2024-02-02T14:59:32.973 INFO:root:teuthology version: 0.0.1.dev256+gd9fdb22 2024-02-02T14:59:32.973 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/dbench}} --verbose --name vshankar-2024-02-02_11:17:23-fs-wip-vshankar-testing1-reef-2024-02-02-1152-testing-default-smithi --owner scheduled_vshankar@teuthology --archive /home/teuthworker/archive/vshankar-2024-02-02_11:17:23-fs-wip-vshankar-testing1-reef-2024-02-02-1152-testing-default-smithi/7543409 -- /home/teuthworker/archive/vshankar-2024-02-02_11:17:23-fs-wip-vshankar-testing1-reef-2024-02-02-1152-testing-default-smithi/7543409/orig.config.yaml 2024-02-02T14:59:33.032 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-02T14:59:33.130 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/vshankar-2024-02-02_11:17:23-fs-wip-vshankar-testing1-reef-2024-02-02-1152-testing-default-smithi/7543409 branch: wip-vshankar-testing1-reef-2024-02-02-1152 description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/dbench}} email: vshankar@redhat.com first_in_suite: false job_id: '7543409' kernel: client: branch: testing flavor: default kdb: true sha1: cd42140e628558a450ba1d39a9d0835a2fa62cda ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: vshankar-2024-02-02_11:17:23-fs-wip-vshankar-testing1-reef-2024-02-02-1152-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-vshankar-testing1-reef-2024-02-02-1152 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - slow metadata IO - SLOW_OPS - slow request sha1: 0fb327883c124fd83155c94db0804f823b805a12 subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: 0fb327883c124fd83155c94db0804f823b805a12 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=legacy - wsync syntax: v2 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-vshankar-testing1-reef-2024-02-02-1152 sha1: 0fb327883c124fd83155c94db0804f823b805a12 owner: scheduled_vshankar@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 1628 sha1: 0fb327883c124fd83155c94db0804f823b805a12 sleep_before_teardown: 0 subset: 50/1024 suite: fs suite_branch: wip-vshankar-testing1-reef-2024-02-02-1152 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_0fb327883c124fd83155c94db0804f823b805a12/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 0fb327883c124fd83155c94db0804f823b805a12 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi001.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBH1rn2VC4T8VnaJuVwz+S1DdI4E7WVLu7YZZWz0pzYDe25xLhAUK2WtqwrPTA3U3fIB0taj0HtHpwttu1J9yK+U= smithi071.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBA4b+FN1PwkTwL2SUBH4cuQNZlKGLXLMASayfQMdZpKqmE9KMIwLXafmB8FDCeE6DOZEP1cBujIXfptse6F6jj0= smithi142.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJ+9kvY0NLYoBysi0hGw7rDPk0gJW8sqe6LiuU2miCRPy1oChyg3cTXZufOoR0OzCMCGWpCi60qpdFaEulx7JFw= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools:rhel8 --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: d9fdb2209e15b39d9f061fd85399f352ce0f0894 timestamp: 2024-02-02_11:17:23 tube: smithi use_shaman: true user: vshankar verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-02-02T14:59:33.130 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_0fb327883c124fd83155c94db0804f823b805a12/qa; will attempt to use it 2024-02-02T14:59:33.131 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_0fb327883c124fd83155c94db0804f823b805a12/qa/tasks 2024-02-02T14:59:33.131 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-02-02T14:59:33.133 INFO:teuthology.task.internal:Checking packages... 2024-02-02T14:59:33.158 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '0fb327883c124fd83155c94db0804f823b805a12' 2024-02-02T14:59:33.159 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-02-02T14:59:33.159 INFO:teuthology.packaging:ref: None 2024-02-02T14:59:33.159 INFO:teuthology.packaging:tag: None 2024-02-02T14:59:33.159 INFO:teuthology.packaging:branch: wip-vshankar-testing1-reef-2024-02-02-1152 2024-02-02T14:59:33.159 INFO:teuthology.packaging:sha1: 0fb327883c124fd83155c94db0804f823b805a12 2024-02-02T14:59:33.159 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-vshankar-testing1-reef-2024-02-02-1152 2024-02-02T14:59:33.406 INFO:teuthology.task.internal:Found packages for ceph version 18.2.1-325.g0fb32788 2024-02-02T14:59:33.407 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-02-02T14:59:33.413 INFO:teuthology.task.internal:no buildpackages task found 2024-02-02T14:59:33.413 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-02-02T14:59:33.452 INFO:teuthology.task.internal:Saving configuration 2024-02-02T14:59:33.473 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-02-02T14:59:33.479 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-02T14:59:33.506 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi001.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-02-02_11:17:23-fs-wip-vshankar-testing1-reef-2024-02-02-1152-testing-default-smithi/7543409', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-02 14:54:37.324227', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBH1rn2VC4T8VnaJuVwz+S1DdI4E7WVLu7YZZWz0pzYDe25xLhAUK2WtqwrPTA3U3fIB0taj0HtHpwttu1J9yK+U='} 2024-02-02T14:59:33.530 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi071.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-02-02_11:17:23-fs-wip-vshankar-testing1-reef-2024-02-02-1152-testing-default-smithi/7543409', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-02 14:54:37.326251', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBA4b+FN1PwkTwL2SUBH4cuQNZlKGLXLMASayfQMdZpKqmE9KMIwLXafmB8FDCeE6DOZEP1cBujIXfptse6F6jj0='} 2024-02-02T14:59:33.554 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi142.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-02-02_11:17:23-fs-wip-vshankar-testing1-reef-2024-02-02-1152-testing-default-smithi/7543409', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-02 14:54:37.327759', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJ+9kvY0NLYoBysi0hGw7rDPk0gJW8sqe6LiuU2miCRPy1oChyg3cTXZufOoR0OzCMCGWpCi60qpdFaEulx7JFw='} 2024-02-02T14:59:33.554 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-02-02T14:59:33.560 INFO:teuthology.task.internal:roles: ubuntu@smithi001.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-02-02T14:59:33.561 INFO:teuthology.task.internal:roles: ubuntu@smithi071.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-02-02T14:59:33.561 INFO:teuthology.task.internal:roles: ubuntu@smithi142.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-02-02T14:59:33.561 INFO:teuthology.run_tasks:Running task console_log... 2024-02-02T14:59:33.670 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f4b724d30d0>, signals=[15]) 2024-02-02T14:59:33.670 INFO:teuthology.run_tasks:Running task internal.connect... 2024-02-02T14:59:33.677 INFO:teuthology.task.internal:Opening connections... 2024-02-02T14:59:33.677 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi001.front.sepia.ceph.com 2024-02-02T14:59:33.679 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi001.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-02T14:59:33.753 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi071.front.sepia.ceph.com 2024-02-02T14:59:33.753 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-02T14:59:33.826 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi142.front.sepia.ceph.com 2024-02-02T14:59:33.827 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-02T14:59:33.899 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-02-02T14:59:33.905 DEBUG:teuthology.orchestra.run.smithi001:> uname -m 2024-02-02T14:59:33.927 INFO:teuthology.orchestra.run.smithi001.stdout:x86_64 2024-02-02T14:59:33.927 DEBUG:teuthology.orchestra.run.smithi001:> cat /etc/os-release 2024-02-02T14:59:33.988 INFO:teuthology.orchestra.run.smithi001.stdout:NAME="Red Hat Enterprise Linux" 2024-02-02T14:59:33.988 INFO:teuthology.orchestra.run.smithi001.stdout:VERSION="8.6 (Ootpa)" 2024-02-02T14:59:33.988 INFO:teuthology.orchestra.run.smithi001.stdout:ID="rhel" 2024-02-02T14:59:33.988 INFO:teuthology.orchestra.run.smithi001.stdout:ID_LIKE="fedora" 2024-02-02T14:59:33.988 INFO:teuthology.orchestra.run.smithi001.stdout:VERSION_ID="8.6" 2024-02-02T14:59:33.988 INFO:teuthology.orchestra.run.smithi001.stdout:PLATFORM_ID="platform:el8" 2024-02-02T14:59:33.988 INFO:teuthology.orchestra.run.smithi001.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-02-02T14:59:33.988 INFO:teuthology.orchestra.run.smithi001.stdout:ANSI_COLOR="0;31" 2024-02-02T14:59:33.988 INFO:teuthology.orchestra.run.smithi001.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-02-02T14:59:33.988 INFO:teuthology.orchestra.run.smithi001.stdout:HOME_URL="https://www.redhat.com/" 2024-02-02T14:59:33.988 INFO:teuthology.orchestra.run.smithi001.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-02-02T14:59:33.988 INFO:teuthology.orchestra.run.smithi001.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-02T14:59:33.989 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T14:59:33.989 INFO:teuthology.orchestra.run.smithi001.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-02-02T14:59:33.989 INFO:teuthology.orchestra.run.smithi001.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-02-02T14:59:33.989 INFO:teuthology.orchestra.run.smithi001.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-02-02T14:59:33.989 INFO:teuthology.orchestra.run.smithi001.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-02-02T14:59:33.989 INFO:teuthology.lock.ops:Updating smithi001.front.sepia.ceph.com on lock server 2024-02-02T14:59:34.011 DEBUG:teuthology.orchestra.run.smithi071:> uname -m 2024-02-02T14:59:34.032 INFO:teuthology.orchestra.run.smithi071.stdout:x86_64 2024-02-02T14:59:34.032 DEBUG:teuthology.orchestra.run.smithi071:> cat /etc/os-release 2024-02-02T14:59:34.092 INFO:teuthology.orchestra.run.smithi071.stdout:NAME="Red Hat Enterprise Linux" 2024-02-02T14:59:34.093 INFO:teuthology.orchestra.run.smithi071.stdout:VERSION="8.6 (Ootpa)" 2024-02-02T14:59:34.093 INFO:teuthology.orchestra.run.smithi071.stdout:ID="rhel" 2024-02-02T14:59:34.093 INFO:teuthology.orchestra.run.smithi071.stdout:ID_LIKE="fedora" 2024-02-02T14:59:34.093 INFO:teuthology.orchestra.run.smithi071.stdout:VERSION_ID="8.6" 2024-02-02T14:59:34.093 INFO:teuthology.orchestra.run.smithi071.stdout:PLATFORM_ID="platform:el8" 2024-02-02T14:59:34.093 INFO:teuthology.orchestra.run.smithi071.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-02-02T14:59:34.093 INFO:teuthology.orchestra.run.smithi071.stdout:ANSI_COLOR="0;31" 2024-02-02T14:59:34.093 INFO:teuthology.orchestra.run.smithi071.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-02-02T14:59:34.093 INFO:teuthology.orchestra.run.smithi071.stdout:HOME_URL="https://www.redhat.com/" 2024-02-02T14:59:34.093 INFO:teuthology.orchestra.run.smithi071.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-02-02T14:59:34.093 INFO:teuthology.orchestra.run.smithi071.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-02T14:59:34.093 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-02T14:59:34.093 INFO:teuthology.orchestra.run.smithi071.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-02-02T14:59:34.093 INFO:teuthology.orchestra.run.smithi071.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-02-02T14:59:34.094 INFO:teuthology.orchestra.run.smithi071.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-02-02T14:59:34.094 INFO:teuthology.orchestra.run.smithi071.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-02-02T14:59:34.094 INFO:teuthology.lock.ops:Updating smithi071.front.sepia.ceph.com on lock server 2024-02-02T14:59:34.120 DEBUG:teuthology.orchestra.run.smithi142:> uname -m 2024-02-02T14:59:34.140 INFO:teuthology.orchestra.run.smithi142.stdout:x86_64 2024-02-02T14:59:34.140 DEBUG:teuthology.orchestra.run.smithi142:> cat /etc/os-release 2024-02-02T14:59:34.199 INFO:teuthology.orchestra.run.smithi142.stdout:NAME="Red Hat Enterprise Linux" 2024-02-02T14:59:34.199 INFO:teuthology.orchestra.run.smithi142.stdout:VERSION="8.6 (Ootpa)" 2024-02-02T14:59:34.199 INFO:teuthology.orchestra.run.smithi142.stdout:ID="rhel" 2024-02-02T14:59:34.199 INFO:teuthology.orchestra.run.smithi142.stdout:ID_LIKE="fedora" 2024-02-02T14:59:34.199 INFO:teuthology.orchestra.run.smithi142.stdout:VERSION_ID="8.6" 2024-02-02T14:59:34.199 INFO:teuthology.orchestra.run.smithi142.stdout:PLATFORM_ID="platform:el8" 2024-02-02T14:59:34.199 INFO:teuthology.orchestra.run.smithi142.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-02-02T14:59:34.199 INFO:teuthology.orchestra.run.smithi142.stdout:ANSI_COLOR="0;31" 2024-02-02T14:59:34.200 INFO:teuthology.orchestra.run.smithi142.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-02-02T14:59:34.200 INFO:teuthology.orchestra.run.smithi142.stdout:HOME_URL="https://www.redhat.com/" 2024-02-02T14:59:34.200 INFO:teuthology.orchestra.run.smithi142.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-02-02T14:59:34.200 INFO:teuthology.orchestra.run.smithi142.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-02T14:59:34.200 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-02-02T14:59:34.200 INFO:teuthology.orchestra.run.smithi142.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-02-02T14:59:34.200 INFO:teuthology.orchestra.run.smithi142.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-02-02T14:59:34.200 INFO:teuthology.orchestra.run.smithi142.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-02-02T14:59:34.200 INFO:teuthology.orchestra.run.smithi142.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-02-02T14:59:34.200 INFO:teuthology.lock.ops:Updating smithi142.front.sepia.ceph.com on lock server 2024-02-02T14:59:34.219 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-02-02T14:59:34.230 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-02-02T14:59:34.252 INFO:teuthology.task.internal:Checking for old test directory... 2024-02-02T14:59:34.252 DEBUG:teuthology.orchestra.run.smithi001:> test '!' -e /home/ubuntu/cephtest 2024-02-02T14:59:34.255 DEBUG:teuthology.orchestra.run.smithi071:> test '!' -e /home/ubuntu/cephtest 2024-02-02T14:59:34.257 DEBUG:teuthology.orchestra.run.smithi142:> test '!' -e /home/ubuntu/cephtest 2024-02-02T14:59:34.275 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-02-02T14:59:34.281 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-02-02T14:59:34.282 DEBUG:teuthology.orchestra.run.smithi001:> test -z $(ls -A /var/lib/ceph) 2024-02-02T14:59:34.315 DEBUG:teuthology.orchestra.run.smithi071:> test -z $(ls -A /var/lib/ceph) 2024-02-02T14:59:34.317 DEBUG:teuthology.orchestra.run.smithi142:> test -z $(ls -A /var/lib/ceph) 2024-02-02T14:59:34.346 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-02-02T14:59:34.424 INFO:teuthology.run_tasks:Running task kernel... 2024-02-02T14:59:34.439 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': 'cd42140e628558a450ba1d39a9d0835a2fa62cda'} 2024-02-02T14:59:34.440 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-02-02T14:59:34.440 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-02-02T14:59:34.440 DEBUG:teuthology.orchestra.run.smithi001:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-02T14:59:34.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-02T14:59:34.461 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&ref=testing 2024-02-02T14:59:34.799 DEBUG:teuthology.task.kernel:sha1 for client.0 is cd42140e628558a450ba1d39a9d0835a2fa62cda 2024-02-02T14:59:34.799 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "cd42140e628558a450ba1d39a9d0835a2fa62cda"... 2024-02-02T14:59:34.799 DEBUG:teuthology.orchestra.run.smithi001:> uname -r 2024-02-02T14:59:34.820 INFO:teuthology.orchestra.run.smithi001.stdout:4.18.0-513.9.1.el8_9.x86_64 2024-02-02T14:59:34.820 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-513.9.1.el8_9.x86_64 vs cd42140e628558a450ba1d39a9d0835a2fa62cda 2024-02-02T14:59:34.821 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-02-02T14:59:34.821 DEBUG:teuthology.orchestra.run.smithi001:> sudo yum upgrade -y linux-firmware 2024-02-02T14:59:36.358 INFO:teuthology.orchestra.run.smithi001.stdout:Updating Subscription Management repositories. 2024-02-02T14:59:36.358 INFO:teuthology.orchestra.run.smithi001.stdout:Unable to read consumer identity 2024-02-02T14:59:36.494 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T14:59:36.494 INFO:teuthology.orchestra.run.smithi001.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-02-02T14:59:36.494 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T14:59:38.296 INFO:teuthology.orchestra.run.smithi001.stdout:Copr repo for python3-asyncssh owned by ceph 14 kB/s | 3.5 kB 00:00 2024-02-02T14:59:40.996 INFO:teuthology.orchestra.run.smithi001.stdout:Extra Packages for Enterprise Linux 6.0 MB/s | 16 MB 00:02 2024-02-02T14:59:45.046 INFO:teuthology.orchestra.run.smithi001.stdout:lab-extras 266 kB/s | 24 kB 00:00 2024-02-02T14:59:45.898 INFO:teuthology.orchestra.run.smithi001.stdout:Dependencies resolved. 2024-02-02T14:59:45.900 INFO:teuthology.orchestra.run.smithi001.stdout:Nothing to do. 2024-02-02T14:59:45.900 INFO:teuthology.orchestra.run.smithi001.stdout:Complete! 2024-02-02T14:59:46.053 INFO:teuthology.task.kernel:Downloading kernel cd42140e628558a450ba1d39a9d0835a2fa62cda on client.0... 2024-02-02T14:59:46.054 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=cd42140e628558a450ba1d39a9d0835a2fa62cda 2024-02-02T14:59:46.206 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/cd42140e628558a450ba1d39a9d0835a2fa62cda/centos/8/flavors/default/x86_64/ 2024-02-02T14:59:46.206 DEBUG:teuthology.orchestra.run.smithi001:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.7.0_gcd42140e6285-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/cd42140e628558a450ba1d39a9d0835a2fa62cda/centos/8/flavors/default/x86_64/ --input-file=- 2024-02-02T14:59:46.209 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi001.front.sepia.ceph.com to complete... 2024-02-02T15:00:08.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02 15:00:08 URL:https://4.chacra.ceph.com/r/kernel/testing/cd42140e628558a450ba1d39a9d0835a2fa62cda/centos/8/flavors/default/x86_64/kernel-6.7.0_gcd42140e6285-1.x86_64.rpm [394673092/394673092] -> "/tmp/kernel.x86_64.rpm" [1] 2024-02-02T15:00:08.304 INFO:teuthology.orchestra.run.smithi001.stderr:FINISHED --2024-02-02 15:00:08-- 2024-02-02T15:00:08.305 INFO:teuthology.orchestra.run.smithi001.stderr:Total wall clock time: 22s 2024-02-02T15:00:08.305 INFO:teuthology.orchestra.run.smithi001.stderr:Downloaded: 1 files, 376M in 22s (17.2 MB/s) 2024-02-02T15:00:08.307 INFO:teuthology.task.kernel:Installing kernel cd42140e628558a450ba1d39a9d0835a2fa62cda on client.0... 2024-02-02T15:00:08.308 DEBUG:teuthology.orchestra.run.smithi001:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-02-02T15:00:10.285 INFO:teuthology.orchestra.run.smithi001.stdout:Verifying... ######################################## 2024-02-02T15:00:10.402 INFO:teuthology.orchestra.run.smithi001.stdout:Preparing... ######################################## 2024-02-02T15:00:10.556 INFO:teuthology.orchestra.run.smithi001.stdout:Updating / installing... 2024-02-02T15:00:54.568 INFO:teuthology.orchestra.run.smithi001.stdout:kernel-6.7.0_gcd42140e6285-1 ######################################## 2024-02-02T15:00:56.181 INFO:teuthology.orchestra.run.smithi001.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-02-02T15:01:19.424 INFO:teuthology.orchestra.run.smithi001.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-02-02T15:03:37.502 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi001.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-02-02T15:03:37.502 DEBUG:teuthology.orchestra.run.smithi001:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-02-02T15:03:37.561 INFO:teuthology.orchestra.run.smithi001.stdout:/boot/System.map-6.7.0-gcd42140e6285 2024-02-02T15:03:37.561 INFO:teuthology.orchestra.run.smithi001.stdout:/boot/config-6.7.0-gcd42140e6285 2024-02-02T15:03:37.562 INFO:teuthology.orchestra.run.smithi001.stdout:/boot/vmlinuz-6.7.0-gcd42140e6285 2024-02-02T15:03:37.562 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285 2024-02-02T15:03:37.562 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel 2024-02-02T15:03:37.562 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch 2024-02-02T15:03:37.562 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86 2024-02-02T15:03:37.562 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto 2024-02-02T15:03:37.562 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-02-02T15:03:37.562 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-02-02T15:03:37.562 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-02-02T15:03:37.562 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-02-02T15:03:37.562 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-02-02T15:03:37.562 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-02-02T15:03:37.563 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-02-02T15:03:37.563 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/crc32c-intel.ko 2024-02-02T15:03:37.563 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-02-02T15:03:37.563 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-02-02T15:03:37.563 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-02-02T15:03:37.563 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-02-02T15:03:37.563 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/serpent-avx2.ko 2024-02-02T15:03:37.563 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-02-02T15:03:37.563 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-02-02T15:03:37.563 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-02-02T15:03:37.563 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-02-02T15:03:37.563 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-02-02T15:03:37.563 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-02-02T15:03:37.563 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-02-02T15:03:37.564 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/kernel 2024-02-02T15:03:37.564 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/kernel/cpu 2024-02-02T15:03:37.564 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/kernel/cpu/mce 2024-02-02T15:03:37.564 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-02-02T15:03:37.564 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/kvm 2024-02-02T15:03:37.564 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/kvm/kvm-amd.ko 2024-02-02T15:03:37.564 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/kvm/kvm-intel.ko 2024-02-02T15:03:37.564 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/arch/x86/kvm/kvm.ko 2024-02-02T15:03:37.564 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto 2024-02-02T15:03:37.564 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/ansi_cprng.ko 2024-02-02T15:03:37.564 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/anubis.ko 2024-02-02T15:03:37.564 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/arc4.ko 2024-02-02T15:03:37.564 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/async_tx 2024-02-02T15:03:37.565 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/async_tx/async_memcpy.ko 2024-02-02T15:03:37.565 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/async_tx/async_pq.ko 2024-02-02T15:03:37.565 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/async_tx/async_raid6_recov.ko 2024-02-02T15:03:37.565 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/async_tx/async_tx.ko 2024-02-02T15:03:37.565 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/async_tx/async_xor.ko 2024-02-02T15:03:37.565 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/async_tx/raid6test.ko 2024-02-02T15:03:37.565 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/authenc.ko 2024-02-02T15:03:37.565 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/authencesn.ko 2024-02-02T15:03:37.565 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/blake2b_generic.ko 2024-02-02T15:03:37.565 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/blowfish_common.ko 2024-02-02T15:03:37.565 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/blowfish_generic.ko 2024-02-02T15:03:37.566 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/camellia_generic.ko 2024-02-02T15:03:37.566 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/cast5_generic.ko 2024-02-02T15:03:37.566 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/cast6_generic.ko 2024-02-02T15:03:37.566 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/cast_common.ko 2024-02-02T15:03:37.566 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/ccm.ko 2024-02-02T15:03:37.566 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/cmac.ko 2024-02-02T15:03:37.566 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/crc32_generic.ko 2024-02-02T15:03:37.566 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/crypto_engine.ko 2024-02-02T15:03:37.566 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/crypto_user.ko 2024-02-02T15:03:37.566 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/cts.ko 2024-02-02T15:03:37.567 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/des_generic.ko 2024-02-02T15:03:37.567 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/dh_generic.ko 2024-02-02T15:03:37.567 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/echainiv.ko 2024-02-02T15:03:37.567 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/essiv.ko 2024-02-02T15:03:37.567 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/fcrypt.ko 2024-02-02T15:03:37.567 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/khazad.ko 2024-02-02T15:03:37.567 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/lz4.ko 2024-02-02T15:03:37.567 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/lz4hc.ko 2024-02-02T15:03:37.567 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/md4.ko 2024-02-02T15:03:37.567 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/michael_mic.ko 2024-02-02T15:03:37.567 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/pcbc.ko 2024-02-02T15:03:37.567 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/pcrypt.ko 2024-02-02T15:03:37.567 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/rmd160.ko 2024-02-02T15:03:37.567 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/seed.ko 2024-02-02T15:03:37.568 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/serpent_generic.ko 2024-02-02T15:03:37.568 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/tcrypt.ko 2024-02-02T15:03:37.568 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/tea.ko 2024-02-02T15:03:37.568 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/twofish_common.ko 2024-02-02T15:03:37.568 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/twofish_generic.ko 2024-02-02T15:03:37.568 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/vmac.ko 2024-02-02T15:03:37.568 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/wp512.ko 2024-02-02T15:03:37.568 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/xcbc.ko 2024-02-02T15:03:37.568 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/xor.ko 2024-02-02T15:03:37.568 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/crypto/xxhash_generic.ko 2024-02-02T15:03:37.568 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers 2024-02-02T15:03:37.568 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/acpi 2024-02-02T15:03:37.568 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/acpi/acpi_ipmi.ko 2024-02-02T15:03:37.568 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/acpi/acpi_pad.ko 2024-02-02T15:03:37.568 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/acpi/custom_method.ko 2024-02-02T15:03:37.569 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/acpi/ec_sys.ko 2024-02-02T15:03:37.569 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/acpi/platform_profile.ko 2024-02-02T15:03:37.569 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/acpi/sbs.ko 2024-02-02T15:03:37.569 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/acpi/sbshc.ko 2024-02-02T15:03:37.569 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/acpi/video.ko 2024-02-02T15:03:37.569 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata 2024-02-02T15:03:37.569 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/acard-ahci.ko 2024-02-02T15:03:37.569 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/ahci_platform.ko 2024-02-02T15:03:37.569 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/ata_generic.ko 2024-02-02T15:03:37.569 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/libahci_platform.ko 2024-02-02T15:03:37.569 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_acpi.ko 2024-02-02T15:03:37.569 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_ali.ko 2024-02-02T15:03:37.569 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_amd.ko 2024-02-02T15:03:37.569 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_artop.ko 2024-02-02T15:03:37.569 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_atiixp.ko 2024-02-02T15:03:37.570 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_atp867x.ko 2024-02-02T15:03:37.570 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_cmd640.ko 2024-02-02T15:03:37.570 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_cmd64x.ko 2024-02-02T15:03:37.570 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_cypress.ko 2024-02-02T15:03:37.570 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_efar.ko 2024-02-02T15:03:37.570 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_hpt366.ko 2024-02-02T15:03:37.570 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_hpt37x.ko 2024-02-02T15:03:37.570 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_hpt3x2n.ko 2024-02-02T15:03:37.570 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_hpt3x3.ko 2024-02-02T15:03:37.570 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_it8213.ko 2024-02-02T15:03:37.570 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_it821x.ko 2024-02-02T15:03:37.570 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_jmicron.ko 2024-02-02T15:03:37.570 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_marvell.ko 2024-02-02T15:03:37.571 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_mpiix.ko 2024-02-02T15:03:37.571 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_netcell.ko 2024-02-02T15:03:37.571 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_ninja32.ko 2024-02-02T15:03:37.571 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_ns87410.ko 2024-02-02T15:03:37.571 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_ns87415.ko 2024-02-02T15:03:37.571 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_oldpiix.ko 2024-02-02T15:03:37.571 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_opti.ko 2024-02-02T15:03:37.571 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_optidma.ko 2024-02-02T15:03:37.572 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_pcmcia.ko 2024-02-02T15:03:37.572 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_pdc2027x.ko 2024-02-02T15:03:37.572 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-02-02T15:03:37.572 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_piccolo.ko 2024-02-02T15:03:37.572 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_rdc.ko 2024-02-02T15:03:37.572 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_sch.ko 2024-02-02T15:03:37.572 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_serverworks.ko 2024-02-02T15:03:37.572 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_sil680.ko 2024-02-02T15:03:37.572 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_sis.ko 2024-02-02T15:03:37.572 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_sl82c105.ko 2024-02-02T15:03:37.572 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_triflex.ko 2024-02-02T15:03:37.572 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pata_via.ko 2024-02-02T15:03:37.572 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/pdc_adma.ko 2024-02-02T15:03:37.572 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/sata_inic162x.ko 2024-02-02T15:03:37.572 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/sata_mv.ko 2024-02-02T15:03:37.573 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/sata_nv.ko 2024-02-02T15:03:37.573 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/sata_promise.ko 2024-02-02T15:03:37.573 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/sata_qstor.ko 2024-02-02T15:03:37.573 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/sata_sil.ko 2024-02-02T15:03:37.573 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/sata_sil24.ko 2024-02-02T15:03:37.573 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/sata_sis.ko 2024-02-02T15:03:37.573 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/sata_svw.ko 2024-02-02T15:03:37.573 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/sata_sx4.ko 2024-02-02T15:03:37.573 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/sata_uli.ko 2024-02-02T15:03:37.573 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/sata_via.ko 2024-02-02T15:03:37.574 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ata/sata_vsc.ko 2024-02-02T15:03:37.574 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/auxdisplay 2024-02-02T15:03:37.574 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/auxdisplay/ks0108.ko 2024-02-02T15:03:37.574 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/base 2024-02-02T15:03:37.574 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/base/regmap 2024-02-02T15:03:37.574 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/base/regmap/regmap-i2c.ko 2024-02-02T15:03:37.575 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/base/regmap/regmap-mmio.ko 2024-02-02T15:03:37.575 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/bcma 2024-02-02T15:03:37.575 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/bcma/bcma.ko 2024-02-02T15:03:37.575 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block 2024-02-02T15:03:37.575 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/aoe 2024-02-02T15:03:37.575 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/aoe/aoe.ko 2024-02-02T15:03:37.575 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/brd.ko 2024-02-02T15:03:37.576 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/drbd 2024-02-02T15:03:37.576 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/drbd/drbd.ko 2024-02-02T15:03:37.576 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/floppy.ko 2024-02-02T15:03:37.576 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/loop.ko 2024-02-02T15:03:37.576 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/mtip32xx 2024-02-02T15:03:37.576 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-02-02T15:03:37.576 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/nbd.ko 2024-02-02T15:03:37.576 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/null_blk 2024-02-02T15:03:37.576 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/null_blk/null_blk.ko 2024-02-02T15:03:37.576 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/pktcdvd.ko 2024-02-02T15:03:37.576 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/rbd.ko 2024-02-02T15:03:37.576 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/virtio_blk.ko 2024-02-02T15:03:37.576 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/xen-blkback 2024-02-02T15:03:37.576 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-02-02T15:03:37.577 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/xen-blkfront.ko 2024-02-02T15:03:37.577 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/zram 2024-02-02T15:03:37.577 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/block/zram/zram.ko 2024-02-02T15:03:37.577 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char 2024-02-02T15:03:37.577 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/hangcheck-timer.ko 2024-02-02T15:03:37.577 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/hw_random 2024-02-02T15:03:37.578 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/hw_random/amd-rng.ko 2024-02-02T15:03:37.578 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/hw_random/intel-rng.ko 2024-02-02T15:03:37.578 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-02-02T15:03:37.578 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/hw_random/via-rng.ko 2024-02-02T15:03:37.578 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/hw_random/virtio-rng.ko 2024-02-02T15:03:37.578 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/ipmi 2024-02-02T15:03:37.578 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-02-02T15:03:37.578 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-02-02T15:03:37.578 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-02-02T15:03:37.578 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/ipmi/ipmi_si.ko 2024-02-02T15:03:37.578 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-02-02T15:03:37.578 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-02-02T15:03:37.578 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/lp.ko 2024-02-02T15:03:37.578 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/mwave 2024-02-02T15:03:37.578 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/mwave/mwave.ko 2024-02-02T15:03:37.579 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/ppdev.ko 2024-02-02T15:03:37.579 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/tlclk.ko 2024-02-02T15:03:37.579 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/tpm 2024-02-02T15:03:37.579 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/tpm/tpm.ko 2024-02-02T15:03:37.579 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/tpm/tpm_atmel.ko 2024-02-02T15:03:37.579 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/tpm/tpm_infineon.ko 2024-02-02T15:03:37.579 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/tpm/tpm_nsc.ko 2024-02-02T15:03:37.579 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/tpm/tpm_tis.ko 2024-02-02T15:03:37.579 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-02-02T15:03:37.579 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/uv_mmtimer.ko 2024-02-02T15:03:37.579 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/char/virtio_console.ko 2024-02-02T15:03:37.579 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/cpufreq 2024-02-02T15:03:37.579 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-02-02T15:03:37.579 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-02-02T15:03:37.579 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/cpufreq/p4-clockmod.ko 2024-02-02T15:03:37.579 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-02-02T15:03:37.579 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/cpufreq/powernow-k8.ko 2024-02-02T15:03:37.580 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/cpufreq/speedstep-lib.ko 2024-02-02T15:03:37.580 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/crypto 2024-02-02T15:03:37.580 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/crypto/ccp 2024-02-02T15:03:37.580 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-02-02T15:03:37.580 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/crypto/ccp/ccp.ko 2024-02-02T15:03:37.580 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/crypto/intel 2024-02-02T15:03:37.580 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/crypto/intel/qat 2024-02-02T15:03:37.580 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/crypto/intel/qat/qat_common 2024-02-02T15:03:37.580 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-02-02T15:03:37.580 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-02-02T15:03:37.580 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-02-02T15:03:37.580 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/crypto/padlock-aes.ko 2024-02-02T15:03:37.580 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/crypto/padlock-sha.ko 2024-02-02T15:03:37.581 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/crypto/virtio 2024-02-02T15:03:37.581 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-02-02T15:03:37.581 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/dca 2024-02-02T15:03:37.581 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/dca/dca.ko 2024-02-02T15:03:37.581 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/devfreq 2024-02-02T15:03:37.581 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-02-02T15:03:37.581 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/dma 2024-02-02T15:03:37.581 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/dma/dw 2024-02-02T15:03:37.581 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/dma/dw/dw_dmac.ko 2024-02-02T15:03:37.581 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/dma/ioat 2024-02-02T15:03:37.581 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/dma/ioat/ioatdma.ko 2024-02-02T15:03:37.581 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/edac 2024-02-02T15:03:37.581 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/edac/amd64_edac.ko 2024-02-02T15:03:37.581 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/edac/e752x_edac.ko 2024-02-02T15:03:37.582 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/edac/edac_mce_amd.ko 2024-02-02T15:03:37.582 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/edac/i3000_edac.ko 2024-02-02T15:03:37.582 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/edac/i3200_edac.ko 2024-02-02T15:03:37.582 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/edac/i5100_edac.ko 2024-02-02T15:03:37.582 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/edac/i5400_edac.ko 2024-02-02T15:03:37.582 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/edac/i7300_edac.ko 2024-02-02T15:03:37.582 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/edac/i7core_edac.ko 2024-02-02T15:03:37.582 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/edac/i82975x_edac.ko 2024-02-02T15:03:37.582 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/edac/ie31200_edac.ko 2024-02-02T15:03:37.582 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/edac/sb_edac.ko 2024-02-02T15:03:37.582 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/edac/x38_edac.ko 2024-02-02T15:03:37.582 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/firmware 2024-02-02T15:03:37.582 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/firmware/edd.ko 2024-02-02T15:03:37.582 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/firmware/iscsi_ibft.ko 2024-02-02T15:03:37.582 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/gpio 2024-02-02T15:03:37.583 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/gpio/gpio-ich.ko 2024-02-02T15:03:37.583 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/gpio/gpio-viperboard.ko 2024-02-02T15:03:37.583 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid 2024-02-02T15:03:37.583 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-appleir.ko 2024-02-02T15:03:37.583 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-aureal.ko 2024-02-02T15:03:37.583 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-axff.ko 2024-02-02T15:03:37.583 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-dr.ko 2024-02-02T15:03:37.583 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-elecom.ko 2024-02-02T15:03:37.583 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-elo.ko 2024-02-02T15:03:37.583 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-emsff.ko 2024-02-02T15:03:37.583 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-gaff.ko 2024-02-02T15:03:37.583 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-gt683r.ko 2024-02-02T15:03:37.583 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-gyration.ko 2024-02-02T15:03:37.583 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-holtek-kbd.ko 2024-02-02T15:03:37.583 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-holtek-mouse.ko 2024-02-02T15:03:37.583 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-holtekff.ko 2024-02-02T15:03:37.584 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-hyperv.ko 2024-02-02T15:03:37.584 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-icade.ko 2024-02-02T15:03:37.584 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-keytouch.ko 2024-02-02T15:03:37.584 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-kye.ko 2024-02-02T15:03:37.584 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-lcpower.ko 2024-02-02T15:03:37.584 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-led.ko 2024-02-02T15:03:37.584 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-lenovo.ko 2024-02-02T15:03:37.584 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-logitech-dj.ko 2024-02-02T15:03:37.584 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-02-02T15:03:37.584 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-multitouch.ko 2024-02-02T15:03:37.584 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-ortek.ko 2024-02-02T15:03:37.584 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-penmount.ko 2024-02-02T15:03:37.584 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-petalynx.ko 2024-02-02T15:03:37.584 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-picolcd.ko 2024-02-02T15:03:37.584 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-pl.ko 2024-02-02T15:03:37.585 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-plantronics.ko 2024-02-02T15:03:37.585 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-primax.ko 2024-02-02T15:03:37.585 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-rmi.ko 2024-02-02T15:03:37.585 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-roccat-arvo.ko 2024-02-02T15:03:37.585 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-roccat-common.ko 2024-02-02T15:03:37.585 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-roccat-isku.ko 2024-02-02T15:03:37.585 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-roccat-kone.ko 2024-02-02T15:03:37.585 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-02-02T15:03:37.585 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-roccat-konepure.ko 2024-02-02T15:03:37.585 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-02-02T15:03:37.585 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-roccat-lua.ko 2024-02-02T15:03:37.585 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-roccat-pyra.ko 2024-02-02T15:03:37.585 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-roccat-ryos.ko 2024-02-02T15:03:37.585 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-roccat-savu.ko 2024-02-02T15:03:37.585 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-roccat.ko 2024-02-02T15:03:37.585 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-saitek.ko 2024-02-02T15:03:37.586 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-samsung.ko 2024-02-02T15:03:37.586 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-sensor-hub.ko 2024-02-02T15:03:37.586 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-sjoy.ko 2024-02-02T15:03:37.586 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-sony.ko 2024-02-02T15:03:37.586 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-speedlink.ko 2024-02-02T15:03:37.586 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-steelseries.ko 2024-02-02T15:03:37.586 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-sunplus.ko 2024-02-02T15:03:37.586 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-thrustmaster.ko 2024-02-02T15:03:37.586 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-tivo.ko 2024-02-02T15:03:37.586 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-tmff.ko 2024-02-02T15:03:37.586 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-topseed.ko 2024-02-02T15:03:37.586 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-twinhan.ko 2024-02-02T15:03:37.586 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-uclogic.ko 2024-02-02T15:03:37.586 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-waltop.ko 2024-02-02T15:03:37.586 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-wiimote.ko 2024-02-02T15:03:37.587 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-xinmo.ko 2024-02-02T15:03:37.587 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-zpff.ko 2024-02-02T15:03:37.587 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/hid-zydacron.ko 2024-02-02T15:03:37.587 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/uhid.ko 2024-02-02T15:03:37.587 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hid/wacom.ko 2024-02-02T15:03:37.587 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hv 2024-02-02T15:03:37.587 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hv/hv_balloon.ko 2024-02-02T15:03:37.587 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hv/hv_utils.ko 2024-02-02T15:03:37.587 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hv/hv_vmbus.ko 2024-02-02T15:03:37.587 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon 2024-02-02T15:03:37.587 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/abituguru.ko 2024-02-02T15:03:37.587 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/abituguru3.ko 2024-02-02T15:03:37.587 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/acpi_power_meter.ko 2024-02-02T15:03:37.587 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/ad7414.ko 2024-02-02T15:03:37.588 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/ad7418.ko 2024-02-02T15:03:37.588 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/adc128d818.ko 2024-02-02T15:03:37.588 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/adm1025.ko 2024-02-02T15:03:37.588 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/adm1026.ko 2024-02-02T15:03:37.588 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/adm1029.ko 2024-02-02T15:03:37.588 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/adm1031.ko 2024-02-02T15:03:37.588 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/adm9240.ko 2024-02-02T15:03:37.588 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/ads7828.ko 2024-02-02T15:03:37.588 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/adt7410.ko 2024-02-02T15:03:37.588 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/adt7411.ko 2024-02-02T15:03:37.588 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/adt7462.ko 2024-02-02T15:03:37.588 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/adt7470.ko 2024-02-02T15:03:37.588 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/adt7475.ko 2024-02-02T15:03:37.588 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/adt7x10.ko 2024-02-02T15:03:37.589 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/amc6821.ko 2024-02-02T15:03:37.589 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/applesmc.ko 2024-02-02T15:03:37.589 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/asb100.ko 2024-02-02T15:03:37.589 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/asc7621.ko 2024-02-02T15:03:37.589 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/asus_atk0110.ko 2024-02-02T15:03:37.589 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/atxp1.ko 2024-02-02T15:03:37.589 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/coretemp.ko 2024-02-02T15:03:37.589 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-02-02T15:03:37.589 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/dme1737.ko 2024-02-02T15:03:37.589 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/ds1621.ko 2024-02-02T15:03:37.589 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/ds620.ko 2024-02-02T15:03:37.589 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/emc1403.ko 2024-02-02T15:03:37.589 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/emc6w201.ko 2024-02-02T15:03:37.589 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/f71805f.ko 2024-02-02T15:03:37.589 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/f71882fg.ko 2024-02-02T15:03:37.590 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/f75375s.ko 2024-02-02T15:03:37.590 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/fam15h_power.ko 2024-02-02T15:03:37.590 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/fschmd.ko 2024-02-02T15:03:37.590 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/g760a.ko 2024-02-02T15:03:37.590 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/g762.ko 2024-02-02T15:03:37.590 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/gl518sm.ko 2024-02-02T15:03:37.590 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/gl520sm.ko 2024-02-02T15:03:37.590 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/hwmon-vid.ko 2024-02-02T15:03:37.590 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/i5500_temp.ko 2024-02-02T15:03:37.590 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/i5k_amb.ko 2024-02-02T15:03:37.590 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/ibmaem.ko 2024-02-02T15:03:37.590 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/ibmpex.ko 2024-02-02T15:03:37.590 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/ina209.ko 2024-02-02T15:03:37.590 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/ina2xx.ko 2024-02-02T15:03:37.590 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/it87.ko 2024-02-02T15:03:37.591 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/k10temp.ko 2024-02-02T15:03:37.591 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/k8temp.ko 2024-02-02T15:03:37.591 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/lineage-pem.ko 2024-02-02T15:03:37.591 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/lm63.ko 2024-02-02T15:03:37.591 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/lm73.ko 2024-02-02T15:03:37.591 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/lm75.ko 2024-02-02T15:03:37.591 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/lm77.ko 2024-02-02T15:03:37.591 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/lm78.ko 2024-02-02T15:03:37.591 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/lm80.ko 2024-02-02T15:03:37.591 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/lm83.ko 2024-02-02T15:03:37.591 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/lm85.ko 2024-02-02T15:03:37.591 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/lm87.ko 2024-02-02T15:03:37.591 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/lm90.ko 2024-02-02T15:03:37.591 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/lm92.ko 2024-02-02T15:03:37.591 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/lm93.ko 2024-02-02T15:03:37.592 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/lm95234.ko 2024-02-02T15:03:37.592 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/lm95241.ko 2024-02-02T15:03:37.592 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/lm95245.ko 2024-02-02T15:03:37.592 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/ltc2945.ko 2024-02-02T15:03:37.592 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/ltc4151.ko 2024-02-02T15:03:37.592 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/ltc4215.ko 2024-02-02T15:03:37.592 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/ltc4222.ko 2024-02-02T15:03:37.592 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/ltc4245.ko 2024-02-02T15:03:37.592 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/ltc4260.ko 2024-02-02T15:03:37.592 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/ltc4261.ko 2024-02-02T15:03:37.592 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/max16065.ko 2024-02-02T15:03:37.592 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/max1619.ko 2024-02-02T15:03:37.592 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/max1668.ko 2024-02-02T15:03:37.592 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/max197.ko 2024-02-02T15:03:37.592 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/max6639.ko 2024-02-02T15:03:37.593 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/max6650.ko 2024-02-02T15:03:37.593 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/max6697.ko 2024-02-02T15:03:37.593 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/mcp3021.ko 2024-02-02T15:03:37.593 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/nct6683.ko 2024-02-02T15:03:37.593 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/nct6775-core.ko 2024-02-02T15:03:37.593 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/nct6775.ko 2024-02-02T15:03:37.593 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/nct7802.ko 2024-02-02T15:03:37.593 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/pc87360.ko 2024-02-02T15:03:37.593 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/pc87427.ko 2024-02-02T15:03:37.593 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/pcf8591.ko 2024-02-02T15:03:37.593 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/pmbus 2024-02-02T15:03:37.593 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-02-02T15:03:37.593 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-02-02T15:03:37.593 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-02-02T15:03:37.593 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/pmbus/max16064.ko 2024-02-02T15:03:37.593 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/pmbus/max34440.ko 2024-02-02T15:03:37.593 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/pmbus/max8688.ko 2024-02-02T15:03:37.594 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-02-02T15:03:37.594 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-02-02T15:03:37.594 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-02-02T15:03:37.594 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-02-02T15:03:37.594 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-02-02T15:03:37.594 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-02-02T15:03:37.594 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/powr1220.ko 2024-02-02T15:03:37.594 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/sch5627.ko 2024-02-02T15:03:37.594 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/sch5636.ko 2024-02-02T15:03:37.594 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/sch56xx-common.ko 2024-02-02T15:03:37.594 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/sht15.ko 2024-02-02T15:03:37.594 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/sht21.ko 2024-02-02T15:03:37.594 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/shtc1.ko 2024-02-02T15:03:37.594 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/sis5595.ko 2024-02-02T15:03:37.594 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/smsc47b397.ko 2024-02-02T15:03:37.594 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/smsc47m1.ko 2024-02-02T15:03:37.594 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/smsc47m192.ko 2024-02-02T15:03:37.595 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/thmc50.ko 2024-02-02T15:03:37.595 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/tmp102.ko 2024-02-02T15:03:37.595 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/tmp103.ko 2024-02-02T15:03:37.595 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/tmp401.ko 2024-02-02T15:03:37.595 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/tmp421.ko 2024-02-02T15:03:37.595 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/via-cputemp.ko 2024-02-02T15:03:37.595 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/via686a.ko 2024-02-02T15:03:37.595 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/vt1211.ko 2024-02-02T15:03:37.595 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/vt8231.ko 2024-02-02T15:03:37.595 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/w83627ehf.ko 2024-02-02T15:03:37.595 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/w83627hf.ko 2024-02-02T15:03:37.595 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/w83781d.ko 2024-02-02T15:03:37.595 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/w83791d.ko 2024-02-02T15:03:37.595 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/w83792d.ko 2024-02-02T15:03:37.595 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/w83793.ko 2024-02-02T15:03:37.595 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/w83795.ko 2024-02-02T15:03:37.596 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/w83l785ts.ko 2024-02-02T15:03:37.596 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/hwmon/w83l786ng.ko 2024-02-02T15:03:37.596 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c 2024-02-02T15:03:37.596 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/algos 2024-02-02T15:03:37.596 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-02-02T15:03:37.596 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-02-02T15:03:37.596 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses 2024-02-02T15:03:37.596 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-02-02T15:03:37.596 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-02-02T15:03:37.596 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-02-02T15:03:37.596 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-02-02T15:03:37.596 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-02-02T15:03:37.596 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-02-02T15:03:37.596 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-02-02T15:03:37.596 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-02-02T15:03:37.596 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-i801.ko 2024-02-02T15:03:37.597 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-isch.ko 2024-02-02T15:03:37.597 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-02-02T15:03:37.597 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-02-02T15:03:37.597 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-02-02T15:03:37.597 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-parport.ko 2024-02-02T15:03:37.597 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-02-02T15:03:37.597 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-02-02T15:03:37.597 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-02-02T15:03:37.597 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-02-02T15:03:37.597 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-02-02T15:03:37.597 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-02-02T15:03:37.597 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-via.ko 2024-02-02T15:03:37.597 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-02-02T15:03:37.597 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-02-02T15:03:37.597 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/i2c-dev.ko 2024-02-02T15:03:37.597 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/i2c-mux.ko 2024-02-02T15:03:37.597 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-02-02T15:03:37.598 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/i2c-smbus.ko 2024-02-02T15:03:37.598 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/i2c/i2c-stub.ko 2024-02-02T15:03:37.598 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband 2024-02-02T15:03:37.598 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/core 2024-02-02T15:03:37.598 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/core/ib_cm.ko 2024-02-02T15:03:37.598 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/core/ib_core.ko 2024-02-02T15:03:37.598 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/core/ib_umad.ko 2024-02-02T15:03:37.598 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-02-02T15:03:37.598 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/core/iw_cm.ko 2024-02-02T15:03:37.598 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/core/rdma_cm.ko 2024-02-02T15:03:37.598 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-02-02T15:03:37.598 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/hw 2024-02-02T15:03:37.598 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/hw/cxgb4 2024-02-02T15:03:37.598 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-02-02T15:03:37.598 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/hw/mlx4 2024-02-02T15:03:37.598 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-02-02T15:03:37.598 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/hw/mlx5 2024-02-02T15:03:37.599 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-02-02T15:03:37.599 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/hw/mthca 2024-02-02T15:03:37.599 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-02-02T15:03:37.599 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/ulp 2024-02-02T15:03:37.599 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/ulp/ipoib 2024-02-02T15:03:37.599 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-02-02T15:03:37.599 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/ulp/iser 2024-02-02T15:03:37.599 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-02-02T15:03:37.599 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/ulp/isert 2024-02-02T15:03:37.599 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-02-02T15:03:37.599 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/ulp/srp 2024-02-02T15:03:37.599 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-02-02T15:03:37.599 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/ulp/srpt 2024-02-02T15:03:37.599 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-02-02T15:03:37.599 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input 2024-02-02T15:03:37.600 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/gameport 2024-02-02T15:03:37.600 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-02-02T15:03:37.600 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/gameport/fm801-gp.ko 2024-02-02T15:03:37.600 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/gameport/gameport.ko 2024-02-02T15:03:37.600 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/joydev.ko 2024-02-02T15:03:37.600 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/keyboard 2024-02-02T15:03:37.600 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/keyboard/gpio_keys.ko 2024-02-02T15:03:37.600 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/mouse 2024-02-02T15:03:37.600 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/mouse/appletouch.ko 2024-02-02T15:03:37.600 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/mouse/bcm5974.ko 2024-02-02T15:03:37.600 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/mouse/cyapatp.ko 2024-02-02T15:03:37.600 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/mouse/elan_i2c.ko 2024-02-02T15:03:37.600 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/mouse/sermouse.ko 2024-02-02T15:03:37.600 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-02-02T15:03:37.600 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/mouse/synaptics_usb.ko 2024-02-02T15:03:37.600 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/mouse/vsxxxaa.ko 2024-02-02T15:03:37.601 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/rmi4 2024-02-02T15:03:37.601 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/rmi4/rmi_core.ko 2024-02-02T15:03:37.601 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/serio 2024-02-02T15:03:37.601 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/serio/altera_ps2.ko 2024-02-02T15:03:37.601 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/serio/arc_ps2.ko 2024-02-02T15:03:37.601 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-02-02T15:03:37.601 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/serio/serio_raw.ko 2024-02-02T15:03:37.601 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/input/sparse-keymap.ko 2024-02-02T15:03:37.601 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/leds 2024-02-02T15:03:37.601 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/leds/leds-blinkm.ko 2024-02-02T15:03:37.601 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/leds/leds-lm3530.ko 2024-02-02T15:03:37.601 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/leds/leds-lp3944.ko 2024-02-02T15:03:37.601 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/leds/leds-ss4200.ko 2024-02-02T15:03:37.601 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/leds/trigger 2024-02-02T15:03:37.602 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-02-02T15:03:37.602 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-02-02T15:03:37.602 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-02-02T15:03:37.602 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-02-02T15:03:37.602 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-02-02T15:03:37.602 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-02-02T15:03:37.602 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-02-02T15:03:37.602 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-02-02T15:03:37.602 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-02-02T15:03:37.602 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md 2024-02-02T15:03:37.602 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/bcache 2024-02-02T15:03:37.602 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/bcache/bcache.ko 2024-02-02T15:03:37.602 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/dm-bio-prison.ko 2024-02-02T15:03:37.602 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/dm-cache-smq.ko 2024-02-02T15:03:37.602 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/dm-cache.ko 2024-02-02T15:03:37.602 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/dm-crypt.ko 2024-02-02T15:03:37.603 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/dm-delay.ko 2024-02-02T15:03:37.603 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/dm-flakey.ko 2024-02-02T15:03:37.603 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/dm-log-userspace.ko 2024-02-02T15:03:37.603 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/dm-multipath.ko 2024-02-02T15:03:37.603 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/dm-queue-length.ko 2024-02-02T15:03:37.603 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/dm-raid.ko 2024-02-02T15:03:37.603 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/dm-round-robin.ko 2024-02-02T15:03:37.603 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/dm-service-time.ko 2024-02-02T15:03:37.603 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/dm-switch.ko 2024-02-02T15:03:37.603 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/dm-thin-pool.ko 2024-02-02T15:03:37.603 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/dm-verity.ko 2024-02-02T15:03:37.603 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/faulty.ko 2024-02-02T15:03:37.603 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/linear.ko 2024-02-02T15:03:37.603 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/multipath.ko 2024-02-02T15:03:37.603 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/persistent-data 2024-02-02T15:03:37.603 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-02-02T15:03:37.603 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/raid0.ko 2024-02-02T15:03:37.604 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/raid1.ko 2024-02-02T15:03:37.604 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/raid10.ko 2024-02-02T15:03:37.604 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/md/raid456.ko 2024-02-02T15:03:37.604 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/message 2024-02-02T15:03:37.604 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/message/fusion 2024-02-02T15:03:37.604 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/message/fusion/mptbase.ko 2024-02-02T15:03:37.604 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/message/fusion/mptctl.ko 2024-02-02T15:03:37.604 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/message/fusion/mptfc.ko 2024-02-02T15:03:37.604 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/message/fusion/mptlan.ko 2024-02-02T15:03:37.604 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/message/fusion/mptsas.ko 2024-02-02T15:03:37.604 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/message/fusion/mptscsih.ko 2024-02-02T15:03:37.604 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/message/fusion/mptspi.ko 2024-02-02T15:03:37.604 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/mfd 2024-02-02T15:03:37.604 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/mfd/lpc_ich.ko 2024-02-02T15:03:37.604 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/mfd/lpc_sch.ko 2024-02-02T15:03:37.604 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/mfd/mfd-core.ko 2024-02-02T15:03:37.605 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/mfd/sm501.ko 2024-02-02T15:03:37.605 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/mfd/viperboard.ko 2024-02-02T15:03:37.605 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/mfd/vx855.ko 2024-02-02T15:03:37.605 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/mfd/wl1273-core.ko 2024-02-02T15:03:37.605 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc 2024-02-02T15:03:37.605 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/altera-stapl 2024-02-02T15:03:37.605 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-02-02T15:03:37.605 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/apds9802als.ko 2024-02-02T15:03:37.605 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/apds990x.ko 2024-02-02T15:03:37.605 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/bh1770glc.ko 2024-02-02T15:03:37.605 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/cb710 2024-02-02T15:03:37.605 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/cb710/cb710.ko 2024-02-02T15:03:37.605 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/eeprom 2024-02-02T15:03:37.605 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/eeprom/at24.ko 2024-02-02T15:03:37.605 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-02-02T15:03:37.605 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/eeprom/max6875.ko 2024-02-02T15:03:37.606 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/enclosure.ko 2024-02-02T15:03:37.606 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/hpilo.ko 2024-02-02T15:03:37.606 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/isl29003.ko 2024-02-02T15:03:37.606 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/isl29020.ko 2024-02-02T15:03:37.606 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/lis3lv02d 2024-02-02T15:03:37.606 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-02-02T15:03:37.606 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-02-02T15:03:37.606 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/mei 2024-02-02T15:03:37.606 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/mei/mei-me.ko 2024-02-02T15:03:37.606 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/mei/mei-txe.ko 2024-02-02T15:03:37.606 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/mei/mei.ko 2024-02-02T15:03:37.606 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/sgi-gru 2024-02-02T15:03:37.606 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/sgi-gru/gru.ko 2024-02-02T15:03:37.606 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/sgi-xp 2024-02-02T15:03:37.606 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/sgi-xp/xp.ko 2024-02-02T15:03:37.607 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/sgi-xp/xpc.ko 2024-02-02T15:03:37.607 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-02-02T15:03:37.607 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/tifm_7xx1.ko 2024-02-02T15:03:37.607 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/tifm_core.ko 2024-02-02T15:03:37.607 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/tsl2550.ko 2024-02-02T15:03:37.607 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/vmw_balloon.ko 2024-02-02T15:03:37.607 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/vmw_vmci 2024-02-02T15:03:37.607 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-02-02T15:03:37.607 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net 2024-02-02T15:03:37.607 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/bonding 2024-02-02T15:03:37.607 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/bonding/bonding.ko 2024-02-02T15:03:37.608 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/dsa 2024-02-02T15:03:37.608 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/dsa/b53 2024-02-02T15:03:37.608 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/dsa/b53/b53_common.ko 2024-02-02T15:03:37.608 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/dsa/bcm-sf2.ko 2024-02-02T15:03:37.608 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/dsa/mv88e6060.ko 2024-02-02T15:03:37.608 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/dsa/mv88e6xxx 2024-02-02T15:03:37.608 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-02-02T15:03:37.608 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/dummy.ko 2024-02-02T15:03:37.608 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/eql.ko 2024-02-02T15:03:37.608 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet 2024-02-02T15:03:37.608 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/3com 2024-02-02T15:03:37.608 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-02-02T15:03:37.609 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-02-02T15:03:37.609 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-02-02T15:03:37.609 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-02-02T15:03:37.609 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/8390 2024-02-02T15:03:37.609 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/8390/8390.ko 2024-02-02T15:03:37.609 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-02-02T15:03:37.609 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-02-02T15:03:37.609 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-02-02T15:03:37.609 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/adaptec 2024-02-02T15:03:37.609 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-02-02T15:03:37.609 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/agere 2024-02-02T15:03:37.609 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/agere/et131x.ko 2024-02-02T15:03:37.609 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/alteon 2024-02-02T15:03:37.609 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-02-02T15:03:37.609 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/altera 2024-02-02T15:03:37.610 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-02-02T15:03:37.610 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/amd 2024-02-02T15:03:37.610 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-02-02T15:03:37.610 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-02-02T15:03:37.610 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-02-02T15:03:37.610 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/atheros 2024-02-02T15:03:37.611 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/atheros/alx 2024-02-02T15:03:37.611 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-02-02T15:03:37.611 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/atheros/atl1c 2024-02-02T15:03:37.611 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-02-02T15:03:37.611 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/atheros/atl1e 2024-02-02T15:03:37.611 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-02-02T15:03:37.612 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/atheros/atlx 2024-02-02T15:03:37.612 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-02-02T15:03:37.612 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-02-02T15:03:37.612 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/broadcom 2024-02-02T15:03:37.612 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-02-02T15:03:37.612 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-02-02T15:03:37.612 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-02-02T15:03:37.612 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-02-02T15:03:37.612 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-02-02T15:03:37.613 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/broadcom/genet 2024-02-02T15:03:37.613 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-02-02T15:03:37.613 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-02-02T15:03:37.613 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/brocade 2024-02-02T15:03:37.613 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/brocade/bna 2024-02-02T15:03:37.613 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-02-02T15:03:37.613 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/chelsio 2024-02-02T15:03:37.613 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/chelsio/cxgb 2024-02-02T15:03:37.613 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-02-02T15:03:37.614 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-02-02T15:03:37.614 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-02-02T15:03:37.614 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-02-02T15:03:37.614 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-02-02T15:03:37.614 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-02-02T15:03:37.614 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-02-02T15:03:37.614 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-02-02T15:03:37.614 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-02-02T15:03:37.614 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/cisco 2024-02-02T15:03:37.615 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/cisco/enic 2024-02-02T15:03:37.615 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-02-02T15:03:37.615 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/dec 2024-02-02T15:03:37.615 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/dec/tulip 2024-02-02T15:03:37.615 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-02-02T15:03:37.615 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-02-02T15:03:37.615 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-02-02T15:03:37.615 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-02-02T15:03:37.615 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-02-02T15:03:37.615 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-02-02T15:03:37.615 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/dlink 2024-02-02T15:03:37.615 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-02-02T15:03:37.615 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-02-02T15:03:37.615 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/dnet.ko 2024-02-02T15:03:37.615 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/emulex 2024-02-02T15:03:37.616 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/emulex/benet 2024-02-02T15:03:37.616 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-02-02T15:03:37.616 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/ethoc.ko 2024-02-02T15:03:37.616 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/fealnx.ko 2024-02-02T15:03:37.616 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel 2024-02-02T15:03:37.616 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/e100.ko 2024-02-02T15:03:37.616 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/e1000 2024-02-02T15:03:37.616 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-02-02T15:03:37.616 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/e1000e 2024-02-02T15:03:37.616 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-02-02T15:03:37.616 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/fm10k 2024-02-02T15:03:37.616 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-02-02T15:03:37.616 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/i40e 2024-02-02T15:03:37.616 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-02-02T15:03:37.616 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/iavf 2024-02-02T15:03:37.617 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-02-02T15:03:37.617 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/igb 2024-02-02T15:03:37.617 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-02-02T15:03:37.617 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/igbvf 2024-02-02T15:03:37.617 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-02-02T15:03:37.617 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/ixgbe 2024-02-02T15:03:37.617 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-02-02T15:03:37.617 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/ixgbevf 2024-02-02T15:03:37.617 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-02-02T15:03:37.617 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/jme.ko 2024-02-02T15:03:37.617 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/marvell 2024-02-02T15:03:37.617 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-02-02T15:03:37.617 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/marvell/skge.ko 2024-02-02T15:03:37.617 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-02-02T15:03:37.617 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/mellanox 2024-02-02T15:03:37.618 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/mellanox/mlx4 2024-02-02T15:03:37.618 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-02-02T15:03:37.618 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-02-02T15:03:37.618 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/mellanox/mlx5 2024-02-02T15:03:37.618 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-02-02T15:03:37.618 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-02-02T15:03:37.618 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/micrel 2024-02-02T15:03:37.618 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-02-02T15:03:37.618 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/myricom 2024-02-02T15:03:37.618 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/myricom/myri10ge 2024-02-02T15:03:37.618 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-02-02T15:03:37.618 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/natsemi 2024-02-02T15:03:37.618 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-02-02T15:03:37.619 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-02-02T15:03:37.619 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/neterion 2024-02-02T15:03:37.619 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-02-02T15:03:37.619 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/nvidia 2024-02-02T15:03:37.619 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-02-02T15:03:37.619 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/packetengines 2024-02-02T15:03:37.619 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-02-02T15:03:37.619 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-02-02T15:03:37.619 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/qlogic 2024-02-02T15:03:37.619 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/qlogic/netxen 2024-02-02T15:03:37.619 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-02-02T15:03:37.619 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-02-02T15:03:37.619 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-02-02T15:03:37.619 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-02-02T15:03:37.619 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/rdc 2024-02-02T15:03:37.619 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-02-02T15:03:37.619 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/realtek 2024-02-02T15:03:37.620 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-02-02T15:03:37.620 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-02-02T15:03:37.620 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/realtek/atp.ko 2024-02-02T15:03:37.620 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-02-02T15:03:37.620 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/rocker 2024-02-02T15:03:37.620 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-02-02T15:03:37.620 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/sfc 2024-02-02T15:03:37.620 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-02-02T15:03:37.620 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/silan 2024-02-02T15:03:37.620 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-02-02T15:03:37.620 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/sis 2024-02-02T15:03:37.620 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/sis/sis190.ko 2024-02-02T15:03:37.620 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/sis/sis900.ko 2024-02-02T15:03:37.621 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/smsc 2024-02-02T15:03:37.621 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-02-02T15:03:37.621 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-02-02T15:03:37.621 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-02-02T15:03:37.621 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-02-02T15:03:37.621 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/stmicro 2024-02-02T15:03:37.621 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/stmicro/stmmac 2024-02-02T15:03:37.621 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-02-02T15:03:37.621 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-02-02T15:03:37.621 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/sun 2024-02-02T15:03:37.621 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/sun/cassini.ko 2024-02-02T15:03:37.621 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/sun/niu.ko 2024-02-02T15:03:37.621 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/sun/sungem.ko 2024-02-02T15:03:37.621 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-02-02T15:03:37.622 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/tehuti 2024-02-02T15:03:37.622 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-02-02T15:03:37.622 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/ti 2024-02-02T15:03:37.622 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/ti/tlan.ko 2024-02-02T15:03:37.622 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/via 2024-02-02T15:03:37.622 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-02-02T15:03:37.622 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-02-02T15:03:37.622 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/wiznet 2024-02-02T15:03:37.622 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-02-02T15:03:37.622 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-02-02T15:03:37.622 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/xircom 2024-02-02T15:03:37.622 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-02-02T15:03:37.622 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/geneve.ko 2024-02-02T15:03:37.622 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/hyperv 2024-02-02T15:03:37.622 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-02-02T15:03:37.623 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ieee802154 2024-02-02T15:03:37.623 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ieee802154/fakelb.ko 2024-02-02T15:03:37.623 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ifb.ko 2024-02-02T15:03:37.623 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ipvlan 2024-02-02T15:03:37.623 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ipvlan/ipvlan.ko 2024-02-02T15:03:37.623 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/macvlan.ko 2024-02-02T15:03:37.623 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/macvtap.ko 2024-02-02T15:03:37.623 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/mdio 2024-02-02T15:03:37.623 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/mdio.ko 2024-02-02T15:03:37.623 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-02-02T15:03:37.623 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-02-02T15:03:37.623 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/mdio/mdio-regmap.ko 2024-02-02T15:03:37.623 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/mii.ko 2024-02-02T15:03:37.624 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/net_failover.ko 2024-02-02T15:03:37.624 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/netconsole.ko 2024-02-02T15:03:37.624 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/nlmon.ko 2024-02-02T15:03:37.624 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/pcs 2024-02-02T15:03:37.624 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/pcs/pcs-lynx.ko 2024-02-02T15:03:37.624 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-02-02T15:03:37.624 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy 2024-02-02T15:03:37.624 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/amd.ko 2024-02-02T15:03:37.624 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-02-02T15:03:37.624 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-02-02T15:03:37.624 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/bcm7xxx.ko 2024-02-02T15:03:37.624 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/bcm87xx.ko 2024-02-02T15:03:37.624 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/broadcom.ko 2024-02-02T15:03:37.625 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/cicada.ko 2024-02-02T15:03:37.625 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/davicom.ko 2024-02-02T15:03:37.625 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/dp83640.ko 2024-02-02T15:03:37.625 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/et1011c.ko 2024-02-02T15:03:37.625 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/icplus.ko 2024-02-02T15:03:37.625 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/lxt.ko 2024-02-02T15:03:37.625 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/marvell.ko 2024-02-02T15:03:37.625 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/micrel.ko 2024-02-02T15:03:37.625 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/national.ko 2024-02-02T15:03:37.625 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/phylink.ko 2024-02-02T15:03:37.625 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/qsemi.ko 2024-02-02T15:03:37.625 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/realtek.ko 2024-02-02T15:03:37.625 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/smsc.ko 2024-02-02T15:03:37.625 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/ste10Xp.ko 2024-02-02T15:03:37.626 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/phy/vitesse.ko 2024-02-02T15:03:37.626 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ppp 2024-02-02T15:03:37.626 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ppp/bsd_comp.ko 2024-02-02T15:03:37.626 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ppp/ppp_async.ko 2024-02-02T15:03:37.626 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ppp/ppp_deflate.ko 2024-02-02T15:03:37.626 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ppp/ppp_generic.ko 2024-02-02T15:03:37.626 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ppp/ppp_mppe.ko 2024-02-02T15:03:37.626 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ppp/ppp_synctty.ko 2024-02-02T15:03:37.626 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ppp/pppoe.ko 2024-02-02T15:03:37.626 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ppp/pppox.ko 2024-02-02T15:03:37.626 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/ppp/pptp.ko 2024-02-02T15:03:37.626 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/slip 2024-02-02T15:03:37.626 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/slip/slhc.ko 2024-02-02T15:03:37.626 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/slip/slip.ko 2024-02-02T15:03:37.626 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/sungem_phy.ko 2024-02-02T15:03:37.627 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/tap.ko 2024-02-02T15:03:37.627 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/team 2024-02-02T15:03:37.627 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/team/team.ko 2024-02-02T15:03:37.627 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/team/team_mode_activebackup.ko 2024-02-02T15:03:37.627 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/team/team_mode_broadcast.ko 2024-02-02T15:03:37.627 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-02-02T15:03:37.627 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/team/team_mode_random.ko 2024-02-02T15:03:37.627 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-02-02T15:03:37.627 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/tun.ko 2024-02-02T15:03:37.627 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/veth.ko 2024-02-02T15:03:37.627 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/virtio_net.ko 2024-02-02T15:03:37.627 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/vmxnet3 2024-02-02T15:03:37.627 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-02-02T15:03:37.627 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/vxlan 2024-02-02T15:03:37.627 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/vxlan/vxlan.ko 2024-02-02T15:03:37.627 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/xen-netback 2024-02-02T15:03:37.628 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/xen-netback/xen-netback.ko 2024-02-02T15:03:37.628 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/net/xen-netfront.ko 2024-02-02T15:03:37.628 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ntb 2024-02-02T15:03:37.628 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ntb/ntb.ko 2024-02-02T15:03:37.628 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/nvme 2024-02-02T15:03:37.628 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/nvme/host 2024-02-02T15:03:37.628 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/nvme/host/nvme-core.ko 2024-02-02T15:03:37.628 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/nvme/host/nvme.ko 2024-02-02T15:03:37.628 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/parport 2024-02-02T15:03:37.628 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/parport/parport.ko 2024-02-02T15:03:37.628 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/parport/parport_cs.ko 2024-02-02T15:03:37.628 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/parport/parport_pc.ko 2024-02-02T15:03:37.628 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/parport/parport_serial.ko 2024-02-02T15:03:37.629 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pci 2024-02-02T15:03:37.629 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pci/hotplug 2024-02-02T15:03:37.629 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-02-02T15:03:37.629 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pci/pcie 2024-02-02T15:03:37.629 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pci/pcie/aer_inject.ko 2024-02-02T15:03:37.629 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pci/xen-pcifront.ko 2024-02-02T15:03:37.629 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pcmcia 2024-02-02T15:03:37.629 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pcmcia/i82092.ko 2024-02-02T15:03:37.629 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pcmcia/pd6729.ko 2024-02-02T15:03:37.629 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pcmcia/yenta_socket.ko 2024-02-02T15:03:37.629 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pinctrl 2024-02-02T15:03:37.629 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pinctrl/intel 2024-02-02T15:03:37.629 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-02-02T15:03:37.629 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform 2024-02-02T15:03:37.629 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86 2024-02-02T15:03:37.629 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/acer-wmi.ko 2024-02-02T15:03:37.630 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/acerhdf.ko 2024-02-02T15:03:37.630 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/apple-gmux.ko 2024-02-02T15:03:37.630 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/asus-laptop.ko 2024-02-02T15:03:37.630 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-02-02T15:03:37.630 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/asus-wmi.ko 2024-02-02T15:03:37.630 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/classmate-laptop.ko 2024-02-02T15:03:37.630 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-02-02T15:03:37.630 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-02-02T15:03:37.630 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-02-02T15:03:37.630 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-02-02T15:03:37.630 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/hdaps.ko 2024-02-02T15:03:37.630 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/intel 2024-02-02T15:03:37.630 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-02-02T15:03:37.630 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/intel_ips.ko 2024-02-02T15:03:37.630 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/msi-wmi.ko 2024-02-02T15:03:37.630 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/mxm-wmi.ko 2024-02-02T15:03:37.631 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-02-02T15:03:37.631 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/samsung-laptop.ko 2024-02-02T15:03:37.631 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/samsung-q10.ko 2024-02-02T15:03:37.631 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/topstar-laptop.ko 2024-02-02T15:03:37.631 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-02-02T15:03:37.631 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/toshiba_haps.ko 2024-02-02T15:03:37.631 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/wmi-bmof.ko 2024-02-02T15:03:37.631 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/platform/x86/wmi.ko 2024-02-02T15:03:37.631 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/powercap 2024-02-02T15:03:37.631 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/powercap/intel_rapl_common.ko 2024-02-02T15:03:37.631 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/powercap/intel_rapl_msr.ko 2024-02-02T15:03:37.631 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pps 2024-02-02T15:03:37.631 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pps/clients 2024-02-02T15:03:37.631 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pps/clients/pps-gpio.ko 2024-02-02T15:03:37.631 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pps/clients/pps-ldisc.ko 2024-02-02T15:03:37.632 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pps/clients/pps_parport.ko 2024-02-02T15:03:37.632 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/pps/pps_core.ko 2024-02-02T15:03:37.632 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ptp 2024-02-02T15:03:37.632 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ptp/ptp.ko 2024-02-02T15:03:37.632 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ptp/ptp_kvm.ko 2024-02-02T15:03:37.632 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc 2024-02-02T15:03:37.632 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-bq32k.ko 2024-02-02T15:03:37.632 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-ds1286.ko 2024-02-02T15:03:37.632 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-ds1307.ko 2024-02-02T15:03:37.632 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-ds1374.ko 2024-02-02T15:03:37.632 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-ds1511.ko 2024-02-02T15:03:37.632 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-ds1553.ko 2024-02-02T15:03:37.632 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-ds1672.ko 2024-02-02T15:03:37.633 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-ds1742.ko 2024-02-02T15:03:37.633 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-ds2404.ko 2024-02-02T15:03:37.633 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-ds3232.ko 2024-02-02T15:03:37.633 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-em3027.ko 2024-02-02T15:03:37.633 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-fm3130.ko 2024-02-02T15:03:37.633 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-isl12022.ko 2024-02-02T15:03:37.633 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-isl1208.ko 2024-02-02T15:03:37.633 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-m41t80.ko 2024-02-02T15:03:37.633 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-m48t35.ko 2024-02-02T15:03:37.633 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-m48t59.ko 2024-02-02T15:03:37.633 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-max6900.ko 2024-02-02T15:03:37.633 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-msm6242.ko 2024-02-02T15:03:37.633 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-pcf2127.ko 2024-02-02T15:03:37.633 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-pcf85063.ko 2024-02-02T15:03:37.633 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-pcf8523.ko 2024-02-02T15:03:37.633 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-pcf8563.ko 2024-02-02T15:03:37.634 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-pcf8583.ko 2024-02-02T15:03:37.634 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-rp5c01.ko 2024-02-02T15:03:37.634 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-rs5c372.ko 2024-02-02T15:03:37.634 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-02-02T15:03:37.634 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-rx8025.ko 2024-02-02T15:03:37.634 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-rx8581.ko 2024-02-02T15:03:37.634 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-02-02T15:03:37.634 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/rtc/rtc-x1205.ko 2024-02-02T15:03:37.634 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi 2024-02-02T15:03:37.634 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/3w-9xxx.ko 2024-02-02T15:03:37.634 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/3w-sas.ko 2024-02-02T15:03:37.634 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/3w-xxxx.ko 2024-02-02T15:03:37.634 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/BusLogic.ko 2024-02-02T15:03:37.634 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/a100u2w.ko 2024-02-02T15:03:37.634 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/aacraid 2024-02-02T15:03:37.634 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/aacraid/aacraid.ko 2024-02-02T15:03:37.635 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/advansys.ko 2024-02-02T15:03:37.635 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/aic7xxx 2024-02-02T15:03:37.635 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-02-02T15:03:37.635 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-02-02T15:03:37.635 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/am53c974.ko 2024-02-02T15:03:37.635 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/arcmsr 2024-02-02T15:03:37.635 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-02-02T15:03:37.635 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/atp870u.ko 2024-02-02T15:03:37.635 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/be2iscsi 2024-02-02T15:03:37.635 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-02-02T15:03:37.635 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/bfa 2024-02-02T15:03:37.635 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/bfa/bfa.ko 2024-02-02T15:03:37.635 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/bnx2fc 2024-02-02T15:03:37.635 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-02-02T15:03:37.635 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/bnx2i 2024-02-02T15:03:37.636 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-02-02T15:03:37.636 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/ch.ko 2024-02-02T15:03:37.636 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/csiostor 2024-02-02T15:03:37.636 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/csiostor/csiostor.ko 2024-02-02T15:03:37.636 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/cxgbi 2024-02-02T15:03:37.636 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/cxgbi/cxgb3i 2024-02-02T15:03:37.636 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-02-02T15:03:37.636 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/cxgbi/cxgb4i 2024-02-02T15:03:37.636 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-02-02T15:03:37.636 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-02-02T15:03:37.636 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/dc395x.ko 2024-02-02T15:03:37.636 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/device_handler 2024-02-02T15:03:37.636 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-02-02T15:03:37.637 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-02-02T15:03:37.637 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-02-02T15:03:37.637 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-02-02T15:03:37.637 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/dmx3191d.ko 2024-02-02T15:03:37.637 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/esas2r 2024-02-02T15:03:37.637 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/esas2r/esas2r.ko 2024-02-02T15:03:37.637 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/esp_scsi.ko 2024-02-02T15:03:37.637 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/fcoe 2024-02-02T15:03:37.637 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/fcoe/fcoe.ko 2024-02-02T15:03:37.637 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-02-02T15:03:37.637 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/fnic 2024-02-02T15:03:37.638 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/fnic/fnic.ko 2024-02-02T15:03:37.638 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/hpsa.ko 2024-02-02T15:03:37.638 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/hptiop.ko 2024-02-02T15:03:37.638 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/hv_storvsc.ko 2024-02-02T15:03:37.638 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/initio.ko 2024-02-02T15:03:37.638 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/ipr.ko 2024-02-02T15:03:37.638 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/ips.ko 2024-02-02T15:03:37.638 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/isci 2024-02-02T15:03:37.638 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/isci/isci.ko 2024-02-02T15:03:37.638 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-02-02T15:03:37.638 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/iscsi_tcp.ko 2024-02-02T15:03:37.638 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/libfc 2024-02-02T15:03:37.638 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/libfc/libfc.ko 2024-02-02T15:03:37.638 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/libiscsi.ko 2024-02-02T15:03:37.639 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/libiscsi_tcp.ko 2024-02-02T15:03:37.639 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/libsas 2024-02-02T15:03:37.639 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/libsas/libsas.ko 2024-02-02T15:03:37.639 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/lpfc 2024-02-02T15:03:37.639 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/lpfc/lpfc.ko 2024-02-02T15:03:37.639 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/megaraid 2024-02-02T15:03:37.639 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/megaraid.ko 2024-02-02T15:03:37.639 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-02-02T15:03:37.640 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-02-02T15:03:37.640 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-02-02T15:03:37.640 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/mpt3sas 2024-02-02T15:03:37.640 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-02-02T15:03:37.640 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/mvsas 2024-02-02T15:03:37.640 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/mvsas/mvsas.ko 2024-02-02T15:03:37.640 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/mvumi.ko 2024-02-02T15:03:37.640 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/pm8001 2024-02-02T15:03:37.640 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-02-02T15:03:37.640 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/pmcraid.ko 2024-02-02T15:03:37.640 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/qla1280.ko 2024-02-02T15:03:37.640 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/qla2xxx 2024-02-02T15:03:37.640 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-02-02T15:03:37.641 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-02-02T15:03:37.641 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/qla4xxx 2024-02-02T15:03:37.641 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-02-02T15:03:37.641 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/raid_class.ko 2024-02-02T15:03:37.641 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/scsi_debug.ko 2024-02-02T15:03:37.641 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/scsi_transport_fc.ko 2024-02-02T15:03:37.641 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-02-02T15:03:37.641 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/scsi_transport_sas.ko 2024-02-02T15:03:37.641 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/scsi_transport_spi.ko 2024-02-02T15:03:37.641 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/scsi_transport_srp.ko 2024-02-02T15:03:37.641 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/ses.ko 2024-02-02T15:03:37.642 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/st.ko 2024-02-02T15:03:37.642 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/stex.ko 2024-02-02T15:03:37.642 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/sym53c8xx_2 2024-02-02T15:03:37.642 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-02-02T15:03:37.642 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/virtio_scsi.ko 2024-02-02T15:03:37.642 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/vmw_pvscsi.ko 2024-02-02T15:03:37.642 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/scsi/wd719x.ko 2024-02-02T15:03:37.643 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ssb 2024-02-02T15:03:37.643 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ssb/ssb.ko 2024-02-02T15:03:37.643 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/target 2024-02-02T15:03:37.643 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/target/iscsi 2024-02-02T15:03:37.643 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-02-02T15:03:37.643 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/target/loopback 2024-02-02T15:03:37.644 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/target/loopback/tcm_loop.ko 2024-02-02T15:03:37.644 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/target/target_core_file.ko 2024-02-02T15:03:37.644 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/target/target_core_iblock.ko 2024-02-02T15:03:37.644 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/target/target_core_mod.ko 2024-02-02T15:03:37.644 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/target/target_core_pscsi.ko 2024-02-02T15:03:37.644 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/target/target_core_user.ko 2024-02-02T15:03:37.644 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/target/tcm_fc 2024-02-02T15:03:37.644 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-02-02T15:03:37.644 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal 2024-02-02T15:03:37.645 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel 2024-02-02T15:03:37.645 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/int340x_thermal 2024-02-02T15:03:37.645 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-02-02T15:03:37.645 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-02-02T15:03:37.645 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-02-02T15:03:37.645 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-02-02T15:03:37.645 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-02-02T15:03:37.645 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-02-02T15:03:37.645 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-02-02T15:03:37.645 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-02-02T15:03:37.645 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-02-02T15:03:37.645 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-02-02T15:03:37.646 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-02-02T15:03:37.646 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-02-02T15:03:37.646 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-02-02T15:03:37.646 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-02-02T15:03:37.646 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-02-02T15:03:37.646 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-02-02T15:03:37.646 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-02-02T15:03:37.646 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-02-02T15:03:37.646 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/tty 2024-02-02T15:03:37.647 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/tty/ipwireless 2024-02-02T15:03:37.647 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-02-02T15:03:37.647 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/tty/n_gsm.ko 2024-02-02T15:03:37.647 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/tty/n_hdlc.ko 2024-02-02T15:03:37.647 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/tty/nozomi.ko 2024-02-02T15:03:37.647 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/tty/serial 2024-02-02T15:03:37.647 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/tty/serial/8250 2024-02-02T15:03:37.647 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-02-02T15:03:37.647 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/tty/serial/arc_uart.ko 2024-02-02T15:03:37.647 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/tty/serial/jsm 2024-02-02T15:03:37.647 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/tty/serial/jsm/jsm.ko 2024-02-02T15:03:37.647 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ufs 2024-02-02T15:03:37.647 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ufs/core 2024-02-02T15:03:37.647 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ufs/core/ufshcd-core.ko 2024-02-02T15:03:37.647 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ufs/host 2024-02-02T15:03:37.648 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-02-02T15:03:37.648 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/uio 2024-02-02T15:03:37.648 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/uio/uio.ko 2024-02-02T15:03:37.648 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/uio/uio_aec.ko 2024-02-02T15:03:37.648 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/uio/uio_cif.ko 2024-02-02T15:03:37.648 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/uio/uio_pci_generic.ko 2024-02-02T15:03:37.648 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/uio/uio_sercos3.ko 2024-02-02T15:03:37.648 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb 2024-02-02T15:03:37.648 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/class 2024-02-02T15:03:37.648 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/class/cdc-acm.ko 2024-02-02T15:03:37.648 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/class/cdc-wdm.ko 2024-02-02T15:03:37.648 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/class/usblp.ko 2024-02-02T15:03:37.648 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/class/usbtmc.ko 2024-02-02T15:03:37.648 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/host 2024-02-02T15:03:37.648 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/host/sl811-hcd.ko 2024-02-02T15:03:37.648 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/image 2024-02-02T15:03:37.649 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/image/mdc800.ko 2024-02-02T15:03:37.649 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/image/microtek.ko 2024-02-02T15:03:37.649 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc 2024-02-02T15:03:37.649 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/adutux.ko 2024-02-02T15:03:37.649 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/appledisplay.ko 2024-02-02T15:03:37.649 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/emi26.ko 2024-02-02T15:03:37.649 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/emi62.ko 2024-02-02T15:03:37.649 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/ezusb.ko 2024-02-02T15:03:37.649 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/idmouse.ko 2024-02-02T15:03:37.649 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/iowarrior.ko 2024-02-02T15:03:37.649 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/isight_firmware.ko 2024-02-02T15:03:37.649 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/ldusb.ko 2024-02-02T15:03:37.649 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/legousbtower.ko 2024-02-02T15:03:37.649 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/sisusbvga 2024-02-02T15:03:37.649 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-02-02T15:03:37.649 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/trancevibrator.ko 2024-02-02T15:03:37.649 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/usb3503.ko 2024-02-02T15:03:37.650 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/usblcd.ko 2024-02-02T15:03:37.650 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/usbsevseg.ko 2024-02-02T15:03:37.650 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/uss720.ko 2024-02-02T15:03:37.650 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/misc/yurex.ko 2024-02-02T15:03:37.650 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/phy 2024-02-02T15:03:37.650 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/phy/phy-generic.ko 2024-02-02T15:03:37.650 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial 2024-02-02T15:03:37.650 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/aircable.ko 2024-02-02T15:03:37.650 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/ark3116.ko 2024-02-02T15:03:37.650 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/belkin_sa.ko 2024-02-02T15:03:37.650 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/ch341.ko 2024-02-02T15:03:37.650 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/cp210x.ko 2024-02-02T15:03:37.650 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/cyberjack.ko 2024-02-02T15:03:37.651 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/cypress_m8.ko 2024-02-02T15:03:37.651 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/digi_acceleport.ko 2024-02-02T15:03:37.651 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/empeg.ko 2024-02-02T15:03:37.651 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/ftdi_sio.ko 2024-02-02T15:03:37.651 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/garmin_gps.ko 2024-02-02T15:03:37.651 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/io_edgeport.ko 2024-02-02T15:03:37.651 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/io_ti.ko 2024-02-02T15:03:37.651 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/ipaq.ko 2024-02-02T15:03:37.651 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/ipw.ko 2024-02-02T15:03:37.652 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/ir-usb.ko 2024-02-02T15:03:37.652 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-02-02T15:03:37.652 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/keyspan.ko 2024-02-02T15:03:37.652 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/keyspan_pda.ko 2024-02-02T15:03:37.652 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/kl5kusb105.ko 2024-02-02T15:03:37.652 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/kobil_sct.ko 2024-02-02T15:03:37.652 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/mct_u232.ko 2024-02-02T15:03:37.652 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/mos7720.ko 2024-02-02T15:03:37.652 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/mos7840.ko 2024-02-02T15:03:37.652 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/navman.ko 2024-02-02T15:03:37.652 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/omninet.ko 2024-02-02T15:03:37.652 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/opticon.ko 2024-02-02T15:03:37.652 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/option.ko 2024-02-02T15:03:37.652 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/oti6858.ko 2024-02-02T15:03:37.653 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/pl2303.ko 2024-02-02T15:03:37.653 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/qcaux.ko 2024-02-02T15:03:37.653 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/qcserial.ko 2024-02-02T15:03:37.653 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/quatech2.ko 2024-02-02T15:03:37.653 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/safe_serial.ko 2024-02-02T15:03:37.653 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/sierra.ko 2024-02-02T15:03:37.653 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/spcp8x5.ko 2024-02-02T15:03:37.653 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/ssu100.ko 2024-02-02T15:03:37.653 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/symbolserial.ko 2024-02-02T15:03:37.653 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-02-02T15:03:37.653 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-02-02T15:03:37.653 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/usb_debug.ko 2024-02-02T15:03:37.653 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/usb_wwan.ko 2024-02-02T15:03:37.653 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/visor.ko 2024-02-02T15:03:37.653 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/whiteheat.ko 2024-02-02T15:03:37.654 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/serial/xsens_mt.ko 2024-02-02T15:03:37.654 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/storage 2024-02-02T15:03:37.654 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/storage/uas.ko 2024-02-02T15:03:37.654 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/storage/ums-alauda.ko 2024-02-02T15:03:37.654 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/storage/ums-cypress.ko 2024-02-02T15:03:37.654 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/storage/ums-datafab.ko 2024-02-02T15:03:37.654 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-02-02T15:03:37.654 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/storage/ums-freecom.ko 2024-02-02T15:03:37.654 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/storage/ums-isd200.ko 2024-02-02T15:03:37.654 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-02-02T15:03:37.654 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/storage/ums-karma.ko 2024-02-02T15:03:37.654 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/storage/ums-onetouch.ko 2024-02-02T15:03:37.654 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/storage/ums-realtek.ko 2024-02-02T15:03:37.654 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/storage/ums-sddr09.ko 2024-02-02T15:03:37.654 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/storage/ums-sddr55.ko 2024-02-02T15:03:37.654 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/storage/ums-usbat.ko 2024-02-02T15:03:37.655 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/storage/usb-storage.ko 2024-02-02T15:03:37.655 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/usbip 2024-02-02T15:03:37.655 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/usbip/usbip-core.ko 2024-02-02T15:03:37.655 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/usbip/usbip-host.ko 2024-02-02T15:03:37.655 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-02-02T15:03:37.655 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/vfio 2024-02-02T15:03:37.655 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/vfio/pci 2024-02-02T15:03:37.655 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-02-02T15:03:37.655 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/vfio/pci/vfio-pci.ko 2024-02-02T15:03:37.655 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/vfio/vfio.ko 2024-02-02T15:03:37.655 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-02-02T15:03:37.655 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/vhost 2024-02-02T15:03:37.655 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/vhost/vhost.ko 2024-02-02T15:03:37.655 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/vhost/vhost_iotlb.ko 2024-02-02T15:03:37.655 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/vhost/vhost_net.ko 2024-02-02T15:03:37.656 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/vhost/vhost_scsi.ko 2024-02-02T15:03:37.656 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/video 2024-02-02T15:03:37.656 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/video/backlight 2024-02-02T15:03:37.656 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/video/backlight/apple_bl.ko 2024-02-02T15:03:37.656 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/video/backlight/lcd.ko 2024-02-02T15:03:37.656 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/video/backlight/platform_lcd.ko 2024-02-02T15:03:37.656 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/virtio 2024-02-02T15:03:37.656 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/virtio/virtio.ko 2024-02-02T15:03:37.656 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/virtio/virtio_balloon.ko 2024-02-02T15:03:37.656 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/virtio/virtio_mmio.ko 2024-02-02T15:03:37.656 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/virtio/virtio_pci.ko 2024-02-02T15:03:37.656 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-02-02T15:03:37.656 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-02-02T15:03:37.657 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/virtio/virtio_ring.ko 2024-02-02T15:03:37.657 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1 2024-02-02T15:03:37.657 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1/masters 2024-02-02T15:03:37.657 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1/masters/ds2482.ko 2024-02-02T15:03:37.657 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1/masters/ds2490.ko 2024-02-02T15:03:37.657 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1/slaves 2024-02-02T15:03:37.657 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-02-02T15:03:37.657 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-02-02T15:03:37.657 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-02-02T15:03:37.657 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-02-02T15:03:37.657 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-02-02T15:03:37.657 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-02-02T15:03:37.657 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-02-02T15:03:37.657 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-02-02T15:03:37.657 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-02-02T15:03:37.657 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1/slaves/w1_smem.ko 2024-02-02T15:03:37.657 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1/slaves/w1_therm.ko 2024-02-02T15:03:37.658 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/w1/wire.ko 2024-02-02T15:03:37.658 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog 2024-02-02T15:03:37.658 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/alim1535_wdt.ko 2024-02-02T15:03:37.658 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/alim7101_wdt.ko 2024-02-02T15:03:37.658 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/f71808e_wdt.ko 2024-02-02T15:03:37.658 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/hpwdt.ko 2024-02-02T15:03:37.658 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/i6300esb.ko 2024-02-02T15:03:37.658 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-02-02T15:03:37.658 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/iTCO_wdt.ko 2024-02-02T15:03:37.658 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/ib700wdt.ko 2024-02-02T15:03:37.658 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/ibmasr.ko 2024-02-02T15:03:37.658 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-02-02T15:03:37.658 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/it8712f_wdt.ko 2024-02-02T15:03:37.658 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/it87_wdt.ko 2024-02-02T15:03:37.658 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/machzwd.ko 2024-02-02T15:03:37.658 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/nv_tco.ko 2024-02-02T15:03:37.659 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/pcwd_pci.ko 2024-02-02T15:03:37.659 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/pcwd_usb.ko 2024-02-02T15:03:37.659 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-02-02T15:03:37.659 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/sch311x_wdt.ko 2024-02-02T15:03:37.659 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/softdog.ko 2024-02-02T15:03:37.659 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/sp5100_tco.ko 2024-02-02T15:03:37.659 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/via_wdt.ko 2024-02-02T15:03:37.659 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-02-02T15:03:37.659 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/w83877f_wdt.ko 2024-02-02T15:03:37.659 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/w83977f_wdt.ko 2024-02-02T15:03:37.659 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/wdt_pci.ko 2024-02-02T15:03:37.659 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/watchdog/xen_wdt.ko 2024-02-02T15:03:37.659 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/xen 2024-02-02T15:03:37.659 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/xen/xen-acpi-processor.ko 2024-02-02T15:03:37.660 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/xen/xen-evtchn.ko 2024-02-02T15:03:37.660 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/xen/xen-gntalloc.ko 2024-02-02T15:03:37.660 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/xen/xen-gntdev.ko 2024-02-02T15:03:37.660 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/xen/xen-pciback 2024-02-02T15:03:37.660 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-02-02T15:03:37.660 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/xen/xen-privcmd.ko 2024-02-02T15:03:37.660 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/xen/xenfs 2024-02-02T15:03:37.660 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/drivers/xen/xenfs/xenfs.ko 2024-02-02T15:03:37.660 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs 2024-02-02T15:03:37.660 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/9p 2024-02-02T15:03:37.660 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/9p/9p.ko 2024-02-02T15:03:37.661 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/affs 2024-02-02T15:03:37.661 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/affs/affs.ko 2024-02-02T15:03:37.661 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/befs 2024-02-02T15:03:37.661 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/befs/befs.ko 2024-02-02T15:03:37.661 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/binfmt_misc.ko 2024-02-02T15:03:37.661 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/btrfs 2024-02-02T15:03:37.661 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/btrfs/btrfs.ko 2024-02-02T15:03:37.661 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/cachefiles 2024-02-02T15:03:37.661 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/cachefiles/cachefiles.ko 2024-02-02T15:03:37.661 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ceph 2024-02-02T15:03:37.661 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ceph/ceph.ko 2024-02-02T15:03:37.661 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/coda 2024-02-02T15:03:37.661 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/coda/coda.ko 2024-02-02T15:03:37.661 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/cramfs 2024-02-02T15:03:37.661 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/cramfs/cramfs.ko 2024-02-02T15:03:37.661 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/dlm 2024-02-02T15:03:37.662 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/dlm/dlm.ko 2024-02-02T15:03:37.662 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ecryptfs 2024-02-02T15:03:37.662 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ecryptfs/ecryptfs.ko 2024-02-02T15:03:37.662 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/f2fs 2024-02-02T15:03:37.662 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/f2fs/f2fs.ko 2024-02-02T15:03:37.662 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/fat 2024-02-02T15:03:37.662 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/fat/fat.ko 2024-02-02T15:03:37.662 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/fat/msdos.ko 2024-02-02T15:03:37.662 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/fat/vfat.ko 2024-02-02T15:03:37.662 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/fscache 2024-02-02T15:03:37.662 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/fscache/fscache.ko 2024-02-02T15:03:37.662 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/fuse 2024-02-02T15:03:37.662 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/fuse/cuse.ko 2024-02-02T15:03:37.662 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/fuse/fuse.ko 2024-02-02T15:03:37.662 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/gfs2 2024-02-02T15:03:37.662 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/gfs2/gfs2.ko 2024-02-02T15:03:37.663 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/hfs 2024-02-02T15:03:37.663 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/hfs/hfs.ko 2024-02-02T15:03:37.663 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/hfsplus 2024-02-02T15:03:37.663 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/hfsplus/hfsplus.ko 2024-02-02T15:03:37.663 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/isofs 2024-02-02T15:03:37.663 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/isofs/isofs.ko 2024-02-02T15:03:37.663 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/jfs 2024-02-02T15:03:37.663 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/jfs/jfs.ko 2024-02-02T15:03:37.663 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/lockd 2024-02-02T15:03:37.663 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/lockd/lockd.ko 2024-02-02T15:03:37.663 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/minix 2024-02-02T15:03:37.664 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/minix/minix.ko 2024-02-02T15:03:37.664 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/netfs 2024-02-02T15:03:37.664 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/netfs/netfs.ko 2024-02-02T15:03:37.664 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nfs 2024-02-02T15:03:37.664 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nfs/blocklayout 2024-02-02T15:03:37.664 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-02-02T15:03:37.664 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nfs/filelayout 2024-02-02T15:03:37.664 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-02-02T15:03:37.664 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nfs/flexfilelayout 2024-02-02T15:03:37.664 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-02-02T15:03:37.664 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nfs/nfs.ko 2024-02-02T15:03:37.664 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nfs/nfsv3.ko 2024-02-02T15:03:37.665 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nfs/nfsv4.ko 2024-02-02T15:03:37.665 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nfs_common 2024-02-02T15:03:37.665 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nfs_common/grace.ko 2024-02-02T15:03:37.665 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nfs_common/nfs_acl.ko 2024-02-02T15:03:37.665 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nfsd 2024-02-02T15:03:37.665 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nfsd/nfsd.ko 2024-02-02T15:03:37.665 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nilfs2 2024-02-02T15:03:37.665 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nilfs2/nilfs2.ko 2024-02-02T15:03:37.665 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls 2024-02-02T15:03:37.665 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/mac-celtic.ko 2024-02-02T15:03:37.665 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/mac-centeuro.ko 2024-02-02T15:03:37.665 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/mac-croatian.ko 2024-02-02T15:03:37.665 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/mac-cyrillic.ko 2024-02-02T15:03:37.665 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/mac-gaelic.ko 2024-02-02T15:03:37.665 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/mac-greek.ko 2024-02-02T15:03:37.665 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/mac-iceland.ko 2024-02-02T15:03:37.666 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/mac-inuit.ko 2024-02-02T15:03:37.666 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/mac-roman.ko 2024-02-02T15:03:37.666 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/mac-romanian.ko 2024-02-02T15:03:37.666 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/mac-turkish.ko 2024-02-02T15:03:37.666 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp1250.ko 2024-02-02T15:03:37.666 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp1251.ko 2024-02-02T15:03:37.666 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp1255.ko 2024-02-02T15:03:37.666 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp737.ko 2024-02-02T15:03:37.666 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp775.ko 2024-02-02T15:03:37.666 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp850.ko 2024-02-02T15:03:37.666 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp852.ko 2024-02-02T15:03:37.666 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp855.ko 2024-02-02T15:03:37.666 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp857.ko 2024-02-02T15:03:37.667 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp860.ko 2024-02-02T15:03:37.667 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp861.ko 2024-02-02T15:03:37.667 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp862.ko 2024-02-02T15:03:37.667 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp863.ko 2024-02-02T15:03:37.667 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp864.ko 2024-02-02T15:03:37.667 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp865.ko 2024-02-02T15:03:37.667 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp866.ko 2024-02-02T15:03:37.668 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp869.ko 2024-02-02T15:03:37.668 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp874.ko 2024-02-02T15:03:37.668 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp932.ko 2024-02-02T15:03:37.668 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp936.ko 2024-02-02T15:03:37.668 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp949.ko 2024-02-02T15:03:37.668 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_cp950.ko 2024-02-02T15:03:37.668 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_euc-jp.ko 2024-02-02T15:03:37.668 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_iso8859-1.ko 2024-02-02T15:03:37.668 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_iso8859-13.ko 2024-02-02T15:03:37.668 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_iso8859-14.ko 2024-02-02T15:03:37.668 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_iso8859-15.ko 2024-02-02T15:03:37.668 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_iso8859-2.ko 2024-02-02T15:03:37.668 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_iso8859-3.ko 2024-02-02T15:03:37.669 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_iso8859-4.ko 2024-02-02T15:03:37.669 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_iso8859-5.ko 2024-02-02T15:03:37.669 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_iso8859-6.ko 2024-02-02T15:03:37.669 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_iso8859-7.ko 2024-02-02T15:03:37.669 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_iso8859-9.ko 2024-02-02T15:03:37.669 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_koi8-r.ko 2024-02-02T15:03:37.670 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_koi8-ru.ko 2024-02-02T15:03:37.670 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_koi8-u.ko 2024-02-02T15:03:37.670 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_ucs2_utils.ko 2024-02-02T15:03:37.670 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/nls/nls_utf8.ko 2024-02-02T15:03:37.670 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ocfs2 2024-02-02T15:03:37.670 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ocfs2/cluster 2024-02-02T15:03:37.671 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-02-02T15:03:37.671 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ocfs2/dlm 2024-02-02T15:03:37.671 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-02-02T15:03:37.671 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ocfs2/dlmfs 2024-02-02T15:03:37.671 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-02-02T15:03:37.671 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ocfs2/ocfs2.ko 2024-02-02T15:03:37.671 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-02-02T15:03:37.672 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-02-02T15:03:37.672 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-02-02T15:03:37.672 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/overlayfs 2024-02-02T15:03:37.672 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/overlayfs/overlay.ko 2024-02-02T15:03:37.672 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/pstore 2024-02-02T15:03:37.672 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/pstore/ramoops.ko 2024-02-02T15:03:37.672 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/reiserfs 2024-02-02T15:03:37.672 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/reiserfs/reiserfs.ko 2024-02-02T15:03:37.672 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/romfs 2024-02-02T15:03:37.672 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/romfs/romfs.ko 2024-02-02T15:03:37.672 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/smb 2024-02-02T15:03:37.673 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/smb/client 2024-02-02T15:03:37.673 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/smb/client/cifs.ko 2024-02-02T15:03:37.673 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/smb/common 2024-02-02T15:03:37.673 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/smb/common/cifs_arc4.ko 2024-02-02T15:03:37.673 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/smb/common/cifs_md4.ko 2024-02-02T15:03:37.673 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/squashfs 2024-02-02T15:03:37.673 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/squashfs/squashfs.ko 2024-02-02T15:03:37.674 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/sysv 2024-02-02T15:03:37.674 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/sysv/sysv.ko 2024-02-02T15:03:37.674 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/udf 2024-02-02T15:03:37.674 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/udf/udf.ko 2024-02-02T15:03:37.674 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ufs 2024-02-02T15:03:37.674 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/ufs/ufs.ko 2024-02-02T15:03:37.674 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/xfs 2024-02-02T15:03:37.674 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/fs/xfs/xfs.ko 2024-02-02T15:03:37.674 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/kernel 2024-02-02T15:03:37.674 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/kernel/rcu 2024-02-02T15:03:37.674 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/kernel/rcu/rcutorture.ko 2024-02-02T15:03:37.674 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/kernel/torture.ko 2024-02-02T15:03:37.674 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/kernel/trace 2024-02-02T15:03:37.674 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-02-02T15:03:37.674 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib 2024-02-02T15:03:37.675 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/asn1_encoder.ko 2024-02-02T15:03:37.675 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/crc-itu-t.ko 2024-02-02T15:03:37.675 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/crc8.ko 2024-02-02T15:03:37.675 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/crypto 2024-02-02T15:03:37.675 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/crypto/libarc4.ko 2024-02-02T15:03:37.675 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/crypto/libdes.ko 2024-02-02T15:03:37.675 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/libcrc32c.ko 2024-02-02T15:03:37.675 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/lru_cache.ko 2024-02-02T15:03:37.675 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/lz4 2024-02-02T15:03:37.675 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/lz4/lz4_compress.ko 2024-02-02T15:03:37.675 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/lz4/lz4hc_compress.ko 2024-02-02T15:03:37.675 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/math 2024-02-02T15:03:37.675 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/math/cordic.ko 2024-02-02T15:03:37.675 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/raid6 2024-02-02T15:03:37.675 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/raid6/raid6_pq.ko 2024-02-02T15:03:37.676 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/reed_solomon 2024-02-02T15:03:37.676 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/reed_solomon/reed_solomon.ko 2024-02-02T15:03:37.676 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/ts_bm.ko 2024-02-02T15:03:37.676 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/ts_fsm.ko 2024-02-02T15:03:37.676 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/ts_kmp.ko 2024-02-02T15:03:37.676 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/zstd 2024-02-02T15:03:37.676 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/lib/zstd/zstd_compress.ko 2024-02-02T15:03:37.676 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/mm 2024-02-02T15:03:37.676 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/mm/hwpoison-inject.ko 2024-02-02T15:03:37.676 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net 2024-02-02T15:03:37.676 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/6lowpan 2024-02-02T15:03:37.676 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/6lowpan/6lowpan.ko 2024-02-02T15:03:37.676 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/6lowpan/nhc_dest.ko 2024-02-02T15:03:37.676 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/6lowpan/nhc_fragment.ko 2024-02-02T15:03:37.676 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/6lowpan/nhc_hop.ko 2024-02-02T15:03:37.677 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/6lowpan/nhc_ipv6.ko 2024-02-02T15:03:37.677 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/6lowpan/nhc_mobility.ko 2024-02-02T15:03:37.677 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/6lowpan/nhc_routing.ko 2024-02-02T15:03:37.677 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/6lowpan/nhc_udp.ko 2024-02-02T15:03:37.677 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/802 2024-02-02T15:03:37.677 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/802/garp.ko 2024-02-02T15:03:37.677 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/802/mrp.ko 2024-02-02T15:03:37.677 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/802/p8022.ko 2024-02-02T15:03:37.677 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/802/psnap.ko 2024-02-02T15:03:37.677 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/802/stp.ko 2024-02-02T15:03:37.677 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/8021q 2024-02-02T15:03:37.677 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/8021q/8021q.ko 2024-02-02T15:03:37.677 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/9p 2024-02-02T15:03:37.677 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/9p/9pnet.ko 2024-02-02T15:03:37.677 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/9p/9pnet_fd.ko 2024-02-02T15:03:37.677 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/9p/9pnet_rdma.ko 2024-02-02T15:03:37.678 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/9p/9pnet_virtio.ko 2024-02-02T15:03:37.678 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/batman-adv 2024-02-02T15:03:37.678 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/batman-adv/batman-adv.ko 2024-02-02T15:03:37.678 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge 2024-02-02T15:03:37.678 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/br_netfilter.ko 2024-02-02T15:03:37.678 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/bridge.ko 2024-02-02T15:03:37.678 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter 2024-02-02T15:03:37.678 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-02-02T15:03:37.678 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_among.ko 2024-02-02T15:03:37.678 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_arp.ko 2024-02-02T15:03:37.678 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-02-02T15:03:37.678 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-02-02T15:03:37.678 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_ip.ko 2024-02-02T15:03:37.678 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-02-02T15:03:37.678 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_limit.ko 2024-02-02T15:03:37.678 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_log.ko 2024-02-02T15:03:37.678 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_mark.ko 2024-02-02T15:03:37.679 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-02-02T15:03:37.679 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-02-02T15:03:37.679 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-02-02T15:03:37.679 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-02-02T15:03:37.679 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_snat.ko 2024-02-02T15:03:37.679 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_stp.ko 2024-02-02T15:03:37.679 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-02-02T15:03:37.679 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-02-02T15:03:37.679 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-02-02T15:03:37.679 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-02-02T15:03:37.679 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/ebtables.ko 2024-02-02T15:03:37.679 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-02-02T15:03:37.679 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-02-02T15:03:37.679 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-02-02T15:03:37.679 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ceph 2024-02-02T15:03:37.679 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ceph/libceph.ko 2024-02-02T15:03:37.679 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/core 2024-02-02T15:03:37.680 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/core/failover.ko 2024-02-02T15:03:37.680 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/core/pktgen.ko 2024-02-02T15:03:37.680 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/dccp 2024-02-02T15:03:37.680 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/dccp/dccp.ko 2024-02-02T15:03:37.680 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/dccp/dccp_diag.ko 2024-02-02T15:03:37.680 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/dccp/dccp_ipv4.ko 2024-02-02T15:03:37.680 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/dccp/dccp_ipv6.ko 2024-02-02T15:03:37.680 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/dns_resolver 2024-02-02T15:03:37.680 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/dns_resolver/dns_resolver.ko 2024-02-02T15:03:37.680 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/dsa 2024-02-02T15:03:37.680 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/dsa/dsa_core.ko 2024-02-02T15:03:37.680 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/dsa/tag_brcm.ko 2024-02-02T15:03:37.680 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/dsa/tag_dsa.ko 2024-02-02T15:03:37.680 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/dsa/tag_none.ko 2024-02-02T15:03:37.681 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/dsa/tag_trailer.ko 2024-02-02T15:03:37.681 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ieee802154 2024-02-02T15:03:37.681 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ieee802154/6lowpan 2024-02-02T15:03:37.681 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-02-02T15:03:37.681 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ieee802154/ieee802154.ko 2024-02-02T15:03:37.681 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ieee802154/ieee802154_socket.ko 2024-02-02T15:03:37.681 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4 2024-02-02T15:03:37.681 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/ah4.ko 2024-02-02T15:03:37.681 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/esp4.ko 2024-02-02T15:03:37.681 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/fou.ko 2024-02-02T15:03:37.681 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/gre.ko 2024-02-02T15:03:37.681 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/inet_diag.ko 2024-02-02T15:03:37.681 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/ip_gre.ko 2024-02-02T15:03:37.681 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/ip_tunnel.ko 2024-02-02T15:03:37.682 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/ip_vti.ko 2024-02-02T15:03:37.682 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/ipcomp.ko 2024-02-02T15:03:37.682 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/ipip.ko 2024-02-02T15:03:37.682 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter 2024-02-02T15:03:37.682 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/arp_tables.ko 2024-02-02T15:03:37.682 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-02-02T15:03:37.682 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-02-02T15:03:37.682 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-02-02T15:03:37.682 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-02-02T15:03:37.682 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-02-02T15:03:37.682 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-02-02T15:03:37.682 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-02-02T15:03:37.682 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-02-02T15:03:37.682 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-02-02T15:03:37.682 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/iptable_security.ko 2024-02-02T15:03:37.682 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-02-02T15:03:37.683 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-02-02T15:03:37.683 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-02-02T15:03:37.683 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-02-02T15:03:37.683 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-02-02T15:03:37.683 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-02-02T15:03:37.683 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-02-02T15:03:37.683 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-02-02T15:03:37.683 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-02-02T15:03:37.683 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-02-02T15:03:37.683 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/tcp_bic.ko 2024-02-02T15:03:37.683 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/tcp_dctcp.ko 2024-02-02T15:03:37.683 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/tcp_diag.ko 2024-02-02T15:03:37.683 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/tcp_highspeed.ko 2024-02-02T15:03:37.683 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/tcp_htcp.ko 2024-02-02T15:03:37.683 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/tcp_hybla.ko 2024-02-02T15:03:37.683 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/tcp_illinois.ko 2024-02-02T15:03:37.683 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/tcp_lp.ko 2024-02-02T15:03:37.684 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/tcp_scalable.ko 2024-02-02T15:03:37.684 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/tcp_vegas.ko 2024-02-02T15:03:37.684 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/tcp_veno.ko 2024-02-02T15:03:37.684 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/tcp_westwood.ko 2024-02-02T15:03:37.684 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/tcp_yeah.ko 2024-02-02T15:03:37.684 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/tunnel4.ko 2024-02-02T15:03:37.684 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/udp_diag.ko 2024-02-02T15:03:37.684 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/udp_tunnel.ko 2024-02-02T15:03:37.684 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv4/xfrm4_tunnel.ko 2024-02-02T15:03:37.684 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6 2024-02-02T15:03:37.684 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/ah6.ko 2024-02-02T15:03:37.684 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/esp6.ko 2024-02-02T15:03:37.684 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/fou6.ko 2024-02-02T15:03:37.684 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/ip6_tunnel.ko 2024-02-02T15:03:37.684 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-02-02T15:03:37.684 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/ip6_vti.ko 2024-02-02T15:03:37.685 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/ipcomp6.ko 2024-02-02T15:03:37.685 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter 2024-02-02T15:03:37.685 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-02-02T15:03:37.685 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-02-02T15:03:37.685 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-02-02T15:03:37.685 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-02-02T15:03:37.685 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-02-02T15:03:37.685 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-02-02T15:03:37.685 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-02-02T15:03:37.685 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-02-02T15:03:37.685 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-02-02T15:03:37.685 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-02-02T15:03:37.685 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-02-02T15:03:37.685 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-02-02T15:03:37.685 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-02-02T15:03:37.685 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-02-02T15:03:37.686 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-02-02T15:03:37.686 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-02-02T15:03:37.686 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-02-02T15:03:37.686 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-02-02T15:03:37.686 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-02-02T15:03:37.686 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-02-02T15:03:37.686 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-02-02T15:03:37.686 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-02-02T15:03:37.686 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-02-02T15:03:37.686 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-02-02T15:03:37.686 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/sit.ko 2024-02-02T15:03:37.686 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/tunnel6.ko 2024-02-02T15:03:37.686 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/ipv6/xfrm6_tunnel.ko 2024-02-02T15:03:37.686 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/key 2024-02-02T15:03:37.686 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/key/af_key.ko 2024-02-02T15:03:37.686 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/l2tp 2024-02-02T15:03:37.686 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/l2tp/l2tp_core.ko 2024-02-02T15:03:37.687 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/l2tp/l2tp_debugfs.ko 2024-02-02T15:03:37.687 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/l2tp/l2tp_eth.ko 2024-02-02T15:03:37.687 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/l2tp/l2tp_ip.ko 2024-02-02T15:03:37.687 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/l2tp/l2tp_ip6.ko 2024-02-02T15:03:37.687 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/l2tp/l2tp_netlink.ko 2024-02-02T15:03:37.687 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/l2tp/l2tp_ppp.ko 2024-02-02T15:03:37.687 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/llc 2024-02-02T15:03:37.687 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/llc/llc.ko 2024-02-02T15:03:37.687 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/mac802154 2024-02-02T15:03:37.687 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/mac802154/mac802154.ko 2024-02-02T15:03:37.687 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/mpls 2024-02-02T15:03:37.687 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/mpls/mpls_gso.ko 2024-02-02T15:03:37.687 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter 2024-02-02T15:03:37.687 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset 2024-02-02T15:03:37.687 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset/ip_set.ko 2024-02-02T15:03:37.688 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-02-02T15:03:37.688 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-02-02T15:03:37.688 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-02-02T15:03:37.688 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-02-02T15:03:37.688 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-02-02T15:03:37.688 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-02-02T15:03:37.688 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-02-02T15:03:37.688 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-02-02T15:03:37.688 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-02-02T15:03:37.688 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-02-02T15:03:37.688 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-02-02T15:03:37.688 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-02-02T15:03:37.688 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-02-02T15:03:37.689 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-02-02T15:03:37.689 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-02-02T15:03:37.689 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipvs 2024-02-02T15:03:37.689 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipvs/ip_vs.ko 2024-02-02T15:03:37.689 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-02-02T15:03:37.689 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-02-02T15:03:37.689 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-02-02T15:03:37.689 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-02-02T15:03:37.689 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-02-02T15:03:37.689 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-02-02T15:03:37.689 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-02-02T15:03:37.689 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-02-02T15:03:37.689 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-02-02T15:03:37.689 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-02-02T15:03:37.689 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-02-02T15:03:37.689 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-02-02T15:03:37.689 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-02-02T15:03:37.690 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_conncount.ko 2024-02-02T15:03:37.690 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_conntrack.ko 2024-02-02T15:03:37.690 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-02-02T15:03:37.690 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-02-02T15:03:37.690 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-02-02T15:03:37.690 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_conntrack_h323.ko 2024-02-02T15:03:37.690 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_conntrack_irc.ko 2024-02-02T15:03:37.690 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-02-02T15:03:37.690 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-02-02T15:03:37.690 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-02-02T15:03:37.690 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_conntrack_sane.ko 2024-02-02T15:03:37.690 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_conntrack_sip.ko 2024-02-02T15:03:37.690 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-02-02T15:03:37.690 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-02-02T15:03:37.690 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_dup_netdev.ko 2024-02-02T15:03:37.691 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_flow_table.ko 2024-02-02T15:03:37.691 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_flow_table_inet.ko 2024-02-02T15:03:37.691 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_log_syslog.ko 2024-02-02T15:03:37.691 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_nat.ko 2024-02-02T15:03:37.691 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_nat_amanda.ko 2024-02-02T15:03:37.691 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_nat_ftp.ko 2024-02-02T15:03:37.691 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_nat_irc.ko 2024-02-02T15:03:37.691 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_nat_sip.ko 2024-02-02T15:03:37.691 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_nat_tftp.ko 2024-02-02T15:03:37.691 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_synproxy_core.ko 2024-02-02T15:03:37.691 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nf_tables.ko 2024-02-02T15:03:37.691 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nfnetlink.ko 2024-02-02T15:03:37.691 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nfnetlink_acct.ko 2024-02-02T15:03:37.691 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nfnetlink_log.ko 2024-02-02T15:03:37.692 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nfnetlink_osf.ko 2024-02-02T15:03:37.692 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nfnetlink_queue.ko 2024-02-02T15:03:37.692 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_chain_nat.ko 2024-02-02T15:03:37.692 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_compat.ko 2024-02-02T15:03:37.692 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_ct.ko 2024-02-02T15:03:37.692 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_dup_netdev.ko 2024-02-02T15:03:37.692 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_fib.ko 2024-02-02T15:03:37.692 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_fwd_netdev.ko 2024-02-02T15:03:37.692 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_hash.ko 2024-02-02T15:03:37.692 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_limit.ko 2024-02-02T15:03:37.692 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_log.ko 2024-02-02T15:03:37.692 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_masq.ko 2024-02-02T15:03:37.692 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_nat.ko 2024-02-02T15:03:37.692 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_numgen.ko 2024-02-02T15:03:37.693 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_queue.ko 2024-02-02T15:03:37.693 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_quota.ko 2024-02-02T15:03:37.693 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_redir.ko 2024-02-02T15:03:37.693 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_reject.ko 2024-02-02T15:03:37.693 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_reject_inet.ko 2024-02-02T15:03:37.693 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_socket.ko 2024-02-02T15:03:37.693 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/nft_tunnel.ko 2024-02-02T15:03:37.693 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_AUDIT.ko 2024-02-02T15:03:37.693 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_CHECKSUM.ko 2024-02-02T15:03:37.693 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_CLASSIFY.ko 2024-02-02T15:03:37.693 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-02-02T15:03:37.693 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_CT.ko 2024-02-02T15:03:37.693 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_DSCP.ko 2024-02-02T15:03:37.693 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_HL.ko 2024-02-02T15:03:37.693 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_HMARK.ko 2024-02-02T15:03:37.694 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_IDLETIMER.ko 2024-02-02T15:03:37.694 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_LED.ko 2024-02-02T15:03:37.694 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_LOG.ko 2024-02-02T15:03:37.694 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_MASQUERADE.ko 2024-02-02T15:03:37.694 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_NETMAP.ko 2024-02-02T15:03:37.694 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_NFLOG.ko 2024-02-02T15:03:37.694 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_NFQUEUE.ko 2024-02-02T15:03:37.694 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_RATEEST.ko 2024-02-02T15:03:37.694 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_REDIRECT.ko 2024-02-02T15:03:37.694 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_SECMARK.ko 2024-02-02T15:03:37.694 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_TCPMSS.ko 2024-02-02T15:03:37.694 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-02-02T15:03:37.694 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_TEE.ko 2024-02-02T15:03:37.694 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_TPROXY.ko 2024-02-02T15:03:37.694 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_TRACE.ko 2024-02-02T15:03:37.694 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_addrtype.ko 2024-02-02T15:03:37.695 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_bpf.ko 2024-02-02T15:03:37.695 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_cgroup.ko 2024-02-02T15:03:37.695 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_cluster.ko 2024-02-02T15:03:37.695 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_comment.ko 2024-02-02T15:03:37.695 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_connbytes.ko 2024-02-02T15:03:37.695 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_connlabel.ko 2024-02-02T15:03:37.695 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_connlimit.ko 2024-02-02T15:03:37.695 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_connmark.ko 2024-02-02T15:03:37.695 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_conntrack.ko 2024-02-02T15:03:37.695 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_cpu.ko 2024-02-02T15:03:37.695 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_dccp.ko 2024-02-02T15:03:37.695 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_devgroup.ko 2024-02-02T15:03:37.695 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_dscp.ko 2024-02-02T15:03:37.695 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_ecn.ko 2024-02-02T15:03:37.695 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_esp.ko 2024-02-02T15:03:37.695 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_hashlimit.ko 2024-02-02T15:03:37.695 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_helper.ko 2024-02-02T15:03:37.696 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_hl.ko 2024-02-02T15:03:37.696 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_ipcomp.ko 2024-02-02T15:03:37.696 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_iprange.ko 2024-02-02T15:03:37.696 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_ipvs.ko 2024-02-02T15:03:37.696 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_l2tp.ko 2024-02-02T15:03:37.696 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_length.ko 2024-02-02T15:03:37.696 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_limit.ko 2024-02-02T15:03:37.696 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_mac.ko 2024-02-02T15:03:37.696 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_mark.ko 2024-02-02T15:03:37.696 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_multiport.ko 2024-02-02T15:03:37.696 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_nat.ko 2024-02-02T15:03:37.696 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_nfacct.ko 2024-02-02T15:03:37.696 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_osf.ko 2024-02-02T15:03:37.696 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_owner.ko 2024-02-02T15:03:37.697 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_physdev.ko 2024-02-02T15:03:37.697 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_pkttype.ko 2024-02-02T15:03:37.697 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_policy.ko 2024-02-02T15:03:37.697 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_quota.ko 2024-02-02T15:03:37.697 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_rateest.ko 2024-02-02T15:03:37.697 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_realm.ko 2024-02-02T15:03:37.697 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_recent.ko 2024-02-02T15:03:37.697 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_sctp.ko 2024-02-02T15:03:37.697 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_set.ko 2024-02-02T15:03:37.697 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_socket.ko 2024-02-02T15:03:37.697 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_state.ko 2024-02-02T15:03:37.697 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_statistic.ko 2024-02-02T15:03:37.697 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_string.ko 2024-02-02T15:03:37.697 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_tcpmss.ko 2024-02-02T15:03:37.697 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_time.ko 2024-02-02T15:03:37.697 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netfilter/xt_u32.ko 2024-02-02T15:03:37.698 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netlink 2024-02-02T15:03:37.698 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/netlink/netlink_diag.ko 2024-02-02T15:03:37.698 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/nsh 2024-02-02T15:03:37.698 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/nsh/nsh.ko 2024-02-02T15:03:37.698 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/openvswitch 2024-02-02T15:03:37.698 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/openvswitch/openvswitch.ko 2024-02-02T15:03:37.698 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/openvswitch/vport-geneve.ko 2024-02-02T15:03:37.698 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/openvswitch/vport-gre.ko 2024-02-02T15:03:37.698 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/openvswitch/vport-vxlan.ko 2024-02-02T15:03:37.698 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/packet 2024-02-02T15:03:37.698 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/packet/af_packet_diag.ko 2024-02-02T15:03:37.698 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/rds 2024-02-02T15:03:37.698 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/rds/rds.ko 2024-02-02T15:03:37.698 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/rds/rds_rdma.ko 2024-02-02T15:03:37.698 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/rds/rds_tcp.ko 2024-02-02T15:03:37.699 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched 2024-02-02T15:03:37.699 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/act_csum.ko 2024-02-02T15:03:37.699 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/act_gact.ko 2024-02-02T15:03:37.699 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/act_ipt.ko 2024-02-02T15:03:37.699 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/act_mirred.ko 2024-02-02T15:03:37.699 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/act_nat.ko 2024-02-02T15:03:37.699 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/act_pedit.ko 2024-02-02T15:03:37.699 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/act_police.ko 2024-02-02T15:03:37.699 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/act_simple.ko 2024-02-02T15:03:37.699 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/act_skbedit.ko 2024-02-02T15:03:37.699 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/act_vlan.ko 2024-02-02T15:03:37.699 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/cls_basic.ko 2024-02-02T15:03:37.699 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/cls_bpf.ko 2024-02-02T15:03:37.699 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/cls_flow.ko 2024-02-02T15:03:37.699 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/cls_fw.ko 2024-02-02T15:03:37.699 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/cls_route.ko 2024-02-02T15:03:37.700 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/cls_u32.ko 2024-02-02T15:03:37.700 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/em_cmp.ko 2024-02-02T15:03:37.700 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/em_ipset.ko 2024-02-02T15:03:37.700 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/em_meta.ko 2024-02-02T15:03:37.700 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/em_nbyte.ko 2024-02-02T15:03:37.700 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/em_text.ko 2024-02-02T15:03:37.700 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/em_u32.ko 2024-02-02T15:03:37.700 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_choke.ko 2024-02-02T15:03:37.700 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_codel.ko 2024-02-02T15:03:37.700 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_drr.ko 2024-02-02T15:03:37.700 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_fq.ko 2024-02-02T15:03:37.700 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_gred.ko 2024-02-02T15:03:37.700 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_hfsc.ko 2024-02-02T15:03:37.701 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_hhf.ko 2024-02-02T15:03:37.701 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_htb.ko 2024-02-02T15:03:37.701 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_ingress.ko 2024-02-02T15:03:37.701 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_mqprio.ko 2024-02-02T15:03:37.701 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_mqprio_lib.ko 2024-02-02T15:03:37.701 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_multiq.ko 2024-02-02T15:03:37.701 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_netem.ko 2024-02-02T15:03:37.701 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_pie.ko 2024-02-02T15:03:37.701 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_plug.ko 2024-02-02T15:03:37.701 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_prio.ko 2024-02-02T15:03:37.701 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_qfq.ko 2024-02-02T15:03:37.701 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_red.ko 2024-02-02T15:03:37.701 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_sfb.ko 2024-02-02T15:03:37.701 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_sfq.ko 2024-02-02T15:03:37.701 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_tbf.ko 2024-02-02T15:03:37.702 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sched/sch_teql.ko 2024-02-02T15:03:37.702 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sctp 2024-02-02T15:03:37.702 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sctp/sctp.ko 2024-02-02T15:03:37.702 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sctp/sctp_diag.ko 2024-02-02T15:03:37.702 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sunrpc 2024-02-02T15:03:37.702 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sunrpc/auth_gss 2024-02-02T15:03:37.702 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-02-02T15:03:37.702 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-02-02T15:03:37.703 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sunrpc/sunrpc.ko 2024-02-02T15:03:37.703 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sunrpc/xprtrdma 2024-02-02T15:03:37.703 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-02-02T15:03:37.703 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/tipc 2024-02-02T15:03:37.703 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/tipc/diag.ko 2024-02-02T15:03:37.703 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/tipc/tipc.ko 2024-02-02T15:03:37.703 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/unix 2024-02-02T15:03:37.703 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/unix/unix_diag.ko 2024-02-02T15:03:37.703 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/vmw_vsock 2024-02-02T15:03:37.703 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-02-02T15:03:37.703 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-02-02T15:03:37.703 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/vmw_vsock/vsock.ko 2024-02-02T15:03:37.703 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/vmw_vsock/vsock_diag.ko 2024-02-02T15:03:37.703 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/vmw_vsock/vsock_loopback.ko 2024-02-02T15:03:37.703 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/xfrm 2024-02-02T15:03:37.703 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/net/xfrm/xfrm_ipcomp.ko 2024-02-02T15:03:37.704 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/security 2024-02-02T15:03:37.704 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/security/keys 2024-02-02T15:03:37.704 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/security/keys/encrypted-keys 2024-02-02T15:03:37.704 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-02-02T15:03:37.704 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/security/keys/trusted-keys 2024-02-02T15:03:37.704 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/security/keys/trusted-keys/trusted.ko 2024-02-02T15:03:37.704 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/virt 2024-02-02T15:03:37.704 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/virt/lib 2024-02-02T15:03:37.704 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/kernel/virt/lib/irqbypass.ko 2024-02-02T15:03:37.704 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/modules.alias 2024-02-02T15:03:37.704 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/modules.alias.bin 2024-02-02T15:03:37.704 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/modules.builtin 2024-02-02T15:03:37.705 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/modules.builtin.bin 2024-02-02T15:03:37.705 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/modules.builtin.modinfo 2024-02-02T15:03:37.705 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/modules.dep 2024-02-02T15:03:37.705 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/modules.dep.bin 2024-02-02T15:03:37.705 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/modules.devname 2024-02-02T15:03:37.705 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/modules.order 2024-02-02T15:03:37.706 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/modules.softdep 2024-02-02T15:03:37.706 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/modules.symbols 2024-02-02T15:03:37.706 INFO:teuthology.orchestra.run.smithi001.stdout:/lib/modules/6.7.0-gcd42140e6285/modules.symbols.bin 2024-02-02T15:03:37.707 DEBUG:teuthology.task.kernel:get_image_version: 6.7.0-gcd42140e6285 2024-02-02T15:03:37.707 DEBUG:teuthology.orchestra.run.smithi001:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-02-02T15:03:37.736 INFO:teuthology.orchestra.run.smithi001.stdout:postinstall scriptlet (using /bin/sh): 2024-02-02T15:03:37.736 INFO:teuthology.orchestra.run.smithi001.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.7.0-gcd42140e6285 -a -r /boot/System.map-6.7.0-gcd42140e6285 ]; then 2024-02-02T15:03:37.737 INFO:teuthology.orchestra.run.smithi001.stdout:cp /boot/vmlinuz-6.7.0-gcd42140e6285 /boot/.vmlinuz-6.7.0-gcd42140e6285-rpm 2024-02-02T15:03:37.737 INFO:teuthology.orchestra.run.smithi001.stdout:cp /boot/System.map-6.7.0-gcd42140e6285 /boot/.System.map-6.7.0-gcd42140e6285-rpm 2024-02-02T15:03:37.737 INFO:teuthology.orchestra.run.smithi001.stdout:rm -f /boot/vmlinuz-6.7.0-gcd42140e6285 /boot/System.map-6.7.0-gcd42140e6285 2024-02-02T15:03:37.737 INFO:teuthology.orchestra.run.smithi001.stdout:/sbin/installkernel 6.7.0-gcd42140e6285 /boot/.vmlinuz-6.7.0-gcd42140e6285-rpm /boot/.System.map-6.7.0-gcd42140e6285-rpm 2024-02-02T15:03:37.737 INFO:teuthology.orchestra.run.smithi001.stdout:rm -f /boot/.vmlinuz-6.7.0-gcd42140e6285-rpm /boot/.System.map-6.7.0-gcd42140e6285-rpm 2024-02-02T15:03:37.737 INFO:teuthology.orchestra.run.smithi001.stdout:fi 2024-02-02T15:03:37.737 INFO:teuthology.orchestra.run.smithi001.stdout:preuninstall scriptlet (using /bin/sh): 2024-02-02T15:03:37.737 INFO:teuthology.orchestra.run.smithi001.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2024-02-02T15:03:37.737 INFO:teuthology.orchestra.run.smithi001.stdout:new-kernel-pkg --remove 6.7.0-gcd42140e6285 --rminitrd --initrdfile=/boot/initramfs-6.7.0-gcd42140e6285.img 2024-02-02T15:03:37.737 INFO:teuthology.orchestra.run.smithi001.stdout:elif [ -x /usr/bin/kernel-install ]; then 2024-02-02T15:03:37.737 INFO:teuthology.orchestra.run.smithi001.stdout:kernel-install remove 6.7.0-gcd42140e6285 2024-02-02T15:03:37.737 INFO:teuthology.orchestra.run.smithi001.stdout:fi 2024-02-02T15:03:37.737 INFO:teuthology.orchestra.run.smithi001.stdout:postuninstall scriptlet (using /bin/sh): 2024-02-02T15:03:37.737 INFO:teuthology.orchestra.run.smithi001.stdout:if [ -x /sbin/update-bootloader ]; then 2024-02-02T15:03:37.738 INFO:teuthology.orchestra.run.smithi001.stdout:/sbin/update-bootloader --remove 6.7.0-gcd42140e6285 2024-02-02T15:03:37.738 INFO:teuthology.orchestra.run.smithi001.stdout:fi 2024-02-02T15:03:37.739 DEBUG:teuthology.orchestra.run.smithi001:> sudo rpm -qi grub2-tools 2024-02-02T15:03:37.822 INFO:teuthology.orchestra.run.smithi001.stdout:Name : grub2-tools 2024-02-02T15:03:37.822 INFO:teuthology.orchestra.run.smithi001.stdout:Epoch : 1 2024-02-02T15:03:37.822 INFO:teuthology.orchestra.run.smithi001.stdout:Version : 2.02 2024-02-02T15:03:37.822 INFO:teuthology.orchestra.run.smithi001.stdout:Release : 123.el8 2024-02-02T15:03:37.822 INFO:teuthology.orchestra.run.smithi001.stdout:Architecture: x86_64 2024-02-02T15:03:37.822 INFO:teuthology.orchestra.run.smithi001.stdout:Install Date: Thu 14 Dec 2023 12:39:42 AM UTC 2024-02-02T15:03:37.822 INFO:teuthology.orchestra.run.smithi001.stdout:Group : System Environment/Base 2024-02-02T15:03:37.822 INFO:teuthology.orchestra.run.smithi001.stdout:Size : 9265450 2024-02-02T15:03:37.822 INFO:teuthology.orchestra.run.smithi001.stdout:License : GPLv3+ 2024-02-02T15:03:37.822 INFO:teuthology.orchestra.run.smithi001.stdout:Signature : RSA/SHA256, Thu 31 Mar 2022 07:24:35 PM UTC, Key ID 199e2f91fd431d51 2024-02-02T15:03:37.822 INFO:teuthology.orchestra.run.smithi001.stdout:Source RPM : grub2-2.02-123.el8.src.rpm 2024-02-02T15:03:37.822 INFO:teuthology.orchestra.run.smithi001.stdout:Build Date : Mon 28 Mar 2022 02:12:09 PM UTC 2024-02-02T15:03:37.822 INFO:teuthology.orchestra.run.smithi001.stdout:Build Host : x86-vm-09.build.eng.bos.redhat.com 2024-02-02T15:03:37.822 INFO:teuthology.orchestra.run.smithi001.stdout:Relocations : (not relocatable) 2024-02-02T15:03:37.823 INFO:teuthology.orchestra.run.smithi001.stdout:Packager : Red Hat, Inc. 2024-02-02T15:03:37.823 INFO:teuthology.orchestra.run.smithi001.stdout:Vendor : Red Hat, Inc. 2024-02-02T15:03:37.823 INFO:teuthology.orchestra.run.smithi001.stdout:URL : http://www.gnu.org/software/grub/ 2024-02-02T15:03:37.823 INFO:teuthology.orchestra.run.smithi001.stdout:Summary : Support tools for GRUB. 2024-02-02T15:03:37.823 INFO:teuthology.orchestra.run.smithi001.stdout:Description : 2024-02-02T15:03:37.823 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:03:37.823 INFO:teuthology.orchestra.run.smithi001.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-02-02T15:03:37.823 INFO:teuthology.orchestra.run.smithi001.stdout:customizable bootloader with modular architecture. It supports a rich 2024-02-02T15:03:37.823 INFO:teuthology.orchestra.run.smithi001.stdout:variety of kernel formats, file systems, computer architectures and 2024-02-02T15:03:37.823 INFO:teuthology.orchestra.run.smithi001.stdout:hardware devices. 2024-02-02T15:03:37.823 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:03:37.823 INFO:teuthology.orchestra.run.smithi001.stdout:This subpackage provides tools for support of all platforms. 2024-02-02T15:03:37.824 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-02-02T15:03:37.824 INFO:teuthology.task.kernel:Updating grub on smithi001 to boot 6.7.0-gcd42140e6285 2024-02-02T15:03:37.824 DEBUG:teuthology.orchestra.run.smithi001:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-02-02T15:03:38.599 INFO:teuthology.orchestra.run.smithi001.stderr:Generating grub configuration file ... 2024-02-02T15:03:41.039 INFO:teuthology.orchestra.run.smithi001.stderr:done 2024-02-02T15:03:41.041 DEBUG:teuthology.orchestra.run.smithi001:> mktemp 2024-02-02T15:03:41.061 INFO:teuthology.orchestra.run.smithi001.stdout:/tmp/tmp.OnJTmBmAf6 2024-02-02T15:03:41.061 DEBUG:teuthology.orchestra.run.smithi001:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.OnJTmBmAf6 2024-02-02T15:03:41.140 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod 0666 /tmp/tmp.OnJTmBmAf6 2024-02-02T15:03:41.303 DEBUG:teuthology.orchestra.remote:smithi001:/tmp/tmp.OnJTmBmAf6 is 6KB 2024-02-02T15:03:41.329 DEBUG:teuthology.orchestra.run.smithi001:> rm -fr /tmp/tmp.OnJTmBmAf6 2024-02-02T15:03:41.350 DEBUG:teuthology.orchestra.run.smithi001:> sudo /bin/ls /boot/loader/entries || true 2024-02-02T15:03:41.431 INFO:teuthology.orchestra.run.smithi001.stdout:2db68f91c8f90721a6596efe657a5324-0-rescue.conf 2024-02-02T15:03:41.431 INFO:teuthology.orchestra.run.smithi001.stdout:2db68f91c8f90721a6596efe657a5324-6.7.0-gcd42140e6285.conf 2024-02-02T15:03:41.431 INFO:teuthology.orchestra.run.smithi001.stdout:963f8b6c99184de696879b3f7da9ecce-0-rescue.conf 2024-02-02T15:03:41.431 INFO:teuthology.orchestra.run.smithi001.stdout:963f8b6c99184de696879b3f7da9ecce-4.18.0-372.9.1.el8.x86_64.conf 2024-02-02T15:03:41.431 INFO:teuthology.orchestra.run.smithi001.stdout:963f8b6c99184de696879b3f7da9ecce-4.18.0-513.9.1.el8_9.x86_64.conf 2024-02-02T15:03:41.432 DEBUG:teuthology.orchestra.run.smithi001:> sudo grub2-set-default 2db68f91c8f90721a6596efe657a5324-6.7.0-gcd42140e6285 2024-02-02T15:03:41.621 DEBUG:teuthology.orchestra.run.smithi001:> sudo shutdown -r now 2024-02-02T15:04:11.624 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-02-02T15:04:11.624 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi001.front.sepia.ceph.com' 2024-02-02T15:04:11.625 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi001.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-02T15:05:11.628 DEBUG:teuthology.orchestra.remote:timed out 2024-02-02T15:05:20.629 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi001.front.sepia.ceph.com' 2024-02-02T15:05:20.630 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi001.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-02T15:05:21.996 DEBUG:teuthology.orchestra.run.smithi001:> true 2024-02-02T15:05:22.510 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi001.front.sepia.ceph.com' 2024-02-02T15:05:22.511 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.7.0-gcd42140e6285"... 2024-02-02T15:05:22.511 DEBUG:teuthology.orchestra.run.smithi001:> uname -r 2024-02-02T15:05:22.550 INFO:teuthology.orchestra.run.smithi001.stdout:6.7.0-gcd42140e6285 2024-02-02T15:05:22.550 DEBUG:teuthology.task.kernel:current kernel version is 6.7.0-gcd42140e6285 vs 6.7.0-gcd42140e6285 2024-02-02T15:05:22.550 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-02T15:05:22.550 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2024-02-02T15:05:23.553 DEBUG:teuthology.parallel:result is None 2024-02-02T15:05:23.553 INFO:teuthology.run_tasks:Running task internal.base... 2024-02-02T15:05:23.562 INFO:teuthology.task.internal:Creating test directory... 2024-02-02T15:05:23.562 DEBUG:teuthology.orchestra.run.smithi001:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-02T15:05:23.567 DEBUG:teuthology.orchestra.run.smithi071:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-02T15:05:23.570 DEBUG:teuthology.orchestra.run.smithi142:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-02T15:05:23.605 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-02-02T15:05:23.611 INFO:teuthology.run_tasks:Running task internal.archive... 2024-02-02T15:05:23.638 INFO:teuthology.task.internal:Creating archive directory... 2024-02-02T15:05:23.638 DEBUG:teuthology.orchestra.run.smithi001:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-02T15:05:23.649 DEBUG:teuthology.orchestra.run.smithi071:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-02T15:05:23.651 DEBUG:teuthology.orchestra.run.smithi142:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-02T15:05:23.703 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-02-02T15:05:23.710 INFO:teuthology.task.internal:Enabling coredump saving... 2024-02-02T15:05:23.710 DEBUG:teuthology.orchestra.run.smithi071:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-02T15:05:23.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-02T15:05:23.735 DEBUG:teuthology.orchestra.run.smithi142:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-02T15:05:23.753 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-02T15:05:23.754 DEBUG:teuthology.orchestra.run.smithi001:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-02T15:05:23.758 DEBUG:teuthology.orchestra.run.smithi071:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-02T15:05:23.778 DEBUG:teuthology.orchestra.run.smithi142:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-02T15:05:23.823 INFO:teuthology.orchestra.run.smithi071.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-02T15:05:23.841 INFO:teuthology.orchestra.run.smithi071.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-02T15:05:23.850 INFO:teuthology.orchestra.run.smithi142.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-02T15:05:23.869 INFO:teuthology.orchestra.run.smithi142.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-02T15:05:23.958 INFO:teuthology.orchestra.run.smithi001.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-02T15:05:23.993 INFO:teuthology.orchestra.run.smithi001.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-02T15:05:23.997 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-02-02T15:05:24.005 INFO:teuthology.task.internal:Configuring sudo... 2024-02-02T15:05:24.005 DEBUG:teuthology.orchestra.run.smithi001:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-02T15:05:24.009 DEBUG:teuthology.orchestra.run.smithi071:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-02T15:05:24.011 DEBUG:teuthology.orchestra.run.smithi142:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-02T15:05:24.070 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-02-02T15:05:24.080 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-02-02T15:05:24.080 DEBUG:teuthology.orchestra.run.smithi001:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-02T15:05:24.084 DEBUG:teuthology.orchestra.run.smithi071:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-02T15:05:24.087 DEBUG:teuthology.orchestra.run.smithi142:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-02T15:05:24.120 DEBUG:teuthology.orchestra.run.smithi001:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-02T15:05:24.231 DEBUG:teuthology.orchestra.run.smithi001:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-02T15:05:24.311 DEBUG:teuthology.orchestra.run.smithi001:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-02T15:05:24.379 DEBUG:teuthology.orchestra.run.smithi001:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-02T15:05:24.460 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-02-02T15:05:24.460 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-02T15:05:24.525 DEBUG:teuthology.orchestra.run.smithi071:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-02T15:05:24.571 DEBUG:teuthology.orchestra.run.smithi071:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-02T15:05:24.626 DEBUG:teuthology.orchestra.run.smithi071:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-02T15:05:24.712 DEBUG:teuthology.orchestra.run.smithi071:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-02T15:05:24.755 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-02-02T15:05:24.756 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-02T15:05:24.824 DEBUG:teuthology.orchestra.run.smithi142:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-02T15:05:24.870 DEBUG:teuthology.orchestra.run.smithi142:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-02T15:05:24.929 DEBUG:teuthology.orchestra.run.smithi142:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-02T15:05:25.009 DEBUG:teuthology.orchestra.run.smithi142:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-02T15:05:25.059 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-02-02T15:05:25.059 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-02T15:05:25.129 DEBUG:teuthology.orchestra.run.smithi001:> sudo service rsyslog restart 2024-02-02T15:05:25.133 DEBUG:teuthology.orchestra.run.smithi071:> sudo service rsyslog restart 2024-02-02T15:05:25.135 DEBUG:teuthology.orchestra.run.smithi142:> sudo service rsyslog restart 2024-02-02T15:05:25.216 INFO:teuthology.orchestra.run.smithi071.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-02T15:05:25.241 INFO:teuthology.orchestra.run.smithi142.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-02T15:05:25.242 INFO:teuthology.orchestra.run.smithi001.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-02T15:05:25.684 INFO:teuthology.run_tasks:Running task internal.timer... 2024-02-02T15:05:25.691 INFO:teuthology.task.internal:Starting timer... 2024-02-02T15:05:25.692 INFO:teuthology.run_tasks:Running task pcp... 2024-02-02T15:05:25.726 INFO:teuthology.run_tasks:Running task selinux... 2024-02-02T15:05:25.758 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-02-02T15:05:25.758 DEBUG:teuthology.orchestra.run.smithi001:> sudo service auditd rotate 2024-02-02T15:05:25.901 INFO:teuthology.orchestra.run.smithi001.stdout:Rotating logs: 2024-02-02T15:05:25.904 DEBUG:teuthology.orchestra.run.smithi071:> sudo service auditd rotate 2024-02-02T15:05:25.997 INFO:teuthology.orchestra.run.smithi071.stdout:Rotating logs: 2024-02-02T15:05:25.998 DEBUG:teuthology.orchestra.run.smithi142:> sudo service auditd rotate 2024-02-02T15:05:26.075 INFO:teuthology.orchestra.run.smithi142.stdout:Rotating logs: 2024-02-02T15:05:26.077 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-02-02T15:05:26.077 DEBUG:teuthology.orchestra.run.smithi001:> /usr/sbin/getenforce 2024-02-02T15:05:26.129 INFO:teuthology.orchestra.run.smithi001.stdout:Permissive 2024-02-02T15:05:26.130 DEBUG:teuthology.orchestra.run.smithi071:> /usr/sbin/getenforce 2024-02-02T15:05:26.152 INFO:teuthology.orchestra.run.smithi071.stdout:Permissive 2024-02-02T15:05:26.153 DEBUG:teuthology.orchestra.run.smithi142:> /usr/sbin/getenforce 2024-02-02T15:05:26.205 INFO:teuthology.orchestra.run.smithi142.stdout:Permissive 2024-02-02T15:05:26.205 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi001.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi071.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi142.front.sepia.ceph.com': 'permissive'} 2024-02-02T15:05:26.205 DEBUG:teuthology.orchestra.run.smithi001:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-02T15:05:26.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-02T15:05:26.271 DEBUG:teuthology.orchestra.run.smithi071:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-02T15:05:26.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-02T15:05:26.299 DEBUG:teuthology.orchestra.run.smithi142:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-02T15:05:26.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-02T15:05:26.329 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-02-02T15:05:26.329 DEBUG:teuthology.orchestra.run.smithi001:> sudo /usr/sbin/setenforce permissive 2024-02-02T15:05:26.395 DEBUG:teuthology.orchestra.run.smithi071:> sudo /usr/sbin/setenforce permissive 2024-02-02T15:05:26.431 DEBUG:teuthology.orchestra.run.smithi142:> sudo /usr/sbin/setenforce permissive 2024-02-02T15:05:26.466 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-02-02T15:05:26.477 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-02-02T15:05:26.482 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-02-02T15:05:26.656 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-02-02T15:05:26.681 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-02-02T15:05:26.682 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi001.front.sepia.ceph.com,smithi071.front.sepia.ceph.com,smithi142.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-02-02T15:14:12.852 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi001.front.sepia.ceph.com'), Remote(name='ubuntu@smithi071.front.sepia.ceph.com'), Remote(name='ubuntu@smithi142.front.sepia.ceph.com')] 2024-02-02T15:14:12.853 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi001.front.sepia.ceph.com' 2024-02-02T15:14:12.854 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi001.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-02T15:14:12.953 DEBUG:teuthology.orchestra.run.smithi001:> true 2024-02-02T15:14:13.064 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi001.front.sepia.ceph.com' 2024-02-02T15:14:13.064 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-02-02T15:14:13.065 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-02T15:14:13.140 DEBUG:teuthology.orchestra.run.smithi071:> true 2024-02-02T15:14:13.210 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-02-02T15:14:13.210 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi142.front.sepia.ceph.com' 2024-02-02T15:14:13.211 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-02T15:14:13.284 DEBUG:teuthology.orchestra.run.smithi142:> true 2024-02-02T15:14:13.356 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi142.front.sepia.ceph.com' 2024-02-02T15:14:13.357 INFO:teuthology.run_tasks:Running task clock... 2024-02-02T15:14:13.367 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-02-02T15:14:13.367 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-02T15:14:13.367 DEBUG:teuthology.orchestra.run.smithi001:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-02T15:14:13.372 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-02T15:14:13.372 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-02T15:14:13.374 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-02T15:14:13.374 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-02T15:14:13.403 INFO:teuthology.orchestra.run.smithi071.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-02T15:14:13.418 INFO:teuthology.orchestra.run.smithi071.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-02T15:14:13.433 INFO:teuthology.orchestra.run.smithi142.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-02T15:14:13.440 INFO:teuthology.orchestra.run.smithi001.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-02T15:14:13.449 INFO:teuthology.orchestra.run.smithi142.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-02T15:14:13.474 INFO:teuthology.orchestra.run.smithi001.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-02T15:14:13.498 INFO:teuthology.orchestra.run.smithi071.stderr:sudo: ntpd: command not found 2024-02-02T15:14:13.510 INFO:teuthology.orchestra.run.smithi071.stdout:506 Cannot talk to daemon 2024-02-02T15:14:13.524 INFO:teuthology.orchestra.run.smithi071.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-02T15:14:13.530 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: ntpd: command not found 2024-02-02T15:14:13.539 INFO:teuthology.orchestra.run.smithi071.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-02T15:14:13.542 INFO:teuthology.orchestra.run.smithi142.stdout:506 Cannot talk to daemon 2024-02-02T15:14:13.558 INFO:teuthology.orchestra.run.smithi142.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-02T15:14:13.564 INFO:teuthology.orchestra.run.smithi001.stderr:sudo: ntpd: command not found 2024-02-02T15:14:13.572 INFO:teuthology.orchestra.run.smithi142.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-02T15:14:13.592 INFO:teuthology.orchestra.run.smithi001.stdout:506 Cannot talk to daemon 2024-02-02T15:14:13.618 INFO:teuthology.orchestra.run.smithi071.stderr:bash: ntpq: command not found 2024-02-02T15:14:13.621 INFO:teuthology.orchestra.run.smithi071.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-02T15:14:13.621 INFO:teuthology.orchestra.run.smithi071.stdout:=============================================================================== 2024-02-02T15:14:13.621 INFO:teuthology.orchestra.run.smithi071.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-02T15:14:13.621 INFO:teuthology.orchestra.run.smithi071.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-02T15:14:13.621 INFO:teuthology.orchestra.run.smithi071.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-02T15:14:13.621 INFO:teuthology.orchestra.run.smithi071.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-02T15:14:13.626 INFO:teuthology.orchestra.run.smithi001.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-02T15:14:13.652 INFO:teuthology.orchestra.run.smithi142.stderr:bash: ntpq: command not found 2024-02-02T15:14:13.656 INFO:teuthology.orchestra.run.smithi142.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-02T15:14:13.656 INFO:teuthology.orchestra.run.smithi142.stdout:=============================================================================== 2024-02-02T15:14:13.656 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-02T15:14:13.656 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-02T15:14:13.656 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-02T15:14:13.656 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-02T15:14:13.658 INFO:teuthology.orchestra.run.smithi001.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-02T15:14:13.753 INFO:teuthology.orchestra.run.smithi001.stderr:bash: ntpq: command not found 2024-02-02T15:14:13.760 INFO:teuthology.orchestra.run.smithi001.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-02T15:14:13.760 INFO:teuthology.orchestra.run.smithi001.stdout:=============================================================================== 2024-02-02T15:14:13.760 INFO:teuthology.orchestra.run.smithi001.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-02T15:14:13.760 INFO:teuthology.orchestra.run.smithi001.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-02T15:14:13.760 INFO:teuthology.orchestra.run.smithi001.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-02T15:14:13.760 INFO:teuthology.orchestra.run.smithi001.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-02T15:14:13.761 INFO:teuthology.run_tasks:Running task pexec... 2024-02-02T15:14:13.771 INFO:teuthology.task.pexec:Executing custom commands... 2024-02-02T15:14:13.771 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi001.front.sepia.ceph.com 2024-02-02T15:14:13.772 DEBUG:teuthology.orchestra.run.smithi001:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-02-02T15:14:13.772 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi071.front.sepia.ceph.com 2024-02-02T15:14:13.772 DEBUG:teuthology.orchestra.run.smithi071:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-02-02T15:14:13.772 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi142.front.sepia.ceph.com 2024-02-02T15:14:13.772 DEBUG:teuthology.orchestra.run.smithi142:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-02-02T15:14:14.146 INFO:teuthology.orchestra.run.smithi142.stdout:Updating Subscription Management repositories. 2024-02-02T15:14:14.146 INFO:teuthology.orchestra.run.smithi071.stdout:Updating Subscription Management repositories. 2024-02-02T15:14:14.283 INFO:teuthology.orchestra.run.smithi001.stdout:Updating Subscription Management repositories. 2024-02-02T15:14:14.934 INFO:teuthology.orchestra.run.smithi142.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 149 kB/s | 4.1 kB 00:00 2024-02-02T15:14:14.953 INFO:teuthology.orchestra.run.smithi071.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 158 kB/s | 4.1 kB 00:00 2024-02-02T15:14:15.167 INFO:teuthology.orchestra.run.smithi142.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 165 kB/s | 4.5 kB 00:00 2024-02-02T15:14:15.191 INFO:teuthology.orchestra.run.smithi071.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 153 kB/s | 4.5 kB 00:00 2024-02-02T15:14:15.197 INFO:teuthology.orchestra.run.smithi001.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 108 kB/s | 4.1 kB 00:00 2024-02-02T15:14:15.559 INFO:teuthology.orchestra.run.smithi001.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 138 kB/s | 4.5 kB 00:00 2024-02-02T15:14:18.012 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-02-02T15:14:18.013 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-02-02T15:14:18.013 INFO:teuthology.orchestra.run.smithi071.stdout: Package Architecture Version Repository Size 2024-02-02T15:14:18.013 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-02-02T15:14:18.013 INFO:teuthology.orchestra.run.smithi071.stdout:Resetting modules: 2024-02-02T15:14:18.013 INFO:teuthology.orchestra.run.smithi071.stdout: container-tools 2024-02-02T15:14:18.013 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-02T15:14:18.013 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-02-02T15:14:18.014 INFO:teuthology.orchestra.run.smithi071.stdout:================================================================================ 2024-02-02T15:14:18.014 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-02T15:14:18.244 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-02-02T15:14:18.245 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-02-02T15:14:18.245 INFO:teuthology.orchestra.run.smithi142.stdout: Package Architecture Version Repository Size 2024-02-02T15:14:18.245 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-02-02T15:14:18.245 INFO:teuthology.orchestra.run.smithi142.stdout:Resetting modules: 2024-02-02T15:14:18.245 INFO:teuthology.orchestra.run.smithi142.stdout: container-tools 2024-02-02T15:14:18.245 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-02-02T15:14:18.245 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction Summary 2024-02-02T15:14:18.245 INFO:teuthology.orchestra.run.smithi142.stdout:================================================================================ 2024-02-02T15:14:18.245 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-02-02T15:14:19.105 INFO:teuthology.orchestra.run.smithi001.stdout:Dependencies resolved. 2024-02-02T15:14:19.106 INFO:teuthology.orchestra.run.smithi001.stdout:================================================================================ 2024-02-02T15:14:19.106 INFO:teuthology.orchestra.run.smithi001.stdout: Package Architecture Version Repository Size 2024-02-02T15:14:19.106 INFO:teuthology.orchestra.run.smithi001.stdout:================================================================================ 2024-02-02T15:14:19.106 INFO:teuthology.orchestra.run.smithi001.stdout:Resetting modules: 2024-02-02T15:14:19.106 INFO:teuthology.orchestra.run.smithi001.stdout: container-tools 2024-02-02T15:14:19.107 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:14:19.107 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction Summary 2024-02-02T15:14:19.107 INFO:teuthology.orchestra.run.smithi001.stdout:================================================================================ 2024-02-02T15:14:19.107 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:14:22.760 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-02-02T15:14:23.280 INFO:teuthology.orchestra.run.smithi071.stdout:Updating Subscription Management repositories. 2024-02-02T15:14:23.283 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-02-02T15:14:23.754 INFO:teuthology.orchestra.run.smithi142.stdout:Updating Subscription Management repositories. 2024-02-02T15:14:24.141 INFO:teuthology.orchestra.run.smithi071.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 107 kB/s | 4.1 kB 00:00 2024-02-02T15:14:24.383 INFO:teuthology.orchestra.run.smithi071.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 119 kB/s | 4.5 kB 00:00 2024-02-02T15:14:24.662 INFO:teuthology.orchestra.run.smithi142.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 107 kB/s | 4.1 kB 00:00 2024-02-02T15:14:24.779 INFO:teuthology.orchestra.run.smithi001.stdout:Complete! 2024-02-02T15:14:24.911 INFO:teuthology.orchestra.run.smithi142.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 123 kB/s | 4.5 kB 00:00 2024-02-02T15:14:25.423 INFO:teuthology.orchestra.run.smithi001.stdout:Updating Subscription Management repositories. 2024-02-02T15:14:26.435 INFO:teuthology.orchestra.run.smithi001.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 99 kB/s | 4.1 kB 00:00 2024-02-02T15:14:26.819 INFO:teuthology.orchestra.run.smithi001.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 110 kB/s | 4.5 kB 00:00 2024-02-02T15:14:27.198 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-02-02T15:14:27.352 INFO:teuthology.orchestra.run.smithi071.stdout:========================================================================================================== 2024-02-02T15:14:27.352 INFO:teuthology.orchestra.run.smithi071.stdout: Package Arch Version Repository Size 2024-02-02T15:14:27.352 INFO:teuthology.orchestra.run.smithi071.stdout:========================================================================================================== 2024-02-02T15:14:27.352 INFO:teuthology.orchestra.run.smithi071.stdout:Installing group/module packages: 2024-02-02T15:14:27.352 INFO:teuthology.orchestra.run.smithi071.stdout: aardvark-dns x86_64 2:1.7.0-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 1.0 M 2024-02-02T15:14:27.352 INFO:teuthology.orchestra.run.smithi071.stdout: buildah x86_64 1:1.31.3-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 8.8 M 2024-02-02T15:14:27.352 INFO:teuthology.orchestra.run.smithi071.stdout: cockpit-podman noarch 75-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 739 k 2024-02-02T15:14:27.352 INFO:teuthology.orchestra.run.smithi071.stdout: crun x86_64 1.8.7-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 239 k 2024-02-02T15:14:27.353 INFO:teuthology.orchestra.run.smithi071.stdout: netavark x86_64 2:1.7.0-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-02-02T15:14:27.353 INFO:teuthology.orchestra.run.smithi071.stdout: python3-podman noarch 4.6.0-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 153 k 2024-02-02T15:14:27.353 INFO:teuthology.orchestra.run.smithi071.stdout: skopeo x86_64 2:1.13.3-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 8.1 M 2024-02-02T15:14:27.353 INFO:teuthology.orchestra.run.smithi071.stdout: toolbox x86_64 0.0.99.4-5.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 2.5 M 2024-02-02T15:14:27.353 INFO:teuthology.orchestra.run.smithi071.stdout: udica noarch 0.2.6-20.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 49 k 2024-02-02T15:14:27.353 INFO:teuthology.orchestra.run.smithi071.stdout:Installing dependencies: 2024-02-02T15:14:27.353 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2024-02-02T15:14:27.353 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2024-02-02T15:14:27.353 INFO:teuthology.orchestra.run.smithi071.stdout: yajl x86_64 2.1.0-12.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2024-02-02T15:14:27.353 INFO:teuthology.orchestra.run.smithi071.stdout:Installing weak dependencies: 2024-02-02T15:14:27.353 INFO:teuthology.orchestra.run.smithi071.stdout: criu-libs x86_64 3.18-4.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 39 k 2024-02-02T15:14:27.353 INFO:teuthology.orchestra.run.smithi071.stdout:Installing module profiles: 2024-02-02T15:14:27.353 INFO:teuthology.orchestra.run.smithi071.stdout: container-tools/common 2024-02-02T15:14:27.354 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-02T15:14:27.354 INFO:teuthology.orchestra.run.smithi071.stdout:Enabling module streams: 2024-02-02T15:14:27.354 INFO:teuthology.orchestra.run.smithi071.stdout: container-tools rhel8 2024-02-02T15:14:27.354 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-02T15:14:27.354 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-02-02T15:14:27.354 INFO:teuthology.orchestra.run.smithi071.stdout:========================================================================================================== 2024-02-02T15:14:27.354 INFO:teuthology.orchestra.run.smithi071.stdout:Install 13 Packages 2024-02-02T15:14:27.354 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-02T15:14:27.355 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 26 M 2024-02-02T15:14:27.355 INFO:teuthology.orchestra.run.smithi071.stdout:Installed size: 87 M 2024-02-02T15:14:27.355 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-02-02T15:14:27.794 INFO:teuthology.orchestra.run.smithi071.stdout:(1/13): python3-pyxdg-0.25-16.el8.noarch.rpm 213 kB/s | 94 kB 00:00 2024-02-02T15:14:27.819 INFO:teuthology.orchestra.run.smithi071.stdout:(2/13): python3-pytoml-0.1.14-5.git7dea353.el8. 53 kB/s | 25 kB 00:00 2024-02-02T15:14:27.862 INFO:teuthology.orchestra.run.smithi071.stdout:(3/13): aardvark-dns-1.7.0-1.module+el8.9.0+203 2.0 MB/s | 1.0 MB 00:00 2024-02-02T15:14:27.931 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-02-02T15:14:28.054 INFO:teuthology.orchestra.run.smithi071.stdout:(4/13): cockpit-podman-75-1.module+el8.9.0+2032 3.1 MB/s | 739 kB 00:00 2024-02-02T15:14:28.083 INFO:teuthology.orchestra.run.smithi142.stdout:========================================================================================================== 2024-02-02T15:14:28.084 INFO:teuthology.orchestra.run.smithi142.stdout: Package Arch Version Repository Size 2024-02-02T15:14:28.084 INFO:teuthology.orchestra.run.smithi142.stdout:========================================================================================================== 2024-02-02T15:14:28.084 INFO:teuthology.orchestra.run.smithi142.stdout:Installing group/module packages: 2024-02-02T15:14:28.084 INFO:teuthology.orchestra.run.smithi142.stdout: aardvark-dns x86_64 2:1.7.0-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 1.0 M 2024-02-02T15:14:28.084 INFO:teuthology.orchestra.run.smithi142.stdout: buildah x86_64 1:1.31.3-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 8.8 M 2024-02-02T15:14:28.084 INFO:teuthology.orchestra.run.smithi142.stdout: cockpit-podman noarch 75-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 739 k 2024-02-02T15:14:28.084 INFO:teuthology.orchestra.run.smithi142.stdout: crun x86_64 1.8.7-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 239 k 2024-02-02T15:14:28.084 INFO:teuthology.orchestra.run.smithi142.stdout: netavark x86_64 2:1.7.0-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-02-02T15:14:28.084 INFO:teuthology.orchestra.run.smithi142.stdout: python3-podman noarch 4.6.0-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 153 k 2024-02-02T15:14:28.084 INFO:teuthology.orchestra.run.smithi142.stdout: skopeo x86_64 2:1.13.3-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 8.1 M 2024-02-02T15:14:28.084 INFO:teuthology.orchestra.run.smithi142.stdout: toolbox x86_64 0.0.99.4-5.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 2.5 M 2024-02-02T15:14:28.084 INFO:teuthology.orchestra.run.smithi142.stdout: udica noarch 0.2.6-20.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 49 k 2024-02-02T15:14:28.085 INFO:teuthology.orchestra.run.smithi142.stdout:Installing dependencies: 2024-02-02T15:14:28.085 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2024-02-02T15:14:28.085 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2024-02-02T15:14:28.085 INFO:teuthology.orchestra.run.smithi142.stdout: yajl x86_64 2.1.0-12.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2024-02-02T15:14:28.085 INFO:teuthology.orchestra.run.smithi142.stdout:Installing weak dependencies: 2024-02-02T15:14:28.085 INFO:teuthology.orchestra.run.smithi142.stdout: criu-libs x86_64 3.18-4.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 39 k 2024-02-02T15:14:28.085 INFO:teuthology.orchestra.run.smithi142.stdout:Installing module profiles: 2024-02-02T15:14:28.085 INFO:teuthology.orchestra.run.smithi142.stdout: container-tools/common 2024-02-02T15:14:28.085 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-02-02T15:14:28.085 INFO:teuthology.orchestra.run.smithi142.stdout:Enabling module streams: 2024-02-02T15:14:28.085 INFO:teuthology.orchestra.run.smithi142.stdout: container-tools rhel8 2024-02-02T15:14:28.085 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-02-02T15:14:28.085 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction Summary 2024-02-02T15:14:28.085 INFO:teuthology.orchestra.run.smithi142.stdout:========================================================================================================== 2024-02-02T15:14:28.086 INFO:teuthology.orchestra.run.smithi142.stdout:Install 13 Packages 2024-02-02T15:14:28.086 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-02-02T15:14:28.086 INFO:teuthology.orchestra.run.smithi142.stdout:Total download size: 26 M 2024-02-02T15:14:28.086 INFO:teuthology.orchestra.run.smithi142.stdout:Installed size: 87 M 2024-02-02T15:14:28.086 INFO:teuthology.orchestra.run.smithi142.stdout:Downloading Packages: 2024-02-02T15:14:28.096 INFO:teuthology.orchestra.run.smithi071.stdout:(5/13): criu-libs-3.18-4.module+el8.9.0+20326+3 168 kB/s | 39 kB 00:00 2024-02-02T15:14:28.271 INFO:teuthology.orchestra.run.smithi071.stdout:(6/13): buildah-1.31.3-1.module+el8.9.0+20326+3 19 MB/s | 8.8 MB 00:00 2024-02-02T15:14:28.363 INFO:teuthology.orchestra.run.smithi071.stdout:(7/13): crun-1.8.7-1.module+el8.9.0+20326+38708 775 kB/s | 239 kB 00:00 2024-02-02T15:14:28.383 INFO:teuthology.orchestra.run.smithi142.stdout:(1/13): python3-pytoml-0.1.14-5.git7dea353.el8. 83 kB/s | 25 kB 00:00 2024-02-02T15:14:28.416 INFO:teuthology.orchestra.run.smithi142.stdout:(2/13): python3-pyxdg-0.25-16.el8.noarch.rpm 285 kB/s | 94 kB 00:00 2024-02-02T15:14:28.438 INFO:teuthology.orchestra.run.smithi071.stdout:(8/13): netavark-1.7.0-1.module+el8.9.0+20326+3 11 MB/s | 3.7 MB 00:00 2024-02-02T15:14:28.466 INFO:teuthology.orchestra.run.smithi142.stdout:(3/13): aardvark-dns-1.7.0-1.module+el8.9.0+203 2.7 MB/s | 1.0 MB 00:00 2024-02-02T15:14:28.513 INFO:teuthology.orchestra.run.smithi071.stdout:(9/13): python3-podman-4.6.0-1.module+el8.9.0+2 634 kB/s | 153 kB 00:00 2024-02-02T15:14:28.675 INFO:teuthology.orchestra.run.smithi142.stdout:(4/13): buildah-1.31.3-1.module+el8.9.0+20326+3 30 MB/s | 8.8 MB 00:00 2024-02-02T15:14:28.692 INFO:teuthology.orchestra.run.smithi142.stdout:(5/13): criu-libs-3.18-4.module+el8.9.0+20326+3 175 kB/s | 39 kB 00:00 2024-02-02T15:14:28.725 INFO:teuthology.orchestra.run.smithi142.stdout:(6/13): cockpit-podman-75-1.module+el8.9.0+2032 2.3 MB/s | 739 kB 00:00 2024-02-02T15:14:28.739 INFO:teuthology.orchestra.run.smithi071.stdout:(10/13): udica-0.2.6-20.module+el8.9.0+20326+38 219 kB/s | 49 kB 00:00 2024-02-02T15:14:28.792 INFO:teuthology.orchestra.run.smithi142.stdout:(7/13): crun-1.8.7-1.module+el8.9.0+20326+38708 2.0 MB/s | 239 kB 00:00 2024-02-02T15:14:28.797 INFO:teuthology.orchestra.run.smithi071.stdout:(11/13): toolbox-0.0.99.4-5.module+el8.9.0+2032 7.0 MB/s | 2.5 MB 00:00 2024-02-02T15:14:28.918 INFO:teuthology.orchestra.run.smithi142.stdout:(8/13): netavark-1.7.0-1.module+el8.9.0+20326+3 16 MB/s | 3.7 MB 00:00 2024-02-02T15:14:28.943 INFO:teuthology.orchestra.run.smithi142.stdout:(9/13): python3-podman-4.6.0-1.module+el8.9.0+2 706 kB/s | 153 kB 00:00 2024-02-02T15:14:28.981 INFO:teuthology.orchestra.run.smithi071.stdout:(12/13): skopeo-1.13.3-1.module+el8.9.0+20326+3 13 MB/s | 8.1 MB 00:00 2024-02-02T15:14:28.997 INFO:teuthology.orchestra.run.smithi071.stdout:(13/13): yajl-2.1.0-12.el8.x86_64.rpm 158 kB/s | 41 kB 00:00 2024-02-02T15:14:28.998 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-02-02T15:14:28.998 INFO:teuthology.orchestra.run.smithi071.stdout:Total 16 MB/s | 26 MB 00:01 2024-02-02T15:14:29.126 INFO:teuthology.orchestra.run.smithi142.stdout:(10/13): skopeo-1.13.3-1.module+el8.9.0+20326+3 24 MB/s | 8.1 MB 00:00 2024-02-02T15:14:29.193 INFO:teuthology.orchestra.run.smithi142.stdout:(11/13): udica-0.2.6-20.module+el8.9.0+20326+38 197 kB/s | 49 kB 00:00 2024-02-02T15:14:29.210 INFO:teuthology.orchestra.run.smithi142.stdout:(12/13): yajl-2.1.0-12.el8.x86_64.rpm 492 kB/s | 41 kB 00:00 2024-02-02T15:14:29.283 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-02-02T15:14:29.287 INFO:teuthology.orchestra.run.smithi142.stdout:(13/13): toolbox-0.0.99.4-5.module+el8.9.0+2032 6.8 MB/s | 2.5 MB 00:00 2024-02-02T15:14:29.288 INFO:teuthology.orchestra.run.smithi142.stdout:-------------------------------------------------------------------------------- 2024-02-02T15:14:29.288 INFO:teuthology.orchestra.run.smithi142.stdout:Total 21 MB/s | 26 MB 00:01 2024-02-02T15:14:29.304 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-02-02T15:14:29.304 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-02-02T15:14:29.534 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-02-02T15:14:29.543 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-02-02T15:14:29.572 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction check 2024-02-02T15:14:29.593 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction check succeeded. 2024-02-02T15:14:29.593 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction test 2024-02-02T15:14:29.821 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction test succeeded. 2024-02-02T15:14:29.829 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction 2024-02-02T15:14:30.121 INFO:teuthology.orchestra.run.smithi001.stdout:Dependencies resolved. 2024-02-02T15:14:30.135 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-02-02T15:14:30.282 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : yajl-2.1.0-12.el8.x86_64 1/13 2024-02-02T15:14:30.295 INFO:teuthology.orchestra.run.smithi001.stdout:========================================================================================================== 2024-02-02T15:14:30.295 INFO:teuthology.orchestra.run.smithi001.stdout: Package Arch Version Repository Size 2024-02-02T15:14:30.295 INFO:teuthology.orchestra.run.smithi001.stdout:========================================================================================================== 2024-02-02T15:14:30.295 INFO:teuthology.orchestra.run.smithi001.stdout:Installing group/module packages: 2024-02-02T15:14:30.295 INFO:teuthology.orchestra.run.smithi001.stdout: aardvark-dns x86_64 2:1.7.0-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 1.0 M 2024-02-02T15:14:30.295 INFO:teuthology.orchestra.run.smithi001.stdout: buildah x86_64 1:1.31.3-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 8.8 M 2024-02-02T15:14:30.295 INFO:teuthology.orchestra.run.smithi001.stdout: cockpit-podman noarch 75-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 739 k 2024-02-02T15:14:30.295 INFO:teuthology.orchestra.run.smithi001.stdout: crun x86_64 1.8.7-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 239 k 2024-02-02T15:14:30.296 INFO:teuthology.orchestra.run.smithi001.stdout: netavark x86_64 2:1.7.0-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-02-02T15:14:30.296 INFO:teuthology.orchestra.run.smithi001.stdout: python3-podman noarch 4.6.0-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 153 k 2024-02-02T15:14:30.296 INFO:teuthology.orchestra.run.smithi001.stdout: skopeo x86_64 2:1.13.3-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 8.1 M 2024-02-02T15:14:30.296 INFO:teuthology.orchestra.run.smithi001.stdout: toolbox x86_64 0.0.99.4-5.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 2.5 M 2024-02-02T15:14:30.296 INFO:teuthology.orchestra.run.smithi001.stdout: udica noarch 0.2.6-20.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 49 k 2024-02-02T15:14:30.296 INFO:teuthology.orchestra.run.smithi001.stdout:Installing dependencies: 2024-02-02T15:14:30.296 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2024-02-02T15:14:30.296 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2024-02-02T15:14:30.296 INFO:teuthology.orchestra.run.smithi001.stdout: yajl x86_64 2.1.0-12.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2024-02-02T15:14:30.296 INFO:teuthology.orchestra.run.smithi001.stdout:Installing weak dependencies: 2024-02-02T15:14:30.296 INFO:teuthology.orchestra.run.smithi001.stdout: criu-libs x86_64 3.18-4.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 39 k 2024-02-02T15:14:30.296 INFO:teuthology.orchestra.run.smithi001.stdout:Installing module profiles: 2024-02-02T15:14:30.296 INFO:teuthology.orchestra.run.smithi001.stdout: container-tools/common 2024-02-02T15:14:30.296 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:14:30.297 INFO:teuthology.orchestra.run.smithi001.stdout:Enabling module streams: 2024-02-02T15:14:30.297 INFO:teuthology.orchestra.run.smithi001.stdout: container-tools rhel8 2024-02-02T15:14:30.297 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:14:30.297 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction Summary 2024-02-02T15:14:30.297 INFO:teuthology.orchestra.run.smithi001.stdout:========================================================================================================== 2024-02-02T15:14:30.297 INFO:teuthology.orchestra.run.smithi001.stdout:Install 13 Packages 2024-02-02T15:14:30.297 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:14:30.298 INFO:teuthology.orchestra.run.smithi001.stdout:Total download size: 26 M 2024-02-02T15:14:30.298 INFO:teuthology.orchestra.run.smithi001.stdout:Installed size: 87 M 2024-02-02T15:14:30.298 INFO:teuthology.orchestra.run.smithi001.stdout:Downloading Packages: 2024-02-02T15:14:30.487 INFO:teuthology.orchestra.run.smithi142.stdout: Preparing : 1/1 2024-02-02T15:14:30.523 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 2/13 2024-02-02T15:14:30.569 INFO:teuthology.orchestra.run.smithi001.stdout:(1/13): aardvark-dns-1.7.0-1.module+el8.9.0+203 3.7 MB/s | 1.0 MB 00:00 2024-02-02T15:14:30.594 INFO:teuthology.orchestra.run.smithi001.stdout:(2/13): python3-pytoml-0.1.14-5.git7dea353.el8. 84 kB/s | 25 kB 00:00 2024-02-02T15:14:30.619 INFO:teuthology.orchestra.run.smithi001.stdout:(3/13): python3-pyxdg-0.25-16.el8.noarch.rpm 293 kB/s | 94 kB 00:00 2024-02-02T15:14:30.677 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : yajl-2.1.0-12.el8.x86_64 1/13 2024-02-02T15:14:30.686 INFO:teuthology.orchestra.run.smithi001.stdout:(4/13): cockpit-podman-75-1.module+el8.9.0+2032 7.9 MB/s | 739 kB 00:00 2024-02-02T15:14:30.761 INFO:teuthology.orchestra.run.smithi001.stdout:(5/13): criu-libs-3.18-4.module+el8.9.0+20326+3 277 kB/s | 39 kB 00:00 2024-02-02T15:14:30.803 INFO:teuthology.orchestra.run.smithi001.stdout:(6/13): crun-1.8.7-1.module+el8.9.0+20326+38708 2.0 MB/s | 239 kB 00:00 2024-02-02T15:14:30.892 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 2/13 2024-02-02T15:14:31.004 INFO:teuthology.orchestra.run.smithi001.stdout:(7/13): buildah-1.31.3-1.module+el8.9.0+20326+3 20 MB/s | 8.8 MB 00:00 2024-02-02T15:14:31.029 INFO:teuthology.orchestra.run.smithi001.stdout:(8/13): python3-podman-4.6.0-1.module+el8.9.0+2 680 kB/s | 153 kB 00:00 2024-02-02T15:14:31.033 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : aardvark-dns-2:1.7.0-1.module+el8.9.0+20326+387084 3/13 2024-02-02T15:14:31.179 INFO:teuthology.orchestra.run.smithi001.stdout:(9/13): netavark-1.7.0-1.module+el8.9.0+20326+3 8.9 MB/s | 3.7 MB 00:00 2024-02-02T15:14:31.233 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : netavark-2:1.7.0-1.module+el8.9.0+20326+387084d0.x 4/13 2024-02-02T15:14:31.271 INFO:teuthology.orchestra.run.smithi001.stdout:(10/13): toolbox-0.0.99.4-5.module+el8.9.0+2032 10 MB/s | 2.5 MB 00:00 2024-02-02T15:14:31.330 INFO:teuthology.orchestra.run.smithi001.stdout:(11/13): udica-0.2.6-20.module+el8.9.0+20326+38 329 kB/s | 49 kB 00:00 2024-02-02T15:14:31.363 INFO:teuthology.orchestra.run.smithi001.stdout:(12/13): yajl-2.1.0-12.el8.x86_64.rpm 447 kB/s | 41 kB 00:00 2024-02-02T15:14:31.377 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : aardvark-dns-2:1.7.0-1.module+el8.9.0+20326+387084 3/13 2024-02-02T15:14:31.379 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 5/13 2024-02-02T15:14:31.505 INFO:teuthology.orchestra.run.smithi001.stdout:(13/13): skopeo-1.13.3-1.module+el8.9.0+20326+3 16 MB/s | 8.1 MB 00:00 2024-02-02T15:14:31.506 INFO:teuthology.orchestra.run.smithi001.stdout:-------------------------------------------------------------------------------- 2024-02-02T15:14:31.506 INFO:teuthology.orchestra.run.smithi001.stdout:Total 21 MB/s | 26 MB 00:01 2024-02-02T15:14:31.596 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 6/13 2024-02-02T15:14:31.610 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : netavark-2:1.7.0-1.module+el8.9.0+20326+387084d0.x 4/13 2024-02-02T15:14:31.799 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 5/13 2024-02-02T15:14:31.972 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction check 2024-02-02T15:14:31.999 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction check succeeded. 2024-02-02T15:14:31.999 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction test 2024-02-02T15:14:32.109 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 6/13 2024-02-02T15:14:32.521 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction test succeeded. 2024-02-02T15:14:32.521 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-podman-4.6.0-1.module+el8.9.0+20326+387084 7/13 2024-02-02T15:14:32.531 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction 2024-02-02T15:14:32.739 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : buildah-1:1.31.3-1.module+el8.9.0+20326+387084d0.x 8/13 2024-02-02T15:14:32.965 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : crun-1.8.7-1.module+el8.9.0+20326+387084d0.x86_64 9/13 2024-02-02T15:14:33.066 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-podman-4.6.0-1.module+el8.9.0+20326+387084 7/13 2024-02-02T15:14:33.218 INFO:teuthology.orchestra.run.smithi001.stdout: Preparing : 1/1 2024-02-02T15:14:33.307 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : buildah-1:1.31.3-1.module+el8.9.0+20326+387084d0.x 8/13 2024-02-02T15:14:33.340 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : udica-0.2.6-20.module+el8.9.0+20326+387084d0.noarc 10/13 2024-02-02T15:14:33.363 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : yajl-2.1.0-12.el8.x86_64 1/13 2024-02-02T15:14:33.534 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : crun-1.8.7-1.module+el8.9.0+20326+387084d0.x86_64 9/13 2024-02-02T15:14:33.604 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 2/13 2024-02-02T15:14:33.927 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : udica-0.2.6-20.module+el8.9.0+20326+387084d0.noarc 10/13 2024-02-02T15:14:34.172 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : aardvark-dns-2:1.7.0-1.module+el8.9.0+20326+387084 3/13 2024-02-02T15:14:34.230 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : toolbox-0.0.99.4-5.module+el8.9.0+20326+387084d0.x 11/13 2024-02-02T15:14:34.363 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : netavark-2:1.7.0-1.module+el8.9.0+20326+387084d0.x 4/13 2024-02-02T15:14:34.485 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : skopeo-2:1.13.3-1.module+el8.9.0+20326+387084d0.x8 12/13 2024-02-02T15:14:34.570 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 5/13 2024-02-02T15:14:34.669 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : cockpit-podman-75-1.module+el8.9.0+20326+387084d0. 13/13 2024-02-02T15:14:34.800 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 6/13 2024-02-02T15:14:34.808 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : toolbox-0.0.99.4-5.module+el8.9.0+20326+387084d0.x 11/13 2024-02-02T15:14:35.055 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : skopeo-2:1.13.3-1.module+el8.9.0+20326+387084d0.x8 12/13 2024-02-02T15:14:35.099 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: cockpit-podman-75-1.module+el8.9.0+20326+387084d0. 13/13 2024-02-02T15:14:35.099 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/13 2024-02-02T15:14:35.099 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 2/13 2024-02-02T15:14:35.099 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : aardvark-dns-2:1.7.0-1.module+el8.9.0+20326+387084 3/13 2024-02-02T15:14:35.099 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : buildah-1:1.31.3-1.module+el8.9.0+20326+387084d0.x 4/13 2024-02-02T15:14:35.099 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : cockpit-podman-75-1.module+el8.9.0+20326+387084d0. 5/13 2024-02-02T15:14:35.099 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 6/13 2024-02-02T15:14:35.099 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : crun-1.8.7-1.module+el8.9.0+20326+387084d0.x86_64 7/13 2024-02-02T15:14:35.099 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : netavark-2:1.7.0-1.module+el8.9.0+20326+387084d0.x 8/13 2024-02-02T15:14:35.099 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-podman-4.6.0-1.module+el8.9.0+20326+387084 9/13 2024-02-02T15:14:35.099 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : skopeo-2:1.13.3-1.module+el8.9.0+20326+387084d0.x8 10/13 2024-02-02T15:14:35.099 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : toolbox-0.0.99.4-5.module+el8.9.0+20326+387084d0.x 11/13 2024-02-02T15:14:35.099 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : udica-0.2.6-20.module+el8.9.0+20326+387084d0.noarc 12/13 2024-02-02T15:14:35.255 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : cockpit-podman-75-1.module+el8.9.0+20326+387084d0. 13/13 2024-02-02T15:14:35.671 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: cockpit-podman-75-1.module+el8.9.0+20326+387084d0. 13/13 2024-02-02T15:14:35.671 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/13 2024-02-02T15:14:35.671 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 2/13 2024-02-02T15:14:35.671 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : aardvark-dns-2:1.7.0-1.module+el8.9.0+20326+387084 3/13 2024-02-02T15:14:35.671 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : buildah-1:1.31.3-1.module+el8.9.0+20326+387084d0.x 4/13 2024-02-02T15:14:35.671 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : cockpit-podman-75-1.module+el8.9.0+20326+387084d0. 5/13 2024-02-02T15:14:35.671 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 6/13 2024-02-02T15:14:35.671 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : crun-1.8.7-1.module+el8.9.0+20326+387084d0.x86_64 7/13 2024-02-02T15:14:35.671 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : netavark-2:1.7.0-1.module+el8.9.0+20326+387084d0.x 8/13 2024-02-02T15:14:35.671 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-podman-4.6.0-1.module+el8.9.0+20326+387084 9/13 2024-02-02T15:14:35.671 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : skopeo-2:1.13.3-1.module+el8.9.0+20326+387084d0.x8 10/13 2024-02-02T15:14:35.672 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : toolbox-0.0.99.4-5.module+el8.9.0+20326+387084d0.x 11/13 2024-02-02T15:14:35.672 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : udica-0.2.6-20.module+el8.9.0+20326+387084d0.noarc 12/13 2024-02-02T15:14:35.803 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-podman-4.6.0-1.module+el8.9.0+20326+387084 7/13 2024-02-02T15:14:36.032 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : buildah-1:1.31.3-1.module+el8.9.0+20326+387084d0.x 8/13 2024-02-02T15:14:36.209 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : crun-1.8.7-1.module+el8.9.0+20326+387084d0.x86_64 9/13 2024-02-02T15:14:36.659 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : udica-0.2.6-20.module+el8.9.0+20326+387084d0.noarc 10/13 2024-02-02T15:14:37.672 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : toolbox-0.0.99.4-5.module+el8.9.0+20326+387084d0.x 11/13 2024-02-02T15:14:37.929 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : skopeo-2:1.13.3-1.module+el8.9.0+20326+387084d0.x8 12/13 2024-02-02T15:14:38.076 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : cockpit-podman-75-1.module+el8.9.0+20326+387084d0. 13/13 2024-02-02T15:14:38.773 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: cockpit-podman-75-1.module+el8.9.0+20326+387084d0. 13/13 2024-02-02T15:14:38.773 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/13 2024-02-02T15:14:38.773 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 2/13 2024-02-02T15:14:38.773 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : aardvark-dns-2:1.7.0-1.module+el8.9.0+20326+387084 3/13 2024-02-02T15:14:38.773 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : buildah-1:1.31.3-1.module+el8.9.0+20326+387084d0.x 4/13 2024-02-02T15:14:38.773 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : cockpit-podman-75-1.module+el8.9.0+20326+387084d0. 5/13 2024-02-02T15:14:38.773 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 6/13 2024-02-02T15:14:38.773 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : crun-1.8.7-1.module+el8.9.0+20326+387084d0.x86_64 7/13 2024-02-02T15:14:38.774 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : netavark-2:1.7.0-1.module+el8.9.0+20326+387084d0.x 8/13 2024-02-02T15:14:38.774 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-podman-4.6.0-1.module+el8.9.0+20326+387084 9/13 2024-02-02T15:14:38.774 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : skopeo-2:1.13.3-1.module+el8.9.0+20326+387084d0.x8 10/13 2024-02-02T15:14:38.774 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : toolbox-0.0.99.4-5.module+el8.9.0+20326+387084d0.x 11/13 2024-02-02T15:14:38.774 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : udica-0.2.6-20.module+el8.9.0+20326+387084d0.noarc 12/13 2024-02-02T15:14:40.359 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 13/13 2024-02-02T15:14:40.359 INFO:teuthology.orchestra.run.smithi071.stdout:Installed products updated. 2024-02-02T15:14:40.883 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-02T15:14:40.883 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-02-02T15:14:40.883 INFO:teuthology.orchestra.run.smithi071.stdout: aardvark-dns-2:1.7.0-1.module+el8.9.0+20326+387084d0.x86_64 2024-02-02T15:14:40.883 INFO:teuthology.orchestra.run.smithi071.stdout: buildah-1:1.31.3-1.module+el8.9.0+20326+387084d0.x86_64 2024-02-02T15:14:40.883 INFO:teuthology.orchestra.run.smithi071.stdout: cockpit-podman-75-1.module+el8.9.0+20326+387084d0.noarch 2024-02-02T15:14:40.883 INFO:teuthology.orchestra.run.smithi071.stdout: criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86_64 2024-02-02T15:14:40.883 INFO:teuthology.orchestra.run.smithi071.stdout: crun-1.8.7-1.module+el8.9.0+20326+387084d0.x86_64 2024-02-02T15:14:40.883 INFO:teuthology.orchestra.run.smithi071.stdout: netavark-2:1.7.0-1.module+el8.9.0+20326+387084d0.x86_64 2024-02-02T15:14:40.883 INFO:teuthology.orchestra.run.smithi071.stdout: python3-podman-4.6.0-1.module+el8.9.0+20326+387084d0.noarch 2024-02-02T15:14:40.883 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2024-02-02T15:14:40.884 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyxdg-0.25-16.el8.noarch 2024-02-02T15:14:40.884 INFO:teuthology.orchestra.run.smithi071.stdout: skopeo-2:1.13.3-1.module+el8.9.0+20326+387084d0.x86_64 2024-02-02T15:14:40.884 INFO:teuthology.orchestra.run.smithi071.stdout: toolbox-0.0.99.4-5.module+el8.9.0+20326+387084d0.x86_64 2024-02-02T15:14:40.884 INFO:teuthology.orchestra.run.smithi071.stdout: udica-0.2.6-20.module+el8.9.0+20326+387084d0.noarch 2024-02-02T15:14:40.884 INFO:teuthology.orchestra.run.smithi071.stdout: yajl-2.1.0-12.el8.x86_64 2024-02-02T15:14:40.884 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-02T15:14:40.884 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-02-02T15:14:41.048 DEBUG:teuthology.parallel:result is None 2024-02-02T15:14:41.275 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 13/13 2024-02-02T15:14:41.275 INFO:teuthology.orchestra.run.smithi142.stdout:Installed products updated. 2024-02-02T15:14:41.864 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-02-02T15:14:41.864 INFO:teuthology.orchestra.run.smithi142.stdout:Installed: 2024-02-02T15:14:41.864 INFO:teuthology.orchestra.run.smithi142.stdout: aardvark-dns-2:1.7.0-1.module+el8.9.0+20326+387084d0.x86_64 2024-02-02T15:14:41.864 INFO:teuthology.orchestra.run.smithi142.stdout: buildah-1:1.31.3-1.module+el8.9.0+20326+387084d0.x86_64 2024-02-02T15:14:41.864 INFO:teuthology.orchestra.run.smithi142.stdout: cockpit-podman-75-1.module+el8.9.0+20326+387084d0.noarch 2024-02-02T15:14:41.864 INFO:teuthology.orchestra.run.smithi142.stdout: criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86_64 2024-02-02T15:14:41.864 INFO:teuthology.orchestra.run.smithi142.stdout: crun-1.8.7-1.module+el8.9.0+20326+387084d0.x86_64 2024-02-02T15:14:41.864 INFO:teuthology.orchestra.run.smithi142.stdout: netavark-2:1.7.0-1.module+el8.9.0+20326+387084d0.x86_64 2024-02-02T15:14:41.864 INFO:teuthology.orchestra.run.smithi142.stdout: python3-podman-4.6.0-1.module+el8.9.0+20326+387084d0.noarch 2024-02-02T15:14:41.865 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2024-02-02T15:14:41.865 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyxdg-0.25-16.el8.noarch 2024-02-02T15:14:41.865 INFO:teuthology.orchestra.run.smithi142.stdout: skopeo-2:1.13.3-1.module+el8.9.0+20326+387084d0.x86_64 2024-02-02T15:14:41.865 INFO:teuthology.orchestra.run.smithi142.stdout: toolbox-0.0.99.4-5.module+el8.9.0+20326+387084d0.x86_64 2024-02-02T15:14:41.865 INFO:teuthology.orchestra.run.smithi142.stdout: udica-0.2.6-20.module+el8.9.0+20326+387084d0.noarch 2024-02-02T15:14:41.865 INFO:teuthology.orchestra.run.smithi142.stdout: yajl-2.1.0-12.el8.x86_64 2024-02-02T15:14:41.865 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-02-02T15:14:41.865 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-02-02T15:14:42.083 DEBUG:teuthology.parallel:result is None 2024-02-02T15:14:44.279 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 13/13 2024-02-02T15:14:44.279 INFO:teuthology.orchestra.run.smithi001.stdout:Installed products updated. 2024-02-02T15:14:44.921 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:14:44.922 INFO:teuthology.orchestra.run.smithi001.stdout:Installed: 2024-02-02T15:14:44.922 INFO:teuthology.orchestra.run.smithi001.stdout: aardvark-dns-2:1.7.0-1.module+el8.9.0+20326+387084d0.x86_64 2024-02-02T15:14:44.922 INFO:teuthology.orchestra.run.smithi001.stdout: buildah-1:1.31.3-1.module+el8.9.0+20326+387084d0.x86_64 2024-02-02T15:14:44.922 INFO:teuthology.orchestra.run.smithi001.stdout: cockpit-podman-75-1.module+el8.9.0+20326+387084d0.noarch 2024-02-02T15:14:44.922 INFO:teuthology.orchestra.run.smithi001.stdout: criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86_64 2024-02-02T15:14:44.922 INFO:teuthology.orchestra.run.smithi001.stdout: crun-1.8.7-1.module+el8.9.0+20326+387084d0.x86_64 2024-02-02T15:14:44.922 INFO:teuthology.orchestra.run.smithi001.stdout: netavark-2:1.7.0-1.module+el8.9.0+20326+387084d0.x86_64 2024-02-02T15:14:44.922 INFO:teuthology.orchestra.run.smithi001.stdout: python3-podman-4.6.0-1.module+el8.9.0+20326+387084d0.noarch 2024-02-02T15:14:44.922 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2024-02-02T15:14:44.922 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pyxdg-0.25-16.el8.noarch 2024-02-02T15:14:44.922 INFO:teuthology.orchestra.run.smithi001.stdout: skopeo-2:1.13.3-1.module+el8.9.0+20326+387084d0.x86_64 2024-02-02T15:14:44.922 INFO:teuthology.orchestra.run.smithi001.stdout: toolbox-0.0.99.4-5.module+el8.9.0+20326+387084d0.x86_64 2024-02-02T15:14:44.922 INFO:teuthology.orchestra.run.smithi001.stdout: udica-0.2.6-20.module+el8.9.0+20326+387084d0.noarch 2024-02-02T15:14:44.923 INFO:teuthology.orchestra.run.smithi001.stdout: yajl-2.1.0-12.el8.x86_64 2024-02-02T15:14:44.923 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:14:44.923 INFO:teuthology.orchestra.run.smithi001.stdout:Complete! 2024-02-02T15:14:45.176 DEBUG:teuthology.parallel:result is None 2024-02-02T15:14:45.176 INFO:teuthology.run_tasks:Running task install... 2024-02-02T15:14:45.188 DEBUG:teuthology.task.install:project ceph 2024-02-02T15:14:45.188 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '0fb327883c124fd83155c94db0804f823b805a12'}} 2024-02-02T15:14:45.188 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '0fb327883c124fd83155c94db0804f823b805a12'} 2024-02-02T15:14:45.188 INFO:teuthology.task.install:Using flavor: default 2024-02-02T15:14:45.194 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-02-02T15:14:45.194 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2024-02-02T15:14:45.194 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '0fb327883c124fd83155c94db0804f823b805a12', 'tag': None, 'wait_for_package': False} 2024-02-02T15:14:45.194 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=0fb327883c124fd83155c94db0804f823b805a12 2024-02-02T15:14:45.196 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '0fb327883c124fd83155c94db0804f823b805a12', 'tag': None, 'wait_for_package': False} 2024-02-02T15:14:45.197 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=0fb327883c124fd83155c94db0804f823b805a12 2024-02-02T15:14:45.198 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '0fb327883c124fd83155c94db0804f823b805a12', 'tag': None, 'wait_for_package': False} 2024-02-02T15:14:45.198 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=0fb327883c124fd83155c94db0804f823b805a12 2024-02-02T15:14:45.430 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing1-reef-2024-02-02-1152/0fb327883c124fd83155c94db0804f823b805a12/centos/8/flavors/default/ 2024-02-02T15:14:45.430 INFO:teuthology.task.install.rpm:Package version is 18.2.1-325.g0fb32788 2024-02-02T15:14:45.434 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing1-reef-2024-02-02-1152/0fb327883c124fd83155c94db0804f823b805a12/centos/8/flavors/default/ 2024-02-02T15:14:45.434 INFO:teuthology.task.install.rpm:Package version is 18.2.1-325.g0fb32788 2024-02-02T15:14:45.443 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing1-reef-2024-02-02-1152/0fb327883c124fd83155c94db0804f823b805a12/centos/8/flavors/default/ 2024-02-02T15:14:45.443 INFO:teuthology.task.install.rpm:Package version is 18.2.1-325.g0fb32788 2024-02-02T15:14:45.605 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing1-reef-2024-02-02-1152/0fb327883c124fd83155c94db0804f823b805a12/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing1-reef-2024-02-02-1152/0fb327883c124fd83155c94db0804f823b805a12/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing1-reef-2024-02-02-1152/0fb327883c124fd83155c94db0804f823b805a12/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-02-02T15:14:45.605 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-02-02T15:14:45.605 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-02-02T15:14:45.610 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing1-reef-2024-02-02-1152/0fb327883c124fd83155c94db0804f823b805a12/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing1-reef-2024-02-02-1152/0fb327883c124fd83155c94db0804f823b805a12/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing1-reef-2024-02-02-1152/0fb327883c124fd83155c94db0804f823b805a12/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-02-02T15:14:45.610 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-02-02T15:14:45.611 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-02-02T15:14:45.625 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing1-reef-2024-02-02-1152/0fb327883c124fd83155c94db0804f823b805a12/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing1-reef-2024-02-02-1152/0fb327883c124fd83155c94db0804f823b805a12/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing1-reef-2024-02-02-1152/0fb327883c124fd83155c94db0804f823b805a12/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-02-02T15:14:45.625 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-02-02T15:14:45.625 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-02-02T15:14:45.639 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-02-02T15:14:45.640 DEBUG:teuthology.orchestra.run.smithi142:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/0fb327883c124fd83155c94db0804f823b805a12/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-02-02T15:14:45.659 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-02-02T15:14:45.659 DEBUG:teuthology.orchestra.run.smithi071:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/0fb327883c124fd83155c94db0804f823b805a12/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-02-02T15:14:45.677 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-02-02T15:14:45.677 DEBUG:teuthology.orchestra.run.smithi001:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/0fb327883c124fd83155c94db0804f823b805a12/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-02-02T15:14:45.711 DEBUG:teuthology.orchestra.run.smithi142:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-02-02T15:14:45.735 DEBUG:teuthology.orchestra.run.smithi071:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-02-02T15:14:45.784 DEBUG:teuthology.orchestra.run.smithi001:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-02-02T15:14:45.792 DEBUG:teuthology.orchestra.run.smithi142:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-02-02T15:14:45.816 DEBUG:teuthology.orchestra.run.smithi071:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-02-02T15:14:45.859 INFO:teuthology.orchestra.run.smithi142.stdout:check_obsoletes = 1 2024-02-02T15:14:45.861 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum clean all 2024-02-02T15:14:45.875 DEBUG:teuthology.orchestra.run.smithi001:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-02-02T15:14:45.885 INFO:teuthology.orchestra.run.smithi071.stdout:check_obsoletes = 1 2024-02-02T15:14:45.886 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum clean all 2024-02-02T15:14:45.940 INFO:teuthology.orchestra.run.smithi001.stdout:check_obsoletes = 1 2024-02-02T15:14:45.944 DEBUG:teuthology.orchestra.run.smithi001:> sudo yum clean all 2024-02-02T15:14:46.236 INFO:teuthology.orchestra.run.smithi142.stdout:Updating Subscription Management repositories. 2024-02-02T15:14:46.263 INFO:teuthology.orchestra.run.smithi071.stdout:Updating Subscription Management repositories. 2024-02-02T15:14:46.421 INFO:teuthology.orchestra.run.smithi001.stdout:Updating Subscription Management repositories. 2024-02-02T15:14:46.677 INFO:teuthology.orchestra.run.smithi142.stdout:57 files removed 2024-02-02T15:14:46.717 INFO:teuthology.orchestra.run.smithi071.stdout:57 files removed 2024-02-02T15:14:46.724 DEBUG:teuthology.orchestra.run.smithi142:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-02-02T15:14:46.763 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-02-02T15:14:47.048 INFO:teuthology.orchestra.run.smithi001.stdout:57 files removed 2024-02-02T15:14:47.060 INFO:teuthology.orchestra.run.smithi142.stdout:Updating Subscription Management repositories. 2024-02-02T15:14:47.101 INFO:teuthology.orchestra.run.smithi071.stdout:Updating Subscription Management repositories. 2024-02-02T15:14:47.106 DEBUG:teuthology.orchestra.run.smithi001:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-02-02T15:14:47.587 INFO:teuthology.orchestra.run.smithi001.stdout:Updating Subscription Management repositories. 2024-02-02T15:14:48.163 INFO:teuthology.orchestra.run.smithi071.stdout:ceph packages for x86_64 168 kB/s | 80 kB 00:00 2024-02-02T15:14:48.165 INFO:teuthology.orchestra.run.smithi142.stdout:ceph packages for x86_64 149 kB/s | 80 kB 00:00 2024-02-02T15:14:48.525 INFO:teuthology.orchestra.run.smithi142.stdout:ceph noarch packages 29 kB/s | 9.5 kB 00:00 2024-02-02T15:14:48.537 INFO:teuthology.orchestra.run.smithi071.stdout:ceph noarch packages 28 kB/s | 9.5 kB 00:00 2024-02-02T15:14:48.548 INFO:teuthology.orchestra.run.smithi001.stdout:ceph packages for x86_64 254 kB/s | 80 kB 00:00 2024-02-02T15:14:48.880 INFO:teuthology.orchestra.run.smithi071.stdout:ceph source packages 4.7 kB/s | 1.5 kB 00:00 2024-02-02T15:14:48.912 INFO:teuthology.orchestra.run.smithi142.stdout:ceph source packages 4.1 kB/s | 1.5 kB 00:00 2024-02-02T15:14:48.932 INFO:teuthology.orchestra.run.smithi001.stdout:ceph noarch packages 28 kB/s | 9.5 kB 00:00 2024-02-02T15:14:49.030 INFO:teuthology.orchestra.run.smithi071.stdout:Copr repo for python3-asyncssh owned by ceph 28 kB/s | 3.5 kB 00:00 2024-02-02T15:14:49.076 INFO:teuthology.orchestra.run.smithi142.stdout:Copr repo for python3-asyncssh owned by ceph 25 kB/s | 3.5 kB 00:00 2024-02-02T15:14:49.216 INFO:teuthology.orchestra.run.smithi001.stdout:ceph source packages 5.8 kB/s | 1.5 kB 00:00 2024-02-02T15:14:49.359 INFO:teuthology.orchestra.run.smithi001.stdout:Copr repo for python3-asyncssh owned by ceph 30 kB/s | 3.5 kB 00:00 2024-02-02T15:14:50.327 INFO:teuthology.orchestra.run.smithi142.stdout:Extra Packages for Enterprise Linux 13 MB/s | 16 MB 00:01 2024-02-02T15:14:50.620 INFO:teuthology.orchestra.run.smithi001.stdout:Extra Packages for Enterprise Linux 13 MB/s | 16 MB 00:01 2024-02-02T15:14:50.650 INFO:teuthology.orchestra.run.smithi071.stdout:Extra Packages for Enterprise Linux 10 MB/s | 16 MB 00:01 2024-02-02T15:14:54.110 INFO:teuthology.orchestra.run.smithi142.stdout:lab-extras 84 kB/s | 24 kB 00:00 2024-02-02T15:14:54.181 INFO:teuthology.orchestra.run.smithi071.stdout:lab-extras 422 kB/s | 24 kB 00:00 2024-02-02T15:14:54.641 INFO:teuthology.orchestra.run.smithi071.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 23 MB/s | 8.8 MB 00:00 2024-02-02T15:14:55.035 INFO:teuthology.orchestra.run.smithi001.stdout:lab-extras 398 kB/s | 24 kB 00:00 2024-02-02T15:14:55.320 INFO:teuthology.orchestra.run.smithi142.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 57 MB/s | 65 MB 00:01 2024-02-02T15:14:56.562 INFO:teuthology.orchestra.run.smithi001.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 45 MB/s | 65 MB 00:01 2024-02-02T15:14:57.192 INFO:teuthology.orchestra.run.smithi071.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 64 MB/s | 65 MB 00:01 2024-02-02T15:15:06.230 INFO:teuthology.orchestra.run.smithi142.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 49 MB/s | 59 MB 00:01 2024-02-02T15:15:07.542 INFO:teuthology.orchestra.run.smithi071.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 63 MB/s | 59 MB 00:00 2024-02-02T15:15:07.855 INFO:teuthology.orchestra.run.smithi001.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 9.7 MB/s | 8.8 MB 00:00 2024-02-02T15:15:11.048 INFO:teuthology.orchestra.run.smithi001.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 48 MB/s | 59 MB 00:01 2024-02-02T15:15:16.513 INFO:teuthology.orchestra.run.smithi142.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 24 MB/s | 8.8 MB 00:00 2024-02-02T15:15:22.729 INFO:teuthology.orchestra.run.smithi071.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2024-02-02T15:15:22.730 INFO:teuthology.orchestra.run.smithi071.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2024-02-02T15:15:22.748 INFO:teuthology.orchestra.run.smithi071.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2024-02-02T15:15:22.749 INFO:teuthology.orchestra.run.smithi071.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2024-02-02T15:15:22.751 INFO:teuthology.orchestra.run.smithi071.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2024-02-02T15:15:22.752 INFO:teuthology.orchestra.run.smithi071.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2024-02-02T15:15:22.754 INFO:teuthology.orchestra.run.smithi071.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2024-02-02T15:15:22.758 INFO:teuthology.orchestra.run.smithi071.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2024-02-02T15:15:22.760 INFO:teuthology.orchestra.run.smithi071.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2024-02-02T15:15:22.967 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-02-02T15:15:22.981 INFO:teuthology.orchestra.run.smithi071.stdout:=========================================================================================================================== 2024-02-02T15:15:22.981 INFO:teuthology.orchestra.run.smithi071.stdout: Package Arch Version Repository Size 2024-02-02T15:15:22.981 INFO:teuthology.orchestra.run.smithi071.stdout:=========================================================================================================================== 2024-02-02T15:15:22.981 INFO:teuthology.orchestra.run.smithi071.stdout:Installing: 2024-02-02T15:15:22.981 INFO:teuthology.orchestra.run.smithi071.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-13.el8_9 rhel-8-for-x86_64-baseos-rpms 144 k 2024-02-02T15:15:22.982 INFO:teuthology.orchestra.run.smithi071.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2024-02-02T15:15:22.982 INFO:teuthology.orchestra.run.smithi071.stdout: ceph x86_64 2:18.2.1-325.g0fb32788.el8 ceph 6.4 k 2024-02-02T15:15:22.982 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-base x86_64 2:18.2.1-325.g0fb32788.el8 ceph 5.4 M 2024-02-02T15:15:22.982 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-fuse x86_64 2:18.2.1-325.g0fb32788.el8 ceph 852 k 2024-02-02T15:15:22.982 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-immutable-object-cache x86_64 2:18.2.1-325.g0fb32788.el8 ceph 165 k 2024-02-02T15:15:22.982 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr x86_64 2:18.2.1-325.g0fb32788.el8 ceph 1.6 M 2024-02-02T15:15:22.982 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-cephadm noarch 2:18.2.1-325.g0fb32788.el8 ceph-noarch 132 k 2024-02-02T15:15:22.982 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-dashboard noarch 2:18.2.1-325.g0fb32788.el8 ceph-noarch 1.8 M 2024-02-02T15:15:22.982 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.1-325.g0fb32788.el8 ceph-noarch 7.4 M 2024-02-02T15:15:22.982 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-rook noarch 2:18.2.1-325.g0fb32788.el8 ceph-noarch 50 k 2024-02-02T15:15:22.982 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-radosgw x86_64 2:18.2.1-325.g0fb32788.el8 ceph 13 M 2024-02-02T15:15:22.983 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-test x86_64 2:18.2.1-325.g0fb32788.el8 ceph 47 M 2024-02-02T15:15:22.983 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-volume noarch 2:18.2.1-325.g0fb32788.el8 ceph-noarch 262 k 2024-02-02T15:15:22.983 INFO:teuthology.orchestra.run.smithi071.stdout: cephadm noarch 2:18.2.1-325.g0fb32788.el8 ceph-noarch 168 k 2024-02-02T15:15:22.983 INFO:teuthology.orchestra.run.smithi071.stdout: cephfs-mirror x86_64 2:18.2.1-325.g0fb32788.el8 ceph 258 k 2024-02-02T15:15:22.983 INFO:teuthology.orchestra.run.smithi071.stdout: cephfs-top noarch 2:18.2.1-325.g0fb32788.el8 ceph-noarch 18 k 2024-02-02T15:15:22.983 INFO:teuthology.orchestra.run.smithi071.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 62 k 2024-02-02T15:15:22.983 INFO:teuthology.orchestra.run.smithi071.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2024-02-02T15:15:22.983 INFO:teuthology.orchestra.run.smithi071.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2024-02-02T15:15:22.983 INFO:teuthology.orchestra.run.smithi071.stdout: libcephfs-devel x86_64 2:18.2.1-325.g0fb32788.el8 ceph 31 k 2024-02-02T15:15:22.983 INFO:teuthology.orchestra.run.smithi071.stdout: libcephfs2 x86_64 2:18.2.1-325.g0fb32788.el8 ceph 775 k 2024-02-02T15:15:22.983 INFO:teuthology.orchestra.run.smithi071.stdout: librados-devel x86_64 2:18.2.1-325.g0fb32788.el8 ceph 139 k 2024-02-02T15:15:22.984 INFO:teuthology.orchestra.run.smithi071.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2024-02-02T15:15:22.984 INFO:teuthology.orchestra.run.smithi071.stdout: libuuid-devel x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 100 k 2024-02-02T15:15:22.984 INFO:teuthology.orchestra.run.smithi071.stdout: openssl-devel x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 2.3 M 2024-02-02T15:15:22.984 INFO:teuthology.orchestra.run.smithi071.stdout: postgresql x86_64 10.23-3.module+el8.9.0+20831+78489779 rhel-8-for-x86_64-appstream-rpms 1.5 M 2024-02-02T15:15:22.984 INFO:teuthology.orchestra.run.smithi071.stdout: postgresql-contrib x86_64 10.23-3.module+el8.9.0+20831+78489779 rhel-8-for-x86_64-appstream-rpms 811 k 2024-02-02T15:15:22.984 INFO:teuthology.orchestra.run.smithi071.stdout: postgresql-server x86_64 10.23-3.module+el8.9.0+20831+78489779 rhel-8-for-x86_64-appstream-rpms 5.1 M 2024-02-02T15:15:22.984 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cephfs x86_64 2:18.2.1-325.g0fb32788.el8 ceph 197 k 2024-02-02T15:15:22.984 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rados x86_64 2:18.2.1-325.g0fb32788.el8 ceph 371 k 2024-02-02T15:15:22.984 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rbd x86_64 2:18.2.1-325.g0fb32788.el8 ceph 365 k 2024-02-02T15:15:22.984 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rgw x86_64 2:18.2.1-325.g0fb32788.el8 ceph 107 k 2024-02-02T15:15:22.985 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-fuse x86_64 2:18.2.1-325.g0fb32788.el8 ceph 94 k 2024-02-02T15:15:22.985 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-mirror x86_64 2:18.2.1-325.g0fb32788.el8 ceph 3.4 M 2024-02-02T15:15:22.985 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-nbd x86_64 2:18.2.1-325.g0fb32788.el8 ceph 181 k 2024-02-02T15:15:22.985 INFO:teuthology.orchestra.run.smithi071.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 73 k 2024-02-02T15:15:22.985 INFO:teuthology.orchestra.run.smithi071.stdout:Upgrading: 2024-02-02T15:15:22.985 INFO:teuthology.orchestra.run.smithi071.stdout: elfutils-libelf x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 232 k 2024-02-02T15:15:22.985 INFO:teuthology.orchestra.run.smithi071.stdout: elfutils-libs x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 303 k 2024-02-02T15:15:22.985 INFO:teuthology.orchestra.run.smithi071.stdout: iproute x86_64 6.2.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 854 k 2024-02-02T15:15:22.985 INFO:teuthology.orchestra.run.smithi071.stdout: libblkid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-02-02T15:15:22.985 INFO:teuthology.orchestra.run.smithi071.stdout: libfdisk x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 254 k 2024-02-02T15:15:22.986 INFO:teuthology.orchestra.run.smithi071.stdout: libmount x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 237 k 2024-02-02T15:15:22.986 INFO:teuthology.orchestra.run.smithi071.stdout: librados2 x86_64 2:18.2.1-325.g0fb32788.el8 ceph 3.7 M 2024-02-02T15:15:22.986 INFO:teuthology.orchestra.run.smithi071.stdout: librbd1 x86_64 2:18.2.1-325.g0fb32788.el8 ceph 3.6 M 2024-02-02T15:15:22.986 INFO:teuthology.orchestra.run.smithi071.stdout: libsmartcols x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 179 k 2024-02-02T15:15:22.986 INFO:teuthology.orchestra.run.smithi071.stdout: libuuid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 99 k 2024-02-02T15:15:22.986 INFO:teuthology.orchestra.run.smithi071.stdout: openssl x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 711 k 2024-02-02T15:15:22.986 INFO:teuthology.orchestra.run.smithi071.stdout: openssl-libs x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 1.5 M 2024-02-02T15:15:22.986 INFO:teuthology.orchestra.run.smithi071.stdout: util-linux x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 2.5 M 2024-02-02T15:15:22.986 INFO:teuthology.orchestra.run.smithi071.stdout: util-linux-user x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 103 k 2024-02-02T15:15:22.986 INFO:teuthology.orchestra.run.smithi071.stdout: xfsdump x86_64 3.1.8-7.el8_9 rhel-8-for-x86_64-baseos-rpms 333 k 2024-02-02T15:15:22.986 INFO:teuthology.orchestra.run.smithi071.stdout: xfsprogs x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 1.1 M 2024-02-02T15:15:22.987 INFO:teuthology.orchestra.run.smithi071.stdout:Installing dependencies: 2024-02-02T15:15:22.987 INFO:teuthology.orchestra.run.smithi071.stdout: automake noarch 1.16.1-8.el8 rhel-8-for-x86_64-appstream-rpms 714 k 2024-02-02T15:15:22.987 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-common x86_64 2:18.2.1-325.g0fb32788.el8 ceph 22 M 2024-02-02T15:15:22.987 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-grafana-dashboards noarch 2:18.2.1-325.g0fb32788.el8 ceph-noarch 23 k 2024-02-02T15:15:22.987 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mds x86_64 2:18.2.1-325.g0fb32788.el8 ceph 2.3 M 2024-02-02T15:15:22.987 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-modules-core noarch 2:18.2.1-325.g0fb32788.el8 ceph-noarch 242 k 2024-02-02T15:15:22.987 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mon x86_64 2:18.2.1-325.g0fb32788.el8 ceph 4.6 M 2024-02-02T15:15:22.987 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-osd x86_64 2:18.2.1-325.g0fb32788.el8 ceph 18 M 2024-02-02T15:15:22.987 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-prometheus-alerts noarch 2:18.2.1-325.g0fb32788.el8 ceph-noarch 15 k 2024-02-02T15:15:22.987 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-selinux x86_64 2:18.2.1-325.g0fb32788.el8 ceph 24 k 2024-02-02T15:15:22.988 INFO:teuthology.orchestra.run.smithi071.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2024-02-02T15:15:22.988 INFO:teuthology.orchestra.run.smithi071.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-02-02T15:15:22.988 INFO:teuthology.orchestra.run.smithi071.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2024-02-02T15:15:22.988 INFO:teuthology.orchestra.run.smithi071.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2024-02-02T15:15:22.988 INFO:teuthology.orchestra.run.smithi071.stdout: krb5-devel x86_64 1.18.2-26.el8_9 rhel-8-for-x86_64-baseos-rpms 562 k 2024-02-02T15:15:22.988 INFO:teuthology.orchestra.run.smithi071.stdout: libcephsqlite x86_64 2:18.2.1-325.g0fb32788.el8 ceph 141 k 2024-02-02T15:15:22.988 INFO:teuthology.orchestra.run.smithi071.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2024-02-02T15:15:22.988 INFO:teuthology.orchestra.run.smithi071.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2024-02-02T15:15:22.988 INFO:teuthology.orchestra.run.smithi071.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-02-02T15:15:22.988 INFO:teuthology.orchestra.run.smithi071.stdout: libpq x86_64 13.11-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2024-02-02T15:15:22.988 INFO:teuthology.orchestra.run.smithi071.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2024-02-02T15:15:22.989 INFO:teuthology.orchestra.run.smithi071.stdout: librabbitmq x86_64 0.9.0-4.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2024-02-02T15:15:22.989 INFO:teuthology.orchestra.run.smithi071.stdout: libradosstriper1 x86_64 2:18.2.1-325.g0fb32788.el8 ceph 474 k 2024-02-02T15:15:22.989 INFO:teuthology.orchestra.run.smithi071.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2024-02-02T15:15:22.989 INFO:teuthology.orchestra.run.smithi071.stdout: librgw2 x86_64 2:18.2.1-325.g0fb32788.el8 ceph 6.5 M 2024-02-02T15:15:22.989 INFO:teuthology.orchestra.run.smithi071.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2024-02-02T15:15:22.989 INFO:teuthology.orchestra.run.smithi071.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2024-02-02T15:15:22.989 INFO:teuthology.orchestra.run.smithi071.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-02-02T15:15:22.989 INFO:teuthology.orchestra.run.smithi071.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2024-02-02T15:15:22.989 INFO:teuthology.orchestra.run.smithi071.stdout: libzstd-devel x86_64 1.4.4-1.el8 rhel-8-for-x86_64-baseos-rpms 44 k 2024-02-02T15:15:22.989 INFO:teuthology.orchestra.run.smithi071.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2024-02-02T15:15:22.989 INFO:teuthology.orchestra.run.smithi071.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2024-02-02T15:15:22.990 INFO:teuthology.orchestra.run.smithi071.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2024-02-02T15:15:22.990 INFO:teuthology.orchestra.run.smithi071.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2024-02-02T15:15:22.990 INFO:teuthology.orchestra.run.smithi071.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2024-02-02T15:15:22.990 INFO:teuthology.orchestra.run.smithi071.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2024-02-02T15:15:22.990 INFO:teuthology.orchestra.run.smithi071.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-02-02T15:15:22.990 INFO:teuthology.orchestra.run.smithi071.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-02-02T15:15:22.990 INFO:teuthology.orchestra.run.smithi071.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2024-02-02T15:15:22.990 INFO:teuthology.orchestra.run.smithi071.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-02-02T15:15:22.990 INFO:teuthology.orchestra.run.smithi071.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-02-02T15:15:22.990 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-02-02T15:15:22.990 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ceph-argparse x86_64 2:18.2.1-325.g0fb32788.el8 ceph 45 k 2024-02-02T15:15:22.990 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ceph-common x86_64 2:18.2.1-325.g0fb32788.el8 ceph 123 k 2024-02-02T15:15:22.991 INFO:teuthology.orchestra.run.smithi071.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-02-02T15:15:22.991 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2024-02-02T15:15:22.991 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-02-02T15:15:22.991 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-02-02T15:15:22.991 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cryptography x86_64 3.2.1-6.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2024-02-02T15:15:22.991 INFO:teuthology.orchestra.run.smithi071.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-02-02T15:15:22.991 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-02-02T15:15:22.991 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-02-02T15:15:22.991 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2024-02-02T15:15:22.991 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2024-02-02T15:15:22.991 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2024-02-02T15:15:22.991 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-02-02T15:15:22.992 INFO:teuthology.orchestra.run.smithi071.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-02-02T15:15:22.992 INFO:teuthology.orchestra.run.smithi071.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-02-02T15:15:22.992 INFO:teuthology.orchestra.run.smithi071.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2024-02-02T15:15:22.992 INFO:teuthology.orchestra.run.smithi071.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2024-02-02T15:15:22.992 INFO:teuthology.orchestra.run.smithi071.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-02-02T15:15:22.992 INFO:teuthology.orchestra.run.smithi071.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-02-02T15:15:22.992 INFO:teuthology.orchestra.run.smithi071.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-02-02T15:15:22.992 INFO:teuthology.orchestra.run.smithi071.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2024-02-02T15:15:22.992 INFO:teuthology.orchestra.run.smithi071.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2024-02-02T15:15:22.992 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-02-02T15:15:22.992 INFO:teuthology.orchestra.run.smithi071.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-02-02T15:15:22.992 INFO:teuthology.orchestra.run.smithi071.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2024-02-02T15:15:22.993 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2024-02-02T15:15:22.993 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2024-02-02T15:15:22.993 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2024-02-02T15:15:22.993 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2024-02-02T15:15:22.993 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2024-02-02T15:15:22.993 INFO:teuthology.orchestra.run.smithi071.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-02-02T15:15:22.993 INFO:teuthology.orchestra.run.smithi071.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-02-02T15:15:22.993 INFO:teuthology.orchestra.run.smithi071.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-02-02T15:15:22.993 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-02-02T15:15:22.993 INFO:teuthology.orchestra.run.smithi071.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2024-02-02T15:15:22.994 INFO:teuthology.orchestra.run.smithi071.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-02-02T15:15:22.994 INFO:teuthology.orchestra.run.smithi071.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-02-02T15:15:22.994 INFO:teuthology.orchestra.run.smithi071.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-02-02T15:15:22.994 INFO:teuthology.orchestra.run.smithi071.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-02-02T15:15:22.994 INFO:teuthology.orchestra.run.smithi071.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-02-02T15:15:22.994 INFO:teuthology.orchestra.run.smithi071.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-02-02T15:15:22.994 INFO:teuthology.orchestra.run.smithi071.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-02-02T15:15:22.994 INFO:teuthology.orchestra.run.smithi071.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-02-02T15:15:22.994 INFO:teuthology.orchestra.run.smithi071.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2024-02-02T15:15:22.994 INFO:teuthology.orchestra.run.smithi071.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-02-02T15:15:22.994 INFO:teuthology.orchestra.run.smithi071.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2024-02-02T15:15:22.994 INFO:teuthology.orchestra.run.smithi071.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-02-02T15:15:22.995 INFO:teuthology.orchestra.run.smithi071.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2024-02-02T15:15:22.995 INFO:teuthology.orchestra.run.smithi071.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2024-02-02T15:15:22.995 INFO:teuthology.orchestra.run.smithi071.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2024-02-02T15:15:22.995 INFO:teuthology.orchestra.run.smithi071.stdout:Enabling module streams: 2024-02-02T15:15:22.995 INFO:teuthology.orchestra.run.smithi071.stdout: postgresql 10 2024-02-02T15:15:22.995 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-02T15:15:22.995 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-02-02T15:15:22.995 INFO:teuthology.orchestra.run.smithi071.stdout:=========================================================================================================================== 2024-02-02T15:15:22.995 INFO:teuthology.orchestra.run.smithi071.stdout:Install 129 Packages 2024-02-02T15:15:22.995 INFO:teuthology.orchestra.run.smithi071.stdout:Upgrade 16 Packages 2024-02-02T15:15:22.996 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-02T15:15:22.996 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 212 M 2024-02-02T15:15:22.996 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-02-02T15:15:23.232 INFO:teuthology.orchestra.run.smithi071.stdout:(1/145): ceph-18.2.1-325.g0fb32788.el8.x86_64.r 33 kB/s | 6.4 kB 00:00 2024-02-02T15:15:23.551 INFO:teuthology.orchestra.run.smithi142.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2024-02-02T15:15:23.552 INFO:teuthology.orchestra.run.smithi142.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2024-02-02T15:15:23.570 INFO:teuthology.orchestra.run.smithi142.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2024-02-02T15:15:23.571 INFO:teuthology.orchestra.run.smithi142.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2024-02-02T15:15:23.573 INFO:teuthology.orchestra.run.smithi142.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2024-02-02T15:15:23.574 INFO:teuthology.orchestra.run.smithi142.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2024-02-02T15:15:23.577 INFO:teuthology.orchestra.run.smithi142.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2024-02-02T15:15:23.580 INFO:teuthology.orchestra.run.smithi142.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2024-02-02T15:15:23.582 INFO:teuthology.orchestra.run.smithi142.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2024-02-02T15:15:23.616 INFO:teuthology.orchestra.run.smithi071.stdout:(2/145): ceph-fuse-18.2.1-325.g0fb32788.el8.x86 2.2 MB/s | 852 kB 00:00 2024-02-02T15:15:23.758 INFO:teuthology.orchestra.run.smithi071.stdout:(3/145): ceph-base-18.2.1-325.g0fb32788.el8.x86 7.5 MB/s | 5.4 MB 00:00 2024-02-02T15:15:23.783 INFO:teuthology.orchestra.run.smithi071.stdout:(4/145): ceph-immutable-object-cache-18.2.1-325 990 kB/s | 165 kB 00:00 2024-02-02T15:15:23.789 INFO:teuthology.orchestra.run.smithi142.stdout:Dependencies resolved. 2024-02-02T15:15:23.803 INFO:teuthology.orchestra.run.smithi142.stdout:=========================================================================================================================== 2024-02-02T15:15:23.803 INFO:teuthology.orchestra.run.smithi142.stdout: Package Arch Version Repository Size 2024-02-02T15:15:23.804 INFO:teuthology.orchestra.run.smithi142.stdout:=========================================================================================================================== 2024-02-02T15:15:23.804 INFO:teuthology.orchestra.run.smithi142.stdout:Installing: 2024-02-02T15:15:23.804 INFO:teuthology.orchestra.run.smithi142.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-13.el8_9 rhel-8-for-x86_64-baseos-rpms 144 k 2024-02-02T15:15:23.804 INFO:teuthology.orchestra.run.smithi142.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2024-02-02T15:15:23.804 INFO:teuthology.orchestra.run.smithi142.stdout: ceph x86_64 2:18.2.1-325.g0fb32788.el8 ceph 6.4 k 2024-02-02T15:15:23.804 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-base x86_64 2:18.2.1-325.g0fb32788.el8 ceph 5.4 M 2024-02-02T15:15:23.804 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-fuse x86_64 2:18.2.1-325.g0fb32788.el8 ceph 852 k 2024-02-02T15:15:23.804 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-immutable-object-cache x86_64 2:18.2.1-325.g0fb32788.el8 ceph 165 k 2024-02-02T15:15:23.804 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr x86_64 2:18.2.1-325.g0fb32788.el8 ceph 1.6 M 2024-02-02T15:15:23.804 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-cephadm noarch 2:18.2.1-325.g0fb32788.el8 ceph-noarch 132 k 2024-02-02T15:15:23.804 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-dashboard noarch 2:18.2.1-325.g0fb32788.el8 ceph-noarch 1.8 M 2024-02-02T15:15:23.804 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.1-325.g0fb32788.el8 ceph-noarch 7.4 M 2024-02-02T15:15:23.805 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-rook noarch 2:18.2.1-325.g0fb32788.el8 ceph-noarch 50 k 2024-02-02T15:15:23.805 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-radosgw x86_64 2:18.2.1-325.g0fb32788.el8 ceph 13 M 2024-02-02T15:15:23.805 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-test x86_64 2:18.2.1-325.g0fb32788.el8 ceph 47 M 2024-02-02T15:15:23.805 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-volume noarch 2:18.2.1-325.g0fb32788.el8 ceph-noarch 262 k 2024-02-02T15:15:23.805 INFO:teuthology.orchestra.run.smithi142.stdout: cephadm noarch 2:18.2.1-325.g0fb32788.el8 ceph-noarch 168 k 2024-02-02T15:15:23.805 INFO:teuthology.orchestra.run.smithi142.stdout: cephfs-mirror x86_64 2:18.2.1-325.g0fb32788.el8 ceph 258 k 2024-02-02T15:15:23.805 INFO:teuthology.orchestra.run.smithi142.stdout: cephfs-top noarch 2:18.2.1-325.g0fb32788.el8 ceph-noarch 18 k 2024-02-02T15:15:23.805 INFO:teuthology.orchestra.run.smithi142.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 62 k 2024-02-02T15:15:23.805 INFO:teuthology.orchestra.run.smithi142.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2024-02-02T15:15:23.805 INFO:teuthology.orchestra.run.smithi142.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2024-02-02T15:15:23.805 INFO:teuthology.orchestra.run.smithi142.stdout: libcephfs-devel x86_64 2:18.2.1-325.g0fb32788.el8 ceph 31 k 2024-02-02T15:15:23.805 INFO:teuthology.orchestra.run.smithi142.stdout: libcephfs2 x86_64 2:18.2.1-325.g0fb32788.el8 ceph 775 k 2024-02-02T15:15:23.805 INFO:teuthology.orchestra.run.smithi142.stdout: librados-devel x86_64 2:18.2.1-325.g0fb32788.el8 ceph 139 k 2024-02-02T15:15:23.805 INFO:teuthology.orchestra.run.smithi142.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2024-02-02T15:15:23.806 INFO:teuthology.orchestra.run.smithi142.stdout: libuuid-devel x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 100 k 2024-02-02T15:15:23.806 INFO:teuthology.orchestra.run.smithi142.stdout: openssl-devel x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 2.3 M 2024-02-02T15:15:23.806 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql x86_64 10.23-3.module+el8.9.0+20831+78489779 rhel-8-for-x86_64-appstream-rpms 1.5 M 2024-02-02T15:15:23.806 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-contrib x86_64 10.23-3.module+el8.9.0+20831+78489779 rhel-8-for-x86_64-appstream-rpms 811 k 2024-02-02T15:15:23.806 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-server x86_64 10.23-3.module+el8.9.0+20831+78489779 rhel-8-for-x86_64-appstream-rpms 5.1 M 2024-02-02T15:15:23.806 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cephfs x86_64 2:18.2.1-325.g0fb32788.el8 ceph 197 k 2024-02-02T15:15:23.806 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rados x86_64 2:18.2.1-325.g0fb32788.el8 ceph 371 k 2024-02-02T15:15:23.806 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rbd x86_64 2:18.2.1-325.g0fb32788.el8 ceph 365 k 2024-02-02T15:15:23.806 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rgw x86_64 2:18.2.1-325.g0fb32788.el8 ceph 107 k 2024-02-02T15:15:23.806 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-fuse x86_64 2:18.2.1-325.g0fb32788.el8 ceph 94 k 2024-02-02T15:15:23.806 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-mirror x86_64 2:18.2.1-325.g0fb32788.el8 ceph 3.4 M 2024-02-02T15:15:23.806 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-nbd x86_64 2:18.2.1-325.g0fb32788.el8 ceph 181 k 2024-02-02T15:15:23.806 INFO:teuthology.orchestra.run.smithi142.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 73 k 2024-02-02T15:15:23.807 INFO:teuthology.orchestra.run.smithi142.stdout:Upgrading: 2024-02-02T15:15:23.807 INFO:teuthology.orchestra.run.smithi142.stdout: elfutils-libelf x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 232 k 2024-02-02T15:15:23.807 INFO:teuthology.orchestra.run.smithi142.stdout: elfutils-libs x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 303 k 2024-02-02T15:15:23.807 INFO:teuthology.orchestra.run.smithi142.stdout: iproute x86_64 6.2.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 854 k 2024-02-02T15:15:23.807 INFO:teuthology.orchestra.run.smithi142.stdout: libblkid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-02-02T15:15:23.807 INFO:teuthology.orchestra.run.smithi142.stdout: libfdisk x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 254 k 2024-02-02T15:15:23.807 INFO:teuthology.orchestra.run.smithi142.stdout: libmount x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 237 k 2024-02-02T15:15:23.807 INFO:teuthology.orchestra.run.smithi142.stdout: librados2 x86_64 2:18.2.1-325.g0fb32788.el8 ceph 3.7 M 2024-02-02T15:15:23.807 INFO:teuthology.orchestra.run.smithi142.stdout: librbd1 x86_64 2:18.2.1-325.g0fb32788.el8 ceph 3.6 M 2024-02-02T15:15:23.807 INFO:teuthology.orchestra.run.smithi142.stdout: libsmartcols x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 179 k 2024-02-02T15:15:23.807 INFO:teuthology.orchestra.run.smithi142.stdout: libuuid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 99 k 2024-02-02T15:15:23.807 INFO:teuthology.orchestra.run.smithi142.stdout: openssl x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 711 k 2024-02-02T15:15:23.807 INFO:teuthology.orchestra.run.smithi142.stdout: openssl-libs x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 1.5 M 2024-02-02T15:15:23.808 INFO:teuthology.orchestra.run.smithi142.stdout: util-linux x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 2.5 M 2024-02-02T15:15:23.808 INFO:teuthology.orchestra.run.smithi142.stdout: util-linux-user x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 103 k 2024-02-02T15:15:23.808 INFO:teuthology.orchestra.run.smithi142.stdout: xfsdump x86_64 3.1.8-7.el8_9 rhel-8-for-x86_64-baseos-rpms 333 k 2024-02-02T15:15:23.808 INFO:teuthology.orchestra.run.smithi142.stdout: xfsprogs x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 1.1 M 2024-02-02T15:15:23.808 INFO:teuthology.orchestra.run.smithi142.stdout:Installing dependencies: 2024-02-02T15:15:23.808 INFO:teuthology.orchestra.run.smithi142.stdout: automake noarch 1.16.1-8.el8 rhel-8-for-x86_64-appstream-rpms 714 k 2024-02-02T15:15:23.808 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-common x86_64 2:18.2.1-325.g0fb32788.el8 ceph 22 M 2024-02-02T15:15:23.808 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-grafana-dashboards noarch 2:18.2.1-325.g0fb32788.el8 ceph-noarch 23 k 2024-02-02T15:15:23.808 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mds x86_64 2:18.2.1-325.g0fb32788.el8 ceph 2.3 M 2024-02-02T15:15:23.808 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-modules-core noarch 2:18.2.1-325.g0fb32788.el8 ceph-noarch 242 k 2024-02-02T15:15:23.808 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mon x86_64 2:18.2.1-325.g0fb32788.el8 ceph 4.6 M 2024-02-02T15:15:23.808 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-osd x86_64 2:18.2.1-325.g0fb32788.el8 ceph 18 M 2024-02-02T15:15:23.808 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-prometheus-alerts noarch 2:18.2.1-325.g0fb32788.el8 ceph-noarch 15 k 2024-02-02T15:15:23.809 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-selinux x86_64 2:18.2.1-325.g0fb32788.el8 ceph 24 k 2024-02-02T15:15:23.809 INFO:teuthology.orchestra.run.smithi142.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2024-02-02T15:15:23.809 INFO:teuthology.orchestra.run.smithi142.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-02-02T15:15:23.809 INFO:teuthology.orchestra.run.smithi142.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2024-02-02T15:15:23.809 INFO:teuthology.orchestra.run.smithi142.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2024-02-02T15:15:23.809 INFO:teuthology.orchestra.run.smithi142.stdout: krb5-devel x86_64 1.18.2-26.el8_9 rhel-8-for-x86_64-baseos-rpms 562 k 2024-02-02T15:15:23.809 INFO:teuthology.orchestra.run.smithi142.stdout: libcephsqlite x86_64 2:18.2.1-325.g0fb32788.el8 ceph 141 k 2024-02-02T15:15:23.809 INFO:teuthology.orchestra.run.smithi142.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2024-02-02T15:15:23.809 INFO:teuthology.orchestra.run.smithi142.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2024-02-02T15:15:23.809 INFO:teuthology.orchestra.run.smithi142.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-02-02T15:15:23.809 INFO:teuthology.orchestra.run.smithi142.stdout: libpq x86_64 13.11-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2024-02-02T15:15:23.809 INFO:teuthology.orchestra.run.smithi142.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2024-02-02T15:15:23.809 INFO:teuthology.orchestra.run.smithi142.stdout: librabbitmq x86_64 0.9.0-4.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2024-02-02T15:15:23.809 INFO:teuthology.orchestra.run.smithi142.stdout: libradosstriper1 x86_64 2:18.2.1-325.g0fb32788.el8 ceph 474 k 2024-02-02T15:15:23.810 INFO:teuthology.orchestra.run.smithi142.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2024-02-02T15:15:23.810 INFO:teuthology.orchestra.run.smithi142.stdout: librgw2 x86_64 2:18.2.1-325.g0fb32788.el8 ceph 6.5 M 2024-02-02T15:15:23.810 INFO:teuthology.orchestra.run.smithi142.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2024-02-02T15:15:23.810 INFO:teuthology.orchestra.run.smithi142.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2024-02-02T15:15:23.810 INFO:teuthology.orchestra.run.smithi142.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-02-02T15:15:23.810 INFO:teuthology.orchestra.run.smithi142.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2024-02-02T15:15:23.810 INFO:teuthology.orchestra.run.smithi142.stdout: libzstd-devel x86_64 1.4.4-1.el8 rhel-8-for-x86_64-baseos-rpms 44 k 2024-02-02T15:15:23.810 INFO:teuthology.orchestra.run.smithi142.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2024-02-02T15:15:23.810 INFO:teuthology.orchestra.run.smithi142.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2024-02-02T15:15:23.810 INFO:teuthology.orchestra.run.smithi142.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2024-02-02T15:15:23.810 INFO:teuthology.orchestra.run.smithi142.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2024-02-02T15:15:23.810 INFO:teuthology.orchestra.run.smithi142.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2024-02-02T15:15:23.810 INFO:teuthology.orchestra.run.smithi142.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2024-02-02T15:15:23.811 INFO:teuthology.orchestra.run.smithi142.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-02-02T15:15:23.811 INFO:teuthology.orchestra.run.smithi142.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-02-02T15:15:23.811 INFO:teuthology.orchestra.run.smithi142.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2024-02-02T15:15:23.811 INFO:teuthology.orchestra.run.smithi142.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-02-02T15:15:23.811 INFO:teuthology.orchestra.run.smithi142.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-02-02T15:15:23.811 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-02-02T15:15:23.811 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ceph-argparse x86_64 2:18.2.1-325.g0fb32788.el8 ceph 45 k 2024-02-02T15:15:23.811 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ceph-common x86_64 2:18.2.1-325.g0fb32788.el8 ceph 123 k 2024-02-02T15:15:23.811 INFO:teuthology.orchestra.run.smithi142.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-02-02T15:15:23.811 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2024-02-02T15:15:23.811 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-02-02T15:15:23.811 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-02-02T15:15:23.811 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cryptography x86_64 3.2.1-6.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2024-02-02T15:15:23.811 INFO:teuthology.orchestra.run.smithi142.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-02-02T15:15:23.812 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-02-02T15:15:23.812 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-02-02T15:15:23.812 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2024-02-02T15:15:23.812 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2024-02-02T15:15:23.812 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2024-02-02T15:15:23.812 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-02-02T15:15:23.812 INFO:teuthology.orchestra.run.smithi142.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-02-02T15:15:23.812 INFO:teuthology.orchestra.run.smithi142.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-02-02T15:15:23.812 INFO:teuthology.orchestra.run.smithi142.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2024-02-02T15:15:23.812 INFO:teuthology.orchestra.run.smithi142.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2024-02-02T15:15:23.812 INFO:teuthology.orchestra.run.smithi142.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-02-02T15:15:23.812 INFO:teuthology.orchestra.run.smithi142.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-02-02T15:15:23.812 INFO:teuthology.orchestra.run.smithi142.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-02-02T15:15:23.813 INFO:teuthology.orchestra.run.smithi142.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2024-02-02T15:15:23.813 INFO:teuthology.orchestra.run.smithi142.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2024-02-02T15:15:23.813 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-02-02T15:15:23.813 INFO:teuthology.orchestra.run.smithi142.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-02-02T15:15:23.813 INFO:teuthology.orchestra.run.smithi142.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2024-02-02T15:15:23.813 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2024-02-02T15:15:23.813 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2024-02-02T15:15:23.813 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2024-02-02T15:15:23.813 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2024-02-02T15:15:23.813 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2024-02-02T15:15:23.813 INFO:teuthology.orchestra.run.smithi142.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-02-02T15:15:23.813 INFO:teuthology.orchestra.run.smithi142.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-02-02T15:15:23.813 INFO:teuthology.orchestra.run.smithi142.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-02-02T15:15:23.813 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-02-02T15:15:23.814 INFO:teuthology.orchestra.run.smithi142.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2024-02-02T15:15:23.814 INFO:teuthology.orchestra.run.smithi142.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-02-02T15:15:23.814 INFO:teuthology.orchestra.run.smithi142.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-02-02T15:15:23.814 INFO:teuthology.orchestra.run.smithi142.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-02-02T15:15:23.814 INFO:teuthology.orchestra.run.smithi142.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-02-02T15:15:23.814 INFO:teuthology.orchestra.run.smithi142.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-02-02T15:15:23.814 INFO:teuthology.orchestra.run.smithi142.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-02-02T15:15:23.814 INFO:teuthology.orchestra.run.smithi142.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-02-02T15:15:23.814 INFO:teuthology.orchestra.run.smithi142.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-02-02T15:15:23.814 INFO:teuthology.orchestra.run.smithi142.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2024-02-02T15:15:23.814 INFO:teuthology.orchestra.run.smithi142.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-02-02T15:15:23.814 INFO:teuthology.orchestra.run.smithi142.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2024-02-02T15:15:23.814 INFO:teuthology.orchestra.run.smithi142.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-02-02T15:15:23.814 INFO:teuthology.orchestra.run.smithi142.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2024-02-02T15:15:23.815 INFO:teuthology.orchestra.run.smithi142.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2024-02-02T15:15:23.815 INFO:teuthology.orchestra.run.smithi142.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2024-02-02T15:15:23.815 INFO:teuthology.orchestra.run.smithi142.stdout:Enabling module streams: 2024-02-02T15:15:23.815 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql 10 2024-02-02T15:15:23.815 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-02-02T15:15:23.815 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction Summary 2024-02-02T15:15:23.815 INFO:teuthology.orchestra.run.smithi142.stdout:=========================================================================================================================== 2024-02-02T15:15:23.815 INFO:teuthology.orchestra.run.smithi142.stdout:Install 129 Packages 2024-02-02T15:15:23.815 INFO:teuthology.orchestra.run.smithi142.stdout:Upgrade 16 Packages 2024-02-02T15:15:23.815 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-02-02T15:15:23.816 INFO:teuthology.orchestra.run.smithi142.stdout:Total download size: 212 M 2024-02-02T15:15:23.816 INFO:teuthology.orchestra.run.smithi142.stdout:Downloading Packages: 2024-02-02T15:15:23.967 INFO:teuthology.orchestra.run.smithi071.stdout:(5/145): ceph-mds-18.2.1-325.g0fb32788.el8.x86_ 11 MB/s | 2.3 MB 00:00 2024-02-02T15:15:24.092 INFO:teuthology.orchestra.run.smithi071.stdout:(6/145): ceph-mgr-18.2.1-325.g0fb32788.el8.x86_ 5.0 MB/s | 1.6 MB 00:00 2024-02-02T15:15:24.330 INFO:teuthology.orchestra.run.smithi142.stdout:(1/145): ceph-18.2.1-325.g0fb32788.el8.x86_64.r 19 kB/s | 6.4 kB 00:00 2024-02-02T15:15:24.467 INFO:teuthology.orchestra.run.smithi071.stdout:(7/145): ceph-mon-18.2.1-325.g0fb32788.el8.x86_ 9.2 MB/s | 4.6 MB 00:00 2024-02-02T15:15:24.781 INFO:teuthology.orchestra.run.smithi142.stdout:(2/145): ceph-fuse-18.2.1-325.g0fb32788.el8.x86 1.8 MB/s | 852 kB 00:00 2024-02-02T15:15:24.914 INFO:teuthology.orchestra.run.smithi142.stdout:(3/145): ceph-immutable-object-cache-18.2.1-325 1.2 MB/s | 165 kB 00:00 2024-02-02T15:15:25.582 INFO:teuthology.orchestra.run.smithi142.stdout:(4/145): ceph-mds-18.2.1-325.g0fb32788.el8.x86_ 3.4 MB/s | 2.3 MB 00:00 2024-02-02T15:15:25.726 INFO:teuthology.orchestra.run.smithi071.stdout:(8/145): ceph-common-18.2.1-325.g0fb32788.el8.x 8.1 MB/s | 22 MB 00:02 2024-02-02T15:15:25.868 INFO:teuthology.orchestra.run.smithi071.stdout:(9/145): ceph-selinux-18.2.1-325.g0fb32788.el8. 172 kB/s | 24 kB 00:00 2024-02-02T15:15:25.940 INFO:teuthology.orchestra.run.smithi142.stdout:(5/145): ceph-mgr-18.2.1-325.g0fb32788.el8.x86_ 4.4 MB/s | 1.6 MB 00:00 2024-02-02T15:15:26.551 INFO:teuthology.orchestra.run.smithi071.stdout:(10/145): ceph-radosgw-18.2.1-325.g0fb32788.el8 6.3 MB/s | 13 MB 00:02 2024-02-02T15:15:26.835 INFO:teuthology.orchestra.run.smithi071.stdout:(11/145): cephfs-mirror-18.2.1-325.g0fb32788.el 913 kB/s | 258 kB 00:00 2024-02-02T15:15:26.840 INFO:teuthology.orchestra.run.smithi142.stdout:(6/145): ceph-base-18.2.1-325.g0fb32788.el8.x86 1.9 MB/s | 5.4 MB 00:02 2024-02-02T15:15:26.990 INFO:teuthology.orchestra.run.smithi142.stdout:(7/145): ceph-mon-18.2.1-325.g0fb32788.el8.x86_ 4.4 MB/s | 4.6 MB 00:01 2024-02-02T15:15:27.002 INFO:teuthology.orchestra.run.smithi071.stdout:(12/145): ceph-osd-18.2.1-325.g0fb32788.el8.x86 6.1 MB/s | 18 MB 00:02 2024-02-02T15:15:27.818 INFO:teuthology.orchestra.run.smithi071.stdout:(13/145): libcephfs-devel-18.2.1-325.g0fb32788. 31 kB/s | 31 kB 00:00 2024-02-02T15:15:28.052 INFO:teuthology.orchestra.run.smithi071.stdout:(14/145): libcephsqlite-18.2.1-325.g0fb32788.el 607 kB/s | 141 kB 00:00 2024-02-02T15:15:28.210 INFO:teuthology.orchestra.run.smithi001.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2024-02-02T15:15:28.212 INFO:teuthology.orchestra.run.smithi001.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2024-02-02T15:15:28.219 INFO:teuthology.orchestra.run.smithi071.stdout:(15/145): librados-devel-18.2.1-325.g0fb32788.e 834 kB/s | 139 kB 00:00 2024-02-02T15:15:28.232 INFO:teuthology.orchestra.run.smithi001.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2024-02-02T15:15:28.233 INFO:teuthology.orchestra.run.smithi001.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2024-02-02T15:15:28.235 INFO:teuthology.orchestra.run.smithi001.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2024-02-02T15:15:28.237 INFO:teuthology.orchestra.run.smithi001.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2024-02-02T15:15:28.240 INFO:teuthology.orchestra.run.smithi001.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2024-02-02T15:15:28.244 INFO:teuthology.orchestra.run.smithi001.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2024-02-02T15:15:28.246 INFO:teuthology.orchestra.run.smithi001.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2024-02-02T15:15:28.436 INFO:teuthology.orchestra.run.smithi071.stdout:(16/145): libradosstriper1-18.2.1-325.g0fb32788 2.1 MB/s | 474 kB 00:00 2024-02-02T15:15:28.479 INFO:teuthology.orchestra.run.smithi001.stdout:Dependencies resolved. 2024-02-02T15:15:28.495 INFO:teuthology.orchestra.run.smithi001.stdout:=========================================================================================================================== 2024-02-02T15:15:28.495 INFO:teuthology.orchestra.run.smithi001.stdout: Package Arch Version Repository Size 2024-02-02T15:15:28.495 INFO:teuthology.orchestra.run.smithi001.stdout:=========================================================================================================================== 2024-02-02T15:15:28.495 INFO:teuthology.orchestra.run.smithi001.stdout:Installing: 2024-02-02T15:15:28.496 INFO:teuthology.orchestra.run.smithi001.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-13.el8_9 rhel-8-for-x86_64-baseos-rpms 144 k 2024-02-02T15:15:28.496 INFO:teuthology.orchestra.run.smithi001.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2024-02-02T15:15:28.496 INFO:teuthology.orchestra.run.smithi001.stdout: ceph x86_64 2:18.2.1-325.g0fb32788.el8 ceph 6.4 k 2024-02-02T15:15:28.496 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-base x86_64 2:18.2.1-325.g0fb32788.el8 ceph 5.4 M 2024-02-02T15:15:28.496 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-fuse x86_64 2:18.2.1-325.g0fb32788.el8 ceph 852 k 2024-02-02T15:15:28.496 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-immutable-object-cache x86_64 2:18.2.1-325.g0fb32788.el8 ceph 165 k 2024-02-02T15:15:28.496 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr x86_64 2:18.2.1-325.g0fb32788.el8 ceph 1.6 M 2024-02-02T15:15:28.496 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-cephadm noarch 2:18.2.1-325.g0fb32788.el8 ceph-noarch 132 k 2024-02-02T15:15:28.496 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-dashboard noarch 2:18.2.1-325.g0fb32788.el8 ceph-noarch 1.8 M 2024-02-02T15:15:28.496 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.1-325.g0fb32788.el8 ceph-noarch 7.4 M 2024-02-02T15:15:28.496 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-rook noarch 2:18.2.1-325.g0fb32788.el8 ceph-noarch 50 k 2024-02-02T15:15:28.496 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-radosgw x86_64 2:18.2.1-325.g0fb32788.el8 ceph 13 M 2024-02-02T15:15:28.497 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-test x86_64 2:18.2.1-325.g0fb32788.el8 ceph 47 M 2024-02-02T15:15:28.497 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-volume noarch 2:18.2.1-325.g0fb32788.el8 ceph-noarch 262 k 2024-02-02T15:15:28.497 INFO:teuthology.orchestra.run.smithi001.stdout: cephadm noarch 2:18.2.1-325.g0fb32788.el8 ceph-noarch 168 k 2024-02-02T15:15:28.497 INFO:teuthology.orchestra.run.smithi001.stdout: cephfs-mirror x86_64 2:18.2.1-325.g0fb32788.el8 ceph 258 k 2024-02-02T15:15:28.497 INFO:teuthology.orchestra.run.smithi001.stdout: cephfs-top noarch 2:18.2.1-325.g0fb32788.el8 ceph-noarch 18 k 2024-02-02T15:15:28.497 INFO:teuthology.orchestra.run.smithi001.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 62 k 2024-02-02T15:15:28.497 INFO:teuthology.orchestra.run.smithi001.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2024-02-02T15:15:28.497 INFO:teuthology.orchestra.run.smithi001.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2024-02-02T15:15:28.497 INFO:teuthology.orchestra.run.smithi001.stdout: libcephfs-devel x86_64 2:18.2.1-325.g0fb32788.el8 ceph 31 k 2024-02-02T15:15:28.497 INFO:teuthology.orchestra.run.smithi001.stdout: libcephfs2 x86_64 2:18.2.1-325.g0fb32788.el8 ceph 775 k 2024-02-02T15:15:28.497 INFO:teuthology.orchestra.run.smithi001.stdout: librados-devel x86_64 2:18.2.1-325.g0fb32788.el8 ceph 139 k 2024-02-02T15:15:28.497 INFO:teuthology.orchestra.run.smithi001.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2024-02-02T15:15:28.497 INFO:teuthology.orchestra.run.smithi001.stdout: libuuid-devel x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 100 k 2024-02-02T15:15:28.497 INFO:teuthology.orchestra.run.smithi001.stdout: openssl-devel x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 2.3 M 2024-02-02T15:15:28.497 INFO:teuthology.orchestra.run.smithi001.stdout: postgresql x86_64 10.23-3.module+el8.9.0+20831+78489779 rhel-8-for-x86_64-appstream-rpms 1.5 M 2024-02-02T15:15:28.498 INFO:teuthology.orchestra.run.smithi001.stdout: postgresql-contrib x86_64 10.23-3.module+el8.9.0+20831+78489779 rhel-8-for-x86_64-appstream-rpms 811 k 2024-02-02T15:15:28.498 INFO:teuthology.orchestra.run.smithi001.stdout: postgresql-server x86_64 10.23-3.module+el8.9.0+20831+78489779 rhel-8-for-x86_64-appstream-rpms 5.1 M 2024-02-02T15:15:28.498 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cephfs x86_64 2:18.2.1-325.g0fb32788.el8 ceph 197 k 2024-02-02T15:15:28.498 INFO:teuthology.orchestra.run.smithi001.stdout: python3-rados x86_64 2:18.2.1-325.g0fb32788.el8 ceph 371 k 2024-02-02T15:15:28.498 INFO:teuthology.orchestra.run.smithi001.stdout: python3-rbd x86_64 2:18.2.1-325.g0fb32788.el8 ceph 365 k 2024-02-02T15:15:28.498 INFO:teuthology.orchestra.run.smithi001.stdout: python3-rgw x86_64 2:18.2.1-325.g0fb32788.el8 ceph 107 k 2024-02-02T15:15:28.498 INFO:teuthology.orchestra.run.smithi001.stdout: rbd-fuse x86_64 2:18.2.1-325.g0fb32788.el8 ceph 94 k 2024-02-02T15:15:28.498 INFO:teuthology.orchestra.run.smithi001.stdout: rbd-mirror x86_64 2:18.2.1-325.g0fb32788.el8 ceph 3.4 M 2024-02-02T15:15:28.498 INFO:teuthology.orchestra.run.smithi001.stdout: rbd-nbd x86_64 2:18.2.1-325.g0fb32788.el8 ceph 181 k 2024-02-02T15:15:28.498 INFO:teuthology.orchestra.run.smithi001.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 73 k 2024-02-02T15:15:28.498 INFO:teuthology.orchestra.run.smithi001.stdout:Upgrading: 2024-02-02T15:15:28.498 INFO:teuthology.orchestra.run.smithi001.stdout: elfutils-libelf x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 232 k 2024-02-02T15:15:28.498 INFO:teuthology.orchestra.run.smithi001.stdout: elfutils-libs x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 303 k 2024-02-02T15:15:28.498 INFO:teuthology.orchestra.run.smithi001.stdout: iproute x86_64 6.2.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 854 k 2024-02-02T15:15:28.499 INFO:teuthology.orchestra.run.smithi001.stdout: libblkid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-02-02T15:15:28.499 INFO:teuthology.orchestra.run.smithi001.stdout: libfdisk x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 254 k 2024-02-02T15:15:28.499 INFO:teuthology.orchestra.run.smithi001.stdout: libmount x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 237 k 2024-02-02T15:15:28.499 INFO:teuthology.orchestra.run.smithi001.stdout: librados2 x86_64 2:18.2.1-325.g0fb32788.el8 ceph 3.7 M 2024-02-02T15:15:28.499 INFO:teuthology.orchestra.run.smithi001.stdout: librbd1 x86_64 2:18.2.1-325.g0fb32788.el8 ceph 3.6 M 2024-02-02T15:15:28.499 INFO:teuthology.orchestra.run.smithi001.stdout: libsmartcols x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 179 k 2024-02-02T15:15:28.499 INFO:teuthology.orchestra.run.smithi001.stdout: libuuid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 99 k 2024-02-02T15:15:28.499 INFO:teuthology.orchestra.run.smithi001.stdout: openssl x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 711 k 2024-02-02T15:15:28.499 INFO:teuthology.orchestra.run.smithi001.stdout: openssl-libs x86_64 1:1.1.1k-12.el8_9 rhel-8-for-x86_64-baseos-rpms 1.5 M 2024-02-02T15:15:28.499 INFO:teuthology.orchestra.run.smithi001.stdout: util-linux x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 2.5 M 2024-02-02T15:15:28.499 INFO:teuthology.orchestra.run.smithi001.stdout: util-linux-user x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 103 k 2024-02-02T15:15:28.499 INFO:teuthology.orchestra.run.smithi001.stdout: xfsdump x86_64 3.1.8-7.el8_9 rhel-8-for-x86_64-baseos-rpms 333 k 2024-02-02T15:15:28.499 INFO:teuthology.orchestra.run.smithi001.stdout: xfsprogs x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 1.1 M 2024-02-02T15:15:28.500 INFO:teuthology.orchestra.run.smithi001.stdout:Installing dependencies: 2024-02-02T15:15:28.500 INFO:teuthology.orchestra.run.smithi001.stdout: automake noarch 1.16.1-8.el8 rhel-8-for-x86_64-appstream-rpms 714 k 2024-02-02T15:15:28.500 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-common x86_64 2:18.2.1-325.g0fb32788.el8 ceph 22 M 2024-02-02T15:15:28.500 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-grafana-dashboards noarch 2:18.2.1-325.g0fb32788.el8 ceph-noarch 23 k 2024-02-02T15:15:28.500 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mds x86_64 2:18.2.1-325.g0fb32788.el8 ceph 2.3 M 2024-02-02T15:15:28.500 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-modules-core noarch 2:18.2.1-325.g0fb32788.el8 ceph-noarch 242 k 2024-02-02T15:15:28.500 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mon x86_64 2:18.2.1-325.g0fb32788.el8 ceph 4.6 M 2024-02-02T15:15:28.500 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-osd x86_64 2:18.2.1-325.g0fb32788.el8 ceph 18 M 2024-02-02T15:15:28.500 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-prometheus-alerts noarch 2:18.2.1-325.g0fb32788.el8 ceph-noarch 15 k 2024-02-02T15:15:28.500 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-selinux x86_64 2:18.2.1-325.g0fb32788.el8 ceph 24 k 2024-02-02T15:15:28.500 INFO:teuthology.orchestra.run.smithi001.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2024-02-02T15:15:28.500 INFO:teuthology.orchestra.run.smithi001.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-02-02T15:15:28.500 INFO:teuthology.orchestra.run.smithi001.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2024-02-02T15:15:28.501 INFO:teuthology.orchestra.run.smithi001.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2024-02-02T15:15:28.501 INFO:teuthology.orchestra.run.smithi001.stdout: krb5-devel x86_64 1.18.2-26.el8_9 rhel-8-for-x86_64-baseos-rpms 562 k 2024-02-02T15:15:28.501 INFO:teuthology.orchestra.run.smithi001.stdout: libcephsqlite x86_64 2:18.2.1-325.g0fb32788.el8 ceph 141 k 2024-02-02T15:15:28.501 INFO:teuthology.orchestra.run.smithi001.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2024-02-02T15:15:28.501 INFO:teuthology.orchestra.run.smithi001.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2024-02-02T15:15:28.501 INFO:teuthology.orchestra.run.smithi001.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-02-02T15:15:28.501 INFO:teuthology.orchestra.run.smithi001.stdout: libpq x86_64 13.11-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2024-02-02T15:15:28.501 INFO:teuthology.orchestra.run.smithi001.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2024-02-02T15:15:28.501 INFO:teuthology.orchestra.run.smithi001.stdout: librabbitmq x86_64 0.9.0-4.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2024-02-02T15:15:28.501 INFO:teuthology.orchestra.run.smithi001.stdout: libradosstriper1 x86_64 2:18.2.1-325.g0fb32788.el8 ceph 474 k 2024-02-02T15:15:28.501 INFO:teuthology.orchestra.run.smithi001.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2024-02-02T15:15:28.501 INFO:teuthology.orchestra.run.smithi001.stdout: librgw2 x86_64 2:18.2.1-325.g0fb32788.el8 ceph 6.5 M 2024-02-02T15:15:28.501 INFO:teuthology.orchestra.run.smithi001.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2024-02-02T15:15:28.502 INFO:teuthology.orchestra.run.smithi001.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2024-02-02T15:15:28.502 INFO:teuthology.orchestra.run.smithi001.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-02-02T15:15:28.502 INFO:teuthology.orchestra.run.smithi001.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2024-02-02T15:15:28.502 INFO:teuthology.orchestra.run.smithi001.stdout: libzstd-devel x86_64 1.4.4-1.el8 rhel-8-for-x86_64-baseos-rpms 44 k 2024-02-02T15:15:28.502 INFO:teuthology.orchestra.run.smithi001.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2024-02-02T15:15:28.502 INFO:teuthology.orchestra.run.smithi001.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2024-02-02T15:15:28.502 INFO:teuthology.orchestra.run.smithi001.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2024-02-02T15:15:28.502 INFO:teuthology.orchestra.run.smithi001.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2024-02-02T15:15:28.502 INFO:teuthology.orchestra.run.smithi001.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2024-02-02T15:15:28.502 INFO:teuthology.orchestra.run.smithi001.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2024-02-02T15:15:28.502 INFO:teuthology.orchestra.run.smithi001.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2024-02-02T15:15:28.502 INFO:teuthology.orchestra.run.smithi001.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-02-02T15:15:28.502 INFO:teuthology.orchestra.run.smithi001.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2024-02-02T15:15:28.503 INFO:teuthology.orchestra.run.smithi001.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-02-02T15:15:28.503 INFO:teuthology.orchestra.run.smithi001.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-02-02T15:15:28.503 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-02-02T15:15:28.503 INFO:teuthology.orchestra.run.smithi001.stdout: python3-ceph-argparse x86_64 2:18.2.1-325.g0fb32788.el8 ceph 45 k 2024-02-02T15:15:28.503 INFO:teuthology.orchestra.run.smithi001.stdout: python3-ceph-common x86_64 2:18.2.1-325.g0fb32788.el8 ceph 123 k 2024-02-02T15:15:28.503 INFO:teuthology.orchestra.run.smithi001.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-02-02T15:15:28.503 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2024-02-02T15:15:28.503 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-02-02T15:15:28.503 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-02-02T15:15:28.503 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cryptography x86_64 3.2.1-6.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2024-02-02T15:15:28.503 INFO:teuthology.orchestra.run.smithi001.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-02-02T15:15:28.503 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-02-02T15:15:28.504 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-02-02T15:15:28.504 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2024-02-02T15:15:28.504 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2024-02-02T15:15:28.504 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2024-02-02T15:15:28.504 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-02-02T15:15:28.504 INFO:teuthology.orchestra.run.smithi001.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-02-02T15:15:28.504 INFO:teuthology.orchestra.run.smithi001.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-02-02T15:15:28.504 INFO:teuthology.orchestra.run.smithi001.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2024-02-02T15:15:28.504 INFO:teuthology.orchestra.run.smithi001.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2024-02-02T15:15:28.504 INFO:teuthology.orchestra.run.smithi001.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-02-02T15:15:28.504 INFO:teuthology.orchestra.run.smithi001.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-02-02T15:15:28.504 INFO:teuthology.orchestra.run.smithi001.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-02-02T15:15:28.505 INFO:teuthology.orchestra.run.smithi001.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2024-02-02T15:15:28.505 INFO:teuthology.orchestra.run.smithi001.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2024-02-02T15:15:28.505 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-02-02T15:15:28.505 INFO:teuthology.orchestra.run.smithi001.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-02-02T15:15:28.505 INFO:teuthology.orchestra.run.smithi001.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2024-02-02T15:15:28.505 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2024-02-02T15:15:28.505 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2024-02-02T15:15:28.505 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2024-02-02T15:15:28.505 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2024-02-02T15:15:28.505 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2024-02-02T15:15:28.505 INFO:teuthology.orchestra.run.smithi001.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-02-02T15:15:28.505 INFO:teuthology.orchestra.run.smithi001.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-02-02T15:15:28.505 INFO:teuthology.orchestra.run.smithi001.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-02-02T15:15:28.506 INFO:teuthology.orchestra.run.smithi001.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-02-02T15:15:28.506 INFO:teuthology.orchestra.run.smithi001.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2024-02-02T15:15:28.506 INFO:teuthology.orchestra.run.smithi001.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-02-02T15:15:28.506 INFO:teuthology.orchestra.run.smithi001.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-02-02T15:15:28.506 INFO:teuthology.orchestra.run.smithi001.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-02-02T15:15:28.506 INFO:teuthology.orchestra.run.smithi001.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-02-02T15:15:28.506 INFO:teuthology.orchestra.run.smithi001.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-02-02T15:15:28.506 INFO:teuthology.orchestra.run.smithi001.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-02-02T15:15:28.506 INFO:teuthology.orchestra.run.smithi001.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-02-02T15:15:28.506 INFO:teuthology.orchestra.run.smithi001.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-02-02T15:15:28.506 INFO:teuthology.orchestra.run.smithi001.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2024-02-02T15:15:28.506 INFO:teuthology.orchestra.run.smithi001.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-02-02T15:15:28.506 INFO:teuthology.orchestra.run.smithi001.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2024-02-02T15:15:28.506 INFO:teuthology.orchestra.run.smithi001.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-02-02T15:15:28.506 INFO:teuthology.orchestra.run.smithi001.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2024-02-02T15:15:28.507 INFO:teuthology.orchestra.run.smithi001.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2024-02-02T15:15:28.507 INFO:teuthology.orchestra.run.smithi001.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2024-02-02T15:15:28.507 INFO:teuthology.orchestra.run.smithi001.stdout:Enabling module streams: 2024-02-02T15:15:28.507 INFO:teuthology.orchestra.run.smithi001.stdout: postgresql 10 2024-02-02T15:15:28.507 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:15:28.507 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction Summary 2024-02-02T15:15:28.507 INFO:teuthology.orchestra.run.smithi001.stdout:=========================================================================================================================== 2024-02-02T15:15:28.507 INFO:teuthology.orchestra.run.smithi001.stdout:Install 129 Packages 2024-02-02T15:15:28.507 INFO:teuthology.orchestra.run.smithi001.stdout:Upgrade 16 Packages 2024-02-02T15:15:28.507 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:15:28.508 INFO:teuthology.orchestra.run.smithi001.stdout:Total download size: 212 M 2024-02-02T15:15:28.508 INFO:teuthology.orchestra.run.smithi001.stdout:Downloading Packages: 2024-02-02T15:15:28.527 INFO:teuthology.orchestra.run.smithi071.stdout:(17/145): libcephfs2-18.2.1-325.g0fb32788.el8.x 508 kB/s | 775 kB 00:01 2024-02-02T15:15:28.573 INFO:teuthology.orchestra.run.smithi142.stdout:(8/145): ceph-common-18.2.1-325.g0fb32788.el8.x 4.7 MB/s | 22 MB 00:04 2024-02-02T15:15:28.620 INFO:teuthology.orchestra.run.smithi071.stdout:(18/145): python3-ceph-argparse-18.2.1-325.g0fb 487 kB/s | 45 kB 00:00 2024-02-02T15:15:28.682 INFO:teuthology.orchestra.run.smithi142.stdout:(9/145): ceph-selinux-18.2.1-325.g0fb32788.el8. 225 kB/s | 24 kB 00:00 2024-02-02T15:15:28.745 INFO:teuthology.orchestra.run.smithi071.stdout:(19/145): python3-ceph-common-18.2.1-325.g0fb32 991 kB/s | 123 kB 00:00 2024-02-02T15:15:28.882 INFO:teuthology.orchestra.run.smithi142.stdout:(10/145): ceph-osd-18.2.1-325.g0fb32788.el8.x86 8.7 MB/s | 18 MB 00:02 2024-02-02T15:15:28.920 INFO:teuthology.orchestra.run.smithi071.stdout:(20/145): python3-cephfs-18.2.1-325.g0fb32788.e 1.1 MB/s | 197 kB 00:00 2024-02-02T15:15:28.949 INFO:teuthology.orchestra.run.smithi001.stdout:(1/145): ceph-18.2.1-325.g0fb32788.el8.x86_64.r 22 kB/s | 6.4 kB 00:00 2024-02-02T15:15:28.999 INFO:teuthology.orchestra.run.smithi142.stdout:(11/145): cephfs-mirror-18.2.1-325.g0fb32788.el 2.2 MB/s | 258 kB 00:00 2024-02-02T15:15:29.124 INFO:teuthology.orchestra.run.smithi142.stdout:(12/145): libcephfs-devel-18.2.1-325.g0fb32788. 248 kB/s | 31 kB 00:00 2024-02-02T15:15:29.362 INFO:teuthology.orchestra.run.smithi071.stdout:(21/145): librgw2-18.2.1-325.g0fb32788.el8.x86_ 7.1 MB/s | 6.5 MB 00:00 2024-02-02T15:15:29.387 INFO:teuthology.orchestra.run.smithi071.stdout:(22/145): python3-rados-18.2.1-325.g0fb32788.el 794 kB/s | 371 kB 00:00 2024-02-02T15:15:29.458 INFO:teuthology.orchestra.run.smithi142.stdout:(13/145): ceph-radosgw-18.2.1-325.g0fb32788.el8 5.3 MB/s | 13 MB 00:02 2024-02-02T15:15:29.479 INFO:teuthology.orchestra.run.smithi071.stdout:(23/145): python3-rbd-18.2.1-325.g0fb32788.el8. 3.1 MB/s | 365 kB 00:00 2024-02-02T15:15:29.492 INFO:teuthology.orchestra.run.smithi142.stdout:(14/145): libcephfs2-18.2.1-325.g0fb32788.el8.x 2.1 MB/s | 775 kB 00:00 2024-02-02T15:15:29.513 INFO:teuthology.orchestra.run.smithi071.stdout:(24/145): python3-rgw-18.2.1-325.g0fb32788.el8. 851 kB/s | 107 kB 00:00 2024-02-02T15:15:29.546 INFO:teuthology.orchestra.run.smithi071.stdout:(25/145): rbd-fuse-18.2.1-325.g0fb32788.el8.x86 1.4 MB/s | 94 kB 00:00 2024-02-02T15:15:29.559 INFO:teuthology.orchestra.run.smithi142.stdout:(15/145): libcephsqlite-18.2.1-325.g0fb32788.el 1.4 MB/s | 141 kB 00:00 2024-02-02T15:15:29.584 INFO:teuthology.orchestra.run.smithi142.stdout:(16/145): librados-devel-18.2.1-325.g0fb32788.e 1.5 MB/s | 139 kB 00:00 2024-02-02T15:15:29.722 INFO:teuthology.orchestra.run.smithi071.stdout:(26/145): rbd-nbd-18.2.1-325.g0fb32788.el8.x86_ 1.0 MB/s | 181 kB 00:00 2024-02-02T15:15:29.725 INFO:teuthology.orchestra.run.smithi142.stdout:(17/145): libradosstriper1-18.2.1-325.g0fb32788 2.8 MB/s | 474 kB 00:00 2024-02-02T15:15:29.782 INFO:teuthology.orchestra.run.smithi001.stdout:(2/145): ceph-fuse-18.2.1-325.g0fb32788.el8.x86 1.0 MB/s | 852 kB 00:00 2024-02-02T15:15:29.813 INFO:teuthology.orchestra.run.smithi071.stdout:(27/145): ceph-grafana-dashboards-18.2.1-325.g0 252 kB/s | 23 kB 00:00 2024-02-02T15:15:29.826 INFO:teuthology.orchestra.run.smithi142.stdout:(18/145): python3-ceph-argparse-18.2.1-325.g0fb 446 kB/s | 45 kB 00:00 2024-02-02T15:15:29.897 INFO:teuthology.orchestra.run.smithi071.stdout:(28/145): ceph-mgr-cephadm-18.2.1-325.g0fb32788 1.5 MB/s | 132 kB 00:00 2024-02-02T15:15:29.934 INFO:teuthology.orchestra.run.smithi142.stdout:(19/145): python3-ceph-common-18.2.1-325.g0fb32 1.1 MB/s | 123 kB 00:00 2024-02-02T15:15:30.099 INFO:teuthology.orchestra.run.smithi001.stdout:(3/145): ceph-immutable-object-cache-18.2.1-325 523 kB/s | 165 kB 00:00 2024-02-02T15:15:30.306 INFO:teuthology.orchestra.run.smithi071.stdout:(29/145): ceph-mgr-dashboard-18.2.1-325.g0fb327 4.3 MB/s | 1.8 MB 00:00 2024-02-02T15:15:30.560 INFO:teuthology.orchestra.run.smithi142.stdout:(20/145): librgw2-18.2.1-325.g0fb32788.el8.x86_ 6.7 MB/s | 6.5 MB 00:00 2024-02-02T15:15:30.585 INFO:teuthology.orchestra.run.smithi142.stdout:(21/145): python3-cephfs-18.2.1-325.g0fb32788.e 303 kB/s | 197 kB 00:00 2024-02-02T15:15:30.710 INFO:teuthology.orchestra.run.smithi142.stdout:(22/145): python3-rbd-18.2.1-325.g0fb32788.el8. 2.9 MB/s | 365 kB 00:00 2024-02-02T15:15:30.769 INFO:teuthology.orchestra.run.smithi142.stdout:(23/145): python3-rados-18.2.1-325.g0fb32788.el 2.0 MB/s | 371 kB 00:00 2024-02-02T15:15:30.828 INFO:teuthology.orchestra.run.smithi142.stdout:(24/145): python3-rgw-18.2.1-325.g0fb32788.el8. 909 kB/s | 107 kB 00:00 2024-02-02T15:15:30.881 INFO:teuthology.orchestra.run.smithi071.stdout:(30/145): rbd-mirror-18.2.1-325.g0fb32788.el8.x 2.5 MB/s | 3.4 MB 00:01 2024-02-02T15:15:30.898 INFO:teuthology.orchestra.run.smithi142.stdout:(25/145): rbd-fuse-18.2.1-325.g0fb32788.el8.x86 731 kB/s | 94 kB 00:00 2024-02-02T15:15:31.020 INFO:teuthology.orchestra.run.smithi142.stdout:(26/145): rbd-nbd-18.2.1-325.g0fb32788.el8.x86_ 1.5 MB/s | 181 kB 00:00 2024-02-02T15:15:31.065 INFO:teuthology.orchestra.run.smithi071.stdout:(31/145): ceph-mgr-modules-core-18.2.1-325.g0fb 1.3 MB/s | 242 kB 00:00 2024-02-02T15:15:31.137 INFO:teuthology.orchestra.run.smithi142.stdout:(27/145): ceph-grafana-dashboards-18.2.1-325.g0 198 kB/s | 23 kB 00:00 2024-02-02T15:15:31.198 INFO:teuthology.orchestra.run.smithi071.stdout:(32/145): ceph-mgr-rook-18.2.1-325.g0fb32788.el 377 kB/s | 50 kB 00:00 2024-02-02T15:15:31.279 INFO:teuthology.orchestra.run.smithi142.stdout:(28/145): ceph-mgr-cephadm-18.2.1-325.g0fb32788 931 kB/s | 132 kB 00:00 2024-02-02T15:15:31.307 INFO:teuthology.orchestra.run.smithi071.stdout:(33/145): ceph-prometheus-alerts-18.2.1-325.g0f 134 kB/s | 15 kB 00:00 2024-02-02T15:15:31.457 INFO:teuthology.orchestra.run.smithi071.stdout:(34/145): ceph-mgr-diskprediction-local-18.2.1- 6.4 MB/s | 7.4 MB 00:01 2024-02-02T15:15:31.504 INFO:teuthology.orchestra.run.smithi142.stdout:(29/145): rbd-mirror-18.2.1-325.g0fb32788.el8.x 5.1 MB/s | 3.4 MB 00:00 2024-02-02T15:15:31.507 INFO:teuthology.orchestra.run.smithi071.stdout:(35/145): ceph-volume-18.2.1-325.g0fb32788.el8. 1.3 MB/s | 262 kB 00:00 2024-02-02T15:15:31.558 INFO:teuthology.orchestra.run.smithi071.stdout:(36/145): cephadm-18.2.1-325.g0fb32788.el8.noar 1.6 MB/s | 168 kB 00:00 2024-02-02T15:15:31.608 INFO:teuthology.orchestra.run.smithi001.stdout:(4/145): ceph-base-18.2.1-325.g0fb32788.el8.x86 1.8 MB/s | 5.4 MB 00:02 2024-02-02T15:15:31.608 INFO:teuthology.orchestra.run.smithi071.stdout:(37/145): cephfs-top-18.2.1-325.g0fb32788.el8.n 179 kB/s | 18 kB 00:00 2024-02-02T15:15:31.846 INFO:teuthology.orchestra.run.smithi142.stdout:(30/145): ceph-mgr-dashboard-18.2.1-325.g0fb327 3.1 MB/s | 1.8 MB 00:00 2024-02-02T15:15:32.008 INFO:teuthology.orchestra.run.smithi071.stdout:(38/145): liboath-2.6.2-3.el8.x86_64.rpm 147 kB/s | 59 kB 00:00 2024-02-02T15:15:32.016 INFO:teuthology.orchestra.run.smithi001.stdout:(5/145): ceph-mgr-18.2.1-325.g0fb32788.el8.x86_ 3.8 MB/s | 1.6 MB 00:00 2024-02-02T15:15:32.030 INFO:teuthology.orchestra.run.smithi142.stdout:(31/145): ceph-mgr-modules-core-18.2.1-325.g0fb 1.3 MB/s | 242 kB 00:00 2024-02-02T15:15:32.083 INFO:teuthology.orchestra.run.smithi001.stdout:(6/145): ceph-mds-18.2.1-325.g0fb32788.el8.x86_ 1.2 MB/s | 2.3 MB 00:01 2024-02-02T15:15:32.130 INFO:teuthology.orchestra.run.smithi142.stdout:(32/145): ceph-mgr-rook-18.2.1-325.g0fb32788.el 503 kB/s | 50 kB 00:00 2024-02-02T15:15:32.143 INFO:teuthology.orchestra.run.smithi071.stdout:(39/145): gperftools-libs-2.7-9.el8.x86_64.rpm 523 kB/s | 306 kB 00:00 2024-02-02T15:15:32.175 INFO:teuthology.orchestra.run.smithi071.stdout:(40/145): libunwind-1.3.1-3.el8.x86_64.rpm 450 kB/s | 75 kB 00:00 2024-02-02T15:15:32.205 INFO:teuthology.orchestra.run.smithi142.stdout:(33/145): ceph-prometheus-alerts-18.2.1-325.g0f 193 kB/s | 15 kB 00:00 2024-02-02T15:15:32.309 INFO:teuthology.orchestra.run.smithi071.stdout:(41/145): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 327 kB/s | 44 kB 00:00 2024-02-02T15:15:32.359 INFO:teuthology.orchestra.run.smithi071.stdout:(42/145): python3-asyncssh-2.7.0-2.el8.noarch.r 2.2 MB/s | 476 kB 00:00 2024-02-02T15:15:32.372 INFO:teuthology.orchestra.run.smithi142.stdout:(34/145): ceph-volume-18.2.1-325.g0fb32788.el8. 1.5 MB/s | 262 kB 00:00 2024-02-02T15:15:32.468 INFO:teuthology.orchestra.run.smithi071.stdout:(43/145): python3-cachetools-3.1.1-4.el8.noarch 304 kB/s | 33 kB 00:00 2024-02-02T15:15:32.497 INFO:teuthology.orchestra.run.smithi142.stdout:(35/145): cephadm-18.2.1-325.g0fb32788.el8.noar 1.3 MB/s | 168 kB 00:00 2024-02-02T15:15:32.509 INFO:teuthology.orchestra.run.smithi071.stdout:(44/145): python3-beautifulsoup4-4.6.3-2.el8.1. 924 kB/s | 185 kB 00:00 2024-02-02T15:15:32.560 INFO:teuthology.orchestra.run.smithi071.stdout:(45/145): python3-certifi-2018.10.15-7.el8.noar 178 kB/s | 16 kB 00:00 2024-02-02T15:15:32.631 INFO:teuthology.orchestra.run.smithi142.stdout:(36/145): ceph-mgr-diskprediction-local-18.2.1- 6.6 MB/s | 7.4 MB 00:01 2024-02-02T15:15:32.656 INFO:teuthology.orchestra.run.smithi142.stdout:(37/145): cephfs-top-18.2.1-325.g0fb32788.el8.n 113 kB/s | 18 kB 00:00 2024-02-02T15:15:32.733 INFO:teuthology.orchestra.run.smithi001.stdout:(7/145): ceph-mon-18.2.1-325.g0fb32788.el8.x86_ 6.4 MB/s | 4.6 MB 00:00 2024-02-02T15:15:33.023 INFO:teuthology.orchestra.run.smithi142.stdout:(38/145): liboath-2.6.2-3.el8.x86_64.rpm 160 kB/s | 59 kB 00:00 2024-02-02T15:15:33.035 INFO:teuthology.orchestra.run.smithi071.stdout:(46/145): ceph-test-18.2.1-325.g0fb32788.el8.x8 6.5 MB/s | 47 MB 00:07 2024-02-02T15:15:33.052 INFO:teuthology.orchestra.run.smithi071.stdout:(47/145): python3-cheroot-8.5.2-1.el8.noarch.rp 318 kB/s | 173 kB 00:00 2024-02-02T15:15:33.110 INFO:teuthology.orchestra.run.smithi071.stdout:(48/145): python3-cherrypy-18.4.0-1.el8.noarch. 699 kB/s | 384 kB 00:00 2024-02-02T15:15:33.161 INFO:teuthology.orchestra.run.smithi071.stdout:(49/145): python3-jaraco-6.2-6.el8.noarch.rpm 102 kB/s | 11 kB 00:00 2024-02-02T15:15:33.173 INFO:teuthology.orchestra.run.smithi142.stdout:(39/145): gperftools-libs-2.7-9.el8.x86_64.rpm 565 kB/s | 306 kB 00:00 2024-02-02T15:15:33.199 INFO:teuthology.orchestra.run.smithi142.stdout:(40/145): libunwind-1.3.1-3.el8.x86_64.rpm 428 kB/s | 75 kB 00:00 2024-02-02T15:15:33.211 INFO:teuthology.orchestra.run.smithi071.stdout:(50/145): python3-jaraco-functools-2.0-4.el8.no 174 kB/s | 18 kB 00:00 2024-02-02T15:15:33.316 INFO:teuthology.orchestra.run.smithi142.stdout:(41/145): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 373 kB/s | 44 kB 00:00 2024-02-02T15:15:33.328 INFO:teuthology.orchestra.run.smithi071.stdout:(51/145): python3-logutils-0.3.5-11.el8.noarch. 418 kB/s | 49 kB 00:00 2024-02-02T15:15:33.399 INFO:teuthology.orchestra.run.smithi142.stdout:(42/145): python3-asyncssh-2.7.0-2.el8.noarch.r 2.1 MB/s | 476 kB 00:00 2024-02-02T15:15:33.454 INFO:teuthology.orchestra.run.smithi071.stdout:(52/145): python3-more-itertools-7.2.0-3.el8.no 470 kB/s | 59 kB 00:00 2024-02-02T15:15:33.479 INFO:teuthology.orchestra.run.smithi071.stdout:(53/145): python3-google-auth-1.1.1-10.el8.noar 183 kB/s | 81 kB 00:00 2024-02-02T15:15:33.491 INFO:teuthology.orchestra.run.smithi142.stdout:(43/145): python3-beautifulsoup4-4.6.3-2.el8.1. 1.0 MB/s | 185 kB 00:00 2024-02-02T15:15:33.512 INFO:teuthology.orchestra.run.smithi071.stdout:(54/145): python3-kubernetes-11.0.0-6.el8.noarc 3.3 MB/s | 1.2 MB 00:00 2024-02-02T15:15:33.516 INFO:teuthology.orchestra.run.smithi142.stdout:(44/145): python3-cachetools-3.1.1-4.el8.noarch 282 kB/s | 33 kB 00:00 2024-02-02T15:15:33.598 INFO:teuthology.orchestra.run.smithi071.stdout:(55/145): python3-natsort-7.1.1-2.el8.noarch.rp 418 kB/s | 60 kB 00:00 2024-02-02T15:15:33.608 INFO:teuthology.orchestra.run.smithi142.stdout:(45/145): python3-certifi-2018.10.15-7.el8.noar 139 kB/s | 16 kB 00:00 2024-02-02T15:15:33.630 INFO:teuthology.orchestra.run.smithi071.stdout:(56/145): python3-portend-2.6-1.el8.noarch.rpm 133 kB/s | 16 kB 00:00 2024-02-02T15:15:33.650 INFO:teuthology.orchestra.run.smithi142.stdout:(46/145): python3-cheroot-8.5.2-1.el8.noarch.rp 1.3 MB/s | 173 kB 00:00 2024-02-02T15:15:33.651 INFO:teuthology.orchestra.run.smithi001.stdout:(8/145): ceph-radosgw-18.2.1-325.g0fb32788.el8. 14 MB/s | 13 MB 00:00 2024-02-02T15:15:33.705 INFO:teuthology.orchestra.run.smithi071.stdout:(57/145): python3-repoze-lru-0.7-6.el8.noarch.r 317 kB/s | 34 kB 00:00 2024-02-02T15:15:33.717 INFO:teuthology.orchestra.run.smithi001.stdout:(9/145): ceph-selinux-18.2.1-325.g0fb32788.el8. 367 kB/s | 24 kB 00:00 2024-02-02T15:15:33.730 INFO:teuthology.orchestra.run.smithi071.stdout:(58/145): python3-routes-2.4.1-12.el8.noarch.rp 1.9 MB/s | 196 kB 00:00 2024-02-02T15:15:33.759 INFO:teuthology.orchestra.run.smithi142.stdout:(47/145): python3-google-auth-1.1.1-10.el8.noar 748 kB/s | 81 kB 00:00 2024-02-02T15:15:33.772 INFO:teuthology.orchestra.run.smithi071.stdout:(59/145): python3-pecan-1.3.2-9.el8.noarch.rpm 965 kB/s | 283 kB 00:00 2024-02-02T15:15:33.816 INFO:teuthology.orchestra.run.smithi142.stdout:(48/145): python3-cherrypy-18.4.0-1.el8.noarch. 1.8 MB/s | 384 kB 00:00 2024-02-02T15:15:33.831 INFO:teuthology.orchestra.run.smithi071.stdout:(60/145): python3-rsa-4.9-2.el8.noarch.rpm 511 kB/s | 64 kB 00:00 2024-02-02T15:15:33.860 INFO:teuthology.orchestra.run.smithi142.stdout:(49/145): python3-jaraco-6.2-6.el8.noarch.rpm 111 kB/s | 11 kB 00:00 2024-02-02T15:15:33.864 INFO:teuthology.orchestra.run.smithi071.stdout:(61/145): python3-simplegeneric-0.8.1-17.el8.no 140 kB/s | 19 kB 00:00 2024-02-02T15:15:33.890 INFO:teuthology.orchestra.run.smithi071.stdout:(62/145): python3-singledispatch-3.4.0.3-18.el8 204 kB/s | 24 kB 00:00 2024-02-02T15:15:33.910 INFO:teuthology.orchestra.run.smithi142.stdout:(50/145): python3-jaraco-functools-2.0-4.el8.no 187 kB/s | 18 kB 00:00 2024-02-02T15:15:33.948 INFO:teuthology.orchestra.run.smithi071.stdout:(63/145): python3-tempora-1.14.1-5.el8.noarch.r 250 kB/s | 29 kB 00:00 2024-02-02T15:15:33.974 INFO:teuthology.orchestra.run.smithi071.stdout:(64/145): python3-trustme-0.6.0-4.el8.noarch.rp 253 kB/s | 27 kB 00:00 2024-02-02T15:15:34.002 INFO:teuthology.orchestra.run.smithi142.stdout:(51/145): python3-logutils-0.3.5-11.el8.noarch. 533 kB/s | 49 kB 00:00 2024-02-02T15:15:34.090 INFO:teuthology.orchestra.run.smithi071.stdout:(65/145): python3-waitress-1.4.3-1.el8.noarch.r 1.2 MB/s | 255 kB 00:00 2024-02-02T15:15:34.094 INFO:teuthology.orchestra.run.smithi142.stdout:(52/145): python3-kubernetes-11.0.0-6.el8.noarc 5.0 MB/s | 1.2 MB 00:00 2024-02-02T15:15:34.115 INFO:teuthology.orchestra.run.smithi071.stdout:(66/145): python3-websocket-client-0.56.0-5.el8 432 kB/s | 61 kB 00:00 2024-02-02T15:15:34.119 INFO:teuthology.orchestra.run.smithi142.stdout:(53/145): python3-more-itertools-7.2.0-3.el8.no 501 kB/s | 59 kB 00:00 2024-02-02T15:15:34.141 INFO:teuthology.orchestra.run.smithi071.stdout:(67/145): python3-webob-1.8.5-1.el8.1.noarch.rp 1.3 MB/s | 251 kB 00:00 2024-02-02T15:15:34.195 INFO:teuthology.orchestra.run.smithi142.stdout:(54/145): python3-natsort-7.1.1-2.el8.noarch.rp 601 kB/s | 60 kB 00:00 2024-02-02T15:15:34.216 INFO:teuthology.orchestra.run.smithi071.stdout:(68/145): python3-webtest-2.0.33-1.el8.noarch.r 682 kB/s | 85 kB 00:00 2024-02-02T15:15:34.241 INFO:teuthology.orchestra.run.smithi071.stdout:(69/145): python3-zc-lockfile-2.0-2.el8.noarch. 183 kB/s | 23 kB 00:00 2024-02-02T15:15:34.303 INFO:teuthology.orchestra.run.smithi142.stdout:(55/145): python3-pecan-1.3.2-9.el8.noarch.rpm 1.5 MB/s | 283 kB 00:00 2024-02-02T15:15:34.308 INFO:teuthology.orchestra.run.smithi071.stdout:(70/145): libverto-devel-0.3.0-5.el8.x86_64.rpm 270 kB/s | 18 kB 00:00 2024-02-02T15:15:34.328 INFO:teuthology.orchestra.run.smithi142.stdout:(56/145): python3-portend-2.6-1.el8.noarch.rpm 117 kB/s | 16 kB 00:00 2024-02-02T15:15:34.333 INFO:teuthology.orchestra.run.smithi071.stdout:(71/145): xmlstarlet-1.6.1-20.el8.x86_64.rpm 594 kB/s | 70 kB 00:00 2024-02-02T15:15:34.359 INFO:teuthology.orchestra.run.smithi071.stdout:(72/145): python3-jwt-1.6.1-2.el8.noarch.rpm 852 kB/s | 43 kB 00:00 2024-02-02T15:15:34.401 INFO:teuthology.orchestra.run.smithi071.stdout:(73/145): thrift-0.13.0-2.el8.x86_64.rpm 6.7 MB/s | 1.7 MB 00:00 2024-02-02T15:15:34.420 INFO:teuthology.orchestra.run.smithi142.stdout:(57/145): python3-repoze-lru-0.7-6.el8.noarch.r 288 kB/s | 34 kB 00:00 2024-02-02T15:15:34.426 INFO:teuthology.orchestra.run.smithi071.stdout:(74/145): python3-pycparser-2.14-14.el8.noarch. 1.7 MB/s | 109 kB 00:00 2024-02-02T15:15:34.451 INFO:teuthology.orchestra.run.smithi071.stdout:(75/145): python3-oauthlib-2.1.0-1.el8.noarch.r 1.3 MB/s | 155 kB 00:00 2024-02-02T15:15:34.454 INFO:teuthology.orchestra.run.smithi142.stdout:(58/145): python3-routes-2.4.1-12.el8.noarch.rp 1.5 MB/s | 196 kB 00:00 2024-02-02T15:15:34.485 INFO:teuthology.orchestra.run.smithi071.stdout:(76/145): libaio-devel-0.3.112-1.el8.x86_64.rpm 353 kB/s | 19 kB 00:00 2024-02-02T15:15:34.510 INFO:teuthology.orchestra.run.smithi071.stdout:(77/145): python3-requests-oauthlib-1.0.0-1.el8 391 kB/s | 43 kB 00:00 2024-02-02T15:15:34.535 INFO:teuthology.orchestra.run.smithi071.stdout:(78/145): libzstd-devel-1.4.4-1.el8.x86_64.rpm 524 kB/s | 44 kB 00:00 2024-02-02T15:15:34.537 INFO:teuthology.orchestra.run.smithi142.stdout:(59/145): python3-rsa-4.9-2.el8.noarch.rpm 550 kB/s | 64 kB 00:00 2024-02-02T15:15:34.563 INFO:teuthology.orchestra.run.smithi142.stdout:(60/145): python3-simplegeneric-0.8.1-17.el8.no 172 kB/s | 19 kB 00:00 2024-02-02T15:15:34.585 INFO:teuthology.orchestra.run.smithi071.stdout:(79/145): pcre2-utf16-10.32-2.el8.x86_64.rpm 3.0 MB/s | 229 kB 00:00 2024-02-02T15:15:34.611 INFO:teuthology.orchestra.run.smithi071.stdout:(80/145): pcre2-utf32-10.32-2.el8.x86_64.rpm 2.9 MB/s | 220 kB 00:00 2024-02-02T15:15:34.636 INFO:teuthology.orchestra.run.smithi071.stdout:(81/145): pcre2-devel-10.32-2.el8.x86_64.rpm 3.9 MB/s | 605 kB 00:00 2024-02-02T15:15:34.646 INFO:teuthology.orchestra.run.smithi142.stdout:(61/145): python3-singledispatch-3.4.0.3-18.el8 220 kB/s | 24 kB 00:00 2024-02-02T15:15:34.661 INFO:teuthology.orchestra.run.smithi071.stdout:(82/145): keyutils-libs-devel-1.5.10-9.el8.x86_ 959 kB/s | 48 kB 00:00 2024-02-02T15:15:34.672 INFO:teuthology.orchestra.run.smithi142.stdout:(62/145): python3-tempora-1.14.1-5.el8.noarch.r 269 kB/s | 29 kB 00:00 2024-02-02T15:15:34.687 INFO:teuthology.orchestra.run.smithi071.stdout:(83/145): libselinux-devel-2.9-5.el8.x86_64.rpm 1.9 MB/s | 200 kB 00:00 2024-02-02T15:15:34.712 INFO:teuthology.orchestra.run.smithi071.stdout:(84/145): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-02-02T15:15:34.737 INFO:teuthology.orchestra.run.smithi071.stdout:(85/145): libsepol-devel-2.9-3.el8.x86_64.rpm 866 kB/s | 87 kB 00:00 2024-02-02T15:15:34.762 INFO:teuthology.orchestra.run.smithi071.stdout:(86/145): libcom_err-devel-1.45.6-4.el8.x86_64. 511 kB/s | 39 kB 00:00 2024-02-02T15:15:34.788 INFO:teuthology.orchestra.run.smithi071.stdout:(87/145): fuse-2.9.7-17.el8.x86_64.rpm 1.1 MB/s | 83 kB 00:00 2024-02-02T15:15:34.797 INFO:teuthology.orchestra.run.smithi142.stdout:(63/145): python3-trustme-0.6.0-4.el8.noarch.rp 183 kB/s | 27 kB 00:00 2024-02-02T15:15:34.813 INFO:teuthology.orchestra.run.smithi071.stdout:(88/145): libquadmath-8.5.0-20.el8.x86_64.rpm 3.4 MB/s | 172 kB 00:00 2024-02-02T15:15:34.838 INFO:teuthology.orchestra.run.smithi071.stdout:(89/145): libuuid-devel-2.32.1-43.el8.x86_64.rp 1.9 MB/s | 100 kB 00:00 2024-02-02T15:15:34.856 INFO:teuthology.orchestra.run.smithi142.stdout:(64/145): python3-waitress-1.4.3-1.el8.noarch.r 1.4 MB/s | 255 kB 00:00 2024-02-02T15:15:34.864 INFO:teuthology.orchestra.run.smithi071.stdout:(90/145): python3-cffi-1.11.5-6.el8.x86_64.rpm 4.6 MB/s | 238 kB 00:00 2024-02-02T15:15:34.897 INFO:teuthology.orchestra.run.smithi071.stdout:(91/145): libgfortran-8.5.0-20.el8.x86_64.rpm 3.9 MB/s | 645 kB 00:00 2024-02-02T15:15:34.914 INFO:teuthology.orchestra.run.smithi071.stdout:(92/145): xfsprogs-devel-5.0.0-12.el8.x86_64.rp 1.4 MB/s | 73 kB 00:00 2024-02-02T15:15:34.948 INFO:teuthology.orchestra.run.smithi071.stdout:(93/145): python3-cryptography-3.2.1-6.el8.x86_ 5.0 MB/s | 559 kB 00:00 2024-02-02T15:15:34.964 INFO:teuthology.orchestra.run.smithi142.stdout:(65/145): python3-webob-1.8.5-1.el8.1.noarch.rp 1.5 MB/s | 251 kB 00:00 2024-02-02T15:15:34.975 INFO:teuthology.orchestra.run.smithi071.stdout:(94/145): elfutils-libelf-devel-0.189-3.el8.x86 1.0 MB/s | 62 kB 00:00 2024-02-02T15:15:34.998 INFO:teuthology.orchestra.run.smithi142.stdout:(66/145): python3-websocket-client-0.56.0-5.el8 431 kB/s | 61 kB 00:00 2024-02-02T15:15:34.998 INFO:teuthology.orchestra.run.smithi071.stdout:(95/145): NetworkManager-initscripts-updown-1.4 1.4 MB/s | 144 kB 00:00 2024-02-02T15:15:35.023 INFO:teuthology.orchestra.run.smithi071.stdout:(96/145): librabbitmq-0.9.0-4.el8.x86_64.rpm 935 kB/s | 47 kB 00:00 2024-02-02T15:15:35.065 INFO:teuthology.orchestra.run.smithi142.stdout:(67/145): python3-webtest-2.0.33-1.el8.noarch.r 852 kB/s | 85 kB 00:00 2024-02-02T15:15:35.082 INFO:teuthology.orchestra.run.smithi071.stdout:(97/145): krb5-devel-1.18.2-26.el8_9.x86_64.rpm 4.1 MB/s | 562 kB 00:00 2024-02-02T15:15:35.108 INFO:teuthology.orchestra.run.smithi071.stdout:(98/145): python3-jsonpointer-1.10-11.el8.noarc 239 kB/s | 20 kB 00:00 2024-02-02T15:15:35.115 INFO:teuthology.orchestra.run.smithi142.stdout:(68/145): python3-zc-lockfile-2.0-2.el8.noarch. 196 kB/s | 23 kB 00:00 2024-02-02T15:15:35.133 INFO:teuthology.orchestra.run.smithi071.stdout:(99/145): python3-prettytable-0.7.2-14.el8.noar 877 kB/s | 44 kB 00:00 2024-02-02T15:15:35.158 INFO:teuthology.orchestra.run.smithi071.stdout:(100/145): python3-pyasn1-modules-0.3.7-6.el8.n 2.2 MB/s | 110 kB 00:00 2024-02-02T15:15:35.170 INFO:teuthology.orchestra.run.smithi142.stdout:(69/145): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.2 MB/s | 70 kB 00:00 2024-02-02T15:15:35.183 INFO:teuthology.orchestra.run.smithi071.stdout:(101/145): python3-jsonpatch-1.21-2.el8.noarch. 529 kB/s | 27 kB 00:00 2024-02-02T15:15:35.208 INFO:teuthology.orchestra.run.smithi071.stdout:(102/145): python3-pyasn1-0.3.7-6.el8.noarch.rp 2.5 MB/s | 126 kB 00:00 2024-02-02T15:15:35.232 INFO:teuthology.orchestra.run.smithi142.stdout:(70/145): libverto-devel-0.3.0-5.el8.x86_64.rpm 288 kB/s | 18 kB 00:00 2024-02-02T15:15:35.242 INFO:teuthology.orchestra.run.smithi071.stdout:(103/145): python3-werkzeug-0.12.2-4.el8.noarch 7.7 MB/s | 457 kB 00:00 2024-02-02T15:15:35.274 INFO:teuthology.orchestra.run.smithi142.stdout:(71/145): thrift-0.13.0-2.el8.x86_64.rpm 8.3 MB/s | 1.7 MB 00:00 2024-02-02T15:15:35.300 INFO:teuthology.orchestra.run.smithi142.stdout:(72/145): python3-jwt-1.6.1-2.el8.noarch.rpm 636 kB/s | 43 kB 00:00 2024-02-02T15:15:35.342 INFO:teuthology.orchestra.run.smithi142.stdout:(73/145): python3-pycparser-2.14-14.el8.noarch. 3.0 MB/s | 109 kB 00:00 2024-02-02T15:15:35.391 INFO:teuthology.orchestra.run.smithi142.stdout:(74/145): python3-oauthlib-2.1.0-1.el8.noarch.r 1.3 MB/s | 155 kB 00:00 2024-02-02T15:15:35.426 INFO:teuthology.orchestra.run.smithi142.stdout:(75/145): python3-requests-oauthlib-1.0.0-1.el8 511 kB/s | 43 kB 00:00 2024-02-02T15:15:35.451 INFO:teuthology.orchestra.run.smithi142.stdout:(76/145): libaio-devel-0.3.112-1.el8.x86_64.rpm 314 kB/s | 19 kB 00:00 2024-02-02T15:15:35.476 INFO:teuthology.orchestra.run.smithi071.stdout:(104/145): bison-3.0.4-10.el8.x86_64.rpm 2.5 MB/s | 688 kB 00:00 2024-02-02T15:15:35.476 INFO:teuthology.orchestra.run.smithi142.stdout:(77/145): libzstd-devel-1.4.4-1.el8.x86_64.rpm 875 kB/s | 44 kB 00:00 2024-02-02T15:15:35.510 INFO:teuthology.orchestra.run.smithi142.stdout:(78/145): pcre2-devel-10.32-2.el8.x86_64.rpm 10 MB/s | 605 kB 00:00 2024-02-02T15:15:35.526 INFO:teuthology.orchestra.run.smithi142.stdout:(79/145): pcre2-utf16-10.32-2.el8.x86_64.rpm 4.5 MB/s | 229 kB 00:00 2024-02-02T15:15:35.552 INFO:teuthology.orchestra.run.smithi142.stdout:(80/145): pcre2-utf32-10.32-2.el8.x86_64.rpm 5.1 MB/s | 220 kB 00:00 2024-02-02T15:15:35.577 INFO:teuthology.orchestra.run.smithi142.stdout:(81/145): libselinux-devel-2.9-5.el8.x86_64.rpm 3.9 MB/s | 200 kB 00:00 2024-02-02T15:15:35.629 INFO:teuthology.orchestra.run.smithi142.stdout:(82/145): keyutils-libs-devel-1.5.10-9.el8.x86_ 628 kB/s | 48 kB 00:00 2024-02-02T15:15:35.651 INFO:teuthology.orchestra.run.smithi071.stdout:(105/145): flex-2.6.1-9.el8.x86_64.rpm 782 kB/s | 320 kB 00:00 2024-02-02T15:15:35.661 INFO:teuthology.orchestra.run.smithi142.stdout:(83/145): libsepol-devel-2.9-3.el8.x86_64.rpm 1.0 MB/s | 87 kB 00:00 2024-02-02T15:15:35.686 INFO:teuthology.orchestra.run.smithi142.stdout:(84/145): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.0 MB/s | 58 kB 00:00 2024-02-02T15:15:35.701 INFO:teuthology.orchestra.run.smithi071.stdout:(106/145): openssl-devel-1.1.1k-12.el8_9.x86_64 3.3 MB/s | 2.3 MB 00:00 2024-02-02T15:15:35.712 INFO:teuthology.orchestra.run.smithi142.stdout:(85/145): libcom_err-devel-1.45.6-4.el8.x86_64. 768 kB/s | 39 kB 00:00 2024-02-02T15:15:35.727 INFO:teuthology.orchestra.run.smithi071.stdout:(107/145): python3-markupsafe-0.23-19.el8.x86_6 516 kB/s | 39 kB 00:00 2024-02-02T15:15:35.737 INFO:teuthology.orchestra.run.smithi142.stdout:(86/145): fuse-2.9.7-17.el8.x86_64.rpm 1.6 MB/s | 83 kB 00:00 2024-02-02T15:15:35.760 INFO:teuthology.orchestra.run.smithi071.stdout:(108/145): libtool-2.4.6-25.el8.x86_64.rpm 2.4 MB/s | 709 kB 00:00 2024-02-02T15:15:35.771 INFO:teuthology.orchestra.run.smithi142.stdout:(87/145): libgfortran-8.5.0-20.el8.x86_64.rpm 11 MB/s | 645 kB 00:00 2024-02-02T15:15:35.785 INFO:teuthology.orchestra.run.smithi071.stdout:(109/145): lttng-ust-2.8.1-11.el8.x86_64.rpm 3.0 MB/s | 259 kB 00:00 2024-02-02T15:15:35.796 INFO:teuthology.orchestra.run.smithi142.stdout:(88/145): libquadmath-8.5.0-20.el8.x86_64.rpm 2.9 MB/s | 172 kB 00:00 2024-02-02T15:15:35.810 INFO:teuthology.orchestra.run.smithi071.stdout:(110/145): uuid-1.6.2-43.el8.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-02-02T15:15:35.821 INFO:teuthology.orchestra.run.smithi142.stdout:(89/145): libuuid-devel-2.32.1-43.el8.x86_64.rp 1.9 MB/s | 100 kB 00:00 2024-02-02T15:15:35.836 INFO:teuthology.orchestra.run.smithi071.stdout:(111/145): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.0 MB/s | 103 kB 00:00 2024-02-02T15:15:35.861 INFO:teuthology.orchestra.run.smithi071.stdout:(112/145): oniguruma-6.8.2-2.el8.x86_64.rpm 1.4 MB/s | 187 kB 00:00 2024-02-02T15:15:35.880 INFO:teuthology.orchestra.run.smithi142.stdout:(90/145): python3-cffi-1.11.5-6.el8.x86_64.rpm 2.8 MB/s | 238 kB 00:00 2024-02-02T15:15:35.895 INFO:teuthology.orchestra.run.smithi071.stdout:(113/145): python3-jinja2-2.10.1-3.el8.noarch.r 9.0 MB/s | 538 kB 00:00 2024-02-02T15:15:35.914 INFO:teuthology.orchestra.run.smithi142.stdout:(91/145): python3-cryptography-3.2.1-6.el8.x86_ 5.9 MB/s | 559 kB 00:00 2024-02-02T15:15:35.939 INFO:teuthology.orchestra.run.smithi142.stdout:(92/145): xfsprogs-devel-5.0.0-12.el8.x86_64.rp 1.2 MB/s | 73 kB 00:00 2024-02-02T15:15:35.954 INFO:teuthology.orchestra.run.smithi071.stdout:(114/145): socat-1.7.4.1-1.el8.x86_64.rpm 5.4 MB/s | 323 kB 00:00 2024-02-02T15:15:35.964 INFO:teuthology.orchestra.run.smithi142.stdout:(93/145): NetworkManager-initscripts-updown-1.4 2.8 MB/s | 144 kB 00:00 2024-02-02T15:15:35.989 INFO:teuthology.orchestra.run.smithi142.stdout:(94/145): elfutils-libelf-devel-0.189-3.el8.x86 1.2 MB/s | 62 kB 00:00 2024-02-02T15:15:36.023 INFO:teuthology.orchestra.run.smithi142.stdout:(95/145): krb5-devel-1.18.2-26.el8_9.x86_64.rpm 9.4 MB/s | 562 kB 00:00 2024-02-02T15:15:36.048 INFO:teuthology.orchestra.run.smithi142.stdout:(96/145): librabbitmq-0.9.0-4.el8.x86_64.rpm 803 kB/s | 47 kB 00:00 2024-02-02T15:15:36.061 INFO:teuthology.orchestra.run.smithi071.stdout:(115/145): python3-numpy-1.14.3-10.el8.x86_64.r 19 MB/s | 3.7 MB 00:00 2024-02-02T15:15:36.096 INFO:teuthology.orchestra.run.smithi071.stdout:(116/145): python3-numpy-f2py-1.14.3-10.el8.x86 1.6 MB/s | 225 kB 00:00 2024-02-02T15:15:36.107 INFO:teuthology.orchestra.run.smithi142.stdout:(97/145): python3-jsonpointer-1.10-11.el8.noarc 344 kB/s | 20 kB 00:00 2024-02-02T15:15:36.129 INFO:teuthology.orchestra.run.smithi071.stdout:(117/145): librdkafka-0.11.4-3.el8.x86_64.rpm 5.1 MB/s | 354 kB 00:00 2024-02-02T15:15:36.149 INFO:teuthology.orchestra.run.smithi142.stdout:(98/145): python3-prettytable-0.7.2-14.el8.noar 1.0 MB/s | 44 kB 00:00 2024-02-02T15:15:36.188 INFO:teuthology.orchestra.run.smithi071.stdout:(118/145): python3-babel-2.5.1-7.el8.noarch.rpm 13 MB/s | 4.8 MB 00:00 2024-02-02T15:15:36.199 INFO:teuthology.orchestra.run.smithi142.stdout:(99/145): openssl-devel-1.1.1k-12.el8_9.x86_64. 13 MB/s | 2.3 MB 00:00 2024-02-02T15:15:36.213 INFO:teuthology.orchestra.run.smithi071.stdout:(119/145): python3-mako-1.0.6-14.el8.noarch.rpm 1.8 MB/s | 157 kB 00:00 2024-02-02T15:15:36.216 INFO:teuthology.orchestra.run.smithi142.stdout:(100/145): python3-pyasn1-modules-0.3.7-6.el8.n 1.6 MB/s | 110 kB 00:00 2024-02-02T15:15:36.250 INFO:teuthology.orchestra.run.smithi142.stdout:(101/145): python3-jsonpatch-1.21-2.el8.noarch. 530 kB/s | 27 kB 00:00 2024-02-02T15:15:36.275 INFO:teuthology.orchestra.run.smithi142.stdout:(102/145): python3-pyasn1-0.3.7-6.el8.noarch.rp 2.1 MB/s | 126 kB 00:00 2024-02-02T15:15:36.280 INFO:teuthology.orchestra.run.smithi071.stdout:(120/145): automake-1.16.1-8.el8.noarch.rpm 10 MB/s | 714 kB 00:00 2024-02-02T15:15:36.372 INFO:teuthology.orchestra.run.smithi071.stdout:(121/145): openblas-threads-0.3.15-6.el8.x86_64 19 MB/s | 5.2 MB 00:00 2024-02-02T15:15:36.397 INFO:teuthology.orchestra.run.smithi071.stdout:(122/145): libpq-13.11-1.el8.x86_64.rpm 1.7 MB/s | 198 kB 00:00 2024-02-02T15:15:36.417 INFO:teuthology.orchestra.run.smithi142.stdout:(103/145): python3-werkzeug-0.12.2-4.el8.noarch 2.7 MB/s | 457 kB 00:00 2024-02-02T15:15:36.422 INFO:teuthology.orchestra.run.smithi071.stdout:(123/145): jq-1.6-7.el8.x86_64.rpm 4.0 MB/s | 202 kB 00:00 2024-02-02T15:15:36.448 INFO:teuthology.orchestra.run.smithi071.stdout:(124/145): python3-pytz-2017.2-11.el8.noarch.rp 1.1 MB/s | 54 kB 00:00 2024-02-02T15:15:36.459 INFO:teuthology.orchestra.run.smithi001.stdout:(10/145): ceph-test-18.2.1-325.g0fb32788.el8.x8 17 MB/s | 47 MB 00:02 2024-02-02T15:15:36.515 INFO:teuthology.orchestra.run.smithi071.stdout:(125/145): openblas-0.3.15-6.el8.x86_64.rpm 15 MB/s | 5.0 MB 00:00 2024-02-02T15:15:36.526 INFO:teuthology.orchestra.run.smithi001.stdout:(11/145): cephfs-mirror-18.2.1-325.g0fb32788.el 3.8 MB/s | 258 kB 00:00 2024-02-02T15:15:36.565 INFO:teuthology.orchestra.run.smithi071.stdout:(126/145): postgresql-contrib-10.23-3.module+el 6.8 MB/s | 811 kB 00:00 2024-02-02T15:15:36.593 INFO:teuthology.orchestra.run.smithi001.stdout:(12/145): libcephfs-devel-18.2.1-325.g0fb32788. 466 kB/s | 31 kB 00:00 2024-02-02T15:15:36.607 INFO:teuthology.orchestra.run.smithi071.stdout:(127/145): postgresql-10.23-3.module+el8.9.0+20 8.2 MB/s | 1.5 MB 00:00 2024-02-02T15:15:36.634 INFO:teuthology.orchestra.run.smithi142.stdout:(104/145): bison-3.0.4-10.el8.x86_64.rpm 1.9 MB/s | 688 kB 00:00 2024-02-02T15:15:36.668 INFO:teuthology.orchestra.run.smithi142.stdout:(105/145): flex-2.6.1-9.el8.x86_64.rpm 1.2 MB/s | 320 kB 00:00 2024-02-02T15:15:36.685 INFO:teuthology.orchestra.run.smithi001.stdout:(13/145): libcephfs2-18.2.1-325.g0fb32788.el8.x 8.3 MB/s | 775 kB 00:00 2024-02-02T15:15:36.701 INFO:teuthology.orchestra.run.smithi142.stdout:(106/145): libtool-2.4.6-25.el8.x86_64.rpm 10 MB/s | 709 kB 00:00 2024-02-02T15:15:36.727 INFO:teuthology.orchestra.run.smithi142.stdout:(107/145): python3-markupsafe-0.23-19.el8.x86_6 662 kB/s | 39 kB 00:00 2024-02-02T15:15:36.741 INFO:teuthology.orchestra.run.smithi071.stdout:(128/145): postgresql-server-10.23-3.module+el8 22 MB/s | 5.1 MB 00:00 2024-02-02T15:15:36.743 INFO:teuthology.orchestra.run.smithi001.stdout:(14/145): libcephsqlite-18.2.1-325.g0fb32788.el 2.4 MB/s | 141 kB 00:00 2024-02-02T15:15:36.785 INFO:teuthology.orchestra.run.smithi142.stdout:(108/145): lttng-ust-2.8.1-11.el8.x86_64.rpm 3.0 MB/s | 259 kB 00:00 2024-02-02T15:15:36.827 INFO:teuthology.orchestra.run.smithi142.stdout:(109/145): oniguruma-6.8.2-2.el8.x86_64.rpm 1.8 MB/s | 187 kB 00:00 2024-02-02T15:15:36.853 INFO:teuthology.orchestra.run.smithi142.stdout:(110/145): uuid-1.6.2-43.el8.x86_64.rpm 954 kB/s | 64 kB 00:00 2024-02-02T15:15:36.878 INFO:teuthology.orchestra.run.smithi142.stdout:(111/145): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.0 MB/s | 103 kB 00:00 2024-02-02T15:15:36.927 INFO:teuthology.orchestra.run.smithi001.stdout:(15/145): librados-devel-18.2.1-325.g0fb32788.e 759 kB/s | 139 kB 00:00 2024-02-02T15:15:36.928 INFO:teuthology.orchestra.run.smithi142.stdout:(112/145): python3-jinja2-2.10.1-3.el8.noarch.r 11 MB/s | 538 kB 00:00 2024-02-02T15:15:36.949 INFO:teuthology.orchestra.run.smithi071.stdout:(129/145): python3-scipy-1.0.0-21.module+el8.9. 36 MB/s | 14 MB 00:00 2024-02-02T15:15:36.999 INFO:teuthology.orchestra.run.smithi071.stdout:(130/145): elfutils-libelf-0.189-3.el8.x86_64.r 4.6 MB/s | 232 kB 00:00 2024-02-02T15:15:37.044 INFO:teuthology.orchestra.run.smithi001.stdout:(16/145): libradosstriper1-18.2.1-325.g0fb32788 4.0 MB/s | 474 kB 00:00 2024-02-02T15:15:37.053 INFO:teuthology.orchestra.run.smithi142.stdout:(113/145): python3-babel-2.5.1-7.el8.noarch.rpm 24 MB/s | 4.8 MB 00:00 2024-02-02T15:15:37.058 INFO:teuthology.orchestra.run.smithi071.stdout:(131/145): elfutils-libs-0.189-3.el8.x86_64.rpm 5.1 MB/s | 303 kB 00:00 2024-02-02T15:15:37.096 INFO:teuthology.orchestra.run.smithi142.stdout:(114/145): socat-1.7.4.1-1.el8.x86_64.rpm 7.6 MB/s | 323 kB 00:00 2024-02-02T15:15:37.108 INFO:teuthology.orchestra.run.smithi071.stdout:(132/145): libmount-2.32.1-43.el8.x86_64.rpm 4.6 MB/s | 237 kB 00:00 2024-02-02T15:15:37.146 INFO:teuthology.orchestra.run.smithi142.stdout:(115/145): python3-numpy-1.14.3-10.el8.x86_64.r 17 MB/s | 3.7 MB 00:00 2024-02-02T15:15:37.150 INFO:teuthology.orchestra.run.smithi071.stdout:(133/145): libsmartcols-2.32.1-43.el8.x86_64.rp 4.2 MB/s | 179 kB 00:00 2024-02-02T15:15:37.171 INFO:teuthology.orchestra.run.smithi142.stdout:(116/145): python3-numpy-f2py-1.14.3-10.el8.x86 2.9 MB/s | 225 kB 00:00 2024-02-02T15:15:37.196 INFO:teuthology.orchestra.run.smithi142.stdout:(117/145): librdkafka-0.11.4-3.el8.x86_64.rpm 6.9 MB/s | 354 kB 00:00 2024-02-02T15:15:37.200 INFO:teuthology.orchestra.run.smithi071.stdout:(134/145): libuuid-2.32.1-43.el8.x86_64.rpm 2.0 MB/s | 99 kB 00:00 2024-02-02T15:15:37.255 INFO:teuthology.orchestra.run.smithi142.stdout:(118/145): python3-mako-1.0.6-14.el8.noarch.rpm 2.6 MB/s | 157 kB 00:00 2024-02-02T15:15:37.301 INFO:teuthology.orchestra.run.smithi071.stdout:(135/145): util-linux-2.32.1-43.el8.x86_64.rpm 25 MB/s | 2.5 MB 00:00 2024-02-02T15:15:37.351 INFO:teuthology.orchestra.run.smithi071.stdout:(136/145): util-linux-user-2.32.1-43.el8.x86_64 2.0 MB/s | 103 kB 00:00 2024-02-02T15:15:37.364 INFO:teuthology.orchestra.run.smithi142.stdout:(119/145): openblas-threads-0.3.15-6.el8.x86_64 27 MB/s | 5.2 MB 00:00 2024-02-02T15:15:37.409 INFO:teuthology.orchestra.run.smithi071.stdout:(137/145): iproute-6.2.0-5.el8_9.x86_64.rpm 14 MB/s | 854 kB 00:00 2024-02-02T15:15:37.423 INFO:teuthology.orchestra.run.smithi142.stdout:(120/145): automake-1.16.1-8.el8.noarch.rpm 12 MB/s | 714 kB 00:00 2024-02-02T15:15:37.460 INFO:teuthology.orchestra.run.smithi071.stdout:(138/145): libblkid-2.32.1-43.el8.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2024-02-02T15:15:37.514 INFO:teuthology.orchestra.run.smithi142.stdout:(121/145): openblas-0.3.15-6.el8.x86_64.rpm 19 MB/s | 5.0 MB 00:00 2024-02-02T15:15:37.535 INFO:teuthology.orchestra.run.smithi071.stdout:(139/145): libfdisk-2.32.1-43.el8.x86_64.rpm 3.3 MB/s | 254 kB 00:00 2024-02-02T15:15:37.540 INFO:teuthology.orchestra.run.smithi142.stdout:(122/145): libpq-13.11-1.el8.x86_64.rpm 1.7 MB/s | 198 kB 00:00 2024-02-02T15:15:37.544 INFO:teuthology.orchestra.run.smithi001.stdout:(17/145): librgw2-18.2.1-325.g0fb32788.el8.x86_ 13 MB/s | 6.5 MB 00:00 2024-02-02T15:15:37.564 INFO:teuthology.orchestra.run.smithi142.stdout:(123/145): jq-1.6-7.el8.x86_64.rpm 4.0 MB/s | 202 kB 00:00 2024-02-02T15:15:37.582 INFO:teuthology.orchestra.run.smithi142.stdout:(124/145): python3-pytz-2017.2-11.el8.noarch.rp 1.3 MB/s | 54 kB 00:00 2024-02-02T15:15:37.593 INFO:teuthology.orchestra.run.smithi071.stdout:(140/145): openssl-1.1.1k-12.el8_9.x86_64.rpm 12 MB/s | 711 kB 00:00 2024-02-02T15:15:37.611 INFO:teuthology.orchestra.run.smithi001.stdout:(18/145): python3-ceph-argparse-18.2.1-325.g0fb 672 kB/s | 45 kB 00:00 2024-02-02T15:15:37.632 INFO:teuthology.orchestra.run.smithi142.stdout:(125/145): postgresql-10.23-3.module+el8.9.0+20 22 MB/s | 1.5 MB 00:00 2024-02-02T15:15:37.669 INFO:teuthology.orchestra.run.smithi071.stdout:(141/145): openssl-libs-1.1.1k-12.el8_9.x86_64. 20 MB/s | 1.5 MB 00:00 2024-02-02T15:15:37.678 INFO:teuthology.orchestra.run.smithi001.stdout:(19/145): python3-ceph-common-18.2.1-325.g0fb32 1.8 MB/s | 123 kB 00:00 2024-02-02T15:15:37.693 INFO:teuthology.orchestra.run.smithi142.stdout:(126/145): postgresql-contrib-10.23-3.module+el 7.1 MB/s | 811 kB 00:00 2024-02-02T15:15:37.719 INFO:teuthology.orchestra.run.smithi071.stdout:(142/145): xfsdump-3.1.8-7.el8_9.x86_64.rpm 6.5 MB/s | 333 kB 00:00 2024-02-02T15:15:37.753 INFO:teuthology.orchestra.run.smithi001.stdout:(20/145): python3-cephfs-18.2.1-325.g0fb32788.e 2.6 MB/s | 197 kB 00:00 2024-02-02T15:15:37.778 INFO:teuthology.orchestra.run.smithi071.stdout:(143/145): librbd1-18.2.1-325.g0fb32788.el8.x86 3.4 MB/s | 3.6 MB 00:01 2024-02-02T15:15:37.820 INFO:teuthology.orchestra.run.smithi071.stdout:(144/145): xfsprogs-5.0.0-12.el8.x86_64.rpm 11 MB/s | 1.1 MB 00:00 2024-02-02T15:15:37.824 INFO:teuthology.orchestra.run.smithi142.stdout:(127/145): postgresql-server-10.23-3.module+el8 26 MB/s | 5.1 MB 00:00 2024-02-02T15:15:37.829 INFO:teuthology.orchestra.run.smithi001.stdout:(21/145): python3-rados-18.2.1-325.g0fb32788.el 4.8 MB/s | 371 kB 00:00 2024-02-02T15:15:37.904 INFO:teuthology.orchestra.run.smithi001.stdout:(22/145): python3-rbd-18.2.1-325.g0fb32788.el8. 4.8 MB/s | 365 kB 00:00 2024-02-02T15:15:37.979 INFO:teuthology.orchestra.run.smithi001.stdout:(23/145): python3-rgw-18.2.1-325.g0fb32788.el8. 1.4 MB/s | 107 kB 00:00 2024-02-02T15:15:38.038 INFO:teuthology.orchestra.run.smithi001.stdout:(24/145): rbd-fuse-18.2.1-325.g0fb32788.el8.x86 1.6 MB/s | 94 kB 00:00 2024-02-02T15:15:38.050 INFO:teuthology.orchestra.run.smithi142.stdout:(128/145): python3-scipy-1.0.0-21.module+el8.9. 39 MB/s | 14 MB 00:00 2024-02-02T15:15:38.253 INFO:teuthology.orchestra.run.smithi071.stdout:(145/145): librados2-18.2.1-325.g0fb32788.el8.x 2.2 MB/s | 3.7 MB 00:01 2024-02-02T15:15:38.256 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-02-02T15:15:38.256 INFO:teuthology.orchestra.run.smithi071.stdout:Total 14 MB/s | 212 MB 00:15 2024-02-02T15:15:38.413 INFO:teuthology.orchestra.run.smithi001.stdout:(25/145): rbd-mirror-18.2.1-325.g0fb32788.el8.x 9.2 MB/s | 3.4 MB 00:00 2024-02-02T15:15:38.656 INFO:teuthology.orchestra.run.smithi001.stdout:(26/145): ceph-common-18.2.1-325.g0fb32788.el8. 2.2 MB/s | 22 MB 00:09 2024-02-02T15:15:38.666 INFO:teuthology.orchestra.run.smithi142.stdout:(129/145): ceph-test-18.2.1-325.g0fb32788.el8.x 4.7 MB/s | 47 MB 00:09 2024-02-02T15:15:38.681 INFO:teuthology.orchestra.run.smithi001.stdout:(27/145): rbd-nbd-18.2.1-325.g0fb32788.el8.x86_ 678 kB/s | 181 kB 00:00 2024-02-02T15:15:38.698 INFO:teuthology.orchestra.run.smithi001.stdout:(28/145): ceph-grafana-dashboards-18.2.1-325.g0 558 kB/s | 23 kB 00:00 2024-02-02T15:15:38.708 INFO:teuthology.orchestra.run.smithi142.stdout:(130/145): elfutils-libelf-0.189-3.el8.x86_64.r 5.5 MB/s | 232 kB 00:00 2024-02-02T15:15:38.748 INFO:teuthology.orchestra.run.smithi001.stdout:(29/145): ceph-mgr-cephadm-18.2.1-325.g0fb32788 1.9 MB/s | 132 kB 00:00 2024-02-02T15:15:38.750 INFO:teuthology.orchestra.run.smithi142.stdout:(131/145): elfutils-libs-0.189-3.el8.x86_64.rpm 7.2 MB/s | 303 kB 00:00 2024-02-02T15:15:38.792 INFO:teuthology.orchestra.run.smithi142.stdout:(132/145): libmount-2.32.1-43.el8.x86_64.rpm 5.6 MB/s | 237 kB 00:00 2024-02-02T15:15:38.859 INFO:teuthology.orchestra.run.smithi142.stdout:(133/145): libsmartcols-2.32.1-43.el8.x86_64.rp 2.6 MB/s | 179 kB 00:00 2024-02-02T15:15:38.926 INFO:teuthology.orchestra.run.smithi142.stdout:(134/145): librbd1-18.2.1-325.g0fb32788.el8.x86 4.1 MB/s | 3.6 MB 00:00 2024-02-02T15:15:38.951 INFO:teuthology.orchestra.run.smithi142.stdout:(135/145): libuuid-2.32.1-43.el8.x86_64.rpm 1.1 MB/s | 99 kB 00:00 2024-02-02T15:15:38.993 INFO:teuthology.orchestra.run.smithi142.stdout:(136/145): util-linux-user-2.32.1-43.el8.x86_64 2.4 MB/s | 103 kB 00:00 2024-02-02T15:15:39.083 INFO:teuthology.orchestra.run.smithi142.stdout:(137/145): util-linux-2.32.1-43.el8.x86_64.rpm 16 MB/s | 2.5 MB 00:00 2024-02-02T15:15:39.128 INFO:teuthology.orchestra.run.smithi142.stdout:(138/145): iproute-6.2.0-5.el8_9.x86_64.rpm 6.2 MB/s | 854 kB 00:00 2024-02-02T15:15:39.140 INFO:teuthology.orchestra.run.smithi001.stdout:(30/145): ceph-mgr-dashboard-18.2.1-325.g0fb327 4.0 MB/s | 1.8 MB 00:00 2024-02-02T15:15:39.153 INFO:teuthology.orchestra.run.smithi142.stdout:(139/145): libblkid-2.32.1-43.el8.x86_64.rpm 3.1 MB/s | 220 kB 00:00 2024-02-02T15:15:39.211 INFO:teuthology.orchestra.run.smithi142.stdout:(140/145): librados2-18.2.1-325.g0fb32788.el8.x 2.6 MB/s | 3.7 MB 00:01 2024-02-02T15:15:39.224 INFO:teuthology.orchestra.run.smithi001.stdout:(31/145): ceph-mgr-modules-core-18.2.1-325.g0fb 2.8 MB/s | 242 kB 00:00 2024-02-02T15:15:39.236 INFO:teuthology.orchestra.run.smithi142.stdout:(141/145): libfdisk-2.32.1-43.el8.x86_64.rpm 2.3 MB/s | 254 kB 00:00 2024-02-02T15:15:39.271 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-02-02T15:15:39.312 INFO:teuthology.orchestra.run.smithi142.stdout:(142/145): openssl-1.1.1k-12.el8_9.x86_64.rpm 4.4 MB/s | 711 kB 00:00 2024-02-02T15:15:39.316 INFO:teuthology.orchestra.run.smithi001.stdout:(32/145): ceph-mgr-diskprediction-local-18.2.1- 13 MB/s | 7.4 MB 00:00 2024-02-02T15:15:39.345 INFO:teuthology.orchestra.run.smithi142.stdout:(143/145): xfsdump-3.1.8-7.el8_9.x86_64.rpm 3.2 MB/s | 333 kB 00:00 2024-02-02T15:15:39.374 INFO:teuthology.orchestra.run.smithi001.stdout:(33/145): ceph-mgr-rook-18.2.1-325.g0fb32788.el 335 kB/s | 50 kB 00:00 2024-02-02T15:15:39.395 INFO:teuthology.orchestra.run.smithi142.stdout:(144/145): xfsprogs-5.0.0-12.el8.x86_64.rpm 13 MB/s | 1.1 MB 00:00 2024-02-02T15:15:39.399 INFO:teuthology.orchestra.run.smithi001.stdout:(34/145): ceph-prometheus-alerts-18.2.1-325.g0f 174 kB/s | 15 kB 00:00 2024-02-02T15:15:39.429 INFO:teuthology.orchestra.run.smithi142.stdout:(145/145): openssl-libs-1.1.1k-12.el8_9.x86_64. 6.8 MB/s | 1.5 MB 00:00 2024-02-02T15:15:39.432 INFO:teuthology.orchestra.run.smithi142.stdout:-------------------------------------------------------------------------------- 2024-02-02T15:15:39.432 INFO:teuthology.orchestra.run.smithi142.stdout:Total 14 MB/s | 212 MB 00:15 2024-02-02T15:15:39.441 INFO:teuthology.orchestra.run.smithi001.stdout:(35/145): ceph-volume-18.2.1-325.g0fb32788.el8. 3.8 MB/s | 262 kB 00:00 2024-02-02T15:15:39.467 INFO:teuthology.orchestra.run.smithi001.stdout:(36/145): cephadm-18.2.1-325.g0fb32788.el8.noar 2.5 MB/s | 168 kB 00:00 2024-02-02T15:15:39.509 INFO:teuthology.orchestra.run.smithi001.stdout:(37/145): cephfs-top-18.2.1-325.g0fb32788.el8.n 269 kB/s | 18 kB 00:00 2024-02-02T15:15:39.664 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-02-02T15:15:39.664 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-02-02T15:15:39.868 INFO:teuthology.orchestra.run.smithi001.stdout:(38/145): ceph-osd-18.2.1-325.g0fb32788.el8.x86 2.3 MB/s | 18 MB 00:07 2024-02-02T15:15:39.893 INFO:teuthology.orchestra.run.smithi001.stdout:(39/145): liboath-2.6.2-3.el8.x86_64.rpm 153 kB/s | 59 kB 00:00 2024-02-02T15:15:40.160 INFO:teuthology.orchestra.run.smithi001.stdout:(40/145): python3-asyncssh-2.7.0-2.el8.noarch.r 1.7 MB/s | 476 kB 00:00 2024-02-02T15:15:40.185 INFO:teuthology.orchestra.run.smithi001.stdout:(41/145): libunwind-1.3.1-3.el8.x86_64.rpm 237 kB/s | 75 kB 00:00 2024-02-02T15:15:40.227 INFO:teuthology.orchestra.run.smithi001.stdout:(42/145): gperftools-libs-2.7-9.el8.x86_64.rpm 403 kB/s | 306 kB 00:00 2024-02-02T15:15:40.252 INFO:teuthology.orchestra.run.smithi001.stdout:(43/145): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 474 kB/s | 44 kB 00:00 2024-02-02T15:15:40.344 INFO:teuthology.orchestra.run.smithi001.stdout:(44/145): python3-certifi-2018.10.15-7.el8.noar 178 kB/s | 16 kB 00:00 2024-02-02T15:15:40.369 INFO:teuthology.orchestra.run.smithi001.stdout:(45/145): python3-cachetools-3.1.1-4.el8.noarch 232 kB/s | 33 kB 00:00 2024-02-02T15:15:40.394 INFO:teuthology.orchestra.run.smithi001.stdout:(46/145): python3-beautifulsoup4-4.6.3-2.el8.1. 885 kB/s | 185 kB 00:00 2024-02-02T15:15:40.428 INFO:teuthology.orchestra.run.smithi001.stdout:(47/145): python3-cheroot-8.5.2-1.el8.noarch.rp 2.0 MB/s | 173 kB 00:00 2024-02-02T15:15:40.472 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction check 2024-02-02T15:15:40.478 INFO:teuthology.orchestra.run.smithi001.stdout:(48/145): python3-google-auth-1.1.1-10.el8.noar 973 kB/s | 81 kB 00:00 2024-02-02T15:15:40.504 INFO:teuthology.orchestra.run.smithi001.stdout:(49/145): python3-jaraco-6.2-6.el8.noarch.rpm 147 kB/s | 11 kB 00:00 2024-02-02T15:15:40.554 INFO:teuthology.orchestra.run.smithi001.stdout:(50/145): python3-jaraco-functools-2.0-4.el8.no 234 kB/s | 18 kB 00:00 2024-02-02T15:15:40.638 INFO:teuthology.orchestra.run.smithi001.stdout:(51/145): python3-logutils-0.3.5-11.el8.noarch. 585 kB/s | 49 kB 00:00 2024-02-02T15:15:40.705 INFO:teuthology.orchestra.run.smithi001.stdout:(52/145): python3-cherrypy-18.4.0-1.el8.noarch. 1.1 MB/s | 384 kB 00:00 2024-02-02T15:15:40.730 INFO:teuthology.orchestra.run.smithi001.stdout:(53/145): python3-more-itertools-7.2.0-3.el8.no 642 kB/s | 59 kB 00:00 2024-02-02T15:15:40.789 INFO:teuthology.orchestra.run.smithi001.stdout:(54/145): python3-kubernetes-11.0.0-6.el8.noarc 4.1 MB/s | 1.2 MB 00:00 2024-02-02T15:15:40.822 INFO:teuthology.orchestra.run.smithi001.stdout:(55/145): python3-natsort-7.1.1-2.el8.noarch.rp 515 kB/s | 60 kB 00:00 2024-02-02T15:15:40.847 INFO:teuthology.orchestra.run.smithi001.stdout:(56/145): python3-pecan-1.3.2-9.el8.noarch.rpm 2.4 MB/s | 283 kB 00:00 2024-02-02T15:15:40.851 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-02-02T15:15:40.861 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-02-02T15:15:40.866 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction check succeeded. 2024-02-02T15:15:40.866 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction test 2024-02-02T15:15:40.873 INFO:teuthology.orchestra.run.smithi001.stdout:(57/145): python3-portend-2.6-1.el8.noarch.rpm 188 kB/s | 16 kB 00:00 2024-02-02T15:15:40.932 INFO:teuthology.orchestra.run.smithi001.stdout:(58/145): python3-repoze-lru-0.7-6.el8.noarch.r 309 kB/s | 34 kB 00:00 2024-02-02T15:15:40.957 INFO:teuthology.orchestra.run.smithi001.stdout:(59/145): python3-routes-2.4.1-12.el8.noarch.rp 1.8 MB/s | 196 kB 00:00 2024-02-02T15:15:40.982 INFO:teuthology.orchestra.run.smithi001.stdout:(60/145): python3-rsa-4.9-2.el8.noarch.rpm 587 kB/s | 64 kB 00:00 2024-02-02T15:15:41.008 INFO:teuthology.orchestra.run.smithi001.stdout:(61/145): python3-simplegeneric-0.8.1-17.el8.no 249 kB/s | 19 kB 00:00 2024-02-02T15:15:41.033 INFO:teuthology.orchestra.run.smithi001.stdout:(62/145): python3-singledispatch-3.4.0.3-18.el8 317 kB/s | 24 kB 00:00 2024-02-02T15:15:41.058 INFO:teuthology.orchestra.run.smithi001.stdout:(63/145): python3-tempora-1.14.1-5.el8.noarch.r 389 kB/s | 29 kB 00:00 2024-02-02T15:15:41.092 INFO:teuthology.orchestra.run.smithi001.stdout:(64/145): python3-trustme-0.6.0-4.el8.noarch.rp 328 kB/s | 27 kB 00:00 2024-02-02T15:15:41.125 INFO:teuthology.orchestra.run.smithi001.stdout:(65/145): python3-waitress-1.4.3-1.el8.noarch.r 2.7 MB/s | 255 kB 00:00 2024-02-02T15:15:41.151 INFO:teuthology.orchestra.run.smithi001.stdout:(66/145): python3-webob-1.8.5-1.el8.1.noarch.rp 2.7 MB/s | 251 kB 00:00 2024-02-02T15:15:41.184 INFO:teuthology.orchestra.run.smithi001.stdout:(67/145): python3-websocket-client-0.56.0-5.el8 665 kB/s | 61 kB 00:00 2024-02-02T15:15:41.209 INFO:teuthology.orchestra.run.smithi001.stdout:(68/145): python3-webtest-2.0.33-1.el8.noarch.r 1.0 MB/s | 85 kB 00:00 2024-02-02T15:15:41.260 INFO:teuthology.orchestra.run.smithi001.stdout:(69/145): python3-zc-lockfile-2.0-2.el8.noarch. 211 kB/s | 23 kB 00:00 2024-02-02T15:15:41.285 INFO:teuthology.orchestra.run.smithi001.stdout:(70/145): xmlstarlet-1.6.1-20.el8.x86_64.rpm 925 kB/s | 70 kB 00:00 2024-02-02T15:15:41.327 INFO:teuthology.orchestra.run.smithi001.stdout:(71/145): libverto-devel-0.3.0-5.el8.x86_64.rpm 268 kB/s | 18 kB 00:00 2024-02-02T15:15:41.344 INFO:teuthology.orchestra.run.smithi001.stdout:(72/145): python3-jwt-1.6.1-2.el8.noarch.rpm 856 kB/s | 43 kB 00:00 2024-02-02T15:15:41.361 INFO:teuthology.orchestra.run.smithi001.stdout:(73/145): python3-oauthlib-2.1.0-1.el8.noarch.r 4.6 MB/s | 155 kB 00:00 2024-02-02T15:15:41.386 INFO:teuthology.orchestra.run.smithi001.stdout:(74/145): python3-pycparser-2.14-14.el8.noarch. 2.6 MB/s | 109 kB 00:00 2024-02-02T15:15:41.411 INFO:teuthology.orchestra.run.smithi001.stdout:(75/145): python3-requests-oauthlib-1.0.0-1.el8 851 kB/s | 43 kB 00:00 2024-02-02T15:15:41.437 INFO:teuthology.orchestra.run.smithi001.stdout:(76/145): libaio-devel-0.3.112-1.el8.x86_64.rpm 372 kB/s | 19 kB 00:00 2024-02-02T15:15:41.462 INFO:teuthology.orchestra.run.smithi001.stdout:(77/145): libzstd-devel-1.4.4-1.el8.x86_64.rpm 875 kB/s | 44 kB 00:00 2024-02-02T15:15:41.504 INFO:teuthology.orchestra.run.smithi001.stdout:(78/145): pcre2-devel-10.32-2.el8.x86_64.rpm 8.8 MB/s | 605 kB 00:00 2024-02-02T15:15:41.538 INFO:teuthology.orchestra.run.smithi001.stdout:(79/145): pcre2-utf16-10.32-2.el8.x86_64.rpm 3.0 MB/s | 229 kB 00:00 2024-02-02T15:15:41.571 INFO:teuthology.orchestra.run.smithi001.stdout:(80/145): pcre2-utf32-10.32-2.el8.x86_64.rpm 3.2 MB/s | 220 kB 00:00 2024-02-02T15:15:41.597 INFO:teuthology.orchestra.run.smithi001.stdout:(81/145): libselinux-devel-2.9-5.el8.x86_64.rpm 3.3 MB/s | 200 kB 00:00 2024-02-02T15:15:41.622 INFO:teuthology.orchestra.run.smithi001.stdout:(82/145): keyutils-libs-devel-1.5.10-9.el8.x86_ 961 kB/s | 48 kB 00:00 2024-02-02T15:15:41.647 INFO:teuthology.orchestra.run.smithi001.stdout:(83/145): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2024-02-02T15:15:41.673 INFO:teuthology.orchestra.run.smithi001.stdout:(84/145): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-02-02T15:15:41.729 INFO:teuthology.orchestra.run.smithi001.stdout:(85/145): libcom_err-devel-1.45.6-4.el8.x86_64. 475 kB/s | 39 kB 00:00 2024-02-02T15:15:41.757 INFO:teuthology.orchestra.run.smithi001.stdout:(86/145): fuse-2.9.7-17.el8.x86_64.rpm 996 kB/s | 83 kB 00:00 2024-02-02T15:15:41.799 INFO:teuthology.orchestra.run.smithi001.stdout:(87/145): libgfortran-8.5.0-20.el8.x86_64.rpm 9.1 MB/s | 645 kB 00:00 2024-02-02T15:15:41.824 INFO:teuthology.orchestra.run.smithi001.stdout:(88/145): libquadmath-8.5.0-20.el8.x86_64.rpm 2.5 MB/s | 172 kB 00:00 2024-02-02T15:15:41.849 INFO:teuthology.orchestra.run.smithi001.stdout:(89/145): libuuid-devel-2.32.1-43.el8.x86_64.rp 1.9 MB/s | 100 kB 00:00 2024-02-02T15:15:41.883 INFO:teuthology.orchestra.run.smithi001.stdout:(90/145): python3-cffi-1.11.5-6.el8.x86_64.rpm 4.0 MB/s | 238 kB 00:00 2024-02-02T15:15:41.934 INFO:teuthology.orchestra.run.smithi001.stdout:(91/145): python3-cryptography-3.2.1-6.el8.x86_ 6.5 MB/s | 559 kB 00:00 2024-02-02T15:15:41.959 INFO:teuthology.orchestra.run.smithi001.stdout:(92/145): xfsprogs-devel-5.0.0-12.el8.x86_64.rp 966 kB/s | 73 kB 00:00 2024-02-02T15:15:41.984 INFO:teuthology.orchestra.run.smithi001.stdout:(93/145): NetworkManager-initscripts-updown-1.4 2.8 MB/s | 144 kB 00:00 2024-02-02T15:15:42.009 INFO:teuthology.orchestra.run.smithi001.stdout:(94/145): elfutils-libelf-devel-0.189-3.el8.x86 1.2 MB/s | 62 kB 00:00 2024-02-02T15:15:42.051 INFO:teuthology.orchestra.run.smithi001.stdout:(95/145): thrift-0.13.0-2.el8.x86_64.rpm 2.0 MB/s | 1.7 MB 00:00 2024-02-02T15:15:42.064 INFO:teuthology.orchestra.run.smithi142.stdout:Transaction test succeeded. 2024-02-02T15:15:42.074 INFO:teuthology.orchestra.run.smithi142.stdout:Running transaction 2024-02-02T15:15:42.077 INFO:teuthology.orchestra.run.smithi001.stdout:(96/145): librabbitmq-0.9.0-4.el8.x86_64.rpm 702 kB/s | 47 kB 00:00 2024-02-02T15:15:42.118 INFO:teuthology.orchestra.run.smithi001.stdout:(97/145): krb5-devel-1.18.2-26.el8_9.x86_64.rpm 4.1 MB/s | 562 kB 00:00 2024-02-02T15:15:42.144 INFO:teuthology.orchestra.run.smithi001.stdout:(98/145): python3-jsonpointer-1.10-11.el8.noarc 342 kB/s | 20 kB 00:00 2024-02-02T15:15:42.190 INFO:teuthology.orchestra.run.smithi001.stdout:(99/145): python3-prettytable-0.7.2-14.el8.noar 614 kB/s | 44 kB 00:00 2024-02-02T15:15:42.219 INFO:teuthology.orchestra.run.smithi001.stdout:(100/145): python3-pyasn1-modules-0.3.7-6.el8.n 1.4 MB/s | 110 kB 00:00 2024-02-02T15:15:42.245 INFO:teuthology.orchestra.run.smithi001.stdout:(101/145): python3-jsonpatch-1.21-2.el8.noarch. 494 kB/s | 27 kB 00:00 2024-02-02T15:15:42.270 INFO:teuthology.orchestra.run.smithi001.stdout:(102/145): python3-pyasn1-0.3.7-6.el8.noarch.rp 2.5 MB/s | 126 kB 00:00 2024-02-02T15:15:42.312 INFO:teuthology.orchestra.run.smithi001.stdout:(103/145): python3-werkzeug-0.12.2-4.el8.noarch 6.7 MB/s | 457 kB 00:00 2024-02-02T15:15:42.362 INFO:teuthology.orchestra.run.smithi001.stdout:(104/145): bison-3.0.4-10.el8.x86_64.rpm 7.3 MB/s | 688 kB 00:00 2024-02-02T15:15:42.396 INFO:teuthology.orchestra.run.smithi001.stdout:(105/145): flex-2.6.1-9.el8.x86_64.rpm 3.7 MB/s | 320 kB 00:00 2024-02-02T15:15:42.421 INFO:teuthology.orchestra.run.smithi001.stdout:(106/145): python3-markupsafe-0.23-19.el8.x86_6 1.5 MB/s | 39 kB 00:00 2024-02-02T15:15:42.455 INFO:teuthology.orchestra.run.smithi001.stdout:(107/145): libtool-2.4.6-25.el8.x86_64.rpm 7.5 MB/s | 709 kB 00:00 2024-02-02T15:15:42.505 INFO:teuthology.orchestra.run.smithi001.stdout:(108/145): openssl-devel-1.1.1k-12.el8_9.x86_64 5.1 MB/s | 2.3 MB 00:00 2024-02-02T15:15:42.539 INFO:teuthology.orchestra.run.smithi001.stdout:(109/145): lttng-ust-2.8.1-11.el8.x86_64.rpm 2.2 MB/s | 259 kB 00:00 2024-02-02T15:15:42.564 INFO:teuthology.orchestra.run.smithi001.stdout:(110/145): oniguruma-6.8.2-2.el8.x86_64.rpm 1.7 MB/s | 187 kB 00:00 2024-02-02T15:15:42.589 INFO:teuthology.orchestra.run.smithi001.stdout:(111/145): uuid-1.6.2-43.el8.x86_64.rpm 765 kB/s | 64 kB 00:00 2024-02-02T15:15:42.614 INFO:teuthology.orchestra.run.smithi001.stdout:(112/145): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.0 MB/s | 103 kB 00:00 2024-02-02T15:15:42.682 INFO:teuthology.orchestra.run.smithi001.stdout:(113/145): python3-jinja2-2.10.1-3.el8.noarch.r 5.7 MB/s | 538 kB 00:00 2024-02-02T15:15:42.732 INFO:teuthology.orchestra.run.smithi001.stdout:(114/145): socat-1.7.4.1-1.el8.x86_64.rpm 6.4 MB/s | 323 kB 00:00 2024-02-02T15:15:42.774 INFO:teuthology.orchestra.run.smithi001.stdout:(115/145): python3-numpy-f2py-1.14.3-10.el8.x86 5.3 MB/s | 225 kB 00:00 2024-02-02T15:15:42.824 INFO:teuthology.orchestra.run.smithi001.stdout:(116/145): librdkafka-0.11.4-3.el8.x86_64.rpm 6.9 MB/s | 354 kB 00:00 2024-02-02T15:15:42.916 INFO:teuthology.orchestra.run.smithi001.stdout:(117/145): python3-numpy-1.14.3-10.el8.x86_64.r 12 MB/s | 3.7 MB 00:00 2024-02-02T15:15:42.992 INFO:teuthology.orchestra.run.smithi001.stdout:(118/145): python3-babel-2.5.1-7.el8.noarch.rpm 11 MB/s | 4.8 MB 00:00 2024-02-02T15:15:43.017 INFO:teuthology.orchestra.run.smithi001.stdout:(119/145): python3-mako-1.0.6-14.el8.noarch.rpm 1.5 MB/s | 157 kB 00:00 2024-02-02T15:15:43.084 INFO:teuthology.orchestra.run.smithi001.stdout:(120/145): automake-1.16.1-8.el8.noarch.rpm 10 MB/s | 714 kB 00:00 2024-02-02T15:15:43.142 INFO:teuthology.orchestra.run.smithi001.stdout:(121/145): libpq-13.11-1.el8.x86_64.rpm 3.3 MB/s | 198 kB 00:00 2024-02-02T15:15:43.184 INFO:teuthology.orchestra.run.smithi001.stdout:(122/145): jq-1.6-7.el8.x86_64.rpm 4.8 MB/s | 202 kB 00:00 2024-02-02T15:15:43.218 INFO:teuthology.orchestra.run.smithi001.stdout:(123/145): python3-pytz-2017.2-11.el8.noarch.rp 1.6 MB/s | 54 kB 00:00 2024-02-02T15:15:43.452 INFO:teuthology.orchestra.run.smithi001.stdout:(124/145): openblas-threads-0.3.15-6.el8.x86_64 8.3 MB/s | 5.2 MB 00:00 2024-02-02T15:15:43.519 INFO:teuthology.orchestra.run.smithi001.stdout:(125/145): postgresql-contrib-10.23-3.module+el 12 MB/s | 811 kB 00:00 2024-02-02T15:15:43.619 INFO:teuthology.orchestra.run.smithi001.stdout:(126/145): openblas-0.3.15-6.el8.x86_64.rpm 8.1 MB/s | 5.0 MB 00:00 2024-02-02T15:15:43.661 INFO:teuthology.orchestra.run.smithi001.stdout:(127/145): postgresql-10.23-3.module+el8.9.0+20 3.4 MB/s | 1.5 MB 00:00 2024-02-02T15:15:43.836 INFO:teuthology.orchestra.run.smithi001.stdout:(128/145): postgresql-server-10.23-3.module+el8 16 MB/s | 5.1 MB 00:00 2024-02-02T15:15:44.112 INFO:teuthology.orchestra.run.smithi001.stdout:(129/145): python3-scipy-1.0.0-21.module+el8.9. 28 MB/s | 14 MB 00:00 2024-02-02T15:15:44.153 INFO:teuthology.orchestra.run.smithi001.stdout:(130/145): elfutils-libelf-0.189-3.el8.x86_64.r 5.8 MB/s | 232 kB 00:00 2024-02-02T15:15:44.195 INFO:teuthology.orchestra.run.smithi001.stdout:(131/145): elfutils-libs-0.189-3.el8.x86_64.rpm 7.2 MB/s | 303 kB 00:00 2024-02-02T15:15:44.237 INFO:teuthology.orchestra.run.smithi001.stdout:(132/145): libmount-2.32.1-43.el8.x86_64.rpm 5.6 MB/s | 237 kB 00:00 2024-02-02T15:15:44.279 INFO:teuthology.orchestra.run.smithi001.stdout:(133/145): libsmartcols-2.32.1-43.el8.x86_64.rp 4.2 MB/s | 179 kB 00:00 2024-02-02T15:15:44.346 INFO:teuthology.orchestra.run.smithi001.stdout:(134/145): librbd1-18.2.1-325.g0fb32788.el8.x86 7.0 MB/s | 3.6 MB 00:00 2024-02-02T15:15:44.371 INFO:teuthology.orchestra.run.smithi001.stdout:(135/145): libuuid-2.32.1-43.el8.x86_64.rpm 1.1 MB/s | 99 kB 00:00 2024-02-02T15:15:44.413 INFO:teuthology.orchestra.run.smithi001.stdout:(136/145): util-linux-user-2.32.1-43.el8.x86_64 2.4 MB/s | 103 kB 00:00 2024-02-02T15:15:44.489 INFO:teuthology.orchestra.run.smithi001.stdout:(137/145): util-linux-2.32.1-43.el8.x86_64.rpm 17 MB/s | 2.5 MB 00:00 2024-02-02T15:15:44.547 INFO:teuthology.orchestra.run.smithi001.stdout:(138/145): iproute-6.2.0-5.el8_9.x86_64.rpm 6.3 MB/s | 854 kB 00:00 2024-02-02T15:15:44.572 INFO:teuthology.orchestra.run.smithi001.stdout:(139/145): libblkid-2.32.1-43.el8.x86_64.rpm 2.6 MB/s | 220 kB 00:00 2024-02-02T15:15:44.606 INFO:teuthology.orchestra.run.smithi001.stdout:(140/145): libfdisk-2.32.1-43.el8.x86_64.rpm 4.2 MB/s | 254 kB 00:00 2024-02-02T15:15:44.648 INFO:teuthology.orchestra.run.smithi001.stdout:(141/145): openssl-1.1.1k-12.el8_9.x86_64.rpm 9.2 MB/s | 711 kB 00:00 2024-02-02T15:15:44.698 INFO:teuthology.orchestra.run.smithi001.stdout:(142/145): xfsdump-3.1.8-7.el8_9.x86_64.rpm 6.6 MB/s | 333 kB 00:00 2024-02-02T15:15:44.740 INFO:teuthology.orchestra.run.smithi001.stdout:(143/145): openssl-libs-1.1.1k-12.el8_9.x86_64. 11 MB/s | 1.5 MB 00:00 2024-02-02T15:15:44.791 INFO:teuthology.orchestra.run.smithi001.stdout:(144/145): xfsprogs-5.0.0-12.el8.x86_64.rpm 12 MB/s | 1.1 MB 00:00 2024-02-02T15:15:44.891 INFO:teuthology.orchestra.run.smithi001.stdout:(145/145): librados2-18.2.1-325.g0fb32788.el8.x 3.0 MB/s | 3.7 MB 00:01 2024-02-02T15:15:44.894 INFO:teuthology.orchestra.run.smithi001.stdout:-------------------------------------------------------------------------------- 2024-02-02T15:15:44.894 INFO:teuthology.orchestra.run.smithi001.stdout:Total 13 MB/s | 212 MB 00:16 2024-02-02T15:15:46.449 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-02-02T15:15:46.583 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/1 2024-02-02T15:15:46.845 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/161 2024-02-02T15:15:47.022 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction check 2024-02-02T15:15:47.058 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/161 2024-02-02T15:15:47.270 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/161 2024-02-02T15:15:47.454 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : libuuid-2.32.1-43.el8.x86_64 3/161 2024-02-02T15:15:47.494 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction check succeeded. 2024-02-02T15:15:47.494 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction test 2024-02-02T15:15:47.529 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: libuuid-2.32.1-43.el8.x86_64 3/161 2024-02-02T15:15:47.729 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : libblkid-2.32.1-43.el8.x86_64 4/161 2024-02-02T15:15:47.799 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: libblkid-2.32.1-43.el8.x86_64 4/161 2024-02-02T15:15:47.975 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/161 2024-02-02T15:15:48.052 INFO:teuthology.orchestra.run.smithi142.stdout: Preparing : 1/1 2024-02-02T15:15:48.188 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/1 2024-02-02T15:15:48.194 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : librabbitmq-0.9.0-4.el8.x86_64 6/161 2024-02-02T15:15:48.412 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 7/161 2024-02-02T15:15:48.475 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/161 2024-02-02T15:15:48.499 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 7/161 2024-02-02T15:15:48.694 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/161 2024-02-02T15:15:48.720 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/161 2024-02-02T15:15:48.882 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/161 2024-02-02T15:15:49.103 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : libuuid-2.32.1-43.el8.x86_64 3/161 2024-02-02T15:15:49.123 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/161 2024-02-02T15:15:49.160 INFO:teuthology.orchestra.run.smithi001.stdout:Transaction test succeeded. 2024-02-02T15:15:49.173 INFO:teuthology.orchestra.run.smithi001.stdout:Running transaction 2024-02-02T15:15:49.205 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libuuid-2.32.1-43.el8.x86_64 3/161 2024-02-02T15:15:49.345 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : librados2-2:18.2.1-325.g0fb32788.el8.x86_64 9/161 2024-02-02T15:15:49.411 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : libblkid-2.32.1-43.el8.x86_64 4/161 2024-02-02T15:15:49.466 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libblkid-2.32.1-43.el8.x86_64 4/161 2024-02-02T15:15:49.699 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/161 2024-02-02T15:15:49.738 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: librados2-2:18.2.1-325.g0fb32788.el8.x86_64 9/161 2024-02-02T15:15:49.928 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : librbd1-2:18.2.1-325.g0fb32788.el8.x86_64 10/161 2024-02-02T15:15:49.935 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : librabbitmq-0.9.0-4.el8.x86_64 6/161 2024-02-02T15:15:50.056 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: librbd1-2:18.2.1-325.g0fb32788.el8.x86_64 10/161 2024-02-02T15:15:50.178 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 7/161 2024-02-02T15:15:50.278 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libcephfs2-2:18.2.1-325.g0fb32788.el8.x86_64 11/161 2024-02-02T15:15:50.283 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 7/161 2024-02-02T15:15:50.377 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: libcephfs2-2:18.2.1-325.g0fb32788.el8.x86_64 11/161 2024-02-02T15:15:50.516 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/161 2024-02-02T15:15:50.663 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-rados-2:18.2.1-325.g0fb32788.el8.x86_64 12/161 2024-02-02T15:15:50.931 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/161 2024-02-02T15:15:50.967 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 13/161 2024-02-02T15:15:51.042 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 13/161 2024-02-02T15:15:51.227 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : librados2-2:18.2.1-325.g0fb32788.el8.x86_64 9/161 2024-02-02T15:15:51.292 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libpq-13.11-1.el8.x86_64 14/161 2024-02-02T15:15:51.548 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 15/161 2024-02-02T15:15:51.606 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librados2-2:18.2.1-325.g0fb32788.el8.x86_64 9/161 2024-02-02T15:15:51.806 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/161 2024-02-02T15:15:51.835 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : librbd1-2:18.2.1-325.g0fb32788.el8.x86_64 10/161 2024-02-02T15:15:51.982 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librbd1-2:18.2.1-325.g0fb32788.el8.x86_64 10/161 2024-02-02T15:15:52.165 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 17/161 2024-02-02T15:15:52.210 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libcephfs2-2:18.2.1-325.g0fb32788.el8.x86_64 11/161 2024-02-02T15:15:52.295 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libcephfs2-2:18.2.1-325.g0fb32788.el8.x86_64 11/161 2024-02-02T15:15:52.399 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : postgresql-10.23-3.module+el8.9.0+20831+78489779 18/161 2024-02-02T15:15:52.553 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libcephsqlite-2:18.2.1-325.g0fb32788.el8.x86_64 19/161 2024-02-02T15:15:52.662 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-rados-2:18.2.1-325.g0fb32788.el8.x86_64 12/161 2024-02-02T15:15:52.685 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: libcephsqlite-2:18.2.1-325.g0fb32788.el8.x86_64 19/161 2024-02-02T15:15:52.844 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libradosstriper1-2:18.2.1-325.g0fb32788.el8.x86_ 20/161 2024-02-02T15:15:52.932 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: libradosstriper1-2:18.2.1-325.g0fb32788.el8.x86_ 20/161 2024-02-02T15:15:53.089 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 21/161 2024-02-02T15:15:53.102 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 13/161 2024-02-02T15:15:53.186 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 13/161 2024-02-02T15:15:53.394 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 22/161 2024-02-02T15:15:53.425 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libpq-13.11-1.el8.x86_64 14/161 2024-02-02T15:15:53.649 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 23/161 2024-02-02T15:15:53.691 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 15/161 2024-02-02T15:15:53.836 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 24/161 2024-02-02T15:15:53.914 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/161 2024-02-02T15:15:53.958 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 24/161 2024-02-02T15:15:54.136 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 25/161 2024-02-02T15:15:54.333 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 17/161 2024-02-02T15:15:54.598 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : postgresql-10.23-3.module+el8.9.0+20831+78489779 18/161 2024-02-02T15:15:54.759 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 25/161 2024-02-02T15:15:54.793 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libcephsqlite-2:18.2.1-325.g0fb32788.el8.x86_64 19/161 2024-02-02T15:15:54.911 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 26/161 2024-02-02T15:15:54.926 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libcephsqlite-2:18.2.1-325.g0fb32788.el8.x86_64 19/161 2024-02-02T15:15:55.136 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libradosstriper1-2:18.2.1-325.g0fb32788.el8.x86_ 20/161 2024-02-02T15:15:55.153 INFO:teuthology.orchestra.run.smithi001.stdout: Preparing : 1/1 2024-02-02T15:15:55.209 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libradosstriper1-2:18.2.1-325.g0fb32788.el8.x86_ 20/161 2024-02-02T15:15:55.355 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/1 2024-02-02T15:15:55.413 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 21/161 2024-02-02T15:15:55.514 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 26/161 2024-02-02T15:15:55.583 INFO:teuthology.orchestra.run.smithi001.stdout: Upgrading : openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/161 2024-02-02T15:15:55.702 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 22/161 2024-02-02T15:15:55.739 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : openblas-0.3.15-6.el8.x86_64 27/161 2024-02-02T15:15:55.864 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: openssl-libs-1:1.1.1k-12.el8_9.x86_64 1/161 2024-02-02T15:15:55.957 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 23/161 2024-02-02T15:15:56.044 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/161 2024-02-02T15:15:56.151 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 24/161 2024-02-02T15:15:56.249 INFO:teuthology.orchestra.run.smithi001.stdout: Upgrading : libuuid-2.32.1-43.el8.x86_64 3/161 2024-02-02T15:15:56.268 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 24/161 2024-02-02T15:15:56.339 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: libuuid-2.32.1-43.el8.x86_64 3/161 2024-02-02T15:15:56.420 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 27/161 2024-02-02T15:15:56.434 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 25/161 2024-02-02T15:15:56.507 INFO:teuthology.orchestra.run.smithi001.stdout: Upgrading : libblkid-2.32.1-43.el8.x86_64 4/161 2024-02-02T15:15:56.601 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: libblkid-2.32.1-43.el8.x86_64 4/161 2024-02-02T15:15:56.716 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 28/161 2024-02-02T15:15:56.932 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 29/161 2024-02-02T15:15:56.938 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/161 2024-02-02T15:15:57.055 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 25/161 2024-02-02T15:15:57.121 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : librabbitmq-0.9.0-4.el8.x86_64 6/161 2024-02-02T15:15:57.237 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 30/161 2024-02-02T15:15:57.268 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 26/161 2024-02-02T15:15:57.340 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 7/161 2024-02-02T15:15:57.408 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-ceph-common-2:18.2.1-325.g0fb32788.el8.x 31/161 2024-02-02T15:15:57.479 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 7/161 2024-02-02T15:15:57.587 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-ceph-argparse-2:18.2.1-325.g0fb32788.el8 32/161 2024-02-02T15:15:57.690 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/161 2024-02-02T15:15:57.839 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cephfs-2:18.2.1-325.g0fb32788.el8.x86_64 33/161 2024-02-02T15:15:57.869 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 26/161 2024-02-02T15:15:58.052 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 34/161 2024-02-02T15:15:58.170 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/161 2024-02-02T15:15:58.316 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : openblas-0.3.15-6.el8.x86_64 27/161 2024-02-02T15:15:58.357 INFO:teuthology.orchestra.run.smithi001.stdout: Upgrading : librados2-2:18.2.1-325.g0fb32788.el8.x86_64 9/161 2024-02-02T15:15:58.832 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: librados2-2:18.2.1-325.g0fb32788.el8.x86_64 9/161 2024-02-02T15:15:59.031 INFO:teuthology.orchestra.run.smithi001.stdout: Upgrading : librbd1-2:18.2.1-325.g0fb32788.el8.x86_64 10/161 2024-02-02T15:15:59.039 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 27/161 2024-02-02T15:15:59.180 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: librbd1-2:18.2.1-325.g0fb32788.el8.x86_64 10/161 2024-02-02T15:15:59.332 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 28/161 2024-02-02T15:15:59.365 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libcephfs2-2:18.2.1-325.g0fb32788.el8.x86_64 11/161 2024-02-02T15:15:59.506 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: libcephfs2-2:18.2.1-325.g0fb32788.el8.x86_64 11/161 2024-02-02T15:15:59.531 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 29/161 2024-02-02T15:15:59.791 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 34/161 2024-02-02T15:15:59.835 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 30/161 2024-02-02T15:15:59.923 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-rados-2:18.2.1-325.g0fb32788.el8.x86_64 12/161 2024-02-02T15:16:00.048 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-ceph-common-2:18.2.1-325.g0fb32788.el8.x 31/161 2024-02-02T15:16:00.187 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-ceph-argparse-2:18.2.1-325.g0fb32788.el8 32/161 2024-02-02T15:16:00.219 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 35/161 2024-02-02T15:16:00.224 INFO:teuthology.orchestra.run.smithi001.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 13/161 2024-02-02T15:16:00.336 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 13/161 2024-02-02T15:16:00.491 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cephfs-2:18.2.1-325.g0fb32788.el8.x86_64 33/161 2024-02-02T15:16:00.568 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libpq-13.11-1.el8.x86_64 14/161 2024-02-02T15:16:00.809 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 34/161 2024-02-02T15:16:00.933 INFO:teuthology.orchestra.run.smithi001.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 15/161 2024-02-02T15:16:01.123 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/161 2024-02-02T15:16:01.199 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 36/161 2024-02-02T15:16:01.555 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 37/161 2024-02-02T15:16:01.593 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 17/161 2024-02-02T15:16:01.768 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 38/161 2024-02-02T15:16:01.863 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : postgresql-10.23-3.module+el8.9.0+20831+78489779 18/161 2024-02-02T15:16:02.020 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-rsa-4.9-2.el8.noarch 39/161 2024-02-02T15:16:02.023 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libcephsqlite-2:18.2.1-325.g0fb32788.el8.x86_64 19/161 2024-02-02T15:16:02.158 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: libcephsqlite-2:18.2.1-325.g0fb32788.el8.x86_64 19/161 2024-02-02T15:16:02.236 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 40/161 2024-02-02T15:16:02.473 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libradosstriper1-2:18.2.1-325.g0fb32788.el8.x86_ 20/161 2024-02-02T15:16:02.476 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-rbd-2:18.2.1-325.g0fb32788.el8.x86_64 41/161 2024-02-02T15:16:02.554 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 34/161 2024-02-02T15:16:02.587 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: libradosstriper1-2:18.2.1-325.g0fb32788.el8.x86_ 20/161 2024-02-02T15:16:02.640 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : librados-devel-2:18.2.1-325.g0fb32788.el8.x86_64 42/161 2024-02-02T15:16:02.748 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 21/161 2024-02-02T15:16:02.867 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : libmount-2.32.1-43.el8.x86_64 43/161 2024-02-02T15:16:02.931 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: libmount-2.32.1-43.el8.x86_64 43/161 2024-02-02T15:16:02.960 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 35/161 2024-02-02T15:16:03.041 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 22/161 2024-02-02T15:16:03.167 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : libfdisk-2.32.1-43.el8.x86_64 44/161 2024-02-02T15:16:03.249 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: libfdisk-2.32.1-43.el8.x86_64 44/161 2024-02-02T15:16:03.291 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 23/161 2024-02-02T15:16:03.473 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 24/161 2024-02-02T15:16:03.522 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libuuid-devel-2.32.1-43.el8.x86_64 45/161 2024-02-02T15:16:03.649 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 24/161 2024-02-02T15:16:03.761 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 46/161 2024-02-02T15:16:03.845 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 25/161 2024-02-02T15:16:03.927 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 36/161 2024-02-02T15:16:04.010 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : libsmartcols-2.32.1-43.el8.x86_64 47/161 2024-02-02T15:16:04.329 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 37/161 2024-02-02T15:16:04.478 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: libsmartcols-2.32.1-43.el8.x86_64 47/161 2024-02-02T15:16:04.575 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 25/161 2024-02-02T15:16:04.592 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 38/161 2024-02-02T15:16:04.751 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : util-linux-2.32.1-43.el8.x86_64 48/161 2024-02-02T15:16:04.773 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 26/161 2024-02-02T15:16:04.843 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: util-linux-2.32.1-43.el8.x86_64 48/161 2024-02-02T15:16:04.844 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-rsa-4.9-2.el8.noarch 39/161 2024-02-02T15:16:05.067 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : automake-1.16.1-8.el8.noarch 49/161 2024-02-02T15:16:05.086 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 40/161 2024-02-02T15:16:05.233 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : uuid-1.6.2-43.el8.x86_64 50/161 2024-02-02T15:16:05.349 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 50/161 2024-02-02T15:16:05.371 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-rbd-2:18.2.1-325.g0fb32788.el8.x86_64 41/161 2024-02-02T15:16:05.471 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 26/161 2024-02-02T15:16:05.508 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 51/161 2024-02-02T15:16:05.572 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 51/161 2024-02-02T15:16:05.602 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : librados-devel-2:18.2.1-325.g0fb32788.el8.x86_64 42/161 2024-02-02T15:16:05.673 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : openblas-0.3.15-6.el8.x86_64 27/161 2024-02-02T15:16:05.746 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : jq-1.6-7.el8.x86_64 52/161 2024-02-02T15:16:05.850 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : libmount-2.32.1-43.el8.x86_64 43/161 2024-02-02T15:16:05.927 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libmount-2.32.1-43.el8.x86_64 43/161 2024-02-02T15:16:05.948 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 53/161 2024-02-02T15:16:06.123 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 54/161 2024-02-02T15:16:06.191 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : libfdisk-2.32.1-43.el8.x86_64 44/161 2024-02-02T15:16:06.295 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libfdisk-2.32.1-43.el8.x86_64 44/161 2024-02-02T15:16:06.343 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 55/161 2024-02-02T15:16:06.554 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : fuse-2.9.7-17.el8.x86_64 56/161 2024-02-02T15:16:06.565 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libuuid-devel-2.32.1-43.el8.x86_64 45/161 2024-02-02T15:16:06.641 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 27/161 2024-02-02T15:16:06.774 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 57/161 2024-02-02T15:16:06.807 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 46/161 2024-02-02T15:16:06.955 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 58/161 2024-02-02T15:16:07.027 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 28/161 2024-02-02T15:16:07.038 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : libsmartcols-2.32.1-43.el8.x86_64 47/161 2024-02-02T15:16:07.238 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 29/161 2024-02-02T15:16:07.488 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 30/161 2024-02-02T15:16:07.490 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libsmartcols-2.32.1-43.el8.x86_64 47/161 2024-02-02T15:16:07.561 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 59/161 2024-02-02T15:16:07.671 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-ceph-common-2:18.2.1-325.g0fb32788.el8.x 31/161 2024-02-02T15:16:07.762 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 60/161 2024-02-02T15:16:07.803 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-ceph-argparse-2:18.2.1-325.g0fb32788.el8 32/161 2024-02-02T15:16:07.900 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : util-linux-2.32.1-43.el8.x86_64 48/161 2024-02-02T15:16:07.994 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: util-linux-2.32.1-43.el8.x86_64 48/161 2024-02-02T15:16:08.053 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 61/161 2024-02-02T15:16:08.121 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-cephfs-2:18.2.1-325.g0fb32788.el8.x86_64 33/161 2024-02-02T15:16:08.225 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : automake-1.16.1-8.el8.noarch 49/161 2024-02-02T15:16:08.354 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 62/161 2024-02-02T15:16:08.441 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : uuid-1.6.2-43.el8.x86_64 50/161 2024-02-02T15:16:08.461 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 34/161 2024-02-02T15:16:08.510 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 50/161 2024-02-02T15:16:08.596 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 63/161 2024-02-02T15:16:08.708 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 51/161 2024-02-02T15:16:08.788 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 64/161 2024-02-02T15:16:08.818 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 51/161 2024-02-02T15:16:09.002 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 65/161 2024-02-02T15:16:09.058 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : jq-1.6-7.el8.x86_64 52/161 2024-02-02T15:16:09.273 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 53/161 2024-02-02T15:16:09.406 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 66/161 2024-02-02T15:16:09.459 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 54/161 2024-02-02T15:16:09.651 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 55/161 2024-02-02T15:16:09.734 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cryptography-3.2.1-6.el8.x86_64 67/161 2024-02-02T15:16:09.920 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 68/161 2024-02-02T15:16:09.937 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : fuse-2.9.7-17.el8.x86_64 56/161 2024-02-02T15:16:10.135 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 69/161 2024-02-02T15:16:10.181 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 57/161 2024-02-02T15:16:10.378 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 70/161 2024-02-02T15:16:10.413 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 58/161 2024-02-02T15:16:10.577 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 59/161 2024-02-02T15:16:10.665 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 34/161 2024-02-02T15:16:10.705 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 71/161 2024-02-02T15:16:10.770 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 60/161 2024-02-02T15:16:10.948 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 72/161 2024-02-02T15:16:11.059 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 35/161 2024-02-02T15:16:11.094 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 61/161 2024-02-02T15:16:11.191 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 73/161 2024-02-02T15:16:11.412 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 62/161 2024-02-02T15:16:11.418 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 74/161 2024-02-02T15:16:11.613 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 75/161 2024-02-02T15:16:11.662 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 63/161 2024-02-02T15:16:11.842 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : krb5-devel-1.18.2-26.el8_9.x86_64 76/161 2024-02-02T15:16:11.861 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 64/161 2024-02-02T15:16:12.023 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 77/161 2024-02-02T15:16:12.120 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 65/161 2024-02-02T15:16:12.206 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 36/161 2024-02-02T15:16:12.227 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 78/161 2024-02-02T15:16:12.484 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 79/161 2024-02-02T15:16:12.530 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 66/161 2024-02-02T15:16:12.561 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 37/161 2024-02-02T15:16:12.772 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 80/161 2024-02-02T15:16:12.840 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 38/161 2024-02-02T15:16:12.866 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cryptography-3.2.1-6.el8.x86_64 67/161 2024-02-02T15:16:12.986 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 81/161 2024-02-02T15:16:13.078 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-rsa-4.9-2.el8.noarch 39/161 2024-02-02T15:16:13.102 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 68/161 2024-02-02T15:16:13.184 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 82/161 2024-02-02T15:16:13.322 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 40/161 2024-02-02T15:16:13.377 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 69/161 2024-02-02T15:16:13.383 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 83/161 2024-02-02T15:16:13.521 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-rbd-2:18.2.1-325.g0fb32788.el8.x86_64 41/161 2024-02-02T15:16:13.628 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 84/161 2024-02-02T15:16:13.669 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 70/161 2024-02-02T15:16:13.735 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : librados-devel-2:18.2.1-325.g0fb32788.el8.x86_64 42/161 2024-02-02T15:16:13.868 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 85/161 2024-02-02T15:16:13.929 INFO:teuthology.orchestra.run.smithi001.stdout: Upgrading : libmount-2.32.1-43.el8.x86_64 43/161 2024-02-02T15:16:14.046 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 71/161 2024-02-02T15:16:14.077 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: libmount-2.32.1-43.el8.x86_64 43/161 2024-02-02T15:16:14.093 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 86/161 2024-02-02T15:16:14.254 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 87/161 2024-02-02T15:16:14.288 INFO:teuthology.orchestra.run.smithi001.stdout: Upgrading : libfdisk-2.32.1-43.el8.x86_64 44/161 2024-02-02T15:16:14.348 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 72/161 2024-02-02T15:16:14.380 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: libfdisk-2.32.1-43.el8.x86_64 44/161 2024-02-02T15:16:14.495 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 88/161 2024-02-02T15:16:14.585 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libuuid-devel-2.32.1-43.el8.x86_64 45/161 2024-02-02T15:16:14.590 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 73/161 2024-02-02T15:16:14.743 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 89/161 2024-02-02T15:16:14.783 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 46/161 2024-02-02T15:16:14.859 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 74/161 2024-02-02T15:16:14.929 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 90/161 2024-02-02T15:16:15.036 INFO:teuthology.orchestra.run.smithi001.stdout: Upgrading : libsmartcols-2.32.1-43.el8.x86_64 47/161 2024-02-02T15:16:15.105 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 75/161 2024-02-02T15:16:15.209 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-portend-2.6-1.el8.noarch 91/161 2024-02-02T15:16:15.383 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : krb5-devel-1.18.2-26.el8_9.x86_64 76/161 2024-02-02T15:16:15.439 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 92/161 2024-02-02T15:16:15.590 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 77/161 2024-02-02T15:16:15.672 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 93/161 2024-02-02T15:16:15.677 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: libsmartcols-2.32.1-43.el8.x86_64 47/161 2024-02-02T15:16:15.812 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 78/161 2024-02-02T15:16:15.888 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 94/161 2024-02-02T15:16:16.014 INFO:teuthology.orchestra.run.smithi001.stdout: Upgrading : util-linux-2.32.1-43.el8.x86_64 48/161 2024-02-02T15:16:16.144 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 79/161 2024-02-02T15:16:16.145 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: util-linux-2.32.1-43.el8.x86_64 48/161 2024-02-02T15:16:16.409 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : automake-1.16.1-8.el8.noarch 49/161 2024-02-02T15:16:16.447 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 80/161 2024-02-02T15:16:16.580 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : uuid-1.6.2-43.el8.x86_64 50/161 2024-02-02T15:16:16.673 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 50/161 2024-02-02T15:16:16.678 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 81/161 2024-02-02T15:16:16.834 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 82/161 2024-02-02T15:16:16.855 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 51/161 2024-02-02T15:16:16.946 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 51/161 2024-02-02T15:16:16.996 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 95/161 2024-02-02T15:16:17.066 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 83/161 2024-02-02T15:16:17.159 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : jq-1.6-7.el8.x86_64 52/161 2024-02-02T15:16:17.293 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 96/161 2024-02-02T15:16:17.354 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 84/161 2024-02-02T15:16:17.369 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 53/161 2024-02-02T15:16:17.538 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 97/161 2024-02-02T15:16:17.540 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 54/161 2024-02-02T15:16:17.631 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 85/161 2024-02-02T15:16:17.727 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 55/161 2024-02-02T15:16:17.840 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 98/161 2024-02-02T15:16:17.843 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 86/161 2024-02-02T15:16:17.953 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : fuse-2.9.7-17.el8.x86_64 56/161 2024-02-02T15:16:18.046 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 87/161 2024-02-02T15:16:18.131 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 99/161 2024-02-02T15:16:18.154 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 57/161 2024-02-02T15:16:18.310 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 88/161 2024-02-02T15:16:18.347 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 100/161 2024-02-02T15:16:18.399 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 58/161 2024-02-02T15:16:18.577 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 89/161 2024-02-02T15:16:18.611 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 59/161 2024-02-02T15:16:18.804 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 90/161 2024-02-02T15:16:18.854 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 60/161 2024-02-02T15:16:19.144 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 101/161 2024-02-02T15:16:19.172 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 61/161 2024-02-02T15:16:19.177 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-portend-2.6-1.el8.noarch 91/161 2024-02-02T15:16:19.330 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : librgw2-2:18.2.1-325.g0fb32788.el8.x86_64 102/161 2024-02-02T15:16:19.433 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 62/161 2024-02-02T15:16:19.442 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: librgw2-2:18.2.1-325.g0fb32788.el8.x86_64 102/161 2024-02-02T15:16:19.463 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 92/161 2024-02-02T15:16:19.631 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-rgw-2:18.2.1-325.g0fb32788.el8.x86_64 103/161 2024-02-02T15:16:19.638 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 93/161 2024-02-02T15:16:19.677 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 63/161 2024-02-02T15:16:19.842 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 64/161 2024-02-02T15:16:19.879 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 94/161 2024-02-02T15:16:20.060 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 65/161 2024-02-02T15:16:20.550 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 66/161 2024-02-02T15:16:20.845 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-cryptography-3.2.1-6.el8.x86_64 67/161 2024-02-02T15:16:20.986 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 95/161 2024-02-02T15:16:21.077 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 68/161 2024-02-02T15:16:21.322 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 69/161 2024-02-02T15:16:21.343 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 96/161 2024-02-02T15:16:21.583 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 70/161 2024-02-02T15:16:21.613 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 97/161 2024-02-02T15:16:21.932 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 71/161 2024-02-02T15:16:21.955 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 98/161 2024-02-02T15:16:22.156 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 72/161 2024-02-02T15:16:22.248 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 99/161 2024-02-02T15:16:22.383 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 73/161 2024-02-02T15:16:22.439 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 100/161 2024-02-02T15:16:22.632 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 74/161 2024-02-02T15:16:22.841 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 75/161 2024-02-02T15:16:23.131 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : krb5-devel-1.18.2-26.el8_9.x86_64 76/161 2024-02-02T15:16:23.195 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 101/161 2024-02-02T15:16:23.250 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-common-2:18.2.1-325.g0fb32788.el8.x86_64 104/161 2024-02-02T15:16:23.341 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 77/161 2024-02-02T15:16:23.388 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : librgw2-2:18.2.1-325.g0fb32788.el8.x86_64 102/161 2024-02-02T15:16:23.448 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librgw2-2:18.2.1-325.g0fb32788.el8.x86_64 102/161 2024-02-02T15:16:23.533 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 78/161 2024-02-02T15:16:23.588 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-common-2:18.2.1-325.g0fb32788.el8.x86_64 104/161 2024-02-02T15:16:23.656 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : python3-rgw-2:18.2.1-325.g0fb32788.el8.x86_64 103/161 2024-02-02T15:16:23.807 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 79/161 2024-02-02T15:16:24.029 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 80/161 2024-02-02T15:16:24.167 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-common-2:18.2.1-325.g0fb32788.el8.x86_64 104/161 2024-02-02T15:16:24.184 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 81/161 2024-02-02T15:16:24.386 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 82/161 2024-02-02T15:16:24.428 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-base-2:18.2.1-325.g0fb32788.el8.x86_64 105/161 2024-02-02T15:16:24.520 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-base-2:18.2.1-325.g0fb32788.el8.x86_64 105/161 2024-02-02T15:16:24.590 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 83/161 2024-02-02T15:16:24.713 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-selinux-2:18.2.1-325.g0fb32788.el8.x86_64 106/161 2024-02-02T15:16:24.857 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 84/161 2024-02-02T15:16:25.047 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 85/161 2024-02-02T15:16:25.247 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 86/161 2024-02-02T15:16:25.403 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 87/161 2024-02-02T15:16:25.683 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 88/161 2024-02-02T15:16:25.875 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 89/161 2024-02-02T15:16:26.053 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 90/161 2024-02-02T15:16:26.350 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-portend-2.6-1.el8.noarch 91/161 2024-02-02T15:16:26.603 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 92/161 2024-02-02T15:16:26.790 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 93/161 2024-02-02T15:16:27.008 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 94/161 2024-02-02T15:16:27.526 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-common-2:18.2.1-325.g0fb32788.el8.x86_64 104/161 2024-02-02T15:16:27.822 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-common-2:18.2.1-325.g0fb32788.el8.x86_64 104/161 2024-02-02T15:16:28.399 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-common-2:18.2.1-325.g0fb32788.el8.x86_64 104/161 2024-02-02T15:16:28.429 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 95/161 2024-02-02T15:16:28.744 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-base-2:18.2.1-325.g0fb32788.el8.x86_64 105/161 2024-02-02T15:16:28.751 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 96/161 2024-02-02T15:16:28.828 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-base-2:18.2.1-325.g0fb32788.el8.x86_64 105/161 2024-02-02T15:16:29.019 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 97/161 2024-02-02T15:16:29.021 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-selinux-2:18.2.1-325.g0fb32788.el8.x86_64 106/161 2024-02-02T15:16:29.377 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 98/161 2024-02-02T15:16:30.253 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 99/161 2024-02-02T15:16:30.496 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 100/161 2024-02-02T15:16:31.393 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 101/161 2024-02-02T15:16:31.602 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : librgw2-2:18.2.1-325.g0fb32788.el8.x86_64 102/161 2024-02-02T15:16:31.693 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: librgw2-2:18.2.1-325.g0fb32788.el8.x86_64 102/161 2024-02-02T15:16:31.870 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : python3-rgw-2:18.2.1-325.g0fb32788.el8.x86_64 103/161 2024-02-02T15:16:36.916 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-common-2:18.2.1-325.g0fb32788.el8.x86_64 104/161 2024-02-02T15:16:37.111 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-common-2:18.2.1-325.g0fb32788.el8.x86_64 104/161 2024-02-02T15:16:37.814 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-common-2:18.2.1-325.g0fb32788.el8.x86_64 104/161 2024-02-02T15:16:38.086 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-base-2:18.2.1-325.g0fb32788.el8.x86_64 105/161 2024-02-02T15:16:38.248 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-base-2:18.2.1-325.g0fb32788.el8.x86_64 105/161 2024-02-02T15:16:38.393 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-selinux-2:18.2.1-325.g0fb32788.el8.x86_64 106/161 2024-02-02T15:16:43.987 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-selinux-2:18.2.1-325.g0fb32788.el8.x86_64 106/161 2024-02-02T15:16:43.987 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /sys 2024-02-02T15:16:43.987 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /proc 2024-02-02T15:16:43.987 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /dev 2024-02-02T15:16:43.987 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /run 2024-02-02T15:16:43.987 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /mnt 2024-02-02T15:16:43.987 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /var/tmp 2024-02-02T15:16:43.988 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /home 2024-02-02T15:16:43.988 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /tmp 2024-02-02T15:16:43.988 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /dev 2024-02-02T15:16:43.988 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-02T15:16:44.367 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-volume-2:18.2.1-325.g0fb32788.el8.noarch 107/161 2024-02-02T15:16:45.933 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-volume-2:18.2.1-325.g0fb32788.el8.noarch 107/161 2024-02-02T15:16:46.166 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-osd-2:18.2.1-325.g0fb32788.el8.x86_64 108/161 2024-02-02T15:16:46.386 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-osd-2:18.2.1-325.g0fb32788.el8.x86_64 108/161 2024-02-02T15:16:46.583 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mds-2:18.2.1-325.g0fb32788.el8.x86_64 109/161 2024-02-02T15:16:47.038 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mds-2:18.2.1-325.g0fb32788.el8.x86_64 109/161 2024-02-02T15:16:47.208 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mon-2:18.2.1-325.g0fb32788.el8.x86_64 110/161 2024-02-02T15:16:47.226 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mon-2:18.2.1-325.g0fb32788.el8.x86_64 110/161 2024-02-02T15:16:48.691 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-selinux-2:18.2.1-325.g0fb32788.el8.x86_64 106/161 2024-02-02T15:16:48.691 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /sys 2024-02-02T15:16:48.691 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /proc 2024-02-02T15:16:48.691 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /dev 2024-02-02T15:16:48.692 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /run 2024-02-02T15:16:48.692 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /mnt 2024-02-02T15:16:48.692 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /var/tmp 2024-02-02T15:16:48.692 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /home 2024-02-02T15:16:48.692 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /tmp 2024-02-02T15:16:48.692 INFO:teuthology.orchestra.run.smithi142.stdout:skipping the directory /dev 2024-02-02T15:16:48.692 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-02-02T15:16:48.941 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: cephadm-2:18.2.1-325.g0fb32788.el8.noarch 111/161 2024-02-02T15:16:49.094 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-volume-2:18.2.1-325.g0fb32788.el8.noarch 107/161 2024-02-02T15:16:49.109 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : cephadm-2:18.2.1-325.g0fb32788.el8.noarch 111/161 2024-02-02T15:16:49.230 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-prometheus-alerts-2:18.2.1-325.g0fb32788.el 112/161 2024-02-02T15:16:49.448 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-grafana-dashboards-2:18.2.1-325.g0fb32788.e 113/161 2024-02-02T15:16:49.673 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-cephadm-2:18.2.1-325.g0fb32788.el8.noar 114/161 2024-02-02T15:16:49.896 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.1-325.g0fb32788.el8.noar 114/161 2024-02-02T15:16:50.181 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-dashboard-2:18.2.1-325.g0fb32788.el8.no 115/161 2024-02-02T15:16:50.667 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-volume-2:18.2.1-325.g0fb32788.el8.noarch 107/161 2024-02-02T15:16:50.883 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-osd-2:18.2.1-325.g0fb32788.el8.x86_64 108/161 2024-02-02T15:16:51.104 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-osd-2:18.2.1-325.g0fb32788.el8.x86_64 108/161 2024-02-02T15:16:51.146 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.1-325.g0fb32788.el8.no 115/161 2024-02-02T15:16:51.318 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mds-2:18.2.1-325.g0fb32788.el8.x86_64 109/161 2024-02-02T15:16:51.348 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.1-325.g0fb3 116/161 2024-02-02T15:16:51.442 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.1-325.g0fb3 116/161 2024-02-02T15:16:51.776 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mds-2:18.2.1-325.g0fb32788.el8.x86_64 109/161 2024-02-02T15:16:51.810 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-modules-core-2:18.2.1-325.g0fb32788.el8 117/161 2024-02-02T15:16:51.926 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mon-2:18.2.1-325.g0fb32788.el8.x86_64 110/161 2024-02-02T15:16:51.944 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mon-2:18.2.1-325.g0fb32788.el8.x86_64 110/161 2024-02-02T15:16:52.023 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-2:18.2.1-325.g0fb32788.el8.x86_64 118/161 2024-02-02T15:16:52.061 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mgr-2:18.2.1-325.g0fb32788.el8.x86_64 118/161 2024-02-02T15:16:52.215 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-rook-2:18.2.1-325.g0fb32788.el8.noarch 119/161 2024-02-02T15:16:52.227 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.1-325.g0fb32788.el8.noarch 119/161 2024-02-02T15:16:53.507 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-2:18.2.1-325.g0fb32788.el8.x86_64 120/161 2024-02-02T15:16:53.698 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: cephadm-2:18.2.1-325.g0fb32788.el8.noarch 111/161 2024-02-02T15:16:53.764 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-radosgw-2:18.2.1-325.g0fb32788.el8.x86_64 121/161 2024-02-02T15:16:53.799 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-radosgw-2:18.2.1-325.g0fb32788.el8.x86_64 121/161 2024-02-02T15:16:53.885 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : cephadm-2:18.2.1-325.g0fb32788.el8.noarch 111/161 2024-02-02T15:16:53.957 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-immutable-object-cache-2:18.2.1-325.g0fb327 122/161 2024-02-02T15:16:54.000 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.1-325.g0fb327 122/161 2024-02-02T15:16:54.048 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-prometheus-alerts-2:18.2.1-325.g0fb32788.el 112/161 2024-02-02T15:16:54.164 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : cephfs-mirror-2:18.2.1-325.g0fb32788.el8.x86_64 123/161 2024-02-02T15:16:54.235 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-grafana-dashboards-2:18.2.1-325.g0fb32788.e 113/161 2024-02-02T15:16:54.510 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: cephfs-mirror-2:18.2.1-325.g0fb32788.el8.x86_64 123/161 2024-02-02T15:16:54.525 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-cephadm-2:18.2.1-325.g0fb32788.el8.noar 114/161 2024-02-02T15:16:54.723 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : rbd-mirror-2:18.2.1-325.g0fb32788.el8.x86_64 124/161 2024-02-02T15:16:54.749 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.1-325.g0fb32788.el8.noar 114/161 2024-02-02T15:16:55.023 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-dashboard-2:18.2.1-325.g0fb32788.el8.no 115/161 2024-02-02T15:16:55.989 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.1-325.g0fb32788.el8.no 115/161 2024-02-02T15:16:56.200 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.1-325.g0fb3 116/161 2024-02-02T15:16:56.294 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.1-325.g0fb3 116/161 2024-02-02T15:16:56.694 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-modules-core-2:18.2.1-325.g0fb32788.el8 117/161 2024-02-02T15:16:56.915 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-2:18.2.1-325.g0fb32788.el8.x86_64 118/161 2024-02-02T15:16:56.953 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-2:18.2.1-325.g0fb32788.el8.x86_64 118/161 2024-02-02T15:16:57.133 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-mgr-rook-2:18.2.1-325.g0fb32788.el8.noarch 119/161 2024-02-02T15:16:57.148 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.1-325.g0fb32788.el8.noarch 119/161 2024-02-02T15:16:58.441 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-2:18.2.1-325.g0fb32788.el8.x86_64 120/161 2024-02-02T15:16:58.681 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-radosgw-2:18.2.1-325.g0fb32788.el8.x86_64 121/161 2024-02-02T15:16:58.716 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-radosgw-2:18.2.1-325.g0fb32788.el8.x86_64 121/161 2024-02-02T15:16:58.925 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-immutable-object-cache-2:18.2.1-325.g0fb327 122/161 2024-02-02T15:16:58.968 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.1-325.g0fb327 122/161 2024-02-02T15:16:59.145 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: rbd-mirror-2:18.2.1-325.g0fb32788.el8.x86_64 124/161 2024-02-02T15:16:59.183 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : cephfs-mirror-2:18.2.1-325.g0fb32788.el8.x86_64 123/161 2024-02-02T15:16:59.529 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: cephfs-mirror-2:18.2.1-325.g0fb32788.el8.x86_64 123/161 2024-02-02T15:16:59.742 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : rbd-mirror-2:18.2.1-325.g0fb32788.el8.x86_64 124/161 2024-02-02T15:17:00.075 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-test-2:18.2.1-325.g0fb32788.el8.x86_64 125/161 2024-02-02T15:17:00.220 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-selinux-2:18.2.1-325.g0fb32788.el8.x86_64 106/161 2024-02-02T15:17:00.220 INFO:teuthology.orchestra.run.smithi001.stdout:skipping the directory /sys 2024-02-02T15:17:00.220 INFO:teuthology.orchestra.run.smithi001.stdout:skipping the directory /proc 2024-02-02T15:17:00.221 INFO:teuthology.orchestra.run.smithi001.stdout:skipping the directory /dev 2024-02-02T15:17:00.221 INFO:teuthology.orchestra.run.smithi001.stdout:skipping the directory /run 2024-02-02T15:17:00.221 INFO:teuthology.orchestra.run.smithi001.stdout:skipping the directory /mnt 2024-02-02T15:17:00.221 INFO:teuthology.orchestra.run.smithi001.stdout:skipping the directory /var/tmp 2024-02-02T15:17:00.221 INFO:teuthology.orchestra.run.smithi001.stdout:skipping the directory /home 2024-02-02T15:17:00.221 INFO:teuthology.orchestra.run.smithi001.stdout:skipping the directory /tmp 2024-02-02T15:17:00.221 INFO:teuthology.orchestra.run.smithi001.stdout:skipping the directory /dev 2024-02-02T15:17:00.221 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:17:00.343 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : openssl-devel-1:1.1.1k-12.el8_9.x86_64 126/161 2024-02-02T15:17:00.628 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-volume-2:18.2.1-325.g0fb32788.el8.noarch 107/161 2024-02-02T15:17:00.670 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 127/161 2024-02-02T15:17:01.072 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-fuse-2:18.2.1-325.g0fb32788.el8.x86_64 128/161 2024-02-02T15:17:01.389 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : postgresql-contrib-10.23-3.module+el8.9.0+20831+ 129/161 2024-02-02T15:17:01.621 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libtool-2.4.6-25.el8.x86_64 130/161 2024-02-02T15:17:01.635 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 130/161 2024-02-02T15:17:01.841 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : util-linux-user-2.32.1-43.el8.x86_64 131/161 2024-02-02T15:17:02.051 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 132/161 2024-02-02T15:17:02.247 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libcephfs-devel-2:18.2.1-325.g0fb32788.el8.x86_6 133/161 2024-02-02T15:17:02.442 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-volume-2:18.2.1-325.g0fb32788.el8.noarch 107/161 2024-02-02T15:17:02.633 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-osd-2:18.2.1-325.g0fb32788.el8.x86_64 108/161 2024-02-02T15:17:02.913 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-osd-2:18.2.1-325.g0fb32788.el8.x86_64 108/161 2024-02-02T15:17:03.040 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-mds-2:18.2.1-325.g0fb32788.el8.x86_64 109/161 2024-02-02T15:17:03.592 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-mds-2:18.2.1-325.g0fb32788.el8.x86_64 109/161 2024-02-02T15:17:03.766 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-mon-2:18.2.1-325.g0fb32788.el8.x86_64 110/161 2024-02-02T15:17:03.809 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-mon-2:18.2.1-325.g0fb32788.el8.x86_64 110/161 2024-02-02T15:17:04.168 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: rbd-mirror-2:18.2.1-325.g0fb32788.el8.x86_64 124/161 2024-02-02T15:17:04.479 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: postgresql-server-10.23-3.module+el8.9.0+20831+7 134/161 2024-02-02T15:17:04.779 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : postgresql-server-10.23-3.module+el8.9.0+20831+7 134/161 2024-02-02T15:17:04.818 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: postgresql-server-10.23-3.module+el8.9.0+20831+7 134/161 2024-02-02T15:17:05.155 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-test-2:18.2.1-325.g0fb32788.el8.x86_64 125/161 2024-02-02T15:17:05.263 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 135/161 2024-02-02T15:17:05.461 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : openssl-devel-1:1.1.1k-12.el8_9.x86_64 126/161 2024-02-02T15:17:05.620 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : iproute-6.2.0-5.el8_9.x86_64 136/161 2024-02-02T15:17:05.756 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 127/161 2024-02-02T15:17:05.858 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : xfsdump-3.1.8-7.el8_9.x86_64 137/161 2024-02-02T15:17:06.054 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : cephfs-top-2:18.2.1-325.g0fb32788.el8.noarch 138/161 2024-02-02T15:17:06.190 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : ceph-fuse-2:18.2.1-325.g0fb32788.el8.x86_64 128/161 2024-02-02T15:17:06.270 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : rbd-fuse-2:18.2.1-325.g0fb32788.el8.x86_64 139/161 2024-02-02T15:17:06.592 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : postgresql-contrib-10.23-3.module+el8.9.0+20831+ 129/161 2024-02-02T15:17:06.592 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : rbd-nbd-2:18.2.1-325.g0fb32788.el8.x86_64 140/161 2024-02-02T15:17:06.708 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: cephadm-2:18.2.1-325.g0fb32788.el8.noarch 111/161 2024-02-02T15:17:06.848 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libtool-2.4.6-25.el8.x86_64 130/161 2024-02-02T15:17:06.862 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 130/161 2024-02-02T15:17:06.873 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : openssl-1:1.1.1k-12.el8_9.x86_64 141/161 2024-02-02T15:17:06.894 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : cephadm-2:18.2.1-325.g0fb32788.el8.noarch 111/161 2024-02-02T15:17:07.023 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-prometheus-alerts-2:18.2.1-325.g0fb32788.el 112/161 2024-02-02T15:17:07.102 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : util-linux-user-2.32.1-43.el8.x86_64 131/161 2024-02-02T15:17:07.103 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : flex-2.6.1-9.el8.x86_64 142/161 2024-02-02T15:17:07.180 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 142/161 2024-02-02T15:17:07.207 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-grafana-dashboards-2:18.2.1-325.g0fb32788.e 113/161 2024-02-02T15:17:07.312 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 132/161 2024-02-02T15:17:07.428 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : bison-3.0.4-10.el8.x86_64 143/161 2024-02-02T15:17:07.442 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 143/161 2024-02-02T15:17:07.456 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-mgr-cephadm-2:18.2.1-325.g0fb32788.el8.noar 114/161 2024-02-02T15:17:07.549 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libcephfs-devel-2:18.2.1-325.g0fb32788.el8.x86_6 133/161 2024-02-02T15:17:07.612 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : NetworkManager-initscripts-updown-1:1.40.16-13.e 144/161 2024-02-02T15:17:07.621 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-13.e 144/161 2024-02-02T15:17:07.736 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.1-325.g0fb32788.el8.noar 114/161 2024-02-02T15:17:07.747 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 145/161 2024-02-02T15:17:07.982 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : elfutils-libs-0.186-1.el8.x86_64 146/161 2024-02-02T15:17:07.998 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-mgr-dashboard-2:18.2.1-325.g0fb32788.el8.no 115/161 2024-02-02T15:17:08.172 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 147/161 2024-02-02T15:17:08.340 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 148/161 2024-02-02T15:17:08.341 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 149/161 2024-02-02T15:17:08.615 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 149/161 2024-02-02T15:17:08.615 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 150/161 2024-02-02T15:17:08.840 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 150/161 2024-02-02T15:17:08.840 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 151/161 2024-02-02T15:17:09.093 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 151/161 2024-02-02T15:17:09.155 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.1-325.g0fb32788.el8.no 115/161 2024-02-02T15:17:09.364 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.1-325.g0fb3 116/161 2024-02-02T15:17:09.446 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 152/161 2024-02-02T15:17:09.503 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.1-325.g0fb3 116/161 2024-02-02T15:17:09.614 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : util-linux-user-2.32.1-35.el8.x86_64 153/161 2024-02-02T15:17:09.850 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: postgresql-server-10.23-3.module+el8.9.0+20831+7 134/161 2024-02-02T15:17:09.863 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : util-linux-2.32.1-35.el8.x86_64 154/161 2024-02-02T15:17:09.863 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : libfdisk-2.32.1-35.el8.x86_64 155/161 2024-02-02T15:17:09.906 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-mgr-modules-core-2:18.2.1-325.g0fb32788.el8 117/161 2024-02-02T15:17:10.082 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-mgr-2:18.2.1-325.g0fb32788.el8.x86_64 118/161 2024-02-02T15:17:10.105 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: libfdisk-2.32.1-35.el8.x86_64 155/161 2024-02-02T15:17:10.105 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : libmount-2.32.1-35.el8.x86_64 156/161 2024-02-02T15:17:10.152 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-mgr-2:18.2.1-325.g0fb32788.el8.x86_64 118/161 2024-02-02T15:17:10.206 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : postgresql-server-10.23-3.module+el8.9.0+20831+7 134/161 2024-02-02T15:17:10.243 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: postgresql-server-10.23-3.module+el8.9.0+20831+7 134/161 2024-02-02T15:17:10.312 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-mgr-rook-2:18.2.1-325.g0fb32788.el8.noarch 119/161 2024-02-02T15:17:10.341 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.1-325.g0fb32788.el8.noarch 119/161 2024-02-02T15:17:10.380 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: libmount-2.32.1-35.el8.x86_64 156/161 2024-02-02T15:17:10.380 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : libblkid-2.32.1-35.el8.x86_64 157/161 2024-02-02T15:17:10.614 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: libblkid-2.32.1-35.el8.x86_64 157/161 2024-02-02T15:17:10.614 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : libuuid-2.32.1-35.el8.x86_64 158/161 2024-02-02T15:17:10.709 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 135/161 2024-02-02T15:17:10.872 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: libuuid-2.32.1-35.el8.x86_64 158/161 2024-02-02T15:17:10.872 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : libsmartcols-2.32.1-35.el8.x86_64 159/161 2024-02-02T15:17:11.113 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: libsmartcols-2.32.1-35.el8.x86_64 159/161 2024-02-02T15:17:11.138 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : iproute-6.2.0-5.el8_9.x86_64 136/161 2024-02-02T15:17:11.313 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : elfutils-libelf-0.186-1.el8.x86_64 160/161 2024-02-02T15:17:11.314 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 161/161 2024-02-02T15:17:11.435 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : xfsdump-3.1.8-7.el8_9.x86_64 137/161 2024-02-02T15:17:11.665 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : cephfs-top-2:18.2.1-325.g0fb32788.el8.noarch 138/161 2024-02-02T15:17:11.783 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-2:18.2.1-325.g0fb32788.el8.x86_64 120/161 2024-02-02T15:17:11.906 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : rbd-fuse-2:18.2.1-325.g0fb32788.el8.x86_64 139/161 2024-02-02T15:17:12.040 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-radosgw-2:18.2.1-325.g0fb32788.el8.x86_64 121/161 2024-02-02T15:17:12.104 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-radosgw-2:18.2.1-325.g0fb32788.el8.x86_64 121/161 2024-02-02T15:17:12.193 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 161/161 2024-02-02T15:17:12.193 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-2:18.2.1-325.g0fb32788.el8.x86_64 1/161 2024-02-02T15:17:12.193 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-base-2:18.2.1-325.g0fb32788.el8.x86_64 2/161 2024-02-02T15:17:12.193 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-common-2:18.2.1-325.g0fb32788.el8.x86_64 3/161 2024-02-02T15:17:12.193 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-fuse-2:18.2.1-325.g0fb32788.el8.x86_64 4/161 2024-02-02T15:17:12.193 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-immutable-object-cache-2:18.2.1-325.g0fb327 5/161 2024-02-02T15:17:12.193 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mds-2:18.2.1-325.g0fb32788.el8.x86_64 6/161 2024-02-02T15:17:12.194 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-2:18.2.1-325.g0fb32788.el8.x86_64 7/161 2024-02-02T15:17:12.194 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mon-2:18.2.1-325.g0fb32788.el8.x86_64 8/161 2024-02-02T15:17:12.194 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-osd-2:18.2.1-325.g0fb32788.el8.x86_64 9/161 2024-02-02T15:17:12.194 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-radosgw-2:18.2.1-325.g0fb32788.el8.x86_64 10/161 2024-02-02T15:17:12.194 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-selinux-2:18.2.1-325.g0fb32788.el8.x86_64 11/161 2024-02-02T15:17:12.194 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-test-2:18.2.1-325.g0fb32788.el8.x86_64 12/161 2024-02-02T15:17:12.194 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : cephfs-mirror-2:18.2.1-325.g0fb32788.el8.x86_64 13/161 2024-02-02T15:17:12.194 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libcephfs-devel-2:18.2.1-325.g0fb32788.el8.x86_6 14/161 2024-02-02T15:17:12.194 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libcephfs2-2:18.2.1-325.g0fb32788.el8.x86_64 15/161 2024-02-02T15:17:12.194 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libcephsqlite-2:18.2.1-325.g0fb32788.el8.x86_64 16/161 2024-02-02T15:17:12.194 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librados-devel-2:18.2.1-325.g0fb32788.el8.x86_64 17/161 2024-02-02T15:17:12.194 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libradosstriper1-2:18.2.1-325.g0fb32788.el8.x86_ 18/161 2024-02-02T15:17:12.194 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librgw2-2:18.2.1-325.g0fb32788.el8.x86_64 19/161 2024-02-02T15:17:12.194 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-ceph-argparse-2:18.2.1-325.g0fb32788.el8 20/161 2024-02-02T15:17:12.194 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-ceph-common-2:18.2.1-325.g0fb32788.el8.x 21/161 2024-02-02T15:17:12.195 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cephfs-2:18.2.1-325.g0fb32788.el8.x86_64 22/161 2024-02-02T15:17:12.195 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-rados-2:18.2.1-325.g0fb32788.el8.x86_64 23/161 2024-02-02T15:17:12.195 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-rbd-2:18.2.1-325.g0fb32788.el8.x86_64 24/161 2024-02-02T15:17:12.195 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-rgw-2:18.2.1-325.g0fb32788.el8.x86_64 25/161 2024-02-02T15:17:12.195 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : rbd-fuse-2:18.2.1-325.g0fb32788.el8.x86_64 26/161 2024-02-02T15:17:12.195 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : rbd-mirror-2:18.2.1-325.g0fb32788.el8.x86_64 27/161 2024-02-02T15:17:12.195 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : rbd-nbd-2:18.2.1-325.g0fb32788.el8.x86_64 28/161 2024-02-02T15:17:12.195 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-grafana-dashboards-2:18.2.1-325.g0fb32788.e 29/161 2024-02-02T15:17:12.208 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-cephadm-2:18.2.1-325.g0fb32788.el8.noar 30/161 2024-02-02T15:17:12.208 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-dashboard-2:18.2.1-325.g0fb32788.el8.no 31/161 2024-02-02T15:17:12.208 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.1-325.g0fb3 32/161 2024-02-02T15:17:12.208 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-modules-core-2:18.2.1-325.g0fb32788.el8 33/161 2024-02-02T15:17:12.208 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-rook-2:18.2.1-325.g0fb32788.el8.noarch 34/161 2024-02-02T15:17:12.208 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-prometheus-alerts-2:18.2.1-325.g0fb32788.el 35/161 2024-02-02T15:17:12.208 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-volume-2:18.2.1-325.g0fb32788.el8.noarch 36/161 2024-02-02T15:17:12.208 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : cephadm-2:18.2.1-325.g0fb32788.el8.noarch 37/161 2024-02-02T15:17:12.208 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : cephfs-top-2:18.2.1-325.g0fb32788.el8.noarch 38/161 2024-02-02T15:17:12.208 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/161 2024-02-02T15:17:12.208 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/161 2024-02-02T15:17:12.209 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/161 2024-02-02T15:17:12.209 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/161 2024-02-02T15:17:12.209 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/161 2024-02-02T15:17:12.209 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/161 2024-02-02T15:17:12.209 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/161 2024-02-02T15:17:12.209 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/161 2024-02-02T15:17:12.209 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/161 2024-02-02T15:17:12.209 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/161 2024-02-02T15:17:12.209 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/161 2024-02-02T15:17:12.209 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/161 2024-02-02T15:17:12.209 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/161 2024-02-02T15:17:12.209 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/161 2024-02-02T15:17:12.209 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/161 2024-02-02T15:17:12.209 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/161 2024-02-02T15:17:12.210 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/161 2024-02-02T15:17:12.210 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/161 2024-02-02T15:17:12.210 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/161 2024-02-02T15:17:12.210 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/161 2024-02-02T15:17:12.210 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/161 2024-02-02T15:17:12.210 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/161 2024-02-02T15:17:12.210 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/161 2024-02-02T15:17:12.210 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/161 2024-02-02T15:17:12.210 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/161 2024-02-02T15:17:12.210 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/161 2024-02-02T15:17:12.210 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/161 2024-02-02T15:17:12.210 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/161 2024-02-02T15:17:12.210 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/161 2024-02-02T15:17:12.211 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/161 2024-02-02T15:17:12.211 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/161 2024-02-02T15:17:12.211 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/161 2024-02-02T15:17:12.211 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/161 2024-02-02T15:17:12.211 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 72/161 2024-02-02T15:17:12.211 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 73/161 2024-02-02T15:17:12.211 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 74/161 2024-02-02T15:17:12.211 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 75/161 2024-02-02T15:17:12.211 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 76/161 2024-02-02T15:17:12.211 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 77/161 2024-02-02T15:17:12.211 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 78/161 2024-02-02T15:17:12.211 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 79/161 2024-02-02T15:17:12.211 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/161 2024-02-02T15:17:12.211 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 81/161 2024-02-02T15:17:12.212 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 82/161 2024-02-02T15:17:12.212 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 83/161 2024-02-02T15:17:12.212 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 84/161 2024-02-02T15:17:12.212 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 85/161 2024-02-02T15:17:12.212 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 86/161 2024-02-02T15:17:12.212 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 87/161 2024-02-02T15:17:12.212 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 88/161 2024-02-02T15:17:12.212 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 89/161 2024-02-02T15:17:12.212 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libuuid-devel-2.32.1-43.el8.x86_64 90/161 2024-02-02T15:17:12.212 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 91/161 2024-02-02T15:17:12.212 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cryptography-3.2.1-6.el8.x86_64 92/161 2024-02-02T15:17:12.212 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 93/161 2024-02-02T15:17:12.212 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-13.e 94/161 2024-02-02T15:17:12.212 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 95/161 2024-02-02T15:17:12.213 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : krb5-devel-1.18.2-26.el8_9.x86_64 96/161 2024-02-02T15:17:12.213 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librabbitmq-0.9.0-4.el8.x86_64 97/161 2024-02-02T15:17:12.213 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : openssl-devel-1:1.1.1k-12.el8_9.x86_64 98/161 2024-02-02T15:17:12.213 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 99/161 2024-02-02T15:17:12.213 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 100/161 2024-02-02T15:17:12.213 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 101/161 2024-02-02T15:17:12.213 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 102/161 2024-02-02T15:17:12.213 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 103/161 2024-02-02T15:17:12.213 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 104/161 2024-02-02T15:17:12.213 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : bison-3.0.4-10.el8.x86_64 105/161 2024-02-02T15:17:12.213 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : flex-2.6.1-9.el8.x86_64 106/161 2024-02-02T15:17:12.213 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 107/161 2024-02-02T15:17:12.213 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 108/161 2024-02-02T15:17:12.213 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 109/161 2024-02-02T15:17:12.214 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 110/161 2024-02-02T15:17:12.214 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 111/161 2024-02-02T15:17:12.214 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 112/161 2024-02-02T15:17:12.214 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 113/161 2024-02-02T15:17:12.214 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 114/161 2024-02-02T15:17:12.214 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 115/161 2024-02-02T15:17:12.214 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 116/161 2024-02-02T15:17:12.214 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 117/161 2024-02-02T15:17:12.214 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 118/161 2024-02-02T15:17:12.214 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 119/161 2024-02-02T15:17:12.214 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 120/161 2024-02-02T15:17:12.214 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 121/161 2024-02-02T15:17:12.214 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : automake-1.16.1-8.el8.noarch 122/161 2024-02-02T15:17:12.214 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libpq-13.11-1.el8.x86_64 123/161 2024-02-02T15:17:12.214 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : jq-1.6-7.el8.x86_64 124/161 2024-02-02T15:17:12.215 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 125/161 2024-02-02T15:17:12.215 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : postgresql-10.23-3.module+el8.9.0+20831+78489779 126/161 2024-02-02T15:17:12.215 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : postgresql-contrib-10.23-3.module+el8.9.0+20831+ 127/161 2024-02-02T15:17:12.215 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : postgresql-server-10.23-3.module+el8.9.0+20831+7 128/161 2024-02-02T15:17:12.215 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 129/161 2024-02-02T15:17:12.215 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librados2-2:18.2.1-325.g0fb32788.el8.x86_64 130/161 2024-02-02T15:17:12.215 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 131/161 2024-02-02T15:17:12.215 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librbd1-2:18.2.1-325.g0fb32788.el8.x86_64 132/161 2024-02-02T15:17:12.215 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 133/161 2024-02-02T15:17:12.215 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 134/161 2024-02-02T15:17:12.215 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : elfutils-libelf-0.186-1.el8.x86_64 135/161 2024-02-02T15:17:12.215 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 136/161 2024-02-02T15:17:12.215 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : elfutils-libs-0.186-1.el8.x86_64 137/161 2024-02-02T15:17:12.215 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libmount-2.32.1-43.el8.x86_64 138/161 2024-02-02T15:17:12.215 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libmount-2.32.1-35.el8.x86_64 139/161 2024-02-02T15:17:12.216 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libsmartcols-2.32.1-43.el8.x86_64 140/161 2024-02-02T15:17:12.216 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libsmartcols-2.32.1-35.el8.x86_64 141/161 2024-02-02T15:17:12.216 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libuuid-2.32.1-43.el8.x86_64 142/161 2024-02-02T15:17:12.216 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libuuid-2.32.1-35.el8.x86_64 143/161 2024-02-02T15:17:12.216 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : util-linux-2.32.1-43.el8.x86_64 144/161 2024-02-02T15:17:12.216 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : util-linux-2.32.1-35.el8.x86_64 145/161 2024-02-02T15:17:12.216 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : util-linux-user-2.32.1-43.el8.x86_64 146/161 2024-02-02T15:17:12.216 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : util-linux-user-2.32.1-35.el8.x86_64 147/161 2024-02-02T15:17:12.216 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : iproute-6.2.0-5.el8_9.x86_64 148/161 2024-02-02T15:17:12.216 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 149/161 2024-02-02T15:17:12.216 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libblkid-2.32.1-43.el8.x86_64 150/161 2024-02-02T15:17:12.216 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libblkid-2.32.1-35.el8.x86_64 151/161 2024-02-02T15:17:12.216 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libfdisk-2.32.1-43.el8.x86_64 152/161 2024-02-02T15:17:12.216 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libfdisk-2.32.1-35.el8.x86_64 153/161 2024-02-02T15:17:12.217 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : openssl-1:1.1.1k-12.el8_9.x86_64 154/161 2024-02-02T15:17:12.217 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 155/161 2024-02-02T15:17:12.217 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : openssl-libs-1:1.1.1k-12.el8_9.x86_64 156/161 2024-02-02T15:17:12.217 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 157/161 2024-02-02T15:17:12.217 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : xfsdump-3.1.8-7.el8_9.x86_64 158/161 2024-02-02T15:17:12.217 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 159/161 2024-02-02T15:17:12.217 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 160/161 2024-02-02T15:17:12.224 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : rbd-nbd-2:18.2.1-325.g0fb32788.el8.x86_64 140/161 2024-02-02T15:17:12.254 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-immutable-object-cache-2:18.2.1-325.g0fb327 122/161 2024-02-02T15:17:12.326 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.1-325.g0fb327 122/161 2024-02-02T15:17:12.531 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : cephfs-mirror-2:18.2.1-325.g0fb32788.el8.x86_64 123/161 2024-02-02T15:17:12.543 INFO:teuthology.orchestra.run.smithi142.stdout: Upgrading : openssl-1:1.1.1k-12.el8_9.x86_64 141/161 2024-02-02T15:17:12.805 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : flex-2.6.1-9.el8.x86_64 142/161 2024-02-02T15:17:12.885 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 142/161 2024-02-02T15:17:12.957 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: cephfs-mirror-2:18.2.1-325.g0fb32788.el8.x86_64 123/161 2024-02-02T15:17:13.123 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : rbd-mirror-2:18.2.1-325.g0fb32788.el8.x86_64 124/161 2024-02-02T15:17:13.130 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : bison-3.0.4-10.el8.x86_64 143/161 2024-02-02T15:17:13.145 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 143/161 2024-02-02T15:17:13.347 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : NetworkManager-initscripts-updown-1:1.40.16-13.e 144/161 2024-02-02T15:17:13.358 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-13.e 144/161 2024-02-02T15:17:13.541 INFO:teuthology.orchestra.run.smithi142.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 145/161 2024-02-02T15:17:13.766 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : elfutils-libs-0.186-1.el8.x86_64 146/161 2024-02-02T15:17:13.974 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 147/161 2024-02-02T15:17:14.143 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 148/161 2024-02-02T15:17:14.143 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 149/161 2024-02-02T15:17:14.492 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 149/161 2024-02-02T15:17:14.492 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 150/161 2024-02-02T15:17:14.768 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 150/161 2024-02-02T15:17:14.768 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 151/161 2024-02-02T15:17:15.045 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 151/161 2024-02-02T15:17:15.290 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 152/161 2024-02-02T15:17:15.507 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : util-linux-user-2.32.1-35.el8.x86_64 153/161 2024-02-02T15:17:15.799 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : util-linux-2.32.1-35.el8.x86_64 154/161 2024-02-02T15:17:15.799 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : libfdisk-2.32.1-35.el8.x86_64 155/161 2024-02-02T15:17:16.074 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libfdisk-2.32.1-35.el8.x86_64 155/161 2024-02-02T15:17:16.074 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : libmount-2.32.1-35.el8.x86_64 156/161 2024-02-02T15:17:16.358 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libmount-2.32.1-35.el8.x86_64 156/161 2024-02-02T15:17:16.358 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : libblkid-2.32.1-35.el8.x86_64 157/161 2024-02-02T15:17:16.649 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libblkid-2.32.1-35.el8.x86_64 157/161 2024-02-02T15:17:16.649 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : libuuid-2.32.1-35.el8.x86_64 158/161 2024-02-02T15:17:16.957 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libuuid-2.32.1-35.el8.x86_64 158/161 2024-02-02T15:17:16.958 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : libsmartcols-2.32.1-35.el8.x86_64 159/161 2024-02-02T15:17:17.216 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: libsmartcols-2.32.1-35.el8.x86_64 159/161 2024-02-02T15:17:17.424 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : elfutils-libelf-0.186-1.el8.x86_64 160/161 2024-02-02T15:17:17.425 INFO:teuthology.orchestra.run.smithi142.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 161/161 2024-02-02T15:17:18.258 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: rbd-mirror-2:18.2.1-325.g0fb32788.el8.x86_64 124/161 2024-02-02T15:17:18.393 INFO:teuthology.orchestra.run.smithi142.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 161/161 2024-02-02T15:17:18.393 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-2:18.2.1-325.g0fb32788.el8.x86_64 1/161 2024-02-02T15:17:18.393 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-base-2:18.2.1-325.g0fb32788.el8.x86_64 2/161 2024-02-02T15:17:18.394 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-common-2:18.2.1-325.g0fb32788.el8.x86_64 3/161 2024-02-02T15:17:18.394 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-fuse-2:18.2.1-325.g0fb32788.el8.x86_64 4/161 2024-02-02T15:17:18.394 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-immutable-object-cache-2:18.2.1-325.g0fb327 5/161 2024-02-02T15:17:18.394 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mds-2:18.2.1-325.g0fb32788.el8.x86_64 6/161 2024-02-02T15:17:18.394 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-2:18.2.1-325.g0fb32788.el8.x86_64 7/161 2024-02-02T15:17:18.394 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mon-2:18.2.1-325.g0fb32788.el8.x86_64 8/161 2024-02-02T15:17:18.394 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-osd-2:18.2.1-325.g0fb32788.el8.x86_64 9/161 2024-02-02T15:17:18.394 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-radosgw-2:18.2.1-325.g0fb32788.el8.x86_64 10/161 2024-02-02T15:17:18.394 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-selinux-2:18.2.1-325.g0fb32788.el8.x86_64 11/161 2024-02-02T15:17:18.395 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-test-2:18.2.1-325.g0fb32788.el8.x86_64 12/161 2024-02-02T15:17:18.395 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : cephfs-mirror-2:18.2.1-325.g0fb32788.el8.x86_64 13/161 2024-02-02T15:17:18.395 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libcephfs-devel-2:18.2.1-325.g0fb32788.el8.x86_6 14/161 2024-02-02T15:17:18.395 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libcephfs2-2:18.2.1-325.g0fb32788.el8.x86_64 15/161 2024-02-02T15:17:18.395 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libcephsqlite-2:18.2.1-325.g0fb32788.el8.x86_64 16/161 2024-02-02T15:17:18.395 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librados-devel-2:18.2.1-325.g0fb32788.el8.x86_64 17/161 2024-02-02T15:17:18.395 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libradosstriper1-2:18.2.1-325.g0fb32788.el8.x86_ 18/161 2024-02-02T15:17:18.395 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librgw2-2:18.2.1-325.g0fb32788.el8.x86_64 19/161 2024-02-02T15:17:18.395 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-ceph-argparse-2:18.2.1-325.g0fb32788.el8 20/161 2024-02-02T15:17:18.395 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-ceph-common-2:18.2.1-325.g0fb32788.el8.x 21/161 2024-02-02T15:17:18.396 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cephfs-2:18.2.1-325.g0fb32788.el8.x86_64 22/161 2024-02-02T15:17:18.396 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-rados-2:18.2.1-325.g0fb32788.el8.x86_64 23/161 2024-02-02T15:17:18.396 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-rbd-2:18.2.1-325.g0fb32788.el8.x86_64 24/161 2024-02-02T15:17:18.396 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-rgw-2:18.2.1-325.g0fb32788.el8.x86_64 25/161 2024-02-02T15:17:18.396 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : rbd-fuse-2:18.2.1-325.g0fb32788.el8.x86_64 26/161 2024-02-02T15:17:18.396 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : rbd-mirror-2:18.2.1-325.g0fb32788.el8.x86_64 27/161 2024-02-02T15:17:18.396 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : rbd-nbd-2:18.2.1-325.g0fb32788.el8.x86_64 28/161 2024-02-02T15:17:18.396 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-grafana-dashboards-2:18.2.1-325.g0fb32788.e 29/161 2024-02-02T15:17:18.396 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-cephadm-2:18.2.1-325.g0fb32788.el8.noar 30/161 2024-02-02T15:17:18.397 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-dashboard-2:18.2.1-325.g0fb32788.el8.no 31/161 2024-02-02T15:17:18.397 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.1-325.g0fb3 32/161 2024-02-02T15:17:18.397 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-modules-core-2:18.2.1-325.g0fb32788.el8 33/161 2024-02-02T15:17:18.397 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-mgr-rook-2:18.2.1-325.g0fb32788.el8.noarch 34/161 2024-02-02T15:17:18.397 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-prometheus-alerts-2:18.2.1-325.g0fb32788.el 35/161 2024-02-02T15:17:18.397 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : ceph-volume-2:18.2.1-325.g0fb32788.el8.noarch 36/161 2024-02-02T15:17:18.397 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : cephadm-2:18.2.1-325.g0fb32788.el8.noarch 37/161 2024-02-02T15:17:18.397 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : cephfs-top-2:18.2.1-325.g0fb32788.el8.noarch 38/161 2024-02-02T15:17:18.397 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/161 2024-02-02T15:17:18.397 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/161 2024-02-02T15:17:18.397 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/161 2024-02-02T15:17:18.398 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/161 2024-02-02T15:17:18.398 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/161 2024-02-02T15:17:18.398 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/161 2024-02-02T15:17:18.398 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/161 2024-02-02T15:17:18.398 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/161 2024-02-02T15:17:18.398 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/161 2024-02-02T15:17:18.398 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/161 2024-02-02T15:17:18.398 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/161 2024-02-02T15:17:18.399 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/161 2024-02-02T15:17:18.399 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/161 2024-02-02T15:17:18.399 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/161 2024-02-02T15:17:18.399 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/161 2024-02-02T15:17:18.399 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/161 2024-02-02T15:17:18.399 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/161 2024-02-02T15:17:18.399 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/161 2024-02-02T15:17:18.400 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/161 2024-02-02T15:17:18.400 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/161 2024-02-02T15:17:18.400 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/161 2024-02-02T15:17:18.400 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/161 2024-02-02T15:17:18.400 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/161 2024-02-02T15:17:18.400 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/161 2024-02-02T15:17:18.400 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/161 2024-02-02T15:17:18.400 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/161 2024-02-02T15:17:18.400 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/161 2024-02-02T15:17:18.400 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/161 2024-02-02T15:17:18.401 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/161 2024-02-02T15:17:18.401 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/161 2024-02-02T15:17:18.401 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/161 2024-02-02T15:17:18.401 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/161 2024-02-02T15:17:18.401 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/161 2024-02-02T15:17:18.401 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 72/161 2024-02-02T15:17:18.401 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 73/161 2024-02-02T15:17:18.401 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 74/161 2024-02-02T15:17:18.401 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 75/161 2024-02-02T15:17:18.401 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 76/161 2024-02-02T15:17:18.402 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 77/161 2024-02-02T15:17:18.402 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 78/161 2024-02-02T15:17:18.402 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 79/161 2024-02-02T15:17:18.402 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/161 2024-02-02T15:17:18.402 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 81/161 2024-02-02T15:17:18.402 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 82/161 2024-02-02T15:17:18.402 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 83/161 2024-02-02T15:17:18.403 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 84/161 2024-02-02T15:17:18.403 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 85/161 2024-02-02T15:17:18.403 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 86/161 2024-02-02T15:17:18.403 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 87/161 2024-02-02T15:17:18.403 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 88/161 2024-02-02T15:17:18.403 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 89/161 2024-02-02T15:17:18.403 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libuuid-devel-2.32.1-43.el8.x86_64 90/161 2024-02-02T15:17:18.403 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 91/161 2024-02-02T15:17:18.403 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-cryptography-3.2.1-6.el8.x86_64 92/161 2024-02-02T15:17:18.403 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 93/161 2024-02-02T15:17:18.403 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-13.e 94/161 2024-02-02T15:17:18.404 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 95/161 2024-02-02T15:17:18.404 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : krb5-devel-1.18.2-26.el8_9.x86_64 96/161 2024-02-02T15:17:18.404 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librabbitmq-0.9.0-4.el8.x86_64 97/161 2024-02-02T15:17:18.404 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openssl-devel-1:1.1.1k-12.el8_9.x86_64 98/161 2024-02-02T15:17:18.404 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 99/161 2024-02-02T15:17:18.404 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 100/161 2024-02-02T15:17:18.404 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 101/161 2024-02-02T15:17:18.405 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 102/161 2024-02-02T15:17:18.405 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 103/161 2024-02-02T15:17:18.405 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 104/161 2024-02-02T15:17:18.405 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : bison-3.0.4-10.el8.x86_64 105/161 2024-02-02T15:17:18.405 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : flex-2.6.1-9.el8.x86_64 106/161 2024-02-02T15:17:18.405 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 107/161 2024-02-02T15:17:18.405 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 108/161 2024-02-02T15:17:18.405 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 109/161 2024-02-02T15:17:18.405 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 110/161 2024-02-02T15:17:18.405 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 111/161 2024-02-02T15:17:18.405 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 112/161 2024-02-02T15:17:18.406 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 113/161 2024-02-02T15:17:18.406 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 114/161 2024-02-02T15:17:18.406 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 115/161 2024-02-02T15:17:18.406 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 116/161 2024-02-02T15:17:18.406 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 117/161 2024-02-02T15:17:18.406 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 118/161 2024-02-02T15:17:18.406 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 119/161 2024-02-02T15:17:18.406 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 120/161 2024-02-02T15:17:18.407 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 121/161 2024-02-02T15:17:18.407 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : automake-1.16.1-8.el8.noarch 122/161 2024-02-02T15:17:18.407 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libpq-13.11-1.el8.x86_64 123/161 2024-02-02T15:17:18.407 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : jq-1.6-7.el8.x86_64 124/161 2024-02-02T15:17:18.407 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 125/161 2024-02-02T15:17:18.407 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : postgresql-10.23-3.module+el8.9.0+20831+78489779 126/161 2024-02-02T15:17:18.407 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : postgresql-contrib-10.23-3.module+el8.9.0+20831+ 127/161 2024-02-02T15:17:18.407 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : postgresql-server-10.23-3.module+el8.9.0+20831+7 128/161 2024-02-02T15:17:18.407 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 129/161 2024-02-02T15:17:18.407 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librados2-2:18.2.1-325.g0fb32788.el8.x86_64 130/161 2024-02-02T15:17:18.408 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 131/161 2024-02-02T15:17:18.408 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librbd1-2:18.2.1-325.g0fb32788.el8.x86_64 132/161 2024-02-02T15:17:18.408 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 133/161 2024-02-02T15:17:18.408 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 134/161 2024-02-02T15:17:18.408 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : elfutils-libelf-0.186-1.el8.x86_64 135/161 2024-02-02T15:17:18.408 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 136/161 2024-02-02T15:17:18.408 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : elfutils-libs-0.186-1.el8.x86_64 137/161 2024-02-02T15:17:18.408 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libmount-2.32.1-43.el8.x86_64 138/161 2024-02-02T15:17:18.408 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libmount-2.32.1-35.el8.x86_64 139/161 2024-02-02T15:17:18.408 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libsmartcols-2.32.1-43.el8.x86_64 140/161 2024-02-02T15:17:18.408 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libsmartcols-2.32.1-35.el8.x86_64 141/161 2024-02-02T15:17:18.409 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libuuid-2.32.1-43.el8.x86_64 142/161 2024-02-02T15:17:18.409 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libuuid-2.32.1-35.el8.x86_64 143/161 2024-02-02T15:17:18.409 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : util-linux-2.32.1-43.el8.x86_64 144/161 2024-02-02T15:17:18.409 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : util-linux-2.32.1-35.el8.x86_64 145/161 2024-02-02T15:17:18.409 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : util-linux-user-2.32.1-43.el8.x86_64 146/161 2024-02-02T15:17:18.409 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : util-linux-user-2.32.1-35.el8.x86_64 147/161 2024-02-02T15:17:18.409 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : iproute-6.2.0-5.el8_9.x86_64 148/161 2024-02-02T15:17:18.409 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 149/161 2024-02-02T15:17:18.409 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libblkid-2.32.1-43.el8.x86_64 150/161 2024-02-02T15:17:18.409 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libblkid-2.32.1-35.el8.x86_64 151/161 2024-02-02T15:17:18.409 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libfdisk-2.32.1-43.el8.x86_64 152/161 2024-02-02T15:17:18.409 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : libfdisk-2.32.1-35.el8.x86_64 153/161 2024-02-02T15:17:18.410 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openssl-1:1.1.1k-12.el8_9.x86_64 154/161 2024-02-02T15:17:18.410 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 155/161 2024-02-02T15:17:18.410 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openssl-libs-1:1.1.1k-12.el8_9.x86_64 156/161 2024-02-02T15:17:18.410 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 157/161 2024-02-02T15:17:18.410 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : xfsdump-3.1.8-7.el8_9.x86_64 158/161 2024-02-02T15:17:18.410 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 159/161 2024-02-02T15:17:18.410 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 160/161 2024-02-02T15:17:18.524 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 161/161 2024-02-02T15:17:18.524 INFO:teuthology.orchestra.run.smithi071.stdout:Installed products updated. 2024-02-02T15:17:19.103 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-02T15:17:19.103 INFO:teuthology.orchestra.run.smithi071.stdout:Upgraded: 2024-02-02T15:17:19.103 INFO:teuthology.orchestra.run.smithi071.stdout: elfutils-libelf-0.189-3.el8.x86_64 2024-02-02T15:17:19.103 INFO:teuthology.orchestra.run.smithi071.stdout: elfutils-libs-0.189-3.el8.x86_64 2024-02-02T15:17:19.103 INFO:teuthology.orchestra.run.smithi071.stdout: iproute-6.2.0-5.el8_9.x86_64 2024-02-02T15:17:19.104 INFO:teuthology.orchestra.run.smithi071.stdout: libblkid-2.32.1-43.el8.x86_64 2024-02-02T15:17:19.104 INFO:teuthology.orchestra.run.smithi071.stdout: libfdisk-2.32.1-43.el8.x86_64 2024-02-02T15:17:19.104 INFO:teuthology.orchestra.run.smithi071.stdout: libmount-2.32.1-43.el8.x86_64 2024-02-02T15:17:19.104 INFO:teuthology.orchestra.run.smithi071.stdout: librados2-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:19.104 INFO:teuthology.orchestra.run.smithi071.stdout: librbd1-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:19.104 INFO:teuthology.orchestra.run.smithi071.stdout: libsmartcols-2.32.1-43.el8.x86_64 2024-02-02T15:17:19.104 INFO:teuthology.orchestra.run.smithi071.stdout: libuuid-2.32.1-43.el8.x86_64 2024-02-02T15:17:19.104 INFO:teuthology.orchestra.run.smithi071.stdout: openssl-1:1.1.1k-12.el8_9.x86_64 2024-02-02T15:17:19.104 INFO:teuthology.orchestra.run.smithi071.stdout: openssl-libs-1:1.1.1k-12.el8_9.x86_64 2024-02-02T15:17:19.104 INFO:teuthology.orchestra.run.smithi071.stdout: util-linux-2.32.1-43.el8.x86_64 2024-02-02T15:17:19.104 INFO:teuthology.orchestra.run.smithi071.stdout: util-linux-user-2.32.1-43.el8.x86_64 2024-02-02T15:17:19.104 INFO:teuthology.orchestra.run.smithi071.stdout: xfsdump-3.1.8-7.el8_9.x86_64 2024-02-02T15:17:19.104 INFO:teuthology.orchestra.run.smithi071.stdout: xfsprogs-5.0.0-12.el8.x86_64 2024-02-02T15:17:19.104 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-02-02T15:17:19.105 INFO:teuthology.orchestra.run.smithi071.stdout: NetworkManager-initscripts-updown-1:1.40.16-13.el8_9.noarch 2024-02-02T15:17:19.105 INFO:teuthology.orchestra.run.smithi071.stdout: automake-1.16.1-8.el8.noarch 2024-02-02T15:17:19.105 INFO:teuthology.orchestra.run.smithi071.stdout: bison-3.0.4-10.el8.x86_64 2024-02-02T15:17:19.105 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:19.105 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-base-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:19.105 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-common-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:19.105 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-fuse-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:19.105 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-grafana-dashboards-2:18.2.1-325.g0fb32788.el8.noarch 2024-02-02T15:17:19.105 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-immutable-object-cache-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:19.105 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mds-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:19.105 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:19.105 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-cephadm-2:18.2.1-325.g0fb32788.el8.noarch 2024-02-02T15:17:19.105 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-dashboard-2:18.2.1-325.g0fb32788.el8.noarch 2024-02-02T15:17:19.105 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-diskprediction-local-2:18.2.1-325.g0fb32788.el8.noarch 2024-02-02T15:17:19.105 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-modules-core-2:18.2.1-325.g0fb32788.el8.noarch 2024-02-02T15:17:19.106 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-rook-2:18.2.1-325.g0fb32788.el8.noarch 2024-02-02T15:17:19.106 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mon-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:19.106 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-osd-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:19.106 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-prometheus-alerts-2:18.2.1-325.g0fb32788.el8.noarch 2024-02-02T15:17:19.106 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-radosgw-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:19.106 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-selinux-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:19.106 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-test-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:19.106 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-volume-2:18.2.1-325.g0fb32788.el8.noarch 2024-02-02T15:17:19.106 INFO:teuthology.orchestra.run.smithi071.stdout: cephadm-2:18.2.1-325.g0fb32788.el8.noarch 2024-02-02T15:17:19.106 INFO:teuthology.orchestra.run.smithi071.stdout: cephfs-mirror-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:19.106 INFO:teuthology.orchestra.run.smithi071.stdout: cephfs-top-2:18.2.1-325.g0fb32788.el8.noarch 2024-02-02T15:17:19.106 INFO:teuthology.orchestra.run.smithi071.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2024-02-02T15:17:19.106 INFO:teuthology.orchestra.run.smithi071.stdout: flex-2.6.1-9.el8.x86_64 2024-02-02T15:17:19.106 INFO:teuthology.orchestra.run.smithi071.stdout: fuse-2.9.7-17.el8.x86_64 2024-02-02T15:17:19.107 INFO:teuthology.orchestra.run.smithi071.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-02-02T15:17:19.107 INFO:teuthology.orchestra.run.smithi071.stdout: jq-1.6-7.el8.x86_64 2024-02-02T15:17:19.107 INFO:teuthology.orchestra.run.smithi071.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2024-02-02T15:17:19.107 INFO:teuthology.orchestra.run.smithi071.stdout: krb5-devel-1.18.2-26.el8_9.x86_64 2024-02-02T15:17:19.107 INFO:teuthology.orchestra.run.smithi071.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-02-02T15:17:19.107 INFO:teuthology.orchestra.run.smithi071.stdout: libcephfs-devel-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:19.107 INFO:teuthology.orchestra.run.smithi071.stdout: libcephfs2-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:19.107 INFO:teuthology.orchestra.run.smithi071.stdout: libcephsqlite-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:19.107 INFO:teuthology.orchestra.run.smithi071.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2024-02-02T15:17:19.107 INFO:teuthology.orchestra.run.smithi071.stdout: libgfortran-8.5.0-20.el8.x86_64 2024-02-02T15:17:19.107 INFO:teuthology.orchestra.run.smithi071.stdout: liboath-2.6.2-3.el8.x86_64 2024-02-02T15:17:19.107 INFO:teuthology.orchestra.run.smithi071.stdout: libpq-13.11-1.el8.x86_64 2024-02-02T15:17:19.107 INFO:teuthology.orchestra.run.smithi071.stdout: libquadmath-8.5.0-20.el8.x86_64 2024-02-02T15:17:19.107 INFO:teuthology.orchestra.run.smithi071.stdout: librabbitmq-0.9.0-4.el8.x86_64 2024-02-02T15:17:19.107 INFO:teuthology.orchestra.run.smithi071.stdout: librados-devel-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:19.108 INFO:teuthology.orchestra.run.smithi071.stdout: libradosstriper1-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:19.108 INFO:teuthology.orchestra.run.smithi071.stdout: librdkafka-0.11.4-3.el8.x86_64 2024-02-02T15:17:19.108 INFO:teuthology.orchestra.run.smithi071.stdout: librgw2-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:19.108 INFO:teuthology.orchestra.run.smithi071.stdout: libselinux-devel-2.9-5.el8.x86_64 2024-02-02T15:17:19.108 INFO:teuthology.orchestra.run.smithi071.stdout: libsepol-devel-2.9-3.el8.x86_64 2024-02-02T15:17:19.108 INFO:teuthology.orchestra.run.smithi071.stdout: libtool-2.4.6-25.el8.x86_64 2024-02-02T15:17:19.108 INFO:teuthology.orchestra.run.smithi071.stdout: libunwind-1.3.1-3.el8.x86_64 2024-02-02T15:17:19.108 INFO:teuthology.orchestra.run.smithi071.stdout: libuuid-devel-2.32.1-43.el8.x86_64 2024-02-02T15:17:19.108 INFO:teuthology.orchestra.run.smithi071.stdout: libverto-devel-0.3.0-5.el8.x86_64 2024-02-02T15:17:19.108 INFO:teuthology.orchestra.run.smithi071.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2024-02-02T15:17:19.108 INFO:teuthology.orchestra.run.smithi071.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-02-02T15:17:19.108 INFO:teuthology.orchestra.run.smithi071.stdout: oniguruma-6.8.2-2.el8.x86_64 2024-02-02T15:17:19.108 INFO:teuthology.orchestra.run.smithi071.stdout: openblas-0.3.15-6.el8.x86_64 2024-02-02T15:17:19.108 INFO:teuthology.orchestra.run.smithi071.stdout: openblas-threads-0.3.15-6.el8.x86_64 2024-02-02T15:17:19.108 INFO:teuthology.orchestra.run.smithi071.stdout: openssl-devel-1:1.1.1k-12.el8_9.x86_64 2024-02-02T15:17:19.109 INFO:teuthology.orchestra.run.smithi071.stdout: pcre2-devel-10.32-2.el8.x86_64 2024-02-02T15:17:19.109 INFO:teuthology.orchestra.run.smithi071.stdout: pcre2-utf16-10.32-2.el8.x86_64 2024-02-02T15:17:19.109 INFO:teuthology.orchestra.run.smithi071.stdout: pcre2-utf32-10.32-2.el8.x86_64 2024-02-02T15:17:19.109 INFO:teuthology.orchestra.run.smithi071.stdout: postgresql-10.23-3.module+el8.9.0+20831+78489779.x86_64 2024-02-02T15:17:19.109 INFO:teuthology.orchestra.run.smithi071.stdout: postgresql-contrib-10.23-3.module+el8.9.0+20831+78489779.x86_64 2024-02-02T15:17:19.109 INFO:teuthology.orchestra.run.smithi071.stdout: postgresql-server-10.23-3.module+el8.9.0+20831+78489779.x86_64 2024-02-02T15:17:19.109 INFO:teuthology.orchestra.run.smithi071.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-02-02T15:17:19.109 INFO:teuthology.orchestra.run.smithi071.stdout: python3-babel-2.5.1-7.el8.noarch 2024-02-02T15:17:19.109 INFO:teuthology.orchestra.run.smithi071.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-02-02T15:17:19.109 INFO:teuthology.orchestra.run.smithi071.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-02-02T15:17:19.109 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-02-02T15:17:19.109 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ceph-argparse-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:19.109 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ceph-common-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:19.109 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cephfs-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:19.109 INFO:teuthology.orchestra.run.smithi071.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-02-02T15:17:19.110 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-02-02T15:17:19.110 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-02-02T15:17:19.110 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-02-02T15:17:19.110 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cryptography-3.2.1-6.el8.x86_64 2024-02-02T15:17:19.110 INFO:teuthology.orchestra.run.smithi071.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-02-02T15:17:19.110 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-6.2-6.el8.noarch 2024-02-02T15:17:19.110 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-02-02T15:17:19.110 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jinja2-2.10.1-3.el8.noarch 2024-02-02T15:17:19.110 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-02-02T15:17:19.110 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-02-02T15:17:19.110 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-02-02T15:17:19.110 INFO:teuthology.orchestra.run.smithi071.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-02-02T15:17:19.110 INFO:teuthology.orchestra.run.smithi071.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-02-02T15:17:19.110 INFO:teuthology.orchestra.run.smithi071.stdout: python3-mako-1.0.6-14.el8.noarch 2024-02-02T15:17:19.111 INFO:teuthology.orchestra.run.smithi071.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-02-02T15:17:19.111 INFO:teuthology.orchestra.run.smithi071.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-02-02T15:17:19.111 INFO:teuthology.orchestra.run.smithi071.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-02-02T15:17:19.111 INFO:teuthology.orchestra.run.smithi071.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-02-02T15:17:19.111 INFO:teuthology.orchestra.run.smithi071.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-02-02T15:17:19.111 INFO:teuthology.orchestra.run.smithi071.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-02-02T15:17:19.111 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-02-02T15:17:19.111 INFO:teuthology.orchestra.run.smithi071.stdout: python3-portend-2.6-1.el8.noarch 2024-02-02T15:17:19.111 INFO:teuthology.orchestra.run.smithi071.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-02-02T15:17:19.111 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-02-02T15:17:19.111 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-02-02T15:17:19.111 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-02-02T15:17:19.111 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pycparser-2.14-14.el8.noarch 2024-02-02T15:17:19.111 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pytz-2017.2-11.el8.noarch 2024-02-02T15:17:19.111 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rados-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:19.112 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rbd-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:19.112 INFO:teuthology.orchestra.run.smithi071.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-02-02T15:17:19.112 INFO:teuthology.orchestra.run.smithi071.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-02-02T15:17:19.112 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rgw-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:19.112 INFO:teuthology.orchestra.run.smithi071.stdout: python3-routes-2.4.1-12.el8.noarch 2024-02-02T15:17:19.112 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rsa-4.9-2.el8.noarch 2024-02-02T15:17:19.112 INFO:teuthology.orchestra.run.smithi071.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2024-02-02T15:17:19.112 INFO:teuthology.orchestra.run.smithi071.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-02-02T15:17:19.112 INFO:teuthology.orchestra.run.smithi071.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-02-02T15:17:19.112 INFO:teuthology.orchestra.run.smithi071.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-02-02T15:17:19.112 INFO:teuthology.orchestra.run.smithi071.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-02-02T15:17:19.112 INFO:teuthology.orchestra.run.smithi071.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-02-02T15:17:19.112 INFO:teuthology.orchestra.run.smithi071.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-02-02T15:17:19.112 INFO:teuthology.orchestra.run.smithi071.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-02-02T15:17:19.112 INFO:teuthology.orchestra.run.smithi071.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-02-02T15:17:19.113 INFO:teuthology.orchestra.run.smithi071.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-02-02T15:17:19.113 INFO:teuthology.orchestra.run.smithi071.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-02-02T15:17:19.113 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-fuse-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:19.113 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-mirror-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:19.113 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-nbd-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:19.113 INFO:teuthology.orchestra.run.smithi071.stdout: socat-1.7.4.1-1.el8.x86_64 2024-02-02T15:17:19.113 INFO:teuthology.orchestra.run.smithi071.stdout: thrift-0.13.0-2.el8.x86_64 2024-02-02T15:17:19.113 INFO:teuthology.orchestra.run.smithi071.stdout: uuid-1.6.2-43.el8.x86_64 2024-02-02T15:17:19.113 INFO:teuthology.orchestra.run.smithi071.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2024-02-02T15:17:19.113 INFO:teuthology.orchestra.run.smithi071.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-02-02T15:17:19.114 INFO:teuthology.orchestra.run.smithi071.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2024-02-02T15:17:19.114 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-02-02T15:17:19.114 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-02-02T15:17:19.598 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-test-2:18.2.1-325.g0fb32788.el8.x86_64 125/161 2024-02-02T15:17:19.892 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : openssl-devel-1:1.1.1k-12.el8_9.x86_64 126/161 2024-02-02T15:17:20.186 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 127/161 2024-02-02T15:17:20.622 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : ceph-fuse-2:18.2.1-325.g0fb32788.el8.x86_64 128/161 2024-02-02T15:17:20.914 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : postgresql-contrib-10.23-3.module+el8.9.0+20831+ 129/161 2024-02-02T15:17:20.978 DEBUG:teuthology.parallel:result is None 2024-02-02T15:17:21.138 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libtool-2.4.6-25.el8.x86_64 130/161 2024-02-02T15:17:21.169 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 130/161 2024-02-02T15:17:21.384 INFO:teuthology.orchestra.run.smithi001.stdout: Upgrading : util-linux-user-2.32.1-43.el8.x86_64 131/161 2024-02-02T15:17:21.619 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 132/161 2024-02-02T15:17:21.831 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libcephfs-devel-2:18.2.1-325.g0fb32788.el8.x86_6 133/161 2024-02-02T15:17:24.905 INFO:teuthology.orchestra.run.smithi142.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 161/161 2024-02-02T15:17:24.905 INFO:teuthology.orchestra.run.smithi142.stdout:Installed products updated. 2024-02-02T15:17:25.435 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: postgresql-server-10.23-3.module+el8.9.0+20831+7 134/161 2024-02-02T15:17:25.508 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-02-02T15:17:25.508 INFO:teuthology.orchestra.run.smithi142.stdout:Upgraded: 2024-02-02T15:17:25.508 INFO:teuthology.orchestra.run.smithi142.stdout: elfutils-libelf-0.189-3.el8.x86_64 2024-02-02T15:17:25.508 INFO:teuthology.orchestra.run.smithi142.stdout: elfutils-libs-0.189-3.el8.x86_64 2024-02-02T15:17:25.508 INFO:teuthology.orchestra.run.smithi142.stdout: iproute-6.2.0-5.el8_9.x86_64 2024-02-02T15:17:25.509 INFO:teuthology.orchestra.run.smithi142.stdout: libblkid-2.32.1-43.el8.x86_64 2024-02-02T15:17:25.509 INFO:teuthology.orchestra.run.smithi142.stdout: libfdisk-2.32.1-43.el8.x86_64 2024-02-02T15:17:25.509 INFO:teuthology.orchestra.run.smithi142.stdout: libmount-2.32.1-43.el8.x86_64 2024-02-02T15:17:25.509 INFO:teuthology.orchestra.run.smithi142.stdout: librados2-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:25.509 INFO:teuthology.orchestra.run.smithi142.stdout: librbd1-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:25.509 INFO:teuthology.orchestra.run.smithi142.stdout: libsmartcols-2.32.1-43.el8.x86_64 2024-02-02T15:17:25.509 INFO:teuthology.orchestra.run.smithi142.stdout: libuuid-2.32.1-43.el8.x86_64 2024-02-02T15:17:25.509 INFO:teuthology.orchestra.run.smithi142.stdout: openssl-1:1.1.1k-12.el8_9.x86_64 2024-02-02T15:17:25.509 INFO:teuthology.orchestra.run.smithi142.stdout: openssl-libs-1:1.1.1k-12.el8_9.x86_64 2024-02-02T15:17:25.509 INFO:teuthology.orchestra.run.smithi142.stdout: util-linux-2.32.1-43.el8.x86_64 2024-02-02T15:17:25.509 INFO:teuthology.orchestra.run.smithi142.stdout: util-linux-user-2.32.1-43.el8.x86_64 2024-02-02T15:17:25.509 INFO:teuthology.orchestra.run.smithi142.stdout: xfsdump-3.1.8-7.el8_9.x86_64 2024-02-02T15:17:25.509 INFO:teuthology.orchestra.run.smithi142.stdout: xfsprogs-5.0.0-12.el8.x86_64 2024-02-02T15:17:25.509 INFO:teuthology.orchestra.run.smithi142.stdout:Installed: 2024-02-02T15:17:25.510 INFO:teuthology.orchestra.run.smithi142.stdout: NetworkManager-initscripts-updown-1:1.40.16-13.el8_9.noarch 2024-02-02T15:17:25.510 INFO:teuthology.orchestra.run.smithi142.stdout: automake-1.16.1-8.el8.noarch 2024-02-02T15:17:25.510 INFO:teuthology.orchestra.run.smithi142.stdout: bison-3.0.4-10.el8.x86_64 2024-02-02T15:17:25.510 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:25.510 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-base-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:25.510 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-common-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:25.510 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-fuse-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:25.510 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-grafana-dashboards-2:18.2.1-325.g0fb32788.el8.noarch 2024-02-02T15:17:25.510 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-immutable-object-cache-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:25.510 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mds-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:25.510 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:25.510 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-cephadm-2:18.2.1-325.g0fb32788.el8.noarch 2024-02-02T15:17:25.510 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-dashboard-2:18.2.1-325.g0fb32788.el8.noarch 2024-02-02T15:17:25.510 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-diskprediction-local-2:18.2.1-325.g0fb32788.el8.noarch 2024-02-02T15:17:25.511 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-modules-core-2:18.2.1-325.g0fb32788.el8.noarch 2024-02-02T15:17:25.511 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-rook-2:18.2.1-325.g0fb32788.el8.noarch 2024-02-02T15:17:25.511 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mon-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:25.511 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-osd-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:25.511 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-prometheus-alerts-2:18.2.1-325.g0fb32788.el8.noarch 2024-02-02T15:17:25.511 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-radosgw-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:25.511 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-selinux-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:25.511 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-test-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:25.511 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-volume-2:18.2.1-325.g0fb32788.el8.noarch 2024-02-02T15:17:25.511 INFO:teuthology.orchestra.run.smithi142.stdout: cephadm-2:18.2.1-325.g0fb32788.el8.noarch 2024-02-02T15:17:25.511 INFO:teuthology.orchestra.run.smithi142.stdout: cephfs-mirror-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:25.511 INFO:teuthology.orchestra.run.smithi142.stdout: cephfs-top-2:18.2.1-325.g0fb32788.el8.noarch 2024-02-02T15:17:25.511 INFO:teuthology.orchestra.run.smithi142.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2024-02-02T15:17:25.511 INFO:teuthology.orchestra.run.smithi142.stdout: flex-2.6.1-9.el8.x86_64 2024-02-02T15:17:25.512 INFO:teuthology.orchestra.run.smithi142.stdout: fuse-2.9.7-17.el8.x86_64 2024-02-02T15:17:25.512 INFO:teuthology.orchestra.run.smithi142.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-02-02T15:17:25.512 INFO:teuthology.orchestra.run.smithi142.stdout: jq-1.6-7.el8.x86_64 2024-02-02T15:17:25.512 INFO:teuthology.orchestra.run.smithi142.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2024-02-02T15:17:25.512 INFO:teuthology.orchestra.run.smithi142.stdout: krb5-devel-1.18.2-26.el8_9.x86_64 2024-02-02T15:17:25.512 INFO:teuthology.orchestra.run.smithi142.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-02-02T15:17:25.512 INFO:teuthology.orchestra.run.smithi142.stdout: libcephfs-devel-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:25.512 INFO:teuthology.orchestra.run.smithi142.stdout: libcephfs2-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:25.512 INFO:teuthology.orchestra.run.smithi142.stdout: libcephsqlite-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:25.512 INFO:teuthology.orchestra.run.smithi142.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2024-02-02T15:17:25.512 INFO:teuthology.orchestra.run.smithi142.stdout: libgfortran-8.5.0-20.el8.x86_64 2024-02-02T15:17:25.512 INFO:teuthology.orchestra.run.smithi142.stdout: liboath-2.6.2-3.el8.x86_64 2024-02-02T15:17:25.513 INFO:teuthology.orchestra.run.smithi142.stdout: libpq-13.11-1.el8.x86_64 2024-02-02T15:17:25.513 INFO:teuthology.orchestra.run.smithi142.stdout: libquadmath-8.5.0-20.el8.x86_64 2024-02-02T15:17:25.513 INFO:teuthology.orchestra.run.smithi142.stdout: librabbitmq-0.9.0-4.el8.x86_64 2024-02-02T15:17:25.513 INFO:teuthology.orchestra.run.smithi142.stdout: librados-devel-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:25.513 INFO:teuthology.orchestra.run.smithi142.stdout: libradosstriper1-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:25.513 INFO:teuthology.orchestra.run.smithi142.stdout: librdkafka-0.11.4-3.el8.x86_64 2024-02-02T15:17:25.513 INFO:teuthology.orchestra.run.smithi142.stdout: librgw2-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:25.513 INFO:teuthology.orchestra.run.smithi142.stdout: libselinux-devel-2.9-5.el8.x86_64 2024-02-02T15:17:25.513 INFO:teuthology.orchestra.run.smithi142.stdout: libsepol-devel-2.9-3.el8.x86_64 2024-02-02T15:17:25.513 INFO:teuthology.orchestra.run.smithi142.stdout: libtool-2.4.6-25.el8.x86_64 2024-02-02T15:17:25.513 INFO:teuthology.orchestra.run.smithi142.stdout: libunwind-1.3.1-3.el8.x86_64 2024-02-02T15:17:25.513 INFO:teuthology.orchestra.run.smithi142.stdout: libuuid-devel-2.32.1-43.el8.x86_64 2024-02-02T15:17:25.513 INFO:teuthology.orchestra.run.smithi142.stdout: libverto-devel-0.3.0-5.el8.x86_64 2024-02-02T15:17:25.514 INFO:teuthology.orchestra.run.smithi142.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2024-02-02T15:17:25.514 INFO:teuthology.orchestra.run.smithi142.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-02-02T15:17:25.514 INFO:teuthology.orchestra.run.smithi142.stdout: oniguruma-6.8.2-2.el8.x86_64 2024-02-02T15:17:25.514 INFO:teuthology.orchestra.run.smithi142.stdout: openblas-0.3.15-6.el8.x86_64 2024-02-02T15:17:25.514 INFO:teuthology.orchestra.run.smithi142.stdout: openblas-threads-0.3.15-6.el8.x86_64 2024-02-02T15:17:25.514 INFO:teuthology.orchestra.run.smithi142.stdout: openssl-devel-1:1.1.1k-12.el8_9.x86_64 2024-02-02T15:17:25.514 INFO:teuthology.orchestra.run.smithi142.stdout: pcre2-devel-10.32-2.el8.x86_64 2024-02-02T15:17:25.514 INFO:teuthology.orchestra.run.smithi142.stdout: pcre2-utf16-10.32-2.el8.x86_64 2024-02-02T15:17:25.514 INFO:teuthology.orchestra.run.smithi142.stdout: pcre2-utf32-10.32-2.el8.x86_64 2024-02-02T15:17:25.514 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-10.23-3.module+el8.9.0+20831+78489779.x86_64 2024-02-02T15:17:25.514 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-contrib-10.23-3.module+el8.9.0+20831+78489779.x86_64 2024-02-02T15:17:25.514 INFO:teuthology.orchestra.run.smithi142.stdout: postgresql-server-10.23-3.module+el8.9.0+20831+78489779.x86_64 2024-02-02T15:17:25.514 INFO:teuthology.orchestra.run.smithi142.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-02-02T15:17:25.514 INFO:teuthology.orchestra.run.smithi142.stdout: python3-babel-2.5.1-7.el8.noarch 2024-02-02T15:17:25.515 INFO:teuthology.orchestra.run.smithi142.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-02-02T15:17:25.515 INFO:teuthology.orchestra.run.smithi142.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-02-02T15:17:25.515 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-02-02T15:17:25.515 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ceph-argparse-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:25.515 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ceph-common-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:25.515 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cephfs-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:25.515 INFO:teuthology.orchestra.run.smithi142.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-02-02T15:17:25.515 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-02-02T15:17:25.515 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-02-02T15:17:25.515 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-02-02T15:17:25.515 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cryptography-3.2.1-6.el8.x86_64 2024-02-02T15:17:25.515 INFO:teuthology.orchestra.run.smithi142.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-02-02T15:17:25.515 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-6.2-6.el8.noarch 2024-02-02T15:17:25.515 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-02-02T15:17:25.516 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jinja2-2.10.1-3.el8.noarch 2024-02-02T15:17:25.516 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-02-02T15:17:25.516 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-02-02T15:17:25.516 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-02-02T15:17:25.516 INFO:teuthology.orchestra.run.smithi142.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-02-02T15:17:25.516 INFO:teuthology.orchestra.run.smithi142.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-02-02T15:17:25.516 INFO:teuthology.orchestra.run.smithi142.stdout: python3-mako-1.0.6-14.el8.noarch 2024-02-02T15:17:25.516 INFO:teuthology.orchestra.run.smithi142.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-02-02T15:17:25.516 INFO:teuthology.orchestra.run.smithi142.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-02-02T15:17:25.516 INFO:teuthology.orchestra.run.smithi142.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-02-02T15:17:25.516 INFO:teuthology.orchestra.run.smithi142.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-02-02T15:17:25.516 INFO:teuthology.orchestra.run.smithi142.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-02-02T15:17:25.516 INFO:teuthology.orchestra.run.smithi142.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-02-02T15:17:25.516 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-02-02T15:17:25.516 INFO:teuthology.orchestra.run.smithi142.stdout: python3-portend-2.6-1.el8.noarch 2024-02-02T15:17:25.517 INFO:teuthology.orchestra.run.smithi142.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-02-02T15:17:25.517 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-02-02T15:17:25.517 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-02-02T15:17:25.517 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-02-02T15:17:25.517 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pycparser-2.14-14.el8.noarch 2024-02-02T15:17:25.517 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pytz-2017.2-11.el8.noarch 2024-02-02T15:17:25.517 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rados-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:25.517 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rbd-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:25.517 INFO:teuthology.orchestra.run.smithi142.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-02-02T15:17:25.517 INFO:teuthology.orchestra.run.smithi142.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-02-02T15:17:25.517 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rgw-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:25.517 INFO:teuthology.orchestra.run.smithi142.stdout: python3-routes-2.4.1-12.el8.noarch 2024-02-02T15:17:25.517 INFO:teuthology.orchestra.run.smithi142.stdout: python3-rsa-4.9-2.el8.noarch 2024-02-02T15:17:25.517 INFO:teuthology.orchestra.run.smithi142.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2024-02-02T15:17:25.518 INFO:teuthology.orchestra.run.smithi142.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-02-02T15:17:25.518 INFO:teuthology.orchestra.run.smithi142.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-02-02T15:17:25.518 INFO:teuthology.orchestra.run.smithi142.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-02-02T15:17:25.518 INFO:teuthology.orchestra.run.smithi142.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-02-02T15:17:25.518 INFO:teuthology.orchestra.run.smithi142.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-02-02T15:17:25.518 INFO:teuthology.orchestra.run.smithi142.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-02-02T15:17:25.518 INFO:teuthology.orchestra.run.smithi142.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-02-02T15:17:25.518 INFO:teuthology.orchestra.run.smithi142.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-02-02T15:17:25.518 INFO:teuthology.orchestra.run.smithi142.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-02-02T15:17:25.518 INFO:teuthology.orchestra.run.smithi142.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-02-02T15:17:25.518 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-fuse-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:25.518 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-mirror-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:25.518 INFO:teuthology.orchestra.run.smithi142.stdout: rbd-nbd-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:25.518 INFO:teuthology.orchestra.run.smithi142.stdout: socat-1.7.4.1-1.el8.x86_64 2024-02-02T15:17:25.518 INFO:teuthology.orchestra.run.smithi142.stdout: thrift-0.13.0-2.el8.x86_64 2024-02-02T15:17:25.519 INFO:teuthology.orchestra.run.smithi142.stdout: uuid-1.6.2-43.el8.x86_64 2024-02-02T15:17:25.519 INFO:teuthology.orchestra.run.smithi142.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2024-02-02T15:17:25.519 INFO:teuthology.orchestra.run.smithi142.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-02-02T15:17:25.519 INFO:teuthology.orchestra.run.smithi142.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2024-02-02T15:17:25.519 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-02-02T15:17:25.519 INFO:teuthology.orchestra.run.smithi142.stdout:Complete! 2024-02-02T15:17:25.765 DEBUG:teuthology.parallel:result is None 2024-02-02T15:17:25.838 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : postgresql-server-10.23-3.module+el8.9.0+20831+7 134/161 2024-02-02T15:17:25.909 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: postgresql-server-10.23-3.module+el8.9.0+20831+7 134/161 2024-02-02T15:17:26.318 INFO:teuthology.orchestra.run.smithi001.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 135/161 2024-02-02T15:17:26.680 INFO:teuthology.orchestra.run.smithi001.stdout: Upgrading : iproute-6.2.0-5.el8_9.x86_64 136/161 2024-02-02T15:17:26.901 INFO:teuthology.orchestra.run.smithi001.stdout: Upgrading : xfsdump-3.1.8-7.el8_9.x86_64 137/161 2024-02-02T15:17:27.103 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : cephfs-top-2:18.2.1-325.g0fb32788.el8.noarch 138/161 2024-02-02T15:17:27.311 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : rbd-fuse-2:18.2.1-325.g0fb32788.el8.x86_64 139/161 2024-02-02T15:17:27.696 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : rbd-nbd-2:18.2.1-325.g0fb32788.el8.x86_64 140/161 2024-02-02T15:17:28.034 INFO:teuthology.orchestra.run.smithi001.stdout: Upgrading : openssl-1:1.1.1k-12.el8_9.x86_64 141/161 2024-02-02T15:17:28.220 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : flex-2.6.1-9.el8.x86_64 142/161 2024-02-02T15:17:28.347 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 142/161 2024-02-02T15:17:28.551 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : bison-3.0.4-10.el8.x86_64 143/161 2024-02-02T15:17:28.582 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 143/161 2024-02-02T15:17:28.726 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : NetworkManager-initscripts-updown-1:1.40.16-13.e 144/161 2024-02-02T15:17:28.751 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-13.e 144/161 2024-02-02T15:17:28.893 INFO:teuthology.orchestra.run.smithi001.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 145/161 2024-02-02T15:17:29.093 INFO:teuthology.orchestra.run.smithi001.stdout: Cleanup : elfutils-libs-0.186-1.el8.x86_64 146/161 2024-02-02T15:17:29.281 INFO:teuthology.orchestra.run.smithi001.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 147/161 2024-02-02T15:17:29.481 INFO:teuthology.orchestra.run.smithi001.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 148/161 2024-02-02T15:17:29.481 INFO:teuthology.orchestra.run.smithi001.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 149/161 2024-02-02T15:17:29.751 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 149/161 2024-02-02T15:17:29.751 INFO:teuthology.orchestra.run.smithi001.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 150/161 2024-02-02T15:17:30.028 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 150/161 2024-02-02T15:17:30.028 INFO:teuthology.orchestra.run.smithi001.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 151/161 2024-02-02T15:17:30.308 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 151/161 2024-02-02T15:17:30.522 INFO:teuthology.orchestra.run.smithi001.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 152/161 2024-02-02T15:17:30.697 INFO:teuthology.orchestra.run.smithi001.stdout: Cleanup : util-linux-user-2.32.1-35.el8.x86_64 153/161 2024-02-02T15:17:30.928 INFO:teuthology.orchestra.run.smithi001.stdout: Cleanup : util-linux-2.32.1-35.el8.x86_64 154/161 2024-02-02T15:17:30.928 INFO:teuthology.orchestra.run.smithi001.stdout: Cleanup : libfdisk-2.32.1-35.el8.x86_64 155/161 2024-02-02T15:17:31.199 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: libfdisk-2.32.1-35.el8.x86_64 155/161 2024-02-02T15:17:31.199 INFO:teuthology.orchestra.run.smithi001.stdout: Cleanup : libmount-2.32.1-35.el8.x86_64 156/161 2024-02-02T15:17:31.490 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: libmount-2.32.1-35.el8.x86_64 156/161 2024-02-02T15:17:31.490 INFO:teuthology.orchestra.run.smithi001.stdout: Cleanup : libblkid-2.32.1-35.el8.x86_64 157/161 2024-02-02T15:17:31.715 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: libblkid-2.32.1-35.el8.x86_64 157/161 2024-02-02T15:17:31.715 INFO:teuthology.orchestra.run.smithi001.stdout: Cleanup : libuuid-2.32.1-35.el8.x86_64 158/161 2024-02-02T15:17:31.973 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: libuuid-2.32.1-35.el8.x86_64 158/161 2024-02-02T15:17:31.974 INFO:teuthology.orchestra.run.smithi001.stdout: Cleanup : libsmartcols-2.32.1-35.el8.x86_64 159/161 2024-02-02T15:17:32.240 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: libsmartcols-2.32.1-35.el8.x86_64 159/161 2024-02-02T15:17:32.382 INFO:teuthology.orchestra.run.smithi001.stdout: Cleanup : elfutils-libelf-0.186-1.el8.x86_64 160/161 2024-02-02T15:17:32.382 INFO:teuthology.orchestra.run.smithi001.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 161/161 2024-02-02T15:17:32.988 INFO:teuthology.orchestra.run.smithi001.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 161/161 2024-02-02T15:17:32.988 INFO:teuthology.orchestra.run.smithi001.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-02-02T15:17:32.988 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:17:33.661 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-2:18.2.1-325.g0fb32788.el8.x86_64 1/161 2024-02-02T15:17:33.661 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-base-2:18.2.1-325.g0fb32788.el8.x86_64 2/161 2024-02-02T15:17:33.661 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-common-2:18.2.1-325.g0fb32788.el8.x86_64 3/161 2024-02-02T15:17:33.661 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-fuse-2:18.2.1-325.g0fb32788.el8.x86_64 4/161 2024-02-02T15:17:33.661 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-immutable-object-cache-2:18.2.1-325.g0fb327 5/161 2024-02-02T15:17:33.661 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-mds-2:18.2.1-325.g0fb32788.el8.x86_64 6/161 2024-02-02T15:17:33.662 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-mgr-2:18.2.1-325.g0fb32788.el8.x86_64 7/161 2024-02-02T15:17:33.662 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-mon-2:18.2.1-325.g0fb32788.el8.x86_64 8/161 2024-02-02T15:17:33.662 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-osd-2:18.2.1-325.g0fb32788.el8.x86_64 9/161 2024-02-02T15:17:33.662 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-radosgw-2:18.2.1-325.g0fb32788.el8.x86_64 10/161 2024-02-02T15:17:33.662 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-selinux-2:18.2.1-325.g0fb32788.el8.x86_64 11/161 2024-02-02T15:17:33.662 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-test-2:18.2.1-325.g0fb32788.el8.x86_64 12/161 2024-02-02T15:17:33.662 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : cephfs-mirror-2:18.2.1-325.g0fb32788.el8.x86_64 13/161 2024-02-02T15:17:33.663 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libcephfs-devel-2:18.2.1-325.g0fb32788.el8.x86_6 14/161 2024-02-02T15:17:33.663 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libcephfs2-2:18.2.1-325.g0fb32788.el8.x86_64 15/161 2024-02-02T15:17:33.663 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libcephsqlite-2:18.2.1-325.g0fb32788.el8.x86_64 16/161 2024-02-02T15:17:33.664 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : librados-devel-2:18.2.1-325.g0fb32788.el8.x86_64 17/161 2024-02-02T15:17:33.664 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libradosstriper1-2:18.2.1-325.g0fb32788.el8.x86_ 18/161 2024-02-02T15:17:33.664 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : librgw2-2:18.2.1-325.g0fb32788.el8.x86_64 19/161 2024-02-02T15:17:33.664 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-ceph-argparse-2:18.2.1-325.g0fb32788.el8 20/161 2024-02-02T15:17:33.664 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-ceph-common-2:18.2.1-325.g0fb32788.el8.x 21/161 2024-02-02T15:17:33.664 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-cephfs-2:18.2.1-325.g0fb32788.el8.x86_64 22/161 2024-02-02T15:17:33.664 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-rados-2:18.2.1-325.g0fb32788.el8.x86_64 23/161 2024-02-02T15:17:33.664 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-rbd-2:18.2.1-325.g0fb32788.el8.x86_64 24/161 2024-02-02T15:17:33.664 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-rgw-2:18.2.1-325.g0fb32788.el8.x86_64 25/161 2024-02-02T15:17:33.664 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : rbd-fuse-2:18.2.1-325.g0fb32788.el8.x86_64 26/161 2024-02-02T15:17:33.676 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : rbd-mirror-2:18.2.1-325.g0fb32788.el8.x86_64 27/161 2024-02-02T15:17:33.676 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : rbd-nbd-2:18.2.1-325.g0fb32788.el8.x86_64 28/161 2024-02-02T15:17:33.677 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-grafana-dashboards-2:18.2.1-325.g0fb32788.e 29/161 2024-02-02T15:17:33.677 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-mgr-cephadm-2:18.2.1-325.g0fb32788.el8.noar 30/161 2024-02-02T15:17:33.677 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-mgr-dashboard-2:18.2.1-325.g0fb32788.el8.no 31/161 2024-02-02T15:17:33.677 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.1-325.g0fb3 32/161 2024-02-02T15:17:33.677 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-mgr-modules-core-2:18.2.1-325.g0fb32788.el8 33/161 2024-02-02T15:17:33.677 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-mgr-rook-2:18.2.1-325.g0fb32788.el8.noarch 34/161 2024-02-02T15:17:33.677 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-prometheus-alerts-2:18.2.1-325.g0fb32788.el 35/161 2024-02-02T15:17:33.677 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : ceph-volume-2:18.2.1-325.g0fb32788.el8.noarch 36/161 2024-02-02T15:17:33.677 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : cephadm-2:18.2.1-325.g0fb32788.el8.noarch 37/161 2024-02-02T15:17:33.677 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : cephfs-top-2:18.2.1-325.g0fb32788.el8.noarch 38/161 2024-02-02T15:17:33.677 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/161 2024-02-02T15:17:33.677 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/161 2024-02-02T15:17:33.677 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/161 2024-02-02T15:17:33.677 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/161 2024-02-02T15:17:33.678 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/161 2024-02-02T15:17:33.678 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/161 2024-02-02T15:17:33.678 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/161 2024-02-02T15:17:33.678 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/161 2024-02-02T15:17:33.678 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/161 2024-02-02T15:17:33.678 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/161 2024-02-02T15:17:33.678 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/161 2024-02-02T15:17:33.678 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/161 2024-02-02T15:17:33.678 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/161 2024-02-02T15:17:33.678 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/161 2024-02-02T15:17:33.678 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/161 2024-02-02T15:17:33.678 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/161 2024-02-02T15:17:33.678 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/161 2024-02-02T15:17:33.678 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/161 2024-02-02T15:17:33.678 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/161 2024-02-02T15:17:33.679 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/161 2024-02-02T15:17:33.679 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/161 2024-02-02T15:17:33.679 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/161 2024-02-02T15:17:33.679 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/161 2024-02-02T15:17:33.679 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/161 2024-02-02T15:17:33.679 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/161 2024-02-02T15:17:33.679 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/161 2024-02-02T15:17:33.679 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/161 2024-02-02T15:17:33.679 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/161 2024-02-02T15:17:33.679 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/161 2024-02-02T15:17:33.679 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/161 2024-02-02T15:17:33.679 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/161 2024-02-02T15:17:33.679 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/161 2024-02-02T15:17:33.679 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/161 2024-02-02T15:17:33.679 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 72/161 2024-02-02T15:17:33.680 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 73/161 2024-02-02T15:17:33.680 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 74/161 2024-02-02T15:17:33.680 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 75/161 2024-02-02T15:17:33.680 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 76/161 2024-02-02T15:17:33.680 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 77/161 2024-02-02T15:17:33.680 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 78/161 2024-02-02T15:17:33.680 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 79/161 2024-02-02T15:17:33.680 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/161 2024-02-02T15:17:33.680 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 81/161 2024-02-02T15:17:33.680 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 82/161 2024-02-02T15:17:33.680 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 83/161 2024-02-02T15:17:33.680 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 84/161 2024-02-02T15:17:33.680 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 85/161 2024-02-02T15:17:33.681 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 86/161 2024-02-02T15:17:33.681 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 87/161 2024-02-02T15:17:33.681 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 88/161 2024-02-02T15:17:33.681 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 89/161 2024-02-02T15:17:33.681 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libuuid-devel-2.32.1-43.el8.x86_64 90/161 2024-02-02T15:17:33.681 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 91/161 2024-02-02T15:17:33.681 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-cryptography-3.2.1-6.el8.x86_64 92/161 2024-02-02T15:17:33.681 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 93/161 2024-02-02T15:17:33.681 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-13.e 94/161 2024-02-02T15:17:33.681 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 95/161 2024-02-02T15:17:33.681 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : krb5-devel-1.18.2-26.el8_9.x86_64 96/161 2024-02-02T15:17:33.681 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : librabbitmq-0.9.0-4.el8.x86_64 97/161 2024-02-02T15:17:33.681 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : openssl-devel-1:1.1.1k-12.el8_9.x86_64 98/161 2024-02-02T15:17:33.681 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 99/161 2024-02-02T15:17:33.681 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 100/161 2024-02-02T15:17:33.682 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 101/161 2024-02-02T15:17:33.682 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 102/161 2024-02-02T15:17:33.682 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 103/161 2024-02-02T15:17:33.682 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 104/161 2024-02-02T15:17:33.682 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : bison-3.0.4-10.el8.x86_64 105/161 2024-02-02T15:17:33.682 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : flex-2.6.1-9.el8.x86_64 106/161 2024-02-02T15:17:33.682 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 107/161 2024-02-02T15:17:33.682 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 108/161 2024-02-02T15:17:33.682 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 109/161 2024-02-02T15:17:33.682 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 110/161 2024-02-02T15:17:33.682 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 111/161 2024-02-02T15:17:33.682 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 112/161 2024-02-02T15:17:33.682 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 113/161 2024-02-02T15:17:33.682 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 114/161 2024-02-02T15:17:33.682 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 115/161 2024-02-02T15:17:33.683 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 116/161 2024-02-02T15:17:33.683 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 117/161 2024-02-02T15:17:33.683 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 118/161 2024-02-02T15:17:33.683 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 119/161 2024-02-02T15:17:33.683 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 120/161 2024-02-02T15:17:33.683 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 121/161 2024-02-02T15:17:33.683 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : automake-1.16.1-8.el8.noarch 122/161 2024-02-02T15:17:33.683 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libpq-13.11-1.el8.x86_64 123/161 2024-02-02T15:17:33.683 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : jq-1.6-7.el8.x86_64 124/161 2024-02-02T15:17:33.683 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 125/161 2024-02-02T15:17:33.683 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : postgresql-10.23-3.module+el8.9.0+20831+78489779 126/161 2024-02-02T15:17:33.683 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : postgresql-contrib-10.23-3.module+el8.9.0+20831+ 127/161 2024-02-02T15:17:33.683 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : postgresql-server-10.23-3.module+el8.9.0+20831+7 128/161 2024-02-02T15:17:33.683 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c3 129/161 2024-02-02T15:17:33.683 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : librados2-2:18.2.1-325.g0fb32788.el8.x86_64 130/161 2024-02-02T15:17:33.683 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 131/161 2024-02-02T15:17:33.684 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : librbd1-2:18.2.1-325.g0fb32788.el8.x86_64 132/161 2024-02-02T15:17:33.684 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 133/161 2024-02-02T15:17:33.684 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 134/161 2024-02-02T15:17:33.684 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : elfutils-libelf-0.186-1.el8.x86_64 135/161 2024-02-02T15:17:33.684 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 136/161 2024-02-02T15:17:33.684 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : elfutils-libs-0.186-1.el8.x86_64 137/161 2024-02-02T15:17:33.684 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libmount-2.32.1-43.el8.x86_64 138/161 2024-02-02T15:17:33.684 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libmount-2.32.1-35.el8.x86_64 139/161 2024-02-02T15:17:33.684 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libsmartcols-2.32.1-43.el8.x86_64 140/161 2024-02-02T15:17:33.684 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libsmartcols-2.32.1-35.el8.x86_64 141/161 2024-02-02T15:17:33.684 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libuuid-2.32.1-43.el8.x86_64 142/161 2024-02-02T15:17:33.684 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libuuid-2.32.1-35.el8.x86_64 143/161 2024-02-02T15:17:33.684 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : util-linux-2.32.1-43.el8.x86_64 144/161 2024-02-02T15:17:33.684 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : util-linux-2.32.1-35.el8.x86_64 145/161 2024-02-02T15:17:33.684 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : util-linux-user-2.32.1-43.el8.x86_64 146/161 2024-02-02T15:17:33.685 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : util-linux-user-2.32.1-35.el8.x86_64 147/161 2024-02-02T15:17:33.685 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : iproute-6.2.0-5.el8_9.x86_64 148/161 2024-02-02T15:17:33.685 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 149/161 2024-02-02T15:17:33.685 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libblkid-2.32.1-43.el8.x86_64 150/161 2024-02-02T15:17:33.685 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libblkid-2.32.1-35.el8.x86_64 151/161 2024-02-02T15:17:33.685 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libfdisk-2.32.1-43.el8.x86_64 152/161 2024-02-02T15:17:33.685 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : libfdisk-2.32.1-35.el8.x86_64 153/161 2024-02-02T15:17:33.685 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : openssl-1:1.1.1k-12.el8_9.x86_64 154/161 2024-02-02T15:17:33.685 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 155/161 2024-02-02T15:17:33.685 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : openssl-libs-1:1.1.1k-12.el8_9.x86_64 156/161 2024-02-02T15:17:33.685 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 157/161 2024-02-02T15:17:33.685 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : xfsdump-3.1.8-7.el8_9.x86_64 158/161 2024-02-02T15:17:33.685 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 159/161 2024-02-02T15:17:33.685 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 160/161 2024-02-02T15:17:41.386 INFO:teuthology.orchestra.run.smithi001.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 161/161 2024-02-02T15:17:41.386 INFO:teuthology.orchestra.run.smithi001.stdout:Installed products updated. 2024-02-02T15:17:42.037 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:17:42.037 INFO:teuthology.orchestra.run.smithi001.stdout:Upgraded: 2024-02-02T15:17:42.037 INFO:teuthology.orchestra.run.smithi001.stdout: elfutils-libelf-0.189-3.el8.x86_64 2024-02-02T15:17:42.037 INFO:teuthology.orchestra.run.smithi001.stdout: elfutils-libs-0.189-3.el8.x86_64 2024-02-02T15:17:42.037 INFO:teuthology.orchestra.run.smithi001.stdout: iproute-6.2.0-5.el8_9.x86_64 2024-02-02T15:17:42.037 INFO:teuthology.orchestra.run.smithi001.stdout: libblkid-2.32.1-43.el8.x86_64 2024-02-02T15:17:42.037 INFO:teuthology.orchestra.run.smithi001.stdout: libfdisk-2.32.1-43.el8.x86_64 2024-02-02T15:17:42.037 INFO:teuthology.orchestra.run.smithi001.stdout: libmount-2.32.1-43.el8.x86_64 2024-02-02T15:17:42.037 INFO:teuthology.orchestra.run.smithi001.stdout: librados2-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:42.037 INFO:teuthology.orchestra.run.smithi001.stdout: librbd1-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:42.038 INFO:teuthology.orchestra.run.smithi001.stdout: libsmartcols-2.32.1-43.el8.x86_64 2024-02-02T15:17:42.038 INFO:teuthology.orchestra.run.smithi001.stdout: libuuid-2.32.1-43.el8.x86_64 2024-02-02T15:17:42.038 INFO:teuthology.orchestra.run.smithi001.stdout: openssl-1:1.1.1k-12.el8_9.x86_64 2024-02-02T15:17:42.038 INFO:teuthology.orchestra.run.smithi001.stdout: openssl-libs-1:1.1.1k-12.el8_9.x86_64 2024-02-02T15:17:42.038 INFO:teuthology.orchestra.run.smithi001.stdout: util-linux-2.32.1-43.el8.x86_64 2024-02-02T15:17:42.038 INFO:teuthology.orchestra.run.smithi001.stdout: util-linux-user-2.32.1-43.el8.x86_64 2024-02-02T15:17:42.038 INFO:teuthology.orchestra.run.smithi001.stdout: xfsdump-3.1.8-7.el8_9.x86_64 2024-02-02T15:17:42.038 INFO:teuthology.orchestra.run.smithi001.stdout: xfsprogs-5.0.0-12.el8.x86_64 2024-02-02T15:17:42.038 INFO:teuthology.orchestra.run.smithi001.stdout:Installed: 2024-02-02T15:17:42.038 INFO:teuthology.orchestra.run.smithi001.stdout: NetworkManager-initscripts-updown-1:1.40.16-13.el8_9.noarch 2024-02-02T15:17:42.038 INFO:teuthology.orchestra.run.smithi001.stdout: automake-1.16.1-8.el8.noarch 2024-02-02T15:17:42.038 INFO:teuthology.orchestra.run.smithi001.stdout: bison-3.0.4-10.el8.x86_64 2024-02-02T15:17:42.038 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:42.038 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-base-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:42.039 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-common-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:42.039 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-fuse-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:42.039 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-grafana-dashboards-2:18.2.1-325.g0fb32788.el8.noarch 2024-02-02T15:17:42.039 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-immutable-object-cache-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:42.039 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mds-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:42.039 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:42.039 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-cephadm-2:18.2.1-325.g0fb32788.el8.noarch 2024-02-02T15:17:42.039 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-dashboard-2:18.2.1-325.g0fb32788.el8.noarch 2024-02-02T15:17:42.039 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-diskprediction-local-2:18.2.1-325.g0fb32788.el8.noarch 2024-02-02T15:17:42.039 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-modules-core-2:18.2.1-325.g0fb32788.el8.noarch 2024-02-02T15:17:42.039 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mgr-rook-2:18.2.1-325.g0fb32788.el8.noarch 2024-02-02T15:17:42.039 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-mon-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:42.039 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-osd-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:42.039 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-prometheus-alerts-2:18.2.1-325.g0fb32788.el8.noarch 2024-02-02T15:17:42.040 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-radosgw-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:42.040 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-selinux-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:42.040 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-test-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:42.040 INFO:teuthology.orchestra.run.smithi001.stdout: ceph-volume-2:18.2.1-325.g0fb32788.el8.noarch 2024-02-02T15:17:42.040 INFO:teuthology.orchestra.run.smithi001.stdout: cephadm-2:18.2.1-325.g0fb32788.el8.noarch 2024-02-02T15:17:42.040 INFO:teuthology.orchestra.run.smithi001.stdout: cephfs-mirror-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:42.040 INFO:teuthology.orchestra.run.smithi001.stdout: cephfs-top-2:18.2.1-325.g0fb32788.el8.noarch 2024-02-02T15:17:42.040 INFO:teuthology.orchestra.run.smithi001.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2024-02-02T15:17:42.040 INFO:teuthology.orchestra.run.smithi001.stdout: flex-2.6.1-9.el8.x86_64 2024-02-02T15:17:42.040 INFO:teuthology.orchestra.run.smithi001.stdout: fuse-2.9.7-17.el8.x86_64 2024-02-02T15:17:42.040 INFO:teuthology.orchestra.run.smithi001.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-02-02T15:17:42.040 INFO:teuthology.orchestra.run.smithi001.stdout: jq-1.6-7.el8.x86_64 2024-02-02T15:17:42.040 INFO:teuthology.orchestra.run.smithi001.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2024-02-02T15:17:42.041 INFO:teuthology.orchestra.run.smithi001.stdout: krb5-devel-1.18.2-26.el8_9.x86_64 2024-02-02T15:17:42.041 INFO:teuthology.orchestra.run.smithi001.stdout: libaio-devel-0.3.112-1.el8.x86_64 2024-02-02T15:17:42.041 INFO:teuthology.orchestra.run.smithi001.stdout: libcephfs-devel-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:42.041 INFO:teuthology.orchestra.run.smithi001.stdout: libcephfs2-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:42.041 INFO:teuthology.orchestra.run.smithi001.stdout: libcephsqlite-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:42.041 INFO:teuthology.orchestra.run.smithi001.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2024-02-02T15:17:42.041 INFO:teuthology.orchestra.run.smithi001.stdout: libgfortran-8.5.0-20.el8.x86_64 2024-02-02T15:17:42.041 INFO:teuthology.orchestra.run.smithi001.stdout: liboath-2.6.2-3.el8.x86_64 2024-02-02T15:17:42.041 INFO:teuthology.orchestra.run.smithi001.stdout: libpq-13.11-1.el8.x86_64 2024-02-02T15:17:42.041 INFO:teuthology.orchestra.run.smithi001.stdout: libquadmath-8.5.0-20.el8.x86_64 2024-02-02T15:17:42.041 INFO:teuthology.orchestra.run.smithi001.stdout: librabbitmq-0.9.0-4.el8.x86_64 2024-02-02T15:17:42.041 INFO:teuthology.orchestra.run.smithi001.stdout: librados-devel-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:42.041 INFO:teuthology.orchestra.run.smithi001.stdout: libradosstriper1-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:42.041 INFO:teuthology.orchestra.run.smithi001.stdout: librdkafka-0.11.4-3.el8.x86_64 2024-02-02T15:17:42.042 INFO:teuthology.orchestra.run.smithi001.stdout: librgw2-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:42.042 INFO:teuthology.orchestra.run.smithi001.stdout: libselinux-devel-2.9-5.el8.x86_64 2024-02-02T15:17:42.042 INFO:teuthology.orchestra.run.smithi001.stdout: libsepol-devel-2.9-3.el8.x86_64 2024-02-02T15:17:42.042 INFO:teuthology.orchestra.run.smithi001.stdout: libtool-2.4.6-25.el8.x86_64 2024-02-02T15:17:42.042 INFO:teuthology.orchestra.run.smithi001.stdout: libunwind-1.3.1-3.el8.x86_64 2024-02-02T15:17:42.042 INFO:teuthology.orchestra.run.smithi001.stdout: libuuid-devel-2.32.1-43.el8.x86_64 2024-02-02T15:17:42.042 INFO:teuthology.orchestra.run.smithi001.stdout: libverto-devel-0.3.0-5.el8.x86_64 2024-02-02T15:17:42.042 INFO:teuthology.orchestra.run.smithi001.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2024-02-02T15:17:42.042 INFO:teuthology.orchestra.run.smithi001.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-02-02T15:17:42.042 INFO:teuthology.orchestra.run.smithi001.stdout: oniguruma-6.8.2-2.el8.x86_64 2024-02-02T15:17:42.042 INFO:teuthology.orchestra.run.smithi001.stdout: openblas-0.3.15-6.el8.x86_64 2024-02-02T15:17:42.042 INFO:teuthology.orchestra.run.smithi001.stdout: openblas-threads-0.3.15-6.el8.x86_64 2024-02-02T15:17:42.042 INFO:teuthology.orchestra.run.smithi001.stdout: openssl-devel-1:1.1.1k-12.el8_9.x86_64 2024-02-02T15:17:42.042 INFO:teuthology.orchestra.run.smithi001.stdout: pcre2-devel-10.32-2.el8.x86_64 2024-02-02T15:17:42.042 INFO:teuthology.orchestra.run.smithi001.stdout: pcre2-utf16-10.32-2.el8.x86_64 2024-02-02T15:17:42.043 INFO:teuthology.orchestra.run.smithi001.stdout: pcre2-utf32-10.32-2.el8.x86_64 2024-02-02T15:17:42.043 INFO:teuthology.orchestra.run.smithi001.stdout: postgresql-10.23-3.module+el8.9.0+20831+78489779.x86_64 2024-02-02T15:17:42.043 INFO:teuthology.orchestra.run.smithi001.stdout: postgresql-contrib-10.23-3.module+el8.9.0+20831+78489779.x86_64 2024-02-02T15:17:42.043 INFO:teuthology.orchestra.run.smithi001.stdout: postgresql-server-10.23-3.module+el8.9.0+20831+78489779.x86_64 2024-02-02T15:17:42.043 INFO:teuthology.orchestra.run.smithi001.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-02-02T15:17:42.043 INFO:teuthology.orchestra.run.smithi001.stdout: python3-babel-2.5.1-7.el8.noarch 2024-02-02T15:17:42.043 INFO:teuthology.orchestra.run.smithi001.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-02-02T15:17:42.043 INFO:teuthology.orchestra.run.smithi001.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-02-02T15:17:42.043 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-02-02T15:17:42.043 INFO:teuthology.orchestra.run.smithi001.stdout: python3-ceph-argparse-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:42.043 INFO:teuthology.orchestra.run.smithi001.stdout: python3-ceph-common-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:42.043 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cephfs-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:42.043 INFO:teuthology.orchestra.run.smithi001.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-02-02T15:17:42.043 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-02-02T15:17:42.043 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-02-02T15:17:42.043 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-02-02T15:17:42.044 INFO:teuthology.orchestra.run.smithi001.stdout: python3-cryptography-3.2.1-6.el8.x86_64 2024-02-02T15:17:42.044 INFO:teuthology.orchestra.run.smithi001.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-02-02T15:17:42.044 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco-6.2-6.el8.noarch 2024-02-02T15:17:42.044 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-02-02T15:17:42.044 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jinja2-2.10.1-3.el8.noarch 2024-02-02T15:17:42.044 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-02-02T15:17:42.044 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-02-02T15:17:42.044 INFO:teuthology.orchestra.run.smithi001.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-02-02T15:17:42.044 INFO:teuthology.orchestra.run.smithi001.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-02-02T15:17:42.044 INFO:teuthology.orchestra.run.smithi001.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-02-02T15:17:42.044 INFO:teuthology.orchestra.run.smithi001.stdout: python3-mako-1.0.6-14.el8.noarch 2024-02-02T15:17:42.044 INFO:teuthology.orchestra.run.smithi001.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-02-02T15:17:42.045 INFO:teuthology.orchestra.run.smithi001.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-02-02T15:17:42.045 INFO:teuthology.orchestra.run.smithi001.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-02-02T15:17:42.045 INFO:teuthology.orchestra.run.smithi001.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-02-02T15:17:42.045 INFO:teuthology.orchestra.run.smithi001.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-02-02T15:17:42.045 INFO:teuthology.orchestra.run.smithi001.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-02-02T15:17:42.045 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-02-02T15:17:42.045 INFO:teuthology.orchestra.run.smithi001.stdout: python3-portend-2.6-1.el8.noarch 2024-02-02T15:17:42.045 INFO:teuthology.orchestra.run.smithi001.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-02-02T15:17:42.045 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-02-02T15:17:42.045 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-02-02T15:17:42.045 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-02-02T15:17:42.045 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pycparser-2.14-14.el8.noarch 2024-02-02T15:17:42.045 INFO:teuthology.orchestra.run.smithi001.stdout: python3-pytz-2017.2-11.el8.noarch 2024-02-02T15:17:42.045 INFO:teuthology.orchestra.run.smithi001.stdout: python3-rados-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:42.045 INFO:teuthology.orchestra.run.smithi001.stdout: python3-rbd-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:42.045 INFO:teuthology.orchestra.run.smithi001.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-02-02T15:17:42.045 INFO:teuthology.orchestra.run.smithi001.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-02-02T15:17:42.046 INFO:teuthology.orchestra.run.smithi001.stdout: python3-rgw-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:42.046 INFO:teuthology.orchestra.run.smithi001.stdout: python3-routes-2.4.1-12.el8.noarch 2024-02-02T15:17:42.046 INFO:teuthology.orchestra.run.smithi001.stdout: python3-rsa-4.9-2.el8.noarch 2024-02-02T15:17:42.046 INFO:teuthology.orchestra.run.smithi001.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2024-02-02T15:17:42.046 INFO:teuthology.orchestra.run.smithi001.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-02-02T15:17:42.046 INFO:teuthology.orchestra.run.smithi001.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-02-02T15:17:42.046 INFO:teuthology.orchestra.run.smithi001.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-02-02T15:17:42.046 INFO:teuthology.orchestra.run.smithi001.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-02-02T15:17:42.046 INFO:teuthology.orchestra.run.smithi001.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-02-02T15:17:42.046 INFO:teuthology.orchestra.run.smithi001.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-02-02T15:17:42.046 INFO:teuthology.orchestra.run.smithi001.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-02-02T15:17:42.046 INFO:teuthology.orchestra.run.smithi001.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-02-02T15:17:42.046 INFO:teuthology.orchestra.run.smithi001.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-02-02T15:17:42.046 INFO:teuthology.orchestra.run.smithi001.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-02-02T15:17:42.046 INFO:teuthology.orchestra.run.smithi001.stdout: rbd-fuse-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:42.046 INFO:teuthology.orchestra.run.smithi001.stdout: rbd-mirror-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:42.046 INFO:teuthology.orchestra.run.smithi001.stdout: rbd-nbd-2:18.2.1-325.g0fb32788.el8.x86_64 2024-02-02T15:17:42.047 INFO:teuthology.orchestra.run.smithi001.stdout: socat-1.7.4.1-1.el8.x86_64 2024-02-02T15:17:42.047 INFO:teuthology.orchestra.run.smithi001.stdout: thrift-0.13.0-2.el8.x86_64 2024-02-02T15:17:42.047 INFO:teuthology.orchestra.run.smithi001.stdout: uuid-1.6.2-43.el8.x86_64 2024-02-02T15:17:42.047 INFO:teuthology.orchestra.run.smithi001.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2024-02-02T15:17:42.047 INFO:teuthology.orchestra.run.smithi001.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-02-02T15:17:42.047 INFO:teuthology.orchestra.run.smithi001.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2024-02-02T15:17:42.047 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:17:42.047 INFO:teuthology.orchestra.run.smithi001.stdout:Complete! 2024-02-02T15:17:42.413 DEBUG:teuthology.parallel:result is None 2024-02-02T15:17:42.413 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=0fb327883c124fd83155c94db0804f823b805a12 2024-02-02T15:17:42.690 DEBUG:teuthology.orchestra.run.smithi001:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-02-02T15:17:42.742 INFO:teuthology.orchestra.run.smithi001.stdout:18.2.1-325.g0fb32788.el8 2024-02-02T15:17:42.743 INFO:teuthology.packaging:The installed version of ceph is 18.2.1-325.g0fb32788.el8 2024-02-02T15:17:42.743 INFO:teuthology.task.install:The correct ceph version 18.2.1-325.g0fb32788 is installed. 2024-02-02T15:17:42.745 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=0fb327883c124fd83155c94db0804f823b805a12 2024-02-02T15:17:42.884 DEBUG:teuthology.orchestra.run.smithi071:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-02-02T15:17:42.912 INFO:teuthology.orchestra.run.smithi071.stdout:18.2.1-325.g0fb32788.el8 2024-02-02T15:17:42.913 INFO:teuthology.packaging:The installed version of ceph is 18.2.1-325.g0fb32788.el8 2024-02-02T15:17:42.913 INFO:teuthology.task.install:The correct ceph version 18.2.1-325.g0fb32788 is installed. 2024-02-02T15:17:42.914 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=0fb327883c124fd83155c94db0804f823b805a12 2024-02-02T15:17:43.141 DEBUG:teuthology.orchestra.run.smithi142:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-02-02T15:17:43.170 INFO:teuthology.orchestra.run.smithi142.stdout:18.2.1-325.g0fb32788.el8 2024-02-02T15:17:43.170 INFO:teuthology.packaging:The installed version of ceph is 18.2.1-325.g0fb32788.el8 2024-02-02T15:17:43.170 INFO:teuthology.task.install:The correct ceph version 18.2.1-325.g0fb32788 is installed. 2024-02-02T15:17:43.172 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-02-02T15:17:43.172 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-02-02T15:17:43.172 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-02T15:17:43.260 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-02-02T15:17:43.260 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-02T15:17:43.309 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-02-02T15:17:43.309 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-02T15:17:43.357 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-02-02T15:17:43.357 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-02-02T15:17:43.357 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/usr/bin/daemon-helper 2024-02-02T15:17:43.421 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-02T15:17:43.526 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-02-02T15:17:43.526 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/usr/bin/daemon-helper 2024-02-02T15:17:43.560 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-02T15:17:43.630 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-02-02T15:17:43.631 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/usr/bin/daemon-helper 2024-02-02T15:17:43.663 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-02T15:17:43.735 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-02-02T15:17:43.735 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-02-02T15:17:43.735 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-02T15:17:43.800 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-02T15:17:43.904 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-02-02T15:17:43.905 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-02T15:17:43.937 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-02T15:17:44.006 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-02-02T15:17:44.006 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-02T15:17:44.039 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-02T15:17:44.111 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-02-02T15:17:44.111 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-02-02T15:17:44.111 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/usr/bin/stdin-killer 2024-02-02T15:17:44.176 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-02T15:17:44.281 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-02-02T15:17:44.281 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/usr/bin/stdin-killer 2024-02-02T15:17:44.313 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-02T15:17:44.383 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-02-02T15:17:44.383 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/usr/bin/stdin-killer 2024-02-02T15:17:44.416 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-02T15:17:44.486 INFO:teuthology.run_tasks:Running task cephadm... 2024-02-02T15:17:44.642 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '0fb327883c124fd83155c94db0804f823b805a12', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-02-02T15:17:44.642 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 2024-02-02T15:17:44.643 INFO:tasks.cephadm:Cluster fsid is 371a409c-c1de-11ee-95b6-87774f69a715 2024-02-02T15:17:44.643 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-02-02T15:17:44.643 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.1', 'mon.b': '172.21.15.71', 'mon.c': '172.21.15.142'} 2024-02-02T15:17:44.643 INFO:tasks.cephadm:First mon is mon.a on smithi001 2024-02-02T15:17:44.643 INFO:tasks.cephadm:First mgr is x 2024-02-02T15:17:44.643 INFO:tasks.cephadm:Normalizing hostnames... 2024-02-02T15:17:44.644 DEBUG:teuthology.orchestra.run.smithi001:> sudo hostname $(hostname -s) 2024-02-02T15:17:44.710 DEBUG:teuthology.orchestra.run.smithi071:> sudo hostname $(hostname -s) 2024-02-02T15:17:44.743 DEBUG:teuthology.orchestra.run.smithi142:> sudo hostname $(hostname -s) 2024-02-02T15:17:44.776 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-02-02T15:17:44.777 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=0fb327883c124fd83155c94db0804f823b805a12 2024-02-02T15:17:44.916 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '0fb327883c124fd83155c94db0804f823b805a12', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=arm64,AVAILABLE_ARCH=arm64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/76444/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.2.1-325-g0fb32788', 'node_name': '172.21.4.63+confusa01', 'job_name': 'ceph-dev-new-build/ARCH=arm64,AVAILABLE_ARCH=arm64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.2.1-325.g0fb32788'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-vshankar-testing1-reef-2024-02-02-1152/0fb327883c124fd83155c94db0804f823b805a12/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2024-02-02 12:10:42.652616', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-vshankar-testing1-reef-2024-02-02-1152', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-vshankar-testing1-reef-2024-02-02-1152/0fb327883c124fd83155c94db0804f823b805a12/centos/8/flavors/default/', 'archs': ['source', 'x86_64', 'arm64'], 'distro': 'centos'}] 2024-02-02T15:17:45.054 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref wip-vshankar-testing1-reef-2024-02-02-1152, sha1 0fb327883c124fd83155c94db0804f823b805a12 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F8%2Fx86_64&flavor=default&sha1=0fb327883c124fd83155c94db0804f823b805a12 2024-02-02T15:17:45.056 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/wip-vshankar-testing1-reef-2024-02-02-1152/0fb327883c124fd83155c94db0804f823b805a12/centos/8/x86_64/flavors/default/cephadm 2024-02-02T15:17:45.056 DEBUG:teuthology.orchestra.run.smithi001:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-vshankar-testing1-reef-2024-02-02-1152/0fb327883c124fd83155c94db0804f823b805a12/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-02T15:17:46.226 INFO:teuthology.orchestra.run.smithi001.stdout:-rw-rw-r--. 1 ubuntu ubuntu 690687 Feb 2 15:17 /home/ubuntu/cephtest/cephadm 2024-02-02T15:17:46.227 DEBUG:teuthology.orchestra.run.smithi071:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-vshankar-testing1-reef-2024-02-02-1152/0fb327883c124fd83155c94db0804f823b805a12/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-02T15:17:46.981 INFO:teuthology.orchestra.run.smithi071.stdout:-rw-rw-r--. 1 ubuntu ubuntu 690687 Feb 2 15:17 /home/ubuntu/cephtest/cephadm 2024-02-02T15:17:46.981 DEBUG:teuthology.orchestra.run.smithi142:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-vshankar-testing1-reef-2024-02-02-1152/0fb327883c124fd83155c94db0804f823b805a12/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-02T15:17:47.720 INFO:teuthology.orchestra.run.smithi142.stdout:-rw-rw-r--. 1 ubuntu ubuntu 690687 Feb 2 15:17 /home/ubuntu/cephtest/cephadm 2024-02-02T15:17:47.720 DEBUG:teuthology.orchestra.run.smithi001:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-02T15:17:47.769 DEBUG:teuthology.orchestra.run.smithi071:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-02T15:17:47.791 DEBUG:teuthology.orchestra.run.smithi142:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-02T15:17:47.829 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 on all hosts... 2024-02-02T15:17:47.829 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 pull 2024-02-02T15:17:47.833 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 pull 2024-02-02T15:17:47.835 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 pull 2024-02-02T15:17:49.005 INFO:teuthology.orchestra.run.smithi071.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12... 2024-02-02T15:17:49.221 INFO:teuthology.orchestra.run.smithi142.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12... 2024-02-02T15:17:50.395 INFO:teuthology.orchestra.run.smithi001.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12... 2024-02-02T15:18:14.383 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-02-02T15:18:14.384 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_version": "ceph version 18.2.1-325-g0fb32788 (0fb327883c124fd83155c94db0804f823b805a12) reef (stable)", 2024-02-02T15:18:14.384 INFO:teuthology.orchestra.run.smithi071.stdout: "image_id": "0b0faa63c314f8e41fd42dfab8075e0b1c615cb55b7b11e23bb9e546a5596317", 2024-02-02T15:18:14.384 INFO:teuthology.orchestra.run.smithi071.stdout: "repo_digests": [ 2024-02-02T15:18:14.384 INFO:teuthology.orchestra.run.smithi071.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8ebf664d52e12c0cb95aadac3a6c74d14d559a05f707b1f192f422fd9fd06632" 2024-02-02T15:18:14.384 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-02-02T15:18:14.384 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-02-02T15:18:18.392 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:18:18.392 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_version": "ceph version 18.2.1-325-g0fb32788 (0fb327883c124fd83155c94db0804f823b805a12) reef (stable)", 2024-02-02T15:18:18.392 INFO:teuthology.orchestra.run.smithi001.stdout: "image_id": "0b0faa63c314f8e41fd42dfab8075e0b1c615cb55b7b11e23bb9e546a5596317", 2024-02-02T15:18:18.392 INFO:teuthology.orchestra.run.smithi001.stdout: "repo_digests": [ 2024-02-02T15:18:18.392 INFO:teuthology.orchestra.run.smithi001.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8ebf664d52e12c0cb95aadac3a6c74d14d559a05f707b1f192f422fd9fd06632" 2024-02-02T15:18:18.392 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-02-02T15:18:18.392 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:18:18.865 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-02-02T15:18:18.865 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_version": "ceph version 18.2.1-325-g0fb32788 (0fb327883c124fd83155c94db0804f823b805a12) reef (stable)", 2024-02-02T15:18:18.865 INFO:teuthology.orchestra.run.smithi142.stdout: "image_id": "0b0faa63c314f8e41fd42dfab8075e0b1c615cb55b7b11e23bb9e546a5596317", 2024-02-02T15:18:18.865 INFO:teuthology.orchestra.run.smithi142.stdout: "repo_digests": [ 2024-02-02T15:18:18.865 INFO:teuthology.orchestra.run.smithi142.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:8ebf664d52e12c0cb95aadac3a6c74d14d559a05f707b1f192f422fd9fd06632" 2024-02-02T15:18:18.865 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-02-02T15:18:18.865 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-02-02T15:18:18.884 DEBUG:teuthology.orchestra.run.smithi001:> sudo mkdir -p /etc/ceph 2024-02-02T15:18:18.955 DEBUG:teuthology.orchestra.run.smithi071:> sudo mkdir -p /etc/ceph 2024-02-02T15:18:18.992 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /etc/ceph 2024-02-02T15:18:19.022 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod 777 /etc/ceph 2024-02-02T15:18:19.086 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod 777 /etc/ceph 2024-02-02T15:18:19.117 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod 777 /etc/ceph 2024-02-02T15:18:19.143 INFO:tasks.cephadm:Writing seed config... 2024-02-02T15:18:19.162 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-02-02T15:18:19.162 INFO:tasks.cephadm: override: [client] debug client = 20 2024-02-02T15:18:19.162 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-02-02T15:18:19.162 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-02-02T15:18:19.162 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-02-02T15:18:19.163 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-02-02T15:18:19.163 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-02-02T15:18:19.163 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-02-02T15:18:19.163 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-02-02T15:18:19.163 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-02-02T15:18:19.163 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-02-02T15:18:19.163 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-02-02T15:18:19.163 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-02-02T15:18:19.163 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-02-02T15:18:19.163 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-02-02T15:18:19.163 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-02-02T15:18:19.163 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-02-02T15:18:19.163 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-02-02T15:18:19.163 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-02-02T15:18:19.164 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-02-02T15:18:19.164 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-02-02T15:18:19.164 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-02-02T15:18:19.164 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-02-02T15:18:19.164 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-02-02T15:18:19.164 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-02-02T15:18:19.164 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-02-02T15:18:19.164 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-02-02T15:18:19.164 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-02-02T15:18:19.164 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-02-02T15:18:19.164 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-02-02T15:18:19.164 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-02-02T15:18:19.164 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-02-02T15:18:19.165 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-02-02T15:18:19.165 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-02-02T15:18:19.165 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-02-02T15:18:19.165 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-02-02T15:18:19.165 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-02-02T15:18:19.165 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-02-02T15:18:19.165 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-02-02T15:18:19.165 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-02-02T15:18:19.165 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-02-02T15:18:19.165 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-02-02T15:18:19.165 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-02-02T15:18:19.165 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-02-02T15:18:19.165 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-02-02T15:18:19.166 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-02-02T15:18:19.166 DEBUG:teuthology.orchestra.run.smithi001:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-02-02T15:18:19.207 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 371a409c-c1de-11ee-95b6-87774f69a715 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-02-02T15:18:19.207 DEBUG:teuthology.orchestra.run.smithi001:mon.a> sudo journalctl -f -n 0 -u ceph-371a409c-c1de-11ee-95b6-87774f69a715@mon.a.service 2024-02-02T15:18:19.251 DEBUG:teuthology.orchestra.run.smithi001:mgr.x> sudo journalctl -f -n 0 -u ceph-371a409c-c1de-11ee-95b6-87774f69a715@mgr.x.service 2024-02-02T15:18:19.295 INFO:tasks.cephadm:Bootstrapping... 2024-02-02T15:18:19.296 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 -v bootstrap --fsid 371a409c-c1de-11ee-95b6-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.1 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-02-02T15:18:19.315 INFO:journalctl@ceph.mon.a.smithi001.stdout:-- Logs begin at Fri 2024-02-02 15:05:03 UTC. -- 2024-02-02T15:18:19.361 INFO:journalctl@ceph.mgr.x.smithi001.stdout:-- Logs begin at Fri 2024-02-02 15:05:03 UTC. -- 2024-02-02T15:18:19.517 INFO:teuthology.orchestra.run.smithi001.stdout:-------------------------------------------------------------------------------- 2024-02-02T15:18:19.518 INFO:teuthology.orchestra.run.smithi001.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12', '-v', 'bootstrap', '--fsid', '371a409c-c1de-11ee-95b6-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.1', '--skip-admin-label'] 2024-02-02T15:18:19.565 INFO:teuthology.orchestra.run.smithi001.stdout:/bin/podman: stdout 4.6.1 2024-02-02T15:18:19.566 INFO:teuthology.orchestra.run.smithi001.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-02-02T15:18:19.566 INFO:teuthology.orchestra.run.smithi001.stdout:Verifying podman|docker is present... 2024-02-02T15:18:19.610 INFO:teuthology.orchestra.run.smithi001.stdout:/bin/podman: stdout 4.6.1 2024-02-02T15:18:19.611 INFO:teuthology.orchestra.run.smithi001.stdout:Verifying lvm2 is present... 2024-02-02T15:18:19.611 INFO:teuthology.orchestra.run.smithi001.stdout:Verifying time synchronization is in place... 2024-02-02T15:18:19.624 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-02-02T15:18:19.624 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-02-02T15:18:19.637 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-02-02T15:18:19.637 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout inactive 2024-02-02T15:18:19.650 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout enabled 2024-02-02T15:18:19.662 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout active 2024-02-02T15:18:19.663 INFO:teuthology.orchestra.run.smithi001.stdout:Unit chronyd.service is enabled and running 2024-02-02T15:18:19.663 INFO:teuthology.orchestra.run.smithi001.stdout:Repeating the final host check... 2024-02-02T15:18:19.707 INFO:teuthology.orchestra.run.smithi001.stdout:/bin/podman: stdout 4.6.1 2024-02-02T15:18:19.707 INFO:teuthology.orchestra.run.smithi001.stdout:podman (/bin/podman) version 4.6.1 is present 2024-02-02T15:18:19.707 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl is present 2024-02-02T15:18:19.707 INFO:teuthology.orchestra.run.smithi001.stdout:lvcreate is present 2024-02-02T15:18:19.719 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-02-02T15:18:19.719 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-02-02T15:18:19.731 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-02-02T15:18:19.731 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout inactive 2024-02-02T15:18:19.744 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout enabled 2024-02-02T15:18:19.757 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout active 2024-02-02T15:18:19.757 INFO:teuthology.orchestra.run.smithi001.stdout:Unit chronyd.service is enabled and running 2024-02-02T15:18:19.757 INFO:teuthology.orchestra.run.smithi001.stdout:Host looks OK 2024-02-02T15:18:19.757 INFO:teuthology.orchestra.run.smithi001.stdout:Cluster fsid: 371a409c-c1de-11ee-95b6-87774f69a715 2024-02-02T15:18:19.757 INFO:teuthology.orchestra.run.smithi001.stdout:Acquiring lock 140005066963432 on /run/cephadm/371a409c-c1de-11ee-95b6-87774f69a715.lock 2024-02-02T15:18:19.758 INFO:teuthology.orchestra.run.smithi001.stdout:Lock 140005066963432 acquired on /run/cephadm/371a409c-c1de-11ee-95b6-87774f69a715.lock 2024-02-02T15:18:19.758 INFO:teuthology.orchestra.run.smithi001.stdout:Verifying IP 172.21.15.1 port 3300 ... 2024-02-02T15:18:19.758 INFO:teuthology.orchestra.run.smithi001.stdout:Verifying IP 172.21.15.1 port 6789 ... 2024-02-02T15:18:19.759 INFO:teuthology.orchestra.run.smithi001.stdout:Base mon IP(s) is [172.21.15.1:3300, 172.21.15.1:6789], mon addrv is [v2:172.21.15.1:3300,v1:172.21.15.1:6789] 2024-02-02T15:18:19.766 INFO:teuthology.orchestra.run.smithi001.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp metric 100 2024-02-02T15:18:19.766 INFO:teuthology.orchestra.run.smithi001.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.1 metric 100 2024-02-02T15:18:19.773 INFO:teuthology.orchestra.run.smithi001.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-02-02T15:18:19.773 INFO:teuthology.orchestra.run.smithi001.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-02-02T15:18:19.773 INFO:teuthology.orchestra.run.smithi001.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1799sec hoplimit 64 pref medium 2024-02-02T15:18:19.780 INFO:teuthology.orchestra.run.smithi001.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-02-02T15:18:19.780 INFO:teuthology.orchestra.run.smithi001.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-02-02T15:18:19.780 INFO:teuthology.orchestra.run.smithi001.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-02-02T15:18:19.780 INFO:teuthology.orchestra.run.smithi001.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-02-02T15:18:19.780 INFO:teuthology.orchestra.run.smithi001.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:15e8/64 scope link 2024-02-02T15:18:19.780 INFO:teuthology.orchestra.run.smithi001.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-02-02T15:18:19.781 INFO:teuthology.orchestra.run.smithi001.stdout:Mon IP `172.21.15.1` is in CIDR network `172.21.0.0/20` 2024-02-02T15:18:19.781 INFO:teuthology.orchestra.run.smithi001.stdout:Mon IP `172.21.15.1` is in CIDR network `172.21.0.0/20` 2024-02-02T15:18:19.781 INFO:teuthology.orchestra.run.smithi001.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-02-02T15:18:19.782 INFO:teuthology.orchestra.run.smithi001.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-02-02T15:18:19.783 INFO:teuthology.orchestra.run.smithi001.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12... 2024-02-02T15:18:20.181 INFO:teuthology.orchestra.run.smithi001.stdout:/bin/podman: stdout 0b0faa63c314f8e41fd42dfab8075e0b1c615cb55b7b11e23bb9e546a5596317 2024-02-02T15:18:20.181 INFO:teuthology.orchestra.run.smithi001.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12... 2024-02-02T15:18:20.181 INFO:teuthology.orchestra.run.smithi001.stdout:/bin/podman: stderr Getting image source signatures 2024-02-02T15:18:20.181 INFO:teuthology.orchestra.run.smithi001.stdout:/bin/podman: stderr Copying blob sha256:58df4e03ba066accf3a7e07426bab5f1c867e27a139801c642bdafb42b9a2d9a 2024-02-02T15:18:20.181 INFO:teuthology.orchestra.run.smithi001.stdout:/bin/podman: stderr Copying blob sha256:98d9bab01b1845b50db92efcbe742540e191ab5ccb83fa27c87e2f0ea05049a3 2024-02-02T15:18:20.182 INFO:teuthology.orchestra.run.smithi001.stdout:/bin/podman: stderr Copying config sha256:0b0faa63c314f8e41fd42dfab8075e0b1c615cb55b7b11e23bb9e546a5596317 2024-02-02T15:18:20.182 INFO:teuthology.orchestra.run.smithi001.stdout:/bin/podman: stderr Writing manifest to image destination 2024-02-02T15:18:21.027 INFO:teuthology.orchestra.run.smithi001.stdout:ceph: stdout ceph version 18.2.1-325-g0fb32788 (0fb327883c124fd83155c94db0804f823b805a12) reef (stable) 2024-02-02T15:18:21.028 INFO:teuthology.orchestra.run.smithi001.stdout:Ceph version: ceph version 18.2.1-325-g0fb32788 (0fb327883c124fd83155c94db0804f823b805a12) reef (stable) 2024-02-02T15:18:21.028 INFO:teuthology.orchestra.run.smithi001.stdout:Extracting ceph user uid/gid from container image... 2024-02-02T15:18:21.663 INFO:teuthology.orchestra.run.smithi001.stdout:stat: stdout 167 167 2024-02-02T15:18:21.663 INFO:teuthology.orchestra.run.smithi001.stdout:Creating initial keys... 2024-02-02T15:18:22.326 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph-authtool: stdout AQC9B71lZLFPOhAA5CuH6r74ZupFC3BuoUF2eg== 2024-02-02T15:18:22.985 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph-authtool: stdout AQC+B71lPklvJRAAvgEarND4GJ/XFGOG2ZGjFQ== 2024-02-02T15:18:23.633 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph-authtool: stdout AQC/B71lQe2MEBAADJiKgBPtX9eHhtoygRmCRw== 2024-02-02T15:18:23.634 INFO:teuthology.orchestra.run.smithi001.stdout:Creating initial monmap... 2024-02-02T15:18:24.324 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-02-02T15:18:24.325 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2024-02-02T15:18:24.325 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 371a409c-c1de-11ee-95b6-87774f69a715 2024-02-02T15:18:24.325 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-02-02T15:18:24.325 INFO:teuthology.orchestra.run.smithi001.stdout:monmaptool for a [v2:172.21.15.1:3300,v1:172.21.15.1:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-02-02T15:18:24.325 INFO:teuthology.orchestra.run.smithi001.stdout:setting min_mon_release = pacific 2024-02-02T15:18:24.325 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/monmaptool: set fsid to 371a409c-c1de-11ee-95b6-87774f69a715 2024-02-02T15:18:24.325 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-02-02T15:18:24.325 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:18:24.325 INFO:teuthology.orchestra.run.smithi001.stdout:Creating mon... 2024-02-02T15:18:24.975 INFO:teuthology.orchestra.run.smithi001.stdout:create mon.a on 2024-02-02T15:18:25.226 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2024-02-02T15:18:25.420 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-02-02T15:18:25.664 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-371a409c-c1de-11ee-95b6-87774f69a715.target → /etc/systemd/system/ceph-371a409c-c1de-11ee-95b6-87774f69a715.target. 2024-02-02T15:18:25.664 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-371a409c-c1de-11ee-95b6-87774f69a715.target → /etc/systemd/system/ceph-371a409c-c1de-11ee-95b6-87774f69a715.target. 2024-02-02T15:18:26.009 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-371a409c-c1de-11ee-95b6-87774f69a715@mon.a 2024-02-02T15:18:26.009 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stderr Failed to reset failed state of unit ceph-371a409c-c1de-11ee-95b6-87774f69a715@mon.a.service: Unit ceph-371a409c-c1de-11ee-95b6-87774f69a715@mon.a.service not loaded. 2024-02-02T15:18:26.199 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-371a409c-c1de-11ee-95b6-87774f69a715.target.wants/ceph-371a409c-c1de-11ee-95b6-87774f69a715@mon.a.service → /etc/systemd/system/ceph-371a409c-c1de-11ee-95b6-87774f69a715@.service. 2024-02-02T15:18:26.290 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:26 smithi001 systemd[1]: Starting Ceph mon.a for 371a409c-c1de-11ee-95b6-87774f69a715... 2024-02-02T15:18:26.738 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-02T15:18:26.738 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout disabled 2024-02-02T15:18:26.751 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:26 smithi001 bash[34946]: 01e23085ca754f1fa8ed3aa18226ffb7be9cc555a6a6387e49270e30936ff09e 2024-02-02T15:18:26.751 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:26 smithi001 systemd[1]: Started Ceph mon.a for 371a409c-c1de-11ee-95b6-87774f69a715. 2024-02-02T15:18:26.751 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:26 smithi001 ceph-mon[34984]: mkfs 371a409c-c1de-11ee-95b6-87774f69a715 2024-02-02T15:18:26.755 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-02T15:18:26.755 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout inactive 2024-02-02T15:18:26.755 INFO:teuthology.orchestra.run.smithi001.stdout:firewalld.service is not enabled 2024-02-02T15:18:26.756 INFO:teuthology.orchestra.run.smithi001.stdout:Not possible to enable service . firewalld.service is not available 2024-02-02T15:18:26.756 INFO:teuthology.orchestra.run.smithi001.stdout:Waiting for mon to start... 2024-02-02T15:18:26.756 INFO:teuthology.orchestra.run.smithi001.stdout:Waiting for mon... 2024-02-02T15:18:26.975 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:26 smithi001 ceph-mon[34984]: mon.a is new leader, mons a in quorum (ranks 0) 2024-02-02T15:18:27.631 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout cluster: 2024-02-02T15:18:27.637 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout id: 371a409c-c1de-11ee-95b6-87774f69a715 2024-02-02T15:18:27.638 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-02-02T15:18:27.638 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2024-02-02T15:18:27.638 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout services: 2024-02-02T15:18:27.638 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.544102s) 2024-02-02T15:18:27.638 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-02-02T15:18:27.638 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-02-02T15:18:27.638 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2024-02-02T15:18:27.638 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout data: 2024-02-02T15:18:27.638 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-02-02T15:18:27.638 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-02-02T15:18:27.638 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-02-02T15:18:27.638 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout pgs: 2024-02-02T15:18:27.638 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2024-02-02T15:18:27.639 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:27.201+0000 7f712de00700 1 Processor -- start 2024-02-02T15:18:27.639 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:27.201+0000 7f712de00700 1 -- start start 2024-02-02T15:18:27.639 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:27.201+0000 7f712de00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f712807be60 0x7f712807a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:18:27.639 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:27.201+0000 7f712de00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f712807a890 con 0x7f712807be60 2024-02-02T15:18:27.639 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:27.202+0000 7f7126a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f712807be60 0x7f712807a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:18:27.639 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:27.202+0000 7f7126a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f712807be60 0x7f712807a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46334/0 (socket says 172.21.15.1:46334) 2024-02-02T15:18:27.639 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:27.202+0000 7f7126a00700 1 -- 172.21.15.1:0/3026093423 learned_addr learned my addr 172.21.15.1:0/3026093423 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:18:27.639 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:27.202+0000 7f7126a00700 1 -- 172.21.15.1:0/3026093423 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f712807a9d0 con 0x7f712807be60 2024-02-02T15:18:27.639 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:27.203+0000 7f7126a00700 1 --2- 172.21.15.1:0/3026093423 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f712807be60 0x7f712807a2c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f71180098a0 tx=0x7f7118009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ab04403e9c318c7 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:18:27.639 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:27.203+0000 7f7125600700 1 -- 172.21.15.1:0/3026093423 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7118004160 con 0x7f712807be60 2024-02-02T15:18:27.639 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:27.203+0000 7f7125600700 1 -- 172.21.15.1:0/3026093423 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71180042c0 con 0x7f712807be60 2024-02-02T15:18:27.639 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:27.203+0000 7f7125600700 1 -- 172.21.15.1:0/3026093423 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f71180045a0 con 0x7f712807be60 2024-02-02T15:18:27.639 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:27.203+0000 7f712de00700 1 -- 172.21.15.1:0/3026093423 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f712807be60 msgr2=0x7f712807a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:18:27.640 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:27.203+0000 7f712de00700 1 --2- 172.21.15.1:0/3026093423 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f712807be60 0x7f712807a2c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f71180098a0 tx=0x7f7118009bb0 comp rx=0 tx=0).stop 2024-02-02T15:18:27.640 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:27.204+0000 7f712de00700 1 -- 172.21.15.1:0/3026093423 shutdown_connections 2024-02-02T15:18:27.640 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:27.204+0000 7f712de00700 1 --2- 172.21.15.1:0/3026093423 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f712807be60 0x7f712807a2c0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:18:27.640 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:27.204+0000 7f712de00700 1 -- 172.21.15.1:0/3026093423 >> 172.21.15.1:0/3026093423 conn(0x7f7128101c20 msgr2=0x7f7128104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:18:27.640 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:27.204+0000 7f712de00700 1 -- 172.21.15.1:0/3026093423 shutdown_connections 2024-02-02T15:18:27.640 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:27.204+0000 7f712de00700 1 -- 172.21.15.1:0/3026093423 wait complete. 2024-02-02T15:18:27.640 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:27.205+0000 7f712de00700 1 Processor -- start 2024-02-02T15:18:27.640 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:27.205+0000 7f712de00700 1 -- start start 2024-02-02T15:18:27.640 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:27.206+0000 7f712de00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f712807be60 0x7f712819bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:18:27.640 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:27.206+0000 7f712de00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f712819c230 con 0x7f712807be60 2024-02-02T15:18:27.640 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:27.206+0000 7f7126a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f712807be60 0x7f712819bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:18:27.640 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:27.206+0000 7f7126a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f712807be60 0x7f712819bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46348/0 (socket says 172.21.15.1:46348) 2024-02-02T15:18:27.640 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:27.206+0000 7f7126a00700 1 -- 172.21.15.1:0/3460390752 learned_addr learned my addr 172.21.15.1:0/3460390752 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:18:27.640 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:27.207+0000 7f7126a00700 1 -- 172.21.15.1:0/3460390752 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7118009550 con 0x7f712807be60 2024-02-02T15:18:27.641 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:27.207+0000 7f7126a00700 1 --2- 172.21.15.1:0/3460390752 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f712807be60 0x7f712819bcf0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f712807b330 tx=0x7f7118004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:18:27.641 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:27.207+0000 7f712ca00700 1 -- 172.21.15.1:0/3460390752 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7118011630 con 0x7f712807be60 2024-02-02T15:18:27.641 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:27.207+0000 7f712ca00700 1 -- 172.21.15.1:0/3460390752 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7118011790 con 0x7f712807be60 2024-02-02T15:18:27.641 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:27.207+0000 7f712ca00700 1 -- 172.21.15.1:0/3460390752 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7118011a30 con 0x7f712807be60 2024-02-02T15:18:27.641 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:27.207+0000 7f712de00700 1 -- 172.21.15.1:0/3460390752 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f712819c430 con 0x7f712807be60 2024-02-02T15:18:27.641 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:27.208+0000 7f712de00700 1 -- 172.21.15.1:0/3460390752 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f712819c850 con 0x7f712807be60 2024-02-02T15:18:27.641 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:27.208+0000 7f712ca00700 1 -- 172.21.15.1:0/3460390752 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f7118022070 con 0x7f712807be60 2024-02-02T15:18:27.641 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:27.208+0000 7f712ca00700 1 -- 172.21.15.1:0/3460390752 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f711801c450 con 0x7f712807be60 2024-02-02T15:18:27.641 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:27.209+0000 7f712de00700 1 -- 172.21.15.1:0/3460390752 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7128195c00 con 0x7f712807be60 2024-02-02T15:18:27.641 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:27.212+0000 7f712ca00700 1 -- 172.21.15.1:0/3460390752 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72491 (secure 0 0 0) 0x7f711801b450 con 0x7f712807be60 2024-02-02T15:18:27.641 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:27.288+0000 7f712de00700 1 -- 172.21.15.1:0/3460390752 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f712802cc70 con 0x7f712807be60 2024-02-02T15:18:27.641 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:27.288+0000 7f712ca00700 1 -- 172.21.15.1:0/3460390752 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7f711801b5f0 con 0x7f712807be60 2024-02-02T15:18:27.641 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:27.289+0000 7f712de00700 1 -- 172.21.15.1:0/3460390752 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f712807be60 msgr2=0x7f712819bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:18:27.642 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:27.289+0000 7f712de00700 1 --2- 172.21.15.1:0/3460390752 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f712807be60 0x7f712819bcf0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f712807b330 tx=0x7f7118004750 comp rx=0 tx=0).stop 2024-02-02T15:18:27.642 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:27.289+0000 7f712de00700 1 -- 172.21.15.1:0/3460390752 shutdown_connections 2024-02-02T15:18:27.642 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:27.289+0000 7f712de00700 1 --2- 172.21.15.1:0/3460390752 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f712807be60 0x7f712819bcf0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:18:27.642 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:27.289+0000 7f712de00700 1 -- 172.21.15.1:0/3460390752 >> 172.21.15.1:0/3460390752 conn(0x7f7128101c20 msgr2=0x7f71281035b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:18:27.642 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:27.289+0000 7f712de00700 1 -- 172.21.15.1:0/3460390752 shutdown_connections 2024-02-02T15:18:27.642 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:27.290+0000 7f712de00700 1 -- 172.21.15.1:0/3460390752 wait complete. 2024-02-02T15:18:27.642 INFO:teuthology.orchestra.run.smithi001.stdout:mon is available 2024-02-02T15:18:27.642 INFO:teuthology.orchestra.run.smithi001.stdout:Assimilating anything we can from ceph.conf... 2024-02-02T15:18:27.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:27 smithi001 ceph-mon[34984]: mon.a is new leader, mons a in quorum (ranks 0) 2024-02-02T15:18:27.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:27 smithi001 ceph-mon[34984]: monmap e1: 1 mons at {a=[v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-02T15:18:27.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:27 smithi001 ceph-mon[34984]: fsmap 2024-02-02T15:18:27.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:27 smithi001 ceph-mon[34984]: osdmap e1: 0 total, 0 up, 0 in 2024-02-02T15:18:27.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:27 smithi001 ceph-mon[34984]: mgrmap e1: no daemons active 2024-02-02T15:18:27.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:27 smithi001 ceph-mon[34984]: from='client.? 172.21.15.1:0/3460390752' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-02-02T15:18:28.521 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2024-02-02T15:18:28.521 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout [global] 2024-02-02T15:18:28.521 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout fsid = 371a409c-c1de-11ee-95b6-87774f69a715 2024-02-02T15:18:28.521 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.1:3300,v1:172.21.15.1:6789] 2024-02-02T15:18:28.522 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-02-02T15:18:28.522 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-02-02T15:18:28.522 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-02-02T15:18:28.522 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-02-02T15:18:28.522 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2024-02-02T15:18:28.522 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout [mgr] 2024-02-02T15:18:28.522 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-02-02T15:18:28.522 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2024-02-02T15:18:28.522 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout [osd] 2024-02-02T15:18:28.522 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-02-02T15:18:28.522 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-02-02T15:18:28.522 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-02-02T15:18:28.522 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-02-02T15:18:28.522 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-02-02T15:18:28.523 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:28.066+0000 7f8f95600700 1 Processor -- start 2024-02-02T15:18:28.523 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:28.066+0000 7f8f95600700 1 -- start start 2024-02-02T15:18:28.523 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:28.066+0000 7f8f95600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8f9007be40 0x7f8f9007a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:18:28.523 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:28.066+0000 7f8f95600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f9007a870 con 0x7f8f9007be40 2024-02-02T15:18:28.523 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:28.067+0000 7f8f8ea00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8f9007be40 0x7f8f9007a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:18:28.523 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:28.067+0000 7f8f8ea00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8f9007be40 0x7f8f9007a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46360/0 (socket says 172.21.15.1:46360) 2024-02-02T15:18:28.523 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:28.067+0000 7f8f8ea00700 1 -- 172.21.15.1:0/3278358392 learned_addr learned my addr 172.21.15.1:0/3278358392 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:18:28.523 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:28.067+0000 7f8f8ea00700 1 -- 172.21.15.1:0/3278358392 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f9007a9b0 con 0x7f8f9007be40 2024-02-02T15:18:28.523 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:28.068+0000 7f8f8ea00700 1 --2- 172.21.15.1:0/3278358392 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8f9007be40 0x7f8f9007a2a0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f8f800098a0 tx=0x7f8f80009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b014bcd402a1d316 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:18:28.523 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:28.068+0000 7f8f8d600700 1 -- 172.21.15.1:0/3278358392 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8f80004160 con 0x7f8f9007be40 2024-02-02T15:18:28.523 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:28.068+0000 7f8f8d600700 1 -- 172.21.15.1:0/3278358392 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f800042c0 con 0x7f8f9007be40 2024-02-02T15:18:28.523 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:28.068+0000 7f8f8d600700 1 -- 172.21.15.1:0/3278358392 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8f800045a0 con 0x7f8f9007be40 2024-02-02T15:18:28.523 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:28.068+0000 7f8f95600700 1 -- 172.21.15.1:0/3278358392 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8f9007be40 msgr2=0x7f8f9007a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:18:28.523 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:28.068+0000 7f8f95600700 1 --2- 172.21.15.1:0/3278358392 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8f9007be40 0x7f8f9007a2a0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f8f800098a0 tx=0x7f8f80009bb0 comp rx=0 tx=0).stop 2024-02-02T15:18:28.524 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:28.069+0000 7f8f95600700 1 -- 172.21.15.1:0/3278358392 shutdown_connections 2024-02-02T15:18:28.524 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:28.069+0000 7f8f95600700 1 --2- 172.21.15.1:0/3278358392 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8f9007be40 0x7f8f9007a2a0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:18:28.524 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:28.069+0000 7f8f95600700 1 -- 172.21.15.1:0/3278358392 >> 172.21.15.1:0/3278358392 conn(0x7f8f90101c20 msgr2=0x7f8f90104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:18:28.524 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:28.069+0000 7f8f95600700 1 -- 172.21.15.1:0/3278358392 shutdown_connections 2024-02-02T15:18:28.524 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:28.069+0000 7f8f95600700 1 -- 172.21.15.1:0/3278358392 wait complete. 2024-02-02T15:18:28.524 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:28.070+0000 7f8f95600700 1 Processor -- start 2024-02-02T15:18:28.524 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:28.070+0000 7f8f95600700 1 -- start start 2024-02-02T15:18:28.524 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:28.070+0000 7f8f95600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8f9007be40 0x7f8f9019bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:18:28.524 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:28.070+0000 7f8f95600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f9019c240 con 0x7f8f9007be40 2024-02-02T15:18:28.524 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:28.070+0000 7f8f8ea00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8f9007be40 0x7f8f9019bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:18:28.524 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:28.070+0000 7f8f8ea00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8f9007be40 0x7f8f9019bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46372/0 (socket says 172.21.15.1:46372) 2024-02-02T15:18:28.524 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:28.070+0000 7f8f8ea00700 1 -- 172.21.15.1:0/76299563 learned_addr learned my addr 172.21.15.1:0/76299563 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:18:28.524 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:28.071+0000 7f8f8ea00700 1 -- 172.21.15.1:0/76299563 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f80009550 con 0x7f8f9007be40 2024-02-02T15:18:28.525 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:28.071+0000 7f8f8ea00700 1 --2- 172.21.15.1:0/76299563 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8f9007be40 0x7f8f9019bd00 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f8f9007b310 tx=0x7f8f80004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:18:28.525 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:28.071+0000 7f8f87400700 1 -- 172.21.15.1:0/76299563 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8f80011630 con 0x7f8f9007be40 2024-02-02T15:18:28.525 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:28.071+0000 7f8f87400700 1 -- 172.21.15.1:0/76299563 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f80011790 con 0x7f8f9007be40 2024-02-02T15:18:28.525 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:28.071+0000 7f8f87400700 1 -- 172.21.15.1:0/76299563 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8f80011a30 con 0x7f8f9007be40 2024-02-02T15:18:28.525 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:28.071+0000 7f8f95600700 1 -- 172.21.15.1:0/76299563 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f9019c440 con 0x7f8f9007be40 2024-02-02T15:18:28.525 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:28.071+0000 7f8f95600700 1 -- 172.21.15.1:0/76299563 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f9019c860 con 0x7f8f9007be40 2024-02-02T15:18:28.525 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:28.071+0000 7f8f87400700 1 -- 172.21.15.1:0/76299563 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f8f80022070 con 0x7f8f9007be40 2024-02-02T15:18:28.525 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:28.072+0000 7f8f87400700 1 -- 172.21.15.1:0/76299563 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f8f8001c450 con 0x7f8f9007be40 2024-02-02T15:18:28.527 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:28.072+0000 7f8f95600700 1 -- 172.21.15.1:0/76299563 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f90195bb0 con 0x7f8f9007be40 2024-02-02T15:18:28.527 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:28.075+0000 7f8f87400700 1 -- 172.21.15.1:0/76299563 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72491 (secure 0 0 0) 0x7f8f8001b450 con 0x7f8f9007be40 2024-02-02T15:18:28.527 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:28.148+0000 7f8f95600700 1 -- 172.21.15.1:0/76299563 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f8f9002cf20 con 0x7f8f9007be40 2024-02-02T15:18:28.527 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:28.152+0000 7f8f87400700 1 -- 172.21.15.1:0/76299563 <== mon.0 v2:172.21.15.1:3300/0 7 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f8f8001b5f0 con 0x7f8f9007be40 2024-02-02T15:18:28.527 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:28.152+0000 7f8f87400700 1 -- 172.21.15.1:0/76299563 <== mon.0 v2:172.21.15.1:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+427 (secure 0 0 0) 0x7f8f800375f0 con 0x7f8f9007be40 2024-02-02T15:18:28.527 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:28.153+0000 7f8f95600700 1 -- 172.21.15.1:0/76299563 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8f9007be40 msgr2=0x7f8f9019bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:18:28.527 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:28.153+0000 7f8f95600700 1 --2- 172.21.15.1:0/76299563 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8f9007be40 0x7f8f9019bd00 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f8f9007b310 tx=0x7f8f80004750 comp rx=0 tx=0).stop 2024-02-02T15:18:28.527 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:28.153+0000 7f8f95600700 1 -- 172.21.15.1:0/76299563 shutdown_connections 2024-02-02T15:18:28.527 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:28.153+0000 7f8f95600700 1 --2- 172.21.15.1:0/76299563 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8f9007be40 0x7f8f9019bd00 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:18:28.527 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:28.153+0000 7f8f95600700 1 -- 172.21.15.1:0/76299563 >> 172.21.15.1:0/76299563 conn(0x7f8f90101c20 msgr2=0x7f8f90103600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:18:28.528 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:28.153+0000 7f8f95600700 1 -- 172.21.15.1:0/76299563 shutdown_connections 2024-02-02T15:18:28.528 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:28.154+0000 7f8f95600700 1 -- 172.21.15.1:0/76299563 wait complete. 2024-02-02T15:18:28.528 INFO:teuthology.orchestra.run.smithi001.stdout:Generating new minimal ceph.conf... 2024-02-02T15:18:28.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:28 smithi001 ceph-mon[34984]: from='client.? 172.21.15.1:0/76299563' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-02-02T15:18:28.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:28 smithi001 ceph-mon[34984]: from='client.? 172.21.15.1:0/76299563' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-02-02T15:18:29.326 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:28.925+0000 7feaf3800700 1 Processor -- start 2024-02-02T15:18:29.327 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:28.925+0000 7feaf3800700 1 -- start start 2024-02-02T15:18:29.327 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:28.925+0000 7feaf3800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feaec106620 0x7feaec106a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:18:29.327 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:28.925+0000 7feaf3800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaec107010 con 0x7feaec106620 2024-02-02T15:18:29.327 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:28.926+0000 7feaf0e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feaec106620 0x7feaec106a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:18:29.327 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:28.926+0000 7feaf0e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feaec106620 0x7feaec106a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46374/0 (socket says 172.21.15.1:46374) 2024-02-02T15:18:29.327 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:28.926+0000 7feaf0e00700 1 -- 172.21.15.1:0/2762487142 learned_addr learned my addr 172.21.15.1:0/2762487142 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:18:29.327 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:28.926+0000 7feaf0e00700 1 -- 172.21.15.1:0/2762487142 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaec107820 con 0x7feaec106620 2024-02-02T15:18:29.327 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:28.926+0000 7feaf0e00700 1 --2- 172.21.15.1:0/2762487142 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feaec106620 0x7feaec106a40 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7feadc0098a0 tx=0x7feadc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c77624b09cf0499b server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:18:29.327 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:28.927+0000 7feaeb400700 1 -- 172.21.15.1:0/2762487142 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feadc004160 con 0x7feaec106620 2024-02-02T15:18:29.327 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:28.927+0000 7feaeb400700 1 -- 172.21.15.1:0/2762487142 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7feadc0042c0 con 0x7feaec106620 2024-02-02T15:18:29.327 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:28.927+0000 7feaeb400700 1 -- 172.21.15.1:0/2762487142 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feadc011a00 con 0x7feaec106620 2024-02-02T15:18:29.328 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:28.927+0000 7feaf3800700 1 -- 172.21.15.1:0/2762487142 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feaec106620 msgr2=0x7feaec106a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:18:29.328 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:28.927+0000 7feaf3800700 1 --2- 172.21.15.1:0/2762487142 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feaec106620 0x7feaec106a40 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7feadc0098a0 tx=0x7feadc009bb0 comp rx=0 tx=0).stop 2024-02-02T15:18:29.328 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:28.927+0000 7feaf3800700 1 -- 172.21.15.1:0/2762487142 shutdown_connections 2024-02-02T15:18:29.328 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:28.927+0000 7feaf3800700 1 --2- 172.21.15.1:0/2762487142 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feaec106620 0x7feaec106a40 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:18:29.328 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:28.927+0000 7feaf3800700 1 -- 172.21.15.1:0/2762487142 >> 172.21.15.1:0/2762487142 conn(0x7feaec101c20 msgr2=0x7feaec104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:18:29.328 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:28.927+0000 7feaf3800700 1 -- 172.21.15.1:0/2762487142 shutdown_connections 2024-02-02T15:18:29.328 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:28.928+0000 7feaf3800700 1 -- 172.21.15.1:0/2762487142 wait complete. 2024-02-02T15:18:29.328 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:28.929+0000 7feaf3800700 1 Processor -- start 2024-02-02T15:18:29.328 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:28.929+0000 7feaf3800700 1 -- start start 2024-02-02T15:18:29.328 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:28.930+0000 7feaf3800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feaec106620 0x7feaec19a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:18:29.328 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:28.930+0000 7feaf3800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaec19a8c0 con 0x7feaec106620 2024-02-02T15:18:29.328 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:28.930+0000 7feaf0e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feaec106620 0x7feaec19a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:18:29.328 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:28.930+0000 7feaf0e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feaec106620 0x7feaec19a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46376/0 (socket says 172.21.15.1:46376) 2024-02-02T15:18:29.328 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:28.930+0000 7feaf0e00700 1 -- 172.21.15.1:0/933067237 learned_addr learned my addr 172.21.15.1:0/933067237 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:18:29.328 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:28.930+0000 7feaf0e00700 1 -- 172.21.15.1:0/933067237 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feadc009550 con 0x7feaec106620 2024-02-02T15:18:29.329 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:28.931+0000 7feaf0e00700 1 --2- 172.21.15.1:0/933067237 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feaec106620 0x7feaec19a380 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7feadc009130 tx=0x7feadc011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:18:29.329 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:28.931+0000 7feae9600700 1 -- 172.21.15.1:0/933067237 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feadc003ab0 con 0x7feaec106620 2024-02-02T15:18:29.329 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:28.931+0000 7feaf3800700 1 -- 172.21.15.1:0/933067237 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaec19aac0 con 0x7feaec106620 2024-02-02T15:18:29.329 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:28.931+0000 7feae9600700 1 -- 172.21.15.1:0/933067237 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7feadc003c10 con 0x7feaec106620 2024-02-02T15:18:29.329 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:28.931+0000 7feae9600700 1 -- 172.21.15.1:0/933067237 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feadc0247d0 con 0x7feaec106620 2024-02-02T15:18:29.329 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:28.931+0000 7feaf3800700 1 -- 172.21.15.1:0/933067237 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaec19aee0 con 0x7feaec106620 2024-02-02T15:18:29.329 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:28.931+0000 7feae9600700 1 -- 172.21.15.1:0/933067237 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7feadc024d00 con 0x7feaec106620 2024-02-02T15:18:29.329 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:28.931+0000 7feae9600700 1 -- 172.21.15.1:0/933067237 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7feadc019450 con 0x7feaec106620 2024-02-02T15:18:29.329 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:28.932+0000 7feaf3800700 1 -- 172.21.15.1:0/933067237 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feaec062370 con 0x7feaec106620 2024-02-02T15:18:29.329 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:28.935+0000 7feae9600700 1 -- 172.21.15.1:0/933067237 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72491 (secure 0 0 0) 0x7feadc01abb0 con 0x7feaec106620 2024-02-02T15:18:29.329 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:29.008+0000 7feaf3800700 1 -- 172.21.15.1:0/933067237 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7feaec19deb0 con 0x7feaec106620 2024-02-02T15:18:29.329 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:29.009+0000 7feae9600700 1 -- 172.21.15.1:0/933067237 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+173 (secure 0 0 0) 0x7feadc01ad50 con 0x7feaec106620 2024-02-02T15:18:29.329 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:29.010+0000 7feaf3800700 1 -- 172.21.15.1:0/933067237 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feaec106620 msgr2=0x7feaec19a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:18:29.330 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:29.010+0000 7feaf3800700 1 --2- 172.21.15.1:0/933067237 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feaec106620 0x7feaec19a380 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7feadc009130 tx=0x7feadc011ad0 comp rx=0 tx=0).stop 2024-02-02T15:18:29.330 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:29.010+0000 7feaf3800700 1 -- 172.21.15.1:0/933067237 shutdown_connections 2024-02-02T15:18:29.330 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:29.010+0000 7feaf3800700 1 --2- 172.21.15.1:0/933067237 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feaec106620 0x7feaec19a380 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:18:29.330 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:29.010+0000 7feaf3800700 1 -- 172.21.15.1:0/933067237 >> 172.21.15.1:0/933067237 conn(0x7feaec101c20 msgr2=0x7feaec103310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:18:29.330 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:29.010+0000 7feaf3800700 1 -- 172.21.15.1:0/933067237 shutdown_connections 2024-02-02T15:18:29.330 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:29.010+0000 7feaf3800700 1 -- 172.21.15.1:0/933067237 wait complete. 2024-02-02T15:18:29.330 INFO:teuthology.orchestra.run.smithi001.stdout:Restarting the monitor... 2024-02-02T15:18:29.628 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:29 smithi001 systemd[1]: Stopping Ceph mon.a for 371a409c-c1de-11ee-95b6-87774f69a715... 2024-02-02T15:18:29.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:29 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mon-a[34961]: 2024-02-02T15:18:29.626+0000 7f86f0000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-02T15:18:29.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:29 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mon-a[34961]: 2024-02-02T15:18:29.626+0000 7f86f0000700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-02-02T15:18:29.927 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:29 smithi001 bash[35314]: ceph-371a409c-c1de-11ee-95b6-87774f69a715-mon-a 2024-02-02T15:18:30.575 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 systemd[1]: ceph-371a409c-c1de-11ee-95b6-87774f69a715@mon.a.service: Succeeded. 2024-02-02T15:18:30.575 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 systemd[1]: Stopped Ceph mon.a for 371a409c-c1de-11ee-95b6-87774f69a715. 2024-02-02T15:18:30.575 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 systemd[1]: Starting Ceph mon.a for 371a409c-c1de-11ee-95b6-87774f69a715... 2024-02-02T15:18:30.668 INFO:teuthology.orchestra.run.smithi001.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-02-02T15:18:30.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 bash[35433]: 5fecb94058f06665ce84cb63f17dd2b4cf9c2a7ddd63fe9f5c284717d43fd481 2024-02-02T15:18:30.942 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 systemd[1]: Started Ceph mon.a for 371a409c-c1de-11ee-95b6-87774f69a715. 2024-02-02T15:18:30.942 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: set uid:gid to 167:167 (ceph:ceph) 2024-02-02T15:18:30.942 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: ceph version 18.2.1-325-g0fb32788 (0fb327883c124fd83155c94db0804f823b805a12) reef (stable), process ceph-mon, pid 7 2024-02-02T15:18:30.942 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: pidfile_write: ignore empty --pid-file 2024-02-02T15:18:30.942 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: load: jerasure load: lrc 2024-02-02T15:18:30.942 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: RocksDB version: 7.9.2 2024-02-02T15:18:30.942 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Git sha 0 2024-02-02T15:18:30.942 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Compile date 2024-02-02 06:52:11 2024-02-02T15:18:30.942 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: DB SUMMARY 2024-02-02T15:18:30.942 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: DB Session ID: BFHCNUWSSRQ0OOH7BEES 2024-02-02T15:18:30.942 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: CURRENT file: CURRENT 2024-02-02T15:18:30.942 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: IDENTITY file: IDENTITY 2024-02-02T15:18:30.942 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-02-02T15:18:30.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-02-02T15:18:30.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 97952 ; 2024-02-02T15:18:30.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.error_if_exists: 0 2024-02-02T15:18:30.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.create_if_missing: 0 2024-02-02T15:18:30.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.paranoid_checks: 1 2024-02-02T15:18:30.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.flush_verify_memtable_count: 1 2024-02-02T15:18:30.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-02-02T15:18:30.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-02-02T15:18:30.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.env: 0x55f64207d720 2024-02-02T15:18:30.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.fs: PosixFileSystem 2024-02-02T15:18:30.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.info_log: 0x55f644867440 2024-02-02T15:18:30.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.max_file_opening_threads: 16 2024-02-02T15:18:30.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.statistics: (nil) 2024-02-02T15:18:30.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.use_fsync: 0 2024-02-02T15:18:30.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.max_log_file_size: 0 2024-02-02T15:18:30.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-02T15:18:30.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.log_file_time_to_roll: 0 2024-02-02T15:18:30.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.keep_log_file_num: 1000 2024-02-02T15:18:30.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.recycle_log_file_num: 0 2024-02-02T15:18:30.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.allow_fallocate: 1 2024-02-02T15:18:30.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.allow_mmap_reads: 0 2024-02-02T15:18:30.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.allow_mmap_writes: 0 2024-02-02T15:18:30.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.use_direct_reads: 0 2024-02-02T15:18:30.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-02T15:18:30.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.create_missing_column_families: 0 2024-02-02T15:18:30.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.db_log_dir: 2024-02-02T15:18:30.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.wal_dir: 2024-02-02T15:18:30.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.table_cache_numshardbits: 6 2024-02-02T15:18:30.945 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.WAL_ttl_seconds: 0 2024-02-02T15:18:30.945 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.WAL_size_limit_MB: 0 2024-02-02T15:18:30.945 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-02T15:18:30.945 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-02T15:18:30.945 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.is_fd_close_on_exec: 1 2024-02-02T15:18:30.945 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.advise_random_on_open: 1 2024-02-02T15:18:30.945 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.db_write_buffer_size: 0 2024-02-02T15:18:30.945 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.write_buffer_manager: 0x55f643af8320 2024-02-02T15:18:30.945 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-02T15:18:30.945 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-02T15:18:30.945 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.use_adaptive_mutex: 0 2024-02-02T15:18:30.945 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.rate_limiter: (nil) 2024-02-02T15:18:30.945 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-02T15:18:30.945 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.wal_recovery_mode: 2 2024-02-02T15:18:30.946 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.enable_thread_tracking: 0 2024-02-02T15:18:30.946 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.enable_pipelined_write: 0 2024-02-02T15:18:30.946 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.unordered_write: 0 2024-02-02T15:18:30.946 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-02T15:18:30.946 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-02T15:18:30.946 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-02T15:18:30.946 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-02T15:18:30.946 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.row_cache: None 2024-02-02T15:18:30.946 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.wal_filter: None 2024-02-02T15:18:30.946 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-02T15:18:30.946 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.allow_ingest_behind: 0 2024-02-02T15:18:30.946 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.two_write_queues: 0 2024-02-02T15:18:30.946 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.manual_wal_flush: 0 2024-02-02T15:18:30.946 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.wal_compression: 0 2024-02-02T15:18:30.947 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.atomic_flush: 0 2024-02-02T15:18:30.947 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-02T15:18:30.947 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.persist_stats_to_disk: 0 2024-02-02T15:18:30.947 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.write_dbid_to_manifest: 0 2024-02-02T15:18:30.947 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.log_readahead_size: 0 2024-02-02T15:18:30.947 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-02-02T15:18:30.947 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.best_efforts_recovery: 0 2024-02-02T15:18:30.947 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-02-02T15:18:30.947 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-02-02T15:18:30.947 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.allow_data_in_errors: 0 2024-02-02T15:18:30.947 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.db_host_id: __hostname__ 2024-02-02T15:18:30.947 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.enforce_single_del_contracts: true 2024-02-02T15:18:30.947 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.max_background_jobs: 2 2024-02-02T15:18:30.947 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.max_background_compactions: -1 2024-02-02T15:18:30.947 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.max_subcompactions: 1 2024-02-02T15:18:30.948 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-02T15:18:30.948 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-02T15:18:30.948 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.delayed_write_rate : 16777216 2024-02-02T15:18:30.948 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.max_total_wal_size: 0 2024-02-02T15:18:30.948 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-02T15:18:30.948 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.stats_dump_period_sec: 600 2024-02-02T15:18:30.948 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.stats_persist_period_sec: 600 2024-02-02T15:18:30.948 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-02T15:18:30.948 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.max_open_files: -1 2024-02-02T15:18:30.948 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.bytes_per_sync: 0 2024-02-02T15:18:30.948 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.wal_bytes_per_sync: 0 2024-02-02T15:18:30.948 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.strict_bytes_per_sync: 0 2024-02-02T15:18:30.948 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.compaction_readahead_size: 0 2024-02-02T15:18:30.948 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.max_background_flushes: -1 2024-02-02T15:18:30.948 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Compression algorithms supported: 2024-02-02T15:18:30.948 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-02T15:18:30.949 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: kZSTD supported: 0 2024-02-02T15:18:30.949 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: kXpressCompression supported: 0 2024-02-02T15:18:30.949 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: kLZ4HCCompression supported: 1 2024-02-02T15:18:30.949 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: kZlibCompression supported: 1 2024-02-02T15:18:30.949 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: kSnappyCompression supported: 1 2024-02-02T15:18:30.949 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: kLZ4Compression supported: 1 2024-02-02T15:18:30.949 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: kBZip2Compression supported: 0 2024-02-02T15:18:30.949 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Fast CRC32 supported: Supported on x86 2024-02-02T15:18:30.949 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: DMutex implementation: pthread_mutex_t 2024-02-02T15:18:30.949 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-02-02T15:18:30.949 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-02-02T15:18:30.949 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-02T15:18:30.949 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.merge_operator: 2024-02-02T15:18:30.949 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.compaction_filter: None 2024-02-02T15:18:30.949 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.compaction_filter_factory: None 2024-02-02T15:18:30.949 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.sst_partitioner_factory: None 2024-02-02T15:18:30.949 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.memtable_factory: SkipListFactory 2024-02-02T15:18:30.950 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.table_factory: BlockBasedTable 2024-02-02T15:18:30.950 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55f644867540) 2024-02-02T15:18:30.950 INFO:journalctl@ceph.mon.a.smithi001.stdout: cache_index_and_filter_blocks: 1 2024-02-02T15:18:30.950 INFO:journalctl@ceph.mon.a.smithi001.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-02T15:18:30.950 INFO:journalctl@ceph.mon.a.smithi001.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-02T15:18:30.950 INFO:journalctl@ceph.mon.a.smithi001.stdout: pin_top_level_index_and_filter: 1 2024-02-02T15:18:30.950 INFO:journalctl@ceph.mon.a.smithi001.stdout: index_type: 0 2024-02-02T15:18:30.950 INFO:journalctl@ceph.mon.a.smithi001.stdout: data_block_index_type: 0 2024-02-02T15:18:30.950 INFO:journalctl@ceph.mon.a.smithi001.stdout: index_shortening: 1 2024-02-02T15:18:30.950 INFO:journalctl@ceph.mon.a.smithi001.stdout: data_block_hash_table_util_ratio: 0.750000 2024-02-02T15:18:30.950 INFO:journalctl@ceph.mon.a.smithi001.stdout: checksum: 4 2024-02-02T15:18:30.950 INFO:journalctl@ceph.mon.a.smithi001.stdout: no_block_cache: 0 2024-02-02T15:18:30.950 INFO:journalctl@ceph.mon.a.smithi001.stdout: block_cache: 0x55f643b791f0 2024-02-02T15:18:30.950 INFO:journalctl@ceph.mon.a.smithi001.stdout: block_cache_name: BinnedLRUCache 2024-02-02T15:18:30.950 INFO:journalctl@ceph.mon.a.smithi001.stdout: block_cache_options: 2024-02-02T15:18:30.951 INFO:journalctl@ceph.mon.a.smithi001.stdout: capacity : 536870912 2024-02-02T15:18:30.951 INFO:journalctl@ceph.mon.a.smithi001.stdout: num_shard_bits : 4 2024-02-02T15:18:30.951 INFO:journalctl@ceph.mon.a.smithi001.stdout: strict_capacity_limit : 0 2024-02-02T15:18:30.951 INFO:journalctl@ceph.mon.a.smithi001.stdout: high_pri_pool_ratio: 0.000 2024-02-02T15:18:30.951 INFO:journalctl@ceph.mon.a.smithi001.stdout: block_cache_compressed: (nil) 2024-02-02T15:18:30.951 INFO:journalctl@ceph.mon.a.smithi001.stdout: persistent_cache: (nil) 2024-02-02T15:18:30.951 INFO:journalctl@ceph.mon.a.smithi001.stdout: block_size: 4096 2024-02-02T15:18:30.951 INFO:journalctl@ceph.mon.a.smithi001.stdout: block_size_deviation: 10 2024-02-02T15:18:30.951 INFO:journalctl@ceph.mon.a.smithi001.stdout: block_restart_interval: 16 2024-02-02T15:18:30.951 INFO:journalctl@ceph.mon.a.smithi001.stdout: index_block_restart_interval: 1 2024-02-02T15:18:30.951 INFO:journalctl@ceph.mon.a.smithi001.stdout: metadata_block_size: 4096 2024-02-02T15:18:30.951 INFO:journalctl@ceph.mon.a.smithi001.stdout: partition_filters: 0 2024-02-02T15:18:30.951 INFO:journalctl@ceph.mon.a.smithi001.stdout: use_delta_encoding: 1 2024-02-02T15:18:30.951 INFO:journalctl@ceph.mon.a.smithi001.stdout: filter_policy: bloomfilter 2024-02-02T15:18:30.951 INFO:journalctl@ceph.mon.a.smithi001.stdout: whole_key_filtering: 1 2024-02-02T15:18:30.952 INFO:journalctl@ceph.mon.a.smithi001.stdout: verify_compression: 0 2024-02-02T15:18:30.952 INFO:journalctl@ceph.mon.a.smithi001.stdout: read_amp_bytes_per_bit: 0 2024-02-02T15:18:30.952 INFO:journalctl@ceph.mon.a.smithi001.stdout: format_version: 5 2024-02-02T15:18:30.952 INFO:journalctl@ceph.mon.a.smithi001.stdout: enable_index_compression: 1 2024-02-02T15:18:30.952 INFO:journalctl@ceph.mon.a.smithi001.stdout: block_align: 0 2024-02-02T15:18:30.952 INFO:journalctl@ceph.mon.a.smithi001.stdout: max_auto_readahead_size: 262144 2024-02-02T15:18:30.952 INFO:journalctl@ceph.mon.a.smithi001.stdout: prepopulate_block_cache: 0 2024-02-02T15:18:30.952 INFO:journalctl@ceph.mon.a.smithi001.stdout: initial_auto_readahead_size: 8192 2024-02-02T15:18:30.952 INFO:journalctl@ceph.mon.a.smithi001.stdout: num_file_reads_for_auto_readahead: 2 2024-02-02T15:18:30.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.write_buffer_size: 33554432 2024-02-02T15:18:30.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.max_write_buffer_number: 2 2024-02-02T15:18:30.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.compression: NoCompression 2024-02-02T15:18:30.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.bottommost_compression: Disabled 2024-02-02T15:18:30.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.prefix_extractor: nullptr 2024-02-02T15:18:30.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-02T15:18:30.952 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.num_levels: 7 2024-02-02T15:18:30.953 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-02T15:18:30.953 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-02T15:18:30.953 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-02T15:18:30.953 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-02T15:18:30.953 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-02T15:18:30.953 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-02T15:18:30.953 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-02T15:18:30.953 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-02T15:18:30.953 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-02-02T15:18:30.953 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-02T15:18:30.953 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-02-02T15:18:30.953 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-02-02T15:18:30.953 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.compression_opts.window_bits: -14 2024-02-02T15:18:30.953 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.compression_opts.level: 32767 2024-02-02T15:18:30.953 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.compression_opts.strategy: 0 2024-02-02T15:18:30.953 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-02T15:18:30.954 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-02T15:18:30.954 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-02-02T15:18:30.954 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-02-02T15:18:30.954 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.compression_opts.enabled: false 2024-02-02T15:18:30.954 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-02-02T15:18:30.954 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-02T15:18:30.954 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-02T15:18:30.954 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-02T15:18:30.954 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.target_file_size_base: 67108864 2024-02-02T15:18:30.954 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.target_file_size_multiplier: 1 2024-02-02T15:18:30.954 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-02T15:18:30.954 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-02T15:18:30.954 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-02T15:18:30.954 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-02T15:18:30.954 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-02T15:18:30.954 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-02T15:18:30.955 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-02T15:18:30.955 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-02T15:18:30.955 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-02T15:18:30.955 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-02T15:18:30.955 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-02T15:18:30.955 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-02T15:18:30.955 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-02-02T15:18:30.955 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.arena_block_size: 1048576 2024-02-02T15:18:30.955 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-02T15:18:30.955 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-02T15:18:30.955 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.disable_auto_compactions: 0 2024-02-02T15:18:30.955 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-02T15:18:30.956 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-02T15:18:30.956 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-02T15:18:30.956 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-02T15:18:30.956 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-02T15:18:30.956 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-02T15:18:30.956 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-02T15:18:30.956 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-02T15:18:30.956 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-02T15:18:30.956 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-02T15:18:30.956 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.table_properties_collectors: 2024-02-02T15:18:30.957 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.inplace_update_support: 0 2024-02-02T15:18:30.957 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.inplace_update_num_locks: 10000 2024-02-02T15:18:30.957 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-02T15:18:30.957 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-02T15:18:30.957 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.memtable_huge_page_size: 0 2024-02-02T15:18:30.957 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.bloom_locality: 0 2024-02-02T15:18:30.957 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.max_successive_merges: 0 2024-02-02T15:18:30.957 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.optimize_filters_for_hits: 0 2024-02-02T15:18:30.957 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.paranoid_file_checks: 0 2024-02-02T15:18:30.957 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.force_consistency_checks: 1 2024-02-02T15:18:30.957 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.report_bg_io_stats: 0 2024-02-02T15:18:30.957 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.ttl: 2592000 2024-02-02T15:18:30.957 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.periodic_compaction_seconds: 0 2024-02-02T15:18:30.957 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-02-02T15:18:30.958 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-02-02T15:18:30.958 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.enable_blob_files: false 2024-02-02T15:18:30.958 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.min_blob_size: 0 2024-02-02T15:18:30.958 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.blob_file_size: 268435456 2024-02-02T15:18:30.958 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.blob_compression_type: NoCompression 2024-02-02T15:18:30.958 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.enable_blob_garbage_collection: false 2024-02-02T15:18:30.958 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-02-02T15:18:30.958 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-02-02T15:18:30.958 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-02-02T15:18:30.958 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.blob_file_starting_level: 0 2024-02-02T15:18:30.958 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-02-02T15:18:30.958 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-02-02T15:18:30.958 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-02-02T15:18:30.958 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: cd32694a-5b93-48c7-a7da-59b2cd2a9d87 2024-02-02T15:18:30.959 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: EVENT_LOG_v1 {"time_micros": 1706887110677756, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-02-02T15:18:30.959 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-02-02T15:18:30.959 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: EVENT_LOG_v1 {"time_micros": 1706887110679513, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 93316, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 309, "table_properties": {"data_size": 91314, "index_size": 284, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13717, "raw_average_key_size": 50, "raw_value_size": 83843, "raw_average_value_size": 308, "num_data_blocks": 11, "num_entries": 272, "num_filter_entries": 272, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1706887110, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "cd32694a-5b93-48c7-a7da-59b2cd2a9d87", "db_session_id": "BFHCNUWSSRQ0OOH7BEES", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-02-02T15:18:30.959 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: EVENT_LOG_v1 {"time_micros": 1706887110679665, "job": 1, "event": "recovery_finished"} 2024-02-02T15:18:30.959 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-02-02T15:18:30.959 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-02-02T15:18:30.959 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55f643c16000 2024-02-02T15:18:30.959 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: DB pointer 0x55f643c02000 2024-02-02T15:18:30.959 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-02-02T15:18:30.959 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-02-02T15:18:30.959 INFO:journalctl@ceph.mon.a.smithi001.stdout: ** DB Stats ** 2024-02-02T15:18:30.959 INFO:journalctl@ceph.mon.a.smithi001.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-02T15:18:30.959 INFO:journalctl@ceph.mon.a.smithi001.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-02T15:18:30.959 INFO:journalctl@ceph.mon.a.smithi001.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-02T15:18:30.960 INFO:journalctl@ceph.mon.a.smithi001.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-02T15:18:30.960 INFO:journalctl@ceph.mon.a.smithi001.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-02T15:18:30.960 INFO:journalctl@ceph.mon.a.smithi001.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-02T15:18:30.960 INFO:journalctl@ceph.mon.a.smithi001.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-02T15:18:30.960 INFO:journalctl@ceph.mon.a.smithi001.stdout: 2024-02-02T15:18:30.960 INFO:journalctl@ceph.mon.a.smithi001.stdout: ** Compaction Stats [default] ** 2024-02-02T15:18:30.960 INFO:journalctl@ceph.mon.a.smithi001.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-02T15:18:30.960 INFO:journalctl@ceph.mon.a.smithi001.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-02-02T15:18:30.960 INFO:journalctl@ceph.mon.a.smithi001.stdout: L0 2/0 92.96 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 81.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-02T15:18:30.960 INFO:journalctl@ceph.mon.a.smithi001.stdout: Sum 2/0 92.96 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 81.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-02T15:18:30.960 INFO:journalctl@ceph.mon.a.smithi001.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 81.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-02T15:18:30.960 INFO:journalctl@ceph.mon.a.smithi001.stdout: 2024-02-02T15:18:30.960 INFO:journalctl@ceph.mon.a.smithi001.stdout: ** Compaction Stats [default] ** 2024-02-02T15:18:30.960 INFO:journalctl@ceph.mon.a.smithi001.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-02T15:18:30.961 INFO:journalctl@ceph.mon.a.smithi001.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-02T15:18:30.961 INFO:journalctl@ceph.mon.a.smithi001.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 81.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-02T15:18:30.961 INFO:journalctl@ceph.mon.a.smithi001.stdout: 2024-02-02T15:18:30.961 INFO:journalctl@ceph.mon.a.smithi001.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-02-02T15:18:30.961 INFO:journalctl@ceph.mon.a.smithi001.stdout: 2024-02-02T15:18:30.961 INFO:journalctl@ceph.mon.a.smithi001.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-02T15:18:30.961 INFO:journalctl@ceph.mon.a.smithi001.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-02-02T15:18:30.961 INFO:journalctl@ceph.mon.a.smithi001.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-02T15:18:30.961 INFO:journalctl@ceph.mon.a.smithi001.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-02-02T15:18:30.961 INFO:journalctl@ceph.mon.a.smithi001.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-02-02T15:18:30.961 INFO:journalctl@ceph.mon.a.smithi001.stdout: AddFile(Keys): cumulative 0, interval 0 2024-02-02T15:18:30.961 INFO:journalctl@ceph.mon.a.smithi001.stdout: Cumulative compaction: 0.00 GB write, 12.32 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-02T15:18:30.961 INFO:journalctl@ceph.mon.a.smithi001.stdout: Interval compaction: 0.00 GB write, 12.32 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-02T15:18:30.962 INFO:journalctl@ceph.mon.a.smithi001.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-02T15:18:30.962 INFO:journalctl@ceph.mon.a.smithi001.stdout: Block cache BinnedLRUCache@0x55f643b791f0#7 capacity: 512.00 MB usage: 1.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-02-02T15:18:30.962 INFO:journalctl@ceph.mon.a.smithi001.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-02-02T15:18:30.962 INFO:journalctl@ceph.mon.a.smithi001.stdout: 2024-02-02T15:18:30.962 INFO:journalctl@ceph.mon.a.smithi001.stdout: ** File Read Latency Histogram By Level [default] ** 2024-02-02T15:18:30.962 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: starting mon.a rank 0 at public addrs [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] at bind addrs [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 371a409c-c1de-11ee-95b6-87774f69a715 2024-02-02T15:18:30.962 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: mon.a@-1(???) e1 preinit fsid 371a409c-c1de-11ee-95b6-87774f69a715 2024-02-02T15:18:30.962 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: mon.a@-1(???).mds e1 new map 2024-02-02T15:18:30.962 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: mon.a@-1(???).mds e1 print_map 2024-02-02T15:18:30.962 INFO:journalctl@ceph.mon.a.smithi001.stdout: e1 2024-02-02T15:18:30.962 INFO:journalctl@ceph.mon.a.smithi001.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-02T15:18:30.962 INFO:journalctl@ceph.mon.a.smithi001.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-02T15:18:30.962 INFO:journalctl@ceph.mon.a.smithi001.stdout: legacy client fscid: -1 2024-02-02T15:18:30.962 INFO:journalctl@ceph.mon.a.smithi001.stdout: 2024-02-02T15:18:30.962 INFO:journalctl@ceph.mon.a.smithi001.stdout: No filesystems configured 2024-02-02T15:18:30.963 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-02T15:18:30.963 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-02T15:18:30.963 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-02T15:18:30.963 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-02T15:18:30.963 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-02-02T15:18:30.963 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: mon.a is new leader, mons a in quorum (ranks 0) 2024-02-02T15:18:30.963 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: monmap e1: 1 mons at {a=[v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-02T15:18:30.963 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: fsmap 2024-02-02T15:18:30.963 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: osdmap e1: 0 total, 0 up, 0 in 2024-02-02T15:18:30.963 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:30 smithi001 ceph-mon[35470]: mgrmap e1: no daemons active 2024-02-02T15:18:31.500 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:31.088+0000 7f5fa1c00700 1 Processor -- start 2024-02-02T15:18:31.500 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:31.088+0000 7f5fa1c00700 1 -- start start 2024-02-02T15:18:31.500 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:31.088+0000 7f5fa1c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f9c106620 0x7f5f9c106a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:18:31.500 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:31.088+0000 7f5fa1c00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9c107010 con 0x7f5f9c106620 2024-02-02T15:18:31.501 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:31.089+0000 7f5f9aa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f9c106620 0x7f5f9c106a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:18:31.501 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:31.089+0000 7f5f9aa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f9c106620 0x7f5f9c106a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46382/0 (socket says 172.21.15.1:46382) 2024-02-02T15:18:31.501 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:31.089+0000 7f5f9aa00700 1 -- 172.21.15.1:0/318562383 learned_addr learned my addr 172.21.15.1:0/318562383 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:18:31.501 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:31.089+0000 7f5f9aa00700 1 -- 172.21.15.1:0/318562383 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f9c107820 con 0x7f5f9c106620 2024-02-02T15:18:31.501 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:31.090+0000 7f5f9aa00700 1 --2- 172.21.15.1:0/318562383 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f9c106620 0x7f5f9c106a40 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c0098a0 tx=0x7f5f8c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=222f82fb31c2f3eb server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:18:31.501 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:31.090+0000 7f5f99600700 1 -- 172.21.15.1:0/318562383 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5f8c004160 con 0x7f5f9c106620 2024-02-02T15:18:31.501 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:31.090+0000 7f5f99600700 1 -- 172.21.15.1:0/318562383 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5f8c0042c0 con 0x7f5f9c106620 2024-02-02T15:18:31.501 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:31.090+0000 7f5f99600700 1 -- 172.21.15.1:0/318562383 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5f8c011a00 con 0x7f5f9c106620 2024-02-02T15:18:31.501 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:31.091+0000 7f5fa1c00700 1 -- 172.21.15.1:0/318562383 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f9c106620 msgr2=0x7f5f9c106a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:18:31.501 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:31.091+0000 7f5fa1c00700 1 --2- 172.21.15.1:0/318562383 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f9c106620 0x7f5f9c106a40 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c0098a0 tx=0x7f5f8c009bb0 comp rx=0 tx=0).stop 2024-02-02T15:18:31.501 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:31.091+0000 7f5fa1c00700 1 -- 172.21.15.1:0/318562383 shutdown_connections 2024-02-02T15:18:31.501 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:31.091+0000 7f5fa1c00700 1 --2- 172.21.15.1:0/318562383 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f9c106620 0x7f5f9c106a40 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:18:31.502 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:31.091+0000 7f5fa1c00700 1 -- 172.21.15.1:0/318562383 >> 172.21.15.1:0/318562383 conn(0x7f5f9c101c20 msgr2=0x7f5f9c104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:18:31.502 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:31.091+0000 7f5fa1c00700 1 -- 172.21.15.1:0/318562383 shutdown_connections 2024-02-02T15:18:31.502 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:31.091+0000 7f5fa1c00700 1 -- 172.21.15.1:0/318562383 wait complete. 2024-02-02T15:18:31.502 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:31.092+0000 7f5fa1c00700 1 Processor -- start 2024-02-02T15:18:31.502 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:31.092+0000 7f5fa1c00700 1 -- start start 2024-02-02T15:18:31.502 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:31.092+0000 7f5fa1c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f9c106620 0x7f5f9c19a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:18:31.502 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:31.092+0000 7f5fa1c00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9c19a8c0 con 0x7f5f9c106620 2024-02-02T15:18:31.502 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:31.092+0000 7f5f9aa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f9c106620 0x7f5f9c19a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:18:31.502 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:31.092+0000 7f5f9aa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f9c106620 0x7f5f9c19a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46392/0 (socket says 172.21.15.1:46392) 2024-02-02T15:18:31.502 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:31.092+0000 7f5f9aa00700 1 -- 172.21.15.1:0/527174333 learned_addr learned my addr 172.21.15.1:0/527174333 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:18:31.502 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:31.093+0000 7f5f9aa00700 1 -- 172.21.15.1:0/527174333 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f8c009550 con 0x7f5f9c106620 2024-02-02T15:18:31.502 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:31.093+0000 7f5f9aa00700 1 --2- 172.21.15.1:0/527174333 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f9c106620 0x7f5f9c19a380 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c009130 tx=0x7f5f8c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:18:31.503 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:31.093+0000 7f5f93400700 1 -- 172.21.15.1:0/527174333 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5f8c003ab0 con 0x7f5f9c106620 2024-02-02T15:18:31.503 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:31.093+0000 7f5f93400700 1 -- 172.21.15.1:0/527174333 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f5f8c003c10 con 0x7f5f9c106620 2024-02-02T15:18:31.503 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:31.093+0000 7f5fa1c00700 1 -- 172.21.15.1:0/527174333 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f9c19aac0 con 0x7f5f9c106620 2024-02-02T15:18:31.503 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:31.094+0000 7f5f93400700 1 -- 172.21.15.1:0/527174333 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5f8c0247d0 con 0x7f5f9c106620 2024-02-02T15:18:31.503 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:31.094+0000 7f5fa1c00700 1 -- 172.21.15.1:0/527174333 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f9c19aee0 con 0x7f5f9c106620 2024-02-02T15:18:31.503 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:31.094+0000 7f5f93400700 1 -- 172.21.15.1:0/527174333 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f5f8c024d50 con 0x7f5f9c106620 2024-02-02T15:18:31.503 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:31.094+0000 7f5f93400700 1 -- 172.21.15.1:0/527174333 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f5f8c0193c0 con 0x7f5f9c106620 2024-02-02T15:18:31.503 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:31.095+0000 7f5fa1c00700 1 -- 172.21.15.1:0/527174333 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f9c062370 con 0x7f5f9c106620 2024-02-02T15:18:31.503 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:31.098+0000 7f5f93400700 1 -- 172.21.15.1:0/527174333 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72491 (secure 0 0 0) 0x7f5f8c01abb0 con 0x7f5f9c106620 2024-02-02T15:18:31.503 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:31.170+0000 7f5fa1c00700 1 -- 172.21.15.1:0/527174333 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f5f9c07a750 con 0x7f5f9c106620 2024-02-02T15:18:31.503 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:31.173+0000 7f5f93400700 1 -- 172.21.15.1:0/527174333 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f5f8c01ad50 con 0x7f5f9c106620 2024-02-02T15:18:31.503 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:31.174+0000 7f5fa1c00700 1 -- 172.21.15.1:0/527174333 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f9c106620 msgr2=0x7f5f9c19a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:18:31.504 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:31.174+0000 7f5fa1c00700 1 --2- 172.21.15.1:0/527174333 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f9c106620 0x7f5f9c19a380 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c009130 tx=0x7f5f8c011ad0 comp rx=0 tx=0).stop 2024-02-02T15:18:31.504 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:31.174+0000 7f5fa1c00700 1 -- 172.21.15.1:0/527174333 shutdown_connections 2024-02-02T15:18:31.504 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:31.174+0000 7f5fa1c00700 1 --2- 172.21.15.1:0/527174333 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f9c106620 0x7f5f9c19a380 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:18:31.504 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:31.174+0000 7f5fa1c00700 1 -- 172.21.15.1:0/527174333 >> 172.21.15.1:0/527174333 conn(0x7f5f9c101c20 msgr2=0x7f5f9c103310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:18:31.504 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:31.174+0000 7f5fa1c00700 1 -- 172.21.15.1:0/527174333 shutdown_connections 2024-02-02T15:18:31.504 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:31.174+0000 7f5fa1c00700 1 -- 172.21.15.1:0/527174333 wait complete. 2024-02-02T15:18:31.504 INFO:teuthology.orchestra.run.smithi001.stdout:Wrote config to /etc/ceph/ceph.conf 2024-02-02T15:18:31.523 INFO:teuthology.orchestra.run.smithi001.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-02-02T15:18:31.524 INFO:teuthology.orchestra.run.smithi001.stdout:Creating mgr... 2024-02-02T15:18:31.524 INFO:teuthology.orchestra.run.smithi001.stdout:Verifying port 0.0.0.0:9283 ... 2024-02-02T15:18:31.525 INFO:teuthology.orchestra.run.smithi001.stdout:Verifying port 0.0.0.0:8765 ... 2024-02-02T15:18:31.891 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-371a409c-c1de-11ee-95b6-87774f69a715@mgr.x 2024-02-02T15:18:31.891 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stderr Failed to reset failed state of unit ceph-371a409c-c1de-11ee-95b6-87774f69a715@mgr.x.service: Unit ceph-371a409c-c1de-11ee-95b6-87774f69a715@mgr.x.service not loaded. 2024-02-02T15:18:32.181 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-371a409c-c1de-11ee-95b6-87774f69a715.target.wants/ceph-371a409c-c1de-11ee-95b6-87774f69a715@mgr.x.service → /etc/systemd/system/ceph-371a409c-c1de-11ee-95b6-87774f69a715@.service. 2024-02-02T15:18:32.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:32 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/527174333' entity='client.admin' 2024-02-02T15:18:32.440 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:18:32 smithi001 systemd[1]: Starting Ceph mgr.x for 371a409c-c1de-11ee-95b6-87774f69a715... 2024-02-02T15:18:32.772 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-02T15:18:32.772 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout disabled 2024-02-02T15:18:32.790 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-02T15:18:32.790 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout inactive 2024-02-02T15:18:32.790 INFO:teuthology.orchestra.run.smithi001.stdout:firewalld.service is not enabled 2024-02-02T15:18:32.790 INFO:teuthology.orchestra.run.smithi001.stdout:Not possible to enable service . firewalld.service is not available 2024-02-02T15:18:32.811 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-02T15:18:32.811 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout disabled 2024-02-02T15:18:32.829 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-02T15:18:32.829 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout inactive 2024-02-02T15:18:32.829 INFO:teuthology.orchestra.run.smithi001.stdout:firewalld.service is not enabled 2024-02-02T15:18:32.830 INFO:teuthology.orchestra.run.smithi001.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-02-02T15:18:32.830 INFO:teuthology.orchestra.run.smithi001.stdout:Waiting for mgr to start... 2024-02-02T15:18:32.830 INFO:teuthology.orchestra.run.smithi001.stdout:Waiting for mgr... 2024-02-02T15:18:33.004 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:18:32 smithi001 bash[35707]: 5ead66761a218144447811c2b89b04d375e16a50b6908f98fa5d22bc77ab6c7f 2024-02-02T15:18:33.004 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:18:32 smithi001 systemd[1]: Started Ceph mgr.x for 371a409c-c1de-11ee-95b6-87774f69a715. 2024-02-02T15:18:33.688 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:33 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/3895833370' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-02T15:18:33.689 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:18:33 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:18:33.327+0000 7fdba6368200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-02T15:18:33.689 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:18:33 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:18:33.455+0000 7fdba6368200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-02T15:18:33.689 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:18:33 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:18:33.573+0000 7fdba6368200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-02T15:18:33.689 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:18:33 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:18:33.688+0000 7fdba6368200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-02T15:18:33.697 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2024-02-02T15:18:33.699 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout { 2024-02-02T15:18:33.699 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "fsid": "371a409c-c1de-11ee-95b6-87774f69a715", 2024-02-02T15:18:33.699 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "health": { 2024-02-02T15:18:33.699 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-02T15:18:33.699 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-02T15:18:33.700 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-02T15:18:33.700 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-02-02T15:18:33.700 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-02T15:18:33.700 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-02T15:18:33.700 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 0 2024-02-02T15:18:33.700 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout ], 2024-02-02T15:18:33.700 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-02T15:18:33.700 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "a" 2024-02-02T15:18:33.700 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout ], 2024-02-02T15:18:33.700 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-02-02T15:18:33.700 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-02T15:18:33.700 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-02T15:18:33.700 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-02-02T15:18:33.700 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-02T15:18:33.701 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-02-02T15:18:33.701 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-02T15:18:33.701 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-02T15:18:33.701 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-02T15:18:33.709 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-02T15:18:33.709 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-02T15:18:33.709 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-02T15:18:33.709 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-02T15:18:33.709 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-02T15:18:33.709 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-02-02T15:18:33.709 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-02T15:18:33.709 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-02T15:18:33.709 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-02T15:18:33.709 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-02T15:18:33.709 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-02T15:18:33.709 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-02T15:18:33.709 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-02T15:18:33.709 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-02T15:18:33.710 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-02T15:18:33.710 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-02-02T15:18:33.710 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-02T15:18:33.710 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-02T15:18:33.710 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-02T15:18:33.710 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-02T15:18:33.710 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-02-02T15:18:33.710 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-02T15:18:33.710 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-02T15:18:33.710 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-02T15:18:33.710 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-02T15:18:33.710 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-02T15:18:33.710 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-02T15:18:33.710 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "restful" 2024-02-02T15:18:33.710 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout ], 2024-02-02T15:18:33.710 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-02T15:18:33.711 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-02-02T15:18:33.711 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-02T15:18:33.711 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-02T15:18:33.711 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "modified": "2024-02-02T15:18:26.747079+0000", 2024-02-02T15:18:33.711 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-02T15:18:33.711 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-02-02T15:18:33.711 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-02T15:18:33.711 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout } 2024-02-02T15:18:33.711 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:33.269+0000 7f580b200700 1 Processor -- start 2024-02-02T15:18:33.711 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:33.269+0000 7f580b200700 1 -- start start 2024-02-02T15:18:33.711 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:33.269+0000 7f580b200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f580407bec0 0x7f580407a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:18:33.711 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:33.269+0000 7f580b200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f580407a8f0 con 0x7f580407bec0 2024-02-02T15:18:33.711 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:33.270+0000 7f5809e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f580407bec0 0x7f580407a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:18:33.711 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:33.270+0000 7f5809e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f580407bec0 0x7f580407a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35612/0 (socket says 172.21.15.1:35612) 2024-02-02T15:18:33.711 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:33.270+0000 7f5809e00700 1 -- 172.21.15.1:0/1608603086 learned_addr learned my addr 172.21.15.1:0/1608603086 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:18:33.712 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:33.270+0000 7f5809e00700 1 -- 172.21.15.1:0/1608603086 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f580407aa30 con 0x7f580407bec0 2024-02-02T15:18:33.712 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:33.270+0000 7f5809e00700 1 --2- 172.21.15.1:0/1608603086 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f580407bec0 0x7f580407a320 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f57f40098a0 tx=0x7f57f4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0249883e4a30826 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:18:33.712 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:33.270+0000 7f5808a00700 1 -- 172.21.15.1:0/1608603086 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f57f4004160 con 0x7f580407bec0 2024-02-02T15:18:33.712 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:33.271+0000 7f5808a00700 1 -- 172.21.15.1:0/1608603086 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f57f40042c0 con 0x7f580407bec0 2024-02-02T15:18:33.712 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:33.271+0000 7f5808a00700 1 -- 172.21.15.1:0/1608603086 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f57f4011a00 con 0x7f580407bec0 2024-02-02T15:18:33.712 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:33.271+0000 7f580b200700 1 -- 172.21.15.1:0/1608603086 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f580407bec0 msgr2=0x7f580407a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:18:33.712 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:33.271+0000 7f580b200700 1 --2- 172.21.15.1:0/1608603086 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f580407bec0 0x7f580407a320 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f57f40098a0 tx=0x7f57f4009bb0 comp rx=0 tx=0).stop 2024-02-02T15:18:33.712 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:33.271+0000 7f580b200700 1 -- 172.21.15.1:0/1608603086 shutdown_connections 2024-02-02T15:18:33.712 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:33.271+0000 7f580b200700 1 --2- 172.21.15.1:0/1608603086 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f580407bec0 0x7f580407a320 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:18:33.712 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:33.271+0000 7f580b200700 1 -- 172.21.15.1:0/1608603086 >> 172.21.15.1:0/1608603086 conn(0x7f58041019d0 msgr2=0x7f5804103df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:18:33.712 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:33.271+0000 7f580b200700 1 -- 172.21.15.1:0/1608603086 shutdown_connections 2024-02-02T15:18:33.712 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:33.271+0000 7f580b200700 1 -- 172.21.15.1:0/1608603086 wait complete. 2024-02-02T15:18:33.712 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:33.273+0000 7f580b200700 1 Processor -- start 2024-02-02T15:18:33.713 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:33.273+0000 7f580b200700 1 -- start start 2024-02-02T15:18:33.713 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:33.274+0000 7f580b200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f580407bec0 0x7f580419c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:18:33.713 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:33.274+0000 7f580b200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f580419c8d0 con 0x7f580407bec0 2024-02-02T15:18:33.713 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:33.274+0000 7f5809e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f580407bec0 0x7f580419c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:18:33.713 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:33.274+0000 7f5809e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f580407bec0 0x7f580419c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35614/0 (socket says 172.21.15.1:35614) 2024-02-02T15:18:33.713 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:33.274+0000 7f5809e00700 1 -- 172.21.15.1:0/3895833370 learned_addr learned my addr 172.21.15.1:0/3895833370 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:18:33.713 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:33.274+0000 7f5809e00700 1 -- 172.21.15.1:0/3895833370 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57f4009550 con 0x7f580407bec0 2024-02-02T15:18:33.713 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:33.274+0000 7f5809e00700 1 --2- 172.21.15.1:0/3895833370 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f580407bec0 0x7f580419c390 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f57f4006af0 tx=0x7f57f4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:18:33.713 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:33.275+0000 7f5802a00700 1 -- 172.21.15.1:0/3895833370 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f57f4003a40 con 0x7f580407bec0 2024-02-02T15:18:33.713 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:33.275+0000 7f5802a00700 1 -- 172.21.15.1:0/3895833370 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f57f4003ba0 con 0x7f580407bec0 2024-02-02T15:18:33.713 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:33.275+0000 7f580b200700 1 -- 172.21.15.1:0/3895833370 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f580419cad0 con 0x7f580407bec0 2024-02-02T15:18:33.713 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:33.275+0000 7f5802a00700 1 -- 172.21.15.1:0/3895833370 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f57f401a700 con 0x7f580407bec0 2024-02-02T15:18:33.713 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:33.275+0000 7f580b200700 1 -- 172.21.15.1:0/3895833370 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f580419cef0 con 0x7f580407bec0 2024-02-02T15:18:33.713 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:33.275+0000 7f5802a00700 1 -- 172.21.15.1:0/3895833370 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f57f401ac80 con 0x7f580407bec0 2024-02-02T15:18:33.713 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:33.275+0000 7f5802a00700 1 -- 172.21.15.1:0/3895833370 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f57f4019700 con 0x7f580407bec0 2024-02-02T15:18:33.714 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:33.276+0000 7f580b200700 1 -- 172.21.15.1:0/3895833370 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5804062370 con 0x7f580407bec0 2024-02-02T15:18:33.714 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:33.279+0000 7f5802a00700 1 -- 172.21.15.1:0/3895833370 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72491 (secure 0 0 0) 0x7f57f4024ad0 con 0x7f580407bec0 2024-02-02T15:18:33.714 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:33.355+0000 7f580b200700 1 -- 172.21.15.1:0/3895833370 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f580419fa20 con 0x7f580407bec0 2024-02-02T15:18:33.714 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:33.355+0000 7f5802a00700 1 -- 172.21.15.1:0/3895833370 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f57f4024c70 con 0x7f580407bec0 2024-02-02T15:18:33.714 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:33.356+0000 7f580b200700 1 -- 172.21.15.1:0/3895833370 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f580407bec0 msgr2=0x7f580419c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:18:33.714 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:33.356+0000 7f580b200700 1 --2- 172.21.15.1:0/3895833370 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f580407bec0 0x7f580419c390 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f57f4006af0 tx=0x7f57f4011ad0 comp rx=0 tx=0).stop 2024-02-02T15:18:33.714 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:33.356+0000 7f580b200700 1 -- 172.21.15.1:0/3895833370 shutdown_connections 2024-02-02T15:18:33.714 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:33.356+0000 7f580b200700 1 --2- 172.21.15.1:0/3895833370 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f580407bec0 0x7f580419c390 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:18:33.714 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:33.356+0000 7f580b200700 1 -- 172.21.15.1:0/3895833370 >> 172.21.15.1:0/3895833370 conn(0x7f58041019d0 msgr2=0x7f58041030d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:18:33.714 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:33.357+0000 7f580b200700 1 -- 172.21.15.1:0/3895833370 shutdown_connections 2024-02-02T15:18:33.714 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:33.357+0000 7f580b200700 1 -- 172.21.15.1:0/3895833370 wait complete. 2024-02-02T15:18:33.714 INFO:teuthology.orchestra.run.smithi001.stdout:mgr not available, waiting (1/15)... 2024-02-02T15:18:34.863 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:18:34 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:18:34.560+0000 7fdba6368200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-02T15:18:35.120 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:18:34 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:18:34.862+0000 7fdba6368200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-02T15:18:35.120 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:18:34 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:18:34.977+0000 7fdba6368200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-02T15:18:35.440 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:18:35 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:18:35.119+0000 7fdba6368200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-02T15:18:35.440 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:18:35 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:18:35.320+0000 7fdba6368200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-02T15:18:35.847 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:18:35 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:18:35.496+0000 7fdba6368200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-02T15:18:35.847 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:18:35 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:18:35.611+0000 7fdba6368200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-02T15:18:36.419 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:18:36 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:18:36.115+0000 7fdba6368200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-02T15:18:36.554 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2024-02-02T15:18:36.554 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout { 2024-02-02T15:18:36.554 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "fsid": "371a409c-c1de-11ee-95b6-87774f69a715", 2024-02-02T15:18:36.554 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "health": { 2024-02-02T15:18:36.554 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-02T15:18:36.554 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-02T15:18:36.554 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-02T15:18:36.554 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-02-02T15:18:36.554 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-02T15:18:36.554 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-02T15:18:36.555 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 0 2024-02-02T15:18:36.555 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout ], 2024-02-02T15:18:36.555 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-02T15:18:36.555 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "a" 2024-02-02T15:18:36.555 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout ], 2024-02-02T15:18:36.555 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-02-02T15:18:36.555 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-02T15:18:36.555 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-02T15:18:36.555 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-02-02T15:18:36.555 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-02T15:18:36.555 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-02-02T15:18:36.555 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-02T15:18:36.556 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-02T15:18:36.556 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-02T15:18:36.556 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-02T15:18:36.564 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-02T15:18:36.564 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-02T15:18:36.564 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-02T15:18:36.565 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-02T15:18:36.565 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-02-02T15:18:36.565 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-02T15:18:36.565 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-02T15:18:36.565 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-02T15:18:36.565 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-02T15:18:36.565 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-02T15:18:36.565 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-02T15:18:36.565 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-02T15:18:36.565 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-02T15:18:36.565 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-02T15:18:36.565 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-02-02T15:18:36.566 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-02T15:18:36.566 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-02T15:18:36.566 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-02T15:18:36.566 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-02T15:18:36.566 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-02-02T15:18:36.566 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-02T15:18:36.566 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-02T15:18:36.566 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-02T15:18:36.566 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-02T15:18:36.566 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-02T15:18:36.566 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-02T15:18:36.566 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "restful" 2024-02-02T15:18:36.567 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout ], 2024-02-02T15:18:36.567 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-02T15:18:36.567 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-02-02T15:18:36.567 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-02T15:18:36.567 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-02T15:18:36.567 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "modified": "2024-02-02T15:18:26.747079+0000", 2024-02-02T15:18:36.567 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-02T15:18:36.567 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-02-02T15:18:36.567 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-02T15:18:36.567 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout } 2024-02-02T15:18:36.567 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:36.124+0000 7fea52600700 1 Processor -- start 2024-02-02T15:18:36.567 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:36.124+0000 7fea52600700 1 -- start start 2024-02-02T15:18:36.567 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:36.124+0000 7fea52600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fea4c1070f0 0x7fea4c1094e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:18:36.568 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:36.124+0000 7fea52600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea4c079e10 con 0x7fea4c1070f0 2024-02-02T15:18:36.568 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:36.125+0000 7fea4b400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fea4c1070f0 0x7fea4c1094e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:18:36.568 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:36.125+0000 7fea4b400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fea4c1070f0 0x7fea4c1094e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35626/0 (socket says 172.21.15.1:35626) 2024-02-02T15:18:36.568 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:36.125+0000 7fea4b400700 1 -- 172.21.15.1:0/4219680733 learned_addr learned my addr 172.21.15.1:0/4219680733 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:18:36.568 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:36.125+0000 7fea4b400700 1 -- 172.21.15.1:0/4219680733 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea4c109a20 con 0x7fea4c1070f0 2024-02-02T15:18:36.568 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:36.125+0000 7fea4b400700 1 --2- 172.21.15.1:0/4219680733 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fea4c1070f0 0x7fea4c1094e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fea3c0098a0 tx=0x7fea3c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bb9d3743aed3246 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:18:36.568 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:36.126+0000 7fea4a000700 1 -- 172.21.15.1:0/4219680733 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fea3c004160 con 0x7fea4c1070f0 2024-02-02T15:18:36.568 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:36.126+0000 7fea4a000700 1 -- 172.21.15.1:0/4219680733 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fea3c0042c0 con 0x7fea4c1070f0 2024-02-02T15:18:36.568 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:36.126+0000 7fea4a000700 1 -- 172.21.15.1:0/4219680733 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fea3c011a00 con 0x7fea4c1070f0 2024-02-02T15:18:36.568 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:36.126+0000 7fea52600700 1 -- 172.21.15.1:0/4219680733 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fea4c1070f0 msgr2=0x7fea4c1094e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:18:36.568 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:36.126+0000 7fea52600700 1 --2- 172.21.15.1:0/4219680733 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fea4c1070f0 0x7fea4c1094e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fea3c0098a0 tx=0x7fea3c009bb0 comp rx=0 tx=0).stop 2024-02-02T15:18:36.569 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:36.127+0000 7fea52600700 1 -- 172.21.15.1:0/4219680733 shutdown_connections 2024-02-02T15:18:36.569 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:36.127+0000 7fea52600700 1 --2- 172.21.15.1:0/4219680733 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fea4c1070f0 0x7fea4c1094e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:18:36.569 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:36.127+0000 7fea52600700 1 -- 172.21.15.1:0/4219680733 >> 172.21.15.1:0/4219680733 conn(0x7fea4c100ac0 msgr2=0x7fea4c102ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:18:36.569 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:36.127+0000 7fea52600700 1 -- 172.21.15.1:0/4219680733 shutdown_connections 2024-02-02T15:18:36.569 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:36.127+0000 7fea52600700 1 -- 172.21.15.1:0/4219680733 wait complete. 2024-02-02T15:18:36.569 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:36.128+0000 7fea52600700 1 Processor -- start 2024-02-02T15:18:36.569 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:36.128+0000 7fea52600700 1 -- start start 2024-02-02T15:18:36.569 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:36.128+0000 7fea52600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fea4c1070f0 0x7fea4c106a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:18:36.569 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:36.128+0000 7fea52600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea4c106f90 con 0x7fea4c1070f0 2024-02-02T15:18:36.569 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:36.128+0000 7fea4b400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fea4c1070f0 0x7fea4c106a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:18:36.569 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:36.129+0000 7fea4b400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fea4c1070f0 0x7fea4c106a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35628/0 (socket says 172.21.15.1:35628) 2024-02-02T15:18:36.570 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:36.129+0000 7fea4b400700 1 -- 172.21.15.1:0/3761049422 learned_addr learned my addr 172.21.15.1:0/3761049422 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:18:36.570 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:36.129+0000 7fea4b400700 1 -- 172.21.15.1:0/3761049422 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea3c009550 con 0x7fea4c1070f0 2024-02-02T15:18:36.570 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:36.129+0000 7fea4b400700 1 --2- 172.21.15.1:0/3761049422 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fea4c1070f0 0x7fea4c106a50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fea3c009130 tx=0x7fea3c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:18:36.570 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:36.129+0000 7fea43e00700 1 -- 172.21.15.1:0/3761049422 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fea3c003b70 con 0x7fea4c1070f0 2024-02-02T15:18:36.570 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:36.129+0000 7fea43e00700 1 -- 172.21.15.1:0/3761049422 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fea3c003cd0 con 0x7fea4c1070f0 2024-02-02T15:18:36.570 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:36.129+0000 7fea52600700 1 -- 172.21.15.1:0/3761049422 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea4c1050a0 con 0x7fea4c1070f0 2024-02-02T15:18:36.570 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:36.129+0000 7fea43e00700 1 -- 172.21.15.1:0/3761049422 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fea3c0197d0 con 0x7fea4c1070f0 2024-02-02T15:18:36.570 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:36.130+0000 7fea52600700 1 -- 172.21.15.1:0/3761049422 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea4c1054c0 con 0x7fea4c1070f0 2024-02-02T15:18:36.570 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:36.130+0000 7fea43e00700 1 -- 172.21.15.1:0/3761049422 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fea3c0199c0 con 0x7fea4c1070f0 2024-02-02T15:18:36.570 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:36.130+0000 7fea43e00700 1 -- 172.21.15.1:0/3761049422 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fea3c00b720 con 0x7fea4c1070f0 2024-02-02T15:18:36.571 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:36.131+0000 7fea52600700 1 -- 172.21.15.1:0/3761049422 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea4c062370 con 0x7fea4c1070f0 2024-02-02T15:18:36.571 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:36.134+0000 7fea43e00700 1 -- 172.21.15.1:0/3761049422 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72491 (secure 0 0 0) 0x7fea3c02a060 con 0x7fea4c1070f0 2024-02-02T15:18:36.571 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:36.223+0000 7fea52600700 1 -- 172.21.15.1:0/3761049422 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fea4c194380 con 0x7fea4c1070f0 2024-02-02T15:18:36.571 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:36.223+0000 7fea43e00700 1 -- 172.21.15.1:0/3761049422 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7fea3c019db0 con 0x7fea4c1070f0 2024-02-02T15:18:36.571 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:36.225+0000 7fea52600700 1 -- 172.21.15.1:0/3761049422 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fea4c1070f0 msgr2=0x7fea4c106a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:18:36.571 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:36.225+0000 7fea52600700 1 --2- 172.21.15.1:0/3761049422 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fea4c1070f0 0x7fea4c106a50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fea3c009130 tx=0x7fea3c011ad0 comp rx=0 tx=0).stop 2024-02-02T15:18:36.571 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:36.225+0000 7fea52600700 1 -- 172.21.15.1:0/3761049422 shutdown_connections 2024-02-02T15:18:36.571 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:36.225+0000 7fea52600700 1 --2- 172.21.15.1:0/3761049422 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fea4c1070f0 0x7fea4c106a50 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:18:36.571 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:36.225+0000 7fea52600700 1 -- 172.21.15.1:0/3761049422 >> 172.21.15.1:0/3761049422 conn(0x7fea4c100ac0 msgr2=0x7fea4c102240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:18:36.571 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:36.225+0000 7fea52600700 1 -- 172.21.15.1:0/3761049422 shutdown_connections 2024-02-02T15:18:36.571 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:36.225+0000 7fea52600700 1 -- 172.21.15.1:0/3761049422 wait complete. 2024-02-02T15:18:36.571 INFO:teuthology.orchestra.run.smithi001.stdout:mgr not available, waiting (2/15)... 2024-02-02T15:18:36.689 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:18:36 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:18:36.502+0000 7fdba6368200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-02T15:18:36.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:36 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/3761049422' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-02T15:18:36.940 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:18:36 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:18:36.866+0000 7fdba6368200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-02T15:18:37.440 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:18:36 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:18:36.994+0000 7fdba6368200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-02T15:18:38.303 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:18:37 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:18:37.977+0000 7fdba6368200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-02T15:18:38.304 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:18:38 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:18:38.105+0000 7fdba6368200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-02T15:18:38.686 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:18:38 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:18:38.303+0000 7fdba6368200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-02T15:18:39.368 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:18:39 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:18:39.178+0000 7fdba6368200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-02T15:18:39.368 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:39 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/3679766352' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-02T15:18:39.373 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2024-02-02T15:18:39.373 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout { 2024-02-02T15:18:39.373 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "fsid": "371a409c-c1de-11ee-95b6-87774f69a715", 2024-02-02T15:18:39.373 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "health": { 2024-02-02T15:18:39.373 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-02T15:18:39.374 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-02T15:18:39.374 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-02T15:18:39.374 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-02-02T15:18:39.374 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-02T15:18:39.374 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-02T15:18:39.374 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 0 2024-02-02T15:18:39.374 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout ], 2024-02-02T15:18:39.374 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-02T15:18:39.374 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "a" 2024-02-02T15:18:39.374 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout ], 2024-02-02T15:18:39.374 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-02-02T15:18:39.374 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-02T15:18:39.374 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-02T15:18:39.375 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-02-02T15:18:39.375 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-02T15:18:39.375 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-02-02T15:18:39.375 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-02T15:18:39.375 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-02T15:18:39.375 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-02T15:18:39.383 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-02T15:18:39.383 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-02T15:18:39.383 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-02T15:18:39.383 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-02T15:18:39.383 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-02T15:18:39.383 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-02-02T15:18:39.383 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-02T15:18:39.383 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-02T15:18:39.383 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-02T15:18:39.383 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-02T15:18:39.383 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-02T15:18:39.383 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-02T15:18:39.384 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-02T15:18:39.384 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-02T15:18:39.384 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-02T15:18:39.384 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-02-02T15:18:39.384 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-02T15:18:39.384 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-02T15:18:39.384 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-02T15:18:39.384 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-02T15:18:39.384 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-02-02T15:18:39.384 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-02T15:18:39.384 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-02T15:18:39.384 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-02T15:18:39.384 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-02T15:18:39.385 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-02T15:18:39.385 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-02T15:18:39.385 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "restful" 2024-02-02T15:18:39.385 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout ], 2024-02-02T15:18:39.385 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-02T15:18:39.385 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-02-02T15:18:39.385 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-02T15:18:39.385 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-02T15:18:39.385 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "modified": "2024-02-02T15:18:26.747079+0000", 2024-02-02T15:18:39.385 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-02T15:18:39.385 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-02-02T15:18:39.385 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-02T15:18:39.386 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout } 2024-02-02T15:18:39.386 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:38.954+0000 7f7bdc600700 1 Processor -- start 2024-02-02T15:18:39.386 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:38.954+0000 7f7bdc600700 1 -- start start 2024-02-02T15:18:39.386 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:38.954+0000 7f7bdc600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7bd407be60 0x7f7bd407a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:18:39.386 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:38.954+0000 7f7bdc600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd407a890 con 0x7f7bd407be60 2024-02-02T15:18:39.386 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:38.955+0000 7f7bd9c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7bd407be60 0x7f7bd407a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:18:39.386 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:38.955+0000 7f7bd9c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7bd407be60 0x7f7bd407a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35636/0 (socket says 172.21.15.1:35636) 2024-02-02T15:18:39.386 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:38.955+0000 7f7bd9c00700 1 -- 172.21.15.1:0/2716388233 learned_addr learned my addr 172.21.15.1:0/2716388233 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:18:39.386 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:38.955+0000 7f7bd9c00700 1 -- 172.21.15.1:0/2716388233 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bd407a9d0 con 0x7f7bd407be60 2024-02-02T15:18:39.386 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:38.955+0000 7f7bd9c00700 1 --2- 172.21.15.1:0/2716388233 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7bd407be60 0x7f7bd407a2c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7bc40098a0 tx=0x7f7bc4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ebb0e649a713ff7 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:18:39.386 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:38.956+0000 7f7bd3e00700 1 -- 172.21.15.1:0/2716388233 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7bc4004160 con 0x7f7bd407be60 2024-02-02T15:18:39.386 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:38.956+0000 7f7bd3e00700 1 -- 172.21.15.1:0/2716388233 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7bc40042c0 con 0x7f7bd407be60 2024-02-02T15:18:39.386 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:38.956+0000 7f7bd3e00700 1 -- 172.21.15.1:0/2716388233 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7bc4011a00 con 0x7f7bd407be60 2024-02-02T15:18:39.386 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:38.956+0000 7f7bdc600700 1 -- 172.21.15.1:0/2716388233 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7bd407be60 msgr2=0x7f7bd407a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:18:39.387 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:38.956+0000 7f7bdc600700 1 --2- 172.21.15.1:0/2716388233 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7bd407be60 0x7f7bd407a2c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7bc40098a0 tx=0x7f7bc4009bb0 comp rx=0 tx=0).stop 2024-02-02T15:18:39.387 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:38.956+0000 7f7bdc600700 1 -- 172.21.15.1:0/2716388233 shutdown_connections 2024-02-02T15:18:39.387 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:38.957+0000 7f7bdc600700 1 --2- 172.21.15.1:0/2716388233 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7bd407be60 0x7f7bd407a2c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:18:39.387 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:38.957+0000 7f7bdc600700 1 -- 172.21.15.1:0/2716388233 >> 172.21.15.1:0/2716388233 conn(0x7f7bd4101c20 msgr2=0x7f7bd4104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:18:39.387 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:38.957+0000 7f7bdc600700 1 -- 172.21.15.1:0/2716388233 shutdown_connections 2024-02-02T15:18:39.387 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:38.957+0000 7f7bdc600700 1 -- 172.21.15.1:0/2716388233 wait complete. 2024-02-02T15:18:39.387 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:38.958+0000 7f7bdc600700 1 Processor -- start 2024-02-02T15:18:39.387 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:38.958+0000 7f7bdc600700 1 -- start start 2024-02-02T15:18:39.387 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:38.959+0000 7f7bdc600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7bd407be60 0x7f7bd419c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:18:39.387 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:38.959+0000 7f7bdc600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd419ca80 con 0x7f7bd407be60 2024-02-02T15:18:39.387 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:38.959+0000 7f7bd9c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7bd407be60 0x7f7bd419c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:18:39.387 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:38.959+0000 7f7bd9c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7bd407be60 0x7f7bd419c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35646/0 (socket says 172.21.15.1:35646) 2024-02-02T15:18:39.387 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:38.959+0000 7f7bd9c00700 1 -- 172.21.15.1:0/3679766352 learned_addr learned my addr 172.21.15.1:0/3679766352 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:18:39.387 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:38.960+0000 7f7bd9c00700 1 -- 172.21.15.1:0/3679766352 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bc4009550 con 0x7f7bd407be60 2024-02-02T15:18:39.388 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:38.960+0000 7f7bd9c00700 1 --2- 172.21.15.1:0/3679766352 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7bd407be60 0x7f7bd419c540 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7bc4009130 tx=0x7f7bc4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:18:39.388 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:38.960+0000 7f7bd2000700 1 -- 172.21.15.1:0/3679766352 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7bc4003b70 con 0x7f7bd407be60 2024-02-02T15:18:39.388 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:38.960+0000 7f7bd2000700 1 -- 172.21.15.1:0/3679766352 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f7bc4003cd0 con 0x7f7bd407be60 2024-02-02T15:18:39.388 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:38.960+0000 7f7bdc600700 1 -- 172.21.15.1:0/3679766352 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bd419cc80 con 0x7f7bd407be60 2024-02-02T15:18:39.388 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:38.960+0000 7f7bd2000700 1 -- 172.21.15.1:0/3679766352 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7bc40197d0 con 0x7f7bd407be60 2024-02-02T15:18:39.388 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:38.960+0000 7f7bdc600700 1 -- 172.21.15.1:0/3679766352 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bd419d0a0 con 0x7f7bd407be60 2024-02-02T15:18:39.388 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:38.961+0000 7f7bd2000700 1 -- 172.21.15.1:0/3679766352 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f7bc40199c0 con 0x7f7bd407be60 2024-02-02T15:18:39.388 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:38.961+0000 7f7bd2000700 1 -- 172.21.15.1:0/3679766352 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f7bc400b720 con 0x7f7bd407be60 2024-02-02T15:18:39.388 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:38.962+0000 7f7bdc600700 1 -- 172.21.15.1:0/3679766352 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bd410ca10 con 0x7f7bd407be60 2024-02-02T15:18:39.388 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:38.964+0000 7f7bd2000700 1 -- 172.21.15.1:0/3679766352 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72491 (secure 0 0 0) 0x7f7bc402a060 con 0x7f7bd407be60 2024-02-02T15:18:39.388 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:39.040+0000 7f7bdc600700 1 -- 172.21.15.1:0/3679766352 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f7bd41a00f0 con 0x7f7bd407be60 2024-02-02T15:18:39.388 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:39.041+0000 7f7bd2000700 1 -- 172.21.15.1:0/3679766352 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f7bc4019db0 con 0x7f7bd407be60 2024-02-02T15:18:39.389 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:39.042+0000 7f7bdc600700 1 -- 172.21.15.1:0/3679766352 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7bd407be60 msgr2=0x7f7bd419c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:18:39.389 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:39.042+0000 7f7bdc600700 1 --2- 172.21.15.1:0/3679766352 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7bd407be60 0x7f7bd419c540 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7bc4009130 tx=0x7f7bc4011ad0 comp rx=0 tx=0).stop 2024-02-02T15:18:39.389 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:39.042+0000 7f7bdc600700 1 -- 172.21.15.1:0/3679766352 shutdown_connections 2024-02-02T15:18:39.389 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:39.042+0000 7f7bdc600700 1 --2- 172.21.15.1:0/3679766352 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7bd407be60 0x7f7bd419c540 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:18:39.389 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:39.042+0000 7f7bdc600700 1 -- 172.21.15.1:0/3679766352 >> 172.21.15.1:0/3679766352 conn(0x7f7bd4101c20 msgr2=0x7f7bd4103340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:18:39.389 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:39.042+0000 7f7bdc600700 1 -- 172.21.15.1:0/3679766352 shutdown_connections 2024-02-02T15:18:39.389 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:39.042+0000 7f7bdc600700 1 -- 172.21.15.1:0/3679766352 wait complete. 2024-02-02T15:18:39.389 INFO:teuthology.orchestra.run.smithi001.stdout:mgr not available, waiting (3/15)... 2024-02-02T15:18:39.690 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:18:39 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:18:39.429+0000 7fdba6368200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-02T15:18:40.190 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:18:39 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:18:39.692+0000 7fdba6368200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-02T15:18:41.190 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:18:40 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:18:40.862+0000 7fdba6368200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-02T15:18:41.552 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:18:41 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:18:41.223+0000 7fdba6368200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-02T15:18:41.553 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:18:41 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:18:41.341+0000 7fdba6368200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-02T15:18:41.935 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:18:41 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:18:41.633+0000 7fdba6368200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-02T15:18:41.936 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:41 smithi001 ceph-mon[35470]: Activating manager daemon x 2024-02-02T15:18:41.936 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:41 smithi001 ceph-mon[35470]: mgrmap e2: x(active, starting, since 0.00468625s) 2024-02-02T15:18:41.936 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:41 smithi001 ceph-mon[35470]: from='mgr.14100 172.21.15.1:0/610728247' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-02T15:18:41.936 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:41 smithi001 ceph-mon[35470]: from='mgr.14100 172.21.15.1:0/610728247' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-02T15:18:41.936 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:41 smithi001 ceph-mon[35470]: from='mgr.14100 172.21.15.1:0/610728247' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-02T15:18:41.936 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:41 smithi001 ceph-mon[35470]: from='mgr.14100 172.21.15.1:0/610728247' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-02T15:18:41.936 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:41 smithi001 ceph-mon[35470]: from='mgr.14100 172.21.15.1:0/610728247' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-02T15:18:41.936 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:41 smithi001 ceph-mon[35470]: Manager daemon x is now available 2024-02-02T15:18:41.936 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:41 smithi001 ceph-mon[35470]: from='mgr.14100 172.21.15.1:0/610728247' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-02-02T15:18:41.936 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:41 smithi001 ceph-mon[35470]: from='mgr.14100 172.21.15.1:0/610728247' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-02-02T15:18:41.937 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:41 smithi001 ceph-mon[35470]: from='mgr.14100 172.21.15.1:0/610728247' entity='mgr.x' 2024-02-02T15:18:41.937 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:41 smithi001 ceph-mon[35470]: from='mgr.14100 172.21.15.1:0/610728247' entity='mgr.x' 2024-02-02T15:18:41.937 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:41 smithi001 ceph-mon[35470]: from='mgr.14100 172.21.15.1:0/610728247' entity='mgr.x' 2024-02-02T15:18:42.252 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2024-02-02T15:18:42.252 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout { 2024-02-02T15:18:42.252 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "fsid": "371a409c-c1de-11ee-95b6-87774f69a715", 2024-02-02T15:18:42.253 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "health": { 2024-02-02T15:18:42.253 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-02T15:18:42.253 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-02T15:18:42.253 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-02T15:18:42.253 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-02-02T15:18:42.253 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-02T15:18:42.253 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-02T15:18:42.253 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 0 2024-02-02T15:18:42.253 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout ], 2024-02-02T15:18:42.253 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-02T15:18:42.253 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "a" 2024-02-02T15:18:42.253 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout ], 2024-02-02T15:18:42.253 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "quorum_age": 11, 2024-02-02T15:18:42.253 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-02T15:18:42.253 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-02T15:18:42.254 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-02-02T15:18:42.254 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-02T15:18:42.254 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-02-02T15:18:42.254 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-02T15:18:42.254 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-02T15:18:42.254 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-02T15:18:42.254 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-02T15:18:42.254 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-02T15:18:42.263 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-02T15:18:42.263 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-02T15:18:42.263 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-02T15:18:42.263 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-02-02T15:18:42.263 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-02T15:18:42.263 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-02T15:18:42.263 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-02T15:18:42.263 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-02T15:18:42.263 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-02T15:18:42.263 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-02T15:18:42.264 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-02T15:18:42.264 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-02T15:18:42.264 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-02T15:18:42.264 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-02-02T15:18:42.264 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-02T15:18:42.264 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-02T15:18:42.264 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-02T15:18:42.264 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-02T15:18:42.264 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-02-02T15:18:42.264 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-02T15:18:42.264 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-02T15:18:42.264 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-02T15:18:42.264 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-02T15:18:42.264 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-02T15:18:42.265 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-02T15:18:42.265 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "restful" 2024-02-02T15:18:42.265 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout ], 2024-02-02T15:18:42.265 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-02T15:18:42.265 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-02-02T15:18:42.265 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-02T15:18:42.265 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-02T15:18:42.265 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "modified": "2024-02-02T15:18:26.747079+0000", 2024-02-02T15:18:42.265 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-02T15:18:42.265 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-02-02T15:18:42.265 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-02T15:18:42.265 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout } 2024-02-02T15:18:42.265 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:41.830+0000 7fb415000700 1 Processor -- start 2024-02-02T15:18:42.265 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:41.831+0000 7fb415000700 1 -- start start 2024-02-02T15:18:42.265 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:41.831+0000 7fb415000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb410106620 0x7fb410106a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:18:42.266 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:41.831+0000 7fb415000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb410107010 con 0x7fb410106620 2024-02-02T15:18:42.266 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:41.831+0000 7fb40de00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb410106620 0x7fb410106a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:18:42.266 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:41.831+0000 7fb40de00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb410106620 0x7fb410106a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35724/0 (socket says 172.21.15.1:35724) 2024-02-02T15:18:42.266 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:41.831+0000 7fb40de00700 1 -- 172.21.15.1:0/3928468302 learned_addr learned my addr 172.21.15.1:0/3928468302 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:18:42.266 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:41.832+0000 7fb40de00700 1 -- 172.21.15.1:0/3928468302 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb410107820 con 0x7fb410106620 2024-02-02T15:18:42.266 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:41.832+0000 7fb40de00700 1 --2- 172.21.15.1:0/3928468302 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb410106620 0x7fb410106a40 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb4000098a0 tx=0x7fb400009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80976ea12964acfc server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:18:42.266 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:41.832+0000 7fb40ca00700 1 -- 172.21.15.1:0/3928468302 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb400004160 con 0x7fb410106620 2024-02-02T15:18:42.266 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:41.832+0000 7fb40ca00700 1 -- 172.21.15.1:0/3928468302 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb4000042c0 con 0x7fb410106620 2024-02-02T15:18:42.266 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:41.832+0000 7fb40ca00700 1 -- 172.21.15.1:0/3928468302 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb400011a00 con 0x7fb410106620 2024-02-02T15:18:42.266 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:41.833+0000 7fb415000700 1 -- 172.21.15.1:0/3928468302 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb410106620 msgr2=0x7fb410106a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:18:42.266 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:41.833+0000 7fb415000700 1 --2- 172.21.15.1:0/3928468302 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb410106620 0x7fb410106a40 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb4000098a0 tx=0x7fb400009bb0 comp rx=0 tx=0).stop 2024-02-02T15:18:42.266 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:41.833+0000 7fb415000700 1 -- 172.21.15.1:0/3928468302 shutdown_connections 2024-02-02T15:18:42.266 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:41.833+0000 7fb415000700 1 --2- 172.21.15.1:0/3928468302 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb410106620 0x7fb410106a40 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:18:42.266 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:41.833+0000 7fb415000700 1 -- 172.21.15.1:0/3928468302 >> 172.21.15.1:0/3928468302 conn(0x7fb410101c20 msgr2=0x7fb410104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:18:42.267 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:41.833+0000 7fb415000700 1 -- 172.21.15.1:0/3928468302 shutdown_connections 2024-02-02T15:18:42.267 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:41.833+0000 7fb415000700 1 -- 172.21.15.1:0/3928468302 wait complete. 2024-02-02T15:18:42.267 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:41.834+0000 7fb415000700 1 Processor -- start 2024-02-02T15:18:42.267 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:41.834+0000 7fb415000700 1 -- start start 2024-02-02T15:18:42.267 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:41.834+0000 7fb415000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb410106620 0x7fb41007bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:18:42.267 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:41.834+0000 7fb415000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb41007c170 con 0x7fb410106620 2024-02-02T15:18:42.267 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:41.835+0000 7fb40de00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb410106620 0x7fb41007bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:18:42.267 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:41.835+0000 7fb40de00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb410106620 0x7fb41007bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35736/0 (socket says 172.21.15.1:35736) 2024-02-02T15:18:42.267 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:41.835+0000 7fb40de00700 1 -- 172.21.15.1:0/3718593987 learned_addr learned my addr 172.21.15.1:0/3718593987 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:18:42.267 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:41.835+0000 7fb40de00700 1 -- 172.21.15.1:0/3718593987 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb400009550 con 0x7fb410106620 2024-02-02T15:18:42.267 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:41.835+0000 7fb40de00700 1 --2- 172.21.15.1:0/3718593987 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb410106620 0x7fb41007bc30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb400009130 tx=0x7fb400011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:18:42.267 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:41.835+0000 7fb406a00700 1 -- 172.21.15.1:0/3718593987 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb400003ab0 con 0x7fb410106620 2024-02-02T15:18:42.267 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:41.835+0000 7fb406a00700 1 -- 172.21.15.1:0/3718593987 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb400003c10 con 0x7fb410106620 2024-02-02T15:18:42.267 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:41.835+0000 7fb406a00700 1 -- 172.21.15.1:0/3718593987 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb4000247d0 con 0x7fb410106620 2024-02-02T15:18:42.268 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:41.835+0000 7fb415000700 1 -- 172.21.15.1:0/3718593987 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb41007a280 con 0x7fb410106620 2024-02-02T15:18:42.268 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:41.836+0000 7fb415000700 1 -- 172.21.15.1:0/3718593987 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb41007a6a0 con 0x7fb410106620 2024-02-02T15:18:42.268 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:41.837+0000 7fb406a00700 1 -- 172.21.15.1:0/3718593987 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 2) v1 ==== 45024+0+0 (secure 0 0 0) 0x7fb40001abb0 con 0x7fb410106620 2024-02-02T15:18:42.268 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:41.836+0000 7fb415000700 1 -- 172.21.15.1:0/3718593987 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb410062370 con 0x7fb410106620 2024-02-02T15:18:42.268 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:41.837+0000 7fb406a00700 1 -- 172.21.15.1:0/3718593987 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb40004bb10 con 0x7fb410106620 2024-02-02T15:18:42.268 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:41.839+0000 7fb406a00700 1 -- 172.21.15.1:0/3718593987 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72491 (secure 0 0 0) 0x7fb400018710 con 0x7fb410106620 2024-02-02T15:18:42.268 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:41.914+0000 7fb415000700 1 -- 172.21.15.1:0/3718593987 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fb41010aa50 con 0x7fb410106620 2024-02-02T15:18:42.268 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:41.914+0000 7fb406a00700 1 -- 172.21.15.1:0/3718593987 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7fb400018350 con 0x7fb410106620 2024-02-02T15:18:42.268 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:41.915+0000 7fb415000700 1 -- 172.21.15.1:0/3718593987 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb410106620 msgr2=0x7fb41007bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:18:42.268 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:41.915+0000 7fb415000700 1 --2- 172.21.15.1:0/3718593987 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb410106620 0x7fb41007bc30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb400009130 tx=0x7fb400011ad0 comp rx=0 tx=0).stop 2024-02-02T15:18:42.268 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:41.915+0000 7fb415000700 1 -- 172.21.15.1:0/3718593987 shutdown_connections 2024-02-02T15:18:42.268 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:41.915+0000 7fb415000700 1 --2- 172.21.15.1:0/3718593987 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb410106620 0x7fb41007bc30 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:18:42.268 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:41.915+0000 7fb415000700 1 -- 172.21.15.1:0/3718593987 >> 172.21.15.1:0/3718593987 conn(0x7fb410101c20 msgr2=0x7fb410103310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:18:42.268 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:41.915+0000 7fb415000700 1 -- 172.21.15.1:0/3718593987 shutdown_connections 2024-02-02T15:18:42.269 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:41.915+0000 7fb415000700 1 -- 172.21.15.1:0/3718593987 wait complete. 2024-02-02T15:18:42.269 INFO:teuthology.orchestra.run.smithi001.stdout:mgr not available, waiting (4/15)... 2024-02-02T15:18:43.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:42 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/3718593987' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-02T15:18:43.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:42 smithi001 ceph-mon[35470]: mgrmap e3: x(active, since 1.00888s) 2024-02-02T15:18:45.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:44 smithi001 ceph-mon[35470]: mgrmap e4: x(active, since 2s) 2024-02-02T15:18:48.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:47 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/1335269324' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-02T15:18:50.983 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2024-02-02T15:18:50.983 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout { 2024-02-02T15:18:50.983 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "fsid": "371a409c-c1de-11ee-95b6-87774f69a715", 2024-02-02T15:18:50.983 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "health": { 2024-02-02T15:18:50.983 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-02T15:18:50.984 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-02T15:18:50.984 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-02T15:18:50.984 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-02-02T15:18:50.984 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-02T15:18:50.984 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-02T15:18:50.984 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 0 2024-02-02T15:18:50.984 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout ], 2024-02-02T15:18:50.984 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-02T15:18:50.984 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "a" 2024-02-02T15:18:50.984 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout ], 2024-02-02T15:18:50.984 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "quorum_age": 17, 2024-02-02T15:18:50.984 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-02T15:18:50.984 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-02T15:18:50.984 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-02-02T15:18:50.985 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-02T15:18:50.985 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-02-02T15:18:50.985 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-02T15:18:50.985 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-02T15:18:50.985 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-02T15:18:50.985 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-02T15:18:50.985 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-02T15:18:50.985 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-02T15:18:50.986 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-02T15:18:50.986 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-02T15:18:50.986 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-02-02T15:18:50.986 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-02T15:18:50.986 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-02T15:18:50.987 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-02T15:18:50.987 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-02T15:18:50.987 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-02T15:18:50.987 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-02T15:18:50.987 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-02T15:18:50.994 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-02T15:18:50.994 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-02T15:18:50.994 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-02-02T15:18:50.994 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-02T15:18:50.994 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-02T15:18:50.995 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-02T15:18:50.995 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-02T15:18:50.995 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-02-02T15:18:50.995 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-02T15:18:50.995 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-02T15:18:50.995 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-02T15:18:50.995 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-02T15:18:50.995 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-02T15:18:50.995 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-02T15:18:50.995 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "restful" 2024-02-02T15:18:50.995 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout ], 2024-02-02T15:18:50.995 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-02T15:18:50.995 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-02-02T15:18:50.995 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-02T15:18:50.996 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-02T15:18:50.996 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "modified": "2024-02-02T15:18:26.747079+0000", 2024-02-02T15:18:50.996 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-02T15:18:50.996 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout }, 2024-02-02T15:18:50.996 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-02T15:18:50.996 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout } 2024-02-02T15:18:50.996 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:47.457+0000 7fd796800700 1 Processor -- start 2024-02-02T15:18:50.996 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:47.457+0000 7fd796800700 1 -- start start 2024-02-02T15:18:50.996 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:47.458+0000 7fd796800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd790108800 0x7fd790108c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:18:50.996 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:47.458+0000 7fd796800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7901091f0 con 0x7fd790108800 2024-02-02T15:18:50.996 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:47.458+0000 7fd78f400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd790108800 0x7fd790108c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:18:50.996 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:47.458+0000 7fd78f400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd790108800 0x7fd790108c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55038/0 (socket says 172.21.15.1:55038) 2024-02-02T15:18:50.996 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:47.458+0000 7fd78f400700 1 -- 172.21.15.1:0/4233823095 learned_addr learned my addr 172.21.15.1:0/4233823095 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:18:50.997 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:47.459+0000 7fd78f400700 1 -- 172.21.15.1:0/4233823095 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd790109a00 con 0x7fd790108800 2024-02-02T15:18:50.997 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:47.459+0000 7fd78f400700 1 --2- 172.21.15.1:0/4233823095 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd790108800 0x7fd790108c20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd7800098a0 tx=0x7fd780009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21eec52efdecb2c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:18:50.997 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:47.459+0000 7fd78e000700 1 -- 172.21.15.1:0/4233823095 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd780004160 con 0x7fd790108800 2024-02-02T15:18:50.997 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:47.459+0000 7fd78e000700 1 -- 172.21.15.1:0/4233823095 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd7800042c0 con 0x7fd790108800 2024-02-02T15:18:50.997 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:47.459+0000 7fd78e000700 1 -- 172.21.15.1:0/4233823095 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd780011a00 con 0x7fd790108800 2024-02-02T15:18:50.997 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:47.460+0000 7fd796800700 1 -- 172.21.15.1:0/4233823095 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd790108800 msgr2=0x7fd790108c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:18:50.997 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:47.460+0000 7fd796800700 1 --2- 172.21.15.1:0/4233823095 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd790108800 0x7fd790108c20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd7800098a0 tx=0x7fd780009bb0 comp rx=0 tx=0).stop 2024-02-02T15:18:50.997 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:47.460+0000 7fd796800700 1 -- 172.21.15.1:0/4233823095 shutdown_connections 2024-02-02T15:18:50.997 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:47.460+0000 7fd796800700 1 --2- 172.21.15.1:0/4233823095 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd790108800 0x7fd790108c20 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:18:50.997 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:47.460+0000 7fd796800700 1 -- 172.21.15.1:0/4233823095 >> 172.21.15.1:0/4233823095 conn(0x7fd79007be00 msgr2=0x7fd7901064b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:18:50.997 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:47.460+0000 7fd796800700 1 -- 172.21.15.1:0/4233823095 shutdown_connections 2024-02-02T15:18:50.997 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:47.460+0000 7fd796800700 1 -- 172.21.15.1:0/4233823095 wait complete. 2024-02-02T15:18:50.997 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:47.461+0000 7fd796800700 1 Processor -- start 2024-02-02T15:18:50.997 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:47.462+0000 7fd796800700 1 -- start start 2024-02-02T15:18:50.997 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:47.462+0000 7fd796800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd790108800 0x7fd79019c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:18:50.998 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:47.462+0000 7fd796800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd79019cae0 con 0x7fd790108800 2024-02-02T15:18:50.998 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:47.462+0000 7fd78f400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd790108800 0x7fd79019c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:18:50.998 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:47.462+0000 7fd78f400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd790108800 0x7fd79019c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55046/0 (socket says 172.21.15.1:55046) 2024-02-02T15:18:50.998 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:47.462+0000 7fd78f400700 1 -- 172.21.15.1:0/1335269324 learned_addr learned my addr 172.21.15.1:0/1335269324 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:18:50.998 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:47.463+0000 7fd78f400700 1 -- 172.21.15.1:0/1335269324 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd780009550 con 0x7fd790108800 2024-02-02T15:18:50.998 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:47.463+0000 7fd78f400700 1 --2- 172.21.15.1:0/1335269324 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd790108800 0x7fd79019c5a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd780009130 tx=0x7fd780011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:18:50.998 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:47.463+0000 7fd787e00700 1 -- 172.21.15.1:0/1335269324 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd780003ab0 con 0x7fd790108800 2024-02-02T15:18:50.998 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:47.463+0000 7fd787e00700 1 -- 172.21.15.1:0/1335269324 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fd780003c10 con 0x7fd790108800 2024-02-02T15:18:50.998 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:47.463+0000 7fd796800700 1 -- 172.21.15.1:0/1335269324 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd79019cce0 con 0x7fd790108800 2024-02-02T15:18:50.998 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:47.463+0000 7fd787e00700 1 -- 172.21.15.1:0/1335269324 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd7800247d0 con 0x7fd790108800 2024-02-02T15:18:50.998 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:47.463+0000 7fd796800700 1 -- 172.21.15.1:0/1335269324 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd79019d100 con 0x7fd790108800 2024-02-02T15:18:50.998 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:47.465+0000 7fd796800700 1 -- 172.21.15.1:0/1335269324 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7901965e0 con 0x7fd790108800 2024-02-02T15:18:50.998 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:47.465+0000 7fd787e00700 1 -- 172.21.15.1:0/1335269324 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 4) v1 ==== 45257+0+0 (secure 0 0 0) 0x7fd78001abb0 con 0x7fd790108800 2024-02-02T15:18:50.998 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:47.465+0000 7fd787e00700 1 --2- 172.21.15.1:0/1335269324 >> [v2:172.21.15.1:6800/1162530732,v1:172.21.15.1:6801/1162530732] conn(0x7fd7700382b0 0x7fd77003a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:18:50.998 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:47.465+0000 7fd787e00700 1 -- 172.21.15.1:0/1335269324 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fd78004bf80 con 0x7fd790108800 2024-02-02T15:18:50.999 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:47.465+0000 7fd78ea00700 1 --2- 172.21.15.1:0/1335269324 >> [v2:172.21.15.1:6800/1162530732,v1:172.21.15.1:6801/1162530732] conn(0x7fd7700382b0 0x7fd77003a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:18:50.999 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:47.466+0000 7fd78ea00700 1 --2- 172.21.15.1:0/1335269324 >> [v2:172.21.15.1:6800/1162530732,v1:172.21.15.1:6801/1162530732] conn(0x7fd7700382b0 0x7fd77003a770 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd778006fd0 tx=0x7fd778006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:18:50.999 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:47.471+0000 7fd787e00700 1 -- 172.21.15.1:0/1335269324 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7fd780018920 con 0x7fd790108800 2024-02-02T15:18:50.999 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:47.766+0000 7fd796800700 1 -- 172.21.15.1:0/1335269324 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fd79002d090 con 0x7fd790108800 2024-02-02T15:18:50.999 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:47.766+0000 7fd787e00700 1 -- 172.21.15.1:0/1335269324 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7fd780018350 con 0x7fd790108800 2024-02-02T15:18:50.999 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:47.768+0000 7fd796800700 1 -- 172.21.15.1:0/1335269324 >> [v2:172.21.15.1:6800/1162530732,v1:172.21.15.1:6801/1162530732] conn(0x7fd7700382b0 msgr2=0x7fd77003a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:18:50.999 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:47.768+0000 7fd796800700 1 --2- 172.21.15.1:0/1335269324 >> [v2:172.21.15.1:6800/1162530732,v1:172.21.15.1:6801/1162530732] conn(0x7fd7700382b0 0x7fd77003a770 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd778006fd0 tx=0x7fd778006e40 comp rx=0 tx=0).stop 2024-02-02T15:18:50.999 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:47.768+0000 7fd796800700 1 -- 172.21.15.1:0/1335269324 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd790108800 msgr2=0x7fd79019c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:18:50.999 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:47.768+0000 7fd796800700 1 --2- 172.21.15.1:0/1335269324 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd790108800 0x7fd79019c5a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd780009130 tx=0x7fd780011ad0 comp rx=0 tx=0).stop 2024-02-02T15:18:50.999 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:47.768+0000 7fd796800700 1 -- 172.21.15.1:0/1335269324 shutdown_connections 2024-02-02T15:18:50.999 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:47.768+0000 7fd796800700 1 --2- 172.21.15.1:0/1335269324 >> [v2:172.21.15.1:6800/1162530732,v1:172.21.15.1:6801/1162530732] conn(0x7fd7700382b0 0x7fd77003a770 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:18:50.999 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:47.768+0000 7fd796800700 1 --2- 172.21.15.1:0/1335269324 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd790108800 0x7fd79019c5a0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:18:50.999 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:47.768+0000 7fd796800700 1 -- 172.21.15.1:0/1335269324 >> 172.21.15.1:0/1335269324 conn(0x7fd79007be00 msgr2=0x7fd790106800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:18:50.999 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:47.769+0000 7fd796800700 1 -- 172.21.15.1:0/1335269324 shutdown_connections 2024-02-02T15:18:51.000 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:47.769+0000 7fd796800700 1 -- 172.21.15.1:0/1335269324 wait complete. 2024-02-02T15:18:51.000 INFO:teuthology.orchestra.run.smithi001.stdout:mgr is available 2024-02-02T15:18:51.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:51 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/876031623' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-02-02T15:18:51.950 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2024-02-02T15:18:51.950 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout [global] 2024-02-02T15:18:51.950 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout fsid = 371a409c-c1de-11ee-95b6-87774f69a715 2024-02-02T15:18:51.950 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-02-02T15:18:51.950 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-02-02T15:18:51.950 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-02-02T15:18:51.951 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-02-02T15:18:51.951 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2024-02-02T15:18:51.951 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout [mgr] 2024-02-02T15:18:51.951 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-02-02T15:18:51.951 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 2024-02-02T15:18:51.951 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout [osd] 2024-02-02T15:18:51.951 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-02-02T15:18:51.951 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-02-02T15:18:51.951 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-02-02T15:18:51.951 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-02-02T15:18:51.951 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-02-02T15:18:51.951 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:51.387+0000 7f4d69400700 1 Processor -- start 2024-02-02T15:18:51.951 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:51.387+0000 7f4d69400700 1 -- start start 2024-02-02T15:18:51.951 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:51.387+0000 7f4d69400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4d64106620 0x7f4d64106a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:18:51.951 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:51.387+0000 7f4d69400700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d64107010 con 0x7f4d64106620 2024-02-02T15:18:51.952 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:51.388+0000 7f4d62800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4d64106620 0x7f4d64106a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:18:51.952 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:51.388+0000 7f4d62800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4d64106620 0x7f4d64106a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55058/0 (socket says 172.21.15.1:55058) 2024-02-02T15:18:51.952 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:51.388+0000 7f4d62800700 1 -- 172.21.15.1:0/1175641748 learned_addr learned my addr 172.21.15.1:0/1175641748 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:18:51.952 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:51.388+0000 7f4d62800700 1 -- 172.21.15.1:0/1175641748 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d64107820 con 0x7f4d64106620 2024-02-02T15:18:51.952 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:51.388+0000 7f4d62800700 1 --2- 172.21.15.1:0/1175641748 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4d64106620 0x7f4d64106a40 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f4d540098a0 tx=0x7f4d54009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69b85350e1ec3d1a server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:18:51.952 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:51.389+0000 7f4d61400700 1 -- 172.21.15.1:0/1175641748 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4d54004160 con 0x7f4d64106620 2024-02-02T15:18:51.952 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:51.389+0000 7f4d61400700 1 -- 172.21.15.1:0/1175641748 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4d540042c0 con 0x7f4d64106620 2024-02-02T15:18:51.952 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:51.389+0000 7f4d61400700 1 -- 172.21.15.1:0/1175641748 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4d54011a00 con 0x7f4d64106620 2024-02-02T15:18:51.952 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:51.389+0000 7f4d69400700 1 -- 172.21.15.1:0/1175641748 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4d64106620 msgr2=0x7f4d64106a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:18:51.952 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:51.389+0000 7f4d69400700 1 --2- 172.21.15.1:0/1175641748 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4d64106620 0x7f4d64106a40 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f4d540098a0 tx=0x7f4d54009bb0 comp rx=0 tx=0).stop 2024-02-02T15:18:51.952 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:51.389+0000 7f4d69400700 1 -- 172.21.15.1:0/1175641748 shutdown_connections 2024-02-02T15:18:51.952 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:51.389+0000 7f4d69400700 1 --2- 172.21.15.1:0/1175641748 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4d64106620 0x7f4d64106a40 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:18:51.953 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:51.389+0000 7f4d69400700 1 -- 172.21.15.1:0/1175641748 >> 172.21.15.1:0/1175641748 conn(0x7f4d64101c20 msgr2=0x7f4d64104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:18:51.953 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:51.389+0000 7f4d69400700 1 -- 172.21.15.1:0/1175641748 shutdown_connections 2024-02-02T15:18:51.953 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:51.389+0000 7f4d69400700 1 -- 172.21.15.1:0/1175641748 wait complete. 2024-02-02T15:18:51.953 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:51.390+0000 7f4d69400700 1 Processor -- start 2024-02-02T15:18:51.953 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:51.390+0000 7f4d69400700 1 -- start start 2024-02-02T15:18:51.953 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:51.391+0000 7f4d69400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4d64106620 0x7f4d6419a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:18:51.953 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:51.391+0000 7f4d69400700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6419a8c0 con 0x7f4d64106620 2024-02-02T15:18:51.953 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:51.391+0000 7f4d62800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4d64106620 0x7f4d6419a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:18:51.953 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:51.391+0000 7f4d62800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4d64106620 0x7f4d6419a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55074/0 (socket says 172.21.15.1:55074) 2024-02-02T15:18:51.953 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:51.391+0000 7f4d62800700 1 -- 172.21.15.1:0/876031623 learned_addr learned my addr 172.21.15.1:0/876031623 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:18:51.953 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:51.391+0000 7f4d62800700 1 -- 172.21.15.1:0/876031623 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d54009550 con 0x7f4d64106620 2024-02-02T15:18:51.953 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:51.392+0000 7f4d62800700 1 --2- 172.21.15.1:0/876031623 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4d64106620 0x7f4d6419a380 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4d54009130 tx=0x7f4d54011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:18:51.953 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:51.392+0000 7f4d5b400700 1 -- 172.21.15.1:0/876031623 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4d54003ab0 con 0x7f4d64106620 2024-02-02T15:18:51.953 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:51.392+0000 7f4d5b400700 1 -- 172.21.15.1:0/876031623 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4d54003c10 con 0x7f4d64106620 2024-02-02T15:18:51.953 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:51.392+0000 7f4d5b400700 1 -- 172.21.15.1:0/876031623 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4d540247d0 con 0x7f4d64106620 2024-02-02T15:18:51.954 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:51.392+0000 7f4d69400700 1 -- 172.21.15.1:0/876031623 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d6419aac0 con 0x7f4d64106620 2024-02-02T15:18:51.954 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:51.392+0000 7f4d69400700 1 -- 172.21.15.1:0/876031623 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d6419aee0 con 0x7f4d64106620 2024-02-02T15:18:51.954 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:51.393+0000 7f4d5b400700 1 -- 172.21.15.1:0/876031623 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 4) v1 ==== 45257+0+0 (secure 0 0 0) 0x7f4d5401abb0 con 0x7f4d64106620 2024-02-02T15:18:51.954 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:51.393+0000 7f4d69400700 1 -- 172.21.15.1:0/876031623 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d64062370 con 0x7f4d64106620 2024-02-02T15:18:51.954 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:51.393+0000 7f4d5b400700 1 --2- 172.21.15.1:0/876031623 >> [v2:172.21.15.1:6800/1162530732,v1:172.21.15.1:6801/1162530732] conn(0x7f4d44038300 0x7f4d4403a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:18:51.954 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:51.393+0000 7f4d5b400700 1 -- 172.21.15.1:0/876031623 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f4d5404bee0 con 0x7f4d64106620 2024-02-02T15:18:51.954 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:51.394+0000 7f4d61e00700 1 --2- 172.21.15.1:0/876031623 >> [v2:172.21.15.1:6800/1162530732,v1:172.21.15.1:6801/1162530732] conn(0x7f4d44038300 0x7f4d4403a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:18:51.954 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:51.394+0000 7f4d61e00700 1 --2- 172.21.15.1:0/876031623 >> [v2:172.21.15.1:6800/1162530732,v1:172.21.15.1:6801/1162530732] conn(0x7f4d44038300 0x7f4d4403a7c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c006fd0 tx=0x7f4d4c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:18:51.954 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:51.398+0000 7f4d5b400700 1 -- 172.21.15.1:0/876031623 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f4d54018710 con 0x7f4d64106620 2024-02-02T15:18:51.954 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:51.603+0000 7f4d69400700 1 -- 172.21.15.1:0/876031623 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f4d6419dd40 con 0x7f4d64106620 2024-02-02T15:18:51.954 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:51.604+0000 7f4d5b400700 1 -- 172.21.15.1:0/876031623 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f4d54018350 con 0x7f4d64106620 2024-02-02T15:18:51.954 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:51.606+0000 7f4d69400700 1 -- 172.21.15.1:0/876031623 >> [v2:172.21.15.1:6800/1162530732,v1:172.21.15.1:6801/1162530732] conn(0x7f4d44038300 msgr2=0x7f4d4403a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:18:51.954 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:51.606+0000 7f4d69400700 1 --2- 172.21.15.1:0/876031623 >> [v2:172.21.15.1:6800/1162530732,v1:172.21.15.1:6801/1162530732] conn(0x7f4d44038300 0x7f4d4403a7c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c006fd0 tx=0x7f4d4c006e40 comp rx=0 tx=0).stop 2024-02-02T15:18:51.954 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:51.606+0000 7f4d69400700 1 -- 172.21.15.1:0/876031623 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4d64106620 msgr2=0x7f4d6419a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:18:51.954 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:51.606+0000 7f4d69400700 1 --2- 172.21.15.1:0/876031623 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4d64106620 0x7f4d6419a380 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f4d54009130 tx=0x7f4d54011ad0 comp rx=0 tx=0).stop 2024-02-02T15:18:51.955 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:51.606+0000 7f4d69400700 1 -- 172.21.15.1:0/876031623 shutdown_connections 2024-02-02T15:18:51.955 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:51.606+0000 7f4d69400700 1 --2- 172.21.15.1:0/876031623 >> [v2:172.21.15.1:6800/1162530732,v1:172.21.15.1:6801/1162530732] conn(0x7f4d44038300 0x7f4d4403a7c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:18:51.955 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:51.606+0000 7f4d69400700 1 --2- 172.21.15.1:0/876031623 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4d64106620 0x7f4d6419a380 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:18:51.955 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:51.606+0000 7f4d69400700 1 -- 172.21.15.1:0/876031623 >> 172.21.15.1:0/876031623 conn(0x7f4d64101c20 msgr2=0x7f4d64103310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:18:51.955 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:51.606+0000 7f4d69400700 1 -- 172.21.15.1:0/876031623 shutdown_connections 2024-02-02T15:18:51.955 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:51.606+0000 7f4d69400700 1 -- 172.21.15.1:0/876031623 wait complete. 2024-02-02T15:18:51.955 INFO:teuthology.orchestra.run.smithi001.stdout:Enabling cephadm module... 2024-02-02T15:18:53.033 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:52.306+0000 7fc998c00700 1 Processor -- start 2024-02-02T15:18:53.033 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:52.306+0000 7fc998c00700 1 -- start start 2024-02-02T15:18:53.034 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:52 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/3503784043' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-02-02T15:18:53.034 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:18:52 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: ignoring --setuser ceph since I am not root 2024-02-02T15:18:53.034 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:18:52 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: ignoring --setgroup ceph since I am not root 2024-02-02T15:18:53.038 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:52.306+0000 7fc998c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc994106f30 0x7fc994109320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:18:53.038 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:52.306+0000 7fc998c00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc994079dd0 con 0x7fc994106f30 2024-02-02T15:18:53.039 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:52.306+0000 7fc993400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc994106f30 0x7fc994109320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:18:53.039 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:52.307+0000 7fc993400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc994106f30 0x7fc994109320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55082/0 (socket says 172.21.15.1:55082) 2024-02-02T15:18:53.039 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:52.307+0000 7fc993400700 1 -- 172.21.15.1:0/604855625 learned_addr learned my addr 172.21.15.1:0/604855625 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:18:53.039 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:52.307+0000 7fc993400700 1 -- 172.21.15.1:0/604855625 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc994079f10 con 0x7fc994106f30 2024-02-02T15:18:53.039 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:52.307+0000 7fc993400700 1 --2- 172.21.15.1:0/604855625 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc994106f30 0x7fc994109320 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc9840098a0 tx=0x7fc984009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a4d98774f7395ad server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:18:53.039 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:52.307+0000 7fc992000700 1 -- 172.21.15.1:0/604855625 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc984004160 con 0x7fc994106f30 2024-02-02T15:18:53.039 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:52.307+0000 7fc992000700 1 -- 172.21.15.1:0/604855625 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc9840042c0 con 0x7fc994106f30 2024-02-02T15:18:53.039 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:52.308+0000 7fc992000700 1 -- 172.21.15.1:0/604855625 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc984011a00 con 0x7fc994106f30 2024-02-02T15:18:53.039 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:52.308+0000 7fc998c00700 1 -- 172.21.15.1:0/604855625 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc994106f30 msgr2=0x7fc994109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:18:53.039 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:52.308+0000 7fc998c00700 1 --2- 172.21.15.1:0/604855625 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc994106f30 0x7fc994109320 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc9840098a0 tx=0x7fc984009bb0 comp rx=0 tx=0).stop 2024-02-02T15:18:53.039 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:52.308+0000 7fc998c00700 1 -- 172.21.15.1:0/604855625 shutdown_connections 2024-02-02T15:18:53.039 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:52.308+0000 7fc998c00700 1 --2- 172.21.15.1:0/604855625 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc994106f30 0x7fc994109320 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:18:53.039 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:52.308+0000 7fc998c00700 1 -- 172.21.15.1:0/604855625 >> 172.21.15.1:0/604855625 conn(0x7fc9941008e0 msgr2=0x7fc994102d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:18:53.039 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:52.308+0000 7fc998c00700 1 -- 172.21.15.1:0/604855625 shutdown_connections 2024-02-02T15:18:53.039 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:52.308+0000 7fc998c00700 1 -- 172.21.15.1:0/604855625 wait complete. 2024-02-02T15:18:53.040 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:52.309+0000 7fc998c00700 1 Processor -- start 2024-02-02T15:18:53.040 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:52.309+0000 7fc998c00700 1 -- start start 2024-02-02T15:18:53.040 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:52.310+0000 7fc998c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc994106f30 0x7fc994106890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:18:53.040 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:52.310+0000 7fc998c00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc994106dd0 con 0x7fc994106f30 2024-02-02T15:18:53.040 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:52.310+0000 7fc993400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc994106f30 0x7fc994106890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:18:53.040 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:52.310+0000 7fc993400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc994106f30 0x7fc994106890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55084/0 (socket says 172.21.15.1:55084) 2024-02-02T15:18:53.040 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:52.310+0000 7fc993400700 1 -- 172.21.15.1:0/3503784043 learned_addr learned my addr 172.21.15.1:0/3503784043 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:18:53.040 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:52.310+0000 7fc993400700 1 -- 172.21.15.1:0/3503784043 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc984009550 con 0x7fc994106f30 2024-02-02T15:18:53.040 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:52.311+0000 7fc993400700 1 --2- 172.21.15.1:0/3503784043 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc994106f30 0x7fc994106890 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc984006af0 tx=0x7fc984011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:18:53.040 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:52.311+0000 7fc98be00700 1 -- 172.21.15.1:0/3503784043 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc984003a40 con 0x7fc994106f30 2024-02-02T15:18:53.040 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:52.311+0000 7fc98be00700 1 -- 172.21.15.1:0/3503784043 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc984003ba0 con 0x7fc994106f30 2024-02-02T15:18:53.040 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:52.311+0000 7fc998c00700 1 -- 172.21.15.1:0/3503784043 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc994104ee0 con 0x7fc994106f30 2024-02-02T15:18:53.040 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:52.311+0000 7fc98be00700 1 -- 172.21.15.1:0/3503784043 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc98401a6e0 con 0x7fc994106f30 2024-02-02T15:18:53.040 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:52.311+0000 7fc998c00700 1 -- 172.21.15.1:0/3503784043 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc994105300 con 0x7fc994106f30 2024-02-02T15:18:53.041 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:52.312+0000 7fc98be00700 1 -- 172.21.15.1:0/3503784043 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 4) v1 ==== 45257+0+0 (secure 0 0 0) 0x7fc984024430 con 0x7fc994106f30 2024-02-02T15:18:53.041 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:52.312+0000 7fc998c00700 1 -- 172.21.15.1:0/3503784043 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc99410a590 con 0x7fc994106f30 2024-02-02T15:18:53.041 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:52.312+0000 7fc98be00700 1 --2- 172.21.15.1:0/3503784043 >> [v2:172.21.15.1:6800/1162530732,v1:172.21.15.1:6801/1162530732] conn(0x7fc974038300 0x7fc97403a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:18:53.041 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:52.313+0000 7fc98be00700 1 -- 172.21.15.1:0/3503784043 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc98404c0f0 con 0x7fc994106f30 2024-02-02T15:18:53.041 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:52.313+0000 7fc992a00700 1 --2- 172.21.15.1:0/3503784043 >> [v2:172.21.15.1:6800/1162530732,v1:172.21.15.1:6801/1162530732] conn(0x7fc974038300 0x7fc97403a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:18:53.041 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:52.313+0000 7fc992a00700 1 --2- 172.21.15.1:0/3503784043 >> [v2:172.21.15.1:6800/1162530732,v1:172.21.15.1:6801/1162530732] conn(0x7fc974038300 0x7fc97403a7c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc97c006fd0 tx=0x7fc97c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:18:53.041 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:52.318+0000 7fc98be00700 1 -- 172.21.15.1:0/3503784043 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7fc98407f110 con 0x7fc994106f30 2024-02-02T15:18:53.041 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:52.580+0000 7fc998c00700 1 -- 172.21.15.1:0/3503784043 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7fc99410a780 con 0x7fc994106f30 2024-02-02T15:18:53.041 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:52.696+0000 7fc98be00700 1 -- 172.21.15.1:0/3503784043 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mgrmap(e 5) v1 ==== 45268+0+0 (secure 0 0 0) 0x7fc98402d8c0 con 0x7fc994106f30 2024-02-02T15:18:53.041 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:52.698+0000 7fc98be00700 1 -- 172.21.15.1:0/3503784043 <== mon.0 v2:172.21.15.1:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7fc984024b10 con 0x7fc994106f30 2024-02-02T15:18:53.041 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:52.700+0000 7fc998c00700 1 -- 172.21.15.1:0/3503784043 >> [v2:172.21.15.1:6800/1162530732,v1:172.21.15.1:6801/1162530732] conn(0x7fc974038300 msgr2=0x7fc97403a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:18:53.041 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:52.700+0000 7fc998c00700 1 --2- 172.21.15.1:0/3503784043 >> [v2:172.21.15.1:6800/1162530732,v1:172.21.15.1:6801/1162530732] conn(0x7fc974038300 0x7fc97403a7c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc97c006fd0 tx=0x7fc97c006e40 comp rx=0 tx=0).stop 2024-02-02T15:18:53.041 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:52.700+0000 7fc998c00700 1 -- 172.21.15.1:0/3503784043 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc994106f30 msgr2=0x7fc994106890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:18:53.042 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:52.700+0000 7fc998c00700 1 --2- 172.21.15.1:0/3503784043 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc994106f30 0x7fc994106890 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc984006af0 tx=0x7fc984011ad0 comp rx=0 tx=0).stop 2024-02-02T15:18:53.042 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:52.700+0000 7fc998c00700 1 -- 172.21.15.1:0/3503784043 shutdown_connections 2024-02-02T15:18:53.042 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:52.700+0000 7fc998c00700 1 --2- 172.21.15.1:0/3503784043 >> [v2:172.21.15.1:6800/1162530732,v1:172.21.15.1:6801/1162530732] conn(0x7fc974038300 0x7fc97403a7c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:18:53.042 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:52.700+0000 7fc998c00700 1 --2- 172.21.15.1:0/3503784043 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc994106f30 0x7fc994106890 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:18:53.042 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:52.700+0000 7fc998c00700 1 -- 172.21.15.1:0/3503784043 >> 172.21.15.1:0/3503784043 conn(0x7fc9941008e0 msgr2=0x7fc994101fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:18:53.042 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:52.700+0000 7fc998c00700 1 -- 172.21.15.1:0/3503784043 shutdown_connections 2024-02-02T15:18:53.042 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:52.700+0000 7fc998c00700 1 -- 172.21.15.1:0/3503784043 wait complete. 2024-02-02T15:18:53.675 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:18:53 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:18:53.317+0000 7f808c6b2200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-02T15:18:53.675 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:18:53 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:18:53.441+0000 7f808c6b2200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-02T15:18:53.675 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:18:53 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:18:53.561+0000 7f808c6b2200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-02T15:18:53.675 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:18:53 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:18:53.675+0000 7f808c6b2200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-02T15:18:53.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:53 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/3503784043' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-02-02T15:18:53.928 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:53 smithi001 ceph-mon[35470]: mgrmap e5: x(active, since 11s) 2024-02-02T15:18:54.093 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout { 2024-02-02T15:18:54.094 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-02-02T15:18:54.094 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-02T15:18:54.094 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-02-02T15:18:54.094 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-02-02T15:18:54.094 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout } 2024-02-02T15:18:54.094 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:53.449+0000 7fbf1d200700 1 Processor -- start 2024-02-02T15:18:54.094 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:53.449+0000 7fbf1d200700 1 -- start start 2024-02-02T15:18:54.094 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:53.450+0000 7fbf1d200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf18106620 0x7fbf18106a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:18:54.094 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:53.450+0000 7fbf1d200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf18107010 con 0x7fbf18106620 2024-02-02T15:18:54.094 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:53.450+0000 7fbf15e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf18106620 0x7fbf18106a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:18:54.094 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:53.450+0000 7fbf15e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf18106620 0x7fbf18106a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47318/0 (socket says 172.21.15.1:47318) 2024-02-02T15:18:54.094 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:53.450+0000 7fbf15e00700 1 -- 172.21.15.1:0/2421244761 learned_addr learned my addr 172.21.15.1:0/2421244761 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:18:54.094 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:53.450+0000 7fbf15e00700 1 -- 172.21.15.1:0/2421244761 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf18107820 con 0x7fbf18106620 2024-02-02T15:18:54.094 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:53.451+0000 7fbf15e00700 1 --2- 172.21.15.1:0/2421244761 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf18106620 0x7fbf18106a40 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fbf080098a0 tx=0x7fbf08009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4194684f6647ddb3 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:18:54.095 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:53.451+0000 7fbf14a00700 1 -- 172.21.15.1:0/2421244761 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbf08004160 con 0x7fbf18106620 2024-02-02T15:18:54.095 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:53.451+0000 7fbf14a00700 1 -- 172.21.15.1:0/2421244761 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbf080042c0 con 0x7fbf18106620 2024-02-02T15:18:54.095 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:53.451+0000 7fbf14a00700 1 -- 172.21.15.1:0/2421244761 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbf08011a00 con 0x7fbf18106620 2024-02-02T15:18:54.095 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:53.451+0000 7fbf1d200700 1 -- 172.21.15.1:0/2421244761 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf18106620 msgr2=0x7fbf18106a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:18:54.095 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:53.451+0000 7fbf1d200700 1 --2- 172.21.15.1:0/2421244761 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf18106620 0x7fbf18106a40 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fbf080098a0 tx=0x7fbf08009bb0 comp rx=0 tx=0).stop 2024-02-02T15:18:54.095 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:53.452+0000 7fbf1d200700 1 -- 172.21.15.1:0/2421244761 shutdown_connections 2024-02-02T15:18:54.095 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:53.452+0000 7fbf1d200700 1 --2- 172.21.15.1:0/2421244761 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf18106620 0x7fbf18106a40 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:18:54.095 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:53.452+0000 7fbf1d200700 1 -- 172.21.15.1:0/2421244761 >> 172.21.15.1:0/2421244761 conn(0x7fbf18101c20 msgr2=0x7fbf18104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:18:54.095 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:53.452+0000 7fbf1d200700 1 -- 172.21.15.1:0/2421244761 shutdown_connections 2024-02-02T15:18:54.095 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:53.452+0000 7fbf1d200700 1 -- 172.21.15.1:0/2421244761 wait complete. 2024-02-02T15:18:54.095 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:53.453+0000 7fbf1d200700 1 Processor -- start 2024-02-02T15:18:54.095 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:53.453+0000 7fbf1d200700 1 -- start start 2024-02-02T15:18:54.095 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:53.453+0000 7fbf1d200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf18106620 0x7fbf1819c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:18:54.096 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:53.453+0000 7fbf1d200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf1819cac0 con 0x7fbf18106620 2024-02-02T15:18:54.096 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:53.453+0000 7fbf15e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf18106620 0x7fbf1819c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:18:54.096 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:53.453+0000 7fbf15e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf18106620 0x7fbf1819c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47334/0 (socket says 172.21.15.1:47334) 2024-02-02T15:18:54.096 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:53.453+0000 7fbf15e00700 1 -- 172.21.15.1:0/1559115947 learned_addr learned my addr 172.21.15.1:0/1559115947 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:18:54.096 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:53.454+0000 7fbf15e00700 1 -- 172.21.15.1:0/1559115947 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf08009550 con 0x7fbf18106620 2024-02-02T15:18:54.096 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:53.454+0000 7fbf15e00700 1 --2- 172.21.15.1:0/1559115947 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf18106620 0x7fbf1819c580 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fbf08009130 tx=0x7fbf08011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:18:54.096 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:53.454+0000 7fbf0ea00700 1 -- 172.21.15.1:0/1559115947 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbf08003ab0 con 0x7fbf18106620 2024-02-02T15:18:54.096 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:53.454+0000 7fbf0ea00700 1 -- 172.21.15.1:0/1559115947 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fbf08003c10 con 0x7fbf18106620 2024-02-02T15:18:54.096 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:53.454+0000 7fbf1d200700 1 -- 172.21.15.1:0/1559115947 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf1819ccc0 con 0x7fbf18106620 2024-02-02T15:18:54.096 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:53.454+0000 7fbf0ea00700 1 -- 172.21.15.1:0/1559115947 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbf080247d0 con 0x7fbf18106620 2024-02-02T15:18:54.096 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:53.454+0000 7fbf1d200700 1 -- 172.21.15.1:0/1559115947 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf1819d0e0 con 0x7fbf18106620 2024-02-02T15:18:54.096 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:53.455+0000 7fbf0ea00700 1 -- 172.21.15.1:0/1559115947 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) v1 ==== 45268+0+0 (secure 0 0 0) 0x7fbf0801abb0 con 0x7fbf18106620 2024-02-02T15:18:54.096 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:53.456+0000 7fbf1d200700 1 -- 172.21.15.1:0/1559115947 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf18062370 con 0x7fbf18106620 2024-02-02T15:18:54.096 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:53.456+0000 7fbf0ea00700 1 --2- 172.21.15.1:0/1559115947 >> [v2:172.21.15.1:6800/1162530732,v1:172.21.15.1:6801/1162530732] conn(0x7fbef8038300 0x7fbef803a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:18:54.097 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:53.456+0000 7fbf0ea00700 1 -- 172.21.15.1:0/1559115947 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fbf0804bfc0 con 0x7fbf18106620 2024-02-02T15:18:54.097 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:53.456+0000 7fbf15400700 1 -- 172.21.15.1:0/1559115947 >> [v2:172.21.15.1:6800/1162530732,v1:172.21.15.1:6801/1162530732] conn(0x7fbef8038300 msgr2=0x7fbef803a7c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/1162530732 2024-02-02T15:18:54.097 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:53.456+0000 7fbf15400700 1 --2- 172.21.15.1:0/1559115947 >> [v2:172.21.15.1:6800/1162530732,v1:172.21.15.1:6801/1162530732] conn(0x7fbef8038300 0x7fbef803a7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-02T15:18:54.097 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:53.461+0000 7fbf0ea00700 1 -- 172.21.15.1:0/1559115947 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7fbf08018940 con 0x7fbf18106620 2024-02-02T15:18:54.097 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:53.657+0000 7fbf15400700 1 -- 172.21.15.1:0/1559115947 >> [v2:172.21.15.1:6800/1162530732,v1:172.21.15.1:6801/1162530732] conn(0x7fbef8038300 msgr2=0x7fbef803a7c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/1162530732 2024-02-02T15:18:54.097 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:53.657+0000 7fbf15400700 1 --2- 172.21.15.1:0/1559115947 >> [v2:172.21.15.1:6800/1162530732,v1:172.21.15.1:6801/1162530732] conn(0x7fbef8038300 0x7fbef803a7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-02T15:18:54.097 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:53.727+0000 7fbf1d200700 1 -- 172.21.15.1:0/1559115947 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fbf1819ff40 con 0x7fbf18106620 2024-02-02T15:18:54.097 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:53.728+0000 7fbf0ea00700 1 -- 172.21.15.1:0/1559115947 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7fbf08018350 con 0x7fbf18106620 2024-02-02T15:18:54.097 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:53.730+0000 7fbf1d200700 1 -- 172.21.15.1:0/1559115947 >> [v2:172.21.15.1:6800/1162530732,v1:172.21.15.1:6801/1162530732] conn(0x7fbef8038300 msgr2=0x7fbef803a7c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-02T15:18:54.097 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:53.730+0000 7fbf1d200700 1 --2- 172.21.15.1:0/1559115947 >> [v2:172.21.15.1:6800/1162530732,v1:172.21.15.1:6801/1162530732] conn(0x7fbef8038300 0x7fbef803a7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:18:54.097 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:53.730+0000 7fbf1d200700 1 -- 172.21.15.1:0/1559115947 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf18106620 msgr2=0x7fbf1819c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:18:54.097 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:53.730+0000 7fbf1d200700 1 --2- 172.21.15.1:0/1559115947 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf18106620 0x7fbf1819c580 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fbf08009130 tx=0x7fbf08011ad0 comp rx=0 tx=0).stop 2024-02-02T15:18:54.097 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:53.730+0000 7fbf1d200700 1 -- 172.21.15.1:0/1559115947 shutdown_connections 2024-02-02T15:18:54.097 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:53.730+0000 7fbf1d200700 1 --2- 172.21.15.1:0/1559115947 >> [v2:172.21.15.1:6800/1162530732,v1:172.21.15.1:6801/1162530732] conn(0x7fbef8038300 0x7fbef803a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:18:54.098 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:53.730+0000 7fbf1d200700 1 --2- 172.21.15.1:0/1559115947 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbf18106620 0x7fbf1819c580 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:18:54.098 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:53.730+0000 7fbf1d200700 1 -- 172.21.15.1:0/1559115947 >> 172.21.15.1:0/1559115947 conn(0x7fbf18101c20 msgr2=0x7fbf18103310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:18:54.098 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:53.731+0000 7fbf1d200700 1 -- 172.21.15.1:0/1559115947 shutdown_connections 2024-02-02T15:18:54.098 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:53.731+0000 7fbf1d200700 1 -- 172.21.15.1:0/1559115947 wait complete. 2024-02-02T15:18:54.098 INFO:teuthology.orchestra.run.smithi001.stdout:Waiting for the mgr to restart... 2024-02-02T15:18:54.098 INFO:teuthology.orchestra.run.smithi001.stdout:Waiting for mgr epoch 5... 2024-02-02T15:18:54.846 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:18:54 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/1559115947' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-02-02T15:18:54.847 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:18:54 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:18:54.541+0000 7f808c6b2200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-02T15:18:55.103 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:18:54 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:18:54.846+0000 7f808c6b2200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-02T15:18:55.103 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:18:54 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:18:54.960+0000 7f808c6b2200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-02T15:18:55.440 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:18:55 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:18:55.102+0000 7f808c6b2200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-02T15:18:55.440 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:18:55 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:18:55.303+0000 7f808c6b2200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-02T15:18:55.940 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:18:55 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:18:55.476+0000 7f808c6b2200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-02T15:18:55.940 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:18:55 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:18:55.590+0000 7f808c6b2200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-02T15:18:56.423 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:18:56 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:18:56.069+0000 7f808c6b2200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-02T15:18:56.690 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:18:56 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:18:56.423+0000 7f808c6b2200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-02T15:18:57.190 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:18:56 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:18:56.782+0000 7f808c6b2200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-02T15:18:57.190 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:18:56 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:18:56.909+0000 7f808c6b2200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-02T15:18:58.190 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:18:57 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:18:57.879+0000 7f808c6b2200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-02T15:18:58.190 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:18:58 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:18:58.006+0000 7f808c6b2200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-02T15:18:58.690 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:18:58 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:18:58.203+0000 7f808c6b2200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-02T15:18:59.440 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:18:59 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:18:59.037+0000 7f808c6b2200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-02T15:18:59.440 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:18:59 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:18:59.281+0000 7f808c6b2200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-02T15:18:59.940 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:18:59 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:18:59.540+0000 7f808c6b2200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-02T15:19:01.052 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:19:00 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:19:00.695+0000 7f808c6b2200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-02T15:19:01.428 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:19:01 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:19:01.053+0000 7f808c6b2200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-02T15:19:01.428 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:19:01 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:19:01.171+0000 7f808c6b2200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-02T15:19:01.690 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:19:01 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:19:01.429+0000 7f808c6b2200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-02T15:19:02.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:01 smithi001 ceph-mon[35470]: Active manager daemon x restarted 2024-02-02T15:19:02.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:01 smithi001 ceph-mon[35470]: Activating manager daemon x 2024-02-02T15:19:02.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:01 smithi001 ceph-mon[35470]: osdmap e2: 0 total, 0 up, 0 in 2024-02-02T15:19:02.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:01 smithi001 ceph-mon[35470]: mgrmap e6: x(active, starting, since 0.00387618s) 2024-02-02T15:19:02.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:01 smithi001 ceph-mon[35470]: from='mgr.14122 172.21.15.1:0/954154330' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-02T15:19:02.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:01 smithi001 ceph-mon[35470]: from='mgr.14122 172.21.15.1:0/954154330' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-02T15:19:02.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:01 smithi001 ceph-mon[35470]: from='mgr.14122 172.21.15.1:0/954154330' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-02T15:19:02.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:01 smithi001 ceph-mon[35470]: from='mgr.14122 172.21.15.1:0/954154330' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-02T15:19:02.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:01 smithi001 ceph-mon[35470]: from='mgr.14122 172.21.15.1:0/954154330' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-02T15:19:02.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:01 smithi001 ceph-mon[35470]: Manager daemon x is now available 2024-02-02T15:19:02.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:01 smithi001 ceph-mon[35470]: from='mgr.14122 172.21.15.1:0/954154330' entity='mgr.x' 2024-02-02T15:19:02.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:01 smithi001 ceph-mon[35470]: from='mgr.14122 172.21.15.1:0/954154330' entity='mgr.x' 2024-02-02T15:19:02.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:01 smithi001 ceph-mon[35470]: from='mgr.14122 172.21.15.1:0/954154330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:19:02.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:01 smithi001 ceph-mon[35470]: from='mgr.14122 172.21.15.1:0/954154330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:19:02.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:01 smithi001 ceph-mon[35470]: from='mgr.14122 172.21.15.1:0/954154330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:19:02.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:01 smithi001 ceph-mon[35470]: from='mgr.14122 172.21.15.1:0/954154330' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-02-02T15:19:02.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:01 smithi001 ceph-mon[35470]: from='mgr.14122 172.21.15.1:0/954154330' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-02-02T15:19:02.748 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout { 2024-02-02T15:19:02.749 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:02 smithi001 ceph-mon[35470]: Found migration_current of "None". Setting to last migration. 2024-02-02T15:19:02.749 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:02 smithi001 ceph-mon[35470]: mgrmap e7: x(active, since 1.00714s) 2024-02-02T15:19:02.756 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-02-02T15:19:02.756 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "initialized": true 2024-02-02T15:19:02.756 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout } 2024-02-02T15:19:02.757 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:54.515+0000 7f182a400700 1 Processor -- start 2024-02-02T15:19:02.757 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:54.516+0000 7f182a400700 1 -- start start 2024-02-02T15:19:02.757 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:54.516+0000 7f182a400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1824106620 0x7f1824106a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:19:02.757 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:54.516+0000 7f182a400700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1824107010 con 0x7f1824106620 2024-02-02T15:19:02.757 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:54.516+0000 7f1823400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1824106620 0x7f1824106a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:19:02.757 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:54.516+0000 7f1823400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1824106620 0x7f1824106a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47342/0 (socket says 172.21.15.1:47342) 2024-02-02T15:19:02.757 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:54.516+0000 7f1823400700 1 -- 172.21.15.1:0/1206769343 learned_addr learned my addr 172.21.15.1:0/1206769343 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:19:02.757 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:54.517+0000 7f1823400700 1 -- 172.21.15.1:0/1206769343 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1824107820 con 0x7f1824106620 2024-02-02T15:19:02.757 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:54.517+0000 7f1823400700 1 --2- 172.21.15.1:0/1206769343 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1824106620 0x7f1824106a40 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f18140098a0 tx=0x7f1814009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67ec4c08f5384872 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:19:02.757 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:54.517+0000 7f1822000700 1 -- 172.21.15.1:0/1206769343 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1814004160 con 0x7f1824106620 2024-02-02T15:19:02.757 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:54.517+0000 7f1822000700 1 -- 172.21.15.1:0/1206769343 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f18140042c0 con 0x7f1824106620 2024-02-02T15:19:02.757 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:54.517+0000 7f1822000700 1 -- 172.21.15.1:0/1206769343 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1814011a00 con 0x7f1824106620 2024-02-02T15:19:02.757 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:54.518+0000 7f182a400700 1 -- 172.21.15.1:0/1206769343 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1824106620 msgr2=0x7f1824106a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:19:02.757 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:54.518+0000 7f182a400700 1 --2- 172.21.15.1:0/1206769343 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1824106620 0x7f1824106a40 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f18140098a0 tx=0x7f1814009bb0 comp rx=0 tx=0).stop 2024-02-02T15:19:02.758 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:54.518+0000 7f182a400700 1 -- 172.21.15.1:0/1206769343 shutdown_connections 2024-02-02T15:19:02.758 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:54.518+0000 7f182a400700 1 --2- 172.21.15.1:0/1206769343 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1824106620 0x7f1824106a40 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:02.758 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:54.518+0000 7f182a400700 1 -- 172.21.15.1:0/1206769343 >> 172.21.15.1:0/1206769343 conn(0x7f1824101c20 msgr2=0x7f1824104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:19:02.758 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:54.518+0000 7f182a400700 1 -- 172.21.15.1:0/1206769343 shutdown_connections 2024-02-02T15:19:02.758 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:54.518+0000 7f182a400700 1 -- 172.21.15.1:0/1206769343 wait complete. 2024-02-02T15:19:02.758 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:54.520+0000 7f182a400700 1 Processor -- start 2024-02-02T15:19:02.758 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:54.520+0000 7f182a400700 1 -- start start 2024-02-02T15:19:02.758 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:54.520+0000 7f182a400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1824106620 0x7f182419a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:19:02.758 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:54.520+0000 7f182a400700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f182419a8c0 con 0x7f1824106620 2024-02-02T15:19:02.758 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:54.521+0000 7f1823400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1824106620 0x7f182419a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:19:02.758 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:54.521+0000 7f1823400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1824106620 0x7f182419a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:47356/0 (socket says 172.21.15.1:47356) 2024-02-02T15:19:02.758 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:54.521+0000 7f1823400700 1 -- 172.21.15.1:0/3600364458 learned_addr learned my addr 172.21.15.1:0/3600364458 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:19:02.758 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:54.521+0000 7f1823400700 1 -- 172.21.15.1:0/3600364458 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1814009550 con 0x7f1824106620 2024-02-02T15:19:02.758 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:54.521+0000 7f1823400700 1 --2- 172.21.15.1:0/3600364458 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1824106620 0x7f182419a380 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f1814009130 tx=0x7f1814011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:19:02.759 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:54.522+0000 7f181be00700 1 -- 172.21.15.1:0/3600364458 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1814003ab0 con 0x7f1824106620 2024-02-02T15:19:02.759 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:54.522+0000 7f181be00700 1 -- 172.21.15.1:0/3600364458 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1814003c10 con 0x7f1824106620 2024-02-02T15:19:02.759 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:54.522+0000 7f182a400700 1 -- 172.21.15.1:0/3600364458 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f182419aac0 con 0x7f1824106620 2024-02-02T15:19:02.759 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:54.522+0000 7f181be00700 1 -- 172.21.15.1:0/3600364458 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f18140247d0 con 0x7f1824106620 2024-02-02T15:19:02.759 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:54.522+0000 7f182a400700 1 -- 172.21.15.1:0/3600364458 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f182419aee0 con 0x7f1824106620 2024-02-02T15:19:02.759 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:54.523+0000 7f181be00700 1 -- 172.21.15.1:0/3600364458 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 5) v1 ==== 45268+0+0 (secure 0 0 0) 0x7f181402d3f0 con 0x7f1824106620 2024-02-02T15:19:02.759 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:54.523+0000 7f181be00700 1 --2- 172.21.15.1:0/3600364458 >> [v2:172.21.15.1:6800/1162530732,v1:172.21.15.1:6801/1162530732] conn(0x7f1804038350 0x7f180403a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:19:02.759 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:54.523+0000 7f181be00700 1 -- 172.21.15.1:0/3600364458 --> [v2:172.21.15.1:6800/1162530732,v1:172.21.15.1:6801/1162530732] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f180403af20 con 0x7f1804038350 2024-02-02T15:19:02.759 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:54.523+0000 7f181be00700 1 -- 172.21.15.1:0/3600364458 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f181404c0c0 con 0x7f1824106620 2024-02-02T15:19:02.759 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:54.523+0000 7f1822a00700 1 -- 172.21.15.1:0/3600364458 >> [v2:172.21.15.1:6800/1162530732,v1:172.21.15.1:6801/1162530732] conn(0x7f1804038350 msgr2=0x7f180403a810 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/1162530732 2024-02-02T15:19:02.759 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:54.523+0000 7f1822a00700 1 --2- 172.21.15.1:0/3600364458 >> [v2:172.21.15.1:6800/1162530732,v1:172.21.15.1:6801/1162530732] conn(0x7f1804038350 0x7f180403a810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-02T15:19:02.759 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:54.724+0000 7f1822a00700 1 -- 172.21.15.1:0/3600364458 >> [v2:172.21.15.1:6800/1162530732,v1:172.21.15.1:6801/1162530732] conn(0x7f1804038350 msgr2=0x7f180403a810 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/1162530732 2024-02-02T15:19:02.759 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:54.724+0000 7f1822a00700 1 --2- 172.21.15.1:0/3600364458 >> [v2:172.21.15.1:6800/1162530732,v1:172.21.15.1:6801/1162530732] conn(0x7f1804038350 0x7f180403a810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-02T15:19:02.759 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:55.125+0000 7f1822a00700 1 -- 172.21.15.1:0/3600364458 >> [v2:172.21.15.1:6800/1162530732,v1:172.21.15.1:6801/1162530732] conn(0x7f1804038350 msgr2=0x7f180403a810 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/1162530732 2024-02-02T15:19:02.760 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:55.125+0000 7f1822a00700 1 --2- 172.21.15.1:0/3600364458 >> [v2:172.21.15.1:6800/1162530732,v1:172.21.15.1:6801/1162530732] conn(0x7f1804038350 0x7f180403a810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-02-02T15:19:02.760 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:55.926+0000 7f1822a00700 1 -- 172.21.15.1:0/3600364458 >> [v2:172.21.15.1:6800/1162530732,v1:172.21.15.1:6801/1162530732] conn(0x7f1804038350 msgr2=0x7f180403a810 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/1162530732 2024-02-02T15:19:02.760 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:55.926+0000 7f1822a00700 1 --2- 172.21.15.1:0/3600364458 >> [v2:172.21.15.1:6800/1162530732,v1:172.21.15.1:6801/1162530732] conn(0x7f1804038350 0x7f180403a810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-02-02T15:19:02.760 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:57.528+0000 7f1822a00700 1 -- 172.21.15.1:0/3600364458 >> [v2:172.21.15.1:6800/1162530732,v1:172.21.15.1:6801/1162530732] conn(0x7f1804038350 msgr2=0x7f180403a810 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/1162530732 2024-02-02T15:19:02.760 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:18:57.528+0000 7f1822a00700 1 --2- 172.21.15.1:0/3600364458 >> [v2:172.21.15.1:6800/1162530732,v1:172.21.15.1:6801/1162530732] conn(0x7f1804038350 0x7f180403a810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-02-02T15:19:02.760 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:00.731+0000 7f1822a00700 1 -- 172.21.15.1:0/3600364458 >> [v2:172.21.15.1:6800/1162530732,v1:172.21.15.1:6801/1162530732] conn(0x7f1804038350 msgr2=0x7f180403a810 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/1162530732 2024-02-02T15:19:02.760 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:00.731+0000 7f1822a00700 1 --2- 172.21.15.1:0/3600364458 >> [v2:172.21.15.1:6800/1162530732,v1:172.21.15.1:6801/1162530732] conn(0x7f1804038350 0x7f180403a810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-02-02T15:19:02.760 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:01.434+0000 7f181be00700 1 -- 172.21.15.1:0/3600364458 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mgrmap(e 6) v1 ==== 45035+0+0 (secure 0 0 0) 0x7f181402d8d0 con 0x7f1824106620 2024-02-02T15:19:02.760 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:01.434+0000 7f181be00700 1 -- 172.21.15.1:0/3600364458 >> [v2:172.21.15.1:6800/1162530732,v1:172.21.15.1:6801/1162530732] conn(0x7f1804038350 msgr2=0x7f180403a810 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-02T15:19:02.760 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:01.434+0000 7f181be00700 1 --2- 172.21.15.1:0/3600364458 >> [v2:172.21.15.1:6800/1162530732,v1:172.21.15.1:6801/1162530732] conn(0x7f1804038350 0x7f180403a810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:02.760 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:02.436+0000 7f181be00700 1 -- 172.21.15.1:0/3600364458 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mgrmap(e 7) v1 ==== 45162+0+0 (secure 0 0 0) 0x7f181404d9e0 con 0x7f1824106620 2024-02-02T15:19:02.760 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:02.436+0000 7f181be00700 1 --2- 172.21.15.1:0/3600364458 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7f180403bc30 0x7f180403e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:19:02.760 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:02.436+0000 7f181be00700 1 -- 172.21.15.1:0/3600364458 --> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f180403af20 con 0x7f180403bc30 2024-02-02T15:19:02.760 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:02.437+0000 7f1822a00700 1 --2- 172.21.15.1:0/3600364458 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7f180403bc30 0x7f180403e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:19:02.761 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:02.437+0000 7f1822a00700 1 --2- 172.21.15.1:0/3600364458 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7f180403bc30 0x7f180403e020 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f180c003a10 tx=0x7f180c0092b0 comp rx=0 tx=0).ready entity=mgr.14122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:19:02.761 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:02.438+0000 7f181be00700 1 -- 172.21.15.1:0/3600364458 <== mgr.14122 v2:172.21.15.1:6800/395498296 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7f180403af20 con 0x7f180403bc30 2024-02-02T15:19:02.761 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:02.446+0000 7f182a400700 1 -- 172.21.15.1:0/3600364458 --> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f182419b6c0 con 0x7f180403bc30 2024-02-02T15:19:02.761 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:02.446+0000 7f181be00700 1 -- 172.21.15.1:0/3600364458 <== mgr.14122 v2:172.21.15.1:6800/395498296 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f182419b6c0 con 0x7f180403bc30 2024-02-02T15:19:02.761 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:02.446+0000 7f182a400700 1 -- 172.21.15.1:0/3600364458 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7f180403bc30 msgr2=0x7f180403e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:19:02.761 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:02.446+0000 7f182a400700 1 --2- 172.21.15.1:0/3600364458 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7f180403bc30 0x7f180403e020 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f180c003a10 tx=0x7f180c0092b0 comp rx=0 tx=0).stop 2024-02-02T15:19:02.761 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:02.446+0000 7f182a400700 1 -- 172.21.15.1:0/3600364458 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1824106620 msgr2=0x7f182419a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:19:02.761 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:02.446+0000 7f182a400700 1 --2- 172.21.15.1:0/3600364458 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1824106620 0x7f182419a380 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f1814009130 tx=0x7f1814011ad0 comp rx=0 tx=0).stop 2024-02-02T15:19:02.761 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:02.446+0000 7f182a400700 1 -- 172.21.15.1:0/3600364458 shutdown_connections 2024-02-02T15:19:02.761 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:02.446+0000 7f182a400700 1 --2- 172.21.15.1:0/3600364458 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7f180403bc30 0x7f180403e020 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:02.761 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:02.446+0000 7f182a400700 1 --2- 172.21.15.1:0/3600364458 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1824106620 0x7f182419a380 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:02.761 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:02.447+0000 7f182a400700 1 --2- 172.21.15.1:0/3600364458 >> [v2:172.21.15.1:6800/1162530732,v1:172.21.15.1:6801/1162530732] conn(0x7f1804038350 0x7f180403a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:02.761 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:02.447+0000 7f182a400700 1 -- 172.21.15.1:0/3600364458 >> 172.21.15.1:0/3600364458 conn(0x7f1824101c20 msgr2=0x7f1824103310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:19:02.761 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:02.447+0000 7f182a400700 1 -- 172.21.15.1:0/3600364458 shutdown_connections 2024-02-02T15:19:02.761 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:02.447+0000 7f182a400700 1 -- 172.21.15.1:0/3600364458 wait complete. 2024-02-02T15:19:02.762 INFO:teuthology.orchestra.run.smithi001.stdout:mgr epoch 5 is available 2024-02-02T15:19:02.762 INFO:teuthology.orchestra.run.smithi001.stdout:Setting orchestrator backend to cephadm... 2024-02-02T15:19:03.766 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:03.166+0000 7fa9bb000700 1 Processor -- start 2024-02-02T15:19:03.766 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:03.166+0000 7fa9bb000700 1 -- start start 2024-02-02T15:19:03.766 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:03.166+0000 7fa9bb000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa9b41087e0 0x7fa9b4108c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:19:03.767 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:03.166+0000 7fa9bb000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b41091d0 con 0x7fa9b41087e0 2024-02-02T15:19:03.767 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:03.166+0000 7fa9b3e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa9b41087e0 0x7fa9b4108c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:19:03.767 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:03.167+0000 7fa9b3e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa9b41087e0 0x7fa9b4108c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46640/0 (socket says 172.21.15.1:46640) 2024-02-02T15:19:03.767 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:03.167+0000 7fa9b3e00700 1 -- 172.21.15.1:0/2307503623 learned_addr learned my addr 172.21.15.1:0/2307503623 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:19:03.767 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:03.167+0000 7fa9b3e00700 1 -- 172.21.15.1:0/2307503623 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9b41099e0 con 0x7fa9b41087e0 2024-02-02T15:19:03.767 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:03.167+0000 7fa9b3e00700 1 --2- 172.21.15.1:0/2307503623 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa9b41087e0 0x7fa9b4108c00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa9a40098a0 tx=0x7fa9a4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a7d0411c139d708 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:19:03.767 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:03.167+0000 7fa9b3400700 1 -- 172.21.15.1:0/2307503623 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa9a4004160 con 0x7fa9b41087e0 2024-02-02T15:19:03.767 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:03.167+0000 7fa9b3400700 1 -- 172.21.15.1:0/2307503623 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa9a40042c0 con 0x7fa9b41087e0 2024-02-02T15:19:03.767 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:03.168+0000 7fa9b3400700 1 -- 172.21.15.1:0/2307503623 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa9a4011a00 con 0x7fa9b41087e0 2024-02-02T15:19:03.767 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:03.168+0000 7fa9bb000700 1 -- 172.21.15.1:0/2307503623 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa9b41087e0 msgr2=0x7fa9b4108c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:19:03.767 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:03.168+0000 7fa9bb000700 1 --2- 172.21.15.1:0/2307503623 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa9b41087e0 0x7fa9b4108c00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa9a40098a0 tx=0x7fa9a4009bb0 comp rx=0 tx=0).stop 2024-02-02T15:19:03.767 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:03.168+0000 7fa9bb000700 1 -- 172.21.15.1:0/2307503623 shutdown_connections 2024-02-02T15:19:03.767 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:03.168+0000 7fa9bb000700 1 --2- 172.21.15.1:0/2307503623 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa9b41087e0 0x7fa9b4108c00 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:03.768 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:03.168+0000 7fa9bb000700 1 -- 172.21.15.1:0/2307503623 >> 172.21.15.1:0/2307503623 conn(0x7fa9b407bbd0 msgr2=0x7fa9b41064b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:19:03.768 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:03.168+0000 7fa9bb000700 1 -- 172.21.15.1:0/2307503623 shutdown_connections 2024-02-02T15:19:03.768 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:03.168+0000 7fa9bb000700 1 -- 172.21.15.1:0/2307503623 wait complete. 2024-02-02T15:19:03.768 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:03.169+0000 7fa9bb000700 1 Processor -- start 2024-02-02T15:19:03.768 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:03.170+0000 7fa9bb000700 1 -- start start 2024-02-02T15:19:03.768 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:03.170+0000 7fa9bb000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa9b41087e0 0x7fa9b419c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:19:03.768 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:03.170+0000 7fa9bb000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b419ca40 con 0x7fa9b41087e0 2024-02-02T15:19:03.768 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:03.170+0000 7fa9b3e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa9b41087e0 0x7fa9b419c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:19:03.768 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:03.170+0000 7fa9b3e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa9b41087e0 0x7fa9b419c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46652/0 (socket says 172.21.15.1:46652) 2024-02-02T15:19:03.768 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:03.170+0000 7fa9b3e00700 1 -- 172.21.15.1:0/2076763290 learned_addr learned my addr 172.21.15.1:0/2076763290 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:19:03.768 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:03.170+0000 7fa9b3e00700 1 -- 172.21.15.1:0/2076763290 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9a4009550 con 0x7fa9b41087e0 2024-02-02T15:19:03.768 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:03.171+0000 7fa9b3e00700 1 --2- 172.21.15.1:0/2076763290 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa9b41087e0 0x7fa9b419c500 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa9a4006af0 tx=0x7fa9a4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:19:03.768 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:03.171+0000 7fa9b1600700 1 -- 172.21.15.1:0/2076763290 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa9a4003a40 con 0x7fa9b41087e0 2024-02-02T15:19:03.769 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:03.171+0000 7fa9b1600700 1 -- 172.21.15.1:0/2076763290 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fa9a4003ba0 con 0x7fa9b41087e0 2024-02-02T15:19:03.769 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:03.171+0000 7fa9b1600700 1 -- 172.21.15.1:0/2076763290 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa9a401a700 con 0x7fa9b41087e0 2024-02-02T15:19:03.769 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:03.171+0000 7fa9bb000700 1 -- 172.21.15.1:0/2076763290 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9b419cc40 con 0x7fa9b41087e0 2024-02-02T15:19:03.769 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:03.171+0000 7fa9bb000700 1 -- 172.21.15.1:0/2076763290 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9b419d060 con 0x7fa9b41087e0 2024-02-02T15:19:03.769 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:03.172+0000 7fa9bb000700 1 -- 172.21.15.1:0/2076763290 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9b41965c0 con 0x7fa9b41087e0 2024-02-02T15:19:03.769 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:03.172+0000 7fa9b1600700 1 -- 172.21.15.1:0/2076763290 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 7) v1 ==== 45162+0+0 (secure 0 0 0) 0x7fa9a4024ad0 con 0x7fa9b41087e0 2024-02-02T15:19:03.769 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:03.173+0000 7fa9b1600700 1 --2- 172.21.15.1:0/2076763290 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7fa994038220 0x7fa99403a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:19:03.769 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:03.173+0000 7fa9b1600700 1 -- 172.21.15.1:0/2076763290 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fa9a404bfe0 con 0x7fa9b41087e0 2024-02-02T15:19:03.769 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:03.173+0000 7fa9abe00700 1 --2- 172.21.15.1:0/2076763290 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7fa994038220 0x7fa99403a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:19:03.769 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:03.174+0000 7fa9abe00700 1 --2- 172.21.15.1:0/2076763290 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7fa994038220 0x7fa99403a6e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa99c006fd0 tx=0x7fa99c006e40 comp rx=0 tx=0).ready entity=mgr.14122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:19:03.769 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:03.178+0000 7fa9b1600700 1 -- 172.21.15.1:0/2076763290 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7fa9a4024430 con 0x7fa9b41087e0 2024-02-02T15:19:03.769 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:03.402+0000 7fa9bb000700 1 -- 172.21.15.1:0/2076763290 --> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7fa9b4061180 con 0x7fa994038220 2024-02-02T15:19:03.769 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:03.410+0000 7fa9b1600700 1 -- 172.21.15.1:0/2076763290 <== mgr.14122 v2:172.21.15.1:6800/395498296 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fa9b4061180 con 0x7fa994038220 2024-02-02T15:19:03.769 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:03.412+0000 7fa9bb000700 1 -- 172.21.15.1:0/2076763290 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7fa994038220 msgr2=0x7fa99403a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:19:03.770 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:03.412+0000 7fa9bb000700 1 --2- 172.21.15.1:0/2076763290 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7fa994038220 0x7fa99403a6e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa99c006fd0 tx=0x7fa99c006e40 comp rx=0 tx=0).stop 2024-02-02T15:19:03.770 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:03.412+0000 7fa9bb000700 1 -- 172.21.15.1:0/2076763290 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa9b41087e0 msgr2=0x7fa9b419c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:19:03.770 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:03.412+0000 7fa9bb000700 1 --2- 172.21.15.1:0/2076763290 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa9b41087e0 0x7fa9b419c500 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa9a4006af0 tx=0x7fa9a4011ad0 comp rx=0 tx=0).stop 2024-02-02T15:19:03.770 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:03.412+0000 7fa9bb000700 1 -- 172.21.15.1:0/2076763290 shutdown_connections 2024-02-02T15:19:03.770 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:03.412+0000 7fa9bb000700 1 --2- 172.21.15.1:0/2076763290 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7fa994038220 0x7fa99403a6e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:03.770 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:03.412+0000 7fa9bb000700 1 --2- 172.21.15.1:0/2076763290 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa9b41087e0 0x7fa9b419c500 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:03.770 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:03.412+0000 7fa9bb000700 1 -- 172.21.15.1:0/2076763290 >> 172.21.15.1:0/2076763290 conn(0x7fa9b407bbd0 msgr2=0x7fa9b41067f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:19:03.770 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:03.413+0000 7fa9bb000700 1 -- 172.21.15.1:0/2076763290 shutdown_connections 2024-02-02T15:19:03.770 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:03.413+0000 7fa9bb000700 1 -- 172.21.15.1:0/2076763290 wait complete. 2024-02-02T15:19:03.961 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:03 smithi001 ceph-mon[35470]: from='client.14126 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-02-02T15:19:03.961 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:03 smithi001 ceph-mon[35470]: from='client.14126 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-02-02T15:19:03.961 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:03 smithi001 ceph-mon[35470]: from='mgr.14122 172.21.15.1:0/954154330' entity='mgr.x' 2024-02-02T15:19:03.961 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:03 smithi001 ceph-mon[35470]: from='mgr.14122 172.21.15.1:0/954154330' entity='mgr.x' 2024-02-02T15:19:03.961 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:03 smithi001 ceph-mon[35470]: from='mgr.14122 172.21.15.1:0/954154330' entity='mgr.x' 2024-02-02T15:19:03.961 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:03 smithi001 ceph-mon[35470]: from='mgr.14122 172.21.15.1:0/954154330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:19:04.713 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout value unchanged 2024-02-02T15:19:04.714 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:04.146+0000 7f4aef400700 1 Processor -- start 2024-02-02T15:19:04.714 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:04.146+0000 7f4aef400700 1 -- start start 2024-02-02T15:19:04.714 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:04.147+0000 7f4aef400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ae8106620 0x7f4ae8106a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:19:04.714 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:04.147+0000 7f4aef400700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae8107010 con 0x7f4ae8106620 2024-02-02T15:19:04.714 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:04.147+0000 7f4aeca00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ae8106620 0x7f4ae8106a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:19:04.714 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:04.147+0000 7f4aeca00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ae8106620 0x7f4ae8106a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46658/0 (socket says 172.21.15.1:46658) 2024-02-02T15:19:04.714 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:04.147+0000 7f4aeca00700 1 -- 172.21.15.1:0/2699682296 learned_addr learned my addr 172.21.15.1:0/2699682296 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:19:04.714 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:04.147+0000 7f4aeca00700 1 -- 172.21.15.1:0/2699682296 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ae8107820 con 0x7f4ae8106620 2024-02-02T15:19:04.714 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:04.148+0000 7f4aeca00700 1 --2- 172.21.15.1:0/2699682296 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ae8106620 0x7f4ae8106a40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4ad80098a0 tx=0x7f4ad8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0c0383cd459e911 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:19:04.714 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:04.148+0000 7f4ae7400700 1 -- 172.21.15.1:0/2699682296 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4ad8004160 con 0x7f4ae8106620 2024-02-02T15:19:04.714 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:04.148+0000 7f4ae7400700 1 -- 172.21.15.1:0/2699682296 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4ad80042c0 con 0x7f4ae8106620 2024-02-02T15:19:04.715 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:04.148+0000 7f4ae7400700 1 -- 172.21.15.1:0/2699682296 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4ad8011a00 con 0x7f4ae8106620 2024-02-02T15:19:04.715 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:04.149+0000 7f4aef400700 1 -- 172.21.15.1:0/2699682296 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ae8106620 msgr2=0x7f4ae8106a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:19:04.715 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:04.149+0000 7f4aef400700 1 --2- 172.21.15.1:0/2699682296 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ae8106620 0x7f4ae8106a40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4ad80098a0 tx=0x7f4ad8009bb0 comp rx=0 tx=0).stop 2024-02-02T15:19:04.715 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:04.149+0000 7f4aef400700 1 -- 172.21.15.1:0/2699682296 shutdown_connections 2024-02-02T15:19:04.715 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:04.149+0000 7f4aef400700 1 --2- 172.21.15.1:0/2699682296 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ae8106620 0x7f4ae8106a40 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:04.715 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:04.149+0000 7f4aef400700 1 -- 172.21.15.1:0/2699682296 >> 172.21.15.1:0/2699682296 conn(0x7f4ae8101c20 msgr2=0x7f4ae8104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:19:04.715 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:04.149+0000 7f4aef400700 1 -- 172.21.15.1:0/2699682296 shutdown_connections 2024-02-02T15:19:04.715 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:04.149+0000 7f4aef400700 1 -- 172.21.15.1:0/2699682296 wait complete. 2024-02-02T15:19:04.715 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:04.150+0000 7f4aef400700 1 Processor -- start 2024-02-02T15:19:04.715 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:04.150+0000 7f4aef400700 1 -- start start 2024-02-02T15:19:04.715 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:04.150+0000 7f4aef400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ae8106620 0x7f4ae807bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:19:04.715 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:04.150+0000 7f4aef400700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae807c170 con 0x7f4ae8106620 2024-02-02T15:19:04.715 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:04.151+0000 7f4aeca00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ae8106620 0x7f4ae807bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:19:04.715 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:04.151+0000 7f4aeca00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ae8106620 0x7f4ae807bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46674/0 (socket says 172.21.15.1:46674) 2024-02-02T15:19:04.716 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:04.151+0000 7f4aeca00700 1 -- 172.21.15.1:0/1630428074 learned_addr learned my addr 172.21.15.1:0/1630428074 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:19:04.716 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:04.151+0000 7f4aeca00700 1 -- 172.21.15.1:0/1630428074 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ad8009550 con 0x7f4ae8106620 2024-02-02T15:19:04.716 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:04.151+0000 7f4aeca00700 1 --2- 172.21.15.1:0/1630428074 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ae8106620 0x7f4ae807bc30 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4ad8009130 tx=0x7f4ad8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:19:04.716 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:04.151+0000 7f4ae5600700 1 -- 172.21.15.1:0/1630428074 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4ad8003ab0 con 0x7f4ae8106620 2024-02-02T15:19:04.716 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:04.152+0000 7f4ae5600700 1 -- 172.21.15.1:0/1630428074 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f4ad8003c10 con 0x7f4ae8106620 2024-02-02T15:19:04.716 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:04.152+0000 7f4aef400700 1 -- 172.21.15.1:0/1630428074 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ae807a280 con 0x7f4ae8106620 2024-02-02T15:19:04.716 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:04.152+0000 7f4ae5600700 1 -- 172.21.15.1:0/1630428074 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4ad80247d0 con 0x7f4ae8106620 2024-02-02T15:19:04.716 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:04.152+0000 7f4aef400700 1 -- 172.21.15.1:0/1630428074 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ae807a6a0 con 0x7f4ae8106620 2024-02-02T15:19:04.716 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:04.153+0000 7f4ae5600700 1 -- 172.21.15.1:0/1630428074 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 7) v1 ==== 45162+0+0 (secure 0 0 0) 0x7f4ad801a430 con 0x7f4ae8106620 2024-02-02T15:19:04.716 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:04.153+0000 7f4aef400700 1 -- 172.21.15.1:0/1630428074 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ae8062370 con 0x7f4ae8106620 2024-02-02T15:19:04.716 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:04.153+0000 7f4ae5600700 1 --2- 172.21.15.1:0/1630428074 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7f4ac8038220 0x7f4ac803a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:19:04.717 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:04.153+0000 7f4ae5600700 1 -- 172.21.15.1:0/1630428074 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f4ad804c190 con 0x7f4ae8106620 2024-02-02T15:19:04.717 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:04.153+0000 7f4ae7e00700 1 --2- 172.21.15.1:0/1630428074 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7f4ac8038220 0x7f4ac803a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:19:04.717 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:04.154+0000 7f4ae7e00700 1 --2- 172.21.15.1:0/1630428074 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7f4ac8038220 0x7f4ac803a6e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4ad0006fd0 tx=0x7f4ad0006e40 comp rx=0 tx=0).ready entity=mgr.14122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:19:04.717 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:04.158+0000 7f4ae5600700 1 -- 172.21.15.1:0/1630428074 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f4ad8018b40 con 0x7f4ae8106620 2024-02-02T15:19:04.717 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:04.370+0000 7f4aef400700 1 -- 172.21.15.1:0/1630428074 --> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f4ae8103f70 con 0x7f4ac8038220 2024-02-02T15:19:04.717 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:04.371+0000 7f4ae5600700 1 -- 172.21.15.1:0/1630428074 <== mgr.14122 v2:172.21.15.1:6800/395498296 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f4ae8103f70 con 0x7f4ac8038220 2024-02-02T15:19:04.717 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:04.373+0000 7f4aef400700 1 -- 172.21.15.1:0/1630428074 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7f4ac8038220 msgr2=0x7f4ac803a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:19:04.717 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:04.373+0000 7f4aef400700 1 --2- 172.21.15.1:0/1630428074 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7f4ac8038220 0x7f4ac803a6e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4ad0006fd0 tx=0x7f4ad0006e40 comp rx=0 tx=0).stop 2024-02-02T15:19:04.717 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:04.373+0000 7f4aef400700 1 -- 172.21.15.1:0/1630428074 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ae8106620 msgr2=0x7f4ae807bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:19:04.717 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:04.373+0000 7f4aef400700 1 --2- 172.21.15.1:0/1630428074 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ae8106620 0x7f4ae807bc30 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4ad8009130 tx=0x7f4ad8011ad0 comp rx=0 tx=0).stop 2024-02-02T15:19:04.717 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:04.373+0000 7f4aef400700 1 -- 172.21.15.1:0/1630428074 shutdown_connections 2024-02-02T15:19:04.717 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:04.373+0000 7f4aef400700 1 --2- 172.21.15.1:0/1630428074 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7f4ac8038220 0x7f4ac803a6e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:04.717 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:04.373+0000 7f4aef400700 1 --2- 172.21.15.1:0/1630428074 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ae8106620 0x7f4ae807bc30 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:04.718 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:04.373+0000 7f4aef400700 1 -- 172.21.15.1:0/1630428074 >> 172.21.15.1:0/1630428074 conn(0x7f4ae8101c20 msgr2=0x7f4ae8103310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:19:04.718 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:04.374+0000 7f4aef400700 1 -- 172.21.15.1:0/1630428074 shutdown_connections 2024-02-02T15:19:04.718 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:04.374+0000 7f4aef400700 1 -- 172.21.15.1:0/1630428074 wait complete. 2024-02-02T15:19:04.718 INFO:teuthology.orchestra.run.smithi001.stdout:Generating ssh key... 2024-02-02T15:19:05.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:04 smithi001 ceph-mon[35470]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:19:05.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:04 smithi001 ceph-mon[35470]: from='mgr.14122 172.21.15.1:0/954154330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:19:05.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:04 smithi001 ceph-mon[35470]: mgrmap e8: x(active, since 2s) 2024-02-02T15:19:05.873 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:05.103+0000 7f1c9d800700 1 Processor -- start 2024-02-02T15:19:05.873 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:05.103+0000 7f1c9d800700 1 -- start start 2024-02-02T15:19:05.874 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:05.104+0000 7f1c9d800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c98106620 0x7f1c98106a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:19:05.874 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:05.104+0000 7f1c9d800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c98107010 con 0x7f1c98106620 2024-02-02T15:19:05.874 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:05.104+0000 7f1c96a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c98106620 0x7f1c98106a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:19:05.874 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:05.104+0000 7f1c96a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c98106620 0x7f1c98106a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46676/0 (socket says 172.21.15.1:46676) 2024-02-02T15:19:05.874 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:05.104+0000 7f1c96a00700 1 -- 172.21.15.1:0/3477193395 learned_addr learned my addr 172.21.15.1:0/3477193395 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:19:05.874 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:05.104+0000 7f1c96a00700 1 -- 172.21.15.1:0/3477193395 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c98107820 con 0x7f1c98106620 2024-02-02T15:19:05.874 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:05.105+0000 7f1c96a00700 1 --2- 172.21.15.1:0/3477193395 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c98106620 0x7f1c98106a40 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1c880098a0 tx=0x7f1c88009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=810e7fd7b9a11b4 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:19:05.874 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:05 smithi001 ceph-mon[35470]: [02/Feb/2024:15:19:03] ENGINE Bus STARTING 2024-02-02T15:19:05.874 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:05 smithi001 ceph-mon[35470]: [02/Feb/2024:15:19:04] ENGINE Serving on https://172.21.15.1:7150 2024-02-02T15:19:05.875 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:05 smithi001 ceph-mon[35470]: [02/Feb/2024:15:19:04] ENGINE Serving on http://172.21.15.1:8765 2024-02-02T15:19:05.875 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:05 smithi001 ceph-mon[35470]: [02/Feb/2024:15:19:04] ENGINE Bus STARTED 2024-02-02T15:19:05.875 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:05 smithi001 ceph-mon[35470]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:19:05.875 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:05 smithi001 ceph-mon[35470]: from='mgr.14122 172.21.15.1:0/954154330' entity='mgr.x' 2024-02-02T15:19:05.875 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:05 smithi001 ceph-mon[35470]: from='mgr.14122 172.21.15.1:0/954154330' entity='mgr.x' 2024-02-02T15:19:05.875 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:05.105+0000 7f1c96000700 1 -- 172.21.15.1:0/3477193395 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1c88004160 con 0x7f1c98106620 2024-02-02T15:19:05.875 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:05.105+0000 7f1c96000700 1 -- 172.21.15.1:0/3477193395 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1c880042c0 con 0x7f1c98106620 2024-02-02T15:19:05.876 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:19:05 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: Generating public/private rsa key pair. 2024-02-02T15:19:05.876 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:19:05 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: Your identification has been saved in /tmp/tmp2cb8yd55/key. 2024-02-02T15:19:05.876 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:19:05 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: Your public key has been saved in /tmp/tmp2cb8yd55/key.pub. 2024-02-02T15:19:05.876 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:19:05 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: The key fingerprint is: 2024-02-02T15:19:05.876 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:19:05 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: SHA256:IPoBK3p38vQjt7/rodGCV6k6wC1dwLFK9BtKZ7+HCdc ceph-371a409c-c1de-11ee-95b6-87774f69a715 2024-02-02T15:19:05.876 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:19:05 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: The key's randomart image is: 2024-02-02T15:19:05.876 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:19:05 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: +---[RSA 3072]----+ 2024-02-02T15:19:05.876 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:19:05 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: | .... | 2024-02-02T15:19:05.876 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:19:05 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: | . .o. | 2024-02-02T15:19:05.876 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:19:05 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: | . + B. | 2024-02-02T15:19:05.876 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:19:05 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: | * * =.. . | 2024-02-02T15:19:05.877 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:19:05 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: |. o.+oo.S E | 2024-02-02T15:19:05.877 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:19:05 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: |.. .+.o+ B | 2024-02-02T15:19:05.877 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:19:05 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: |. . +o+ O + | 2024-02-02T15:19:05.877 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:19:05 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: | . . =o+o= . | 2024-02-02T15:19:05.877 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:19:05 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: | o++==o | 2024-02-02T15:19:05.877 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:19:05 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: +----[SHA256]-----+ 2024-02-02T15:19:05.882 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:05.105+0000 7f1c96000700 1 -- 172.21.15.1:0/3477193395 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1c88011a00 con 0x7f1c98106620 2024-02-02T15:19:05.882 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:05.106+0000 7f1c9d800700 1 -- 172.21.15.1:0/3477193395 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c98106620 msgr2=0x7f1c98106a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:19:05.882 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:05.106+0000 7f1c9d800700 1 --2- 172.21.15.1:0/3477193395 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c98106620 0x7f1c98106a40 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1c880098a0 tx=0x7f1c88009bb0 comp rx=0 tx=0).stop 2024-02-02T15:19:05.882 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:05.106+0000 7f1c9d800700 1 -- 172.21.15.1:0/3477193395 shutdown_connections 2024-02-02T15:19:05.882 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:05.106+0000 7f1c9d800700 1 --2- 172.21.15.1:0/3477193395 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c98106620 0x7f1c98106a40 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:05.882 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:05.106+0000 7f1c9d800700 1 -- 172.21.15.1:0/3477193395 >> 172.21.15.1:0/3477193395 conn(0x7f1c98101c20 msgr2=0x7f1c98104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:19:05.882 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:05.106+0000 7f1c9d800700 1 -- 172.21.15.1:0/3477193395 shutdown_connections 2024-02-02T15:19:05.882 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:05.106+0000 7f1c9d800700 1 -- 172.21.15.1:0/3477193395 wait complete. 2024-02-02T15:19:05.882 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:05.107+0000 7f1c9d800700 1 Processor -- start 2024-02-02T15:19:05.882 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:05.107+0000 7f1c9d800700 1 -- start start 2024-02-02T15:19:05.882 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:05.107+0000 7f1c9d800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c98106620 0x7f1c9819a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:19:05.882 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:05.107+0000 7f1c9d800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9819a8c0 con 0x7f1c98106620 2024-02-02T15:19:05.882 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:05.107+0000 7f1c96a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c98106620 0x7f1c9819a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:19:05.883 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:05.107+0000 7f1c96a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c98106620 0x7f1c9819a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46678/0 (socket says 172.21.15.1:46678) 2024-02-02T15:19:05.883 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:05.107+0000 7f1c96a00700 1 -- 172.21.15.1:0/3212963187 learned_addr learned my addr 172.21.15.1:0/3212963187 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:19:05.883 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:05.108+0000 7f1c96a00700 1 -- 172.21.15.1:0/3212963187 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c88009550 con 0x7f1c98106620 2024-02-02T15:19:05.883 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:05.108+0000 7f1c96a00700 1 --2- 172.21.15.1:0/3212963187 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c98106620 0x7f1c9819a380 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1c88009130 tx=0x7f1c88011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:19:05.883 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:05.108+0000 7f1c8f400700 1 -- 172.21.15.1:0/3212963187 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1c88003ab0 con 0x7f1c98106620 2024-02-02T15:19:05.883 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:05.108+0000 7f1c8f400700 1 -- 172.21.15.1:0/3212963187 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1c88003c10 con 0x7f1c98106620 2024-02-02T15:19:05.883 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:05.108+0000 7f1c8f400700 1 -- 172.21.15.1:0/3212963187 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1c880247d0 con 0x7f1c98106620 2024-02-02T15:19:05.884 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:05.108+0000 7f1c9d800700 1 -- 172.21.15.1:0/3212963187 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c9819aac0 con 0x7f1c98106620 2024-02-02T15:19:05.884 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:05.108+0000 7f1c9d800700 1 -- 172.21.15.1:0/3212963187 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c9819aee0 con 0x7f1c98106620 2024-02-02T15:19:05.884 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:05.109+0000 7f1c8f400700 1 -- 172.21.15.1:0/3212963187 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 8) v1 ==== 45268+0+0 (secure 0 0 0) 0x7f1c8801abb0 con 0x7f1c98106620 2024-02-02T15:19:05.884 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:05.109+0000 7f1c9d800700 1 -- 172.21.15.1:0/3212963187 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c98062370 con 0x7f1c98106620 2024-02-02T15:19:05.884 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:05.110+0000 7f1c8f400700 1 --2- 172.21.15.1:0/3212963187 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7f1c78038350 0x7f1c7803a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:19:05.884 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:05.110+0000 7f1c8f400700 1 -- 172.21.15.1:0/3212963187 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f1c8804c370 con 0x7f1c98106620 2024-02-02T15:19:05.884 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:05.110+0000 7f1c8fe00700 1 --2- 172.21.15.1:0/3212963187 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7f1c78038350 0x7f1c7803a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:19:05.885 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:05.111+0000 7f1c8fe00700 1 --2- 172.21.15.1:0/3212963187 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7f1c78038350 0x7f1c7803a810 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1c80006fd0 tx=0x7f1c80006e40 comp rx=0 tx=0).ready entity=mgr.14122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:19:05.885 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:05.115+0000 7f1c8f400700 1 -- 172.21.15.1:0/3212963187 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f1c88018530 con 0x7f1c98106620 2024-02-02T15:19:05.885 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:05.324+0000 7f1c9d800700 1 -- 172.21.15.1:0/3212963187 --> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f1c9819dd40 con 0x7f1c78038350 2024-02-02T15:19:05.885 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:05.553+0000 7f1c8f400700 1 -- 172.21.15.1:0/3212963187 <== mgr.14122 v2:172.21.15.1:6800/395498296 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f1c9819dd40 con 0x7f1c78038350 2024-02-02T15:19:05.885 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:05.556+0000 7f1c9d800700 1 -- 172.21.15.1:0/3212963187 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7f1c78038350 msgr2=0x7f1c7803a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:19:05.885 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:05.556+0000 7f1c9d800700 1 --2- 172.21.15.1:0/3212963187 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7f1c78038350 0x7f1c7803a810 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1c80006fd0 tx=0x7f1c80006e40 comp rx=0 tx=0).stop 2024-02-02T15:19:05.885 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:05.556+0000 7f1c9d800700 1 -- 172.21.15.1:0/3212963187 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c98106620 msgr2=0x7f1c9819a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:19:05.886 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:05.556+0000 7f1c9d800700 1 --2- 172.21.15.1:0/3212963187 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c98106620 0x7f1c9819a380 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1c88009130 tx=0x7f1c88011ad0 comp rx=0 tx=0).stop 2024-02-02T15:19:05.886 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:05.556+0000 7f1c9d800700 1 -- 172.21.15.1:0/3212963187 shutdown_connections 2024-02-02T15:19:05.886 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:05.556+0000 7f1c9d800700 1 --2- 172.21.15.1:0/3212963187 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7f1c78038350 0x7f1c7803a810 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:05.886 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:05.556+0000 7f1c9d800700 1 --2- 172.21.15.1:0/3212963187 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1c98106620 0x7f1c9819a380 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:05.886 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:05.556+0000 7f1c9d800700 1 -- 172.21.15.1:0/3212963187 >> 172.21.15.1:0/3212963187 conn(0x7f1c98101c20 msgr2=0x7f1c98103310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:19:05.886 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:05.556+0000 7f1c9d800700 1 -- 172.21.15.1:0/3212963187 shutdown_connections 2024-02-02T15:19:05.887 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:05.556+0000 7f1c9d800700 1 -- 172.21.15.1:0/3212963187 wait complete. 2024-02-02T15:19:06.948 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:06 smithi001 ceph-mon[35470]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:19:06.948 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:06 smithi001 ceph-mon[35470]: Generating ssh key... 2024-02-02T15:19:07.083 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDE1d+fQVaW2LJFp4HGw10d1r9j3ZuxDZ/8uAf0tV2ReKkI/+KvK1Y03ELM8Ho6f1S53Du8SOhwYiK+TzDXIkN8IYb3nyisiarqb625CXWDfn/piNfEPVyEU90lbXnHnbCRYyqK4FsKpzSqjQyvh625tIwJ4TXPRJ3oqciYDEv+7P7oWUGvEwtUWGv1prmhfy0AQ6rg4fzBFRZZZGXRA+1xQ/sZ69+rdtWTXH1d13RqLzhKcMhBQFk7sIC9g/57fwrXhPNL3mS7d6G92B1m7eig8+Xod3fF43a2g18CJwA6kM1/gbAbtfCnesbsbXQLTl6MSDOxis+ZmUeq3mMdJ+LQ8Y37/SZPnPPQbSEOU1Ehzjp8zn9jpmE1BM2MwuunfR90yh3e3o4VCEljXI59cccXx03Ro1h8nby4HRafSm8IQGOT801mDdy2ica98QSzX9b7o6IVHZS17Q8wQdwRDT6MlLhf1jROaynneW58a7em8rlAA2huRAxIE6kbNfaag9s= ceph-371a409c-c1de-11ee-95b6-87774f69a715 2024-02-02T15:19:07.084 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:06.511+0000 7f3b3aa00700 1 Processor -- start 2024-02-02T15:19:07.084 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:06.511+0000 7f3b3aa00700 1 -- start start 2024-02-02T15:19:07.084 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:06.512+0000 7f3b3aa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3b3407be60 0x7f3b3407a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:19:07.084 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:06.512+0000 7f3b3aa00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b3407a890 con 0x7f3b3407be60 2024-02-02T15:19:07.084 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:06.512+0000 7f3b39600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3b3407be60 0x7f3b3407a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:19:07.084 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:06.512+0000 7f3b39600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3b3407be60 0x7f3b3407a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46682/0 (socket says 172.21.15.1:46682) 2024-02-02T15:19:07.084 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:06.512+0000 7f3b39600700 1 -- 172.21.15.1:0/808012736 learned_addr learned my addr 172.21.15.1:0/808012736 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:19:07.084 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:06.512+0000 7f3b39600700 1 -- 172.21.15.1:0/808012736 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b3407a9d0 con 0x7f3b3407be60 2024-02-02T15:19:07.084 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:06.512+0000 7f3b39600700 1 --2- 172.21.15.1:0/808012736 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3b3407be60 0x7f3b3407a2c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3b240098a0 tx=0x7f3b24009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae1cd9c297c83982 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:19:07.084 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:06.513+0000 7f3b33e00700 1 -- 172.21.15.1:0/808012736 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3b24004160 con 0x7f3b3407be60 2024-02-02T15:19:07.084 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:06.513+0000 7f3b33e00700 1 -- 172.21.15.1:0/808012736 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3b240042c0 con 0x7f3b3407be60 2024-02-02T15:19:07.084 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:06.513+0000 7f3b33e00700 1 -- 172.21.15.1:0/808012736 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3b24011a00 con 0x7f3b3407be60 2024-02-02T15:19:07.085 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:06.513+0000 7f3b3aa00700 1 -- 172.21.15.1:0/808012736 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3b3407be60 msgr2=0x7f3b3407a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:19:07.085 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:06.513+0000 7f3b3aa00700 1 --2- 172.21.15.1:0/808012736 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3b3407be60 0x7f3b3407a2c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3b240098a0 tx=0x7f3b24009bb0 comp rx=0 tx=0).stop 2024-02-02T15:19:07.085 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:06.513+0000 7f3b3aa00700 1 -- 172.21.15.1:0/808012736 shutdown_connections 2024-02-02T15:19:07.085 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:06.513+0000 7f3b3aa00700 1 --2- 172.21.15.1:0/808012736 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3b3407be60 0x7f3b3407a2c0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:07.085 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:06.513+0000 7f3b3aa00700 1 -- 172.21.15.1:0/808012736 >> 172.21.15.1:0/808012736 conn(0x7f3b341019d0 msgr2=0x7f3b34103df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:19:07.085 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:06.514+0000 7f3b3aa00700 1 -- 172.21.15.1:0/808012736 shutdown_connections 2024-02-02T15:19:07.085 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:06.514+0000 7f3b3aa00700 1 -- 172.21.15.1:0/808012736 wait complete. 2024-02-02T15:19:07.085 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:06.514+0000 7f3b3aa00700 1 Processor -- start 2024-02-02T15:19:07.085 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:06.515+0000 7f3b3aa00700 1 -- start start 2024-02-02T15:19:07.085 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:06.515+0000 7f3b3aa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3b3407be60 0x7f3b3419c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:19:07.085 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:06.515+0000 7f3b3aa00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b3419c850 con 0x7f3b3407be60 2024-02-02T15:19:07.085 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:06.515+0000 7f3b39600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3b3407be60 0x7f3b3419c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:19:07.085 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:06.515+0000 7f3b39600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3b3407be60 0x7f3b3419c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46694/0 (socket says 172.21.15.1:46694) 2024-02-02T15:19:07.086 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:06.515+0000 7f3b39600700 1 -- 172.21.15.1:0/186903970 learned_addr learned my addr 172.21.15.1:0/186903970 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:19:07.086 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:06.515+0000 7f3b39600700 1 -- 172.21.15.1:0/186903970 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b24009550 con 0x7f3b3407be60 2024-02-02T15:19:07.086 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:06.516+0000 7f3b39600700 1 --2- 172.21.15.1:0/186903970 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3b3407be60 0x7f3b3419c310 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3b24006af0 tx=0x7f3b24011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:19:07.086 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:06.516+0000 7f3b32000700 1 -- 172.21.15.1:0/186903970 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3b24003a40 con 0x7f3b3407be60 2024-02-02T15:19:07.086 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:06.516+0000 7f3b32000700 1 -- 172.21.15.1:0/186903970 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3b24003ba0 con 0x7f3b3407be60 2024-02-02T15:19:07.086 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:06.516+0000 7f3b3aa00700 1 -- 172.21.15.1:0/186903970 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b3419ca50 con 0x7f3b3407be60 2024-02-02T15:19:07.086 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:06.516+0000 7f3b32000700 1 -- 172.21.15.1:0/186903970 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3b2401a6e0 con 0x7f3b3407be60 2024-02-02T15:19:07.086 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:06.516+0000 7f3b3aa00700 1 -- 172.21.15.1:0/186903970 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b3419ce70 con 0x7f3b3407be60 2024-02-02T15:19:07.086 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:06.517+0000 7f3b32000700 1 -- 172.21.15.1:0/186903970 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 8) v1 ==== 45268+0+0 (secure 0 0 0) 0x7f3b24024a60 con 0x7f3b3407be60 2024-02-02T15:19:07.086 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:06.517+0000 7f3b3aa00700 1 -- 172.21.15.1:0/186903970 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b34062370 con 0x7f3b3407be60 2024-02-02T15:19:07.086 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:06.518+0000 7f3b32000700 1 --2- 172.21.15.1:0/186903970 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7f3b14038350 0x7f3b1403a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:19:07.086 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:06.518+0000 7f3b32000700 1 -- 172.21.15.1:0/186903970 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f3b2404c260 con 0x7f3b3407be60 2024-02-02T15:19:07.086 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:06.518+0000 7f3b38c00700 1 --2- 172.21.15.1:0/186903970 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7f3b14038350 0x7f3b1403a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:19:07.086 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:06.518+0000 7f3b38c00700 1 --2- 172.21.15.1:0/186903970 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7f3b14038350 0x7f3b1403a810 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c006fd0 tx=0x7f3b1c006e40 comp rx=0 tx=0).ready entity=mgr.14122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:19:07.087 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:06.523+0000 7f3b32000700 1 -- 172.21.15.1:0/186903970 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f3b24024430 con 0x7f3b3407be60 2024-02-02T15:19:07.087 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:06.731+0000 7f3b3aa00700 1 -- 172.21.15.1:0/186903970 --> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f3b34103d30 con 0x7f3b14038350 2024-02-02T15:19:07.087 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:06.731+0000 7f3b32000700 1 -- 172.21.15.1:0/186903970 <== mgr.14122 v2:172.21.15.1:6800/395498296 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f3b34103d30 con 0x7f3b14038350 2024-02-02T15:19:07.087 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:06.734+0000 7f3b3aa00700 1 -- 172.21.15.1:0/186903970 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7f3b14038350 msgr2=0x7f3b1403a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:19:07.087 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:06.734+0000 7f3b3aa00700 1 --2- 172.21.15.1:0/186903970 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7f3b14038350 0x7f3b1403a810 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c006fd0 tx=0x7f3b1c006e40 comp rx=0 tx=0).stop 2024-02-02T15:19:07.087 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:06.734+0000 7f3b3aa00700 1 -- 172.21.15.1:0/186903970 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3b3407be60 msgr2=0x7f3b3419c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:19:07.087 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:06.734+0000 7f3b3aa00700 1 --2- 172.21.15.1:0/186903970 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3b3407be60 0x7f3b3419c310 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3b24006af0 tx=0x7f3b24011ad0 comp rx=0 tx=0).stop 2024-02-02T15:19:07.087 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:06.734+0000 7f3b3aa00700 1 -- 172.21.15.1:0/186903970 shutdown_connections 2024-02-02T15:19:07.087 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:06.734+0000 7f3b3aa00700 1 --2- 172.21.15.1:0/186903970 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7f3b14038350 0x7f3b1403a810 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:07.087 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:06.734+0000 7f3b3aa00700 1 --2- 172.21.15.1:0/186903970 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3b3407be60 0x7f3b3419c310 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:07.087 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:06.734+0000 7f3b3aa00700 1 -- 172.21.15.1:0/186903970 >> 172.21.15.1:0/186903970 conn(0x7f3b341019d0 msgr2=0x7f3b34103110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:19:07.087 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:06.734+0000 7f3b3aa00700 1 -- 172.21.15.1:0/186903970 shutdown_connections 2024-02-02T15:19:07.087 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:06.734+0000 7f3b3aa00700 1 -- 172.21.15.1:0/186903970 wait complete. 2024-02-02T15:19:07.087 INFO:teuthology.orchestra.run.smithi001.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-02-02T15:19:07.087 INFO:teuthology.orchestra.run.smithi001.stdout:Adding key to root@localhost authorized_keys... 2024-02-02T15:19:07.088 INFO:teuthology.orchestra.run.smithi001.stdout:Adding host smithi001... 2024-02-02T15:19:08.092 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:07 smithi001 ceph-mon[35470]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:19:08.983 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:08 smithi001 ceph-mon[35470]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi001", "addr": "172.21.15.1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:19:10.041 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:09 smithi001 ceph-mon[35470]: Deploying cephadm binary to smithi001 2024-02-02T15:19:11.104 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout Added host 'smithi001' with addr '172.21.15.1' 2024-02-02T15:19:11.104 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:07.467+0000 7fdce7200700 1 Processor -- start 2024-02-02T15:19:11.104 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:07.467+0000 7fdce7200700 1 -- start start 2024-02-02T15:19:11.104 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:07.467+0000 7fdce7200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdce0106410 0x7fdce0106830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:19:11.104 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:07.467+0000 7fdce7200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdce0106e00 con 0x7fdce0106410 2024-02-02T15:19:11.104 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:07.468+0000 7fdce5e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdce0106410 0x7fdce0106830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:19:11.104 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:07.468+0000 7fdce5e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdce0106410 0x7fdce0106830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46696/0 (socket says 172.21.15.1:46696) 2024-02-02T15:19:11.105 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:07.468+0000 7fdce5e00700 1 -- 172.21.15.1:0/1131236592 learned_addr learned my addr 172.21.15.1:0/1131236592 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:19:11.105 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:07.468+0000 7fdce5e00700 1 -- 172.21.15.1:0/1131236592 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdce0107610 con 0x7fdce0106410 2024-02-02T15:19:11.105 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:07.468+0000 7fdce5e00700 1 --2- 172.21.15.1:0/1131236592 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdce0106410 0x7fdce0106830 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fdcd00098a0 tx=0x7fdcd0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e48f905a8861c63e server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:19:11.105 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:07.469+0000 7fdce4a00700 1 -- 172.21.15.1:0/1131236592 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdcd0004160 con 0x7fdce0106410 2024-02-02T15:19:11.105 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:07.469+0000 7fdce4a00700 1 -- 172.21.15.1:0/1131236592 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fdcd00042c0 con 0x7fdce0106410 2024-02-02T15:19:11.105 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:07.469+0000 7fdce4a00700 1 -- 172.21.15.1:0/1131236592 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdcd0011a00 con 0x7fdce0106410 2024-02-02T15:19:11.105 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:07.469+0000 7fdce7200700 1 -- 172.21.15.1:0/1131236592 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdce0106410 msgr2=0x7fdce0106830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:19:11.105 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:07.469+0000 7fdce7200700 1 --2- 172.21.15.1:0/1131236592 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdce0106410 0x7fdce0106830 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fdcd00098a0 tx=0x7fdcd0009bb0 comp rx=0 tx=0).stop 2024-02-02T15:19:11.105 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:07.469+0000 7fdce7200700 1 -- 172.21.15.1:0/1131236592 shutdown_connections 2024-02-02T15:19:11.105 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:07.469+0000 7fdce7200700 1 --2- 172.21.15.1:0/1131236592 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdce0106410 0x7fdce0106830 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:11.105 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:07.469+0000 7fdce7200700 1 -- 172.21.15.1:0/1131236592 >> 172.21.15.1:0/1131236592 conn(0x7fdce01019d0 msgr2=0x7fdce0103df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:19:11.105 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:07.469+0000 7fdce7200700 1 -- 172.21.15.1:0/1131236592 shutdown_connections 2024-02-02T15:19:11.105 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:07.470+0000 7fdce7200700 1 -- 172.21.15.1:0/1131236592 wait complete. 2024-02-02T15:19:11.105 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:07.470+0000 7fdce7200700 1 Processor -- start 2024-02-02T15:19:11.106 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:07.471+0000 7fdce7200700 1 -- start start 2024-02-02T15:19:11.106 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:07.471+0000 7fdce7200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdce0106410 0x7fdce019c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:19:11.106 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:07.471+0000 7fdce7200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdce019c890 con 0x7fdce0106410 2024-02-02T15:19:11.106 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:07.471+0000 7fdce5e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdce0106410 0x7fdce019c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:19:11.106 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:07.471+0000 7fdce5e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdce0106410 0x7fdce019c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46702/0 (socket says 172.21.15.1:46702) 2024-02-02T15:19:11.106 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:07.471+0000 7fdce5e00700 1 -- 172.21.15.1:0/4268958365 learned_addr learned my addr 172.21.15.1:0/4268958365 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:19:11.106 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:07.472+0000 7fdce5e00700 1 -- 172.21.15.1:0/4268958365 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcd0009550 con 0x7fdce0106410 2024-02-02T15:19:11.106 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:07.472+0000 7fdce5e00700 1 --2- 172.21.15.1:0/4268958365 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdce0106410 0x7fdce019c350 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0006af0 tx=0x7fdcd0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:19:11.106 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:07.472+0000 7fdcdea00700 1 -- 172.21.15.1:0/4268958365 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdcd0003a40 con 0x7fdce0106410 2024-02-02T15:19:11.106 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:07.472+0000 7fdce7200700 1 -- 172.21.15.1:0/4268958365 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdce019ca90 con 0x7fdce0106410 2024-02-02T15:19:11.106 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:07.472+0000 7fdcdea00700 1 -- 172.21.15.1:0/4268958365 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fdcd0003ba0 con 0x7fdce0106410 2024-02-02T15:19:11.106 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:07.472+0000 7fdcdea00700 1 -- 172.21.15.1:0/4268958365 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdcd001a700 con 0x7fdce0106410 2024-02-02T15:19:11.106 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:07.472+0000 7fdce7200700 1 -- 172.21.15.1:0/4268958365 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdce019ceb0 con 0x7fdce0106410 2024-02-02T15:19:11.106 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:07.473+0000 7fdcdea00700 1 -- 172.21.15.1:0/4268958365 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 8) v1 ==== 45268+0+0 (secure 0 0 0) 0x7fdcd0024ad0 con 0x7fdce0106410 2024-02-02T15:19:11.107 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:07.473+0000 7fdce7200700 1 -- 172.21.15.1:0/4268958365 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdce0062370 con 0x7fdce0106410 2024-02-02T15:19:11.107 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:07.474+0000 7fdcdea00700 1 --2- 172.21.15.1:0/4268958365 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7fdcc0038350 0x7fdcc003a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:19:11.107 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:07.474+0000 7fdcdea00700 1 -- 172.21.15.1:0/4268958365 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fdcd004c200 con 0x7fdce0106410 2024-02-02T15:19:11.107 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:07.474+0000 7fdce5400700 1 --2- 172.21.15.1:0/4268958365 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7fdcc0038350 0x7fdcc003a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:19:11.107 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:07.474+0000 7fdce5400700 1 --2- 172.21.15.1:0/4268958365 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7fdcc0038350 0x7fdcc003a810 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fdcc8006fd0 tx=0x7fdcc8006e40 comp rx=0 tx=0).ready entity=mgr.14122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:19:11.107 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:07.479+0000 7fdcdea00700 1 -- 172.21.15.1:0/4268958365 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7fdcd0018b80 con 0x7fdce0106410 2024-02-02T15:19:11.107 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:07.693+0000 7fdce7200700 1 -- 172.21.15.1:0/4268958365 --> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi001", "addr": "172.21.15.1", "target": ["mon-mgr", ""]}) v1 -- 0x7fdce0103d10 con 0x7fdcc0038350 2024-02-02T15:19:11.107 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:10.747+0000 7fdcdea00700 1 -- 172.21.15.1:0/4268958365 <== mgr.14122 v2:172.21.15.1:6800/395498296 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7fdce0103d10 con 0x7fdcc0038350 2024-02-02T15:19:11.107 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:10.750+0000 7fdce7200700 1 -- 172.21.15.1:0/4268958365 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7fdcc0038350 msgr2=0x7fdcc003a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:19:11.107 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:10.750+0000 7fdce7200700 1 --2- 172.21.15.1:0/4268958365 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7fdcc0038350 0x7fdcc003a810 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fdcc8006fd0 tx=0x7fdcc8006e40 comp rx=0 tx=0).stop 2024-02-02T15:19:11.107 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:10.750+0000 7fdce7200700 1 -- 172.21.15.1:0/4268958365 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdce0106410 msgr2=0x7fdce019c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:19:11.107 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:10.750+0000 7fdce7200700 1 --2- 172.21.15.1:0/4268958365 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdce0106410 0x7fdce019c350 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fdcd0006af0 tx=0x7fdcd0011ad0 comp rx=0 tx=0).stop 2024-02-02T15:19:11.107 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:10.750+0000 7fdce7200700 1 -- 172.21.15.1:0/4268958365 shutdown_connections 2024-02-02T15:19:11.107 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:10.750+0000 7fdce7200700 1 --2- 172.21.15.1:0/4268958365 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7fdcc0038350 0x7fdcc003a810 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:11.108 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:10.750+0000 7fdce7200700 1 --2- 172.21.15.1:0/4268958365 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdce0106410 0x7fdce019c350 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:11.108 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:10.750+0000 7fdce7200700 1 -- 172.21.15.1:0/4268958365 >> 172.21.15.1:0/4268958365 conn(0x7fdce01019d0 msgr2=0x7fdce01030f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:19:11.108 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:10.750+0000 7fdce7200700 1 -- 172.21.15.1:0/4268958365 shutdown_connections 2024-02-02T15:19:11.108 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:10.750+0000 7fdce7200700 1 -- 172.21.15.1:0/4268958365 wait complete. 2024-02-02T15:19:11.108 INFO:teuthology.orchestra.run.smithi001.stdout:Deploying unmanaged mon service... 2024-02-02T15:19:11.938 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:11 smithi001 ceph-mon[35470]: from='mgr.14122 172.21.15.1:0/954154330' entity='mgr.x' 2024-02-02T15:19:11.939 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:11 smithi001 ceph-mon[35470]: Added host smithi001 2024-02-02T15:19:11.939 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:11 smithi001 ceph-mon[35470]: from='mgr.14122 172.21.15.1:0/954154330' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:19:12.301 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-02-02T15:19:12.301 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:11.535+0000 7f3340a00700 1 Processor -- start 2024-02-02T15:19:12.301 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:11.535+0000 7f3340a00700 1 -- start start 2024-02-02T15:19:12.301 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:11.536+0000 7f3340a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f333807be60 0x7f333807a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:19:12.302 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:11.536+0000 7f3340a00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f333807a890 con 0x7f333807be60 2024-02-02T15:19:12.302 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:11.536+0000 7f333e000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f333807be60 0x7f333807a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:19:12.302 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:11.536+0000 7f333e000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f333807be60 0x7f333807a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46708/0 (socket says 172.21.15.1:46708) 2024-02-02T15:19:12.302 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:11.536+0000 7f333e000700 1 -- 172.21.15.1:0/1419515876 learned_addr learned my addr 172.21.15.1:0/1419515876 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:19:12.302 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:11.537+0000 7f333e000700 1 -- 172.21.15.1:0/1419515876 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f333807a9d0 con 0x7f333807be60 2024-02-02T15:19:12.302 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:11.537+0000 7f333e000700 1 --2- 172.21.15.1:0/1419515876 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f333807be60 0x7f333807a2c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f33280098a0 tx=0x7f3328009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9827685ba5cda636 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:19:12.302 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:11.537+0000 7f333cc00700 1 -- 172.21.15.1:0/1419515876 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3328004160 con 0x7f333807be60 2024-02-02T15:19:12.302 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:11.537+0000 7f333cc00700 1 -- 172.21.15.1:0/1419515876 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f33280042c0 con 0x7f333807be60 2024-02-02T15:19:12.303 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:11.537+0000 7f333cc00700 1 -- 172.21.15.1:0/1419515876 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3328011a00 con 0x7f333807be60 2024-02-02T15:19:12.303 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:11.538+0000 7f3340a00700 1 -- 172.21.15.1:0/1419515876 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f333807be60 msgr2=0x7f333807a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:19:12.303 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:11.538+0000 7f3340a00700 1 --2- 172.21.15.1:0/1419515876 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f333807be60 0x7f333807a2c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f33280098a0 tx=0x7f3328009bb0 comp rx=0 tx=0).stop 2024-02-02T15:19:12.303 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:11.538+0000 7f3340a00700 1 -- 172.21.15.1:0/1419515876 shutdown_connections 2024-02-02T15:19:12.303 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:11.538+0000 7f3340a00700 1 --2- 172.21.15.1:0/1419515876 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f333807be60 0x7f333807a2c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:12.303 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:11.538+0000 7f3340a00700 1 -- 172.21.15.1:0/1419515876 >> 172.21.15.1:0/1419515876 conn(0x7f3338101c20 msgr2=0x7f3338104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:19:12.304 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:11.538+0000 7f3340a00700 1 -- 172.21.15.1:0/1419515876 shutdown_connections 2024-02-02T15:19:12.304 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:11.538+0000 7f3340a00700 1 -- 172.21.15.1:0/1419515876 wait complete. 2024-02-02T15:19:12.304 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:11.539+0000 7f3340a00700 1 Processor -- start 2024-02-02T15:19:12.304 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:11.539+0000 7f3340a00700 1 -- start start 2024-02-02T15:19:12.304 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:11.540+0000 7f3340a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f333807be60 0x7f333819c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:19:12.304 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:11.540+0000 7f3340a00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f333819ca80 con 0x7f333807be60 2024-02-02T15:19:12.304 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:11.540+0000 7f333e000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f333807be60 0x7f333819c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:19:12.304 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:11.540+0000 7f333e000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f333807be60 0x7f333819c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46716/0 (socket says 172.21.15.1:46716) 2024-02-02T15:19:12.304 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:11.540+0000 7f333e000700 1 -- 172.21.15.1:0/2770021151 learned_addr learned my addr 172.21.15.1:0/2770021151 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:19:12.304 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:11.540+0000 7f333e000700 1 -- 172.21.15.1:0/2770021151 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3328009550 con 0x7f333807be60 2024-02-02T15:19:12.304 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:11.541+0000 7f333e000700 1 --2- 172.21.15.1:0/2770021151 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f333807be60 0x7f333819c540 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3328009130 tx=0x7f3328011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:19:12.304 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:11.541+0000 7f3336a00700 1 -- 172.21.15.1:0/2770021151 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3328003b70 con 0x7f333807be60 2024-02-02T15:19:12.305 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:11.541+0000 7f3336a00700 1 -- 172.21.15.1:0/2770021151 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f3328003cd0 con 0x7f333807be60 2024-02-02T15:19:12.305 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:11.541+0000 7f3340a00700 1 -- 172.21.15.1:0/2770021151 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f333819cc80 con 0x7f333807be60 2024-02-02T15:19:12.305 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:11.541+0000 7f3336a00700 1 -- 172.21.15.1:0/2770021151 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f33280197d0 con 0x7f333807be60 2024-02-02T15:19:12.305 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:11.541+0000 7f3340a00700 1 -- 172.21.15.1:0/2770021151 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f333819d0a0 con 0x7f333807be60 2024-02-02T15:19:12.305 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:11.542+0000 7f3336a00700 1 -- 172.21.15.1:0/2770021151 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 8) v1 ==== 45268+0+0 (secure 0 0 0) 0x7f332801a430 con 0x7f333807be60 2024-02-02T15:19:12.305 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:11.542+0000 7f332fe00700 1 -- 172.21.15.1:0/2770021151 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33080052f0 con 0x7f333807be60 2024-02-02T15:19:12.305 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:11.543+0000 7f3336a00700 1 --2- 172.21.15.1:0/2770021151 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7f3318038350 0x7f331803a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:19:12.305 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:11.543+0000 7f3336a00700 1 -- 172.21.15.1:0/2770021151 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f332804c4a0 con 0x7f333807be60 2024-02-02T15:19:12.305 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:11.543+0000 7f333d600700 1 --2- 172.21.15.1:0/2770021151 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7f3318038350 0x7f331803a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:19:12.305 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:11.544+0000 7f333d600700 1 --2- 172.21.15.1:0/2770021151 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7f3318038350 0x7f331803a810 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3320006fd0 tx=0x7f3320006e40 comp rx=0 tx=0).ready entity=mgr.14122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:19:12.305 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:11.548+0000 7f3336a00700 1 -- 172.21.15.1:0/2770021151 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f332802d9a0 con 0x7f333807be60 2024-02-02T15:19:12.305 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:11.794+0000 7f332fe00700 1 -- 172.21.15.1:0/2770021151 --> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f3308000bc0 con 0x7f3318038350 2024-02-02T15:19:12.305 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:11.799+0000 7f3336a00700 1 -- 172.21.15.1:0/2770021151 <== mgr.14122 v2:172.21.15.1:6800/395498296 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f3308000bc0 con 0x7f3318038350 2024-02-02T15:19:12.305 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:11.801+0000 7f332fe00700 1 -- 172.21.15.1:0/2770021151 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7f3318038350 msgr2=0x7f331803a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:19:12.305 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:11.801+0000 7f332fe00700 1 --2- 172.21.15.1:0/2770021151 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7f3318038350 0x7f331803a810 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3320006fd0 tx=0x7f3320006e40 comp rx=0 tx=0).stop 2024-02-02T15:19:12.306 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:11.801+0000 7f332fe00700 1 -- 172.21.15.1:0/2770021151 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f333807be60 msgr2=0x7f333819c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:19:12.306 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:11.801+0000 7f332fe00700 1 --2- 172.21.15.1:0/2770021151 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f333807be60 0x7f333819c540 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3328009130 tx=0x7f3328011ad0 comp rx=0 tx=0).stop 2024-02-02T15:19:12.306 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:11.801+0000 7f332fe00700 1 -- 172.21.15.1:0/2770021151 shutdown_connections 2024-02-02T15:19:12.306 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:11.801+0000 7f332fe00700 1 --2- 172.21.15.1:0/2770021151 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7f3318038350 0x7f331803a810 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:12.306 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:11.801+0000 7f332fe00700 1 --2- 172.21.15.1:0/2770021151 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f333807be60 0x7f333819c540 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:12.306 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:11.802+0000 7f332fe00700 1 -- 172.21.15.1:0/2770021151 >> 172.21.15.1:0/2770021151 conn(0x7f3338101c20 msgr2=0x7f3338103340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:19:12.306 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:11.802+0000 7f332fe00700 1 -- 172.21.15.1:0/2770021151 shutdown_connections 2024-02-02T15:19:12.306 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:11.802+0000 7f332fe00700 1 -- 172.21.15.1:0/2770021151 wait complete. 2024-02-02T15:19:12.306 INFO:teuthology.orchestra.run.smithi001.stdout:Deploying unmanaged mgr service... 2024-02-02T15:19:13.097 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:12 smithi001 ceph-mon[35470]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:19:13.097 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:12 smithi001 ceph-mon[35470]: Saving service mon spec with placement count:5 2024-02-02T15:19:13.097 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:12 smithi001 ceph-mon[35470]: from='mgr.14122 172.21.15.1:0/954154330' entity='mgr.x' 2024-02-02T15:19:13.097 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:12 smithi001 ceph-mon[35470]: from='mgr.14122 172.21.15.1:0/954154330' entity='mgr.x' 2024-02-02T15:19:13.097 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:12 smithi001 ceph-mon[35470]: from='mgr.14122 172.21.15.1:0/954154330' entity='mgr.x' 2024-02-02T15:19:13.231 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-02-02T15:19:13.231 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:12.702+0000 7f1b9b200700 1 Processor -- start 2024-02-02T15:19:13.231 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:12.702+0000 7f1b9b200700 1 -- start start 2024-02-02T15:19:13.231 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:12.702+0000 7f1b9b200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b9407be60 0x7f1b9407a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:19:13.231 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:12.702+0000 7f1b9b200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9407a890 con 0x7f1b9407be60 2024-02-02T15:19:13.231 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:12.703+0000 7f1b93e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b9407be60 0x7f1b9407a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:19:13.231 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:12.703+0000 7f1b93e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b9407be60 0x7f1b9407a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46722/0 (socket says 172.21.15.1:46722) 2024-02-02T15:19:13.232 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:12.703+0000 7f1b93e00700 1 -- 172.21.15.1:0/3606907334 learned_addr learned my addr 172.21.15.1:0/3606907334 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:19:13.232 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:12.703+0000 7f1b93e00700 1 -- 172.21.15.1:0/3606907334 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b9407a9d0 con 0x7f1b9407be60 2024-02-02T15:19:13.232 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:12.703+0000 7f1b93e00700 1 --2- 172.21.15.1:0/3606907334 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b9407be60 0x7f1b9407a2c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1b840098a0 tx=0x7f1b84009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34c53cd3af24cab6 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:19:13.232 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:12.703+0000 7f1b92a00700 1 -- 172.21.15.1:0/3606907334 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1b84004160 con 0x7f1b9407be60 2024-02-02T15:19:13.232 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:12.704+0000 7f1b92a00700 1 -- 172.21.15.1:0/3606907334 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b840042c0 con 0x7f1b9407be60 2024-02-02T15:19:13.232 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:12.704+0000 7f1b92a00700 1 -- 172.21.15.1:0/3606907334 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1b840119b0 con 0x7f1b9407be60 2024-02-02T15:19:13.232 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:12.704+0000 7f1b9b200700 1 -- 172.21.15.1:0/3606907334 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b9407be60 msgr2=0x7f1b9407a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:19:13.232 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:12.704+0000 7f1b9b200700 1 --2- 172.21.15.1:0/3606907334 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b9407be60 0x7f1b9407a2c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1b840098a0 tx=0x7f1b84009bb0 comp rx=0 tx=0).stop 2024-02-02T15:19:13.232 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:12.704+0000 7f1b9b200700 1 -- 172.21.15.1:0/3606907334 shutdown_connections 2024-02-02T15:19:13.232 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:12.704+0000 7f1b9b200700 1 --2- 172.21.15.1:0/3606907334 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b9407be60 0x7f1b9407a2c0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:13.232 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:12.704+0000 7f1b9b200700 1 -- 172.21.15.1:0/3606907334 >> 172.21.15.1:0/3606907334 conn(0x7f1b94101c20 msgr2=0x7f1b94104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:19:13.232 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:12.704+0000 7f1b9b200700 1 -- 172.21.15.1:0/3606907334 shutdown_connections 2024-02-02T15:19:13.233 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:12.704+0000 7f1b9b200700 1 -- 172.21.15.1:0/3606907334 wait complete. 2024-02-02T15:19:13.233 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:12.706+0000 7f1b9b200700 1 Processor -- start 2024-02-02T15:19:13.233 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:12.706+0000 7f1b9b200700 1 -- start start 2024-02-02T15:19:13.233 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:12.706+0000 7f1b9b200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b9407be60 0x7f1b9419c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:19:13.233 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:12.706+0000 7f1b9b200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9419cac0 con 0x7f1b9407be60 2024-02-02T15:19:13.233 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:12.707+0000 7f1b93e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b9407be60 0x7f1b9419c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:19:13.233 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:12.707+0000 7f1b93e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b9407be60 0x7f1b9419c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46730/0 (socket says 172.21.15.1:46730) 2024-02-02T15:19:13.234 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:12.707+0000 7f1b93e00700 1 -- 172.21.15.1:0/359562764 learned_addr learned my addr 172.21.15.1:0/359562764 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:19:13.234 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:12.707+0000 7f1b93e00700 1 -- 172.21.15.1:0/359562764 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b84009550 con 0x7f1b9407be60 2024-02-02T15:19:13.234 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:12.707+0000 7f1b93e00700 1 --2- 172.21.15.1:0/359562764 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b9407be60 0x7f1b9419c580 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1b84003f00 tx=0x7f1b84011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:19:13.234 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:12.708+0000 7f1b90c00700 1 -- 172.21.15.1:0/359562764 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1b84003b20 con 0x7f1b9407be60 2024-02-02T15:19:13.234 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:12.708+0000 7f1b90c00700 1 -- 172.21.15.1:0/359562764 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b84003c80 con 0x7f1b9407be60 2024-02-02T15:19:13.235 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:12.708+0000 7f1b90c00700 1 -- 172.21.15.1:0/359562764 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1b84019770 con 0x7f1b9407be60 2024-02-02T15:19:13.235 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:12.708+0000 7f1b9b200700 1 -- 172.21.15.1:0/359562764 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b9419ccc0 con 0x7f1b9407be60 2024-02-02T15:19:13.235 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:12.708+0000 7f1b9b200700 1 -- 172.21.15.1:0/359562764 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b9419d0e0 con 0x7f1b9407be60 2024-02-02T15:19:13.235 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:12.709+0000 7f1b90c00700 1 -- 172.21.15.1:0/359562764 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 8) v1 ==== 45268+0+0 (secure 0 0 0) 0x7f1b8401ab60 con 0x7f1b9407be60 2024-02-02T15:19:13.235 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:12.709+0000 7f1b90c00700 1 --2- 172.21.15.1:0/359562764 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7f1b740383a0 0x7f1b7403a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:19:13.235 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:12.709+0000 7f1b90c00700 1 -- 172.21.15.1:0/359562764 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f1b8404c4f0 con 0x7f1b9407be60 2024-02-02T15:19:13.235 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:12.709+0000 7f1b9b200700 1 -- 172.21.15.1:0/359562764 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b9410ca60 con 0x7f1b9407be60 2024-02-02T15:19:13.236 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:12.710+0000 7f1b93400700 1 --2- 172.21.15.1:0/359562764 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7f1b740383a0 0x7f1b7403a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:19:13.236 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:12.710+0000 7f1b93400700 1 --2- 172.21.15.1:0/359562764 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7f1b740383a0 0x7f1b7403a860 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c006fd0 tx=0x7f1b7c006e40 comp rx=0 tx=0).ready entity=mgr.14122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:19:13.236 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:12.715+0000 7f1b90c00700 1 -- 172.21.15.1:0/359562764 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f1b8401a430 con 0x7f1b9407be60 2024-02-02T15:19:13.236 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:12.947+0000 7f1b9b200700 1 -- 172.21.15.1:0/359562764 --> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f1b94103f60 con 0x7f1b740383a0 2024-02-02T15:19:13.236 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:12.952+0000 7f1b90c00700 1 -- 172.21.15.1:0/359562764 <== mgr.14122 v2:172.21.15.1:6800/395498296 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f1b94103f60 con 0x7f1b740383a0 2024-02-02T15:19:13.236 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:12.954+0000 7f1b9b200700 1 -- 172.21.15.1:0/359562764 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7f1b740383a0 msgr2=0x7f1b7403a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:19:13.237 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:12.954+0000 7f1b9b200700 1 --2- 172.21.15.1:0/359562764 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7f1b740383a0 0x7f1b7403a860 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c006fd0 tx=0x7f1b7c006e40 comp rx=0 tx=0).stop 2024-02-02T15:19:13.237 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:12.954+0000 7f1b9b200700 1 -- 172.21.15.1:0/359562764 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b9407be60 msgr2=0x7f1b9419c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:19:13.237 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:12.954+0000 7f1b9b200700 1 --2- 172.21.15.1:0/359562764 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b9407be60 0x7f1b9419c580 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1b84003f00 tx=0x7f1b84011a30 comp rx=0 tx=0).stop 2024-02-02T15:19:13.237 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:12.954+0000 7f1b9b200700 1 -- 172.21.15.1:0/359562764 shutdown_connections 2024-02-02T15:19:13.237 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:12.954+0000 7f1b9b200700 1 --2- 172.21.15.1:0/359562764 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7f1b740383a0 0x7f1b7403a860 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:13.237 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:12.954+0000 7f1b9b200700 1 --2- 172.21.15.1:0/359562764 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1b9407be60 0x7f1b9419c580 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:13.237 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:12.954+0000 7f1b9b200700 1 -- 172.21.15.1:0/359562764 >> 172.21.15.1:0/359562764 conn(0x7f1b94101c20 msgr2=0x7f1b94103340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:19:13.237 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:12.954+0000 7f1b9b200700 1 -- 172.21.15.1:0/359562764 shutdown_connections 2024-02-02T15:19:13.237 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:12.954+0000 7f1b9b200700 1 -- 172.21.15.1:0/359562764 wait complete. 2024-02-02T15:19:14.338 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:13.747+0000 7f40fe200700 1 Processor -- start 2024-02-02T15:19:14.338 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:13.747+0000 7f40fe200700 1 -- start start 2024-02-02T15:19:14.338 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:13.747+0000 7f40fe200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40f807be40 0x7f40f807a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:19:14.338 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:13.747+0000 7f40fe200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f807a870 con 0x7f40f807be40 2024-02-02T15:19:14.338 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:13.748+0000 7f40f7400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40f807be40 0x7f40f807a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:19:14.338 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:13.748+0000 7f40f7400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40f807be40 0x7f40f807a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59788/0 (socket says 172.21.15.1:59788) 2024-02-02T15:19:14.338 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:13.748+0000 7f40f7400700 1 -- 172.21.15.1:0/889506477 learned_addr learned my addr 172.21.15.1:0/889506477 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:19:14.338 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:13.748+0000 7f40f7400700 1 -- 172.21.15.1:0/889506477 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40f807a9b0 con 0x7f40f807be40 2024-02-02T15:19:14.338 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:13.748+0000 7f40f7400700 1 --2- 172.21.15.1:0/889506477 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40f807be40 0x7f40f807a2a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f40e80098a0 tx=0x7f40e8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35fbead683fe602d server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:19:14.338 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:13.749+0000 7f40f6000700 1 -- 172.21.15.1:0/889506477 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f40e8004160 con 0x7f40f807be40 2024-02-02T15:19:14.338 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:13.749+0000 7f40f6000700 1 -- 172.21.15.1:0/889506477 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f40e80042c0 con 0x7f40f807be40 2024-02-02T15:19:14.339 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:13.749+0000 7f40f6000700 1 -- 172.21.15.1:0/889506477 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f40e80119b0 con 0x7f40f807be40 2024-02-02T15:19:14.339 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:13.749+0000 7f40fe200700 1 -- 172.21.15.1:0/889506477 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40f807be40 msgr2=0x7f40f807a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:19:14.339 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:13.749+0000 7f40fe200700 1 --2- 172.21.15.1:0/889506477 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40f807be40 0x7f40f807a2a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f40e80098a0 tx=0x7f40e8009bb0 comp rx=0 tx=0).stop 2024-02-02T15:19:14.339 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:13.749+0000 7f40fe200700 1 -- 172.21.15.1:0/889506477 shutdown_connections 2024-02-02T15:19:14.339 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:13.749+0000 7f40fe200700 1 --2- 172.21.15.1:0/889506477 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40f807be40 0x7f40f807a2a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:14.339 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:13.749+0000 7f40fe200700 1 -- 172.21.15.1:0/889506477 >> 172.21.15.1:0/889506477 conn(0x7f40f8101c20 msgr2=0x7f40f8104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:19:14.339 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:13.749+0000 7f40fe200700 1 -- 172.21.15.1:0/889506477 shutdown_connections 2024-02-02T15:19:14.339 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:13.749+0000 7f40fe200700 1 -- 172.21.15.1:0/889506477 wait complete. 2024-02-02T15:19:14.339 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:13.751+0000 7f40fe200700 1 Processor -- start 2024-02-02T15:19:14.339 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:13.751+0000 7f40fe200700 1 -- start start 2024-02-02T15:19:14.339 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:13.751+0000 7f40fe200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40f807be40 0x7f40f819c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:19:14.339 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:13.752+0000 7f40fe200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f819cab0 con 0x7f40f807be40 2024-02-02T15:19:14.339 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:13.752+0000 7f40f7400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40f807be40 0x7f40f819c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:19:14.340 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:13.752+0000 7f40f7400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40f807be40 0x7f40f819c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59794/0 (socket says 172.21.15.1:59794) 2024-02-02T15:19:14.340 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:13.752+0000 7f40f7400700 1 -- 172.21.15.1:0/1271526557 learned_addr learned my addr 172.21.15.1:0/1271526557 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:19:14.340 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:13.752+0000 7f40f7400700 1 -- 172.21.15.1:0/1271526557 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40e8009550 con 0x7f40f807be40 2024-02-02T15:19:14.340 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:13.752+0000 7f40f7400700 1 --2- 172.21.15.1:0/1271526557 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40f807be40 0x7f40f819c570 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f40e8003f00 tx=0x7f40e8011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:19:14.340 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:13.752+0000 7f40efe00700 1 -- 172.21.15.1:0/1271526557 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f40e8003b20 con 0x7f40f807be40 2024-02-02T15:19:14.340 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:13.753+0000 7f40efe00700 1 -- 172.21.15.1:0/1271526557 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f40e8003c80 con 0x7f40f807be40 2024-02-02T15:19:14.340 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:13.753+0000 7f40efe00700 1 -- 172.21.15.1:0/1271526557 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f40e8019770 con 0x7f40f807be40 2024-02-02T15:19:14.340 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:13.753+0000 7f40fe200700 1 -- 172.21.15.1:0/1271526557 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40f819ccb0 con 0x7f40f807be40 2024-02-02T15:19:14.340 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:13.753+0000 7f40fe200700 1 -- 172.21.15.1:0/1271526557 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40f819d0d0 con 0x7f40f807be40 2024-02-02T15:19:14.340 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:13.754+0000 7f40fe200700 1 -- 172.21.15.1:0/1271526557 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40f810ca20 con 0x7f40f807be40 2024-02-02T15:19:14.340 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:13.754+0000 7f40efe00700 1 -- 172.21.15.1:0/1271526557 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 8) v1 ==== 45268+0+0 (secure 0 0 0) 0x7f40e801ab60 con 0x7f40f807be40 2024-02-02T15:19:14.340 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:13.754+0000 7f40efe00700 1 --2- 172.21.15.1:0/1271526557 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7f40d8038350 0x7f40d803a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:19:14.340 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:13.754+0000 7f40efe00700 1 -- 172.21.15.1:0/1271526557 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f40e804c4f0 con 0x7f40f807be40 2024-02-02T15:19:14.341 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:13.754+0000 7f40f6a00700 1 --2- 172.21.15.1:0/1271526557 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7f40d8038350 0x7f40d803a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:19:14.341 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:13.755+0000 7f40f6a00700 1 --2- 172.21.15.1:0/1271526557 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7f40d8038350 0x7f40d803a810 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f40e0006fd0 tx=0x7f40e0006e40 comp rx=0 tx=0).ready entity=mgr.14122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:19:14.341 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:13.759+0000 7f40efe00700 1 -- 172.21.15.1:0/1271526557 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f40e801a430 con 0x7f40f807be40 2024-02-02T15:19:14.341 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:13.979+0000 7f40fe200700 1 -- 172.21.15.1:0/1271526557 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f40f810cc10 con 0x7f40f807be40 2024-02-02T15:19:14.341 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:13.983+0000 7f40efe00700 1 -- 172.21.15.1:0/1271526557 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f40e802db40 con 0x7f40f807be40 2024-02-02T15:19:14.341 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:13.986+0000 7f40fe200700 1 -- 172.21.15.1:0/1271526557 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7f40d8038350 msgr2=0x7f40d803a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:19:14.341 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:13.986+0000 7f40fe200700 1 --2- 172.21.15.1:0/1271526557 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7f40d8038350 0x7f40d803a810 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f40e0006fd0 tx=0x7f40e0006e40 comp rx=0 tx=0).stop 2024-02-02T15:19:14.341 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:13.986+0000 7f40fe200700 1 -- 172.21.15.1:0/1271526557 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40f807be40 msgr2=0x7f40f819c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:19:14.341 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:13.986+0000 7f40fe200700 1 --2- 172.21.15.1:0/1271526557 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40f807be40 0x7f40f819c570 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f40e8003f00 tx=0x7f40e8011a30 comp rx=0 tx=0).stop 2024-02-02T15:19:14.341 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:13.986+0000 7f40fe200700 1 -- 172.21.15.1:0/1271526557 shutdown_connections 2024-02-02T15:19:14.341 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:13.986+0000 7f40fe200700 1 --2- 172.21.15.1:0/1271526557 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7f40d8038350 0x7f40d803a810 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:14.341 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:13.986+0000 7f40fe200700 1 --2- 172.21.15.1:0/1271526557 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40f807be40 0x7f40f819c570 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:14.341 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:13.986+0000 7f40fe200700 1 -- 172.21.15.1:0/1271526557 >> 172.21.15.1:0/1271526557 conn(0x7f40f8101c20 msgr2=0x7f40f81032f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:19:14.341 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:13.986+0000 7f40fe200700 1 -- 172.21.15.1:0/1271526557 shutdown_connections 2024-02-02T15:19:14.342 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:13.986+0000 7f40fe200700 1 -- 172.21.15.1:0/1271526557 wait complete. 2024-02-02T15:19:14.342 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:13 smithi001 ceph-mon[35470]: from='client.14146 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:19:14.342 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:13 smithi001 ceph-mon[35470]: Saving service mgr spec with placement count:2 2024-02-02T15:19:14.342 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:13 smithi001 ceph-mon[35470]: from='mgr.14122 172.21.15.1:0/954154330' entity='mgr.x' 2024-02-02T15:19:14.342 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:13 smithi001 ceph-mon[35470]: from='mgr.14122 172.21.15.1:0/954154330' entity='mgr.x' 2024-02-02T15:19:15.233 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:14 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/1271526557' entity='client.admin' 2024-02-02T15:19:15.390 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:14.776+0000 7f6f7e400700 1 Processor -- start 2024-02-02T15:19:15.390 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:14.776+0000 7f6f7e400700 1 -- start start 2024-02-02T15:19:15.390 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:14.776+0000 7f6f7e400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6f78106620 0x7f6f78106a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:19:15.390 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:14.776+0000 7f6f7e400700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f78107010 con 0x7f6f78106620 2024-02-02T15:19:15.390 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:14.777+0000 7f6f77400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6f78106620 0x7f6f78106a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:19:15.390 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:14.777+0000 7f6f77400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6f78106620 0x7f6f78106a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59800/0 (socket says 172.21.15.1:59800) 2024-02-02T15:19:15.390 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:14.777+0000 7f6f77400700 1 -- 172.21.15.1:0/1576124839 learned_addr learned my addr 172.21.15.1:0/1576124839 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:19:15.391 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:14.777+0000 7f6f77400700 1 -- 172.21.15.1:0/1576124839 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f78107820 con 0x7f6f78106620 2024-02-02T15:19:15.391 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:14.778+0000 7f6f77400700 1 --2- 172.21.15.1:0/1576124839 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6f78106620 0x7f6f78106a40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6f680098a0 tx=0x7f6f68009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acfb9f47c3405fcb server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:19:15.391 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:14.778+0000 7f6f76000700 1 -- 172.21.15.1:0/1576124839 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6f68004160 con 0x7f6f78106620 2024-02-02T15:19:15.391 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:14.778+0000 7f6f76000700 1 -- 172.21.15.1:0/1576124839 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6f680042c0 con 0x7f6f78106620 2024-02-02T15:19:15.391 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:14.778+0000 7f6f76000700 1 -- 172.21.15.1:0/1576124839 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6f680119b0 con 0x7f6f78106620 2024-02-02T15:19:15.391 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:14.779+0000 7f6f7e400700 1 -- 172.21.15.1:0/1576124839 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6f78106620 msgr2=0x7f6f78106a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:19:15.391 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:14.779+0000 7f6f7e400700 1 --2- 172.21.15.1:0/1576124839 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6f78106620 0x7f6f78106a40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6f680098a0 tx=0x7f6f68009bb0 comp rx=0 tx=0).stop 2024-02-02T15:19:15.391 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:14.779+0000 7f6f7e400700 1 -- 172.21.15.1:0/1576124839 shutdown_connections 2024-02-02T15:19:15.391 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:14.779+0000 7f6f7e400700 1 --2- 172.21.15.1:0/1576124839 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6f78106620 0x7f6f78106a40 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:15.391 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:14.779+0000 7f6f7e400700 1 -- 172.21.15.1:0/1576124839 >> 172.21.15.1:0/1576124839 conn(0x7f6f78101c20 msgr2=0x7f6f78104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:19:15.391 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:14.779+0000 7f6f7e400700 1 -- 172.21.15.1:0/1576124839 shutdown_connections 2024-02-02T15:19:15.391 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:14.779+0000 7f6f7e400700 1 -- 172.21.15.1:0/1576124839 wait complete. 2024-02-02T15:19:15.391 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:14.780+0000 7f6f7e400700 1 Processor -- start 2024-02-02T15:19:15.391 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:14.780+0000 7f6f7e400700 1 -- start start 2024-02-02T15:19:15.392 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:14.781+0000 7f6f7e400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6f78106620 0x7f6f7807bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:19:15.392 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:14.781+0000 7f6f7e400700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f7807c170 con 0x7f6f78106620 2024-02-02T15:19:15.392 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:14.781+0000 7f6f77400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6f78106620 0x7f6f7807bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:19:15.392 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:14.781+0000 7f6f77400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6f78106620 0x7f6f7807bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59812/0 (socket says 172.21.15.1:59812) 2024-02-02T15:19:15.392 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:14.781+0000 7f6f77400700 1 -- 172.21.15.1:0/983783824 learned_addr learned my addr 172.21.15.1:0/983783824 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:19:15.392 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:14.781+0000 7f6f77400700 1 -- 172.21.15.1:0/983783824 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f68009550 con 0x7f6f78106620 2024-02-02T15:19:15.392 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:14.781+0000 7f6f77400700 1 --2- 172.21.15.1:0/983783824 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6f78106620 0x7f6f7807bc30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6f68003e60 tx=0x7f6f68011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:19:15.392 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:14.782+0000 7f6f6fe00700 1 -- 172.21.15.1:0/983783824 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6f68003a60 con 0x7f6f78106620 2024-02-02T15:19:15.392 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:14.782+0000 7f6f6fe00700 1 -- 172.21.15.1:0/983783824 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6f68003bc0 con 0x7f6f78106620 2024-02-02T15:19:15.392 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:14.782+0000 7f6f7e400700 1 -- 172.21.15.1:0/983783824 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f7807a280 con 0x7f6f78106620 2024-02-02T15:19:15.392 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:14.782+0000 7f6f6fe00700 1 -- 172.21.15.1:0/983783824 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6f680196c0 con 0x7f6f78106620 2024-02-02T15:19:15.392 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:14.782+0000 7f6f7e400700 1 -- 172.21.15.1:0/983783824 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f7807a6a0 con 0x7f6f78106620 2024-02-02T15:19:15.392 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:14.783+0000 7f6f7e400700 1 -- 172.21.15.1:0/983783824 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f7810aa50 con 0x7f6f78106620 2024-02-02T15:19:15.392 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:14.783+0000 7f6f6fe00700 1 -- 172.21.15.1:0/983783824 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 8) v1 ==== 45268+0+0 (secure 0 0 0) 0x7f6f6801ab60 con 0x7f6f78106620 2024-02-02T15:19:15.393 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:14.784+0000 7f6f6fe00700 1 --2- 172.21.15.1:0/983783824 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7f6f580382e0 0x7f6f5803a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:19:15.393 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:14.784+0000 7f6f6fe00700 1 -- 172.21.15.1:0/983783824 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f6f6804c420 con 0x7f6f78106620 2024-02-02T15:19:15.393 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:14.784+0000 7f6f76a00700 1 --2- 172.21.15.1:0/983783824 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7f6f580382e0 0x7f6f5803a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:19:15.393 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:14.785+0000 7f6f76a00700 1 --2- 172.21.15.1:0/983783824 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7f6f580382e0 0x7f6f5803a7a0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6f60006fd0 tx=0x7f6f60006e40 comp rx=0 tx=0).ready entity=mgr.14122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:19:15.393 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:14.789+0000 7f6f6fe00700 1 -- 172.21.15.1:0/983783824 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f6f68010790 con 0x7f6f78106620 2024-02-02T15:19:15.393 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:15.021+0000 7f6f7e400700 1 -- 172.21.15.1:0/983783824 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f6f7810ac70 con 0x7f6f78106620 2024-02-02T15:19:15.393 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:15.024+0000 7f6f6fe00700 1 -- 172.21.15.1:0/983783824 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f6f68010380 con 0x7f6f78106620 2024-02-02T15:19:15.393 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:15.027+0000 7f6f7e400700 1 -- 172.21.15.1:0/983783824 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7f6f580382e0 msgr2=0x7f6f5803a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:19:15.393 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:15.027+0000 7f6f7e400700 1 --2- 172.21.15.1:0/983783824 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7f6f580382e0 0x7f6f5803a7a0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6f60006fd0 tx=0x7f6f60006e40 comp rx=0 tx=0).stop 2024-02-02T15:19:15.393 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:15.027+0000 7f6f7e400700 1 -- 172.21.15.1:0/983783824 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6f78106620 msgr2=0x7f6f7807bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:19:15.393 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:15.027+0000 7f6f7e400700 1 --2- 172.21.15.1:0/983783824 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6f78106620 0x7f6f7807bc30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6f68003e60 tx=0x7f6f68011a30 comp rx=0 tx=0).stop 2024-02-02T15:19:15.393 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:15.027+0000 7f6f7e400700 1 -- 172.21.15.1:0/983783824 shutdown_connections 2024-02-02T15:19:15.393 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:15.027+0000 7f6f7e400700 1 --2- 172.21.15.1:0/983783824 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7f6f580382e0 0x7f6f5803a7a0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:15.393 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:15.027+0000 7f6f7e400700 1 --2- 172.21.15.1:0/983783824 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6f78106620 0x7f6f7807bc30 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:15.394 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:15.027+0000 7f6f7e400700 1 -- 172.21.15.1:0/983783824 >> 172.21.15.1:0/983783824 conn(0x7f6f78101c20 msgr2=0x7f6f78103310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:19:15.394 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:15.027+0000 7f6f7e400700 1 -- 172.21.15.1:0/983783824 shutdown_connections 2024-02-02T15:19:15.394 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:15.027+0000 7f6f7e400700 1 -- 172.21.15.1:0/983783824 wait complete. 2024-02-02T15:19:15.394 INFO:teuthology.orchestra.run.smithi001.stdout:Enabling the dashboard module... 2024-02-02T15:19:16.313 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:16 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/983783824' entity='client.admin' 2024-02-02T15:19:16.313 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:16 smithi001 ceph-mon[35470]: from='mgr.14122 172.21.15.1:0/954154330' entity='mgr.x' 2024-02-02T15:19:17.286 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:17 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/3061579861' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-02-02T15:19:17.286 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:19:17 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: ignoring --setuser ceph since I am not root 2024-02-02T15:19:17.286 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:19:17 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: ignoring --setgroup ceph since I am not root 2024-02-02T15:19:17.433 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:15.809+0000 7f2e0d600700 1 Processor -- start 2024-02-02T15:19:17.433 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:15.810+0000 7f2e0d600700 1 -- start start 2024-02-02T15:19:17.434 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:15.810+0000 7f2e0d600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e08106620 0x7f2e08106a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:19:17.434 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:15.810+0000 7f2e0d600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e08107010 con 0x7f2e08106620 2024-02-02T15:19:17.434 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:15.810+0000 7f2e06a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e08106620 0x7f2e08106a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:19:17.434 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:15.810+0000 7f2e06a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e08106620 0x7f2e08106a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59820/0 (socket says 172.21.15.1:59820) 2024-02-02T15:19:17.434 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:15.810+0000 7f2e06a00700 1 -- 172.21.15.1:0/3550401208 learned_addr learned my addr 172.21.15.1:0/3550401208 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:19:17.434 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:15.811+0000 7f2e06a00700 1 -- 172.21.15.1:0/3550401208 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e08107820 con 0x7f2e08106620 2024-02-02T15:19:17.434 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:15.811+0000 7f2e06a00700 1 --2- 172.21.15.1:0/3550401208 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e08106620 0x7f2e08106a40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2df80098a0 tx=0x7f2df8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6621eb92615b7f35 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:19:17.434 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:15.811+0000 7f2e05600700 1 -- 172.21.15.1:0/3550401208 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2df8004160 con 0x7f2e08106620 2024-02-02T15:19:17.434 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:15.811+0000 7f2e05600700 1 -- 172.21.15.1:0/3550401208 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2df80042c0 con 0x7f2e08106620 2024-02-02T15:19:17.434 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:15.811+0000 7f2e05600700 1 -- 172.21.15.1:0/3550401208 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2df80119b0 con 0x7f2e08106620 2024-02-02T15:19:17.434 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:15.812+0000 7f2e0d600700 1 -- 172.21.15.1:0/3550401208 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e08106620 msgr2=0x7f2e08106a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:19:17.434 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:15.812+0000 7f2e0d600700 1 --2- 172.21.15.1:0/3550401208 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e08106620 0x7f2e08106a40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2df80098a0 tx=0x7f2df8009bb0 comp rx=0 tx=0).stop 2024-02-02T15:19:17.434 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:15.812+0000 7f2e0d600700 1 -- 172.21.15.1:0/3550401208 shutdown_connections 2024-02-02T15:19:17.434 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:15.812+0000 7f2e0d600700 1 --2- 172.21.15.1:0/3550401208 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e08106620 0x7f2e08106a40 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:17.435 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:15.812+0000 7f2e0d600700 1 -- 172.21.15.1:0/3550401208 >> 172.21.15.1:0/3550401208 conn(0x7f2e08101c20 msgr2=0x7f2e08104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:19:17.435 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:15.812+0000 7f2e0d600700 1 -- 172.21.15.1:0/3550401208 shutdown_connections 2024-02-02T15:19:17.435 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:15.812+0000 7f2e0d600700 1 -- 172.21.15.1:0/3550401208 wait complete. 2024-02-02T15:19:17.435 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:15.813+0000 7f2e0d600700 1 Processor -- start 2024-02-02T15:19:17.435 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:15.814+0000 7f2e0d600700 1 -- start start 2024-02-02T15:19:17.435 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:15.814+0000 7f2e0d600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e08106620 0x7f2e0807bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:19:17.435 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:15.814+0000 7f2e0d600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e0807c170 con 0x7f2e08106620 2024-02-02T15:19:17.435 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:15.814+0000 7f2e06a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e08106620 0x7f2e0807bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:19:17.435 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:15.814+0000 7f2e06a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e08106620 0x7f2e0807bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59822/0 (socket says 172.21.15.1:59822) 2024-02-02T15:19:17.435 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:15.814+0000 7f2e06a00700 1 -- 172.21.15.1:0/3061579861 learned_addr learned my addr 172.21.15.1:0/3061579861 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:19:17.435 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:15.815+0000 7f2e06a00700 1 -- 172.21.15.1:0/3061579861 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2df8009550 con 0x7f2e08106620 2024-02-02T15:19:17.435 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:15.815+0000 7f2e06a00700 1 --2- 172.21.15.1:0/3061579861 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e08106620 0x7f2e0807bc30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2df8003e60 tx=0x7f2df8011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:19:17.435 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:15.815+0000 7f2dff400700 1 -- 172.21.15.1:0/3061579861 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2df8003a60 con 0x7f2e08106620 2024-02-02T15:19:17.435 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:15.816+0000 7f2dff400700 1 -- 172.21.15.1:0/3061579861 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2df8003bc0 con 0x7f2e08106620 2024-02-02T15:19:17.436 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:15.816+0000 7f2dff400700 1 -- 172.21.15.1:0/3061579861 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2df80196c0 con 0x7f2e08106620 2024-02-02T15:19:17.436 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:15.816+0000 7f2e0d600700 1 -- 172.21.15.1:0/3061579861 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e0807a280 con 0x7f2e08106620 2024-02-02T15:19:17.436 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:15.816+0000 7f2e0d600700 1 -- 172.21.15.1:0/3061579861 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e0807a6a0 con 0x7f2e08106620 2024-02-02T15:19:17.436 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:15.817+0000 7f2e0d600700 1 -- 172.21.15.1:0/3061579861 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e0810aa50 con 0x7f2e08106620 2024-02-02T15:19:17.436 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:15.817+0000 7f2dff400700 1 -- 172.21.15.1:0/3061579861 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 8) v1 ==== 45268+0+0 (secure 0 0 0) 0x7f2df801ab60 con 0x7f2e08106620 2024-02-02T15:19:17.436 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:15.818+0000 7f2dff400700 1 --2- 172.21.15.1:0/3061579861 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7f2de8038330 0x7f2de803a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:19:17.436 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:15.818+0000 7f2dff400700 1 -- 172.21.15.1:0/3061579861 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f2df804c4e0 con 0x7f2e08106620 2024-02-02T15:19:17.436 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:15.818+0000 7f2e06000700 1 --2- 172.21.15.1:0/3061579861 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7f2de8038330 0x7f2de803a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:19:17.436 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:15.819+0000 7f2e06000700 1 --2- 172.21.15.1:0/3061579861 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7f2de8038330 0x7f2de803a7f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2df0006fd0 tx=0x7f2df0006e40 comp rx=0 tx=0).ready entity=mgr.14122 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:19:17.436 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:15.823+0000 7f2dff400700 1 -- 172.21.15.1:0/3061579861 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f2df8010790 con 0x7f2e08106620 2024-02-02T15:19:17.436 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:16.109+0000 7f2e0d600700 1 -- 172.21.15.1:0/3061579861 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f2e0810ac70 con 0x7f2e08106620 2024-02-02T15:19:17.436 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:17.029+0000 7f2dff400700 1 -- 172.21.15.1:0/3061579861 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mgrmap(e 9) v1 ==== 45281+0+0 (secure 0 0 0) 0x7f2df801a430 con 0x7f2e08106620 2024-02-02T15:19:17.437 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:17.031+0000 7f2dff400700 1 -- 172.21.15.1:0/3061579861 <== mon.0 v2:172.21.15.1:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f2df801a950 con 0x7f2e08106620 2024-02-02T15:19:17.437 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:17.034+0000 7f2e0d600700 1 -- 172.21.15.1:0/3061579861 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7f2de8038330 msgr2=0x7f2de803a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:19:17.437 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:17.034+0000 7f2e0d600700 1 --2- 172.21.15.1:0/3061579861 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7f2de8038330 0x7f2de803a7f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2df0006fd0 tx=0x7f2df0006e40 comp rx=0 tx=0).stop 2024-02-02T15:19:17.437 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:17.034+0000 7f2e0d600700 1 -- 172.21.15.1:0/3061579861 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e08106620 msgr2=0x7f2e0807bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:19:17.437 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:17.034+0000 7f2e0d600700 1 --2- 172.21.15.1:0/3061579861 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e08106620 0x7f2e0807bc30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2df8003e60 tx=0x7f2df8011a30 comp rx=0 tx=0).stop 2024-02-02T15:19:17.437 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:17.035+0000 7f2e0d600700 1 -- 172.21.15.1:0/3061579861 shutdown_connections 2024-02-02T15:19:17.437 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:17.035+0000 7f2e0d600700 1 --2- 172.21.15.1:0/3061579861 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7f2de8038330 0x7f2de803a7f0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:17.437 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:17.035+0000 7f2e0d600700 1 --2- 172.21.15.1:0/3061579861 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e08106620 0x7f2e0807bc30 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:17.437 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:17.035+0000 7f2e0d600700 1 -- 172.21.15.1:0/3061579861 >> 172.21.15.1:0/3061579861 conn(0x7f2e08101c20 msgr2=0x7f2e08103310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:19:17.437 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:17.035+0000 7f2e0d600700 1 -- 172.21.15.1:0/3061579861 shutdown_connections 2024-02-02T15:19:17.437 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:17.035+0000 7f2e0d600700 1 -- 172.21.15.1:0/3061579861 wait complete. 2024-02-02T15:19:17.815 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:19:17 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:19:17.687+0000 7fd8d6780200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-02T15:19:18.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:18 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/3061579861' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-02-02T15:19:18.128 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:18 smithi001 ceph-mon[35470]: mgrmap e9: x(active, since 15s) 2024-02-02T15:19:18.128 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:19:17 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:19:17.815+0000 7fd8d6780200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-02T15:19:18.129 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:19:17 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:19:17.938+0000 7fd8d6780200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-02T15:19:18.129 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:19:18 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:19:18.052+0000 7fd8d6780200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-02T15:19:18.404 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout { 2024-02-02T15:19:18.404 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-02-02T15:19:18.404 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-02T15:19:18.405 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-02-02T15:19:18.405 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-02-02T15:19:18.405 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout } 2024-02-02T15:19:18.405 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:17.821+0000 7faa84200700 1 Processor -- start 2024-02-02T15:19:18.405 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:17.821+0000 7faa84200700 1 -- start start 2024-02-02T15:19:18.405 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:17.822+0000 7faa84200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa7c07be40 0x7faa7c07a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:19:18.405 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:17.822+0000 7faa84200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa7c07a870 con 0x7faa7c07be40 2024-02-02T15:19:18.405 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:17.822+0000 7faa81800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa7c07be40 0x7faa7c07a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:19:18.405 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:17.822+0000 7faa81800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa7c07be40 0x7faa7c07a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59832/0 (socket says 172.21.15.1:59832) 2024-02-02T15:19:18.405 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:17.822+0000 7faa81800700 1 -- 172.21.15.1:0/698747176 learned_addr learned my addr 172.21.15.1:0/698747176 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:19:18.405 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:17.823+0000 7faa81800700 1 -- 172.21.15.1:0/698747176 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa7c07a9b0 con 0x7faa7c07be40 2024-02-02T15:19:18.405 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:17.823+0000 7faa81800700 1 --2- 172.21.15.1:0/698747176 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa7c07be40 0x7faa7c07a2a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7faa6c0098a0 tx=0x7faa6c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73847a10aec37901 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:19:18.405 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:17.823+0000 7faa7be00700 1 -- 172.21.15.1:0/698747176 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faa6c004160 con 0x7faa7c07be40 2024-02-02T15:19:18.405 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:17.823+0000 7faa7be00700 1 -- 172.21.15.1:0/698747176 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa6c0042c0 con 0x7faa7c07be40 2024-02-02T15:19:18.406 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:17.823+0000 7faa7be00700 1 -- 172.21.15.1:0/698747176 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faa6c0119b0 con 0x7faa7c07be40 2024-02-02T15:19:18.406 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:17.824+0000 7faa84200700 1 -- 172.21.15.1:0/698747176 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa7c07be40 msgr2=0x7faa7c07a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:19:18.406 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:17.824+0000 7faa84200700 1 --2- 172.21.15.1:0/698747176 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa7c07be40 0x7faa7c07a2a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7faa6c0098a0 tx=0x7faa6c009bb0 comp rx=0 tx=0).stop 2024-02-02T15:19:18.406 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:17.824+0000 7faa84200700 1 -- 172.21.15.1:0/698747176 shutdown_connections 2024-02-02T15:19:18.406 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:17.824+0000 7faa84200700 1 --2- 172.21.15.1:0/698747176 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa7c07be40 0x7faa7c07a2a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:18.406 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:17.824+0000 7faa84200700 1 -- 172.21.15.1:0/698747176 >> 172.21.15.1:0/698747176 conn(0x7faa7c101c20 msgr2=0x7faa7c104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:19:18.406 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:17.824+0000 7faa84200700 1 -- 172.21.15.1:0/698747176 shutdown_connections 2024-02-02T15:19:18.406 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:17.824+0000 7faa84200700 1 -- 172.21.15.1:0/698747176 wait complete. 2024-02-02T15:19:18.406 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:17.825+0000 7faa84200700 1 Processor -- start 2024-02-02T15:19:18.406 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:17.825+0000 7faa84200700 1 -- start start 2024-02-02T15:19:18.406 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:17.825+0000 7faa84200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa7c07be40 0x7faa7c19c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:19:18.406 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:17.825+0000 7faa84200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa7c19caa0 con 0x7faa7c07be40 2024-02-02T15:19:18.406 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:17.826+0000 7faa81800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa7c07be40 0x7faa7c19c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:19:18.406 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:17.826+0000 7faa81800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa7c07be40 0x7faa7c19c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59848/0 (socket says 172.21.15.1:59848) 2024-02-02T15:19:18.406 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:17.826+0000 7faa81800700 1 -- 172.21.15.1:0/600012131 learned_addr learned my addr 172.21.15.1:0/600012131 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:19:18.407 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:17.826+0000 7faa81800700 1 -- 172.21.15.1:0/600012131 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa6c009550 con 0x7faa7c07be40 2024-02-02T15:19:18.407 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:17.826+0000 7faa81800700 1 --2- 172.21.15.1:0/600012131 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa7c07be40 0x7faa7c19c560 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7faa6c003f00 tx=0x7faa6c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:19:18.407 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:17.826+0000 7faa7a000700 1 -- 172.21.15.1:0/600012131 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faa6c003b20 con 0x7faa7c07be40 2024-02-02T15:19:18.407 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:17.827+0000 7faa84200700 1 -- 172.21.15.1:0/600012131 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa7c19cca0 con 0x7faa7c07be40 2024-02-02T15:19:18.407 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:17.827+0000 7faa7a000700 1 -- 172.21.15.1:0/600012131 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa6c003c80 con 0x7faa7c07be40 2024-02-02T15:19:18.407 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:17.827+0000 7faa7a000700 1 -- 172.21.15.1:0/600012131 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7faa6c019770 con 0x7faa7c07be40 2024-02-02T15:19:18.407 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:17.827+0000 7faa84200700 1 -- 172.21.15.1:0/600012131 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa7c19d0c0 con 0x7faa7c07be40 2024-02-02T15:19:18.407 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:17.828+0000 7faa84200700 1 -- 172.21.15.1:0/600012131 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa7c196620 con 0x7faa7c07be40 2024-02-02T15:19:18.407 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:17.828+0000 7faa7a000700 1 -- 172.21.15.1:0/600012131 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 9) v1 ==== 45281+0+0 (secure 0 0 0) 0x7faa6c01ab60 con 0x7faa7c07be40 2024-02-02T15:19:18.407 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:17.828+0000 7faa7a000700 1 --2- 172.21.15.1:0/600012131 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7faa5c0383a0 0x7faa5c03a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:19:18.407 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:17.828+0000 7faa7a000700 1 -- 172.21.15.1:0/600012131 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7faa6c04c500 con 0x7faa7c07be40 2024-02-02T15:19:18.407 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:17.828+0000 7faa80e00700 1 -- 172.21.15.1:0/600012131 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7faa5c0383a0 msgr2=0x7faa5c03a860 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/395498296 2024-02-02T15:19:18.407 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:17.829+0000 7faa80e00700 1 --2- 172.21.15.1:0/600012131 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7faa5c0383a0 0x7faa5c03a860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-02T15:19:18.407 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:17.837+0000 7faa7a000700 1 -- 172.21.15.1:0/600012131 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7faa6c01a430 con 0x7faa7c07be40 2024-02-02T15:19:18.408 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:18.030+0000 7faa80e00700 1 -- 172.21.15.1:0/600012131 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7faa5c0383a0 msgr2=0x7faa5c03a860 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/395498296 2024-02-02T15:19:18.408 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:18.030+0000 7faa80e00700 1 --2- 172.21.15.1:0/600012131 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7faa5c0383a0 0x7faa5c03a860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-02T15:19:18.408 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:18.103+0000 7faa84200700 1 -- 172.21.15.1:0/600012131 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7faa7c062370 con 0x7faa7c07be40 2024-02-02T15:19:18.408 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:18.103+0000 7faa7a000700 1 -- 172.21.15.1:0/600012131 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7faa6c02db40 con 0x7faa7c07be40 2024-02-02T15:19:18.408 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:18.105+0000 7faa84200700 1 -- 172.21.15.1:0/600012131 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7faa5c0383a0 msgr2=0x7faa5c03a860 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-02T15:19:18.408 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:18.105+0000 7faa84200700 1 --2- 172.21.15.1:0/600012131 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7faa5c0383a0 0x7faa5c03a860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:18.408 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:18.106+0000 7faa84200700 1 -- 172.21.15.1:0/600012131 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa7c07be40 msgr2=0x7faa7c19c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:19:18.408 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:18.106+0000 7faa84200700 1 --2- 172.21.15.1:0/600012131 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa7c07be40 0x7faa7c19c560 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7faa6c003f00 tx=0x7faa6c011a30 comp rx=0 tx=0).stop 2024-02-02T15:19:18.408 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:18.106+0000 7faa84200700 1 -- 172.21.15.1:0/600012131 shutdown_connections 2024-02-02T15:19:18.408 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:18.106+0000 7faa84200700 1 --2- 172.21.15.1:0/600012131 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7faa5c0383a0 0x7faa5c03a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:18.408 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:18.106+0000 7faa84200700 1 --2- 172.21.15.1:0/600012131 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa7c07be40 0x7faa7c19c560 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:18.408 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:18.106+0000 7faa84200700 1 -- 172.21.15.1:0/600012131 >> 172.21.15.1:0/600012131 conn(0x7faa7c101c20 msgr2=0x7faa7c103370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:19:18.408 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:18.106+0000 7faa84200700 1 -- 172.21.15.1:0/600012131 shutdown_connections 2024-02-02T15:19:18.408 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:18.106+0000 7faa84200700 1 -- 172.21.15.1:0/600012131 wait complete. 2024-02-02T15:19:18.408 INFO:teuthology.orchestra.run.smithi001.stdout:Waiting for the mgr to restart... 2024-02-02T15:19:18.409 INFO:teuthology.orchestra.run.smithi001.stdout:Waiting for mgr epoch 9... 2024-02-02T15:19:19.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:19 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/600012131' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-02-02T15:19:19.190 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:19:18 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:19:18.922+0000 7fd8d6780200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-02T15:19:19.482 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:19:19 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:19:19.226+0000 7fd8d6780200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-02T15:19:19.482 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:19:19 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:19:19.339+0000 7fd8d6780200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-02T15:19:19.482 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:19:19 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:19:19.481+0000 7fd8d6780200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-02T15:19:19.865 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:19:19 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:19:19.691+0000 7fd8d6780200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-02T15:19:19.866 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:19:19 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:19:19.865+0000 7fd8d6780200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-02T15:19:20.190 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:19:19 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:19:19.979+0000 7fd8d6780200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-02T15:19:20.813 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:19:20 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:19:20.458+0000 7fd8d6780200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-02T15:19:21.173 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:19:20 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:19:20.813+0000 7fd8d6780200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-02T15:19:21.440 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:19:21 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:19:21.173+0000 7fd8d6780200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-02T15:19:21.440 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:19:21 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:19:21.300+0000 7fd8d6780200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-02T15:19:22.599 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:19:22 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:19:22.271+0000 7fd8d6780200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-02T15:19:22.600 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:19:22 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:19:22.400+0000 7fd8d6780200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-02T15:19:22.600 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:19:22 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:19:22.599+0000 7fd8d6780200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-02T15:19:23.690 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:19:23 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:19:23.434+0000 7fd8d6780200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-02T15:19:23.690 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:19:23 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:19:23.678+0000 7fd8d6780200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-02T15:19:24.190 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:19:23 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:19:23.936+0000 7fd8d6780200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-02T15:19:25.440 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:19:25 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:19:25.096+0000 7fd8d6780200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-02T15:19:25.833 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:19:25 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:19:25.458+0000 7fd8d6780200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-02T15:19:25.833 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:19:25 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:19:25.576+0000 7fd8d6780200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-02T15:19:26.190 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:19:25 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:19:25.833+0000 7fd8d6780200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-02T15:19:27.079 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:26 smithi001 ceph-mon[35470]: Active manager daemon x restarted 2024-02-02T15:19:27.079 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:26 smithi001 ceph-mon[35470]: Activating manager daemon x 2024-02-02T15:19:27.079 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:26 smithi001 ceph-mon[35470]: osdmap e3: 0 total, 0 up, 0 in 2024-02-02T15:19:27.080 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:26 smithi001 ceph-mon[35470]: mgrmap e10: x(active, starting, since 0.00406036s) 2024-02-02T15:19:27.080 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:26 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-02T15:19:27.080 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:26 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-02T15:19:27.080 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:26 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-02T15:19:27.080 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:26 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-02T15:19:27.080 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:26 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-02T15:19:27.080 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:26 smithi001 ceph-mon[35470]: Manager daemon x is now available 2024-02-02T15:19:27.080 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:26 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:19:27.080 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:26 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:19:27.080 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:26 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-02-02T15:19:27.080 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:26 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-02-02T15:19:27.225 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout { 2024-02-02T15:19:27.225 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-02-02T15:19:27.225 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout "initialized": true 2024-02-02T15:19:27.226 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout } 2024-02-02T15:19:27.226 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:18.891+0000 7f25c0c00700 1 Processor -- start 2024-02-02T15:19:27.226 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:18.891+0000 7f25c0c00700 1 -- start start 2024-02-02T15:19:27.226 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:18.891+0000 7f25c0c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f25bc106620 0x7f25bc106a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:19:27.226 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:18.891+0000 7f25c0c00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25bc107010 con 0x7f25bc106620 2024-02-02T15:19:27.226 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:18.891+0000 7f25b9e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f25bc106620 0x7f25bc106a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:19:27.226 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:18.891+0000 7f25b9e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f25bc106620 0x7f25bc106a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59856/0 (socket says 172.21.15.1:59856) 2024-02-02T15:19:27.226 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:18.892+0000 7f25b9e00700 1 -- 172.21.15.1:0/1546502685 learned_addr learned my addr 172.21.15.1:0/1546502685 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:19:27.226 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:18.892+0000 7f25b9e00700 1 -- 172.21.15.1:0/1546502685 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25bc107820 con 0x7f25bc106620 2024-02-02T15:19:27.226 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:18.892+0000 7f25b9e00700 1 --2- 172.21.15.1:0/1546502685 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f25bc106620 0x7f25bc106a40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f25ac0098a0 tx=0x7f25ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b3bc869f6d99e1 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:19:27.226 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:18.892+0000 7f25b8a00700 1 -- 172.21.15.1:0/1546502685 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f25ac004160 con 0x7f25bc106620 2024-02-02T15:19:27.226 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:18.892+0000 7f25b8a00700 1 -- 172.21.15.1:0/1546502685 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f25ac0042c0 con 0x7f25bc106620 2024-02-02T15:19:27.226 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:18.892+0000 7f25b8a00700 1 -- 172.21.15.1:0/1546502685 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f25ac0119b0 con 0x7f25bc106620 2024-02-02T15:19:27.227 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:18.893+0000 7f25c0c00700 1 -- 172.21.15.1:0/1546502685 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f25bc106620 msgr2=0x7f25bc106a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:19:27.227 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:18.893+0000 7f25c0c00700 1 --2- 172.21.15.1:0/1546502685 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f25bc106620 0x7f25bc106a40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f25ac0098a0 tx=0x7f25ac009bb0 comp rx=0 tx=0).stop 2024-02-02T15:19:27.227 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:18.893+0000 7f25c0c00700 1 -- 172.21.15.1:0/1546502685 shutdown_connections 2024-02-02T15:19:27.227 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:18.893+0000 7f25c0c00700 1 --2- 172.21.15.1:0/1546502685 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f25bc106620 0x7f25bc106a40 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:27.227 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:18.893+0000 7f25c0c00700 1 -- 172.21.15.1:0/1546502685 >> 172.21.15.1:0/1546502685 conn(0x7f25bc101c20 msgr2=0x7f25bc104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:19:27.227 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:18.893+0000 7f25c0c00700 1 -- 172.21.15.1:0/1546502685 shutdown_connections 2024-02-02T15:19:27.227 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:18.893+0000 7f25c0c00700 1 -- 172.21.15.1:0/1546502685 wait complete. 2024-02-02T15:19:27.227 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:18.894+0000 7f25c0c00700 1 Processor -- start 2024-02-02T15:19:27.227 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:18.894+0000 7f25c0c00700 1 -- start start 2024-02-02T15:19:27.227 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:18.895+0000 7f25c0c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f25bc106620 0x7f25bc19a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:19:27.227 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:18.895+0000 7f25c0c00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25bc19a8e0 con 0x7f25bc106620 2024-02-02T15:19:27.227 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:18.895+0000 7f25b9e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f25bc106620 0x7f25bc19a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:19:27.227 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:18.895+0000 7f25b9e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f25bc106620 0x7f25bc19a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59858/0 (socket says 172.21.15.1:59858) 2024-02-02T15:19:27.227 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:18.895+0000 7f25b9e00700 1 -- 172.21.15.1:0/517176669 learned_addr learned my addr 172.21.15.1:0/517176669 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:19:27.228 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:18.895+0000 7f25b9e00700 1 -- 172.21.15.1:0/517176669 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25ac009550 con 0x7f25bc106620 2024-02-02T15:19:27.228 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:18.896+0000 7f25b9e00700 1 --2- 172.21.15.1:0/517176669 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f25bc106620 0x7f25bc19a3a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f25ac006af0 tx=0x7f25ac011a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:19:27.228 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:18.896+0000 7f25b2a00700 1 -- 172.21.15.1:0/517176669 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f25ac003a40 con 0x7f25bc106620 2024-02-02T15:19:27.228 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:18.896+0000 7f25c0c00700 1 -- 172.21.15.1:0/517176669 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25bc19aae0 con 0x7f25bc106620 2024-02-02T15:19:27.228 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:18.896+0000 7f25b2a00700 1 -- 172.21.15.1:0/517176669 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f25ac003ba0 con 0x7f25bc106620 2024-02-02T15:19:27.228 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:18.896+0000 7f25b2a00700 1 -- 172.21.15.1:0/517176669 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f25ac019690 con 0x7f25bc106620 2024-02-02T15:19:27.228 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:18.896+0000 7f25c0c00700 1 -- 172.21.15.1:0/517176669 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25bc19af00 con 0x7f25bc106620 2024-02-02T15:19:27.228 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:18.897+0000 7f25b2a00700 1 -- 172.21.15.1:0/517176669 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 9) v1 ==== 45281+0+0 (secure 0 0 0) 0x7f25ac019900 con 0x7f25bc106620 2024-02-02T15:19:27.228 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:18.898+0000 7f25b2a00700 1 --2- 172.21.15.1:0/517176669 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7f259c038350 0x7f259c03a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:19:27.228 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:18.898+0000 7f25b2a00700 1 -- 172.21.15.1:0/517176669 --> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f259c03af20 con 0x7f259c038350 2024-02-02T15:19:27.228 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:18.898+0000 7f25b2a00700 1 -- 172.21.15.1:0/517176669 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f25ac04c320 con 0x7f25bc106620 2024-02-02T15:19:27.228 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:18.898+0000 7f25b9400700 1 -- 172.21.15.1:0/517176669 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7f259c038350 msgr2=0x7f259c03a810 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/395498296 2024-02-02T15:19:27.228 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:18.898+0000 7f25b9400700 1 --2- 172.21.15.1:0/517176669 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7f259c038350 0x7f259c03a810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-02T15:19:27.231 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:19.099+0000 7f25b9400700 1 -- 172.21.15.1:0/517176669 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7f259c038350 msgr2=0x7f259c03a810 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/395498296 2024-02-02T15:19:27.231 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:19.099+0000 7f25b9400700 1 --2- 172.21.15.1:0/517176669 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7f259c038350 0x7f259c03a810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-02T15:19:27.231 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:19.500+0000 7f25b9400700 1 -- 172.21.15.1:0/517176669 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7f259c038350 msgr2=0x7f259c03a810 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/395498296 2024-02-02T15:19:27.231 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:19.500+0000 7f25b9400700 1 --2- 172.21.15.1:0/517176669 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7f259c038350 0x7f259c03a810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-02-02T15:19:27.231 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:20.301+0000 7f25b9400700 1 -- 172.21.15.1:0/517176669 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7f259c038350 msgr2=0x7f259c03a810 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/395498296 2024-02-02T15:19:27.231 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:20.301+0000 7f25b9400700 1 --2- 172.21.15.1:0/517176669 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7f259c038350 0x7f259c03a810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-02-02T15:19:27.231 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:21.902+0000 7f25b9400700 1 -- 172.21.15.1:0/517176669 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7f259c038350 msgr2=0x7f259c03a810 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/395498296 2024-02-02T15:19:27.231 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:21.903+0000 7f25b9400700 1 --2- 172.21.15.1:0/517176669 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7f259c038350 0x7f259c03a810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-02-02T15:19:27.231 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:25.106+0000 7f25b9400700 1 -- 172.21.15.1:0/517176669 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7f259c038350 msgr2=0x7f259c03a810 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/395498296 2024-02-02T15:19:27.231 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:25.106+0000 7f25b9400700 1 --2- 172.21.15.1:0/517176669 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7f259c038350 0x7f259c03a810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-02-02T15:19:27.232 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:25.837+0000 7f25b2a00700 1 -- 172.21.15.1:0/517176669 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mgrmap(e 10) v1 ==== 45048+0+0 (secure 0 0 0) 0x7f25ac02dcd0 con 0x7f25bc106620 2024-02-02T15:19:27.232 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:25.837+0000 7f25b2a00700 1 -- 172.21.15.1:0/517176669 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7f259c038350 msgr2=0x7f259c03a810 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-02T15:19:27.232 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:25.837+0000 7f25b2a00700 1 --2- 172.21.15.1:0/517176669 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7f259c038350 0x7f259c03a810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:27.232 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:26.839+0000 7f25b2a00700 1 -- 172.21.15.1:0/517176669 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mgrmap(e 11) v1 ==== 45175+0+0 (secure 0 0 0) 0x7f25ac04ddf0 con 0x7f25bc106620 2024-02-02T15:19:27.232 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:26.840+0000 7f25b2a00700 1 --2- 172.21.15.1:0/517176669 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f259c03bdc0 0x7f259c03e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:19:27.232 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:26.840+0000 7f25b2a00700 1 -- 172.21.15.1:0/517176669 --> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f259c03af20 con 0x7f259c03bdc0 2024-02-02T15:19:27.232 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:26.840+0000 7f25b9400700 1 --2- 172.21.15.1:0/517176669 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f259c03bdc0 0x7f259c03e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:19:27.232 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:26.842+0000 7f25b9400700 1 --2- 172.21.15.1:0/517176669 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f259c03bdc0 0x7f259c03e1b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f25a4003a10 tx=0x7f25a4009270 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:19:27.232 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:26.843+0000 7f25b2a00700 1 -- 172.21.15.1:0/517176669 <== mgr.14154 v2:172.21.15.1:6800/3768989981 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7f259c03af20 con 0x7f259c03bdc0 2024-02-02T15:19:27.232 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:26.849+0000 7f25abe00700 1 -- 172.21.15.1:0/517176669 --> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f258c002600 con 0x7f259c03bdc0 2024-02-02T15:19:27.232 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:26.850+0000 7f25b2a00700 1 -- 172.21.15.1:0/517176669 <== mgr.14154 v2:172.21.15.1:6800/3768989981 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f258c002600 con 0x7f259c03bdc0 2024-02-02T15:19:27.232 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:26.850+0000 7f25abe00700 1 -- 172.21.15.1:0/517176669 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f259c03bdc0 msgr2=0x7f259c03e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:19:27.232 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:26.850+0000 7f25abe00700 1 --2- 172.21.15.1:0/517176669 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f259c03bdc0 0x7f259c03e1b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f25a4003a10 tx=0x7f25a4009270 comp rx=0 tx=0).stop 2024-02-02T15:19:27.232 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:26.850+0000 7f25abe00700 1 -- 172.21.15.1:0/517176669 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f25bc106620 msgr2=0x7f25bc19a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:19:27.233 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:26.850+0000 7f25abe00700 1 --2- 172.21.15.1:0/517176669 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f25bc106620 0x7f25bc19a3a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f25ac006af0 tx=0x7f25ac011a10 comp rx=0 tx=0).stop 2024-02-02T15:19:27.233 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:26.850+0000 7f25abe00700 1 -- 172.21.15.1:0/517176669 shutdown_connections 2024-02-02T15:19:27.233 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:26.850+0000 7f25abe00700 1 --2- 172.21.15.1:0/517176669 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f259c03bdc0 0x7f259c03e1b0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:27.233 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:26.850+0000 7f25abe00700 1 --2- 172.21.15.1:0/517176669 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f25bc106620 0x7f25bc19a3a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:27.233 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:26.850+0000 7f25abe00700 1 --2- 172.21.15.1:0/517176669 >> [v2:172.21.15.1:6800/395498296,v1:172.21.15.1:6801/395498296] conn(0x7f259c038350 0x7f259c03a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:27.233 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:26.850+0000 7f25abe00700 1 -- 172.21.15.1:0/517176669 >> 172.21.15.1:0/517176669 conn(0x7f25bc101c20 msgr2=0x7f25bc103310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:19:27.233 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:26.851+0000 7f25abe00700 1 -- 172.21.15.1:0/517176669 shutdown_connections 2024-02-02T15:19:27.233 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:26.851+0000 7f25abe00700 1 -- 172.21.15.1:0/517176669 wait complete. 2024-02-02T15:19:27.233 INFO:teuthology.orchestra.run.smithi001.stdout:mgr epoch 9 is available 2024-02-02T15:19:27.233 INFO:teuthology.orchestra.run.smithi001.stdout:Generating a dashboard self-signed certificate... 2024-02-02T15:19:28.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:27 smithi001 ceph-mon[35470]: mgrmap e11: x(active, since 1.00721s) 2024-02-02T15:19:28.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:27 smithi001 ceph-mon[35470]: from='client.14158 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-02-02T15:19:28.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:27 smithi001 ceph-mon[35470]: from='client.14158 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-02-02T15:19:28.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:27 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:19:28.412 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-02-02T15:19:28.412 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:27.655+0000 7f961b800700 1 Processor -- start 2024-02-02T15:19:28.412 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:27.655+0000 7f961b800700 1 -- start start 2024-02-02T15:19:28.412 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:27.655+0000 7f961b800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f961407be60 0x7f961407a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:19:28.412 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:27.655+0000 7f961b800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f961407a890 con 0x7f961407be60 2024-02-02T15:19:28.412 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:27.656+0000 7f9618e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f961407be60 0x7f961407a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:19:28.412 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:27.656+0000 7f9618e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f961407be60 0x7f961407a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36082/0 (socket says 172.21.15.1:36082) 2024-02-02T15:19:28.412 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:27.656+0000 7f9618e00700 1 -- 172.21.15.1:0/2631653490 learned_addr learned my addr 172.21.15.1:0/2631653490 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:19:28.412 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:27.656+0000 7f9618e00700 1 -- 172.21.15.1:0/2631653490 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f961407a9d0 con 0x7f961407be60 2024-02-02T15:19:28.412 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:27.657+0000 7f9618e00700 1 --2- 172.21.15.1:0/2631653490 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f961407be60 0x7f961407a2c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f96040098a0 tx=0x7f9604009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a8a46828e0edec0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:19:28.412 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:27.657+0000 7f9613400700 1 -- 172.21.15.1:0/2631653490 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9604004160 con 0x7f961407be60 2024-02-02T15:19:28.413 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:27.657+0000 7f9613400700 1 -- 172.21.15.1:0/2631653490 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f96040042c0 con 0x7f961407be60 2024-02-02T15:19:28.413 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:27.657+0000 7f9613400700 1 -- 172.21.15.1:0/2631653490 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f96040119b0 con 0x7f961407be60 2024-02-02T15:19:28.413 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:27.657+0000 7f961b800700 1 -- 172.21.15.1:0/2631653490 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f961407be60 msgr2=0x7f961407a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:19:28.413 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:27.657+0000 7f961b800700 1 --2- 172.21.15.1:0/2631653490 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f961407be60 0x7f961407a2c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f96040098a0 tx=0x7f9604009bb0 comp rx=0 tx=0).stop 2024-02-02T15:19:28.413 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:27.658+0000 7f961b800700 1 -- 172.21.15.1:0/2631653490 shutdown_connections 2024-02-02T15:19:28.413 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:27.658+0000 7f961b800700 1 --2- 172.21.15.1:0/2631653490 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f961407be60 0x7f961407a2c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:28.413 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:27.658+0000 7f961b800700 1 -- 172.21.15.1:0/2631653490 >> 172.21.15.1:0/2631653490 conn(0x7f9614101c20 msgr2=0x7f9614104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:19:28.413 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:27.658+0000 7f961b800700 1 -- 172.21.15.1:0/2631653490 shutdown_connections 2024-02-02T15:19:28.413 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:27.658+0000 7f961b800700 1 -- 172.21.15.1:0/2631653490 wait complete. 2024-02-02T15:19:28.413 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:27.659+0000 7f961b800700 1 Processor -- start 2024-02-02T15:19:28.413 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:27.660+0000 7f961b800700 1 -- start start 2024-02-02T15:19:28.413 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:27.660+0000 7f961b800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f961407be60 0x7f961419c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:19:28.413 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:27.660+0000 7f961b800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f961419cac0 con 0x7f961407be60 2024-02-02T15:19:28.413 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:27.660+0000 7f9618e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f961407be60 0x7f961419c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:19:28.414 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:27.660+0000 7f9618e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f961407be60 0x7f961419c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36086/0 (socket says 172.21.15.1:36086) 2024-02-02T15:19:28.414 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:27.660+0000 7f9618e00700 1 -- 172.21.15.1:0/2565625279 learned_addr learned my addr 172.21.15.1:0/2565625279 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:19:28.414 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:27.661+0000 7f9618e00700 1 -- 172.21.15.1:0/2565625279 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9604009550 con 0x7f961407be60 2024-02-02T15:19:28.414 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:27.661+0000 7f9618e00700 1 --2- 172.21.15.1:0/2565625279 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f961407be60 0x7f961419c580 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9604003f00 tx=0x7f9604011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:19:28.414 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:27.661+0000 7f9611600700 1 -- 172.21.15.1:0/2565625279 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9604003b20 con 0x7f961407be60 2024-02-02T15:19:28.414 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:27.661+0000 7f9611600700 1 -- 172.21.15.1:0/2565625279 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9604003c80 con 0x7f961407be60 2024-02-02T15:19:28.414 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:27.661+0000 7f961b800700 1 -- 172.21.15.1:0/2565625279 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f961419ccc0 con 0x7f961407be60 2024-02-02T15:19:28.414 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:27.661+0000 7f9611600700 1 -- 172.21.15.1:0/2565625279 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9604019770 con 0x7f961407be60 2024-02-02T15:19:28.414 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:27.661+0000 7f961b800700 1 -- 172.21.15.1:0/2565625279 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f961419d0e0 con 0x7f961407be60 2024-02-02T15:19:28.414 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:27.663+0000 7f960aa00700 1 -- 172.21.15.1:0/2565625279 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95e40052f0 con 0x7f961407be60 2024-02-02T15:19:28.414 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:27.663+0000 7f9611600700 1 -- 172.21.15.1:0/2565625279 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 11) v1 ==== 45175+0+0 (secure 0 0 0) 0x7f960401a430 con 0x7f961407be60 2024-02-02T15:19:28.414 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:27.663+0000 7f9611600700 1 --2- 172.21.15.1:0/2565625279 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f95f4038270 0x7f95f403a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:19:28.414 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:27.663+0000 7f9611600700 1 -- 172.21.15.1:0/2565625279 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f960404c3e0 con 0x7f961407be60 2024-02-02T15:19:28.414 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:27.664+0000 7f9613e00700 1 --2- 172.21.15.1:0/2565625279 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f95f4038270 0x7f95f403a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:19:28.415 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:27.664+0000 7f9613e00700 1 --2- 172.21.15.1:0/2565625279 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f95f4038270 0x7f95f403a730 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f95fc006fd0 tx=0x7f95fc006e40 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:19:28.415 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:27.669+0000 7f9611600700 1 -- 172.21.15.1:0/2565625279 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f960402d960 con 0x7f961407be60 2024-02-02T15:19:28.415 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:27.903+0000 7f960aa00700 1 -- 172.21.15.1:0/2565625279 --> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f95e4000bc0 con 0x7f95f4038270 2024-02-02T15:19:28.415 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:28.017+0000 7f9611600700 1 -- 172.21.15.1:0/2565625279 <== mgr.14154 v2:172.21.15.1:6800/3768989981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f95e4000bc0 con 0x7f95f4038270 2024-02-02T15:19:28.415 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:28.020+0000 7f960aa00700 1 -- 172.21.15.1:0/2565625279 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f95f4038270 msgr2=0x7f95f403a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:19:28.415 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:28.020+0000 7f960aa00700 1 --2- 172.21.15.1:0/2565625279 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f95f4038270 0x7f95f403a730 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f95fc006fd0 tx=0x7f95fc006e40 comp rx=0 tx=0).stop 2024-02-02T15:19:28.415 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:28.020+0000 7f960aa00700 1 -- 172.21.15.1:0/2565625279 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f961407be60 msgr2=0x7f961419c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:19:28.415 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:28.020+0000 7f960aa00700 1 --2- 172.21.15.1:0/2565625279 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f961407be60 0x7f961419c580 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9604003f00 tx=0x7f9604011a30 comp rx=0 tx=0).stop 2024-02-02T15:19:28.415 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:28.020+0000 7f960aa00700 1 -- 172.21.15.1:0/2565625279 shutdown_connections 2024-02-02T15:19:28.415 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:28.020+0000 7f960aa00700 1 --2- 172.21.15.1:0/2565625279 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f95f4038270 0x7f95f403a730 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:28.415 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:28.020+0000 7f960aa00700 1 --2- 172.21.15.1:0/2565625279 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f961407be60 0x7f961419c580 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:28.415 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:28.020+0000 7f960aa00700 1 -- 172.21.15.1:0/2565625279 >> 172.21.15.1:0/2565625279 conn(0x7f9614101c20 msgr2=0x7f9614103340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:19:28.415 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:28.020+0000 7f960aa00700 1 -- 172.21.15.1:0/2565625279 shutdown_connections 2024-02-02T15:19:28.415 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:28.020+0000 7f960aa00700 1 -- 172.21.15.1:0/2565625279 wait complete. 2024-02-02T15:19:28.415 INFO:teuthology.orchestra.run.smithi001.stdout:Creating initial admin user... 2024-02-02T15:19:29.271 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:29 smithi001 ceph-mon[35470]: [02/Feb/2024:15:19:27] ENGINE Bus STARTING 2024-02-02T15:19:29.271 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:29 smithi001 ceph-mon[35470]: [02/Feb/2024:15:19:27] ENGINE Serving on http://172.21.15.1:8765 2024-02-02T15:19:29.272 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:29 smithi001 ceph-mon[35470]: [02/Feb/2024:15:19:27] ENGINE Serving on https://172.21.15.1:7150 2024-02-02T15:19:29.272 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:29 smithi001 ceph-mon[35470]: [02/Feb/2024:15:19:27] ENGINE Bus STARTED 2024-02-02T15:19:29.272 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:29 smithi001 ceph-mon[35470]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:19:29.272 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:29 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:19:29.272 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:29 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:19:29.272 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:29 smithi001 ceph-mon[35470]: mgrmap e12: x(active, since 2s) 2024-02-02T15:19:29.272 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:29 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:19:30.013 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$o5JxIDu1OVaxbH6RlGfNPuRi0gF0aw17ZzsWVhaAqFOnfyCus4lMy", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1706887169, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-02-02T15:19:30.013 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:29.041+0000 7eff88600700 1 Processor -- start 2024-02-02T15:19:30.013 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:29.042+0000 7eff88600700 1 -- start start 2024-02-02T15:19:30.013 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:29.042+0000 7eff88600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff801070f0 0x7eff801094e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:19:30.013 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:29.042+0000 7eff88600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff80079e10 con 0x7eff801070f0 2024-02-02T15:19:30.014 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:29.042+0000 7eff85c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff801070f0 0x7eff801094e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:19:30.014 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:29.042+0000 7eff85c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff801070f0 0x7eff801094e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36092/0 (socket says 172.21.15.1:36092) 2024-02-02T15:19:30.014 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:29.042+0000 7eff85c00700 1 -- 172.21.15.1:0/4249258117 learned_addr learned my addr 172.21.15.1:0/4249258117 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:19:30.014 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:29.043+0000 7eff85c00700 1 -- 172.21.15.1:0/4249258117 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff80109a20 con 0x7eff801070f0 2024-02-02T15:19:30.014 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:29.043+0000 7eff85c00700 1 --2- 172.21.15.1:0/4249258117 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff801070f0 0x7eff801094e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7eff700098a0 tx=0x7eff70009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92886cbd2b16877b server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:19:30.014 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:29.043+0000 7eff7fe00700 1 -- 172.21.15.1:0/4249258117 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7eff70004160 con 0x7eff801070f0 2024-02-02T15:19:30.014 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:29.043+0000 7eff7fe00700 1 -- 172.21.15.1:0/4249258117 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff700042c0 con 0x7eff801070f0 2024-02-02T15:19:30.014 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:29.043+0000 7eff7fe00700 1 -- 172.21.15.1:0/4249258117 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7eff700119b0 con 0x7eff801070f0 2024-02-02T15:19:30.014 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:29.044+0000 7eff88600700 1 -- 172.21.15.1:0/4249258117 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff801070f0 msgr2=0x7eff801094e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:19:30.014 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:29.044+0000 7eff88600700 1 --2- 172.21.15.1:0/4249258117 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff801070f0 0x7eff801094e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7eff700098a0 tx=0x7eff70009bb0 comp rx=0 tx=0).stop 2024-02-02T15:19:30.014 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:29.044+0000 7eff88600700 1 -- 172.21.15.1:0/4249258117 shutdown_connections 2024-02-02T15:19:30.014 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:29.044+0000 7eff88600700 1 --2- 172.21.15.1:0/4249258117 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff801070f0 0x7eff801094e0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:30.014 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:29.044+0000 7eff88600700 1 -- 172.21.15.1:0/4249258117 >> 172.21.15.1:0/4249258117 conn(0x7eff80100ac0 msgr2=0x7eff80102ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:19:30.015 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:29.044+0000 7eff88600700 1 -- 172.21.15.1:0/4249258117 shutdown_connections 2024-02-02T15:19:30.015 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:29.044+0000 7eff88600700 1 -- 172.21.15.1:0/4249258117 wait complete. 2024-02-02T15:19:30.015 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:29.045+0000 7eff88600700 1 Processor -- start 2024-02-02T15:19:30.015 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:29.045+0000 7eff88600700 1 -- start start 2024-02-02T15:19:30.015 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:29.046+0000 7eff88600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff801070f0 0x7eff8019c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:19:30.015 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:29.046+0000 7eff88600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff8019caa0 con 0x7eff801070f0 2024-02-02T15:19:30.015 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:29.046+0000 7eff85c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff801070f0 0x7eff8019c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:19:30.015 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:29.046+0000 7eff85c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff801070f0 0x7eff8019c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36102/0 (socket says 172.21.15.1:36102) 2024-02-02T15:19:30.015 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:29.046+0000 7eff85c00700 1 -- 172.21.15.1:0/588381873 learned_addr learned my addr 172.21.15.1:0/588381873 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:19:30.015 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:29.046+0000 7eff85c00700 1 -- 172.21.15.1:0/588381873 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff70009550 con 0x7eff801070f0 2024-02-02T15:19:30.015 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:29.047+0000 7eff85c00700 1 --2- 172.21.15.1:0/588381873 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff801070f0 0x7eff8019c560 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7eff70003f00 tx=0x7eff70011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:19:30.015 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:29.047+0000 7eff7e000700 1 -- 172.21.15.1:0/588381873 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7eff70003b20 con 0x7eff801070f0 2024-02-02T15:19:30.015 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:29.047+0000 7eff7e000700 1 -- 172.21.15.1:0/588381873 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff70003c80 con 0x7eff801070f0 2024-02-02T15:19:30.015 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:29.047+0000 7eff7e000700 1 -- 172.21.15.1:0/588381873 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7eff70019770 con 0x7eff801070f0 2024-02-02T15:19:30.015 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:29.047+0000 7eff88600700 1 -- 172.21.15.1:0/588381873 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff8019cca0 con 0x7eff801070f0 2024-02-02T15:19:30.016 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:29.047+0000 7eff88600700 1 -- 172.21.15.1:0/588381873 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff8019d0c0 con 0x7eff801070f0 2024-02-02T15:19:30.016 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:29.048+0000 7eff7e000700 1 -- 172.21.15.1:0/588381873 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 12) v1 ==== 45281+0+0 (secure 0 0 0) 0x7eff7001a430 con 0x7eff801070f0 2024-02-02T15:19:30.016 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:29.048+0000 7eff88600700 1 -- 172.21.15.1:0/588381873 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff80062370 con 0x7eff801070f0 2024-02-02T15:19:30.016 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:29.048+0000 7eff7e000700 1 --2- 172.21.15.1:0/588381873 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7eff600383a0 0x7eff6003a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:19:30.016 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:29.049+0000 7eff7e000700 1 -- 172.21.15.1:0/588381873 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7eff7004c5e0 con 0x7eff801070f0 2024-02-02T15:19:30.016 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:29.049+0000 7eff85200700 1 --2- 172.21.15.1:0/588381873 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7eff600383a0 0x7eff6003a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:19:30.016 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:29.050+0000 7eff85200700 1 --2- 172.21.15.1:0/588381873 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7eff600383a0 0x7eff6003a860 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7eff68006fd0 tx=0x7eff68006e40 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:19:30.016 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:29.054+0000 7eff7e000700 1 -- 172.21.15.1:0/588381873 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7eff7002d9a0 con 0x7eff801070f0 2024-02-02T15:19:30.016 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:29.288+0000 7eff88600700 1 -- 172.21.15.1:0/588381873 --> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7eff8019fbc0 con 0x7eff600383a0 2024-02-02T15:19:30.016 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:29.536+0000 7eff7e000700 1 -- 172.21.15.1:0/588381873 <== mgr.14154 v2:172.21.15.1:6800/3768989981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7eff8019fbc0 con 0x7eff600383a0 2024-02-02T15:19:30.016 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:29.538+0000 7eff88600700 1 -- 172.21.15.1:0/588381873 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7eff600383a0 msgr2=0x7eff6003a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:19:30.016 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:29.538+0000 7eff88600700 1 --2- 172.21.15.1:0/588381873 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7eff600383a0 0x7eff6003a860 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7eff68006fd0 tx=0x7eff68006e40 comp rx=0 tx=0).stop 2024-02-02T15:19:30.016 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:29.538+0000 7eff88600700 1 -- 172.21.15.1:0/588381873 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff801070f0 msgr2=0x7eff8019c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:19:30.017 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:29.538+0000 7eff88600700 1 --2- 172.21.15.1:0/588381873 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff801070f0 0x7eff8019c560 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7eff70003f00 tx=0x7eff70011a30 comp rx=0 tx=0).stop 2024-02-02T15:19:30.017 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:29.538+0000 7eff88600700 1 -- 172.21.15.1:0/588381873 shutdown_connections 2024-02-02T15:19:30.017 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:29.538+0000 7eff88600700 1 --2- 172.21.15.1:0/588381873 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7eff600383a0 0x7eff6003a860 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:30.017 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:29.538+0000 7eff88600700 1 --2- 172.21.15.1:0/588381873 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff801070f0 0x7eff8019c560 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:30.017 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:29.538+0000 7eff88600700 1 -- 172.21.15.1:0/588381873 >> 172.21.15.1:0/588381873 conn(0x7eff80100ac0 msgr2=0x7eff801021a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:19:30.017 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:29.538+0000 7eff88600700 1 -- 172.21.15.1:0/588381873 shutdown_connections 2024-02-02T15:19:30.017 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:29.538+0000 7eff88600700 1 -- 172.21.15.1:0/588381873 wait complete. 2024-02-02T15:19:30.017 INFO:teuthology.orchestra.run.smithi001.stdout:Fetching dashboard port number... 2024-02-02T15:19:30.861 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:30 smithi001 ceph-mon[35470]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:19:30.861 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:30 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:19:31.008 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stdout 8443 2024-02-02T15:19:31.008 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:30.444+0000 7f202c400700 1 Processor -- start 2024-02-02T15:19:31.008 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:30.444+0000 7f202c400700 1 -- start start 2024-02-02T15:19:31.008 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:30.445+0000 7f202c400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f202407be60 0x7f202407a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:19:31.008 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:30.445+0000 7f202c400700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f202407a890 con 0x7f202407be60 2024-02-02T15:19:31.008 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:30.445+0000 7f2029a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f202407be60 0x7f202407a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:19:31.008 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:30.445+0000 7f2029a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f202407be60 0x7f202407a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36106/0 (socket says 172.21.15.1:36106) 2024-02-02T15:19:31.008 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:30.445+0000 7f2029a00700 1 -- 172.21.15.1:0/4038873928 learned_addr learned my addr 172.21.15.1:0/4038873928 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:19:31.008 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:30.445+0000 7f2029a00700 1 -- 172.21.15.1:0/4038873928 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f202407a9d0 con 0x7f202407be60 2024-02-02T15:19:31.008 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:30.446+0000 7f2029a00700 1 --2- 172.21.15.1:0/4038873928 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f202407be60 0x7f202407a2c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f20140098a0 tx=0x7f2014009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=187e567dbac7d533 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:19:31.009 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:30.446+0000 7f2023e00700 1 -- 172.21.15.1:0/4038873928 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2014004160 con 0x7f202407be60 2024-02-02T15:19:31.009 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:30.446+0000 7f2023e00700 1 -- 172.21.15.1:0/4038873928 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f20140042c0 con 0x7f202407be60 2024-02-02T15:19:31.009 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:30.446+0000 7f2023e00700 1 -- 172.21.15.1:0/4038873928 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f20140119b0 con 0x7f202407be60 2024-02-02T15:19:31.009 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:30.447+0000 7f202c400700 1 -- 172.21.15.1:0/4038873928 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f202407be60 msgr2=0x7f202407a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:19:31.009 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:30.447+0000 7f202c400700 1 --2- 172.21.15.1:0/4038873928 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f202407be60 0x7f202407a2c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f20140098a0 tx=0x7f2014009bb0 comp rx=0 tx=0).stop 2024-02-02T15:19:31.009 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:30.447+0000 7f202c400700 1 -- 172.21.15.1:0/4038873928 shutdown_connections 2024-02-02T15:19:31.009 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:30.447+0000 7f202c400700 1 --2- 172.21.15.1:0/4038873928 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f202407be60 0x7f202407a2c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:31.009 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:30.447+0000 7f202c400700 1 -- 172.21.15.1:0/4038873928 >> 172.21.15.1:0/4038873928 conn(0x7f2024101c20 msgr2=0x7f2024104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:19:31.009 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:30.447+0000 7f202c400700 1 -- 172.21.15.1:0/4038873928 shutdown_connections 2024-02-02T15:19:31.009 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:30.447+0000 7f202c400700 1 -- 172.21.15.1:0/4038873928 wait complete. 2024-02-02T15:19:31.009 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:30.448+0000 7f202c400700 1 Processor -- start 2024-02-02T15:19:31.009 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:30.449+0000 7f202c400700 1 -- start start 2024-02-02T15:19:31.009 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:30.449+0000 7f202c400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f202407be60 0x7f202419c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:19:31.009 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:30.449+0000 7f202c400700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f202419cac0 con 0x7f202407be60 2024-02-02T15:19:31.009 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:30.449+0000 7f2029a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f202407be60 0x7f202419c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:19:31.010 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:30.449+0000 7f2029a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f202407be60 0x7f202419c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36122/0 (socket says 172.21.15.1:36122) 2024-02-02T15:19:31.010 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:30.449+0000 7f2029a00700 1 -- 172.21.15.1:0/3467023355 learned_addr learned my addr 172.21.15.1:0/3467023355 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:19:31.010 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:30.450+0000 7f2029a00700 1 -- 172.21.15.1:0/3467023355 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2014009550 con 0x7f202407be60 2024-02-02T15:19:31.010 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:30.450+0000 7f2029a00700 1 --2- 172.21.15.1:0/3467023355 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f202407be60 0x7f202419c580 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2014003f00 tx=0x7f2014011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:19:31.010 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:30.450+0000 7f2022000700 1 -- 172.21.15.1:0/3467023355 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2014003b20 con 0x7f202407be60 2024-02-02T15:19:31.010 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:30.450+0000 7f2022000700 1 -- 172.21.15.1:0/3467023355 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2014003c80 con 0x7f202407be60 2024-02-02T15:19:31.010 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:30.450+0000 7f202c400700 1 -- 172.21.15.1:0/3467023355 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f202419ccc0 con 0x7f202407be60 2024-02-02T15:19:31.010 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:30.450+0000 7f2022000700 1 -- 172.21.15.1:0/3467023355 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2014019770 con 0x7f202407be60 2024-02-02T15:19:31.010 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:30.450+0000 7f202c400700 1 -- 172.21.15.1:0/3467023355 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f202419d0e0 con 0x7f202407be60 2024-02-02T15:19:31.010 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:30.451+0000 7f2022000700 1 -- 172.21.15.1:0/3467023355 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 12) v1 ==== 45281+0+0 (secure 0 0 0) 0x7f201401ab60 con 0x7f202407be60 2024-02-02T15:19:31.010 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:30.451+0000 7f202c400700 1 -- 172.21.15.1:0/3467023355 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2024062370 con 0x7f202407be60 2024-02-02T15:19:31.010 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:30.452+0000 7f2022000700 1 --2- 172.21.15.1:0/3467023355 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f20040383a0 0x7f200403a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:19:31.010 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:30.452+0000 7f2022000700 1 -- 172.21.15.1:0/3467023355 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f201404c620 con 0x7f202407be60 2024-02-02T15:19:31.010 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:30.452+0000 7f2029000700 1 --2- 172.21.15.1:0/3467023355 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f20040383a0 0x7f200403a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:19:31.011 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:30.452+0000 7f2029000700 1 --2- 172.21.15.1:0/3467023355 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f20040383a0 0x7f200403a860 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f200c006fd0 tx=0x7f200c006e40 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:19:31.011 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:30.457+0000 7f2022000700 1 -- 172.21.15.1:0/3467023355 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f201401a430 con 0x7f202407be60 2024-02-02T15:19:31.011 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:30.664+0000 7f202c400700 1 -- 172.21.15.1:0/3467023355 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f202419fb20 con 0x7f202407be60 2024-02-02T15:19:31.011 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:30.664+0000 7f2022000700 1 -- 172.21.15.1:0/3467023355 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f201402db00 con 0x7f202407be60 2024-02-02T15:19:31.011 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:30.667+0000 7f202c400700 1 -- 172.21.15.1:0/3467023355 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f20040383a0 msgr2=0x7f200403a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:19:31.011 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:30.667+0000 7f202c400700 1 --2- 172.21.15.1:0/3467023355 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f20040383a0 0x7f200403a860 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f200c006fd0 tx=0x7f200c006e40 comp rx=0 tx=0).stop 2024-02-02T15:19:31.011 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:30.667+0000 7f202c400700 1 -- 172.21.15.1:0/3467023355 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f202407be60 msgr2=0x7f202419c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:19:31.011 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:30.667+0000 7f202c400700 1 --2- 172.21.15.1:0/3467023355 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f202407be60 0x7f202419c580 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2014003f00 tx=0x7f2014011a30 comp rx=0 tx=0).stop 2024-02-02T15:19:31.011 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:30.667+0000 7f202c400700 1 -- 172.21.15.1:0/3467023355 shutdown_connections 2024-02-02T15:19:31.011 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:30.667+0000 7f202c400700 1 --2- 172.21.15.1:0/3467023355 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f20040383a0 0x7f200403a860 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:31.011 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:30.667+0000 7f202c400700 1 --2- 172.21.15.1:0/3467023355 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f202407be60 0x7f202419c580 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:31.011 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:30.667+0000 7f202c400700 1 -- 172.21.15.1:0/3467023355 >> 172.21.15.1:0/3467023355 conn(0x7f2024101c20 msgr2=0x7f2024103340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:19:31.011 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:30.667+0000 7f202c400700 1 -- 172.21.15.1:0/3467023355 shutdown_connections 2024-02-02T15:19:31.011 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:30.667+0000 7f202c400700 1 -- 172.21.15.1:0/3467023355 wait complete. 2024-02-02T15:19:31.026 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-02T15:19:31.027 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout disabled 2024-02-02T15:19:31.043 INFO:teuthology.orchestra.run.smithi001.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-02T15:19:31.043 INFO:teuthology.orchestra.run.smithi001.stdout:systemctl: stdout inactive 2024-02-02T15:19:31.043 INFO:teuthology.orchestra.run.smithi001.stdout:firewalld.service is not enabled 2024-02-02T15:19:31.043 INFO:teuthology.orchestra.run.smithi001.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-02-02T15:19:31.045 INFO:teuthology.orchestra.run.smithi001.stdout:Ceph Dashboard is now available at: 2024-02-02T15:19:31.046 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:19:31.046 INFO:teuthology.orchestra.run.smithi001.stdout: URL: https://smithi001.front.sepia.ceph.com:8443/ 2024-02-02T15:19:31.046 INFO:teuthology.orchestra.run.smithi001.stdout: User: admin 2024-02-02T15:19:31.046 INFO:teuthology.orchestra.run.smithi001.stdout: Password: jcjos67j2q 2024-02-02T15:19:31.046 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:19:31.046 INFO:teuthology.orchestra.run.smithi001.stdout:Saving cluster configuration to /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/config directory 2024-02-02T15:19:31.047 INFO:teuthology.orchestra.run.smithi001.stdout:Enabling autotune for osd_memory_target 2024-02-02T15:19:31.869 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:31 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/3467023355' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-02-02T15:19:32.035 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:31.469+0000 7fa92d600700 1 Processor -- start 2024-02-02T15:19:32.035 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:31.470+0000 7fa92d600700 1 -- start start 2024-02-02T15:19:32.035 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:31.470+0000 7fa92d600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa92807be60 0x7fa92807a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:19:32.035 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:31.470+0000 7fa92d600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa92807a890 con 0x7fa92807be60 2024-02-02T15:19:32.036 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:31.470+0000 7fa926a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa92807be60 0x7fa92807a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:19:32.036 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:31.470+0000 7fa926a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa92807be60 0x7fa92807a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36136/0 (socket says 172.21.15.1:36136) 2024-02-02T15:19:32.036 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:31.470+0000 7fa926a00700 1 -- 172.21.15.1:0/1123767724 learned_addr learned my addr 172.21.15.1:0/1123767724 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:19:32.036 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:31.471+0000 7fa926a00700 1 -- 172.21.15.1:0/1123767724 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa92807a9d0 con 0x7fa92807be60 2024-02-02T15:19:32.036 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:31.471+0000 7fa926a00700 1 --2- 172.21.15.1:0/1123767724 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa92807be60 0x7fa92807a2c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa9180098a0 tx=0x7fa918009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=817c41a10382abe7 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:19:32.036 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:31.471+0000 7fa925600700 1 -- 172.21.15.1:0/1123767724 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa918004160 con 0x7fa92807be60 2024-02-02T15:19:32.036 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:31.471+0000 7fa925600700 1 -- 172.21.15.1:0/1123767724 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa9180042c0 con 0x7fa92807be60 2024-02-02T15:19:32.036 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:31.471+0000 7fa925600700 1 -- 172.21.15.1:0/1123767724 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa9180119b0 con 0x7fa92807be60 2024-02-02T15:19:32.036 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:31.472+0000 7fa92d600700 1 -- 172.21.15.1:0/1123767724 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa92807be60 msgr2=0x7fa92807a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:19:32.036 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:31.472+0000 7fa92d600700 1 --2- 172.21.15.1:0/1123767724 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa92807be60 0x7fa92807a2c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa9180098a0 tx=0x7fa918009bb0 comp rx=0 tx=0).stop 2024-02-02T15:19:32.036 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:31.472+0000 7fa92d600700 1 -- 172.21.15.1:0/1123767724 shutdown_connections 2024-02-02T15:19:32.036 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:31.472+0000 7fa92d600700 1 --2- 172.21.15.1:0/1123767724 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa92807be60 0x7fa92807a2c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:32.036 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:31.472+0000 7fa92d600700 1 -- 172.21.15.1:0/1123767724 >> 172.21.15.1:0/1123767724 conn(0x7fa928101c20 msgr2=0x7fa928104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:19:32.037 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:31.472+0000 7fa92d600700 1 -- 172.21.15.1:0/1123767724 shutdown_connections 2024-02-02T15:19:32.037 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:31.472+0000 7fa92d600700 1 -- 172.21.15.1:0/1123767724 wait complete. 2024-02-02T15:19:32.037 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:31.473+0000 7fa92d600700 1 Processor -- start 2024-02-02T15:19:32.037 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:31.473+0000 7fa92d600700 1 -- start start 2024-02-02T15:19:32.037 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:31.473+0000 7fa92d600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa92807be60 0x7fa92819c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:19:32.037 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:31.473+0000 7fa92d600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa92819cac0 con 0x7fa92807be60 2024-02-02T15:19:32.037 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:31.473+0000 7fa926a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa92807be60 0x7fa92819c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:19:32.037 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:31.473+0000 7fa926a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa92807be60 0x7fa92819c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36148/0 (socket says 172.21.15.1:36148) 2024-02-02T15:19:32.037 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:31.474+0000 7fa926a00700 1 -- 172.21.15.1:0/1308560473 learned_addr learned my addr 172.21.15.1:0/1308560473 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:19:32.037 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:31.474+0000 7fa926a00700 1 -- 172.21.15.1:0/1308560473 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa918009550 con 0x7fa92807be60 2024-02-02T15:19:32.037 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:31.474+0000 7fa926a00700 1 --2- 172.21.15.1:0/1308560473 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa92807be60 0x7fa92819c580 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa918003f00 tx=0x7fa918011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:19:32.037 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:31.474+0000 7fa91f400700 1 -- 172.21.15.1:0/1308560473 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa918003b20 con 0x7fa92807be60 2024-02-02T15:19:32.037 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:31.474+0000 7fa91f400700 1 -- 172.21.15.1:0/1308560473 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa918003c80 con 0x7fa92807be60 2024-02-02T15:19:32.038 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:31.474+0000 7fa91f400700 1 -- 172.21.15.1:0/1308560473 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa918019770 con 0x7fa92807be60 2024-02-02T15:19:32.038 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:31.474+0000 7fa92d600700 1 -- 172.21.15.1:0/1308560473 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa92819ccc0 con 0x7fa92807be60 2024-02-02T15:19:32.038 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:31.474+0000 7fa92d600700 1 -- 172.21.15.1:0/1308560473 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa92819d0e0 con 0x7fa92807be60 2024-02-02T15:19:32.038 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:31.475+0000 7fa91f400700 1 -- 172.21.15.1:0/1308560473 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 12) v1 ==== 45281+0+0 (secure 0 0 0) 0x7fa91801ab60 con 0x7fa92807be60 2024-02-02T15:19:32.038 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:31.475+0000 7fa92d600700 1 -- 172.21.15.1:0/1308560473 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa928062370 con 0x7fa92807be60 2024-02-02T15:19:32.038 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:31.476+0000 7fa91f400700 1 --2- 172.21.15.1:0/1308560473 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fa9080383a0 0x7fa90803a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:19:32.038 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:31.476+0000 7fa91f400700 1 -- 172.21.15.1:0/1308560473 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa91804c620 con 0x7fa92807be60 2024-02-02T15:19:32.038 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:31.476+0000 7fa926000700 1 --2- 172.21.15.1:0/1308560473 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fa9080383a0 0x7fa90803a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:19:32.038 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:31.477+0000 7fa926000700 1 --2- 172.21.15.1:0/1308560473 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fa9080383a0 0x7fa90803a860 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa910006fd0 tx=0x7fa910006e40 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:19:32.038 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:31.481+0000 7fa91f400700 1 -- 172.21.15.1:0/1308560473 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7fa91801a430 con 0x7fa92807be60 2024-02-02T15:19:32.038 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:31.689+0000 7fa92d600700 1 -- 172.21.15.1:0/1308560473 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7fa92819fb20 con 0x7fa92807be60 2024-02-02T15:19:32.038 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:31.690+0000 7fa91f400700 1 -- 172.21.15.1:0/1308560473 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7fa91802db00 con 0x7fa92807be60 2024-02-02T15:19:32.038 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:31.692+0000 7fa92d600700 1 -- 172.21.15.1:0/1308560473 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fa9080383a0 msgr2=0x7fa90803a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:19:32.038 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:31.692+0000 7fa92d600700 1 --2- 172.21.15.1:0/1308560473 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fa9080383a0 0x7fa90803a860 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa910006fd0 tx=0x7fa910006e40 comp rx=0 tx=0).stop 2024-02-02T15:19:32.039 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:31.692+0000 7fa92d600700 1 -- 172.21.15.1:0/1308560473 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa92807be60 msgr2=0x7fa92819c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:19:32.039 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:31.692+0000 7fa92d600700 1 --2- 172.21.15.1:0/1308560473 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa92807be60 0x7fa92819c580 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa918003f00 tx=0x7fa918011a30 comp rx=0 tx=0).stop 2024-02-02T15:19:32.039 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:31.692+0000 7fa92d600700 1 -- 172.21.15.1:0/1308560473 shutdown_connections 2024-02-02T15:19:32.039 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:31.692+0000 7fa92d600700 1 --2- 172.21.15.1:0/1308560473 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fa9080383a0 0x7fa90803a860 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:32.039 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:31.692+0000 7fa92d600700 1 --2- 172.21.15.1:0/1308560473 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa92807be60 0x7fa92819c580 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:32.039 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:31.692+0000 7fa92d600700 1 -- 172.21.15.1:0/1308560473 >> 172.21.15.1:0/1308560473 conn(0x7fa928101c20 msgr2=0x7fa928103340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:19:32.039 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:31.692+0000 7fa92d600700 1 -- 172.21.15.1:0/1308560473 shutdown_connections 2024-02-02T15:19:32.039 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:31.692+0000 7fa92d600700 1 -- 172.21.15.1:0/1308560473 wait complete. 2024-02-02T15:19:33.079 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:32.401+0000 7f2ea4e00700 1 Processor -- start 2024-02-02T15:19:33.079 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:32.401+0000 7f2ea4e00700 1 -- start start 2024-02-02T15:19:33.080 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:32.402+0000 7f2ea4e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2ea007be60 0x7f2ea007a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:19:33.080 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:32.402+0000 7f2ea4e00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea007a890 con 0x7f2ea007be60 2024-02-02T15:19:33.080 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:32.402+0000 7f2e9de00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2ea007be60 0x7f2ea007a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:19:33.080 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:32.402+0000 7f2e9de00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2ea007be60 0x7f2ea007a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36160/0 (socket says 172.21.15.1:36160) 2024-02-02T15:19:33.080 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:32.402+0000 7f2e9de00700 1 -- 172.21.15.1:0/3152321944 learned_addr learned my addr 172.21.15.1:0/3152321944 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:19:33.080 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:32.403+0000 7f2e9de00700 1 -- 172.21.15.1:0/3152321944 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ea007a9d0 con 0x7f2ea007be60 2024-02-02T15:19:33.080 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:32.403+0000 7f2e9de00700 1 --2- 172.21.15.1:0/3152321944 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2ea007be60 0x7f2ea007a2c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2e900098a0 tx=0x7f2e90009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3478c38af6f9604d server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:19:33.081 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:32.403+0000 7f2e9d400700 1 -- 172.21.15.1:0/3152321944 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2e90004160 con 0x7f2ea007be60 2024-02-02T15:19:33.081 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:32.403+0000 7f2e9d400700 1 -- 172.21.15.1:0/3152321944 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e900042c0 con 0x7f2ea007be60 2024-02-02T15:19:33.081 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:32.403+0000 7f2e9d400700 1 -- 172.21.15.1:0/3152321944 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2e900119b0 con 0x7f2ea007be60 2024-02-02T15:19:33.081 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:32.404+0000 7f2ea4e00700 1 -- 172.21.15.1:0/3152321944 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2ea007be60 msgr2=0x7f2ea007a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:19:33.081 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:32.404+0000 7f2ea4e00700 1 --2- 172.21.15.1:0/3152321944 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2ea007be60 0x7f2ea007a2c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2e900098a0 tx=0x7f2e90009bb0 comp rx=0 tx=0).stop 2024-02-02T15:19:33.081 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:32.404+0000 7f2ea4e00700 1 -- 172.21.15.1:0/3152321944 shutdown_connections 2024-02-02T15:19:33.081 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:32.404+0000 7f2ea4e00700 1 --2- 172.21.15.1:0/3152321944 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2ea007be60 0x7f2ea007a2c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:33.081 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:32.404+0000 7f2ea4e00700 1 -- 172.21.15.1:0/3152321944 >> 172.21.15.1:0/3152321944 conn(0x7f2ea0101c20 msgr2=0x7f2ea0104040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:19:33.081 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:32.404+0000 7f2ea4e00700 1 -- 172.21.15.1:0/3152321944 shutdown_connections 2024-02-02T15:19:33.081 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:32.404+0000 7f2ea4e00700 1 -- 172.21.15.1:0/3152321944 wait complete. 2024-02-02T15:19:33.081 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:32.405+0000 7f2ea4e00700 1 Processor -- start 2024-02-02T15:19:33.081 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:32.405+0000 7f2ea4e00700 1 -- start start 2024-02-02T15:19:33.081 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:32.405+0000 7f2ea4e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2ea007be60 0x7f2ea019c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:19:33.082 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:32.405+0000 7f2ea4e00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ea019cac0 con 0x7f2ea007be60 2024-02-02T15:19:33.082 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:32.406+0000 7f2e9de00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2ea007be60 0x7f2ea019c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:19:33.082 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:32.406+0000 7f2e9de00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2ea007be60 0x7f2ea019c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36176/0 (socket says 172.21.15.1:36176) 2024-02-02T15:19:33.082 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:32.406+0000 7f2e9de00700 1 -- 172.21.15.1:0/4062561306 learned_addr learned my addr 172.21.15.1:0/4062561306 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:19:33.082 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:32.406+0000 7f2e9de00700 1 -- 172.21.15.1:0/4062561306 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e90009550 con 0x7f2ea007be60 2024-02-02T15:19:33.082 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:32.406+0000 7f2e9de00700 1 --2- 172.21.15.1:0/4062561306 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2ea007be60 0x7f2ea019c580 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2e90003f00 tx=0x7f2e90011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:19:33.082 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:32.406+0000 7f2e96a00700 1 -- 172.21.15.1:0/4062561306 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2e90003b20 con 0x7f2ea007be60 2024-02-02T15:19:33.082 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:32.406+0000 7f2e96a00700 1 -- 172.21.15.1:0/4062561306 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e90003c80 con 0x7f2ea007be60 2024-02-02T15:19:33.082 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:32.406+0000 7f2e96a00700 1 -- 172.21.15.1:0/4062561306 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2e90019770 con 0x7f2ea007be60 2024-02-02T15:19:33.082 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:32.407+0000 7f2ea4e00700 1 -- 172.21.15.1:0/4062561306 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ea019ccc0 con 0x7f2ea007be60 2024-02-02T15:19:33.082 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:32.407+0000 7f2ea4e00700 1 -- 172.21.15.1:0/4062561306 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ea019d0e0 con 0x7f2ea007be60 2024-02-02T15:19:33.082 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:32.408+0000 7f2e96a00700 1 -- 172.21.15.1:0/4062561306 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 12) v1 ==== 45281+0+0 (secure 0 0 0) 0x7f2e9001ab60 con 0x7f2ea007be60 2024-02-02T15:19:33.082 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:32.408+0000 7f2ea4e00700 1 -- 172.21.15.1:0/4062561306 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ea010ca60 con 0x7f2ea007be60 2024-02-02T15:19:33.082 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:32.408+0000 7f2e96a00700 1 --2- 172.21.15.1:0/4062561306 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f2e80038350 0x7f2e8003a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:19:33.083 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:32.408+0000 7f2e96a00700 1 -- 172.21.15.1:0/4062561306 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f2e9004c670 con 0x7f2ea007be60 2024-02-02T15:19:33.083 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:32.408+0000 7f2e97e00700 1 --2- 172.21.15.1:0/4062561306 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f2e80038350 0x7f2e8003a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:19:33.083 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:32.409+0000 7f2e97e00700 1 --2- 172.21.15.1:0/4062561306 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f2e80038350 0x7f2e8003a810 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2e88006fd0 tx=0x7f2e88006e40 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:19:33.083 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:32.413+0000 7f2e96a00700 1 -- 172.21.15.1:0/4062561306 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f2e9001a430 con 0x7f2ea007be60 2024-02-02T15:19:33.083 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:32.538+0000 7f2e96a00700 1 -- 172.21.15.1:0/4062561306 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mgrmap(e 13) v1 ==== 45323+0+0 (secure 0 0 0) 0x7f2e9002bd40 con 0x7f2ea007be60 2024-02-02T15:19:33.083 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:32.732+0000 7f2ea4e00700 1 -- 172.21.15.1:0/4062561306 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f2ea010cc20 con 0x7f2ea007be60 2024-02-02T15:19:33.083 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:32.734+0000 7f2e96a00700 1 -- 172.21.15.1:0/4062561306 <== mon.0 v2:172.21.15.1:3300/0 8 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (secure 0 0 0) 0x7f2e9004eaf0 con 0x7f2ea007be60 2024-02-02T15:19:33.083 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:32.736+0000 7f2ea4e00700 1 -- 172.21.15.1:0/4062561306 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f2e80038350 msgr2=0x7f2e8003a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:19:33.083 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:32.736+0000 7f2ea4e00700 1 --2- 172.21.15.1:0/4062561306 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f2e80038350 0x7f2e8003a810 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2e88006fd0 tx=0x7f2e88006e40 comp rx=0 tx=0).stop 2024-02-02T15:19:33.083 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:32.736+0000 7f2ea4e00700 1 -- 172.21.15.1:0/4062561306 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2ea007be60 msgr2=0x7f2ea019c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:19:33.083 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:32.736+0000 7f2ea4e00700 1 --2- 172.21.15.1:0/4062561306 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2ea007be60 0x7f2ea019c580 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2e90003f00 tx=0x7f2e90011a30 comp rx=0 tx=0).stop 2024-02-02T15:19:33.083 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:32.737+0000 7f2ea4e00700 1 -- 172.21.15.1:0/4062561306 shutdown_connections 2024-02-02T15:19:33.084 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:32.737+0000 7f2ea4e00700 1 --2- 172.21.15.1:0/4062561306 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f2e80038350 0x7f2e8003a810 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:33.084 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:32.737+0000 7f2ea4e00700 1 --2- 172.21.15.1:0/4062561306 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2ea007be60 0x7f2ea019c580 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:33.084 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:32.737+0000 7f2ea4e00700 1 -- 172.21.15.1:0/4062561306 >> 172.21.15.1:0/4062561306 conn(0x7f2ea0101c20 msgr2=0x7f2ea0103340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:19:33.084 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:32.737+0000 7f2ea4e00700 1 -- 172.21.15.1:0/4062561306 shutdown_connections 2024-02-02T15:19:33.084 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr 2024-02-02T15:19:32.737+0000 7f2ea4e00700 1 -- 172.21.15.1:0/4062561306 wait complete. 2024-02-02T15:19:33.084 INFO:teuthology.orchestra.run.smithi001.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-02-02T15:19:33.084 INFO:teuthology.orchestra.run.smithi001.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-02-02T15:19:33.084 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:19:33.084 INFO:teuthology.orchestra.run.smithi001.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-02-02T15:19:33.084 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:19:33.084 INFO:teuthology.orchestra.run.smithi001.stdout:Or, if you are only running a single cluster on this host: 2024-02-02T15:19:33.084 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:19:33.084 INFO:teuthology.orchestra.run.smithi001.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-02-02T15:19:33.085 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:19:33.085 INFO:teuthology.orchestra.run.smithi001.stdout:Please consider enabling telemetry to help improve Ceph: 2024-02-02T15:19:33.085 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:19:33.085 INFO:teuthology.orchestra.run.smithi001.stdout: ceph telemetry on 2024-02-02T15:19:33.085 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:19:33.085 INFO:teuthology.orchestra.run.smithi001.stdout:For more information see: 2024-02-02T15:19:33.085 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:19:33.085 INFO:teuthology.orchestra.run.smithi001.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-02-02T15:19:33.085 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:19:33.085 INFO:teuthology.orchestra.run.smithi001.stdout:Bootstrap complete. 2024-02-02T15:19:33.137 INFO:tasks.cephadm:Fetching config... 2024-02-02T15:19:33.137 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-02-02T15:19:33.137 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-02-02T15:19:33.179 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-02-02T15:19:33.179 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-02-02T15:19:33.180 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-02-02T15:19:33.263 INFO:tasks.cephadm:Fetching mon keyring... 2024-02-02T15:19:33.263 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-02-02T15:19:33.264 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.a/keyring of=/dev/stdout 2024-02-02T15:19:33.379 INFO:tasks.cephadm:Fetching pub ssh key... 2024-02-02T15:19:33.379 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-02-02T15:19:33.379 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-02-02T15:19:33.462 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-02-02T15:19:33.462 DEBUG:teuthology.orchestra.run.smithi001:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDE1d+fQVaW2LJFp4HGw10d1r9j3ZuxDZ/8uAf0tV2ReKkI/+KvK1Y03ELM8Ho6f1S53Du8SOhwYiK+TzDXIkN8IYb3nyisiarqb625CXWDfn/piNfEPVyEU90lbXnHnbCRYyqK4FsKpzSqjQyvh625tIwJ4TXPRJ3oqciYDEv+7P7oWUGvEwtUWGv1prmhfy0AQ6rg4fzBFRZZZGXRA+1xQ/sZ69+rdtWTXH1d13RqLzhKcMhBQFk7sIC9g/57fwrXhPNL3mS7d6G92B1m7eig8+Xod3fF43a2g18CJwA6kM1/gbAbtfCnesbsbXQLTl6MSDOxis+ZmUeq3mMdJ+LQ8Y37/SZPnPPQbSEOU1Ehzjp8zn9jpmE1BM2MwuunfR90yh3e3o4VCEljXI59cccXx03Ro1h8nby4HRafSm8IQGOT801mDdy2ica98QSzX9b7o6IVHZS17Q8wQdwRDT6MlLhf1jROaynneW58a7em8rlAA2huRAxIE6kbNfaag9s= ceph-371a409c-c1de-11ee-95b6-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-02T15:19:33.589 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:33 smithi001 ceph-mon[35470]: mgrmap e13: x(active, since 6s) 2024-02-02T15:19:33.589 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:33 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/4062561306' entity='client.admin' 2024-02-02T15:19:33.595 INFO:teuthology.orchestra.run.smithi001.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDE1d+fQVaW2LJFp4HGw10d1r9j3ZuxDZ/8uAf0tV2ReKkI/+KvK1Y03ELM8Ho6f1S53Du8SOhwYiK+TzDXIkN8IYb3nyisiarqb625CXWDfn/piNfEPVyEU90lbXnHnbCRYyqK4FsKpzSqjQyvh625tIwJ4TXPRJ3oqciYDEv+7P7oWUGvEwtUWGv1prmhfy0AQ6rg4fzBFRZZZGXRA+1xQ/sZ69+rdtWTXH1d13RqLzhKcMhBQFk7sIC9g/57fwrXhPNL3mS7d6G92B1m7eig8+Xod3fF43a2g18CJwA6kM1/gbAbtfCnesbsbXQLTl6MSDOxis+ZmUeq3mMdJ+LQ8Y37/SZPnPPQbSEOU1Ehzjp8zn9jpmE1BM2MwuunfR90yh3e3o4VCEljXI59cccXx03Ro1h8nby4HRafSm8IQGOT801mDdy2ica98QSzX9b7o6IVHZS17Q8wQdwRDT6MlLhf1jROaynneW58a7em8rlAA2huRAxIE6kbNfaag9s= ceph-371a409c-c1de-11ee-95b6-87774f69a715 2024-02-02T15:19:33.624 DEBUG:teuthology.orchestra.run.smithi071:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDE1d+fQVaW2LJFp4HGw10d1r9j3ZuxDZ/8uAf0tV2ReKkI/+KvK1Y03ELM8Ho6f1S53Du8SOhwYiK+TzDXIkN8IYb3nyisiarqb625CXWDfn/piNfEPVyEU90lbXnHnbCRYyqK4FsKpzSqjQyvh625tIwJ4TXPRJ3oqciYDEv+7P7oWUGvEwtUWGv1prmhfy0AQ6rg4fzBFRZZZGXRA+1xQ/sZ69+rdtWTXH1d13RqLzhKcMhBQFk7sIC9g/57fwrXhPNL3mS7d6G92B1m7eig8+Xod3fF43a2g18CJwA6kM1/gbAbtfCnesbsbXQLTl6MSDOxis+ZmUeq3mMdJ+LQ8Y37/SZPnPPQbSEOU1Ehzjp8zn9jpmE1BM2MwuunfR90yh3e3o4VCEljXI59cccXx03Ro1h8nby4HRafSm8IQGOT801mDdy2ica98QSzX9b7o6IVHZS17Q8wQdwRDT6MlLhf1jROaynneW58a7em8rlAA2huRAxIE6kbNfaag9s= ceph-371a409c-c1de-11ee-95b6-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-02T15:19:33.671 INFO:teuthology.orchestra.run.smithi071.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDE1d+fQVaW2LJFp4HGw10d1r9j3ZuxDZ/8uAf0tV2ReKkI/+KvK1Y03ELM8Ho6f1S53Du8SOhwYiK+TzDXIkN8IYb3nyisiarqb625CXWDfn/piNfEPVyEU90lbXnHnbCRYyqK4FsKpzSqjQyvh625tIwJ4TXPRJ3oqciYDEv+7P7oWUGvEwtUWGv1prmhfy0AQ6rg4fzBFRZZZGXRA+1xQ/sZ69+rdtWTXH1d13RqLzhKcMhBQFk7sIC9g/57fwrXhPNL3mS7d6G92B1m7eig8+Xod3fF43a2g18CJwA6kM1/gbAbtfCnesbsbXQLTl6MSDOxis+ZmUeq3mMdJ+LQ8Y37/SZPnPPQbSEOU1Ehzjp8zn9jpmE1BM2MwuunfR90yh3e3o4VCEljXI59cccXx03Ro1h8nby4HRafSm8IQGOT801mDdy2ica98QSzX9b7o6IVHZS17Q8wQdwRDT6MlLhf1jROaynneW58a7em8rlAA2huRAxIE6kbNfaag9s= ceph-371a409c-c1de-11ee-95b6-87774f69a715 2024-02-02T15:19:33.684 DEBUG:teuthology.orchestra.run.smithi142:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDE1d+fQVaW2LJFp4HGw10d1r9j3ZuxDZ/8uAf0tV2ReKkI/+KvK1Y03ELM8Ho6f1S53Du8SOhwYiK+TzDXIkN8IYb3nyisiarqb625CXWDfn/piNfEPVyEU90lbXnHnbCRYyqK4FsKpzSqjQyvh625tIwJ4TXPRJ3oqciYDEv+7P7oWUGvEwtUWGv1prmhfy0AQ6rg4fzBFRZZZGXRA+1xQ/sZ69+rdtWTXH1d13RqLzhKcMhBQFk7sIC9g/57fwrXhPNL3mS7d6G92B1m7eig8+Xod3fF43a2g18CJwA6kM1/gbAbtfCnesbsbXQLTl6MSDOxis+ZmUeq3mMdJ+LQ8Y37/SZPnPPQbSEOU1Ehzjp8zn9jpmE1BM2MwuunfR90yh3e3o4VCEljXI59cccXx03Ro1h8nby4HRafSm8IQGOT801mDdy2ica98QSzX9b7o6IVHZS17Q8wQdwRDT6MlLhf1jROaynneW58a7em8rlAA2huRAxIE6kbNfaag9s= ceph-371a409c-c1de-11ee-95b6-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-02T15:19:33.730 INFO:teuthology.orchestra.run.smithi142.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDE1d+fQVaW2LJFp4HGw10d1r9j3ZuxDZ/8uAf0tV2ReKkI/+KvK1Y03ELM8Ho6f1S53Du8SOhwYiK+TzDXIkN8IYb3nyisiarqb625CXWDfn/piNfEPVyEU90lbXnHnbCRYyqK4FsKpzSqjQyvh625tIwJ4TXPRJ3oqciYDEv+7P7oWUGvEwtUWGv1prmhfy0AQ6rg4fzBFRZZZGXRA+1xQ/sZ69+rdtWTXH1d13RqLzhKcMhBQFk7sIC9g/57fwrXhPNL3mS7d6G92B1m7eig8+Xod3fF43a2g18CJwA6kM1/gbAbtfCnesbsbXQLTl6MSDOxis+ZmUeq3mMdJ+LQ8Y37/SZPnPPQbSEOU1Ehzjp8zn9jpmE1BM2MwuunfR90yh3e3o4VCEljXI59cccXx03Ro1h8nby4HRafSm8IQGOT801mDdy2ica98QSzX9b7o6IVHZS17Q8wQdwRDT6MlLhf1jROaynneW58a7em8rlAA2huRAxIE6kbNfaag9s= ceph-371a409c-c1de-11ee-95b6-87774f69a715 2024-02-02T15:19:33.742 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-02-02T15:19:34.108 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.a/config 2024-02-02T15:19:35.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:35.461+0000 7f421b600700 1 -- 172.21.15.1:0/3845559198 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4214100850 msgr2=0x7f4214100c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:19:35.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:35.461+0000 7f421b600700 1 --2- 172.21.15.1:0/3845559198 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4214100850 0x7f4214100c70 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4204009910 tx=0x7f4204009c20 comp rx=0 tx=0).stop 2024-02-02T15:19:35.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:35.461+0000 7f421b600700 1 -- 172.21.15.1:0/3845559198 shutdown_connections 2024-02-02T15:19:35.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:35.461+0000 7f421b600700 1 --2- 172.21.15.1:0/3845559198 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4214100850 0x7f4214100c70 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:35.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:35.461+0000 7f421b600700 1 -- 172.21.15.1:0/3845559198 >> 172.21.15.1:0/3845559198 conn(0x7f42140fbdd0 msgr2=0x7f42140fe230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:19:35.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:35.461+0000 7f421b600700 1 -- 172.21.15.1:0/3845559198 shutdown_connections 2024-02-02T15:19:35.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:35.462+0000 7f421b600700 1 -- 172.21.15.1:0/3845559198 wait complete. 2024-02-02T15:19:35.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:35.463+0000 7f421b600700 1 Processor -- start 2024-02-02T15:19:35.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:35.463+0000 7f421b600700 1 -- start start 2024-02-02T15:19:35.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:35.463+0000 7f421b600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4214100850 0x7f4214197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:19:35.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:35.463+0000 7f421b600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4214198090 con 0x7f4214100850 2024-02-02T15:19:35.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:35.463+0000 7f4218c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4214100850 0x7f4214197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:19:35.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:35.463+0000 7f4218c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4214100850 0x7f4214197b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55162/0 (socket says 172.21.15.1:55162) 2024-02-02T15:19:35.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:35.463+0000 7f4218c00700 1 -- 172.21.15.1:0/3930346514 learned_addr learned my addr 172.21.15.1:0/3930346514 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:19:35.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:35.464+0000 7f4218c00700 1 -- 172.21.15.1:0/3930346514 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42040095f0 con 0x7f4214100850 2024-02-02T15:19:35.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:35.464+0000 7f4218c00700 1 --2- 172.21.15.1:0/3930346514 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4214100850 0x7f4214197b50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4204004f40 tx=0x7f42040057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:19:35.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:35.464+0000 7f4211600700 1 -- 172.21.15.1:0/3930346514 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f420401d070 con 0x7f4214100850 2024-02-02T15:19:35.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:35.464+0000 7f4211600700 1 -- 172.21.15.1:0/3930346514 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f420400bc10 con 0x7f4214100850 2024-02-02T15:19:35.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:35.464+0000 7f4211600700 1 -- 172.21.15.1:0/3930346514 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4204003a70 con 0x7f4214100850 2024-02-02T15:19:35.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:35.465+0000 7f421b600700 1 -- 172.21.15.1:0/3930346514 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4214198290 con 0x7f4214100850 2024-02-02T15:19:35.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:35.465+0000 7f421b600700 1 -- 172.21.15.1:0/3930346514 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4214198730 con 0x7f4214100850 2024-02-02T15:19:35.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:35.466+0000 7f4211600700 1 -- 172.21.15.1:0/3930346514 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) v1 ==== 45323+0+0 (secure 0 0 0) 0x7f4204003ce0 con 0x7f4214100850 2024-02-02T15:19:35.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:35.466+0000 7f421b600700 1 -- 172.21.15.1:0/3930346514 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42141065a0 con 0x7f4214100850 2024-02-02T15:19:35.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:35.467+0000 7f4211600700 1 --2- 172.21.15.1:0/3930346514 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f41f40383f0 0x7f41f403a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:19:35.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:35.467+0000 7f4211600700 1 -- 172.21.15.1:0/3930346514 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f420404bfe0 con 0x7f4214100850 2024-02-02T15:19:35.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:35.467+0000 7f4213e00700 1 --2- 172.21.15.1:0/3930346514 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f41f40383f0 0x7f41f403a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:19:35.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:35.468+0000 7f4213e00700 1 --2- 172.21.15.1:0/3930346514 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f41f40383f0 0x7f41f403a8b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f41fc006fd0 tx=0x7f41fc006e40 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:19:35.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:35.472+0000 7f4211600700 1 -- 172.21.15.1:0/3930346514 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f420402aba0 con 0x7f4214100850 2024-02-02T15:19:35.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:35.684+0000 7f421b600700 1 -- 172.21.15.1:0/3930346514 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f421419b510 con 0x7f4214100850 2024-02-02T15:19:35.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:35.688+0000 7f4211600700 1 -- 172.21.15.1:0/3930346514 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f4204026070 con 0x7f4214100850 2024-02-02T15:19:35.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:35.690+0000 7f421b600700 1 -- 172.21.15.1:0/3930346514 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f41f40383f0 msgr2=0x7f41f403a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:19:35.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:35.690+0000 7f421b600700 1 --2- 172.21.15.1:0/3930346514 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f41f40383f0 0x7f41f403a8b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f41fc006fd0 tx=0x7f41fc006e40 comp rx=0 tx=0).stop 2024-02-02T15:19:35.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:35.690+0000 7f421b600700 1 -- 172.21.15.1:0/3930346514 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4214100850 msgr2=0x7f4214197b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:19:35.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:35.690+0000 7f421b600700 1 --2- 172.21.15.1:0/3930346514 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4214100850 0x7f4214197b50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4204004f40 tx=0x7f42040057c0 comp rx=0 tx=0).stop 2024-02-02T15:19:35.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:35.691+0000 7f421b600700 1 -- 172.21.15.1:0/3930346514 shutdown_connections 2024-02-02T15:19:35.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:35.691+0000 7f421b600700 1 --2- 172.21.15.1:0/3930346514 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f41f40383f0 0x7f41f403a8b0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:35.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:35.691+0000 7f421b600700 1 --2- 172.21.15.1:0/3930346514 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4214100850 0x7f4214197b50 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:35.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:35.691+0000 7f421b600700 1 -- 172.21.15.1:0/3930346514 >> 172.21.15.1:0/3930346514 conn(0x7f42140fbdd0 msgr2=0x7f4214073010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:19:35.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:35.691+0000 7f421b600700 1 -- 172.21.15.1:0/3930346514 shutdown_connections 2024-02-02T15:19:35.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:35.691+0000 7f421b600700 1 -- 172.21.15.1:0/3930346514 wait complete. 2024-02-02T15:19:36.048 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-02-02T15:19:36.048 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-02-02T15:19:36.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:36 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:19:36.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:36 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:19:36.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:36 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi001", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:19:36.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:36 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:19:36.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:36 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:19:36.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:36 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:19:36.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:36 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/3930346514' entity='client.admin' 2024-02-02T15:19:36.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:36 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:19:36.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:36 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:19:36.429 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.a/config 2024-02-02T15:19:37.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:37.841+0000 7f7a40c00700 1 -- 172.21.15.1:0/3775252922 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7a3c100850 msgr2=0x7f7a3c100c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:19:37.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:37.841+0000 7f7a40c00700 1 --2- 172.21.15.1:0/3775252922 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7a3c100850 0x7f7a3c100c70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c009910 tx=0x7f7a2c009c20 comp rx=0 tx=0).stop 2024-02-02T15:19:37.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:37.841+0000 7f7a40c00700 1 -- 172.21.15.1:0/3775252922 shutdown_connections 2024-02-02T15:19:37.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:37.841+0000 7f7a40c00700 1 --2- 172.21.15.1:0/3775252922 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7a3c100850 0x7f7a3c100c70 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:37.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:37.841+0000 7f7a40c00700 1 -- 172.21.15.1:0/3775252922 >> 172.21.15.1:0/3775252922 conn(0x7f7a3c0fbdd0 msgr2=0x7f7a3c0fe230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:19:37.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:37.841+0000 7f7a40c00700 1 -- 172.21.15.1:0/3775252922 shutdown_connections 2024-02-02T15:19:37.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:37.841+0000 7f7a40c00700 1 -- 172.21.15.1:0/3775252922 wait complete. 2024-02-02T15:19:37.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:37.842+0000 7f7a40c00700 1 Processor -- start 2024-02-02T15:19:37.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:37.842+0000 7f7a40c00700 1 -- start start 2024-02-02T15:19:37.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:37.842+0000 7f7a40c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7a3c100850 0x7f7a3c197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:19:37.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:37.843+0000 7f7a40c00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3c198090 con 0x7f7a3c100850 2024-02-02T15:19:37.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:37.843+0000 7f7a39e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7a3c100850 0x7f7a3c197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:19:37.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:37.843+0000 7f7a39e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7a3c100850 0x7f7a3c197b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55174/0 (socket says 172.21.15.1:55174) 2024-02-02T15:19:37.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:37.843+0000 7f7a39e00700 1 -- 172.21.15.1:0/3853983692 learned_addr learned my addr 172.21.15.1:0/3853983692 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:19:37.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:37.843+0000 7f7a39e00700 1 -- 172.21.15.1:0/3853983692 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a2c0095f0 con 0x7f7a3c100850 2024-02-02T15:19:37.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:37.843+0000 7f7a39e00700 1 --2- 172.21.15.1:0/3853983692 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7a3c100850 0x7f7a3c197b50 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c006010 tx=0x7f7a2c0057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:19:37.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:37.844+0000 7f7a32a00700 1 -- 172.21.15.1:0/3853983692 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7a2c01d070 con 0x7f7a3c100850 2024-02-02T15:19:37.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:37.844+0000 7f7a40c00700 1 -- 172.21.15.1:0/3853983692 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a3c198290 con 0x7f7a3c100850 2024-02-02T15:19:37.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:37.844+0000 7f7a32a00700 1 -- 172.21.15.1:0/3853983692 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a2c0044e0 con 0x7f7a3c100850 2024-02-02T15:19:37.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:37.844+0000 7f7a32a00700 1 -- 172.21.15.1:0/3853983692 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7a2c0039b0 con 0x7f7a3c100850 2024-02-02T15:19:37.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:37.844+0000 7f7a40c00700 1 -- 172.21.15.1:0/3853983692 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a3c1986d0 con 0x7f7a3c100850 2024-02-02T15:19:37.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:37.845+0000 7f7a32a00700 1 -- 172.21.15.1:0/3853983692 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) v1 ==== 45323+0+0 (secure 0 0 0) 0x7f7a2c00baf0 con 0x7f7a3c100850 2024-02-02T15:19:37.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:37.845+0000 7f7a40c00700 1 -- 172.21.15.1:0/3853983692 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a3c1065a0 con 0x7f7a3c100850 2024-02-02T15:19:37.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:37.845+0000 7f7a32a00700 1 --2- 172.21.15.1:0/3853983692 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f7a1c0383f0 0x7f7a1c03a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:19:37.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:37.845+0000 7f7a32a00700 1 -- 172.21.15.1:0/3853983692 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f7a2c04c380 con 0x7f7a3c100850 2024-02-02T15:19:37.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:37.845+0000 7f7a39400700 1 --2- 172.21.15.1:0/3853983692 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f7a1c0383f0 0x7f7a1c03a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:19:37.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:37.846+0000 7f7a39400700 1 --2- 172.21.15.1:0/3853983692 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f7a1c0383f0 0x7f7a1c03a8b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7a24006fd0 tx=0x7f7a24006e40 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:19:37.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:37.851+0000 7f7a32a00700 1 -- 172.21.15.1:0/3853983692 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f7a2c02a950 con 0x7f7a3c100850 2024-02-02T15:19:38.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:38.063+0000 7f7a40c00700 1 -- 172.21.15.1:0/3853983692 --> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f7a3c073720 con 0x7f7a1c0383f0 2024-02-02T15:19:38.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:38.067+0000 7f7a32a00700 1 -- 172.21.15.1:0/3853983692 <== mgr.14154 v2:172.21.15.1:6800/3768989981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f7a3c073720 con 0x7f7a1c0383f0 2024-02-02T15:19:38.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:38.069+0000 7f7a40c00700 1 -- 172.21.15.1:0/3853983692 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f7a1c0383f0 msgr2=0x7f7a1c03a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:19:38.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:38.069+0000 7f7a40c00700 1 --2- 172.21.15.1:0/3853983692 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f7a1c0383f0 0x7f7a1c03a8b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7a24006fd0 tx=0x7f7a24006e40 comp rx=0 tx=0).stop 2024-02-02T15:19:38.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:38.069+0000 7f7a40c00700 1 -- 172.21.15.1:0/3853983692 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7a3c100850 msgr2=0x7f7a3c197b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:19:38.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:38.069+0000 7f7a40c00700 1 --2- 172.21.15.1:0/3853983692 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7a3c100850 0x7f7a3c197b50 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c006010 tx=0x7f7a2c0057c0 comp rx=0 tx=0).stop 2024-02-02T15:19:38.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:38.069+0000 7f7a40c00700 1 -- 172.21.15.1:0/3853983692 shutdown_connections 2024-02-02T15:19:38.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:38.069+0000 7f7a40c00700 1 --2- 172.21.15.1:0/3853983692 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f7a1c0383f0 0x7f7a1c03a8b0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:38.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:38.069+0000 7f7a40c00700 1 --2- 172.21.15.1:0/3853983692 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7a3c100850 0x7f7a3c197b50 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:38.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:38.069+0000 7f7a40c00700 1 -- 172.21.15.1:0/3853983692 >> 172.21.15.1:0/3853983692 conn(0x7f7a3c0fbdd0 msgr2=0x7f7a3c073010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:19:38.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:38.070+0000 7f7a40c00700 1 -- 172.21.15.1:0/3853983692 shutdown_connections 2024-02-02T15:19:38.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:38.070+0000 7f7a40c00700 1 -- 172.21.15.1:0/3853983692 wait complete. 2024-02-02T15:19:38.427 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi071 2024-02-02T15:19:38.428 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-02-02T15:19:38.428 DEBUG:teuthology.orchestra.run.smithi071:> dd of=/etc/ceph/ceph.conf 2024-02-02T15:19:38.448 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-02-02T15:19:38.449 DEBUG:teuthology.orchestra.run.smithi071:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-02T15:19:38.509 INFO:tasks.cephadm:Adding host smithi071 to orchestrator... 2024-02-02T15:19:38.509 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph orch host add smithi071 2024-02-02T15:19:38.887 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.a/config 2024-02-02T15:19:39.347 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:39 smithi001 ceph-mon[35470]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:19:39.347 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:39 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:19:39.348 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:39 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:19:39.348 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:39 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:19:39.348 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:39 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:19:39.348 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:39 smithi001 ceph-mon[35470]: Updating smithi001:/etc/ceph/ceph.conf 2024-02-02T15:19:40.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:40.026+0000 7f0f2de00700 1 -- 172.21.15.1:0/1394347914 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0f281089c0 msgr2=0x7f0f28108da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:19:40.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:40.026+0000 7f0f2de00700 1 --2- 172.21.15.1:0/1394347914 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0f281089c0 0x7f0f28108da0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0f18009910 tx=0x7f0f18009c20 comp rx=0 tx=0).stop 2024-02-02T15:19:40.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:40.026+0000 7f0f2de00700 1 -- 172.21.15.1:0/1394347914 shutdown_connections 2024-02-02T15:19:40.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:40.026+0000 7f0f2de00700 1 --2- 172.21.15.1:0/1394347914 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0f281089c0 0x7f0f28108da0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:40.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:40.026+0000 7f0f2de00700 1 -- 172.21.15.1:0/1394347914 >> 172.21.15.1:0/1394347914 conn(0x7f0f280fe4f0 msgr2=0x7f0f28100910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:19:40.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:40.026+0000 7f0f2de00700 1 -- 172.21.15.1:0/1394347914 shutdown_connections 2024-02-02T15:19:40.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:40.027+0000 7f0f2de00700 1 -- 172.21.15.1:0/1394347914 wait complete. 2024-02-02T15:19:40.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:40.028+0000 7f0f2de00700 1 Processor -- start 2024-02-02T15:19:40.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:40.028+0000 7f0f2de00700 1 -- start start 2024-02-02T15:19:40.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:40.028+0000 7f0f2de00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0f281089c0 0x7f0f28197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:19:40.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:40.028+0000 7f0f2de00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f28198480 con 0x7f0f281089c0 2024-02-02T15:19:40.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:40.028+0000 7f0f26a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0f281089c0 0x7f0f28197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:19:40.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:40.028+0000 7f0f26a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0f281089c0 0x7f0f28197f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55190/0 (socket says 172.21.15.1:55190) 2024-02-02T15:19:40.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:40.028+0000 7f0f26a00700 1 -- 172.21.15.1:0/593372397 learned_addr learned my addr 172.21.15.1:0/593372397 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:19:40.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:40.029+0000 7f0f26a00700 1 -- 172.21.15.1:0/593372397 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f180095f0 con 0x7f0f281089c0 2024-02-02T15:19:40.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:40.029+0000 7f0f26a00700 1 --2- 172.21.15.1:0/593372397 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0f281089c0 0x7f0f28197f40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0f18006010 tx=0x7f0f18004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:19:40.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:40.029+0000 7f0f2ca00700 1 -- 172.21.15.1:0/593372397 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0f1801d070 con 0x7f0f281089c0 2024-02-02T15:19:40.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:40.029+0000 7f0f2ca00700 1 -- 172.21.15.1:0/593372397 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f1800bbb0 con 0x7f0f281089c0 2024-02-02T15:19:40.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:40.029+0000 7f0f2ca00700 1 -- 172.21.15.1:0/593372397 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0f18003930 con 0x7f0f281089c0 2024-02-02T15:19:40.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:40.029+0000 7f0f2de00700 1 -- 172.21.15.1:0/593372397 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f281986e0 con 0x7f0f281089c0 2024-02-02T15:19:40.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:40.030+0000 7f0f2de00700 1 -- 172.21.15.1:0/593372397 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f28198b60 con 0x7f0f281089c0 2024-02-02T15:19:40.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:40.031+0000 7f0f2ca00700 1 -- 172.21.15.1:0/593372397 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) v1 ==== 45323+0+0 (secure 0 0 0) 0x7f0f1800bd20 con 0x7f0f281089c0 2024-02-02T15:19:40.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:40.031+0000 7f0f2de00700 1 -- 172.21.15.1:0/593372397 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f28062370 con 0x7f0f281089c0 2024-02-02T15:19:40.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:40.031+0000 7f0f2ca00700 1 --2- 172.21.15.1:0/593372397 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f0f080383f0 0x7f0f0803a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:19:40.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:40.031+0000 7f0f2ca00700 1 -- 172.21.15.1:0/593372397 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f0f1804c0c0 con 0x7f0f281089c0 2024-02-02T15:19:40.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:40.031+0000 7f0f26000700 1 --2- 172.21.15.1:0/593372397 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f0f080383f0 0x7f0f0803a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:19:40.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:40.032+0000 7f0f26000700 1 --2- 172.21.15.1:0/593372397 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f0f080383f0 0x7f0f0803a8b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0f10006fd0 tx=0x7f0f10006e40 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:19:40.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:40.036+0000 7f0f2ca00700 1 -- 172.21.15.1:0/593372397 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f0f1807f110 con 0x7f0f281089c0 2024-02-02T15:19:40.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:40.276+0000 7f0f2de00700 1 -- 172.21.15.1:0/593372397 --> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi071", "target": ["mon-mgr", ""]}) v1 -- 0x7f0f281066f0 con 0x7f0f080383f0 2024-02-02T15:19:40.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:40 smithi001 ceph-mon[35470]: Updating smithi001:/var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-02T15:19:40.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:40 smithi001 ceph-mon[35470]: Updating smithi001:/etc/ceph/ceph.client.admin.keyring 2024-02-02T15:19:41.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:41 smithi001 ceph-mon[35470]: Updating smithi001:/var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-02T15:19:41.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:41 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:19:41.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:41 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:19:41.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:41 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:19:42.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:42 smithi001 ceph-mon[35470]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi071", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:19:42.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:42 smithi001 ceph-mon[35470]: Deploying cephadm binary to smithi071 2024-02-02T15:19:42.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:42.578+0000 7f0f2ca00700 1 -- 172.21.15.1:0/593372397 <== mgr.14154 v2:172.21.15.1:6800/3768989981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f0f281066f0 con 0x7f0f080383f0 2024-02-02T15:19:42.579 INFO:teuthology.orchestra.run.smithi001.stdout:Added host 'smithi071' with addr '172.21.15.71' 2024-02-02T15:19:42.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:42.581+0000 7f0f2de00700 1 -- 172.21.15.1:0/593372397 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f0f080383f0 msgr2=0x7f0f0803a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:19:42.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:42.581+0000 7f0f2de00700 1 --2- 172.21.15.1:0/593372397 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f0f080383f0 0x7f0f0803a8b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0f10006fd0 tx=0x7f0f10006e40 comp rx=0 tx=0).stop 2024-02-02T15:19:42.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:42.581+0000 7f0f2de00700 1 -- 172.21.15.1:0/593372397 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0f281089c0 msgr2=0x7f0f28197f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:19:42.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:42.581+0000 7f0f2de00700 1 --2- 172.21.15.1:0/593372397 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0f281089c0 0x7f0f28197f40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0f18006010 tx=0x7f0f18004ff0 comp rx=0 tx=0).stop 2024-02-02T15:19:42.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:42.581+0000 7f0f2de00700 1 -- 172.21.15.1:0/593372397 shutdown_connections 2024-02-02T15:19:42.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:42.581+0000 7f0f2de00700 1 --2- 172.21.15.1:0/593372397 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f0f080383f0 0x7f0f0803a8b0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:42.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:42.581+0000 7f0f2de00700 1 --2- 172.21.15.1:0/593372397 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0f281089c0 0x7f0f28197f40 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:42.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:42.581+0000 7f0f2de00700 1 -- 172.21.15.1:0/593372397 >> 172.21.15.1:0/593372397 conn(0x7f0f280fe4f0 msgr2=0x7f0f2810adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:19:42.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:42.582+0000 7f0f2de00700 1 -- 172.21.15.1:0/593372397 shutdown_connections 2024-02-02T15:19:42.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:42.582+0000 7f0f2de00700 1 -- 172.21.15.1:0/593372397 wait complete. 2024-02-02T15:19:42.949 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph orch host ls --format=json 2024-02-02T15:19:43.301 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.a/config 2024-02-02T15:19:43.734 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:43 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:19:43.735 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:43 smithi001 ceph-mon[35470]: Added host smithi071 2024-02-02T15:19:43.735 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:43 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:19:43.735 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:43 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:19:43.735 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:43 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:19:44.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:44.352+0000 7f2cede00700 1 -- 172.21.15.1:0/654826343 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2ce81067d0 msgr2=0x7f2ce8106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:19:44.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:44.352+0000 7f2cede00700 1 --2- 172.21.15.1:0/654826343 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2ce81067d0 0x7f2ce8106bb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8009910 tx=0x7f2cd8009c20 comp rx=0 tx=0).stop 2024-02-02T15:19:44.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:44.352+0000 7f2cede00700 1 -- 172.21.15.1:0/654826343 shutdown_connections 2024-02-02T15:19:44.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:44.352+0000 7f2cede00700 1 --2- 172.21.15.1:0/654826343 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2ce81067d0 0x7f2ce8106bb0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:44.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:44.352+0000 7f2cede00700 1 -- 172.21.15.1:0/654826343 >> 172.21.15.1:0/654826343 conn(0x7f2ce8075240 msgr2=0x7f2ce8075650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:19:44.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:44.352+0000 7f2cede00700 1 -- 172.21.15.1:0/654826343 shutdown_connections 2024-02-02T15:19:44.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:44.352+0000 7f2cede00700 1 -- 172.21.15.1:0/654826343 wait complete. 2024-02-02T15:19:44.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:44.353+0000 7f2cede00700 1 Processor -- start 2024-02-02T15:19:44.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:44.354+0000 7f2cede00700 1 -- start start 2024-02-02T15:19:44.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:44.354+0000 7f2cede00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2ce81067d0 0x7f2ce80ff7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:19:44.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:44.354+0000 7f2cede00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce8101730 con 0x7f2ce81067d0 2024-02-02T15:19:44.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:44.354+0000 7f2ce6a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2ce81067d0 0x7f2ce80ff7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:19:44.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:44.354+0000 7f2ce6a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2ce81067d0 0x7f2ce80ff7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60424/0 (socket says 172.21.15.1:60424) 2024-02-02T15:19:44.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:44.355+0000 7f2ce6a00700 1 -- 172.21.15.1:0/2900559807 learned_addr learned my addr 172.21.15.1:0/2900559807 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:19:44.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:44.355+0000 7f2ce6a00700 1 -- 172.21.15.1:0/2900559807 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cd80095f0 con 0x7f2ce81067d0 2024-02-02T15:19:44.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:44.355+0000 7f2ce6a00700 1 --2- 172.21.15.1:0/2900559807 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2ce81067d0 0x7f2ce80ff7b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8006010 tx=0x7f2cd8004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:19:44.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:44.355+0000 7f2ceca00700 1 -- 172.21.15.1:0/2900559807 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2cd801d070 con 0x7f2ce81067d0 2024-02-02T15:19:44.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:44.355+0000 7f2ceca00700 1 -- 172.21.15.1:0/2900559807 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2cd800bc10 con 0x7f2ce81067d0 2024-02-02T15:19:44.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:44.355+0000 7f2cede00700 1 -- 172.21.15.1:0/2900559807 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ce80ffcf0 con 0x7f2ce81067d0 2024-02-02T15:19:44.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:44.355+0000 7f2ceca00700 1 -- 172.21.15.1:0/2900559807 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2cd8003910 con 0x7f2ce81067d0 2024-02-02T15:19:44.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:44.356+0000 7f2cede00700 1 -- 172.21.15.1:0/2900559807 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ce8100170 con 0x7f2ce81067d0 2024-02-02T15:19:44.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:44.356+0000 7f2ceca00700 1 -- 172.21.15.1:0/2900559807 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) v1 ==== 45323+0+0 (secure 0 0 0) 0x7f2cd800bd80 con 0x7f2ce81067d0 2024-02-02T15:19:44.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:44.356+0000 7f2cede00700 1 -- 172.21.15.1:0/2900559807 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ce8105a80 con 0x7f2ce81067d0 2024-02-02T15:19:44.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:44.357+0000 7f2ceca00700 1 --2- 172.21.15.1:0/2900559807 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f2cc80383f0 0x7f2cc803a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:19:44.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:44.357+0000 7f2ceca00700 1 -- 172.21.15.1:0/2900559807 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f2cd804c090 con 0x7f2ce81067d0 2024-02-02T15:19:44.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:44.357+0000 7f2cdfe00700 1 --2- 172.21.15.1:0/2900559807 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f2cc80383f0 0x7f2cc803a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:19:44.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:44.358+0000 7f2cdfe00700 1 --2- 172.21.15.1:0/2900559807 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f2cc80383f0 0x7f2cc803a8b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2cd0006fd0 tx=0x7f2cd0006e40 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:19:44.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:44.362+0000 7f2ceca00700 1 -- 172.21.15.1:0/2900559807 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f2cd802abb0 con 0x7f2ce81067d0 2024-02-02T15:19:44.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:44.574+0000 7f2cede00700 1 -- 172.21.15.1:0/2900559807 --> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2ce8100470 con 0x7f2cc80383f0 2024-02-02T15:19:44.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:44.576+0000 7f2ceca00700 1 -- 172.21.15.1:0/2900559807 <== mgr.14154 v2:172.21.15.1:6800/3768989981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+158 (secure 0 0 0) 0x7f2ce8100470 con 0x7f2cc80383f0 2024-02-02T15:19:44.576 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:19:44.576 INFO:teuthology.orchestra.run.smithi001.stdout:[{"addr": "172.21.15.1", "hostname": "smithi001", "labels": [], "status": ""}, {"addr": "172.21.15.71", "hostname": "smithi071", "labels": [], "status": ""}] 2024-02-02T15:19:44.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:44.578+0000 7f2cede00700 1 -- 172.21.15.1:0/2900559807 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f2cc80383f0 msgr2=0x7f2cc803a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:19:44.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:44.578+0000 7f2cede00700 1 --2- 172.21.15.1:0/2900559807 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f2cc80383f0 0x7f2cc803a8b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2cd0006fd0 tx=0x7f2cd0006e40 comp rx=0 tx=0).stop 2024-02-02T15:19:44.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:44.578+0000 7f2cede00700 1 -- 172.21.15.1:0/2900559807 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2ce81067d0 msgr2=0x7f2ce80ff7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:19:44.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:44.578+0000 7f2cede00700 1 --2- 172.21.15.1:0/2900559807 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2ce81067d0 0x7f2ce80ff7b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8006010 tx=0x7f2cd8004ff0 comp rx=0 tx=0).stop 2024-02-02T15:19:44.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:44.578+0000 7f2cede00700 1 -- 172.21.15.1:0/2900559807 shutdown_connections 2024-02-02T15:19:44.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:44.578+0000 7f2cede00700 1 --2- 172.21.15.1:0/2900559807 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f2cc80383f0 0x7f2cc803a8b0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:44.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:44.578+0000 7f2cede00700 1 --2- 172.21.15.1:0/2900559807 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2ce81067d0 0x7f2ce80ff7b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:44.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:44.578+0000 7f2cede00700 1 -- 172.21.15.1:0/2900559807 >> 172.21.15.1:0/2900559807 conn(0x7f2ce8075240 msgr2=0x7f2ce80fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:19:44.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:44.578+0000 7f2cede00700 1 -- 172.21.15.1:0/2900559807 shutdown_connections 2024-02-02T15:19:44.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:44.578+0000 7f2cede00700 1 -- 172.21.15.1:0/2900559807 wait complete. 2024-02-02T15:19:44.937 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:44 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:19:44.963 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi142 2024-02-02T15:19:44.963 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-02-02T15:19:44.964 DEBUG:teuthology.orchestra.run.smithi142:> dd of=/etc/ceph/ceph.conf 2024-02-02T15:19:44.984 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-02-02T15:19:44.984 DEBUG:teuthology.orchestra.run.smithi142:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-02T15:19:45.044 INFO:tasks.cephadm:Adding host smithi142 to orchestrator... 2024-02-02T15:19:45.045 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph orch host add smithi142 2024-02-02T15:19:45.396 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.a/config 2024-02-02T15:19:46.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:45 smithi001 ceph-mon[35470]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-02T15:19:46.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:46.437+0000 7f391f600700 1 -- 172.21.15.1:0/3687941379 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3918073170 msgr2=0x7f3918073550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:19:46.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:46.437+0000 7f391f600700 1 --2- 172.21.15.1:0/3687941379 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3918073170 0x7f3918073550 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3908009910 tx=0x7f3908009c20 comp rx=0 tx=0).stop 2024-02-02T15:19:46.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:46.437+0000 7f391f600700 1 -- 172.21.15.1:0/3687941379 shutdown_connections 2024-02-02T15:19:46.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:46.437+0000 7f391f600700 1 --2- 172.21.15.1:0/3687941379 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3918073170 0x7f3918073550 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:46.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:46.437+0000 7f391f600700 1 -- 172.21.15.1:0/3687941379 >> 172.21.15.1:0/3687941379 conn(0x7f39180fc300 msgr2=0x7f39180fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:19:46.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:46.437+0000 7f391f600700 1 -- 172.21.15.1:0/3687941379 shutdown_connections 2024-02-02T15:19:46.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:46.437+0000 7f391f600700 1 -- 172.21.15.1:0/3687941379 wait complete. 2024-02-02T15:19:46.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:46.438+0000 7f391f600700 1 Processor -- start 2024-02-02T15:19:46.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:46.438+0000 7f391f600700 1 -- start start 2024-02-02T15:19:46.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:46.439+0000 7f391f600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3918073170 0x7f3918197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:19:46.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:46.439+0000 7f391f600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3918198430 con 0x7f3918073170 2024-02-02T15:19:46.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:46.439+0000 7f391cc00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3918073170 0x7f3918197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:19:46.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:46.439+0000 7f391cc00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3918073170 0x7f3918197ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60436/0 (socket says 172.21.15.1:60436) 2024-02-02T15:19:46.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:46.439+0000 7f391cc00700 1 -- 172.21.15.1:0/3444631644 learned_addr learned my addr 172.21.15.1:0/3444631644 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:19:46.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:46.439+0000 7f391cc00700 1 -- 172.21.15.1:0/3444631644 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39080095f0 con 0x7f3918073170 2024-02-02T15:19:46.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:46.440+0000 7f391cc00700 1 --2- 172.21.15.1:0/3444631644 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3918073170 0x7f3918197ef0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3908006010 tx=0x7f3908004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:19:46.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:46.440+0000 7f3915600700 1 -- 172.21.15.1:0/3444631644 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f390801d070 con 0x7f3918073170 2024-02-02T15:19:46.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:46.440+0000 7f3915600700 1 -- 172.21.15.1:0/3444631644 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39080044a0 con 0x7f3918073170 2024-02-02T15:19:46.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:46.440+0000 7f391f600700 1 -- 172.21.15.1:0/3444631644 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3918198630 con 0x7f3918073170 2024-02-02T15:19:46.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:46.440+0000 7f3915600700 1 -- 172.21.15.1:0/3444631644 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3908003a70 con 0x7f3918073170 2024-02-02T15:19:46.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:46.440+0000 7f391f600700 1 -- 172.21.15.1:0/3444631644 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3918198a50 con 0x7f3918073170 2024-02-02T15:19:46.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:46.441+0000 7f3915600700 1 -- 172.21.15.1:0/3444631644 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) v1 ==== 45323+0+0 (secure 0 0 0) 0x7f3908003c60 con 0x7f3918073170 2024-02-02T15:19:46.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:46.442+0000 7f3915600700 1 --2- 172.21.15.1:0/3444631644 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f38f80383f0 0x7f38f803a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:19:46.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:46.442+0000 7f3915600700 1 -- 172.21.15.1:0/3444631644 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f390804c100 con 0x7f3918073170 2024-02-02T15:19:46.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:46.442+0000 7f3917e00700 1 --2- 172.21.15.1:0/3444631644 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f38f80383f0 0x7f38f803a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:19:46.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:46.442+0000 7f391f600700 1 -- 172.21.15.1:0/3444631644 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3918062370 con 0x7f3918073170 2024-02-02T15:19:46.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:46.443+0000 7f3917e00700 1 --2- 172.21.15.1:0/3444631644 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f38f80383f0 0x7f38f803a8b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3900006fd0 tx=0x7f3900006e40 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:19:46.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:46.448+0000 7f3915600700 1 -- 172.21.15.1:0/3444631644 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f3908026020 con 0x7f3918073170 2024-02-02T15:19:46.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:46.662+0000 7f391f600700 1 -- 172.21.15.1:0/3444631644 --> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi142", "target": ["mon-mgr", ""]}) v1 -- 0x7f391819c8a0 con 0x7f38f80383f0 2024-02-02T15:19:47.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:47 smithi001 ceph-mon[35470]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-02T15:19:47.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:47 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:19:47.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:47 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:19:47.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:47 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:19:47.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:47 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:19:47.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:47 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi071", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:19:47.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:47 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:19:47.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:47 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:19:47.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:47 smithi001 ceph-mon[35470]: Updating smithi071:/etc/ceph/ceph.conf 2024-02-02T15:19:47.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:47 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:19:47.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:47 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:19:47.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:47 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:19:48.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:48 smithi001 ceph-mon[35470]: Updating smithi071:/var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-02T15:19:48.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:48 smithi001 ceph-mon[35470]: Updating smithi071:/etc/ceph/ceph.client.admin.keyring 2024-02-02T15:19:48.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:48 smithi001 ceph-mon[35470]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi142", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:19:48.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:48 smithi001 ceph-mon[35470]: Updating smithi071:/var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-02T15:19:48.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:48 smithi001 ceph-mon[35470]: Deploying cephadm binary to smithi142 2024-02-02T15:19:48.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:48.876+0000 7f3915600700 1 -- 172.21.15.1:0/3444631644 <== mgr.14154 v2:172.21.15.1:6800/3768989981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f391819c8a0 con 0x7f38f80383f0 2024-02-02T15:19:48.876 INFO:teuthology.orchestra.run.smithi001.stdout:Added host 'smithi142' with addr '172.21.15.142' 2024-02-02T15:19:48.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:48.878+0000 7f391f600700 1 -- 172.21.15.1:0/3444631644 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f38f80383f0 msgr2=0x7f38f803a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:19:48.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:48.878+0000 7f391f600700 1 --2- 172.21.15.1:0/3444631644 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f38f80383f0 0x7f38f803a8b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3900006fd0 tx=0x7f3900006e40 comp rx=0 tx=0).stop 2024-02-02T15:19:48.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:48.878+0000 7f391f600700 1 -- 172.21.15.1:0/3444631644 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3918073170 msgr2=0x7f3918197ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:19:48.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:48.878+0000 7f391f600700 1 --2- 172.21.15.1:0/3444631644 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3918073170 0x7f3918197ef0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3908006010 tx=0x7f3908004ff0 comp rx=0 tx=0).stop 2024-02-02T15:19:48.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:48.878+0000 7f391f600700 1 -- 172.21.15.1:0/3444631644 shutdown_connections 2024-02-02T15:19:48.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:48.878+0000 7f391f600700 1 --2- 172.21.15.1:0/3444631644 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f38f80383f0 0x7f38f803a8b0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:48.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:48.878+0000 7f391f600700 1 --2- 172.21.15.1:0/3444631644 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3918073170 0x7f3918197ef0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:48.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:48.878+0000 7f391f600700 1 -- 172.21.15.1:0/3444631644 >> 172.21.15.1:0/3444631644 conn(0x7f39180fc300 msgr2=0x7f3918107310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:19:48.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:48.879+0000 7f391f600700 1 -- 172.21.15.1:0/3444631644 shutdown_connections 2024-02-02T15:19:48.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:48.879+0000 7f391f600700 1 -- 172.21.15.1:0/3444631644 wait complete. 2024-02-02T15:19:49.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:49 smithi001 ceph-mon[35470]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-02T15:19:49.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:49 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:19:49.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:49 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:19:49.238 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph orch host ls --format=json 2024-02-02T15:19:49.593 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.a/config 2024-02-02T15:19:50.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:50 smithi001 ceph-mon[35470]: Added host smithi142 2024-02-02T15:19:50.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:50 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:19:50.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:50 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:19:50.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:50.693+0000 7f9442000700 1 -- 172.21.15.1:0/4146271220 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f943c1067d0 msgr2=0x7f943c106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:19:50.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:50.693+0000 7f9442000700 1 --2- 172.21.15.1:0/4146271220 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f943c1067d0 0x7f943c106bb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f942c009910 tx=0x7f942c009c20 comp rx=0 tx=0).stop 2024-02-02T15:19:50.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:50.693+0000 7f9442000700 1 -- 172.21.15.1:0/4146271220 shutdown_connections 2024-02-02T15:19:50.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:50.693+0000 7f9442000700 1 --2- 172.21.15.1:0/4146271220 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f943c1067d0 0x7f943c106bb0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:50.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:50.693+0000 7f9442000700 1 -- 172.21.15.1:0/4146271220 >> 172.21.15.1:0/4146271220 conn(0x7f943c0fc300 msgr2=0x7f943c0fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:19:50.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:50.693+0000 7f9442000700 1 -- 172.21.15.1:0/4146271220 shutdown_connections 2024-02-02T15:19:50.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:50.693+0000 7f9442000700 1 -- 172.21.15.1:0/4146271220 wait complete. 2024-02-02T15:19:50.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:50.694+0000 7f9442000700 1 Processor -- start 2024-02-02T15:19:50.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:50.694+0000 7f9442000700 1 -- start start 2024-02-02T15:19:50.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:50.695+0000 7f9442000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f943c1067d0 0x7f943c197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:19:50.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:50.695+0000 7f9442000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f943c198400 con 0x7f943c1067d0 2024-02-02T15:19:50.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:50.695+0000 7f943b400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f943c1067d0 0x7f943c197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:19:50.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:50.695+0000 7f943b400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f943c1067d0 0x7f943c197ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60448/0 (socket says 172.21.15.1:60448) 2024-02-02T15:19:50.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:50.695+0000 7f943b400700 1 -- 172.21.15.1:0/13761810 learned_addr learned my addr 172.21.15.1:0/13761810 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:19:50.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:50.695+0000 7f943b400700 1 -- 172.21.15.1:0/13761810 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f942c0095f0 con 0x7f943c1067d0 2024-02-02T15:19:50.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:50.695+0000 7f943b400700 1 --2- 172.21.15.1:0/13761810 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f943c1067d0 0x7f943c197ec0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f942c004f40 tx=0x7f942c005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:19:50.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:50.696+0000 7f9433e00700 1 -- 172.21.15.1:0/13761810 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f942c01d070 con 0x7f943c1067d0 2024-02-02T15:19:50.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:50.696+0000 7f9433e00700 1 -- 172.21.15.1:0/13761810 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f942c004370 con 0x7f943c1067d0 2024-02-02T15:19:50.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:50.696+0000 7f9433e00700 1 -- 172.21.15.1:0/13761810 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f942c003970 con 0x7f943c1067d0 2024-02-02T15:19:50.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:50.696+0000 7f9442000700 1 -- 172.21.15.1:0/13761810 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f943c198660 con 0x7f943c1067d0 2024-02-02T15:19:50.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:50.696+0000 7f9442000700 1 -- 172.21.15.1:0/13761810 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f943c198ae0 con 0x7f943c1067d0 2024-02-02T15:19:50.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:50.697+0000 7f9442000700 1 -- 172.21.15.1:0/13761810 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f943c062370 con 0x7f943c1067d0 2024-02-02T15:19:50.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:50.697+0000 7f9433e00700 1 -- 172.21.15.1:0/13761810 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) v1 ==== 45323+0+0 (secure 0 0 0) 0x7f942c003be0 con 0x7f943c1067d0 2024-02-02T15:19:50.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:50.697+0000 7f9433e00700 1 --2- 172.21.15.1:0/13761810 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f941c0383f0 0x7f941c03a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:19:50.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:50.698+0000 7f9433e00700 1 -- 172.21.15.1:0/13761810 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f942c04c070 con 0x7f943c1067d0 2024-02-02T15:19:50.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:50.698+0000 7f943aa00700 1 --2- 172.21.15.1:0/13761810 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f941c0383f0 0x7f941c03a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:19:50.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:50.698+0000 7f943aa00700 1 --2- 172.21.15.1:0/13761810 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f941c0383f0 0x7f941c03a8b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9424006fd0 tx=0x7f9424006e40 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:19:50.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:50.702+0000 7f9433e00700 1 -- 172.21.15.1:0/13761810 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f942c02a590 con 0x7f943c1067d0 2024-02-02T15:19:50.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:50.919+0000 7f9442000700 1 -- 172.21.15.1:0/13761810 --> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f943c073f80 con 0x7f941c0383f0 2024-02-02T15:19:50.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:50.921+0000 7f9433e00700 1 -- 172.21.15.1:0/13761810 <== mgr.14154 v2:172.21.15.1:6800/3768989981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+238 (secure 0 0 0) 0x7f943c073f80 con 0x7f941c0383f0 2024-02-02T15:19:50.921 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:19:50.921 INFO:teuthology.orchestra.run.smithi001.stdout:[{"addr": "172.21.15.1", "hostname": "smithi001", "labels": [], "status": ""}, {"addr": "172.21.15.71", "hostname": "smithi071", "labels": [], "status": ""}, {"addr": "172.21.15.142", "hostname": "smithi142", "labels": [], "status": ""}] 2024-02-02T15:19:50.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:50.923+0000 7f9442000700 1 -- 172.21.15.1:0/13761810 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f941c0383f0 msgr2=0x7f941c03a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:19:50.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:50.923+0000 7f9442000700 1 --2- 172.21.15.1:0/13761810 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f941c0383f0 0x7f941c03a8b0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9424006fd0 tx=0x7f9424006e40 comp rx=0 tx=0).stop 2024-02-02T15:19:50.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:50.923+0000 7f9442000700 1 -- 172.21.15.1:0/13761810 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f943c1067d0 msgr2=0x7f943c197ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:19:50.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:50.923+0000 7f9442000700 1 --2- 172.21.15.1:0/13761810 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f943c1067d0 0x7f943c197ec0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f942c004f40 tx=0x7f942c005020 comp rx=0 tx=0).stop 2024-02-02T15:19:50.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:50.923+0000 7f9442000700 1 -- 172.21.15.1:0/13761810 shutdown_connections 2024-02-02T15:19:50.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:50.923+0000 7f9442000700 1 --2- 172.21.15.1:0/13761810 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f941c0383f0 0x7f941c03a8b0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:50.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:50.923+0000 7f9442000700 1 --2- 172.21.15.1:0/13761810 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f943c1067d0 0x7f943c197ec0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:50.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:50.923+0000 7f9442000700 1 -- 172.21.15.1:0/13761810 >> 172.21.15.1:0/13761810 conn(0x7f943c0fc300 msgr2=0x7f943c108c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:19:50.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:50.923+0000 7f9442000700 1 -- 172.21.15.1:0/13761810 shutdown_connections 2024-02-02T15:19:50.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:50.923+0000 7f9442000700 1 -- 172.21.15.1:0/13761810 wait complete. 2024-02-02T15:19:51.268 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:51 smithi001 ceph-mon[35470]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-02T15:19:51.268 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:51 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:19:51.268 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:51 smithi001 ceph-mon[35470]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-02T15:19:51.296 INFO:tasks.cephadm:Setting crush tunables to default 2024-02-02T15:19:51.296 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph osd crush tunables default 2024-02-02T15:19:51.649 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.a/config 2024-02-02T15:19:52.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:52 smithi001 ceph-mon[35470]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-02T15:19:52.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:52.707+0000 7f19e7200700 1 -- 172.21.15.1:0/1626941323 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f19e01067d0 msgr2=0x7f19e0106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:19:52.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:52.707+0000 7f19e7200700 1 --2- 172.21.15.1:0/1626941323 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f19e01067d0 0x7f19e0106bb0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f19d0009910 tx=0x7f19d0009c20 comp rx=0 tx=0).stop 2024-02-02T15:19:52.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:52.707+0000 7f19e7200700 1 -- 172.21.15.1:0/1626941323 shutdown_connections 2024-02-02T15:19:52.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:52.707+0000 7f19e7200700 1 --2- 172.21.15.1:0/1626941323 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f19e01067d0 0x7f19e0106bb0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:52.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:52.707+0000 7f19e7200700 1 -- 172.21.15.1:0/1626941323 >> 172.21.15.1:0/1626941323 conn(0x7f19e00fc300 msgr2=0x7f19e00fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:19:52.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:52.707+0000 7f19e7200700 1 -- 172.21.15.1:0/1626941323 shutdown_connections 2024-02-02T15:19:52.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:52.707+0000 7f19e7200700 1 -- 172.21.15.1:0/1626941323 wait complete. 2024-02-02T15:19:52.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:52.708+0000 7f19e7200700 1 Processor -- start 2024-02-02T15:19:52.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:52.709+0000 7f19e7200700 1 -- start start 2024-02-02T15:19:52.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:52.709+0000 7f19e7200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f19e01067d0 0x7f19e0197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:19:52.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:52.709+0000 7f19e7200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e0198400 con 0x7f19e01067d0 2024-02-02T15:19:52.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:52.709+0000 7f19dfe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f19e01067d0 0x7f19e0197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:19:52.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:52.709+0000 7f19dfe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f19e01067d0 0x7f19e0197ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60454/0 (socket says 172.21.15.1:60454) 2024-02-02T15:19:52.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:52.709+0000 7f19dfe00700 1 -- 172.21.15.1:0/480496878 learned_addr learned my addr 172.21.15.1:0/480496878 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:19:52.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:52.709+0000 7f19dfe00700 1 -- 172.21.15.1:0/480496878 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19d00095f0 con 0x7f19e01067d0 2024-02-02T15:19:52.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:52.710+0000 7f19dfe00700 1 --2- 172.21.15.1:0/480496878 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f19e01067d0 0x7f19e0197ec0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f19d0006010 tx=0x7f19d0004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:19:52.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:52.710+0000 7f19dcc00700 1 -- 172.21.15.1:0/480496878 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f19d001d070 con 0x7f19e01067d0 2024-02-02T15:19:52.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:52.710+0000 7f19dcc00700 1 -- 172.21.15.1:0/480496878 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f19d000bbb0 con 0x7f19e01067d0 2024-02-02T15:19:52.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:52.710+0000 7f19dcc00700 1 -- 172.21.15.1:0/480496878 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f19d0003930 con 0x7f19e01067d0 2024-02-02T15:19:52.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:52.710+0000 7f19e7200700 1 -- 172.21.15.1:0/480496878 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19e0198660 con 0x7f19e01067d0 2024-02-02T15:19:52.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:52.710+0000 7f19e7200700 1 -- 172.21.15.1:0/480496878 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19e0198ae0 con 0x7f19e01067d0 2024-02-02T15:19:52.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:52.711+0000 7f19dcc00700 1 -- 172.21.15.1:0/480496878 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) v1 ==== 45323+0+0 (secure 0 0 0) 0x7f19d000b6f0 con 0x7f19e01067d0 2024-02-02T15:19:52.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:52.711+0000 7f19e7200700 1 -- 172.21.15.1:0/480496878 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19e0062370 con 0x7f19e01067d0 2024-02-02T15:19:52.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:52.712+0000 7f19dcc00700 1 --2- 172.21.15.1:0/480496878 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f19c00383f0 0x7f19c003a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:19:52.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:52.712+0000 7f19dcc00700 1 -- 172.21.15.1:0/480496878 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f19d004c120 con 0x7f19e01067d0 2024-02-02T15:19:52.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:52.712+0000 7f19df400700 1 --2- 172.21.15.1:0/480496878 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f19c00383f0 0x7f19c003a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:19:52.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:52.713+0000 7f19df400700 1 --2- 172.21.15.1:0/480496878 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f19c00383f0 0x7f19c003a8b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f19c8006fd0 tx=0x7f19c8006e40 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:19:52.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:52.717+0000 7f19dcc00700 1 -- 172.21.15.1:0/480496878 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f19d0029540 con 0x7f19e01067d0 2024-02-02T15:19:52.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:52.935+0000 7f19e7200700 1 -- 172.21.15.1:0/480496878 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f19e0199220 con 0x7f19e01067d0 2024-02-02T15:19:53.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:53.618+0000 7f19dcc00700 1 -- 172.21.15.1:0/480496878 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f19d0026020 con 0x7f19e01067d0 2024-02-02T15:19:53.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:53.620+0000 7f19e7200700 1 -- 172.21.15.1:0/480496878 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f19c00383f0 msgr2=0x7f19c003a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:19:53.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:53.620+0000 7f19e7200700 1 --2- 172.21.15.1:0/480496878 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f19c00383f0 0x7f19c003a8b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f19c8006fd0 tx=0x7f19c8006e40 comp rx=0 tx=0).stop 2024-02-02T15:19:53.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:53.620+0000 7f19e7200700 1 -- 172.21.15.1:0/480496878 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f19e01067d0 msgr2=0x7f19e0197ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:19:53.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:53.620+0000 7f19e7200700 1 --2- 172.21.15.1:0/480496878 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f19e01067d0 0x7f19e0197ec0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f19d0006010 tx=0x7f19d0004ff0 comp rx=0 tx=0).stop 2024-02-02T15:19:53.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:53.620+0000 7f19e7200700 1 -- 172.21.15.1:0/480496878 shutdown_connections 2024-02-02T15:19:53.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:53.620+0000 7f19e7200700 1 --2- 172.21.15.1:0/480496878 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f19c00383f0 0x7f19c003a8b0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:53.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:53.620+0000 7f19e7200700 1 --2- 172.21.15.1:0/480496878 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f19e01067d0 0x7f19e0197ec0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:53.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:53.620+0000 7f19e7200700 1 -- 172.21.15.1:0/480496878 >> 172.21.15.1:0/480496878 conn(0x7f19e00fc300 msgr2=0x7f19e0108c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:19:53.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:53.620+0000 7f19e7200700 1 -- 172.21.15.1:0/480496878 shutdown_connections 2024-02-02T15:19:53.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:19:53.621+0000 7f19e7200700 1 -- 172.21.15.1:0/480496878 wait complete. 2024-02-02T15:19:53.621 INFO:teuthology.orchestra.run.smithi001.stderr:adjusted tunables profile to default 2024-02-02T15:19:53.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:53 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:19:53.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:53 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:19:53.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:53 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:19:53.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:53 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:19:53.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:53 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi142", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:19:53.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:53 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:19:53.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:53 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:19:53.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:53 smithi001 ceph-mon[35470]: Updating smithi142:/etc/ceph/ceph.conf 2024-02-02T15:19:53.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:53 smithi001 ceph-mon[35470]: Updating smithi142:/var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-02T15:19:53.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:53 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/480496878' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-02-02T15:19:53.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:53 smithi001 ceph-mon[35470]: Updating smithi142:/etc/ceph/ceph.client.admin.keyring 2024-02-02T15:19:53.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:53 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:19:53.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:53 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:19:53.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:53 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:19:54.002 INFO:tasks.cephadm:Adding mon.a on smithi001 2024-02-02T15:19:54.003 INFO:tasks.cephadm:Adding mon.b on smithi071 2024-02-02T15:19:54.003 INFO:tasks.cephadm:Adding mon.c on smithi142 2024-02-02T15:19:54.003 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph orch apply mon '3;smithi001:172.21.15.1=a;smithi071:172.21.15.71=b;smithi142:172.21.15.142=c' 2024-02-02T15:19:54.248 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-02T15:19:54.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:54 smithi001 ceph-mon[35470]: Updating smithi142:/var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-02T15:19:54.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:54 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/480496878' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-02-02T15:19:54.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:54 smithi001 ceph-mon[35470]: osdmap e4: 0 total, 0 up, 0 in 2024-02-02T15:19:54.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:54 smithi001 ceph-mon[35470]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-02T15:19:55.193 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:19:55.194+0000 7f34e9c53700 1 -- 172.21.15.142:0/4007899648 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f34e41067d0 msgr2=0x7f34e4106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:19:55.194 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:19:55.194+0000 7f34e9c53700 1 --2- 172.21.15.142:0/4007899648 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f34e41067d0 0x7f34e4106bb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f34d4009910 tx=0x7f34d4009c20 comp rx=0 tx=0).stop 2024-02-02T15:19:55.194 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:19:55.194+0000 7f34e9c53700 1 -- 172.21.15.142:0/4007899648 shutdown_connections 2024-02-02T15:19:55.194 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:19:55.194+0000 7f34e9c53700 1 --2- 172.21.15.142:0/4007899648 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f34e41067d0 0x7f34e4106bb0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:55.194 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:19:55.194+0000 7f34e9c53700 1 -- 172.21.15.142:0/4007899648 >> 172.21.15.142:0/4007899648 conn(0x7f34e4075240 msgr2=0x7f34e4075650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:19:55.194 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:19:55.194+0000 7f34e9c53700 1 -- 172.21.15.142:0/4007899648 shutdown_connections 2024-02-02T15:19:55.194 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:19:55.194+0000 7f34e9c53700 1 -- 172.21.15.142:0/4007899648 wait complete. 2024-02-02T15:19:55.196 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:19:55.195+0000 7f34e9c53700 1 Processor -- start 2024-02-02T15:19:55.196 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:19:55.195+0000 7f34e9c53700 1 -- start start 2024-02-02T15:19:55.196 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:19:55.195+0000 7f34e9c53700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f34e41067d0 0x7f34e40ff7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:19:55.196 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:19:55.195+0000 7f34e9c53700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34e4101730 con 0x7f34e41067d0 2024-02-02T15:19:55.196 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:19:55.196+0000 7f34e37fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f34e41067d0 0x7f34e40ff7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:19:55.196 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:19:55.196+0000 7f34e37fe700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f34e41067d0 0x7f34e40ff7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.142:36366/0 (socket says 172.21.15.142:36366) 2024-02-02T15:19:55.196 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:19:55.196+0000 7f34e37fe700 1 -- 172.21.15.142:0/1686067573 learned_addr learned my addr 172.21.15.142:0/1686067573 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-02-02T15:19:55.196 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:19:55.196+0000 7f34e37fe700 1 -- 172.21.15.142:0/1686067573 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34d40095f0 con 0x7f34e41067d0 2024-02-02T15:19:55.196 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:19:55.196+0000 7f34e37fe700 1 --2- 172.21.15.142:0/1686067573 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f34e41067d0 0x7f34e40ff7b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f34d4004f40 tx=0x7f34d4005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:19:55.197 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:19:55.196+0000 7f34e0ff9700 1 -- 172.21.15.142:0/1686067573 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f34d401d070 con 0x7f34e41067d0 2024-02-02T15:19:55.197 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:19:55.196+0000 7f34e9c53700 1 -- 172.21.15.142:0/1686067573 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34e40ffcf0 con 0x7f34e41067d0 2024-02-02T15:19:55.197 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:19:55.196+0000 7f34e0ff9700 1 -- 172.21.15.142:0/1686067573 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f34d4004370 con 0x7f34e41067d0 2024-02-02T15:19:55.197 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:19:55.196+0000 7f34e0ff9700 1 -- 172.21.15.142:0/1686067573 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f34d40039b0 con 0x7f34e41067d0 2024-02-02T15:19:55.197 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:19:55.196+0000 7f34e9c53700 1 -- 172.21.15.142:0/1686067573 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34e4100170 con 0x7f34e41067d0 2024-02-02T15:19:55.198 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:19:55.197+0000 7f34e0ff9700 1 -- 172.21.15.142:0/1686067573 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) v1 ==== 45323+0+0 (secure 0 0 0) 0x7f34d4003c70 con 0x7f34e41067d0 2024-02-02T15:19:55.198 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:19:55.197+0000 7f34e9c53700 1 -- 172.21.15.142:0/1686067573 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34e4105a80 con 0x7f34e41067d0 2024-02-02T15:19:55.198 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:19:55.197+0000 7f34e0ff9700 1 --2- 172.21.15.142:0/1686067573 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f34cc0383a0 0x7f34cc03a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:19:55.198 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:19:55.197+0000 7f34e0ff9700 1 -- 172.21.15.142:0/1686067573 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f34d404c210 con 0x7f34e41067d0 2024-02-02T15:19:55.198 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:19:55.198+0000 7f34e2ffd700 1 --2- 172.21.15.142:0/1686067573 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f34cc0383a0 0x7f34cc03a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:19:55.198 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:19:55.198+0000 7f34e2ffd700 1 --2- 172.21.15.142:0/1686067573 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f34cc0383a0 0x7f34cc03a860 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f34d8006fd0 tx=0x7f34d8006e40 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:19:55.202 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:19:55.203+0000 7f34e0ff9700 1 -- 172.21.15.142:0/1686067573 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f34d4029360 con 0x7f34e41067d0 2024-02-02T15:19:55.397 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:19:55.398+0000 7f34e9c53700 1 -- 172.21.15.142:0/1686067573 --> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi001:172.21.15.1=a;smithi071:172.21.15.71=b;smithi142:172.21.15.142=c", "target": ["mon-mgr", ""]}) v1 -- 0x7f34e4100470 con 0x7f34cc0383a0 2024-02-02T15:19:55.404 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:19:55.405+0000 7f34e0ff9700 1 -- 172.21.15.142:0/1686067573 <== mgr.14154 v2:172.21.15.1:6800/3768989981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f34e4100470 con 0x7f34cc0383a0 2024-02-02T15:19:55.404 INFO:teuthology.orchestra.run.smithi142.stdout:Scheduled mon update... 2024-02-02T15:19:55.406 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:19:55.407+0000 7f34e9c53700 1 -- 172.21.15.142:0/1686067573 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f34cc0383a0 msgr2=0x7f34cc03a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:19:55.406 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:19:55.407+0000 7f34e9c53700 1 --2- 172.21.15.142:0/1686067573 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f34cc0383a0 0x7f34cc03a860 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f34d8006fd0 tx=0x7f34d8006e40 comp rx=0 tx=0).stop 2024-02-02T15:19:55.406 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:19:55.407+0000 7f34e9c53700 1 -- 172.21.15.142:0/1686067573 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f34e41067d0 msgr2=0x7f34e40ff7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:19:55.406 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:19:55.407+0000 7f34e9c53700 1 --2- 172.21.15.142:0/1686067573 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f34e41067d0 0x7f34e40ff7b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f34d4004f40 tx=0x7f34d4005020 comp rx=0 tx=0).stop 2024-02-02T15:19:55.407 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:19:55.407+0000 7f34e9c53700 1 -- 172.21.15.142:0/1686067573 shutdown_connections 2024-02-02T15:19:55.407 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:19:55.407+0000 7f34e9c53700 1 --2- 172.21.15.142:0/1686067573 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f34cc0383a0 0x7f34cc03a860 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:55.407 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:19:55.407+0000 7f34e9c53700 1 --2- 172.21.15.142:0/1686067573 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f34e41067d0 0x7f34e40ff7b0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:55.407 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:19:55.407+0000 7f34e9c53700 1 -- 172.21.15.142:0/1686067573 >> 172.21.15.142:0/1686067573 conn(0x7f34e4075240 msgr2=0x7f34e40fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:19:55.407 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:19:55.407+0000 7f34e9c53700 1 -- 172.21.15.142:0/1686067573 shutdown_connections 2024-02-02T15:19:55.407 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:19:55.407+0000 7f34e9c53700 1 -- 172.21.15.142:0/1686067573 wait complete. 2024-02-02T15:19:55.810 DEBUG:teuthology.orchestra.run.smithi071:mon.b> sudo journalctl -f -n 0 -u ceph-371a409c-c1de-11ee-95b6-87774f69a715@mon.b.service 2024-02-02T15:19:55.813 DEBUG:teuthology.orchestra.run.smithi142:mon.c> sudo journalctl -f -n 0 -u ceph-371a409c-c1de-11ee-95b6-87774f69a715@mon.c.service 2024-02-02T15:19:55.815 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-02T15:19:55.815 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph mon dump -f json 2024-02-02T15:19:55.847 INFO:journalctl@ceph.mon.c.smithi142.stdout:-- Logs begin at Fri 2024-02-02 14:58:43 UTC. -- 2024-02-02T15:19:55.849 INFO:journalctl@ceph.mon.b.smithi071.stdout:-- Logs begin at Fri 2024-02-02 14:58:45 UTC. -- 2024-02-02T15:19:56.220 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-02T15:19:56.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:56 smithi001 ceph-mon[35470]: from='client.14190 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi001:172.21.15.1=a;smithi071:172.21.15.71=b;smithi142:172.21.15.142=c", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:19:56.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:56 smithi001 ceph-mon[35470]: Saving service mon spec with placement smithi001:172.21.15.1=a;smithi071:172.21.15.71=b;smithi142:172.21.15.142=c;count:3 2024-02-02T15:19:56.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:56 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:19:56.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:56 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:19:56.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:56 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:19:56.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:56 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:19:56.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:56 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:19:56.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:56 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-02T15:19:56.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:56 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:19:56.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:56 smithi001 ceph-mon[35470]: Deploying daemon mon.c on smithi142 2024-02-02T15:19:56.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:56 smithi001 ceph-mon[35470]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-02T15:19:57.424 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:19:57.425+0000 7fab19154700 1 -- 172.21.15.142:0/2768545533 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fab14105730 msgr2=0x7fab14105b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:19:57.425 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:19:57.425+0000 7fab19154700 1 --2- 172.21.15.142:0/2768545533 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fab14105730 0x7fab14105b10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fab04009910 tx=0x7fab04009c20 comp rx=0 tx=0).stop 2024-02-02T15:19:57.425 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:19:57.425+0000 7fab19154700 1 -- 172.21.15.142:0/2768545533 shutdown_connections 2024-02-02T15:19:57.425 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:19:57.425+0000 7fab19154700 1 --2- 172.21.15.142:0/2768545533 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fab14105730 0x7fab14105b10 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:57.425 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:19:57.425+0000 7fab19154700 1 -- 172.21.15.142:0/2768545533 >> 172.21.15.142:0/2768545533 conn(0x7fab14075240 msgr2=0x7fab14075650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:19:57.425 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:19:57.425+0000 7fab19154700 1 -- 172.21.15.142:0/2768545533 shutdown_connections 2024-02-02T15:19:57.425 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:19:57.425+0000 7fab19154700 1 -- 172.21.15.142:0/2768545533 wait complete. 2024-02-02T15:19:57.427 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:19:57.426+0000 7fab19154700 1 Processor -- start 2024-02-02T15:19:57.427 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:19:57.426+0000 7fab19154700 1 -- start start 2024-02-02T15:19:57.427 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:19:57.426+0000 7fab19154700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fab14105730 0x7fab14196b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:19:57.427 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:19:57.426+0000 7fab19154700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab14197090 con 0x7fab14105730 2024-02-02T15:19:57.427 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:19:57.427+0000 7fab12d9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fab14105730 0x7fab14196b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:19:57.427 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:19:57.427+0000 7fab12d9d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fab14105730 0x7fab14196b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.142:36390/0 (socket says 172.21.15.142:36390) 2024-02-02T15:19:57.427 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:19:57.427+0000 7fab12d9d700 1 -- 172.21.15.142:0/1434325514 learned_addr learned my addr 172.21.15.142:0/1434325514 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-02-02T15:19:57.427 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:19:57.427+0000 7fab12d9d700 1 -- 172.21.15.142:0/1434325514 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab040095f0 con 0x7fab14105730 2024-02-02T15:19:57.428 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:19:57.427+0000 7fab12d9d700 1 --2- 172.21.15.142:0/1434325514 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fab14105730 0x7fab14196b50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fab04005850 tx=0x7fab04004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:19:57.428 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:19:57.427+0000 7faafbfff700 1 -- 172.21.15.142:0/1434325514 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fab0401d070 con 0x7fab14105730 2024-02-02T15:19:57.428 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:19:57.427+0000 7fab19154700 1 -- 172.21.15.142:0/1434325514 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab14197290 con 0x7fab14105730 2024-02-02T15:19:57.428 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:19:57.427+0000 7fab19154700 1 -- 172.21.15.142:0/1434325514 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab141976b0 con 0x7fab14105730 2024-02-02T15:19:57.428 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:19:57.427+0000 7faafbfff700 1 -- 172.21.15.142:0/1434325514 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fab04004370 con 0x7fab14105730 2024-02-02T15:19:57.428 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:19:57.427+0000 7faafbfff700 1 -- 172.21.15.142:0/1434325514 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fab04003ab0 con 0x7fab14105730 2024-02-02T15:19:57.429 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:19:57.428+0000 7fab19154700 1 -- 172.21.15.142:0/1434325514 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab14103970 con 0x7fab14105730 2024-02-02T15:19:57.429 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:19:57.428+0000 7faafbfff700 1 -- 172.21.15.142:0/1434325514 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) v1 ==== 45323+0+0 (secure 0 0 0) 0x7fab04003c10 con 0x7fab14105730 2024-02-02T15:19:57.429 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:19:57.428+0000 7faafbfff700 1 --2- 172.21.15.142:0/1434325514 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7faafc0383a0 0x7faafc03a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:19:57.429 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:19:57.429+0000 7faafbfff700 1 -- 172.21.15.142:0/1434325514 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fab0404c290 con 0x7fab14105730 2024-02-02T15:19:57.429 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:19:57.429+0000 7fab1259c700 1 --2- 172.21.15.142:0/1434325514 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7faafc0383a0 0x7faafc03a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:19:57.429 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:19:57.429+0000 7fab1259c700 1 --2- 172.21.15.142:0/1434325514 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7faafc0383a0 0x7faafc03a860 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fab08006fd0 tx=0x7fab08006e40 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:19:57.433 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:19:57.435+0000 7faafbfff700 1 -- 172.21.15.142:0/1434325514 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7fab0400b750 con 0x7fab14105730 2024-02-02T15:19:57.719 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:19:57.719+0000 7fab19154700 1 -- 172.21.15.142:0/1434325514 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fab14068530 con 0x7fab14105730 2024-02-02T15:19:57.719 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:19:57.720+0000 7faafbfff700 1 -- 172.21.15.142:0/1434325514 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+732 (secure 0 0 0) 0x7fab04026030 con 0x7fab14105730 2024-02-02T15:19:57.720 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-02-02T15:19:57.720 INFO:teuthology.orchestra.run.smithi142.stdout:{"epoch":1,"fsid":"371a409c-c1de-11ee-95b6-87774f69a715","modified":"2024-02-02T15:18:23.966833Z","created":"2024-02-02T15:18:23.966833Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:3300","nonce":0},{"type":"v1","addr":"172.21.15.1:6789","nonce":0}]},"addr":"172.21.15.1:6789/0","public_addr":"172.21.15.1:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-02T15:19:57.722 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:19:57.723+0000 7fab19154700 1 -- 172.21.15.142:0/1434325514 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7faafc0383a0 msgr2=0x7faafc03a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:19:57.722 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:19:57.723+0000 7fab19154700 1 --2- 172.21.15.142:0/1434325514 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7faafc0383a0 0x7faafc03a860 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fab08006fd0 tx=0x7fab08006e40 comp rx=0 tx=0).stop 2024-02-02T15:19:57.722 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:19:57.723+0000 7fab19154700 1 -- 172.21.15.142:0/1434325514 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fab14105730 msgr2=0x7fab14196b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:19:57.722 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:19:57.723+0000 7fab19154700 1 --2- 172.21.15.142:0/1434325514 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fab14105730 0x7fab14196b50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fab04005850 tx=0x7fab04004ff0 comp rx=0 tx=0).stop 2024-02-02T15:19:57.722 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:19:57.723+0000 7fab19154700 1 -- 172.21.15.142:0/1434325514 shutdown_connections 2024-02-02T15:19:57.722 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:19:57.723+0000 7fab19154700 1 --2- 172.21.15.142:0/1434325514 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7faafc0383a0 0x7faafc03a860 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:57.722 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:19:57.723+0000 7fab19154700 1 --2- 172.21.15.142:0/1434325514 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fab14105730 0x7fab14196b50 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:19:57.722 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:19:57.723+0000 7fab19154700 1 -- 172.21.15.142:0/1434325514 >> 172.21.15.142:0/1434325514 conn(0x7fab14075240 msgr2=0x7fab140fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:19:57.723 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:19:57.723+0000 7fab19154700 1 -- 172.21.15.142:0/1434325514 shutdown_connections 2024-02-02T15:19:57.723 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:19:57.723+0000 7fab19154700 1 -- 172.21.15.142:0/1434325514 wait complete. 2024-02-02T15:19:57.723 INFO:teuthology.orchestra.run.smithi142.stderr:dumped monmap epoch 1 2024-02-02T15:19:58.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:19:57 smithi001 ceph-mon[35470]: from='client.? 172.21.15.142:0/1434325514' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-02T15:19:59.098 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-02T15:19:59.098 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph mon dump -f json 2024-02-02T15:19:59.127 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:19:58 smithi142 systemd[1]: Started Ceph mon.c for 371a409c-c1de-11ee-95b6-87774f69a715. 2024-02-02T15:19:59.127 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:19:58 smithi142 ceph-mon[45728]: rocksdb: EVENT_LOG_v1 {"time_micros": 1706887198691520, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-02-02T15:19:59.127 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:19:58 smithi142 ceph-mon[45728]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-02-02T15:19:59.127 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:19:58 smithi142 ceph-mon[45728]: rocksdb: EVENT_LOG_v1 {"time_micros": 1706887198692076, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1706887198, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "8b680b0e-8ca3-4495-94d3-85fe11146f82", "db_session_id": "3FBPF1K78AZZ1HB3WU7Z", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-02-02T15:19:59.127 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:19:58 smithi142 ceph-mon[45728]: rocksdb: EVENT_LOG_v1 {"time_micros": 1706887198692167, "job": 1, "event": "recovery_finished"} 2024-02-02T15:19:59.127 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:19:58 smithi142 ceph-mon[45728]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-02-02T15:19:59.127 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:19:58 smithi142 ceph-mon[45728]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-02-02T15:19:59.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:19:58 smithi142 ceph-mon[45728]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x559e9d30a000 2024-02-02T15:19:59.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:19:58 smithi142 ceph-mon[45728]: rocksdb: DB pointer 0x559e9d2f6000 2024-02-02T15:19:59.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:19:58 smithi142 ceph-mon[45728]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-02-02T15:19:59.128 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:19:58 smithi142 ceph-mon[45728]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-02-02T15:19:59.128 INFO:journalctl@ceph.mon.c.smithi142.stdout: ** DB Stats ** 2024-02-02T15:19:59.128 INFO:journalctl@ceph.mon.c.smithi142.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-02T15:19:59.128 INFO:journalctl@ceph.mon.c.smithi142.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-02T15:19:59.128 INFO:journalctl@ceph.mon.c.smithi142.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-02T15:19:59.128 INFO:journalctl@ceph.mon.c.smithi142.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-02T15:19:59.128 INFO:journalctl@ceph.mon.c.smithi142.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-02T15:19:59.129 INFO:journalctl@ceph.mon.c.smithi142.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-02T15:19:59.129 INFO:journalctl@ceph.mon.c.smithi142.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-02T15:19:59.129 INFO:journalctl@ceph.mon.c.smithi142.stdout: 2024-02-02T15:19:59.129 INFO:journalctl@ceph.mon.c.smithi142.stdout: ** Compaction Stats [default] ** 2024-02-02T15:19:59.129 INFO:journalctl@ceph.mon.c.smithi142.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-02T15:19:59.129 INFO:journalctl@ceph.mon.c.smithi142.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-02-02T15:19:59.129 INFO:journalctl@ceph.mon.c.smithi142.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-02T15:19:59.129 INFO:journalctl@ceph.mon.c.smithi142.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-02T15:19:59.129 INFO:journalctl@ceph.mon.c.smithi142.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-02T15:19:59.129 INFO:journalctl@ceph.mon.c.smithi142.stdout: 2024-02-02T15:19:59.129 INFO:journalctl@ceph.mon.c.smithi142.stdout: ** Compaction Stats [default] ** 2024-02-02T15:19:59.129 INFO:journalctl@ceph.mon.c.smithi142.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-02T15:19:59.130 INFO:journalctl@ceph.mon.c.smithi142.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-02T15:19:59.130 INFO:journalctl@ceph.mon.c.smithi142.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-02T15:19:59.130 INFO:journalctl@ceph.mon.c.smithi142.stdout: 2024-02-02T15:19:59.130 INFO:journalctl@ceph.mon.c.smithi142.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-02-02T15:19:59.130 INFO:journalctl@ceph.mon.c.smithi142.stdout: 2024-02-02T15:19:59.130 INFO:journalctl@ceph.mon.c.smithi142.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-02T15:19:59.130 INFO:journalctl@ceph.mon.c.smithi142.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-02-02T15:19:59.130 INFO:journalctl@ceph.mon.c.smithi142.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-02T15:19:59.130 INFO:journalctl@ceph.mon.c.smithi142.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-02-02T15:19:59.130 INFO:journalctl@ceph.mon.c.smithi142.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-02-02T15:19:59.130 INFO:journalctl@ceph.mon.c.smithi142.stdout: AddFile(Keys): cumulative 0, interval 0 2024-02-02T15:19:59.130 INFO:journalctl@ceph.mon.c.smithi142.stdout: Cumulative compaction: 0.00 GB write, 0.50 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-02T15:19:59.130 INFO:journalctl@ceph.mon.c.smithi142.stdout: Interval compaction: 0.00 GB write, 0.50 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-02T15:19:59.131 INFO:journalctl@ceph.mon.c.smithi142.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-02T15:19:59.131 INFO:journalctl@ceph.mon.c.smithi142.stdout: Block cache BinnedLRUCache@0x559e9d26d350#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-02-02T15:19:59.131 INFO:journalctl@ceph.mon.c.smithi142.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-02-02T15:19:59.131 INFO:journalctl@ceph.mon.c.smithi142.stdout: 2024-02-02T15:19:59.131 INFO:journalctl@ceph.mon.c.smithi142.stdout: ** File Read Latency Histogram By Level [default] ** 2024-02-02T15:19:59.131 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:19:58 smithi142 ceph-mon[45728]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-02-02T15:19:59.131 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:19:58 smithi142 ceph-mon[45728]: using public_addr v2:172.21.15.142:0/0 -> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] 2024-02-02T15:19:59.131 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:19:58 smithi142 ceph-mon[45728]: starting mon.c rank -1 at public addrs [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] at bind addrs [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 371a409c-c1de-11ee-95b6-87774f69a715 2024-02-02T15:19:59.131 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:19:58 smithi142 ceph-mon[45728]: mon.c@-1(???) e0 preinit fsid 371a409c-c1de-11ee-95b6-87774f69a715 2024-02-02T15:19:59.131 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:19:58 smithi142 ceph-mon[45728]: mon.c@-1(synchronizing).mds e1 new map 2024-02-02T15:19:59.131 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:19:58 smithi142 ceph-mon[45728]: mon.c@-1(synchronizing).mds e1 print_map 2024-02-02T15:19:59.132 INFO:journalctl@ceph.mon.c.smithi142.stdout: e1 2024-02-02T15:19:59.132 INFO:journalctl@ceph.mon.c.smithi142.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-02T15:19:59.132 INFO:journalctl@ceph.mon.c.smithi142.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-02T15:19:59.132 INFO:journalctl@ceph.mon.c.smithi142.stdout: legacy client fscid: -1 2024-02-02T15:19:59.132 INFO:journalctl@ceph.mon.c.smithi142.stdout: 2024-02-02T15:19:59.132 INFO:journalctl@ceph.mon.c.smithi142.stdout: No filesystems configured 2024-02-02T15:19:59.132 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:19:58 smithi142 ceph-mon[45728]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-02-02T15:19:59.132 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:19:58 smithi142 ceph-mon[45728]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-02-02T15:19:59.132 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:19:58 smithi142 ceph-mon[45728]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-02-02T15:19:59.132 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:19:58 smithi142 ceph-mon[45728]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-02-02T15:19:59.132 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:19:58 smithi142 ceph-mon[45728]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-02-02T15:19:59.133 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:19:58 smithi142 ceph-mon[45728]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-02-02T15:19:59.133 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:19:58 smithi142 ceph-mon[45728]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-02T15:19:59.133 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:19:58 smithi142 ceph-mon[45728]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-02T15:19:59.133 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:19:58 smithi142 ceph-mon[45728]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-02T15:19:59.133 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:19:58 smithi142 ceph-mon[45728]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-02T15:19:59.133 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:19:58 smithi142 ceph-mon[45728]: Added host smithi142 2024-02-02T15:19:59.133 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:19:58 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:19:59.133 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:19:58 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:19:59.133 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:19:58 smithi142 ceph-mon[45728]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-02T15:19:59.133 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:19:58 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:19:59.133 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:19:58 smithi142 ceph-mon[45728]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-02T15:19:59.133 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:19:58 smithi142 ceph-mon[45728]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-02T15:19:59.133 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:19:58 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:19:59.134 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:19:58 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:19:59.134 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:19:58 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:19:59.134 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:19:58 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:19:59.134 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:19:58 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi142", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:19:59.134 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:19:58 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:19:59.134 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:19:58 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:19:59.134 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:19:58 smithi142 ceph-mon[45728]: Updating smithi142:/etc/ceph/ceph.conf 2024-02-02T15:19:59.134 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:19:58 smithi142 ceph-mon[45728]: Updating smithi142:/var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-02T15:19:59.134 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:19:58 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/480496878' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-02-02T15:19:59.134 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:19:58 smithi142 ceph-mon[45728]: Updating smithi142:/etc/ceph/ceph.client.admin.keyring 2024-02-02T15:19:59.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:19:58 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:19:59.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:19:58 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:19:59.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:19:58 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:19:59.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:19:58 smithi142 ceph-mon[45728]: Updating smithi142:/var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-02T15:19:59.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:19:58 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/480496878' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-02-02T15:19:59.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:19:58 smithi142 ceph-mon[45728]: osdmap e4: 0 total, 0 up, 0 in 2024-02-02T15:19:59.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:19:58 smithi142 ceph-mon[45728]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-02T15:19:59.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:19:58 smithi142 ceph-mon[45728]: from='client.14190 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi001:172.21.15.1=a;smithi071:172.21.15.71=b;smithi142:172.21.15.142=c", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:19:59.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:19:58 smithi142 ceph-mon[45728]: Saving service mon spec with placement smithi001:172.21.15.1=a;smithi071:172.21.15.71=b;smithi142:172.21.15.142=c;count:3 2024-02-02T15:19:59.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:19:58 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:19:59.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:19:58 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:19:59.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:19:58 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:19:59.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:19:58 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:19:59.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:19:58 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:19:59.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:19:58 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-02T15:19:59.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:19:58 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:19:59.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:19:58 smithi142 ceph-mon[45728]: Deploying daemon mon.c on smithi142 2024-02-02T15:19:59.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:19:58 smithi142 ceph-mon[45728]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-02T15:19:59.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:19:58 smithi142 ceph-mon[45728]: from='client.? 172.21.15.142:0/1434325514' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-02T15:19:59.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:19:58 smithi142 ceph-mon[45728]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-02-02T15:19:59.136 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:19:58 smithi142 ceph-mon[45728]: mon.c@-1(synchronizing) e1 handle_conf_change mon_allow_pool_delete 2024-02-02T15:19:59.323 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.c/config 2024-02-02T15:20:04.010 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:04.009+0000 7f08c27e2700 1 -- 172.21.15.142:0/2031559762 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f08a80035c0 msgr2=0x7f08a8005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:20:04.010 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:04.009+0000 7f08c27e2700 1 --2- 172.21.15.142:0/2031559762 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f08a80035c0 0x7f08a8005a50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f08bc0ff720 tx=0x7f08a4014ec0 comp rx=0 tx=0).stop 2024-02-02T15:20:04.010 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:04.009+0000 7f08c27e2700 1 -- 172.21.15.142:0/2031559762 shutdown_connections 2024-02-02T15:20:04.010 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:04.009+0000 7f08c27e2700 1 --2- 172.21.15.142:0/2031559762 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f08a80035c0 0x7f08a8005a50 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:20:04.010 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:04.009+0000 7f08c27e2700 1 -- 172.21.15.142:0/2031559762 >> 172.21.15.142:0/2031559762 conn(0x7f08bc0fb300 msgr2=0x7f08bc0fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:20:04.010 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:04.009+0000 7f08c27e2700 1 -- 172.21.15.142:0/2031559762 shutdown_connections 2024-02-02T15:20:04.010 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:04.009+0000 7f08c27e2700 1 -- 172.21.15.142:0/2031559762 wait complete. 2024-02-02T15:20:04.011 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:04.010+0000 7f08c27e2700 1 Processor -- start 2024-02-02T15:20:04.012 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:04.010+0000 7f08c27e2700 1 -- start start 2024-02-02T15:20:04.012 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:04.011+0000 7f08c27e2700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08a80035c0 0x7f08bc196c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:20:04.012 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:04.011+0000 7f08c27e2700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f08bc197170 0x7f08bc1975f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:20:04.012 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:04.011+0000 7f08c27e2700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08bc19fca0 con 0x7f08bc197170 2024-02-02T15:20:04.012 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:04.011+0000 7f08c27e2700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08bc19fde0 con 0x7f08a80035c0 2024-02-02T15:20:04.012 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:04.011+0000 7f08bbfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08a80035c0 0x7f08bc196c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:20:04.012 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:04.011+0000 7f08bbfff700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08a80035c0 0x7f08bc196c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:40330/0 (socket says 172.21.15.142:40330) 2024-02-02T15:20:04.012 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:04.011+0000 7f08bbfff700 1 -- 172.21.15.142:0/2604503560 learned_addr learned my addr 172.21.15.142:0/2604503560 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-02-02T15:20:04.012 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:04.011+0000 7f08bbfff700 1 -- 172.21.15.142:0/2604503560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08a80035c0 msgr2=0x7f08bc196c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-02-02T15:20:04.012 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:04.011+0000 7f08bbfff700 1 -- 172.21.15.142:0/2604503560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08a80035c0 msgr2=0x7f08bc196c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-02-02T15:20:04.012 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:04.011+0000 7f08bbfff700 1 --2- 172.21.15.142:0/2604503560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08a80035c0 0x7f08bc196c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-02-02T15:20:04.012 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:04.011+0000 7f08bbfff700 1 --2- 172.21.15.142:0/2604503560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08a80035c0 0x7f08bc196c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-02T15:20:04.013 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:04.012+0000 7f08bb7fe700 1 --2- 172.21.15.142:0/2604503560 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f08bc197170 0x7f08bc1975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:20:04.014 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:04.012+0000 7f08bb7fe700 1 -- 172.21.15.142:0/2604503560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08a80035c0 msgr2=0x7f08bc196c30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-02T15:20:04.014 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:04.012+0000 7f08bb7fe700 1 --2- 172.21.15.142:0/2604503560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08a80035c0 0x7f08bc196c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:20:04.014 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:04.012+0000 7f08bb7fe700 1 -- 172.21.15.142:0/2604503560 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08a4014910 con 0x7f08bc197170 2024-02-02T15:20:04.014 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:04.012+0000 7f08bb7fe700 1 --2- 172.21.15.142:0/2604503560 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f08bc197170 0x7f08bc1975f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f08ac00b4d0 tx=0x7f08ac00b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:20:04.014 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:04.012+0000 7f08b97fa700 1 -- 172.21.15.142:0/2604503560 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f08ac011630 con 0x7f08bc197170 2024-02-02T15:20:04.014 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:04.013+0000 7f08b97fa700 1 -- 172.21.15.142:0/2604503560 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f08ac011c50 con 0x7f08bc197170 2024-02-02T15:20:04.014 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:04.013+0000 7f08c27e2700 1 -- 172.21.15.142:0/2604503560 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08bc19ffe0 con 0x7f08bc197170 2024-02-02T15:20:04.014 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:04.013+0000 7f08b97fa700 1 -- 172.21.15.142:0/2604503560 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f08ac010d90 con 0x7f08bc197170 2024-02-02T15:20:04.014 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:04.013+0000 7f08c27e2700 1 -- 172.21.15.142:0/2604503560 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08bc1a0500 con 0x7f08bc197170 2024-02-02T15:20:04.014 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:04.014+0000 7f08b97fa700 1 -- 172.21.15.142:0/2604503560 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) v1 ==== 45323+0+0 (secure 0 0 0) 0x7f08ac01a400 con 0x7f08bc197170 2024-02-02T15:20:04.015 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:04.014+0000 7f08c27e2700 1 -- 172.21.15.142:0/2604503560 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08bc062370 con 0x7f08bc197170 2024-02-02T15:20:04.015 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:04.014+0000 7f08b97fa700 1 --2- 172.21.15.142:0/2604503560 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f089c0381e0 0x7f089c03a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:20:04.015 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:04.014+0000 7f08b97fa700 1 -- 172.21.15.142:0/2604503560 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f08ac04ac00 con 0x7f08bc197170 2024-02-02T15:20:04.015 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:04.014+0000 7f08bbfff700 1 --2- 172.21.15.142:0/2604503560 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f089c0381e0 0x7f089c03a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:20:04.015 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:04.015+0000 7f08bbfff700 1 --2- 172.21.15.142:0/2604503560 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f089c0381e0 0x7f089c03a6a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f08a400f470 tx=0x7f08a400f3c0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:20:04.019 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:04.020+0000 7f08b97fa700 1 -- 172.21.15.142:0/2604503560 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f08ac017680 con 0x7f08bc197170 2024-02-02T15:20:04.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:03 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-02T15:20:04.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:03 smithi142 ceph-mon[45728]: mon.a calling monitor election 2024-02-02T15:20:04.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:03 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-02T15:20:04.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:03 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-02T15:20:04.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:03 smithi142 ceph-mon[45728]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-02T15:20:04.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:03 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-02T15:20:04.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:03 smithi142 ceph-mon[45728]: mon.c calling monitor election 2024-02-02T15:20:04.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:03 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-02T15:20:04.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:03 smithi142 ceph-mon[45728]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-02T15:20:04.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:03 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-02T15:20:04.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:03 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-02T15:20:04.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:03 smithi142 ceph-mon[45728]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-02T15:20:04.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:03 smithi142 ceph-mon[45728]: monmap e2: 2 mons at {a=[v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0],c=[v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-02T15:20:04.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:03 smithi142 ceph-mon[45728]: fsmap 2024-02-02T15:20:04.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:03 smithi142 ceph-mon[45728]: osdmap e4: 0 total, 0 up, 0 in 2024-02-02T15:20:04.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:03 smithi142 ceph-mon[45728]: mgrmap e13: x(active, since 37s) 2024-02-02T15:20:04.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:03 smithi142 ceph-mon[45728]: overall HEALTH_OK 2024-02-02T15:20:04.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:03 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:04.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:03 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:04.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:03 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-02T15:20:04.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:03 smithi001 ceph-mon[35470]: mon.a calling monitor election 2024-02-02T15:20:04.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:03 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-02T15:20:04.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:03 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-02T15:20:04.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:03 smithi001 ceph-mon[35470]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-02T15:20:04.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:03 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-02T15:20:04.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:03 smithi001 ceph-mon[35470]: mon.c calling monitor election 2024-02-02T15:20:04.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:03 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-02T15:20:04.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:03 smithi001 ceph-mon[35470]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-02T15:20:04.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:03 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-02T15:20:04.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:03 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-02T15:20:04.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:03 smithi001 ceph-mon[35470]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-02T15:20:04.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:03 smithi001 ceph-mon[35470]: monmap e2: 2 mons at {a=[v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0],c=[v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-02T15:20:04.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:03 smithi001 ceph-mon[35470]: fsmap 2024-02-02T15:20:04.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:03 smithi001 ceph-mon[35470]: osdmap e4: 0 total, 0 up, 0 in 2024-02-02T15:20:04.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:03 smithi001 ceph-mon[35470]: mgrmap e13: x(active, since 37s) 2024-02-02T15:20:04.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:03 smithi001 ceph-mon[35470]: overall HEALTH_OK 2024-02-02T15:20:04.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:03 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:04.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:03 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:04.291 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:04.290+0000 7f08c27e2700 1 -- 172.21.15.142:0/2604503560 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f08bc069320 con 0x7f08bc197170 2024-02-02T15:20:04.291 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:04.291+0000 7f08b97fa700 1 -- 172.21.15.142:0/2604503560 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1002 (secure 0 0 0) 0x7f08ac0172c0 con 0x7f08bc197170 2024-02-02T15:20:04.291 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-02-02T15:20:04.292 INFO:teuthology.orchestra.run.smithi142.stdout:{"epoch":2,"fsid":"371a409c-c1de-11ee-95b6-87774f69a715","modified":"2024-02-02T15:19:58.722864Z","created":"2024-02-02T15:18:23.966833Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:3300","nonce":0},{"type":"v1","addr":"172.21.15.1:6789","nonce":0}]},"addr":"172.21.15.1:6789/0","public_addr":"172.21.15.1:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:3300","nonce":0},{"type":"v1","addr":"172.21.15.142:6789","nonce":0}]},"addr":"172.21.15.142:6789/0","public_addr":"172.21.15.142:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-02-02T15:20:04.294 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:04.293+0000 7f08c27e2700 1 -- 172.21.15.142:0/2604503560 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f089c0381e0 msgr2=0x7f089c03a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:20:04.294 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:04.293+0000 7f08c27e2700 1 --2- 172.21.15.142:0/2604503560 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f089c0381e0 0x7f089c03a6a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f08a400f470 tx=0x7f08a400f3c0 comp rx=0 tx=0).stop 2024-02-02T15:20:04.294 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:04.293+0000 7f08c27e2700 1 -- 172.21.15.142:0/2604503560 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f08bc197170 msgr2=0x7f08bc1975f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:20:04.294 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:04.293+0000 7f08c27e2700 1 --2- 172.21.15.142:0/2604503560 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f08bc197170 0x7f08bc1975f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f08ac00b4d0 tx=0x7f08ac00b890 comp rx=0 tx=0).stop 2024-02-02T15:20:04.294 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:04.293+0000 7f08c27e2700 1 -- 172.21.15.142:0/2604503560 shutdown_connections 2024-02-02T15:20:04.294 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:04.293+0000 7f08c27e2700 1 --2- 172.21.15.142:0/2604503560 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f089c0381e0 0x7f089c03a6a0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:20:04.294 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:04.293+0000 7f08c27e2700 1 --2- 172.21.15.142:0/2604503560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08a80035c0 0x7f08bc196c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:20:04.294 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:04.293+0000 7f08c27e2700 1 --2- 172.21.15.142:0/2604503560 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f08bc197170 0x7f08bc1975f0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:20:04.294 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:04.293+0000 7f08c27e2700 1 -- 172.21.15.142:0/2604503560 >> 172.21.15.142:0/2604503560 conn(0x7f08bc0fb300 msgr2=0x7f08bc18d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:20:04.294 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:04.293+0000 7f08c27e2700 1 -- 172.21.15.142:0/2604503560 shutdown_connections 2024-02-02T15:20:04.294 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:04.293+0000 7f08c27e2700 1 -- 172.21.15.142:0/2604503560 wait complete. 2024-02-02T15:20:04.295 INFO:teuthology.orchestra.run.smithi142.stderr:dumped monmap epoch 2 2024-02-02T15:20:04.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:04 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:04.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:04 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-02T15:20:04.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:04 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:20:04.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:04 smithi142 ceph-mon[45728]: Deploying daemon mon.b on smithi071 2024-02-02T15:20:04.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:04 smithi142 ceph-mon[45728]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-02T15:20:04.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:04 smithi142 ceph-mon[45728]: from='client.? 172.21.15.142:0/2604503560' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-02T15:20:04.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:04 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-02T15:20:05.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:04 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:05.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:04 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-02T15:20:05.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:04 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:20:05.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:04 smithi001 ceph-mon[35470]: Deploying daemon mon.b on smithi071 2024-02-02T15:20:05.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:04 smithi001 ceph-mon[35470]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-02T15:20:05.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:04 smithi001 ceph-mon[35470]: from='client.? 172.21.15.142:0/2604503560' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-02T15:20:05.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:04 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-02T15:20:05.712 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-02T15:20:05.712 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph mon dump -f json 2024-02-02T15:20:05.936 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.c/config 2024-02-02T15:20:06.190 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:20:05 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:20:05.722+0000 7fd892000700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-02-02T15:20:06.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:06 smithi071 ceph-mon[44707]: mon.b@-1(synchronizing) e2 handle_conf_change mon_allow_pool_delete 2024-02-02T15:20:06.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:06 smithi071 systemd[1]: Started Ceph mon.b for 371a409c-c1de-11ee-95b6-87774f69a715. 2024-02-02T15:20:11.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:11 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-02T15:20:11.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:11 smithi142 ceph-mon[45728]: mon.c calling monitor election 2024-02-02T15:20:11.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:11 smithi142 ceph-mon[45728]: mon.a calling monitor election 2024-02-02T15:20:11.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:11 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-02T15:20:11.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:11 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-02T15:20:11.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:11 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-02T15:20:11.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:11 smithi142 ceph-mon[45728]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-02T15:20:11.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:11 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-02T15:20:11.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:11 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-02T15:20:11.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:11 smithi142 ceph-mon[45728]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-02T15:20:11.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:11 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-02T15:20:11.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:11 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-02T15:20:11.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:11 smithi142 ceph-mon[45728]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-02T15:20:11.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:11 smithi142 ceph-mon[45728]: monmap e3: 3 mons at {a=[v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0],b=[v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0],c=[v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-02T15:20:11.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:11 smithi142 ceph-mon[45728]: fsmap 2024-02-02T15:20:11.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:11 smithi142 ceph-mon[45728]: osdmap e4: 0 total, 0 up, 0 in 2024-02-02T15:20:11.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:11 smithi142 ceph-mon[45728]: mgrmap e13: x(active, since 45s) 2024-02-02T15:20:11.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:11 smithi142 ceph-mon[45728]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-02-02T15:20:11.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:11 smithi142 ceph-mon[45728]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-02-02T15:20:11.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:11 smithi142 ceph-mon[45728]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-02-02T15:20:11.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:11 smithi142 ceph-mon[45728]: mon.b (rank 2) addr [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] is down (out of quorum) 2024-02-02T15:20:11.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:11 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:11.922 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:11 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:11.922 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:11 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:11.922 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:11 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:11.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:11 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-02T15:20:11.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:11 smithi001 ceph-mon[35470]: mon.c calling monitor election 2024-02-02T15:20:11.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:11 smithi001 ceph-mon[35470]: mon.a calling monitor election 2024-02-02T15:20:11.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:11 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-02T15:20:11.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:11 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-02T15:20:11.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:11 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-02T15:20:11.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:11 smithi001 ceph-mon[35470]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-02T15:20:11.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:11 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-02T15:20:11.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:11 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-02T15:20:11.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:11 smithi001 ceph-mon[35470]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-02T15:20:11.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:11 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-02T15:20:11.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:11 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-02T15:20:11.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:11 smithi001 ceph-mon[35470]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-02T15:20:11.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:11 smithi001 ceph-mon[35470]: monmap e3: 3 mons at {a=[v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0],b=[v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0],c=[v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-02T15:20:11.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:11 smithi001 ceph-mon[35470]: fsmap 2024-02-02T15:20:11.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:11 smithi001 ceph-mon[35470]: osdmap e4: 0 total, 0 up, 0 in 2024-02-02T15:20:11.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:11 smithi001 ceph-mon[35470]: mgrmap e13: x(active, since 45s) 2024-02-02T15:20:11.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:11 smithi001 ceph-mon[35470]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-02-02T15:20:11.942 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:11 smithi001 ceph-mon[35470]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-02-02T15:20:11.942 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:11 smithi001 ceph-mon[35470]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-02-02T15:20:11.942 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:11 smithi001 ceph-mon[35470]: mon.b (rank 2) addr [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] is down (out of quorum) 2024-02-02T15:20:11.942 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:11 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:11.942 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:11 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:11.942 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:11 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:11.942 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:11 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:12.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:12 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:20:12.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:12 smithi142 ceph-mon[45728]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-02T15:20:12.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:12 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-02T15:20:12.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:12 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:12.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:12 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:12.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:12 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:12.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:12 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:12.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:12 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:20:12.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:12 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:20:12.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:12 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:20:12.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:12 smithi001 ceph-mon[35470]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-02T15:20:12.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:12 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-02T15:20:12.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:12 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:12.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:12 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:12.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:12 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:12.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:12 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:12.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:12 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:20:12.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:12 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:20:12.963 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:12.963+0000 7f5611990700 1 -- 172.21.15.142:0/986475836 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f55f4006220 msgr2=0x7f55f40086b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:20:12.964 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:12.963+0000 7f5611990700 1 --2- 172.21.15.142:0/986475836 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f55f4006220 0x7f55f40086b0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f560c100060 tx=0x7f55fc00fe90 comp rx=0 tx=0).stop 2024-02-02T15:20:12.964 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:12.963+0000 7f5611990700 1 -- 172.21.15.142:0/986475836 shutdown_connections 2024-02-02T15:20:12.964 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:12.963+0000 7f5611990700 1 --2- 172.21.15.142:0/986475836 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f55f4006220 0x7f55f40086b0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:20:12.964 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:12.963+0000 7f5611990700 1 -- 172.21.15.142:0/986475836 >> 172.21.15.142:0/986475836 conn(0x7f560c074a90 msgr2=0x7f560c074ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:20:12.964 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:12.963+0000 7f5611990700 1 -- 172.21.15.142:0/986475836 shutdown_connections 2024-02-02T15:20:12.964 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:12.963+0000 7f5611990700 1 -- 172.21.15.142:0/986475836 wait complete. 2024-02-02T15:20:12.968 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:12.964+0000 7f5611990700 1 Processor -- start 2024-02-02T15:20:12.968 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:12.964+0000 7f5611990700 1 -- start start 2024-02-02T15:20:12.968 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:12.964+0000 7f5611990700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f55f4006220 0x7f560c196060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:20:12.968 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:12.964+0000 7f5611990700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f560c1965a0 0x7f560c19aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:20:12.968 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:12.964+0000 7f5611990700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f560c19af50 0x7f560c19b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:20:12.968 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:12.964+0000 7f5611990700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f560c196c90 con 0x7f560c19af50 2024-02-02T15:20:12.969 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:12.964+0000 7f5611990700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f560c1a2cc0 con 0x7f55f4006220 2024-02-02T15:20:12.969 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:12.964+0000 7f5611990700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f560c1a2e00 con 0x7f560c1965a0 2024-02-02T15:20:12.969 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:12.965+0000 7f560a7fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f560c1965a0 0x7f560c19aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:20:12.969 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:12.965+0000 7f560a7fc700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f560c1965a0 0x7f560c19aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:50430/0 (socket says 172.21.15.142:50430) 2024-02-02T15:20:12.969 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:12.965+0000 7f560a7fc700 1 -- 172.21.15.142:0/831471566 learned_addr learned my addr 172.21.15.142:0/831471566 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-02-02T15:20:12.969 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:12.965+0000 7f560a7fc700 1 -- 172.21.15.142:0/831471566 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f560c1965a0 msgr2=0x7f560c19aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-02-02T15:20:12.969 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:12.965+0000 7f560a7fc700 1 -- 172.21.15.142:0/831471566 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f560c1965a0 msgr2=0x7f560c19aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-02-02T15:20:12.970 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:12.965+0000 7f560a7fc700 1 --2- 172.21.15.142:0/831471566 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f560c1965a0 0x7f560c19aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-02-02T15:20:12.970 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:12.965+0000 7f560b7fe700 1 --2- 172.21.15.142:0/831471566 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f560c19af50 0x7f560c19b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:20:12.970 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:12.965+0000 7f560a7fc700 1 --2- 172.21.15.142:0/831471566 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f560c1965a0 0x7f560c19aa10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-02T15:20:12.970 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:12.965+0000 7f560affd700 1 --2- 172.21.15.142:0/831471566 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f55f4006220 0x7f560c196060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:20:12.970 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:12.965+0000 7f560affd700 1 -- 172.21.15.142:0/831471566 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f55f4006220 msgr2=0x7f560c196060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-02-02T15:20:12.970 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:12.965+0000 7f560affd700 1 -- 172.21.15.142:0/831471566 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f55f4006220 msgr2=0x7f560c196060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-02-02T15:20:12.970 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:12.965+0000 7f560affd700 1 --2- 172.21.15.142:0/831471566 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f55f4006220 0x7f560c196060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-02-02T15:20:12.970 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:12.965+0000 7f560affd700 1 --2- 172.21.15.142:0/831471566 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f55f4006220 0x7f560c196060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-02T15:20:12.970 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:12.965+0000 7f560b7fe700 1 -- 172.21.15.142:0/831471566 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f560c1965a0 msgr2=0x7f560c19aa10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-02T15:20:12.970 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:12.965+0000 7f560b7fe700 1 --2- 172.21.15.142:0/831471566 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f560c1965a0 0x7f560c19aa10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:20:12.970 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:12.965+0000 7f560b7fe700 1 -- 172.21.15.142:0/831471566 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f55f4006220 msgr2=0x7f560c196060 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-02T15:20:12.970 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:12.965+0000 7f560b7fe700 1 --2- 172.21.15.142:0/831471566 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f55f4006220 0x7f560c196060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:20:12.970 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:12.965+0000 7f560b7fe700 1 -- 172.21.15.142:0/831471566 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55fc00f8e0 con 0x7f560c19af50 2024-02-02T15:20:12.970 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:12.965+0000 7f560b7fe700 1 --2- 172.21.15.142:0/831471566 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f560c19af50 0x7f560c19b400 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f560400bd50 tx=0x7f560400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:20:12.971 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:12.966+0000 7f561098e700 1 -- 172.21.15.142:0/831471566 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f560400cd10 con 0x7f560c19af50 2024-02-02T15:20:12.971 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:12.966+0000 7f561098e700 1 -- 172.21.15.142:0/831471566 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f560400ce70 con 0x7f560c19af50 2024-02-02T15:20:12.971 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:12.966+0000 7f5611990700 1 -- 172.21.15.142:0/831471566 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f560c1a30e0 con 0x7f560c19af50 2024-02-02T15:20:12.971 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:12.966+0000 7f561098e700 1 -- 172.21.15.142:0/831471566 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5604012840 con 0x7f560c19af50 2024-02-02T15:20:12.971 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:12.966+0000 7f5611990700 1 -- 172.21.15.142:0/831471566 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f560c1a35e0 con 0x7f560c19af50 2024-02-02T15:20:12.971 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:12.967+0000 7f5611990700 1 -- 172.21.15.142:0/831471566 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f560c190460 con 0x7f560c19af50 2024-02-02T15:20:12.972 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:12.969+0000 7f561098e700 1 -- 172.21.15.142:0/831471566 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) v1 ==== 45323+0+0 (secure 0 0 0) 0x7f5604014b00 con 0x7f560c19af50 2024-02-02T15:20:12.972 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:12.969+0000 7f561098e700 1 --2- 172.21.15.142:0/831471566 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f55f8038820 0x7f55f803ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:20:12.972 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:12.969+0000 7f561098e700 1 -- 172.21.15.142:0/831471566 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f560404acf0 con 0x7f560c19af50 2024-02-02T15:20:12.972 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:12.969+0000 7f560affd700 1 --2- 172.21.15.142:0/831471566 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f55f8038820 0x7f55f803ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:20:12.972 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:12.970+0000 7f560affd700 1 --2- 172.21.15.142:0/831471566 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f55f8038820 0x7f55f803ace0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f55fc00f550 tx=0x7f55fc00f4a0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:20:12.976 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:12.976+0000 7f561098e700 1 -- 172.21.15.142:0/831471566 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f5604013bd0 con 0x7f560c19af50 2024-02-02T15:20:13.239 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:13.238+0000 7f5611990700 1 -- 172.21.15.142:0/831471566 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f560c18fe40 con 0x7f560c19af50 2024-02-02T15:20:13.239 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:13.239+0000 7f561098e700 1 -- 172.21.15.142:0/831471566 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1266 (secure 0 0 0) 0x7f5604019030 con 0x7f560c19af50 2024-02-02T15:20:13.239 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-02-02T15:20:13.240 INFO:teuthology.orchestra.run.smithi142.stdout:{"epoch":3,"fsid":"371a409c-c1de-11ee-95b6-87774f69a715","modified":"2024-02-02T15:20:06.397959Z","created":"2024-02-02T15:18:23.966833Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:3300","nonce":0},{"type":"v1","addr":"172.21.15.1:6789","nonce":0}]},"addr":"172.21.15.1:6789/0","public_addr":"172.21.15.1:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:3300","nonce":0},{"type":"v1","addr":"172.21.15.142:6789","nonce":0}]},"addr":"172.21.15.142:6789/0","public_addr":"172.21.15.142:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:3300","nonce":0},{"type":"v1","addr":"172.21.15.71:6789","nonce":0}]},"addr":"172.21.15.71:6789/0","public_addr":"172.21.15.71:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-02-02T15:20:13.241 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:13.241+0000 7f5611990700 1 -- 172.21.15.142:0/831471566 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f55f8038820 msgr2=0x7f55f803ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:20:13.241 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:13.241+0000 7f5611990700 1 --2- 172.21.15.142:0/831471566 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f55f8038820 0x7f55f803ace0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f55fc00f550 tx=0x7f55fc00f4a0 comp rx=0 tx=0).stop 2024-02-02T15:20:13.242 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:13.241+0000 7f5611990700 1 -- 172.21.15.142:0/831471566 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f560c19af50 msgr2=0x7f560c19b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:20:13.242 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:13.241+0000 7f5611990700 1 --2- 172.21.15.142:0/831471566 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f560c19af50 0x7f560c19b400 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f560400bd50 tx=0x7f560400be30 comp rx=0 tx=0).stop 2024-02-02T15:20:13.242 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:13.241+0000 7f5611990700 1 -- 172.21.15.142:0/831471566 shutdown_connections 2024-02-02T15:20:13.242 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:13.241+0000 7f5611990700 1 --2- 172.21.15.142:0/831471566 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f560c19af50 0x7f560c19b400 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:20:13.242 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:13.241+0000 7f5611990700 1 --2- 172.21.15.142:0/831471566 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f55f4006220 0x7f560c196060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:20:13.242 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:13.241+0000 7f5611990700 1 --2- 172.21.15.142:0/831471566 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f55f8038820 0x7f55f803ace0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:20:13.242 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:13.241+0000 7f5611990700 1 --2- 172.21.15.142:0/831471566 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f560c1965a0 0x7f560c19aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:20:13.242 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:13.241+0000 7f5611990700 1 -- 172.21.15.142:0/831471566 >> 172.21.15.142:0/831471566 conn(0x7f560c074a90 msgr2=0x7f560c0fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:20:13.242 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:13.241+0000 7f5611990700 1 -- 172.21.15.142:0/831471566 shutdown_connections 2024-02-02T15:20:13.242 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:13.241+0000 7f5611990700 1 -- 172.21.15.142:0/831471566 wait complete. 2024-02-02T15:20:13.243 INFO:teuthology.orchestra.run.smithi142.stderr:dumped monmap epoch 3 2024-02-02T15:20:13.624 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-02-02T15:20:13.624 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph config generate-minimal-conf 2024-02-02T15:20:13.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:13 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-02T15:20:13.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:13 smithi071 ceph-mon[44707]: mon.c calling monitor election 2024-02-02T15:20:13.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:13 smithi071 ceph-mon[44707]: mon.a calling monitor election 2024-02-02T15:20:13.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:13 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-02T15:20:13.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:13 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-02T15:20:13.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:13 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-02T15:20:13.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:13 smithi071 ceph-mon[44707]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-02T15:20:13.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:13 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-02T15:20:13.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:13 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-02T15:20:13.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:13 smithi071 ceph-mon[44707]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-02T15:20:13.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:13 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-02T15:20:13.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:13 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-02T15:20:13.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:13 smithi071 ceph-mon[44707]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-02T15:20:13.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:13 smithi071 ceph-mon[44707]: monmap e3: 3 mons at {a=[v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0],b=[v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0],c=[v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-02T15:20:13.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:13 smithi071 ceph-mon[44707]: fsmap 2024-02-02T15:20:13.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:13 smithi071 ceph-mon[44707]: osdmap e4: 0 total, 0 up, 0 in 2024-02-02T15:20:13.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:13 smithi071 ceph-mon[44707]: mgrmap e13: x(active, since 45s) 2024-02-02T15:20:13.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:13 smithi071 ceph-mon[44707]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-02-02T15:20:13.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:13 smithi071 ceph-mon[44707]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-02-02T15:20:13.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:13 smithi071 ceph-mon[44707]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-02-02T15:20:13.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:13 smithi071 ceph-mon[44707]: mon.b (rank 2) addr [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] is down (out of quorum) 2024-02-02T15:20:13.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:13 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:13.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:13 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:13.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:13 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:13.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:13 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:13.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:13 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:20:13.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:13 smithi071 ceph-mon[44707]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-02T15:20:13.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:13 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-02T15:20:13.867 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:13 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:13.867 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:13 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:13.867 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:13 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:13.867 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:13 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:13.867 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:13 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:20:13.867 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:13 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:20:14.000 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.a/config 2024-02-02T15:20:14.246 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:13 smithi001 ceph-mon[35470]: mon.b calling monitor election 2024-02-02T15:20:14.246 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:13 smithi001 ceph-mon[35470]: mon.b calling monitor election 2024-02-02T15:20:14.246 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:13 smithi001 ceph-mon[35470]: mon.c calling monitor election 2024-02-02T15:20:14.246 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:13 smithi001 ceph-mon[35470]: mon.a calling monitor election 2024-02-02T15:20:14.246 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:13 smithi001 ceph-mon[35470]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-02-02T15:20:14.247 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:13 smithi001 ceph-mon[35470]: monmap e3: 3 mons at {a=[v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0],b=[v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0],c=[v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-02T15:20:14.247 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:13 smithi001 ceph-mon[35470]: fsmap 2024-02-02T15:20:14.247 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:13 smithi001 ceph-mon[35470]: osdmap e4: 0 total, 0 up, 0 in 2024-02-02T15:20:14.247 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:13 smithi001 ceph-mon[35470]: mgrmap e13: x(active, since 47s) 2024-02-02T15:20:14.247 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:13 smithi001 ceph-mon[35470]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-02-02T15:20:14.247 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:13 smithi001 ceph-mon[35470]: Cluster is now healthy 2024-02-02T15:20:14.247 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:13 smithi001 ceph-mon[35470]: overall HEALTH_OK 2024-02-02T15:20:14.247 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:13 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:14.247 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:13 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:14.247 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:13 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:14.248 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:13 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:14.248 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:13 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:14.248 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:13 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:14.248 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:13 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:14.248 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:13 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-02T15:20:14.248 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:13 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-02T15:20:14.248 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:13 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:20:14.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:13 smithi071 ceph-mon[44707]: mon.b calling monitor election 2024-02-02T15:20:14.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:13 smithi071 ceph-mon[44707]: mon.b calling monitor election 2024-02-02T15:20:14.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:13 smithi071 ceph-mon[44707]: mon.c calling monitor election 2024-02-02T15:20:14.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:13 smithi071 ceph-mon[44707]: mon.a calling monitor election 2024-02-02T15:20:14.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:13 smithi071 ceph-mon[44707]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-02-02T15:20:14.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:13 smithi071 ceph-mon[44707]: monmap e3: 3 mons at {a=[v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0],b=[v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0],c=[v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-02T15:20:14.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:13 smithi071 ceph-mon[44707]: fsmap 2024-02-02T15:20:14.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:13 smithi071 ceph-mon[44707]: osdmap e4: 0 total, 0 up, 0 in 2024-02-02T15:20:14.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:13 smithi071 ceph-mon[44707]: mgrmap e13: x(active, since 47s) 2024-02-02T15:20:14.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:13 smithi071 ceph-mon[44707]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-02-02T15:20:14.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:13 smithi071 ceph-mon[44707]: Cluster is now healthy 2024-02-02T15:20:14.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:13 smithi071 ceph-mon[44707]: overall HEALTH_OK 2024-02-02T15:20:14.366 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:13 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:14.366 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:13 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:14.366 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:13 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:14.366 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:13 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:14.366 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:13 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:14.366 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:13 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:14.366 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:13 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:14.366 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:13 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-02T15:20:14.366 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:13 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-02T15:20:14.366 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:13 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:20:14.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:13 smithi142 ceph-mon[45728]: mon.b calling monitor election 2024-02-02T15:20:14.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:13 smithi142 ceph-mon[45728]: mon.b calling monitor election 2024-02-02T15:20:14.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:13 smithi142 ceph-mon[45728]: mon.c calling monitor election 2024-02-02T15:20:14.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:13 smithi142 ceph-mon[45728]: mon.a calling monitor election 2024-02-02T15:20:14.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:13 smithi142 ceph-mon[45728]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-02-02T15:20:14.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:13 smithi142 ceph-mon[45728]: monmap e3: 3 mons at {a=[v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0],b=[v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0],c=[v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-02T15:20:14.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:13 smithi142 ceph-mon[45728]: fsmap 2024-02-02T15:20:14.421 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:13 smithi142 ceph-mon[45728]: osdmap e4: 0 total, 0 up, 0 in 2024-02-02T15:20:14.421 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:13 smithi142 ceph-mon[45728]: mgrmap e13: x(active, since 47s) 2024-02-02T15:20:14.421 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:13 smithi142 ceph-mon[45728]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-02-02T15:20:14.421 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:13 smithi142 ceph-mon[45728]: Cluster is now healthy 2024-02-02T15:20:14.421 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:13 smithi142 ceph-mon[45728]: overall HEALTH_OK 2024-02-02T15:20:14.421 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:13 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:14.421 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:13 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:14.421 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:13 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:14.421 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:13 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:14.421 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:13 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:14.421 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:13 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:14.421 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:13 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:14.421 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:13 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-02T15:20:14.421 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:13 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-02T15:20:14.421 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:13 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:20:15.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:14 smithi071 ceph-mon[44707]: Reconfiguring mon.a (unknown last config time)... 2024-02-02T15:20:15.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:14 smithi071 ceph-mon[44707]: Reconfiguring daemon mon.a on smithi001 2024-02-02T15:20:15.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:14 smithi071 ceph-mon[44707]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-02T15:20:15.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:14 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-02T15:20:15.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:15.383+0000 7fb9bb000700 1 -- 172.21.15.1:0/4027605541 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9b4106ed0 msgr2=0x7fb9b41072b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:20:15.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:15.383+0000 7fb9bb000700 1 --2- 172.21.15.1:0/4027605541 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9b4106ed0 0x7fb9b41072b0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb9a40098f0 tx=0x7fb9a4009c00 comp rx=0 tx=0).stop 2024-02-02T15:20:15.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:15.383+0000 7fb9bb000700 1 -- 172.21.15.1:0/4027605541 shutdown_connections 2024-02-02T15:20:15.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:15.383+0000 7fb9bb000700 1 --2- 172.21.15.1:0/4027605541 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9b4069220 0x7fb9b410db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:20:15.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:15.383+0000 7fb9bb000700 1 --2- 172.21.15.1:0/4027605541 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9b4106ed0 0x7fb9b41072b0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:20:15.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:15.383+0000 7fb9bb000700 1 --2- 172.21.15.1:0/4027605541 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9b4068860 0x7fb9b4068ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:20:15.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:15.383+0000 7fb9bb000700 1 -- 172.21.15.1:0/4027605541 >> 172.21.15.1:0/4027605541 conn(0x7fb9b4076d40 msgr2=0x7fb9b4077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:20:15.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:15.383+0000 7fb9bb000700 1 -- 172.21.15.1:0/4027605541 shutdown_connections 2024-02-02T15:20:15.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:15.384+0000 7fb9bb000700 1 -- 172.21.15.1:0/4027605541 wait complete. 2024-02-02T15:20:15.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:15.384+0000 7fb9bb000700 1 Processor -- start 2024-02-02T15:20:15.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:15.385+0000 7fb9bb000700 1 -- start start 2024-02-02T15:20:15.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:15.385+0000 7fb9bb000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9b4068860 0x7fb9b40ffcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:20:15.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:15.385+0000 7fb9bb000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9b4069220 0x7fb9b4100230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:20:15.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:15.385+0000 7fb9bb000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9b4106ed0 0x7fb9b41007b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:20:15.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:15.385+0000 7fb9bb000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b41a52a0 con 0x7fb9b4106ed0 2024-02-02T15:20:15.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:15.385+0000 7fb9bb000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b41a5410 con 0x7fb9b4069220 2024-02-02T15:20:15.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:15.385+0000 7fb9bb000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b41a55d0 con 0x7fb9b4068860 2024-02-02T15:20:15.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:15.385+0000 7fb9b9000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9b4106ed0 0x7fb9b41007b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:20:15.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:15.385+0000 7fb9b9000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9b4106ed0 0x7fb9b41007b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45420/0 (socket says 172.21.15.1:45420) 2024-02-02T15:20:15.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:15.385+0000 7fb9b9000700 1 -- 172.21.15.1:0/1939889844 learned_addr learned my addr 172.21.15.1:0/1939889844 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:20:15.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:15.385+0000 7fb9b3400700 1 --2- 172.21.15.1:0/1939889844 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9b4069220 0x7fb9b4100230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:20:15.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:15.385+0000 7fb9b3e00700 1 --2- 172.21.15.1:0/1939889844 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9b4068860 0x7fb9b40ffcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:20:15.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:15.386+0000 7fb9b9000700 1 -- 172.21.15.1:0/1939889844 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9b4068860 msgr2=0x7fb9b40ffcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:20:15.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:15.386+0000 7fb9b9000700 1 --2- 172.21.15.1:0/1939889844 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9b4068860 0x7fb9b40ffcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:20:15.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:15.386+0000 7fb9b9000700 1 -- 172.21.15.1:0/1939889844 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9b4069220 msgr2=0x7fb9b4100230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:20:15.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:15.386+0000 7fb9b9000700 1 --2- 172.21.15.1:0/1939889844 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9b4069220 0x7fb9b4100230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:20:15.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:15.386+0000 7fb9b9000700 1 -- 172.21.15.1:0/1939889844 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9a4009520 con 0x7fb9b4106ed0 2024-02-02T15:20:15.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:15.386+0000 7fb9b3400700 1 --2- 172.21.15.1:0/1939889844 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9b4069220 0x7fb9b4100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:20:15.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:15.386+0000 7fb9b9000700 1 --2- 172.21.15.1:0/1939889844 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9b4106ed0 0x7fb9b41007b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac007f10 tx=0x7fb9ac00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:20:15.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:15.386+0000 7fb9b0c00700 1 -- 172.21.15.1:0/1939889844 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ac009ea0 con 0x7fb9b4106ed0 2024-02-02T15:20:15.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:15.386+0000 7fb9b0c00700 1 -- 172.21.15.1:0/1939889844 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb9ac00f040 con 0x7fb9b4106ed0 2024-02-02T15:20:15.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:15.386+0000 7fb9b0c00700 1 -- 172.21.15.1:0/1939889844 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ac004760 con 0x7fb9b4106ed0 2024-02-02T15:20:15.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:15.386+0000 7fb9bb000700 1 -- 172.21.15.1:0/1939889844 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9b41a58b0 con 0x7fb9b4106ed0 2024-02-02T15:20:15.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:15.387+0000 7fb9bb000700 1 -- 172.21.15.1:0/1939889844 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9b41a5d60 con 0x7fb9b4106ed0 2024-02-02T15:20:15.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:15.387+0000 7fb9bb000700 1 -- 172.21.15.1:0/1939889844 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9b404ea40 con 0x7fb9b4106ed0 2024-02-02T15:20:15.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:15.388+0000 7fb9b0c00700 1 -- 172.21.15.1:0/1939889844 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 13) v1 ==== 45323+0+0 (secure 0 0 0) 0x7fb9ac003a00 con 0x7fb9b4106ed0 2024-02-02T15:20:15.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:15.388+0000 7fb9b0c00700 1 --2- 172.21.15.1:0/1939889844 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fb9940385d0 0x7fb99403aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:20:15.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:15.388+0000 7fb9b0c00700 1 -- 172.21.15.1:0/1939889844 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb9ac04a690 con 0x7fb9b4106ed0 2024-02-02T15:20:15.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:15.388+0000 7fb9b3e00700 1 --2- 172.21.15.1:0/1939889844 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fb9940385d0 0x7fb99403aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:20:15.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:15.389+0000 7fb9b3e00700 1 --2- 172.21.15.1:0/1939889844 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fb9940385d0 0x7fb99403aa90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb9a4005e90 tx=0x7fb9a4019040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:20:15.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:15.393+0000 7fb9b0c00700 1 -- 172.21.15.1:0/1939889844 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7fb9ac009360 con 0x7fb9b4106ed0 2024-02-02T15:20:15.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:14 smithi142 ceph-mon[45728]: Reconfiguring mon.a (unknown last config time)... 2024-02-02T15:20:15.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:14 smithi142 ceph-mon[45728]: Reconfiguring daemon mon.a on smithi001 2024-02-02T15:20:15.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:14 smithi142 ceph-mon[45728]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-02T15:20:15.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:14 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-02T15:20:15.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:14 smithi001 ceph-mon[35470]: Reconfiguring mon.a (unknown last config time)... 2024-02-02T15:20:15.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:14 smithi001 ceph-mon[35470]: Reconfiguring daemon mon.a on smithi001 2024-02-02T15:20:15.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:14 smithi001 ceph-mon[35470]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-02T15:20:15.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:14 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-02T15:20:15.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:15.602+0000 7fb9bb000700 1 -- 172.21.15.1:0/1939889844 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fb9b41a6110 con 0x7fb9b4106ed0 2024-02-02T15:20:15.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:15.602+0000 7fb9b0c00700 1 -- 172.21.15.1:0/1939889844 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+271 (secure 0 0 0) 0x7fb9ac012350 con 0x7fb9b4106ed0 2024-02-02T15:20:15.601 INFO:teuthology.orchestra.run.smithi001.stdout:# minimal ceph.conf for 371a409c-c1de-11ee-95b6-87774f69a715 2024-02-02T15:20:15.602 INFO:teuthology.orchestra.run.smithi001.stdout:[global] 2024-02-02T15:20:15.602 INFO:teuthology.orchestra.run.smithi001.stdout: fsid = 371a409c-c1de-11ee-95b6-87774f69a715 2024-02-02T15:20:15.602 INFO:teuthology.orchestra.run.smithi001.stdout: mon_host = [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] 2024-02-02T15:20:15.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:15.605+0000 7fb9bb000700 1 -- 172.21.15.1:0/1939889844 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fb9940385d0 msgr2=0x7fb99403aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:20:15.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:15.605+0000 7fb9bb000700 1 --2- 172.21.15.1:0/1939889844 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fb9940385d0 0x7fb99403aa90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fb9a4005e90 tx=0x7fb9a4019040 comp rx=0 tx=0).stop 2024-02-02T15:20:15.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:15.605+0000 7fb9bb000700 1 -- 172.21.15.1:0/1939889844 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9b4106ed0 msgr2=0x7fb9b41007b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:20:15.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:15.605+0000 7fb9bb000700 1 --2- 172.21.15.1:0/1939889844 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9b4106ed0 0x7fb9b41007b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac007f10 tx=0x7fb9ac00d3b0 comp rx=0 tx=0).stop 2024-02-02T15:20:15.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:15.605+0000 7fb9bb000700 1 -- 172.21.15.1:0/1939889844 shutdown_connections 2024-02-02T15:20:15.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:15.605+0000 7fb9bb000700 1 --2- 172.21.15.1:0/1939889844 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9b4106ed0 0x7fb9b41007b0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:20:15.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:15.605+0000 7fb9bb000700 1 --2- 172.21.15.1:0/1939889844 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fb9940385d0 0x7fb99403aa90 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:20:15.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:15.605+0000 7fb9bb000700 1 --2- 172.21.15.1:0/1939889844 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9b4068860 0x7fb9b40ffcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:20:15.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:15.605+0000 7fb9bb000700 1 --2- 172.21.15.1:0/1939889844 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9b4069220 0x7fb9b4100230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:20:15.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:15.605+0000 7fb9bb000700 1 -- 172.21.15.1:0/1939889844 >> 172.21.15.1:0/1939889844 conn(0x7fb9b4076d40 msgr2=0x7fb9b40fe800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:20:15.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:15.605+0000 7fb9bb000700 1 -- 172.21.15.1:0/1939889844 shutdown_connections 2024-02-02T15:20:15.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:15.605+0000 7fb9bb000700 1 -- 172.21.15.1:0/1939889844 wait complete. 2024-02-02T15:20:15.945 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-02-02T15:20:15.946 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-02-02T15:20:15.946 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/etc/ceph/ceph.conf 2024-02-02T15:20:16.016 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-02-02T15:20:16.016 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-02T15:20:16.123 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-02-02T15:20:16.123 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/etc/ceph/ceph.conf 2024-02-02T15:20:16.154 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-02-02T15:20:16.154 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-02T15:20:16.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:16 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:16.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:16 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:16.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:16 smithi001 ceph-mon[35470]: Reconfiguring mon.b (monmap changed)... 2024-02-02T15:20:16.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:16 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-02T15:20:16.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:16 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-02T15:20:16.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:16 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:20:16.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:16 smithi001 ceph-mon[35470]: Reconfiguring daemon mon.b on smithi071 2024-02-02T15:20:16.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:16 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/1939889844' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:20:16.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:16 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:16.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:16 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:16.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:16 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:16.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:16 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-02T15:20:16.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:16 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-02T15:20:16.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:16 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:20:16.220 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-02-02T15:20:16.220 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/ceph/ceph.conf 2024-02-02T15:20:16.251 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-02-02T15:20:16.251 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-02T15:20:16.315 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:16 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:16.315 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:16 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:16.315 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:16 smithi142 ceph-mon[45728]: Reconfiguring mon.b (monmap changed)... 2024-02-02T15:20:16.315 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:16 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-02T15:20:16.316 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:16 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-02T15:20:16.316 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:16 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:20:16.316 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:16 smithi142 ceph-mon[45728]: Reconfiguring daemon mon.b on smithi071 2024-02-02T15:20:16.316 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:16 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/1939889844' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:20:16.316 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:16 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:16.316 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:16 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:16.316 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:16 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:16.316 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:16 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-02T15:20:16.316 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:16 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-02T15:20:16.316 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:16 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:20:16.318 INFO:tasks.cephadm:Adding mgr.x on smithi001 2024-02-02T15:20:16.318 INFO:tasks.cephadm:Adding mgr.y on smithi071 2024-02-02T15:20:16.319 INFO:tasks.cephadm:Adding mgr.z on smithi142 2024-02-02T15:20:16.319 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph orch apply mgr '3;smithi001=x;smithi071=y;smithi142=z' 2024-02-02T15:20:16.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:16 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:16.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:16 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:16.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:16 smithi071 ceph-mon[44707]: Reconfiguring mon.b (monmap changed)... 2024-02-02T15:20:16.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:16 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-02T15:20:16.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:16 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-02T15:20:16.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:16 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:20:16.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:16 smithi071 ceph-mon[44707]: Reconfiguring daemon mon.b on smithi071 2024-02-02T15:20:16.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:16 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/1939889844' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:20:16.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:16 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:16.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:16 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:16.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:16 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:16.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:16 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-02T15:20:16.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:16 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-02T15:20:16.368 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:16 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:20:16.875 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.c/config 2024-02-02T15:20:17.302 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:17 smithi142 ceph-mon[45728]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-02T15:20:17.302 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:17 smithi142 ceph-mon[45728]: Reconfiguring mon.c (monmap changed)... 2024-02-02T15:20:17.302 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:17 smithi142 ceph-mon[45728]: Reconfiguring daemon mon.c on smithi142 2024-02-02T15:20:17.302 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:17 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:17.302 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:17 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:17.302 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:17 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:20:17.302 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:17 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:20:17.302 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:17 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:20:17.303 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:17 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:17.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:17 smithi071 ceph-mon[44707]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-02T15:20:17.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:17 smithi071 ceph-mon[44707]: Reconfiguring mon.c (monmap changed)... 2024-02-02T15:20:17.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:17 smithi071 ceph-mon[44707]: Reconfiguring daemon mon.c on smithi142 2024-02-02T15:20:17.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:17 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:17.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:17 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:17.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:17 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:20:17.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:17 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:20:17.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:17 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:20:17.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:17 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:17.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:17 smithi001 ceph-mon[35470]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-02T15:20:17.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:17 smithi001 ceph-mon[35470]: Reconfiguring mon.c (monmap changed)... 2024-02-02T15:20:17.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:17 smithi001 ceph-mon[35470]: Reconfiguring daemon mon.c on smithi142 2024-02-02T15:20:17.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:17 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:17.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:17 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:17.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:17 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:20:17.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:17 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:20:17.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:17 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:20:17.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:17 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:17.833 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:17.830+0000 7f4a670b2700 1 -- 172.21.15.142:0/1507048059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a60101870 msgr2=0x7f4a60108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:20:17.833 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:17.830+0000 7f4a670b2700 1 --2- 172.21.15.142:0/1507048059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a60101870 0x7f4a60108120 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4a5800b0a0 tx=0x7f4a5800b3b0 comp rx=0 tx=0).stop 2024-02-02T15:20:17.833 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:17.830+0000 7f4a670b2700 1 -- 172.21.15.142:0/1507048059 shutdown_connections 2024-02-02T15:20:17.833 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:17.830+0000 7f4a670b2700 1 --2- 172.21.15.142:0/1507048059 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a60101870 0x7f4a60108120 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:20:17.833 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:17.830+0000 7f4a670b2700 1 --2- 172.21.15.142:0/1507048059 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a600ffcd0 0x7f4a601000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:20:17.833 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:17.830+0000 7f4a670b2700 1 --2- 172.21.15.142:0/1507048059 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4a60100ed0 0x7f4a60101330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:20:17.833 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:17.830+0000 7f4a670b2700 1 -- 172.21.15.142:0/1507048059 >> 172.21.15.142:0/1507048059 conn(0x7f4a600fb250 msgr2=0x7f4a600fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:20:17.833 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:17.830+0000 7f4a670b2700 1 -- 172.21.15.142:0/1507048059 shutdown_connections 2024-02-02T15:20:17.833 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:17.830+0000 7f4a670b2700 1 -- 172.21.15.142:0/1507048059 wait complete. 2024-02-02T15:20:17.833 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:17.830+0000 7f4a670b2700 1 Processor -- start 2024-02-02T15:20:17.833 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:17.830+0000 7f4a670b2700 1 -- start start 2024-02-02T15:20:17.833 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:17.830+0000 7f4a670b2700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a600ffcd0 0x7f4a60196be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:20:17.834 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:17.830+0000 7f4a670b2700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a60100ed0 0x7f4a60197120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:20:17.834 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:17.830+0000 7f4a670b2700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4a60101870 0x7f4a6019e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:20:17.834 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:17.830+0000 7f4a670b2700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a6019e770 con 0x7f4a60101870 2024-02-02T15:20:17.834 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:17.830+0000 7f4a670b2700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a6019e8b0 con 0x7f4a60100ed0 2024-02-02T15:20:17.834 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:17.830+0000 7f4a670b2700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a6019ea40 con 0x7f4a600ffcd0 2024-02-02T15:20:17.834 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:17.831+0000 7f4a64e4e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a600ffcd0 0x7f4a60196be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:20:17.834 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:17.831+0000 7f4a64e4e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a600ffcd0 0x7f4a60196be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:50444/0 (socket says 172.21.15.142:50444) 2024-02-02T15:20:17.834 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:17.831+0000 7f4a64e4e700 1 -- 172.21.15.142:0/4117324193 learned_addr learned my addr 172.21.15.142:0/4117324193 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-02-02T15:20:17.834 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:17.831+0000 7f4a6564f700 1 --2- 172.21.15.142:0/4117324193 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4a60101870 0x7f4a6019e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:20:17.834 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:17.831+0000 7f4a5ffff700 1 --2- 172.21.15.142:0/4117324193 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a60100ed0 0x7f4a60197120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:20:17.834 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:17.831+0000 7f4a64e4e700 1 -- 172.21.15.142:0/4117324193 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a60100ed0 msgr2=0x7f4a60197120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:20:17.834 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:17.831+0000 7f4a64e4e700 1 --2- 172.21.15.142:0/4117324193 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a60100ed0 0x7f4a60197120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:20:17.835 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:17.831+0000 7f4a64e4e700 1 -- 172.21.15.142:0/4117324193 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4a60101870 msgr2=0x7f4a6019e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:20:17.835 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:17.831+0000 7f4a64e4e700 1 --2- 172.21.15.142:0/4117324193 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4a60101870 0x7f4a6019e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:20:17.835 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:17.831+0000 7f4a64e4e700 1 -- 172.21.15.142:0/4117324193 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a58009d00 con 0x7f4a600ffcd0 2024-02-02T15:20:17.835 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:17.831+0000 7f4a6564f700 1 --2- 172.21.15.142:0/4117324193 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4a60101870 0x7f4a6019e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:20:17.835 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:17.831+0000 7f4a64e4e700 1 --2- 172.21.15.142:0/4117324193 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a600ffcd0 0x7f4a60196be0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4a5400b530 tx=0x7f4a5400b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:20:17.835 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:17.831+0000 7f4a5dffb700 1 -- 172.21.15.142:0/4117324193 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a54011830 con 0x7f4a600ffcd0 2024-02-02T15:20:17.835 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:17.831+0000 7f4a5dffb700 1 -- 172.21.15.142:0/4117324193 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a54011e50 con 0x7f4a600ffcd0 2024-02-02T15:20:17.835 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:17.831+0000 7f4a5dffb700 1 -- 172.21.15.142:0/4117324193 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a5400f2e0 con 0x7f4a600ffcd0 2024-02-02T15:20:17.835 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:17.831+0000 7f4a670b2700 1 -- 172.21.15.142:0/4117324193 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a6019ed20 con 0x7f4a600ffcd0 2024-02-02T15:20:17.835 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:17.831+0000 7f4a670b2700 1 -- 172.21.15.142:0/4117324193 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a6019f170 con 0x7f4a600ffcd0 2024-02-02T15:20:17.835 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:17.832+0000 7f4a5dffb700 1 -- 172.21.15.142:0/4117324193 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 13) v1 ==== 45323+0+0 (secure 0 0 0) 0x7f4a5400f440 con 0x7f4a600ffcd0 2024-02-02T15:20:17.835 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:17.832+0000 7f4a5dffb700 1 --2- 172.21.15.142:0/4117324193 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f4a500385d0 0x7f4a5003aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:20:17.835 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:17.832+0000 7f4a5dffb700 1 -- 172.21.15.142:0/4117324193 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f4a5404a890 con 0x7f4a600ffcd0 2024-02-02T15:20:17.835 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:17.832+0000 7f4a670b2700 1 -- 172.21.15.142:0/4117324193 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a6010b700 con 0x7f4a600ffcd0 2024-02-02T15:20:17.835 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:17.833+0000 7f4a5ffff700 1 --2- 172.21.15.142:0/4117324193 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f4a500385d0 0x7f4a5003aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:20:17.836 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:17.833+0000 7f4a5ffff700 1 --2- 172.21.15.142:0/4117324193 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f4a500385d0 0x7f4a5003aa90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c005fd0 tx=0x7f4a4c00a560 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:20:17.838 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:17.837+0000 7f4a5dffb700 1 -- 172.21.15.142:0/4117324193 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f4a540178b0 con 0x7f4a600ffcd0 2024-02-02T15:20:18.036 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:18.036+0000 7f4a670b2700 1 -- 172.21.15.142:0/4117324193 --> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi001=x;smithi071=y;smithi142=z", "target": ["mon-mgr", ""]}) v1 -- 0x7f4a600fd130 con 0x7f4a500385d0 2024-02-02T15:20:18.044 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:18.044+0000 7f4a5dffb700 1 -- 172.21.15.142:0/4117324193 <== mgr.14154 v2:172.21.15.1:6800/3768989981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f4a600fd130 con 0x7f4a500385d0 2024-02-02T15:20:18.044 INFO:teuthology.orchestra.run.smithi142.stdout:Scheduled mgr update... 2024-02-02T15:20:18.046 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:18.046+0000 7f4a670b2700 1 -- 172.21.15.142:0/4117324193 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f4a500385d0 msgr2=0x7f4a5003aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:20:18.046 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:18.046+0000 7f4a670b2700 1 --2- 172.21.15.142:0/4117324193 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f4a500385d0 0x7f4a5003aa90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c005fd0 tx=0x7f4a4c00a560 comp rx=0 tx=0).stop 2024-02-02T15:20:18.046 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:18.046+0000 7f4a670b2700 1 -- 172.21.15.142:0/4117324193 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a600ffcd0 msgr2=0x7f4a60196be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:20:18.046 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:18.046+0000 7f4a670b2700 1 --2- 172.21.15.142:0/4117324193 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a600ffcd0 0x7f4a60196be0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4a5400b530 tx=0x7f4a5400b8f0 comp rx=0 tx=0).stop 2024-02-02T15:20:18.047 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:18.046+0000 7f4a670b2700 1 -- 172.21.15.142:0/4117324193 shutdown_connections 2024-02-02T15:20:18.047 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:18.046+0000 7f4a670b2700 1 --2- 172.21.15.142:0/4117324193 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4a60101870 0x7f4a6019e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:20:18.047 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:18.046+0000 7f4a670b2700 1 --2- 172.21.15.142:0/4117324193 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f4a500385d0 0x7f4a5003aa90 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:20:18.047 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:18.046+0000 7f4a670b2700 1 --2- 172.21.15.142:0/4117324193 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a600ffcd0 0x7f4a60196be0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:20:18.047 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:18.046+0000 7f4a670b2700 1 --2- 172.21.15.142:0/4117324193 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a60100ed0 0x7f4a60197120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:20:18.047 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:18.046+0000 7f4a670b2700 1 -- 172.21.15.142:0/4117324193 >> 172.21.15.142:0/4117324193 conn(0x7f4a600fb250 msgr2=0x7f4a600fca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:20:18.047 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:18.046+0000 7f4a670b2700 1 -- 172.21.15.142:0/4117324193 shutdown_connections 2024-02-02T15:20:18.047 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:20:18.046+0000 7f4a670b2700 1 -- 172.21.15.142:0/4117324193 wait complete. 2024-02-02T15:20:18.437 DEBUG:teuthology.orchestra.run.smithi071:mgr.y> sudo journalctl -f -n 0 -u ceph-371a409c-c1de-11ee-95b6-87774f69a715@mgr.y.service 2024-02-02T15:20:18.439 DEBUG:teuthology.orchestra.run.smithi142:mgr.z> sudo journalctl -f -n 0 -u ceph-371a409c-c1de-11ee-95b6-87774f69a715@mgr.z.service 2024-02-02T15:20:18.442 INFO:tasks.cephadm:Deploying OSDs... 2024-02-02T15:20:18.442 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-02-02T15:20:18.442 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/scratch_devs of=/dev/stdout 2024-02-02T15:20:18.472 INFO:journalctl@ceph.mgr.y.smithi071.stdout:-- Logs begin at Fri 2024-02-02 14:58:45 UTC. -- 2024-02-02T15:20:18.472 INFO:journalctl@ceph.mgr.z.smithi142.stdout:-- Logs begin at Fri 2024-02-02 14:58:43 UTC. -- 2024-02-02T15:20:18.487 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-02T15:20:18.488 DEBUG:teuthology.orchestra.run.smithi001:> stat /dev/vg_nvme/lv_1 2024-02-02T15:20:18.572 INFO:teuthology.orchestra.run.smithi001.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-02T15:20:18.572 INFO:teuthology.orchestra.run.smithi001.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-02T15:20:18.572 INFO:teuthology.orchestra.run.smithi001.stdout:Device: 5h/5d Inode: 452 Links: 1 2024-02-02T15:20:18.573 INFO:teuthology.orchestra.run.smithi001.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-02T15:20:18.573 INFO:teuthology.orchestra.run.smithi001.stdout:Context: system_u:object_r:device_t:s0 2024-02-02T15:20:18.573 INFO:teuthology.orchestra.run.smithi001.stdout:Access: 2024-02-02 15:19:36.265640477 +0000 2024-02-02T15:20:18.573 INFO:teuthology.orchestra.run.smithi001.stdout:Modify: 2024-02-02 15:13:40.828510102 +0000 2024-02-02T15:20:18.573 INFO:teuthology.orchestra.run.smithi001.stdout:Change: 2024-02-02 15:13:40.828510102 +0000 2024-02-02T15:20:18.573 INFO:teuthology.orchestra.run.smithi001.stdout: Birth: 2024-02-02 15:13:40.808510354 +0000 2024-02-02T15:20:18.573 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-02T15:20:18.678 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records in 2024-02-02T15:20:18.678 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records out 2024-02-02T15:20:18.679 INFO:teuthology.orchestra.run.smithi001.stderr:512 bytes copied, 0.000181856 s, 2.8 MB/s 2024-02-02T15:20:18.681 DEBUG:teuthology.orchestra.run.smithi001:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-02T15:20:18.725 DEBUG:teuthology.orchestra.run.smithi001:> stat /dev/vg_nvme/lv_2 2024-02-02T15:20:18.808 INFO:teuthology.orchestra.run.smithi001.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-02T15:20:18.808 INFO:teuthology.orchestra.run.smithi001.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-02T15:20:18.808 INFO:teuthology.orchestra.run.smithi001.stdout:Device: 5h/5d Inode: 458 Links: 1 2024-02-02T15:20:18.808 INFO:teuthology.orchestra.run.smithi001.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-02T15:20:18.808 INFO:teuthology.orchestra.run.smithi001.stdout:Context: system_u:object_r:device_t:s0 2024-02-02T15:20:18.808 INFO:teuthology.orchestra.run.smithi001.stdout:Access: 2024-02-02 15:19:36.330639640 +0000 2024-02-02T15:20:18.808 INFO:teuthology.orchestra.run.smithi001.stdout:Modify: 2024-02-02 15:13:41.303504113 +0000 2024-02-02T15:20:18.808 INFO:teuthology.orchestra.run.smithi001.stdout:Change: 2024-02-02 15:13:41.303504113 +0000 2024-02-02T15:20:18.808 INFO:teuthology.orchestra.run.smithi001.stdout: Birth: 2024-02-02 15:13:41.281504390 +0000 2024-02-02T15:20:18.809 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-02T15:20:18.909 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records in 2024-02-02T15:20:18.910 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records out 2024-02-02T15:20:18.910 INFO:teuthology.orchestra.run.smithi001.stderr:512 bytes copied, 0.000177278 s, 2.9 MB/s 2024-02-02T15:20:18.912 DEBUG:teuthology.orchestra.run.smithi001:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-02T15:20:18.955 DEBUG:teuthology.orchestra.run.smithi001:> stat /dev/vg_nvme/lv_3 2024-02-02T15:20:19.038 INFO:teuthology.orchestra.run.smithi001.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-02T15:20:19.038 INFO:teuthology.orchestra.run.smithi001.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-02T15:20:19.038 INFO:teuthology.orchestra.run.smithi001.stdout:Device: 5h/5d Inode: 464 Links: 1 2024-02-02T15:20:19.038 INFO:teuthology.orchestra.run.smithi001.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-02T15:20:19.038 INFO:teuthology.orchestra.run.smithi001.stdout:Context: system_u:object_r:device_t:s0 2024-02-02T15:20:19.038 INFO:teuthology.orchestra.run.smithi001.stdout:Access: 2024-02-02 15:19:36.402638712 +0000 2024-02-02T15:20:19.038 INFO:teuthology.orchestra.run.smithi001.stdout:Modify: 2024-02-02 15:13:41.777498137 +0000 2024-02-02T15:20:19.038 INFO:teuthology.orchestra.run.smithi001.stdout:Change: 2024-02-02 15:13:41.777498137 +0000 2024-02-02T15:20:19.038 INFO:teuthology.orchestra.run.smithi001.stdout: Birth: 2024-02-02 15:13:41.754498427 +0000 2024-02-02T15:20:19.039 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-02T15:20:19.139 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records in 2024-02-02T15:20:19.140 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records out 2024-02-02T15:20:19.140 INFO:teuthology.orchestra.run.smithi001.stderr:512 bytes copied, 0.000175503 s, 2.9 MB/s 2024-02-02T15:20:19.142 DEBUG:teuthology.orchestra.run.smithi001:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-02T15:20:19.184 DEBUG:teuthology.orchestra.run.smithi001:> stat /dev/vg_nvme/lv_4 2024-02-02T15:20:19.266 INFO:teuthology.orchestra.run.smithi001.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-02T15:20:19.267 INFO:teuthology.orchestra.run.smithi001.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-02T15:20:19.267 INFO:teuthology.orchestra.run.smithi001.stdout:Device: 5h/5d Inode: 470 Links: 1 2024-02-02T15:20:19.267 INFO:teuthology.orchestra.run.smithi001.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-02T15:20:19.267 INFO:teuthology.orchestra.run.smithi001.stdout:Context: system_u:object_r:device_t:s0 2024-02-02T15:20:19.267 INFO:teuthology.orchestra.run.smithi001.stdout:Access: 2024-02-02 15:19:36.470637835 +0000 2024-02-02T15:20:19.267 INFO:teuthology.orchestra.run.smithi001.stdout:Modify: 2024-02-02 15:13:42.243492262 +0000 2024-02-02T15:20:19.267 INFO:teuthology.orchestra.run.smithi001.stdout:Change: 2024-02-02 15:13:42.243492262 +0000 2024-02-02T15:20:19.267 INFO:teuthology.orchestra.run.smithi001.stdout: Birth: 2024-02-02 15:13:42.224492501 +0000 2024-02-02T15:20:19.267 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-02T15:20:19.344 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:19 smithi142 ceph-mon[45728]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-02T15:20:19.344 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:19 smithi142 ceph-mon[45728]: from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi001=x;smithi071=y;smithi142=z", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:20:19.344 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:19 smithi142 ceph-mon[45728]: Saving service mgr spec with placement smithi001=x;smithi071=y;smithi142=z;count:3 2024-02-02T15:20:19.344 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:19 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:19.344 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:19 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:20:19.344 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:19 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:20:19.344 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:19 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:20:19.344 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:19 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:19.345 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:19 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-02T15:20:19.345 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:19 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-02T15:20:19.345 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:19 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-02T15:20:19.345 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:19 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:20:19.345 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:19 smithi142 ceph-mon[45728]: Deploying daemon mgr.z on smithi142 2024-02-02T15:20:19.362 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:19 smithi001 ceph-mon[35470]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-02T15:20:19.362 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:19 smithi001 ceph-mon[35470]: from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi001=x;smithi071=y;smithi142=z", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:20:19.363 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:19 smithi001 ceph-mon[35470]: Saving service mgr spec with placement smithi001=x;smithi071=y;smithi142=z;count:3 2024-02-02T15:20:19.363 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:19 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:19.363 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:19 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:20:19.363 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:19 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:20:19.363 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:19 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:20:19.363 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:19 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:19.363 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:19 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-02T15:20:19.363 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:19 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-02T15:20:19.363 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:19 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-02T15:20:19.363 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:19 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:20:19.363 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:19 smithi001 ceph-mon[35470]: Deploying daemon mgr.z on smithi142 2024-02-02T15:20:19.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:19 smithi071 ceph-mon[44707]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-02T15:20:19.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:19 smithi071 ceph-mon[44707]: from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi001=x;smithi071=y;smithi142=z", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:20:19.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:19 smithi071 ceph-mon[44707]: Saving service mgr spec with placement smithi001=x;smithi071=y;smithi142=z;count:3 2024-02-02T15:20:19.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:19 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:19.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:19 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:20:19.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:19 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:20:19.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:19 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:20:19.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:19 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:19.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:19 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-02T15:20:19.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:19 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-02T15:20:19.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:19 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-02T15:20:19.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:19 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:20:19.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:19 smithi071 ceph-mon[44707]: Deploying daemon mgr.z on smithi142 2024-02-02T15:20:19.368 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records in 2024-02-02T15:20:19.368 INFO:teuthology.orchestra.run.smithi001.stderr:1+0 records out 2024-02-02T15:20:19.368 INFO:teuthology.orchestra.run.smithi001.stderr:512 bytes copied, 0.000169717 s, 3.0 MB/s 2024-02-02T15:20:19.371 DEBUG:teuthology.orchestra.run.smithi001:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-02T15:20:19.414 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-02-02T15:20:19.414 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/scratch_devs of=/dev/stdout 2024-02-02T15:20:19.437 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-02T15:20:19.437 DEBUG:teuthology.orchestra.run.smithi071:> stat /dev/vg_nvme/lv_1 2024-02-02T15:20:19.498 INFO:teuthology.orchestra.run.smithi071.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-02T15:20:19.498 INFO:teuthology.orchestra.run.smithi071.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-02T15:20:19.498 INFO:teuthology.orchestra.run.smithi071.stdout:Device: 6h/6d Inode: 176507 Links: 1 2024-02-02T15:20:19.498 INFO:teuthology.orchestra.run.smithi071.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-02T15:20:19.498 INFO:teuthology.orchestra.run.smithi071.stdout:Context: system_u:object_r:device_t:s0 2024-02-02T15:20:19.498 INFO:teuthology.orchestra.run.smithi071.stdout:Access: 2024-02-02 15:20:05.991285361 +0000 2024-02-02T15:20:19.498 INFO:teuthology.orchestra.run.smithi071.stdout:Modify: 2024-02-02 15:12:22.181123334 +0000 2024-02-02T15:20:19.498 INFO:teuthology.orchestra.run.smithi071.stdout:Change: 2024-02-02 15:12:22.181123334 +0000 2024-02-02T15:20:19.498 INFO:teuthology.orchestra.run.smithi071.stdout: Birth: - 2024-02-02T15:20:19.499 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-02T15:20:19.568 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records in 2024-02-02T15:20:19.568 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records out 2024-02-02T15:20:19.568 INFO:teuthology.orchestra.run.smithi071.stderr:512 bytes copied, 0.000280432 s, 1.8 MB/s 2024-02-02T15:20:19.570 DEBUG:teuthology.orchestra.run.smithi071:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-02T15:20:19.630 DEBUG:teuthology.orchestra.run.smithi071:> stat /dev/vg_nvme/lv_2 2024-02-02T15:20:19.670 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Feb 02 15:20:19 smithi142 systemd[1]: Starting Ceph mgr.z for 371a409c-c1de-11ee-95b6-87774f69a715... 2024-02-02T15:20:19.689 INFO:teuthology.orchestra.run.smithi071.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-02T15:20:19.689 INFO:teuthology.orchestra.run.smithi071.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-02T15:20:19.689 INFO:teuthology.orchestra.run.smithi071.stdout:Device: 6h/6d Inode: 179303 Links: 1 2024-02-02T15:20:19.689 INFO:teuthology.orchestra.run.smithi071.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-02T15:20:19.689 INFO:teuthology.orchestra.run.smithi071.stdout:Context: system_u:object_r:device_t:s0 2024-02-02T15:20:19.689 INFO:teuthology.orchestra.run.smithi071.stdout:Access: 2024-02-02 15:20:05.991285361 +0000 2024-02-02T15:20:19.689 INFO:teuthology.orchestra.run.smithi071.stdout:Modify: 2024-02-02 15:12:22.532116881 +0000 2024-02-02T15:20:19.689 INFO:teuthology.orchestra.run.smithi071.stdout:Change: 2024-02-02 15:12:22.532116881 +0000 2024-02-02T15:20:19.689 INFO:teuthology.orchestra.run.smithi071.stdout: Birth: - 2024-02-02T15:20:19.690 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-02T15:20:19.756 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records in 2024-02-02T15:20:19.756 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records out 2024-02-02T15:20:19.756 INFO:teuthology.orchestra.run.smithi071.stderr:512 bytes copied, 0.00022199 s, 2.3 MB/s 2024-02-02T15:20:19.757 DEBUG:teuthology.orchestra.run.smithi071:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-02T15:20:19.816 DEBUG:teuthology.orchestra.run.smithi071:> stat /dev/vg_nvme/lv_3 2024-02-02T15:20:19.875 INFO:teuthology.orchestra.run.smithi071.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-02T15:20:19.875 INFO:teuthology.orchestra.run.smithi071.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-02T15:20:19.875 INFO:teuthology.orchestra.run.smithi071.stdout:Device: 6h/6d Inode: 179379 Links: 1 2024-02-02T15:20:19.875 INFO:teuthology.orchestra.run.smithi071.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-02T15:20:19.875 INFO:teuthology.orchestra.run.smithi071.stdout:Context: system_u:object_r:device_t:s0 2024-02-02T15:20:19.875 INFO:teuthology.orchestra.run.smithi071.stdout:Access: 2024-02-02 15:20:05.991285361 +0000 2024-02-02T15:20:19.875 INFO:teuthology.orchestra.run.smithi071.stdout:Modify: 2024-02-02 15:12:22.907109988 +0000 2024-02-02T15:20:19.875 INFO:teuthology.orchestra.run.smithi071.stdout:Change: 2024-02-02 15:12:22.907109988 +0000 2024-02-02T15:20:19.875 INFO:teuthology.orchestra.run.smithi071.stdout: Birth: - 2024-02-02T15:20:19.876 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-02T15:20:19.941 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records in 2024-02-02T15:20:19.941 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records out 2024-02-02T15:20:19.941 INFO:teuthology.orchestra.run.smithi071.stderr:512 bytes copied, 0.000224588 s, 2.3 MB/s 2024-02-02T15:20:19.942 DEBUG:teuthology.orchestra.run.smithi071:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-02T15:20:20.001 DEBUG:teuthology.orchestra.run.smithi071:> stat /dev/vg_nvme/lv_4 2024-02-02T15:20:20.058 INFO:teuthology.orchestra.run.smithi071.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-02T15:20:20.058 INFO:teuthology.orchestra.run.smithi071.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-02T15:20:20.058 INFO:teuthology.orchestra.run.smithi071.stdout:Device: 6h/6d Inode: 175727 Links: 1 2024-02-02T15:20:20.058 INFO:teuthology.orchestra.run.smithi071.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-02T15:20:20.058 INFO:teuthology.orchestra.run.smithi071.stdout:Context: system_u:object_r:device_t:s0 2024-02-02T15:20:20.058 INFO:teuthology.orchestra.run.smithi071.stdout:Access: 2024-02-02 15:20:05.991285361 +0000 2024-02-02T15:20:20.058 INFO:teuthology.orchestra.run.smithi071.stdout:Modify: 2024-02-02 15:12:23.248103720 +0000 2024-02-02T15:20:20.058 INFO:teuthology.orchestra.run.smithi071.stdout:Change: 2024-02-02 15:12:23.248103720 +0000 2024-02-02T15:20:20.058 INFO:teuthology.orchestra.run.smithi071.stdout: Birth: - 2024-02-02T15:20:20.059 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-02T15:20:20.124 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records in 2024-02-02T15:20:20.124 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records out 2024-02-02T15:20:20.125 INFO:teuthology.orchestra.run.smithi071.stderr:512 bytes copied, 0.000248475 s, 2.1 MB/s 2024-02-02T15:20:20.126 DEBUG:teuthology.orchestra.run.smithi071:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-02T15:20:20.169 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Feb 02 15:20:19 smithi142 ceph-mgr[47613]: -- 172.21.15.142:0/1750245029 <== mon.1 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ced7ae71e0 con 0x55ced7abcc00 2024-02-02T15:20:20.170 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Feb 02 15:20:19 smithi142 bash[47575]: 47f3ec4035f2f2b5bd256f449d61a1ef4a3575a2f6434b24bd1756d9277ec398 2024-02-02T15:20:20.170 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Feb 02 15:20:19 smithi142 systemd[1]: Started Ceph mgr.z for 371a409c-c1de-11ee-95b6-87774f69a715. 2024-02-02T15:20:20.183 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-02-02T15:20:20.183 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/scratch_devs of=/dev/stdout 2024-02-02T15:20:20.202 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-02T15:20:20.202 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/vg_nvme/lv_1 2024-02-02T15:20:20.260 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-02T15:20:20.260 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-02T15:20:20.260 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 6h/6d Inode: 179618 Links: 1 2024-02-02T15:20:20.260 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-02T15:20:20.260 INFO:teuthology.orchestra.run.smithi142.stdout:Context: system_u:object_r:device_t:s0 2024-02-02T15:20:20.260 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2024-02-02 15:20:19.391670853 +0000 2024-02-02T15:20:20.260 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2024-02-02 15:12:28.384162456 +0000 2024-02-02T15:20:20.260 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2024-02-02 15:12:28.384162456 +0000 2024-02-02T15:20:20.261 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: - 2024-02-02T15:20:20.261 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-02T15:20:20.327 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Feb 02 15:20:20 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-z[47590]: 2024-02-02T15:20:20.265+0000 7f24fb277200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-02T15:20:20.329 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2024-02-02T15:20:20.329 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2024-02-02T15:20:20.330 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.000206817 s, 2.5 MB/s 2024-02-02T15:20:20.331 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-02T15:20:20.390 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/vg_nvme/lv_2 2024-02-02T15:20:20.447 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-02T15:20:20.447 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-02T15:20:20.447 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 6h/6d Inode: 179690 Links: 1 2024-02-02T15:20:20.447 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-02T15:20:20.448 INFO:teuthology.orchestra.run.smithi142.stdout:Context: system_u:object_r:device_t:s0 2024-02-02T15:20:20.448 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2024-02-02 15:20:19.391670853 +0000 2024-02-02T15:20:20.448 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2024-02-02 15:12:28.733155591 +0000 2024-02-02T15:20:20.448 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2024-02-02 15:12:28.733155591 +0000 2024-02-02T15:20:20.448 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: - 2024-02-02T15:20:20.448 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-02T15:20:20.513 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2024-02-02T15:20:20.514 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2024-02-02T15:20:20.514 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.000244051 s, 2.1 MB/s 2024-02-02T15:20:20.515 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-02T15:20:20.539 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Feb 02 15:20:20 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-z[47590]: 2024-02-02T15:20:20.362+0000 7f24fb277200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-02T15:20:20.540 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Feb 02 15:20:20 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-z[47590]: 2024-02-02T15:20:20.452+0000 7f24fb277200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-02T15:20:20.557 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/vg_nvme/lv_3 2024-02-02T15:20:20.615 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-02T15:20:20.615 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-02T15:20:20.615 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 6h/6d Inode: 173608 Links: 1 2024-02-02T15:20:20.615 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-02T15:20:20.616 INFO:teuthology.orchestra.run.smithi142.stdout:Context: system_u:object_r:device_t:s0 2024-02-02T15:20:20.616 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2024-02-02 15:20:19.391670853 +0000 2024-02-02T15:20:20.616 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2024-02-02 15:12:29.110148175 +0000 2024-02-02T15:20:20.616 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2024-02-02 15:12:29.110148175 +0000 2024-02-02T15:20:20.616 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: - 2024-02-02T15:20:20.616 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-02T15:20:20.682 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2024-02-02T15:20:20.682 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2024-02-02T15:20:20.682 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.00019381 s, 2.6 MB/s 2024-02-02T15:20:20.683 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-02T15:20:20.741 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/vg_nvme/lv_4 2024-02-02T15:20:20.799 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-02T15:20:20.800 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-02T15:20:20.800 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 6h/6d Inode: 176530 Links: 1 2024-02-02T15:20:20.800 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-02T15:20:20.800 INFO:teuthology.orchestra.run.smithi142.stdout:Context: system_u:object_r:device_t:s0 2024-02-02T15:20:20.800 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2024-02-02 15:20:19.392670834 +0000 2024-02-02T15:20:20.800 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2024-02-02 15:12:29.459141311 +0000 2024-02-02T15:20:20.800 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2024-02-02 15:12:29.459141311 +0000 2024-02-02T15:20:20.800 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: - 2024-02-02T15:20:20.800 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-02T15:20:20.866 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2024-02-02T15:20:20.866 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2024-02-02T15:20:20.866 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.000233452 s, 2.2 MB/s 2024-02-02T15:20:20.867 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-02T15:20:20.926 INFO:tasks.cephadm:Deploying osd.0 on smithi001 with /dev/vg_nvme/lv_4... 2024-02-02T15:20:20.926 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-02-02T15:20:20.957 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:20 smithi142 ceph-mon[45728]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-02T15:20:20.957 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:20 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:20.957 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:20 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:20.957 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:20 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:20.957 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:20 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-02T15:20:20.958 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:20 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-02T15:20:20.958 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:20 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-02T15:20:20.958 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:20 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:20:20.958 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:20 smithi142 ceph-mon[45728]: Deploying daemon mgr.y on smithi071 2024-02-02T15:20:20.958 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Feb 02 15:20:20 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-z[47590]: 2024-02-02T15:20:20.539+0000 7f24fb277200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-02T15:20:21.287 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.a/config 2024-02-02T15:20:21.364 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Feb 02 15:20:21 smithi071 systemd[1]: Starting Ceph mgr.y for 371a409c-c1de-11ee-95b6-87774f69a715... 2024-02-02T15:20:21.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:20 smithi071 ceph-mon[44707]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-02T15:20:21.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:20 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:21.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:20 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:21.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:20 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:21.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:20 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-02T15:20:21.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:20 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-02T15:20:21.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:20 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-02T15:20:21.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:20 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:20:21.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:20 smithi071 ceph-mon[44707]: Deploying daemon mgr.y on smithi071 2024-02-02T15:20:21.415 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:20 smithi001 ceph-mon[35470]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-02T15:20:21.415 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:20 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:21.415 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:20 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:21.415 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:20 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:21.416 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:20 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-02T15:20:21.416 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:20 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-02T15:20:21.416 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:20 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-02T15:20:21.416 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:20 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:20:21.416 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:20 smithi001 ceph-mon[35470]: Deploying daemon mgr.y on smithi071 2024-02-02T15:20:21.416 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Feb 02 15:20:21 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-z[47590]: 2024-02-02T15:20:21.174+0000 7f24fb277200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-02T15:20:21.670 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Feb 02 15:20:21 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-z[47590]: 2024-02-02T15:20:21.415+0000 7f24fb277200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-02T15:20:21.670 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Feb 02 15:20:21 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-z[47590]: 2024-02-02T15:20:21.499+0000 7f24fb277200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-02T15:20:21.670 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Feb 02 15:20:21 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-z[47590]: 2024-02-02T15:20:21.606+0000 7f24fb277200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-02T15:20:21.864 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Feb 02 15:20:21 smithi071 ceph-mgr[46210]: -- 172.21.15.71:0/3068855890 <== mon.2 v2:172.21.15.71:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561ceba551e0 con 0x561ceba2ac00 2024-02-02T15:20:21.864 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Feb 02 15:20:21 smithi071 bash[46172]: ee6fbe6035563c746f5fb44c022dc21e0289a77b5ae72624ca794e8861f68ac6 2024-02-02T15:20:21.864 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Feb 02 15:20:21 smithi071 systemd[1]: Started Ceph mgr.y for 371a409c-c1de-11ee-95b6-87774f69a715. 2024-02-02T15:20:22.047 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Feb 02 15:20:21 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-y[46187]: 2024-02-02T15:20:21.949+0000 7f4e9ef49200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-02T15:20:22.170 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Feb 02 15:20:21 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-z[47590]: 2024-02-02T15:20:21.764+0000 7f24fb277200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-02T15:20:22.170 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Feb 02 15:20:21 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-z[47590]: 2024-02-02T15:20:21.892+0000 7f24fb277200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-02T15:20:22.170 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Feb 02 15:20:21 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-z[47590]: 2024-02-02T15:20:21.989+0000 7f24fb277200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-02T15:20:22.240 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Feb 02 15:20:22 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-y[46187]: 2024-02-02T15:20:22.046+0000 7f4e9ef49200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-02T15:20:22.240 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Feb 02 15:20:22 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-y[46187]: 2024-02-02T15:20:22.143+0000 7f4e9ef49200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-02T15:20:22.364 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Feb 02 15:20:22 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-y[46187]: 2024-02-02T15:20:22.240+0000 7f4e9ef49200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-02T15:20:22.655 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Feb 02 15:20:22 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-z[47590]: 2024-02-02T15:20:22.388+0000 7f24fb277200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-02T15:20:22.899 INFO:teuthology.orchestra.run.smithi001.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-02-02T15:20:22.899 INFO:teuthology.orchestra.run.smithi001.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-02-02T15:20:22.899 INFO:teuthology.orchestra.run.smithi001.stderr: stderr: 10+0 records in 2024-02-02T15:20:22.899 INFO:teuthology.orchestra.run.smithi001.stderr:10+0 records out 2024-02-02T15:20:22.899 INFO:teuthology.orchestra.run.smithi001.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.030597 s, 343 MB/s 2024-02-02T15:20:22.899 INFO:teuthology.orchestra.run.smithi001.stderr:--> Zapping successful for: 2024-02-02T15:20:22.920 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Feb 02 15:20:22 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-z[47590]: 2024-02-02T15:20:22.674+0000 7f24fb277200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-02T15:20:22.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:22 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:22.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:22 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:22.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:22 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:22.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:22 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:22.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:22 smithi142 ceph-mon[45728]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-02T15:20:22.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:22 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:20:22.936 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:22 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:22.936 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:22 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:22.936 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:22 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:22.936 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:22 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:22.936 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:22 smithi001 ceph-mon[35470]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-02T15:20:22.937 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:22 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:20:23.114 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Feb 02 15:20:22 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-y[46187]: 2024-02-02T15:20:22.885+0000 7f4e9ef49200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-02T15:20:23.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:22 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:23.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:22 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:23.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:22 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:23.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:22 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:23.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:22 smithi071 ceph-mon[44707]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-02T15:20:23.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:22 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:20:23.279 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph orch daemon add osd smithi001:vg_nvme/lv_4 2024-02-02T15:20:23.420 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Feb 02 15:20:22 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-z[47590]: 2024-02-02T15:20:22.967+0000 7f24fb277200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-02T15:20:23.420 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Feb 02 15:20:23 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-z[47590]: 2024-02-02T15:20:23.063+0000 7f24fb277200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-02T15:20:23.470 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Feb 02 15:20:23 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-y[46187]: 2024-02-02T15:20:23.125+0000 7f4e9ef49200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-02T15:20:23.470 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Feb 02 15:20:23 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-y[46187]: 2024-02-02T15:20:23.208+0000 7f4e9ef49200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-02T15:20:23.470 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Feb 02 15:20:23 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-y[46187]: 2024-02-02T15:20:23.315+0000 7f4e9ef49200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-02T15:20:23.470 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Feb 02 15:20:23 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-y[46187]: 2024-02-02T15:20:23.469+0000 7f4e9ef49200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-02T15:20:23.721 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Feb 02 15:20:23 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-y[46187]: 2024-02-02T15:20:23.596+0000 7f4e9ef49200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-02T15:20:23.721 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Feb 02 15:20:23 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-y[46187]: 2024-02-02T15:20:23.680+0000 7f4e9ef49200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-02T15:20:23.782 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.a/config 2024-02-02T15:20:23.846 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:23 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:23.847 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:23 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:23.847 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:23 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:23.847 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:23 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:23.847 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:23 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:20:23.847 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:23 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:20:23.847 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:23 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:23.847 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:23 smithi001 ceph-mon[35470]: Reconfiguring mgr.x (unknown last config time)... 2024-02-02T15:20:23.847 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:23 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-02T15:20:23.847 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:23 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-02T15:20:23.847 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:23 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:20:23.847 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:23 smithi001 ceph-mon[35470]: Reconfiguring daemon mgr.x on smithi001 2024-02-02T15:20:24.049 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:23 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:24.049 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:23 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:24.049 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:23 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:24.049 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:23 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:24.049 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:23 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:20:24.049 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:23 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:20:24.049 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:23 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:24.049 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:23 smithi071 ceph-mon[44707]: Reconfiguring mgr.x (unknown last config time)... 2024-02-02T15:20:24.049 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:23 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-02T15:20:24.049 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:23 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-02T15:20:24.049 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:23 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:20:24.049 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:23 smithi071 ceph-mon[44707]: Reconfiguring daemon mgr.x on smithi001 2024-02-02T15:20:24.055 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Feb 02 15:20:23 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-z[47590]: 2024-02-02T15:20:23.817+0000 7f24fb277200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-02T15:20:24.055 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Feb 02 15:20:23 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-z[47590]: 2024-02-02T15:20:23.910+0000 7f24fb277200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-02T15:20:24.055 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:23 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:24.055 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:23 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:24.055 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:23 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:24.055 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:23 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:24.055 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:23 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:20:24.056 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:23 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:20:24.056 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:23 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:24.056 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:23 smithi142 ceph-mon[45728]: Reconfiguring mgr.x (unknown last config time)... 2024-02-02T15:20:24.056 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:23 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-02T15:20:24.056 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:23 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-02T15:20:24.056 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:23 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:20:24.056 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:23 smithi142 ceph-mon[45728]: Reconfiguring daemon mgr.x on smithi001 2024-02-02T15:20:24.315 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Feb 02 15:20:24 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-y[46187]: 2024-02-02T15:20:24.048+0000 7f4e9ef49200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-02T15:20:24.420 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Feb 02 15:20:24 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-z[47590]: 2024-02-02T15:20:24.055+0000 7f24fb277200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-02T15:20:24.597 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Feb 02 15:20:24 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-y[46187]: 2024-02-02T15:20:24.314+0000 7f4e9ef49200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-02T15:20:24.597 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Feb 02 15:20:24 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-y[46187]: 2024-02-02T15:20:24.596+0000 7f4e9ef49200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-02T15:20:24.864 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Feb 02 15:20:24 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-y[46187]: 2024-02-02T15:20:24.689+0000 7f4e9ef49200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-02T15:20:25.053 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Feb 02 15:20:24 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-z[47590]: 2024-02-02T15:20:24.685+0000 7f24fb277200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-02T15:20:25.054 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Feb 02 15:20:24 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-z[47590]: 2024-02-02T15:20:24.864+0000 7f24fb277200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-02T15:20:25.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:25.109+0000 7f8bbfc00700 1 -- 172.21.15.1:0/2282571735 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bb8102970 msgr2=0x7f8bb8109220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:20:25.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:25.109+0000 7f8bbfc00700 1 --2- 172.21.15.1:0/2282571735 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bb8102970 0x7f8bb8109220 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8bb0009f90 tx=0x7f8bb000b330 comp rx=0 tx=0).stop 2024-02-02T15:20:25.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:25.109+0000 7f8bbfc00700 1 -- 172.21.15.1:0/2282571735 shutdown_connections 2024-02-02T15:20:25.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:25.109+0000 7f8bbfc00700 1 --2- 172.21.15.1:0/2282571735 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bb8102970 0x7f8bb8109220 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:20:25.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:25.109+0000 7f8bbfc00700 1 --2- 172.21.15.1:0/2282571735 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8bb8100dd0 0x7f8bb81011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:20:25.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:25.109+0000 7f8bbfc00700 1 --2- 172.21.15.1:0/2282571735 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bb8101fd0 0x7f8bb8102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:20:25.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:25.109+0000 7f8bbfc00700 1 -- 172.21.15.1:0/2282571735 >> 172.21.15.1:0/2282571735 conn(0x7f8bb80fc330 msgr2=0x7f8bb80fe7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:20:25.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:25.109+0000 7f8bbfc00700 1 -- 172.21.15.1:0/2282571735 shutdown_connections 2024-02-02T15:20:25.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:25.109+0000 7f8bbfc00700 1 -- 172.21.15.1:0/2282571735 wait complete. 2024-02-02T15:20:25.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:25.111+0000 7f8bbfc00700 1 Processor -- start 2024-02-02T15:20:25.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:25.111+0000 7f8bbfc00700 1 -- start start 2024-02-02T15:20:25.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:25.112+0000 7f8bbfc00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bb8100dd0 0x7f8bb8198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:20:25.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:25.112+0000 7f8bbfc00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8bb8101fd0 0x7f8bb81992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:20:25.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:25.112+0000 7f8bbfc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bb8102970 0x7f8bb81a0370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:20:25.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:25.112+0000 7f8bbfc00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb81a0940 con 0x7f8bb8100dd0 2024-02-02T15:20:25.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:25.112+0000 7f8bbfc00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb81a0a80 con 0x7f8bb8101fd0 2024-02-02T15:20:25.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:25.112+0000 7f8bbfc00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb81a0c10 con 0x7f8bb8102970 2024-02-02T15:20:25.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:25.112+0000 7f8bb7e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8bb8101fd0 0x7f8bb81992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:20:25.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:25.112+0000 7f8bbd200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bb8100dd0 0x7f8bb8198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:20:25.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:25.112+0000 7f8bbdc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bb8102970 0x7f8bb81a0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:20:25.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:25.112+0000 7f8bb7e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8bb8101fd0 0x7f8bb81992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:46202/0 (socket says 172.21.15.1:46202) 2024-02-02T15:20:25.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:25.112+0000 7f8bb7e00700 1 -- 172.21.15.1:0/3606648304 learned_addr learned my addr 172.21.15.1:0/3606648304 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:20:25.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:25.112+0000 7f8bb7e00700 1 -- 172.21.15.1:0/3606648304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bb8102970 msgr2=0x7f8bb81a0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:20:25.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:25.112+0000 7f8bb7e00700 1 --2- 172.21.15.1:0/3606648304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bb8102970 0x7f8bb81a0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:20:25.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:25.112+0000 7f8bb7e00700 1 -- 172.21.15.1:0/3606648304 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bb8100dd0 msgr2=0x7f8bb8198db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:20:25.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:25.112+0000 7f8bb7e00700 1 --2- 172.21.15.1:0/3606648304 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bb8100dd0 0x7f8bb8198db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:20:25.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:25.112+0000 7f8bb7e00700 1 -- 172.21.15.1:0/3606648304 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ba80095f0 con 0x7f8bb8101fd0 2024-02-02T15:20:25.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:25.112+0000 7f8bbdc00700 1 --2- 172.21.15.1:0/3606648304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bb8102970 0x7f8bb81a0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:20:25.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:25.112+0000 7f8bbd200700 1 --2- 172.21.15.1:0/3606648304 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bb8100dd0 0x7f8bb8198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:20:25.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:25.113+0000 7f8bb7e00700 1 --2- 172.21.15.1:0/3606648304 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8bb8101fd0 0x7f8bb81992f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8ba000e9a0 tx=0x7f8ba000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:20:25.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:25.113+0000 7f8bb5600700 1 -- 172.21.15.1:0/3606648304 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba0009790 con 0x7f8bb8101fd0 2024-02-02T15:20:25.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:25.113+0000 7f8bb5600700 1 -- 172.21.15.1:0/3606648304 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ba0010430 con 0x7f8bb8101fd0 2024-02-02T15:20:25.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:25.113+0000 7f8bb5600700 1 -- 172.21.15.1:0/3606648304 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba0018680 con 0x7f8bb8101fd0 2024-02-02T15:20:25.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:25.113+0000 7f8bbfc00700 1 -- 172.21.15.1:0/3606648304 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bb0009c40 con 0x7f8bb8101fd0 2024-02-02T15:20:25.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:25.113+0000 7f8bbfc00700 1 -- 172.21.15.1:0/3606648304 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bb81a11b0 con 0x7f8bb8101fd0 2024-02-02T15:20:25.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:25.114+0000 7f8bb5600700 1 -- 172.21.15.1:0/3606648304 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 13) v1 ==== 45323+0+0 (secure 0 0 0) 0x7f8ba00187e0 con 0x7f8bb8101fd0 2024-02-02T15:20:25.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:25.114+0000 7f8bbfc00700 1 -- 172.21.15.1:0/3606648304 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bb8193420 con 0x7f8bb8101fd0 2024-02-02T15:20:25.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:25.114+0000 7f8bb5600700 1 --2- 172.21.15.1:0/3606648304 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f8b980385d0 0x7f8b9803aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:20:25.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:25.115+0000 7f8bb5600700 1 -- 172.21.15.1:0/3606648304 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f8ba0014070 con 0x7f8bb8101fd0 2024-02-02T15:20:25.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:25.115+0000 7f8bbd200700 1 --2- 172.21.15.1:0/3606648304 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f8b980385d0 0x7f8b9803aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:20:25.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:25.115+0000 7f8bbd200700 1 --2- 172.21.15.1:0/3606648304 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f8b980385d0 0x7f8b9803aa90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8101e30 tx=0x7f8ba8009310 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:20:25.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:25.120+0000 7f8bb5600700 1 -- 172.21.15.1:0/3606648304 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f8ba001f310 con 0x7f8bb8101fd0 2024-02-02T15:20:25.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:25 smithi001 ceph-mon[35470]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-02T15:20:25.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:25 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:25.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:25 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:25.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:25 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:20:25.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:25 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:20:25.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:25 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:20:25.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:25 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:25.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:25.338+0000 7f8bbfc00700 1 -- 172.21.15.1:0/3606648304 --> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f8bb8061b40 con 0x7f8b980385d0 2024-02-02T15:20:25.420 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Feb 02 15:20:25 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-z[47590]: 2024-02-02T15:20:25.053+0000 7f24fb277200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-02T15:20:25.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:25 smithi142 ceph-mon[45728]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-02T15:20:25.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:25 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:25.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:25 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:25.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:25 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:20:25.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:25 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:20:25.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:25 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:20:25.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:25 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:25.435 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:25 smithi071 ceph-mon[44707]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-02T15:20:25.436 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:25 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:25.436 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:25 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:25.436 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:25 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:20:25.436 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:25 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:20:25.436 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:25 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:20:25.436 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:25 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:25.864 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Feb 02 15:20:25 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-y[46187]: 2024-02-02T15:20:25.435+0000 7f4e9ef49200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-02T15:20:25.864 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Feb 02 15:20:25 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-y[46187]: 2024-02-02T15:20:25.530+0000 7f4e9ef49200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-02T15:20:25.864 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Feb 02 15:20:25 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-y[46187]: 2024-02-02T15:20:25.675+0000 7f4e9ef49200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-02T15:20:26.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:26 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-02T15:20:26.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:26 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-02T15:20:26.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:26 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:20:26.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:26 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:26.229 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Feb 02 15:20:25 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-z[47590]: 2024-02-02T15:20:25.966+0000 7f24fb277200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-02T15:20:26.230 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:26 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-02T15:20:26.230 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:26 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-02T15:20:26.230 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:26 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:20:26.230 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:26 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:26.476 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Feb 02 15:20:26 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-y[46187]: 2024-02-02T15:20:26.296+0000 7f4e9ef49200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-02T15:20:26.476 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:26 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-02T15:20:26.477 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:26 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-02T15:20:26.477 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:26 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:20:26.477 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:26 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:26.501 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Feb 02 15:20:26 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-z[47590]: 2024-02-02T15:20:26.229+0000 7f24fb277200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-02T15:20:26.501 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Feb 02 15:20:26 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-z[47590]: 2024-02-02T15:20:26.315+0000 7f24fb277200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-02T15:20:26.864 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Feb 02 15:20:26 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-y[46187]: 2024-02-02T15:20:26.476+0000 7f4e9ef49200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-02T15:20:26.864 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Feb 02 15:20:26 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-y[46187]: 2024-02-02T15:20:26.667+0000 7f4e9ef49200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-02T15:20:26.919 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Feb 02 15:20:26 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-z[47590]: 2024-02-02T15:20:26.501+0000 7f24fb277200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-02T15:20:27.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:27.128+0000 7f8bb5600700 1 -- 172.21.15.1:0/3606648304 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mgrmap(e 14) v1 ==== 90168+0+0 (secure 0 0 0) 0x7f8ba0051080 con 0x7f8bb8101fd0 2024-02-02T15:20:27.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:27 smithi142 ceph-mon[45728]: from='client.24125 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:20:27.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:27 smithi142 ceph-mon[45728]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-02T15:20:27.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:27 smithi142 ceph-mon[45728]: Standby manager daemon z started 2024-02-02T15:20:27.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:27 smithi142 ceph-mon[45728]: from='mgr.? 172.21.15.142:0/2080348839' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-02T15:20:27.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:27 smithi142 ceph-mon[45728]: from='mgr.? 172.21.15.142:0/2080348839' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-02T15:20:27.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:27 smithi142 ceph-mon[45728]: from='mgr.? 172.21.15.142:0/2080348839' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-02T15:20:27.421 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:27 smithi142 ceph-mon[45728]: from='mgr.? 172.21.15.142:0/2080348839' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-02T15:20:27.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:27 smithi001 ceph-mon[35470]: from='client.24125 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:20:27.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:27 smithi001 ceph-mon[35470]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-02T15:20:27.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:27 smithi001 ceph-mon[35470]: Standby manager daemon z started 2024-02-02T15:20:27.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:27 smithi001 ceph-mon[35470]: from='mgr.? 172.21.15.142:0/2080348839' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-02T15:20:27.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:27 smithi001 ceph-mon[35470]: from='mgr.? 172.21.15.142:0/2080348839' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-02T15:20:27.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:27 smithi001 ceph-mon[35470]: from='mgr.? 172.21.15.142:0/2080348839' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-02T15:20:27.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:27 smithi001 ceph-mon[35470]: from='mgr.? 172.21.15.142:0/2080348839' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-02T15:20:27.579 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:27 smithi071 ceph-mon[44707]: from='client.24125 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:20:27.579 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:27 smithi071 ceph-mon[44707]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-02T15:20:27.579 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:27 smithi071 ceph-mon[44707]: Standby manager daemon z started 2024-02-02T15:20:27.579 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:27 smithi071 ceph-mon[44707]: from='mgr.? 172.21.15.142:0/2080348839' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-02T15:20:27.579 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:27 smithi071 ceph-mon[44707]: from='mgr.? 172.21.15.142:0/2080348839' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-02T15:20:27.579 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:27 smithi071 ceph-mon[44707]: from='mgr.? 172.21.15.142:0/2080348839' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-02T15:20:27.579 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:27 smithi071 ceph-mon[44707]: from='mgr.? 172.21.15.142:0/2080348839' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-02T15:20:27.844 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Feb 02 15:20:27 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-y[46187]: 2024-02-02T15:20:27.578+0000 7f4e9ef49200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-02T15:20:27.844 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Feb 02 15:20:27 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-y[46187]: 2024-02-02T15:20:27.844+0000 7f4e9ef49200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-02T15:20:28.114 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Feb 02 15:20:27 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-y[46187]: 2024-02-02T15:20:27.931+0000 7f4e9ef49200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-02T15:20:28.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:28 smithi142 ceph-mon[45728]: mgrmap e14: x(active, since 61s), standbys: z 2024-02-02T15:20:28.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:28 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-02T15:20:28.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:28 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/3932148080' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73e8a07f-3560-4c19-85ed-030d4d8f0f23"}]: dispatch 2024-02-02T15:20:28.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:28 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/3932148080' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "73e8a07f-3560-4c19-85ed-030d4d8f0f23"}]': finished 2024-02-02T15:20:28.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:28 smithi142 ceph-mon[45728]: osdmap e5: 1 total, 0 up, 1 in 2024-02-02T15:20:28.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:28 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-02T15:20:28.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:28 smithi001 ceph-mon[35470]: mgrmap e14: x(active, since 61s), standbys: z 2024-02-02T15:20:28.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:28 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-02T15:20:28.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:28 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/3932148080' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73e8a07f-3560-4c19-85ed-030d4d8f0f23"}]: dispatch 2024-02-02T15:20:28.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:28 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/3932148080' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "73e8a07f-3560-4c19-85ed-030d4d8f0f23"}]': finished 2024-02-02T15:20:28.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:28 smithi001 ceph-mon[35470]: osdmap e5: 1 total, 0 up, 1 in 2024-02-02T15:20:28.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:28 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-02T15:20:28.614 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Feb 02 15:20:28 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-y[46187]: 2024-02-02T15:20:28.119+0000 7f4e9ef49200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-02T15:20:28.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:28 smithi071 ceph-mon[44707]: mgrmap e14: x(active, since 61s), standbys: z 2024-02-02T15:20:28.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:28 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-02T15:20:28.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:28 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/3932148080' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "73e8a07f-3560-4c19-85ed-030d4d8f0f23"}]: dispatch 2024-02-02T15:20:28.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:28 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/3932148080' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "73e8a07f-3560-4c19-85ed-030d4d8f0f23"}]': finished 2024-02-02T15:20:28.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:28 smithi071 ceph-mon[44707]: osdmap e5: 1 total, 0 up, 1 in 2024-02-02T15:20:28.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:28 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-02T15:20:29.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:29.143+0000 7f8bb5600700 1 -- 172.21.15.1:0/3606648304 <== mon.2 v2:172.21.15.71:3300/0 8 ==== mgrmap(e 15) v1 ==== 135013+0+0 (secure 0 0 0) 0x7f8ba0017700 con 0x7f8bb8101fd0 2024-02-02T15:20:29.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:29 smithi142 ceph-mon[45728]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-02T15:20:29.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:29 smithi142 ceph-mon[45728]: from='mgr.? 172.21.15.71:0/428413582' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-02-02T15:20:29.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:29 smithi142 ceph-mon[45728]: from='mgr.? 172.21.15.71:0/428413582' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-02T15:20:29.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:29 smithi142 ceph-mon[45728]: Standby manager daemon y started 2024-02-02T15:20:29.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:29 smithi142 ceph-mon[45728]: from='mgr.? 172.21.15.71:0/428413582' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-02-02T15:20:29.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:29 smithi142 ceph-mon[45728]: from='mgr.? 172.21.15.71:0/428413582' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-02T15:20:29.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:29 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/3868962914' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-02T15:20:29.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:29 smithi001 ceph-mon[35470]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-02T15:20:29.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:29 smithi001 ceph-mon[35470]: from='mgr.? 172.21.15.71:0/428413582' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-02-02T15:20:29.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:29 smithi001 ceph-mon[35470]: from='mgr.? 172.21.15.71:0/428413582' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-02T15:20:29.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:29 smithi001 ceph-mon[35470]: Standby manager daemon y started 2024-02-02T15:20:29.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:29 smithi001 ceph-mon[35470]: from='mgr.? 172.21.15.71:0/428413582' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-02-02T15:20:29.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:29 smithi001 ceph-mon[35470]: from='mgr.? 172.21.15.71:0/428413582' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-02T15:20:29.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:29 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/3868962914' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-02T15:20:29.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:29 smithi071 ceph-mon[44707]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-02T15:20:29.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:29 smithi071 ceph-mon[44707]: from='mgr.? 172.21.15.71:0/428413582' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-02-02T15:20:29.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:29 smithi071 ceph-mon[44707]: from='mgr.? 172.21.15.71:0/428413582' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-02T15:20:29.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:29 smithi071 ceph-mon[44707]: Standby manager daemon y started 2024-02-02T15:20:29.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:29 smithi071 ceph-mon[44707]: from='mgr.? 172.21.15.71:0/428413582' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-02-02T15:20:29.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:29 smithi071 ceph-mon[44707]: from='mgr.? 172.21.15.71:0/428413582' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-02T15:20:29.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:29 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/3868962914' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-02T15:20:30.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:30 smithi142 ceph-mon[45728]: mgrmap e15: x(active, since 63s), standbys: z, y 2024-02-02T15:20:30.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:30 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-02T15:20:30.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:30 smithi001 ceph-mon[35470]: mgrmap e15: x(active, since 63s), standbys: z, y 2024-02-02T15:20:30.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:30 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-02T15:20:30.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:30 smithi071 ceph-mon[44707]: mgrmap e15: x(active, since 63s), standbys: z, y 2024-02-02T15:20:30.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:30 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-02T15:20:31.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:31 smithi142 ceph-mon[45728]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-02T15:20:31.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:31 smithi001 ceph-mon[35470]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-02T15:20:31.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:31 smithi071 ceph-mon[44707]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-02T15:20:33.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:33 smithi142 ceph-mon[45728]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-02T15:20:33.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:33 smithi001 ceph-mon[35470]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-02T15:20:33.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:33 smithi071 ceph-mon[44707]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-02T15:20:35.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:35 smithi142 ceph-mon[45728]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-02T15:20:35.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:35 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-02T15:20:35.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:35 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:20:35.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:35 smithi001 ceph-mon[35470]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-02T15:20:35.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:35 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-02T15:20:35.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:35 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:20:35.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:35 smithi071 ceph-mon[44707]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-02T15:20:35.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:35 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-02T15:20:35.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:35 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:20:36.394 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:36 smithi001 ceph-mon[35470]: Deploying daemon osd.0 on smithi001 2024-02-02T15:20:36.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:36 smithi142 ceph-mon[45728]: Deploying daemon osd.0 on smithi001 2024-02-02T15:20:36.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:36 smithi071 ceph-mon[44707]: Deploying daemon osd.0 on smithi001 2024-02-02T15:20:37.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:37 smithi142 ceph-mon[45728]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-02T15:20:37.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:37 smithi001 ceph-mon[35470]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-02T15:20:37.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:37 smithi071 ceph-mon[44707]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-02T15:20:39.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:39 smithi142 ceph-mon[45728]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-02T15:20:39.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:39 smithi001 ceph-mon[35470]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-02T15:20:39.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:39 smithi071 ceph-mon[44707]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-02T15:20:40.393 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:40 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:40.393 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:40 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:40.393 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:40 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:20:40.394 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:40 smithi001 ceph-mon[35470]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-02T15:20:40.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:40 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:40.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:40 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:40.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:40 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:20:40.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:40 smithi071 ceph-mon[44707]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-02T15:20:40.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:40 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:40.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:40 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:40.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:40 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:20:40.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:40 smithi142 ceph-mon[45728]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-02T15:20:42.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:42.251+0000 7f8bb5600700 1 -- 172.21.15.1:0/3606648304 <== mgr.14154 v2:172.21.15.1:6800/3768989981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f8bb8061b40 con 0x7f8b980385d0 2024-02-02T15:20:42.253 INFO:teuthology.orchestra.run.smithi001.stdout:Created osd(s) 0 on host 'smithi001' 2024-02-02T15:20:42.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:42.254+0000 7f8bbfc00700 1 -- 172.21.15.1:0/3606648304 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f8b980385d0 msgr2=0x7f8b9803aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:20:42.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:42.254+0000 7f8bbfc00700 1 --2- 172.21.15.1:0/3606648304 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f8b980385d0 0x7f8b9803aa90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8101e30 tx=0x7f8ba8009310 comp rx=0 tx=0).stop 2024-02-02T15:20:42.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:42.254+0000 7f8bbfc00700 1 -- 172.21.15.1:0/3606648304 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8bb8101fd0 msgr2=0x7f8bb81992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:20:42.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:42.254+0000 7f8bbfc00700 1 --2- 172.21.15.1:0/3606648304 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8bb8101fd0 0x7f8bb81992f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8ba000e9a0 tx=0x7f8ba000ecb0 comp rx=0 tx=0).stop 2024-02-02T15:20:42.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:42.254+0000 7f8bbfc00700 1 -- 172.21.15.1:0/3606648304 shutdown_connections 2024-02-02T15:20:42.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:42.254+0000 7f8bbfc00700 1 --2- 172.21.15.1:0/3606648304 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f8b980385d0 0x7f8b9803aa90 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:20:42.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:42.254+0000 7f8bbfc00700 1 --2- 172.21.15.1:0/3606648304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bb8102970 0x7f8bb81a0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:20:42.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:42.254+0000 7f8bbfc00700 1 --2- 172.21.15.1:0/3606648304 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bb8100dd0 0x7f8bb8198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:20:42.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:42.254+0000 7f8bbfc00700 1 --2- 172.21.15.1:0/3606648304 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8bb8101fd0 0x7f8bb81992f0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:20:42.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:42.254+0000 7f8bbfc00700 1 -- 172.21.15.1:0/3606648304 >> 172.21.15.1:0/3606648304 conn(0x7f8bb80fc330 msgr2=0x7f8bb81078b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:20:42.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:42.255+0000 7f8bbfc00700 1 -- 172.21.15.1:0/3606648304 shutdown_connections 2024-02-02T15:20:42.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:20:42.255+0000 7f8bbfc00700 1 -- 172.21.15.1:0/3606648304 wait complete. 2024-02-02T15:20:42.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:42 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:42.453 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:42 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:42.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:42 smithi001 ceph-mon[35470]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-02T15:20:42.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:42 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:20:42.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:42 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:20:42.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:42 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:42.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:42 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:42.454 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:42 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:42.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:42 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:42.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:42 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:42.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:42 smithi071 ceph-mon[44707]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-02T15:20:42.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:42 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:20:42.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:42 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:20:42.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:42 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:42.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:42 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:42.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:42 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:42.642 DEBUG:teuthology.orchestra.run.smithi001:osd.0> sudo journalctl -f -n 0 -u ceph-371a409c-c1de-11ee-95b6-87774f69a715@osd.0.service 2024-02-02T15:20:42.646 INFO:tasks.cephadm:Deploying osd.1 on smithi071 with /dev/vg_nvme/lv_4... 2024-02-02T15:20:42.646 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-02-02T15:20:42.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:42 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:42.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:42 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:42.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:42 smithi142 ceph-mon[45728]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-02T15:20:42.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:42 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:20:42.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:42 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:20:42.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:42 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:42.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:42 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:42.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:42 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:42.720 INFO:journalctl@ceph.osd.0.smithi001.stdout:-- Logs begin at Fri 2024-02-02 15:05:03 UTC. -- 2024-02-02T15:20:42.876 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.b/config 2024-02-02T15:20:43.277 INFO:journalctl@ceph.osd.0.smithi001.stdout:Feb 02 15:20:43 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-0[52951]: 2024-02-02T15:20:43.096+0000 7f117e6e7640 -1 osd.0 0 log_to_monitors true 2024-02-02T15:20:43.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:43 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:20:43.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:43 smithi071 ceph-mon[44707]: from='osd.0 [v2:172.21.15.1:6802/2591906423,v1:172.21.15.1:6803/2591906423]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-02T15:20:43.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:43 smithi071 ceph-mon[44707]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-02T15:20:43.662 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:43 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:20:43.662 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:43 smithi001 ceph-mon[35470]: from='osd.0 [v2:172.21.15.1:6802/2591906423,v1:172.21.15.1:6803/2591906423]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-02T15:20:43.662 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:43 smithi001 ceph-mon[35470]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-02T15:20:43.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:43 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:20:43.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:43 smithi142 ceph-mon[45728]: from='osd.0 [v2:172.21.15.1:6802/2591906423,v1:172.21.15.1:6803/2591906423]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-02T15:20:43.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:43 smithi142 ceph-mon[45728]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-02T15:20:44.090 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-02-02T15:20:44.091 INFO:teuthology.orchestra.run.smithi071.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-02-02T15:20:44.091 INFO:teuthology.orchestra.run.smithi071.stderr: stderr: 10+0 records in 2024-02-02T15:20:44.091 INFO:teuthology.orchestra.run.smithi071.stderr:10+0 records out 2024-02-02T15:20:44.091 INFO:teuthology.orchestra.run.smithi071.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0128138 s, 818 MB/s 2024-02-02T15:20:44.091 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping successful for: 2024-02-02T15:20:44.431 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:44 smithi071 ceph-mon[44707]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-02T15:20:44.431 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:44 smithi071 ceph-mon[44707]: osdmap e6: 1 total, 0 up, 1 in 2024-02-02T15:20:44.431 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:44 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-02T15:20:44.431 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:44 smithi071 ceph-mon[44707]: from='osd.0 [v2:172.21.15.1:6802/2591906423,v1:172.21.15.1:6803/2591906423]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-02-02T15:20:44.431 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:44 smithi071 ceph-mon[44707]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-02-02T15:20:44.431 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:44 smithi071 ceph-mon[44707]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-02T15:20:44.432 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph orch daemon add osd smithi071:vg_nvme/lv_4 2024-02-02T15:20:44.647 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.b/config 2024-02-02T15:20:44.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:44 smithi142 ceph-mon[45728]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-02T15:20:44.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:44 smithi142 ceph-mon[45728]: osdmap e6: 1 total, 0 up, 1 in 2024-02-02T15:20:44.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:44 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-02T15:20:44.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:44 smithi142 ceph-mon[45728]: from='osd.0 [v2:172.21.15.1:6802/2591906423,v1:172.21.15.1:6803/2591906423]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-02-02T15:20:44.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:44 smithi142 ceph-mon[45728]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-02-02T15:20:44.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:44 smithi142 ceph-mon[45728]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-02T15:20:44.690 INFO:journalctl@ceph.osd.0.smithi001.stdout:Feb 02 15:20:44 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-0[52951]: 2024-02-02T15:20:44.346+0000 7f1172c00700 -1 osd.0 0 waiting for initial osdmap 2024-02-02T15:20:44.690 INFO:journalctl@ceph.osd.0.smithi001.stdout:Feb 02 15:20:44 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-0[52951]: 2024-02-02T15:20:44.350+0000 7f116b200700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-02T15:20:44.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:44 smithi001 ceph-mon[35470]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-02T15:20:44.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:44 smithi001 ceph-mon[35470]: osdmap e6: 1 total, 0 up, 1 in 2024-02-02T15:20:44.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:44 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-02T15:20:44.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:44 smithi001 ceph-mon[35470]: from='osd.0 [v2:172.21.15.1:6802/2591906423,v1:172.21.15.1:6803/2591906423]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-02-02T15:20:44.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:44 smithi001 ceph-mon[35470]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-02-02T15:20:44.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:44 smithi001 ceph-mon[35470]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-02T15:20:45.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:45 smithi071 ceph-mon[44707]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]': finished 2024-02-02T15:20:45.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:45 smithi071 ceph-mon[44707]: osdmap e7: 1 total, 0 up, 1 in 2024-02-02T15:20:45.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:45 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-02T15:20:45.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:45 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-02T15:20:45.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:20:45.618+0000 7f8508b23700 1 -- 172.21.15.71:0/2347241082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8504100ed0 msgr2=0x7f8504101330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:20:45.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:20:45.618+0000 7f8508b23700 1 --2- 172.21.15.71:0/2347241082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8504100ed0 0x7f8504101330 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f84f80099c0 tx=0x7f84f8009cd0 comp rx=0 tx=0).stop 2024-02-02T15:20:45.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:20:45.619+0000 7f8508b23700 1 -- 172.21.15.71:0/2347241082 shutdown_connections 2024-02-02T15:20:45.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:20:45.619+0000 7f8508b23700 1 --2- 172.21.15.71:0/2347241082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8504101870 0x7f8504108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:20:45.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:20:45.619+0000 7f8508b23700 1 --2- 172.21.15.71:0/2347241082 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f85040ffcd0 0x7f85041000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:20:45.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:20:45.619+0000 7f8508b23700 1 --2- 172.21.15.71:0/2347241082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8504100ed0 0x7f8504101330 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:20:45.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:20:45.619+0000 7f8508b23700 1 -- 172.21.15.71:0/2347241082 >> 172.21.15.71:0/2347241082 conn(0x7f85040fb250 msgr2=0x7f85040fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:20:45.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:20:45.619+0000 7f8508b23700 1 -- 172.21.15.71:0/2347241082 shutdown_connections 2024-02-02T15:20:45.619 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:20:45.619+0000 7f8508b23700 1 -- 172.21.15.71:0/2347241082 wait complete. 2024-02-02T15:20:45.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:20:45.619+0000 7f8508b23700 1 Processor -- start 2024-02-02T15:20:45.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:20:45.619+0000 7f8508b23700 1 -- start start 2024-02-02T15:20:45.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:20:45.619+0000 7f8508b23700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85040ffcd0 0x7f8504195ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:20:45.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:20:45.619+0000 7f8508b23700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8504100ed0 0x7f8504196000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:20:45.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:20:45.619+0000 7f8508b23700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8504101870 0x7f850419d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:20:45.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:20:45.619+0000 7f8508b23700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f850419d650 con 0x7f8504101870 2024-02-02T15:20:45.621 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:20:45.619+0000 7f8508b23700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f850419d790 con 0x7f8504100ed0 2024-02-02T15:20:45.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:20:45.619+0000 7f8508b23700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f850419d920 con 0x7f85040ffcd0 2024-02-02T15:20:45.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:20:45.620+0000 7f8501d9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8504100ed0 0x7f8504196000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:20:45.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:20:45.620+0000 7f8501d9b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8504100ed0 0x7f8504196000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:50430/0 (socket says 172.21.15.71:50430) 2024-02-02T15:20:45.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:20:45.620+0000 7f8501d9b700 1 -- 172.21.15.71:0/1122742598 learned_addr learned my addr 172.21.15.71:0/1122742598 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-02T15:20:45.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:20:45.620+0000 7f850259c700 1 --2- 172.21.15.71:0/1122742598 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85040ffcd0 0x7f8504195ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:20:45.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:20:45.620+0000 7f8501d9b700 1 -- 172.21.15.71:0/1122742598 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85040ffcd0 msgr2=0x7f8504195ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:20:45.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:20:45.620+0000 7f8501d9b700 1 --2- 172.21.15.71:0/1122742598 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85040ffcd0 0x7f8504195ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:20:45.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:20:45.620+0000 7f8501d9b700 1 -- 172.21.15.71:0/1122742598 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8504101870 msgr2=0x7f850419d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:20:45.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:20:45.620+0000 7f8501d9b700 1 --2- 172.21.15.71:0/1122742598 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8504101870 0x7f850419d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:20:45.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:20:45.620+0000 7f8501d9b700 1 -- 172.21.15.71:0/1122742598 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84f80095f0 con 0x7f8504100ed0 2024-02-02T15:20:45.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:20:45.620+0000 7f8501d9b700 1 --2- 172.21.15.71:0/1122742598 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8504100ed0 0x7f8504196000 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f84f800c010 tx=0x7f84f8004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:20:45.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:20:45.620+0000 7f84f37fe700 1 -- 172.21.15.71:0/1122742598 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f801d070 con 0x7f8504100ed0 2024-02-02T15:20:45.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:20:45.620+0000 7f8508b23700 1 -- 172.21.15.71:0/1122742598 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f850419dba0 con 0x7f8504100ed0 2024-02-02T15:20:45.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:20:45.620+0000 7f84f37fe700 1 -- 172.21.15.71:0/1122742598 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f84f8022470 con 0x7f8504100ed0 2024-02-02T15:20:45.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:20:45.620+0000 7f84f37fe700 1 -- 172.21.15.71:0/1122742598 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f800f6c0 con 0x7f8504100ed0 2024-02-02T15:20:45.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:20:45.620+0000 7f8508b23700 1 -- 172.21.15.71:0/1122742598 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f850419e090 con 0x7f8504100ed0 2024-02-02T15:20:45.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:20:45.621+0000 7f8508b23700 1 -- 172.21.15.71:0/1122742598 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8504107320 con 0x7f8504100ed0 2024-02-02T15:20:45.623 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:20:45.622+0000 7f84f37fe700 1 -- 172.21.15.71:0/1122742598 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 15) v1 ==== 135013+0+0 (secure 0 0 0) 0x7f84f80225e0 con 0x7f8504100ed0 2024-02-02T15:20:45.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:20:45.623+0000 7f84f37fe700 1 --2- 172.21.15.71:0/1122742598 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f84ec0a0490 0x7f84ec0a2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:20:45.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:20:45.623+0000 7f84f37fe700 1 -- 172.21.15.71:0/1122742598 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1767+0+0 (secure 0 0 0) 0x7f84f80cb530 con 0x7f8504100ed0 2024-02-02T15:20:45.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:20:45.623+0000 7f850259c700 1 --2- 172.21.15.71:0/1122742598 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f84ec0a0490 0x7f84ec0a2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:20:45.624 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:20:45.623+0000 7f850259c700 1 --2- 172.21.15.71:0/1122742598 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f84ec0a0490 0x7f84ec0a2950 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8504100d30 tx=0x7f84f4009190 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:20:45.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:20:45.627+0000 7f84f37fe700 1 -- 172.21.15.71:0/1122742598 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f84f8096b30 con 0x7f8504100ed0 2024-02-02T15:20:45.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:45 smithi142 ceph-mon[45728]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]': finished 2024-02-02T15:20:45.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:45 smithi142 ceph-mon[45728]: osdmap e7: 1 total, 0 up, 1 in 2024-02-02T15:20:45.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:45 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-02T15:20:45.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:45 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-02T15:20:45.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:45 smithi001 ceph-mon[35470]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]': finished 2024-02-02T15:20:45.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:45 smithi001 ceph-mon[35470]: osdmap e7: 1 total, 0 up, 1 in 2024-02-02T15:20:45.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:45 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-02T15:20:45.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:45 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-02T15:20:45.822 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:20:45.822+0000 7f8508b23700 1 -- 172.21.15.71:0/1122742598 --> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f8504061180 con 0x7f84ec0a0490 2024-02-02T15:20:46.541 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:46 smithi071 ceph-mon[44707]: purged_snaps scrub starts 2024-02-02T15:20:46.541 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:46 smithi071 ceph-mon[44707]: purged_snaps scrub ok 2024-02-02T15:20:46.541 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:46 smithi071 ceph-mon[44707]: osd.0 [v2:172.21.15.1:6802/2591906423,v1:172.21.15.1:6803/2591906423] boot 2024-02-02T15:20:46.541 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:46 smithi071 ceph-mon[44707]: osdmap e8: 1 total, 1 up, 1 in 2024-02-02T15:20:46.541 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:46 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-02T15:20:46.541 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:46 smithi071 ceph-mon[44707]: Detected new or changed devices on smithi001 2024-02-02T15:20:46.542 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:46 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:46.542 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:46 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:46.542 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:46 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:20:46.542 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:46 smithi071 ceph-mon[44707]: Adjusting osd_memory_target on smithi001 to 17276M 2024-02-02T15:20:46.542 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:46 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:46.542 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:46 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:20:46.542 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:46 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:20:46.542 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:46 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:46.542 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:46 smithi071 ceph-mon[44707]: from='client.24152 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:20:46.542 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:46 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-02T15:20:46.542 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:46 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-02T15:20:46.542 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:46 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:20:46.542 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:46 smithi071 ceph-mon[44707]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-02T15:20:46.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:46 smithi142 ceph-mon[45728]: purged_snaps scrub starts 2024-02-02T15:20:46.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:46 smithi142 ceph-mon[45728]: purged_snaps scrub ok 2024-02-02T15:20:46.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:46 smithi142 ceph-mon[45728]: osd.0 [v2:172.21.15.1:6802/2591906423,v1:172.21.15.1:6803/2591906423] boot 2024-02-02T15:20:46.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:46 smithi142 ceph-mon[45728]: osdmap e8: 1 total, 1 up, 1 in 2024-02-02T15:20:46.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:46 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-02T15:20:46.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:46 smithi142 ceph-mon[45728]: Detected new or changed devices on smithi001 2024-02-02T15:20:46.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:46 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:46.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:46 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:46.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:46 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:20:46.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:46 smithi142 ceph-mon[45728]: Adjusting osd_memory_target on smithi001 to 17276M 2024-02-02T15:20:46.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:46 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:46.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:46 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:20:46.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:46 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:20:46.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:46 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:46.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:46 smithi142 ceph-mon[45728]: from='client.24152 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:20:46.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:46 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-02T15:20:46.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:46 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-02T15:20:46.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:46 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:20:46.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:46 smithi142 ceph-mon[45728]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-02T15:20:46.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:46 smithi001 ceph-mon[35470]: purged_snaps scrub starts 2024-02-02T15:20:46.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:46 smithi001 ceph-mon[35470]: purged_snaps scrub ok 2024-02-02T15:20:46.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:46 smithi001 ceph-mon[35470]: osd.0 [v2:172.21.15.1:6802/2591906423,v1:172.21.15.1:6803/2591906423] boot 2024-02-02T15:20:46.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:46 smithi001 ceph-mon[35470]: osdmap e8: 1 total, 1 up, 1 in 2024-02-02T15:20:46.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:46 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-02T15:20:46.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:46 smithi001 ceph-mon[35470]: Detected new or changed devices on smithi001 2024-02-02T15:20:46.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:46 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:46.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:46 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:46.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:46 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:20:46.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:46 smithi001 ceph-mon[35470]: Adjusting osd_memory_target on smithi001 to 17276M 2024-02-02T15:20:46.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:46 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:46.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:46 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:20:46.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:46 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:20:46.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:46 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:46.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:46 smithi001 ceph-mon[35470]: from='client.24152 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:20:46.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:46 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-02T15:20:46.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:46 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-02T15:20:46.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:46 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:20:46.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:46 smithi001 ceph-mon[35470]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-02T15:20:48.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:47 smithi071 ceph-mon[44707]: osdmap e9: 1 total, 1 up, 1 in 2024-02-02T15:20:48.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:47 smithi071 ceph-mon[44707]: from='client.? 172.21.15.71:0/2007034786' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e53cffec-0ff9-4ed9-8dcf-ff47d04db2bb"}]: dispatch 2024-02-02T15:20:48.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:47 smithi071 ceph-mon[44707]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e53cffec-0ff9-4ed9-8dcf-ff47d04db2bb"}]: dispatch 2024-02-02T15:20:48.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:47 smithi071 ceph-mon[44707]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e53cffec-0ff9-4ed9-8dcf-ff47d04db2bb"}]': finished 2024-02-02T15:20:48.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:47 smithi071 ceph-mon[44707]: osdmap e10: 2 total, 1 up, 2 in 2024-02-02T15:20:48.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:47 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-02T15:20:48.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:47 smithi142 ceph-mon[45728]: osdmap e9: 1 total, 1 up, 1 in 2024-02-02T15:20:48.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:47 smithi142 ceph-mon[45728]: from='client.? 172.21.15.71:0/2007034786' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e53cffec-0ff9-4ed9-8dcf-ff47d04db2bb"}]: dispatch 2024-02-02T15:20:48.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:47 smithi142 ceph-mon[45728]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e53cffec-0ff9-4ed9-8dcf-ff47d04db2bb"}]: dispatch 2024-02-02T15:20:48.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:47 smithi142 ceph-mon[45728]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e53cffec-0ff9-4ed9-8dcf-ff47d04db2bb"}]': finished 2024-02-02T15:20:48.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:47 smithi142 ceph-mon[45728]: osdmap e10: 2 total, 1 up, 2 in 2024-02-02T15:20:48.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:47 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-02T15:20:48.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:47 smithi001 ceph-mon[35470]: osdmap e9: 1 total, 1 up, 1 in 2024-02-02T15:20:48.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:47 smithi001 ceph-mon[35470]: from='client.? 172.21.15.71:0/2007034786' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e53cffec-0ff9-4ed9-8dcf-ff47d04db2bb"}]: dispatch 2024-02-02T15:20:48.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:47 smithi001 ceph-mon[35470]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e53cffec-0ff9-4ed9-8dcf-ff47d04db2bb"}]: dispatch 2024-02-02T15:20:48.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:47 smithi001 ceph-mon[35470]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e53cffec-0ff9-4ed9-8dcf-ff47d04db2bb"}]': finished 2024-02-02T15:20:48.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:47 smithi001 ceph-mon[35470]: osdmap e10: 2 total, 1 up, 2 in 2024-02-02T15:20:48.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:47 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-02T15:20:49.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:48 smithi071 ceph-mon[44707]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-02T15:20:49.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:48 smithi071 ceph-mon[44707]: from='client.? 172.21.15.71:0/1942204038' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-02T15:20:49.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:48 smithi142 ceph-mon[45728]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-02T15:20:49.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:48 smithi142 ceph-mon[45728]: from='client.? 172.21.15.71:0/1942204038' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-02T15:20:49.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:48 smithi001 ceph-mon[35470]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-02T15:20:49.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:48 smithi001 ceph-mon[35470]: from='client.? 172.21.15.71:0/1942204038' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-02T15:20:51.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:50 smithi142 ceph-mon[45728]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-02T15:20:51.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:50 smithi001 ceph-mon[35470]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-02T15:20:51.341 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:50 smithi071 ceph-mon[44707]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-02T15:20:53.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:52 smithi071 ceph-mon[44707]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-02T15:20:53.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:52 smithi142 ceph-mon[45728]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-02T15:20:53.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:52 smithi001 ceph-mon[35470]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-02T15:20:54.112 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:53 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-02T15:20:54.112 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:53 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:20:54.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:53 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-02T15:20:54.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:53 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:20:54.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:53 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-02T15:20:54.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:53 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:20:55.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:54 smithi071 ceph-mon[44707]: Deploying daemon osd.1 on smithi071 2024-02-02T15:20:55.107 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:54 smithi071 ceph-mon[44707]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-02T15:20:55.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:54 smithi142 ceph-mon[45728]: Deploying daemon osd.1 on smithi071 2024-02-02T15:20:55.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:54 smithi142 ceph-mon[45728]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-02T15:20:55.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:54 smithi001 ceph-mon[35470]: Deploying daemon osd.1 on smithi071 2024-02-02T15:20:55.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:54 smithi001 ceph-mon[35470]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-02T15:20:57.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:56 smithi071 ceph-mon[44707]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-02T15:20:57.169 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:56 smithi142 ceph-mon[45728]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-02T15:20:57.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:56 smithi001 ceph-mon[35470]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-02T15:20:58.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:57 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:58.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:57 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:58.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:57 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:20:58.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:57 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:58.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:57 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:58.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:57 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:20:58.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:57 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:58.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:57 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:58.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:57 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:20:59.284 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:20:59.285+0000 7f84f37fe700 1 -- 172.21.15.71:0/1122742598 <== mgr.14154 v2:172.21.15.1:6800/3768989981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f8504061180 con 0x7f84ec0a0490 2024-02-02T15:20:59.284 INFO:teuthology.orchestra.run.smithi071.stdout:Created osd(s) 1 on host 'smithi071' 2024-02-02T15:20:59.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:20:59.287+0000 7f8508b23700 1 -- 172.21.15.71:0/1122742598 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f84ec0a0490 msgr2=0x7f84ec0a2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:20:59.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:20:59.287+0000 7f8508b23700 1 --2- 172.21.15.71:0/1122742598 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f84ec0a0490 0x7f84ec0a2950 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8504100d30 tx=0x7f84f4009190 comp rx=0 tx=0).stop 2024-02-02T15:20:59.286 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:20:59.287+0000 7f8508b23700 1 -- 172.21.15.71:0/1122742598 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8504100ed0 msgr2=0x7f8504196000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:20:59.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:20:59.287+0000 7f8508b23700 1 --2- 172.21.15.71:0/1122742598 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8504100ed0 0x7f8504196000 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f84f800c010 tx=0x7f84f8004730 comp rx=0 tx=0).stop 2024-02-02T15:20:59.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:20:59.287+0000 7f8508b23700 1 -- 172.21.15.71:0/1122742598 shutdown_connections 2024-02-02T15:20:59.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:20:59.287+0000 7f8508b23700 1 --2- 172.21.15.71:0/1122742598 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8504101870 0x7f850419d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:20:59.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:20:59.287+0000 7f8508b23700 1 --2- 172.21.15.71:0/1122742598 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f84ec0a0490 0x7f84ec0a2950 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:20:59.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:20:59.287+0000 7f8508b23700 1 --2- 172.21.15.71:0/1122742598 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85040ffcd0 0x7f8504195ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:20:59.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:20:59.287+0000 7f8508b23700 1 --2- 172.21.15.71:0/1122742598 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8504100ed0 0x7f8504196000 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:20:59.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:20:59.287+0000 7f8508b23700 1 -- 172.21.15.71:0/1122742598 >> 172.21.15.71:0/1122742598 conn(0x7f85040fb250 msgr2=0x7f8504104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:20:59.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:20:59.287+0000 7f8508b23700 1 -- 172.21.15.71:0/1122742598 shutdown_connections 2024-02-02T15:20:59.287 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:20:59.287+0000 7f8508b23700 1 -- 172.21.15.71:0/1122742598 wait complete. 2024-02-02T15:20:59.309 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:58 smithi071 ceph-mon[44707]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-02T15:20:59.309 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:58 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:59.309 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:58 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:59.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:58 smithi142 ceph-mon[45728]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-02T15:20:59.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:58 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:59.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:58 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:59.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:58 smithi001 ceph-mon[35470]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-02T15:20:59.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:58 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:59.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:58 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:20:59.782 DEBUG:teuthology.orchestra.run.smithi071:osd.1> sudo journalctl -f -n 0 -u ceph-371a409c-c1de-11ee-95b6-87774f69a715@osd.1.service 2024-02-02T15:20:59.784 INFO:tasks.cephadm:Deploying osd.2 on smithi142 with /dev/vg_nvme/lv_4... 2024-02-02T15:20:59.784 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-02-02T15:20:59.813 INFO:journalctl@ceph.osd.1.smithi071.stdout:-- Logs begin at Fri 2024-02-02 14:58:45 UTC. -- 2024-02-02T15:21:00.011 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.c/config 2024-02-02T15:21:00.156 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:59 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:21:00.156 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:59 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:21:00.156 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:59 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:00.156 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:59 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:00.156 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:59 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:21:00.156 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:20:59 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:00.169 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:59 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:21:00.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:59 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:21:00.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:59 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:00.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:59 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:00.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:59 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:21:00.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:20:59 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:00.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:59 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:21:00.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:59 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:21:00.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:59 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:00.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:59 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:00.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:59 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:21:00.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:20:59 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:01.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:00 smithi142 ceph-mon[45728]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-02T15:21:01.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:00 smithi142 ceph-mon[45728]: from='osd.1 [v2:172.21.15.71:6800/747581719,v1:172.21.15.71:6801/747581719]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-02T15:21:01.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:00 smithi142 ceph-mon[45728]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-02T15:21:01.259 INFO:journalctl@ceph.osd.1.smithi071.stdout:Feb 02 15:21:00 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-1[49653]: 2024-02-02T15:21:00.948+0000 7fb5fabcc640 -1 osd.1 0 log_to_monitors true 2024-02-02T15:21:01.260 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:00 smithi071 ceph-mon[44707]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-02T15:21:01.260 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:00 smithi071 ceph-mon[44707]: from='osd.1 [v2:172.21.15.71:6800/747581719,v1:172.21.15.71:6801/747581719]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-02T15:21:01.260 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:00 smithi071 ceph-mon[44707]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-02T15:21:01.340 INFO:teuthology.orchestra.run.smithi142.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-02-02T15:21:01.340 INFO:teuthology.orchestra.run.smithi142.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-02-02T15:21:01.340 INFO:teuthology.orchestra.run.smithi142.stderr: stderr: 10+0 records in 2024-02-02T15:21:01.340 INFO:teuthology.orchestra.run.smithi142.stderr:10+0 records out 2024-02-02T15:21:01.341 INFO:teuthology.orchestra.run.smithi142.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113789 s, 922 MB/s 2024-02-02T15:21:01.341 INFO:teuthology.orchestra.run.smithi142.stderr:--> Zapping successful for: 2024-02-02T15:21:01.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:00 smithi001 ceph-mon[35470]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-02T15:21:01.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:00 smithi001 ceph-mon[35470]: from='osd.1 [v2:172.21.15.71:6800/747581719,v1:172.21.15.71:6801/747581719]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-02T15:21:01.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:00 smithi001 ceph-mon[35470]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-02T15:21:01.775 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph orch daemon add osd smithi142:vg_nvme/lv_4 2024-02-02T15:21:01.994 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.c/config 2024-02-02T15:21:02.696 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:02 smithi142 ceph-mon[45728]: Detected new or changed devices on smithi071 2024-02-02T15:21:02.696 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:02 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:02.696 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:02 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:02.696 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:02 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:21:02.697 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:02 smithi142 ceph-mon[45728]: Adjusting osd_memory_target on smithi071 to 17113M 2024-02-02T15:21:02.697 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:02 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:02.697 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:02 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:21:02.697 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:02 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:21:02.697 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:02 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:02.697 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:02 smithi142 ceph-mon[45728]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-02T15:21:02.697 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:02 smithi142 ceph-mon[45728]: osdmap e11: 2 total, 1 up, 2 in 2024-02-02T15:21:02.697 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:02 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-02T15:21:02.697 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:02 smithi142 ceph-mon[45728]: from='osd.1 [v2:172.21.15.71:6800/747581719,v1:172.21.15.71:6801/747581719]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-02-02T15:21:02.697 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:02 smithi142 ceph-mon[45728]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-02-02T15:21:02.697 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:02 smithi142 ceph-mon[45728]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-02T15:21:02.863 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:02.863+0000 7f09e12bb700 1 -- 172.21.15.142:0/3262627971 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09dc0ffce0 msgr2=0x7f09dc100100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:21:02.863 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:02.863+0000 7f09e12bb700 1 --2- 172.21.15.142:0/3262627971 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09dc0ffce0 0x7f09dc100100 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f09cc009990 tx=0x7f09cc009ca0 comp rx=0 tx=0).stop 2024-02-02T15:21:02.863 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:02.863+0000 7f09e12bb700 1 -- 172.21.15.142:0/3262627971 shutdown_connections 2024-02-02T15:21:02.863 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:02.863+0000 7f09e12bb700 1 --2- 172.21.15.142:0/3262627971 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f09dc101880 0x7f09dc108130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:21:02.864 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:02.863+0000 7f09e12bb700 1 --2- 172.21.15.142:0/3262627971 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09dc0ffce0 0x7f09dc100100 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:21:02.864 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:02.863+0000 7f09e12bb700 1 --2- 172.21.15.142:0/3262627971 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f09dc100ee0 0x7f09dc101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:21:02.864 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:02.863+0000 7f09e12bb700 1 -- 172.21.15.142:0/3262627971 >> 172.21.15.142:0/3262627971 conn(0x7f09dc0fb280 msgr2=0x7f09dc0fd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:21:02.864 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:02.863+0000 7f09e12bb700 1 -- 172.21.15.142:0/3262627971 shutdown_connections 2024-02-02T15:21:02.864 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:02.863+0000 7f09e12bb700 1 -- 172.21.15.142:0/3262627971 wait complete. 2024-02-02T15:21:02.866 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:02.864+0000 7f09e12bb700 1 Processor -- start 2024-02-02T15:21:02.866 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:02.864+0000 7f09e12bb700 1 -- start start 2024-02-02T15:21:02.866 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:02.864+0000 7f09e12bb700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09dc0ffce0 0x7f09dc069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:21:02.866 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:02.864+0000 7f09e12bb700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f09dc100ee0 0x7f09dc19b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:21:02.866 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:02.864+0000 7f09e12bb700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f09dc101880 0x7f09dc19de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:21:02.866 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:02.864+0000 7f09e12bb700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09dc19e600 con 0x7f09dc100ee0 2024-02-02T15:21:02.866 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:02.864+0000 7f09e12bb700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09dc19e770 con 0x7f09dc101880 2024-02-02T15:21:02.867 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:02.864+0000 7f09e12bb700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09dc19e930 con 0x7f09dc0ffce0 2024-02-02T15:21:02.867 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:02.864+0000 7f09daffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09dc0ffce0 0x7f09dc069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:21:02.867 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:02.864+0000 7f09daffd700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09dc0ffce0 0x7f09dc069ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:52698/0 (socket says 172.21.15.142:52698) 2024-02-02T15:21:02.867 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:02.864+0000 7f09daffd700 1 -- 172.21.15.142:0/397585019 learned_addr learned my addr 172.21.15.142:0/397585019 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-02-02T15:21:02.867 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:02.864+0000 7f09daffd700 1 -- 172.21.15.142:0/397585019 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f09dc101880 msgr2=0x7f09dc19de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:21:02.867 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:02.864+0000 7f09db7fe700 1 --2- 172.21.15.142:0/397585019 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f09dc101880 0x7f09dc19de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:21:02.867 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:02.864+0000 7f09daffd700 1 --2- 172.21.15.142:0/397585019 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f09dc101880 0x7f09dc19de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:21:02.867 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:02.864+0000 7f09daffd700 1 -- 172.21.15.142:0/397585019 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f09dc100ee0 msgr2=0x7f09dc19b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:21:02.867 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:02.864+0000 7f09daffd700 1 --2- 172.21.15.142:0/397585019 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f09dc100ee0 0x7f09dc19b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:21:02.867 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:02.864+0000 7f09daffd700 1 -- 172.21.15.142:0/397585019 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09cc0095f0 con 0x7f09dc0ffce0 2024-02-02T15:21:02.867 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:02.865+0000 7f09db7fe700 1 --2- 172.21.15.142:0/397585019 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f09dc101880 0x7f09dc19de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:21:02.867 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:02.865+0000 7f09daffd700 1 --2- 172.21.15.142:0/397585019 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09dc0ffce0 0x7f09dc069ce0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f09cc005b40 tx=0x7f09cc00f740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:21:02.867 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:02.865+0000 7f09d3fff700 1 -- 172.21.15.142:0/397585019 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09cc01d070 con 0x7f09dc0ffce0 2024-02-02T15:21:02.868 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:02.865+0000 7f09d3fff700 1 -- 172.21.15.142:0/397585019 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f09cc003e20 con 0x7f09dc0ffce0 2024-02-02T15:21:02.868 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:02.865+0000 7f09d3fff700 1 -- 172.21.15.142:0/397585019 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09cc0173f0 con 0x7f09dc0ffce0 2024-02-02T15:21:02.868 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:02.865+0000 7f09e12bb700 1 -- 172.21.15.142:0/397585019 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09dc19ebb0 con 0x7f09dc0ffce0 2024-02-02T15:21:02.868 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:02.865+0000 7f09e12bb700 1 -- 172.21.15.142:0/397585019 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09dc19ef70 con 0x7f09dc0ffce0 2024-02-02T15:21:02.868 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:02.866+0000 7f09e12bb700 1 -- 172.21.15.142:0/397585019 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09dc106c70 con 0x7f09dc0ffce0 2024-02-02T15:21:02.869 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:02.867+0000 7f09d3fff700 1 -- 172.21.15.142:0/397585019 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 15) v1 ==== 135013+0+0 (secure 0 0 0) 0x7f09cc003960 con 0x7f09dc0ffce0 2024-02-02T15:21:02.869 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:02.867+0000 7f09d3fff700 1 --2- 172.21.15.142:0/397585019 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f09c80a0490 0x7f09c80a2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:21:02.869 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:02.868+0000 7f09d3fff700 1 -- 172.21.15.142:0/397585019 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2029+0+0 (secure 0 0 0) 0x7f09cc0cb380 con 0x7f09dc0ffce0 2024-02-02T15:21:02.869 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:02.868+0000 7f09da7fc700 1 --2- 172.21.15.142:0/397585019 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f09c80a0490 0x7f09c80a2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:21:02.869 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:02.868+0000 7f09da7fc700 1 --2- 172.21.15.142:0/397585019 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f09c80a0490 0x7f09c80a2950 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f09c4005950 tx=0x7f09c40058e0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:21:02.871 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:02.871+0000 7f09d3fff700 1 -- 172.21.15.142:0/397585019 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f09cc096920 con 0x7f09dc0ffce0 2024-02-02T15:21:02.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:02 smithi001 ceph-mon[35470]: Detected new or changed devices on smithi071 2024-02-02T15:21:02.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:02 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:02.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:02 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:02.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:02 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:21:02.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:02 smithi001 ceph-mon[35470]: Adjusting osd_memory_target on smithi071 to 17113M 2024-02-02T15:21:02.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:02 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:02.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:02 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:21:02.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:02 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:21:02.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:02 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:02.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:02 smithi001 ceph-mon[35470]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-02T15:21:02.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:02 smithi001 ceph-mon[35470]: osdmap e11: 2 total, 1 up, 2 in 2024-02-02T15:21:02.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:02 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-02T15:21:02.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:02 smithi001 ceph-mon[35470]: from='osd.1 [v2:172.21.15.71:6800/747581719,v1:172.21.15.71:6801/747581719]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-02-02T15:21:02.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:02 smithi001 ceph-mon[35470]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-02-02T15:21:02.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:02 smithi001 ceph-mon[35470]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-02T15:21:03.064 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:03.064+0000 7f09e12bb700 1 -- 172.21.15.142:0/397585019 --> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f09dc061180 con 0x7f09c80a0490 2024-02-02T15:21:03.114 INFO:journalctl@ceph.osd.1.smithi071.stdout:Feb 02 15:21:02 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-1[49653]: 2024-02-02T15:21:02.735+0000 7fb5f0230700 -1 osd.1 0 waiting for initial osdmap 2024-02-02T15:21:03.114 INFO:journalctl@ceph.osd.1.smithi071.stdout:Feb 02 15:21:02 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-1[49653]: 2024-02-02T15:21:02.741+0000 7fb5e801d700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-02T15:21:03.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:02 smithi071 ceph-mon[44707]: Detected new or changed devices on smithi071 2024-02-02T15:21:03.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:02 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:03.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:02 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:03.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:02 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:21:03.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:02 smithi071 ceph-mon[44707]: Adjusting osd_memory_target on smithi071 to 17113M 2024-02-02T15:21:03.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:02 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:03.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:02 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:21:03.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:02 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:21:03.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:02 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:03.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:02 smithi071 ceph-mon[44707]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-02T15:21:03.116 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:02 smithi071 ceph-mon[44707]: osdmap e11: 2 total, 1 up, 2 in 2024-02-02T15:21:03.116 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:02 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-02T15:21:03.116 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:02 smithi071 ceph-mon[44707]: from='osd.1 [v2:172.21.15.71:6800/747581719,v1:172.21.15.71:6801/747581719]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-02-02T15:21:03.116 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:02 smithi071 ceph-mon[44707]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-02-02T15:21:03.116 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:02 smithi071 ceph-mon[44707]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-02T15:21:04.070 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:03 smithi142 ceph-mon[45728]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-02-02T15:21:04.070 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:03 smithi142 ceph-mon[45728]: osdmap e12: 2 total, 1 up, 2 in 2024-02-02T15:21:04.071 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:03 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-02T15:21:04.071 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:03 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-02T15:21:04.071 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:03 smithi142 ceph-mon[45728]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:21:04.071 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:03 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-02T15:21:04.071 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:03 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-02T15:21:04.071 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:03 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:21:04.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:03 smithi071 ceph-mon[44707]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-02-02T15:21:04.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:03 smithi071 ceph-mon[44707]: osdmap e12: 2 total, 1 up, 2 in 2024-02-02T15:21:04.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:03 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-02T15:21:04.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:03 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-02T15:21:04.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:03 smithi071 ceph-mon[44707]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:21:04.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:03 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-02T15:21:04.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:03 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-02T15:21:04.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:03 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:21:04.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:03 smithi001 ceph-mon[35470]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-02-02T15:21:04.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:03 smithi001 ceph-mon[35470]: osdmap e12: 2 total, 1 up, 2 in 2024-02-02T15:21:04.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:03 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-02T15:21:04.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:03 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-02T15:21:04.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:03 smithi001 ceph-mon[35470]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:21:04.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:03 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-02T15:21:04.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:03 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-02T15:21:04.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:03 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:21:05.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:04 smithi071 ceph-mon[44707]: purged_snaps scrub starts 2024-02-02T15:21:05.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:04 smithi071 ceph-mon[44707]: purged_snaps scrub ok 2024-02-02T15:21:05.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:04 smithi071 ceph-mon[44707]: osd.1 [v2:172.21.15.71:6800/747581719,v1:172.21.15.71:6801/747581719] boot 2024-02-02T15:21:05.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:04 smithi071 ceph-mon[44707]: osdmap e13: 2 total, 2 up, 2 in 2024-02-02T15:21:05.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:04 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-02T15:21:05.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:04 smithi071 ceph-mon[44707]: pgmap v52: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-02T15:21:05.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:04 smithi142 ceph-mon[45728]: purged_snaps scrub starts 2024-02-02T15:21:05.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:04 smithi142 ceph-mon[45728]: purged_snaps scrub ok 2024-02-02T15:21:05.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:04 smithi142 ceph-mon[45728]: osd.1 [v2:172.21.15.71:6800/747581719,v1:172.21.15.71:6801/747581719] boot 2024-02-02T15:21:05.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:04 smithi142 ceph-mon[45728]: osdmap e13: 2 total, 2 up, 2 in 2024-02-02T15:21:05.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:04 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-02T15:21:05.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:04 smithi142 ceph-mon[45728]: pgmap v52: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-02T15:21:05.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:04 smithi001 ceph-mon[35470]: purged_snaps scrub starts 2024-02-02T15:21:05.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:04 smithi001 ceph-mon[35470]: purged_snaps scrub ok 2024-02-02T15:21:05.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:04 smithi001 ceph-mon[35470]: osd.1 [v2:172.21.15.71:6800/747581719,v1:172.21.15.71:6801/747581719] boot 2024-02-02T15:21:05.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:04 smithi001 ceph-mon[35470]: osdmap e13: 2 total, 2 up, 2 in 2024-02-02T15:21:05.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:04 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-02T15:21:05.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:04 smithi001 ceph-mon[35470]: pgmap v52: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-02T15:21:06.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:05 smithi071 ceph-mon[44707]: osdmap e14: 2 total, 2 up, 2 in 2024-02-02T15:21:06.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:05 smithi071 ceph-mon[44707]: from='client.? 172.21.15.142:0/1500361360' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "856c2347-294c-482b-8595-ad77f4792db4"}]: dispatch 2024-02-02T15:21:06.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:05 smithi071 ceph-mon[44707]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "856c2347-294c-482b-8595-ad77f4792db4"}]: dispatch 2024-02-02T15:21:06.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:05 smithi071 ceph-mon[44707]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "856c2347-294c-482b-8595-ad77f4792db4"}]': finished 2024-02-02T15:21:06.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:05 smithi071 ceph-mon[44707]: osdmap e15: 3 total, 2 up, 3 in 2024-02-02T15:21:06.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:05 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-02T15:21:06.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:05 smithi071 ceph-mon[44707]: from='client.? 172.21.15.142:0/2282120437' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-02T15:21:06.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:05 smithi142 ceph-mon[45728]: osdmap e14: 2 total, 2 up, 2 in 2024-02-02T15:21:06.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:05 smithi142 ceph-mon[45728]: from='client.? 172.21.15.142:0/1500361360' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "856c2347-294c-482b-8595-ad77f4792db4"}]: dispatch 2024-02-02T15:21:06.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:05 smithi142 ceph-mon[45728]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "856c2347-294c-482b-8595-ad77f4792db4"}]: dispatch 2024-02-02T15:21:06.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:05 smithi142 ceph-mon[45728]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "856c2347-294c-482b-8595-ad77f4792db4"}]': finished 2024-02-02T15:21:06.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:05 smithi142 ceph-mon[45728]: osdmap e15: 3 total, 2 up, 3 in 2024-02-02T15:21:06.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:05 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-02T15:21:06.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:05 smithi142 ceph-mon[45728]: from='client.? 172.21.15.142:0/2282120437' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-02T15:21:06.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:05 smithi001 ceph-mon[35470]: osdmap e14: 2 total, 2 up, 2 in 2024-02-02T15:21:06.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:05 smithi001 ceph-mon[35470]: from='client.? 172.21.15.142:0/1500361360' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "856c2347-294c-482b-8595-ad77f4792db4"}]: dispatch 2024-02-02T15:21:06.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:05 smithi001 ceph-mon[35470]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "856c2347-294c-482b-8595-ad77f4792db4"}]: dispatch 2024-02-02T15:21:06.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:05 smithi001 ceph-mon[35470]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "856c2347-294c-482b-8595-ad77f4792db4"}]': finished 2024-02-02T15:21:06.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:05 smithi001 ceph-mon[35470]: osdmap e15: 3 total, 2 up, 3 in 2024-02-02T15:21:06.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:05 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-02T15:21:06.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:05 smithi001 ceph-mon[35470]: from='client.? 172.21.15.142:0/2282120437' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-02T15:21:07.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:06 smithi071 ceph-mon[44707]: pgmap v55: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-02T15:21:07.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:06 smithi142 ceph-mon[45728]: pgmap v55: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-02T15:21:07.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:06 smithi001 ceph-mon[35470]: pgmap v55: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-02T15:21:09.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:08 smithi142 ceph-mon[45728]: pgmap v56: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-02T15:21:09.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:08 smithi001 ceph-mon[35470]: pgmap v56: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-02T15:21:09.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:08 smithi071 ceph-mon[44707]: pgmap v56: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-02T15:21:11.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:10 smithi142 ceph-mon[45728]: pgmap v57: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-02T15:21:11.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:10 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-02T15:21:11.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:10 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:21:11.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:10 smithi001 ceph-mon[35470]: pgmap v57: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-02T15:21:11.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:10 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-02T15:21:11.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:10 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:21:11.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:10 smithi071 ceph-mon[44707]: pgmap v57: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-02T15:21:11.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:10 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-02T15:21:11.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:10 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:21:12.075 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:11 smithi142 ceph-mon[45728]: Deploying daemon osd.2 on smithi142 2024-02-02T15:21:12.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:11 smithi001 ceph-mon[35470]: Deploying daemon osd.2 on smithi142 2024-02-02T15:21:12.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:11 smithi071 ceph-mon[44707]: Deploying daemon osd.2 on smithi142 2024-02-02T15:21:13.071 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:12 smithi142 ceph-mon[45728]: pgmap v58: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-02T15:21:13.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:12 smithi001 ceph-mon[35470]: pgmap v58: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-02T15:21:13.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:12 smithi071 ceph-mon[44707]: pgmap v58: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-02T15:21:15.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:14 smithi142 ceph-mon[45728]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-02T15:21:15.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:14 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:15.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:14 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:15.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:14 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:21:15.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:14 smithi001 ceph-mon[35470]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-02T15:21:15.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:14 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:15.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:14 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:15.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:14 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:21:15.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:14 smithi071 ceph-mon[44707]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-02T15:21:15.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:14 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:15.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:14 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:15.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:14 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:21:16.623 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:16.623+0000 7f09d3fff700 1 -- 172.21.15.142:0/397585019 <== mgr.14154 v2:172.21.15.1:6800/3768989981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f09dc061180 con 0x7f09c80a0490 2024-02-02T15:21:16.624 INFO:teuthology.orchestra.run.smithi142.stdout:Created osd(s) 2 on host 'smithi142' 2024-02-02T15:21:16.626 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:16.625+0000 7f09e12bb700 1 -- 172.21.15.142:0/397585019 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f09c80a0490 msgr2=0x7f09c80a2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:21:16.626 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:16.625+0000 7f09e12bb700 1 --2- 172.21.15.142:0/397585019 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f09c80a0490 0x7f09c80a2950 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f09c4005950 tx=0x7f09c40058e0 comp rx=0 tx=0).stop 2024-02-02T15:21:16.626 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:16.625+0000 7f09e12bb700 1 -- 172.21.15.142:0/397585019 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09dc0ffce0 msgr2=0x7f09dc069ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:21:16.626 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:16.626+0000 7f09e12bb700 1 --2- 172.21.15.142:0/397585019 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09dc0ffce0 0x7f09dc069ce0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f09cc005b40 tx=0x7f09cc00f740 comp rx=0 tx=0).stop 2024-02-02T15:21:16.626 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:16.626+0000 7f09e12bb700 1 -- 172.21.15.142:0/397585019 shutdown_connections 2024-02-02T15:21:16.626 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:16.626+0000 7f09e12bb700 1 --2- 172.21.15.142:0/397585019 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f09dc101880 0x7f09dc19de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:21:16.626 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:16.626+0000 7f09e12bb700 1 --2- 172.21.15.142:0/397585019 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f09c80a0490 0x7f09c80a2950 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:21:16.626 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:16.626+0000 7f09e12bb700 1 --2- 172.21.15.142:0/397585019 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09dc0ffce0 0x7f09dc069ce0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:21:16.626 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:16.626+0000 7f09e12bb700 1 --2- 172.21.15.142:0/397585019 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f09dc100ee0 0x7f09dc19b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:21:16.626 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:16.626+0000 7f09e12bb700 1 -- 172.21.15.142:0/397585019 >> 172.21.15.142:0/397585019 conn(0x7f09dc0fb280 msgr2=0x7f09dc0fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:21:16.627 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:16.626+0000 7f09e12bb700 1 -- 172.21.15.142:0/397585019 shutdown_connections 2024-02-02T15:21:16.627 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:16.626+0000 7f09e12bb700 1 -- 172.21.15.142:0/397585019 wait complete. 2024-02-02T15:21:16.801 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:16 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:16.802 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:16 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:16.802 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:16 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:21:16.802 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:16 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:21:16.802 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:16 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:16.802 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:16 smithi142 ceph-mon[45728]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-02T15:21:16.802 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:16 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:16.802 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:16 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:16.802 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:16 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:21:16.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:16 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:16.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:16 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:16.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:16 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:21:16.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:16 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:21:16.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:16 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:16.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:16 smithi001 ceph-mon[35470]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-02T15:21:16.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:16 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:16.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:16 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:16.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:16 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:21:17.027 DEBUG:teuthology.orchestra.run.smithi142:osd.2> sudo journalctl -f -n 0 -u ceph-371a409c-c1de-11ee-95b6-87774f69a715@osd.2.service 2024-02-02T15:21:17.029 INFO:tasks.cephadm:Deploying osd.3 on smithi001 with /dev/vg_nvme/lv_3... 2024-02-02T15:21:17.029 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-02-02T15:21:17.061 INFO:journalctl@ceph.osd.2.smithi142.stdout:-- Logs begin at Fri 2024-02-02 14:58:43 UTC. -- 2024-02-02T15:21:17.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:16 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:17.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:16 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:17.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:16 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:21:17.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:16 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:21:17.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:16 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:17.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:16 smithi071 ceph-mon[44707]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-02T15:21:17.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:16 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:17.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:16 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:17.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:16 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:21:17.400 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.a/config 2024-02-02T15:21:18.670 INFO:journalctl@ceph.osd.2.smithi142.stdout:Feb 02 15:21:18 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-2[51513]: 2024-02-02T15:21:18.339+0000 7f3d39bab640 -1 osd.2 0 log_to_monitors true 2024-02-02T15:21:18.916 INFO:teuthology.orchestra.run.smithi001.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-02-02T15:21:18.916 INFO:teuthology.orchestra.run.smithi001.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-02-02T15:21:18.917 INFO:teuthology.orchestra.run.smithi001.stderr: stderr: 10+0 records in 2024-02-02T15:21:18.917 INFO:teuthology.orchestra.run.smithi001.stderr:10+0 records out 2024-02-02T15:21:18.917 INFO:teuthology.orchestra.run.smithi001.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0475828 s, 220 MB/s 2024-02-02T15:21:18.917 INFO:teuthology.orchestra.run.smithi001.stderr:--> Zapping successful for: 2024-02-02T15:21:19.004 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:18 smithi142 ceph-mon[45728]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-02T15:21:19.004 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:18 smithi142 ceph-mon[45728]: from='osd.2 [v2:172.21.15.142:6800/377511414,v1:172.21.15.142:6801/377511414]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-02T15:21:19.004 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:18 smithi142 ceph-mon[45728]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-02T15:21:19.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:18 smithi001 ceph-mon[35470]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-02T15:21:19.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:18 smithi001 ceph-mon[35470]: from='osd.2 [v2:172.21.15.142:6800/377511414,v1:172.21.15.142:6801/377511414]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-02T15:21:19.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:18 smithi001 ceph-mon[35470]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-02T15:21:19.299 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph orch daemon add osd smithi001:vg_nvme/lv_3 2024-02-02T15:21:19.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:18 smithi071 ceph-mon[44707]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-02T15:21:19.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:18 smithi071 ceph-mon[44707]: from='osd.2 [v2:172.21.15.142:6800/377511414,v1:172.21.15.142:6801/377511414]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-02T15:21:19.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:18 smithi071 ceph-mon[44707]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-02T15:21:19.659 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.a/config 2024-02-02T15:21:20.090 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:19 smithi001 ceph-mon[35470]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-02T15:21:20.090 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:19 smithi001 ceph-mon[35470]: osdmap e16: 3 total, 2 up, 3 in 2024-02-02T15:21:20.090 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:19 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-02T15:21:20.090 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:19 smithi001 ceph-mon[35470]: from='osd.2 [v2:172.21.15.142:6800/377511414,v1:172.21.15.142:6801/377511414]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-02-02T15:21:20.090 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:19 smithi001 ceph-mon[35470]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-02-02T15:21:20.090 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:19 smithi001 ceph-mon[35470]: Detected new or changed devices on smithi142 2024-02-02T15:21:20.090 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:19 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:20.090 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:19 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:20.090 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:19 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:21:20.090 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:19 smithi001 ceph-mon[35470]: Adjusting osd_memory_target on smithi142 to 17113M 2024-02-02T15:21:20.090 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:19 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:20.090 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:19 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:21:20.091 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:19 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:21:20.091 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:19 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:20.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:19 smithi071 ceph-mon[44707]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-02T15:21:20.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:19 smithi071 ceph-mon[44707]: osdmap e16: 3 total, 2 up, 3 in 2024-02-02T15:21:20.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:19 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-02T15:21:20.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:19 smithi071 ceph-mon[44707]: from='osd.2 [v2:172.21.15.142:6800/377511414,v1:172.21.15.142:6801/377511414]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-02-02T15:21:20.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:19 smithi071 ceph-mon[44707]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-02-02T15:21:20.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:19 smithi071 ceph-mon[44707]: Detected new or changed devices on smithi142 2024-02-02T15:21:20.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:19 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:20.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:19 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:20.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:19 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:21:20.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:19 smithi071 ceph-mon[44707]: Adjusting osd_memory_target on smithi142 to 17113M 2024-02-02T15:21:20.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:19 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:20.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:19 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:21:20.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:19 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:21:20.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:19 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:20.420 INFO:journalctl@ceph.osd.2.smithi142.stdout:Feb 02 15:21:19 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-2[51513]: 2024-02-02T15:21:19.930+0000 7f3d2f20f700 -1 osd.2 0 waiting for initial osdmap 2024-02-02T15:21:20.420 INFO:journalctl@ceph.osd.2.smithi142.stdout:Feb 02 15:21:19 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-2[51513]: 2024-02-02T15:21:19.934+0000 7f3d29000700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-02T15:21:20.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:19 smithi142 ceph-mon[45728]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-02T15:21:20.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:19 smithi142 ceph-mon[45728]: osdmap e16: 3 total, 2 up, 3 in 2024-02-02T15:21:20.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:19 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-02T15:21:20.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:19 smithi142 ceph-mon[45728]: from='osd.2 [v2:172.21.15.142:6800/377511414,v1:172.21.15.142:6801/377511414]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-02-02T15:21:20.421 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:19 smithi142 ceph-mon[45728]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-02-02T15:21:20.421 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:19 smithi142 ceph-mon[45728]: Detected new or changed devices on smithi142 2024-02-02T15:21:20.421 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:19 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:20.421 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:19 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:20.421 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:19 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:21:20.421 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:19 smithi142 ceph-mon[45728]: Adjusting osd_memory_target on smithi142 to 17113M 2024-02-02T15:21:20.421 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:19 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:20.421 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:19 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:21:20.421 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:19 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:21:20.421 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:19 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:20.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:21:20.718+0000 7ff5fea00700 1 -- 172.21.15.1:0/1889404765 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff5f8101860 msgr2=0x7ff5f8103c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:21:20.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:21:20.718+0000 7ff5fea00700 1 --2- 172.21.15.1:0/1889404765 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff5f8101860 0x7ff5f8103c50 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff5e80098f0 tx=0x7ff5e8009c00 comp rx=0 tx=0).stop 2024-02-02T15:21:20.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:21:20.718+0000 7ff5fea00700 1 -- 172.21.15.1:0/1889404765 shutdown_connections 2024-02-02T15:21:20.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:21:20.718+0000 7ff5fea00700 1 --2- 172.21.15.1:0/1889404765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5f8106ac0 0x7ff5f8108f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:21:20.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:21:20.718+0000 7ff5fea00700 1 --2- 172.21.15.1:0/1889404765 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff5f8101860 0x7ff5f8103c50 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:21:20.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:21:20.718+0000 7ff5fea00700 1 --2- 172.21.15.1:0/1889404765 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5f8104190 0x7ff5f8106580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:21:20.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:21:20.718+0000 7ff5fea00700 1 -- 172.21.15.1:0/1889404765 >> 172.21.15.1:0/1889404765 conn(0x7ff5f80fb250 msgr2=0x7ff5f80fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:21:20.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:21:20.718+0000 7ff5fea00700 1 -- 172.21.15.1:0/1889404765 shutdown_connections 2024-02-02T15:21:20.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:21:20.718+0000 7ff5fea00700 1 -- 172.21.15.1:0/1889404765 wait complete. 2024-02-02T15:21:20.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:21:20.719+0000 7ff5fea00700 1 Processor -- start 2024-02-02T15:21:20.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:21:20.719+0000 7ff5fea00700 1 -- start start 2024-02-02T15:21:20.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:21:20.720+0000 7ff5fea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5f8101860 0x7ff5f8198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:21:20.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:21:20.720+0000 7ff5fea00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5f8104190 0x7ff5f8199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:21:20.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:21:20.720+0000 7ff5fea00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff5f8106ac0 0x7ff5f81a02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:21:20.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:21:20.720+0000 7ff5fea00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f81a0870 con 0x7ff5f8106ac0 2024-02-02T15:21:20.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:21:20.720+0000 7ff5fea00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f81a09b0 con 0x7ff5f8104190 2024-02-02T15:21:20.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:21:20.720+0000 7ff5fea00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f81a0b70 con 0x7ff5f8101860 2024-02-02T15:21:20.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:21:20.720+0000 7ff5f7e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5f8101860 0x7ff5f8198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:21:20.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:21:20.720+0000 7ff5f7400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5f8104190 0x7ff5f8199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:21:20.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:21:20.720+0000 7ff5f7e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5f8101860 0x7ff5f8198d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:57298/0 (socket says 172.21.15.1:57298) 2024-02-02T15:21:20.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:21:20.720+0000 7ff5f7e00700 1 -- 172.21.15.1:0/2988839503 learned_addr learned my addr 172.21.15.1:0/2988839503 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:21:20.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:21:20.720+0000 7ff5fca00700 1 --2- 172.21.15.1:0/2988839503 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff5f8106ac0 0x7ff5f81a02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:21:20.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:21:20.720+0000 7ff5f7400700 1 -- 172.21.15.1:0/2988839503 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5f8101860 msgr2=0x7ff5f8198d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:21:20.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:21:20.720+0000 7ff5f7400700 1 --2- 172.21.15.1:0/2988839503 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5f8101860 0x7ff5f8198d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:21:20.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:21:20.720+0000 7ff5f7400700 1 -- 172.21.15.1:0/2988839503 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff5f8106ac0 msgr2=0x7ff5f81a02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:21:20.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:21:20.720+0000 7ff5f7400700 1 --2- 172.21.15.1:0/2988839503 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff5f8106ac0 0x7ff5f81a02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:21:20.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:21:20.720+0000 7ff5f7400700 1 -- 172.21.15.1:0/2988839503 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5e8009520 con 0x7ff5f8104190 2024-02-02T15:21:20.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:21:20.720+0000 7ff5f7e00700 1 --2- 172.21.15.1:0/2988839503 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5f8101860 0x7ff5f8198d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:21:20.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:21:20.720+0000 7ff5fca00700 1 --2- 172.21.15.1:0/2988839503 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff5f8106ac0 0x7ff5f81a02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:21:20.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:21:20.721+0000 7ff5f7400700 1 --2- 172.21.15.1:0/2988839503 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5f8104190 0x7ff5f8199260 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff5e000e9c0 tx=0x7ff5e000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:21:20.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:21:20.721+0000 7ff5f4c00700 1 -- 172.21.15.1:0/2988839503 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e000c9c0 con 0x7ff5f8104190 2024-02-02T15:21:20.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:21:20.721+0000 7ff5fea00700 1 -- 172.21.15.1:0/2988839503 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5f81a0e50 con 0x7ff5f8104190 2024-02-02T15:21:20.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:21:20.721+0000 7ff5f4c00700 1 -- 172.21.15.1:0/2988839503 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff5e0010430 con 0x7ff5f8104190 2024-02-02T15:21:20.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:21:20.721+0000 7ff5f4c00700 1 -- 172.21.15.1:0/2988839503 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e0018680 con 0x7ff5f8104190 2024-02-02T15:21:20.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:21:20.721+0000 7ff5fea00700 1 -- 172.21.15.1:0/2988839503 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5f81a12c0 con 0x7ff5f8104190 2024-02-02T15:21:20.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:21:20.722+0000 7ff5fea00700 1 -- 172.21.15.1:0/2988839503 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5f8108690 con 0x7ff5f8104190 2024-02-02T15:21:20.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:21:20.724+0000 7ff5f4c00700 1 -- 172.21.15.1:0/2988839503 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 15) v1 ==== 135013+0+0 (secure 0 0 0) 0x7ff5e0010860 con 0x7ff5f8104190 2024-02-02T15:21:20.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:21:20.725+0000 7ff5f4c00700 1 --2- 172.21.15.1:0/2988839503 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7ff5d80a0510 0x7ff5d80a29d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:21:20.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:21:20.726+0000 7ff5f4c00700 1 -- 172.21.15.1:0/2988839503 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 2571+0+0 (secure 0 0 0) 0x7ff5e0014070 con 0x7ff5f8104190 2024-02-02T15:21:20.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:21:20.726+0000 7ff5f7e00700 1 --2- 172.21.15.1:0/2988839503 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7ff5d80a0510 0x7ff5d80a29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:21:20.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:21:20.726+0000 7ff5f7e00700 1 --2- 172.21.15.1:0/2988839503 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7ff5d80a0510 0x7ff5d80a29d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff5e800c010 tx=0x7ff5e8019040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:21:20.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:21:20.730+0000 7ff5f4c00700 1 -- 172.21.15.1:0/2988839503 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7ff5e0052110 con 0x7ff5f8104190 2024-02-02T15:21:20.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:21:20.949+0000 7ff5fea00700 1 -- 172.21.15.1:0/2988839503 --> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7ff5f8061180 con 0x7ff5d80a0510 2024-02-02T15:21:21.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:20 smithi001 ceph-mon[35470]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-02T15:21:21.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:20 smithi001 ceph-mon[35470]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-02-02T15:21:21.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:20 smithi001 ceph-mon[35470]: osdmap e17: 3 total, 2 up, 3 in 2024-02-02T15:21:21.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:20 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-02T15:21:21.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:20 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-02T15:21:21.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:20 smithi071 ceph-mon[44707]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-02T15:21:21.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:20 smithi071 ceph-mon[44707]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-02-02T15:21:21.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:20 smithi071 ceph-mon[44707]: osdmap e17: 3 total, 2 up, 3 in 2024-02-02T15:21:21.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:20 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-02T15:21:21.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:20 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-02T15:21:21.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:20 smithi142 ceph-mon[45728]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-02T15:21:21.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:20 smithi142 ceph-mon[45728]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-02-02T15:21:21.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:20 smithi142 ceph-mon[45728]: osdmap e17: 3 total, 2 up, 3 in 2024-02-02T15:21:21.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:20 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-02T15:21:21.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:20 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-02T15:21:22.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:22 smithi001 ceph-mon[35470]: purged_snaps scrub starts 2024-02-02T15:21:22.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:22 smithi001 ceph-mon[35470]: purged_snaps scrub ok 2024-02-02T15:21:22.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:22 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-02T15:21:22.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:22 smithi001 ceph-mon[35470]: osd.2 [v2:172.21.15.142:6800/377511414,v1:172.21.15.142:6801/377511414] boot 2024-02-02T15:21:22.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:22 smithi001 ceph-mon[35470]: osdmap e18: 3 total, 3 up, 3 in 2024-02-02T15:21:22.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:22 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-02T15:21:22.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:22 smithi001 ceph-mon[35470]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:21:22.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:22 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-02T15:21:22.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:22 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-02T15:21:22.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:22 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:21:22.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:22 smithi071 ceph-mon[44707]: purged_snaps scrub starts 2024-02-02T15:21:22.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:22 smithi071 ceph-mon[44707]: purged_snaps scrub ok 2024-02-02T15:21:22.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:22 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-02T15:21:22.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:22 smithi071 ceph-mon[44707]: osd.2 [v2:172.21.15.142:6800/377511414,v1:172.21.15.142:6801/377511414] boot 2024-02-02T15:21:22.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:22 smithi071 ceph-mon[44707]: osdmap e18: 3 total, 3 up, 3 in 2024-02-02T15:21:22.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:22 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-02T15:21:22.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:22 smithi071 ceph-mon[44707]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:21:22.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:22 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-02T15:21:22.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:22 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-02T15:21:22.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:22 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:21:22.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:22 smithi142 ceph-mon[45728]: purged_snaps scrub starts 2024-02-02T15:21:22.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:22 smithi142 ceph-mon[45728]: purged_snaps scrub ok 2024-02-02T15:21:22.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:22 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-02T15:21:22.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:22 smithi142 ceph-mon[45728]: osd.2 [v2:172.21.15.142:6800/377511414,v1:172.21.15.142:6801/377511414] boot 2024-02-02T15:21:22.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:22 smithi142 ceph-mon[45728]: osdmap e18: 3 total, 3 up, 3 in 2024-02-02T15:21:22.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:22 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-02T15:21:22.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:22 smithi142 ceph-mon[45728]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:21:22.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:22 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-02T15:21:22.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:22 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-02T15:21:22.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:22 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:21:23.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:23 smithi071 ceph-mon[44707]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-02T15:21:23.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:23 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-02-02T15:21:23.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:23 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-02-02T15:21:23.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:23 smithi071 ceph-mon[44707]: osdmap e19: 3 total, 3 up, 3 in 2024-02-02T15:21:23.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:23 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-02T15:21:23.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:23 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/2466736263' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4b946ad2-2da9-4e0b-9d61-d039d8669a42"}]: dispatch 2024-02-02T15:21:23.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:23 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-02T15:21:23.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:23 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/2466736263' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4b946ad2-2da9-4e0b-9d61-d039d8669a42"}]': finished 2024-02-02T15:21:23.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:23 smithi071 ceph-mon[44707]: osdmap e20: 4 total, 3 up, 4 in 2024-02-02T15:21:23.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:23 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-02T15:21:23.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:23 smithi142 ceph-mon[45728]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-02T15:21:23.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:23 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-02-02T15:21:23.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:23 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-02-02T15:21:23.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:23 smithi142 ceph-mon[45728]: osdmap e19: 3 total, 3 up, 3 in 2024-02-02T15:21:23.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:23 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-02T15:21:23.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:23 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/2466736263' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4b946ad2-2da9-4e0b-9d61-d039d8669a42"}]: dispatch 2024-02-02T15:21:23.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:23 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-02T15:21:23.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:23 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/2466736263' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4b946ad2-2da9-4e0b-9d61-d039d8669a42"}]': finished 2024-02-02T15:21:23.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:23 smithi142 ceph-mon[45728]: osdmap e20: 4 total, 3 up, 4 in 2024-02-02T15:21:23.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:23 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-02T15:21:23.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:23 smithi001 ceph-mon[35470]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-02T15:21:23.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:23 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-02-02T15:21:23.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:23 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-02-02T15:21:23.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:23 smithi001 ceph-mon[35470]: osdmap e19: 3 total, 3 up, 3 in 2024-02-02T15:21:23.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:23 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-02T15:21:23.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:23 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/2466736263' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4b946ad2-2da9-4e0b-9d61-d039d8669a42"}]: dispatch 2024-02-02T15:21:23.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:23 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-02T15:21:23.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:23 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/2466736263' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4b946ad2-2da9-4e0b-9d61-d039d8669a42"}]': finished 2024-02-02T15:21:23.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:23 smithi001 ceph-mon[35470]: osdmap e20: 4 total, 3 up, 4 in 2024-02-02T15:21:23.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:23 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-02T15:21:24.752 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:24 smithi001 ceph-mon[35470]: pgmap v69: 1 pgs: 1 creating+activating; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-02T15:21:24.753 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:24 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/1765223215' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-02T15:21:24.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:24 smithi071 ceph-mon[44707]: pgmap v69: 1 pgs: 1 creating+activating; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-02T15:21:24.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:24 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/1765223215' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-02T15:21:24.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:24 smithi142 ceph-mon[45728]: pgmap v69: 1 pgs: 1 creating+activating; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-02T15:21:24.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:24 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/1765223215' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-02T15:21:25.190 INFO:journalctl@ceph.osd.0.smithi001.stdout:Feb 02 15:21:24 smithi001 sudo[59115]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-02T15:21:25.190 INFO:journalctl@ceph.osd.0.smithi001.stdout:Feb 02 15:21:24 smithi001 sudo[59115]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-02T15:21:25.190 INFO:journalctl@ceph.osd.0.smithi001.stdout:Feb 02 15:21:24 smithi001 sudo[59115]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-02T15:21:25.190 INFO:journalctl@ceph.osd.0.smithi001.stdout:Feb 02 15:21:24 smithi001 sudo[59115]: pam_unix(sudo:session): session closed for user root 2024-02-02T15:21:25.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:25 smithi001 ceph-mon[35470]: osdmap e21: 4 total, 3 up, 4 in 2024-02-02T15:21:25.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:25 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-02T15:21:25.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:25 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:25.690 INFO:journalctl@ceph.osd.0.smithi001.stdout:Feb 02 15:21:25 smithi001 sudo[59488]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-02T15:21:25.690 INFO:journalctl@ceph.osd.0.smithi001.stdout:Feb 02 15:21:25 smithi001 sudo[59488]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-02T15:21:25.691 INFO:journalctl@ceph.osd.0.smithi001.stdout:Feb 02 15:21:25 smithi001 sudo[59488]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-02T15:21:25.691 INFO:journalctl@ceph.osd.0.smithi001.stdout:Feb 02 15:21:25 smithi001 sudo[59488]: pam_unix(sudo:session): session closed for user root 2024-02-02T15:21:25.860 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:25 smithi071 ceph-mon[44707]: osdmap e21: 4 total, 3 up, 4 in 2024-02-02T15:21:25.861 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:25 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-02T15:21:25.861 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:25 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:25.861 INFO:journalctl@ceph.osd.1.smithi071.stdout:Feb 02 15:21:25 smithi071 sudo[54459]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-02T15:21:25.861 INFO:journalctl@ceph.osd.1.smithi071.stdout:Feb 02 15:21:25 smithi071 sudo[54459]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-02T15:21:25.861 INFO:journalctl@ceph.osd.1.smithi071.stdout:Feb 02 15:21:25 smithi071 sudo[54459]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-02T15:21:25.861 INFO:journalctl@ceph.osd.1.smithi071.stdout:Feb 02 15:21:25 smithi071 sudo[54459]: pam_unix(sudo:session): session closed for user root 2024-02-02T15:21:25.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:25 smithi142 ceph-mon[45728]: osdmap e21: 4 total, 3 up, 4 in 2024-02-02T15:21:25.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:25 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-02T15:21:25.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:25 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:26.114 INFO:journalctl@ceph.osd.1.smithi071.stdout:Feb 02 15:21:25 smithi071 sudo[54462]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-02T15:21:26.114 INFO:journalctl@ceph.osd.1.smithi071.stdout:Feb 02 15:21:25 smithi071 sudo[54462]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-02T15:21:26.114 INFO:journalctl@ceph.osd.1.smithi071.stdout:Feb 02 15:21:25 smithi071 sudo[54462]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-02T15:21:26.114 INFO:journalctl@ceph.osd.1.smithi071.stdout:Feb 02 15:21:25 smithi071 sudo[54462]: pam_unix(sudo:session): session closed for user root 2024-02-02T15:21:26.376 INFO:journalctl@ceph.osd.2.smithi142.stdout:Feb 02 15:21:26 smithi142 sudo[56155]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-02T15:21:26.377 INFO:journalctl@ceph.osd.2.smithi142.stdout:Feb 02 15:21:26 smithi142 sudo[56155]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-02T15:21:26.377 INFO:journalctl@ceph.osd.2.smithi142.stdout:Feb 02 15:21:26 smithi142 sudo[56155]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-02T15:21:26.377 INFO:journalctl@ceph.osd.2.smithi142.stdout:Feb 02 15:21:26 smithi142 sudo[56155]: pam_unix(sudo:session): session closed for user root 2024-02-02T15:21:26.670 INFO:journalctl@ceph.osd.2.smithi142.stdout:Feb 02 15:21:26 smithi142 sudo[56158]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-02T15:21:26.670 INFO:journalctl@ceph.osd.2.smithi142.stdout:Feb 02 15:21:26 smithi142 sudo[56158]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-02T15:21:26.670 INFO:journalctl@ceph.osd.2.smithi142.stdout:Feb 02 15:21:26 smithi142 sudo[56158]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-02T15:21:26.670 INFO:journalctl@ceph.osd.2.smithi142.stdout:Feb 02 15:21:26 smithi142 sudo[56158]: pam_unix(sudo:session): session closed for user root 2024-02-02T15:21:27.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:26 smithi001 sudo[60251]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-02T15:21:27.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:26 smithi001 sudo[60251]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-02T15:21:27.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:26 smithi001 sudo[60251]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-02T15:21:27.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:26 smithi001 sudo[60251]: pam_unix(sudo:session): session closed for user root 2024-02-02T15:21:27.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:21:27.515+0000 7ff5f4c00700 1 -- 172.21.15.1:0/2988839503 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mgrmap(e 16) v1 ==== 135069+0+0 (secure 0 0 0) 0x7ff5e0095890 con 0x7ff5f8104190 2024-02-02T15:21:27.672 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:27 smithi001 sudo[60390]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-02T15:21:27.672 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:27 smithi001 sudo[60390]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-02T15:21:27.672 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:27 smithi001 sudo[60390]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-02T15:21:27.672 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:27 smithi001 sudo[60390]: pam_unix(sudo:session): session closed for user root 2024-02-02T15:21:27.672 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:27 smithi001 ceph-mon[35470]: pgmap v71: 1 pgs: 1 creating+activating; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-02T15:21:27.672 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:27 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:27.672 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:27 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:27.673 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:27 smithi001 ceph-mon[35470]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-02T15:21:27.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:27 smithi071 ceph-mon[44707]: pgmap v71: 1 pgs: 1 creating+activating; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-02T15:21:27.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:27 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:27.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:27 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:27.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:27 smithi071 ceph-mon[44707]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-02T15:21:27.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:27 smithi142 ceph-mon[45728]: pgmap v71: 1 pgs: 1 creating+activating; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-02T15:21:27.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:27 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:27.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:27 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:27.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:27 smithi142 ceph-mon[45728]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-02T15:21:27.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:27 smithi142 sudo[56161]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-02T15:21:27.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:27 smithi142 sudo[56161]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-02T15:21:27.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:27 smithi142 sudo[56161]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-02T15:21:27.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:27 smithi142 sudo[56161]: pam_unix(sudo:session): session closed for user root 2024-02-02T15:21:28.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:27 smithi142 sudo[56164]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-02T15:21:28.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:27 smithi142 sudo[56164]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-02T15:21:28.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:27 smithi142 sudo[56164]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-02T15:21:28.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:27 smithi142 sudo[56164]: pam_unix(sudo:session): session closed for user root 2024-02-02T15:21:28.514 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:28 smithi071 sudo[54465]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-02T15:21:28.515 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:28 smithi071 sudo[54465]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-02T15:21:28.515 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:28 smithi071 sudo[54465]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-02T15:21:28.515 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:28 smithi071 sudo[54465]: pam_unix(sudo:session): session closed for user root 2024-02-02T15:21:28.515 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:28 smithi071 sudo[54468]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-02T15:21:28.515 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:28 smithi071 sudo[54468]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-02T15:21:28.515 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:28 smithi071 sudo[54468]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-02T15:21:28.515 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:28 smithi071 sudo[54468]: pam_unix(sudo:session): session closed for user root 2024-02-02T15:21:28.588 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:28 smithi001 ceph-mon[35470]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-02T15:21:28.588 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:28 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-02T15:21:28.588 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:28 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-02T15:21:28.588 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:28 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-02T15:21:28.588 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:28 smithi001 ceph-mon[35470]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-02T15:21:28.588 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:28 smithi001 ceph-mon[35470]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-02-02T15:21:28.588 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:28 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-02T15:21:28.588 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:28 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-02T15:21:28.588 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:28 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-02T15:21:28.588 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:28 smithi001 ceph-mon[35470]: pgmap v72: 1 pgs: 1 creating+activating; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-02T15:21:28.588 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:28 smithi001 ceph-mon[35470]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-02T15:21:28.589 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:28 smithi001 ceph-mon[35470]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-02T15:21:28.589 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:28 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-02T15:21:28.589 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:28 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-02T15:21:28.589 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:28 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-02T15:21:28.589 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:28 smithi001 ceph-mon[35470]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-02T15:21:28.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:28 smithi071 ceph-mon[44707]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-02T15:21:28.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:28 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-02T15:21:28.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:28 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-02T15:21:28.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:28 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-02T15:21:28.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:28 smithi071 ceph-mon[44707]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-02T15:21:28.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:28 smithi071 ceph-mon[44707]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-02-02T15:21:28.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:28 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-02T15:21:28.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:28 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-02T15:21:28.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:28 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-02T15:21:28.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:28 smithi071 ceph-mon[44707]: pgmap v72: 1 pgs: 1 creating+activating; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-02T15:21:28.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:28 smithi071 ceph-mon[44707]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-02T15:21:28.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:28 smithi071 ceph-mon[44707]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-02T15:21:28.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:28 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-02T15:21:28.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:28 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-02T15:21:28.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:28 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-02T15:21:28.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:28 smithi071 ceph-mon[44707]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-02T15:21:28.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:28 smithi142 ceph-mon[45728]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-02T15:21:28.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:28 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-02T15:21:28.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:28 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-02T15:21:28.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:28 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-02T15:21:28.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:28 smithi142 ceph-mon[45728]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-02T15:21:28.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:28 smithi142 ceph-mon[45728]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-02-02T15:21:28.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:28 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-02T15:21:28.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:28 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-02T15:21:28.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:28 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-02T15:21:28.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:28 smithi142 ceph-mon[45728]: pgmap v72: 1 pgs: 1 creating+activating; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-02T15:21:28.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:28 smithi142 ceph-mon[45728]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-02T15:21:28.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:28 smithi142 ceph-mon[45728]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-02T15:21:28.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:28 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-02T15:21:28.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:28 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-02T15:21:28.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:28 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-02T15:21:28.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:28 smithi142 ceph-mon[45728]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-02T15:21:30.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:29 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-02T15:21:30.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:29 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:21:30.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:29 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-02T15:21:30.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:29 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:21:30.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:29 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-02T15:21:30.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:29 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:21:31.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:30 smithi001 ceph-mon[35470]: pgmap v73: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-02T15:21:31.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:30 smithi001 ceph-mon[35470]: Deploying daemon osd.3 on smithi001 2024-02-02T15:21:31.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:30 smithi071 ceph-mon[44707]: pgmap v73: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-02T15:21:31.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:30 smithi071 ceph-mon[44707]: Deploying daemon osd.3 on smithi001 2024-02-02T15:21:31.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:30 smithi142 ceph-mon[45728]: pgmap v73: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-02T15:21:31.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:30 smithi142 ceph-mon[45728]: Deploying daemon osd.3 on smithi001 2024-02-02T15:21:33.264 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:32 smithi001 ceph-mon[35470]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-02T15:21:33.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:32 smithi071 ceph-mon[44707]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-02T15:21:33.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:32 smithi142 ceph-mon[45728]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-02T15:21:35.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:34 smithi001 ceph-mon[35470]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-02T15:21:35.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:34 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:35.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:34 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:35.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:34 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:21:35.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:34 smithi071 ceph-mon[44707]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-02T15:21:35.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:34 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:35.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:34 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:35.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:34 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:21:35.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:34 smithi142 ceph-mon[45728]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-02T15:21:35.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:34 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:35.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:34 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:35.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:34 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:21:36.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:36 smithi001 ceph-mon[35470]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-02T15:21:37.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:36 smithi071 ceph-mon[44707]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-02T15:21:37.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:36 smithi142 ceph-mon[45728]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-02T15:21:37.798 INFO:teuthology.orchestra.run.smithi001.stdout:Created osd(s) 3 on host 'smithi001' 2024-02-02T15:21:37.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:21:37.799+0000 7ff5f4c00700 1 -- 172.21.15.1:0/2988839503 <== mgr.14154 v2:172.21.15.1:6800/3768989981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ff5f8061180 con 0x7ff5d80a0510 2024-02-02T15:21:37.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:21:37.801+0000 7ff5fea00700 1 -- 172.21.15.1:0/2988839503 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7ff5d80a0510 msgr2=0x7ff5d80a29d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:21:37.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:21:37.801+0000 7ff5fea00700 1 --2- 172.21.15.1:0/2988839503 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7ff5d80a0510 0x7ff5d80a29d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff5e800c010 tx=0x7ff5e8019040 comp rx=0 tx=0).stop 2024-02-02T15:21:37.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:21:37.802+0000 7ff5fea00700 1 -- 172.21.15.1:0/2988839503 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5f8104190 msgr2=0x7ff5f8199260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:21:37.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:21:37.802+0000 7ff5fea00700 1 --2- 172.21.15.1:0/2988839503 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5f8104190 0x7ff5f8199260 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff5e000e9c0 tx=0x7ff5e000ecd0 comp rx=0 tx=0).stop 2024-02-02T15:21:37.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:21:37.802+0000 7ff5fea00700 1 -- 172.21.15.1:0/2988839503 shutdown_connections 2024-02-02T15:21:37.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:21:37.802+0000 7ff5fea00700 1 --2- 172.21.15.1:0/2988839503 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff5f8106ac0 0x7ff5f81a02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:21:37.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:21:37.802+0000 7ff5fea00700 1 --2- 172.21.15.1:0/2988839503 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7ff5d80a0510 0x7ff5d80a29d0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:21:37.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:21:37.802+0000 7ff5fea00700 1 --2- 172.21.15.1:0/2988839503 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5f8101860 0x7ff5f8198d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:21:37.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:21:37.802+0000 7ff5fea00700 1 --2- 172.21.15.1:0/2988839503 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5f8104190 0x7ff5f8199260 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:21:37.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:21:37.802+0000 7ff5fea00700 1 -- 172.21.15.1:0/2988839503 >> 172.21.15.1:0/2988839503 conn(0x7ff5f80fb250 msgr2=0x7ff5f80ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:21:37.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:21:37.802+0000 7ff5fea00700 1 -- 172.21.15.1:0/2988839503 shutdown_connections 2024-02-02T15:21:37.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:21:37.802+0000 7ff5fea00700 1 -- 172.21.15.1:0/2988839503 wait complete. 2024-02-02T15:21:38.037 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:37 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:38.037 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:37 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:38.037 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:37 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:21:38.037 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:37 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:21:38.038 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:37 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:38.038 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:37 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:38.038 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:37 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:38.038 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:37 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:21:38.229 DEBUG:teuthology.orchestra.run.smithi001:osd.3> sudo journalctl -f -n 0 -u ceph-371a409c-c1de-11ee-95b6-87774f69a715@osd.3.service 2024-02-02T15:21:38.234 INFO:tasks.cephadm:Deploying osd.4 on smithi071 with /dev/vg_nvme/lv_3... 2024-02-02T15:21:38.234 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-02-02T15:21:38.267 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:37 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:38.267 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:37 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:38.267 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:37 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:21:38.267 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:37 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:21:38.267 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:37 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:38.268 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:37 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:38.268 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:37 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:38.268 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:37 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:21:38.313 INFO:journalctl@ceph.osd.3.smithi001.stdout:-- Logs begin at Fri 2024-02-02 15:05:03 UTC. -- 2024-02-02T15:21:38.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:37 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:38.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:37 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:38.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:37 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:21:38.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:37 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:21:38.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:37 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:38.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:37 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:38.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:37 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:38.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:37 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:21:38.465 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.b/config 2024-02-02T15:21:39.218 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:38 smithi071 ceph-mon[44707]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-02T15:21:39.223 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:38 smithi001 ceph-mon[35470]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-02T15:21:39.223 INFO:journalctl@ceph.osd.3.smithi001.stdout:Feb 02 15:21:39 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-3[61288]: 2024-02-02T15:21:39.006+0000 7f53c875a640 -1 osd.3 0 log_to_monitors true 2024-02-02T15:21:39.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:38 smithi142 ceph-mon[45728]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-02T15:21:39.665 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-02-02T15:21:39.665 INFO:teuthology.orchestra.run.smithi071.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-02-02T15:21:39.666 INFO:teuthology.orchestra.run.smithi071.stderr: stderr: 10+0 records in 2024-02-02T15:21:39.666 INFO:teuthology.orchestra.run.smithi071.stderr:10+0 records out 2024-02-02T15:21:39.666 INFO:teuthology.orchestra.run.smithi071.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113044 s, 928 MB/s 2024-02-02T15:21:39.666 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping successful for: 2024-02-02T15:21:39.994 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph orch daemon add osd smithi071:vg_nvme/lv_3 2024-02-02T15:21:40.250 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.b/config 2024-02-02T15:21:40.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:39 smithi071 ceph-mon[44707]: from='osd.3 [v2:172.21.15.1:6810/1086124477,v1:172.21.15.1:6811/1086124477]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-02T15:21:40.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:39 smithi142 ceph-mon[45728]: from='osd.3 [v2:172.21.15.1:6810/1086124477,v1:172.21.15.1:6811/1086124477]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-02T15:21:40.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:39 smithi001 ceph-mon[35470]: from='osd.3 [v2:172.21.15.1:6810/1086124477,v1:172.21.15.1:6811/1086124477]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-02T15:21:41.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:21:41.172+0000 7fe89004a700 1 -- 172.21.15.71:0/2722118433 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe888101900 msgr2=0x7fe888103cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:21:41.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:21:41.172+0000 7fe89004a700 1 --2- 172.21.15.71:0/2722118433 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe888101900 0x7fe888103cf0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe884009990 tx=0x7fe884009ca0 comp rx=0 tx=0).stop 2024-02-02T15:21:41.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:21:41.172+0000 7fe89004a700 1 -- 172.21.15.71:0/2722118433 shutdown_connections 2024-02-02T15:21:41.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:21:41.172+0000 7fe89004a700 1 --2- 172.21.15.71:0/2722118433 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe888106b60 0x7fe888108f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:21:41.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:21:41.172+0000 7fe89004a700 1 --2- 172.21.15.71:0/2722118433 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe888101900 0x7fe888103cf0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:21:41.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:21:41.172+0000 7fe89004a700 1 --2- 172.21.15.71:0/2722118433 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe888104230 0x7fe888106620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:21:41.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:21:41.172+0000 7fe89004a700 1 -- 172.21.15.71:0/2722118433 >> 172.21.15.71:0/2722118433 conn(0x7fe8880fb250 msgr2=0x7fe8880fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:21:41.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:21:41.172+0000 7fe89004a700 1 -- 172.21.15.71:0/2722118433 shutdown_connections 2024-02-02T15:21:41.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:21:41.172+0000 7fe89004a700 1 -- 172.21.15.71:0/2722118433 wait complete. 2024-02-02T15:21:41.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:21:41.172+0000 7fe89004a700 1 Processor -- start 2024-02-02T15:21:41.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:21:41.172+0000 7fe89004a700 1 -- start start 2024-02-02T15:21:41.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:21:41.173+0000 7fe89004a700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe888101900 0x7fe888101260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:21:41.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:21:41.173+0000 7fe89004a700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe888104230 0x7fe8880ff8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:21:41.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:21:41.173+0000 7fe89004a700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe888106b60 0x7fe8880ffdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:21:41.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:21:41.173+0000 7fe89004a700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe888100570 con 0x7fe888104230 2024-02-02T15:21:41.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:21:41.173+0000 7fe89004a700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8881006e0 con 0x7fe888106b60 2024-02-02T15:21:41.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:21:41.173+0000 7fe89004a700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8881a0a40 con 0x7fe888101900 2024-02-02T15:21:41.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:21:41.173+0000 7fe88e5e7700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe888106b60 0x7fe8880ffdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:21:41.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:21:41.173+0000 7fe88e5e7700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe888106b60 0x7fe8880ffdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:32796/0 (socket says 172.21.15.71:32796) 2024-02-02T15:21:41.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:21:41.173+0000 7fe88e5e7700 1 -- 172.21.15.71:0/185482537 learned_addr learned my addr 172.21.15.71:0/185482537 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-02T15:21:41.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:21:41.173+0000 7fe88dde6700 1 --2- 172.21.15.71:0/185482537 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe888101900 0x7fe888101260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:21:41.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:21:41.173+0000 7fe88d5e5700 1 --2- 172.21.15.71:0/185482537 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe888104230 0x7fe8880ff8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:21:41.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:21:41.173+0000 7fe88e5e7700 1 -- 172.21.15.71:0/185482537 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe888101900 msgr2=0x7fe888101260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:21:41.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:21:41.173+0000 7fe88e5e7700 1 --2- 172.21.15.71:0/185482537 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe888101900 0x7fe888101260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:21:41.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:21:41.173+0000 7fe88e5e7700 1 -- 172.21.15.71:0/185482537 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe888104230 msgr2=0x7fe8880ff8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:21:41.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:21:41.173+0000 7fe88e5e7700 1 --2- 172.21.15.71:0/185482537 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe888104230 0x7fe8880ff8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:21:41.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:21:41.173+0000 7fe88e5e7700 1 -- 172.21.15.71:0/185482537 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8840095f0 con 0x7fe888106b60 2024-02-02T15:21:41.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:21:41.173+0000 7fe88e5e7700 1 --2- 172.21.15.71:0/185482537 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe888106b60 0x7fe8880ffdf0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe8800094f0 tx=0x7fe88000d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:21:41.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:21:41.173+0000 7fe87effd700 1 -- 172.21.15.71:0/185482537 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe880004db0 con 0x7fe888106b60 2024-02-02T15:21:41.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:21:41.173+0000 7fe87effd700 1 -- 172.21.15.71:0/185482537 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe88000f040 con 0x7fe888106b60 2024-02-02T15:21:41.176 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:21:41.173+0000 7fe89004a700 1 -- 172.21.15.71:0/185482537 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8881a0d20 con 0x7fe888106b60 2024-02-02T15:21:41.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:21:41.173+0000 7fe87effd700 1 -- 172.21.15.71:0/185482537 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe880005980 con 0x7fe888106b60 2024-02-02T15:21:41.177 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:21:41.173+0000 7fe89004a700 1 -- 172.21.15.71:0/185482537 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8881a1170 con 0x7fe888106b60 2024-02-02T15:21:41.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:21:41.174+0000 7fe89004a700 1 -- 172.21.15.71:0/185482537 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8880fcdd0 con 0x7fe888106b60 2024-02-02T15:21:41.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:21:41.175+0000 7fe87effd700 1 -- 172.21.15.71:0/185482537 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 135069+0+0 (secure 0 0 0) 0x7fe880005000 con 0x7fe888106b60 2024-02-02T15:21:41.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:21:41.176+0000 7fe87effd700 1 --2- 172.21.15.71:0/185482537 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fe8740a0520 0x7fe8740a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:21:41.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:21:41.176+0000 7fe87effd700 1 -- 172.21.15.71:0/185482537 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 3389+0+0 (secure 0 0 0) 0x7fe8800ca770 con 0x7fe888106b60 2024-02-02T15:21:41.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:21:41.176+0000 7fe88dde6700 1 --2- 172.21.15.71:0/185482537 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fe8740a0520 0x7fe8740a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:21:41.178 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:21:41.177+0000 7fe88dde6700 1 --2- 172.21.15.71:0/185482537 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fe8740a0520 0x7fe8740a29e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe88400b5c0 tx=0x7fe884005960 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:21:41.180 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:21:41.180+0000 7fe87effd700 1 -- 172.21.15.71:0/185482537 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7fe880095630 con 0x7fe888106b60 2024-02-02T15:21:41.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:40 smithi001 ceph-mon[35470]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-02T15:21:41.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:40 smithi001 ceph-mon[35470]: from='osd.3 [v2:172.21.15.1:6810/1086124477,v1:172.21.15.1:6811/1086124477]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-02T15:21:41.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:40 smithi001 ceph-mon[35470]: osdmap e22: 4 total, 3 up, 4 in 2024-02-02T15:21:41.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:40 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-02T15:21:41.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:40 smithi001 ceph-mon[35470]: from='osd.3 [v2:172.21.15.1:6810/1086124477,v1:172.21.15.1:6811/1086124477]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-02-02T15:21:41.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:40 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:41.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:40 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:41.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:40 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:21:41.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:40 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:21:41.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:40 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:41.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:40 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:21:41.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:40 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:21:41.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:40 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:41.191 INFO:journalctl@ceph.osd.3.smithi001.stdout:Feb 02 15:21:40 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-3[61288]: 2024-02-02T15:21:40.981+0000 7f53bcc00700 -1 osd.3 0 waiting for initial osdmap 2024-02-02T15:21:41.191 INFO:journalctl@ceph.osd.3.smithi001.stdout:Feb 02 15:21:40 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-3[61288]: 2024-02-02T15:21:40.988+0000 7f53b7a00700 -1 osd.3 23 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-02T15:21:41.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:40 smithi071 ceph-mon[44707]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-02T15:21:41.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:40 smithi071 ceph-mon[44707]: from='osd.3 [v2:172.21.15.1:6810/1086124477,v1:172.21.15.1:6811/1086124477]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-02T15:21:41.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:40 smithi071 ceph-mon[44707]: osdmap e22: 4 total, 3 up, 4 in 2024-02-02T15:21:41.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:40 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-02T15:21:41.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:40 smithi071 ceph-mon[44707]: from='osd.3 [v2:172.21.15.1:6810/1086124477,v1:172.21.15.1:6811/1086124477]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-02-02T15:21:41.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:40 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:41.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:40 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:41.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:40 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:21:41.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:40 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:21:41.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:40 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:41.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:40 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:21:41.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:40 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:21:41.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:40 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:41.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:21:41.374+0000 7fe89004a700 1 -- 172.21.15.71:0/185482537 --> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fe888061180 con 0x7fe8740a0520 2024-02-02T15:21:41.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:40 smithi142 ceph-mon[45728]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-02T15:21:41.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:40 smithi142 ceph-mon[45728]: from='osd.3 [v2:172.21.15.1:6810/1086124477,v1:172.21.15.1:6811/1086124477]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-02T15:21:41.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:40 smithi142 ceph-mon[45728]: osdmap e22: 4 total, 3 up, 4 in 2024-02-02T15:21:41.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:40 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-02T15:21:41.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:40 smithi142 ceph-mon[45728]: from='osd.3 [v2:172.21.15.1:6810/1086124477,v1:172.21.15.1:6811/1086124477]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-02-02T15:21:41.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:40 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:41.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:40 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:41.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:40 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:21:41.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:40 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:21:41.421 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:40 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:41.421 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:40 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:21:41.421 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:40 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:21:41.421 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:40 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:42.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:41 smithi071 ceph-mon[44707]: Detected new or changed devices on smithi001 2024-02-02T15:21:42.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:41 smithi071 ceph-mon[44707]: Adjusting osd_memory_target on smithi001 to 8638M 2024-02-02T15:21:42.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:41 smithi071 ceph-mon[44707]: from='osd.3 [v2:172.21.15.1:6810/1086124477,v1:172.21.15.1:6811/1086124477]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]': finished 2024-02-02T15:21:42.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:41 smithi071 ceph-mon[44707]: osdmap e23: 4 total, 3 up, 4 in 2024-02-02T15:21:42.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:41 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-02T15:21:42.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:41 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-02T15:21:42.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:41 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-02T15:21:42.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:41 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-02T15:21:42.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:41 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:21:42.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:41 smithi142 ceph-mon[45728]: Detected new or changed devices on smithi001 2024-02-02T15:21:42.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:41 smithi142 ceph-mon[45728]: Adjusting osd_memory_target on smithi001 to 8638M 2024-02-02T15:21:42.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:41 smithi142 ceph-mon[45728]: from='osd.3 [v2:172.21.15.1:6810/1086124477,v1:172.21.15.1:6811/1086124477]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]': finished 2024-02-02T15:21:42.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:41 smithi142 ceph-mon[45728]: osdmap e23: 4 total, 3 up, 4 in 2024-02-02T15:21:42.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:41 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-02T15:21:42.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:41 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-02T15:21:42.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:41 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-02T15:21:42.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:41 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-02T15:21:42.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:41 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:21:42.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:41 smithi001 ceph-mon[35470]: Detected new or changed devices on smithi001 2024-02-02T15:21:42.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:41 smithi001 ceph-mon[35470]: Adjusting osd_memory_target on smithi001 to 8638M 2024-02-02T15:21:42.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:41 smithi001 ceph-mon[35470]: from='osd.3 [v2:172.21.15.1:6810/1086124477,v1:172.21.15.1:6811/1086124477]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]': finished 2024-02-02T15:21:42.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:41 smithi001 ceph-mon[35470]: osdmap e23: 4 total, 3 up, 4 in 2024-02-02T15:21:42.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:41 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-02T15:21:42.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:41 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-02T15:21:42.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:41 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-02T15:21:42.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:41 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-02T15:21:42.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:41 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:21:43.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:42 smithi071 ceph-mon[44707]: purged_snaps scrub starts 2024-02-02T15:21:43.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:42 smithi071 ceph-mon[44707]: purged_snaps scrub ok 2024-02-02T15:21:43.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:42 smithi071 ceph-mon[44707]: from='client.24239 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:21:43.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:42 smithi071 ceph-mon[44707]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-02T15:21:43.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:42 smithi071 ceph-mon[44707]: osd.3 [v2:172.21.15.1:6810/1086124477,v1:172.21.15.1:6811/1086124477] boot 2024-02-02T15:21:43.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:42 smithi071 ceph-mon[44707]: osdmap e24: 4 total, 4 up, 4 in 2024-02-02T15:21:43.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:42 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-02T15:21:43.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:42 smithi071 ceph-mon[44707]: osdmap e25: 4 total, 4 up, 4 in 2024-02-02T15:21:43.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:42 smithi142 ceph-mon[45728]: purged_snaps scrub starts 2024-02-02T15:21:43.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:42 smithi142 ceph-mon[45728]: purged_snaps scrub ok 2024-02-02T15:21:43.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:42 smithi142 ceph-mon[45728]: from='client.24239 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:21:43.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:42 smithi142 ceph-mon[45728]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-02T15:21:43.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:42 smithi142 ceph-mon[45728]: osd.3 [v2:172.21.15.1:6810/1086124477,v1:172.21.15.1:6811/1086124477] boot 2024-02-02T15:21:43.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:42 smithi142 ceph-mon[45728]: osdmap e24: 4 total, 4 up, 4 in 2024-02-02T15:21:43.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:42 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-02T15:21:43.421 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:42 smithi142 ceph-mon[45728]: osdmap e25: 4 total, 4 up, 4 in 2024-02-02T15:21:43.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:42 smithi001 ceph-mon[35470]: purged_snaps scrub starts 2024-02-02T15:21:43.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:42 smithi001 ceph-mon[35470]: purged_snaps scrub ok 2024-02-02T15:21:43.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:42 smithi001 ceph-mon[35470]: from='client.24239 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:21:43.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:42 smithi001 ceph-mon[35470]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-02T15:21:43.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:42 smithi001 ceph-mon[35470]: osd.3 [v2:172.21.15.1:6810/1086124477,v1:172.21.15.1:6811/1086124477] boot 2024-02-02T15:21:43.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:42 smithi001 ceph-mon[35470]: osdmap e24: 4 total, 4 up, 4 in 2024-02-02T15:21:43.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:42 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-02T15:21:43.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:42 smithi001 ceph-mon[35470]: osdmap e25: 4 total, 4 up, 4 in 2024-02-02T15:21:44.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:43 smithi071 ceph-mon[44707]: from='client.? 172.21.15.71:0/3064462360' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "43e36c43-47a0-4634-a45b-1da14a0e19d0"}]: dispatch 2024-02-02T15:21:44.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:43 smithi071 ceph-mon[44707]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "43e36c43-47a0-4634-a45b-1da14a0e19d0"}]: dispatch 2024-02-02T15:21:44.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:43 smithi071 ceph-mon[44707]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "43e36c43-47a0-4634-a45b-1da14a0e19d0"}]': finished 2024-02-02T15:21:44.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:43 smithi071 ceph-mon[44707]: osdmap e26: 5 total, 4 up, 5 in 2024-02-02T15:21:44.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:43 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-02T15:21:44.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:43 smithi071 ceph-mon[44707]: from='client.? 172.21.15.71:0/2903457139' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-02T15:21:44.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:43 smithi142 ceph-mon[45728]: from='client.? 172.21.15.71:0/3064462360' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "43e36c43-47a0-4634-a45b-1da14a0e19d0"}]: dispatch 2024-02-02T15:21:44.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:43 smithi142 ceph-mon[45728]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "43e36c43-47a0-4634-a45b-1da14a0e19d0"}]: dispatch 2024-02-02T15:21:44.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:43 smithi142 ceph-mon[45728]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "43e36c43-47a0-4634-a45b-1da14a0e19d0"}]': finished 2024-02-02T15:21:44.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:43 smithi142 ceph-mon[45728]: osdmap e26: 5 total, 4 up, 5 in 2024-02-02T15:21:44.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:43 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-02T15:21:44.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:43 smithi142 ceph-mon[45728]: from='client.? 172.21.15.71:0/2903457139' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-02T15:21:44.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:43 smithi001 ceph-mon[35470]: from='client.? 172.21.15.71:0/3064462360' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "43e36c43-47a0-4634-a45b-1da14a0e19d0"}]: dispatch 2024-02-02T15:21:44.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:43 smithi001 ceph-mon[35470]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "43e36c43-47a0-4634-a45b-1da14a0e19d0"}]: dispatch 2024-02-02T15:21:44.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:43 smithi001 ceph-mon[35470]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "43e36c43-47a0-4634-a45b-1da14a0e19d0"}]': finished 2024-02-02T15:21:44.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:43 smithi001 ceph-mon[35470]: osdmap e26: 5 total, 4 up, 5 in 2024-02-02T15:21:44.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:43 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-02T15:21:44.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:43 smithi001 ceph-mon[35470]: from='client.? 172.21.15.71:0/2903457139' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-02T15:21:45.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:45 smithi142 ceph-mon[45728]: pgmap v85: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 148 KiB/s, 0 objects/s recovering 2024-02-02T15:21:45.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:45 smithi142 ceph-mon[45728]: osdmap e27: 5 total, 4 up, 5 in 2024-02-02T15:21:45.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:45 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-02T15:21:45.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:45 smithi001 ceph-mon[35470]: pgmap v85: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 148 KiB/s, 0 objects/s recovering 2024-02-02T15:21:45.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:45 smithi001 ceph-mon[35470]: osdmap e27: 5 total, 4 up, 5 in 2024-02-02T15:21:45.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:45 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-02T15:21:45.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:45 smithi071 ceph-mon[44707]: pgmap v85: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 148 KiB/s, 0 objects/s recovering 2024-02-02T15:21:45.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:45 smithi071 ceph-mon[44707]: osdmap e27: 5 total, 4 up, 5 in 2024-02-02T15:21:45.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:45 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-02T15:21:46.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:46 smithi142 ceph-mon[45728]: osdmap e28: 5 total, 4 up, 5 in 2024-02-02T15:21:46.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:46 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-02T15:21:46.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:46 smithi001 ceph-mon[35470]: osdmap e28: 5 total, 4 up, 5 in 2024-02-02T15:21:46.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:46 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-02T15:21:46.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:46 smithi071 ceph-mon[44707]: osdmap e28: 5 total, 4 up, 5 in 2024-02-02T15:21:46.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:46 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-02T15:21:47.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:47 smithi142 ceph-mon[45728]: pgmap v88: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 148 KiB/s, 0 objects/s recovering 2024-02-02T15:21:47.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:47 smithi001 ceph-mon[35470]: pgmap v88: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 148 KiB/s, 0 objects/s recovering 2024-02-02T15:21:47.474 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:47 smithi071 ceph-mon[44707]: pgmap v88: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 148 KiB/s, 0 objects/s recovering 2024-02-02T15:21:49.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:49 smithi142 ceph-mon[45728]: pgmap v89: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-02-02T15:21:49.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:49 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-02T15:21:49.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:49 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:21:49.437 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:49 smithi071 ceph-mon[44707]: pgmap v89: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-02-02T15:21:49.437 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:49 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-02T15:21:49.437 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:49 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:21:49.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:49 smithi001 ceph-mon[35470]: pgmap v89: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-02-02T15:21:49.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:49 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-02T15:21:49.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:49 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:21:50.347 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:50 smithi071 ceph-mon[44707]: Deploying daemon osd.4 on smithi071 2024-02-02T15:21:50.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:50 smithi142 ceph-mon[45728]: Deploying daemon osd.4 on smithi071 2024-02-02T15:21:50.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:50 smithi001 ceph-mon[35470]: Deploying daemon osd.4 on smithi071 2024-02-02T15:21:51.302 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:51 smithi071 ceph-mon[44707]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-02-02T15:21:51.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:51 smithi142 ceph-mon[45728]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-02-02T15:21:51.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:51 smithi001 ceph-mon[35470]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-02-02T15:21:53.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:53 smithi142 ceph-mon[45728]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-02-02T15:21:53.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:53 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:53.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:53 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:53.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:53 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:21:53.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:53 smithi001 ceph-mon[35470]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-02-02T15:21:53.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:53 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:53.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:53 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:53.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:53 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:21:53.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:53 smithi071 ceph-mon[44707]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-02-02T15:21:53.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:53 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:53.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:53 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:53.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:53 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:21:54.579 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:21:54.579+0000 7fe87effd700 1 -- 172.21.15.71:0/185482537 <== mgr.14154 v2:172.21.15.1:6800/3768989981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fe888061180 con 0x7fe8740a0520 2024-02-02T15:21:54.580 INFO:teuthology.orchestra.run.smithi071.stdout:Created osd(s) 4 on host 'smithi071' 2024-02-02T15:21:54.580 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:21:54.581+0000 7fe89004a700 1 -- 172.21.15.71:0/185482537 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fe8740a0520 msgr2=0x7fe8740a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:21:54.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:21:54.581+0000 7fe89004a700 1 --2- 172.21.15.71:0/185482537 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fe8740a0520 0x7fe8740a29e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe88400b5c0 tx=0x7fe884005960 comp rx=0 tx=0).stop 2024-02-02T15:21:54.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:21:54.581+0000 7fe89004a700 1 -- 172.21.15.71:0/185482537 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe888106b60 msgr2=0x7fe8880ffdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:21:54.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:21:54.581+0000 7fe89004a700 1 --2- 172.21.15.71:0/185482537 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe888106b60 0x7fe8880ffdf0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe8800094f0 tx=0x7fe88000d3b0 comp rx=0 tx=0).stop 2024-02-02T15:21:54.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:21:54.581+0000 7fe89004a700 1 -- 172.21.15.71:0/185482537 shutdown_connections 2024-02-02T15:21:54.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:21:54.581+0000 7fe89004a700 1 --2- 172.21.15.71:0/185482537 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe888106b60 0x7fe8880ffdf0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:21:54.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:21:54.581+0000 7fe89004a700 1 --2- 172.21.15.71:0/185482537 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fe8740a0520 0x7fe8740a29e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:21:54.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:21:54.581+0000 7fe89004a700 1 --2- 172.21.15.71:0/185482537 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe888101900 0x7fe888101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:21:54.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:21:54.581+0000 7fe89004a700 1 --2- 172.21.15.71:0/185482537 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe888104230 0x7fe8880ff8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:21:54.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:21:54.581+0000 7fe89004a700 1 -- 172.21.15.71:0/185482537 >> 172.21.15.71:0/185482537 conn(0x7fe8880fb250 msgr2=0x7fe888102570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:21:54.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:21:54.582+0000 7fe89004a700 1 -- 172.21.15.71:0/185482537 shutdown_connections 2024-02-02T15:21:54.581 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:21:54.582+0000 7fe89004a700 1 -- 172.21.15.71:0/185482537 wait complete. 2024-02-02T15:21:54.937 DEBUG:teuthology.orchestra.run.smithi071:osd.4> sudo journalctl -f -n 0 -u ceph-371a409c-c1de-11ee-95b6-87774f69a715@osd.4.service 2024-02-02T15:21:54.939 INFO:tasks.cephadm:Deploying osd.5 on smithi142 with /dev/vg_nvme/lv_3... 2024-02-02T15:21:54.940 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-02-02T15:21:54.971 INFO:journalctl@ceph.osd.4.smithi071.stdout:-- Logs begin at Fri 2024-02-02 14:58:45 UTC. -- 2024-02-02T15:21:55.173 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.c/config 2024-02-02T15:21:55.292 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:55 smithi071 ceph-mon[44707]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-02-02T15:21:55.292 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:55 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:55.292 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:55 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:55.292 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:55 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:21:55.292 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:55 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:21:55.292 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:55 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:55.292 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:55 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:55.293 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:55 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:55.293 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:55 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:21:55.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:55 smithi142 ceph-mon[45728]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-02-02T15:21:55.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:55 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:55.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:55 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:55.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:55 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:21:55.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:55 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:21:55.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:55 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:55.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:55 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:55.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:55 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:55.294 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:55 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:21:55.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:55 smithi001 ceph-mon[35470]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-02-02T15:21:55.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:55 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:55.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:55 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:55.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:55 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:21:55.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:55 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:21:55.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:55 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:55.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:55 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:55.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:55 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:55.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:55 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:21:56.454 INFO:teuthology.orchestra.run.smithi142.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-02-02T15:21:56.454 INFO:teuthology.orchestra.run.smithi142.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-02-02T15:21:56.454 INFO:teuthology.orchestra.run.smithi142.stderr: stderr: 10+0 records in 2024-02-02T15:21:56.454 INFO:teuthology.orchestra.run.smithi142.stderr:10+0 records out 2024-02-02T15:21:56.455 INFO:teuthology.orchestra.run.smithi142.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113228 s, 926 MB/s 2024-02-02T15:21:56.455 INFO:teuthology.orchestra.run.smithi142.stderr:--> Zapping successful for: 2024-02-02T15:21:56.747 INFO:journalctl@ceph.osd.4.smithi071.stdout:Feb 02 15:21:56 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-4[57502]: 2024-02-02T15:21:56.485+0000 7f637b80a640 -1 osd.4 0 log_to_monitors true 2024-02-02T15:21:56.848 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:56 smithi142 ceph-mon[45728]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-02-02T15:21:56.848 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:56 smithi142 ceph-mon[45728]: from='osd.4 [v2:172.21.15.71:6808/3095380305,v1:172.21.15.71:6809/3095380305]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-02T15:21:56.848 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:56 smithi142 ceph-mon[45728]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-02T15:21:56.882 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph orch daemon add osd smithi142:vg_nvme/lv_3 2024-02-02T15:21:57.103 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.c/config 2024-02-02T15:21:57.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:56 smithi071 ceph-mon[44707]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-02-02T15:21:57.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:56 smithi071 ceph-mon[44707]: from='osd.4 [v2:172.21.15.71:6808/3095380305,v1:172.21.15.71:6809/3095380305]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-02T15:21:57.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:56 smithi071 ceph-mon[44707]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-02T15:21:57.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:56 smithi001 ceph-mon[35470]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-02-02T15:21:57.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:56 smithi001 ceph-mon[35470]: from='osd.4 [v2:172.21.15.71:6808/3095380305,v1:172.21.15.71:6809/3095380305]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-02T15:21:57.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:56 smithi001 ceph-mon[35470]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-02T15:21:57.904 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:57 smithi142 ceph-mon[45728]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-02T15:21:57.904 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:57 smithi142 ceph-mon[45728]: osdmap e29: 5 total, 4 up, 5 in 2024-02-02T15:21:57.904 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:57 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-02T15:21:57.904 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:57 smithi142 ceph-mon[45728]: from='osd.4 [v2:172.21.15.71:6808/3095380305,v1:172.21.15.71:6809/3095380305]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-02-02T15:21:57.905 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:57 smithi142 ceph-mon[45728]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-02-02T15:21:57.905 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:57 smithi142 ceph-mon[45728]: Detected new or changed devices on smithi071 2024-02-02T15:21:57.905 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:57 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:57.905 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:57 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:57.905 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:57 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:21:57.905 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:57 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:21:57.905 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:57 smithi142 ceph-mon[45728]: Adjusting osd_memory_target on smithi071 to 8556M 2024-02-02T15:21:57.905 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:57 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:57.905 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:57 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:21:57.905 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:57 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:21:57.905 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:57 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:58.082 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:58.079+0000 7f1ccf79e700 1 -- 172.21.15.142:0/2196889041 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1cc8100dd0 msgr2=0x7f1cc81011f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:21:58.082 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:58.079+0000 7f1ccf79e700 1 --2- 172.21.15.142:0/2196889041 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1cc8100dd0 0x7f1cc81011f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1cb8009990 tx=0x7f1cb8009ca0 comp rx=0 tx=0).stop 2024-02-02T15:21:58.082 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:58.079+0000 7f1ccf79e700 1 -- 172.21.15.142:0/2196889041 shutdown_connections 2024-02-02T15:21:58.082 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:58.079+0000 7f1ccf79e700 1 --2- 172.21.15.142:0/2196889041 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1cc8102970 0x7f1cc8109220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:21:58.082 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:58.079+0000 7f1ccf79e700 1 --2- 172.21.15.142:0/2196889041 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1cc8100dd0 0x7f1cc81011f0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:21:58.083 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:58.079+0000 7f1ccf79e700 1 --2- 172.21.15.142:0/2196889041 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1cc8101fd0 0x7f1cc8102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:21:58.083 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:58.079+0000 7f1ccf79e700 1 -- 172.21.15.142:0/2196889041 >> 172.21.15.142:0/2196889041 conn(0x7f1cc80fc330 msgr2=0x7f1cc80fe7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:21:58.083 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:58.079+0000 7f1ccf79e700 1 -- 172.21.15.142:0/2196889041 shutdown_connections 2024-02-02T15:21:58.083 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:58.079+0000 7f1ccf79e700 1 -- 172.21.15.142:0/2196889041 wait complete. 2024-02-02T15:21:58.083 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:58.080+0000 7f1ccf79e700 1 Processor -- start 2024-02-02T15:21:58.083 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:58.080+0000 7f1ccf79e700 1 -- start start 2024-02-02T15:21:58.083 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:58.080+0000 7f1ccf79e700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1cc8100dd0 0x7f1cc8196b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:21:58.083 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:58.080+0000 7f1ccf79e700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1cc8101fd0 0x7f1cc8197070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:21:58.083 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:58.080+0000 7f1ccf79e700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1cc8102970 0x7f1cc819e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:21:58.083 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:58.080+0000 7f1ccf79e700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc819e6c0 con 0x7f1cc8101fd0 2024-02-02T15:21:58.083 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:58.080+0000 7f1ccf79e700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc819e800 con 0x7f1cc8100dd0 2024-02-02T15:21:58.083 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:58.080+0000 7f1ccf79e700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc819e990 con 0x7f1cc8102970 2024-02-02T15:21:58.083 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:58.080+0000 7f1ccdd3b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1cc8102970 0x7f1cc819e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:21:58.084 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:58.080+0000 7f1ccdd3b700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1cc8102970 0x7f1cc819e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:48354/0 (socket says 172.21.15.142:48354) 2024-02-02T15:21:58.084 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:58.080+0000 7f1ccdd3b700 1 -- 172.21.15.142:0/826194082 learned_addr learned my addr 172.21.15.142:0/826194082 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-02-02T15:21:58.084 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:58.081+0000 7f1ccd53a700 1 --2- 172.21.15.142:0/826194082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1cc8100dd0 0x7f1cc8196b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:21:58.084 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:58.081+0000 7f1ccdd3b700 1 -- 172.21.15.142:0/826194082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1cc8100dd0 msgr2=0x7f1cc8196b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:21:58.084 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:58.081+0000 7f1ccdd3b700 1 --2- 172.21.15.142:0/826194082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1cc8100dd0 0x7f1cc8196b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:21:58.084 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:58.081+0000 7f1ccdd3b700 1 -- 172.21.15.142:0/826194082 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1cc8101fd0 msgr2=0x7f1cc8197070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:21:58.084 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:58.081+0000 7f1cccd39700 1 --2- 172.21.15.142:0/826194082 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1cc8101fd0 0x7f1cc8197070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:21:58.084 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:58.081+0000 7f1ccdd3b700 1 --2- 172.21.15.142:0/826194082 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1cc8101fd0 0x7f1cc8197070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:21:58.084 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:58.081+0000 7f1ccdd3b700 1 -- 172.21.15.142:0/826194082 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cb80095f0 con 0x7f1cc8102970 2024-02-02T15:21:58.084 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:58.081+0000 7f1ccd53a700 1 --2- 172.21.15.142:0/826194082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1cc8100dd0 0x7f1cc8196b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:21:58.084 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:58.081+0000 7f1cccd39700 1 --2- 172.21.15.142:0/826194082 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1cc8101fd0 0x7f1cc8197070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:21:58.084 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:58.081+0000 7f1ccdd3b700 1 --2- 172.21.15.142:0/826194082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1cc8102970 0x7f1cc819e0f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1cc0007f10 tx=0x7f1cc000d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:21:58.084 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:58.081+0000 7f1cbe7fc700 1 -- 172.21.15.142:0/826194082 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc000ddd0 con 0x7f1cc8102970 2024-02-02T15:21:58.085 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:58.081+0000 7f1cbe7fc700 1 -- 172.21.15.142:0/826194082 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1cc000f040 con 0x7f1cc8102970 2024-02-02T15:21:58.085 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:58.081+0000 7f1ccf79e700 1 -- 172.21.15.142:0/826194082 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cc819ec70 con 0x7f1cc8102970 2024-02-02T15:21:58.085 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:58.081+0000 7f1cbe7fc700 1 -- 172.21.15.142:0/826194082 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc0015400 con 0x7f1cc8102970 2024-02-02T15:21:58.085 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:58.081+0000 7f1ccf79e700 1 -- 172.21.15.142:0/826194082 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cc819f120 con 0x7f1cc8102970 2024-02-02T15:21:58.085 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:58.082+0000 7f1ccf79e700 1 -- 172.21.15.142:0/826194082 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cc810c650 con 0x7f1cc8102970 2024-02-02T15:21:58.086 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:58.083+0000 7f1cbe7fc700 1 -- 172.21.15.142:0/826194082 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 16) v1 ==== 135069+0+0 (secure 0 0 0) 0x7f1cc0004000 con 0x7f1cc8102970 2024-02-02T15:21:58.086 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:58.084+0000 7f1cbe7fc700 1 --2- 172.21.15.142:0/826194082 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f1cb40a05a0 0x7f1cb40a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:21:58.086 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:58.084+0000 7f1cbe7fc700 1 -- 172.21.15.142:0/826194082 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 3821+0+0 (secure 0 0 0) 0x7f1cc00caac0 con 0x7f1cc8102970 2024-02-02T15:21:58.086 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:58.084+0000 7f1ccd53a700 1 --2- 172.21.15.142:0/826194082 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f1cb40a05a0 0x7f1cb40a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:21:58.086 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:58.085+0000 7f1ccd53a700 1 --2- 172.21.15.142:0/826194082 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f1cb40a05a0 0x7f1cb40a2a60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1cb8003790 tx=0x7f1cb8005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:21:58.087 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:58.088+0000 7f1cbe7fc700 1 -- 172.21.15.142:0/826194082 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f1cc0095750 con 0x7f1cc8102970 2024-02-02T15:21:58.114 INFO:journalctl@ceph.osd.4.smithi071.stdout:Feb 02 15:21:57 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-4[57502]: 2024-02-02T15:21:57.759+0000 7f6370e6e700 -1 osd.4 0 waiting for initial osdmap 2024-02-02T15:21:58.114 INFO:journalctl@ceph.osd.4.smithi071.stdout:Feb 02 15:21:57 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-4[57502]: 2024-02-02T15:21:57.766+0000 7f636bc61700 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-02T15:21:58.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:57 smithi071 ceph-mon[44707]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-02T15:21:58.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:57 smithi071 ceph-mon[44707]: osdmap e29: 5 total, 4 up, 5 in 2024-02-02T15:21:58.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:57 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-02T15:21:58.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:57 smithi071 ceph-mon[44707]: from='osd.4 [v2:172.21.15.71:6808/3095380305,v1:172.21.15.71:6809/3095380305]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-02-02T15:21:58.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:57 smithi071 ceph-mon[44707]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-02-02T15:21:58.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:57 smithi071 ceph-mon[44707]: Detected new or changed devices on smithi071 2024-02-02T15:21:58.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:57 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:58.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:57 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:58.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:57 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:21:58.116 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:57 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:21:58.116 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:57 smithi071 ceph-mon[44707]: Adjusting osd_memory_target on smithi071 to 8556M 2024-02-02T15:21:58.116 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:57 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:58.116 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:57 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:21:58.116 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:57 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:21:58.116 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:57 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:58.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:57 smithi001 ceph-mon[35470]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-02T15:21:58.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:57 smithi001 ceph-mon[35470]: osdmap e29: 5 total, 4 up, 5 in 2024-02-02T15:21:58.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:57 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-02T15:21:58.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:57 smithi001 ceph-mon[35470]: from='osd.4 [v2:172.21.15.71:6808/3095380305,v1:172.21.15.71:6809/3095380305]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-02-02T15:21:58.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:57 smithi001 ceph-mon[35470]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-02-02T15:21:58.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:57 smithi001 ceph-mon[35470]: Detected new or changed devices on smithi071 2024-02-02T15:21:58.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:57 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:58.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:57 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:58.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:57 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:21:58.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:57 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:21:58.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:57 smithi001 ceph-mon[35470]: Adjusting osd_memory_target on smithi071 to 8556M 2024-02-02T15:21:58.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:57 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:58.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:57 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:21:58.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:57 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:21:58.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:57 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:21:58.283 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:21:58.282+0000 7f1ccf79e700 1 -- 172.21.15.142:0/826194082 --> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f1cc8061180 con 0x7f1cb40a05a0 2024-02-02T15:21:58.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:58 smithi142 ceph-mon[45728]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-02-02T15:21:58.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:58 smithi142 ceph-mon[45728]: osdmap e30: 5 total, 4 up, 5 in 2024-02-02T15:21:58.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:58 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-02T15:21:58.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:58 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-02T15:21:58.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:58 smithi142 ceph-mon[45728]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-02-02T15:21:58.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:58 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-02T15:21:58.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:58 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-02T15:21:58.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:58 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:21:59.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:58 smithi071 ceph-mon[44707]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-02-02T15:21:59.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:58 smithi071 ceph-mon[44707]: osdmap e30: 5 total, 4 up, 5 in 2024-02-02T15:21:59.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:58 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-02T15:21:59.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:58 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-02T15:21:59.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:58 smithi071 ceph-mon[44707]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-02-02T15:21:59.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:58 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-02T15:21:59.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:58 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-02T15:21:59.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:58 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:21:59.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:58 smithi001 ceph-mon[35470]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-02-02T15:21:59.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:58 smithi001 ceph-mon[35470]: osdmap e30: 5 total, 4 up, 5 in 2024-02-02T15:21:59.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:58 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-02T15:21:59.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:58 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-02T15:21:59.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:58 smithi001 ceph-mon[35470]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 107 MiB used, 358 GiB / 358 GiB avail 2024-02-02T15:21:59.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:58 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-02T15:21:59.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:58 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-02T15:21:59.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:58 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:22:00.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:59 smithi071 ceph-mon[44707]: purged_snaps scrub starts 2024-02-02T15:22:00.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:59 smithi071 ceph-mon[44707]: purged_snaps scrub ok 2024-02-02T15:22:00.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:59 smithi071 ceph-mon[44707]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:22:00.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:59 smithi071 ceph-mon[44707]: osd.4 [v2:172.21.15.71:6808/3095380305,v1:172.21.15.71:6809/3095380305] boot 2024-02-02T15:22:00.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:59 smithi071 ceph-mon[44707]: osdmap e31: 5 total, 5 up, 5 in 2024-02-02T15:22:00.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:21:59 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-02T15:22:00.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:59 smithi142 ceph-mon[45728]: purged_snaps scrub starts 2024-02-02T15:22:00.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:59 smithi142 ceph-mon[45728]: purged_snaps scrub ok 2024-02-02T15:22:00.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:59 smithi142 ceph-mon[45728]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:22:00.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:59 smithi142 ceph-mon[45728]: osd.4 [v2:172.21.15.71:6808/3095380305,v1:172.21.15.71:6809/3095380305] boot 2024-02-02T15:22:00.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:59 smithi142 ceph-mon[45728]: osdmap e31: 5 total, 5 up, 5 in 2024-02-02T15:22:00.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:21:59 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-02T15:22:00.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:59 smithi001 ceph-mon[35470]: purged_snaps scrub starts 2024-02-02T15:22:00.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:59 smithi001 ceph-mon[35470]: purged_snaps scrub ok 2024-02-02T15:22:00.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:59 smithi001 ceph-mon[35470]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:22:00.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:59 smithi001 ceph-mon[35470]: osd.4 [v2:172.21.15.71:6808/3095380305,v1:172.21.15.71:6809/3095380305] boot 2024-02-02T15:22:00.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:59 smithi001 ceph-mon[35470]: osdmap e31: 5 total, 5 up, 5 in 2024-02-02T15:22:00.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:21:59 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-02T15:22:01.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:00 smithi071 ceph-mon[44707]: osdmap e32: 5 total, 5 up, 5 in 2024-02-02T15:22:01.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:00 smithi071 ceph-mon[44707]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-02-02T15:22:01.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:00 smithi071 ceph-mon[44707]: from='client.? 172.21.15.142:0/1688883500' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb48160e-ea7c-42f7-9477-be110bc8a34e"}]: dispatch 2024-02-02T15:22:01.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:00 smithi071 ceph-mon[44707]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb48160e-ea7c-42f7-9477-be110bc8a34e"}]: dispatch 2024-02-02T15:22:01.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:00 smithi071 ceph-mon[44707]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eb48160e-ea7c-42f7-9477-be110bc8a34e"}]': finished 2024-02-02T15:22:01.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:00 smithi071 ceph-mon[44707]: osdmap e33: 6 total, 5 up, 6 in 2024-02-02T15:22:01.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:00 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-02T15:22:01.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:00 smithi142 ceph-mon[45728]: osdmap e32: 5 total, 5 up, 5 in 2024-02-02T15:22:01.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:00 smithi142 ceph-mon[45728]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-02-02T15:22:01.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:00 smithi142 ceph-mon[45728]: from='client.? 172.21.15.142:0/1688883500' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb48160e-ea7c-42f7-9477-be110bc8a34e"}]: dispatch 2024-02-02T15:22:01.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:00 smithi142 ceph-mon[45728]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb48160e-ea7c-42f7-9477-be110bc8a34e"}]: dispatch 2024-02-02T15:22:01.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:00 smithi142 ceph-mon[45728]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eb48160e-ea7c-42f7-9477-be110bc8a34e"}]': finished 2024-02-02T15:22:01.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:00 smithi142 ceph-mon[45728]: osdmap e33: 6 total, 5 up, 6 in 2024-02-02T15:22:01.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:00 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-02T15:22:01.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:00 smithi001 ceph-mon[35470]: osdmap e32: 5 total, 5 up, 5 in 2024-02-02T15:22:01.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:00 smithi001 ceph-mon[35470]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-02-02T15:22:01.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:00 smithi001 ceph-mon[35470]: from='client.? 172.21.15.142:0/1688883500' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb48160e-ea7c-42f7-9477-be110bc8a34e"}]: dispatch 2024-02-02T15:22:01.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:00 smithi001 ceph-mon[35470]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb48160e-ea7c-42f7-9477-be110bc8a34e"}]: dispatch 2024-02-02T15:22:01.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:00 smithi001 ceph-mon[35470]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eb48160e-ea7c-42f7-9477-be110bc8a34e"}]': finished 2024-02-02T15:22:01.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:00 smithi001 ceph-mon[35470]: osdmap e33: 6 total, 5 up, 6 in 2024-02-02T15:22:01.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:00 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-02T15:22:02.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:01 smithi071 ceph-mon[44707]: from='client.? 172.21.15.142:0/1206769703' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-02T15:22:02.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:01 smithi142 ceph-mon[45728]: from='client.? 172.21.15.142:0/1206769703' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-02T15:22:02.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:01 smithi001 ceph-mon[35470]: from='client.? 172.21.15.142:0/1206769703' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-02T15:22:03.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:02 smithi071 ceph-mon[44707]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-02-02T15:22:03.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:02 smithi142 ceph-mon[45728]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-02-02T15:22:03.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:02 smithi001 ceph-mon[35470]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-02-02T15:22:05.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:04 smithi142 ceph-mon[45728]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-02-02T15:22:05.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:04 smithi001 ceph-mon[35470]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-02-02T15:22:05.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:04 smithi071 ceph-mon[44707]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-02-02T15:22:07.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:06 smithi071 ceph-mon[44707]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-02-02T15:22:07.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:06 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-02T15:22:07.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:06 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:22:07.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:06 smithi071 ceph-mon[44707]: Deploying daemon osd.5 on smithi142 2024-02-02T15:22:07.152 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:06 smithi142 ceph-mon[45728]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-02-02T15:22:07.152 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:06 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-02T15:22:07.152 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:06 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:22:07.152 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:06 smithi142 ceph-mon[45728]: Deploying daemon osd.5 on smithi142 2024-02-02T15:22:07.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:06 smithi001 ceph-mon[35470]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-02-02T15:22:07.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:06 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-02T15:22:07.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:06 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:22:07.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:06 smithi001 ceph-mon[35470]: Deploying daemon osd.5 on smithi142 2024-02-02T15:22:09.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:08 smithi001 ceph-mon[35470]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-02-02T15:22:09.192 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:08 smithi142 ceph-mon[45728]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-02-02T15:22:09.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:08 smithi071 ceph-mon[44707]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-02-02T15:22:10.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:10 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:10.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:10 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:10.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:10 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:22:10.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:10 smithi142 ceph-mon[45728]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-02-02T15:22:11.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:10 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:11.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:10 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:11.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:10 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:22:11.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:10 smithi071 ceph-mon[44707]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-02-02T15:22:11.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:10 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:11.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:10 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:11.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:10 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:22:11.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:10 smithi001 ceph-mon[35470]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-02-02T15:22:11.795 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:22:11.794+0000 7f1cbe7fc700 1 -- 172.21.15.142:0/826194082 <== mgr.14154 v2:172.21.15.1:6800/3768989981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f1cc8061180 con 0x7f1cb40a05a0 2024-02-02T15:22:11.795 INFO:teuthology.orchestra.run.smithi142.stdout:Created osd(s) 5 on host 'smithi142' 2024-02-02T15:22:11.796 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:22:11.796+0000 7f1ccf79e700 1 -- 172.21.15.142:0/826194082 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f1cb40a05a0 msgr2=0x7f1cb40a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:22:11.796 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:22:11.796+0000 7f1ccf79e700 1 --2- 172.21.15.142:0/826194082 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f1cb40a05a0 0x7f1cb40a2a60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1cb8003790 tx=0x7f1cb8005fb0 comp rx=0 tx=0).stop 2024-02-02T15:22:11.796 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:22:11.796+0000 7f1ccf79e700 1 -- 172.21.15.142:0/826194082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1cc8102970 msgr2=0x7f1cc819e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:22:11.796 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:22:11.796+0000 7f1ccf79e700 1 --2- 172.21.15.142:0/826194082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1cc8102970 0x7f1cc819e0f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1cc0007f10 tx=0x7f1cc000d3b0 comp rx=0 tx=0).stop 2024-02-02T15:22:11.796 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:22:11.796+0000 7f1ccf79e700 1 -- 172.21.15.142:0/826194082 shutdown_connections 2024-02-02T15:22:11.796 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:22:11.796+0000 7f1ccf79e700 1 --2- 172.21.15.142:0/826194082 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f1cb40a05a0 0x7f1cb40a2a60 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:22:11.796 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:22:11.796+0000 7f1ccf79e700 1 --2- 172.21.15.142:0/826194082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1cc8102970 0x7f1cc819e0f0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:22:11.797 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:22:11.796+0000 7f1ccf79e700 1 --2- 172.21.15.142:0/826194082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1cc8100dd0 0x7f1cc8196b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:22:11.797 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:22:11.796+0000 7f1ccf79e700 1 --2- 172.21.15.142:0/826194082 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1cc8101fd0 0x7f1cc8197070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:22:11.797 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:22:11.796+0000 7f1ccf79e700 1 -- 172.21.15.142:0/826194082 >> 172.21.15.142:0/826194082 conn(0x7f1cc80fc330 msgr2=0x7f1cc80fe620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:22:11.797 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:22:11.796+0000 7f1ccf79e700 1 -- 172.21.15.142:0/826194082 shutdown_connections 2024-02-02T15:22:11.797 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:22:11.796+0000 7f1ccf79e700 1 -- 172.21.15.142:0/826194082 wait complete. 2024-02-02T15:22:12.172 DEBUG:teuthology.orchestra.run.smithi142:osd.5> sudo journalctl -f -n 0 -u ceph-371a409c-c1de-11ee-95b6-87774f69a715@osd.5.service 2024-02-02T15:22:12.174 INFO:tasks.cephadm:Deploying osd.6 on smithi001 with /dev/vg_nvme/lv_2... 2024-02-02T15:22:12.174 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-02-02T15:22:12.205 INFO:journalctl@ceph.osd.5.smithi142.stdout:-- Logs begin at Fri 2024-02-02 14:58:43 UTC. -- 2024-02-02T15:22:12.545 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.a/config 2024-02-02T15:22:12.624 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:12 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:12.625 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:12 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:12.625 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:12 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:12.625 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:12 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:12.625 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:12 smithi142 ceph-mon[45728]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-02-02T15:22:12.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:12 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:12.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:12 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:12.678 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:12 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:12.679 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:12 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:12.679 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:12 smithi001 ceph-mon[35470]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-02-02T15:22:12.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:12 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:12.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:12 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:12.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:12 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:12.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:12 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:12.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:12 smithi071 ceph-mon[44707]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-02-02T15:22:14.003 INFO:teuthology.orchestra.run.smithi001.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-02-02T15:22:14.003 INFO:teuthology.orchestra.run.smithi001.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-02-02T15:22:14.003 INFO:teuthology.orchestra.run.smithi001.stderr: stderr: 10+0 records in 2024-02-02T15:22:14.003 INFO:teuthology.orchestra.run.smithi001.stderr:10+0 records out 2024-02-02T15:22:14.003 INFO:teuthology.orchestra.run.smithi001.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0303248 s, 346 MB/s 2024-02-02T15:22:14.003 INFO:teuthology.orchestra.run.smithi001.stderr:--> Zapping successful for: 2024-02-02T15:22:14.039 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:13 smithi001 ceph-mon[35470]: from='osd.5 [v2:172.21.15.142:6808/4195149262,v1:172.21.15.142:6809/4195149262]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-02T15:22:14.039 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:13 smithi001 ceph-mon[35470]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-02T15:22:14.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:13 smithi071 ceph-mon[44707]: from='osd.5 [v2:172.21.15.142:6808/4195149262,v1:172.21.15.142:6809/4195149262]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-02T15:22:14.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:13 smithi071 ceph-mon[44707]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-02T15:22:14.116 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:13 smithi142 ceph-mon[45728]: from='osd.5 [v2:172.21.15.142:6808/4195149262,v1:172.21.15.142:6809/4195149262]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-02T15:22:14.117 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:13 smithi142 ceph-mon[45728]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-02T15:22:14.117 INFO:journalctl@ceph.osd.5.smithi142.stdout:Feb 02 15:22:13 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-5[59288]: 2024-02-02T15:22:13.749+0000 7f762e03a640 -1 osd.5 0 log_to_monitors true 2024-02-02T15:22:14.419 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph orch daemon add osd smithi001:vg_nvme/lv_2 2024-02-02T15:22:14.791 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.a/config 2024-02-02T15:22:15.070 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:14 smithi001 ceph-mon[35470]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-02T15:22:15.071 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:14 smithi001 ceph-mon[35470]: osdmap e34: 6 total, 5 up, 6 in 2024-02-02T15:22:15.071 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:14 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-02T15:22:15.071 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:14 smithi001 ceph-mon[35470]: from='osd.5 [v2:172.21.15.142:6808/4195149262,v1:172.21.15.142:6809/4195149262]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-02-02T15:22:15.071 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:14 smithi001 ceph-mon[35470]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-02-02T15:22:15.071 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:14 smithi001 ceph-mon[35470]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-02-02T15:22:15.071 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:14 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:15.071 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:14 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:15.071 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:14 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:22:15.071 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:14 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:22:15.071 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:14 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:15.071 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:14 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:22:15.071 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:14 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:22:15.072 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:14 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:15.072 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:14 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:22:15.072 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:14 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:22:15.072 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:14 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:22:15.072 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:14 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:15.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:14 smithi071 ceph-mon[44707]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-02T15:22:15.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:14 smithi071 ceph-mon[44707]: osdmap e34: 6 total, 5 up, 6 in 2024-02-02T15:22:15.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:14 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-02T15:22:15.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:14 smithi071 ceph-mon[44707]: from='osd.5 [v2:172.21.15.142:6808/4195149262,v1:172.21.15.142:6809/4195149262]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-02-02T15:22:15.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:14 smithi071 ceph-mon[44707]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-02-02T15:22:15.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:14 smithi071 ceph-mon[44707]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-02-02T15:22:15.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:14 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:15.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:14 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:15.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:14 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:22:15.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:14 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:22:15.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:14 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:15.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:14 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:22:15.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:14 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:22:15.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:14 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:15.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:14 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:22:15.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:14 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:22:15.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:14 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:22:15.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:14 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:15.170 INFO:journalctl@ceph.osd.5.smithi142.stdout:Feb 02 15:22:14 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-5[59288]: 2024-02-02T15:22:14.782+0000 7f762369e700 -1 osd.5 0 waiting for initial osdmap 2024-02-02T15:22:15.170 INFO:journalctl@ceph.osd.5.smithi142.stdout:Feb 02 15:22:14 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-5[59288]: 2024-02-02T15:22:14.789+0000 7f761d48f700 -1 osd.5 35 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-02T15:22:15.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:14 smithi142 ceph-mon[45728]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-02T15:22:15.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:14 smithi142 ceph-mon[45728]: osdmap e34: 6 total, 5 up, 6 in 2024-02-02T15:22:15.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:14 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-02T15:22:15.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:14 smithi142 ceph-mon[45728]: from='osd.5 [v2:172.21.15.142:6808/4195149262,v1:172.21.15.142:6809/4195149262]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-02-02T15:22:15.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:14 smithi142 ceph-mon[45728]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-02-02T15:22:15.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:14 smithi142 ceph-mon[45728]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2024-02-02T15:22:15.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:14 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:15.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:14 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:15.172 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:14 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:22:15.172 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:14 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:22:15.172 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:14 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:15.172 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:14 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:22:15.172 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:14 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:22:15.172 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:14 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:15.172 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:14 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:22:15.172 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:14 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:22:15.172 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:14 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:22:15.172 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:14 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:15.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:22:15.848+0000 7ffb66600700 1 -- 172.21.15.1:0/1974573820 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb60100dd0 msgr2=0x7ffb601011f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:22:15.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:22:15.848+0000 7ffb66600700 1 --2- 172.21.15.1:0/1974573820 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb60100dd0 0x7ffb601011f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ffb500099c0 tx=0x7ffb50009cd0 comp rx=0 tx=0).stop 2024-02-02T15:22:15.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:22:15.848+0000 7ffb66600700 1 -- 172.21.15.1:0/1974573820 shutdown_connections 2024-02-02T15:22:15.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:22:15.848+0000 7ffb66600700 1 --2- 172.21.15.1:0/1974573820 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffb60102970 0x7ffb60109220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:22:15.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:22:15.848+0000 7ffb66600700 1 --2- 172.21.15.1:0/1974573820 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb60100dd0 0x7ffb601011f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:22:15.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:22:15.848+0000 7ffb66600700 1 --2- 172.21.15.1:0/1974573820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb60101fd0 0x7ffb60102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:22:15.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:22:15.848+0000 7ffb66600700 1 -- 172.21.15.1:0/1974573820 >> 172.21.15.1:0/1974573820 conn(0x7ffb600fc330 msgr2=0x7ffb600fe7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:22:15.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:22:15.848+0000 7ffb66600700 1 -- 172.21.15.1:0/1974573820 shutdown_connections 2024-02-02T15:22:15.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:22:15.848+0000 7ffb66600700 1 -- 172.21.15.1:0/1974573820 wait complete. 2024-02-02T15:22:15.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:22:15.850+0000 7ffb66600700 1 Processor -- start 2024-02-02T15:22:15.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:22:15.850+0000 7ffb66600700 1 -- start start 2024-02-02T15:22:15.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:22:15.850+0000 7ffb66600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb60100dd0 0x7ffb60198d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:22:15.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:22:15.850+0000 7ffb66600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffb60101fd0 0x7ffb60199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:22:15.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:22:15.850+0000 7ffb66600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb60102970 0x7ffb601a0290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:22:15.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:22:15.850+0000 7ffb66600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb601a0860 con 0x7ffb60102970 2024-02-02T15:22:15.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:22:15.850+0000 7ffb66600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb601a09a0 con 0x7ffb60101fd0 2024-02-02T15:22:15.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:22:15.850+0000 7ffb66600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb601a0b60 con 0x7ffb60100dd0 2024-02-02T15:22:15.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:22:15.850+0000 7ffb5fe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb60102970 0x7ffb601a0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:22:15.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:22:15.850+0000 7ffb5f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb60100dd0 0x7ffb60198d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:22:15.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:22:15.850+0000 7ffb5ea00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffb60101fd0 0x7ffb60199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:22:15.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:22:15.850+0000 7ffb5f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb60100dd0 0x7ffb60198d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:53368/0 (socket says 172.21.15.1:53368) 2024-02-02T15:22:15.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:22:15.850+0000 7ffb5ea00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffb60101fd0 0x7ffb60199250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:44024/0 (socket says 172.21.15.1:44024) 2024-02-02T15:22:15.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:22:15.850+0000 7ffb5f400700 1 -- 172.21.15.1:0/1939015220 learned_addr learned my addr 172.21.15.1:0/1939015220 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:22:15.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:22:15.851+0000 7ffb5fe00700 1 -- 172.21.15.1:0/1939015220 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb60100dd0 msgr2=0x7ffb60198d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:22:15.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:22:15.851+0000 7ffb5fe00700 1 --2- 172.21.15.1:0/1939015220 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb60100dd0 0x7ffb60198d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:22:15.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:22:15.851+0000 7ffb5fe00700 1 -- 172.21.15.1:0/1939015220 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffb60101fd0 msgr2=0x7ffb60199250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:22:15.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:22:15.851+0000 7ffb5fe00700 1 --2- 172.21.15.1:0/1939015220 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffb60101fd0 0x7ffb60199250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:22:15.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:22:15.851+0000 7ffb5fe00700 1 -- 172.21.15.1:0/1939015220 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb500095f0 con 0x7ffb60102970 2024-02-02T15:22:15.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:22:15.851+0000 7ffb5ea00700 1 --2- 172.21.15.1:0/1939015220 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffb60101fd0 0x7ffb60199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:22:15.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:22:15.851+0000 7ffb5f400700 1 --2- 172.21.15.1:0/1939015220 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb60100dd0 0x7ffb60198d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:22:15.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:22:15.851+0000 7ffb5fe00700 1 --2- 172.21.15.1:0/1939015220 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb60102970 0x7ffb601a0290 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ffb58007f10 tx=0x7ffb5800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:22:15.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:22:15.851+0000 7ffb57e00700 1 -- 172.21.15.1:0/1939015220 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb58009ea0 con 0x7ffb60102970 2024-02-02T15:22:15.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:22:15.851+0000 7ffb66600700 1 -- 172.21.15.1:0/1939015220 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb601a0e40 con 0x7ffb60102970 2024-02-02T15:22:15.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:22:15.851+0000 7ffb57e00700 1 -- 172.21.15.1:0/1939015220 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffb5800f040 con 0x7ffb60102970 2024-02-02T15:22:15.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:22:15.851+0000 7ffb57e00700 1 -- 172.21.15.1:0/1939015220 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb58004350 con 0x7ffb60102970 2024-02-02T15:22:15.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:22:15.852+0000 7ffb66600700 1 -- 172.21.15.1:0/1939015220 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb601a1200 con 0x7ffb60102970 2024-02-02T15:22:15.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:22:15.853+0000 7ffb66600700 1 -- 172.21.15.1:0/1939015220 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb6010c690 con 0x7ffb60102970 2024-02-02T15:22:15.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:22:15.854+0000 7ffb57e00700 1 -- 172.21.15.1:0/1939015220 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) v1 ==== 135069+0+0 (secure 0 0 0) 0x7ffb5800db20 con 0x7ffb60102970 2024-02-02T15:22:15.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:22:15.856+0000 7ffb57e00700 1 --2- 172.21.15.1:0/1939015220 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7ffb400a0520 0x7ffb400a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:22:15.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:22:15.856+0000 7ffb57e00700 1 -- 172.21.15.1:0/1939015220 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4566+0+0 (secure 0 0 0) 0x7ffb580ca460 con 0x7ffb60102970 2024-02-02T15:22:15.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:22:15.856+0000 7ffb5f400700 1 --2- 172.21.15.1:0/1939015220 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7ffb400a0520 0x7ffb400a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:22:15.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:22:15.857+0000 7ffb5f400700 1 --2- 172.21.15.1:0/1939015220 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7ffb400a0520 0x7ffb400a29e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ffb50000c00 tx=0x7ffb50005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:22:15.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:22:15.860+0000 7ffb57e00700 1 -- 172.21.15.1:0/1939015220 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7ffb58094e00 con 0x7ffb60102970 2024-02-02T15:22:15.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:15 smithi001 ceph-mon[35470]: Detected new or changed devices on smithi142 2024-02-02T15:22:15.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:15 smithi001 ceph-mon[35470]: Adjusting osd_memory_target on smithi142 to 8556M 2024-02-02T15:22:15.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:15 smithi001 ceph-mon[35470]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-02-02T15:22:15.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:15 smithi001 ceph-mon[35470]: osdmap e35: 6 total, 5 up, 6 in 2024-02-02T15:22:15.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:15 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-02T15:22:15.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:15 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-02T15:22:15.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:15 smithi001 ceph-mon[35470]: osd.5 [v2:172.21.15.142:6808/4195149262,v1:172.21.15.142:6809/4195149262] boot 2024-02-02T15:22:15.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:15 smithi001 ceph-mon[35470]: osdmap e36: 6 total, 6 up, 6 in 2024-02-02T15:22:15.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:15 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-02T15:22:16.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:22:16.079+0000 7ffb66600700 1 -- 172.21.15.1:0/1939015220 --> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7ffb60108910 con 0x7ffb400a0520 2024-02-02T15:22:16.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:15 smithi071 ceph-mon[44707]: Detected new or changed devices on smithi142 2024-02-02T15:22:16.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:15 smithi071 ceph-mon[44707]: Adjusting osd_memory_target on smithi142 to 8556M 2024-02-02T15:22:16.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:15 smithi071 ceph-mon[44707]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-02-02T15:22:16.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:15 smithi071 ceph-mon[44707]: osdmap e35: 6 total, 5 up, 6 in 2024-02-02T15:22:16.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:15 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-02T15:22:16.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:15 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-02T15:22:16.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:15 smithi071 ceph-mon[44707]: osd.5 [v2:172.21.15.142:6808/4195149262,v1:172.21.15.142:6809/4195149262] boot 2024-02-02T15:22:16.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:15 smithi071 ceph-mon[44707]: osdmap e36: 6 total, 6 up, 6 in 2024-02-02T15:22:16.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:15 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-02T15:22:16.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:15 smithi142 ceph-mon[45728]: Detected new or changed devices on smithi142 2024-02-02T15:22:16.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:15 smithi142 ceph-mon[45728]: Adjusting osd_memory_target on smithi142 to 8556M 2024-02-02T15:22:16.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:15 smithi142 ceph-mon[45728]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-02-02T15:22:16.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:15 smithi142 ceph-mon[45728]: osdmap e35: 6 total, 5 up, 6 in 2024-02-02T15:22:16.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:15 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-02T15:22:16.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:15 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-02T15:22:16.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:15 smithi142 ceph-mon[45728]: osd.5 [v2:172.21.15.142:6808/4195149262,v1:172.21.15.142:6809/4195149262] boot 2024-02-02T15:22:16.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:15 smithi142 ceph-mon[45728]: osdmap e36: 6 total, 6 up, 6 in 2024-02-02T15:22:16.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:15 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-02T15:22:16.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:16 smithi001 ceph-mon[35470]: purged_snaps scrub starts 2024-02-02T15:22:16.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:16 smithi001 ceph-mon[35470]: purged_snaps scrub ok 2024-02-02T15:22:16.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:16 smithi001 ceph-mon[35470]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2024-02-02T15:22:16.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:16 smithi001 ceph-mon[35470]: from='client.14316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:22:16.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:16 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-02T15:22:16.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:16 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-02T15:22:16.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:16 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:22:16.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:16 smithi001 ceph-mon[35470]: osdmap e37: 6 total, 6 up, 6 in 2024-02-02T15:22:17.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:16 smithi071 ceph-mon[44707]: purged_snaps scrub starts 2024-02-02T15:22:17.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:16 smithi071 ceph-mon[44707]: purged_snaps scrub ok 2024-02-02T15:22:17.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:16 smithi071 ceph-mon[44707]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2024-02-02T15:22:17.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:16 smithi071 ceph-mon[44707]: from='client.14316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:22:17.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:16 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-02T15:22:17.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:16 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-02T15:22:17.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:16 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:22:17.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:16 smithi071 ceph-mon[44707]: osdmap e37: 6 total, 6 up, 6 in 2024-02-02T15:22:17.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:16 smithi142 ceph-mon[45728]: purged_snaps scrub starts 2024-02-02T15:22:17.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:16 smithi142 ceph-mon[45728]: purged_snaps scrub ok 2024-02-02T15:22:17.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:16 smithi142 ceph-mon[45728]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2024-02-02T15:22:17.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:16 smithi142 ceph-mon[45728]: from='client.14316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:22:17.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:16 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-02T15:22:17.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:16 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-02T15:22:17.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:16 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:22:17.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:16 smithi142 ceph-mon[45728]: osdmap e37: 6 total, 6 up, 6 in 2024-02-02T15:22:19.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:18 smithi071 ceph-mon[44707]: osdmap e38: 6 total, 6 up, 6 in 2024-02-02T15:22:19.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:18 smithi071 ceph-mon[44707]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2024-02-02T15:22:19.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:18 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/897626517' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0ec4501a-5853-4ca6-b325-231f5b72335e"}]: dispatch 2024-02-02T15:22:19.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:18 smithi071 ceph-mon[44707]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0ec4501a-5853-4ca6-b325-231f5b72335e"}]: dispatch 2024-02-02T15:22:19.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:18 smithi071 ceph-mon[44707]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0ec4501a-5853-4ca6-b325-231f5b72335e"}]': finished 2024-02-02T15:22:19.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:18 smithi071 ceph-mon[44707]: osdmap e39: 7 total, 6 up, 7 in 2024-02-02T15:22:19.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:18 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-02T15:22:19.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:18 smithi142 ceph-mon[45728]: osdmap e38: 6 total, 6 up, 6 in 2024-02-02T15:22:19.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:18 smithi142 ceph-mon[45728]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2024-02-02T15:22:19.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:18 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/897626517' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0ec4501a-5853-4ca6-b325-231f5b72335e"}]: dispatch 2024-02-02T15:22:19.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:18 smithi142 ceph-mon[45728]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0ec4501a-5853-4ca6-b325-231f5b72335e"}]: dispatch 2024-02-02T15:22:19.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:18 smithi142 ceph-mon[45728]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0ec4501a-5853-4ca6-b325-231f5b72335e"}]': finished 2024-02-02T15:22:19.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:18 smithi142 ceph-mon[45728]: osdmap e39: 7 total, 6 up, 7 in 2024-02-02T15:22:19.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:18 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-02T15:22:19.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:18 smithi001 ceph-mon[35470]: osdmap e38: 6 total, 6 up, 6 in 2024-02-02T15:22:19.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:18 smithi001 ceph-mon[35470]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2024-02-02T15:22:19.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:18 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/897626517' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0ec4501a-5853-4ca6-b325-231f5b72335e"}]: dispatch 2024-02-02T15:22:19.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:18 smithi001 ceph-mon[35470]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0ec4501a-5853-4ca6-b325-231f5b72335e"}]: dispatch 2024-02-02T15:22:19.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:18 smithi001 ceph-mon[35470]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0ec4501a-5853-4ca6-b325-231f5b72335e"}]': finished 2024-02-02T15:22:19.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:18 smithi001 ceph-mon[35470]: osdmap e39: 7 total, 6 up, 7 in 2024-02-02T15:22:19.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:18 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-02T15:22:20.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:19 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/3297735971' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-02T15:22:20.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:19 smithi071 ceph-mon[44707]: osdmap e40: 7 total, 6 up, 7 in 2024-02-02T15:22:20.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:19 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-02T15:22:20.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:19 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/3297735971' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-02T15:22:20.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:19 smithi142 ceph-mon[45728]: osdmap e40: 7 total, 6 up, 7 in 2024-02-02T15:22:20.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:19 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-02T15:22:20.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:19 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/3297735971' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-02T15:22:20.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:19 smithi001 ceph-mon[35470]: osdmap e40: 7 total, 6 up, 7 in 2024-02-02T15:22:20.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:19 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-02T15:22:21.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:20 smithi071 ceph-mon[44707]: pgmap v117: 1 pgs: 1 peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-02-02T15:22:21.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:20 smithi142 ceph-mon[45728]: pgmap v117: 1 pgs: 1 peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-02-02T15:22:21.189 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:20 smithi001 ceph-mon[35470]: pgmap v117: 1 pgs: 1 peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-02-02T15:22:23.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:22 smithi001 ceph-mon[35470]: pgmap v118: 1 pgs: 1 peering; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 113 KiB/s, 0 objects/s recovering 2024-02-02T15:22:23.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:22 smithi071 ceph-mon[44707]: pgmap v118: 1 pgs: 1 peering; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 113 KiB/s, 0 objects/s recovering 2024-02-02T15:22:23.419 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:22 smithi142 ceph-mon[45728]: pgmap v118: 1 pgs: 1 peering; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 113 KiB/s, 0 objects/s recovering 2024-02-02T15:22:25.065 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:24 smithi001 ceph-mon[35470]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-02-02T15:22:25.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:24 smithi071 ceph-mon[44707]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-02-02T15:22:25.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:24 smithi142 ceph-mon[45728]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-02-02T15:22:26.031 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:25 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-02-02T15:22:26.031 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:25 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:22:26.031 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:25 smithi001 ceph-mon[35470]: Deploying daemon osd.6 on smithi001 2024-02-02T15:22:26.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:25 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-02-02T15:22:26.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:25 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:22:26.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:25 smithi071 ceph-mon[44707]: Deploying daemon osd.6 on smithi001 2024-02-02T15:22:26.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:25 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-02-02T15:22:26.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:25 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:22:26.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:25 smithi142 ceph-mon[45728]: Deploying daemon osd.6 on smithi001 2024-02-02T15:22:27.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:26 smithi001 ceph-mon[35470]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-02T15:22:27.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:26 smithi071 ceph-mon[44707]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-02T15:22:27.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:26 smithi142 ceph-mon[45728]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-02T15:22:29.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:28 smithi001 ceph-mon[35470]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-02-02T15:22:29.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:28 smithi071 ceph-mon[44707]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-02-02T15:22:29.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:28 smithi142 ceph-mon[45728]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-02-02T15:22:31.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:31 smithi071 ceph-mon[44707]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-02-02T15:22:31.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:31 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:31.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:31 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:31.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:31 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:22:31.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:31 smithi142 ceph-mon[45728]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-02-02T15:22:31.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:31 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:31.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:31 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:31.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:31 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:22:31.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:31 smithi001 ceph-mon[35470]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-02-02T15:22:31.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:31 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:31.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:31 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:31.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:31 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:22:32.982 INFO:teuthology.orchestra.run.smithi001.stdout:Created osd(s) 6 on host 'smithi001' 2024-02-02T15:22:32.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:22:32.981+0000 7ffb57e00700 1 -- 172.21.15.1:0/1939015220 <== mgr.14154 v2:172.21.15.1:6800/3768989981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ffb60108910 con 0x7ffb400a0520 2024-02-02T15:22:32.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:22:32.984+0000 7ffb66600700 1 -- 172.21.15.1:0/1939015220 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7ffb400a0520 msgr2=0x7ffb400a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:22:32.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:22:32.984+0000 7ffb66600700 1 --2- 172.21.15.1:0/1939015220 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7ffb400a0520 0x7ffb400a29e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ffb50000c00 tx=0x7ffb50005fb0 comp rx=0 tx=0).stop 2024-02-02T15:22:32.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:22:32.984+0000 7ffb66600700 1 -- 172.21.15.1:0/1939015220 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb60102970 msgr2=0x7ffb601a0290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:22:32.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:22:32.984+0000 7ffb66600700 1 --2- 172.21.15.1:0/1939015220 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb60102970 0x7ffb601a0290 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ffb58007f10 tx=0x7ffb5800d3b0 comp rx=0 tx=0).stop 2024-02-02T15:22:32.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:22:32.984+0000 7ffb66600700 1 -- 172.21.15.1:0/1939015220 shutdown_connections 2024-02-02T15:22:32.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:22:32.984+0000 7ffb66600700 1 --2- 172.21.15.1:0/1939015220 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb60102970 0x7ffb601a0290 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:22:32.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:22:32.984+0000 7ffb66600700 1 --2- 172.21.15.1:0/1939015220 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7ffb400a0520 0x7ffb400a29e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:22:32.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:22:32.984+0000 7ffb66600700 1 --2- 172.21.15.1:0/1939015220 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb60100dd0 0x7ffb60198d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:22:32.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:22:32.984+0000 7ffb66600700 1 --2- 172.21.15.1:0/1939015220 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffb60101fd0 0x7ffb60199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:22:32.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:22:32.984+0000 7ffb66600700 1 -- 172.21.15.1:0/1939015220 >> 172.21.15.1:0/1939015220 conn(0x7ffb600fc330 msgr2=0x7ffb600fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:22:32.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:22:32.984+0000 7ffb66600700 1 -- 172.21.15.1:0/1939015220 shutdown_connections 2024-02-02T15:22:32.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:22:32.984+0000 7ffb66600700 1 -- 172.21.15.1:0/1939015220 wait complete. 2024-02-02T15:22:33.200 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:33 smithi001 ceph-mon[35470]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-02-02T15:22:33.200 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:33 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:33.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:33 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:33.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:33 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:22:33.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:33 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:22:33.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:33 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:33.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:33 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:33.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:33 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:33.201 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:33 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:22:33.383 DEBUG:teuthology.orchestra.run.smithi001:osd.6> sudo journalctl -f -n 0 -u ceph-371a409c-c1de-11ee-95b6-87774f69a715@osd.6.service 2024-02-02T15:22:33.387 INFO:tasks.cephadm:Deploying osd.7 on smithi071 with /dev/vg_nvme/lv_2... 2024-02-02T15:22:33.387 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-02-02T15:22:33.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:33 smithi142 ceph-mon[45728]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-02-02T15:22:33.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:33 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:33.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:33 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:33.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:33 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:22:33.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:33 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:22:33.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:33 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:33.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:33 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:33.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:33 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:33.421 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:33 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:22:33.421 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:33 smithi071 ceph-mon[44707]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-02-02T15:22:33.421 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:33 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:33.421 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:33 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:33.421 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:33 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:22:33.421 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:33 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:22:33.421 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:33 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:33.421 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:33 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:33.421 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:33 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:33.422 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:33 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:22:33.463 INFO:journalctl@ceph.osd.6.smithi001.stdout:-- Logs begin at Fri 2024-02-02 15:05:03 UTC. -- 2024-02-02T15:22:33.617 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.b/config 2024-02-02T15:22:34.433 INFO:journalctl@ceph.osd.6.smithi001.stdout:Feb 02 15:22:34 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-6[69227]: 2024-02-02T15:22:34.156+0000 7f4ffae97640 -1 osd.6 0 log_to_monitors true 2024-02-02T15:22:34.843 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-02-02T15:22:34.843 INFO:teuthology.orchestra.run.smithi071.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-02-02T15:22:34.843 INFO:teuthology.orchestra.run.smithi071.stderr: stderr: 10+0 records in 2024-02-02T15:22:34.843 INFO:teuthology.orchestra.run.smithi071.stderr:10+0 records out 2024-02-02T15:22:34.843 INFO:teuthology.orchestra.run.smithi071.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112037 s, 936 MB/s 2024-02-02T15:22:34.843 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping successful for: 2024-02-02T15:22:35.211 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph orch daemon add osd smithi071:vg_nvme/lv_2 2024-02-02T15:22:35.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:35 smithi142 ceph-mon[45728]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-02-02T15:22:35.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:35 smithi142 ceph-mon[45728]: from='osd.6 [v2:172.21.15.1:6818/1560153428,v1:172.21.15.1:6819/1560153428]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-02T15:22:35.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:35 smithi001 ceph-mon[35470]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-02-02T15:22:35.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:35 smithi001 ceph-mon[35470]: from='osd.6 [v2:172.21.15.1:6818/1560153428,v1:172.21.15.1:6819/1560153428]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-02T15:22:35.469 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.b/config 2024-02-02T15:22:35.579 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:35 smithi071 ceph-mon[44707]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-02-02T15:22:35.580 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:35 smithi071 ceph-mon[44707]: from='osd.6 [v2:172.21.15.1:6818/1560153428,v1:172.21.15.1:6819/1560153428]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-02T15:22:36.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:36 smithi142 ceph-mon[45728]: from='osd.6 [v2:172.21.15.1:6818/1560153428,v1:172.21.15.1:6819/1560153428]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-02-02T15:22:36.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:36 smithi142 ceph-mon[45728]: osdmap e41: 7 total, 6 up, 7 in 2024-02-02T15:22:36.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:36 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-02T15:22:36.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:36 smithi142 ceph-mon[45728]: from='osd.6 [v2:172.21.15.1:6818/1560153428,v1:172.21.15.1:6819/1560153428]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-02-02T15:22:36.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:36 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:36.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:36 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:36.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:36 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:22:36.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:36 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:22:36.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:36 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:22:36.421 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:36 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:36.421 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:36 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:22:36.421 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:36 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:22:36.421 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:36 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:36.440 INFO:journalctl@ceph.osd.6.smithi001.stdout:Feb 02 15:22:36 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-6[69227]: 2024-02-02T15:22:36.145+0000 7f4fef400700 -1 osd.6 0 waiting for initial osdmap 2024-02-02T15:22:36.440 INFO:journalctl@ceph.osd.6.smithi001.stdout:Feb 02 15:22:36 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-6[69227]: 2024-02-02T15:22:36.157+0000 7f4fe7000700 -1 osd.6 42 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-02T15:22:36.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:36 smithi001 ceph-mon[35470]: from='osd.6 [v2:172.21.15.1:6818/1560153428,v1:172.21.15.1:6819/1560153428]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-02-02T15:22:36.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:36 smithi001 ceph-mon[35470]: osdmap e41: 7 total, 6 up, 7 in 2024-02-02T15:22:36.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:36 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-02T15:22:36.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:36 smithi001 ceph-mon[35470]: from='osd.6 [v2:172.21.15.1:6818/1560153428,v1:172.21.15.1:6819/1560153428]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-02-02T15:22:36.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:36 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:36.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:36 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:36.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:36 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:22:36.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:36 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:22:36.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:36 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:22:36.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:36 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:36.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:36 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:22:36.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:36 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:22:36.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:36 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:36.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:22:36.459+0000 7f057060d700 1 -- 172.21.15.71:0/714987135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0568101870 msgr2=0x7f0568108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:22:36.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:22:36.459+0000 7f057060d700 1 --2- 172.21.15.71:0/714987135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0568101870 0x7f0568108120 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f056000b0d0 tx=0x7f056000b3e0 comp rx=0 tx=0).stop 2024-02-02T15:22:36.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:22:36.460+0000 7f057060d700 1 -- 172.21.15.71:0/714987135 shutdown_connections 2024-02-02T15:22:36.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:22:36.460+0000 7f057060d700 1 --2- 172.21.15.71:0/714987135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0568101870 0x7f0568108120 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:22:36.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:22:36.460+0000 7f057060d700 1 --2- 172.21.15.71:0/714987135 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f05680ffcd0 0x7f05681000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:22:36.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:22:36.460+0000 7f057060d700 1 --2- 172.21.15.71:0/714987135 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0568100ed0 0x7f0568101330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:22:36.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:22:36.460+0000 7f057060d700 1 -- 172.21.15.71:0/714987135 >> 172.21.15.71:0/714987135 conn(0x7f05680fb250 msgr2=0x7f05680fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:22:36.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:22:36.460+0000 7f057060d700 1 -- 172.21.15.71:0/714987135 shutdown_connections 2024-02-02T15:22:36.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:22:36.460+0000 7f057060d700 1 -- 172.21.15.71:0/714987135 wait complete. 2024-02-02T15:22:36.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:22:36.460+0000 7f057060d700 1 Processor -- start 2024-02-02T15:22:36.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:22:36.460+0000 7f057060d700 1 -- start start 2024-02-02T15:22:36.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:22:36.460+0000 7f057060d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f05680ffcd0 0x7f0568195ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:22:36.462 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:22:36.460+0000 7f057060d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0568100ed0 0x7f0568196000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:22:36.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:22:36.460+0000 7f057060d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0568101870 0x7f056819d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:22:36.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:22:36.460+0000 7f057060d700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f056819d650 con 0x7f05680ffcd0 2024-02-02T15:22:36.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:22:36.460+0000 7f057060d700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f056819d790 con 0x7f0568100ed0 2024-02-02T15:22:36.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:22:36.460+0000 7f057060d700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f056819d920 con 0x7f0568101870 2024-02-02T15:22:36.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:22:36.461+0000 7f056dba8700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0568100ed0 0x7f0568196000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:22:36.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:22:36.461+0000 7f056dba8700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0568100ed0 0x7f0568196000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:40006/0 (socket says 172.21.15.71:40006) 2024-02-02T15:22:36.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:22:36.461+0000 7f056dba8700 1 -- 172.21.15.71:0/4189580608 learned_addr learned my addr 172.21.15.71:0/4189580608 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-02T15:22:36.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:22:36.461+0000 7f056ebaa700 1 --2- 172.21.15.71:0/4189580608 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0568101870 0x7f056819d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:22:36.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:22:36.461+0000 7f056dba8700 1 -- 172.21.15.71:0/4189580608 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0568101870 msgr2=0x7f056819d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:22:36.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:22:36.461+0000 7f056dba8700 1 --2- 172.21.15.71:0/4189580608 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0568101870 0x7f056819d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:22:36.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:22:36.461+0000 7f056dba8700 1 -- 172.21.15.71:0/4189580608 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f05680ffcd0 msgr2=0x7f0568195ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:22:36.463 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:22:36.461+0000 7f056dba8700 1 --2- 172.21.15.71:0/4189580608 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f05680ffcd0 0x7f0568195ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:22:36.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:22:36.461+0000 7f056dba8700 1 -- 172.21.15.71:0/4189580608 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0560009d00 con 0x7f0568100ed0 2024-02-02T15:22:36.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:22:36.461+0000 7f056ebaa700 1 --2- 172.21.15.71:0/4189580608 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0568101870 0x7f056819d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:22:36.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:22:36.461+0000 7f056dba8700 1 --2- 172.21.15.71:0/4189580608 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0568100ed0 0x7f0568196000 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f055800b530 tx=0x7f055800b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:22:36.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:22:36.461+0000 7f055f7fe700 1 -- 172.21.15.71:0/4189580608 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0558011830 con 0x7f0568100ed0 2024-02-02T15:22:36.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:22:36.461+0000 7f055f7fe700 1 -- 172.21.15.71:0/4189580608 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0558011e50 con 0x7f0568100ed0 2024-02-02T15:22:36.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:22:36.461+0000 7f057060d700 1 -- 172.21.15.71:0/4189580608 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f056819dc00 con 0x7f0568100ed0 2024-02-02T15:22:36.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:22:36.461+0000 7f055f7fe700 1 -- 172.21.15.71:0/4189580608 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f055800f2e0 con 0x7f0568100ed0 2024-02-02T15:22:36.464 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:22:36.461+0000 7f057060d700 1 -- 172.21.15.71:0/4189580608 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f056819e150 con 0x7f0568100ed0 2024-02-02T15:22:36.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:22:36.462+0000 7f057060d700 1 -- 172.21.15.71:0/4189580608 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f056810b550 con 0x7f0568100ed0 2024-02-02T15:22:36.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:22:36.463+0000 7f055f7fe700 1 -- 172.21.15.71:0/4189580608 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 135069+0+0 (secure 0 0 0) 0x7f0558010580 con 0x7f0568100ed0 2024-02-02T15:22:36.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:22:36.464+0000 7f055f7fe700 1 --2- 172.21.15.71:0/4189580608 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f05540a05a0 0x7f05540a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:22:36.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:22:36.464+0000 7f055f7fe700 1 -- 172.21.15.71:0/4189580608 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4685+0+0 (secure 0 0 0) 0x7f05580ca670 con 0x7f0568100ed0 2024-02-02T15:22:36.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:22:36.464+0000 7f056e3a9700 1 --2- 172.21.15.71:0/4189580608 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f05540a05a0 0x7f05540a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:22:36.465 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:22:36.465+0000 7f056e3a9700 1 --2- 172.21.15.71:0/4189580608 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f05540a05a0 0x7f05540a2a60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0568100d30 tx=0x7f0564009470 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:22:36.468 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:22:36.468+0000 7f055f7fe700 1 -- 172.21.15.71:0/4189580608 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f0558094fa0 con 0x7f0568100ed0 2024-02-02T15:22:36.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:36 smithi071 ceph-mon[44707]: from='osd.6 [v2:172.21.15.1:6818/1560153428,v1:172.21.15.1:6819/1560153428]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-02-02T15:22:36.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:36 smithi071 ceph-mon[44707]: osdmap e41: 7 total, 6 up, 7 in 2024-02-02T15:22:36.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:36 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-02T15:22:36.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:36 smithi071 ceph-mon[44707]: from='osd.6 [v2:172.21.15.1:6818/1560153428,v1:172.21.15.1:6819/1560153428]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-02-02T15:22:36.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:36 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:36.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:36 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:36.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:36 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:22:36.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:36 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:22:36.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:36 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:22:36.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:36 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:36.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:36 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:22:36.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:36 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:22:36.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:36 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:36.665 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:22:36.665+0000 7f057060d700 1 -- 172.21.15.71:0/4189580608 --> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f0568061180 con 0x7f05540a05a0 2024-02-02T15:22:37.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:37 smithi071 ceph-mon[44707]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-02-02T15:22:37.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:37 smithi071 ceph-mon[44707]: Detected new or changed devices on smithi001 2024-02-02T15:22:37.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:37 smithi071 ceph-mon[44707]: Adjusting osd_memory_target on smithi001 to 5758M 2024-02-02T15:22:37.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:37 smithi071 ceph-mon[44707]: from='osd.6 [v2:172.21.15.1:6818/1560153428,v1:172.21.15.1:6819/1560153428]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]': finished 2024-02-02T15:22:37.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:37 smithi071 ceph-mon[44707]: osdmap e42: 7 total, 6 up, 7 in 2024-02-02T15:22:37.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:37 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-02T15:22:37.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:37 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-02T15:22:37.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:37 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-02T15:22:37.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:37 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-02T15:22:37.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:37 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:22:37.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:37 smithi142 ceph-mon[45728]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-02-02T15:22:37.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:37 smithi142 ceph-mon[45728]: Detected new or changed devices on smithi001 2024-02-02T15:22:37.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:37 smithi142 ceph-mon[45728]: Adjusting osd_memory_target on smithi001 to 5758M 2024-02-02T15:22:37.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:37 smithi142 ceph-mon[45728]: from='osd.6 [v2:172.21.15.1:6818/1560153428,v1:172.21.15.1:6819/1560153428]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]': finished 2024-02-02T15:22:37.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:37 smithi142 ceph-mon[45728]: osdmap e42: 7 total, 6 up, 7 in 2024-02-02T15:22:37.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:37 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-02T15:22:37.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:37 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-02T15:22:37.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:37 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-02T15:22:37.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:37 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-02T15:22:37.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:37 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:22:37.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:37 smithi001 ceph-mon[35470]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-02-02T15:22:37.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:37 smithi001 ceph-mon[35470]: Detected new or changed devices on smithi001 2024-02-02T15:22:37.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:37 smithi001 ceph-mon[35470]: Adjusting osd_memory_target on smithi001 to 5758M 2024-02-02T15:22:37.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:37 smithi001 ceph-mon[35470]: from='osd.6 [v2:172.21.15.1:6818/1560153428,v1:172.21.15.1:6819/1560153428]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]': finished 2024-02-02T15:22:37.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:37 smithi001 ceph-mon[35470]: osdmap e42: 7 total, 6 up, 7 in 2024-02-02T15:22:37.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:37 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-02T15:22:37.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:37 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-02T15:22:37.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:37 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-02T15:22:37.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:37 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-02T15:22:37.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:37 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:22:38.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:38 smithi142 ceph-mon[45728]: purged_snaps scrub starts 2024-02-02T15:22:38.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:38 smithi142 ceph-mon[45728]: purged_snaps scrub ok 2024-02-02T15:22:38.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:38 smithi142 ceph-mon[45728]: from='client.24302 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:22:38.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:38 smithi142 ceph-mon[45728]: osd.6 [v2:172.21.15.1:6818/1560153428,v1:172.21.15.1:6819/1560153428] boot 2024-02-02T15:22:38.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:38 smithi142 ceph-mon[45728]: osdmap e43: 7 total, 7 up, 7 in 2024-02-02T15:22:38.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:38 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-02T15:22:38.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:38 smithi001 ceph-mon[35470]: purged_snaps scrub starts 2024-02-02T15:22:38.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:38 smithi001 ceph-mon[35470]: purged_snaps scrub ok 2024-02-02T15:22:38.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:38 smithi001 ceph-mon[35470]: from='client.24302 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:22:38.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:38 smithi001 ceph-mon[35470]: osd.6 [v2:172.21.15.1:6818/1560153428,v1:172.21.15.1:6819/1560153428] boot 2024-02-02T15:22:38.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:38 smithi001 ceph-mon[35470]: osdmap e43: 7 total, 7 up, 7 in 2024-02-02T15:22:38.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:38 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-02T15:22:38.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:38 smithi071 ceph-mon[44707]: purged_snaps scrub starts 2024-02-02T15:22:38.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:38 smithi071 ceph-mon[44707]: purged_snaps scrub ok 2024-02-02T15:22:38.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:38 smithi071 ceph-mon[44707]: from='client.24302 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:22:38.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:38 smithi071 ceph-mon[44707]: osd.6 [v2:172.21.15.1:6818/1560153428,v1:172.21.15.1:6819/1560153428] boot 2024-02-02T15:22:38.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:38 smithi071 ceph-mon[44707]: osdmap e43: 7 total, 7 up, 7 in 2024-02-02T15:22:38.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:38 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-02T15:22:39.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:39 smithi142 ceph-mon[45728]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-02-02T15:22:39.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:39 smithi142 ceph-mon[45728]: osdmap e44: 7 total, 7 up, 7 in 2024-02-02T15:22:39.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:39 smithi142 ceph-mon[45728]: from='client.? 172.21.15.71:0/2353637903' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "029debe8-9cdf-44f6-9aea-cabd1d185f93"}]: dispatch 2024-02-02T15:22:39.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:39 smithi142 ceph-mon[45728]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "029debe8-9cdf-44f6-9aea-cabd1d185f93"}]: dispatch 2024-02-02T15:22:39.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:39 smithi142 ceph-mon[45728]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "029debe8-9cdf-44f6-9aea-cabd1d185f93"}]': finished 2024-02-02T15:22:39.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:39 smithi142 ceph-mon[45728]: osdmap e45: 8 total, 7 up, 8 in 2024-02-02T15:22:39.421 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:39 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-02T15:22:39.421 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:39 smithi142 ceph-mon[45728]: from='client.? 172.21.15.71:0/2881605687' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-02T15:22:39.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:39 smithi001 ceph-mon[35470]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-02-02T15:22:39.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:39 smithi001 ceph-mon[35470]: osdmap e44: 7 total, 7 up, 7 in 2024-02-02T15:22:39.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:39 smithi001 ceph-mon[35470]: from='client.? 172.21.15.71:0/2353637903' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "029debe8-9cdf-44f6-9aea-cabd1d185f93"}]: dispatch 2024-02-02T15:22:39.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:39 smithi001 ceph-mon[35470]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "029debe8-9cdf-44f6-9aea-cabd1d185f93"}]: dispatch 2024-02-02T15:22:39.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:39 smithi001 ceph-mon[35470]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "029debe8-9cdf-44f6-9aea-cabd1d185f93"}]': finished 2024-02-02T15:22:39.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:39 smithi001 ceph-mon[35470]: osdmap e45: 8 total, 7 up, 8 in 2024-02-02T15:22:39.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:39 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-02T15:22:39.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:39 smithi001 ceph-mon[35470]: from='client.? 172.21.15.71:0/2881605687' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-02T15:22:39.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:39 smithi071 ceph-mon[44707]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-02-02T15:22:39.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:39 smithi071 ceph-mon[44707]: osdmap e44: 7 total, 7 up, 7 in 2024-02-02T15:22:39.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:39 smithi071 ceph-mon[44707]: from='client.? 172.21.15.71:0/2353637903' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "029debe8-9cdf-44f6-9aea-cabd1d185f93"}]: dispatch 2024-02-02T15:22:39.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:39 smithi071 ceph-mon[44707]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "029debe8-9cdf-44f6-9aea-cabd1d185f93"}]: dispatch 2024-02-02T15:22:39.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:39 smithi071 ceph-mon[44707]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "029debe8-9cdf-44f6-9aea-cabd1d185f93"}]': finished 2024-02-02T15:22:39.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:39 smithi071 ceph-mon[44707]: osdmap e45: 8 total, 7 up, 8 in 2024-02-02T15:22:39.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:39 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-02T15:22:39.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:39 smithi071 ceph-mon[44707]: from='client.? 172.21.15.71:0/2881605687' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-02T15:22:41.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:41 smithi142 ceph-mon[45728]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-02-02T15:22:41.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:41 smithi001 ceph-mon[35470]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-02-02T15:22:41.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:41 smithi071 ceph-mon[44707]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-02-02T15:22:43.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:43 smithi142 ceph-mon[45728]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-02-02T15:22:43.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:43 smithi001 ceph-mon[35470]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-02-02T15:22:43.572 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:43 smithi071 ceph-mon[44707]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-02-02T15:22:45.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:45 smithi071 ceph-mon[44707]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-02-02T15:22:45.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:45 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-02-02T15:22:45.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:45 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:22:45.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:45 smithi142 ceph-mon[45728]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-02-02T15:22:45.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:45 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-02-02T15:22:45.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:45 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:22:45.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:45 smithi001 ceph-mon[35470]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-02-02T15:22:45.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:45 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-02-02T15:22:45.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:45 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:22:46.329 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:46 smithi071 ceph-mon[44707]: Deploying daemon osd.7 on smithi071 2024-02-02T15:22:46.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:46 smithi142 ceph-mon[45728]: Deploying daemon osd.7 on smithi071 2024-02-02T15:22:46.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:46 smithi001 ceph-mon[35470]: Deploying daemon osd.7 on smithi071 2024-02-02T15:22:47.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:47 smithi142 ceph-mon[45728]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-02-02T15:22:47.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:47 smithi001 ceph-mon[35470]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-02-02T15:22:47.446 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:47 smithi071 ceph-mon[44707]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-02-02T15:22:49.296 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:48 smithi071 ceph-mon[44707]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-02-02T15:22:49.296 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:48 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:49.296 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:48 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:49.296 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:48 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:22:49.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:48 smithi142 ceph-mon[45728]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-02-02T15:22:49.421 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:48 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:49.421 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:48 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:49.421 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:48 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:22:49.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:48 smithi001 ceph-mon[35470]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-02-02T15:22:49.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:48 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:49.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:48 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:49.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:48 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:22:50.172 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:22:50.172+0000 7f055f7fe700 1 -- 172.21.15.71:0/4189580608 <== mgr.14154 v2:172.21.15.1:6800/3768989981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f0568061180 con 0x7f05540a05a0 2024-02-02T15:22:50.172 INFO:teuthology.orchestra.run.smithi071.stdout:Created osd(s) 7 on host 'smithi071' 2024-02-02T15:22:50.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:22:50.174+0000 7f057060d700 1 -- 172.21.15.71:0/4189580608 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f05540a05a0 msgr2=0x7f05540a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:22:50.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:22:50.174+0000 7f057060d700 1 --2- 172.21.15.71:0/4189580608 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f05540a05a0 0x7f05540a2a60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0568100d30 tx=0x7f0564009470 comp rx=0 tx=0).stop 2024-02-02T15:22:50.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:22:50.175+0000 7f057060d700 1 -- 172.21.15.71:0/4189580608 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0568100ed0 msgr2=0x7f0568196000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:22:50.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:22:50.175+0000 7f057060d700 1 --2- 172.21.15.71:0/4189580608 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0568100ed0 0x7f0568196000 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f055800b530 tx=0x7f055800b8f0 comp rx=0 tx=0).stop 2024-02-02T15:22:50.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:22:50.175+0000 7f057060d700 1 -- 172.21.15.71:0/4189580608 shutdown_connections 2024-02-02T15:22:50.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:22:50.175+0000 7f057060d700 1 --2- 172.21.15.71:0/4189580608 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f05540a05a0 0x7f05540a2a60 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:22:50.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:22:50.175+0000 7f057060d700 1 --2- 172.21.15.71:0/4189580608 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0568101870 0x7f056819d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:22:50.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:22:50.175+0000 7f057060d700 1 --2- 172.21.15.71:0/4189580608 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f05680ffcd0 0x7f0568195ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:22:50.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:22:50.175+0000 7f057060d700 1 --2- 172.21.15.71:0/4189580608 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0568100ed0 0x7f0568196000 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:22:50.174 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:22:50.175+0000 7f057060d700 1 -- 172.21.15.71:0/4189580608 >> 172.21.15.71:0/4189580608 conn(0x7f05680fb250 msgr2=0x7f05680fd460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:22:50.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:22:50.175+0000 7f057060d700 1 -- 172.21.15.71:0/4189580608 shutdown_connections 2024-02-02T15:22:50.175 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:22:50.175+0000 7f057060d700 1 -- 172.21.15.71:0/4189580608 wait complete. 2024-02-02T15:22:50.553 DEBUG:teuthology.orchestra.run.smithi071:osd.7> sudo journalctl -f -n 0 -u ceph-371a409c-c1de-11ee-95b6-87774f69a715@osd.7.service 2024-02-02T15:22:50.555 INFO:tasks.cephadm:Deploying osd.8 on smithi142 with /dev/vg_nvme/lv_2... 2024-02-02T15:22:50.555 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-02-02T15:22:50.598 INFO:journalctl@ceph.osd.7.smithi071.stdout:-- Logs begin at Fri 2024-02-02 14:58:45 UTC. -- 2024-02-02T15:22:50.762 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:50 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:50.762 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:50 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:50.762 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:50 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:22:50.762 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:50 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:22:50.762 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:50 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:50.762 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:50 smithi071 ceph-mon[44707]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-02-02T15:22:50.762 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:50 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:50.762 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:50 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:50.762 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:50 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:22:50.801 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.c/config 2024-02-02T15:22:50.894 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:50 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:50.894 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:50 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:50.894 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:50 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:22:50.894 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:50 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:22:50.895 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:50 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:50.895 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:50 smithi142 ceph-mon[45728]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-02-02T15:22:50.895 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:50 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:50.895 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:50 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:50.895 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:50 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:22:50.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:50 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:50.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:50 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:50.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:50 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:22:50.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:50 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:22:50.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:50 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:50.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:50 smithi001 ceph-mon[35470]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-02-02T15:22:50.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:50 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:50.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:50 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:50.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:50 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:22:52.075 INFO:teuthology.orchestra.run.smithi142.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-02-02T15:22:52.076 INFO:teuthology.orchestra.run.smithi142.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-02-02T15:22:52.076 INFO:teuthology.orchestra.run.smithi142.stderr: stderr: 10+0 records in 2024-02-02T15:22:52.076 INFO:teuthology.orchestra.run.smithi142.stderr:10+0 records out 2024-02-02T15:22:52.076 INFO:teuthology.orchestra.run.smithi142.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0111824 s, 938 MB/s 2024-02-02T15:22:52.076 INFO:teuthology.orchestra.run.smithi142.stderr:--> Zapping successful for: 2024-02-02T15:22:52.278 INFO:journalctl@ceph.osd.7.smithi071.stdout:Feb 02 15:22:51 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-7[65443]: 2024-02-02T15:22:51.984+0000 7f66cb647640 -1 osd.7 0 log_to_monitors true 2024-02-02T15:22:52.465 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph orch daemon add osd smithi142:vg_nvme/lv_2 2024-02-02T15:22:52.685 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.c/config 2024-02-02T15:22:53.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:52 smithi142 ceph-mon[45728]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-02-02T15:22:53.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:52 smithi142 ceph-mon[45728]: from='osd.7 [v2:172.21.15.71:6816/4080955538,v1:172.21.15.71:6817/4080955538]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-02T15:22:53.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:52 smithi142 ceph-mon[45728]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-02T15:22:53.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:52 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:53.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:52 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:53.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:52 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:22:53.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:52 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:22:53.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:52 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:22:53.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:52 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:53.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:52 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:22:53.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:52 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:22:53.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:52 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:53.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:52 smithi001 ceph-mon[35470]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-02-02T15:22:53.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:52 smithi001 ceph-mon[35470]: from='osd.7 [v2:172.21.15.71:6816/4080955538,v1:172.21.15.71:6817/4080955538]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-02T15:22:53.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:52 smithi001 ceph-mon[35470]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-02T15:22:53.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:52 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:53.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:52 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:53.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:52 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:22:53.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:52 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:22:53.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:52 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:22:53.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:52 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:53.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:52 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:22:53.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:52 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:22:53.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:52 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:53.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:52 smithi071 ceph-mon[44707]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-02-02T15:22:53.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:52 smithi071 ceph-mon[44707]: from='osd.7 [v2:172.21.15.71:6816/4080955538,v1:172.21.15.71:6817/4080955538]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-02T15:22:53.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:52 smithi071 ceph-mon[44707]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-02T15:22:53.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:52 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:53.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:52 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:53.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:52 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:22:53.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:52 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:22:53.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:52 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:22:53.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:52 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:53.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:52 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:22:53.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:52 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:22:53.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:52 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:22:53.766 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:22:53.763+0000 7f0528e46700 1 -- 172.21.15.142:0/2003268688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0524101880 msgr2=0x7f0524108130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:22:53.766 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:22:53.763+0000 7f0528e46700 1 --2- 172.21.15.142:0/2003268688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0524101880 0x7f0524108130 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f051c00b0a0 tx=0x7f051c00b3b0 comp rx=0 tx=0).stop 2024-02-02T15:22:53.766 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:22:53.763+0000 7f0528e46700 1 -- 172.21.15.142:0/2003268688 shutdown_connections 2024-02-02T15:22:53.766 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:22:53.763+0000 7f0528e46700 1 --2- 172.21.15.142:0/2003268688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0524101880 0x7f0524108130 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:22:53.766 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:22:53.763+0000 7f0528e46700 1 --2- 172.21.15.142:0/2003268688 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05240ffce0 0x7f0524100100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:22:53.766 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:22:53.763+0000 7f0528e46700 1 --2- 172.21.15.142:0/2003268688 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0524100ee0 0x7f0524101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:22:53.766 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:22:53.763+0000 7f0528e46700 1 -- 172.21.15.142:0/2003268688 >> 172.21.15.142:0/2003268688 conn(0x7f05240fb280 msgr2=0x7f05240fd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:22:53.766 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:22:53.763+0000 7f0528e46700 1 -- 172.21.15.142:0/2003268688 shutdown_connections 2024-02-02T15:22:53.767 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:22:53.763+0000 7f0528e46700 1 -- 172.21.15.142:0/2003268688 wait complete. 2024-02-02T15:22:53.767 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:22:53.764+0000 7f0528e46700 1 Processor -- start 2024-02-02T15:22:53.767 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:22:53.764+0000 7f0528e46700 1 -- start start 2024-02-02T15:22:53.767 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:22:53.764+0000 7f0528e46700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05240ffce0 0x7f0524195a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:22:53.767 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:22:53.764+0000 7f0528e46700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0524100ee0 0x7f0524195f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:22:53.767 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:22:53.764+0000 7f0528e46700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0524101880 0x7f052419d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:22:53.767 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:22:53.764+0000 7f0528e46700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f052419d5d0 con 0x7f0524100ee0 2024-02-02T15:22:53.767 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:22:53.764+0000 7f0528e46700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f052419d710 con 0x7f05240ffce0 2024-02-02T15:22:53.767 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:22:53.764+0000 7f0528e46700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f052419d8a0 con 0x7f0524101880 2024-02-02T15:22:53.767 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:22:53.764+0000 7f0522d9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0524101880 0x7f052419d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:22:53.767 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:22:53.764+0000 7f0522d9d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0524101880 0x7f052419d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:51970/0 (socket says 172.21.15.142:51970) 2024-02-02T15:22:53.767 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:22:53.764+0000 7f0522d9d700 1 -- 172.21.15.142:0/1897104823 learned_addr learned my addr 172.21.15.142:0/1897104823 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-02-02T15:22:53.767 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:22:53.764+0000 7f0521d9b700 1 --2- 172.21.15.142:0/1897104823 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0524100ee0 0x7f0524195f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:22:53.767 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:22:53.764+0000 7f052259c700 1 --2- 172.21.15.142:0/1897104823 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05240ffce0 0x7f0524195a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:22:53.767 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:22:53.765+0000 7f0522d9d700 1 -- 172.21.15.142:0/1897104823 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05240ffce0 msgr2=0x7f0524195a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:22:53.768 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:22:53.765+0000 7f0522d9d700 1 --2- 172.21.15.142:0/1897104823 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05240ffce0 0x7f0524195a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:22:53.768 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:22:53.765+0000 7f0522d9d700 1 -- 172.21.15.142:0/1897104823 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0524100ee0 msgr2=0x7f0524195f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:22:53.768 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:22:53.765+0000 7f0522d9d700 1 --2- 172.21.15.142:0/1897104823 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0524100ee0 0x7f0524195f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:22:53.768 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:22:53.765+0000 7f0522d9d700 1 -- 172.21.15.142:0/1897104823 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f051c009d00 con 0x7f0524101880 2024-02-02T15:22:53.768 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:22:53.765+0000 7f0521d9b700 1 --2- 172.21.15.142:0/1897104823 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0524100ee0 0x7f0524195f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:22:53.768 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:22:53.765+0000 7f0522d9d700 1 --2- 172.21.15.142:0/1897104823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0524101880 0x7f052419d000 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f051c004d00 tx=0x7f051c004de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:22:53.768 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:22:53.765+0000 7f051b7fe700 1 -- 172.21.15.142:0/1897104823 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051c00e040 con 0x7f0524101880 2024-02-02T15:22:53.768 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:22:53.765+0000 7f051b7fe700 1 -- 172.21.15.142:0/1897104823 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f051c01bd20 con 0x7f0524101880 2024-02-02T15:22:53.768 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:22:53.765+0000 7f0528e46700 1 -- 172.21.15.142:0/1897104823 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f052419db20 con 0x7f0524101880 2024-02-02T15:22:53.768 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:22:53.765+0000 7f051b7fe700 1 -- 172.21.15.142:0/1897104823 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051c005920 con 0x7f0524101880 2024-02-02T15:22:53.768 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:22:53.765+0000 7f0528e46700 1 -- 172.21.15.142:0/1897104823 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f052419dfc0 con 0x7f0524101880 2024-02-02T15:22:53.769 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:22:53.766+0000 7f0528e46700 1 -- 172.21.15.142:0/1897104823 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0524106bd0 con 0x7f0524101880 2024-02-02T15:22:53.769 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:22:53.768+0000 7f051b7fe700 1 -- 172.21.15.142:0/1897104823 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 16) v1 ==== 135069+0+0 (secure 0 0 0) 0x7f051c005040 con 0x7f0524101880 2024-02-02T15:22:53.770 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:22:53.768+0000 7f051b7fe700 1 --2- 172.21.15.142:0/1897104823 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f05100a0520 0x7f05100a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:22:53.770 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:22:53.768+0000 7f051b7fe700 1 -- 172.21.15.142:0/1897104823 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5101+0+0 (secure 0 0 0) 0x7f051c0ccf70 con 0x7f0524101880 2024-02-02T15:22:53.770 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:22:53.769+0000 7f052259c700 1 --2- 172.21.15.142:0/1897104823 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f05100a0520 0x7f05100a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:22:53.770 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:22:53.769+0000 7f052259c700 1 --2- 172.21.15.142:0/1897104823 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f05100a0520 0x7f05100a29e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0514005fd0 tx=0x7f0514005dd0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:22:53.772 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:22:53.772+0000 7f051b7fe700 1 -- 172.21.15.142:0/1897104823 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f051c097830 con 0x7f0524101880 2024-02-02T15:22:53.968 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:22:53.968+0000 7f0528e46700 1 -- 172.21.15.142:0/1897104823 --> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f0524061180 con 0x7f05100a0520 2024-02-02T15:22:54.333 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:53 smithi142 ceph-mon[45728]: Detected new or changed devices on smithi071 2024-02-02T15:22:54.334 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:53 smithi142 ceph-mon[45728]: Adjusting osd_memory_target on smithi071 to 5704M 2024-02-02T15:22:54.334 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:53 smithi142 ceph-mon[45728]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-02-02T15:22:54.334 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:53 smithi142 ceph-mon[45728]: osdmap e46: 8 total, 7 up, 8 in 2024-02-02T15:22:54.334 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:53 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-02T15:22:54.334 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:53 smithi142 ceph-mon[45728]: from='osd.7 [v2:172.21.15.71:6816/4080955538,v1:172.21.15.71:6817/4080955538]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-02-02T15:22:54.334 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:53 smithi142 ceph-mon[45728]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-02-02T15:22:54.364 INFO:journalctl@ceph.osd.7.smithi071.stdout:Feb 02 15:22:53 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-7[65443]: 2024-02-02T15:22:53.950+0000 7f66c0cab700 -1 osd.7 0 waiting for initial osdmap 2024-02-02T15:22:54.364 INFO:journalctl@ceph.osd.7.smithi071.stdout:Feb 02 15:22:53 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-7[65443]: 2024-02-02T15:22:53.962+0000 7f66bcaa0700 -1 osd.7 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-02T15:22:54.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:53 smithi071 ceph-mon[44707]: Detected new or changed devices on smithi071 2024-02-02T15:22:54.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:53 smithi071 ceph-mon[44707]: Adjusting osd_memory_target on smithi071 to 5704M 2024-02-02T15:22:54.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:53 smithi071 ceph-mon[44707]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-02-02T15:22:54.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:53 smithi071 ceph-mon[44707]: osdmap e46: 8 total, 7 up, 8 in 2024-02-02T15:22:54.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:53 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-02T15:22:54.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:53 smithi071 ceph-mon[44707]: from='osd.7 [v2:172.21.15.71:6816/4080955538,v1:172.21.15.71:6817/4080955538]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-02-02T15:22:54.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:53 smithi071 ceph-mon[44707]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-02-02T15:22:54.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:53 smithi001 ceph-mon[35470]: Detected new or changed devices on smithi071 2024-02-02T15:22:54.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:53 smithi001 ceph-mon[35470]: Adjusting osd_memory_target on smithi071 to 5704M 2024-02-02T15:22:54.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:53 smithi001 ceph-mon[35470]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-02-02T15:22:54.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:53 smithi001 ceph-mon[35470]: osdmap e46: 8 total, 7 up, 8 in 2024-02-02T15:22:54.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:53 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-02T15:22:54.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:53 smithi001 ceph-mon[35470]: from='osd.7 [v2:172.21.15.71:6816/4080955538,v1:172.21.15.71:6817/4080955538]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-02-02T15:22:54.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:53 smithi001 ceph-mon[35470]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-02-02T15:22:55.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:54 smithi071 ceph-mon[44707]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-02-02T15:22:55.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:54 smithi071 ceph-mon[44707]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-02-02T15:22:55.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:54 smithi071 ceph-mon[44707]: osdmap e47: 8 total, 7 up, 8 in 2024-02-02T15:22:55.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:54 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-02T15:22:55.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:54 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-02T15:22:55.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:54 smithi071 ceph-mon[44707]: from='client.24316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:22:55.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:54 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-02T15:22:55.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:54 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-02T15:22:55.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:54 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:22:55.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:54 smithi142 ceph-mon[45728]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-02-02T15:22:55.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:54 smithi142 ceph-mon[45728]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-02-02T15:22:55.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:54 smithi142 ceph-mon[45728]: osdmap e47: 8 total, 7 up, 8 in 2024-02-02T15:22:55.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:54 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-02T15:22:55.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:54 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-02T15:22:55.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:54 smithi142 ceph-mon[45728]: from='client.24316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:22:55.421 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:54 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-02T15:22:55.421 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:54 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-02T15:22:55.421 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:54 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:22:55.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:54 smithi001 ceph-mon[35470]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-02-02T15:22:55.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:54 smithi001 ceph-mon[35470]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-02-02T15:22:55.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:54 smithi001 ceph-mon[35470]: osdmap e47: 8 total, 7 up, 8 in 2024-02-02T15:22:55.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:54 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-02T15:22:55.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:54 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-02T15:22:55.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:54 smithi001 ceph-mon[35470]: from='client.24316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:22:55.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:54 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-02T15:22:55.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:54 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-02T15:22:55.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:54 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:22:56.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:55 smithi071 ceph-mon[44707]: purged_snaps scrub starts 2024-02-02T15:22:56.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:55 smithi071 ceph-mon[44707]: purged_snaps scrub ok 2024-02-02T15:22:56.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:55 smithi071 ceph-mon[44707]: osd.7 [v2:172.21.15.71:6816/4080955538,v1:172.21.15.71:6817/4080955538] boot 2024-02-02T15:22:56.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:55 smithi071 ceph-mon[44707]: osdmap e48: 8 total, 8 up, 8 in 2024-02-02T15:22:56.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:55 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-02T15:22:56.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:55 smithi071 ceph-mon[44707]: osdmap e49: 8 total, 8 up, 8 in 2024-02-02T15:22:56.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:55 smithi071 ceph-mon[44707]: from='client.? 172.21.15.142:0/1250333536' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7c90896c-9955-4c1b-aaca-7531dd5457f6"}]: dispatch 2024-02-02T15:22:56.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:55 smithi071 ceph-mon[44707]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7c90896c-9955-4c1b-aaca-7531dd5457f6"}]: dispatch 2024-02-02T15:22:56.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:55 smithi071 ceph-mon[44707]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7c90896c-9955-4c1b-aaca-7531dd5457f6"}]': finished 2024-02-02T15:22:56.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:55 smithi071 ceph-mon[44707]: osdmap e50: 9 total, 8 up, 9 in 2024-02-02T15:22:56.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:55 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-02T15:22:56.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:55 smithi142 ceph-mon[45728]: purged_snaps scrub starts 2024-02-02T15:22:56.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:55 smithi142 ceph-mon[45728]: purged_snaps scrub ok 2024-02-02T15:22:56.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:55 smithi142 ceph-mon[45728]: osd.7 [v2:172.21.15.71:6816/4080955538,v1:172.21.15.71:6817/4080955538] boot 2024-02-02T15:22:56.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:55 smithi142 ceph-mon[45728]: osdmap e48: 8 total, 8 up, 8 in 2024-02-02T15:22:56.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:55 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-02T15:22:56.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:55 smithi142 ceph-mon[45728]: osdmap e49: 8 total, 8 up, 8 in 2024-02-02T15:22:56.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:55 smithi142 ceph-mon[45728]: from='client.? 172.21.15.142:0/1250333536' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7c90896c-9955-4c1b-aaca-7531dd5457f6"}]: dispatch 2024-02-02T15:22:56.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:55 smithi142 ceph-mon[45728]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7c90896c-9955-4c1b-aaca-7531dd5457f6"}]: dispatch 2024-02-02T15:22:56.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:55 smithi142 ceph-mon[45728]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7c90896c-9955-4c1b-aaca-7531dd5457f6"}]': finished 2024-02-02T15:22:56.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:55 smithi142 ceph-mon[45728]: osdmap e50: 9 total, 8 up, 9 in 2024-02-02T15:22:56.421 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:55 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-02T15:22:56.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:55 smithi001 ceph-mon[35470]: purged_snaps scrub starts 2024-02-02T15:22:56.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:55 smithi001 ceph-mon[35470]: purged_snaps scrub ok 2024-02-02T15:22:56.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:55 smithi001 ceph-mon[35470]: osd.7 [v2:172.21.15.71:6816/4080955538,v1:172.21.15.71:6817/4080955538] boot 2024-02-02T15:22:56.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:55 smithi001 ceph-mon[35470]: osdmap e48: 8 total, 8 up, 8 in 2024-02-02T15:22:56.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:55 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-02T15:22:56.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:55 smithi001 ceph-mon[35470]: osdmap e49: 8 total, 8 up, 8 in 2024-02-02T15:22:56.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:55 smithi001 ceph-mon[35470]: from='client.? 172.21.15.142:0/1250333536' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7c90896c-9955-4c1b-aaca-7531dd5457f6"}]: dispatch 2024-02-02T15:22:56.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:55 smithi001 ceph-mon[35470]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7c90896c-9955-4c1b-aaca-7531dd5457f6"}]: dispatch 2024-02-02T15:22:56.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:55 smithi001 ceph-mon[35470]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7c90896c-9955-4c1b-aaca-7531dd5457f6"}]': finished 2024-02-02T15:22:56.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:55 smithi001 ceph-mon[35470]: osdmap e50: 9 total, 8 up, 9 in 2024-02-02T15:22:56.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:55 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-02T15:22:57.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:56 smithi071 ceph-mon[44707]: pgmap v145: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-02-02T15:22:57.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:56 smithi071 ceph-mon[44707]: from='client.? 172.21.15.142:0/2142186782' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-02T15:22:57.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:56 smithi071 ceph-mon[44707]: osdmap e51: 9 total, 8 up, 9 in 2024-02-02T15:22:57.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:56 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-02T15:22:57.419 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:56 smithi142 ceph-mon[45728]: pgmap v145: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-02-02T15:22:57.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:56 smithi142 ceph-mon[45728]: from='client.? 172.21.15.142:0/2142186782' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-02T15:22:57.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:56 smithi142 ceph-mon[45728]: osdmap e51: 9 total, 8 up, 9 in 2024-02-02T15:22:57.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:56 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-02T15:22:57.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:56 smithi001 ceph-mon[35470]: pgmap v145: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2024-02-02T15:22:57.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:56 smithi001 ceph-mon[35470]: from='client.? 172.21.15.142:0/2142186782' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-02T15:22:57.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:56 smithi001 ceph-mon[35470]: osdmap e51: 9 total, 8 up, 9 in 2024-02-02T15:22:57.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:56 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-02T15:22:59.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:58 smithi071 ceph-mon[44707]: osdmap e52: 9 total, 8 up, 9 in 2024-02-02T15:22:59.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:58 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-02T15:22:59.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:22:58 smithi071 ceph-mon[44707]: pgmap v148: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-02-02T15:22:59.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:58 smithi142 ceph-mon[45728]: osdmap e52: 9 total, 8 up, 9 in 2024-02-02T15:22:59.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:58 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-02T15:22:59.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:22:58 smithi142 ceph-mon[45728]: pgmap v148: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-02-02T15:22:59.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:58 smithi001 ceph-mon[35470]: osdmap e52: 9 total, 8 up, 9 in 2024-02-02T15:22:59.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:58 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-02T15:22:59.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:22:58 smithi001 ceph-mon[35470]: pgmap v148: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-02-02T15:23:01.060 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:00 smithi142 ceph-mon[45728]: pgmap v149: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-02-02T15:23:01.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:00 smithi001 ceph-mon[35470]: pgmap v149: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-02-02T15:23:01.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:00 smithi071 ceph-mon[44707]: pgmap v149: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-02-02T15:23:02.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:01 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-02-02T15:23:02.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:01 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:23:02.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:01 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-02-02T15:23:02.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:01 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:23:02.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:01 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-02-02T15:23:02.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:01 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:23:03.284 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:02 smithi142 ceph-mon[45728]: Deploying daemon osd.8 on smithi142 2024-02-02T15:23:03.284 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:02 smithi142 ceph-mon[45728]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-02-02T15:23:03.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:02 smithi071 ceph-mon[44707]: Deploying daemon osd.8 on smithi142 2024-02-02T15:23:03.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:02 smithi071 ceph-mon[44707]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-02-02T15:23:03.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:02 smithi001 ceph-mon[35470]: Deploying daemon osd.8 on smithi142 2024-02-02T15:23:03.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:02 smithi001 ceph-mon[35470]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-02-02T15:23:05.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:04 smithi142 ceph-mon[45728]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-02-02T15:23:05.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:04 smithi071 ceph-mon[44707]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-02-02T15:23:05.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:04 smithi001 ceph-mon[35470]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-02-02T15:23:06.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:06 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:06.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:06 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:06.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:06 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:23:06.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:06 smithi071 ceph-mon[44707]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-02-02T15:23:06.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:06 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:06.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:06 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:06.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:06 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:23:06.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:06 smithi142 ceph-mon[45728]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-02-02T15:23:06.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:06 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:06.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:06 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:06.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:06 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:23:06.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:06 smithi001 ceph-mon[35470]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-02-02T15:23:07.879 INFO:teuthology.orchestra.run.smithi142.stdout:Created osd(s) 8 on host 'smithi142' 2024-02-02T15:23:07.879 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:23:07.878+0000 7f051b7fe700 1 -- 172.21.15.142:0/1897104823 <== mgr.14154 v2:172.21.15.1:6800/3768989981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f0524061180 con 0x7f05100a0520 2024-02-02T15:23:07.881 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:23:07.881+0000 7f0528e46700 1 -- 172.21.15.142:0/1897104823 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f05100a0520 msgr2=0x7f05100a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:23:07.881 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:23:07.881+0000 7f0528e46700 1 --2- 172.21.15.142:0/1897104823 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f05100a0520 0x7f05100a29e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0514005fd0 tx=0x7f0514005dd0 comp rx=0 tx=0).stop 2024-02-02T15:23:07.881 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:23:07.881+0000 7f0528e46700 1 -- 172.21.15.142:0/1897104823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0524101880 msgr2=0x7f052419d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:23:07.881 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:23:07.881+0000 7f0528e46700 1 --2- 172.21.15.142:0/1897104823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0524101880 0x7f052419d000 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f051c004d00 tx=0x7f051c004de0 comp rx=0 tx=0).stop 2024-02-02T15:23:07.881 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:23:07.881+0000 7f0528e46700 1 -- 172.21.15.142:0/1897104823 shutdown_connections 2024-02-02T15:23:07.881 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:23:07.881+0000 7f0528e46700 1 --2- 172.21.15.142:0/1897104823 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f05100a0520 0x7f05100a29e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:23:07.881 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:23:07.881+0000 7f0528e46700 1 --2- 172.21.15.142:0/1897104823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0524101880 0x7f052419d000 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:23:07.881 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:23:07.881+0000 7f0528e46700 1 --2- 172.21.15.142:0/1897104823 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05240ffce0 0x7f0524195a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:23:07.881 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:23:07.881+0000 7f0528e46700 1 --2- 172.21.15.142:0/1897104823 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0524100ee0 0x7f0524195f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:23:07.881 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:23:07.881+0000 7f0528e46700 1 -- 172.21.15.142:0/1897104823 >> 172.21.15.142:0/1897104823 conn(0x7f05240fb280 msgr2=0x7f05240fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:23:07.881 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:23:07.881+0000 7f0528e46700 1 -- 172.21.15.142:0/1897104823 shutdown_connections 2024-02-02T15:23:07.882 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:23:07.881+0000 7f0528e46700 1 -- 172.21.15.142:0/1897104823 wait complete. 2024-02-02T15:23:08.242 DEBUG:teuthology.orchestra.run.smithi142:osd.8> sudo journalctl -f -n 0 -u ceph-371a409c-c1de-11ee-95b6-87774f69a715@osd.8.service 2024-02-02T15:23:08.244 INFO:tasks.cephadm:Deploying osd.9 on smithi001 with /dev/vg_nvme/lv_1... 2024-02-02T15:23:08.244 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-02-02T15:23:08.274 INFO:journalctl@ceph.osd.8.smithi142.stdout:-- Logs begin at Fri 2024-02-02 14:58:43 UTC. -- 2024-02-02T15:23:08.390 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:08 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:08.390 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:08 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:08.390 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:08 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:23:08.390 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:08 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:23:08.390 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:08 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:08.390 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:08 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:08.390 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:08 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:08.390 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:08 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:23:08.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:08 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:08.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:08 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:08.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:08 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:23:08.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:08 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:23:08.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:08 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:08.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:08 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:08.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:08 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:08.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:08 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:23:08.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:08 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:08.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:08 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:08.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:08 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:23:08.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:08 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:23:08.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:08 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:08.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:08 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:08.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:08 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:08.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:08 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:23:08.635 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.a/config 2024-02-02T15:23:09.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:09 smithi142 ceph-mon[45728]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-02-02T15:23:09.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:09 smithi001 ceph-mon[35470]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-02-02T15:23:09.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:09 smithi071 ceph-mon[44707]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-02-02T15:23:09.920 INFO:journalctl@ceph.osd.8.smithi142.stdout:Feb 02 15:23:09 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-8[67933]: 2024-02-02T15:23:09.550+0000 7f526af09640 -1 osd.8 0 log_to_monitors true 2024-02-02T15:23:10.183 INFO:teuthology.orchestra.run.smithi001.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-02-02T15:23:10.183 INFO:teuthology.orchestra.run.smithi001.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-02-02T15:23:10.183 INFO:teuthology.orchestra.run.smithi001.stderr: stderr: 10+0 records in 2024-02-02T15:23:10.183 INFO:teuthology.orchestra.run.smithi001.stderr:10+0 records out 2024-02-02T15:23:10.183 INFO:teuthology.orchestra.run.smithi001.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0309597 s, 339 MB/s 2024-02-02T15:23:10.184 INFO:teuthology.orchestra.run.smithi001.stderr:--> Zapping successful for: 2024-02-02T15:23:10.311 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:10 smithi142 ceph-mon[45728]: from='osd.8 [v2:172.21.15.142:6816/3942038209,v1:172.21.15.142:6817/3942038209]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-02-02T15:23:10.311 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:10 smithi142 ceph-mon[45728]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-02-02T15:23:10.400 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:10 smithi001 ceph-mon[35470]: from='osd.8 [v2:172.21.15.142:6816/3942038209,v1:172.21.15.142:6817/3942038209]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-02-02T15:23:10.400 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:10 smithi001 ceph-mon[35470]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-02-02T15:23:10.581 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph orch daemon add osd smithi001:vg_nvme/lv_1 2024-02-02T15:23:10.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:10 smithi071 ceph-mon[44707]: from='osd.8 [v2:172.21.15.142:6816/3942038209,v1:172.21.15.142:6817/3942038209]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-02-02T15:23:10.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:10 smithi071 ceph-mon[44707]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-02-02T15:23:10.944 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.a/config 2024-02-02T15:23:11.420 INFO:journalctl@ceph.osd.8.smithi142.stdout:Feb 02 15:23:11 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-8[67933]: 2024-02-02T15:23:11.167+0000 7f526056d700 -1 osd.8 0 waiting for initial osdmap 2024-02-02T15:23:11.420 INFO:journalctl@ceph.osd.8.smithi142.stdout:Feb 02 15:23:11 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-8[67933]: 2024-02-02T15:23:11.181+0000 7f525c362700 -1 osd.8 54 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-02T15:23:11.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:11 smithi142 ceph-mon[45728]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-02-02T15:23:11.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:11 smithi142 ceph-mon[45728]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-02-02T15:23:11.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:11 smithi142 ceph-mon[45728]: osdmap e53: 9 total, 8 up, 9 in 2024-02-02T15:23:11.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:11 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-02T15:23:11.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:11 smithi142 ceph-mon[45728]: from='osd.8 [v2:172.21.15.142:6816/3942038209,v1:172.21.15.142:6817/3942038209]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-02-02T15:23:11.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:11 smithi142 ceph-mon[45728]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-02-02T15:23:11.421 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:11 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:11.421 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:11 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:11.421 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:11 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:23:11.421 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:11 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:23:11.421 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:11 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:23:11.421 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:11 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:11.421 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:11 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:23:11.421 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:11 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:23:11.421 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:11 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:11.424 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:11 smithi001 ceph-mon[35470]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-02-02T15:23:11.424 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:11 smithi001 ceph-mon[35470]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-02-02T15:23:11.424 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:11 smithi001 ceph-mon[35470]: osdmap e53: 9 total, 8 up, 9 in 2024-02-02T15:23:11.425 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:11 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-02T15:23:11.425 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:11 smithi001 ceph-mon[35470]: from='osd.8 [v2:172.21.15.142:6816/3942038209,v1:172.21.15.142:6817/3942038209]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-02-02T15:23:11.425 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:11 smithi001 ceph-mon[35470]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-02-02T15:23:11.425 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:11 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:11.425 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:11 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:11.425 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:11 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:23:11.425 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:11 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:23:11.425 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:11 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:23:11.425 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:11 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:11.425 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:11 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:23:11.425 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:11 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:23:11.425 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:11 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:11.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:11 smithi071 ceph-mon[44707]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-02-02T15:23:11.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:11 smithi071 ceph-mon[44707]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-02-02T15:23:11.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:11 smithi071 ceph-mon[44707]: osdmap e53: 9 total, 8 up, 9 in 2024-02-02T15:23:11.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:11 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-02T15:23:11.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:11 smithi071 ceph-mon[44707]: from='osd.8 [v2:172.21.15.142:6816/3942038209,v1:172.21.15.142:6817/3942038209]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-02-02T15:23:11.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:11 smithi071 ceph-mon[44707]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-02-02T15:23:11.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:11 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:11.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:11 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:11.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:11 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:23:11.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:11 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:23:11.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:11 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:23:11.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:11 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:11.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:11 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:23:11.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:11 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:23:11.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:11 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:12.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:23:12.046+0000 7f555a000700 1 -- 172.21.15.1:0/3134692725 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5554104190 msgr2=0x7f5554106580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:23:12.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:23:12.046+0000 7f555a000700 1 --2- 172.21.15.1:0/3134692725 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5554104190 0x7f5554106580 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f553c009990 tx=0x7f553c009ca0 comp rx=0 tx=0).stop 2024-02-02T15:23:12.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:23:12.046+0000 7f555a000700 1 -- 172.21.15.1:0/3134692725 shutdown_connections 2024-02-02T15:23:12.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:23:12.046+0000 7f555a000700 1 --2- 172.21.15.1:0/3134692725 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5554106ac0 0x7f5554108f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:23:12.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:23:12.046+0000 7f555a000700 1 --2- 172.21.15.1:0/3134692725 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5554101860 0x7f5554103c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:23:12.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:23:12.046+0000 7f555a000700 1 --2- 172.21.15.1:0/3134692725 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5554104190 0x7f5554106580 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:23:12.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:23:12.046+0000 7f555a000700 1 -- 172.21.15.1:0/3134692725 >> 172.21.15.1:0/3134692725 conn(0x7f55540fb250 msgr2=0x7f55540fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:23:12.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:23:12.046+0000 7f555a000700 1 -- 172.21.15.1:0/3134692725 shutdown_connections 2024-02-02T15:23:12.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:23:12.046+0000 7f555a000700 1 -- 172.21.15.1:0/3134692725 wait complete. 2024-02-02T15:23:12.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:23:12.047+0000 7f555a000700 1 Processor -- start 2024-02-02T15:23:12.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:23:12.047+0000 7f555a000700 1 -- start start 2024-02-02T15:23:12.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:23:12.047+0000 7f555a000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5554101860 0x7f5554198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:23:12.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:23:12.047+0000 7f555a000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5554104190 0x7f55541991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:23:12.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:23:12.047+0000 7f555a000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5554106ac0 0x7f55541a0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:23:12.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:23:12.047+0000 7f555a000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55541a0820 con 0x7f5554101860 2024-02-02T15:23:12.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:23:12.047+0000 7f555a000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55541a0960 con 0x7f5554106ac0 2024-02-02T15:23:12.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:23:12.047+0000 7f555a000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55541a0b20 con 0x7f5554104190 2024-02-02T15:23:12.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:23:12.048+0000 7f5553400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5554101860 0x7f5554198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:23:12.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:23:12.048+0000 7f5553400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5554101860 0x7f5554198ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41988/0 (socket says 172.21.15.1:41988) 2024-02-02T15:23:12.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:23:12.048+0000 7f5553400700 1 -- 172.21.15.1:0/261932578 learned_addr learned my addr 172.21.15.1:0/261932578 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:23:12.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:23:12.048+0000 7f5552a00700 1 --2- 172.21.15.1:0/261932578 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5554104190 0x7f55541991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:23:12.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:23:12.048+0000 7f5553e00700 1 --2- 172.21.15.1:0/261932578 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5554106ac0 0x7f55541a0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:23:12.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:23:12.048+0000 7f5553400700 1 -- 172.21.15.1:0/261932578 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5554104190 msgr2=0x7f55541991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:23:12.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:23:12.048+0000 7f5553400700 1 --2- 172.21.15.1:0/261932578 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5554104190 0x7f55541991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:23:12.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:23:12.048+0000 7f5553400700 1 -- 172.21.15.1:0/261932578 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5554106ac0 msgr2=0x7f55541a0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:23:12.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:23:12.048+0000 7f5553400700 1 --2- 172.21.15.1:0/261932578 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5554106ac0 0x7f55541a0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:23:12.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:23:12.048+0000 7f5553400700 1 -- 172.21.15.1:0/261932578 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f553c0095f0 con 0x7f5554101860 2024-02-02T15:23:12.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:23:12.048+0000 7f5553400700 1 --2- 172.21.15.1:0/261932578 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5554101860 0x7f5554198ca0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f554400e9a0 tx=0x7f554400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:23:12.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:23:12.049+0000 7f554be00700 1 -- 172.21.15.1:0/261932578 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5544009790 con 0x7f5554101860 2024-02-02T15:23:12.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:23:12.049+0000 7f554be00700 1 -- 172.21.15.1:0/261932578 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5544010430 con 0x7f5554101860 2024-02-02T15:23:12.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:23:12.049+0000 7f554be00700 1 -- 172.21.15.1:0/261932578 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5544018680 con 0x7f5554101860 2024-02-02T15:23:12.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:23:12.049+0000 7f555a000700 1 -- 172.21.15.1:0/261932578 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55541a0e00 con 0x7f5554101860 2024-02-02T15:23:12.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:23:12.049+0000 7f555a000700 1 -- 172.21.15.1:0/261932578 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55541a11c0 con 0x7f5554101860 2024-02-02T15:23:12.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:23:12.050+0000 7f555a000700 1 -- 172.21.15.1:0/261932578 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5554108690 con 0x7f5554101860 2024-02-02T15:23:12.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:23:12.052+0000 7f554be00700 1 -- 172.21.15.1:0/261932578 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) v1 ==== 135069+0+0 (secure 0 0 0) 0x7f55440105a0 con 0x7f5554101860 2024-02-02T15:23:12.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:23:12.054+0000 7f554be00700 1 --2- 172.21.15.1:0/261932578 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f55340a0520 0x7f55340a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:23:12.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:23:12.054+0000 7f554be00700 1 -- 172.21.15.1:0/261932578 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5549+0+0 (secure 0 0 0) 0x7f5544014070 con 0x7f5554101860 2024-02-02T15:23:12.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:23:12.054+0000 7f5552a00700 1 --2- 172.21.15.1:0/261932578 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f55340a0520 0x7f55340a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:23:12.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:23:12.055+0000 7f5552a00700 1 --2- 172.21.15.1:0/261932578 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f55340a0520 0x7f55340a29e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f555419a1b0 tx=0x7f553c005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:23:12.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:23:12.058+0000 7f554be00700 1 -- 172.21.15.1:0/261932578 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f5544052110 con 0x7f5554101860 2024-02-02T15:23:12.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:23:12.282+0000 7f555a000700 1 -- 172.21.15.1:0/261932578 --> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f5554061b40 con 0x7f55340a0520 2024-02-02T15:23:12.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:12 smithi001 ceph-mon[35470]: Detected new or changed devices on smithi142 2024-02-02T15:23:12.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:12 smithi001 ceph-mon[35470]: Adjusting osd_memory_target on smithi142 to 5704M 2024-02-02T15:23:12.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:12 smithi001 ceph-mon[35470]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-02-02T15:23:12.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:12 smithi001 ceph-mon[35470]: osdmap e54: 9 total, 8 up, 9 in 2024-02-02T15:23:12.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:12 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-02T15:23:12.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:12 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-02T15:23:12.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:12 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-02T15:23:12.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:12 smithi071 ceph-mon[44707]: Detected new or changed devices on smithi142 2024-02-02T15:23:12.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:12 smithi071 ceph-mon[44707]: Adjusting osd_memory_target on smithi142 to 5704M 2024-02-02T15:23:12.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:12 smithi071 ceph-mon[44707]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-02-02T15:23:12.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:12 smithi071 ceph-mon[44707]: osdmap e54: 9 total, 8 up, 9 in 2024-02-02T15:23:12.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:12 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-02T15:23:12.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:12 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-02T15:23:12.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:12 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-02T15:23:12.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:12 smithi142 ceph-mon[45728]: Detected new or changed devices on smithi142 2024-02-02T15:23:12.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:12 smithi142 ceph-mon[45728]: Adjusting osd_memory_target on smithi142 to 5704M 2024-02-02T15:23:12.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:12 smithi142 ceph-mon[45728]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-02-02T15:23:12.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:12 smithi142 ceph-mon[45728]: osdmap e54: 9 total, 8 up, 9 in 2024-02-02T15:23:12.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:12 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-02T15:23:12.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:12 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-02T15:23:12.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:12 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-02T15:23:13.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:13 smithi001 ceph-mon[35470]: purged_snaps scrub starts 2024-02-02T15:23:13.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:13 smithi001 ceph-mon[35470]: purged_snaps scrub ok 2024-02-02T15:23:13.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:13 smithi001 ceph-mon[35470]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-02-02T15:23:13.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:13 smithi001 ceph-mon[35470]: osd.8 [v2:172.21.15.142:6816/3942038209,v1:172.21.15.142:6817/3942038209] boot 2024-02-02T15:23:13.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:13 smithi001 ceph-mon[35470]: osdmap e55: 9 total, 9 up, 9 in 2024-02-02T15:23:13.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:13 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-02T15:23:13.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:13 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-02T15:23:13.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:13 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-02T15:23:13.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:13 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:23:13.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:13 smithi071 ceph-mon[44707]: purged_snaps scrub starts 2024-02-02T15:23:13.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:13 smithi071 ceph-mon[44707]: purged_snaps scrub ok 2024-02-02T15:23:13.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:13 smithi071 ceph-mon[44707]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-02-02T15:23:13.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:13 smithi071 ceph-mon[44707]: osd.8 [v2:172.21.15.142:6816/3942038209,v1:172.21.15.142:6817/3942038209] boot 2024-02-02T15:23:13.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:13 smithi071 ceph-mon[44707]: osdmap e55: 9 total, 9 up, 9 in 2024-02-02T15:23:13.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:13 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-02T15:23:13.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:13 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-02T15:23:13.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:13 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-02T15:23:13.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:13 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:23:13.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:13 smithi142 ceph-mon[45728]: purged_snaps scrub starts 2024-02-02T15:23:13.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:13 smithi142 ceph-mon[45728]: purged_snaps scrub ok 2024-02-02T15:23:13.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:13 smithi142 ceph-mon[45728]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2024-02-02T15:23:13.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:13 smithi142 ceph-mon[45728]: osd.8 [v2:172.21.15.142:6816/3942038209,v1:172.21.15.142:6817/3942038209] boot 2024-02-02T15:23:13.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:13 smithi142 ceph-mon[45728]: osdmap e55: 9 total, 9 up, 9 in 2024-02-02T15:23:13.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:13 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-02T15:23:13.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:13 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-02T15:23:13.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:13 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-02T15:23:13.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:13 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:23:14.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:14 smithi001 ceph-mon[35470]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:23:14.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:14 smithi001 ceph-mon[35470]: osdmap e56: 9 total, 9 up, 9 in 2024-02-02T15:23:14.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:14 smithi071 ceph-mon[44707]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:23:14.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:14 smithi071 ceph-mon[44707]: osdmap e56: 9 total, 9 up, 9 in 2024-02-02T15:23:14.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:14 smithi142 ceph-mon[45728]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi001:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:23:14.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:14 smithi142 ceph-mon[45728]: osdmap e56: 9 total, 9 up, 9 in 2024-02-02T15:23:15.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:15 smithi001 ceph-mon[35470]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2024-02-02T15:23:15.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:15 smithi001 ceph-mon[35470]: osdmap e57: 9 total, 9 up, 9 in 2024-02-02T15:23:15.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:15 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/140085977' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7e165e68-31a8-4598-a577-de3e90ae5c5b"}]: dispatch 2024-02-02T15:23:15.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:15 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/140085977' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7e165e68-31a8-4598-a577-de3e90ae5c5b"}]': finished 2024-02-02T15:23:15.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:15 smithi001 ceph-mon[35470]: osdmap e58: 10 total, 9 up, 10 in 2024-02-02T15:23:15.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:15 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-02T15:23:15.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:15 smithi071 ceph-mon[44707]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2024-02-02T15:23:15.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:15 smithi071 ceph-mon[44707]: osdmap e57: 9 total, 9 up, 9 in 2024-02-02T15:23:15.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:15 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/140085977' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7e165e68-31a8-4598-a577-de3e90ae5c5b"}]: dispatch 2024-02-02T15:23:15.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:15 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/140085977' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7e165e68-31a8-4598-a577-de3e90ae5c5b"}]': finished 2024-02-02T15:23:15.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:15 smithi071 ceph-mon[44707]: osdmap e58: 10 total, 9 up, 10 in 2024-02-02T15:23:15.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:15 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-02T15:23:15.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:15 smithi142 ceph-mon[45728]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2024-02-02T15:23:15.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:15 smithi142 ceph-mon[45728]: osdmap e57: 9 total, 9 up, 9 in 2024-02-02T15:23:15.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:15 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/140085977' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7e165e68-31a8-4598-a577-de3e90ae5c5b"}]: dispatch 2024-02-02T15:23:15.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:15 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/140085977' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7e165e68-31a8-4598-a577-de3e90ae5c5b"}]': finished 2024-02-02T15:23:15.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:15 smithi142 ceph-mon[45728]: osdmap e58: 10 total, 9 up, 10 in 2024-02-02T15:23:15.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:15 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-02T15:23:16.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:16 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/1260546181' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-02T15:23:16.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:16 smithi001 ceph-mon[35470]: osdmap e59: 10 total, 9 up, 10 in 2024-02-02T15:23:16.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:16 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-02T15:23:16.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:16 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/1260546181' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-02T15:23:16.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:16 smithi071 ceph-mon[44707]: osdmap e59: 10 total, 9 up, 10 in 2024-02-02T15:23:16.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:16 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-02T15:23:16.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:16 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/1260546181' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-02T15:23:16.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:16 smithi142 ceph-mon[45728]: osdmap e59: 10 total, 9 up, 10 in 2024-02-02T15:23:16.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:16 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-02T15:23:17.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:17 smithi001 ceph-mon[35470]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2024-02-02T15:23:17.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:17 smithi071 ceph-mon[44707]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2024-02-02T15:23:17.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:17 smithi142 ceph-mon[45728]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2024-02-02T15:23:19.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:19 smithi001 ceph-mon[35470]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-02-02T15:23:19.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:19 smithi071 ceph-mon[44707]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-02-02T15:23:19.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:19 smithi142 ceph-mon[45728]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-02-02T15:23:21.444 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:21 smithi001 ceph-mon[35470]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-02T15:23:21.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:21 smithi071 ceph-mon[44707]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-02T15:23:21.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:21 smithi142 ceph-mon[45728]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-02-02T15:23:22.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:22 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-02-02T15:23:22.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:22 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:23:22.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:22 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-02-02T15:23:22.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:22 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:23:22.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:22 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-02-02T15:23:22.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:22 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:23:23.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:23 smithi001 ceph-mon[35470]: Deploying daemon osd.9 on smithi001 2024-02-02T15:23:23.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:23 smithi001 ceph-mon[35470]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-02-02T15:23:23.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:23 smithi071 ceph-mon[44707]: Deploying daemon osd.9 on smithi001 2024-02-02T15:23:23.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:23 smithi071 ceph-mon[44707]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-02-02T15:23:23.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:23 smithi142 ceph-mon[45728]: Deploying daemon osd.9 on smithi001 2024-02-02T15:23:23.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:23 smithi142 ceph-mon[45728]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-02-02T15:23:25.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:25 smithi001 ceph-mon[35470]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-02-02T15:23:25.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:25 smithi071 ceph-mon[44707]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-02-02T15:23:25.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:25 smithi142 ceph-mon[45728]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-02-02T15:23:27.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:26 smithi071 ceph-mon[44707]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-02-02T15:23:27.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:26 smithi142 ceph-mon[45728]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-02-02T15:23:27.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:26 smithi001 ceph-mon[35470]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-02-02T15:23:27.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:27 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:23:27.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:27 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:27.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:27 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:28.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:27 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:23:28.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:27 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:28.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:27 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:28.169 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:27 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:23:28.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:27 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:28.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:27 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:29.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:28 smithi071 ceph-mon[44707]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-02-02T15:23:29.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:28 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:29.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:28 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:29.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:28 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:23:29.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:28 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:23:29.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:28 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:29.169 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:28 smithi142 ceph-mon[45728]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-02-02T15:23:29.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:28 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:29.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:28 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:29.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:28 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:23:29.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:28 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:23:29.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:28 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:29.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:28 smithi001 ceph-mon[35470]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-02-02T15:23:29.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:28 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:29.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:28 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:29.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:28 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:23:29.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:28 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:23:29.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:28 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:30.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:23:30.192+0000 7f554be00700 1 -- 172.21.15.1:0/261932578 <== mgr.14154 v2:172.21.15.1:6800/3768989981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f5554061b40 con 0x7f55340a0520 2024-02-02T15:23:30.193 INFO:teuthology.orchestra.run.smithi001.stdout:Created osd(s) 9 on host 'smithi001' 2024-02-02T15:23:30.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:23:30.195+0000 7f555a000700 1 -- 172.21.15.1:0/261932578 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f55340a0520 msgr2=0x7f55340a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:23:30.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:23:30.195+0000 7f555a000700 1 --2- 172.21.15.1:0/261932578 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f55340a0520 0x7f55340a29e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f555419a1b0 tx=0x7f553c005fb0 comp rx=0 tx=0).stop 2024-02-02T15:23:30.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:23:30.195+0000 7f555a000700 1 -- 172.21.15.1:0/261932578 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5554101860 msgr2=0x7f5554198ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:23:30.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:23:30.195+0000 7f555a000700 1 --2- 172.21.15.1:0/261932578 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5554101860 0x7f5554198ca0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f554400e9a0 tx=0x7f554400ecb0 comp rx=0 tx=0).stop 2024-02-02T15:23:30.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:23:30.195+0000 7f555a000700 1 -- 172.21.15.1:0/261932578 shutdown_connections 2024-02-02T15:23:30.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:23:30.195+0000 7f555a000700 1 --2- 172.21.15.1:0/261932578 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5554106ac0 0x7f55541a0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:23:30.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:23:30.195+0000 7f555a000700 1 --2- 172.21.15.1:0/261932578 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5554101860 0x7f5554198ca0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:23:30.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:23:30.195+0000 7f555a000700 1 --2- 172.21.15.1:0/261932578 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f55340a0520 0x7f55340a29e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:23:30.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:23:30.195+0000 7f555a000700 1 --2- 172.21.15.1:0/261932578 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5554104190 0x7f55541991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:23:30.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:23:30.195+0000 7f555a000700 1 -- 172.21.15.1:0/261932578 >> 172.21.15.1:0/261932578 conn(0x7f55540fb250 msgr2=0x7f55540ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:23:30.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:23:30.196+0000 7f555a000700 1 -- 172.21.15.1:0/261932578 shutdown_connections 2024-02-02T15:23:30.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:23:30.196+0000 7f555a000700 1 -- 172.21.15.1:0/261932578 wait complete. 2024-02-02T15:23:30.577 DEBUG:teuthology.orchestra.run.smithi001:osd.9> sudo journalctl -f -n 0 -u ceph-371a409c-c1de-11ee-95b6-87774f69a715@osd.9.service 2024-02-02T15:23:30.581 INFO:tasks.cephadm:Deploying osd.10 on smithi071 with /dev/vg_nvme/lv_1... 2024-02-02T15:23:30.582 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-02-02T15:23:30.656 INFO:journalctl@ceph.osd.9.smithi001.stdout:-- Logs begin at Fri 2024-02-02 15:05:03 UTC. -- 2024-02-02T15:23:30.814 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.b/config 2024-02-02T15:23:30.903 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:23:30 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:23:30.821+0000 7fb99a4ff640 -1 osd.9 0 log_to_monitors true 2024-02-02T15:23:31.236 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:31 smithi071 ceph-mon[44707]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2024-02-02T15:23:31.236 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:31 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:31.236 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:31 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:31.236 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:31 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:23:31.236 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:31 smithi071 ceph-mon[44707]: from='osd.9 [v2:172.21.15.1:6826/2716544850,v1:172.21.15.1:6827/2716544850]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-02-02T15:23:31.237 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:31 smithi071 ceph-mon[44707]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-02-02T15:23:31.239 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:31 smithi001 ceph-mon[35470]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2024-02-02T15:23:31.240 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:31 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:31.240 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:31 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:31.240 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:31 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:23:31.240 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:31 smithi001 ceph-mon[35470]: from='osd.9 [v2:172.21.15.1:6826/2716544850,v1:172.21.15.1:6827/2716544850]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-02-02T15:23:31.240 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:31 smithi001 ceph-mon[35470]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-02-02T15:23:31.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:31 smithi142 ceph-mon[45728]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2024-02-02T15:23:31.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:31 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:31.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:31 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:31.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:31 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:23:31.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:31 smithi142 ceph-mon[45728]: from='osd.9 [v2:172.21.15.1:6826/2716544850,v1:172.21.15.1:6827/2716544850]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-02-02T15:23:31.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:31 smithi142 ceph-mon[45728]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-02-02T15:23:32.078 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-02-02T15:23:32.078 INFO:teuthology.orchestra.run.smithi071.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-02-02T15:23:32.078 INFO:teuthology.orchestra.run.smithi071.stderr: stderr: 10+0 records in 2024-02-02T15:23:32.078 INFO:teuthology.orchestra.run.smithi071.stderr:10+0 records out 2024-02-02T15:23:32.078 INFO:teuthology.orchestra.run.smithi071.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112718 s, 930 MB/s 2024-02-02T15:23:32.078 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping successful for: 2024-02-02T15:23:32.419 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph orch daemon add osd smithi071:vg_nvme/lv_1 2024-02-02T15:23:32.639 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.b/config 2024-02-02T15:23:33.086 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:32 smithi071 ceph-mon[44707]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-02-02T15:23:33.086 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:32 smithi071 ceph-mon[44707]: osdmap e60: 10 total, 9 up, 10 in 2024-02-02T15:23:33.086 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:32 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-02T15:23:33.086 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:32 smithi071 ceph-mon[44707]: from='osd.9 [v2:172.21.15.1:6826/2716544850,v1:172.21.15.1:6827/2716544850]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-02-02T15:23:33.086 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:32 smithi071 ceph-mon[44707]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-02-02T15:23:33.086 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:32 smithi071 ceph-mon[44707]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2024-02-02T15:23:33.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:32 smithi142 ceph-mon[45728]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-02-02T15:23:33.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:32 smithi142 ceph-mon[45728]: osdmap e60: 10 total, 9 up, 10 in 2024-02-02T15:23:33.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:32 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-02T15:23:33.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:32 smithi142 ceph-mon[45728]: from='osd.9 [v2:172.21.15.1:6826/2716544850,v1:172.21.15.1:6827/2716544850]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-02-02T15:23:33.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:32 smithi142 ceph-mon[45728]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-02-02T15:23:33.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:32 smithi142 ceph-mon[45728]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2024-02-02T15:23:33.188 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:23:32 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:23:32.783+0000 7fb98ea00700 -1 osd.9 0 waiting for initial osdmap 2024-02-02T15:23:33.188 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:23:32 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:23:32.804+0000 7fb987000700 -1 osd.9 61 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-02T15:23:33.189 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:32 smithi001 ceph-mon[35470]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-02-02T15:23:33.189 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:32 smithi001 ceph-mon[35470]: osdmap e60: 10 total, 9 up, 10 in 2024-02-02T15:23:33.189 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:32 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-02T15:23:33.189 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:32 smithi001 ceph-mon[35470]: from='osd.9 [v2:172.21.15.1:6826/2716544850,v1:172.21.15.1:6827/2716544850]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-02-02T15:23:33.189 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:32 smithi001 ceph-mon[35470]: from='osd.9 ' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]: dispatch 2024-02-02T15:23:33.189 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:32 smithi001 ceph-mon[35470]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2024-02-02T15:23:33.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:23:33.626+0000 7f844b105700 1 -- 172.21.15.71:0/3204999461 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8444101870 msgr2=0x7f8444108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:23:33.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:23:33.626+0000 7f844b105700 1 --2- 172.21.15.71:0/3204999461 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8444101870 0x7f8444108120 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f843c00b0d0 tx=0x7f843c00b3e0 comp rx=0 tx=0).stop 2024-02-02T15:23:33.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:23:33.626+0000 7f844b105700 1 -- 172.21.15.71:0/3204999461 shutdown_connections 2024-02-02T15:23:33.626 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:23:33.626+0000 7f844b105700 1 --2- 172.21.15.71:0/3204999461 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8444101870 0x7f8444108120 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:23:33.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:23:33.626+0000 7f844b105700 1 --2- 172.21.15.71:0/3204999461 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84440ffcd0 0x7f84441000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:23:33.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:23:33.626+0000 7f844b105700 1 --2- 172.21.15.71:0/3204999461 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8444100ed0 0x7f8444101330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:23:33.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:23:33.626+0000 7f844b105700 1 -- 172.21.15.71:0/3204999461 >> 172.21.15.71:0/3204999461 conn(0x7f84440fb250 msgr2=0x7f84440fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:23:33.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:23:33.626+0000 7f844b105700 1 -- 172.21.15.71:0/3204999461 shutdown_connections 2024-02-02T15:23:33.627 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:23:33.626+0000 7f844b105700 1 -- 172.21.15.71:0/3204999461 wait complete. 2024-02-02T15:23:33.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:23:33.627+0000 7f844b105700 1 Processor -- start 2024-02-02T15:23:33.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:23:33.627+0000 7f844b105700 1 -- start start 2024-02-02T15:23:33.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:23:33.627+0000 7f844b105700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84440ffcd0 0x7f8444103c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:23:33.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:23:33.627+0000 7f844b105700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8444100ed0 0x7f84441022b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:23:33.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:23:33.627+0000 7f844b105700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8444101870 0x7f84441027f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:23:33.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:23:33.627+0000 7f844b105700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8444102f90 con 0x7f8444100ed0 2024-02-02T15:23:33.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:23:33.627+0000 7f844b105700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8444103100 con 0x7f8444101870 2024-02-02T15:23:33.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:23:33.627+0000 7f844b105700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84441a0ca0 con 0x7f84440ffcd0 2024-02-02T15:23:33.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:23:33.627+0000 7f84496a2700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8444101870 0x7f84441027f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:23:33.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:23:33.627+0000 7f84496a2700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8444101870 0x7f84441027f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:46680/0 (socket says 172.21.15.71:46680) 2024-02-02T15:23:33.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:23:33.627+0000 7f84496a2700 1 -- 172.21.15.71:0/1734650555 learned_addr learned my addr 172.21.15.71:0/1734650555 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-02-02T15:23:33.629 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:23:33.627+0000 7f8448ea1700 1 --2- 172.21.15.71:0/1734650555 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84440ffcd0 0x7f8444103c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:23:33.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:23:33.627+0000 7f84496a2700 1 -- 172.21.15.71:0/1734650555 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84440ffcd0 msgr2=0x7f8444103c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:23:33.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:23:33.627+0000 7f84496a2700 1 --2- 172.21.15.71:0/1734650555 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84440ffcd0 0x7f8444103c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:23:33.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:23:33.627+0000 7f84496a2700 1 -- 172.21.15.71:0/1734650555 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8444100ed0 msgr2=0x7f84441022b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:23:33.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:23:33.627+0000 7f84496a2700 1 --2- 172.21.15.71:0/1734650555 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8444100ed0 0x7f84441022b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:23:33.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:23:33.627+0000 7f84496a2700 1 -- 172.21.15.71:0/1734650555 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f843c009d00 con 0x7f8444101870 2024-02-02T15:23:33.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:23:33.627+0000 7f8448ea1700 1 --2- 172.21.15.71:0/1734650555 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84440ffcd0 0x7f8444103c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:23:33.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:23:33.627+0000 7f84496a2700 1 --2- 172.21.15.71:0/1734650555 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8444101870 0x7f84441027f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f843c007a00 tx=0x7f843c0094f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:23:33.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:23:33.627+0000 7f8441ffb700 1 -- 172.21.15.71:0/1734650555 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f843c00e040 con 0x7f8444101870 2024-02-02T15:23:33.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:23:33.628+0000 7f844b105700 1 -- 172.21.15.71:0/1734650555 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84441a0ed0 con 0x7f8444101870 2024-02-02T15:23:33.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:23:33.628+0000 7f8441ffb700 1 -- 172.21.15.71:0/1734650555 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f843c012dc0 con 0x7f8444101870 2024-02-02T15:23:33.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:23:33.628+0000 7f8441ffb700 1 -- 172.21.15.71:0/1734650555 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f843c01b450 con 0x7f8444101870 2024-02-02T15:23:33.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:23:33.628+0000 7f844b105700 1 -- 172.21.15.71:0/1734650555 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84441a1360 con 0x7f8444101870 2024-02-02T15:23:33.630 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:23:33.629+0000 7f844b105700 1 -- 172.21.15.71:0/1734650555 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f844410b610 con 0x7f8444101870 2024-02-02T15:23:33.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:23:33.630+0000 7f8441ffb700 1 -- 172.21.15.71:0/1734650555 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 135069+0+0 (secure 0 0 0) 0x7f843c01d610 con 0x7f8444101870 2024-02-02T15:23:33.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:23:33.630+0000 7f8441ffb700 1 --2- 172.21.15.71:0/1734650555 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f84340a0520 0x7f84340a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:23:33.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:23:33.630+0000 7f8441ffb700 1 -- 172.21.15.71:0/1734650555 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5981+0+0 (secure 0 0 0) 0x7f843c0cd580 con 0x7f8444101870 2024-02-02T15:23:33.631 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:23:33.631+0000 7f8448ea1700 1 --2- 172.21.15.71:0/1734650555 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f84340a0520 0x7f84340a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:23:33.632 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:23:33.631+0000 7f8448ea1700 1 --2- 172.21.15.71:0/1734650555 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f84340a0520 0x7f84340a29e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8438005fd0 tx=0x7f8438005f00 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:23:33.633 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:23:33.634+0000 7f8441ffb700 1 -- 172.21.15.71:0/1734650555 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f843c060110 con 0x7f8444101870 2024-02-02T15:23:33.832 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:23:33.832+0000 7f844b105700 1 -- 172.21.15.71:0/1734650555 --> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f8444068f50 con 0x7f84340a0520 2024-02-02T15:23:34.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:33 smithi071 ceph-mon[44707]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]': finished 2024-02-02T15:23:34.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:33 smithi071 ceph-mon[44707]: osdmap e61: 10 total, 9 up, 10 in 2024-02-02T15:23:34.116 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:33 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-02T15:23:34.116 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:33 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-02T15:23:34.116 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:33 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:34.116 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:33 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:34.116 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:33 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:23:34.116 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:33 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:23:34.116 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:33 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:23:34.116 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:33 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:23:34.116 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:33 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:34.116 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:33 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:23:34.116 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:33 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:23:34.116 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:33 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:34.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:33 smithi142 ceph-mon[45728]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]': finished 2024-02-02T15:23:34.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:33 smithi142 ceph-mon[45728]: osdmap e61: 10 total, 9 up, 10 in 2024-02-02T15:23:34.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:33 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-02T15:23:34.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:33 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-02T15:23:34.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:33 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:34.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:33 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:34.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:33 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:23:34.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:33 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:23:34.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:33 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:23:34.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:33 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:23:34.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:33 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:34.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:33 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:23:34.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:33 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:23:34.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:33 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:34.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:33 smithi001 ceph-mon[35470]: from='osd.9 ' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi001", "root=default"]}]': finished 2024-02-02T15:23:34.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:33 smithi001 ceph-mon[35470]: osdmap e61: 10 total, 9 up, 10 in 2024-02-02T15:23:34.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:33 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-02T15:23:34.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:33 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-02T15:23:34.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:33 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:34.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:33 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:34.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:33 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:23:34.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:33 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:23:34.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:33 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:23:34.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:33 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:23:34.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:33 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:34.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:33 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:23:34.192 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:33 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:23:34.192 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:33 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:35.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:34 smithi071 ceph-mon[44707]: purged_snaps scrub starts 2024-02-02T15:23:35.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:34 smithi071 ceph-mon[44707]: purged_snaps scrub ok 2024-02-02T15:23:35.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:34 smithi071 ceph-mon[44707]: Detected new or changed devices on smithi001 2024-02-02T15:23:35.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:34 smithi071 ceph-mon[44707]: Adjusting osd_memory_target on smithi001 to 4319M 2024-02-02T15:23:35.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:34 smithi071 ceph-mon[44707]: osd.9 [v2:172.21.15.1:6826/2716544850,v1:172.21.15.1:6827/2716544850] boot 2024-02-02T15:23:35.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:34 smithi071 ceph-mon[44707]: osdmap e62: 10 total, 10 up, 10 in 2024-02-02T15:23:35.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:34 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-02T15:23:35.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:34 smithi071 ceph-mon[44707]: from='client.24371 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:23:35.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:34 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-02T15:23:35.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:34 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-02T15:23:35.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:34 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:23:35.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:34 smithi071 ceph-mon[44707]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2024-02-02T15:23:35.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:34 smithi142 ceph-mon[45728]: purged_snaps scrub starts 2024-02-02T15:23:35.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:34 smithi142 ceph-mon[45728]: purged_snaps scrub ok 2024-02-02T15:23:35.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:34 smithi142 ceph-mon[45728]: Detected new or changed devices on smithi001 2024-02-02T15:23:35.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:34 smithi142 ceph-mon[45728]: Adjusting osd_memory_target on smithi001 to 4319M 2024-02-02T15:23:35.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:34 smithi142 ceph-mon[45728]: osd.9 [v2:172.21.15.1:6826/2716544850,v1:172.21.15.1:6827/2716544850] boot 2024-02-02T15:23:35.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:34 smithi142 ceph-mon[45728]: osdmap e62: 10 total, 10 up, 10 in 2024-02-02T15:23:35.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:34 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-02T15:23:35.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:34 smithi142 ceph-mon[45728]: from='client.24371 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:23:35.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:34 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-02T15:23:35.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:34 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-02T15:23:35.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:34 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:23:35.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:34 smithi142 ceph-mon[45728]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2024-02-02T15:23:35.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:34 smithi001 ceph-mon[35470]: purged_snaps scrub starts 2024-02-02T15:23:35.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:34 smithi001 ceph-mon[35470]: purged_snaps scrub ok 2024-02-02T15:23:35.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:34 smithi001 ceph-mon[35470]: Detected new or changed devices on smithi001 2024-02-02T15:23:35.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:34 smithi001 ceph-mon[35470]: Adjusting osd_memory_target on smithi001 to 4319M 2024-02-02T15:23:35.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:34 smithi001 ceph-mon[35470]: osd.9 [v2:172.21.15.1:6826/2716544850,v1:172.21.15.1:6827/2716544850] boot 2024-02-02T15:23:35.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:34 smithi001 ceph-mon[35470]: osdmap e62: 10 total, 10 up, 10 in 2024-02-02T15:23:35.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:34 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-02T15:23:35.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:34 smithi001 ceph-mon[35470]: from='client.24371 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:23:35.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:34 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-02T15:23:35.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:34 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-02T15:23:35.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:34 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:23:35.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:34 smithi001 ceph-mon[35470]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2024-02-02T15:23:36.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:35 smithi071 ceph-mon[44707]: osdmap e63: 10 total, 10 up, 10 in 2024-02-02T15:23:36.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:35 smithi071 ceph-mon[44707]: from='client.? 172.21.15.71:0/582435883' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "48cd5fdb-0887-444a-aab8-a1de24282bb8"}]: dispatch 2024-02-02T15:23:36.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:35 smithi071 ceph-mon[44707]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "48cd5fdb-0887-444a-aab8-a1de24282bb8"}]: dispatch 2024-02-02T15:23:36.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:35 smithi071 ceph-mon[44707]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "48cd5fdb-0887-444a-aab8-a1de24282bb8"}]': finished 2024-02-02T15:23:36.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:35 smithi071 ceph-mon[44707]: osdmap e64: 11 total, 10 up, 11 in 2024-02-02T15:23:36.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:35 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-02T15:23:36.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:35 smithi142 ceph-mon[45728]: osdmap e63: 10 total, 10 up, 10 in 2024-02-02T15:23:36.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:35 smithi142 ceph-mon[45728]: from='client.? 172.21.15.71:0/582435883' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "48cd5fdb-0887-444a-aab8-a1de24282bb8"}]: dispatch 2024-02-02T15:23:36.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:35 smithi142 ceph-mon[45728]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "48cd5fdb-0887-444a-aab8-a1de24282bb8"}]: dispatch 2024-02-02T15:23:36.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:35 smithi142 ceph-mon[45728]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "48cd5fdb-0887-444a-aab8-a1de24282bb8"}]': finished 2024-02-02T15:23:36.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:35 smithi142 ceph-mon[45728]: osdmap e64: 11 total, 10 up, 11 in 2024-02-02T15:23:36.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:35 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-02T15:23:36.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:35 smithi001 ceph-mon[35470]: osdmap e63: 10 total, 10 up, 10 in 2024-02-02T15:23:36.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:35 smithi001 ceph-mon[35470]: from='client.? 172.21.15.71:0/582435883' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "48cd5fdb-0887-444a-aab8-a1de24282bb8"}]: dispatch 2024-02-02T15:23:36.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:35 smithi001 ceph-mon[35470]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "48cd5fdb-0887-444a-aab8-a1de24282bb8"}]: dispatch 2024-02-02T15:23:36.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:35 smithi001 ceph-mon[35470]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "48cd5fdb-0887-444a-aab8-a1de24282bb8"}]': finished 2024-02-02T15:23:36.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:35 smithi001 ceph-mon[35470]: osdmap e64: 11 total, 10 up, 11 in 2024-02-02T15:23:36.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:35 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-02T15:23:37.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:36 smithi071 ceph-mon[44707]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2024-02-02T15:23:37.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:36 smithi071 ceph-mon[44707]: from='client.? 172.21.15.71:0/2948325245' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-02T15:23:37.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:36 smithi142 ceph-mon[45728]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2024-02-02T15:23:37.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:36 smithi142 ceph-mon[45728]: from='client.? 172.21.15.71:0/2948325245' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-02T15:23:37.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:36 smithi001 ceph-mon[35470]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2024-02-02T15:23:37.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:36 smithi001 ceph-mon[35470]: from='client.? 172.21.15.71:0/2948325245' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-02T15:23:39.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:38 smithi001 ceph-mon[35470]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2024-02-02T15:23:39.290 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:38 smithi071 ceph-mon[44707]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2024-02-02T15:23:39.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:38 smithi142 ceph-mon[45728]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2024-02-02T15:23:41.209 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:40 smithi071 ceph-mon[44707]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2024-02-02T15:23:41.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:40 smithi142 ceph-mon[45728]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2024-02-02T15:23:41.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:40 smithi001 ceph-mon[35470]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2024-02-02T15:23:42.109 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:41 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-02-02T15:23:42.109 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:41 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:23:42.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:41 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-02-02T15:23:42.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:41 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:23:42.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:41 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-02-02T15:23:42.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:41 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:23:43.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:42 smithi071 ceph-mon[44707]: Deploying daemon osd.10 on smithi071 2024-02-02T15:23:43.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:42 smithi071 ceph-mon[44707]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2024-02-02T15:23:43.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:42 smithi142 ceph-mon[45728]: Deploying daemon osd.10 on smithi071 2024-02-02T15:23:43.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:42 smithi142 ceph-mon[45728]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2024-02-02T15:23:43.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:42 smithi001 ceph-mon[35470]: Deploying daemon osd.10 on smithi071 2024-02-02T15:23:43.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:42 smithi001 ceph-mon[35470]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2024-02-02T15:23:45.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:44 smithi071 ceph-mon[44707]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2024-02-02T15:23:45.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:44 smithi142 ceph-mon[45728]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2024-02-02T15:23:45.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:44 smithi001 ceph-mon[35470]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2024-02-02T15:23:46.300 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:46 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:46.300 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:46 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:46.300 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:46 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:23:46.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:46 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:46.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:46 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:46.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:46 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:23:46.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:46 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:46.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:46 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:46.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:46 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:23:47.085 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:47 smithi071 ceph-mon[44707]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2024-02-02T15:23:47.086 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:47 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:47.086 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:47 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:47.086 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:47 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:23:47.086 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:47 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:23:47.086 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:47 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:47.192 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:23:47.193+0000 7f8441ffb700 1 -- 172.21.15.71:0/1734650555 <== mgr.14154 v2:172.21.15.1:6800/3768989981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f8444068f50 con 0x7f84340a0520 2024-02-02T15:23:47.192 INFO:teuthology.orchestra.run.smithi071.stdout:Created osd(s) 10 on host 'smithi071' 2024-02-02T15:23:47.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:23:47.195+0000 7f844b105700 1 -- 172.21.15.71:0/1734650555 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f84340a0520 msgr2=0x7f84340a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:23:47.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:23:47.195+0000 7f844b105700 1 --2- 172.21.15.71:0/1734650555 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f84340a0520 0x7f84340a29e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8438005fd0 tx=0x7f8438005f00 comp rx=0 tx=0).stop 2024-02-02T15:23:47.194 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:23:47.195+0000 7f844b105700 1 -- 172.21.15.71:0/1734650555 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8444101870 msgr2=0x7f84441027f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:23:47.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:23:47.195+0000 7f844b105700 1 --2- 172.21.15.71:0/1734650555 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8444101870 0x7f84441027f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f843c007a00 tx=0x7f843c0094f0 comp rx=0 tx=0).stop 2024-02-02T15:23:47.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:23:47.195+0000 7f844b105700 1 -- 172.21.15.71:0/1734650555 shutdown_connections 2024-02-02T15:23:47.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:23:47.195+0000 7f844b105700 1 --2- 172.21.15.71:0/1734650555 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8444101870 0x7f84441027f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:23:47.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:23:47.195+0000 7f844b105700 1 --2- 172.21.15.71:0/1734650555 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f84340a0520 0x7f84340a29e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:23:47.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:23:47.195+0000 7f844b105700 1 --2- 172.21.15.71:0/1734650555 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84440ffcd0 0x7f8444103c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:23:47.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:23:47.195+0000 7f844b105700 1 --2- 172.21.15.71:0/1734650555 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8444100ed0 0x7f84441022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:23:47.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:23:47.195+0000 7f844b105700 1 -- 172.21.15.71:0/1734650555 >> 172.21.15.71:0/1734650555 conn(0x7f84440fb250 msgr2=0x7f8444068840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:23:47.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:23:47.195+0000 7f844b105700 1 -- 172.21.15.71:0/1734650555 shutdown_connections 2024-02-02T15:23:47.195 INFO:teuthology.orchestra.run.smithi071.stderr:2024-02-02T15:23:47.195+0000 7f844b105700 1 -- 172.21.15.71:0/1734650555 wait complete. 2024-02-02T15:23:47.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:47 smithi142 ceph-mon[45728]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2024-02-02T15:23:47.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:47 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:47.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:47 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:47.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:47 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:23:47.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:47 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:23:47.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:47 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:47.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:47 smithi001 ceph-mon[35470]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2024-02-02T15:23:47.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:47 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:47.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:47 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:47.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:47 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:23:47.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:47 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:23:47.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:47 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:47.563 DEBUG:teuthology.orchestra.run.smithi071:osd.10> sudo journalctl -f -n 0 -u ceph-371a409c-c1de-11ee-95b6-87774f69a715@osd.10.service 2024-02-02T15:23:47.565 INFO:tasks.cephadm:Deploying osd.11 on smithi142 with /dev/vg_nvme/lv_1... 2024-02-02T15:23:47.565 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-02-02T15:23:47.598 INFO:journalctl@ceph.osd.10.smithi071.stdout:-- Logs begin at Fri 2024-02-02 14:58:45 UTC. -- 2024-02-02T15:23:47.798 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.c/config 2024-02-02T15:23:48.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:48 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:48.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:48 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:48.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:48 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:23:48.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:48 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:48.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:48 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:48.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:48 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:23:48.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:48 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:48.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:48 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:48.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:48 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:23:49.062 INFO:teuthology.orchestra.run.smithi142.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-02-02T15:23:49.062 INFO:teuthology.orchestra.run.smithi142.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-02-02T15:23:49.062 INFO:teuthology.orchestra.run.smithi142.stderr: stderr: 10+0 records in 2024-02-02T15:23:49.062 INFO:teuthology.orchestra.run.smithi142.stderr:10+0 records out 2024-02-02T15:23:49.062 INFO:teuthology.orchestra.run.smithi142.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0140455 s, 747 MB/s 2024-02-02T15:23:49.062 INFO:teuthology.orchestra.run.smithi142.stderr: stderr: 2024-02-02T15:23:49.062 INFO:teuthology.orchestra.run.smithi142.stderr:--> Zapping successful for: 2024-02-02T15:23:49.313 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:49 smithi071 ceph-mon[44707]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2024-02-02T15:23:49.314 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:49 smithi071 ceph-mon[44707]: from='osd.10 [v2:172.21.15.71:6824/84691841,v1:172.21.15.71:6825/84691841]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-02-02T15:23:49.314 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:49 smithi071 ceph-mon[44707]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-02-02T15:23:49.314 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:23:49 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:23:49.020+0000 7f4073674640 -1 osd.10 0 log_to_monitors true 2024-02-02T15:23:49.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:49 smithi142 ceph-mon[45728]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2024-02-02T15:23:49.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:49 smithi142 ceph-mon[45728]: from='osd.10 [v2:172.21.15.71:6824/84691841,v1:172.21.15.71:6825/84691841]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-02-02T15:23:49.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:49 smithi142 ceph-mon[45728]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-02-02T15:23:49.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:49 smithi001 ceph-mon[35470]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2024-02-02T15:23:49.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:49 smithi001 ceph-mon[35470]: from='osd.10 [v2:172.21.15.71:6824/84691841,v1:172.21.15.71:6825/84691841]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-02-02T15:23:49.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:49 smithi001 ceph-mon[35470]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-02-02T15:23:49.506 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph orch daemon add osd smithi142:vg_nvme/lv_1 2024-02-02T15:23:49.727 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.c/config 2024-02-02T15:23:50.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:50 smithi001 ceph-mon[35470]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-02-02T15:23:50.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:50 smithi001 ceph-mon[35470]: osdmap e65: 11 total, 10 up, 11 in 2024-02-02T15:23:50.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:50 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-02T15:23:50.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:50 smithi001 ceph-mon[35470]: from='osd.10 [v2:172.21.15.71:6824/84691841,v1:172.21.15.71:6825/84691841]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-02-02T15:23:50.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:50 smithi001 ceph-mon[35470]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-02-02T15:23:50.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:50 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:50.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:50 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:50.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:50 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:23:50.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:50 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:23:50.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:50 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:23:50.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:50 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:23:50.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:50 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:50.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:50 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:23:50.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:50 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:23:50.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:50 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:50.602 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:50 smithi142 ceph-mon[45728]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-02-02T15:23:50.602 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:50 smithi142 ceph-mon[45728]: osdmap e65: 11 total, 10 up, 11 in 2024-02-02T15:23:50.602 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:50 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-02T15:23:50.602 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:50 smithi142 ceph-mon[45728]: from='osd.10 [v2:172.21.15.71:6824/84691841,v1:172.21.15.71:6825/84691841]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-02-02T15:23:50.602 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:50 smithi142 ceph-mon[45728]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-02-02T15:23:50.602 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:50 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:50.602 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:50 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:50.602 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:50 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:23:50.602 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:50 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:23:50.603 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:50 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:23:50.603 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:50 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:23:50.603 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:50 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:50.603 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:50 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:23:50.603 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:50 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:23:50.603 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:50 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:50.614 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:23:50 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:23:50.178+0000 7f4068cd8700 -1 osd.10 0 waiting for initial osdmap 2024-02-02T15:23:50.614 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:23:50 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:23:50.190+0000 7f4060ac5700 -1 osd.10 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-02T15:23:50.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:50 smithi071 ceph-mon[44707]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-02-02T15:23:50.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:50 smithi071 ceph-mon[44707]: osdmap e65: 11 total, 10 up, 11 in 2024-02-02T15:23:50.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:50 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-02T15:23:50.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:50 smithi071 ceph-mon[44707]: from='osd.10 [v2:172.21.15.71:6824/84691841,v1:172.21.15.71:6825/84691841]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-02-02T15:23:50.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:50 smithi071 ceph-mon[44707]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-02-02T15:23:50.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:50 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:50.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:50 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:50.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:50 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:23:50.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:50 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:23:50.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:50 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:23:50.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:50 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:23:50.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:50 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:50.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:50 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:23:50.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:50 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:23:50.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:50 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:23:50.773 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:23:50.770+0000 7f51b5cc0700 1 -- 172.21.15.142:0/249758324 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51b0074d70 msgr2=0x7f51b00731d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:23:50.773 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:23:50.770+0000 7f51b5cc0700 1 --2- 172.21.15.142:0/249758324 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51b0074d70 0x7f51b00731d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f51a0009990 tx=0x7f51a0009ca0 comp rx=0 tx=0).stop 2024-02-02T15:23:50.773 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:23:50.770+0000 7f51b5cc0700 1 -- 172.21.15.142:0/249758324 shutdown_connections 2024-02-02T15:23:50.773 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:23:50.770+0000 7f51b5cc0700 1 --2- 172.21.15.142:0/249758324 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51b010d3c0 0x7f51b010f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:23:50.773 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:23:50.770+0000 7f51b5cc0700 1 --2- 172.21.15.142:0/249758324 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51b0074d70 0x7f51b00731d0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:23:50.773 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:23:50.771+0000 7f51b5cc0700 1 --2- 172.21.15.142:0/249758324 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f51b00737a0 0x7f51b0073c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:23:50.773 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:23:50.771+0000 7f51b5cc0700 1 -- 172.21.15.142:0/249758324 >> 172.21.15.142:0/249758324 conn(0x7f51b00fc360 msgr2=0x7f51b00fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:23:50.773 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:23:50.771+0000 7f51b5cc0700 1 -- 172.21.15.142:0/249758324 shutdown_connections 2024-02-02T15:23:50.773 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:23:50.771+0000 7f51b5cc0700 1 -- 172.21.15.142:0/249758324 wait complete. 2024-02-02T15:23:50.773 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:23:50.771+0000 7f51b5cc0700 1 Processor -- start 2024-02-02T15:23:50.773 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:23:50.771+0000 7f51b5cc0700 1 -- start start 2024-02-02T15:23:50.774 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:23:50.771+0000 7f51b5cc0700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51b00737a0 0x7f51b019d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:23:50.774 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:23:50.771+0000 7f51b5cc0700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f51b0074d70 0x7f51b019d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:23:50.774 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:23:50.771+0000 7f51b5cc0700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51b010d3c0 0x7f51b01a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:23:50.774 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:23:50.772+0000 7f51b5cc0700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b01a4cc0 con 0x7f51b0074d70 2024-02-02T15:23:50.774 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:23:50.772+0000 7f51b5cc0700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b01a4e00 con 0x7f51b010d3c0 2024-02-02T15:23:50.774 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:23:50.772+0000 7f51b5cc0700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b01a4f90 con 0x7f51b00737a0 2024-02-02T15:23:50.774 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:23:50.772+0000 7f51af7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51b00737a0 0x7f51b019d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:23:50.774 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:23:50.772+0000 7f51af7fe700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51b00737a0 0x7f51b019d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:49124/0 (socket says 172.21.15.142:49124) 2024-02-02T15:23:50.774 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:23:50.772+0000 7f51af7fe700 1 -- 172.21.15.142:0/689037907 learned_addr learned my addr 172.21.15.142:0/689037907 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-02-02T15:23:50.774 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:23:50.772+0000 7f51affff700 1 --2- 172.21.15.142:0/689037907 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51b010d3c0 0x7f51b01a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:23:50.774 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:23:50.772+0000 7f51aeffd700 1 --2- 172.21.15.142:0/689037907 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f51b0074d70 0x7f51b019d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:23:50.774 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:23:50.772+0000 7f51af7fe700 1 -- 172.21.15.142:0/689037907 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51b010d3c0 msgr2=0x7f51b01a46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:23:50.774 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:23:50.772+0000 7f51af7fe700 1 --2- 172.21.15.142:0/689037907 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51b010d3c0 0x7f51b01a46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:23:50.775 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:23:50.772+0000 7f51af7fe700 1 -- 172.21.15.142:0/689037907 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f51b0074d70 msgr2=0x7f51b019d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:23:50.775 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:23:50.772+0000 7f51af7fe700 1 --2- 172.21.15.142:0/689037907 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f51b0074d70 0x7f51b019d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:23:50.775 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:23:50.772+0000 7f51af7fe700 1 -- 172.21.15.142:0/689037907 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51a00095f0 con 0x7f51b00737a0 2024-02-02T15:23:50.775 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:23:50.772+0000 7f51aeffd700 1 --2- 172.21.15.142:0/689037907 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f51b0074d70 0x7f51b019d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:23:50.775 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:23:50.772+0000 7f51af7fe700 1 --2- 172.21.15.142:0/689037907 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51b00737a0 0x7f51b019d130 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f51a0004730 tx=0x7f51a0004760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:23:50.776 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:23:50.776+0000 7f51acff9700 1 -- 172.21.15.142:0/689037907 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a001d070 con 0x7f51b00737a0 2024-02-02T15:23:50.776 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:23:50.776+0000 7f51acff9700 1 -- 172.21.15.142:0/689037907 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51a00054f0 con 0x7f51b00737a0 2024-02-02T15:23:50.776 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:23:50.776+0000 7f51b5cc0700 1 -- 172.21.15.142:0/689037907 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51b01a5210 con 0x7f51b00737a0 2024-02-02T15:23:50.776 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:23:50.776+0000 7f51acff9700 1 -- 172.21.15.142:0/689037907 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a0004c60 con 0x7f51b00737a0 2024-02-02T15:23:50.776 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:23:50.776+0000 7f51b5cc0700 1 -- 172.21.15.142:0/689037907 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51b01a5620 con 0x7f51b00737a0 2024-02-02T15:23:50.777 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:23:50.777+0000 7f51b5cc0700 1 -- 172.21.15.142:0/689037907 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51b0102f10 con 0x7f51b00737a0 2024-02-02T15:23:50.778 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:23:50.778+0000 7f51acff9700 1 -- 172.21.15.142:0/689037907 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 16) v1 ==== 135069+0+0 (secure 0 0 0) 0x7f51a0021400 con 0x7f51b00737a0 2024-02-02T15:23:50.778 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:23:50.778+0000 7f51acff9700 1 --2- 172.21.15.142:0/689037907 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f51980a05a0 0x7f51980a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:23:50.779 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:23:50.779+0000 7f51acff9700 1 -- 172.21.15.142:0/689037907 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6727+0+0 (secure 0 0 0) 0x7f51a0005660 con 0x7f51b00737a0 2024-02-02T15:23:50.779 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:23:50.779+0000 7f51aeffd700 1 --2- 172.21.15.142:0/689037907 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f51980a05a0 0x7f51980a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:23:50.779 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:23:50.779+0000 7f51aeffd700 1 --2- 172.21.15.142:0/689037907 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f51980a05a0 0x7f51980a2a60 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f51a4005fd0 tx=0x7f51a4005dc0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:23:50.783 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:23:50.783+0000 7f51acff9700 1 -- 172.21.15.142:0/689037907 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f51a005b110 con 0x7f51b00737a0 2024-02-02T15:23:50.982 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:23:50.981+0000 7f51b5cc0700 1 -- 172.21.15.142:0/689037907 --> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f51b0061180 con 0x7f51980a05a0 2024-02-02T15:23:51.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:51 smithi001 ceph-mon[35470]: Detected new or changed devices on smithi071 2024-02-02T15:23:51.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:51 smithi001 ceph-mon[35470]: Adjusting osd_memory_target on smithi071 to 4278M 2024-02-02T15:23:51.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:51 smithi001 ceph-mon[35470]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2024-02-02T15:23:51.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:51 smithi001 ceph-mon[35470]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-02-02T15:23:51.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:51 smithi001 ceph-mon[35470]: osdmap e66: 11 total, 10 up, 11 in 2024-02-02T15:23:51.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:51 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-02T15:23:51.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:51 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-02T15:23:51.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:51 smithi001 ceph-mon[35470]: osd.10 [v2:172.21.15.71:6824/84691841,v1:172.21.15.71:6825/84691841] boot 2024-02-02T15:23:51.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:51 smithi001 ceph-mon[35470]: osdmap e67: 11 total, 11 up, 11 in 2024-02-02T15:23:51.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:51 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-02T15:23:51.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:51 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-02T15:23:51.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:51 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-02T15:23:51.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:51 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:23:51.517 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:51 smithi142 ceph-mon[45728]: Detected new or changed devices on smithi071 2024-02-02T15:23:51.517 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:51 smithi142 ceph-mon[45728]: Adjusting osd_memory_target on smithi071 to 4278M 2024-02-02T15:23:51.517 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:51 smithi142 ceph-mon[45728]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2024-02-02T15:23:51.517 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:51 smithi142 ceph-mon[45728]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-02-02T15:23:51.517 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:51 smithi142 ceph-mon[45728]: osdmap e66: 11 total, 10 up, 11 in 2024-02-02T15:23:51.517 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:51 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-02T15:23:51.518 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:51 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-02T15:23:51.518 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:51 smithi142 ceph-mon[45728]: osd.10 [v2:172.21.15.71:6824/84691841,v1:172.21.15.71:6825/84691841] boot 2024-02-02T15:23:51.518 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:51 smithi142 ceph-mon[45728]: osdmap e67: 11 total, 11 up, 11 in 2024-02-02T15:23:51.518 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:51 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-02T15:23:51.518 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:51 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-02T15:23:51.518 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:51 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-02T15:23:51.518 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:51 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:23:51.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:51 smithi071 ceph-mon[44707]: Detected new or changed devices on smithi071 2024-02-02T15:23:51.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:51 smithi071 ceph-mon[44707]: Adjusting osd_memory_target on smithi071 to 4278M 2024-02-02T15:23:51.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:51 smithi071 ceph-mon[44707]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2024-02-02T15:23:51.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:51 smithi071 ceph-mon[44707]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-02-02T15:23:51.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:51 smithi071 ceph-mon[44707]: osdmap e66: 11 total, 10 up, 11 in 2024-02-02T15:23:51.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:51 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-02T15:23:51.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:51 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-02T15:23:51.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:51 smithi071 ceph-mon[44707]: osd.10 [v2:172.21.15.71:6824/84691841,v1:172.21.15.71:6825/84691841] boot 2024-02-02T15:23:51.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:51 smithi071 ceph-mon[44707]: osdmap e67: 11 total, 11 up, 11 in 2024-02-02T15:23:51.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:51 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-02T15:23:51.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:51 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-02T15:23:51.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:51 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-02T15:23:51.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:51 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:23:52.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:52 smithi001 ceph-mon[35470]: purged_snaps scrub starts 2024-02-02T15:23:52.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:52 smithi001 ceph-mon[35470]: purged_snaps scrub ok 2024-02-02T15:23:52.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:52 smithi001 ceph-mon[35470]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:23:52.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:52 smithi001 ceph-mon[35470]: osdmap e68: 11 total, 11 up, 11 in 2024-02-02T15:23:52.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:52 smithi071 ceph-mon[44707]: purged_snaps scrub starts 2024-02-02T15:23:52.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:52 smithi071 ceph-mon[44707]: purged_snaps scrub ok 2024-02-02T15:23:52.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:52 smithi071 ceph-mon[44707]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:23:52.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:52 smithi071 ceph-mon[44707]: osdmap e68: 11 total, 11 up, 11 in 2024-02-02T15:23:52.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:52 smithi142 ceph-mon[45728]: purged_snaps scrub starts 2024-02-02T15:23:52.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:52 smithi142 ceph-mon[45728]: purged_snaps scrub ok 2024-02-02T15:23:52.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:52 smithi142 ceph-mon[45728]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi142:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:23:52.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:52 smithi142 ceph-mon[45728]: osdmap e68: 11 total, 11 up, 11 in 2024-02-02T15:23:53.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:53 smithi001 ceph-mon[35470]: pgmap v191: 1 pgs: 1 remapped+peering; 577 KiB data, 297 MiB used, 983 GiB / 983 GiB avail 2024-02-02T15:23:53.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:53 smithi001 ceph-mon[35470]: from='client.? 172.21.15.142:0/1781092676' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f011dd27-11b0-4166-adfe-76756533163a"}]: dispatch 2024-02-02T15:23:53.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:53 smithi001 ceph-mon[35470]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f011dd27-11b0-4166-adfe-76756533163a"}]: dispatch 2024-02-02T15:23:53.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:53 smithi001 ceph-mon[35470]: osdmap e69: 11 total, 11 up, 11 in 2024-02-02T15:23:53.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:53 smithi001 ceph-mon[35470]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f011dd27-11b0-4166-adfe-76756533163a"}]': finished 2024-02-02T15:23:53.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:53 smithi001 ceph-mon[35470]: osdmap e70: 12 total, 11 up, 12 in 2024-02-02T15:23:53.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:53 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-02T15:23:53.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:53 smithi071 ceph-mon[44707]: pgmap v191: 1 pgs: 1 remapped+peering; 577 KiB data, 297 MiB used, 983 GiB / 983 GiB avail 2024-02-02T15:23:53.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:53 smithi071 ceph-mon[44707]: from='client.? 172.21.15.142:0/1781092676' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f011dd27-11b0-4166-adfe-76756533163a"}]: dispatch 2024-02-02T15:23:53.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:53 smithi071 ceph-mon[44707]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f011dd27-11b0-4166-adfe-76756533163a"}]: dispatch 2024-02-02T15:23:53.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:53 smithi071 ceph-mon[44707]: osdmap e69: 11 total, 11 up, 11 in 2024-02-02T15:23:53.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:53 smithi071 ceph-mon[44707]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f011dd27-11b0-4166-adfe-76756533163a"}]': finished 2024-02-02T15:23:53.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:53 smithi071 ceph-mon[44707]: osdmap e70: 12 total, 11 up, 12 in 2024-02-02T15:23:53.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:53 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-02T15:23:53.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:53 smithi142 ceph-mon[45728]: pgmap v191: 1 pgs: 1 remapped+peering; 577 KiB data, 297 MiB used, 983 GiB / 983 GiB avail 2024-02-02T15:23:53.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:53 smithi142 ceph-mon[45728]: from='client.? 172.21.15.142:0/1781092676' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f011dd27-11b0-4166-adfe-76756533163a"}]: dispatch 2024-02-02T15:23:53.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:53 smithi142 ceph-mon[45728]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f011dd27-11b0-4166-adfe-76756533163a"}]: dispatch 2024-02-02T15:23:53.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:53 smithi142 ceph-mon[45728]: osdmap e69: 11 total, 11 up, 11 in 2024-02-02T15:23:53.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:53 smithi142 ceph-mon[45728]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f011dd27-11b0-4166-adfe-76756533163a"}]': finished 2024-02-02T15:23:53.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:53 smithi142 ceph-mon[45728]: osdmap e70: 12 total, 11 up, 12 in 2024-02-02T15:23:53.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:53 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-02T15:23:54.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:54 smithi001 ceph-mon[35470]: from='client.? 172.21.15.142:0/3951236118' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-02T15:23:54.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:54 smithi071 ceph-mon[44707]: from='client.? 172.21.15.142:0/3951236118' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-02T15:23:54.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:54 smithi142 ceph-mon[45728]: from='client.? 172.21.15.142:0/3951236118' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-02T15:23:55.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:55 smithi001 ceph-mon[35470]: pgmap v194: 1 pgs: 1 remapped+peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2024-02-02T15:23:55.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:55 smithi001 ceph-mon[35470]: osdmap e71: 12 total, 11 up, 12 in 2024-02-02T15:23:55.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:55 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-02T15:23:55.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:55 smithi071 ceph-mon[44707]: pgmap v194: 1 pgs: 1 remapped+peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2024-02-02T15:23:55.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:55 smithi071 ceph-mon[44707]: osdmap e71: 12 total, 11 up, 12 in 2024-02-02T15:23:55.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:55 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-02T15:23:55.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:55 smithi142 ceph-mon[45728]: pgmap v194: 1 pgs: 1 remapped+peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2024-02-02T15:23:55.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:55 smithi142 ceph-mon[45728]: osdmap e71: 12 total, 11 up, 12 in 2024-02-02T15:23:55.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:55 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-02T15:23:56.454 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:56 smithi142 ceph-mon[45728]: osdmap e72: 12 total, 11 up, 12 in 2024-02-02T15:23:56.455 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:56 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-02T15:23:56.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:56 smithi071 ceph-mon[44707]: osdmap e72: 12 total, 11 up, 12 in 2024-02-02T15:23:56.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:56 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-02T15:23:56.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:56 smithi001 ceph-mon[35470]: osdmap e72: 12 total, 11 up, 12 in 2024-02-02T15:23:56.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:56 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-02T15:23:57.491 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:57 smithi142 ceph-mon[45728]: pgmap v197: 1 pgs: 1 remapped+peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2024-02-02T15:23:57.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:57 smithi071 ceph-mon[44707]: pgmap v197: 1 pgs: 1 remapped+peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2024-02-02T15:23:57.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:57 smithi001 ceph-mon[35470]: pgmap v197: 1 pgs: 1 remapped+peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2024-02-02T15:23:59.307 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:59 smithi142 ceph-mon[45728]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 113 KiB/s, 0 objects/s recovering 2024-02-02T15:23:59.307 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:59 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-02-02T15:23:59.307 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:23:59 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:23:59.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:59 smithi071 ceph-mon[44707]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 113 KiB/s, 0 objects/s recovering 2024-02-02T15:23:59.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:59 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-02-02T15:23:59.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:23:59 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:23:59.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:59 smithi001 ceph-mon[35470]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 113 KiB/s, 0 objects/s recovering 2024-02-02T15:23:59.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:59 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-02-02T15:23:59.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:23:59 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:24:00.369 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:00 smithi142 ceph-mon[45728]: Deploying daemon osd.11 on smithi142 2024-02-02T15:24:00.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:00 smithi071 ceph-mon[44707]: Deploying daemon osd.11 on smithi142 2024-02-02T15:24:00.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:00 smithi001 ceph-mon[35470]: Deploying daemon osd.11 on smithi142 2024-02-02T15:24:01.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:01 smithi142 ceph-mon[45728]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-02-02T15:24:01.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:01 smithi071 ceph-mon[44707]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-02-02T15:24:01.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:01 smithi001 ceph-mon[35470]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-02-02T15:24:03.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:03 smithi142 ceph-mon[45728]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-02T15:24:03.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:03 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:03.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:03 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:03.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:03 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:24:03.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:03 smithi071 ceph-mon[44707]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-02T15:24:03.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:03 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:03.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:03 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:03.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:03 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:24:03.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:03 smithi001 ceph-mon[35470]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-02-02T15:24:03.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:03 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:03.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:03 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:03.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:03 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:24:04.690 INFO:teuthology.orchestra.run.smithi142.stdout:Created osd(s) 11 on host 'smithi142' 2024-02-02T15:24:04.691 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:24:04.690+0000 7f51acff9700 1 -- 172.21.15.142:0/689037907 <== mgr.14154 v2:172.21.15.1:6800/3768989981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f51b0061180 con 0x7f51980a05a0 2024-02-02T15:24:04.692 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:24:04.692+0000 7f51b5cc0700 1 -- 172.21.15.142:0/689037907 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f51980a05a0 msgr2=0x7f51980a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:04.692 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:24:04.692+0000 7f51b5cc0700 1 --2- 172.21.15.142:0/689037907 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f51980a05a0 0x7f51980a2a60 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f51a4005fd0 tx=0x7f51a4005dc0 comp rx=0 tx=0).stop 2024-02-02T15:24:04.692 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:24:04.692+0000 7f51b5cc0700 1 -- 172.21.15.142:0/689037907 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51b00737a0 msgr2=0x7f51b019d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:04.693 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:24:04.692+0000 7f51b5cc0700 1 --2- 172.21.15.142:0/689037907 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51b00737a0 0x7f51b019d130 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f51a0004730 tx=0x7f51a0004760 comp rx=0 tx=0).stop 2024-02-02T15:24:04.693 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:24:04.692+0000 7f51b5cc0700 1 -- 172.21.15.142:0/689037907 shutdown_connections 2024-02-02T15:24:04.693 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:24:04.692+0000 7f51b5cc0700 1 --2- 172.21.15.142:0/689037907 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51b010d3c0 0x7f51b01a46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:04.693 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:24:04.692+0000 7f51b5cc0700 1 --2- 172.21.15.142:0/689037907 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f51980a05a0 0x7f51980a2a60 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:04.693 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:24:04.692+0000 7f51b5cc0700 1 --2- 172.21.15.142:0/689037907 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51b00737a0 0x7f51b019d130 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:04.693 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:24:04.692+0000 7f51b5cc0700 1 --2- 172.21.15.142:0/689037907 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f51b0074d70 0x7f51b019d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:04.693 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:24:04.692+0000 7f51b5cc0700 1 -- 172.21.15.142:0/689037907 >> 172.21.15.142:0/689037907 conn(0x7f51b00fc360 msgr2=0x7f51b00fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:24:04.693 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:24:04.692+0000 7f51b5cc0700 1 -- 172.21.15.142:0/689037907 shutdown_connections 2024-02-02T15:24:04.693 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:24:04.692+0000 7f51b5cc0700 1 -- 172.21.15.142:0/689037907 wait complete. 2024-02-02T15:24:05.144 DEBUG:teuthology.orchestra.run.smithi142:osd.11> sudo journalctl -f -n 0 -u ceph-371a409c-c1de-11ee-95b6-87774f69a715@osd.11.service 2024-02-02T15:24:05.147 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-02-02T15:24:05.147 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph osd stat -f json 2024-02-02T15:24:05.178 INFO:journalctl@ceph.osd.11.smithi142.stdout:-- Logs begin at Fri 2024-02-02 14:58:43 UTC. -- 2024-02-02T15:24:05.255 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:05 smithi142 ceph-mon[45728]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-02-02T15:24:05.256 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:05 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:05.256 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:05 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:05.256 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:05 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:24:05.256 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:05 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:24:05.256 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:05 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:05.256 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:05 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:05.256 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:05 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:05.256 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:05 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:24:05.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:05 smithi001 ceph-mon[35470]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-02-02T15:24:05.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:05 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:05.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:05 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:05.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:05 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:24:05.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:05 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:24:05.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:05 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:05.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:05 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:05.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:05 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:05.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:05 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:24:05.523 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.a/config 2024-02-02T15:24:05.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:05 smithi071 ceph-mon[44707]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-02-02T15:24:05.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:05 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:05.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:05 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:05.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:05 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:24:05.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:05 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:24:05.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:05 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:05.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:05 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:05.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:05 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:05.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:05 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:24:06.670 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:24:06 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:24:06.356+0000 7f9aa9ef3640 -1 osd.11 0 log_to_monitors true 2024-02-02T15:24:06.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:06.811+0000 7f4321800700 1 -- 172.21.15.1:0/2536875023 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f431c104190 msgr2=0x7f431c106580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:06.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:06.811+0000 7f4321800700 1 --2- 172.21.15.1:0/2536875023 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f431c104190 0x7f431c106580 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4304009990 tx=0x7f4304009ca0 comp rx=0 tx=0).stop 2024-02-02T15:24:06.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:06.811+0000 7f4321800700 1 -- 172.21.15.1:0/2536875023 shutdown_connections 2024-02-02T15:24:06.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:06.811+0000 7f4321800700 1 --2- 172.21.15.1:0/2536875023 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f431c106ac0 0x7f431c108f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:06.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:06.811+0000 7f4321800700 1 --2- 172.21.15.1:0/2536875023 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f431c101860 0x7f431c103c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:06.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:06.811+0000 7f4321800700 1 --2- 172.21.15.1:0/2536875023 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f431c104190 0x7f431c106580 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:06.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:06.811+0000 7f4321800700 1 -- 172.21.15.1:0/2536875023 >> 172.21.15.1:0/2536875023 conn(0x7f431c0fb250 msgr2=0x7f431c0fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:24:06.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:06.811+0000 7f4321800700 1 -- 172.21.15.1:0/2536875023 shutdown_connections 2024-02-02T15:24:06.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:06.811+0000 7f4321800700 1 -- 172.21.15.1:0/2536875023 wait complete. 2024-02-02T15:24:06.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:06.812+0000 7f4321800700 1 Processor -- start 2024-02-02T15:24:06.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:06.812+0000 7f4321800700 1 -- start start 2024-02-02T15:24:06.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:06.812+0000 7f4321800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f431c101860 0x7f431c198ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:06.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:06.812+0000 7f4321800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f431c104190 0x7f431c199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:06.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:06.812+0000 7f4321800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f431c106ac0 0x7f431c1a0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:06.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:06.812+0000 7f4321800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431c1a0830 con 0x7f431c101860 2024-02-02T15:24:06.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:06.812+0000 7f4321800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431c1a0970 con 0x7f431c104190 2024-02-02T15:24:06.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:06.812+0000 7f4321800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431c1a0b30 con 0x7f431c106ac0 2024-02-02T15:24:06.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:06.813+0000 7f431aa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f431c101860 0x7f431c198ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:06.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:06.813+0000 7f431b400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f431c106ac0 0x7f431c1a0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:06.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:06.813+0000 7f431aa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f431c101860 0x7f431c198ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50402/0 (socket says 172.21.15.1:50402) 2024-02-02T15:24:06.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:06.813+0000 7f431a000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f431c104190 0x7f431c199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:06.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:06.813+0000 7f431aa00700 1 -- 172.21.15.1:0/1789305022 learned_addr learned my addr 172.21.15.1:0/1789305022 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:24:06.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:06.813+0000 7f431b400700 1 -- 172.21.15.1:0/1789305022 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f431c104190 msgr2=0x7f431c199220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:06.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:06.813+0000 7f431b400700 1 --2- 172.21.15.1:0/1789305022 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f431c104190 0x7f431c199220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:06.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:06.813+0000 7f431b400700 1 -- 172.21.15.1:0/1789305022 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f431c101860 msgr2=0x7f431c198ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:06.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:06.813+0000 7f431b400700 1 --2- 172.21.15.1:0/1789305022 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f431c101860 0x7f431c198ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:06.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:06.813+0000 7f431b400700 1 -- 172.21.15.1:0/1789305022 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f430c009520 con 0x7f431c106ac0 2024-02-02T15:24:06.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:06.813+0000 7f431aa00700 1 --2- 172.21.15.1:0/1789305022 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f431c101860 0x7f431c198ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:24:06.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:06.813+0000 7f431b400700 1 --2- 172.21.15.1:0/1789305022 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f431c106ac0 0x7f431c1a0260 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f431400ef90 tx=0x7f431400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:24:06.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:06.814+0000 7f4313400700 1 -- 172.21.15.1:0/1789305022 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4314018070 con 0x7f431c106ac0 2024-02-02T15:24:06.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:06.814+0000 7f4313400700 1 -- 172.21.15.1:0/1789305022 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f431400f040 con 0x7f431c106ac0 2024-02-02T15:24:06.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:06.814+0000 7f4313400700 1 -- 172.21.15.1:0/1789305022 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431400cc30 con 0x7f431c106ac0 2024-02-02T15:24:06.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:06.814+0000 7f4321800700 1 -- 172.21.15.1:0/1789305022 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43040095f0 con 0x7f431c106ac0 2024-02-02T15:24:06.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:06.814+0000 7f4321800700 1 -- 172.21.15.1:0/1789305022 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f431c1a10d0 con 0x7f431c106ac0 2024-02-02T15:24:06.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:06.815+0000 7f4321800700 1 -- 172.21.15.1:0/1789305022 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f431c108690 con 0x7f431c106ac0 2024-02-02T15:24:06.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:06.817+0000 7f4313400700 1 -- 172.21.15.1:0/1789305022 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 16) v1 ==== 135069+0+0 (secure 0 0 0) 0x7f4314003a00 con 0x7f431c106ac0 2024-02-02T15:24:06.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:06.818+0000 7f4313400700 1 --2- 172.21.15.1:0/1789305022 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f42fc0a05a0 0x7f42fc0a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:06.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:06.818+0000 7f4313400700 1 -- 172.21.15.1:0/1789305022 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6831+0+0 (secure 0 0 0) 0x7f43140cb7c0 con 0x7f431c106ac0 2024-02-02T15:24:06.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:06.818+0000 7f431aa00700 1 --2- 172.21.15.1:0/1789305022 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f42fc0a05a0 0x7f42fc0a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:06.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:06.819+0000 7f431aa00700 1 --2- 172.21.15.1:0/1789305022 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f42fc0a05a0 0x7f42fc0a2a60 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f431c19a1c0 tx=0x7f430c009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:24:06.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:06.823+0000 7f4313400700 1 -- 172.21.15.1:0/1789305022 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f4314095920 con 0x7f431c106ac0 2024-02-02T15:24:06.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:06 smithi001 ceph-mon[35470]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-02-02T15:24:06.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:06 smithi001 ceph-mon[35470]: from='osd.11 [v2:172.21.15.142:6824/2290179934,v1:172.21.15.142:6825/2290179934]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-02-02T15:24:06.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:06 smithi001 ceph-mon[35470]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-02-02T15:24:07.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:07.030+0000 7f4321800700 1 -- 172.21.15.1:0/1789305022 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f431c02cc70 con 0x7f431c106ac0 2024-02-02T15:24:07.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:07.030+0000 7f4313400700 1 -- 172.21.15.1:0/1789305022 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v73) v1 ==== 74+0+133 (secure 0 0 0) 0x7f4314098f40 con 0x7f431c106ac0 2024-02-02T15:24:07.030 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:24:07.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:07.033+0000 7f4321800700 1 -- 172.21.15.1:0/1789305022 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f42fc0a05a0 msgr2=0x7f42fc0a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:07.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:07.033+0000 7f4321800700 1 --2- 172.21.15.1:0/1789305022 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f42fc0a05a0 0x7f42fc0a2a60 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f431c19a1c0 tx=0x7f430c009260 comp rx=0 tx=0).stop 2024-02-02T15:24:07.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:07.033+0000 7f4321800700 1 -- 172.21.15.1:0/1789305022 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f431c106ac0 msgr2=0x7f431c1a0260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:07.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:07.033+0000 7f4321800700 1 --2- 172.21.15.1:0/1789305022 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f431c106ac0 0x7f431c1a0260 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f431400ef90 tx=0x7f431400c3a0 comp rx=0 tx=0).stop 2024-02-02T15:24:07.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:07.033+0000 7f4321800700 1 -- 172.21.15.1:0/1789305022 shutdown_connections 2024-02-02T15:24:07.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:07.033+0000 7f4321800700 1 --2- 172.21.15.1:0/1789305022 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f42fc0a05a0 0x7f42fc0a2a60 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:07.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:07.033+0000 7f4321800700 1 --2- 172.21.15.1:0/1789305022 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f431c106ac0 0x7f431c1a0260 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:07.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:07.033+0000 7f4321800700 1 --2- 172.21.15.1:0/1789305022 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f431c101860 0x7f431c198ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:07.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:07.033+0000 7f4321800700 1 --2- 172.21.15.1:0/1789305022 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f431c104190 0x7f431c199220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:07.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:07.033+0000 7f4321800700 1 -- 172.21.15.1:0/1789305022 >> 172.21.15.1:0/1789305022 conn(0x7f431c0fb250 msgr2=0x7f431c0ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:24:07.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:07.033+0000 7f4321800700 1 -- 172.21.15.1:0/1789305022 shutdown_connections 2024-02-02T15:24:07.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:07.033+0000 7f4321800700 1 -- 172.21.15.1:0/1789305022 wait complete. 2024-02-02T15:24:07.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:06 smithi071 ceph-mon[44707]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-02-02T15:24:07.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:06 smithi071 ceph-mon[44707]: from='osd.11 [v2:172.21.15.142:6824/2290179934,v1:172.21.15.142:6825/2290179934]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-02-02T15:24:07.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:06 smithi071 ceph-mon[44707]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-02-02T15:24:07.134 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:06 smithi142 ceph-mon[45728]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-02-02T15:24:07.134 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:06 smithi142 ceph-mon[45728]: from='osd.11 [v2:172.21.15.142:6824/2290179934,v1:172.21.15.142:6825/2290179934]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-02-02T15:24:07.135 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:06 smithi142 ceph-mon[45728]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-02-02T15:24:07.528 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":73,"num_osds":12,"num_up_osds":11,"osd_up_since":1706887430,"num_in_osds":12,"osd_in_since":1706887432,"num_remapped_pgs":0} 2024-02-02T15:24:08.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:07 smithi071 ceph-mon[44707]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-02-02T15:24:08.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:07 smithi071 ceph-mon[44707]: osdmap e73: 12 total, 11 up, 12 in 2024-02-02T15:24:08.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:07 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-02T15:24:08.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:07 smithi071 ceph-mon[44707]: from='osd.11 [v2:172.21.15.142:6824/2290179934,v1:172.21.15.142:6825/2290179934]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-02-02T15:24:08.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:07 smithi071 ceph-mon[44707]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-02-02T15:24:08.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:07 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/1789305022' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-02T15:24:08.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:07 smithi071 ceph-mon[44707]: Detected new or changed devices on smithi142 2024-02-02T15:24:08.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:07 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:08.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:07 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:08.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:07 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:24:08.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:07 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:24:08.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:07 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:24:08.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:07 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:24:08.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:07 smithi071 ceph-mon[44707]: Adjusting osd_memory_target on smithi142 to 4278M 2024-02-02T15:24:08.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:07 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:08.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:07 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:24:08.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:07 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:24:08.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:07 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:08.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:24:07 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:24:07.795+0000 7f9a9f557700 -1 osd.11 0 waiting for initial osdmap 2024-02-02T15:24:08.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:24:07 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:24:07.812+0000 7f9a9ab4b700 -1 osd.11 74 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-02T15:24:08.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:07 smithi142 ceph-mon[45728]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-02-02T15:24:08.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:07 smithi142 ceph-mon[45728]: osdmap e73: 12 total, 11 up, 12 in 2024-02-02T15:24:08.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:07 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-02T15:24:08.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:07 smithi142 ceph-mon[45728]: from='osd.11 [v2:172.21.15.142:6824/2290179934,v1:172.21.15.142:6825/2290179934]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-02-02T15:24:08.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:07 smithi142 ceph-mon[45728]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-02-02T15:24:08.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:07 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/1789305022' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-02T15:24:08.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:07 smithi142 ceph-mon[45728]: Detected new or changed devices on smithi142 2024-02-02T15:24:08.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:07 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:08.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:07 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:08.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:07 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:24:08.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:07 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:24:08.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:07 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:24:08.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:07 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:24:08.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:07 smithi142 ceph-mon[45728]: Adjusting osd_memory_target on smithi142 to 4278M 2024-02-02T15:24:08.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:07 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:08.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:07 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:24:08.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:07 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:24:08.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:07 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:08.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:07 smithi001 ceph-mon[35470]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-02-02T15:24:08.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:07 smithi001 ceph-mon[35470]: osdmap e73: 12 total, 11 up, 12 in 2024-02-02T15:24:08.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:07 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-02T15:24:08.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:07 smithi001 ceph-mon[35470]: from='osd.11 [v2:172.21.15.142:6824/2290179934,v1:172.21.15.142:6825/2290179934]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-02-02T15:24:08.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:07 smithi001 ceph-mon[35470]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]: dispatch 2024-02-02T15:24:08.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:07 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/1789305022' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-02T15:24:08.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:07 smithi001 ceph-mon[35470]: Detected new or changed devices on smithi142 2024-02-02T15:24:08.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:07 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:08.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:07 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:08.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:07 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:24:08.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:07 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:24:08.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:07 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:24:08.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:07 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:24:08.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:07 smithi001 ceph-mon[35470]: Adjusting osd_memory_target on smithi142 to 4278M 2024-02-02T15:24:08.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:07 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:08.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:07 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:24:08.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:07 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:24:08.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:07 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:08.529 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph osd stat -f json 2024-02-02T15:24:08.903 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.a/config 2024-02-02T15:24:08.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:08 smithi001 ceph-mon[35470]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-02-02T15:24:08.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:08 smithi001 ceph-mon[35470]: osdmap e74: 12 total, 11 up, 12 in 2024-02-02T15:24:08.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:08 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-02T15:24:08.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:08 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-02T15:24:08.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:08 smithi001 ceph-mon[35470]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2024-02-02T15:24:09.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:08 smithi071 ceph-mon[44707]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-02-02T15:24:09.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:08 smithi071 ceph-mon[44707]: osdmap e74: 12 total, 11 up, 12 in 2024-02-02T15:24:09.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:08 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-02T15:24:09.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:08 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-02T15:24:09.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:08 smithi071 ceph-mon[44707]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2024-02-02T15:24:09.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:08 smithi142 ceph-mon[45728]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi142", "root=default"]}]': finished 2024-02-02T15:24:09.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:08 smithi142 ceph-mon[45728]: osdmap e74: 12 total, 11 up, 12 in 2024-02-02T15:24:09.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:08 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-02T15:24:09.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:08 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-02T15:24:09.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:08 smithi142 ceph-mon[45728]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2024-02-02T15:24:10.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:10.114+0000 7f6387400700 1 -- 172.21.15.1:0/2347555862 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6380101fd0 msgr2=0x7f6380102430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:10.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:10.114+0000 7f6387400700 1 --2- 172.21.15.1:0/2347555862 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6380101fd0 0x7f6380102430 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f63680099c0 tx=0x7f6368009cd0 comp rx=0 tx=0).stop 2024-02-02T15:24:10.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:10.114+0000 7f6387400700 1 -- 172.21.15.1:0/2347555862 shutdown_connections 2024-02-02T15:24:10.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:10.114+0000 7f6387400700 1 --2- 172.21.15.1:0/2347555862 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6380102970 0x7f6380109220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:10.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:10.114+0000 7f6387400700 1 --2- 172.21.15.1:0/2347555862 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6380100dd0 0x7f63801011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:10.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:10.114+0000 7f6387400700 1 --2- 172.21.15.1:0/2347555862 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6380101fd0 0x7f6380102430 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:10.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:10.114+0000 7f6387400700 1 -- 172.21.15.1:0/2347555862 >> 172.21.15.1:0/2347555862 conn(0x7f63800fc330 msgr2=0x7f63800fe7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:24:10.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:10.114+0000 7f6387400700 1 -- 172.21.15.1:0/2347555862 shutdown_connections 2024-02-02T15:24:10.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:10.114+0000 7f6387400700 1 -- 172.21.15.1:0/2347555862 wait complete. 2024-02-02T15:24:10.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:09 smithi071 ceph-mon[44707]: purged_snaps scrub starts 2024-02-02T15:24:10.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:09 smithi071 ceph-mon[44707]: purged_snaps scrub ok 2024-02-02T15:24:10.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:09 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-02T15:24:10.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:09 smithi071 ceph-mon[44707]: osd.11 [v2:172.21.15.142:6824/2290179934,v1:172.21.15.142:6825/2290179934] boot 2024-02-02T15:24:10.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:09 smithi071 ceph-mon[44707]: osdmap e75: 12 total, 12 up, 12 in 2024-02-02T15:24:10.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:09 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-02T15:24:10.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:10.115+0000 7f6387400700 1 Processor -- start 2024-02-02T15:24:10.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:10.115+0000 7f6387400700 1 -- start start 2024-02-02T15:24:10.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:10.116+0000 7f6387400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6380100dd0 0x7f6380198ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:10.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:10.116+0000 7f6387400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6380101fd0 0x7f6380199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:10.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:10.116+0000 7f6387400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6380102970 0x7f63801a0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:10.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:10.116+0000 7f6387400700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63801a0830 con 0x7f6380102970 2024-02-02T15:24:10.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:10.116+0000 7f6387400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63801a0970 con 0x7f6380100dd0 2024-02-02T15:24:10.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:10.116+0000 7f6387400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63801a0b30 con 0x7f6380101fd0 2024-02-02T15:24:10.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:10.116+0000 7f6384a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6380100dd0 0x7f6380198ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:10.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:10.116+0000 7f6384a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6380100dd0 0x7f6380198ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:35236/0 (socket says 172.21.15.1:35236) 2024-02-02T15:24:10.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:10.116+0000 7f6384a00700 1 -- 172.21.15.1:0/2026616896 learned_addr learned my addr 172.21.15.1:0/2026616896 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:24:10.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:10.116+0000 7f6385400700 1 --2- 172.21.15.1:0/2026616896 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6380102970 0x7f63801a0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:10.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:10.116+0000 7f637fe00700 1 --2- 172.21.15.1:0/2026616896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6380101fd0 0x7f6380199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:10.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:10.117+0000 7f6384a00700 1 -- 172.21.15.1:0/2026616896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6380101fd0 msgr2=0x7f6380199220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:10.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:10.117+0000 7f6384a00700 1 --2- 172.21.15.1:0/2026616896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6380101fd0 0x7f6380199220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:10.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:10.117+0000 7f6384a00700 1 -- 172.21.15.1:0/2026616896 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6380102970 msgr2=0x7f63801a0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:10.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:10.117+0000 7f6384a00700 1 --2- 172.21.15.1:0/2026616896 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6380102970 0x7f63801a0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:10.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:10.117+0000 7f6384a00700 1 -- 172.21.15.1:0/2026616896 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63680095f0 con 0x7f6380100dd0 2024-02-02T15:24:10.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:10.117+0000 7f6385400700 1 --2- 172.21.15.1:0/2026616896 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6380102970 0x7f63801a0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:24:10.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:10.117+0000 7f6384a00700 1 --2- 172.21.15.1:0/2026616896 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6380100dd0 0x7f6380198ce0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f637000d980 tx=0x7f637000dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:24:10.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:10.117+0000 7f637d600700 1 -- 172.21.15.1:0/2026616896 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6370004020 con 0x7f6380100dd0 2024-02-02T15:24:10.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:10.117+0000 7f6387400700 1 -- 172.21.15.1:0/2026616896 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63801a0e10 con 0x7f6380100dd0 2024-02-02T15:24:10.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:10.117+0000 7f637d600700 1 -- 172.21.15.1:0/2026616896 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f637000f690 con 0x7f6380100dd0 2024-02-02T15:24:10.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:10.117+0000 7f637d600700 1 -- 172.21.15.1:0/2026616896 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63700039e0 con 0x7f6380100dd0 2024-02-02T15:24:10.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:10.117+0000 7f6387400700 1 -- 172.21.15.1:0/2026616896 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63801a1230 con 0x7f6380100dd0 2024-02-02T15:24:10.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:10.118+0000 7f6387400700 1 -- 172.21.15.1:0/2026616896 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f638010c620 con 0x7f6380100dd0 2024-02-02T15:24:10.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:10.120+0000 7f637d600700 1 -- 172.21.15.1:0/2026616896 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 135069+0+0 (secure 0 0 0) 0x7f6370004180 con 0x7f6380100dd0 2024-02-02T15:24:10.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:10.121+0000 7f637d600700 1 --2- 172.21.15.1:0/2026616896 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f63600a0520 0x7f63600a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:10.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:10.122+0000 7f637d600700 1 -- 172.21.15.1:0/2026616896 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f63700cb220 con 0x7f6380100dd0 2024-02-02T15:24:10.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:10.122+0000 7f637fe00700 1 --2- 172.21.15.1:0/2026616896 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f63600a0520 0x7f63600a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:10.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:10.122+0000 7f637fe00700 1 --2- 172.21.15.1:0/2026616896 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f63600a0520 0x7f63600a29e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6368006010 tx=0x7f6368005bc0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:24:10.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:10.124+0000 7f637d600700 1 -- 172.21.15.1:0/2026616896 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f63700951c0 con 0x7f6380100dd0 2024-02-02T15:24:10.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:09 smithi142 ceph-mon[45728]: purged_snaps scrub starts 2024-02-02T15:24:10.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:09 smithi142 ceph-mon[45728]: purged_snaps scrub ok 2024-02-02T15:24:10.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:09 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-02T15:24:10.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:09 smithi142 ceph-mon[45728]: osd.11 [v2:172.21.15.142:6824/2290179934,v1:172.21.15.142:6825/2290179934] boot 2024-02-02T15:24:10.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:09 smithi142 ceph-mon[45728]: osdmap e75: 12 total, 12 up, 12 in 2024-02-02T15:24:10.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:09 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-02T15:24:10.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:09 smithi001 ceph-mon[35470]: purged_snaps scrub starts 2024-02-02T15:24:10.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:09 smithi001 ceph-mon[35470]: purged_snaps scrub ok 2024-02-02T15:24:10.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:09 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-02T15:24:10.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:09 smithi001 ceph-mon[35470]: osd.11 [v2:172.21.15.142:6824/2290179934,v1:172.21.15.142:6825/2290179934] boot 2024-02-02T15:24:10.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:09 smithi001 ceph-mon[35470]: osdmap e75: 12 total, 12 up, 12 in 2024-02-02T15:24:10.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:09 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-02T15:24:10.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:10.333+0000 7f6387400700 1 -- 172.21.15.1:0/2026616896 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f63801a18d0 con 0x7f6380100dd0 2024-02-02T15:24:10.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:10.333+0000 7f637d600700 1 -- 172.21.15.1:0/2026616896 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v76) v1 ==== 74+0+133 (secure 0 0 0) 0x7f63700987e0 con 0x7f6380100dd0 2024-02-02T15:24:10.332 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:24:10.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:10.336+0000 7f6387400700 1 -- 172.21.15.1:0/2026616896 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f63600a0520 msgr2=0x7f63600a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:10.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:10.336+0000 7f6387400700 1 --2- 172.21.15.1:0/2026616896 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f63600a0520 0x7f63600a29e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6368006010 tx=0x7f6368005bc0 comp rx=0 tx=0).stop 2024-02-02T15:24:10.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:10.336+0000 7f6387400700 1 -- 172.21.15.1:0/2026616896 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6380100dd0 msgr2=0x7f6380198ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:10.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:10.336+0000 7f6387400700 1 --2- 172.21.15.1:0/2026616896 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6380100dd0 0x7f6380198ce0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f637000d980 tx=0x7f637000dc90 comp rx=0 tx=0).stop 2024-02-02T15:24:10.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:10.336+0000 7f6387400700 1 -- 172.21.15.1:0/2026616896 shutdown_connections 2024-02-02T15:24:10.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:10.336+0000 7f6387400700 1 --2- 172.21.15.1:0/2026616896 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6380102970 0x7f63801a0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:10.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:10.336+0000 7f6387400700 1 --2- 172.21.15.1:0/2026616896 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6380100dd0 0x7f6380198ce0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:10.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:10.336+0000 7f6387400700 1 --2- 172.21.15.1:0/2026616896 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f63600a0520 0x7f63600a29e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:10.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:10.336+0000 7f6387400700 1 --2- 172.21.15.1:0/2026616896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6380101fd0 0x7f6380199220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:10.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:10.336+0000 7f6387400700 1 -- 172.21.15.1:0/2026616896 >> 172.21.15.1:0/2026616896 conn(0x7f63800fc330 msgr2=0x7f63800fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:24:10.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:10.336+0000 7f6387400700 1 -- 172.21.15.1:0/2026616896 shutdown_connections 2024-02-02T15:24:10.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:10.336+0000 7f6387400700 1 -- 172.21.15.1:0/2026616896 wait complete. 2024-02-02T15:24:10.732 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":76,"num_osds":12,"num_up_osds":12,"osd_up_since":1706887448,"num_in_osds":12,"osd_in_since":1706887432,"num_remapped_pgs":0} 2024-02-02T15:24:10.732 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph osd dump --format=json 2024-02-02T15:24:11.093 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.a/config 2024-02-02T15:24:11.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:10 smithi071 ceph-mon[44707]: osdmap e76: 12 total, 12 up, 12 in 2024-02-02T15:24:11.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:10 smithi071 ceph-mon[44707]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-02T15:24:11.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:10 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/2026616896' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-02T15:24:11.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:10 smithi142 ceph-mon[45728]: osdmap e76: 12 total, 12 up, 12 in 2024-02-02T15:24:11.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:10 smithi142 ceph-mon[45728]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-02T15:24:11.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:10 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/2026616896' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-02T15:24:11.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:10 smithi001 ceph-mon[35470]: osdmap e76: 12 total, 12 up, 12 in 2024-02-02T15:24:11.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:10 smithi001 ceph-mon[35470]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-02T15:24:11.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:10 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/2026616896' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-02T15:24:12.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:12.184+0000 7efe12600700 1 -- 172.21.15.1:0/2216083682 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe0c1060a0 msgr2=0x7efe0c10dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:12.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:12.184+0000 7efe12600700 1 --2- 172.21.15.1:0/2216083682 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe0c1060a0 0x7efe0c10dc70 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7efe0400b0a0 tx=0x7efe0400b3b0 comp rx=0 tx=0).stop 2024-02-02T15:24:12.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:12.184+0000 7efe12600700 1 -- 172.21.15.1:0/2216083682 shutdown_connections 2024-02-02T15:24:12.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:12.184+0000 7efe12600700 1 --2- 172.21.15.1:0/2216083682 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe0c1060a0 0x7efe0c10dc70 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:12.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:12.184+0000 7efe12600700 1 --2- 172.21.15.1:0/2216083682 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe0c100fc0 0x7efe0c1013a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:12.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:12.184+0000 7efe12600700 1 --2- 172.21.15.1:0/2216083682 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe0c101970 0x7efe0c1059c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:12.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:12.184+0000 7efe12600700 1 -- 172.21.15.1:0/2216083682 >> 172.21.15.1:0/2216083682 conn(0x7efe0c0fc830 msgr2=0x7efe0c0fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:24:12.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:12.184+0000 7efe12600700 1 -- 172.21.15.1:0/2216083682 shutdown_connections 2024-02-02T15:24:12.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:12.184+0000 7efe12600700 1 -- 172.21.15.1:0/2216083682 wait complete. 2024-02-02T15:24:12.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:12.186+0000 7efe12600700 1 Processor -- start 2024-02-02T15:24:12.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:12.186+0000 7efe12600700 1 -- start start 2024-02-02T15:24:12.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:12.186+0000 7efe12600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe0c100fc0 0x7efe0c073040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:12.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:12.187+0000 7efe12600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe0c101970 0x7efe0c073580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:12.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:12.187+0000 7efe12600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe0c1060a0 0x7efe0c073b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:12.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:12.187+0000 7efe12600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe0c1a5430 con 0x7efe0c1060a0 2024-02-02T15:24:12.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:12.187+0000 7efe12600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe0c1a55a0 con 0x7efe0c100fc0 2024-02-02T15:24:12.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:12.187+0000 7efe12600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe0c1a5760 con 0x7efe0c101970 2024-02-02T15:24:12.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:12.187+0000 7efe0b400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe0c100fc0 0x7efe0c073040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:12.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:12.187+0000 7efe0be00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe0c1060a0 0x7efe0c073b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:12.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:12.187+0000 7efe0b400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe0c100fc0 0x7efe0c073040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:35252/0 (socket says 172.21.15.1:35252) 2024-02-02T15:24:12.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:12.187+0000 7efe0b400700 1 -- 172.21.15.1:0/461034081 learned_addr learned my addr 172.21.15.1:0/461034081 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:24:12.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:12.187+0000 7efe03e00700 1 --2- 172.21.15.1:0/461034081 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe0c101970 0x7efe0c073580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:12.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:12.187+0000 7efe0be00700 1 -- 172.21.15.1:0/461034081 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe0c101970 msgr2=0x7efe0c073580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:12.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:12.187+0000 7efe0be00700 1 --2- 172.21.15.1:0/461034081 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe0c101970 0x7efe0c073580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:12.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:12.187+0000 7efe0be00700 1 -- 172.21.15.1:0/461034081 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe0c100fc0 msgr2=0x7efe0c073040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:12.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:12.187+0000 7efe0be00700 1 --2- 172.21.15.1:0/461034081 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe0c100fc0 0x7efe0c073040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:12.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:12.187+0000 7efe0be00700 1 -- 172.21.15.1:0/461034081 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe04009d00 con 0x7efe0c1060a0 2024-02-02T15:24:12.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:12.187+0000 7efe0b400700 1 --2- 172.21.15.1:0/461034081 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe0c100fc0 0x7efe0c073040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:24:12.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:12.187+0000 7efe0be00700 1 --2- 172.21.15.1:0/461034081 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe0c1060a0 0x7efe0c073b00 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7efe04009fd0 tx=0x7efe0400bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:24:12.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:12.188+0000 7efe08c00700 1 -- 172.21.15.1:0/461034081 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe0400e040 con 0x7efe0c1060a0 2024-02-02T15:24:12.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:12.188+0000 7efe12600700 1 -- 172.21.15.1:0/461034081 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe0c1a59e0 con 0x7efe0c1060a0 2024-02-02T15:24:12.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:12.188+0000 7efe08c00700 1 -- 172.21.15.1:0/461034081 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe040094c0 con 0x7efe0c1060a0 2024-02-02T15:24:12.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:12.188+0000 7efe08c00700 1 -- 172.21.15.1:0/461034081 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe04005040 con 0x7efe0c1060a0 2024-02-02T15:24:12.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:12.188+0000 7efe12600700 1 -- 172.21.15.1:0/461034081 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe0c1a5ea0 con 0x7efe0c1060a0 2024-02-02T15:24:12.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:12.189+0000 7efe12600700 1 -- 172.21.15.1:0/461034081 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe0c04ea40 con 0x7efe0c1060a0 2024-02-02T15:24:12.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:12.191+0000 7efe08c00700 1 -- 172.21.15.1:0/461034081 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) v1 ==== 135069+0+0 (secure 0 0 0) 0x7efe0401d470 con 0x7efe0c1060a0 2024-02-02T15:24:12.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:12.192+0000 7efe08c00700 1 --2- 172.21.15.1:0/461034081 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7efdec0a0520 0x7efdec0a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:12.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:12.192+0000 7efe08c00700 1 -- 172.21.15.1:0/461034081 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7efe040cd1a0 con 0x7efe0c1060a0 2024-02-02T15:24:12.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:12.193+0000 7efe0b400700 1 --2- 172.21.15.1:0/461034081 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7efdec0a0520 0x7efdec0a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:12.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:12.193+0000 7efe0b400700 1 --2- 172.21.15.1:0/461034081 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7efdec0a0520 0x7efdec0a29e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7efdfc005950 tx=0x7efdfc00b500 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:24:12.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:12.195+0000 7efe08c00700 1 -- 172.21.15.1:0/461034081 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7efe040971d0 con 0x7efe0c1060a0 2024-02-02T15:24:12.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:12.406+0000 7efe12600700 1 -- 172.21.15.1:0/461034081 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efe0c1a61c0 con 0x7efe0c1060a0 2024-02-02T15:24:12.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:12.407+0000 7efe08c00700 1 -- 172.21.15.1:0/461034081 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) v1 ==== 74+0+18303 (secure 0 0 0) 0x7efe0409a7f0 con 0x7efe0c1060a0 2024-02-02T15:24:12.406 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:24:12.406 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":76,"fsid":"371a409c-c1de-11ee-95b6-87774f69a715","created":"2024-02-02T15:18:26.746506+0000","modified":"2024-02-02T15:24:09.807430+0000","last_up_change":"2024-02-02T15:24:08.800951+0000","last_in_change":"2024-02-02T15:23:52.784280+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T15:21:22.471754+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"73e8a07f-3560-4c19-85ed-030d4d8f0f23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6803","nonce":2591906423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6805","nonce":2591906423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6809","nonce":2591906423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6807","nonce":2591906423}]},"public_addr":"172.21.15.1:6803/2591906423","cluster_addr":"172.21.15.1:6805/2591906423","heartbeat_back_addr":"172.21.15.1:6809/2591906423","heartbeat_front_addr":"172.21.15.1:6807/2591906423","state":["exists","up"]},{"osd":1,"uuid":"e53cffec-0ff9-4ed9-8dcf-ff47d04db2bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6801","nonce":747581719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6803","nonce":747581719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6807","nonce":747581719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6805","nonce":747581719}]},"public_addr":"172.21.15.71:6801/747581719","cluster_addr":"172.21.15.71:6803/747581719","heartbeat_back_addr":"172.21.15.71:6807/747581719","heartbeat_front_addr":"172.21.15.71:6805/747581719","state":["exists","up"]},{"osd":2,"uuid":"856c2347-294c-482b-8595-ad77f4792db4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6801","nonce":377511414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6803","nonce":377511414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6807","nonce":377511414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6805","nonce":377511414}]},"public_addr":"172.21.15.142:6801/377511414","cluster_addr":"172.21.15.142:6803/377511414","heartbeat_back_addr":"172.21.15.142:6807/377511414","heartbeat_front_addr":"172.21.15.142:6805/377511414","state":["exists","up"]},{"osd":3,"uuid":"4b946ad2-2da9-4e0b-9d61-d039d8669a42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6811","nonce":1086124477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6813","nonce":1086124477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6817","nonce":1086124477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6815","nonce":1086124477}]},"public_addr":"172.21.15.1:6811/1086124477","cluster_addr":"172.21.15.1:6813/1086124477","heartbeat_back_addr":"172.21.15.1:6817/1086124477","heartbeat_front_addr":"172.21.15.1:6815/1086124477","state":["exists","up"]},{"osd":4,"uuid":"43e36c43-47a0-4634-a45b-1da14a0e19d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6809","nonce":3095380305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6811","nonce":3095380305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6815","nonce":3095380305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6813","nonce":3095380305}]},"public_addr":"172.21.15.71:6809/3095380305","cluster_addr":"172.21.15.71:6811/3095380305","heartbeat_back_addr":"172.21.15.71:6815/3095380305","heartbeat_front_addr":"172.21.15.71:6813/3095380305","state":["exists","up"]},{"osd":5,"uuid":"eb48160e-ea7c-42f7-9477-be110bc8a34e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6809","nonce":4195149262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6811","nonce":4195149262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6815","nonce":4195149262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6813","nonce":4195149262}]},"public_addr":"172.21.15.142:6809/4195149262","cluster_addr":"172.21.15.142:6811/4195149262","heartbeat_back_addr":"172.21.15.142:6815/4195149262","heartbeat_front_addr":"172.21.15.142:6813/4195149262","state":["exists","up"]},{"osd":6,"uuid":"0ec4501a-5853-4ca6-b325-231f5b72335e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6819","nonce":1560153428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6821","nonce":1560153428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6825","nonce":1560153428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6823","nonce":1560153428}]},"public_addr":"172.21.15.1:6819/1560153428","cluster_addr":"172.21.15.1:6821/1560153428","heartbeat_back_addr":"172.21.15.1:6825/1560153428","heartbeat_front_addr":"172.21.15.1:6823/1560153428","state":["exists","up"]},{"osd":7,"uuid":"029debe8-9cdf-44f6-9aea-cabd1d185f93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6817","nonce":4080955538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6819","nonce":4080955538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6823","nonce":4080955538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6821","nonce":4080955538}]},"public_addr":"172.21.15.71:6817/4080955538","cluster_addr":"172.21.15.71:6819/4080955538","heartbeat_back_addr":"172.21.15.71:6823/4080955538","heartbeat_front_addr":"172.21.15.71:6821/4080955538","state":["exists","up"]},{"osd":8,"uuid":"7c90896c-9955-4c1b-aaca-7531dd5457f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6817","nonce":3942038209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6819","nonce":3942038209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6823","nonce":3942038209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6821","nonce":3942038209}]},"public_addr":"172.21.15.142:6817/3942038209","cluster_addr":"172.21.15.142:6819/3942038209","heartbeat_back_addr":"172.21.15.142:6823/3942038209","heartbeat_front_addr":"172.21.15.142:6821/3942038209","state":["exists","up"]},{"osd":9,"uuid":"7e165e68-31a8-4598-a577-de3e90ae5c5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6827","nonce":2716544850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6829","nonce":2716544850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6833","nonce":2716544850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6831","nonce":2716544850}]},"public_addr":"172.21.15.1:6827/2716544850","cluster_addr":"172.21.15.1:6829/2716544850","heartbeat_back_addr":"172.21.15.1:6833/2716544850","heartbeat_front_addr":"172.21.15.1:6831/2716544850","state":["exists","up"]},{"osd":10,"uuid":"48cd5fdb-0887-444a-aab8-a1de24282bb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6825","nonce":84691841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6827","nonce":84691841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6831","nonce":84691841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6829","nonce":84691841}]},"public_addr":"172.21.15.71:6825/84691841","cluster_addr":"172.21.15.71:6827/84691841","heartbeat_back_addr":"172.21.15.71:6831/84691841","heartbeat_front_addr":"172.21.15.71:6829/84691841","state":["exists","up"]},{"osd":11,"uuid":"f011dd27-11b0-4166-adfe-76756533163a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6825","nonce":2290179934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6827","nonce":2290179934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6831","nonce":2290179934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6829","nonce":2290179934}]},"public_addr":"172.21.15.142:6825/2290179934","cluster_addr":"172.21.15.142:6827/2290179934","heartbeat_back_addr":"172.21.15.142:6831/2290179934","heartbeat_front_addr":"172.21.15.142:6829/2290179934","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:20:44.060692+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:01.903817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:19.368258+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:40.028037+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:57.513361+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:22:14.805944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:22:35.188439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:22:53.014412+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:23:10.507420+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:23:31.849765+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:23:50.070612+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:24:07.388625+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/4092532060":"2024-02-03T15:19:25.836171+0000","172.21.15.1:6801/395498296":"2024-02-03T15:19:25.836171+0000","172.21.15.1:6800/395498296":"2024-02-03T15:19:25.836171+0000","172.21.15.1:0/1575373489":"2024-02-03T15:19:25.836171+0000","172.21.15.1:0/865983374":"2024-02-03T15:19:01.431596+0000","172.21.15.1:0/3165960227":"2024-02-03T15:19:01.431596+0000","172.21.15.1:0/2350830293":"2024-02-03T15:19:01.431596+0000","172.21.15.1:0/2959690250":"2024-02-03T15:19:25.836171+0000","172.21.15.1:6800/1162530732":"2024-02-03T15:19:01.431596+0000","172.21.15.1:6801/1162530732":"2024-02-03T15:19:01.431596+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T15:24:12.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:12.409+0000 7efe12600700 1 -- 172.21.15.1:0/461034081 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7efdec0a0520 msgr2=0x7efdec0a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:12.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:12.409+0000 7efe12600700 1 --2- 172.21.15.1:0/461034081 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7efdec0a0520 0x7efdec0a29e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7efdfc005950 tx=0x7efdfc00b500 comp rx=0 tx=0).stop 2024-02-02T15:24:12.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:12.409+0000 7efe12600700 1 -- 172.21.15.1:0/461034081 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe0c1060a0 msgr2=0x7efe0c073b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:12.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:12.409+0000 7efe12600700 1 --2- 172.21.15.1:0/461034081 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe0c1060a0 0x7efe0c073b00 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7efe04009fd0 tx=0x7efe0400bce0 comp rx=0 tx=0).stop 2024-02-02T15:24:12.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:12.409+0000 7efe12600700 1 -- 172.21.15.1:0/461034081 shutdown_connections 2024-02-02T15:24:12.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:12.409+0000 7efe12600700 1 --2- 172.21.15.1:0/461034081 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe0c1060a0 0x7efe0c073b00 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:12.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:12.409+0000 7efe12600700 1 --2- 172.21.15.1:0/461034081 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe0c100fc0 0x7efe0c073040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:12.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:12.409+0000 7efe12600700 1 --2- 172.21.15.1:0/461034081 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7efdec0a0520 0x7efdec0a29e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:12.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:12.409+0000 7efe12600700 1 --2- 172.21.15.1:0/461034081 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe0c101970 0x7efe0c073580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:12.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:12.409+0000 7efe12600700 1 -- 172.21.15.1:0/461034081 >> 172.21.15.1:0/461034081 conn(0x7efe0c0fc830 msgr2=0x7efe0c0fd2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:24:12.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:12.409+0000 7efe12600700 1 -- 172.21.15.1:0/461034081 shutdown_connections 2024-02-02T15:24:12.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:12.409+0000 7efe12600700 1 -- 172.21.15.1:0/461034081 wait complete. 2024-02-02T15:24:12.840 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-02T15:21:22.471754+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-02T15:24:12.840 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-02-02T15:24:13.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:12 smithi001 ceph-mon[35470]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-02T15:24:13.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:12 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/461034081' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-02T15:24:13.204 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.a/config 2024-02-02T15:24:13.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:12 smithi071 ceph-mon[44707]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-02T15:24:13.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:12 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/461034081' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-02T15:24:13.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:12 smithi142 ceph-mon[45728]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-02T15:24:13.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:12 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/461034081' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-02T15:24:14.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:14.356+0000 7f83a6a00700 1 -- 172.21.15.1:0/2539982007 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83a0106ed0 msgr2=0x7f83a01072b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:14.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:14.356+0000 7f83a6a00700 1 --2- 172.21.15.1:0/2539982007 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83a0106ed0 0x7f83a01072b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f8390009990 tx=0x7f8390009ca0 comp rx=0 tx=0).stop 2024-02-02T15:24:14.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:14.356+0000 7f83a6a00700 1 -- 172.21.15.1:0/2539982007 shutdown_connections 2024-02-02T15:24:14.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:14.356+0000 7f83a6a00700 1 --2- 172.21.15.1:0/2539982007 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83a0069220 0x7f83a010db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:14.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:14.356+0000 7f83a6a00700 1 --2- 172.21.15.1:0/2539982007 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83a0106ed0 0x7f83a01072b0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:14.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:14.356+0000 7f83a6a00700 1 --2- 172.21.15.1:0/2539982007 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83a0068860 0x7f83a0068ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:14.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:14.356+0000 7f83a6a00700 1 -- 172.21.15.1:0/2539982007 >> 172.21.15.1:0/2539982007 conn(0x7f83a0076d40 msgr2=0x7f83a0077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:24:14.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:14.356+0000 7f83a6a00700 1 -- 172.21.15.1:0/2539982007 shutdown_connections 2024-02-02T15:24:14.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:14.356+0000 7f83a6a00700 1 -- 172.21.15.1:0/2539982007 wait complete. 2024-02-02T15:24:14.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:14.357+0000 7f83a6a00700 1 Processor -- start 2024-02-02T15:24:14.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:14.357+0000 7f83a6a00700 1 -- start start 2024-02-02T15:24:14.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:14.357+0000 7f83a6a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83a0068860 0x7f83a00ffcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:14.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:14.357+0000 7f83a6a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83a0069220 0x7f83a0100230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:14.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:14.357+0000 7f83a6a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83a0106ed0 0x7f83a01007b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:14.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:14.357+0000 7f83a6a00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83a01a51c0 con 0x7f83a0069220 2024-02-02T15:24:14.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:14.357+0000 7f83a6a00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83a01a5330 con 0x7f83a0068860 2024-02-02T15:24:14.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:14.358+0000 7f83a6a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83a01a54f0 con 0x7f83a0106ed0 2024-02-02T15:24:14.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:14.358+0000 7f8397e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83a0069220 0x7f83a0100230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:14.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:14.358+0000 7f839fe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83a0068860 0x7f83a00ffcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:14.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:14.358+0000 7f8397e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83a0069220 0x7f83a0100230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37352/0 (socket says 172.21.15.1:37352) 2024-02-02T15:24:14.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:14.358+0000 7f8397e00700 1 -- 172.21.15.1:0/3225533713 learned_addr learned my addr 172.21.15.1:0/3225533713 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:24:14.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:14.358+0000 7f83a4a00700 1 --2- 172.21.15.1:0/3225533713 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83a0106ed0 0x7f83a01007b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:14.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:14.358+0000 7f8397e00700 1 -- 172.21.15.1:0/3225533713 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83a0106ed0 msgr2=0x7f83a01007b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:14.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:14.358+0000 7f8397e00700 1 --2- 172.21.15.1:0/3225533713 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83a0106ed0 0x7f83a01007b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:14.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:14.358+0000 7f8397e00700 1 -- 172.21.15.1:0/3225533713 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83a0068860 msgr2=0x7f83a00ffcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:14.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:14.358+0000 7f8397e00700 1 --2- 172.21.15.1:0/3225533713 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83a0068860 0x7f83a00ffcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:14.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:14.358+0000 7f8397e00700 1 -- 172.21.15.1:0/3225533713 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83900095f0 con 0x7f83a0069220 2024-02-02T15:24:14.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:14.358+0000 7f83a4a00700 1 --2- 172.21.15.1:0/3225533713 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83a0106ed0 0x7f83a01007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:24:14.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:14.358+0000 7f839fe00700 1 --2- 172.21.15.1:0/3225533713 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83a0068860 0x7f83a00ffcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:24:14.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:14.358+0000 7f8397e00700 1 --2- 172.21.15.1:0/3225533713 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83a0069220 0x7f83a0100230 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f838800e9a0 tx=0x7f838800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:24:14.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:14.359+0000 7f839d600700 1 -- 172.21.15.1:0/3225533713 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8388009790 con 0x7f83a0069220 2024-02-02T15:24:14.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:14.359+0000 7f839d600700 1 -- 172.21.15.1:0/3225533713 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f838800cbd0 con 0x7f83a0069220 2024-02-02T15:24:14.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:14.359+0000 7f83a6a00700 1 -- 172.21.15.1:0/3225533713 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83a01a57d0 con 0x7f83a0069220 2024-02-02T15:24:14.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:14.359+0000 7f839d600700 1 -- 172.21.15.1:0/3225533713 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8388018680 con 0x7f83a0069220 2024-02-02T15:24:14.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:14.359+0000 7f83a6a00700 1 -- 172.21.15.1:0/3225533713 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83a01a5cd0 con 0x7f83a0069220 2024-02-02T15:24:14.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:14.360+0000 7f83a6a00700 1 -- 172.21.15.1:0/3225533713 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83a004ea40 con 0x7f83a0069220 2024-02-02T15:24:14.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:14.362+0000 7f839d600700 1 -- 172.21.15.1:0/3225533713 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) v1 ==== 135069+0+0 (secure 0 0 0) 0x7f838800cd40 con 0x7f83a0069220 2024-02-02T15:24:14.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:14.363+0000 7f839d600700 1 --2- 172.21.15.1:0/3225533713 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f83800a0520 0x7f83800a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:14.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:14.363+0000 7f839d600700 1 -- 172.21.15.1:0/3225533713 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f8388014070 con 0x7f83a0069220 2024-02-02T15:24:14.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:14.364+0000 7f839fe00700 1 --2- 172.21.15.1:0/3225533713 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f83800a0520 0x7f83800a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:14.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:14.364+0000 7f839fe00700 1 --2- 172.21.15.1:0/3225533713 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f83800a0520 0x7f83800a29e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f839000b2e0 tx=0x7f8390005960 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:24:14.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:14.367+0000 7f839d600700 1 -- 172.21.15.1:0/3225533713 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f8388052110 con 0x7f83a0069220 2024-02-02T15:24:14.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:14.577+0000 7f83a6a00700 1 -- 172.21.15.1:0/3225533713 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f83a01a5ff0 con 0x7f83a0069220 2024-02-02T15:24:14.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:14.578+0000 7f839d600700 1 -- 172.21.15.1:0/3225533713 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v76) v1 ==== 93+0+10 (secure 0 0 0) 0x7f83880957e0 con 0x7f83a0069220 2024-02-02T15:24:14.577 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-02-02T15:24:14.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:14.580+0000 7f83a6a00700 1 -- 172.21.15.1:0/3225533713 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f83800a0520 msgr2=0x7f83800a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:14.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:14.580+0000 7f83a6a00700 1 --2- 172.21.15.1:0/3225533713 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f83800a0520 0x7f83800a29e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f839000b2e0 tx=0x7f8390005960 comp rx=0 tx=0).stop 2024-02-02T15:24:14.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:14.580+0000 7f83a6a00700 1 -- 172.21.15.1:0/3225533713 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83a0069220 msgr2=0x7f83a0100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:14.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:14.580+0000 7f83a6a00700 1 --2- 172.21.15.1:0/3225533713 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83a0069220 0x7f83a0100230 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f838800e9a0 tx=0x7f838800ecb0 comp rx=0 tx=0).stop 2024-02-02T15:24:14.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:14.580+0000 7f83a6a00700 1 -- 172.21.15.1:0/3225533713 shutdown_connections 2024-02-02T15:24:14.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:14.580+0000 7f83a6a00700 1 --2- 172.21.15.1:0/3225533713 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f83800a0520 0x7f83800a29e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:14.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:14.580+0000 7f83a6a00700 1 --2- 172.21.15.1:0/3225533713 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83a0106ed0 0x7f83a01007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:14.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:14.580+0000 7f83a6a00700 1 --2- 172.21.15.1:0/3225533713 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83a0068860 0x7f83a00ffcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:14.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:14.580+0000 7f83a6a00700 1 --2- 172.21.15.1:0/3225533713 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83a0069220 0x7f83a0100230 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:14.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:14.580+0000 7f83a6a00700 1 -- 172.21.15.1:0/3225533713 >> 172.21.15.1:0/3225533713 conn(0x7f83a0076d40 msgr2=0x7f83a0104ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:24:14.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:14.580+0000 7f83a6a00700 1 -- 172.21.15.1:0/3225533713 shutdown_connections 2024-02-02T15:24:14.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:14.580+0000 7f83a6a00700 1 -- 172.21.15.1:0/3225533713 wait complete. 2024-02-02T15:24:14.949 INFO:tasks.cephadm:Adding mds.a on smithi001 2024-02-02T15:24:14.949 INFO:tasks.cephadm:Adding mds.d on smithi001 2024-02-02T15:24:14.949 INFO:tasks.cephadm:Adding mds.g on smithi001 2024-02-02T15:24:14.949 INFO:tasks.cephadm:Adding mds.j on smithi001 2024-02-02T15:24:14.949 INFO:tasks.cephadm:Adding mds.b on smithi071 2024-02-02T15:24:14.949 INFO:tasks.cephadm:Adding mds.e on smithi071 2024-02-02T15:24:14.949 INFO:tasks.cephadm:Adding mds.h on smithi071 2024-02-02T15:24:14.949 INFO:tasks.cephadm:Adding mds.k on smithi071 2024-02-02T15:24:14.949 INFO:tasks.cephadm:Adding mds.c on smithi142 2024-02-02T15:24:14.949 INFO:tasks.cephadm:Adding mds.f on smithi142 2024-02-02T15:24:14.949 INFO:tasks.cephadm:Adding mds.i on smithi142 2024-02-02T15:24:14.950 INFO:tasks.cephadm:Adding mds.l on smithi142 2024-02-02T15:24:14.950 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph orch apply mds all '12;smithi001=a;smithi001=d;smithi001=g;smithi001=j;smithi071=b;smithi071=e;smithi071=h;smithi071=k;smithi142=c;smithi142=f;smithi142=i;smithi142=l' 2024-02-02T15:24:15.186 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.c/config 2024-02-02T15:24:15.189 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:14 smithi001 ceph-mon[35470]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-02T15:24:15.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:14 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/3225533713' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-02T15:24:15.289 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:14 smithi142 ceph-mon[45728]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-02T15:24:15.289 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:14 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/3225533713' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-02T15:24:15.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:14 smithi071 ceph-mon[44707]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-02T15:24:15.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:14 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/3225533713' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-02T15:24:16.151 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:24:16.150+0000 7f210a95d700 1 -- 172.21.15.142:0/3988785776 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2104101860 msgr2=0x7f2104103c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:16.152 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:24:16.150+0000 7f210a95d700 1 --2- 172.21.15.142:0/3988785776 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2104101860 0x7f2104103c50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f20f8009990 tx=0x7f20f8009ca0 comp rx=0 tx=0).stop 2024-02-02T15:24:16.152 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:24:16.150+0000 7f210a95d700 1 -- 172.21.15.142:0/3988785776 shutdown_connections 2024-02-02T15:24:16.152 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:24:16.150+0000 7f210a95d700 1 --2- 172.21.15.142:0/3988785776 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2104106ac0 0x7f2104108f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:16.152 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:24:16.150+0000 7f210a95d700 1 --2- 172.21.15.142:0/3988785776 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2104101860 0x7f2104103c50 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:16.152 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:24:16.150+0000 7f210a95d700 1 --2- 172.21.15.142:0/3988785776 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2104104190 0x7f2104106580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:16.152 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:24:16.150+0000 7f210a95d700 1 -- 172.21.15.142:0/3988785776 >> 172.21.15.142:0/3988785776 conn(0x7f21040fb250 msgr2=0x7f21040fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:24:16.152 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:24:16.150+0000 7f210a95d700 1 -- 172.21.15.142:0/3988785776 shutdown_connections 2024-02-02T15:24:16.152 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:24:16.150+0000 7f210a95d700 1 -- 172.21.15.142:0/3988785776 wait complete. 2024-02-02T15:24:16.152 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:24:16.151+0000 7f210a95d700 1 Processor -- start 2024-02-02T15:24:16.152 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:24:16.151+0000 7f210a95d700 1 -- start start 2024-02-02T15:24:16.152 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:24:16.151+0000 7f210a95d700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2104101860 0x7f2104198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:16.152 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:24:16.151+0000 7f210a95d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2104104190 0x7f2104199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:16.153 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:24:16.151+0000 7f210a95d700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2104106ac0 0x7f21041a02c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:16.153 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:24:16.151+0000 7f210a95d700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21041a0890 con 0x7f2104101860 2024-02-02T15:24:16.153 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:24:16.151+0000 7f210a95d700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21041a09d0 con 0x7f2104104190 2024-02-02T15:24:16.153 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:24:16.151+0000 7f210a95d700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21041a0b90 con 0x7f2104106ac0 2024-02-02T15:24:16.153 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:24:16.151+0000 7f2108efa700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2104106ac0 0x7f21041a02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:16.153 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:24:16.151+0000 7f2108efa700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2104106ac0 0x7f21041a02c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.142:35264/0 (socket says 172.21.15.142:35264) 2024-02-02T15:24:16.153 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:24:16.151+0000 7f2108efa700 1 -- 172.21.15.142:0/851698593 learned_addr learned my addr 172.21.15.142:0/851698593 (peer_addr_for_me v2:172.21.15.142:0/0) 2024-02-02T15:24:16.153 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:24:16.151+0000 7f2103fff700 1 --2- 172.21.15.142:0/851698593 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2104101860 0x7f2104198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:16.153 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:24:16.151+0000 7f21037fe700 1 --2- 172.21.15.142:0/851698593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2104104190 0x7f2104199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:16.153 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:24:16.151+0000 7f2108efa700 1 -- 172.21.15.142:0/851698593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2104104190 msgr2=0x7f2104199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:16.153 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:24:16.151+0000 7f2108efa700 1 --2- 172.21.15.142:0/851698593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2104104190 0x7f2104199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:16.153 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:24:16.151+0000 7f2108efa700 1 -- 172.21.15.142:0/851698593 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2104101860 msgr2=0x7f2104198d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:16.154 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:24:16.151+0000 7f2108efa700 1 --2- 172.21.15.142:0/851698593 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2104101860 0x7f2104198d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:16.154 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:24:16.151+0000 7f2108efa700 1 -- 172.21.15.142:0/851698593 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20f80095f0 con 0x7f2104106ac0 2024-02-02T15:24:16.154 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:24:16.151+0000 7f21037fe700 1 --2- 172.21.15.142:0/851698593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2104104190 0x7f2104199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:24:16.154 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:24:16.151+0000 7f2103fff700 1 --2- 172.21.15.142:0/851698593 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2104101860 0x7f2104198d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:24:16.154 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:24:16.151+0000 7f2108efa700 1 --2- 172.21.15.142:0/851698593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2104106ac0 0x7f21041a02c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f20fc00b9e0 tx=0x7f20fc00bda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:24:16.154 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:24:16.152+0000 7f21017fa700 1 -- 172.21.15.142:0/851698593 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20fc00c9b0 con 0x7f2104106ac0 2024-02-02T15:24:16.154 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:24:16.152+0000 7f21017fa700 1 -- 172.21.15.142:0/851698593 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f20fc014440 con 0x7f2104106ac0 2024-02-02T15:24:16.154 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:24:16.152+0000 7f210a95d700 1 -- 172.21.15.142:0/851698593 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21041a0e70 con 0x7f2104106ac0 2024-02-02T15:24:16.154 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:24:16.152+0000 7f21017fa700 1 -- 172.21.15.142:0/851698593 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20fc0125a0 con 0x7f2104106ac0 2024-02-02T15:24:16.154 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:24:16.152+0000 7f210a95d700 1 -- 172.21.15.142:0/851698593 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21041a12b0 con 0x7f2104106ac0 2024-02-02T15:24:16.155 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:24:16.153+0000 7f210a95d700 1 -- 172.21.15.142:0/851698593 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2104108680 con 0x7f2104106ac0 2024-02-02T15:24:16.156 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:24:16.154+0000 7f21017fa700 1 -- 172.21.15.142:0/851698593 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 16) v1 ==== 135069+0+0 (secure 0 0 0) 0x7f20fc0145b0 con 0x7f2104106ac0 2024-02-02T15:24:16.156 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:24:16.154+0000 7f21017fa700 1 --2- 172.21.15.142:0/851698593 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f20ec0a0520 0x7f20ec0a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:16.156 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:24:16.155+0000 7f21017fa700 1 -- 172.21.15.142:0/851698593 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f20fc006e40 con 0x7f2104106ac0 2024-02-02T15:24:16.156 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:24:16.155+0000 7f2103fff700 1 --2- 172.21.15.142:0/851698593 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f20ec0a0520 0x7f20ec0a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:16.156 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:24:16.155+0000 7f2103fff700 1 --2- 172.21.15.142:0/851698593 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f20ec0a0520 0x7f20ec0a29e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f20f800b5c0 tx=0x7f20f8005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:24:16.158 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:24:16.159+0000 7f21017fa700 1 -- 172.21.15.142:0/851698593 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f20fc094bc0 con 0x7f2104106ac0 2024-02-02T15:24:16.353 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:24:16.354+0000 7f210a95d700 1 -- 172.21.15.142:0/851698593 --> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi001=a;smithi001=d;smithi001=g;smithi001=j;smithi071=b;smithi071=e;smithi071=h;smithi071=k;smithi142=c;smithi142=f;smithi142=i;smithi142=l", "target": ["mon-mgr", ""]}) v1 -- 0x7f2104061180 con 0x7f20ec0a0520 2024-02-02T15:24:16.362 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:24:16.363+0000 7f21017fa700 1 -- 172.21.15.142:0/851698593 <== mgr.14154 v2:172.21.15.1:6800/3768989981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f2104061180 con 0x7f20ec0a0520 2024-02-02T15:24:16.362 INFO:teuthology.orchestra.run.smithi142.stdout:Scheduled mds.all update... 2024-02-02T15:24:16.364 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:24:16.365+0000 7f210a95d700 1 -- 172.21.15.142:0/851698593 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f20ec0a0520 msgr2=0x7f20ec0a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:16.364 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:24:16.365+0000 7f210a95d700 1 --2- 172.21.15.142:0/851698593 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f20ec0a0520 0x7f20ec0a29e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f20f800b5c0 tx=0x7f20f8005fb0 comp rx=0 tx=0).stop 2024-02-02T15:24:16.365 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:24:16.365+0000 7f210a95d700 1 -- 172.21.15.142:0/851698593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2104106ac0 msgr2=0x7f21041a02c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:16.365 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:24:16.365+0000 7f210a95d700 1 --2- 172.21.15.142:0/851698593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2104106ac0 0x7f21041a02c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f20fc00b9e0 tx=0x7f20fc00bda0 comp rx=0 tx=0).stop 2024-02-02T15:24:16.365 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:24:16.365+0000 7f210a95d700 1 -- 172.21.15.142:0/851698593 shutdown_connections 2024-02-02T15:24:16.365 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:24:16.365+0000 7f210a95d700 1 --2- 172.21.15.142:0/851698593 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f20ec0a0520 0x7f20ec0a29e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:16.365 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:24:16.365+0000 7f210a95d700 1 --2- 172.21.15.142:0/851698593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2104106ac0 0x7f21041a02c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:16.365 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:24:16.365+0000 7f210a95d700 1 --2- 172.21.15.142:0/851698593 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2104101860 0x7f2104198d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:16.365 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:24:16.365+0000 7f210a95d700 1 --2- 172.21.15.142:0/851698593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2104104190 0x7f2104199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:16.365 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:24:16.365+0000 7f210a95d700 1 -- 172.21.15.142:0/851698593 >> 172.21.15.142:0/851698593 conn(0x7f21040fb250 msgr2=0x7f21040ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:24:16.365 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:24:16.365+0000 7f210a95d700 1 -- 172.21.15.142:0/851698593 shutdown_connections 2024-02-02T15:24:16.365 INFO:teuthology.orchestra.run.smithi142.stderr:2024-02-02T15:24:16.365+0000 7f210a95d700 1 -- 172.21.15.142:0/851698593 wait complete. 2024-02-02T15:24:16.759 DEBUG:teuthology.orchestra.run.smithi001:mds.a> sudo journalctl -f -n 0 -u ceph-371a409c-c1de-11ee-95b6-87774f69a715@mds.a.service 2024-02-02T15:24:16.763 DEBUG:teuthology.orchestra.run.smithi001:mds.d> sudo journalctl -f -n 0 -u ceph-371a409c-c1de-11ee-95b6-87774f69a715@mds.d.service 2024-02-02T15:24:16.809 DEBUG:teuthology.orchestra.run.smithi001:mds.g> sudo journalctl -f -n 0 -u ceph-371a409c-c1de-11ee-95b6-87774f69a715@mds.g.service 2024-02-02T15:24:16.835 INFO:journalctl@ceph.mds.a.smithi001.stdout:-- Logs begin at Fri 2024-02-02 15:05:03 UTC. -- 2024-02-02T15:24:16.841 DEBUG:teuthology.orchestra.run.smithi001:mds.j> sudo journalctl -f -n 0 -u ceph-371a409c-c1de-11ee-95b6-87774f69a715@mds.j.service 2024-02-02T15:24:16.878 INFO:journalctl@ceph.mds.d.smithi001.stdout:-- Logs begin at Fri 2024-02-02 15:05:03 UTC. -- 2024-02-02T15:24:16.880 DEBUG:teuthology.orchestra.run.smithi071:mds.b> sudo journalctl -f -n 0 -u ceph-371a409c-c1de-11ee-95b6-87774f69a715@mds.b.service 2024-02-02T15:24:16.883 DEBUG:teuthology.orchestra.run.smithi071:mds.e> sudo journalctl -f -n 0 -u ceph-371a409c-c1de-11ee-95b6-87774f69a715@mds.e.service 2024-02-02T15:24:16.908 INFO:journalctl@ceph.mds.g.smithi001.stdout:-- Logs begin at Fri 2024-02-02 15:05:03 UTC. -- 2024-02-02T15:24:16.921 INFO:journalctl@ceph.mds.b.smithi071.stdout:-- Logs begin at Fri 2024-02-02 14:58:45 UTC. -- 2024-02-02T15:24:16.923 DEBUG:teuthology.orchestra.run.smithi071:mds.h> sudo journalctl -f -n 0 -u ceph-371a409c-c1de-11ee-95b6-87774f69a715@mds.h.service 2024-02-02T15:24:16.946 INFO:journalctl@ceph.mds.j.smithi001.stdout:-- Logs begin at Fri 2024-02-02 15:05:03 UTC. -- 2024-02-02T15:24:16.952 INFO:journalctl@ceph.mds.e.smithi071.stdout:-- Logs begin at Fri 2024-02-02 14:58:45 UTC. -- 2024-02-02T15:24:16.953 DEBUG:teuthology.orchestra.run.smithi071:mds.k> sudo journalctl -f -n 0 -u ceph-371a409c-c1de-11ee-95b6-87774f69a715@mds.k.service 2024-02-02T15:24:16.981 INFO:journalctl@ceph.mds.h.smithi071.stdout:-- Logs begin at Fri 2024-02-02 14:58:45 UTC. -- 2024-02-02T15:24:16.982 DEBUG:teuthology.orchestra.run.smithi142:mds.c> sudo journalctl -f -n 0 -u ceph-371a409c-c1de-11ee-95b6-87774f69a715@mds.c.service 2024-02-02T15:24:16.985 DEBUG:teuthology.orchestra.run.smithi142:mds.f> sudo journalctl -f -n 0 -u ceph-371a409c-c1de-11ee-95b6-87774f69a715@mds.f.service 2024-02-02T15:24:17.009 INFO:journalctl@ceph.mds.k.smithi071.stdout:-- Logs begin at Fri 2024-02-02 14:58:45 UTC. -- 2024-02-02T15:24:17.010 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:16 smithi142 ceph-mon[45728]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-02T15:24:17.010 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:16 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:17.010 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:16 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:24:17.011 DEBUG:teuthology.orchestra.run.smithi142:mds.i> sudo journalctl -f -n 0 -u ceph-371a409c-c1de-11ee-95b6-87774f69a715@mds.i.service 2024-02-02T15:24:17.016 INFO:journalctl@ceph.mds.c.smithi142.stdout:-- Logs begin at Fri 2024-02-02 14:58:43 UTC. -- 2024-02-02T15:24:17.017 DEBUG:teuthology.orchestra.run.smithi142:mds.l> sudo journalctl -f -n 0 -u ceph-371a409c-c1de-11ee-95b6-87774f69a715@mds.l.service 2024-02-02T15:24:17.040 INFO:journalctl@ceph.mds.f.smithi142.stdout:-- Logs begin at Fri 2024-02-02 14:58:43 UTC. -- 2024-02-02T15:24:17.040 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-02-02T15:24:17.041 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T15:24:17.043 INFO:journalctl@ceph.mds.i.smithi142.stdout:-- Logs begin at Fri 2024-02-02 14:58:43 UTC. -- 2024-02-02T15:24:17.066 INFO:journalctl@ceph.mds.l.smithi142.stdout:-- Logs begin at Fri 2024-02-02 14:58:43 UTC. -- 2024-02-02T15:24:17.101 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:16 smithi001 ceph-mon[35470]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-02T15:24:17.102 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:16 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:17.102 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:16 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:24:17.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:16 smithi071 ceph-mon[44707]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-02T15:24:17.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:16 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:17.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:16 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:24:17.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.260+0000 7fa47a800700 1 -- 172.21.15.1:0/2017916423 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa474106ac0 msgr2=0x7fa474108eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:17.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.260+0000 7fa47a800700 1 --2- 172.21.15.1:0/2017916423 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa474106ac0 0x7fa474108eb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa46c009f90 tx=0x7fa46c00b330 comp rx=0 tx=0).stop 2024-02-02T15:24:17.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.261+0000 7fa47a800700 1 -- 172.21.15.1:0/2017916423 shutdown_connections 2024-02-02T15:24:17.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.261+0000 7fa47a800700 1 --2- 172.21.15.1:0/2017916423 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa474106ac0 0x7fa474108eb0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:17.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.261+0000 7fa47a800700 1 --2- 172.21.15.1:0/2017916423 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa474101860 0x7fa474103c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:17.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.261+0000 7fa47a800700 1 --2- 172.21.15.1:0/2017916423 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa474104190 0x7fa474106580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:17.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.261+0000 7fa47a800700 1 -- 172.21.15.1:0/2017916423 >> 172.21.15.1:0/2017916423 conn(0x7fa4740fb170 msgr2=0x7fa4740fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:24:17.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.261+0000 7fa47a800700 1 -- 172.21.15.1:0/2017916423 shutdown_connections 2024-02-02T15:24:17.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.261+0000 7fa47a800700 1 -- 172.21.15.1:0/2017916423 wait complete. 2024-02-02T15:24:17.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.262+0000 7fa47a800700 1 Processor -- start 2024-02-02T15:24:17.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.262+0000 7fa47a800700 1 -- start start 2024-02-02T15:24:17.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.262+0000 7fa47a800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa474101860 0x7fa474198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:17.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.262+0000 7fa47a800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa474104190 0x7fa474199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:17.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.262+0000 7fa47a800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa474106ac0 0x7fa4741a0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:17.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.262+0000 7fa47a800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4741a0840 con 0x7fa474101860 2024-02-02T15:24:17.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.262+0000 7fa47a800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4741a09b0 con 0x7fa474104190 2024-02-02T15:24:17.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.262+0000 7fa47a800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4741a0b70 con 0x7fa474106ac0 2024-02-02T15:24:17.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.263+0000 7fa473400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa474101860 0x7fa474198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:17.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.263+0000 7fa473400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa474101860 0x7fa474198d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37370/0 (socket says 172.21.15.1:37370) 2024-02-02T15:24:17.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.263+0000 7fa473e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa474106ac0 0x7fa4741a0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:17.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.263+0000 7fa473400700 1 -- 172.21.15.1:0/2974456940 learned_addr learned my addr 172.21.15.1:0/2974456940 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:24:17.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.263+0000 7fa46be00700 1 --2- 172.21.15.1:0/2974456940 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa474104190 0x7fa474199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:17.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.263+0000 7fa473e00700 1 -- 172.21.15.1:0/2974456940 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa474104190 msgr2=0x7fa474199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:17.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.263+0000 7fa473e00700 1 --2- 172.21.15.1:0/2974456940 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa474104190 0x7fa474199240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:17.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.263+0000 7fa473e00700 1 -- 172.21.15.1:0/2974456940 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa474101860 msgr2=0x7fa474198d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:17.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.263+0000 7fa473e00700 1 --2- 172.21.15.1:0/2974456940 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa474101860 0x7fa474198d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:17.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.263+0000 7fa473e00700 1 -- 172.21.15.1:0/2974456940 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa46c009c40 con 0x7fa474106ac0 2024-02-02T15:24:17.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.263+0000 7fa473400700 1 --2- 172.21.15.1:0/2974456940 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa474101860 0x7fa474198d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:24:17.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.263+0000 7fa473e00700 1 --2- 172.21.15.1:0/2974456940 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa474106ac0 0x7fa4741a0270 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa46c00b6c0 tx=0x7fa46c004c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:24:17.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.263+0000 7fa470c00700 1 -- 172.21.15.1:0/2974456940 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa46c00e040 con 0x7fa474106ac0 2024-02-02T15:24:17.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.264+0000 7fa470c00700 1 -- 172.21.15.1:0/2974456940 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa46c00bbf0 con 0x7fa474106ac0 2024-02-02T15:24:17.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.264+0000 7fa47a800700 1 -- 172.21.15.1:0/2974456940 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4741a0df0 con 0x7fa474106ac0 2024-02-02T15:24:17.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.264+0000 7fa470c00700 1 -- 172.21.15.1:0/2974456940 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa46c005040 con 0x7fa474106ac0 2024-02-02T15:24:17.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.264+0000 7fa47a800700 1 -- 172.21.15.1:0/2974456940 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4741a11c0 con 0x7fa474106ac0 2024-02-02T15:24:17.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.265+0000 7fa47a800700 1 -- 172.21.15.1:0/2974456940 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa474108640 con 0x7fa474106ac0 2024-02-02T15:24:17.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.266+0000 7fa470c00700 1 -- 172.21.15.1:0/2974456940 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 16) v1 ==== 135069+0+0 (secure 0 0 0) 0x7fa46c01d470 con 0x7fa474106ac0 2024-02-02T15:24:17.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.268+0000 7fa470c00700 1 --2- 172.21.15.1:0/2974456940 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fa4540a0520 0x7fa4540a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:17.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.268+0000 7fa470c00700 1 -- 172.21.15.1:0/2974456940 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa46c0cce90 con 0x7fa474106ac0 2024-02-02T15:24:17.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.268+0000 7fa473400700 1 --2- 172.21.15.1:0/2974456940 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fa4540a0520 0x7fa4540a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:17.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.268+0000 7fa473400700 1 --2- 172.21.15.1:0/2974456940 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fa4540a0520 0x7fa4540a29e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa47419a1d0 tx=0x7fa464009310 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:24:17.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.270+0000 7fa470c00700 1 -- 172.21.15.1:0/2974456940 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7fa46c096ec0 con 0x7fa474106ac0 2024-02-02T15:24:17.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.483+0000 7fa47a800700 1 -- 172.21.15.1:0/2974456940 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa47402ce40 con 0x7fa474106ac0 2024-02-02T15:24:17.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.485+0000 7fa470c00700 1 -- 172.21.15.1:0/2974456940 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) v1 ==== 74+0+18303 (secure 0 0 0) 0x7fa46c09a4e0 con 0x7fa474106ac0 2024-02-02T15:24:17.484 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:24:17.484 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":76,"fsid":"371a409c-c1de-11ee-95b6-87774f69a715","created":"2024-02-02T15:18:26.746506+0000","modified":"2024-02-02T15:24:09.807430+0000","last_up_change":"2024-02-02T15:24:08.800951+0000","last_in_change":"2024-02-02T15:23:52.784280+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T15:21:22.471754+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"73e8a07f-3560-4c19-85ed-030d4d8f0f23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6803","nonce":2591906423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6805","nonce":2591906423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6809","nonce":2591906423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6807","nonce":2591906423}]},"public_addr":"172.21.15.1:6803/2591906423","cluster_addr":"172.21.15.1:6805/2591906423","heartbeat_back_addr":"172.21.15.1:6809/2591906423","heartbeat_front_addr":"172.21.15.1:6807/2591906423","state":["exists","up"]},{"osd":1,"uuid":"e53cffec-0ff9-4ed9-8dcf-ff47d04db2bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6801","nonce":747581719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6803","nonce":747581719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6807","nonce":747581719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6805","nonce":747581719}]},"public_addr":"172.21.15.71:6801/747581719","cluster_addr":"172.21.15.71:6803/747581719","heartbeat_back_addr":"172.21.15.71:6807/747581719","heartbeat_front_addr":"172.21.15.71:6805/747581719","state":["exists","up"]},{"osd":2,"uuid":"856c2347-294c-482b-8595-ad77f4792db4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6801","nonce":377511414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6803","nonce":377511414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6807","nonce":377511414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6805","nonce":377511414}]},"public_addr":"172.21.15.142:6801/377511414","cluster_addr":"172.21.15.142:6803/377511414","heartbeat_back_addr":"172.21.15.142:6807/377511414","heartbeat_front_addr":"172.21.15.142:6805/377511414","state":["exists","up"]},{"osd":3,"uuid":"4b946ad2-2da9-4e0b-9d61-d039d8669a42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6811","nonce":1086124477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6813","nonce":1086124477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6817","nonce":1086124477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6815","nonce":1086124477}]},"public_addr":"172.21.15.1:6811/1086124477","cluster_addr":"172.21.15.1:6813/1086124477","heartbeat_back_addr":"172.21.15.1:6817/1086124477","heartbeat_front_addr":"172.21.15.1:6815/1086124477","state":["exists","up"]},{"osd":4,"uuid":"43e36c43-47a0-4634-a45b-1da14a0e19d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6809","nonce":3095380305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6811","nonce":3095380305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6815","nonce":3095380305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6813","nonce":3095380305}]},"public_addr":"172.21.15.71:6809/3095380305","cluster_addr":"172.21.15.71:6811/3095380305","heartbeat_back_addr":"172.21.15.71:6815/3095380305","heartbeat_front_addr":"172.21.15.71:6813/3095380305","state":["exists","up"]},{"osd":5,"uuid":"eb48160e-ea7c-42f7-9477-be110bc8a34e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6809","nonce":4195149262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6811","nonce":4195149262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6815","nonce":4195149262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6813","nonce":4195149262}]},"public_addr":"172.21.15.142:6809/4195149262","cluster_addr":"172.21.15.142:6811/4195149262","heartbeat_back_addr":"172.21.15.142:6815/4195149262","heartbeat_front_addr":"172.21.15.142:6813/4195149262","state":["exists","up"]},{"osd":6,"uuid":"0ec4501a-5853-4ca6-b325-231f5b72335e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6819","nonce":1560153428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6821","nonce":1560153428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6825","nonce":1560153428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6823","nonce":1560153428}]},"public_addr":"172.21.15.1:6819/1560153428","cluster_addr":"172.21.15.1:6821/1560153428","heartbeat_back_addr":"172.21.15.1:6825/1560153428","heartbeat_front_addr":"172.21.15.1:6823/1560153428","state":["exists","up"]},{"osd":7,"uuid":"029debe8-9cdf-44f6-9aea-cabd1d185f93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6817","nonce":4080955538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6819","nonce":4080955538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6823","nonce":4080955538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6821","nonce":4080955538}]},"public_addr":"172.21.15.71:6817/4080955538","cluster_addr":"172.21.15.71:6819/4080955538","heartbeat_back_addr":"172.21.15.71:6823/4080955538","heartbeat_front_addr":"172.21.15.71:6821/4080955538","state":["exists","up"]},{"osd":8,"uuid":"7c90896c-9955-4c1b-aaca-7531dd5457f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6817","nonce":3942038209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6819","nonce":3942038209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6823","nonce":3942038209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6821","nonce":3942038209}]},"public_addr":"172.21.15.142:6817/3942038209","cluster_addr":"172.21.15.142:6819/3942038209","heartbeat_back_addr":"172.21.15.142:6823/3942038209","heartbeat_front_addr":"172.21.15.142:6821/3942038209","state":["exists","up"]},{"osd":9,"uuid":"7e165e68-31a8-4598-a577-de3e90ae5c5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6827","nonce":2716544850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6829","nonce":2716544850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6833","nonce":2716544850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6831","nonce":2716544850}]},"public_addr":"172.21.15.1:6827/2716544850","cluster_addr":"172.21.15.1:6829/2716544850","heartbeat_back_addr":"172.21.15.1:6833/2716544850","heartbeat_front_addr":"172.21.15.1:6831/2716544850","state":["exists","up"]},{"osd":10,"uuid":"48cd5fdb-0887-444a-aab8-a1de24282bb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6825","nonce":84691841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6827","nonce":84691841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6831","nonce":84691841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6829","nonce":84691841}]},"public_addr":"172.21.15.71:6825/84691841","cluster_addr":"172.21.15.71:6827/84691841","heartbeat_back_addr":"172.21.15.71:6831/84691841","heartbeat_front_addr":"172.21.15.71:6829/84691841","state":["exists","up"]},{"osd":11,"uuid":"f011dd27-11b0-4166-adfe-76756533163a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6825","nonce":2290179934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6827","nonce":2290179934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6831","nonce":2290179934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6829","nonce":2290179934}]},"public_addr":"172.21.15.142:6825/2290179934","cluster_addr":"172.21.15.142:6827/2290179934","heartbeat_back_addr":"172.21.15.142:6831/2290179934","heartbeat_front_addr":"172.21.15.142:6829/2290179934","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:20:44.060692+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:01.903817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:19.368258+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:40.028037+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:57.513361+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:22:14.805944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:22:35.188439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:22:53.014412+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:23:10.507420+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:23:31.849765+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:23:50.070612+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:24:07.388625+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/4092532060":"2024-02-03T15:19:25.836171+0000","172.21.15.1:6801/395498296":"2024-02-03T15:19:25.836171+0000","172.21.15.1:6800/395498296":"2024-02-03T15:19:25.836171+0000","172.21.15.1:0/1575373489":"2024-02-03T15:19:25.836171+0000","172.21.15.1:0/865983374":"2024-02-03T15:19:01.431596+0000","172.21.15.1:0/3165960227":"2024-02-03T15:19:01.431596+0000","172.21.15.1:0/2350830293":"2024-02-03T15:19:01.431596+0000","172.21.15.1:0/2959690250":"2024-02-03T15:19:25.836171+0000","172.21.15.1:6800/1162530732":"2024-02-03T15:19:01.431596+0000","172.21.15.1:6801/1162530732":"2024-02-03T15:19:01.431596+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T15:24:17.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.488+0000 7fa47a800700 1 -- 172.21.15.1:0/2974456940 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fa4540a0520 msgr2=0x7fa4540a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:17.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.488+0000 7fa47a800700 1 --2- 172.21.15.1:0/2974456940 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fa4540a0520 0x7fa4540a29e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa47419a1d0 tx=0x7fa464009310 comp rx=0 tx=0).stop 2024-02-02T15:24:17.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.488+0000 7fa47a800700 1 -- 172.21.15.1:0/2974456940 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa474106ac0 msgr2=0x7fa4741a0270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:17.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.488+0000 7fa47a800700 1 --2- 172.21.15.1:0/2974456940 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa474106ac0 0x7fa4741a0270 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa46c00b6c0 tx=0x7fa46c004c60 comp rx=0 tx=0).stop 2024-02-02T15:24:17.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.488+0000 7fa47a800700 1 -- 172.21.15.1:0/2974456940 shutdown_connections 2024-02-02T15:24:17.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.488+0000 7fa47a800700 1 --2- 172.21.15.1:0/2974456940 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fa4540a0520 0x7fa4540a29e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:17.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.488+0000 7fa47a800700 1 --2- 172.21.15.1:0/2974456940 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa474106ac0 0x7fa4741a0270 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:17.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.488+0000 7fa47a800700 1 --2- 172.21.15.1:0/2974456940 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa474101860 0x7fa474198d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:17.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.488+0000 7fa47a800700 1 --2- 172.21.15.1:0/2974456940 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa474104190 0x7fa474199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:17.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.488+0000 7fa47a800700 1 -- 172.21.15.1:0/2974456940 >> 172.21.15.1:0/2974456940 conn(0x7fa4740fb170 msgr2=0x7fa4741024d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:24:17.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.488+0000 7fa47a800700 1 -- 172.21.15.1:0/2974456940 shutdown_connections 2024-02-02T15:24:17.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.488+0000 7fa47a800700 1 -- 172.21.15.1:0/2974456940 wait complete. 2024-02-02T15:24:17.507 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-02T15:21:22.471754+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-02T15:24:17.507 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-02T15:24:17.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.709+0000 7f6bb1c00700 1 -- 172.21.15.1:0/4206557067 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6bac100e20 msgr2=0x7f6bac101240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:17.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.709+0000 7f6bb1c00700 1 --2- 172.21.15.1:0/4206557067 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6bac100e20 0x7f6bac101240 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c009990 tx=0x7f6b9c009ca0 comp rx=0 tx=0).stop 2024-02-02T15:24:17.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.710+0000 7f6bb1c00700 1 -- 172.21.15.1:0/4206557067 shutdown_connections 2024-02-02T15:24:17.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.710+0000 7f6bb1c00700 1 --2- 172.21.15.1:0/4206557067 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6bac102930 0x7f6bac109250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:17.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.710+0000 7f6bb1c00700 1 --2- 172.21.15.1:0/4206557067 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6bac100e20 0x7f6bac101240 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:17.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.710+0000 7f6bb1c00700 1 --2- 172.21.15.1:0/4206557067 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6bac101f70 0x7f6bac1023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:17.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.710+0000 7f6bb1c00700 1 -- 172.21.15.1:0/4206557067 >> 172.21.15.1:0/4206557067 conn(0x7f6bac0fc390 msgr2=0x7f6bac0fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:24:17.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.710+0000 7f6bb1c00700 1 -- 172.21.15.1:0/4206557067 shutdown_connections 2024-02-02T15:24:17.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.710+0000 7f6bb1c00700 1 -- 172.21.15.1:0/4206557067 wait complete. 2024-02-02T15:24:17.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.711+0000 7f6bb1c00700 1 Processor -- start 2024-02-02T15:24:17.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.711+0000 7f6bb1c00700 1 -- start start 2024-02-02T15:24:17.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.712+0000 7f6bb1c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6bac100e20 0x7f6bac196b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:17.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.712+0000 7f6bb1c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6bac101f70 0x7f6bac197060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:17.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.712+0000 7f6bb1c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6bac102930 0x7f6bac19e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:17.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.712+0000 7f6bb1c00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bac19e6a0 con 0x7f6bac100e20 2024-02-02T15:24:17.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.712+0000 7f6bb1c00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bac19e7e0 con 0x7f6bac102930 2024-02-02T15:24:17.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.712+0000 7f6bb1c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bac19e970 con 0x7f6bac101f70 2024-02-02T15:24:17.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.712+0000 7f6baaa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6bac100e20 0x7f6bac196b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:17.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.712+0000 7f6baaa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6bac100e20 0x7f6bac196b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37392/0 (socket says 172.21.15.1:37392) 2024-02-02T15:24:17.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.712+0000 7f6baaa00700 1 -- 172.21.15.1:0/4026671658 learned_addr learned my addr 172.21.15.1:0/4026671658 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:24:17.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.712+0000 7f6baa000700 1 --2- 172.21.15.1:0/4026671658 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6bac101f70 0x7f6bac197060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:17.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.712+0000 7f6bab400700 1 --2- 172.21.15.1:0/4026671658 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6bac102930 0x7f6bac19e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:17.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.712+0000 7f6baaa00700 1 -- 172.21.15.1:0/4026671658 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6bac101f70 msgr2=0x7f6bac197060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:17.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.712+0000 7f6baaa00700 1 --2- 172.21.15.1:0/4026671658 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6bac101f70 0x7f6bac197060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:17.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.712+0000 7f6baaa00700 1 -- 172.21.15.1:0/4026671658 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6bac102930 msgr2=0x7f6bac19e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:17.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.713+0000 7f6baaa00700 1 --2- 172.21.15.1:0/4026671658 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6bac102930 0x7f6bac19e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:17.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.713+0000 7f6baaa00700 1 -- 172.21.15.1:0/4026671658 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b9c0095f0 con 0x7f6bac100e20 2024-02-02T15:24:17.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.713+0000 7f6bab400700 1 --2- 172.21.15.1:0/4026671658 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6bac102930 0x7f6bac19e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:24:17.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.713+0000 7f6baaa00700 1 --2- 172.21.15.1:0/4026671658 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6bac100e20 0x7f6bac196b20 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c00c010 tx=0x7f6b9c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:24:17.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.713+0000 7f6ba3400700 1 -- 172.21.15.1:0/4026671658 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c01d070 con 0x7f6bac100e20 2024-02-02T15:24:17.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.713+0000 7f6ba3400700 1 -- 172.21.15.1:0/4026671658 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b9c0054f0 con 0x7f6bac100e20 2024-02-02T15:24:17.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.713+0000 7f6ba3400700 1 -- 172.21.15.1:0/4026671658 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c004ca0 con 0x7f6bac100e20 2024-02-02T15:24:17.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.713+0000 7f6bb1c00700 1 -- 172.21.15.1:0/4026671658 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bac19ebf0 con 0x7f6bac100e20 2024-02-02T15:24:17.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.713+0000 7f6bb1c00700 1 -- 172.21.15.1:0/4026671658 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bac19ef60 con 0x7f6bac100e20 2024-02-02T15:24:17.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.714+0000 7f6bb1c00700 1 -- 172.21.15.1:0/4026671658 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bac10c670 con 0x7f6bac100e20 2024-02-02T15:24:17.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.716+0000 7f6ba3400700 1 -- 172.21.15.1:0/4026671658 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) v1 ==== 135069+0+0 (secure 0 0 0) 0x7f6b9c00fbc0 con 0x7f6bac100e20 2024-02-02T15:24:17.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.717+0000 7f6ba3400700 1 --2- 172.21.15.1:0/4026671658 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f6b8c0a05a0 0x7f6b8c0a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:17.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.718+0000 7f6ba3400700 1 -- 172.21.15.1:0/4026671658 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f6b9c0ccc90 con 0x7f6bac100e20 2024-02-02T15:24:17.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.718+0000 7f6baa000700 1 --2- 172.21.15.1:0/4026671658 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f6b8c0a05a0 0x7f6b8c0a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:17.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.718+0000 7f6baa000700 1 --2- 172.21.15.1:0/4026671658 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f6b8c0a05a0 0x7f6b8c0a2a60 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6b94007c60 tx=0x7f6b940073d0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:24:17.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.720+0000 7f6ba3400700 1 -- 172.21.15.1:0/4026671658 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f6b9c096cc0 con 0x7f6bac100e20 2024-02-02T15:24:17.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.934+0000 7f6bb1c00700 1 -- 172.21.15.1:0/4026671658 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6bac107d10 con 0x7f6bac100e20 2024-02-02T15:24:17.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.935+0000 7f6ba3400700 1 -- 172.21.15.1:0/4026671658 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v76) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6b9c09a2e0 con 0x7f6bac100e20 2024-02-02T15:24:17.934 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-02-02T15:24:17.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.937+0000 7f6bb1c00700 1 -- 172.21.15.1:0/4026671658 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f6b8c0a05a0 msgr2=0x7f6b8c0a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:17.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.937+0000 7f6bb1c00700 1 --2- 172.21.15.1:0/4026671658 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f6b8c0a05a0 0x7f6b8c0a2a60 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6b94007c60 tx=0x7f6b940073d0 comp rx=0 tx=0).stop 2024-02-02T15:24:17.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.937+0000 7f6bb1c00700 1 -- 172.21.15.1:0/4026671658 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6bac100e20 msgr2=0x7f6bac196b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:17.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.937+0000 7f6bb1c00700 1 --2- 172.21.15.1:0/4026671658 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6bac100e20 0x7f6bac196b20 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c00c010 tx=0x7f6b9c004730 comp rx=0 tx=0).stop 2024-02-02T15:24:17.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.937+0000 7f6bb1c00700 1 -- 172.21.15.1:0/4026671658 shutdown_connections 2024-02-02T15:24:17.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.937+0000 7f6bb1c00700 1 --2- 172.21.15.1:0/4026671658 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6bac102930 0x7f6bac19e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:17.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.937+0000 7f6bb1c00700 1 --2- 172.21.15.1:0/4026671658 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6bac100e20 0x7f6bac196b20 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:17.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.937+0000 7f6bb1c00700 1 --2- 172.21.15.1:0/4026671658 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f6b8c0a05a0 0x7f6b8c0a2a60 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:17.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.937+0000 7f6bb1c00700 1 --2- 172.21.15.1:0/4026671658 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6bac101f70 0x7f6bac197060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:17.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.937+0000 7f6bb1c00700 1 -- 172.21.15.1:0/4026671658 >> 172.21.15.1:0/4026671658 conn(0x7f6bac0fc390 msgr2=0x7f6bac0fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:24:17.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.938+0000 7f6bb1c00700 1 -- 172.21.15.1:0/4026671658 shutdown_connections 2024-02-02T15:24:17.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:17.938+0000 7f6bb1c00700 1 -- 172.21.15.1:0/4026671658 wait complete. 2024-02-02T15:24:17.955 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T15:24:18.083 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:17 smithi142 ceph-mon[45728]: from='client.24433 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi001=a;smithi001=d;smithi001=g;smithi001=j;smithi071=b;smithi071=e;smithi071=h;smithi071=k;smithi142=c;smithi142=f;smithi142=i;smithi142=l", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:24:18.084 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:17 smithi142 ceph-mon[45728]: Saving service mds.all spec with placement smithi001=a;smithi001=d;smithi001=g;smithi001=j;smithi071=b;smithi071=e;smithi071=h;smithi071=k;smithi142=c;smithi142=f;smithi142=i;smithi142=l;count:12 2024-02-02T15:24:18.084 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:17 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:24:18.084 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:17 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:24:18.084 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:17 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:18.084 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:17 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-02T15:24:18.084 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:17 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-02T15:24:18.084 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:17 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:24:18.084 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:17 smithi142 ceph-mon[45728]: Deploying daemon mds.l on smithi142 2024-02-02T15:24:18.084 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:17 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/2974456940' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-02T15:24:18.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:17 smithi071 ceph-mon[44707]: from='client.24433 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi001=a;smithi001=d;smithi001=g;smithi001=j;smithi071=b;smithi071=e;smithi071=h;smithi071=k;smithi142=c;smithi142=f;smithi142=i;smithi142=l", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:24:18.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:17 smithi071 ceph-mon[44707]: Saving service mds.all spec with placement smithi001=a;smithi001=d;smithi001=g;smithi001=j;smithi071=b;smithi071=e;smithi071=h;smithi071=k;smithi142=c;smithi142=f;smithi142=i;smithi142=l;count:12 2024-02-02T15:24:18.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:17 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:24:18.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:17 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:24:18.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:17 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:18.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:17 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-02T15:24:18.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:17 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-02T15:24:18.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:17 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:24:18.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:17 smithi071 ceph-mon[44707]: Deploying daemon mds.l on smithi142 2024-02-02T15:24:18.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:17 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/2974456940' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-02T15:24:18.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:18.153+0000 7fa86cc00700 1 -- 172.21.15.1:0/2954196955 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa868100d40 msgr2=0x7fa868101160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:18.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:18.153+0000 7fa86cc00700 1 --2- 172.21.15.1:0/2954196955 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa868100d40 0x7fa868101160 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa8580099c0 tx=0x7fa858009cd0 comp rx=0 tx=0).stop 2024-02-02T15:24:18.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:18.153+0000 7fa86cc00700 1 -- 172.21.15.1:0/2954196955 shutdown_connections 2024-02-02T15:24:18.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:18.153+0000 7fa86cc00700 1 --2- 172.21.15.1:0/2954196955 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa8681028d0 0x7fa868109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:18.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:18.153+0000 7fa86cc00700 1 --2- 172.21.15.1:0/2954196955 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa868100d40 0x7fa868101160 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:18.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:18.153+0000 7fa86cc00700 1 --2- 172.21.15.1:0/2954196955 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa868101f30 0x7fa868102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:18.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:18.153+0000 7fa86cc00700 1 -- 172.21.15.1:0/2954196955 >> 172.21.15.1:0/2954196955 conn(0x7fa8680fc290 msgr2=0x7fa8680fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:24:18.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:18.153+0000 7fa86cc00700 1 -- 172.21.15.1:0/2954196955 shutdown_connections 2024-02-02T15:24:18.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:18.153+0000 7fa86cc00700 1 -- 172.21.15.1:0/2954196955 wait complete. 2024-02-02T15:24:18.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:18.155+0000 7fa86cc00700 1 Processor -- start 2024-02-02T15:24:18.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:18.155+0000 7fa86cc00700 1 -- start start 2024-02-02T15:24:18.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:18.155+0000 7fa86cc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa868100d40 0x7fa868074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:18.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:18.156+0000 7fa86cc00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa868101f30 0x7fa868073130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:18.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:18.156+0000 7fa86cc00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa8681028d0 0x7fa868073670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:18.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:18.156+0000 7fa86cc00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa868073e10 con 0x7fa8681028d0 2024-02-02T15:24:18.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:18.156+0000 7fa86cc00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa868073f80 con 0x7fa868101f30 2024-02-02T15:24:18.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:18.156+0000 7fa86cc00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8681a0ac0 con 0x7fa868100d40 2024-02-02T15:24:18.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:18.156+0000 7fa865e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa868100d40 0x7fa868074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:18.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:18.156+0000 7fa865e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa868100d40 0x7fa868074af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:57058/0 (socket says 172.21.15.1:57058) 2024-02-02T15:24:18.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:18.156+0000 7fa865e00700 1 -- 172.21.15.1:0/2121584540 learned_addr learned my addr 172.21.15.1:0/2121584540 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:24:18.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:18.156+0000 7fa866800700 1 --2- 172.21.15.1:0/2121584540 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa8681028d0 0x7fa868073670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:18.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:18.156+0000 7fa865400700 1 --2- 172.21.15.1:0/2121584540 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa868101f30 0x7fa868073130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:18.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:18.156+0000 7fa865e00700 1 -- 172.21.15.1:0/2121584540 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa868101f30 msgr2=0x7fa868073130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:18.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:18.156+0000 7fa865e00700 1 --2- 172.21.15.1:0/2121584540 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa868101f30 0x7fa868073130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:18.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:18.156+0000 7fa865e00700 1 -- 172.21.15.1:0/2121584540 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa8681028d0 msgr2=0x7fa868073670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:18.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:18.156+0000 7fa865e00700 1 --2- 172.21.15.1:0/2121584540 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa8681028d0 0x7fa868073670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:18.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:18.156+0000 7fa865e00700 1 -- 172.21.15.1:0/2121584540 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8580095f0 con 0x7fa868100d40 2024-02-02T15:24:18.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:18.156+0000 7fa865e00700 1 --2- 172.21.15.1:0/2121584540 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa868100d40 0x7fa868074af0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa8580098e0 tx=0x7fa85800f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:24:18.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:18.157+0000 7fa85ea00700 1 -- 172.21.15.1:0/2121584540 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa85801d070 con 0x7fa868100d40 2024-02-02T15:24:18.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:18.157+0000 7fa85ea00700 1 -- 172.21.15.1:0/2121584540 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa858005510 con 0x7fa868100d40 2024-02-02T15:24:18.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:18.157+0000 7fa86cc00700 1 -- 172.21.15.1:0/2121584540 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8681a0d40 con 0x7fa868100d40 2024-02-02T15:24:18.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:18.157+0000 7fa85ea00700 1 -- 172.21.15.1:0/2121584540 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa858004c00 con 0x7fa868100d40 2024-02-02T15:24:18.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:18.157+0000 7fa86cc00700 1 -- 172.21.15.1:0/2121584540 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8681a1160 con 0x7fa868100d40 2024-02-02T15:24:18.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:18.158+0000 7fa86cc00700 1 -- 172.21.15.1:0/2121584540 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa868107cc0 con 0x7fa868100d40 2024-02-02T15:24:18.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:18.160+0000 7fa85ea00700 1 -- 172.21.15.1:0/2121584540 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 16) v1 ==== 135069+0+0 (secure 0 0 0) 0x7fa85800fb80 con 0x7fa868100d40 2024-02-02T15:24:18.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:18.161+0000 7fa85ea00700 1 --2- 172.21.15.1:0/2121584540 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fa8480a0520 0x7fa8480a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:18.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:18.161+0000 7fa85ea00700 1 -- 172.21.15.1:0/2121584540 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa8580cc7d0 con 0x7fa868100d40 2024-02-02T15:24:18.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:18.161+0000 7fa865400700 1 --2- 172.21.15.1:0/2121584540 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fa8480a0520 0x7fa8480a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:18.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:18.161+0000 7fa865400700 1 --2- 172.21.15.1:0/2121584540 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fa8480a0520 0x7fa8480a29e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa85000bbd0 tx=0x7fa85000b5b0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:24:18.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:18.163+0000 7fa85ea00700 1 -- 172.21.15.1:0/2121584540 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7fa858096800 con 0x7fa868100d40 2024-02-02T15:24:18.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:17 smithi001 ceph-mon[35470]: from='client.24433 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi001=a;smithi001=d;smithi001=g;smithi001=j;smithi071=b;smithi071=e;smithi071=h;smithi071=k;smithi142=c;smithi142=f;smithi142=i;smithi142=l", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:24:18.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:17 smithi001 ceph-mon[35470]: Saving service mds.all spec with placement smithi001=a;smithi001=d;smithi001=g;smithi001=j;smithi071=b;smithi071=e;smithi071=h;smithi071=k;smithi142=c;smithi142=f;smithi142=i;smithi142=l;count:12 2024-02-02T15:24:18.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:17 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:24:18.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:17 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:24:18.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:17 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:18.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:17 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-02T15:24:18.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:17 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-02T15:24:18.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:17 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:24:18.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:17 smithi001 ceph-mon[35470]: Deploying daemon mds.l on smithi142 2024-02-02T15:24:18.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:17 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/2974456940' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-02T15:24:18.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:18.418+0000 7fa86cc00700 1 -- 172.21.15.1:0/2121584540 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa86802cc70 con 0x7fa868100d40 2024-02-02T15:24:18.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:18.419+0000 7fa85ea00700 1 -- 172.21.15.1:0/2121584540 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+501 (secure 0 0 0) 0x7fa858099e20 con 0x7fa868100d40 2024-02-02T15:24:18.418 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:24:18.418 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-02-02T15:24:18.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:18.421+0000 7fa86cc00700 1 -- 172.21.15.1:0/2121584540 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fa8480a0520 msgr2=0x7fa8480a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:18.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:18.421+0000 7fa86cc00700 1 --2- 172.21.15.1:0/2121584540 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fa8480a0520 0x7fa8480a29e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa85000bbd0 tx=0x7fa85000b5b0 comp rx=0 tx=0).stop 2024-02-02T15:24:18.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:18.421+0000 7fa86cc00700 1 -- 172.21.15.1:0/2121584540 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa868100d40 msgr2=0x7fa868074af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:18.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:18.422+0000 7fa86cc00700 1 --2- 172.21.15.1:0/2121584540 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa868100d40 0x7fa868074af0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa8580098e0 tx=0x7fa85800f690 comp rx=0 tx=0).stop 2024-02-02T15:24:18.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:18.422+0000 7fa86cc00700 1 -- 172.21.15.1:0/2121584540 shutdown_connections 2024-02-02T15:24:18.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:18.422+0000 7fa86cc00700 1 --2- 172.21.15.1:0/2121584540 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa8681028d0 0x7fa868073670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:18.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:18.422+0000 7fa86cc00700 1 --2- 172.21.15.1:0/2121584540 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fa8480a0520 0x7fa8480a29e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:18.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:18.422+0000 7fa86cc00700 1 --2- 172.21.15.1:0/2121584540 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa868100d40 0x7fa868074af0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:18.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:18.422+0000 7fa86cc00700 1 --2- 172.21.15.1:0/2121584540 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa868101f30 0x7fa868073130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:18.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:18.422+0000 7fa86cc00700 1 -- 172.21.15.1:0/2121584540 >> 172.21.15.1:0/2121584540 conn(0x7fa8680fc290 msgr2=0x7fa8680fe570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:24:18.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:18.422+0000 7fa86cc00700 1 -- 172.21.15.1:0/2121584540 shutdown_connections 2024-02-02T15:24:18.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:18.422+0000 7fa86cc00700 1 -- 172.21.15.1:0/2121584540 wait complete. 2024-02-02T15:24:18.422 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 1 2024-02-02T15:24:18.670 INFO:journalctl@ceph.mds.l.smithi142.stdout:Feb 02 15:24:18 smithi142 systemd[1]: Starting Ceph mds.l for 371a409c-c1de-11ee-95b6-87774f69a715... 2024-02-02T15:24:19.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:18 smithi071 ceph-mon[44707]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-02T15:24:19.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:18 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/4026671658' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-02T15:24:19.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:18 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/2121584540' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:24:19.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:18 smithi071 ceph-mon[44707]: mds.? [v2:172.21.15.142:6832/7648986,v1:172.21.15.142:6833/7648986] up:boot 2024-02-02T15:24:19.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:18 smithi071 ceph-mon[44707]: fsmap 1 up:standby 2024-02-02T15:24:19.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:18 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-02-02T15:24:19.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:18 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:19.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:18 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:19.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:18 smithi142 ceph-mon[45728]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-02T15:24:19.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:18 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/4026671658' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-02T15:24:19.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:18 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/2121584540' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:24:19.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:18 smithi142 ceph-mon[45728]: mds.? [v2:172.21.15.142:6832/7648986,v1:172.21.15.142:6833/7648986] up:boot 2024-02-02T15:24:19.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:18 smithi142 ceph-mon[45728]: fsmap 1 up:standby 2024-02-02T15:24:19.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:18 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-02-02T15:24:19.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:18 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:19.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:18 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:19.171 INFO:journalctl@ceph.mds.l.smithi142.stdout:Feb 02 15:24:18 smithi142 ceph-mds[82195]: -- 172.21.15.142:0/1834200693 <== mon.1 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5647c05ad1e0 con 0x5647c0582800 2024-02-02T15:24:19.171 INFO:journalctl@ceph.mds.l.smithi142.stdout:Feb 02 15:24:18 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mds-l[82172]: starting mds.l at 2024-02-02T15:24:19.171 INFO:journalctl@ceph.mds.l.smithi142.stdout:Feb 02 15:24:18 smithi142 bash[82158]: 9614f16ef39c41bdc74e34832dc6b4865057dc14f97952131fd7e54bda5cb2f0 2024-02-02T15:24:19.171 INFO:journalctl@ceph.mds.l.smithi142.stdout:Feb 02 15:24:18 smithi142 systemd[1]: Started Ceph mds.l for 371a409c-c1de-11ee-95b6-87774f69a715. 2024-02-02T15:24:19.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:18 smithi001 ceph-mon[35470]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-02T15:24:19.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:18 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/4026671658' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-02T15:24:19.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:18 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/2121584540' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:24:19.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:18 smithi001 ceph-mon[35470]: mds.? [v2:172.21.15.142:6832/7648986,v1:172.21.15.142:6833/7648986] up:boot 2024-02-02T15:24:19.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:18 smithi001 ceph-mon[35470]: fsmap 1 up:standby 2024-02-02T15:24:19.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:18 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-02-02T15:24:19.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:18 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:19.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:18 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:19.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:19 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:19.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:19 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-02T15:24:19.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:19 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-02T15:24:19.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:19 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:24:19.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:19 smithi071 ceph-mon[44707]: Deploying daemon mds.k on smithi071 2024-02-02T15:24:20.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:19 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:20.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:19 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-02T15:24:20.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:19 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-02T15:24:20.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:19 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:24:20.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:19 smithi142 ceph-mon[45728]: Deploying daemon mds.k on smithi071 2024-02-02T15:24:20.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:19 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:20.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:19 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-02T15:24:20.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:19 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-02T15:24:20.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:19 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:24:20.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:19 smithi001 ceph-mon[35470]: Deploying daemon mds.k on smithi071 2024-02-02T15:24:20.308 INFO:journalctl@ceph.mds.k.smithi071.stdout:Feb 02 15:24:20 smithi071 systemd[1]: Starting Ceph mds.k for 371a409c-c1de-11ee-95b6-87774f69a715... 2024-02-02T15:24:20.439 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T15:24:20.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:20.666+0000 7f9f10a00700 1 -- 172.21.15.1:0/1914364769 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9f08074d80 msgr2=0x7f9f080731d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:20.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:20.666+0000 7f9f10a00700 1 --2- 172.21.15.1:0/1914364769 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9f08074d80 0x7f9f080731d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9ef8009990 tx=0x7f9ef8009ca0 comp rx=0 tx=0).stop 2024-02-02T15:24:20.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:20.666+0000 7f9f10a00700 1 -- 172.21.15.1:0/1914364769 shutdown_connections 2024-02-02T15:24:20.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:20.666+0000 7f9f10a00700 1 --2- 172.21.15.1:0/1914364769 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9f08108c10 0x7f9f0810b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:20.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:20.666+0000 7f9f10a00700 1 --2- 172.21.15.1:0/1914364769 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9f08074d80 0x7f9f080731d0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:20.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:20.666+0000 7f9f10a00700 1 --2- 172.21.15.1:0/1914364769 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f080737a0 0x7f9f08073c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:20.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:20.666+0000 7f9f10a00700 1 -- 172.21.15.1:0/1914364769 >> 172.21.15.1:0/1914364769 conn(0x7f9f080fc2c0 msgr2=0x7f9f080fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:24:20.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:20.667+0000 7f9f10a00700 1 -- 172.21.15.1:0/1914364769 shutdown_connections 2024-02-02T15:24:20.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:20.667+0000 7f9f10a00700 1 -- 172.21.15.1:0/1914364769 wait complete. 2024-02-02T15:24:20.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:20.668+0000 7f9f10a00700 1 Processor -- start 2024-02-02T15:24:20.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:20.668+0000 7f9f10a00700 1 -- start start 2024-02-02T15:24:20.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:20.669+0000 7f9f10a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f080737a0 0x7f9f08198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:20.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:20.669+0000 7f9f10a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9f08074d80 0x7f9f081991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:20.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:20.669+0000 7f9f10a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9f08108c10 0x7f9f081a0240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:20.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:20.669+0000 7f9f10a00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f081a0810 con 0x7f9f08074d80 2024-02-02T15:24:20.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:20.669+0000 7f9f10a00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f081a0980 con 0x7f9f080737a0 2024-02-02T15:24:20.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:20.669+0000 7f9f10a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f081a0b40 con 0x7f9f08108c10 2024-02-02T15:24:20.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:20.669+0000 7f9f0e000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f080737a0 0x7f9f08198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:20.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:20.669+0000 7f9f0d600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9f08074d80 0x7f9f081991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:20.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:20.669+0000 7f9f0e000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f080737a0 0x7f9f08198ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:39650/0 (socket says 172.21.15.1:39650) 2024-02-02T15:24:20.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:20.669+0000 7f9f0d600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9f08074d80 0x7f9f081991e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37426/0 (socket says 172.21.15.1:37426) 2024-02-02T15:24:20.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:20.669+0000 7f9f0e000700 1 -- 172.21.15.1:0/3747327893 learned_addr learned my addr 172.21.15.1:0/3747327893 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:24:20.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:20.669+0000 7f9f0ea00700 1 --2- 172.21.15.1:0/3747327893 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9f08108c10 0x7f9f081a0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:20.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:20.670+0000 7f9f0d600700 1 -- 172.21.15.1:0/3747327893 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9f08108c10 msgr2=0x7f9f081a0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:20.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:20.670+0000 7f9f0d600700 1 --2- 172.21.15.1:0/3747327893 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9f08108c10 0x7f9f081a0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:20.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:20.670+0000 7f9f0d600700 1 -- 172.21.15.1:0/3747327893 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f080737a0 msgr2=0x7f9f08198ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:20.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:20.670+0000 7f9f0d600700 1 --2- 172.21.15.1:0/3747327893 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f080737a0 0x7f9f08198ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:20.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:20.670+0000 7f9f0d600700 1 -- 172.21.15.1:0/3747327893 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ef0009520 con 0x7f9f08074d80 2024-02-02T15:24:20.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:20.670+0000 7f9f0e000700 1 --2- 172.21.15.1:0/3747327893 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f080737a0 0x7f9f08198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:24:20.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:20.670+0000 7f9f0d600700 1 --2- 172.21.15.1:0/3747327893 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9f08074d80 0x7f9f081991e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9ef000e9c0 tx=0x7f9ef000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:24:20.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:20.670+0000 7f9f06a00700 1 -- 172.21.15.1:0/3747327893 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef000cb70 con 0x7f9f08074d80 2024-02-02T15:24:20.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:20.670+0000 7f9f06a00700 1 -- 172.21.15.1:0/3747327893 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ef000ee60 con 0x7f9f08074d80 2024-02-02T15:24:20.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:20.670+0000 7f9f10a00700 1 -- 172.21.15.1:0/3747327893 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ef80095f0 con 0x7f9f08074d80 2024-02-02T15:24:20.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:20.670+0000 7f9f06a00700 1 -- 172.21.15.1:0/3747327893 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef0018680 con 0x7f9f08074d80 2024-02-02T15:24:20.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:20.670+0000 7f9f10a00700 1 -- 172.21.15.1:0/3747327893 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f081a11a0 con 0x7f9f08074d80 2024-02-02T15:24:20.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:20.671+0000 7f9f10a00700 1 -- 172.21.15.1:0/3747327893 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f081072c0 con 0x7f9f08074d80 2024-02-02T15:24:20.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:20.673+0000 7f9f06a00700 1 -- 172.21.15.1:0/3747327893 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) v1 ==== 135069+0+0 (secure 0 0 0) 0x7f9ef0010430 con 0x7f9f08074d80 2024-02-02T15:24:20.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:20.674+0000 7f9f06a00700 1 --2- 172.21.15.1:0/3747327893 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f9ee80a0520 0x7f9ee80a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:20.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:20.675+0000 7f9f06a00700 1 -- 172.21.15.1:0/3747327893 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f9ef0014070 con 0x7f9f08074d80 2024-02-02T15:24:20.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:20.675+0000 7f9f0e000700 1 --2- 172.21.15.1:0/3747327893 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f9ee80a0520 0x7f9ee80a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:20.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:20.675+0000 7f9f0e000700 1 --2- 172.21.15.1:0/3747327893 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f9ee80a0520 0x7f9ee80a29e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f9f08199b70 tx=0x7f9ef800b260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:24:20.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:20.682+0000 7f9f06a00700 1 -- 172.21.15.1:0/3747327893 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f9ef00958d0 con 0x7f9f08074d80 2024-02-02T15:24:20.824 INFO:journalctl@ceph.mds.k.smithi071.stdout:Feb 02 15:24:20 smithi071 ceph-mds[78661]: -- 172.21.15.71:0/3669390763 <== mon.2 v2:172.21.15.71:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5611597131e0 con 0x5611596e9000 2024-02-02T15:24:20.825 INFO:journalctl@ceph.mds.k.smithi071.stdout:Feb 02 15:24:20 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mds-k[78638]: starting mds.k at 2024-02-02T15:24:20.825 INFO:journalctl@ceph.mds.k.smithi071.stdout:Feb 02 15:24:20 smithi071 bash[78623]: 98bbf5cf46e0c1632c3532f6a631e108c8ce474fb9797d2246bf56b16b0358bc 2024-02-02T15:24:20.825 INFO:journalctl@ceph.mds.k.smithi071.stdout:Feb 02 15:24:20 smithi071 systemd[1]: Started Ceph mds.k for 371a409c-c1de-11ee-95b6-87774f69a715. 2024-02-02T15:24:20.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:20.949+0000 7f9f10a00700 1 -- 172.21.15.1:0/3747327893 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9f0802d100 con 0x7f9f08074d80 2024-02-02T15:24:20.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:20.949+0000 7f9f06a00700 1 -- 172.21.15.1:0/3747327893 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2008 (secure 0 0 0) 0x7f9ef000ce60 con 0x7f9f08074d80 2024-02-02T15:24:20.949 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:24:20.949 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/7648986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":7648986},{"type":"v1","addr":"172.21.15.142:6833","nonce":7648986}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/3482746094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3482746094},{"type":"v1","addr":"172.21.15.71:6833","nonce":3482746094}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2024-02-02T15:24:20.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:20.952+0000 7f9f10a00700 1 -- 172.21.15.1:0/3747327893 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f9ee80a0520 msgr2=0x7f9ee80a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:20.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:20.952+0000 7f9f10a00700 1 --2- 172.21.15.1:0/3747327893 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f9ee80a0520 0x7f9ee80a29e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f9f08199b70 tx=0x7f9ef800b260 comp rx=0 tx=0).stop 2024-02-02T15:24:20.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:20.952+0000 7f9f10a00700 1 -- 172.21.15.1:0/3747327893 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9f08074d80 msgr2=0x7f9f081991e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:20.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:20.952+0000 7f9f10a00700 1 --2- 172.21.15.1:0/3747327893 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9f08074d80 0x7f9f081991e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9ef000e9c0 tx=0x7f9ef000ecd0 comp rx=0 tx=0).stop 2024-02-02T15:24:20.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:20.952+0000 7f9f10a00700 1 -- 172.21.15.1:0/3747327893 shutdown_connections 2024-02-02T15:24:20.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:20.952+0000 7f9f10a00700 1 --2- 172.21.15.1:0/3747327893 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f9ee80a0520 0x7f9ee80a29e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:20.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:20.952+0000 7f9f10a00700 1 --2- 172.21.15.1:0/3747327893 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9f08108c10 0x7f9f081a0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:20.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:20.952+0000 7f9f10a00700 1 --2- 172.21.15.1:0/3747327893 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9f080737a0 0x7f9f08198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:20.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:20.952+0000 7f9f10a00700 1 --2- 172.21.15.1:0/3747327893 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9f08074d80 0x7f9f081991e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:20.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:20.952+0000 7f9f10a00700 1 -- 172.21.15.1:0/3747327893 >> 172.21.15.1:0/3747327893 conn(0x7f9f080fc2c0 msgr2=0x7f9f080fe6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:24:20.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:20.952+0000 7f9f10a00700 1 -- 172.21.15.1:0/3747327893 shutdown_connections 2024-02-02T15:24:20.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:20.952+0000 7f9f10a00700 1 -- 172.21.15.1:0/3747327893 wait complete. 2024-02-02T15:24:20.954 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 3 2024-02-02T15:24:21.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:20 smithi071 ceph-mon[44707]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-02T15:24:21.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:20 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:21.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:20 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:21.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:20 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:21.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:20 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-02T15:24:21.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:20 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-02T15:24:21.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:20 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:24:21.136 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:20 smithi001 ceph-mon[35470]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-02T15:24:21.136 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:20 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:21.136 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:20 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:21.137 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:20 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:21.137 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:20 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-02T15:24:21.137 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:20 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-02T15:24:21.137 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:20 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:24:21.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:20 smithi142 ceph-mon[45728]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-02T15:24:21.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:20 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:21.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:20 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:21.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:20 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:21.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:20 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-02T15:24:21.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:20 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-02T15:24:21.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:20 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:24:22.086 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:21 smithi001 ceph-mon[35470]: Deploying daemon mds.j on smithi001 2024-02-02T15:24:22.086 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:21 smithi001 ceph-mon[35470]: mds.? [v2:172.21.15.71:6832/3482746094,v1:172.21.15.71:6833/3482746094] up:boot 2024-02-02T15:24:22.087 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:21 smithi001 ceph-mon[35470]: fsmap 2 up:standby 2024-02-02T15:24:22.087 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:21 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-02-02T15:24:22.087 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:21 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/3747327893' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:24:22.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:21 smithi071 ceph-mon[44707]: Deploying daemon mds.j on smithi001 2024-02-02T15:24:22.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:21 smithi071 ceph-mon[44707]: mds.? [v2:172.21.15.71:6832/3482746094,v1:172.21.15.71:6833/3482746094] up:boot 2024-02-02T15:24:22.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:21 smithi071 ceph-mon[44707]: fsmap 2 up:standby 2024-02-02T15:24:22.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:21 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-02-02T15:24:22.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:21 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/3747327893' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:24:22.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:21 smithi142 ceph-mon[45728]: Deploying daemon mds.j on smithi001 2024-02-02T15:24:22.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:21 smithi142 ceph-mon[45728]: mds.? [v2:172.21.15.71:6832/3482746094,v1:172.21.15.71:6833/3482746094] up:boot 2024-02-02T15:24:22.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:21 smithi142 ceph-mon[45728]: fsmap 2 up:standby 2024-02-02T15:24:22.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:21 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-02-02T15:24:22.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:21 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/3747327893' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:24:22.440 INFO:journalctl@ceph.mds.j.smithi001.stdout:Feb 02 15:24:22 smithi001 systemd[1]: Starting Ceph mds.j for 371a409c-c1de-11ee-95b6-87774f69a715... 2024-02-02T15:24:22.980 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T15:24:23.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:22 smithi071 ceph-mon[44707]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-02T15:24:23.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:22 smithi142 ceph-mon[45728]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-02T15:24:23.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:23.188+0000 7fb710000700 1 -- 172.21.15.1:0/1743422687 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb7080737a0 msgr2=0x7fb708073c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:23.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:23.188+0000 7fb710000700 1 --2- 172.21.15.1:0/1743422687 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb7080737a0 0x7fb708073c20 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb6f0009990 tx=0x7fb6f0009ca0 comp rx=0 tx=0).stop 2024-02-02T15:24:23.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:23.189+0000 7fb710000700 1 -- 172.21.15.1:0/1743422687 shutdown_connections 2024-02-02T15:24:23.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:23.189+0000 7fb710000700 1 --2- 172.21.15.1:0/1743422687 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb708108c10 0x7fb70810b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:23.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:23.189+0000 7fb710000700 1 --2- 172.21.15.1:0/1743422687 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb708074d80 0x7fb7080731d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:23.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:23.189+0000 7fb710000700 1 --2- 172.21.15.1:0/1743422687 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb7080737a0 0x7fb708073c20 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:23.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:23.189+0000 7fb710000700 1 -- 172.21.15.1:0/1743422687 >> 172.21.15.1:0/1743422687 conn(0x7fb7080fc2c0 msgr2=0x7fb7080fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:24:23.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:23.189+0000 7fb710000700 1 -- 172.21.15.1:0/1743422687 shutdown_connections 2024-02-02T15:24:23.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:23.189+0000 7fb710000700 1 -- 172.21.15.1:0/1743422687 wait complete. 2024-02-02T15:24:23.190 INFO:journalctl@ceph.mds.j.smithi001.stdout:Feb 02 15:24:22 smithi001 ceph-mds[83417]: -- 172.21.15.1:0/1225355547 <== mon.0 v2:172.21.15.1:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5582cec631e0 con 0x5582cec3a800 2024-02-02T15:24:23.190 INFO:journalctl@ceph.mds.j.smithi001.stdout:Feb 02 15:24:22 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mds-j[83394]: starting mds.j at 2024-02-02T15:24:23.190 INFO:journalctl@ceph.mds.j.smithi001.stdout:Feb 02 15:24:22 smithi001 bash[83379]: 32b061251d8cb75ecee5cda36f5e8dff13d793809d2b955f00f1f752626ca24e 2024-02-02T15:24:23.190 INFO:journalctl@ceph.mds.j.smithi001.stdout:Feb 02 15:24:22 smithi001 systemd[1]: Started Ceph mds.j for 371a409c-c1de-11ee-95b6-87774f69a715. 2024-02-02T15:24:23.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:22 smithi001 ceph-mon[35470]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-02T15:24:23.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:23.190+0000 7fb710000700 1 Processor -- start 2024-02-02T15:24:23.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:23.190+0000 7fb710000700 1 -- start start 2024-02-02T15:24:23.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:23.191+0000 7fb710000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7080737a0 0x7fb708198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:23.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:23.191+0000 7fb710000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb708074d80 0x7fb7081991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:23.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:23.191+0000 7fb710000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb708108c10 0x7fb7081a0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:23.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:23.191+0000 7fb710000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7081a07f0 con 0x7fb708108c10 2024-02-02T15:24:23.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:23.191+0000 7fb710000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7081a0960 con 0x7fb708074d80 2024-02-02T15:24:23.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:23.191+0000 7fb710000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7081a0b20 con 0x7fb7080737a0 2024-02-02T15:24:23.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:23.191+0000 7fb70e000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb708108c10 0x7fb7081a0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:23.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:23.191+0000 7fb70d600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7080737a0 0x7fb708198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:23.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:23.191+0000 7fb70d600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7080737a0 0x7fb708198c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:40888/0 (socket says 172.21.15.1:40888) 2024-02-02T15:24:23.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:23.191+0000 7fb70d600700 1 -- 172.21.15.1:0/1238480336 learned_addr learned my addr 172.21.15.1:0/1238480336 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:24:23.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:23.191+0000 7fb70cc00700 1 --2- 172.21.15.1:0/1238480336 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb708074d80 0x7fb7081991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:23.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:23.192+0000 7fb70d600700 1 -- 172.21.15.1:0/1238480336 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb708074d80 msgr2=0x7fb7081991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:23.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:23.192+0000 7fb70d600700 1 --2- 172.21.15.1:0/1238480336 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb708074d80 0x7fb7081991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:23.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:23.192+0000 7fb70d600700 1 -- 172.21.15.1:0/1238480336 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb708108c10 msgr2=0x7fb7081a0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:23.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:23.192+0000 7fb70d600700 1 --2- 172.21.15.1:0/1238480336 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb708108c10 0x7fb7081a0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:23.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:23.192+0000 7fb70d600700 1 -- 172.21.15.1:0/1238480336 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6f00095f0 con 0x7fb7080737a0 2024-02-02T15:24:23.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:23.192+0000 7fb70e000700 1 --2- 172.21.15.1:0/1238480336 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb708108c10 0x7fb7081a0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:24:23.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:23.192+0000 7fb70d600700 1 --2- 172.21.15.1:0/1238480336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7080737a0 0x7fb708198c80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb6f800e9a0 tx=0x7fb6f800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:24:23.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:23.192+0000 7fb706000700 1 -- 172.21.15.1:0/1238480336 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f8009790 con 0x7fb7080737a0 2024-02-02T15:24:23.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:23.192+0000 7fb706000700 1 -- 172.21.15.1:0/1238480336 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6f8010430 con 0x7fb7080737a0 2024-02-02T15:24:23.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:23.192+0000 7fb710000700 1 -- 172.21.15.1:0/1238480336 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7081a0e00 con 0x7fb7080737a0 2024-02-02T15:24:23.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:23.192+0000 7fb706000700 1 -- 172.21.15.1:0/1238480336 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f8018680 con 0x7fb7080737a0 2024-02-02T15:24:23.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:23.192+0000 7fb710000700 1 -- 172.21.15.1:0/1238480336 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7081a1280 con 0x7fb7080737a0 2024-02-02T15:24:23.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:23.193+0000 7fb710000700 1 -- 172.21.15.1:0/1238480336 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7081072c0 con 0x7fb7080737a0 2024-02-02T15:24:23.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:23.195+0000 7fb706000700 1 -- 172.21.15.1:0/1238480336 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 16) v1 ==== 135069+0+0 (secure 0 0 0) 0x7fb6f80105a0 con 0x7fb7080737a0 2024-02-02T15:24:23.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:23.196+0000 7fb706000700 1 --2- 172.21.15.1:0/1238480336 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fb6e80a05a0 0x7fb6e80a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:23.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:23.197+0000 7fb706000700 1 -- 172.21.15.1:0/1238480336 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fb6f8014070 con 0x7fb7080737a0 2024-02-02T15:24:23.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:23.197+0000 7fb70cc00700 1 --2- 172.21.15.1:0/1238480336 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fb6e80a05a0 0x7fb6e80a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:23.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:23.197+0000 7fb70cc00700 1 --2- 172.21.15.1:0/1238480336 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fb6e80a05a0 0x7fb6e80a2a60 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb6f000b2e0 tx=0x7fb6f0005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:24:23.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:23.200+0000 7fb706000700 1 -- 172.21.15.1:0/1238480336 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7fb6f8052110 con 0x7fb7080737a0 2024-02-02T15:24:23.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:23.471+0000 7fb710000700 1 -- 172.21.15.1:0/1238480336 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb70802cc70 con 0x7fb7080737a0 2024-02-02T15:24:23.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:23.472+0000 7fb706000700 1 -- 172.21.15.1:0/1238480336 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2008 (secure 0 0 0) 0x7fb6f80956a0 con 0x7fb7080737a0 2024-02-02T15:24:23.473 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:24:23.473 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/7648986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":7648986},{"type":"v1","addr":"172.21.15.142:6833","nonce":7648986}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/3482746094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3482746094},{"type":"v1","addr":"172.21.15.71:6833","nonce":3482746094}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2024-02-02T15:24:23.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:23.475+0000 7fb710000700 1 -- 172.21.15.1:0/1238480336 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fb6e80a05a0 msgr2=0x7fb6e80a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:23.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:23.475+0000 7fb710000700 1 --2- 172.21.15.1:0/1238480336 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fb6e80a05a0 0x7fb6e80a2a60 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb6f000b2e0 tx=0x7fb6f0005fb0 comp rx=0 tx=0).stop 2024-02-02T15:24:23.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:23.475+0000 7fb710000700 1 -- 172.21.15.1:0/1238480336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7080737a0 msgr2=0x7fb708198c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:23.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:23.475+0000 7fb710000700 1 --2- 172.21.15.1:0/1238480336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7080737a0 0x7fb708198c80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb6f800e9a0 tx=0x7fb6f800ecb0 comp rx=0 tx=0).stop 2024-02-02T15:24:23.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:23.475+0000 7fb710000700 1 -- 172.21.15.1:0/1238480336 shutdown_connections 2024-02-02T15:24:23.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:23.475+0000 7fb710000700 1 --2- 172.21.15.1:0/1238480336 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb708108c10 0x7fb7081a0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:23.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:23.475+0000 7fb710000700 1 --2- 172.21.15.1:0/1238480336 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fb6e80a05a0 0x7fb6e80a2a60 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:23.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:23.475+0000 7fb710000700 1 --2- 172.21.15.1:0/1238480336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7080737a0 0x7fb708198c80 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:23.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:23.475+0000 7fb710000700 1 --2- 172.21.15.1:0/1238480336 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb708074d80 0x7fb7081991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:23.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:23.475+0000 7fb710000700 1 -- 172.21.15.1:0/1238480336 >> 172.21.15.1:0/1238480336 conn(0x7fb7080fc2c0 msgr2=0x7fb7080fe6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:24:23.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:23.476+0000 7fb710000700 1 -- 172.21.15.1:0/1238480336 shutdown_connections 2024-02-02T15:24:23.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:23.476+0000 7fb710000700 1 -- 172.21.15.1:0/1238480336 wait complete. 2024-02-02T15:24:23.477 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 3 2024-02-02T15:24:24.135 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:23 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:24.135 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:23 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:24.135 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:23 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:24.135 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:23 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-02T15:24:24.135 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:23 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-02T15:24:24.135 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:23 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:24:24.135 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:23 smithi001 ceph-mon[35470]: Deploying daemon mds.d on smithi001 2024-02-02T15:24:24.135 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:23 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/1238480336' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:24:24.135 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:23 smithi001 ceph-mon[35470]: mds.? [v2:172.21.15.1:6834/2922240448,v1:172.21.15.1:6835/2922240448] up:boot 2024-02-02T15:24:24.136 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:23 smithi001 ceph-mon[35470]: fsmap 3 up:standby 2024-02-02T15:24:24.136 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:23 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-02-02T15:24:24.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:23 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:24.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:23 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:24.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:23 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:24.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:23 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-02T15:24:24.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:23 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-02T15:24:24.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:23 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:24:24.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:23 smithi071 ceph-mon[44707]: Deploying daemon mds.d on smithi001 2024-02-02T15:24:24.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:23 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/1238480336' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:24:24.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:23 smithi071 ceph-mon[44707]: mds.? [v2:172.21.15.1:6834/2922240448,v1:172.21.15.1:6835/2922240448] up:boot 2024-02-02T15:24:24.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:23 smithi071 ceph-mon[44707]: fsmap 3 up:standby 2024-02-02T15:24:24.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:23 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-02-02T15:24:24.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:23 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:24.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:23 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:24.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:23 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:24.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:23 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-02T15:24:24.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:23 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-02T15:24:24.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:23 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:24:24.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:23 smithi142 ceph-mon[45728]: Deploying daemon mds.d on smithi001 2024-02-02T15:24:24.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:23 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/1238480336' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:24:24.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:23 smithi142 ceph-mon[45728]: mds.? [v2:172.21.15.1:6834/2922240448,v1:172.21.15.1:6835/2922240448] up:boot 2024-02-02T15:24:24.421 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:23 smithi142 ceph-mon[45728]: fsmap 3 up:standby 2024-02-02T15:24:24.421 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:23 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-02-02T15:24:24.940 INFO:journalctl@ceph.mds.d.smithi001.stdout:Feb 02 15:24:24 smithi001 systemd[1]: Starting Ceph mds.d for 371a409c-c1de-11ee-95b6-87774f69a715... 2024-02-02T15:24:25.366 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:24 smithi071 ceph-mon[44707]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-02T15:24:25.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:24 smithi142 ceph-mon[45728]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-02T15:24:25.424 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:24 smithi001 ceph-mon[35470]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-02T15:24:25.497 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T15:24:25.690 INFO:journalctl@ceph.mds.d.smithi001.stdout:Feb 02 15:24:25 smithi001 ceph-mds[83804]: -- 172.21.15.1:0/3756999989 <== mon.0 v2:172.21.15.1:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5627075791e0 con 0x562707550c00 2024-02-02T15:24:25.690 INFO:journalctl@ceph.mds.d.smithi001.stdout:Feb 02 15:24:25 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mds-d[83781]: starting mds.d at 2024-02-02T15:24:25.690 INFO:journalctl@ceph.mds.d.smithi001.stdout:Feb 02 15:24:25 smithi001 bash[83766]: eb4b8787cfb667704f42d46154a1e5299d8b41d850a0cec63d12d5d891587974 2024-02-02T15:24:25.690 INFO:journalctl@ceph.mds.d.smithi001.stdout:Feb 02 15:24:25 smithi001 systemd[1]: Started Ceph mds.d for 371a409c-c1de-11ee-95b6-87774f69a715. 2024-02-02T15:24:25.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:25.707+0000 7f2a82c00700 1 -- 172.21.15.1:0/3871768705 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a7c101860 msgr2=0x7f2a7c103c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:25.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:25.707+0000 7f2a82c00700 1 --2- 172.21.15.1:0/3871768705 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a7c101860 0x7f2a7c103c50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c009990 tx=0x7f2a6c009ca0 comp rx=0 tx=0).stop 2024-02-02T15:24:25.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:25.707+0000 7f2a82c00700 1 -- 172.21.15.1:0/3871768705 shutdown_connections 2024-02-02T15:24:25.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:25.707+0000 7f2a82c00700 1 --2- 172.21.15.1:0/3871768705 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a7c106ac0 0x7f2a7c108eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:25.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:25.707+0000 7f2a82c00700 1 --2- 172.21.15.1:0/3871768705 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a7c101860 0x7f2a7c103c50 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:25.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:25.707+0000 7f2a82c00700 1 --2- 172.21.15.1:0/3871768705 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a7c104190 0x7f2a7c106580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:25.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:25.707+0000 7f2a82c00700 1 -- 172.21.15.1:0/3871768705 >> 172.21.15.1:0/3871768705 conn(0x7f2a7c0fb170 msgr2=0x7f2a7c0fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:24:25.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:25.707+0000 7f2a82c00700 1 -- 172.21.15.1:0/3871768705 shutdown_connections 2024-02-02T15:24:25.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:25.707+0000 7f2a82c00700 1 -- 172.21.15.1:0/3871768705 wait complete. 2024-02-02T15:24:25.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:25.709+0000 7f2a82c00700 1 Processor -- start 2024-02-02T15:24:25.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:25.709+0000 7f2a82c00700 1 -- start start 2024-02-02T15:24:25.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:25.709+0000 7f2a82c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a7c101860 0x7f2a7c198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:25.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:25.709+0000 7f2a82c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a7c104190 0x7f2a7c199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:25.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:25.709+0000 7f2a82c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a7c106ac0 0x7f2a7c1a01c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:25.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:25.709+0000 7f2a82c00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a7c1a0790 con 0x7f2a7c106ac0 2024-02-02T15:24:25.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:25.709+0000 7f2a82c00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a7c1a0900 con 0x7f2a7c104190 2024-02-02T15:24:25.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:25.709+0000 7f2a82c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a7c1a0ac0 con 0x7f2a7c101860 2024-02-02T15:24:25.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:25.709+0000 7f2a73e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a7c104190 0x7f2a7c199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:25.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:25.709+0000 7f2a73e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a7c104190 0x7f2a7c199190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:33614/0 (socket says 172.21.15.1:33614) 2024-02-02T15:24:25.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:25.709+0000 7f2a73e00700 1 -- 172.21.15.1:0/1634941540 learned_addr learned my addr 172.21.15.1:0/1634941540 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:24:25.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:25.709+0000 7f2a80c00700 1 --2- 172.21.15.1:0/1634941540 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a7c106ac0 0x7f2a7c1a01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:25.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:25.710+0000 7f2a7be00700 1 --2- 172.21.15.1:0/1634941540 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a7c101860 0x7f2a7c198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:25.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:25.710+0000 7f2a73e00700 1 -- 172.21.15.1:0/1634941540 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a7c101860 msgr2=0x7f2a7c198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:25.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:25.710+0000 7f2a73e00700 1 --2- 172.21.15.1:0/1634941540 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a7c101860 0x7f2a7c198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:25.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:25.710+0000 7f2a73e00700 1 -- 172.21.15.1:0/1634941540 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a7c106ac0 msgr2=0x7f2a7c1a01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:25.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:25.710+0000 7f2a73e00700 1 --2- 172.21.15.1:0/1634941540 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a7c106ac0 0x7f2a7c1a01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:25.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:25.710+0000 7f2a73e00700 1 -- 172.21.15.1:0/1634941540 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a6c0095f0 con 0x7f2a7c104190 2024-02-02T15:24:25.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:25.710+0000 7f2a73e00700 1 --2- 172.21.15.1:0/1634941540 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a7c104190 0x7f2a7c199190 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2a6400e9a0 tx=0x7f2a6400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:24:25.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:25.710+0000 7f2a79600700 1 -- 172.21.15.1:0/1634941540 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a64009790 con 0x7f2a7c104190 2024-02-02T15:24:25.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:25.710+0000 7f2a79600700 1 -- 172.21.15.1:0/1634941540 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a64010430 con 0x7f2a7c104190 2024-02-02T15:24:25.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:25.710+0000 7f2a79600700 1 -- 172.21.15.1:0/1634941540 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a64018680 con 0x7f2a7c104190 2024-02-02T15:24:25.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:25.710+0000 7f2a82c00700 1 -- 172.21.15.1:0/1634941540 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a7c1a0da0 con 0x7f2a7c104190 2024-02-02T15:24:25.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:25.711+0000 7f2a82c00700 1 -- 172.21.15.1:0/1634941540 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a7c1a11a0 con 0x7f2a7c104190 2024-02-02T15:24:25.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:25.711+0000 7f2a82c00700 1 -- 172.21.15.1:0/1634941540 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a7c108640 con 0x7f2a7c104190 2024-02-02T15:24:25.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:25.713+0000 7f2a79600700 1 -- 172.21.15.1:0/1634941540 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 135069+0+0 (secure 0 0 0) 0x7f2a640105a0 con 0x7f2a7c104190 2024-02-02T15:24:25.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:25.715+0000 7f2a79600700 1 --2- 172.21.15.1:0/1634941540 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f2a5c0a05a0 0x7f2a5c0a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:25.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:25.715+0000 7f2a79600700 1 -- 172.21.15.1:0/1634941540 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f2a64014070 con 0x7f2a7c104190 2024-02-02T15:24:25.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:25.715+0000 7f2a7be00700 1 --2- 172.21.15.1:0/1634941540 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f2a5c0a05a0 0x7f2a5c0a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:25.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:25.715+0000 7f2a7be00700 1 --2- 172.21.15.1:0/1634941540 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f2a5c0a05a0 0x7f2a5c0a2a60 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c00b2e0 tx=0x7f2a6c005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:24:25.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:25.717+0000 7f2a79600700 1 -- 172.21.15.1:0/1634941540 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f2a64095850 con 0x7f2a7c104190 2024-02-02T15:24:25.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:25.975+0000 7f2a82c00700 1 -- 172.21.15.1:0/1634941540 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2a7c02cc70 con 0x7f2a7c104190 2024-02-02T15:24:25.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:25.975+0000 7f2a79600700 1 -- 172.21.15.1:0/1634941540 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3516 (secure 0 0 0) 0x7f2a64098e70 con 0x7f2a7c104190 2024-02-02T15:24:25.976 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:24:25.976 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2922240448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2922240448},{"type":"v1","addr":"172.21.15.1:6835","nonce":2922240448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2729497431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2729497431},{"type":"v1","addr":"172.21.15.1:6837","nonce":2729497431}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/7648986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":7648986},{"type":"v1","addr":"172.21.15.142:6833","nonce":7648986}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/3482746094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3482746094},{"type":"v1","addr":"172.21.15.71:6833","nonce":3482746094}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2024-02-02T15:24:25.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:25.978+0000 7f2a82c00700 1 -- 172.21.15.1:0/1634941540 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f2a5c0a05a0 msgr2=0x7f2a5c0a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:25.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:25.978+0000 7f2a82c00700 1 --2- 172.21.15.1:0/1634941540 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f2a5c0a05a0 0x7f2a5c0a2a60 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c00b2e0 tx=0x7f2a6c005fb0 comp rx=0 tx=0).stop 2024-02-02T15:24:25.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:25.978+0000 7f2a82c00700 1 -- 172.21.15.1:0/1634941540 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a7c104190 msgr2=0x7f2a7c199190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:25.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:25.978+0000 7f2a82c00700 1 --2- 172.21.15.1:0/1634941540 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a7c104190 0x7f2a7c199190 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2a6400e9a0 tx=0x7f2a6400ecb0 comp rx=0 tx=0).stop 2024-02-02T15:24:25.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:25.978+0000 7f2a82c00700 1 -- 172.21.15.1:0/1634941540 shutdown_connections 2024-02-02T15:24:25.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:25.978+0000 7f2a82c00700 1 --2- 172.21.15.1:0/1634941540 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a7c106ac0 0x7f2a7c1a01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:25.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:25.978+0000 7f2a82c00700 1 --2- 172.21.15.1:0/1634941540 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f2a5c0a05a0 0x7f2a5c0a2a60 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:25.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:25.978+0000 7f2a82c00700 1 --2- 172.21.15.1:0/1634941540 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a7c101860 0x7f2a7c198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:25.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:25.978+0000 7f2a82c00700 1 --2- 172.21.15.1:0/1634941540 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a7c104190 0x7f2a7c199190 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:25.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:25.978+0000 7f2a82c00700 1 -- 172.21.15.1:0/1634941540 >> 172.21.15.1:0/1634941540 conn(0x7f2a7c0fb170 msgr2=0x7f2a7c0ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:24:25.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:25.978+0000 7f2a82c00700 1 -- 172.21.15.1:0/1634941540 shutdown_connections 2024-02-02T15:24:25.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:25.978+0000 7f2a82c00700 1 -- 172.21.15.1:0/1634941540 wait complete. 2024-02-02T15:24:25.980 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 5 2024-02-02T15:24:26.563 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:26 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:26.563 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:26 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:26.563 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:26 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:26.563 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:26 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-02T15:24:26.563 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:26 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-02T15:24:26.563 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:26 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:24:26.563 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:26 smithi071 ceph-mon[44707]: Deploying daemon mds.e on smithi071 2024-02-02T15:24:26.563 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:26 smithi071 ceph-mon[44707]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-02T15:24:26.564 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:26 smithi071 ceph-mon[44707]: mds.? [v2:172.21.15.1:6836/2729497431,v1:172.21.15.1:6837/2729497431] up:boot 2024-02-02T15:24:26.564 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:26 smithi071 ceph-mon[44707]: fsmap 4 up:standby 2024-02-02T15:24:26.564 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:26 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-02-02T15:24:26.564 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:26 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/1634941540' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:24:26.864 INFO:journalctl@ceph.mds.e.smithi071.stdout:Feb 02 15:24:26 smithi071 systemd[1]: Starting Ceph mds.e for 371a409c-c1de-11ee-95b6-87774f69a715... 2024-02-02T15:24:26.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:26 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:26.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:26 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:26.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:26 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:26.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:26 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-02T15:24:26.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:26 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-02T15:24:26.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:26 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:24:26.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:26 smithi142 ceph-mon[45728]: Deploying daemon mds.e on smithi071 2024-02-02T15:24:26.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:26 smithi142 ceph-mon[45728]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-02T15:24:26.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:26 smithi142 ceph-mon[45728]: mds.? [v2:172.21.15.1:6836/2729497431,v1:172.21.15.1:6837/2729497431] up:boot 2024-02-02T15:24:26.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:26 smithi142 ceph-mon[45728]: fsmap 4 up:standby 2024-02-02T15:24:26.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:26 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-02-02T15:24:26.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:26 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/1634941540' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:24:26.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:26 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:26.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:26 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:26.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:26 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:26.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:26 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-02T15:24:26.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:26 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-02T15:24:26.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:26 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:24:26.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:26 smithi001 ceph-mon[35470]: Deploying daemon mds.e on smithi071 2024-02-02T15:24:26.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:26 smithi001 ceph-mon[35470]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-02T15:24:26.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:26 smithi001 ceph-mon[35470]: mds.? [v2:172.21.15.1:6836/2729497431,v1:172.21.15.1:6837/2729497431] up:boot 2024-02-02T15:24:26.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:26 smithi001 ceph-mon[35470]: fsmap 4 up:standby 2024-02-02T15:24:26.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:26 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-02-02T15:24:26.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:26 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/1634941540' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:24:27.364 INFO:journalctl@ceph.mds.e.smithi071.stdout:Feb 02 15:24:27 smithi071 ceph-mds[79002]: -- 172.21.15.71:0/1905758203 <== mon.2 v2:172.21.15.71:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c3777471e0 con 0x55c37771c800 2024-02-02T15:24:27.364 INFO:journalctl@ceph.mds.e.smithi071.stdout:Feb 02 15:24:27 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mds-e[78979]: starting mds.e at 2024-02-02T15:24:27.364 INFO:journalctl@ceph.mds.e.smithi071.stdout:Feb 02 15:24:27 smithi071 bash[78964]: 4ca66c9f3221385cb059d4d4ea3e5412328ea6f5d61c9600f5b5fa42e010f24b 2024-02-02T15:24:27.364 INFO:journalctl@ceph.mds.e.smithi071.stdout:Feb 02 15:24:27 smithi071 systemd[1]: Started Ceph mds.e for 371a409c-c1de-11ee-95b6-87774f69a715. 2024-02-02T15:24:27.998 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T15:24:28.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:28.208+0000 7f3212800700 1 -- 172.21.15.1:0/2093722275 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f320c101f30 msgr2=0x7f320c102390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:28.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:28.208+0000 7f3212800700 1 --2- 172.21.15.1:0/2093722275 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f320c101f30 0x7f320c102390 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f31f40099c0 tx=0x7f31f4009cd0 comp rx=0 tx=0).stop 2024-02-02T15:24:28.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:28.208+0000 7f3212800700 1 -- 172.21.15.1:0/2093722275 shutdown_connections 2024-02-02T15:24:28.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:28.208+0000 7f3212800700 1 --2- 172.21.15.1:0/2093722275 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f320c1028d0 0x7f320c109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:28.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:28.208+0000 7f3212800700 1 --2- 172.21.15.1:0/2093722275 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f320c100d40 0x7f320c101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:28.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:28.208+0000 7f3212800700 1 --2- 172.21.15.1:0/2093722275 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f320c101f30 0x7f320c102390 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:28.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:28.208+0000 7f3212800700 1 -- 172.21.15.1:0/2093722275 >> 172.21.15.1:0/2093722275 conn(0x7f320c0fc290 msgr2=0x7f320c0fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:24:28.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:28.208+0000 7f3212800700 1 -- 172.21.15.1:0/2093722275 shutdown_connections 2024-02-02T15:24:28.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:28.208+0000 7f3212800700 1 -- 172.21.15.1:0/2093722275 wait complete. 2024-02-02T15:24:28.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:28.209+0000 7f3212800700 1 Processor -- start 2024-02-02T15:24:28.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:28.209+0000 7f3212800700 1 -- start start 2024-02-02T15:24:28.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:28.210+0000 7f3212800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f320c100d40 0x7f320c198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:28.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:28.210+0000 7f3212800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f320c101f30 0x7f320c1991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:28.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:28.210+0000 7f3212800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f320c1028d0 0x7f320c1a0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:28.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:28.210+0000 7f3212800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f320c1a07f0 con 0x7f320c100d40 2024-02-02T15:24:28.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:28.210+0000 7f3212800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f320c1a0960 con 0x7f320c101f30 2024-02-02T15:24:28.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:28.210+0000 7f3212800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f320c1a0b20 con 0x7f320c1028d0 2024-02-02T15:24:28.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:28.210+0000 7f320b400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f320c100d40 0x7f320c198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:28.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:28.210+0000 7f320b400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f320c100d40 0x7f320c198c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48180/0 (socket says 172.21.15.1:48180) 2024-02-02T15:24:28.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:28.210+0000 7f320b400700 1 -- 172.21.15.1:0/4030041740 learned_addr learned my addr 172.21.15.1:0/4030041740 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:24:28.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:28.210+0000 7f320be00700 1 --2- 172.21.15.1:0/4030041740 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f320c1028d0 0x7f320c1a0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:28.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:28.210+0000 7f320aa00700 1 --2- 172.21.15.1:0/4030041740 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f320c101f30 0x7f320c1991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:28.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:28.210+0000 7f320b400700 1 -- 172.21.15.1:0/4030041740 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f320c1028d0 msgr2=0x7f320c1a0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:28.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:28.210+0000 7f320b400700 1 --2- 172.21.15.1:0/4030041740 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f320c1028d0 0x7f320c1a0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:28.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:28.210+0000 7f320b400700 1 -- 172.21.15.1:0/4030041740 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f320c101f30 msgr2=0x7f320c1991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:28.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:28.210+0000 7f320b400700 1 --2- 172.21.15.1:0/4030041740 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f320c101f30 0x7f320c1991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:28.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:28.210+0000 7f320b400700 1 -- 172.21.15.1:0/4030041740 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31f40095f0 con 0x7f320c100d40 2024-02-02T15:24:28.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:28.211+0000 7f320b400700 1 --2- 172.21.15.1:0/4030041740 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f320c100d40 0x7f320c198c80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f31fc00e970 tx=0x7f31fc00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:24:28.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:28.211+0000 7f3203e00700 1 -- 172.21.15.1:0/4030041740 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31fc009790 con 0x7f320c100d40 2024-02-02T15:24:28.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:28.211+0000 7f3203e00700 1 -- 172.21.15.1:0/4030041740 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f31fc00c7e0 con 0x7f320c100d40 2024-02-02T15:24:28.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:28.211+0000 7f3212800700 1 -- 172.21.15.1:0/4030041740 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f320c1a0e00 con 0x7f320c100d40 2024-02-02T15:24:28.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:28.211+0000 7f3203e00700 1 -- 172.21.15.1:0/4030041740 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31fc003c70 con 0x7f320c100d40 2024-02-02T15:24:28.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:28.211+0000 7f3212800700 1 -- 172.21.15.1:0/4030041740 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f320c1a1280 con 0x7f320c100d40 2024-02-02T15:24:28.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:28.212+0000 7f3212800700 1 -- 172.21.15.1:0/4030041740 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f320c193390 con 0x7f320c100d40 2024-02-02T15:24:28.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:28.214+0000 7f3203e00700 1 -- 172.21.15.1:0/4030041740 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) v1 ==== 135069+0+0 (secure 0 0 0) 0x7f31fc00cc50 con 0x7f320c100d40 2024-02-02T15:24:28.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:28.215+0000 7f3203e00700 1 --2- 172.21.15.1:0/4030041740 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f31ec0a05a0 0x7f31ec0a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:28.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:28.216+0000 7f3203e00700 1 -- 172.21.15.1:0/4030041740 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f31fc014070 con 0x7f320c100d40 2024-02-02T15:24:28.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:28.216+0000 7f320aa00700 1 --2- 172.21.15.1:0/4030041740 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f31ec0a05a0 0x7f31ec0a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:28.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:28.216+0000 7f320aa00700 1 --2- 172.21.15.1:0/4030041740 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f31ec0a05a0 0x7f31ec0a2a60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f31f400b580 tx=0x7f31f4005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:24:28.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:28.219+0000 7f3203e00700 1 -- 172.21.15.1:0/4030041740 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f31fc095670 con 0x7f320c100d40 2024-02-02T15:24:28.345 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:28 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:28.346 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:28 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:28.346 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:28 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:28.346 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:28 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-02T15:24:28.346 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:28 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-02T15:24:28.346 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:28 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:24:28.346 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:28 smithi142 ceph-mon[45728]: Deploying daemon mds.f on smithi142 2024-02-02T15:24:28.346 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:28 smithi142 ceph-mon[45728]: mds.? [v2:172.21.15.71:6834/388207814,v1:172.21.15.71:6835/388207814] up:boot 2024-02-02T15:24:28.346 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:28 smithi142 ceph-mon[45728]: fsmap 5 up:standby 2024-02-02T15:24:28.346 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:28 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-02-02T15:24:28.346 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:28 smithi142 ceph-mon[45728]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-02T15:24:28.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:28 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:28.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:28 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:28.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:28 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:28.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:28 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-02T15:24:28.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:28 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-02T15:24:28.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:28 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:24:28.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:28 smithi001 ceph-mon[35470]: Deploying daemon mds.f on smithi142 2024-02-02T15:24:28.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:28 smithi001 ceph-mon[35470]: mds.? [v2:172.21.15.71:6834/388207814,v1:172.21.15.71:6835/388207814] up:boot 2024-02-02T15:24:28.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:28 smithi001 ceph-mon[35470]: fsmap 5 up:standby 2024-02-02T15:24:28.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:28 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-02-02T15:24:28.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:28 smithi001 ceph-mon[35470]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-02T15:24:28.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:28.480+0000 7f3212800700 1 -- 172.21.15.1:0/4030041740 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f320c02cc70 con 0x7f320c100d40 2024-02-02T15:24:28.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:28.480+0000 7f3203e00700 1 -- 172.21.15.1:0/4030041740 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4270 (secure 0 0 0) 0x7f31fc098c90 con 0x7f320c100d40 2024-02-02T15:24:28.481 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:24:28.481 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2922240448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2922240448},{"type":"v1","addr":"172.21.15.1:6835","nonce":2922240448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2729497431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2729497431},{"type":"v1","addr":"172.21.15.1:6837","nonce":2729497431}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/7648986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":7648986},{"type":"v1","addr":"172.21.15.142:6833","nonce":7648986}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/3482746094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3482746094},{"type":"v1","addr":"172.21.15.71:6833","nonce":3482746094}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/388207814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":388207814},{"type":"v1","addr":"172.21.15.71:6835","nonce":388207814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[]} 2024-02-02T15:24:28.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:28.483+0000 7f3212800700 1 -- 172.21.15.1:0/4030041740 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f31ec0a05a0 msgr2=0x7f31ec0a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:28.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:28.483+0000 7f3212800700 1 --2- 172.21.15.1:0/4030041740 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f31ec0a05a0 0x7f31ec0a2a60 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f31f400b580 tx=0x7f31f4005fb0 comp rx=0 tx=0).stop 2024-02-02T15:24:28.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:28.483+0000 7f3212800700 1 -- 172.21.15.1:0/4030041740 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f320c100d40 msgr2=0x7f320c198c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:28.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:28.483+0000 7f3212800700 1 --2- 172.21.15.1:0/4030041740 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f320c100d40 0x7f320c198c80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f31fc00e970 tx=0x7f31fc00ec80 comp rx=0 tx=0).stop 2024-02-02T15:24:28.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:28.483+0000 7f3212800700 1 -- 172.21.15.1:0/4030041740 shutdown_connections 2024-02-02T15:24:28.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:28.483+0000 7f3212800700 1 --2- 172.21.15.1:0/4030041740 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f31ec0a05a0 0x7f31ec0a2a60 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:28.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:28.483+0000 7f3212800700 1 --2- 172.21.15.1:0/4030041740 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f320c1028d0 0x7f320c1a0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:28.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:28.483+0000 7f3212800700 1 --2- 172.21.15.1:0/4030041740 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f320c100d40 0x7f320c198c80 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:28.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:28.483+0000 7f3212800700 1 --2- 172.21.15.1:0/4030041740 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f320c101f30 0x7f320c1991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:28.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:28.483+0000 7f3212800700 1 -- 172.21.15.1:0/4030041740 >> 172.21.15.1:0/4030041740 conn(0x7f320c0fc290 msgr2=0x7f320c107870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:24:28.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:28.483+0000 7f3212800700 1 -- 172.21.15.1:0/4030041740 shutdown_connections 2024-02-02T15:24:28.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:28.483+0000 7f3212800700 1 -- 172.21.15.1:0/4030041740 wait complete. 2024-02-02T15:24:28.485 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 6 2024-02-02T15:24:28.595 INFO:journalctl@ceph.mds.f.smithi142.stdout:Feb 02 15:24:28 smithi142 systemd[1]: Starting Ceph mds.f for 371a409c-c1de-11ee-95b6-87774f69a715... 2024-02-02T15:24:28.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:28 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:28.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:28 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:28.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:28 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:28.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:28 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-02T15:24:28.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:28 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-02T15:24:28.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:28 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:24:28.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:28 smithi071 ceph-mon[44707]: Deploying daemon mds.f on smithi142 2024-02-02T15:24:28.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:28 smithi071 ceph-mon[44707]: mds.? [v2:172.21.15.71:6834/388207814,v1:172.21.15.71:6835/388207814] up:boot 2024-02-02T15:24:28.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:28 smithi071 ceph-mon[44707]: fsmap 5 up:standby 2024-02-02T15:24:28.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:28 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-02-02T15:24:28.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:28 smithi071 ceph-mon[44707]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-02T15:24:29.170 INFO:journalctl@ceph.mds.f.smithi142.stdout:Feb 02 15:24:29 smithi142 ceph-mds[82539]: -- 172.21.15.142:0/926252827 <== mon.1 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x557a12fd11e0 con 0x557a12fa7000 2024-02-02T15:24:29.170 INFO:journalctl@ceph.mds.f.smithi142.stdout:Feb 02 15:24:29 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mds-f[82516]: starting mds.f at 2024-02-02T15:24:29.170 INFO:journalctl@ceph.mds.f.smithi142.stdout:Feb 02 15:24:29 smithi142 bash[82501]: 8ef5a11decce61dfe800be78d3172468cb8d39a1ec6f78214e02acc2c2467e88 2024-02-02T15:24:29.170 INFO:journalctl@ceph.mds.f.smithi142.stdout:Feb 02 15:24:29 smithi142 systemd[1]: Started Ceph mds.f for 371a409c-c1de-11ee-95b6-87774f69a715. 2024-02-02T15:24:29.535 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:29 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/4030041740' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:24:29.535 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:29 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:29.535 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:29 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:29.535 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:29 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:29.535 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:29 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-02T15:24:29.535 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:29 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-02T15:24:29.535 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:29 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:24:29.535 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:29 smithi142 ceph-mon[45728]: Deploying daemon mds.c on smithi142 2024-02-02T15:24:29.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:29 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/4030041740' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:24:29.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:29 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:29.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:29 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:29.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:29 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:29.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:29 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-02T15:24:29.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:29 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-02T15:24:29.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:29 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:24:29.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:29 smithi071 ceph-mon[44707]: Deploying daemon mds.c on smithi142 2024-02-02T15:24:29.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:29 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/4030041740' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:24:29.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:29 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:29.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:29 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:29.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:29 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:29.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:29 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-02T15:24:29.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:29 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-02T15:24:29.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:29 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:24:29.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:29 smithi001 ceph-mon[35470]: Deploying daemon mds.c on smithi142 2024-02-02T15:24:30.432 INFO:journalctl@ceph.mds.c.smithi142.stdout:Feb 02 15:24:30 smithi142 systemd[1]: Starting Ceph mds.c for 371a409c-c1de-11ee-95b6-87774f69a715... 2024-02-02T15:24:30.503 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T15:24:30.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:30.720+0000 7f3f92600700 1 -- 172.21.15.1:0/2332427155 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3f8c108c00 msgr2=0x7f3f8c10b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:30.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:30.720+0000 7f3f92600700 1 --2- 172.21.15.1:0/2332427155 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3f8c108c00 0x7f3f8c10b090 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3f8400b0a0 tx=0x7f3f8400b3b0 comp rx=0 tx=0).stop 2024-02-02T15:24:30.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:30.720+0000 7f3f92600700 1 -- 172.21.15.1:0/2332427155 shutdown_connections 2024-02-02T15:24:30.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:30.720+0000 7f3f92600700 1 --2- 172.21.15.1:0/2332427155 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3f8c108c00 0x7f3f8c10b090 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:30.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:30.720+0000 7f3f92600700 1 --2- 172.21.15.1:0/2332427155 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f8c074d80 0x7f3f8c0731d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:30.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:30.720+0000 7f3f92600700 1 --2- 172.21.15.1:0/2332427155 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f8c0737a0 0x7f3f8c073c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:30.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:30.720+0000 7f3f92600700 1 -- 172.21.15.1:0/2332427155 >> 172.21.15.1:0/2332427155 conn(0x7f3f8c0fc290 msgr2=0x7f3f8c0fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:24:30.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:30.720+0000 7f3f92600700 1 -- 172.21.15.1:0/2332427155 shutdown_connections 2024-02-02T15:24:30.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:30.720+0000 7f3f92600700 1 -- 172.21.15.1:0/2332427155 wait complete. 2024-02-02T15:24:30.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:30.721+0000 7f3f92600700 1 Processor -- start 2024-02-02T15:24:30.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:30.722+0000 7f3f92600700 1 -- start start 2024-02-02T15:24:30.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:30.722+0000 7f3f92600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3f8c0737a0 0x7f3f8c198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:30.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:30.722+0000 7f3f92600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f8c074d80 0x7f3f8c199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:30.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:30.722+0000 7f3f92600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f8c108c00 0x7f3f8c1a02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:30.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:30.722+0000 7f3f92600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f8c1a08c0 con 0x7f3f8c0737a0 2024-02-02T15:24:30.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:30.722+0000 7f3f92600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f8c1a0a00 con 0x7f3f8c108c00 2024-02-02T15:24:30.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:30.722+0000 7f3f92600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f8c1a0b90 con 0x7f3f8c074d80 2024-02-02T15:24:30.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:30.722+0000 7f3f8b400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3f8c0737a0 0x7f3f8c198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:30.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:30.722+0000 7f3f8b400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3f8c0737a0 0x7f3f8c198d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48200/0 (socket says 172.21.15.1:48200) 2024-02-02T15:24:30.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:30.722+0000 7f3f8b400700 1 -- 172.21.15.1:0/1371299654 learned_addr learned my addr 172.21.15.1:0/1371299654 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:24:30.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:30.722+0000 7f3f8aa00700 1 --2- 172.21.15.1:0/1371299654 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f8c074d80 0x7f3f8c199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:30.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:30.722+0000 7f3f8be00700 1 --2- 172.21.15.1:0/1371299654 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f8c108c00 0x7f3f8c1a02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:30.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:30.723+0000 7f3f8b400700 1 -- 172.21.15.1:0/1371299654 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f8c074d80 msgr2=0x7f3f8c199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:30.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:30.723+0000 7f3f8b400700 1 --2- 172.21.15.1:0/1371299654 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f8c074d80 0x7f3f8c199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:30.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:30.723+0000 7f3f8b400700 1 -- 172.21.15.1:0/1371299654 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f8c108c00 msgr2=0x7f3f8c1a02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:30.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:30.723+0000 7f3f8b400700 1 --2- 172.21.15.1:0/1371299654 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f8c108c00 0x7f3f8c1a02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:30.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:30.723+0000 7f3f8b400700 1 -- 172.21.15.1:0/1371299654 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f84009d00 con 0x7f3f8c0737a0 2024-02-02T15:24:30.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:30.723+0000 7f3f8aa00700 1 --2- 172.21.15.1:0/1371299654 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f8c074d80 0x7f3f8c199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:24:30.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:30.723+0000 7f3f8b400700 1 --2- 172.21.15.1:0/1371299654 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3f8c0737a0 0x7f3f8c198d40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c00d950 tx=0x7f3f7c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:24:30.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:30.723+0000 7f3f83e00700 1 -- 172.21.15.1:0/1371299654 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f7c00f890 con 0x7f3f8c0737a0 2024-02-02T15:24:30.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:30.723+0000 7f3f83e00700 1 -- 172.21.15.1:0/1371299654 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3f7c010460 con 0x7f3f8c0737a0 2024-02-02T15:24:30.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:30.723+0000 7f3f92600700 1 -- 172.21.15.1:0/1371299654 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f8c1a0e70 con 0x7f3f8c0737a0 2024-02-02T15:24:30.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:30.723+0000 7f3f83e00700 1 -- 172.21.15.1:0/1371299654 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f7c00e610 con 0x7f3f8c0737a0 2024-02-02T15:24:30.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:30.723+0000 7f3f92600700 1 -- 172.21.15.1:0/1371299654 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f8c1a1290 con 0x7f3f8c0737a0 2024-02-02T15:24:30.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:30.724+0000 7f3f92600700 1 -- 172.21.15.1:0/1371299654 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f8c1072b0 con 0x7f3f8c0737a0 2024-02-02T15:24:30.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:30.726+0000 7f3f83e00700 1 -- 172.21.15.1:0/1371299654 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) v1 ==== 135069+0+0 (secure 0 0 0) 0x7f3f7c0108d0 con 0x7f3f8c0737a0 2024-02-02T15:24:30.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:30.727+0000 7f3f83e00700 1 --2- 172.21.15.1:0/1371299654 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f3f6c0a0520 0x7f3f6c0a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:30.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:30.728+0000 7f3f83e00700 1 -- 172.21.15.1:0/1371299654 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f3f7c0cb270 con 0x7f3f8c0737a0 2024-02-02T15:24:30.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:30.728+0000 7f3f8aa00700 1 --2- 172.21.15.1:0/1371299654 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f3f6c0a0520 0x7f3f6c0a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:30.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:30.728+0000 7f3f8aa00700 1 --2- 172.21.15.1:0/1371299654 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f3f6c0a0520 0x7f3f6c0a29e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c074aa0 tx=0x7f3f74009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:24:30.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:30.732+0000 7f3f83e00700 1 -- 172.21.15.1:0/1371299654 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f3f7c0952b0 con 0x7f3f8c0737a0 2024-02-02T15:24:30.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:30 smithi001 ceph-mon[35470]: mds.? [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] up:boot 2024-02-02T15:24:30.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:30 smithi001 ceph-mon[35470]: fsmap 6 up:standby 2024-02-02T15:24:30.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:30 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-02-02T15:24:30.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:30 smithi001 ceph-mon[35470]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-02T15:24:31.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:31.001+0000 7f3f92600700 1 -- 172.21.15.1:0/1371299654 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3f8c02d090 con 0x7f3f8c0737a0 2024-02-02T15:24:31.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:31.002+0000 7f3f83e00700 1 -- 172.21.15.1:0/1371299654 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5030 (secure 0 0 0) 0x7f3f7c0988d0 con 0x7f3f8c0737a0 2024-02-02T15:24:31.002 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:24:31.002 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2922240448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2922240448},{"type":"v1","addr":"172.21.15.1:6835","nonce":2922240448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2729497431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2729497431},{"type":"v1","addr":"172.21.15.1:6837","nonce":2729497431}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/7648986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":7648986},{"type":"v1","addr":"172.21.15.142:6833","nonce":7648986}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/3482746094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3482746094},{"type":"v1","addr":"172.21.15.71:6833","nonce":3482746094}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/388207814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":388207814},{"type":"v1","addr":"172.21.15.71:6835","nonce":388207814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24502,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/1052606547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1052606547},{"type":"v1","addr":"172.21.15.142:6835","nonce":1052606547}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7}],"filesystems":[]} 2024-02-02T15:24:31.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:31.005+0000 7f3f92600700 1 -- 172.21.15.1:0/1371299654 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f3f6c0a0520 msgr2=0x7f3f6c0a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:31.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:31.005+0000 7f3f92600700 1 --2- 172.21.15.1:0/1371299654 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f3f6c0a0520 0x7f3f6c0a29e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c074aa0 tx=0x7f3f74009260 comp rx=0 tx=0).stop 2024-02-02T15:24:31.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:31.005+0000 7f3f92600700 1 -- 172.21.15.1:0/1371299654 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3f8c0737a0 msgr2=0x7f3f8c198d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:31.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:31.005+0000 7f3f92600700 1 --2- 172.21.15.1:0/1371299654 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3f8c0737a0 0x7f3f8c198d40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c00d950 tx=0x7f3f7c00dc60 comp rx=0 tx=0).stop 2024-02-02T15:24:31.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:31.005+0000 7f3f92600700 1 -- 172.21.15.1:0/1371299654 shutdown_connections 2024-02-02T15:24:31.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:31.005+0000 7f3f92600700 1 --2- 172.21.15.1:0/1371299654 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3f8c108c00 0x7f3f8c1a02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:31.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:31.005+0000 7f3f92600700 1 --2- 172.21.15.1:0/1371299654 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3f8c0737a0 0x7f3f8c198d40 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:31.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:31.005+0000 7f3f92600700 1 --2- 172.21.15.1:0/1371299654 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f3f6c0a0520 0x7f3f6c0a29e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:31.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:31.005+0000 7f3f92600700 1 --2- 172.21.15.1:0/1371299654 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3f8c074d80 0x7f3f8c199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:31.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:31.005+0000 7f3f92600700 1 -- 172.21.15.1:0/1371299654 >> 172.21.15.1:0/1371299654 conn(0x7f3f8c0fc290 msgr2=0x7f3f8c0fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:24:31.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:31.005+0000 7f3f92600700 1 -- 172.21.15.1:0/1371299654 shutdown_connections 2024-02-02T15:24:31.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:31.005+0000 7f3f92600700 1 -- 172.21.15.1:0/1371299654 wait complete. 2024-02-02T15:24:31.006 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 7 2024-02-02T15:24:31.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:30 smithi071 ceph-mon[44707]: mds.? [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] up:boot 2024-02-02T15:24:31.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:30 smithi071 ceph-mon[44707]: fsmap 6 up:standby 2024-02-02T15:24:31.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:30 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-02-02T15:24:31.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:30 smithi071 ceph-mon[44707]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-02T15:24:31.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:30 smithi142 ceph-mon[45728]: mds.? [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] up:boot 2024-02-02T15:24:31.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:30 smithi142 ceph-mon[45728]: fsmap 6 up:standby 2024-02-02T15:24:31.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:30 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-02-02T15:24:31.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:30 smithi142 ceph-mon[45728]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-02T15:24:31.170 INFO:journalctl@ceph.mds.c.smithi142.stdout:Feb 02 15:24:30 smithi142 ceph-mds[82880]: -- 172.21.15.142:0/1796808372 <== mon.1 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5649a3deb1e0 con 0x5649a3dc1000 2024-02-02T15:24:31.170 INFO:journalctl@ceph.mds.c.smithi142.stdout:Feb 02 15:24:30 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mds-c[82857]: starting mds.c at 2024-02-02T15:24:31.170 INFO:journalctl@ceph.mds.c.smithi142.stdout:Feb 02 15:24:30 smithi142 bash[82842]: cde6423bf9d3ea9026af635d7a89f9d6cbe8e49036b9cd5817faa94db8ad9490 2024-02-02T15:24:31.171 INFO:journalctl@ceph.mds.c.smithi142.stdout:Feb 02 15:24:30 smithi142 systemd[1]: Started Ceph mds.c for 371a409c-c1de-11ee-95b6-87774f69a715. 2024-02-02T15:24:31.942 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:31 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:31.942 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:31 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:31.942 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:31 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:31.942 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:31 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-02T15:24:31.942 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:31 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-02T15:24:31.942 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:31 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:24:31.942 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:31 smithi071 ceph-mon[44707]: Deploying daemon mds.h on smithi071 2024-02-02T15:24:31.942 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:31 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/1371299654' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:24:31.942 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:31 smithi071 ceph-mon[44707]: mds.? [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] up:boot 2024-02-02T15:24:31.943 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:31 smithi071 ceph-mon[44707]: fsmap 7 up:standby 2024-02-02T15:24:31.943 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:31 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-02-02T15:24:32.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:31 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:32.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:31 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:32.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:31 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:32.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:31 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-02T15:24:32.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:31 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-02T15:24:32.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:31 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:24:32.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:31 smithi001 ceph-mon[35470]: Deploying daemon mds.h on smithi071 2024-02-02T15:24:32.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:31 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/1371299654' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:24:32.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:31 smithi001 ceph-mon[35470]: mds.? [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] up:boot 2024-02-02T15:24:32.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:31 smithi001 ceph-mon[35470]: fsmap 7 up:standby 2024-02-02T15:24:32.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:31 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-02-02T15:24:32.196 INFO:journalctl@ceph.mds.h.smithi071.stdout:Feb 02 15:24:32 smithi071 systemd[1]: Starting Ceph mds.h for 371a409c-c1de-11ee-95b6-87774f69a715... 2024-02-02T15:24:32.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:31 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:32.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:31 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:32.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:31 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:32.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:31 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-02T15:24:32.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:31 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-02T15:24:32.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:31 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:24:32.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:31 smithi142 ceph-mon[45728]: Deploying daemon mds.h on smithi071 2024-02-02T15:24:32.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:31 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/1371299654' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:24:32.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:31 smithi142 ceph-mon[45728]: mds.? [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] up:boot 2024-02-02T15:24:32.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:31 smithi142 ceph-mon[45728]: fsmap 7 up:standby 2024-02-02T15:24:32.421 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:31 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-02-02T15:24:32.864 INFO:journalctl@ceph.mds.h.smithi071.stdout:Feb 02 15:24:32 smithi071 ceph-mds[79343]: -- 172.21.15.71:0/3289894717 <== mon.2 v2:172.21.15.71:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a0d36971e0 con 0x55a0d366d000 2024-02-02T15:24:32.864 INFO:journalctl@ceph.mds.h.smithi071.stdout:Feb 02 15:24:32 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mds-h[79320]: starting mds.h at 2024-02-02T15:24:32.864 INFO:journalctl@ceph.mds.h.smithi071.stdout:Feb 02 15:24:32 smithi071 bash[79305]: 4898fcbe16ed45ef92194bbc439a1cf3955139ea65d6cd3c5fb6a39df461b701 2024-02-02T15:24:32.864 INFO:journalctl@ceph.mds.h.smithi071.stdout:Feb 02 15:24:32 smithi071 systemd[1]: Started Ceph mds.h for 371a409c-c1de-11ee-95b6-87774f69a715. 2024-02-02T15:24:33.025 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T15:24:33.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:33.235+0000 7febb2400700 1 -- 172.21.15.1:0/3055805993 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febac1028e0 msgr2=0x7febac109190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:33.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:33.235+0000 7febb2400700 1 --2- 172.21.15.1:0/3055805993 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febac1028e0 0x7febac109190 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7feba4009fc0 tx=0x7feba400b330 comp rx=0 tx=0).stop 2024-02-02T15:24:33.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:33.235+0000 7febb2400700 1 -- 172.21.15.1:0/3055805993 shutdown_connections 2024-02-02T15:24:33.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:33.235+0000 7febb2400700 1 --2- 172.21.15.1:0/3055805993 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febac1028e0 0x7febac109190 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:33.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:33.235+0000 7febb2400700 1 --2- 172.21.15.1:0/3055805993 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7febac100d50 0x7febac101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:33.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:33.235+0000 7febb2400700 1 --2- 172.21.15.1:0/3055805993 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7febac101f40 0x7febac1023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:33.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:33.235+0000 7febb2400700 1 -- 172.21.15.1:0/3055805993 >> 172.21.15.1:0/3055805993 conn(0x7febac0fc2c0 msgr2=0x7febac0fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:24:33.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:33.235+0000 7febb2400700 1 -- 172.21.15.1:0/3055805993 shutdown_connections 2024-02-02T15:24:33.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:33.235+0000 7febb2400700 1 -- 172.21.15.1:0/3055805993 wait complete. 2024-02-02T15:24:33.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:33.236+0000 7febb2400700 1 Processor -- start 2024-02-02T15:24:33.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:33.236+0000 7febb2400700 1 -- start start 2024-02-02T15:24:33.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:33.236+0000 7febb2400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febac100d50 0x7febac198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:33.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:33.236+0000 7febb2400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7febac101f40 0x7febac199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:33.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:33.236+0000 7febb2400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7febac1028e0 0x7febac1a02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:33.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:33.236+0000 7febb2400700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febac1a08c0 con 0x7febac101f40 2024-02-02T15:24:33.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:33.236+0000 7febb2400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febac1a0a00 con 0x7febac100d50 2024-02-02T15:24:33.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:33.236+0000 7febb2400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febac1a0b90 con 0x7febac1028e0 2024-02-02T15:24:33.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:33.237+0000 7febaaa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7febac101f40 0x7febac199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:33.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:33.237+0000 7febaaa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7febac101f40 0x7febac199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58956/0 (socket says 172.21.15.1:58956) 2024-02-02T15:24:33.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:33.237+0000 7febaaa00700 1 -- 172.21.15.1:0/3782915049 learned_addr learned my addr 172.21.15.1:0/3782915049 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:24:33.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:33.237+0000 7febab400700 1 --2- 172.21.15.1:0/3782915049 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febac100d50 0x7febac198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:33.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:33.237+0000 7febabe00700 1 --2- 172.21.15.1:0/3782915049 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7febac1028e0 0x7febac1a02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:33.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:33.237+0000 7febab400700 1 -- 172.21.15.1:0/3782915049 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7febac1028e0 msgr2=0x7febac1a02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:33.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:33.237+0000 7febab400700 1 --2- 172.21.15.1:0/3782915049 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7febac1028e0 0x7febac1a02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:33.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:33.237+0000 7febab400700 1 -- 172.21.15.1:0/3782915049 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7febac101f40 msgr2=0x7febac199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:33.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:33.237+0000 7febab400700 1 --2- 172.21.15.1:0/3782915049 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7febac101f40 0x7febac199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:33.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:33.237+0000 7febab400700 1 -- 172.21.15.1:0/3782915049 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb940095f0 con 0x7febac100d50 2024-02-02T15:24:33.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:33.237+0000 7febaaa00700 1 --2- 172.21.15.1:0/3782915049 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7febac101f40 0x7febac199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:24:33.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:33.237+0000 7febab400700 1 --2- 172.21.15.1:0/3782915049 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febac100d50 0x7febac198d40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7feb9c00e9c0 tx=0x7feb9c00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:24:33.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:33.237+0000 7feba3e00700 1 -- 172.21.15.1:0/3782915049 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb9c004220 con 0x7febac100d50 2024-02-02T15:24:33.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:33.238+0000 7feba3e00700 1 -- 172.21.15.1:0/3782915049 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feb9c004380 con 0x7febac100d50 2024-02-02T15:24:33.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:33.238+0000 7febb2400700 1 -- 172.21.15.1:0/3782915049 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feba4009c40 con 0x7febac100d50 2024-02-02T15:24:33.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:33.238+0000 7feba3e00700 1 -- 172.21.15.1:0/3782915049 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb9c003be0 con 0x7febac100d50 2024-02-02T15:24:33.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:33.238+0000 7febb2400700 1 -- 172.21.15.1:0/3782915049 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febac1a1190 con 0x7febac100d50 2024-02-02T15:24:33.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:33.239+0000 7febb2400700 1 -- 172.21.15.1:0/3782915049 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febac10c630 con 0x7febac100d50 2024-02-02T15:24:33.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:33.240+0000 7feba3e00700 1 -- 172.21.15.1:0/3782915049 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 135069+0+0 (secure 0 0 0) 0x7feb9c003d40 con 0x7febac100d50 2024-02-02T15:24:33.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:33.242+0000 7feba3e00700 1 --2- 172.21.15.1:0/3782915049 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7feb8c0a05a0 0x7feb8c0a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:33.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:33.242+0000 7feba3e00700 1 -- 172.21.15.1:0/3782915049 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7feb9c014070 con 0x7febac100d50 2024-02-02T15:24:33.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:33.242+0000 7febaaa00700 1 --2- 172.21.15.1:0/3782915049 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7feb8c0a05a0 0x7feb8c0a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:33.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:33.242+0000 7febaaa00700 1 --2- 172.21.15.1:0/3782915049 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7feb8c0a05a0 0x7feb8c0a2a60 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7feb94005d10 tx=0x7feb94009310 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:24:33.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:33.246+0000 7feba3e00700 1 -- 172.21.15.1:0/3782915049 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7feb9c095720 con 0x7febac100d50 2024-02-02T15:24:33.283 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:32 smithi142 ceph-mon[45728]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-02T15:24:33.283 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:32 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:33.283 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:32 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:33.283 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:32 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:33.284 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:32 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-02T15:24:33.284 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:32 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-02T15:24:33.284 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:32 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:24:33.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:32 smithi071 ceph-mon[44707]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-02T15:24:33.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:32 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:33.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:32 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:33.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:32 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:33.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:32 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-02T15:24:33.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:32 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-02T15:24:33.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:32 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:24:33.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:32 smithi001 ceph-mon[35470]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-02T15:24:33.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:32 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:33.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:32 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:33.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:32 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:33.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:32 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-02T15:24:33.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:32 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-02T15:24:33.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:32 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:24:33.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:33.502+0000 7febb2400700 1 -- 172.21.15.1:0/3782915049 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7febac1a1800 con 0x7febac100d50 2024-02-02T15:24:33.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:33.503+0000 7feba3e00700 1 -- 172.21.15.1:0/3782915049 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6544 (secure 0 0 0) 0x7feb9c098d40 con 0x7febac100d50 2024-02-02T15:24:33.503 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:24:33.503 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2922240448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2922240448},{"type":"v1","addr":"172.21.15.1:6835","nonce":2922240448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2729497431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2729497431},{"type":"v1","addr":"172.21.15.1:6837","nonce":2729497431}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/7648986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":7648986},{"type":"v1","addr":"172.21.15.142:6833","nonce":7648986}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/3482746094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3482746094},{"type":"v1","addr":"172.21.15.71:6833","nonce":3482746094}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/388207814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":388207814},{"type":"v1","addr":"172.21.15.71:6835","nonce":388207814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24502,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/1052606547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1052606547},{"type":"v1","addr":"172.21.15.142:6835","nonce":1052606547}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24514,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/2870289019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2870289019},{"type":"v1","addr":"172.21.15.142:6837","nonce":2870289019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24518,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/628016199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":628016199},{"type":"v1","addr":"172.21.15.71:6837","nonce":628016199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[]} 2024-02-02T15:24:33.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:33.505+0000 7febb2400700 1 -- 172.21.15.1:0/3782915049 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7feb8c0a05a0 msgr2=0x7feb8c0a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:33.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:33.505+0000 7febb2400700 1 --2- 172.21.15.1:0/3782915049 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7feb8c0a05a0 0x7feb8c0a2a60 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7feb94005d10 tx=0x7feb94009310 comp rx=0 tx=0).stop 2024-02-02T15:24:33.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:33.506+0000 7febb2400700 1 -- 172.21.15.1:0/3782915049 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febac100d50 msgr2=0x7febac198d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:33.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:33.506+0000 7febb2400700 1 --2- 172.21.15.1:0/3782915049 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febac100d50 0x7febac198d40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7feb9c00e9c0 tx=0x7feb9c00ecd0 comp rx=0 tx=0).stop 2024-02-02T15:24:33.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:33.506+0000 7febb2400700 1 -- 172.21.15.1:0/3782915049 shutdown_connections 2024-02-02T15:24:33.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:33.506+0000 7febb2400700 1 --2- 172.21.15.1:0/3782915049 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7feb8c0a05a0 0x7feb8c0a2a60 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:33.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:33.506+0000 7febb2400700 1 --2- 172.21.15.1:0/3782915049 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7febac1028e0 0x7febac1a02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:33.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:33.506+0000 7febb2400700 1 --2- 172.21.15.1:0/3782915049 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febac100d50 0x7febac198d40 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:33.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:33.506+0000 7febb2400700 1 --2- 172.21.15.1:0/3782915049 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7febac101f40 0x7febac199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:33.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:33.506+0000 7febb2400700 1 -- 172.21.15.1:0/3782915049 >> 172.21.15.1:0/3782915049 conn(0x7febac0fc2c0 msgr2=0x7febac107770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:24:33.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:33.506+0000 7febb2400700 1 -- 172.21.15.1:0/3782915049 shutdown_connections 2024-02-02T15:24:33.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:33.506+0000 7febb2400700 1 -- 172.21.15.1:0/3782915049 wait complete. 2024-02-02T15:24:33.507 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 9 2024-02-02T15:24:34.291 INFO:journalctl@ceph.mds.i.smithi142.stdout:Feb 02 15:24:34 smithi142 systemd[1]: Starting Ceph mds.i for 371a409c-c1de-11ee-95b6-87774f69a715... 2024-02-02T15:24:34.292 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:33 smithi142 ceph-mon[45728]: Deploying daemon mds.i on smithi142 2024-02-02T15:24:34.292 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:33 smithi142 ceph-mon[45728]: mds.? [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] up:boot 2024-02-02T15:24:34.292 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:33 smithi142 ceph-mon[45728]: fsmap 8 up:standby 2024-02-02T15:24:34.292 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:33 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-02T15:24:34.292 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:33 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/3782915049' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:24:34.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:33 smithi071 ceph-mon[44707]: Deploying daemon mds.i on smithi142 2024-02-02T15:24:34.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:33 smithi071 ceph-mon[44707]: mds.? [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] up:boot 2024-02-02T15:24:34.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:33 smithi071 ceph-mon[44707]: fsmap 8 up:standby 2024-02-02T15:24:34.367 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:33 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-02T15:24:34.367 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:33 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/3782915049' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:24:34.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:33 smithi001 ceph-mon[35470]: Deploying daemon mds.i on smithi142 2024-02-02T15:24:34.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:33 smithi001 ceph-mon[35470]: mds.? [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] up:boot 2024-02-02T15:24:34.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:33 smithi001 ceph-mon[35470]: fsmap 8 up:standby 2024-02-02T15:24:34.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:33 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-02T15:24:34.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:33 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/3782915049' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:24:34.920 INFO:journalctl@ceph.mds.i.smithi142.stdout:Feb 02 15:24:34 smithi142 ceph-mds[83221]: -- 172.21.15.142:0/731090072 <== mon.1 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55f4086e91e0 con 0x55f4086bec00 2024-02-02T15:24:34.920 INFO:journalctl@ceph.mds.i.smithi142.stdout:Feb 02 15:24:34 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mds-i[83198]: starting mds.i at 2024-02-02T15:24:34.920 INFO:journalctl@ceph.mds.i.smithi142.stdout:Feb 02 15:24:34 smithi142 bash[83183]: cda2f53d586d073ae4695b60d006d91ad8ac6fc91bc396f156d7d350ee6b563f 2024-02-02T15:24:34.920 INFO:journalctl@ceph.mds.i.smithi142.stdout:Feb 02 15:24:34 smithi142 systemd[1]: Started Ceph mds.i for 371a409c-c1de-11ee-95b6-87774f69a715. 2024-02-02T15:24:35.217 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:34 smithi001 ceph-mon[35470]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-02T15:24:35.218 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:34 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:35.218 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:34 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:35.218 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:34 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:35.218 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:34 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-02T15:24:35.218 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:34 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-02T15:24:35.218 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:34 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:24:35.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:34 smithi071 ceph-mon[44707]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-02T15:24:35.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:34 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:35.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:34 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:35.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:34 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:35.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:34 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-02T15:24:35.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:34 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-02T15:24:35.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:34 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:24:35.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:34 smithi142 ceph-mon[45728]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-02T15:24:35.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:34 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:35.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:34 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:35.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:34 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:35.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:34 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-02T15:24:35.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:34 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-02T15:24:35.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:34 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:24:35.524 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T15:24:35.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:35.726+0000 7ff67f000700 1 -- 172.21.15.1:0/261549936 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff678106a20 msgr2=0x7ff678108e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:35.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:35.726+0000 7ff67f000700 1 --2- 172.21.15.1:0/261549936 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff678106a20 0x7ff678108e90 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff67000b0a0 tx=0x7ff67000b3b0 comp rx=0 tx=0).stop 2024-02-02T15:24:35.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:35.726+0000 7ff67f000700 1 -- 172.21.15.1:0/261549936 shutdown_connections 2024-02-02T15:24:35.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:35.726+0000 7ff67f000700 1 --2- 172.21.15.1:0/261549936 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff678106a20 0x7ff678108e90 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:35.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:35.726+0000 7ff67f000700 1 --2- 172.21.15.1:0/261549936 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff6781017c0 0x7ff678103bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:35.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:35.726+0000 7ff67f000700 1 --2- 172.21.15.1:0/261549936 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff6781040f0 0x7ff6781064e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:35.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:35.726+0000 7ff67f000700 1 -- 172.21.15.1:0/261549936 >> 172.21.15.1:0/261549936 conn(0x7ff6780fb170 msgr2=0x7ff6780fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:24:35.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:35.726+0000 7ff67f000700 1 -- 172.21.15.1:0/261549936 shutdown_connections 2024-02-02T15:24:35.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:35.726+0000 7ff67f000700 1 -- 172.21.15.1:0/261549936 wait complete. 2024-02-02T15:24:35.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:35.727+0000 7ff67f000700 1 Processor -- start 2024-02-02T15:24:35.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:35.727+0000 7ff67f000700 1 -- start start 2024-02-02T15:24:35.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:35.728+0000 7ff67f000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff6781017c0 0x7ff678198d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:35.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:35.728+0000 7ff67f000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff6781040f0 0x7ff678199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:35.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:35.728+0000 7ff67f000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff678106a20 0x7ff6781a0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:35.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:35.728+0000 7ff67f000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6781a0850 con 0x7ff6781017c0 2024-02-02T15:24:35.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:35.728+0000 7ff67f000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6781a09c0 con 0x7ff678106a20 2024-02-02T15:24:35.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:35.728+0000 7ff67f000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6781a0b80 con 0x7ff6781040f0 2024-02-02T15:24:35.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:35.728+0000 7ff677e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff6781017c0 0x7ff678198d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:35.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:35.728+0000 7ff67d000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff678106a20 0x7ff6781a0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:35.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:35.728+0000 7ff677e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff6781017c0 0x7ff678198d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58978/0 (socket says 172.21.15.1:58978) 2024-02-02T15:24:35.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:35.728+0000 7ff677e00700 1 -- 172.21.15.1:0/274316111 learned_addr learned my addr 172.21.15.1:0/274316111 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:24:35.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:35.728+0000 7ff677400700 1 --2- 172.21.15.1:0/274316111 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff6781040f0 0x7ff678199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:35.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:35.728+0000 7ff67d000700 1 -- 172.21.15.1:0/274316111 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff6781040f0 msgr2=0x7ff678199250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:35.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:35.728+0000 7ff67d000700 1 --2- 172.21.15.1:0/274316111 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff6781040f0 0x7ff678199250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:35.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:35.728+0000 7ff67d000700 1 -- 172.21.15.1:0/274316111 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff6781017c0 msgr2=0x7ff678198d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:35.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:35.728+0000 7ff67d000700 1 --2- 172.21.15.1:0/274316111 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff6781017c0 0x7ff678198d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:35.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:35.728+0000 7ff67d000700 1 -- 172.21.15.1:0/274316111 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff670009d00 con 0x7ff678106a20 2024-02-02T15:24:35.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:35.728+0000 7ff677e00700 1 --2- 172.21.15.1:0/274316111 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff6781017c0 0x7ff678198d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:24:35.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:35.729+0000 7ff67d000700 1 --2- 172.21.15.1:0/274316111 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff678106a20 0x7ff6781a0280 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff670009fd0 tx=0x7ff670007930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:24:35.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:35.729+0000 7ff674c00700 1 -- 172.21.15.1:0/274316111 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff67000e040 con 0x7ff678106a20 2024-02-02T15:24:35.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:35.729+0000 7ff674c00700 1 -- 172.21.15.1:0/274316111 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff670004c50 con 0x7ff678106a20 2024-02-02T15:24:35.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:35.729+0000 7ff67f000700 1 -- 172.21.15.1:0/274316111 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6781a0e00 con 0x7ff678106a20 2024-02-02T15:24:35.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:35.729+0000 7ff674c00700 1 -- 172.21.15.1:0/274316111 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff67001b6c0 con 0x7ff678106a20 2024-02-02T15:24:35.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:35.729+0000 7ff67f000700 1 -- 172.21.15.1:0/274316111 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6781a11d0 con 0x7ff678106a20 2024-02-02T15:24:35.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:35.730+0000 7ff67f000700 1 -- 172.21.15.1:0/274316111 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6781085a0 con 0x7ff678106a20 2024-02-02T15:24:35.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:35.732+0000 7ff674c00700 1 -- 172.21.15.1:0/274316111 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 135069+0+0 (secure 0 0 0) 0x7ff67001d470 con 0x7ff678106a20 2024-02-02T15:24:35.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:35.733+0000 7ff674c00700 1 --2- 172.21.15.1:0/274316111 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7ff6580a0520 0x7ff6580a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:35.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:35.733+0000 7ff674c00700 1 -- 172.21.15.1:0/274316111 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff6700cce30 con 0x7ff678106a20 2024-02-02T15:24:35.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:35.733+0000 7ff677e00700 1 --2- 172.21.15.1:0/274316111 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7ff6580a0520 0x7ff6580a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:35.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:35.734+0000 7ff677e00700 1 --2- 172.21.15.1:0/274316111 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7ff6580a0520 0x7ff6580a29e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff67819a1e0 tx=0x7ff668009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:24:35.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:35.735+0000 7ff674c00700 1 -- 172.21.15.1:0/274316111 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7ff670096e60 con 0x7ff678106a20 2024-02-02T15:24:36.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:36.033+0000 7ff67f000700 1 -- 172.21.15.1:0/274316111 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff67802cc70 con 0x7ff678106a20 2024-02-02T15:24:36.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:36.034+0000 7ff674c00700 1 -- 172.21.15.1:0/274316111 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7303 (secure 0 0 0) 0x7ff67009a480 con 0x7ff678106a20 2024-02-02T15:24:36.034 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:24:36.034 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2922240448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2922240448},{"type":"v1","addr":"172.21.15.1:6835","nonce":2922240448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2729497431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2729497431},{"type":"v1","addr":"172.21.15.1:6837","nonce":2729497431}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/7648986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":7648986},{"type":"v1","addr":"172.21.15.142:6833","nonce":7648986}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/3482746094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3482746094},{"type":"v1","addr":"172.21.15.71:6833","nonce":3482746094}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/388207814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":388207814},{"type":"v1","addr":"172.21.15.71:6835","nonce":388207814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24502,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/1052606547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1052606547},{"type":"v1","addr":"172.21.15.142:6835","nonce":1052606547}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24514,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/2870289019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2870289019},{"type":"v1","addr":"172.21.15.142:6837","nonce":2870289019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24518,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/628016199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":628016199},{"type":"v1","addr":"172.21.15.71:6837","nonce":628016199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24529,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6839/321675421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":321675421},{"type":"v1","addr":"172.21.15.142:6839","nonce":321675421}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2024-02-02T15:24:36.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:36.036+0000 7ff67f000700 1 -- 172.21.15.1:0/274316111 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7ff6580a0520 msgr2=0x7ff6580a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:36.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:36.036+0000 7ff67f000700 1 --2- 172.21.15.1:0/274316111 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7ff6580a0520 0x7ff6580a29e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff67819a1e0 tx=0x7ff668009260 comp rx=0 tx=0).stop 2024-02-02T15:24:36.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:36.036+0000 7ff67f000700 1 -- 172.21.15.1:0/274316111 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff678106a20 msgr2=0x7ff6781a0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:36.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:36.036+0000 7ff67f000700 1 --2- 172.21.15.1:0/274316111 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff678106a20 0x7ff6781a0280 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff670009fd0 tx=0x7ff670007930 comp rx=0 tx=0).stop 2024-02-02T15:24:36.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:36.036+0000 7ff67f000700 1 -- 172.21.15.1:0/274316111 shutdown_connections 2024-02-02T15:24:36.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:36.036+0000 7ff67f000700 1 --2- 172.21.15.1:0/274316111 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff678106a20 0x7ff6781a0280 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:36.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:36.036+0000 7ff67f000700 1 --2- 172.21.15.1:0/274316111 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff6781017c0 0x7ff678198d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:36.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:36.036+0000 7ff67f000700 1 --2- 172.21.15.1:0/274316111 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7ff6580a0520 0x7ff6580a29e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:36.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:36.036+0000 7ff67f000700 1 --2- 172.21.15.1:0/274316111 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff6781040f0 0x7ff678199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:36.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:36.036+0000 7ff67f000700 1 -- 172.21.15.1:0/274316111 >> 172.21.15.1:0/274316111 conn(0x7ff6780fb170 msgr2=0x7ff6780ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:24:36.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:36.036+0000 7ff67f000700 1 -- 172.21.15.1:0/274316111 shutdown_connections 2024-02-02T15:24:36.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:36.037+0000 7ff67f000700 1 -- 172.21.15.1:0/274316111 wait complete. 2024-02-02T15:24:36.038 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 10 2024-02-02T15:24:36.189 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:35 smithi001 ceph-mon[35470]: Deploying daemon mds.g on smithi001 2024-02-02T15:24:36.189 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:35 smithi001 ceph-mon[35470]: mds.? [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] up:boot 2024-02-02T15:24:36.189 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:35 smithi001 ceph-mon[35470]: fsmap 9 up:standby 2024-02-02T15:24:36.189 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:35 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-02T15:24:36.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:35 smithi071 ceph-mon[44707]: Deploying daemon mds.g on smithi001 2024-02-02T15:24:36.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:35 smithi071 ceph-mon[44707]: mds.? [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] up:boot 2024-02-02T15:24:36.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:35 smithi071 ceph-mon[44707]: fsmap 9 up:standby 2024-02-02T15:24:36.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:35 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-02T15:24:36.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:35 smithi142 ceph-mon[45728]: Deploying daemon mds.g on smithi001 2024-02-02T15:24:36.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:35 smithi142 ceph-mon[45728]: mds.? [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] up:boot 2024-02-02T15:24:36.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:35 smithi142 ceph-mon[45728]: fsmap 9 up:standby 2024-02-02T15:24:36.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:35 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-02T15:24:36.440 INFO:journalctl@ceph.mds.g.smithi001.stdout:Feb 02 15:24:36 smithi001 systemd[1]: Starting Ceph mds.g for 371a409c-c1de-11ee-95b6-87774f69a715... 2024-02-02T15:24:37.190 INFO:journalctl@ceph.mds.g.smithi001.stdout:Feb 02 15:24:36 smithi001 bash[84334]: ebb0bc4afb884140e33582e25d34ac2eacfa5068171c5c56a32665d47c6e38e9 2024-02-02T15:24:37.190 INFO:journalctl@ceph.mds.g.smithi001.stdout:Feb 02 15:24:36 smithi001 systemd[1]: Started Ceph mds.g for 371a409c-c1de-11ee-95b6-87774f69a715. 2024-02-02T15:24:37.190 INFO:journalctl@ceph.mds.g.smithi001.stdout:Feb 02 15:24:36 smithi001 ceph-mds[84372]: -- 172.21.15.1:0/630164910 <== mon.1 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a8f13f71e0 con 0x55a8f13ce800 2024-02-02T15:24:37.190 INFO:journalctl@ceph.mds.g.smithi001.stdout:Feb 02 15:24:36 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mds-g[84349]: starting mds.g at 2024-02-02T15:24:37.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:36 smithi001 ceph-mon[35470]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-02T15:24:37.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:36 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/274316111' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:24:37.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:36 smithi071 ceph-mon[44707]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-02T15:24:37.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:36 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/274316111' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:24:37.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:36 smithi142 ceph-mon[45728]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-02T15:24:37.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:36 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/274316111' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:24:38.057 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T15:24:38.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:38.265+0000 7fe844800700 1 -- 172.21.15.1:0/2145191329 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe83c074d80 msgr2=0x7fe83c0731d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:38.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:38.265+0000 7fe844800700 1 --2- 172.21.15.1:0/2145191329 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe83c074d80 0x7fe83c0731d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe82c009990 tx=0x7fe82c009ca0 comp rx=0 tx=0).stop 2024-02-02T15:24:38.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:38.265+0000 7fe844800700 1 -- 172.21.15.1:0/2145191329 shutdown_connections 2024-02-02T15:24:38.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:38.265+0000 7fe844800700 1 --2- 172.21.15.1:0/2145191329 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe83c108c00 0x7fe83c10b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:38.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:38.265+0000 7fe844800700 1 --2- 172.21.15.1:0/2145191329 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe83c074d80 0x7fe83c0731d0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:38.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:38.265+0000 7fe844800700 1 --2- 172.21.15.1:0/2145191329 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe83c0737a0 0x7fe83c073c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:38.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:38.265+0000 7fe844800700 1 -- 172.21.15.1:0/2145191329 >> 172.21.15.1:0/2145191329 conn(0x7fe83c0fc290 msgr2=0x7fe83c0fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:24:38.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:38.265+0000 7fe844800700 1 -- 172.21.15.1:0/2145191329 shutdown_connections 2024-02-02T15:24:38.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:38.265+0000 7fe844800700 1 -- 172.21.15.1:0/2145191329 wait complete. 2024-02-02T15:24:38.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:38.267+0000 7fe844800700 1 Processor -- start 2024-02-02T15:24:38.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:38.267+0000 7fe844800700 1 -- start start 2024-02-02T15:24:38.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:38.268+0000 7fe844800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe83c0737a0 0x7fe83c198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:38.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:38.268+0000 7fe844800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe83c074d80 0x7fe83c1991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:38.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:38.268+0000 7fe844800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe83c108c00 0x7fe83c1a0210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:38.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:38.268+0000 7fe844800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe83c1a07e0 con 0x7fe83c074d80 2024-02-02T15:24:38.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:38.268+0000 7fe844800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe83c1a0950 con 0x7fe83c0737a0 2024-02-02T15:24:38.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:38.268+0000 7fe844800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe83c1a0b10 con 0x7fe83c108c00 2024-02-02T15:24:38.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:38.268+0000 7fe841e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe83c0737a0 0x7fe83c198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:38.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:38.268+0000 7fe841400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe83c074d80 0x7fe83c1991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:38.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:38.268+0000 7fe841e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe83c0737a0 0x7fe83c198c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42752/0 (socket says 172.21.15.1:42752) 2024-02-02T15:24:38.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:38.268+0000 7fe841e00700 1 -- 172.21.15.1:0/3737828466 learned_addr learned my addr 172.21.15.1:0/3737828466 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:24:38.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:38.268+0000 7fe842800700 1 --2- 172.21.15.1:0/3737828466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe83c108c00 0x7fe83c1a0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:38.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:38.268+0000 7fe841400700 1 -- 172.21.15.1:0/3737828466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe83c108c00 msgr2=0x7fe83c1a0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:38.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:38.268+0000 7fe841400700 1 --2- 172.21.15.1:0/3737828466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe83c108c00 0x7fe83c1a0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:38.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:38.268+0000 7fe841400700 1 -- 172.21.15.1:0/3737828466 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe83c0737a0 msgr2=0x7fe83c198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:38.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:38.268+0000 7fe841400700 1 --2- 172.21.15.1:0/3737828466 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe83c0737a0 0x7fe83c198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:38.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:38.268+0000 7fe841400700 1 -- 172.21.15.1:0/3737828466 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe82c0095f0 con 0x7fe83c074d80 2024-02-02T15:24:38.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:38.269+0000 7fe841400700 1 --2- 172.21.15.1:0/3737828466 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe83c074d80 0x7fe83c1991b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe82400e9c0 tx=0x7fe82400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:24:38.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:38.269+0000 7fe83aa00700 1 -- 172.21.15.1:0/3737828466 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe824004220 con 0x7fe83c074d80 2024-02-02T15:24:38.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:38.269+0000 7fe83aa00700 1 -- 172.21.15.1:0/3737828466 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe824004380 con 0x7fe83c074d80 2024-02-02T15:24:38.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:38.269+0000 7fe844800700 1 -- 172.21.15.1:0/3737828466 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe83c1a0df0 con 0x7fe83c074d80 2024-02-02T15:24:38.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:38.269+0000 7fe83aa00700 1 -- 172.21.15.1:0/3737828466 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe824003be0 con 0x7fe83c074d80 2024-02-02T15:24:38.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:38.269+0000 7fe844800700 1 -- 172.21.15.1:0/3737828466 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe83c1a1280 con 0x7fe83c074d80 2024-02-02T15:24:38.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:38.270+0000 7fe844800700 1 -- 172.21.15.1:0/3737828466 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe83c1072b0 con 0x7fe83c074d80 2024-02-02T15:24:38.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:38.272+0000 7fe83aa00700 1 -- 172.21.15.1:0/3737828466 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) v1 ==== 135069+0+0 (secure 0 0 0) 0x7fe82400c7c0 con 0x7fe83c074d80 2024-02-02T15:24:38.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:38.273+0000 7fe83aa00700 1 --2- 172.21.15.1:0/3737828466 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fe81c0a05a0 0x7fe81c0a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:38.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:38.273+0000 7fe83aa00700 1 -- 172.21.15.1:0/3737828466 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe824014070 con 0x7fe83c074d80 2024-02-02T15:24:38.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:38.274+0000 7fe841e00700 1 --2- 172.21.15.1:0/3737828466 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fe81c0a05a0 0x7fe81c0a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:38.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:38.274+0000 7fe841e00700 1 --2- 172.21.15.1:0/3737828466 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fe81c0a05a0 0x7fe81c0a2a60 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe82c00b2e0 tx=0x7fe82c005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:24:38.275 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:38 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:38.275 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:38 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:38.275 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:38 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:38.275 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:38 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-02T15:24:38.275 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:38 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-02T15:24:38.275 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:38 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:24:38.275 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:38 smithi001 ceph-mon[35470]: Deploying daemon mds.a on smithi001 2024-02-02T15:24:38.275 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:38 smithi001 ceph-mon[35470]: mds.? [v2:172.21.15.1:6838/1218303683,v1:172.21.15.1:6839/1218303683] up:boot 2024-02-02T15:24:38.276 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:38 smithi001 ceph-mon[35470]: fsmap 10 up:standby 2024-02-02T15:24:38.276 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:38 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-02-02T15:24:38.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:38.278+0000 7fe83aa00700 1 -- 172.21.15.1:0/3737828466 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7fe824095750 con 0x7fe83c074d80 2024-02-02T15:24:38.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:38 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:38.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:38 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:38.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:38 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:38.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:38 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-02T15:24:38.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:38 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-02T15:24:38.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:38 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:24:38.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:38 smithi071 ceph-mon[44707]: Deploying daemon mds.a on smithi001 2024-02-02T15:24:38.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:38 smithi071 ceph-mon[44707]: mds.? [v2:172.21.15.1:6838/1218303683,v1:172.21.15.1:6839/1218303683] up:boot 2024-02-02T15:24:38.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:38 smithi071 ceph-mon[44707]: fsmap 10 up:standby 2024-02-02T15:24:38.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:38 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-02-02T15:24:38.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:38 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:38.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:38 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:38.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:38 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:38.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:38 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-02T15:24:38.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:38 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-02T15:24:38.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:38 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:24:38.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:38 smithi142 ceph-mon[45728]: Deploying daemon mds.a on smithi001 2024-02-02T15:24:38.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:38 smithi142 ceph-mon[45728]: mds.? [v2:172.21.15.1:6838/1218303683,v1:172.21.15.1:6839/1218303683] up:boot 2024-02-02T15:24:38.421 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:38 smithi142 ceph-mon[45728]: fsmap 10 up:standby 2024-02-02T15:24:38.421 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:38 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-02-02T15:24:38.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:38.564+0000 7fe844800700 1 -- 172.21.15.1:0/3737828466 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe83c02cc70 con 0x7fe83c074d80 2024-02-02T15:24:38.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:38.564+0000 7fe83aa00700 1 -- 172.21.15.1:0/3737828466 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8058 (secure 0 0 0) 0x7fe824098d70 con 0x7fe83c074d80 2024-02-02T15:24:38.565 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:24:38.565 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2922240448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2922240448},{"type":"v1","addr":"172.21.15.1:6835","nonce":2922240448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2729497431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2729497431},{"type":"v1","addr":"172.21.15.1:6837","nonce":2729497431}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14517,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/1218303683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":1218303683},{"type":"v1","addr":"172.21.15.1:6839","nonce":1218303683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/7648986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":7648986},{"type":"v1","addr":"172.21.15.142:6833","nonce":7648986}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/3482746094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3482746094},{"type":"v1","addr":"172.21.15.71:6833","nonce":3482746094}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/388207814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":388207814},{"type":"v1","addr":"172.21.15.71:6835","nonce":388207814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24502,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/1052606547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1052606547},{"type":"v1","addr":"172.21.15.142:6835","nonce":1052606547}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24514,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/2870289019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2870289019},{"type":"v1","addr":"172.21.15.142:6837","nonce":2870289019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24518,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/628016199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":628016199},{"type":"v1","addr":"172.21.15.71:6837","nonce":628016199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24529,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6839/321675421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":321675421},{"type":"v1","addr":"172.21.15.142:6839","nonce":321675421}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2024-02-02T15:24:38.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:38.567+0000 7fe844800700 1 -- 172.21.15.1:0/3737828466 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fe81c0a05a0 msgr2=0x7fe81c0a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:38.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:38.567+0000 7fe844800700 1 --2- 172.21.15.1:0/3737828466 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fe81c0a05a0 0x7fe81c0a2a60 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe82c00b2e0 tx=0x7fe82c005fb0 comp rx=0 tx=0).stop 2024-02-02T15:24:38.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:38.567+0000 7fe844800700 1 -- 172.21.15.1:0/3737828466 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe83c074d80 msgr2=0x7fe83c1991b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:38.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:38.567+0000 7fe844800700 1 --2- 172.21.15.1:0/3737828466 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe83c074d80 0x7fe83c1991b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe82400e9c0 tx=0x7fe82400ecd0 comp rx=0 tx=0).stop 2024-02-02T15:24:38.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:38.567+0000 7fe844800700 1 -- 172.21.15.1:0/3737828466 shutdown_connections 2024-02-02T15:24:38.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:38.567+0000 7fe844800700 1 --2- 172.21.15.1:0/3737828466 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fe81c0a05a0 0x7fe81c0a2a60 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:38.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:38.567+0000 7fe844800700 1 --2- 172.21.15.1:0/3737828466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe83c108c00 0x7fe83c1a0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:38.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:38.567+0000 7fe844800700 1 --2- 172.21.15.1:0/3737828466 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe83c0737a0 0x7fe83c198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:38.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:38.567+0000 7fe844800700 1 --2- 172.21.15.1:0/3737828466 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe83c074d80 0x7fe83c1991b0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:38.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:38.567+0000 7fe844800700 1 -- 172.21.15.1:0/3737828466 >> 172.21.15.1:0/3737828466 conn(0x7fe83c0fc290 msgr2=0x7fe83c0fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:24:38.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:38.567+0000 7fe844800700 1 -- 172.21.15.1:0/3737828466 shutdown_connections 2024-02-02T15:24:38.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:38.567+0000 7fe844800700 1 -- 172.21.15.1:0/3737828466 wait complete. 2024-02-02T15:24:38.568 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 11 2024-02-02T15:24:39.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:39 smithi001 ceph-mon[35470]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-02T15:24:39.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:39 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/3737828466' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:24:39.191 INFO:journalctl@ceph.mds.a.smithi001.stdout:Feb 02 15:24:38 smithi001 systemd[1]: Starting Ceph mds.a for 371a409c-c1de-11ee-95b6-87774f69a715... 2024-02-02T15:24:39.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:39 smithi071 ceph-mon[44707]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-02T15:24:39.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:39 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/3737828466' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:24:39.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:39 smithi142 ceph-mon[45728]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-02T15:24:39.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:39 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/3737828466' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:24:40.190 INFO:journalctl@ceph.mds.a.smithi001.stdout:Feb 02 15:24:39 smithi001 ceph-mds[84758]: -- 172.21.15.1:0/351596165 <== mon.0 v2:172.21.15.1:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5636359791e0 con 0x563635950800 2024-02-02T15:24:40.190 INFO:journalctl@ceph.mds.a.smithi001.stdout:Feb 02 15:24:39 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mds-a[84735]: starting mds.a at 2024-02-02T15:24:40.190 INFO:journalctl@ceph.mds.a.smithi001.stdout:Feb 02 15:24:39 smithi001 bash[84721]: 9bd8bce92f7c363c9d013a701d7fbde9594232b4c0cde3ee2e2bb09211230631 2024-02-02T15:24:40.190 INFO:journalctl@ceph.mds.a.smithi001.stdout:Feb 02 15:24:39 smithi001 systemd[1]: Started Ceph mds.a for 371a409c-c1de-11ee-95b6-87774f69a715. 2024-02-02T15:24:40.587 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T15:24:40.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:40.800+0000 7f1a04400700 1 -- 172.21.15.1:0/2340902676 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f19fc1028d0 msgr2=0x7f19fc109180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:40.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:40.800+0000 7f1a04400700 1 --2- 172.21.15.1:0/2340902676 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f19fc1028d0 0x7f19fc109180 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f19f400b0d0 tx=0x7f19f400b3e0 comp rx=0 tx=0).stop 2024-02-02T15:24:40.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:40.800+0000 7f1a04400700 1 -- 172.21.15.1:0/2340902676 shutdown_connections 2024-02-02T15:24:40.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:40.800+0000 7f1a04400700 1 --2- 172.21.15.1:0/2340902676 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f19fc1028d0 0x7f19fc109180 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:40.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:40.800+0000 7f1a04400700 1 --2- 172.21.15.1:0/2340902676 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19fc100d40 0x7f19fc101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:40.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:40.800+0000 7f1a04400700 1 --2- 172.21.15.1:0/2340902676 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f19fc101f30 0x7f19fc102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:40.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:40.800+0000 7f1a04400700 1 -- 172.21.15.1:0/2340902676 >> 172.21.15.1:0/2340902676 conn(0x7f19fc0fc290 msgr2=0x7f19fc0fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:24:40.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:40.800+0000 7f1a04400700 1 -- 172.21.15.1:0/2340902676 shutdown_connections 2024-02-02T15:24:40.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:40.801+0000 7f1a04400700 1 -- 172.21.15.1:0/2340902676 wait complete. 2024-02-02T15:24:40.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:40.801+0000 7f1a04400700 1 Processor -- start 2024-02-02T15:24:40.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:40.802+0000 7f1a04400700 1 -- start start 2024-02-02T15:24:40.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:40.802+0000 7f1a04400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f19fc100d40 0x7f19fc196b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:40.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:40.802+0000 7f1a04400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19fc101f30 0x7f19fc1970d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:40.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:40.802+0000 7f1a04400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f19fc1028d0 0x7f19fc19e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:40.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:40.802+0000 7f1a04400700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19fc19e710 con 0x7f19fc1028d0 2024-02-02T15:24:40.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:40.802+0000 7f1a04400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19fc19e850 con 0x7f19fc100d40 2024-02-02T15:24:40.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:40.802+0000 7f1a04400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19fc19e9e0 con 0x7f19fc101f30 2024-02-02T15:24:40.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:40.802+0000 7f1a02400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f19fc1028d0 0x7f19fc19e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:40.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:40.802+0000 7f1a02400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f19fc1028d0 0x7f19fc19e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59048/0 (socket says 172.21.15.1:59048) 2024-02-02T15:24:40.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:40.802+0000 7f1a02400700 1 -- 172.21.15.1:0/2512861361 learned_addr learned my addr 172.21.15.1:0/2512861361 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:24:40.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:40.802+0000 7f1a01000700 1 --2- 172.21.15.1:0/2512861361 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19fc101f30 0x7f19fc1970d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:40.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:40.803+0000 7f1a02400700 1 -- 172.21.15.1:0/2512861361 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19fc101f30 msgr2=0x7f19fc1970d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:40.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:40.803+0000 7f1a02400700 1 --2- 172.21.15.1:0/2512861361 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19fc101f30 0x7f19fc1970d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:40.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:40.803+0000 7f1a02400700 1 -- 172.21.15.1:0/2512861361 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f19fc100d40 msgr2=0x7f19fc196b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:40.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:40.803+0000 7f1a02400700 1 --2- 172.21.15.1:0/2512861361 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f19fc100d40 0x7f19fc196b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:40.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:40.803+0000 7f1a02400700 1 -- 172.21.15.1:0/2512861361 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19f4009d00 con 0x7f19fc1028d0 2024-02-02T15:24:40.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:40.803+0000 7f1a02400700 1 --2- 172.21.15.1:0/2512861361 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f19fc1028d0 0x7f19fc19e140 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f19f4007990 tx=0x7f19f4008eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:24:40.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:40.803+0000 7f19fa000700 1 -- 172.21.15.1:0/2512861361 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f400e040 con 0x7f19fc1028d0 2024-02-02T15:24:40.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:40.803+0000 7f19fa000700 1 -- 172.21.15.1:0/2512861361 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f19f400be30 con 0x7f19fc1028d0 2024-02-02T15:24:40.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:40.803+0000 7f1a04400700 1 -- 172.21.15.1:0/2512861361 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19fc19ec60 con 0x7f19fc1028d0 2024-02-02T15:24:40.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:40.803+0000 7f19fa000700 1 -- 172.21.15.1:0/2512861361 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19f401d6d0 con 0x7f19fc1028d0 2024-02-02T15:24:40.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:40.803+0000 7f1a04400700 1 -- 172.21.15.1:0/2512861361 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19fc19f0e0 con 0x7f19fc1028d0 2024-02-02T15:24:40.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:40.804+0000 7f1a04400700 1 -- 172.21.15.1:0/2512861361 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19fc107bd0 con 0x7f19fc1028d0 2024-02-02T15:24:40.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:40.806+0000 7f19fa000700 1 -- 172.21.15.1:0/2512861361 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) v1 ==== 135069+0+0 (secure 0 0 0) 0x7f19f40051b0 con 0x7f19fc1028d0 2024-02-02T15:24:40.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:40.807+0000 7f19fa000700 1 --2- 172.21.15.1:0/2512861361 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f19dc0a05a0 0x7f19dc0a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:40.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:40.808+0000 7f19fa000700 1 -- 172.21.15.1:0/2512861361 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f19f40cc8d0 con 0x7f19fc1028d0 2024-02-02T15:24:40.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:40.808+0000 7f1a01a00700 1 --2- 172.21.15.1:0/2512861361 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f19dc0a05a0 0x7f19dc0a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:40.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:40.808+0000 7f1a01a00700 1 --2- 172.21.15.1:0/2512861361 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f19dc0a05a0 0x7f19dc0a2a60 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f19fc101d90 tx=0x7f19ec0096c0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:24:40.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:40.812+0000 7f19fa000700 1 -- 172.21.15.1:0/2512861361 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f19f4096900 con 0x7f19fc1028d0 2024-02-02T15:24:40.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:40 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:40.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:40 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:40.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:40 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:40.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:40 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-02T15:24:40.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:40 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-02T15:24:40.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:40 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:24:40.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:40 smithi001 ceph-mon[35470]: Deploying daemon mds.b on smithi071 2024-02-02T15:24:40.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:40 smithi001 ceph-mon[35470]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-02T15:24:40.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:40 smithi001 ceph-mon[35470]: mds.? [v2:172.21.15.1:6840/2059197496,v1:172.21.15.1:6841/2059197496] up:boot 2024-02-02T15:24:40.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:40 smithi001 ceph-mon[35470]: fsmap 11 up:standby 2024-02-02T15:24:40.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:40 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-02-02T15:24:41.030 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:40 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:41.030 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:40 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:41.030 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:40 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:41.030 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:40 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-02T15:24:41.030 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:40 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-02T15:24:41.030 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:40 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:24:41.030 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:40 smithi071 ceph-mon[44707]: Deploying daemon mds.b on smithi071 2024-02-02T15:24:41.030 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:40 smithi071 ceph-mon[44707]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-02T15:24:41.030 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:40 smithi071 ceph-mon[44707]: mds.? [v2:172.21.15.1:6840/2059197496,v1:172.21.15.1:6841/2059197496] up:boot 2024-02-02T15:24:41.030 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:40 smithi071 ceph-mon[44707]: fsmap 11 up:standby 2024-02-02T15:24:41.030 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:40 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-02-02T15:24:41.031 INFO:journalctl@ceph.mds.b.smithi071.stdout:Feb 02 15:24:41 smithi071 systemd[1]: Starting Ceph mds.b for 371a409c-c1de-11ee-95b6-87774f69a715... 2024-02-02T15:24:41.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:41.075+0000 7f1a04400700 1 -- 172.21.15.1:0/2512861361 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f19fc02cc70 con 0x7f19fc1028d0 2024-02-02T15:24:41.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:41.075+0000 7f19fa000700 1 -- 172.21.15.1:0/2512861361 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+8813 (secure 0 0 0) 0x7f19f4099f20 con 0x7f19fc1028d0 2024-02-02T15:24:41.075 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:24:41.075 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2922240448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2922240448},{"type":"v1","addr":"172.21.15.1:6835","nonce":2922240448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2729497431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2729497431},{"type":"v1","addr":"172.21.15.1:6837","nonce":2729497431}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14517,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/1218303683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":1218303683},{"type":"v1","addr":"172.21.15.1:6839","nonce":1218303683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14529,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/2059197496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2059197496},{"type":"v1","addr":"172.21.15.1:6841","nonce":2059197496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/7648986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":7648986},{"type":"v1","addr":"172.21.15.142:6833","nonce":7648986}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/3482746094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3482746094},{"type":"v1","addr":"172.21.15.71:6833","nonce":3482746094}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/388207814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":388207814},{"type":"v1","addr":"172.21.15.71:6835","nonce":388207814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24502,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/1052606547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1052606547},{"type":"v1","addr":"172.21.15.142:6835","nonce":1052606547}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24514,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/2870289019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2870289019},{"type":"v1","addr":"172.21.15.142:6837","nonce":2870289019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24518,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/628016199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":628016199},{"type":"v1","addr":"172.21.15.71:6837","nonce":628016199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24529,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6839/321675421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":321675421},{"type":"v1","addr":"172.21.15.142:6839","nonce":321675421}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2024-02-02T15:24:41.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:41.078+0000 7f1a04400700 1 -- 172.21.15.1:0/2512861361 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f19dc0a05a0 msgr2=0x7f19dc0a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:41.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:41.078+0000 7f1a04400700 1 --2- 172.21.15.1:0/2512861361 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f19dc0a05a0 0x7f19dc0a2a60 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f19fc101d90 tx=0x7f19ec0096c0 comp rx=0 tx=0).stop 2024-02-02T15:24:41.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:41.078+0000 7f1a04400700 1 -- 172.21.15.1:0/2512861361 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f19fc1028d0 msgr2=0x7f19fc19e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:41.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:41.078+0000 7f1a04400700 1 --2- 172.21.15.1:0/2512861361 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f19fc1028d0 0x7f19fc19e140 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f19f4007990 tx=0x7f19f4008eb0 comp rx=0 tx=0).stop 2024-02-02T15:24:41.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:41.078+0000 7f1a04400700 1 -- 172.21.15.1:0/2512861361 shutdown_connections 2024-02-02T15:24:41.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:41.078+0000 7f1a04400700 1 --2- 172.21.15.1:0/2512861361 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f19fc1028d0 0x7f19fc19e140 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:41.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:41.078+0000 7f1a04400700 1 --2- 172.21.15.1:0/2512861361 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f19fc100d40 0x7f19fc196b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:41.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:41.078+0000 7f1a04400700 1 --2- 172.21.15.1:0/2512861361 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f19dc0a05a0 0x7f19dc0a2a60 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:41.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:41.078+0000 7f1a04400700 1 --2- 172.21.15.1:0/2512861361 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19fc101f30 0x7f19fc1970d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:41.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:41.078+0000 7f1a04400700 1 -- 172.21.15.1:0/2512861361 >> 172.21.15.1:0/2512861361 conn(0x7f19fc0fc290 msgr2=0x7f19fc0fe490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:24:41.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:41.078+0000 7f1a04400700 1 -- 172.21.15.1:0/2512861361 shutdown_connections 2024-02-02T15:24:41.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:41.078+0000 7f1a04400700 1 -- 172.21.15.1:0/2512861361 wait complete. 2024-02-02T15:24:41.079 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 12 2024-02-02T15:24:41.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:40 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:41.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:40 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:41.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:40 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:41.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:40 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-02-02T15:24:41.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:40 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-02-02T15:24:41.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:40 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:24:41.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:40 smithi142 ceph-mon[45728]: Deploying daemon mds.b on smithi071 2024-02-02T15:24:41.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:40 smithi142 ceph-mon[45728]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-02T15:24:41.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:40 smithi142 ceph-mon[45728]: mds.? [v2:172.21.15.1:6840/2059197496,v1:172.21.15.1:6841/2059197496] up:boot 2024-02-02T15:24:41.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:40 smithi142 ceph-mon[45728]: fsmap 11 up:standby 2024-02-02T15:24:41.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:40 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-02-02T15:24:41.819 INFO:journalctl@ceph.mds.b.smithi071.stdout:Feb 02 15:24:41 smithi071 ceph-mds[79683]: -- 172.21.15.71:0/447383405 <== mon.2 v2:172.21.15.71:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55cb5b3b71e0 con 0x55cb5b38d000 2024-02-02T15:24:41.819 INFO:journalctl@ceph.mds.b.smithi071.stdout:Feb 02 15:24:41 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mds-b[79660]: starting mds.b at 2024-02-02T15:24:41.819 INFO:journalctl@ceph.mds.b.smithi071.stdout:Feb 02 15:24:41 smithi071 bash[79646]: 971da35dde5adda3167a797cf2e2165e45829c1a13fc2679c84da54ea0f31202 2024-02-02T15:24:41.819 INFO:journalctl@ceph.mds.b.smithi071.stdout:Feb 02 15:24:41 smithi071 systemd[1]: Started Ceph mds.b for 371a409c-c1de-11ee-95b6-87774f69a715. 2024-02-02T15:24:42.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:41 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/2512861361' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:24:42.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:41 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:42.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:41 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:42.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:41 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:42.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:41 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:42.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:41 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:42.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:41 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:24:42.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:41 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/2512861361' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:24:42.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:41 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:42.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:41 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:42.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:41 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:42.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:41 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:42.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:41 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:42.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:41 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:24:42.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:41 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/2512861361' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:24:42.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:41 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:42.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:41 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:42.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:41 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:42.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:41 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:42.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:41 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:42.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:41 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:24:43.098 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T15:24:43.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:42 smithi071 ceph-mon[44707]: mds.? [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] up:boot 2024-02-02T15:24:43.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:42 smithi071 ceph-mon[44707]: fsmap 12 up:standby 2024-02-02T15:24:43.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:42 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-02-02T15:24:43.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:42 smithi071 ceph-mon[44707]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-02T15:24:43.162 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:42 smithi001 ceph-mon[35470]: mds.? [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] up:boot 2024-02-02T15:24:43.162 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:42 smithi001 ceph-mon[35470]: fsmap 12 up:standby 2024-02-02T15:24:43.162 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:42 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-02-02T15:24:43.162 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:42 smithi001 ceph-mon[35470]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-02T15:24:43.169 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:42 smithi142 ceph-mon[45728]: mds.? [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] up:boot 2024-02-02T15:24:43.169 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:42 smithi142 ceph-mon[45728]: fsmap 12 up:standby 2024-02-02T15:24:43.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:42 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-02-02T15:24:43.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:42 smithi142 ceph-mon[45728]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-02T15:24:43.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.340+0000 7f6c33a00700 1 -- 172.21.15.1:0/3144533223 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c2c1040f0 msgr2=0x7f6c2c1064e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:43.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.340+0000 7f6c33a00700 1 --2- 172.21.15.1:0/3144533223 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c2c1040f0 0x7f6c2c1064e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6c14009990 tx=0x7f6c14009ca0 comp rx=0 tx=0).stop 2024-02-02T15:24:43.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.341+0000 7f6c33a00700 1 -- 172.21.15.1:0/3144533223 shutdown_connections 2024-02-02T15:24:43.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.341+0000 7f6c33a00700 1 --2- 172.21.15.1:0/3144533223 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6c2c106a20 0x7f6c2c108e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:43.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.341+0000 7f6c33a00700 1 --2- 172.21.15.1:0/3144533223 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6c2c1017c0 0x7f6c2c103bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:43.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.341+0000 7f6c33a00700 1 --2- 172.21.15.1:0/3144533223 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c2c1040f0 0x7f6c2c1064e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:43.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.341+0000 7f6c33a00700 1 -- 172.21.15.1:0/3144533223 >> 172.21.15.1:0/3144533223 conn(0x7f6c2c0fb1b0 msgr2=0x7f6c2c0fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:24:43.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.341+0000 7f6c33a00700 1 -- 172.21.15.1:0/3144533223 shutdown_connections 2024-02-02T15:24:43.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.341+0000 7f6c33a00700 1 -- 172.21.15.1:0/3144533223 wait complete. 2024-02-02T15:24:43.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.342+0000 7f6c33a00700 1 Processor -- start 2024-02-02T15:24:43.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.342+0000 7f6c33a00700 1 -- start start 2024-02-02T15:24:43.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.342+0000 7f6c33a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6c2c1017c0 0x7f6c2c101130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:43.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.342+0000 7f6c33a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c2c1040f0 0x7f6c2c0ff770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:43.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.342+0000 7f6c33a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6c2c106a20 0x7f6c2c0ffcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:43.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.342+0000 7f6c33a00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2c19a960 con 0x7f6c2c1017c0 2024-02-02T15:24:43.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.342+0000 7f6c33a00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2c1001f0 con 0x7f6c2c106a20 2024-02-02T15:24:43.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.342+0000 7f6c33a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2c1003b0 con 0x7f6c2c1040f0 2024-02-02T15:24:43.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.342+0000 7f6c2be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c2c1040f0 0x7f6c2c0ff770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:43.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.342+0000 7f6c31000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6c2c1017c0 0x7f6c2c101130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:43.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.342+0000 7f6c2be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c2c1040f0 0x7f6c2c0ff770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:49718/0 (socket says 172.21.15.1:49718) 2024-02-02T15:24:43.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.342+0000 7f6c2be00700 1 -- 172.21.15.1:0/1302858200 learned_addr learned my addr 172.21.15.1:0/1302858200 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:24:43.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.343+0000 7f6c31a00700 1 --2- 172.21.15.1:0/1302858200 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6c2c106a20 0x7f6c2c0ffcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:43.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.343+0000 7f6c2be00700 1 -- 172.21.15.1:0/1302858200 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6c2c106a20 msgr2=0x7f6c2c0ffcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:43.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.343+0000 7f6c2be00700 1 --2- 172.21.15.1:0/1302858200 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6c2c106a20 0x7f6c2c0ffcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:43.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.343+0000 7f6c2be00700 1 -- 172.21.15.1:0/1302858200 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6c2c1017c0 msgr2=0x7f6c2c101130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:43.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.343+0000 7f6c2be00700 1 --2- 172.21.15.1:0/1302858200 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6c2c1017c0 0x7f6c2c101130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:43.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.343+0000 7f6c2be00700 1 -- 172.21.15.1:0/1302858200 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c140095f0 con 0x7f6c2c1040f0 2024-02-02T15:24:43.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.343+0000 7f6c31000700 1 --2- 172.21.15.1:0/1302858200 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6c2c1017c0 0x7f6c2c101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:24:43.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.343+0000 7f6c2be00700 1 --2- 172.21.15.1:0/1302858200 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c2c1040f0 0x7f6c2c0ff770 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6c1400b680 tx=0x7f6c14009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:24:43.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.344+0000 7f6c29600700 1 -- 172.21.15.1:0/1302858200 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1401d070 con 0x7f6c2c1040f0 2024-02-02T15:24:43.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.344+0000 7f6c29600700 1 -- 172.21.15.1:0/1302858200 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c14021410 con 0x7f6c2c1040f0 2024-02-02T15:24:43.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.344+0000 7f6c29600700 1 -- 172.21.15.1:0/1302858200 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c14017650 con 0x7f6c2c1040f0 2024-02-02T15:24:43.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.344+0000 7f6c33a00700 1 -- 172.21.15.1:0/1302858200 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c2c1a5310 con 0x7f6c2c1040f0 2024-02-02T15:24:43.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.344+0000 7f6c33a00700 1 -- 172.21.15.1:0/1302858200 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c2c1a5680 con 0x7f6c2c1040f0 2024-02-02T15:24:43.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.345+0000 7f6c33a00700 1 -- 172.21.15.1:0/1302858200 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c2c0fcd30 con 0x7f6c2c1040f0 2024-02-02T15:24:43.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.348+0000 7f6c29600700 1 -- 172.21.15.1:0/1302858200 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 16) v1 ==== 135069+0+0 (secure 0 0 0) 0x7f6c14021580 con 0x7f6c2c1040f0 2024-02-02T15:24:43.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.349+0000 7f6c29600700 1 --2- 172.21.15.1:0/1302858200 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f6c0c0a0670 0x7f6c0c0a2b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:43.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.349+0000 7f6c29600700 1 -- 172.21.15.1:0/1302858200 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f6c140cc790 con 0x7f6c2c1040f0 2024-02-02T15:24:43.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.350+0000 7f6c31000700 1 --2- 172.21.15.1:0/1302858200 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f6c0c0a0670 0x7f6c0c0a2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:43.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.350+0000 7f6c31000700 1 --2- 172.21.15.1:0/1302858200 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f6c0c0a0670 0x7f6c0c0a2b30 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c011490 tx=0x7f6c1c009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:24:43.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.351+0000 7f6c29600700 1 -- 172.21.15.1:0/1302858200 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f6c14096840 con 0x7f6c2c1040f0 2024-02-02T15:24:43.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.669+0000 7f6c33a00700 1 -- 172.21.15.1:0/1302858200 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6c2c02cc70 con 0x7f6c2c1040f0 2024-02-02T15:24:43.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.670+0000 7f6c29600700 1 -- 172.21.15.1:0/1302858200 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+9571 (secure 0 0 0) 0x7f6c14099e60 con 0x7f6c2c1040f0 2024-02-02T15:24:43.670 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:24:43.670 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2922240448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2922240448},{"type":"v1","addr":"172.21.15.1:6835","nonce":2922240448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2729497431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2729497431},{"type":"v1","addr":"172.21.15.1:6837","nonce":2729497431}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14517,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/1218303683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":1218303683},{"type":"v1","addr":"172.21.15.1:6839","nonce":1218303683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14529,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/2059197496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2059197496},{"type":"v1","addr":"172.21.15.1:6841","nonce":2059197496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/7648986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":7648986},{"type":"v1","addr":"172.21.15.142:6833","nonce":7648986}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/3482746094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3482746094},{"type":"v1","addr":"172.21.15.71:6833","nonce":3482746094}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/388207814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":388207814},{"type":"v1","addr":"172.21.15.71:6835","nonce":388207814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24502,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/1052606547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1052606547},{"type":"v1","addr":"172.21.15.142:6835","nonce":1052606547}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24514,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/2870289019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2870289019},{"type":"v1","addr":"172.21.15.142:6837","nonce":2870289019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24518,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/628016199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":628016199},{"type":"v1","addr":"172.21.15.71:6837","nonce":628016199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24529,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6839/321675421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":321675421},{"type":"v1","addr":"172.21.15.142:6839","nonce":321675421}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24557,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6839/1268086493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1268086493},{"type":"v1","addr":"172.21.15.71:6839","nonce":1268086493}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[]} 2024-02-02T15:24:43.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.672+0000 7f6c33a00700 1 -- 172.21.15.1:0/1302858200 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f6c0c0a0670 msgr2=0x7f6c0c0a2b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:43.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.672+0000 7f6c33a00700 1 --2- 172.21.15.1:0/1302858200 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f6c0c0a0670 0x7f6c0c0a2b30 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c011490 tx=0x7f6c1c009260 comp rx=0 tx=0).stop 2024-02-02T15:24:43.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.672+0000 7f6c33a00700 1 -- 172.21.15.1:0/1302858200 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c2c1040f0 msgr2=0x7f6c2c0ff770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:43.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.672+0000 7f6c33a00700 1 --2- 172.21.15.1:0/1302858200 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c2c1040f0 0x7f6c2c0ff770 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6c1400b680 tx=0x7f6c14009580 comp rx=0 tx=0).stop 2024-02-02T15:24:43.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.672+0000 7f6c33a00700 1 -- 172.21.15.1:0/1302858200 shutdown_connections 2024-02-02T15:24:43.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.672+0000 7f6c33a00700 1 --2- 172.21.15.1:0/1302858200 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6c2c106a20 0x7f6c2c0ffcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:43.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.672+0000 7f6c33a00700 1 --2- 172.21.15.1:0/1302858200 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6c2c1017c0 0x7f6c2c101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:43.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.672+0000 7f6c33a00700 1 --2- 172.21.15.1:0/1302858200 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f6c0c0a0670 0x7f6c0c0a2b30 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:43.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.672+0000 7f6c33a00700 1 --2- 172.21.15.1:0/1302858200 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c2c1040f0 0x7f6c2c0ff770 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:43.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.672+0000 7f6c33a00700 1 -- 172.21.15.1:0/1302858200 >> 172.21.15.1:0/1302858200 conn(0x7f6c2c0fb1b0 msgr2=0x7f6c2c102430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:24:43.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.672+0000 7f6c33a00700 1 -- 172.21.15.1:0/1302858200 shutdown_connections 2024-02-02T15:24:43.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.672+0000 7f6c33a00700 1 -- 172.21.15.1:0/1302858200 wait complete. 2024-02-02T15:24:43.673 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 13 2024-02-02T15:24:43.691 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T15:24:43.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.894+0000 7fc4ef400700 1 -- 172.21.15.1:0/2623159434 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4e80737a0 msgr2=0x7fc4e8073c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:43.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.894+0000 7fc4ef400700 1 --2- 172.21.15.1:0/2623159434 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4e80737a0 0x7fc4e8073c20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc4d0009990 tx=0x7fc4d0009ca0 comp rx=0 tx=0).stop 2024-02-02T15:24:43.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.894+0000 7fc4ef400700 1 -- 172.21.15.1:0/2623159434 shutdown_connections 2024-02-02T15:24:43.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.894+0000 7fc4ef400700 1 --2- 172.21.15.1:0/2623159434 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4e8108c00 0x7fc4e810b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:43.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.894+0000 7fc4ef400700 1 --2- 172.21.15.1:0/2623159434 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4e8074d80 0x7fc4e80731d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:43.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.894+0000 7fc4ef400700 1 --2- 172.21.15.1:0/2623159434 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4e80737a0 0x7fc4e8073c20 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:43.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.894+0000 7fc4ef400700 1 -- 172.21.15.1:0/2623159434 >> 172.21.15.1:0/2623159434 conn(0x7fc4e80fc290 msgr2=0x7fc4e80fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:24:43.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.894+0000 7fc4ef400700 1 -- 172.21.15.1:0/2623159434 shutdown_connections 2024-02-02T15:24:43.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.894+0000 7fc4ef400700 1 -- 172.21.15.1:0/2623159434 wait complete. 2024-02-02T15:24:43.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.895+0000 7fc4ef400700 1 Processor -- start 2024-02-02T15:24:43.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.895+0000 7fc4ef400700 1 -- start start 2024-02-02T15:24:43.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.896+0000 7fc4ef400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4e80737a0 0x7fc4e8198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:43.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.896+0000 7fc4ef400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4e8074d80 0x7fc4e81991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:43.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.896+0000 7fc4ef400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4e8108c00 0x7fc4e81a0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:43.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.896+0000 7fc4ef400700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e81a07f0 con 0x7fc4e8074d80 2024-02-02T15:24:43.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.896+0000 7fc4ef400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e81a0960 con 0x7fc4e8108c00 2024-02-02T15:24:43.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.896+0000 7fc4ef400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e81a0b20 con 0x7fc4e80737a0 2024-02-02T15:24:43.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.896+0000 7fc4ed400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4e8108c00 0x7fc4e81a0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:43.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.896+0000 7fc4eca00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4e80737a0 0x7fc4e8198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:43.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.896+0000 7fc4e7e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4e8074d80 0x7fc4e81991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:43.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.896+0000 7fc4ed400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4e8108c00 0x7fc4e81a0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:46822/0 (socket says 172.21.15.1:46822) 2024-02-02T15:24:43.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.896+0000 7fc4eca00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4e80737a0 0x7fc4e8198c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:49722/0 (socket says 172.21.15.1:49722) 2024-02-02T15:24:43.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.896+0000 7fc4ed400700 1 -- 172.21.15.1:0/2163103893 learned_addr learned my addr 172.21.15.1:0/2163103893 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:24:43.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.897+0000 7fc4ed400700 1 -- 172.21.15.1:0/2163103893 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4e80737a0 msgr2=0x7fc4e8198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:43.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.897+0000 7fc4ed400700 1 --2- 172.21.15.1:0/2163103893 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4e80737a0 0x7fc4e8198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:43.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.897+0000 7fc4ed400700 1 -- 172.21.15.1:0/2163103893 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4e8074d80 msgr2=0x7fc4e81991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:43.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.897+0000 7fc4ed400700 1 --2- 172.21.15.1:0/2163103893 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4e8074d80 0x7fc4e81991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:43.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.897+0000 7fc4ed400700 1 -- 172.21.15.1:0/2163103893 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4d00095f0 con 0x7fc4e8108c00 2024-02-02T15:24:43.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.897+0000 7fc4eca00700 1 --2- 172.21.15.1:0/2163103893 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4e80737a0 0x7fc4e8198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:24:43.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.897+0000 7fc4ed400700 1 --2- 172.21.15.1:0/2163103893 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4e8108c00 0x7fc4e81a0220 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc4e000ef30 tx=0x7fc4e000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:24:43.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.897+0000 7fc4e5600700 1 -- 172.21.15.1:0/2163103893 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e0018070 con 0x7fc4e8108c00 2024-02-02T15:24:43.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.897+0000 7fc4e5600700 1 -- 172.21.15.1:0/2163103893 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc4e000f040 con 0x7fc4e8108c00 2024-02-02T15:24:43.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.897+0000 7fc4ef400700 1 -- 172.21.15.1:0/2163103893 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4e81a0da0 con 0x7fc4e8108c00 2024-02-02T15:24:43.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.897+0000 7fc4e5600700 1 -- 172.21.15.1:0/2163103893 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e000cd00 con 0x7fc4e8108c00 2024-02-02T15:24:43.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.897+0000 7fc4ef400700 1 -- 172.21.15.1:0/2163103893 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4e81a1270 con 0x7fc4e8108c00 2024-02-02T15:24:43.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.898+0000 7fc4ef400700 1 -- 172.21.15.1:0/2163103893 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4e81072b0 con 0x7fc4e8108c00 2024-02-02T15:24:43.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.900+0000 7fc4e5600700 1 -- 172.21.15.1:0/2163103893 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 135069+0+0 (secure 0 0 0) 0x7fc4e0003a00 con 0x7fc4e8108c00 2024-02-02T15:24:43.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.901+0000 7fc4e5600700 1 --2- 172.21.15.1:0/2163103893 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fc4c80a05a0 0x7fc4c80a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:43.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.902+0000 7fc4e5600700 1 -- 172.21.15.1:0/2163103893 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc4e00cafc0 con 0x7fc4e8108c00 2024-02-02T15:24:43.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.902+0000 7fc4eca00700 1 --2- 172.21.15.1:0/2163103893 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fc4c80a05a0 0x7fc4c80a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:43.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.902+0000 7fc4eca00700 1 --2- 172.21.15.1:0/2163103893 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fc4c80a05a0 0x7fc4c80a2a60 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8199b50 tx=0x7fc4d8009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:24:43.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:43.905+0000 7fc4e5600700 1 -- 172.21.15.1:0/2163103893 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7fc4e0094ff0 con 0x7fc4e8108c00 2024-02-02T15:24:43.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:43 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:43.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:43 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:43.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:43 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:43.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:43 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:43.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:43 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:43.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:43 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:43.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:43 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:24:43.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:43 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:24:43.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:43 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:43.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:43 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/1302858200' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:24:43.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:43 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:24:44.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.145+0000 7fc4ef400700 1 -- 172.21.15.1:0/2163103893 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc4e802cc70 con 0x7fc4e8108c00 2024-02-02T15:24:44.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.146+0000 7fc4e5600700 1 -- 172.21.15.1:0/2163103893 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) v1 ==== 74+0+18303 (secure 0 0 0) 0x7fc4e0098610 con 0x7fc4e8108c00 2024-02-02T15:24:44.146 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:24:44.146 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":76,"fsid":"371a409c-c1de-11ee-95b6-87774f69a715","created":"2024-02-02T15:18:26.746506+0000","modified":"2024-02-02T15:24:09.807430+0000","last_up_change":"2024-02-02T15:24:08.800951+0000","last_in_change":"2024-02-02T15:23:52.784280+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T15:21:22.471754+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"73e8a07f-3560-4c19-85ed-030d4d8f0f23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6803","nonce":2591906423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6805","nonce":2591906423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6809","nonce":2591906423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6807","nonce":2591906423}]},"public_addr":"172.21.15.1:6803/2591906423","cluster_addr":"172.21.15.1:6805/2591906423","heartbeat_back_addr":"172.21.15.1:6809/2591906423","heartbeat_front_addr":"172.21.15.1:6807/2591906423","state":["exists","up"]},{"osd":1,"uuid":"e53cffec-0ff9-4ed9-8dcf-ff47d04db2bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6801","nonce":747581719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6803","nonce":747581719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6807","nonce":747581719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6805","nonce":747581719}]},"public_addr":"172.21.15.71:6801/747581719","cluster_addr":"172.21.15.71:6803/747581719","heartbeat_back_addr":"172.21.15.71:6807/747581719","heartbeat_front_addr":"172.21.15.71:6805/747581719","state":["exists","up"]},{"osd":2,"uuid":"856c2347-294c-482b-8595-ad77f4792db4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6801","nonce":377511414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6803","nonce":377511414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6807","nonce":377511414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6805","nonce":377511414}]},"public_addr":"172.21.15.142:6801/377511414","cluster_addr":"172.21.15.142:6803/377511414","heartbeat_back_addr":"172.21.15.142:6807/377511414","heartbeat_front_addr":"172.21.15.142:6805/377511414","state":["exists","up"]},{"osd":3,"uuid":"4b946ad2-2da9-4e0b-9d61-d039d8669a42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6811","nonce":1086124477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6813","nonce":1086124477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6817","nonce":1086124477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6815","nonce":1086124477}]},"public_addr":"172.21.15.1:6811/1086124477","cluster_addr":"172.21.15.1:6813/1086124477","heartbeat_back_addr":"172.21.15.1:6817/1086124477","heartbeat_front_addr":"172.21.15.1:6815/1086124477","state":["exists","up"]},{"osd":4,"uuid":"43e36c43-47a0-4634-a45b-1da14a0e19d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6809","nonce":3095380305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6811","nonce":3095380305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6815","nonce":3095380305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6813","nonce":3095380305}]},"public_addr":"172.21.15.71:6809/3095380305","cluster_addr":"172.21.15.71:6811/3095380305","heartbeat_back_addr":"172.21.15.71:6815/3095380305","heartbeat_front_addr":"172.21.15.71:6813/3095380305","state":["exists","up"]},{"osd":5,"uuid":"eb48160e-ea7c-42f7-9477-be110bc8a34e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6809","nonce":4195149262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6811","nonce":4195149262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6815","nonce":4195149262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6813","nonce":4195149262}]},"public_addr":"172.21.15.142:6809/4195149262","cluster_addr":"172.21.15.142:6811/4195149262","heartbeat_back_addr":"172.21.15.142:6815/4195149262","heartbeat_front_addr":"172.21.15.142:6813/4195149262","state":["exists","up"]},{"osd":6,"uuid":"0ec4501a-5853-4ca6-b325-231f5b72335e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6819","nonce":1560153428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6821","nonce":1560153428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6825","nonce":1560153428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6823","nonce":1560153428}]},"public_addr":"172.21.15.1:6819/1560153428","cluster_addr":"172.21.15.1:6821/1560153428","heartbeat_back_addr":"172.21.15.1:6825/1560153428","heartbeat_front_addr":"172.21.15.1:6823/1560153428","state":["exists","up"]},{"osd":7,"uuid":"029debe8-9cdf-44f6-9aea-cabd1d185f93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6817","nonce":4080955538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6819","nonce":4080955538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6823","nonce":4080955538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6821","nonce":4080955538}]},"public_addr":"172.21.15.71:6817/4080955538","cluster_addr":"172.21.15.71:6819/4080955538","heartbeat_back_addr":"172.21.15.71:6823/4080955538","heartbeat_front_addr":"172.21.15.71:6821/4080955538","state":["exists","up"]},{"osd":8,"uuid":"7c90896c-9955-4c1b-aaca-7531dd5457f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6817","nonce":3942038209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6819","nonce":3942038209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6823","nonce":3942038209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6821","nonce":3942038209}]},"public_addr":"172.21.15.142:6817/3942038209","cluster_addr":"172.21.15.142:6819/3942038209","heartbeat_back_addr":"172.21.15.142:6823/3942038209","heartbeat_front_addr":"172.21.15.142:6821/3942038209","state":["exists","up"]},{"osd":9,"uuid":"7e165e68-31a8-4598-a577-de3e90ae5c5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6827","nonce":2716544850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6829","nonce":2716544850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6833","nonce":2716544850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6831","nonce":2716544850}]},"public_addr":"172.21.15.1:6827/2716544850","cluster_addr":"172.21.15.1:6829/2716544850","heartbeat_back_addr":"172.21.15.1:6833/2716544850","heartbeat_front_addr":"172.21.15.1:6831/2716544850","state":["exists","up"]},{"osd":10,"uuid":"48cd5fdb-0887-444a-aab8-a1de24282bb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6825","nonce":84691841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6827","nonce":84691841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6831","nonce":84691841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6829","nonce":84691841}]},"public_addr":"172.21.15.71:6825/84691841","cluster_addr":"172.21.15.71:6827/84691841","heartbeat_back_addr":"172.21.15.71:6831/84691841","heartbeat_front_addr":"172.21.15.71:6829/84691841","state":["exists","up"]},{"osd":11,"uuid":"f011dd27-11b0-4166-adfe-76756533163a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6825","nonce":2290179934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6827","nonce":2290179934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6831","nonce":2290179934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6829","nonce":2290179934}]},"public_addr":"172.21.15.142:6825/2290179934","cluster_addr":"172.21.15.142:6827/2290179934","heartbeat_back_addr":"172.21.15.142:6831/2290179934","heartbeat_front_addr":"172.21.15.142:6829/2290179934","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:20:44.060692+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:01.903817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:19.368258+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:40.028037+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:57.513361+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:22:14.805944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:22:35.188439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:22:53.014412+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:23:10.507420+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:23:31.849765+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:23:50.070612+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:24:07.388625+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/4092532060":"2024-02-03T15:19:25.836171+0000","172.21.15.1:6801/395498296":"2024-02-03T15:19:25.836171+0000","172.21.15.1:6800/395498296":"2024-02-03T15:19:25.836171+0000","172.21.15.1:0/1575373489":"2024-02-03T15:19:25.836171+0000","172.21.15.1:0/865983374":"2024-02-03T15:19:01.431596+0000","172.21.15.1:0/3165960227":"2024-02-03T15:19:01.431596+0000","172.21.15.1:0/2350830293":"2024-02-03T15:19:01.431596+0000","172.21.15.1:0/2959690250":"2024-02-03T15:19:25.836171+0000","172.21.15.1:6800/1162530732":"2024-02-03T15:19:01.431596+0000","172.21.15.1:6801/1162530732":"2024-02-03T15:19:01.431596+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T15:24:44.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.149+0000 7fc4ef400700 1 -- 172.21.15.1:0/2163103893 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fc4c80a05a0 msgr2=0x7fc4c80a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:44.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.149+0000 7fc4ef400700 1 --2- 172.21.15.1:0/2163103893 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fc4c80a05a0 0x7fc4c80a2a60 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8199b50 tx=0x7fc4d8009260 comp rx=0 tx=0).stop 2024-02-02T15:24:44.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.149+0000 7fc4ef400700 1 -- 172.21.15.1:0/2163103893 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4e8108c00 msgr2=0x7fc4e81a0220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:44.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.149+0000 7fc4ef400700 1 --2- 172.21.15.1:0/2163103893 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4e8108c00 0x7fc4e81a0220 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc4e000ef30 tx=0x7fc4e000c3a0 comp rx=0 tx=0).stop 2024-02-02T15:24:44.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.149+0000 7fc4ef400700 1 -- 172.21.15.1:0/2163103893 shutdown_connections 2024-02-02T15:24:44.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.149+0000 7fc4ef400700 1 --2- 172.21.15.1:0/2163103893 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4e8108c00 0x7fc4e81a0220 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:44.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.149+0000 7fc4ef400700 1 --2- 172.21.15.1:0/2163103893 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fc4c80a05a0 0x7fc4c80a2a60 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:44.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.150+0000 7fc4ef400700 1 --2- 172.21.15.1:0/2163103893 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4e80737a0 0x7fc4e8198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:44.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.150+0000 7fc4ef400700 1 --2- 172.21.15.1:0/2163103893 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4e8074d80 0x7fc4e81991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:44.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.150+0000 7fc4ef400700 1 -- 172.21.15.1:0/2163103893 >> 172.21.15.1:0/2163103893 conn(0x7fc4e80fc290 msgr2=0x7fc4e80fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:24:44.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.150+0000 7fc4ef400700 1 -- 172.21.15.1:0/2163103893 shutdown_connections 2024-02-02T15:24:44.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.150+0000 7fc4ef400700 1 -- 172.21.15.1:0/2163103893 wait complete. 2024-02-02T15:24:44.169 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:43 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:44.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:43 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:44.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:43 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:44.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:43 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:44.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:43 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:44.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:43 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:44.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:43 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:24:44.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:43 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:24:44.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:43 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:44.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:43 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/1302858200' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:24:44.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:43 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:24:44.179 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-02T15:21:22.471754+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-02T15:24:44.179 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-02T15:24:44.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:43 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:44.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:43 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:44.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:43 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:44.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:43 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:44.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:43 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:44.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:43 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:44.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:43 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:24:44.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:43 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:24:44.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:43 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:44.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:43 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/1302858200' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:24:44.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:43 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:24:44.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.404+0000 7f9882800700 1 -- 172.21.15.1:0/2763783040 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f987c0ffd30 msgr2=0x7f987c100150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:44.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.404+0000 7f9882800700 1 --2- 172.21.15.1:0/2763783040 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f987c0ffd30 0x7f987c100150 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f986c0099c0 tx=0x7f986c009cd0 comp rx=0 tx=0).stop 2024-02-02T15:24:44.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.404+0000 7f9882800700 1 -- 172.21.15.1:0/2763783040 shutdown_connections 2024-02-02T15:24:44.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.404+0000 7f9882800700 1 --2- 172.21.15.1:0/2763783040 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f987c1018c0 0x7f987c108170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:44.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.404+0000 7f9882800700 1 --2- 172.21.15.1:0/2763783040 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f987c0ffd30 0x7f987c100150 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:44.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.404+0000 7f9882800700 1 --2- 172.21.15.1:0/2763783040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f987c100f20 0x7f987c101380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:44.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.404+0000 7f9882800700 1 -- 172.21.15.1:0/2763783040 >> 172.21.15.1:0/2763783040 conn(0x7f987c0fb2a0 msgr2=0x7f987c0fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:24:44.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.404+0000 7f9882800700 1 -- 172.21.15.1:0/2763783040 shutdown_connections 2024-02-02T15:24:44.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.404+0000 7f9882800700 1 -- 172.21.15.1:0/2763783040 wait complete. 2024-02-02T15:24:44.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.405+0000 7f9882800700 1 Processor -- start 2024-02-02T15:24:44.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.406+0000 7f9882800700 1 -- start start 2024-02-02T15:24:44.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.406+0000 7f9882800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f987c0ffd30 0x7f987c103cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:44.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.406+0000 7f9882800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f987c100f20 0x7f987c102300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:44.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.406+0000 7f9882800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f987c1018c0 0x7f987c102840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:44.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.406+0000 7f9882800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f987c102fe0 con 0x7f987c1018c0 2024-02-02T15:24:44.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.406+0000 7f9882800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f987c103150 con 0x7f987c0ffd30 2024-02-02T15:24:44.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.406+0000 7f9882800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f987c1a0b70 con 0x7f987c100f20 2024-02-02T15:24:44.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.406+0000 7f987b400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f987c0ffd30 0x7f987c103cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:44.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.406+0000 7f987aa00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f987c100f20 0x7f987c102300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:44.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.406+0000 7f987b400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f987c0ffd30 0x7f987c103cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:46846/0 (socket says 172.21.15.1:46846) 2024-02-02T15:24:44.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.406+0000 7f987b400700 1 -- 172.21.15.1:0/176713365 learned_addr learned my addr 172.21.15.1:0/176713365 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:24:44.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.406+0000 7f987be00700 1 --2- 172.21.15.1:0/176713365 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f987c1018c0 0x7f987c102840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:44.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.406+0000 7f987b400700 1 -- 172.21.15.1:0/176713365 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f987c100f20 msgr2=0x7f987c102300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:44.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.406+0000 7f987b400700 1 --2- 172.21.15.1:0/176713365 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f987c100f20 0x7f987c102300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:44.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.406+0000 7f987b400700 1 -- 172.21.15.1:0/176713365 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f987c1018c0 msgr2=0x7f987c102840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:44.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.406+0000 7f987b400700 1 --2- 172.21.15.1:0/176713365 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f987c1018c0 0x7f987c102840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:44.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.407+0000 7f987b400700 1 -- 172.21.15.1:0/176713365 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9864009520 con 0x7f987c0ffd30 2024-02-02T15:24:44.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.407+0000 7f987b400700 1 --2- 172.21.15.1:0/176713365 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f987c0ffd30 0x7f987c103cc0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f986c009990 tx=0x7f986c00f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:24:44.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.407+0000 7f9873e00700 1 -- 172.21.15.1:0/176713365 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f986c01d070 con 0x7f987c0ffd30 2024-02-02T15:24:44.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.407+0000 7f9873e00700 1 -- 172.21.15.1:0/176713365 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f986c005030 con 0x7f987c0ffd30 2024-02-02T15:24:44.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.407+0000 7f9873e00700 1 -- 172.21.15.1:0/176713365 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f986c004a70 con 0x7f987c0ffd30 2024-02-02T15:24:44.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.407+0000 7f9882800700 1 -- 172.21.15.1:0/176713365 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f986c0095f0 con 0x7f987c0ffd30 2024-02-02T15:24:44.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.407+0000 7f9882800700 1 -- 172.21.15.1:0/176713365 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f987c1a1030 con 0x7f987c0ffd30 2024-02-02T15:24:44.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.408+0000 7f9882800700 1 -- 172.21.15.1:0/176713365 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f987c10b590 con 0x7f987c0ffd30 2024-02-02T15:24:44.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.410+0000 7f9873e00700 1 -- 172.21.15.1:0/176713365 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 135069+0+0 (secure 0 0 0) 0x7f986c005410 con 0x7f987c0ffd30 2024-02-02T15:24:44.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.411+0000 7f9873e00700 1 --2- 172.21.15.1:0/176713365 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f985c0a0520 0x7f985c0a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:44.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.411+0000 7f9873e00700 1 -- 172.21.15.1:0/176713365 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f986c0cc300 con 0x7f987c0ffd30 2024-02-02T15:24:44.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.412+0000 7f987aa00700 1 --2- 172.21.15.1:0/176713365 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f985c0a0520 0x7f985c0a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:44.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.412+0000 7f987aa00700 1 --2- 172.21.15.1:0/176713365 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f985c0a0520 0x7f985c0a29e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9864009df0 tx=0x7f9864009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:24:44.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.415+0000 7f9873e00700 1 -- 172.21.15.1:0/176713365 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f986c096330 con 0x7f987c0ffd30 2024-02-02T15:24:44.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.662+0000 7f9882800700 1 -- 172.21.15.1:0/176713365 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f987c1a15e0 con 0x7f987c0ffd30 2024-02-02T15:24:44.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.662+0000 7f9873e00700 1 -- 172.21.15.1:0/176713365 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v76) v1 ==== 93+0+10 (secure 0 0 0) 0x7f986c099950 con 0x7f987c0ffd30 2024-02-02T15:24:44.662 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-02-02T15:24:44.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.665+0000 7f9882800700 1 -- 172.21.15.1:0/176713365 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f985c0a0520 msgr2=0x7f985c0a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:44.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.665+0000 7f9882800700 1 --2- 172.21.15.1:0/176713365 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f985c0a0520 0x7f985c0a29e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9864009df0 tx=0x7f9864009260 comp rx=0 tx=0).stop 2024-02-02T15:24:44.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.665+0000 7f9882800700 1 -- 172.21.15.1:0/176713365 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f987c0ffd30 msgr2=0x7f987c103cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:44.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.665+0000 7f9882800700 1 --2- 172.21.15.1:0/176713365 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f987c0ffd30 0x7f987c103cc0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f986c009990 tx=0x7f986c00f690 comp rx=0 tx=0).stop 2024-02-02T15:24:44.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.665+0000 7f9882800700 1 -- 172.21.15.1:0/176713365 shutdown_connections 2024-02-02T15:24:44.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.665+0000 7f9882800700 1 --2- 172.21.15.1:0/176713365 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f987c1018c0 0x7f987c102840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:44.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.665+0000 7f9882800700 1 --2- 172.21.15.1:0/176713365 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f987c0ffd30 0x7f987c103cc0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:44.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.665+0000 7f9882800700 1 --2- 172.21.15.1:0/176713365 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f985c0a0520 0x7f985c0a29e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:44.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.665+0000 7f9882800700 1 --2- 172.21.15.1:0/176713365 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f987c100f20 0x7f987c102300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:44.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.665+0000 7f9882800700 1 -- 172.21.15.1:0/176713365 >> 172.21.15.1:0/176713365 conn(0x7f987c0fb2a0 msgr2=0x7f987c068da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:24:44.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.665+0000 7f9882800700 1 -- 172.21.15.1:0/176713365 shutdown_connections 2024-02-02T15:24:44.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.666+0000 7f9882800700 1 -- 172.21.15.1:0/176713365 wait complete. 2024-02-02T15:24:44.686 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-02-02T15:24:44.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.902+0000 7ff44fc00700 1 -- 172.21.15.1:0/2959269502 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff448074d80 msgr2=0x7ff4480731d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:44.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.902+0000 7ff44fc00700 1 --2- 172.21.15.1:0/2959269502 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff448074d80 0x7ff4480731d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff438009990 tx=0x7ff438009ca0 comp rx=0 tx=0).stop 2024-02-02T15:24:44.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.902+0000 7ff44fc00700 1 -- 172.21.15.1:0/2959269502 shutdown_connections 2024-02-02T15:24:44.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.902+0000 7ff44fc00700 1 --2- 172.21.15.1:0/2959269502 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff44810d320 0x7ff44810f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:44.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.902+0000 7ff44fc00700 1 --2- 172.21.15.1:0/2959269502 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff448074d80 0x7ff4480731d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:44.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.902+0000 7ff44fc00700 1 --2- 172.21.15.1:0/2959269502 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4480737a0 0x7ff448073c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:44.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.902+0000 7ff44fc00700 1 -- 172.21.15.1:0/2959269502 >> 172.21.15.1:0/2959269502 conn(0x7ff4480fc2c0 msgr2=0x7ff4480fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:24:44.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.902+0000 7ff44fc00700 1 -- 172.21.15.1:0/2959269502 shutdown_connections 2024-02-02T15:24:44.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.902+0000 7ff44fc00700 1 -- 172.21.15.1:0/2959269502 wait complete. 2024-02-02T15:24:44.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.903+0000 7ff44fc00700 1 Processor -- start 2024-02-02T15:24:44.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.904+0000 7ff44fc00700 1 -- start start 2024-02-02T15:24:44.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.904+0000 7ff44fc00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4480737a0 0x7ff44819d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:44.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.904+0000 7ff44fc00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff448074d80 0x7ff44819d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:44.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.904+0000 7ff44fc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff44810d320 0x7ff4481a4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:44.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.904+0000 7ff44fc00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4481a4be0 con 0x7ff448074d80 2024-02-02T15:24:44.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.904+0000 7ff44fc00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4481a4d20 con 0x7ff4480737a0 2024-02-02T15:24:44.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.904+0000 7ff44fc00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4481a4eb0 con 0x7ff44810d320 2024-02-02T15:24:44.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.904+0000 7ff44d200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4480737a0 0x7ff44819d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:44.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.904+0000 7ff44d200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4480737a0 0x7ff44819d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:46868/0 (socket says 172.21.15.1:46868) 2024-02-02T15:24:44.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.904+0000 7ff44d200700 1 -- 172.21.15.1:0/3985063156 learned_addr learned my addr 172.21.15.1:0/3985063156 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:24:44.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.904+0000 7ff447e00700 1 --2- 172.21.15.1:0/3985063156 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff448074d80 0x7ff44819d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:44.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.904+0000 7ff44dc00700 1 --2- 172.21.15.1:0/3985063156 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff44810d320 0x7ff4481a4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:44.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.905+0000 7ff44d200700 1 -- 172.21.15.1:0/3985063156 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff44810d320 msgr2=0x7ff4481a4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:44.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.905+0000 7ff44d200700 1 --2- 172.21.15.1:0/3985063156 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff44810d320 0x7ff4481a4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:44.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.905+0000 7ff44d200700 1 -- 172.21.15.1:0/3985063156 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff448074d80 msgr2=0x7ff44819d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:44.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.905+0000 7ff44d200700 1 --2- 172.21.15.1:0/3985063156 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff448074d80 0x7ff44819d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:44.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.905+0000 7ff44d200700 1 -- 172.21.15.1:0/3985063156 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4380095f0 con 0x7ff4480737a0 2024-02-02T15:24:44.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.905+0000 7ff44d200700 1 --2- 172.21.15.1:0/3985063156 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4480737a0 0x7ff44819d060 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff43800b680 tx=0x7ff4380057b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:24:44.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.905+0000 7ff445600700 1 -- 172.21.15.1:0/3985063156 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff43801d070 con 0x7ff4480737a0 2024-02-02T15:24:44.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.905+0000 7ff445600700 1 -- 172.21.15.1:0/3985063156 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff438022470 con 0x7ff4480737a0 2024-02-02T15:24:44.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.905+0000 7ff44fc00700 1 -- 172.21.15.1:0/3985063156 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4481a5130 con 0x7ff4480737a0 2024-02-02T15:24:44.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.905+0000 7ff445600700 1 -- 172.21.15.1:0/3985063156 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff438017650 con 0x7ff4480737a0 2024-02-02T15:24:44.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.905+0000 7ff44fc00700 1 -- 172.21.15.1:0/3985063156 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4481a5550 con 0x7ff4480737a0 2024-02-02T15:24:44.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.906+0000 7ff44fc00700 1 -- 172.21.15.1:0/3985063156 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff448102e70 con 0x7ff4480737a0 2024-02-02T15:24:44.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.908+0000 7ff445600700 1 -- 172.21.15.1:0/3985063156 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 135069+0+0 (secure 0 0 0) 0x7ff4380225e0 con 0x7ff4480737a0 2024-02-02T15:24:44.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.909+0000 7ff445600700 1 --2- 172.21.15.1:0/3985063156 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7ff4280a0520 0x7ff4280a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:44.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.910+0000 7ff445600700 1 -- 172.21.15.1:0/3985063156 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff4380cd110 con 0x7ff4480737a0 2024-02-02T15:24:44.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.910+0000 7ff447e00700 1 --2- 172.21.15.1:0/3985063156 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7ff4280a0520 0x7ff4280a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:44.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.910+0000 7ff447e00700 1 --2- 172.21.15.1:0/3985063156 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7ff4280a0520 0x7ff4280a29e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff448074aa0 tx=0x7ff430009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:24:44.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:44.912+0000 7ff445600700 1 -- 172.21.15.1:0/3985063156 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7ff438097140 con 0x7ff4480737a0 2024-02-02T15:24:44.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:44 smithi001 ceph-mon[35470]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-02T15:24:44.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:44 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/2163103893' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-02T15:24:44.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:44 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/176713365' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-02T15:24:45.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:44 smithi142 ceph-mon[45728]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-02T15:24:45.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:44 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/2163103893' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-02T15:24:45.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:44 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/176713365' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-02T15:24:45.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:45.180+0000 7ff44fc00700 1 -- 172.21.15.1:0/3985063156 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7ff44802cec0 con 0x7ff4480737a0 2024-02-02T15:24:45.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:45.180+0000 7ff445600700 1 -- 172.21.15.1:0/3985063156 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v76) v1 ==== 84+0+65 (secure 0 0 0) 0x7ff43809a760 con 0x7ff4480737a0 2024-02-02T15:24:45.181 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:24:45.181 INFO:teuthology.orchestra.run.smithi001.stdout:[ 2024-02-02T15:24:45.181 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:24:45.181 INFO:teuthology.orchestra.run.smithi001.stdout: "poolnum": 1, 2024-02-02T15:24:45.181 INFO:teuthology.orchestra.run.smithi001.stdout: "poolname": ".mgr" 2024-02-02T15:24:45.181 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:24:45.181 INFO:teuthology.orchestra.run.smithi001.stdout:] 2024-02-02T15:24:45.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:45.183+0000 7ff44fc00700 1 -- 172.21.15.1:0/3985063156 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7ff4280a0520 msgr2=0x7ff4280a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:45.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:45.183+0000 7ff44fc00700 1 --2- 172.21.15.1:0/3985063156 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7ff4280a0520 0x7ff4280a29e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff448074aa0 tx=0x7ff430009260 comp rx=0 tx=0).stop 2024-02-02T15:24:45.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:45.183+0000 7ff44fc00700 1 -- 172.21.15.1:0/3985063156 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4480737a0 msgr2=0x7ff44819d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:45.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:45.183+0000 7ff44fc00700 1 --2- 172.21.15.1:0/3985063156 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4480737a0 0x7ff44819d060 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff43800b680 tx=0x7ff4380057b0 comp rx=0 tx=0).stop 2024-02-02T15:24:45.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:45.183+0000 7ff44fc00700 1 -- 172.21.15.1:0/3985063156 shutdown_connections 2024-02-02T15:24:45.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:45.183+0000 7ff44fc00700 1 --2- 172.21.15.1:0/3985063156 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7ff4280a0520 0x7ff4280a29e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:45.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:45.183+0000 7ff44fc00700 1 --2- 172.21.15.1:0/3985063156 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff44810d320 0x7ff4481a4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:45.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:45.183+0000 7ff44fc00700 1 --2- 172.21.15.1:0/3985063156 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4480737a0 0x7ff44819d060 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:45.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:45.183+0000 7ff44fc00700 1 --2- 172.21.15.1:0/3985063156 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff448074d80 0x7ff44819d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:45.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:45.183+0000 7ff44fc00700 1 -- 172.21.15.1:0/3985063156 >> 172.21.15.1:0/3985063156 conn(0x7ff4480fc2c0 msgr2=0x7ff4480fe6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:24:45.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:45.183+0000 7ff44fc00700 1 -- 172.21.15.1:0/3985063156 shutdown_connections 2024-02-02T15:24:45.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:45.184+0000 7ff44fc00700 1 -- 172.21.15.1:0/3985063156 wait complete. 2024-02-02T15:24:45.205 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-02-02T15:24:45.205 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-02-02T15:24:45.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:44 smithi071 ceph-mon[44707]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-02T15:24:45.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:44 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/2163103893' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-02T15:24:45.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:44 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/176713365' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-02T15:24:45.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:45.426+0000 7fac7fe00700 1 -- 172.21.15.1:0/3395398015 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fac781041c0 msgr2=0x7fac781065b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:45.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:45.426+0000 7fac7fe00700 1 --2- 172.21.15.1:0/3395398015 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fac781041c0 0x7fac781065b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fac60009990 tx=0x7fac60009ca0 comp rx=0 tx=0).stop 2024-02-02T15:24:45.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:45.426+0000 7fac7fe00700 1 -- 172.21.15.1:0/3395398015 shutdown_connections 2024-02-02T15:24:45.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:45.426+0000 7fac7fe00700 1 --2- 172.21.15.1:0/3395398015 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fac78106af0 0x7fac78108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:45.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:45.426+0000 7fac7fe00700 1 --2- 172.21.15.1:0/3395398015 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac78101890 0x7fac78103c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:45.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:45.426+0000 7fac7fe00700 1 --2- 172.21.15.1:0/3395398015 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fac781041c0 0x7fac781065b0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:45.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:45.426+0000 7fac7fe00700 1 -- 172.21.15.1:0/3395398015 >> 172.21.15.1:0/3395398015 conn(0x7fac780fb260 msgr2=0x7fac780fd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:24:45.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:45.427+0000 7fac7fe00700 1 -- 172.21.15.1:0/3395398015 shutdown_connections 2024-02-02T15:24:45.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:45.427+0000 7fac7fe00700 1 -- 172.21.15.1:0/3395398015 wait complete. 2024-02-02T15:24:45.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:45.428+0000 7fac7fe00700 1 Processor -- start 2024-02-02T15:24:45.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:45.428+0000 7fac7fe00700 1 -- start start 2024-02-02T15:24:45.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:45.428+0000 7fac7fe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac78101890 0x7fac78101200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:45.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:45.428+0000 7fac7fe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fac781041c0 0x7fac780ff840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:45.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:45.428+0000 7fac7fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fac78106af0 0x7fac780ffd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:45.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:45.428+0000 7fac7fe00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac78100520 con 0x7fac781041c0 2024-02-02T15:24:45.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:45.428+0000 7fac7fe00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac78100690 con 0x7fac78101890 2024-02-02T15:24:45.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:45.428+0000 7fac7fe00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac781a0b30 con 0x7fac78106af0 2024-02-02T15:24:45.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:45.428+0000 7fac7d400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac78101890 0x7fac78101200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:45.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:45.428+0000 7fac7ca00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fac781041c0 0x7fac780ff840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:45.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:45.428+0000 7fac7d400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac78101890 0x7fac78101200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:46888/0 (socket says 172.21.15.1:46888) 2024-02-02T15:24:45.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:45.428+0000 7fac7ca00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fac781041c0 0x7fac780ff840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53234/0 (socket says 172.21.15.1:53234) 2024-02-02T15:24:45.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:45.428+0000 7fac7d400700 1 -- 172.21.15.1:0/1129526482 learned_addr learned my addr 172.21.15.1:0/1129526482 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:24:45.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:45.428+0000 7fac7de00700 1 --2- 172.21.15.1:0/1129526482 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fac78106af0 0x7fac780ffd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:45.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:45.429+0000 7fac7d400700 1 -- 172.21.15.1:0/1129526482 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fac78106af0 msgr2=0x7fac780ffd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:45.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:45.429+0000 7fac7d400700 1 --2- 172.21.15.1:0/1129526482 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fac78106af0 0x7fac780ffd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:45.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:45.429+0000 7fac7d400700 1 -- 172.21.15.1:0/1129526482 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fac781041c0 msgr2=0x7fac780ff840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:45.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:45.429+0000 7fac7d400700 1 --2- 172.21.15.1:0/1129526482 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fac781041c0 0x7fac780ff840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:45.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:45.429+0000 7fac7d400700 1 -- 172.21.15.1:0/1129526482 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac600095f0 con 0x7fac78101890 2024-02-02T15:24:45.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:45.429+0000 7fac7de00700 1 --2- 172.21.15.1:0/1129526482 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fac78106af0 0x7fac780ffd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:24:45.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:45.429+0000 7fac7d400700 1 --2- 172.21.15.1:0/1129526482 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac78101890 0x7fac78101200 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fac6800e9a0 tx=0x7fac6800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:24:45.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:45.429+0000 7fac76000700 1 -- 172.21.15.1:0/1129526482 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac68009790 con 0x7fac78101890 2024-02-02T15:24:45.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:45.429+0000 7fac76000700 1 -- 172.21.15.1:0/1129526482 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fac68010430 con 0x7fac78101890 2024-02-02T15:24:45.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:45.429+0000 7fac76000700 1 -- 172.21.15.1:0/1129526482 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac68018680 con 0x7fac78101890 2024-02-02T15:24:45.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:45.429+0000 7fac7fe00700 1 -- 172.21.15.1:0/1129526482 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac781a0e10 con 0x7fac78101890 2024-02-02T15:24:45.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:45.430+0000 7fac7fe00700 1 -- 172.21.15.1:0/1129526482 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac781a11e0 con 0x7fac78101890 2024-02-02T15:24:45.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:45.431+0000 7fac7fe00700 1 -- 172.21.15.1:0/1129526482 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac7810c650 con 0x7fac78101890 2024-02-02T15:24:45.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:45.434+0000 7fac76000700 1 -- 172.21.15.1:0/1129526482 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 135069+0+0 (secure 0 0 0) 0x7fac680105a0 con 0x7fac78101890 2024-02-02T15:24:45.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:45.436+0000 7fac76000700 1 --2- 172.21.15.1:0/1129526482 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fac580a05a0 0x7fac580a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:45.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:45.436+0000 7fac76000700 1 -- 172.21.15.1:0/1129526482 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fac68014070 con 0x7fac78101890 2024-02-02T15:24:45.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:45.436+0000 7fac7ca00700 1 --2- 172.21.15.1:0/1129526482 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fac580a05a0 0x7fac580a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:45.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:45.437+0000 7fac7ca00700 1 --2- 172.21.15.1:0/1129526482 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fac580a05a0 0x7fac580a2a60 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fac6000b2e0 tx=0x7fac60005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:24:45.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:45.439+0000 7fac76000700 1 -- 172.21.15.1:0/1129526482 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7fac68052110 con 0x7fac78101890 2024-02-02T15:24:45.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:45.665+0000 7fac7fe00700 1 -- 172.21.15.1:0/1129526482 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fac781a18b0 con 0x7fac78101890 2024-02-02T15:24:46.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:46.251+0000 7fac76000700 1 -- 172.21.15.1:0/1129526482 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v77) v1 ==== 138+0+0 (secure 0 0 0) 0x7fac680990b0 con 0x7fac78101890 2024-02-02T15:24:46.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:46.256+0000 7fac7fe00700 1 -- 172.21.15.1:0/1129526482 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fac580a05a0 msgr2=0x7fac580a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:46.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:46.256+0000 7fac7fe00700 1 --2- 172.21.15.1:0/1129526482 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fac580a05a0 0x7fac580a2a60 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fac6000b2e0 tx=0x7fac60005fb0 comp rx=0 tx=0).stop 2024-02-02T15:24:46.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:46.256+0000 7fac7fe00700 1 -- 172.21.15.1:0/1129526482 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac78101890 msgr2=0x7fac78101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:46.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:46.256+0000 7fac7fe00700 1 --2- 172.21.15.1:0/1129526482 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac78101890 0x7fac78101200 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fac6800e9a0 tx=0x7fac6800ecb0 comp rx=0 tx=0).stop 2024-02-02T15:24:46.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:46.256+0000 7fac7fe00700 1 -- 172.21.15.1:0/1129526482 shutdown_connections 2024-02-02T15:24:46.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:46.256+0000 7fac7fe00700 1 --2- 172.21.15.1:0/1129526482 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fac580a05a0 0x7fac580a2a60 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:46.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:46.256+0000 7fac7fe00700 1 --2- 172.21.15.1:0/1129526482 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fac78106af0 0x7fac780ffd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:46.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:46.256+0000 7fac7fe00700 1 --2- 172.21.15.1:0/1129526482 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac78101890 0x7fac78101200 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:46.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:46.256+0000 7fac7fe00700 1 --2- 172.21.15.1:0/1129526482 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fac781041c0 0x7fac780ff840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:46.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:46.256+0000 7fac7fe00700 1 -- 172.21.15.1:0/1129526482 >> 172.21.15.1:0/1129526482 conn(0x7fac780fb260 msgr2=0x7fac78102d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:24:46.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:46.257+0000 7fac7fe00700 1 -- 172.21.15.1:0/1129526482 shutdown_connections 2024-02-02T15:24:46.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:46.257+0000 7fac7fe00700 1 -- 172.21.15.1:0/1129526482 wait complete. 2024-02-02T15:24:46.259 INFO:teuthology.orchestra.run.smithi001.stderr:pool 'cephfs_metadata' created 2024-02-02T15:24:46.278 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-02-02T15:24:46.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:46.483+0000 7f7975e00700 1 -- 172.21.15.1:0/1457277 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7970101f70 msgr2=0x7f79701023f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:46.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:46.483+0000 7f7975e00700 1 --2- 172.21.15.1:0/1457277 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7970101f70 0x7f79701023f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7958009990 tx=0x7f7958009ca0 comp rx=0 tx=0).stop 2024-02-02T15:24:46.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:46.483+0000 7f7975e00700 1 -- 172.21.15.1:0/1457277 shutdown_connections 2024-02-02T15:24:46.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:46.483+0000 7f7975e00700 1 --2- 172.21.15.1:0/1457277 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7970102930 0x7f7970109250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:46.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:46.483+0000 7f7975e00700 1 --2- 172.21.15.1:0/1457277 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7970100e20 0x7f7970101240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:46.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:46.483+0000 7f7975e00700 1 --2- 172.21.15.1:0/1457277 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7970101f70 0x7f79701023f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:46.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:46.483+0000 7f7975e00700 1 -- 172.21.15.1:0/1457277 >> 172.21.15.1:0/1457277 conn(0x7f79700fc390 msgr2=0x7f79700fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:24:46.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:46.483+0000 7f7975e00700 1 -- 172.21.15.1:0/1457277 shutdown_connections 2024-02-02T15:24:46.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:46.483+0000 7f7975e00700 1 -- 172.21.15.1:0/1457277 wait complete. 2024-02-02T15:24:46.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:46.484+0000 7f7975e00700 1 Processor -- start 2024-02-02T15:24:46.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:46.484+0000 7f7975e00700 1 -- start start 2024-02-02T15:24:46.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:46.484+0000 7f7975e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7970100e20 0x7f7970074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:46.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:46.484+0000 7f7975e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7970101f70 0x7f7970073130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:46.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:46.484+0000 7f7975e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7970102930 0x7f7970073670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:46.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:46.484+0000 7f7975e00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7970073e10 con 0x7f7970102930 2024-02-02T15:24:46.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:46.484+0000 7f7975e00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7970073f80 con 0x7f7970101f70 2024-02-02T15:24:46.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:46.484+0000 7f7975e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79701a0b40 con 0x7f7970100e20 2024-02-02T15:24:46.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:46.485+0000 7f796f400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7970102930 0x7f7970073670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:46.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:46.485+0000 7f796ea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7970100e20 0x7f7970074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:46.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:46.485+0000 7f796f400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7970102930 0x7f7970073670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53256/0 (socket says 172.21.15.1:53256) 2024-02-02T15:24:46.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:46.485+0000 7f796f400700 1 -- 172.21.15.1:0/1553662510 learned_addr learned my addr 172.21.15.1:0/1553662510 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:24:46.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:46.485+0000 7f796e000700 1 --2- 172.21.15.1:0/1553662510 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7970101f70 0x7f7970073130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:46.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:46.485+0000 7f796f400700 1 -- 172.21.15.1:0/1553662510 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7970100e20 msgr2=0x7f7970074af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:46.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:46.485+0000 7f796f400700 1 --2- 172.21.15.1:0/1553662510 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7970100e20 0x7f7970074af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:46.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:46.485+0000 7f796f400700 1 -- 172.21.15.1:0/1553662510 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7970101f70 msgr2=0x7f7970073130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:46.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:46.485+0000 7f796f400700 1 --2- 172.21.15.1:0/1553662510 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7970101f70 0x7f7970073130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:46.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:46.485+0000 7f796f400700 1 -- 172.21.15.1:0/1553662510 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79580095f0 con 0x7f7970102930 2024-02-02T15:24:46.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:46.485+0000 7f796f400700 1 --2- 172.21.15.1:0/1553662510 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7970102930 0x7f7970073670 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7968007ee0 tx=0x7f796800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:24:46.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:46.486+0000 7f7974a00700 1 -- 172.21.15.1:0/1553662510 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7968017070 con 0x7f7970102930 2024-02-02T15:24:46.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:46.486+0000 7f7975e00700 1 -- 172.21.15.1:0/1553662510 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79701a0e20 con 0x7f7970102930 2024-02-02T15:24:46.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:46.486+0000 7f7974a00700 1 -- 172.21.15.1:0/1553662510 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f796800f040 con 0x7f7970102930 2024-02-02T15:24:46.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:46.486+0000 7f7974a00700 1 -- 172.21.15.1:0/1553662510 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7968005580 con 0x7f7970102930 2024-02-02T15:24:46.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:46.486+0000 7f7975e00700 1 -- 172.21.15.1:0/1553662510 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79701a12a0 con 0x7f7970102930 2024-02-02T15:24:46.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:46.487+0000 7f7975e00700 1 -- 172.21.15.1:0/1553662510 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7970107d40 con 0x7f7970102930 2024-02-02T15:24:46.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:46.489+0000 7f7974a00700 1 -- 172.21.15.1:0/1553662510 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) v1 ==== 135069+0+0 (secure 0 0 0) 0x7f796800db20 con 0x7f7970102930 2024-02-02T15:24:46.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:46.490+0000 7f7974a00700 1 --2- 172.21.15.1:0/1553662510 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f79500a0520 0x7f79500a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:46.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:46.490+0000 7f7974a00700 1 -- 172.21.15.1:0/1553662510 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7497+0+0 (secure 0 0 0) 0x7f79680cae40 con 0x7f7970102930 2024-02-02T15:24:46.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:46.490+0000 7f796ea00700 1 --2- 172.21.15.1:0/1553662510 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f79500a0520 0x7f79500a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:46.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:46.491+0000 7f796ea00700 1 --2- 172.21.15.1:0/1553662510 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f79500a0520 0x7f79500a29e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7960009c60 tx=0x7f7960009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:24:46.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:46.495+0000 7f7974a00700 1 -- 172.21.15.1:0/1553662510 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f7968094c70 con 0x7f7970102930 2024-02-02T15:24:46.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:46 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:46.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:46 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:46.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:46 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:46.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:46 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:46.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:46 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/3985063156' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-02-02T15:24:46.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:46 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:46.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:46 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:46.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:46 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:46.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:46 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:46.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:46 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/3985063156' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-02-02T15:24:46.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:46 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:46.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:46 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:46.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:46 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:46.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:46 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:46.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:46 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/3985063156' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-02-02T15:24:46.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:46.709+0000 7f7975e00700 1 -- 172.21.15.1:0/1553662510 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7f797002cc70 con 0x7f7970102930 2024-02-02T15:24:47.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:47.252+0000 7f7974a00700 1 -- 172.21.15.1:0/1553662510 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v78) v1 ==== 171+0+0 (secure 0 0 0) 0x7f7968098290 con 0x7f7970102930 2024-02-02T15:24:47.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:47.257+0000 7f7975e00700 1 -- 172.21.15.1:0/1553662510 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f79500a0520 msgr2=0x7f79500a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:47.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:47.257+0000 7f7975e00700 1 --2- 172.21.15.1:0/1553662510 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f79500a0520 0x7f79500a29e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7960009c60 tx=0x7f7960009260 comp rx=0 tx=0).stop 2024-02-02T15:24:47.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:47.257+0000 7f7975e00700 1 -- 172.21.15.1:0/1553662510 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7970102930 msgr2=0x7f7970073670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:47.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:47.257+0000 7f7975e00700 1 --2- 172.21.15.1:0/1553662510 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7970102930 0x7f7970073670 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7968007ee0 tx=0x7f796800d3b0 comp rx=0 tx=0).stop 2024-02-02T15:24:47.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:47.258+0000 7f7975e00700 1 -- 172.21.15.1:0/1553662510 shutdown_connections 2024-02-02T15:24:47.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:47.258+0000 7f7975e00700 1 --2- 172.21.15.1:0/1553662510 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7970102930 0x7f7970073670 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:47.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:47.258+0000 7f7975e00700 1 --2- 172.21.15.1:0/1553662510 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f79500a0520 0x7f79500a29e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:47.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:47.258+0000 7f7975e00700 1 --2- 172.21.15.1:0/1553662510 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7970100e20 0x7f7970074af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:47.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:47.258+0000 7f7975e00700 1 --2- 172.21.15.1:0/1553662510 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7970101f70 0x7f7970073130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:47.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:47.258+0000 7f7975e00700 1 -- 172.21.15.1:0/1553662510 >> 172.21.15.1:0/1553662510 conn(0x7f79700fc390 msgr2=0x7f79700fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:24:47.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:47.259+0000 7f7975e00700 1 -- 172.21.15.1:0/1553662510 shutdown_connections 2024-02-02T15:24:47.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:47.259+0000 7f7975e00700 1 -- 172.21.15.1:0/1553662510 wait complete. 2024-02-02T15:24:47.262 INFO:teuthology.orchestra.run.smithi001.stderr:pool 'cephfs_data' created 2024-02-02T15:24:47.291 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-02-02T15:24:47.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:47.491+0000 7fca14600700 1 -- 172.21.15.1:0/3923233426 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca0c0737a0 msgr2=0x7fca0c073c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:47.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:47.491+0000 7fca14600700 1 --2- 172.21.15.1:0/3923233426 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca0c0737a0 0x7fca0c073c20 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc9f40098f0 tx=0x7fc9f4009c00 comp rx=0 tx=0).stop 2024-02-02T15:24:47.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:47.491+0000 7fca14600700 1 -- 172.21.15.1:0/3923233426 shutdown_connections 2024-02-02T15:24:47.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:47.491+0000 7fca14600700 1 --2- 172.21.15.1:0/3923233426 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca0c108ce0 0x7fca0c10b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:47.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:47.491+0000 7fca14600700 1 --2- 172.21.15.1:0/3923233426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca0c074d80 0x7fca0c0731d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:47.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:47.491+0000 7fca14600700 1 --2- 172.21.15.1:0/3923233426 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca0c0737a0 0x7fca0c073c20 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:47.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:47.491+0000 7fca14600700 1 -- 172.21.15.1:0/3923233426 >> 172.21.15.1:0/3923233426 conn(0x7fca0c0fc390 msgr2=0x7fca0c0fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:24:47.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:47.491+0000 7fca14600700 1 -- 172.21.15.1:0/3923233426 shutdown_connections 2024-02-02T15:24:47.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:47.492+0000 7fca14600700 1 -- 172.21.15.1:0/3923233426 wait complete. 2024-02-02T15:24:47.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:47.493+0000 7fca14600700 1 Processor -- start 2024-02-02T15:24:47.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:47.493+0000 7fca14600700 1 -- start start 2024-02-02T15:24:47.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:47.494+0000 7fca14600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca0c0737a0 0x7fca0c198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:47.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:47.494+0000 7fca14600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca0c074d80 0x7fca0c1992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:47.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:47.494+0000 7fca14600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca0c108ce0 0x7fca0c1a0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:47.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:47.494+0000 7fca14600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca0c1a08e0 con 0x7fca0c108ce0 2024-02-02T15:24:47.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:47.494+0000 7fca14600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca0c1a0a50 con 0x7fca0c0737a0 2024-02-02T15:24:47.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:47.494+0000 7fca14600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca0c1a0c10 con 0x7fca0c074d80 2024-02-02T15:24:47.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:47.494+0000 7fca11c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca0c0737a0 0x7fca0c198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:47.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:47.494+0000 7fca11c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca0c0737a0 0x7fca0c198da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:46924/0 (socket says 172.21.15.1:46924) 2024-02-02T15:24:47.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:47.494+0000 7fca12600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca0c108ce0 0x7fca0c1a0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:47.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:47.494+0000 7fca11c00700 1 -- 172.21.15.1:0/1357720947 learned_addr learned my addr 172.21.15.1:0/1357720947 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:24:47.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:47.494+0000 7fca11200700 1 --2- 172.21.15.1:0/1357720947 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca0c074d80 0x7fca0c1992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:47.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:47.494+0000 7fca11c00700 1 -- 172.21.15.1:0/1357720947 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca0c074d80 msgr2=0x7fca0c1992e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:47.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:47.494+0000 7fca11c00700 1 --2- 172.21.15.1:0/1357720947 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca0c074d80 0x7fca0c1992e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:47.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:47.494+0000 7fca11c00700 1 -- 172.21.15.1:0/1357720947 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca0c108ce0 msgr2=0x7fca0c1a0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:47.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:47.494+0000 7fca11c00700 1 --2- 172.21.15.1:0/1357720947 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca0c108ce0 0x7fca0c1a0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:47.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:47.494+0000 7fca11c00700 1 -- 172.21.15.1:0/1357720947 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9f4009520 con 0x7fca0c0737a0 2024-02-02T15:24:47.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:47.495+0000 7fca11c00700 1 --2- 172.21.15.1:0/1357720947 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca0c0737a0 0x7fca0c198da0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc00e9a0 tx=0x7fc9fc00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:24:47.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:47.495+0000 7fca0a000700 1 -- 172.21.15.1:0/1357720947 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9fc009790 con 0x7fca0c0737a0 2024-02-02T15:24:47.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:47.495+0000 7fca0a000700 1 -- 172.21.15.1:0/1357720947 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc9fc010430 con 0x7fca0c0737a0 2024-02-02T15:24:47.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:47.495+0000 7fca14600700 1 -- 172.21.15.1:0/1357720947 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca0c1a0ef0 con 0x7fca0c0737a0 2024-02-02T15:24:47.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:47.495+0000 7fca0a000700 1 -- 172.21.15.1:0/1357720947 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9fc018680 con 0x7fca0c0737a0 2024-02-02T15:24:47.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:47.495+0000 7fca14600700 1 -- 172.21.15.1:0/1357720947 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca0c1a1310 con 0x7fca0c0737a0 2024-02-02T15:24:47.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:47.496+0000 7fca14600700 1 -- 172.21.15.1:0/1357720947 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca0c107390 con 0x7fca0c0737a0 2024-02-02T15:24:47.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:47.498+0000 7fca0a000700 1 -- 172.21.15.1:0/1357720947 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 135069+0+0 (secure 0 0 0) 0x7fc9fc0105a0 con 0x7fca0c0737a0 2024-02-02T15:24:47.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:47.499+0000 7fca0a000700 1 --2- 172.21.15.1:0/1357720947 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fc9ec0a0520 0x7fc9ec0a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:47.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:47.499+0000 7fca0a000700 1 -- 172.21.15.1:0/1357720947 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7879+0+0 (secure 0 0 0) 0x7fc9fc014070 con 0x7fca0c0737a0 2024-02-02T15:24:47.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:47.500+0000 7fca11200700 1 --2- 172.21.15.1:0/1357720947 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fc9ec0a0520 0x7fc9ec0a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:47.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:47.500+0000 7fca11200700 1 --2- 172.21.15.1:0/1357720947 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fc9ec0a0520 0x7fc9ec0a29e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc9f4009fd0 tx=0x7fc9f4019040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:24:47.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:47.502+0000 7fca0a000700 1 -- 172.21.15.1:0/1357720947 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7fc9fc095880 con 0x7fca0c0737a0 2024-02-02T15:24:47.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:47 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/1129526482' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-02-02T15:24:47.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:47 smithi071 ceph-mon[44707]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-02-02T15:24:47.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:47 smithi071 ceph-mon[44707]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-02T15:24:47.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:47 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:47.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:47 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:47.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:47 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:47.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:47 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:24:47.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:47 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:24:47.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:47 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:47.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:47 smithi071 ceph-mon[44707]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-02-02T15:24:47.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:47 smithi071 ceph-mon[44707]: osdmap e77: 12 total, 12 up, 12 in 2024-02-02T15:24:47.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:47 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/1553662510' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-02-02T15:24:47.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:47 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/1129526482' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-02-02T15:24:47.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:47 smithi142 ceph-mon[45728]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-02-02T15:24:47.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:47 smithi142 ceph-mon[45728]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-02T15:24:47.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:47 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:47.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:47 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:47.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:47 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:47.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:47 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:24:47.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:47 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:24:47.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:47 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:47.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:47 smithi142 ceph-mon[45728]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-02-02T15:24:47.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:47 smithi142 ceph-mon[45728]: osdmap e77: 12 total, 12 up, 12 in 2024-02-02T15:24:47.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:47 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/1553662510' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-02-02T15:24:47.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:47 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/1129526482' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-02-02T15:24:47.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:47 smithi001 ceph-mon[35470]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-02-02T15:24:47.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:47 smithi001 ceph-mon[35470]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-02T15:24:47.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:47 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:47.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:47 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:47.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:47 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:47.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:47 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:24:47.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:47 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:24:47.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:47 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:24:47.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:47 smithi001 ceph-mon[35470]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-02-02T15:24:47.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:47 smithi001 ceph-mon[35470]: osdmap e77: 12 total, 12 up, 12 in 2024-02-02T15:24:47.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:47 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/1553662510' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-02-02T15:24:47.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:47.758+0000 7fca14600700 1 -- 172.21.15.1:0/1357720947 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fca0c02cc70 con 0x7fca0c0737a0 2024-02-02T15:24:47.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:47.769+0000 7fca0a000700 1 -- 172.21.15.1:0/1357720947 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7fc9fc0260c0 con 0x7fca0c0737a0 2024-02-02T15:24:47.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:47.775+0000 7fca14600700 1 -- 172.21.15.1:0/1357720947 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fc9ec0a0520 msgr2=0x7fc9ec0a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:47.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:47.775+0000 7fca14600700 1 --2- 172.21.15.1:0/1357720947 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fc9ec0a0520 0x7fc9ec0a29e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc9f4009fd0 tx=0x7fc9f4019040 comp rx=0 tx=0).stop 2024-02-02T15:24:47.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:47.775+0000 7fca14600700 1 -- 172.21.15.1:0/1357720947 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca0c0737a0 msgr2=0x7fca0c198da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:47.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:47.775+0000 7fca14600700 1 --2- 172.21.15.1:0/1357720947 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca0c0737a0 0x7fca0c198da0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc00e9a0 tx=0x7fc9fc00ecb0 comp rx=0 tx=0).stop 2024-02-02T15:24:47.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:47.775+0000 7fca14600700 1 -- 172.21.15.1:0/1357720947 shutdown_connections 2024-02-02T15:24:47.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:47.775+0000 7fca14600700 1 --2- 172.21.15.1:0/1357720947 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca0c108ce0 0x7fca0c1a0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:47.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:47.775+0000 7fca14600700 1 --2- 172.21.15.1:0/1357720947 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca0c0737a0 0x7fca0c198da0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:47.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:47.775+0000 7fca14600700 1 --2- 172.21.15.1:0/1357720947 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fc9ec0a0520 0x7fc9ec0a29e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:47.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:47.775+0000 7fca14600700 1 --2- 172.21.15.1:0/1357720947 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca0c074d80 0x7fca0c1992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:47.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:47.775+0000 7fca14600700 1 -- 172.21.15.1:0/1357720947 >> 172.21.15.1:0/1357720947 conn(0x7fca0c0fc390 msgr2=0x7fca0c0fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:24:47.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:47.776+0000 7fca14600700 1 -- 172.21.15.1:0/1357720947 shutdown_connections 2024-02-02T15:24:47.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:47.776+0000 7fca14600700 1 -- 172.21.15.1:0/1357720947 wait complete. 2024-02-02T15:24:47.784 INFO:teuthology.orchestra.run.smithi001.stderr:new fs with metadata pool 2 and data pool 3 2024-02-02T15:24:47.808 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-02-02T15:24:47.808 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-02-02T15:24:48.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:48.012+0000 7f4199a00700 1 -- 172.21.15.1:0/3206318337 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4194100e20 msgr2=0x7f4194101240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:48.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:48.012+0000 7f4199a00700 1 --2- 172.21.15.1:0/3206318337 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4194100e20 0x7f4194101240 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4184009990 tx=0x7f4184009ca0 comp rx=0 tx=0).stop 2024-02-02T15:24:48.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:48.012+0000 7f4199a00700 1 -- 172.21.15.1:0/3206318337 shutdown_connections 2024-02-02T15:24:48.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:48.012+0000 7f4199a00700 1 --2- 172.21.15.1:0/3206318337 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4194102930 0x7f4194109250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:48.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:48.012+0000 7f4199a00700 1 --2- 172.21.15.1:0/3206318337 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4194100e20 0x7f4194101240 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:48.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:48.012+0000 7f4199a00700 1 --2- 172.21.15.1:0/3206318337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4194101f70 0x7f41941023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:48.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:48.012+0000 7f4199a00700 1 -- 172.21.15.1:0/3206318337 >> 172.21.15.1:0/3206318337 conn(0x7f41940fc390 msgr2=0x7f41940fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:24:48.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:48.012+0000 7f4199a00700 1 -- 172.21.15.1:0/3206318337 shutdown_connections 2024-02-02T15:24:48.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:48.012+0000 7f4199a00700 1 -- 172.21.15.1:0/3206318337 wait complete. 2024-02-02T15:24:48.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:48.013+0000 7f4199a00700 1 Processor -- start 2024-02-02T15:24:48.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:48.013+0000 7f4199a00700 1 -- start start 2024-02-02T15:24:48.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:48.013+0000 7f4199a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4194100e20 0x7f4194196ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:48.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:48.014+0000 7f4199a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4194101f70 0x7f41941970e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:48.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:48.014+0000 7f4199a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4194102930 0x7f419419e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:48.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:48.014+0000 7f4199a00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f419419e720 con 0x7f4194102930 2024-02-02T15:24:48.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:48.014+0000 7f4199a00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f419419e860 con 0x7f4194100e20 2024-02-02T15:24:48.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:48.014+0000 7f4199a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f419419e9f0 con 0x7f4194101f70 2024-02-02T15:24:48.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:48.014+0000 7f4193400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4194102930 0x7f419419e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:48.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:48.014+0000 7f4193400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4194102930 0x7f419419e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53292/0 (socket says 172.21.15.1:53292) 2024-02-02T15:24:48.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:48.014+0000 7f4193400700 1 -- 172.21.15.1:0/3816227875 learned_addr learned my addr 172.21.15.1:0/3816227875 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:24:48.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:48.014+0000 7f4192a00700 1 --2- 172.21.15.1:0/3816227875 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4194100e20 0x7f4194196ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:48.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:48.014+0000 7f4192000700 1 --2- 172.21.15.1:0/3816227875 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4194101f70 0x7f41941970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:48.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:48.014+0000 7f4193400700 1 -- 172.21.15.1:0/3816227875 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4194101f70 msgr2=0x7f41941970e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:48.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:48.014+0000 7f4193400700 1 --2- 172.21.15.1:0/3816227875 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4194101f70 0x7f41941970e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:48.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:48.014+0000 7f4193400700 1 -- 172.21.15.1:0/3816227875 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4194100e20 msgr2=0x7f4194196ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:48.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:48.014+0000 7f4193400700 1 --2- 172.21.15.1:0/3816227875 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4194100e20 0x7f4194196ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:48.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:48.014+0000 7f4193400700 1 -- 172.21.15.1:0/3816227875 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41840095f0 con 0x7f4194102930 2024-02-02T15:24:48.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:48.014+0000 7f4193400700 1 --2- 172.21.15.1:0/3816227875 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4194102930 0x7f419419e150 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f418c007ee0 tx=0x7f418c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:24:48.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:48.015+0000 7f418b400700 1 -- 172.21.15.1:0/3816227875 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f418c017070 con 0x7f4194102930 2024-02-02T15:24:48.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:48.015+0000 7f418b400700 1 -- 172.21.15.1:0/3816227875 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f418c00f040 con 0x7f4194102930 2024-02-02T15:24:48.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:48.015+0000 7f4199a00700 1 -- 172.21.15.1:0/3816227875 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f419419ecd0 con 0x7f4194102930 2024-02-02T15:24:48.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:48.015+0000 7f418b400700 1 -- 172.21.15.1:0/3816227875 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f418c005580 con 0x7f4194102930 2024-02-02T15:24:48.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:48.015+0000 7f4199a00700 1 -- 172.21.15.1:0/3816227875 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f419419f190 con 0x7f4194102930 2024-02-02T15:24:48.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:48.016+0000 7f4199a00700 1 -- 172.21.15.1:0/3816227875 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4194108450 con 0x7f4194102930 2024-02-02T15:24:48.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:48.018+0000 7f418b400700 1 -- 172.21.15.1:0/3816227875 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) v1 ==== 135069+0+0 (secure 0 0 0) 0x7f418c00db20 con 0x7f4194102930 2024-02-02T15:24:48.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:48.019+0000 7f418b400700 1 --2- 172.21.15.1:0/3816227875 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f41740a0520 0x7f41740a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:48.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:48.019+0000 7f418b400700 1 -- 172.21.15.1:0/3816227875 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f418c0cb5b0 con 0x7f4194102930 2024-02-02T15:24:48.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:48.020+0000 7f4192a00700 1 --2- 172.21.15.1:0/3816227875 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f41740a0520 0x7f41740a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:48.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:48.020+0000 7f4192a00700 1 --2- 172.21.15.1:0/3816227875 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f41740a0520 0x7f41740a29e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f418400b5c0 tx=0x7f4184005fd0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:24:48.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:48.022+0000 7f418b400700 1 -- 172.21.15.1:0/3816227875 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f418c095200 con 0x7f4194102930 2024-02-02T15:24:48.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:48.246+0000 7f4199a00700 1 -- 172.21.15.1:0/3816227875 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f419402cd00 con 0x7f4194102930 2024-02-02T15:24:48.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:48 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/1553662510' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-02-02T15:24:48.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:48 smithi071 ceph-mon[44707]: osdmap e78: 12 total, 12 up, 12 in 2024-02-02T15:24:48.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:48 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/1357720947' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-02-02T15:24:48.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:48 smithi071 ceph-mon[44707]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-02-02T15:24:48.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:48 smithi071 ceph-mon[44707]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-02-02T15:24:48.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:48 smithi071 ceph-mon[44707]: osdmap e79: 12 total, 12 up, 12 in 2024-02-02T15:24:48.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:48 smithi071 ceph-mon[44707]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-02-02T15:24:48.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:48 smithi071 ceph-mon[44707]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-02-02T15:24:48.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:48 smithi071 ceph-mon[44707]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-02-02T15:24:48.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:48 smithi071 ceph-mon[44707]: pgmap v230: 97 pgs: 5 creating+peering, 1 active+clean, 91 unknown; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-02T15:24:48.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:48 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/3816227875' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-02-02T15:24:48.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:48 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/1553662510' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-02-02T15:24:48.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:48 smithi142 ceph-mon[45728]: osdmap e78: 12 total, 12 up, 12 in 2024-02-02T15:24:48.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:48 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/1357720947' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-02-02T15:24:48.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:48 smithi142 ceph-mon[45728]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-02-02T15:24:48.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:48 smithi142 ceph-mon[45728]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-02-02T15:24:48.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:48 smithi142 ceph-mon[45728]: osdmap e79: 12 total, 12 up, 12 in 2024-02-02T15:24:48.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:48 smithi142 ceph-mon[45728]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-02-02T15:24:48.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:48 smithi142 ceph-mon[45728]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-02-02T15:24:48.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:48 smithi142 ceph-mon[45728]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-02-02T15:24:48.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:48 smithi142 ceph-mon[45728]: pgmap v230: 97 pgs: 5 creating+peering, 1 active+clean, 91 unknown; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-02T15:24:48.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:48 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/3816227875' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-02-02T15:24:48.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:48 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/1553662510' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-02-02T15:24:48.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:48 smithi001 ceph-mon[35470]: osdmap e78: 12 total, 12 up, 12 in 2024-02-02T15:24:48.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:48 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/1357720947' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-02-02T15:24:48.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:48 smithi001 ceph-mon[35470]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-02-02T15:24:48.692 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:48 smithi001 ceph-mon[35470]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-02-02T15:24:48.692 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:48 smithi001 ceph-mon[35470]: osdmap e79: 12 total, 12 up, 12 in 2024-02-02T15:24:48.692 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:48 smithi001 ceph-mon[35470]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-02-02T15:24:48.692 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:48 smithi001 ceph-mon[35470]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-02-02T15:24:48.692 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:48 smithi001 ceph-mon[35470]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-02-02T15:24:48.692 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:48 smithi001 ceph-mon[35470]: pgmap v230: 97 pgs: 5 creating+peering, 1 active+clean, 91 unknown; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-02T15:24:48.692 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:48 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/3816227875' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-02-02T15:24:48.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:48.773+0000 7f418b400700 1 -- 172.21.15.1:0/3816227875 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v80) v1 ==== 157+0+0 (secure 0 0 0) 0x7f418c098820 con 0x7f4194102930 2024-02-02T15:24:48.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:48.776+0000 7f4199a00700 1 -- 172.21.15.1:0/3816227875 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f41740a0520 msgr2=0x7f41740a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:48.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:48.776+0000 7f4199a00700 1 --2- 172.21.15.1:0/3816227875 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f41740a0520 0x7f41740a29e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f418400b5c0 tx=0x7f4184005fd0 comp rx=0 tx=0).stop 2024-02-02T15:24:48.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:48.776+0000 7f4199a00700 1 -- 172.21.15.1:0/3816227875 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4194102930 msgr2=0x7f419419e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:48.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:48.776+0000 7f4199a00700 1 --2- 172.21.15.1:0/3816227875 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4194102930 0x7f419419e150 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f418c007ee0 tx=0x7f418c00d3b0 comp rx=0 tx=0).stop 2024-02-02T15:24:48.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:48.777+0000 7f4199a00700 1 -- 172.21.15.1:0/3816227875 shutdown_connections 2024-02-02T15:24:48.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:48.777+0000 7f4199a00700 1 --2- 172.21.15.1:0/3816227875 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4194102930 0x7f419419e150 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:48.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:48.777+0000 7f4199a00700 1 --2- 172.21.15.1:0/3816227875 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4194100e20 0x7f4194196ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:48.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:48.777+0000 7f4199a00700 1 --2- 172.21.15.1:0/3816227875 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f41740a0520 0x7f41740a29e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:48.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:48.777+0000 7f4199a00700 1 --2- 172.21.15.1:0/3816227875 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4194101f70 0x7f41941970e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:48.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:48.777+0000 7f4199a00700 1 -- 172.21.15.1:0/3816227875 >> 172.21.15.1:0/3816227875 conn(0x7f41940fc390 msgr2=0x7f4194105590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:24:48.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:48.777+0000 7f4199a00700 1 -- 172.21.15.1:0/3816227875 shutdown_connections 2024-02-02T15:24:48.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:48.777+0000 7f4199a00700 1 -- 172.21.15.1:0/3816227875 wait complete. 2024-02-02T15:24:48.796 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-02-02T15:24:49.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:49.004+0000 7fdd47400700 1 -- 172.21.15.1:0/660225431 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdd401017d0 msgr2=0x7fdd40103bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:49.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:49.004+0000 7fdd47400700 1 --2- 172.21.15.1:0/660225431 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdd401017d0 0x7fdd40103bc0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fdd30009990 tx=0x7fdd30009ca0 comp rx=0 tx=0).stop 2024-02-02T15:24:49.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:49.004+0000 7fdd47400700 1 -- 172.21.15.1:0/660225431 shutdown_connections 2024-02-02T15:24:49.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:49.004+0000 7fdd47400700 1 --2- 172.21.15.1:0/660225431 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdd40106a30 0x7fdd40108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:49.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:49.004+0000 7fdd47400700 1 --2- 172.21.15.1:0/660225431 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdd401017d0 0x7fdd40103bc0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:49.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:49.004+0000 7fdd47400700 1 --2- 172.21.15.1:0/660225431 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd40104100 0x7fdd401064f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:49.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:49.004+0000 7fdd47400700 1 -- 172.21.15.1:0/660225431 >> 172.21.15.1:0/660225431 conn(0x7fdd400fb200 msgr2=0x7fdd400fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:24:49.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:49.004+0000 7fdd47400700 1 -- 172.21.15.1:0/660225431 shutdown_connections 2024-02-02T15:24:49.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:49.004+0000 7fdd47400700 1 -- 172.21.15.1:0/660225431 wait complete. 2024-02-02T15:24:49.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:49.005+0000 7fdd47400700 1 Processor -- start 2024-02-02T15:24:49.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:49.005+0000 7fdd47400700 1 -- start start 2024-02-02T15:24:49.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:49.006+0000 7fdd47400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdd401017d0 0x7fdd40101140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:49.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:49.006+0000 7fdd47400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdd40104100 0x7fdd400ff780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:49.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:49.006+0000 7fdd47400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd40106a30 0x7fdd400ffcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:49.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:49.006+0000 7fdd47400700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd40100460 con 0x7fdd40104100 2024-02-02T15:24:49.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:49.006+0000 7fdd47400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd401005d0 con 0x7fdd401017d0 2024-02-02T15:24:49.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:49.006+0000 7fdd47400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd401a0a60 con 0x7fdd40106a30 2024-02-02T15:24:49.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:49.006+0000 7fdd44a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdd401017d0 0x7fdd40101140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:49.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:49.006+0000 7fdd3fe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdd40104100 0x7fdd400ff780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:49.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:49.006+0000 7fdd44a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdd401017d0 0x7fdd40101140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:46972/0 (socket says 172.21.15.1:46972) 2024-02-02T15:24:49.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:49.006+0000 7fdd44a00700 1 -- 172.21.15.1:0/4032103013 learned_addr learned my addr 172.21.15.1:0/4032103013 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:24:49.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:49.006+0000 7fdd45400700 1 --2- 172.21.15.1:0/4032103013 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd40106a30 0x7fdd400ffcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:49.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:49.006+0000 7fdd44a00700 1 -- 172.21.15.1:0/4032103013 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd40106a30 msgr2=0x7fdd400ffcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:49.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:49.006+0000 7fdd44a00700 1 --2- 172.21.15.1:0/4032103013 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd40106a30 0x7fdd400ffcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:49.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:49.006+0000 7fdd44a00700 1 -- 172.21.15.1:0/4032103013 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdd40104100 msgr2=0x7fdd400ff780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:49.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:49.006+0000 7fdd44a00700 1 --2- 172.21.15.1:0/4032103013 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdd40104100 0x7fdd400ff780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:49.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:49.006+0000 7fdd44a00700 1 -- 172.21.15.1:0/4032103013 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd300095f0 con 0x7fdd401017d0 2024-02-02T15:24:49.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:49.007+0000 7fdd44a00700 1 --2- 172.21.15.1:0/4032103013 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdd401017d0 0x7fdd40101140 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdd3000b680 tx=0x7fdd300057b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:24:49.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:49.007+0000 7fdd3d600700 1 -- 172.21.15.1:0/4032103013 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd3001d070 con 0x7fdd401017d0 2024-02-02T15:24:49.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:49.007+0000 7fdd3d600700 1 -- 172.21.15.1:0/4032103013 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd3000fbc0 con 0x7fdd401017d0 2024-02-02T15:24:49.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:49.007+0000 7fdd3d600700 1 -- 172.21.15.1:0/4032103013 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd30017650 con 0x7fdd401017d0 2024-02-02T15:24:49.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:49.007+0000 7fdd47400700 1 -- 172.21.15.1:0/4032103013 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd401a0ce0 con 0x7fdd401017d0 2024-02-02T15:24:49.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:49.007+0000 7fdd47400700 1 -- 172.21.15.1:0/4032103013 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd401a10b0 con 0x7fdd401017d0 2024-02-02T15:24:49.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:49.008+0000 7fdd47400700 1 -- 172.21.15.1:0/4032103013 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd4010c5f0 con 0x7fdd401017d0 2024-02-02T15:24:49.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:49.010+0000 7fdd3d600700 1 -- 172.21.15.1:0/4032103013 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 135069+0+0 (secure 0 0 0) 0x7fdd30022470 con 0x7fdd401017d0 2024-02-02T15:24:49.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:49.011+0000 7fdd3d600700 1 --2- 172.21.15.1:0/4032103013 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fdd200a0520 0x7fdd200a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:49.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:49.011+0000 7fdd3d600700 1 -- 172.21.15.1:0/4032103013 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 8205+0+0 (secure 0 0 0) 0x7fdd300cd6e0 con 0x7fdd401017d0 2024-02-02T15:24:49.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:49.011+0000 7fdd3fe00700 1 --2- 172.21.15.1:0/4032103013 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fdd200a0520 0x7fdd200a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:49.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:49.012+0000 7fdd3fe00700 1 --2- 172.21.15.1:0/4032103013 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fdd200a0520 0x7fdd200a29e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fdd28007c60 tx=0x7fdd280073d0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:24:49.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:49.013+0000 7fdd3d600700 1 -- 172.21.15.1:0/4032103013 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7fdd300972d0 con 0x7fdd401017d0 2024-02-02T15:24:49.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:49.226+0000 7fdd47400700 1 -- 172.21.15.1:0/4032103013 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fdd401a1710 con 0x7fdd401017d0 2024-02-02T15:24:50.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:49 smithi071 ceph-mon[44707]: mds.? [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] up:active 2024-02-02T15:24:50.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:49 smithi071 ceph-mon[44707]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-02T15:24:50.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:49 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/3816227875' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-02-02T15:24:50.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:49 smithi071 ceph-mon[44707]: osdmap e80: 12 total, 12 up, 12 in 2024-02-02T15:24:50.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:49 smithi071 ceph-mon[44707]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-02T15:24:50.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:49 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/4032103013' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-02-02T15:24:50.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:49 smithi071 ceph-mon[44707]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-02-02T15:24:50.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:49 smithi142 ceph-mon[45728]: mds.? [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] up:active 2024-02-02T15:24:50.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:49 smithi142 ceph-mon[45728]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-02T15:24:50.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:49 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/3816227875' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-02-02T15:24:50.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:49 smithi142 ceph-mon[45728]: osdmap e80: 12 total, 12 up, 12 in 2024-02-02T15:24:50.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:49 smithi142 ceph-mon[45728]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-02T15:24:50.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:49 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/4032103013' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-02-02T15:24:50.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:49 smithi142 ceph-mon[45728]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-02-02T15:24:50.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:49 smithi001 ceph-mon[35470]: mds.? [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] up:active 2024-02-02T15:24:50.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:49 smithi001 ceph-mon[35470]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-02T15:24:50.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:49 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/3816227875' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-02-02T15:24:50.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:49 smithi001 ceph-mon[35470]: osdmap e80: 12 total, 12 up, 12 in 2024-02-02T15:24:50.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:49 smithi001 ceph-mon[35470]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-02T15:24:50.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:49 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/4032103013' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-02-02T15:24:50.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:49 smithi001 ceph-mon[35470]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-02-02T15:24:50.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:50.789+0000 7fdd3d600700 1 -- 172.21.15.1:0/4032103013 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v82) v1 ==== 228+0+0 (secure 0 0 0) 0x7fdd3009a8f0 con 0x7fdd401017d0 2024-02-02T15:24:50.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:50.795+0000 7fdd47400700 1 -- 172.21.15.1:0/4032103013 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fdd200a0520 msgr2=0x7fdd200a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:50.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:50.795+0000 7fdd47400700 1 --2- 172.21.15.1:0/4032103013 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fdd200a0520 0x7fdd200a29e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fdd28007c60 tx=0x7fdd280073d0 comp rx=0 tx=0).stop 2024-02-02T15:24:50.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:50.795+0000 7fdd47400700 1 -- 172.21.15.1:0/4032103013 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdd401017d0 msgr2=0x7fdd40101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:50.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:50.795+0000 7fdd47400700 1 --2- 172.21.15.1:0/4032103013 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdd401017d0 0x7fdd40101140 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdd3000b680 tx=0x7fdd300057b0 comp rx=0 tx=0).stop 2024-02-02T15:24:50.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:50.795+0000 7fdd47400700 1 -- 172.21.15.1:0/4032103013 shutdown_connections 2024-02-02T15:24:50.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:50.795+0000 7fdd47400700 1 --2- 172.21.15.1:0/4032103013 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fdd200a0520 0x7fdd200a29e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:50.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:50.795+0000 7fdd47400700 1 --2- 172.21.15.1:0/4032103013 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd40106a30 0x7fdd400ffcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:50.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:50.795+0000 7fdd47400700 1 --2- 172.21.15.1:0/4032103013 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdd401017d0 0x7fdd40101140 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:50.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:50.795+0000 7fdd47400700 1 --2- 172.21.15.1:0/4032103013 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdd40104100 0x7fdd400ff780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:50.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:50.795+0000 7fdd47400700 1 -- 172.21.15.1:0/4032103013 >> 172.21.15.1:0/4032103013 conn(0x7fdd400fb200 msgr2=0x7fdd40102c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:24:50.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:50.797+0000 7fdd47400700 1 -- 172.21.15.1:0/4032103013 shutdown_connections 2024-02-02T15:24:50.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:50.797+0000 7fdd47400700 1 -- 172.21.15.1:0/4032103013 wait complete. 2024-02-02T15:24:50.799 INFO:teuthology.orchestra.run.smithi001.stderr:pool 'cephfs_data_ec' created 2024-02-02T15:24:50.820 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-02-02T15:24:51.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:51.025+0000 7ff5c7200700 1 -- 172.21.15.1:0/310388011 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff5c0074d80 msgr2=0x7ff5c00731d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:51.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:51.025+0000 7ff5c7200700 1 --2- 172.21.15.1:0/310388011 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff5c0074d80 0x7ff5c00731d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0009990 tx=0x7ff5b0009ca0 comp rx=0 tx=0).stop 2024-02-02T15:24:51.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:51.025+0000 7ff5c7200700 1 -- 172.21.15.1:0/310388011 shutdown_connections 2024-02-02T15:24:51.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:51.025+0000 7ff5c7200700 1 --2- 172.21.15.1:0/310388011 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5c0108ce0 0x7ff5c010b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:51.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:51.025+0000 7ff5c7200700 1 --2- 172.21.15.1:0/310388011 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff5c0074d80 0x7ff5c00731d0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:51.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:51.025+0000 7ff5c7200700 1 --2- 172.21.15.1:0/310388011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5c00737a0 0x7ff5c0073c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:51.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:51.025+0000 7ff5c7200700 1 -- 172.21.15.1:0/310388011 >> 172.21.15.1:0/310388011 conn(0x7ff5c00fc390 msgr2=0x7ff5c00fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:24:51.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:51.025+0000 7ff5c7200700 1 -- 172.21.15.1:0/310388011 shutdown_connections 2024-02-02T15:24:51.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:51.025+0000 7ff5c7200700 1 -- 172.21.15.1:0/310388011 wait complete. 2024-02-02T15:24:51.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:51.026+0000 7ff5c7200700 1 Processor -- start 2024-02-02T15:24:51.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:51.026+0000 7ff5c7200700 1 -- start start 2024-02-02T15:24:51.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:51.027+0000 7ff5c7200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5c00737a0 0x7ff5c0198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:51.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:51.027+0000 7ff5c7200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff5c0074d80 0x7ff5c0199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:51.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:51.027+0000 7ff5c7200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5c0108ce0 0x7ff5c01a0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:51.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:51.027+0000 7ff5c7200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c01a08d0 con 0x7ff5c0074d80 2024-02-02T15:24:51.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:51.027+0000 7ff5c7200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c01a0a10 con 0x7ff5c00737a0 2024-02-02T15:24:51.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:51.027+0000 7ff5c7200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c01a0ba0 con 0x7ff5c0108ce0 2024-02-02T15:24:51.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:51.027+0000 7ff5bf400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff5c0074d80 0x7ff5c0199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:51.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:51.027+0000 7ff5bf400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff5c0074d80 0x7ff5c0199290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53350/0 (socket says 172.21.15.1:53350) 2024-02-02T15:24:51.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:51.027+0000 7ff5bf400700 1 -- 172.21.15.1:0/2795004084 learned_addr learned my addr 172.21.15.1:0/2795004084 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:24:51.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:51.027+0000 7ff5bfe00700 1 --2- 172.21.15.1:0/2795004084 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5c00737a0 0x7ff5c0198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:51.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:51.027+0000 7ff5c5200700 1 --2- 172.21.15.1:0/2795004084 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5c0108ce0 0x7ff5c01a0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:51.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:51.027+0000 7ff5bf400700 1 -- 172.21.15.1:0/2795004084 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5c0108ce0 msgr2=0x7ff5c01a0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:51.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:51.027+0000 7ff5bf400700 1 --2- 172.21.15.1:0/2795004084 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5c0108ce0 0x7ff5c01a0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:51.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:51.027+0000 7ff5bf400700 1 -- 172.21.15.1:0/2795004084 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5c00737a0 msgr2=0x7ff5c0198d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:51.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:51.027+0000 7ff5bf400700 1 --2- 172.21.15.1:0/2795004084 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5c00737a0 0x7ff5c0198d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:51.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:51.027+0000 7ff5bf400700 1 -- 172.21.15.1:0/2795004084 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5b00095f0 con 0x7ff5c0074d80 2024-02-02T15:24:51.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:51.028+0000 7ff5bf400700 1 --2- 172.21.15.1:0/2795004084 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff5c0074d80 0x7ff5c0199290 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff5a800e970 tx=0x7ff5a800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:24:51.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:51.028+0000 7ff5bcc00700 1 -- 172.21.15.1:0/2795004084 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a8009790 con 0x7ff5c0074d80 2024-02-02T15:24:51.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:51.028+0000 7ff5bcc00700 1 -- 172.21.15.1:0/2795004084 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff5a800c7e0 con 0x7ff5c0074d80 2024-02-02T15:24:51.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:51.028+0000 7ff5bcc00700 1 -- 172.21.15.1:0/2795004084 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a8003c70 con 0x7ff5c0074d80 2024-02-02T15:24:51.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:51.028+0000 7ff5c7200700 1 -- 172.21.15.1:0/2795004084 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5c01a0e80 con 0x7ff5c0074d80 2024-02-02T15:24:51.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:51.028+0000 7ff5c7200700 1 -- 172.21.15.1:0/2795004084 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5c01a1300 con 0x7ff5c0074d80 2024-02-02T15:24:51.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:51.029+0000 7ff5c7200700 1 -- 172.21.15.1:0/2795004084 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5c0107390 con 0x7ff5c0074d80 2024-02-02T15:24:51.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:51.032+0000 7ff5bcc00700 1 -- 172.21.15.1:0/2795004084 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) v1 ==== 135069+0+0 (secure 0 0 0) 0x7ff5a800cc50 con 0x7ff5c0074d80 2024-02-02T15:24:51.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:51.034+0000 7ff5bcc00700 1 --2- 172.21.15.1:0/2795004084 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7ff5a00a05a0 0x7ff5a00a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:51.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:51.034+0000 7ff5bcc00700 1 -- 172.21.15.1:0/2795004084 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8698+0+0 (secure 0 0 0) 0x7ff5a8014070 con 0x7ff5c0074d80 2024-02-02T15:24:51.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:51.034+0000 7ff5bfe00700 1 --2- 172.21.15.1:0/2795004084 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7ff5a00a05a0 0x7ff5a00a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:51.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:51.035+0000 7ff5bfe00700 1 --2- 172.21.15.1:0/2795004084 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7ff5a00a05a0 0x7ff5a00a2a60 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff5b000b2e0 tx=0x7ff5b0005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:24:51.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:51.036+0000 7ff5bcc00700 1 -- 172.21.15.1:0/2795004084 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7ff5a8095780 con 0x7ff5c0074d80 2024-02-02T15:24:51.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:50 smithi071 ceph-mon[44707]: pgmap v233: 97 pgs: 5 creating+peering, 44 active+clean, 48 unknown; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 560 B/s wr, 8 op/s 2024-02-02T15:24:51.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:50 smithi071 ceph-mon[44707]: osdmap e81: 12 total, 12 up, 12 in 2024-02-02T15:24:51.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:50 smithi142 ceph-mon[45728]: pgmap v233: 97 pgs: 5 creating+peering, 44 active+clean, 48 unknown; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 560 B/s wr, 8 op/s 2024-02-02T15:24:51.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:50 smithi142 ceph-mon[45728]: osdmap e81: 12 total, 12 up, 12 in 2024-02-02T15:24:51.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:50 smithi001 ceph-mon[35470]: pgmap v233: 97 pgs: 5 creating+peering, 44 active+clean, 48 unknown; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 560 B/s wr, 8 op/s 2024-02-02T15:24:51.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:50 smithi001 ceph-mon[35470]: osdmap e81: 12 total, 12 up, 12 in 2024-02-02T15:24:51.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:51.247+0000 7ff5c7200700 1 -- 172.21.15.1:0/2795004084 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7ff5c002cec0 con 0x7ff5c0074d80 2024-02-02T15:24:51.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:51.793+0000 7ff5bcc00700 1 -- 172.21.15.1:0/2795004084 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v83) v1 ==== 169+0+0 (secure 0 0 0) 0x7ff5a8003dd0 con 0x7ff5c0074d80 2024-02-02T15:24:51.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:51.800+0000 7ff5c7200700 1 -- 172.21.15.1:0/2795004084 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7ff5a00a05a0 msgr2=0x7ff5a00a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:51.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:51.800+0000 7ff5c7200700 1 --2- 172.21.15.1:0/2795004084 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7ff5a00a05a0 0x7ff5a00a2a60 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff5b000b2e0 tx=0x7ff5b0005fb0 comp rx=0 tx=0).stop 2024-02-02T15:24:51.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:51.800+0000 7ff5c7200700 1 -- 172.21.15.1:0/2795004084 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff5c0074d80 msgr2=0x7ff5c0199290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:51.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:51.800+0000 7ff5c7200700 1 --2- 172.21.15.1:0/2795004084 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff5c0074d80 0x7ff5c0199290 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff5a800e970 tx=0x7ff5a800ec80 comp rx=0 tx=0).stop 2024-02-02T15:24:51.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:51.801+0000 7ff5c7200700 1 -- 172.21.15.1:0/2795004084 shutdown_connections 2024-02-02T15:24:51.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:51.801+0000 7ff5c7200700 1 --2- 172.21.15.1:0/2795004084 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7ff5a00a05a0 0x7ff5a00a2a60 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:51.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:51.801+0000 7ff5c7200700 1 --2- 172.21.15.1:0/2795004084 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5c0108ce0 0x7ff5c01a0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:51.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:51.801+0000 7ff5c7200700 1 --2- 172.21.15.1:0/2795004084 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5c00737a0 0x7ff5c0198d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:51.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:51.801+0000 7ff5c7200700 1 --2- 172.21.15.1:0/2795004084 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff5c0074d80 0x7ff5c0199290 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:51.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:51.801+0000 7ff5c7200700 1 -- 172.21.15.1:0/2795004084 >> 172.21.15.1:0/2795004084 conn(0x7ff5c00fc390 msgr2=0x7ff5c00fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:24:51.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:51.802+0000 7ff5c7200700 1 -- 172.21.15.1:0/2795004084 shutdown_connections 2024-02-02T15:24:51.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:51.802+0000 7ff5c7200700 1 -- 172.21.15.1:0/2795004084 wait complete. 2024-02-02T15:24:51.807 INFO:teuthology.orchestra.run.smithi001.stderr:set pool 4 allow_ec_overwrites to true 2024-02-02T15:24:51.834 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-02-02T15:24:52.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.037+0000 7f3856200700 1 -- 172.21.15.1:0/3418139473 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3850102930 msgr2=0x7f3850109250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:52.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.037+0000 7f3856200700 1 --2- 172.21.15.1:0/3418139473 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3850102930 0x7f3850109250 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f384800b0a0 tx=0x7f384800b3b0 comp rx=0 tx=0).stop 2024-02-02T15:24:52.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.037+0000 7f3856200700 1 -- 172.21.15.1:0/3418139473 shutdown_connections 2024-02-02T15:24:52.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.037+0000 7f3856200700 1 --2- 172.21.15.1:0/3418139473 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3850102930 0x7f3850109250 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:52.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.037+0000 7f3856200700 1 --2- 172.21.15.1:0/3418139473 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3850100e20 0x7f3850101240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:52.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.037+0000 7f3856200700 1 --2- 172.21.15.1:0/3418139473 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3850101f70 0x7f38501023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:52.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.037+0000 7f3856200700 1 -- 172.21.15.1:0/3418139473 >> 172.21.15.1:0/3418139473 conn(0x7f38500fc390 msgr2=0x7f38500fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:24:52.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.037+0000 7f3856200700 1 -- 172.21.15.1:0/3418139473 shutdown_connections 2024-02-02T15:24:52.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.037+0000 7f3856200700 1 -- 172.21.15.1:0/3418139473 wait complete. 2024-02-02T15:24:52.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.038+0000 7f3856200700 1 Processor -- start 2024-02-02T15:24:52.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.039+0000 7f3856200700 1 -- start start 2024-02-02T15:24:52.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.039+0000 7f3856200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3850100e20 0x7f3850198df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:52.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.039+0000 7f3856200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3850101f70 0x7f3850199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:52.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.039+0000 7f3856200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3850102930 0x7f38501a03a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:52.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.039+0000 7f3856200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38501a0970 con 0x7f3850101f70 2024-02-02T15:24:52.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.039+0000 7f3856200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38501a0ab0 con 0x7f3850100e20 2024-02-02T15:24:52.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.039+0000 7f3856200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38501a0c40 con 0x7f3850102930 2024-02-02T15:24:52.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.039+0000 7f384ea00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3850101f70 0x7f3850199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:52.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.039+0000 7f384ea00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3850101f70 0x7f3850199330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53374/0 (socket says 172.21.15.1:53374) 2024-02-02T15:24:52.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.039+0000 7f384ea00700 1 -- 172.21.15.1:0/628172116 learned_addr learned my addr 172.21.15.1:0/628172116 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:24:52.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.039+0000 7f384f400700 1 --2- 172.21.15.1:0/628172116 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3850100e20 0x7f3850198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:52.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.039+0000 7f384fe00700 1 --2- 172.21.15.1:0/628172116 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3850102930 0x7f38501a03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:52.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.040+0000 7f384f400700 1 -- 172.21.15.1:0/628172116 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3850102930 msgr2=0x7f38501a03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:52.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.040+0000 7f384f400700 1 --2- 172.21.15.1:0/628172116 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3850102930 0x7f38501a03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:52.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.040+0000 7f384f400700 1 -- 172.21.15.1:0/628172116 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3850101f70 msgr2=0x7f3850199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:52.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.040+0000 7f384f400700 1 --2- 172.21.15.1:0/628172116 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3850101f70 0x7f3850199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:52.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.040+0000 7f384f400700 1 -- 172.21.15.1:0/628172116 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3848009d00 con 0x7f3850100e20 2024-02-02T15:24:52.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.040+0000 7f384f400700 1 --2- 172.21.15.1:0/628172116 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3850100e20 0x7f3850198df0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f384000e9f0 tx=0x7f384000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:24:52.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.040+0000 7f3847e00700 1 -- 172.21.15.1:0/628172116 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f384000c9c0 con 0x7f3850100e20 2024-02-02T15:24:52.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.040+0000 7f3847e00700 1 -- 172.21.15.1:0/628172116 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3840010430 con 0x7f3850100e20 2024-02-02T15:24:52.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.040+0000 7f3856200700 1 -- 172.21.15.1:0/628172116 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38501a0f20 con 0x7f3850100e20 2024-02-02T15:24:52.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.040+0000 7f3847e00700 1 -- 172.21.15.1:0/628172116 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3840018680 con 0x7f3850100e20 2024-02-02T15:24:52.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.040+0000 7f3856200700 1 -- 172.21.15.1:0/628172116 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38501a1340 con 0x7f3850100e20 2024-02-02T15:24:52.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.041+0000 7f3856200700 1 -- 172.21.15.1:0/628172116 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f385010c670 con 0x7f3850100e20 2024-02-02T15:24:52.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.044+0000 7f3847e00700 1 -- 172.21.15.1:0/628172116 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 135069+0+0 (secure 0 0 0) 0x7f3840010860 con 0x7f3850100e20 2024-02-02T15:24:52.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.045+0000 7f3847e00700 1 --2- 172.21.15.1:0/628172116 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f38300a0520 0x7f38300a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:52.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.045+0000 7f3847e00700 1 -- 172.21.15.1:0/628172116 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8698+0+0 (secure 0 0 0) 0x7f3840014070 con 0x7f3850100e20 2024-02-02T15:24:52.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.045+0000 7f384ea00700 1 --2- 172.21.15.1:0/628172116 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f38300a0520 0x7f38300a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:52.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.046+0000 7f384ea00700 1 --2- 172.21.15.1:0/628172116 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f38300a0520 0x7f38300a29e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3838009d20 tx=0x7f3838009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:24:52.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.047+0000 7f3847e00700 1 -- 172.21.15.1:0/628172116 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f38400957e0 con 0x7f3850100e20 2024-02-02T15:24:52.114 INFO:journalctl@ceph.mds.b.smithi071.stdout:Feb 02 15:24:51 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mds-b[79660]: 2024-02-02T15:24:51.822+0000 7f2045bbd700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-02-02T15:24:52.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:51 smithi071 ceph-mon[44707]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-02-02T15:24:52.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:51 smithi071 ceph-mon[44707]: osdmap e82: 12 total, 12 up, 12 in 2024-02-02T15:24:52.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:51 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/2795004084' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-02-02T15:24:52.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:51 smithi142 ceph-mon[45728]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-02-02T15:24:52.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:51 smithi142 ceph-mon[45728]: osdmap e82: 12 total, 12 up, 12 in 2024-02-02T15:24:52.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:51 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/2795004084' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-02-02T15:24:52.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:51 smithi001 ceph-mon[35470]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-02-02T15:24:52.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:51 smithi001 ceph-mon[35470]: osdmap e82: 12 total, 12 up, 12 in 2024-02-02T15:24:52.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:51 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/2795004084' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-02-02T15:24:52.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.308+0000 7f3856200700 1 -- 172.21.15.1:0/628172116 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7f38501a1ae0 con 0x7f3850100e20 2024-02-02T15:24:52.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.316+0000 7f3847e00700 1 -- 172.21.15.1:0/628172116 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) v1 ==== 137+0+0 (secure 0 0 0) 0x7f3840098e00 con 0x7f3850100e20 2024-02-02T15:24:52.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.318+0000 7f3856200700 1 -- 172.21.15.1:0/628172116 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f38300a0520 msgr2=0x7f38300a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:52.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.318+0000 7f3856200700 1 --2- 172.21.15.1:0/628172116 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f38300a0520 0x7f38300a29e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3838009d20 tx=0x7f3838009260 comp rx=0 tx=0).stop 2024-02-02T15:24:52.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.318+0000 7f3856200700 1 -- 172.21.15.1:0/628172116 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3850100e20 msgr2=0x7f3850198df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:52.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.318+0000 7f3856200700 1 --2- 172.21.15.1:0/628172116 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3850100e20 0x7f3850198df0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f384000e9f0 tx=0x7f384000ed00 comp rx=0 tx=0).stop 2024-02-02T15:24:52.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.319+0000 7f3856200700 1 -- 172.21.15.1:0/628172116 shutdown_connections 2024-02-02T15:24:52.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.319+0000 7f3856200700 1 --2- 172.21.15.1:0/628172116 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f38300a0520 0x7f38300a29e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:52.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.319+0000 7f3856200700 1 --2- 172.21.15.1:0/628172116 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3850102930 0x7f38501a03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:52.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.319+0000 7f3856200700 1 --2- 172.21.15.1:0/628172116 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3850100e20 0x7f3850198df0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:52.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.319+0000 7f3856200700 1 --2- 172.21.15.1:0/628172116 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3850101f70 0x7f3850199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:52.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.319+0000 7f3856200700 1 -- 172.21.15.1:0/628172116 >> 172.21.15.1:0/628172116 conn(0x7f38500fc390 msgr2=0x7f38500fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:24:52.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.319+0000 7f3856200700 1 -- 172.21.15.1:0/628172116 shutdown_connections 2024-02-02T15:24:52.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.319+0000 7f3856200700 1 -- 172.21.15.1:0/628172116 wait complete. 2024-02-02T15:24:52.320 INFO:teuthology.orchestra.run.smithi001.stderr:added data pool 4 to fsmap 2024-02-02T15:24:52.338 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T15:24:52.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.535+0000 7f3e8da00700 1 -- 172.21.15.1:0/14472098 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3e881028d0 msgr2=0x7f3e88109180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:52.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.535+0000 7f3e8da00700 1 --2- 172.21.15.1:0/14472098 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3e881028d0 0x7f3e88109180 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3e80009fc0 tx=0x7f3e8000b330 comp rx=0 tx=0).stop 2024-02-02T15:24:52.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.535+0000 7f3e8da00700 1 -- 172.21.15.1:0/14472098 shutdown_connections 2024-02-02T15:24:52.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.535+0000 7f3e8da00700 1 --2- 172.21.15.1:0/14472098 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3e881028d0 0x7f3e88109180 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:52.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.535+0000 7f3e8da00700 1 --2- 172.21.15.1:0/14472098 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3e88100d40 0x7f3e88101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:52.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.535+0000 7f3e8da00700 1 --2- 172.21.15.1:0/14472098 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3e88101f30 0x7f3e88102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:52.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.535+0000 7f3e8da00700 1 -- 172.21.15.1:0/14472098 >> 172.21.15.1:0/14472098 conn(0x7f3e880fc290 msgr2=0x7f3e880fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:24:52.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.535+0000 7f3e8da00700 1 -- 172.21.15.1:0/14472098 shutdown_connections 2024-02-02T15:24:52.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.535+0000 7f3e8da00700 1 -- 172.21.15.1:0/14472098 wait complete. 2024-02-02T15:24:52.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.536+0000 7f3e8da00700 1 Processor -- start 2024-02-02T15:24:52.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.536+0000 7f3e8da00700 1 -- start start 2024-02-02T15:24:52.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.536+0000 7f3e8da00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3e88100d40 0x7f3e88198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:52.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.536+0000 7f3e8da00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3e88101f30 0x7f3e88199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:52.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.536+0000 7f3e8da00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3e881028d0 0x7f3e881a0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:52.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.537+0000 7f3e8da00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e881a08d0 con 0x7f3e88101f30 2024-02-02T15:24:52.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.537+0000 7f3e8da00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e881a0a10 con 0x7f3e88100d40 2024-02-02T15:24:52.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.537+0000 7f3e8da00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e881a0ba0 con 0x7f3e881028d0 2024-02-02T15:24:52.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.537+0000 7f3e86a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3e88100d40 0x7f3e88198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:52.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.537+0000 7f3e86000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3e88101f30 0x7f3e88199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:52.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.537+0000 7f3e86a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3e88100d40 0x7f3e88198d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:47030/0 (socket says 172.21.15.1:47030) 2024-02-02T15:24:52.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.537+0000 7f3e86a00700 1 -- 172.21.15.1:0/3781541447 learned_addr learned my addr 172.21.15.1:0/3781541447 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:24:52.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.537+0000 7f3e87400700 1 --2- 172.21.15.1:0/3781541447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3e881028d0 0x7f3e881a0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:52.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.537+0000 7f3e86a00700 1 -- 172.21.15.1:0/3781541447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3e881028d0 msgr2=0x7f3e881a0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:52.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.537+0000 7f3e86a00700 1 --2- 172.21.15.1:0/3781541447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3e881028d0 0x7f3e881a0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:52.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.537+0000 7f3e86a00700 1 -- 172.21.15.1:0/3781541447 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3e88101f30 msgr2=0x7f3e88199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:52.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.537+0000 7f3e86a00700 1 --2- 172.21.15.1:0/3781541447 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3e88101f30 0x7f3e88199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:52.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.537+0000 7f3e86a00700 1 -- 172.21.15.1:0/3781541447 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e80009c40 con 0x7f3e88100d40 2024-02-02T15:24:52.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.537+0000 7f3e87400700 1 --2- 172.21.15.1:0/3781541447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3e881028d0 0x7f3e881a0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:24:52.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.537+0000 7f3e86000700 1 --2- 172.21.15.1:0/3781541447 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3e88101f30 0x7f3e88199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:24:52.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.537+0000 7f3e86a00700 1 --2- 172.21.15.1:0/3781541447 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3e88100d40 0x7f3e88198d50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3e7800e970 tx=0x7f3e7800ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:24:52.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.538+0000 7f3e7f400700 1 -- 172.21.15.1:0/3781541447 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e78009790 con 0x7f3e88100d40 2024-02-02T15:24:52.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.538+0000 7f3e8da00700 1 -- 172.21.15.1:0/3781541447 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e881a0e80 con 0x7f3e88100d40 2024-02-02T15:24:52.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.538+0000 7f3e7f400700 1 -- 172.21.15.1:0/3781541447 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3e7800eeb0 con 0x7f3e88100d40 2024-02-02T15:24:52.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.538+0000 7f3e7f400700 1 -- 172.21.15.1:0/3781541447 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e78003d20 con 0x7f3e88100d40 2024-02-02T15:24:52.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.538+0000 7f3e8da00700 1 -- 172.21.15.1:0/3781541447 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e881a12b0 con 0x7f3e88100d40 2024-02-02T15:24:52.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.539+0000 7f3e8da00700 1 -- 172.21.15.1:0/3781541447 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e8810c5b0 con 0x7f3e88100d40 2024-02-02T15:24:52.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.541+0000 7f3e7f400700 1 -- 172.21.15.1:0/3781541447 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 135069+0+0 (secure 0 0 0) 0x7f3e78004870 con 0x7f3e88100d40 2024-02-02T15:24:52.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.542+0000 7f3e7f400700 1 --2- 172.21.15.1:0/3781541447 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f3e680a0520 0x7f3e680a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:52.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.542+0000 7f3e7f400700 1 -- 172.21.15.1:0/3781541447 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f3e78014070 con 0x7f3e88100d40 2024-02-02T15:24:52.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.542+0000 7f3e86000700 1 --2- 172.21.15.1:0/3781541447 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f3e680a0520 0x7f3e680a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:52.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.543+0000 7f3e86000700 1 --2- 172.21.15.1:0/3781541447 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f3e680a0520 0x7f3e680a29e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3e70005fd0 tx=0x7f3e70009310 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:24:52.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.544+0000 7f3e7f400700 1 -- 172.21.15.1:0/3781541447 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f3e78095900 con 0x7f3e88100d40 2024-02-02T15:24:52.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.807+0000 7f3e8da00700 1 -- 172.21.15.1:0/3781541447 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3e881a1920 con 0x7f3e88100d40 2024-02-02T15:24:52.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.808+0000 7f3e7f400700 1 -- 172.21.15.1:0/3781541447 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+10755 (secure 0 0 0) 0x7f3e78098f20 con 0x7f3e88100d40 2024-02-02T15:24:52.808 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:24:52.808 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2922240448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2922240448},{"type":"v1","addr":"172.21.15.1:6835","nonce":2922240448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2729497431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2729497431},{"type":"v1","addr":"172.21.15.1:6837","nonce":2729497431}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14517,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/1218303683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":1218303683},{"type":"v1","addr":"172.21.15.1:6839","nonce":1218303683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14529,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/2059197496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2059197496},{"type":"v1","addr":"172.21.15.1:6841","nonce":2059197496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/7648986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":7648986},{"type":"v1","addr":"172.21.15.142:6833","nonce":7648986}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/3482746094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3482746094},{"type":"v1","addr":"172.21.15.71:6833","nonce":3482746094}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/388207814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":388207814},{"type":"v1","addr":"172.21.15.71:6835","nonce":388207814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24502,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/1052606547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1052606547},{"type":"v1","addr":"172.21.15.142:6835","nonce":1052606547}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24514,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/2870289019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2870289019},{"type":"v1","addr":"172.21.15.142:6837","nonce":2870289019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24518,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/628016199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":628016199},{"type":"v1","addr":"172.21.15.71:6837","nonce":628016199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24529,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6839/321675421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":321675421},{"type":"v1","addr":"172.21.15.142:6839","nonce":321675421}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T15:24:47.761435+0000","modified":"2024-02-02T15:24:52.311168+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24557},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/1268086493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1268086493},{"type":"v1","addr":"172.21.15.71:6839","nonce":1268086493}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-02T15:24:52.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.810+0000 7f3e8da00700 1 -- 172.21.15.1:0/3781541447 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f3e680a0520 msgr2=0x7f3e680a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:52.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.810+0000 7f3e8da00700 1 --2- 172.21.15.1:0/3781541447 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f3e680a0520 0x7f3e680a29e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3e70005fd0 tx=0x7f3e70009310 comp rx=0 tx=0).stop 2024-02-02T15:24:52.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.810+0000 7f3e8da00700 1 -- 172.21.15.1:0/3781541447 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3e88100d40 msgr2=0x7f3e88198d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:52.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.810+0000 7f3e8da00700 1 --2- 172.21.15.1:0/3781541447 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3e88100d40 0x7f3e88198d50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3e7800e970 tx=0x7f3e7800ed30 comp rx=0 tx=0).stop 2024-02-02T15:24:52.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.811+0000 7f3e8da00700 1 -- 172.21.15.1:0/3781541447 shutdown_connections 2024-02-02T15:24:52.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.811+0000 7f3e8da00700 1 --2- 172.21.15.1:0/3781541447 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f3e680a0520 0x7f3e680a29e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:52.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.811+0000 7f3e8da00700 1 --2- 172.21.15.1:0/3781541447 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3e881028d0 0x7f3e881a0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:52.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.811+0000 7f3e8da00700 1 --2- 172.21.15.1:0/3781541447 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3e88100d40 0x7f3e88198d50 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:52.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.811+0000 7f3e8da00700 1 --2- 172.21.15.1:0/3781541447 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3e88101f30 0x7f3e88199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:52.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.811+0000 7f3e8da00700 1 -- 172.21.15.1:0/3781541447 >> 172.21.15.1:0/3781541447 conn(0x7f3e880fc290 msgr2=0x7f3e881077c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:24:52.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.811+0000 7f3e8da00700 1 -- 172.21.15.1:0/3781541447 shutdown_connections 2024-02-02T15:24:52.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:52.811+0000 7f3e8da00700 1 -- 172.21.15.1:0/3781541447 wait complete. 2024-02-02T15:24:52.812 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 17 2024-02-02T15:24:52.829 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T15:24:53.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.027+0000 7f63ddc00700 1 -- 172.21.15.1:0/2720223717 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f63d8108c00 msgr2=0x7f63d810b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:53.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.027+0000 7f63ddc00700 1 --2- 172.21.15.1:0/2720223717 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f63d8108c00 0x7f63d810b090 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f63d000b0a0 tx=0x7f63d000b3b0 comp rx=0 tx=0).stop 2024-02-02T15:24:53.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.027+0000 7f63ddc00700 1 -- 172.21.15.1:0/2720223717 shutdown_connections 2024-02-02T15:24:53.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.027+0000 7f63ddc00700 1 --2- 172.21.15.1:0/2720223717 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f63d8108c00 0x7f63d810b090 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:53.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.027+0000 7f63ddc00700 1 --2- 172.21.15.1:0/2720223717 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63d8074d80 0x7f63d80731d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:53.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.027+0000 7f63ddc00700 1 --2- 172.21.15.1:0/2720223717 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f63d80737a0 0x7f63d8073c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:53.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.027+0000 7f63ddc00700 1 -- 172.21.15.1:0/2720223717 >> 172.21.15.1:0/2720223717 conn(0x7f63d80fc290 msgr2=0x7f63d80fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:24:53.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.027+0000 7f63ddc00700 1 -- 172.21.15.1:0/2720223717 shutdown_connections 2024-02-02T15:24:53.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.027+0000 7f63ddc00700 1 -- 172.21.15.1:0/2720223717 wait complete. 2024-02-02T15:24:53.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.028+0000 7f63ddc00700 1 Processor -- start 2024-02-02T15:24:53.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.029+0000 7f63ddc00700 1 -- start start 2024-02-02T15:24:53.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.029+0000 7f63ddc00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f63d80737a0 0x7f63d8198de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:53.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.029+0000 7f63ddc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63d8074d80 0x7f63d8199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:53.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.029+0000 7f63ddc00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f63d8108c00 0x7f63d81a0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:53.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.029+0000 7f63ddc00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d81a0960 con 0x7f63d80737a0 2024-02-02T15:24:53.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.029+0000 7f63ddc00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d81a0aa0 con 0x7f63d8108c00 2024-02-02T15:24:53.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.029+0000 7f63ddc00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d81a0c30 con 0x7f63d8074d80 2024-02-02T15:24:53.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.029+0000 7f63d6a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f63d80737a0 0x7f63d8198de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:53.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.029+0000 7f63d6000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63d8074d80 0x7f63d8199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:53.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.029+0000 7f63d6a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f63d80737a0 0x7f63d8198de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33640/0 (socket says 172.21.15.1:33640) 2024-02-02T15:24:53.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.030+0000 7f63d6000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63d8074d80 0x7f63d8199320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:60266/0 (socket says 172.21.15.1:60266) 2024-02-02T15:24:53.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.030+0000 7f63d6a00700 1 -- 172.21.15.1:0/3324049945 learned_addr learned my addr 172.21.15.1:0/3324049945 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:24:53.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.030+0000 7f63d7400700 1 --2- 172.21.15.1:0/3324049945 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f63d8108c00 0x7f63d81a0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:53.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.030+0000 7f63d6000700 1 -- 172.21.15.1:0/3324049945 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f63d8108c00 msgr2=0x7f63d81a0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:53.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.030+0000 7f63d6000700 1 --2- 172.21.15.1:0/3324049945 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f63d8108c00 0x7f63d81a0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:53.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.030+0000 7f63d6000700 1 -- 172.21.15.1:0/3324049945 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f63d80737a0 msgr2=0x7f63d8198de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:53.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.030+0000 7f63d6000700 1 --2- 172.21.15.1:0/3324049945 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f63d80737a0 0x7f63d8198de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:53.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.030+0000 7f63d6000700 1 -- 172.21.15.1:0/3324049945 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63d0009d00 con 0x7f63d8074d80 2024-02-02T15:24:53.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.030+0000 7f63d6a00700 1 --2- 172.21.15.1:0/3324049945 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f63d80737a0 0x7f63d8198de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:24:53.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.030+0000 7f63d7400700 1 --2- 172.21.15.1:0/3324049945 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f63d8108c00 0x7f63d81a0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:24:53.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.030+0000 7f63d6000700 1 --2- 172.21.15.1:0/3324049945 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63d8074d80 0x7f63d8199320 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f63c000e9c0 tx=0x7f63c000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:24:53.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.030+0000 7f63cf400700 1 -- 172.21.15.1:0/3324049945 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c0004220 con 0x7f63d8074d80 2024-02-02T15:24:53.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.030+0000 7f63cf400700 1 -- 172.21.15.1:0/3324049945 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63c0004380 con 0x7f63d8074d80 2024-02-02T15:24:53.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.030+0000 7f63ddc00700 1 -- 172.21.15.1:0/3324049945 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63d81a0f10 con 0x7f63d8074d80 2024-02-02T15:24:53.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.031+0000 7f63cf400700 1 -- 172.21.15.1:0/3324049945 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c0003be0 con 0x7f63d8074d80 2024-02-02T15:24:53.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.031+0000 7f63ddc00700 1 -- 172.21.15.1:0/3324049945 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63d81a1390 con 0x7f63d8074d80 2024-02-02T15:24:53.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.032+0000 7f63ddc00700 1 -- 172.21.15.1:0/3324049945 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63d81072b0 con 0x7f63d8074d80 2024-02-02T15:24:53.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.034+0000 7f63cf400700 1 -- 172.21.15.1:0/3324049945 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 16) v1 ==== 135069+0+0 (secure 0 0 0) 0x7f63c0003d40 con 0x7f63d8074d80 2024-02-02T15:24:53.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.035+0000 7f63cf400700 1 --2- 172.21.15.1:0/3324049945 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f63b80a0520 0x7f63b80a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:53.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.035+0000 7f63cf400700 1 -- 172.21.15.1:0/3324049945 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f63c0014070 con 0x7f63d8074d80 2024-02-02T15:24:53.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.035+0000 7f63d6a00700 1 --2- 172.21.15.1:0/3324049945 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f63b80a0520 0x7f63b80a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:53.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.036+0000 7f63d6a00700 1 --2- 172.21.15.1:0/3324049945 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f63b80a0520 0x7f63b80a29e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f63c8011550 tx=0x7f63c8009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:24:53.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.038+0000 7f63cf400700 1 -- 172.21.15.1:0/3324049945 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f63c0095840 con 0x7f63d8074d80 2024-02-02T15:24:53.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:52 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/2795004084' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-02-02T15:24:53.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:52 smithi071 ceph-mon[44707]: osdmap e83: 12 total, 12 up, 12 in 2024-02-02T15:24:53.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:52 smithi071 ceph-mon[44707]: pgmap v236: 129 pgs: 12 creating+peering, 80 active+clean, 37 unknown; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 10 op/s 2024-02-02T15:24:53.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:52 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/628172116' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-02-02T15:24:53.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:52 smithi071 ceph-mon[44707]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-02-02T15:24:53.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:52 smithi071 ceph-mon[44707]: osdmap e84: 12 total, 12 up, 12 in 2024-02-02T15:24:53.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:52 smithi071 ceph-mon[44707]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-02-02T15:24:53.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:52 smithi071 ceph-mon[44707]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-02T15:24:53.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:52 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/2795004084' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-02-02T15:24:53.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:52 smithi142 ceph-mon[45728]: osdmap e83: 12 total, 12 up, 12 in 2024-02-02T15:24:53.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:52 smithi142 ceph-mon[45728]: pgmap v236: 129 pgs: 12 creating+peering, 80 active+clean, 37 unknown; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 10 op/s 2024-02-02T15:24:53.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:52 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/628172116' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-02-02T15:24:53.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:52 smithi142 ceph-mon[45728]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-02-02T15:24:53.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:52 smithi142 ceph-mon[45728]: osdmap e84: 12 total, 12 up, 12 in 2024-02-02T15:24:53.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:52 smithi142 ceph-mon[45728]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-02-02T15:24:53.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:52 smithi142 ceph-mon[45728]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-02T15:24:53.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:52 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/2795004084' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-02-02T15:24:53.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:52 smithi001 ceph-mon[35470]: osdmap e83: 12 total, 12 up, 12 in 2024-02-02T15:24:53.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:52 smithi001 ceph-mon[35470]: pgmap v236: 129 pgs: 12 creating+peering, 80 active+clean, 37 unknown; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 10 op/s 2024-02-02T15:24:53.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:52 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/628172116' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-02-02T15:24:53.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:52 smithi001 ceph-mon[35470]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-02-02T15:24:53.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:52 smithi001 ceph-mon[35470]: osdmap e84: 12 total, 12 up, 12 in 2024-02-02T15:24:53.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:52 smithi001 ceph-mon[35470]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-02-02T15:24:53.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:52 smithi001 ceph-mon[35470]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-02T15:24:53.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.254+0000 7f63ddc00700 1 -- 172.21.15.1:0/3324049945 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f63d802cee0 con 0x7f63d8074d80 2024-02-02T15:24:53.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.256+0000 7f63cf400700 1 -- 172.21.15.1:0/3324049945 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+23841 (secure 0 0 0) 0x7f63c0098e60 con 0x7f63d8074d80 2024-02-02T15:24:53.256 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:24:53.256 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":84,"fsid":"371a409c-c1de-11ee-95b6-87774f69a715","created":"2024-02-02T15:18:26.746506+0000","modified":"2024-02-02T15:24:52.310888+0000","last_up_change":"2024-02-02T15:24:08.800951+0000","last_in_change":"2024-02-02T15:23:52.784280+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T15:21:22.471754+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-02T15:24:46.195779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-02T15:24:47.226403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-02T15:24:50.300462+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"73e8a07f-3560-4c19-85ed-030d4d8f0f23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6803","nonce":2591906423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6805","nonce":2591906423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6809","nonce":2591906423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6807","nonce":2591906423}]},"public_addr":"172.21.15.1:6803/2591906423","cluster_addr":"172.21.15.1:6805/2591906423","heartbeat_back_addr":"172.21.15.1:6809/2591906423","heartbeat_front_addr":"172.21.15.1:6807/2591906423","state":["exists","up"]},{"osd":1,"uuid":"e53cffec-0ff9-4ed9-8dcf-ff47d04db2bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6801","nonce":747581719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6803","nonce":747581719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6807","nonce":747581719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6805","nonce":747581719}]},"public_addr":"172.21.15.71:6801/747581719","cluster_addr":"172.21.15.71:6803/747581719","heartbeat_back_addr":"172.21.15.71:6807/747581719","heartbeat_front_addr":"172.21.15.71:6805/747581719","state":["exists","up"]},{"osd":2,"uuid":"856c2347-294c-482b-8595-ad77f4792db4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6801","nonce":377511414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6803","nonce":377511414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6807","nonce":377511414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6805","nonce":377511414}]},"public_addr":"172.21.15.142:6801/377511414","cluster_addr":"172.21.15.142:6803/377511414","heartbeat_back_addr":"172.21.15.142:6807/377511414","heartbeat_front_addr":"172.21.15.142:6805/377511414","state":["exists","up"]},{"osd":3,"uuid":"4b946ad2-2da9-4e0b-9d61-d039d8669a42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6811","nonce":1086124477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6813","nonce":1086124477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6817","nonce":1086124477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6815","nonce":1086124477}]},"public_addr":"172.21.15.1:6811/1086124477","cluster_addr":"172.21.15.1:6813/1086124477","heartbeat_back_addr":"172.21.15.1:6817/1086124477","heartbeat_front_addr":"172.21.15.1:6815/1086124477","state":["exists","up"]},{"osd":4,"uuid":"43e36c43-47a0-4634-a45b-1da14a0e19d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6809","nonce":3095380305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6811","nonce":3095380305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6815","nonce":3095380305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6813","nonce":3095380305}]},"public_addr":"172.21.15.71:6809/3095380305","cluster_addr":"172.21.15.71:6811/3095380305","heartbeat_back_addr":"172.21.15.71:6815/3095380305","heartbeat_front_addr":"172.21.15.71:6813/3095380305","state":["exists","up"]},{"osd":5,"uuid":"eb48160e-ea7c-42f7-9477-be110bc8a34e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6809","nonce":4195149262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6811","nonce":4195149262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6815","nonce":4195149262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6813","nonce":4195149262}]},"public_addr":"172.21.15.142:6809/4195149262","cluster_addr":"172.21.15.142:6811/4195149262","heartbeat_back_addr":"172.21.15.142:6815/4195149262","heartbeat_front_addr":"172.21.15.142:6813/4195149262","state":["exists","up"]},{"osd":6,"uuid":"0ec4501a-5853-4ca6-b325-231f5b72335e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6819","nonce":1560153428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6821","nonce":1560153428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6825","nonce":1560153428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6823","nonce":1560153428}]},"public_addr":"172.21.15.1:6819/1560153428","cluster_addr":"172.21.15.1:6821/1560153428","heartbeat_back_addr":"172.21.15.1:6825/1560153428","heartbeat_front_addr":"172.21.15.1:6823/1560153428","state":["exists","up"]},{"osd":7,"uuid":"029debe8-9cdf-44f6-9aea-cabd1d185f93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6817","nonce":4080955538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6819","nonce":4080955538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6823","nonce":4080955538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6821","nonce":4080955538}]},"public_addr":"172.21.15.71:6817/4080955538","cluster_addr":"172.21.15.71:6819/4080955538","heartbeat_back_addr":"172.21.15.71:6823/4080955538","heartbeat_front_addr":"172.21.15.71:6821/4080955538","state":["exists","up"]},{"osd":8,"uuid":"7c90896c-9955-4c1b-aaca-7531dd5457f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6817","nonce":3942038209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6819","nonce":3942038209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6823","nonce":3942038209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6821","nonce":3942038209}]},"public_addr":"172.21.15.142:6817/3942038209","cluster_addr":"172.21.15.142:6819/3942038209","heartbeat_back_addr":"172.21.15.142:6823/3942038209","heartbeat_front_addr":"172.21.15.142:6821/3942038209","state":["exists","up"]},{"osd":9,"uuid":"7e165e68-31a8-4598-a577-de3e90ae5c5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6827","nonce":2716544850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6829","nonce":2716544850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6833","nonce":2716544850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6831","nonce":2716544850}]},"public_addr":"172.21.15.1:6827/2716544850","cluster_addr":"172.21.15.1:6829/2716544850","heartbeat_back_addr":"172.21.15.1:6833/2716544850","heartbeat_front_addr":"172.21.15.1:6831/2716544850","state":["exists","up"]},{"osd":10,"uuid":"48cd5fdb-0887-444a-aab8-a1de24282bb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6825","nonce":84691841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6827","nonce":84691841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6831","nonce":84691841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6829","nonce":84691841}]},"public_addr":"172.21.15.71:6825/84691841","cluster_addr":"172.21.15.71:6827/84691841","heartbeat_back_addr":"172.21.15.71:6831/84691841","heartbeat_front_addr":"172.21.15.71:6829/84691841","state":["exists","up"]},{"osd":11,"uuid":"f011dd27-11b0-4166-adfe-76756533163a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6825","nonce":2290179934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6827","nonce":2290179934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6831","nonce":2290179934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6829","nonce":2290179934}]},"public_addr":"172.21.15.142:6825/2290179934","cluster_addr":"172.21.15.142:6827/2290179934","heartbeat_back_addr":"172.21.15.142:6831/2290179934","heartbeat_front_addr":"172.21.15.142:6829/2290179934","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:20:44.060692+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:01.903817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:19.368258+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:40.028037+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:57.513361+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:22:14.805944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:22:35.188439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:22:53.014412+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:23:10.507420+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:23:31.849765+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:23:50.070612+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:24:07.388625+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/4092532060":"2024-02-03T15:19:25.836171+0000","172.21.15.1:6801/395498296":"2024-02-03T15:19:25.836171+0000","172.21.15.1:6800/395498296":"2024-02-03T15:19:25.836171+0000","172.21.15.1:0/1575373489":"2024-02-03T15:19:25.836171+0000","172.21.15.1:0/865983374":"2024-02-03T15:19:01.431596+0000","172.21.15.1:0/3165960227":"2024-02-03T15:19:01.431596+0000","172.21.15.1:0/2350830293":"2024-02-03T15:19:01.431596+0000","172.21.15.1:0/2959690250":"2024-02-03T15:19:25.836171+0000","172.21.15.1:6800/1162530732":"2024-02-03T15:19:01.431596+0000","172.21.15.1:6801/1162530732":"2024-02-03T15:19:01.431596+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T15:24:53.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.259+0000 7f63ddc00700 1 -- 172.21.15.1:0/3324049945 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f63b80a0520 msgr2=0x7f63b80a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:53.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.259+0000 7f63ddc00700 1 --2- 172.21.15.1:0/3324049945 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f63b80a0520 0x7f63b80a29e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f63c8011550 tx=0x7f63c8009260 comp rx=0 tx=0).stop 2024-02-02T15:24:53.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.259+0000 7f63ddc00700 1 -- 172.21.15.1:0/3324049945 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63d8074d80 msgr2=0x7f63d8199320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:53.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.259+0000 7f63ddc00700 1 --2- 172.21.15.1:0/3324049945 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63d8074d80 0x7f63d8199320 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f63c000e9c0 tx=0x7f63c000ecd0 comp rx=0 tx=0).stop 2024-02-02T15:24:53.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.259+0000 7f63ddc00700 1 -- 172.21.15.1:0/3324049945 shutdown_connections 2024-02-02T15:24:53.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.259+0000 7f63ddc00700 1 --2- 172.21.15.1:0/3324049945 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f63d8108c00 0x7f63d81a0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:53.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.259+0000 7f63ddc00700 1 --2- 172.21.15.1:0/3324049945 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f63d80737a0 0x7f63d8198de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:53.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.259+0000 7f63ddc00700 1 --2- 172.21.15.1:0/3324049945 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f63b80a0520 0x7f63b80a29e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:53.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.259+0000 7f63ddc00700 1 --2- 172.21.15.1:0/3324049945 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63d8074d80 0x7f63d8199320 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:53.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.259+0000 7f63ddc00700 1 -- 172.21.15.1:0/3324049945 >> 172.21.15.1:0/3324049945 conn(0x7f63d80fc290 msgr2=0x7f63d80fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:24:53.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.259+0000 7f63ddc00700 1 -- 172.21.15.1:0/3324049945 shutdown_connections 2024-02-02T15:24:53.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.259+0000 7f63ddc00700 1 -- 172.21.15.1:0/3324049945 wait complete. 2024-02-02T15:24:53.278 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T15:24:53.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.476+0000 7fa7c2400700 1 -- 172.21.15.1:0/2776713662 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa7bc100d40 msgr2=0x7fa7bc101160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:53.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.476+0000 7fa7c2400700 1 --2- 172.21.15.1:0/2776713662 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa7bc100d40 0x7fa7bc101160 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac0099c0 tx=0x7fa7ac009cd0 comp rx=0 tx=0).stop 2024-02-02T15:24:53.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.476+0000 7fa7c2400700 1 -- 172.21.15.1:0/2776713662 shutdown_connections 2024-02-02T15:24:53.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.476+0000 7fa7c2400700 1 --2- 172.21.15.1:0/2776713662 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7bc1028d0 0x7fa7bc109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:53.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.476+0000 7fa7c2400700 1 --2- 172.21.15.1:0/2776713662 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa7bc100d40 0x7fa7bc101160 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:53.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.476+0000 7fa7c2400700 1 --2- 172.21.15.1:0/2776713662 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7bc101f30 0x7fa7bc102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:53.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.476+0000 7fa7c2400700 1 -- 172.21.15.1:0/2776713662 >> 172.21.15.1:0/2776713662 conn(0x7fa7bc0fc290 msgr2=0x7fa7bc0fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:24:53.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.476+0000 7fa7c2400700 1 -- 172.21.15.1:0/2776713662 shutdown_connections 2024-02-02T15:24:53.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.476+0000 7fa7c2400700 1 -- 172.21.15.1:0/2776713662 wait complete. 2024-02-02T15:24:53.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.477+0000 7fa7c2400700 1 Processor -- start 2024-02-02T15:24:53.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.477+0000 7fa7c2400700 1 -- start start 2024-02-02T15:24:53.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.478+0000 7fa7c2400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa7bc100d40 0x7fa7bc074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:53.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.478+0000 7fa7c2400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7bc101f30 0x7fa7bc073130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:53.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.478+0000 7fa7c2400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7bc1028d0 0x7fa7bc073670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:53.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.478+0000 7fa7c2400700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7bc073e10 con 0x7fa7bc100d40 2024-02-02T15:24:53.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.478+0000 7fa7c2400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7bc073f80 con 0x7fa7bc101f30 2024-02-02T15:24:53.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.478+0000 7fa7c2400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7bc1a0ab0 con 0x7fa7bc1028d0 2024-02-02T15:24:53.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.478+0000 7fa7bb400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa7bc100d40 0x7fa7bc074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:53.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.478+0000 7fa7baa00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7bc101f30 0x7fa7bc073130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:53.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.478+0000 7fa7baa00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7bc101f30 0x7fa7bc073130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:56468/0 (socket says 172.21.15.1:56468) 2024-02-02T15:24:53.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.478+0000 7fa7baa00700 1 -- 172.21.15.1:0/3427659543 learned_addr learned my addr 172.21.15.1:0/3427659543 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:24:53.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.478+0000 7fa7bbe00700 1 --2- 172.21.15.1:0/3427659543 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7bc1028d0 0x7fa7bc073670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:53.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.478+0000 7fa7baa00700 1 -- 172.21.15.1:0/3427659543 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7bc1028d0 msgr2=0x7fa7bc073670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:53.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.478+0000 7fa7baa00700 1 --2- 172.21.15.1:0/3427659543 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7bc1028d0 0x7fa7bc073670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:53.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.478+0000 7fa7baa00700 1 -- 172.21.15.1:0/3427659543 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa7bc100d40 msgr2=0x7fa7bc074af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:53.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.478+0000 7fa7baa00700 1 --2- 172.21.15.1:0/3427659543 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa7bc100d40 0x7fa7bc074af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:53.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.478+0000 7fa7baa00700 1 -- 172.21.15.1:0/3427659543 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7ac0095f0 con 0x7fa7bc101f30 2024-02-02T15:24:53.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.479+0000 7fa7baa00700 1 --2- 172.21.15.1:0/3427659543 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7bc101f30 0x7fa7bc073130 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa7a400e9a0 tx=0x7fa7a400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:24:53.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.479+0000 7fa7b3e00700 1 -- 172.21.15.1:0/3427659543 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a4009790 con 0x7fa7bc101f30 2024-02-02T15:24:53.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.479+0000 7fa7b3e00700 1 -- 172.21.15.1:0/3427659543 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa7a4010430 con 0x7fa7bc101f30 2024-02-02T15:24:53.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.479+0000 7fa7c2400700 1 -- 172.21.15.1:0/3427659543 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7bc1a0d90 con 0x7fa7bc101f30 2024-02-02T15:24:53.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.479+0000 7fa7b3e00700 1 -- 172.21.15.1:0/3427659543 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a4018680 con 0x7fa7bc101f30 2024-02-02T15:24:53.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.479+0000 7fa7c2400700 1 -- 172.21.15.1:0/3427659543 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7bc1a1240 con 0x7fa7bc101f30 2024-02-02T15:24:53.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.480+0000 7fa7c2400700 1 -- 172.21.15.1:0/3427659543 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7bc107c70 con 0x7fa7bc101f30 2024-02-02T15:24:53.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.482+0000 7fa7b3e00700 1 -- 172.21.15.1:0/3427659543 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 135069+0+0 (secure 0 0 0) 0x7fa7a40105a0 con 0x7fa7bc101f30 2024-02-02T15:24:53.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.483+0000 7fa7b3e00700 1 --2- 172.21.15.1:0/3427659543 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fa79c0a0520 0x7fa79c0a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:53.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.484+0000 7fa7b3e00700 1 -- 172.21.15.1:0/3427659543 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fa7a4014070 con 0x7fa7bc101f30 2024-02-02T15:24:53.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.484+0000 7fa7bb400700 1 --2- 172.21.15.1:0/3427659543 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fa79c0a0520 0x7fa79c0a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:53.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.484+0000 7fa7bb400700 1 --2- 172.21.15.1:0/3427659543 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fa79c0a0520 0x7fa79c0a29e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac009990 tx=0x7fa7ac005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:24:53.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.486+0000 7fa7b3e00700 1 -- 172.21.15.1:0/3427659543 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7fa7a40958b0 con 0x7fa7bc101f30 2024-02-02T15:24:53.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.702+0000 7fa7c2400700 1 -- 172.21.15.1:0/3427659543 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa7bc02ce40 con 0x7fa7bc101f30 2024-02-02T15:24:53.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.704+0000 7fa7b3e00700 1 -- 172.21.15.1:0/3427659543 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+23841 (secure 0 0 0) 0x7fa7a4026090 con 0x7fa7bc101f30 2024-02-02T15:24:53.704 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:24:53.704 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":84,"fsid":"371a409c-c1de-11ee-95b6-87774f69a715","created":"2024-02-02T15:18:26.746506+0000","modified":"2024-02-02T15:24:52.310888+0000","last_up_change":"2024-02-02T15:24:08.800951+0000","last_in_change":"2024-02-02T15:23:52.784280+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T15:21:22.471754+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-02T15:24:46.195779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-02T15:24:47.226403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-02T15:24:50.300462+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"73e8a07f-3560-4c19-85ed-030d4d8f0f23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6803","nonce":2591906423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6805","nonce":2591906423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6809","nonce":2591906423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6807","nonce":2591906423}]},"public_addr":"172.21.15.1:6803/2591906423","cluster_addr":"172.21.15.1:6805/2591906423","heartbeat_back_addr":"172.21.15.1:6809/2591906423","heartbeat_front_addr":"172.21.15.1:6807/2591906423","state":["exists","up"]},{"osd":1,"uuid":"e53cffec-0ff9-4ed9-8dcf-ff47d04db2bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6801","nonce":747581719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6803","nonce":747581719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6807","nonce":747581719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6805","nonce":747581719}]},"public_addr":"172.21.15.71:6801/747581719","cluster_addr":"172.21.15.71:6803/747581719","heartbeat_back_addr":"172.21.15.71:6807/747581719","heartbeat_front_addr":"172.21.15.71:6805/747581719","state":["exists","up"]},{"osd":2,"uuid":"856c2347-294c-482b-8595-ad77f4792db4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6801","nonce":377511414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6803","nonce":377511414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6807","nonce":377511414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6805","nonce":377511414}]},"public_addr":"172.21.15.142:6801/377511414","cluster_addr":"172.21.15.142:6803/377511414","heartbeat_back_addr":"172.21.15.142:6807/377511414","heartbeat_front_addr":"172.21.15.142:6805/377511414","state":["exists","up"]},{"osd":3,"uuid":"4b946ad2-2da9-4e0b-9d61-d039d8669a42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6811","nonce":1086124477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6813","nonce":1086124477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6817","nonce":1086124477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6815","nonce":1086124477}]},"public_addr":"172.21.15.1:6811/1086124477","cluster_addr":"172.21.15.1:6813/1086124477","heartbeat_back_addr":"172.21.15.1:6817/1086124477","heartbeat_front_addr":"172.21.15.1:6815/1086124477","state":["exists","up"]},{"osd":4,"uuid":"43e36c43-47a0-4634-a45b-1da14a0e19d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6809","nonce":3095380305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6811","nonce":3095380305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6815","nonce":3095380305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6813","nonce":3095380305}]},"public_addr":"172.21.15.71:6809/3095380305","cluster_addr":"172.21.15.71:6811/3095380305","heartbeat_back_addr":"172.21.15.71:6815/3095380305","heartbeat_front_addr":"172.21.15.71:6813/3095380305","state":["exists","up"]},{"osd":5,"uuid":"eb48160e-ea7c-42f7-9477-be110bc8a34e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6809","nonce":4195149262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6811","nonce":4195149262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6815","nonce":4195149262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6813","nonce":4195149262}]},"public_addr":"172.21.15.142:6809/4195149262","cluster_addr":"172.21.15.142:6811/4195149262","heartbeat_back_addr":"172.21.15.142:6815/4195149262","heartbeat_front_addr":"172.21.15.142:6813/4195149262","state":["exists","up"]},{"osd":6,"uuid":"0ec4501a-5853-4ca6-b325-231f5b72335e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6819","nonce":1560153428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6821","nonce":1560153428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6825","nonce":1560153428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6823","nonce":1560153428}]},"public_addr":"172.21.15.1:6819/1560153428","cluster_addr":"172.21.15.1:6821/1560153428","heartbeat_back_addr":"172.21.15.1:6825/1560153428","heartbeat_front_addr":"172.21.15.1:6823/1560153428","state":["exists","up"]},{"osd":7,"uuid":"029debe8-9cdf-44f6-9aea-cabd1d185f93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6817","nonce":4080955538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6819","nonce":4080955538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6823","nonce":4080955538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6821","nonce":4080955538}]},"public_addr":"172.21.15.71:6817/4080955538","cluster_addr":"172.21.15.71:6819/4080955538","heartbeat_back_addr":"172.21.15.71:6823/4080955538","heartbeat_front_addr":"172.21.15.71:6821/4080955538","state":["exists","up"]},{"osd":8,"uuid":"7c90896c-9955-4c1b-aaca-7531dd5457f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6817","nonce":3942038209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6819","nonce":3942038209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6823","nonce":3942038209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6821","nonce":3942038209}]},"public_addr":"172.21.15.142:6817/3942038209","cluster_addr":"172.21.15.142:6819/3942038209","heartbeat_back_addr":"172.21.15.142:6823/3942038209","heartbeat_front_addr":"172.21.15.142:6821/3942038209","state":["exists","up"]},{"osd":9,"uuid":"7e165e68-31a8-4598-a577-de3e90ae5c5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6827","nonce":2716544850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6829","nonce":2716544850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6833","nonce":2716544850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6831","nonce":2716544850}]},"public_addr":"172.21.15.1:6827/2716544850","cluster_addr":"172.21.15.1:6829/2716544850","heartbeat_back_addr":"172.21.15.1:6833/2716544850","heartbeat_front_addr":"172.21.15.1:6831/2716544850","state":["exists","up"]},{"osd":10,"uuid":"48cd5fdb-0887-444a-aab8-a1de24282bb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6825","nonce":84691841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6827","nonce":84691841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6831","nonce":84691841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6829","nonce":84691841}]},"public_addr":"172.21.15.71:6825/84691841","cluster_addr":"172.21.15.71:6827/84691841","heartbeat_back_addr":"172.21.15.71:6831/84691841","heartbeat_front_addr":"172.21.15.71:6829/84691841","state":["exists","up"]},{"osd":11,"uuid":"f011dd27-11b0-4166-adfe-76756533163a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6825","nonce":2290179934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6827","nonce":2290179934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6831","nonce":2290179934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6829","nonce":2290179934}]},"public_addr":"172.21.15.142:6825/2290179934","cluster_addr":"172.21.15.142:6827/2290179934","heartbeat_back_addr":"172.21.15.142:6831/2290179934","heartbeat_front_addr":"172.21.15.142:6829/2290179934","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:20:44.060692+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:01.903817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:19.368258+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:40.028037+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:57.513361+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:22:14.805944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:22:35.188439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:22:53.014412+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:23:10.507420+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:23:31.849765+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:23:50.070612+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:24:07.388625+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/4092532060":"2024-02-03T15:19:25.836171+0000","172.21.15.1:6801/395498296":"2024-02-03T15:19:25.836171+0000","172.21.15.1:6800/395498296":"2024-02-03T15:19:25.836171+0000","172.21.15.1:0/1575373489":"2024-02-03T15:19:25.836171+0000","172.21.15.1:0/865983374":"2024-02-03T15:19:01.431596+0000","172.21.15.1:0/3165960227":"2024-02-03T15:19:01.431596+0000","172.21.15.1:0/2350830293":"2024-02-03T15:19:01.431596+0000","172.21.15.1:0/2959690250":"2024-02-03T15:19:25.836171+0000","172.21.15.1:6800/1162530732":"2024-02-03T15:19:01.431596+0000","172.21.15.1:6801/1162530732":"2024-02-03T15:19:01.431596+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T15:24:53.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.707+0000 7fa7c2400700 1 -- 172.21.15.1:0/3427659543 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fa79c0a0520 msgr2=0x7fa79c0a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:53.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.707+0000 7fa7c2400700 1 --2- 172.21.15.1:0/3427659543 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fa79c0a0520 0x7fa79c0a29e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac009990 tx=0x7fa7ac005fb0 comp rx=0 tx=0).stop 2024-02-02T15:24:53.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.707+0000 7fa7c2400700 1 -- 172.21.15.1:0/3427659543 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7bc101f30 msgr2=0x7fa7bc073130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:53.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.707+0000 7fa7c2400700 1 --2- 172.21.15.1:0/3427659543 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7bc101f30 0x7fa7bc073130 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa7a400e9a0 tx=0x7fa7a400ecb0 comp rx=0 tx=0).stop 2024-02-02T15:24:53.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.707+0000 7fa7c2400700 1 -- 172.21.15.1:0/3427659543 shutdown_connections 2024-02-02T15:24:53.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.707+0000 7fa7c2400700 1 --2- 172.21.15.1:0/3427659543 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fa79c0a0520 0x7fa79c0a29e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:53.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.707+0000 7fa7c2400700 1 --2- 172.21.15.1:0/3427659543 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7bc1028d0 0x7fa7bc073670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:53.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.707+0000 7fa7c2400700 1 --2- 172.21.15.1:0/3427659543 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa7bc100d40 0x7fa7bc074af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:53.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.707+0000 7fa7c2400700 1 --2- 172.21.15.1:0/3427659543 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7bc101f30 0x7fa7bc073130 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:53.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.707+0000 7fa7c2400700 1 -- 172.21.15.1:0/3427659543 >> 172.21.15.1:0/3427659543 conn(0x7fa7bc0fc290 msgr2=0x7fa7bc0fe520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:24:53.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.707+0000 7fa7c2400700 1 -- 172.21.15.1:0/3427659543 shutdown_connections 2024-02-02T15:24:53.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.707+0000 7fa7c2400700 1 -- 172.21.15.1:0/3427659543 wait complete. 2024-02-02T15:24:53.741 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T15:24:53.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.939+0000 7f8e77800700 1 -- 172.21.15.1:0/4066430513 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8e70101f30 msgr2=0x7f8e70102390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:53.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.939+0000 7f8e77800700 1 --2- 172.21.15.1:0/4066430513 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8e70101f30 0x7f8e70102390 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8e580099c0 tx=0x7f8e58009cd0 comp rx=0 tx=0).stop 2024-02-02T15:24:53.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.939+0000 7f8e77800700 1 -- 172.21.15.1:0/4066430513 shutdown_connections 2024-02-02T15:24:53.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.939+0000 7f8e77800700 1 --2- 172.21.15.1:0/4066430513 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e701028d0 0x7f8e70109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:53.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.939+0000 7f8e77800700 1 --2- 172.21.15.1:0/4066430513 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e70100d40 0x7f8e70101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:53.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.939+0000 7f8e77800700 1 --2- 172.21.15.1:0/4066430513 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8e70101f30 0x7f8e70102390 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:53.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.939+0000 7f8e77800700 1 -- 172.21.15.1:0/4066430513 >> 172.21.15.1:0/4066430513 conn(0x7f8e700fc290 msgr2=0x7f8e700fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:24:53.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.939+0000 7f8e77800700 1 -- 172.21.15.1:0/4066430513 shutdown_connections 2024-02-02T15:24:53.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.939+0000 7f8e77800700 1 -- 172.21.15.1:0/4066430513 wait complete. 2024-02-02T15:24:53.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.940+0000 7f8e77800700 1 Processor -- start 2024-02-02T15:24:53.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.940+0000 7f8e77800700 1 -- start start 2024-02-02T15:24:53.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.940+0000 7f8e77800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e70100d40 0x7f8e70198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:53.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.940+0000 7f8e77800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e70101f30 0x7f8e701991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:53.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.940+0000 7f8e77800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8e701028d0 0x7f8e701a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:53.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.940+0000 7f8e77800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e701a07c0 con 0x7f8e701028d0 2024-02-02T15:24:53.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.940+0000 7f8e77800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e701a0930 con 0x7f8e70100d40 2024-02-02T15:24:53.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.940+0000 7f8e77800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e701a0af0 con 0x7f8e70101f30 2024-02-02T15:24:53.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.941+0000 7f8e75800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8e701028d0 0x7f8e701a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:53.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.941+0000 7f8e75800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8e701028d0 0x7f8e701a01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33676/0 (socket says 172.21.15.1:33676) 2024-02-02T15:24:53.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.941+0000 7f8e75800700 1 -- 172.21.15.1:0/3585958389 learned_addr learned my addr 172.21.15.1:0/3585958389 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:24:53.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.941+0000 7f8e74e00700 1 --2- 172.21.15.1:0/3585958389 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e70100d40 0x7f8e70198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:53.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.941+0000 7f8e6fe00700 1 --2- 172.21.15.1:0/3585958389 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e70101f30 0x7f8e701991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:53.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.941+0000 7f8e75800700 1 -- 172.21.15.1:0/3585958389 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e70101f30 msgr2=0x7f8e701991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:53.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.941+0000 7f8e75800700 1 --2- 172.21.15.1:0/3585958389 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e70101f30 0x7f8e701991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:53.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.941+0000 7f8e75800700 1 -- 172.21.15.1:0/3585958389 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e70100d40 msgr2=0x7f8e70198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:53.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.941+0000 7f8e75800700 1 --2- 172.21.15.1:0/3585958389 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e70100d40 0x7f8e70198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:53.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.941+0000 7f8e75800700 1 -- 172.21.15.1:0/3585958389 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e580095f0 con 0x7f8e701028d0 2024-02-02T15:24:53.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.941+0000 7f8e75800700 1 --2- 172.21.15.1:0/3585958389 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8e701028d0 0x7f8e701a01f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8e6800ef30 tx=0x7f8e6800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:24:53.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.942+0000 7f8e6d600700 1 -- 172.21.15.1:0/3585958389 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e68018070 con 0x7f8e701028d0 2024-02-02T15:24:53.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.942+0000 7f8e6d600700 1 -- 172.21.15.1:0/3585958389 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8e6800f040 con 0x7f8e701028d0 2024-02-02T15:24:53.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.942+0000 7f8e6d600700 1 -- 172.21.15.1:0/3585958389 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e6800cd00 con 0x7f8e701028d0 2024-02-02T15:24:53.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.942+0000 7f8e77800700 1 -- 172.21.15.1:0/3585958389 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e701a0d70 con 0x7f8e701028d0 2024-02-02T15:24:53.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.942+0000 7f8e77800700 1 -- 172.21.15.1:0/3585958389 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e701a11d0 con 0x7f8e701028d0 2024-02-02T15:24:53.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.943+0000 7f8e77800700 1 -- 172.21.15.1:0/3585958389 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e7010c5a0 con 0x7f8e701028d0 2024-02-02T15:24:53.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.945+0000 7f8e6d600700 1 -- 172.21.15.1:0/3585958389 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) v1 ==== 135069+0+0 (secure 0 0 0) 0x7f8e68003a00 con 0x7f8e701028d0 2024-02-02T15:24:53.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.946+0000 7f8e6d600700 1 --2- 172.21.15.1:0/3585958389 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f8e500a05a0 0x7f8e500a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:53.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.946+0000 7f8e6d600700 1 -- 172.21.15.1:0/3585958389 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f8e680cbfe0 con 0x7f8e701028d0 2024-02-02T15:24:53.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.946+0000 7f8e74e00700 1 --2- 172.21.15.1:0/3585958389 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f8e500a05a0 0x7f8e500a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:53.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.947+0000 7f8e74e00700 1 --2- 172.21.15.1:0/3585958389 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f8e500a05a0 0x7f8e500a2a60 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8e7019a150 tx=0x7f8e60009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:24:53.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:53.949+0000 7f8e6d600700 1 -- 172.21.15.1:0/3585958389 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f8e680959e0 con 0x7f8e701028d0 2024-02-02T15:24:54.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:53 smithi071 ceph-mon[44707]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-02T15:24:54.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:53 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/3781541447' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:24:54.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:53 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/3324049945' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-02T15:24:54.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:53 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/3427659543' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-02T15:24:54.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:53 smithi142 ceph-mon[45728]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-02T15:24:54.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:53 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/3781541447' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:24:54.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:53 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/3324049945' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-02T15:24:54.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:53 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/3427659543' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-02T15:24:54.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:53 smithi001 ceph-mon[35470]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-02T15:24:54.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:53 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/3781541447' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:24:54.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:53 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/3324049945' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-02T15:24:54.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:53 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/3427659543' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-02T15:24:54.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.205+0000 7f8e77800700 1 -- 172.21.15.1:0/3585958389 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8e7010c890 con 0x7f8e701028d0 2024-02-02T15:24:54.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.206+0000 7f8e6d600700 1 -- 172.21.15.1:0/3585958389 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+10755 (secure 0 0 0) 0x7f8e68099000 con 0x7f8e701028d0 2024-02-02T15:24:54.206 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:24:54.206 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2922240448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2922240448},{"type":"v1","addr":"172.21.15.1:6835","nonce":2922240448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2729497431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2729497431},{"type":"v1","addr":"172.21.15.1:6837","nonce":2729497431}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14517,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6839/1218303683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":1218303683},{"type":"v1","addr":"172.21.15.1:6839","nonce":1218303683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14529,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6841/2059197496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2059197496},{"type":"v1","addr":"172.21.15.1:6841","nonce":2059197496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24457,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6833/7648986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":7648986},{"type":"v1","addr":"172.21.15.142:6833","nonce":7648986}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24458,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/3482746094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3482746094},{"type":"v1","addr":"172.21.15.71:6833","nonce":3482746094}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/388207814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":388207814},{"type":"v1","addr":"172.21.15.71:6835","nonce":388207814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24502,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6835/1052606547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1052606547},{"type":"v1","addr":"172.21.15.142:6835","nonce":1052606547}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24514,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6837/2870289019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2870289019},{"type":"v1","addr":"172.21.15.142:6837","nonce":2870289019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24518,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/628016199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":628016199},{"type":"v1","addr":"172.21.15.71:6837","nonce":628016199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24529,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.142:6839/321675421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":321675421},{"type":"v1","addr":"172.21.15.142:6839","nonce":321675421}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-02-02T15:24:47.761435+0000","modified":"2024-02-02T15:24:52.311168+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24557},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/1268086493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1268086493},{"type":"v1","addr":"172.21.15.71:6839","nonce":1268086493}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-02-02T15:24:54.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.209+0000 7f8e77800700 1 -- 172.21.15.1:0/3585958389 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f8e500a05a0 msgr2=0x7f8e500a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:54.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.209+0000 7f8e77800700 1 --2- 172.21.15.1:0/3585958389 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f8e500a05a0 0x7f8e500a2a60 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8e7019a150 tx=0x7f8e60009260 comp rx=0 tx=0).stop 2024-02-02T15:24:54.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.209+0000 7f8e77800700 1 -- 172.21.15.1:0/3585958389 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8e701028d0 msgr2=0x7f8e701a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:54.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.209+0000 7f8e77800700 1 --2- 172.21.15.1:0/3585958389 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8e701028d0 0x7f8e701a01f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8e6800ef30 tx=0x7f8e6800c3a0 comp rx=0 tx=0).stop 2024-02-02T15:24:54.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.209+0000 7f8e77800700 1 -- 172.21.15.1:0/3585958389 shutdown_connections 2024-02-02T15:24:54.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.209+0000 7f8e77800700 1 --2- 172.21.15.1:0/3585958389 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8e701028d0 0x7f8e701a01f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:54.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.209+0000 7f8e77800700 1 --2- 172.21.15.1:0/3585958389 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e70100d40 0x7f8e70198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:54.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.209+0000 7f8e77800700 1 --2- 172.21.15.1:0/3585958389 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f8e500a05a0 0x7f8e500a2a60 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:54.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.209+0000 7f8e77800700 1 --2- 172.21.15.1:0/3585958389 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8e70101f30 0x7f8e701991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:54.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.209+0000 7f8e77800700 1 -- 172.21.15.1:0/3585958389 >> 172.21.15.1:0/3585958389 conn(0x7f8e700fc290 msgr2=0x7f8e700fe530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:24:54.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.209+0000 7f8e77800700 1 -- 172.21.15.1:0/3585958389 shutdown_connections 2024-02-02T15:24:54.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.209+0000 7f8e77800700 1 -- 172.21.15.1:0/3585958389 wait complete. 2024-02-02T15:24:54.210 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 17 2024-02-02T15:24:54.227 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T15:24:54.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.424+0000 7f30aea00700 1 -- 172.21.15.1:0/2709891325 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30a8101f30 msgr2=0x7f30a8102390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:54.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.424+0000 7f30aea00700 1 --2- 172.21.15.1:0/2709891325 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30a8101f30 0x7f30a8102390 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f30900099c0 tx=0x7f3090009cd0 comp rx=0 tx=0).stop 2024-02-02T15:24:54.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.424+0000 7f30aea00700 1 -- 172.21.15.1:0/2709891325 shutdown_connections 2024-02-02T15:24:54.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.424+0000 7f30aea00700 1 --2- 172.21.15.1:0/2709891325 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30a81028d0 0x7f30a8109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:54.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.424+0000 7f30aea00700 1 --2- 172.21.15.1:0/2709891325 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f30a8100d40 0x7f30a8101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:54.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.424+0000 7f30aea00700 1 --2- 172.21.15.1:0/2709891325 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30a8101f30 0x7f30a8102390 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:54.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.424+0000 7f30aea00700 1 -- 172.21.15.1:0/2709891325 >> 172.21.15.1:0/2709891325 conn(0x7f30a80fc290 msgr2=0x7f30a80fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:24:54.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.424+0000 7f30aea00700 1 -- 172.21.15.1:0/2709891325 shutdown_connections 2024-02-02T15:24:54.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.424+0000 7f30aea00700 1 -- 172.21.15.1:0/2709891325 wait complete. 2024-02-02T15:24:54.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.425+0000 7f30aea00700 1 Processor -- start 2024-02-02T15:24:54.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.425+0000 7f30aea00700 1 -- start start 2024-02-02T15:24:54.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.425+0000 7f30aea00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f30a8100d40 0x7f30a8198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:54.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.425+0000 7f30aea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30a8101f30 0x7f30a81991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:54.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.425+0000 7f30aea00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30a81028d0 0x7f30a81a01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:54.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.425+0000 7f30aea00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30a81a07b0 con 0x7f30a8100d40 2024-02-02T15:24:54.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.425+0000 7f30aea00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30a81a0920 con 0x7f30a81028d0 2024-02-02T15:24:54.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.425+0000 7f30aea00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30a81a0ae0 con 0x7f30a8101f30 2024-02-02T15:24:54.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.426+0000 7f30a7e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f30a8100d40 0x7f30a8198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:54.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.426+0000 7f30a7e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f30a8100d40 0x7f30a8198c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33694/0 (socket says 172.21.15.1:33694) 2024-02-02T15:24:54.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.426+0000 7f30a7e00700 1 -- 172.21.15.1:0/2494680206 learned_addr learned my addr 172.21.15.1:0/2494680206 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:24:54.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.426+0000 7f30aca00700 1 --2- 172.21.15.1:0/2494680206 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30a81028d0 0x7f30a81a01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:54.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.426+0000 7f30a7400700 1 --2- 172.21.15.1:0/2494680206 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30a8101f30 0x7f30a81991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:54.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.426+0000 7f30a7e00700 1 -- 172.21.15.1:0/2494680206 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30a8101f30 msgr2=0x7f30a81991b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:54.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.426+0000 7f30a7e00700 1 --2- 172.21.15.1:0/2494680206 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30a8101f30 0x7f30a81991b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:54.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.426+0000 7f30a7e00700 1 -- 172.21.15.1:0/2494680206 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30a81028d0 msgr2=0x7f30a81a01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:54.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.426+0000 7f30a7e00700 1 --2- 172.21.15.1:0/2494680206 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30a81028d0 0x7f30a81a01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:54.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.426+0000 7f30a7e00700 1 -- 172.21.15.1:0/2494680206 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3098009520 con 0x7f30a8100d40 2024-02-02T15:24:54.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.426+0000 7f30aca00700 1 --2- 172.21.15.1:0/2494680206 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30a81028d0 0x7f30a81a01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:24:54.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.426+0000 7f30a7e00700 1 --2- 172.21.15.1:0/2494680206 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f30a8100d40 0x7f30a8198c70 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f309800ea50 tx=0x7f309800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:24:54.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.427+0000 7f30a4c00700 1 -- 172.21.15.1:0/2494680206 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f309800cbd0 con 0x7f30a8100d40 2024-02-02T15:24:54.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.427+0000 7f30aea00700 1 -- 172.21.15.1:0/2494680206 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30900095f0 con 0x7f30a8100d40 2024-02-02T15:24:54.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.427+0000 7f30a4c00700 1 -- 172.21.15.1:0/2494680206 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f309800cd30 con 0x7f30a8100d40 2024-02-02T15:24:54.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.427+0000 7f30a4c00700 1 -- 172.21.15.1:0/2494680206 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3098004a50 con 0x7f30a8100d40 2024-02-02T15:24:54.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.427+0000 7f30aea00700 1 -- 172.21.15.1:0/2494680206 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30a81a1140 con 0x7f30a8100d40 2024-02-02T15:24:54.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.428+0000 7f30aea00700 1 -- 172.21.15.1:0/2494680206 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30a810c5a0 con 0x7f30a8100d40 2024-02-02T15:24:54.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.430+0000 7f30a4c00700 1 -- 172.21.15.1:0/2494680206 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) v1 ==== 135069+0+0 (secure 0 0 0) 0x7f3098004bb0 con 0x7f30a8100d40 2024-02-02T15:24:54.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.431+0000 7f30a4c00700 1 --2- 172.21.15.1:0/2494680206 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f30880a0520 0x7f30880a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:54.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.431+0000 7f30a4c00700 1 -- 172.21.15.1:0/2494680206 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f3098014070 con 0x7f30a8100d40 2024-02-02T15:24:54.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.431+0000 7f30a7400700 1 --2- 172.21.15.1:0/2494680206 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f30880a0520 0x7f30880a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:54.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.432+0000 7f30a7400700 1 --2- 172.21.15.1:0/2494680206 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f30880a0520 0x7f30880a29e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f309000b580 tx=0x7f3090005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:24:54.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.435+0000 7f30a4c00700 1 -- 172.21.15.1:0/2494680206 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f3098095a90 con 0x7f30a8100d40 2024-02-02T15:24:54.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.650+0000 7f30aea00700 1 -- 172.21.15.1:0/2494680206 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f30a81a16f0 con 0x7f30a8100d40 2024-02-02T15:24:54.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.652+0000 7f30a4c00700 1 -- 172.21.15.1:0/2494680206 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+23841 (secure 0 0 0) 0x7f3098004e60 con 0x7f30a8100d40 2024-02-02T15:24:54.652 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:24:54.653 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":84,"fsid":"371a409c-c1de-11ee-95b6-87774f69a715","created":"2024-02-02T15:18:26.746506+0000","modified":"2024-02-02T15:24:52.310888+0000","last_up_change":"2024-02-02T15:24:08.800951+0000","last_in_change":"2024-02-02T15:23:52.784280+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T15:21:22.471754+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-02T15:24:46.195779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-02T15:24:47.226403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-02T15:24:50.300462+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"73e8a07f-3560-4c19-85ed-030d4d8f0f23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6803","nonce":2591906423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6805","nonce":2591906423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6809","nonce":2591906423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6807","nonce":2591906423}]},"public_addr":"172.21.15.1:6803/2591906423","cluster_addr":"172.21.15.1:6805/2591906423","heartbeat_back_addr":"172.21.15.1:6809/2591906423","heartbeat_front_addr":"172.21.15.1:6807/2591906423","state":["exists","up"]},{"osd":1,"uuid":"e53cffec-0ff9-4ed9-8dcf-ff47d04db2bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6801","nonce":747581719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6803","nonce":747581719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6807","nonce":747581719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6805","nonce":747581719}]},"public_addr":"172.21.15.71:6801/747581719","cluster_addr":"172.21.15.71:6803/747581719","heartbeat_back_addr":"172.21.15.71:6807/747581719","heartbeat_front_addr":"172.21.15.71:6805/747581719","state":["exists","up"]},{"osd":2,"uuid":"856c2347-294c-482b-8595-ad77f4792db4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6801","nonce":377511414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6803","nonce":377511414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6807","nonce":377511414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6805","nonce":377511414}]},"public_addr":"172.21.15.142:6801/377511414","cluster_addr":"172.21.15.142:6803/377511414","heartbeat_back_addr":"172.21.15.142:6807/377511414","heartbeat_front_addr":"172.21.15.142:6805/377511414","state":["exists","up"]},{"osd":3,"uuid":"4b946ad2-2da9-4e0b-9d61-d039d8669a42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6811","nonce":1086124477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6813","nonce":1086124477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6817","nonce":1086124477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6815","nonce":1086124477}]},"public_addr":"172.21.15.1:6811/1086124477","cluster_addr":"172.21.15.1:6813/1086124477","heartbeat_back_addr":"172.21.15.1:6817/1086124477","heartbeat_front_addr":"172.21.15.1:6815/1086124477","state":["exists","up"]},{"osd":4,"uuid":"43e36c43-47a0-4634-a45b-1da14a0e19d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6809","nonce":3095380305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6811","nonce":3095380305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6815","nonce":3095380305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6813","nonce":3095380305}]},"public_addr":"172.21.15.71:6809/3095380305","cluster_addr":"172.21.15.71:6811/3095380305","heartbeat_back_addr":"172.21.15.71:6815/3095380305","heartbeat_front_addr":"172.21.15.71:6813/3095380305","state":["exists","up"]},{"osd":5,"uuid":"eb48160e-ea7c-42f7-9477-be110bc8a34e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6809","nonce":4195149262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6811","nonce":4195149262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6815","nonce":4195149262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6813","nonce":4195149262}]},"public_addr":"172.21.15.142:6809/4195149262","cluster_addr":"172.21.15.142:6811/4195149262","heartbeat_back_addr":"172.21.15.142:6815/4195149262","heartbeat_front_addr":"172.21.15.142:6813/4195149262","state":["exists","up"]},{"osd":6,"uuid":"0ec4501a-5853-4ca6-b325-231f5b72335e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6819","nonce":1560153428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6821","nonce":1560153428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6825","nonce":1560153428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6823","nonce":1560153428}]},"public_addr":"172.21.15.1:6819/1560153428","cluster_addr":"172.21.15.1:6821/1560153428","heartbeat_back_addr":"172.21.15.1:6825/1560153428","heartbeat_front_addr":"172.21.15.1:6823/1560153428","state":["exists","up"]},{"osd":7,"uuid":"029debe8-9cdf-44f6-9aea-cabd1d185f93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6817","nonce":4080955538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6819","nonce":4080955538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6823","nonce":4080955538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6821","nonce":4080955538}]},"public_addr":"172.21.15.71:6817/4080955538","cluster_addr":"172.21.15.71:6819/4080955538","heartbeat_back_addr":"172.21.15.71:6823/4080955538","heartbeat_front_addr":"172.21.15.71:6821/4080955538","state":["exists","up"]},{"osd":8,"uuid":"7c90896c-9955-4c1b-aaca-7531dd5457f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6817","nonce":3942038209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6819","nonce":3942038209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6823","nonce":3942038209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6821","nonce":3942038209}]},"public_addr":"172.21.15.142:6817/3942038209","cluster_addr":"172.21.15.142:6819/3942038209","heartbeat_back_addr":"172.21.15.142:6823/3942038209","heartbeat_front_addr":"172.21.15.142:6821/3942038209","state":["exists","up"]},{"osd":9,"uuid":"7e165e68-31a8-4598-a577-de3e90ae5c5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6827","nonce":2716544850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6829","nonce":2716544850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6833","nonce":2716544850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6831","nonce":2716544850}]},"public_addr":"172.21.15.1:6827/2716544850","cluster_addr":"172.21.15.1:6829/2716544850","heartbeat_back_addr":"172.21.15.1:6833/2716544850","heartbeat_front_addr":"172.21.15.1:6831/2716544850","state":["exists","up"]},{"osd":10,"uuid":"48cd5fdb-0887-444a-aab8-a1de24282bb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6825","nonce":84691841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6827","nonce":84691841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6831","nonce":84691841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6829","nonce":84691841}]},"public_addr":"172.21.15.71:6825/84691841","cluster_addr":"172.21.15.71:6827/84691841","heartbeat_back_addr":"172.21.15.71:6831/84691841","heartbeat_front_addr":"172.21.15.71:6829/84691841","state":["exists","up"]},{"osd":11,"uuid":"f011dd27-11b0-4166-adfe-76756533163a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6825","nonce":2290179934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6827","nonce":2290179934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6831","nonce":2290179934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6829","nonce":2290179934}]},"public_addr":"172.21.15.142:6825/2290179934","cluster_addr":"172.21.15.142:6827/2290179934","heartbeat_back_addr":"172.21.15.142:6831/2290179934","heartbeat_front_addr":"172.21.15.142:6829/2290179934","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:20:44.060692+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:01.903817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:19.368258+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:40.028037+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:57.513361+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:22:14.805944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:22:35.188439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:22:53.014412+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:23:10.507420+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:23:31.849765+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:23:50.070612+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:24:07.388625+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/4092532060":"2024-02-03T15:19:25.836171+0000","172.21.15.1:6801/395498296":"2024-02-03T15:19:25.836171+0000","172.21.15.1:6800/395498296":"2024-02-03T15:19:25.836171+0000","172.21.15.1:0/1575373489":"2024-02-03T15:19:25.836171+0000","172.21.15.1:0/865983374":"2024-02-03T15:19:01.431596+0000","172.21.15.1:0/3165960227":"2024-02-03T15:19:01.431596+0000","172.21.15.1:0/2350830293":"2024-02-03T15:19:01.431596+0000","172.21.15.1:0/2959690250":"2024-02-03T15:19:25.836171+0000","172.21.15.1:6800/1162530732":"2024-02-03T15:19:01.431596+0000","172.21.15.1:6801/1162530732":"2024-02-03T15:19:01.431596+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T15:24:54.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.654+0000 7f30aea00700 1 -- 172.21.15.1:0/2494680206 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f30880a0520 msgr2=0x7f30880a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:54.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.654+0000 7f30aea00700 1 --2- 172.21.15.1:0/2494680206 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f30880a0520 0x7f30880a29e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f309000b580 tx=0x7f3090005fb0 comp rx=0 tx=0).stop 2024-02-02T15:24:54.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.654+0000 7f30aea00700 1 -- 172.21.15.1:0/2494680206 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f30a8100d40 msgr2=0x7f30a8198c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:54.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.654+0000 7f30aea00700 1 --2- 172.21.15.1:0/2494680206 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f30a8100d40 0x7f30a8198c70 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f309800ea50 tx=0x7f309800ed60 comp rx=0 tx=0).stop 2024-02-02T15:24:54.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.654+0000 7f30aea00700 1 -- 172.21.15.1:0/2494680206 shutdown_connections 2024-02-02T15:24:54.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.654+0000 7f30aea00700 1 --2- 172.21.15.1:0/2494680206 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30a81028d0 0x7f30a81a01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:54.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.654+0000 7f30aea00700 1 --2- 172.21.15.1:0/2494680206 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f30a8100d40 0x7f30a8198c70 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:54.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.654+0000 7f30aea00700 1 --2- 172.21.15.1:0/2494680206 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f30880a0520 0x7f30880a29e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:54.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.654+0000 7f30aea00700 1 --2- 172.21.15.1:0/2494680206 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30a8101f30 0x7f30a81991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:54.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.654+0000 7f30aea00700 1 -- 172.21.15.1:0/2494680206 >> 172.21.15.1:0/2494680206 conn(0x7f30a80fc290 msgr2=0x7f30a80fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:24:54.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.655+0000 7f30aea00700 1 -- 172.21.15.1:0/2494680206 shutdown_connections 2024-02-02T15:24:54.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.655+0000 7f30aea00700 1 -- 172.21.15.1:0/2494680206 wait complete. 2024-02-02T15:24:54.673 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-02-02T15:24:47.761435+0000', 'modified': '2024-02-02T15:24:52.311168+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24557}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24557': {'gid': 24557, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.71:6839/1268086493', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6838', 'nonce': 1268086493}, {'type': 'v1', 'addr': '172.21.15.71:6839', 'nonce': 1268086493}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2024-02-02T15:24:54.673 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-02-02T15:24:54.673 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-02-02T15:24:54.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.873+0000 7f8feac00700 1 -- 172.21.15.1:0/1833028640 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8fe4102990 msgr2=0x7f8fe4109240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:54.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.873+0000 7f8feac00700 1 --2- 172.21.15.1:0/1833028640 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8fe4102990 0x7f8fe4109240 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc00b0a0 tx=0x7f8fdc00b3b0 comp rx=0 tx=0).stop 2024-02-02T15:24:54.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.873+0000 7f8feac00700 1 -- 172.21.15.1:0/1833028640 shutdown_connections 2024-02-02T15:24:54.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.873+0000 7f8feac00700 1 --2- 172.21.15.1:0/1833028640 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8fe4102990 0x7f8fe4109240 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:54.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.873+0000 7f8feac00700 1 --2- 172.21.15.1:0/1833028640 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8fe4100e00 0x7f8fe4101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:54.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.873+0000 7f8feac00700 1 --2- 172.21.15.1:0/1833028640 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8fe4101ff0 0x7f8fe4102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:54.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.873+0000 7f8feac00700 1 -- 172.21.15.1:0/1833028640 >> 172.21.15.1:0/1833028640 conn(0x7f8fe40fc370 msgr2=0x7f8fe40fe7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:24:54.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.873+0000 7f8feac00700 1 -- 172.21.15.1:0/1833028640 shutdown_connections 2024-02-02T15:24:54.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.873+0000 7f8feac00700 1 -- 172.21.15.1:0/1833028640 wait complete. 2024-02-02T15:24:54.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.874+0000 7f8feac00700 1 Processor -- start 2024-02-02T15:24:54.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.874+0000 7f8feac00700 1 -- start start 2024-02-02T15:24:54.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.874+0000 7f8feac00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8fe4100e00 0x7f8fe4198e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:54.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.874+0000 7f8feac00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8fe4101ff0 0x7f8fe4199340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:54.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.875+0000 7f8feac00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8fe4102990 0x7f8fe41a03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:54.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.875+0000 7f8feac00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe41a0980 con 0x7f8fe4100e00 2024-02-02T15:24:54.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.875+0000 7f8feac00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe41a0ac0 con 0x7f8fe4102990 2024-02-02T15:24:54.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.875+0000 7f8feac00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe41a0c50 con 0x7f8fe4101ff0 2024-02-02T15:24:54.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.875+0000 7f8fe3e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8fe4100e00 0x7f8fe4198e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:54.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.875+0000 7f8fe3e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8fe4100e00 0x7f8fe4198e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33714/0 (socket says 172.21.15.1:33714) 2024-02-02T15:24:54.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.875+0000 7f8fe3e00700 1 -- 172.21.15.1:0/734218145 learned_addr learned my addr 172.21.15.1:0/734218145 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:24:54.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.875+0000 7f8fe8c00700 1 --2- 172.21.15.1:0/734218145 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8fe4102990 0x7f8fe41a03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:54.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.875+0000 7f8fe3400700 1 --2- 172.21.15.1:0/734218145 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8fe4101ff0 0x7f8fe4199340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:54.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.875+0000 7f8fe3e00700 1 -- 172.21.15.1:0/734218145 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8fe4101ff0 msgr2=0x7f8fe4199340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:54.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.875+0000 7f8fe3e00700 1 --2- 172.21.15.1:0/734218145 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8fe4101ff0 0x7f8fe4199340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:54.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.875+0000 7f8fe3e00700 1 -- 172.21.15.1:0/734218145 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8fe4102990 msgr2=0x7f8fe41a03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:54.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.875+0000 7f8fe3e00700 1 --2- 172.21.15.1:0/734218145 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8fe4102990 0x7f8fe41a03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:54.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.875+0000 7f8fe3e00700 1 -- 172.21.15.1:0/734218145 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fdc009d00 con 0x7f8fe4100e00 2024-02-02T15:24:54.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.875+0000 7f8fe8c00700 1 --2- 172.21.15.1:0/734218145 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8fe4102990 0x7f8fe41a03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:24:54.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.875+0000 7f8fe3e00700 1 --2- 172.21.15.1:0/734218145 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8fe4100e00 0x7f8fe4198e00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8fd400e970 tx=0x7f8fd400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:24:54.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.876+0000 7f8fe0c00700 1 -- 172.21.15.1:0/734218145 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd4009790 con 0x7f8fe4100e00 2024-02-02T15:24:54.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.876+0000 7f8fe0c00700 1 -- 172.21.15.1:0/734218145 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8fd400c7e0 con 0x7f8fe4100e00 2024-02-02T15:24:54.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.876+0000 7f8fe0c00700 1 -- 172.21.15.1:0/734218145 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd4003c70 con 0x7f8fe4100e00 2024-02-02T15:24:54.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.876+0000 7f8feac00700 1 -- 172.21.15.1:0/734218145 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fe41a0f30 con 0x7f8fe4100e00 2024-02-02T15:24:54.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.876+0000 7f8feac00700 1 -- 172.21.15.1:0/734218145 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fe41a12a0 con 0x7f8fe4100e00 2024-02-02T15:24:54.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.879+0000 7f8fe0c00700 1 -- 172.21.15.1:0/734218145 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) v1 ==== 135069+0+0 (secure 0 0 0) 0x7f8fd400cc50 con 0x7f8fe4100e00 2024-02-02T15:24:54.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.881+0000 7f8ffe641b80 1 Processor -- start 2024-02-02T15:24:54.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.881+0000 7f8fe0c00700 1 --2- 172.21.15.1:0/734218145 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f8fc40a0520 0x7f8fc40a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:54.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.881+0000 7f8ffe641b80 1 -- start start 2024-02-02T15:24:54.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.882+0000 7f8fe0c00700 1 -- 172.21.15.1:0/734218145 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f8fd4014070 con 0x7f8fe4100e00 2024-02-02T15:24:54.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.882+0000 7f8fe8c00700 1 --2- 172.21.15.1:0/734218145 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f8fc40a0520 0x7f8fc40a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:54.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.882+0000 7f8ffe641b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621cebdf130 0x5621cebdf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:54.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.882+0000 7f8ffe641b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5621cebdfa90 0x5621cebe03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:54.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.882+0000 7f8ffe641b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621cebe0930 0x5621cebe0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:54.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.882+0000 7f8ffe641b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621ceb52470 con 0x5621cebe0930 2024-02-02T15:24:54.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.882+0000 7f8ffe641b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621ceb36a50 con 0x5621cebdfa90 2024-02-02T15:24:54.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.882+0000 7f8ffe641b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621ceb34410 con 0x5621cebdf130 2024-02-02T15:24:54.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.882+0000 7f8fe3e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621cebdf130 0x5621cebdf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:54.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.882+0000 7f8fe3e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621cebdf130 0x5621cebdf550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:60348/0 (socket says 172.21.15.1:60348) 2024-02-02T15:24:54.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.882+0000 7f8fe3400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5621cebdfa90 0x5621cebe03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:54.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.882+0000 7f8fe3e00700 1 -- 172.21.15.1:0/2795815855 learned_addr learned my addr 172.21.15.1:0/2795815855 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:24:54.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.882+0000 7f8fe8c00700 1 --2- 172.21.15.1:0/2795815855 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621cebe0930 0x5621cebe0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:54.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.882+0000 7f8fe3e00700 1 -- 172.21.15.1:0/2795815855 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5621cebdfa90 msgr2=0x5621cebe03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:24:54.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.882+0000 7f8fe3e00700 1 --2- 172.21.15.1:0/2795815855 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5621cebdfa90 0x5621cebe03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:54.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.883+0000 7f8fe3e00700 1 -- 172.21.15.1:0/2795815855 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621cebe0930 msgr2=0x5621cebe0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:24:54.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.883+0000 7f8fe3e00700 1 --2- 172.21.15.1:0/2795815855 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621cebe0930 0x5621cebe0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:54.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.883+0000 7f8fe3e00700 1 -- 172.21.15.1:0/2795815855 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5621ce9b1360 con 0x5621cebdf130 2024-02-02T15:24:54.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.883+0000 7f8fe3400700 1 --2- 172.21.15.1:0/2795815855 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5621cebdfa90 0x5621cebe03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:24:54.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.883+0000 7f8fe8c00700 1 --2- 172.21.15.1:0/734218145 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f8fc40a0520 0x7f8fc40a29e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc00b780 tx=0x7f8fdc0095a0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:24:54.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.883+0000 7f8fe3e00700 1 --2- 172.21.15.1:0/2795815855 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621cebdf130 0x5621cebdf550 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8fd4099340 tx=0x7f8fd4098d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8da156d500147564 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:24:54.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.883+0000 7f8fd9200700 1 -- 172.21.15.1:0/2795815855 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd40993b0 con 0x5621cebdf130 2024-02-02T15:24:54.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.883+0000 7f8fd9200700 1 -- 172.21.15.1:0/2795815855 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8fd409d1e0 con 0x5621cebdf130 2024-02-02T15:24:54.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.883+0000 7f8fd9200700 1 -- 172.21.15.1:0/2795815855 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd4099610 con 0x5621cebdf130 2024-02-02T15:24:54.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.884+0000 7f8ffe641b80 1 -- 172.21.15.1:0/2795815855 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621cebdf130 msgr2=0x5621cebdf550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:54.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.884+0000 7f8ffe641b80 1 --2- 172.21.15.1:0/2795815855 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621cebdf130 0x5621cebdf550 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8fd4099340 tx=0x7f8fd4098d90 comp rx=0 tx=0).stop 2024-02-02T15:24:54.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.884+0000 7f8ffe641b80 1 -- 172.21.15.1:0/2795815855 shutdown_connections 2024-02-02T15:24:54.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.884+0000 7f8ffe641b80 1 --2- 172.21.15.1:0/2795815855 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621cebe0930 0x5621cebe0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:54.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.884+0000 7f8ffe641b80 1 --2- 172.21.15.1:0/2795815855 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621cebdf130 0x5621cebdf550 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:54.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.884+0000 7f8ffe641b80 1 --2- 172.21.15.1:0/2795815855 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5621cebdfa90 0x5621cebe03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:54.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.884+0000 7f8ffe641b80 1 -- 172.21.15.1:0/2795815855 >> 172.21.15.1:0/2795815855 conn(0x5621ceb3dd60 msgr2=0x5621cebd9e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:24:54.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.884+0000 7f8ffe641b80 1 -- 172.21.15.1:0/2795815855 shutdown_connections 2024-02-02T15:24:54.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.884+0000 7f8ffe641b80 1 -- 172.21.15.1:0/2795815855 wait complete. 2024-02-02T15:24:54.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.884+0000 7f8ffe641b80 1 -- start start 2024-02-02T15:24:54.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.885+0000 7f8ffe641b80 1 Processor -- start 2024-02-02T15:24:54.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.885+0000 7f8ffe641b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5621cebdf130 0x5621cebe8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:54.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.885+0000 7f8ffe641b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621cebdfa90 0x5621cebe9430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:54.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.885+0000 7f8ffe641b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621cebe0930 0x5621cebf0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:54.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.885+0000 7f8ffe641b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621ceb2fd00 con 0x5621cebe0930 2024-02-02T15:24:54.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.885+0000 7f8ffe641b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621ceacbd30 con 0x5621cebdf130 2024-02-02T15:24:54.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.885+0000 7f8ffe641b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621ceacb9d0 con 0x5621cebdfa90 2024-02-02T15:24:54.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.885+0000 7f8ffe641b80 10 client.0 _ll_register_callbacks cb 0x5621ceacd240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:24:54.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.885+0000 7f8fe3400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621cebdfa90 0x5621cebe9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:54.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.885+0000 7f8fe3400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621cebdfa90 0x5621cebe9430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:60356/0 (socket says 172.21.15.1:60356) 2024-02-02T15:24:54.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.885+0000 7f8fe3400700 1 -- 172.21.15.1:0/837482799 learned_addr learned my addr 172.21.15.1:0/837482799 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:24:54.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.885+0000 7f8fe8c00700 1 --2- 172.21.15.1:0/837482799 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621cebe0930 0x5621cebf0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:54.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.885+0000 7f8fe3e00700 1 --2- 172.21.15.1:0/837482799 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5621cebdf130 0x5621cebe8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:54.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.886+0000 7f8fe3400700 1 -- 172.21.15.1:0/837482799 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5621cebdf130 msgr2=0x5621cebe8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:24:54.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.886+0000 7f8fe3400700 1 --2- 172.21.15.1:0/837482799 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5621cebdf130 0x5621cebe8ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:54.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.886+0000 7f8fe3400700 1 -- 172.21.15.1:0/837482799 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621cebe0930 msgr2=0x5621cebf0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:24:54.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.886+0000 7f8fe3400700 1 --2- 172.21.15.1:0/837482799 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621cebe0930 0x5621cebf0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:54.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.886+0000 7f8fe3400700 1 -- 172.21.15.1:0/837482799 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fcc0098e0 con 0x5621cebdfa90 2024-02-02T15:24:54.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.886+0000 7f8fe3400700 1 --2- 172.21.15.1:0/837482799 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621cebdfa90 0x5621cebe9430 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc012980 tx=0x7f8fcc012c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5577e6970fa94aea server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:24:54.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.886+0000 7f8fd1600700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:24:54.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.886+0000 7f8fd1600700 1 -- 172.21.15.1:0/837482799 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fcc017070 con 0x5621cebdfa90 2024-02-02T15:24:54.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.886+0000 7f8ffe641b80 1 -- 172.21.15.1:0/837482799 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5621cebd20a0 con 0x5621cebdfa90 2024-02-02T15:24:54.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.886+0000 7f8fd1600700 1 -- 172.21.15.1:0/837482799 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f8fcc004030 con 0x5621cebdfa90 2024-02-02T15:24:54.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.887+0000 7f8fd1600700 1 -- 172.21.15.1:0/837482799 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8fcc0041f0 con 0x5621cebdfa90 2024-02-02T15:24:54.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.887+0000 7f8ffe641b80 10 client.24670 fetch_fsmap learned FSMap version 17 2024-02-02T15:24:54.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.887+0000 7f8ffe641b80 1 -- 172.21.15.1:0/837482799 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x7f8fe41a12a0 con 0x5621cebdfa90 2024-02-02T15:24:54.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.887+0000 7f8fd1600700 1 -- 172.21.15.1:0/837482799 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5621cebd20a0 con 0x5621cebdfa90 2024-02-02T15:24:54.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.887+0000 7f8fd1600700 1 -- 172.21.15.1:0/837482799 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 17) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f8fcc00f670 con 0x5621cebdfa90 2024-02-02T15:24:54.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.887+0000 7f8ffe641b80 10 client.24670 fetch_fsmap finished waiting for FSMap version 17 2024-02-02T15:24:54.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.887+0000 7f8ffe641b80 10 client.24670 resolve_mds: resolved name 'b' to daemon mds.b 2024-02-02T15:24:54.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.887+0000 7f8ffe641b80 20 client.24670 populate_metadata read hostname 'smithi001' 2024-02-02T15:24:54.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.887+0000 7f8ffe641b80 1 --2- 172.21.15.1:0/837482799 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5621cebf47e0 0x5621cebf35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:54.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.887+0000 7f8ffe641b80 4 client.24670 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:24:54.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.887+0000 7f8ffe641b80 1 -- 172.21.15.1:0/837482799 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5621ceb28d80 con 0x5621cebf47e0 2024-02-02T15:24:54.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.888+0000 7f8fe3e00700 1 --2- 172.21.15.1:0/837482799 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5621cebf47e0 0x5621cebf35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:54.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.888+0000 7f8fd1600700 10 client.24670 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:24:54.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.888+0000 7f8fe3e00700 1 --2- 172.21.15.1:0/837482799 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5621cebf47e0 0x5621cebf35d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e01616c0940acd23 server_cookie=3cdec21895b7a7f7 in_seq=0 out_seq=0 2024-02-02T15:24:54.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.889+0000 7f8fd1600700 1 -- 172.21.15.1:0/837482799 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5621ceb28d80 con 0x5621cebf47e0 2024-02-02T15:24:54.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.889+0000 7f8fd1600700 10 client.24670 handle_command_reply: tid=0 2024-02-02T15:24:54.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.890+0000 7f8ffe641b80 1 client.24670 shutdown 2024-02-02T15:24:54.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.890+0000 7f8ffe641b80 1 -- 172.21.15.1:0/837482799 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621cebdfa90 msgr2=0x5621cebe9430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:54.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.890+0000 7f8ffe641b80 1 --2- 172.21.15.1:0/837482799 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621cebdfa90 0x5621cebe9430 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc012980 tx=0x7f8fcc012c90 comp rx=0 tx=0).stop 2024-02-02T15:24:54.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.890+0000 7f8ffe641b80 1 -- 172.21.15.1:0/837482799 shutdown_connections 2024-02-02T15:24:54.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.890+0000 7f8ffe641b80 1 --2- 172.21.15.1:0/837482799 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5621cebf47e0 0x5621cebf35d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:54.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.890+0000 7f8ffe641b80 1 --2- 172.21.15.1:0/837482799 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621cebe0930 0x5621cebf0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:54.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.890+0000 7f8ffe641b80 1 --2- 172.21.15.1:0/837482799 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5621cebdf130 0x5621cebe8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:54.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.890+0000 7f8fd1600700 0 client.24670 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:24:54.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.890+0000 7f8ffe641b80 1 --2- 172.21.15.1:0/837482799 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621cebdfa90 0x5621cebe9430 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:54.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.890+0000 7f8ffe641b80 1 -- 172.21.15.1:0/837482799 >> 172.21.15.1:0/837482799 conn(0x5621ceb3dd60 msgr2=0x5621cebd9570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:24:54.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.890+0000 7f8ffe641b80 1 -- 172.21.15.1:0/837482799 shutdown_connections 2024-02-02T15:24:54.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.890+0000 7f8ffe641b80 1 -- 172.21.15.1:0/837482799 wait complete. 2024-02-02T15:24:54.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.890+0000 7f8ffe641b80 20 client.24670 trim_cache size 0 max 16384 2024-02-02T15:24:54.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.908+0000 7f8ffe641b80 1 Processor -- start 2024-02-02T15:24:54.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.908+0000 7f8ffe641b80 1 -- start start 2024-02-02T15:24:54.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.908+0000 7f8ffe641b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621ceb3dd60 0x5621cebe6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:54.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.908+0000 7f8ffe641b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621cebdf130 0x5621cebfcf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:54.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.908+0000 7f8ffe641b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5621cebdfa90 0x5621cebf35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:54.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.908+0000 7f8ffe641b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621ce9b3650 con 0x5621cebdf130 2024-02-02T15:24:54.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.908+0000 7f8ffe641b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621ceb28d80 con 0x5621cebdfa90 2024-02-02T15:24:54.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.908+0000 7f8ffe641b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621cebdb070 con 0x5621ceb3dd60 2024-02-02T15:24:54.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.909+0000 7f8fe3400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621cebdf130 0x5621cebfcf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:54.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.909+0000 7f8fe3400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621cebdf130 0x5621cebfcf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33756/0 (socket says 172.21.15.1:33756) 2024-02-02T15:24:54.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.909+0000 7f8fe3400700 1 -- 172.21.15.1:0/3516636158 learned_addr learned my addr 172.21.15.1:0/3516636158 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:24:54.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.909+0000 7f8fe3e00700 1 --2- 172.21.15.1:0/3516636158 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621ceb3dd60 0x5621cebe6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:54.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.909+0000 7f8fe8c00700 1 --2- 172.21.15.1:0/3516636158 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5621cebdfa90 0x5621cebf35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:54.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.909+0000 7f8fe3400700 1 -- 172.21.15.1:0/3516636158 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621ceb3dd60 msgr2=0x5621cebe6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:24:54.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.909+0000 7f8fe3400700 1 --2- 172.21.15.1:0/3516636158 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621ceb3dd60 0x5621cebe6cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:54.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.909+0000 7f8fe3400700 1 -- 172.21.15.1:0/3516636158 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5621cebdfa90 msgr2=0x5621cebf35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:24:54.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.909+0000 7f8fe3400700 1 --2- 172.21.15.1:0/3516636158 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5621cebdfa90 0x5621cebf35d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:54.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.909+0000 7f8fe3400700 1 -- 172.21.15.1:0/3516636158 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5621ce95db10 con 0x5621cebdf130 2024-02-02T15:24:54.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.909+0000 7f8fe8c00700 1 --2- 172.21.15.1:0/3516636158 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5621cebdfa90 0x5621cebf35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:24:54.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.909+0000 7f8fe3400700 1 --2- 172.21.15.1:0/3516636158 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621cebdf130 0x5621cebfcf40 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc012950 tx=0x7f8fcc005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd91bce0dfca8bae server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:24:54.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.909+0000 7f8fd9c00700 1 -- 172.21.15.1:0/3516636158 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fcc017040 con 0x5621cebdf130 2024-02-02T15:24:54.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.909+0000 7f8fd9c00700 1 -- 172.21.15.1:0/3516636158 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8fcc0055d0 con 0x5621cebdf130 2024-02-02T15:24:54.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.909+0000 7f8fd9c00700 1 -- 172.21.15.1:0/3516636158 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fcc003b90 con 0x5621cebdf130 2024-02-02T15:24:54.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.910+0000 7f8ffe641b80 1 -- 172.21.15.1:0/3516636158 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621cebdf130 msgr2=0x5621cebfcf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:54.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.910+0000 7f8ffe641b80 1 --2- 172.21.15.1:0/3516636158 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621cebdf130 0x5621cebfcf40 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8fcc012950 tx=0x7f8fcc005dc0 comp rx=0 tx=0).stop 2024-02-02T15:24:54.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.910+0000 7f8ffe641b80 1 -- 172.21.15.1:0/3516636158 shutdown_connections 2024-02-02T15:24:54.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.910+0000 7f8ffe641b80 1 --2- 172.21.15.1:0/3516636158 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5621cebdfa90 0x5621cebf35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:54.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.910+0000 7f8ffe641b80 1 --2- 172.21.15.1:0/3516636158 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621ceb3dd60 0x5621cebe6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:54.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.910+0000 7f8ffe641b80 1 --2- 172.21.15.1:0/3516636158 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621cebdf130 0x5621cebfcf40 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:54.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.910+0000 7f8ffe641b80 1 -- 172.21.15.1:0/3516636158 >> 172.21.15.1:0/3516636158 conn(0x5621cebda1c0 msgr2=0x5621cebf0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:24:54.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.910+0000 7f8ffe641b80 1 -- 172.21.15.1:0/3516636158 shutdown_connections 2024-02-02T15:24:54.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.910+0000 7f8ffe641b80 1 -- 172.21.15.1:0/3516636158 wait complete. 2024-02-02T15:24:54.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.911+0000 7f8ffe641b80 1 -- start start 2024-02-02T15:24:54.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.911+0000 7f8ffe641b80 1 Processor -- start 2024-02-02T15:24:54.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.911+0000 7f8ffe641b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621ceb3dd60 0x5621cebe5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:54.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.911+0000 7f8ffe641b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621cebdf130 0x5621cebfcf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:54.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.911+0000 7f8ffe641b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5621cebdfa90 0x5621cec45cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:54.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.912+0000 7f8ffe641b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621cec46240 con 0x5621ceb3dd60 2024-02-02T15:24:54.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.912+0000 7f8ffe641b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621cec463b0 con 0x5621cebdfa90 2024-02-02T15:24:54.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.912+0000 7f8ffe641b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621cec46570 con 0x5621cebdf130 2024-02-02T15:24:54.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.912+0000 7f8ffe641b80 10 client.0 _ll_register_callbacks cb 0x5621ceacd240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:24:54.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.912+0000 7f8fe3e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621ceb3dd60 0x5621cebe5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:54.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.912+0000 7f8fe3e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621ceb3dd60 0x5621cebe5fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33762/0 (socket says 172.21.15.1:33762) 2024-02-02T15:24:54.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.912+0000 7f8fe3e00700 1 -- 172.21.15.1:0/1973658852 learned_addr learned my addr 172.21.15.1:0/1973658852 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:24:54.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.912+0000 7f8fe8c00700 1 --2- 172.21.15.1:0/1973658852 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5621cebdfa90 0x5621cec45cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:54.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.912+0000 7f8fe3400700 1 --2- 172.21.15.1:0/1973658852 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621cebdf130 0x5621cebfcf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:54.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.912+0000 7f8fe3e00700 1 -- 172.21.15.1:0/1973658852 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621cebdf130 msgr2=0x5621cebfcf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:24:54.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.912+0000 7f8fe3e00700 1 --2- 172.21.15.1:0/1973658852 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621cebdf130 0x5621cebfcf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:54.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.912+0000 7f8fe3e00700 1 -- 172.21.15.1:0/1973658852 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5621cebdfa90 msgr2=0x5621cec45cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:24:54.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.912+0000 7f8fe3e00700 1 --2- 172.21.15.1:0/1973658852 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5621cebdfa90 0x5621cec45cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:54.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.912+0000 7f8fe3e00700 1 -- 172.21.15.1:0/1973658852 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fe41a12a0 con 0x5621ceb3dd60 2024-02-02T15:24:54.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.912+0000 7f8fe3400700 1 --2- 172.21.15.1:0/1973658852 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621cebdf130 0x5621cebfcf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:24:54.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.912+0000 7f8fe3e00700 1 --2- 172.21.15.1:0/1973658852 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621ceb3dd60 0x5621cebe5fc0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8fd4000c00 tx=0x7f8fd4003710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f14b9e88f1ffb83f server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:24:54.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.912+0000 7f8fd1600700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:24:54.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.912+0000 7f8fd1600700 1 -- 172.21.15.1:0/1973658852 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd40977d0 con 0x5621ceb3dd60 2024-02-02T15:24:54.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.913+0000 7f8fd1600700 1 -- 172.21.15.1:0/1973658852 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f8fd40993b0 con 0x5621ceb3dd60 2024-02-02T15:24:54.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.913+0000 7f8ffe641b80 1 -- 172.21.15.1:0/1973658852 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8fd409d1e0 con 0x5621ceb3dd60 2024-02-02T15:24:54.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.913+0000 7f8fd1600700 1 -- 172.21.15.1:0/1973658852 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8fd40a1420 con 0x5621ceb3dd60 2024-02-02T15:24:54.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.913+0000 7f8ffe641b80 10 client.14649 fetch_fsmap learned FSMap version 17 2024-02-02T15:24:54.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.913+0000 7f8ffe641b80 1 -- 172.21.15.1:0/1973658852 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x5621ceaf5e50 con 0x5621ceb3dd60 2024-02-02T15:24:54.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.913+0000 7f8fd1600700 1 -- 172.21.15.1:0/1973658852 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8fd409d1e0 con 0x5621ceb3dd60 2024-02-02T15:24:54.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.913+0000 7f8fd1600700 1 -- 172.21.15.1:0/1973658852 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 17) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f8fd4098930 con 0x5621ceb3dd60 2024-02-02T15:24:54.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.913+0000 7f8ffe641b80 10 client.14649 fetch_fsmap finished waiting for FSMap version 17 2024-02-02T15:24:54.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.913+0000 7f8ffe641b80 10 client.14649 resolve_mds: resolved name 'b' to daemon mds.b 2024-02-02T15:24:54.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.913+0000 7f8ffe641b80 20 client.14649 populate_metadata read hostname 'smithi001' 2024-02-02T15:24:54.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.913+0000 7f8ffe641b80 1 --2- 172.21.15.1:0/1973658852 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5621cebe0930 0x5621cec47940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:54.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.913+0000 7f8ffe641b80 4 client.14649 mds_command: new command op to 24557 tid=0[{"prefix": "status"}] 2024-02-02T15:24:54.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.913+0000 7f8ffe641b80 1 -- 172.21.15.1:0/1973658852 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5621ceaeef00 con 0x5621cebe0930 2024-02-02T15:24:54.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.914+0000 7f8fe3400700 1 --2- 172.21.15.1:0/1973658852 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5621cebe0930 0x5621cec47940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:54.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.914+0000 7f8fe3400700 1 --2- 172.21.15.1:0/1973658852 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5621cebe0930 0x5621cec47940 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12634e22a1b091fc server_cookie=f1d06a428ca288e6 in_seq=0 out_seq=0 2024-02-02T15:24:54.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.914+0000 7f8fd1600700 10 client.14649 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:24:54.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.914+0000 7f8fd1600700 1 -- 172.21.15.1:0/1973658852 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x5621ceaeef00 con 0x5621cebe0930 2024-02-02T15:24:54.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.914+0000 7f8fd1600700 10 client.14649 handle_command_reply: tid=0 2024-02-02T15:24:54.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.914+0000 7f8ffe641b80 1 client.14649 shutdown 2024-02-02T15:24:54.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.914+0000 7f8ffe641b80 1 -- 172.21.15.1:0/1973658852 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621ceb3dd60 msgr2=0x5621cebe5fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:54.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.914+0000 7f8ffe641b80 1 --2- 172.21.15.1:0/1973658852 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621ceb3dd60 0x5621cebe5fc0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8fd4000c00 tx=0x7f8fd4003710 comp rx=0 tx=0).stop 2024-02-02T15:24:54.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.915+0000 7f8ffe641b80 1 -- 172.21.15.1:0/1973658852 shutdown_connections 2024-02-02T15:24:54.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.915+0000 7f8ffe641b80 1 --2- 172.21.15.1:0/1973658852 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5621cebe0930 0x5621cec47940 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:54.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.915+0000 7f8ffe641b80 1 --2- 172.21.15.1:0/1973658852 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5621cebdfa90 0x5621cec45cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:54.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.915+0000 7f8ffe641b80 1 --2- 172.21.15.1:0/1973658852 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621ceb3dd60 0x5621cebe5fc0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:54.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.915+0000 7f8fd1600700 0 client.14649 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:24:54.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.915+0000 7f8ffe641b80 1 --2- 172.21.15.1:0/1973658852 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621cebdf130 0x5621cebfcf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:54.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.915+0000 7f8ffe641b80 1 -- 172.21.15.1:0/1973658852 >> 172.21.15.1:0/1973658852 conn(0x5621cebda1c0 msgr2=0x5621cebe6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:24:54.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.915+0000 7f8ffe641b80 1 -- 172.21.15.1:0/1973658852 shutdown_connections 2024-02-02T15:24:54.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.915+0000 7f8ffe641b80 1 -- 172.21.15.1:0/1973658852 wait complete. 2024-02-02T15:24:54.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.915+0000 7f8ffe641b80 20 client.14649 trim_cache size 0 max 16384 2024-02-02T15:24:54.920 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:24:54.920 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "371a409c-c1de-11ee-95b6-87774f69a715", 2024-02-02T15:24:54.920 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 0, 2024-02-02T15:24:54.920 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 24557, 2024-02-02T15:24:54.920 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2024-02-02T15:24:54.920 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2024-02-02T15:24:54.920 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2024-02-02T15:24:54.920 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 7.149667773, 2024-02-02T15:24:54.920 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 17, 2024-02-02T15:24:54.920 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 84, 2024-02-02T15:24:54.920 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 83, 2024-02-02T15:24:54.920 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 13.480362680000001 2024-02-02T15:24:54.920 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:24:54.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.915+0000 7f8feac00700 1 -- 172.21.15.1:0/734218145 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f8fc40a0520 msgr2=0x7f8fc40a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:54.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.915+0000 7f8feac00700 1 --2- 172.21.15.1:0/734218145 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f8fc40a0520 0x7f8fc40a29e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc00b780 tx=0x7f8fdc0095a0 comp rx=0 tx=0).stop 2024-02-02T15:24:54.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.915+0000 7f8feac00700 1 -- 172.21.15.1:0/734218145 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8fe4100e00 msgr2=0x7f8fe4198e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:54.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.915+0000 7f8feac00700 1 --2- 172.21.15.1:0/734218145 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8fe4100e00 0x7f8fe4198e00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8fd400e970 tx=0x7f8fd400ec80 comp rx=0 tx=0).stop 2024-02-02T15:24:54.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.916+0000 7f8feac00700 1 -- 172.21.15.1:0/734218145 shutdown_connections 2024-02-02T15:24:54.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.916+0000 7f8feac00700 1 --2- 172.21.15.1:0/734218145 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8fe4102990 0x7f8fe41a03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:54.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.916+0000 7f8feac00700 1 --2- 172.21.15.1:0/734218145 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8fe4100e00 0x7f8fe4198e00 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:54.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.916+0000 7f8feac00700 1 --2- 172.21.15.1:0/734218145 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f8fc40a0520 0x7f8fc40a29e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:54.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.916+0000 7f8feac00700 1 --2- 172.21.15.1:0/734218145 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8fe4101ff0 0x7f8fe4199340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:54.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.916+0000 7f8feac00700 1 -- 172.21.15.1:0/734218145 >> 172.21.15.1:0/734218145 conn(0x7f8fe40fc370 msgr2=0x7f8fe4107880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:24:54.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.916+0000 7f8feac00700 1 -- 172.21.15.1:0/734218145 shutdown_connections 2024-02-02T15:24:54.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:54.916+0000 7f8feac00700 1 -- 172.21.15.1:0/734218145 wait complete. 2024-02-02T15:24:54.932 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:54 smithi001 ceph-mon[35470]: pgmap v238: 129 pgs: 7 creating+peering, 112 active+clean, 10 unknown; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s wr, 3 op/s 2024-02-02T15:24:54.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:54 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/3585958389' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:24:54.933 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:54 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/2494680206' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-02T15:24:54.934 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-02-02T15:24:54.934 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-02-02T15:24:54.934 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph fs ls 2024-02-02T15:24:55.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:54 smithi071 ceph-mon[44707]: pgmap v238: 129 pgs: 7 creating+peering, 112 active+clean, 10 unknown; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s wr, 3 op/s 2024-02-02T15:24:55.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:54 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/3585958389' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:24:55.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:54 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/2494680206' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-02T15:24:55.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:54 smithi142 ceph-mon[45728]: pgmap v238: 129 pgs: 7 creating+peering, 112 active+clean, 10 unknown; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s wr, 3 op/s 2024-02-02T15:24:55.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:54 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/3585958389' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:24:55.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:54 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/2494680206' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-02T15:24:55.298 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.a/config 2024-02-02T15:24:56.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:55 smithi071 ceph-mon[44707]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-02-02T15:24:56.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:55 smithi071 ceph-mon[44707]: Cluster is now healthy 2024-02-02T15:24:56.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:55 smithi142 ceph-mon[45728]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-02-02T15:24:56.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:55 smithi142 ceph-mon[45728]: Cluster is now healthy 2024-02-02T15:24:56.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:55 smithi001 ceph-mon[35470]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-02-02T15:24:56.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:55 smithi001 ceph-mon[35470]: Cluster is now healthy 2024-02-02T15:24:56.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:56.537+0000 7f9294a00700 1 -- 172.21.15.1:0/4143755603 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f928c1060a0 msgr2=0x7f928c10dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:56.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:56.537+0000 7f9294a00700 1 --2- 172.21.15.1:0/4143755603 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f928c1060a0 0x7f928c10dc70 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9284009990 tx=0x7f9284009ca0 comp rx=0 tx=0).stop 2024-02-02T15:24:56.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:56.537+0000 7f9294a00700 1 -- 172.21.15.1:0/4143755603 shutdown_connections 2024-02-02T15:24:56.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:56.537+0000 7f9294a00700 1 --2- 172.21.15.1:0/4143755603 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f928c1060a0 0x7f928c10dc70 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:56.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:56.537+0000 7f9294a00700 1 --2- 172.21.15.1:0/4143755603 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f928c100fc0 0x7f928c1013a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:56.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:56.537+0000 7f9294a00700 1 --2- 172.21.15.1:0/4143755603 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f928c101970 0x7f928c1059c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:56.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:56.537+0000 7f9294a00700 1 -- 172.21.15.1:0/4143755603 >> 172.21.15.1:0/4143755603 conn(0x7f928c0fc830 msgr2=0x7f928c0fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:24:56.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:56.537+0000 7f9294a00700 1 -- 172.21.15.1:0/4143755603 shutdown_connections 2024-02-02T15:24:56.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:56.537+0000 7f9294a00700 1 -- 172.21.15.1:0/4143755603 wait complete. 2024-02-02T15:24:56.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:56.538+0000 7f9294a00700 1 Processor -- start 2024-02-02T15:24:56.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:56.538+0000 7f9294a00700 1 -- start start 2024-02-02T15:24:56.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:56.539+0000 7f9294a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f928c100fc0 0x7f928c073040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:56.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:56.539+0000 7f9294a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f928c101970 0x7f928c073580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:56.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:56.539+0000 7f9294a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f928c1060a0 0x7f928c073ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:56.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:56.539+0000 7f9294a00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f928c1a52b0 con 0x7f928c101970 2024-02-02T15:24:56.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:56.539+0000 7f9294a00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f928c1a5420 con 0x7f928c100fc0 2024-02-02T15:24:56.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:56.539+0000 7f9294a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f928c1a55e0 con 0x7f928c1060a0 2024-02-02T15:24:56.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:56.539+0000 7f9292000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f928c100fc0 0x7f928c073040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:56.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:56.539+0000 7f9292000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f928c100fc0 0x7f928c073040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:56586/0 (socket says 172.21.15.1:56586) 2024-02-02T15:24:56.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:56.539+0000 7f9292000700 1 -- 172.21.15.1:0/2605009207 learned_addr learned my addr 172.21.15.1:0/2605009207 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:24:56.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:56.539+0000 7f9291600700 1 --2- 172.21.15.1:0/2605009207 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f928c101970 0x7f928c073580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:56.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:56.539+0000 7f9292a00700 1 --2- 172.21.15.1:0/2605009207 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f928c1060a0 0x7f928c073ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:56.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:56.540+0000 7f9292000700 1 -- 172.21.15.1:0/2605009207 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f928c1060a0 msgr2=0x7f928c073ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:56.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:56.540+0000 7f9292000700 1 --2- 172.21.15.1:0/2605009207 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f928c1060a0 0x7f928c073ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:56.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:56.540+0000 7f9292000700 1 -- 172.21.15.1:0/2605009207 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f928c101970 msgr2=0x7f928c073580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:56.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:56.540+0000 7f9292000700 1 --2- 172.21.15.1:0/2605009207 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f928c101970 0x7f928c073580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:56.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:56.540+0000 7f9292000700 1 -- 172.21.15.1:0/2605009207 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92840095f0 con 0x7f928c100fc0 2024-02-02T15:24:56.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:56.540+0000 7f9292000700 1 --2- 172.21.15.1:0/2605009207 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f928c100fc0 0x7f928c073040 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f927c00ef30 tx=0x7f927c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:24:56.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:56.540+0000 7f928aa00700 1 -- 172.21.15.1:0/2605009207 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f927c018070 con 0x7f928c100fc0 2024-02-02T15:24:56.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:56.541+0000 7f928aa00700 1 -- 172.21.15.1:0/2605009207 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f927c00f040 con 0x7f928c100fc0 2024-02-02T15:24:56.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:56.541+0000 7f9294a00700 1 -- 172.21.15.1:0/2605009207 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f928c1a5860 con 0x7f928c100fc0 2024-02-02T15:24:56.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:56.541+0000 7f928aa00700 1 -- 172.21.15.1:0/2605009207 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f927c00cd80 con 0x7f928c100fc0 2024-02-02T15:24:56.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:56.541+0000 7f9294a00700 1 -- 172.21.15.1:0/2605009207 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f928c1a5d10 con 0x7f928c100fc0 2024-02-02T15:24:56.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:56.542+0000 7f9294a00700 1 -- 172.21.15.1:0/2605009207 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f928c04ea40 con 0x7f928c100fc0 2024-02-02T15:24:56.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:56.544+0000 7f928aa00700 1 -- 172.21.15.1:0/2605009207 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 135069+0+0 (secure 0 0 0) 0x7f927c003a00 con 0x7f928c100fc0 2024-02-02T15:24:56.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:56.545+0000 7f928aa00700 1 --2- 172.21.15.1:0/2605009207 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f926c0a0520 0x7f926c0a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:24:56.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:56.545+0000 7f928aa00700 1 -- 172.21.15.1:0/2605009207 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f927c0cc0d0 con 0x7f928c100fc0 2024-02-02T15:24:56.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:56.545+0000 7f9291600700 1 --2- 172.21.15.1:0/2605009207 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f926c0a0520 0x7f926c0a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:24:56.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:56.546+0000 7f9291600700 1 --2- 172.21.15.1:0/2605009207 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f926c0a0520 0x7f926c0a29e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f928c0747e0 tx=0x7f9274009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:24:56.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:56.547+0000 7f928aa00700 1 -- 172.21.15.1:0/2605009207 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f927c095ab0 con 0x7f928c100fc0 2024-02-02T15:24:56.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:56.799+0000 7f9294a00700 1 -- 172.21.15.1:0/2605009207 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f928c1a60c0 con 0x7f928c100fc0 2024-02-02T15:24:56.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:56.800+0000 7f928aa00700 1 -- 172.21.15.1:0/2605009207 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+88 (secure 0 0 0) 0x7f927c0990d0 con 0x7f928c100fc0 2024-02-02T15:24:56.800 INFO:teuthology.orchestra.run.smithi001.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-02-02T15:24:56.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:56.803+0000 7f9294a00700 1 -- 172.21.15.1:0/2605009207 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f926c0a0520 msgr2=0x7f926c0a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:56.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:56.803+0000 7f9294a00700 1 --2- 172.21.15.1:0/2605009207 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f926c0a0520 0x7f926c0a29e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f928c0747e0 tx=0x7f9274009260 comp rx=0 tx=0).stop 2024-02-02T15:24:56.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:56.803+0000 7f9294a00700 1 -- 172.21.15.1:0/2605009207 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f928c100fc0 msgr2=0x7f928c073040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:24:56.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:56.803+0000 7f9294a00700 1 --2- 172.21.15.1:0/2605009207 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f928c100fc0 0x7f928c073040 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f927c00ef30 tx=0x7f927c00c3a0 comp rx=0 tx=0).stop 2024-02-02T15:24:56.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:56.803+0000 7f9294a00700 1 -- 172.21.15.1:0/2605009207 shutdown_connections 2024-02-02T15:24:56.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:56.803+0000 7f9294a00700 1 --2- 172.21.15.1:0/2605009207 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f926c0a0520 0x7f926c0a29e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:56.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:56.803+0000 7f9294a00700 1 --2- 172.21.15.1:0/2605009207 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f928c1060a0 0x7f928c073ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:56.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:56.803+0000 7f9294a00700 1 --2- 172.21.15.1:0/2605009207 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f928c100fc0 0x7f928c073040 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:56.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:56.803+0000 7f9294a00700 1 --2- 172.21.15.1:0/2605009207 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f928c101970 0x7f928c073580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:24:56.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:56.803+0000 7f9294a00700 1 -- 172.21.15.1:0/2605009207 >> 172.21.15.1:0/2605009207 conn(0x7f928c0fc830 msgr2=0x7f928c0fd300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:24:56.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:56.803+0000 7f9294a00700 1 -- 172.21.15.1:0/2605009207 shutdown_connections 2024-02-02T15:24:56.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:24:56.803+0000 7f9294a00700 1 -- 172.21.15.1:0/2605009207 wait complete. 2024-02-02T15:24:57.172 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-02-02T15:24:57.172 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-02-02T15:24:57.172 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi001.front.sepia.ceph.com 2024-02-02T15:24:57.172 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-02-02T15:24:57.172 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-02-02T15:24:57.172 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-02-02T15:24:57.172 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-02-02T15:24:57.172 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-02-02T15:24:57.172 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-02-02T15:24:57.173 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T15:24:57.173 DEBUG:teuthology.orchestra.run.smithi001:> ip addr 2024-02-02T15:24:57.216 INFO:teuthology.orchestra.run.smithi001.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-02T15:24:57.216 INFO:teuthology.orchestra.run.smithi001.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-02T15:24:57.216 INFO:teuthology.orchestra.run.smithi001.stdout: inet 127.0.0.1/8 scope host lo 2024-02-02T15:24:57.216 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-02-02T15:24:57.216 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 ::1/128 scope host 2024-02-02T15:24:57.216 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-02-02T15:24:57.217 INFO:teuthology.orchestra.run.smithi001.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-02T15:24:57.217 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:6b:f6:7a brd ff:ff:ff:ff:ff:ff 2024-02-02T15:24:57.217 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp7s0f0 2024-02-02T15:24:57.217 INFO:teuthology.orchestra.run.smithi001.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-02T15:24:57.217 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:6b:f6:7b brd ff:ff:ff:ff:ff:ff 2024-02-02T15:24:57.217 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp7s0f1 2024-02-02T15:24:57.217 INFO:teuthology.orchestra.run.smithi001.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-02-02T15:24:57.217 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:bd:15:e8 brd ff:ff:ff:ff:ff:ff 2024-02-02T15:24:57.217 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp1s0f0 2024-02-02T15:24:57.217 INFO:teuthology.orchestra.run.smithi001.stdout: inet 172.21.15.1/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-02-02T15:24:57.217 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft 42022sec preferred_lft 42022sec 2024-02-02T15:24:57.217 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 fe80::ec4:7aff:febd:15e8/64 scope link 2024-02-02T15:24:57.217 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-02-02T15:24:57.217 INFO:teuthology.orchestra.run.smithi001.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-02T15:24:57.218 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:bd:15:e9 brd ff:ff:ff:ff:ff:ff 2024-02-02T15:24:57.218 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp1s0f1 2024-02-02T15:24:57.219 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-02-02T15:24:57.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T15:24:57.219 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec bash -c ' 2024-02-02T15:24:57.219 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2024-02-02T15:24:57.219 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link add name ceph-brx type bridge 2024-02-02T15:24:57.219 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip addr flush dev ceph-brx 2024-02-02T15:24:57.219 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link set ceph-brx up 2024-02-02T15:24:57.219 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-02-02T15:24:57.219 DEBUG:teuthology.orchestra.run.smithi001:> ') 2024-02-02T15:24:57.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:57 smithi001 ceph-mon[35470]: pgmap v239: 129 pgs: 3 creating+peering, 126 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 2 op/s 2024-02-02T15:24:57.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:57 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/2605009207' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-02-02T15:24:57.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:57 smithi071 ceph-mon[44707]: pgmap v239: 129 pgs: 3 creating+peering, 126 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 2 op/s 2024-02-02T15:24:57.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:57 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/2605009207' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-02-02T15:24:57.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:57 smithi142 ceph-mon[45728]: pgmap v239: 129 pgs: 3 creating+peering, 126 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 2 op/s 2024-02-02T15:24:57.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:57 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/2605009207' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-02-02T15:24:57.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T15:24:57.683 DEBUG:teuthology.orchestra.run.smithi001:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-02-02T15:24:57.752 INFO:teuthology.orchestra.run.smithi001.stdout:1 2024-02-02T15:24:57.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T15:24:57.755 DEBUG:teuthology.orchestra.run.smithi001:> route 2024-02-02T15:24:57.816 INFO:teuthology.orchestra.run.smithi001.stdout:Kernel IP routing table 2024-02-02T15:24:57.816 INFO:teuthology.orchestra.run.smithi001.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-02-02T15:24:57.816 INFO:teuthology.orchestra.run.smithi001.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-02-02T15:24:57.816 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-02-02T15:24:57.816 INFO:teuthology.orchestra.run.smithi001.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-02-02T15:24:57.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T15:24:57.817 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec bash -c ' 2024-02-02T15:24:57.817 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2024-02-02T15:24:57.817 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-02-02T15:24:57.818 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-02-02T15:24:57.818 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-02-02T15:24:57.818 DEBUG:teuthology.orchestra.run.smithi001:> ') 2024-02-02T15:24:58.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T15:24:58.458 DEBUG:teuthology.orchestra.run.smithi001:> ip netns list 2024-02-02T15:24:58.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T15:24:58.501 DEBUG:teuthology.orchestra.run.smithi001:> ip netns list-id 2024-02-02T15:24:58.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T15:24:58.584 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec bash -c ' 2024-02-02T15:24:58.584 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2024-02-02T15:24:58.584 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-02-02T15:24:58.584 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-02-02T15:24:58.584 DEBUG:teuthology.orchestra.run.smithi001:> ') 2024-02-02T15:24:58.763 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-02-02T15:24:58.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T15:24:58.764 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec bash -c ' 2024-02-02T15:24:58.764 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2024-02-02T15:24:58.764 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-02-02T15:24:58.764 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-02-02T15:24:58.764 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-02-02T15:24:58.764 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-02-02T15:24:58.764 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-02-02T15:24:58.764 DEBUG:teuthology.orchestra.run.smithi001:> ') 2024-02-02T15:24:59.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T15:24:59.033 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec bash -c ' 2024-02-02T15:24:59.033 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2024-02-02T15:24:59.033 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link set brx.0 up 2024-02-02T15:24:59.033 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link set dev brx.0 master ceph-brx 2024-02-02T15:24:59.033 DEBUG:teuthology.orchestra.run.smithi001:> ') 2024-02-02T15:24:59.201 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-02-02T15:24:59.201 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-02T15:24:59.201 DEBUG:teuthology.orchestra.run.smithi001:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-02-02T15:24:59.243 INFO:teuthology.orchestra.run.smithi001.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-02-02T15:24:59.243 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-02T15:24:59.243 DEBUG:teuthology.orchestra.run.smithi001:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-02-02T15:24:59.260 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:24:59 smithi001 ceph-mon[35470]: pgmap v240: 129 pgs: 129 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 2 op/s 2024-02-02T15:24:59.299 DEBUG:teuthology.orchestra.run.smithi001:> sudo modprobe fuse 2024-02-02T15:24:59.406 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-02-02T15:24:59.406 DEBUG:teuthology.orchestra.run.smithi001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-02-02T15:24:59.470 INFO:teuthology.orchestra.run.smithi001.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-02-02T15:24:59.472 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-02-02T15:24:59.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T15:24:59.473 DEBUG:teuthology.orchestra.run.smithi001:> ls /sys/fs/fuse/connections 2024-02-02T15:24:59.514 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-02-02T15:24:59.514 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-02-02T15:24:59.557 DEBUG:teuthology.orchestra.run.smithi001:> sudo modprobe fuse 2024-02-02T15:24:59.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:24:59 smithi071 ceph-mon[44707]: pgmap v240: 129 pgs: 129 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 2 op/s 2024-02-02T15:24:59.626 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-02-02T15:24:59.626 DEBUG:teuthology.orchestra.run.smithi001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-02-02T15:24:59.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:24:59 smithi142 ceph-mon[45728]: pgmap v240: 129 pgs: 129 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 2 op/s 2024-02-02T15:24:59.696 INFO:teuthology.orchestra.run.smithi001.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-02-02T15:24:59.701 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-02-02T15:24:59.701 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T15:24:59.701 DEBUG:teuthology.orchestra.run.smithi001:> ls /sys/fs/fuse/connections 2024-02-02T15:24:59.746 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi001.stderr:2024-02-02T15:24:59.746+0000 7f3607e255c0 -1 init, newargv = 0x55b07f248200 newargc=15 2024-02-02T15:24:59.746 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi001.stderr:ceph-fuse[87173]: starting ceph client 2024-02-02T15:24:59.755 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi001.stderr:ceph-fuse[87173]: starting fuse 2024-02-02T15:25:00.845 DEBUG:teuthology.orchestra.run.smithi001:> sudo modprobe fuse 2024-02-02T15:25:00.921 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-02-02T15:25:00.921 DEBUG:teuthology.orchestra.run.smithi001:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2024-02-02T15:25:00.988 INFO:teuthology.orchestra.run.smithi001.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2024-02-02T15:25:00.991 DEBUG:teuthology.orchestra.run:got remote process result: 32 2024-02-02T15:25:00.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T15:25:00.991 DEBUG:teuthology.orchestra.run.smithi001:> ls /sys/fs/fuse/connections 2024-02-02T15:25:01.033 INFO:teuthology.orchestra.run.smithi001.stdout:121 2024-02-02T15:25:01.033 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [121] 2024-02-02T15:25:01.033 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2024-02-02T15:25:01.033 DEBUG:teuthology.orchestra.run.smithi001:> import glob 2024-02-02T15:25:01.033 DEBUG:teuthology.orchestra.run.smithi001:> import re 2024-02-02T15:25:01.033 DEBUG:teuthology.orchestra.run.smithi001:> import os 2024-02-02T15:25:01.033 DEBUG:teuthology.orchestra.run.smithi001:> import subprocess 2024-02-02T15:25:01.033 DEBUG:teuthology.orchestra.run.smithi001:> 2024-02-02T15:25:01.033 DEBUG:teuthology.orchestra.run.smithi001:> def _find_admin_socket(client_name): 2024-02-02T15:25:01.033 DEBUG:teuthology.orchestra.run.smithi001:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-02-02T15:25:01.034 DEBUG:teuthology.orchestra.run.smithi001:> files = glob.glob(asok_path) 2024-02-02T15:25:01.034 DEBUG:teuthology.orchestra.run.smithi001:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-02-02T15:25:01.034 DEBUG:teuthology.orchestra.run.smithi001:> 2024-02-02T15:25:01.034 DEBUG:teuthology.orchestra.run.smithi001:> # Given a non-glob path, it better be there 2024-02-02T15:25:01.034 DEBUG:teuthology.orchestra.run.smithi001:> if "*" not in asok_path: 2024-02-02T15:25:01.034 DEBUG:teuthology.orchestra.run.smithi001:> assert(len(files) == 1) 2024-02-02T15:25:01.034 DEBUG:teuthology.orchestra.run.smithi001:> return files[0] 2024-02-02T15:25:01.034 DEBUG:teuthology.orchestra.run.smithi001:> 2024-02-02T15:25:01.034 DEBUG:teuthology.orchestra.run.smithi001:> for f in files: 2024-02-02T15:25:01.034 DEBUG:teuthology.orchestra.run.smithi001:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-02-02T15:25:01.034 DEBUG:teuthology.orchestra.run.smithi001:> if os.path.exists("/proc/{0}".format(pid)): 2024-02-02T15:25:01.034 DEBUG:teuthology.orchestra.run.smithi001:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-02-02T15:25:01.034 DEBUG:teuthology.orchestra.run.smithi001:> contents = proc_f.read() 2024-02-02T15:25:01.034 DEBUG:teuthology.orchestra.run.smithi001:> if mountpoint in contents: 2024-02-02T15:25:01.034 DEBUG:teuthology.orchestra.run.smithi001:> return f 2024-02-02T15:25:01.034 DEBUG:teuthology.orchestra.run.smithi001:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-02-02T15:25:01.034 DEBUG:teuthology.orchestra.run.smithi001:> 2024-02-02T15:25:01.034 DEBUG:teuthology.orchestra.run.smithi001:> print(_find_admin_socket("client.admin")) 2024-02-02T15:25:01.035 DEBUG:teuthology.orchestra.run.smithi001:> ' 2024-02-02T15:25:01.218 INFO:teuthology.orchestra.run.smithi001.stdout:/var/run/ceph/ceph-client.admin.87173.asok 2024-02-02T15:25:01.393 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.87173.asok 2024-02-02T15:25:01.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T15:25:01.393 DEBUG:teuthology.orchestra.run.smithi001:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.87173.asok status 2024-02-02T15:25:01.490 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:01 smithi001 ceph-mon[35470]: pgmap v241: 129 pgs: 129 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 883 B/s wr, 0 op/s 2024-02-02T15:25:01.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:01 smithi071 ceph-mon[44707]: pgmap v241: 129 pgs: 129 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 883 B/s wr, 0 op/s 2024-02-02T15:25:01.620 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:25:01.620 INFO:teuthology.orchestra.run.smithi001.stdout: "metadata": { 2024-02-02T15:25:01.620 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_sha1": "0fb327883c124fd83155c94db0804f823b805a12", 2024-02-02T15:25:01.620 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_version": "ceph version 18.2.1-325-g0fb32788 (0fb327883c124fd83155c94db0804f823b805a12) reef (stable)", 2024-02-02T15:25:01.620 INFO:teuthology.orchestra.run.smithi001.stdout: "entity_id": "admin", 2024-02-02T15:25:01.620 INFO:teuthology.orchestra.run.smithi001.stdout: "hostname": "smithi001", 2024-02-02T15:25:01.620 INFO:teuthology.orchestra.run.smithi001.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-02-02T15:25:01.620 INFO:teuthology.orchestra.run.smithi001.stdout: "pid": "87173", 2024-02-02T15:25:01.620 INFO:teuthology.orchestra.run.smithi001.stdout: "root": "/" 2024-02-02T15:25:01.620 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:25:01.620 INFO:teuthology.orchestra.run.smithi001.stdout: "dentry_count": 0, 2024-02-02T15:25:01.621 INFO:teuthology.orchestra.run.smithi001.stdout: "dentry_pinned_count": 0, 2024-02-02T15:25:01.621 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 14661, 2024-02-02T15:25:01.621 INFO:teuthology.orchestra.run.smithi001.stdout: "inst": { 2024-02-02T15:25:01.621 INFO:teuthology.orchestra.run.smithi001.stdout: "name": { 2024-02-02T15:25:01.621 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "client", 2024-02-02T15:25:01.621 INFO:teuthology.orchestra.run.smithi001.stdout: "num": 14661 2024-02-02T15:25:01.621 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:25:01.621 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": { 2024-02-02T15:25:01.621 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-02-02T15:25:01.621 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "192.168.0.1:0", 2024-02-02T15:25:01.621 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4029335009 2024-02-02T15:25:01.621 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:25:01.621 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:25:01.621 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": { 2024-02-02T15:25:01.621 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-02-02T15:25:01.621 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "192.168.0.1:0", 2024-02-02T15:25:01.622 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 4029335009 2024-02-02T15:25:01.622 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:25:01.622 INFO:teuthology.orchestra.run.smithi001.stdout: "inst_str": "client.14661 192.168.0.1:0/4029335009", 2024-02-02T15:25:01.622 INFO:teuthology.orchestra.run.smithi001.stdout: "addr_str": "192.168.0.1:0/4029335009", 2024-02-02T15:25:01.622 INFO:teuthology.orchestra.run.smithi001.stdout: "inode_count": 1, 2024-02-02T15:25:01.622 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_epoch": 17, 2024-02-02T15:25:01.622 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_epoch": 84, 2024-02-02T15:25:01.622 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_epoch_barrier": 0, 2024-02-02T15:25:01.622 INFO:teuthology.orchestra.run.smithi001.stdout: "blocklisted": false, 2024-02-02T15:25:01.622 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs" 2024-02-02T15:25:01.622 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:25:01.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T15:25:01.634 DEBUG:teuthology.orchestra.run.smithi001:> stat --file-system '--printf=%T 2024-02-02T15:25:01.634 DEBUG:teuthology.orchestra.run.smithi001:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-02-02T15:25:01.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:01 smithi142 ceph-mon[45728]: pgmap v241: 129 pgs: 129 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 883 B/s wr, 0 op/s 2024-02-02T15:25:01.677 INFO:teuthology.orchestra.run.smithi001.stdout:fuseblk 2024-02-02T15:25:01.677 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-02-02T15:25:01.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T15:25:01.677 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-02-02T15:25:01.784 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T15:25:01.784 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-02-02T15:25:01.922 INFO:teuthology.orchestra.run.smithi001.stdout:# file: . 2024-02-02T15:25:01.922 INFO:teuthology.orchestra.run.smithi001.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-02-02T15:25:01.922 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:25:01.922 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-02-02T15:25:01.922 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T15:25:01.964 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T15:25:02.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.201+0000 7fce67c00700 1 -- 172.21.15.1:0/4099426738 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fce60101f30 msgr2=0x7fce60102390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:02.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.201+0000 7fce67c00700 1 --2- 172.21.15.1:0/4099426738 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fce60101f30 0x7fce60102390 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fce480099c0 tx=0x7fce48009cd0 comp rx=0 tx=0).stop 2024-02-02T15:25:02.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.201+0000 7fce67c00700 1 -- 172.21.15.1:0/4099426738 shutdown_connections 2024-02-02T15:25:02.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.201+0000 7fce67c00700 1 --2- 172.21.15.1:0/4099426738 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce601028d0 0x7fce60109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:02.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.201+0000 7fce67c00700 1 --2- 172.21.15.1:0/4099426738 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce60100d40 0x7fce60101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:02.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.201+0000 7fce67c00700 1 --2- 172.21.15.1:0/4099426738 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fce60101f30 0x7fce60102390 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:02.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.202+0000 7fce67c00700 1 -- 172.21.15.1:0/4099426738 >> 172.21.15.1:0/4099426738 conn(0x7fce600fc290 msgr2=0x7fce600fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:02.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.202+0000 7fce67c00700 1 -- 172.21.15.1:0/4099426738 shutdown_connections 2024-02-02T15:25:02.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.202+0000 7fce67c00700 1 -- 172.21.15.1:0/4099426738 wait complete. 2024-02-02T15:25:02.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.203+0000 7fce67c00700 1 Processor -- start 2024-02-02T15:25:02.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.203+0000 7fce67c00700 1 -- start start 2024-02-02T15:25:02.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.203+0000 7fce67c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fce60100d40 0x7fce60198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:02.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.203+0000 7fce67c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce60101f30 0x7fce601991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:02.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.203+0000 7fce67c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce601028d0 0x7fce601a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:02.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.203+0000 7fce67c00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce601a07c0 con 0x7fce60100d40 2024-02-02T15:25:02.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.203+0000 7fce67c00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce601a0930 con 0x7fce60101f30 2024-02-02T15:25:02.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.203+0000 7fce67c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce601a0af0 con 0x7fce601028d0 2024-02-02T15:25:02.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.203+0000 7fce65200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fce60100d40 0x7fce60198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:02.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.204+0000 7fce65200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fce60100d40 0x7fce60198c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33780/0 (socket says 172.21.15.1:33780) 2024-02-02T15:25:02.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.204+0000 7fce65200700 1 -- 172.21.15.1:0/400144641 learned_addr learned my addr 172.21.15.1:0/400144641 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:25:02.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.204+0000 7fce5fe00700 1 --2- 172.21.15.1:0/400144641 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce60101f30 0x7fce601991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:02.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.204+0000 7fce65c00700 1 --2- 172.21.15.1:0/400144641 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce601028d0 0x7fce601a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:02.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.204+0000 7fce65200700 1 -- 172.21.15.1:0/400144641 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce601028d0 msgr2=0x7fce601a01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:02.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.204+0000 7fce65200700 1 --2- 172.21.15.1:0/400144641 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce601028d0 0x7fce601a01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:02.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.204+0000 7fce65200700 1 -- 172.21.15.1:0/400144641 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce60101f30 msgr2=0x7fce601991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:02.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.204+0000 7fce65200700 1 --2- 172.21.15.1:0/400144641 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce60101f30 0x7fce601991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:02.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.204+0000 7fce65200700 1 -- 172.21.15.1:0/400144641 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce480095f0 con 0x7fce60100d40 2024-02-02T15:25:02.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.204+0000 7fce65200700 1 --2- 172.21.15.1:0/400144641 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fce60100d40 0x7fce60198c80 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fce5000e970 tx=0x7fce5000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:02.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.204+0000 7fce5d600700 1 -- 172.21.15.1:0/400144641 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce50009790 con 0x7fce60100d40 2024-02-02T15:25:02.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.204+0000 7fce67c00700 1 -- 172.21.15.1:0/400144641 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce601a0dd0 con 0x7fce60100d40 2024-02-02T15:25:02.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.204+0000 7fce5d600700 1 -- 172.21.15.1:0/400144641 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fce5000c7e0 con 0x7fce60100d40 2024-02-02T15:25:02.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.205+0000 7fce67c00700 1 -- 172.21.15.1:0/400144641 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce601a1250 con 0x7fce60100d40 2024-02-02T15:25:02.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.205+0000 7fce5d600700 1 -- 172.21.15.1:0/400144641 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce50003c10 con 0x7fce60100d40 2024-02-02T15:25:02.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.205+0000 7fce67c00700 1 -- 172.21.15.1:0/400144641 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce6010c5a0 con 0x7fce60100d40 2024-02-02T15:25:02.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.208+0000 7fce5d600700 1 -- 172.21.15.1:0/400144641 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) v1 ==== 135069+0+0 (secure 0 0 0) 0x7fce5000cc50 con 0x7fce60100d40 2024-02-02T15:25:02.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.209+0000 7fce5d600700 1 --2- 172.21.15.1:0/400144641 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fce400a0520 0x7fce400a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:02.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.209+0000 7fce5d600700 1 -- 172.21.15.1:0/400144641 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fce50014070 con 0x7fce60100d40 2024-02-02T15:25:02.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.209+0000 7fce5fe00700 1 --2- 172.21.15.1:0/400144641 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fce400a0520 0x7fce400a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:02.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.210+0000 7fce5fe00700 1 --2- 172.21.15.1:0/400144641 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fce400a0520 0x7fce400a29e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fce4800b580 tx=0x7fce48005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:02.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.212+0000 7fce5d600700 1 -- 172.21.15.1:0/400144641 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7fce50095840 con 0x7fce60100d40 2024-02-02T15:25:02.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.423+0000 7fce67c00700 1 -- 172.21.15.1:0/400144641 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fce6010c730 con 0x7fce60100d40 2024-02-02T15:25:02.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.425+0000 7fce5d600700 1 -- 172.21.15.1:0/400144641 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+23841 (secure 0 0 0) 0x7fce50098e60 con 0x7fce60100d40 2024-02-02T15:25:02.425 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:25:02.425 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":84,"fsid":"371a409c-c1de-11ee-95b6-87774f69a715","created":"2024-02-02T15:18:26.746506+0000","modified":"2024-02-02T15:24:52.310888+0000","last_up_change":"2024-02-02T15:24:08.800951+0000","last_in_change":"2024-02-02T15:23:52.784280+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T15:21:22.471754+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-02T15:24:46.195779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-02T15:24:47.226403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-02T15:24:50.300462+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"73e8a07f-3560-4c19-85ed-030d4d8f0f23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6803","nonce":2591906423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6805","nonce":2591906423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6809","nonce":2591906423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6807","nonce":2591906423}]},"public_addr":"172.21.15.1:6803/2591906423","cluster_addr":"172.21.15.1:6805/2591906423","heartbeat_back_addr":"172.21.15.1:6809/2591906423","heartbeat_front_addr":"172.21.15.1:6807/2591906423","state":["exists","up"]},{"osd":1,"uuid":"e53cffec-0ff9-4ed9-8dcf-ff47d04db2bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6801","nonce":747581719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6803","nonce":747581719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6807","nonce":747581719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6805","nonce":747581719}]},"public_addr":"172.21.15.71:6801/747581719","cluster_addr":"172.21.15.71:6803/747581719","heartbeat_back_addr":"172.21.15.71:6807/747581719","heartbeat_front_addr":"172.21.15.71:6805/747581719","state":["exists","up"]},{"osd":2,"uuid":"856c2347-294c-482b-8595-ad77f4792db4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6801","nonce":377511414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6803","nonce":377511414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6807","nonce":377511414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6805","nonce":377511414}]},"public_addr":"172.21.15.142:6801/377511414","cluster_addr":"172.21.15.142:6803/377511414","heartbeat_back_addr":"172.21.15.142:6807/377511414","heartbeat_front_addr":"172.21.15.142:6805/377511414","state":["exists","up"]},{"osd":3,"uuid":"4b946ad2-2da9-4e0b-9d61-d039d8669a42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6811","nonce":1086124477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6813","nonce":1086124477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6817","nonce":1086124477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6815","nonce":1086124477}]},"public_addr":"172.21.15.1:6811/1086124477","cluster_addr":"172.21.15.1:6813/1086124477","heartbeat_back_addr":"172.21.15.1:6817/1086124477","heartbeat_front_addr":"172.21.15.1:6815/1086124477","state":["exists","up"]},{"osd":4,"uuid":"43e36c43-47a0-4634-a45b-1da14a0e19d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6809","nonce":3095380305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6811","nonce":3095380305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6815","nonce":3095380305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6813","nonce":3095380305}]},"public_addr":"172.21.15.71:6809/3095380305","cluster_addr":"172.21.15.71:6811/3095380305","heartbeat_back_addr":"172.21.15.71:6815/3095380305","heartbeat_front_addr":"172.21.15.71:6813/3095380305","state":["exists","up"]},{"osd":5,"uuid":"eb48160e-ea7c-42f7-9477-be110bc8a34e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6809","nonce":4195149262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6811","nonce":4195149262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6815","nonce":4195149262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6813","nonce":4195149262}]},"public_addr":"172.21.15.142:6809/4195149262","cluster_addr":"172.21.15.142:6811/4195149262","heartbeat_back_addr":"172.21.15.142:6815/4195149262","heartbeat_front_addr":"172.21.15.142:6813/4195149262","state":["exists","up"]},{"osd":6,"uuid":"0ec4501a-5853-4ca6-b325-231f5b72335e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6819","nonce":1560153428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6821","nonce":1560153428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6825","nonce":1560153428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6823","nonce":1560153428}]},"public_addr":"172.21.15.1:6819/1560153428","cluster_addr":"172.21.15.1:6821/1560153428","heartbeat_back_addr":"172.21.15.1:6825/1560153428","heartbeat_front_addr":"172.21.15.1:6823/1560153428","state":["exists","up"]},{"osd":7,"uuid":"029debe8-9cdf-44f6-9aea-cabd1d185f93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6817","nonce":4080955538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6819","nonce":4080955538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6823","nonce":4080955538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6821","nonce":4080955538}]},"public_addr":"172.21.15.71:6817/4080955538","cluster_addr":"172.21.15.71:6819/4080955538","heartbeat_back_addr":"172.21.15.71:6823/4080955538","heartbeat_front_addr":"172.21.15.71:6821/4080955538","state":["exists","up"]},{"osd":8,"uuid":"7c90896c-9955-4c1b-aaca-7531dd5457f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6817","nonce":3942038209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6819","nonce":3942038209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6823","nonce":3942038209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6821","nonce":3942038209}]},"public_addr":"172.21.15.142:6817/3942038209","cluster_addr":"172.21.15.142:6819/3942038209","heartbeat_back_addr":"172.21.15.142:6823/3942038209","heartbeat_front_addr":"172.21.15.142:6821/3942038209","state":["exists","up"]},{"osd":9,"uuid":"7e165e68-31a8-4598-a577-de3e90ae5c5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6827","nonce":2716544850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6829","nonce":2716544850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6833","nonce":2716544850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6831","nonce":2716544850}]},"public_addr":"172.21.15.1:6827/2716544850","cluster_addr":"172.21.15.1:6829/2716544850","heartbeat_back_addr":"172.21.15.1:6833/2716544850","heartbeat_front_addr":"172.21.15.1:6831/2716544850","state":["exists","up"]},{"osd":10,"uuid":"48cd5fdb-0887-444a-aab8-a1de24282bb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6825","nonce":84691841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6827","nonce":84691841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6831","nonce":84691841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6829","nonce":84691841}]},"public_addr":"172.21.15.71:6825/84691841","cluster_addr":"172.21.15.71:6827/84691841","heartbeat_back_addr":"172.21.15.71:6831/84691841","heartbeat_front_addr":"172.21.15.71:6829/84691841","state":["exists","up"]},{"osd":11,"uuid":"f011dd27-11b0-4166-adfe-76756533163a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6825","nonce":2290179934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6827","nonce":2290179934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6831","nonce":2290179934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6829","nonce":2290179934}]},"public_addr":"172.21.15.142:6825/2290179934","cluster_addr":"172.21.15.142:6827/2290179934","heartbeat_back_addr":"172.21.15.142:6831/2290179934","heartbeat_front_addr":"172.21.15.142:6829/2290179934","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:20:44.060692+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:01.903817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:19.368258+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:40.028037+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:57.513361+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:22:14.805944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:22:35.188439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:22:53.014412+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:23:10.507420+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:23:31.849765+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:23:50.070612+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:24:07.388625+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/4092532060":"2024-02-03T15:19:25.836171+0000","172.21.15.1:6801/395498296":"2024-02-03T15:19:25.836171+0000","172.21.15.1:6800/395498296":"2024-02-03T15:19:25.836171+0000","172.21.15.1:0/1575373489":"2024-02-03T15:19:25.836171+0000","172.21.15.1:0/865983374":"2024-02-03T15:19:01.431596+0000","172.21.15.1:0/3165960227":"2024-02-03T15:19:01.431596+0000","172.21.15.1:0/2350830293":"2024-02-03T15:19:01.431596+0000","172.21.15.1:0/2959690250":"2024-02-03T15:19:25.836171+0000","172.21.15.1:6800/1162530732":"2024-02-03T15:19:01.431596+0000","172.21.15.1:6801/1162530732":"2024-02-03T15:19:01.431596+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T15:25:02.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.427+0000 7fce67c00700 1 -- 172.21.15.1:0/400144641 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fce400a0520 msgr2=0x7fce400a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:02.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.427+0000 7fce67c00700 1 --2- 172.21.15.1:0/400144641 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fce400a0520 0x7fce400a29e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fce4800b580 tx=0x7fce48005fb0 comp rx=0 tx=0).stop 2024-02-02T15:25:02.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.427+0000 7fce67c00700 1 -- 172.21.15.1:0/400144641 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fce60100d40 msgr2=0x7fce60198c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:02.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.427+0000 7fce67c00700 1 --2- 172.21.15.1:0/400144641 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fce60100d40 0x7fce60198c80 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fce5000e970 tx=0x7fce5000ec80 comp rx=0 tx=0).stop 2024-02-02T15:25:02.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.427+0000 7fce67c00700 1 -- 172.21.15.1:0/400144641 shutdown_connections 2024-02-02T15:25:02.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.427+0000 7fce67c00700 1 --2- 172.21.15.1:0/400144641 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fce400a0520 0x7fce400a29e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:02.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.427+0000 7fce67c00700 1 --2- 172.21.15.1:0/400144641 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce601028d0 0x7fce601a01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:02.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.427+0000 7fce67c00700 1 --2- 172.21.15.1:0/400144641 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fce60100d40 0x7fce60198c80 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:02.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.427+0000 7fce67c00700 1 --2- 172.21.15.1:0/400144641 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce60101f30 0x7fce601991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:02.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.427+0000 7fce67c00700 1 -- 172.21.15.1:0/400144641 >> 172.21.15.1:0/400144641 conn(0x7fce600fc290 msgr2=0x7fce600fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:02.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.427+0000 7fce67c00700 1 -- 172.21.15.1:0/400144641 shutdown_connections 2024-02-02T15:25:02.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.427+0000 7fce67c00700 1 -- 172.21.15.1:0/400144641 wait complete. 2024-02-02T15:25:02.446 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-02T15:21:22.471754+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-02T15:24:46.195779+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-02T15:24:47.226403+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-02-02T15:24:50.300462+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-02-02T15:25:02.446 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-02T15:25:02.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.648+0000 7f26ea200700 1 -- 172.21.15.1:0/431588135 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f26e4101f70 msgr2=0x7f26e41023f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:02.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.648+0000 7f26ea200700 1 --2- 172.21.15.1:0/431588135 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f26e4101f70 0x7f26e41023f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f26cc009990 tx=0x7f26cc009ca0 comp rx=0 tx=0).stop 2024-02-02T15:25:02.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.648+0000 7f26ea200700 1 -- 172.21.15.1:0/431588135 shutdown_connections 2024-02-02T15:25:02.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.648+0000 7f26ea200700 1 --2- 172.21.15.1:0/431588135 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f26e4102930 0x7f26e4109250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:02.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.648+0000 7f26ea200700 1 --2- 172.21.15.1:0/431588135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26e4100e20 0x7f26e4101240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:02.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.648+0000 7f26ea200700 1 --2- 172.21.15.1:0/431588135 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f26e4101f70 0x7f26e41023f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:02.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.648+0000 7f26ea200700 1 -- 172.21.15.1:0/431588135 >> 172.21.15.1:0/431588135 conn(0x7f26e40fc390 msgr2=0x7f26e40fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:02.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.648+0000 7f26ea200700 1 -- 172.21.15.1:0/431588135 shutdown_connections 2024-02-02T15:25:02.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.648+0000 7f26ea200700 1 -- 172.21.15.1:0/431588135 wait complete. 2024-02-02T15:25:02.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.649+0000 7f26ea200700 1 Processor -- start 2024-02-02T15:25:02.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.649+0000 7f26ea200700 1 -- start start 2024-02-02T15:25:02.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.649+0000 7f26ea200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f26e4100e20 0x7f26e4074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:02.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.649+0000 7f26ea200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f26e4101f70 0x7f26e4073130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:02.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.649+0000 7f26ea200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26e4102930 0x7f26e4073670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:02.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.649+0000 7f26ea200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e4073e10 con 0x7f26e4100e20 2024-02-02T15:25:02.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.649+0000 7f26ea200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e4073f80 con 0x7f26e4102930 2024-02-02T15:25:02.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.649+0000 7f26ea200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e41a0af0 con 0x7f26e4101f70 2024-02-02T15:25:02.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.649+0000 7f26e3400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f26e4100e20 0x7f26e4074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:02.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.649+0000 7f26e3400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f26e4100e20 0x7f26e4074af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33794/0 (socket says 172.21.15.1:33794) 2024-02-02T15:25:02.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.649+0000 7f26e3400700 1 -- 172.21.15.1:0/2268060501 learned_addr learned my addr 172.21.15.1:0/2268060501 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:25:02.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.650+0000 7f26e2a00700 1 --2- 172.21.15.1:0/2268060501 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f26e4101f70 0x7f26e4073130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:02.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.650+0000 7f26e3e00700 1 --2- 172.21.15.1:0/2268060501 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26e4102930 0x7f26e4073670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:02.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.650+0000 7f26e3400700 1 -- 172.21.15.1:0/2268060501 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f26e4101f70 msgr2=0x7f26e4073130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:02.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.650+0000 7f26e3400700 1 --2- 172.21.15.1:0/2268060501 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f26e4101f70 0x7f26e4073130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:02.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.650+0000 7f26e3400700 1 -- 172.21.15.1:0/2268060501 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26e4102930 msgr2=0x7f26e4073670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:02.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.650+0000 7f26e3400700 1 --2- 172.21.15.1:0/2268060501 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26e4102930 0x7f26e4073670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:02.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.650+0000 7f26e3400700 1 -- 172.21.15.1:0/2268060501 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26cc0095f0 con 0x7f26e4100e20 2024-02-02T15:25:02.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.650+0000 7f26e2a00700 1 --2- 172.21.15.1:0/2268060501 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f26e4101f70 0x7f26e4073130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:25:02.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.650+0000 7f26e3400700 1 --2- 172.21.15.1:0/2268060501 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f26e4100e20 0x7f26e4074af0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f26d400e9a0 tx=0x7f26d400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:02.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.650+0000 7f26dbe00700 1 -- 172.21.15.1:0/2268060501 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d4009790 con 0x7f26e4100e20 2024-02-02T15:25:02.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.650+0000 7f26dbe00700 1 -- 172.21.15.1:0/2268060501 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f26d400cbd0 con 0x7f26e4100e20 2024-02-02T15:25:02.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.650+0000 7f26ea200700 1 -- 172.21.15.1:0/2268060501 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26e41a0dd0 con 0x7f26e4100e20 2024-02-02T15:25:02.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.650+0000 7f26dbe00700 1 -- 172.21.15.1:0/2268060501 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d4018680 con 0x7f26e4100e20 2024-02-02T15:25:02.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.651+0000 7f26ea200700 1 -- 172.21.15.1:0/2268060501 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26e41a1250 con 0x7f26e4100e20 2024-02-02T15:25:02.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.651+0000 7f26ea200700 1 -- 172.21.15.1:0/2268060501 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26e4107d40 con 0x7f26e4100e20 2024-02-02T15:25:02.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.654+0000 7f26dbe00700 1 -- 172.21.15.1:0/2268060501 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) v1 ==== 135069+0+0 (secure 0 0 0) 0x7f26d400cd40 con 0x7f26e4100e20 2024-02-02T15:25:02.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.655+0000 7f26dbe00700 1 --2- 172.21.15.1:0/2268060501 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f26c40a05a0 0x7f26c40a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:02.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.655+0000 7f26dbe00700 1 -- 172.21.15.1:0/2268060501 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f26d4014070 con 0x7f26e4100e20 2024-02-02T15:25:02.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.655+0000 7f26e2a00700 1 --2- 172.21.15.1:0/2268060501 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f26c40a05a0 0x7f26c40a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:02.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.656+0000 7f26e2a00700 1 --2- 172.21.15.1:0/2268060501 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f26c40a05a0 0x7f26c40a2a60 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f26cc00b2e0 tx=0x7f26cc005fd0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:02.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.657+0000 7f26dbe00700 1 -- 172.21.15.1:0/2268060501 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f26d4095c80 con 0x7f26e4100e20 2024-02-02T15:25:02.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.867+0000 7f26ea200700 1 -- 172.21.15.1:0/2268060501 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f26e402cee0 con 0x7f26e4100e20 2024-02-02T15:25:02.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.867+0000 7f26dbe00700 1 -- 172.21.15.1:0/2268060501 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7f26d40992a0 con 0x7f26e4100e20 2024-02-02T15:25:02.868 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-02-02T15:25:02.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.870+0000 7f26ea200700 1 -- 172.21.15.1:0/2268060501 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f26c40a05a0 msgr2=0x7f26c40a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:02.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.870+0000 7f26ea200700 1 --2- 172.21.15.1:0/2268060501 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f26c40a05a0 0x7f26c40a2a60 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f26cc00b2e0 tx=0x7f26cc005fd0 comp rx=0 tx=0).stop 2024-02-02T15:25:02.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.870+0000 7f26ea200700 1 -- 172.21.15.1:0/2268060501 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f26e4100e20 msgr2=0x7f26e4074af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:02.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.870+0000 7f26ea200700 1 --2- 172.21.15.1:0/2268060501 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f26e4100e20 0x7f26e4074af0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f26d400e9a0 tx=0x7f26d400ed60 comp rx=0 tx=0).stop 2024-02-02T15:25:02.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.870+0000 7f26ea200700 1 -- 172.21.15.1:0/2268060501 shutdown_connections 2024-02-02T15:25:02.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.870+0000 7f26ea200700 1 --2- 172.21.15.1:0/2268060501 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f26e4102930 0x7f26e4073670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:02.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.870+0000 7f26ea200700 1 --2- 172.21.15.1:0/2268060501 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f26e4100e20 0x7f26e4074af0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:02.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.870+0000 7f26ea200700 1 --2- 172.21.15.1:0/2268060501 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f26c40a05a0 0x7f26c40a2a60 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:02.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.870+0000 7f26ea200700 1 --2- 172.21.15.1:0/2268060501 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f26e4101f70 0x7f26e4073130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:02.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.870+0000 7f26ea200700 1 -- 172.21.15.1:0/2268060501 >> 172.21.15.1:0/2268060501 conn(0x7f26e40fc390 msgr2=0x7f26e40fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:02.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.870+0000 7f26ea200700 1 -- 172.21.15.1:0/2268060501 shutdown_connections 2024-02-02T15:25:02.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:02.870+0000 7f26ea200700 1 -- 172.21.15.1:0/2268060501 wait complete. 2024-02-02T15:25:02.888 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-02T15:25:03.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.088+0000 7f9433e00700 1 -- 172.21.15.1:0/3190832477 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f942c108ce0 msgr2=0x7f942c10b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:03.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.088+0000 7f9433e00700 1 --2- 172.21.15.1:0/3190832477 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f942c108ce0 0x7f942c10b170 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f942400b0a0 tx=0x7f942400b3b0 comp rx=0 tx=0).stop 2024-02-02T15:25:03.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.088+0000 7f9433e00700 1 -- 172.21.15.1:0/3190832477 shutdown_connections 2024-02-02T15:25:03.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.088+0000 7f9433e00700 1 --2- 172.21.15.1:0/3190832477 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f942c108ce0 0x7f942c10b170 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:03.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.088+0000 7f9433e00700 1 --2- 172.21.15.1:0/3190832477 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f942c074d80 0x7f942c0731d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:03.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.088+0000 7f9433e00700 1 --2- 172.21.15.1:0/3190832477 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f942c0737a0 0x7f942c073c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:03.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.088+0000 7f9433e00700 1 -- 172.21.15.1:0/3190832477 >> 172.21.15.1:0/3190832477 conn(0x7f942c0fc390 msgr2=0x7f942c0fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:03.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.088+0000 7f9433e00700 1 -- 172.21.15.1:0/3190832477 shutdown_connections 2024-02-02T15:25:03.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.088+0000 7f9433e00700 1 -- 172.21.15.1:0/3190832477 wait complete. 2024-02-02T15:25:03.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.089+0000 7f9433e00700 1 Processor -- start 2024-02-02T15:25:03.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.089+0000 7f9433e00700 1 -- start start 2024-02-02T15:25:03.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.090+0000 7f9433e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f942c0737a0 0x7f942c198df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:03.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.090+0000 7f9433e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f942c074d80 0x7f942c199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:03.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.090+0000 7f9433e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f942c108ce0 0x7f942c1a03a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:03.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.090+0000 7f9433e00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942c1a0970 con 0x7f942c0737a0 2024-02-02T15:25:03.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.090+0000 7f9433e00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942c1a0ab0 con 0x7f942c074d80 2024-02-02T15:25:03.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.090+0000 7f9433e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942c1a0c40 con 0x7f942c108ce0 2024-02-02T15:25:03.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.090+0000 7f9431400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f942c0737a0 0x7f942c198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:03.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.090+0000 7f9431400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f942c0737a0 0x7f942c198df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43262/0 (socket says 172.21.15.1:43262) 2024-02-02T15:25:03.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.090+0000 7f9431400700 1 -- 172.21.15.1:0/3263919851 learned_addr learned my addr 172.21.15.1:0/3263919851 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:25:03.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.090+0000 7f9430a00700 1 --2- 172.21.15.1:0/3263919851 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f942c074d80 0x7f942c199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:03.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.090+0000 7f9431e00700 1 --2- 172.21.15.1:0/3263919851 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f942c108ce0 0x7f942c1a03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:03.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.090+0000 7f9431400700 1 -- 172.21.15.1:0/3263919851 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f942c108ce0 msgr2=0x7f942c1a03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:03.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.090+0000 7f9431400700 1 --2- 172.21.15.1:0/3263919851 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f942c108ce0 0x7f942c1a03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:03.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.090+0000 7f9431400700 1 -- 172.21.15.1:0/3263919851 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f942c074d80 msgr2=0x7f942c199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:03.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.090+0000 7f9431400700 1 --2- 172.21.15.1:0/3263919851 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f942c074d80 0x7f942c199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:03.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.090+0000 7f9431400700 1 -- 172.21.15.1:0/3263919851 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9424009d00 con 0x7f942c0737a0 2024-02-02T15:25:03.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.091+0000 7f9431400700 1 --2- 172.21.15.1:0/3263919851 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f942c0737a0 0x7f942c198df0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f941c00e9a0 tx=0x7f941c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:03.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.091+0000 7f942a000700 1 -- 172.21.15.1:0/3263919851 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f941c009790 con 0x7f942c0737a0 2024-02-02T15:25:03.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.091+0000 7f942a000700 1 -- 172.21.15.1:0/3263919851 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f941c010430 con 0x7f942c0737a0 2024-02-02T15:25:03.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.091+0000 7f942a000700 1 -- 172.21.15.1:0/3263919851 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f941c018680 con 0x7f942c0737a0 2024-02-02T15:25:03.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.091+0000 7f9433e00700 1 -- 172.21.15.1:0/3263919851 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f942c1a0f20 con 0x7f942c0737a0 2024-02-02T15:25:03.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.091+0000 7f9433e00700 1 -- 172.21.15.1:0/3263919851 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f942c1a1340 con 0x7f942c0737a0 2024-02-02T15:25:03.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.092+0000 7f9433e00700 1 -- 172.21.15.1:0/3263919851 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f942c107390 con 0x7f942c0737a0 2024-02-02T15:25:03.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.094+0000 7f942a000700 1 -- 172.21.15.1:0/3263919851 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) v1 ==== 135069+0+0 (secure 0 0 0) 0x7f941c0105a0 con 0x7f942c0737a0 2024-02-02T15:25:03.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.096+0000 7f942a000700 1 --2- 172.21.15.1:0/3263919851 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f940c0a05a0 0x7f940c0a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:03.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.096+0000 7f942a000700 1 -- 172.21.15.1:0/3263919851 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f941c014070 con 0x7f942c0737a0 2024-02-02T15:25:03.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.096+0000 7f9430a00700 1 --2- 172.21.15.1:0/3263919851 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f940c0a05a0 0x7f940c0a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:03.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.097+0000 7f9430a00700 1 --2- 172.21.15.1:0/3263919851 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f940c0a05a0 0x7f940c0a2a60 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f942c074aa0 tx=0x7f9414009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:03.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.100+0000 7f942a000700 1 -- 172.21.15.1:0/3263919851 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f941c052110 con 0x7f942c0737a0 2024-02-02T15:25:03.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.314+0000 7f9433e00700 1 -- 172.21.15.1:0/3263919851 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f942c061a60 con 0x7f942c0737a0 2024-02-02T15:25:03.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.315+0000 7f942a000700 1 -- 172.21.15.1:0/3263919851 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) v1 ==== 104+0+11 (secure 0 0 0) 0x7f941c095680 con 0x7f942c0737a0 2024-02-02T15:25:03.315 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-02-02T15:25:03.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.317+0000 7f9433e00700 1 -- 172.21.15.1:0/3263919851 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f940c0a05a0 msgr2=0x7f940c0a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:03.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.317+0000 7f9433e00700 1 --2- 172.21.15.1:0/3263919851 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f940c0a05a0 0x7f940c0a2a60 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f942c074aa0 tx=0x7f9414009260 comp rx=0 tx=0).stop 2024-02-02T15:25:03.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.317+0000 7f9433e00700 1 -- 172.21.15.1:0/3263919851 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f942c0737a0 msgr2=0x7f942c198df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:03.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.317+0000 7f9433e00700 1 --2- 172.21.15.1:0/3263919851 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f942c0737a0 0x7f942c198df0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f941c00e9a0 tx=0x7f941c00ecb0 comp rx=0 tx=0).stop 2024-02-02T15:25:03.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.317+0000 7f9433e00700 1 -- 172.21.15.1:0/3263919851 shutdown_connections 2024-02-02T15:25:03.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.317+0000 7f9433e00700 1 --2- 172.21.15.1:0/3263919851 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f940c0a05a0 0x7f940c0a2a60 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:03.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.317+0000 7f9433e00700 1 --2- 172.21.15.1:0/3263919851 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f942c108ce0 0x7f942c1a03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:03.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.317+0000 7f9433e00700 1 --2- 172.21.15.1:0/3263919851 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f942c0737a0 0x7f942c198df0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:03.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.317+0000 7f9433e00700 1 --2- 172.21.15.1:0/3263919851 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f942c074d80 0x7f942c199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:03.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.317+0000 7f9433e00700 1 -- 172.21.15.1:0/3263919851 >> 172.21.15.1:0/3263919851 conn(0x7f942c0fc390 msgr2=0x7f942c10af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:03.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.317+0000 7f9433e00700 1 -- 172.21.15.1:0/3263919851 shutdown_connections 2024-02-02T15:25:03.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.317+0000 7f9433e00700 1 -- 172.21.15.1:0/3263919851 wait complete. 2024-02-02T15:25:03.335 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-02T15:25:03.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.536+0000 7f50bbc00700 1 -- 172.21.15.1:0/2225330485 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f50b4101f80 msgr2=0x7f50b4102400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:03.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.536+0000 7f50bbc00700 1 --2- 172.21.15.1:0/2225330485 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f50b4101f80 0x7f50b4102400 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f509c009990 tx=0x7f509c009ca0 comp rx=0 tx=0).stop 2024-02-02T15:25:03.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.536+0000 7f50bbc00700 1 -- 172.21.15.1:0/2225330485 shutdown_connections 2024-02-02T15:25:03.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.536+0000 7f50bbc00700 1 --2- 172.21.15.1:0/2225330485 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50b4102940 0x7f50b4109260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:03.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.536+0000 7f50bbc00700 1 --2- 172.21.15.1:0/2225330485 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50b4100e30 0x7f50b4101250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:03.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.536+0000 7f50bbc00700 1 --2- 172.21.15.1:0/2225330485 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f50b4101f80 0x7f50b4102400 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:03.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.536+0000 7f50bbc00700 1 -- 172.21.15.1:0/2225330485 >> 172.21.15.1:0/2225330485 conn(0x7f50b40fc3a0 msgr2=0x7f50b40fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:03.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.536+0000 7f50bbc00700 1 -- 172.21.15.1:0/2225330485 shutdown_connections 2024-02-02T15:25:03.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.536+0000 7f50bbc00700 1 -- 172.21.15.1:0/2225330485 wait complete. 2024-02-02T15:25:03.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.537+0000 7f50bbc00700 1 Processor -- start 2024-02-02T15:25:03.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.537+0000 7f50bbc00700 1 -- start start 2024-02-02T15:25:03.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.538+0000 7f50bbc00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f50b4100e30 0x7f50b4198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:03.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.538+0000 7f50bbc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50b4101f80 0x7f50b4199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:03.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.538+0000 7f50bbc00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50b4102940 0x7f50b41a02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:03.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.538+0000 7f50bbc00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50b41a08c0 con 0x7f50b4100e30 2024-02-02T15:25:03.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.538+0000 7f50bbc00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50b41a0a00 con 0x7f50b4102940 2024-02-02T15:25:03.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.538+0000 7f50bbc00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50b41a0b90 con 0x7f50b4101f80 2024-02-02T15:25:03.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.538+0000 7f50b9200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f50b4100e30 0x7f50b4198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:03.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.538+0000 7f50b9200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f50b4100e30 0x7f50b4198d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43274/0 (socket says 172.21.15.1:43274) 2024-02-02T15:25:03.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.538+0000 7f50b9200700 1 -- 172.21.15.1:0/1760114764 learned_addr learned my addr 172.21.15.1:0/1760114764 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:25:03.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.538+0000 7f50b3e00700 1 --2- 172.21.15.1:0/1760114764 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50b4101f80 0x7f50b4199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:03.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.538+0000 7f50b9c00700 1 --2- 172.21.15.1:0/1760114764 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50b4102940 0x7f50b41a02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:03.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.539+0000 7f50b9200700 1 -- 172.21.15.1:0/1760114764 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50b4101f80 msgr2=0x7f50b4199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:03.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.539+0000 7f50b9200700 1 --2- 172.21.15.1:0/1760114764 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50b4101f80 0x7f50b4199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:03.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.539+0000 7f50b9200700 1 -- 172.21.15.1:0/1760114764 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50b4102940 msgr2=0x7f50b41a02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:03.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.539+0000 7f50b9200700 1 --2- 172.21.15.1:0/1760114764 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50b4102940 0x7f50b41a02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:03.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.539+0000 7f50b9200700 1 -- 172.21.15.1:0/1760114764 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f509c0095f0 con 0x7f50b4100e30 2024-02-02T15:25:03.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.539+0000 7f50b3e00700 1 --2- 172.21.15.1:0/1760114764 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50b4101f80 0x7f50b4199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:25:03.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.539+0000 7f50b9c00700 1 --2- 172.21.15.1:0/1760114764 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50b4102940 0x7f50b41a02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:25:03.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.539+0000 7f50b9200700 1 --2- 172.21.15.1:0/1760114764 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f50b4100e30 0x7f50b4198d40 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f50a400e9a0 tx=0x7f50a400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:03.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.539+0000 7f50b1600700 1 -- 172.21.15.1:0/1760114764 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a4009790 con 0x7f50b4100e30 2024-02-02T15:25:03.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.539+0000 7f50b1600700 1 -- 172.21.15.1:0/1760114764 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f50a4010430 con 0x7f50b4100e30 2024-02-02T15:25:03.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.539+0000 7f50b1600700 1 -- 172.21.15.1:0/1760114764 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a4018680 con 0x7f50b4100e30 2024-02-02T15:25:03.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.539+0000 7f50bbc00700 1 -- 172.21.15.1:0/1760114764 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50b41a0e70 con 0x7f50b4100e30 2024-02-02T15:25:03.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.539+0000 7f50bbc00700 1 -- 172.21.15.1:0/1760114764 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50b41a1240 con 0x7f50b4100e30 2024-02-02T15:25:03.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.540+0000 7f50bbc00700 1 -- 172.21.15.1:0/1760114764 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50b4193450 con 0x7f50b4100e30 2024-02-02T15:25:03.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.542+0000 7f50b1600700 1 -- 172.21.15.1:0/1760114764 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) v1 ==== 135069+0+0 (secure 0 0 0) 0x7f50a40105a0 con 0x7f50b4100e30 2024-02-02T15:25:03.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.544+0000 7f50b1600700 1 --2- 172.21.15.1:0/1760114764 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f50940a0520 0x7f50940a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:03.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.544+0000 7f50b1600700 1 -- 172.21.15.1:0/1760114764 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f50a4014070 con 0x7f50b4100e30 2024-02-02T15:25:03.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.544+0000 7f50b3e00700 1 --2- 172.21.15.1:0/1760114764 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f50940a0520 0x7f50940a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:03.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.545+0000 7f50b3e00700 1 --2- 172.21.15.1:0/1760114764 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f50940a0520 0x7f50940a29e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f509c00b2e0 tx=0x7f509c005960 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:03.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.548+0000 7f50b1600700 1 -- 172.21.15.1:0/1760114764 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f50a4052110 con 0x7f50b4100e30 2024-02-02T15:25:03.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:03 smithi071 ceph-mon[44707]: pgmap v242: 129 pgs: 129 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 818 B/s wr, 0 op/s 2024-02-02T15:25:03.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:03 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/400144641' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-02T15:25:03.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:03 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/2268060501' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-02T15:25:03.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:03 smithi142 ceph-mon[45728]: pgmap v242: 129 pgs: 129 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 818 B/s wr, 0 op/s 2024-02-02T15:25:03.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:03 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/400144641' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-02T15:25:03.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:03 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/2268060501' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-02T15:25:03.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:03 smithi001 ceph-mon[35470]: pgmap v242: 129 pgs: 129 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 818 B/s wr, 0 op/s 2024-02-02T15:25:03.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:03 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/400144641' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-02T15:25:03.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:03 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/2268060501' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-02T15:25:03.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.760+0000 7f50bbc00700 1 -- 172.21.15.1:0/1760114764 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f50b40619f0 con 0x7f50b4100e30 2024-02-02T15:25:03.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.761+0000 7f50b1600700 1 -- 172.21.15.1:0/1760114764 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) v1 ==== 100+0+11 (secure 0 0 0) 0x7f50a40956a0 con 0x7f50b4100e30 2024-02-02T15:25:03.761 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 64 2024-02-02T15:25:03.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.763+0000 7f50bbc00700 1 -- 172.21.15.1:0/1760114764 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f50940a0520 msgr2=0x7f50940a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:03.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.763+0000 7f50bbc00700 1 --2- 172.21.15.1:0/1760114764 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f50940a0520 0x7f50940a29e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f509c00b2e0 tx=0x7f509c005960 comp rx=0 tx=0).stop 2024-02-02T15:25:03.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.763+0000 7f50bbc00700 1 -- 172.21.15.1:0/1760114764 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f50b4100e30 msgr2=0x7f50b4198d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:03.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.763+0000 7f50bbc00700 1 --2- 172.21.15.1:0/1760114764 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f50b4100e30 0x7f50b4198d40 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f50a400e9a0 tx=0x7f50a400ecb0 comp rx=0 tx=0).stop 2024-02-02T15:25:03.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.763+0000 7f50bbc00700 1 -- 172.21.15.1:0/1760114764 shutdown_connections 2024-02-02T15:25:03.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.763+0000 7f50bbc00700 1 --2- 172.21.15.1:0/1760114764 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50b4102940 0x7f50b41a02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:03.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.763+0000 7f50bbc00700 1 --2- 172.21.15.1:0/1760114764 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f50b4100e30 0x7f50b4198d40 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:03.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.763+0000 7f50bbc00700 1 --2- 172.21.15.1:0/1760114764 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f50940a0520 0x7f50940a29e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:03.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.763+0000 7f50bbc00700 1 --2- 172.21.15.1:0/1760114764 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50b4101f80 0x7f50b4199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:03.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.763+0000 7f50bbc00700 1 -- 172.21.15.1:0/1760114764 >> 172.21.15.1:0/1760114764 conn(0x7f50b40fc3a0 msgr2=0x7f50b4107900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:03.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.763+0000 7f50bbc00700 1 -- 172.21.15.1:0/1760114764 shutdown_connections 2024-02-02T15:25:03.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.764+0000 7f50bbc00700 1 -- 172.21.15.1:0/1760114764 wait complete. 2024-02-02T15:25:03.782 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-02-02T15:25:03.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.984+0000 7fdb0de00700 1 -- 172.21.15.1:0/388121200 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb08102940 msgr2=0x7fdb08109260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:03.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.984+0000 7fdb0de00700 1 --2- 172.21.15.1:0/388121200 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb08102940 0x7fdb08109260 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fdb0000b0a0 tx=0x7fdb0000b3b0 comp rx=0 tx=0).stop 2024-02-02T15:25:03.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.984+0000 7fdb0de00700 1 -- 172.21.15.1:0/388121200 shutdown_connections 2024-02-02T15:25:03.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.984+0000 7fdb0de00700 1 --2- 172.21.15.1:0/388121200 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb08102940 0x7fdb08109260 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:03.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.984+0000 7fdb0de00700 1 --2- 172.21.15.1:0/388121200 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb08100e30 0x7fdb08101250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:03.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.984+0000 7fdb0de00700 1 --2- 172.21.15.1:0/388121200 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdb08101f80 0x7fdb08102400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:03.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.984+0000 7fdb0de00700 1 -- 172.21.15.1:0/388121200 >> 172.21.15.1:0/388121200 conn(0x7fdb080fc3a0 msgr2=0x7fdb080fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:03.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.985+0000 7fdb0de00700 1 -- 172.21.15.1:0/388121200 shutdown_connections 2024-02-02T15:25:03.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.985+0000 7fdb0de00700 1 -- 172.21.15.1:0/388121200 wait complete. 2024-02-02T15:25:03.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.985+0000 7fdb0de00700 1 Processor -- start 2024-02-02T15:25:03.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.986+0000 7fdb0de00700 1 -- start start 2024-02-02T15:25:03.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.986+0000 7fdb0de00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb08100e30 0x7fdb08198e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:03.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.986+0000 7fdb0de00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb08101f80 0x7fdb081993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:03.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.986+0000 7fdb0de00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdb08102940 0x7fdb081a0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:03.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.986+0000 7fdb0de00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb081a0a00 con 0x7fdb08101f80 2024-02-02T15:25:03.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.986+0000 7fdb0de00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb081a0b40 con 0x7fdb08100e30 2024-02-02T15:25:03.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.986+0000 7fdb0de00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb081a0cd0 con 0x7fdb08102940 2024-02-02T15:25:03.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.986+0000 7fdb06000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb08101f80 0x7fdb081993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:03.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.986+0000 7fdb06000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb08101f80 0x7fdb081993c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43298/0 (socket says 172.21.15.1:43298) 2024-02-02T15:25:03.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.986+0000 7fdb06000700 1 -- 172.21.15.1:0/1714225429 learned_addr learned my addr 172.21.15.1:0/1714225429 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:25:03.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.986+0000 7fdb06a00700 1 --2- 172.21.15.1:0/1714225429 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb08100e30 0x7fdb08198e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:03.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.986+0000 7fdb07400700 1 --2- 172.21.15.1:0/1714225429 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdb08102940 0x7fdb081a0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:03.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.987+0000 7fdb06000700 1 -- 172.21.15.1:0/1714225429 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdb08102940 msgr2=0x7fdb081a0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:03.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.987+0000 7fdb06000700 1 --2- 172.21.15.1:0/1714225429 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdb08102940 0x7fdb081a0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:03.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.987+0000 7fdb06000700 1 -- 172.21.15.1:0/1714225429 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb08100e30 msgr2=0x7fdb08198e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:03.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.987+0000 7fdb06000700 1 --2- 172.21.15.1:0/1714225429 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb08100e30 0x7fdb08198e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:03.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.987+0000 7fdb06000700 1 -- 172.21.15.1:0/1714225429 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb00009d00 con 0x7fdb08101f80 2024-02-02T15:25:03.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.987+0000 7fdb06a00700 1 --2- 172.21.15.1:0/1714225429 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb08100e30 0x7fdb08198e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:25:03.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.987+0000 7fdb06000700 1 --2- 172.21.15.1:0/1714225429 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb08101f80 0x7fdb081993c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fdaf000e9a0 tx=0x7fdaf000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:03.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.987+0000 7fdb0ca00700 1 -- 172.21.15.1:0/1714225429 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf0009790 con 0x7fdb08101f80 2024-02-02T15:25:03.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.987+0000 7fdb0ca00700 1 -- 172.21.15.1:0/1714225429 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdaf0010430 con 0x7fdb08101f80 2024-02-02T15:25:03.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.987+0000 7fdb0ca00700 1 -- 172.21.15.1:0/1714225429 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf0018680 con 0x7fdb08101f80 2024-02-02T15:25:03.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.987+0000 7fdb0de00700 1 -- 172.21.15.1:0/1714225429 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb081a0fb0 con 0x7fdb08101f80 2024-02-02T15:25:03.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.987+0000 7fdb0de00700 1 -- 172.21.15.1:0/1714225429 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb081a1430 con 0x7fdb08101f80 2024-02-02T15:25:03.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.988+0000 7fdb0de00700 1 -- 172.21.15.1:0/1714225429 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb0810c680 con 0x7fdb08101f80 2024-02-02T15:25:03.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.991+0000 7fdb0ca00700 1 -- 172.21.15.1:0/1714225429 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) v1 ==== 135069+0+0 (secure 0 0 0) 0x7fdaf00105a0 con 0x7fdb08101f80 2024-02-02T15:25:03.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.992+0000 7fdb0ca00700 1 --2- 172.21.15.1:0/1714225429 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fdae80a05a0 0x7fdae80a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:03.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.992+0000 7fdb0ca00700 1 -- 172.21.15.1:0/1714225429 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fdaf0014070 con 0x7fdb08101f80 2024-02-02T15:25:03.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.993+0000 7fdb06a00700 1 --2- 172.21.15.1:0/1714225429 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fdae80a05a0 0x7fdae80a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:03.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.993+0000 7fdb06a00700 1 --2- 172.21.15.1:0/1714225429 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fdae80a05a0 0x7fdae80a2a60 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fdaf8011490 tx=0x7fdaf8009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:03.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:03.994+0000 7fdb0ca00700 1 -- 172.21.15.1:0/1714225429 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7fdaf0095870 con 0x7fdb08101f80 2024-02-02T15:25:04.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.210+0000 7fdb0de00700 1 -- 172.21.15.1:0/1714225429 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fdb081a1a40 con 0x7fdb08101f80 2024-02-02T15:25:04.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.211+0000 7fdb0ca00700 1 -- 172.21.15.1:0/1714225429 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v84) v1 ==== 103+0+11 (secure 0 0 0) 0x7fdaf0026020 con 0x7fdb08101f80 2024-02-02T15:25:04.211 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-02-02T15:25:04.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.213+0000 7fdb0de00700 1 -- 172.21.15.1:0/1714225429 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fdae80a05a0 msgr2=0x7fdae80a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:04.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.213+0000 7fdb0de00700 1 --2- 172.21.15.1:0/1714225429 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fdae80a05a0 0x7fdae80a2a60 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fdaf8011490 tx=0x7fdaf8009260 comp rx=0 tx=0).stop 2024-02-02T15:25:04.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.213+0000 7fdb0de00700 1 -- 172.21.15.1:0/1714225429 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb08101f80 msgr2=0x7fdb081993c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:04.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.213+0000 7fdb0de00700 1 --2- 172.21.15.1:0/1714225429 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb08101f80 0x7fdb081993c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fdaf000e9a0 tx=0x7fdaf000ecb0 comp rx=0 tx=0).stop 2024-02-02T15:25:04.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.213+0000 7fdb0de00700 1 -- 172.21.15.1:0/1714225429 shutdown_connections 2024-02-02T15:25:04.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.213+0000 7fdb0de00700 1 --2- 172.21.15.1:0/1714225429 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fdae80a05a0 0x7fdae80a2a60 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:04.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.213+0000 7fdb0de00700 1 --2- 172.21.15.1:0/1714225429 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdb08102940 0x7fdb081a0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:04.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.213+0000 7fdb0de00700 1 --2- 172.21.15.1:0/1714225429 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb08100e30 0x7fdb08198e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:04.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.213+0000 7fdb0de00700 1 --2- 172.21.15.1:0/1714225429 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb08101f80 0x7fdb081993c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:04.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.213+0000 7fdb0de00700 1 -- 172.21.15.1:0/1714225429 >> 172.21.15.1:0/1714225429 conn(0x7fdb080fc3a0 msgr2=0x7fdb08107850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:04.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.213+0000 7fdb0de00700 1 -- 172.21.15.1:0/1714225429 shutdown_connections 2024-02-02T15:25:04.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.213+0000 7fdb0de00700 1 -- 172.21.15.1:0/1714225429 wait complete. 2024-02-02T15:25:04.233 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-02-02T15:25:04.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.430+0000 7fb30d600700 1 -- 172.21.15.1:0/897323042 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb308102990 msgr2=0x7fb308109240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:04.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.430+0000 7fb30d600700 1 --2- 172.21.15.1:0/897323042 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb308102990 0x7fb308109240 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fb300009fc0 tx=0x7fb30000b330 comp rx=0 tx=0).stop 2024-02-02T15:25:04.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.430+0000 7fb30d600700 1 -- 172.21.15.1:0/897323042 shutdown_connections 2024-02-02T15:25:04.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.430+0000 7fb30d600700 1 --2- 172.21.15.1:0/897323042 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb308102990 0x7fb308109240 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:04.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.430+0000 7fb30d600700 1 --2- 172.21.15.1:0/897323042 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb308100e00 0x7fb308101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:04.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.430+0000 7fb30d600700 1 --2- 172.21.15.1:0/897323042 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb308101ff0 0x7fb308102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:04.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.430+0000 7fb30d600700 1 -- 172.21.15.1:0/897323042 >> 172.21.15.1:0/897323042 conn(0x7fb3080fc370 msgr2=0x7fb3080fe7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:04.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.430+0000 7fb30d600700 1 -- 172.21.15.1:0/897323042 shutdown_connections 2024-02-02T15:25:04.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.430+0000 7fb30d600700 1 -- 172.21.15.1:0/897323042 wait complete. 2024-02-02T15:25:04.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.431+0000 7fb30d600700 1 Processor -- start 2024-02-02T15:25:04.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.431+0000 7fb30d600700 1 -- start start 2024-02-02T15:25:04.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.431+0000 7fb30d600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb308100e00 0x7fb308198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:04.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.431+0000 7fb30d600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb308101ff0 0x7fb3081993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:04.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.431+0000 7fb30d600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb308102990 0x7fb3081a0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:04.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.431+0000 7fb30d600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3081a0a10 con 0x7fb308101ff0 2024-02-02T15:25:04.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.431+0000 7fb30d600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3081a0b50 con 0x7fb308102990 2024-02-02T15:25:04.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.431+0000 7fb30d600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3081a0ce0 con 0x7fb308100e00 2024-02-02T15:25:04.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.432+0000 7fb306000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb308101ff0 0x7fb3081993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:04.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.432+0000 7fb306000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb308101ff0 0x7fb3081993d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43316/0 (socket says 172.21.15.1:43316) 2024-02-02T15:25:04.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.432+0000 7fb306000700 1 -- 172.21.15.1:0/1567245175 learned_addr learned my addr 172.21.15.1:0/1567245175 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:25:04.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.432+0000 7fb307400700 1 --2- 172.21.15.1:0/1567245175 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb308102990 0x7fb3081a0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:04.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.432+0000 7fb306a00700 1 --2- 172.21.15.1:0/1567245175 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb308100e00 0x7fb308198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:04.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.432+0000 7fb306000700 1 -- 172.21.15.1:0/1567245175 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb308100e00 msgr2=0x7fb308198e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:04.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.432+0000 7fb306000700 1 --2- 172.21.15.1:0/1567245175 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb308100e00 0x7fb308198e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:04.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.432+0000 7fb306000700 1 -- 172.21.15.1:0/1567245175 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb308102990 msgr2=0x7fb3081a0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:04.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.432+0000 7fb306000700 1 --2- 172.21.15.1:0/1567245175 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb308102990 0x7fb3081a0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:04.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.432+0000 7fb306000700 1 -- 172.21.15.1:0/1567245175 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb300009c40 con 0x7fb308101ff0 2024-02-02T15:25:04.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.432+0000 7fb306000700 1 --2- 172.21.15.1:0/1567245175 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb308101ff0 0x7fb3081993d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb2f000e9c0 tx=0x7fb2f000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:04.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.433+0000 7fb2ff400700 1 -- 172.21.15.1:0/1567245175 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f0004a30 con 0x7fb308101ff0 2024-02-02T15:25:04.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.433+0000 7fb2ff400700 1 -- 172.21.15.1:0/1567245175 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb2f000cc80 con 0x7fb308101ff0 2024-02-02T15:25:04.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.433+0000 7fb30d600700 1 -- 172.21.15.1:0/1567245175 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3081a0fc0 con 0x7fb308101ff0 2024-02-02T15:25:04.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.433+0000 7fb2ff400700 1 -- 172.21.15.1:0/1567245175 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f00054e0 con 0x7fb308101ff0 2024-02-02T15:25:04.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.433+0000 7fb30d600700 1 -- 172.21.15.1:0/1567245175 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3081a1440 con 0x7fb308101ff0 2024-02-02T15:25:04.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.434+0000 7fb30d600700 1 -- 172.21.15.1:0/1567245175 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb30810c670 con 0x7fb308101ff0 2024-02-02T15:25:04.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.436+0000 7fb2ff400700 1 -- 172.21.15.1:0/1567245175 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) v1 ==== 135069+0+0 (secure 0 0 0) 0x7fb2f0004b90 con 0x7fb308101ff0 2024-02-02T15:25:04.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.438+0000 7fb2ff400700 1 --2- 172.21.15.1:0/1567245175 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fb2e80a05a0 0x7fb2e80a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:04.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.438+0000 7fb2ff400700 1 -- 172.21.15.1:0/1567245175 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fb2f0014070 con 0x7fb308101ff0 2024-02-02T15:25:04.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.438+0000 7fb306a00700 1 --2- 172.21.15.1:0/1567245175 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fb2e80a05a0 0x7fb2e80a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:04.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.439+0000 7fb306a00700 1 --2- 172.21.15.1:0/1567245175 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fb2e80a05a0 0x7fb2e80a2a60 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb308101e50 tx=0x7fb2f8009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:04.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.442+0000 7fb2ff400700 1 -- 172.21.15.1:0/1567245175 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7fb2f0095570 con 0x7fb308101ff0 2024-02-02T15:25:04.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:04 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/3263919851' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-02T15:25:04.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:04 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/1760114764' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-02T15:25:04.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:04 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/1714225429' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-02T15:25:04.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.669+0000 7fb30d600700 1 -- 172.21.15.1:0/1567245175 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb30810c830 con 0x7fb308101ff0 2024-02-02T15:25:04.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.669+0000 7fb2ff400700 1 -- 172.21.15.1:0/1567245175 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v84) v1 ==== 81+0+579 (secure 0 0 0) 0x7fb2f0098b90 con 0x7fb308101ff0 2024-02-02T15:25:04.669 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/4092532060 2024-02-03T15:19:25.836171+0000 2024-02-02T15:25:04.669 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:6801/395498296 2024-02-03T15:19:25.836171+0000 2024-02-02T15:25:04.669 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:6800/395498296 2024-02-03T15:19:25.836171+0000 2024-02-02T15:25:04.669 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/1575373489 2024-02-03T15:19:25.836171+0000 2024-02-02T15:25:04.669 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/865983374 2024-02-03T15:19:01.431596+0000 2024-02-02T15:25:04.670 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/3165960227 2024-02-03T15:19:01.431596+0000 2024-02-02T15:25:04.670 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/2350830293 2024-02-03T15:19:01.431596+0000 2024-02-02T15:25:04.670 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/2959690250 2024-02-03T15:19:25.836171+0000 2024-02-02T15:25:04.670 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:6800/1162530732 2024-02-03T15:19:01.431596+0000 2024-02-02T15:25:04.670 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:6801/1162530732 2024-02-03T15:19:01.431596+0000 2024-02-02T15:25:04.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:04 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/3263919851' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-02T15:25:04.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:04 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/1760114764' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-02T15:25:04.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:04 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/1714225429' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-02T15:25:04.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.672+0000 7fb30d600700 1 -- 172.21.15.1:0/1567245175 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fb2e80a05a0 msgr2=0x7fb2e80a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:04.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.672+0000 7fb30d600700 1 --2- 172.21.15.1:0/1567245175 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fb2e80a05a0 0x7fb2e80a2a60 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb308101e50 tx=0x7fb2f8009260 comp rx=0 tx=0).stop 2024-02-02T15:25:04.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.672+0000 7fb30d600700 1 -- 172.21.15.1:0/1567245175 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb308101ff0 msgr2=0x7fb3081993d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:04.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.672+0000 7fb30d600700 1 --2- 172.21.15.1:0/1567245175 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb308101ff0 0x7fb3081993d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fb2f000e9c0 tx=0x7fb2f000ecd0 comp rx=0 tx=0).stop 2024-02-02T15:25:04.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.672+0000 7fb30d600700 1 -- 172.21.15.1:0/1567245175 shutdown_connections 2024-02-02T15:25:04.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.672+0000 7fb30d600700 1 --2- 172.21.15.1:0/1567245175 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb308102990 0x7fb3081a0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:04.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.672+0000 7fb30d600700 1 --2- 172.21.15.1:0/1567245175 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fb2e80a05a0 0x7fb2e80a2a60 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:04.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.672+0000 7fb30d600700 1 --2- 172.21.15.1:0/1567245175 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb308100e00 0x7fb308198e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:04.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.672+0000 7fb30d600700 1 --2- 172.21.15.1:0/1567245175 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb308101ff0 0x7fb3081993d0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:04.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.672+0000 7fb30d600700 1 -- 172.21.15.1:0/1567245175 >> 172.21.15.1:0/1567245175 conn(0x7fb3080fc370 msgr2=0x7fb308107890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:04.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.672+0000 7fb30d600700 1 -- 172.21.15.1:0/1567245175 shutdown_connections 2024-02-02T15:25:04.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.672+0000 7fb30d600700 1 -- 172.21.15.1:0/1567245175 wait complete. 2024-02-02T15:25:04.673 INFO:teuthology.orchestra.run.smithi001.stderr:listed 10 entries 2024-02-02T15:25:04.687 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:04 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/3263919851' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-02T15:25:04.687 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:04 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/1760114764' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-02T15:25:04.687 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:04 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/1714225429' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-02T15:25:04.689 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-02-02T15:25:04.690 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T15:25:04.733 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T15:25:04.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.970+0000 7f8928a00700 1 -- 172.21.15.1:0/3809018250 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8920101f30 msgr2=0x7f8920102390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:04.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.970+0000 7f8928a00700 1 --2- 172.21.15.1:0/3809018250 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8920101f30 0x7f8920102390 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f89080099c0 tx=0x7f8908009cd0 comp rx=0 tx=0).stop 2024-02-02T15:25:04.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.971+0000 7f8928a00700 1 -- 172.21.15.1:0/3809018250 shutdown_connections 2024-02-02T15:25:04.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.971+0000 7f8928a00700 1 --2- 172.21.15.1:0/3809018250 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f89201028d0 0x7f8920109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:04.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.971+0000 7f8928a00700 1 --2- 172.21.15.1:0/3809018250 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8920100d40 0x7f8920101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:04.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.971+0000 7f8928a00700 1 --2- 172.21.15.1:0/3809018250 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8920101f30 0x7f8920102390 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:04.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.971+0000 7f8928a00700 1 -- 172.21.15.1:0/3809018250 >> 172.21.15.1:0/3809018250 conn(0x7f89200fc290 msgr2=0x7f89200fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:04.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.971+0000 7f8928a00700 1 -- 172.21.15.1:0/3809018250 shutdown_connections 2024-02-02T15:25:04.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.971+0000 7f8928a00700 1 -- 172.21.15.1:0/3809018250 wait complete. 2024-02-02T15:25:04.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.972+0000 7f8928a00700 1 Processor -- start 2024-02-02T15:25:04.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.972+0000 7f8928a00700 1 -- start start 2024-02-02T15:25:04.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.972+0000 7f8928a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8920100d40 0x7f8920074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:04.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.972+0000 7f8928a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8920101f30 0x7f8920073130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:04.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.972+0000 7f8928a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89201028d0 0x7f8920073670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:04.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.972+0000 7f8928a00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8920073e10 con 0x7f8920100d40 2024-02-02T15:25:04.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.972+0000 7f8928a00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8920073f80 con 0x7f8920101f30 2024-02-02T15:25:04.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.972+0000 7f8928a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89201a0aa0 con 0x7f89201028d0 2024-02-02T15:25:04.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.972+0000 7f8926000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8920100d40 0x7f8920074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:04.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.972+0000 7f8926000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8920100d40 0x7f8920074af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43340/0 (socket says 172.21.15.1:43340) 2024-02-02T15:25:04.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.972+0000 7f8926000700 1 -- 172.21.15.1:0/2180963642 learned_addr learned my addr 172.21.15.1:0/2180963642 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:25:04.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.973+0000 7f8926a00700 1 --2- 172.21.15.1:0/2180963642 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89201028d0 0x7f8920073670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:04.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.973+0000 7f8925600700 1 --2- 172.21.15.1:0/2180963642 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8920101f30 0x7f8920073130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:04.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.973+0000 7f8926000700 1 -- 172.21.15.1:0/2180963642 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89201028d0 msgr2=0x7f8920073670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:04.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.973+0000 7f8926000700 1 --2- 172.21.15.1:0/2180963642 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89201028d0 0x7f8920073670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:04.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.973+0000 7f8926000700 1 -- 172.21.15.1:0/2180963642 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8920101f30 msgr2=0x7f8920073130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:04.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.973+0000 7f8926000700 1 --2- 172.21.15.1:0/2180963642 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8920101f30 0x7f8920073130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:04.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.973+0000 7f8926000700 1 -- 172.21.15.1:0/2180963642 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89080095f0 con 0x7f8920100d40 2024-02-02T15:25:04.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.973+0000 7f8926000700 1 --2- 172.21.15.1:0/2180963642 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8920100d40 0x7f8920074af0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f891000e970 tx=0x7f891000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:04.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.973+0000 7f891ea00700 1 -- 172.21.15.1:0/2180963642 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8910009790 con 0x7f8920100d40 2024-02-02T15:25:04.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.973+0000 7f891ea00700 1 -- 172.21.15.1:0/2180963642 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f891000c7e0 con 0x7f8920100d40 2024-02-02T15:25:04.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.973+0000 7f8928a00700 1 -- 172.21.15.1:0/2180963642 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89201a0d80 con 0x7f8920100d40 2024-02-02T15:25:04.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.973+0000 7f891ea00700 1 -- 172.21.15.1:0/2180963642 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8910003c70 con 0x7f8920100d40 2024-02-02T15:25:04.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.974+0000 7f8928a00700 1 -- 172.21.15.1:0/2180963642 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89201a1220 con 0x7f8920100d40 2024-02-02T15:25:04.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.974+0000 7f8928a00700 1 -- 172.21.15.1:0/2180963642 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8920107c70 con 0x7f8920100d40 2024-02-02T15:25:04.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.976+0000 7f891ea00700 1 -- 172.21.15.1:0/2180963642 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) v1 ==== 135069+0+0 (secure 0 0 0) 0x7f891000cc50 con 0x7f8920100d40 2024-02-02T15:25:04.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.978+0000 7f891ea00700 1 --2- 172.21.15.1:0/2180963642 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f89000a05a0 0x7f89000a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:04.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.979+0000 7f891ea00700 1 -- 172.21.15.1:0/2180963642 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f8910014070 con 0x7f8920100d40 2024-02-02T15:25:04.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.979+0000 7f8925600700 1 --2- 172.21.15.1:0/2180963642 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f89000a05a0 0x7f89000a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:04.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.979+0000 7f8925600700 1 --2- 172.21.15.1:0/2180963642 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f89000a05a0 0x7f89000a2a60 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f890800b580 tx=0x7f8908005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:04.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:04.982+0000 7f891ea00700 1 -- 172.21.15.1:0/2180963642 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f89100956a0 con 0x7f8920100d40 2024-02-02T15:25:05.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.199+0000 7f8928a00700 1 -- 172.21.15.1:0/2180963642 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f892002d090 con 0x7f8920100d40 2024-02-02T15:25:05.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.200+0000 7f891ea00700 1 -- 172.21.15.1:0/2180963642 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+23841 (secure 0 0 0) 0x7f8910003dd0 con 0x7f8920100d40 2024-02-02T15:25:05.201 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:25:05.201 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":84,"fsid":"371a409c-c1de-11ee-95b6-87774f69a715","created":"2024-02-02T15:18:26.746506+0000","modified":"2024-02-02T15:24:52.310888+0000","last_up_change":"2024-02-02T15:24:08.800951+0000","last_in_change":"2024-02-02T15:23:52.784280+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T15:21:22.471754+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-02T15:24:46.195779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-02T15:24:47.226403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-02T15:24:50.300462+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"73e8a07f-3560-4c19-85ed-030d4d8f0f23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6803","nonce":2591906423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6805","nonce":2591906423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6809","nonce":2591906423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6807","nonce":2591906423}]},"public_addr":"172.21.15.1:6803/2591906423","cluster_addr":"172.21.15.1:6805/2591906423","heartbeat_back_addr":"172.21.15.1:6809/2591906423","heartbeat_front_addr":"172.21.15.1:6807/2591906423","state":["exists","up"]},{"osd":1,"uuid":"e53cffec-0ff9-4ed9-8dcf-ff47d04db2bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6801","nonce":747581719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6803","nonce":747581719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6807","nonce":747581719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6805","nonce":747581719}]},"public_addr":"172.21.15.71:6801/747581719","cluster_addr":"172.21.15.71:6803/747581719","heartbeat_back_addr":"172.21.15.71:6807/747581719","heartbeat_front_addr":"172.21.15.71:6805/747581719","state":["exists","up"]},{"osd":2,"uuid":"856c2347-294c-482b-8595-ad77f4792db4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6801","nonce":377511414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6803","nonce":377511414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6807","nonce":377511414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6805","nonce":377511414}]},"public_addr":"172.21.15.142:6801/377511414","cluster_addr":"172.21.15.142:6803/377511414","heartbeat_back_addr":"172.21.15.142:6807/377511414","heartbeat_front_addr":"172.21.15.142:6805/377511414","state":["exists","up"]},{"osd":3,"uuid":"4b946ad2-2da9-4e0b-9d61-d039d8669a42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6811","nonce":1086124477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6813","nonce":1086124477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6817","nonce":1086124477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6815","nonce":1086124477}]},"public_addr":"172.21.15.1:6811/1086124477","cluster_addr":"172.21.15.1:6813/1086124477","heartbeat_back_addr":"172.21.15.1:6817/1086124477","heartbeat_front_addr":"172.21.15.1:6815/1086124477","state":["exists","up"]},{"osd":4,"uuid":"43e36c43-47a0-4634-a45b-1da14a0e19d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6809","nonce":3095380305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6811","nonce":3095380305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6815","nonce":3095380305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6813","nonce":3095380305}]},"public_addr":"172.21.15.71:6809/3095380305","cluster_addr":"172.21.15.71:6811/3095380305","heartbeat_back_addr":"172.21.15.71:6815/3095380305","heartbeat_front_addr":"172.21.15.71:6813/3095380305","state":["exists","up"]},{"osd":5,"uuid":"eb48160e-ea7c-42f7-9477-be110bc8a34e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6809","nonce":4195149262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6811","nonce":4195149262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6815","nonce":4195149262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6813","nonce":4195149262}]},"public_addr":"172.21.15.142:6809/4195149262","cluster_addr":"172.21.15.142:6811/4195149262","heartbeat_back_addr":"172.21.15.142:6815/4195149262","heartbeat_front_addr":"172.21.15.142:6813/4195149262","state":["exists","up"]},{"osd":6,"uuid":"0ec4501a-5853-4ca6-b325-231f5b72335e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6819","nonce":1560153428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6821","nonce":1560153428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6825","nonce":1560153428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6823","nonce":1560153428}]},"public_addr":"172.21.15.1:6819/1560153428","cluster_addr":"172.21.15.1:6821/1560153428","heartbeat_back_addr":"172.21.15.1:6825/1560153428","heartbeat_front_addr":"172.21.15.1:6823/1560153428","state":["exists","up"]},{"osd":7,"uuid":"029debe8-9cdf-44f6-9aea-cabd1d185f93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6817","nonce":4080955538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6819","nonce":4080955538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6823","nonce":4080955538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6821","nonce":4080955538}]},"public_addr":"172.21.15.71:6817/4080955538","cluster_addr":"172.21.15.71:6819/4080955538","heartbeat_back_addr":"172.21.15.71:6823/4080955538","heartbeat_front_addr":"172.21.15.71:6821/4080955538","state":["exists","up"]},{"osd":8,"uuid":"7c90896c-9955-4c1b-aaca-7531dd5457f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6817","nonce":3942038209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6819","nonce":3942038209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6823","nonce":3942038209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6821","nonce":3942038209}]},"public_addr":"172.21.15.142:6817/3942038209","cluster_addr":"172.21.15.142:6819/3942038209","heartbeat_back_addr":"172.21.15.142:6823/3942038209","heartbeat_front_addr":"172.21.15.142:6821/3942038209","state":["exists","up"]},{"osd":9,"uuid":"7e165e68-31a8-4598-a577-de3e90ae5c5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6827","nonce":2716544850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6829","nonce":2716544850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6833","nonce":2716544850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6831","nonce":2716544850}]},"public_addr":"172.21.15.1:6827/2716544850","cluster_addr":"172.21.15.1:6829/2716544850","heartbeat_back_addr":"172.21.15.1:6833/2716544850","heartbeat_front_addr":"172.21.15.1:6831/2716544850","state":["exists","up"]},{"osd":10,"uuid":"48cd5fdb-0887-444a-aab8-a1de24282bb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6825","nonce":84691841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6827","nonce":84691841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6831","nonce":84691841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6829","nonce":84691841}]},"public_addr":"172.21.15.71:6825/84691841","cluster_addr":"172.21.15.71:6827/84691841","heartbeat_back_addr":"172.21.15.71:6831/84691841","heartbeat_front_addr":"172.21.15.71:6829/84691841","state":["exists","up"]},{"osd":11,"uuid":"f011dd27-11b0-4166-adfe-76756533163a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6825","nonce":2290179934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6827","nonce":2290179934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6831","nonce":2290179934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6829","nonce":2290179934}]},"public_addr":"172.21.15.142:6825/2290179934","cluster_addr":"172.21.15.142:6827/2290179934","heartbeat_back_addr":"172.21.15.142:6831/2290179934","heartbeat_front_addr":"172.21.15.142:6829/2290179934","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:20:44.060692+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:01.903817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:19.368258+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:40.028037+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:57.513361+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:22:14.805944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:22:35.188439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:22:53.014412+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:23:10.507420+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:23:31.849765+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:23:50.070612+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:24:07.388625+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/4092532060":"2024-02-03T15:19:25.836171+0000","172.21.15.1:6801/395498296":"2024-02-03T15:19:25.836171+0000","172.21.15.1:6800/395498296":"2024-02-03T15:19:25.836171+0000","172.21.15.1:0/1575373489":"2024-02-03T15:19:25.836171+0000","172.21.15.1:0/865983374":"2024-02-03T15:19:01.431596+0000","172.21.15.1:0/3165960227":"2024-02-03T15:19:01.431596+0000","172.21.15.1:0/2350830293":"2024-02-03T15:19:01.431596+0000","172.21.15.1:0/2959690250":"2024-02-03T15:19:25.836171+0000","172.21.15.1:6800/1162530732":"2024-02-03T15:19:01.431596+0000","172.21.15.1:6801/1162530732":"2024-02-03T15:19:01.431596+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T15:25:05.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.203+0000 7f8928a00700 1 -- 172.21.15.1:0/2180963642 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f89000a05a0 msgr2=0x7f89000a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:05.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.203+0000 7f8928a00700 1 --2- 172.21.15.1:0/2180963642 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f89000a05a0 0x7f89000a2a60 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f890800b580 tx=0x7f8908005fb0 comp rx=0 tx=0).stop 2024-02-02T15:25:05.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.203+0000 7f8928a00700 1 -- 172.21.15.1:0/2180963642 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8920100d40 msgr2=0x7f8920074af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:05.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.203+0000 7f8928a00700 1 --2- 172.21.15.1:0/2180963642 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8920100d40 0x7f8920074af0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f891000e970 tx=0x7f891000ec80 comp rx=0 tx=0).stop 2024-02-02T15:25:05.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.203+0000 7f8928a00700 1 -- 172.21.15.1:0/2180963642 shutdown_connections 2024-02-02T15:25:05.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.203+0000 7f8928a00700 1 --2- 172.21.15.1:0/2180963642 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f89000a05a0 0x7f89000a2a60 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:05.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.203+0000 7f8928a00700 1 --2- 172.21.15.1:0/2180963642 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89201028d0 0x7f8920073670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:05.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.203+0000 7f8928a00700 1 --2- 172.21.15.1:0/2180963642 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8920100d40 0x7f8920074af0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:05.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.203+0000 7f8928a00700 1 --2- 172.21.15.1:0/2180963642 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8920101f30 0x7f8920073130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:05.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.203+0000 7f8928a00700 1 -- 172.21.15.1:0/2180963642 >> 172.21.15.1:0/2180963642 conn(0x7f89200fc290 msgr2=0x7f89200fe530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:05.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.203+0000 7f8928a00700 1 -- 172.21.15.1:0/2180963642 shutdown_connections 2024-02-02T15:25:05.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.203+0000 7f8928a00700 1 -- 172.21.15.1:0/2180963642 wait complete. 2024-02-02T15:25:05.223 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-02T15:21:22.471754+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-02T15:24:46.195779+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-02T15:24:47.226403+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-02-02T15:24:50.300462+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-02-02T15:25:05.223 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-02T15:25:05.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.423+0000 7f0968000700 1 -- 172.21.15.1:0/4168342248 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0960101f70 msgr2=0x7f09601023f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:05.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.423+0000 7f0968000700 1 --2- 172.21.15.1:0/4168342248 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0960101f70 0x7f09601023f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f09480098f0 tx=0x7f0948009c00 comp rx=0 tx=0).stop 2024-02-02T15:25:05.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.423+0000 7f0968000700 1 -- 172.21.15.1:0/4168342248 shutdown_connections 2024-02-02T15:25:05.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.423+0000 7f0968000700 1 --2- 172.21.15.1:0/4168342248 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0960102930 0x7f0960109250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:05.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.423+0000 7f0968000700 1 --2- 172.21.15.1:0/4168342248 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0960100e20 0x7f0960101240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:05.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.423+0000 7f0968000700 1 --2- 172.21.15.1:0/4168342248 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0960101f70 0x7f09601023f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:05.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.423+0000 7f0968000700 1 -- 172.21.15.1:0/4168342248 >> 172.21.15.1:0/4168342248 conn(0x7f09600fc390 msgr2=0x7f09600fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:05.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.423+0000 7f0968000700 1 -- 172.21.15.1:0/4168342248 shutdown_connections 2024-02-02T15:25:05.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.423+0000 7f0968000700 1 -- 172.21.15.1:0/4168342248 wait complete. 2024-02-02T15:25:05.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.425+0000 7f0968000700 1 Processor -- start 2024-02-02T15:25:05.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.425+0000 7f0968000700 1 -- start start 2024-02-02T15:25:05.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.425+0000 7f0968000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0960100e20 0x7f0960198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:05.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.425+0000 7f0968000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0960101f70 0x7f0960199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:05.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.425+0000 7f0968000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0960102930 0x7f09601a0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:05.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.425+0000 7f0968000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09601a08d0 con 0x7f0960102930 2024-02-02T15:25:05.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.425+0000 7f0968000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09601a0a10 con 0x7f0960100e20 2024-02-02T15:25:05.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.425+0000 7f0968000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09601a0ba0 con 0x7f0960101f70 2024-02-02T15:25:05.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.426+0000 7f0966000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0960102930 0x7f09601a0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:05.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.426+0000 7f0966000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0960102930 0x7f09601a0300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43360/0 (socket says 172.21.15.1:43360) 2024-02-02T15:25:05.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.426+0000 7f0966000700 1 -- 172.21.15.1:0/2115548995 learned_addr learned my addr 172.21.15.1:0/2115548995 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:25:05.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.426+0000 7f0965600700 1 --2- 172.21.15.1:0/2115548995 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0960100e20 0x7f0960198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:05.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.426+0000 7f0964c00700 1 --2- 172.21.15.1:0/2115548995 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0960101f70 0x7f0960199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:05.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.426+0000 7f0966000700 1 -- 172.21.15.1:0/2115548995 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0960101f70 msgr2=0x7f0960199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:05.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.426+0000 7f0966000700 1 --2- 172.21.15.1:0/2115548995 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0960101f70 0x7f0960199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:05.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.426+0000 7f0966000700 1 -- 172.21.15.1:0/2115548995 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0960100e20 msgr2=0x7f0960198d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:05.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.426+0000 7f0966000700 1 --2- 172.21.15.1:0/2115548995 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0960100e20 0x7f0960198d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:05.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.426+0000 7f0966000700 1 -- 172.21.15.1:0/2115548995 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09500095f0 con 0x7f0960102930 2024-02-02T15:25:05.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.426+0000 7f0965600700 1 --2- 172.21.15.1:0/2115548995 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0960100e20 0x7f0960198d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:25:05.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.426+0000 7f0966000700 1 --2- 172.21.15.1:0/2115548995 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0960102930 0x7f09601a0300 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0958007ee0 tx=0x7f095800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:05.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.427+0000 7f095e000700 1 -- 172.21.15.1:0/2115548995 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f095800dd20 con 0x7f0960102930 2024-02-02T15:25:05.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.427+0000 7f095e000700 1 -- 172.21.15.1:0/2115548995 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f095800f040 con 0x7f0960102930 2024-02-02T15:25:05.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.427+0000 7f095e000700 1 -- 172.21.15.1:0/2115548995 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09580048a0 con 0x7f0960102930 2024-02-02T15:25:05.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.427+0000 7f0968000700 1 -- 172.21.15.1:0/2115548995 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0948009520 con 0x7f0960102930 2024-02-02T15:25:05.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.427+0000 7f0968000700 1 -- 172.21.15.1:0/2115548995 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09601a11b0 con 0x7f0960102930 2024-02-02T15:25:05.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.428+0000 7f0968000700 1 -- 172.21.15.1:0/2115548995 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0960193460 con 0x7f0960102930 2024-02-02T15:25:05.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.430+0000 7f095e000700 1 -- 172.21.15.1:0/2115548995 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) v1 ==== 135069+0+0 (secure 0 0 0) 0x7f0958003a00 con 0x7f0960102930 2024-02-02T15:25:05.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.431+0000 7f095e000700 1 --2- 172.21.15.1:0/2115548995 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f09400a05a0 0x7f09400a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:05.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.431+0000 7f095e000700 1 -- 172.21.15.1:0/2115548995 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f09580cbbd0 con 0x7f0960102930 2024-02-02T15:25:05.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.431+0000 7f0965600700 1 --2- 172.21.15.1:0/2115548995 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f09400a05a0 0x7f09400a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:05.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.432+0000 7f0965600700 1 --2- 172.21.15.1:0/2115548995 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f09400a05a0 0x7f09400a2a60 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0950005fd0 tx=0x7f0950009310 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:05.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.433+0000 7f095e000700 1 -- 172.21.15.1:0/2115548995 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f0958095520 con 0x7f0960102930 2024-02-02T15:25:05.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:05 smithi071 ceph-mon[44707]: pgmap v243: 129 pgs: 129 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 88 B/s wr, 0 op/s 2024-02-02T15:25:05.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:05 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/1567245175' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-02T15:25:05.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:05 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/2180963642' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-02T15:25:05.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.646+0000 7f0968000700 1 -- 172.21.15.1:0/2115548995 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f096002cee0 con 0x7f0960102930 2024-02-02T15:25:05.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.646+0000 7f095e000700 1 -- 172.21.15.1:0/2115548995 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0958023020 con 0x7f0960102930 2024-02-02T15:25:05.646 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-02-02T15:25:05.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.649+0000 7f0968000700 1 -- 172.21.15.1:0/2115548995 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f09400a05a0 msgr2=0x7f09400a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:05.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.649+0000 7f0968000700 1 --2- 172.21.15.1:0/2115548995 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f09400a05a0 0x7f09400a2a60 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0950005fd0 tx=0x7f0950009310 comp rx=0 tx=0).stop 2024-02-02T15:25:05.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.649+0000 7f0968000700 1 -- 172.21.15.1:0/2115548995 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0960102930 msgr2=0x7f09601a0300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:05.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.649+0000 7f0968000700 1 --2- 172.21.15.1:0/2115548995 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0960102930 0x7f09601a0300 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0958007ee0 tx=0x7f095800d3b0 comp rx=0 tx=0).stop 2024-02-02T15:25:05.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.649+0000 7f0968000700 1 -- 172.21.15.1:0/2115548995 shutdown_connections 2024-02-02T15:25:05.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.649+0000 7f0968000700 1 --2- 172.21.15.1:0/2115548995 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0960102930 0x7f09601a0300 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:05.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.649+0000 7f0968000700 1 --2- 172.21.15.1:0/2115548995 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0960100e20 0x7f0960198d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:05.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.649+0000 7f0968000700 1 --2- 172.21.15.1:0/2115548995 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f09400a05a0 0x7f09400a2a60 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:05.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.649+0000 7f0968000700 1 --2- 172.21.15.1:0/2115548995 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0960101f70 0x7f0960199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:05.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.649+0000 7f0968000700 1 -- 172.21.15.1:0/2115548995 >> 172.21.15.1:0/2115548995 conn(0x7f09600fc390 msgr2=0x7f0960107920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:05.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.649+0000 7f0968000700 1 -- 172.21.15.1:0/2115548995 shutdown_connections 2024-02-02T15:25:05.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.649+0000 7f0968000700 1 -- 172.21.15.1:0/2115548995 wait complete. 2024-02-02T15:25:05.665 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:05 smithi001 ceph-mon[35470]: pgmap v243: 129 pgs: 129 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 88 B/s wr, 0 op/s 2024-02-02T15:25:05.665 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:05 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/1567245175' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-02T15:25:05.665 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:05 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/2180963642' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-02T15:25:05.667 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-02T15:25:05.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:05 smithi142 ceph-mon[45728]: pgmap v243: 129 pgs: 129 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 88 B/s wr, 0 op/s 2024-02-02T15:25:05.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:05 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/1567245175' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-02T15:25:05.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:05 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/2180963642' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-02T15:25:05.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.866+0000 7fb1c1e00700 1 -- 172.21.15.1:0/2063041084 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb1bc101f70 msgr2=0x7fb1bc1023f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:05.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.866+0000 7fb1c1e00700 1 --2- 172.21.15.1:0/2063041084 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb1bc101f70 0x7fb1bc1023f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fb1a4009990 tx=0x7fb1a4009ca0 comp rx=0 tx=0).stop 2024-02-02T15:25:05.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.866+0000 7fb1c1e00700 1 -- 172.21.15.1:0/2063041084 shutdown_connections 2024-02-02T15:25:05.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.866+0000 7fb1c1e00700 1 --2- 172.21.15.1:0/2063041084 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb1bc102930 0x7fb1bc109250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:05.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.866+0000 7fb1c1e00700 1 --2- 172.21.15.1:0/2063041084 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb1bc100e20 0x7fb1bc101240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:05.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.866+0000 7fb1c1e00700 1 --2- 172.21.15.1:0/2063041084 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb1bc101f70 0x7fb1bc1023f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:05.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.866+0000 7fb1c1e00700 1 -- 172.21.15.1:0/2063041084 >> 172.21.15.1:0/2063041084 conn(0x7fb1bc0fc390 msgr2=0x7fb1bc0fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:05.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.866+0000 7fb1c1e00700 1 -- 172.21.15.1:0/2063041084 shutdown_connections 2024-02-02T15:25:05.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.866+0000 7fb1c1e00700 1 -- 172.21.15.1:0/2063041084 wait complete. 2024-02-02T15:25:05.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.867+0000 7fb1c1e00700 1 Processor -- start 2024-02-02T15:25:05.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.867+0000 7fb1c1e00700 1 -- start start 2024-02-02T15:25:05.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.868+0000 7fb1c1e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb1bc100e20 0x7fb1bc198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:05.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.868+0000 7fb1c1e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb1bc101f70 0x7fb1bc1991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:05.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.868+0000 7fb1c1e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb1bc102930 0x7fb1bc1a0240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:05.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.868+0000 7fb1c1e00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1bc1a0810 con 0x7fb1bc101f70 2024-02-02T15:25:05.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.868+0000 7fb1c1e00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1bc1a0980 con 0x7fb1bc100e20 2024-02-02T15:25:05.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.868+0000 7fb1c1e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1bc1a0b40 con 0x7fb1bc102930 2024-02-02T15:25:05.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.868+0000 7fb1ba000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb1bc101f70 0x7fb1bc1991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:05.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.868+0000 7fb1baa00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb1bc100e20 0x7fb1bc198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:05.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.868+0000 7fb1baa00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb1bc100e20 0x7fb1bc198ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:45460/0 (socket says 172.21.15.1:45460) 2024-02-02T15:25:05.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.868+0000 7fb1baa00700 1 -- 172.21.15.1:0/856708753 learned_addr learned my addr 172.21.15.1:0/856708753 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:25:05.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.868+0000 7fb1bb400700 1 --2- 172.21.15.1:0/856708753 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb1bc102930 0x7fb1bc1a0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:05.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.868+0000 7fb1baa00700 1 -- 172.21.15.1:0/856708753 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb1bc102930 msgr2=0x7fb1bc1a0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:05.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.868+0000 7fb1baa00700 1 --2- 172.21.15.1:0/856708753 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb1bc102930 0x7fb1bc1a0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:05.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.868+0000 7fb1baa00700 1 -- 172.21.15.1:0/856708753 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb1bc101f70 msgr2=0x7fb1bc1991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:05.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.868+0000 7fb1baa00700 1 --2- 172.21.15.1:0/856708753 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb1bc101f70 0x7fb1bc1991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:05.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.868+0000 7fb1baa00700 1 -- 172.21.15.1:0/856708753 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1ac009520 con 0x7fb1bc100e20 2024-02-02T15:25:05.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.868+0000 7fb1ba000700 1 --2- 172.21.15.1:0/856708753 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb1bc101f70 0x7fb1bc1991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:25:05.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.869+0000 7fb1baa00700 1 --2- 172.21.15.1:0/856708753 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb1bc100e20 0x7fb1bc198ca0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac00eb30 tx=0x7fb1ac00ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:05.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.869+0000 7fb1c0a00700 1 -- 172.21.15.1:0/856708753 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1ac009880 con 0x7fb1bc100e20 2024-02-02T15:25:05.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.869+0000 7fb1c1e00700 1 -- 172.21.15.1:0/856708753 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1a40095f0 con 0x7fb1bc100e20 2024-02-02T15:25:05.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.869+0000 7fb1c0a00700 1 -- 172.21.15.1:0/856708753 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb1ac0044e0 con 0x7fb1bc100e20 2024-02-02T15:25:05.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.869+0000 7fb1c0a00700 1 -- 172.21.15.1:0/856708753 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1ac003d80 con 0x7fb1bc100e20 2024-02-02T15:25:05.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.869+0000 7fb1c1e00700 1 -- 172.21.15.1:0/856708753 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1bc1a1170 con 0x7fb1bc100e20 2024-02-02T15:25:05.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.870+0000 7fb1c1e00700 1 -- 172.21.15.1:0/856708753 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1bc10c620 con 0x7fb1bc100e20 2024-02-02T15:25:05.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.872+0000 7fb1c0a00700 1 -- 172.21.15.1:0/856708753 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 135069+0+0 (secure 0 0 0) 0x7fb1ac003f20 con 0x7fb1bc100e20 2024-02-02T15:25:05.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.873+0000 7fb1c0a00700 1 --2- 172.21.15.1:0/856708753 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fb19c0a0520 0x7fb19c0a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:05.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.873+0000 7fb1c0a00700 1 -- 172.21.15.1:0/856708753 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fb1ac014070 con 0x7fb1bc100e20 2024-02-02T15:25:05.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.873+0000 7fb1ba000700 1 --2- 172.21.15.1:0/856708753 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fb19c0a0520 0x7fb19c0a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:05.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.874+0000 7fb1ba000700 1 --2- 172.21.15.1:0/856708753 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fb19c0a0520 0x7fb19c0a29e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb1a400b620 tx=0x7fb1a4005960 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:05.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:05.876+0000 7fb1c0a00700 1 -- 172.21.15.1:0/856708753 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7fb1ac099570 con 0x7fb1bc100e20 2024-02-02T15:25:06.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.086+0000 7fb1c1e00700 1 -- 172.21.15.1:0/856708753 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb1bc1a1720 con 0x7fb1bc100e20 2024-02-02T15:25:06.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.087+0000 7fb1c0a00700 1 -- 172.21.15.1:0/856708753 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb1ac099100 con 0x7fb1bc100e20 2024-02-02T15:25:06.087 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-02-02T15:25:06.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.089+0000 7fb1c1e00700 1 -- 172.21.15.1:0/856708753 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fb19c0a0520 msgr2=0x7fb19c0a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:06.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.089+0000 7fb1c1e00700 1 --2- 172.21.15.1:0/856708753 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fb19c0a0520 0x7fb19c0a29e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fb1a400b620 tx=0x7fb1a4005960 comp rx=0 tx=0).stop 2024-02-02T15:25:06.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.089+0000 7fb1c1e00700 1 -- 172.21.15.1:0/856708753 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb1bc100e20 msgr2=0x7fb1bc198ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:06.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.089+0000 7fb1c1e00700 1 --2- 172.21.15.1:0/856708753 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb1bc100e20 0x7fb1bc198ca0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac00eb30 tx=0x7fb1ac00ee40 comp rx=0 tx=0).stop 2024-02-02T15:25:06.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.089+0000 7fb1c1e00700 1 -- 172.21.15.1:0/856708753 shutdown_connections 2024-02-02T15:25:06.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.089+0000 7fb1c1e00700 1 --2- 172.21.15.1:0/856708753 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fb19c0a0520 0x7fb19c0a29e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:06.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.089+0000 7fb1c1e00700 1 --2- 172.21.15.1:0/856708753 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb1bc102930 0x7fb1bc1a0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:06.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.089+0000 7fb1c1e00700 1 --2- 172.21.15.1:0/856708753 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb1bc100e20 0x7fb1bc198ca0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:06.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.089+0000 7fb1c1e00700 1 --2- 172.21.15.1:0/856708753 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb1bc101f70 0x7fb1bc1991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:06.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.089+0000 7fb1c1e00700 1 -- 172.21.15.1:0/856708753 >> 172.21.15.1:0/856708753 conn(0x7fb1bc0fc390 msgr2=0x7fb1bc0fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:06.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.090+0000 7fb1c1e00700 1 -- 172.21.15.1:0/856708753 shutdown_connections 2024-02-02T15:25:06.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.090+0000 7fb1c1e00700 1 -- 172.21.15.1:0/856708753 wait complete. 2024-02-02T15:25:06.108 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-02T15:25:06.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.310+0000 7f0f7d000700 1 -- 172.21.15.1:0/2364594413 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0f78100e20 msgr2=0x7f0f78101240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:06.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.310+0000 7f0f7d000700 1 --2- 172.21.15.1:0/2364594413 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0f78100e20 0x7f0f78101240 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0f68009990 tx=0x7f0f68009ca0 comp rx=0 tx=0).stop 2024-02-02T15:25:06.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.310+0000 7f0f7d000700 1 -- 172.21.15.1:0/2364594413 shutdown_connections 2024-02-02T15:25:06.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.310+0000 7f0f7d000700 1 --2- 172.21.15.1:0/2364594413 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0f78102930 0x7f0f78109250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:06.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.310+0000 7f0f7d000700 1 --2- 172.21.15.1:0/2364594413 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0f78100e20 0x7f0f78101240 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:06.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.310+0000 7f0f7d000700 1 --2- 172.21.15.1:0/2364594413 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f78101f70 0x7f0f781023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:06.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.310+0000 7f0f7d000700 1 -- 172.21.15.1:0/2364594413 >> 172.21.15.1:0/2364594413 conn(0x7f0f780fc390 msgr2=0x7f0f780fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:06.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.310+0000 7f0f7d000700 1 -- 172.21.15.1:0/2364594413 shutdown_connections 2024-02-02T15:25:06.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.310+0000 7f0f7d000700 1 -- 172.21.15.1:0/2364594413 wait complete. 2024-02-02T15:25:06.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.311+0000 7f0f7d000700 1 Processor -- start 2024-02-02T15:25:06.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.311+0000 7f0f7d000700 1 -- start start 2024-02-02T15:25:06.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.311+0000 7f0f7d000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0f78100e20 0x7f0f78074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:06.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.311+0000 7f0f7d000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0f78101f70 0x7f0f78073130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:06.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.311+0000 7f0f7d000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f78102930 0x7f0f78073670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:06.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.311+0000 7f0f7d000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f78073e10 con 0x7f0f78101f70 2024-02-02T15:25:06.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.311+0000 7f0f7d000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f78073f80 con 0x7f0f78100e20 2024-02-02T15:25:06.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.311+0000 7f0f7d000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f781a0c00 con 0x7f0f78102930 2024-02-02T15:25:06.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.312+0000 7f0f75e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0f78100e20 0x7f0f78074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:06.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.312+0000 7f0f75400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0f78101f70 0x7f0f78073130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:06.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.312+0000 7f0f75e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0f78100e20 0x7f0f78074af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:45478/0 (socket says 172.21.15.1:45478) 2024-02-02T15:25:06.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.312+0000 7f0f75400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0f78101f70 0x7f0f78073130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43384/0 (socket says 172.21.15.1:43384) 2024-02-02T15:25:06.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.312+0000 7f0f75e00700 1 -- 172.21.15.1:0/13321305 learned_addr learned my addr 172.21.15.1:0/13321305 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:25:06.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.312+0000 7f0f76800700 1 --2- 172.21.15.1:0/13321305 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f78102930 0x7f0f78073670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:06.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.312+0000 7f0f75400700 1 -- 172.21.15.1:0/13321305 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f78102930 msgr2=0x7f0f78073670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:06.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.312+0000 7f0f75400700 1 --2- 172.21.15.1:0/13321305 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f78102930 0x7f0f78073670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:06.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.312+0000 7f0f75400700 1 -- 172.21.15.1:0/13321305 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0f78100e20 msgr2=0x7f0f78074af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:06.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.312+0000 7f0f75400700 1 --2- 172.21.15.1:0/13321305 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0f78100e20 0x7f0f78074af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:06.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.312+0000 7f0f75400700 1 -- 172.21.15.1:0/13321305 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f60009520 con 0x7f0f78101f70 2024-02-02T15:25:06.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.312+0000 7f0f75e00700 1 --2- 172.21.15.1:0/13321305 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0f78100e20 0x7f0f78074af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:25:06.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.312+0000 7f0f75400700 1 --2- 172.21.15.1:0/13321305 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0f78101f70 0x7f0f78073130 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0f6000e9f0 tx=0x7f0f6000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:06.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.313+0000 7f0f6ea00700 1 -- 172.21.15.1:0/13321305 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f6000cb70 con 0x7f0f78101f70 2024-02-02T15:25:06.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.313+0000 7f0f6ea00700 1 -- 172.21.15.1:0/13321305 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f6000ee90 con 0x7f0f78101f70 2024-02-02T15:25:06.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.313+0000 7f0f6ea00700 1 -- 172.21.15.1:0/13321305 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f60003cf0 con 0x7f0f78101f70 2024-02-02T15:25:06.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.313+0000 7f0f7d000700 1 -- 172.21.15.1:0/13321305 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f680095f0 con 0x7f0f78101f70 2024-02-02T15:25:06.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.313+0000 7f0f7d000700 1 -- 172.21.15.1:0/13321305 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f781a11b0 con 0x7f0f78101f70 2024-02-02T15:25:06.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.314+0000 7f0f7d000700 1 -- 172.21.15.1:0/13321305 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f78108450 con 0x7f0f78101f70 2024-02-02T15:25:06.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.316+0000 7f0f6ea00700 1 -- 172.21.15.1:0/13321305 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) v1 ==== 135069+0+0 (secure 0 0 0) 0x7f0f6000ccd0 con 0x7f0f78101f70 2024-02-02T15:25:06.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.317+0000 7f0f6ea00700 1 --2- 172.21.15.1:0/13321305 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f0f580a05a0 0x7f0f580a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:06.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.317+0000 7f0f6ea00700 1 -- 172.21.15.1:0/13321305 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f0f60014070 con 0x7f0f78101f70 2024-02-02T15:25:06.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.317+0000 7f0f75e00700 1 --2- 172.21.15.1:0/13321305 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f0f580a05a0 0x7f0f580a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:06.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.318+0000 7f0f75e00700 1 --2- 172.21.15.1:0/13321305 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f0f580a05a0 0x7f0f580a2a60 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0f68006010 tx=0x7f0f6800b260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:06.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.321+0000 7f0f6ea00700 1 -- 172.21.15.1:0/13321305 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f0f60095760 con 0x7f0f78101f70 2024-02-02T15:25:06.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:06 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/2115548995' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-02T15:25:06.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:06 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/856708753' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-02T15:25:06.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.531+0000 7f0f7d000700 1 -- 172.21.15.1:0/13321305 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0f780619f0 con 0x7f0f78101f70 2024-02-02T15:25:06.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.532+0000 7f0f6ea00700 1 -- 172.21.15.1:0/13321305 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0f60098d80 con 0x7f0f78101f70 2024-02-02T15:25:06.532 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 64 2024-02-02T15:25:06.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.534+0000 7f0f7d000700 1 -- 172.21.15.1:0/13321305 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f0f580a05a0 msgr2=0x7f0f580a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:06.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.534+0000 7f0f7d000700 1 --2- 172.21.15.1:0/13321305 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f0f580a05a0 0x7f0f580a2a60 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0f68006010 tx=0x7f0f6800b260 comp rx=0 tx=0).stop 2024-02-02T15:25:06.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.534+0000 7f0f7d000700 1 -- 172.21.15.1:0/13321305 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0f78101f70 msgr2=0x7f0f78073130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:06.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.534+0000 7f0f7d000700 1 --2- 172.21.15.1:0/13321305 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0f78101f70 0x7f0f78073130 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0f6000e9f0 tx=0x7f0f6000ed00 comp rx=0 tx=0).stop 2024-02-02T15:25:06.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.534+0000 7f0f7d000700 1 -- 172.21.15.1:0/13321305 shutdown_connections 2024-02-02T15:25:06.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.534+0000 7f0f7d000700 1 --2- 172.21.15.1:0/13321305 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f0f580a05a0 0x7f0f580a2a60 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:06.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.534+0000 7f0f7d000700 1 --2- 172.21.15.1:0/13321305 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f78102930 0x7f0f78073670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:06.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.534+0000 7f0f7d000700 1 --2- 172.21.15.1:0/13321305 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0f78100e20 0x7f0f78074af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:06.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.534+0000 7f0f7d000700 1 --2- 172.21.15.1:0/13321305 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0f78101f70 0x7f0f78073130 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:06.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.534+0000 7f0f7d000700 1 -- 172.21.15.1:0/13321305 >> 172.21.15.1:0/13321305 conn(0x7f0f780fc390 msgr2=0x7f0f78105590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:06.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.534+0000 7f0f7d000700 1 -- 172.21.15.1:0/13321305 shutdown_connections 2024-02-02T15:25:06.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.534+0000 7f0f7d000700 1 -- 172.21.15.1:0/13321305 wait complete. 2024-02-02T15:25:06.552 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-02-02T15:25:06.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:06 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/2115548995' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-02T15:25:06.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:06 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/856708753' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-02T15:25:06.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:06 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/2115548995' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-02T15:25:06.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:06 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/856708753' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-02T15:25:06.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.750+0000 7fa98f200700 1 -- 172.21.15.1:0/1155016717 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa988074d80 msgr2=0x7fa9880731d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:06.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.750+0000 7fa98f200700 1 --2- 172.21.15.1:0/1155016717 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa988074d80 0x7fa9880731d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa978009990 tx=0x7fa978009ca0 comp rx=0 tx=0).stop 2024-02-02T15:25:06.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.751+0000 7fa98f200700 1 -- 172.21.15.1:0/1155016717 shutdown_connections 2024-02-02T15:25:06.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.751+0000 7fa98f200700 1 --2- 172.21.15.1:0/1155016717 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa988108ce0 0x7fa98810b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:06.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.751+0000 7fa98f200700 1 --2- 172.21.15.1:0/1155016717 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa988074d80 0x7fa9880731d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:06.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.751+0000 7fa98f200700 1 --2- 172.21.15.1:0/1155016717 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa9880737a0 0x7fa988073c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:06.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.751+0000 7fa98f200700 1 -- 172.21.15.1:0/1155016717 >> 172.21.15.1:0/1155016717 conn(0x7fa9880fc390 msgr2=0x7fa9880fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:06.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.751+0000 7fa98f200700 1 -- 172.21.15.1:0/1155016717 shutdown_connections 2024-02-02T15:25:06.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.751+0000 7fa98f200700 1 -- 172.21.15.1:0/1155016717 wait complete. 2024-02-02T15:25:06.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.752+0000 7fa98f200700 1 Processor -- start 2024-02-02T15:25:06.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.752+0000 7fa98f200700 1 -- start start 2024-02-02T15:25:06.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.752+0000 7fa98f200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa9880737a0 0x7fa988198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:06.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.752+0000 7fa98f200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa988074d80 0x7fa988199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:06.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.752+0000 7fa98f200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa988108ce0 0x7fa9881a0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:06.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.752+0000 7fa98f200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9881a08d0 con 0x7fa988108ce0 2024-02-02T15:25:06.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.752+0000 7fa98f200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9881a0a10 con 0x7fa9880737a0 2024-02-02T15:25:06.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.752+0000 7fa98f200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9881a0ba0 con 0x7fa988074d80 2024-02-02T15:25:06.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.752+0000 7fa98d200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa988108ce0 0x7fa9881a0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:06.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.753+0000 7fa987e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa9880737a0 0x7fa988198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:06.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.753+0000 7fa987400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa988074d80 0x7fa988199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:06.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.753+0000 7fa987400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa988074d80 0x7fa988199290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:57484/0 (socket says 172.21.15.1:57484) 2024-02-02T15:25:06.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.753+0000 7fa98d200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa988108ce0 0x7fa9881a0300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43400/0 (socket says 172.21.15.1:43400) 2024-02-02T15:25:06.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.753+0000 7fa987e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa9880737a0 0x7fa988198d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:45500/0 (socket says 172.21.15.1:45500) 2024-02-02T15:25:06.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.753+0000 7fa98d200700 1 -- 172.21.15.1:0/3841819504 learned_addr learned my addr 172.21.15.1:0/3841819504 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:25:06.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.753+0000 7fa987e00700 1 -- 172.21.15.1:0/3841819504 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa988074d80 msgr2=0x7fa988199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:06.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.753+0000 7fa987e00700 1 --2- 172.21.15.1:0/3841819504 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa988074d80 0x7fa988199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:06.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.753+0000 7fa987e00700 1 -- 172.21.15.1:0/3841819504 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa988108ce0 msgr2=0x7fa9881a0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:06.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.753+0000 7fa987e00700 1 --2- 172.21.15.1:0/3841819504 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa988108ce0 0x7fa9881a0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:06.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.753+0000 7fa987e00700 1 -- 172.21.15.1:0/3841819504 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9780095f0 con 0x7fa9880737a0 2024-02-02T15:25:06.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.753+0000 7fa987400700 1 --2- 172.21.15.1:0/3841819504 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa988074d80 0x7fa988199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:25:06.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.753+0000 7fa98d200700 1 --2- 172.21.15.1:0/3841819504 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa988108ce0 0x7fa9881a0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:25:06.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.753+0000 7fa987e00700 1 --2- 172.21.15.1:0/3841819504 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa9880737a0 0x7fa988198d50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa97800b680 tx=0x7fa978005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:06.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.754+0000 7fa984c00700 1 -- 172.21.15.1:0/3841819504 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa97801d070 con 0x7fa9880737a0 2024-02-02T15:25:06.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.754+0000 7fa98f200700 1 -- 172.21.15.1:0/3841819504 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9881a0e20 con 0x7fa9880737a0 2024-02-02T15:25:06.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.754+0000 7fa984c00700 1 -- 172.21.15.1:0/3841819504 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa978021410 con 0x7fa9880737a0 2024-02-02T15:25:06.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.754+0000 7fa984c00700 1 -- 172.21.15.1:0/3841819504 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa978021ea0 con 0x7fa9880737a0 2024-02-02T15:25:06.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.754+0000 7fa98f200700 1 -- 172.21.15.1:0/3841819504 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9881a11e0 con 0x7fa9880737a0 2024-02-02T15:25:06.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.755+0000 7fa98f200700 1 -- 172.21.15.1:0/3841819504 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa988107390 con 0x7fa9880737a0 2024-02-02T15:25:06.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.757+0000 7fa984c00700 1 -- 172.21.15.1:0/3841819504 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 135069+0+0 (secure 0 0 0) 0x7fa9780217f0 con 0x7fa9880737a0 2024-02-02T15:25:06.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.758+0000 7fa984c00700 1 --2- 172.21.15.1:0/3841819504 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fa9680a0520 0x7fa9680a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:06.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.758+0000 7fa984c00700 1 -- 172.21.15.1:0/3841819504 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fa9780cd1c0 con 0x7fa9880737a0 2024-02-02T15:25:06.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.758+0000 7fa987400700 1 --2- 172.21.15.1:0/3841819504 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fa9680a0520 0x7fa9680a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:06.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.759+0000 7fa987400700 1 --2- 172.21.15.1:0/3841819504 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fa9680a0520 0x7fa9680a29e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa988074aa0 tx=0x7fa970009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:06.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.760+0000 7fa984c00700 1 -- 172.21.15.1:0/3841819504 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7fa978096ba0 con 0x7fa9880737a0 2024-02-02T15:25:06.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.971+0000 7fa98f200700 1 -- 172.21.15.1:0/3841819504 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fa98802ce00 con 0x7fa9880737a0 2024-02-02T15:25:06.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.971+0000 7fa984c00700 1 -- 172.21.15.1:0/3841819504 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v84) v1 ==== 103+0+11 (secure 0 0 0) 0x7fa978030020 con 0x7fa9880737a0 2024-02-02T15:25:06.971 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-02-02T15:25:06.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.974+0000 7fa98f200700 1 -- 172.21.15.1:0/3841819504 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fa9680a0520 msgr2=0x7fa9680a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:06.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.974+0000 7fa98f200700 1 --2- 172.21.15.1:0/3841819504 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fa9680a0520 0x7fa9680a29e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa988074aa0 tx=0x7fa970009260 comp rx=0 tx=0).stop 2024-02-02T15:25:06.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.974+0000 7fa98f200700 1 -- 172.21.15.1:0/3841819504 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa9880737a0 msgr2=0x7fa988198d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:06.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.974+0000 7fa98f200700 1 --2- 172.21.15.1:0/3841819504 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa9880737a0 0x7fa988198d50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa97800b680 tx=0x7fa978005740 comp rx=0 tx=0).stop 2024-02-02T15:25:06.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.974+0000 7fa98f200700 1 -- 172.21.15.1:0/3841819504 shutdown_connections 2024-02-02T15:25:06.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.974+0000 7fa98f200700 1 --2- 172.21.15.1:0/3841819504 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa988108ce0 0x7fa9881a0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:06.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.974+0000 7fa98f200700 1 --2- 172.21.15.1:0/3841819504 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa9880737a0 0x7fa988198d50 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:06.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.974+0000 7fa98f200700 1 --2- 172.21.15.1:0/3841819504 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fa9680a0520 0x7fa9680a29e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:06.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.974+0000 7fa98f200700 1 --2- 172.21.15.1:0/3841819504 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa988074d80 0x7fa988199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:06.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.974+0000 7fa98f200700 1 -- 172.21.15.1:0/3841819504 >> 172.21.15.1:0/3841819504 conn(0x7fa9880fc390 msgr2=0x7fa9880fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:06.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.974+0000 7fa98f200700 1 -- 172.21.15.1:0/3841819504 shutdown_connections 2024-02-02T15:25:06.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:06.974+0000 7fa98f200700 1 -- 172.21.15.1:0/3841819504 wait complete. 2024-02-02T15:25:06.992 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-02-02T15:25:07.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:07.191+0000 7f3589e00700 1 -- 172.21.15.1:0/2693328215 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3584101ff0 msgr2=0x7f3584102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:07.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:07.191+0000 7f3589e00700 1 --2- 172.21.15.1:0/2693328215 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3584101ff0 0x7f3584102450 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f356c0099c0 tx=0x7f356c009cd0 comp rx=0 tx=0).stop 2024-02-02T15:25:07.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:07.191+0000 7f3589e00700 1 -- 172.21.15.1:0/2693328215 shutdown_connections 2024-02-02T15:25:07.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:07.191+0000 7f3589e00700 1 --2- 172.21.15.1:0/2693328215 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3584102990 0x7f3584109240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:07.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:07.191+0000 7f3589e00700 1 --2- 172.21.15.1:0/2693328215 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3584100e00 0x7f3584101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:07.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:07.191+0000 7f3589e00700 1 --2- 172.21.15.1:0/2693328215 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3584101ff0 0x7f3584102450 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:07.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:07.191+0000 7f3589e00700 1 -- 172.21.15.1:0/2693328215 >> 172.21.15.1:0/2693328215 conn(0x7f35840fc370 msgr2=0x7f35840fe7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:07.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:07.191+0000 7f3589e00700 1 -- 172.21.15.1:0/2693328215 shutdown_connections 2024-02-02T15:25:07.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:07.192+0000 7f3589e00700 1 -- 172.21.15.1:0/2693328215 wait complete. 2024-02-02T15:25:07.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:07.193+0000 7f3589e00700 1 Processor -- start 2024-02-02T15:25:07.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:07.193+0000 7f3589e00700 1 -- start start 2024-02-02T15:25:07.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:07.194+0000 7f3589e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3584100e00 0x7f3584196b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:07.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:07.194+0000 7f3589e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3584101ff0 0x7f3584197090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:07.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:07.194+0000 7f3589e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3584102990 0x7f358419e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:07.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:07.194+0000 7f3589e00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f358419e6d0 con 0x7f3584102990 2024-02-02T15:25:07.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:07.194+0000 7f3589e00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f358419e810 con 0x7f3584101ff0 2024-02-02T15:25:07.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:07.194+0000 7f3589e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f358419e9a0 con 0x7f3584100e00 2024-02-02T15:25:07.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:07.194+0000 7f3582a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3584100e00 0x7f3584196b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:07.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:07.194+0000 7f3582000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3584101ff0 0x7f3584197090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:07.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:07.194+0000 7f3582a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3584100e00 0x7f3584196b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:57488/0 (socket says 172.21.15.1:57488) 2024-02-02T15:25:07.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:07.194+0000 7f3582a00700 1 -- 172.21.15.1:0/2969565018 learned_addr learned my addr 172.21.15.1:0/2969565018 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:25:07.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:07.194+0000 7f3582000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3584101ff0 0x7f3584197090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:45524/0 (socket says 172.21.15.1:45524) 2024-02-02T15:25:07.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:07.194+0000 7f3583400700 1 --2- 172.21.15.1:0/2969565018 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3584102990 0x7f358419e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:07.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:07.194+0000 7f3582000700 1 -- 172.21.15.1:0/2969565018 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3584100e00 msgr2=0x7f3584196b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:07.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:07.194+0000 7f3582000700 1 --2- 172.21.15.1:0/2969565018 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3584100e00 0x7f3584196b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:07.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:07.194+0000 7f3582000700 1 -- 172.21.15.1:0/2969565018 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3584102990 msgr2=0x7f358419e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:07.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:07.194+0000 7f3582000700 1 --2- 172.21.15.1:0/2969565018 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3584102990 0x7f358419e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:07.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:07.194+0000 7f3582000700 1 -- 172.21.15.1:0/2969565018 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f356c0095f0 con 0x7f3584101ff0 2024-02-02T15:25:07.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:07.194+0000 7f3582a00700 1 --2- 172.21.15.1:0/2969565018 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3584100e00 0x7f3584196b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:25:07.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:07.195+0000 7f3582000700 1 --2- 172.21.15.1:0/2969565018 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3584101ff0 0x7f3584197090 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f356c009fd0 tx=0x7f356c00f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:07.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:07.195+0000 7f3588a00700 1 -- 172.21.15.1:0/2969565018 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356c01d070 con 0x7f3584101ff0 2024-02-02T15:25:07.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:07.195+0000 7f3589e00700 1 -- 172.21.15.1:0/2969565018 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f358419ec20 con 0x7f3584101ff0 2024-02-02T15:25:07.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:07.195+0000 7f3588a00700 1 -- 172.21.15.1:0/2969565018 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f356c005050 con 0x7f3584101ff0 2024-02-02T15:25:07.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:07.195+0000 7f3588a00700 1 -- 172.21.15.1:0/2969565018 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356c004a70 con 0x7f3584101ff0 2024-02-02T15:25:07.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:07.195+0000 7f3589e00700 1 -- 172.21.15.1:0/2969565018 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f358419f040 con 0x7f3584101ff0 2024-02-02T15:25:07.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:07.196+0000 7f3589e00700 1 -- 172.21.15.1:0/2969565018 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3584108390 con 0x7f3584101ff0 2024-02-02T15:25:07.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:07.199+0000 7f3588a00700 1 -- 172.21.15.1:0/2969565018 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 135069+0+0 (secure 0 0 0) 0x7f356c005450 con 0x7f3584101ff0 2024-02-02T15:25:07.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:07.200+0000 7f3588a00700 1 --2- 172.21.15.1:0/2969565018 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f35640a0520 0x7f35640a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:07.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:07.200+0000 7f3588a00700 1 -- 172.21.15.1:0/2969565018 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f356c0ccaa0 con 0x7f3584101ff0 2024-02-02T15:25:07.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:07.201+0000 7f3582a00700 1 --2- 172.21.15.1:0/2969565018 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f35640a0520 0x7f35640a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:07.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:07.201+0000 7f3582a00700 1 --2- 172.21.15.1:0/2969565018 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f35640a0520 0x7f35640a29e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3574005950 tx=0x7f3574009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:07.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:07.202+0000 7f3588a00700 1 -- 172.21.15.1:0/2969565018 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f356c096480 con 0x7f3584101ff0 2024-02-02T15:25:07.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:07.429+0000 7f3589e00700 1 -- 172.21.15.1:0/2969565018 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f358402cc70 con 0x7f3584101ff0 2024-02-02T15:25:07.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:07.430+0000 7f3588a00700 1 -- 172.21.15.1:0/2969565018 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v84) v1 ==== 81+0+579 (secure 0 0 0) 0x7f356c099aa0 con 0x7f3584101ff0 2024-02-02T15:25:07.430 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/4092532060 2024-02-03T15:19:25.836171+0000 2024-02-02T15:25:07.430 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:6801/395498296 2024-02-03T15:19:25.836171+0000 2024-02-02T15:25:07.430 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:6800/395498296 2024-02-03T15:19:25.836171+0000 2024-02-02T15:25:07.430 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/1575373489 2024-02-03T15:19:25.836171+0000 2024-02-02T15:25:07.430 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/865983374 2024-02-03T15:19:01.431596+0000 2024-02-02T15:25:07.430 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/3165960227 2024-02-03T15:19:01.431596+0000 2024-02-02T15:25:07.430 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/2350830293 2024-02-03T15:19:01.431596+0000 2024-02-02T15:25:07.430 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/2959690250 2024-02-03T15:19:25.836171+0000 2024-02-02T15:25:07.430 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:6800/1162530732 2024-02-03T15:19:01.431596+0000 2024-02-02T15:25:07.431 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:6801/1162530732 2024-02-03T15:19:01.431596+0000 2024-02-02T15:25:07.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:07.432+0000 7f3589e00700 1 -- 172.21.15.1:0/2969565018 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f35640a0520 msgr2=0x7f35640a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:07.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:07.432+0000 7f3589e00700 1 --2- 172.21.15.1:0/2969565018 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f35640a0520 0x7f35640a29e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3574005950 tx=0x7f3574009260 comp rx=0 tx=0).stop 2024-02-02T15:25:07.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:07.432+0000 7f3589e00700 1 -- 172.21.15.1:0/2969565018 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3584101ff0 msgr2=0x7f3584197090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:07.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:07.432+0000 7f3589e00700 1 --2- 172.21.15.1:0/2969565018 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3584101ff0 0x7f3584197090 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f356c009fd0 tx=0x7f356c00f690 comp rx=0 tx=0).stop 2024-02-02T15:25:07.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:07.433+0000 7f3589e00700 1 -- 172.21.15.1:0/2969565018 shutdown_connections 2024-02-02T15:25:07.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:07.433+0000 7f3589e00700 1 --2- 172.21.15.1:0/2969565018 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3584102990 0x7f358419e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:07.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:07.433+0000 7f3589e00700 1 --2- 172.21.15.1:0/2969565018 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f35640a0520 0x7f35640a29e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:07.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:07.433+0000 7f3589e00700 1 --2- 172.21.15.1:0/2969565018 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3584100e00 0x7f3584196b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:07.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:07.433+0000 7f3589e00700 1 --2- 172.21.15.1:0/2969565018 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3584101ff0 0x7f3584197090 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:07.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:07.433+0000 7f3589e00700 1 -- 172.21.15.1:0/2969565018 >> 172.21.15.1:0/2969565018 conn(0x7f35840fc370 msgr2=0x7f35841054d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:07.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:07.433+0000 7f3589e00700 1 -- 172.21.15.1:0/2969565018 shutdown_connections 2024-02-02T15:25:07.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:07.433+0000 7f3589e00700 1 -- 172.21.15.1:0/2969565018 wait complete. 2024-02-02T15:25:07.434 INFO:teuthology.orchestra.run.smithi001.stderr:listed 10 entries 2024-02-02T15:25:07.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:07 smithi001 ceph-mon[35470]: pgmap v244: 129 pgs: 129 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-02-02T15:25:07.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:07 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/13321305' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-02T15:25:07.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:07 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/3841819504' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-02T15:25:07.452 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi001.front.sepia.ceph.com... 2024-02-02T15:25:07.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T15:25:07.452 DEBUG:teuthology.orchestra.run.smithi001:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-02-02T15:25:07.515 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi001.stderr:ceph-fuse[87173]: fuse finished with error 0 and tester_r 0 2024-02-02T15:25:07.519 INFO:teuthology.orchestra.run:waiting for 300 2024-02-02T15:25:07.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:07 smithi071 ceph-mon[44707]: pgmap v244: 129 pgs: 129 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-02-02T15:25:07.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:07 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/13321305' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-02T15:25:07.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:07 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/3841819504' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-02T15:25:07.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:07 smithi142 ceph-mon[45728]: pgmap v244: 129 pgs: 129 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-02-02T15:25:07.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:07 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/13321305' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-02T15:25:07.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:07 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/3841819504' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-02T15:25:08.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:08 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/2969565018' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-02T15:25:08.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:08 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/2969565018' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-02T15:25:08.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:08 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/2969565018' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-02T15:25:09.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:09 smithi071 ceph-mon[44707]: pgmap v245: 129 pgs: 129 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-02-02T15:25:09.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:09 smithi142 ceph-mon[45728]: pgmap v245: 129 pgs: 129 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-02-02T15:25:09.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:09 smithi001 ceph-mon[35470]: pgmap v245: 129 pgs: 129 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-02-02T15:25:11.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:11 smithi071 ceph-mon[44707]: pgmap v246: 129 pgs: 129 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-02-02T15:25:11.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:11 smithi142 ceph-mon[45728]: pgmap v246: 129 pgs: 129 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-02-02T15:25:11.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:11 smithi001 ceph-mon[35470]: pgmap v246: 129 pgs: 129 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-02-02T15:25:13.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:13 smithi071 ceph-mon[44707]: pgmap v247: 129 pgs: 129 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-02-02T15:25:13.620 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi001.front.sepia.ceph.com 2024-02-02T15:25:13.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T15:25:13.621 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-02-02T15:25:13.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:13 smithi142 ceph-mon[45728]: pgmap v247: 129 pgs: 129 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-02-02T15:25:13.676 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T15:25:13.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:13 smithi001 ceph-mon[35470]: pgmap v247: 129 pgs: 129 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-02-02T15:25:13.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:13.879+0000 7f8b41200700 1 -- 172.21.15.1:0/1573886806 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8b3c1028d0 msgr2=0x7f8b3c109180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:13.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:13.879+0000 7f8b41200700 1 --2- 172.21.15.1:0/1573886806 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8b3c1028d0 0x7f8b3c109180 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8b3400b0d0 tx=0x7f8b3400b3e0 comp rx=0 tx=0).stop 2024-02-02T15:25:13.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:13.879+0000 7f8b41200700 1 -- 172.21.15.1:0/1573886806 shutdown_connections 2024-02-02T15:25:13.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:13.879+0000 7f8b41200700 1 --2- 172.21.15.1:0/1573886806 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8b3c1028d0 0x7f8b3c109180 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:13.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:13.879+0000 7f8b41200700 1 --2- 172.21.15.1:0/1573886806 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b3c100d40 0x7f8b3c101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:13.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:13.879+0000 7f8b41200700 1 --2- 172.21.15.1:0/1573886806 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b3c101f30 0x7f8b3c102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:13.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:13.879+0000 7f8b41200700 1 -- 172.21.15.1:0/1573886806 >> 172.21.15.1:0/1573886806 conn(0x7f8b3c0fc290 msgr2=0x7f8b3c0fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:13.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:13.879+0000 7f8b41200700 1 -- 172.21.15.1:0/1573886806 shutdown_connections 2024-02-02T15:25:13.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:13.879+0000 7f8b41200700 1 -- 172.21.15.1:0/1573886806 wait complete. 2024-02-02T15:25:13.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:13.880+0000 7f8b41200700 1 Processor -- start 2024-02-02T15:25:13.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:13.880+0000 7f8b41200700 1 -- start start 2024-02-02T15:25:13.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:13.881+0000 7f8b41200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b3c100d40 0x7f8b3c198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:13.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:13.881+0000 7f8b41200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8b3c101f30 0x7f8b3c199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:13.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:13.881+0000 7f8b41200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b3c1028d0 0x7f8b3c1a0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:13.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:13.881+0000 7f8b41200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3c1a08d0 con 0x7f8b3c101f30 2024-02-02T15:25:13.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:13.881+0000 7f8b41200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3c1a0a10 con 0x7f8b3c1028d0 2024-02-02T15:25:13.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:13.881+0000 7f8b41200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3c1a0ba0 con 0x7f8b3c100d40 2024-02-02T15:25:13.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:13.881+0000 7f8b39400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8b3c101f30 0x7f8b3c199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:13.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:13.881+0000 7f8b39400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8b3c101f30 0x7f8b3c199290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:32930/0 (socket says 172.21.15.1:32930) 2024-02-02T15:25:13.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:13.881+0000 7f8b39400700 1 -- 172.21.15.1:0/2841706208 learned_addr learned my addr 172.21.15.1:0/2841706208 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:25:13.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:13.881+0000 7f8b39e00700 1 --2- 172.21.15.1:0/2841706208 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b3c100d40 0x7f8b3c198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:13.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:13.881+0000 7f8b3a800700 1 --2- 172.21.15.1:0/2841706208 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b3c1028d0 0x7f8b3c1a0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:13.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:13.881+0000 7f8b39400700 1 -- 172.21.15.1:0/2841706208 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b3c100d40 msgr2=0x7f8b3c198d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:13.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:13.881+0000 7f8b39400700 1 --2- 172.21.15.1:0/2841706208 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b3c100d40 0x7f8b3c198d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:13.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:13.881+0000 7f8b39400700 1 -- 172.21.15.1:0/2841706208 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b3c1028d0 msgr2=0x7f8b3c1a0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:13.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:13.881+0000 7f8b39400700 1 --2- 172.21.15.1:0/2841706208 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b3c1028d0 0x7f8b3c1a0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:13.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:13.881+0000 7f8b39400700 1 -- 172.21.15.1:0/2841706208 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b34009d00 con 0x7f8b3c101f30 2024-02-02T15:25:13.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:13.882+0000 7f8b39400700 1 --2- 172.21.15.1:0/2841706208 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8b3c101f30 0x7f8b3c199290 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8b2400e9a0 tx=0x7f8b2400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:13.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:13.882+0000 7f8b32a00700 1 -- 172.21.15.1:0/2841706208 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b24009790 con 0x7f8b3c101f30 2024-02-02T15:25:13.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:13.882+0000 7f8b32a00700 1 -- 172.21.15.1:0/2841706208 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b24010430 con 0x7f8b3c101f30 2024-02-02T15:25:13.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:13.882+0000 7f8b41200700 1 -- 172.21.15.1:0/2841706208 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b3c1a0e80 con 0x7f8b3c101f30 2024-02-02T15:25:13.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:13.882+0000 7f8b32a00700 1 -- 172.21.15.1:0/2841706208 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b24018680 con 0x7f8b3c101f30 2024-02-02T15:25:13.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:13.882+0000 7f8b41200700 1 -- 172.21.15.1:0/2841706208 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b3c1a1350 con 0x7f8b3c101f30 2024-02-02T15:25:13.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:13.883+0000 7f8b41200700 1 -- 172.21.15.1:0/2841706208 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b3c10c5b0 con 0x7f8b3c101f30 2024-02-02T15:25:13.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:13.885+0000 7f8b32a00700 1 -- 172.21.15.1:0/2841706208 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) v1 ==== 135069+0+0 (secure 0 0 0) 0x7f8b240105a0 con 0x7f8b3c101f30 2024-02-02T15:25:13.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:13.887+0000 7f8b32a00700 1 --2- 172.21.15.1:0/2841706208 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f8b1c0a05a0 0x7f8b1c0a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:13.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:13.887+0000 7f8b32a00700 1 -- 172.21.15.1:0/2841706208 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f8b24014070 con 0x7f8b3c101f30 2024-02-02T15:25:13.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:13.887+0000 7f8b39e00700 1 --2- 172.21.15.1:0/2841706208 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f8b1c0a05a0 0x7f8b1c0a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:13.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:13.888+0000 7f8b39e00700 1 --2- 172.21.15.1:0/2841706208 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f8b1c0a05a0 0x7f8b1c0a2a60 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c101d90 tx=0x7f8b2c009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:13.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:13.892+0000 7f8b32a00700 1 -- 172.21.15.1:0/2841706208 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f8b24052110 con 0x7f8b3c101f30 2024-02-02T15:25:14.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.106+0000 7f8b41200700 1 -- 172.21.15.1:0/2841706208 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8b3c1a1960 con 0x7f8b3c101f30 2024-02-02T15:25:14.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.107+0000 7f8b32a00700 1 -- 172.21.15.1:0/2841706208 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+23841 (secure 0 0 0) 0x7f8b24095680 con 0x7f8b3c101f30 2024-02-02T15:25:14.108 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:25:14.108 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":84,"fsid":"371a409c-c1de-11ee-95b6-87774f69a715","created":"2024-02-02T15:18:26.746506+0000","modified":"2024-02-02T15:24:52.310888+0000","last_up_change":"2024-02-02T15:24:08.800951+0000","last_in_change":"2024-02-02T15:23:52.784280+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T15:21:22.471754+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-02T15:24:46.195779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-02T15:24:47.226403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-02T15:24:50.300462+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"73e8a07f-3560-4c19-85ed-030d4d8f0f23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6803","nonce":2591906423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6805","nonce":2591906423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6809","nonce":2591906423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6807","nonce":2591906423}]},"public_addr":"172.21.15.1:6803/2591906423","cluster_addr":"172.21.15.1:6805/2591906423","heartbeat_back_addr":"172.21.15.1:6809/2591906423","heartbeat_front_addr":"172.21.15.1:6807/2591906423","state":["exists","up"]},{"osd":1,"uuid":"e53cffec-0ff9-4ed9-8dcf-ff47d04db2bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6801","nonce":747581719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6803","nonce":747581719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6807","nonce":747581719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6805","nonce":747581719}]},"public_addr":"172.21.15.71:6801/747581719","cluster_addr":"172.21.15.71:6803/747581719","heartbeat_back_addr":"172.21.15.71:6807/747581719","heartbeat_front_addr":"172.21.15.71:6805/747581719","state":["exists","up"]},{"osd":2,"uuid":"856c2347-294c-482b-8595-ad77f4792db4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6801","nonce":377511414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6803","nonce":377511414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6807","nonce":377511414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6805","nonce":377511414}]},"public_addr":"172.21.15.142:6801/377511414","cluster_addr":"172.21.15.142:6803/377511414","heartbeat_back_addr":"172.21.15.142:6807/377511414","heartbeat_front_addr":"172.21.15.142:6805/377511414","state":["exists","up"]},{"osd":3,"uuid":"4b946ad2-2da9-4e0b-9d61-d039d8669a42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6811","nonce":1086124477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6813","nonce":1086124477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6817","nonce":1086124477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6815","nonce":1086124477}]},"public_addr":"172.21.15.1:6811/1086124477","cluster_addr":"172.21.15.1:6813/1086124477","heartbeat_back_addr":"172.21.15.1:6817/1086124477","heartbeat_front_addr":"172.21.15.1:6815/1086124477","state":["exists","up"]},{"osd":4,"uuid":"43e36c43-47a0-4634-a45b-1da14a0e19d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6809","nonce":3095380305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6811","nonce":3095380305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6815","nonce":3095380305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6813","nonce":3095380305}]},"public_addr":"172.21.15.71:6809/3095380305","cluster_addr":"172.21.15.71:6811/3095380305","heartbeat_back_addr":"172.21.15.71:6815/3095380305","heartbeat_front_addr":"172.21.15.71:6813/3095380305","state":["exists","up"]},{"osd":5,"uuid":"eb48160e-ea7c-42f7-9477-be110bc8a34e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6809","nonce":4195149262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6811","nonce":4195149262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6815","nonce":4195149262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6813","nonce":4195149262}]},"public_addr":"172.21.15.142:6809/4195149262","cluster_addr":"172.21.15.142:6811/4195149262","heartbeat_back_addr":"172.21.15.142:6815/4195149262","heartbeat_front_addr":"172.21.15.142:6813/4195149262","state":["exists","up"]},{"osd":6,"uuid":"0ec4501a-5853-4ca6-b325-231f5b72335e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6819","nonce":1560153428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6821","nonce":1560153428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6825","nonce":1560153428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6823","nonce":1560153428}]},"public_addr":"172.21.15.1:6819/1560153428","cluster_addr":"172.21.15.1:6821/1560153428","heartbeat_back_addr":"172.21.15.1:6825/1560153428","heartbeat_front_addr":"172.21.15.1:6823/1560153428","state":["exists","up"]},{"osd":7,"uuid":"029debe8-9cdf-44f6-9aea-cabd1d185f93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6817","nonce":4080955538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6819","nonce":4080955538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6823","nonce":4080955538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6821","nonce":4080955538}]},"public_addr":"172.21.15.71:6817/4080955538","cluster_addr":"172.21.15.71:6819/4080955538","heartbeat_back_addr":"172.21.15.71:6823/4080955538","heartbeat_front_addr":"172.21.15.71:6821/4080955538","state":["exists","up"]},{"osd":8,"uuid":"7c90896c-9955-4c1b-aaca-7531dd5457f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6817","nonce":3942038209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6819","nonce":3942038209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6823","nonce":3942038209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6821","nonce":3942038209}]},"public_addr":"172.21.15.142:6817/3942038209","cluster_addr":"172.21.15.142:6819/3942038209","heartbeat_back_addr":"172.21.15.142:6823/3942038209","heartbeat_front_addr":"172.21.15.142:6821/3942038209","state":["exists","up"]},{"osd":9,"uuid":"7e165e68-31a8-4598-a577-de3e90ae5c5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6827","nonce":2716544850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6829","nonce":2716544850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6833","nonce":2716544850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6831","nonce":2716544850}]},"public_addr":"172.21.15.1:6827/2716544850","cluster_addr":"172.21.15.1:6829/2716544850","heartbeat_back_addr":"172.21.15.1:6833/2716544850","heartbeat_front_addr":"172.21.15.1:6831/2716544850","state":["exists","up"]},{"osd":10,"uuid":"48cd5fdb-0887-444a-aab8-a1de24282bb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6825","nonce":84691841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6827","nonce":84691841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6831","nonce":84691841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6829","nonce":84691841}]},"public_addr":"172.21.15.71:6825/84691841","cluster_addr":"172.21.15.71:6827/84691841","heartbeat_back_addr":"172.21.15.71:6831/84691841","heartbeat_front_addr":"172.21.15.71:6829/84691841","state":["exists","up"]},{"osd":11,"uuid":"f011dd27-11b0-4166-adfe-76756533163a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6825","nonce":2290179934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6827","nonce":2290179934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6831","nonce":2290179934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6829","nonce":2290179934}]},"public_addr":"172.21.15.142:6825/2290179934","cluster_addr":"172.21.15.142:6827/2290179934","heartbeat_back_addr":"172.21.15.142:6831/2290179934","heartbeat_front_addr":"172.21.15.142:6829/2290179934","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:20:44.060692+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:01.903817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:19.368258+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:40.028037+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:57.513361+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:22:14.805944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:22:35.188439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:22:53.014412+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:23:10.507420+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:23:31.849765+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:23:50.070612+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:24:07.388625+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/4092532060":"2024-02-03T15:19:25.836171+0000","172.21.15.1:6801/395498296":"2024-02-03T15:19:25.836171+0000","172.21.15.1:6800/395498296":"2024-02-03T15:19:25.836171+0000","172.21.15.1:0/1575373489":"2024-02-03T15:19:25.836171+0000","172.21.15.1:0/865983374":"2024-02-03T15:19:01.431596+0000","172.21.15.1:0/3165960227":"2024-02-03T15:19:01.431596+0000","172.21.15.1:0/2350830293":"2024-02-03T15:19:01.431596+0000","172.21.15.1:0/2959690250":"2024-02-03T15:19:25.836171+0000","172.21.15.1:6800/1162530732":"2024-02-03T15:19:01.431596+0000","172.21.15.1:6801/1162530732":"2024-02-03T15:19:01.431596+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T15:25:14.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.110+0000 7f8b41200700 1 -- 172.21.15.1:0/2841706208 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f8b1c0a05a0 msgr2=0x7f8b1c0a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:14.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.110+0000 7f8b41200700 1 --2- 172.21.15.1:0/2841706208 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f8b1c0a05a0 0x7f8b1c0a2a60 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c101d90 tx=0x7f8b2c009260 comp rx=0 tx=0).stop 2024-02-02T15:25:14.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.110+0000 7f8b41200700 1 -- 172.21.15.1:0/2841706208 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8b3c101f30 msgr2=0x7f8b3c199290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:14.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.110+0000 7f8b41200700 1 --2- 172.21.15.1:0/2841706208 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8b3c101f30 0x7f8b3c199290 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8b2400e9a0 tx=0x7f8b2400ecb0 comp rx=0 tx=0).stop 2024-02-02T15:25:14.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.110+0000 7f8b41200700 1 -- 172.21.15.1:0/2841706208 shutdown_connections 2024-02-02T15:25:14.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.110+0000 7f8b41200700 1 --2- 172.21.15.1:0/2841706208 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b3c1028d0 0x7f8b3c1a0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:14.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.110+0000 7f8b41200700 1 --2- 172.21.15.1:0/2841706208 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f8b1c0a05a0 0x7f8b1c0a2a60 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:14.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.110+0000 7f8b41200700 1 --2- 172.21.15.1:0/2841706208 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b3c100d40 0x7f8b3c198d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:14.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.110+0000 7f8b41200700 1 --2- 172.21.15.1:0/2841706208 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8b3c101f30 0x7f8b3c199290 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:14.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.110+0000 7f8b41200700 1 -- 172.21.15.1:0/2841706208 >> 172.21.15.1:0/2841706208 conn(0x7f8b3c0fc290 msgr2=0x7f8b3c1077c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:14.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.110+0000 7f8b41200700 1 -- 172.21.15.1:0/2841706208 shutdown_connections 2024-02-02T15:25:14.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.110+0000 7f8b41200700 1 -- 172.21.15.1:0/2841706208 wait complete. 2024-02-02T15:25:14.129 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T15:25:14.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.367+0000 7f1590800700 1 -- 172.21.15.1:0/2390177941 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15881040f0 msgr2=0x7f15881064e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:14.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.367+0000 7f1590800700 1 --2- 172.21.15.1:0/2390177941 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15881040f0 0x7f15881064e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f15700098f0 tx=0x7f1570009c00 comp rx=0 tx=0).stop 2024-02-02T15:25:14.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.367+0000 7f1590800700 1 -- 172.21.15.1:0/2390177941 shutdown_connections 2024-02-02T15:25:14.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.367+0000 7f1590800700 1 --2- 172.21.15.1:0/2390177941 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1588106a20 0x7f1588108e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:14.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.367+0000 7f1590800700 1 --2- 172.21.15.1:0/2390177941 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15881017c0 0x7f1588103bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:14.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.367+0000 7f1590800700 1 --2- 172.21.15.1:0/2390177941 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15881040f0 0x7f15881064e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:14.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.367+0000 7f1590800700 1 -- 172.21.15.1:0/2390177941 >> 172.21.15.1:0/2390177941 conn(0x7f15880fb170 msgr2=0x7f15880fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:14.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.367+0000 7f1590800700 1 -- 172.21.15.1:0/2390177941 shutdown_connections 2024-02-02T15:25:14.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.367+0000 7f1590800700 1 -- 172.21.15.1:0/2390177941 wait complete. 2024-02-02T15:25:14.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.368+0000 7f1590800700 1 Processor -- start 2024-02-02T15:25:14.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.368+0000 7f1590800700 1 -- start start 2024-02-02T15:25:14.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.369+0000 7f1590800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15881017c0 0x7f1588101130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:14.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.369+0000 7f1590800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15881040f0 0x7f15880ff770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:14.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.369+0000 7f1590800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1588106a20 0x7f15880ffcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:14.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.369+0000 7f1590800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1588100450 con 0x7f1588106a20 2024-02-02T15:25:14.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.369+0000 7f1590800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15881005c0 con 0x7f15881040f0 2024-02-02T15:25:14.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.369+0000 7f1590800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15881a0ab0 con 0x7f15881017c0 2024-02-02T15:25:14.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.369+0000 7f158e800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1588106a20 0x7f15880ffcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:14.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.369+0000 7f158e800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1588106a20 0x7f15880ffcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:32950/0 (socket says 172.21.15.1:32950) 2024-02-02T15:25:14.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.369+0000 7f158e800700 1 -- 172.21.15.1:0/4082597440 learned_addr learned my addr 172.21.15.1:0/4082597440 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:25:14.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.369+0000 7f158de00700 1 --2- 172.21.15.1:0/4082597440 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15881017c0 0x7f1588101130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:14.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.369+0000 7f158d400700 1 --2- 172.21.15.1:0/4082597440 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15881040f0 0x7f15880ff770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:14.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.369+0000 7f158e800700 1 -- 172.21.15.1:0/4082597440 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15881017c0 msgr2=0x7f1588101130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:14.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.369+0000 7f158e800700 1 --2- 172.21.15.1:0/4082597440 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15881017c0 0x7f1588101130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:14.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.369+0000 7f158e800700 1 -- 172.21.15.1:0/4082597440 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15881040f0 msgr2=0x7f15880ff770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:14.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.369+0000 7f158e800700 1 --2- 172.21.15.1:0/4082597440 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15881040f0 0x7f15880ff770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:14.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.369+0000 7f158e800700 1 -- 172.21.15.1:0/4082597440 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15780095f0 con 0x7f1588106a20 2024-02-02T15:25:14.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.370+0000 7f158e800700 1 --2- 172.21.15.1:0/4082597440 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1588106a20 0x7f15880ffcb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f158000ef30 tx=0x7f158000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:14.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.370+0000 7f1586a00700 1 -- 172.21.15.1:0/4082597440 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1580018070 con 0x7f1588106a20 2024-02-02T15:25:14.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.370+0000 7f1590800700 1 -- 172.21.15.1:0/4082597440 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1570009520 con 0x7f1588106a20 2024-02-02T15:25:14.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.370+0000 7f1586a00700 1 -- 172.21.15.1:0/4082597440 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f158000f040 con 0x7f1588106a20 2024-02-02T15:25:14.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.370+0000 7f1586a00700 1 -- 172.21.15.1:0/4082597440 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f158000cd00 con 0x7f1588106a20 2024-02-02T15:25:14.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.370+0000 7f1590800700 1 -- 172.21.15.1:0/4082597440 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15881a1100 con 0x7f1588106a20 2024-02-02T15:25:14.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.371+0000 7f1590800700 1 -- 172.21.15.1:0/4082597440 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15880fccf0 con 0x7f1588106a20 2024-02-02T15:25:14.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.373+0000 7f1586a00700 1 -- 172.21.15.1:0/4082597440 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) v1 ==== 135069+0+0 (secure 0 0 0) 0x7f1580003a00 con 0x7f1588106a20 2024-02-02T15:25:14.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.374+0000 7f1586a00700 1 --2- 172.21.15.1:0/4082597440 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f15680a0520 0x7f15680a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:14.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.374+0000 7f1586a00700 1 -- 172.21.15.1:0/4082597440 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f15800cb690 con 0x7f1588106a20 2024-02-02T15:25:14.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.374+0000 7f158de00700 1 --2- 172.21.15.1:0/4082597440 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f15680a0520 0x7f15680a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:14.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.375+0000 7f158de00700 1 --2- 172.21.15.1:0/4082597440 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f15680a0520 0x7f15680a29e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1578005fd0 tx=0x7f1578009310 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:14.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.378+0000 7f1586a00700 1 -- 172.21.15.1:0/4082597440 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f1580095070 con 0x7f1588106a20 2024-02-02T15:25:14.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:14 smithi001 ceph-mon[35470]: pgmap v248: 129 pgs: 129 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-02-02T15:25:14.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:14 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/2841706208' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-02T15:25:14.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.591+0000 7f1590800700 1 -- 172.21.15.1:0/4082597440 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f15880619f0 con 0x7f1588106a20 2024-02-02T15:25:14.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.593+0000 7f1586a00700 1 -- 172.21.15.1:0/4082597440 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+23841 (secure 0 0 0) 0x7f1580077e60 con 0x7f1588106a20 2024-02-02T15:25:14.593 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:25:14.593 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":84,"fsid":"371a409c-c1de-11ee-95b6-87774f69a715","created":"2024-02-02T15:18:26.746506+0000","modified":"2024-02-02T15:24:52.310888+0000","last_up_change":"2024-02-02T15:24:08.800951+0000","last_in_change":"2024-02-02T15:23:52.784280+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T15:21:22.471754+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-02T15:24:46.195779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-02T15:24:47.226403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-02T15:24:50.300462+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"73e8a07f-3560-4c19-85ed-030d4d8f0f23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6803","nonce":2591906423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6805","nonce":2591906423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6809","nonce":2591906423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6807","nonce":2591906423}]},"public_addr":"172.21.15.1:6803/2591906423","cluster_addr":"172.21.15.1:6805/2591906423","heartbeat_back_addr":"172.21.15.1:6809/2591906423","heartbeat_front_addr":"172.21.15.1:6807/2591906423","state":["exists","up"]},{"osd":1,"uuid":"e53cffec-0ff9-4ed9-8dcf-ff47d04db2bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6801","nonce":747581719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6803","nonce":747581719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6807","nonce":747581719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6805","nonce":747581719}]},"public_addr":"172.21.15.71:6801/747581719","cluster_addr":"172.21.15.71:6803/747581719","heartbeat_back_addr":"172.21.15.71:6807/747581719","heartbeat_front_addr":"172.21.15.71:6805/747581719","state":["exists","up"]},{"osd":2,"uuid":"856c2347-294c-482b-8595-ad77f4792db4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6801","nonce":377511414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6803","nonce":377511414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6807","nonce":377511414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6805","nonce":377511414}]},"public_addr":"172.21.15.142:6801/377511414","cluster_addr":"172.21.15.142:6803/377511414","heartbeat_back_addr":"172.21.15.142:6807/377511414","heartbeat_front_addr":"172.21.15.142:6805/377511414","state":["exists","up"]},{"osd":3,"uuid":"4b946ad2-2da9-4e0b-9d61-d039d8669a42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6811","nonce":1086124477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6813","nonce":1086124477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6817","nonce":1086124477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6815","nonce":1086124477}]},"public_addr":"172.21.15.1:6811/1086124477","cluster_addr":"172.21.15.1:6813/1086124477","heartbeat_back_addr":"172.21.15.1:6817/1086124477","heartbeat_front_addr":"172.21.15.1:6815/1086124477","state":["exists","up"]},{"osd":4,"uuid":"43e36c43-47a0-4634-a45b-1da14a0e19d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6809","nonce":3095380305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6811","nonce":3095380305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6815","nonce":3095380305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6813","nonce":3095380305}]},"public_addr":"172.21.15.71:6809/3095380305","cluster_addr":"172.21.15.71:6811/3095380305","heartbeat_back_addr":"172.21.15.71:6815/3095380305","heartbeat_front_addr":"172.21.15.71:6813/3095380305","state":["exists","up"]},{"osd":5,"uuid":"eb48160e-ea7c-42f7-9477-be110bc8a34e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6809","nonce":4195149262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6811","nonce":4195149262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6815","nonce":4195149262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6813","nonce":4195149262}]},"public_addr":"172.21.15.142:6809/4195149262","cluster_addr":"172.21.15.142:6811/4195149262","heartbeat_back_addr":"172.21.15.142:6815/4195149262","heartbeat_front_addr":"172.21.15.142:6813/4195149262","state":["exists","up"]},{"osd":6,"uuid":"0ec4501a-5853-4ca6-b325-231f5b72335e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6819","nonce":1560153428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6821","nonce":1560153428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6825","nonce":1560153428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6823","nonce":1560153428}]},"public_addr":"172.21.15.1:6819/1560153428","cluster_addr":"172.21.15.1:6821/1560153428","heartbeat_back_addr":"172.21.15.1:6825/1560153428","heartbeat_front_addr":"172.21.15.1:6823/1560153428","state":["exists","up"]},{"osd":7,"uuid":"029debe8-9cdf-44f6-9aea-cabd1d185f93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6817","nonce":4080955538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6819","nonce":4080955538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6823","nonce":4080955538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6821","nonce":4080955538}]},"public_addr":"172.21.15.71:6817/4080955538","cluster_addr":"172.21.15.71:6819/4080955538","heartbeat_back_addr":"172.21.15.71:6823/4080955538","heartbeat_front_addr":"172.21.15.71:6821/4080955538","state":["exists","up"]},{"osd":8,"uuid":"7c90896c-9955-4c1b-aaca-7531dd5457f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6817","nonce":3942038209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6819","nonce":3942038209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6823","nonce":3942038209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6821","nonce":3942038209}]},"public_addr":"172.21.15.142:6817/3942038209","cluster_addr":"172.21.15.142:6819/3942038209","heartbeat_back_addr":"172.21.15.142:6823/3942038209","heartbeat_front_addr":"172.21.15.142:6821/3942038209","state":["exists","up"]},{"osd":9,"uuid":"7e165e68-31a8-4598-a577-de3e90ae5c5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6827","nonce":2716544850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6829","nonce":2716544850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6833","nonce":2716544850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6831","nonce":2716544850}]},"public_addr":"172.21.15.1:6827/2716544850","cluster_addr":"172.21.15.1:6829/2716544850","heartbeat_back_addr":"172.21.15.1:6833/2716544850","heartbeat_front_addr":"172.21.15.1:6831/2716544850","state":["exists","up"]},{"osd":10,"uuid":"48cd5fdb-0887-444a-aab8-a1de24282bb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6825","nonce":84691841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6827","nonce":84691841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6831","nonce":84691841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6829","nonce":84691841}]},"public_addr":"172.21.15.71:6825/84691841","cluster_addr":"172.21.15.71:6827/84691841","heartbeat_back_addr":"172.21.15.71:6831/84691841","heartbeat_front_addr":"172.21.15.71:6829/84691841","state":["exists","up"]},{"osd":11,"uuid":"f011dd27-11b0-4166-adfe-76756533163a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6825","nonce":2290179934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6827","nonce":2290179934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6831","nonce":2290179934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6829","nonce":2290179934}]},"public_addr":"172.21.15.142:6825/2290179934","cluster_addr":"172.21.15.142:6827/2290179934","heartbeat_back_addr":"172.21.15.142:6831/2290179934","heartbeat_front_addr":"172.21.15.142:6829/2290179934","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:20:44.060692+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:01.903817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:19.368258+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:40.028037+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:57.513361+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:22:14.805944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:22:35.188439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:22:53.014412+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:23:10.507420+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:23:31.849765+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:23:50.070612+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:24:07.388625+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/4092532060":"2024-02-03T15:19:25.836171+0000","172.21.15.1:6801/395498296":"2024-02-03T15:19:25.836171+0000","172.21.15.1:6800/395498296":"2024-02-03T15:19:25.836171+0000","172.21.15.1:0/1575373489":"2024-02-03T15:19:25.836171+0000","172.21.15.1:0/865983374":"2024-02-03T15:19:01.431596+0000","172.21.15.1:0/3165960227":"2024-02-03T15:19:01.431596+0000","172.21.15.1:0/2350830293":"2024-02-03T15:19:01.431596+0000","172.21.15.1:0/2959690250":"2024-02-03T15:19:25.836171+0000","172.21.15.1:6800/1162530732":"2024-02-03T15:19:01.431596+0000","172.21.15.1:6801/1162530732":"2024-02-03T15:19:01.431596+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T15:25:14.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.595+0000 7f1590800700 1 -- 172.21.15.1:0/4082597440 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f15680a0520 msgr2=0x7f15680a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:14.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.595+0000 7f1590800700 1 --2- 172.21.15.1:0/4082597440 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f15680a0520 0x7f15680a29e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1578005fd0 tx=0x7f1578009310 comp rx=0 tx=0).stop 2024-02-02T15:25:14.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.595+0000 7f1590800700 1 -- 172.21.15.1:0/4082597440 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1588106a20 msgr2=0x7f15880ffcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:14.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.595+0000 7f1590800700 1 --2- 172.21.15.1:0/4082597440 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1588106a20 0x7f15880ffcb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f158000ef30 tx=0x7f158000c3a0 comp rx=0 tx=0).stop 2024-02-02T15:25:14.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.595+0000 7f1590800700 1 -- 172.21.15.1:0/4082597440 shutdown_connections 2024-02-02T15:25:14.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.595+0000 7f1590800700 1 --2- 172.21.15.1:0/4082597440 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1588106a20 0x7f15880ffcb0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:14.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.595+0000 7f1590800700 1 --2- 172.21.15.1:0/4082597440 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f15680a0520 0x7f15680a29e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:14.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.595+0000 7f1590800700 1 --2- 172.21.15.1:0/4082597440 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15881017c0 0x7f1588101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:14.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.595+0000 7f1590800700 1 --2- 172.21.15.1:0/4082597440 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15881040f0 0x7f15880ff770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:14.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.595+0000 7f1590800700 1 -- 172.21.15.1:0/4082597440 >> 172.21.15.1:0/4082597440 conn(0x7f15880fb170 msgr2=0x7f1588102430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:14.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.595+0000 7f1590800700 1 -- 172.21.15.1:0/4082597440 shutdown_connections 2024-02-02T15:25:14.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.596+0000 7f1590800700 1 -- 172.21.15.1:0/4082597440 wait complete. 2024-02-02T15:25:14.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:14 smithi071 ceph-mon[44707]: pgmap v248: 129 pgs: 129 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-02-02T15:25:14.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:14 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/2841706208' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-02T15:25:14.615 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-02-02T15:25:14.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:14 smithi142 ceph-mon[45728]: pgmap v248: 129 pgs: 129 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-02-02T15:25:14.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:14 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/2841706208' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-02T15:25:14.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.815+0000 7f0b11200700 1 -- 172.21.15.1:0/1976767782 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0b0c102930 msgr2=0x7f0b0c109250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:14.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.815+0000 7f0b11200700 1 --2- 172.21.15.1:0/1976767782 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0b0c102930 0x7f0b0c109250 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0b0400b0a0 tx=0x7f0b0400b3b0 comp rx=0 tx=0).stop 2024-02-02T15:25:14.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.815+0000 7f0b11200700 1 -- 172.21.15.1:0/1976767782 shutdown_connections 2024-02-02T15:25:14.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.815+0000 7f0b11200700 1 --2- 172.21.15.1:0/1976767782 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0b0c102930 0x7f0b0c109250 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:14.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.815+0000 7f0b11200700 1 --2- 172.21.15.1:0/1976767782 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b0c100e20 0x7f0b0c101240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:14.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.815+0000 7f0b11200700 1 --2- 172.21.15.1:0/1976767782 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b0c101f70 0x7f0b0c1023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:14.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.815+0000 7f0b11200700 1 -- 172.21.15.1:0/1976767782 >> 172.21.15.1:0/1976767782 conn(0x7f0b0c0fc390 msgr2=0x7f0b0c0fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:14.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.815+0000 7f0b11200700 1 -- 172.21.15.1:0/1976767782 shutdown_connections 2024-02-02T15:25:14.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.815+0000 7f0b11200700 1 -- 172.21.15.1:0/1976767782 wait complete. 2024-02-02T15:25:14.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.816+0000 7f0b11200700 1 Processor -- start 2024-02-02T15:25:14.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.817+0000 7f0b11200700 1 -- start start 2024-02-02T15:25:14.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.817+0000 7f0b11200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b0c100e20 0x7f0b0c198df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:14.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.817+0000 7f0b11200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b0c101f70 0x7f0b0c199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:14.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.817+0000 7f0b11200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0b0c102930 0x7f0b0c1a03a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:14.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.817+0000 7f0b11200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0c1a0970 con 0x7f0b0c102930 2024-02-02T15:25:14.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.817+0000 7f0b11200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0c1a0ab0 con 0x7f0b0c101f70 2024-02-02T15:25:14.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.817+0000 7f0b11200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b0c1a0c40 con 0x7f0b0c100e20 2024-02-02T15:25:14.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.817+0000 7f0b09e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b0c100e20 0x7f0b0c198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:14.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.817+0000 7f0b0a800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0b0c102930 0x7f0b0c1a03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:14.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.817+0000 7f0b09e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b0c100e20 0x7f0b0c198df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:37004/0 (socket says 172.21.15.1:37004) 2024-02-02T15:25:14.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.817+0000 7f0b09e00700 1 -- 172.21.15.1:0/3204368829 learned_addr learned my addr 172.21.15.1:0/3204368829 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:25:14.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.817+0000 7f0b09400700 1 --2- 172.21.15.1:0/3204368829 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b0c101f70 0x7f0b0c199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:14.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.818+0000 7f0b0a800700 1 -- 172.21.15.1:0/3204368829 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b0c100e20 msgr2=0x7f0b0c198df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:14.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.818+0000 7f0b0a800700 1 --2- 172.21.15.1:0/3204368829 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b0c100e20 0x7f0b0c198df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:14.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.818+0000 7f0b0a800700 1 -- 172.21.15.1:0/3204368829 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b0c101f70 msgr2=0x7f0b0c199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:14.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.818+0000 7f0b0a800700 1 --2- 172.21.15.1:0/3204368829 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b0c101f70 0x7f0b0c199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:14.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.818+0000 7f0b0a800700 1 -- 172.21.15.1:0/3204368829 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b04009d00 con 0x7f0b0c102930 2024-02-02T15:25:14.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.818+0000 7f0b09400700 1 --2- 172.21.15.1:0/3204368829 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b0c101f70 0x7f0b0c199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:25:14.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.818+0000 7f0b0a800700 1 --2- 172.21.15.1:0/3204368829 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0b0c102930 0x7f0b0c1a03a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f0b0400b070 tx=0x7f0b0400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:14.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.818+0000 7f0b02a00700 1 -- 172.21.15.1:0/3204368829 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b0400e040 con 0x7f0b0c102930 2024-02-02T15:25:14.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.818+0000 7f0b02a00700 1 -- 172.21.15.1:0/3204368829 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b04009480 con 0x7f0b0c102930 2024-02-02T15:25:14.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.818+0000 7f0b11200700 1 -- 172.21.15.1:0/3204368829 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b0c1a0ec0 con 0x7f0b0c102930 2024-02-02T15:25:14.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.818+0000 7f0b02a00700 1 -- 172.21.15.1:0/3204368829 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b040052a0 con 0x7f0b0c102930 2024-02-02T15:25:14.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.818+0000 7f0b11200700 1 -- 172.21.15.1:0/3204368829 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b0c1a1230 con 0x7f0b0c102930 2024-02-02T15:25:14.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.819+0000 7f0b11200700 1 -- 172.21.15.1:0/3204368829 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b0c10c670 con 0x7f0b0c102930 2024-02-02T15:25:14.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.821+0000 7f0b02a00700 1 -- 172.21.15.1:0/3204368829 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) v1 ==== 135069+0+0 (secure 0 0 0) 0x7f0b0401d480 con 0x7f0b0c102930 2024-02-02T15:25:14.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.822+0000 7f0b02a00700 1 --2- 172.21.15.1:0/3204368829 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f0aec0a0520 0x7f0aec0a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:14.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.823+0000 7f0b02a00700 1 -- 172.21.15.1:0/3204368829 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f0b040cce40 con 0x7f0b0c102930 2024-02-02T15:25:14.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.823+0000 7f0b09e00700 1 --2- 172.21.15.1:0/3204368829 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f0aec0a0520 0x7f0aec0a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:14.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.823+0000 7f0b09e00700 1 --2- 172.21.15.1:0/3204368829 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f0aec0a0520 0x7f0aec0a29e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0afc009d20 tx=0x7f0afc009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:14.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:14.826+0000 7f0b02a00700 1 -- 172.21.15.1:0/3204368829 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f0b04096820 con 0x7f0b0c102930 2024-02-02T15:25:15.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:15.088+0000 7f0b11200700 1 -- 172.21.15.1:0/3204368829 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f0b0c1a1840 con 0x7f0b0c102930 2024-02-02T15:25:15.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:15.258+0000 7f0b02a00700 1 -- 172.21.15.1:0/3204368829 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) v1 ==== 118+0+0 (secure 0 0 0) 0x7f0b04099e40 con 0x7f0b0c102930 2024-02-02T15:25:15.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:15.260+0000 7f0b11200700 1 -- 172.21.15.1:0/3204368829 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f0aec0a0520 msgr2=0x7f0aec0a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:15.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:15.260+0000 7f0b11200700 1 --2- 172.21.15.1:0/3204368829 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f0aec0a0520 0x7f0aec0a29e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0afc009d20 tx=0x7f0afc009260 comp rx=0 tx=0).stop 2024-02-02T15:25:15.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:15.260+0000 7f0b11200700 1 -- 172.21.15.1:0/3204368829 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0b0c102930 msgr2=0x7f0b0c1a03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:15.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:15.260+0000 7f0b11200700 1 --2- 172.21.15.1:0/3204368829 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0b0c102930 0x7f0b0c1a03a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f0b0400b070 tx=0x7f0b0400bc80 comp rx=0 tx=0).stop 2024-02-02T15:25:15.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:15.260+0000 7f0b11200700 1 -- 172.21.15.1:0/3204368829 shutdown_connections 2024-02-02T15:25:15.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:15.260+0000 7f0b11200700 1 --2- 172.21.15.1:0/3204368829 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0b0c102930 0x7f0b0c1a03a0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:15.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:15.260+0000 7f0b11200700 1 --2- 172.21.15.1:0/3204368829 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f0aec0a0520 0x7f0aec0a29e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:15.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:15.260+0000 7f0b11200700 1 --2- 172.21.15.1:0/3204368829 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b0c100e20 0x7f0b0c198df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:15.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:15.260+0000 7f0b11200700 1 --2- 172.21.15.1:0/3204368829 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b0c101f70 0x7f0b0c199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:15.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:15.260+0000 7f0b11200700 1 -- 172.21.15.1:0/3204368829 >> 172.21.15.1:0/3204368829 conn(0x7f0b0c0fc390 msgr2=0x7f0b0c0fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:15.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:15.261+0000 7f0b11200700 1 -- 172.21.15.1:0/3204368829 shutdown_connections 2024-02-02T15:25:15.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:15.261+0000 7f0b11200700 1 -- 172.21.15.1:0/3204368829 wait complete. 2024-02-02T15:25:15.281 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-02-02T15:25:15.281 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-02-02T15:25:15.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:15.486+0000 7f1a3c200700 1 -- 172.21.15.1:0/484298723 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1a34100e20 msgr2=0x7f1a34101240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:15.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:15.486+0000 7f1a3c200700 1 --2- 172.21.15.1:0/484298723 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1a34100e20 0x7f1a34101240 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1a24009990 tx=0x7f1a24009ca0 comp rx=0 tx=0).stop 2024-02-02T15:25:15.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:15.486+0000 7f1a3c200700 1 -- 172.21.15.1:0/484298723 shutdown_connections 2024-02-02T15:25:15.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:15.486+0000 7f1a3c200700 1 --2- 172.21.15.1:0/484298723 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a34102930 0x7f1a34109250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:15.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:15.486+0000 7f1a3c200700 1 --2- 172.21.15.1:0/484298723 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1a34100e20 0x7f1a34101240 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:15.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:15.486+0000 7f1a3c200700 1 --2- 172.21.15.1:0/484298723 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a34101f70 0x7f1a341023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:15.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:15.486+0000 7f1a3c200700 1 -- 172.21.15.1:0/484298723 >> 172.21.15.1:0/484298723 conn(0x7f1a340fc390 msgr2=0x7f1a340fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:15.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:15.486+0000 7f1a3c200700 1 -- 172.21.15.1:0/484298723 shutdown_connections 2024-02-02T15:25:15.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:15.487+0000 7f1a3c200700 1 -- 172.21.15.1:0/484298723 wait complete. 2024-02-02T15:25:15.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:15.487+0000 7f1a3c200700 1 Processor -- start 2024-02-02T15:25:15.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:15.488+0000 7f1a3c200700 1 -- start start 2024-02-02T15:25:15.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:15.488+0000 7f1a3c200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1a34100e20 0x7f1a34074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:15.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:15.488+0000 7f1a3c200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a34101f70 0x7f1a34073130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:15.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:15.488+0000 7f1a3c200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a34102930 0x7f1a34073670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:15.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:15.488+0000 7f1a3c200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a34073e10 con 0x7f1a34100e20 2024-02-02T15:25:15.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:15.488+0000 7f1a3c200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a34073f80 con 0x7f1a34101f70 2024-02-02T15:25:15.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:15.488+0000 7f1a3c200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a341a0b30 con 0x7f1a34102930 2024-02-02T15:25:15.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:15.488+0000 7f1a39800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1a34100e20 0x7f1a34074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:15.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:15.488+0000 7f1a39800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1a34100e20 0x7f1a34074af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:32990/0 (socket says 172.21.15.1:32990) 2024-02-02T15:25:15.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:15.488+0000 7f1a39800700 1 -- 172.21.15.1:0/112922513 learned_addr learned my addr 172.21.15.1:0/112922513 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:25:15.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:15.488+0000 7f1a38e00700 1 --2- 172.21.15.1:0/112922513 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a34101f70 0x7f1a34073130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:15.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:15.488+0000 7f1a3a200700 1 --2- 172.21.15.1:0/112922513 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a34102930 0x7f1a34073670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:15.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:15.489+0000 7f1a39800700 1 -- 172.21.15.1:0/112922513 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a34102930 msgr2=0x7f1a34073670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:15.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:15.489+0000 7f1a39800700 1 --2- 172.21.15.1:0/112922513 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a34102930 0x7f1a34073670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:15.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:15.489+0000 7f1a39800700 1 -- 172.21.15.1:0/112922513 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a34101f70 msgr2=0x7f1a34073130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:15.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:15.489+0000 7f1a39800700 1 --2- 172.21.15.1:0/112922513 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a34101f70 0x7f1a34073130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:15.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:15.489+0000 7f1a39800700 1 -- 172.21.15.1:0/112922513 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a2c009c40 con 0x7f1a34100e20 2024-02-02T15:25:15.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:15.489+0000 7f1a39800700 1 --2- 172.21.15.1:0/112922513 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1a34100e20 0x7f1a34074af0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1a2400b620 tx=0x7f1a24005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:15.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:15.489+0000 7f1a32000700 1 -- 172.21.15.1:0/112922513 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a2401d070 con 0x7f1a34100e20 2024-02-02T15:25:15.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:15.489+0000 7f1a3c200700 1 -- 172.21.15.1:0/112922513 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a240095f0 con 0x7f1a34100e20 2024-02-02T15:25:15.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:15.489+0000 7f1a32000700 1 -- 172.21.15.1:0/112922513 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1a24022470 con 0x7f1a34100e20 2024-02-02T15:25:15.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:15.489+0000 7f1a32000700 1 -- 172.21.15.1:0/112922513 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a24017650 con 0x7f1a34100e20 2024-02-02T15:25:15.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:15.489+0000 7f1a3c200700 1 -- 172.21.15.1:0/112922513 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a341a1040 con 0x7f1a34100e20 2024-02-02T15:25:15.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:15.490+0000 7f1a3c200700 1 -- 172.21.15.1:0/112922513 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a341083f0 con 0x7f1a34100e20 2024-02-02T15:25:15.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:15.492+0000 7f1a32000700 1 -- 172.21.15.1:0/112922513 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) v1 ==== 135069+0+0 (secure 0 0 0) 0x7f1a240225e0 con 0x7f1a34100e20 2024-02-02T15:25:15.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:15.493+0000 7f1a32000700 1 --2- 172.21.15.1:0/112922513 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f1a140a0520 0x7f1a140a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:15.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:15.494+0000 7f1a32000700 1 -- 172.21.15.1:0/112922513 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f1a240ccfe0 con 0x7f1a34100e20 2024-02-02T15:25:15.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:15.494+0000 7f1a38e00700 1 --2- 172.21.15.1:0/112922513 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f1a140a0520 0x7f1a140a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:15.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:15.494+0000 7f1a38e00700 1 --2- 172.21.15.1:0/112922513 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f1a140a0520 0x7f1a140a29e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c009c60 tx=0x7f1a1c009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:15.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:15.497+0000 7f1a32000700 1 -- 172.21.15.1:0/112922513 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f1a24096b40 con 0x7f1a34100e20 2024-02-02T15:25:15.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:15 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/4082597440' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-02T15:25:15.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:15 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/3204368829' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-02-02T15:25:15.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:15 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/4082597440' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-02T15:25:15.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:15 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/3204368829' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-02-02T15:25:15.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:15 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/4082597440' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-02T15:25:15.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:15 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/3204368829' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-02-02T15:25:15.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:15.756+0000 7f1a3c200700 1 -- 172.21.15.1:0/112922513 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) v1 -- 0x7f1a34061950 con 0x7f1a34100e20 2024-02-02T15:25:16.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:16.208+0000 7f1a32000700 1 -- 172.21.15.1:0/112922513 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v19) v1 ==== 105+0+0 (secure 0 0 0) 0x7f1a24027020 con 0x7f1a34100e20 2024-02-02T15:25:16.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:16.211+0000 7f1a3c200700 1 -- 172.21.15.1:0/112922513 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f1a140a0520 msgr2=0x7f1a140a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:16.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:16.211+0000 7f1a3c200700 1 --2- 172.21.15.1:0/112922513 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f1a140a0520 0x7f1a140a29e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1a1c009c60 tx=0x7f1a1c009260 comp rx=0 tx=0).stop 2024-02-02T15:25:16.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:16.211+0000 7f1a3c200700 1 -- 172.21.15.1:0/112922513 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1a34100e20 msgr2=0x7f1a34074af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:16.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:16.211+0000 7f1a3c200700 1 --2- 172.21.15.1:0/112922513 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1a34100e20 0x7f1a34074af0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1a2400b620 tx=0x7f1a24005740 comp rx=0 tx=0).stop 2024-02-02T15:25:16.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:16.213+0000 7f1a3c200700 1 -- 172.21.15.1:0/112922513 shutdown_connections 2024-02-02T15:25:16.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:16.213+0000 7f1a3c200700 1 --2- 172.21.15.1:0/112922513 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f1a140a0520 0x7f1a140a29e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:16.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:16.213+0000 7f1a3c200700 1 --2- 172.21.15.1:0/112922513 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1a34102930 0x7f1a34073670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:16.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:16.213+0000 7f1a3c200700 1 --2- 172.21.15.1:0/112922513 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1a34100e20 0x7f1a34074af0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:16.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:16.213+0000 7f1a3c200700 1 --2- 172.21.15.1:0/112922513 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1a34101f70 0x7f1a34073130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:16.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:16.213+0000 7f1a3c200700 1 -- 172.21.15.1:0/112922513 >> 172.21.15.1:0/112922513 conn(0x7f1a340fc390 msgr2=0x7f1a34105530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:16.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:16.214+0000 7f1a3c200700 1 -- 172.21.15.1:0/112922513 shutdown_connections 2024-02-02T15:25:16.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:16.214+0000 7f1a3c200700 1 -- 172.21.15.1:0/112922513 wait complete. 2024-02-02T15:25:16.236 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-02-02T15:25:16.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:16.440+0000 7fa7c8c00700 1 -- 172.21.15.1:0/1944474288 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa7c4101f70 msgr2=0x7fa7c41023f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:16.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:16.440+0000 7fa7c8c00700 1 --2- 172.21.15.1:0/1944474288 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa7c4101f70 0x7fa7c41023f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac009990 tx=0x7fa7ac009ca0 comp rx=0 tx=0).stop 2024-02-02T15:25:16.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:16.440+0000 7fa7c8c00700 1 -- 172.21.15.1:0/1944474288 shutdown_connections 2024-02-02T15:25:16.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:16.440+0000 7fa7c8c00700 1 --2- 172.21.15.1:0/1944474288 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7c4102930 0x7fa7c4109250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:16.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:16.440+0000 7fa7c8c00700 1 --2- 172.21.15.1:0/1944474288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7c4100e20 0x7fa7c4101240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:16.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:16.440+0000 7fa7c8c00700 1 --2- 172.21.15.1:0/1944474288 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa7c4101f70 0x7fa7c41023f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:16.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:16.440+0000 7fa7c8c00700 1 -- 172.21.15.1:0/1944474288 >> 172.21.15.1:0/1944474288 conn(0x7fa7c40fc390 msgr2=0x7fa7c40fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:16.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:16.440+0000 7fa7c8c00700 1 -- 172.21.15.1:0/1944474288 shutdown_connections 2024-02-02T15:25:16.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:16.440+0000 7fa7c8c00700 1 -- 172.21.15.1:0/1944474288 wait complete. 2024-02-02T15:25:16.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:16.441+0000 7fa7c8c00700 1 Processor -- start 2024-02-02T15:25:16.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:16.441+0000 7fa7c8c00700 1 -- start start 2024-02-02T15:25:16.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:16.441+0000 7fa7c8c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7c4100e20 0x7fa7c419d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:16.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:16.441+0000 7fa7c8c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7c4101f70 0x7fa7c419d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:16.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:16.441+0000 7fa7c8c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa7c4102930 0x7fa7c41a46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:16.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:16.442+0000 7fa7c8c00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c41a4c80 con 0x7fa7c4102930 2024-02-02T15:25:16.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:16.442+0000 7fa7c8c00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c41a4dc0 con 0x7fa7c4101f70 2024-02-02T15:25:16.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:16.442+0000 7fa7c8c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c41a4f50 con 0x7fa7c4100e20 2024-02-02T15:25:16.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:16.442+0000 7fa7c2800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa7c4102930 0x7fa7c41a46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:16.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:16.442+0000 7fa7c1e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7c4100e20 0x7fa7c419d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:16.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:16.442+0000 7fa7c2800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa7c4102930 0x7fa7c41a46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:32998/0 (socket says 172.21.15.1:32998) 2024-02-02T15:25:16.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:16.442+0000 7fa7c2800700 1 -- 172.21.15.1:0/3328853055 learned_addr learned my addr 172.21.15.1:0/3328853055 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:25:16.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:16.442+0000 7fa7c1400700 1 --2- 172.21.15.1:0/3328853055 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7c4101f70 0x7fa7c419d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:16.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:16.442+0000 7fa7c1e00700 1 -- 172.21.15.1:0/3328853055 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7c4101f70 msgr2=0x7fa7c419d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:16.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:16.442+0000 7fa7c1e00700 1 --2- 172.21.15.1:0/3328853055 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7c4101f70 0x7fa7c419d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:16.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:16.442+0000 7fa7c1e00700 1 -- 172.21.15.1:0/3328853055 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa7c4102930 msgr2=0x7fa7c41a46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:16.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:16.442+0000 7fa7c1e00700 1 --2- 172.21.15.1:0/3328853055 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa7c4102930 0x7fa7c41a46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:16.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:16.442+0000 7fa7c1e00700 1 -- 172.21.15.1:0/3328853055 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7ac0095f0 con 0x7fa7c4100e20 2024-02-02T15:25:16.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:16.442+0000 7fa7c1400700 1 --2- 172.21.15.1:0/3328853055 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7c4101f70 0x7fa7c419d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:25:16.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:16.442+0000 7fa7c2800700 1 --2- 172.21.15.1:0/3328853055 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa7c4102930 0x7fa7c41a46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:25:16.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:16.442+0000 7fa7c1e00700 1 --2- 172.21.15.1:0/3328853055 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7c4100e20 0x7fa7c419d100 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa7b400e9f0 tx=0x7fa7b400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:16.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:16.443+0000 7fa7baa00700 1 -- 172.21.15.1:0/3328853055 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b40096d0 con 0x7fa7c4100e20 2024-02-02T15:25:16.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:16.443+0000 7fa7c8c00700 1 -- 172.21.15.1:0/3328853055 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7c41a5230 con 0x7fa7c4100e20 2024-02-02T15:25:16.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:16.443+0000 7fa7baa00700 1 -- 172.21.15.1:0/3328853055 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa7b400cbd0 con 0x7fa7c4100e20 2024-02-02T15:25:16.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:16.443+0000 7fa7baa00700 1 -- 172.21.15.1:0/3328853055 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b4018680 con 0x7fa7c4100e20 2024-02-02T15:25:16.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:16.443+0000 7fa7c8c00700 1 -- 172.21.15.1:0/3328853055 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7c41a5790 con 0x7fa7c4100e20 2024-02-02T15:25:16.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:16.444+0000 7fa7c8c00700 1 -- 172.21.15.1:0/3328853055 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7c4105d30 con 0x7fa7c4100e20 2024-02-02T15:25:16.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:16.446+0000 7fa7baa00700 1 -- 172.21.15.1:0/3328853055 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 16) v1 ==== 135069+0+0 (secure 0 0 0) 0x7fa7b400cd40 con 0x7fa7c4100e20 2024-02-02T15:25:16.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:16.447+0000 7fa7baa00700 1 --2- 172.21.15.1:0/3328853055 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fa7a40a0520 0x7fa7a40a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:16.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:16.448+0000 7fa7baa00700 1 -- 172.21.15.1:0/3328853055 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fa7b4014070 con 0x7fa7c4100e20 2024-02-02T15:25:16.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:16.448+0000 7fa7c1400700 1 --2- 172.21.15.1:0/3328853055 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fa7a40a0520 0x7fa7a40a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:16.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:16.448+0000 7fa7c1400700 1 --2- 172.21.15.1:0/3328853055 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fa7a40a0520 0x7fa7a40a29e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac000c00 tx=0x7fa7ac005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:16.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:16.450+0000 7fa7baa00700 1 -- 172.21.15.1:0/3328853055 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7fa7b4095970 con 0x7fa7c4100e20 2024-02-02T15:25:16.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:16 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/3204368829' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-02-02T15:25:16.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:16 smithi071 ceph-mon[44707]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-02T15:25:16.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:16 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/112922513' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-02-02T15:25:16.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:16 smithi071 ceph-mon[44707]: pgmap v249: 129 pgs: 129 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-02-02T15:25:16.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:16 smithi071 ceph-mon[44707]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-02-02T15:25:16.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:16 smithi071 ceph-mon[44707]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-02-02T15:25:16.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:16 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/112922513' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-02-02T15:25:16.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:16 smithi071 ceph-mon[44707]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-02-02T15:25:16.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:16 smithi071 ceph-mon[44707]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-02-02T15:25:16.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:16 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/3204368829' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-02-02T15:25:16.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:16 smithi142 ceph-mon[45728]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-02T15:25:16.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:16 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/112922513' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-02-02T15:25:16.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:16 smithi142 ceph-mon[45728]: pgmap v249: 129 pgs: 129 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-02-02T15:25:16.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:16 smithi142 ceph-mon[45728]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-02-02T15:25:16.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:16 smithi142 ceph-mon[45728]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-02-02T15:25:16.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:16 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/112922513' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-02-02T15:25:16.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:16 smithi142 ceph-mon[45728]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-02-02T15:25:16.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:16 smithi142 ceph-mon[45728]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-02-02T15:25:16.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:16 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/3204368829' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-02-02T15:25:16.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:16 smithi001 ceph-mon[35470]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-02-02T15:25:16.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:16 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/112922513' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-02-02T15:25:16.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:16 smithi001 ceph-mon[35470]: pgmap v249: 129 pgs: 129 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-02-02T15:25:16.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:16 smithi001 ceph-mon[35470]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-02-02T15:25:16.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:16 smithi001 ceph-mon[35470]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-02-02T15:25:16.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:16 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/112922513' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-02-02T15:25:16.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:16 smithi001 ceph-mon[35470]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-02-02T15:25:16.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:16 smithi001 ceph-mon[35470]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-02-02T15:25:16.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:16.710+0000 7fa7c8c00700 1 -- 172.21.15.1:0/3328853055 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7fa7c402cc70 con 0x7fa7c4100e20 2024-02-02T15:25:17.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:17.215+0000 7fa7baa00700 1 -- 172.21.15.1:0/3328853055 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v20) v1 ==== 121+0+0 (secure 0 0 0) 0x7fa7b4098f90 con 0x7fa7c4100e20 2024-02-02T15:25:17.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:17.218+0000 7fa7c8c00700 1 -- 172.21.15.1:0/3328853055 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fa7a40a0520 msgr2=0x7fa7a40a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:17.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:17.218+0000 7fa7c8c00700 1 --2- 172.21.15.1:0/3328853055 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fa7a40a0520 0x7fa7a40a29e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac000c00 tx=0x7fa7ac005fb0 comp rx=0 tx=0).stop 2024-02-02T15:25:17.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:17.218+0000 7fa7c8c00700 1 -- 172.21.15.1:0/3328853055 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7c4100e20 msgr2=0x7fa7c419d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:17.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:17.218+0000 7fa7c8c00700 1 --2- 172.21.15.1:0/3328853055 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7c4100e20 0x7fa7c419d100 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa7b400e9f0 tx=0x7fa7b400ed00 comp rx=0 tx=0).stop 2024-02-02T15:25:17.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:17.218+0000 7fa7c8c00700 1 -- 172.21.15.1:0/3328853055 shutdown_connections 2024-02-02T15:25:17.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:17.218+0000 7fa7c8c00700 1 --2- 172.21.15.1:0/3328853055 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa7c4102930 0x7fa7c41a46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:17.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:17.218+0000 7fa7c8c00700 1 --2- 172.21.15.1:0/3328853055 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fa7a40a0520 0x7fa7a40a29e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:17.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:17.218+0000 7fa7c8c00700 1 --2- 172.21.15.1:0/3328853055 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa7c4100e20 0x7fa7c419d100 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:17.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:17.218+0000 7fa7c8c00700 1 --2- 172.21.15.1:0/3328853055 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7c4101f70 0x7fa7c419d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:17.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:17.218+0000 7fa7c8c00700 1 -- 172.21.15.1:0/3328853055 >> 172.21.15.1:0/3328853055 conn(0x7fa7c40fc390 msgr2=0x7fa7c40fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:17.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:17.219+0000 7fa7c8c00700 1 -- 172.21.15.1:0/3328853055 shutdown_connections 2024-02-02T15:25:17.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:17.219+0000 7fa7c8c00700 1 -- 172.21.15.1:0/3328853055 wait complete. 2024-02-02T15:25:17.242 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-02-02T15:25:17.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:17.444+0000 7fda85e00700 1 -- 172.21.15.1:0/3906811500 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda80100e30 msgr2=0x7fda80101250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:17.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:17.444+0000 7fda85e00700 1 --2- 172.21.15.1:0/3906811500 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda80100e30 0x7fda80101250 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fda70009990 tx=0x7fda70009ca0 comp rx=0 tx=0).stop 2024-02-02T15:25:17.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:17.444+0000 7fda85e00700 1 -- 172.21.15.1:0/3906811500 shutdown_connections 2024-02-02T15:25:17.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:17.444+0000 7fda85e00700 1 --2- 172.21.15.1:0/3906811500 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fda80102940 0x7fda80109260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:17.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:17.444+0000 7fda85e00700 1 --2- 172.21.15.1:0/3906811500 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda80100e30 0x7fda80101250 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:17.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:17.444+0000 7fda85e00700 1 --2- 172.21.15.1:0/3906811500 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda80101f80 0x7fda80102400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:17.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:17.444+0000 7fda85e00700 1 -- 172.21.15.1:0/3906811500 >> 172.21.15.1:0/3906811500 conn(0x7fda800fc3a0 msgr2=0x7fda800fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:17.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:17.444+0000 7fda85e00700 1 -- 172.21.15.1:0/3906811500 shutdown_connections 2024-02-02T15:25:17.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:17.444+0000 7fda85e00700 1 -- 172.21.15.1:0/3906811500 wait complete. 2024-02-02T15:25:17.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:17.445+0000 7fda85e00700 1 Processor -- start 2024-02-02T15:25:17.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:17.446+0000 7fda85e00700 1 -- start start 2024-02-02T15:25:17.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:17.446+0000 7fda85e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda80100e30 0x7fda80074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:17.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:17.446+0000 7fda85e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fda80101f80 0x7fda80073130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:17.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:17.446+0000 7fda85e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda80102940 0x7fda80073670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:17.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:17.446+0000 7fda85e00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda80073e10 con 0x7fda80100e30 2024-02-02T15:25:17.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:17.446+0000 7fda85e00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda80073f80 con 0x7fda80101f80 2024-02-02T15:25:17.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:17.446+0000 7fda85e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda801a0b80 con 0x7fda80102940 2024-02-02T15:25:17.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:17.446+0000 7fda7ea00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda80100e30 0x7fda80074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:17.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:17.446+0000 7fda7ea00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda80100e30 0x7fda80074af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33020/0 (socket says 172.21.15.1:33020) 2024-02-02T15:25:17.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:17.446+0000 7fda7ea00700 1 -- 172.21.15.1:0/3473384249 learned_addr learned my addr 172.21.15.1:0/3473384249 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:25:17.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:17.446+0000 7fda7e000700 1 --2- 172.21.15.1:0/3473384249 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fda80101f80 0x7fda80073130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:17.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:17.446+0000 7fda7f400700 1 --2- 172.21.15.1:0/3473384249 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda80102940 0x7fda80073670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:17.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:17.447+0000 7fda7ea00700 1 -- 172.21.15.1:0/3473384249 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda80102940 msgr2=0x7fda80073670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:17.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:17.447+0000 7fda7ea00700 1 --2- 172.21.15.1:0/3473384249 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda80102940 0x7fda80073670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:17.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:17.447+0000 7fda7ea00700 1 -- 172.21.15.1:0/3473384249 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fda80101f80 msgr2=0x7fda80073130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:17.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:17.447+0000 7fda7ea00700 1 --2- 172.21.15.1:0/3473384249 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fda80101f80 0x7fda80073130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:17.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:17.447+0000 7fda7ea00700 1 -- 172.21.15.1:0/3473384249 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda700095f0 con 0x7fda80100e30 2024-02-02T15:25:17.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:17.447+0000 7fda7ea00700 1 --2- 172.21.15.1:0/3473384249 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda80100e30 0x7fda80074af0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fda7000b620 tx=0x7fda70005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:17.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:17.447+0000 7fda84a00700 1 -- 172.21.15.1:0/3473384249 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda7001d070 con 0x7fda80100e30 2024-02-02T15:25:17.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:17.447+0000 7fda84a00700 1 -- 172.21.15.1:0/3473384249 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda70022470 con 0x7fda80100e30 2024-02-02T15:25:17.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:17.447+0000 7fda85e00700 1 -- 172.21.15.1:0/3473384249 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda801a0e00 con 0x7fda80100e30 2024-02-02T15:25:17.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:17.447+0000 7fda84a00700 1 -- 172.21.15.1:0/3473384249 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda70017650 con 0x7fda80100e30 2024-02-02T15:25:17.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:17.447+0000 7fda85e00700 1 -- 172.21.15.1:0/3473384249 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda801a11d0 con 0x7fda80100e30 2024-02-02T15:25:17.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:17.448+0000 7fda85e00700 1 -- 172.21.15.1:0/3473384249 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda8010c680 con 0x7fda80100e30 2024-02-02T15:25:17.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:17.450+0000 7fda84a00700 1 -- 172.21.15.1:0/3473384249 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) v1 ==== 135069+0+0 (secure 0 0 0) 0x7fda700225e0 con 0x7fda80100e30 2024-02-02T15:25:17.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:17.452+0000 7fda84a00700 1 --2- 172.21.15.1:0/3473384249 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fda600a05a0 0x7fda600a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:17.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:17.453+0000 7fda84a00700 1 -- 172.21.15.1:0/3473384249 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fda700cd1b0 con 0x7fda80100e30 2024-02-02T15:25:17.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:17.453+0000 7fda7e000700 1 --2- 172.21.15.1:0/3473384249 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fda600a05a0 0x7fda600a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:17.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:17.453+0000 7fda7e000700 1 --2- 172.21.15.1:0/3473384249 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fda600a05a0 0x7fda600a2a60 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fda68009c60 tx=0x7fda68009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:17.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:17.457+0000 7fda84a00700 1 -- 172.21.15.1:0/3473384249 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7fda70096b90 con 0x7fda80100e30 2024-02-02T15:25:17.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:17 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/3328853055' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-02-02T15:25:17.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:17 smithi071 ceph-mon[44707]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-02-02T15:25:17.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:17 smithi071 ceph-mon[44707]: mds.? [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] up:active 2024-02-02T15:25:17.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:17 smithi071 ceph-mon[44707]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-02-02T15:25:17.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:17 smithi071 ceph-mon[44707]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-02-02T15:25:17.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:17 smithi071 ceph-mon[44707]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-02-02T15:25:17.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:17 smithi071 ceph-mon[44707]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-02-02T15:25:17.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:17 smithi071 ceph-mon[44707]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-02-02T15:25:17.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:17 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/3328853055' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-02-02T15:25:17.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:17 smithi142 ceph-mon[45728]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-02-02T15:25:17.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:17 smithi142 ceph-mon[45728]: mds.? [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] up:active 2024-02-02T15:25:17.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:17 smithi142 ceph-mon[45728]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-02-02T15:25:17.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:17 smithi142 ceph-mon[45728]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-02-02T15:25:17.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:17 smithi142 ceph-mon[45728]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-02-02T15:25:17.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:17 smithi142 ceph-mon[45728]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-02-02T15:25:17.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:17 smithi142 ceph-mon[45728]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-02-02T15:25:17.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:17 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/3328853055' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-02-02T15:25:17.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:17 smithi001 ceph-mon[35470]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-02-02T15:25:17.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:17 smithi001 ceph-mon[35470]: mds.? [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] up:active 2024-02-02T15:25:17.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:17 smithi001 ceph-mon[35470]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-02-02T15:25:17.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:17 smithi001 ceph-mon[35470]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-02-02T15:25:17.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:17 smithi001 ceph-mon[35470]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-02-02T15:25:17.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:17 smithi001 ceph-mon[35470]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-02-02T15:25:17.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:17 smithi001 ceph-mon[35470]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-02-02T15:25:17.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:17.713+0000 7fda85e00700 1 -- 172.21.15.1:0/3473384249 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7fda80107d00 con 0x7fda80100e30 2024-02-02T15:25:18.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:18.221+0000 7fda84a00700 1 -- 172.21.15.1:0/3473384249 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v22) v1 ==== 115+0+0 (secure 0 0 0) 0x7fda7009a1b0 con 0x7fda80100e30 2024-02-02T15:25:18.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:18.224+0000 7fda85e00700 1 -- 172.21.15.1:0/3473384249 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fda600a05a0 msgr2=0x7fda600a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:18.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:18.224+0000 7fda85e00700 1 --2- 172.21.15.1:0/3473384249 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fda600a05a0 0x7fda600a2a60 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fda68009c60 tx=0x7fda68009260 comp rx=0 tx=0).stop 2024-02-02T15:25:18.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:18.224+0000 7fda85e00700 1 -- 172.21.15.1:0/3473384249 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda80100e30 msgr2=0x7fda80074af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:18.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:18.224+0000 7fda85e00700 1 --2- 172.21.15.1:0/3473384249 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda80100e30 0x7fda80074af0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fda7000b620 tx=0x7fda70005740 comp rx=0 tx=0).stop 2024-02-02T15:25:18.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:18.224+0000 7fda85e00700 1 -- 172.21.15.1:0/3473384249 shutdown_connections 2024-02-02T15:25:18.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:18.224+0000 7fda85e00700 1 --2- 172.21.15.1:0/3473384249 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fda600a05a0 0x7fda600a2a60 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:18.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:18.224+0000 7fda85e00700 1 --2- 172.21.15.1:0/3473384249 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda80102940 0x7fda80073670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:18.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:18.224+0000 7fda85e00700 1 --2- 172.21.15.1:0/3473384249 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda80100e30 0x7fda80074af0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:18.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:18.224+0000 7fda85e00700 1 --2- 172.21.15.1:0/3473384249 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fda80101f80 0x7fda80073130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:18.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:18.224+0000 7fda85e00700 1 -- 172.21.15.1:0/3473384249 >> 172.21.15.1:0/3473384249 conn(0x7fda800fc3a0 msgr2=0x7fda800fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:18.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:18.224+0000 7fda85e00700 1 -- 172.21.15.1:0/3473384249 shutdown_connections 2024-02-02T15:25:18.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:18.224+0000 7fda85e00700 1 -- 172.21.15.1:0/3473384249 wait complete. 2024-02-02T15:25:18.251 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-02-02T15:25:18.252 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 '' 2024-02-02T15:25:18.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:18.455+0000 7f408e200700 1 -- 172.21.15.1:0/21720877 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4088108ce0 msgr2=0x7f408810b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:18.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:18.455+0000 7f408e200700 1 --2- 172.21.15.1:0/21720877 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4088108ce0 0x7f408810b170 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f408000b0a0 tx=0x7f408000b3b0 comp rx=0 tx=0).stop 2024-02-02T15:25:18.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:18.455+0000 7f408e200700 1 -- 172.21.15.1:0/21720877 shutdown_connections 2024-02-02T15:25:18.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:18.455+0000 7f408e200700 1 --2- 172.21.15.1:0/21720877 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4088108ce0 0x7f408810b170 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:18.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:18.455+0000 7f408e200700 1 --2- 172.21.15.1:0/21720877 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4088074d80 0x7f40880731d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:18.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:18.455+0000 7f408e200700 1 --2- 172.21.15.1:0/21720877 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40880737a0 0x7f4088073c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:18.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:18.455+0000 7f408e200700 1 -- 172.21.15.1:0/21720877 >> 172.21.15.1:0/21720877 conn(0x7f40880fc390 msgr2=0x7f40880fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:18.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:18.455+0000 7f408e200700 1 -- 172.21.15.1:0/21720877 shutdown_connections 2024-02-02T15:25:18.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:18.455+0000 7f408e200700 1 -- 172.21.15.1:0/21720877 wait complete. 2024-02-02T15:25:18.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:18.456+0000 7f408e200700 1 Processor -- start 2024-02-02T15:25:18.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:18.457+0000 7f408e200700 1 -- start start 2024-02-02T15:25:18.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:18.457+0000 7f408e200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40880737a0 0x7f4088198df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:18.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:18.457+0000 7f408e200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4088074d80 0x7f4088199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:18.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:18.457+0000 7f408e200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4088108ce0 0x7f40881a03a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:18.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:18.457+0000 7f408e200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40881a0970 con 0x7f4088108ce0 2024-02-02T15:25:18.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:18.457+0000 7f408e200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40881a0ab0 con 0x7f4088074d80 2024-02-02T15:25:18.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:18.457+0000 7f408e200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40881a0c40 con 0x7f40880737a0 2024-02-02T15:25:18.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:18.457+0000 7f4087e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4088108ce0 0x7f40881a03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:18.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:18.457+0000 7f4087400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40880737a0 0x7f4088198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:18.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:18.457+0000 7f4087e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4088108ce0 0x7f40881a03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33028/0 (socket says 172.21.15.1:33028) 2024-02-02T15:25:18.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:18.457+0000 7f4087e00700 1 -- 172.21.15.1:0/187011424 learned_addr learned my addr 172.21.15.1:0/187011424 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:25:18.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:18.457+0000 7f4086a00700 1 --2- 172.21.15.1:0/187011424 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4088074d80 0x7f4088199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:18.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:18.458+0000 7f4087e00700 1 -- 172.21.15.1:0/187011424 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40880737a0 msgr2=0x7f4088198df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:18.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:18.458+0000 7f4087e00700 1 --2- 172.21.15.1:0/187011424 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40880737a0 0x7f4088198df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:18.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:18.458+0000 7f4087e00700 1 -- 172.21.15.1:0/187011424 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4088074d80 msgr2=0x7f4088199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:18.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:18.458+0000 7f4087e00700 1 --2- 172.21.15.1:0/187011424 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4088074d80 0x7f4088199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:18.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:18.458+0000 7f4087e00700 1 -- 172.21.15.1:0/187011424 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4080009d00 con 0x7f4088108ce0 2024-02-02T15:25:18.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:18.458+0000 7f4087400700 1 --2- 172.21.15.1:0/187011424 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40880737a0 0x7f4088198df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:25:18.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:18.458+0000 7f4086a00700 1 --2- 172.21.15.1:0/187011424 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4088074d80 0x7f4088199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:25:18.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:18.458+0000 7f4087e00700 1 --2- 172.21.15.1:0/187011424 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4088108ce0 0x7f40881a03a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4080009fd0 tx=0x7f408000bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:18.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:18.458+0000 7f407fe00700 1 -- 172.21.15.1:0/187011424 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f408000e040 con 0x7f4088108ce0 2024-02-02T15:25:18.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:18.458+0000 7f407fe00700 1 -- 172.21.15.1:0/187011424 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f40800094a0 con 0x7f4088108ce0 2024-02-02T15:25:18.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:18.458+0000 7f407fe00700 1 -- 172.21.15.1:0/187011424 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f408001b6c0 con 0x7f4088108ce0 2024-02-02T15:25:18.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:18.458+0000 7f408e200700 1 -- 172.21.15.1:0/187011424 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40881a0ec0 con 0x7f4088108ce0 2024-02-02T15:25:18.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:18.458+0000 7f408e200700 1 -- 172.21.15.1:0/187011424 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40881a1230 con 0x7f4088108ce0 2024-02-02T15:25:18.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:18.459+0000 7f408e200700 1 -- 172.21.15.1:0/187011424 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4088107390 con 0x7f4088108ce0 2024-02-02T15:25:18.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:18.461+0000 7f407fe00700 1 -- 172.21.15.1:0/187011424 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) v1 ==== 135069+0+0 (secure 0 0 0) 0x7f408001d470 con 0x7f4088108ce0 2024-02-02T15:25:18.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:18.462+0000 7f407fe00700 1 --2- 172.21.15.1:0/187011424 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f40680a0520 0x7f40680a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:18.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:18.463+0000 7f407fe00700 1 -- 172.21.15.1:0/187011424 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f40800ccdc0 con 0x7f4088108ce0 2024-02-02T15:25:18.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:18.463+0000 7f4087400700 1 --2- 172.21.15.1:0/187011424 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f40680a0520 0x7f40680a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:18.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:18.463+0000 7f4087400700 1 --2- 172.21.15.1:0/187011424 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f40680a0520 0x7f40680a29e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4078011490 tx=0x7f4078009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:18.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:18.467+0000 7f407fe00700 1 -- 172.21.15.1:0/187011424 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f40800967a0 con 0x7f4088108ce0 2024-02-02T15:25:18.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:18 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/3473384249' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-02-02T15:25:18.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:18 smithi071 ceph-mon[44707]: pgmap v250: 129 pgs: 129 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-02-02T15:25:18.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:18 smithi071 ceph-mon[44707]: mds.? [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] up:active 2024-02-02T15:25:18.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:18 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/3473384249' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-02-02T15:25:18.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:18 smithi071 ceph-mon[44707]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-02-02T15:25:18.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:18 smithi071 ceph-mon[44707]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-02-02T15:25:18.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:18 smithi071 ceph-mon[44707]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-02-02T15:25:18.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:18 smithi071 ceph-mon[44707]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-02-02T15:25:18.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:18 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/3473384249' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-02-02T15:25:18.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:18 smithi142 ceph-mon[45728]: pgmap v250: 129 pgs: 129 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-02-02T15:25:18.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:18 smithi142 ceph-mon[45728]: mds.? [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] up:active 2024-02-02T15:25:18.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:18 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/3473384249' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-02-02T15:25:18.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:18 smithi142 ceph-mon[45728]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-02-02T15:25:18.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:18 smithi142 ceph-mon[45728]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-02-02T15:25:18.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:18 smithi142 ceph-mon[45728]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-02-02T15:25:18.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:18 smithi142 ceph-mon[45728]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-02-02T15:25:18.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:18 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/3473384249' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-02-02T15:25:18.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:18 smithi001 ceph-mon[35470]: pgmap v250: 129 pgs: 129 active+clean; 581 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-02-02T15:25:18.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:18 smithi001 ceph-mon[35470]: mds.? [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] up:active 2024-02-02T15:25:18.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:18 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/3473384249' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-02-02T15:25:18.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:18 smithi001 ceph-mon[35470]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-02-02T15:25:18.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:18 smithi001 ceph-mon[35470]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-02-02T15:25:18.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:18 smithi001 ceph-mon[35470]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-02-02T15:25:18.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:18 smithi001 ceph-mon[35470]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-02-02T15:25:18.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:18.713+0000 7f408e200700 1 -- 172.21.15.1:0/187011424 --> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7f4088061210 con 0x7f40680a0520 2024-02-02T15:25:19.190 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:25:18 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:25:18.719+0000 7fd82cc00700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-02T15:25:19.190 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:25:18 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:25:18.719+0000 7fd82cc00700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-02T15:25:19.190 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:25:18 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:25:18.719+0000 7fd82cc00700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-02T15:25:19.190 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:25:18 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:25:18.719+0000 7fd82cc00700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-02T15:25:19.190 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:25:18 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:25:18.719+0000 7fd82cc00700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-02T15:25:19.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.267+0000 7f407fe00700 1 -- 172.21.15.1:0/187011424 <== mgr.14154 v2:172.21.15.1:6800/3768989981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f4088061210 con 0x7f40680a0520 2024-02-02T15:25:19.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.270+0000 7f408e200700 1 -- 172.21.15.1:0/187011424 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f40680a0520 msgr2=0x7f40680a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:19.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.270+0000 7f408e200700 1 --2- 172.21.15.1:0/187011424 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f40680a0520 0x7f40680a29e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4078011490 tx=0x7f4078009260 comp rx=0 tx=0).stop 2024-02-02T15:25:19.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.270+0000 7f408e200700 1 -- 172.21.15.1:0/187011424 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4088108ce0 msgr2=0x7f40881a03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:19.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.270+0000 7f408e200700 1 --2- 172.21.15.1:0/187011424 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4088108ce0 0x7f40881a03a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4080009fd0 tx=0x7f408000bce0 comp rx=0 tx=0).stop 2024-02-02T15:25:19.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.270+0000 7f408e200700 1 -- 172.21.15.1:0/187011424 shutdown_connections 2024-02-02T15:25:19.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.270+0000 7f408e200700 1 --2- 172.21.15.1:0/187011424 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4088108ce0 0x7f40881a03a0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:19.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.270+0000 7f408e200700 1 --2- 172.21.15.1:0/187011424 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f40680a0520 0x7f40680a29e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:19.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.270+0000 7f408e200700 1 --2- 172.21.15.1:0/187011424 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40880737a0 0x7f4088198df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:19.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.270+0000 7f408e200700 1 --2- 172.21.15.1:0/187011424 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4088074d80 0x7f4088199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:19.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.270+0000 7f408e200700 1 -- 172.21.15.1:0/187011424 >> 172.21.15.1:0/187011424 conn(0x7f40880fc390 msgr2=0x7f408810af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:19.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.270+0000 7f408e200700 1 -- 172.21.15.1:0/187011424 shutdown_connections 2024-02-02T15:25:19.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.270+0000 7f408e200700 1 -- 172.21.15.1:0/187011424 wait complete. 2024-02-02T15:25:19.291 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_0 2024-02-02T15:25:19.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.492+0000 7f670b400700 1 -- 172.21.15.1:0/201837498 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6704101f70 msgr2=0x7f67041023f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:19.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.492+0000 7f670b400700 1 --2- 172.21.15.1:0/201837498 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6704101f70 0x7f67041023f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f66ec009990 tx=0x7f66ec009ca0 comp rx=0 tx=0).stop 2024-02-02T15:25:19.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.492+0000 7f670b400700 1 -- 172.21.15.1:0/201837498 shutdown_connections 2024-02-02T15:25:19.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.492+0000 7f670b400700 1 --2- 172.21.15.1:0/201837498 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6704102930 0x7f6704109250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:19.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.492+0000 7f670b400700 1 --2- 172.21.15.1:0/201837498 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6704100e20 0x7f6704101240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:19.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.492+0000 7f670b400700 1 --2- 172.21.15.1:0/201837498 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6704101f70 0x7f67041023f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:19.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.492+0000 7f670b400700 1 -- 172.21.15.1:0/201837498 >> 172.21.15.1:0/201837498 conn(0x7f67040fc390 msgr2=0x7f67040fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:19.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.492+0000 7f670b400700 1 -- 172.21.15.1:0/201837498 shutdown_connections 2024-02-02T15:25:19.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.492+0000 7f670b400700 1 -- 172.21.15.1:0/201837498 wait complete. 2024-02-02T15:25:19.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.493+0000 7f670b400700 1 Processor -- start 2024-02-02T15:25:19.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.494+0000 7f670b400700 1 -- start start 2024-02-02T15:25:19.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.494+0000 7f670b400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6704100e20 0x7f6704198d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:19.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.494+0000 7f670b400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6704101f70 0x7f6704199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:19.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.494+0000 7f670b400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6704102930 0x7f67041a0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:19.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.494+0000 7f670b400700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67041a0850 con 0x7f6704102930 2024-02-02T15:25:19.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.494+0000 7f670b400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67041a09c0 con 0x7f6704101f70 2024-02-02T15:25:19.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.494+0000 7f670b400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67041a0b80 con 0x7f6704100e20 2024-02-02T15:25:19.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.494+0000 7f6709400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6704102930 0x7f67041a0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:19.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.494+0000 7f6708a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6704100e20 0x7f6704198d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:19.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.494+0000 7f6703e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6704101f70 0x7f6704199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:19.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.494+0000 7f6709400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6704102930 0x7f67041a0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33078/0 (socket says 172.21.15.1:33078) 2024-02-02T15:25:19.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.494+0000 7f6709400700 1 -- 172.21.15.1:0/4191295496 learned_addr learned my addr 172.21.15.1:0/4191295496 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:25:19.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.495+0000 7f6703e00700 1 -- 172.21.15.1:0/4191295496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6704100e20 msgr2=0x7f6704198d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:19.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.495+0000 7f6703e00700 1 --2- 172.21.15.1:0/4191295496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6704100e20 0x7f6704198d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:19.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.495+0000 7f6703e00700 1 -- 172.21.15.1:0/4191295496 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6704102930 msgr2=0x7f67041a0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:19.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.495+0000 7f6703e00700 1 --2- 172.21.15.1:0/4191295496 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6704102930 0x7f67041a0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:19.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.495+0000 7f6703e00700 1 -- 172.21.15.1:0/4191295496 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66ec0095f0 con 0x7f6704101f70 2024-02-02T15:25:19.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.495+0000 7f6709400700 1 --2- 172.21.15.1:0/4191295496 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6704102930 0x7f67041a0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:25:19.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.495+0000 7f6703e00700 1 --2- 172.21.15.1:0/4191295496 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6704101f70 0x7f6704199250 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f66ec00b620 tx=0x7f66ec004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:19.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.495+0000 7f6701600700 1 -- 172.21.15.1:0/4191295496 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ec01d070 con 0x7f6704101f70 2024-02-02T15:25:19.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.495+0000 7f670b400700 1 -- 172.21.15.1:0/4191295496 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67041a0e00 con 0x7f6704101f70 2024-02-02T15:25:19.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.495+0000 7f6701600700 1 -- 172.21.15.1:0/4191295496 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f66ec021410 con 0x7f6704101f70 2024-02-02T15:25:19.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.495+0000 7f6701600700 1 -- 172.21.15.1:0/4191295496 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ec017650 con 0x7f6704101f70 2024-02-02T15:25:19.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.495+0000 7f670b400700 1 -- 172.21.15.1:0/4191295496 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67041a11d0 con 0x7f6704101f70 2024-02-02T15:25:19.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.496+0000 7f670b400700 1 -- 172.21.15.1:0/4191295496 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f670410c670 con 0x7f6704101f70 2024-02-02T15:25:19.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.498+0000 7f6701600700 1 -- 172.21.15.1:0/4191295496 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) v1 ==== 135069+0+0 (secure 0 0 0) 0x7f66ec0217f0 con 0x7f6704101f70 2024-02-02T15:25:19.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.499+0000 7f6701600700 1 --2- 172.21.15.1:0/4191295496 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f66e40a0520 0x7f66e40a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:19.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.500+0000 7f6701600700 1 -- 172.21.15.1:0/4191295496 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f66ec0cd5e0 con 0x7f6704101f70 2024-02-02T15:25:19.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.500+0000 7f6708a00700 1 --2- 172.21.15.1:0/4191295496 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f66e40a0520 0x7f66e40a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:19.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.500+0000 7f6708a00700 1 --2- 172.21.15.1:0/4191295496 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f66e40a0520 0x7f66e40a29e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f670419a1e0 tx=0x7f66f4009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:19.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.502+0000 7f6701600700 1 -- 172.21.15.1:0/4191295496 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f66ec096fc0 con 0x7f6704101f70 2024-02-02T15:25:19.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.750+0000 7f670b400700 1 -- 172.21.15.1:0/4191295496 --> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7f6704108940 con 0x7f66e40a0520 2024-02-02T15:25:19.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.754+0000 7f6701600700 1 -- 172.21.15.1:0/4191295496 <== mgr.14154 v2:172.21.15.1:6800/3768989981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7f6704108940 con 0x7f66e40a0520 2024-02-02T15:25:19.754 INFO:teuthology.orchestra.run.smithi001.stdout:/volumes/_nogroup/sv_0/2d38b71b-2eab-4bcb-853f-7684f5d45e03 2024-02-02T15:25:19.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.756+0000 7f670b400700 1 -- 172.21.15.1:0/4191295496 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f66e40a0520 msgr2=0x7f66e40a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:19.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.756+0000 7f670b400700 1 --2- 172.21.15.1:0/4191295496 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f66e40a0520 0x7f66e40a29e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f670419a1e0 tx=0x7f66f4009260 comp rx=0 tx=0).stop 2024-02-02T15:25:19.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.756+0000 7f670b400700 1 -- 172.21.15.1:0/4191295496 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6704101f70 msgr2=0x7f6704199250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:19.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.756+0000 7f670b400700 1 --2- 172.21.15.1:0/4191295496 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6704101f70 0x7f6704199250 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f66ec00b620 tx=0x7f66ec004730 comp rx=0 tx=0).stop 2024-02-02T15:25:19.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.756+0000 7f670b400700 1 -- 172.21.15.1:0/4191295496 shutdown_connections 2024-02-02T15:25:19.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.756+0000 7f670b400700 1 --2- 172.21.15.1:0/4191295496 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6704102930 0x7f67041a0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:19.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.756+0000 7f670b400700 1 --2- 172.21.15.1:0/4191295496 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f66e40a0520 0x7f66e40a29e0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:19.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.756+0000 7f670b400700 1 --2- 172.21.15.1:0/4191295496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6704100e20 0x7f6704198d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:19.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.756+0000 7f670b400700 1 --2- 172.21.15.1:0/4191295496 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6704101f70 0x7f6704199250 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:19.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.756+0000 7f670b400700 1 -- 172.21.15.1:0/4191295496 >> 172.21.15.1:0/4191295496 conn(0x7f67040fc390 msgr2=0x7f67040fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:19.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.756+0000 7f670b400700 1 -- 172.21.15.1:0/4191295496 shutdown_connections 2024-02-02T15:25:19.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.756+0000 7f670b400700 1 -- 172.21.15.1:0/4191295496 wait complete. 2024-02-02T15:25:19.775 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 '' 2024-02-02T15:25:19.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.980+0000 7f11b2800700 1 -- 172.21.15.1:0/635477390 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f11ac100e30 msgr2=0x7f11ac101250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:19.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.980+0000 7f11b2800700 1 --2- 172.21.15.1:0/635477390 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f11ac100e30 0x7f11ac101250 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f119c009990 tx=0x7f119c009ca0 comp rx=0 tx=0).stop 2024-02-02T15:25:19.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.980+0000 7f11b2800700 1 -- 172.21.15.1:0/635477390 shutdown_connections 2024-02-02T15:25:19.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.980+0000 7f11b2800700 1 --2- 172.21.15.1:0/635477390 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11ac102940 0x7f11ac109260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:19.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.980+0000 7f11b2800700 1 --2- 172.21.15.1:0/635477390 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f11ac100e30 0x7f11ac101250 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:19.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.980+0000 7f11b2800700 1 --2- 172.21.15.1:0/635477390 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11ac101f80 0x7f11ac102400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:19.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.980+0000 7f11b2800700 1 -- 172.21.15.1:0/635477390 >> 172.21.15.1:0/635477390 conn(0x7f11ac0fc3a0 msgr2=0x7f11ac0fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:19.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.980+0000 7f11b2800700 1 -- 172.21.15.1:0/635477390 shutdown_connections 2024-02-02T15:25:19.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.980+0000 7f11b2800700 1 -- 172.21.15.1:0/635477390 wait complete. 2024-02-02T15:25:19.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.981+0000 7f11b2800700 1 Processor -- start 2024-02-02T15:25:19.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.981+0000 7f11b2800700 1 -- start start 2024-02-02T15:25:19.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.981+0000 7f11b2800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f11ac100e30 0x7f11ac196b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:19.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.981+0000 7f11b2800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11ac101f80 0x7f11ac197040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:19.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.981+0000 7f11b2800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11ac102940 0x7f11ac19e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:19.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.981+0000 7f11b2800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11ac19e680 con 0x7f11ac100e30 2024-02-02T15:25:19.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.981+0000 7f11b2800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11ac19e7c0 con 0x7f11ac102940 2024-02-02T15:25:19.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.982+0000 7f11b2800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11ac19e950 con 0x7f11ac101f80 2024-02-02T15:25:19.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.982+0000 7f11ab400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f11ac100e30 0x7f11ac196b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:19.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.982+0000 7f11ab400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f11ac100e30 0x7f11ac196b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33096/0 (socket says 172.21.15.1:33096) 2024-02-02T15:25:19.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.982+0000 7f11ab400700 1 -- 172.21.15.1:0/3188963511 learned_addr learned my addr 172.21.15.1:0/3188963511 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:25:19.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.982+0000 7f11aaa00700 1 --2- 172.21.15.1:0/3188963511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11ac101f80 0x7f11ac197040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:19.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.982+0000 7f11abe00700 1 --2- 172.21.15.1:0/3188963511 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11ac102940 0x7f11ac19e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:19.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.982+0000 7f11ab400700 1 -- 172.21.15.1:0/3188963511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11ac101f80 msgr2=0x7f11ac197040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:19.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.982+0000 7f11ab400700 1 --2- 172.21.15.1:0/3188963511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11ac101f80 0x7f11ac197040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:19.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.982+0000 7f11ab400700 1 -- 172.21.15.1:0/3188963511 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11ac102940 msgr2=0x7f11ac19e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:19.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.982+0000 7f11ab400700 1 --2- 172.21.15.1:0/3188963511 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11ac102940 0x7f11ac19e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:19.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.982+0000 7f11ab400700 1 -- 172.21.15.1:0/3188963511 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f119c0095f0 con 0x7f11ac100e30 2024-02-02T15:25:19.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.983+0000 7f11ab400700 1 --2- 172.21.15.1:0/3188963511 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f11ac100e30 0x7f11ac196b00 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f119c005910 tx=0x7f119c005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:19.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.983+0000 7f11a3e00700 1 -- 172.21.15.1:0/3188963511 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f119c01d070 con 0x7f11ac100e30 2024-02-02T15:25:19.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.983+0000 7f11a3e00700 1 -- 172.21.15.1:0/3188963511 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f119c003f60 con 0x7f11ac100e30 2024-02-02T15:25:19.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.983+0000 7f11b2800700 1 -- 172.21.15.1:0/3188963511 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11ac19ebd0 con 0x7f11ac100e30 2024-02-02T15:25:19.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.983+0000 7f11a3e00700 1 -- 172.21.15.1:0/3188963511 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f119c022c30 con 0x7f11ac100e30 2024-02-02T15:25:19.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.983+0000 7f11b2800700 1 -- 172.21.15.1:0/3188963511 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11ac19f050 con 0x7f11ac100e30 2024-02-02T15:25:19.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.984+0000 7f11b2800700 1 -- 172.21.15.1:0/3188963511 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11ac107d70 con 0x7f11ac100e30 2024-02-02T15:25:19.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.986+0000 7f11a3e00700 1 -- 172.21.15.1:0/3188963511 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 16) v1 ==== 135069+0+0 (secure 0 0 0) 0x7f119c0040d0 con 0x7f11ac100e30 2024-02-02T15:25:19.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.988+0000 7f11a3e00700 1 --2- 172.21.15.1:0/3188963511 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f118c0a05a0 0x7f118c0a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:19.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.988+0000 7f11a3e00700 1 -- 172.21.15.1:0/3188963511 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f119c0ccfe0 con 0x7f11ac100e30 2024-02-02T15:25:19.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.988+0000 7f11aaa00700 1 --2- 172.21.15.1:0/3188963511 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f118c0a05a0 0x7f118c0a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:19.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.989+0000 7f11aaa00700 1 --2- 172.21.15.1:0/3188963511 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f118c0a05a0 0x7f118c0a2a60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1194007c60 tx=0x7f11940073d0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:19.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:19.992+0000 7f11a3e00700 1 -- 172.21.15.1:0/3188963511 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f119c096a00 con 0x7f11ac100e30 2024-02-02T15:25:20.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.238+0000 7f11b2800700 1 -- 172.21.15.1:0/3188963511 --> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7f11ac061180 con 0x7f118c0a05a0 2024-02-02T15:25:20.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.248+0000 7f11a3e00700 1 -- 172.21.15.1:0/3188963511 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mgrmap(e 17) v1 ==== 135120+0+0 (secure 0 0 0) 0x7f119c096dd0 con 0x7f11ac100e30 2024-02-02T15:25:20.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.269+0000 7f11a3e00700 1 -- 172.21.15.1:0/3188963511 <== mgr.14154 v2:172.21.15.1:6800/3768989981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f11ac061180 con 0x7f118c0a05a0 2024-02-02T15:25:20.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.271+0000 7f11b2800700 1 -- 172.21.15.1:0/3188963511 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f118c0a05a0 msgr2=0x7f118c0a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:20.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.271+0000 7f11b2800700 1 --2- 172.21.15.1:0/3188963511 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f118c0a05a0 0x7f118c0a2a60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f1194007c60 tx=0x7f11940073d0 comp rx=0 tx=0).stop 2024-02-02T15:25:20.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.271+0000 7f11b2800700 1 -- 172.21.15.1:0/3188963511 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f11ac100e30 msgr2=0x7f11ac196b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:20.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.271+0000 7f11b2800700 1 --2- 172.21.15.1:0/3188963511 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f11ac100e30 0x7f11ac196b00 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f119c005910 tx=0x7f119c005740 comp rx=0 tx=0).stop 2024-02-02T15:25:20.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.271+0000 7f11b2800700 1 -- 172.21.15.1:0/3188963511 shutdown_connections 2024-02-02T15:25:20.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.271+0000 7f11b2800700 1 --2- 172.21.15.1:0/3188963511 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11ac102940 0x7f11ac19e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:20.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.271+0000 7f11b2800700 1 --2- 172.21.15.1:0/3188963511 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f11ac100e30 0x7f11ac196b00 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:20.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.271+0000 7f11b2800700 1 --2- 172.21.15.1:0/3188963511 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f118c0a05a0 0x7f118c0a2a60 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:20.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.271+0000 7f11b2800700 1 --2- 172.21.15.1:0/3188963511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f11ac101f80 0x7f11ac197040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:20.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.271+0000 7f11b2800700 1 -- 172.21.15.1:0/3188963511 >> 172.21.15.1:0/3188963511 conn(0x7f11ac0fc3a0 msgr2=0x7f11ac0fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:20.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.272+0000 7f11b2800700 1 -- 172.21.15.1:0/3188963511 shutdown_connections 2024-02-02T15:25:20.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.272+0000 7f11b2800700 1 -- 172.21.15.1:0/3188963511 wait complete. 2024-02-02T15:25:20.290 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_1 2024-02-02T15:25:20.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.490+0000 7f0421000700 1 -- 172.21.15.1:0/4092025253 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f041c100e20 msgr2=0x7f041c101240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:20.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.490+0000 7f0421000700 1 --2- 172.21.15.1:0/4092025253 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f041c100e20 0x7f041c101240 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f040c0098f0 tx=0x7f040c009c00 comp rx=0 tx=0).stop 2024-02-02T15:25:20.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.490+0000 7f0421000700 1 -- 172.21.15.1:0/4092025253 shutdown_connections 2024-02-02T15:25:20.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.490+0000 7f0421000700 1 --2- 172.21.15.1:0/4092025253 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f041c102930 0x7f041c109250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:20.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.490+0000 7f0421000700 1 --2- 172.21.15.1:0/4092025253 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f041c100e20 0x7f041c101240 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:20.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.490+0000 7f0421000700 1 --2- 172.21.15.1:0/4092025253 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f041c101f70 0x7f041c1023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:20.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.490+0000 7f0421000700 1 -- 172.21.15.1:0/4092025253 >> 172.21.15.1:0/4092025253 conn(0x7f041c0fc390 msgr2=0x7f041c0fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:20.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.490+0000 7f0421000700 1 -- 172.21.15.1:0/4092025253 shutdown_connections 2024-02-02T15:25:20.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.490+0000 7f0421000700 1 -- 172.21.15.1:0/4092025253 wait complete. 2024-02-02T15:25:20.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.491+0000 7f0421000700 1 Processor -- start 2024-02-02T15:25:20.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.492+0000 7f0421000700 1 -- start start 2024-02-02T15:25:20.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.492+0000 7f0421000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f041c100e20 0x7f041c196b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:20.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.492+0000 7f0421000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f041c101f70 0x7f041c1970a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:20.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.492+0000 7f0421000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f041c102930 0x7f041c19e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:20.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.492+0000 7f0421000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f041c19e6e0 con 0x7f041c101f70 2024-02-02T15:25:20.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.492+0000 7f0421000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f041c19e820 con 0x7f041c102930 2024-02-02T15:25:20.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.492+0000 7f0421000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f041c19e9b0 con 0x7f041c100e20 2024-02-02T15:25:20.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.492+0000 7f0419e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f041c100e20 0x7f041c196b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:20.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.492+0000 7f0419e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f041c100e20 0x7f041c196b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:37146/0 (socket says 172.21.15.1:37146) 2024-02-02T15:25:20.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.492+0000 7f0419e00700 1 -- 172.21.15.1:0/980243376 learned_addr learned my addr 172.21.15.1:0/980243376 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:25:20.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.492+0000 7f0419400700 1 --2- 172.21.15.1:0/980243376 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f041c101f70 0x7f041c1970a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:20.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.492+0000 7f041a800700 1 --2- 172.21.15.1:0/980243376 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f041c102930 0x7f041c19e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:20.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.492+0000 7f0419400700 1 -- 172.21.15.1:0/980243376 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f041c100e20 msgr2=0x7f041c196b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:20.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.492+0000 7f0419400700 1 --2- 172.21.15.1:0/980243376 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f041c100e20 0x7f041c196b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:20.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.492+0000 7f0419400700 1 -- 172.21.15.1:0/980243376 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f041c102930 msgr2=0x7f041c19e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:20.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.492+0000 7f0419400700 1 --2- 172.21.15.1:0/980243376 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f041c102930 0x7f041c19e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:20.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.492+0000 7f0419400700 1 -- 172.21.15.1:0/980243376 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f040c009520 con 0x7f041c101f70 2024-02-02T15:25:20.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.493+0000 7f0419400700 1 --2- 172.21.15.1:0/980243376 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f041c101f70 0x7f041c1970a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f040400e9c0 tx=0x7f040400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:20.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.493+0000 7f0412a00700 1 -- 172.21.15.1:0/980243376 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f040400c9c0 con 0x7f041c101f70 2024-02-02T15:25:20.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.493+0000 7f0412a00700 1 -- 172.21.15.1:0/980243376 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0404010430 con 0x7f041c101f70 2024-02-02T15:25:20.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.493+0000 7f0421000700 1 -- 172.21.15.1:0/980243376 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f041c19ec90 con 0x7f041c101f70 2024-02-02T15:25:20.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.493+0000 7f0412a00700 1 -- 172.21.15.1:0/980243376 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0404018680 con 0x7f041c101f70 2024-02-02T15:25:20.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.493+0000 7f0421000700 1 -- 172.21.15.1:0/980243376 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f041c19f1a0 con 0x7f041c101f70 2024-02-02T15:25:20.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.494+0000 7f0421000700 1 -- 172.21.15.1:0/980243376 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f041c10c670 con 0x7f041c101f70 2024-02-02T15:25:20.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.496+0000 7f0412a00700 1 -- 172.21.15.1:0/980243376 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) v1 ==== 135120+0+0 (secure 0 0 0) 0x7f0404010860 con 0x7f041c101f70 2024-02-02T15:25:20.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.497+0000 7f0412a00700 1 --2- 172.21.15.1:0/980243376 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f03fc0a05b0 0x7f03fc0a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:20.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.497+0000 7f0412a00700 1 -- 172.21.15.1:0/980243376 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f0404014070 con 0x7f041c101f70 2024-02-02T15:25:20.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.498+0000 7f0419e00700 1 --2- 172.21.15.1:0/980243376 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f03fc0a05b0 0x7f03fc0a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:20.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.498+0000 7f0419e00700 1 --2- 172.21.15.1:0/980243376 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f03fc0a05b0 0x7f03fc0a2a70 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f040c00c010 tx=0x7f040c019040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:20.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.501+0000 7f0412a00700 1 -- 172.21.15.1:0/980243376 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f0404052110 con 0x7f041c101f70 2024-02-02T15:25:20.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:20 smithi071 ceph-mon[44707]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:25:20.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:20 smithi071 ceph-mon[44707]: mds.? [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] up:active 2024-02-02T15:25:20.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:20 smithi071 ceph-mon[44707]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-02-02T15:25:20.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:20 smithi071 ceph-mon[44707]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-02-02T15:25:20.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:20 smithi071 ceph-mon[44707]: Cluster is now healthy 2024-02-02T15:25:20.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:20 smithi071 ceph-mon[44707]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 8 up:standby 2024-02-02T15:25:20.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:20 smithi071 ceph-mon[44707]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-02-02T15:25:20.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:20 smithi071 ceph-mon[44707]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-02-02T15:25:20.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:20 smithi142 ceph-mon[45728]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:25:20.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:20 smithi142 ceph-mon[45728]: mds.? [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] up:active 2024-02-02T15:25:20.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:20 smithi142 ceph-mon[45728]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-02-02T15:25:20.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:20 smithi142 ceph-mon[45728]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-02-02T15:25:20.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:20 smithi142 ceph-mon[45728]: Cluster is now healthy 2024-02-02T15:25:20.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:20 smithi142 ceph-mon[45728]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 8 up:standby 2024-02-02T15:25:20.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:20 smithi142 ceph-mon[45728]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-02-02T15:25:20.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:20 smithi142 ceph-mon[45728]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-02-02T15:25:20.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:20 smithi001 ceph-mon[35470]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:25:20.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:20 smithi001 ceph-mon[35470]: mds.? [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] up:active 2024-02-02T15:25:20.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:20 smithi001 ceph-mon[35470]: daemon mds.f assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-02-02T15:25:20.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:20 smithi001 ceph-mon[35470]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-02-02T15:25:20.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:20 smithi001 ceph-mon[35470]: Cluster is now healthy 2024-02-02T15:25:20.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:20 smithi001 ceph-mon[35470]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 8 up:standby 2024-02-02T15:25:20.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:20 smithi001 ceph-mon[35470]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-02-02T15:25:20.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:20 smithi001 ceph-mon[35470]: daemon mds.f is now active in filesystem cephfs as rank 4 2024-02-02T15:25:20.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.748+0000 7f0421000700 1 -- 172.21.15.1:0/980243376 --> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f041c061180 con 0x7f03fc0a05b0 2024-02-02T15:25:20.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.753+0000 7f0412a00700 1 -- 172.21.15.1:0/980243376 <== mgr.14154 v2:172.21.15.1:6800/3768989981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7f041c061180 con 0x7f03fc0a05b0 2024-02-02T15:25:20.753 INFO:teuthology.orchestra.run.smithi001.stdout:/volumes/_nogroup/sv_1/2234ea52-cd6c-42c8-816f-3ba5d3b919ec 2024-02-02T15:25:20.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.755+0000 7f0421000700 1 -- 172.21.15.1:0/980243376 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f03fc0a05b0 msgr2=0x7f03fc0a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:20.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.755+0000 7f0421000700 1 --2- 172.21.15.1:0/980243376 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f03fc0a05b0 0x7f03fc0a2a70 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f040c00c010 tx=0x7f040c019040 comp rx=0 tx=0).stop 2024-02-02T15:25:20.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.755+0000 7f0421000700 1 -- 172.21.15.1:0/980243376 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f041c101f70 msgr2=0x7f041c1970a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:20.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.755+0000 7f0421000700 1 --2- 172.21.15.1:0/980243376 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f041c101f70 0x7f041c1970a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f040400e9c0 tx=0x7f040400ecd0 comp rx=0 tx=0).stop 2024-02-02T15:25:20.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.755+0000 7f0421000700 1 -- 172.21.15.1:0/980243376 shutdown_connections 2024-02-02T15:25:20.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.755+0000 7f0421000700 1 --2- 172.21.15.1:0/980243376 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f041c102930 0x7f041c19e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:20.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.755+0000 7f0421000700 1 --2- 172.21.15.1:0/980243376 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f03fc0a05b0 0x7f03fc0a2a70 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:20.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.755+0000 7f0421000700 1 --2- 172.21.15.1:0/980243376 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f041c100e20 0x7f041c196b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:20.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.755+0000 7f0421000700 1 --2- 172.21.15.1:0/980243376 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f041c101f70 0x7f041c1970a0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:20.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.755+0000 7f0421000700 1 -- 172.21.15.1:0/980243376 >> 172.21.15.1:0/980243376 conn(0x7f041c0fc390 msgr2=0x7f041c0fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:20.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.755+0000 7f0421000700 1 -- 172.21.15.1:0/980243376 shutdown_connections 2024-02-02T15:25:20.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.755+0000 7f0421000700 1 -- 172.21.15.1:0/980243376 wait complete. 2024-02-02T15:25:20.774 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T15:25:20.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.971+0000 7fd283e00700 1 -- 172.21.15.1:0/305193036 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd27c101f40 msgr2=0x7fd27c1023a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:20.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.971+0000 7fd283e00700 1 --2- 172.21.15.1:0/305193036 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd27c101f40 0x7fd27c1023a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd2640099c0 tx=0x7fd264009cd0 comp rx=0 tx=0).stop 2024-02-02T15:25:20.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.972+0000 7fd283e00700 1 -- 172.21.15.1:0/305193036 shutdown_connections 2024-02-02T15:25:20.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.972+0000 7fd283e00700 1 --2- 172.21.15.1:0/305193036 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd27c1028e0 0x7fd27c109190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:20.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.972+0000 7fd283e00700 1 --2- 172.21.15.1:0/305193036 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd27c100d50 0x7fd27c101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:20.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.972+0000 7fd283e00700 1 --2- 172.21.15.1:0/305193036 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd27c101f40 0x7fd27c1023a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:20.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.972+0000 7fd283e00700 1 -- 172.21.15.1:0/305193036 >> 172.21.15.1:0/305193036 conn(0x7fd27c0fc2c0 msgr2=0x7fd27c0fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:20.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.972+0000 7fd283e00700 1 -- 172.21.15.1:0/305193036 shutdown_connections 2024-02-02T15:25:20.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.972+0000 7fd283e00700 1 -- 172.21.15.1:0/305193036 wait complete. 2024-02-02T15:25:20.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.973+0000 7fd283e00700 1 Processor -- start 2024-02-02T15:25:20.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.973+0000 7fd283e00700 1 -- start start 2024-02-02T15:25:20.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.973+0000 7fd283e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd27c100d50 0x7fd27c198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:20.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.973+0000 7fd283e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd27c101f40 0x7fd27c1991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:20.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.973+0000 7fd283e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd27c1028e0 0x7fd27c1a01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:20.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.973+0000 7fd283e00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd27c1a07a0 con 0x7fd27c101f40 2024-02-02T15:25:20.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.973+0000 7fd283e00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd27c1a0910 con 0x7fd27c100d50 2024-02-02T15:25:20.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.973+0000 7fd283e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd27c1a0ad0 con 0x7fd27c1028e0 2024-02-02T15:25:20.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.974+0000 7fd280a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd27c101f40 0x7fd27c1991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:20.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.974+0000 7fd281400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd27c100d50 0x7fd27c198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:20.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.974+0000 7fd280a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd27c101f40 0x7fd27c1991a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33134/0 (socket says 172.21.15.1:33134) 2024-02-02T15:25:20.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.974+0000 7fd280a00700 1 -- 172.21.15.1:0/1010496905 learned_addr learned my addr 172.21.15.1:0/1010496905 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:25:20.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.974+0000 7fd281e00700 1 --2- 172.21.15.1:0/1010496905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd27c1028e0 0x7fd27c1a01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:20.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.974+0000 7fd280a00700 1 -- 172.21.15.1:0/1010496905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd27c1028e0 msgr2=0x7fd27c1a01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:20.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.974+0000 7fd280a00700 1 --2- 172.21.15.1:0/1010496905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd27c1028e0 0x7fd27c1a01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:20.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.974+0000 7fd280a00700 1 -- 172.21.15.1:0/1010496905 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd27c100d50 msgr2=0x7fd27c198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:20.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.974+0000 7fd280a00700 1 --2- 172.21.15.1:0/1010496905 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd27c100d50 0x7fd27c198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:20.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.974+0000 7fd280a00700 1 -- 172.21.15.1:0/1010496905 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2640095f0 con 0x7fd27c101f40 2024-02-02T15:25:20.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.974+0000 7fd281400700 1 --2- 172.21.15.1:0/1010496905 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd27c100d50 0x7fd27c198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:25:20.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.974+0000 7fd280a00700 1 --2- 172.21.15.1:0/1010496905 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd27c101f40 0x7fd27c1991a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd264009fd0 tx=0x7fd26400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:20.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.975+0000 7fd27a000700 1 -- 172.21.15.1:0/1010496905 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd26401d070 con 0x7fd27c101f40 2024-02-02T15:25:20.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.975+0000 7fd27a000700 1 -- 172.21.15.1:0/1010496905 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd264005050 con 0x7fd27c101f40 2024-02-02T15:25:20.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.975+0000 7fd283e00700 1 -- 172.21.15.1:0/1010496905 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd27c1a0d50 con 0x7fd27c101f40 2024-02-02T15:25:20.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.975+0000 7fd27a000700 1 -- 172.21.15.1:0/1010496905 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd264004b30 con 0x7fd27c101f40 2024-02-02T15:25:20.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.975+0000 7fd283e00700 1 -- 172.21.15.1:0/1010496905 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd27c1a1170 con 0x7fd27c101f40 2024-02-02T15:25:20.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.976+0000 7fd283e00700 1 -- 172.21.15.1:0/1010496905 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd27c1933a0 con 0x7fd27c101f40 2024-02-02T15:25:20.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.978+0000 7fd27a000700 1 -- 172.21.15.1:0/1010496905 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) v1 ==== 135120+0+0 (secure 0 0 0) 0x7fd264004c90 con 0x7fd27c101f40 2024-02-02T15:25:20.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.980+0000 7fd27a000700 1 --2- 172.21.15.1:0/1010496905 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fd25c0a05b0 0x7fd25c0a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:20.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.980+0000 7fd27a000700 1 -- 172.21.15.1:0/1010496905 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fd2640ccaf0 con 0x7fd27c101f40 2024-02-02T15:25:20.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.980+0000 7fd281400700 1 --2- 172.21.15.1:0/1010496905 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fd25c0a05b0 0x7fd25c0a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:20.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.981+0000 7fd281400700 1 --2- 172.21.15.1:0/1010496905 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fd25c0a05b0 0x7fd25c0a2a70 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd27c19a130 tx=0x7fd26c009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:20.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:20.984+0000 7fd27a000700 1 -- 172.21.15.1:0/1010496905 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7fd264096370 con 0x7fd27c101f40 2024-02-02T15:25:21.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.246+0000 7fd283e00700 1 -- 172.21.15.1:0/1010496905 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd27c02cc70 con 0x7fd27c101f40 2024-02-02T15:25:21.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.247+0000 7fd27a000700 1 -- 172.21.15.1:0/1010496905 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10866 (secure 0 0 0) 0x7fd264099990 con 0x7fd27c101f40 2024-02-02T15:25:21.247 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:25:21.247 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2922240448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2922240448},{"type":"v1","addr":"172.21.15.1:6835","nonce":2922240448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2729497431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2729497431},{"type":"v1","addr":"172.21.15.1:6837","nonce":2729497431}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-02T15:24:47.761435+0000","modified":"2024-02-02T15:25:20.251601+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24529,"mds_2":24518,"mds_3":24514,"mds_4":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14517":{"gid":14517,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/1218303683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":1218303683},{"type":"v1","addr":"172.21.15.1:6839","nonce":1218303683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14529":{"gid":14529,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/2059197496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2059197496},{"type":"v1","addr":"172.21.15.1:6841","nonce":2059197496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/7648986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":7648986},{"type":"v1","addr":"172.21.15.142:6833","nonce":7648986}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/3482746094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3482746094},{"type":"v1","addr":"172.21.15.71:6833","nonce":3482746094}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/388207814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":388207814},{"type":"v1","addr":"172.21.15.71:6835","nonce":388207814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6835/1052606547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1052606547},{"type":"v1","addr":"172.21.15.142:6835","nonce":1052606547}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.142:6837/2870289019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2870289019},{"type":"v1","addr":"172.21.15.142:6837","nonce":2870289019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.71:6837/628016199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":628016199},{"type":"v1","addr":"172.21.15.71:6837","nonce":628016199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.142:6839/321675421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":321675421},{"type":"v1","addr":"172.21.15.142:6839","nonce":321675421}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/1268086493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1268086493},{"type":"v1","addr":"172.21.15.71:6839","nonce":1268086493}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-02T15:25:21.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.249+0000 7fd283e00700 1 -- 172.21.15.1:0/1010496905 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fd25c0a05b0 msgr2=0x7fd25c0a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:21.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.249+0000 7fd283e00700 1 --2- 172.21.15.1:0/1010496905 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fd25c0a05b0 0x7fd25c0a2a70 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd27c19a130 tx=0x7fd26c009260 comp rx=0 tx=0).stop 2024-02-02T15:25:21.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.249+0000 7fd283e00700 1 -- 172.21.15.1:0/1010496905 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd27c101f40 msgr2=0x7fd27c1991a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:21.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.249+0000 7fd283e00700 1 --2- 172.21.15.1:0/1010496905 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd27c101f40 0x7fd27c1991a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd264009fd0 tx=0x7fd26400f690 comp rx=0 tx=0).stop 2024-02-02T15:25:21.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.249+0000 7fd283e00700 1 -- 172.21.15.1:0/1010496905 shutdown_connections 2024-02-02T15:25:21.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.249+0000 7fd283e00700 1 --2- 172.21.15.1:0/1010496905 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fd25c0a05b0 0x7fd25c0a2a70 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:21.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.249+0000 7fd283e00700 1 --2- 172.21.15.1:0/1010496905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd27c1028e0 0x7fd27c1a01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:21.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.249+0000 7fd283e00700 1 --2- 172.21.15.1:0/1010496905 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd27c100d50 0x7fd27c198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:21.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.249+0000 7fd283e00700 1 --2- 172.21.15.1:0/1010496905 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd27c101f40 0x7fd27c1991a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:21.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.249+0000 7fd283e00700 1 -- 172.21.15.1:0/1010496905 >> 172.21.15.1:0/1010496905 conn(0x7fd27c0fc2c0 msgr2=0x7fd27c107840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:21.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.249+0000 7fd283e00700 1 -- 172.21.15.1:0/1010496905 shutdown_connections 2024-02-02T15:25:21.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.249+0000 7fd283e00700 1 -- 172.21.15.1:0/1010496905 wait complete. 2024-02-02T15:25:21.250 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 27 2024-02-02T15:25:21.268 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T15:25:21.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.467+0000 7f1da1600700 1 -- 172.21.15.1:0/3200686474 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d9c101f30 msgr2=0x7f1d9c102390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:21.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.467+0000 7f1da1600700 1 --2- 172.21.15.1:0/3200686474 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d9c101f30 0x7f1d9c102390 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1d840099c0 tx=0x7f1d84009cd0 comp rx=0 tx=0).stop 2024-02-02T15:25:21.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.467+0000 7f1da1600700 1 -- 172.21.15.1:0/3200686474 shutdown_connections 2024-02-02T15:25:21.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.467+0000 7f1da1600700 1 --2- 172.21.15.1:0/3200686474 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d9c1028d0 0x7f1d9c109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:21.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.467+0000 7f1da1600700 1 --2- 172.21.15.1:0/3200686474 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d9c100d40 0x7f1d9c101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:21.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.467+0000 7f1da1600700 1 --2- 172.21.15.1:0/3200686474 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d9c101f30 0x7f1d9c102390 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:21.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.467+0000 7f1da1600700 1 -- 172.21.15.1:0/3200686474 >> 172.21.15.1:0/3200686474 conn(0x7f1d9c0fc290 msgr2=0x7f1d9c0fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:21.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.467+0000 7f1da1600700 1 -- 172.21.15.1:0/3200686474 shutdown_connections 2024-02-02T15:25:21.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.467+0000 7f1da1600700 1 -- 172.21.15.1:0/3200686474 wait complete. 2024-02-02T15:25:21.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.468+0000 7f1da1600700 1 Processor -- start 2024-02-02T15:25:21.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.468+0000 7f1da1600700 1 -- start start 2024-02-02T15:25:21.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.468+0000 7f1da1600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d9c100d40 0x7f1d9c074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:21.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.468+0000 7f1da1600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d9c101f30 0x7f1d9c073130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:21.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.468+0000 7f1da1600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d9c1028d0 0x7f1d9c073670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:21.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.468+0000 7f1da1600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9c073e10 con 0x7f1d9c100d40 2024-02-02T15:25:21.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.468+0000 7f1da1600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9c073f80 con 0x7f1d9c101f30 2024-02-02T15:25:21.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.468+0000 7f1da1600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d9c1a0a50 con 0x7f1d9c1028d0 2024-02-02T15:25:21.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.468+0000 7f1d9aa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d9c100d40 0x7f1d9c074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:21.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.469+0000 7f1d9aa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d9c100d40 0x7f1d9c074af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33138/0 (socket says 172.21.15.1:33138) 2024-02-02T15:25:21.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.469+0000 7f1d9aa00700 1 -- 172.21.15.1:0/2979015657 learned_addr learned my addr 172.21.15.1:0/2979015657 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:25:21.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.469+0000 7f1d9a000700 1 --2- 172.21.15.1:0/2979015657 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d9c101f30 0x7f1d9c073130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:21.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.469+0000 7f1d9b400700 1 --2- 172.21.15.1:0/2979015657 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d9c1028d0 0x7f1d9c073670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:21.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.469+0000 7f1d9aa00700 1 -- 172.21.15.1:0/2979015657 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d9c1028d0 msgr2=0x7f1d9c073670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:21.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.469+0000 7f1d9aa00700 1 --2- 172.21.15.1:0/2979015657 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d9c1028d0 0x7f1d9c073670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:21.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.469+0000 7f1d9aa00700 1 -- 172.21.15.1:0/2979015657 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d9c101f30 msgr2=0x7f1d9c073130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:21.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.469+0000 7f1d9aa00700 1 --2- 172.21.15.1:0/2979015657 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d9c101f30 0x7f1d9c073130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:21.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.469+0000 7f1d9aa00700 1 -- 172.21.15.1:0/2979015657 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d8c009520 con 0x7f1d9c100d40 2024-02-02T15:25:21.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.469+0000 7f1d9aa00700 1 --2- 172.21.15.1:0/2979015657 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d9c100d40 0x7f1d9c074af0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c00ea50 tx=0x7f1d8c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:21.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.469+0000 7f1d93400700 1 -- 172.21.15.1:0/2979015657 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8c00cbd0 con 0x7f1d9c100d40 2024-02-02T15:25:21.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.470+0000 7f1d93400700 1 -- 172.21.15.1:0/2979015657 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1d8c00cd30 con 0x7f1d9c100d40 2024-02-02T15:25:21.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.470+0000 7f1da1600700 1 -- 172.21.15.1:0/2979015657 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d840095f0 con 0x7f1d9c100d40 2024-02-02T15:25:21.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.470+0000 7f1d93400700 1 -- 172.21.15.1:0/2979015657 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d8c018680 con 0x7f1d9c100d40 2024-02-02T15:25:21.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.470+0000 7f1da1600700 1 -- 172.21.15.1:0/2979015657 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d9c1a10b0 con 0x7f1d9c100d40 2024-02-02T15:25:21.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.471+0000 7f1da1600700 1 -- 172.21.15.1:0/2979015657 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d9c107c70 con 0x7f1d9c100d40 2024-02-02T15:25:21.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.473+0000 7f1d93400700 1 -- 172.21.15.1:0/2979015657 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) v1 ==== 135120+0+0 (secure 0 0 0) 0x7f1d8c010430 con 0x7f1d9c100d40 2024-02-02T15:25:21.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.474+0000 7f1d93400700 1 --2- 172.21.15.1:0/2979015657 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f1d7c0a05b0 0x7f1d7c0a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:21.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.474+0000 7f1d93400700 1 -- 172.21.15.1:0/2979015657 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f1d8c014070 con 0x7f1d9c100d40 2024-02-02T15:25:21.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.474+0000 7f1d9a000700 1 --2- 172.21.15.1:0/2979015657 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f1d7c0a05b0 0x7f1d7c0a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:21.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.475+0000 7f1d9a000700 1 --2- 172.21.15.1:0/2979015657 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f1d7c0a05b0 0x7f1d7c0a2a70 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1d8400b580 tx=0x7f1d84005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:21.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.478+0000 7f1d93400700 1 -- 172.21.15.1:0/2979015657 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f1d8c095a30 con 0x7f1d9c100d40 2024-02-02T15:25:21.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:21 smithi071 ceph-mon[44707]: from='client.24806 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:25:21.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:21 smithi071 ceph-mon[44707]: pgmap v251: 129 pgs: 129 active+clean; 582 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 5 op/s 2024-02-02T15:25:21.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:21 smithi071 ceph-mon[44707]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:25:21.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:21 smithi071 ceph-mon[44707]: mds.? [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] up:active 2024-02-02T15:25:21.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:21 smithi071 ceph-mon[44707]: fsmap cephfs:5 7 up:standby 5 up:active 2024-02-02T15:25:21.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:21 smithi071 ceph-mon[44707]: mgrmap e17: x(active, since 5m), standbys: z, y 2024-02-02T15:25:21.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:21 smithi071 ceph-mon[44707]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:25:21.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:21 smithi142 ceph-mon[45728]: from='client.24806 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:25:21.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:21 smithi142 ceph-mon[45728]: pgmap v251: 129 pgs: 129 active+clean; 582 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 5 op/s 2024-02-02T15:25:21.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:21 smithi142 ceph-mon[45728]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:25:21.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:21 smithi142 ceph-mon[45728]: mds.? [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] up:active 2024-02-02T15:25:21.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:21 smithi142 ceph-mon[45728]: fsmap cephfs:5 7 up:standby 5 up:active 2024-02-02T15:25:21.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:21 smithi142 ceph-mon[45728]: mgrmap e17: x(active, since 5m), standbys: z, y 2024-02-02T15:25:21.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:21 smithi142 ceph-mon[45728]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:25:21.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:21 smithi001 ceph-mon[35470]: from='client.24806 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:25:21.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:21 smithi001 ceph-mon[35470]: pgmap v251: 129 pgs: 129 active+clean; 582 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 5 op/s 2024-02-02T15:25:21.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:21 smithi001 ceph-mon[35470]: from='client.14793 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:25:21.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:21 smithi001 ceph-mon[35470]: mds.? [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] up:active 2024-02-02T15:25:21.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:21 smithi001 ceph-mon[35470]: fsmap cephfs:5 7 up:standby 5 up:active 2024-02-02T15:25:21.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:21 smithi001 ceph-mon[35470]: mgrmap e17: x(active, since 5m), standbys: z, y 2024-02-02T15:25:21.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:21 smithi001 ceph-mon[35470]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:25:21.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.692+0000 7f1da1600700 1 -- 172.21.15.1:0/2979015657 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1d9c02ced0 con 0x7f1d9c100d40 2024-02-02T15:25:21.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.694+0000 7f1d93400700 1 -- 172.21.15.1:0/2979015657 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+23841 (secure 0 0 0) 0x7f1d8c099050 con 0x7f1d9c100d40 2024-02-02T15:25:21.694 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:25:21.694 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":84,"fsid":"371a409c-c1de-11ee-95b6-87774f69a715","created":"2024-02-02T15:18:26.746506+0000","modified":"2024-02-02T15:24:52.310888+0000","last_up_change":"2024-02-02T15:24:08.800951+0000","last_in_change":"2024-02-02T15:23:52.784280+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T15:21:22.471754+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-02T15:24:46.195779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-02T15:24:47.226403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-02T15:24:50.300462+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"73e8a07f-3560-4c19-85ed-030d4d8f0f23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6803","nonce":2591906423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6805","nonce":2591906423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6809","nonce":2591906423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6807","nonce":2591906423}]},"public_addr":"172.21.15.1:6803/2591906423","cluster_addr":"172.21.15.1:6805/2591906423","heartbeat_back_addr":"172.21.15.1:6809/2591906423","heartbeat_front_addr":"172.21.15.1:6807/2591906423","state":["exists","up"]},{"osd":1,"uuid":"e53cffec-0ff9-4ed9-8dcf-ff47d04db2bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6801","nonce":747581719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6803","nonce":747581719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6807","nonce":747581719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6805","nonce":747581719}]},"public_addr":"172.21.15.71:6801/747581719","cluster_addr":"172.21.15.71:6803/747581719","heartbeat_back_addr":"172.21.15.71:6807/747581719","heartbeat_front_addr":"172.21.15.71:6805/747581719","state":["exists","up"]},{"osd":2,"uuid":"856c2347-294c-482b-8595-ad77f4792db4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6801","nonce":377511414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6803","nonce":377511414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6807","nonce":377511414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6805","nonce":377511414}]},"public_addr":"172.21.15.142:6801/377511414","cluster_addr":"172.21.15.142:6803/377511414","heartbeat_back_addr":"172.21.15.142:6807/377511414","heartbeat_front_addr":"172.21.15.142:6805/377511414","state":["exists","up"]},{"osd":3,"uuid":"4b946ad2-2da9-4e0b-9d61-d039d8669a42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6811","nonce":1086124477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6813","nonce":1086124477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6817","nonce":1086124477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6815","nonce":1086124477}]},"public_addr":"172.21.15.1:6811/1086124477","cluster_addr":"172.21.15.1:6813/1086124477","heartbeat_back_addr":"172.21.15.1:6817/1086124477","heartbeat_front_addr":"172.21.15.1:6815/1086124477","state":["exists","up"]},{"osd":4,"uuid":"43e36c43-47a0-4634-a45b-1da14a0e19d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6809","nonce":3095380305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6811","nonce":3095380305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6815","nonce":3095380305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6813","nonce":3095380305}]},"public_addr":"172.21.15.71:6809/3095380305","cluster_addr":"172.21.15.71:6811/3095380305","heartbeat_back_addr":"172.21.15.71:6815/3095380305","heartbeat_front_addr":"172.21.15.71:6813/3095380305","state":["exists","up"]},{"osd":5,"uuid":"eb48160e-ea7c-42f7-9477-be110bc8a34e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6809","nonce":4195149262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6811","nonce":4195149262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6815","nonce":4195149262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6813","nonce":4195149262}]},"public_addr":"172.21.15.142:6809/4195149262","cluster_addr":"172.21.15.142:6811/4195149262","heartbeat_back_addr":"172.21.15.142:6815/4195149262","heartbeat_front_addr":"172.21.15.142:6813/4195149262","state":["exists","up"]},{"osd":6,"uuid":"0ec4501a-5853-4ca6-b325-231f5b72335e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6819","nonce":1560153428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6821","nonce":1560153428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6825","nonce":1560153428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6823","nonce":1560153428}]},"public_addr":"172.21.15.1:6819/1560153428","cluster_addr":"172.21.15.1:6821/1560153428","heartbeat_back_addr":"172.21.15.1:6825/1560153428","heartbeat_front_addr":"172.21.15.1:6823/1560153428","state":["exists","up"]},{"osd":7,"uuid":"029debe8-9cdf-44f6-9aea-cabd1d185f93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6817","nonce":4080955538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6819","nonce":4080955538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6823","nonce":4080955538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6821","nonce":4080955538}]},"public_addr":"172.21.15.71:6817/4080955538","cluster_addr":"172.21.15.71:6819/4080955538","heartbeat_back_addr":"172.21.15.71:6823/4080955538","heartbeat_front_addr":"172.21.15.71:6821/4080955538","state":["exists","up"]},{"osd":8,"uuid":"7c90896c-9955-4c1b-aaca-7531dd5457f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6817","nonce":3942038209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6819","nonce":3942038209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6823","nonce":3942038209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6821","nonce":3942038209}]},"public_addr":"172.21.15.142:6817/3942038209","cluster_addr":"172.21.15.142:6819/3942038209","heartbeat_back_addr":"172.21.15.142:6823/3942038209","heartbeat_front_addr":"172.21.15.142:6821/3942038209","state":["exists","up"]},{"osd":9,"uuid":"7e165e68-31a8-4598-a577-de3e90ae5c5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6827","nonce":2716544850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6829","nonce":2716544850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6833","nonce":2716544850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6831","nonce":2716544850}]},"public_addr":"172.21.15.1:6827/2716544850","cluster_addr":"172.21.15.1:6829/2716544850","heartbeat_back_addr":"172.21.15.1:6833/2716544850","heartbeat_front_addr":"172.21.15.1:6831/2716544850","state":["exists","up"]},{"osd":10,"uuid":"48cd5fdb-0887-444a-aab8-a1de24282bb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6825","nonce":84691841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6827","nonce":84691841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6831","nonce":84691841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6829","nonce":84691841}]},"public_addr":"172.21.15.71:6825/84691841","cluster_addr":"172.21.15.71:6827/84691841","heartbeat_back_addr":"172.21.15.71:6831/84691841","heartbeat_front_addr":"172.21.15.71:6829/84691841","state":["exists","up"]},{"osd":11,"uuid":"f011dd27-11b0-4166-adfe-76756533163a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6825","nonce":2290179934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6827","nonce":2290179934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6831","nonce":2290179934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6829","nonce":2290179934}]},"public_addr":"172.21.15.142:6825/2290179934","cluster_addr":"172.21.15.142:6827/2290179934","heartbeat_back_addr":"172.21.15.142:6831/2290179934","heartbeat_front_addr":"172.21.15.142:6829/2290179934","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:20:44.060692+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:01.903817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:19.368258+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:40.028037+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:57.513361+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:22:14.805944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:22:35.188439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:22:53.014412+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:23:10.507420+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:23:31.849765+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:23:50.070612+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:24:07.388625+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/4092532060":"2024-02-03T15:19:25.836171+0000","172.21.15.1:6801/395498296":"2024-02-03T15:19:25.836171+0000","172.21.15.1:6800/395498296":"2024-02-03T15:19:25.836171+0000","172.21.15.1:0/1575373489":"2024-02-03T15:19:25.836171+0000","172.21.15.1:0/865983374":"2024-02-03T15:19:01.431596+0000","172.21.15.1:0/3165960227":"2024-02-03T15:19:01.431596+0000","172.21.15.1:0/2350830293":"2024-02-03T15:19:01.431596+0000","172.21.15.1:0/2959690250":"2024-02-03T15:19:25.836171+0000","172.21.15.1:6800/1162530732":"2024-02-03T15:19:01.431596+0000","172.21.15.1:6801/1162530732":"2024-02-03T15:19:01.431596+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T15:25:21.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.696+0000 7f1da1600700 1 -- 172.21.15.1:0/2979015657 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f1d7c0a05b0 msgr2=0x7f1d7c0a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:21.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.696+0000 7f1da1600700 1 --2- 172.21.15.1:0/2979015657 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f1d7c0a05b0 0x7f1d7c0a2a70 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1d8400b580 tx=0x7f1d84005fb0 comp rx=0 tx=0).stop 2024-02-02T15:25:21.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.696+0000 7f1da1600700 1 -- 172.21.15.1:0/2979015657 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d9c100d40 msgr2=0x7f1d9c074af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:21.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.696+0000 7f1da1600700 1 --2- 172.21.15.1:0/2979015657 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d9c100d40 0x7f1d9c074af0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c00ea50 tx=0x7f1d8c00ed60 comp rx=0 tx=0).stop 2024-02-02T15:25:21.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.696+0000 7f1da1600700 1 -- 172.21.15.1:0/2979015657 shutdown_connections 2024-02-02T15:25:21.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.696+0000 7f1da1600700 1 --2- 172.21.15.1:0/2979015657 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f1d7c0a05b0 0x7f1d7c0a2a70 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:21.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.696+0000 7f1da1600700 1 --2- 172.21.15.1:0/2979015657 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1d9c1028d0 0x7f1d9c073670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:21.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.696+0000 7f1da1600700 1 --2- 172.21.15.1:0/2979015657 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1d9c100d40 0x7f1d9c074af0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:21.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.696+0000 7f1da1600700 1 --2- 172.21.15.1:0/2979015657 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d9c101f30 0x7f1d9c073130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:21.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.696+0000 7f1da1600700 1 -- 172.21.15.1:0/2979015657 >> 172.21.15.1:0/2979015657 conn(0x7f1d9c0fc290 msgr2=0x7f1d9c0fe570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:21.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.696+0000 7f1da1600700 1 -- 172.21.15.1:0/2979015657 shutdown_connections 2024-02-02T15:25:21.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.696+0000 7f1da1600700 1 -- 172.21.15.1:0/2979015657 wait complete. 2024-02-02T15:25:21.715 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-02-02T15:25:21.715 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-02T15:25:21.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.912+0000 7f033ae00700 1 -- 172.21.15.1:0/521852776 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0334101f30 msgr2=0x7f0334102390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:21.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.912+0000 7f033ae00700 1 --2- 172.21.15.1:0/521852776 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0334101f30 0x7f0334102390 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f031c0099c0 tx=0x7f031c009cd0 comp rx=0 tx=0).stop 2024-02-02T15:25:21.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.912+0000 7f033ae00700 1 -- 172.21.15.1:0/521852776 shutdown_connections 2024-02-02T15:25:21.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.912+0000 7f033ae00700 1 --2- 172.21.15.1:0/521852776 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f03341028d0 0x7f0334109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:21.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.912+0000 7f033ae00700 1 --2- 172.21.15.1:0/521852776 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0334100d40 0x7f0334101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:21.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.912+0000 7f033ae00700 1 --2- 172.21.15.1:0/521852776 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0334101f30 0x7f0334102390 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:21.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.912+0000 7f033ae00700 1 -- 172.21.15.1:0/521852776 >> 172.21.15.1:0/521852776 conn(0x7f03340fc290 msgr2=0x7f03340fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:21.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.912+0000 7f033ae00700 1 -- 172.21.15.1:0/521852776 shutdown_connections 2024-02-02T15:25:21.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.912+0000 7f033ae00700 1 -- 172.21.15.1:0/521852776 wait complete. 2024-02-02T15:25:21.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.913+0000 7f033ae00700 1 Processor -- start 2024-02-02T15:25:21.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.914+0000 7f033ae00700 1 -- start start 2024-02-02T15:25:21.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.914+0000 7f033ae00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0334100d40 0x7f0334198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:21.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.914+0000 7f033ae00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0334101f30 0x7f03341991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:21.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.914+0000 7f033ae00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03341028d0 0x7f03341a01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:21.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.914+0000 7f033ae00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03341a07a0 con 0x7f03341028d0 2024-02-02T15:25:21.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.914+0000 7f033ae00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03341a0910 con 0x7f0334100d40 2024-02-02T15:25:21.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.914+0000 7f033ae00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03341a0ad0 con 0x7f0334101f30 2024-02-02T15:25:21.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.914+0000 7f0338e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03341028d0 0x7f03341a01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:21.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.914+0000 7f0338e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03341028d0 0x7f03341a01d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33156/0 (socket says 172.21.15.1:33156) 2024-02-02T15:25:21.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.914+0000 7f0338e00700 1 -- 172.21.15.1:0/364698335 learned_addr learned my addr 172.21.15.1:0/364698335 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:25:21.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.914+0000 7f0333e00700 1 --2- 172.21.15.1:0/364698335 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0334100d40 0x7f0334198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:21.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.914+0000 7f0333400700 1 --2- 172.21.15.1:0/364698335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0334101f30 0x7f03341991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:21.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.915+0000 7f0333e00700 1 -- 172.21.15.1:0/364698335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0334101f30 msgr2=0x7f03341991a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:21.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.915+0000 7f0333e00700 1 --2- 172.21.15.1:0/364698335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0334101f30 0x7f03341991a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:21.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.915+0000 7f0333e00700 1 -- 172.21.15.1:0/364698335 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03341028d0 msgr2=0x7f03341a01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:21.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.915+0000 7f0333e00700 1 --2- 172.21.15.1:0/364698335 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03341028d0 0x7f03341a01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:21.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.915+0000 7f0333e00700 1 -- 172.21.15.1:0/364698335 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f031c0095f0 con 0x7f0334100d40 2024-02-02T15:25:21.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.915+0000 7f0338e00700 1 --2- 172.21.15.1:0/364698335 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03341028d0 0x7f03341a01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:25:21.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.915+0000 7f0333e00700 1 --2- 172.21.15.1:0/364698335 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0334100d40 0x7f0334198c60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f032400e9c0 tx=0x7f032400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:21.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.915+0000 7f0330c00700 1 -- 172.21.15.1:0/364698335 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0324004220 con 0x7f0334100d40 2024-02-02T15:25:21.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.915+0000 7f0330c00700 1 -- 172.21.15.1:0/364698335 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0324004380 con 0x7f0334100d40 2024-02-02T15:25:21.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.915+0000 7f0330c00700 1 -- 172.21.15.1:0/364698335 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0324003be0 con 0x7f0334100d40 2024-02-02T15:25:21.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.915+0000 7f033ae00700 1 -- 172.21.15.1:0/364698335 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03341a0db0 con 0x7f0334100d40 2024-02-02T15:25:21.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.915+0000 7f033ae00700 1 -- 172.21.15.1:0/364698335 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03341a11b0 con 0x7f0334100d40 2024-02-02T15:25:21.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.916+0000 7f033ae00700 1 -- 172.21.15.1:0/364698335 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f033410c5d0 con 0x7f0334100d40 2024-02-02T15:25:21.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.918+0000 7f0330c00700 1 -- 172.21.15.1:0/364698335 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 135120+0+0 (secure 0 0 0) 0x7f0324003d40 con 0x7f0334100d40 2024-02-02T15:25:21.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.920+0000 7f0330c00700 1 --2- 172.21.15.1:0/364698335 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f03140a05b0 0x7f03140a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:21.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.920+0000 7f0330c00700 1 -- 172.21.15.1:0/364698335 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f0324014070 con 0x7f0334100d40 2024-02-02T15:25:21.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.920+0000 7f0333400700 1 --2- 172.21.15.1:0/364698335 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f03140a05b0 0x7f03140a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:21.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.921+0000 7f0333400700 1 --2- 172.21.15.1:0/364698335 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f03140a05b0 0x7f03140a2a70 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f031c00b580 tx=0x7f031c005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:21.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:21.922+0000 7f0330c00700 1 -- 172.21.15.1:0/364698335 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f03240957d0 con 0x7f0334100d40 2024-02-02T15:25:22.114 INFO:journalctl@ceph.mds.b.smithi071.stdout:Feb 02 15:25:21 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mds-b[79660]: 2024-02-02T15:25:21.824+0000 7f2045bbd700 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-02-02T15:25:22.114 INFO:journalctl@ceph.mds.b.smithi071.stdout:Feb 02 15:25:21 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mds-b[79660]: 2024-02-02T15:25:21.824+0000 7f2045bbd700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-02-02T15:25:22.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:22.128+0000 7f033ae00700 1 -- 172.21.15.1:0/364698335 --> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0334108880 con 0x7f03140a05b0 2024-02-02T15:25:22.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:22.144+0000 7f0330c00700 1 -- 172.21.15.1:0/364698335 <== mgr.14154 v2:172.21.15.1:6800/3768989981 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+423119 (secure 0 0 0) 0x7f0334108880 con 0x7f03140a05b0 2024-02-02T15:25:22.150 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:25:22.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:22.149+0000 7f033ae00700 1 -- 172.21.15.1:0/364698335 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f03140a05b0 msgr2=0x7f03140a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:22.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:22.149+0000 7f033ae00700 1 --2- 172.21.15.1:0/364698335 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f03140a05b0 0x7f03140a2a70 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f031c00b580 tx=0x7f031c005fb0 comp rx=0 tx=0).stop 2024-02-02T15:25:22.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:22.149+0000 7f033ae00700 1 -- 172.21.15.1:0/364698335 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0334100d40 msgr2=0x7f0334198c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:22.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:22.149+0000 7f033ae00700 1 --2- 172.21.15.1:0/364698335 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0334100d40 0x7f0334198c60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f032400e9c0 tx=0x7f032400ecd0 comp rx=0 tx=0).stop 2024-02-02T15:25:22.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:22.149+0000 7f033ae00700 1 -- 172.21.15.1:0/364698335 shutdown_connections 2024-02-02T15:25:22.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:22.149+0000 7f033ae00700 1 --2- 172.21.15.1:0/364698335 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f03341028d0 0x7f03341a01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:22.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:22.149+0000 7f033ae00700 1 --2- 172.21.15.1:0/364698335 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0334100d40 0x7f0334198c60 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:22.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:22.149+0000 7f033ae00700 1 --2- 172.21.15.1:0/364698335 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f03140a05b0 0x7f03140a2a70 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:22.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:22.149+0000 7f033ae00700 1 --2- 172.21.15.1:0/364698335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0334101f30 0x7f03341991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:22.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:22.149+0000 7f033ae00700 1 -- 172.21.15.1:0/364698335 >> 172.21.15.1:0/364698335 conn(0x7f03340fc290 msgr2=0x7f03340fe520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:22.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:22.149+0000 7f033ae00700 1 -- 172.21.15.1:0/364698335 shutdown_connections 2024-02-02T15:25:22.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:22.149+0000 7f033ae00700 1 -- 172.21.15.1:0/364698335 wait complete. 2024-02-02T15:25:22.155 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-02-02T15:25:22.167 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":252,"stamp":"2024-02-02T15:25:21.912083+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":633691,"num_objects":77,"num_object_clones":0,"num_object_copies":232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":152,"num_read_kb":164,"num_write":355,"num_write_kb":3031,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":49,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":290,"ondisk_log_size":290,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335336,"kb_used_data":11144,"kb_used_omap":20,"kb_used_meta":324075,"kb_avail":1124606488,"statfs":{"total":1151940427776,"available":1151597043712,"internally_reserved":0,"allocated":11411456,"data_stored":7291811,"data_compressed":249417,"data_compressed_allocated":331776,"data_compressed_original":2162688,"omap_allocated":21372,"internal_metadata":331852932},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":38871,"num_objects":53,"num_object_clones":0,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":16,"num_read_kb":45,"num_write":127,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003833"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.800354+0000","last_change":"2024-02-02T15:24:47.793308+0000","last_active":"2024-02-02T15:24:51.800354+0000","last_peered":"2024-02-02T15:24:51.800354+0000","last_clean":"2024-02-02T15:24:51.800354+0000","last_became_active":"2024-02-02T15:24:47.792420+0000","last_became_peered":"2024-02-02T15:24:47.792420+0000","last_unstale":"2024-02-02T15:24:51.800354+0000","last_undegraded":"2024-02-02T15:24:51.800354+0000","last_fullsized":"2024-02-02T15:24:51.800354+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:39:09.871317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.808238+0000","last_change":"2024-02-02T15:24:47.791272+0000","last_active":"2024-02-02T15:24:51.808238+0000","last_peered":"2024-02-02T15:24:51.808238+0000","last_clean":"2024-02-02T15:24:51.808238+0000","last_became_active":"2024-02-02T15:24:47.790902+0000","last_became_peered":"2024-02-02T15:24:47.790902+0000","last_unstale":"2024-02-02T15:24:51.808238+0000","last_undegraded":"2024-02-02T15:24:51.808238+0000","last_fullsized":"2024-02-02T15:24:51.808238+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:31:11.620873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.805893+0000","last_change":"2024-02-02T15:24:47.798712+0000","last_active":"2024-02-02T15:24:51.805893+0000","last_peered":"2024-02-02T15:24:51.805893+0000","last_clean":"2024-02-02T15:24:51.805893+0000","last_became_active":"2024-02-02T15:24:47.798405+0000","last_became_peered":"2024-02-02T15:24:47.798405+0000","last_unstale":"2024-02-02T15:24:51.805893+0000","last_undegraded":"2024-02-02T15:24:51.805893+0000","last_fullsized":"2024-02-02T15:24:51.805893+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:37:15.238070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.800684+0000","last_change":"2024-02-02T15:24:47.786456+0000","last_active":"2024-02-02T15:24:51.800684+0000","last_peered":"2024-02-02T15:24:51.800684+0000","last_clean":"2024-02-02T15:24:51.800684+0000","last_became_active":"2024-02-02T15:24:47.785520+0000","last_became_peered":"2024-02-02T15:24:47.785520+0000","last_unstale":"2024-02-02T15:24:51.800684+0000","last_undegraded":"2024-02-02T15:24:51.800684+0000","last_fullsized":"2024-02-02T15:24:51.800684+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:16:04.003459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.801660+0000","last_change":"2024-02-02T15:24:47.780084+0000","last_active":"2024-02-02T15:24:51.801660+0000","last_peered":"2024-02-02T15:24:51.801660+0000","last_clean":"2024-02-02T15:24:51.801660+0000","last_became_active":"2024-02-02T15:24:47.779667+0000","last_became_peered":"2024-02-02T15:24:47.779667+0000","last_unstale":"2024-02-02T15:24:51.801660+0000","last_undegraded":"2024-02-02T15:24:51.801660+0000","last_fullsized":"2024-02-02T15:24:51.801660+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:30:15.377691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.810959+0000","last_change":"2024-02-02T15:24:47.786173+0000","last_active":"2024-02-02T15:24:51.810959+0000","last_peered":"2024-02-02T15:24:51.810959+0000","last_clean":"2024-02-02T15:24:51.810959+0000","last_became_active":"2024-02-02T15:24:47.785301+0000","last_became_peered":"2024-02-02T15:24:47.785301+0000","last_unstale":"2024-02-02T15:24:51.810959+0000","last_undegraded":"2024-02-02T15:24:51.810959+0000","last_fullsized":"2024-02-02T15:24:51.810959+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:14:58.524317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.800609+0000","last_change":"2024-02-02T15:24:47.790913+0000","last_active":"2024-02-02T15:24:51.800609+0000","last_peered":"2024-02-02T15:24:51.800609+0000","last_clean":"2024-02-02T15:24:51.800609+0000","last_became_active":"2024-02-02T15:24:47.790566+0000","last_became_peered":"2024-02-02T15:24:47.790566+0000","last_unstale":"2024-02-02T15:24:51.800609+0000","last_undegraded":"2024-02-02T15:24:51.800609+0000","last_fullsized":"2024-02-02T15:24:51.800609+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:34:51.954830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:52.000920+0000","last_change":"2024-02-02T15:24:47.786465+0000","last_active":"2024-02-02T15:24:52.000920+0000","last_peered":"2024-02-02T15:24:52.000920+0000","last_clean":"2024-02-02T15:24:52.000920+0000","last_became_active":"2024-02-02T15:24:47.786175+0000","last_became_peered":"2024-02-02T15:24:47.786175+0000","last_unstale":"2024-02-02T15:24:52.000920+0000","last_undegraded":"2024-02-02T15:24:52.000920+0000","last_fullsized":"2024-02-02T15:24:52.000920+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:42:13.748850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.795081+0000","last_change":"2024-02-02T15:24:47.787115+0000","last_active":"2024-02-02T15:24:51.795081+0000","last_peered":"2024-02-02T15:24:51.795081+0000","last_clean":"2024-02-02T15:24:51.795081+0000","last_became_active":"2024-02-02T15:24:47.786804+0000","last_became_peered":"2024-02-02T15:24:47.786804+0000","last_unstale":"2024-02-02T15:24:51.795081+0000","last_undegraded":"2024-02-02T15:24:51.795081+0000","last_fullsized":"2024-02-02T15:24:51.795081+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T03:06:17.873243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.811379+0000","last_change":"2024-02-02T15:24:47.787211+0000","last_active":"2024-02-02T15:24:51.811379+0000","last_peered":"2024-02-02T15:24:51.811379+0000","last_clean":"2024-02-02T15:24:51.811379+0000","last_became_active":"2024-02-02T15:24:47.786888+0000","last_became_peered":"2024-02-02T15:24:47.786888+0000","last_unstale":"2024-02-02T15:24:51.811379+0000","last_undegraded":"2024-02-02T15:24:51.811379+0000","last_fullsized":"2024-02-02T15:24:51.811379+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:14:21.442787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.800451+0000","last_change":"2024-02-02T15:24:47.798962+0000","last_active":"2024-02-02T15:24:51.800451+0000","last_peered":"2024-02-02T15:24:51.800451+0000","last_clean":"2024-02-02T15:24:51.800451+0000","last_became_active":"2024-02-02T15:24:47.798657+0000","last_became_peered":"2024-02-02T15:24:47.798657+0000","last_unstale":"2024-02-02T15:24:51.800451+0000","last_undegraded":"2024-02-02T15:24:51.800451+0000","last_fullsized":"2024-02-02T15:24:51.800451+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:53:37.316027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.800567+0000","last_change":"2024-02-02T15:24:47.782647+0000","last_active":"2024-02-02T15:24:51.800567+0000","last_peered":"2024-02-02T15:24:51.800567+0000","last_clean":"2024-02-02T15:24:51.800567+0000","last_became_active":"2024-02-02T15:24:47.782344+0000","last_became_peered":"2024-02-02T15:24:47.782344+0000","last_unstale":"2024-02-02T15:24:51.800567+0000","last_undegraded":"2024-02-02T15:24:51.800567+0000","last_fullsized":"2024-02-02T15:24:51.800567+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:06:54.874657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.794123+0000","last_change":"2024-02-02T15:24:47.787032+0000","last_active":"2024-02-02T15:24:51.794123+0000","last_peered":"2024-02-02T15:24:51.794123+0000","last_clean":"2024-02-02T15:24:51.794123+0000","last_became_active":"2024-02-02T15:24:47.786710+0000","last_became_peered":"2024-02-02T15:24:47.786710+0000","last_unstale":"2024-02-02T15:24:51.794123+0000","last_undegraded":"2024-02-02T15:24:51.794123+0000","last_fullsized":"2024-02-02T15:24:51.794123+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:26:15.429316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.804049+0000","last_change":"2024-02-02T15:24:47.785708+0000","last_active":"2024-02-02T15:24:51.804049+0000","last_peered":"2024-02-02T15:24:51.804049+0000","last_clean":"2024-02-02T15:24:51.804049+0000","last_became_active":"2024-02-02T15:24:47.785280+0000","last_became_peered":"2024-02-02T15:24:47.785280+0000","last_unstale":"2024-02-02T15:24:51.804049+0000","last_undegraded":"2024-02-02T15:24:51.804049+0000","last_fullsized":"2024-02-02T15:24:51.804049+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T02:21:23.224933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.804879+0000","last_change":"2024-02-02T15:24:47.783904+0000","last_active":"2024-02-02T15:24:51.804879+0000","last_peered":"2024-02-02T15:24:51.804879+0000","last_clean":"2024-02-02T15:24:51.804879+0000","last_became_active":"2024-02-02T15:24:47.783599+0000","last_became_peered":"2024-02-02T15:24:47.783599+0000","last_unstale":"2024-02-02T15:24:51.804879+0000","last_undegraded":"2024-02-02T15:24:51.804879+0000","last_fullsized":"2024-02-02T15:24:51.804879+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:35:45.805527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.794661+0000","last_change":"2024-02-02T15:24:47.795770+0000","last_active":"2024-02-02T15:24:51.794661+0000","last_peered":"2024-02-02T15:24:51.794661+0000","last_clean":"2024-02-02T15:24:51.794661+0000","last_became_active":"2024-02-02T15:24:47.795433+0000","last_became_peered":"2024-02-02T15:24:47.795433+0000","last_unstale":"2024-02-02T15:24:51.794661+0000","last_undegraded":"2024-02-02T15:24:51.794661+0000","last_fullsized":"2024-02-02T15:24:51.794661+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:50:07.496998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.800219+0000","last_change":"2024-02-02T15:24:47.793684+0000","last_active":"2024-02-02T15:24:51.800219+0000","last_peered":"2024-02-02T15:24:51.800219+0000","last_clean":"2024-02-02T15:24:51.800219+0000","last_became_active":"2024-02-02T15:24:47.792853+0000","last_became_peered":"2024-02-02T15:24:47.792853+0000","last_unstale":"2024-02-02T15:24:51.800219+0000","last_undegraded":"2024-02-02T15:24:51.800219+0000","last_fullsized":"2024-02-02T15:24:51.800219+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:30:40.379379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.806672+0000","last_change":"2024-02-02T15:24:47.780935+0000","last_active":"2024-02-02T15:24:51.806672+0000","last_peered":"2024-02-02T15:24:51.806672+0000","last_clean":"2024-02-02T15:24:51.806672+0000","last_became_active":"2024-02-02T15:24:47.780642+0000","last_became_peered":"2024-02-02T15:24:47.780642+0000","last_unstale":"2024-02-02T15:24:51.806672+0000","last_undegraded":"2024-02-02T15:24:51.806672+0000","last_fullsized":"2024-02-02T15:24:51.806672+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:36:37.615368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.946993+0000","last_change":"2024-02-02T15:24:47.791051+0000","last_active":"2024-02-02T15:24:51.946993+0000","last_peered":"2024-02-02T15:24:51.946993+0000","last_clean":"2024-02-02T15:24:51.946993+0000","last_became_active":"2024-02-02T15:24:47.790732+0000","last_became_peered":"2024-02-02T15:24:47.790732+0000","last_unstale":"2024-02-02T15:24:51.946993+0000","last_undegraded":"2024-02-02T15:24:51.946993+0000","last_fullsized":"2024-02-02T15:24:51.946993+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:45:05.155144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.946914+0000","last_change":"2024-02-02T15:24:51.814699+0000","last_active":"2024-02-02T15:24:51.946914+0000","last_peered":"2024-02-02T15:24:51.946914+0000","last_clean":"2024-02-02T15:24:51.946914+0000","last_became_active":"2024-02-02T15:24:51.814251+0000","last_became_peered":"2024-02-02T15:24:51.814251+0000","last_unstale":"2024-02-02T15:24:51.946914+0000","last_undegraded":"2024-02-02T15:24:51.946914+0000","last_fullsized":"2024-02-02T15:24:51.946914+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:27:21.674922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"84'11","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:25:18.234367+0000","last_change":"2024-02-02T15:24:47.269429+0000","last_active":"2024-02-02T15:25:18.234367+0000","last_peered":"2024-02-02T15:25:18.234367+0000","last_clean":"2024-02-02T15:25:18.234367+0000","last_became_active":"2024-02-02T15:24:47.269023+0000","last_became_peered":"2024-02-02T15:24:47.269023+0000","last_unstale":"2024-02-02T15:25:18.234367+0000","last_undegraded":"2024-02-02T15:25:18.234367+0000","last_fullsized":"2024-02-02T15:25:18.234367+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T03:11:46.344438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:52.001073+0000","last_change":"2024-02-02T15:24:47.785900+0000","last_active":"2024-02-02T15:24:52.001073+0000","last_peered":"2024-02-02T15:24:52.001073+0000","last_clean":"2024-02-02T15:24:52.001073+0000","last_became_active":"2024-02-02T15:24:47.781994+0000","last_became_peered":"2024-02-02T15:24:47.781994+0000","last_unstale":"2024-02-02T15:24:52.001073+0000","last_undegraded":"2024-02-02T15:24:52.001073+0000","last_fullsized":"2024-02-02T15:24:52.001073+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:00:52.432637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:24:53.093098+0000","last_change":"2024-02-02T15:24:51.825655+0000","last_active":"2024-02-02T15:24:53.093098+0000","last_peered":"2024-02-02T15:24:53.093098+0000","last_clean":"2024-02-02T15:24:53.093098+0000","last_became_active":"2024-02-02T15:24:51.825199+0000","last_became_peered":"2024-02-02T15:24:51.825199+0000","last_unstale":"2024-02-02T15:24:53.093098+0000","last_undegraded":"2024-02-02T15:24:53.093098+0000","last_fullsized":"2024-02-02T15:24:53.093098+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:50:32.283394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"79'1","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.804946+0000","last_change":"2024-02-02T15:24:47.281869+0000","last_active":"2024-02-02T15:24:51.804946+0000","last_peered":"2024-02-02T15:24:51.804946+0000","last_clean":"2024-02-02T15:24:51.804946+0000","last_became_active":"2024-02-02T15:24:47.280825+0000","last_became_peered":"2024-02-02T15:24:47.280825+0000","last_unstale":"2024-02-02T15:24:51.804946+0000","last_undegraded":"2024-02-02T15:24:51.804946+0000","last_fullsized":"2024-02-02T15:24:51.804946+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:24:19.688389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.806330+0000","last_change":"2024-02-02T15:24:47.795893+0000","last_active":"2024-02-02T15:24:51.806330+0000","last_peered":"2024-02-02T15:24:51.806330+0000","last_clean":"2024-02-02T15:24:51.806330+0000","last_became_active":"2024-02-02T15:24:47.795491+0000","last_became_peered":"2024-02-02T15:24:47.795491+0000","last_unstale":"2024-02-02T15:24:51.806330+0000","last_undegraded":"2024-02-02T15:24:51.806330+0000","last_fullsized":"2024-02-02T15:24:51.806330+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:54:38.923036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:24:53.094510+0000","last_change":"2024-02-02T15:24:51.826980+0000","last_active":"2024-02-02T15:24:53.094510+0000","last_peered":"2024-02-02T15:24:53.094510+0000","last_clean":"2024-02-02T15:24:53.094510+0000","last_became_active":"2024-02-02T15:24:51.826221+0000","last_became_peered":"2024-02-02T15:24:51.826221+0000","last_unstale":"2024-02-02T15:24:53.094510+0000","last_undegraded":"2024-02-02T15:24:53.094510+0000","last_fullsized":"2024-02-02T15:24:53.094510+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:45:17.126174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"84'4","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:25:17.226185+0000","last_change":"2024-02-02T15:24:47.272219+0000","last_active":"2024-02-02T15:25:17.226185+0000","last_peered":"2024-02-02T15:25:17.226185+0000","last_clean":"2024-02-02T15:25:17.226185+0000","last_became_active":"2024-02-02T15:24:47.271899+0000","last_became_peered":"2024-02-02T15:24:47.271899+0000","last_unstale":"2024-02-02T15:25:17.226185+0000","last_undegraded":"2024-02-02T15:25:17.226185+0000","last_fullsized":"2024-02-02T15:25:17.226185+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:44:27.370783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.795158+0000","last_change":"2024-02-02T15:24:47.783434+0000","last_active":"2024-02-02T15:24:51.795158+0000","last_peered":"2024-02-02T15:24:51.795158+0000","last_clean":"2024-02-02T15:24:51.795158+0000","last_became_active":"2024-02-02T15:24:47.782130+0000","last_became_peered":"2024-02-02T15:24:47.782130+0000","last_unstale":"2024-02-02T15:24:51.795158+0000","last_undegraded":"2024-02-02T15:24:51.795158+0000","last_fullsized":"2024-02-02T15:24:51.795158+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:32:25.981913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:24:53.094617+0000","last_change":"2024-02-02T15:24:51.810791+0000","last_active":"2024-02-02T15:24:53.094617+0000","last_peered":"2024-02-02T15:24:53.094617+0000","last_clean":"2024-02-02T15:24:53.094617+0000","last_became_active":"2024-02-02T15:24:51.810379+0000","last_became_peered":"2024-02-02T15:24:51.810379+0000","last_unstale":"2024-02-02T15:24:53.094617+0000","last_undegraded":"2024-02-02T15:24:53.094617+0000","last_fullsized":"2024-02-02T15:24:53.094617+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:49:44.635820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.802389+0000","last_change":"2024-02-02T15:24:47.271225+0000","last_active":"2024-02-02T15:24:51.802389+0000","last_peered":"2024-02-02T15:24:51.802389+0000","last_clean":"2024-02-02T15:24:51.802389+0000","last_became_active":"2024-02-02T15:24:47.270870+0000","last_became_peered":"2024-02-02T15:24:47.270870+0000","last_unstale":"2024-02-02T15:24:51.802389+0000","last_undegraded":"2024-02-02T15:24:51.802389+0000","last_fullsized":"2024-02-02T15:24:51.802389+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:16:55.659950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.801631+0000","last_change":"2024-02-02T15:24:47.791367+0000","last_active":"2024-02-02T15:24:51.801631+0000","last_peered":"2024-02-02T15:24:51.801631+0000","last_clean":"2024-02-02T15:24:51.801631+0000","last_became_active":"2024-02-02T15:24:47.790835+0000","last_became_peered":"2024-02-02T15:24:47.790835+0000","last_unstale":"2024-02-02T15:24:51.801631+0000","last_undegraded":"2024-02-02T15:24:51.801631+0000","last_fullsized":"2024-02-02T15:24:51.801631+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:02:51.807571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.946981+0000","last_change":"2024-02-02T15:24:51.807328+0000","last_active":"2024-02-02T15:24:51.946981+0000","last_peered":"2024-02-02T15:24:51.946981+0000","last_clean":"2024-02-02T15:24:51.946981+0000","last_became_active":"2024-02-02T15:24:51.806853+0000","last_became_peered":"2024-02-02T15:24:51.806853+0000","last_unstale":"2024-02-02T15:24:51.946981+0000","last_undegraded":"2024-02-02T15:24:51.946981+0000","last_fullsized":"2024-02-02T15:24:51.946981+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:46:13.898450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"84'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:25:20.260444+0000","last_change":"2024-02-02T15:24:47.268192+0000","last_active":"2024-02-02T15:25:20.260444+0000","last_peered":"2024-02-02T15:25:20.260444+0000","last_clean":"2024-02-02T15:25:20.260444+0000","last_became_active":"2024-02-02T15:24:47.267796+0000","last_became_peered":"2024-02-02T15:24:47.267796+0000","last_unstale":"2024-02-02T15:25:20.260444+0000","last_undegraded":"2024-02-02T15:25:20.260444+0000","last_fullsized":"2024-02-02T15:25:20.260444+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:12:13.161690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.801719+0000","last_change":"2024-02-02T15:24:47.782976+0000","last_active":"2024-02-02T15:24:51.801719+0000","last_peered":"2024-02-02T15:24:51.801719+0000","last_clean":"2024-02-02T15:24:51.801719+0000","last_became_active":"2024-02-02T15:24:47.782640+0000","last_became_peered":"2024-02-02T15:24:47.782640+0000","last_unstale":"2024-02-02T15:24:51.801719+0000","last_undegraded":"2024-02-02T15:24:51.801719+0000","last_fullsized":"2024-02-02T15:24:51.801719+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T02:09:54.851418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.946963+0000","last_change":"2024-02-02T15:24:51.814799+0000","last_active":"2024-02-02T15:24:51.946963+0000","last_peered":"2024-02-02T15:24:51.946963+0000","last_clean":"2024-02-02T15:24:51.946963+0000","last_became_active":"2024-02-02T15:24:51.814263+0000","last_became_peered":"2024-02-02T15:24:51.814263+0000","last_unstale":"2024-02-02T15:24:51.946963+0000","last_undegraded":"2024-02-02T15:24:51.946963+0000","last_fullsized":"2024-02-02T15:24:51.946963+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:52:02.786496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"84'7","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:25:18.235072+0000","last_change":"2024-02-02T15:24:47.273204+0000","last_active":"2024-02-02T15:25:18.235072+0000","last_peered":"2024-02-02T15:25:18.235072+0000","last_clean":"2024-02-02T15:25:18.235072+0000","last_became_active":"2024-02-02T15:24:47.272888+0000","last_became_peered":"2024-02-02T15:24:47.272888+0000","last_unstale":"2024-02-02T15:25:18.235072+0000","last_undegraded":"2024-02-02T15:25:18.235072+0000","last_fullsized":"2024-02-02T15:25:18.235072+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:26:25.188915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.797918+0000","last_change":"2024-02-02T15:24:47.788645+0000","last_active":"2024-02-02T15:24:51.797918+0000","last_peered":"2024-02-02T15:24:51.797918+0000","last_clean":"2024-02-02T15:24:51.797918+0000","last_became_active":"2024-02-02T15:24:47.787478+0000","last_became_peered":"2024-02-02T15:24:47.787478+0000","last_unstale":"2024-02-02T15:24:51.797918+0000","last_undegraded":"2024-02-02T15:24:51.797918+0000","last_fullsized":"2024-02-02T15:24:51.797918+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:24:45.327030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:24:52.868735+0000","last_change":"2024-02-02T15:24:51.822681+0000","last_active":"2024-02-02T15:24:52.868735+0000","last_peered":"2024-02-02T15:24:52.868735+0000","last_clean":"2024-02-02T15:24:52.868735+0000","last_became_active":"2024-02-02T15:24:51.822202+0000","last_became_peered":"2024-02-02T15:24:51.822202+0000","last_unstale":"2024-02-02T15:24:52.868735+0000","last_undegraded":"2024-02-02T15:24:52.868735+0000","last_fullsized":"2024-02-02T15:24:52.868735+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:05:43.751558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"84'2","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:25:17.229705+0000","last_change":"2024-02-02T15:24:47.271095+0000","last_active":"2024-02-02T15:25:17.229705+0000","last_peered":"2024-02-02T15:25:17.229705+0000","last_clean":"2024-02-02T15:25:17.229705+0000","last_became_active":"2024-02-02T15:24:47.270696+0000","last_became_peered":"2024-02-02T15:24:47.270696+0000","last_unstale":"2024-02-02T15:25:17.229705+0000","last_undegraded":"2024-02-02T15:25:17.229705+0000","last_fullsized":"2024-02-02T15:25:17.229705+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T02:20:59.370217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:52.000970+0000","last_change":"2024-02-02T15:24:47.791699+0000","last_active":"2024-02-02T15:24:52.000970+0000","last_peered":"2024-02-02T15:24:52.000970+0000","last_clean":"2024-02-02T15:24:52.000970+0000","last_became_active":"2024-02-02T15:24:47.787280+0000","last_became_peered":"2024-02-02T15:24:47.787280+0000","last_unstale":"2024-02-02T15:24:52.000970+0000","last_undegraded":"2024-02-02T15:24:52.000970+0000","last_fullsized":"2024-02-02T15:24:52.000970+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:18:10.316393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.812325+0000","last_change":"2024-02-02T15:24:51.810050+0000","last_active":"2024-02-02T15:24:51.812325+0000","last_peered":"2024-02-02T15:24:51.812325+0000","last_clean":"2024-02-02T15:24:51.812325+0000","last_became_active":"2024-02-02T15:24:51.809621+0000","last_became_peered":"2024-02-02T15:24:51.809621+0000","last_unstale":"2024-02-02T15:24:51.812325+0000","last_undegraded":"2024-02-02T15:24:51.812325+0000","last_fullsized":"2024-02-02T15:24:51.812325+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:52:35.885973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"84'3","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:25:19.241022+0000","last_change":"2024-02-02T15:24:47.267973+0000","last_active":"2024-02-02T15:25:19.241022+0000","last_peered":"2024-02-02T15:25:19.241022+0000","last_clean":"2024-02-02T15:25:19.241022+0000","last_became_active":"2024-02-02T15:24:47.267649+0000","last_became_peered":"2024-02-02T15:24:47.267649+0000","last_unstale":"2024-02-02T15:25:19.241022+0000","last_undegraded":"2024-02-02T15:25:19.241022+0000","last_fullsized":"2024-02-02T15:25:19.241022+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T02:46:56.844570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.802331+0000","last_change":"2024-02-02T15:24:47.798636+0000","last_active":"2024-02-02T15:24:51.802331+0000","last_peered":"2024-02-02T15:24:51.802331+0000","last_clean":"2024-02-02T15:24:51.802331+0000","last_became_active":"2024-02-02T15:24:47.798328+0000","last_became_peered":"2024-02-02T15:24:47.798328+0000","last_unstale":"2024-02-02T15:24:51.802331+0000","last_undegraded":"2024-02-02T15:24:51.802331+0000","last_fullsized":"2024-02-02T15:24:51.802331+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:37:43.687336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:52.000988+0000","last_change":"2024-02-02T15:24:51.811312+0000","last_active":"2024-02-02T15:24:52.000988+0000","last_peered":"2024-02-02T15:24:52.000988+0000","last_clean":"2024-02-02T15:24:52.000988+0000","last_became_active":"2024-02-02T15:24:51.810932+0000","last_became_peered":"2024-02-02T15:24:51.810932+0000","last_unstale":"2024-02-02T15:24:52.000988+0000","last_undegraded":"2024-02-02T15:24:52.000988+0000","last_fullsized":"2024-02-02T15:24:52.000988+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:10:31.530518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"84'4","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:25:20.260121+0000","last_change":"2024-02-02T15:24:47.271483+0000","last_active":"2024-02-02T15:25:20.260121+0000","last_peered":"2024-02-02T15:25:20.260121+0000","last_clean":"2024-02-02T15:25:20.260121+0000","last_became_active":"2024-02-02T15:24:47.271160+0000","last_became_peered":"2024-02-02T15:24:47.271160+0000","last_unstale":"2024-02-02T15:25:20.260121+0000","last_undegraded":"2024-02-02T15:25:20.260121+0000","last_fullsized":"2024-02-02T15:25:20.260121+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:29:20.135351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.795145+0000","last_change":"2024-02-02T15:24:47.786492+0000","last_active":"2024-02-02T15:24:51.795145+0000","last_peered":"2024-02-02T15:24:51.795145+0000","last_clean":"2024-02-02T15:24:51.795145+0000","last_became_active":"2024-02-02T15:24:47.786103+0000","last_became_peered":"2024-02-02T15:24:47.786103+0000","last_unstale":"2024-02-02T15:24:51.795145+0000","last_undegraded":"2024-02-02T15:24:51.795145+0000","last_fullsized":"2024-02-02T15:24:51.795145+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:24:23.666596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:24:53.093337+0000","last_change":"2024-02-02T15:24:51.819941+0000","last_active":"2024-02-02T15:24:53.093337+0000","last_peered":"2024-02-02T15:24:53.093337+0000","last_clean":"2024-02-02T15:24:53.093337+0000","last_became_active":"2024-02-02T15:24:51.818066+0000","last_became_peered":"2024-02-02T15:24:51.818066+0000","last_unstale":"2024-02-02T15:24:53.093337+0000","last_undegraded":"2024-02-02T15:24:53.093337+0000","last_fullsized":"2024-02-02T15:24:53.093337+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:55:51.550988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"84'4","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:25:19.241307+0000","last_change":"2024-02-02T15:24:47.268541+0000","last_active":"2024-02-02T15:25:19.241307+0000","last_peered":"2024-02-02T15:25:19.241307+0000","last_clean":"2024-02-02T15:25:19.241307+0000","last_became_active":"2024-02-02T15:24:47.268184+0000","last_became_peered":"2024-02-02T15:24:47.268184+0000","last_unstale":"2024-02-02T15:25:19.241307+0000","last_undegraded":"2024-02-02T15:25:19.241307+0000","last_fullsized":"2024-02-02T15:25:19.241307+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:30:12.187297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.806739+0000","last_change":"2024-02-02T15:24:47.787792+0000","last_active":"2024-02-02T15:24:51.806739+0000","last_peered":"2024-02-02T15:24:51.806739+0000","last_clean":"2024-02-02T15:24:51.806739+0000","last_became_active":"2024-02-02T15:24:47.787479+0000","last_became_peered":"2024-02-02T15:24:47.787479+0000","last_unstale":"2024-02-02T15:24:51.806739+0000","last_undegraded":"2024-02-02T15:24:51.806739+0000","last_fullsized":"2024-02-02T15:24:51.806739+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T03:24:06.544232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:24:53.094691+0000","last_change":"2024-02-02T15:24:51.826639+0000","last_active":"2024-02-02T15:24:53.094691+0000","last_peered":"2024-02-02T15:24:53.094691+0000","last_clean":"2024-02-02T15:24:53.094691+0000","last_became_active":"2024-02-02T15:24:51.825089+0000","last_became_peered":"2024-02-02T15:24:51.825089+0000","last_unstale":"2024-02-02T15:24:53.094691+0000","last_undegraded":"2024-02-02T15:24:53.094691+0000","last_fullsized":"2024-02-02T15:24:53.094691+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:41:44.592603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"84'5","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:25:18.233151+0000","last_change":"2024-02-02T15:24:47.281490+0000","last_active":"2024-02-02T15:25:18.233151+0000","last_peered":"2024-02-02T15:25:18.233151+0000","last_clean":"2024-02-02T15:25:18.233151+0000","last_became_active":"2024-02-02T15:24:47.281137+0000","last_became_peered":"2024-02-02T15:24:47.281137+0000","last_unstale":"2024-02-02T15:25:18.233151+0000","last_undegraded":"2024-02-02T15:25:18.233151+0000","last_fullsized":"2024-02-02T15:25:18.233151+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:42:08.508881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.795016+0000","last_change":"2024-02-02T15:24:47.781135+0000","last_active":"2024-02-02T15:24:51.795016+0000","last_peered":"2024-02-02T15:24:51.795016+0000","last_clean":"2024-02-02T15:24:51.795016+0000","last_became_active":"2024-02-02T15:24:47.780817+0000","last_became_peered":"2024-02-02T15:24:47.780817+0000","last_unstale":"2024-02-02T15:24:51.795016+0000","last_undegraded":"2024-02-02T15:24:51.795016+0000","last_fullsized":"2024-02-02T15:24:51.795016+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:33:05.862934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:24:53.093584+0000","last_change":"2024-02-02T15:24:51.822935+0000","last_active":"2024-02-02T15:24:53.093584+0000","last_peered":"2024-02-02T15:24:53.093584+0000","last_clean":"2024-02-02T15:24:53.093584+0000","last_became_active":"2024-02-02T15:24:51.822222+0000","last_became_peered":"2024-02-02T15:24:51.822222+0000","last_unstale":"2024-02-02T15:24:53.093584+0000","last_undegraded":"2024-02-02T15:24:53.093584+0000","last_fullsized":"2024-02-02T15:24:53.093584+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:54:44.578112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"84'24","reported_seq":44,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:25:20.263806+0000","last_change":"2024-02-02T15:24:47.270604+0000","last_active":"2024-02-02T15:25:20.263806+0000","last_peered":"2024-02-02T15:25:20.263806+0000","last_clean":"2024-02-02T15:25:20.263806+0000","last_became_active":"2024-02-02T15:24:47.270261+0000","last_became_peered":"2024-02-02T15:24:47.270261+0000","last_unstale":"2024-02-02T15:25:20.263806+0000","last_undegraded":"2024-02-02T15:25:20.263806+0000","last_fullsized":"2024-02-02T15:25:20.263806+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:56:33.767423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38518,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":4,"num_read_kb":35,"num_write":26,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.801998+0000","last_change":"2024-02-02T15:24:47.789663+0000","last_active":"2024-02-02T15:24:51.801998+0000","last_peered":"2024-02-02T15:24:51.801998+0000","last_clean":"2024-02-02T15:24:51.801998+0000","last_became_active":"2024-02-02T15:24:47.789352+0000","last_became_peered":"2024-02-02T15:24:47.789352+0000","last_unstale":"2024-02-02T15:24:51.801998+0000","last_undegraded":"2024-02-02T15:24:51.801998+0000","last_fullsized":"2024-02-02T15:24:51.801998+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:13:37.991350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.816451+0000","last_change":"2024-02-02T15:24:51.816021+0000","last_active":"2024-02-02T15:24:51.816451+0000","last_peered":"2024-02-02T15:24:51.816451+0000","last_clean":"2024-02-02T15:24:51.816451+0000","last_became_active":"2024-02-02T15:24:51.815698+0000","last_became_peered":"2024-02-02T15:24:51.815698+0000","last_unstale":"2024-02-02T15:24:51.816451+0000","last_undegraded":"2024-02-02T15:24:51.816451+0000","last_fullsized":"2024-02-02T15:24:51.816451+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:11:10.670195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"84'4","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:25:16.219059+0000","last_change":"2024-02-02T15:24:47.272522+0000","last_active":"2024-02-02T15:25:16.219059+0000","last_peered":"2024-02-02T15:25:16.219059+0000","last_clean":"2024-02-02T15:25:16.219059+0000","last_became_active":"2024-02-02T15:24:47.272087+0000","last_became_peered":"2024-02-02T15:24:47.272087+0000","last_unstale":"2024-02-02T15:25:16.219059+0000","last_undegraded":"2024-02-02T15:25:16.219059+0000","last_fullsized":"2024-02-02T15:25:16.219059+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:51:49.484821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.793946+0000","last_change":"2024-02-02T15:24:47.798089+0000","last_active":"2024-02-02T15:24:51.793946+0000","last_peered":"2024-02-02T15:24:51.793946+0000","last_clean":"2024-02-02T15:24:51.793946+0000","last_became_active":"2024-02-02T15:24:47.793914+0000","last_became_peered":"2024-02-02T15:24:47.793914+0000","last_unstale":"2024-02-02T15:24:51.793946+0000","last_undegraded":"2024-02-02T15:24:51.793946+0000","last_fullsized":"2024-02-02T15:24:51.793946+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:33:39.927550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:24:52.868395+0000","last_change":"2024-02-02T15:24:51.823098+0000","last_active":"2024-02-02T15:24:52.868395+0000","last_peered":"2024-02-02T15:24:52.868395+0000","last_clean":"2024-02-02T15:24:52.868395+0000","last_became_active":"2024-02-02T15:24:51.822634+0000","last_became_peered":"2024-02-02T15:24:51.822634+0000","last_unstale":"2024-02-02T15:24:52.868395+0000","last_undegraded":"2024-02-02T15:24:52.868395+0000","last_fullsized":"2024-02-02T15:24:52.868395+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T03:18:07.420196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"84'5","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:25:19.241109+0000","last_change":"2024-02-02T15:24:47.268962+0000","last_active":"2024-02-02T15:25:19.241109+0000","last_peered":"2024-02-02T15:25:19.241109+0000","last_clean":"2024-02-02T15:25:19.241109+0000","last_became_active":"2024-02-02T15:24:47.268607+0000","last_became_peered":"2024-02-02T15:24:47.268607+0000","last_unstale":"2024-02-02T15:25:19.241109+0000","last_undegraded":"2024-02-02T15:25:19.241109+0000","last_fullsized":"2024-02-02T15:25:19.241109+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:31:06.330437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.800131+0000","last_change":"2024-02-02T15:24:47.792205+0000","last_active":"2024-02-02T15:24:51.800131+0000","last_peered":"2024-02-02T15:24:51.800131+0000","last_clean":"2024-02-02T15:24:51.800131+0000","last_became_active":"2024-02-02T15:24:47.791874+0000","last_became_peered":"2024-02-02T15:24:47.791874+0000","last_unstale":"2024-02-02T15:24:51.800131+0000","last_undegraded":"2024-02-02T15:24:51.800131+0000","last_fullsized":"2024-02-02T15:24:51.800131+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:33:38.795921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:24:53.094439+0000","last_change":"2024-02-02T15:24:51.825499+0000","last_active":"2024-02-02T15:24:53.094439+0000","last_peered":"2024-02-02T15:24:53.094439+0000","last_clean":"2024-02-02T15:24:53.094439+0000","last_became_active":"2024-02-02T15:24:51.825049+0000","last_became_peered":"2024-02-02T15:24:51.825049+0000","last_unstale":"2024-02-02T15:24:53.094439+0000","last_undegraded":"2024-02-02T15:24:53.094439+0000","last_fullsized":"2024-02-02T15:24:53.094439+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:41:05.430891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"84'6","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:25:18.233586+0000","last_change":"2024-02-02T15:24:47.267683+0000","last_active":"2024-02-02T15:25:18.233586+0000","last_peered":"2024-02-02T15:25:18.233586+0000","last_clean":"2024-02-02T15:25:18.233586+0000","last_became_active":"2024-02-02T15:24:47.267324+0000","last_became_peered":"2024-02-02T15:24:47.267324+0000","last_unstale":"2024-02-02T15:25:18.233586+0000","last_undegraded":"2024-02-02T15:25:18.233586+0000","last_fullsized":"2024-02-02T15:25:18.233586+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:03:31.326791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":932,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.799771+0000","last_change":"2024-02-02T15:24:47.792509+0000","last_active":"2024-02-02T15:24:51.799771+0000","last_peered":"2024-02-02T15:24:51.799771+0000","last_clean":"2024-02-02T15:24:51.799771+0000","last_became_active":"2024-02-02T15:24:47.791662+0000","last_became_peered":"2024-02-02T15:24:47.791662+0000","last_unstale":"2024-02-02T15:24:51.799771+0000","last_undegraded":"2024-02-02T15:24:51.799771+0000","last_fullsized":"2024-02-02T15:24:51.799771+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:42:50.009971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:24:52.315377+0000","last_change":"2024-02-02T15:24:51.815417+0000","last_active":"2024-02-02T15:24:52.315377+0000","last_peered":"2024-02-02T15:24:52.315377+0000","last_clean":"2024-02-02T15:24:52.315377+0000","last_became_active":"2024-02-02T15:24:51.815021+0000","last_became_peered":"2024-02-02T15:24:51.815021+0000","last_unstale":"2024-02-02T15:24:52.315377+0000","last_undegraded":"2024-02-02T15:24:52.315377+0000","last_fullsized":"2024-02-02T15:24:52.315377+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:22:43.790920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"84'15","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:25:17.230723+0000","last_change":"2024-02-02T15:24:47.274152+0000","last_active":"2024-02-02T15:25:17.230723+0000","last_peered":"2024-02-02T15:25:17.230723+0000","last_clean":"2024-02-02T15:25:17.230723+0000","last_became_active":"2024-02-02T15:24:47.273720+0000","last_became_peered":"2024-02-02T15:24:47.273720+0000","last_unstale":"2024-02-02T15:25:17.230723+0000","last_undegraded":"2024-02-02T15:25:17.230723+0000","last_fullsized":"2024-02-02T15:25:17.230723+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:38:05.280309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.803323+0000","last_change":"2024-02-02T15:24:47.790681+0000","last_active":"2024-02-02T15:24:51.803323+0000","last_peered":"2024-02-02T15:24:51.803323+0000","last_clean":"2024-02-02T15:24:51.803323+0000","last_became_active":"2024-02-02T15:24:47.789952+0000","last_became_peered":"2024-02-02T15:24:47.789952+0000","last_unstale":"2024-02-02T15:24:51.803323+0000","last_undegraded":"2024-02-02T15:24:51.803323+0000","last_fullsized":"2024-02-02T15:24:51.803323+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:21:03.103398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.813237+0000","last_change":"2024-02-02T15:24:51.812676+0000","last_active":"2024-02-02T15:24:51.813237+0000","last_peered":"2024-02-02T15:24:51.813237+0000","last_clean":"2024-02-02T15:24:51.813237+0000","last_became_active":"2024-02-02T15:24:51.812212+0000","last_became_peered":"2024-02-02T15:24:51.812212+0000","last_unstale":"2024-02-02T15:24:51.813237+0000","last_undegraded":"2024-02-02T15:24:51.813237+0000","last_fullsized":"2024-02-02T15:24:51.813237+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:48:01.944379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"79'1","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.802403+0000","last_change":"2024-02-02T15:24:47.271707+0000","last_active":"2024-02-02T15:24:51.802403+0000","last_peered":"2024-02-02T15:24:51.802403+0000","last_clean":"2024-02-02T15:24:51.802403+0000","last_became_active":"2024-02-02T15:24:47.271379+0000","last_became_peered":"2024-02-02T15:24:47.271379+0000","last_unstale":"2024-02-02T15:24:51.802403+0000","last_undegraded":"2024-02-02T15:24:51.802403+0000","last_fullsized":"2024-02-02T15:24:51.802403+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:00:24.571385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.799394+0000","last_change":"2024-02-02T15:24:47.788360+0000","last_active":"2024-02-02T15:24:51.799394+0000","last_peered":"2024-02-02T15:24:51.799394+0000","last_clean":"2024-02-02T15:24:51.799394+0000","last_became_active":"2024-02-02T15:24:47.787945+0000","last_became_peered":"2024-02-02T15:24:47.787945+0000","last_unstale":"2024-02-02T15:24:51.799394+0000","last_undegraded":"2024-02-02T15:24:51.799394+0000","last_fullsized":"2024-02-02T15:24:51.799394+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:37:54.531487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:24:52.316113+0000","last_change":"2024-02-02T15:24:51.821320+0000","last_active":"2024-02-02T15:24:52.316113+0000","last_peered":"2024-02-02T15:24:52.316113+0000","last_clean":"2024-02-02T15:24:52.316113+0000","last_became_active":"2024-02-02T15:24:51.820853+0000","last_became_peered":"2024-02-02T15:24:51.820853+0000","last_unstale":"2024-02-02T15:24:52.316113+0000","last_undegraded":"2024-02-02T15:24:52.316113+0000","last_fullsized":"2024-02-02T15:24:52.316113+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:20:12.304517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"84'1","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:24:53.093698+0000","last_change":"2024-02-02T15:24:47.278291+0000","last_active":"2024-02-02T15:24:53.093698+0000","last_peered":"2024-02-02T15:24:53.093698+0000","last_clean":"2024-02-02T15:24:53.093698+0000","last_became_active":"2024-02-02T15:24:47.276163+0000","last_became_peered":"2024-02-02T15:24:47.276163+0000","last_unstale":"2024-02-02T15:24:53.093698+0000","last_undegraded":"2024-02-02T15:24:53.093698+0000","last_fullsized":"2024-02-02T15:24:53.093698+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:53:50.312627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.811070+0000","last_change":"2024-02-02T15:24:47.791290+0000","last_active":"2024-02-02T15:24:51.811070+0000","last_peered":"2024-02-02T15:24:51.811070+0000","last_clean":"2024-02-02T15:24:51.811070+0000","last_became_active":"2024-02-02T15:24:47.790987+0000","last_became_peered":"2024-02-02T15:24:47.790987+0000","last_unstale":"2024-02-02T15:24:51.811070+0000","last_undegraded":"2024-02-02T15:24:51.811070+0000","last_fullsized":"2024-02-02T15:24:51.811070+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:38:08.803338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:24:52.315363+0000","last_change":"2024-02-02T15:24:51.813050+0000","last_active":"2024-02-02T15:24:52.315363+0000","last_peered":"2024-02-02T15:24:52.315363+0000","last_clean":"2024-02-02T15:24:52.315363+0000","last_became_active":"2024-02-02T15:24:51.812634+0000","last_became_peered":"2024-02-02T15:24:51.812634+0000","last_unstale":"2024-02-02T15:24:52.315363+0000","last_undegraded":"2024-02-02T15:24:52.315363+0000","last_fullsized":"2024-02-02T15:24:52.315363+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:06:56.428431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"84'5","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:25:20.261463+0000","last_change":"2024-02-02T15:24:47.274033+0000","last_active":"2024-02-02T15:25:20.261463+0000","last_peered":"2024-02-02T15:25:20.261463+0000","last_clean":"2024-02-02T15:25:20.261463+0000","last_became_active":"2024-02-02T15:24:47.273672+0000","last_became_peered":"2024-02-02T15:24:47.273672+0000","last_unstale":"2024-02-02T15:25:20.261463+0000","last_undegraded":"2024-02-02T15:25:20.261463+0000","last_fullsized":"2024-02-02T15:25:20.261463+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:51:46.926397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.811312+0000","last_change":"2024-02-02T15:24:47.786533+0000","last_active":"2024-02-02T15:24:51.811312+0000","last_peered":"2024-02-02T15:24:51.811312+0000","last_clean":"2024-02-02T15:24:51.811312+0000","last_became_active":"2024-02-02T15:24:47.785436+0000","last_became_peered":"2024-02-02T15:24:47.785436+0000","last_unstale":"2024-02-02T15:24:51.811312+0000","last_undegraded":"2024-02-02T15:24:51.811312+0000","last_fullsized":"2024-02-02T15:24:51.811312+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:23:45.910800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:24:52.315298+0000","last_change":"2024-02-02T15:24:51.814326+0000","last_active":"2024-02-02T15:24:52.315298+0000","last_peered":"2024-02-02T15:24:52.315298+0000","last_clean":"2024-02-02T15:24:52.315298+0000","last_became_active":"2024-02-02T15:24:51.813909+0000","last_became_peered":"2024-02-02T15:24:51.813909+0000","last_unstale":"2024-02-02T15:24:52.315298+0000","last_undegraded":"2024-02-02T15:24:52.315298+0000","last_fullsized":"2024-02-02T15:24:52.315298+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:16:12.902718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.799212+0000","last_change":"2024-02-02T15:24:47.788308+0000","last_active":"2024-02-02T15:24:51.799212+0000","last_peered":"2024-02-02T15:24:51.799212+0000","last_clean":"2024-02-02T15:24:51.799212+0000","last_became_active":"2024-02-02T15:24:47.787988+0000","last_became_peered":"2024-02-02T15:24:47.787988+0000","last_unstale":"2024-02-02T15:24:51.799212+0000","last_undegraded":"2024-02-02T15:24:51.799212+0000","last_fullsized":"2024-02-02T15:24:51.799212+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:44:38.103758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"84'4","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:25:17.230363+0000","last_change":"2024-02-02T15:24:47.274437+0000","last_active":"2024-02-02T15:25:17.230363+0000","last_peered":"2024-02-02T15:25:17.230363+0000","last_clean":"2024-02-02T15:25:17.230363+0000","last_became_active":"2024-02-02T15:24:47.274010+0000","last_became_peered":"2024-02-02T15:24:47.274010+0000","last_unstale":"2024-02-02T15:25:17.230363+0000","last_undegraded":"2024-02-02T15:25:17.230363+0000","last_fullsized":"2024-02-02T15:25:17.230363+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:10:36.862190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"84'1","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:24:53.094023+0000","last_change":"2024-02-02T15:24:51.825698+0000","last_active":"2024-02-02T15:24:53.094023+0000","last_peered":"2024-02-02T15:24:53.094023+0000","last_clean":"2024-02-02T15:24:53.094023+0000","last_became_active":"2024-02-02T15:24:51.825295+0000","last_became_peered":"2024-02-02T15:24:51.825295+0000","last_unstale":"2024-02-02T15:24:53.094023+0000","last_undegraded":"2024-02-02T15:24:53.094023+0000","last_fullsized":"2024-02-02T15:24:53.094023+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:17:09.838124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:52.001031+0000","last_change":"2024-02-02T15:24:47.781836+0000","last_active":"2024-02-02T15:24:52.001031+0000","last_peered":"2024-02-02T15:24:52.001031+0000","last_clean":"2024-02-02T15:24:52.001031+0000","last_became_active":"2024-02-02T15:24:47.781472+0000","last_became_peered":"2024-02-02T15:24:47.781472+0000","last_unstale":"2024-02-02T15:24:52.001031+0000","last_undegraded":"2024-02-02T15:24:52.001031+0000","last_fullsized":"2024-02-02T15:24:52.001031+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:46:33.714191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"84'6","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:25:18.234742+0000","last_change":"2024-02-02T15:24:47.269087+0000","last_active":"2024-02-02T15:25:18.234742+0000","last_peered":"2024-02-02T15:25:18.234742+0000","last_clean":"2024-02-02T15:25:18.234742+0000","last_became_active":"2024-02-02T15:24:47.268743+0000","last_became_peered":"2024-02-02T15:24:47.268743+0000","last_unstale":"2024-02-02T15:25:18.234742+0000","last_undegraded":"2024-02-02T15:25:18.234742+0000","last_fullsized":"2024-02-02T15:25:18.234742+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:03:04.323079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":334,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:24:53.094546+0000","last_change":"2024-02-02T15:23:55.191735+0000","last_active":"2024-02-02T15:24:53.094546+0000","last_peered":"2024-02-02T15:24:53.094546+0000","last_clean":"2024-02-02T15:24:53.094546+0000","last_became_active":"2024-02-02T15:23:55.191119+0000","last_became_peered":"2024-02-02T15:23:55.191119+0000","last_unstale":"2024-02-02T15:24:53.094546+0000","last_undegraded":"2024-02-02T15:24:53.094546+0000","last_fullsized":"2024-02-02T15:24:53.094546+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:21:22.472631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:21:22.472631+0000","last_clean_scrub_stamp":"2024-02-02T15:21:22.472631+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:24:44.153461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.823480+0000","last_change":"2024-02-02T15:24:51.822842+0000","last_active":"2024-02-02T15:24:51.823480+0000","last_peered":"2024-02-02T15:24:51.823480+0000","last_clean":"2024-02-02T15:24:51.823480+0000","last_became_active":"2024-02-02T15:24:51.822372+0000","last_became_peered":"2024-02-02T15:24:51.822372+0000","last_unstale":"2024-02-02T15:24:51.823480+0000","last_undegraded":"2024-02-02T15:24:51.823480+0000","last_fullsized":"2024-02-02T15:24:51.823480+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:45:29.631422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.793629+0000","last_change":"2024-02-02T15:24:47.786507+0000","last_active":"2024-02-02T15:24:51.793629+0000","last_peered":"2024-02-02T15:24:51.793629+0000","last_clean":"2024-02-02T15:24:51.793629+0000","last_became_active":"2024-02-02T15:24:47.786125+0000","last_became_peered":"2024-02-02T15:24:47.786125+0000","last_unstale":"2024-02-02T15:24:51.793629+0000","last_undegraded":"2024-02-02T15:24:51.793629+0000","last_fullsized":"2024-02-02T15:24:51.793629+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:33:09.014258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"84'4","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:25:17.228222+0000","last_change":"2024-02-02T15:24:47.271661+0000","last_active":"2024-02-02T15:25:17.228222+0000","last_peered":"2024-02-02T15:25:17.228222+0000","last_clean":"2024-02-02T15:25:17.228222+0000","last_became_active":"2024-02-02T15:24:47.271258+0000","last_became_peered":"2024-02-02T15:24:47.271258+0000","last_unstale":"2024-02-02T15:25:17.228222+0000","last_undegraded":"2024-02-02T15:25:17.228222+0000","last_fullsized":"2024-02-02T15:25:17.228222+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:14:38.391361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:24:52.868393+0000","last_change":"2024-02-02T15:24:51.808979+0000","last_active":"2024-02-02T15:24:52.868393+0000","last_peered":"2024-02-02T15:24:52.868393+0000","last_clean":"2024-02-02T15:24:52.868393+0000","last_became_active":"2024-02-02T15:24:51.808557+0000","last_became_peered":"2024-02-02T15:24:51.808557+0000","last_unstale":"2024-02-02T15:24:52.868393+0000","last_undegraded":"2024-02-02T15:24:52.868393+0000","last_fullsized":"2024-02-02T15:24:52.868393+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T02:38:34.697137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"84'1","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:24:52.316127+0000","last_change":"2024-02-02T15:24:47.281396+0000","last_active":"2024-02-02T15:24:52.316127+0000","last_peered":"2024-02-02T15:24:52.316127+0000","last_clean":"2024-02-02T15:24:52.316127+0000","last_became_active":"2024-02-02T15:24:47.281036+0000","last_became_peered":"2024-02-02T15:24:47.281036+0000","last_unstale":"2024-02-02T15:24:52.316127+0000","last_undegraded":"2024-02-02T15:24:52.316127+0000","last_fullsized":"2024-02-02T15:24:52.316127+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:54:47.139151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.794267+0000","last_change":"2024-02-02T15:24:47.791057+0000","last_active":"2024-02-02T15:24:51.794267+0000","last_peered":"2024-02-02T15:24:51.794267+0000","last_clean":"2024-02-02T15:24:51.794267+0000","last_became_active":"2024-02-02T15:24:47.790647+0000","last_became_peered":"2024-02-02T15:24:47.790647+0000","last_unstale":"2024-02-02T15:24:51.794267+0000","last_undegraded":"2024-02-02T15:24:51.794267+0000","last_fullsized":"2024-02-02T15:24:51.794267+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T02:49:42.067700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:52.000949+0000","last_change":"2024-02-02T15:24:51.817792+0000","last_active":"2024-02-02T15:24:52.000949+0000","last_peered":"2024-02-02T15:24:52.000949+0000","last_clean":"2024-02-02T15:24:52.000949+0000","last_became_active":"2024-02-02T15:24:51.815800+0000","last_became_peered":"2024-02-02T15:24:51.815800+0000","last_unstale":"2024-02-02T15:24:52.000949+0000","last_undegraded":"2024-02-02T15:24:52.000949+0000","last_fullsized":"2024-02-02T15:24:52.000949+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:28:10.440251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"84'7","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:25:18.236406+0000","last_change":"2024-02-02T15:24:47.277098+0000","last_active":"2024-02-02T15:25:18.236406+0000","last_peered":"2024-02-02T15:25:18.236406+0000","last_clean":"2024-02-02T15:25:18.236406+0000","last_became_active":"2024-02-02T15:24:47.275867+0000","last_became_peered":"2024-02-02T15:24:47.275867+0000","last_unstale":"2024-02-02T15:25:18.236406+0000","last_undegraded":"2024-02-02T15:25:18.236406+0000","last_fullsized":"2024-02-02T15:25:18.236406+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:35:21.015918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:52.001109+0000","last_change":"2024-02-02T15:24:47.784873+0000","last_active":"2024-02-02T15:24:52.001109+0000","last_peered":"2024-02-02T15:24:52.001109+0000","last_clean":"2024-02-02T15:24:52.001109+0000","last_became_active":"2024-02-02T15:24:47.784550+0000","last_became_peered":"2024-02-02T15:24:47.784550+0000","last_unstale":"2024-02-02T15:24:52.001109+0000","last_undegraded":"2024-02-02T15:24:52.001109+0000","last_fullsized":"2024-02-02T15:24:52.001109+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:13:37.513754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.946878+0000","last_change":"2024-02-02T15:24:51.805131+0000","last_active":"2024-02-02T15:24:51.946878+0000","last_peered":"2024-02-02T15:24:51.946878+0000","last_clean":"2024-02-02T15:24:51.946878+0000","last_became_active":"2024-02-02T15:24:51.804705+0000","last_became_peered":"2024-02-02T15:24:51.804705+0000","last_unstale":"2024-02-02T15:24:51.946878+0000","last_undegraded":"2024-02-02T15:24:51.946878+0000","last_fullsized":"2024-02-02T15:24:51.946878+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T02:12:59.945050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"84'7","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:25:19.240901+0000","last_change":"2024-02-02T15:24:47.266469+0000","last_active":"2024-02-02T15:25:19.240901+0000","last_peered":"2024-02-02T15:25:19.240901+0000","last_clean":"2024-02-02T15:25:19.240901+0000","last_became_active":"2024-02-02T15:24:47.266094+0000","last_became_peered":"2024-02-02T15:24:47.266094+0000","last_unstale":"2024-02-02T15:25:19.240901+0000","last_undegraded":"2024-02-02T15:25:19.240901+0000","last_fullsized":"2024-02-02T15:25:19.240901+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:51:41.324698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.808312+0000","last_change":"2024-02-02T15:24:47.793051+0000","last_active":"2024-02-02T15:24:51.808312+0000","last_peered":"2024-02-02T15:24:51.808312+0000","last_clean":"2024-02-02T15:24:51.808312+0000","last_became_active":"2024-02-02T15:24:47.792645+0000","last_became_peered":"2024-02-02T15:24:47.792645+0000","last_unstale":"2024-02-02T15:24:51.808312+0000","last_undegraded":"2024-02-02T15:24:51.808312+0000","last_fullsized":"2024-02-02T15:24:51.808312+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:47:17.229407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:24:53.094426+0000","last_change":"2024-02-02T15:24:51.821048+0000","last_active":"2024-02-02T15:24:53.094426+0000","last_peered":"2024-02-02T15:24:53.094426+0000","last_clean":"2024-02-02T15:24:53.094426+0000","last_became_active":"2024-02-02T15:24:51.820661+0000","last_became_peered":"2024-02-02T15:24:51.820661+0000","last_unstale":"2024-02-02T15:24:53.094426+0000","last_undegraded":"2024-02-02T15:24:53.094426+0000","last_fullsized":"2024-02-02T15:24:53.094426+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:45:52.401807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"84'2","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:25:16.219044+0000","last_change":"2024-02-02T15:24:47.273716+0000","last_active":"2024-02-02T15:25:16.219044+0000","last_peered":"2024-02-02T15:25:16.219044+0000","last_clean":"2024-02-02T15:25:16.219044+0000","last_became_active":"2024-02-02T15:24:47.273438+0000","last_became_peered":"2024-02-02T15:24:47.273438+0000","last_unstale":"2024-02-02T15:25:16.219044+0000","last_undegraded":"2024-02-02T15:25:16.219044+0000","last_fullsized":"2024-02-02T15:25:16.219044+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:52:16.498119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.798696+0000","last_change":"2024-02-02T15:24:47.782950+0000","last_active":"2024-02-02T15:24:51.798696+0000","last_peered":"2024-02-02T15:24:51.798696+0000","last_clean":"2024-02-02T15:24:51.798696+0000","last_became_active":"2024-02-02T15:24:47.782605+0000","last_became_peered":"2024-02-02T15:24:47.782605+0000","last_unstale":"2024-02-02T15:24:51.798696+0000","last_undegraded":"2024-02-02T15:24:51.798696+0000","last_fullsized":"2024-02-02T15:24:51.798696+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:57:34.125985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:24:53.094614+0000","last_change":"2024-02-02T15:24:51.822596+0000","last_active":"2024-02-02T15:24:53.094614+0000","last_peered":"2024-02-02T15:24:53.094614+0000","last_clean":"2024-02-02T15:24:53.094614+0000","last_became_active":"2024-02-02T15:24:51.822138+0000","last_became_peered":"2024-02-02T15:24:51.822138+0000","last_unstale":"2024-02-02T15:24:53.094614+0000","last_undegraded":"2024-02-02T15:24:53.094614+0000","last_fullsized":"2024-02-02T15:24:53.094614+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:03:11.339557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"84'3","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:25:16.217495+0000","last_change":"2024-02-02T15:24:47.278478+0000","last_active":"2024-02-02T15:25:16.217495+0000","last_peered":"2024-02-02T15:25:16.217495+0000","last_clean":"2024-02-02T15:25:16.217495+0000","last_became_active":"2024-02-02T15:24:47.278084+0000","last_became_peered":"2024-02-02T15:24:47.278084+0000","last_unstale":"2024-02-02T15:25:16.217495+0000","last_undegraded":"2024-02-02T15:25:16.217495+0000","last_fullsized":"2024-02-02T15:25:16.217495+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:10:23.044968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.798379+0000","last_change":"2024-02-02T15:24:47.795291+0000","last_active":"2024-02-02T15:24:51.798379+0000","last_peered":"2024-02-02T15:24:51.798379+0000","last_clean":"2024-02-02T15:24:51.798379+0000","last_became_active":"2024-02-02T15:24:47.794514+0000","last_became_peered":"2024-02-02T15:24:47.794514+0000","last_unstale":"2024-02-02T15:24:51.798379+0000","last_undegraded":"2024-02-02T15:24:51.798379+0000","last_fullsized":"2024-02-02T15:24:51.798379+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:32:12.722258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:24:53.093493+0000","last_change":"2024-02-02T15:24:51.825985+0000","last_active":"2024-02-02T15:24:53.093493+0000","last_peered":"2024-02-02T15:24:53.093493+0000","last_clean":"2024-02-02T15:24:53.093493+0000","last_became_active":"2024-02-02T15:24:51.825588+0000","last_became_peered":"2024-02-02T15:24:51.825588+0000","last_unstale":"2024-02-02T15:24:53.093493+0000","last_undegraded":"2024-02-02T15:24:53.093493+0000","last_fullsized":"2024-02-02T15:24:53.093493+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:32:39.910258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.794667+0000","last_change":"2024-02-02T15:24:47.272954+0000","last_active":"2024-02-02T15:24:51.794667+0000","last_peered":"2024-02-02T15:24:51.794667+0000","last_clean":"2024-02-02T15:24:51.794667+0000","last_became_active":"2024-02-02T15:24:47.272090+0000","last_became_peered":"2024-02-02T15:24:47.272090+0000","last_unstale":"2024-02-02T15:24:51.794667+0000","last_undegraded":"2024-02-02T15:24:51.794667+0000","last_fullsized":"2024-02-02T15:24:51.794667+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:01:53.602026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:52.001055+0000","last_change":"2024-02-02T15:24:47.784188+0000","last_active":"2024-02-02T15:24:52.001055+0000","last_peered":"2024-02-02T15:24:52.001055+0000","last_clean":"2024-02-02T15:24:52.001055+0000","last_became_active":"2024-02-02T15:24:47.783872+0000","last_became_peered":"2024-02-02T15:24:47.783872+0000","last_unstale":"2024-02-02T15:24:52.001055+0000","last_undegraded":"2024-02-02T15:24:52.001055+0000","last_fullsized":"2024-02-02T15:24:52.001055+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:45:45.213255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:24:53.093842+0000","last_change":"2024-02-02T15:24:51.815263+0000","last_active":"2024-02-02T15:24:53.093842+0000","last_peered":"2024-02-02T15:24:53.093842+0000","last_clean":"2024-02-02T15:24:53.093842+0000","last_became_active":"2024-02-02T15:24:51.814849+0000","last_became_peered":"2024-02-02T15:24:51.814849+0000","last_unstale":"2024-02-02T15:24:53.093842+0000","last_undegraded":"2024-02-02T15:24:53.093842+0000","last_fullsized":"2024-02-02T15:24:53.093842+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:05:32.987359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"84'4","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:25:16.220014+0000","last_change":"2024-02-02T15:24:47.270534+0000","last_active":"2024-02-02T15:25:16.220014+0000","last_peered":"2024-02-02T15:25:16.220014+0000","last_clean":"2024-02-02T15:25:16.220014+0000","last_became_active":"2024-02-02T15:24:47.270110+0000","last_became_peered":"2024-02-02T15:24:47.270110+0000","last_unstale":"2024-02-02T15:25:16.220014+0000","last_undegraded":"2024-02-02T15:25:16.220014+0000","last_fullsized":"2024-02-02T15:25:16.220014+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T02:59:52.297315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.801786+0000","last_change":"2024-02-02T15:24:47.791826+0000","last_active":"2024-02-02T15:24:51.801786+0000","last_peered":"2024-02-02T15:24:51.801786+0000","last_clean":"2024-02-02T15:24:51.801786+0000","last_became_active":"2024-02-02T15:24:47.791016+0000","last_became_peered":"2024-02-02T15:24:47.791016+0000","last_unstale":"2024-02-02T15:24:51.801786+0000","last_undegraded":"2024-02-02T15:24:51.801786+0000","last_fullsized":"2024-02-02T15:24:51.801786+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:47:14.296876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:24:53.093305+0000","last_change":"2024-02-02T15:24:51.806741+0000","last_active":"2024-02-02T15:24:53.093305+0000","last_peered":"2024-02-02T15:24:53.093305+0000","last_clean":"2024-02-02T15:24:53.093305+0000","last_became_active":"2024-02-02T15:24:51.806085+0000","last_became_peered":"2024-02-02T15:24:51.806085+0000","last_unstale":"2024-02-02T15:24:53.093305+0000","last_undegraded":"2024-02-02T15:24:53.093305+0000","last_fullsized":"2024-02-02T15:24:53.093305+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:53:39.944222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.795735+0000","last_change":"2024-02-02T15:24:47.278515+0000","last_active":"2024-02-02T15:24:51.795735+0000","last_peered":"2024-02-02T15:24:51.795735+0000","last_clean":"2024-02-02T15:24:51.795735+0000","last_became_active":"2024-02-02T15:24:47.278145+0000","last_became_peered":"2024-02-02T15:24:47.278145+0000","last_unstale":"2024-02-02T15:24:51.795735+0000","last_undegraded":"2024-02-02T15:24:51.795735+0000","last_fullsized":"2024-02-02T15:24:51.795735+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:48:55.118003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.800565+0000","last_change":"2024-02-02T15:24:47.791996+0000","last_active":"2024-02-02T15:24:51.800565+0000","last_peered":"2024-02-02T15:24:51.800565+0000","last_clean":"2024-02-02T15:24:51.800565+0000","last_became_active":"2024-02-02T15:24:47.791624+0000","last_became_peered":"2024-02-02T15:24:47.791624+0000","last_unstale":"2024-02-02T15:24:51.800565+0000","last_undegraded":"2024-02-02T15:24:51.800565+0000","last_fullsized":"2024-02-02T15:24:51.800565+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:33:27.963740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.827314+0000","last_change":"2024-02-02T15:24:51.826835+0000","last_active":"2024-02-02T15:24:51.827314+0000","last_peered":"2024-02-02T15:24:51.827314+0000","last_clean":"2024-02-02T15:24:51.827314+0000","last_became_active":"2024-02-02T15:24:51.826310+0000","last_became_peered":"2024-02-02T15:24:51.826310+0000","last_unstale":"2024-02-02T15:24:51.827314+0000","last_undegraded":"2024-02-02T15:24:51.827314+0000","last_fullsized":"2024-02-02T15:24:51.827314+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:48:16.687048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'3","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:25:19.241138+0000","last_change":"2024-02-02T15:24:47.268865+0000","last_active":"2024-02-02T15:25:19.241138+0000","last_peered":"2024-02-02T15:25:19.241138+0000","last_clean":"2024-02-02T15:25:19.241138+0000","last_became_active":"2024-02-02T15:24:47.268523+0000","last_became_peered":"2024-02-02T15:24:47.268523+0000","last_unstale":"2024-02-02T15:25:19.241138+0000","last_undegraded":"2024-02-02T15:25:19.241138+0000","last_fullsized":"2024-02-02T15:25:19.241138+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:45:16.957512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.799303+0000","last_change":"2024-02-02T15:24:47.792601+0000","last_active":"2024-02-02T15:24:51.799303+0000","last_peered":"2024-02-02T15:24:51.799303+0000","last_clean":"2024-02-02T15:24:51.799303+0000","last_became_active":"2024-02-02T15:24:47.792240+0000","last_became_peered":"2024-02-02T15:24:47.792240+0000","last_unstale":"2024-02-02T15:24:51.799303+0000","last_undegraded":"2024-02-02T15:24:51.799303+0000","last_fullsized":"2024-02-02T15:24:51.799303+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:51:25.205859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.806905+0000","last_change":"2024-02-02T15:24:51.806267+0000","last_active":"2024-02-02T15:24:51.806905+0000","last_peered":"2024-02-02T15:24:51.806905+0000","last_clean":"2024-02-02T15:24:51.806905+0000","last_became_active":"2024-02-02T15:24:51.805877+0000","last_became_peered":"2024-02-02T15:24:51.805877+0000","last_unstale":"2024-02-02T15:24:51.806905+0000","last_undegraded":"2024-02-02T15:24:51.806905+0000","last_fullsized":"2024-02-02T15:24:51.806905+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:00:47.747377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.794156+0000","last_change":"2024-02-02T15:24:47.271057+0000","last_active":"2024-02-02T15:24:51.794156+0000","last_peered":"2024-02-02T15:24:51.794156+0000","last_clean":"2024-02-02T15:24:51.794156+0000","last_became_active":"2024-02-02T15:24:47.270737+0000","last_became_peered":"2024-02-02T15:24:47.270737+0000","last_unstale":"2024-02-02T15:24:51.794156+0000","last_undegraded":"2024-02-02T15:24:51.794156+0000","last_fullsized":"2024-02-02T15:24:51.794156+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:17:00.212426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.799556+0000","last_change":"2024-02-02T15:24:47.794108+0000","last_active":"2024-02-02T15:24:51.799556+0000","last_peered":"2024-02-02T15:24:51.799556+0000","last_clean":"2024-02-02T15:24:51.799556+0000","last_became_active":"2024-02-02T15:24:47.793890+0000","last_became_peered":"2024-02-02T15:24:47.793890+0000","last_unstale":"2024-02-02T15:24:51.799556+0000","last_undegraded":"2024-02-02T15:24:51.799556+0000","last_fullsized":"2024-02-02T15:24:51.799556+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T03:05:51.704385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.798477+0000","last_change":"2024-02-02T15:24:47.796234+0000","last_active":"2024-02-02T15:24:51.798477+0000","last_peered":"2024-02-02T15:24:51.798477+0000","last_clean":"2024-02-02T15:24:51.798477+0000","last_became_active":"2024-02-02T15:24:47.795474+0000","last_became_peered":"2024-02-02T15:24:47.795474+0000","last_unstale":"2024-02-02T15:24:51.798477+0000","last_undegraded":"2024-02-02T15:24:51.798477+0000","last_fullsized":"2024-02-02T15:24:51.798477+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:25:28.059500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.946901+0000","last_change":"2024-02-02T15:24:47.786930+0000","last_active":"2024-02-02T15:24:51.946901+0000","last_peered":"2024-02-02T15:24:51.946901+0000","last_clean":"2024-02-02T15:24:51.946901+0000","last_became_active":"2024-02-02T15:24:47.786615+0000","last_became_peered":"2024-02-02T15:24:47.786615+0000","last_unstale":"2024-02-02T15:24:51.946901+0000","last_undegraded":"2024-02-02T15:24:51.946901+0000","last_fullsized":"2024-02-02T15:24:51.946901+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:17:23.231511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.946924+0000","last_change":"2024-02-02T15:24:47.797877+0000","last_active":"2024-02-02T15:24:51.946924+0000","last_peered":"2024-02-02T15:24:51.946924+0000","last_clean":"2024-02-02T15:24:51.946924+0000","last_became_active":"2024-02-02T15:24:47.797559+0000","last_became_peered":"2024-02-02T15:24:47.797559+0000","last_unstale":"2024-02-02T15:24:51.946924+0000","last_undegraded":"2024-02-02T15:24:51.946924+0000","last_fullsized":"2024-02-02T15:24:51.946924+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:13:55.529796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.804307+0000","last_change":"2024-02-02T15:24:47.790270+0000","last_active":"2024-02-02T15:24:51.804307+0000","last_peered":"2024-02-02T15:24:51.804307+0000","last_clean":"2024-02-02T15:24:51.804307+0000","last_became_active":"2024-02-02T15:24:47.789928+0000","last_became_peered":"2024-02-02T15:24:47.789928+0000","last_unstale":"2024-02-02T15:24:51.804307+0000","last_undegraded":"2024-02-02T15:24:51.804307+0000","last_fullsized":"2024-02-02T15:24:51.804307+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T02:34:44.885803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.794371+0000","last_change":"2024-02-02T15:24:47.786586+0000","last_active":"2024-02-02T15:24:51.794371+0000","last_peered":"2024-02-02T15:24:51.794371+0000","last_clean":"2024-02-02T15:24:51.794371+0000","last_became_active":"2024-02-02T15:24:47.785555+0000","last_became_peered":"2024-02-02T15:24:47.785555+0000","last_unstale":"2024-02-02T15:24:51.794371+0000","last_undegraded":"2024-02-02T15:24:51.794371+0000","last_fullsized":"2024-02-02T15:24:51.794371+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:37:47.537815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.799602+0000","last_change":"2024-02-02T15:24:47.797861+0000","last_active":"2024-02-02T15:24:51.799602+0000","last_peered":"2024-02-02T15:24:51.799602+0000","last_clean":"2024-02-02T15:24:51.799602+0000","last_became_active":"2024-02-02T15:24:47.797543+0000","last_became_peered":"2024-02-02T15:24:47.797543+0000","last_unstale":"2024-02-02T15:24:51.799602+0000","last_undegraded":"2024-02-02T15:24:51.799602+0000","last_fullsized":"2024-02-02T15:24:51.799602+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:09:37.710909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.947003+0000","last_change":"2024-02-02T15:24:47.796221+0000","last_active":"2024-02-02T15:24:51.947003+0000","last_peered":"2024-02-02T15:24:51.947003+0000","last_clean":"2024-02-02T15:24:51.947003+0000","last_became_active":"2024-02-02T15:24:47.795845+0000","last_became_peered":"2024-02-02T15:24:47.795845+0000","last_unstale":"2024-02-02T15:24:51.947003+0000","last_undegraded":"2024-02-02T15:24:51.947003+0000","last_fullsized":"2024-02-02T15:24:51.947003+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:54:55.637660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:52.001128+0000","last_change":"2024-02-02T15:24:47.783086+0000","last_active":"2024-02-02T15:24:52.001128+0000","last_peered":"2024-02-02T15:24:52.001128+0000","last_clean":"2024-02-02T15:24:52.001128+0000","last_became_active":"2024-02-02T15:24:47.781826+0000","last_became_peered":"2024-02-02T15:24:47.781826+0000","last_unstale":"2024-02-02T15:24:52.001128+0000","last_undegraded":"2024-02-02T15:24:52.001128+0000","last_fullsized":"2024-02-02T15:24:52.001128+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:32:05.623591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.799100+0000","last_change":"2024-02-02T15:24:47.789151+0000","last_active":"2024-02-02T15:24:51.799100+0000","last_peered":"2024-02-02T15:24:51.799100+0000","last_clean":"2024-02-02T15:24:51.799100+0000","last_became_active":"2024-02-02T15:24:47.785809+0000","last_became_peered":"2024-02-02T15:24:47.785809+0000","last_unstale":"2024-02-02T15:24:51.799100+0000","last_undegraded":"2024-02-02T15:24:51.799100+0000","last_fullsized":"2024-02-02T15:24:51.799100+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:18:16.013352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:52.001013+0000","last_change":"2024-02-02T15:24:47.789905+0000","last_active":"2024-02-02T15:24:52.001013+0000","last_peered":"2024-02-02T15:24:52.001013+0000","last_clean":"2024-02-02T15:24:52.001013+0000","last_became_active":"2024-02-02T15:24:47.788903+0000","last_became_peered":"2024-02-02T15:24:47.788903+0000","last_unstale":"2024-02-02T15:24:52.001013+0000","last_undegraded":"2024-02-02T15:24:52.001013+0000","last_fullsized":"2024-02-02T15:24:52.001013+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T02:49:46.118232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.795514+0000","last_change":"2024-02-02T15:24:47.795928+0000","last_active":"2024-02-02T15:24:51.795514+0000","last_peered":"2024-02-02T15:24:51.795514+0000","last_clean":"2024-02-02T15:24:51.795514+0000","last_became_active":"2024-02-02T15:24:47.795511+0000","last_became_peered":"2024-02-02T15:24:47.795511+0000","last_unstale":"2024-02-02T15:24:51.795514+0000","last_undegraded":"2024-02-02T15:24:51.795514+0000","last_fullsized":"2024-02-02T15:24:51.795514+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:55:31.444331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.794050+0000","last_change":"2024-02-02T15:24:47.794368+0000","last_active":"2024-02-02T15:24:51.794050+0000","last_peered":"2024-02-02T15:24:51.794050+0000","last_clean":"2024-02-02T15:24:51.794050+0000","last_became_active":"2024-02-02T15:24:47.793808+0000","last_became_peered":"2024-02-02T15:24:47.793808+0000","last_unstale":"2024-02-02T15:24:51.794050+0000","last_undegraded":"2024-02-02T15:24:51.794050+0000","last_fullsized":"2024-02-02T15:24:51.794050+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T02:39:00.099808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.799168+0000","last_change":"2024-02-02T15:24:47.796358+0000","last_active":"2024-02-02T15:24:51.799168+0000","last_peered":"2024-02-02T15:24:51.799168+0000","last_clean":"2024-02-02T15:24:51.799168+0000","last_became_active":"2024-02-02T15:24:47.795800+0000","last_became_peered":"2024-02-02T15:24:47.795800+0000","last_unstale":"2024-02-02T15:24:51.799168+0000","last_undegraded":"2024-02-02T15:24:51.799168+0000","last_fullsized":"2024-02-02T15:24:51.799168+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:49:40.019787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":16384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":250199,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":43200,"num_objects":74,"num_object_clones":0,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":0,"num_read":16,"num_read_kb":45,"num_write":173,"num_write_kb":92,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":49,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":348160,"data_stored":61788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152,"ondisk_log_size":152,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547217,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":925696,"data_stored":459202,"data_compressed":11732,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 15:25:12 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.63900000000000001}]},{"osd":1,"last update":"Fri Feb 2 15:25:12 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.59999999999999998}]},{"osd":2,"last update":"Fri Feb 2 15:25:12 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.47899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":4,"last update":"Fri Feb 2 15:25:12 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.69799999999999995}]},{"osd":5,"last update":"Fri Feb 2 15:25:12 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.625}]},{"osd":6,"last update":"Fri Feb 2 15:25:12 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.80900000000000005}]},{"osd":7,"last update":"Fri Feb 2 15:25:12 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.75},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.76200000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]},{"osd":9,"last update":"Fri Feb 2 15:25:12 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.78700000000000003}]},{"osd":10,"last update":"Fri Feb 2 15:25:12 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.77400000000000002}]}]},{"osd":10,"up_from":67,"seq":287762808851,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1052672,"data_stored":1084928,"data_compressed":47943,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":1,"last update":"Fri Feb 2 15:24:51 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.443,"15min":0.443},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.435}]},{"osd":2,"last update":"Fri Feb 2 15:24:51 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.51100000000000001}]},{"osd":3,"last update":"Fri Feb 2 15:24:51 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":4.4109999999999996,"5min":4.4109999999999996,"15min":4.4109999999999996},"last":0.55000000000000004}]},{"osd":4,"last update":"Fri Feb 2 15:24:51 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.57699999999999996}]},{"osd":5,"last update":"Fri Feb 2 15:24:51 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.69299999999999995}]},{"osd":6,"last update":"Fri Feb 2 15:24:51 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":4.4459999999999997,"5min":4.4459999999999997,"15min":4.4459999999999997},"last":0.746},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.68400000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40300000000000002}]},{"osd":8,"last update":"Fri Feb 2 15:24:51 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.70699999999999996}]},{"osd":9,"last update":"Fri Feb 2 15:24:51 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":4.4269999999999996,"5min":4.4269999999999996,"15min":4.4269999999999996},"last":0.73399999999999999}]},{"osd":11,"last update":"Fri Feb 2 15:25:11 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.5680000000000001,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.76700000000000002}]}]},{"osd":9,"up_from":62,"seq":266287972375,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":933888,"data_stored":454870,"data_compressed":11732,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 15:24:35 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.50900000000000001}]},{"osd":1,"last update":"Fri Feb 2 15:24:35 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.752},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":0.64700000000000002}]},{"osd":2,"last update":"Fri Feb 2 15:24:35 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.71399999999999997}]},{"osd":3,"last update":"Fri Feb 2 15:24:35 2024","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.296,"5min":1.296,"15min":1.296},"last":0.56499999999999995}]},{"osd":4,"last update":"Fri Feb 2 15:24:35 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.381,"5min":1.381,"15min":1.381},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.70599999999999996}]},{"osd":5,"last update":"Fri Feb 2 15:24:35 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":3.895,"5min":3.895,"15min":3.895},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.373,"5min":1.373,"15min":1.373},"last":0.78900000000000003}]},{"osd":6,"last update":"Fri Feb 2 15:24:35 2024","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.6739999999999999,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":0.64500000000000002}]},{"osd":7,"last update":"Fri Feb 2 15:24:35 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.77600000000000002}]},{"osd":8,"last update":"Fri Feb 2 15:24:35 2024","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.7789999999999999,"5min":1.7789999999999999,"15min":1.7789999999999999},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":0.84799999999999998}]},{"osd":10,"last update":"Fri Feb 2 15:24:56 2024","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.87},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.83999999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86299999999999999}]}]},{"osd":8,"up_from":55,"seq":236223201307,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":925696,"data_stored":454369,"data_compressed":11732,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 15:25:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.58199999999999996,"15min":0.58299999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.73699999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.59399999999999997,"15min":0.60099999999999998},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.77000000000000002,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.55700000000000005}]},{"osd":1,"last update":"Fri Feb 2 15:25:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.56899999999999995,"15min":0.56499999999999995},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.57399999999999995,"15min":0.56399999999999995},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.79500000000000004}]},{"osd":2,"last update":"Fri Feb 2 15:25:16 2024","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.42099999999999999,"15min":0.41699999999999998},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.63400000000000001,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.627},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.42299999999999999,"15min":0.41699999999999998},"min":{"1min":0.307,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.61299999999999999,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.72299999999999998}]},{"osd":3,"last update":"Fri Feb 2 15:25:16 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.621,"15min":0.627},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.76400000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.61699999999999999,"15min":0.621},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.79500000000000004,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.875}]},{"osd":4,"last update":"Fri Feb 2 15:25:16 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.60299999999999998,"15min":0.59899999999999998},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.62,"5min":0.59499999999999997,"15min":0.59099999999999997},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.93300000000000005}]},{"osd":5,"last update":"Fri Feb 2 15:25:16 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.51000000000000001,"15min":0.503},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.50800000000000001,"15min":0.502},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.84799999999999998}]},{"osd":6,"last update":"Fri Feb 2 15:25:16 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.67200000000000004,"15min":0.67500000000000004},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.83299999999999996,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.69199999999999995,"15min":0.69299999999999995},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.80900000000000005,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.86799999999999999}]},{"osd":7,"last update":"Fri Feb 2 15:25:16 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.64700000000000002,"15min":0.64100000000000001},"min":{"1min":0.49199999999999999,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.67400000000000004,"15min":0.67000000000000004},"min":{"1min":0.45800000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.88700000000000001,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.89600000000000002}]},{"osd":9,"last update":"Fri Feb 2 15:24:36 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.996}]},{"osd":10,"last update":"Fri Feb 2 15:24:53 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":1.0169999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98799999999999999}]}]},{"osd":7,"up_from":48,"seq":206158430239,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":1051816,"data_compressed":47943,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 15:23:58 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.439},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.38500000000000001}]},{"osd":1,"last update":"Fri Feb 2 15:24:59 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.42799999999999999,"15min":0.42599999999999999},"min":{"1min":0.28100000000000003,"5min":0.247,"15min":0.247},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.442},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.40899999999999997,"15min":0.40699999999999997},"min":{"1min":0.29399999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.64500000000000002,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.46400000000000002}]},{"osd":2,"last update":"Fri Feb 2 15:24:59 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.79100000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.67600000000000005}]},{"osd":3,"last update":"Fri Feb 2 15:24:59 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.60899999999999999,"15min":0.59999999999999998},"min":{"1min":0.41099999999999998,"5min":0.374,"15min":0.374},"max":{"1min":2.746,"5min":2.746,"15min":2.746},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.65500000000000003}]},{"osd":4,"last update":"Fri Feb 2 15:24:59 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.502,"15min":0.5},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.5,"15min":0.496},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.49199999999999999}]},{"osd":5,"last update":"Fri Feb 2 15:24:59 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.63,"15min":0.61899999999999999},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":2.3370000000000002,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.63,"5min":0.623,"15min":0.622},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.83999999999999997}]},{"osd":6,"last update":"Fri Feb 2 15:24:59 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.68999999999999995,"15min":0.69199999999999995},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.67700000000000005,"15min":0.66800000000000004},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.778,"5min":1.778,"15min":1.778},"last":0.61299999999999999}]},{"osd":8,"last update":"Fri Feb 2 15:25:16 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.70499999999999996,"15min":0.70099999999999996},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":2.069,"5min":2.069,"15min":2.069},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.72899999999999998,"15min":0.73299999999999998},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.995,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.79500000000000004}]},{"osd":9,"last update":"Fri Feb 2 15:24:37 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.70699999999999996}]},{"osd":10,"last update":"Fri Feb 2 15:24:51 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.64400000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81599999999999995}]}]},{"osd":6,"up_from":43,"seq":184683593762,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":913408,"data_stored":454052,"data_compressed":11732,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 15:24:43 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.48899999999999999,"15min":0.48199999999999998},"min":{"1min":0.33800000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.51800000000000002,"15min":0.51200000000000001},"min":{"1min":0.39200000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.371}]},{"osd":1,"last update":"Fri Feb 2 15:24:43 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.63900000000000001,"15min":0.63600000000000001},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.65300000000000002,"15min":0.64900000000000002},"min":{"1min":0.54600000000000004,"5min":0.501,"15min":0.501},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.88900000000000001}]},{"osd":2,"last update":"Fri Feb 2 15:24:43 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.69999999999999996,"15min":0.69399999999999995},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.97599999999999998,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.71499999999999997,"15min":0.71199999999999997},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.0409999999999999,"5min":1.151,"15min":1.151},"last":0.80200000000000005}]},{"osd":3,"last update":"Fri Feb 2 15:24:43 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.63900000000000001,"15min":0.63700000000000001},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.432},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.66900000000000004,"15min":0.66300000000000003},"min":{"1min":0.41799999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.99099999999999999}]},{"osd":4,"last update":"Fri Feb 2 15:24:43 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.71899999999999997,"15min":0.71699999999999997},"min":{"1min":0.50600000000000001,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.046,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.76000000000000001,"15min":0.755},"min":{"1min":0.53700000000000003,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.2250000000000001,"5min":1.228,"15min":1.228},"last":0.64600000000000002}]},{"osd":5,"last update":"Fri Feb 2 15:24:43 2024","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.78100000000000003,"15min":0.77600000000000002},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.81799999999999995,"15min":0.81399999999999995},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.202,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.85499999999999998}]},{"osd":7,"last update":"Fri Feb 2 15:24:57 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.83199999999999996,"15min":0.83099999999999996},"min":{"1min":0.59199999999999997,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.073,"5min":1.256,"15min":1.256},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.876,"15min":0.878},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.103,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.747}]},{"osd":8,"last update":"Fri Feb 2 15:25:14 2024","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.90000000000000002,"15min":0.90100000000000002},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.212,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.91700000000000004,"15min":0.91300000000000003},"min":{"1min":0.57699999999999996,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.78100000000000003}]},{"osd":9,"last update":"Fri Feb 2 15:24:37 2024","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.405,"5min":1.405,"15min":1.405},"last":1.018},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.874}]},{"osd":10,"last update":"Fri Feb 2 15:24:53 2024","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.93700000000000006}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97699999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574900,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":921600,"data_stored":457757,"data_compressed":11732,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 15:25:12 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.51100000000000001,"15min":0.50800000000000001},"min":{"1min":0.39200000000000002,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.52500000000000002,"15min":0.52100000000000002},"min":{"1min":0.40100000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.68999999999999995}]},{"osd":2,"last update":"Fri Feb 2 15:24:30 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.53400000000000003,"15min":0.54000000000000004},"min":{"1min":0.34399999999999997,"5min":0.31,"15min":0.31},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.53900000000000003,"15min":0.53800000000000003},"min":{"1min":0.379,"5min":0.315,"15min":0.315},"max":{"1min":0.79700000000000004,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.65300000000000002}]},{"osd":3,"last update":"Fri Feb 2 15:24:50 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.54100000000000004,"15min":0.52700000000000002},"min":{"1min":0.39800000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.747,"5min":0.748,"15min":0.748},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.55000000000000004,"15min":0.54400000000000004},"min":{"1min":0.48599999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.80100000000000005}]},{"osd":4,"last update":"Fri Feb 2 15:25:04 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.39600000000000002,"15min":0.36899999999999999},"min":{"1min":0.32900000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.66800000000000004,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.40000000000000002,"15min":0.375},"min":{"1min":0.27200000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.623}]},{"osd":5,"last update":"Fri Feb 2 15:24:19 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.53600000000000003,"15min":0.53300000000000003},"min":{"1min":0.40699999999999997,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.55500000000000005,"15min":0.54600000000000004},"min":{"1min":0.41399999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.79000000000000004}]},{"osd":6,"last update":"Fri Feb 2 15:24:44 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.59399999999999997,"15min":0.58599999999999997},"min":{"1min":0.49099999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.752,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.751},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.58499999999999996,"15min":0.58099999999999996},"min":{"1min":0.503,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.70499999999999996}]},{"osd":7,"last update":"Fri Feb 2 15:25:00 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.81100000000000005,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.53700000000000003,"15min":0.53300000000000003},"min":{"1min":0.371,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.72699999999999998,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.77900000000000003}]},{"osd":8,"last update":"Fri Feb 2 15:25:14 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.63400000000000001,"15min":0.63},"min":{"1min":0.434,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.65700000000000003,"15min":0.64700000000000002},"min":{"1min":0.51100000000000001,"5min":0.442,"15min":0.442},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.87}]},{"osd":9,"last update":"Fri Feb 2 15:24:36 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.88},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.90600000000000003}]},{"osd":10,"last update":"Fri Feb 2 15:24:51 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.85599999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93000000000000005}]}]},{"osd":0,"up_from":8,"seq":34359738424,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":456092,"data_compressed":11732,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Feb 2 15:25:10 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.55800000000000005,"15min":0.54400000000000004},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.90400000000000003,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.628,"5min":0.57399999999999995,"15min":0.54500000000000004},"min":{"1min":0.35799999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.89800000000000002}]},{"osd":2,"last update":"Fri Feb 2 15:24:29 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.60699999999999998,"15min":0.56699999999999995},"min":{"1min":0.48999999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.872,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.67500000000000004,"15min":0.60499999999999998},"min":{"1min":0.45700000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":4.6550000000000002,"5min":4.6550000000000002,"15min":4.6550000000000002},"last":0.51100000000000001}]},{"osd":3,"last update":"Fri Feb 2 15:23:45 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.55100000000000005,"15min":0.53700000000000003},"min":{"1min":0.33800000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.57599999999999996,"15min":0.56599999999999995},"min":{"1min":0.46300000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.39500000000000002}]},{"osd":4,"last update":"Fri Feb 2 15:25:05 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.65400000000000003,"15min":0.63700000000000001},"min":{"1min":0.433,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.091,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":1.02},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.64200000000000002,"15min":0.626},"min":{"1min":0.46700000000000003,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.1279999999999999,"5min":1.141,"15min":1.141},"last":0.65700000000000003}]},{"osd":5,"last update":"Fri Feb 2 15:25:21 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68200000000000005,"15min":0.68300000000000005},"min":{"1min":0.50700000000000001,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.024,"5min":1.042,"15min":1.042},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.70999999999999996,"15min":0.67600000000000005},"min":{"1min":0.42099999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.054,"5min":4.5890000000000004,"15min":4.5890000000000004},"last":0.58099999999999996}]},{"osd":6,"last update":"Fri Feb 2 15:24:43 2024","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.70999999999999996,"15min":0.69899999999999995},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.73499999999999999,"15min":0.72899999999999998},"min":{"1min":0.46600000000000003,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.72799999999999998}]},{"osd":7,"last update":"Fri Feb 2 15:23:55 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.86099999999999999}]},{"osd":8,"last update":"Fri Feb 2 15:25:21 2024","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.77700000000000002,"15min":0.76900000000000002},"min":{"1min":0.56899999999999995,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.70699999999999996}]},{"osd":9,"last update":"Fri Feb 2 15:24:36 2024","interfaces":[{"interface":"back","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.97299999999999998},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.76000000000000001}]},{"osd":10,"last update":"Fri Feb 2 15:24:56 2024","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":1.089}]},{"osd":11,"last update":"Fri Feb 2 15:25:10 2024","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":0.753},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":1.1499999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411377,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":933888,"data_stored":458365,"data_compressed":11732,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 15:24:24 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.54500000000000004,"15min":0.54300000000000004},"min":{"1min":0.36499999999999999,"5min":0.253,"15min":0.253},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.52600000000000002,"15min":0.52000000000000002},"min":{"1min":0.32600000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.628}]},{"osd":1,"last update":"Fri Feb 2 15:24:24 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.503,"15min":0.497},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.53600000000000003,"15min":0.52400000000000002},"min":{"1min":0.34599999999999997,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.74399999999999999}]},{"osd":3,"last update":"Fri Feb 2 15:24:45 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.58599999999999997,"15min":0.56699999999999995},"min":{"1min":0.34699999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.623,"5min":0.57199999999999995,"15min":0.55000000000000004},"min":{"1min":0.441,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.90600000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.81000000000000005}]},{"osd":4,"last update":"Fri Feb 2 15:23:59 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.51000000000000001,"15min":0.504},"min":{"1min":0.39900000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.52700000000000002,"15min":0.51400000000000001},"min":{"1min":0.45900000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.75600000000000001}]},{"osd":5,"last update":"Fri Feb 2 15:24:21 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.41699999999999998,"15min":0.40500000000000003},"min":{"1min":0.32200000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.41299999999999998,"15min":0.40400000000000003},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.42999999999999999}]},{"osd":6,"last update":"Fri Feb 2 15:24:39 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.61899999999999999,"15min":0.61699999999999999},"min":{"1min":0.41899999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.626,"15min":0.628},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.67700000000000005}]},{"osd":7,"last update":"Fri Feb 2 15:23:57 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.79700000000000004}]},{"osd":8,"last update":"Fri Feb 2 15:25:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.41899999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.84199999999999997,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.40699999999999997,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.70799999999999996,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.47999999999999998}]},{"osd":9,"last update":"Fri Feb 2 15:24:35 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.70299999999999996}]},{"osd":10,"last update":"Fri Feb 2 15:24:51 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.68899999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77200000000000002}]}]},{"osd":3,"up_from":24,"seq":103079215148,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":921600,"data_stored":454156,"data_compressed":11732,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 15:24:44 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.42899999999999999,"15min":0.41499999999999998},"min":{"1min":0.33000000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.79000000000000004,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.45700000000000002,"15min":0.45000000000000001},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.78300000000000003,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.53800000000000003}]},{"osd":1,"last update":"Fri Feb 2 15:24:44 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.60999999999999999,"15min":0.58199999999999996},"min":{"1min":0.40100000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.60199999999999998,"15min":0.59199999999999997},"min":{"1min":0.44600000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.73299999999999998}]},{"osd":2,"last update":"Fri Feb 2 15:24:44 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.623,"15min":0.61899999999999999},"min":{"1min":0.44500000000000001,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.81899999999999995,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.67500000000000004,"15min":0.64800000000000002},"min":{"1min":0.47699999999999998,"5min":0.443,"15min":0.443},"max":{"1min":1.016,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":0.56799999999999995}]},{"osd":4,"last update":"Fri Feb 2 15:24:59 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.625},"min":{"1min":0.40400000000000003,"5min":0.376,"15min":0.376},"max":{"1min":0.82099999999999995,"5min":1.022,"15min":1.022},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.70999999999999996,"15min":0.69199999999999995},"min":{"1min":0.61499999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0329999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.81100000000000005}]},{"osd":5,"last update":"Fri Feb 2 15:24:21 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.72299999999999998,"15min":0.71699999999999997},"min":{"1min":0.55000000000000004,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0780000000000001,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.64500000000000002,"15min":0.64100000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.48099999999999998}]},{"osd":6,"last update":"Fri Feb 2 15:24:44 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.72499999999999998,"15min":0.70999999999999996},"min":{"1min":0.51800000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.69599999999999995,"15min":0.68999999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.51700000000000002}]},{"osd":7,"last update":"Fri Feb 2 15:24:59 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.75,"15min":0.74099999999999999},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.73599999999999999,"15min":0.72999999999999998},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.97299999999999998,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.61799999999999999}]},{"osd":8,"last update":"Fri Feb 2 15:24:14 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.65800000000000003}]},{"osd":9,"last update":"Fri Feb 2 15:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.60199999999999998}]},{"osd":10,"last update":"Fri Feb 2 15:24:54 2024","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.83599999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74299999999999999}]}]},{"osd":4,"up_from":31,"seq":133143986219,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":929792,"data_stored":455006,"data_compressed":11732,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 15:24:03 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.52300000000000002,"15min":0.52100000000000002},"min":{"1min":0.371,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.78600000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.53800000000000003,"15min":0.53600000000000003},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.47799999999999998}]},{"osd":1,"last update":"Fri Feb 2 15:25:05 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.35299999999999998,"15min":0.33500000000000002},"min":{"1min":0.26600000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.432,"5min":0.36499999999999999,"15min":0.34699999999999998},"min":{"1min":0.28299999999999997,"5min":0.255,"15min":0.255},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.434}]},{"osd":2,"last update":"Fri Feb 2 15:25:05 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.52400000000000002,"15min":0.50700000000000001},"min":{"1min":0.35699999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.52800000000000002,"15min":0.51100000000000001},"min":{"1min":0.40699999999999997,"5min":0.38,"15min":0.38},"max":{"1min":0.753,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.56599999999999995}]},{"osd":3,"last update":"Fri Feb 2 15:25:05 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.56599999999999995,"15min":0.56399999999999995},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.55800000000000005,"15min":0.56000000000000005},"min":{"1min":0.38400000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.70899999999999996,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.59299999999999997}]},{"osd":5,"last update":"Fri Feb 2 15:24:19 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.58899999999999997,"15min":0.58199999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.60299999999999998,"15min":0.60099999999999998},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.61099999999999999}]},{"osd":6,"last update":"Fri Feb 2 15:24:40 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.64300000000000002,"15min":0.63400000000000001},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.65200000000000002,"15min":0.64100000000000001},"min":{"1min":0.45700000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.335,"5min":1.335,"15min":1.335},"last":0.747}]},{"osd":7,"last update":"Fri Feb 2 15:24:59 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.49099999999999999,"15min":0.48299999999999998},"min":{"1min":0.34000000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.53600000000000003,"15min":0.53200000000000003},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.61899999999999999}]},{"osd":8,"last update":"Fri Feb 2 15:25:16 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.65200000000000002,"15min":0.65000000000000002},"min":{"1min":0.42499999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.84299999999999997,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.65600000000000003,"15min":0.65100000000000002},"min":{"1min":0.46999999999999997,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.76700000000000002}]},{"osd":9,"last update":"Fri Feb 2 15:24:35 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.69499999999999995}]},{"osd":10,"last update":"Fri Feb 2 15:24:53 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.64100000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80800000000000005}]}]},{"osd":5,"up_from":36,"seq":154618822694,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":1051198,"data_compressed":47943,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 15:25:20 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.53000000000000003,"15min":0.51500000000000001},"min":{"1min":0.38600000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.53900000000000003,"15min":0.52600000000000002},"min":{"1min":0.39600000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.83599999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.58299999999999996}]},{"osd":1,"last update":"Fri Feb 2 15:25:20 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.56000000000000005,"15min":0.55000000000000004},"min":{"1min":0.46500000000000002,"5min":0.252,"15min":0.252},"max":{"1min":0.69999999999999996,"5min":1.018,"15min":1.018},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.621,"5min":0.56499999999999995,"15min":0.54700000000000004},"min":{"1min":0.46400000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.77500000000000002,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.77500000000000002}]},{"osd":2,"last update":"Fri Feb 2 15:25:20 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.39500000000000002,"15min":0.38300000000000001},"min":{"1min":0.312,"5min":0.215,"15min":0.215},"max":{"1min":0.59499999999999997,"5min":0.63,"15min":0.63},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.40999999999999998,"15min":0.40000000000000002},"min":{"1min":0.318,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.72699999999999998,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.55600000000000005}]},{"osd":3,"last update":"Fri Feb 2 15:25:20 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.54700000000000004,"15min":0.52300000000000002},"min":{"1min":0.378,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.80600000000000005,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.58299999999999996,"15min":0.56599999999999995},"min":{"1min":0.52200000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.64500000000000002}]},{"osd":4,"last update":"Fri Feb 2 15:25:20 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.57499999999999996,"15min":0.56200000000000006},"min":{"1min":0.45700000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.79600000000000004,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.58699999999999997,"15min":0.57599999999999996},"min":{"1min":0.47099999999999997,"5min":0.316,"15min":0.316},"max":{"1min":0.81899999999999995,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.73699999999999999}]},{"osd":6,"last update":"Fri Feb 2 15:24:41 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.625,"15min":0.629},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.82299999999999995,"5min":0.878,"15min":0.878},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.62,"5min":0.63,"15min":0.63200000000000001},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.86899999999999999,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.67300000000000004}]},{"osd":7,"last update":"Fri Feb 2 15:25:00 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.622,"15min":0.62},"min":{"1min":0.46400000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.629,"5min":0.60799999999999998,"15min":0.60399999999999998},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.79200000000000004,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.72799999999999998}]},{"osd":8,"last update":"Fri Feb 2 15:25:14 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.55100000000000005,"15min":0.55400000000000005},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.83999999999999997,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.58699999999999997,"15min":0.59199999999999997},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.80100000000000005,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.83399999999999996}]},{"osd":9,"last update":"Fri Feb 2 15:24:36 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.80800000000000005}]},{"osd":10,"last update":"Fri Feb 2 15:24:53 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.76600000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34572,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34572,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36211,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36211,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36211,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36211,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36211,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":4377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":40598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-02T15:25:22.176 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-02T15:25:22.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:22.407+0000 7ff187e00700 1 -- 172.21.15.1:0/1163852948 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff180100d40 msgr2=0x7ff180101160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:22.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:22.407+0000 7ff187e00700 1 --2- 172.21.15.1:0/1163852948 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff180100d40 0x7ff180101160 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff1700099c0 tx=0x7ff170009cd0 comp rx=0 tx=0).stop 2024-02-02T15:25:22.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:22.407+0000 7ff187e00700 1 -- 172.21.15.1:0/1163852948 shutdown_connections 2024-02-02T15:25:22.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:22.407+0000 7ff187e00700 1 --2- 172.21.15.1:0/1163852948 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff1801028d0 0x7ff180109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:22.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:22.407+0000 7ff187e00700 1 --2- 172.21.15.1:0/1163852948 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff180100d40 0x7ff180101160 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:22.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:22.407+0000 7ff187e00700 1 --2- 172.21.15.1:0/1163852948 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff180101f30 0x7ff180102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:22.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:22.407+0000 7ff187e00700 1 -- 172.21.15.1:0/1163852948 >> 172.21.15.1:0/1163852948 conn(0x7ff1800fc290 msgr2=0x7ff1800fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:22.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:22.407+0000 7ff187e00700 1 -- 172.21.15.1:0/1163852948 shutdown_connections 2024-02-02T15:25:22.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:22.407+0000 7ff187e00700 1 -- 172.21.15.1:0/1163852948 wait complete. 2024-02-02T15:25:22.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:22.409+0000 7ff187e00700 1 Processor -- start 2024-02-02T15:25:22.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:22.409+0000 7ff187e00700 1 -- start start 2024-02-02T15:25:22.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:22.410+0000 7ff187e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff180100d40 0x7ff180198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:22.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:22.410+0000 7ff187e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff180101f30 0x7ff1801991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:22.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:22.410+0000 7ff187e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff1801028d0 0x7ff1801a0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:22.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:22.410+0000 7ff187e00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1801a0800 con 0x7ff180101f30 2024-02-02T15:25:22.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:22.410+0000 7ff187e00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1801a0970 con 0x7ff1801028d0 2024-02-02T15:25:22.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:22.410+0000 7ff187e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1801a0b30 con 0x7ff180100d40 2024-02-02T15:25:22.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:22.410+0000 7ff184a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff180101f30 0x7ff1801991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:22.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:22.410+0000 7ff184a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff180101f30 0x7ff1801991d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33176/0 (socket says 172.21.15.1:33176) 2024-02-02T15:25:22.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:22.410+0000 7ff184a00700 1 -- 172.21.15.1:0/2159453581 learned_addr learned my addr 172.21.15.1:0/2159453581 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:25:22.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:22.410+0000 7ff185400700 1 --2- 172.21.15.1:0/2159453581 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff180100d40 0x7ff180198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:22.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:22.410+0000 7ff185e00700 1 --2- 172.21.15.1:0/2159453581 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff1801028d0 0x7ff1801a0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:22.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:22.410+0000 7ff184a00700 1 -- 172.21.15.1:0/2159453581 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff180100d40 msgr2=0x7ff180198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:22.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:22.410+0000 7ff184a00700 1 --2- 172.21.15.1:0/2159453581 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff180100d40 0x7ff180198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:22.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:22.410+0000 7ff184a00700 1 -- 172.21.15.1:0/2159453581 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff1801028d0 msgr2=0x7ff1801a0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:22.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:22.410+0000 7ff184a00700 1 --2- 172.21.15.1:0/2159453581 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff1801028d0 0x7ff1801a0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:22.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:22.410+0000 7ff184a00700 1 -- 172.21.15.1:0/2159453581 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1700095f0 con 0x7ff180101f30 2024-02-02T15:25:22.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:22.411+0000 7ff184a00700 1 --2- 172.21.15.1:0/2159453581 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff180101f30 0x7ff1801991d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff16800e9a0 tx=0x7ff16800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:22.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:22.411+0000 7ff17e000700 1 -- 172.21.15.1:0/2159453581 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff168009790 con 0x7ff180101f30 2024-02-02T15:25:22.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:22.411+0000 7ff17e000700 1 -- 172.21.15.1:0/2159453581 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff168010430 con 0x7ff180101f30 2024-02-02T15:25:22.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:22.411+0000 7ff187e00700 1 -- 172.21.15.1:0/2159453581 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1801a0e10 con 0x7ff180101f30 2024-02-02T15:25:22.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:22.411+0000 7ff17e000700 1 -- 172.21.15.1:0/2159453581 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff168018680 con 0x7ff180101f30 2024-02-02T15:25:22.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:22.411+0000 7ff187e00700 1 -- 172.21.15.1:0/2159453581 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1801a1290 con 0x7ff180101f30 2024-02-02T15:25:22.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:22.412+0000 7ff187e00700 1 -- 172.21.15.1:0/2159453581 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff180193390 con 0x7ff180101f30 2024-02-02T15:25:22.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:22.414+0000 7ff17e000700 1 -- 172.21.15.1:0/2159453581 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) v1 ==== 135120+0+0 (secure 0 0 0) 0x7ff1680105a0 con 0x7ff180101f30 2024-02-02T15:25:22.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:22.415+0000 7ff17e000700 1 --2- 172.21.15.1:0/2159453581 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7ff1600a05b0 0x7ff1600a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:22.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:22.415+0000 7ff17e000700 1 -- 172.21.15.1:0/2159453581 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7ff168014070 con 0x7ff180101f30 2024-02-02T15:25:22.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:22.416+0000 7ff185400700 1 --2- 172.21.15.1:0/2159453581 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7ff1600a05b0 0x7ff1600a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:22.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:22.416+0000 7ff185400700 1 --2- 172.21.15.1:0/2159453581 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7ff1600a05b0 0x7ff1600a2a70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff180199b60 tx=0x7ff170005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:22.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:22.419+0000 7ff17e000700 1 -- 172.21.15.1:0/2159453581 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7ff168052110 con 0x7ff180101f30 2024-02-02T15:25:22.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:22 smithi001 ceph-mon[35470]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:25:22.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:22 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/1010496905' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:25:22.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:22 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/2979015657' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-02T15:25:22.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:22 smithi071 ceph-mon[44707]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:25:22.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:22 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/1010496905' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:25:22.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:22 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/2979015657' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-02T15:25:22.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:22.626+0000 7ff187e00700 1 -- 172.21.15.1:0/2159453581 --> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff180061180 con 0x7ff1600a05b0 2024-02-02T15:25:22.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:22.639+0000 7ff17e000700 1 -- 172.21.15.1:0/2159453581 <== mgr.14154 v2:172.21.15.1:6800/3768989981 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+423119 (secure 0 0 0) 0x7ff180061180 con 0x7ff1600a05b0 2024-02-02T15:25:22.645 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:25:22.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:22.644+0000 7ff187e00700 1 -- 172.21.15.1:0/2159453581 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7ff1600a05b0 msgr2=0x7ff1600a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:22.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:22.644+0000 7ff187e00700 1 --2- 172.21.15.1:0/2159453581 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7ff1600a05b0 0x7ff1600a2a70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff180199b60 tx=0x7ff170005fb0 comp rx=0 tx=0).stop 2024-02-02T15:25:22.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:22.644+0000 7ff187e00700 1 -- 172.21.15.1:0/2159453581 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff180101f30 msgr2=0x7ff1801991d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:22.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:22.644+0000 7ff187e00700 1 --2- 172.21.15.1:0/2159453581 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff180101f30 0x7ff1801991d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff16800e9a0 tx=0x7ff16800ecb0 comp rx=0 tx=0).stop 2024-02-02T15:25:22.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:22.645+0000 7ff187e00700 1 -- 172.21.15.1:0/2159453581 shutdown_connections 2024-02-02T15:25:22.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:22.645+0000 7ff187e00700 1 --2- 172.21.15.1:0/2159453581 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff1801028d0 0x7ff1801a0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:22.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:22.645+0000 7ff187e00700 1 --2- 172.21.15.1:0/2159453581 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7ff1600a05b0 0x7ff1600a2a70 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:22.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:22.645+0000 7ff187e00700 1 --2- 172.21.15.1:0/2159453581 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff180100d40 0x7ff180198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:22.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:22.645+0000 7ff187e00700 1 --2- 172.21.15.1:0/2159453581 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff180101f30 0x7ff1801991d0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:22.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:22.645+0000 7ff187e00700 1 -- 172.21.15.1:0/2159453581 >> 172.21.15.1:0/2159453581 conn(0x7ff1800fc290 msgr2=0x7ff180107870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:22.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:22.645+0000 7ff187e00700 1 -- 172.21.15.1:0/2159453581 shutdown_connections 2024-02-02T15:25:22.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:22.645+0000 7ff187e00700 1 -- 172.21.15.1:0/2159453581 wait complete. 2024-02-02T15:25:22.650 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-02-02T15:25:22.663 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":252,"stamp":"2024-02-02T15:25:21.912083+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":633691,"num_objects":77,"num_object_clones":0,"num_object_copies":232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":152,"num_read_kb":164,"num_write":355,"num_write_kb":3031,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":49,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":290,"ondisk_log_size":290,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335336,"kb_used_data":11144,"kb_used_omap":20,"kb_used_meta":324075,"kb_avail":1124606488,"statfs":{"total":1151940427776,"available":1151597043712,"internally_reserved":0,"allocated":11411456,"data_stored":7291811,"data_compressed":249417,"data_compressed_allocated":331776,"data_compressed_original":2162688,"omap_allocated":21372,"internal_metadata":331852932},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":38871,"num_objects":53,"num_object_clones":0,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":16,"num_read_kb":45,"num_write":127,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003833"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.800354+0000","last_change":"2024-02-02T15:24:47.793308+0000","last_active":"2024-02-02T15:24:51.800354+0000","last_peered":"2024-02-02T15:24:51.800354+0000","last_clean":"2024-02-02T15:24:51.800354+0000","last_became_active":"2024-02-02T15:24:47.792420+0000","last_became_peered":"2024-02-02T15:24:47.792420+0000","last_unstale":"2024-02-02T15:24:51.800354+0000","last_undegraded":"2024-02-02T15:24:51.800354+0000","last_fullsized":"2024-02-02T15:24:51.800354+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:39:09.871317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.808238+0000","last_change":"2024-02-02T15:24:47.791272+0000","last_active":"2024-02-02T15:24:51.808238+0000","last_peered":"2024-02-02T15:24:51.808238+0000","last_clean":"2024-02-02T15:24:51.808238+0000","last_became_active":"2024-02-02T15:24:47.790902+0000","last_became_peered":"2024-02-02T15:24:47.790902+0000","last_unstale":"2024-02-02T15:24:51.808238+0000","last_undegraded":"2024-02-02T15:24:51.808238+0000","last_fullsized":"2024-02-02T15:24:51.808238+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:31:11.620873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.805893+0000","last_change":"2024-02-02T15:24:47.798712+0000","last_active":"2024-02-02T15:24:51.805893+0000","last_peered":"2024-02-02T15:24:51.805893+0000","last_clean":"2024-02-02T15:24:51.805893+0000","last_became_active":"2024-02-02T15:24:47.798405+0000","last_became_peered":"2024-02-02T15:24:47.798405+0000","last_unstale":"2024-02-02T15:24:51.805893+0000","last_undegraded":"2024-02-02T15:24:51.805893+0000","last_fullsized":"2024-02-02T15:24:51.805893+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:37:15.238070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.800684+0000","last_change":"2024-02-02T15:24:47.786456+0000","last_active":"2024-02-02T15:24:51.800684+0000","last_peered":"2024-02-02T15:24:51.800684+0000","last_clean":"2024-02-02T15:24:51.800684+0000","last_became_active":"2024-02-02T15:24:47.785520+0000","last_became_peered":"2024-02-02T15:24:47.785520+0000","last_unstale":"2024-02-02T15:24:51.800684+0000","last_undegraded":"2024-02-02T15:24:51.800684+0000","last_fullsized":"2024-02-02T15:24:51.800684+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:16:04.003459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.801660+0000","last_change":"2024-02-02T15:24:47.780084+0000","last_active":"2024-02-02T15:24:51.801660+0000","last_peered":"2024-02-02T15:24:51.801660+0000","last_clean":"2024-02-02T15:24:51.801660+0000","last_became_active":"2024-02-02T15:24:47.779667+0000","last_became_peered":"2024-02-02T15:24:47.779667+0000","last_unstale":"2024-02-02T15:24:51.801660+0000","last_undegraded":"2024-02-02T15:24:51.801660+0000","last_fullsized":"2024-02-02T15:24:51.801660+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:30:15.377691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.810959+0000","last_change":"2024-02-02T15:24:47.786173+0000","last_active":"2024-02-02T15:24:51.810959+0000","last_peered":"2024-02-02T15:24:51.810959+0000","last_clean":"2024-02-02T15:24:51.810959+0000","last_became_active":"2024-02-02T15:24:47.785301+0000","last_became_peered":"2024-02-02T15:24:47.785301+0000","last_unstale":"2024-02-02T15:24:51.810959+0000","last_undegraded":"2024-02-02T15:24:51.810959+0000","last_fullsized":"2024-02-02T15:24:51.810959+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:14:58.524317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.800609+0000","last_change":"2024-02-02T15:24:47.790913+0000","last_active":"2024-02-02T15:24:51.800609+0000","last_peered":"2024-02-02T15:24:51.800609+0000","last_clean":"2024-02-02T15:24:51.800609+0000","last_became_active":"2024-02-02T15:24:47.790566+0000","last_became_peered":"2024-02-02T15:24:47.790566+0000","last_unstale":"2024-02-02T15:24:51.800609+0000","last_undegraded":"2024-02-02T15:24:51.800609+0000","last_fullsized":"2024-02-02T15:24:51.800609+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:34:51.954830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:52.000920+0000","last_change":"2024-02-02T15:24:47.786465+0000","last_active":"2024-02-02T15:24:52.000920+0000","last_peered":"2024-02-02T15:24:52.000920+0000","last_clean":"2024-02-02T15:24:52.000920+0000","last_became_active":"2024-02-02T15:24:47.786175+0000","last_became_peered":"2024-02-02T15:24:47.786175+0000","last_unstale":"2024-02-02T15:24:52.000920+0000","last_undegraded":"2024-02-02T15:24:52.000920+0000","last_fullsized":"2024-02-02T15:24:52.000920+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:42:13.748850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.795081+0000","last_change":"2024-02-02T15:24:47.787115+0000","last_active":"2024-02-02T15:24:51.795081+0000","last_peered":"2024-02-02T15:24:51.795081+0000","last_clean":"2024-02-02T15:24:51.795081+0000","last_became_active":"2024-02-02T15:24:47.786804+0000","last_became_peered":"2024-02-02T15:24:47.786804+0000","last_unstale":"2024-02-02T15:24:51.795081+0000","last_undegraded":"2024-02-02T15:24:51.795081+0000","last_fullsized":"2024-02-02T15:24:51.795081+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T03:06:17.873243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.811379+0000","last_change":"2024-02-02T15:24:47.787211+0000","last_active":"2024-02-02T15:24:51.811379+0000","last_peered":"2024-02-02T15:24:51.811379+0000","last_clean":"2024-02-02T15:24:51.811379+0000","last_became_active":"2024-02-02T15:24:47.786888+0000","last_became_peered":"2024-02-02T15:24:47.786888+0000","last_unstale":"2024-02-02T15:24:51.811379+0000","last_undegraded":"2024-02-02T15:24:51.811379+0000","last_fullsized":"2024-02-02T15:24:51.811379+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:14:21.442787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.800451+0000","last_change":"2024-02-02T15:24:47.798962+0000","last_active":"2024-02-02T15:24:51.800451+0000","last_peered":"2024-02-02T15:24:51.800451+0000","last_clean":"2024-02-02T15:24:51.800451+0000","last_became_active":"2024-02-02T15:24:47.798657+0000","last_became_peered":"2024-02-02T15:24:47.798657+0000","last_unstale":"2024-02-02T15:24:51.800451+0000","last_undegraded":"2024-02-02T15:24:51.800451+0000","last_fullsized":"2024-02-02T15:24:51.800451+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:53:37.316027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.800567+0000","last_change":"2024-02-02T15:24:47.782647+0000","last_active":"2024-02-02T15:24:51.800567+0000","last_peered":"2024-02-02T15:24:51.800567+0000","last_clean":"2024-02-02T15:24:51.800567+0000","last_became_active":"2024-02-02T15:24:47.782344+0000","last_became_peered":"2024-02-02T15:24:47.782344+0000","last_unstale":"2024-02-02T15:24:51.800567+0000","last_undegraded":"2024-02-02T15:24:51.800567+0000","last_fullsized":"2024-02-02T15:24:51.800567+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:06:54.874657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.794123+0000","last_change":"2024-02-02T15:24:47.787032+0000","last_active":"2024-02-02T15:24:51.794123+0000","last_peered":"2024-02-02T15:24:51.794123+0000","last_clean":"2024-02-02T15:24:51.794123+0000","last_became_active":"2024-02-02T15:24:47.786710+0000","last_became_peered":"2024-02-02T15:24:47.786710+0000","last_unstale":"2024-02-02T15:24:51.794123+0000","last_undegraded":"2024-02-02T15:24:51.794123+0000","last_fullsized":"2024-02-02T15:24:51.794123+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:26:15.429316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.804049+0000","last_change":"2024-02-02T15:24:47.785708+0000","last_active":"2024-02-02T15:24:51.804049+0000","last_peered":"2024-02-02T15:24:51.804049+0000","last_clean":"2024-02-02T15:24:51.804049+0000","last_became_active":"2024-02-02T15:24:47.785280+0000","last_became_peered":"2024-02-02T15:24:47.785280+0000","last_unstale":"2024-02-02T15:24:51.804049+0000","last_undegraded":"2024-02-02T15:24:51.804049+0000","last_fullsized":"2024-02-02T15:24:51.804049+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T02:21:23.224933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.804879+0000","last_change":"2024-02-02T15:24:47.783904+0000","last_active":"2024-02-02T15:24:51.804879+0000","last_peered":"2024-02-02T15:24:51.804879+0000","last_clean":"2024-02-02T15:24:51.804879+0000","last_became_active":"2024-02-02T15:24:47.783599+0000","last_became_peered":"2024-02-02T15:24:47.783599+0000","last_unstale":"2024-02-02T15:24:51.804879+0000","last_undegraded":"2024-02-02T15:24:51.804879+0000","last_fullsized":"2024-02-02T15:24:51.804879+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:35:45.805527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.794661+0000","last_change":"2024-02-02T15:24:47.795770+0000","last_active":"2024-02-02T15:24:51.794661+0000","last_peered":"2024-02-02T15:24:51.794661+0000","last_clean":"2024-02-02T15:24:51.794661+0000","last_became_active":"2024-02-02T15:24:47.795433+0000","last_became_peered":"2024-02-02T15:24:47.795433+0000","last_unstale":"2024-02-02T15:24:51.794661+0000","last_undegraded":"2024-02-02T15:24:51.794661+0000","last_fullsized":"2024-02-02T15:24:51.794661+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:50:07.496998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.800219+0000","last_change":"2024-02-02T15:24:47.793684+0000","last_active":"2024-02-02T15:24:51.800219+0000","last_peered":"2024-02-02T15:24:51.800219+0000","last_clean":"2024-02-02T15:24:51.800219+0000","last_became_active":"2024-02-02T15:24:47.792853+0000","last_became_peered":"2024-02-02T15:24:47.792853+0000","last_unstale":"2024-02-02T15:24:51.800219+0000","last_undegraded":"2024-02-02T15:24:51.800219+0000","last_fullsized":"2024-02-02T15:24:51.800219+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:30:40.379379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.806672+0000","last_change":"2024-02-02T15:24:47.780935+0000","last_active":"2024-02-02T15:24:51.806672+0000","last_peered":"2024-02-02T15:24:51.806672+0000","last_clean":"2024-02-02T15:24:51.806672+0000","last_became_active":"2024-02-02T15:24:47.780642+0000","last_became_peered":"2024-02-02T15:24:47.780642+0000","last_unstale":"2024-02-02T15:24:51.806672+0000","last_undegraded":"2024-02-02T15:24:51.806672+0000","last_fullsized":"2024-02-02T15:24:51.806672+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:36:37.615368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.946993+0000","last_change":"2024-02-02T15:24:47.791051+0000","last_active":"2024-02-02T15:24:51.946993+0000","last_peered":"2024-02-02T15:24:51.946993+0000","last_clean":"2024-02-02T15:24:51.946993+0000","last_became_active":"2024-02-02T15:24:47.790732+0000","last_became_peered":"2024-02-02T15:24:47.790732+0000","last_unstale":"2024-02-02T15:24:51.946993+0000","last_undegraded":"2024-02-02T15:24:51.946993+0000","last_fullsized":"2024-02-02T15:24:51.946993+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:45:05.155144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.946914+0000","last_change":"2024-02-02T15:24:51.814699+0000","last_active":"2024-02-02T15:24:51.946914+0000","last_peered":"2024-02-02T15:24:51.946914+0000","last_clean":"2024-02-02T15:24:51.946914+0000","last_became_active":"2024-02-02T15:24:51.814251+0000","last_became_peered":"2024-02-02T15:24:51.814251+0000","last_unstale":"2024-02-02T15:24:51.946914+0000","last_undegraded":"2024-02-02T15:24:51.946914+0000","last_fullsized":"2024-02-02T15:24:51.946914+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:27:21.674922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"84'11","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:25:18.234367+0000","last_change":"2024-02-02T15:24:47.269429+0000","last_active":"2024-02-02T15:25:18.234367+0000","last_peered":"2024-02-02T15:25:18.234367+0000","last_clean":"2024-02-02T15:25:18.234367+0000","last_became_active":"2024-02-02T15:24:47.269023+0000","last_became_peered":"2024-02-02T15:24:47.269023+0000","last_unstale":"2024-02-02T15:25:18.234367+0000","last_undegraded":"2024-02-02T15:25:18.234367+0000","last_fullsized":"2024-02-02T15:25:18.234367+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T03:11:46.344438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:52.001073+0000","last_change":"2024-02-02T15:24:47.785900+0000","last_active":"2024-02-02T15:24:52.001073+0000","last_peered":"2024-02-02T15:24:52.001073+0000","last_clean":"2024-02-02T15:24:52.001073+0000","last_became_active":"2024-02-02T15:24:47.781994+0000","last_became_peered":"2024-02-02T15:24:47.781994+0000","last_unstale":"2024-02-02T15:24:52.001073+0000","last_undegraded":"2024-02-02T15:24:52.001073+0000","last_fullsized":"2024-02-02T15:24:52.001073+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:00:52.432637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:24:53.093098+0000","last_change":"2024-02-02T15:24:51.825655+0000","last_active":"2024-02-02T15:24:53.093098+0000","last_peered":"2024-02-02T15:24:53.093098+0000","last_clean":"2024-02-02T15:24:53.093098+0000","last_became_active":"2024-02-02T15:24:51.825199+0000","last_became_peered":"2024-02-02T15:24:51.825199+0000","last_unstale":"2024-02-02T15:24:53.093098+0000","last_undegraded":"2024-02-02T15:24:53.093098+0000","last_fullsized":"2024-02-02T15:24:53.093098+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:50:32.283394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"79'1","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.804946+0000","last_change":"2024-02-02T15:24:47.281869+0000","last_active":"2024-02-02T15:24:51.804946+0000","last_peered":"2024-02-02T15:24:51.804946+0000","last_clean":"2024-02-02T15:24:51.804946+0000","last_became_active":"2024-02-02T15:24:47.280825+0000","last_became_peered":"2024-02-02T15:24:47.280825+0000","last_unstale":"2024-02-02T15:24:51.804946+0000","last_undegraded":"2024-02-02T15:24:51.804946+0000","last_fullsized":"2024-02-02T15:24:51.804946+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:24:19.688389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.806330+0000","last_change":"2024-02-02T15:24:47.795893+0000","last_active":"2024-02-02T15:24:51.806330+0000","last_peered":"2024-02-02T15:24:51.806330+0000","last_clean":"2024-02-02T15:24:51.806330+0000","last_became_active":"2024-02-02T15:24:47.795491+0000","last_became_peered":"2024-02-02T15:24:47.795491+0000","last_unstale":"2024-02-02T15:24:51.806330+0000","last_undegraded":"2024-02-02T15:24:51.806330+0000","last_fullsized":"2024-02-02T15:24:51.806330+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:54:38.923036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:24:53.094510+0000","last_change":"2024-02-02T15:24:51.826980+0000","last_active":"2024-02-02T15:24:53.094510+0000","last_peered":"2024-02-02T15:24:53.094510+0000","last_clean":"2024-02-02T15:24:53.094510+0000","last_became_active":"2024-02-02T15:24:51.826221+0000","last_became_peered":"2024-02-02T15:24:51.826221+0000","last_unstale":"2024-02-02T15:24:53.094510+0000","last_undegraded":"2024-02-02T15:24:53.094510+0000","last_fullsized":"2024-02-02T15:24:53.094510+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:45:17.126174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"84'4","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:25:17.226185+0000","last_change":"2024-02-02T15:24:47.272219+0000","last_active":"2024-02-02T15:25:17.226185+0000","last_peered":"2024-02-02T15:25:17.226185+0000","last_clean":"2024-02-02T15:25:17.226185+0000","last_became_active":"2024-02-02T15:24:47.271899+0000","last_became_peered":"2024-02-02T15:24:47.271899+0000","last_unstale":"2024-02-02T15:25:17.226185+0000","last_undegraded":"2024-02-02T15:25:17.226185+0000","last_fullsized":"2024-02-02T15:25:17.226185+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:44:27.370783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.795158+0000","last_change":"2024-02-02T15:24:47.783434+0000","last_active":"2024-02-02T15:24:51.795158+0000","last_peered":"2024-02-02T15:24:51.795158+0000","last_clean":"2024-02-02T15:24:51.795158+0000","last_became_active":"2024-02-02T15:24:47.782130+0000","last_became_peered":"2024-02-02T15:24:47.782130+0000","last_unstale":"2024-02-02T15:24:51.795158+0000","last_undegraded":"2024-02-02T15:24:51.795158+0000","last_fullsized":"2024-02-02T15:24:51.795158+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:32:25.981913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:24:53.094617+0000","last_change":"2024-02-02T15:24:51.810791+0000","last_active":"2024-02-02T15:24:53.094617+0000","last_peered":"2024-02-02T15:24:53.094617+0000","last_clean":"2024-02-02T15:24:53.094617+0000","last_became_active":"2024-02-02T15:24:51.810379+0000","last_became_peered":"2024-02-02T15:24:51.810379+0000","last_unstale":"2024-02-02T15:24:53.094617+0000","last_undegraded":"2024-02-02T15:24:53.094617+0000","last_fullsized":"2024-02-02T15:24:53.094617+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:49:44.635820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.802389+0000","last_change":"2024-02-02T15:24:47.271225+0000","last_active":"2024-02-02T15:24:51.802389+0000","last_peered":"2024-02-02T15:24:51.802389+0000","last_clean":"2024-02-02T15:24:51.802389+0000","last_became_active":"2024-02-02T15:24:47.270870+0000","last_became_peered":"2024-02-02T15:24:47.270870+0000","last_unstale":"2024-02-02T15:24:51.802389+0000","last_undegraded":"2024-02-02T15:24:51.802389+0000","last_fullsized":"2024-02-02T15:24:51.802389+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:16:55.659950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.801631+0000","last_change":"2024-02-02T15:24:47.791367+0000","last_active":"2024-02-02T15:24:51.801631+0000","last_peered":"2024-02-02T15:24:51.801631+0000","last_clean":"2024-02-02T15:24:51.801631+0000","last_became_active":"2024-02-02T15:24:47.790835+0000","last_became_peered":"2024-02-02T15:24:47.790835+0000","last_unstale":"2024-02-02T15:24:51.801631+0000","last_undegraded":"2024-02-02T15:24:51.801631+0000","last_fullsized":"2024-02-02T15:24:51.801631+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:02:51.807571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.946981+0000","last_change":"2024-02-02T15:24:51.807328+0000","last_active":"2024-02-02T15:24:51.946981+0000","last_peered":"2024-02-02T15:24:51.946981+0000","last_clean":"2024-02-02T15:24:51.946981+0000","last_became_active":"2024-02-02T15:24:51.806853+0000","last_became_peered":"2024-02-02T15:24:51.806853+0000","last_unstale":"2024-02-02T15:24:51.946981+0000","last_undegraded":"2024-02-02T15:24:51.946981+0000","last_fullsized":"2024-02-02T15:24:51.946981+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:46:13.898450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"84'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:25:20.260444+0000","last_change":"2024-02-02T15:24:47.268192+0000","last_active":"2024-02-02T15:25:20.260444+0000","last_peered":"2024-02-02T15:25:20.260444+0000","last_clean":"2024-02-02T15:25:20.260444+0000","last_became_active":"2024-02-02T15:24:47.267796+0000","last_became_peered":"2024-02-02T15:24:47.267796+0000","last_unstale":"2024-02-02T15:25:20.260444+0000","last_undegraded":"2024-02-02T15:25:20.260444+0000","last_fullsized":"2024-02-02T15:25:20.260444+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:12:13.161690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.801719+0000","last_change":"2024-02-02T15:24:47.782976+0000","last_active":"2024-02-02T15:24:51.801719+0000","last_peered":"2024-02-02T15:24:51.801719+0000","last_clean":"2024-02-02T15:24:51.801719+0000","last_became_active":"2024-02-02T15:24:47.782640+0000","last_became_peered":"2024-02-02T15:24:47.782640+0000","last_unstale":"2024-02-02T15:24:51.801719+0000","last_undegraded":"2024-02-02T15:24:51.801719+0000","last_fullsized":"2024-02-02T15:24:51.801719+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T02:09:54.851418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.946963+0000","last_change":"2024-02-02T15:24:51.814799+0000","last_active":"2024-02-02T15:24:51.946963+0000","last_peered":"2024-02-02T15:24:51.946963+0000","last_clean":"2024-02-02T15:24:51.946963+0000","last_became_active":"2024-02-02T15:24:51.814263+0000","last_became_peered":"2024-02-02T15:24:51.814263+0000","last_unstale":"2024-02-02T15:24:51.946963+0000","last_undegraded":"2024-02-02T15:24:51.946963+0000","last_fullsized":"2024-02-02T15:24:51.946963+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:52:02.786496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"84'7","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:25:18.235072+0000","last_change":"2024-02-02T15:24:47.273204+0000","last_active":"2024-02-02T15:25:18.235072+0000","last_peered":"2024-02-02T15:25:18.235072+0000","last_clean":"2024-02-02T15:25:18.235072+0000","last_became_active":"2024-02-02T15:24:47.272888+0000","last_became_peered":"2024-02-02T15:24:47.272888+0000","last_unstale":"2024-02-02T15:25:18.235072+0000","last_undegraded":"2024-02-02T15:25:18.235072+0000","last_fullsized":"2024-02-02T15:25:18.235072+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:26:25.188915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.797918+0000","last_change":"2024-02-02T15:24:47.788645+0000","last_active":"2024-02-02T15:24:51.797918+0000","last_peered":"2024-02-02T15:24:51.797918+0000","last_clean":"2024-02-02T15:24:51.797918+0000","last_became_active":"2024-02-02T15:24:47.787478+0000","last_became_peered":"2024-02-02T15:24:47.787478+0000","last_unstale":"2024-02-02T15:24:51.797918+0000","last_undegraded":"2024-02-02T15:24:51.797918+0000","last_fullsized":"2024-02-02T15:24:51.797918+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:24:45.327030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:24:52.868735+0000","last_change":"2024-02-02T15:24:51.822681+0000","last_active":"2024-02-02T15:24:52.868735+0000","last_peered":"2024-02-02T15:24:52.868735+0000","last_clean":"2024-02-02T15:24:52.868735+0000","last_became_active":"2024-02-02T15:24:51.822202+0000","last_became_peered":"2024-02-02T15:24:51.822202+0000","last_unstale":"2024-02-02T15:24:52.868735+0000","last_undegraded":"2024-02-02T15:24:52.868735+0000","last_fullsized":"2024-02-02T15:24:52.868735+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:05:43.751558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"84'2","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:25:17.229705+0000","last_change":"2024-02-02T15:24:47.271095+0000","last_active":"2024-02-02T15:25:17.229705+0000","last_peered":"2024-02-02T15:25:17.229705+0000","last_clean":"2024-02-02T15:25:17.229705+0000","last_became_active":"2024-02-02T15:24:47.270696+0000","last_became_peered":"2024-02-02T15:24:47.270696+0000","last_unstale":"2024-02-02T15:25:17.229705+0000","last_undegraded":"2024-02-02T15:25:17.229705+0000","last_fullsized":"2024-02-02T15:25:17.229705+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T02:20:59.370217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:52.000970+0000","last_change":"2024-02-02T15:24:47.791699+0000","last_active":"2024-02-02T15:24:52.000970+0000","last_peered":"2024-02-02T15:24:52.000970+0000","last_clean":"2024-02-02T15:24:52.000970+0000","last_became_active":"2024-02-02T15:24:47.787280+0000","last_became_peered":"2024-02-02T15:24:47.787280+0000","last_unstale":"2024-02-02T15:24:52.000970+0000","last_undegraded":"2024-02-02T15:24:52.000970+0000","last_fullsized":"2024-02-02T15:24:52.000970+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:18:10.316393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.812325+0000","last_change":"2024-02-02T15:24:51.810050+0000","last_active":"2024-02-02T15:24:51.812325+0000","last_peered":"2024-02-02T15:24:51.812325+0000","last_clean":"2024-02-02T15:24:51.812325+0000","last_became_active":"2024-02-02T15:24:51.809621+0000","last_became_peered":"2024-02-02T15:24:51.809621+0000","last_unstale":"2024-02-02T15:24:51.812325+0000","last_undegraded":"2024-02-02T15:24:51.812325+0000","last_fullsized":"2024-02-02T15:24:51.812325+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:52:35.885973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"84'3","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:25:19.241022+0000","last_change":"2024-02-02T15:24:47.267973+0000","last_active":"2024-02-02T15:25:19.241022+0000","last_peered":"2024-02-02T15:25:19.241022+0000","last_clean":"2024-02-02T15:25:19.241022+0000","last_became_active":"2024-02-02T15:24:47.267649+0000","last_became_peered":"2024-02-02T15:24:47.267649+0000","last_unstale":"2024-02-02T15:25:19.241022+0000","last_undegraded":"2024-02-02T15:25:19.241022+0000","last_fullsized":"2024-02-02T15:25:19.241022+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T02:46:56.844570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.802331+0000","last_change":"2024-02-02T15:24:47.798636+0000","last_active":"2024-02-02T15:24:51.802331+0000","last_peered":"2024-02-02T15:24:51.802331+0000","last_clean":"2024-02-02T15:24:51.802331+0000","last_became_active":"2024-02-02T15:24:47.798328+0000","last_became_peered":"2024-02-02T15:24:47.798328+0000","last_unstale":"2024-02-02T15:24:51.802331+0000","last_undegraded":"2024-02-02T15:24:51.802331+0000","last_fullsized":"2024-02-02T15:24:51.802331+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:37:43.687336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:52.000988+0000","last_change":"2024-02-02T15:24:51.811312+0000","last_active":"2024-02-02T15:24:52.000988+0000","last_peered":"2024-02-02T15:24:52.000988+0000","last_clean":"2024-02-02T15:24:52.000988+0000","last_became_active":"2024-02-02T15:24:51.810932+0000","last_became_peered":"2024-02-02T15:24:51.810932+0000","last_unstale":"2024-02-02T15:24:52.000988+0000","last_undegraded":"2024-02-02T15:24:52.000988+0000","last_fullsized":"2024-02-02T15:24:52.000988+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:10:31.530518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"84'4","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:25:20.260121+0000","last_change":"2024-02-02T15:24:47.271483+0000","last_active":"2024-02-02T15:25:20.260121+0000","last_peered":"2024-02-02T15:25:20.260121+0000","last_clean":"2024-02-02T15:25:20.260121+0000","last_became_active":"2024-02-02T15:24:47.271160+0000","last_became_peered":"2024-02-02T15:24:47.271160+0000","last_unstale":"2024-02-02T15:25:20.260121+0000","last_undegraded":"2024-02-02T15:25:20.260121+0000","last_fullsized":"2024-02-02T15:25:20.260121+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:29:20.135351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.795145+0000","last_change":"2024-02-02T15:24:47.786492+0000","last_active":"2024-02-02T15:24:51.795145+0000","last_peered":"2024-02-02T15:24:51.795145+0000","last_clean":"2024-02-02T15:24:51.795145+0000","last_became_active":"2024-02-02T15:24:47.786103+0000","last_became_peered":"2024-02-02T15:24:47.786103+0000","last_unstale":"2024-02-02T15:24:51.795145+0000","last_undegraded":"2024-02-02T15:24:51.795145+0000","last_fullsized":"2024-02-02T15:24:51.795145+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:24:23.666596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:24:53.093337+0000","last_change":"2024-02-02T15:24:51.819941+0000","last_active":"2024-02-02T15:24:53.093337+0000","last_peered":"2024-02-02T15:24:53.093337+0000","last_clean":"2024-02-02T15:24:53.093337+0000","last_became_active":"2024-02-02T15:24:51.818066+0000","last_became_peered":"2024-02-02T15:24:51.818066+0000","last_unstale":"2024-02-02T15:24:53.093337+0000","last_undegraded":"2024-02-02T15:24:53.093337+0000","last_fullsized":"2024-02-02T15:24:53.093337+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:55:51.550988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"84'4","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:25:19.241307+0000","last_change":"2024-02-02T15:24:47.268541+0000","last_active":"2024-02-02T15:25:19.241307+0000","last_peered":"2024-02-02T15:25:19.241307+0000","last_clean":"2024-02-02T15:25:19.241307+0000","last_became_active":"2024-02-02T15:24:47.268184+0000","last_became_peered":"2024-02-02T15:24:47.268184+0000","last_unstale":"2024-02-02T15:25:19.241307+0000","last_undegraded":"2024-02-02T15:25:19.241307+0000","last_fullsized":"2024-02-02T15:25:19.241307+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:30:12.187297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.806739+0000","last_change":"2024-02-02T15:24:47.787792+0000","last_active":"2024-02-02T15:24:51.806739+0000","last_peered":"2024-02-02T15:24:51.806739+0000","last_clean":"2024-02-02T15:24:51.806739+0000","last_became_active":"2024-02-02T15:24:47.787479+0000","last_became_peered":"2024-02-02T15:24:47.787479+0000","last_unstale":"2024-02-02T15:24:51.806739+0000","last_undegraded":"2024-02-02T15:24:51.806739+0000","last_fullsized":"2024-02-02T15:24:51.806739+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T03:24:06.544232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:24:53.094691+0000","last_change":"2024-02-02T15:24:51.826639+0000","last_active":"2024-02-02T15:24:53.094691+0000","last_peered":"2024-02-02T15:24:53.094691+0000","last_clean":"2024-02-02T15:24:53.094691+0000","last_became_active":"2024-02-02T15:24:51.825089+0000","last_became_peered":"2024-02-02T15:24:51.825089+0000","last_unstale":"2024-02-02T15:24:53.094691+0000","last_undegraded":"2024-02-02T15:24:53.094691+0000","last_fullsized":"2024-02-02T15:24:53.094691+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:41:44.592603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"84'5","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:25:18.233151+0000","last_change":"2024-02-02T15:24:47.281490+0000","last_active":"2024-02-02T15:25:18.233151+0000","last_peered":"2024-02-02T15:25:18.233151+0000","last_clean":"2024-02-02T15:25:18.233151+0000","last_became_active":"2024-02-02T15:24:47.281137+0000","last_became_peered":"2024-02-02T15:24:47.281137+0000","last_unstale":"2024-02-02T15:25:18.233151+0000","last_undegraded":"2024-02-02T15:25:18.233151+0000","last_fullsized":"2024-02-02T15:25:18.233151+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:42:08.508881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.795016+0000","last_change":"2024-02-02T15:24:47.781135+0000","last_active":"2024-02-02T15:24:51.795016+0000","last_peered":"2024-02-02T15:24:51.795016+0000","last_clean":"2024-02-02T15:24:51.795016+0000","last_became_active":"2024-02-02T15:24:47.780817+0000","last_became_peered":"2024-02-02T15:24:47.780817+0000","last_unstale":"2024-02-02T15:24:51.795016+0000","last_undegraded":"2024-02-02T15:24:51.795016+0000","last_fullsized":"2024-02-02T15:24:51.795016+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:33:05.862934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:24:53.093584+0000","last_change":"2024-02-02T15:24:51.822935+0000","last_active":"2024-02-02T15:24:53.093584+0000","last_peered":"2024-02-02T15:24:53.093584+0000","last_clean":"2024-02-02T15:24:53.093584+0000","last_became_active":"2024-02-02T15:24:51.822222+0000","last_became_peered":"2024-02-02T15:24:51.822222+0000","last_unstale":"2024-02-02T15:24:53.093584+0000","last_undegraded":"2024-02-02T15:24:53.093584+0000","last_fullsized":"2024-02-02T15:24:53.093584+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:54:44.578112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"84'24","reported_seq":44,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:25:20.263806+0000","last_change":"2024-02-02T15:24:47.270604+0000","last_active":"2024-02-02T15:25:20.263806+0000","last_peered":"2024-02-02T15:25:20.263806+0000","last_clean":"2024-02-02T15:25:20.263806+0000","last_became_active":"2024-02-02T15:24:47.270261+0000","last_became_peered":"2024-02-02T15:24:47.270261+0000","last_unstale":"2024-02-02T15:25:20.263806+0000","last_undegraded":"2024-02-02T15:25:20.263806+0000","last_fullsized":"2024-02-02T15:25:20.263806+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:56:33.767423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38518,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":4,"num_read_kb":35,"num_write":26,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.801998+0000","last_change":"2024-02-02T15:24:47.789663+0000","last_active":"2024-02-02T15:24:51.801998+0000","last_peered":"2024-02-02T15:24:51.801998+0000","last_clean":"2024-02-02T15:24:51.801998+0000","last_became_active":"2024-02-02T15:24:47.789352+0000","last_became_peered":"2024-02-02T15:24:47.789352+0000","last_unstale":"2024-02-02T15:24:51.801998+0000","last_undegraded":"2024-02-02T15:24:51.801998+0000","last_fullsized":"2024-02-02T15:24:51.801998+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:13:37.991350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.816451+0000","last_change":"2024-02-02T15:24:51.816021+0000","last_active":"2024-02-02T15:24:51.816451+0000","last_peered":"2024-02-02T15:24:51.816451+0000","last_clean":"2024-02-02T15:24:51.816451+0000","last_became_active":"2024-02-02T15:24:51.815698+0000","last_became_peered":"2024-02-02T15:24:51.815698+0000","last_unstale":"2024-02-02T15:24:51.816451+0000","last_undegraded":"2024-02-02T15:24:51.816451+0000","last_fullsized":"2024-02-02T15:24:51.816451+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:11:10.670195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"84'4","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:25:16.219059+0000","last_change":"2024-02-02T15:24:47.272522+0000","last_active":"2024-02-02T15:25:16.219059+0000","last_peered":"2024-02-02T15:25:16.219059+0000","last_clean":"2024-02-02T15:25:16.219059+0000","last_became_active":"2024-02-02T15:24:47.272087+0000","last_became_peered":"2024-02-02T15:24:47.272087+0000","last_unstale":"2024-02-02T15:25:16.219059+0000","last_undegraded":"2024-02-02T15:25:16.219059+0000","last_fullsized":"2024-02-02T15:25:16.219059+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:51:49.484821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.793946+0000","last_change":"2024-02-02T15:24:47.798089+0000","last_active":"2024-02-02T15:24:51.793946+0000","last_peered":"2024-02-02T15:24:51.793946+0000","last_clean":"2024-02-02T15:24:51.793946+0000","last_became_active":"2024-02-02T15:24:47.793914+0000","last_became_peered":"2024-02-02T15:24:47.793914+0000","last_unstale":"2024-02-02T15:24:51.793946+0000","last_undegraded":"2024-02-02T15:24:51.793946+0000","last_fullsized":"2024-02-02T15:24:51.793946+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:33:39.927550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:24:52.868395+0000","last_change":"2024-02-02T15:24:51.823098+0000","last_active":"2024-02-02T15:24:52.868395+0000","last_peered":"2024-02-02T15:24:52.868395+0000","last_clean":"2024-02-02T15:24:52.868395+0000","last_became_active":"2024-02-02T15:24:51.822634+0000","last_became_peered":"2024-02-02T15:24:51.822634+0000","last_unstale":"2024-02-02T15:24:52.868395+0000","last_undegraded":"2024-02-02T15:24:52.868395+0000","last_fullsized":"2024-02-02T15:24:52.868395+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T03:18:07.420196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"84'5","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:25:19.241109+0000","last_change":"2024-02-02T15:24:47.268962+0000","last_active":"2024-02-02T15:25:19.241109+0000","last_peered":"2024-02-02T15:25:19.241109+0000","last_clean":"2024-02-02T15:25:19.241109+0000","last_became_active":"2024-02-02T15:24:47.268607+0000","last_became_peered":"2024-02-02T15:24:47.268607+0000","last_unstale":"2024-02-02T15:25:19.241109+0000","last_undegraded":"2024-02-02T15:25:19.241109+0000","last_fullsized":"2024-02-02T15:25:19.241109+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:31:06.330437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.800131+0000","last_change":"2024-02-02T15:24:47.792205+0000","last_active":"2024-02-02T15:24:51.800131+0000","last_peered":"2024-02-02T15:24:51.800131+0000","last_clean":"2024-02-02T15:24:51.800131+0000","last_became_active":"2024-02-02T15:24:47.791874+0000","last_became_peered":"2024-02-02T15:24:47.791874+0000","last_unstale":"2024-02-02T15:24:51.800131+0000","last_undegraded":"2024-02-02T15:24:51.800131+0000","last_fullsized":"2024-02-02T15:24:51.800131+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:33:38.795921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:24:53.094439+0000","last_change":"2024-02-02T15:24:51.825499+0000","last_active":"2024-02-02T15:24:53.094439+0000","last_peered":"2024-02-02T15:24:53.094439+0000","last_clean":"2024-02-02T15:24:53.094439+0000","last_became_active":"2024-02-02T15:24:51.825049+0000","last_became_peered":"2024-02-02T15:24:51.825049+0000","last_unstale":"2024-02-02T15:24:53.094439+0000","last_undegraded":"2024-02-02T15:24:53.094439+0000","last_fullsized":"2024-02-02T15:24:53.094439+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:41:05.430891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"84'6","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:25:18.233586+0000","last_change":"2024-02-02T15:24:47.267683+0000","last_active":"2024-02-02T15:25:18.233586+0000","last_peered":"2024-02-02T15:25:18.233586+0000","last_clean":"2024-02-02T15:25:18.233586+0000","last_became_active":"2024-02-02T15:24:47.267324+0000","last_became_peered":"2024-02-02T15:24:47.267324+0000","last_unstale":"2024-02-02T15:25:18.233586+0000","last_undegraded":"2024-02-02T15:25:18.233586+0000","last_fullsized":"2024-02-02T15:25:18.233586+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:03:31.326791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":932,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.799771+0000","last_change":"2024-02-02T15:24:47.792509+0000","last_active":"2024-02-02T15:24:51.799771+0000","last_peered":"2024-02-02T15:24:51.799771+0000","last_clean":"2024-02-02T15:24:51.799771+0000","last_became_active":"2024-02-02T15:24:47.791662+0000","last_became_peered":"2024-02-02T15:24:47.791662+0000","last_unstale":"2024-02-02T15:24:51.799771+0000","last_undegraded":"2024-02-02T15:24:51.799771+0000","last_fullsized":"2024-02-02T15:24:51.799771+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:42:50.009971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:24:52.315377+0000","last_change":"2024-02-02T15:24:51.815417+0000","last_active":"2024-02-02T15:24:52.315377+0000","last_peered":"2024-02-02T15:24:52.315377+0000","last_clean":"2024-02-02T15:24:52.315377+0000","last_became_active":"2024-02-02T15:24:51.815021+0000","last_became_peered":"2024-02-02T15:24:51.815021+0000","last_unstale":"2024-02-02T15:24:52.315377+0000","last_undegraded":"2024-02-02T15:24:52.315377+0000","last_fullsized":"2024-02-02T15:24:52.315377+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:22:43.790920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"84'15","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:25:17.230723+0000","last_change":"2024-02-02T15:24:47.274152+0000","last_active":"2024-02-02T15:25:17.230723+0000","last_peered":"2024-02-02T15:25:17.230723+0000","last_clean":"2024-02-02T15:25:17.230723+0000","last_became_active":"2024-02-02T15:24:47.273720+0000","last_became_peered":"2024-02-02T15:24:47.273720+0000","last_unstale":"2024-02-02T15:25:17.230723+0000","last_undegraded":"2024-02-02T15:25:17.230723+0000","last_fullsized":"2024-02-02T15:25:17.230723+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:38:05.280309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.803323+0000","last_change":"2024-02-02T15:24:47.790681+0000","last_active":"2024-02-02T15:24:51.803323+0000","last_peered":"2024-02-02T15:24:51.803323+0000","last_clean":"2024-02-02T15:24:51.803323+0000","last_became_active":"2024-02-02T15:24:47.789952+0000","last_became_peered":"2024-02-02T15:24:47.789952+0000","last_unstale":"2024-02-02T15:24:51.803323+0000","last_undegraded":"2024-02-02T15:24:51.803323+0000","last_fullsized":"2024-02-02T15:24:51.803323+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:21:03.103398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.813237+0000","last_change":"2024-02-02T15:24:51.812676+0000","last_active":"2024-02-02T15:24:51.813237+0000","last_peered":"2024-02-02T15:24:51.813237+0000","last_clean":"2024-02-02T15:24:51.813237+0000","last_became_active":"2024-02-02T15:24:51.812212+0000","last_became_peered":"2024-02-02T15:24:51.812212+0000","last_unstale":"2024-02-02T15:24:51.813237+0000","last_undegraded":"2024-02-02T15:24:51.813237+0000","last_fullsized":"2024-02-02T15:24:51.813237+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:48:01.944379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"79'1","reported_seq":17,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.802403+0000","last_change":"2024-02-02T15:24:47.271707+0000","last_active":"2024-02-02T15:24:51.802403+0000","last_peered":"2024-02-02T15:24:51.802403+0000","last_clean":"2024-02-02T15:24:51.802403+0000","last_became_active":"2024-02-02T15:24:47.271379+0000","last_became_peered":"2024-02-02T15:24:47.271379+0000","last_unstale":"2024-02-02T15:24:51.802403+0000","last_undegraded":"2024-02-02T15:24:51.802403+0000","last_fullsized":"2024-02-02T15:24:51.802403+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:00:24.571385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.799394+0000","last_change":"2024-02-02T15:24:47.788360+0000","last_active":"2024-02-02T15:24:51.799394+0000","last_peered":"2024-02-02T15:24:51.799394+0000","last_clean":"2024-02-02T15:24:51.799394+0000","last_became_active":"2024-02-02T15:24:47.787945+0000","last_became_peered":"2024-02-02T15:24:47.787945+0000","last_unstale":"2024-02-02T15:24:51.799394+0000","last_undegraded":"2024-02-02T15:24:51.799394+0000","last_fullsized":"2024-02-02T15:24:51.799394+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:37:54.531487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:24:52.316113+0000","last_change":"2024-02-02T15:24:51.821320+0000","last_active":"2024-02-02T15:24:52.316113+0000","last_peered":"2024-02-02T15:24:52.316113+0000","last_clean":"2024-02-02T15:24:52.316113+0000","last_became_active":"2024-02-02T15:24:51.820853+0000","last_became_peered":"2024-02-02T15:24:51.820853+0000","last_unstale":"2024-02-02T15:24:52.316113+0000","last_undegraded":"2024-02-02T15:24:52.316113+0000","last_fullsized":"2024-02-02T15:24:52.316113+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:20:12.304517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"84'1","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:24:53.093698+0000","last_change":"2024-02-02T15:24:47.278291+0000","last_active":"2024-02-02T15:24:53.093698+0000","last_peered":"2024-02-02T15:24:53.093698+0000","last_clean":"2024-02-02T15:24:53.093698+0000","last_became_active":"2024-02-02T15:24:47.276163+0000","last_became_peered":"2024-02-02T15:24:47.276163+0000","last_unstale":"2024-02-02T15:24:53.093698+0000","last_undegraded":"2024-02-02T15:24:53.093698+0000","last_fullsized":"2024-02-02T15:24:53.093698+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:53:50.312627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.811070+0000","last_change":"2024-02-02T15:24:47.791290+0000","last_active":"2024-02-02T15:24:51.811070+0000","last_peered":"2024-02-02T15:24:51.811070+0000","last_clean":"2024-02-02T15:24:51.811070+0000","last_became_active":"2024-02-02T15:24:47.790987+0000","last_became_peered":"2024-02-02T15:24:47.790987+0000","last_unstale":"2024-02-02T15:24:51.811070+0000","last_undegraded":"2024-02-02T15:24:51.811070+0000","last_fullsized":"2024-02-02T15:24:51.811070+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:38:08.803338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:24:52.315363+0000","last_change":"2024-02-02T15:24:51.813050+0000","last_active":"2024-02-02T15:24:52.315363+0000","last_peered":"2024-02-02T15:24:52.315363+0000","last_clean":"2024-02-02T15:24:52.315363+0000","last_became_active":"2024-02-02T15:24:51.812634+0000","last_became_peered":"2024-02-02T15:24:51.812634+0000","last_unstale":"2024-02-02T15:24:52.315363+0000","last_undegraded":"2024-02-02T15:24:52.315363+0000","last_fullsized":"2024-02-02T15:24:52.315363+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:06:56.428431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"84'5","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:25:20.261463+0000","last_change":"2024-02-02T15:24:47.274033+0000","last_active":"2024-02-02T15:25:20.261463+0000","last_peered":"2024-02-02T15:25:20.261463+0000","last_clean":"2024-02-02T15:25:20.261463+0000","last_became_active":"2024-02-02T15:24:47.273672+0000","last_became_peered":"2024-02-02T15:24:47.273672+0000","last_unstale":"2024-02-02T15:25:20.261463+0000","last_undegraded":"2024-02-02T15:25:20.261463+0000","last_fullsized":"2024-02-02T15:25:20.261463+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:51:46.926397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.811312+0000","last_change":"2024-02-02T15:24:47.786533+0000","last_active":"2024-02-02T15:24:51.811312+0000","last_peered":"2024-02-02T15:24:51.811312+0000","last_clean":"2024-02-02T15:24:51.811312+0000","last_became_active":"2024-02-02T15:24:47.785436+0000","last_became_peered":"2024-02-02T15:24:47.785436+0000","last_unstale":"2024-02-02T15:24:51.811312+0000","last_undegraded":"2024-02-02T15:24:51.811312+0000","last_fullsized":"2024-02-02T15:24:51.811312+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:23:45.910800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:24:52.315298+0000","last_change":"2024-02-02T15:24:51.814326+0000","last_active":"2024-02-02T15:24:52.315298+0000","last_peered":"2024-02-02T15:24:52.315298+0000","last_clean":"2024-02-02T15:24:52.315298+0000","last_became_active":"2024-02-02T15:24:51.813909+0000","last_became_peered":"2024-02-02T15:24:51.813909+0000","last_unstale":"2024-02-02T15:24:52.315298+0000","last_undegraded":"2024-02-02T15:24:52.315298+0000","last_fullsized":"2024-02-02T15:24:52.315298+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:16:12.902718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.799212+0000","last_change":"2024-02-02T15:24:47.788308+0000","last_active":"2024-02-02T15:24:51.799212+0000","last_peered":"2024-02-02T15:24:51.799212+0000","last_clean":"2024-02-02T15:24:51.799212+0000","last_became_active":"2024-02-02T15:24:47.787988+0000","last_became_peered":"2024-02-02T15:24:47.787988+0000","last_unstale":"2024-02-02T15:24:51.799212+0000","last_undegraded":"2024-02-02T15:24:51.799212+0000","last_fullsized":"2024-02-02T15:24:51.799212+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:44:38.103758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"84'4","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:25:17.230363+0000","last_change":"2024-02-02T15:24:47.274437+0000","last_active":"2024-02-02T15:25:17.230363+0000","last_peered":"2024-02-02T15:25:17.230363+0000","last_clean":"2024-02-02T15:25:17.230363+0000","last_became_active":"2024-02-02T15:24:47.274010+0000","last_became_peered":"2024-02-02T15:24:47.274010+0000","last_unstale":"2024-02-02T15:25:17.230363+0000","last_undegraded":"2024-02-02T15:25:17.230363+0000","last_fullsized":"2024-02-02T15:25:17.230363+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:10:36.862190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"84'1","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:24:53.094023+0000","last_change":"2024-02-02T15:24:51.825698+0000","last_active":"2024-02-02T15:24:53.094023+0000","last_peered":"2024-02-02T15:24:53.094023+0000","last_clean":"2024-02-02T15:24:53.094023+0000","last_became_active":"2024-02-02T15:24:51.825295+0000","last_became_peered":"2024-02-02T15:24:51.825295+0000","last_unstale":"2024-02-02T15:24:53.094023+0000","last_undegraded":"2024-02-02T15:24:53.094023+0000","last_fullsized":"2024-02-02T15:24:53.094023+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:17:09.838124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:52.001031+0000","last_change":"2024-02-02T15:24:47.781836+0000","last_active":"2024-02-02T15:24:52.001031+0000","last_peered":"2024-02-02T15:24:52.001031+0000","last_clean":"2024-02-02T15:24:52.001031+0000","last_became_active":"2024-02-02T15:24:47.781472+0000","last_became_peered":"2024-02-02T15:24:47.781472+0000","last_unstale":"2024-02-02T15:24:52.001031+0000","last_undegraded":"2024-02-02T15:24:52.001031+0000","last_fullsized":"2024-02-02T15:24:52.001031+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:46:33.714191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"84'6","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:25:18.234742+0000","last_change":"2024-02-02T15:24:47.269087+0000","last_active":"2024-02-02T15:25:18.234742+0000","last_peered":"2024-02-02T15:25:18.234742+0000","last_clean":"2024-02-02T15:25:18.234742+0000","last_became_active":"2024-02-02T15:24:47.268743+0000","last_became_peered":"2024-02-02T15:24:47.268743+0000","last_unstale":"2024-02-02T15:25:18.234742+0000","last_undegraded":"2024-02-02T15:25:18.234742+0000","last_fullsized":"2024-02-02T15:25:18.234742+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:03:04.323079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":334,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:24:53.094546+0000","last_change":"2024-02-02T15:23:55.191735+0000","last_active":"2024-02-02T15:24:53.094546+0000","last_peered":"2024-02-02T15:24:53.094546+0000","last_clean":"2024-02-02T15:24:53.094546+0000","last_became_active":"2024-02-02T15:23:55.191119+0000","last_became_peered":"2024-02-02T15:23:55.191119+0000","last_unstale":"2024-02-02T15:24:53.094546+0000","last_undegraded":"2024-02-02T15:24:53.094546+0000","last_fullsized":"2024-02-02T15:24:53.094546+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:21:22.472631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:21:22.472631+0000","last_clean_scrub_stamp":"2024-02-02T15:21:22.472631+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:24:44.153461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.823480+0000","last_change":"2024-02-02T15:24:51.822842+0000","last_active":"2024-02-02T15:24:51.823480+0000","last_peered":"2024-02-02T15:24:51.823480+0000","last_clean":"2024-02-02T15:24:51.823480+0000","last_became_active":"2024-02-02T15:24:51.822372+0000","last_became_peered":"2024-02-02T15:24:51.822372+0000","last_unstale":"2024-02-02T15:24:51.823480+0000","last_undegraded":"2024-02-02T15:24:51.823480+0000","last_fullsized":"2024-02-02T15:24:51.823480+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:45:29.631422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.793629+0000","last_change":"2024-02-02T15:24:47.786507+0000","last_active":"2024-02-02T15:24:51.793629+0000","last_peered":"2024-02-02T15:24:51.793629+0000","last_clean":"2024-02-02T15:24:51.793629+0000","last_became_active":"2024-02-02T15:24:47.786125+0000","last_became_peered":"2024-02-02T15:24:47.786125+0000","last_unstale":"2024-02-02T15:24:51.793629+0000","last_undegraded":"2024-02-02T15:24:51.793629+0000","last_fullsized":"2024-02-02T15:24:51.793629+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:33:09.014258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"84'4","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:25:17.228222+0000","last_change":"2024-02-02T15:24:47.271661+0000","last_active":"2024-02-02T15:25:17.228222+0000","last_peered":"2024-02-02T15:25:17.228222+0000","last_clean":"2024-02-02T15:25:17.228222+0000","last_became_active":"2024-02-02T15:24:47.271258+0000","last_became_peered":"2024-02-02T15:24:47.271258+0000","last_unstale":"2024-02-02T15:25:17.228222+0000","last_undegraded":"2024-02-02T15:25:17.228222+0000","last_fullsized":"2024-02-02T15:25:17.228222+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:14:38.391361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:24:52.868393+0000","last_change":"2024-02-02T15:24:51.808979+0000","last_active":"2024-02-02T15:24:52.868393+0000","last_peered":"2024-02-02T15:24:52.868393+0000","last_clean":"2024-02-02T15:24:52.868393+0000","last_became_active":"2024-02-02T15:24:51.808557+0000","last_became_peered":"2024-02-02T15:24:51.808557+0000","last_unstale":"2024-02-02T15:24:52.868393+0000","last_undegraded":"2024-02-02T15:24:52.868393+0000","last_fullsized":"2024-02-02T15:24:52.868393+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T02:38:34.697137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"84'1","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:24:52.316127+0000","last_change":"2024-02-02T15:24:47.281396+0000","last_active":"2024-02-02T15:24:52.316127+0000","last_peered":"2024-02-02T15:24:52.316127+0000","last_clean":"2024-02-02T15:24:52.316127+0000","last_became_active":"2024-02-02T15:24:47.281036+0000","last_became_peered":"2024-02-02T15:24:47.281036+0000","last_unstale":"2024-02-02T15:24:52.316127+0000","last_undegraded":"2024-02-02T15:24:52.316127+0000","last_fullsized":"2024-02-02T15:24:52.316127+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:54:47.139151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.794267+0000","last_change":"2024-02-02T15:24:47.791057+0000","last_active":"2024-02-02T15:24:51.794267+0000","last_peered":"2024-02-02T15:24:51.794267+0000","last_clean":"2024-02-02T15:24:51.794267+0000","last_became_active":"2024-02-02T15:24:47.790647+0000","last_became_peered":"2024-02-02T15:24:47.790647+0000","last_unstale":"2024-02-02T15:24:51.794267+0000","last_undegraded":"2024-02-02T15:24:51.794267+0000","last_fullsized":"2024-02-02T15:24:51.794267+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T02:49:42.067700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:52.000949+0000","last_change":"2024-02-02T15:24:51.817792+0000","last_active":"2024-02-02T15:24:52.000949+0000","last_peered":"2024-02-02T15:24:52.000949+0000","last_clean":"2024-02-02T15:24:52.000949+0000","last_became_active":"2024-02-02T15:24:51.815800+0000","last_became_peered":"2024-02-02T15:24:51.815800+0000","last_unstale":"2024-02-02T15:24:52.000949+0000","last_undegraded":"2024-02-02T15:24:52.000949+0000","last_fullsized":"2024-02-02T15:24:52.000949+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:28:10.440251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"84'7","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:25:18.236406+0000","last_change":"2024-02-02T15:24:47.277098+0000","last_active":"2024-02-02T15:25:18.236406+0000","last_peered":"2024-02-02T15:25:18.236406+0000","last_clean":"2024-02-02T15:25:18.236406+0000","last_became_active":"2024-02-02T15:24:47.275867+0000","last_became_peered":"2024-02-02T15:24:47.275867+0000","last_unstale":"2024-02-02T15:25:18.236406+0000","last_undegraded":"2024-02-02T15:25:18.236406+0000","last_fullsized":"2024-02-02T15:25:18.236406+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:35:21.015918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:52.001109+0000","last_change":"2024-02-02T15:24:47.784873+0000","last_active":"2024-02-02T15:24:52.001109+0000","last_peered":"2024-02-02T15:24:52.001109+0000","last_clean":"2024-02-02T15:24:52.001109+0000","last_became_active":"2024-02-02T15:24:47.784550+0000","last_became_peered":"2024-02-02T15:24:47.784550+0000","last_unstale":"2024-02-02T15:24:52.001109+0000","last_undegraded":"2024-02-02T15:24:52.001109+0000","last_fullsized":"2024-02-02T15:24:52.001109+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:13:37.513754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.946878+0000","last_change":"2024-02-02T15:24:51.805131+0000","last_active":"2024-02-02T15:24:51.946878+0000","last_peered":"2024-02-02T15:24:51.946878+0000","last_clean":"2024-02-02T15:24:51.946878+0000","last_became_active":"2024-02-02T15:24:51.804705+0000","last_became_peered":"2024-02-02T15:24:51.804705+0000","last_unstale":"2024-02-02T15:24:51.946878+0000","last_undegraded":"2024-02-02T15:24:51.946878+0000","last_fullsized":"2024-02-02T15:24:51.946878+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T02:12:59.945050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"84'7","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:25:19.240901+0000","last_change":"2024-02-02T15:24:47.266469+0000","last_active":"2024-02-02T15:25:19.240901+0000","last_peered":"2024-02-02T15:25:19.240901+0000","last_clean":"2024-02-02T15:25:19.240901+0000","last_became_active":"2024-02-02T15:24:47.266094+0000","last_became_peered":"2024-02-02T15:24:47.266094+0000","last_unstale":"2024-02-02T15:25:19.240901+0000","last_undegraded":"2024-02-02T15:25:19.240901+0000","last_fullsized":"2024-02-02T15:25:19.240901+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:51:41.324698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.808312+0000","last_change":"2024-02-02T15:24:47.793051+0000","last_active":"2024-02-02T15:24:51.808312+0000","last_peered":"2024-02-02T15:24:51.808312+0000","last_clean":"2024-02-02T15:24:51.808312+0000","last_became_active":"2024-02-02T15:24:47.792645+0000","last_became_peered":"2024-02-02T15:24:47.792645+0000","last_unstale":"2024-02-02T15:24:51.808312+0000","last_undegraded":"2024-02-02T15:24:51.808312+0000","last_fullsized":"2024-02-02T15:24:51.808312+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:47:17.229407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:24:53.094426+0000","last_change":"2024-02-02T15:24:51.821048+0000","last_active":"2024-02-02T15:24:53.094426+0000","last_peered":"2024-02-02T15:24:53.094426+0000","last_clean":"2024-02-02T15:24:53.094426+0000","last_became_active":"2024-02-02T15:24:51.820661+0000","last_became_peered":"2024-02-02T15:24:51.820661+0000","last_unstale":"2024-02-02T15:24:53.094426+0000","last_undegraded":"2024-02-02T15:24:53.094426+0000","last_fullsized":"2024-02-02T15:24:53.094426+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:45:52.401807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"84'2","reported_seq":18,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:25:16.219044+0000","last_change":"2024-02-02T15:24:47.273716+0000","last_active":"2024-02-02T15:25:16.219044+0000","last_peered":"2024-02-02T15:25:16.219044+0000","last_clean":"2024-02-02T15:25:16.219044+0000","last_became_active":"2024-02-02T15:24:47.273438+0000","last_became_peered":"2024-02-02T15:24:47.273438+0000","last_unstale":"2024-02-02T15:25:16.219044+0000","last_undegraded":"2024-02-02T15:25:16.219044+0000","last_fullsized":"2024-02-02T15:25:16.219044+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:52:16.498119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.798696+0000","last_change":"2024-02-02T15:24:47.782950+0000","last_active":"2024-02-02T15:24:51.798696+0000","last_peered":"2024-02-02T15:24:51.798696+0000","last_clean":"2024-02-02T15:24:51.798696+0000","last_became_active":"2024-02-02T15:24:47.782605+0000","last_became_peered":"2024-02-02T15:24:47.782605+0000","last_unstale":"2024-02-02T15:24:51.798696+0000","last_undegraded":"2024-02-02T15:24:51.798696+0000","last_fullsized":"2024-02-02T15:24:51.798696+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:57:34.125985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:24:53.094614+0000","last_change":"2024-02-02T15:24:51.822596+0000","last_active":"2024-02-02T15:24:53.094614+0000","last_peered":"2024-02-02T15:24:53.094614+0000","last_clean":"2024-02-02T15:24:53.094614+0000","last_became_active":"2024-02-02T15:24:51.822138+0000","last_became_peered":"2024-02-02T15:24:51.822138+0000","last_unstale":"2024-02-02T15:24:53.094614+0000","last_undegraded":"2024-02-02T15:24:53.094614+0000","last_fullsized":"2024-02-02T15:24:53.094614+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:03:11.339557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"84'3","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:25:16.217495+0000","last_change":"2024-02-02T15:24:47.278478+0000","last_active":"2024-02-02T15:25:16.217495+0000","last_peered":"2024-02-02T15:25:16.217495+0000","last_clean":"2024-02-02T15:25:16.217495+0000","last_became_active":"2024-02-02T15:24:47.278084+0000","last_became_peered":"2024-02-02T15:24:47.278084+0000","last_unstale":"2024-02-02T15:25:16.217495+0000","last_undegraded":"2024-02-02T15:25:16.217495+0000","last_fullsized":"2024-02-02T15:25:16.217495+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:10:23.044968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.798379+0000","last_change":"2024-02-02T15:24:47.795291+0000","last_active":"2024-02-02T15:24:51.798379+0000","last_peered":"2024-02-02T15:24:51.798379+0000","last_clean":"2024-02-02T15:24:51.798379+0000","last_became_active":"2024-02-02T15:24:47.794514+0000","last_became_peered":"2024-02-02T15:24:47.794514+0000","last_unstale":"2024-02-02T15:24:51.798379+0000","last_undegraded":"2024-02-02T15:24:51.798379+0000","last_fullsized":"2024-02-02T15:24:51.798379+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:32:12.722258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:24:53.093493+0000","last_change":"2024-02-02T15:24:51.825985+0000","last_active":"2024-02-02T15:24:53.093493+0000","last_peered":"2024-02-02T15:24:53.093493+0000","last_clean":"2024-02-02T15:24:53.093493+0000","last_became_active":"2024-02-02T15:24:51.825588+0000","last_became_peered":"2024-02-02T15:24:51.825588+0000","last_unstale":"2024-02-02T15:24:53.093493+0000","last_undegraded":"2024-02-02T15:24:53.093493+0000","last_fullsized":"2024-02-02T15:24:53.093493+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:32:39.910258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.794667+0000","last_change":"2024-02-02T15:24:47.272954+0000","last_active":"2024-02-02T15:24:51.794667+0000","last_peered":"2024-02-02T15:24:51.794667+0000","last_clean":"2024-02-02T15:24:51.794667+0000","last_became_active":"2024-02-02T15:24:47.272090+0000","last_became_peered":"2024-02-02T15:24:47.272090+0000","last_unstale":"2024-02-02T15:24:51.794667+0000","last_undegraded":"2024-02-02T15:24:51.794667+0000","last_fullsized":"2024-02-02T15:24:51.794667+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:01:53.602026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:52.001055+0000","last_change":"2024-02-02T15:24:47.784188+0000","last_active":"2024-02-02T15:24:52.001055+0000","last_peered":"2024-02-02T15:24:52.001055+0000","last_clean":"2024-02-02T15:24:52.001055+0000","last_became_active":"2024-02-02T15:24:47.783872+0000","last_became_peered":"2024-02-02T15:24:47.783872+0000","last_unstale":"2024-02-02T15:24:52.001055+0000","last_undegraded":"2024-02-02T15:24:52.001055+0000","last_fullsized":"2024-02-02T15:24:52.001055+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:45:45.213255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:24:53.093842+0000","last_change":"2024-02-02T15:24:51.815263+0000","last_active":"2024-02-02T15:24:53.093842+0000","last_peered":"2024-02-02T15:24:53.093842+0000","last_clean":"2024-02-02T15:24:53.093842+0000","last_became_active":"2024-02-02T15:24:51.814849+0000","last_became_peered":"2024-02-02T15:24:51.814849+0000","last_unstale":"2024-02-02T15:24:53.093842+0000","last_undegraded":"2024-02-02T15:24:53.093842+0000","last_fullsized":"2024-02-02T15:24:53.093842+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:05:32.987359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"84'4","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:25:16.220014+0000","last_change":"2024-02-02T15:24:47.270534+0000","last_active":"2024-02-02T15:25:16.220014+0000","last_peered":"2024-02-02T15:25:16.220014+0000","last_clean":"2024-02-02T15:25:16.220014+0000","last_became_active":"2024-02-02T15:24:47.270110+0000","last_became_peered":"2024-02-02T15:24:47.270110+0000","last_unstale":"2024-02-02T15:25:16.220014+0000","last_undegraded":"2024-02-02T15:25:16.220014+0000","last_fullsized":"2024-02-02T15:25:16.220014+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T02:59:52.297315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.801786+0000","last_change":"2024-02-02T15:24:47.791826+0000","last_active":"2024-02-02T15:24:51.801786+0000","last_peered":"2024-02-02T15:24:51.801786+0000","last_clean":"2024-02-02T15:24:51.801786+0000","last_became_active":"2024-02-02T15:24:47.791016+0000","last_became_peered":"2024-02-02T15:24:47.791016+0000","last_unstale":"2024-02-02T15:24:51.801786+0000","last_undegraded":"2024-02-02T15:24:51.801786+0000","last_fullsized":"2024-02-02T15:24:51.801786+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:47:14.296876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:24:53.093305+0000","last_change":"2024-02-02T15:24:51.806741+0000","last_active":"2024-02-02T15:24:53.093305+0000","last_peered":"2024-02-02T15:24:53.093305+0000","last_clean":"2024-02-02T15:24:53.093305+0000","last_became_active":"2024-02-02T15:24:51.806085+0000","last_became_peered":"2024-02-02T15:24:51.806085+0000","last_unstale":"2024-02-02T15:24:53.093305+0000","last_undegraded":"2024-02-02T15:24:53.093305+0000","last_fullsized":"2024-02-02T15:24:53.093305+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:53:39.944222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":16,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.795735+0000","last_change":"2024-02-02T15:24:47.278515+0000","last_active":"2024-02-02T15:24:51.795735+0000","last_peered":"2024-02-02T15:24:51.795735+0000","last_clean":"2024-02-02T15:24:51.795735+0000","last_became_active":"2024-02-02T15:24:47.278145+0000","last_became_peered":"2024-02-02T15:24:47.278145+0000","last_unstale":"2024-02-02T15:24:51.795735+0000","last_undegraded":"2024-02-02T15:24:51.795735+0000","last_fullsized":"2024-02-02T15:24:51.795735+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:48:55.118003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.800565+0000","last_change":"2024-02-02T15:24:47.791996+0000","last_active":"2024-02-02T15:24:51.800565+0000","last_peered":"2024-02-02T15:24:51.800565+0000","last_clean":"2024-02-02T15:24:51.800565+0000","last_became_active":"2024-02-02T15:24:47.791624+0000","last_became_peered":"2024-02-02T15:24:47.791624+0000","last_unstale":"2024-02-02T15:24:51.800565+0000","last_undegraded":"2024-02-02T15:24:51.800565+0000","last_fullsized":"2024-02-02T15:24:51.800565+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:33:27.963740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.827314+0000","last_change":"2024-02-02T15:24:51.826835+0000","last_active":"2024-02-02T15:24:51.827314+0000","last_peered":"2024-02-02T15:24:51.827314+0000","last_clean":"2024-02-02T15:24:51.827314+0000","last_became_active":"2024-02-02T15:24:51.826310+0000","last_became_peered":"2024-02-02T15:24:51.826310+0000","last_unstale":"2024-02-02T15:24:51.827314+0000","last_undegraded":"2024-02-02T15:24:51.827314+0000","last_fullsized":"2024-02-02T15:24:51.827314+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:48:16.687048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'3","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-02-02T15:25:19.241138+0000","last_change":"2024-02-02T15:24:47.268865+0000","last_active":"2024-02-02T15:25:19.241138+0000","last_peered":"2024-02-02T15:25:19.241138+0000","last_clean":"2024-02-02T15:25:19.241138+0000","last_became_active":"2024-02-02T15:24:47.268523+0000","last_became_peered":"2024-02-02T15:24:47.268523+0000","last_unstale":"2024-02-02T15:25:19.241138+0000","last_undegraded":"2024-02-02T15:25:19.241138+0000","last_fullsized":"2024-02-02T15:25:19.241138+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:45:16.957512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.799303+0000","last_change":"2024-02-02T15:24:47.792601+0000","last_active":"2024-02-02T15:24:51.799303+0000","last_peered":"2024-02-02T15:24:51.799303+0000","last_clean":"2024-02-02T15:24:51.799303+0000","last_became_active":"2024-02-02T15:24:47.792240+0000","last_became_peered":"2024-02-02T15:24:47.792240+0000","last_unstale":"2024-02-02T15:24:51.799303+0000","last_undegraded":"2024-02-02T15:24:51.799303+0000","last_fullsized":"2024-02-02T15:24:51.799303+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:51:25.205859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.806905+0000","last_change":"2024-02-02T15:24:51.806267+0000","last_active":"2024-02-02T15:24:51.806905+0000","last_peered":"2024-02-02T15:24:51.806905+0000","last_clean":"2024-02-02T15:24:51.806905+0000","last_became_active":"2024-02-02T15:24:51.805877+0000","last_became_peered":"2024-02-02T15:24:51.805877+0000","last_unstale":"2024-02-02T15:24:51.806905+0000","last_undegraded":"2024-02-02T15:24:51.806905+0000","last_fullsized":"2024-02-02T15:24:51.806905+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:00:47.747377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":18,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.794156+0000","last_change":"2024-02-02T15:24:47.271057+0000","last_active":"2024-02-02T15:24:51.794156+0000","last_peered":"2024-02-02T15:24:51.794156+0000","last_clean":"2024-02-02T15:24:51.794156+0000","last_became_active":"2024-02-02T15:24:47.270737+0000","last_became_peered":"2024-02-02T15:24:47.270737+0000","last_unstale":"2024-02-02T15:24:51.794156+0000","last_undegraded":"2024-02-02T15:24:51.794156+0000","last_fullsized":"2024-02-02T15:24:51.794156+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:17:00.212426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.799556+0000","last_change":"2024-02-02T15:24:47.794108+0000","last_active":"2024-02-02T15:24:51.799556+0000","last_peered":"2024-02-02T15:24:51.799556+0000","last_clean":"2024-02-02T15:24:51.799556+0000","last_became_active":"2024-02-02T15:24:47.793890+0000","last_became_peered":"2024-02-02T15:24:47.793890+0000","last_unstale":"2024-02-02T15:24:51.799556+0000","last_undegraded":"2024-02-02T15:24:51.799556+0000","last_fullsized":"2024-02-02T15:24:51.799556+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T03:05:51.704385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.798477+0000","last_change":"2024-02-02T15:24:47.796234+0000","last_active":"2024-02-02T15:24:51.798477+0000","last_peered":"2024-02-02T15:24:51.798477+0000","last_clean":"2024-02-02T15:24:51.798477+0000","last_became_active":"2024-02-02T15:24:47.795474+0000","last_became_peered":"2024-02-02T15:24:47.795474+0000","last_unstale":"2024-02-02T15:24:51.798477+0000","last_undegraded":"2024-02-02T15:24:51.798477+0000","last_fullsized":"2024-02-02T15:24:51.798477+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:25:28.059500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.946901+0000","last_change":"2024-02-02T15:24:47.786930+0000","last_active":"2024-02-02T15:24:51.946901+0000","last_peered":"2024-02-02T15:24:51.946901+0000","last_clean":"2024-02-02T15:24:51.946901+0000","last_became_active":"2024-02-02T15:24:47.786615+0000","last_became_peered":"2024-02-02T15:24:47.786615+0000","last_unstale":"2024-02-02T15:24:51.946901+0000","last_undegraded":"2024-02-02T15:24:51.946901+0000","last_fullsized":"2024-02-02T15:24:51.946901+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:17:23.231511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.946924+0000","last_change":"2024-02-02T15:24:47.797877+0000","last_active":"2024-02-02T15:24:51.946924+0000","last_peered":"2024-02-02T15:24:51.946924+0000","last_clean":"2024-02-02T15:24:51.946924+0000","last_became_active":"2024-02-02T15:24:47.797559+0000","last_became_peered":"2024-02-02T15:24:47.797559+0000","last_unstale":"2024-02-02T15:24:51.946924+0000","last_undegraded":"2024-02-02T15:24:51.946924+0000","last_fullsized":"2024-02-02T15:24:51.946924+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:13:55.529796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.804307+0000","last_change":"2024-02-02T15:24:47.790270+0000","last_active":"2024-02-02T15:24:51.804307+0000","last_peered":"2024-02-02T15:24:51.804307+0000","last_clean":"2024-02-02T15:24:51.804307+0000","last_became_active":"2024-02-02T15:24:47.789928+0000","last_became_peered":"2024-02-02T15:24:47.789928+0000","last_unstale":"2024-02-02T15:24:51.804307+0000","last_undegraded":"2024-02-02T15:24:51.804307+0000","last_fullsized":"2024-02-02T15:24:51.804307+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T02:34:44.885803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.794371+0000","last_change":"2024-02-02T15:24:47.786586+0000","last_active":"2024-02-02T15:24:51.794371+0000","last_peered":"2024-02-02T15:24:51.794371+0000","last_clean":"2024-02-02T15:24:51.794371+0000","last_became_active":"2024-02-02T15:24:47.785555+0000","last_became_peered":"2024-02-02T15:24:47.785555+0000","last_unstale":"2024-02-02T15:24:51.794371+0000","last_undegraded":"2024-02-02T15:24:51.794371+0000","last_fullsized":"2024-02-02T15:24:51.794371+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:37:47.537815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.799602+0000","last_change":"2024-02-02T15:24:47.797861+0000","last_active":"2024-02-02T15:24:51.799602+0000","last_peered":"2024-02-02T15:24:51.799602+0000","last_clean":"2024-02-02T15:24:51.799602+0000","last_became_active":"2024-02-02T15:24:47.797543+0000","last_became_peered":"2024-02-02T15:24:47.797543+0000","last_unstale":"2024-02-02T15:24:51.799602+0000","last_undegraded":"2024-02-02T15:24:51.799602+0000","last_fullsized":"2024-02-02T15:24:51.799602+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:09:37.710909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.947003+0000","last_change":"2024-02-02T15:24:47.796221+0000","last_active":"2024-02-02T15:24:51.947003+0000","last_peered":"2024-02-02T15:24:51.947003+0000","last_clean":"2024-02-02T15:24:51.947003+0000","last_became_active":"2024-02-02T15:24:47.795845+0000","last_became_peered":"2024-02-02T15:24:47.795845+0000","last_unstale":"2024-02-02T15:24:51.947003+0000","last_undegraded":"2024-02-02T15:24:51.947003+0000","last_fullsized":"2024-02-02T15:24:51.947003+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:54:55.637660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:52.001128+0000","last_change":"2024-02-02T15:24:47.783086+0000","last_active":"2024-02-02T15:24:52.001128+0000","last_peered":"2024-02-02T15:24:52.001128+0000","last_clean":"2024-02-02T15:24:52.001128+0000","last_became_active":"2024-02-02T15:24:47.781826+0000","last_became_peered":"2024-02-02T15:24:47.781826+0000","last_unstale":"2024-02-02T15:24:52.001128+0000","last_undegraded":"2024-02-02T15:24:52.001128+0000","last_fullsized":"2024-02-02T15:24:52.001128+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:32:05.623591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.799100+0000","last_change":"2024-02-02T15:24:47.789151+0000","last_active":"2024-02-02T15:24:51.799100+0000","last_peered":"2024-02-02T15:24:51.799100+0000","last_clean":"2024-02-02T15:24:51.799100+0000","last_became_active":"2024-02-02T15:24:47.785809+0000","last_became_peered":"2024-02-02T15:24:47.785809+0000","last_unstale":"2024-02-02T15:24:51.799100+0000","last_undegraded":"2024-02-02T15:24:51.799100+0000","last_fullsized":"2024-02-02T15:24:51.799100+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:18:16.013352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:52.001013+0000","last_change":"2024-02-02T15:24:47.789905+0000","last_active":"2024-02-02T15:24:52.001013+0000","last_peered":"2024-02-02T15:24:52.001013+0000","last_clean":"2024-02-02T15:24:52.001013+0000","last_became_active":"2024-02-02T15:24:47.788903+0000","last_became_peered":"2024-02-02T15:24:47.788903+0000","last_unstale":"2024-02-02T15:24:52.001013+0000","last_undegraded":"2024-02-02T15:24:52.001013+0000","last_fullsized":"2024-02-02T15:24:52.001013+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T02:49:46.118232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.795514+0000","last_change":"2024-02-02T15:24:47.795928+0000","last_active":"2024-02-02T15:24:51.795514+0000","last_peered":"2024-02-02T15:24:51.795514+0000","last_clean":"2024-02-02T15:24:51.795514+0000","last_became_active":"2024-02-02T15:24:47.795511+0000","last_became_peered":"2024-02-02T15:24:47.795511+0000","last_unstale":"2024-02-02T15:24:51.795514+0000","last_undegraded":"2024-02-02T15:24:51.795514+0000","last_fullsized":"2024-02-02T15:24:51.795514+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:55:31.444331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.794050+0000","last_change":"2024-02-02T15:24:47.794368+0000","last_active":"2024-02-02T15:24:51.794050+0000","last_peered":"2024-02-02T15:24:51.794050+0000","last_clean":"2024-02-02T15:24:51.794050+0000","last_became_active":"2024-02-02T15:24:47.793808+0000","last_became_peered":"2024-02-02T15:24:47.793808+0000","last_unstale":"2024-02-02T15:24:51.794050+0000","last_undegraded":"2024-02-02T15:24:51.794050+0000","last_fullsized":"2024-02-02T15:24:51.794050+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T02:39:00.099808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":15,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-02-02T15:24:51.799168+0000","last_change":"2024-02-02T15:24:47.796358+0000","last_active":"2024-02-02T15:24:51.799168+0000","last_peered":"2024-02-02T15:24:51.799168+0000","last_clean":"2024-02-02T15:24:51.799168+0000","last_became_active":"2024-02-02T15:24:47.795800+0000","last_became_peered":"2024-02-02T15:24:47.795800+0000","last_unstale":"2024-02-02T15:24:51.799168+0000","last_undegraded":"2024-02-02T15:24:51.799168+0000","last_fullsized":"2024-02-02T15:24:51.799168+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:49:40.019787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":16384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":250199,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":43200,"num_objects":74,"num_object_clones":0,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":0,"num_read":16,"num_read_kb":45,"num_write":173,"num_write_kb":92,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":49,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":348160,"data_stored":61788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152,"ondisk_log_size":152,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547217,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":925696,"data_stored":459202,"data_compressed":11732,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 15:25:12 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.63900000000000001}]},{"osd":1,"last update":"Fri Feb 2 15:25:12 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.59999999999999998}]},{"osd":2,"last update":"Fri Feb 2 15:25:12 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.47899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":4,"last update":"Fri Feb 2 15:25:12 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.69799999999999995}]},{"osd":5,"last update":"Fri Feb 2 15:25:12 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.625}]},{"osd":6,"last update":"Fri Feb 2 15:25:12 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.80900000000000005}]},{"osd":7,"last update":"Fri Feb 2 15:25:12 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.75},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.76200000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]},{"osd":9,"last update":"Fri Feb 2 15:25:12 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.78700000000000003}]},{"osd":10,"last update":"Fri Feb 2 15:25:12 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.77400000000000002}]}]},{"osd":10,"up_from":67,"seq":287762808851,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1052672,"data_stored":1084928,"data_compressed":47943,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":1,"last update":"Fri Feb 2 15:24:51 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.443,"15min":0.443},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.435}]},{"osd":2,"last update":"Fri Feb 2 15:24:51 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.51100000000000001}]},{"osd":3,"last update":"Fri Feb 2 15:24:51 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":4.4109999999999996,"5min":4.4109999999999996,"15min":4.4109999999999996},"last":0.55000000000000004}]},{"osd":4,"last update":"Fri Feb 2 15:24:51 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.57699999999999996}]},{"osd":5,"last update":"Fri Feb 2 15:24:51 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.69299999999999995}]},{"osd":6,"last update":"Fri Feb 2 15:24:51 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":4.4459999999999997,"5min":4.4459999999999997,"15min":4.4459999999999997},"last":0.746},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.3740000000000001,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.68400000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40300000000000002}]},{"osd":8,"last update":"Fri Feb 2 15:24:51 2024","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.70699999999999996}]},{"osd":9,"last update":"Fri Feb 2 15:24:51 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":4.4269999999999996,"5min":4.4269999999999996,"15min":4.4269999999999996},"last":0.73399999999999999}]},{"osd":11,"last update":"Fri Feb 2 15:25:11 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.5680000000000001,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.76700000000000002}]}]},{"osd":9,"up_from":62,"seq":266287972375,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":933888,"data_stored":454870,"data_compressed":11732,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 15:24:35 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.50900000000000001}]},{"osd":1,"last update":"Fri Feb 2 15:24:35 2024","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.752},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":0.64700000000000002}]},{"osd":2,"last update":"Fri Feb 2 15:24:35 2024","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.71399999999999997}]},{"osd":3,"last update":"Fri Feb 2 15:24:35 2024","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.296,"5min":1.296,"15min":1.296},"last":0.56499999999999995}]},{"osd":4,"last update":"Fri Feb 2 15:24:35 2024","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.381,"5min":1.381,"15min":1.381},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.70599999999999996}]},{"osd":5,"last update":"Fri Feb 2 15:24:35 2024","interfaces":[{"interface":"back","average":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":3.895,"5min":3.895,"15min":3.895},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.373,"5min":1.373,"15min":1.373},"last":0.78900000000000003}]},{"osd":6,"last update":"Fri Feb 2 15:24:35 2024","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.6739999999999999,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":0.64500000000000002}]},{"osd":7,"last update":"Fri Feb 2 15:24:35 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.77600000000000002}]},{"osd":8,"last update":"Fri Feb 2 15:24:35 2024","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.7789999999999999,"5min":1.7789999999999999,"15min":1.7789999999999999},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":0.84799999999999998}]},{"osd":10,"last update":"Fri Feb 2 15:24:56 2024","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.87},{"interface":"front","average":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"last":0.83999999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86299999999999999}]}]},{"osd":8,"up_from":55,"seq":236223201307,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":925696,"data_stored":454369,"data_compressed":11732,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 15:25:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.58199999999999996,"15min":0.58299999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.73699999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.59399999999999997,"15min":0.60099999999999998},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.77000000000000002,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.55700000000000005}]},{"osd":1,"last update":"Fri Feb 2 15:25:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.56899999999999995,"15min":0.56499999999999995},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.57399999999999995,"15min":0.56399999999999995},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.79500000000000004}]},{"osd":2,"last update":"Fri Feb 2 15:25:16 2024","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.42099999999999999,"15min":0.41699999999999998},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.63400000000000001,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.627},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.42299999999999999,"15min":0.41699999999999998},"min":{"1min":0.307,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.61299999999999999,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.72299999999999998}]},{"osd":3,"last update":"Fri Feb 2 15:25:16 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.621,"15min":0.627},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.76400000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.61699999999999999,"15min":0.621},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.79500000000000004,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.875}]},{"osd":4,"last update":"Fri Feb 2 15:25:16 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.60299999999999998,"15min":0.59899999999999998},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.62,"5min":0.59499999999999997,"15min":0.59099999999999997},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.93300000000000005}]},{"osd":5,"last update":"Fri Feb 2 15:25:16 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.51000000000000001,"15min":0.503},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.50800000000000001,"15min":0.502},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.84799999999999998}]},{"osd":6,"last update":"Fri Feb 2 15:25:16 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.67200000000000004,"15min":0.67500000000000004},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.83299999999999996,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.97199999999999998},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.69199999999999995,"15min":0.69299999999999995},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.80900000000000005,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.86799999999999999}]},{"osd":7,"last update":"Fri Feb 2 15:25:16 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.64700000000000002,"15min":0.64100000000000001},"min":{"1min":0.49199999999999999,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.67400000000000004,"15min":0.67000000000000004},"min":{"1min":0.45800000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.88700000000000001,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.89600000000000002}]},{"osd":9,"last update":"Fri Feb 2 15:24:36 2024","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.996}]},{"osd":10,"last update":"Fri Feb 2 15:24:53 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":1.0169999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98799999999999999}]}]},{"osd":7,"up_from":48,"seq":206158430239,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":1051816,"data_compressed":47943,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 15:23:58 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.439},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.38500000000000001}]},{"osd":1,"last update":"Fri Feb 2 15:24:59 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.42799999999999999,"15min":0.42599999999999999},"min":{"1min":0.28100000000000003,"5min":0.247,"15min":0.247},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.442},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.40899999999999997,"15min":0.40699999999999997},"min":{"1min":0.29399999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.64500000000000002,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.46400000000000002}]},{"osd":2,"last update":"Fri Feb 2 15:24:59 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.79100000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.67600000000000005}]},{"osd":3,"last update":"Fri Feb 2 15:24:59 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.60899999999999999,"15min":0.59999999999999998},"min":{"1min":0.41099999999999998,"5min":0.374,"15min":0.374},"max":{"1min":2.746,"5min":2.746,"15min":2.746},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.65500000000000003}]},{"osd":4,"last update":"Fri Feb 2 15:24:59 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.502,"15min":0.5},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.5,"15min":0.496},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.49199999999999999}]},{"osd":5,"last update":"Fri Feb 2 15:24:59 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.63,"15min":0.61899999999999999},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":2.3370000000000002,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.63,"5min":0.623,"15min":0.622},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.83999999999999997}]},{"osd":6,"last update":"Fri Feb 2 15:24:59 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.68999999999999995,"15min":0.69199999999999995},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.67700000000000005,"15min":0.66800000000000004},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.778,"5min":1.778,"15min":1.778},"last":0.61299999999999999}]},{"osd":8,"last update":"Fri Feb 2 15:25:16 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.70499999999999996,"15min":0.70099999999999996},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":2.069,"5min":2.069,"15min":2.069},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.72899999999999998,"15min":0.73299999999999998},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.995,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.79500000000000004}]},{"osd":9,"last update":"Fri Feb 2 15:24:37 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.70699999999999996}]},{"osd":10,"last update":"Fri Feb 2 15:24:51 2024","interfaces":[{"interface":"back","average":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.64400000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81599999999999995}]}]},{"osd":6,"up_from":43,"seq":184683593762,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":913408,"data_stored":454052,"data_compressed":11732,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 15:24:43 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.48899999999999999,"15min":0.48199999999999998},"min":{"1min":0.33800000000000002,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.51800000000000002,"15min":0.51200000000000001},"min":{"1min":0.39200000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.371}]},{"osd":1,"last update":"Fri Feb 2 15:24:43 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.63900000000000001,"15min":0.63600000000000001},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.91000000000000003,"5min":0.91000000000000003,"15min":0.91000000000000003},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.65300000000000002,"15min":0.64900000000000002},"min":{"1min":0.54600000000000004,"5min":0.501,"15min":0.501},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.88900000000000001}]},{"osd":2,"last update":"Fri Feb 2 15:24:43 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.69999999999999996,"15min":0.69399999999999995},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.97599999999999998,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.71499999999999997,"15min":0.71199999999999997},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.0409999999999999,"5min":1.151,"15min":1.151},"last":0.80200000000000005}]},{"osd":3,"last update":"Fri Feb 2 15:24:43 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.63900000000000001,"15min":0.63700000000000001},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.432},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.66900000000000004,"15min":0.66300000000000003},"min":{"1min":0.41799999999999998,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.99099999999999999}]},{"osd":4,"last update":"Fri Feb 2 15:24:43 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.71899999999999997,"15min":0.71699999999999997},"min":{"1min":0.50600000000000001,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.046,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.76000000000000001,"15min":0.755},"min":{"1min":0.53700000000000003,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.2250000000000001,"5min":1.228,"15min":1.228},"last":0.64600000000000002}]},{"osd":5,"last update":"Fri Feb 2 15:24:43 2024","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.78100000000000003,"15min":0.77600000000000002},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.81799999999999995,"15min":0.81399999999999995},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":1.202,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.85499999999999998}]},{"osd":7,"last update":"Fri Feb 2 15:24:57 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.83199999999999996,"15min":0.83099999999999996},"min":{"1min":0.59199999999999997,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.073,"5min":1.256,"15min":1.256},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.876,"15min":0.878},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.103,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.747}]},{"osd":8,"last update":"Fri Feb 2 15:25:14 2024","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.90000000000000002,"15min":0.90100000000000002},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.212,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.91700000000000004,"15min":0.91300000000000003},"min":{"1min":0.57699999999999996,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.78100000000000003}]},{"osd":9,"last update":"Fri Feb 2 15:24:37 2024","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.405,"5min":1.405,"15min":1.405},"last":1.018},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.4870000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.874}]},{"osd":10,"last update":"Fri Feb 2 15:24:53 2024","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.93700000000000006}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97699999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574900,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":921600,"data_stored":457757,"data_compressed":11732,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 15:25:12 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.51100000000000001,"15min":0.50800000000000001},"min":{"1min":0.39200000000000002,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.52500000000000002,"15min":0.52100000000000002},"min":{"1min":0.40100000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.68999999999999995}]},{"osd":2,"last update":"Fri Feb 2 15:24:30 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.53400000000000003,"15min":0.54000000000000004},"min":{"1min":0.34399999999999997,"5min":0.31,"15min":0.31},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.53900000000000003,"15min":0.53800000000000003},"min":{"1min":0.379,"5min":0.315,"15min":0.315},"max":{"1min":0.79700000000000004,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.65300000000000002}]},{"osd":3,"last update":"Fri Feb 2 15:24:50 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.54100000000000004,"15min":0.52700000000000002},"min":{"1min":0.39800000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.747,"5min":0.748,"15min":0.748},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.55000000000000004,"15min":0.54400000000000004},"min":{"1min":0.48599999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.80100000000000005}]},{"osd":4,"last update":"Fri Feb 2 15:25:04 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.39600000000000002,"15min":0.36899999999999999},"min":{"1min":0.32900000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.66800000000000004,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.40000000000000002,"15min":0.375},"min":{"1min":0.27200000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.623}]},{"osd":5,"last update":"Fri Feb 2 15:24:19 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.53600000000000003,"15min":0.53300000000000003},"min":{"1min":0.40699999999999997,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.55500000000000005,"15min":0.54600000000000004},"min":{"1min":0.41399999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.79000000000000004}]},{"osd":6,"last update":"Fri Feb 2 15:24:44 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.59399999999999997,"15min":0.58599999999999997},"min":{"1min":0.49099999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.752,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.751},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.58499999999999996,"15min":0.58099999999999996},"min":{"1min":0.503,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.70499999999999996}]},{"osd":7,"last update":"Fri Feb 2 15:25:00 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.81100000000000005,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.53700000000000003,"15min":0.53300000000000003},"min":{"1min":0.371,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.72699999999999998,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.77900000000000003}]},{"osd":8,"last update":"Fri Feb 2 15:25:14 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.63400000000000001,"15min":0.63},"min":{"1min":0.434,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.65700000000000003,"15min":0.64700000000000002},"min":{"1min":0.51100000000000001,"5min":0.442,"15min":0.442},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.87}]},{"osd":9,"last update":"Fri Feb 2 15:24:36 2024","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.88},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.90600000000000003}]},{"osd":10,"last update":"Fri Feb 2 15:24:51 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.85599999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93000000000000005}]}]},{"osd":0,"up_from":8,"seq":34359738424,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":456092,"data_compressed":11732,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Feb 2 15:25:10 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.55800000000000005,"15min":0.54400000000000004},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.90400000000000003,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.628,"5min":0.57399999999999995,"15min":0.54500000000000004},"min":{"1min":0.35799999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.89800000000000002}]},{"osd":2,"last update":"Fri Feb 2 15:24:29 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.60699999999999998,"15min":0.56699999999999995},"min":{"1min":0.48999999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.872,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.67500000000000004,"15min":0.60499999999999998},"min":{"1min":0.45700000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":4.6550000000000002,"5min":4.6550000000000002,"15min":4.6550000000000002},"last":0.51100000000000001}]},{"osd":3,"last update":"Fri Feb 2 15:23:45 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.55100000000000005,"15min":0.53700000000000003},"min":{"1min":0.33800000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.57599999999999996,"15min":0.56599999999999995},"min":{"1min":0.46300000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.39500000000000002}]},{"osd":4,"last update":"Fri Feb 2 15:25:05 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.65400000000000003,"15min":0.63700000000000001},"min":{"1min":0.433,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.091,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":1.02},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.64200000000000002,"15min":0.626},"min":{"1min":0.46700000000000003,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.1279999999999999,"5min":1.141,"15min":1.141},"last":0.65700000000000003}]},{"osd":5,"last update":"Fri Feb 2 15:25:21 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68200000000000005,"15min":0.68300000000000005},"min":{"1min":0.50700000000000001,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.024,"5min":1.042,"15min":1.042},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.70999999999999996,"15min":0.67600000000000005},"min":{"1min":0.42099999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.054,"5min":4.5890000000000004,"15min":4.5890000000000004},"last":0.58099999999999996}]},{"osd":6,"last update":"Fri Feb 2 15:24:43 2024","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.70999999999999996,"15min":0.69899999999999995},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.73499999999999999,"15min":0.72899999999999998},"min":{"1min":0.46600000000000003,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.72799999999999998}]},{"osd":7,"last update":"Fri Feb 2 15:23:55 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.86099999999999999}]},{"osd":8,"last update":"Fri Feb 2 15:25:21 2024","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.77700000000000002,"15min":0.76900000000000002},"min":{"1min":0.56899999999999995,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.70699999999999996}]},{"osd":9,"last update":"Fri Feb 2 15:24:36 2024","interfaces":[{"interface":"back","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.97299999999999998},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.76000000000000001}]},{"osd":10,"last update":"Fri Feb 2 15:24:56 2024","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":1.089}]},{"osd":11,"last update":"Fri Feb 2 15:25:10 2024","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":0.753},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":1.1499999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411377,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":933888,"data_stored":458365,"data_compressed":11732,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 15:24:24 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.54500000000000004,"15min":0.54300000000000004},"min":{"1min":0.36499999999999999,"5min":0.253,"15min":0.253},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.52600000000000002,"15min":0.52000000000000002},"min":{"1min":0.32600000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.628}]},{"osd":1,"last update":"Fri Feb 2 15:24:24 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.503,"15min":0.497},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.53600000000000003,"15min":0.52400000000000002},"min":{"1min":0.34599999999999997,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.74399999999999999}]},{"osd":3,"last update":"Fri Feb 2 15:24:45 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.58599999999999997,"15min":0.56699999999999995},"min":{"1min":0.34699999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.623,"5min":0.57199999999999995,"15min":0.55000000000000004},"min":{"1min":0.441,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.90600000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.81000000000000005}]},{"osd":4,"last update":"Fri Feb 2 15:23:59 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.51000000000000001,"15min":0.504},"min":{"1min":0.39900000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.52700000000000002,"15min":0.51400000000000001},"min":{"1min":0.45900000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.75600000000000001}]},{"osd":5,"last update":"Fri Feb 2 15:24:21 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.41699999999999998,"15min":0.40500000000000003},"min":{"1min":0.32200000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.41299999999999998,"15min":0.40400000000000003},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.42999999999999999}]},{"osd":6,"last update":"Fri Feb 2 15:24:39 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.61899999999999999,"15min":0.61699999999999999},"min":{"1min":0.41899999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.626,"15min":0.628},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.67700000000000005}]},{"osd":7,"last update":"Fri Feb 2 15:23:57 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.79700000000000004}]},{"osd":8,"last update":"Fri Feb 2 15:25:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.41899999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.84199999999999997,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.40699999999999997,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.70799999999999996,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.47999999999999998}]},{"osd":9,"last update":"Fri Feb 2 15:24:35 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.70299999999999996}]},{"osd":10,"last update":"Fri Feb 2 15:24:51 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.68899999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77200000000000002}]}]},{"osd":3,"up_from":24,"seq":103079215148,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":921600,"data_stored":454156,"data_compressed":11732,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 15:24:44 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.42899999999999999,"15min":0.41499999999999998},"min":{"1min":0.33000000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.79000000000000004,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.45700000000000002,"15min":0.45000000000000001},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.78300000000000003,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.53800000000000003}]},{"osd":1,"last update":"Fri Feb 2 15:24:44 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.60999999999999999,"15min":0.58199999999999996},"min":{"1min":0.40100000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.60199999999999998,"15min":0.59199999999999997},"min":{"1min":0.44600000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.73299999999999998}]},{"osd":2,"last update":"Fri Feb 2 15:24:44 2024","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.623,"15min":0.61899999999999999},"min":{"1min":0.44500000000000001,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.81899999999999995,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.67500000000000004,"15min":0.64800000000000002},"min":{"1min":0.47699999999999998,"5min":0.443,"15min":0.443},"max":{"1min":1.016,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":0.56799999999999995}]},{"osd":4,"last update":"Fri Feb 2 15:24:59 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.625},"min":{"1min":0.40400000000000003,"5min":0.376,"15min":0.376},"max":{"1min":0.82099999999999995,"5min":1.022,"15min":1.022},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.70999999999999996,"15min":0.69199999999999995},"min":{"1min":0.61499999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0329999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.81100000000000005}]},{"osd":5,"last update":"Fri Feb 2 15:24:21 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.72299999999999998,"15min":0.71699999999999997},"min":{"1min":0.55000000000000004,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0780000000000001,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.64500000000000002,"15min":0.64100000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.48099999999999998}]},{"osd":6,"last update":"Fri Feb 2 15:24:44 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.72499999999999998,"15min":0.70999999999999996},"min":{"1min":0.51800000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.69599999999999995,"15min":0.68999999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.51700000000000002}]},{"osd":7,"last update":"Fri Feb 2 15:24:59 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.75,"15min":0.74099999999999999},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.73599999999999999,"15min":0.72999999999999998},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.97299999999999998,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.61799999999999999}]},{"osd":8,"last update":"Fri Feb 2 15:24:14 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.65800000000000003}]},{"osd":9,"last update":"Fri Feb 2 15:24:34 2024","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.60199999999999998}]},{"osd":10,"last update":"Fri Feb 2 15:24:54 2024","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.83599999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74299999999999999}]}]},{"osd":4,"up_from":31,"seq":133143986219,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":929792,"data_stored":455006,"data_compressed":11732,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 15:24:03 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.52300000000000002,"15min":0.52100000000000002},"min":{"1min":0.371,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.78600000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.53800000000000003,"15min":0.53600000000000003},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.47799999999999998}]},{"osd":1,"last update":"Fri Feb 2 15:25:05 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.35299999999999998,"15min":0.33500000000000002},"min":{"1min":0.26600000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.432,"5min":0.36499999999999999,"15min":0.34699999999999998},"min":{"1min":0.28299999999999997,"5min":0.255,"15min":0.255},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.434}]},{"osd":2,"last update":"Fri Feb 2 15:25:05 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.52400000000000002,"15min":0.50700000000000001},"min":{"1min":0.35699999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.52800000000000002,"15min":0.51100000000000001},"min":{"1min":0.40699999999999997,"5min":0.38,"15min":0.38},"max":{"1min":0.753,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.56599999999999995}]},{"osd":3,"last update":"Fri Feb 2 15:25:05 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.56599999999999995,"15min":0.56399999999999995},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.55800000000000005,"15min":0.56000000000000005},"min":{"1min":0.38400000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.70899999999999996,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.59299999999999997}]},{"osd":5,"last update":"Fri Feb 2 15:24:19 2024","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.58899999999999997,"15min":0.58199999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.60299999999999998,"15min":0.60099999999999998},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.61099999999999999}]},{"osd":6,"last update":"Fri Feb 2 15:24:40 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.64300000000000002,"15min":0.63400000000000001},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.65200000000000002,"15min":0.64100000000000001},"min":{"1min":0.45700000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.335,"5min":1.335,"15min":1.335},"last":0.747}]},{"osd":7,"last update":"Fri Feb 2 15:24:59 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.49099999999999999,"15min":0.48299999999999998},"min":{"1min":0.34000000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.53600000000000003,"15min":0.53200000000000003},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.61899999999999999}]},{"osd":8,"last update":"Fri Feb 2 15:25:16 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.65200000000000002,"15min":0.65000000000000002},"min":{"1min":0.42499999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.84299999999999997,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.65600000000000003,"15min":0.65100000000000002},"min":{"1min":0.46999999999999997,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.76700000000000002}]},{"osd":9,"last update":"Fri Feb 2 15:24:35 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.69499999999999995}]},{"osd":10,"last update":"Fri Feb 2 15:24:53 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.64100000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80800000000000005}]}]},{"osd":5,"up_from":36,"seq":154618822694,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":1051198,"data_compressed":47943,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 15:25:20 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.53000000000000003,"15min":0.51500000000000001},"min":{"1min":0.38600000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.53900000000000003,"15min":0.52600000000000002},"min":{"1min":0.39600000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.83599999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.58299999999999996}]},{"osd":1,"last update":"Fri Feb 2 15:25:20 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.56000000000000005,"15min":0.55000000000000004},"min":{"1min":0.46500000000000002,"5min":0.252,"15min":0.252},"max":{"1min":0.69999999999999996,"5min":1.018,"15min":1.018},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.621,"5min":0.56499999999999995,"15min":0.54700000000000004},"min":{"1min":0.46400000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.77500000000000002,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.77500000000000002}]},{"osd":2,"last update":"Fri Feb 2 15:25:20 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.39500000000000002,"15min":0.38300000000000001},"min":{"1min":0.312,"5min":0.215,"15min":0.215},"max":{"1min":0.59499999999999997,"5min":0.63,"15min":0.63},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.40999999999999998,"15min":0.40000000000000002},"min":{"1min":0.318,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.72699999999999998,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.55600000000000005}]},{"osd":3,"last update":"Fri Feb 2 15:25:20 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.54700000000000004,"15min":0.52300000000000002},"min":{"1min":0.378,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.80600000000000005,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.58299999999999996,"15min":0.56599999999999995},"min":{"1min":0.52200000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.64500000000000002}]},{"osd":4,"last update":"Fri Feb 2 15:25:20 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.57499999999999996,"15min":0.56200000000000006},"min":{"1min":0.45700000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.79600000000000004,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.58699999999999997,"15min":0.57599999999999996},"min":{"1min":0.47099999999999997,"5min":0.316,"15min":0.316},"max":{"1min":0.81899999999999995,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.73699999999999999}]},{"osd":6,"last update":"Fri Feb 2 15:24:41 2024","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.625,"15min":0.629},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.82299999999999995,"5min":0.878,"15min":0.878},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.62,"5min":0.63,"15min":0.63200000000000001},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.86899999999999999,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.67300000000000004}]},{"osd":7,"last update":"Fri Feb 2 15:25:00 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.622,"15min":0.62},"min":{"1min":0.46400000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.629,"5min":0.60799999999999998,"15min":0.60399999999999998},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.79200000000000004,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.72799999999999998}]},{"osd":8,"last update":"Fri Feb 2 15:25:14 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.55100000000000005,"15min":0.55400000000000005},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.83999999999999997,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.58699999999999997,"15min":0.59199999999999997},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.80100000000000005,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.83399999999999996}]},{"osd":9,"last update":"Fri Feb 2 15:24:36 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.86899999999999999},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.80800000000000005}]},{"osd":10,"last update":"Fri Feb 2 15:24:53 2024","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.76600000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34572,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34572,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36211,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36211,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36211,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36211,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36211,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":4377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":40598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-02T15:25:22.670 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-02-02T15:25:22.671 INFO:tasks.cephadm:Setting up client nodes... 2024-02-02T15:25:22.671 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-02-02T15:25:22.672 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:22 smithi142 ceph-mon[45728]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:25:22.672 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:22 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/1010496905' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:25:22.672 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:22 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/2979015657' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-02T15:25:23.071 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.a/config 2024-02-02T15:25:23.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:23 smithi071 ceph-mon[44707]: pgmap v252: 129 pgs: 129 active+clean; 619 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6.3 KiB/s wr, 11 op/s 2024-02-02T15:25:23.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:23 smithi071 ceph-mon[44707]: from='client.24836 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-02T15:25:23.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:23 smithi071 ceph-mon[44707]: from='client.14823 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-02T15:25:23.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:23 smithi142 ceph-mon[45728]: pgmap v252: 129 pgs: 129 active+clean; 619 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6.3 KiB/s wr, 11 op/s 2024-02-02T15:25:23.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:23 smithi142 ceph-mon[45728]: from='client.24836 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-02T15:25:23.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:23 smithi142 ceph-mon[45728]: from='client.14823 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-02T15:25:23.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:23 smithi001 ceph-mon[35470]: pgmap v252: 129 pgs: 129 active+clean; 619 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6.3 KiB/s wr, 11 op/s 2024-02-02T15:25:23.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:23 smithi001 ceph-mon[35470]: from='client.24836 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-02T15:25:23.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:23 smithi001 ceph-mon[35470]: from='client.14823 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-02T15:25:24.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:24.435+0000 7f0e66600700 1 -- 172.21.15.1:0/1339221777 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e60100e00 msgr2=0x7f0e60101220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:24.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:24.435+0000 7f0e66600700 1 --2- 172.21.15.1:0/1339221777 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e60100e00 0x7f0e60101220 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0e50009990 tx=0x7f0e50009ca0 comp rx=0 tx=0).stop 2024-02-02T15:25:24.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:24.435+0000 7f0e66600700 1 -- 172.21.15.1:0/1339221777 shutdown_connections 2024-02-02T15:25:24.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:24.435+0000 7f0e66600700 1 --2- 172.21.15.1:0/1339221777 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e60102920 0x7f0e60109240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:24.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:24.435+0000 7f0e66600700 1 --2- 172.21.15.1:0/1339221777 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e60100e00 0x7f0e60101220 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:24.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:24.435+0000 7f0e66600700 1 --2- 172.21.15.1:0/1339221777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e60101f60 0x7f0e601023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:24.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:24.435+0000 7f0e66600700 1 -- 172.21.15.1:0/1339221777 >> 172.21.15.1:0/1339221777 conn(0x7f0e600fc3c0 msgr2=0x7f0e600fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:24.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:24.435+0000 7f0e66600700 1 -- 172.21.15.1:0/1339221777 shutdown_connections 2024-02-02T15:25:24.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:24.435+0000 7f0e66600700 1 -- 172.21.15.1:0/1339221777 wait complete. 2024-02-02T15:25:24.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:24.436+0000 7f0e66600700 1 Processor -- start 2024-02-02T15:25:24.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:24.436+0000 7f0e66600700 1 -- start start 2024-02-02T15:25:24.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:24.437+0000 7f0e66600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e60100e00 0x7f0e60074ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:24.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:24.437+0000 7f0e66600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e60101f60 0x7f0e60073130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:24.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:24.437+0000 7f0e66600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e60102920 0x7f0e60073670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:24.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:24.437+0000 7f0e66600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e60073df0 con 0x7f0e60100e00 2024-02-02T15:25:24.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:24.437+0000 7f0e66600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e60073f60 con 0x7f0e60101f60 2024-02-02T15:25:24.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:24.437+0000 7f0e66600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e601a0b40 con 0x7f0e60102920 2024-02-02T15:25:24.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:24.437+0000 7f0e5f400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e60100e00 0x7f0e60074ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:24.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:24.437+0000 7f0e5f400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e60100e00 0x7f0e60074ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33468/0 (socket says 172.21.15.1:33468) 2024-02-02T15:25:24.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:24.437+0000 7f0e5f400700 1 -- 172.21.15.1:0/1695393882 learned_addr learned my addr 172.21.15.1:0/1695393882 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:25:24.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:24.437+0000 7f0e5ea00700 1 --2- 172.21.15.1:0/1695393882 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e60101f60 0x7f0e60073130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:24.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:24.437+0000 7f0e5fe00700 1 --2- 172.21.15.1:0/1695393882 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e60102920 0x7f0e60073670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:24.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:24.438+0000 7f0e5ea00700 1 -- 172.21.15.1:0/1695393882 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e60102920 msgr2=0x7f0e60073670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:24.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:24.438+0000 7f0e5ea00700 1 --2- 172.21.15.1:0/1695393882 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e60102920 0x7f0e60073670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:24.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:24.438+0000 7f0e5ea00700 1 -- 172.21.15.1:0/1695393882 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e60100e00 msgr2=0x7f0e60074ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:24.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:24.438+0000 7f0e5ea00700 1 --2- 172.21.15.1:0/1695393882 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e60100e00 0x7f0e60074ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:24.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:24.438+0000 7f0e5ea00700 1 -- 172.21.15.1:0/1695393882 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e500095f0 con 0x7f0e60101f60 2024-02-02T15:25:24.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:24.438+0000 7f0e5f400700 1 --2- 172.21.15.1:0/1695393882 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e60100e00 0x7f0e60074ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:25:24.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:24.438+0000 7f0e5ea00700 1 --2- 172.21.15.1:0/1695393882 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e60101f60 0x7f0e60073130 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0e4800e9f0 tx=0x7f0e4800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:24.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:24.438+0000 7f0e57e00700 1 -- 172.21.15.1:0/1695393882 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e4800c9c0 con 0x7f0e60101f60 2024-02-02T15:25:24.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:24.438+0000 7f0e57e00700 1 -- 172.21.15.1:0/1695393882 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e48010430 con 0x7f0e60101f60 2024-02-02T15:25:24.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:24.438+0000 7f0e57e00700 1 -- 172.21.15.1:0/1695393882 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e48018680 con 0x7f0e60101f60 2024-02-02T15:25:24.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:24.438+0000 7f0e66600700 1 -- 172.21.15.1:0/1695393882 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e601a0e20 con 0x7f0e60101f60 2024-02-02T15:25:24.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:24.438+0000 7f0e66600700 1 -- 172.21.15.1:0/1695393882 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e601a1230 con 0x7f0e60101f60 2024-02-02T15:25:24.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:24.439+0000 7f0e66600700 1 -- 172.21.15.1:0/1695393882 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e6010c6d0 con 0x7f0e60101f60 2024-02-02T15:25:24.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:24 smithi001 ceph-mon[35470]: pgmap v253: 129 pgs: 129 active+clean; 621 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 7.3 KiB/s wr, 17 op/s 2024-02-02T15:25:24.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:24.441+0000 7f0e57e00700 1 -- 172.21.15.1:0/1695393882 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 135120+0+0 (secure 0 0 0) 0x7f0e48010860 con 0x7f0e60101f60 2024-02-02T15:25:24.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:24.443+0000 7f0e57e00700 1 --2- 172.21.15.1:0/1695393882 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f0e400a05b0 0x7f0e400a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:24.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:24.443+0000 7f0e57e00700 1 -- 172.21.15.1:0/1695393882 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f0e48014070 con 0x7f0e60101f60 2024-02-02T15:25:24.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:24.443+0000 7f0e5f400700 1 --2- 172.21.15.1:0/1695393882 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f0e400a05b0 0x7f0e400a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:24.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:24.444+0000 7f0e5f400700 1 --2- 172.21.15.1:0/1695393882 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f0e400a05b0 0x7f0e400a2a70 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0e50006010 tx=0x7f0e5000b260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:24.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:24.448+0000 7f0e57e00700 1 -- 172.21.15.1:0/1695393882 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f0e48052110 con 0x7f0e60101f60 2024-02-02T15:25:24.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:24 smithi071 ceph-mon[44707]: pgmap v253: 129 pgs: 129 active+clean; 621 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 7.3 KiB/s wr, 17 op/s 2024-02-02T15:25:24.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:24 smithi142 ceph-mon[45728]: pgmap v253: 129 pgs: 129 active+clean; 621 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 7.3 KiB/s wr, 17 op/s 2024-02-02T15:25:24.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:24.738+0000 7f0e66600700 1 -- 172.21.15.1:0/1695393882 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f0e60107d00 con 0x7f0e60101f60 2024-02-02T15:25:24.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:24.744+0000 7f0e57e00700 1 -- 172.21.15.1:0/1695393882 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v31) v1 ==== 170+0+59 (secure 0 0 0) 0x7f0e48095660 con 0x7f0e60101f60 2024-02-02T15:25:24.744 INFO:teuthology.orchestra.run.smithi001.stdout:[client.0] 2024-02-02T15:25:24.744 INFO:teuthology.orchestra.run.smithi001.stdout: key = AQBkCb1lJDgiLBAAU33H3SBsjd9H1eveI7EPZA== 2024-02-02T15:25:24.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:24.746+0000 7f0e66600700 1 -- 172.21.15.1:0/1695393882 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f0e400a05b0 msgr2=0x7f0e400a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:24.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:24.746+0000 7f0e66600700 1 --2- 172.21.15.1:0/1695393882 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f0e400a05b0 0x7f0e400a2a70 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0e50006010 tx=0x7f0e5000b260 comp rx=0 tx=0).stop 2024-02-02T15:25:24.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:24.746+0000 7f0e66600700 1 -- 172.21.15.1:0/1695393882 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e60101f60 msgr2=0x7f0e60073130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:24.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:24.746+0000 7f0e66600700 1 --2- 172.21.15.1:0/1695393882 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e60101f60 0x7f0e60073130 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0e4800e9f0 tx=0x7f0e4800ed00 comp rx=0 tx=0).stop 2024-02-02T15:25:24.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:24.747+0000 7f0e66600700 1 -- 172.21.15.1:0/1695393882 shutdown_connections 2024-02-02T15:25:24.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:24.747+0000 7f0e66600700 1 --2- 172.21.15.1:0/1695393882 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f0e400a05b0 0x7f0e400a2a70 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:24.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:24.747+0000 7f0e66600700 1 --2- 172.21.15.1:0/1695393882 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e60102920 0x7f0e60073670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:24.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:24.747+0000 7f0e66600700 1 --2- 172.21.15.1:0/1695393882 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e60100e00 0x7f0e60074ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:24.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:24.747+0000 7f0e66600700 1 --2- 172.21.15.1:0/1695393882 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e60101f60 0x7f0e60073130 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:24.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:24.747+0000 7f0e66600700 1 -- 172.21.15.1:0/1695393882 >> 172.21.15.1:0/1695393882 conn(0x7f0e600fc3c0 msgr2=0x7f0e600fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:24.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:24.747+0000 7f0e66600700 1 -- 172.21.15.1:0/1695393882 shutdown_connections 2024-02-02T15:25:24.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:24.747+0000 7f0e66600700 1 -- 172.21.15.1:0/1695393882 wait complete. 2024-02-02T15:25:25.259 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-02-02T15:25:25.259 DEBUG:teuthology.orchestra.run.smithi001:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-02-02T15:25:25.259 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-02-02T15:25:25.355 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-02-02T15:25:25.355 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-02-02T15:25:25.355 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph mgr dump --format=json 2024-02-02T15:25:25.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:25 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/1695393882' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-02T15:25:25.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:25 smithi071 ceph-mon[44707]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-02T15:25:25.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:25 smithi071 ceph-mon[44707]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-02T15:25:25.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:25 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/1695393882' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-02T15:25:25.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:25 smithi142 ceph-mon[45728]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-02T15:25:25.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:25 smithi142 ceph-mon[45728]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-02T15:25:25.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:25 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/1695393882' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-02T15:25:25.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:25 smithi001 ceph-mon[35470]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-02T15:25:25.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:25 smithi001 ceph-mon[35470]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-02T15:25:25.760 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.a/config 2024-02-02T15:25:26.583 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:26 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 1]}]: dispatch 2024-02-02T15:25:26.583 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:26 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-02-02T15:25:26.583 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:26 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-02-02T15:25:26.583 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:26 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]: dispatch 2024-02-02T15:25:26.584 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:26 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-02-02T15:25:26.584 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:26 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6]}]: dispatch 2024-02-02T15:25:26.584 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:26 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]: dispatch 2024-02-02T15:25:26.584 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:26 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2024-02-02T15:25:26.584 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:26 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [5, 10]}]: dispatch 2024-02-02T15:25:26.584 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:26 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]: dispatch 2024-02-02T15:25:26.584 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:26 smithi001 ceph-mon[35470]: pgmap v254: 129 pgs: 129 active+clean; 621 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 7.4 KiB/s wr, 19 op/s 2024-02-02T15:25:26.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:26 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 1]}]: dispatch 2024-02-02T15:25:26.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:26 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-02-02T15:25:26.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:26 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-02-02T15:25:26.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:26 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]: dispatch 2024-02-02T15:25:26.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:26 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-02-02T15:25:26.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:26 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6]}]: dispatch 2024-02-02T15:25:26.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:26 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]: dispatch 2024-02-02T15:25:26.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:26 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2024-02-02T15:25:26.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:26 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [5, 10]}]: dispatch 2024-02-02T15:25:26.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:26 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]: dispatch 2024-02-02T15:25:26.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:26 smithi071 ceph-mon[44707]: pgmap v254: 129 pgs: 129 active+clean; 621 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 7.4 KiB/s wr, 19 op/s 2024-02-02T15:25:26.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:26 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 1]}]: dispatch 2024-02-02T15:25:26.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:26 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-02-02T15:25:26.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:26 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-02-02T15:25:26.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:26 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]: dispatch 2024-02-02T15:25:26.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:26 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-02-02T15:25:26.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:26 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6]}]: dispatch 2024-02-02T15:25:26.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:26 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]: dispatch 2024-02-02T15:25:26.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:26 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2024-02-02T15:25:26.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:26 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [5, 10]}]: dispatch 2024-02-02T15:25:26.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:26 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]: dispatch 2024-02-02T15:25:26.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:26 smithi142 ceph-mon[45728]: pgmap v254: 129 pgs: 129 active+clean; 621 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 7.4 KiB/s wr, 19 op/s 2024-02-02T15:25:27.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:27.166+0000 7fe251800700 1 -- 172.21.15.1:0/248041696 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe24c100fc0 msgr2=0x7fe24c1013a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:27.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:27.166+0000 7fe251800700 1 --2- 172.21.15.1:0/248041696 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe24c100fc0 0x7fe24c1013a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe23c009990 tx=0x7fe23c009ca0 comp rx=0 tx=0).stop 2024-02-02T15:25:27.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:27.166+0000 7fe251800700 1 -- 172.21.15.1:0/248041696 shutdown_connections 2024-02-02T15:25:27.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:27.166+0000 7fe251800700 1 --2- 172.21.15.1:0/248041696 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe24c1060a0 0x7fe24c10dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:27.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:27.166+0000 7fe251800700 1 --2- 172.21.15.1:0/248041696 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe24c100fc0 0x7fe24c1013a0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:27.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:27.166+0000 7fe251800700 1 --2- 172.21.15.1:0/248041696 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe24c101970 0x7fe24c1059c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:27.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:27.166+0000 7fe251800700 1 -- 172.21.15.1:0/248041696 >> 172.21.15.1:0/248041696 conn(0x7fe24c0fc830 msgr2=0x7fe24c0fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:27.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:27.166+0000 7fe251800700 1 -- 172.21.15.1:0/248041696 shutdown_connections 2024-02-02T15:25:27.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:27.166+0000 7fe251800700 1 -- 172.21.15.1:0/248041696 wait complete. 2024-02-02T15:25:27.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:27.167+0000 7fe251800700 1 Processor -- start 2024-02-02T15:25:27.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:27.168+0000 7fe251800700 1 -- start start 2024-02-02T15:25:27.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:27.168+0000 7fe251800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe24c100fc0 0x7fe24c196f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:27.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:27.168+0000 7fe251800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe24c101970 0x7fe24c1974d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:27.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:27.168+0000 7fe251800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe24c1060a0 0x7fe24c19b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:27.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:27.168+0000 7fe251800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe24c19bf50 con 0x7fe24c1060a0 2024-02-02T15:25:27.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:27.168+0000 7fe251800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe24c19c0c0 con 0x7fe24c101970 2024-02-02T15:25:27.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:27.168+0000 7fe251800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe24c19c280 con 0x7fe24c100fc0 2024-02-02T15:25:27.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:27.169+0000 7fe24aa00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe24c100fc0 0x7fe24c196f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:27.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:27.169+0000 7fe24b400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe24c1060a0 0x7fe24c19b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:27.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:27.169+0000 7fe24aa00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe24c100fc0 0x7fe24c196f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:60398/0 (socket says 172.21.15.1:60398) 2024-02-02T15:25:27.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:27.169+0000 7fe24aa00700 1 -- 172.21.15.1:0/3453544269 learned_addr learned my addr 172.21.15.1:0/3453544269 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:25:27.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:27.169+0000 7fe243e00700 1 --2- 172.21.15.1:0/3453544269 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe24c101970 0x7fe24c1974d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:27.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:27.169+0000 7fe24aa00700 1 -- 172.21.15.1:0/3453544269 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe24c101970 msgr2=0x7fe24c1974d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:27.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:27.169+0000 7fe24aa00700 1 --2- 172.21.15.1:0/3453544269 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe24c101970 0x7fe24c1974d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:27.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:27.169+0000 7fe24aa00700 1 -- 172.21.15.1:0/3453544269 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe24c1060a0 msgr2=0x7fe24c19b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:27.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:27.169+0000 7fe24aa00700 1 --2- 172.21.15.1:0/3453544269 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe24c1060a0 0x7fe24c19b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:27.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:27.169+0000 7fe24aa00700 1 -- 172.21.15.1:0/3453544269 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe23c0095f0 con 0x7fe24c100fc0 2024-02-02T15:25:27.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:27.169+0000 7fe24b400700 1 --2- 172.21.15.1:0/3453544269 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe24c1060a0 0x7fe24c19b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:25:27.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:27.169+0000 7fe24aa00700 1 --2- 172.21.15.1:0/3453544269 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe24c100fc0 0x7fe24c196f90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe23c005910 tx=0x7fe23c005720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:27.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:27.170+0000 7fe243400700 1 -- 172.21.15.1:0/3453544269 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe23c01d070 con 0x7fe24c100fc0 2024-02-02T15:25:27.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:27.170+0000 7fe251800700 1 -- 172.21.15.1:0/3453544269 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe24c19c500 con 0x7fe24c100fc0 2024-02-02T15:25:27.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:27.170+0000 7fe243400700 1 -- 172.21.15.1:0/3453544269 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe23c022470 con 0x7fe24c100fc0 2024-02-02T15:25:27.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:27.170+0000 7fe243400700 1 -- 172.21.15.1:0/3453544269 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe23c017650 con 0x7fe24c100fc0 2024-02-02T15:25:27.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:27.170+0000 7fe251800700 1 -- 172.21.15.1:0/3453544269 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe24c073140 con 0x7fe24c100fc0 2024-02-02T15:25:27.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:27.171+0000 7fe251800700 1 -- 172.21.15.1:0/3453544269 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe24c04ea40 con 0x7fe24c100fc0 2024-02-02T15:25:27.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:27.173+0000 7fe243400700 1 -- 172.21.15.1:0/3453544269 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 17) v1 ==== 135120+0+0 (secure 0 0 0) 0x7fe23c0225e0 con 0x7fe24c100fc0 2024-02-02T15:25:27.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:27.174+0000 7fe243400700 1 --2- 172.21.15.1:0/3453544269 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fe22c0a0630 0x7fe22c0a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:27.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:27.175+0000 7fe243400700 1 -- 172.21.15.1:0/3453544269 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fe23c0cd390 con 0x7fe24c100fc0 2024-02-02T15:25:27.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:27.175+0000 7fe243e00700 1 --2- 172.21.15.1:0/3453544269 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fe22c0a0630 0x7fe22c0a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:27.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:27.175+0000 7fe243e00700 1 --2- 172.21.15.1:0/3453544269 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fe22c0a0630 0x7fe22c0a2af0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe24c1985b0 tx=0x7fe234009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:27.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:27.177+0000 7fe243400700 1 -- 172.21.15.1:0/3453544269 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7fe23c096af0 con 0x7fe24c100fc0 2024-02-02T15:25:27.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:27.454+0000 7fe251800700 1 -- 172.21.15.1:0/3453544269 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fe24c073420 con 0x7fe24c100fc0 2024-02-02T15:25:27.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:27.459+0000 7fe243400700 1 -- 172.21.15.1:0/3453544269 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+258821 (secure 0 0 0) 0x7fe23c031090 con 0x7fe24c100fc0 2024-02-02T15:25:27.462 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:25:27.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:27.463+0000 7fe251800700 1 -- 172.21.15.1:0/3453544269 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fe22c0a0630 msgr2=0x7fe22c0a2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:27.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:27.463+0000 7fe251800700 1 --2- 172.21.15.1:0/3453544269 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fe22c0a0630 0x7fe22c0a2af0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe24c1985b0 tx=0x7fe234009260 comp rx=0 tx=0).stop 2024-02-02T15:25:27.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:27.463+0000 7fe251800700 1 -- 172.21.15.1:0/3453544269 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe24c100fc0 msgr2=0x7fe24c196f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:27.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:27.463+0000 7fe251800700 1 --2- 172.21.15.1:0/3453544269 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe24c100fc0 0x7fe24c196f90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe23c005910 tx=0x7fe23c005720 comp rx=0 tx=0).stop 2024-02-02T15:25:27.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:27.463+0000 7fe251800700 1 -- 172.21.15.1:0/3453544269 shutdown_connections 2024-02-02T15:25:27.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:27.463+0000 7fe251800700 1 --2- 172.21.15.1:0/3453544269 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe24c1060a0 0x7fe24c19b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:27.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:27.463+0000 7fe251800700 1 --2- 172.21.15.1:0/3453544269 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fe22c0a0630 0x7fe22c0a2af0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:27.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:27.463+0000 7fe251800700 1 --2- 172.21.15.1:0/3453544269 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe24c100fc0 0x7fe24c196f90 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:27.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:27.463+0000 7fe251800700 1 --2- 172.21.15.1:0/3453544269 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe24c101970 0x7fe24c1974d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:27.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:27.463+0000 7fe251800700 1 -- 172.21.15.1:0/3453544269 >> 172.21.15.1:0/3453544269 conn(0x7fe24c0fc830 msgr2=0x7fe24c0fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:27.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:27.464+0000 7fe251800700 1 -- 172.21.15.1:0/3453544269 shutdown_connections 2024-02-02T15:25:27.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:27.464+0000 7fe251800700 1 -- 172.21.15.1:0/3453544269 wait complete. 2024-02-02T15:25:27.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:27 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 1]}]': finished 2024-02-02T15:25:27.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:27 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-02-02T15:25:27.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:27 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-02-02T15:25:27.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:27 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]': finished 2024-02-02T15:25:27.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:27 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-02-02T15:25:27.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:27 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6]}]': finished 2024-02-02T15:25:27.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:27 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]': finished 2024-02-02T15:25:27.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:27 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2024-02-02T15:25:27.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:27 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [5, 10]}]': finished 2024-02-02T15:25:27.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:27 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]': finished 2024-02-02T15:25:27.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:27 smithi071 ceph-mon[44707]: osdmap e85: 12 total, 12 up, 12 in 2024-02-02T15:25:27.657 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:27 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 1]}]': finished 2024-02-02T15:25:27.657 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:27 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-02-02T15:25:27.657 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:27 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-02-02T15:25:27.657 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:27 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]': finished 2024-02-02T15:25:27.657 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:27 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-02-02T15:25:27.657 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:27 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6]}]': finished 2024-02-02T15:25:27.657 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:27 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]': finished 2024-02-02T15:25:27.658 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:27 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2024-02-02T15:25:27.658 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:27 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [5, 10]}]': finished 2024-02-02T15:25:27.658 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:27 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]': finished 2024-02-02T15:25:27.658 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:27 smithi001 ceph-mon[35470]: osdmap e85: 12 total, 12 up, 12 in 2024-02-02T15:25:27.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:27 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 1]}]': finished 2024-02-02T15:25:27.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:27 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-02-02T15:25:27.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:27 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-02-02T15:25:27.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:27 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]': finished 2024-02-02T15:25:27.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:27 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-02-02T15:25:27.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:27 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [5, 6]}]': finished 2024-02-02T15:25:27.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:27 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [9, 11]}]': finished 2024-02-02T15:25:27.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:27 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2024-02-02T15:25:27.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:27 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [5, 10]}]': finished 2024-02-02T15:25:27.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:27 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.16", "id": [9, 8]}]': finished 2024-02-02T15:25:27.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:27 smithi142 ceph-mon[45728]: osdmap e85: 12 total, 12 up, 12 in 2024-02-02T15:25:27.896 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":17,"active_gid":14154,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6800","nonce":3768989981},{"type":"v1","addr":"172.21.15.1:6801","nonce":3768989981}]},"active_addr":"172.21.15.1:6801/3768989981","active_change":"2024-02-02T15:19:25.836329+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":24115,"name":"z","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24119,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.1:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.1:0","nonce":3236204176}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.1:0","nonce":3997770901}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.1:0","nonce":1574620681}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.1:0","nonce":2256116842}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.1:0","nonce":1943250276}]}]} 2024-02-02T15:25:27.901 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-02-02T15:25:27.901 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-02-02T15:25:27.902 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph osd dump --format=json 2024-02-02T15:25:28.270 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.a/config 2024-02-02T15:25:28.552 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:28 smithi001 ceph-mon[35470]: osdmap e86: 12 total, 12 up, 12 in 2024-02-02T15:25:28.553 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:28 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/3453544269' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-02T15:25:28.553 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:28 smithi001 ceph-mon[35470]: pgmap v257: 129 pgs: 129 active+clean; 626 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.1 KiB/s wr, 24 op/s 2024-02-02T15:25:28.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:28 smithi071 ceph-mon[44707]: osdmap e86: 12 total, 12 up, 12 in 2024-02-02T15:25:28.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:28 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/3453544269' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-02T15:25:28.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:28 smithi071 ceph-mon[44707]: pgmap v257: 129 pgs: 129 active+clean; 626 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.1 KiB/s wr, 24 op/s 2024-02-02T15:25:28.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:28 smithi142 ceph-mon[45728]: osdmap e86: 12 total, 12 up, 12 in 2024-02-02T15:25:28.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:28 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/3453544269' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-02T15:25:28.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:28 smithi142 ceph-mon[45728]: pgmap v257: 129 pgs: 129 active+clean; 626 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.1 KiB/s wr, 24 op/s 2024-02-02T15:25:29.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:29.474+0000 7efc0aa00700 1 -- 172.21.15.1:0/2580427472 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efc041060a0 msgr2=0x7efc0410dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:29.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:29.474+0000 7efc0aa00700 1 --2- 172.21.15.1:0/2580427472 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efc041060a0 0x7efc0410dc70 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7efbfc00b0a0 tx=0x7efbfc00b3b0 comp rx=0 tx=0).stop 2024-02-02T15:25:29.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:29.474+0000 7efc0aa00700 1 -- 172.21.15.1:0/2580427472 shutdown_connections 2024-02-02T15:25:29.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:29.474+0000 7efc0aa00700 1 --2- 172.21.15.1:0/2580427472 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efc041060a0 0x7efc0410dc70 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:29.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:29.474+0000 7efc0aa00700 1 --2- 172.21.15.1:0/2580427472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc04100fc0 0x7efc041013a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:29.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:29.474+0000 7efc0aa00700 1 --2- 172.21.15.1:0/2580427472 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc04101970 0x7efc041059c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:29.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:29.474+0000 7efc0aa00700 1 -- 172.21.15.1:0/2580427472 >> 172.21.15.1:0/2580427472 conn(0x7efc040fc830 msgr2=0x7efc040fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:29.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:29.474+0000 7efc0aa00700 1 -- 172.21.15.1:0/2580427472 shutdown_connections 2024-02-02T15:25:29.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:29.474+0000 7efc0aa00700 1 -- 172.21.15.1:0/2580427472 wait complete. 2024-02-02T15:25:29.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:29.475+0000 7efc0aa00700 1 Processor -- start 2024-02-02T15:25:29.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:29.475+0000 7efc0aa00700 1 -- start start 2024-02-02T15:25:29.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:29.475+0000 7efc0aa00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc04100fc0 0x7efc04196f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:29.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:29.475+0000 7efc0aa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efc04101970 0x7efc041974d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:29.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:29.475+0000 7efc0aa00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc041060a0 0x7efc0419b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:29.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:29.475+0000 7efc0aa00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc0419bf00 con 0x7efc04101970 2024-02-02T15:25:29.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:29.475+0000 7efc0aa00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc0419c070 con 0x7efc04100fc0 2024-02-02T15:25:29.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:29.475+0000 7efc0aa00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc0419c230 con 0x7efc041060a0 2024-02-02T15:25:29.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:29.476+0000 7efc03400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efc04101970 0x7efc041974d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:29.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:29.476+0000 7efc03400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efc04101970 0x7efc041974d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33514/0 (socket says 172.21.15.1:33514) 2024-02-02T15:25:29.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:29.476+0000 7efc03e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc04100fc0 0x7efc04196f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:29.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:29.476+0000 7efc03400700 1 -- 172.21.15.1:0/2393712228 learned_addr learned my addr 172.21.15.1:0/2393712228 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:25:29.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:29.476+0000 7efc08a00700 1 --2- 172.21.15.1:0/2393712228 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc041060a0 0x7efc0419b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:29.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:29.476+0000 7efc03e00700 1 -- 172.21.15.1:0/2393712228 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc041060a0 msgr2=0x7efc0419b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:29.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:29.476+0000 7efc03e00700 1 --2- 172.21.15.1:0/2393712228 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc041060a0 0x7efc0419b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:29.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:29.476+0000 7efc03e00700 1 -- 172.21.15.1:0/2393712228 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efc04101970 msgr2=0x7efc041974d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:29.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:29.476+0000 7efc03e00700 1 --2- 172.21.15.1:0/2393712228 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efc04101970 0x7efc041974d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:29.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:29.476+0000 7efc03e00700 1 -- 172.21.15.1:0/2393712228 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbfc009d00 con 0x7efc04100fc0 2024-02-02T15:25:29.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:29.476+0000 7efc03400700 1 --2- 172.21.15.1:0/2393712228 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efc04101970 0x7efc041974d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:25:29.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:29.476+0000 7efc03e00700 1 --2- 172.21.15.1:0/2393712228 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc04100fc0 0x7efc04196f90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7efbf400e9f0 tx=0x7efbf400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:29.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:29.477+0000 7efc00c00700 1 -- 172.21.15.1:0/2393712228 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf400c9c0 con 0x7efc04100fc0 2024-02-02T15:25:29.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:29.477+0000 7efc00c00700 1 -- 172.21.15.1:0/2393712228 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efbf4010430 con 0x7efc04100fc0 2024-02-02T15:25:29.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:29.477+0000 7efc00c00700 1 -- 172.21.15.1:0/2393712228 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf4018680 con 0x7efc04100fc0 2024-02-02T15:25:29.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:29.477+0000 7efc0aa00700 1 -- 172.21.15.1:0/2393712228 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc0419c510 con 0x7efc04100fc0 2024-02-02T15:25:29.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:29.477+0000 7efc0aa00700 1 -- 172.21.15.1:0/2393712228 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc041a3d70 con 0x7efc04100fc0 2024-02-02T15:25:29.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:29.478+0000 7efc0aa00700 1 -- 172.21.15.1:0/2393712228 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc0404ea40 con 0x7efc04100fc0 2024-02-02T15:25:29.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:29.480+0000 7efc00c00700 1 -- 172.21.15.1:0/2393712228 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 135120+0+0 (secure 0 0 0) 0x7efbf4010860 con 0x7efc04100fc0 2024-02-02T15:25:29.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:29.481+0000 7efc00c00700 1 --2- 172.21.15.1:0/2393712228 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7efbe40a0630 0x7efbe40a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:29.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:29.481+0000 7efc00c00700 1 -- 172.21.15.1:0/2393712228 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7efbf4014070 con 0x7efc04100fc0 2024-02-02T15:25:29.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:29.482+0000 7efc03400700 1 --2- 172.21.15.1:0/2393712228 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7efbe40a0630 0x7efbe40a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:29.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:29.482+0000 7efc03400700 1 --2- 172.21.15.1:0/2393712228 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7efbe40a0630 0x7efbe40a2af0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7efc04198560 tx=0x7efbec009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:29.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:29.484+0000 7efc00c00700 1 -- 172.21.15.1:0/2393712228 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7efbf40957c0 con 0x7efc04100fc0 2024-02-02T15:25:29.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:29.693+0000 7efc0aa00700 1 -- 172.21.15.1:0/2393712228 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efc041a4060 con 0x7efc04100fc0 2024-02-02T15:25:29.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:29.695+0000 7efc00c00700 1 -- 172.21.15.1:0/2393712228 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+24312 (secure 0 0 0) 0x7efbf4026090 con 0x7efc04100fc0 2024-02-02T15:25:29.695 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:25:29.696 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":86,"fsid":"371a409c-c1de-11ee-95b6-87774f69a715","created":"2024-02-02T15:18:26.746506+0000","modified":"2024-02-02T15:25:27.269296+0000","last_up_change":"2024-02-02T15:24:08.800951+0000","last_in_change":"2024-02-02T15:23:52.784280+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T15:21:22.471754+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-02T15:24:46.195779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-02T15:24:47.226403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-02T15:24:50.300462+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"73e8a07f-3560-4c19-85ed-030d4d8f0f23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6803","nonce":2591906423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6805","nonce":2591906423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6809","nonce":2591906423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6807","nonce":2591906423}]},"public_addr":"172.21.15.1:6803/2591906423","cluster_addr":"172.21.15.1:6805/2591906423","heartbeat_back_addr":"172.21.15.1:6809/2591906423","heartbeat_front_addr":"172.21.15.1:6807/2591906423","state":["exists","up"]},{"osd":1,"uuid":"e53cffec-0ff9-4ed9-8dcf-ff47d04db2bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6801","nonce":747581719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6803","nonce":747581719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6807","nonce":747581719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6805","nonce":747581719}]},"public_addr":"172.21.15.71:6801/747581719","cluster_addr":"172.21.15.71:6803/747581719","heartbeat_back_addr":"172.21.15.71:6807/747581719","heartbeat_front_addr":"172.21.15.71:6805/747581719","state":["exists","up"]},{"osd":2,"uuid":"856c2347-294c-482b-8595-ad77f4792db4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6801","nonce":377511414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6803","nonce":377511414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6807","nonce":377511414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6805","nonce":377511414}]},"public_addr":"172.21.15.142:6801/377511414","cluster_addr":"172.21.15.142:6803/377511414","heartbeat_back_addr":"172.21.15.142:6807/377511414","heartbeat_front_addr":"172.21.15.142:6805/377511414","state":["exists","up"]},{"osd":3,"uuid":"4b946ad2-2da9-4e0b-9d61-d039d8669a42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6811","nonce":1086124477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6813","nonce":1086124477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6817","nonce":1086124477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6815","nonce":1086124477}]},"public_addr":"172.21.15.1:6811/1086124477","cluster_addr":"172.21.15.1:6813/1086124477","heartbeat_back_addr":"172.21.15.1:6817/1086124477","heartbeat_front_addr":"172.21.15.1:6815/1086124477","state":["exists","up"]},{"osd":4,"uuid":"43e36c43-47a0-4634-a45b-1da14a0e19d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6809","nonce":3095380305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6811","nonce":3095380305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6815","nonce":3095380305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6813","nonce":3095380305}]},"public_addr":"172.21.15.71:6809/3095380305","cluster_addr":"172.21.15.71:6811/3095380305","heartbeat_back_addr":"172.21.15.71:6815/3095380305","heartbeat_front_addr":"172.21.15.71:6813/3095380305","state":["exists","up"]},{"osd":5,"uuid":"eb48160e-ea7c-42f7-9477-be110bc8a34e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6809","nonce":4195149262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6811","nonce":4195149262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6815","nonce":4195149262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6813","nonce":4195149262}]},"public_addr":"172.21.15.142:6809/4195149262","cluster_addr":"172.21.15.142:6811/4195149262","heartbeat_back_addr":"172.21.15.142:6815/4195149262","heartbeat_front_addr":"172.21.15.142:6813/4195149262","state":["exists","up"]},{"osd":6,"uuid":"0ec4501a-5853-4ca6-b325-231f5b72335e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6819","nonce":1560153428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6821","nonce":1560153428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6825","nonce":1560153428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6823","nonce":1560153428}]},"public_addr":"172.21.15.1:6819/1560153428","cluster_addr":"172.21.15.1:6821/1560153428","heartbeat_back_addr":"172.21.15.1:6825/1560153428","heartbeat_front_addr":"172.21.15.1:6823/1560153428","state":["exists","up"]},{"osd":7,"uuid":"029debe8-9cdf-44f6-9aea-cabd1d185f93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6817","nonce":4080955538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6819","nonce":4080955538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6823","nonce":4080955538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6821","nonce":4080955538}]},"public_addr":"172.21.15.71:6817/4080955538","cluster_addr":"172.21.15.71:6819/4080955538","heartbeat_back_addr":"172.21.15.71:6823/4080955538","heartbeat_front_addr":"172.21.15.71:6821/4080955538","state":["exists","up"]},{"osd":8,"uuid":"7c90896c-9955-4c1b-aaca-7531dd5457f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6817","nonce":3942038209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6819","nonce":3942038209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6823","nonce":3942038209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6821","nonce":3942038209}]},"public_addr":"172.21.15.142:6817/3942038209","cluster_addr":"172.21.15.142:6819/3942038209","heartbeat_back_addr":"172.21.15.142:6823/3942038209","heartbeat_front_addr":"172.21.15.142:6821/3942038209","state":["exists","up"]},{"osd":9,"uuid":"7e165e68-31a8-4598-a577-de3e90ae5c5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6827","nonce":2716544850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6829","nonce":2716544850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6833","nonce":2716544850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6831","nonce":2716544850}]},"public_addr":"172.21.15.1:6827/2716544850","cluster_addr":"172.21.15.1:6829/2716544850","heartbeat_back_addr":"172.21.15.1:6833/2716544850","heartbeat_front_addr":"172.21.15.1:6831/2716544850","state":["exists","up"]},{"osd":10,"uuid":"48cd5fdb-0887-444a-aab8-a1de24282bb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6825","nonce":84691841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6827","nonce":84691841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6831","nonce":84691841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6829","nonce":84691841}]},"public_addr":"172.21.15.71:6825/84691841","cluster_addr":"172.21.15.71:6827/84691841","heartbeat_back_addr":"172.21.15.71:6831/84691841","heartbeat_front_addr":"172.21.15.71:6829/84691841","state":["exists","up"]},{"osd":11,"uuid":"f011dd27-11b0-4166-adfe-76756533163a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6825","nonce":2290179934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6827","nonce":2290179934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6831","nonce":2290179934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6829","nonce":2290179934}]},"public_addr":"172.21.15.142:6825/2290179934","cluster_addr":"172.21.15.142:6827/2290179934","heartbeat_back_addr":"172.21.15.142:6831/2290179934","heartbeat_front_addr":"172.21.15.142:6829/2290179934","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:20:44.060692+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:01.903817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:19.368258+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:40.028037+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:57.513361+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:22:14.805944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:22:35.188439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:22:53.014412+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:23:10.507420+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:23:31.849765+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:23:50.070612+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:24:07.388625+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":5,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/4092532060":"2024-02-03T15:19:25.836171+0000","172.21.15.1:6801/395498296":"2024-02-03T15:19:25.836171+0000","172.21.15.1:6800/395498296":"2024-02-03T15:19:25.836171+0000","172.21.15.1:0/1575373489":"2024-02-03T15:19:25.836171+0000","172.21.15.1:0/865983374":"2024-02-03T15:19:01.431596+0000","172.21.15.1:0/3165960227":"2024-02-03T15:19:01.431596+0000","172.21.15.1:0/2350830293":"2024-02-03T15:19:01.431596+0000","172.21.15.1:0/2959690250":"2024-02-03T15:19:25.836171+0000","172.21.15.1:6800/1162530732":"2024-02-03T15:19:01.431596+0000","172.21.15.1:6801/1162530732":"2024-02-03T15:19:01.431596+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T15:25:29.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:29.698+0000 7efc0aa00700 1 -- 172.21.15.1:0/2393712228 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7efbe40a0630 msgr2=0x7efbe40a2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:29.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:29.698+0000 7efc0aa00700 1 --2- 172.21.15.1:0/2393712228 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7efbe40a0630 0x7efbe40a2af0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7efc04198560 tx=0x7efbec009260 comp rx=0 tx=0).stop 2024-02-02T15:25:29.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:29.698+0000 7efc0aa00700 1 -- 172.21.15.1:0/2393712228 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc04100fc0 msgr2=0x7efc04196f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:29.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:29.698+0000 7efc0aa00700 1 --2- 172.21.15.1:0/2393712228 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc04100fc0 0x7efc04196f90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7efbf400e9f0 tx=0x7efbf400ed00 comp rx=0 tx=0).stop 2024-02-02T15:25:29.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:29.698+0000 7efc0aa00700 1 -- 172.21.15.1:0/2393712228 shutdown_connections 2024-02-02T15:25:29.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:29.698+0000 7efc0aa00700 1 --2- 172.21.15.1:0/2393712228 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7efbe40a0630 0x7efbe40a2af0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:29.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:29.698+0000 7efc0aa00700 1 --2- 172.21.15.1:0/2393712228 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc041060a0 0x7efc0419b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:29.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:29.698+0000 7efc0aa00700 1 --2- 172.21.15.1:0/2393712228 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc04100fc0 0x7efc04196f90 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:29.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:29.698+0000 7efc0aa00700 1 --2- 172.21.15.1:0/2393712228 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efc04101970 0x7efc041974d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:29.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:29.698+0000 7efc0aa00700 1 -- 172.21.15.1:0/2393712228 >> 172.21.15.1:0/2393712228 conn(0x7efc040fc830 msgr2=0x7efc040fd300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:29.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:29.698+0000 7efc0aa00700 1 -- 172.21.15.1:0/2393712228 shutdown_connections 2024-02-02T15:25:29.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:29.698+0000 7efc0aa00700 1 -- 172.21.15.1:0/2393712228 wait complete. 2024-02-02T15:25:30.023 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:29 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/2393712228' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-02T15:25:30.051 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-02-02T15:25:30.052 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph osd dump --format=json 2024-02-02T15:25:30.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:29 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/2393712228' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-02T15:25:30.169 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:29 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/2393712228' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-02T15:25:30.422 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.a/config 2024-02-02T15:25:31.060 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:30 smithi001 ceph-mon[35470]: pgmap v258: 129 pgs: 129 active+clean; 626 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 2.7 KiB/s wr, 17 op/s; 8 B/s, 1 keys/s, 0 objects/s recovering 2024-02-02T15:25:31.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:30 smithi071 ceph-mon[44707]: pgmap v258: 129 pgs: 129 active+clean; 626 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 2.7 KiB/s wr, 17 op/s; 8 B/s, 1 keys/s, 0 objects/s recovering 2024-02-02T15:25:31.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:30 smithi142 ceph-mon[45728]: pgmap v258: 129 pgs: 129 active+clean; 626 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 2.7 KiB/s wr, 17 op/s; 8 B/s, 1 keys/s, 0 objects/s recovering 2024-02-02T15:25:31.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:31.871+0000 7fdf95600700 1 -- 172.21.15.1:0/1337158070 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf90068860 msgr2=0x7fdf90068ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:31.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:31.871+0000 7fdf95600700 1 --2- 172.21.15.1:0/1337158070 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf90068860 0x7fdf90068ce0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdf780098f0 tx=0x7fdf78009c00 comp rx=0 tx=0).stop 2024-02-02T15:25:31.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:31.871+0000 7fdf95600700 1 -- 172.21.15.1:0/1337158070 shutdown_connections 2024-02-02T15:25:31.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:31.871+0000 7fdf95600700 1 --2- 172.21.15.1:0/1337158070 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf90069220 0x7fdf9010db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:31.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:31.871+0000 7fdf95600700 1 --2- 172.21.15.1:0/1337158070 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf90106ed0 0x7fdf901072b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:31.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:31.871+0000 7fdf95600700 1 --2- 172.21.15.1:0/1337158070 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf90068860 0x7fdf90068ce0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:31.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:31.871+0000 7fdf95600700 1 -- 172.21.15.1:0/1337158070 >> 172.21.15.1:0/1337158070 conn(0x7fdf90076d40 msgr2=0x7fdf90077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:31.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:31.871+0000 7fdf95600700 1 -- 172.21.15.1:0/1337158070 shutdown_connections 2024-02-02T15:25:31.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:31.871+0000 7fdf95600700 1 -- 172.21.15.1:0/1337158070 wait complete. 2024-02-02T15:25:31.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:31.872+0000 7fdf95600700 1 Processor -- start 2024-02-02T15:25:31.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:31.872+0000 7fdf95600700 1 -- start start 2024-02-02T15:25:31.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:31.872+0000 7fdf95600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf90068860 0x7fdf90198fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:31.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:31.872+0000 7fdf95600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf90069220 0x7fdf90199520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:31.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:31.872+0000 7fdf95600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf90106ed0 0x7fdf9019d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:31.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:31.873+0000 7fdf95600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9019dfa0 con 0x7fdf90068860 2024-02-02T15:25:31.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:31.873+0000 7fdf95600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9019e110 con 0x7fdf90069220 2024-02-02T15:25:31.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:31.873+0000 7fdf95600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9019e2d0 con 0x7fdf90106ed0 2024-02-02T15:25:31.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:31.873+0000 7fdf8ea00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf90068860 0x7fdf90198fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:31.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:31.873+0000 7fdf8ea00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf90068860 0x7fdf90198fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33530/0 (socket says 172.21.15.1:33530) 2024-02-02T15:25:31.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:31.873+0000 7fdf8f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf90106ed0 0x7fdf9019d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:31.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:31.873+0000 7fdf8ea00700 1 -- 172.21.15.1:0/3285676145 learned_addr learned my addr 172.21.15.1:0/3285676145 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:25:31.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:31.873+0000 7fdf8e000700 1 --2- 172.21.15.1:0/3285676145 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf90069220 0x7fdf90199520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:31.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:31.873+0000 7fdf8ea00700 1 -- 172.21.15.1:0/3285676145 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf90106ed0 msgr2=0x7fdf9019d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:31.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:31.873+0000 7fdf8ea00700 1 --2- 172.21.15.1:0/3285676145 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf90106ed0 0x7fdf9019d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:31.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:31.873+0000 7fdf8ea00700 1 -- 172.21.15.1:0/3285676145 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf90069220 msgr2=0x7fdf90199520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:31.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:31.873+0000 7fdf8ea00700 1 --2- 172.21.15.1:0/3285676145 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf90069220 0x7fdf90199520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:31.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:31.873+0000 7fdf8ea00700 1 -- 172.21.15.1:0/3285676145 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf78009520 con 0x7fdf90068860 2024-02-02T15:25:31.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:31.873+0000 7fdf8e000700 1 --2- 172.21.15.1:0/3285676145 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf90069220 0x7fdf90199520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:25:31.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:31.874+0000 7fdf8ea00700 1 --2- 172.21.15.1:0/3285676145 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf90068860 0x7fdf90198fe0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fdf8000e9c0 tx=0x7fdf8000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:31.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:31.874+0000 7fdf87400700 1 -- 172.21.15.1:0/3285676145 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf80004a30 con 0x7fdf90068860 2024-02-02T15:25:31.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:31.874+0000 7fdf95600700 1 -- 172.21.15.1:0/3285676145 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf9019e5b0 con 0x7fdf90068860 2024-02-02T15:25:31.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:31.874+0000 7fdf87400700 1 -- 172.21.15.1:0/3285676145 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf8000cc80 con 0x7fdf90068860 2024-02-02T15:25:31.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:31.874+0000 7fdf87400700 1 -- 172.21.15.1:0/3285676145 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf800054e0 con 0x7fdf90068860 2024-02-02T15:25:31.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:31.874+0000 7fdf95600700 1 -- 172.21.15.1:0/3285676145 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf901a5e10 con 0x7fdf90068860 2024-02-02T15:25:31.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:31.875+0000 7fdf95600700 1 -- 172.21.15.1:0/3285676145 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf9004ea40 con 0x7fdf90068860 2024-02-02T15:25:31.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:31.877+0000 7fdf87400700 1 -- 172.21.15.1:0/3285676145 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) v1 ==== 135120+0+0 (secure 0 0 0) 0x7fdf80004b90 con 0x7fdf90068860 2024-02-02T15:25:31.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:31.879+0000 7fdf87400700 1 --2- 172.21.15.1:0/3285676145 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fdf700a05b0 0x7fdf700a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:31.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:31.879+0000 7fdf87400700 1 -- 172.21.15.1:0/3285676145 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fdf80014070 con 0x7fdf90068860 2024-02-02T15:25:31.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:31.879+0000 7fdf8e000700 1 --2- 172.21.15.1:0/3285676145 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fdf700a05b0 0x7fdf700a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:31.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:31.880+0000 7fdf8e000700 1 --2- 172.21.15.1:0/3285676145 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fdf700a05b0 0x7fdf700a2a70 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fdf78000c00 tx=0x7fdf780058e0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:31.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:31.883+0000 7fdf87400700 1 -- 172.21.15.1:0/3285676145 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7fdf80095600 con 0x7fdf90068860 2024-02-02T15:25:32.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:32.095+0000 7fdf95600700 1 -- 172.21.15.1:0/3285676145 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdf9019a340 con 0x7fdf90068860 2024-02-02T15:25:32.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:32.097+0000 7fdf87400700 1 -- 172.21.15.1:0/3285676145 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+24312 (secure 0 0 0) 0x7fdf80098c20 con 0x7fdf90068860 2024-02-02T15:25:32.096 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:25:32.096 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":86,"fsid":"371a409c-c1de-11ee-95b6-87774f69a715","created":"2024-02-02T15:18:26.746506+0000","modified":"2024-02-02T15:25:27.269296+0000","last_up_change":"2024-02-02T15:24:08.800951+0000","last_in_change":"2024-02-02T15:23:52.784280+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T15:21:22.471754+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-02T15:24:46.195779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-02T15:24:47.226403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-02T15:24:50.300462+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"73e8a07f-3560-4c19-85ed-030d4d8f0f23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6803","nonce":2591906423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6805","nonce":2591906423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6809","nonce":2591906423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6807","nonce":2591906423}]},"public_addr":"172.21.15.1:6803/2591906423","cluster_addr":"172.21.15.1:6805/2591906423","heartbeat_back_addr":"172.21.15.1:6809/2591906423","heartbeat_front_addr":"172.21.15.1:6807/2591906423","state":["exists","up"]},{"osd":1,"uuid":"e53cffec-0ff9-4ed9-8dcf-ff47d04db2bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6801","nonce":747581719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6803","nonce":747581719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6807","nonce":747581719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6805","nonce":747581719}]},"public_addr":"172.21.15.71:6801/747581719","cluster_addr":"172.21.15.71:6803/747581719","heartbeat_back_addr":"172.21.15.71:6807/747581719","heartbeat_front_addr":"172.21.15.71:6805/747581719","state":["exists","up"]},{"osd":2,"uuid":"856c2347-294c-482b-8595-ad77f4792db4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6801","nonce":377511414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6803","nonce":377511414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6807","nonce":377511414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6805","nonce":377511414}]},"public_addr":"172.21.15.142:6801/377511414","cluster_addr":"172.21.15.142:6803/377511414","heartbeat_back_addr":"172.21.15.142:6807/377511414","heartbeat_front_addr":"172.21.15.142:6805/377511414","state":["exists","up"]},{"osd":3,"uuid":"4b946ad2-2da9-4e0b-9d61-d039d8669a42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6811","nonce":1086124477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6813","nonce":1086124477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6817","nonce":1086124477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6815","nonce":1086124477}]},"public_addr":"172.21.15.1:6811/1086124477","cluster_addr":"172.21.15.1:6813/1086124477","heartbeat_back_addr":"172.21.15.1:6817/1086124477","heartbeat_front_addr":"172.21.15.1:6815/1086124477","state":["exists","up"]},{"osd":4,"uuid":"43e36c43-47a0-4634-a45b-1da14a0e19d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6809","nonce":3095380305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6811","nonce":3095380305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6815","nonce":3095380305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6813","nonce":3095380305}]},"public_addr":"172.21.15.71:6809/3095380305","cluster_addr":"172.21.15.71:6811/3095380305","heartbeat_back_addr":"172.21.15.71:6815/3095380305","heartbeat_front_addr":"172.21.15.71:6813/3095380305","state":["exists","up"]},{"osd":5,"uuid":"eb48160e-ea7c-42f7-9477-be110bc8a34e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6809","nonce":4195149262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6811","nonce":4195149262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6815","nonce":4195149262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6813","nonce":4195149262}]},"public_addr":"172.21.15.142:6809/4195149262","cluster_addr":"172.21.15.142:6811/4195149262","heartbeat_back_addr":"172.21.15.142:6815/4195149262","heartbeat_front_addr":"172.21.15.142:6813/4195149262","state":["exists","up"]},{"osd":6,"uuid":"0ec4501a-5853-4ca6-b325-231f5b72335e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6819","nonce":1560153428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6821","nonce":1560153428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6825","nonce":1560153428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6823","nonce":1560153428}]},"public_addr":"172.21.15.1:6819/1560153428","cluster_addr":"172.21.15.1:6821/1560153428","heartbeat_back_addr":"172.21.15.1:6825/1560153428","heartbeat_front_addr":"172.21.15.1:6823/1560153428","state":["exists","up"]},{"osd":7,"uuid":"029debe8-9cdf-44f6-9aea-cabd1d185f93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6817","nonce":4080955538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6819","nonce":4080955538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6823","nonce":4080955538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6821","nonce":4080955538}]},"public_addr":"172.21.15.71:6817/4080955538","cluster_addr":"172.21.15.71:6819/4080955538","heartbeat_back_addr":"172.21.15.71:6823/4080955538","heartbeat_front_addr":"172.21.15.71:6821/4080955538","state":["exists","up"]},{"osd":8,"uuid":"7c90896c-9955-4c1b-aaca-7531dd5457f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6817","nonce":3942038209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6819","nonce":3942038209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6823","nonce":3942038209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6821","nonce":3942038209}]},"public_addr":"172.21.15.142:6817/3942038209","cluster_addr":"172.21.15.142:6819/3942038209","heartbeat_back_addr":"172.21.15.142:6823/3942038209","heartbeat_front_addr":"172.21.15.142:6821/3942038209","state":["exists","up"]},{"osd":9,"uuid":"7e165e68-31a8-4598-a577-de3e90ae5c5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6827","nonce":2716544850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6829","nonce":2716544850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6833","nonce":2716544850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6831","nonce":2716544850}]},"public_addr":"172.21.15.1:6827/2716544850","cluster_addr":"172.21.15.1:6829/2716544850","heartbeat_back_addr":"172.21.15.1:6833/2716544850","heartbeat_front_addr":"172.21.15.1:6831/2716544850","state":["exists","up"]},{"osd":10,"uuid":"48cd5fdb-0887-444a-aab8-a1de24282bb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6825","nonce":84691841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6827","nonce":84691841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6831","nonce":84691841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6829","nonce":84691841}]},"public_addr":"172.21.15.71:6825/84691841","cluster_addr":"172.21.15.71:6827/84691841","heartbeat_back_addr":"172.21.15.71:6831/84691841","heartbeat_front_addr":"172.21.15.71:6829/84691841","state":["exists","up"]},{"osd":11,"uuid":"f011dd27-11b0-4166-adfe-76756533163a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6825","nonce":2290179934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6827","nonce":2290179934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6831","nonce":2290179934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6829","nonce":2290179934}]},"public_addr":"172.21.15.142:6825/2290179934","cluster_addr":"172.21.15.142:6827/2290179934","heartbeat_back_addr":"172.21.15.142:6831/2290179934","heartbeat_front_addr":"172.21.15.142:6829/2290179934","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:20:44.060692+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:01.903817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:19.368258+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:40.028037+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:57.513361+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:22:14.805944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:22:35.188439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:22:53.014412+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:23:10.507420+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:23:31.849765+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:23:50.070612+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:24:07.388625+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":5,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/4092532060":"2024-02-03T15:19:25.836171+0000","172.21.15.1:6801/395498296":"2024-02-03T15:19:25.836171+0000","172.21.15.1:6800/395498296":"2024-02-03T15:19:25.836171+0000","172.21.15.1:0/1575373489":"2024-02-03T15:19:25.836171+0000","172.21.15.1:0/865983374":"2024-02-03T15:19:01.431596+0000","172.21.15.1:0/3165960227":"2024-02-03T15:19:01.431596+0000","172.21.15.1:0/2350830293":"2024-02-03T15:19:01.431596+0000","172.21.15.1:0/2959690250":"2024-02-03T15:19:25.836171+0000","172.21.15.1:6800/1162530732":"2024-02-03T15:19:01.431596+0000","172.21.15.1:6801/1162530732":"2024-02-03T15:19:01.431596+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T15:25:32.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:32.099+0000 7fdf95600700 1 -- 172.21.15.1:0/3285676145 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fdf700a05b0 msgr2=0x7fdf700a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:32.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:32.099+0000 7fdf95600700 1 --2- 172.21.15.1:0/3285676145 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fdf700a05b0 0x7fdf700a2a70 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fdf78000c00 tx=0x7fdf780058e0 comp rx=0 tx=0).stop 2024-02-02T15:25:32.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:32.099+0000 7fdf95600700 1 -- 172.21.15.1:0/3285676145 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf90068860 msgr2=0x7fdf90198fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:32.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:32.099+0000 7fdf95600700 1 --2- 172.21.15.1:0/3285676145 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf90068860 0x7fdf90198fe0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fdf8000e9c0 tx=0x7fdf8000ecd0 comp rx=0 tx=0).stop 2024-02-02T15:25:32.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:32.099+0000 7fdf95600700 1 -- 172.21.15.1:0/3285676145 shutdown_connections 2024-02-02T15:25:32.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:32.099+0000 7fdf95600700 1 --2- 172.21.15.1:0/3285676145 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fdf700a05b0 0x7fdf700a2a70 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:32.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:32.099+0000 7fdf95600700 1 --2- 172.21.15.1:0/3285676145 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf90106ed0 0x7fdf9019d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:32.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:32.099+0000 7fdf95600700 1 --2- 172.21.15.1:0/3285676145 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf90068860 0x7fdf90198fe0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:32.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:32.099+0000 7fdf95600700 1 --2- 172.21.15.1:0/3285676145 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf90069220 0x7fdf90199520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:32.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:32.099+0000 7fdf95600700 1 -- 172.21.15.1:0/3285676145 >> 172.21.15.1:0/3285676145 conn(0x7fdf90076d40 msgr2=0x7fdf9010beb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:32.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:32.099+0000 7fdf95600700 1 -- 172.21.15.1:0/3285676145 shutdown_connections 2024-02-02T15:25:32.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:32.099+0000 7fdf95600700 1 -- 172.21.15.1:0/3285676145 wait complete. 2024-02-02T15:25:32.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:32 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/3285676145' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-02T15:25:32.533 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-02-02T15:25:32.533 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-02-02T15:25:32.534 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-02-02T15:25:32.534 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-02-02T15:25:32.535 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-02-02T15:25:32.535 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-02-02T15:25:32.535 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2024-02-02T15:25:32.536 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2024-02-02T15:25:32.536 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph tell osd.8 flush_pg_stats 2024-02-02T15:25:32.537 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph tell osd.9 flush_pg_stats 2024-02-02T15:25:32.537 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph tell osd.10 flush_pg_stats 2024-02-02T15:25:32.537 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph tell osd.11 flush_pg_stats 2024-02-02T15:25:32.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:32 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/3285676145' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-02T15:25:32.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:32 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/3285676145' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-02T15:25:33.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:33 smithi071 ceph-mon[44707]: pgmap v259: 129 pgs: 129 active+clean; 634 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 2.5 KiB/s wr, 13 op/s; 8 B/s, 1 keys/s, 1 objects/s recovering 2024-02-02T15:25:33.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:33 smithi142 ceph-mon[45728]: pgmap v259: 129 pgs: 129 active+clean; 634 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 2.5 KiB/s wr, 13 op/s; 8 B/s, 1 keys/s, 1 objects/s recovering 2024-02-02T15:25:33.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:33 smithi001 ceph-mon[35470]: pgmap v259: 129 pgs: 129 active+clean; 634 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 2.5 KiB/s wr, 13 op/s; 8 B/s, 1 keys/s, 1 objects/s recovering 2024-02-02T15:25:33.982 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.a/config 2024-02-02T15:25:34.800 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.a/config 2024-02-02T15:25:35.304 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.a/config 2024-02-02T15:25:35.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:35.384+0000 7fdc41800700 1 -- 172.21.15.1:0/3262122981 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdc3c075270 msgr2=0x7fdc3c075650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:35.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:35.384+0000 7fdc41800700 1 --2- 172.21.15.1:0/3262122981 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdc3c075270 0x7fdc3c075650 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c009990 tx=0x7fdc2c009ca0 comp rx=0 tx=0).stop 2024-02-02T15:25:35.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:35.384+0000 7fdc41800700 1 -- 172.21.15.1:0/3262122981 shutdown_connections 2024-02-02T15:25:35.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:35.384+0000 7fdc41800700 1 --2- 172.21.15.1:0/3262122981 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc3c111c30 0x7fdc3c114020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:35.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:35.384+0000 7fdc41800700 1 --2- 172.21.15.1:0/3262122981 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdc3c075270 0x7fdc3c075650 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:35.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:35.384+0000 7fdc41800700 1 --2- 172.21.15.1:0/3262122981 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdc3c075b90 0x7fdc3c1116f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:35.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:35.384+0000 7fdc41800700 1 -- 172.21.15.1:0/3262122981 >> 172.21.15.1:0/3262122981 conn(0x7fdc3c0fea40 msgr2=0x7fdc3c100e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:35.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:35.384+0000 7fdc41800700 1 -- 172.21.15.1:0/3262122981 shutdown_connections 2024-02-02T15:25:35.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:35.384+0000 7fdc41800700 1 -- 172.21.15.1:0/3262122981 wait complete. 2024-02-02T15:25:35.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:35.385+0000 7fdc41800700 1 Processor -- start 2024-02-02T15:25:35.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:35.386+0000 7fdc41800700 1 -- start start 2024-02-02T15:25:35.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:35.386+0000 7fdc41800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdc3c075270 0x7fdc3c19d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:35.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:35.386+0000 7fdc41800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdc3c075b90 0x7fdc3c19d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:35.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:35.386+0000 7fdc41800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc3c111c30 0x7fdc3c1a1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:35.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:35.386+0000 7fdc41800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc3c1a2450 con 0x7fdc3c075b90 2024-02-02T15:25:35.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:35.386+0000 7fdc41800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc3c1a25c0 con 0x7fdc3c075270 2024-02-02T15:25:35.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:35.386+0000 7fdc41800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc3c1a2780 con 0x7fdc3c111c30 2024-02-02T15:25:35.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:35.386+0000 7fdc3a000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdc3c075b90 0x7fdc3c19d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:35.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:35.386+0000 7fdc3aa00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdc3c075270 0x7fdc3c19d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:35.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:35.386+0000 7fdc3b400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc3c111c30 0x7fdc3c1a1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:35.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:35.386+0000 7fdc3a000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdc3c075b90 0x7fdc3c19d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48206/0 (socket says 172.21.15.1:48206) 2024-02-02T15:25:35.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:35.386+0000 7fdc3a000700 1 -- 172.21.15.1:0/1641208843 learned_addr learned my addr 172.21.15.1:0/1641208843 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:25:35.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:35.387+0000 7fdc3b400700 1 -- 172.21.15.1:0/1641208843 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdc3c075270 msgr2=0x7fdc3c19d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:35.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:35.387+0000 7fdc3b400700 1 --2- 172.21.15.1:0/1641208843 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdc3c075270 0x7fdc3c19d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:35.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:35.387+0000 7fdc3b400700 1 -- 172.21.15.1:0/1641208843 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdc3c075b90 msgr2=0x7fdc3c19d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:35.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:35.387+0000 7fdc3b400700 1 --2- 172.21.15.1:0/1641208843 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdc3c075b90 0x7fdc3c19d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:35.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:35.387+0000 7fdc3b400700 1 -- 172.21.15.1:0/1641208843 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc2c0095f0 con 0x7fdc3c111c30 2024-02-02T15:25:35.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:35.387+0000 7fdc3b400700 1 --2- 172.21.15.1:0/1641208843 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc3c111c30 0x7fdc3c1a1e40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c079560 tx=0x7fdc3400ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:35.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:35.387+0000 7fdc33400700 1 -- 172.21.15.1:0/1641208843 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc34018070 con 0x7fdc3c111c30 2024-02-02T15:25:35.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:35.387+0000 7fdc41800700 1 -- 172.21.15.1:0/1641208843 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc3c1a2a00 con 0x7fdc3c111c30 2024-02-02T15:25:35.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:35.387+0000 7fdc33400700 1 -- 172.21.15.1:0/1641208843 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdc3400f040 con 0x7fdc3c111c30 2024-02-02T15:25:35.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:35.387+0000 7fdc33400700 1 -- 172.21.15.1:0/1641208843 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc3400ccd0 con 0x7fdc3c111c30 2024-02-02T15:25:35.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:35.388+0000 7fdc41800700 1 -- 172.21.15.1:0/1641208843 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc3c1aa2c0 con 0x7fdc3c111c30 2024-02-02T15:25:35.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:35.388+0000 7fdc41800700 1 -- 172.21.15.1:0/1641208843 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fdc3c04ea40 con 0x7fdc3c111c30 2024-02-02T15:25:35.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:35.391+0000 7fdc33400700 1 -- 172.21.15.1:0/1641208843 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 17) v1 ==== 135120+0+0 (secure 0 0 0) 0x7fdc34007500 con 0x7fdc3c111c30 2024-02-02T15:25:35.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:35.392+0000 7fdc33400700 1 --2- 172.21.15.1:0/1641208843 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fdc1c0a0630 0x7fdc1c0a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:35.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:35.392+0000 7fdc33400700 1 -- 172.21.15.1:0/1641208843 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fdc340cc220 con 0x7fdc3c111c30 2024-02-02T15:25:35.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:35.392+0000 7fdc3aa00700 1 --2- 172.21.15.1:0/1641208843 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fdc1c0a0630 0x7fdc1c0a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:35.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:35.392+0000 7fdc33400700 1 --2- 172.21.15.1:0/1641208843 >> [v2:172.21.15.1:6810/1086124477,v1:172.21.15.1:6811/1086124477] conn(0x7fdc1c0ab650 0x7fdc1c0adaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:35.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:35.392+0000 7fdc33400700 1 -- 172.21.15.1:0/1641208843 --> [v2:172.21.15.1:6810/1086124477,v1:172.21.15.1:6811/1086124477] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdc1c0ae1c0 con 0x7fdc1c0ab650 2024-02-02T15:25:35.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:35.393+0000 7fdc33400700 1 -- 172.21.15.1:0/1641208843 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc340cc630 con 0x7fdc3c111c30 2024-02-02T15:25:35.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:35.393+0000 7fdc3a000700 1 --2- 172.21.15.1:0/1641208843 >> [v2:172.21.15.1:6810/1086124477,v1:172.21.15.1:6811/1086124477] conn(0x7fdc1c0ab650 0x7fdc1c0adaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:35.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:35.393+0000 7fdc3aa00700 1 --2- 172.21.15.1:0/1641208843 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fdc1c0a0630 0x7fdc1c0a2af0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c00b2e0 tx=0x7fdc2c005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:35.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:35.394+0000 7fdc3a000700 1 --2- 172.21.15.1:0/1641208843 >> [v2:172.21.15.1:6810/1086124477,v1:172.21.15.1:6811/1086124477] conn(0x7fdc1c0ab650 0x7fdc1c0adaf0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:35.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:35.394+0000 7fdc33400700 1 -- 172.21.15.1:0/1641208843 <== osd.3 v2:172.21.15.1:6810/1086124477 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fdc1c0ae1c0 con 0x7fdc1c0ab650 2024-02-02T15:25:35.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:35.411+0000 7fdc41800700 1 -- 172.21.15.1:0/1641208843 --> [v2:172.21.15.1:6810/1086124477,v1:172.21.15.1:6811/1086124477] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fdc3c117680 con 0x7fdc1c0ab650 2024-02-02T15:25:35.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:35.412+0000 7fdc33400700 1 -- 172.21.15.1:0/1641208843 <== osd.3 v2:172.21.15.1:6810/1086124477 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fdc24009e90 con 0x7fdc1c0ab650 2024-02-02T15:25:35.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:35.412+0000 7fdc41800700 1 -- 172.21.15.1:0/1641208843 >> [v2:172.21.15.1:6810/1086124477,v1:172.21.15.1:6811/1086124477] conn(0x7fdc1c0ab650 msgr2=0x7fdc1c0adaf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:35.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:35.412+0000 7fdc41800700 1 --2- 172.21.15.1:0/1641208843 >> [v2:172.21.15.1:6810/1086124477,v1:172.21.15.1:6811/1086124477] conn(0x7fdc1c0ab650 0x7fdc1c0adaf0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:35.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:35.412+0000 7fdc41800700 1 -- 172.21.15.1:0/1641208843 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fdc1c0a0630 msgr2=0x7fdc1c0a2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:35.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:35.413+0000 7fdc41800700 1 --2- 172.21.15.1:0/1641208843 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fdc1c0a0630 0x7fdc1c0a2af0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c00b2e0 tx=0x7fdc2c005fb0 comp rx=0 tx=0).stop 2024-02-02T15:25:35.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:35.413+0000 7fdc41800700 1 -- 172.21.15.1:0/1641208843 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc3c111c30 msgr2=0x7fdc3c1a1e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:35.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:35.413+0000 7fdc41800700 1 --2- 172.21.15.1:0/1641208843 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdc3c111c30 0x7fdc3c1a1e40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c079560 tx=0x7fdc3400ef90 comp rx=0 tx=0).stop 2024-02-02T15:25:35.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:35.413+0000 7fdc3b400700 1 -- 172.21.15.1:0/1641208843 reap_dead start 2024-02-02T15:25:35.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:35.413+0000 7fdc41800700 1 -- 172.21.15.1:0/1641208843 shutdown_connections 2024-02-02T15:25:35.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:35.413+0000 7fdc41800700 1 -- 172.21.15.1:0/1641208843 >> 172.21.15.1:0/1641208843 conn(0x7fdc3c0fea40 msgr2=0x7fdc3c100e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:35.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:35.413+0000 7fdc41800700 1 -- 172.21.15.1:0/1641208843 shutdown_connections 2024-02-02T15:25:35.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:35.413+0000 7fdc41800700 1 -- 172.21.15.1:0/1641208843 wait complete. 2024-02-02T15:25:35.575 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:35 smithi001 ceph-mon[35470]: pgmap v260: 129 pgs: 129 active+clean; 634 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 2.4 KiB/s wr, 12 op/s; 8 B/s, 1 keys/s, 1 objects/s recovering 2024-02-02T15:25:35.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:35 smithi071 ceph-mon[44707]: pgmap v260: 129 pgs: 129 active+clean; 634 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 2.4 KiB/s wr, 12 op/s; 8 B/s, 1 keys/s, 1 objects/s recovering 2024-02-02T15:25:35.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:35 smithi142 ceph-mon[45728]: pgmap v260: 129 pgs: 129 active+clean; 634 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 2.4 KiB/s wr, 12 op/s; 8 B/s, 1 keys/s, 1 objects/s recovering 2024-02-02T15:25:35.852 INFO:teuthology.orchestra.run.smithi001.stdout:103079215152 2024-02-02T15:25:35.853 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-02-02T15:25:36.069 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.a/config 2024-02-02T15:25:36.371 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.a/config 2024-02-02T15:25:36.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:36.465+0000 7f2121000700 1 -- 172.21.15.1:0/2036617926 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f211c073170 msgr2=0x7f211c073550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:36.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:36.465+0000 7f2121000700 1 --2- 172.21.15.1:0/2036617926 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f211c073170 0x7f211c073550 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f210c009990 tx=0x7f210c009ca0 comp rx=0 tx=0).stop 2024-02-02T15:25:36.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:36.465+0000 7f2121000700 1 -- 172.21.15.1:0/2036617926 shutdown_connections 2024-02-02T15:25:36.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:36.465+0000 7f2121000700 1 --2- 172.21.15.1:0/2036617926 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f211c10d5a0 0x7f211c10fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:36.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:36.465+0000 7f2121000700 1 --2- 172.21.15.1:0/2036617926 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f211c073170 0x7f211c073550 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:36.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:36.465+0000 7f2121000700 1 --2- 172.21.15.1:0/2036617926 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f211c073a90 0x7f211c10d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:36.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:36.465+0000 7f2121000700 1 -- 172.21.15.1:0/2036617926 >> 172.21.15.1:0/2036617926 conn(0x7f211c0fc830 msgr2=0x7f211c0fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:36.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:36.465+0000 7f2121000700 1 -- 172.21.15.1:0/2036617926 shutdown_connections 2024-02-02T15:25:36.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:36.465+0000 7f2121000700 1 -- 172.21.15.1:0/2036617926 wait complete. 2024-02-02T15:25:36.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:36.466+0000 7f2121000700 1 Processor -- start 2024-02-02T15:25:36.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:36.466+0000 7f2121000700 1 -- start start 2024-02-02T15:25:36.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:36.467+0000 7f2121000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f211c073170 0x7f211c1990a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:36.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:36.467+0000 7f2121000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f211c073a90 0x7f211c1995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:36.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:36.467+0000 7f2121000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f211c10d5a0 0x7f211c19da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:36.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:36.467+0000 7f2121000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f211c19e060 con 0x7f211c073a90 2024-02-02T15:25:36.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:36.467+0000 7f2121000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f211c19e1d0 con 0x7f211c073170 2024-02-02T15:25:36.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:36.467+0000 7f2121000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f211c19e390 con 0x7f211c10d5a0 2024-02-02T15:25:36.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:36.467+0000 7f2119e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f211c073170 0x7f211c1990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:36.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:36.467+0000 7f2119400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f211c073a90 0x7f211c1995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:36.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:36.467+0000 7f2119e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f211c073170 0x7f211c1990a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:47116/0 (socket says 172.21.15.1:47116) 2024-02-02T15:25:36.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:36.467+0000 7f2119e00700 1 -- 172.21.15.1:0/512565864 learned_addr learned my addr 172.21.15.1:0/512565864 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:25:36.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:36.467+0000 7f211a800700 1 --2- 172.21.15.1:0/512565864 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f211c10d5a0 0x7f211c19da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:36.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:36.468+0000 7f2119e00700 1 -- 172.21.15.1:0/512565864 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f211c10d5a0 msgr2=0x7f211c19da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:36.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:36.468+0000 7f2119e00700 1 --2- 172.21.15.1:0/512565864 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f211c10d5a0 0x7f211c19da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:36.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:36.468+0000 7f2119e00700 1 -- 172.21.15.1:0/512565864 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f211c073a90 msgr2=0x7f211c1995e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:36.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:36.468+0000 7f2119e00700 1 --2- 172.21.15.1:0/512565864 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f211c073a90 0x7f211c1995e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:36.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:36.468+0000 7f2119e00700 1 -- 172.21.15.1:0/512565864 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f210c0095f0 con 0x7f211c073170 2024-02-02T15:25:36.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:36.468+0000 7f211a800700 1 --2- 172.21.15.1:0/512565864 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f211c10d5a0 0x7f211c19da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:25:36.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:36.468+0000 7f2119e00700 1 --2- 172.21.15.1:0/512565864 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f211c073170 0x7f211c1990a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f210c005910 tx=0x7f210c005720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:36.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:36.468+0000 7f2112a00700 1 -- 172.21.15.1:0/512565864 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f210c01d070 con 0x7f211c073170 2024-02-02T15:25:36.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:36.468+0000 7f2121000700 1 -- 172.21.15.1:0/512565864 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f211c19e610 con 0x7f211c073170 2024-02-02T15:25:36.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:36.468+0000 7f2112a00700 1 -- 172.21.15.1:0/512565864 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f210c022470 con 0x7f211c073170 2024-02-02T15:25:36.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:36.468+0000 7f2112a00700 1 -- 172.21.15.1:0/512565864 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f210c017650 con 0x7f211c073170 2024-02-02T15:25:36.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:36.469+0000 7f2121000700 1 -- 172.21.15.1:0/512565864 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f211c1a5ed0 con 0x7f211c073170 2024-02-02T15:25:36.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:36.469+0000 7f2121000700 1 -- 172.21.15.1:0/512565864 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f211c1131c0 con 0x7f211c073170 2024-02-02T15:25:36.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:36.471+0000 7f2112a00700 1 -- 172.21.15.1:0/512565864 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 135120+0+0 (secure 0 0 0) 0x7f210c0225e0 con 0x7f211c073170 2024-02-02T15:25:36.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:36.473+0000 7f2112a00700 1 --2- 172.21.15.1:0/512565864 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f20fc0a0630 0x7f20fc0a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:36.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:36.473+0000 7f2112a00700 1 -- 172.21.15.1:0/512565864 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f210c0cdb80 con 0x7f211c073170 2024-02-02T15:25:36.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:36.473+0000 7f2119400700 1 --2- 172.21.15.1:0/512565864 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f20fc0a0630 0x7f20fc0a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:36.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:36.473+0000 7f2112a00700 1 --2- 172.21.15.1:0/512565864 >> [v2:172.21.15.142:6800/377511414,v1:172.21.15.142:6801/377511414] conn(0x7f20fc0ab650 0x7f20fc0adaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:36.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:36.473+0000 7f2112a00700 1 -- 172.21.15.1:0/512565864 --> [v2:172.21.15.142:6800/377511414,v1:172.21.15.142:6801/377511414] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f20fc0ae1c0 con 0x7f20fc0ab650 2024-02-02T15:25:36.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:36.473+0000 7f2112a00700 1 -- 172.21.15.1:0/512565864 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f210c0cdef0 con 0x7f211c073170 2024-02-02T15:25:36.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:36.474+0000 7f211a800700 1 --2- 172.21.15.1:0/512565864 >> [v2:172.21.15.142:6800/377511414,v1:172.21.15.142:6801/377511414] conn(0x7f20fc0ab650 0x7f20fc0adaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:36.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:36.474+0000 7f2119400700 1 --2- 172.21.15.1:0/512565864 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f20fc0a0630 0x7f20fc0a2af0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f211c19a6c0 tx=0x7f2104009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:36.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:36.474+0000 7f211a800700 1 --2- 172.21.15.1:0/512565864 >> [v2:172.21.15.142:6800/377511414,v1:172.21.15.142:6801/377511414] conn(0x7f20fc0ab650 0x7f20fc0adaf0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:36.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:36.475+0000 7f2112a00700 1 -- 172.21.15.1:0/512565864 <== osd.2 v2:172.21.15.142:6800/377511414 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f20fc0ae1c0 con 0x7f20fc0ab650 2024-02-02T15:25:36.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:36.493+0000 7f2121000700 1 -- 172.21.15.1:0/512565864 --> [v2:172.21.15.142:6800/377511414,v1:172.21.15.142:6801/377511414] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f211c04ea40 con 0x7f20fc0ab650 2024-02-02T15:25:36.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:36.493+0000 7f2112a00700 1 -- 172.21.15.1:0/512565864 <== osd.2 v2:172.21.15.142:6800/377511414 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f211c04ea40 con 0x7f20fc0ab650 2024-02-02T15:25:36.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:36.494+0000 7f2121000700 1 -- 172.21.15.1:0/512565864 >> [v2:172.21.15.142:6800/377511414,v1:172.21.15.142:6801/377511414] conn(0x7f20fc0ab650 msgr2=0x7f20fc0adaf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:36.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:36.494+0000 7f2121000700 1 --2- 172.21.15.1:0/512565864 >> [v2:172.21.15.142:6800/377511414,v1:172.21.15.142:6801/377511414] conn(0x7f20fc0ab650 0x7f20fc0adaf0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:36.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:36.494+0000 7f2121000700 1 -- 172.21.15.1:0/512565864 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f20fc0a0630 msgr2=0x7f20fc0a2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:36.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:36.494+0000 7f2121000700 1 --2- 172.21.15.1:0/512565864 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f20fc0a0630 0x7f20fc0a2af0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f211c19a6c0 tx=0x7f2104009260 comp rx=0 tx=0).stop 2024-02-02T15:25:36.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:36.494+0000 7f2121000700 1 -- 172.21.15.1:0/512565864 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f211c073170 msgr2=0x7f211c1990a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:36.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:36.494+0000 7f2121000700 1 --2- 172.21.15.1:0/512565864 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f211c073170 0x7f211c1990a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f210c005910 tx=0x7f210c005720 comp rx=0 tx=0).stop 2024-02-02T15:25:36.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:36.494+0000 7f211a800700 1 -- 172.21.15.1:0/512565864 reap_dead start 2024-02-02T15:25:36.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:36.494+0000 7f2121000700 1 -- 172.21.15.1:0/512565864 shutdown_connections 2024-02-02T15:25:36.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:36.494+0000 7f2121000700 1 -- 172.21.15.1:0/512565864 >> 172.21.15.1:0/512565864 conn(0x7f211c0fc830 msgr2=0x7f211c10f770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:36.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:36.494+0000 7f2121000700 1 -- 172.21.15.1:0/512565864 shutdown_connections 2024-02-02T15:25:36.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:36.494+0000 7f2121000700 1 -- 172.21.15.1:0/512565864 wait complete. 2024-02-02T15:25:36.621 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.a/config 2024-02-02T15:25:36.672 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.a/config 2024-02-02T15:25:36.986 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.a/config 2024-02-02T15:25:37.490 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:37 smithi001 ceph-mon[35470]: pgmap v261: 129 pgs: 129 active+clean; 634 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.3 KiB/s wr, 8 op/s; 7 B/s, 1 keys/s, 0 objects/s recovering 2024-02-02T15:25:37.511 INFO:teuthology.orchestra.run.smithi001.stdout:77309411381 2024-02-02T15:25:37.511 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-02-02T15:25:37.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:37 smithi071 ceph-mon[44707]: pgmap v261: 129 pgs: 129 active+clean; 634 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.3 KiB/s wr, 8 op/s; 7 B/s, 1 keys/s, 0 objects/s recovering 2024-02-02T15:25:37.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:37 smithi142 ceph-mon[45728]: pgmap v261: 129 pgs: 129 active+clean; 634 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.3 KiB/s wr, 8 op/s; 7 B/s, 1 keys/s, 0 objects/s recovering 2024-02-02T15:25:38.635 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.a/config 2024-02-02T15:25:38.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.534+0000 7f45b2800700 1 -- 172.21.15.1:0/3675462 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45ac073170 msgr2=0x7f45ac073550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:38.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.534+0000 7f45b2800700 1 --2- 172.21.15.1:0/3675462 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45ac073170 0x7f45ac073550 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f459c009990 tx=0x7f459c009ca0 comp rx=0 tx=0).stop 2024-02-02T15:25:38.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.534+0000 7f45b2800700 1 -- 172.21.15.1:0/3675462 shutdown_connections 2024-02-02T15:25:38.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.534+0000 7f45b2800700 1 --2- 172.21.15.1:0/3675462 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45ac10d5a0 0x7f45ac10fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:38.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.534+0000 7f45b2800700 1 --2- 172.21.15.1:0/3675462 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45ac073170 0x7f45ac073550 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:38.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.534+0000 7f45b2800700 1 --2- 172.21.15.1:0/3675462 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f45ac073a90 0x7f45ac10d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:38.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.534+0000 7f45b2800700 1 -- 172.21.15.1:0/3675462 >> 172.21.15.1:0/3675462 conn(0x7f45ac0fc830 msgr2=0x7f45ac0fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:38.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.534+0000 7f45b2800700 1 -- 172.21.15.1:0/3675462 shutdown_connections 2024-02-02T15:25:38.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.534+0000 7f45b2800700 1 -- 172.21.15.1:0/3675462 wait complete. 2024-02-02T15:25:38.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.535+0000 7f45b2800700 1 Processor -- start 2024-02-02T15:25:38.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.535+0000 7f45b2800700 1 -- start start 2024-02-02T15:25:38.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.536+0000 7f45b2800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45ac073170 0x7f45ac199020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:38.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.536+0000 7f45b2800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f45ac073a90 0x7f45ac199560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:38.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.536+0000 7f45b2800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45ac10d5a0 0x7f45ac19d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:38.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.536+0000 7f45b2800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45ac19dfe0 con 0x7f45ac073a90 2024-02-02T15:25:38.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.536+0000 7f45b2800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45ac19e150 con 0x7f45ac10d5a0 2024-02-02T15:25:38.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.536+0000 7f45b2800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45ac19e310 con 0x7f45ac073170 2024-02-02T15:25:38.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.536+0000 7f45aaa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f45ac073a90 0x7f45ac199560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:38.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.536+0000 7f45ab400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45ac073170 0x7f45ac199020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:38.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.536+0000 7f45aaa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f45ac073a90 0x7f45ac199560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48250/0 (socket says 172.21.15.1:48250) 2024-02-02T15:25:38.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.536+0000 7f45aaa00700 1 -- 172.21.15.1:0/2583822636 learned_addr learned my addr 172.21.15.1:0/2583822636 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:25:38.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.537+0000 7f45abe00700 1 --2- 172.21.15.1:0/2583822636 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45ac10d5a0 0x7f45ac19d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:38.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.537+0000 7f45ab400700 1 -- 172.21.15.1:0/2583822636 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45ac10d5a0 msgr2=0x7f45ac19d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:38.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.537+0000 7f45ab400700 1 --2- 172.21.15.1:0/2583822636 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45ac10d5a0 0x7f45ac19d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:38.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.537+0000 7f45ab400700 1 -- 172.21.15.1:0/2583822636 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f45ac073a90 msgr2=0x7f45ac199560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:38.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.537+0000 7f45ab400700 1 --2- 172.21.15.1:0/2583822636 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f45ac073a90 0x7f45ac199560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:38.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.537+0000 7f45ab400700 1 -- 172.21.15.1:0/2583822636 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f459c0095f0 con 0x7f45ac073170 2024-02-02T15:25:38.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.537+0000 7f45ab400700 1 --2- 172.21.15.1:0/2583822636 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45ac073170 0x7f45ac199020 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f459c00b2e0 tx=0x7f459c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:38.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.538+0000 7f45a3e00700 1 -- 172.21.15.1:0/2583822636 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f459c01d070 con 0x7f45ac073170 2024-02-02T15:25:38.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.538+0000 7f45a3e00700 1 -- 172.21.15.1:0/2583822636 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f459c005030 con 0x7f45ac073170 2024-02-02T15:25:38.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.538+0000 7f45a3e00700 1 -- 172.21.15.1:0/2583822636 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f459c004a00 con 0x7f45ac073170 2024-02-02T15:25:38.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.538+0000 7f45b2800700 1 -- 172.21.15.1:0/2583822636 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45ac19e590 con 0x7f45ac073170 2024-02-02T15:25:38.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.538+0000 7f45b2800700 1 -- 172.21.15.1:0/2583822636 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45ac1a5c80 con 0x7f45ac073170 2024-02-02T15:25:38.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.539+0000 7f45b2800700 1 -- 172.21.15.1:0/2583822636 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f45ac113170 con 0x7f45ac073170 2024-02-02T15:25:38.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.542+0000 7f45a3e00700 1 -- 172.21.15.1:0/2583822636 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 17) v1 ==== 135120+0+0 (secure 0 0 0) 0x7f459c005430 con 0x7f45ac073170 2024-02-02T15:25:38.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.543+0000 7f45a3e00700 1 --2- 172.21.15.1:0/2583822636 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f458c0a0630 0x7f458c0a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:38.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.543+0000 7f45a3e00700 1 -- 172.21.15.1:0/2583822636 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f459c0ccd50 con 0x7f45ac073170 2024-02-02T15:25:38.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.543+0000 7f45aaa00700 1 --2- 172.21.15.1:0/2583822636 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f458c0a0630 0x7f458c0a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:38.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.543+0000 7f45a3e00700 1 --2- 172.21.15.1:0/2583822636 >> [v2:172.21.15.71:6800/747581719,v1:172.21.15.71:6801/747581719] conn(0x7f458c0ab650 0x7f458c0adaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:38.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.543+0000 7f45a3e00700 1 -- 172.21.15.1:0/2583822636 --> [v2:172.21.15.71:6800/747581719,v1:172.21.15.71:6801/747581719] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f458c0ae1c0 con 0x7f458c0ab650 2024-02-02T15:25:38.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.543+0000 7f45a3e00700 1 -- 172.21.15.1:0/2583822636 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f459c0cf330 con 0x7f45ac073170 2024-02-02T15:25:38.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.544+0000 7f45aaa00700 1 --2- 172.21.15.1:0/2583822636 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f458c0a0630 0x7f458c0a2af0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f45ac19a640 tx=0x7f4594005f90 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:38.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.544+0000 7f45abe00700 1 --2- 172.21.15.1:0/2583822636 >> [v2:172.21.15.71:6800/747581719,v1:172.21.15.71:6801/747581719] conn(0x7f458c0ab650 0x7f458c0adaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:38.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.544+0000 7f45abe00700 1 --2- 172.21.15.1:0/2583822636 >> [v2:172.21.15.71:6800/747581719,v1:172.21.15.71:6801/747581719] conn(0x7f458c0ab650 0x7f458c0adaf0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:38.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.545+0000 7f45a3e00700 1 -- 172.21.15.1:0/2583822636 <== osd.1 v2:172.21.15.71:6800/747581719 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f458c0ae1c0 con 0x7f458c0ab650 2024-02-02T15:25:38.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.563+0000 7f45b2800700 1 -- 172.21.15.1:0/2583822636 --> [v2:172.21.15.71:6800/747581719,v1:172.21.15.71:6801/747581719] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f45ac04ea40 con 0x7f458c0ab650 2024-02-02T15:25:38.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.564+0000 7f45a3e00700 1 -- 172.21.15.1:0/2583822636 <== osd.1 v2:172.21.15.71:6800/747581719 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f45ac04ea40 con 0x7f458c0ab650 2024-02-02T15:25:38.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.564+0000 7f45b2800700 1 -- 172.21.15.1:0/2583822636 >> [v2:172.21.15.71:6800/747581719,v1:172.21.15.71:6801/747581719] conn(0x7f458c0ab650 msgr2=0x7f458c0adaf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:38.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.564+0000 7f45b2800700 1 --2- 172.21.15.1:0/2583822636 >> [v2:172.21.15.71:6800/747581719,v1:172.21.15.71:6801/747581719] conn(0x7f458c0ab650 0x7f458c0adaf0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:38.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.565+0000 7f45b2800700 1 -- 172.21.15.1:0/2583822636 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f458c0a0630 msgr2=0x7f458c0a2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:38.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.565+0000 7f45b2800700 1 --2- 172.21.15.1:0/2583822636 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f458c0a0630 0x7f458c0a2af0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f45ac19a640 tx=0x7f4594005f90 comp rx=0 tx=0).stop 2024-02-02T15:25:38.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.565+0000 7f45b2800700 1 -- 172.21.15.1:0/2583822636 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45ac073170 msgr2=0x7f45ac199020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:38.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.565+0000 7f45b2800700 1 --2- 172.21.15.1:0/2583822636 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45ac073170 0x7f45ac199020 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f459c00b2e0 tx=0x7f459c004730 comp rx=0 tx=0).stop 2024-02-02T15:25:38.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.565+0000 7f45abe00700 1 -- 172.21.15.1:0/2583822636 reap_dead start 2024-02-02T15:25:38.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.565+0000 7f45b2800700 1 -- 172.21.15.1:0/2583822636 shutdown_connections 2024-02-02T15:25:38.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.565+0000 7f45b2800700 1 -- 172.21.15.1:0/2583822636 >> 172.21.15.1:0/2583822636 conn(0x7f45ac0fc830 msgr2=0x7f45ac105000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:38.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.565+0000 7f45b2800700 1 -- 172.21.15.1:0/2583822636 shutdown_connections 2024-02-02T15:25:38.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.565+0000 7f45b2800700 1 -- 172.21.15.1:0/2583822636 wait complete. 2024-02-02T15:25:38.736 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.a/config 2024-02-02T15:25:39.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:39 smithi001 ceph-mon[35470]: pgmap v262: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.7 KiB/s wr, 10 op/s; 6 B/s, 0 keys/s, 0 objects/s recovering 2024-02-02T15:25:39.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:39 smithi071 ceph-mon[44707]: pgmap v262: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.7 KiB/s wr, 10 op/s; 6 B/s, 0 keys/s, 0 objects/s recovering 2024-02-02T15:25:39.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:39 smithi142 ceph-mon[45728]: pgmap v262: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 1.7 KiB/s wr, 10 op/s; 6 B/s, 0 keys/s, 0 objects/s recovering 2024-02-02T15:25:39.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.871+0000 7f3c78400700 1 -- 172.21.15.1:0/2485860223 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3c70106ed0 msgr2=0x7f3c701072b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:39.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.871+0000 7f3c78400700 1 --2- 172.21.15.1:0/2485860223 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3c70106ed0 0x7f3c701072b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3c60009990 tx=0x7f3c60009ca0 comp rx=0 tx=0).stop 2024-02-02T15:25:39.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.871+0000 7f3c78400700 1 -- 172.21.15.1:0/2485860223 shutdown_connections 2024-02-02T15:25:39.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.871+0000 7f3c78400700 1 --2- 172.21.15.1:0/2485860223 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c70069220 0x7f3c7010db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:39.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.871+0000 7f3c78400700 1 --2- 172.21.15.1:0/2485860223 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3c70106ed0 0x7f3c701072b0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:39.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.871+0000 7f3c78400700 1 --2- 172.21.15.1:0/2485860223 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c70068860 0x7f3c70068ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:39.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.871+0000 7f3c78400700 1 -- 172.21.15.1:0/2485860223 >> 172.21.15.1:0/2485860223 conn(0x7f3c70076d40 msgr2=0x7f3c70077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:39.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.871+0000 7f3c78400700 1 -- 172.21.15.1:0/2485860223 shutdown_connections 2024-02-02T15:25:39.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.871+0000 7f3c78400700 1 -- 172.21.15.1:0/2485860223 wait complete. 2024-02-02T15:25:39.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.872+0000 7f3c78400700 1 Processor -- start 2024-02-02T15:25:39.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.872+0000 7f3c78400700 1 -- start start 2024-02-02T15:25:39.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.872+0000 7f3c78400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c70068860 0x7f3c700ffcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:39.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.872+0000 7f3c78400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3c70069220 0x7f3c70100230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:39.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.873+0000 7f3c78400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c70106ed0 0x7f3c701a4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:39.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.873+0000 7f3c78400700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c70100b20 con 0x7f3c70069220 2024-02-02T15:25:39.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.873+0000 7f3c78400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c701a54e0 con 0x7f3c70106ed0 2024-02-02T15:25:39.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.873+0000 7f3c78400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c701a5650 con 0x7f3c70068860 2024-02-02T15:25:39.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.873+0000 7f3c75a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c70068860 0x7f3c700ffcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:39.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.873+0000 7f3c75000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3c70069220 0x7f3c70100230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:39.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.873+0000 7f3c75a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c70068860 0x7f3c700ffcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:45586/0 (socket says 172.21.15.1:45586) 2024-02-02T15:25:39.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.873+0000 7f3c75a00700 1 -- 172.21.15.1:0/3145883553 learned_addr learned my addr 172.21.15.1:0/3145883553 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:25:39.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.873+0000 7f3c76400700 1 --2- 172.21.15.1:0/3145883553 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c70106ed0 0x7f3c701a4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:39.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.873+0000 7f3c75a00700 1 -- 172.21.15.1:0/3145883553 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c70106ed0 msgr2=0x7f3c701a4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:39.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.873+0000 7f3c75a00700 1 --2- 172.21.15.1:0/3145883553 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c70106ed0 0x7f3c701a4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:39.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.873+0000 7f3c75a00700 1 -- 172.21.15.1:0/3145883553 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3c70069220 msgr2=0x7f3c70100230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:39.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.873+0000 7f3c75a00700 1 --2- 172.21.15.1:0/3145883553 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3c70069220 0x7f3c70100230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:39.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.873+0000 7f3c75a00700 1 -- 172.21.15.1:0/3145883553 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c600095f0 con 0x7f3c70068860 2024-02-02T15:25:39.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.873+0000 7f3c75a00700 1 --2- 172.21.15.1:0/3145883553 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c70068860 0x7f3c700ffcf0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3c60005910 tx=0x7f3c60005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:39.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.874+0000 7f3c6e000700 1 -- 172.21.15.1:0/3145883553 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c6001d070 con 0x7f3c70068860 2024-02-02T15:25:39.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.874+0000 7f3c6e000700 1 -- 172.21.15.1:0/3145883553 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3c60022470 con 0x7f3c70068860 2024-02-02T15:25:39.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.874+0000 7f3c78400700 1 -- 172.21.15.1:0/3145883553 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c701a5870 con 0x7f3c70068860 2024-02-02T15:25:39.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.874+0000 7f3c6e000700 1 -- 172.21.15.1:0/3145883553 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c60017650 con 0x7f3c70068860 2024-02-02T15:25:39.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.874+0000 7f3c78400700 1 -- 172.21.15.1:0/3145883553 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c701a5ce0 con 0x7f3c70068860 2024-02-02T15:25:39.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.875+0000 7f3c78400700 1 -- 172.21.15.1:0/3145883553 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3c7004ea40 con 0x7f3c70068860 2024-02-02T15:25:39.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.877+0000 7f3c6e000700 1 -- 172.21.15.1:0/3145883553 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 17) v1 ==== 135120+0+0 (secure 0 0 0) 0x7f3c6000fad0 con 0x7f3c70068860 2024-02-02T15:25:39.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.878+0000 7f3c6e000700 1 --2- 172.21.15.1:0/3145883553 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f3c500a0630 0x7f3c500a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:39.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.879+0000 7f3c6e000700 1 -- 172.21.15.1:0/3145883553 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f3c600cdc80 con 0x7f3c70068860 2024-02-02T15:25:39.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.879+0000 7f3c6e000700 1 --2- 172.21.15.1:0/3145883553 >> [v2:172.21.15.71:6816/4080955538,v1:172.21.15.71:6817/4080955538] conn(0x7f3c500ab650 0x7f3c500adaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:39.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.879+0000 7f3c6e000700 1 -- 172.21.15.1:0/3145883553 --> [v2:172.21.15.71:6816/4080955538,v1:172.21.15.71:6817/4080955538] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3c500ae1c0 con 0x7f3c500ab650 2024-02-02T15:25:39.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.879+0000 7f3c6e000700 1 -- 172.21.15.1:0/3145883553 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3c600cdff0 con 0x7f3c70068860 2024-02-02T15:25:39.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.879+0000 7f3c76400700 1 --2- 172.21.15.1:0/3145883553 >> [v2:172.21.15.71:6816/4080955538,v1:172.21.15.71:6817/4080955538] conn(0x7f3c500ab650 0x7f3c500adaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:39.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.880+0000 7f3c76400700 1 --2- 172.21.15.1:0/3145883553 >> [v2:172.21.15.71:6816/4080955538,v1:172.21.15.71:6817/4080955538] conn(0x7f3c500ab650 0x7f3c500adaf0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:39.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.880+0000 7f3c75000700 1 --2- 172.21.15.1:0/3145883553 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f3c500a0630 0x7f3c500a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:39.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.881+0000 7f3c6e000700 1 -- 172.21.15.1:0/3145883553 <== osd.7 v2:172.21.15.71:6816/4080955538 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f3c500ae1c0 con 0x7f3c500ab650 2024-02-02T15:25:39.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.881+0000 7f3c75000700 1 --2- 172.21.15.1:0/3145883553 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f3c500a0630 0x7f3c500a2af0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3c701014e0 tx=0x7f3c58009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:39.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.901+0000 7f3c78400700 1 -- 172.21.15.1:0/3145883553 --> [v2:172.21.15.71:6816/4080955538,v1:172.21.15.71:6817/4080955538] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3c70110fb0 con 0x7f3c500ab650 2024-02-02T15:25:39.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.902+0000 7f3c6e000700 1 -- 172.21.15.1:0/3145883553 <== osd.7 v2:172.21.15.71:6816/4080955538 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f3c6800f040 con 0x7f3c500ab650 2024-02-02T15:25:39.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.902+0000 7f3c78400700 1 -- 172.21.15.1:0/3145883553 >> [v2:172.21.15.71:6816/4080955538,v1:172.21.15.71:6817/4080955538] conn(0x7f3c500ab650 msgr2=0x7f3c500adaf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:39.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.902+0000 7f3c78400700 1 --2- 172.21.15.1:0/3145883553 >> [v2:172.21.15.71:6816/4080955538,v1:172.21.15.71:6817/4080955538] conn(0x7f3c500ab650 0x7f3c500adaf0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:39.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.902+0000 7f3c78400700 1 -- 172.21.15.1:0/3145883553 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f3c500a0630 msgr2=0x7f3c500a2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:39.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.902+0000 7f3c78400700 1 --2- 172.21.15.1:0/3145883553 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f3c500a0630 0x7f3c500a2af0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3c701014e0 tx=0x7f3c58009260 comp rx=0 tx=0).stop 2024-02-02T15:25:39.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.902+0000 7f3c78400700 1 -- 172.21.15.1:0/3145883553 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c70068860 msgr2=0x7f3c700ffcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:39.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.902+0000 7f3c78400700 1 --2- 172.21.15.1:0/3145883553 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c70068860 0x7f3c700ffcf0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3c60005910 tx=0x7f3c60005740 comp rx=0 tx=0).stop 2024-02-02T15:25:39.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.902+0000 7f3c76400700 1 -- 172.21.15.1:0/3145883553 reap_dead start 2024-02-02T15:25:39.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.902+0000 7f3c78400700 1 -- 172.21.15.1:0/3145883553 shutdown_connections 2024-02-02T15:25:39.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.902+0000 7f3c78400700 1 -- 172.21.15.1:0/3145883553 >> 172.21.15.1:0/3145883553 conn(0x7f3c70076d40 msgr2=0x7f3c700fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:39.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.902+0000 7f3c78400700 1 -- 172.21.15.1:0/3145883553 shutdown_connections 2024-02-02T15:25:39.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:38.902+0000 7f3c78400700 1 -- 172.21.15.1:0/3145883553 wait complete. 2024-02-02T15:25:40.174 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.a/config 2024-02-02T15:25:40.177 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.a/config 2024-02-02T15:25:41.220 INFO:teuthology.orchestra.run.smithi001.stdout:55834574904 2024-02-02T15:25:41.221 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-02-02T15:25:41.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:41.257+0000 7f5735c00700 1 -- 172.21.15.1:0/3877284699 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5730105d00 msgr2=0x7f5730103790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:41.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:41.257+0000 7f5735c00700 1 --2- 172.21.15.1:0/3877284699 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5730105d00 0x7f5730103790 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5720009990 tx=0x7f5720009ca0 comp rx=0 tx=0).stop 2024-02-02T15:25:41.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:41.258+0000 7f5735c00700 1 -- 172.21.15.1:0/3877284699 shutdown_connections 2024-02-02T15:25:41.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:41.258+0000 7f5735c00700 1 --2- 172.21.15.1:0/3877284699 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5730068860 0x7f5730068ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:41.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:41.258+0000 7f5735c00700 1 --2- 172.21.15.1:0/3877284699 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5730105d00 0x7f5730103790 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:41.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:41.258+0000 7f5735c00700 1 --2- 172.21.15.1:0/3877284699 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5730103cd0 0x7f5730104150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:41.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:41.258+0000 7f5735c00700 1 -- 172.21.15.1:0/3877284699 >> 172.21.15.1:0/3877284699 conn(0x7f5730076d40 msgr2=0x7f5730077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:41.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:41.259+0000 7f5735c00700 1 -- 172.21.15.1:0/3877284699 shutdown_connections 2024-02-02T15:25:41.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:41.259+0000 7f5735c00700 1 -- 172.21.15.1:0/3877284699 wait complete. 2024-02-02T15:25:41.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:41.260+0000 7f5735c00700 1 Processor -- start 2024-02-02T15:25:41.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:41.264+0000 7f5735c00700 1 -- start start 2024-02-02T15:25:41.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:41.265+0000 7f5735c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5730068860 0x7f573006b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:41.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:41.265+0000 7f5735c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5730103cd0 0x7f573006b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:41.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:41.265+0000 7f5735c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5730105d00 0x7f57301b19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:41.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:41.266+0000 7f5735c00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57301b1fa0 con 0x7f5730103cd0 2024-02-02T15:25:41.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:41.266+0000 7f5735c00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57301b2110 con 0x7f5730105d00 2024-02-02T15:25:41.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:41.266+0000 7f5735c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57301b22d0 con 0x7f5730068860 2024-02-02T15:25:41.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:41.266+0000 7f572e000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5730103cd0 0x7f573006b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:41.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:41.266+0000 7f572ea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5730068860 0x7f573006b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:41.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:41.266+0000 7f572ea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5730068860 0x7f573006b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:45608/0 (socket says 172.21.15.1:45608) 2024-02-02T15:25:41.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:41.266+0000 7f572e000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5730103cd0 0x7f573006b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48292/0 (socket says 172.21.15.1:48292) 2024-02-02T15:25:41.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:41.266+0000 7f572ea00700 1 -- 172.21.15.1:0/2019773162 learned_addr learned my addr 172.21.15.1:0/2019773162 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:25:41.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:41.266+0000 7f572f400700 1 --2- 172.21.15.1:0/2019773162 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5730105d00 0x7f57301b19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:41.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:41.267+0000 7f572ea00700 1 -- 172.21.15.1:0/2019773162 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5730105d00 msgr2=0x7f57301b19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:41.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:41.267+0000 7f572ea00700 1 --2- 172.21.15.1:0/2019773162 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5730105d00 0x7f57301b19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:41.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:41.267+0000 7f572ea00700 1 -- 172.21.15.1:0/2019773162 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5730103cd0 msgr2=0x7f573006b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:41.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:41.267+0000 7f572ea00700 1 --2- 172.21.15.1:0/2019773162 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5730103cd0 0x7f573006b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:41.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:41.267+0000 7f572ea00700 1 -- 172.21.15.1:0/2019773162 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57200095f0 con 0x7f5730068860 2024-02-02T15:25:41.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:41.267+0000 7f572f400700 1 --2- 172.21.15.1:0/2019773162 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5730105d00 0x7f57301b19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:25:41.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:41.267+0000 7f572e000700 1 --2- 172.21.15.1:0/2019773162 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5730103cd0 0x7f573006b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:25:41.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:41.267+0000 7f572ea00700 1 --2- 172.21.15.1:0/2019773162 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5730068860 0x7f573006b280 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5720005b40 tx=0x7f5720005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:41.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:41.268+0000 7f5727400700 1 -- 172.21.15.1:0/2019773162 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f572001d070 con 0x7f5730068860 2024-02-02T15:25:41.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:41.268+0000 7f5727400700 1 -- 172.21.15.1:0/2019773162 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5720021410 con 0x7f5730068860 2024-02-02T15:25:41.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:41.268+0000 7f5727400700 1 -- 172.21.15.1:0/2019773162 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5720017650 con 0x7f5730068860 2024-02-02T15:25:41.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:41.268+0000 7f5735c00700 1 -- 172.21.15.1:0/2019773162 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57301b2550 con 0x7f5730068860 2024-02-02T15:25:41.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:41.268+0000 7f5735c00700 1 -- 172.21.15.1:0/2019773162 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57301b2910 con 0x7f5730068860 2024-02-02T15:25:41.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:41.269+0000 7f5735c00700 1 -- 172.21.15.1:0/2019773162 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5730117610 con 0x7f5730068860 2024-02-02T15:25:41.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:41.272+0000 7f5727400700 1 -- 172.21.15.1:0/2019773162 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 17) v1 ==== 135120+0+0 (secure 0 0 0) 0x7f5720053050 con 0x7f5730068860 2024-02-02T15:25:41.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:41.274+0000 7f5727400700 1 --2- 172.21.15.1:0/2019773162 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f57100a0630 0x7f57100a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:41.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:41.274+0000 7f5727400700 1 -- 172.21.15.1:0/2019773162 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f5720051070 con 0x7f5730068860 2024-02-02T15:25:41.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:41.274+0000 7f5727400700 1 --2- 172.21.15.1:0/2019773162 >> [v2:172.21.15.142:6824/2290179934,v1:172.21.15.142:6825/2290179934] conn(0x7f57100ab650 0x7f57100adaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:41.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:41.274+0000 7f5727400700 1 -- 172.21.15.1:0/2019773162 --> [v2:172.21.15.142:6824/2290179934,v1:172.21.15.142:6825/2290179934] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f57100ae1c0 con 0x7f57100ab650 2024-02-02T15:25:41.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:41.274+0000 7f5727400700 1 -- 172.21.15.1:0/2019773162 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57200cd7f0 con 0x7f5730068860 2024-02-02T15:25:41.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:41.274+0000 7f572e000700 1 --2- 172.21.15.1:0/2019773162 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f57100a0630 0x7f57100a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:41.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:41.275+0000 7f572f400700 1 --2- 172.21.15.1:0/2019773162 >> [v2:172.21.15.142:6824/2290179934,v1:172.21.15.142:6825/2290179934] conn(0x7f57100ab650 0x7f57100adaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:41.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:41.275+0000 7f572f400700 1 --2- 172.21.15.1:0/2019773162 >> [v2:172.21.15.142:6824/2290179934,v1:172.21.15.142:6825/2290179934] conn(0x7f57100ab650 0x7f57100adaf0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:41.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:41.275+0000 7f572e000700 1 --2- 172.21.15.1:0/2019773162 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f57100a0630 0x7f57100a2af0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f573006c8a0 tx=0x7f5718009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:41.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:41.276+0000 7f5727400700 1 -- 172.21.15.1:0/2019773162 <== osd.11 v2:172.21.15.142:6824/2290179934 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f57100ae1c0 con 0x7f57100ab650 2024-02-02T15:25:41.283 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:41 smithi001 ceph-mon[35470]: pgmap v263: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.5 KiB/s wr, 10 op/s; 5 B/s, 0 keys/s, 0 objects/s recovering 2024-02-02T15:25:41.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:41.296+0000 7f5735c00700 1 -- 172.21.15.1:0/2019773162 --> [v2:172.21.15.142:6824/2290179934,v1:172.21.15.142:6825/2290179934] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f573004ea40 con 0x7f57100ab650 2024-02-02T15:25:41.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:41.297+0000 7f5727400700 1 -- 172.21.15.1:0/2019773162 <== osd.11 v2:172.21.15.142:6824/2290179934 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f573004ea40 con 0x7f57100ab650 2024-02-02T15:25:41.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:41.297+0000 7f5735c00700 1 -- 172.21.15.1:0/2019773162 >> [v2:172.21.15.142:6824/2290179934,v1:172.21.15.142:6825/2290179934] conn(0x7f57100ab650 msgr2=0x7f57100adaf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:41.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:41.297+0000 7f5735c00700 1 --2- 172.21.15.1:0/2019773162 >> [v2:172.21.15.142:6824/2290179934,v1:172.21.15.142:6825/2290179934] conn(0x7f57100ab650 0x7f57100adaf0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:41.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:41.297+0000 7f5735c00700 1 -- 172.21.15.1:0/2019773162 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f57100a0630 msgr2=0x7f57100a2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:41.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:41.298+0000 7f5735c00700 1 --2- 172.21.15.1:0/2019773162 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f57100a0630 0x7f57100a2af0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f573006c8a0 tx=0x7f5718009260 comp rx=0 tx=0).stop 2024-02-02T15:25:41.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:41.298+0000 7f5735c00700 1 -- 172.21.15.1:0/2019773162 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5730068860 msgr2=0x7f573006b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:41.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:41.298+0000 7f5735c00700 1 --2- 172.21.15.1:0/2019773162 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5730068860 0x7f573006b280 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5720005b40 tx=0x7f5720005740 comp rx=0 tx=0).stop 2024-02-02T15:25:41.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:41.298+0000 7f572f400700 1 -- 172.21.15.1:0/2019773162 reap_dead start 2024-02-02T15:25:41.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:41.298+0000 7f5735c00700 1 -- 172.21.15.1:0/2019773162 shutdown_connections 2024-02-02T15:25:41.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:41.298+0000 7f5735c00700 1 -- 172.21.15.1:0/2019773162 >> 172.21.15.1:0/2019773162 conn(0x7f5730076d40 msgr2=0x7f57300fda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:41.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:41.298+0000 7f5735c00700 1 -- 172.21.15.1:0/2019773162 shutdown_connections 2024-02-02T15:25:41.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:41.298+0000 7f5735c00700 1 -- 172.21.15.1:0/2019773162 wait complete. 2024-02-02T15:25:41.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:41 smithi071 ceph-mon[44707]: pgmap v263: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.5 KiB/s wr, 10 op/s; 5 B/s, 0 keys/s, 0 objects/s recovering 2024-02-02T15:25:41.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:41 smithi142 ceph-mon[45728]: pgmap v263: 129 pgs: 129 active+clean; 638 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.5 KiB/s wr, 10 op/s; 5 B/s, 0 keys/s, 0 objects/s recovering 2024-02-02T15:25:41.928 INFO:teuthology.orchestra.run.smithi001.stdout:206158430243 2024-02-02T15:25:41.928 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph osd last-stat-seq osd.7 2024-02-02T15:25:42.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.124+0000 7f6a83000700 1 -- 172.21.15.1:0/3245192517 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a7c103cd0 msgr2=0x7f6a7c104150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:42.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.124+0000 7f6a83000700 1 --2- 172.21.15.1:0/3245192517 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a7c103cd0 0x7f6a7c104150 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6a64009990 tx=0x7f6a64009ca0 comp rx=0 tx=0).stop 2024-02-02T15:25:42.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.124+0000 7f6a83000700 1 -- 172.21.15.1:0/3245192517 shutdown_connections 2024-02-02T15:25:42.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.124+0000 7f6a83000700 1 --2- 172.21.15.1:0/3245192517 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a7c068860 0x7f6a7c068ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:42.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.124+0000 7f6a83000700 1 --2- 172.21.15.1:0/3245192517 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a7c105d00 0x7f6a7c103790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:42.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.124+0000 7f6a83000700 1 --2- 172.21.15.1:0/3245192517 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a7c103cd0 0x7f6a7c104150 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:42.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.124+0000 7f6a83000700 1 -- 172.21.15.1:0/3245192517 >> 172.21.15.1:0/3245192517 conn(0x7f6a7c076d40 msgr2=0x7f6a7c077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:42.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.124+0000 7f6a83000700 1 -- 172.21.15.1:0/3245192517 shutdown_connections 2024-02-02T15:25:42.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.124+0000 7f6a83000700 1 -- 172.21.15.1:0/3245192517 wait complete. 2024-02-02T15:25:42.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.125+0000 7f6a83000700 1 Processor -- start 2024-02-02T15:25:42.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.125+0000 7f6a83000700 1 -- start start 2024-02-02T15:25:42.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.126+0000 7f6a83000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a7c068860 0x7f6a7c19d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:42.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.126+0000 7f6a83000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a7c103cd0 0x7f6a7c19d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:42.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.126+0000 7f6a83000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a7c105d00 0x7f6a7c1a1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:42.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.126+0000 7f6a83000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c1a2420 con 0x7f6a7c068860 2024-02-02T15:25:42.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.126+0000 7f6a83000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c1a2590 con 0x7f6a7c103cd0 2024-02-02T15:25:42.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.126+0000 7f6a83000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c1a2750 con 0x7f6a7c105d00 2024-02-02T15:25:42.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.126+0000 7f6a7be00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a7c068860 0x7f6a7c19d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:42.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.126+0000 7f6a7be00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a7c068860 0x7f6a7c19d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48342/0 (socket says 172.21.15.1:48342) 2024-02-02T15:25:42.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.126+0000 7f6a7be00700 1 -- 172.21.15.1:0/2417808630 learned_addr learned my addr 172.21.15.1:0/2417808630 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:25:42.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.126+0000 7f6a81000700 1 --2- 172.21.15.1:0/2417808630 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a7c105d00 0x7f6a7c1a1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:42.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.127+0000 7f6a7be00700 1 -- 172.21.15.1:0/2417808630 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a7c105d00 msgr2=0x7f6a7c1a1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:42.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.127+0000 7f6a7be00700 1 --2- 172.21.15.1:0/2417808630 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a7c105d00 0x7f6a7c1a1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:42.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.127+0000 7f6a7b400700 1 --2- 172.21.15.1:0/2417808630 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a7c103cd0 0x7f6a7c19d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:42.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.127+0000 7f6a7be00700 1 -- 172.21.15.1:0/2417808630 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a7c103cd0 msgr2=0x7f6a7c19d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:42.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.127+0000 7f6a7be00700 1 --2- 172.21.15.1:0/2417808630 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a7c103cd0 0x7f6a7c19d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:42.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.127+0000 7f6a7be00700 1 -- 172.21.15.1:0/2417808630 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a6c009520 con 0x7f6a7c068860 2024-02-02T15:25:42.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.127+0000 7f6a7b400700 1 --2- 172.21.15.1:0/2417808630 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a7c103cd0 0x7f6a7c19d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:25:42.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.127+0000 7f6a7be00700 1 --2- 172.21.15.1:0/2417808630 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a7c068860 0x7f6a7c19d460 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c00eb30 tx=0x7f6a6c00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:42.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.127+0000 7f6a78c00700 1 -- 172.21.15.1:0/2417808630 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a6c009880 con 0x7f6a7c068860 2024-02-02T15:25:42.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.127+0000 7f6a78c00700 1 -- 172.21.15.1:0/2417808630 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a6c0044e0 con 0x7f6a7c068860 2024-02-02T15:25:42.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.127+0000 7f6a83000700 1 -- 172.21.15.1:0/2417808630 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a640095f0 con 0x7f6a7c068860 2024-02-02T15:25:42.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.127+0000 7f6a78c00700 1 -- 172.21.15.1:0/2417808630 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a6c003e90 con 0x7f6a7c068860 2024-02-02T15:25:42.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.128+0000 7f6a83000700 1 -- 172.21.15.1:0/2417808630 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a7c1aa0c0 con 0x7f6a7c068860 2024-02-02T15:25:42.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.129+0000 7f6a83000700 1 -- 172.21.15.1:0/2417808630 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6a7c04ea40 con 0x7f6a7c068860 2024-02-02T15:25:42.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.132+0000 7f6a78c00700 1 -- 172.21.15.1:0/2417808630 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) v1 ==== 135120+0+0 (secure 0 0 0) 0x7f6a6c004000 con 0x7f6a7c068860 2024-02-02T15:25:42.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.134+0000 7f6a78c00700 1 --2- 172.21.15.1:0/2417808630 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f6a5c0a0630 0x7f6a5c0a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:42.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.134+0000 7f6a78c00700 1 -- 172.21.15.1:0/2417808630 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f6a6c014070 con 0x7f6a7c068860 2024-02-02T15:25:42.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.134+0000 7f6a7b400700 1 --2- 172.21.15.1:0/2417808630 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f6a5c0a0630 0x7f6a5c0a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:42.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.134+0000 7f6a78c00700 1 --2- 172.21.15.1:0/2417808630 >> [v2:172.21.15.1:6802/2591906423,v1:172.21.15.1:6803/2591906423] conn(0x7f6a5c0ab4e0 0x7f6a5c0ad980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:42.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.134+0000 7f6a78c00700 1 -- 172.21.15.1:0/2417808630 --> [v2:172.21.15.1:6802/2591906423,v1:172.21.15.1:6803/2591906423] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6a5c0ae050 con 0x7f6a5c0ab4e0 2024-02-02T15:25:42.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.134+0000 7f6a78c00700 1 -- 172.21.15.1:0/2417808630 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a6c0cc600 con 0x7f6a7c068860 2024-02-02T15:25:42.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.135+0000 7f6a81000700 1 --2- 172.21.15.1:0/2417808630 >> [v2:172.21.15.1:6802/2591906423,v1:172.21.15.1:6803/2591906423] conn(0x7f6a5c0ab4e0 0x7f6a5c0ad980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:42.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.135+0000 7f6a7b400700 1 --2- 172.21.15.1:0/2417808630 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f6a5c0a0630 0x7f6a5c0a2af0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6a6400b2e0 tx=0x7f6a64005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:42.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.135+0000 7f6a81000700 1 --2- 172.21.15.1:0/2417808630 >> [v2:172.21.15.1:6802/2591906423,v1:172.21.15.1:6803/2591906423] conn(0x7f6a5c0ab4e0 0x7f6a5c0ad980 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:42.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.136+0000 7f6a78c00700 1 -- 172.21.15.1:0/2417808630 <== osd.0 v2:172.21.15.1:6802/2591906423 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f6a5c0ae050 con 0x7f6a5c0ab4e0 2024-02-02T15:25:42.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.154+0000 7f6a83000700 1 -- 172.21.15.1:0/2417808630 --> [v2:172.21.15.1:6802/2591906423,v1:172.21.15.1:6803/2591906423] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6a7c1175e0 con 0x7f6a5c0ab4e0 2024-02-02T15:25:42.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.155+0000 7f6a78c00700 1 -- 172.21.15.1:0/2417808630 <== osd.0 v2:172.21.15.1:6802/2591906423 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f6a7400f040 con 0x7f6a5c0ab4e0 2024-02-02T15:25:42.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.155+0000 7f6a83000700 1 -- 172.21.15.1:0/2417808630 >> [v2:172.21.15.1:6802/2591906423,v1:172.21.15.1:6803/2591906423] conn(0x7f6a5c0ab4e0 msgr2=0x7f6a5c0ad980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:42.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.155+0000 7f6a83000700 1 --2- 172.21.15.1:0/2417808630 >> [v2:172.21.15.1:6802/2591906423,v1:172.21.15.1:6803/2591906423] conn(0x7f6a5c0ab4e0 0x7f6a5c0ad980 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:42.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.155+0000 7f6a83000700 1 -- 172.21.15.1:0/2417808630 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f6a5c0a0630 msgr2=0x7f6a5c0a2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:42.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.155+0000 7f6a83000700 1 --2- 172.21.15.1:0/2417808630 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f6a5c0a0630 0x7f6a5c0a2af0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6a6400b2e0 tx=0x7f6a64005fb0 comp rx=0 tx=0).stop 2024-02-02T15:25:42.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.155+0000 7f6a83000700 1 -- 172.21.15.1:0/2417808630 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a7c068860 msgr2=0x7f6a7c19d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:42.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.155+0000 7f6a83000700 1 --2- 172.21.15.1:0/2417808630 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a7c068860 0x7f6a7c19d460 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c00eb30 tx=0x7f6a6c00ee40 comp rx=0 tx=0).stop 2024-02-02T15:25:42.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.156+0000 7f6a81000700 1 -- 172.21.15.1:0/2417808630 reap_dead start 2024-02-02T15:25:42.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.156+0000 7f6a83000700 1 -- 172.21.15.1:0/2417808630 shutdown_connections 2024-02-02T15:25:42.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.156+0000 7f6a83000700 1 -- 172.21.15.1:0/2417808630 >> 172.21.15.1:0/2417808630 conn(0x7f6a7c076d40 msgr2=0x7f6a7c0fda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:42.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.156+0000 7f6a83000700 1 -- 172.21.15.1:0/2417808630 shutdown_connections 2024-02-02T15:25:42.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.156+0000 7f6a83000700 1 -- 172.21.15.1:0/2417808630 wait complete. 2024-02-02T15:25:42.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.080+0000 7f8bce000700 1 -- 172.21.15.1:0/4068553519 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bc8068860 msgr2=0x7f8bc8068ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:42.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.080+0000 7f8bce000700 1 --2- 172.21.15.1:0/4068553519 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bc8068860 0x7f8bc8068ce0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8bc000b0a0 tx=0x7f8bc000b3b0 comp rx=0 tx=0).stop 2024-02-02T15:25:42.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.080+0000 7f8bce000700 1 -- 172.21.15.1:0/4068553519 shutdown_connections 2024-02-02T15:25:42.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.080+0000 7f8bce000700 1 --2- 172.21.15.1:0/4068553519 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bc8068860 0x7f8bc8068ce0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:42.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.080+0000 7f8bce000700 1 --2- 172.21.15.1:0/4068553519 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8bc8105d00 0x7f8bc8103790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:42.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.080+0000 7f8bce000700 1 --2- 172.21.15.1:0/4068553519 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bc8103cd0 0x7f8bc8104150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:42.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.080+0000 7f8bce000700 1 -- 172.21.15.1:0/4068553519 >> 172.21.15.1:0/4068553519 conn(0x7f8bc8076d40 msgr2=0x7f8bc8077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:42.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.080+0000 7f8bce000700 1 -- 172.21.15.1:0/4068553519 shutdown_connections 2024-02-02T15:25:42.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.080+0000 7f8bce000700 1 -- 172.21.15.1:0/4068553519 wait complete. 2024-02-02T15:25:42.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.081+0000 7f8bce000700 1 Processor -- start 2024-02-02T15:25:42.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.081+0000 7f8bce000700 1 -- start start 2024-02-02T15:25:42.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.082+0000 7f8bce000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bc8068860 0x7f8bc819d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:42.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.082+0000 7f8bce000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8bc8103cd0 0x7f8bc819db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:42.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.082+0000 7f8bce000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bc8105d00 0x7f8bc81a1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:42.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.082+0000 7f8bce000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc81a2600 con 0x7f8bc8105d00 2024-02-02T15:25:42.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.082+0000 7f8bce000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc81a2770 con 0x7f8bc8103cd0 2024-02-02T15:25:42.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.082+0000 7f8bce000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc81a2930 con 0x7f8bc8068860 2024-02-02T15:25:42.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.082+0000 7f8bc6a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8bc8103cd0 0x7f8bc819db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:42.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.082+0000 7f8bc7e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bc8105d00 0x7f8bc81a1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:42.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.082+0000 7f8bc6a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8bc8103cd0 0x7f8bc819db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:47176/0 (socket says 172.21.15.1:47176) 2024-02-02T15:25:42.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.082+0000 7f8bc7400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bc8068860 0x7f8bc819d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:42.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.082+0000 7f8bc6a00700 1 -- 172.21.15.1:0/649600900 learned_addr learned my addr 172.21.15.1:0/649600900 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:25:42.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.082+0000 7f8bc6a00700 1 -- 172.21.15.1:0/649600900 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bc8068860 msgr2=0x7f8bc819d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:42.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.082+0000 7f8bc6a00700 1 --2- 172.21.15.1:0/649600900 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bc8068860 0x7f8bc819d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:42.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.082+0000 7f8bc6a00700 1 -- 172.21.15.1:0/649600900 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bc8105d00 msgr2=0x7f8bc81a1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:42.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.082+0000 7f8bc6a00700 1 --2- 172.21.15.1:0/649600900 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bc8105d00 0x7f8bc81a1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:42.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.082+0000 7f8bc6a00700 1 -- 172.21.15.1:0/649600900 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bb0009520 con 0x7f8bc8103cd0 2024-02-02T15:25:42.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.082+0000 7f8bc7400700 1 --2- 172.21.15.1:0/649600900 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bc8068860 0x7f8bc819d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:25:42.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.083+0000 7f8bc6a00700 1 --2- 172.21.15.1:0/649600900 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8bc8103cd0 0x7f8bc819db80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8bb000ea50 tx=0x7f8bb000ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:42.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.083+0000 7f8bbfe00700 1 -- 172.21.15.1:0/649600900 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb000cbd0 con 0x7f8bc8103cd0 2024-02-02T15:25:42.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.083+0000 7f8bbfe00700 1 -- 172.21.15.1:0/649600900 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8bb000cd30 con 0x7f8bc8103cd0 2024-02-02T15:25:42.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.083+0000 7f8bbfe00700 1 -- 172.21.15.1:0/649600900 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb0018680 con 0x7f8bc8103cd0 2024-02-02T15:25:42.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.083+0000 7f8bce000700 1 -- 172.21.15.1:0/649600900 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bc0009d00 con 0x7f8bc8103cd0 2024-02-02T15:25:42.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.083+0000 7f8bce000700 1 -- 172.21.15.1:0/649600900 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bc81aa2a0 con 0x7f8bc8103cd0 2024-02-02T15:25:42.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.084+0000 7f8bce000700 1 -- 172.21.15.1:0/649600900 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8bc8117560 con 0x7f8bc8103cd0 2024-02-02T15:25:42.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.088+0000 7f8bbfe00700 1 -- 172.21.15.1:0/649600900 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 135120+0+0 (secure 0 0 0) 0x7f8bb0010430 con 0x7f8bc8103cd0 2024-02-02T15:25:42.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.089+0000 7f8bbfe00700 1 --2- 172.21.15.1:0/649600900 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f8ba80a05b0 0x7f8ba80a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:42.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.089+0000 7f8bbfe00700 1 -- 172.21.15.1:0/649600900 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f8bb0014070 con 0x7f8bc8103cd0 2024-02-02T15:25:42.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.090+0000 7f8bc7400700 1 --2- 172.21.15.1:0/649600900 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f8ba80a05b0 0x7f8ba80a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:42.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.090+0000 7f8bbfe00700 1 --2- 172.21.15.1:0/649600900 >> [v2:172.21.15.1:6826/2716544850,v1:172.21.15.1:6827/2716544850] conn(0x7f8ba80ab5d0 0x7f8ba80ada70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:42.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.090+0000 7f8bbfe00700 1 -- 172.21.15.1:0/649600900 --> [v2:172.21.15.1:6826/2716544850,v1:172.21.15.1:6827/2716544850] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8ba80ae140 con 0x7f8ba80ab5d0 2024-02-02T15:25:42.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.090+0000 7f8bbfe00700 1 -- 172.21.15.1:0/649600900 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8bb00ccdc0 con 0x7f8bc8103cd0 2024-02-02T15:25:42.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.090+0000 7f8bc7e00700 1 --2- 172.21.15.1:0/649600900 >> [v2:172.21.15.1:6826/2716544850,v1:172.21.15.1:6827/2716544850] conn(0x7f8ba80ab5d0 0x7f8ba80ada70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:42.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.090+0000 7f8bc7400700 1 --2- 172.21.15.1:0/649600900 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f8ba80a05b0 0x7f8ba80a2a70 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8009fd0 tx=0x7f8bb8009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:42.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.091+0000 7f8bc7e00700 1 --2- 172.21.15.1:0/649600900 >> [v2:172.21.15.1:6826/2716544850,v1:172.21.15.1:6827/2716544850] conn(0x7f8ba80ab5d0 0x7f8ba80ada70 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:42.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.092+0000 7f8bbfe00700 1 -- 172.21.15.1:0/649600900 <== osd.9 v2:172.21.15.1:6826/2716544850 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f8ba80ae140 con 0x7f8ba80ab5d0 2024-02-02T15:25:42.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.112+0000 7f8bce000700 1 -- 172.21.15.1:0/649600900 --> [v2:172.21.15.1:6826/2716544850,v1:172.21.15.1:6827/2716544850] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8bc804ea40 con 0x7f8ba80ab5d0 2024-02-02T15:25:42.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.113+0000 7f8bbfe00700 1 -- 172.21.15.1:0/649600900 <== osd.9 v2:172.21.15.1:6826/2716544850 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f8bc804ea40 con 0x7f8ba80ab5d0 2024-02-02T15:25:42.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.113+0000 7f8bce000700 1 -- 172.21.15.1:0/649600900 >> [v2:172.21.15.1:6826/2716544850,v1:172.21.15.1:6827/2716544850] conn(0x7f8ba80ab5d0 msgr2=0x7f8ba80ada70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:42.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.113+0000 7f8bce000700 1 --2- 172.21.15.1:0/649600900 >> [v2:172.21.15.1:6826/2716544850,v1:172.21.15.1:6827/2716544850] conn(0x7f8ba80ab5d0 0x7f8ba80ada70 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:42.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.113+0000 7f8bce000700 1 -- 172.21.15.1:0/649600900 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f8ba80a05b0 msgr2=0x7f8ba80a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:42.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.114+0000 7f8bce000700 1 --2- 172.21.15.1:0/649600900 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f8ba80a05b0 0x7f8ba80a2a70 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8009fd0 tx=0x7f8bb8009260 comp rx=0 tx=0).stop 2024-02-02T15:25:42.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.114+0000 7f8bce000700 1 -- 172.21.15.1:0/649600900 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8bc8103cd0 msgr2=0x7f8bc819db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:42.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.114+0000 7f8bce000700 1 --2- 172.21.15.1:0/649600900 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8bc8103cd0 0x7f8bc819db80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8bb000ea50 tx=0x7f8bb000ed60 comp rx=0 tx=0).stop 2024-02-02T15:25:42.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.114+0000 7f8bc7e00700 1 -- 172.21.15.1:0/649600900 reap_dead start 2024-02-02T15:25:42.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.114+0000 7f8bce000700 1 -- 172.21.15.1:0/649600900 shutdown_connections 2024-02-02T15:25:42.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.114+0000 7f8bce000700 1 -- 172.21.15.1:0/649600900 >> 172.21.15.1:0/649600900 conn(0x7f8bc8076d40 msgr2=0x7f8bc80fda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:42.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.114+0000 7f8bce000700 1 -- 172.21.15.1:0/649600900 shutdown_connections 2024-02-02T15:25:42.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:42.114+0000 7f8bce000700 1 -- 172.21.15.1:0/649600900 wait complete. 2024-02-02T15:25:43.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.055+0000 7fd9a7400700 1 -- 172.21.15.1:0/3701247599 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9a0073a00 msgr2=0x7fd9a0111830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:43.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.055+0000 7fd9a7400700 1 --2- 172.21.15.1:0/3701247599 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9a0073a00 0x7fd9a0111830 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd988009990 tx=0x7fd988009ca0 comp rx=0 tx=0).stop 2024-02-02T15:25:43.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.055+0000 7fd9a7400700 1 -- 172.21.15.1:0/3701247599 shutdown_connections 2024-02-02T15:25:43.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.055+0000 7fd9a7400700 1 --2- 172.21.15.1:0/3701247599 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd9a0111d70 0x7fd9a0114160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:43.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.055+0000 7fd9a7400700 1 --2- 172.21.15.1:0/3701247599 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd9a00730e0 0x7fd9a00734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:43.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.055+0000 7fd9a7400700 1 --2- 172.21.15.1:0/3701247599 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9a0073a00 0x7fd9a0111830 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:43.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.055+0000 7fd9a7400700 1 -- 172.21.15.1:0/3701247599 >> 172.21.15.1:0/3701247599 conn(0x7fd9a00fc830 msgr2=0x7fd9a00fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:43.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.055+0000 7fd9a7400700 1 -- 172.21.15.1:0/3701247599 shutdown_connections 2024-02-02T15:25:43.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.055+0000 7fd9a7400700 1 -- 172.21.15.1:0/3701247599 wait complete. 2024-02-02T15:25:43.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.056+0000 7fd9a7400700 1 Processor -- start 2024-02-02T15:25:43.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.056+0000 7fd9a7400700 1 -- start start 2024-02-02T15:25:43.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.057+0000 7fd9a7400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9a00730e0 0x7fd9a019d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:43.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.057+0000 7fd9a7400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd9a0073a00 0x7fd9a019da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:43.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.057+0000 7fd9a7400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd9a0111d70 0x7fd9a01a1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:43.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.057+0000 7fd9a7400700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a01a2490 con 0x7fd9a0073a00 2024-02-02T15:25:43.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.057+0000 7fd9a7400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a01a2600 con 0x7fd9a00730e0 2024-02-02T15:25:43.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.057+0000 7fd9a7400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a01a27c0 con 0x7fd9a0111d70 2024-02-02T15:25:43.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.057+0000 7fd9a4a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9a00730e0 0x7fd9a019d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:43.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.057+0000 7fd9a4a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9a00730e0 0x7fd9a019d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:55290/0 (socket says 172.21.15.1:55290) 2024-02-02T15:25:43.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.057+0000 7fd9a4a00700 1 -- 172.21.15.1:0/3170336538 learned_addr learned my addr 172.21.15.1:0/3170336538 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:25:43.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.057+0000 7fd99fe00700 1 --2- 172.21.15.1:0/3170336538 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd9a0073a00 0x7fd9a019da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:43.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.057+0000 7fd9a5400700 1 --2- 172.21.15.1:0/3170336538 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd9a0111d70 0x7fd9a01a1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:43.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.058+0000 7fd9a4a00700 1 -- 172.21.15.1:0/3170336538 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd9a0111d70 msgr2=0x7fd9a01a1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:43.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.058+0000 7fd9a4a00700 1 --2- 172.21.15.1:0/3170336538 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd9a0111d70 0x7fd9a01a1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:43.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.058+0000 7fd9a4a00700 1 -- 172.21.15.1:0/3170336538 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd9a0073a00 msgr2=0x7fd9a019da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:43.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.058+0000 7fd9a4a00700 1 --2- 172.21.15.1:0/3170336538 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd9a0073a00 0x7fd9a019da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:43.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.058+0000 7fd9a4a00700 1 -- 172.21.15.1:0/3170336538 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9880095f0 con 0x7fd9a00730e0 2024-02-02T15:25:43.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.058+0000 7fd99fe00700 1 --2- 172.21.15.1:0/3170336538 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd9a0073a00 0x7fd9a019da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:25:43.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.058+0000 7fd9a4a00700 1 --2- 172.21.15.1:0/3170336538 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9a00730e0 0x7fd9a019d4d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd99000e9a0 tx=0x7fd99000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:43.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.058+0000 7fd99d600700 1 -- 172.21.15.1:0/3170336538 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd99000ca10 con 0x7fd9a00730e0 2024-02-02T15:25:43.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.058+0000 7fd99d600700 1 -- 172.21.15.1:0/3170336538 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd99000cb70 con 0x7fd9a00730e0 2024-02-02T15:25:43.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.058+0000 7fd9a7400700 1 -- 172.21.15.1:0/3170336538 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9a01a2aa0 con 0x7fd9a00730e0 2024-02-02T15:25:43.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.059+0000 7fd99d600700 1 -- 172.21.15.1:0/3170336538 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd990018680 con 0x7fd9a00730e0 2024-02-02T15:25:43.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.059+0000 7fd9a7400700 1 -- 172.21.15.1:0/3170336538 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9a01051d0 con 0x7fd9a00730e0 2024-02-02T15:25:43.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.060+0000 7fd9a7400700 1 -- 172.21.15.1:0/3170336538 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd9a004ea40 con 0x7fd9a00730e0 2024-02-02T15:25:43.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.062+0000 7fd99d600700 1 -- 172.21.15.1:0/3170336538 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 135120+0+0 (secure 0 0 0) 0x7fd99000cce0 con 0x7fd9a00730e0 2024-02-02T15:25:43.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.063+0000 7fd99d600700 1 --2- 172.21.15.1:0/3170336538 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fd9800a05b0 0x7fd9800a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:43.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.063+0000 7fd99d600700 1 -- 172.21.15.1:0/3170336538 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fd990014070 con 0x7fd9a00730e0 2024-02-02T15:25:43.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.063+0000 7fd99fe00700 1 --2- 172.21.15.1:0/3170336538 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fd9800a05b0 0x7fd9800a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:43.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.064+0000 7fd99d600700 1 --2- 172.21.15.1:0/3170336538 >> [v2:172.21.15.142:6816/3942038209,v1:172.21.15.142:6817/3942038209] conn(0x7fd9800ab5d0 0x7fd9800ada70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:43.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.064+0000 7fd99d600700 1 -- 172.21.15.1:0/3170336538 --> [v2:172.21.15.142:6816/3942038209,v1:172.21.15.142:6817/3942038209] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd9800ae140 con 0x7fd9800ab5d0 2024-02-02T15:25:43.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.064+0000 7fd99d600700 1 -- 172.21.15.1:0/3170336538 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd9900cc7d0 con 0x7fd9a00730e0 2024-02-02T15:25:43.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.064+0000 7fd9a5400700 1 --2- 172.21.15.1:0/3170336538 >> [v2:172.21.15.142:6816/3942038209,v1:172.21.15.142:6817/3942038209] conn(0x7fd9800ab5d0 0x7fd9800ada70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:43.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.064+0000 7fd99fe00700 1 --2- 172.21.15.1:0/3170336538 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fd9800a05b0 0x7fd9800a2a70 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd9a019eaf0 tx=0x7fd988005fd0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:43.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.065+0000 7fd9a5400700 1 --2- 172.21.15.1:0/3170336538 >> [v2:172.21.15.142:6816/3942038209,v1:172.21.15.142:6817/3942038209] conn(0x7fd9800ab5d0 0x7fd9800ada70 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:43.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.066+0000 7fd99d600700 1 -- 172.21.15.1:0/3170336538 <== osd.8 v2:172.21.15.142:6816/3942038209 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fd9800ae140 con 0x7fd9800ab5d0 2024-02-02T15:25:43.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.083+0000 7fd9a7400700 1 -- 172.21.15.1:0/3170336538 --> [v2:172.21.15.142:6816/3942038209,v1:172.21.15.142:6817/3942038209] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd9a0117630 con 0x7fd9800ab5d0 2024-02-02T15:25:43.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.083+0000 7fd99d600700 1 -- 172.21.15.1:0/3170336538 <== osd.8 v2:172.21.15.142:6816/3942038209 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fd99800f040 con 0x7fd9800ab5d0 2024-02-02T15:25:43.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.084+0000 7fd9a7400700 1 -- 172.21.15.1:0/3170336538 >> [v2:172.21.15.142:6816/3942038209,v1:172.21.15.142:6817/3942038209] conn(0x7fd9800ab5d0 msgr2=0x7fd9800ada70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:43.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.084+0000 7fd9a7400700 1 --2- 172.21.15.1:0/3170336538 >> [v2:172.21.15.142:6816/3942038209,v1:172.21.15.142:6817/3942038209] conn(0x7fd9800ab5d0 0x7fd9800ada70 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:43.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.084+0000 7fd9a7400700 1 -- 172.21.15.1:0/3170336538 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fd9800a05b0 msgr2=0x7fd9800a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:43.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.084+0000 7fd9a7400700 1 --2- 172.21.15.1:0/3170336538 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fd9800a05b0 0x7fd9800a2a70 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fd9a019eaf0 tx=0x7fd988005fd0 comp rx=0 tx=0).stop 2024-02-02T15:25:43.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.084+0000 7fd9a7400700 1 -- 172.21.15.1:0/3170336538 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9a00730e0 msgr2=0x7fd9a019d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:43.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.084+0000 7fd9a7400700 1 --2- 172.21.15.1:0/3170336538 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd9a00730e0 0x7fd9a019d4d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd99000e9a0 tx=0x7fd99000ecb0 comp rx=0 tx=0).stop 2024-02-02T15:25:43.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.084+0000 7fd9a5400700 1 -- 172.21.15.1:0/3170336538 reap_dead start 2024-02-02T15:25:43.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.084+0000 7fd9a7400700 1 -- 172.21.15.1:0/3170336538 shutdown_connections 2024-02-02T15:25:43.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.084+0000 7fd9a7400700 1 -- 172.21.15.1:0/3170336538 >> 172.21.15.1:0/3170336538 conn(0x7fd9a00fc830 msgr2=0x7fd9a00fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:43.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.084+0000 7fd9a7400700 1 -- 172.21.15.1:0/3170336538 shutdown_connections 2024-02-02T15:25:43.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.084+0000 7fd9a7400700 1 -- 172.21.15.1:0/3170336538 wait complete. 2024-02-02T15:25:43.330 INFO:teuthology.orchestra.run.smithi001.stdout:34359738429 2024-02-02T15:25:43.330 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-02-02T15:25:43.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.346+0000 7f5d9a000700 1 -- 172.21.15.1:0/2852297230 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d94107f50 msgr2=0x7f5d94108330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:43.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.346+0000 7f5d9a000700 1 --2- 172.21.15.1:0/2852297230 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d94107f50 0x7f5d94108330 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5d84009990 tx=0x7f5d84009ca0 comp rx=0 tx=0).stop 2024-02-02T15:25:43.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.346+0000 7f5d9a000700 1 -- 172.21.15.1:0/2852297230 shutdown_connections 2024-02-02T15:25:43.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.346+0000 7f5d9a000700 1 --2- 172.21.15.1:0/2852297230 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d94113e90 0x7f5d94116280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:43.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.346+0000 7f5d9a000700 1 --2- 172.21.15.1:0/2852297230 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d94107f50 0x7f5d94108330 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:43.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.346+0000 7f5d9a000700 1 --2- 172.21.15.1:0/2852297230 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5d94108870 0x7f5d94113950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:43.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.346+0000 7f5d9a000700 1 -- 172.21.15.1:0/2852297230 >> 172.21.15.1:0/2852297230 conn(0x7f5d9406cf40 msgr2=0x7f5d9406d350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:43.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.347+0000 7f5d9a000700 1 -- 172.21.15.1:0/2852297230 shutdown_connections 2024-02-02T15:25:43.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.347+0000 7f5d9a000700 1 -- 172.21.15.1:0/2852297230 wait complete. 2024-02-02T15:25:43.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.348+0000 7f5d9a000700 1 Processor -- start 2024-02-02T15:25:43.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.349+0000 7f5d9a000700 1 -- start start 2024-02-02T15:25:43.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.349+0000 7f5d9a000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5d94107f50 0x7f5d941a76c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:43.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.349+0000 7f5d9a000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d94108870 0x7f5d941a7c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:43.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.349+0000 7f5d9a000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d94113e90 0x7f5d941a1740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:43.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.349+0000 7f5d9a000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d941a1db0 con 0x7f5d94108870 2024-02-02T15:25:43.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.349+0000 7f5d9a000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d941a1f20 con 0x7f5d94107f50 2024-02-02T15:25:43.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.349+0000 7f5d9a000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d941a20e0 con 0x7f5d94113e90 2024-02-02T15:25:43.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.349+0000 7f5d93400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5d94107f50 0x7f5d941a76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:43.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.349+0000 7f5d92a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d94108870 0x7f5d941a7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:43.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.349+0000 7f5d93400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5d94107f50 0x7f5d941a76c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:55316/0 (socket says 172.21.15.1:55316) 2024-02-02T15:25:43.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.349+0000 7f5d93400700 1 -- 172.21.15.1:0/1963009590 learned_addr learned my addr 172.21.15.1:0/1963009590 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:25:43.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.350+0000 7f5d93e00700 1 --2- 172.21.15.1:0/1963009590 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d94113e90 0x7f5d941a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:43.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.350+0000 7f5d93400700 1 -- 172.21.15.1:0/1963009590 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d94113e90 msgr2=0x7f5d941a1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:43.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.350+0000 7f5d93400700 1 --2- 172.21.15.1:0/1963009590 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5d94113e90 0x7f5d941a1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:43.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.350+0000 7f5d93400700 1 -- 172.21.15.1:0/1963009590 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d94108870 msgr2=0x7f5d941a7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:43.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.350+0000 7f5d93400700 1 --2- 172.21.15.1:0/1963009590 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5d94108870 0x7f5d941a7c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:43.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.350+0000 7f5d93400700 1 -- 172.21.15.1:0/1963009590 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d840095f0 con 0x7f5d94107f50 2024-02-02T15:25:43.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.350+0000 7f5d93400700 1 --2- 172.21.15.1:0/1963009590 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5d94107f50 0x7f5d941a76c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5d84000c00 tx=0x7f5d840047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:43.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.350+0000 7f5d8be00700 1 -- 172.21.15.1:0/1963009590 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8401d070 con 0x7f5d94107f50 2024-02-02T15:25:43.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.351+0000 7f5d9a000700 1 -- 172.21.15.1:0/1963009590 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d941a2360 con 0x7f5d94107f50 2024-02-02T15:25:43.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.351+0000 7f5d8be00700 1 -- 172.21.15.1:0/1963009590 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5d84021410 con 0x7f5d94107f50 2024-02-02T15:25:43.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.351+0000 7f5d8be00700 1 -- 172.21.15.1:0/1963009590 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d84017650 con 0x7f5d94107f50 2024-02-02T15:25:43.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.351+0000 7f5d9a000700 1 -- 172.21.15.1:0/1963009590 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d941ae6b0 con 0x7f5d94107f50 2024-02-02T15:25:43.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.352+0000 7f5d9a000700 1 -- 172.21.15.1:0/1963009590 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5d94119800 con 0x7f5d94107f50 2024-02-02T15:25:43.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.354+0000 7f5d8be00700 1 -- 172.21.15.1:0/1963009590 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 135120+0+0 (secure 0 0 0) 0x7f5d840217f0 con 0x7f5d94107f50 2024-02-02T15:25:43.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.355+0000 7f5d8be00700 1 --2- 172.21.15.1:0/1963009590 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f5d740a05b0 0x7f5d740a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:43.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.355+0000 7f5d8be00700 1 -- 172.21.15.1:0/1963009590 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f5d840cdc10 con 0x7f5d94107f50 2024-02-02T15:25:43.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.355+0000 7f5d92a00700 1 --2- 172.21.15.1:0/1963009590 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f5d740a05b0 0x7f5d740a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:43.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.356+0000 7f5d8be00700 1 --2- 172.21.15.1:0/1963009590 >> [v2:172.21.15.1:6818/1560153428,v1:172.21.15.1:6819/1560153428] conn(0x7f5d740ab550 0x7f5d740ad9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:43.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.356+0000 7f5d8be00700 1 -- 172.21.15.1:0/1963009590 --> [v2:172.21.15.1:6818/1560153428,v1:172.21.15.1:6819/1560153428] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5d740ae0c0 con 0x7f5d740ab550 2024-02-02T15:25:43.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.356+0000 7f5d8be00700 1 -- 172.21.15.1:0/1963009590 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d840cdf80 con 0x7f5d94107f50 2024-02-02T15:25:43.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.356+0000 7f5d93e00700 1 --2- 172.21.15.1:0/1963009590 >> [v2:172.21.15.1:6818/1560153428,v1:172.21.15.1:6819/1560153428] conn(0x7f5d740ab550 0x7f5d740ad9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:43.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.356+0000 7f5d92a00700 1 --2- 172.21.15.1:0/1963009590 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f5d740a05b0 0x7f5d740a2a70 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5d941a2bb0 tx=0x7f5d7c00b410 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:43.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.356+0000 7f5d93e00700 1 --2- 172.21.15.1:0/1963009590 >> [v2:172.21.15.1:6818/1560153428,v1:172.21.15.1:6819/1560153428] conn(0x7f5d740ab550 0x7f5d740ad9f0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:43.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.357+0000 7f5d8be00700 1 -- 172.21.15.1:0/1963009590 <== osd.6 v2:172.21.15.1:6818/1560153428 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f5d740ae0c0 con 0x7f5d740ab550 2024-02-02T15:25:43.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.381+0000 7f5d9a000700 1 -- 172.21.15.1:0/1963009590 --> [v2:172.21.15.1:6818/1560153428,v1:172.21.15.1:6819/1560153428] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5d9404ea40 con 0x7f5d740ab550 2024-02-02T15:25:43.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.381+0000 7f5d8be00700 1 -- 172.21.15.1:0/1963009590 <== osd.6 v2:172.21.15.1:6818/1560153428 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f5d9404ea40 con 0x7f5d740ab550 2024-02-02T15:25:43.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.382+0000 7f5d9a000700 1 -- 172.21.15.1:0/1963009590 >> [v2:172.21.15.1:6818/1560153428,v1:172.21.15.1:6819/1560153428] conn(0x7f5d740ab550 msgr2=0x7f5d740ad9f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:43.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.382+0000 7f5d9a000700 1 --2- 172.21.15.1:0/1963009590 >> [v2:172.21.15.1:6818/1560153428,v1:172.21.15.1:6819/1560153428] conn(0x7f5d740ab550 0x7f5d740ad9f0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:43.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.382+0000 7f5d9a000700 1 -- 172.21.15.1:0/1963009590 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f5d740a05b0 msgr2=0x7f5d740a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:43.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.382+0000 7f5d9a000700 1 --2- 172.21.15.1:0/1963009590 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f5d740a05b0 0x7f5d740a2a70 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5d941a2bb0 tx=0x7f5d7c00b410 comp rx=0 tx=0).stop 2024-02-02T15:25:43.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.382+0000 7f5d9a000700 1 -- 172.21.15.1:0/1963009590 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5d94107f50 msgr2=0x7f5d941a76c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:43.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.382+0000 7f5d9a000700 1 --2- 172.21.15.1:0/1963009590 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5d94107f50 0x7f5d941a76c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5d84000c00 tx=0x7f5d840047b0 comp rx=0 tx=0).stop 2024-02-02T15:25:43.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.382+0000 7f5d93e00700 1 -- 172.21.15.1:0/1963009590 reap_dead start 2024-02-02T15:25:43.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.382+0000 7f5d9a000700 1 -- 172.21.15.1:0/1963009590 shutdown_connections 2024-02-02T15:25:43.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.382+0000 7f5d9a000700 1 -- 172.21.15.1:0/1963009590 >> 172.21.15.1:0/1963009590 conn(0x7f5d9406cf40 msgr2=0x7f5d9406f820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:43.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.382+0000 7f5d9a000700 1 -- 172.21.15.1:0/1963009590 shutdown_connections 2024-02-02T15:25:43.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.382+0000 7f5d9a000700 1 -- 172.21.15.1:0/1963009590 wait complete. 2024-02-02T15:25:43.401 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:43 smithi001 ceph-mon[35470]: pgmap v264: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.7 KiB/s wr, 11 op/s; 0 B/s, 0 objects/s recovering 2024-02-02T15:25:43.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:43 smithi071 ceph-mon[44707]: pgmap v264: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.7 KiB/s wr, 11 op/s; 0 B/s, 0 objects/s recovering 2024-02-02T15:25:43.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:43 smithi142 ceph-mon[45728]: pgmap v264: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.7 KiB/s wr, 11 op/s; 0 B/s, 0 objects/s recovering 2024-02-02T15:25:43.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.796+0000 7f7fd5800700 1 -- 172.21.15.1:0/3851717462 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7fd0101d50 msgr2=0x7f7fd0102130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:43.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.796+0000 7f7fd5800700 1 --2- 172.21.15.1:0/3851717462 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7fd0101d50 0x7f7fd0102130 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7fc0009990 tx=0x7f7fc0009ca0 comp rx=0 tx=0).stop 2024-02-02T15:25:43.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.796+0000 7f7fd5800700 1 -- 172.21.15.1:0/3851717462 shutdown_connections 2024-02-02T15:25:43.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.796+0000 7f7fd5800700 1 --2- 172.21.15.1:0/3851717462 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7fd010b180 0x7f7fd010d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:43.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.796+0000 7f7fd5800700 1 --2- 172.21.15.1:0/3851717462 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7fd0101d50 0x7f7fd0102130 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:43.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.796+0000 7f7fd5800700 1 --2- 172.21.15.1:0/3851717462 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7fd0102670 0x7f7fd010ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:43.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.796+0000 7f7fd5800700 1 -- 172.21.15.1:0/3851717462 >> 172.21.15.1:0/3851717462 conn(0x7f7fd00fb5e0 msgr2=0x7f7fd00fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:43.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.796+0000 7f7fd5800700 1 -- 172.21.15.1:0/3851717462 shutdown_connections 2024-02-02T15:25:43.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.796+0000 7f7fd5800700 1 -- 172.21.15.1:0/3851717462 wait complete. 2024-02-02T15:25:43.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.797+0000 7f7fd5800700 1 Processor -- start 2024-02-02T15:25:43.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.798+0000 7f7fd5800700 1 -- start start 2024-02-02T15:25:43.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.798+0000 7f7fd5800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7fd0101d50 0x7f7fd01099f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:43.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.798+0000 7f7fd5800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7fd0102670 0x7f7fd0109f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:43.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.798+0000 7f7fd5800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7fd010b180 0x7f7fd01a1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:43.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.798+0000 7f7fd5800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd01a2280 con 0x7f7fd0102670 2024-02-02T15:25:43.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.798+0000 7f7fd5800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd01a23f0 con 0x7f7fd010b180 2024-02-02T15:25:43.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.798+0000 7f7fd5800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd01a25b0 con 0x7f7fd0101d50 2024-02-02T15:25:43.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.799+0000 7f7fcfe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7fd0101d50 0x7f7fd01099f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:43.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.799+0000 7f7fcfe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7fd0101d50 0x7f7fd01099f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:43134/0 (socket says 172.21.15.1:43134) 2024-02-02T15:25:43.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.799+0000 7f7fcfe00700 1 -- 172.21.15.1:0/3841978174 learned_addr learned my addr 172.21.15.1:0/3841978174 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:25:43.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.799+0000 7f7fd4e00700 1 --2- 172.21.15.1:0/3841978174 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7fd010b180 0x7f7fd01a1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:43.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.799+0000 7f7fcf400700 1 --2- 172.21.15.1:0/3841978174 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7fd0102670 0x7f7fd0109f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:43.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.799+0000 7f7fcfe00700 1 -- 172.21.15.1:0/3841978174 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7fd010b180 msgr2=0x7f7fd01a1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:43.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.799+0000 7f7fcfe00700 1 --2- 172.21.15.1:0/3841978174 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7fd010b180 0x7f7fd01a1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:43.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.799+0000 7f7fcfe00700 1 -- 172.21.15.1:0/3841978174 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7fd0102670 msgr2=0x7f7fd0109f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:43.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.799+0000 7f7fcfe00700 1 --2- 172.21.15.1:0/3841978174 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7fd0102670 0x7f7fd0109f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:43.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.799+0000 7f7fcfe00700 1 -- 172.21.15.1:0/3841978174 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fc00095f0 con 0x7f7fd0101d50 2024-02-02T15:25:43.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.799+0000 7f7fcfe00700 1 --2- 172.21.15.1:0/3841978174 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7fd0101d50 0x7f7fd01099f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7fc0000c00 tx=0x7f7fc0004790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:43.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.800+0000 7f7fccc00700 1 -- 172.21.15.1:0/3841978174 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc001d070 con 0x7f7fd0101d50 2024-02-02T15:25:43.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.800+0000 7f7fd5800700 1 -- 172.21.15.1:0/3841978174 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fd01a2830 con 0x7f7fd0101d50 2024-02-02T15:25:43.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.800+0000 7f7fccc00700 1 -- 172.21.15.1:0/3841978174 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7fc0021410 con 0x7f7fd0101d50 2024-02-02T15:25:43.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.800+0000 7f7fccc00700 1 -- 172.21.15.1:0/3841978174 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc0017650 con 0x7f7fd0101d50 2024-02-02T15:25:43.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.800+0000 7f7fd5800700 1 -- 172.21.15.1:0/3841978174 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fd01aa0f0 con 0x7f7fd0101d50 2024-02-02T15:25:43.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.801+0000 7f7fd5800700 1 -- 172.21.15.1:0/3841978174 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7fd0110dd0 con 0x7f7fd0101d50 2024-02-02T15:25:43.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.803+0000 7f7fccc00700 1 -- 172.21.15.1:0/3841978174 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 17) v1 ==== 135120+0+0 (secure 0 0 0) 0x7f7fc0021580 con 0x7f7fd0101d50 2024-02-02T15:25:43.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.804+0000 7f7fccc00700 1 --2- 172.21.15.1:0/3841978174 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f7fb00a05b0 0x7f7fb00a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:43.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.805+0000 7f7fccc00700 1 -- 172.21.15.1:0/3841978174 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f7fc00cd770 con 0x7f7fd0101d50 2024-02-02T15:25:43.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.805+0000 7f7fcf400700 1 --2- 172.21.15.1:0/3841978174 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f7fb00a05b0 0x7f7fb00a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:43.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.805+0000 7f7fccc00700 1 --2- 172.21.15.1:0/3841978174 >> [v2:172.21.15.71:6808/3095380305,v1:172.21.15.71:6809/3095380305] conn(0x7f7fb00ab5d0 0x7f7fb00ada70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:43.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.805+0000 7f7fccc00700 1 -- 172.21.15.1:0/3841978174 --> [v2:172.21.15.71:6808/3095380305,v1:172.21.15.71:6809/3095380305] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7fb00ae140 con 0x7f7fb00ab5d0 2024-02-02T15:25:43.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.805+0000 7f7fccc00700 1 -- 172.21.15.1:0/3841978174 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7fc00cdae0 con 0x7f7fd0101d50 2024-02-02T15:25:43.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.805+0000 7f7fd4e00700 1 --2- 172.21.15.1:0/3841978174 >> [v2:172.21.15.71:6808/3095380305,v1:172.21.15.71:6809/3095380305] conn(0x7f7fb00ab5d0 0x7f7fb00ada70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:43.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.806+0000 7f7fcf400700 1 --2- 172.21.15.1:0/3841978174 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f7fb00a05b0 0x7f7fb00a2a70 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7fd010b010 tx=0x7f7fb8009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:43.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.806+0000 7f7fd4e00700 1 --2- 172.21.15.1:0/3841978174 >> [v2:172.21.15.71:6808/3095380305,v1:172.21.15.71:6809/3095380305] conn(0x7f7fb00ab5d0 0x7f7fb00ada70 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:43.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.807+0000 7f7fccc00700 1 -- 172.21.15.1:0/3841978174 <== osd.4 v2:172.21.15.71:6808/3095380305 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f7fb00ae140 con 0x7f7fb00ab5d0 2024-02-02T15:25:43.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.824+0000 7f7fd5800700 1 -- 172.21.15.1:0/3841978174 --> [v2:172.21.15.71:6808/3095380305,v1:172.21.15.71:6809/3095380305] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7fd004f290 con 0x7f7fb00ab5d0 2024-02-02T15:25:43.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.824+0000 7f7fccc00700 1 -- 172.21.15.1:0/3841978174 <== osd.4 v2:172.21.15.71:6808/3095380305 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f7fd004f290 con 0x7f7fb00ab5d0 2024-02-02T15:25:43.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.825+0000 7f7fd5800700 1 -- 172.21.15.1:0/3841978174 >> [v2:172.21.15.71:6808/3095380305,v1:172.21.15.71:6809/3095380305] conn(0x7f7fb00ab5d0 msgr2=0x7f7fb00ada70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:43.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.825+0000 7f7fd5800700 1 --2- 172.21.15.1:0/3841978174 >> [v2:172.21.15.71:6808/3095380305,v1:172.21.15.71:6809/3095380305] conn(0x7f7fb00ab5d0 0x7f7fb00ada70 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:43.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.825+0000 7f7fd5800700 1 -- 172.21.15.1:0/3841978174 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f7fb00a05b0 msgr2=0x7f7fb00a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:43.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.825+0000 7f7fd5800700 1 --2- 172.21.15.1:0/3841978174 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f7fb00a05b0 0x7f7fb00a2a70 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7fd010b010 tx=0x7f7fb8009260 comp rx=0 tx=0).stop 2024-02-02T15:25:43.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.825+0000 7f7fd5800700 1 -- 172.21.15.1:0/3841978174 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7fd0101d50 msgr2=0x7f7fd01099f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:43.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.825+0000 7f7fd5800700 1 --2- 172.21.15.1:0/3841978174 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7fd0101d50 0x7f7fd01099f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7fc0000c00 tx=0x7f7fc0004790 comp rx=0 tx=0).stop 2024-02-02T15:25:43.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.825+0000 7f7fd4e00700 1 -- 172.21.15.1:0/3841978174 reap_dead start 2024-02-02T15:25:43.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.825+0000 7f7fd5800700 1 -- 172.21.15.1:0/3841978174 shutdown_connections 2024-02-02T15:25:43.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.825+0000 7f7fd5800700 1 -- 172.21.15.1:0/3841978174 >> 172.21.15.1:0/3841978174 conn(0x7f7fd00fb5e0 msgr2=0x7f7fd010d420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:43.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.825+0000 7f7fd5800700 1 -- 172.21.15.1:0/3841978174 shutdown_connections 2024-02-02T15:25:43.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.825+0000 7f7fd5800700 1 -- 172.21.15.1:0/3841978174 wait complete. 2024-02-02T15:25:43.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.947+0000 7ff6b2400700 1 -- 172.21.15.1:0/2432914662 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff6ac073170 msgr2=0x7ff6ac073550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:43.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.947+0000 7ff6b2400700 1 --2- 172.21.15.1:0/2432914662 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff6ac073170 0x7ff6ac073550 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff69c009990 tx=0x7ff69c009ca0 comp rx=0 tx=0).stop 2024-02-02T15:25:43.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.948+0000 7ff6b2400700 1 -- 172.21.15.1:0/2432914662 shutdown_connections 2024-02-02T15:25:43.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.948+0000 7ff6b2400700 1 --2- 172.21.15.1:0/2432914662 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff6ac10d5a0 0x7ff6ac10fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:43.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.948+0000 7ff6b2400700 1 --2- 172.21.15.1:0/2432914662 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff6ac073170 0x7ff6ac073550 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:43.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.948+0000 7ff6b2400700 1 --2- 172.21.15.1:0/2432914662 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff6ac073a90 0x7ff6ac10d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:43.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.948+0000 7ff6b2400700 1 -- 172.21.15.1:0/2432914662 >> 172.21.15.1:0/2432914662 conn(0x7ff6ac0fc830 msgr2=0x7ff6ac0fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:43.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.948+0000 7ff6b2400700 1 -- 172.21.15.1:0/2432914662 shutdown_connections 2024-02-02T15:25:43.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.948+0000 7ff6b2400700 1 -- 172.21.15.1:0/2432914662 wait complete. 2024-02-02T15:25:43.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.949+0000 7ff6b2400700 1 Processor -- start 2024-02-02T15:25:43.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.949+0000 7ff6b2400700 1 -- start start 2024-02-02T15:25:43.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.949+0000 7ff6b2400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff6ac073170 0x7ff6ac199070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:43.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.949+0000 7ff6b2400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff6ac073a90 0x7ff6ac1995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:43.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.949+0000 7ff6b2400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff6ac10d5a0 0x7ff6ac19da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:43.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.949+0000 7ff6b2400700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ac19e030 con 0x7ff6ac073170 2024-02-02T15:25:43.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.949+0000 7ff6b2400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ac19e1a0 con 0x7ff6ac073a90 2024-02-02T15:25:43.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.949+0000 7ff6b2400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ac19e360 con 0x7ff6ac10d5a0 2024-02-02T15:25:43.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.949+0000 7ff6ab400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff6ac073170 0x7ff6ac199070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:43.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.950+0000 7ff6ab400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff6ac073170 0x7ff6ac199070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51854/0 (socket says 172.21.15.1:51854) 2024-02-02T15:25:43.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.950+0000 7ff6ab400700 1 -- 172.21.15.1:0/3788790611 learned_addr learned my addr 172.21.15.1:0/3788790611 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:25:43.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.950+0000 7ff6abe00700 1 --2- 172.21.15.1:0/3788790611 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff6ac10d5a0 0x7ff6ac19da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:43.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.950+0000 7ff6aaa00700 1 --2- 172.21.15.1:0/3788790611 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff6ac073a90 0x7ff6ac1995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:43.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.950+0000 7ff6ab400700 1 -- 172.21.15.1:0/3788790611 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff6ac10d5a0 msgr2=0x7ff6ac19da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:43.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.950+0000 7ff6ab400700 1 --2- 172.21.15.1:0/3788790611 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff6ac10d5a0 0x7ff6ac19da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:43.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.950+0000 7ff6ab400700 1 -- 172.21.15.1:0/3788790611 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff6ac073a90 msgr2=0x7ff6ac1995b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:43.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.950+0000 7ff6ab400700 1 --2- 172.21.15.1:0/3788790611 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff6ac073a90 0x7ff6ac1995b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:43.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.950+0000 7ff6ab400700 1 -- 172.21.15.1:0/3788790611 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff69c0095f0 con 0x7ff6ac073170 2024-02-02T15:25:43.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.950+0000 7ff6abe00700 1 --2- 172.21.15.1:0/3788790611 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff6ac10d5a0 0x7ff6ac19da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:25:43.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.950+0000 7ff6aaa00700 1 --2- 172.21.15.1:0/3788790611 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff6ac073a90 0x7ff6ac1995b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:25:43.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.950+0000 7ff6ab400700 1 --2- 172.21.15.1:0/3788790611 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff6ac073170 0x7ff6ac199070 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff69c005910 tx=0x7ff69c004710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:43.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.951+0000 7ff6a3e00700 1 -- 172.21.15.1:0/3788790611 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69c01d070 con 0x7ff6ac073170 2024-02-02T15:25:43.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.951+0000 7ff6a3e00700 1 -- 172.21.15.1:0/3788790611 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff69c022470 con 0x7ff6ac073170 2024-02-02T15:25:43.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.951+0000 7ff6b2400700 1 -- 172.21.15.1:0/3788790611 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6ac19e5e0 con 0x7ff6ac073170 2024-02-02T15:25:43.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.951+0000 7ff6a3e00700 1 -- 172.21.15.1:0/3788790611 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69c017650 con 0x7ff6ac073170 2024-02-02T15:25:43.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.951+0000 7ff6b2400700 1 -- 172.21.15.1:0/3788790611 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6ac1a5ea0 con 0x7ff6ac073170 2024-02-02T15:25:43.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.952+0000 7ff6b2400700 1 -- 172.21.15.1:0/3788790611 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff6ac1131c0 con 0x7ff6ac073170 2024-02-02T15:25:43.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.954+0000 7ff6a3e00700 1 -- 172.21.15.1:0/3788790611 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) v1 ==== 135120+0+0 (secure 0 0 0) 0x7ff69c0225e0 con 0x7ff6ac073170 2024-02-02T15:25:43.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.956+0000 7ff6a3e00700 1 --2- 172.21.15.1:0/3788790611 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7ff68c0a0630 0x7ff68c0a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:43.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.956+0000 7ff6a3e00700 1 -- 172.21.15.1:0/3788790611 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ff69c0cd3d0 con 0x7ff6ac073170 2024-02-02T15:25:43.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.956+0000 7ff6aaa00700 1 --2- 172.21.15.1:0/3788790611 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7ff68c0a0630 0x7ff68c0a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:43.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.956+0000 7ff6a3e00700 1 --2- 172.21.15.1:0/3788790611 >> [v2:172.21.15.142:6808/4195149262,v1:172.21.15.142:6809/4195149262] conn(0x7ff68c0ab650 0x7ff68c0adaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:43.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.956+0000 7ff6a3e00700 1 -- 172.21.15.1:0/3788790611 --> [v2:172.21.15.142:6808/4195149262,v1:172.21.15.142:6809/4195149262] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff68c0ae1c0 con 0x7ff68c0ab650 2024-02-02T15:25:43.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.956+0000 7ff6a3e00700 1 -- 172.21.15.1:0/3788790611 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff69c0cd7d0 con 0x7ff6ac073170 2024-02-02T15:25:43.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.957+0000 7ff6abe00700 1 --2- 172.21.15.1:0/3788790611 >> [v2:172.21.15.142:6808/4195149262,v1:172.21.15.142:6809/4195149262] conn(0x7ff68c0ab650 0x7ff68c0adaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:43.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.957+0000 7ff6aaa00700 1 --2- 172.21.15.1:0/3788790611 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7ff68c0a0630 0x7ff68c0a2af0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac19a690 tx=0x7ff694009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:43.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.957+0000 7ff6abe00700 1 --2- 172.21.15.1:0/3788790611 >> [v2:172.21.15.142:6808/4195149262,v1:172.21.15.142:6809/4195149262] conn(0x7ff68c0ab650 0x7ff68c0adaf0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:43.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.958+0000 7ff6a3e00700 1 -- 172.21.15.1:0/3788790611 <== osd.5 v2:172.21.15.142:6808/4195149262 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7ff68c0ae1c0 con 0x7ff68c0ab650 2024-02-02T15:25:43.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.982+0000 7ff6b2400700 1 -- 172.21.15.1:0/3788790611 --> [v2:172.21.15.142:6808/4195149262,v1:172.21.15.142:6809/4195149262] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff6ac04ea40 con 0x7ff68c0ab650 2024-02-02T15:25:43.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.983+0000 7ff6a3e00700 1 -- 172.21.15.1:0/3788790611 <== osd.5 v2:172.21.15.142:6808/4195149262 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7ff6ac04ea40 con 0x7ff68c0ab650 2024-02-02T15:25:43.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.983+0000 7ff6b2400700 1 -- 172.21.15.1:0/3788790611 >> [v2:172.21.15.142:6808/4195149262,v1:172.21.15.142:6809/4195149262] conn(0x7ff68c0ab650 msgr2=0x7ff68c0adaf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:43.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.983+0000 7ff6b2400700 1 --2- 172.21.15.1:0/3788790611 >> [v2:172.21.15.142:6808/4195149262,v1:172.21.15.142:6809/4195149262] conn(0x7ff68c0ab650 0x7ff68c0adaf0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:43.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.983+0000 7ff6b2400700 1 -- 172.21.15.1:0/3788790611 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7ff68c0a0630 msgr2=0x7ff68c0a2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:43.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.983+0000 7ff6b2400700 1 --2- 172.21.15.1:0/3788790611 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7ff68c0a0630 0x7ff68c0a2af0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac19a690 tx=0x7ff694009260 comp rx=0 tx=0).stop 2024-02-02T15:25:43.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.983+0000 7ff6b2400700 1 -- 172.21.15.1:0/3788790611 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff6ac073170 msgr2=0x7ff6ac199070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:43.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.983+0000 7ff6b2400700 1 --2- 172.21.15.1:0/3788790611 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff6ac073170 0x7ff6ac199070 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff69c005910 tx=0x7ff69c004710 comp rx=0 tx=0).stop 2024-02-02T15:25:43.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.983+0000 7ff6abe00700 1 -- 172.21.15.1:0/3788790611 reap_dead start 2024-02-02T15:25:43.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.983+0000 7ff6b2400700 1 -- 172.21.15.1:0/3788790611 shutdown_connections 2024-02-02T15:25:43.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.983+0000 7ff6b2400700 1 -- 172.21.15.1:0/3788790611 >> 172.21.15.1:0/3788790611 conn(0x7ff6ac0fc830 msgr2=0x7ff6ac10f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:43.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.984+0000 7ff6b2400700 1 -- 172.21.15.1:0/3788790611 shutdown_connections 2024-02-02T15:25:43.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:43.984+0000 7ff6b2400700 1 -- 172.21.15.1:0/3788790611 wait complete. 2024-02-02T15:25:44.292 INFO:teuthology.orchestra.run.smithi001.stdout:236223201312 2024-02-02T15:25:44.292 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph osd last-stat-seq osd.8 2024-02-02T15:25:44.801 INFO:teuthology.orchestra.run.smithi001.stdout:154618822699 2024-02-02T15:25:44.801 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-02-02T15:25:45.029 INFO:teuthology.orchestra.run.smithi001.stdout:133143986224 2024-02-02T15:25:45.030 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-02-02T15:25:45.071 INFO:teuthology.orchestra.run.smithi001.stdout:266287972380 2024-02-02T15:25:45.072 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph osd last-stat-seq osd.9 2024-02-02T15:25:45.294 INFO:teuthology.orchestra.run.smithi001.stdout:322122547222 2024-02-02T15:25:45.294 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph osd last-stat-seq osd.11 2024-02-02T15:25:45.360 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:45 smithi001 ceph-mon[35470]: pgmap v265: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 767 B/s wr, 10 op/s 2024-02-02T15:25:45.577 INFO:teuthology.orchestra.run.smithi001.stdout:184683593768 2024-02-02T15:25:45.578 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph osd last-stat-seq osd.6 2024-02-02T15:25:45.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:45 smithi071 ceph-mon[44707]: pgmap v265: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 767 B/s wr, 10 op/s 2024-02-02T15:25:45.649 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.a/config 2024-02-02T15:25:45.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:45 smithi142 ceph-mon[45728]: pgmap v265: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 767 B/s wr, 10 op/s 2024-02-02T15:25:45.701 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.a/config 2024-02-02T15:25:46.662 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.a/config 2024-02-02T15:25:46.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:46.893+0000 7f2b7a200700 1 -- 172.21.15.1:0/2600706877 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b741060a0 msgr2=0x7f2b7410dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:46.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:46.893+0000 7f2b7a200700 1 --2- 172.21.15.1:0/2600706877 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b741060a0 0x7f2b7410dc70 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c00b0a0 tx=0x7f2b6c00b3b0 comp rx=0 tx=0).stop 2024-02-02T15:25:46.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:46.893+0000 7f2b7a200700 1 -- 172.21.15.1:0/2600706877 shutdown_connections 2024-02-02T15:25:46.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:46.893+0000 7f2b7a200700 1 --2- 172.21.15.1:0/2600706877 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b741060a0 0x7f2b7410dc70 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:46.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:46.893+0000 7f2b7a200700 1 --2- 172.21.15.1:0/2600706877 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b74100fc0 0x7f2b741013a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:46.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:46.893+0000 7f2b7a200700 1 --2- 172.21.15.1:0/2600706877 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2b74101970 0x7f2b741059c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:46.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:46.893+0000 7f2b7a200700 1 -- 172.21.15.1:0/2600706877 >> 172.21.15.1:0/2600706877 conn(0x7f2b740fc830 msgr2=0x7f2b740fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:46.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:46.893+0000 7f2b7a200700 1 -- 172.21.15.1:0/2600706877 shutdown_connections 2024-02-02T15:25:46.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:46.893+0000 7f2b7a200700 1 -- 172.21.15.1:0/2600706877 wait complete. 2024-02-02T15:25:46.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:46.894+0000 7f2b7a200700 1 Processor -- start 2024-02-02T15:25:46.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:46.895+0000 7f2b7a200700 1 -- start start 2024-02-02T15:25:46.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:46.895+0000 7f2b7a200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b74100fc0 0x7f2b74199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:46.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:46.895+0000 7f2b7a200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2b74101970 0x7f2b74199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:46.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:46.895+0000 7f2b7a200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b741060a0 0x7f2b7419dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:46.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:46.895+0000 7f2b7a200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7419e1f0 con 0x7f2b741060a0 2024-02-02T15:25:46.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:46.895+0000 7f2b7a200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7419e360 con 0x7f2b74101970 2024-02-02T15:25:46.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:46.895+0000 7f2b7a200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7419e520 con 0x7f2b74100fc0 2024-02-02T15:25:46.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:46.895+0000 7f2b73e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b741060a0 0x7f2b7419dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:46.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:46.895+0000 7f2b72a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2b74101970 0x7f2b74199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:46.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:46.895+0000 7f2b73e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b741060a0 0x7f2b7419dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51868/0 (socket says 172.21.15.1:51868) 2024-02-02T15:25:46.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:46.895+0000 7f2b73400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b74100fc0 0x7f2b74199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:46.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:46.895+0000 7f2b73e00700 1 -- 172.21.15.1:0/2389259608 learned_addr learned my addr 172.21.15.1:0/2389259608 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:25:46.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:46.896+0000 7f2b72a00700 1 -- 172.21.15.1:0/2389259608 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b74100fc0 msgr2=0x7f2b74199230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:46.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:46.896+0000 7f2b72a00700 1 --2- 172.21.15.1:0/2389259608 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b74100fc0 0x7f2b74199230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:46.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:46.896+0000 7f2b72a00700 1 -- 172.21.15.1:0/2389259608 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b741060a0 msgr2=0x7f2b7419dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:46.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:46.896+0000 7f2b72a00700 1 --2- 172.21.15.1:0/2389259608 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b741060a0 0x7f2b7419dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:46.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:46.896+0000 7f2b72a00700 1 -- 172.21.15.1:0/2389259608 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b6c009d00 con 0x7f2b74101970 2024-02-02T15:25:46.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:46.896+0000 7f2b72a00700 1 --2- 172.21.15.1:0/2389259608 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2b74101970 0x7f2b74199770 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c00e9c0 tx=0x7f2b5c00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:46.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:46.897+0000 7f2b6be00700 1 -- 172.21.15.1:0/2389259608 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c00c9c0 con 0x7f2b74101970 2024-02-02T15:25:46.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:46.897+0000 7f2b6be00700 1 -- 172.21.15.1:0/2389259608 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b5c010430 con 0x7f2b74101970 2024-02-02T15:25:46.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:46.897+0000 7f2b7a200700 1 -- 172.21.15.1:0/2389259608 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b7419e800 con 0x7f2b74101970 2024-02-02T15:25:46.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:46.897+0000 7f2b6be00700 1 -- 172.21.15.1:0/2389259608 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b5c018680 con 0x7f2b74101970 2024-02-02T15:25:46.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:46.897+0000 7f2b7a200700 1 -- 172.21.15.1:0/2389259608 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b741a6060 con 0x7f2b74101970 2024-02-02T15:25:46.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:46.898+0000 7f2b7a200700 1 -- 172.21.15.1:0/2389259608 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2b74110ff0 con 0x7f2b74101970 2024-02-02T15:25:46.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:46.900+0000 7f2b6be00700 1 -- 172.21.15.1:0/2389259608 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 135120+0+0 (secure 0 0 0) 0x7f2b5c00cb20 con 0x7f2b74101970 2024-02-02T15:25:46.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:46.902+0000 7f2b6be00700 1 --2- 172.21.15.1:0/2389259608 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f2b540a05b0 0x7f2b540a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:46.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:46.903+0000 7f2b6be00700 1 -- 172.21.15.1:0/2389259608 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f2b5c014070 con 0x7f2b74101970 2024-02-02T15:25:46.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:46.903+0000 7f2b73400700 1 --2- 172.21.15.1:0/2389259608 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f2b540a05b0 0x7f2b540a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:46.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:46.903+0000 7f2b6be00700 1 --2- 172.21.15.1:0/2389259608 >> [v2:172.21.15.71:6824/84691841,v1:172.21.15.71:6825/84691841] conn(0x7f2b540ab5d0 0x7f2b540ada70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:46.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:46.903+0000 7f2b6be00700 1 -- 172.21.15.1:0/2389259608 --> [v2:172.21.15.71:6824/84691841,v1:172.21.15.71:6825/84691841] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2b540ae140 con 0x7f2b540ab5d0 2024-02-02T15:25:46.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:46.903+0000 7f2b6be00700 1 -- 172.21.15.1:0/2389259608 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2b5c0cc550 con 0x7f2b74101970 2024-02-02T15:25:46.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:46.904+0000 7f2b73400700 1 --2- 172.21.15.1:0/2389259608 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f2b540a05b0 0x7f2b540a2a70 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2b64007be0 tx=0x7f2b640073d0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:46.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:46.904+0000 7f2b73e00700 1 --2- 172.21.15.1:0/2389259608 >> [v2:172.21.15.71:6824/84691841,v1:172.21.15.71:6825/84691841] conn(0x7f2b540ab5d0 0x7f2b540ada70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:46.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:46.904+0000 7f2b73e00700 1 --2- 172.21.15.1:0/2389259608 >> [v2:172.21.15.71:6824/84691841,v1:172.21.15.71:6825/84691841] conn(0x7f2b540ab5d0 0x7f2b540ada70 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:46.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:46.905+0000 7f2b6be00700 1 -- 172.21.15.1:0/2389259608 <== osd.10 v2:172.21.15.71:6824/84691841 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f2b540ae140 con 0x7f2b540ab5d0 2024-02-02T15:25:46.913 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.a/config 2024-02-02T15:25:46.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:46.924+0000 7f2b7a200700 1 -- 172.21.15.1:0/2389259608 --> [v2:172.21.15.71:6824/84691841,v1:172.21.15.71:6825/84691841] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2b7404ea40 con 0x7f2b540ab5d0 2024-02-02T15:25:46.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:46.925+0000 7f2b6be00700 1 -- 172.21.15.1:0/2389259608 <== osd.10 v2:172.21.15.71:6824/84691841 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f2b7404ea40 con 0x7f2b540ab5d0 2024-02-02T15:25:46.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:46.925+0000 7f2b7a200700 1 -- 172.21.15.1:0/2389259608 >> [v2:172.21.15.71:6824/84691841,v1:172.21.15.71:6825/84691841] conn(0x7f2b540ab5d0 msgr2=0x7f2b540ada70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:46.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:46.925+0000 7f2b7a200700 1 --2- 172.21.15.1:0/2389259608 >> [v2:172.21.15.71:6824/84691841,v1:172.21.15.71:6825/84691841] conn(0x7f2b540ab5d0 0x7f2b540ada70 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:46.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:46.926+0000 7f2b7a200700 1 -- 172.21.15.1:0/2389259608 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f2b540a05b0 msgr2=0x7f2b540a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:46.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:46.926+0000 7f2b7a200700 1 --2- 172.21.15.1:0/2389259608 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f2b540a05b0 0x7f2b540a2a70 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2b64007be0 tx=0x7f2b640073d0 comp rx=0 tx=0).stop 2024-02-02T15:25:46.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:46.926+0000 7f2b7a200700 1 -- 172.21.15.1:0/2389259608 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2b74101970 msgr2=0x7f2b74199770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:46.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:46.926+0000 7f2b7a200700 1 --2- 172.21.15.1:0/2389259608 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2b74101970 0x7f2b74199770 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c00e9c0 tx=0x7f2b5c00ecd0 comp rx=0 tx=0).stop 2024-02-02T15:25:46.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:46.926+0000 7f2b73e00700 1 -- 172.21.15.1:0/2389259608 reap_dead start 2024-02-02T15:25:46.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:46.926+0000 7f2b7a200700 1 -- 172.21.15.1:0/2389259608 shutdown_connections 2024-02-02T15:25:46.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:46.926+0000 7f2b7a200700 1 -- 172.21.15.1:0/2389259608 >> 172.21.15.1:0/2389259608 conn(0x7f2b740fc830 msgr2=0x7f2b740fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:46.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:46.926+0000 7f2b7a200700 1 -- 172.21.15.1:0/2389259608 shutdown_connections 2024-02-02T15:25:46.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:46.926+0000 7f2b7a200700 1 -- 172.21.15.1:0/2389259608 wait complete. 2024-02-02T15:25:47.361 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:47 smithi001 ceph-mon[35470]: pgmap v266: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 9 op/s 2024-02-02T15:25:47.362 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:47 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:25:47.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:47 smithi071 ceph-mon[44707]: pgmap v266: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 9 op/s 2024-02-02T15:25:47.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:47 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:25:47.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:47 smithi142 ceph-mon[45728]: pgmap v266: 129 pgs: 129 active+clean; 641 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 767 B/s wr, 9 op/s 2024-02-02T15:25:47.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:47 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:25:48.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.244+0000 7f9544200700 1 -- 172.21.15.1:0/3751801267 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f953c1060a0 msgr2=0x7f953c10dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:48.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.244+0000 7f9544200700 1 --2- 172.21.15.1:0/3751801267 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f953c1060a0 0x7f953c10dc70 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f953400b0a0 tx=0x7f953400b3b0 comp rx=0 tx=0).stop 2024-02-02T15:25:48.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.244+0000 7f9544200700 1 -- 172.21.15.1:0/3751801267 shutdown_connections 2024-02-02T15:25:48.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.244+0000 7f9544200700 1 --2- 172.21.15.1:0/3751801267 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f953c1060a0 0x7f953c10dc70 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:48.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.244+0000 7f9544200700 1 --2- 172.21.15.1:0/3751801267 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f953c100fc0 0x7f953c1013a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:48.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.244+0000 7f9544200700 1 --2- 172.21.15.1:0/3751801267 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f953c101970 0x7f953c1059c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:48.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.244+0000 7f9544200700 1 -- 172.21.15.1:0/3751801267 >> 172.21.15.1:0/3751801267 conn(0x7f953c0fc830 msgr2=0x7f953c0fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:48.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.244+0000 7f9544200700 1 -- 172.21.15.1:0/3751801267 shutdown_connections 2024-02-02T15:25:48.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.244+0000 7f9544200700 1 -- 172.21.15.1:0/3751801267 wait complete. 2024-02-02T15:25:48.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.245+0000 7f9544200700 1 Processor -- start 2024-02-02T15:25:48.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.245+0000 7f9544200700 1 -- start start 2024-02-02T15:25:48.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.245+0000 7f9544200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f953c100fc0 0x7f953c073040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:48.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.245+0000 7f9544200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f953c101970 0x7f953c073580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:48.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.245+0000 7f9544200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f953c1060a0 0x7f953c073ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:48.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.245+0000 7f9544200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f953c1a9910 con 0x7f953c1060a0 2024-02-02T15:25:48.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.245+0000 7f9544200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f953c1a9a50 con 0x7f953c100fc0 2024-02-02T15:25:48.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.245+0000 7f9544200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f953c1a9b90 con 0x7f953c101970 2024-02-02T15:25:48.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.246+0000 7f9542200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f953c1060a0 0x7f953c073ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:48.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.246+0000 7f9540e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f953c101970 0x7f953c073580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:48.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.246+0000 7f9542200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f953c1060a0 0x7f953c073ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51886/0 (socket says 172.21.15.1:51886) 2024-02-02T15:25:48.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.246+0000 7f9542200700 1 -- 172.21.15.1:0/4024452290 learned_addr learned my addr 172.21.15.1:0/4024452290 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:25:48.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.246+0000 7f9541800700 1 --2- 172.21.15.1:0/4024452290 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f953c100fc0 0x7f953c073040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:48.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.246+0000 7f9540e00700 1 -- 172.21.15.1:0/4024452290 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f953c100fc0 msgr2=0x7f953c073040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:48.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.246+0000 7f9540e00700 1 --2- 172.21.15.1:0/4024452290 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f953c100fc0 0x7f953c073040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:48.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.246+0000 7f9540e00700 1 -- 172.21.15.1:0/4024452290 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f953c1060a0 msgr2=0x7f953c073ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:48.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.246+0000 7f9540e00700 1 --2- 172.21.15.1:0/4024452290 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f953c1060a0 0x7f953c073ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:48.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.246+0000 7f9540e00700 1 -- 172.21.15.1:0/4024452290 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9534009d00 con 0x7f953c101970 2024-02-02T15:25:48.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.246+0000 7f9542200700 1 --2- 172.21.15.1:0/4024452290 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f953c1060a0 0x7f953c073ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:25:48.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.247+0000 7f9540e00700 1 --2- 172.21.15.1:0/4024452290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f953c101970 0x7f953c073580 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f952400e9f0 tx=0x7f952400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:48.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.247+0000 7f953a000700 1 -- 172.21.15.1:0/4024452290 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f952400c9c0 con 0x7f953c101970 2024-02-02T15:25:48.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.247+0000 7f953a000700 1 -- 172.21.15.1:0/4024452290 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9524010430 con 0x7f953c101970 2024-02-02T15:25:48.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.247+0000 7f9544200700 1 -- 172.21.15.1:0/4024452290 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f953c1a9e70 con 0x7f953c101970 2024-02-02T15:25:48.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.247+0000 7f953a000700 1 -- 172.21.15.1:0/4024452290 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9524018680 con 0x7f953c101970 2024-02-02T15:25:48.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.247+0000 7f9544200700 1 -- 172.21.15.1:0/4024452290 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f953c1aa330 con 0x7f953c101970 2024-02-02T15:25:48.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.248+0000 7f9544200700 1 -- 172.21.15.1:0/4024452290 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f953c04ea40 con 0x7f953c101970 2024-02-02T15:25:48.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.250+0000 7f953a000700 1 -- 172.21.15.1:0/4024452290 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 17) v1 ==== 135120+0+0 (secure 0 0 0) 0x7f95240105a0 con 0x7f953c101970 2024-02-02T15:25:48.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.251+0000 7f953a000700 1 --2- 172.21.15.1:0/4024452290 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f951c0a0630 0x7f951c0a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:48.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.252+0000 7f953a000700 1 -- 172.21.15.1:0/4024452290 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f9524014070 con 0x7f953c101970 2024-02-02T15:25:48.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.252+0000 7f9541800700 1 --2- 172.21.15.1:0/4024452290 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f951c0a0630 0x7f951c0a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:48.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.252+0000 7f9541800700 1 --2- 172.21.15.1:0/4024452290 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f951c0a0630 0x7f951c0a2af0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f952c009c60 tx=0x7f952c009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:48.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.253+0000 7f953a000700 1 -- 172.21.15.1:0/4024452290 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f9524095940 con 0x7f953c101970 2024-02-02T15:25:48.394 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.a/config 2024-02-02T15:25:48.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.424+0000 7f3da6200700 1 -- 172.21.15.1:0/3626994587 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3da0105d00 msgr2=0x7f3da0103790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:48.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.424+0000 7f3da6200700 1 --2- 172.21.15.1:0/3626994587 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3da0105d00 0x7f3da0103790 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3d90009990 tx=0x7f3d90009ca0 comp rx=0 tx=0).stop 2024-02-02T15:25:48.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.424+0000 7f3da6200700 1 -- 172.21.15.1:0/3626994587 shutdown_connections 2024-02-02T15:25:48.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.424+0000 7f3da6200700 1 --2- 172.21.15.1:0/3626994587 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3da0068860 0x7f3da0068ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:48.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.424+0000 7f3da6200700 1 --2- 172.21.15.1:0/3626994587 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3da0105d00 0x7f3da0103790 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:48.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.424+0000 7f3da6200700 1 --2- 172.21.15.1:0/3626994587 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3da0103cd0 0x7f3da0104150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:48.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.424+0000 7f3da6200700 1 -- 172.21.15.1:0/3626994587 >> 172.21.15.1:0/3626994587 conn(0x7f3da0076d40 msgr2=0x7f3da0077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:48.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.425+0000 7f3da6200700 1 -- 172.21.15.1:0/3626994587 shutdown_connections 2024-02-02T15:25:48.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.425+0000 7f3da6200700 1 -- 172.21.15.1:0/3626994587 wait complete. 2024-02-02T15:25:48.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.426+0000 7f3da6200700 1 Processor -- start 2024-02-02T15:25:48.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.426+0000 7f3da6200700 1 -- start start 2024-02-02T15:25:48.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.427+0000 7f3da6200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3da0068860 0x7f3da019d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:48.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.427+0000 7f3da6200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3da0103cd0 0x7f3da019da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:48.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.427+0000 7f3da6200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3da0105d00 0x7f3da01a1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:48.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.427+0000 7f3da6200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3da01a24a0 con 0x7f3da0103cd0 2024-02-02T15:25:48.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.427+0000 7f3da6200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3da01a2610 con 0x7f3da0068860 2024-02-02T15:25:48.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.427+0000 7f3da6200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3da01a27d0 con 0x7f3da0105d00 2024-02-02T15:25:48.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.428+0000 7f3d9fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3da0105d00 0x7f3da01a1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:48.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.428+0000 7f3d9f400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3da0068860 0x7f3da019d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:48.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.428+0000 7f3d9fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3da0105d00 0x7f3da01a1e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:43216/0 (socket says 172.21.15.1:43216) 2024-02-02T15:25:48.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.428+0000 7f3d9fe00700 1 -- 172.21.15.1:0/3145921591 learned_addr learned my addr 172.21.15.1:0/3145921591 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:25:48.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.428+0000 7f3d9ea00700 1 --2- 172.21.15.1:0/3145921591 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3da0103cd0 0x7f3da019da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:48.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.428+0000 7f3d9fe00700 1 -- 172.21.15.1:0/3145921591 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3da0068860 msgr2=0x7f3da019d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:48.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.428+0000 7f3d9fe00700 1 --2- 172.21.15.1:0/3145921591 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3da0068860 0x7f3da019d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:48.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.428+0000 7f3d9fe00700 1 -- 172.21.15.1:0/3145921591 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3da0103cd0 msgr2=0x7f3da019da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:48.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.428+0000 7f3d9fe00700 1 --2- 172.21.15.1:0/3145921591 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3da0103cd0 0x7f3da019da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:48.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.428+0000 7f3d9fe00700 1 -- 172.21.15.1:0/3145921591 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d900095f0 con 0x7f3da0105d00 2024-02-02T15:25:48.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.428+0000 7f3d9ea00700 1 --2- 172.21.15.1:0/3145921591 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3da0103cd0 0x7f3da019da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:25:48.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.429+0000 7f3d9fe00700 1 --2- 172.21.15.1:0/3145921591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3da0105d00 0x7f3da01a1e90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3d9800efc0 tx=0x7f3d9800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:48.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.429+0000 7f3d97e00700 1 -- 172.21.15.1:0/3145921591 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d98018070 con 0x7f3da0105d00 2024-02-02T15:25:48.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.429+0000 7f3d97e00700 1 -- 172.21.15.1:0/3145921591 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d9800f040 con 0x7f3da0105d00 2024-02-02T15:25:48.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.429+0000 7f3da6200700 1 -- 172.21.15.1:0/3145921591 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3da01a2a50 con 0x7f3da0105d00 2024-02-02T15:25:48.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.429+0000 7f3d97e00700 1 -- 172.21.15.1:0/3145921591 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d9800cc10 con 0x7f3da0105d00 2024-02-02T15:25:48.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.429+0000 7f3da6200700 1 -- 172.21.15.1:0/3145921591 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3da01aa310 con 0x7f3da0105d00 2024-02-02T15:25:48.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.432+0000 7f3da6200700 1 -- 172.21.15.1:0/3145921591 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3da004ea40 con 0x7f3da0105d00 2024-02-02T15:25:48.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.433+0000 7f3d97e00700 1 -- 172.21.15.1:0/3145921591 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 17) v1 ==== 135120+0+0 (secure 0 0 0) 0x7f3d98003a00 con 0x7f3da0105d00 2024-02-02T15:25:48.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.435+0000 7f3d97e00700 1 --2- 172.21.15.1:0/3145921591 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f3d800a05f0 0x7f3d800a2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:48.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.435+0000 7f3d97e00700 1 -- 172.21.15.1:0/3145921591 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f3d980cb850 con 0x7f3da0105d00 2024-02-02T15:25:48.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.435+0000 7f3d9f400700 1 --2- 172.21.15.1:0/3145921591 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f3d800a05f0 0x7f3d800a2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:48.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.436+0000 7f3d9f400700 1 --2- 172.21.15.1:0/3145921591 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f3d800a05f0 0x7f3d800a2ab0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3d9000b2e0 tx=0x7f3d90005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:48.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.439+0000 7f3d97e00700 1 -- 172.21.15.1:0/3145921591 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f3d98094fb0 con 0x7f3da0105d00 2024-02-02T15:25:48.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.551+0000 7f9544200700 1 -- 172.21.15.1:0/4024452290 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f953c074570 con 0x7f953c101970 2024-02-02T15:25:48.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.552+0000 7f953a000700 1 -- 172.21.15.1:0/4024452290 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9524026020 con 0x7f953c101970 2024-02-02T15:25:48.552 INFO:teuthology.orchestra.run.smithi001.stdout:77309411383 2024-02-02T15:25:48.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.554+0000 7f9544200700 1 -- 172.21.15.1:0/4024452290 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f951c0a0630 msgr2=0x7f951c0a2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:48.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.554+0000 7f9544200700 1 --2- 172.21.15.1:0/4024452290 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f951c0a0630 0x7f951c0a2af0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f952c009c60 tx=0x7f952c009260 comp rx=0 tx=0).stop 2024-02-02T15:25:48.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.554+0000 7f9544200700 1 -- 172.21.15.1:0/4024452290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f953c101970 msgr2=0x7f953c073580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:48.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.554+0000 7f9544200700 1 --2- 172.21.15.1:0/4024452290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f953c101970 0x7f953c073580 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f952400e9f0 tx=0x7f952400ed00 comp rx=0 tx=0).stop 2024-02-02T15:25:48.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.554+0000 7f9544200700 1 -- 172.21.15.1:0/4024452290 shutdown_connections 2024-02-02T15:25:48.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.554+0000 7f9544200700 1 --2- 172.21.15.1:0/4024452290 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f953c1060a0 0x7f953c073ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:48.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.554+0000 7f9544200700 1 --2- 172.21.15.1:0/4024452290 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f953c100fc0 0x7f953c073040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:48.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.554+0000 7f9544200700 1 --2- 172.21.15.1:0/4024452290 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f951c0a0630 0x7f951c0a2af0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:48.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.554+0000 7f9544200700 1 --2- 172.21.15.1:0/4024452290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f953c101970 0x7f953c073580 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:48.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.554+0000 7f9544200700 1 -- 172.21.15.1:0/4024452290 >> 172.21.15.1:0/4024452290 conn(0x7f953c0fc830 msgr2=0x7f953c0fd270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:48.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.554+0000 7f9544200700 1 -- 172.21.15.1:0/4024452290 shutdown_connections 2024-02-02T15:25:48.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.554+0000 7f9544200700 1 -- 172.21.15.1:0/4024452290 wait complete. 2024-02-02T15:25:48.572 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.a/config 2024-02-02T15:25:48.574 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:48 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:25:48.574 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:48 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:25:48.574 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:48 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:25:48.574 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:48 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:25:48.574 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:48 smithi001 ceph-mon[35470]: pgmap v267: 129 pgs: 129 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1023 B/s wr, 11 op/s 2024-02-02T15:25:48.607 INFO:teuthology.orchestra.run.smithi001.stdout:287762808857 2024-02-02T15:25:48.607 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph osd last-stat-seq osd.10 2024-02-02T15:25:48.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.693+0000 7f3da6200700 1 -- 172.21.15.1:0/3145921591 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7f3da01aa4d0 con 0x7f3da0105d00 2024-02-02T15:25:48.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.693+0000 7f3d97e00700 1 -- 172.21.15.1:0/3145921591 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3d980985d0 con 0x7f3da0105d00 2024-02-02T15:25:48.694 INFO:teuthology.orchestra.run.smithi001.stdout:236223201313 2024-02-02T15:25:48.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.696+0000 7f3da6200700 1 -- 172.21.15.1:0/3145921591 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f3d800a05f0 msgr2=0x7f3d800a2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:48.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.696+0000 7f3da6200700 1 --2- 172.21.15.1:0/3145921591 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f3d800a05f0 0x7f3d800a2ab0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3d9000b2e0 tx=0x7f3d90005fb0 comp rx=0 tx=0).stop 2024-02-02T15:25:48.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.696+0000 7f3da6200700 1 -- 172.21.15.1:0/3145921591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3da0105d00 msgr2=0x7f3da01a1e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:48.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.696+0000 7f3da6200700 1 --2- 172.21.15.1:0/3145921591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3da0105d00 0x7f3da01a1e90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3d9800efc0 tx=0x7f3d9800c3a0 comp rx=0 tx=0).stop 2024-02-02T15:25:48.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.696+0000 7f3da6200700 1 -- 172.21.15.1:0/3145921591 shutdown_connections 2024-02-02T15:25:48.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.696+0000 7f3da6200700 1 --2- 172.21.15.1:0/3145921591 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f3d800a05f0 0x7f3d800a2ab0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:48.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.696+0000 7f3da6200700 1 --2- 172.21.15.1:0/3145921591 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3da0105d00 0x7f3da01a1e90 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:48.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.696+0000 7f3da6200700 1 --2- 172.21.15.1:0/3145921591 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3da0068860 0x7f3da019d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:48.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.696+0000 7f3da6200700 1 --2- 172.21.15.1:0/3145921591 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3da0103cd0 0x7f3da019da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:48.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.696+0000 7f3da6200700 1 -- 172.21.15.1:0/3145921591 >> 172.21.15.1:0/3145921591 conn(0x7f3da0076d40 msgr2=0x7f3da00fda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:48.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.696+0000 7f3da6200700 1 -- 172.21.15.1:0/3145921591 shutdown_connections 2024-02-02T15:25:48.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:48.696+0000 7f3da6200700 1 -- 172.21.15.1:0/3145921591 wait complete. 2024-02-02T15:25:48.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:48 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:25:48.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:48 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:25:48.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:48 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:25:48.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:48 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:25:48.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:48 smithi071 ceph-mon[44707]: pgmap v267: 129 pgs: 129 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1023 B/s wr, 11 op/s 2024-02-02T15:25:48.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:48 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:25:48.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:48 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:25:48.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:48 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:25:48.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:48 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:25:48.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:48 smithi142 ceph-mon[45728]: pgmap v267: 129 pgs: 129 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1023 B/s wr, 11 op/s 2024-02-02T15:25:49.648 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:49 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/4024452290' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-02T15:25:49.648 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:49 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/3145921591' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-02-02T15:25:49.671 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411381 got 77309411383 for osd.2 2024-02-02T15:25:49.671 DEBUG:teuthology.parallel:result is None 2024-02-02T15:25:49.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:49 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/4024452290' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-02T15:25:49.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:49 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/3145921591' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-02-02T15:25:49.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:49 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/4024452290' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-02T15:25:49.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:49 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/3145921591' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-02-02T15:25:50.003 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.a/config 2024-02-02T15:25:50.047 INFO:tasks.cephadm.ceph_manager.ceph:need seq 236223201312 got 236223201313 for osd.8 2024-02-02T15:25:50.047 DEBUG:teuthology.parallel:result is None 2024-02-02T15:25:50.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:49.883+0000 7f6c02e00700 1 -- 172.21.15.1:0/3655295005 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6bfc106ed0 msgr2=0x7f6bfc1072b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:50.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:49.883+0000 7f6c02e00700 1 --2- 172.21.15.1:0/3655295005 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6bfc106ed0 0x7f6bfc1072b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6bec009990 tx=0x7f6bec009ca0 comp rx=0 tx=0).stop 2024-02-02T15:25:50.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:49.883+0000 7f6c02e00700 1 -- 172.21.15.1:0/3655295005 shutdown_connections 2024-02-02T15:25:50.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:49.883+0000 7f6c02e00700 1 --2- 172.21.15.1:0/3655295005 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6bfc069220 0x7f6bfc10db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:50.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:49.883+0000 7f6c02e00700 1 --2- 172.21.15.1:0/3655295005 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6bfc106ed0 0x7f6bfc1072b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:50.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:49.883+0000 7f6c02e00700 1 --2- 172.21.15.1:0/3655295005 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6bfc068860 0x7f6bfc068ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:50.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:49.883+0000 7f6c02e00700 1 -- 172.21.15.1:0/3655295005 >> 172.21.15.1:0/3655295005 conn(0x7f6bfc076d40 msgr2=0x7f6bfc077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:50.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:49.883+0000 7f6c02e00700 1 -- 172.21.15.1:0/3655295005 shutdown_connections 2024-02-02T15:25:50.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:49.883+0000 7f6c02e00700 1 -- 172.21.15.1:0/3655295005 wait complete. 2024-02-02T15:25:50.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:49.884+0000 7f6c02e00700 1 Processor -- start 2024-02-02T15:25:50.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:49.884+0000 7f6c02e00700 1 -- start start 2024-02-02T15:25:50.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:49.884+0000 7f6c02e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6bfc068860 0x7f6bfc196d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:50.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:49.884+0000 7f6c02e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6bfc069220 0x7f6bfc1972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:50.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:49.884+0000 7f6c02e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6bfc106ed0 0x7f6bfc19b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:50.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:49.884+0000 7f6c02e00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bfc19bd40 con 0x7f6bfc106ed0 2024-02-02T15:25:50.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:49.884+0000 7f6c02e00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bfc19beb0 con 0x7f6bfc068860 2024-02-02T15:25:50.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:49.884+0000 7f6c02e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bfc19c070 con 0x7f6bfc069220 2024-02-02T15:25:50.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:49.885+0000 7f6bfb400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6bfc069220 0x7f6bfc1972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:50.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:49.885+0000 7f6c00e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6bfc106ed0 0x7f6bfc19b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:50.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:49.885+0000 7f6bfb400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6bfc069220 0x7f6bfc1972c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:43226/0 (socket says 172.21.15.1:43226) 2024-02-02T15:25:50.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:49.885+0000 7f6bfbe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6bfc068860 0x7f6bfc196d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:50.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:49.885+0000 7f6bfb400700 1 -- 172.21.15.1:0/3659536830 learned_addr learned my addr 172.21.15.1:0/3659536830 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:25:50.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:49.885+0000 7f6bfb400700 1 -- 172.21.15.1:0/3659536830 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6bfc068860 msgr2=0x7f6bfc196d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:50.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:49.885+0000 7f6bfb400700 1 --2- 172.21.15.1:0/3659536830 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6bfc068860 0x7f6bfc196d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:50.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:49.885+0000 7f6bfb400700 1 -- 172.21.15.1:0/3659536830 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6bfc106ed0 msgr2=0x7f6bfc19b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:50.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:49.885+0000 7f6bfb400700 1 --2- 172.21.15.1:0/3659536830 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6bfc106ed0 0x7f6bfc19b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:50.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:49.885+0000 7f6bfb400700 1 -- 172.21.15.1:0/3659536830 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bec0095f0 con 0x7f6bfc069220 2024-02-02T15:25:50.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:49.886+0000 7f6bfb400700 1 --2- 172.21.15.1:0/3659536830 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6bfc069220 0x7f6bfc1972c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6be400e9a0 tx=0x7f6be400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:50.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:49.886+0000 7f6bf8c00700 1 -- 172.21.15.1:0/3659536830 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6be4009790 con 0x7f6bfc069220 2024-02-02T15:25:50.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:49.886+0000 7f6bf8c00700 1 -- 172.21.15.1:0/3659536830 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6be4010430 con 0x7f6bfc069220 2024-02-02T15:25:50.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:49.886+0000 7f6bf8c00700 1 -- 172.21.15.1:0/3659536830 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6be4018680 con 0x7f6bfc069220 2024-02-02T15:25:50.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:49.886+0000 7f6c02e00700 1 -- 172.21.15.1:0/3659536830 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bfc19c350 con 0x7f6bfc069220 2024-02-02T15:25:50.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:49.886+0000 7f6c02e00700 1 -- 172.21.15.1:0/3659536830 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bfc1a3bb0 con 0x7f6bfc069220 2024-02-02T15:25:50.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:49.887+0000 7f6c02e00700 1 -- 172.21.15.1:0/3659536830 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bfc04ea40 con 0x7f6bfc069220 2024-02-02T15:25:50.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:49.889+0000 7f6bf8c00700 1 -- 172.21.15.1:0/3659536830 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 17) v1 ==== 135120+0+0 (secure 0 0 0) 0x7f6be40105a0 con 0x7f6bfc069220 2024-02-02T15:25:50.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:49.890+0000 7f6bf8c00700 1 --2- 172.21.15.1:0/3659536830 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f6bdc0a0630 0x7f6bdc0a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:50.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:49.890+0000 7f6bf8c00700 1 -- 172.21.15.1:0/3659536830 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f6be4014070 con 0x7f6bfc069220 2024-02-02T15:25:50.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:49.891+0000 7f6bfbe00700 1 --2- 172.21.15.1:0/3659536830 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f6bdc0a0630 0x7f6bdc0a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:50.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:49.891+0000 7f6bfbe00700 1 --2- 172.21.15.1:0/3659536830 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f6bdc0a0630 0x7f6bdc0a2af0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6bec00b2e0 tx=0x7f6bec005960 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:50.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:49.895+0000 7f6bf8c00700 1 -- 172.21.15.1:0/3659536830 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f6be4052110 con 0x7f6bfc069220 2024-02-02T15:25:50.078 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.a/config 2024-02-02T15:25:50.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:50.138+0000 7f6c02e00700 1 -- 172.21.15.1:0/3659536830 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f6bfc1980e0 con 0x7f6bfc069220 2024-02-02T15:25:50.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:50.139+0000 7f6bf8c00700 1 -- 172.21.15.1:0/3659536830 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6be40956b0 con 0x7f6bfc069220 2024-02-02T15:25:50.139 INFO:teuthology.orchestra.run.smithi001.stdout:103079215155 2024-02-02T15:25:50.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:50.143+0000 7f6c02e00700 1 -- 172.21.15.1:0/3659536830 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f6bdc0a0630 msgr2=0x7f6bdc0a2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:50.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:50.143+0000 7f6c02e00700 1 --2- 172.21.15.1:0/3659536830 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f6bdc0a0630 0x7f6bdc0a2af0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6bec00b2e0 tx=0x7f6bec005960 comp rx=0 tx=0).stop 2024-02-02T15:25:50.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:50.143+0000 7f6c02e00700 1 -- 172.21.15.1:0/3659536830 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6bfc069220 msgr2=0x7f6bfc1972c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:50.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:50.143+0000 7f6c02e00700 1 --2- 172.21.15.1:0/3659536830 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6bfc069220 0x7f6bfc1972c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6be400e9a0 tx=0x7f6be400ecb0 comp rx=0 tx=0).stop 2024-02-02T15:25:50.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:50.143+0000 7f6c02e00700 1 -- 172.21.15.1:0/3659536830 shutdown_connections 2024-02-02T15:25:50.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:50.143+0000 7f6c02e00700 1 --2- 172.21.15.1:0/3659536830 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6bfc106ed0 0x7f6bfc19b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:50.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:50.143+0000 7f6c02e00700 1 --2- 172.21.15.1:0/3659536830 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6bfc068860 0x7f6bfc196d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:50.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:50.143+0000 7f6c02e00700 1 --2- 172.21.15.1:0/3659536830 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f6bdc0a0630 0x7f6bdc0a2af0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:50.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:50.143+0000 7f6c02e00700 1 --2- 172.21.15.1:0/3659536830 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6bfc069220 0x7f6bfc1972c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:50.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:50.143+0000 7f6c02e00700 1 -- 172.21.15.1:0/3659536830 >> 172.21.15.1:0/3659536830 conn(0x7f6bfc076d40 msgr2=0x7f6bfc0fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:50.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:50.143+0000 7f6c02e00700 1 -- 172.21.15.1:0/3659536830 shutdown_connections 2024-02-02T15:25:50.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:50.143+0000 7f6c02e00700 1 -- 172.21.15.1:0/3659536830 wait complete. 2024-02-02T15:25:50.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:50.550+0000 7f50de200700 1 -- 172.21.15.1:0/942627843 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f50d810d610 msgr2=0x7f50d810fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:50.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:50.550+0000 7f50de200700 1 --2- 172.21.15.1:0/942627843 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f50d810d610 0x7f50d810fa20 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f50d000b0a0 tx=0x7f50d000b3b0 comp rx=0 tx=0).stop 2024-02-02T15:25:50.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:50.550+0000 7f50de200700 1 -- 172.21.15.1:0/942627843 shutdown_connections 2024-02-02T15:25:50.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:50.550+0000 7f50de200700 1 --2- 172.21.15.1:0/942627843 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f50d810d610 0x7f50d810fa20 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:50.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:50.550+0000 7f50de200700 1 --2- 172.21.15.1:0/942627843 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50d8073110 0x7f50d80734f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:50.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:50.550+0000 7f50de200700 1 --2- 172.21.15.1:0/942627843 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50d8073a30 0x7f50d810d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:50.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:50.550+0000 7f50de200700 1 -- 172.21.15.1:0/942627843 >> 172.21.15.1:0/942627843 conn(0x7f50d80fc860 msgr2=0x7f50d80fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:50.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:50.550+0000 7f50de200700 1 -- 172.21.15.1:0/942627843 shutdown_connections 2024-02-02T15:25:50.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:50.550+0000 7f50de200700 1 -- 172.21.15.1:0/942627843 wait complete. 2024-02-02T15:25:50.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:50.551+0000 7f50de200700 1 Processor -- start 2024-02-02T15:25:50.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:50.552+0000 7f50de200700 1 -- start start 2024-02-02T15:25:50.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:50.552+0000 7f50de200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50d8073110 0x7f50d81991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:50.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:50.552+0000 7f50de200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50d8073a30 0x7f50d8199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:50.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:50.552+0000 7f50de200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f50d810d610 0x7f50d819dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:50.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:50.552+0000 7f50de200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d819e1b0 con 0x7f50d810d610 2024-02-02T15:25:50.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:50.552+0000 7f50de200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d819e320 con 0x7f50d8073a30 2024-02-02T15:25:50.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:50.552+0000 7f50de200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d819e4e0 con 0x7f50d8073110 2024-02-02T15:25:50.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:50.552+0000 7f50d7400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50d8073110 0x7f50d81991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:50.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:50.552+0000 7f50d7e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f50d810d610 0x7f50d819dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:50.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:50.552+0000 7f50d7400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50d8073110 0x7f50d81991f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:43238/0 (socket says 172.21.15.1:43238) 2024-02-02T15:25:50.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:50.552+0000 7f50d7400700 1 -- 172.21.15.1:0/60122757 learned_addr learned my addr 172.21.15.1:0/60122757 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:25:50.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:50.552+0000 7f50d6a00700 1 --2- 172.21.15.1:0/60122757 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50d8073a30 0x7f50d8199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:50.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:50.553+0000 7f50d7400700 1 -- 172.21.15.1:0/60122757 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50d8073a30 msgr2=0x7f50d8199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:50.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:50.553+0000 7f50d7400700 1 --2- 172.21.15.1:0/60122757 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50d8073a30 0x7f50d8199730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:50.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:50.553+0000 7f50d7400700 1 -- 172.21.15.1:0/60122757 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f50d810d610 msgr2=0x7f50d819dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:50.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:50.553+0000 7f50d7400700 1 --2- 172.21.15.1:0/60122757 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f50d810d610 0x7f50d819dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:50.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:50.553+0000 7f50d7400700 1 -- 172.21.15.1:0/60122757 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50d0009d00 con 0x7f50d8073110 2024-02-02T15:25:50.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:50.553+0000 7f50d7e00700 1 --2- 172.21.15.1:0/60122757 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f50d810d610 0x7f50d819dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:25:50.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:50.553+0000 7f50d7400700 1 --2- 172.21.15.1:0/60122757 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50d8073110 0x7f50d81991f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f50c800e9a0 tx=0x7f50c800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:50.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:50.553+0000 7f50cfe00700 1 -- 172.21.15.1:0/60122757 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c8009790 con 0x7f50d8073110 2024-02-02T15:25:50.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:50.553+0000 7f50cfe00700 1 -- 172.21.15.1:0/60122757 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f50c8010430 con 0x7f50d8073110 2024-02-02T15:25:50.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:50.553+0000 7f50cfe00700 1 -- 172.21.15.1:0/60122757 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c8018680 con 0x7f50d8073110 2024-02-02T15:25:50.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:50.553+0000 7f50de200700 1 -- 172.21.15.1:0/60122757 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50d819e7c0 con 0x7f50d8073110 2024-02-02T15:25:50.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:50.553+0000 7f50de200700 1 -- 172.21.15.1:0/60122757 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50d81a6020 con 0x7f50d8073110 2024-02-02T15:25:50.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:50.554+0000 7f50de200700 1 -- 172.21.15.1:0/60122757 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50d8107f30 con 0x7f50d8073110 2024-02-02T15:25:50.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:50.556+0000 7f50cfe00700 1 -- 172.21.15.1:0/60122757 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 17) v1 ==== 135120+0+0 (secure 0 0 0) 0x7f50c80105a0 con 0x7f50d8073110 2024-02-02T15:25:50.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:50.557+0000 7f50cfe00700 1 --2- 172.21.15.1:0/60122757 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f50b80a0630 0x7f50b80a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:50.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:50.558+0000 7f50cfe00700 1 -- 172.21.15.1:0/60122757 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f50c8014070 con 0x7f50d8073110 2024-02-02T15:25:50.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:50.558+0000 7f50d6a00700 1 --2- 172.21.15.1:0/60122757 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f50b80a0630 0x7f50b80a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:50.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:50.558+0000 7f50d6a00700 1 --2- 172.21.15.1:0/60122757 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f50b80a0630 0x7f50b80a2af0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f50d819a810 tx=0x7f50c0009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:50.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:50.560+0000 7f50cfe00700 1 -- 172.21.15.1:0/60122757 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f50c8095860 con 0x7f50d8073110 2024-02-02T15:25:50.584 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:50 smithi001 ceph-mon[35470]: pgmap v268: 129 pgs: 129 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 597 B/s wr, 10 op/s 2024-02-02T15:25:50.584 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:50 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/3659536830' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-02T15:25:50.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:50.800+0000 7f50de200700 1 -- 172.21.15.1:0/60122757 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f50d804ea40 con 0x7f50d8073110 2024-02-02T15:25:50.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:50.801+0000 7f50cfe00700 1 -- 172.21.15.1:0/60122757 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f50c8098e80 con 0x7f50d8073110 2024-02-02T15:25:50.801 INFO:teuthology.orchestra.run.smithi001.stdout:154618822700 2024-02-02T15:25:50.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:50.803+0000 7f50de200700 1 -- 172.21.15.1:0/60122757 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f50b80a0630 msgr2=0x7f50b80a2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:50.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:50.803+0000 7f50de200700 1 --2- 172.21.15.1:0/60122757 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f50b80a0630 0x7f50b80a2af0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f50d819a810 tx=0x7f50c0009260 comp rx=0 tx=0).stop 2024-02-02T15:25:50.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:50.803+0000 7f50de200700 1 -- 172.21.15.1:0/60122757 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50d8073110 msgr2=0x7f50d81991f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:50.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:50.803+0000 7f50de200700 1 --2- 172.21.15.1:0/60122757 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50d8073110 0x7f50d81991f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f50c800e9a0 tx=0x7f50c800ecb0 comp rx=0 tx=0).stop 2024-02-02T15:25:50.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:50.803+0000 7f50de200700 1 -- 172.21.15.1:0/60122757 shutdown_connections 2024-02-02T15:25:50.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:50.803+0000 7f50de200700 1 --2- 172.21.15.1:0/60122757 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f50d810d610 0x7f50d819dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:50.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:50.803+0000 7f50de200700 1 --2- 172.21.15.1:0/60122757 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f50b80a0630 0x7f50b80a2af0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:50.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:50.803+0000 7f50de200700 1 --2- 172.21.15.1:0/60122757 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f50d8073110 0x7f50d81991f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:50.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:50.803+0000 7f50de200700 1 --2- 172.21.15.1:0/60122757 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f50d8073a30 0x7f50d8199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:50.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:50.803+0000 7f50de200700 1 -- 172.21.15.1:0/60122757 >> 172.21.15.1:0/60122757 conn(0x7f50d80fc860 msgr2=0x7f50d8105070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:50.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:50.804+0000 7f50de200700 1 -- 172.21.15.1:0/60122757 shutdown_connections 2024-02-02T15:25:50.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:50.804+0000 7f50de200700 1 -- 172.21.15.1:0/60122757 wait complete. 2024-02-02T15:25:50.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:50 smithi071 ceph-mon[44707]: pgmap v268: 129 pgs: 129 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 597 B/s wr, 10 op/s 2024-02-02T15:25:50.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:50 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/3659536830' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-02T15:25:50.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:50 smithi142 ceph-mon[45728]: pgmap v268: 129 pgs: 129 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 597 B/s wr, 10 op/s 2024-02-02T15:25:50.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:50 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/3659536830' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-02T15:25:51.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.554+0000 7f150a000700 1 -- 172.21.15.1:0/2835085248 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1504100760 msgr2=0x7f1504100b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:51.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.554+0000 7f150a000700 1 --2- 172.21.15.1:0/2835085248 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1504100760 0x7f1504100b40 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f14f4009990 tx=0x7f14f4009ca0 comp rx=0 tx=0).stop 2024-02-02T15:25:51.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.555+0000 7f150a000700 1 -- 172.21.15.1:0/2835085248 shutdown_connections 2024-02-02T15:25:51.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.555+0000 7f150a000700 1 --2- 172.21.15.1:0/2835085248 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f150410d7d0 0x7f150410fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:51.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.555+0000 7f150a000700 1 --2- 172.21.15.1:0/2835085248 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1504100760 0x7f1504100b40 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:51.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.555+0000 7f150a000700 1 --2- 172.21.15.1:0/2835085248 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1504101080 0x7f150410d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:51.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.555+0000 7f150a000700 1 -- 172.21.15.1:0/2835085248 >> 172.21.15.1:0/2835085248 conn(0x7f1504076d40 msgr2=0x7f1504077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:51.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.555+0000 7f150a000700 1 -- 172.21.15.1:0/2835085248 shutdown_connections 2024-02-02T15:25:51.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.555+0000 7f150a000700 1 -- 172.21.15.1:0/2835085248 wait complete. 2024-02-02T15:25:51.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.556+0000 7f150a000700 1 Processor -- start 2024-02-02T15:25:51.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.556+0000 7f150a000700 1 -- start start 2024-02-02T15:25:51.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.557+0000 7f150a000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1504100760 0x7f15041990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:51.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.557+0000 7f150a000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1504101080 0x7f1504199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:51.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.557+0000 7f150a000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f150410d7d0 0x7f150419da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:51.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.557+0000 7f150a000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f150419e090 con 0x7f1504100760 2024-02-02T15:25:51.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.557+0000 7f150a000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f150419e200 con 0x7f150410d7d0 2024-02-02T15:25:51.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.557+0000 7f150a000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f150419e3c0 con 0x7f1504101080 2024-02-02T15:25:51.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.557+0000 7f1503400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1504100760 0x7f15041990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:51.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.557+0000 7f1503400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1504100760 0x7f15041990d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51956/0 (socket says 172.21.15.1:51956) 2024-02-02T15:25:51.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.557+0000 7f1503400700 1 -- 172.21.15.1:0/4243832332 learned_addr learned my addr 172.21.15.1:0/4243832332 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:25:51.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.557+0000 7f1502a00700 1 --2- 172.21.15.1:0/4243832332 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1504101080 0x7f1504199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:51.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.557+0000 7f1503e00700 1 --2- 172.21.15.1:0/4243832332 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f150410d7d0 0x7f150419da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:51.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.558+0000 7f1503400700 1 -- 172.21.15.1:0/4243832332 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1504101080 msgr2=0x7f1504199610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:51.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.558+0000 7f1503400700 1 --2- 172.21.15.1:0/4243832332 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1504101080 0x7f1504199610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:51.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.558+0000 7f1503400700 1 -- 172.21.15.1:0/4243832332 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f150410d7d0 msgr2=0x7f150419da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:51.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.558+0000 7f1503400700 1 --2- 172.21.15.1:0/4243832332 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f150410d7d0 0x7f150419da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:51.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.558+0000 7f1503400700 1 -- 172.21.15.1:0/4243832332 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14f40095f0 con 0x7f1504100760 2024-02-02T15:25:51.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.558+0000 7f1503400700 1 --2- 172.21.15.1:0/4243832332 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1504100760 0x7f15041990d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f14f4000c00 tx=0x7f14f4005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:51.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.558+0000 7f14fbe00700 1 -- 172.21.15.1:0/4243832332 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f401d070 con 0x7f1504100760 2024-02-02T15:25:51.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.558+0000 7f150a000700 1 -- 172.21.15.1:0/4243832332 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f150419e640 con 0x7f1504100760 2024-02-02T15:25:51.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.558+0000 7f14fbe00700 1 -- 172.21.15.1:0/4243832332 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f14f4022470 con 0x7f1504100760 2024-02-02T15:25:51.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.558+0000 7f14fbe00700 1 -- 172.21.15.1:0/4243832332 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f4017650 con 0x7f1504100760 2024-02-02T15:25:51.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.559+0000 7f150a000700 1 -- 172.21.15.1:0/4243832332 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15041a5f00 con 0x7f1504100760 2024-02-02T15:25:51.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.560+0000 7f150a000700 1 -- 172.21.15.1:0/4243832332 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f150404ea40 con 0x7f1504100760 2024-02-02T15:25:51.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.563+0000 7f14fbe00700 1 -- 172.21.15.1:0/4243832332 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) v1 ==== 135120+0+0 (secure 0 0 0) 0x7f14f40225e0 con 0x7f1504100760 2024-02-02T15:25:51.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.565+0000 7f14fbe00700 1 --2- 172.21.15.1:0/4243832332 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f14e40a05b0 0x7f14e40a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:51.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.565+0000 7f14fbe00700 1 -- 172.21.15.1:0/4243832332 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f14f40cd960 con 0x7f1504100760 2024-02-02T15:25:51.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.565+0000 7f1502a00700 1 --2- 172.21.15.1:0/4243832332 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f14e40a05b0 0x7f14e40a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:51.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.566+0000 7f1502a00700 1 --2- 172.21.15.1:0/4243832332 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f14e40a05b0 0x7f14e40a2a70 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f150419a6f0 tx=0x7f14ec009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:51.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.566+0000 7f14fbe00700 1 -- 172.21.15.1:0/4243832332 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f14f40970c0 con 0x7f1504100760 2024-02-02T15:25:51.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:51 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/60122757' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-02T15:25:51.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.791+0000 7f150a000700 1 -- 172.21.15.1:0/4243832332 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f15041a6090 con 0x7f1504100760 2024-02-02T15:25:51.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.792+0000 7f14fbe00700 1 -- 172.21.15.1:0/4243832332 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f14f409a6e0 con 0x7f1504100760 2024-02-02T15:25:51.792 INFO:teuthology.orchestra.run.smithi001.stdout:55834574907 2024-02-02T15:25:51.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.795+0000 7f150a000700 1 -- 172.21.15.1:0/4243832332 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f14e40a05b0 msgr2=0x7f14e40a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:51.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.795+0000 7f150a000700 1 --2- 172.21.15.1:0/4243832332 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f14e40a05b0 0x7f14e40a2a70 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f150419a6f0 tx=0x7f14ec009260 comp rx=0 tx=0).stop 2024-02-02T15:25:51.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.795+0000 7f150a000700 1 -- 172.21.15.1:0/4243832332 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1504100760 msgr2=0x7f15041990d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:51.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.795+0000 7f150a000700 1 --2- 172.21.15.1:0/4243832332 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1504100760 0x7f15041990d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f14f4000c00 tx=0x7f14f4005740 comp rx=0 tx=0).stop 2024-02-02T15:25:51.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.795+0000 7f150a000700 1 -- 172.21.15.1:0/4243832332 shutdown_connections 2024-02-02T15:25:51.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.795+0000 7f150a000700 1 --2- 172.21.15.1:0/4243832332 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f150410d7d0 0x7f150419da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:51.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.795+0000 7f150a000700 1 --2- 172.21.15.1:0/4243832332 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1504100760 0x7f15041990d0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:51.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.795+0000 7f150a000700 1 --2- 172.21.15.1:0/4243832332 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f14e40a05b0 0x7f14e40a2a70 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:51.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.795+0000 7f150a000700 1 --2- 172.21.15.1:0/4243832332 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1504101080 0x7f1504199610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:51.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.795+0000 7f150a000700 1 -- 172.21.15.1:0/4243832332 >> 172.21.15.1:0/4243832332 conn(0x7f1504076d40 msgr2=0x7f150410ec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:51.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.795+0000 7f150a000700 1 -- 172.21.15.1:0/4243832332 shutdown_connections 2024-02-02T15:25:51.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.795+0000 7f150a000700 1 -- 172.21.15.1:0/4243832332 wait complete. 2024-02-02T15:25:51.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:51 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/60122757' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-02T15:25:51.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:51 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/60122757' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-02T15:25:52.076 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.a/config 2024-02-02T15:25:52.399 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.a/config 2024-02-02T15:25:52.427 INFO:tasks.cephadm.ceph_manager.ceph:need seq 103079215152 got 103079215155 for osd.3 2024-02-02T15:25:52.427 DEBUG:teuthology.parallel:result is None 2024-02-02T15:25:52.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.947+0000 7fad43e00700 1 -- 172.21.15.1:0/3586548951 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad3c073170 msgr2=0x7fad3c073550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:52.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.947+0000 7fad43e00700 1 --2- 172.21.15.1:0/3586548951 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad3c073170 0x7fad3c073550 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fad2c009990 tx=0x7fad2c009ca0 comp rx=0 tx=0).stop 2024-02-02T15:25:52.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.948+0000 7fad43e00700 1 -- 172.21.15.1:0/3586548951 shutdown_connections 2024-02-02T15:25:52.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.948+0000 7fad43e00700 1 --2- 172.21.15.1:0/3586548951 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad3c10d590 0x7fad3c10fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:52.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.948+0000 7fad43e00700 1 --2- 172.21.15.1:0/3586548951 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad3c073170 0x7fad3c073550 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:52.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.948+0000 7fad43e00700 1 --2- 172.21.15.1:0/3586548951 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad3c073a90 0x7fad3c10d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:52.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.948+0000 7fad43e00700 1 -- 172.21.15.1:0/3586548951 >> 172.21.15.1:0/3586548951 conn(0x7fad3c0fc7e0 msgr2=0x7fad3c0fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:52.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.948+0000 7fad43e00700 1 -- 172.21.15.1:0/3586548951 shutdown_connections 2024-02-02T15:25:52.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.948+0000 7fad43e00700 1 -- 172.21.15.1:0/3586548951 wait complete. 2024-02-02T15:25:52.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.949+0000 7fad43e00700 1 Processor -- start 2024-02-02T15:25:52.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.949+0000 7fad43e00700 1 -- start start 2024-02-02T15:25:52.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.949+0000 7fad43e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad3c073170 0x7fad3c199080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:52.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.949+0000 7fad43e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad3c073a90 0x7fad3c1995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:52.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.949+0000 7fad43e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad3c10d590 0x7fad3c19da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:52.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.949+0000 7fad43e00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad3c19e040 con 0x7fad3c073a90 2024-02-02T15:25:52.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.949+0000 7fad43e00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad3c19e1b0 con 0x7fad3c073170 2024-02-02T15:25:52.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.949+0000 7fad43e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad3c19e370 con 0x7fad3c10d590 2024-02-02T15:25:52.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.950+0000 7fad40a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad3c073a90 0x7fad3c1995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:52.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.950+0000 7fad40a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad3c073a90 0x7fad3c1995c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51982/0 (socket says 172.21.15.1:51982) 2024-02-02T15:25:52.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.950+0000 7fad40a00700 1 -- 172.21.15.1:0/160152089 learned_addr learned my addr 172.21.15.1:0/160152089 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:25:52.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.950+0000 7fad41400700 1 --2- 172.21.15.1:0/160152089 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad3c073170 0x7fad3c199080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:52.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.950+0000 7fad41e00700 1 --2- 172.21.15.1:0/160152089 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad3c10d590 0x7fad3c19da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:52.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.950+0000 7fad41e00700 1 -- 172.21.15.1:0/160152089 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad3c073170 msgr2=0x7fad3c199080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:52.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.950+0000 7fad41e00700 1 --2- 172.21.15.1:0/160152089 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad3c073170 0x7fad3c199080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:52.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.950+0000 7fad41e00700 1 -- 172.21.15.1:0/160152089 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad3c073a90 msgr2=0x7fad3c1995c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:52.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.950+0000 7fad41e00700 1 --2- 172.21.15.1:0/160152089 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad3c073a90 0x7fad3c1995c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:52.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.950+0000 7fad41e00700 1 -- 172.21.15.1:0/160152089 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad2c0095f0 con 0x7fad3c10d590 2024-02-02T15:25:52.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.950+0000 7fad40a00700 1 --2- 172.21.15.1:0/160152089 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad3c073a90 0x7fad3c1995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:25:52.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.951+0000 7fad41e00700 1 --2- 172.21.15.1:0/160152089 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad3c10d590 0x7fad3c19da30 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fad3400efc0 tx=0x7fad3400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:52.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.951+0000 7fad3a000700 1 -- 172.21.15.1:0/160152089 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad34018070 con 0x7fad3c10d590 2024-02-02T15:25:52.579 INFO:teuthology.orchestra.run.smithi001.stdout:206158430245 2024-02-02T15:25:52.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.951+0000 7fad3a000700 1 -- 172.21.15.1:0/160152089 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fad3400f040 con 0x7fad3c10d590 2024-02-02T15:25:52.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.951+0000 7fad3a000700 1 -- 172.21.15.1:0/160152089 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad3400cd40 con 0x7fad3c10d590 2024-02-02T15:25:52.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.951+0000 7fad43e00700 1 -- 172.21.15.1:0/160152089 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad3c19e5f0 con 0x7fad3c10d590 2024-02-02T15:25:52.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.951+0000 7fad43e00700 1 -- 172.21.15.1:0/160152089 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad3c1a5eb0 con 0x7fad3c10d590 2024-02-02T15:25:52.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.952+0000 7fad43e00700 1 -- 172.21.15.1:0/160152089 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad3c04ea40 con 0x7fad3c10d590 2024-02-02T15:25:52.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.954+0000 7fad3a000700 1 -- 172.21.15.1:0/160152089 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 17) v1 ==== 135120+0+0 (secure 0 0 0) 0x7fad34003a00 con 0x7fad3c10d590 2024-02-02T15:25:52.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.956+0000 7fad3a000700 1 --2- 172.21.15.1:0/160152089 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fad1c0a0630 0x7fad1c0a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:52.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.956+0000 7fad3a000700 1 -- 172.21.15.1:0/160152089 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fad340cc2b0 con 0x7fad3c10d590 2024-02-02T15:25:52.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.956+0000 7fad41400700 1 --2- 172.21.15.1:0/160152089 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fad1c0a0630 0x7fad1c0a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:52.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.957+0000 7fad41400700 1 --2- 172.21.15.1:0/160152089 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fad1c0a0630 0x7fad1c0a2af0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fad2c00b2e0 tx=0x7fad2c005fd0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:52.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:51.958+0000 7fad3a000700 1 -- 172.21.15.1:0/160152089 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7fad34095a10 con 0x7fad3c10d590 2024-02-02T15:25:52.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:52.188+0000 7fad43e00700 1 -- 172.21.15.1:0/160152089 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fad3c19a3e0 con 0x7fad3c10d590 2024-02-02T15:25:52.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:52.189+0000 7fad3a000700 1 -- 172.21.15.1:0/160152089 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fad34099030 con 0x7fad3c10d590 2024-02-02T15:25:52.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:52.191+0000 7fad43e00700 1 -- 172.21.15.1:0/160152089 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fad1c0a0630 msgr2=0x7fad1c0a2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:52.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:52.191+0000 7fad43e00700 1 --2- 172.21.15.1:0/160152089 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fad1c0a0630 0x7fad1c0a2af0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fad2c00b2e0 tx=0x7fad2c005fd0 comp rx=0 tx=0).stop 2024-02-02T15:25:52.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:52.191+0000 7fad43e00700 1 -- 172.21.15.1:0/160152089 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad3c10d590 msgr2=0x7fad3c19da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:52.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:52.191+0000 7fad43e00700 1 --2- 172.21.15.1:0/160152089 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad3c10d590 0x7fad3c19da30 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fad3400efc0 tx=0x7fad3400c3a0 comp rx=0 tx=0).stop 2024-02-02T15:25:52.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:52.191+0000 7fad43e00700 1 -- 172.21.15.1:0/160152089 shutdown_connections 2024-02-02T15:25:52.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:52.191+0000 7fad43e00700 1 --2- 172.21.15.1:0/160152089 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fad1c0a0630 0x7fad1c0a2af0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:52.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:52.191+0000 7fad43e00700 1 --2- 172.21.15.1:0/160152089 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fad3c10d590 0x7fad3c19da30 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:52.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:52.191+0000 7fad43e00700 1 --2- 172.21.15.1:0/160152089 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fad3c073170 0x7fad3c199080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:52.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:52.191+0000 7fad43e00700 1 --2- 172.21.15.1:0/160152089 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fad3c073a90 0x7fad3c1995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:52.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:52.191+0000 7fad43e00700 1 -- 172.21.15.1:0/160152089 >> 172.21.15.1:0/160152089 conn(0x7fad3c0fc7e0 msgr2=0x7fad3c10f750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:52.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:52.191+0000 7fad43e00700 1 -- 172.21.15.1:0/160152089 shutdown_connections 2024-02-02T15:25:52.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:52.191+0000 7fad43e00700 1 -- 172.21.15.1:0/160152089 wait complete. 2024-02-02T15:25:52.660 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:52 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/4243832332' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-02T15:25:52.660 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:52 smithi001 ceph-mon[35470]: pgmap v269: 129 pgs: 129 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 597 B/s wr, 11 op/s 2024-02-02T15:25:52.660 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:52 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/160152089' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-02-02T15:25:52.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:52 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/4243832332' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-02T15:25:52.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:52 smithi071 ceph-mon[44707]: pgmap v269: 129 pgs: 129 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 597 B/s wr, 11 op/s 2024-02-02T15:25:52.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:52 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/160152089' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-02-02T15:25:52.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:52 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/4243832332' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-02T15:25:52.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:52 smithi142 ceph-mon[45728]: pgmap v269: 129 pgs: 129 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 597 B/s wr, 11 op/s 2024-02-02T15:25:52.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:52 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/160152089' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-02-02T15:25:53.038 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574904 got 55834574907 for osd.1 2024-02-02T15:25:53.038 DEBUG:teuthology.parallel:result is None 2024-02-02T15:25:53.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:52.528+0000 7f7ed5400700 1 -- 172.21.15.1:0/3213887843 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ed0068860 msgr2=0x7f7ed0068ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:53.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:52.528+0000 7f7ed5400700 1 --2- 172.21.15.1:0/3213887843 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ed0068860 0x7f7ed0068ce0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7ec800b0a0 tx=0x7f7ec800b3b0 comp rx=0 tx=0).stop 2024-02-02T15:25:53.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:52.528+0000 7f7ed5400700 1 -- 172.21.15.1:0/3213887843 shutdown_connections 2024-02-02T15:25:53.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:52.528+0000 7f7ed5400700 1 --2- 172.21.15.1:0/3213887843 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ed0068860 0x7f7ed0068ce0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:53.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:52.528+0000 7f7ed5400700 1 --2- 172.21.15.1:0/3213887843 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ed0105d00 0x7f7ed0103790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:53.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:52.528+0000 7f7ed5400700 1 --2- 172.21.15.1:0/3213887843 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ed0103cd0 0x7f7ed0104150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:53.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:52.528+0000 7f7ed5400700 1 -- 172.21.15.1:0/3213887843 >> 172.21.15.1:0/3213887843 conn(0x7f7ed0076d40 msgr2=0x7f7ed0077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:53.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:52.529+0000 7f7ed5400700 1 -- 172.21.15.1:0/3213887843 shutdown_connections 2024-02-02T15:25:53.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:52.529+0000 7f7ed5400700 1 -- 172.21.15.1:0/3213887843 wait complete. 2024-02-02T15:25:53.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:52.529+0000 7f7ed5400700 1 Processor -- start 2024-02-02T15:25:53.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:52.530+0000 7f7ed5400700 1 -- start start 2024-02-02T15:25:53.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:52.530+0000 7f7ed5400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ed0068860 0x7f7ed019d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:53.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:52.530+0000 7f7ed5400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ed0103cd0 0x7f7ed019db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:53.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:52.530+0000 7f7ed5400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ed0105d00 0x7f7ed01a1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:53.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:52.530+0000 7f7ed5400700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ed01a2600 con 0x7f7ed0068860 2024-02-02T15:25:53.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:52.530+0000 7f7ed5400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ed01a2770 con 0x7f7ed0103cd0 2024-02-02T15:25:53.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:52.530+0000 7f7ed5400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ed01a2930 con 0x7f7ed0105d00 2024-02-02T15:25:53.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:52.530+0000 7f7ece800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ed0068860 0x7f7ed019d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:53.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:52.530+0000 7f7ecf200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ed0105d00 0x7f7ed01a1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:53.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:52.530+0000 7f7ecf200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ed0105d00 0x7f7ed01a1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:43290/0 (socket says 172.21.15.1:43290) 2024-02-02T15:25:53.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:52.530+0000 7f7ecf200700 1 -- 172.21.15.1:0/1022539109 learned_addr learned my addr 172.21.15.1:0/1022539109 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:25:53.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:52.530+0000 7f7ecde00700 1 --2- 172.21.15.1:0/1022539109 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ed0103cd0 0x7f7ed019db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:53.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:52.531+0000 7f7ecf200700 1 -- 172.21.15.1:0/1022539109 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ed0103cd0 msgr2=0x7f7ed019db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:53.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:52.531+0000 7f7ecf200700 1 --2- 172.21.15.1:0/1022539109 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ed0103cd0 0x7f7ed019db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:53.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:52.531+0000 7f7ecf200700 1 -- 172.21.15.1:0/1022539109 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ed0068860 msgr2=0x7f7ed019d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:53.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:52.531+0000 7f7ecf200700 1 --2- 172.21.15.1:0/1022539109 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ed0068860 0x7f7ed019d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:53.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:52.531+0000 7f7ecf200700 1 -- 172.21.15.1:0/1022539109 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ec8009d00 con 0x7f7ed0105d00 2024-02-02T15:25:53.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:52.531+0000 7f7ecf200700 1 --2- 172.21.15.1:0/1022539109 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ed0105d00 0x7f7ed01a1ff0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7ec8009fd0 tx=0x7f7ec80094c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:53.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:52.531+0000 7f7ec7400700 1 -- 172.21.15.1:0/1022539109 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ec800e040 con 0x7f7ed0105d00 2024-02-02T15:25:53.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:52.531+0000 7f7ed5400700 1 -- 172.21.15.1:0/1022539109 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ed01a2bb0 con 0x7f7ed0105d00 2024-02-02T15:25:53.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:52.532+0000 7f7ec7400700 1 -- 172.21.15.1:0/1022539109 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ec801d470 con 0x7f7ed0105d00 2024-02-02T15:25:53.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:52.532+0000 7f7ec7400700 1 -- 172.21.15.1:0/1022539109 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ec80052a0 con 0x7f7ed0105d00 2024-02-02T15:25:53.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:52.532+0000 7f7ed5400700 1 -- 172.21.15.1:0/1022539109 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ed01aa470 con 0x7f7ed0105d00 2024-02-02T15:25:53.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:52.533+0000 7f7ed5400700 1 -- 172.21.15.1:0/1022539109 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ed004ea40 con 0x7f7ed0105d00 2024-02-02T15:25:53.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:52.535+0000 7f7ec7400700 1 -- 172.21.15.1:0/1022539109 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 17) v1 ==== 135120+0+0 (secure 0 0 0) 0x7f7ec801d5e0 con 0x7f7ed0105d00 2024-02-02T15:25:53.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:52.536+0000 7f7ec7400700 1 --2- 172.21.15.1:0/1022539109 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f7eb00a05b0 0x7f7eb00a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:53.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:52.536+0000 7f7ec7400700 1 -- 172.21.15.1:0/1022539109 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f7ec80cd6b0 con 0x7f7ed0105d00 2024-02-02T15:25:53.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:52.536+0000 7f7ece800700 1 --2- 172.21.15.1:0/1022539109 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f7eb00a05b0 0x7f7eb00a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:53.171 INFO:teuthology.orchestra.run.smithi001.stdout:133143986226 2024-02-02T15:25:53.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:52.537+0000 7f7ece800700 1 --2- 172.21.15.1:0/1022539109 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f7eb00a05b0 0x7f7eb00a2a70 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0009be0 tx=0x7f7ec0009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:53.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:52.538+0000 7f7ec7400700 1 -- 172.21.15.1:0/1022539109 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f7ec8096e10 con 0x7f7ed0105d00 2024-02-02T15:25:53.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:52.766+0000 7f7ed5400700 1 -- 172.21.15.1:0/1022539109 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f7ed01aa600 con 0x7f7ed0105d00 2024-02-02T15:25:53.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:52.767+0000 7f7ec7400700 1 -- 172.21.15.1:0/1022539109 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7ec809a430 con 0x7f7ed0105d00 2024-02-02T15:25:53.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:52.769+0000 7f7ed5400700 1 -- 172.21.15.1:0/1022539109 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f7eb00a05b0 msgr2=0x7f7eb00a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:53.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:52.769+0000 7f7ed5400700 1 --2- 172.21.15.1:0/1022539109 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f7eb00a05b0 0x7f7eb00a2a70 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7ec0009be0 tx=0x7f7ec0009260 comp rx=0 tx=0).stop 2024-02-02T15:25:53.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:52.769+0000 7f7ed5400700 1 -- 172.21.15.1:0/1022539109 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ed0105d00 msgr2=0x7f7ed01a1ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:53.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:52.769+0000 7f7ed5400700 1 --2- 172.21.15.1:0/1022539109 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ed0105d00 0x7f7ed01a1ff0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7ec8009fd0 tx=0x7f7ec80094c0 comp rx=0 tx=0).stop 2024-02-02T15:25:53.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:52.769+0000 7f7ed5400700 1 -- 172.21.15.1:0/1022539109 shutdown_connections 2024-02-02T15:25:53.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:52.769+0000 7f7ed5400700 1 --2- 172.21.15.1:0/1022539109 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f7eb00a05b0 0x7f7eb00a2a70 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:53.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:52.769+0000 7f7ed5400700 1 --2- 172.21.15.1:0/1022539109 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7ed0105d00 0x7f7ed01a1ff0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:53.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:52.769+0000 7f7ed5400700 1 --2- 172.21.15.1:0/1022539109 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7ed0068860 0x7f7ed019d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:53.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:52.769+0000 7f7ed5400700 1 --2- 172.21.15.1:0/1022539109 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7ed0103cd0 0x7f7ed019db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:53.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:52.769+0000 7f7ed5400700 1 -- 172.21.15.1:0/1022539109 >> 172.21.15.1:0/1022539109 conn(0x7f7ed0076d40 msgr2=0x7f7ed00fda90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:53.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:52.769+0000 7f7ed5400700 1 -- 172.21.15.1:0/1022539109 shutdown_connections 2024-02-02T15:25:53.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:52.769+0000 7f7ed5400700 1 -- 172.21.15.1:0/1022539109 wait complete. 2024-02-02T15:25:53.174 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.a/config 2024-02-02T15:25:53.372 INFO:tasks.cephadm.ceph_manager.ceph:need seq 154618822699 got 154618822700 for osd.5 2024-02-02T15:25:53.373 DEBUG:teuthology.parallel:result is None 2024-02-02T15:25:53.501 INFO:tasks.cephadm.ceph_manager.ceph:need seq 206158430243 got 206158430245 for osd.7 2024-02-02T15:25:53.501 DEBUG:teuthology.parallel:result is None 2024-02-02T15:25:53.598 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:53 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/1022539109' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-02T15:25:53.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:53 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/1022539109' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-02T15:25:53.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:53 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/1022539109' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-02T15:25:54.597 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986224 got 133143986226 for osd.4 2024-02-02T15:25:54.597 DEBUG:teuthology.parallel:result is None 2024-02-02T15:25:54.758 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.a/config 2024-02-02T15:25:54.852 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:54 smithi001 ceph-mon[35470]: pgmap v270: 129 pgs: 129 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-02-02T15:25:54.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:54 smithi071 ceph-mon[44707]: pgmap v270: 129 pgs: 129 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-02-02T15:25:54.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:54.865+0000 7f6773000700 1 -- 172.21.15.1:0/2565357087 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f676c106ed0 msgr2=0x7f676c1072b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:54.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:54.865+0000 7f6773000700 1 --2- 172.21.15.1:0/2565357087 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f676c106ed0 0x7f676c1072b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f675c009990 tx=0x7f675c009ca0 comp rx=0 tx=0).stop 2024-02-02T15:25:54.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:54.865+0000 7f6773000700 1 -- 172.21.15.1:0/2565357087 shutdown_connections 2024-02-02T15:25:54.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:54.865+0000 7f6773000700 1 --2- 172.21.15.1:0/2565357087 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f676c069220 0x7f676c10db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:54.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:54.865+0000 7f6773000700 1 --2- 172.21.15.1:0/2565357087 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f676c106ed0 0x7f676c1072b0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:54.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:54.865+0000 7f6773000700 1 --2- 172.21.15.1:0/2565357087 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f676c068860 0x7f676c068ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:54.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:54.865+0000 7f6773000700 1 -- 172.21.15.1:0/2565357087 >> 172.21.15.1:0/2565357087 conn(0x7f676c076d40 msgr2=0x7f676c077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:54.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:54.865+0000 7f6773000700 1 -- 172.21.15.1:0/2565357087 shutdown_connections 2024-02-02T15:25:54.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:54.865+0000 7f6773000700 1 -- 172.21.15.1:0/2565357087 wait complete. 2024-02-02T15:25:54.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:54.866+0000 7f6773000700 1 Processor -- start 2024-02-02T15:25:54.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:54.866+0000 7f6773000700 1 -- start start 2024-02-02T15:25:54.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:54.867+0000 7f6773000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f676c068860 0x7f676c06b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:54.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:54.867+0000 7f6773000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f676c069220 0x7f676c06b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:54.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:54.867+0000 7f6773000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f676c106ed0 0x7f676c1ab330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:54.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:54.867+0000 7f6773000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f676c1ab900 con 0x7f676c069220 2024-02-02T15:25:54.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:54.867+0000 7f6773000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f676c1aba40 con 0x7f676c068860 2024-02-02T15:25:54.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:54.867+0000 7f6773000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f676c1abc00 con 0x7f676c106ed0 2024-02-02T15:25:54.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:54.867+0000 7f676be00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f676c068860 0x7f676c06b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:54.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:54.867+0000 7f676be00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f676c068860 0x7f676c06b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59162/0 (socket says 172.21.15.1:59162) 2024-02-02T15:25:54.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:54.867+0000 7f676be00700 1 -- 172.21.15.1:0/4152947887 learned_addr learned my addr 172.21.15.1:0/4152947887 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:25:54.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:54.867+0000 7f6771000700 1 --2- 172.21.15.1:0/4152947887 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f676c106ed0 0x7f676c1ab330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:54.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:54.867+0000 7f676be00700 1 -- 172.21.15.1:0/4152947887 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f676c106ed0 msgr2=0x7f676c1ab330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:54.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:54.867+0000 7f676be00700 1 --2- 172.21.15.1:0/4152947887 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f676c106ed0 0x7f676c1ab330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:54.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:54.867+0000 7f676be00700 1 -- 172.21.15.1:0/4152947887 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f676c069220 msgr2=0x7f676c06b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:54.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:54.867+0000 7f676be00700 1 --2- 172.21.15.1:0/4152947887 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f676c069220 0x7f676c06b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:54.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:54.867+0000 7f676be00700 1 -- 172.21.15.1:0/4152947887 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f675c0095f0 con 0x7f676c068860 2024-02-02T15:25:54.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:54.868+0000 7f676be00700 1 --2- 172.21.15.1:0/4152947887 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f676c068860 0x7f676c06b250 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f675c005910 tx=0x7f675c005720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:54.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:54.868+0000 7f6768c00700 1 -- 172.21.15.1:0/4152947887 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f675c01d070 con 0x7f676c068860 2024-02-02T15:25:54.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:54.868+0000 7f6773000700 1 -- 172.21.15.1:0/4152947887 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f676c1abe80 con 0x7f676c068860 2024-02-02T15:25:54.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:54.868+0000 7f6768c00700 1 -- 172.21.15.1:0/4152947887 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f675c022470 con 0x7f676c068860 2024-02-02T15:25:54.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:54.868+0000 7f6768c00700 1 -- 172.21.15.1:0/4152947887 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f675c017650 con 0x7f676c068860 2024-02-02T15:25:54.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:54.868+0000 7f6773000700 1 -- 172.21.15.1:0/4152947887 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f676c1ac290 con 0x7f676c068860 2024-02-02T15:25:54.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:54.869+0000 7f6773000700 1 -- 172.21.15.1:0/4152947887 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f676c04ea40 con 0x7f676c068860 2024-02-02T15:25:54.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:54.872+0000 7f6768c00700 1 -- 172.21.15.1:0/4152947887 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 135120+0+0 (secure 0 0 0) 0x7f675c054080 con 0x7f676c068860 2024-02-02T15:25:54.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:54.873+0000 7f6768c00700 1 --2- 172.21.15.1:0/4152947887 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f674c0a0630 0x7f674c0a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:54.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:54.874+0000 7f6768c00700 1 -- 172.21.15.1:0/4152947887 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f675c052070 con 0x7f676c068860 2024-02-02T15:25:54.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:54.874+0000 7f676b400700 1 --2- 172.21.15.1:0/4152947887 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f674c0a0630 0x7f674c0a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:54.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:54.875+0000 7f676b400700 1 --2- 172.21.15.1:0/4152947887 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f674c0a0630 0x7f674c0a2af0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6754005950 tx=0x7f67540058e0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:54.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:54.875+0000 7f6768c00700 1 -- 172.21.15.1:0/4152947887 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f675c096cb0 con 0x7f676c068860 2024-02-02T15:25:54.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:54 smithi142 ceph-mon[45728]: pgmap v270: 129 pgs: 129 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-02-02T15:25:55.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.124+0000 7f6773000700 1 -- 172.21.15.1:0/4152947887 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7f676c06c5b0 con 0x7f676c068860 2024-02-02T15:25:55.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.125+0000 7f6768c00700 1 -- 172.21.15.1:0/4152947887 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f675c027090 con 0x7f676c068860 2024-02-02T15:25:55.125 INFO:teuthology.orchestra.run.smithi001.stdout:266287972383 2024-02-02T15:25:55.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.127+0000 7f6773000700 1 -- 172.21.15.1:0/4152947887 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f674c0a0630 msgr2=0x7f674c0a2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:55.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.127+0000 7f6773000700 1 --2- 172.21.15.1:0/4152947887 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f674c0a0630 0x7f674c0a2af0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6754005950 tx=0x7f67540058e0 comp rx=0 tx=0).stop 2024-02-02T15:25:55.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.127+0000 7f6773000700 1 -- 172.21.15.1:0/4152947887 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f676c068860 msgr2=0x7f676c06b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:55.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.127+0000 7f6773000700 1 --2- 172.21.15.1:0/4152947887 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f676c068860 0x7f676c06b250 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f675c005910 tx=0x7f675c005720 comp rx=0 tx=0).stop 2024-02-02T15:25:55.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.128+0000 7f6773000700 1 -- 172.21.15.1:0/4152947887 shutdown_connections 2024-02-02T15:25:55.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.128+0000 7f6773000700 1 --2- 172.21.15.1:0/4152947887 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f674c0a0630 0x7f674c0a2af0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:55.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.128+0000 7f6773000700 1 --2- 172.21.15.1:0/4152947887 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f676c106ed0 0x7f676c1ab330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:55.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.128+0000 7f6773000700 1 --2- 172.21.15.1:0/4152947887 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f676c068860 0x7f676c06b250 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:55.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.128+0000 7f6773000700 1 --2- 172.21.15.1:0/4152947887 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f676c069220 0x7f676c06b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:55.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.128+0000 7f6773000700 1 -- 172.21.15.1:0/4152947887 >> 172.21.15.1:0/4152947887 conn(0x7f676c076d40 msgr2=0x7f676c0fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:55.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.128+0000 7f6773000700 1 -- 172.21.15.1:0/4152947887 shutdown_connections 2024-02-02T15:25:55.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.128+0000 7f6773000700 1 -- 172.21.15.1:0/4152947887 wait complete. 2024-02-02T15:25:55.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.166+0000 7f2011200700 1 -- 172.21.15.1:0/3618356414 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f200c075b90 msgr2=0x7f200c111710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:55.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.166+0000 7f2011200700 1 --2- 172.21.15.1:0/3618356414 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f200c075b90 0x7f200c111710 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1ff4009990 tx=0x7f1ff4009ca0 comp rx=0 tx=0).stop 2024-02-02T15:25:55.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.166+0000 7f2011200700 1 -- 172.21.15.1:0/3618356414 shutdown_connections 2024-02-02T15:25:55.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.166+0000 7f2011200700 1 --2- 172.21.15.1:0/3618356414 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f200c111c50 0x7f200c114040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:55.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.166+0000 7f2011200700 1 --2- 172.21.15.1:0/3618356414 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f200c075270 0x7f200c075650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:55.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.166+0000 7f2011200700 1 --2- 172.21.15.1:0/3618356414 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f200c075b90 0x7f200c111710 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:55.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.166+0000 7f2011200700 1 -- 172.21.15.1:0/3618356414 >> 172.21.15.1:0/3618356414 conn(0x7f200c0fea80 msgr2=0x7f200c100ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:55.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.166+0000 7f2011200700 1 -- 172.21.15.1:0/3618356414 shutdown_connections 2024-02-02T15:25:55.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.166+0000 7f2011200700 1 -- 172.21.15.1:0/3618356414 wait complete. 2024-02-02T15:25:55.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.167+0000 7f2011200700 1 Processor -- start 2024-02-02T15:25:55.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.168+0000 7f2011200700 1 -- start start 2024-02-02T15:25:55.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.168+0000 7f2011200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f200c075270 0x7f200c19d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:55.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.168+0000 7f2011200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f200c075b90 0x7f200c19dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:55.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.168+0000 7f2011200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f200c111c50 0x7f200c1a1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:55.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.168+0000 7f2011200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f200c1a2540 con 0x7f200c075270 2024-02-02T15:25:55.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.168+0000 7f2011200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f200c1a26b0 con 0x7f200c111c50 2024-02-02T15:25:55.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.168+0000 7f2011200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f200c1a2870 con 0x7f200c075b90 2024-02-02T15:25:55.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.169+0000 7f2009e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f200c075270 0x7f200c19d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:55.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.169+0000 7f200a800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f200c111c50 0x7f200c1a1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:55.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.169+0000 7f200a800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f200c111c50 0x7f200c1a1f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59182/0 (socket says 172.21.15.1:59182) 2024-02-02T15:25:55.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.169+0000 7f2009400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f200c075b90 0x7f200c19dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:55.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.169+0000 7f200a800700 1 -- 172.21.15.1:0/2135702021 learned_addr learned my addr 172.21.15.1:0/2135702021 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:25:55.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.169+0000 7f200a800700 1 -- 172.21.15.1:0/2135702021 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f200c075b90 msgr2=0x7f200c19dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:55.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.169+0000 7f200a800700 1 --2- 172.21.15.1:0/2135702021 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f200c075b90 0x7f200c19dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:55.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.169+0000 7f200a800700 1 -- 172.21.15.1:0/2135702021 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f200c075270 msgr2=0x7f200c19d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:55.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.169+0000 7f200a800700 1 --2- 172.21.15.1:0/2135702021 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f200c075270 0x7f200c19d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:55.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.169+0000 7f200a800700 1 -- 172.21.15.1:0/2135702021 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ff40095f0 con 0x7f200c111c50 2024-02-02T15:25:55.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.169+0000 7f2009e00700 1 --2- 172.21.15.1:0/2135702021 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f200c075270 0x7f200c19d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:25:55.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.169+0000 7f2009400700 1 --2- 172.21.15.1:0/2135702021 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f200c075b90 0x7f200c19dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:25:55.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.169+0000 7f200a800700 1 --2- 172.21.15.1:0/2135702021 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f200c111c50 0x7f200c1a1f30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2004009fd0 tx=0x7f200400ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:55.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.170+0000 7f2002a00700 1 -- 172.21.15.1:0/2135702021 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2004018070 con 0x7f200c111c50 2024-02-02T15:25:55.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.170+0000 7f2011200700 1 -- 172.21.15.1:0/2135702021 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f200c1a2af0 con 0x7f200c111c50 2024-02-02T15:25:55.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.170+0000 7f2002a00700 1 -- 172.21.15.1:0/2135702021 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f200400f040 con 0x7f200c111c50 2024-02-02T15:25:55.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.170+0000 7f2002a00700 1 -- 172.21.15.1:0/2135702021 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f200400cdc0 con 0x7f200c111c50 2024-02-02T15:25:55.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.170+0000 7f2011200700 1 -- 172.21.15.1:0/2135702021 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f200c1aa3b0 con 0x7f200c111c50 2024-02-02T15:25:55.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.171+0000 7f2011200700 1 -- 172.21.15.1:0/2135702021 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f200c04ea40 con 0x7f200c111c50 2024-02-02T15:25:55.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.173+0000 7f2002a00700 1 -- 172.21.15.1:0/2135702021 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 135120+0+0 (secure 0 0 0) 0x7f2004004ad0 con 0x7f200c111c50 2024-02-02T15:25:55.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.174+0000 7f2002a00700 1 --2- 172.21.15.1:0/2135702021 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f1fec0a0630 0x7f1fec0a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:55.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.174+0000 7f2002a00700 1 -- 172.21.15.1:0/2135702021 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f20040cc240 con 0x7f200c111c50 2024-02-02T15:25:55.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.174+0000 7f2009e00700 1 --2- 172.21.15.1:0/2135702021 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f1fec0a0630 0x7f1fec0a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:55.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.175+0000 7f2009e00700 1 --2- 172.21.15.1:0/2135702021 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f1fec0a0630 0x7f1fec0a2af0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc0099b0 tx=0x7f1ffc008040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:55.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.176+0000 7f2002a00700 1 -- 172.21.15.1:0/2135702021 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f20040959a0 con 0x7f200c111c50 2024-02-02T15:25:55.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.403+0000 7f2011200700 1 -- 172.21.15.1:0/2135702021 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7f200c1aa5e0 con 0x7f200c111c50 2024-02-02T15:25:55.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.403+0000 7f2002a00700 1 -- 172.21.15.1:0/2135702021 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f2004098fc0 con 0x7f200c111c50 2024-02-02T15:25:55.403 INFO:teuthology.orchestra.run.smithi001.stdout:322122547224 2024-02-02T15:25:55.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.406+0000 7f2011200700 1 -- 172.21.15.1:0/2135702021 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f1fec0a0630 msgr2=0x7f1fec0a2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:55.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.406+0000 7f2011200700 1 --2- 172.21.15.1:0/2135702021 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f1fec0a0630 0x7f1fec0a2af0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1ffc0099b0 tx=0x7f1ffc008040 comp rx=0 tx=0).stop 2024-02-02T15:25:55.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.406+0000 7f2011200700 1 -- 172.21.15.1:0/2135702021 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f200c111c50 msgr2=0x7f200c1a1f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:55.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.406+0000 7f2011200700 1 --2- 172.21.15.1:0/2135702021 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f200c111c50 0x7f200c1a1f30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2004009fd0 tx=0x7f200400ef90 comp rx=0 tx=0).stop 2024-02-02T15:25:55.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.406+0000 7f2011200700 1 -- 172.21.15.1:0/2135702021 shutdown_connections 2024-02-02T15:25:55.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.406+0000 7f2011200700 1 --2- 172.21.15.1:0/2135702021 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f200c111c50 0x7f200c1a1f30 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:55.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.406+0000 7f2011200700 1 --2- 172.21.15.1:0/2135702021 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f200c075270 0x7f200c19d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:55.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.406+0000 7f2011200700 1 --2- 172.21.15.1:0/2135702021 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f1fec0a0630 0x7f1fec0a2af0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:55.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.406+0000 7f2011200700 1 --2- 172.21.15.1:0/2135702021 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f200c075b90 0x7f200c19dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:55.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.406+0000 7f2011200700 1 -- 172.21.15.1:0/2135702021 >> 172.21.15.1:0/2135702021 conn(0x7f200c0fea80 msgr2=0x7f200c100e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:55.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.406+0000 7f2011200700 1 -- 172.21.15.1:0/2135702021 shutdown_connections 2024-02-02T15:25:55.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.406+0000 7f2011200700 1 -- 172.21.15.1:0/2135702021 wait complete. 2024-02-02T15:25:55.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.607+0000 7f76da200700 1 -- 172.21.15.1:0/3700396521 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76d4101980 msgr2=0x7f76d41059d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:55.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.607+0000 7f76da200700 1 --2- 172.21.15.1:0/3700396521 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76d4101980 0x7f76d41059d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f76bc009990 tx=0x7f76bc009ca0 comp rx=0 tx=0).stop 2024-02-02T15:25:55.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.607+0000 7f76da200700 1 -- 172.21.15.1:0/3700396521 shutdown_connections 2024-02-02T15:25:55.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.607+0000 7f76da200700 1 --2- 172.21.15.1:0/3700396521 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f76d41060b0 0x7f76d410dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:55.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.607+0000 7f76da200700 1 --2- 172.21.15.1:0/3700396521 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76d4100fd0 0x7f76d41013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:55.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.607+0000 7f76da200700 1 --2- 172.21.15.1:0/3700396521 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76d4101980 0x7f76d41059d0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:55.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.607+0000 7f76da200700 1 -- 172.21.15.1:0/3700396521 >> 172.21.15.1:0/3700396521 conn(0x7f76d40fc860 msgr2=0x7f76d40fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:55.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.607+0000 7f76da200700 1 -- 172.21.15.1:0/3700396521 shutdown_connections 2024-02-02T15:25:55.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.608+0000 7f76da200700 1 -- 172.21.15.1:0/3700396521 wait complete. 2024-02-02T15:25:55.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.609+0000 7f76da200700 1 Processor -- start 2024-02-02T15:25:55.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.609+0000 7f76da200700 1 -- start start 2024-02-02T15:25:55.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.609+0000 7f76da200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76d4100fd0 0x7f76d4199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:55.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.609+0000 7f76da200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f76d4101980 0x7f76d41996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:55.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.609+0000 7f76da200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76d41060b0 0x7f76d419da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:55.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.609+0000 7f76da200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76d419e030 con 0x7f76d4100fd0 2024-02-02T15:25:55.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.609+0000 7f76da200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76d419e1a0 con 0x7f76d41060b0 2024-02-02T15:25:55.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.609+0000 7f76da200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76d419e360 con 0x7f76d4101980 2024-02-02T15:25:55.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.609+0000 7f76d3400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76d4100fd0 0x7f76d4199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:55.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.609+0000 7f76d3400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76d4100fd0 0x7f76d4199160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:32772/0 (socket says 172.21.15.1:32772) 2024-02-02T15:25:55.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.609+0000 7f76d3400700 1 -- 172.21.15.1:0/1494987278 learned_addr learned my addr 172.21.15.1:0/1494987278 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:25:55.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.610+0000 7f76d3e00700 1 --2- 172.21.15.1:0/1494987278 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76d41060b0 0x7f76d419da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:55.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.610+0000 7f76d2a00700 1 --2- 172.21.15.1:0/1494987278 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f76d4101980 0x7f76d41996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:55.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.610+0000 7f76d3400700 1 -- 172.21.15.1:0/1494987278 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f76d4101980 msgr2=0x7f76d41996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:55.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.610+0000 7f76d3400700 1 --2- 172.21.15.1:0/1494987278 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f76d4101980 0x7f76d41996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:55.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.610+0000 7f76d3400700 1 -- 172.21.15.1:0/1494987278 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76d41060b0 msgr2=0x7f76d419da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:55.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.610+0000 7f76d3400700 1 --2- 172.21.15.1:0/1494987278 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76d41060b0 0x7f76d419da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:55.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.610+0000 7f76d3400700 1 -- 172.21.15.1:0/1494987278 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76bc0095f0 con 0x7f76d4100fd0 2024-02-02T15:25:55.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.610+0000 7f76d3e00700 1 --2- 172.21.15.1:0/1494987278 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76d41060b0 0x7f76d419da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:25:55.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.610+0000 7f76d3400700 1 --2- 172.21.15.1:0/1494987278 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76d4100fd0 0x7f76d4199160 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f76c400e9a0 tx=0x7f76c400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:55.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.611+0000 7f76cbe00700 1 -- 172.21.15.1:0/1494987278 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c4009790 con 0x7f76d4100fd0 2024-02-02T15:25:55.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.611+0000 7f76cbe00700 1 -- 172.21.15.1:0/1494987278 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f76c4010430 con 0x7f76d4100fd0 2024-02-02T15:25:55.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.611+0000 7f76da200700 1 -- 172.21.15.1:0/1494987278 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76d419e640 con 0x7f76d4100fd0 2024-02-02T15:25:55.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.611+0000 7f76cbe00700 1 -- 172.21.15.1:0/1494987278 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c4018680 con 0x7f76d4100fd0 2024-02-02T15:25:55.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.611+0000 7f76da200700 1 -- 172.21.15.1:0/1494987278 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76d41a5ea0 con 0x7f76d4100fd0 2024-02-02T15:25:55.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.612+0000 7f76da200700 1 -- 172.21.15.1:0/1494987278 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76d404ea40 con 0x7f76d4100fd0 2024-02-02T15:25:55.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.614+0000 7f76cbe00700 1 -- 172.21.15.1:0/1494987278 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) v1 ==== 135120+0+0 (secure 0 0 0) 0x7f76c40105a0 con 0x7f76d4100fd0 2024-02-02T15:25:55.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.615+0000 7f76cbe00700 1 --2- 172.21.15.1:0/1494987278 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f76b40a05b0 0x7f76b40a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:55.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.615+0000 7f76cbe00700 1 -- 172.21.15.1:0/1494987278 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f76c4014070 con 0x7f76d4100fd0 2024-02-02T15:25:55.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.615+0000 7f76d2a00700 1 --2- 172.21.15.1:0/1494987278 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f76b40a05b0 0x7f76b40a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:55.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.616+0000 7f76d2a00700 1 --2- 172.21.15.1:0/1494987278 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f76b40a05b0 0x7f76b40a2a70 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f76d419a730 tx=0x7f76bc005fd0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:55.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.619+0000 7f76cbe00700 1 -- 172.21.15.1:0/1494987278 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f76c4052110 con 0x7f76d4100fd0 2024-02-02T15:25:55.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:55 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/4152947887' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-02-02T15:25:55.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:55 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/2135702021' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-02-02T15:25:55.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:55 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/4152947887' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-02-02T15:25:55.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:55 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/2135702021' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-02-02T15:25:55.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.864+0000 7f76da200700 1 -- 172.21.15.1:0/1494987278 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f76d41a6140 con 0x7f76d4100fd0 2024-02-02T15:25:55.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.865+0000 7f76cbe00700 1 -- 172.21.15.1:0/1494987278 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f76c40956f0 con 0x7f76d4100fd0 2024-02-02T15:25:55.865 INFO:teuthology.orchestra.run.smithi001.stdout:34359738431 2024-02-02T15:25:55.869 INFO:tasks.cephadm.ceph_manager.ceph:need seq 266287972380 got 266287972383 for osd.9 2024-02-02T15:25:55.869 DEBUG:teuthology.parallel:result is None 2024-02-02T15:25:55.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.868+0000 7f76da200700 1 -- 172.21.15.1:0/1494987278 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f76b40a05b0 msgr2=0x7f76b40a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:55.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.868+0000 7f76da200700 1 --2- 172.21.15.1:0/1494987278 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f76b40a05b0 0x7f76b40a2a70 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f76d419a730 tx=0x7f76bc005fd0 comp rx=0 tx=0).stop 2024-02-02T15:25:55.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.868+0000 7f76da200700 1 -- 172.21.15.1:0/1494987278 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76d4100fd0 msgr2=0x7f76d4199160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:55.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.868+0000 7f76da200700 1 --2- 172.21.15.1:0/1494987278 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76d4100fd0 0x7f76d4199160 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f76c400e9a0 tx=0x7f76c400ecb0 comp rx=0 tx=0).stop 2024-02-02T15:25:55.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.869+0000 7f76da200700 1 -- 172.21.15.1:0/1494987278 shutdown_connections 2024-02-02T15:25:55.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.869+0000 7f76da200700 1 --2- 172.21.15.1:0/1494987278 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f76d41060b0 0x7f76d419da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:55.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.869+0000 7f76da200700 1 --2- 172.21.15.1:0/1494987278 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f76d4100fd0 0x7f76d4199160 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:55.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.869+0000 7f76da200700 1 --2- 172.21.15.1:0/1494987278 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f76b40a05b0 0x7f76b40a2a70 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:55.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.869+0000 7f76da200700 1 --2- 172.21.15.1:0/1494987278 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f76d4101980 0x7f76d41996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:55.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.869+0000 7f76da200700 1 -- 172.21.15.1:0/1494987278 >> 172.21.15.1:0/1494987278 conn(0x7f76d40fc860 msgr2=0x7f76d40fd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:55.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.870+0000 7f76da200700 1 -- 172.21.15.1:0/1494987278 shutdown_connections 2024-02-02T15:25:55.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:55.870+0000 7f76da200700 1 -- 172.21.15.1:0/1494987278 wait complete. 2024-02-02T15:25:55.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:55 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/4152947887' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-02-02T15:25:55.962 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:55 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/2135702021' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-02-02T15:25:56.435 INFO:tasks.cephadm.ceph_manager.ceph:need seq 322122547222 got 322122547224 for osd.11 2024-02-02T15:25:56.435 DEBUG:teuthology.parallel:result is None 2024-02-02T15:25:56.579 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:56 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/1494987278' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-02T15:25:56.580 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:56 smithi001 ceph-mon[35470]: pgmap v271: 129 pgs: 129 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-02-02T15:25:56.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:56.670+0000 7f4f71800700 1 -- 172.21.15.1:0/418242227 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f6c106ed0 msgr2=0x7f4f6c1072b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:56.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:56.670+0000 7f4f71800700 1 --2- 172.21.15.1:0/418242227 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f6c106ed0 0x7f4f6c1072b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c009990 tx=0x7f4f5c009ca0 comp rx=0 tx=0).stop 2024-02-02T15:25:56.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:56.670+0000 7f4f71800700 1 -- 172.21.15.1:0/418242227 shutdown_connections 2024-02-02T15:25:56.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:56.670+0000 7f4f71800700 1 --2- 172.21.15.1:0/418242227 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f6c069220 0x7f4f6c10db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:56.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:56.670+0000 7f4f71800700 1 --2- 172.21.15.1:0/418242227 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f6c106ed0 0x7f4f6c1072b0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:56.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:56.670+0000 7f4f71800700 1 --2- 172.21.15.1:0/418242227 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f6c068860 0x7f4f6c068ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:56.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:56.670+0000 7f4f71800700 1 -- 172.21.15.1:0/418242227 >> 172.21.15.1:0/418242227 conn(0x7f4f6c076d40 msgr2=0x7f4f6c077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:56.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:56.670+0000 7f4f71800700 1 -- 172.21.15.1:0/418242227 shutdown_connections 2024-02-02T15:25:56.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:56.670+0000 7f4f71800700 1 -- 172.21.15.1:0/418242227 wait complete. 2024-02-02T15:25:56.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:56.671+0000 7f4f71800700 1 Processor -- start 2024-02-02T15:25:56.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:56.672+0000 7f4f71800700 1 -- start start 2024-02-02T15:25:56.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:56.672+0000 7f4f71800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f6c068860 0x7f4f6c1990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:56.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:56.672+0000 7f4f71800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f6c069220 0x7f4f6c199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:56.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:56.672+0000 7f4f71800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f6c106ed0 0x7f4f6c19da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:56.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:56.672+0000 7f4f71800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6c19e080 con 0x7f4f6c106ed0 2024-02-02T15:25:56.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:56.672+0000 7f4f71800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6c19e1f0 con 0x7f4f6c069220 2024-02-02T15:25:56.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:56.672+0000 7f4f71800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6c19e3b0 con 0x7f4f6c068860 2024-02-02T15:25:56.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:56.672+0000 7f4f6b400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f6c106ed0 0x7f4f6c19da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:56.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:56.672+0000 7f4f6aa00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f6c068860 0x7f4f6c1990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:56.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:56.672+0000 7f4f6b400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f6c106ed0 0x7f4f6c19da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:32804/0 (socket says 172.21.15.1:32804) 2024-02-02T15:25:56.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:56.672+0000 7f4f6b400700 1 -- 172.21.15.1:0/1905795801 learned_addr learned my addr 172.21.15.1:0/1905795801 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:25:56.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:56.672+0000 7f4f6a000700 1 --2- 172.21.15.1:0/1905795801 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f6c069220 0x7f4f6c199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:56.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:56.673+0000 7f4f6b400700 1 -- 172.21.15.1:0/1905795801 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f6c068860 msgr2=0x7f4f6c1990c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:56.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:56.673+0000 7f4f6b400700 1 --2- 172.21.15.1:0/1905795801 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f6c068860 0x7f4f6c1990c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:56.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:56.673+0000 7f4f6b400700 1 -- 172.21.15.1:0/1905795801 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f6c069220 msgr2=0x7f4f6c199600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:56.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:56.673+0000 7f4f6b400700 1 --2- 172.21.15.1:0/1905795801 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f6c069220 0x7f4f6c199600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:56.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:56.673+0000 7f4f6b400700 1 -- 172.21.15.1:0/1905795801 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f54009520 con 0x7f4f6c106ed0 2024-02-02T15:25:56.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:56.673+0000 7f4f6b400700 1 --2- 172.21.15.1:0/1905795801 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f6c106ed0 0x7f4f6c19da70 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4f6400c0f0 tx=0x7f4f6400c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:56.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:56.673+0000 7f4f63400700 1 -- 172.21.15.1:0/1905795801 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f64003a00 con 0x7f4f6c106ed0 2024-02-02T15:25:56.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:56.673+0000 7f4f63400700 1 -- 172.21.15.1:0/1905795801 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f6400e030 con 0x7f4f6c106ed0 2024-02-02T15:25:56.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:56.673+0000 7f4f71800700 1 -- 172.21.15.1:0/1905795801 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f64009c40 con 0x7f4f6c106ed0 2024-02-02T15:25:56.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:56.673+0000 7f4f63400700 1 -- 172.21.15.1:0/1905795801 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f64004650 con 0x7f4f6c106ed0 2024-02-02T15:25:56.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:56.674+0000 7f4f71800700 1 -- 172.21.15.1:0/1905795801 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f5c0095f0 con 0x7f4f6c106ed0 2024-02-02T15:25:56.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:56.675+0000 7f4f71800700 1 -- 172.21.15.1:0/1905795801 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f6c04ea40 con 0x7f4f6c106ed0 2024-02-02T15:25:56.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:56.676+0000 7f4f63400700 1 -- 172.21.15.1:0/1905795801 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) v1 ==== 135120+0+0 (secure 0 0 0) 0x7f4f64003cf0 con 0x7f4f6c106ed0 2024-02-02T15:25:56.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:56.678+0000 7f4f63400700 1 --2- 172.21.15.1:0/1905795801 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f4f4c0a05b0 0x7f4f4c0a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:56.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:56.678+0000 7f4f63400700 1 -- 172.21.15.1:0/1905795801 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f4f640cbc80 con 0x7f4f6c106ed0 2024-02-02T15:25:56.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:56.678+0000 7f4f6aa00700 1 --2- 172.21.15.1:0/1905795801 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f4f4c0a05b0 0x7f4f4c0a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:56.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:56.679+0000 7f4f6aa00700 1 --2- 172.21.15.1:0/1905795801 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f4f4c0a05b0 0x7f4f4c0a2a70 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c00b2e0 tx=0x7f4f5c005fd0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:56.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:56.682+0000 7f4f63400700 1 -- 172.21.15.1:0/1905795801 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f4f640953e0 con 0x7f4f6c106ed0 2024-02-02T15:25:56.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:56 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/1494987278' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-02T15:25:56.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:56 smithi071 ceph-mon[44707]: pgmap v271: 129 pgs: 129 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-02-02T15:25:56.878 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738429 got 34359738431 for osd.0 2024-02-02T15:25:56.878 DEBUG:teuthology.parallel:result is None 2024-02-02T15:25:56.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:56.897+0000 7f4f71800700 1 -- 172.21.15.1:0/1905795801 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f4f6c1a5e20 con 0x7f4f6c106ed0 2024-02-02T15:25:56.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:56.898+0000 7f4f63400700 1 -- 172.21.15.1:0/1905795801 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4f64098a00 con 0x7f4f6c106ed0 2024-02-02T15:25:56.898 INFO:teuthology.orchestra.run.smithi001.stdout:184683593770 2024-02-02T15:25:56.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:56.900+0000 7f4f71800700 1 -- 172.21.15.1:0/1905795801 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f4f4c0a05b0 msgr2=0x7f4f4c0a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:56.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:56.900+0000 7f4f71800700 1 --2- 172.21.15.1:0/1905795801 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f4f4c0a05b0 0x7f4f4c0a2a70 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c00b2e0 tx=0x7f4f5c005fd0 comp rx=0 tx=0).stop 2024-02-02T15:25:56.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:56.900+0000 7f4f71800700 1 -- 172.21.15.1:0/1905795801 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f6c106ed0 msgr2=0x7f4f6c19da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:56.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:56.900+0000 7f4f71800700 1 --2- 172.21.15.1:0/1905795801 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f6c106ed0 0x7f4f6c19da70 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4f6400c0f0 tx=0x7f4f6400c400 comp rx=0 tx=0).stop 2024-02-02T15:25:56.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:56.900+0000 7f4f71800700 1 -- 172.21.15.1:0/1905795801 shutdown_connections 2024-02-02T15:25:56.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:56.900+0000 7f4f71800700 1 --2- 172.21.15.1:0/1905795801 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f6c106ed0 0x7f4f6c19da70 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:56.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:56.900+0000 7f4f71800700 1 --2- 172.21.15.1:0/1905795801 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f4f4c0a05b0 0x7f4f4c0a2a70 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:56.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:56.900+0000 7f4f71800700 1 --2- 172.21.15.1:0/1905795801 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f6c068860 0x7f4f6c1990c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:56.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:56.900+0000 7f4f71800700 1 --2- 172.21.15.1:0/1905795801 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f6c069220 0x7f4f6c199600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:56.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:56.900+0000 7f4f71800700 1 -- 172.21.15.1:0/1905795801 >> 172.21.15.1:0/1905795801 conn(0x7f4f6c076d40 msgr2=0x7f4f6c10beb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:56.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:56.900+0000 7f4f71800700 1 -- 172.21.15.1:0/1905795801 shutdown_connections 2024-02-02T15:25:56.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:56.900+0000 7f4f71800700 1 -- 172.21.15.1:0/1905795801 wait complete. 2024-02-02T15:25:56.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:56 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/1494987278' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-02T15:25:56.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:56 smithi142 ceph-mon[45728]: pgmap v271: 129 pgs: 129 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-02-02T15:25:57.399 INFO:tasks.cephadm.ceph_manager.ceph:need seq 184683593768 got 184683593770 for osd.6 2024-02-02T15:25:57.399 DEBUG:teuthology.parallel:result is None 2024-02-02T15:25:57.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:57.684+0000 7f680ba00700 1 -- 172.21.15.1:0/4142923229 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6804073a90 msgr2=0x7f680410d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:57.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:57.684+0000 7f680ba00700 1 --2- 172.21.15.1:0/4142923229 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6804073a90 0x7f680410d060 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f67ec009990 tx=0x7f67ec009ca0 comp rx=0 tx=0).stop 2024-02-02T15:25:57.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:57.684+0000 7f680ba00700 1 -- 172.21.15.1:0/4142923229 shutdown_connections 2024-02-02T15:25:57.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:57.684+0000 7f680ba00700 1 --2- 172.21.15.1:0/4142923229 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f680410d5a0 0x7f680410fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:57.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:57.684+0000 7f680ba00700 1 --2- 172.21.15.1:0/4142923229 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6804073170 0x7f6804073550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:57.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:57.684+0000 7f680ba00700 1 --2- 172.21.15.1:0/4142923229 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6804073a90 0x7f680410d060 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:57.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:57.684+0000 7f680ba00700 1 -- 172.21.15.1:0/4142923229 >> 172.21.15.1:0/4142923229 conn(0x7f68040fc830 msgr2=0x7f68040fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:57.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:57.684+0000 7f680ba00700 1 -- 172.21.15.1:0/4142923229 shutdown_connections 2024-02-02T15:25:57.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:57.684+0000 7f680ba00700 1 -- 172.21.15.1:0/4142923229 wait complete. 2024-02-02T15:25:57.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:57.686+0000 7f680ba00700 1 Processor -- start 2024-02-02T15:25:57.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:57.686+0000 7f680ba00700 1 -- start start 2024-02-02T15:25:57.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:57.687+0000 7f680ba00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6804073170 0x7f6804199030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:57.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:57.687+0000 7f680ba00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6804073a90 0x7f6804199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:57.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:57.687+0000 7f680ba00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f680410d5a0 0x7f680419d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:57.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:57.687+0000 7f680ba00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f680419dff0 con 0x7f6804073170 2024-02-02T15:25:57.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:57.687+0000 7f680ba00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f680419e160 con 0x7f680410d5a0 2024-02-02T15:25:57.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:57.687+0000 7f680ba00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f680419e320 con 0x7f6804073a90 2024-02-02T15:25:57.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:57.687+0000 7f6809000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6804073170 0x7f6804199030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:57.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:57.687+0000 7f6809000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6804073170 0x7f6804199030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:32816/0 (socket says 172.21.15.1:32816) 2024-02-02T15:25:57.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:57.687+0000 7f6809000700 1 -- 172.21.15.1:0/2122991972 learned_addr learned my addr 172.21.15.1:0/2122991972 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:25:57.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:57.687+0000 7f6803e00700 1 --2- 172.21.15.1:0/2122991972 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6804073a90 0x7f6804199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:57.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:57.687+0000 7f6809a00700 1 --2- 172.21.15.1:0/2122991972 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f680410d5a0 0x7f680419d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:57.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:57.687+0000 7f6809000700 1 -- 172.21.15.1:0/2122991972 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6804073a90 msgr2=0x7f6804199570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:57.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:57.687+0000 7f6809000700 1 --2- 172.21.15.1:0/2122991972 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6804073a90 0x7f6804199570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:57.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:57.687+0000 7f6809000700 1 -- 172.21.15.1:0/2122991972 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f680410d5a0 msgr2=0x7f680419d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:57.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:57.687+0000 7f6809000700 1 --2- 172.21.15.1:0/2122991972 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f680410d5a0 0x7f680419d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:57.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:57.687+0000 7f6809000700 1 -- 172.21.15.1:0/2122991972 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67ec0095f0 con 0x7f6804073170 2024-02-02T15:25:57.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:57.687+0000 7f6803e00700 1 --2- 172.21.15.1:0/2122991972 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6804073a90 0x7f6804199570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:25:57.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:57.688+0000 7f6809000700 1 --2- 172.21.15.1:0/2122991972 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6804073170 0x7f6804199030 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f67f400e9a0 tx=0x7f67f400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:57.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:57.688+0000 7f6801600700 1 -- 172.21.15.1:0/2122991972 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f4009790 con 0x7f6804073170 2024-02-02T15:25:57.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:57.688+0000 7f680ba00700 1 -- 172.21.15.1:0/2122991972 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f680419e600 con 0x7f6804073170 2024-02-02T15:25:57.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:57.688+0000 7f6801600700 1 -- 172.21.15.1:0/2122991972 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f67f4010430 con 0x7f6804073170 2024-02-02T15:25:57.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:57.688+0000 7f6801600700 1 -- 172.21.15.1:0/2122991972 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f4018680 con 0x7f6804073170 2024-02-02T15:25:57.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:57.688+0000 7f680ba00700 1 -- 172.21.15.1:0/2122991972 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68041a5e60 con 0x7f6804073170 2024-02-02T15:25:57.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:57.690+0000 7f680ba00700 1 -- 172.21.15.1:0/2122991972 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67dc005320 con 0x7f6804073170 2024-02-02T15:25:57.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:57.691+0000 7f6801600700 1 -- 172.21.15.1:0/2122991972 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) v1 ==== 135120+0+0 (secure 0 0 0) 0x7f67f40105a0 con 0x7f6804073170 2024-02-02T15:25:57.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:57.693+0000 7f6801600700 1 --2- 172.21.15.1:0/2122991972 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f67e40a05b0 0x7f67e40a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:57.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:57.693+0000 7f6801600700 1 -- 172.21.15.1:0/2122991972 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f67f4014070 con 0x7f6804073170 2024-02-02T15:25:57.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:57.693+0000 7f6803e00700 1 --2- 172.21.15.1:0/2122991972 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f67e40a05b0 0x7f67e40a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:57.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:57.694+0000 7f6803e00700 1 --2- 172.21.15.1:0/2122991972 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f67e40a05b0 0x7f67e40a2a70 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f67ec00b2e0 tx=0x7f67ec005960 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:57.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:57.697+0000 7f6801600700 1 -- 172.21.15.1:0/2122991972 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f67f4052110 con 0x7f6804073170 2024-02-02T15:25:57.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:57 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/1905795801' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-02-02T15:25:57.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:57 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/1905795801' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-02-02T15:25:57.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:57.927+0000 7f680ba00700 1 -- 172.21.15.1:0/2122991972 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7f67dc005190 con 0x7f6804073170 2024-02-02T15:25:57.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:57.927+0000 7f6801600700 1 -- 172.21.15.1:0/2122991972 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f67f40cf030 con 0x7f6804073170 2024-02-02T15:25:57.927 INFO:teuthology.orchestra.run.smithi001.stdout:287762808858 2024-02-02T15:25:57.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:57.930+0000 7f680ba00700 1 -- 172.21.15.1:0/2122991972 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f67e40a05b0 msgr2=0x7f67e40a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:57.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:57.930+0000 7f680ba00700 1 --2- 172.21.15.1:0/2122991972 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f67e40a05b0 0x7f67e40a2a70 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f67ec00b2e0 tx=0x7f67ec005960 comp rx=0 tx=0).stop 2024-02-02T15:25:57.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:57.930+0000 7f680ba00700 1 -- 172.21.15.1:0/2122991972 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6804073170 msgr2=0x7f6804199030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:57.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:57.930+0000 7f680ba00700 1 --2- 172.21.15.1:0/2122991972 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6804073170 0x7f6804199030 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f67f400e9a0 tx=0x7f67f400ecb0 comp rx=0 tx=0).stop 2024-02-02T15:25:57.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:57.930+0000 7f680ba00700 1 -- 172.21.15.1:0/2122991972 shutdown_connections 2024-02-02T15:25:57.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:57.930+0000 7f680ba00700 1 --2- 172.21.15.1:0/2122991972 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f680410d5a0 0x7f680419d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:57.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:57.930+0000 7f680ba00700 1 --2- 172.21.15.1:0/2122991972 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6804073170 0x7f6804199030 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:57.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:57.930+0000 7f680ba00700 1 --2- 172.21.15.1:0/2122991972 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f67e40a05b0 0x7f67e40a2a70 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:57.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:57.930+0000 7f680ba00700 1 --2- 172.21.15.1:0/2122991972 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6804073a90 0x7f6804199570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:57.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:57.930+0000 7f680ba00700 1 -- 172.21.15.1:0/2122991972 >> 172.21.15.1:0/2122991972 conn(0x7f68040fc830 msgr2=0x7f680410f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:57.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:57.930+0000 7f680ba00700 1 -- 172.21.15.1:0/2122991972 shutdown_connections 2024-02-02T15:25:57.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:57.930+0000 7f680ba00700 1 -- 172.21.15.1:0/2122991972 wait complete. 2024-02-02T15:25:57.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:57 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/1905795801' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-02-02T15:25:58.320 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808857 got 287762808858 for osd.10 2024-02-02T15:25:58.320 DEBUG:teuthology.parallel:result is None 2024-02-02T15:25:58.320 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-02-02T15:25:58.321 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph pg dump --format=json 2024-02-02T15:25:58.705 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.a/config 2024-02-02T15:25:59.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:58 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:25:59.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:58 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:25:59.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:58 smithi142 ceph-mon[45728]: pgmap v272: 129 pgs: 129 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 10 op/s 2024-02-02T15:25:59.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:58 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/2122991972' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-02-02T15:25:59.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:58 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:25:59.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:58 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:25:59.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:25:58 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:25:59.178 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:58 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:25:59.179 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:58 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:25:59.179 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:58 smithi001 ceph-mon[35470]: pgmap v272: 129 pgs: 129 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 10 op/s 2024-02-02T15:25:59.179 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:58 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/2122991972' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-02-02T15:25:59.179 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:58 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:25:59.179 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:58 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:25:59.179 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:25:58 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:25:59.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:58 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:25:59.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:58 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:25:59.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:58 smithi071 ceph-mon[44707]: pgmap v272: 129 pgs: 129 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 10 op/s 2024-02-02T15:25:59.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:58 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/2122991972' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-02-02T15:25:59.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:58 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:25:59.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:58 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:25:59.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:25:58 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' 2024-02-02T15:25:59.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:59.857+0000 7f7c04e00700 1 -- 172.21.15.1:0/654875170 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7c001060a0 msgr2=0x7f7c0010dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:59.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:59.857+0000 7f7c04e00700 1 --2- 172.21.15.1:0/654875170 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7c001060a0 0x7f7c0010dc70 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7bf800b0a0 tx=0x7f7bf800b3b0 comp rx=0 tx=0).stop 2024-02-02T15:25:59.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:59.857+0000 7f7c04e00700 1 -- 172.21.15.1:0/654875170 shutdown_connections 2024-02-02T15:25:59.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:59.857+0000 7f7c04e00700 1 --2- 172.21.15.1:0/654875170 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7c001060a0 0x7f7c0010dc70 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:59.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:59.857+0000 7f7c04e00700 1 --2- 172.21.15.1:0/654875170 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c00100fc0 0x7f7c001013a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:59.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:59.857+0000 7f7c04e00700 1 --2- 172.21.15.1:0/654875170 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c00101970 0x7f7c001059c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:59.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:59.857+0000 7f7c04e00700 1 -- 172.21.15.1:0/654875170 >> 172.21.15.1:0/654875170 conn(0x7f7c000fc830 msgr2=0x7f7c000fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:25:59.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:59.857+0000 7f7c04e00700 1 -- 172.21.15.1:0/654875170 shutdown_connections 2024-02-02T15:25:59.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:59.857+0000 7f7c04e00700 1 -- 172.21.15.1:0/654875170 wait complete. 2024-02-02T15:25:59.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:59.858+0000 7f7c04e00700 1 Processor -- start 2024-02-02T15:25:59.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:59.858+0000 7f7c04e00700 1 -- start start 2024-02-02T15:25:59.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:59.858+0000 7f7c04e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7c00100fc0 0x7f7c00199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:59.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:59.858+0000 7f7c04e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c00101970 0x7f7c001996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:59.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:59.858+0000 7f7c04e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c001060a0 0x7f7c0019db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:59.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:59.858+0000 7f7c04e00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c0019e150 con 0x7f7c00100fc0 2024-02-02T15:25:59.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:59.858+0000 7f7c04e00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c0019e2c0 con 0x7f7c00101970 2024-02-02T15:25:59.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:59.858+0000 7f7c04e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c0019e480 con 0x7f7c001060a0 2024-02-02T15:25:59.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:59.859+0000 7f7bfde00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7c00100fc0 0x7f7c00199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:59.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:59.859+0000 7f7bfde00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7c00100fc0 0x7f7c00199190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:32838/0 (socket says 172.21.15.1:32838) 2024-02-02T15:25:59.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:59.859+0000 7f7bfde00700 1 -- 172.21.15.1:0/2774001760 learned_addr learned my addr 172.21.15.1:0/2774001760 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:25:59.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:59.859+0000 7f7bfd400700 1 --2- 172.21.15.1:0/2774001760 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c00101970 0x7f7c001996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:59.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:59.859+0000 7f7bfe800700 1 --2- 172.21.15.1:0/2774001760 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c001060a0 0x7f7c0019db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:59.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:59.859+0000 7f7bfde00700 1 -- 172.21.15.1:0/2774001760 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c001060a0 msgr2=0x7f7c0019db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:59.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:59.859+0000 7f7bfde00700 1 --2- 172.21.15.1:0/2774001760 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c001060a0 0x7f7c0019db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:59.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:59.859+0000 7f7bfde00700 1 -- 172.21.15.1:0/2774001760 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c00101970 msgr2=0x7f7c001996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:25:59.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:59.859+0000 7f7bfde00700 1 --2- 172.21.15.1:0/2774001760 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c00101970 0x7f7c001996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:25:59.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:59.859+0000 7f7bfde00700 1 -- 172.21.15.1:0/2774001760 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bf8009d00 con 0x7f7c00100fc0 2024-02-02T15:25:59.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:59.859+0000 7f7bfd400700 1 --2- 172.21.15.1:0/2774001760 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c00101970 0x7f7c001996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:25:59.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:59.859+0000 7f7bfde00700 1 --2- 172.21.15.1:0/2774001760 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7c00100fc0 0x7f7c00199190 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7bf000e9a0 tx=0x7f7bf000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:59.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:59.860+0000 7f7bf6a00700 1 -- 172.21.15.1:0/2774001760 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf0009790 con 0x7f7c00100fc0 2024-02-02T15:25:59.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:59.860+0000 7f7bf6a00700 1 -- 172.21.15.1:0/2774001760 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7bf0010430 con 0x7f7c00100fc0 2024-02-02T15:25:59.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:59.860+0000 7f7c04e00700 1 -- 172.21.15.1:0/2774001760 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c0019e760 con 0x7f7c00100fc0 2024-02-02T15:25:59.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:59.860+0000 7f7bf6a00700 1 -- 172.21.15.1:0/2774001760 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf0018680 con 0x7f7c00100fc0 2024-02-02T15:25:59.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:59.860+0000 7f7c04e00700 1 -- 172.21.15.1:0/2774001760 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c001a5fc0 con 0x7f7c00100fc0 2024-02-02T15:25:59.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:59.861+0000 7f7c04e00700 1 -- 172.21.15.1:0/2774001760 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c0004ea40 con 0x7f7c00100fc0 2024-02-02T15:25:59.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:59.863+0000 7f7bf6a00700 1 -- 172.21.15.1:0/2774001760 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) v1 ==== 135120+0+0 (secure 0 0 0) 0x7f7bf00105a0 con 0x7f7c00100fc0 2024-02-02T15:25:59.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:59.864+0000 7f7bf6a00700 1 --2- 172.21.15.1:0/2774001760 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f7be00a05b0 0x7f7be00a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:25:59.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:59.864+0000 7f7bf6a00700 1 -- 172.21.15.1:0/2774001760 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f7bf0014070 con 0x7f7c00100fc0 2024-02-02T15:25:59.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:59.864+0000 7f7bfd400700 1 --2- 172.21.15.1:0/2774001760 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f7be00a05b0 0x7f7be00a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:25:59.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:59.865+0000 7f7bfd400700 1 --2- 172.21.15.1:0/2774001760 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f7be00a05b0 0x7f7be00a2a70 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7c0019a7b0 tx=0x7f7be8009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:25:59.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:25:59.868+0000 7f7bf6a00700 1 -- 172.21.15.1:0/2774001760 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f7bf0052110 con 0x7f7c00100fc0 2024-02-02T15:26:00.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:00.074+0000 7f7c04e00700 1 -- 172.21.15.1:0/2774001760 --> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7c00073f80 con 0x7f7be00a05b0 2024-02-02T15:26:00.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:00.088+0000 7f7bf6a00700 1 -- 172.21.15.1:0/2774001760 <== mgr.14154 v2:172.21.15.1:6800/3768989981 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427242 (secure 0 0 0) 0x7f7c00073f80 con 0x7f7be00a05b0 2024-02-02T15:26:00.095 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:26:00.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:00.093+0000 7f7c04e00700 1 -- 172.21.15.1:0/2774001760 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f7be00a05b0 msgr2=0x7f7be00a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:00.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:00.093+0000 7f7c04e00700 1 --2- 172.21.15.1:0/2774001760 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f7be00a05b0 0x7f7be00a2a70 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7c0019a7b0 tx=0x7f7be8009260 comp rx=0 tx=0).stop 2024-02-02T15:26:00.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:00.093+0000 7f7c04e00700 1 -- 172.21.15.1:0/2774001760 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7c00100fc0 msgr2=0x7f7c00199190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:00.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:00.093+0000 7f7c04e00700 1 --2- 172.21.15.1:0/2774001760 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7c00100fc0 0x7f7c00199190 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7bf000e9a0 tx=0x7f7bf000ecb0 comp rx=0 tx=0).stop 2024-02-02T15:26:00.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:00.093+0000 7f7c04e00700 1 -- 172.21.15.1:0/2774001760 shutdown_connections 2024-02-02T15:26:00.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:00.093+0000 7f7c04e00700 1 --2- 172.21.15.1:0/2774001760 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f7be00a05b0 0x7f7be00a2a70 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:00.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:00.093+0000 7f7c04e00700 1 --2- 172.21.15.1:0/2774001760 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c001060a0 0x7f7c0019db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:00.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:00.093+0000 7f7c04e00700 1 --2- 172.21.15.1:0/2774001760 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7c00100fc0 0x7f7c00199190 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:00.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:00.093+0000 7f7c04e00700 1 --2- 172.21.15.1:0/2774001760 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c00101970 0x7f7c001996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:00.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:00.093+0000 7f7c04e00700 1 -- 172.21.15.1:0/2774001760 >> 172.21.15.1:0/2774001760 conn(0x7f7c000fc830 msgr2=0x7f7c000fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:00.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:00.093+0000 7f7c04e00700 1 -- 172.21.15.1:0/2774001760 shutdown_connections 2024-02-02T15:26:00.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:00.093+0000 7f7c04e00700 1 -- 172.21.15.1:0/2774001760 wait complete. 2024-02-02T15:26:00.099 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-02-02T15:26:00.514 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":273,"stamp":"2024-02-02T15:25:59.923025+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657632,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":542,"num_read_kb":396,"num_write":421,"num_write_kb":3077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1180804,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":347,"ondisk_log_size":347,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335916,"kb_used_data":11668,"kb_used_omap":20,"kb_used_meta":324075,"kb_avail":1124605908,"statfs":{"total":1151940427776,"available":1151596449792,"internally_reserved":0,"allocated":11948032,"data_stored":7677571,"data_compressed":320901,"data_compressed_allocated":430080,"data_compressed_original":2359296,"omap_allocated":21372,"internal_metadata":331852932},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":111,"num_read_kb":58,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003398"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.275844+0000","last_change":"2024-02-02T15:24:47.793308+0000","last_active":"2024-02-02T15:25:26.275844+0000","last_peered":"2024-02-02T15:25:26.275844+0000","last_clean":"2024-02-02T15:25:26.275844+0000","last_became_active":"2024-02-02T15:24:47.792420+0000","last_became_peered":"2024-02-02T15:24:47.792420+0000","last_unstale":"2024-02-02T15:25:26.275844+0000","last_undegraded":"2024-02-02T15:25:26.275844+0000","last_fullsized":"2024-02-02T15:25:26.275844+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:39:09.871317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.280986+0000","last_change":"2024-02-02T15:24:47.791272+0000","last_active":"2024-02-02T15:25:26.280986+0000","last_peered":"2024-02-02T15:25:26.280986+0000","last_clean":"2024-02-02T15:25:26.280986+0000","last_became_active":"2024-02-02T15:24:47.790902+0000","last_became_peered":"2024-02-02T15:24:47.790902+0000","last_unstale":"2024-02-02T15:25:26.280986+0000","last_undegraded":"2024-02-02T15:25:26.280986+0000","last_fullsized":"2024-02-02T15:25:26.280986+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:31:11.620873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.280219+0000","last_change":"2024-02-02T15:24:47.798712+0000","last_active":"2024-02-02T15:25:26.280219+0000","last_peered":"2024-02-02T15:25:26.280219+0000","last_clean":"2024-02-02T15:25:26.280219+0000","last_became_active":"2024-02-02T15:24:47.798405+0000","last_became_peered":"2024-02-02T15:24:47.798405+0000","last_unstale":"2024-02-02T15:25:26.280219+0000","last_undegraded":"2024-02-02T15:25:26.280219+0000","last_fullsized":"2024-02-02T15:25:26.280219+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:37:15.238070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.275855+0000","last_change":"2024-02-02T15:24:47.786456+0000","last_active":"2024-02-02T15:25:26.275855+0000","last_peered":"2024-02-02T15:25:26.275855+0000","last_clean":"2024-02-02T15:25:26.275855+0000","last_became_active":"2024-02-02T15:24:47.785520+0000","last_became_peered":"2024-02-02T15:24:47.785520+0000","last_unstale":"2024-02-02T15:25:26.275855+0000","last_undegraded":"2024-02-02T15:25:26.275855+0000","last_fullsized":"2024-02-02T15:25:26.275855+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:16:04.003459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.277322+0000","last_change":"2024-02-02T15:24:47.780084+0000","last_active":"2024-02-02T15:25:26.277322+0000","last_peered":"2024-02-02T15:25:26.277322+0000","last_clean":"2024-02-02T15:25:26.277322+0000","last_became_active":"2024-02-02T15:24:47.779667+0000","last_became_peered":"2024-02-02T15:24:47.779667+0000","last_unstale":"2024-02-02T15:25:26.277322+0000","last_undegraded":"2024-02-02T15:25:26.277322+0000","last_fullsized":"2024-02-02T15:25:26.277322+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:30:15.377691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.277446+0000","last_change":"2024-02-02T15:24:47.786173+0000","last_active":"2024-02-02T15:25:26.277446+0000","last_peered":"2024-02-02T15:25:26.277446+0000","last_clean":"2024-02-02T15:25:26.277446+0000","last_became_active":"2024-02-02T15:24:47.785301+0000","last_became_peered":"2024-02-02T15:24:47.785301+0000","last_unstale":"2024-02-02T15:25:26.277446+0000","last_undegraded":"2024-02-02T15:25:26.277446+0000","last_fullsized":"2024-02-02T15:25:26.277446+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:14:58.524317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.275443+0000","last_change":"2024-02-02T15:24:47.790913+0000","last_active":"2024-02-02T15:25:26.275443+0000","last_peered":"2024-02-02T15:25:26.275443+0000","last_clean":"2024-02-02T15:25:26.275443+0000","last_became_active":"2024-02-02T15:24:47.790566+0000","last_became_peered":"2024-02-02T15:24:47.790566+0000","last_unstale":"2024-02-02T15:25:26.275443+0000","last_undegraded":"2024-02-02T15:25:26.275443+0000","last_fullsized":"2024-02-02T15:25:26.275443+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:34:51.954830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.276550+0000","last_change":"2024-02-02T15:24:47.786465+0000","last_active":"2024-02-02T15:25:26.276550+0000","last_peered":"2024-02-02T15:25:26.276550+0000","last_clean":"2024-02-02T15:25:26.276550+0000","last_became_active":"2024-02-02T15:24:47.786175+0000","last_became_peered":"2024-02-02T15:24:47.786175+0000","last_unstale":"2024-02-02T15:25:26.276550+0000","last_undegraded":"2024-02-02T15:25:26.276550+0000","last_fullsized":"2024-02-02T15:25:26.276550+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:42:13.748850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.273552+0000","last_change":"2024-02-02T15:24:47.787115+0000","last_active":"2024-02-02T15:25:26.273552+0000","last_peered":"2024-02-02T15:25:26.273552+0000","last_clean":"2024-02-02T15:25:26.273552+0000","last_became_active":"2024-02-02T15:24:47.786804+0000","last_became_peered":"2024-02-02T15:24:47.786804+0000","last_unstale":"2024-02-02T15:25:26.273552+0000","last_undegraded":"2024-02-02T15:25:26.273552+0000","last_fullsized":"2024-02-02T15:25:26.273552+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T03:06:17.873243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.277877+0000","last_change":"2024-02-02T15:24:47.787211+0000","last_active":"2024-02-02T15:25:26.277877+0000","last_peered":"2024-02-02T15:25:26.277877+0000","last_clean":"2024-02-02T15:25:26.277877+0000","last_became_active":"2024-02-02T15:24:47.786888+0000","last_became_peered":"2024-02-02T15:24:47.786888+0000","last_unstale":"2024-02-02T15:25:26.277877+0000","last_undegraded":"2024-02-02T15:25:26.277877+0000","last_fullsized":"2024-02-02T15:25:26.277877+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:14:21.442787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.275882+0000","last_change":"2024-02-02T15:24:47.798962+0000","last_active":"2024-02-02T15:25:26.275882+0000","last_peered":"2024-02-02T15:25:26.275882+0000","last_clean":"2024-02-02T15:25:26.275882+0000","last_became_active":"2024-02-02T15:24:47.798657+0000","last_became_peered":"2024-02-02T15:24:47.798657+0000","last_unstale":"2024-02-02T15:25:26.275882+0000","last_undegraded":"2024-02-02T15:25:26.275882+0000","last_fullsized":"2024-02-02T15:25:26.275882+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:53:37.316027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.275776+0000","last_change":"2024-02-02T15:24:47.782647+0000","last_active":"2024-02-02T15:25:26.275776+0000","last_peered":"2024-02-02T15:25:26.275776+0000","last_clean":"2024-02-02T15:25:26.275776+0000","last_became_active":"2024-02-02T15:24:47.782344+0000","last_became_peered":"2024-02-02T15:24:47.782344+0000","last_unstale":"2024-02-02T15:25:26.275776+0000","last_undegraded":"2024-02-02T15:25:26.275776+0000","last_fullsized":"2024-02-02T15:25:26.275776+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:06:54.874657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.272126+0000","last_change":"2024-02-02T15:24:47.787032+0000","last_active":"2024-02-02T15:25:26.272126+0000","last_peered":"2024-02-02T15:25:26.272126+0000","last_clean":"2024-02-02T15:25:26.272126+0000","last_became_active":"2024-02-02T15:24:47.786710+0000","last_became_peered":"2024-02-02T15:24:47.786710+0000","last_unstale":"2024-02-02T15:25:26.272126+0000","last_undegraded":"2024-02-02T15:25:26.272126+0000","last_fullsized":"2024-02-02T15:25:26.272126+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:26:15.429316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.278945+0000","last_change":"2024-02-02T15:24:47.785708+0000","last_active":"2024-02-02T15:25:26.278945+0000","last_peered":"2024-02-02T15:25:26.278945+0000","last_clean":"2024-02-02T15:25:26.278945+0000","last_became_active":"2024-02-02T15:24:47.785280+0000","last_became_peered":"2024-02-02T15:24:47.785280+0000","last_unstale":"2024-02-02T15:25:26.278945+0000","last_undegraded":"2024-02-02T15:25:26.278945+0000","last_fullsized":"2024-02-02T15:25:26.278945+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T02:21:23.224933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.278605+0000","last_change":"2024-02-02T15:24:47.783904+0000","last_active":"2024-02-02T15:25:26.278605+0000","last_peered":"2024-02-02T15:25:26.278605+0000","last_clean":"2024-02-02T15:25:26.278605+0000","last_became_active":"2024-02-02T15:24:47.783599+0000","last_became_peered":"2024-02-02T15:24:47.783599+0000","last_unstale":"2024-02-02T15:25:26.278605+0000","last_undegraded":"2024-02-02T15:25:26.278605+0000","last_fullsized":"2024-02-02T15:25:26.278605+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:35:45.805527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.272807+0000","last_change":"2024-02-02T15:24:47.795770+0000","last_active":"2024-02-02T15:25:26.272807+0000","last_peered":"2024-02-02T15:25:26.272807+0000","last_clean":"2024-02-02T15:25:26.272807+0000","last_became_active":"2024-02-02T15:24:47.795433+0000","last_became_peered":"2024-02-02T15:24:47.795433+0000","last_unstale":"2024-02-02T15:25:26.272807+0000","last_undegraded":"2024-02-02T15:25:26.272807+0000","last_fullsized":"2024-02-02T15:25:26.272807+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:50:07.496998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.275747+0000","last_change":"2024-02-02T15:24:47.793684+0000","last_active":"2024-02-02T15:25:26.275747+0000","last_peered":"2024-02-02T15:25:26.275747+0000","last_clean":"2024-02-02T15:25:26.275747+0000","last_became_active":"2024-02-02T15:24:47.792853+0000","last_became_peered":"2024-02-02T15:24:47.792853+0000","last_unstale":"2024-02-02T15:25:26.275747+0000","last_undegraded":"2024-02-02T15:25:26.275747+0000","last_fullsized":"2024-02-02T15:25:26.275747+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:30:40.379379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.282537+0000","last_change":"2024-02-02T15:24:47.780935+0000","last_active":"2024-02-02T15:25:26.282537+0000","last_peered":"2024-02-02T15:25:26.282537+0000","last_clean":"2024-02-02T15:25:26.282537+0000","last_became_active":"2024-02-02T15:24:47.780642+0000","last_became_peered":"2024-02-02T15:24:47.780642+0000","last_unstale":"2024-02-02T15:25:26.282537+0000","last_undegraded":"2024-02-02T15:25:26.282537+0000","last_fullsized":"2024-02-02T15:25:26.282537+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:36:37.615368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.280797+0000","last_change":"2024-02-02T15:24:47.791051+0000","last_active":"2024-02-02T15:25:26.280797+0000","last_peered":"2024-02-02T15:25:26.280797+0000","last_clean":"2024-02-02T15:25:26.280797+0000","last_became_active":"2024-02-02T15:24:47.790732+0000","last_became_peered":"2024-02-02T15:24:47.790732+0000","last_unstale":"2024-02-02T15:25:26.280797+0000","last_undegraded":"2024-02-02T15:25:26.280797+0000","last_fullsized":"2024-02-02T15:25:26.280797+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:45:05.155144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.279514+0000","last_change":"2024-02-02T15:24:51.814699+0000","last_active":"2024-02-02T15:25:26.279514+0000","last_peered":"2024-02-02T15:25:26.279514+0000","last_clean":"2024-02-02T15:25:26.279514+0000","last_became_active":"2024-02-02T15:24:51.814251+0000","last_became_peered":"2024-02-02T15:24:51.814251+0000","last_unstale":"2024-02-02T15:25:26.279514+0000","last_undegraded":"2024-02-02T15:25:26.279514+0000","last_fullsized":"2024-02-02T15:25:26.279514+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:27:21.674922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"84'13","reported_seq":71,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T15:25:57.332488+0000","last_change":"2024-02-02T15:24:47.269429+0000","last_active":"2024-02-02T15:25:57.332488+0000","last_peered":"2024-02-02T15:25:57.332488+0000","last_clean":"2024-02-02T15:25:57.332488+0000","last_became_active":"2024-02-02T15:24:47.269023+0000","last_became_peered":"2024-02-02T15:24:47.269023+0000","last_unstale":"2024-02-02T15:25:57.332488+0000","last_undegraded":"2024-02-02T15:25:57.332488+0000","last_fullsized":"2024-02-02T15:25:57.332488+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T03:11:46.344438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":40,"num_read_kb":40,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.276412+0000","last_change":"2024-02-02T15:24:47.785900+0000","last_active":"2024-02-02T15:25:26.276412+0000","last_peered":"2024-02-02T15:25:26.276412+0000","last_clean":"2024-02-02T15:25:26.276412+0000","last_became_active":"2024-02-02T15:24:47.781994+0000","last_became_peered":"2024-02-02T15:24:47.781994+0000","last_unstale":"2024-02-02T15:25:26.276412+0000","last_undegraded":"2024-02-02T15:25:26.276412+0000","last_fullsized":"2024-02-02T15:25:26.276412+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:00:52.432637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.275056+0000","last_change":"2024-02-02T15:24:51.825655+0000","last_active":"2024-02-02T15:25:26.275056+0000","last_peered":"2024-02-02T15:25:26.275056+0000","last_clean":"2024-02-02T15:25:26.275056+0000","last_became_active":"2024-02-02T15:24:51.825199+0000","last_became_peered":"2024-02-02T15:24:51.825199+0000","last_unstale":"2024-02-02T15:25:26.275056+0000","last_undegraded":"2024-02-02T15:25:26.275056+0000","last_fullsized":"2024-02-02T15:25:26.275056+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:50:32.283394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"84'2","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.278347+0000","last_change":"2024-02-02T15:24:47.281869+0000","last_active":"2024-02-02T15:25:26.278347+0000","last_peered":"2024-02-02T15:25:26.278347+0000","last_clean":"2024-02-02T15:25:26.278347+0000","last_became_active":"2024-02-02T15:24:47.280825+0000","last_became_peered":"2024-02-02T15:24:47.280825+0000","last_unstale":"2024-02-02T15:25:26.278347+0000","last_undegraded":"2024-02-02T15:25:26.278347+0000","last_fullsized":"2024-02-02T15:25:26.278347+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:24:19.688389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.280594+0000","last_change":"2024-02-02T15:24:47.795893+0000","last_active":"2024-02-02T15:25:26.280594+0000","last_peered":"2024-02-02T15:25:26.280594+0000","last_clean":"2024-02-02T15:25:26.280594+0000","last_became_active":"2024-02-02T15:24:47.795491+0000","last_became_peered":"2024-02-02T15:24:47.795491+0000","last_unstale":"2024-02-02T15:25:26.280594+0000","last_undegraded":"2024-02-02T15:25:26.280594+0000","last_fullsized":"2024-02-02T15:25:26.280594+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:54:38.923036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.275861+0000","last_change":"2024-02-02T15:24:51.826980+0000","last_active":"2024-02-02T15:25:26.275861+0000","last_peered":"2024-02-02T15:25:26.275861+0000","last_clean":"2024-02-02T15:25:26.275861+0000","last_became_active":"2024-02-02T15:24:51.826221+0000","last_became_peered":"2024-02-02T15:24:51.826221+0000","last_unstale":"2024-02-02T15:25:26.275861+0000","last_undegraded":"2024-02-02T15:25:26.275861+0000","last_fullsized":"2024-02-02T15:25:26.275861+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:45:17.126174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"84'12","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.278436+0000","last_change":"2024-02-02T15:24:47.272219+0000","last_active":"2024-02-02T15:25:26.278436+0000","last_peered":"2024-02-02T15:25:26.278436+0000","last_clean":"2024-02-02T15:25:26.278436+0000","last_became_active":"2024-02-02T15:24:47.271899+0000","last_became_peered":"2024-02-02T15:24:47.271899+0000","last_unstale":"2024-02-02T15:25:26.278436+0000","last_undegraded":"2024-02-02T15:25:26.278436+0000","last_fullsized":"2024-02-02T15:25:26.278436+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:44:27.370783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.273053+0000","last_change":"2024-02-02T15:24:47.783434+0000","last_active":"2024-02-02T15:25:26.273053+0000","last_peered":"2024-02-02T15:25:26.273053+0000","last_clean":"2024-02-02T15:25:26.273053+0000","last_became_active":"2024-02-02T15:24:47.782130+0000","last_became_peered":"2024-02-02T15:24:47.782130+0000","last_unstale":"2024-02-02T15:25:26.273053+0000","last_undegraded":"2024-02-02T15:25:26.273053+0000","last_fullsized":"2024-02-02T15:25:26.273053+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:32:25.981913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.275051+0000","last_change":"2024-02-02T15:24:51.810791+0000","last_active":"2024-02-02T15:25:26.275051+0000","last_peered":"2024-02-02T15:25:26.275051+0000","last_clean":"2024-02-02T15:25:26.275051+0000","last_became_active":"2024-02-02T15:24:51.810379+0000","last_became_peered":"2024-02-02T15:24:51.810379+0000","last_unstale":"2024-02-02T15:25:26.275051+0000","last_undegraded":"2024-02-02T15:25:26.275051+0000","last_fullsized":"2024-02-02T15:25:26.275051+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:49:44.635820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.277871+0000","last_change":"2024-02-02T15:24:47.271225+0000","last_active":"2024-02-02T15:25:26.277871+0000","last_peered":"2024-02-02T15:25:26.277871+0000","last_clean":"2024-02-02T15:25:26.277871+0000","last_became_active":"2024-02-02T15:24:47.270870+0000","last_became_peered":"2024-02-02T15:24:47.270870+0000","last_unstale":"2024-02-02T15:25:26.277871+0000","last_undegraded":"2024-02-02T15:25:26.277871+0000","last_fullsized":"2024-02-02T15:25:26.277871+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:16:55.659950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.272911+0000","last_change":"2024-02-02T15:24:47.791367+0000","last_active":"2024-02-02T15:25:26.272911+0000","last_peered":"2024-02-02T15:25:26.272911+0000","last_clean":"2024-02-02T15:25:26.272911+0000","last_became_active":"2024-02-02T15:24:47.790835+0000","last_became_peered":"2024-02-02T15:24:47.790835+0000","last_unstale":"2024-02-02T15:25:26.272911+0000","last_undegraded":"2024-02-02T15:25:26.272911+0000","last_fullsized":"2024-02-02T15:25:26.272911+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:02:51.807571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.280640+0000","last_change":"2024-02-02T15:24:51.807328+0000","last_active":"2024-02-02T15:25:26.280640+0000","last_peered":"2024-02-02T15:25:26.280640+0000","last_clean":"2024-02-02T15:25:26.280640+0000","last_became_active":"2024-02-02T15:24:51.806853+0000","last_became_peered":"2024-02-02T15:24:51.806853+0000","last_unstale":"2024-02-02T15:25:26.280640+0000","last_undegraded":"2024-02-02T15:25:26.280640+0000","last_fullsized":"2024-02-02T15:25:26.280640+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:46:13.898450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"84'5","reported_seq":61,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T15:25:54.311460+0000","last_change":"2024-02-02T15:24:47.268192+0000","last_active":"2024-02-02T15:25:54.311460+0000","last_peered":"2024-02-02T15:25:54.311460+0000","last_clean":"2024-02-02T15:25:54.311460+0000","last_became_active":"2024-02-02T15:24:47.267796+0000","last_became_peered":"2024-02-02T15:24:47.267796+0000","last_unstale":"2024-02-02T15:25:54.311460+0000","last_undegraded":"2024-02-02T15:25:54.311460+0000","last_fullsized":"2024-02-02T15:25:54.311460+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:12:13.161690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":38,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.277181+0000","last_change":"2024-02-02T15:24:47.782976+0000","last_active":"2024-02-02T15:25:26.277181+0000","last_peered":"2024-02-02T15:25:26.277181+0000","last_clean":"2024-02-02T15:25:26.277181+0000","last_became_active":"2024-02-02T15:24:47.782640+0000","last_became_peered":"2024-02-02T15:24:47.782640+0000","last_unstale":"2024-02-02T15:25:26.277181+0000","last_undegraded":"2024-02-02T15:25:26.277181+0000","last_fullsized":"2024-02-02T15:25:26.277181+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T02:09:54.851418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.281037+0000","last_change":"2024-02-02T15:24:51.814799+0000","last_active":"2024-02-02T15:25:26.281037+0000","last_peered":"2024-02-02T15:25:26.281037+0000","last_clean":"2024-02-02T15:25:26.281037+0000","last_became_active":"2024-02-02T15:24:51.814263+0000","last_became_peered":"2024-02-02T15:24:51.814263+0000","last_unstale":"2024-02-02T15:25:26.281037+0000","last_undegraded":"2024-02-02T15:25:26.281037+0000","last_fullsized":"2024-02-02T15:25:26.281037+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:52:02.786496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"84'7","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.277357+0000","last_change":"2024-02-02T15:24:47.273204+0000","last_active":"2024-02-02T15:25:26.277357+0000","last_peered":"2024-02-02T15:25:26.277357+0000","last_clean":"2024-02-02T15:25:26.277357+0000","last_became_active":"2024-02-02T15:24:47.272888+0000","last_became_peered":"2024-02-02T15:24:47.272888+0000","last_unstale":"2024-02-02T15:25:26.277357+0000","last_undegraded":"2024-02-02T15:25:26.277357+0000","last_fullsized":"2024-02-02T15:25:26.277357+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:26:25.188915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.272783+0000","last_change":"2024-02-02T15:24:47.788645+0000","last_active":"2024-02-02T15:25:26.272783+0000","last_peered":"2024-02-02T15:25:26.272783+0000","last_clean":"2024-02-02T15:25:26.272783+0000","last_became_active":"2024-02-02T15:24:47.787478+0000","last_became_peered":"2024-02-02T15:24:47.787478+0000","last_unstale":"2024-02-02T15:25:26.272783+0000","last_undegraded":"2024-02-02T15:25:26.272783+0000","last_fullsized":"2024-02-02T15:25:26.272783+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:24:45.327030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.275632+0000","last_change":"2024-02-02T15:24:51.822681+0000","last_active":"2024-02-02T15:25:26.275632+0000","last_peered":"2024-02-02T15:25:26.275632+0000","last_clean":"2024-02-02T15:25:26.275632+0000","last_became_active":"2024-02-02T15:24:51.822202+0000","last_became_peered":"2024-02-02T15:24:51.822202+0000","last_unstale":"2024-02-02T15:25:26.275632+0000","last_undegraded":"2024-02-02T15:25:26.275632+0000","last_fullsized":"2024-02-02T15:25:26.275632+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:05:43.751558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"84'3","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.271740+0000","last_change":"2024-02-02T15:24:47.271095+0000","last_active":"2024-02-02T15:25:26.271740+0000","last_peered":"2024-02-02T15:25:26.271740+0000","last_clean":"2024-02-02T15:25:26.271740+0000","last_became_active":"2024-02-02T15:24:47.270696+0000","last_became_peered":"2024-02-02T15:24:47.270696+0000","last_unstale":"2024-02-02T15:25:26.271740+0000","last_undegraded":"2024-02-02T15:25:26.271740+0000","last_fullsized":"2024-02-02T15:25:26.271740+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T02:20:59.370217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.276719+0000","last_change":"2024-02-02T15:24:47.791699+0000","last_active":"2024-02-02T15:25:26.276719+0000","last_peered":"2024-02-02T15:25:26.276719+0000","last_clean":"2024-02-02T15:25:26.276719+0000","last_became_active":"2024-02-02T15:24:47.787280+0000","last_became_peered":"2024-02-02T15:24:47.787280+0000","last_unstale":"2024-02-02T15:25:26.276719+0000","last_undegraded":"2024-02-02T15:25:26.276719+0000","last_fullsized":"2024-02-02T15:25:26.276719+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:18:10.316393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.280531+0000","last_change":"2024-02-02T15:24:51.810050+0000","last_active":"2024-02-02T15:25:26.280531+0000","last_peered":"2024-02-02T15:25:26.280531+0000","last_clean":"2024-02-02T15:25:26.280531+0000","last_became_active":"2024-02-02T15:24:51.809621+0000","last_became_peered":"2024-02-02T15:24:51.809621+0000","last_unstale":"2024-02-02T15:25:26.280531+0000","last_undegraded":"2024-02-02T15:25:26.280531+0000","last_fullsized":"2024-02-02T15:25:26.280531+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:52:35.885973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"84'3","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.272677+0000","last_change":"2024-02-02T15:24:47.267973+0000","last_active":"2024-02-02T15:25:26.272677+0000","last_peered":"2024-02-02T15:25:26.272677+0000","last_clean":"2024-02-02T15:25:26.272677+0000","last_became_active":"2024-02-02T15:24:47.267649+0000","last_became_peered":"2024-02-02T15:24:47.267649+0000","last_unstale":"2024-02-02T15:25:26.272677+0000","last_undegraded":"2024-02-02T15:25:26.272677+0000","last_fullsized":"2024-02-02T15:25:26.272677+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T02:46:56.844570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.281458+0000","last_change":"2024-02-02T15:24:47.798636+0000","last_active":"2024-02-02T15:25:26.281458+0000","last_peered":"2024-02-02T15:25:26.281458+0000","last_clean":"2024-02-02T15:25:26.281458+0000","last_became_active":"2024-02-02T15:24:47.798328+0000","last_became_peered":"2024-02-02T15:24:47.798328+0000","last_unstale":"2024-02-02T15:25:26.281458+0000","last_undegraded":"2024-02-02T15:25:26.281458+0000","last_fullsized":"2024-02-02T15:25:26.281458+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:37:43.687336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.277684+0000","last_change":"2024-02-02T15:24:51.811312+0000","last_active":"2024-02-02T15:25:26.277684+0000","last_peered":"2024-02-02T15:25:26.277684+0000","last_clean":"2024-02-02T15:25:26.277684+0000","last_became_active":"2024-02-02T15:24:51.810932+0000","last_became_peered":"2024-02-02T15:24:51.810932+0000","last_unstale":"2024-02-02T15:25:26.277684+0000","last_undegraded":"2024-02-02T15:25:26.277684+0000","last_fullsized":"2024-02-02T15:25:26.277684+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:10:31.530518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"84'4","reported_seq":94,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T15:25:55.312231+0000","last_change":"2024-02-02T15:24:47.271483+0000","last_active":"2024-02-02T15:25:55.312231+0000","last_peered":"2024-02-02T15:25:55.312231+0000","last_clean":"2024-02-02T15:25:55.312231+0000","last_became_active":"2024-02-02T15:24:47.271160+0000","last_became_peered":"2024-02-02T15:24:47.271160+0000","last_unstale":"2024-02-02T15:25:55.312231+0000","last_undegraded":"2024-02-02T15:25:55.312231+0000","last_fullsized":"2024-02-02T15:25:55.312231+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:29:20.135351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":72,"num_read_kb":72,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.273506+0000","last_change":"2024-02-02T15:24:47.786492+0000","last_active":"2024-02-02T15:25:26.273506+0000","last_peered":"2024-02-02T15:25:26.273506+0000","last_clean":"2024-02-02T15:25:26.273506+0000","last_became_active":"2024-02-02T15:24:47.786103+0000","last_became_peered":"2024-02-02T15:24:47.786103+0000","last_unstale":"2024-02-02T15:25:26.273506+0000","last_undegraded":"2024-02-02T15:25:26.273506+0000","last_fullsized":"2024-02-02T15:25:26.273506+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:24:23.666596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.280381+0000","last_change":"2024-02-02T15:24:51.819941+0000","last_active":"2024-02-02T15:25:26.280381+0000","last_peered":"2024-02-02T15:25:26.280381+0000","last_clean":"2024-02-02T15:25:26.280381+0000","last_became_active":"2024-02-02T15:24:51.818066+0000","last_became_peered":"2024-02-02T15:24:51.818066+0000","last_unstale":"2024-02-02T15:25:26.280381+0000","last_undegraded":"2024-02-02T15:25:26.280381+0000","last_fullsized":"2024-02-02T15:25:26.280381+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:55:51.550988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"84'4","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.276022+0000","last_change":"2024-02-02T15:24:47.268541+0000","last_active":"2024-02-02T15:25:26.276022+0000","last_peered":"2024-02-02T15:25:26.276022+0000","last_clean":"2024-02-02T15:25:26.276022+0000","last_became_active":"2024-02-02T15:24:47.268184+0000","last_became_peered":"2024-02-02T15:24:47.268184+0000","last_unstale":"2024-02-02T15:25:26.276022+0000","last_undegraded":"2024-02-02T15:25:26.276022+0000","last_fullsized":"2024-02-02T15:25:26.276022+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:30:12.187297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.280697+0000","last_change":"2024-02-02T15:24:47.787792+0000","last_active":"2024-02-02T15:25:26.280697+0000","last_peered":"2024-02-02T15:25:26.280697+0000","last_clean":"2024-02-02T15:25:26.280697+0000","last_became_active":"2024-02-02T15:24:47.787479+0000","last_became_peered":"2024-02-02T15:24:47.787479+0000","last_unstale":"2024-02-02T15:25:26.280697+0000","last_undegraded":"2024-02-02T15:25:26.280697+0000","last_fullsized":"2024-02-02T15:25:26.280697+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T03:24:06.544232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.276031+0000","last_change":"2024-02-02T15:24:51.826639+0000","last_active":"2024-02-02T15:25:26.276031+0000","last_peered":"2024-02-02T15:25:26.276031+0000","last_clean":"2024-02-02T15:25:26.276031+0000","last_became_active":"2024-02-02T15:24:51.825089+0000","last_became_peered":"2024-02-02T15:24:51.825089+0000","last_unstale":"2024-02-02T15:25:26.276031+0000","last_undegraded":"2024-02-02T15:25:26.276031+0000","last_fullsized":"2024-02-02T15:25:26.276031+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:41:44.592603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.272658+0000","last_change":"2024-02-02T15:24:47.281490+0000","last_active":"2024-02-02T15:25:26.272658+0000","last_peered":"2024-02-02T15:25:26.272658+0000","last_clean":"2024-02-02T15:25:26.272658+0000","last_became_active":"2024-02-02T15:24:47.281137+0000","last_became_peered":"2024-02-02T15:24:47.281137+0000","last_unstale":"2024-02-02T15:25:26.272658+0000","last_undegraded":"2024-02-02T15:25:26.272658+0000","last_fullsized":"2024-02-02T15:25:26.272658+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:42:08.508881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.273131+0000","last_change":"2024-02-02T15:24:47.781135+0000","last_active":"2024-02-02T15:25:26.273131+0000","last_peered":"2024-02-02T15:25:26.273131+0000","last_clean":"2024-02-02T15:25:26.273131+0000","last_became_active":"2024-02-02T15:24:47.780817+0000","last_became_peered":"2024-02-02T15:24:47.780817+0000","last_unstale":"2024-02-02T15:25:26.273131+0000","last_undegraded":"2024-02-02T15:25:26.273131+0000","last_fullsized":"2024-02-02T15:25:26.273131+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:33:05.862934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T15:25:27.286403+0000","last_change":"2024-02-02T15:25:27.286403+0000","last_active":"2024-02-02T15:25:27.286403+0000","last_peered":"2024-02-02T15:25:27.286403+0000","last_clean":"2024-02-02T15:25:27.286403+0000","last_became_active":"2024-02-02T15:25:27.285824+0000","last_became_peered":"2024-02-02T15:25:27.285824+0000","last_unstale":"2024-02-02T15:25:27.286403+0000","last_undegraded":"2024-02-02T15:25:27.286403+0000","last_fullsized":"2024-02-02T15:25:27.286403+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:11:25.986947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,10],"acting":[9,8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"86'35","reported_seq":133,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T15:25:55.321807+0000","last_change":"2024-02-02T15:24:47.270604+0000","last_active":"2024-02-02T15:25:55.321807+0000","last_peered":"2024-02-02T15:25:55.321807+0000","last_clean":"2024-02-02T15:25:55.321807+0000","last_became_active":"2024-02-02T15:24:47.270261+0000","last_became_peered":"2024-02-02T15:24:47.270261+0000","last_unstale":"2024-02-02T15:25:55.321807+0000","last_undegraded":"2024-02-02T15:25:55.321807+0000","last_fullsized":"2024-02-02T15:25:55.321807+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:56:33.767423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60396,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":80,"num_read_kb":100,"num_write":37,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.278171+0000","last_change":"2024-02-02T15:24:47.789663+0000","last_active":"2024-02-02T15:25:26.278171+0000","last_peered":"2024-02-02T15:25:26.278171+0000","last_clean":"2024-02-02T15:25:26.278171+0000","last_became_active":"2024-02-02T15:24:47.789352+0000","last_became_peered":"2024-02-02T15:24:47.789352+0000","last_unstale":"2024-02-02T15:25:26.278171+0000","last_undegraded":"2024-02-02T15:25:26.278171+0000","last_fullsized":"2024-02-02T15:25:26.278171+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:13:37.991350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.273255+0000","last_change":"2024-02-02T15:24:51.816021+0000","last_active":"2024-02-02T15:25:26.273255+0000","last_peered":"2024-02-02T15:25:26.273255+0000","last_clean":"2024-02-02T15:25:26.273255+0000","last_became_active":"2024-02-02T15:24:51.815698+0000","last_became_peered":"2024-02-02T15:24:51.815698+0000","last_unstale":"2024-02-02T15:25:26.273255+0000","last_undegraded":"2024-02-02T15:25:26.273255+0000","last_fullsized":"2024-02-02T15:25:26.273255+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:11:10.670195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"84'9","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T15:25:27.295680+0000","last_change":"2024-02-02T15:25:27.295339+0000","last_active":"2024-02-02T15:25:27.295680+0000","last_peered":"2024-02-02T15:25:27.295680+0000","last_clean":"2024-02-02T15:25:27.295680+0000","last_became_active":"2024-02-02T15:25:27.281605+0000","last_became_peered":"2024-02-02T15:25:27.281605+0000","last_unstale":"2024-02-02T15:25:27.295680+0000","last_undegraded":"2024-02-02T15:25:27.295680+0000","last_fullsized":"2024-02-02T15:25:27.295680+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T02:35:36.952261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.271927+0000","last_change":"2024-02-02T15:24:47.798089+0000","last_active":"2024-02-02T15:25:26.271927+0000","last_peered":"2024-02-02T15:25:26.271927+0000","last_clean":"2024-02-02T15:25:26.271927+0000","last_became_active":"2024-02-02T15:24:47.793914+0000","last_became_peered":"2024-02-02T15:24:47.793914+0000","last_unstale":"2024-02-02T15:25:26.271927+0000","last_undegraded":"2024-02-02T15:25:26.271927+0000","last_fullsized":"2024-02-02T15:25:26.271927+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:33:39.927550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.276733+0000","last_change":"2024-02-02T15:24:51.823098+0000","last_active":"2024-02-02T15:25:26.276733+0000","last_peered":"2024-02-02T15:25:26.276733+0000","last_clean":"2024-02-02T15:25:26.276733+0000","last_became_active":"2024-02-02T15:24:51.822634+0000","last_became_peered":"2024-02-02T15:24:51.822634+0000","last_unstale":"2024-02-02T15:25:26.276733+0000","last_undegraded":"2024-02-02T15:25:26.276733+0000","last_fullsized":"2024-02-02T15:25:26.276733+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T03:18:07.420196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.272888+0000","last_change":"2024-02-02T15:24:47.268962+0000","last_active":"2024-02-02T15:25:26.272888+0000","last_peered":"2024-02-02T15:25:26.272888+0000","last_clean":"2024-02-02T15:25:26.272888+0000","last_became_active":"2024-02-02T15:24:47.268607+0000","last_became_peered":"2024-02-02T15:24:47.268607+0000","last_unstale":"2024-02-02T15:25:26.272888+0000","last_undegraded":"2024-02-02T15:25:26.272888+0000","last_fullsized":"2024-02-02T15:25:26.272888+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:31:06.330437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.275646+0000","last_change":"2024-02-02T15:24:47.792205+0000","last_active":"2024-02-02T15:25:26.275646+0000","last_peered":"2024-02-02T15:25:26.275646+0000","last_clean":"2024-02-02T15:25:26.275646+0000","last_became_active":"2024-02-02T15:24:47.791874+0000","last_became_peered":"2024-02-02T15:24:47.791874+0000","last_unstale":"2024-02-02T15:25:26.275646+0000","last_undegraded":"2024-02-02T15:25:26.275646+0000","last_fullsized":"2024-02-02T15:25:26.275646+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:33:38.795921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.282390+0000","last_change":"2024-02-02T15:24:51.825499+0000","last_active":"2024-02-02T15:25:26.282390+0000","last_peered":"2024-02-02T15:25:26.282390+0000","last_clean":"2024-02-02T15:25:26.282390+0000","last_became_active":"2024-02-02T15:24:51.825049+0000","last_became_peered":"2024-02-02T15:24:51.825049+0000","last_unstale":"2024-02-02T15:25:26.282390+0000","last_undegraded":"2024-02-02T15:25:26.282390+0000","last_fullsized":"2024-02-02T15:25:26.282390+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:41:05.430891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"84'6","reported_seq":66,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T15:25:58.323282+0000","last_change":"2024-02-02T15:24:47.267683+0000","last_active":"2024-02-02T15:25:58.323282+0000","last_peered":"2024-02-02T15:25:58.323282+0000","last_clean":"2024-02-02T15:25:58.323282+0000","last_became_active":"2024-02-02T15:24:47.267324+0000","last_became_peered":"2024-02-02T15:24:47.267324+0000","last_unstale":"2024-02-02T15:25:58.323282+0000","last_undegraded":"2024-02-02T15:25:58.323282+0000","last_fullsized":"2024-02-02T15:25:58.323282+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:03:31.326791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":932,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":43,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.277864+0000","last_change":"2024-02-02T15:24:47.792509+0000","last_active":"2024-02-02T15:25:26.277864+0000","last_peered":"2024-02-02T15:25:26.277864+0000","last_clean":"2024-02-02T15:25:26.277864+0000","last_became_active":"2024-02-02T15:24:47.791662+0000","last_became_peered":"2024-02-02T15:24:47.791662+0000","last_unstale":"2024-02-02T15:25:26.277864+0000","last_undegraded":"2024-02-02T15:25:26.277864+0000","last_fullsized":"2024-02-02T15:25:26.277864+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:42:50.009971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T15:25:27.282514+0000","last_change":"2024-02-02T15:25:27.282514+0000","last_active":"2024-02-02T15:25:27.282514+0000","last_peered":"2024-02-02T15:25:27.282514+0000","last_clean":"2024-02-02T15:25:27.282514+0000","last_became_active":"2024-02-02T15:25:27.281336+0000","last_became_peered":"2024-02-02T15:25:27.281336+0000","last_unstale":"2024-02-02T15:25:27.282514+0000","last_undegraded":"2024-02-02T15:25:27.282514+0000","last_fullsized":"2024-02-02T15:25:27.282514+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:50:39.728813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"84'17","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.276131+0000","last_change":"2024-02-02T15:24:47.274152+0000","last_active":"2024-02-02T15:25:26.276131+0000","last_peered":"2024-02-02T15:25:26.276131+0000","last_clean":"2024-02-02T15:25:26.276131+0000","last_became_active":"2024-02-02T15:24:47.273720+0000","last_became_peered":"2024-02-02T15:24:47.273720+0000","last_unstale":"2024-02-02T15:25:26.276131+0000","last_undegraded":"2024-02-02T15:25:26.276131+0000","last_fullsized":"2024-02-02T15:25:26.276131+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:38:05.280309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.272549+0000","last_change":"2024-02-02T15:24:47.790681+0000","last_active":"2024-02-02T15:25:26.272549+0000","last_peered":"2024-02-02T15:25:26.272549+0000","last_clean":"2024-02-02T15:25:26.272549+0000","last_became_active":"2024-02-02T15:24:47.789952+0000","last_became_peered":"2024-02-02T15:24:47.789952+0000","last_unstale":"2024-02-02T15:25:26.272549+0000","last_undegraded":"2024-02-02T15:25:26.272549+0000","last_fullsized":"2024-02-02T15:25:26.272549+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:21:03.103398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.272725+0000","last_change":"2024-02-02T15:24:51.812676+0000","last_active":"2024-02-02T15:25:26.272725+0000","last_peered":"2024-02-02T15:25:26.272725+0000","last_clean":"2024-02-02T15:25:26.272725+0000","last_became_active":"2024-02-02T15:24:51.812212+0000","last_became_peered":"2024-02-02T15:24:51.812212+0000","last_unstale":"2024-02-02T15:25:26.272725+0000","last_undegraded":"2024-02-02T15:25:26.272725+0000","last_fullsized":"2024-02-02T15:25:26.272725+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:48:01.944379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'2","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.281427+0000","last_change":"2024-02-02T15:24:47.271707+0000","last_active":"2024-02-02T15:25:26.281427+0000","last_peered":"2024-02-02T15:25:26.281427+0000","last_clean":"2024-02-02T15:25:26.281427+0000","last_became_active":"2024-02-02T15:24:47.271379+0000","last_became_peered":"2024-02-02T15:24:47.271379+0000","last_unstale":"2024-02-02T15:25:26.281427+0000","last_undegraded":"2024-02-02T15:25:26.281427+0000","last_fullsized":"2024-02-02T15:25:26.281427+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:00:24.571385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.276273+0000","last_change":"2024-02-02T15:24:47.788360+0000","last_active":"2024-02-02T15:25:26.276273+0000","last_peered":"2024-02-02T15:25:26.276273+0000","last_clean":"2024-02-02T15:25:26.276273+0000","last_became_active":"2024-02-02T15:24:47.787945+0000","last_became_peered":"2024-02-02T15:24:47.787945+0000","last_unstale":"2024-02-02T15:25:26.276273+0000","last_undegraded":"2024-02-02T15:25:26.276273+0000","last_fullsized":"2024-02-02T15:25:26.276273+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:37:54.531487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.277086+0000","last_change":"2024-02-02T15:24:51.821320+0000","last_active":"2024-02-02T15:25:26.277086+0000","last_peered":"2024-02-02T15:25:26.277086+0000","last_clean":"2024-02-02T15:25:26.277086+0000","last_became_active":"2024-02-02T15:24:51.820853+0000","last_became_peered":"2024-02-02T15:24:51.820853+0000","last_unstale":"2024-02-02T15:25:26.277086+0000","last_undegraded":"2024-02-02T15:25:26.277086+0000","last_fullsized":"2024-02-02T15:25:26.277086+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:20:12.304517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"84'1","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.282116+0000","last_change":"2024-02-02T15:24:47.278291+0000","last_active":"2024-02-02T15:25:26.282116+0000","last_peered":"2024-02-02T15:25:26.282116+0000","last_clean":"2024-02-02T15:25:26.282116+0000","last_became_active":"2024-02-02T15:24:47.276163+0000","last_became_peered":"2024-02-02T15:24:47.276163+0000","last_unstale":"2024-02-02T15:25:26.282116+0000","last_undegraded":"2024-02-02T15:25:26.282116+0000","last_fullsized":"2024-02-02T15:25:26.282116+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:53:50.312627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.277505+0000","last_change":"2024-02-02T15:24:47.791290+0000","last_active":"2024-02-02T15:25:26.277505+0000","last_peered":"2024-02-02T15:25:26.277505+0000","last_clean":"2024-02-02T15:25:26.277505+0000","last_became_active":"2024-02-02T15:24:47.790987+0000","last_became_peered":"2024-02-02T15:24:47.790987+0000","last_unstale":"2024-02-02T15:25:26.277505+0000","last_undegraded":"2024-02-02T15:25:26.277505+0000","last_fullsized":"2024-02-02T15:25:26.277505+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:38:08.803338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.271792+0000","last_change":"2024-02-02T15:24:51.813050+0000","last_active":"2024-02-02T15:25:26.271792+0000","last_peered":"2024-02-02T15:25:26.271792+0000","last_clean":"2024-02-02T15:25:26.271792+0000","last_became_active":"2024-02-02T15:24:51.812634+0000","last_became_peered":"2024-02-02T15:24:51.812634+0000","last_unstale":"2024-02-02T15:25:26.271792+0000","last_undegraded":"2024-02-02T15:25:26.271792+0000","last_fullsized":"2024-02-02T15:25:26.271792+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:06:56.428431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"84'5","reported_seq":60,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T15:25:54.329299+0000","last_change":"2024-02-02T15:24:47.274033+0000","last_active":"2024-02-02T15:25:54.329299+0000","last_peered":"2024-02-02T15:25:54.329299+0000","last_clean":"2024-02-02T15:25:54.329299+0000","last_became_active":"2024-02-02T15:24:47.273672+0000","last_became_peered":"2024-02-02T15:24:47.273672+0000","last_unstale":"2024-02-02T15:25:54.329299+0000","last_undegraded":"2024-02-02T15:25:54.329299+0000","last_fullsized":"2024-02-02T15:25:54.329299+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:51:46.926397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":37,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.277973+0000","last_change":"2024-02-02T15:24:47.786533+0000","last_active":"2024-02-02T15:25:26.277973+0000","last_peered":"2024-02-02T15:25:26.277973+0000","last_clean":"2024-02-02T15:25:26.277973+0000","last_became_active":"2024-02-02T15:24:47.785436+0000","last_became_peered":"2024-02-02T15:24:47.785436+0000","last_unstale":"2024-02-02T15:25:26.277973+0000","last_undegraded":"2024-02-02T15:25:26.277973+0000","last_fullsized":"2024-02-02T15:25:26.277973+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:23:45.910800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.272128+0000","last_change":"2024-02-02T15:24:51.814326+0000","last_active":"2024-02-02T15:25:26.272128+0000","last_peered":"2024-02-02T15:25:26.272128+0000","last_clean":"2024-02-02T15:25:26.272128+0000","last_became_active":"2024-02-02T15:24:51.813909+0000","last_became_peered":"2024-02-02T15:24:51.813909+0000","last_unstale":"2024-02-02T15:25:26.272128+0000","last_undegraded":"2024-02-02T15:25:26.272128+0000","last_fullsized":"2024-02-02T15:25:26.272128+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:16:12.902718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T15:25:27.287818+0000","last_change":"2024-02-02T15:25:27.287818+0000","last_active":"2024-02-02T15:25:27.287818+0000","last_peered":"2024-02-02T15:25:27.287818+0000","last_clean":"2024-02-02T15:25:27.287818+0000","last_became_active":"2024-02-02T15:25:27.287481+0000","last_became_peered":"2024-02-02T15:25:27.287481+0000","last_unstale":"2024-02-02T15:25:27.287818+0000","last_undegraded":"2024-02-02T15:25:27.287818+0000","last_fullsized":"2024-02-02T15:25:27.287818+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:14:32.820969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"84'5","reported_seq":62,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T15:25:56.317891+0000","last_change":"2024-02-02T15:24:47.274437+0000","last_active":"2024-02-02T15:25:56.317891+0000","last_peered":"2024-02-02T15:25:56.317891+0000","last_clean":"2024-02-02T15:25:56.317891+0000","last_became_active":"2024-02-02T15:24:47.274010+0000","last_became_peered":"2024-02-02T15:24:47.274010+0000","last_unstale":"2024-02-02T15:25:56.317891+0000","last_undegraded":"2024-02-02T15:25:56.317891+0000","last_fullsized":"2024-02-02T15:25:56.317891+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:10:36.862190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":39,"num_read_kb":39,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"84'1","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.275771+0000","last_change":"2024-02-02T15:24:51.825698+0000","last_active":"2024-02-02T15:25:26.275771+0000","last_peered":"2024-02-02T15:25:26.275771+0000","last_clean":"2024-02-02T15:25:26.275771+0000","last_became_active":"2024-02-02T15:24:51.825295+0000","last_became_peered":"2024-02-02T15:24:51.825295+0000","last_unstale":"2024-02-02T15:25:26.275771+0000","last_undegraded":"2024-02-02T15:25:26.275771+0000","last_fullsized":"2024-02-02T15:25:26.275771+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:17:09.838124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.276296+0000","last_change":"2024-02-02T15:24:47.781836+0000","last_active":"2024-02-02T15:25:26.276296+0000","last_peered":"2024-02-02T15:25:26.276296+0000","last_clean":"2024-02-02T15:25:26.276296+0000","last_became_active":"2024-02-02T15:24:47.781472+0000","last_became_peered":"2024-02-02T15:24:47.781472+0000","last_unstale":"2024-02-02T15:25:26.276296+0000","last_undegraded":"2024-02-02T15:25:26.276296+0000","last_fullsized":"2024-02-02T15:25:26.276296+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:46:33.714191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"84'6","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.275947+0000","last_change":"2024-02-02T15:24:47.269087+0000","last_active":"2024-02-02T15:25:26.275947+0000","last_peered":"2024-02-02T15:25:26.275947+0000","last_clean":"2024-02-02T15:25:26.275947+0000","last_became_active":"2024-02-02T15:24:47.268743+0000","last_became_peered":"2024-02-02T15:24:47.268743+0000","last_unstale":"2024-02-02T15:25:26.275947+0000","last_undegraded":"2024-02-02T15:25:26.275947+0000","last_fullsized":"2024-02-02T15:25:26.275947+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:03:04.323079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":336,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.277648+0000","last_change":"2024-02-02T15:23:55.191735+0000","last_active":"2024-02-02T15:25:26.277648+0000","last_peered":"2024-02-02T15:25:26.277648+0000","last_clean":"2024-02-02T15:25:26.277648+0000","last_became_active":"2024-02-02T15:23:55.191119+0000","last_became_peered":"2024-02-02T15:23:55.191119+0000","last_unstale":"2024-02-02T15:25:26.277648+0000","last_undegraded":"2024-02-02T15:25:26.277648+0000","last_fullsized":"2024-02-02T15:25:26.277648+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:21:22.472631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:21:22.472631+0000","last_clean_scrub_stamp":"2024-02-02T15:21:22.472631+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:24:44.153461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.273124+0000","last_change":"2024-02-02T15:24:51.822842+0000","last_active":"2024-02-02T15:25:26.273124+0000","last_peered":"2024-02-02T15:25:26.273124+0000","last_clean":"2024-02-02T15:25:26.273124+0000","last_became_active":"2024-02-02T15:24:51.822372+0000","last_became_peered":"2024-02-02T15:24:51.822372+0000","last_unstale":"2024-02-02T15:25:26.273124+0000","last_undegraded":"2024-02-02T15:25:26.273124+0000","last_fullsized":"2024-02-02T15:25:26.273124+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:45:29.631422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.271614+0000","last_change":"2024-02-02T15:24:47.786507+0000","last_active":"2024-02-02T15:25:26.271614+0000","last_peered":"2024-02-02T15:25:26.271614+0000","last_clean":"2024-02-02T15:25:26.271614+0000","last_became_active":"2024-02-02T15:24:47.786125+0000","last_became_peered":"2024-02-02T15:24:47.786125+0000","last_unstale":"2024-02-02T15:25:26.271614+0000","last_undegraded":"2024-02-02T15:25:26.271614+0000","last_fullsized":"2024-02-02T15:25:26.271614+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:33:09.014258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"84'4","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T15:25:27.628074+0000","last_change":"2024-02-02T15:25:27.628074+0000","last_active":"2024-02-02T15:25:27.628074+0000","last_peered":"2024-02-02T15:25:27.628074+0000","last_clean":"2024-02-02T15:25:27.628074+0000","last_became_active":"2024-02-02T15:25:27.625103+0000","last_became_peered":"2024-02-02T15:25:27.625103+0000","last_unstale":"2024-02-02T15:25:27.628074+0000","last_undegraded":"2024-02-02T15:25:27.628074+0000","last_fullsized":"2024-02-02T15:25:27.628074+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:30:35.540456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.276587+0000","last_change":"2024-02-02T15:24:51.808979+0000","last_active":"2024-02-02T15:25:26.276587+0000","last_peered":"2024-02-02T15:25:26.276587+0000","last_clean":"2024-02-02T15:25:26.276587+0000","last_became_active":"2024-02-02T15:24:51.808557+0000","last_became_peered":"2024-02-02T15:24:51.808557+0000","last_unstale":"2024-02-02T15:25:26.276587+0000","last_undegraded":"2024-02-02T15:25:26.276587+0000","last_fullsized":"2024-02-02T15:25:26.276587+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T02:38:34.697137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"84'4","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.278273+0000","last_change":"2024-02-02T15:24:47.281396+0000","last_active":"2024-02-02T15:25:26.278273+0000","last_peered":"2024-02-02T15:25:26.278273+0000","last_clean":"2024-02-02T15:25:26.278273+0000","last_became_active":"2024-02-02T15:24:47.281036+0000","last_became_peered":"2024-02-02T15:24:47.281036+0000","last_unstale":"2024-02-02T15:25:26.278273+0000","last_undegraded":"2024-02-02T15:25:26.278273+0000","last_fullsized":"2024-02-02T15:25:26.278273+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:54:47.139151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.271949+0000","last_change":"2024-02-02T15:24:47.791057+0000","last_active":"2024-02-02T15:25:26.271949+0000","last_peered":"2024-02-02T15:25:26.271949+0000","last_clean":"2024-02-02T15:25:26.271949+0000","last_became_active":"2024-02-02T15:24:47.790647+0000","last_became_peered":"2024-02-02T15:24:47.790647+0000","last_unstale":"2024-02-02T15:25:26.271949+0000","last_undegraded":"2024-02-02T15:25:26.271949+0000","last_fullsized":"2024-02-02T15:25:26.271949+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T02:49:42.067700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.276636+0000","last_change":"2024-02-02T15:24:51.817792+0000","last_active":"2024-02-02T15:25:26.276636+0000","last_peered":"2024-02-02T15:25:26.276636+0000","last_clean":"2024-02-02T15:25:26.276636+0000","last_became_active":"2024-02-02T15:24:51.815800+0000","last_became_peered":"2024-02-02T15:24:51.815800+0000","last_unstale":"2024-02-02T15:25:26.276636+0000","last_undegraded":"2024-02-02T15:25:26.276636+0000","last_fullsized":"2024-02-02T15:25:26.276636+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:28:10.440251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"86'10","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T15:25:27.280528+0000","last_change":"2024-02-02T15:24:47.277098+0000","last_active":"2024-02-02T15:25:27.280528+0000","last_peered":"2024-02-02T15:25:27.280528+0000","last_clean":"2024-02-02T15:25:27.280528+0000","last_became_active":"2024-02-02T15:24:47.275867+0000","last_became_peered":"2024-02-02T15:24:47.275867+0000","last_unstale":"2024-02-02T15:25:27.280528+0000","last_undegraded":"2024-02-02T15:25:27.280528+0000","last_fullsized":"2024-02-02T15:25:27.280528+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:35:21.015918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.277243+0000","last_change":"2024-02-02T15:24:47.784873+0000","last_active":"2024-02-02T15:25:26.277243+0000","last_peered":"2024-02-02T15:25:26.277243+0000","last_clean":"2024-02-02T15:25:26.277243+0000","last_became_active":"2024-02-02T15:24:47.784550+0000","last_became_peered":"2024-02-02T15:24:47.784550+0000","last_unstale":"2024-02-02T15:25:26.277243+0000","last_undegraded":"2024-02-02T15:25:26.277243+0000","last_fullsized":"2024-02-02T15:25:26.277243+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:13:37.513754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.279176+0000","last_change":"2024-02-02T15:24:51.805131+0000","last_active":"2024-02-02T15:25:26.279176+0000","last_peered":"2024-02-02T15:25:26.279176+0000","last_clean":"2024-02-02T15:25:26.279176+0000","last_became_active":"2024-02-02T15:24:51.804705+0000","last_became_peered":"2024-02-02T15:24:51.804705+0000","last_unstale":"2024-02-02T15:25:26.279176+0000","last_undegraded":"2024-02-02T15:25:26.279176+0000","last_fullsized":"2024-02-02T15:25:26.279176+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T02:12:59.945050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.276174+0000","last_change":"2024-02-02T15:24:47.266469+0000","last_active":"2024-02-02T15:25:26.276174+0000","last_peered":"2024-02-02T15:25:26.276174+0000","last_clean":"2024-02-02T15:25:26.276174+0000","last_became_active":"2024-02-02T15:24:47.266094+0000","last_became_peered":"2024-02-02T15:24:47.266094+0000","last_unstale":"2024-02-02T15:25:26.276174+0000","last_undegraded":"2024-02-02T15:25:26.276174+0000","last_fullsized":"2024-02-02T15:25:26.276174+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:51:41.324698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.281014+0000","last_change":"2024-02-02T15:24:47.793051+0000","last_active":"2024-02-02T15:25:26.281014+0000","last_peered":"2024-02-02T15:25:26.281014+0000","last_clean":"2024-02-02T15:25:26.281014+0000","last_became_active":"2024-02-02T15:24:47.792645+0000","last_became_peered":"2024-02-02T15:24:47.792645+0000","last_unstale":"2024-02-02T15:25:26.281014+0000","last_undegraded":"2024-02-02T15:25:26.281014+0000","last_fullsized":"2024-02-02T15:25:26.281014+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:47:17.229407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T15:25:27.282505+0000","last_change":"2024-02-02T15:25:27.282505+0000","last_active":"2024-02-02T15:25:27.282505+0000","last_peered":"2024-02-02T15:25:27.282505+0000","last_clean":"2024-02-02T15:25:27.282505+0000","last_became_active":"2024-02-02T15:25:27.282039+0000","last_became_peered":"2024-02-02T15:25:27.282039+0000","last_unstale":"2024-02-02T15:25:27.282505+0000","last_undegraded":"2024-02-02T15:25:27.282505+0000","last_fullsized":"2024-02-02T15:25:27.282505+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:31:29.700848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"84'2","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.272816+0000","last_change":"2024-02-02T15:24:47.273716+0000","last_active":"2024-02-02T15:25:26.272816+0000","last_peered":"2024-02-02T15:25:26.272816+0000","last_clean":"2024-02-02T15:25:26.272816+0000","last_became_active":"2024-02-02T15:24:47.273438+0000","last_became_peered":"2024-02-02T15:24:47.273438+0000","last_unstale":"2024-02-02T15:25:26.272816+0000","last_undegraded":"2024-02-02T15:25:26.272816+0000","last_fullsized":"2024-02-02T15:25:26.272816+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:52:16.498119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.275707+0000","last_change":"2024-02-02T15:24:47.782950+0000","last_active":"2024-02-02T15:25:26.275707+0000","last_peered":"2024-02-02T15:25:26.275707+0000","last_clean":"2024-02-02T15:25:26.275707+0000","last_became_active":"2024-02-02T15:24:47.782605+0000","last_became_peered":"2024-02-02T15:24:47.782605+0000","last_unstale":"2024-02-02T15:25:26.275707+0000","last_undegraded":"2024-02-02T15:25:26.275707+0000","last_fullsized":"2024-02-02T15:25:26.275707+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:57:34.125985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T15:25:27.288805+0000","last_change":"2024-02-02T15:25:27.288179+0000","last_active":"2024-02-02T15:25:27.288805+0000","last_peered":"2024-02-02T15:25:27.288805+0000","last_clean":"2024-02-02T15:25:27.288805+0000","last_became_active":"2024-02-02T15:25:27.287753+0000","last_became_peered":"2024-02-02T15:25:27.287753+0000","last_unstale":"2024-02-02T15:25:27.288805+0000","last_undegraded":"2024-02-02T15:25:27.288805+0000","last_fullsized":"2024-02-02T15:25:27.288805+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:02:46.854950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"84'7","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.278283+0000","last_change":"2024-02-02T15:24:47.278478+0000","last_active":"2024-02-02T15:25:26.278283+0000","last_peered":"2024-02-02T15:25:26.278283+0000","last_clean":"2024-02-02T15:25:26.278283+0000","last_became_active":"2024-02-02T15:24:47.278084+0000","last_became_peered":"2024-02-02T15:24:47.278084+0000","last_unstale":"2024-02-02T15:25:26.278283+0000","last_undegraded":"2024-02-02T15:25:26.278283+0000","last_fullsized":"2024-02-02T15:25:26.278283+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:10:23.044968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.276242+0000","last_change":"2024-02-02T15:24:47.795291+0000","last_active":"2024-02-02T15:25:26.276242+0000","last_peered":"2024-02-02T15:25:26.276242+0000","last_clean":"2024-02-02T15:25:26.276242+0000","last_became_active":"2024-02-02T15:24:47.794514+0000","last_became_peered":"2024-02-02T15:24:47.794514+0000","last_unstale":"2024-02-02T15:25:26.276242+0000","last_undegraded":"2024-02-02T15:25:26.276242+0000","last_fullsized":"2024-02-02T15:25:26.276242+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:32:12.722258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"84'1","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.275640+0000","last_change":"2024-02-02T15:24:51.825985+0000","last_active":"2024-02-02T15:25:26.275640+0000","last_peered":"2024-02-02T15:25:26.275640+0000","last_clean":"2024-02-02T15:25:26.275640+0000","last_became_active":"2024-02-02T15:24:51.825588+0000","last_became_peered":"2024-02-02T15:24:51.825588+0000","last_unstale":"2024-02-02T15:25:26.275640+0000","last_undegraded":"2024-02-02T15:25:26.275640+0000","last_fullsized":"2024-02-02T15:25:26.275640+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:32:39.910258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"84'3","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T15:25:27.298932+0000","last_change":"2024-02-02T15:25:27.298553+0000","last_active":"2024-02-02T15:25:27.298932+0000","last_peered":"2024-02-02T15:25:27.298932+0000","last_clean":"2024-02-02T15:25:27.298932+0000","last_became_active":"2024-02-02T15:25:27.282469+0000","last_became_peered":"2024-02-02T15:25:27.282469+0000","last_unstale":"2024-02-02T15:25:27.298932+0000","last_undegraded":"2024-02-02T15:25:27.298932+0000","last_fullsized":"2024-02-02T15:25:27.298932+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:54:14.740951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.276116+0000","last_change":"2024-02-02T15:24:47.784188+0000","last_active":"2024-02-02T15:25:26.276116+0000","last_peered":"2024-02-02T15:25:26.276116+0000","last_clean":"2024-02-02T15:25:26.276116+0000","last_became_active":"2024-02-02T15:24:47.783872+0000","last_became_peered":"2024-02-02T15:24:47.783872+0000","last_unstale":"2024-02-02T15:25:26.276116+0000","last_undegraded":"2024-02-02T15:25:26.276116+0000","last_fullsized":"2024-02-02T15:25:26.276116+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:45:45.213255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.275577+0000","last_change":"2024-02-02T15:24:51.815263+0000","last_active":"2024-02-02T15:25:26.275577+0000","last_peered":"2024-02-02T15:25:26.275577+0000","last_clean":"2024-02-02T15:25:26.275577+0000","last_became_active":"2024-02-02T15:24:51.814849+0000","last_became_peered":"2024-02-02T15:24:51.814849+0000","last_unstale":"2024-02-02T15:25:26.275577+0000","last_undegraded":"2024-02-02T15:25:26.275577+0000","last_fullsized":"2024-02-02T15:25:26.275577+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:05:32.987359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"84'4","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.277429+0000","last_change":"2024-02-02T15:24:47.270534+0000","last_active":"2024-02-02T15:25:26.277429+0000","last_peered":"2024-02-02T15:25:26.277429+0000","last_clean":"2024-02-02T15:25:26.277429+0000","last_became_active":"2024-02-02T15:24:47.270110+0000","last_became_peered":"2024-02-02T15:24:47.270110+0000","last_unstale":"2024-02-02T15:25:26.277429+0000","last_undegraded":"2024-02-02T15:25:26.277429+0000","last_fullsized":"2024-02-02T15:25:26.277429+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T02:59:52.297315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.275557+0000","last_change":"2024-02-02T15:24:47.791826+0000","last_active":"2024-02-02T15:25:26.275557+0000","last_peered":"2024-02-02T15:25:26.275557+0000","last_clean":"2024-02-02T15:25:26.275557+0000","last_became_active":"2024-02-02T15:24:47.791016+0000","last_became_peered":"2024-02-02T15:24:47.791016+0000","last_unstale":"2024-02-02T15:25:26.275557+0000","last_undegraded":"2024-02-02T15:25:26.275557+0000","last_fullsized":"2024-02-02T15:25:26.275557+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:47:14.296876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.272974+0000","last_change":"2024-02-02T15:24:51.806741+0000","last_active":"2024-02-02T15:25:26.272974+0000","last_peered":"2024-02-02T15:25:26.272974+0000","last_clean":"2024-02-02T15:25:26.272974+0000","last_became_active":"2024-02-02T15:24:51.806085+0000","last_became_peered":"2024-02-02T15:24:51.806085+0000","last_unstale":"2024-02-02T15:25:26.272974+0000","last_undegraded":"2024-02-02T15:25:26.272974+0000","last_fullsized":"2024-02-02T15:25:26.272974+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:53:39.944222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"84'5","reported_seq":62,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T15:25:55.326324+0000","last_change":"2024-02-02T15:24:47.278515+0000","last_active":"2024-02-02T15:25:55.326324+0000","last_peered":"2024-02-02T15:25:55.326324+0000","last_clean":"2024-02-02T15:25:55.326324+0000","last_became_active":"2024-02-02T15:24:47.278145+0000","last_became_peered":"2024-02-02T15:24:47.278145+0000","last_unstale":"2024-02-02T15:25:55.326324+0000","last_undegraded":"2024-02-02T15:25:55.326324+0000","last_fullsized":"2024-02-02T15:25:55.326324+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:48:55.118003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":39,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.275926+0000","last_change":"2024-02-02T15:24:47.791996+0000","last_active":"2024-02-02T15:25:26.275926+0000","last_peered":"2024-02-02T15:25:26.275926+0000","last_clean":"2024-02-02T15:25:26.275926+0000","last_became_active":"2024-02-02T15:24:47.791624+0000","last_became_peered":"2024-02-02T15:24:47.791624+0000","last_unstale":"2024-02-02T15:25:26.275926+0000","last_undegraded":"2024-02-02T15:25:26.275926+0000","last_fullsized":"2024-02-02T15:25:26.275926+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:33:27.963740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.279130+0000","last_change":"2024-02-02T15:24:51.826835+0000","last_active":"2024-02-02T15:25:26.279130+0000","last_peered":"2024-02-02T15:25:26.279130+0000","last_clean":"2024-02-02T15:25:26.279130+0000","last_became_active":"2024-02-02T15:24:51.826310+0000","last_became_peered":"2024-02-02T15:24:51.826310+0000","last_unstale":"2024-02-02T15:25:26.279130+0000","last_undegraded":"2024-02-02T15:25:26.279130+0000","last_fullsized":"2024-02-02T15:25:26.279130+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:48:16.687048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'3","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.272976+0000","last_change":"2024-02-02T15:24:47.268865+0000","last_active":"2024-02-02T15:25:26.272976+0000","last_peered":"2024-02-02T15:25:26.272976+0000","last_clean":"2024-02-02T15:25:26.272976+0000","last_became_active":"2024-02-02T15:24:47.268523+0000","last_became_peered":"2024-02-02T15:24:47.268523+0000","last_unstale":"2024-02-02T15:25:26.272976+0000","last_undegraded":"2024-02-02T15:25:26.272976+0000","last_fullsized":"2024-02-02T15:25:26.272976+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:45:16.957512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.275647+0000","last_change":"2024-02-02T15:24:47.792601+0000","last_active":"2024-02-02T15:25:26.275647+0000","last_peered":"2024-02-02T15:25:26.275647+0000","last_clean":"2024-02-02T15:25:26.275647+0000","last_became_active":"2024-02-02T15:24:47.792240+0000","last_became_peered":"2024-02-02T15:24:47.792240+0000","last_unstale":"2024-02-02T15:25:26.275647+0000","last_undegraded":"2024-02-02T15:25:26.275647+0000","last_fullsized":"2024-02-02T15:25:26.275647+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:51:25.205859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.272255+0000","last_change":"2024-02-02T15:24:51.806267+0000","last_active":"2024-02-02T15:25:26.272255+0000","last_peered":"2024-02-02T15:25:26.272255+0000","last_clean":"2024-02-02T15:25:26.272255+0000","last_became_active":"2024-02-02T15:24:51.805877+0000","last_became_peered":"2024-02-02T15:24:51.805877+0000","last_unstale":"2024-02-02T15:25:26.272255+0000","last_undegraded":"2024-02-02T15:25:26.272255+0000","last_fullsized":"2024-02-02T15:25:26.272255+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:00:47.747377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'5","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.272379+0000","last_change":"2024-02-02T15:24:47.271057+0000","last_active":"2024-02-02T15:25:26.272379+0000","last_peered":"2024-02-02T15:25:26.272379+0000","last_clean":"2024-02-02T15:25:26.272379+0000","last_became_active":"2024-02-02T15:24:47.270737+0000","last_became_peered":"2024-02-02T15:24:47.270737+0000","last_unstale":"2024-02-02T15:25:26.272379+0000","last_undegraded":"2024-02-02T15:25:26.272379+0000","last_fullsized":"2024-02-02T15:25:26.272379+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:17:00.212426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.276183+0000","last_change":"2024-02-02T15:24:47.794108+0000","last_active":"2024-02-02T15:25:26.276183+0000","last_peered":"2024-02-02T15:25:26.276183+0000","last_clean":"2024-02-02T15:25:26.276183+0000","last_became_active":"2024-02-02T15:24:47.793890+0000","last_became_peered":"2024-02-02T15:24:47.793890+0000","last_unstale":"2024-02-02T15:25:26.276183+0000","last_undegraded":"2024-02-02T15:25:26.276183+0000","last_fullsized":"2024-02-02T15:25:26.276183+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T03:05:51.704385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.276351+0000","last_change":"2024-02-02T15:24:47.796234+0000","last_active":"2024-02-02T15:25:26.276351+0000","last_peered":"2024-02-02T15:25:26.276351+0000","last_clean":"2024-02-02T15:25:26.276351+0000","last_became_active":"2024-02-02T15:24:47.795474+0000","last_became_peered":"2024-02-02T15:24:47.795474+0000","last_unstale":"2024-02-02T15:25:26.276351+0000","last_undegraded":"2024-02-02T15:25:26.276351+0000","last_fullsized":"2024-02-02T15:25:26.276351+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:25:28.059500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.279417+0000","last_change":"2024-02-02T15:24:47.786930+0000","last_active":"2024-02-02T15:25:26.279417+0000","last_peered":"2024-02-02T15:25:26.279417+0000","last_clean":"2024-02-02T15:25:26.279417+0000","last_became_active":"2024-02-02T15:24:47.786615+0000","last_became_peered":"2024-02-02T15:24:47.786615+0000","last_unstale":"2024-02-02T15:25:26.279417+0000","last_undegraded":"2024-02-02T15:25:26.279417+0000","last_fullsized":"2024-02-02T15:25:26.279417+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:17:23.231511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.279172+0000","last_change":"2024-02-02T15:24:47.797877+0000","last_active":"2024-02-02T15:25:26.279172+0000","last_peered":"2024-02-02T15:25:26.279172+0000","last_clean":"2024-02-02T15:25:26.279172+0000","last_became_active":"2024-02-02T15:24:47.797559+0000","last_became_peered":"2024-02-02T15:24:47.797559+0000","last_unstale":"2024-02-02T15:25:26.279172+0000","last_undegraded":"2024-02-02T15:25:26.279172+0000","last_fullsized":"2024-02-02T15:25:26.279172+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:13:55.529796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.279344+0000","last_change":"2024-02-02T15:24:47.790270+0000","last_active":"2024-02-02T15:25:26.279344+0000","last_peered":"2024-02-02T15:25:26.279344+0000","last_clean":"2024-02-02T15:25:26.279344+0000","last_became_active":"2024-02-02T15:24:47.789928+0000","last_became_peered":"2024-02-02T15:24:47.789928+0000","last_unstale":"2024-02-02T15:25:26.279344+0000","last_undegraded":"2024-02-02T15:25:26.279344+0000","last_fullsized":"2024-02-02T15:25:26.279344+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T02:34:44.885803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T15:25:27.298415+0000","last_change":"2024-02-02T15:25:27.298415+0000","last_active":"2024-02-02T15:25:27.298415+0000","last_peered":"2024-02-02T15:25:27.298415+0000","last_clean":"2024-02-02T15:25:27.298415+0000","last_became_active":"2024-02-02T15:25:27.297990+0000","last_became_peered":"2024-02-02T15:25:27.297990+0000","last_unstale":"2024-02-02T15:25:27.298415+0000","last_undegraded":"2024-02-02T15:25:27.298415+0000","last_fullsized":"2024-02-02T15:25:27.298415+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:38:02.040613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.277083+0000","last_change":"2024-02-02T15:24:47.797861+0000","last_active":"2024-02-02T15:25:26.277083+0000","last_peered":"2024-02-02T15:25:26.277083+0000","last_clean":"2024-02-02T15:25:26.277083+0000","last_became_active":"2024-02-02T15:24:47.797543+0000","last_became_peered":"2024-02-02T15:24:47.797543+0000","last_unstale":"2024-02-02T15:25:26.277083+0000","last_undegraded":"2024-02-02T15:25:26.277083+0000","last_fullsized":"2024-02-02T15:25:26.277083+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:09:37.710909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.278967+0000","last_change":"2024-02-02T15:24:47.796221+0000","last_active":"2024-02-02T15:25:26.278967+0000","last_peered":"2024-02-02T15:25:26.278967+0000","last_clean":"2024-02-02T15:25:26.278967+0000","last_became_active":"2024-02-02T15:24:47.795845+0000","last_became_peered":"2024-02-02T15:24:47.795845+0000","last_unstale":"2024-02-02T15:25:26.278967+0000","last_undegraded":"2024-02-02T15:25:26.278967+0000","last_fullsized":"2024-02-02T15:25:26.278967+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:54:55.637660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.277498+0000","last_change":"2024-02-02T15:24:47.783086+0000","last_active":"2024-02-02T15:25:26.277498+0000","last_peered":"2024-02-02T15:25:26.277498+0000","last_clean":"2024-02-02T15:25:26.277498+0000","last_became_active":"2024-02-02T15:24:47.781826+0000","last_became_peered":"2024-02-02T15:24:47.781826+0000","last_unstale":"2024-02-02T15:25:26.277498+0000","last_undegraded":"2024-02-02T15:25:26.277498+0000","last_fullsized":"2024-02-02T15:25:26.277498+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:32:05.623591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T15:25:27.285214+0000","last_change":"2024-02-02T15:25:27.284461+0000","last_active":"2024-02-02T15:25:27.285214+0000","last_peered":"2024-02-02T15:25:27.285214+0000","last_clean":"2024-02-02T15:25:27.285214+0000","last_became_active":"2024-02-02T15:25:27.284080+0000","last_became_peered":"2024-02-02T15:25:27.284080+0000","last_unstale":"2024-02-02T15:25:27.285214+0000","last_undegraded":"2024-02-02T15:25:27.285214+0000","last_fullsized":"2024-02-02T15:25:27.285214+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:43:40.124825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.279045+0000","last_change":"2024-02-02T15:24:47.789905+0000","last_active":"2024-02-02T15:25:26.279045+0000","last_peered":"2024-02-02T15:25:26.279045+0000","last_clean":"2024-02-02T15:25:26.279045+0000","last_became_active":"2024-02-02T15:24:47.788903+0000","last_became_peered":"2024-02-02T15:24:47.788903+0000","last_unstale":"2024-02-02T15:25:26.279045+0000","last_undegraded":"2024-02-02T15:25:26.279045+0000","last_fullsized":"2024-02-02T15:25:26.279045+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T02:49:46.118232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.273118+0000","last_change":"2024-02-02T15:24:47.795928+0000","last_active":"2024-02-02T15:25:26.273118+0000","last_peered":"2024-02-02T15:25:26.273118+0000","last_clean":"2024-02-02T15:25:26.273118+0000","last_became_active":"2024-02-02T15:24:47.795511+0000","last_became_peered":"2024-02-02T15:24:47.795511+0000","last_unstale":"2024-02-02T15:25:26.273118+0000","last_undegraded":"2024-02-02T15:25:26.273118+0000","last_fullsized":"2024-02-02T15:25:26.273118+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:55:31.444331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.271980+0000","last_change":"2024-02-02T15:24:47.794368+0000","last_active":"2024-02-02T15:25:26.271980+0000","last_peered":"2024-02-02T15:25:26.271980+0000","last_clean":"2024-02-02T15:25:26.271980+0000","last_became_active":"2024-02-02T15:24:47.793808+0000","last_became_peered":"2024-02-02T15:24:47.793808+0000","last_unstale":"2024-02-02T15:25:26.271980+0000","last_undegraded":"2024-02-02T15:25:26.271980+0000","last_fullsized":"2024-02-02T15:25:26.271980+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T02:39:00.099808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.276848+0000","last_change":"2024-02-02T15:24:47.796358+0000","last_active":"2024-02-02T15:25:26.276848+0000","last_peered":"2024-02-02T15:25:26.276848+0000","last_clean":"2024-02-02T15:25:26.276848+0000","last_became_active":"2024-02-02T15:24:47.795800+0000","last_became_peered":"2024-02-02T15:24:47.795800+0000","last_unstale":"2024-02-02T15:25:26.276848+0000","last_undegraded":"2024-02-02T15:25:26.276848+0000","last_fullsized":"2024-02-02T15:25:26.276848+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:49:40.019787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":250199,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67018,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":406,"num_read_kb":277,"num_write":238,"num_write_kb":137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":201054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":208,"ondisk_log_size":208,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547226,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":478056,"data_compressed":17689,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 15:25:12 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.63900000000000001}]},{"osd":1,"last update":"Fri Feb 2 15:25:12 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.69799999999999995}]},{"osd":2,"last update":"Fri Feb 2 15:25:12 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.41699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":4,"last update":"Fri Feb 2 15:25:12 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.60499999999999998}]},{"osd":5,"last update":"Fri Feb 2 15:25:12 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.50800000000000001}]},{"osd":6,"last update":"Fri Feb 2 15:25:12 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.66900000000000004}]},{"osd":7,"last update":"Fri Feb 2 15:25:12 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.78100000000000003}]},{"osd":8,"last update":"Fri Feb 2 15:25:50 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.748},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.72499999999999998}]},{"osd":9,"last update":"Fri Feb 2 15:25:12 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.74199999999999999}]},{"osd":10,"last update":"Fri Feb 2 15:25:12 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.79400000000000004}]}]},{"osd":10,"up_from":67,"seq":287762808859,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1097728,"data_stored":1125592,"data_compressed":53900,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 15:25:48 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.42899999999999999}]},{"osd":1,"last update":"Fri Feb 2 15:25:53 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.439,"15min":0.441},"min":{"1min":0.32200000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.63300000000000001,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.45900000000000002,"15min":0.46100000000000002},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.36299999999999999}]},{"osd":2,"last update":"Fri Feb 2 15:25:53 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.58799999999999997,"15min":0.59199999999999997},"min":{"1min":0.40000000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.72499999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.59399999999999997,"15min":0.59499999999999997},"min":{"1min":0.42599999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.74399999999999999,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.53600000000000003}]},{"osd":3,"last update":"Fri Feb 2 15:25:53 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.58999999999999997,"15min":0.59799999999999998},"min":{"1min":0.39900000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.73299999999999998,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.499},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.78700000000000003,"15min":0.82399999999999995},"min":{"1min":0.40600000000000003,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.70399999999999996,"5min":4.4109999999999996,"15min":4.4109999999999996},"last":0.58799999999999997}]},{"osd":4,"last update":"Fri Feb 2 15:25:53 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.36899999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.78400000000000003,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.52000000000000002,"15min":0.52500000000000002},"min":{"1min":0.375,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.65200000000000002,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.49199999999999999}]},{"osd":5,"last update":"Fri Feb 2 15:25:53 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.65600000000000003,"15min":0.65800000000000003},"min":{"1min":0.45500000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.77200000000000002,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.66300000000000003,"15min":0.66000000000000003},"min":{"1min":0.47099999999999997,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.86499999999999999,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.751}]},{"osd":6,"last update":"Fri Feb 2 15:25:53 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.80100000000000005,"15min":0.82699999999999996},"min":{"1min":0.52300000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.82799999999999996,"5min":4.4459999999999997,"15min":4.4459999999999997},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.71999999999999997,"15min":0.72899999999999998},"min":{"1min":0.44500000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.83899999999999997,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.623}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40300000000000002}]},{"osd":8,"last update":"Fri Feb 2 15:25:53 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.71999999999999997,"15min":0.72399999999999998},"min":{"1min":0.496,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.92100000000000004,"5min":1.413,"15min":1.413},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.71899999999999997,"15min":0.72199999999999998},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.89800000000000002,"5min":1.036,"15min":1.036},"last":0.73699999999999999}]},{"osd":9,"last update":"Fri Feb 2 15:25:53 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.77600000000000002,"15min":0.78400000000000003},"min":{"1min":0.58899999999999997,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.88300000000000001,"5min":1.506,"15min":1.506},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.89800000000000002,"15min":0.93200000000000005},"min":{"1min":0.55400000000000005,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.86799999999999999,"5min":4.4269999999999996,"15min":4.4269999999999996},"last":0.63300000000000001}]},{"osd":11,"last update":"Fri Feb 2 15:25:11 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.5680000000000001,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.70299999999999996}]}]},{"osd":9,"up_from":62,"seq":266287972384,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":482062,"data_compressed":17689,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 15:25:37 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.56399999999999995,"15min":0.56899999999999995},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.0960000000000001,"5min":1.196,"15min":1.196},"last":0.48299999999999998}]},{"osd":1,"last update":"Fri Feb 2 15:25:37 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.73099999999999998,"15min":0.72299999999999998},"min":{"1min":0.44900000000000001,"5min":0.38,"15min":0.38},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.70199999999999996,"15min":0.70599999999999996},"min":{"1min":0.53900000000000003,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.93500000000000005,"5min":1.198,"15min":1.198},"last":0.80100000000000005}]},{"osd":2,"last update":"Fri Feb 2 15:25:37 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.72699999999999998,"15min":0.73499999999999999},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.0009999999999999,"5min":1.135,"15min":1.135},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.48699999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.70999999999999996}]},{"osd":3,"last update":"Fri Feb 2 15:25:37 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.72599999999999998,"15min":0.72299999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.625,"5min":0.71199999999999997,"15min":0.72599999999999998},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.1359999999999999,"5min":1.296,"15min":1.296},"last":0.59799999999999998}]},{"osd":4,"last update":"Fri Feb 2 15:25:37 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.70899999999999996,"15min":0.71899999999999997},"min":{"1min":0.39900000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.80500000000000005,"5min":1.381,"15min":1.381},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.83099999999999996,"15min":0.83699999999999997},"min":{"1min":0.53400000000000003,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.92800000000000005}]},{"osd":5,"last update":"Fri Feb 2 15:25:37 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.90000000000000002,"15min":0.93200000000000005},"min":{"1min":0.57399999999999995,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.93100000000000005,"5min":3.895,"15min":3.895},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.85799999999999998,"15min":0.86199999999999999},"min":{"1min":0.47699999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.81399999999999995}]},{"osd":6,"last update":"Fri Feb 2 15:24:35 2024","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.6739999999999999,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":0.64500000000000002}]},{"osd":7,"last update":"Fri Feb 2 15:25:37 2024","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.91300000000000003,"15min":0.91900000000000004},"min":{"1min":0.64000000000000001,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.88,"15min":0.89800000000000002},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.0549999999999999,"5min":1.413,"15min":1.413},"last":0.86399999999999999}]},{"osd":8,"last update":"Fri Feb 2 15:25:37 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.90400000000000003,"15min":0.92500000000000004},"min":{"1min":0.66000000000000003,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":0.90100000000000002,"5min":1.7789999999999999,"15min":1.7789999999999999},"last":0.754},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.96399999999999997,"15min":0.97599999999999998},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.27,"5min":1.538,"15min":1.538},"last":0.77200000000000002}]},{"osd":10,"last update":"Fri Feb 2 15:25:56 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.93200000000000005,"15min":0.92700000000000005},"min":{"1min":0.71399999999999997,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.389,"5min":1.389,"15min":1.389},"last":1.0429999999999999},{"interface":"front","average":{"1min":0.874,"5min":0.86199999999999999,"15min":0.85999999999999999},"min":{"1min":0.70499999999999996,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":0.873}]},{"osd":11,"last update":"Fri Feb 2 15:25:51 2024","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.474,"5min":1.474,"15min":1.474},"last":1.006}]}]},{"osd":8,"up_from":55,"seq":236223201316,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":478013,"data_compressed":17689,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 15:25:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.58199999999999996,"15min":0.58299999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.73699999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.59399999999999997,"15min":0.60099999999999998},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.77000000000000002,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.47599999999999998}]},{"osd":1,"last update":"Fri Feb 2 15:25:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.56899999999999995,"15min":0.56499999999999995},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.57399999999999995,"15min":0.56399999999999995},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.63200000000000001}]},{"osd":2,"last update":"Fri Feb 2 15:25:16 2024","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.42099999999999999,"15min":0.41699999999999998},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.63400000000000001,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.42299999999999999,"15min":0.41699999999999998},"min":{"1min":0.307,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.61299999999999999,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.55100000000000005}]},{"osd":3,"last update":"Fri Feb 2 15:25:16 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.621,"15min":0.627},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.76400000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.61699999999999999,"15min":0.621},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.79500000000000004,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.56499999999999995}]},{"osd":4,"last update":"Fri Feb 2 15:25:16 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.60299999999999998,"15min":0.59899999999999998},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.62,"5min":0.59499999999999997,"15min":0.59099999999999997},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.45400000000000001}]},{"osd":5,"last update":"Fri Feb 2 15:25:16 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.51000000000000001,"15min":0.503},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.50800000000000001,"15min":0.502},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.65000000000000002}]},{"osd":6,"last update":"Fri Feb 2 15:25:16 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.67200000000000004,"15min":0.67500000000000004},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.83299999999999996,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.69199999999999995,"15min":0.69299999999999995},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.80900000000000005,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.70799999999999996}]},{"osd":7,"last update":"Fri Feb 2 15:25:16 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.64700000000000002,"15min":0.64100000000000001},"min":{"1min":0.49199999999999999,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.67400000000000004,"15min":0.67000000000000004},"min":{"1min":0.45800000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.88700000000000001,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.66900000000000004}]},{"osd":9,"last update":"Fri Feb 2 15:25:38 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.70799999999999996,"15min":0.69299999999999995},"min":{"1min":0.56000000000000005,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.72999999999999998,"15min":0.71499999999999997},"min":{"1min":0.62,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.73299999999999998}]},{"osd":10,"last update":"Fri Feb 2 15:25:56 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.73099999999999998,"15min":0.71699999999999997},"min":{"1min":0.56200000000000006,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.94599999999999995,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.72899999999999998,"15min":0.71299999999999997},"min":{"1min":0.63600000000000001,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.81100000000000005}]},{"osd":11,"last update":"Fri Feb 2 15:25:48 2024","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.748},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.77500000000000002}]}]},{"osd":7,"up_from":48,"seq":206158430247,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":1074698,"data_compressed":53900,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 15:23:58 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.439},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.38500000000000001}]},{"osd":1,"last update":"Fri Feb 2 15:24:59 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.42799999999999999,"15min":0.42599999999999999},"min":{"1min":0.28100000000000003,"5min":0.247,"15min":0.247},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.40899999999999997,"15min":0.40699999999999997},"min":{"1min":0.29399999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.64500000000000002,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.46899999999999997}]},{"osd":2,"last update":"Fri Feb 2 15:24:59 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.79100000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.69399999999999995}]},{"osd":3,"last update":"Fri Feb 2 15:24:59 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.60899999999999999,"15min":0.59999999999999998},"min":{"1min":0.41099999999999998,"5min":0.374,"15min":0.374},"max":{"1min":2.746,"5min":2.746,"15min":2.746},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.59799999999999998}]},{"osd":4,"last update":"Fri Feb 2 15:24:59 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.502,"15min":0.5},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.5,"15min":0.496},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.49099999999999999}]},{"osd":5,"last update":"Fri Feb 2 15:24:59 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.63,"15min":0.61899999999999999},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":2.3370000000000002,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.63,"5min":0.623,"15min":0.622},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.626}]},{"osd":6,"last update":"Fri Feb 2 15:24:59 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.68999999999999995,"15min":0.69199999999999995},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.67700000000000005,"15min":0.66800000000000004},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.778,"5min":1.778,"15min":1.778},"last":0.71099999999999997}]},{"osd":8,"last update":"Fri Feb 2 15:25:16 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.70499999999999996,"15min":0.70099999999999996},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":2.069,"5min":2.069,"15min":2.069},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.72899999999999998,"15min":0.73299999999999998},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.995,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.78400000000000003}]},{"osd":9,"last update":"Fri Feb 2 15:25:40 2024","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.76100000000000001,"15min":0.747},"min":{"1min":0.497,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":4.9550000000000001,"5min":4.9550000000000001,"15min":4.9550000000000001},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.753,"15min":0.73899999999999999},"min":{"1min":0.54500000000000004,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":4.0759999999999996,"5min":4.0759999999999996,"15min":4.0759999999999996},"last":0.83799999999999997}]},{"osd":10,"last update":"Fri Feb 2 15:25:55 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.47099999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.82499999999999996,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.495,"5min":0.436,"15min":0.436},"max":{"1min":0.88800000000000001,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.748}]},{"osd":11,"last update":"Fri Feb 2 15:25:51 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.85599999999999998}]}]},{"osd":6,"up_from":43,"seq":184683593771,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":950272,"data_stored":473422,"data_compressed":17689,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 15:25:44 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.497,"15min":0.48499999999999999},"min":{"1min":0.35599999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.67900000000000005,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51900000000000002,"15min":0.51200000000000001},"min":{"1min":0.371,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.67200000000000004,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.50600000000000001}]},{"osd":1,"last update":"Fri Feb 2 15:25:44 2024","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":0.72199999999999998,"15min":0.66400000000000003},"min":{"1min":0.47999999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":6.8650000000000002,"5min":6.8650000000000002,"15min":6.8650000000000002},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.66400000000000003,"15min":0.65200000000000002},"min":{"1min":0.50600000000000001,"5min":0.501,"15min":0.501},"max":{"1min":0.88900000000000001,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.59899999999999998}]},{"osd":2,"last update":"Fri Feb 2 15:25:44 2024","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":0.79700000000000004,"15min":0.72699999999999998},"min":{"1min":0.57199999999999995,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":6.9320000000000004,"5min":6.9320000000000004,"15min":6.9320000000000004},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.753,"15min":0.72499999999999998},"min":{"1min":0.52600000000000002,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":3.008,"5min":3.008,"15min":3.008},"last":0.69299999999999995}]},{"osd":3,"last update":"Fri Feb 2 15:25:44 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.628,"15min":0.63300000000000001},"min":{"1min":0.432,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.875,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.67400000000000004,"15min":0.66400000000000003},"min":{"1min":0.45100000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.63100000000000001}]},{"osd":4,"last update":"Fri Feb 2 15:25:44 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.72999999999999998,"15min":0.71999999999999997},"min":{"1min":0.53100000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.95599999999999996,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76200000000000001,"15min":0.75600000000000001},"min":{"1min":0.55700000000000005,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.0489999999999999,"5min":1.228,"15min":1.228},"last":0.65700000000000003}]},{"osd":5,"last update":"Fri Feb 2 15:25:44 2024","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.79100000000000004,"15min":0.77900000000000003},"min":{"1min":0.55100000000000005,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":1.268,"5min":0.90900000000000003,"15min":0.84499999999999997},"min":{"1min":0.61199999999999999,"5min":0.442,"15min":0.442},"max":{"1min":6.9820000000000002,"5min":6.9820000000000002,"15min":6.9820000000000002},"last":0.77900000000000003}]},{"osd":7,"last update":"Fri Feb 2 15:24:57 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.83199999999999996,"15min":0.83099999999999996},"min":{"1min":0.59199999999999997,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.073,"5min":1.256,"15min":1.256},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.876,"15min":0.878},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.103,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.82099999999999995}]},{"osd":8,"last update":"Fri Feb 2 15:25:14 2024","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.90000000000000002,"15min":0.90100000000000002},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.212,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.91700000000000004,"15min":0.91300000000000003},"min":{"1min":0.57699999999999996,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.81000000000000005}]},{"osd":9,"last update":"Fri Feb 2 15:25:39 2024","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.93999999999999995,"15min":0.94599999999999995},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.2410000000000001,"5min":1.405,"15min":1.405},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.96799999999999997,"15min":0.97699999999999998},"min":{"1min":0.72199999999999998,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.1120000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.90900000000000003}]},{"osd":10,"last update":"Fri Feb 2 15:25:56 2024","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1,"15min":0.96399999999999997},"min":{"1min":0.68300000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":6.9960000000000004,"5min":6.9960000000000004,"15min":6.9960000000000004},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.97799999999999998,"15min":0.98699999999999999},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.1879999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.877}]},{"osd":11,"last update":"Fri Feb 2 15:25:51 2024","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.90100000000000002},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":3.032,"5min":3.032,"15min":3.032},"last":0.96799999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574909,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":534848,"data_compressed":17689,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 15:25:12 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.51100000000000001,"15min":0.50800000000000001},"min":{"1min":0.39200000000000002,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.52500000000000002,"15min":0.52100000000000002},"min":{"1min":0.40100000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.56200000000000006}]},{"osd":2,"last update":"Fri Feb 2 15:25:34 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.55200000000000005,"15min":0.54600000000000004},"min":{"1min":0.39400000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.56200000000000006,"15min":0.54600000000000004},"min":{"1min":0.377,"5min":0.315,"15min":0.315},"max":{"1min":0.81599999999999995,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.44900000000000001}]},{"osd":3,"last update":"Fri Feb 2 15:25:50 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.56899999999999995,"15min":0.53700000000000003},"min":{"1min":0.434,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.57999999999999996,"15min":0.55400000000000005},"min":{"1min":0.45800000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.746}]},{"osd":4,"last update":"Fri Feb 2 15:25:04 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.39600000000000002,"15min":0.36899999999999999},"min":{"1min":0.32900000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.66800000000000004,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.40000000000000002,"15min":0.375},"min":{"1min":0.27200000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.59499999999999997}]},{"osd":5,"last update":"Fri Feb 2 15:25:21 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.56000000000000005,"15min":0.54100000000000004},"min":{"1min":0.39600000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.58199999999999996,"15min":0.55500000000000005},"min":{"1min":0.45300000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.72199999999999998}]},{"osd":6,"last update":"Fri Feb 2 15:25:46 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.621,"15min":0.59499999999999997},"min":{"1min":0.50800000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":6.1950000000000003},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.60599999999999998,"15min":0.58799999999999997},"min":{"1min":0.48099999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.70099999999999996}]},{"osd":7,"last update":"Fri Feb 2 15:25:00 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.81100000000000005,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.53700000000000003,"15min":0.53300000000000003},"min":{"1min":0.371,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.72699999999999998,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.76300000000000001}]},{"osd":8,"last update":"Fri Feb 2 15:25:14 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.63400000000000001,"15min":0.63},"min":{"1min":0.434,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.65700000000000003,"15min":0.64700000000000002},"min":{"1min":0.51100000000000001,"5min":0.442,"15min":0.442},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.78500000000000003}]},{"osd":9,"last update":"Fri Feb 2 15:25:38 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.71199999999999997,"15min":0.70499999999999996},"min":{"1min":0.52300000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.749,"15min":0.74299999999999999},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.02,"5min":1.121,"15min":1.121},"last":0.85899999999999999}]},{"osd":10,"last update":"Fri Feb 2 15:25:56 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.69499999999999995,"15min":0.68500000000000005},"min":{"1min":0.57399999999999995,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.72699999999999998,"15min":0.71499999999999997},"min":{"1min":0.57999999999999996,"5min":0.5,"15min":0.5},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.84499999999999997}]},{"osd":11,"last update":"Fri Feb 2 15:25:50 2024","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.81100000000000005}]}]},{"osd":0,"up_from":8,"seq":34359738432,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":974848,"data_stored":474878,"data_compressed":17689,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Feb 2 15:25:10 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.55800000000000005,"15min":0.54400000000000004},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.90400000000000003,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.628,"5min":0.57399999999999995,"15min":0.54500000000000004},"min":{"1min":0.35799999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.46899999999999997}]},{"osd":2,"last update":"Fri Feb 2 15:25:30 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.65300000000000002,"15min":0.58299999999999996},"min":{"1min":0.54500000000000004,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.70799999999999996,"15min":0.61599999999999999},"min":{"1min":0.50900000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.2789999999999999,"5min":4.6550000000000002,"15min":4.6550000000000002},"last":0.77800000000000002}]},{"osd":3,"last update":"Fri Feb 2 15:25:48 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.5},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":0.90300000000000002}]},{"osd":4,"last update":"Fri Feb 2 15:25:05 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.65400000000000003,"15min":0.63700000000000001},"min":{"1min":0.433,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.091,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.64200000000000002,"15min":0.626},"min":{"1min":0.46700000000000003,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.1279999999999999,"5min":1.141,"15min":1.141},"last":0.85099999999999998}]},{"osd":5,"last update":"Fri Feb 2 15:25:21 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68200000000000005,"15min":0.68300000000000005},"min":{"1min":0.50700000000000001,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.024,"5min":1.042,"15min":1.042},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.70999999999999996,"15min":0.67600000000000005},"min":{"1min":0.42099999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.054,"5min":4.5890000000000004,"15min":4.5890000000000004},"last":0.98599999999999999}]},{"osd":6,"last update":"Fri Feb 2 15:25:43 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.72899999999999998,"15min":0.70499999999999996},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.494,"5min":1.494,"15min":1.494},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.74099999999999999,"15min":0.73099999999999998},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.96199999999999997}]},{"osd":7,"last update":"Fri Feb 2 15:25:55 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":1.026},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.86499999999999999}]},{"osd":8,"last update":"Fri Feb 2 15:25:21 2024","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.77700000000000002,"15min":0.76900000000000002},"min":{"1min":0.56899999999999995,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":1.0069999999999999}]},{"osd":9,"last update":"Fri Feb 2 15:25:40 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.96399999999999997,"15min":0.98699999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0980000000000001,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.97299999999999998},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.95399999999999996,"15min":0.96799999999999997},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.411,"5min":1.411,"15min":1.411},"last":0.94599999999999995}]},{"osd":10,"last update":"Fri Feb 2 15:24:56 2024","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.878,"15min":0.878},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.159,"5min":1.159,"15min":1.159},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.95499999999999996}]},{"osd":11,"last update":"Fri Feb 2 15:25:10 2024","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":1.0469999999999999},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.91600000000000004}]}]},{"osd":2,"up_from":18,"seq":77309411386,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":534952,"data_compressed":17689,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 15:25:26 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54500000000000004,"15min":0.54300000000000004},"min":{"1min":0.40200000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.74299999999999999,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.629},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.52900000000000003,"15min":0.52100000000000002},"min":{"1min":0.39600000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.77200000000000002,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.41999999999999998}]},{"osd":1,"last update":"Fri Feb 2 15:25:26 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.52600000000000002,"15min":0.504},"min":{"1min":0.39200000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.56200000000000006,"15min":0.53300000000000003},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.76400000000000001}]},{"osd":3,"last update":"Fri Feb 2 15:25:46 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.60099999999999998,"15min":0.57199999999999995},"min":{"1min":0.46100000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.75600000000000001,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.623,"5min":0.58799999999999997,"15min":0.55600000000000005},"min":{"1min":0.42799999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.81000000000000005,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.53900000000000003}]},{"osd":4,"last update":"Fri Feb 2 15:25:50 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.69799999999999995}]},{"osd":5,"last update":"Fri Feb 2 15:25:26 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.432,"15min":0.40999999999999998},"min":{"1min":0.32400000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.81399999999999995,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.42399999999999999,"15min":0.40799999999999997},"min":{"1min":0.33900000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.63,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.44}]},{"osd":6,"last update":"Fri Feb 2 15:25:40 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.625,"15min":0.61899999999999999},"min":{"1min":0.46100000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.79500000000000004,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.628,"5min":0.625,"15min":0.627},"min":{"1min":0.437,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.62}]},{"osd":7,"last update":"Fri Feb 2 15:25:52 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.79000000000000004}]},{"osd":8,"last update":"Fri Feb 2 15:25:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.41899999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.84199999999999997,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.40699999999999997,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.70799999999999996,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.55600000000000005}]},{"osd":9,"last update":"Fri Feb 2 15:25:40 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.66900000000000004,"15min":0.66700000000000004},"min":{"1min":0.501,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.751,"15min":0.746},"min":{"1min":0.60299999999999998,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.92500000000000004,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.88300000000000001}]},{"osd":10,"last update":"Fri Feb 2 15:25:52 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.68300000000000005,"15min":0.67200000000000004},"min":{"1min":0.628,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.68100000000000005,"15min":0.67100000000000004},"min":{"1min":0.56299999999999994,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.86799999999999999,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.73099999999999998}]},{"osd":11,"last update":"Fri Feb 2 15:25:46 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.75700000000000001}]}]},{"osd":3,"up_from":24,"seq":103079215157,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":962560,"data_stored":473178,"data_compressed":17689,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 15:25:44 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.442,"15min":0.41899999999999998},"min":{"1min":0.34899999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.69699999999999995,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.46600000000000003,"15min":0.45300000000000001},"min":{"1min":0.33300000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.73099999999999998,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.435}]},{"osd":1,"last update":"Fri Feb 2 15:25:44 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.65000000000000002,"15min":0.59599999999999997},"min":{"1min":0.52900000000000003,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.0649999999999999,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.622,"15min":0.59899999999999998},"min":{"1min":0.49099999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.879,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.59399999999999997}]},{"osd":2,"last update":"Fri Feb 2 15:25:44 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.63300000000000001,"15min":0.623},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.70299999999999996,"15min":0.65700000000000003},"min":{"1min":0.56799999999999995,"5min":0.443,"15min":0.443},"max":{"1min":1.1319999999999999,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":0.65700000000000003}]},{"osd":4,"last update":"Fri Feb 2 15:24:59 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.625},"min":{"1min":0.40400000000000003,"5min":0.376,"15min":0.376},"max":{"1min":0.82099999999999995,"5min":1.022,"15min":1.022},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.70999999999999996,"15min":0.69199999999999995},"min":{"1min":0.61499999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0329999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.89600000000000002}]},{"osd":5,"last update":"Fri Feb 2 15:25:24 2024","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.74199999999999999,"15min":0.72399999999999998},"min":{"1min":0.52700000000000002,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.1419999999999999,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.66300000000000003,"15min":0.64700000000000002},"min":{"1min":0.48099999999999998,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.52700000000000002}]},{"osd":6,"last update":"Fri Feb 2 15:25:44 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.74199999999999999,"15min":0.71599999999999997},"min":{"1min":0.57099999999999995,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.1100000000000001,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.68999999999999995,"15min":0.68799999999999994},"min":{"1min":0.40500000000000003,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.90600000000000003,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.54400000000000004}]},{"osd":7,"last update":"Fri Feb 2 15:24:59 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.75,"15min":0.74099999999999999},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.73599999999999999,"15min":0.72999999999999998},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.97299999999999998,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.623}]},{"osd":8,"last update":"Fri Feb 2 15:24:14 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.65800000000000003}]},{"osd":9,"last update":"Fri Feb 2 15:25:37 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.94599999999999995,"15min":0.95399999999999996},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.83999999999999997,"15min":0.84999999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0449999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.63100000000000001}]},{"osd":10,"last update":"Fri Feb 2 15:24:54 2024","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.96699999999999997}]},{"osd":11,"last update":"Fri Feb 2 15:25:49 2024","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":1.0009999999999999},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.81200000000000006}]}]},{"osd":4,"up_from":31,"seq":133143986228,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":958464,"data_stored":477888,"data_compressed":17689,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 15:24:03 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.52300000000000002,"15min":0.52100000000000002},"min":{"1min":0.371,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.78600000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.53800000000000003,"15min":0.53600000000000003},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.47799999999999998}]},{"osd":1,"last update":"Fri Feb 2 15:25:05 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.35299999999999998,"15min":0.33500000000000002},"min":{"1min":0.26600000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.432,"5min":0.36499999999999999,"15min":0.34699999999999998},"min":{"1min":0.28299999999999997,"5min":0.255,"15min":0.255},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.71399999999999997}]},{"osd":2,"last update":"Fri Feb 2 15:25:05 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.52400000000000002,"15min":0.50700000000000001},"min":{"1min":0.35699999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.872},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.52800000000000002,"15min":0.51100000000000001},"min":{"1min":0.40699999999999997,"5min":0.38,"15min":0.38},"max":{"1min":0.753,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.63700000000000001}]},{"osd":3,"last update":"Fri Feb 2 15:25:05 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.56599999999999995,"15min":0.56399999999999995},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.55800000000000005,"15min":0.56000000000000005},"min":{"1min":0.38400000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.70899999999999996,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.57599999999999996}]},{"osd":5,"last update":"Fri Feb 2 15:25:21 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.60399999999999998,"15min":0.58699999999999997},"min":{"1min":0.40000000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.61199999999999999,"15min":0.60399999999999998},"min":{"1min":0.439,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.80600000000000005,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.68700000000000006}]},{"osd":6,"last update":"Fri Feb 2 15:25:41 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.63600000000000001,"15min":0.63200000000000001},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.752,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.64200000000000002},"min":{"1min":0.49199999999999999,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.84499999999999997,"5min":1.335,"15min":1.335},"last":0.82199999999999995}]},{"osd":7,"last update":"Fri Feb 2 15:24:59 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.49099999999999999,"15min":0.48299999999999998},"min":{"1min":0.34000000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.53600000000000003,"15min":0.53200000000000003},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.76000000000000001}]},{"osd":8,"last update":"Fri Feb 2 15:25:16 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.65200000000000002,"15min":0.65000000000000002},"min":{"1min":0.42499999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.84299999999999997,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.65600000000000003,"15min":0.65100000000000002},"min":{"1min":0.46999999999999997,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":1.0089999999999999}]},{"osd":9,"last update":"Fri Feb 2 15:25:36 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.71899999999999997,"15min":0.72299999999999998},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.87,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.753,"5min":0.72899999999999998,"15min":0.72499999999999998},"min":{"1min":0.53200000000000003,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.889,"5min":1.889,"15min":1.889},"last":0.85799999999999998}]},{"osd":10,"last update":"Fri Feb 2 15:25:57 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.67700000000000005,"15min":0.67300000000000004},"min":{"1min":0.48199999999999998,"5min":0.379,"15min":0.379},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.94099999999999995},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.66800000000000004,"15min":0.66700000000000004},"min":{"1min":0.55200000000000005,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.79100000000000004,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.72799999999999998}]},{"osd":11,"last update":"Fri Feb 2 15:25:48 2024","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.97499999999999998},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.84699999999999998}]}]},{"osd":5,"up_from":36,"seq":154618822702,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":1069984,"data_compressed":53900,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 15:25:20 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.53000000000000003,"15min":0.51500000000000001},"min":{"1min":0.38600000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.53900000000000003,"15min":0.52600000000000002},"min":{"1min":0.39600000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.83599999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.46300000000000002}]},{"osd":1,"last update":"Fri Feb 2 15:25:20 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.56000000000000005,"15min":0.55000000000000004},"min":{"1min":0.46500000000000002,"5min":0.252,"15min":0.252},"max":{"1min":0.69999999999999996,"5min":1.018,"15min":1.018},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.621,"5min":0.56499999999999995,"15min":0.54700000000000004},"min":{"1min":0.46400000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.77500000000000002,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.63700000000000001}]},{"osd":2,"last update":"Fri Feb 2 15:25:20 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.39500000000000002,"15min":0.38300000000000001},"min":{"1min":0.312,"5min":0.215,"15min":0.215},"max":{"1min":0.59499999999999997,"5min":0.63,"15min":0.63},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.40999999999999998,"15min":0.40000000000000002},"min":{"1min":0.318,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.72699999999999998,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.35399999999999998}]},{"osd":3,"last update":"Fri Feb 2 15:25:20 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.54700000000000004,"15min":0.52300000000000002},"min":{"1min":0.378,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.80600000000000005,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.58299999999999996,"15min":0.56599999999999995},"min":{"1min":0.52200000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.47899999999999998}]},{"osd":4,"last update":"Fri Feb 2 15:25:20 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.57499999999999996,"15min":0.56200000000000006},"min":{"1min":0.45700000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.79600000000000004,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.58699999999999997,"15min":0.57599999999999996},"min":{"1min":0.47099999999999997,"5min":0.316,"15min":0.316},"max":{"1min":0.81899999999999995,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.60799999999999998}]},{"osd":6,"last update":"Fri Feb 2 15:25:42 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.63300000000000001,"15min":0.63200000000000001},"min":{"1min":0.51700000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.84599999999999997,"5min":0.878,"15min":0.878},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.63400000000000001,"15min":0.63300000000000001},"min":{"1min":0.57599999999999996,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.81999999999999995,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.622}]},{"osd":7,"last update":"Fri Feb 2 15:25:00 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.622,"15min":0.62},"min":{"1min":0.46400000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.629,"5min":0.60799999999999998,"15min":0.60399999999999998},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.79200000000000004,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.51200000000000001}]},{"osd":8,"last update":"Fri Feb 2 15:25:14 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.55100000000000005,"15min":0.55400000000000005},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.83999999999999997,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.58699999999999997,"15min":0.59199999999999997},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.80100000000000005,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.52500000000000002}]},{"osd":9,"last update":"Fri Feb 2 15:25:38 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.69399999999999995,"15min":0.68400000000000005},"min":{"1min":0.63,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.88,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.71299999999999997,"15min":0.70499999999999996},"min":{"1min":0.63500000000000001,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.91100000000000003,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.54500000000000004}]},{"osd":10,"last update":"Fri Feb 2 15:25:54 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.70899999999999996,"15min":0.69999999999999996},"min":{"1min":0.57699999999999996,"5min":0.497,"15min":0.497},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.749},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.72799999999999998,"15min":0.72299999999999998},"min":{"1min":0.54000000000000004,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1,"5min":1,"15min":1},"last":0.66800000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34572,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34572,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36211,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36211,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36211,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36211,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36211,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":62100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":62204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":62476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-02T15:26:00.521 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph pg dump --format=json 2024-02-02T15:26:00.891 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.a/config 2024-02-02T15:26:01.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:00 smithi071 ceph-mon[44707]: pgmap v273: 129 pgs: 129 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-02-02T15:26:01.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:00 smithi071 ceph-mon[44707]: from='client.14985 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-02T15:26:01.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:00 smithi142 ceph-mon[45728]: pgmap v273: 129 pgs: 129 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-02-02T15:26:01.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:00 smithi142 ceph-mon[45728]: from='client.14985 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-02T15:26:01.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:00 smithi001 ceph-mon[35470]: pgmap v273: 129 pgs: 129 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 9 op/s 2024-02-02T15:26:01.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:00 smithi001 ceph-mon[35470]: from='client.14985 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-02T15:26:02.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:02.188+0000 7fcd99c00700 1 -- 172.21.15.1:0/2487439316 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcd94069220 msgr2=0x7fcd9410db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:02.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:02.188+0000 7fcd99c00700 1 --2- 172.21.15.1:0/2487439316 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcd94069220 0x7fcd9410db80 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c00b0a0 tx=0x7fcd8c00b3b0 comp rx=0 tx=0).stop 2024-02-02T15:26:02.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:02.188+0000 7fcd99c00700 1 -- 172.21.15.1:0/2487439316 shutdown_connections 2024-02-02T15:26:02.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:02.188+0000 7fcd99c00700 1 --2- 172.21.15.1:0/2487439316 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcd94069220 0x7fcd9410db80 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:02.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:02.188+0000 7fcd99c00700 1 --2- 172.21.15.1:0/2487439316 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd94106ed0 0x7fcd941072b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:02.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:02.188+0000 7fcd99c00700 1 --2- 172.21.15.1:0/2487439316 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd94068860 0x7fcd94068ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:02.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:02.188+0000 7fcd99c00700 1 -- 172.21.15.1:0/2487439316 >> 172.21.15.1:0/2487439316 conn(0x7fcd94076d40 msgr2=0x7fcd94077150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:02.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:02.188+0000 7fcd99c00700 1 -- 172.21.15.1:0/2487439316 shutdown_connections 2024-02-02T15:26:02.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:02.188+0000 7fcd99c00700 1 -- 172.21.15.1:0/2487439316 wait complete. 2024-02-02T15:26:02.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:02.189+0000 7fcd99c00700 1 Processor -- start 2024-02-02T15:26:02.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:02.189+0000 7fcd99c00700 1 -- start start 2024-02-02T15:26:02.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:02.189+0000 7fcd99c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd94068860 0x7fcd94199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:02.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:02.189+0000 7fcd99c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd94069220 0x7fcd941996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:02.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:02.189+0000 7fcd99c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcd94106ed0 0x7fcd9419db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:02.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:02.189+0000 7fcd99c00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9419e130 con 0x7fcd94106ed0 2024-02-02T15:26:02.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:02.189+0000 7fcd99c00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9419e2a0 con 0x7fcd94068860 2024-02-02T15:26:02.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:02.189+0000 7fcd99c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9419e460 con 0x7fcd94069220 2024-02-02T15:26:02.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:02.190+0000 7fcd93400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcd94106ed0 0x7fcd9419db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:02.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:02.190+0000 7fcd92000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd94069220 0x7fcd941996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:02.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:02.190+0000 7fcd92a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd94068860 0x7fcd94199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:02.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:02.190+0000 7fcd92a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd94068860 0x7fcd94199170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59266/0 (socket says 172.21.15.1:59266) 2024-02-02T15:26:02.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:02.190+0000 7fcd92000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd94069220 0x7fcd941996b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:32814/0 (socket says 172.21.15.1:32814) 2024-02-02T15:26:02.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:02.190+0000 7fcd92a00700 1 -- 172.21.15.1:0/3307032703 learned_addr learned my addr 172.21.15.1:0/3307032703 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:02.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:02.190+0000 7fcd92a00700 1 -- 172.21.15.1:0/3307032703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd94069220 msgr2=0x7fcd941996b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:02.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:02.190+0000 7fcd92a00700 1 --2- 172.21.15.1:0/3307032703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd94069220 0x7fcd941996b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:02.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:02.190+0000 7fcd92a00700 1 -- 172.21.15.1:0/3307032703 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcd94106ed0 msgr2=0x7fcd9419db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:02.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:02.190+0000 7fcd92a00700 1 --2- 172.21.15.1:0/3307032703 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcd94106ed0 0x7fcd9419db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:02.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:02.190+0000 7fcd92a00700 1 -- 172.21.15.1:0/3307032703 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd8c009d00 con 0x7fcd94068860 2024-02-02T15:26:02.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:02.190+0000 7fcd93400700 1 --2- 172.21.15.1:0/3307032703 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcd94106ed0 0x7fcd9419db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:26:02.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:02.190+0000 7fcd92a00700 1 --2- 172.21.15.1:0/3307032703 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd94068860 0x7fcd94199170 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fcd8400e9a0 tx=0x7fcd8400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:02.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:02.191+0000 7fcd8b400700 1 -- 172.21.15.1:0/3307032703 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd84009790 con 0x7fcd94068860 2024-02-02T15:26:02.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:02.191+0000 7fcd99c00700 1 -- 172.21.15.1:0/3307032703 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd9419e740 con 0x7fcd94068860 2024-02-02T15:26:02.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:02.191+0000 7fcd8b400700 1 -- 172.21.15.1:0/3307032703 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd84010430 con 0x7fcd94068860 2024-02-02T15:26:02.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:02.191+0000 7fcd99c00700 1 -- 172.21.15.1:0/3307032703 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd941a5fa0 con 0x7fcd94068860 2024-02-02T15:26:02.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:02.191+0000 7fcd8b400700 1 -- 172.21.15.1:0/3307032703 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd84018680 con 0x7fcd94068860 2024-02-02T15:26:02.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:02.192+0000 7fcd99c00700 1 -- 172.21.15.1:0/3307032703 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd9404ea40 con 0x7fcd94068860 2024-02-02T15:26:02.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:02.194+0000 7fcd8b400700 1 -- 172.21.15.1:0/3307032703 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 135120+0+0 (secure 0 0 0) 0x7fcd84010810 con 0x7fcd94068860 2024-02-02T15:26:02.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:02.195+0000 7fcd8b400700 1 --2- 172.21.15.1:0/3307032703 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fcd740a0630 0x7fcd740a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:02.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:02.195+0000 7fcd8b400700 1 -- 172.21.15.1:0/3307032703 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fcd84014070 con 0x7fcd94068860 2024-02-02T15:26:02.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:02.196+0000 7fcd92000700 1 --2- 172.21.15.1:0/3307032703 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fcd740a0630 0x7fcd740a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:02.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:02.196+0000 7fcd92000700 1 --2- 172.21.15.1:0/3307032703 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fcd740a0630 0x7fcd740a2af0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fcd9419a790 tx=0x7fcd7c009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:02.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:02.198+0000 7fcd8b400700 1 -- 172.21.15.1:0/3307032703 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7fcd84052110 con 0x7fcd94068860 2024-02-02T15:26:02.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:02.404+0000 7fcd99c00700 1 -- 172.21.15.1:0/3307032703 --> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcd9410c5e0 con 0x7fcd740a0630 2024-02-02T15:26:02.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:02.417+0000 7fcd8b400700 1 -- 172.21.15.1:0/3307032703 <== mgr.14154 v2:172.21.15.1:6800/3768989981 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427306 (secure 0 0 0) 0x7fcd9410c5e0 con 0x7fcd740a0630 2024-02-02T15:26:02.423 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:26:02.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:02.422+0000 7fcd99c00700 1 -- 172.21.15.1:0/3307032703 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fcd740a0630 msgr2=0x7fcd740a2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:02.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:02.422+0000 7fcd99c00700 1 --2- 172.21.15.1:0/3307032703 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fcd740a0630 0x7fcd740a2af0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fcd9419a790 tx=0x7fcd7c009260 comp rx=0 tx=0).stop 2024-02-02T15:26:02.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:02.422+0000 7fcd99c00700 1 -- 172.21.15.1:0/3307032703 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd94068860 msgr2=0x7fcd94199170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:02.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:02.422+0000 7fcd99c00700 1 --2- 172.21.15.1:0/3307032703 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd94068860 0x7fcd94199170 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fcd8400e9a0 tx=0x7fcd8400ecb0 comp rx=0 tx=0).stop 2024-02-02T15:26:02.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:02.423+0000 7fcd99c00700 1 -- 172.21.15.1:0/3307032703 shutdown_connections 2024-02-02T15:26:02.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:02.423+0000 7fcd99c00700 1 --2- 172.21.15.1:0/3307032703 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcd94106ed0 0x7fcd9419db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:02.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:02.423+0000 7fcd99c00700 1 --2- 172.21.15.1:0/3307032703 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcd94068860 0x7fcd94199170 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:02.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:02.423+0000 7fcd99c00700 1 --2- 172.21.15.1:0/3307032703 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fcd740a0630 0x7fcd740a2af0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:02.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:02.423+0000 7fcd99c00700 1 --2- 172.21.15.1:0/3307032703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcd94069220 0x7fcd941996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:02.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:02.423+0000 7fcd99c00700 1 -- 172.21.15.1:0/3307032703 >> 172.21.15.1:0/3307032703 conn(0x7fcd94076d40 msgr2=0x7fcd9410bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:02.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:02.423+0000 7fcd99c00700 1 -- 172.21.15.1:0/3307032703 shutdown_connections 2024-02-02T15:26:02.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:02.423+0000 7fcd99c00700 1 -- 172.21.15.1:0/3307032703 wait complete. 2024-02-02T15:26:02.427 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-02-02T15:26:02.810 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":274,"stamp":"2024-02-02T15:26:01.923562+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657632,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":572,"num_read_kb":411,"num_write":421,"num_write_kb":3077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1180804,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":347,"ondisk_log_size":347,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":335916,"kb_used_data":11668,"kb_used_omap":20,"kb_used_meta":324075,"kb_avail":1124605908,"statfs":{"total":1151940427776,"available":1151596449792,"internally_reserved":0,"allocated":11948032,"data_stored":7677571,"data_compressed":320901,"data_compressed_allocated":430080,"data_compressed_original":2359296,"omap_allocated":21372,"internal_metadata":331852932},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":127,"num_read_kb":64,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003360"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.275844+0000","last_change":"2024-02-02T15:24:47.793308+0000","last_active":"2024-02-02T15:25:26.275844+0000","last_peered":"2024-02-02T15:25:26.275844+0000","last_clean":"2024-02-02T15:25:26.275844+0000","last_became_active":"2024-02-02T15:24:47.792420+0000","last_became_peered":"2024-02-02T15:24:47.792420+0000","last_unstale":"2024-02-02T15:25:26.275844+0000","last_undegraded":"2024-02-02T15:25:26.275844+0000","last_fullsized":"2024-02-02T15:25:26.275844+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:39:09.871317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.280986+0000","last_change":"2024-02-02T15:24:47.791272+0000","last_active":"2024-02-02T15:25:26.280986+0000","last_peered":"2024-02-02T15:25:26.280986+0000","last_clean":"2024-02-02T15:25:26.280986+0000","last_became_active":"2024-02-02T15:24:47.790902+0000","last_became_peered":"2024-02-02T15:24:47.790902+0000","last_unstale":"2024-02-02T15:25:26.280986+0000","last_undegraded":"2024-02-02T15:25:26.280986+0000","last_fullsized":"2024-02-02T15:25:26.280986+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:31:11.620873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.280219+0000","last_change":"2024-02-02T15:24:47.798712+0000","last_active":"2024-02-02T15:25:26.280219+0000","last_peered":"2024-02-02T15:25:26.280219+0000","last_clean":"2024-02-02T15:25:26.280219+0000","last_became_active":"2024-02-02T15:24:47.798405+0000","last_became_peered":"2024-02-02T15:24:47.798405+0000","last_unstale":"2024-02-02T15:25:26.280219+0000","last_undegraded":"2024-02-02T15:25:26.280219+0000","last_fullsized":"2024-02-02T15:25:26.280219+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:37:15.238070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.275855+0000","last_change":"2024-02-02T15:24:47.786456+0000","last_active":"2024-02-02T15:25:26.275855+0000","last_peered":"2024-02-02T15:25:26.275855+0000","last_clean":"2024-02-02T15:25:26.275855+0000","last_became_active":"2024-02-02T15:24:47.785520+0000","last_became_peered":"2024-02-02T15:24:47.785520+0000","last_unstale":"2024-02-02T15:25:26.275855+0000","last_undegraded":"2024-02-02T15:25:26.275855+0000","last_fullsized":"2024-02-02T15:25:26.275855+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:16:04.003459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.277322+0000","last_change":"2024-02-02T15:24:47.780084+0000","last_active":"2024-02-02T15:25:26.277322+0000","last_peered":"2024-02-02T15:25:26.277322+0000","last_clean":"2024-02-02T15:25:26.277322+0000","last_became_active":"2024-02-02T15:24:47.779667+0000","last_became_peered":"2024-02-02T15:24:47.779667+0000","last_unstale":"2024-02-02T15:25:26.277322+0000","last_undegraded":"2024-02-02T15:25:26.277322+0000","last_fullsized":"2024-02-02T15:25:26.277322+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:30:15.377691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.277446+0000","last_change":"2024-02-02T15:24:47.786173+0000","last_active":"2024-02-02T15:25:26.277446+0000","last_peered":"2024-02-02T15:25:26.277446+0000","last_clean":"2024-02-02T15:25:26.277446+0000","last_became_active":"2024-02-02T15:24:47.785301+0000","last_became_peered":"2024-02-02T15:24:47.785301+0000","last_unstale":"2024-02-02T15:25:26.277446+0000","last_undegraded":"2024-02-02T15:25:26.277446+0000","last_fullsized":"2024-02-02T15:25:26.277446+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:14:58.524317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.275443+0000","last_change":"2024-02-02T15:24:47.790913+0000","last_active":"2024-02-02T15:25:26.275443+0000","last_peered":"2024-02-02T15:25:26.275443+0000","last_clean":"2024-02-02T15:25:26.275443+0000","last_became_active":"2024-02-02T15:24:47.790566+0000","last_became_peered":"2024-02-02T15:24:47.790566+0000","last_unstale":"2024-02-02T15:25:26.275443+0000","last_undegraded":"2024-02-02T15:25:26.275443+0000","last_fullsized":"2024-02-02T15:25:26.275443+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:34:51.954830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.276550+0000","last_change":"2024-02-02T15:24:47.786465+0000","last_active":"2024-02-02T15:25:26.276550+0000","last_peered":"2024-02-02T15:25:26.276550+0000","last_clean":"2024-02-02T15:25:26.276550+0000","last_became_active":"2024-02-02T15:24:47.786175+0000","last_became_peered":"2024-02-02T15:24:47.786175+0000","last_unstale":"2024-02-02T15:25:26.276550+0000","last_undegraded":"2024-02-02T15:25:26.276550+0000","last_fullsized":"2024-02-02T15:25:26.276550+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:42:13.748850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.273552+0000","last_change":"2024-02-02T15:24:47.787115+0000","last_active":"2024-02-02T15:25:26.273552+0000","last_peered":"2024-02-02T15:25:26.273552+0000","last_clean":"2024-02-02T15:25:26.273552+0000","last_became_active":"2024-02-02T15:24:47.786804+0000","last_became_peered":"2024-02-02T15:24:47.786804+0000","last_unstale":"2024-02-02T15:25:26.273552+0000","last_undegraded":"2024-02-02T15:25:26.273552+0000","last_fullsized":"2024-02-02T15:25:26.273552+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T03:06:17.873243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.277877+0000","last_change":"2024-02-02T15:24:47.787211+0000","last_active":"2024-02-02T15:25:26.277877+0000","last_peered":"2024-02-02T15:25:26.277877+0000","last_clean":"2024-02-02T15:25:26.277877+0000","last_became_active":"2024-02-02T15:24:47.786888+0000","last_became_peered":"2024-02-02T15:24:47.786888+0000","last_unstale":"2024-02-02T15:25:26.277877+0000","last_undegraded":"2024-02-02T15:25:26.277877+0000","last_fullsized":"2024-02-02T15:25:26.277877+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:14:21.442787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.275882+0000","last_change":"2024-02-02T15:24:47.798962+0000","last_active":"2024-02-02T15:25:26.275882+0000","last_peered":"2024-02-02T15:25:26.275882+0000","last_clean":"2024-02-02T15:25:26.275882+0000","last_became_active":"2024-02-02T15:24:47.798657+0000","last_became_peered":"2024-02-02T15:24:47.798657+0000","last_unstale":"2024-02-02T15:25:26.275882+0000","last_undegraded":"2024-02-02T15:25:26.275882+0000","last_fullsized":"2024-02-02T15:25:26.275882+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:53:37.316027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.275776+0000","last_change":"2024-02-02T15:24:47.782647+0000","last_active":"2024-02-02T15:25:26.275776+0000","last_peered":"2024-02-02T15:25:26.275776+0000","last_clean":"2024-02-02T15:25:26.275776+0000","last_became_active":"2024-02-02T15:24:47.782344+0000","last_became_peered":"2024-02-02T15:24:47.782344+0000","last_unstale":"2024-02-02T15:25:26.275776+0000","last_undegraded":"2024-02-02T15:25:26.275776+0000","last_fullsized":"2024-02-02T15:25:26.275776+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:06:54.874657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.272126+0000","last_change":"2024-02-02T15:24:47.787032+0000","last_active":"2024-02-02T15:25:26.272126+0000","last_peered":"2024-02-02T15:25:26.272126+0000","last_clean":"2024-02-02T15:25:26.272126+0000","last_became_active":"2024-02-02T15:24:47.786710+0000","last_became_peered":"2024-02-02T15:24:47.786710+0000","last_unstale":"2024-02-02T15:25:26.272126+0000","last_undegraded":"2024-02-02T15:25:26.272126+0000","last_fullsized":"2024-02-02T15:25:26.272126+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:26:15.429316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.278945+0000","last_change":"2024-02-02T15:24:47.785708+0000","last_active":"2024-02-02T15:25:26.278945+0000","last_peered":"2024-02-02T15:25:26.278945+0000","last_clean":"2024-02-02T15:25:26.278945+0000","last_became_active":"2024-02-02T15:24:47.785280+0000","last_became_peered":"2024-02-02T15:24:47.785280+0000","last_unstale":"2024-02-02T15:25:26.278945+0000","last_undegraded":"2024-02-02T15:25:26.278945+0000","last_fullsized":"2024-02-02T15:25:26.278945+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T02:21:23.224933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.278605+0000","last_change":"2024-02-02T15:24:47.783904+0000","last_active":"2024-02-02T15:25:26.278605+0000","last_peered":"2024-02-02T15:25:26.278605+0000","last_clean":"2024-02-02T15:25:26.278605+0000","last_became_active":"2024-02-02T15:24:47.783599+0000","last_became_peered":"2024-02-02T15:24:47.783599+0000","last_unstale":"2024-02-02T15:25:26.278605+0000","last_undegraded":"2024-02-02T15:25:26.278605+0000","last_fullsized":"2024-02-02T15:25:26.278605+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:35:45.805527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.272807+0000","last_change":"2024-02-02T15:24:47.795770+0000","last_active":"2024-02-02T15:25:26.272807+0000","last_peered":"2024-02-02T15:25:26.272807+0000","last_clean":"2024-02-02T15:25:26.272807+0000","last_became_active":"2024-02-02T15:24:47.795433+0000","last_became_peered":"2024-02-02T15:24:47.795433+0000","last_unstale":"2024-02-02T15:25:26.272807+0000","last_undegraded":"2024-02-02T15:25:26.272807+0000","last_fullsized":"2024-02-02T15:25:26.272807+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:50:07.496998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.275747+0000","last_change":"2024-02-02T15:24:47.793684+0000","last_active":"2024-02-02T15:25:26.275747+0000","last_peered":"2024-02-02T15:25:26.275747+0000","last_clean":"2024-02-02T15:25:26.275747+0000","last_became_active":"2024-02-02T15:24:47.792853+0000","last_became_peered":"2024-02-02T15:24:47.792853+0000","last_unstale":"2024-02-02T15:25:26.275747+0000","last_undegraded":"2024-02-02T15:25:26.275747+0000","last_fullsized":"2024-02-02T15:25:26.275747+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:30:40.379379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.282537+0000","last_change":"2024-02-02T15:24:47.780935+0000","last_active":"2024-02-02T15:25:26.282537+0000","last_peered":"2024-02-02T15:25:26.282537+0000","last_clean":"2024-02-02T15:25:26.282537+0000","last_became_active":"2024-02-02T15:24:47.780642+0000","last_became_peered":"2024-02-02T15:24:47.780642+0000","last_unstale":"2024-02-02T15:25:26.282537+0000","last_undegraded":"2024-02-02T15:25:26.282537+0000","last_fullsized":"2024-02-02T15:25:26.282537+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:36:37.615368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.280797+0000","last_change":"2024-02-02T15:24:47.791051+0000","last_active":"2024-02-02T15:25:26.280797+0000","last_peered":"2024-02-02T15:25:26.280797+0000","last_clean":"2024-02-02T15:25:26.280797+0000","last_became_active":"2024-02-02T15:24:47.790732+0000","last_became_peered":"2024-02-02T15:24:47.790732+0000","last_unstale":"2024-02-02T15:25:26.280797+0000","last_undegraded":"2024-02-02T15:25:26.280797+0000","last_fullsized":"2024-02-02T15:25:26.280797+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:45:05.155144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.279514+0000","last_change":"2024-02-02T15:24:51.814699+0000","last_active":"2024-02-02T15:25:26.279514+0000","last_peered":"2024-02-02T15:25:26.279514+0000","last_clean":"2024-02-02T15:25:26.279514+0000","last_became_active":"2024-02-02T15:24:51.814251+0000","last_became_peered":"2024-02-02T15:24:51.814251+0000","last_unstale":"2024-02-02T15:25:26.279514+0000","last_undegraded":"2024-02-02T15:25:26.279514+0000","last_fullsized":"2024-02-02T15:25:26.279514+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:27:21.674922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"84'13","reported_seq":71,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T15:25:57.332488+0000","last_change":"2024-02-02T15:24:47.269429+0000","last_active":"2024-02-02T15:25:57.332488+0000","last_peered":"2024-02-02T15:25:57.332488+0000","last_clean":"2024-02-02T15:25:57.332488+0000","last_became_active":"2024-02-02T15:24:47.269023+0000","last_became_peered":"2024-02-02T15:24:47.269023+0000","last_unstale":"2024-02-02T15:25:57.332488+0000","last_undegraded":"2024-02-02T15:25:57.332488+0000","last_fullsized":"2024-02-02T15:25:57.332488+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T03:11:46.344438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":40,"num_read_kb":40,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.276412+0000","last_change":"2024-02-02T15:24:47.785900+0000","last_active":"2024-02-02T15:25:26.276412+0000","last_peered":"2024-02-02T15:25:26.276412+0000","last_clean":"2024-02-02T15:25:26.276412+0000","last_became_active":"2024-02-02T15:24:47.781994+0000","last_became_peered":"2024-02-02T15:24:47.781994+0000","last_unstale":"2024-02-02T15:25:26.276412+0000","last_undegraded":"2024-02-02T15:25:26.276412+0000","last_fullsized":"2024-02-02T15:25:26.276412+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:00:52.432637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.275056+0000","last_change":"2024-02-02T15:24:51.825655+0000","last_active":"2024-02-02T15:25:26.275056+0000","last_peered":"2024-02-02T15:25:26.275056+0000","last_clean":"2024-02-02T15:25:26.275056+0000","last_became_active":"2024-02-02T15:24:51.825199+0000","last_became_peered":"2024-02-02T15:24:51.825199+0000","last_unstale":"2024-02-02T15:25:26.275056+0000","last_undegraded":"2024-02-02T15:25:26.275056+0000","last_fullsized":"2024-02-02T15:25:26.275056+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:50:32.283394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"84'2","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.278347+0000","last_change":"2024-02-02T15:24:47.281869+0000","last_active":"2024-02-02T15:25:26.278347+0000","last_peered":"2024-02-02T15:25:26.278347+0000","last_clean":"2024-02-02T15:25:26.278347+0000","last_became_active":"2024-02-02T15:24:47.280825+0000","last_became_peered":"2024-02-02T15:24:47.280825+0000","last_unstale":"2024-02-02T15:25:26.278347+0000","last_undegraded":"2024-02-02T15:25:26.278347+0000","last_fullsized":"2024-02-02T15:25:26.278347+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:24:19.688389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.280594+0000","last_change":"2024-02-02T15:24:47.795893+0000","last_active":"2024-02-02T15:25:26.280594+0000","last_peered":"2024-02-02T15:25:26.280594+0000","last_clean":"2024-02-02T15:25:26.280594+0000","last_became_active":"2024-02-02T15:24:47.795491+0000","last_became_peered":"2024-02-02T15:24:47.795491+0000","last_unstale":"2024-02-02T15:25:26.280594+0000","last_undegraded":"2024-02-02T15:25:26.280594+0000","last_fullsized":"2024-02-02T15:25:26.280594+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:54:38.923036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.275861+0000","last_change":"2024-02-02T15:24:51.826980+0000","last_active":"2024-02-02T15:25:26.275861+0000","last_peered":"2024-02-02T15:25:26.275861+0000","last_clean":"2024-02-02T15:25:26.275861+0000","last_became_active":"2024-02-02T15:24:51.826221+0000","last_became_peered":"2024-02-02T15:24:51.826221+0000","last_unstale":"2024-02-02T15:25:26.275861+0000","last_undegraded":"2024-02-02T15:25:26.275861+0000","last_fullsized":"2024-02-02T15:25:26.275861+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:45:17.126174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"84'12","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.278436+0000","last_change":"2024-02-02T15:24:47.272219+0000","last_active":"2024-02-02T15:25:26.278436+0000","last_peered":"2024-02-02T15:25:26.278436+0000","last_clean":"2024-02-02T15:25:26.278436+0000","last_became_active":"2024-02-02T15:24:47.271899+0000","last_became_peered":"2024-02-02T15:24:47.271899+0000","last_unstale":"2024-02-02T15:25:26.278436+0000","last_undegraded":"2024-02-02T15:25:26.278436+0000","last_fullsized":"2024-02-02T15:25:26.278436+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:44:27.370783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.273053+0000","last_change":"2024-02-02T15:24:47.783434+0000","last_active":"2024-02-02T15:25:26.273053+0000","last_peered":"2024-02-02T15:25:26.273053+0000","last_clean":"2024-02-02T15:25:26.273053+0000","last_became_active":"2024-02-02T15:24:47.782130+0000","last_became_peered":"2024-02-02T15:24:47.782130+0000","last_unstale":"2024-02-02T15:25:26.273053+0000","last_undegraded":"2024-02-02T15:25:26.273053+0000","last_fullsized":"2024-02-02T15:25:26.273053+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:32:25.981913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.275051+0000","last_change":"2024-02-02T15:24:51.810791+0000","last_active":"2024-02-02T15:25:26.275051+0000","last_peered":"2024-02-02T15:25:26.275051+0000","last_clean":"2024-02-02T15:25:26.275051+0000","last_became_active":"2024-02-02T15:24:51.810379+0000","last_became_peered":"2024-02-02T15:24:51.810379+0000","last_unstale":"2024-02-02T15:25:26.275051+0000","last_undegraded":"2024-02-02T15:25:26.275051+0000","last_fullsized":"2024-02-02T15:25:26.275051+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:49:44.635820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.277871+0000","last_change":"2024-02-02T15:24:47.271225+0000","last_active":"2024-02-02T15:25:26.277871+0000","last_peered":"2024-02-02T15:25:26.277871+0000","last_clean":"2024-02-02T15:25:26.277871+0000","last_became_active":"2024-02-02T15:24:47.270870+0000","last_became_peered":"2024-02-02T15:24:47.270870+0000","last_unstale":"2024-02-02T15:25:26.277871+0000","last_undegraded":"2024-02-02T15:25:26.277871+0000","last_fullsized":"2024-02-02T15:25:26.277871+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:16:55.659950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.272911+0000","last_change":"2024-02-02T15:24:47.791367+0000","last_active":"2024-02-02T15:25:26.272911+0000","last_peered":"2024-02-02T15:25:26.272911+0000","last_clean":"2024-02-02T15:25:26.272911+0000","last_became_active":"2024-02-02T15:24:47.790835+0000","last_became_peered":"2024-02-02T15:24:47.790835+0000","last_unstale":"2024-02-02T15:25:26.272911+0000","last_undegraded":"2024-02-02T15:25:26.272911+0000","last_fullsized":"2024-02-02T15:25:26.272911+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:02:51.807571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.280640+0000","last_change":"2024-02-02T15:24:51.807328+0000","last_active":"2024-02-02T15:25:26.280640+0000","last_peered":"2024-02-02T15:25:26.280640+0000","last_clean":"2024-02-02T15:25:26.280640+0000","last_became_active":"2024-02-02T15:24:51.806853+0000","last_became_peered":"2024-02-02T15:24:51.806853+0000","last_unstale":"2024-02-02T15:25:26.280640+0000","last_undegraded":"2024-02-02T15:25:26.280640+0000","last_fullsized":"2024-02-02T15:25:26.280640+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:46:13.898450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"84'5","reported_seq":66,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T15:25:59.319030+0000","last_change":"2024-02-02T15:24:47.268192+0000","last_active":"2024-02-02T15:25:59.319030+0000","last_peered":"2024-02-02T15:25:59.319030+0000","last_clean":"2024-02-02T15:25:59.319030+0000","last_became_active":"2024-02-02T15:24:47.267796+0000","last_became_peered":"2024-02-02T15:24:47.267796+0000","last_unstale":"2024-02-02T15:25:59.319030+0000","last_undegraded":"2024-02-02T15:25:59.319030+0000","last_fullsized":"2024-02-02T15:25:59.319030+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:12:13.161690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":43,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.277181+0000","last_change":"2024-02-02T15:24:47.782976+0000","last_active":"2024-02-02T15:25:26.277181+0000","last_peered":"2024-02-02T15:25:26.277181+0000","last_clean":"2024-02-02T15:25:26.277181+0000","last_became_active":"2024-02-02T15:24:47.782640+0000","last_became_peered":"2024-02-02T15:24:47.782640+0000","last_unstale":"2024-02-02T15:25:26.277181+0000","last_undegraded":"2024-02-02T15:25:26.277181+0000","last_fullsized":"2024-02-02T15:25:26.277181+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T02:09:54.851418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.281037+0000","last_change":"2024-02-02T15:24:51.814799+0000","last_active":"2024-02-02T15:25:26.281037+0000","last_peered":"2024-02-02T15:25:26.281037+0000","last_clean":"2024-02-02T15:25:26.281037+0000","last_became_active":"2024-02-02T15:24:51.814263+0000","last_became_peered":"2024-02-02T15:24:51.814263+0000","last_unstale":"2024-02-02T15:25:26.281037+0000","last_undegraded":"2024-02-02T15:25:26.281037+0000","last_fullsized":"2024-02-02T15:25:26.281037+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:52:02.786496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"84'7","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.277357+0000","last_change":"2024-02-02T15:24:47.273204+0000","last_active":"2024-02-02T15:25:26.277357+0000","last_peered":"2024-02-02T15:25:26.277357+0000","last_clean":"2024-02-02T15:25:26.277357+0000","last_became_active":"2024-02-02T15:24:47.272888+0000","last_became_peered":"2024-02-02T15:24:47.272888+0000","last_unstale":"2024-02-02T15:25:26.277357+0000","last_undegraded":"2024-02-02T15:25:26.277357+0000","last_fullsized":"2024-02-02T15:25:26.277357+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:26:25.188915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.272783+0000","last_change":"2024-02-02T15:24:47.788645+0000","last_active":"2024-02-02T15:25:26.272783+0000","last_peered":"2024-02-02T15:25:26.272783+0000","last_clean":"2024-02-02T15:25:26.272783+0000","last_became_active":"2024-02-02T15:24:47.787478+0000","last_became_peered":"2024-02-02T15:24:47.787478+0000","last_unstale":"2024-02-02T15:25:26.272783+0000","last_undegraded":"2024-02-02T15:25:26.272783+0000","last_fullsized":"2024-02-02T15:25:26.272783+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:24:45.327030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.275632+0000","last_change":"2024-02-02T15:24:51.822681+0000","last_active":"2024-02-02T15:25:26.275632+0000","last_peered":"2024-02-02T15:25:26.275632+0000","last_clean":"2024-02-02T15:25:26.275632+0000","last_became_active":"2024-02-02T15:24:51.822202+0000","last_became_peered":"2024-02-02T15:24:51.822202+0000","last_unstale":"2024-02-02T15:25:26.275632+0000","last_undegraded":"2024-02-02T15:25:26.275632+0000","last_fullsized":"2024-02-02T15:25:26.275632+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:05:43.751558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"84'3","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.271740+0000","last_change":"2024-02-02T15:24:47.271095+0000","last_active":"2024-02-02T15:25:26.271740+0000","last_peered":"2024-02-02T15:25:26.271740+0000","last_clean":"2024-02-02T15:25:26.271740+0000","last_became_active":"2024-02-02T15:24:47.270696+0000","last_became_peered":"2024-02-02T15:24:47.270696+0000","last_unstale":"2024-02-02T15:25:26.271740+0000","last_undegraded":"2024-02-02T15:25:26.271740+0000","last_fullsized":"2024-02-02T15:25:26.271740+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T02:20:59.370217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.276719+0000","last_change":"2024-02-02T15:24:47.791699+0000","last_active":"2024-02-02T15:25:26.276719+0000","last_peered":"2024-02-02T15:25:26.276719+0000","last_clean":"2024-02-02T15:25:26.276719+0000","last_became_active":"2024-02-02T15:24:47.787280+0000","last_became_peered":"2024-02-02T15:24:47.787280+0000","last_unstale":"2024-02-02T15:25:26.276719+0000","last_undegraded":"2024-02-02T15:25:26.276719+0000","last_fullsized":"2024-02-02T15:25:26.276719+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:18:10.316393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.280531+0000","last_change":"2024-02-02T15:24:51.810050+0000","last_active":"2024-02-02T15:25:26.280531+0000","last_peered":"2024-02-02T15:25:26.280531+0000","last_clean":"2024-02-02T15:25:26.280531+0000","last_became_active":"2024-02-02T15:24:51.809621+0000","last_became_peered":"2024-02-02T15:24:51.809621+0000","last_unstale":"2024-02-02T15:25:26.280531+0000","last_undegraded":"2024-02-02T15:25:26.280531+0000","last_fullsized":"2024-02-02T15:25:26.280531+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:52:35.885973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"84'3","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.272677+0000","last_change":"2024-02-02T15:24:47.267973+0000","last_active":"2024-02-02T15:25:26.272677+0000","last_peered":"2024-02-02T15:25:26.272677+0000","last_clean":"2024-02-02T15:25:26.272677+0000","last_became_active":"2024-02-02T15:24:47.267649+0000","last_became_peered":"2024-02-02T15:24:47.267649+0000","last_unstale":"2024-02-02T15:25:26.272677+0000","last_undegraded":"2024-02-02T15:25:26.272677+0000","last_fullsized":"2024-02-02T15:25:26.272677+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T02:46:56.844570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.281458+0000","last_change":"2024-02-02T15:24:47.798636+0000","last_active":"2024-02-02T15:25:26.281458+0000","last_peered":"2024-02-02T15:25:26.281458+0000","last_clean":"2024-02-02T15:25:26.281458+0000","last_became_active":"2024-02-02T15:24:47.798328+0000","last_became_peered":"2024-02-02T15:24:47.798328+0000","last_unstale":"2024-02-02T15:25:26.281458+0000","last_undegraded":"2024-02-02T15:25:26.281458+0000","last_fullsized":"2024-02-02T15:25:26.281458+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:37:43.687336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.277684+0000","last_change":"2024-02-02T15:24:51.811312+0000","last_active":"2024-02-02T15:25:26.277684+0000","last_peered":"2024-02-02T15:25:26.277684+0000","last_clean":"2024-02-02T15:25:26.277684+0000","last_became_active":"2024-02-02T15:24:51.810932+0000","last_became_peered":"2024-02-02T15:24:51.810932+0000","last_unstale":"2024-02-02T15:25:26.277684+0000","last_undegraded":"2024-02-02T15:25:26.277684+0000","last_fullsized":"2024-02-02T15:25:26.277684+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:10:31.530518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"84'4","reported_seq":104,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T15:26:00.319951+0000","last_change":"2024-02-02T15:24:47.271483+0000","last_active":"2024-02-02T15:26:00.319951+0000","last_peered":"2024-02-02T15:26:00.319951+0000","last_clean":"2024-02-02T15:26:00.319951+0000","last_became_active":"2024-02-02T15:24:47.271160+0000","last_became_peered":"2024-02-02T15:24:47.271160+0000","last_unstale":"2024-02-02T15:26:00.319951+0000","last_undegraded":"2024-02-02T15:26:00.319951+0000","last_fullsized":"2024-02-02T15:26:00.319951+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:29:20.135351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":82,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.273506+0000","last_change":"2024-02-02T15:24:47.786492+0000","last_active":"2024-02-02T15:25:26.273506+0000","last_peered":"2024-02-02T15:25:26.273506+0000","last_clean":"2024-02-02T15:25:26.273506+0000","last_became_active":"2024-02-02T15:24:47.786103+0000","last_became_peered":"2024-02-02T15:24:47.786103+0000","last_unstale":"2024-02-02T15:25:26.273506+0000","last_undegraded":"2024-02-02T15:25:26.273506+0000","last_fullsized":"2024-02-02T15:25:26.273506+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:24:23.666596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.280381+0000","last_change":"2024-02-02T15:24:51.819941+0000","last_active":"2024-02-02T15:25:26.280381+0000","last_peered":"2024-02-02T15:25:26.280381+0000","last_clean":"2024-02-02T15:25:26.280381+0000","last_became_active":"2024-02-02T15:24:51.818066+0000","last_became_peered":"2024-02-02T15:24:51.818066+0000","last_unstale":"2024-02-02T15:25:26.280381+0000","last_undegraded":"2024-02-02T15:25:26.280381+0000","last_fullsized":"2024-02-02T15:25:26.280381+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:55:51.550988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"84'4","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.276022+0000","last_change":"2024-02-02T15:24:47.268541+0000","last_active":"2024-02-02T15:25:26.276022+0000","last_peered":"2024-02-02T15:25:26.276022+0000","last_clean":"2024-02-02T15:25:26.276022+0000","last_became_active":"2024-02-02T15:24:47.268184+0000","last_became_peered":"2024-02-02T15:24:47.268184+0000","last_unstale":"2024-02-02T15:25:26.276022+0000","last_undegraded":"2024-02-02T15:25:26.276022+0000","last_fullsized":"2024-02-02T15:25:26.276022+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:30:12.187297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.280697+0000","last_change":"2024-02-02T15:24:47.787792+0000","last_active":"2024-02-02T15:25:26.280697+0000","last_peered":"2024-02-02T15:25:26.280697+0000","last_clean":"2024-02-02T15:25:26.280697+0000","last_became_active":"2024-02-02T15:24:47.787479+0000","last_became_peered":"2024-02-02T15:24:47.787479+0000","last_unstale":"2024-02-02T15:25:26.280697+0000","last_undegraded":"2024-02-02T15:25:26.280697+0000","last_fullsized":"2024-02-02T15:25:26.280697+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T03:24:06.544232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.276031+0000","last_change":"2024-02-02T15:24:51.826639+0000","last_active":"2024-02-02T15:25:26.276031+0000","last_peered":"2024-02-02T15:25:26.276031+0000","last_clean":"2024-02-02T15:25:26.276031+0000","last_became_active":"2024-02-02T15:24:51.825089+0000","last_became_peered":"2024-02-02T15:24:51.825089+0000","last_unstale":"2024-02-02T15:25:26.276031+0000","last_undegraded":"2024-02-02T15:25:26.276031+0000","last_fullsized":"2024-02-02T15:25:26.276031+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:41:44.592603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.272658+0000","last_change":"2024-02-02T15:24:47.281490+0000","last_active":"2024-02-02T15:25:26.272658+0000","last_peered":"2024-02-02T15:25:26.272658+0000","last_clean":"2024-02-02T15:25:26.272658+0000","last_became_active":"2024-02-02T15:24:47.281137+0000","last_became_peered":"2024-02-02T15:24:47.281137+0000","last_unstale":"2024-02-02T15:25:26.272658+0000","last_undegraded":"2024-02-02T15:25:26.272658+0000","last_fullsized":"2024-02-02T15:25:26.272658+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:42:08.508881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.273131+0000","last_change":"2024-02-02T15:24:47.781135+0000","last_active":"2024-02-02T15:25:26.273131+0000","last_peered":"2024-02-02T15:25:26.273131+0000","last_clean":"2024-02-02T15:25:26.273131+0000","last_became_active":"2024-02-02T15:24:47.780817+0000","last_became_peered":"2024-02-02T15:24:47.780817+0000","last_unstale":"2024-02-02T15:25:26.273131+0000","last_undegraded":"2024-02-02T15:25:26.273131+0000","last_fullsized":"2024-02-02T15:25:26.273131+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:33:05.862934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T15:25:27.286403+0000","last_change":"2024-02-02T15:25:27.286403+0000","last_active":"2024-02-02T15:25:27.286403+0000","last_peered":"2024-02-02T15:25:27.286403+0000","last_clean":"2024-02-02T15:25:27.286403+0000","last_became_active":"2024-02-02T15:25:27.285824+0000","last_became_peered":"2024-02-02T15:25:27.285824+0000","last_unstale":"2024-02-02T15:25:27.286403+0000","last_undegraded":"2024-02-02T15:25:27.286403+0000","last_fullsized":"2024-02-02T15:25:27.286403+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:11:25.986947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,10],"acting":[9,8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"86'35","reported_seq":143,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T15:26:00.327968+0000","last_change":"2024-02-02T15:24:47.270604+0000","last_active":"2024-02-02T15:26:00.327968+0000","last_peered":"2024-02-02T15:26:00.327968+0000","last_clean":"2024-02-02T15:26:00.327968+0000","last_became_active":"2024-02-02T15:24:47.270261+0000","last_became_peered":"2024-02-02T15:24:47.270261+0000","last_unstale":"2024-02-02T15:26:00.327968+0000","last_undegraded":"2024-02-02T15:26:00.327968+0000","last_fullsized":"2024-02-02T15:26:00.327968+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:56:33.767423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60396,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":90,"num_read_kb":105,"num_write":37,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.278171+0000","last_change":"2024-02-02T15:24:47.789663+0000","last_active":"2024-02-02T15:25:26.278171+0000","last_peered":"2024-02-02T15:25:26.278171+0000","last_clean":"2024-02-02T15:25:26.278171+0000","last_became_active":"2024-02-02T15:24:47.789352+0000","last_became_peered":"2024-02-02T15:24:47.789352+0000","last_unstale":"2024-02-02T15:25:26.278171+0000","last_undegraded":"2024-02-02T15:25:26.278171+0000","last_fullsized":"2024-02-02T15:25:26.278171+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:13:37.991350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.273255+0000","last_change":"2024-02-02T15:24:51.816021+0000","last_active":"2024-02-02T15:25:26.273255+0000","last_peered":"2024-02-02T15:25:26.273255+0000","last_clean":"2024-02-02T15:25:26.273255+0000","last_became_active":"2024-02-02T15:24:51.815698+0000","last_became_peered":"2024-02-02T15:24:51.815698+0000","last_unstale":"2024-02-02T15:25:26.273255+0000","last_undegraded":"2024-02-02T15:25:26.273255+0000","last_fullsized":"2024-02-02T15:25:26.273255+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:11:10.670195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"84'9","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T15:25:27.295680+0000","last_change":"2024-02-02T15:25:27.295339+0000","last_active":"2024-02-02T15:25:27.295680+0000","last_peered":"2024-02-02T15:25:27.295680+0000","last_clean":"2024-02-02T15:25:27.295680+0000","last_became_active":"2024-02-02T15:25:27.281605+0000","last_became_peered":"2024-02-02T15:25:27.281605+0000","last_unstale":"2024-02-02T15:25:27.295680+0000","last_undegraded":"2024-02-02T15:25:27.295680+0000","last_fullsized":"2024-02-02T15:25:27.295680+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T02:35:36.952261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.271927+0000","last_change":"2024-02-02T15:24:47.798089+0000","last_active":"2024-02-02T15:25:26.271927+0000","last_peered":"2024-02-02T15:25:26.271927+0000","last_clean":"2024-02-02T15:25:26.271927+0000","last_became_active":"2024-02-02T15:24:47.793914+0000","last_became_peered":"2024-02-02T15:24:47.793914+0000","last_unstale":"2024-02-02T15:25:26.271927+0000","last_undegraded":"2024-02-02T15:25:26.271927+0000","last_fullsized":"2024-02-02T15:25:26.271927+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:33:39.927550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.276733+0000","last_change":"2024-02-02T15:24:51.823098+0000","last_active":"2024-02-02T15:25:26.276733+0000","last_peered":"2024-02-02T15:25:26.276733+0000","last_clean":"2024-02-02T15:25:26.276733+0000","last_became_active":"2024-02-02T15:24:51.822634+0000","last_became_peered":"2024-02-02T15:24:51.822634+0000","last_unstale":"2024-02-02T15:25:26.276733+0000","last_undegraded":"2024-02-02T15:25:26.276733+0000","last_fullsized":"2024-02-02T15:25:26.276733+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T03:18:07.420196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"84'5","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.272888+0000","last_change":"2024-02-02T15:24:47.268962+0000","last_active":"2024-02-02T15:25:26.272888+0000","last_peered":"2024-02-02T15:25:26.272888+0000","last_clean":"2024-02-02T15:25:26.272888+0000","last_became_active":"2024-02-02T15:24:47.268607+0000","last_became_peered":"2024-02-02T15:24:47.268607+0000","last_unstale":"2024-02-02T15:25:26.272888+0000","last_undegraded":"2024-02-02T15:25:26.272888+0000","last_fullsized":"2024-02-02T15:25:26.272888+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:31:06.330437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.275646+0000","last_change":"2024-02-02T15:24:47.792205+0000","last_active":"2024-02-02T15:25:26.275646+0000","last_peered":"2024-02-02T15:25:26.275646+0000","last_clean":"2024-02-02T15:25:26.275646+0000","last_became_active":"2024-02-02T15:24:47.791874+0000","last_became_peered":"2024-02-02T15:24:47.791874+0000","last_unstale":"2024-02-02T15:25:26.275646+0000","last_undegraded":"2024-02-02T15:25:26.275646+0000","last_fullsized":"2024-02-02T15:25:26.275646+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:33:38.795921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.282390+0000","last_change":"2024-02-02T15:24:51.825499+0000","last_active":"2024-02-02T15:25:26.282390+0000","last_peered":"2024-02-02T15:25:26.282390+0000","last_clean":"2024-02-02T15:25:26.282390+0000","last_became_active":"2024-02-02T15:24:51.825049+0000","last_became_peered":"2024-02-02T15:24:51.825049+0000","last_unstale":"2024-02-02T15:25:26.282390+0000","last_undegraded":"2024-02-02T15:25:26.282390+0000","last_fullsized":"2024-02-02T15:25:26.282390+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:41:05.430891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"84'6","reported_seq":66,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T15:25:58.323282+0000","last_change":"2024-02-02T15:24:47.267683+0000","last_active":"2024-02-02T15:25:58.323282+0000","last_peered":"2024-02-02T15:25:58.323282+0000","last_clean":"2024-02-02T15:25:58.323282+0000","last_became_active":"2024-02-02T15:24:47.267324+0000","last_became_peered":"2024-02-02T15:24:47.267324+0000","last_unstale":"2024-02-02T15:25:58.323282+0000","last_undegraded":"2024-02-02T15:25:58.323282+0000","last_fullsized":"2024-02-02T15:25:58.323282+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:03:31.326791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":932,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":43,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.277864+0000","last_change":"2024-02-02T15:24:47.792509+0000","last_active":"2024-02-02T15:25:26.277864+0000","last_peered":"2024-02-02T15:25:26.277864+0000","last_clean":"2024-02-02T15:25:26.277864+0000","last_became_active":"2024-02-02T15:24:47.791662+0000","last_became_peered":"2024-02-02T15:24:47.791662+0000","last_unstale":"2024-02-02T15:25:26.277864+0000","last_undegraded":"2024-02-02T15:25:26.277864+0000","last_fullsized":"2024-02-02T15:25:26.277864+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:42:50.009971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T15:25:27.282514+0000","last_change":"2024-02-02T15:25:27.282514+0000","last_active":"2024-02-02T15:25:27.282514+0000","last_peered":"2024-02-02T15:25:27.282514+0000","last_clean":"2024-02-02T15:25:27.282514+0000","last_became_active":"2024-02-02T15:25:27.281336+0000","last_became_peered":"2024-02-02T15:25:27.281336+0000","last_unstale":"2024-02-02T15:25:27.282514+0000","last_undegraded":"2024-02-02T15:25:27.282514+0000","last_fullsized":"2024-02-02T15:25:27.282514+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:50:39.728813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"84'17","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.276131+0000","last_change":"2024-02-02T15:24:47.274152+0000","last_active":"2024-02-02T15:25:26.276131+0000","last_peered":"2024-02-02T15:25:26.276131+0000","last_clean":"2024-02-02T15:25:26.276131+0000","last_became_active":"2024-02-02T15:24:47.273720+0000","last_became_peered":"2024-02-02T15:24:47.273720+0000","last_unstale":"2024-02-02T15:25:26.276131+0000","last_undegraded":"2024-02-02T15:25:26.276131+0000","last_fullsized":"2024-02-02T15:25:26.276131+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:38:05.280309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.272549+0000","last_change":"2024-02-02T15:24:47.790681+0000","last_active":"2024-02-02T15:25:26.272549+0000","last_peered":"2024-02-02T15:25:26.272549+0000","last_clean":"2024-02-02T15:25:26.272549+0000","last_became_active":"2024-02-02T15:24:47.789952+0000","last_became_peered":"2024-02-02T15:24:47.789952+0000","last_unstale":"2024-02-02T15:25:26.272549+0000","last_undegraded":"2024-02-02T15:25:26.272549+0000","last_fullsized":"2024-02-02T15:25:26.272549+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:21:03.103398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.272725+0000","last_change":"2024-02-02T15:24:51.812676+0000","last_active":"2024-02-02T15:25:26.272725+0000","last_peered":"2024-02-02T15:25:26.272725+0000","last_clean":"2024-02-02T15:25:26.272725+0000","last_became_active":"2024-02-02T15:24:51.812212+0000","last_became_peered":"2024-02-02T15:24:51.812212+0000","last_unstale":"2024-02-02T15:25:26.272725+0000","last_undegraded":"2024-02-02T15:25:26.272725+0000","last_fullsized":"2024-02-02T15:25:26.272725+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:48:01.944379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'2","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.281427+0000","last_change":"2024-02-02T15:24:47.271707+0000","last_active":"2024-02-02T15:25:26.281427+0000","last_peered":"2024-02-02T15:25:26.281427+0000","last_clean":"2024-02-02T15:25:26.281427+0000","last_became_active":"2024-02-02T15:24:47.271379+0000","last_became_peered":"2024-02-02T15:24:47.271379+0000","last_unstale":"2024-02-02T15:25:26.281427+0000","last_undegraded":"2024-02-02T15:25:26.281427+0000","last_fullsized":"2024-02-02T15:25:26.281427+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:00:24.571385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.276273+0000","last_change":"2024-02-02T15:24:47.788360+0000","last_active":"2024-02-02T15:25:26.276273+0000","last_peered":"2024-02-02T15:25:26.276273+0000","last_clean":"2024-02-02T15:25:26.276273+0000","last_became_active":"2024-02-02T15:24:47.787945+0000","last_became_peered":"2024-02-02T15:24:47.787945+0000","last_unstale":"2024-02-02T15:25:26.276273+0000","last_undegraded":"2024-02-02T15:25:26.276273+0000","last_fullsized":"2024-02-02T15:25:26.276273+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:37:54.531487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.277086+0000","last_change":"2024-02-02T15:24:51.821320+0000","last_active":"2024-02-02T15:25:26.277086+0000","last_peered":"2024-02-02T15:25:26.277086+0000","last_clean":"2024-02-02T15:25:26.277086+0000","last_became_active":"2024-02-02T15:24:51.820853+0000","last_became_peered":"2024-02-02T15:24:51.820853+0000","last_unstale":"2024-02-02T15:25:26.277086+0000","last_undegraded":"2024-02-02T15:25:26.277086+0000","last_fullsized":"2024-02-02T15:25:26.277086+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:20:12.304517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"84'1","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.282116+0000","last_change":"2024-02-02T15:24:47.278291+0000","last_active":"2024-02-02T15:25:26.282116+0000","last_peered":"2024-02-02T15:25:26.282116+0000","last_clean":"2024-02-02T15:25:26.282116+0000","last_became_active":"2024-02-02T15:24:47.276163+0000","last_became_peered":"2024-02-02T15:24:47.276163+0000","last_unstale":"2024-02-02T15:25:26.282116+0000","last_undegraded":"2024-02-02T15:25:26.282116+0000","last_fullsized":"2024-02-02T15:25:26.282116+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:53:50.312627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.277505+0000","last_change":"2024-02-02T15:24:47.791290+0000","last_active":"2024-02-02T15:25:26.277505+0000","last_peered":"2024-02-02T15:25:26.277505+0000","last_clean":"2024-02-02T15:25:26.277505+0000","last_became_active":"2024-02-02T15:24:47.790987+0000","last_became_peered":"2024-02-02T15:24:47.790987+0000","last_unstale":"2024-02-02T15:25:26.277505+0000","last_undegraded":"2024-02-02T15:25:26.277505+0000","last_fullsized":"2024-02-02T15:25:26.277505+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:38:08.803338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.271792+0000","last_change":"2024-02-02T15:24:51.813050+0000","last_active":"2024-02-02T15:25:26.271792+0000","last_peered":"2024-02-02T15:25:26.271792+0000","last_clean":"2024-02-02T15:25:26.271792+0000","last_became_active":"2024-02-02T15:24:51.812634+0000","last_became_peered":"2024-02-02T15:24:51.812634+0000","last_unstale":"2024-02-02T15:25:26.271792+0000","last_undegraded":"2024-02-02T15:25:26.271792+0000","last_fullsized":"2024-02-02T15:25:26.271792+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:06:56.428431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"84'5","reported_seq":65,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T15:25:59.335894+0000","last_change":"2024-02-02T15:24:47.274033+0000","last_active":"2024-02-02T15:25:59.335894+0000","last_peered":"2024-02-02T15:25:59.335894+0000","last_clean":"2024-02-02T15:25:59.335894+0000","last_became_active":"2024-02-02T15:24:47.273672+0000","last_became_peered":"2024-02-02T15:24:47.273672+0000","last_unstale":"2024-02-02T15:25:59.335894+0000","last_undegraded":"2024-02-02T15:25:59.335894+0000","last_fullsized":"2024-02-02T15:25:59.335894+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:51:46.926397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":42,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.277973+0000","last_change":"2024-02-02T15:24:47.786533+0000","last_active":"2024-02-02T15:25:26.277973+0000","last_peered":"2024-02-02T15:25:26.277973+0000","last_clean":"2024-02-02T15:25:26.277973+0000","last_became_active":"2024-02-02T15:24:47.785436+0000","last_became_peered":"2024-02-02T15:24:47.785436+0000","last_unstale":"2024-02-02T15:25:26.277973+0000","last_undegraded":"2024-02-02T15:25:26.277973+0000","last_fullsized":"2024-02-02T15:25:26.277973+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:23:45.910800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.272128+0000","last_change":"2024-02-02T15:24:51.814326+0000","last_active":"2024-02-02T15:25:26.272128+0000","last_peered":"2024-02-02T15:25:26.272128+0000","last_clean":"2024-02-02T15:25:26.272128+0000","last_became_active":"2024-02-02T15:24:51.813909+0000","last_became_peered":"2024-02-02T15:24:51.813909+0000","last_unstale":"2024-02-02T15:25:26.272128+0000","last_undegraded":"2024-02-02T15:25:26.272128+0000","last_fullsized":"2024-02-02T15:25:26.272128+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:16:12.902718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T15:25:27.287818+0000","last_change":"2024-02-02T15:25:27.287818+0000","last_active":"2024-02-02T15:25:27.287818+0000","last_peered":"2024-02-02T15:25:27.287818+0000","last_clean":"2024-02-02T15:25:27.287818+0000","last_became_active":"2024-02-02T15:25:27.287481+0000","last_became_peered":"2024-02-02T15:25:27.287481+0000","last_unstale":"2024-02-02T15:25:27.287818+0000","last_undegraded":"2024-02-02T15:25:27.287818+0000","last_fullsized":"2024-02-02T15:25:27.287818+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:14:32.820969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"84'5","reported_seq":62,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T15:25:56.317891+0000","last_change":"2024-02-02T15:24:47.274437+0000","last_active":"2024-02-02T15:25:56.317891+0000","last_peered":"2024-02-02T15:25:56.317891+0000","last_clean":"2024-02-02T15:25:56.317891+0000","last_became_active":"2024-02-02T15:24:47.274010+0000","last_became_peered":"2024-02-02T15:24:47.274010+0000","last_unstale":"2024-02-02T15:25:56.317891+0000","last_undegraded":"2024-02-02T15:25:56.317891+0000","last_fullsized":"2024-02-02T15:25:56.317891+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:10:36.862190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":39,"num_read_kb":39,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"84'1","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.275771+0000","last_change":"2024-02-02T15:24:51.825698+0000","last_active":"2024-02-02T15:25:26.275771+0000","last_peered":"2024-02-02T15:25:26.275771+0000","last_clean":"2024-02-02T15:25:26.275771+0000","last_became_active":"2024-02-02T15:24:51.825295+0000","last_became_peered":"2024-02-02T15:24:51.825295+0000","last_unstale":"2024-02-02T15:25:26.275771+0000","last_undegraded":"2024-02-02T15:25:26.275771+0000","last_fullsized":"2024-02-02T15:25:26.275771+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:17:09.838124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.276296+0000","last_change":"2024-02-02T15:24:47.781836+0000","last_active":"2024-02-02T15:25:26.276296+0000","last_peered":"2024-02-02T15:25:26.276296+0000","last_clean":"2024-02-02T15:25:26.276296+0000","last_became_active":"2024-02-02T15:24:47.781472+0000","last_became_peered":"2024-02-02T15:24:47.781472+0000","last_unstale":"2024-02-02T15:25:26.276296+0000","last_undegraded":"2024-02-02T15:25:26.276296+0000","last_fullsized":"2024-02-02T15:25:26.276296+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:46:33.714191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"84'6","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.275947+0000","last_change":"2024-02-02T15:24:47.269087+0000","last_active":"2024-02-02T15:25:26.275947+0000","last_peered":"2024-02-02T15:25:26.275947+0000","last_clean":"2024-02-02T15:25:26.275947+0000","last_became_active":"2024-02-02T15:24:47.268743+0000","last_became_peered":"2024-02-02T15:24:47.268743+0000","last_unstale":"2024-02-02T15:25:26.275947+0000","last_undegraded":"2024-02-02T15:25:26.275947+0000","last_fullsized":"2024-02-02T15:25:26.275947+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:03:04.323079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":336,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.277648+0000","last_change":"2024-02-02T15:23:55.191735+0000","last_active":"2024-02-02T15:25:26.277648+0000","last_peered":"2024-02-02T15:25:26.277648+0000","last_clean":"2024-02-02T15:25:26.277648+0000","last_became_active":"2024-02-02T15:23:55.191119+0000","last_became_peered":"2024-02-02T15:23:55.191119+0000","last_unstale":"2024-02-02T15:25:26.277648+0000","last_undegraded":"2024-02-02T15:25:26.277648+0000","last_fullsized":"2024-02-02T15:25:26.277648+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:21:22.472631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:21:22.472631+0000","last_clean_scrub_stamp":"2024-02-02T15:21:22.472631+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:24:44.153461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.273124+0000","last_change":"2024-02-02T15:24:51.822842+0000","last_active":"2024-02-02T15:25:26.273124+0000","last_peered":"2024-02-02T15:25:26.273124+0000","last_clean":"2024-02-02T15:25:26.273124+0000","last_became_active":"2024-02-02T15:24:51.822372+0000","last_became_peered":"2024-02-02T15:24:51.822372+0000","last_unstale":"2024-02-02T15:25:26.273124+0000","last_undegraded":"2024-02-02T15:25:26.273124+0000","last_fullsized":"2024-02-02T15:25:26.273124+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:45:29.631422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.271614+0000","last_change":"2024-02-02T15:24:47.786507+0000","last_active":"2024-02-02T15:25:26.271614+0000","last_peered":"2024-02-02T15:25:26.271614+0000","last_clean":"2024-02-02T15:25:26.271614+0000","last_became_active":"2024-02-02T15:24:47.786125+0000","last_became_peered":"2024-02-02T15:24:47.786125+0000","last_unstale":"2024-02-02T15:25:26.271614+0000","last_undegraded":"2024-02-02T15:25:26.271614+0000","last_fullsized":"2024-02-02T15:25:26.271614+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:33:09.014258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"84'4","reported_seq":35,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T15:25:27.628074+0000","last_change":"2024-02-02T15:25:27.628074+0000","last_active":"2024-02-02T15:25:27.628074+0000","last_peered":"2024-02-02T15:25:27.628074+0000","last_clean":"2024-02-02T15:25:27.628074+0000","last_became_active":"2024-02-02T15:25:27.625103+0000","last_became_peered":"2024-02-02T15:25:27.625103+0000","last_unstale":"2024-02-02T15:25:27.628074+0000","last_undegraded":"2024-02-02T15:25:27.628074+0000","last_fullsized":"2024-02-02T15:25:27.628074+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:30:35.540456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.276587+0000","last_change":"2024-02-02T15:24:51.808979+0000","last_active":"2024-02-02T15:25:26.276587+0000","last_peered":"2024-02-02T15:25:26.276587+0000","last_clean":"2024-02-02T15:25:26.276587+0000","last_became_active":"2024-02-02T15:24:51.808557+0000","last_became_peered":"2024-02-02T15:24:51.808557+0000","last_unstale":"2024-02-02T15:25:26.276587+0000","last_undegraded":"2024-02-02T15:25:26.276587+0000","last_fullsized":"2024-02-02T15:25:26.276587+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T02:38:34.697137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"84'4","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.278273+0000","last_change":"2024-02-02T15:24:47.281396+0000","last_active":"2024-02-02T15:25:26.278273+0000","last_peered":"2024-02-02T15:25:26.278273+0000","last_clean":"2024-02-02T15:25:26.278273+0000","last_became_active":"2024-02-02T15:24:47.281036+0000","last_became_peered":"2024-02-02T15:24:47.281036+0000","last_unstale":"2024-02-02T15:25:26.278273+0000","last_undegraded":"2024-02-02T15:25:26.278273+0000","last_fullsized":"2024-02-02T15:25:26.278273+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:54:47.139151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.271949+0000","last_change":"2024-02-02T15:24:47.791057+0000","last_active":"2024-02-02T15:25:26.271949+0000","last_peered":"2024-02-02T15:25:26.271949+0000","last_clean":"2024-02-02T15:25:26.271949+0000","last_became_active":"2024-02-02T15:24:47.790647+0000","last_became_peered":"2024-02-02T15:24:47.790647+0000","last_unstale":"2024-02-02T15:25:26.271949+0000","last_undegraded":"2024-02-02T15:25:26.271949+0000","last_fullsized":"2024-02-02T15:25:26.271949+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T02:49:42.067700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.276636+0000","last_change":"2024-02-02T15:24:51.817792+0000","last_active":"2024-02-02T15:25:26.276636+0000","last_peered":"2024-02-02T15:25:26.276636+0000","last_clean":"2024-02-02T15:25:26.276636+0000","last_became_active":"2024-02-02T15:24:51.815800+0000","last_became_peered":"2024-02-02T15:24:51.815800+0000","last_unstale":"2024-02-02T15:25:26.276636+0000","last_undegraded":"2024-02-02T15:25:26.276636+0000","last_fullsized":"2024-02-02T15:25:26.276636+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:28:10.440251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"86'10","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T15:25:27.280528+0000","last_change":"2024-02-02T15:24:47.277098+0000","last_active":"2024-02-02T15:25:27.280528+0000","last_peered":"2024-02-02T15:25:27.280528+0000","last_clean":"2024-02-02T15:25:27.280528+0000","last_became_active":"2024-02-02T15:24:47.275867+0000","last_became_peered":"2024-02-02T15:24:47.275867+0000","last_unstale":"2024-02-02T15:25:27.280528+0000","last_undegraded":"2024-02-02T15:25:27.280528+0000","last_fullsized":"2024-02-02T15:25:27.280528+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:35:21.015918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.277243+0000","last_change":"2024-02-02T15:24:47.784873+0000","last_active":"2024-02-02T15:25:26.277243+0000","last_peered":"2024-02-02T15:25:26.277243+0000","last_clean":"2024-02-02T15:25:26.277243+0000","last_became_active":"2024-02-02T15:24:47.784550+0000","last_became_peered":"2024-02-02T15:24:47.784550+0000","last_unstale":"2024-02-02T15:25:26.277243+0000","last_undegraded":"2024-02-02T15:25:26.277243+0000","last_fullsized":"2024-02-02T15:25:26.277243+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:13:37.513754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.279176+0000","last_change":"2024-02-02T15:24:51.805131+0000","last_active":"2024-02-02T15:25:26.279176+0000","last_peered":"2024-02-02T15:25:26.279176+0000","last_clean":"2024-02-02T15:25:26.279176+0000","last_became_active":"2024-02-02T15:24:51.804705+0000","last_became_peered":"2024-02-02T15:24:51.804705+0000","last_unstale":"2024-02-02T15:25:26.279176+0000","last_undegraded":"2024-02-02T15:25:26.279176+0000","last_fullsized":"2024-02-02T15:25:26.279176+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T02:12:59.945050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"84'7","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.276174+0000","last_change":"2024-02-02T15:24:47.266469+0000","last_active":"2024-02-02T15:25:26.276174+0000","last_peered":"2024-02-02T15:25:26.276174+0000","last_clean":"2024-02-02T15:25:26.276174+0000","last_became_active":"2024-02-02T15:24:47.266094+0000","last_became_peered":"2024-02-02T15:24:47.266094+0000","last_unstale":"2024-02-02T15:25:26.276174+0000","last_undegraded":"2024-02-02T15:25:26.276174+0000","last_fullsized":"2024-02-02T15:25:26.276174+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:51:41.324698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.281014+0000","last_change":"2024-02-02T15:24:47.793051+0000","last_active":"2024-02-02T15:25:26.281014+0000","last_peered":"2024-02-02T15:25:26.281014+0000","last_clean":"2024-02-02T15:25:26.281014+0000","last_became_active":"2024-02-02T15:24:47.792645+0000","last_became_peered":"2024-02-02T15:24:47.792645+0000","last_unstale":"2024-02-02T15:25:26.281014+0000","last_undegraded":"2024-02-02T15:25:26.281014+0000","last_fullsized":"2024-02-02T15:25:26.281014+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:47:17.229407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T15:25:27.282505+0000","last_change":"2024-02-02T15:25:27.282505+0000","last_active":"2024-02-02T15:25:27.282505+0000","last_peered":"2024-02-02T15:25:27.282505+0000","last_clean":"2024-02-02T15:25:27.282505+0000","last_became_active":"2024-02-02T15:25:27.282039+0000","last_became_peered":"2024-02-02T15:25:27.282039+0000","last_unstale":"2024-02-02T15:25:27.282505+0000","last_undegraded":"2024-02-02T15:25:27.282505+0000","last_fullsized":"2024-02-02T15:25:27.282505+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:31:29.700848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"84'2","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.272816+0000","last_change":"2024-02-02T15:24:47.273716+0000","last_active":"2024-02-02T15:25:26.272816+0000","last_peered":"2024-02-02T15:25:26.272816+0000","last_clean":"2024-02-02T15:25:26.272816+0000","last_became_active":"2024-02-02T15:24:47.273438+0000","last_became_peered":"2024-02-02T15:24:47.273438+0000","last_unstale":"2024-02-02T15:25:26.272816+0000","last_undegraded":"2024-02-02T15:25:26.272816+0000","last_fullsized":"2024-02-02T15:25:26.272816+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:52:16.498119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.275707+0000","last_change":"2024-02-02T15:24:47.782950+0000","last_active":"2024-02-02T15:25:26.275707+0000","last_peered":"2024-02-02T15:25:26.275707+0000","last_clean":"2024-02-02T15:25:26.275707+0000","last_became_active":"2024-02-02T15:24:47.782605+0000","last_became_peered":"2024-02-02T15:24:47.782605+0000","last_unstale":"2024-02-02T15:25:26.275707+0000","last_undegraded":"2024-02-02T15:25:26.275707+0000","last_fullsized":"2024-02-02T15:25:26.275707+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:57:34.125985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T15:25:27.288805+0000","last_change":"2024-02-02T15:25:27.288179+0000","last_active":"2024-02-02T15:25:27.288805+0000","last_peered":"2024-02-02T15:25:27.288805+0000","last_clean":"2024-02-02T15:25:27.288805+0000","last_became_active":"2024-02-02T15:25:27.287753+0000","last_became_peered":"2024-02-02T15:25:27.287753+0000","last_unstale":"2024-02-02T15:25:27.288805+0000","last_undegraded":"2024-02-02T15:25:27.288805+0000","last_fullsized":"2024-02-02T15:25:27.288805+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:02:46.854950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"84'7","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.278283+0000","last_change":"2024-02-02T15:24:47.278478+0000","last_active":"2024-02-02T15:25:26.278283+0000","last_peered":"2024-02-02T15:25:26.278283+0000","last_clean":"2024-02-02T15:25:26.278283+0000","last_became_active":"2024-02-02T15:24:47.278084+0000","last_became_peered":"2024-02-02T15:24:47.278084+0000","last_unstale":"2024-02-02T15:25:26.278283+0000","last_undegraded":"2024-02-02T15:25:26.278283+0000","last_fullsized":"2024-02-02T15:25:26.278283+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:10:23.044968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.276242+0000","last_change":"2024-02-02T15:24:47.795291+0000","last_active":"2024-02-02T15:25:26.276242+0000","last_peered":"2024-02-02T15:25:26.276242+0000","last_clean":"2024-02-02T15:25:26.276242+0000","last_became_active":"2024-02-02T15:24:47.794514+0000","last_became_peered":"2024-02-02T15:24:47.794514+0000","last_unstale":"2024-02-02T15:25:26.276242+0000","last_undegraded":"2024-02-02T15:25:26.276242+0000","last_fullsized":"2024-02-02T15:25:26.276242+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:32:12.722258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"84'1","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.275640+0000","last_change":"2024-02-02T15:24:51.825985+0000","last_active":"2024-02-02T15:25:26.275640+0000","last_peered":"2024-02-02T15:25:26.275640+0000","last_clean":"2024-02-02T15:25:26.275640+0000","last_became_active":"2024-02-02T15:24:51.825588+0000","last_became_peered":"2024-02-02T15:24:51.825588+0000","last_unstale":"2024-02-02T15:25:26.275640+0000","last_undegraded":"2024-02-02T15:25:26.275640+0000","last_fullsized":"2024-02-02T15:25:26.275640+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:32:39.910258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"84'3","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T15:25:27.298932+0000","last_change":"2024-02-02T15:25:27.298553+0000","last_active":"2024-02-02T15:25:27.298932+0000","last_peered":"2024-02-02T15:25:27.298932+0000","last_clean":"2024-02-02T15:25:27.298932+0000","last_became_active":"2024-02-02T15:25:27.282469+0000","last_became_peered":"2024-02-02T15:25:27.282469+0000","last_unstale":"2024-02-02T15:25:27.298932+0000","last_undegraded":"2024-02-02T15:25:27.298932+0000","last_fullsized":"2024-02-02T15:25:27.298932+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:54:14.740951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.276116+0000","last_change":"2024-02-02T15:24:47.784188+0000","last_active":"2024-02-02T15:25:26.276116+0000","last_peered":"2024-02-02T15:25:26.276116+0000","last_clean":"2024-02-02T15:25:26.276116+0000","last_became_active":"2024-02-02T15:24:47.783872+0000","last_became_peered":"2024-02-02T15:24:47.783872+0000","last_unstale":"2024-02-02T15:25:26.276116+0000","last_undegraded":"2024-02-02T15:25:26.276116+0000","last_fullsized":"2024-02-02T15:25:26.276116+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:45:45.213255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.275577+0000","last_change":"2024-02-02T15:24:51.815263+0000","last_active":"2024-02-02T15:25:26.275577+0000","last_peered":"2024-02-02T15:25:26.275577+0000","last_clean":"2024-02-02T15:25:26.275577+0000","last_became_active":"2024-02-02T15:24:51.814849+0000","last_became_peered":"2024-02-02T15:24:51.814849+0000","last_unstale":"2024-02-02T15:25:26.275577+0000","last_undegraded":"2024-02-02T15:25:26.275577+0000","last_fullsized":"2024-02-02T15:25:26.275577+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:05:32.987359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"84'4","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.277429+0000","last_change":"2024-02-02T15:24:47.270534+0000","last_active":"2024-02-02T15:25:26.277429+0000","last_peered":"2024-02-02T15:25:26.277429+0000","last_clean":"2024-02-02T15:25:26.277429+0000","last_became_active":"2024-02-02T15:24:47.270110+0000","last_became_peered":"2024-02-02T15:24:47.270110+0000","last_unstale":"2024-02-02T15:25:26.277429+0000","last_undegraded":"2024-02-02T15:25:26.277429+0000","last_fullsized":"2024-02-02T15:25:26.277429+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T02:59:52.297315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.275557+0000","last_change":"2024-02-02T15:24:47.791826+0000","last_active":"2024-02-02T15:25:26.275557+0000","last_peered":"2024-02-02T15:25:26.275557+0000","last_clean":"2024-02-02T15:25:26.275557+0000","last_became_active":"2024-02-02T15:24:47.791016+0000","last_became_peered":"2024-02-02T15:24:47.791016+0000","last_unstale":"2024-02-02T15:25:26.275557+0000","last_undegraded":"2024-02-02T15:25:26.275557+0000","last_fullsized":"2024-02-02T15:25:26.275557+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:47:14.296876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.272974+0000","last_change":"2024-02-02T15:24:51.806741+0000","last_active":"2024-02-02T15:25:26.272974+0000","last_peered":"2024-02-02T15:25:26.272974+0000","last_clean":"2024-02-02T15:25:26.272974+0000","last_became_active":"2024-02-02T15:24:51.806085+0000","last_became_peered":"2024-02-02T15:24:51.806085+0000","last_unstale":"2024-02-02T15:25:26.272974+0000","last_undegraded":"2024-02-02T15:25:26.272974+0000","last_fullsized":"2024-02-02T15:25:26.272974+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:53:39.944222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"84'5","reported_seq":62,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T15:25:55.326324+0000","last_change":"2024-02-02T15:24:47.278515+0000","last_active":"2024-02-02T15:25:55.326324+0000","last_peered":"2024-02-02T15:25:55.326324+0000","last_clean":"2024-02-02T15:25:55.326324+0000","last_became_active":"2024-02-02T15:24:47.278145+0000","last_became_peered":"2024-02-02T15:24:47.278145+0000","last_unstale":"2024-02-02T15:25:55.326324+0000","last_undegraded":"2024-02-02T15:25:55.326324+0000","last_fullsized":"2024-02-02T15:25:55.326324+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:48:55.118003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":39,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.275926+0000","last_change":"2024-02-02T15:24:47.791996+0000","last_active":"2024-02-02T15:25:26.275926+0000","last_peered":"2024-02-02T15:25:26.275926+0000","last_clean":"2024-02-02T15:25:26.275926+0000","last_became_active":"2024-02-02T15:24:47.791624+0000","last_became_peered":"2024-02-02T15:24:47.791624+0000","last_unstale":"2024-02-02T15:25:26.275926+0000","last_undegraded":"2024-02-02T15:25:26.275926+0000","last_fullsized":"2024-02-02T15:25:26.275926+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:33:27.963740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.279130+0000","last_change":"2024-02-02T15:24:51.826835+0000","last_active":"2024-02-02T15:25:26.279130+0000","last_peered":"2024-02-02T15:25:26.279130+0000","last_clean":"2024-02-02T15:25:26.279130+0000","last_became_active":"2024-02-02T15:24:51.826310+0000","last_became_peered":"2024-02-02T15:24:51.826310+0000","last_unstale":"2024-02-02T15:25:26.279130+0000","last_undegraded":"2024-02-02T15:25:26.279130+0000","last_fullsized":"2024-02-02T15:25:26.279130+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:48:16.687048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'3","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.272976+0000","last_change":"2024-02-02T15:24:47.268865+0000","last_active":"2024-02-02T15:25:26.272976+0000","last_peered":"2024-02-02T15:25:26.272976+0000","last_clean":"2024-02-02T15:25:26.272976+0000","last_became_active":"2024-02-02T15:24:47.268523+0000","last_became_peered":"2024-02-02T15:24:47.268523+0000","last_unstale":"2024-02-02T15:25:26.272976+0000","last_undegraded":"2024-02-02T15:25:26.272976+0000","last_fullsized":"2024-02-02T15:25:26.272976+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:45:16.957512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.275647+0000","last_change":"2024-02-02T15:24:47.792601+0000","last_active":"2024-02-02T15:25:26.275647+0000","last_peered":"2024-02-02T15:25:26.275647+0000","last_clean":"2024-02-02T15:25:26.275647+0000","last_became_active":"2024-02-02T15:24:47.792240+0000","last_became_peered":"2024-02-02T15:24:47.792240+0000","last_unstale":"2024-02-02T15:25:26.275647+0000","last_undegraded":"2024-02-02T15:25:26.275647+0000","last_fullsized":"2024-02-02T15:25:26.275647+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:51:25.205859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.272255+0000","last_change":"2024-02-02T15:24:51.806267+0000","last_active":"2024-02-02T15:25:26.272255+0000","last_peered":"2024-02-02T15:25:26.272255+0000","last_clean":"2024-02-02T15:25:26.272255+0000","last_became_active":"2024-02-02T15:24:51.805877+0000","last_became_peered":"2024-02-02T15:24:51.805877+0000","last_unstale":"2024-02-02T15:25:26.272255+0000","last_undegraded":"2024-02-02T15:25:26.272255+0000","last_fullsized":"2024-02-02T15:25:26.272255+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:00:47.747377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'5","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.272379+0000","last_change":"2024-02-02T15:24:47.271057+0000","last_active":"2024-02-02T15:25:26.272379+0000","last_peered":"2024-02-02T15:25:26.272379+0000","last_clean":"2024-02-02T15:25:26.272379+0000","last_became_active":"2024-02-02T15:24:47.270737+0000","last_became_peered":"2024-02-02T15:24:47.270737+0000","last_unstale":"2024-02-02T15:25:26.272379+0000","last_undegraded":"2024-02-02T15:25:26.272379+0000","last_fullsized":"2024-02-02T15:25:26.272379+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:17:00.212426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.276183+0000","last_change":"2024-02-02T15:24:47.794108+0000","last_active":"2024-02-02T15:25:26.276183+0000","last_peered":"2024-02-02T15:25:26.276183+0000","last_clean":"2024-02-02T15:25:26.276183+0000","last_became_active":"2024-02-02T15:24:47.793890+0000","last_became_peered":"2024-02-02T15:24:47.793890+0000","last_unstale":"2024-02-02T15:25:26.276183+0000","last_undegraded":"2024-02-02T15:25:26.276183+0000","last_fullsized":"2024-02-02T15:25:26.276183+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T03:05:51.704385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.276351+0000","last_change":"2024-02-02T15:24:47.796234+0000","last_active":"2024-02-02T15:25:26.276351+0000","last_peered":"2024-02-02T15:25:26.276351+0000","last_clean":"2024-02-02T15:25:26.276351+0000","last_became_active":"2024-02-02T15:24:47.795474+0000","last_became_peered":"2024-02-02T15:24:47.795474+0000","last_unstale":"2024-02-02T15:25:26.276351+0000","last_undegraded":"2024-02-02T15:25:26.276351+0000","last_fullsized":"2024-02-02T15:25:26.276351+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:25:28.059500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.279417+0000","last_change":"2024-02-02T15:24:47.786930+0000","last_active":"2024-02-02T15:25:26.279417+0000","last_peered":"2024-02-02T15:25:26.279417+0000","last_clean":"2024-02-02T15:25:26.279417+0000","last_became_active":"2024-02-02T15:24:47.786615+0000","last_became_peered":"2024-02-02T15:24:47.786615+0000","last_unstale":"2024-02-02T15:25:26.279417+0000","last_undegraded":"2024-02-02T15:25:26.279417+0000","last_fullsized":"2024-02-02T15:25:26.279417+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:17:23.231511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.279172+0000","last_change":"2024-02-02T15:24:47.797877+0000","last_active":"2024-02-02T15:25:26.279172+0000","last_peered":"2024-02-02T15:25:26.279172+0000","last_clean":"2024-02-02T15:25:26.279172+0000","last_became_active":"2024-02-02T15:24:47.797559+0000","last_became_peered":"2024-02-02T15:24:47.797559+0000","last_unstale":"2024-02-02T15:25:26.279172+0000","last_undegraded":"2024-02-02T15:25:26.279172+0000","last_fullsized":"2024-02-02T15:25:26.279172+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:13:55.529796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.279344+0000","last_change":"2024-02-02T15:24:47.790270+0000","last_active":"2024-02-02T15:25:26.279344+0000","last_peered":"2024-02-02T15:25:26.279344+0000","last_clean":"2024-02-02T15:25:26.279344+0000","last_became_active":"2024-02-02T15:24:47.789928+0000","last_became_peered":"2024-02-02T15:24:47.789928+0000","last_unstale":"2024-02-02T15:25:26.279344+0000","last_undegraded":"2024-02-02T15:25:26.279344+0000","last_fullsized":"2024-02-02T15:25:26.279344+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T02:34:44.885803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T15:25:27.298415+0000","last_change":"2024-02-02T15:25:27.298415+0000","last_active":"2024-02-02T15:25:27.298415+0000","last_peered":"2024-02-02T15:25:27.298415+0000","last_clean":"2024-02-02T15:25:27.298415+0000","last_became_active":"2024-02-02T15:25:27.297990+0000","last_became_peered":"2024-02-02T15:25:27.297990+0000","last_unstale":"2024-02-02T15:25:27.298415+0000","last_undegraded":"2024-02-02T15:25:27.298415+0000","last_fullsized":"2024-02-02T15:25:27.298415+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:38:02.040613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.277083+0000","last_change":"2024-02-02T15:24:47.797861+0000","last_active":"2024-02-02T15:25:26.277083+0000","last_peered":"2024-02-02T15:25:26.277083+0000","last_clean":"2024-02-02T15:25:26.277083+0000","last_became_active":"2024-02-02T15:24:47.797543+0000","last_became_peered":"2024-02-02T15:24:47.797543+0000","last_unstale":"2024-02-02T15:25:26.277083+0000","last_undegraded":"2024-02-02T15:25:26.277083+0000","last_fullsized":"2024-02-02T15:25:26.277083+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:09:37.710909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.278967+0000","last_change":"2024-02-02T15:24:47.796221+0000","last_active":"2024-02-02T15:25:26.278967+0000","last_peered":"2024-02-02T15:25:26.278967+0000","last_clean":"2024-02-02T15:25:26.278967+0000","last_became_active":"2024-02-02T15:24:47.795845+0000","last_became_peered":"2024-02-02T15:24:47.795845+0000","last_unstale":"2024-02-02T15:25:26.278967+0000","last_undegraded":"2024-02-02T15:25:26.278967+0000","last_fullsized":"2024-02-02T15:25:26.278967+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:54:55.637660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.277498+0000","last_change":"2024-02-02T15:24:47.783086+0000","last_active":"2024-02-02T15:25:26.277498+0000","last_peered":"2024-02-02T15:25:26.277498+0000","last_clean":"2024-02-02T15:25:26.277498+0000","last_became_active":"2024-02-02T15:24:47.781826+0000","last_became_peered":"2024-02-02T15:24:47.781826+0000","last_unstale":"2024-02-02T15:25:26.277498+0000","last_undegraded":"2024-02-02T15:25:26.277498+0000","last_fullsized":"2024-02-02T15:25:26.277498+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:32:05.623591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-02-02T15:25:27.285214+0000","last_change":"2024-02-02T15:25:27.284461+0000","last_active":"2024-02-02T15:25:27.285214+0000","last_peered":"2024-02-02T15:25:27.285214+0000","last_clean":"2024-02-02T15:25:27.285214+0000","last_became_active":"2024-02-02T15:25:27.284080+0000","last_became_peered":"2024-02-02T15:25:27.284080+0000","last_unstale":"2024-02-02T15:25:27.285214+0000","last_undegraded":"2024-02-02T15:25:27.285214+0000","last_fullsized":"2024-02-02T15:25:27.285214+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:43:40.124825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.279045+0000","last_change":"2024-02-02T15:24:47.789905+0000","last_active":"2024-02-02T15:25:26.279045+0000","last_peered":"2024-02-02T15:25:26.279045+0000","last_clean":"2024-02-02T15:25:26.279045+0000","last_became_active":"2024-02-02T15:24:47.788903+0000","last_became_peered":"2024-02-02T15:24:47.788903+0000","last_unstale":"2024-02-02T15:25:26.279045+0000","last_undegraded":"2024-02-02T15:25:26.279045+0000","last_fullsized":"2024-02-02T15:25:26.279045+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T02:49:46.118232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.273118+0000","last_change":"2024-02-02T15:24:47.795928+0000","last_active":"2024-02-02T15:25:26.273118+0000","last_peered":"2024-02-02T15:25:26.273118+0000","last_clean":"2024-02-02T15:25:26.273118+0000","last_became_active":"2024-02-02T15:24:47.795511+0000","last_became_peered":"2024-02-02T15:24:47.795511+0000","last_unstale":"2024-02-02T15:25:26.273118+0000","last_undegraded":"2024-02-02T15:25:26.273118+0000","last_fullsized":"2024-02-02T15:25:26.273118+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:55:31.444331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.271980+0000","last_change":"2024-02-02T15:24:47.794368+0000","last_active":"2024-02-02T15:25:26.271980+0000","last_peered":"2024-02-02T15:25:26.271980+0000","last_clean":"2024-02-02T15:25:26.271980+0000","last_became_active":"2024-02-02T15:24:47.793808+0000","last_became_peered":"2024-02-02T15:24:47.793808+0000","last_unstale":"2024-02-02T15:25:26.271980+0000","last_undegraded":"2024-02-02T15:25:26.271980+0000","last_fullsized":"2024-02-02T15:25:26.271980+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T02:39:00.099808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-02-02T15:25:26.276848+0000","last_change":"2024-02-02T15:24:47.796358+0000","last_active":"2024-02-02T15:25:26.276848+0000","last_peered":"2024-02-02T15:25:26.276848+0000","last_clean":"2024-02-02T15:25:26.276848+0000","last_became_active":"2024-02-02T15:24:47.795800+0000","last_became_peered":"2024-02-02T15:24:47.795800+0000","last_unstale":"2024-02-02T15:25:26.276848+0000","last_undegraded":"2024-02-02T15:25:26.276848+0000","last_fullsized":"2024-02-02T15:25:26.276848+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:49:40.019787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":250199,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67018,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":436,"num_read_kb":292,"num_write":238,"num_write_kb":137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":201054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":208,"ondisk_log_size":208,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547226,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":478056,"data_compressed":17689,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 15:25:12 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.63900000000000001}]},{"osd":1,"last update":"Fri Feb 2 15:25:12 2024","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.69799999999999995}]},{"osd":2,"last update":"Fri Feb 2 15:25:12 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.41699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":4,"last update":"Fri Feb 2 15:25:12 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.60499999999999998}]},{"osd":5,"last update":"Fri Feb 2 15:25:12 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.50800000000000001}]},{"osd":6,"last update":"Fri Feb 2 15:25:12 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.325,"5min":1.325,"15min":1.325},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.66900000000000004}]},{"osd":7,"last update":"Fri Feb 2 15:25:12 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.78100000000000003}]},{"osd":8,"last update":"Fri Feb 2 15:25:50 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.748},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.72499999999999998}]},{"osd":9,"last update":"Fri Feb 2 15:25:12 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.74199999999999999}]},{"osd":10,"last update":"Fri Feb 2 15:25:12 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.79400000000000004}]}]},{"osd":10,"up_from":67,"seq":287762808860,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1097728,"data_stored":1125592,"data_compressed":53900,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 15:25:48 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.434},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.48099999999999998}]},{"osd":1,"last update":"Fri Feb 2 15:25:53 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.439,"15min":0.441},"min":{"1min":0.32200000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.63300000000000001,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.377},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.45900000000000002,"15min":0.46100000000000002},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.39900000000000002}]},{"osd":2,"last update":"Fri Feb 2 15:25:53 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.58799999999999997,"15min":0.59199999999999997},"min":{"1min":0.40000000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.72499999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.59399999999999997,"15min":0.59499999999999997},"min":{"1min":0.42599999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.74399999999999999,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.497}]},{"osd":3,"last update":"Fri Feb 2 15:25:53 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.58999999999999997,"15min":0.59799999999999998},"min":{"1min":0.39900000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.73299999999999998,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.78700000000000003,"15min":0.82399999999999995},"min":{"1min":0.40600000000000003,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.70399999999999996,"5min":4.4109999999999996,"15min":4.4109999999999996},"last":0.56799999999999995}]},{"osd":4,"last update":"Fri Feb 2 15:25:53 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.36899999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.78400000000000003,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.52000000000000002,"15min":0.52500000000000002},"min":{"1min":0.375,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.65200000000000002,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.44600000000000001}]},{"osd":5,"last update":"Fri Feb 2 15:25:53 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.65600000000000003,"15min":0.65800000000000003},"min":{"1min":0.45500000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.77200000000000002,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.66300000000000003,"15min":0.66000000000000003},"min":{"1min":0.47099999999999997,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.86499999999999999,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.66000000000000003}]},{"osd":6,"last update":"Fri Feb 2 15:25:53 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.80100000000000005,"15min":0.82699999999999996},"min":{"1min":0.52300000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.82799999999999996,"5min":4.4459999999999997,"15min":4.4459999999999997},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.71999999999999997,"15min":0.72899999999999998},"min":{"1min":0.44500000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.83899999999999997,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.60299999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40300000000000002}]},{"osd":8,"last update":"Fri Feb 2 15:25:53 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.71999999999999997,"15min":0.72399999999999998},"min":{"1min":0.496,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.92100000000000004,"5min":1.413,"15min":1.413},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.71899999999999997,"15min":0.72199999999999998},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.89800000000000002,"5min":1.036,"15min":1.036},"last":0.71099999999999997}]},{"osd":9,"last update":"Fri Feb 2 15:25:53 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.77600000000000002,"15min":0.78400000000000003},"min":{"1min":0.58899999999999997,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.88300000000000001,"5min":1.506,"15min":1.506},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.89800000000000002,"15min":0.93200000000000005},"min":{"1min":0.55400000000000005,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.86799999999999999,"5min":4.4269999999999996,"15min":4.4269999999999996},"last":0.55300000000000005}]},{"osd":11,"last update":"Fri Feb 2 15:25:11 2024","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.5680000000000001,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.753}]}]},{"osd":9,"up_from":62,"seq":266287972384,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":482062,"data_compressed":17689,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 15:25:37 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.56399999999999995,"15min":0.56899999999999995},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.0960000000000001,"5min":1.196,"15min":1.196},"last":0.48299999999999998}]},{"osd":1,"last update":"Fri Feb 2 15:25:37 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.73099999999999998,"15min":0.72299999999999998},"min":{"1min":0.44900000000000001,"5min":0.38,"15min":0.38},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.70199999999999996,"15min":0.70599999999999996},"min":{"1min":0.53900000000000003,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.93500000000000005,"5min":1.198,"15min":1.198},"last":0.80100000000000005}]},{"osd":2,"last update":"Fri Feb 2 15:25:37 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.72699999999999998,"15min":0.73499999999999999},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.0009999999999999,"5min":1.135,"15min":1.135},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.48699999999999999,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.70999999999999996}]},{"osd":3,"last update":"Fri Feb 2 15:25:37 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.72599999999999998,"15min":0.72299999999999998},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.625,"5min":0.71199999999999997,"15min":0.72599999999999998},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.1359999999999999,"5min":1.296,"15min":1.296},"last":0.59799999999999998}]},{"osd":4,"last update":"Fri Feb 2 15:25:37 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.70899999999999996,"15min":0.71899999999999997},"min":{"1min":0.39900000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.80500000000000005,"5min":1.381,"15min":1.381},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.83099999999999996,"15min":0.83699999999999997},"min":{"1min":0.53400000000000003,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.92800000000000005}]},{"osd":5,"last update":"Fri Feb 2 15:25:37 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.90000000000000002,"15min":0.93200000000000005},"min":{"1min":0.57399999999999995,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.93100000000000005,"5min":3.895,"15min":3.895},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.85799999999999998,"15min":0.86199999999999999},"min":{"1min":0.47699999999999998,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.5089999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.81399999999999995}]},{"osd":6,"last update":"Fri Feb 2 15:24:35 2024","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.6739999999999999,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":0.64500000000000002}]},{"osd":7,"last update":"Fri Feb 2 15:25:37 2024","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.91300000000000003,"15min":0.91900000000000004},"min":{"1min":0.64000000000000001,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.347,"5min":1.347,"15min":1.347},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.88,"15min":0.89800000000000002},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.0549999999999999,"5min":1.413,"15min":1.413},"last":0.86399999999999999}]},{"osd":8,"last update":"Fri Feb 2 15:25:37 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.90400000000000003,"15min":0.92500000000000004},"min":{"1min":0.66000000000000003,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":0.90100000000000002,"5min":1.7789999999999999,"15min":1.7789999999999999},"last":0.754},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.96399999999999997,"15min":0.97599999999999998},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":1.27,"5min":1.538,"15min":1.538},"last":0.77200000000000002}]},{"osd":10,"last update":"Fri Feb 2 15:25:56 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.93200000000000005,"15min":0.92700000000000005},"min":{"1min":0.71399999999999997,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.389,"5min":1.389,"15min":1.389},"last":1.0429999999999999},{"interface":"front","average":{"1min":0.874,"5min":0.86199999999999999,"15min":0.85999999999999999},"min":{"1min":0.70499999999999996,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":0.873}]},{"osd":11,"last update":"Fri Feb 2 15:25:51 2024","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.474,"5min":1.474,"15min":1.474},"last":1.006}]}]},{"osd":8,"up_from":55,"seq":236223201316,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":966656,"data_stored":478013,"data_compressed":17689,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 15:25:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.58199999999999996,"15min":0.58299999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.73699999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.59399999999999997,"15min":0.60099999999999998},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.77000000000000002,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.47599999999999998}]},{"osd":1,"last update":"Fri Feb 2 15:25:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.56899999999999995,"15min":0.56499999999999995},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.57399999999999995,"15min":0.56399999999999995},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.63200000000000001}]},{"osd":2,"last update":"Fri Feb 2 15:25:16 2024","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.42099999999999999,"15min":0.41699999999999998},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.63400000000000001,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.42299999999999999,"15min":0.41699999999999998},"min":{"1min":0.307,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.61299999999999999,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.55100000000000005}]},{"osd":3,"last update":"Fri Feb 2 15:25:16 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.621,"15min":0.627},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.76400000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.61699999999999999,"15min":0.621},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.79500000000000004,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.56499999999999995}]},{"osd":4,"last update":"Fri Feb 2 15:25:16 2024","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.60299999999999998,"15min":0.59899999999999998},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.62,"5min":0.59499999999999997,"15min":0.59099999999999997},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.45400000000000001}]},{"osd":5,"last update":"Fri Feb 2 15:25:16 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.51000000000000001,"15min":0.503},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.50800000000000001,"15min":0.502},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.65000000000000002}]},{"osd":6,"last update":"Fri Feb 2 15:25:16 2024","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.67200000000000004,"15min":0.67500000000000004},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.83299999999999996,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.69199999999999995,"15min":0.69299999999999995},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.80900000000000005,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.70799999999999996}]},{"osd":7,"last update":"Fri Feb 2 15:25:16 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.64700000000000002,"15min":0.64100000000000001},"min":{"1min":0.49199999999999999,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.67400000000000004,"15min":0.67000000000000004},"min":{"1min":0.45800000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.88700000000000001,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.66900000000000004}]},{"osd":9,"last update":"Fri Feb 2 15:25:38 2024","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.70799999999999996,"15min":0.69299999999999995},"min":{"1min":0.56000000000000005,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.72999999999999998,"15min":0.71499999999999997},"min":{"1min":0.62,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.73299999999999998}]},{"osd":10,"last update":"Fri Feb 2 15:25:56 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.73099999999999998,"15min":0.71699999999999997},"min":{"1min":0.56200000000000006,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.94599999999999995,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.72899999999999998,"15min":0.71299999999999997},"min":{"1min":0.63600000000000001,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.81100000000000005}]},{"osd":11,"last update":"Fri Feb 2 15:25:48 2024","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.748},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.77500000000000002}]}]},{"osd":7,"up_from":48,"seq":206158430248,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":1074698,"data_compressed":53900,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 15:23:58 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.439},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.38500000000000001}]},{"osd":1,"last update":"Fri Feb 2 15:24:59 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.42799999999999999,"15min":0.42599999999999999},"min":{"1min":0.28100000000000003,"5min":0.247,"15min":0.247},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.40899999999999997,"15min":0.40699999999999997},"min":{"1min":0.29399999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.64500000000000002,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.47599999999999998}]},{"osd":2,"last update":"Fri Feb 2 15:24:59 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.79100000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.57899999999999996}]},{"osd":3,"last update":"Fri Feb 2 15:24:59 2024","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.60899999999999999,"15min":0.59999999999999998},"min":{"1min":0.41099999999999998,"5min":0.374,"15min":0.374},"max":{"1min":2.746,"5min":2.746,"15min":2.746},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.69999999999999996}]},{"osd":4,"last update":"Fri Feb 2 15:24:59 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.502,"15min":0.5},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.5,"15min":0.496},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.501}]},{"osd":5,"last update":"Fri Feb 2 15:24:59 2024","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.63,"15min":0.61899999999999999},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":2.3370000000000002,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.63,"5min":0.623,"15min":0.622},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.629}]},{"osd":6,"last update":"Fri Feb 2 15:24:59 2024","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.68999999999999995,"15min":0.69199999999999995},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.67700000000000005,"15min":0.66800000000000004},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.778,"5min":1.778,"15min":1.778},"last":0.71399999999999997}]},{"osd":8,"last update":"Fri Feb 2 15:25:16 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.70499999999999996,"15min":0.70099999999999996},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":2.069,"5min":2.069,"15min":2.069},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.72899999999999998,"15min":0.73299999999999998},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.995,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.66200000000000003}]},{"osd":9,"last update":"Fri Feb 2 15:25:40 2024","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.76100000000000001,"15min":0.747},"min":{"1min":0.497,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":4.9550000000000001,"5min":4.9550000000000001,"15min":4.9550000000000001},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.753,"15min":0.73899999999999999},"min":{"1min":0.54500000000000004,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":4.0759999999999996,"5min":4.0759999999999996,"15min":4.0759999999999996},"last":0.80300000000000005}]},{"osd":10,"last update":"Fri Feb 2 15:25:55 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.47099999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.82499999999999996,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.75},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.495,"5min":0.436,"15min":0.436},"max":{"1min":0.88800000000000001,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.73799999999999999}]},{"osd":11,"last update":"Fri Feb 2 15:25:51 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.76100000000000001}]}]},{"osd":6,"up_from":43,"seq":184683593771,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":950272,"data_stored":473422,"data_compressed":17689,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 15:25:44 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.497,"15min":0.48499999999999999},"min":{"1min":0.35599999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.67900000000000005,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51900000000000002,"15min":0.51200000000000001},"min":{"1min":0.371,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.67200000000000004,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.50600000000000001}]},{"osd":1,"last update":"Fri Feb 2 15:25:44 2024","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":0.72199999999999998,"15min":0.66400000000000003},"min":{"1min":0.47999999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":6.8650000000000002,"5min":6.8650000000000002,"15min":6.8650000000000002},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.66400000000000003,"15min":0.65200000000000002},"min":{"1min":0.50600000000000001,"5min":0.501,"15min":0.501},"max":{"1min":0.88900000000000001,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.59899999999999998}]},{"osd":2,"last update":"Fri Feb 2 15:25:44 2024","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":0.79700000000000004,"15min":0.72699999999999998},"min":{"1min":0.57199999999999995,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":6.9320000000000004,"5min":6.9320000000000004,"15min":6.9320000000000004},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.753,"15min":0.72499999999999998},"min":{"1min":0.52600000000000002,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":3.008,"5min":3.008,"15min":3.008},"last":0.69299999999999995}]},{"osd":3,"last update":"Fri Feb 2 15:25:44 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.628,"15min":0.63300000000000001},"min":{"1min":0.432,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.875,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.67400000000000004,"15min":0.66400000000000003},"min":{"1min":0.45100000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.63100000000000001}]},{"osd":4,"last update":"Fri Feb 2 15:25:44 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.72999999999999998,"15min":0.71999999999999997},"min":{"1min":0.53100000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.95599999999999996,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76200000000000001,"15min":0.75600000000000001},"min":{"1min":0.55700000000000005,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.0489999999999999,"5min":1.228,"15min":1.228},"last":0.65700000000000003}]},{"osd":5,"last update":"Fri Feb 2 15:25:44 2024","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.79100000000000004,"15min":0.77900000000000003},"min":{"1min":0.55100000000000005,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":1.268,"5min":0.90900000000000003,"15min":0.84499999999999997},"min":{"1min":0.61199999999999999,"5min":0.442,"15min":0.442},"max":{"1min":6.9820000000000002,"5min":6.9820000000000002,"15min":6.9820000000000002},"last":0.77900000000000003}]},{"osd":7,"last update":"Fri Feb 2 15:24:57 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.83199999999999996,"15min":0.83099999999999996},"min":{"1min":0.59199999999999997,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.073,"5min":1.256,"15min":1.256},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.876,"15min":0.878},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.103,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.82099999999999995}]},{"osd":8,"last update":"Fri Feb 2 15:25:14 2024","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.90000000000000002,"15min":0.90100000000000002},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.212,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.94399999999999995,"5min":0.91700000000000004,"15min":0.91300000000000003},"min":{"1min":0.57699999999999996,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.81000000000000005}]},{"osd":9,"last update":"Fri Feb 2 15:25:39 2024","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.93999999999999995,"15min":0.94599999999999995},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.2410000000000001,"5min":1.405,"15min":1.405},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.96799999999999997,"15min":0.97699999999999998},"min":{"1min":0.72199999999999998,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.1120000000000001,"5min":1.4870000000000001,"15min":1.4870000000000001},"last":0.90900000000000003}]},{"osd":10,"last update":"Fri Feb 2 15:25:56 2024","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1,"15min":0.96399999999999997},"min":{"1min":0.68300000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":6.9960000000000004,"5min":6.9960000000000004,"15min":6.9960000000000004},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.97799999999999998,"15min":0.98699999999999999},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.1879999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.877}]},{"osd":11,"last update":"Fri Feb 2 15:25:51 2024","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.90100000000000002},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":3.032,"5min":3.032,"15min":3.032},"last":0.96799999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574909,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":534848,"data_compressed":17689,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 15:25:12 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.51100000000000001,"15min":0.50800000000000001},"min":{"1min":0.39200000000000002,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.52500000000000002,"15min":0.52100000000000002},"min":{"1min":0.40100000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.56200000000000006}]},{"osd":2,"last update":"Fri Feb 2 15:25:34 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.55200000000000005,"15min":0.54600000000000004},"min":{"1min":0.39400000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.56200000000000006,"15min":0.54600000000000004},"min":{"1min":0.377,"5min":0.315,"15min":0.315},"max":{"1min":0.81599999999999995,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.44900000000000001}]},{"osd":3,"last update":"Fri Feb 2 15:25:50 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.56899999999999995,"15min":0.53700000000000003},"min":{"1min":0.434,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.57999999999999996,"15min":0.55400000000000005},"min":{"1min":0.45800000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.746}]},{"osd":4,"last update":"Fri Feb 2 15:25:04 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.39600000000000002,"15min":0.36899999999999999},"min":{"1min":0.32900000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.66800000000000004,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.40000000000000002,"15min":0.375},"min":{"1min":0.27200000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.59499999999999997}]},{"osd":5,"last update":"Fri Feb 2 15:25:21 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.56000000000000005,"15min":0.54100000000000004},"min":{"1min":0.39600000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.58199999999999996,"15min":0.55500000000000005},"min":{"1min":0.45300000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.72199999999999998}]},{"osd":6,"last update":"Fri Feb 2 15:25:46 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.621,"15min":0.59499999999999997},"min":{"1min":0.50800000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":6.1950000000000003},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.60599999999999998,"15min":0.58799999999999997},"min":{"1min":0.48099999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.70099999999999996}]},{"osd":7,"last update":"Fri Feb 2 15:25:00 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.81100000000000005,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.53700000000000003,"15min":0.53300000000000003},"min":{"1min":0.371,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.72699999999999998,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.76300000000000001}]},{"osd":8,"last update":"Fri Feb 2 15:25:14 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.63400000000000001,"15min":0.63},"min":{"1min":0.434,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.65700000000000003,"15min":0.64700000000000002},"min":{"1min":0.51100000000000001,"5min":0.442,"15min":0.442},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.78500000000000003}]},{"osd":9,"last update":"Fri Feb 2 15:25:38 2024","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.71199999999999997,"15min":0.70499999999999996},"min":{"1min":0.52300000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.749,"15min":0.74299999999999999},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.02,"5min":1.121,"15min":1.121},"last":0.85899999999999999}]},{"osd":10,"last update":"Fri Feb 2 15:25:56 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.69499999999999995,"15min":0.68500000000000005},"min":{"1min":0.57399999999999995,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.72699999999999998,"15min":0.71499999999999997},"min":{"1min":0.57999999999999996,"5min":0.5,"15min":0.5},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.84499999999999997}]},{"osd":11,"last update":"Fri Feb 2 15:25:50 2024","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.81100000000000005}]}]},{"osd":0,"up_from":8,"seq":34359738433,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":974848,"data_stored":474878,"data_compressed":17689,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Feb 2 15:25:10 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.55800000000000005,"15min":0.54400000000000004},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.90400000000000003,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.628,"5min":0.57399999999999995,"15min":0.54500000000000004},"min":{"1min":0.35799999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.55800000000000005}]},{"osd":2,"last update":"Fri Feb 2 15:25:30 2024","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.65300000000000002,"15min":0.58299999999999996},"min":{"1min":0.54500000000000004,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.70799999999999996,"15min":0.61599999999999999},"min":{"1min":0.50900000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.2789999999999999,"5min":4.6550000000000002,"15min":4.6550000000000002},"last":0.78200000000000003}]},{"osd":3,"last update":"Fri Feb 2 15:25:48 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.625},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":0.53900000000000003}]},{"osd":4,"last update":"Fri Feb 2 15:25:05 2024","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.65400000000000003,"15min":0.63700000000000001},"min":{"1min":0.433,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.091,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.64200000000000002,"15min":0.626},"min":{"1min":0.46700000000000003,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.1279999999999999,"5min":1.141,"15min":1.141},"last":0.70799999999999996}]},{"osd":5,"last update":"Fri Feb 2 15:25:21 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68200000000000005,"15min":0.68300000000000005},"min":{"1min":0.50700000000000001,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.024,"5min":1.042,"15min":1.042},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.70999999999999996,"15min":0.67600000000000005},"min":{"1min":0.42099999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.054,"5min":4.5890000000000004,"15min":4.5890000000000004},"last":0.69099999999999995}]},{"osd":6,"last update":"Fri Feb 2 15:25:43 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.72899999999999998,"15min":0.70499999999999996},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.494,"5min":1.494,"15min":1.494},"last":0.92000000000000004},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.74099999999999999,"15min":0.73099999999999998},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.82399999999999995}]},{"osd":7,"last update":"Fri Feb 2 15:25:55 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.72799999999999998}]},{"osd":8,"last update":"Fri Feb 2 15:25:21 2024","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.77700000000000002,"15min":0.76900000000000002},"min":{"1min":0.56899999999999995,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.84699999999999998}]},{"osd":9,"last update":"Fri Feb 2 15:25:40 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.96399999999999997,"15min":0.98699999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0980000000000001,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":1.0209999999999999},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.95399999999999996,"15min":0.96799999999999997},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.411,"5min":1.411,"15min":1.411},"last":0.93799999999999994}]},{"osd":10,"last update":"Fri Feb 2 15:26:01 2024","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.879,"15min":0.878},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.80200000000000005},{"interface":"front","average":{"1min":1,"5min":0.97199999999999998,"15min":0.96799999999999997},"min":{"1min":0.66100000000000003,"5min":0.63,"15min":0.63},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.96399999999999997}]},{"osd":11,"last update":"Fri Feb 2 15:25:10 2024","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":1.002}]}]},{"osd":2,"up_from":18,"seq":77309411386,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":534952,"data_compressed":17689,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 15:25:26 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54500000000000004,"15min":0.54300000000000004},"min":{"1min":0.40200000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.74299999999999999,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.629},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.52900000000000003,"15min":0.52100000000000002},"min":{"1min":0.39600000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.77200000000000002,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.41999999999999998}]},{"osd":1,"last update":"Fri Feb 2 15:25:26 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.52600000000000002,"15min":0.504},"min":{"1min":0.39200000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.56200000000000006,"15min":0.53300000000000003},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.76400000000000001}]},{"osd":3,"last update":"Fri Feb 2 15:25:46 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.60099999999999998,"15min":0.57199999999999995},"min":{"1min":0.46100000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.75600000000000001,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.623,"5min":0.58799999999999997,"15min":0.55600000000000005},"min":{"1min":0.42799999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.81000000000000005,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.53900000000000003}]},{"osd":4,"last update":"Fri Feb 2 15:25:50 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.69799999999999995}]},{"osd":5,"last update":"Fri Feb 2 15:25:26 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.432,"15min":0.40999999999999998},"min":{"1min":0.32400000000000001,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.81399999999999995,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.42399999999999999,"15min":0.40799999999999997},"min":{"1min":0.33900000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.63,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.44}]},{"osd":6,"last update":"Fri Feb 2 15:25:40 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.625,"15min":0.61899999999999999},"min":{"1min":0.46100000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.79500000000000004,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.628,"5min":0.625,"15min":0.627},"min":{"1min":0.437,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.62}]},{"osd":7,"last update":"Fri Feb 2 15:25:52 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.79000000000000004}]},{"osd":8,"last update":"Fri Feb 2 15:25:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.41899999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.84199999999999997,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.40699999999999997,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.70799999999999996,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.55600000000000005}]},{"osd":9,"last update":"Fri Feb 2 15:25:40 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.66900000000000004,"15min":0.66700000000000004},"min":{"1min":0.501,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.751,"15min":0.746},"min":{"1min":0.60299999999999998,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.92500000000000004,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.88300000000000001}]},{"osd":10,"last update":"Fri Feb 2 15:25:52 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.68300000000000005,"15min":0.67200000000000004},"min":{"1min":0.628,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.68100000000000005,"15min":0.67100000000000004},"min":{"1min":0.56299999999999994,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.86799999999999999,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.73099999999999998}]},{"osd":11,"last update":"Fri Feb 2 15:25:46 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.75700000000000001}]}]},{"osd":3,"up_from":24,"seq":103079215157,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":962560,"data_stored":473178,"data_compressed":17689,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 15:25:44 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.442,"15min":0.41899999999999998},"min":{"1min":0.34899999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.69699999999999995,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.46600000000000003,"15min":0.45300000000000001},"min":{"1min":0.33300000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.73099999999999998,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.435}]},{"osd":1,"last update":"Fri Feb 2 15:25:44 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.65000000000000002,"15min":0.59599999999999997},"min":{"1min":0.52900000000000003,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.0649999999999999,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.622,"15min":0.59899999999999998},"min":{"1min":0.49099999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.879,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.59399999999999997}]},{"osd":2,"last update":"Fri Feb 2 15:25:44 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.63300000000000001,"15min":0.623},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.70299999999999996,"15min":0.65700000000000003},"min":{"1min":0.56799999999999995,"5min":0.443,"15min":0.443},"max":{"1min":1.1319999999999999,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":0.65700000000000003}]},{"osd":4,"last update":"Fri Feb 2 15:24:59 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.625},"min":{"1min":0.40400000000000003,"5min":0.376,"15min":0.376},"max":{"1min":0.82099999999999995,"5min":1.022,"15min":1.022},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.70999999999999996,"15min":0.69199999999999995},"min":{"1min":0.61499999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0329999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.89600000000000002}]},{"osd":5,"last update":"Fri Feb 2 15:25:24 2024","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.74199999999999999,"15min":0.72399999999999998},"min":{"1min":0.52700000000000002,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.1419999999999999,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.66300000000000003,"15min":0.64700000000000002},"min":{"1min":0.48099999999999998,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.52700000000000002}]},{"osd":6,"last update":"Fri Feb 2 15:25:44 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.74199999999999999,"15min":0.71599999999999997},"min":{"1min":0.57099999999999995,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.1100000000000001,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.68999999999999995,"15min":0.68799999999999994},"min":{"1min":0.40500000000000003,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.90600000000000003,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.54400000000000004}]},{"osd":7,"last update":"Fri Feb 2 15:24:59 2024","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.75,"15min":0.74099999999999999},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.73599999999999999,"15min":0.72999999999999998},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.97299999999999998,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.623}]},{"osd":8,"last update":"Fri Feb 2 15:24:14 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.65800000000000003}]},{"osd":9,"last update":"Fri Feb 2 15:25:37 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.94599999999999995,"15min":0.95399999999999996},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.83999999999999997,"15min":0.84999999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0449999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.63100000000000001}]},{"osd":10,"last update":"Fri Feb 2 15:24:54 2024","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.1379999999999999,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.96699999999999997}]},{"osd":11,"last update":"Fri Feb 2 15:25:49 2024","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":1.0009999999999999},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.81200000000000006}]}]},{"osd":4,"up_from":31,"seq":133143986228,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":958464,"data_stored":477888,"data_compressed":17689,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 15:24:03 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.52300000000000002,"15min":0.52100000000000002},"min":{"1min":0.371,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.78600000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.53800000000000003,"15min":0.53600000000000003},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.47799999999999998}]},{"osd":1,"last update":"Fri Feb 2 15:25:05 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.35299999999999998,"15min":0.33500000000000002},"min":{"1min":0.26600000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.432,"5min":0.36499999999999999,"15min":0.34699999999999998},"min":{"1min":0.28299999999999997,"5min":0.255,"15min":0.255},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.71399999999999997}]},{"osd":2,"last update":"Fri Feb 2 15:25:05 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.52400000000000002,"15min":0.50700000000000001},"min":{"1min":0.35699999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.872},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.52800000000000002,"15min":0.51100000000000001},"min":{"1min":0.40699999999999997,"5min":0.38,"15min":0.38},"max":{"1min":0.753,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.63700000000000001}]},{"osd":3,"last update":"Fri Feb 2 15:25:05 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.56599999999999995,"15min":0.56399999999999995},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.55800000000000005,"15min":0.56000000000000005},"min":{"1min":0.38400000000000001,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.70899999999999996,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.57599999999999996}]},{"osd":5,"last update":"Fri Feb 2 15:25:21 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.60399999999999998,"15min":0.58699999999999997},"min":{"1min":0.40000000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.61199999999999999,"15min":0.60399999999999998},"min":{"1min":0.439,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.80600000000000005,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.68700000000000006}]},{"osd":6,"last update":"Fri Feb 2 15:25:41 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.63600000000000001,"15min":0.63200000000000001},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.752,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.64200000000000002},"min":{"1min":0.49199999999999999,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.84499999999999997,"5min":1.335,"15min":1.335},"last":0.82199999999999995}]},{"osd":7,"last update":"Fri Feb 2 15:24:59 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.49099999999999999,"15min":0.48299999999999998},"min":{"1min":0.34000000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.53600000000000003,"15min":0.53200000000000003},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.76000000000000001}]},{"osd":8,"last update":"Fri Feb 2 15:25:16 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.65200000000000002,"15min":0.65000000000000002},"min":{"1min":0.42499999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.84299999999999997,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.65600000000000003,"15min":0.65100000000000002},"min":{"1min":0.46999999999999997,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":1.0089999999999999}]},{"osd":9,"last update":"Fri Feb 2 15:25:36 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.71899999999999997,"15min":0.72299999999999998},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.87,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.753,"5min":0.72899999999999998,"15min":0.72499999999999998},"min":{"1min":0.53200000000000003,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.889,"5min":1.889,"15min":1.889},"last":0.85799999999999998}]},{"osd":10,"last update":"Fri Feb 2 15:25:57 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.67700000000000005,"15min":0.67300000000000004},"min":{"1min":0.48199999999999998,"5min":0.379,"15min":0.379},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.94099999999999995},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.66800000000000004,"15min":0.66700000000000004},"min":{"1min":0.55200000000000005,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.79100000000000004,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.72799999999999998}]},{"osd":11,"last update":"Fri Feb 2 15:25:48 2024","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.97499999999999998},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.84699999999999998}]}]},{"osd":5,"up_from":36,"seq":154618822703,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":1069984,"data_compressed":53900,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 15:25:20 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.53000000000000003,"15min":0.51500000000000001},"min":{"1min":0.38600000000000001,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.53900000000000003,"15min":0.52600000000000002},"min":{"1min":0.39600000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.83599999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.51500000000000001}]},{"osd":1,"last update":"Fri Feb 2 15:25:20 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.56000000000000005,"15min":0.55000000000000004},"min":{"1min":0.46500000000000002,"5min":0.252,"15min":0.252},"max":{"1min":0.69999999999999996,"5min":1.018,"15min":1.018},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.621,"5min":0.56499999999999995,"15min":0.54700000000000004},"min":{"1min":0.46400000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.77500000000000002,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.61099999999999999}]},{"osd":2,"last update":"Fri Feb 2 15:25:20 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.39500000000000002,"15min":0.38300000000000001},"min":{"1min":0.312,"5min":0.215,"15min":0.215},"max":{"1min":0.59499999999999997,"5min":0.63,"15min":0.63},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.40999999999999998,"15min":0.40000000000000002},"min":{"1min":0.318,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.72699999999999998,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.64900000000000002}]},{"osd":3,"last update":"Fri Feb 2 15:25:20 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.54700000000000004,"15min":0.52300000000000002},"min":{"1min":0.378,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.80600000000000005,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.58299999999999996,"15min":0.56599999999999995},"min":{"1min":0.52200000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.63200000000000001}]},{"osd":4,"last update":"Fri Feb 2 15:25:20 2024","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.57499999999999996,"15min":0.56200000000000006},"min":{"1min":0.45700000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.79600000000000004,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.58699999999999997,"15min":0.57599999999999996},"min":{"1min":0.47099999999999997,"5min":0.316,"15min":0.316},"max":{"1min":0.81899999999999995,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.74199999999999999}]},{"osd":6,"last update":"Fri Feb 2 15:25:42 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.63300000000000001,"15min":0.63200000000000001},"min":{"1min":0.51700000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.84599999999999997,"5min":0.878,"15min":0.878},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.63400000000000001,"15min":0.63300000000000001},"min":{"1min":0.57599999999999996,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.81999999999999995,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.751}]},{"osd":7,"last update":"Fri Feb 2 15:25:00 2024","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.622,"15min":0.62},"min":{"1min":0.46400000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.629,"5min":0.60799999999999998,"15min":0.60399999999999998},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.79200000000000004,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.71699999999999997}]},{"osd":8,"last update":"Fri Feb 2 15:25:14 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.55100000000000005,"15min":0.55400000000000005},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.83999999999999997,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.58699999999999997,"15min":0.59199999999999997},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.80100000000000005,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.83099999999999996}]},{"osd":9,"last update":"Fri Feb 2 15:25:38 2024","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.69399999999999995,"15min":0.68400000000000005},"min":{"1min":0.63,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.88,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.71299999999999997,"15min":0.70499999999999996},"min":{"1min":0.63500000000000001,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.91100000000000003,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.77900000000000003}]},{"osd":10,"last update":"Fri Feb 2 15:25:54 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.70899999999999996,"15min":0.69999999999999996},"min":{"1min":0.57699999999999996,"5min":0.497,"15min":0.497},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.874},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.72799999999999998,"15min":0.72299999999999998},"min":{"1min":0.54000000000000004,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1,"5min":1,"15min":1},"last":0.81799999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34572,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34572,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36211,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36211,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36211,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36211,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36211,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":62100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":62204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":62476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-02T15:26:02.817 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-02-02T15:26:02.818 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-02-02T15:26:02.818 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-02-02T15:26:02.818 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph health --format=json 2024-02-02T15:26:03.187 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.a/config 2024-02-02T15:26:03.481 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:03 smithi001 ceph-mon[35470]: pgmap v274: 129 pgs: 129 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-02-02T15:26:03.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:03 smithi071 ceph-mon[44707]: pgmap v274: 129 pgs: 129 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-02-02T15:26:03.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:03 smithi142 ceph-mon[45728]: pgmap v274: 129 pgs: 129 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-02-02T15:26:04.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:04.347+0000 7f6f59200700 1 -- 172.21.15.1:0/908201929 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f541060a0 msgr2=0x7f6f5410dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:04.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:04.347+0000 7f6f59200700 1 --2- 172.21.15.1:0/908201929 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f541060a0 0x7f6f5410dc70 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6f4c00b0a0 tx=0x7f6f4c00b3b0 comp rx=0 tx=0).stop 2024-02-02T15:26:04.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:04.347+0000 7f6f59200700 1 -- 172.21.15.1:0/908201929 shutdown_connections 2024-02-02T15:26:04.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:04.347+0000 7f6f59200700 1 --2- 172.21.15.1:0/908201929 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f541060a0 0x7f6f5410dc70 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:04.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:04.347+0000 7f6f59200700 1 --2- 172.21.15.1:0/908201929 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6f54100fc0 0x7f6f541013a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:04.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:04.347+0000 7f6f59200700 1 --2- 172.21.15.1:0/908201929 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6f54101970 0x7f6f541059c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:04.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:04.347+0000 7f6f59200700 1 -- 172.21.15.1:0/908201929 >> 172.21.15.1:0/908201929 conn(0x7f6f540fc830 msgr2=0x7f6f540fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:04.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:04.347+0000 7f6f59200700 1 -- 172.21.15.1:0/908201929 shutdown_connections 2024-02-02T15:26:04.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:04.347+0000 7f6f59200700 1 -- 172.21.15.1:0/908201929 wait complete. 2024-02-02T15:26:04.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:04.348+0000 7f6f59200700 1 Processor -- start 2024-02-02T15:26:04.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:04.348+0000 7f6f59200700 1 -- start start 2024-02-02T15:26:04.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:04.348+0000 7f6f59200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f54100fc0 0x7f6f54073040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:04.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:04.348+0000 7f6f59200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6f54101970 0x7f6f54073580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:04.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:04.348+0000 7f6f59200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6f541060a0 0x7f6f54073b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:04.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:04.348+0000 7f6f59200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f541a5400 con 0x7f6f54101970 2024-02-02T15:26:04.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:04.348+0000 7f6f59200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f541a5570 con 0x7f6f541060a0 2024-02-02T15:26:04.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:04.348+0000 7f6f59200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f541a5730 con 0x7f6f54100fc0 2024-02-02T15:26:04.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:04.349+0000 7f6f51e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f54100fc0 0x7f6f54073040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:04.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:04.349+0000 7f6f51400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6f54101970 0x7f6f54073580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:04.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:04.349+0000 7f6f51e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f54100fc0 0x7f6f54073040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:56962/0 (socket says 172.21.15.1:56962) 2024-02-02T15:26:04.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:04.349+0000 7f6f51e00700 1 -- 172.21.15.1:0/4060802326 learned_addr learned my addr 172.21.15.1:0/4060802326 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:04.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:04.349+0000 7f6f52800700 1 --2- 172.21.15.1:0/4060802326 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6f541060a0 0x7f6f54073b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:04.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:04.349+0000 7f6f51400700 1 -- 172.21.15.1:0/4060802326 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f54100fc0 msgr2=0x7f6f54073040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:04.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:04.349+0000 7f6f51400700 1 --2- 172.21.15.1:0/4060802326 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f54100fc0 0x7f6f54073040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:04.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:04.349+0000 7f6f51400700 1 -- 172.21.15.1:0/4060802326 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6f541060a0 msgr2=0x7f6f54073b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:04.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:04.349+0000 7f6f51400700 1 --2- 172.21.15.1:0/4060802326 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6f541060a0 0x7f6f54073b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:04.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:04.349+0000 7f6f51400700 1 -- 172.21.15.1:0/4060802326 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f3c009520 con 0x7f6f54101970 2024-02-02T15:26:04.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:04.349+0000 7f6f51e00700 1 --2- 172.21.15.1:0/4060802326 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f54100fc0 0x7f6f54073040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:26:04.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:04.350+0000 7f6f51400700 1 --2- 172.21.15.1:0/4060802326 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6f54101970 0x7f6f54073580 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c00e9c0 tx=0x7f6f3c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:04.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:04.350+0000 7f6f4aa00700 1 -- 172.21.15.1:0/4060802326 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c00cb70 con 0x7f6f54101970 2024-02-02T15:26:04.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:04.350+0000 7f6f59200700 1 -- 172.21.15.1:0/4060802326 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f4c009d00 con 0x7f6f54101970 2024-02-02T15:26:04.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:04.350+0000 7f6f4aa00700 1 -- 172.21.15.1:0/4060802326 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6f3c00ee60 con 0x7f6f54101970 2024-02-02T15:26:04.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:04.350+0000 7f6f4aa00700 1 -- 172.21.15.1:0/4060802326 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c003c60 con 0x7f6f54101970 2024-02-02T15:26:04.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:04.350+0000 7f6f59200700 1 -- 172.21.15.1:0/4060802326 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f541a5dd0 con 0x7f6f54101970 2024-02-02T15:26:04.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:04.351+0000 7f6f59200700 1 -- 172.21.15.1:0/4060802326 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f5404ea40 con 0x7f6f54101970 2024-02-02T15:26:04.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:04.353+0000 7f6f4aa00700 1 -- 172.21.15.1:0/4060802326 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) v1 ==== 135120+0+0 (secure 0 0 0) 0x7f6f3c003de0 con 0x7f6f54101970 2024-02-02T15:26:04.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:04.354+0000 7f6f4aa00700 1 --2- 172.21.15.1:0/4060802326 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f6f340a0630 0x7f6f340a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:04.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:04.354+0000 7f6f4aa00700 1 -- 172.21.15.1:0/4060802326 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f6f3c014070 con 0x7f6f54101970 2024-02-02T15:26:04.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:04.354+0000 7f6f51e00700 1 --2- 172.21.15.1:0/4060802326 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f6f340a0630 0x7f6f340a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:04.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:04.355+0000 7f6f51e00700 1 --2- 172.21.15.1:0/4060802326 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f6f340a0630 0x7f6f340a2af0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6f44011550 tx=0x7f6f44009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:04.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:04.357+0000 7f6f4aa00700 1 -- 172.21.15.1:0/4060802326 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f6f3c095840 con 0x7f6f54101970 2024-02-02T15:26:04.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:04 smithi001 ceph-mon[35470]: from='client.25019 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-02T15:26:04.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:04.608+0000 7f6f59200700 1 -- 172.21.15.1:0/4060802326 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f6f54074570 con 0x7f6f54101970 2024-02-02T15:26:04.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:04.608+0000 7f6f4aa00700 1 -- 172.21.15.1:0/4060802326 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f6f3c098e60 con 0x7f6f54101970 2024-02-02T15:26:04.609 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:26:04.609 INFO:teuthology.orchestra.run.smithi001.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-02-02T15:26:04.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:04.611+0000 7f6f59200700 1 -- 172.21.15.1:0/4060802326 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f6f340a0630 msgr2=0x7f6f340a2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:04.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:04.611+0000 7f6f59200700 1 --2- 172.21.15.1:0/4060802326 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f6f340a0630 0x7f6f340a2af0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6f44011550 tx=0x7f6f44009260 comp rx=0 tx=0).stop 2024-02-02T15:26:04.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:04.611+0000 7f6f59200700 1 -- 172.21.15.1:0/4060802326 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6f54101970 msgr2=0x7f6f54073580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:04.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:04.611+0000 7f6f59200700 1 --2- 172.21.15.1:0/4060802326 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6f54101970 0x7f6f54073580 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c00e9c0 tx=0x7f6f3c00ecd0 comp rx=0 tx=0).stop 2024-02-02T15:26:04.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:04.611+0000 7f6f59200700 1 -- 172.21.15.1:0/4060802326 shutdown_connections 2024-02-02T15:26:04.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:04.611+0000 7f6f59200700 1 --2- 172.21.15.1:0/4060802326 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6f541060a0 0x7f6f54073b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:04.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:04.611+0000 7f6f59200700 1 --2- 172.21.15.1:0/4060802326 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f6f340a0630 0x7f6f340a2af0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:04.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:04.611+0000 7f6f59200700 1 --2- 172.21.15.1:0/4060802326 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6f54100fc0 0x7f6f54073040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:04.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:04.611+0000 7f6f59200700 1 --2- 172.21.15.1:0/4060802326 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6f54101970 0x7f6f54073580 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:04.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:04.611+0000 7f6f59200700 1 -- 172.21.15.1:0/4060802326 >> 172.21.15.1:0/4060802326 conn(0x7f6f540fc830 msgr2=0x7f6f540fd250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:04.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:04.611+0000 7f6f59200700 1 -- 172.21.15.1:0/4060802326 shutdown_connections 2024-02-02T15:26:04.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:04.611+0000 7f6f59200700 1 -- 172.21.15.1:0/4060802326 wait complete. 2024-02-02T15:26:04.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:04 smithi071 ceph-mon[44707]: from='client.25019 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-02T15:26:04.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:04 smithi142 ceph-mon[45728]: from='client.25019 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-02T15:26:04.987 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-02-02T15:26:04.988 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T15:26:05.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.189+0000 7f5c2f200700 1 -- 172.21.15.1:0/1734767212 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c281028e0 msgr2=0x7f5c28109190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:05.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.189+0000 7f5c2f200700 1 --2- 172.21.15.1:0/1734767212 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c281028e0 0x7f5c28109190 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5c2000b0d0 tx=0x7f5c2000b3e0 comp rx=0 tx=0).stop 2024-02-02T15:26:05.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.189+0000 7f5c2f200700 1 -- 172.21.15.1:0/1734767212 shutdown_connections 2024-02-02T15:26:05.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.189+0000 7f5c2f200700 1 --2- 172.21.15.1:0/1734767212 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c281028e0 0x7f5c28109190 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:05.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.189+0000 7f5c2f200700 1 --2- 172.21.15.1:0/1734767212 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c28100d50 0x7f5c28101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:05.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.189+0000 7f5c2f200700 1 --2- 172.21.15.1:0/1734767212 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5c28101f40 0x7f5c281023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:05.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.189+0000 7f5c2f200700 1 -- 172.21.15.1:0/1734767212 >> 172.21.15.1:0/1734767212 conn(0x7f5c280fc2c0 msgr2=0x7f5c280fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:05.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.189+0000 7f5c2f200700 1 -- 172.21.15.1:0/1734767212 shutdown_connections 2024-02-02T15:26:05.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.189+0000 7f5c2f200700 1 -- 172.21.15.1:0/1734767212 wait complete. 2024-02-02T15:26:05.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.190+0000 7f5c2f200700 1 Processor -- start 2024-02-02T15:26:05.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.190+0000 7f5c2f200700 1 -- start start 2024-02-02T15:26:05.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.190+0000 7f5c2f200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c28100d50 0x7f5c28074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:05.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.190+0000 7f5c2f200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5c28101f40 0x7f5c28073130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:05.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.190+0000 7f5c2f200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c281028e0 0x7f5c28073670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:05.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.190+0000 7f5c2f200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c28073e10 con 0x7f5c28100d50 2024-02-02T15:26:05.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.190+0000 7f5c2f200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c28073f80 con 0x7f5c28101f40 2024-02-02T15:26:05.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.190+0000 7f5c2f200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c281a0bd0 con 0x7f5c281028e0 2024-02-02T15:26:05.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.191+0000 7f5c27e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c28100d50 0x7f5c28074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:05.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.191+0000 7f5c27400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5c28101f40 0x7f5c28073130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:05.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.191+0000 7f5c2d200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c281028e0 0x7f5c28073670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:05.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.191+0000 7f5c2d200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c281028e0 0x7f5c28073670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:56986/0 (socket says 172.21.15.1:56986) 2024-02-02T15:26:05.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.191+0000 7f5c27e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c28100d50 0x7f5c28074af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55038/0 (socket says 172.21.15.1:55038) 2024-02-02T15:26:05.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.191+0000 7f5c27e00700 1 -- 172.21.15.1:0/212477051 learned_addr learned my addr 172.21.15.1:0/212477051 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:05.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.191+0000 7f5c27400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5c28101f40 0x7f5c28073130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:56212/0 (socket says 172.21.15.1:56212) 2024-02-02T15:26:05.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.191+0000 7f5c2d200700 1 -- 172.21.15.1:0/212477051 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5c28101f40 msgr2=0x7f5c28073130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:05.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.191+0000 7f5c2d200700 1 --2- 172.21.15.1:0/212477051 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5c28101f40 0x7f5c28073130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:05.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.191+0000 7f5c2d200700 1 -- 172.21.15.1:0/212477051 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c28100d50 msgr2=0x7f5c28074af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:05.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.191+0000 7f5c2d200700 1 --2- 172.21.15.1:0/212477051 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c28100d50 0x7f5c28074af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:05.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.191+0000 7f5c2d200700 1 -- 172.21.15.1:0/212477051 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c20009d00 con 0x7f5c281028e0 2024-02-02T15:26:05.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.191+0000 7f5c27400700 1 --2- 172.21.15.1:0/212477051 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5c28101f40 0x7f5c28073130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:26:05.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.191+0000 7f5c27e00700 1 --2- 172.21.15.1:0/212477051 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c28100d50 0x7f5c28074af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:26:05.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.191+0000 7f5c2d200700 1 --2- 172.21.15.1:0/212477051 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c281028e0 0x7f5c28073670 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5c20007a30 tx=0x7f5c20008eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:05.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.192+0000 7f5c24c00700 1 -- 172.21.15.1:0/212477051 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c2000e040 con 0x7f5c281028e0 2024-02-02T15:26:05.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.192+0000 7f5c24c00700 1 -- 172.21.15.1:0/212477051 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c2000bd30 con 0x7f5c281028e0 2024-02-02T15:26:05.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.192+0000 7f5c24c00700 1 -- 172.21.15.1:0/212477051 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c200058d0 con 0x7f5c281028e0 2024-02-02T15:26:05.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.192+0000 7f5c2f200700 1 -- 172.21.15.1:0/212477051 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c281a0e50 con 0x7f5c281028e0 2024-02-02T15:26:05.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.192+0000 7f5c2f200700 1 -- 172.21.15.1:0/212477051 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c281a1220 con 0x7f5c281028e0 2024-02-02T15:26:05.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.193+0000 7f5c2f200700 1 -- 172.21.15.1:0/212477051 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c28108240 con 0x7f5c281028e0 2024-02-02T15:26:05.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.195+0000 7f5c24c00700 1 -- 172.21.15.1:0/212477051 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 17) v1 ==== 135120+0+0 (secure 0 0 0) 0x7f5c20005000 con 0x7f5c281028e0 2024-02-02T15:26:05.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.196+0000 7f5c24c00700 1 --2- 172.21.15.1:0/212477051 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f5c080a0630 0x7f5c080a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:05.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.196+0000 7f5c24c00700 1 -- 172.21.15.1:0/212477051 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f5c200cd780 con 0x7f5c281028e0 2024-02-02T15:26:05.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.196+0000 7f5c27e00700 1 --2- 172.21.15.1:0/212477051 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f5c080a0630 0x7f5c080a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:05.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.197+0000 7f5c27e00700 1 --2- 172.21.15.1:0/212477051 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f5c080a0630 0x7f5c080a2af0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5c180105c0 tx=0x7f5c1800b4a0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:05.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.198+0000 7f5c24c00700 1 -- 172.21.15.1:0/212477051 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f5c20097000 con 0x7f5c281028e0 2024-02-02T15:26:05.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.412+0000 7f5c2f200700 1 -- 172.21.15.1:0/212477051 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5c2802ce40 con 0x7f5c281028e0 2024-02-02T15:26:05.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.414+0000 7f5c24c00700 1 -- 172.21.15.1:0/212477051 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+24312 (secure 0 0 0) 0x7f5c2009a620 con 0x7f5c281028e0 2024-02-02T15:26:05.414 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:26:05.415 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":86,"fsid":"371a409c-c1de-11ee-95b6-87774f69a715","created":"2024-02-02T15:18:26.746506+0000","modified":"2024-02-02T15:25:27.269296+0000","last_up_change":"2024-02-02T15:24:08.800951+0000","last_in_change":"2024-02-02T15:23:52.784280+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T15:21:22.471754+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-02T15:24:46.195779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-02T15:24:47.226403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-02T15:24:50.300462+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"73e8a07f-3560-4c19-85ed-030d4d8f0f23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6803","nonce":2591906423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6805","nonce":2591906423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6809","nonce":2591906423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6807","nonce":2591906423}]},"public_addr":"172.21.15.1:6803/2591906423","cluster_addr":"172.21.15.1:6805/2591906423","heartbeat_back_addr":"172.21.15.1:6809/2591906423","heartbeat_front_addr":"172.21.15.1:6807/2591906423","state":["exists","up"]},{"osd":1,"uuid":"e53cffec-0ff9-4ed9-8dcf-ff47d04db2bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6801","nonce":747581719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6803","nonce":747581719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6807","nonce":747581719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6805","nonce":747581719}]},"public_addr":"172.21.15.71:6801/747581719","cluster_addr":"172.21.15.71:6803/747581719","heartbeat_back_addr":"172.21.15.71:6807/747581719","heartbeat_front_addr":"172.21.15.71:6805/747581719","state":["exists","up"]},{"osd":2,"uuid":"856c2347-294c-482b-8595-ad77f4792db4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6801","nonce":377511414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6803","nonce":377511414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6807","nonce":377511414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6805","nonce":377511414}]},"public_addr":"172.21.15.142:6801/377511414","cluster_addr":"172.21.15.142:6803/377511414","heartbeat_back_addr":"172.21.15.142:6807/377511414","heartbeat_front_addr":"172.21.15.142:6805/377511414","state":["exists","up"]},{"osd":3,"uuid":"4b946ad2-2da9-4e0b-9d61-d039d8669a42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6811","nonce":1086124477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6813","nonce":1086124477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6817","nonce":1086124477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6815","nonce":1086124477}]},"public_addr":"172.21.15.1:6811/1086124477","cluster_addr":"172.21.15.1:6813/1086124477","heartbeat_back_addr":"172.21.15.1:6817/1086124477","heartbeat_front_addr":"172.21.15.1:6815/1086124477","state":["exists","up"]},{"osd":4,"uuid":"43e36c43-47a0-4634-a45b-1da14a0e19d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6809","nonce":3095380305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6811","nonce":3095380305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6815","nonce":3095380305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6813","nonce":3095380305}]},"public_addr":"172.21.15.71:6809/3095380305","cluster_addr":"172.21.15.71:6811/3095380305","heartbeat_back_addr":"172.21.15.71:6815/3095380305","heartbeat_front_addr":"172.21.15.71:6813/3095380305","state":["exists","up"]},{"osd":5,"uuid":"eb48160e-ea7c-42f7-9477-be110bc8a34e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6809","nonce":4195149262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6811","nonce":4195149262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6815","nonce":4195149262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6813","nonce":4195149262}]},"public_addr":"172.21.15.142:6809/4195149262","cluster_addr":"172.21.15.142:6811/4195149262","heartbeat_back_addr":"172.21.15.142:6815/4195149262","heartbeat_front_addr":"172.21.15.142:6813/4195149262","state":["exists","up"]},{"osd":6,"uuid":"0ec4501a-5853-4ca6-b325-231f5b72335e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6819","nonce":1560153428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6821","nonce":1560153428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6825","nonce":1560153428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6823","nonce":1560153428}]},"public_addr":"172.21.15.1:6819/1560153428","cluster_addr":"172.21.15.1:6821/1560153428","heartbeat_back_addr":"172.21.15.1:6825/1560153428","heartbeat_front_addr":"172.21.15.1:6823/1560153428","state":["exists","up"]},{"osd":7,"uuid":"029debe8-9cdf-44f6-9aea-cabd1d185f93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6817","nonce":4080955538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6819","nonce":4080955538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6823","nonce":4080955538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6821","nonce":4080955538}]},"public_addr":"172.21.15.71:6817/4080955538","cluster_addr":"172.21.15.71:6819/4080955538","heartbeat_back_addr":"172.21.15.71:6823/4080955538","heartbeat_front_addr":"172.21.15.71:6821/4080955538","state":["exists","up"]},{"osd":8,"uuid":"7c90896c-9955-4c1b-aaca-7531dd5457f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6817","nonce":3942038209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6819","nonce":3942038209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6823","nonce":3942038209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6821","nonce":3942038209}]},"public_addr":"172.21.15.142:6817/3942038209","cluster_addr":"172.21.15.142:6819/3942038209","heartbeat_back_addr":"172.21.15.142:6823/3942038209","heartbeat_front_addr":"172.21.15.142:6821/3942038209","state":["exists","up"]},{"osd":9,"uuid":"7e165e68-31a8-4598-a577-de3e90ae5c5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6827","nonce":2716544850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6829","nonce":2716544850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6833","nonce":2716544850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6831","nonce":2716544850}]},"public_addr":"172.21.15.1:6827/2716544850","cluster_addr":"172.21.15.1:6829/2716544850","heartbeat_back_addr":"172.21.15.1:6833/2716544850","heartbeat_front_addr":"172.21.15.1:6831/2716544850","state":["exists","up"]},{"osd":10,"uuid":"48cd5fdb-0887-444a-aab8-a1de24282bb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6825","nonce":84691841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6827","nonce":84691841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6831","nonce":84691841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6829","nonce":84691841}]},"public_addr":"172.21.15.71:6825/84691841","cluster_addr":"172.21.15.71:6827/84691841","heartbeat_back_addr":"172.21.15.71:6831/84691841","heartbeat_front_addr":"172.21.15.71:6829/84691841","state":["exists","up"]},{"osd":11,"uuid":"f011dd27-11b0-4166-adfe-76756533163a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6825","nonce":2290179934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6827","nonce":2290179934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6831","nonce":2290179934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6829","nonce":2290179934}]},"public_addr":"172.21.15.142:6825/2290179934","cluster_addr":"172.21.15.142:6827/2290179934","heartbeat_back_addr":"172.21.15.142:6831/2290179934","heartbeat_front_addr":"172.21.15.142:6829/2290179934","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:20:44.060692+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:01.903817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:19.368258+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:40.028037+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:57.513361+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:22:14.805944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:22:35.188439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:22:53.014412+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:23:10.507420+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:23:31.849765+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:23:50.070612+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:24:07.388625+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":5,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/4092532060":"2024-02-03T15:19:25.836171+0000","172.21.15.1:6801/395498296":"2024-02-03T15:19:25.836171+0000","172.21.15.1:6800/395498296":"2024-02-03T15:19:25.836171+0000","172.21.15.1:0/1575373489":"2024-02-03T15:19:25.836171+0000","172.21.15.1:0/865983374":"2024-02-03T15:19:01.431596+0000","172.21.15.1:0/3165960227":"2024-02-03T15:19:01.431596+0000","172.21.15.1:0/2350830293":"2024-02-03T15:19:01.431596+0000","172.21.15.1:0/2959690250":"2024-02-03T15:19:25.836171+0000","172.21.15.1:6800/1162530732":"2024-02-03T15:19:01.431596+0000","172.21.15.1:6801/1162530732":"2024-02-03T15:19:01.431596+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T15:26:05.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.417+0000 7f5c2f200700 1 -- 172.21.15.1:0/212477051 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f5c080a0630 msgr2=0x7f5c080a2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:05.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.417+0000 7f5c2f200700 1 --2- 172.21.15.1:0/212477051 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f5c080a0630 0x7f5c080a2af0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5c180105c0 tx=0x7f5c1800b4a0 comp rx=0 tx=0).stop 2024-02-02T15:26:05.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.417+0000 7f5c2f200700 1 -- 172.21.15.1:0/212477051 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c281028e0 msgr2=0x7f5c28073670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:05.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.417+0000 7f5c2f200700 1 --2- 172.21.15.1:0/212477051 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c281028e0 0x7f5c28073670 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5c20007a30 tx=0x7f5c20008eb0 comp rx=0 tx=0).stop 2024-02-02T15:26:05.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.417+0000 7f5c2f200700 1 -- 172.21.15.1:0/212477051 shutdown_connections 2024-02-02T15:26:05.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.417+0000 7f5c2f200700 1 --2- 172.21.15.1:0/212477051 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f5c080a0630 0x7f5c080a2af0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:05.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.417+0000 7f5c2f200700 1 --2- 172.21.15.1:0/212477051 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5c281028e0 0x7f5c28073670 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:05.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.417+0000 7f5c2f200700 1 --2- 172.21.15.1:0/212477051 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5c28100d50 0x7f5c28074af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:05.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.417+0000 7f5c2f200700 1 --2- 172.21.15.1:0/212477051 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5c28101f40 0x7f5c28073130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:05.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.417+0000 7f5c2f200700 1 -- 172.21.15.1:0/212477051 >> 172.21.15.1:0/212477051 conn(0x7f5c280fc2c0 msgr2=0x7f5c28105380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:05.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.417+0000 7f5c2f200700 1 -- 172.21.15.1:0/212477051 shutdown_connections 2024-02-02T15:26:05.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.417+0000 7f5c2f200700 1 -- 172.21.15.1:0/212477051 wait complete. 2024-02-02T15:26:05.436 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-02T15:21:22.471754+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-02T15:24:46.195779+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-02T15:24:47.226403+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-02-02T15:24:50.300462+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-02-02T15:26:05.436 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-02T15:26:05.493 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:05 smithi001 ceph-mon[35470]: pgmap v275: 129 pgs: 129 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-02-02T15:26:05.493 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:05 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/4060802326' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-02T15:26:05.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:05 smithi071 ceph-mon[44707]: pgmap v275: 129 pgs: 129 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-02-02T15:26:05.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:05 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/4060802326' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-02T15:26:05.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.637+0000 7fd221e00700 1 -- 172.21.15.1:0/2231680718 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd21c101f70 msgr2=0x7fd21c1023f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:05.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.637+0000 7fd221e00700 1 --2- 172.21.15.1:0/2231680718 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd21c101f70 0x7fd21c1023f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd204009990 tx=0x7fd204009ca0 comp rx=0 tx=0).stop 2024-02-02T15:26:05.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.637+0000 7fd221e00700 1 -- 172.21.15.1:0/2231680718 shutdown_connections 2024-02-02T15:26:05.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.637+0000 7fd221e00700 1 --2- 172.21.15.1:0/2231680718 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd21c102930 0x7fd21c109250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:05.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.637+0000 7fd221e00700 1 --2- 172.21.15.1:0/2231680718 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd21c100e20 0x7fd21c101240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:05.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.638+0000 7fd221e00700 1 --2- 172.21.15.1:0/2231680718 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd21c101f70 0x7fd21c1023f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:05.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.638+0000 7fd221e00700 1 -- 172.21.15.1:0/2231680718 >> 172.21.15.1:0/2231680718 conn(0x7fd21c0fc390 msgr2=0x7fd21c0fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:05.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.638+0000 7fd221e00700 1 -- 172.21.15.1:0/2231680718 shutdown_connections 2024-02-02T15:26:05.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.638+0000 7fd221e00700 1 -- 172.21.15.1:0/2231680718 wait complete. 2024-02-02T15:26:05.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.639+0000 7fd221e00700 1 Processor -- start 2024-02-02T15:26:05.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.639+0000 7fd221e00700 1 -- start start 2024-02-02T15:26:05.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.639+0000 7fd221e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd21c100e20 0x7fd21c198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:05.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.639+0000 7fd221e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd21c101f70 0x7fd21c199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:05.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.639+0000 7fd221e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd21c102930 0x7fd21c1a0290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:05.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.639+0000 7fd221e00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd21c1a0860 con 0x7fd21c101f70 2024-02-02T15:26:05.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.639+0000 7fd221e00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd21c1a09d0 con 0x7fd21c100e20 2024-02-02T15:26:05.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.639+0000 7fd221e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd21c1a0b90 con 0x7fd21c102930 2024-02-02T15:26:05.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.639+0000 7fd21a000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd21c101f70 0x7fd21c199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:05.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.639+0000 7fd21b400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd21c102930 0x7fd21c1a0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:05.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.639+0000 7fd21aa00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd21c100e20 0x7fd21c198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:05.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.639+0000 7fd21b400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd21c102930 0x7fd21c1a0290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:57000/0 (socket says 172.21.15.1:57000) 2024-02-02T15:26:05.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.639+0000 7fd21a000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd21c101f70 0x7fd21c199260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55054/0 (socket says 172.21.15.1:55054) 2024-02-02T15:26:05.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.639+0000 7fd21b400700 1 -- 172.21.15.1:0/3064891146 learned_addr learned my addr 172.21.15.1:0/3064891146 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:05.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.640+0000 7fd21aa00700 1 -- 172.21.15.1:0/3064891146 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd21c102930 msgr2=0x7fd21c1a0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:05.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.640+0000 7fd21aa00700 1 --2- 172.21.15.1:0/3064891146 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd21c102930 0x7fd21c1a0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:05.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.640+0000 7fd21aa00700 1 -- 172.21.15.1:0/3064891146 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd21c101f70 msgr2=0x7fd21c199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:05.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.640+0000 7fd21aa00700 1 --2- 172.21.15.1:0/3064891146 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd21c101f70 0x7fd21c199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:05.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.640+0000 7fd21aa00700 1 -- 172.21.15.1:0/3064891146 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2040095f0 con 0x7fd21c100e20 2024-02-02T15:26:05.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.640+0000 7fd21a000700 1 --2- 172.21.15.1:0/3064891146 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd21c101f70 0x7fd21c199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:26:05.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.640+0000 7fd21aa00700 1 --2- 172.21.15.1:0/3064891146 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd21c100e20 0x7fd21c198d20 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd20c00e9f0 tx=0x7fd20c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:05.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.640+0000 7fd220a00700 1 -- 172.21.15.1:0/3064891146 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd20c00c9c0 con 0x7fd21c100e20 2024-02-02T15:26:05.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.640+0000 7fd221e00700 1 -- 172.21.15.1:0/3064891146 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd21c1a0e70 con 0x7fd21c100e20 2024-02-02T15:26:05.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.641+0000 7fd220a00700 1 -- 172.21.15.1:0/3064891146 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd20c010430 con 0x7fd21c100e20 2024-02-02T15:26:05.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.641+0000 7fd220a00700 1 -- 172.21.15.1:0/3064891146 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd20c018680 con 0x7fd21c100e20 2024-02-02T15:26:05.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.641+0000 7fd221e00700 1 -- 172.21.15.1:0/3064891146 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd21c1a12a0 con 0x7fd21c100e20 2024-02-02T15:26:05.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.641+0000 7fd221e00700 1 -- 172.21.15.1:0/3064891146 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd21c10c670 con 0x7fd21c100e20 2024-02-02T15:26:05.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.644+0000 7fd220a00700 1 -- 172.21.15.1:0/3064891146 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 135120+0+0 (secure 0 0 0) 0x7fd20c010860 con 0x7fd21c100e20 2024-02-02T15:26:05.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.645+0000 7fd220a00700 1 --2- 172.21.15.1:0/3064891146 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fd1fc0a05b0 0x7fd1fc0a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:05.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.645+0000 7fd220a00700 1 -- 172.21.15.1:0/3064891146 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fd20c014070 con 0x7fd21c100e20 2024-02-02T15:26:05.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.645+0000 7fd21a000700 1 --2- 172.21.15.1:0/3064891146 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fd1fc0a05b0 0x7fd1fc0a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:05.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.646+0000 7fd21a000700 1 --2- 172.21.15.1:0/3064891146 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fd1fc0a05b0 0x7fd1fc0a2a70 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd20400b2e0 tx=0x7fd204005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:05.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.647+0000 7fd220a00700 1 -- 172.21.15.1:0/3064891146 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7fd20c0957f0 con 0x7fd21c100e20 2024-02-02T15:26:05.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:05 smithi142 ceph-mon[45728]: pgmap v275: 129 pgs: 129 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-02-02T15:26:05.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:05 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/4060802326' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-02T15:26:05.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.856+0000 7fd221e00700 1 -- 172.21.15.1:0/3064891146 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd21c1a1970 con 0x7fd21c100e20 2024-02-02T15:26:05.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.857+0000 7fd220a00700 1 -- 172.21.15.1:0/3064891146 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd20c098e10 con 0x7fd21c100e20 2024-02-02T15:26:05.857 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-02-02T15:26:05.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.859+0000 7fd221e00700 1 -- 172.21.15.1:0/3064891146 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fd1fc0a05b0 msgr2=0x7fd1fc0a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:05.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.859+0000 7fd221e00700 1 --2- 172.21.15.1:0/3064891146 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fd1fc0a05b0 0x7fd1fc0a2a70 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd20400b2e0 tx=0x7fd204005fb0 comp rx=0 tx=0).stop 2024-02-02T15:26:05.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.859+0000 7fd221e00700 1 -- 172.21.15.1:0/3064891146 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd21c100e20 msgr2=0x7fd21c198d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:05.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.859+0000 7fd221e00700 1 --2- 172.21.15.1:0/3064891146 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd21c100e20 0x7fd21c198d20 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd20c00e9f0 tx=0x7fd20c00ed00 comp rx=0 tx=0).stop 2024-02-02T15:26:05.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.859+0000 7fd221e00700 1 -- 172.21.15.1:0/3064891146 shutdown_connections 2024-02-02T15:26:05.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.859+0000 7fd221e00700 1 --2- 172.21.15.1:0/3064891146 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fd1fc0a05b0 0x7fd1fc0a2a70 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:05.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.859+0000 7fd221e00700 1 --2- 172.21.15.1:0/3064891146 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd21c102930 0x7fd21c1a0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:05.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.859+0000 7fd221e00700 1 --2- 172.21.15.1:0/3064891146 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd21c100e20 0x7fd21c198d20 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:05.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.859+0000 7fd221e00700 1 --2- 172.21.15.1:0/3064891146 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd21c101f70 0x7fd21c199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:05.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.859+0000 7fd221e00700 1 -- 172.21.15.1:0/3064891146 >> 172.21.15.1:0/3064891146 conn(0x7fd21c0fc390 msgr2=0x7fd21c0fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:05.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.859+0000 7fd221e00700 1 -- 172.21.15.1:0/3064891146 shutdown_connections 2024-02-02T15:26:05.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:05.859+0000 7fd221e00700 1 -- 172.21.15.1:0/3064891146 wait complete. 2024-02-02T15:26:05.877 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-02T15:26:06.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.080+0000 7fecbf000700 1 -- 172.21.15.1:0/2029624554 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fecb8101f80 msgr2=0x7fecb8102400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:06.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.080+0000 7fecbf000700 1 --2- 172.21.15.1:0/2029624554 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fecb8101f80 0x7fecb8102400 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7feca0009990 tx=0x7feca0009ca0 comp rx=0 tx=0).stop 2024-02-02T15:26:06.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.081+0000 7fecbf000700 1 -- 172.21.15.1:0/2029624554 shutdown_connections 2024-02-02T15:26:06.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.081+0000 7fecbf000700 1 --2- 172.21.15.1:0/2029624554 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fecb8102940 0x7fecb8109260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:06.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.081+0000 7fecbf000700 1 --2- 172.21.15.1:0/2029624554 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fecb8100e30 0x7fecb8101250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:06.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.081+0000 7fecbf000700 1 --2- 172.21.15.1:0/2029624554 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fecb8101f80 0x7fecb8102400 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:06.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.081+0000 7fecbf000700 1 -- 172.21.15.1:0/2029624554 >> 172.21.15.1:0/2029624554 conn(0x7fecb80fc3a0 msgr2=0x7fecb80fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:06.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.081+0000 7fecbf000700 1 -- 172.21.15.1:0/2029624554 shutdown_connections 2024-02-02T15:26:06.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.081+0000 7fecbf000700 1 -- 172.21.15.1:0/2029624554 wait complete. 2024-02-02T15:26:06.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.082+0000 7fecbf000700 1 Processor -- start 2024-02-02T15:26:06.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.082+0000 7fecbf000700 1 -- start start 2024-02-02T15:26:06.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.082+0000 7fecbf000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fecb8100e30 0x7fecb8198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:06.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.082+0000 7fecbf000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fecb8101f80 0x7fecb8199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:06.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.082+0000 7fecbf000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fecb8102940 0x7fecb81a0290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:06.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.082+0000 7fecbf000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecb81a0860 con 0x7fecb8100e30 2024-02-02T15:26:06.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.082+0000 7fecbf000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecb81a09d0 con 0x7fecb8102940 2024-02-02T15:26:06.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.082+0000 7fecbf000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecb81a0b90 con 0x7fecb8101f80 2024-02-02T15:26:06.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.082+0000 7fecb7e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fecb8100e30 0x7fecb8198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:06.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.083+0000 7fecb7e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fecb8100e30 0x7fecb8198d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55074/0 (socket says 172.21.15.1:55074) 2024-02-02T15:26:06.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.083+0000 7fecb7e00700 1 -- 172.21.15.1:0/3802943062 learned_addr learned my addr 172.21.15.1:0/3802943062 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:06.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.083+0000 7fecbd000700 1 --2- 172.21.15.1:0/3802943062 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fecb8102940 0x7fecb81a0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:06.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.083+0000 7fecb7400700 1 --2- 172.21.15.1:0/3802943062 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fecb8101f80 0x7fecb8199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:06.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.083+0000 7fecb7e00700 1 -- 172.21.15.1:0/3802943062 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fecb8101f80 msgr2=0x7fecb8199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:06.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.083+0000 7fecb7e00700 1 --2- 172.21.15.1:0/3802943062 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fecb8101f80 0x7fecb8199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:06.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.083+0000 7fecb7e00700 1 -- 172.21.15.1:0/3802943062 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fecb8102940 msgr2=0x7fecb81a0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:06.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.083+0000 7fecb7e00700 1 --2- 172.21.15.1:0/3802943062 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fecb8102940 0x7fecb81a0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:06.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.083+0000 7fecb7e00700 1 -- 172.21.15.1:0/3802943062 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecb0009c40 con 0x7fecb8100e30 2024-02-02T15:26:06.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.083+0000 7fecb7e00700 1 --2- 172.21.15.1:0/3802943062 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fecb8100e30 0x7fecb8198d20 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7feca800e9a0 tx=0x7feca800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:06.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.084+0000 7fecb4c00700 1 -- 172.21.15.1:0/3802943062 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feca8009790 con 0x7fecb8100e30 2024-02-02T15:26:06.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.084+0000 7fecbf000700 1 -- 172.21.15.1:0/3802943062 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feca00095f0 con 0x7fecb8100e30 2024-02-02T15:26:06.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.084+0000 7fecb4c00700 1 -- 172.21.15.1:0/3802943062 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feca800ce80 con 0x7fecb8100e30 2024-02-02T15:26:06.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.084+0000 7fecb4c00700 1 -- 172.21.15.1:0/3802943062 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feca8018680 con 0x7fecb8100e30 2024-02-02T15:26:06.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.084+0000 7fecbf000700 1 -- 172.21.15.1:0/3802943062 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecb81a1190 con 0x7fecb8100e30 2024-02-02T15:26:06.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.085+0000 7fecbf000700 1 -- 172.21.15.1:0/3802943062 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecb8193470 con 0x7fecb8100e30 2024-02-02T15:26:06.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.087+0000 7fecb4c00700 1 -- 172.21.15.1:0/3802943062 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) v1 ==== 135120+0+0 (secure 0 0 0) 0x7feca8010430 con 0x7fecb8100e30 2024-02-02T15:26:06.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.088+0000 7fecb4c00700 1 --2- 172.21.15.1:0/3802943062 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fec980a05b0 0x7fec980a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:06.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.088+0000 7fecb4c00700 1 -- 172.21.15.1:0/3802943062 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7feca8014070 con 0x7fecb8100e30 2024-02-02T15:26:06.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.088+0000 7fecb7400700 1 --2- 172.21.15.1:0/3802943062 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fec980a05b0 0x7fec980a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:06.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.089+0000 7fecb7400700 1 --2- 172.21.15.1:0/3802943062 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fec980a05b0 0x7fec980a2a70 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7feca000b2e0 tx=0x7feca0005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:06.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.092+0000 7fecb4c00700 1 -- 172.21.15.1:0/3802943062 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7feca8052110 con 0x7fecb8100e30 2024-02-02T15:26:06.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.306+0000 7fecbf000700 1 -- 172.21.15.1:0/3802943062 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fecb802cc70 con 0x7fecb8100e30 2024-02-02T15:26:06.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.306+0000 7fecb4c00700 1 -- 172.21.15.1:0/3802943062 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v86) v1 ==== 104+0+11 (secure 0 0 0) 0x7feca8095730 con 0x7fecb8100e30 2024-02-02T15:26:06.306 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-02-02T15:26:06.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.309+0000 7fecbf000700 1 -- 172.21.15.1:0/3802943062 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fec980a05b0 msgr2=0x7fec980a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:06.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.309+0000 7fecbf000700 1 --2- 172.21.15.1:0/3802943062 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fec980a05b0 0x7fec980a2a70 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7feca000b2e0 tx=0x7feca0005fb0 comp rx=0 tx=0).stop 2024-02-02T15:26:06.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.309+0000 7fecbf000700 1 -- 172.21.15.1:0/3802943062 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fecb8100e30 msgr2=0x7fecb8198d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:06.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.309+0000 7fecbf000700 1 --2- 172.21.15.1:0/3802943062 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fecb8100e30 0x7fecb8198d20 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7feca800e9a0 tx=0x7feca800ecb0 comp rx=0 tx=0).stop 2024-02-02T15:26:06.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.309+0000 7fecbf000700 1 -- 172.21.15.1:0/3802943062 shutdown_connections 2024-02-02T15:26:06.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.309+0000 7fecbf000700 1 --2- 172.21.15.1:0/3802943062 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fecb8102940 0x7fecb81a0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:06.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.309+0000 7fecbf000700 1 --2- 172.21.15.1:0/3802943062 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fecb8100e30 0x7fecb8198d20 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:06.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.309+0000 7fecbf000700 1 --2- 172.21.15.1:0/3802943062 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fec980a05b0 0x7fec980a2a70 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:06.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.309+0000 7fecbf000700 1 --2- 172.21.15.1:0/3802943062 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fecb8101f80 0x7fecb8199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:06.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.309+0000 7fecbf000700 1 -- 172.21.15.1:0/3802943062 >> 172.21.15.1:0/3802943062 conn(0x7fecb80fc3a0 msgr2=0x7fecb8107900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:06.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.309+0000 7fecbf000700 1 -- 172.21.15.1:0/3802943062 shutdown_connections 2024-02-02T15:26:06.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.309+0000 7fecbf000700 1 -- 172.21.15.1:0/3802943062 wait complete. 2024-02-02T15:26:06.328 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-02T15:26:06.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.528+0000 7fe198800700 1 -- 172.21.15.1:0/1543793030 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe190100e20 msgr2=0x7fe190101240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:06.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.528+0000 7fe198800700 1 --2- 172.21.15.1:0/1543793030 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe190100e20 0x7fe190101240 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe180009990 tx=0x7fe180009ca0 comp rx=0 tx=0).stop 2024-02-02T15:26:06.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.528+0000 7fe198800700 1 -- 172.21.15.1:0/1543793030 shutdown_connections 2024-02-02T15:26:06.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.528+0000 7fe198800700 1 --2- 172.21.15.1:0/1543793030 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe190102930 0x7fe190109250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:06.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.528+0000 7fe198800700 1 --2- 172.21.15.1:0/1543793030 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe190100e20 0x7fe190101240 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:06.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.528+0000 7fe198800700 1 --2- 172.21.15.1:0/1543793030 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe190101f70 0x7fe1901023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:06.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.528+0000 7fe198800700 1 -- 172.21.15.1:0/1543793030 >> 172.21.15.1:0/1543793030 conn(0x7fe1900fc390 msgr2=0x7fe1900fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:06.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.528+0000 7fe198800700 1 -- 172.21.15.1:0/1543793030 shutdown_connections 2024-02-02T15:26:06.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.528+0000 7fe198800700 1 -- 172.21.15.1:0/1543793030 wait complete. 2024-02-02T15:26:06.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.529+0000 7fe198800700 1 Processor -- start 2024-02-02T15:26:06.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.529+0000 7fe198800700 1 -- start start 2024-02-02T15:26:06.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.530+0000 7fe198800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe190100e20 0x7fe190074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:06.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.530+0000 7fe198800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe190101f70 0x7fe190073130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:06.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.530+0000 7fe198800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe190102930 0x7fe190073670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:06.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.530+0000 7fe198800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe190073e10 con 0x7fe190102930 2024-02-02T15:26:06.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.530+0000 7fe198800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe190073f80 con 0x7fe190100e20 2024-02-02T15:26:06.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.530+0000 7fe198800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1901a0b40 con 0x7fe190101f70 2024-02-02T15:26:06.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.530+0000 7fe195400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe190101f70 0x7fe190073130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:06.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.530+0000 7fe195e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe190100e20 0x7fe190074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:06.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.530+0000 7fe195400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe190101f70 0x7fe190073130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:57026/0 (socket says 172.21.15.1:57026) 2024-02-02T15:26:06.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.530+0000 7fe195400700 1 -- 172.21.15.1:0/2844097177 learned_addr learned my addr 172.21.15.1:0/2844097177 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:06.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.530+0000 7fe196800700 1 --2- 172.21.15.1:0/2844097177 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe190102930 0x7fe190073670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:06.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.530+0000 7fe195400700 1 -- 172.21.15.1:0/2844097177 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe190100e20 msgr2=0x7fe190074af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:06.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.530+0000 7fe195400700 1 --2- 172.21.15.1:0/2844097177 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe190100e20 0x7fe190074af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:06.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.530+0000 7fe195400700 1 -- 172.21.15.1:0/2844097177 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe190102930 msgr2=0x7fe190073670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:06.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.530+0000 7fe195400700 1 --2- 172.21.15.1:0/2844097177 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe190102930 0x7fe190073670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:06.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.530+0000 7fe195400700 1 -- 172.21.15.1:0/2844097177 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1800095f0 con 0x7fe190101f70 2024-02-02T15:26:06.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.531+0000 7fe195e00700 1 --2- 172.21.15.1:0/2844097177 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe190100e20 0x7fe190074af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:26:06.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.531+0000 7fe195400700 1 --2- 172.21.15.1:0/2844097177 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe190101f70 0x7fe190073130 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe17800e9a0 tx=0x7fe17800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:06.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.531+0000 7fe18ea00700 1 -- 172.21.15.1:0/2844097177 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe178009790 con 0x7fe190101f70 2024-02-02T15:26:06.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.531+0000 7fe18ea00700 1 -- 172.21.15.1:0/2844097177 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe178010430 con 0x7fe190101f70 2024-02-02T15:26:06.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.531+0000 7fe198800700 1 -- 172.21.15.1:0/2844097177 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1901a0e20 con 0x7fe190101f70 2024-02-02T15:26:06.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.531+0000 7fe18ea00700 1 -- 172.21.15.1:0/2844097177 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe178018680 con 0x7fe190101f70 2024-02-02T15:26:06.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.531+0000 7fe198800700 1 -- 172.21.15.1:0/2844097177 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1901a12a0 con 0x7fe190101f70 2024-02-02T15:26:06.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.532+0000 7fe198800700 1 -- 172.21.15.1:0/2844097177 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe190107d60 con 0x7fe190101f70 2024-02-02T15:26:06.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.534+0000 7fe18ea00700 1 -- 172.21.15.1:0/2844097177 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 17) v1 ==== 135120+0+0 (secure 0 0 0) 0x7fe1780105a0 con 0x7fe190101f70 2024-02-02T15:26:06.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.535+0000 7fe18ea00700 1 --2- 172.21.15.1:0/2844097177 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fe1700a05b0 0x7fe1700a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:06.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.536+0000 7fe18ea00700 1 -- 172.21.15.1:0/2844097177 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fe178014070 con 0x7fe190101f70 2024-02-02T15:26:06.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.536+0000 7fe195e00700 1 --2- 172.21.15.1:0/2844097177 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fe1700a05b0 0x7fe1700a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:06.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.536+0000 7fe195e00700 1 --2- 172.21.15.1:0/2844097177 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fe1700a05b0 0x7fe1700a2a70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe18000b2e0 tx=0x7fe180005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:06.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.538+0000 7fe18ea00700 1 -- 172.21.15.1:0/2844097177 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7fe178095920 con 0x7fe190101f70 2024-02-02T15:26:06.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:06 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/212477051' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-02T15:26:06.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:06 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/3064891146' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-02T15:26:06.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:06 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/212477051' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-02T15:26:06.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:06 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/3064891146' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-02T15:26:06.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:06 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/212477051' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-02T15:26:06.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:06 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/3064891146' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-02T15:26:06.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.747+0000 7fe198800700 1 -- 172.21.15.1:0/2844097177 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe19002ce00 con 0x7fe190101f70 2024-02-02T15:26:06.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.748+0000 7fe18ea00700 1 -- 172.21.15.1:0/2844097177 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v86) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe178026070 con 0x7fe190101f70 2024-02-02T15:26:06.748 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 64 2024-02-02T15:26:06.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.751+0000 7fe198800700 1 -- 172.21.15.1:0/2844097177 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fe1700a05b0 msgr2=0x7fe1700a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:06.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.751+0000 7fe198800700 1 --2- 172.21.15.1:0/2844097177 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fe1700a05b0 0x7fe1700a2a70 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe18000b2e0 tx=0x7fe180005fb0 comp rx=0 tx=0).stop 2024-02-02T15:26:06.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.751+0000 7fe198800700 1 -- 172.21.15.1:0/2844097177 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe190101f70 msgr2=0x7fe190073130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:06.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.751+0000 7fe198800700 1 --2- 172.21.15.1:0/2844097177 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe190101f70 0x7fe190073130 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe17800e9a0 tx=0x7fe17800ecb0 comp rx=0 tx=0).stop 2024-02-02T15:26:06.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.751+0000 7fe198800700 1 -- 172.21.15.1:0/2844097177 shutdown_connections 2024-02-02T15:26:06.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.751+0000 7fe198800700 1 --2- 172.21.15.1:0/2844097177 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe190102930 0x7fe190073670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:06.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.751+0000 7fe198800700 1 --2- 172.21.15.1:0/2844097177 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe190100e20 0x7fe190074af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:06.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.751+0000 7fe198800700 1 --2- 172.21.15.1:0/2844097177 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fe1700a05b0 0x7fe1700a2a70 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:06.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.751+0000 7fe198800700 1 --2- 172.21.15.1:0/2844097177 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe190101f70 0x7fe190073130 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:06.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.751+0000 7fe198800700 1 -- 172.21.15.1:0/2844097177 >> 172.21.15.1:0/2844097177 conn(0x7fe1900fc390 msgr2=0x7fe1900fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:06.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.751+0000 7fe198800700 1 -- 172.21.15.1:0/2844097177 shutdown_connections 2024-02-02T15:26:06.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.751+0000 7fe198800700 1 -- 172.21.15.1:0/2844097177 wait complete. 2024-02-02T15:26:06.770 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-02-02T15:26:06.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.970+0000 7fe121400700 1 -- 172.21.15.1:0/705053449 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe11c100e20 msgr2=0x7fe11c101240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:06.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.970+0000 7fe121400700 1 --2- 172.21.15.1:0/705053449 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe11c100e20 0x7fe11c101240 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe10c009990 tx=0x7fe10c009ca0 comp rx=0 tx=0).stop 2024-02-02T15:26:06.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.970+0000 7fe121400700 1 -- 172.21.15.1:0/705053449 shutdown_connections 2024-02-02T15:26:06.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.970+0000 7fe121400700 1 --2- 172.21.15.1:0/705053449 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe11c102930 0x7fe11c109250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:06.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.970+0000 7fe121400700 1 --2- 172.21.15.1:0/705053449 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe11c100e20 0x7fe11c101240 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:06.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.970+0000 7fe121400700 1 --2- 172.21.15.1:0/705053449 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe11c101f70 0x7fe11c1023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:06.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.970+0000 7fe121400700 1 -- 172.21.15.1:0/705053449 >> 172.21.15.1:0/705053449 conn(0x7fe11c0fc390 msgr2=0x7fe11c0fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:06.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.970+0000 7fe121400700 1 -- 172.21.15.1:0/705053449 shutdown_connections 2024-02-02T15:26:06.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.970+0000 7fe121400700 1 -- 172.21.15.1:0/705053449 wait complete. 2024-02-02T15:26:06.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.971+0000 7fe121400700 1 Processor -- start 2024-02-02T15:26:06.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.971+0000 7fe121400700 1 -- start start 2024-02-02T15:26:06.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.972+0000 7fe121400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe11c100e20 0x7fe11c196ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:06.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.972+0000 7fe121400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe11c101f70 0x7fe11c197020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:06.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.972+0000 7fe121400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe11c102930 0x7fe11c19e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:06.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.972+0000 7fe121400700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe11c19e660 con 0x7fe11c102930 2024-02-02T15:26:06.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.972+0000 7fe121400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe11c19e7a0 con 0x7fe11c101f70 2024-02-02T15:26:06.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.972+0000 7fe121400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe11c19e930 con 0x7fe11c100e20 2024-02-02T15:26:06.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.972+0000 7fe11a800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe11c100e20 0x7fe11c196ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:06.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.972+0000 7fe119e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe11c101f70 0x7fe11c197020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:06.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.972+0000 7fe11a800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe11c100e20 0x7fe11c196ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:57038/0 (socket says 172.21.15.1:57038) 2024-02-02T15:26:06.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.972+0000 7fe11a800700 1 -- 172.21.15.1:0/396652874 learned_addr learned my addr 172.21.15.1:0/396652874 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:06.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.972+0000 7fe11b200700 1 --2- 172.21.15.1:0/396652874 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe11c102930 0x7fe11c19e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:06.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.972+0000 7fe119e00700 1 -- 172.21.15.1:0/396652874 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe11c100e20 msgr2=0x7fe11c196ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:06.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.972+0000 7fe119e00700 1 --2- 172.21.15.1:0/396652874 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe11c100e20 0x7fe11c196ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:06.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.972+0000 7fe119e00700 1 -- 172.21.15.1:0/396652874 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe11c102930 msgr2=0x7fe11c19e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:06.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.972+0000 7fe119e00700 1 --2- 172.21.15.1:0/396652874 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe11c102930 0x7fe11c19e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:06.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.972+0000 7fe119e00700 1 -- 172.21.15.1:0/396652874 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe104009520 con 0x7fe11c101f70 2024-02-02T15:26:06.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.972+0000 7fe11a800700 1 --2- 172.21.15.1:0/396652874 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe11c100e20 0x7fe11c196ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:26:06.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.973+0000 7fe119e00700 1 --2- 172.21.15.1:0/396652874 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe11c101f70 0x7fe11c197020 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe10400e9f0 tx=0x7fe10400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:06.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.973+0000 7fe113400700 1 -- 172.21.15.1:0/396652874 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe10400cb70 con 0x7fe11c101f70 2024-02-02T15:26:06.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.973+0000 7fe121400700 1 -- 172.21.15.1:0/396652874 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe10c0095f0 con 0x7fe11c101f70 2024-02-02T15:26:06.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.973+0000 7fe113400700 1 -- 172.21.15.1:0/396652874 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe10400ee90 con 0x7fe11c101f70 2024-02-02T15:26:06.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.973+0000 7fe113400700 1 -- 172.21.15.1:0/396652874 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe104003cf0 con 0x7fe11c101f70 2024-02-02T15:26:06.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.973+0000 7fe121400700 1 -- 172.21.15.1:0/396652874 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe11c19efe0 con 0x7fe11c101f70 2024-02-02T15:26:06.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.974+0000 7fe121400700 1 -- 172.21.15.1:0/396652874 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe11c107d60 con 0x7fe11c101f70 2024-02-02T15:26:06.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.976+0000 7fe113400700 1 -- 172.21.15.1:0/396652874 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 135120+0+0 (secure 0 0 0) 0x7fe10400ccd0 con 0x7fe11c101f70 2024-02-02T15:26:06.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.978+0000 7fe113400700 1 --2- 172.21.15.1:0/396652874 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fe0fc0a05b0 0x7fe0fc0a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:06.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.978+0000 7fe113400700 1 -- 172.21.15.1:0/396652874 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fe104014070 con 0x7fe11c101f70 2024-02-02T15:26:06.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.978+0000 7fe11a800700 1 --2- 172.21.15.1:0/396652874 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fe0fc0a05b0 0x7fe0fc0a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:06.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.978+0000 7fe11a800700 1 --2- 172.21.15.1:0/396652874 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fe0fc0a05b0 0x7fe0fc0a2a70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe10c00b2e0 tx=0x7fe10c005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:06.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:06.980+0000 7fe113400700 1 -- 172.21.15.1:0/396652874 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7fe104095910 con 0x7fe11c101f70 2024-02-02T15:26:07.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.191+0000 7fe121400700 1 -- 172.21.15.1:0/396652874 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fe11c02cec0 con 0x7fe11c101f70 2024-02-02T15:26:07.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.192+0000 7fe113400700 1 -- 172.21.15.1:0/396652874 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v86) v1 ==== 103+0+11 (secure 0 0 0) 0x7fe104098f30 con 0x7fe11c101f70 2024-02-02T15:26:07.192 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-02-02T15:26:07.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.194+0000 7fe121400700 1 -- 172.21.15.1:0/396652874 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fe0fc0a05b0 msgr2=0x7fe0fc0a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:07.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.194+0000 7fe121400700 1 --2- 172.21.15.1:0/396652874 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fe0fc0a05b0 0x7fe0fc0a2a70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe10c00b2e0 tx=0x7fe10c005fb0 comp rx=0 tx=0).stop 2024-02-02T15:26:07.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.194+0000 7fe121400700 1 -- 172.21.15.1:0/396652874 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe11c101f70 msgr2=0x7fe11c197020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:07.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.194+0000 7fe121400700 1 --2- 172.21.15.1:0/396652874 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe11c101f70 0x7fe11c197020 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe10400e9f0 tx=0x7fe10400ed00 comp rx=0 tx=0).stop 2024-02-02T15:26:07.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.195+0000 7fe121400700 1 -- 172.21.15.1:0/396652874 shutdown_connections 2024-02-02T15:26:07.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.195+0000 7fe121400700 1 --2- 172.21.15.1:0/396652874 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe11c102930 0x7fe11c19e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:07.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.195+0000 7fe121400700 1 --2- 172.21.15.1:0/396652874 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fe0fc0a05b0 0x7fe0fc0a2a70 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:07.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.195+0000 7fe121400700 1 --2- 172.21.15.1:0/396652874 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe11c100e20 0x7fe11c196ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:07.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.195+0000 7fe121400700 1 --2- 172.21.15.1:0/396652874 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe11c101f70 0x7fe11c197020 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:07.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.195+0000 7fe121400700 1 -- 172.21.15.1:0/396652874 >> 172.21.15.1:0/396652874 conn(0x7fe11c0fc390 msgr2=0x7fe11c0fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:07.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.195+0000 7fe121400700 1 -- 172.21.15.1:0/396652874 shutdown_connections 2024-02-02T15:26:07.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.195+0000 7fe121400700 1 -- 172.21.15.1:0/396652874 wait complete. 2024-02-02T15:26:07.213 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T15:26:07.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.414+0000 7fa093c00700 1 -- 172.21.15.1:0/526479365 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa08c101f30 msgr2=0x7fa08c102390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:07.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.414+0000 7fa093c00700 1 --2- 172.21.15.1:0/526479365 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa08c101f30 0x7fa08c102390 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa0740099c0 tx=0x7fa074009cd0 comp rx=0 tx=0).stop 2024-02-02T15:26:07.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.414+0000 7fa093c00700 1 -- 172.21.15.1:0/526479365 shutdown_connections 2024-02-02T15:26:07.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.414+0000 7fa093c00700 1 --2- 172.21.15.1:0/526479365 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa08c1028d0 0x7fa08c109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:07.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.414+0000 7fa093c00700 1 --2- 172.21.15.1:0/526479365 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa08c100d40 0x7fa08c101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:07.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.414+0000 7fa093c00700 1 --2- 172.21.15.1:0/526479365 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa08c101f30 0x7fa08c102390 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:07.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.414+0000 7fa093c00700 1 -- 172.21.15.1:0/526479365 >> 172.21.15.1:0/526479365 conn(0x7fa08c0fc290 msgr2=0x7fa08c0fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:07.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.414+0000 7fa093c00700 1 -- 172.21.15.1:0/526479365 shutdown_connections 2024-02-02T15:26:07.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.414+0000 7fa093c00700 1 -- 172.21.15.1:0/526479365 wait complete. 2024-02-02T15:26:07.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.415+0000 7fa093c00700 1 Processor -- start 2024-02-02T15:26:07.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.415+0000 7fa093c00700 1 -- start start 2024-02-02T15:26:07.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.415+0000 7fa093c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa08c100d40 0x7fa08c198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:07.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.415+0000 7fa093c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa08c101f30 0x7fa08c1991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:07.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.415+0000 7fa093c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa08c1028d0 0x7fa08c1a01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:07.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.416+0000 7fa093c00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa08c1a07a0 con 0x7fa08c1028d0 2024-02-02T15:26:07.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.416+0000 7fa093c00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa08c1a0910 con 0x7fa08c100d40 2024-02-02T15:26:07.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.416+0000 7fa093c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa08c1a0ad0 con 0x7fa08c101f30 2024-02-02T15:26:07.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.416+0000 7fa091c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa08c1028d0 0x7fa08c1a01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:07.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.416+0000 7fa08be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa08c101f30 0x7fa08c1991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:07.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.416+0000 7fa091200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa08c100d40 0x7fa08c198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:07.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.416+0000 7fa091c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa08c1028d0 0x7fa08c1a01d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55124/0 (socket says 172.21.15.1:55124) 2024-02-02T15:26:07.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.416+0000 7fa091c00700 1 -- 172.21.15.1:0/1675098712 learned_addr learned my addr 172.21.15.1:0/1675098712 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:07.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.416+0000 7fa091c00700 1 -- 172.21.15.1:0/1675098712 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa08c101f30 msgr2=0x7fa08c1991a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:07.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.416+0000 7fa091c00700 1 --2- 172.21.15.1:0/1675098712 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa08c101f30 0x7fa08c1991a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:07.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.416+0000 7fa091c00700 1 -- 172.21.15.1:0/1675098712 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa08c100d40 msgr2=0x7fa08c198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:07.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.416+0000 7fa091c00700 1 --2- 172.21.15.1:0/1675098712 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa08c100d40 0x7fa08c198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:07.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.416+0000 7fa091c00700 1 -- 172.21.15.1:0/1675098712 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa07c009520 con 0x7fa08c1028d0 2024-02-02T15:26:07.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.416+0000 7fa08be00700 1 --2- 172.21.15.1:0/1675098712 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa08c101f30 0x7fa08c1991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:26:07.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.417+0000 7fa091c00700 1 --2- 172.21.15.1:0/1675098712 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa08c1028d0 0x7fa08c1a01d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa08400ef30 tx=0x7fa08400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:07.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.417+0000 7fa089600700 1 -- 172.21.15.1:0/1675098712 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa084018070 con 0x7fa08c1028d0 2024-02-02T15:26:07.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.417+0000 7fa093c00700 1 -- 172.21.15.1:0/1675098712 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0740095f0 con 0x7fa08c1028d0 2024-02-02T15:26:07.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.417+0000 7fa089600700 1 -- 172.21.15.1:0/1675098712 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa08400f040 con 0x7fa08c1028d0 2024-02-02T15:26:07.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.417+0000 7fa089600700 1 -- 172.21.15.1:0/1675098712 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa08400cd00 con 0x7fa08c1028d0 2024-02-02T15:26:07.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.417+0000 7fa093c00700 1 -- 172.21.15.1:0/1675098712 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa08c1a10d0 con 0x7fa08c1028d0 2024-02-02T15:26:07.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.418+0000 7fa093c00700 1 -- 172.21.15.1:0/1675098712 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa08c193340 con 0x7fa08c1028d0 2024-02-02T15:26:07.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.420+0000 7fa089600700 1 -- 172.21.15.1:0/1675098712 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) v1 ==== 135120+0+0 (secure 0 0 0) 0x7fa084003a00 con 0x7fa08c1028d0 2024-02-02T15:26:07.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.422+0000 7fa089600700 1 --2- 172.21.15.1:0/1675098712 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fa06c0a05b0 0x7fa06c0a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:07.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.422+0000 7fa089600700 1 -- 172.21.15.1:0/1675098712 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fa0840cb990 con 0x7fa08c1028d0 2024-02-02T15:26:07.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.422+0000 7fa091200700 1 --2- 172.21.15.1:0/1675098712 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fa06c0a05b0 0x7fa06c0a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:07.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.423+0000 7fa091200700 1 --2- 172.21.15.1:0/1675098712 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fa06c0a05b0 0x7fa06c0a2a70 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa08c19a130 tx=0x7fa07c009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:07.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.426+0000 7fa089600700 1 -- 172.21.15.1:0/1675098712 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7fa0840950f0 con 0x7fa08c1028d0 2024-02-02T15:26:07.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:07 smithi071 ceph-mon[44707]: pgmap v276: 129 pgs: 129 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-02-02T15:26:07.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:07 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/3802943062' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-02T15:26:07.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:07 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/2844097177' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-02T15:26:07.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:07 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/396652874' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-02T15:26:07.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:07 smithi142 ceph-mon[45728]: pgmap v276: 129 pgs: 129 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-02-02T15:26:07.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:07 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/3802943062' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-02T15:26:07.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:07 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/2844097177' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-02T15:26:07.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:07 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/396652874' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-02T15:26:07.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.686+0000 7fa093c00700 1 -- 172.21.15.1:0/1675098712 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa08c02cc70 con 0x7fa08c1028d0 2024-02-02T15:26:07.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.687+0000 7fa089600700 1 -- 172.21.15.1:0/1675098712 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10866 (secure 0 0 0) 0x7fa084098710 con 0x7fa08c1028d0 2024-02-02T15:26:07.687 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:26:07.688 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2922240448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2922240448},{"type":"v1","addr":"172.21.15.1:6835","nonce":2922240448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2729497431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2729497431},{"type":"v1","addr":"172.21.15.1:6837","nonce":2729497431}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-02T15:24:47.761435+0000","modified":"2024-02-02T15:25:20.251601+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24529,"mds_2":24518,"mds_3":24514,"mds_4":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14517":{"gid":14517,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/1218303683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":1218303683},{"type":"v1","addr":"172.21.15.1:6839","nonce":1218303683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14529":{"gid":14529,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/2059197496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2059197496},{"type":"v1","addr":"172.21.15.1:6841","nonce":2059197496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/7648986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":7648986},{"type":"v1","addr":"172.21.15.142:6833","nonce":7648986}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/3482746094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3482746094},{"type":"v1","addr":"172.21.15.71:6833","nonce":3482746094}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/388207814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":388207814},{"type":"v1","addr":"172.21.15.71:6835","nonce":388207814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6835/1052606547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1052606547},{"type":"v1","addr":"172.21.15.142:6835","nonce":1052606547}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.142:6837/2870289019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2870289019},{"type":"v1","addr":"172.21.15.142:6837","nonce":2870289019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.71:6837/628016199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":628016199},{"type":"v1","addr":"172.21.15.71:6837","nonce":628016199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.142:6839/321675421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":321675421},{"type":"v1","addr":"172.21.15.142:6839","nonce":321675421}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/1268086493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1268086493},{"type":"v1","addr":"172.21.15.71:6839","nonce":1268086493}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-02T15:26:07.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:07 smithi001 ceph-mon[35470]: pgmap v276: 129 pgs: 129 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-02-02T15:26:07.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:07 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/3802943062' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-02T15:26:07.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:07 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/2844097177' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-02T15:26:07.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:07 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/396652874' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-02T15:26:07.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.690+0000 7fa093c00700 1 -- 172.21.15.1:0/1675098712 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fa06c0a05b0 msgr2=0x7fa06c0a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:07.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.690+0000 7fa093c00700 1 --2- 172.21.15.1:0/1675098712 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fa06c0a05b0 0x7fa06c0a2a70 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa08c19a130 tx=0x7fa07c009260 comp rx=0 tx=0).stop 2024-02-02T15:26:07.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.690+0000 7fa093c00700 1 -- 172.21.15.1:0/1675098712 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa08c1028d0 msgr2=0x7fa08c1a01d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:07.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.690+0000 7fa093c00700 1 --2- 172.21.15.1:0/1675098712 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa08c1028d0 0x7fa08c1a01d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa08400ef30 tx=0x7fa08400c3a0 comp rx=0 tx=0).stop 2024-02-02T15:26:07.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.690+0000 7fa093c00700 1 -- 172.21.15.1:0/1675098712 shutdown_connections 2024-02-02T15:26:07.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.690+0000 7fa093c00700 1 --2- 172.21.15.1:0/1675098712 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa08c1028d0 0x7fa08c1a01d0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:07.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.690+0000 7fa093c00700 1 --2- 172.21.15.1:0/1675098712 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa08c100d40 0x7fa08c198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:07.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.690+0000 7fa093c00700 1 --2- 172.21.15.1:0/1675098712 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fa06c0a05b0 0x7fa06c0a2a70 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:07.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.690+0000 7fa093c00700 1 --2- 172.21.15.1:0/1675098712 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa08c101f30 0x7fa08c1991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:07.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.690+0000 7fa093c00700 1 -- 172.21.15.1:0/1675098712 >> 172.21.15.1:0/1675098712 conn(0x7fa08c0fc290 msgr2=0x7fa08c107870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:07.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.691+0000 7fa093c00700 1 -- 172.21.15.1:0/1675098712 shutdown_connections 2024-02-02T15:26:07.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.691+0000 7fa093c00700 1 -- 172.21.15.1:0/1675098712 wait complete. 2024-02-02T15:26:07.692 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 27 2024-02-02T15:26:07.710 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T15:26:07.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.911+0000 7f5152200700 1 -- 172.21.15.1:0/852759791 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f514c0ffc70 msgr2=0x7f514c100090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:07.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.911+0000 7f5152200700 1 --2- 172.21.15.1:0/852759791 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f514c0ffc70 0x7f514c100090 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f513c0099c0 tx=0x7f513c009cd0 comp rx=0 tx=0).stop 2024-02-02T15:26:07.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.911+0000 7f5152200700 1 -- 172.21.15.1:0/852759791 shutdown_connections 2024-02-02T15:26:07.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.911+0000 7f5152200700 1 --2- 172.21.15.1:0/852759791 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f514c101800 0x7f514c1080b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:07.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.911+0000 7f5152200700 1 --2- 172.21.15.1:0/852759791 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f514c0ffc70 0x7f514c100090 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:07.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.911+0000 7f5152200700 1 --2- 172.21.15.1:0/852759791 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f514c100e60 0x7f514c1012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:07.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.911+0000 7f5152200700 1 -- 172.21.15.1:0/852759791 >> 172.21.15.1:0/852759791 conn(0x7f514c0fb1c0 msgr2=0x7f514c0fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:07.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.911+0000 7f5152200700 1 -- 172.21.15.1:0/852759791 shutdown_connections 2024-02-02T15:26:07.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.911+0000 7f5152200700 1 -- 172.21.15.1:0/852759791 wait complete. 2024-02-02T15:26:07.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.912+0000 7f5152200700 1 Processor -- start 2024-02-02T15:26:07.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.912+0000 7f5152200700 1 -- start start 2024-02-02T15:26:07.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.912+0000 7f5152200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f514c0ffc70 0x7f514c196a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:07.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.912+0000 7f5152200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f514c100e60 0x7f514c196f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:07.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.912+0000 7f5152200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f514c101800 0x7f514c19dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:07.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.912+0000 7f5152200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f514c19e5c0 con 0x7f514c100e60 2024-02-02T15:26:07.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.912+0000 7f5152200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f514c19e700 con 0x7f514c0ffc70 2024-02-02T15:26:07.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.912+0000 7f5152200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f514c19e890 con 0x7f514c101800 2024-02-02T15:26:07.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.913+0000 7f514aa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f514c100e60 0x7f514c196f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:07.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.913+0000 7f514aa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f514c100e60 0x7f514c196f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55140/0 (socket says 172.21.15.1:55140) 2024-02-02T15:26:07.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.913+0000 7f514aa00700 1 -- 172.21.15.1:0/3368344178 learned_addr learned my addr 172.21.15.1:0/3368344178 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:07.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.913+0000 7f514b400700 1 --2- 172.21.15.1:0/3368344178 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f514c0ffc70 0x7f514c196a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:07.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.913+0000 7f514be00700 1 --2- 172.21.15.1:0/3368344178 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f514c101800 0x7f514c19dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:07.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.913+0000 7f514aa00700 1 -- 172.21.15.1:0/3368344178 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f514c101800 msgr2=0x7f514c19dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:07.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.913+0000 7f514aa00700 1 --2- 172.21.15.1:0/3368344178 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f514c101800 0x7f514c19dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:07.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.913+0000 7f514aa00700 1 -- 172.21.15.1:0/3368344178 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f514c0ffc70 msgr2=0x7f514c196a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:07.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.913+0000 7f514aa00700 1 --2- 172.21.15.1:0/3368344178 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f514c0ffc70 0x7f514c196a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:07.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.913+0000 7f514aa00700 1 -- 172.21.15.1:0/3368344178 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f513c0095f0 con 0x7f514c100e60 2024-02-02T15:26:07.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.913+0000 7f514aa00700 1 --2- 172.21.15.1:0/3368344178 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f514c100e60 0x7f514c196f80 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f513400e9a0 tx=0x7f513400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:07.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.913+0000 7f5143e00700 1 -- 172.21.15.1:0/3368344178 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5134009790 con 0x7f514c100e60 2024-02-02T15:26:07.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.914+0000 7f5152200700 1 -- 172.21.15.1:0/3368344178 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f514c19eb70 con 0x7f514c100e60 2024-02-02T15:26:07.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.914+0000 7f5143e00700 1 -- 172.21.15.1:0/3368344178 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f513400cbd0 con 0x7f514c100e60 2024-02-02T15:26:07.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.914+0000 7f5143e00700 1 -- 172.21.15.1:0/3368344178 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5134018680 con 0x7f514c100e60 2024-02-02T15:26:07.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.914+0000 7f5152200700 1 -- 172.21.15.1:0/3368344178 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f514c19f080 con 0x7f514c100e60 2024-02-02T15:26:07.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.915+0000 7f5152200700 1 -- 172.21.15.1:0/3368344178 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f514c10b4d0 con 0x7f514c100e60 2024-02-02T15:26:07.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.918+0000 7f5143e00700 1 -- 172.21.15.1:0/3368344178 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) v1 ==== 135120+0+0 (secure 0 0 0) 0x7f513400cd40 con 0x7f514c100e60 2024-02-02T15:26:07.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.919+0000 7f5143e00700 1 --2- 172.21.15.1:0/3368344178 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f512c0a05b0 0x7f512c0a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:07.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.919+0000 7f5143e00700 1 -- 172.21.15.1:0/3368344178 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f5134014070 con 0x7f514c100e60 2024-02-02T15:26:07.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.919+0000 7f514b400700 1 --2- 172.21.15.1:0/3368344178 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f512c0a05b0 0x7f512c0a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:07.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.920+0000 7f514b400700 1 --2- 172.21.15.1:0/3368344178 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f512c0a05b0 0x7f512c0a2a70 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f513c009990 tx=0x7f513c005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:07.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:07.921+0000 7f5143e00700 1 -- 172.21.15.1:0/3368344178 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f5134095ad0 con 0x7f514c100e60 2024-02-02T15:26:08.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.132+0000 7f5152200700 1 -- 172.21.15.1:0/3368344178 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f514c10b660 con 0x7f514c100e60 2024-02-02T15:26:08.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.133+0000 7f5143e00700 1 -- 172.21.15.1:0/3368344178 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+24312 (secure 0 0 0) 0x7f51340990f0 con 0x7f514c100e60 2024-02-02T15:26:08.134 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:26:08.134 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":86,"fsid":"371a409c-c1de-11ee-95b6-87774f69a715","created":"2024-02-02T15:18:26.746506+0000","modified":"2024-02-02T15:25:27.269296+0000","last_up_change":"2024-02-02T15:24:08.800951+0000","last_in_change":"2024-02-02T15:23:52.784280+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T15:21:22.471754+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-02T15:24:46.195779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-02T15:24:47.226403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-02T15:24:50.300462+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"73e8a07f-3560-4c19-85ed-030d4d8f0f23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6803","nonce":2591906423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6805","nonce":2591906423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6809","nonce":2591906423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6807","nonce":2591906423}]},"public_addr":"172.21.15.1:6803/2591906423","cluster_addr":"172.21.15.1:6805/2591906423","heartbeat_back_addr":"172.21.15.1:6809/2591906423","heartbeat_front_addr":"172.21.15.1:6807/2591906423","state":["exists","up"]},{"osd":1,"uuid":"e53cffec-0ff9-4ed9-8dcf-ff47d04db2bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6801","nonce":747581719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6803","nonce":747581719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6807","nonce":747581719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6805","nonce":747581719}]},"public_addr":"172.21.15.71:6801/747581719","cluster_addr":"172.21.15.71:6803/747581719","heartbeat_back_addr":"172.21.15.71:6807/747581719","heartbeat_front_addr":"172.21.15.71:6805/747581719","state":["exists","up"]},{"osd":2,"uuid":"856c2347-294c-482b-8595-ad77f4792db4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6801","nonce":377511414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6803","nonce":377511414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6807","nonce":377511414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6805","nonce":377511414}]},"public_addr":"172.21.15.142:6801/377511414","cluster_addr":"172.21.15.142:6803/377511414","heartbeat_back_addr":"172.21.15.142:6807/377511414","heartbeat_front_addr":"172.21.15.142:6805/377511414","state":["exists","up"]},{"osd":3,"uuid":"4b946ad2-2da9-4e0b-9d61-d039d8669a42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6811","nonce":1086124477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6813","nonce":1086124477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6817","nonce":1086124477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6815","nonce":1086124477}]},"public_addr":"172.21.15.1:6811/1086124477","cluster_addr":"172.21.15.1:6813/1086124477","heartbeat_back_addr":"172.21.15.1:6817/1086124477","heartbeat_front_addr":"172.21.15.1:6815/1086124477","state":["exists","up"]},{"osd":4,"uuid":"43e36c43-47a0-4634-a45b-1da14a0e19d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6809","nonce":3095380305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6811","nonce":3095380305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6815","nonce":3095380305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6813","nonce":3095380305}]},"public_addr":"172.21.15.71:6809/3095380305","cluster_addr":"172.21.15.71:6811/3095380305","heartbeat_back_addr":"172.21.15.71:6815/3095380305","heartbeat_front_addr":"172.21.15.71:6813/3095380305","state":["exists","up"]},{"osd":5,"uuid":"eb48160e-ea7c-42f7-9477-be110bc8a34e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6809","nonce":4195149262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6811","nonce":4195149262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6815","nonce":4195149262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6813","nonce":4195149262}]},"public_addr":"172.21.15.142:6809/4195149262","cluster_addr":"172.21.15.142:6811/4195149262","heartbeat_back_addr":"172.21.15.142:6815/4195149262","heartbeat_front_addr":"172.21.15.142:6813/4195149262","state":["exists","up"]},{"osd":6,"uuid":"0ec4501a-5853-4ca6-b325-231f5b72335e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6819","nonce":1560153428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6821","nonce":1560153428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6825","nonce":1560153428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6823","nonce":1560153428}]},"public_addr":"172.21.15.1:6819/1560153428","cluster_addr":"172.21.15.1:6821/1560153428","heartbeat_back_addr":"172.21.15.1:6825/1560153428","heartbeat_front_addr":"172.21.15.1:6823/1560153428","state":["exists","up"]},{"osd":7,"uuid":"029debe8-9cdf-44f6-9aea-cabd1d185f93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6817","nonce":4080955538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6819","nonce":4080955538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6823","nonce":4080955538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6821","nonce":4080955538}]},"public_addr":"172.21.15.71:6817/4080955538","cluster_addr":"172.21.15.71:6819/4080955538","heartbeat_back_addr":"172.21.15.71:6823/4080955538","heartbeat_front_addr":"172.21.15.71:6821/4080955538","state":["exists","up"]},{"osd":8,"uuid":"7c90896c-9955-4c1b-aaca-7531dd5457f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6817","nonce":3942038209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6819","nonce":3942038209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6823","nonce":3942038209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6821","nonce":3942038209}]},"public_addr":"172.21.15.142:6817/3942038209","cluster_addr":"172.21.15.142:6819/3942038209","heartbeat_back_addr":"172.21.15.142:6823/3942038209","heartbeat_front_addr":"172.21.15.142:6821/3942038209","state":["exists","up"]},{"osd":9,"uuid":"7e165e68-31a8-4598-a577-de3e90ae5c5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6827","nonce":2716544850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6829","nonce":2716544850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6833","nonce":2716544850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6831","nonce":2716544850}]},"public_addr":"172.21.15.1:6827/2716544850","cluster_addr":"172.21.15.1:6829/2716544850","heartbeat_back_addr":"172.21.15.1:6833/2716544850","heartbeat_front_addr":"172.21.15.1:6831/2716544850","state":["exists","up"]},{"osd":10,"uuid":"48cd5fdb-0887-444a-aab8-a1de24282bb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6825","nonce":84691841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6827","nonce":84691841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6831","nonce":84691841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6829","nonce":84691841}]},"public_addr":"172.21.15.71:6825/84691841","cluster_addr":"172.21.15.71:6827/84691841","heartbeat_back_addr":"172.21.15.71:6831/84691841","heartbeat_front_addr":"172.21.15.71:6829/84691841","state":["exists","up"]},{"osd":11,"uuid":"f011dd27-11b0-4166-adfe-76756533163a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6825","nonce":2290179934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6827","nonce":2290179934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6831","nonce":2290179934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6829","nonce":2290179934}]},"public_addr":"172.21.15.142:6825/2290179934","cluster_addr":"172.21.15.142:6827/2290179934","heartbeat_back_addr":"172.21.15.142:6831/2290179934","heartbeat_front_addr":"172.21.15.142:6829/2290179934","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:20:44.060692+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:01.903817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:19.368258+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:40.028037+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:57.513361+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:22:14.805944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:22:35.188439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:22:53.014412+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:23:10.507420+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:23:31.849765+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:23:50.070612+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:24:07.388625+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":5,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/4092532060":"2024-02-03T15:19:25.836171+0000","172.21.15.1:6801/395498296":"2024-02-03T15:19:25.836171+0000","172.21.15.1:6800/395498296":"2024-02-03T15:19:25.836171+0000","172.21.15.1:0/1575373489":"2024-02-03T15:19:25.836171+0000","172.21.15.1:0/865983374":"2024-02-03T15:19:01.431596+0000","172.21.15.1:0/3165960227":"2024-02-03T15:19:01.431596+0000","172.21.15.1:0/2350830293":"2024-02-03T15:19:01.431596+0000","172.21.15.1:0/2959690250":"2024-02-03T15:19:25.836171+0000","172.21.15.1:6800/1162530732":"2024-02-03T15:19:01.431596+0000","172.21.15.1:6801/1162530732":"2024-02-03T15:19:01.431596+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T15:26:08.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.136+0000 7f5152200700 1 -- 172.21.15.1:0/3368344178 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f512c0a05b0 msgr2=0x7f512c0a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:08.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.136+0000 7f5152200700 1 --2- 172.21.15.1:0/3368344178 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f512c0a05b0 0x7f512c0a2a70 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f513c009990 tx=0x7f513c005fb0 comp rx=0 tx=0).stop 2024-02-02T15:26:08.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.136+0000 7f5152200700 1 -- 172.21.15.1:0/3368344178 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f514c100e60 msgr2=0x7f514c196f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:08.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.136+0000 7f5152200700 1 --2- 172.21.15.1:0/3368344178 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f514c100e60 0x7f514c196f80 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f513400e9a0 tx=0x7f513400ecb0 comp rx=0 tx=0).stop 2024-02-02T15:26:08.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.136+0000 7f5152200700 1 -- 172.21.15.1:0/3368344178 shutdown_connections 2024-02-02T15:26:08.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.136+0000 7f5152200700 1 --2- 172.21.15.1:0/3368344178 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f512c0a05b0 0x7f512c0a2a70 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:08.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.136+0000 7f5152200700 1 --2- 172.21.15.1:0/3368344178 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f514c101800 0x7f514c19dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:08.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.136+0000 7f5152200700 1 --2- 172.21.15.1:0/3368344178 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f514c0ffc70 0x7f514c196a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:08.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.136+0000 7f5152200700 1 --2- 172.21.15.1:0/3368344178 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f514c100e60 0x7f514c196f80 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:08.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.136+0000 7f5152200700 1 -- 172.21.15.1:0/3368344178 >> 172.21.15.1:0/3368344178 conn(0x7f514c0fb1c0 msgr2=0x7f514c068d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:08.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.136+0000 7f5152200700 1 -- 172.21.15.1:0/3368344178 shutdown_connections 2024-02-02T15:26:08.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.136+0000 7f5152200700 1 -- 172.21.15.1:0/3368344178 wait complete. 2024-02-02T15:26:08.155 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-02T15:21:22.471754+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-02T15:24:46.195779+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-02T15:24:47.226403+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-02-02T15:24:50.300462+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-02-02T15:26:08.156 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-02T15:26:08.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.358+0000 7f800ce00700 1 -- 172.21.15.1:0/1224429053 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8008100e20 msgr2=0x7f8008101240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:08.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.358+0000 7f800ce00700 1 --2- 172.21.15.1:0/1224429053 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8008100e20 0x7f8008101240 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f7ff8009990 tx=0x7f7ff8009ca0 comp rx=0 tx=0).stop 2024-02-02T15:26:08.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.358+0000 7f800ce00700 1 -- 172.21.15.1:0/1224429053 shutdown_connections 2024-02-02T15:26:08.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.358+0000 7f800ce00700 1 --2- 172.21.15.1:0/1224429053 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8008102930 0x7f8008109250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:08.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.358+0000 7f800ce00700 1 --2- 172.21.15.1:0/1224429053 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8008100e20 0x7f8008101240 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:08.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.358+0000 7f800ce00700 1 --2- 172.21.15.1:0/1224429053 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8008101f70 0x7f80081023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:08.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.358+0000 7f800ce00700 1 -- 172.21.15.1:0/1224429053 >> 172.21.15.1:0/1224429053 conn(0x7f80080fc390 msgr2=0x7f80080fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:08.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.358+0000 7f800ce00700 1 -- 172.21.15.1:0/1224429053 shutdown_connections 2024-02-02T15:26:08.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.358+0000 7f800ce00700 1 -- 172.21.15.1:0/1224429053 wait complete. 2024-02-02T15:26:08.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.359+0000 7f800ce00700 1 Processor -- start 2024-02-02T15:26:08.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.359+0000 7f800ce00700 1 -- start start 2024-02-02T15:26:08.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.360+0000 7f800ce00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8008100e20 0x7f8008198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:08.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.360+0000 7f800ce00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8008101f70 0x7f8008199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:08.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.360+0000 7f800ce00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8008102930 0x7f80081a02c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:08.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.360+0000 7f800ce00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80081a0890 con 0x7f8008101f70 2024-02-02T15:26:08.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.360+0000 7f800ce00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80081a0a00 con 0x7f8008102930 2024-02-02T15:26:08.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.360+0000 7f800ce00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80081a0bc0 con 0x7f8008100e20 2024-02-02T15:26:08.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.360+0000 7f8005400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8008101f70 0x7f8008199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:08.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.360+0000 7f8005400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8008101f70 0x7f8008199290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55168/0 (socket says 172.21.15.1:55168) 2024-02-02T15:26:08.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.360+0000 7f8005400700 1 -- 172.21.15.1:0/1734709728 learned_addr learned my addr 172.21.15.1:0/1734709728 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:08.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.360+0000 7f8005e00700 1 --2- 172.21.15.1:0/1734709728 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8008100e20 0x7f8008198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:08.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.360+0000 7f8006800700 1 --2- 172.21.15.1:0/1734709728 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8008102930 0x7f80081a02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:08.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.360+0000 7f8005400700 1 -- 172.21.15.1:0/1734709728 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8008100e20 msgr2=0x7f8008198d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:08.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.360+0000 7f8005400700 1 --2- 172.21.15.1:0/1734709728 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8008100e20 0x7f8008198d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:08.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.360+0000 7f8005400700 1 -- 172.21.15.1:0/1734709728 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8008102930 msgr2=0x7f80081a02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:08.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.360+0000 7f8005400700 1 --2- 172.21.15.1:0/1734709728 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8008102930 0x7f80081a02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:08.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.360+0000 7f8005400700 1 -- 172.21.15.1:0/1734709728 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ff80095f0 con 0x7f8008101f70 2024-02-02T15:26:08.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.361+0000 7f8005400700 1 --2- 172.21.15.1:0/1734709728 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8008101f70 0x7f8008199290 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7ff000e9a0 tx=0x7f7ff000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:08.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.361+0000 7f7ffea00700 1 -- 172.21.15.1:0/1734709728 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff0009790 con 0x7f8008101f70 2024-02-02T15:26:08.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.361+0000 7f7ffea00700 1 -- 172.21.15.1:0/1734709728 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7ff0010430 con 0x7f8008101f70 2024-02-02T15:26:08.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.361+0000 7f800ce00700 1 -- 172.21.15.1:0/1734709728 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80081a0ea0 con 0x7f8008101f70 2024-02-02T15:26:08.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.361+0000 7f7ffea00700 1 -- 172.21.15.1:0/1734709728 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff0018680 con 0x7f8008101f70 2024-02-02T15:26:08.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.361+0000 7f800ce00700 1 -- 172.21.15.1:0/1734709728 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80081a1320 con 0x7f8008101f70 2024-02-02T15:26:08.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.362+0000 7f800ce00700 1 -- 172.21.15.1:0/1734709728 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f800810c670 con 0x7f8008101f70 2024-02-02T15:26:08.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.364+0000 7f7ffea00700 1 -- 172.21.15.1:0/1734709728 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) v1 ==== 135120+0+0 (secure 0 0 0) 0x7f7ff00105a0 con 0x7f8008101f70 2024-02-02T15:26:08.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.365+0000 7f7ffea00700 1 --2- 172.21.15.1:0/1734709728 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f7fe80a0630 0x7f7fe80a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:08.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.365+0000 7f7ffea00700 1 -- 172.21.15.1:0/1734709728 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f7ff0014070 con 0x7f8008101f70 2024-02-02T15:26:08.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.366+0000 7f8005e00700 1 --2- 172.21.15.1:0/1734709728 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f7fe80a0630 0x7f7fe80a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:08.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.366+0000 7f8005e00700 1 --2- 172.21.15.1:0/1734709728 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f7fe80a0630 0x7f7fe80a2af0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7ff800b2e0 tx=0x7f7ff8005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:08.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.369+0000 7f7ffea00700 1 -- 172.21.15.1:0/1734709728 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f7ff0052110 con 0x7f8008101f70 2024-02-02T15:26:08.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:08 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/1675098712' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:26:08.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:08 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/3368344178' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-02T15:26:08.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.580+0000 7f800ce00700 1 -- 172.21.15.1:0/1734709728 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f80081a1960 con 0x7f8008101f70 2024-02-02T15:26:08.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.580+0000 7f7ffea00700 1 -- 172.21.15.1:0/1734709728 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7ff00956b0 con 0x7f8008101f70 2024-02-02T15:26:08.581 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-02-02T15:26:08.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.583+0000 7f800ce00700 1 -- 172.21.15.1:0/1734709728 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f7fe80a0630 msgr2=0x7f7fe80a2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:08.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.583+0000 7f800ce00700 1 --2- 172.21.15.1:0/1734709728 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f7fe80a0630 0x7f7fe80a2af0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7ff800b2e0 tx=0x7f7ff8005fb0 comp rx=0 tx=0).stop 2024-02-02T15:26:08.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.583+0000 7f800ce00700 1 -- 172.21.15.1:0/1734709728 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8008101f70 msgr2=0x7f8008199290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:08.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.583+0000 7f800ce00700 1 --2- 172.21.15.1:0/1734709728 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8008101f70 0x7f8008199290 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7ff000e9a0 tx=0x7f7ff000ecb0 comp rx=0 tx=0).stop 2024-02-02T15:26:08.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.583+0000 7f800ce00700 1 -- 172.21.15.1:0/1734709728 shutdown_connections 2024-02-02T15:26:08.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.583+0000 7f800ce00700 1 --2- 172.21.15.1:0/1734709728 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8008102930 0x7f80081a02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:08.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.583+0000 7f800ce00700 1 --2- 172.21.15.1:0/1734709728 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f7fe80a0630 0x7f7fe80a2af0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:08.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.583+0000 7f800ce00700 1 --2- 172.21.15.1:0/1734709728 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8008100e20 0x7f8008198d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:08.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.583+0000 7f800ce00700 1 --2- 172.21.15.1:0/1734709728 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8008101f70 0x7f8008199290 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:08.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.583+0000 7f800ce00700 1 -- 172.21.15.1:0/1734709728 >> 172.21.15.1:0/1734709728 conn(0x7f80080fc390 msgr2=0x7f80080fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:08.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.583+0000 7f800ce00700 1 -- 172.21.15.1:0/1734709728 shutdown_connections 2024-02-02T15:26:08.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.583+0000 7f800ce00700 1 -- 172.21.15.1:0/1734709728 wait complete. 2024-02-02T15:26:08.602 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-02T15:26:08.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:08 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/1675098712' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:26:08.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:08 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/3368344178' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-02T15:26:08.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:08 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/1675098712' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:26:08.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:08 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/3368344178' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-02T15:26:08.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.803+0000 7f31d2c00700 1 -- 172.21.15.1:0/1028336708 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f31cc101890 msgr2=0x7f31cc103c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:08.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.803+0000 7f31d2c00700 1 --2- 172.21.15.1:0/1028336708 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f31cc101890 0x7f31cc103c80 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f31bc009990 tx=0x7f31bc009ca0 comp rx=0 tx=0).stop 2024-02-02T15:26:08.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.803+0000 7f31d2c00700 1 -- 172.21.15.1:0/1028336708 shutdown_connections 2024-02-02T15:26:08.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.803+0000 7f31d2c00700 1 --2- 172.21.15.1:0/1028336708 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f31cc106af0 0x7f31cc108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:08.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.803+0000 7f31d2c00700 1 --2- 172.21.15.1:0/1028336708 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f31cc101890 0x7f31cc103c80 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:08.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.803+0000 7f31d2c00700 1 --2- 172.21.15.1:0/1028336708 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31cc1041c0 0x7f31cc1065b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:08.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.803+0000 7f31d2c00700 1 -- 172.21.15.1:0/1028336708 >> 172.21.15.1:0/1028336708 conn(0x7f31cc0fb260 msgr2=0x7f31cc0fd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:08.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.803+0000 7f31d2c00700 1 -- 172.21.15.1:0/1028336708 shutdown_connections 2024-02-02T15:26:08.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.803+0000 7f31d2c00700 1 -- 172.21.15.1:0/1028336708 wait complete. 2024-02-02T15:26:08.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.804+0000 7f31d2c00700 1 Processor -- start 2024-02-02T15:26:08.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.804+0000 7f31d2c00700 1 -- start start 2024-02-02T15:26:08.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.805+0000 7f31d2c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f31cc101890 0x7f31cc101200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:08.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.805+0000 7f31d2c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f31cc1041c0 0x7f31cc0ff840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:08.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.805+0000 7f31d2c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31cc106af0 0x7f31cc0ffd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:08.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.805+0000 7f31d2c00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31cc100520 con 0x7f31cc1041c0 2024-02-02T15:26:08.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.805+0000 7f31d2c00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31cc100690 con 0x7f31cc106af0 2024-02-02T15:26:08.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.805+0000 7f31d2c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31cc1a0b20 con 0x7f31cc101890 2024-02-02T15:26:08.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.805+0000 7f31cbe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f31cc101890 0x7f31cc101200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:08.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.805+0000 7f31cb400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f31cc1041c0 0x7f31cc0ff840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:08.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.805+0000 7f31cbe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f31cc101890 0x7f31cc101200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:57106/0 (socket says 172.21.15.1:57106) 2024-02-02T15:26:08.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.805+0000 7f31cbe00700 1 -- 172.21.15.1:0/85229052 learned_addr learned my addr 172.21.15.1:0/85229052 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:08.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.805+0000 7f31d0c00700 1 --2- 172.21.15.1:0/85229052 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31cc106af0 0x7f31cc0ffd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:08.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.806+0000 7f31cbe00700 1 -- 172.21.15.1:0/85229052 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31cc106af0 msgr2=0x7f31cc0ffd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:08.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.806+0000 7f31cbe00700 1 --2- 172.21.15.1:0/85229052 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31cc106af0 0x7f31cc0ffd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:08.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.806+0000 7f31cbe00700 1 -- 172.21.15.1:0/85229052 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f31cc1041c0 msgr2=0x7f31cc0ff840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:08.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.806+0000 7f31cbe00700 1 --2- 172.21.15.1:0/85229052 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f31cc1041c0 0x7f31cc0ff840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:08.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.806+0000 7f31cbe00700 1 -- 172.21.15.1:0/85229052 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31b4009520 con 0x7f31cc101890 2024-02-02T15:26:08.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.806+0000 7f31cb400700 1 --2- 172.21.15.1:0/85229052 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f31cc1041c0 0x7f31cc0ff840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:26:08.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.806+0000 7f31cbe00700 1 --2- 172.21.15.1:0/85229052 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f31cc101890 0x7f31cc101200 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f31bc004740 tx=0x7f31bc009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:08.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.806+0000 7f31c8c00700 1 -- 172.21.15.1:0/85229052 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31bc01d070 con 0x7f31cc101890 2024-02-02T15:26:08.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.806+0000 7f31c8c00700 1 -- 172.21.15.1:0/85229052 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f31bc004830 con 0x7f31cc101890 2024-02-02T15:26:08.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.806+0000 7f31d2c00700 1 -- 172.21.15.1:0/85229052 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31bc0095f0 con 0x7f31cc101890 2024-02-02T15:26:08.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.806+0000 7f31c8c00700 1 -- 172.21.15.1:0/85229052 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31bc0053c0 con 0x7f31cc101890 2024-02-02T15:26:08.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.806+0000 7f31d2c00700 1 -- 172.21.15.1:0/85229052 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31cc1a0fd0 con 0x7f31cc101890 2024-02-02T15:26:08.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.807+0000 7f31d2c00700 1 -- 172.21.15.1:0/85229052 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31cc0fcde0 con 0x7f31cc101890 2024-02-02T15:26:08.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.809+0000 7f31c8c00700 1 -- 172.21.15.1:0/85229052 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 17) v1 ==== 135120+0+0 (secure 0 0 0) 0x7f31bc005560 con 0x7f31cc101890 2024-02-02T15:26:08.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.811+0000 7f31c8c00700 1 --2- 172.21.15.1:0/85229052 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f31ac0a05b0 0x7f31ac0a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:08.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.811+0000 7f31c8c00700 1 -- 172.21.15.1:0/85229052 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f31bc0cd440 con 0x7f31cc101890 2024-02-02T15:26:08.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.811+0000 7f31cb400700 1 --2- 172.21.15.1:0/85229052 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f31ac0a05b0 0x7f31ac0a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:08.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.811+0000 7f31cb400700 1 --2- 172.21.15.1:0/85229052 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f31ac0a05b0 0x7f31ac0a2a70 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f31b4009fd0 tx=0x7f31b4009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:08.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:08.813+0000 7f31c8c00700 1 -- 172.21.15.1:0/85229052 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f31bc096ba0 con 0x7f31cc101890 2024-02-02T15:26:09.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.025+0000 7f31d2c00700 1 -- 172.21.15.1:0/85229052 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f31cc02ce00 con 0x7f31cc101890 2024-02-02T15:26:09.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.026+0000 7f31c8c00700 1 -- 172.21.15.1:0/85229052 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v86) v1 ==== 104+0+11 (secure 0 0 0) 0x7f31bc09a1c0 con 0x7f31cc101890 2024-02-02T15:26:09.026 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-02-02T15:26:09.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.029+0000 7f31d2c00700 1 -- 172.21.15.1:0/85229052 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f31ac0a05b0 msgr2=0x7f31ac0a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:09.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.029+0000 7f31d2c00700 1 --2- 172.21.15.1:0/85229052 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f31ac0a05b0 0x7f31ac0a2a70 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f31b4009fd0 tx=0x7f31b4009260 comp rx=0 tx=0).stop 2024-02-02T15:26:09.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.029+0000 7f31d2c00700 1 -- 172.21.15.1:0/85229052 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f31cc101890 msgr2=0x7f31cc101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:09.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.029+0000 7f31d2c00700 1 --2- 172.21.15.1:0/85229052 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f31cc101890 0x7f31cc101200 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f31bc004740 tx=0x7f31bc009580 comp rx=0 tx=0).stop 2024-02-02T15:26:09.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.029+0000 7f31d2c00700 1 -- 172.21.15.1:0/85229052 shutdown_connections 2024-02-02T15:26:09.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.029+0000 7f31d2c00700 1 --2- 172.21.15.1:0/85229052 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31cc106af0 0x7f31cc0ffd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:09.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.029+0000 7f31d2c00700 1 --2- 172.21.15.1:0/85229052 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f31ac0a05b0 0x7f31ac0a2a70 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:09.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.029+0000 7f31d2c00700 1 --2- 172.21.15.1:0/85229052 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f31cc101890 0x7f31cc101200 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:09.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.029+0000 7f31d2c00700 1 --2- 172.21.15.1:0/85229052 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f31cc1041c0 0x7f31cc0ff840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:09.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.029+0000 7f31d2c00700 1 -- 172.21.15.1:0/85229052 >> 172.21.15.1:0/85229052 conn(0x7f31cc0fb260 msgr2=0x7f31cc102500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:09.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.029+0000 7f31d2c00700 1 -- 172.21.15.1:0/85229052 shutdown_connections 2024-02-02T15:26:09.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.029+0000 7f31d2c00700 1 -- 172.21.15.1:0/85229052 wait complete. 2024-02-02T15:26:09.047 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-02T15:26:09.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.250+0000 7f4367800700 1 -- 172.21.15.1:0/2701225970 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4360101930 msgr2=0x7f4360103d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:09.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.250+0000 7f4367800700 1 --2- 172.21.15.1:0/2701225970 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4360101930 0x7f4360103d20 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4350009990 tx=0x7f4350009ca0 comp rx=0 tx=0).stop 2024-02-02T15:26:09.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.250+0000 7f4367800700 1 -- 172.21.15.1:0/2701225970 shutdown_connections 2024-02-02T15:26:09.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.250+0000 7f4367800700 1 --2- 172.21.15.1:0/2701225970 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4360106b90 0x7f4360108f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:09.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.250+0000 7f4367800700 1 --2- 172.21.15.1:0/2701225970 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4360101930 0x7f4360103d20 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:09.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.250+0000 7f4367800700 1 --2- 172.21.15.1:0/2701225970 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4360104260 0x7f4360106650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:09.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.250+0000 7f4367800700 1 -- 172.21.15.1:0/2701225970 >> 172.21.15.1:0/2701225970 conn(0x7f43600fb260 msgr2=0x7f43600fd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:09.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.250+0000 7f4367800700 1 -- 172.21.15.1:0/2701225970 shutdown_connections 2024-02-02T15:26:09.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.250+0000 7f4367800700 1 -- 172.21.15.1:0/2701225970 wait complete. 2024-02-02T15:26:09.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.251+0000 7f4367800700 1 Processor -- start 2024-02-02T15:26:09.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.251+0000 7f4367800700 1 -- start start 2024-02-02T15:26:09.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.252+0000 7f4367800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4360101930 0x7f4360198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:09.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.252+0000 7f4367800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4360104260 0x7f4360199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:09.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.252+0000 7f4367800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4360106b90 0x7f43601a0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:09.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.252+0000 7f4367800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43601a0840 con 0x7f4360101930 2024-02-02T15:26:09.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.252+0000 7f4367800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43601a09b0 con 0x7f4360104260 2024-02-02T15:26:09.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.252+0000 7f4367800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43601a0b70 con 0x7f4360106b90 2024-02-02T15:26:09.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.252+0000 7f4364e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4360101930 0x7f4360198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:09.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.252+0000 7f4364e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4360101930 0x7f4360198d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55212/0 (socket says 172.21.15.1:55212) 2024-02-02T15:26:09.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.252+0000 7f4364e00700 1 -- 172.21.15.1:0/1776802065 learned_addr learned my addr 172.21.15.1:0/1776802065 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:09.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.252+0000 7f435fe00700 1 --2- 172.21.15.1:0/1776802065 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4360104260 0x7f4360199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:09.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.252+0000 7f4365800700 1 --2- 172.21.15.1:0/1776802065 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4360106b90 0x7f43601a0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:09.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.253+0000 7f4364e00700 1 -- 172.21.15.1:0/1776802065 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4360106b90 msgr2=0x7f43601a0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:09.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.253+0000 7f4364e00700 1 --2- 172.21.15.1:0/1776802065 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4360106b90 0x7f43601a0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:09.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.253+0000 7f4364e00700 1 -- 172.21.15.1:0/1776802065 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4360104260 msgr2=0x7f4360199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:09.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.253+0000 7f4364e00700 1 --2- 172.21.15.1:0/1776802065 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4360104260 0x7f4360199240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:09.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.253+0000 7f4364e00700 1 -- 172.21.15.1:0/1776802065 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43500095f0 con 0x7f4360101930 2024-02-02T15:26:09.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.253+0000 7f4364e00700 1 --2- 172.21.15.1:0/1776802065 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4360101930 0x7f4360198d00 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f435000b680 tx=0x7f43500057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:09.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.253+0000 7f435d600700 1 -- 172.21.15.1:0/1776802065 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f435001d070 con 0x7f4360101930 2024-02-02T15:26:09.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.253+0000 7f435d600700 1 -- 172.21.15.1:0/1776802065 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4350022470 con 0x7f4360101930 2024-02-02T15:26:09.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.253+0000 7f4367800700 1 -- 172.21.15.1:0/1776802065 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43601a0df0 con 0x7f4360101930 2024-02-02T15:26:09.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.253+0000 7f435d600700 1 -- 172.21.15.1:0/1776802065 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4350017650 con 0x7f4360101930 2024-02-02T15:26:09.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.253+0000 7f4367800700 1 -- 172.21.15.1:0/1776802065 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43601a11c0 con 0x7f4360101930 2024-02-02T15:26:09.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.254+0000 7f4367800700 1 -- 172.21.15.1:0/1776802065 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f436010c640 con 0x7f4360101930 2024-02-02T15:26:09.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.256+0000 7f435d600700 1 -- 172.21.15.1:0/1776802065 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) v1 ==== 135120+0+0 (secure 0 0 0) 0x7f43500225e0 con 0x7f4360101930 2024-02-02T15:26:09.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.257+0000 7f435d600700 1 --2- 172.21.15.1:0/1776802065 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f43400a05b0 0x7f43400a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:09.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.258+0000 7f435d600700 1 -- 172.21.15.1:0/1776802065 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f43500cd390 con 0x7f4360101930 2024-02-02T15:26:09.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.258+0000 7f435fe00700 1 --2- 172.21.15.1:0/1776802065 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f43400a05b0 0x7f43400a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:09.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.258+0000 7f435fe00700 1 --2- 172.21.15.1:0/1776802065 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f43400a05b0 0x7f43400a2a70 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4348009c60 tx=0x7f4348009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:09.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.261+0000 7f435d600700 1 -- 172.21.15.1:0/1776802065 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f4350096c10 con 0x7f4360101930 2024-02-02T15:26:09.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:09 smithi001 ceph-mon[35470]: pgmap v277: 129 pgs: 129 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-02-02T15:26:09.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:09 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/1734709728' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-02T15:26:09.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:09 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/85229052' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-02T15:26:09.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.473+0000 7f4367800700 1 -- 172.21.15.1:0/1776802065 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f43601a1770 con 0x7f4360101930 2024-02-02T15:26:09.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.474+0000 7f435d600700 1 -- 172.21.15.1:0/1776802065 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v86) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4350027020 con 0x7f4360101930 2024-02-02T15:26:09.474 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 64 2024-02-02T15:26:09.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.476+0000 7f4367800700 1 -- 172.21.15.1:0/1776802065 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f43400a05b0 msgr2=0x7f43400a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:09.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.476+0000 7f4367800700 1 --2- 172.21.15.1:0/1776802065 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f43400a05b0 0x7f43400a2a70 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4348009c60 tx=0x7f4348009260 comp rx=0 tx=0).stop 2024-02-02T15:26:09.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.476+0000 7f4367800700 1 -- 172.21.15.1:0/1776802065 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4360101930 msgr2=0x7f4360198d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:09.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.476+0000 7f4367800700 1 --2- 172.21.15.1:0/1776802065 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4360101930 0x7f4360198d00 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f435000b680 tx=0x7f43500057b0 comp rx=0 tx=0).stop 2024-02-02T15:26:09.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.477+0000 7f4367800700 1 -- 172.21.15.1:0/1776802065 shutdown_connections 2024-02-02T15:26:09.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.477+0000 7f4367800700 1 --2- 172.21.15.1:0/1776802065 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f43400a05b0 0x7f43400a2a70 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:09.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.477+0000 7f4367800700 1 --2- 172.21.15.1:0/1776802065 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4360106b90 0x7f43601a0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:09.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.477+0000 7f4367800700 1 --2- 172.21.15.1:0/1776802065 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4360101930 0x7f4360198d00 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:09.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.477+0000 7f4367800700 1 --2- 172.21.15.1:0/1776802065 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4360104260 0x7f4360199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:09.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.477+0000 7f4367800700 1 -- 172.21.15.1:0/1776802065 >> 172.21.15.1:0/1776802065 conn(0x7f43600fb260 msgr2=0x7f4360102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:09.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.477+0000 7f4367800700 1 -- 172.21.15.1:0/1776802065 shutdown_connections 2024-02-02T15:26:09.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.477+0000 7f4367800700 1 -- 172.21.15.1:0/1776802065 wait complete. 2024-02-02T15:26:09.496 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-02-02T15:26:09.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:09 smithi071 ceph-mon[44707]: pgmap v277: 129 pgs: 129 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-02-02T15:26:09.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:09 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/1734709728' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-02T15:26:09.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:09 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/85229052' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-02T15:26:09.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:09 smithi142 ceph-mon[45728]: pgmap v277: 129 pgs: 129 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-02-02T15:26:09.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:09 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/1734709728' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-02T15:26:09.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:09 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/85229052' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-02T15:26:09.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.696+0000 7f70a7c00700 1 -- 172.21.15.1:0/1685383586 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f70a0100e20 msgr2=0x7f70a0101240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:09.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.696+0000 7f70a7c00700 1 --2- 172.21.15.1:0/1685383586 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f70a0100e20 0x7f70a0101240 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7090009990 tx=0x7f7090009ca0 comp rx=0 tx=0).stop 2024-02-02T15:26:09.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.696+0000 7f70a7c00700 1 -- 172.21.15.1:0/1685383586 shutdown_connections 2024-02-02T15:26:09.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.696+0000 7f70a7c00700 1 --2- 172.21.15.1:0/1685383586 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70a0102930 0x7f70a0109250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:09.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.696+0000 7f70a7c00700 1 --2- 172.21.15.1:0/1685383586 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f70a0100e20 0x7f70a0101240 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:09.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.696+0000 7f70a7c00700 1 --2- 172.21.15.1:0/1685383586 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f70a0101f70 0x7f70a01023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:09.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.696+0000 7f70a7c00700 1 -- 172.21.15.1:0/1685383586 >> 172.21.15.1:0/1685383586 conn(0x7f70a00fc390 msgr2=0x7f70a00fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:09.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.696+0000 7f70a7c00700 1 -- 172.21.15.1:0/1685383586 shutdown_connections 2024-02-02T15:26:09.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.696+0000 7f70a7c00700 1 -- 172.21.15.1:0/1685383586 wait complete. 2024-02-02T15:26:09.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.698+0000 7f70a7c00700 1 Processor -- start 2024-02-02T15:26:09.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.698+0000 7f70a7c00700 1 -- start start 2024-02-02T15:26:09.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.698+0000 7f70a7c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70a0100e20 0x7f70a0074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:09.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.698+0000 7f70a7c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f70a0101f70 0x7f70a0073130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:09.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.699+0000 7f70a7c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f70a0102930 0x7f70a0073670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:09.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.699+0000 7f70a7c00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a0073e10 con 0x7f70a0101f70 2024-02-02T15:26:09.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.699+0000 7f70a7c00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a0073f80 con 0x7f70a0100e20 2024-02-02T15:26:09.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.699+0000 7f70a7c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a01a0bb0 con 0x7f70a0102930 2024-02-02T15:26:09.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.699+0000 7f709fe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f70a0101f70 0x7f70a0073130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:09.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.699+0000 7f709fe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f70a0101f70 0x7f70a0073130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55220/0 (socket says 172.21.15.1:55220) 2024-02-02T15:26:09.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.699+0000 7f709fe00700 1 -- 172.21.15.1:0/4269452101 learned_addr learned my addr 172.21.15.1:0/4269452101 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:09.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.699+0000 7f70a5200700 1 --2- 172.21.15.1:0/4269452101 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70a0100e20 0x7f70a0074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:09.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.699+0000 7f70a5c00700 1 --2- 172.21.15.1:0/4269452101 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f70a0102930 0x7f70a0073670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:09.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.699+0000 7f709fe00700 1 -- 172.21.15.1:0/4269452101 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f70a0102930 msgr2=0x7f70a0073670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:09.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.699+0000 7f709fe00700 1 --2- 172.21.15.1:0/4269452101 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f70a0102930 0x7f70a0073670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:09.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.699+0000 7f709fe00700 1 -- 172.21.15.1:0/4269452101 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70a0100e20 msgr2=0x7f70a0074af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:09.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.699+0000 7f709fe00700 1 --2- 172.21.15.1:0/4269452101 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70a0100e20 0x7f70a0074af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:09.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.699+0000 7f709fe00700 1 -- 172.21.15.1:0/4269452101 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70900095f0 con 0x7f70a0101f70 2024-02-02T15:26:09.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.699+0000 7f709fe00700 1 --2- 172.21.15.1:0/4269452101 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f70a0101f70 0x7f70a0073130 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f708800e9a0 tx=0x7f708800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:09.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.700+0000 7f709d600700 1 -- 172.21.15.1:0/4269452101 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7088009790 con 0x7f70a0101f70 2024-02-02T15:26:09.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.700+0000 7f709d600700 1 -- 172.21.15.1:0/4269452101 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7088010430 con 0x7f70a0101f70 2024-02-02T15:26:09.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.700+0000 7f70a7c00700 1 -- 172.21.15.1:0/4269452101 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70a01a0e90 con 0x7f70a0101f70 2024-02-02T15:26:09.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.700+0000 7f709d600700 1 -- 172.21.15.1:0/4269452101 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7088018680 con 0x7f70a0101f70 2024-02-02T15:26:09.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.700+0000 7f70a7c00700 1 -- 172.21.15.1:0/4269452101 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70a01a1360 con 0x7f70a0101f70 2024-02-02T15:26:09.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.701+0000 7f70a7c00700 1 -- 172.21.15.1:0/4269452101 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70a0108450 con 0x7f70a0101f70 2024-02-02T15:26:09.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.703+0000 7f709d600700 1 -- 172.21.15.1:0/4269452101 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) v1 ==== 135120+0+0 (secure 0 0 0) 0x7f70880105a0 con 0x7f70a0101f70 2024-02-02T15:26:09.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.704+0000 7f709d600700 1 --2- 172.21.15.1:0/4269452101 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f70800a0630 0x7f70800a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:09.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.704+0000 7f709d600700 1 -- 172.21.15.1:0/4269452101 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f7088014070 con 0x7f70a0101f70 2024-02-02T15:26:09.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.704+0000 7f70a5200700 1 --2- 172.21.15.1:0/4269452101 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f70800a0630 0x7f70800a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:09.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.705+0000 7f70a5200700 1 --2- 172.21.15.1:0/4269452101 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f70800a0630 0x7f70800a2af0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f709000b2e0 tx=0x7f7090005960 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:09.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.708+0000 7f709d600700 1 -- 172.21.15.1:0/4269452101 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f7088052110 con 0x7f70a0101f70 2024-02-02T15:26:09.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.921+0000 7f70a7c00700 1 -- 172.21.15.1:0/4269452101 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f70a002cec0 con 0x7f70a0101f70 2024-02-02T15:26:09.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.922+0000 7f709d600700 1 -- 172.21.15.1:0/4269452101 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v86) v1 ==== 103+0+11 (secure 0 0 0) 0x7f7088095660 con 0x7f70a0101f70 2024-02-02T15:26:09.922 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-02-02T15:26:09.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.924+0000 7f70a7c00700 1 -- 172.21.15.1:0/4269452101 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f70800a0630 msgr2=0x7f70800a2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:09.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.924+0000 7f70a7c00700 1 --2- 172.21.15.1:0/4269452101 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f70800a0630 0x7f70800a2af0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f709000b2e0 tx=0x7f7090005960 comp rx=0 tx=0).stop 2024-02-02T15:26:09.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.924+0000 7f70a7c00700 1 -- 172.21.15.1:0/4269452101 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f70a0101f70 msgr2=0x7f70a0073130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:09.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.924+0000 7f70a7c00700 1 --2- 172.21.15.1:0/4269452101 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f70a0101f70 0x7f70a0073130 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f708800e9a0 tx=0x7f708800ecb0 comp rx=0 tx=0).stop 2024-02-02T15:26:09.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.925+0000 7f70a7c00700 1 -- 172.21.15.1:0/4269452101 shutdown_connections 2024-02-02T15:26:09.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.925+0000 7f70a7c00700 1 --2- 172.21.15.1:0/4269452101 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f70800a0630 0x7f70800a2af0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:09.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.925+0000 7f70a7c00700 1 --2- 172.21.15.1:0/4269452101 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f70a0102930 0x7f70a0073670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:09.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.925+0000 7f70a7c00700 1 --2- 172.21.15.1:0/4269452101 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70a0100e20 0x7f70a0074af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:09.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.925+0000 7f70a7c00700 1 --2- 172.21.15.1:0/4269452101 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f70a0101f70 0x7f70a0073130 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:09.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.925+0000 7f70a7c00700 1 -- 172.21.15.1:0/4269452101 >> 172.21.15.1:0/4269452101 conn(0x7f70a00fc390 msgr2=0x7f70a0105590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:09.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.925+0000 7f70a7c00700 1 -- 172.21.15.1:0/4269452101 shutdown_connections 2024-02-02T15:26:09.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:09.925+0000 7f70a7c00700 1 -- 172.21.15.1:0/4269452101 wait complete. 2024-02-02T15:26:09.944 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T15:26:10.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.141+0000 7f83ce400700 1 -- 172.21.15.1:0/853148738 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83c8108c40 msgr2=0x7f83c810b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:10.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.141+0000 7f83ce400700 1 --2- 172.21.15.1:0/853148738 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83c8108c40 0x7f83c810b0d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f83c000b0a0 tx=0x7f83c000b3b0 comp rx=0 tx=0).stop 2024-02-02T15:26:10.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.141+0000 7f83ce400700 1 -- 172.21.15.1:0/853148738 shutdown_connections 2024-02-02T15:26:10.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.141+0000 7f83ce400700 1 --2- 172.21.15.1:0/853148738 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83c8108c40 0x7f83c810b0d0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:10.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.141+0000 7f83ce400700 1 --2- 172.21.15.1:0/853148738 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83c8074d00 0x7f83c8073150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:10.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.141+0000 7f83ce400700 1 --2- 172.21.15.1:0/853148738 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83c8073720 0x7f83c8073ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:10.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.141+0000 7f83ce400700 1 -- 172.21.15.1:0/853148738 >> 172.21.15.1:0/853148738 conn(0x7f83c80fc2c0 msgr2=0x7f83c80fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:10.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.141+0000 7f83ce400700 1 -- 172.21.15.1:0/853148738 shutdown_connections 2024-02-02T15:26:10.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.141+0000 7f83ce400700 1 -- 172.21.15.1:0/853148738 wait complete. 2024-02-02T15:26:10.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.142+0000 7f83ce400700 1 Processor -- start 2024-02-02T15:26:10.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.142+0000 7f83ce400700 1 -- start start 2024-02-02T15:26:10.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.143+0000 7f83ce400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83c8073720 0x7f83c8198d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:10.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.143+0000 7f83ce400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83c8074d00 0x7f83c8199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:10.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.143+0000 7f83ce400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83c8108c40 0x7f83c81a0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:10.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.143+0000 7f83ce400700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c81a0850 con 0x7f83c8073720 2024-02-02T15:26:10.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.143+0000 7f83ce400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c81a09c0 con 0x7f83c8074d00 2024-02-02T15:26:10.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.143+0000 7f83ce400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c81a0b80 con 0x7f83c8108c40 2024-02-02T15:26:10.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.143+0000 7f83c7400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83c8073720 0x7f83c8198d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:10.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.143+0000 7f83c7400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83c8073720 0x7f83c8198d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55232/0 (socket says 172.21.15.1:55232) 2024-02-02T15:26:10.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.143+0000 7f83c7400700 1 -- 172.21.15.1:0/2534898477 learned_addr learned my addr 172.21.15.1:0/2534898477 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:10.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.143+0000 7f83c6a00700 1 --2- 172.21.15.1:0/2534898477 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83c8074d00 0x7f83c8199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:10.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.143+0000 7f83c7e00700 1 --2- 172.21.15.1:0/2534898477 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83c8108c40 0x7f83c81a0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:10.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.143+0000 7f83c7400700 1 -- 172.21.15.1:0/2534898477 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83c8108c40 msgr2=0x7f83c81a0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:10.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.143+0000 7f83c7400700 1 --2- 172.21.15.1:0/2534898477 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83c8108c40 0x7f83c81a0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:10.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.143+0000 7f83c7400700 1 -- 172.21.15.1:0/2534898477 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83c8074d00 msgr2=0x7f83c8199250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:10.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.143+0000 7f83c7400700 1 --2- 172.21.15.1:0/2534898477 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83c8074d00 0x7f83c8199250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:10.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.143+0000 7f83c7400700 1 -- 172.21.15.1:0/2534898477 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83c0009d00 con 0x7f83c8073720 2024-02-02T15:26:10.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.144+0000 7f83c7400700 1 --2- 172.21.15.1:0/2534898477 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83c8073720 0x7f83c8198d10 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f83b800d980 tx=0x7f83b800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:10.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.144+0000 7f83bfe00700 1 -- 172.21.15.1:0/2534898477 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b8004220 con 0x7f83c8073720 2024-02-02T15:26:10.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.144+0000 7f83bfe00700 1 -- 172.21.15.1:0/2534898477 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f83b8007640 con 0x7f83c8073720 2024-02-02T15:26:10.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.144+0000 7f83bfe00700 1 -- 172.21.15.1:0/2534898477 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b8005690 con 0x7f83c8073720 2024-02-02T15:26:10.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.144+0000 7f83ce400700 1 -- 172.21.15.1:0/2534898477 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83c81a0e60 con 0x7f83c8073720 2024-02-02T15:26:10.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.144+0000 7f83ce400700 1 -- 172.21.15.1:0/2534898477 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83c81a1230 con 0x7f83c8073720 2024-02-02T15:26:10.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.145+0000 7f83ce400700 1 -- 172.21.15.1:0/2534898477 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83c8107340 con 0x7f83c8073720 2024-02-02T15:26:10.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.147+0000 7f83bfe00700 1 -- 172.21.15.1:0/2534898477 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) v1 ==== 135120+0+0 (secure 0 0 0) 0x7f83b8004750 con 0x7f83c8073720 2024-02-02T15:26:10.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.148+0000 7f83bfe00700 1 --2- 172.21.15.1:0/2534898477 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f83a80a0630 0x7f83a80a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:10.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.148+0000 7f83bfe00700 1 -- 172.21.15.1:0/2534898477 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f83b80cbba0 con 0x7f83c8073720 2024-02-02T15:26:10.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.148+0000 7f83c6a00700 1 --2- 172.21.15.1:0/2534898477 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f83a80a0630 0x7f83a80a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:10.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.149+0000 7f83c6a00700 1 --2- 172.21.15.1:0/2534898477 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f83a80a0630 0x7f83a80a2af0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f83b0005950 tx=0x7f83b000b410 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:10.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.152+0000 7f83bfe00700 1 -- 172.21.15.1:0/2534898477 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f83b8095270 con 0x7f83c8073720 2024-02-02T15:26:10.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.415+0000 7f83ce400700 1 -- 172.21.15.1:0/2534898477 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f83c802cf20 con 0x7f83c8073720 2024-02-02T15:26:10.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.415+0000 7f83bfe00700 1 -- 172.21.15.1:0/2534898477 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10866 (secure 0 0 0) 0x7f83b8016020 con 0x7f83c8073720 2024-02-02T15:26:10.416 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:26:10.416 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2922240448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2922240448},{"type":"v1","addr":"172.21.15.1:6835","nonce":2922240448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2729497431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2729497431},{"type":"v1","addr":"172.21.15.1:6837","nonce":2729497431}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-02T15:24:47.761435+0000","modified":"2024-02-02T15:25:20.251601+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24529,"mds_2":24518,"mds_3":24514,"mds_4":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14517":{"gid":14517,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/1218303683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":1218303683},{"type":"v1","addr":"172.21.15.1:6839","nonce":1218303683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14529":{"gid":14529,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/2059197496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2059197496},{"type":"v1","addr":"172.21.15.1:6841","nonce":2059197496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/7648986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":7648986},{"type":"v1","addr":"172.21.15.142:6833","nonce":7648986}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/3482746094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3482746094},{"type":"v1","addr":"172.21.15.71:6833","nonce":3482746094}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/388207814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":388207814},{"type":"v1","addr":"172.21.15.71:6835","nonce":388207814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6835/1052606547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1052606547},{"type":"v1","addr":"172.21.15.142:6835","nonce":1052606547}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.142:6837/2870289019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2870289019},{"type":"v1","addr":"172.21.15.142:6837","nonce":2870289019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.71:6837/628016199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":628016199},{"type":"v1","addr":"172.21.15.71:6837","nonce":628016199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.142:6839/321675421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":321675421},{"type":"v1","addr":"172.21.15.142:6839","nonce":321675421}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/1268086493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1268086493},{"type":"v1","addr":"172.21.15.71:6839","nonce":1268086493}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-02T15:26:10.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.418+0000 7f83ce400700 1 -- 172.21.15.1:0/2534898477 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f83a80a0630 msgr2=0x7f83a80a2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:10.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.418+0000 7f83ce400700 1 --2- 172.21.15.1:0/2534898477 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f83a80a0630 0x7f83a80a2af0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f83b0005950 tx=0x7f83b000b410 comp rx=0 tx=0).stop 2024-02-02T15:26:10.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.418+0000 7f83ce400700 1 -- 172.21.15.1:0/2534898477 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83c8073720 msgr2=0x7f83c8198d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:10.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.418+0000 7f83ce400700 1 --2- 172.21.15.1:0/2534898477 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83c8073720 0x7f83c8198d10 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f83b800d980 tx=0x7f83b800dc90 comp rx=0 tx=0).stop 2024-02-02T15:26:10.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.418+0000 7f83ce400700 1 -- 172.21.15.1:0/2534898477 shutdown_connections 2024-02-02T15:26:10.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.418+0000 7f83ce400700 1 --2- 172.21.15.1:0/2534898477 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f83a80a0630 0x7f83a80a2af0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:10.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.418+0000 7f83ce400700 1 --2- 172.21.15.1:0/2534898477 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83c8108c40 0x7f83c81a0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:10.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.418+0000 7f83ce400700 1 --2- 172.21.15.1:0/2534898477 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83c8073720 0x7f83c8198d10 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:10.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.418+0000 7f83ce400700 1 --2- 172.21.15.1:0/2534898477 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83c8074d00 0x7f83c8199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:10.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.418+0000 7f83ce400700 1 -- 172.21.15.1:0/2534898477 >> 172.21.15.1:0/2534898477 conn(0x7f83c80fc2c0 msgr2=0x7f83c80fe6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:10.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.418+0000 7f83ce400700 1 -- 172.21.15.1:0/2534898477 shutdown_connections 2024-02-02T15:26:10.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.418+0000 7f83ce400700 1 -- 172.21.15.1:0/2534898477 wait complete. 2024-02-02T15:26:10.419 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 27 2024-02-02T15:26:10.437 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T15:26:10.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:10 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/1776802065' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-02T15:26:10.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:10 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/4269452101' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-02T15:26:10.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.634+0000 7fc7c4a00700 1 -- 172.21.15.1:0/989268785 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc7bc101f30 msgr2=0x7fc7bc102390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:10.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.634+0000 7fc7c4a00700 1 --2- 172.21.15.1:0/989268785 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc7bc101f30 0x7fc7bc102390 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc7a40099c0 tx=0x7fc7a4009cd0 comp rx=0 tx=0).stop 2024-02-02T15:26:10.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.634+0000 7fc7c4a00700 1 -- 172.21.15.1:0/989268785 shutdown_connections 2024-02-02T15:26:10.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.634+0000 7fc7c4a00700 1 --2- 172.21.15.1:0/989268785 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7bc1028d0 0x7fc7bc109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:10.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.634+0000 7fc7c4a00700 1 --2- 172.21.15.1:0/989268785 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7bc100d40 0x7fc7bc101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:10.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.634+0000 7fc7c4a00700 1 --2- 172.21.15.1:0/989268785 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc7bc101f30 0x7fc7bc102390 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:10.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.634+0000 7fc7c4a00700 1 -- 172.21.15.1:0/989268785 >> 172.21.15.1:0/989268785 conn(0x7fc7bc0fc290 msgr2=0x7fc7bc0fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:10.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.634+0000 7fc7c4a00700 1 -- 172.21.15.1:0/989268785 shutdown_connections 2024-02-02T15:26:10.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.634+0000 7fc7c4a00700 1 -- 172.21.15.1:0/989268785 wait complete. 2024-02-02T15:26:10.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.635+0000 7fc7c4a00700 1 Processor -- start 2024-02-02T15:26:10.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.635+0000 7fc7c4a00700 1 -- start start 2024-02-02T15:26:10.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.636+0000 7fc7c4a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7bc100d40 0x7fc7bc074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:10.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.636+0000 7fc7c4a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7bc101f30 0x7fc7bc073130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:10.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.636+0000 7fc7c4a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc7bc1028d0 0x7fc7bc073670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:10.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.636+0000 7fc7c4a00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7bc073e10 con 0x7fc7bc1028d0 2024-02-02T15:26:10.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.636+0000 7fc7c4a00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7bc073f80 con 0x7fc7bc101f30 2024-02-02T15:26:10.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.636+0000 7fc7c4a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7bc1a0a70 con 0x7fc7bc100d40 2024-02-02T15:26:10.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.636+0000 7fc7c2000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7bc100d40 0x7fc7bc074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:10.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.636+0000 7fc7c2a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc7bc1028d0 0x7fc7bc073670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:10.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.636+0000 7fc7c1600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7bc101f30 0x7fc7bc073130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:10.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.636+0000 7fc7c2000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7bc100d40 0x7fc7bc074af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:57172/0 (socket says 172.21.15.1:57172) 2024-02-02T15:26:10.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.636+0000 7fc7c2000700 1 -- 172.21.15.1:0/3780855617 learned_addr learned my addr 172.21.15.1:0/3780855617 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:10.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.636+0000 7fc7c2000700 1 -- 172.21.15.1:0/3780855617 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7bc101f30 msgr2=0x7fc7bc073130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:10.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.636+0000 7fc7c2000700 1 --2- 172.21.15.1:0/3780855617 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7bc101f30 0x7fc7bc073130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:10.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.636+0000 7fc7c2000700 1 -- 172.21.15.1:0/3780855617 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc7bc1028d0 msgr2=0x7fc7bc073670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:10.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.636+0000 7fc7c2000700 1 --2- 172.21.15.1:0/3780855617 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc7bc1028d0 0x7fc7bc073670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:10.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.636+0000 7fc7c2000700 1 -- 172.21.15.1:0/3780855617 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7ac009520 con 0x7fc7bc100d40 2024-02-02T15:26:10.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.637+0000 7fc7c2000700 1 --2- 172.21.15.1:0/3780855617 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7bc100d40 0x7fc7bc074af0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac00ea50 tx=0x7fc7ac00ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:10.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.637+0000 7fc7baa00700 1 -- 172.21.15.1:0/3780855617 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7ac00cbd0 con 0x7fc7bc100d40 2024-02-02T15:26:10.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.637+0000 7fc7baa00700 1 -- 172.21.15.1:0/3780855617 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc7ac00cd30 con 0x7fc7bc100d40 2024-02-02T15:26:10.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.637+0000 7fc7baa00700 1 -- 172.21.15.1:0/3780855617 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7ac018680 con 0x7fc7bc100d40 2024-02-02T15:26:10.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.637+0000 7fc7c4a00700 1 -- 172.21.15.1:0/3780855617 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7a40095f0 con 0x7fc7bc100d40 2024-02-02T15:26:10.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.637+0000 7fc7c4a00700 1 -- 172.21.15.1:0/3780855617 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7bc1a1080 con 0x7fc7bc100d40 2024-02-02T15:26:10.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.638+0000 7fc7c4a00700 1 -- 172.21.15.1:0/3780855617 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7bc107cc0 con 0x7fc7bc100d40 2024-02-02T15:26:10.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.640+0000 7fc7baa00700 1 -- 172.21.15.1:0/3780855617 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 17) v1 ==== 135120+0+0 (secure 0 0 0) 0x7fc7ac010430 con 0x7fc7bc100d40 2024-02-02T15:26:10.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.642+0000 7fc7baa00700 1 --2- 172.21.15.1:0/3780855617 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fc79c0a05b0 0x7fc79c0a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:10.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.642+0000 7fc7baa00700 1 -- 172.21.15.1:0/3780855617 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc7ac014070 con 0x7fc7bc100d40 2024-02-02T15:26:10.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.642+0000 7fc7c1600700 1 --2- 172.21.15.1:0/3780855617 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fc79c0a05b0 0x7fc79c0a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:10.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.642+0000 7fc7c1600700 1 --2- 172.21.15.1:0/3780855617 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fc79c0a05b0 0x7fc79c0a2a70 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc7a400b580 tx=0x7fc7a4005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:10.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.644+0000 7fc7baa00700 1 -- 172.21.15.1:0/3780855617 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7fc7ac095fd0 con 0x7fc7bc100d40 2024-02-02T15:26:10.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:10 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/1776802065' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-02T15:26:10.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:10 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/4269452101' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-02T15:26:10.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:10 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/1776802065' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-02T15:26:10.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:10 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/4269452101' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-02T15:26:10.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.861+0000 7fc7c4a00700 1 -- 172.21.15.1:0/3780855617 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc7bc02cf20 con 0x7fc7bc100d40 2024-02-02T15:26:10.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.863+0000 7fc7baa00700 1 -- 172.21.15.1:0/3780855617 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+24312 (secure 0 0 0) 0x7fc7ac0995f0 con 0x7fc7bc100d40 2024-02-02T15:26:10.863 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:26:10.864 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":86,"fsid":"371a409c-c1de-11ee-95b6-87774f69a715","created":"2024-02-02T15:18:26.746506+0000","modified":"2024-02-02T15:25:27.269296+0000","last_up_change":"2024-02-02T15:24:08.800951+0000","last_in_change":"2024-02-02T15:23:52.784280+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T15:21:22.471754+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-02T15:24:46.195779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-02T15:24:47.226403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-02T15:24:50.300462+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"73e8a07f-3560-4c19-85ed-030d4d8f0f23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6803","nonce":2591906423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6805","nonce":2591906423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6809","nonce":2591906423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6807","nonce":2591906423}]},"public_addr":"172.21.15.1:6803/2591906423","cluster_addr":"172.21.15.1:6805/2591906423","heartbeat_back_addr":"172.21.15.1:6809/2591906423","heartbeat_front_addr":"172.21.15.1:6807/2591906423","state":["exists","up"]},{"osd":1,"uuid":"e53cffec-0ff9-4ed9-8dcf-ff47d04db2bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6801","nonce":747581719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6803","nonce":747581719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6807","nonce":747581719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6805","nonce":747581719}]},"public_addr":"172.21.15.71:6801/747581719","cluster_addr":"172.21.15.71:6803/747581719","heartbeat_back_addr":"172.21.15.71:6807/747581719","heartbeat_front_addr":"172.21.15.71:6805/747581719","state":["exists","up"]},{"osd":2,"uuid":"856c2347-294c-482b-8595-ad77f4792db4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6801","nonce":377511414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6803","nonce":377511414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6807","nonce":377511414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6805","nonce":377511414}]},"public_addr":"172.21.15.142:6801/377511414","cluster_addr":"172.21.15.142:6803/377511414","heartbeat_back_addr":"172.21.15.142:6807/377511414","heartbeat_front_addr":"172.21.15.142:6805/377511414","state":["exists","up"]},{"osd":3,"uuid":"4b946ad2-2da9-4e0b-9d61-d039d8669a42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6811","nonce":1086124477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6813","nonce":1086124477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6817","nonce":1086124477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6815","nonce":1086124477}]},"public_addr":"172.21.15.1:6811/1086124477","cluster_addr":"172.21.15.1:6813/1086124477","heartbeat_back_addr":"172.21.15.1:6817/1086124477","heartbeat_front_addr":"172.21.15.1:6815/1086124477","state":["exists","up"]},{"osd":4,"uuid":"43e36c43-47a0-4634-a45b-1da14a0e19d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6809","nonce":3095380305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6811","nonce":3095380305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6815","nonce":3095380305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6813","nonce":3095380305}]},"public_addr":"172.21.15.71:6809/3095380305","cluster_addr":"172.21.15.71:6811/3095380305","heartbeat_back_addr":"172.21.15.71:6815/3095380305","heartbeat_front_addr":"172.21.15.71:6813/3095380305","state":["exists","up"]},{"osd":5,"uuid":"eb48160e-ea7c-42f7-9477-be110bc8a34e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6809","nonce":4195149262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6811","nonce":4195149262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6815","nonce":4195149262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6813","nonce":4195149262}]},"public_addr":"172.21.15.142:6809/4195149262","cluster_addr":"172.21.15.142:6811/4195149262","heartbeat_back_addr":"172.21.15.142:6815/4195149262","heartbeat_front_addr":"172.21.15.142:6813/4195149262","state":["exists","up"]},{"osd":6,"uuid":"0ec4501a-5853-4ca6-b325-231f5b72335e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6819","nonce":1560153428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6821","nonce":1560153428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6825","nonce":1560153428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6823","nonce":1560153428}]},"public_addr":"172.21.15.1:6819/1560153428","cluster_addr":"172.21.15.1:6821/1560153428","heartbeat_back_addr":"172.21.15.1:6825/1560153428","heartbeat_front_addr":"172.21.15.1:6823/1560153428","state":["exists","up"]},{"osd":7,"uuid":"029debe8-9cdf-44f6-9aea-cabd1d185f93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6817","nonce":4080955538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6819","nonce":4080955538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6823","nonce":4080955538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6821","nonce":4080955538}]},"public_addr":"172.21.15.71:6817/4080955538","cluster_addr":"172.21.15.71:6819/4080955538","heartbeat_back_addr":"172.21.15.71:6823/4080955538","heartbeat_front_addr":"172.21.15.71:6821/4080955538","state":["exists","up"]},{"osd":8,"uuid":"7c90896c-9955-4c1b-aaca-7531dd5457f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6817","nonce":3942038209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6819","nonce":3942038209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6823","nonce":3942038209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6821","nonce":3942038209}]},"public_addr":"172.21.15.142:6817/3942038209","cluster_addr":"172.21.15.142:6819/3942038209","heartbeat_back_addr":"172.21.15.142:6823/3942038209","heartbeat_front_addr":"172.21.15.142:6821/3942038209","state":["exists","up"]},{"osd":9,"uuid":"7e165e68-31a8-4598-a577-de3e90ae5c5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6827","nonce":2716544850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6829","nonce":2716544850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6833","nonce":2716544850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6831","nonce":2716544850}]},"public_addr":"172.21.15.1:6827/2716544850","cluster_addr":"172.21.15.1:6829/2716544850","heartbeat_back_addr":"172.21.15.1:6833/2716544850","heartbeat_front_addr":"172.21.15.1:6831/2716544850","state":["exists","up"]},{"osd":10,"uuid":"48cd5fdb-0887-444a-aab8-a1de24282bb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6825","nonce":84691841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6827","nonce":84691841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6831","nonce":84691841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6829","nonce":84691841}]},"public_addr":"172.21.15.71:6825/84691841","cluster_addr":"172.21.15.71:6827/84691841","heartbeat_back_addr":"172.21.15.71:6831/84691841","heartbeat_front_addr":"172.21.15.71:6829/84691841","state":["exists","up"]},{"osd":11,"uuid":"f011dd27-11b0-4166-adfe-76756533163a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6825","nonce":2290179934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6827","nonce":2290179934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6831","nonce":2290179934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6829","nonce":2290179934}]},"public_addr":"172.21.15.142:6825/2290179934","cluster_addr":"172.21.15.142:6827/2290179934","heartbeat_back_addr":"172.21.15.142:6831/2290179934","heartbeat_front_addr":"172.21.15.142:6829/2290179934","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:20:44.060692+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:01.903817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:19.368258+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:40.028037+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:57.513361+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:22:14.805944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:22:35.188439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:22:53.014412+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:23:10.507420+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:23:31.849765+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:23:50.070612+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:24:07.388625+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":5,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/4092532060":"2024-02-03T15:19:25.836171+0000","172.21.15.1:6801/395498296":"2024-02-03T15:19:25.836171+0000","172.21.15.1:6800/395498296":"2024-02-03T15:19:25.836171+0000","172.21.15.1:0/1575373489":"2024-02-03T15:19:25.836171+0000","172.21.15.1:0/865983374":"2024-02-03T15:19:01.431596+0000","172.21.15.1:0/3165960227":"2024-02-03T15:19:01.431596+0000","172.21.15.1:0/2350830293":"2024-02-03T15:19:01.431596+0000","172.21.15.1:0/2959690250":"2024-02-03T15:19:25.836171+0000","172.21.15.1:6800/1162530732":"2024-02-03T15:19:01.431596+0000","172.21.15.1:6801/1162530732":"2024-02-03T15:19:01.431596+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T15:26:10.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.866+0000 7fc7c4a00700 1 -- 172.21.15.1:0/3780855617 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fc79c0a05b0 msgr2=0x7fc79c0a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:10.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.866+0000 7fc7c4a00700 1 --2- 172.21.15.1:0/3780855617 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fc79c0a05b0 0x7fc79c0a2a70 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc7a400b580 tx=0x7fc7a4005fb0 comp rx=0 tx=0).stop 2024-02-02T15:26:10.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.866+0000 7fc7c4a00700 1 -- 172.21.15.1:0/3780855617 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7bc100d40 msgr2=0x7fc7bc074af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:10.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.866+0000 7fc7c4a00700 1 --2- 172.21.15.1:0/3780855617 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7bc100d40 0x7fc7bc074af0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac00ea50 tx=0x7fc7ac00ed60 comp rx=0 tx=0).stop 2024-02-02T15:26:10.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.866+0000 7fc7c4a00700 1 -- 172.21.15.1:0/3780855617 shutdown_connections 2024-02-02T15:26:10.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.866+0000 7fc7c4a00700 1 --2- 172.21.15.1:0/3780855617 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc7bc1028d0 0x7fc7bc073670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:10.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.866+0000 7fc7c4a00700 1 --2- 172.21.15.1:0/3780855617 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fc79c0a05b0 0x7fc79c0a2a70 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:10.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.866+0000 7fc7c4a00700 1 --2- 172.21.15.1:0/3780855617 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc7bc100d40 0x7fc7bc074af0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:10.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.866+0000 7fc7c4a00700 1 --2- 172.21.15.1:0/3780855617 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc7bc101f30 0x7fc7bc073130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:10.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.866+0000 7fc7c4a00700 1 -- 172.21.15.1:0/3780855617 >> 172.21.15.1:0/3780855617 conn(0x7fc7bc0fc290 msgr2=0x7fc7bc0fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:10.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.866+0000 7fc7c4a00700 1 -- 172.21.15.1:0/3780855617 shutdown_connections 2024-02-02T15:26:10.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:10.866+0000 7fc7c4a00700 1 -- 172.21.15.1:0/3780855617 wait complete. 2024-02-02T15:26:10.885 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T15:26:11.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.081+0000 7f0088400700 1 -- 172.21.15.1:0/1104650658 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0080101f30 msgr2=0x7f0080102390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:11.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.081+0000 7f0088400700 1 --2- 172.21.15.1:0/1104650658 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0080101f30 0x7f0080102390 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f00680099c0 tx=0x7f0068009cd0 comp rx=0 tx=0).stop 2024-02-02T15:26:11.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.081+0000 7f0088400700 1 -- 172.21.15.1:0/1104650658 shutdown_connections 2024-02-02T15:26:11.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.081+0000 7f0088400700 1 --2- 172.21.15.1:0/1104650658 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f00801028d0 0x7f0080109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:11.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.081+0000 7f0088400700 1 --2- 172.21.15.1:0/1104650658 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0080100d40 0x7f0080101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:11.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.081+0000 7f0088400700 1 --2- 172.21.15.1:0/1104650658 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0080101f30 0x7f0080102390 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:11.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.081+0000 7f0088400700 1 -- 172.21.15.1:0/1104650658 >> 172.21.15.1:0/1104650658 conn(0x7f00800fc290 msgr2=0x7f00800fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:11.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.081+0000 7f0088400700 1 -- 172.21.15.1:0/1104650658 shutdown_connections 2024-02-02T15:26:11.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.081+0000 7f0088400700 1 -- 172.21.15.1:0/1104650658 wait complete. 2024-02-02T15:26:11.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.082+0000 7f0088400700 1 Processor -- start 2024-02-02T15:26:11.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.082+0000 7f0088400700 1 -- start start 2024-02-02T15:26:11.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.083+0000 7f0088400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0080100d40 0x7f0080198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:11.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.083+0000 7f0088400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0080101f30 0x7f00801991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:11.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.083+0000 7f0088400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f00801028d0 0x7f00801a01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:11.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.083+0000 7f0088400700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00801a07b0 con 0x7f0080100d40 2024-02-02T15:26:11.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.083+0000 7f0088400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00801a0920 con 0x7f0080101f30 2024-02-02T15:26:11.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.083+0000 7f0088400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00801a0ae0 con 0x7f00801028d0 2024-02-02T15:26:11.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.083+0000 7f0085a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0080100d40 0x7f0080198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:11.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.083+0000 7f0085a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0080100d40 0x7f0080198c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55268/0 (socket says 172.21.15.1:55268) 2024-02-02T15:26:11.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.083+0000 7f0085a00700 1 -- 172.21.15.1:0/1659083492 learned_addr learned my addr 172.21.15.1:0/1659083492 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:11.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.083+0000 7f0085000700 1 --2- 172.21.15.1:0/1659083492 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0080101f30 0x7f00801991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:11.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.083+0000 7f0086400700 1 --2- 172.21.15.1:0/1659083492 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f00801028d0 0x7f00801a01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:11.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.084+0000 7f0085a00700 1 -- 172.21.15.1:0/1659083492 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f00801028d0 msgr2=0x7f00801a01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:11.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.084+0000 7f0085a00700 1 --2- 172.21.15.1:0/1659083492 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f00801028d0 0x7f00801a01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:11.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.084+0000 7f0085a00700 1 -- 172.21.15.1:0/1659083492 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0080101f30 msgr2=0x7f00801991b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:11.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.084+0000 7f0085a00700 1 --2- 172.21.15.1:0/1659083492 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0080101f30 0x7f00801991b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:11.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.084+0000 7f0085a00700 1 -- 172.21.15.1:0/1659083492 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0070009520 con 0x7f0080100d40 2024-02-02T15:26:11.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.084+0000 7f0085a00700 1 --2- 172.21.15.1:0/1659083492 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0080100d40 0x7f0080198c70 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f007000ea50 tx=0x7f007000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:11.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.084+0000 7f007e000700 1 -- 172.21.15.1:0/1659083492 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f007000cbd0 con 0x7f0080100d40 2024-02-02T15:26:11.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.084+0000 7f007e000700 1 -- 172.21.15.1:0/1659083492 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f007000cd30 con 0x7f0080100d40 2024-02-02T15:26:11.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.084+0000 7f0088400700 1 -- 172.21.15.1:0/1659083492 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00680095f0 con 0x7f0080100d40 2024-02-02T15:26:11.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.084+0000 7f007e000700 1 -- 172.21.15.1:0/1659083492 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0070018680 con 0x7f0080100d40 2024-02-02T15:26:11.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.084+0000 7f0088400700 1 -- 172.21.15.1:0/1659083492 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00801a10f0 con 0x7f0080100d40 2024-02-02T15:26:11.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.085+0000 7f0088400700 1 -- 172.21.15.1:0/1659083492 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0080193390 con 0x7f0080100d40 2024-02-02T15:26:11.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.087+0000 7f007e000700 1 -- 172.21.15.1:0/1659083492 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) v1 ==== 135120+0+0 (secure 0 0 0) 0x7f0070010430 con 0x7f0080100d40 2024-02-02T15:26:11.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.089+0000 7f007e000700 1 --2- 172.21.15.1:0/1659083492 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f00600a05b0 0x7f00600a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:11.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.089+0000 7f007e000700 1 -- 172.21.15.1:0/1659083492 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f0070014070 con 0x7f0080100d40 2024-02-02T15:26:11.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.089+0000 7f0085000700 1 --2- 172.21.15.1:0/1659083492 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f00600a05b0 0x7f00600a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:11.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.090+0000 7f0085000700 1 --2- 172.21.15.1:0/1659083492 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f00600a05b0 0x7f00600a2a70 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0068009fd0 tx=0x7f0068019040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:11.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.093+0000 7f007e000700 1 -- 172.21.15.1:0/1659083492 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f0070095a30 con 0x7f0080100d40 2024-02-02T15:26:11.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.356+0000 7f0088400700 1 -- 172.21.15.1:0/1659083492 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f008002cf00 con 0x7f0080100d40 2024-02-02T15:26:11.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.357+0000 7f007e000700 1 -- 172.21.15.1:0/1659083492 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10866 (secure 0 0 0) 0x7f0070099050 con 0x7f0080100d40 2024-02-02T15:26:11.357 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:26:11.357 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2922240448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2922240448},{"type":"v1","addr":"172.21.15.1:6835","nonce":2922240448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2729497431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2729497431},{"type":"v1","addr":"172.21.15.1:6837","nonce":2729497431}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-02T15:24:47.761435+0000","modified":"2024-02-02T15:25:20.251601+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24529,"mds_2":24518,"mds_3":24514,"mds_4":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14517":{"gid":14517,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/1218303683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":1218303683},{"type":"v1","addr":"172.21.15.1:6839","nonce":1218303683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14529":{"gid":14529,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/2059197496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2059197496},{"type":"v1","addr":"172.21.15.1:6841","nonce":2059197496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/7648986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":7648986},{"type":"v1","addr":"172.21.15.142:6833","nonce":7648986}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/3482746094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3482746094},{"type":"v1","addr":"172.21.15.71:6833","nonce":3482746094}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/388207814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":388207814},{"type":"v1","addr":"172.21.15.71:6835","nonce":388207814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6835/1052606547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1052606547},{"type":"v1","addr":"172.21.15.142:6835","nonce":1052606547}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.142:6837/2870289019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2870289019},{"type":"v1","addr":"172.21.15.142:6837","nonce":2870289019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.71:6837/628016199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":628016199},{"type":"v1","addr":"172.21.15.71:6837","nonce":628016199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.142:6839/321675421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":321675421},{"type":"v1","addr":"172.21.15.142:6839","nonce":321675421}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/1268086493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1268086493},{"type":"v1","addr":"172.21.15.71:6839","nonce":1268086493}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-02T15:26:11.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.360+0000 7f0088400700 1 -- 172.21.15.1:0/1659083492 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f00600a05b0 msgr2=0x7f00600a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:11.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.360+0000 7f0088400700 1 --2- 172.21.15.1:0/1659083492 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f00600a05b0 0x7f00600a2a70 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0068009fd0 tx=0x7f0068019040 comp rx=0 tx=0).stop 2024-02-02T15:26:11.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.360+0000 7f0088400700 1 -- 172.21.15.1:0/1659083492 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0080100d40 msgr2=0x7f0080198c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:11.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.360+0000 7f0088400700 1 --2- 172.21.15.1:0/1659083492 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0080100d40 0x7f0080198c70 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f007000ea50 tx=0x7f007000ed60 comp rx=0 tx=0).stop 2024-02-02T15:26:11.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.360+0000 7f0088400700 1 -- 172.21.15.1:0/1659083492 shutdown_connections 2024-02-02T15:26:11.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.360+0000 7f0088400700 1 --2- 172.21.15.1:0/1659083492 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f00600a05b0 0x7f00600a2a70 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:11.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.360+0000 7f0088400700 1 --2- 172.21.15.1:0/1659083492 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f00801028d0 0x7f00801a01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:11.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.360+0000 7f0088400700 1 --2- 172.21.15.1:0/1659083492 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0080100d40 0x7f0080198c70 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:11.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.360+0000 7f0088400700 1 --2- 172.21.15.1:0/1659083492 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0080101f30 0x7f00801991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:11.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.360+0000 7f0088400700 1 -- 172.21.15.1:0/1659083492 >> 172.21.15.1:0/1659083492 conn(0x7f00800fc290 msgr2=0x7f0080107870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:11.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.360+0000 7f0088400700 1 -- 172.21.15.1:0/1659083492 shutdown_connections 2024-02-02T15:26:11.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.360+0000 7f0088400700 1 -- 172.21.15.1:0/1659083492 wait complete. 2024-02-02T15:26:11.361 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 27 2024-02-02T15:26:11.379 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-02-02T15:24:47.761435+0000', 'modified': '2024-02-02T15:25:20.251601+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24557, 'mds_1': 24529, 'mds_2': 24518, 'mds_3': 24514, 'mds_4': 24502}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14517': {'gid': 14517, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.1:6839/1218303683', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6838', 'nonce': 1218303683}, {'type': 'v1', 'addr': '172.21.15.1:6839', 'nonce': 1218303683}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_14529': {'gid': 14529, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.1:6841/2059197496', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6840', 'nonce': 2059197496}, {'type': 'v1', 'addr': '172.21.15.1:6841', 'nonce': 2059197496}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24457': {'gid': 24457, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.142:6833/7648986', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6832', 'nonce': 7648986}, {'type': 'v1', 'addr': '172.21.15.142:6833', 'nonce': 7648986}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24458': {'gid': 24458, 'name': 'k', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.71:6833/3482746094', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6832', 'nonce': 3482746094}, {'type': 'v1', 'addr': '172.21.15.71:6833', 'nonce': 3482746094}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24494': {'gid': 24494, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.71:6835/388207814', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6834', 'nonce': 388207814}, {'type': 'v1', 'addr': '172.21.15.71:6835', 'nonce': 388207814}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24502': {'gid': 24502, 'name': 'f', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.142:6835/1052606547', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6834', 'nonce': 1052606547}, {'type': 'v1', 'addr': '172.21.15.142:6835', 'nonce': 1052606547}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24514': {'gid': 24514, 'name': 'c', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.142:6837/2870289019', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6836', 'nonce': 2870289019}, {'type': 'v1', 'addr': '172.21.15.142:6837', 'nonce': 2870289019}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24518': {'gid': 24518, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.71:6837/628016199', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6836', 'nonce': 628016199}, {'type': 'v1', 'addr': '172.21.15.71:6837', 'nonce': 628016199}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24529': {'gid': 24529, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.142:6839/321675421', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6838', 'nonce': 321675421}, {'type': 'v1', 'addr': '172.21.15.142:6839', 'nonce': 321675421}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24557': {'gid': 24557, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.71:6839/1268086493', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6838', 'nonce': 1268086493}, {'type': 'v1', 'addr': '172.21.15.71:6839', 'nonce': 1268086493}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-02-02T15:26:11.379 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-02-02T15:26:11.379 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2024-02-02T15:26:11.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.577+0000 7f429da00700 1 -- 172.21.15.1:0/1681081046 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4298100e00 msgr2=0x7f4298101220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:11.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.577+0000 7f429da00700 1 --2- 172.21.15.1:0/1681081046 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4298100e00 0x7f4298101220 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f42880099c0 tx=0x7f4288009cd0 comp rx=0 tx=0).stop 2024-02-02T15:26:11.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.577+0000 7f429da00700 1 -- 172.21.15.1:0/1681081046 shutdown_connections 2024-02-02T15:26:11.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.577+0000 7f429da00700 1 --2- 172.21.15.1:0/1681081046 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4298102990 0x7f4298109240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:11.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.577+0000 7f429da00700 1 --2- 172.21.15.1:0/1681081046 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4298100e00 0x7f4298101220 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:11.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.577+0000 7f429da00700 1 --2- 172.21.15.1:0/1681081046 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4298101ff0 0x7f4298102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:11.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.577+0000 7f429da00700 1 -- 172.21.15.1:0/1681081046 >> 172.21.15.1:0/1681081046 conn(0x7f42980fc390 msgr2=0x7f42980fe7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:11.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.577+0000 7f429da00700 1 -- 172.21.15.1:0/1681081046 shutdown_connections 2024-02-02T15:26:11.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.577+0000 7f429da00700 1 -- 172.21.15.1:0/1681081046 wait complete. 2024-02-02T15:26:11.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.578+0000 7f429da00700 1 Processor -- start 2024-02-02T15:26:11.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.578+0000 7f429da00700 1 -- start start 2024-02-02T15:26:11.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.578+0000 7f429da00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4298100e00 0x7f4298198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:11.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.578+0000 7f429da00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4298101ff0 0x7f4298199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:11.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.578+0000 7f429da00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4298102990 0x7f42981a02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:11.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.578+0000 7f429da00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42981a0880 con 0x7f4298102990 2024-02-02T15:26:11.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.578+0000 7f429da00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42981a09f0 con 0x7f4298100e00 2024-02-02T15:26:11.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.578+0000 7f429da00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42981a0bb0 con 0x7f4298101ff0 2024-02-02T15:26:11.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.579+0000 7f4297400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4298102990 0x7f42981a02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:11.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.579+0000 7f4297400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4298102990 0x7f42981a02b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55288/0 (socket says 172.21.15.1:55288) 2024-02-02T15:26:11.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.579+0000 7f4297400700 1 -- 172.21.15.1:0/235270189 learned_addr learned my addr 172.21.15.1:0/235270189 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:11.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.579+0000 7f4296a00700 1 --2- 172.21.15.1:0/235270189 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4298100e00 0x7f4298198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:11.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.579+0000 7f4296000700 1 --2- 172.21.15.1:0/235270189 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4298101ff0 0x7f4298199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:11.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.579+0000 7f4297400700 1 -- 172.21.15.1:0/235270189 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4298101ff0 msgr2=0x7f4298199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:11.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.579+0000 7f4297400700 1 --2- 172.21.15.1:0/235270189 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4298101ff0 0x7f4298199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:11.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.579+0000 7f4297400700 1 -- 172.21.15.1:0/235270189 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4298100e00 msgr2=0x7f4298198d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:11.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.579+0000 7f4297400700 1 --2- 172.21.15.1:0/235270189 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4298100e00 0x7f4298198d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:11.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.579+0000 7f4297400700 1 -- 172.21.15.1:0/235270189 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42880095f0 con 0x7f4298102990 2024-02-02T15:26:11.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.579+0000 7f4297400700 1 --2- 172.21.15.1:0/235270189 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4298102990 0x7f42981a02b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f429000ef30 tx=0x7f429000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:11.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.580+0000 7f428f400700 1 -- 172.21.15.1:0/235270189 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4290018070 con 0x7f4298102990 2024-02-02T15:26:11.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.580+0000 7f428f400700 1 -- 172.21.15.1:0/235270189 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f429000f040 con 0x7f4298102990 2024-02-02T15:26:11.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.580+0000 7f429da00700 1 -- 172.21.15.1:0/235270189 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42981a0e30 con 0x7f4298102990 2024-02-02T15:26:11.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.580+0000 7f428f400700 1 -- 172.21.15.1:0/235270189 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429000cd00 con 0x7f4298102990 2024-02-02T15:26:11.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.580+0000 7f429da00700 1 -- 172.21.15.1:0/235270189 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42981a1250 con 0x7f4298102990 2024-02-02T15:26:11.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.583+0000 7f428f400700 1 -- 172.21.15.1:0/235270189 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) v1 ==== 135120+0+0 (secure 0 0 0) 0x7f4290003a00 con 0x7f4298102990 2024-02-02T15:26:11.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.584+0000 7f42b14a1b80 1 Processor -- start 2024-02-02T15:26:11.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.584+0000 7f42b14a1b80 1 -- start start 2024-02-02T15:26:11.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.584+0000 7f428f400700 1 --2- 172.21.15.1:0/235270189 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f42780a05b0 0x7f42780a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:11.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.584+0000 7f42b14a1b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7dc67d130 0x55a7dc67d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:11.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.584+0000 7f428f400700 1 -- 172.21.15.1:0/235270189 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f42900cb990 con 0x7f4298102990 2024-02-02T15:26:11.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.584+0000 7f42b14a1b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a7dc67da90 0x55a7dc67e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:11.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.584+0000 7f42b14a1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7dc67e930 0x55a7dc67edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:11.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.584+0000 7f42b14a1b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7dc5f0470 con 0x55a7dc67d130 2024-02-02T15:26:11.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.584+0000 7f42b14a1b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7dc5d4a50 con 0x55a7dc67da90 2024-02-02T15:26:11.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.584+0000 7f42b14a1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7dc5d2410 con 0x55a7dc67e930 2024-02-02T15:26:11.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.585+0000 7f4296000700 1 --2- 172.21.15.1:0/235270189 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f42780a05b0 0x7f42780a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:11.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.585+0000 7f4296a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7dc67d130 0x55a7dc67d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:11.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.585+0000 7f4296a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7dc67d130 0x55a7dc67d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55300/0 (socket says 172.21.15.1:55300) 2024-02-02T15:26:11.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.585+0000 7f4296a00700 1 -- 172.21.15.1:0/2803553951 learned_addr learned my addr 172.21.15.1:0/2803553951 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:11.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.585+0000 7f4296000700 1 --2- 172.21.15.1:0/2803553951 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a7dc67da90 0x55a7dc67e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:11.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.585+0000 7f4297400700 1 --2- 172.21.15.1:0/2803553951 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7dc67e930 0x55a7dc67edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:11.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.585+0000 7f4296a00700 1 -- 172.21.15.1:0/2803553951 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7dc67e930 msgr2=0x55a7dc67edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:11.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.585+0000 7f4296a00700 1 --2- 172.21.15.1:0/2803553951 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7dc67e930 0x55a7dc67edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:11.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.585+0000 7f4296a00700 1 -- 172.21.15.1:0/2803553951 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a7dc67da90 msgr2=0x55a7dc67e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:11.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.585+0000 7f4296a00700 1 --2- 172.21.15.1:0/2803553951 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a7dc67da90 0x55a7dc67e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:11.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.585+0000 7f4296a00700 1 -- 172.21.15.1:0/2803553951 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7dc44f360 con 0x55a7dc67d130 2024-02-02T15:26:11.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.585+0000 7f4296000700 1 --2- 172.21.15.1:0/235270189 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f42780a05b0 0x7f42780a2a70 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f42800119b0 tx=0x7f4280009bf0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:11.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.585+0000 7f4296a00700 1 --2- 172.21.15.1:0/2803553951 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7dc67d130 0x55a7dc67d550 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f42880099c0 tx=0x7f4288005ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af897274934f746c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:11.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.586+0000 7f4287e00700 1 -- 172.21.15.1:0/2803553951 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f428800bad0 con 0x55a7dc67d130 2024-02-02T15:26:11.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.586+0000 7f4287e00700 1 -- 172.21.15.1:0/2803553951 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f428801e040 con 0x55a7dc67d130 2024-02-02T15:26:11.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.586+0000 7f4287e00700 1 -- 172.21.15.1:0/2803553951 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4288003b30 con 0x55a7dc67d130 2024-02-02T15:26:11.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.586+0000 7f42b14a1b80 1 -- 172.21.15.1:0/2803553951 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7dc67d130 msgr2=0x55a7dc67d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:11.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.586+0000 7f42b14a1b80 1 --2- 172.21.15.1:0/2803553951 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7dc67d130 0x55a7dc67d550 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f42880099c0 tx=0x7f4288005ee0 comp rx=0 tx=0).stop 2024-02-02T15:26:11.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.587+0000 7f42b14a1b80 1 -- 172.21.15.1:0/2803553951 shutdown_connections 2024-02-02T15:26:11.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.587+0000 7f42b14a1b80 1 --2- 172.21.15.1:0/2803553951 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7dc67e930 0x55a7dc67edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:11.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.587+0000 7f42b14a1b80 1 --2- 172.21.15.1:0/2803553951 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7dc67d130 0x55a7dc67d550 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:11.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.587+0000 7f42b14a1b80 1 --2- 172.21.15.1:0/2803553951 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a7dc67da90 0x55a7dc67e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:11.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.587+0000 7f42b14a1b80 1 -- 172.21.15.1:0/2803553951 >> 172.21.15.1:0/2803553951 conn(0x55a7dc5dbd60 msgr2=0x55a7dc677e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:11.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.587+0000 7f42b14a1b80 1 -- 172.21.15.1:0/2803553951 shutdown_connections 2024-02-02T15:26:11.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.587+0000 7f42b14a1b80 1 -- 172.21.15.1:0/2803553951 wait complete. 2024-02-02T15:26:11.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.587+0000 7f42b14a1b80 1 -- start start 2024-02-02T15:26:11.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.587+0000 7f42b14a1b80 1 Processor -- start 2024-02-02T15:26:11.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.588+0000 7f42b14a1b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a7dc67d130 0x55a7dc686e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:11.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.588+0000 7f42b14a1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7dc67da90 0x55a7dc687390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:11.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.588+0000 7f42b14a1b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7dc67e930 0x55a7dc68e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:11.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.588+0000 7f42b14a1b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7dc5cdd00 con 0x55a7dc67e930 2024-02-02T15:26:11.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.588+0000 7f42b14a1b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7dc569d30 con 0x55a7dc67d130 2024-02-02T15:26:11.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.588+0000 7f42b14a1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7dc5699d0 con 0x55a7dc67da90 2024-02-02T15:26:11.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.588+0000 7f42b14a1b80 10 client.0 _ll_register_callbacks cb 0x55a7dc56b240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:26:11.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.588+0000 7f4296a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a7dc67d130 0x55a7dc686e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:11.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.588+0000 7f4297400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7dc67e930 0x55a7dc68e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:11.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.588+0000 7f4296a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a7dc67d130 0x55a7dc686e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:56472/0 (socket says 172.21.15.1:56472) 2024-02-02T15:26:11.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.588+0000 7f4296000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7dc67da90 0x55a7dc687390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:11.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.588+0000 7f4296a00700 1 -- 172.21.15.1:0/1987072544 learned_addr learned my addr 172.21.15.1:0/1987072544 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:11.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.589+0000 7f4296a00700 1 -- 172.21.15.1:0/1987072544 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7dc67da90 msgr2=0x55a7dc687390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:11.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.589+0000 7f4296a00700 1 --2- 172.21.15.1:0/1987072544 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7dc67da90 0x55a7dc687390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:11.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.589+0000 7f4296a00700 1 -- 172.21.15.1:0/1987072544 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7dc67e930 msgr2=0x55a7dc68e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:11.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.589+0000 7f4296a00700 1 --2- 172.21.15.1:0/1987072544 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7dc67e930 0x55a7dc68e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:11.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.589+0000 7f4296a00700 1 -- 172.21.15.1:0/1987072544 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4298109b90 con 0x55a7dc67d130 2024-02-02T15:26:11.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.589+0000 7f4297400700 1 --2- 172.21.15.1:0/1987072544 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7dc67e930 0x55a7dc68e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:26:11.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.589+0000 7f4296a00700 1 --2- 172.21.15.1:0/1987072544 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a7dc67d130 0x55a7dc686e50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f428800b580 tx=0x7f42880106a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d75ac28cb1cdd126 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:11.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.589+0000 7f4284c00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:26:11.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.589+0000 7f4284c00700 1 -- 172.21.15.1:0/1987072544 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4288005270 con 0x55a7dc67d130 2024-02-02T15:26:11.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.589+0000 7f42b14a1b80 1 -- 172.21.15.1:0/1987072544 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a7dc6700a0 con 0x55a7dc67d130 2024-02-02T15:26:11.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.589+0000 7f4284c00700 1 -- 172.21.15.1:0/1987072544 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f42880053d0 con 0x55a7dc67d130 2024-02-02T15:26:11.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.589+0000 7f4284c00700 1 -- 172.21.15.1:0/1987072544 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4288005590 con 0x55a7dc67d130 2024-02-02T15:26:11.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.589+0000 7f42b14a1b80 10 client.25121 fetch_fsmap learned FSMap version 27 2024-02-02T15:26:11.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.590+0000 7f42b14a1b80 1 -- 172.21.15.1:0/1987072544 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55a7dc3fbb10 con 0x55a7dc67d130 2024-02-02T15:26:11.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.590+0000 7f4284c00700 1 -- 172.21.15.1:0/1987072544 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a7dc6700a0 con 0x55a7dc67d130 2024-02-02T15:26:11.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.590+0000 7f4284c00700 1 -- 172.21.15.1:0/1987072544 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f42880223f0 con 0x55a7dc67d130 2024-02-02T15:26:11.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.590+0000 7f42b14a1b80 10 client.25121 fetch_fsmap finished waiting for FSMap version 27 2024-02-02T15:26:11.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.590+0000 7f42b14a1b80 10 client.25121 resolve_mds: resolved name 'f' to daemon mds.f 2024-02-02T15:26:11.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.590+0000 7f42b14a1b80 20 client.25121 populate_metadata read hostname 'smithi001' 2024-02-02T15:26:11.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.590+0000 7f42b14a1b80 1 --2- 172.21.15.1:0/1987072544 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55a7dc691c40 0x55a7dc691300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:11.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.590+0000 7f42b14a1b80 4 client.25121 mds_command: new command op to 24502 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:26:11.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.590+0000 7f42b14a1b80 1 -- 172.21.15.1:0/1987072544 --> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a7dc5c6d80 con 0x55a7dc691c40 2024-02-02T15:26:11.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.590+0000 7f4296000700 1 --2- 172.21.15.1:0/1987072544 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55a7dc691c40 0x55a7dc691300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:11.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.591+0000 7f4296000700 1 --2- 172.21.15.1:0/1987072544 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55a7dc691c40 0x55a7dc691300 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=edaa1ec3c0eeb4ee server_cookie=d3566b3b4fdddaf8 in_seq=0 out_seq=0 2024-02-02T15:26:11.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.591+0000 7f4284c00700 10 client.25121 ms_handle_connect on v2:172.21.15.142:6834/1052606547 2024-02-02T15:26:11.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.592+0000 7f4284c00700 1 -- 172.21.15.1:0/1987072544 <== mds.4 v2:172.21.15.142:6834/1052606547 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55a7dc5c6d80 con 0x55a7dc691c40 2024-02-02T15:26:11.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.592+0000 7f4284c00700 10 client.25121 handle_command_reply: tid=0 2024-02-02T15:26:11.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.592+0000 7f42b14a1b80 1 client.25121 shutdown 2024-02-02T15:26:11.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.592+0000 7f42b14a1b80 1 -- 172.21.15.1:0/1987072544 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a7dc67d130 msgr2=0x55a7dc686e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:11.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.592+0000 7f42b14a1b80 1 --2- 172.21.15.1:0/1987072544 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a7dc67d130 0x55a7dc686e50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f428800b580 tx=0x7f42880106a0 comp rx=0 tx=0).stop 2024-02-02T15:26:11.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.592+0000 7f42b14a1b80 1 -- 172.21.15.1:0/1987072544 shutdown_connections 2024-02-02T15:26:11.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.592+0000 7f42b14a1b80 1 --2- 172.21.15.1:0/1987072544 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55a7dc691c40 0x55a7dc691300 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:11.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.592+0000 7f42b14a1b80 1 --2- 172.21.15.1:0/1987072544 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7dc67e930 0x55a7dc68e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:11.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.592+0000 7f42b14a1b80 1 --2- 172.21.15.1:0/1987072544 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a7dc67d130 0x55a7dc686e50 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:11.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.592+0000 7f4284c00700 0 client.25121 ms_handle_reset on v2:172.21.15.142:6834/1052606547 2024-02-02T15:26:11.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.592+0000 7f42b14a1b80 1 --2- 172.21.15.1:0/1987072544 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7dc67da90 0x55a7dc687390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:11.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.592+0000 7f42b14a1b80 1 -- 172.21.15.1:0/1987072544 >> 172.21.15.1:0/1987072544 conn(0x55a7dc5dbd60 msgr2=0x55a7dc677570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:11.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.592+0000 7f42b14a1b80 1 -- 172.21.15.1:0/1987072544 shutdown_connections 2024-02-02T15:26:11.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.592+0000 7f42b14a1b80 1 -- 172.21.15.1:0/1987072544 wait complete. 2024-02-02T15:26:11.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.592+0000 7f42b14a1b80 20 client.25121 trim_cache size 0 max 16384 2024-02-02T15:26:11.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.610+0000 7f42b14a1b80 1 Processor -- start 2024-02-02T15:26:11.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.611+0000 7f42b14a1b80 1 -- start start 2024-02-02T15:26:11.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.611+0000 7f42b14a1b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a7dc67d130 0x55a7dc677e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:11.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.611+0000 7f42b14a1b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7dc67da90 0x55a7dc69a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:11.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.611+0000 7f42b14a1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7dc67e930 0x55a7dc691300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:11.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.611+0000 7f42b14a1b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7dc686a20 con 0x55a7dc67da90 2024-02-02T15:26:11.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.611+0000 7f42b14a1b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7dc451650 con 0x55a7dc67d130 2024-02-02T15:26:11.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.611+0000 7f42b14a1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7dc5776b0 con 0x55a7dc67e930 2024-02-02T15:26:11.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.611+0000 7f4296000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7dc67da90 0x55a7dc69a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:11.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.611+0000 7f4296a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a7dc67d130 0x55a7dc677e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:11.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.611+0000 7f4296000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7dc67da90 0x55a7dc69a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55316/0 (socket says 172.21.15.1:55316) 2024-02-02T15:26:11.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.611+0000 7f4296000700 1 -- 172.21.15.1:0/3221106309 learned_addr learned my addr 172.21.15.1:0/3221106309 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:11.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.611+0000 7f4297400700 1 --2- 172.21.15.1:0/3221106309 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7dc67e930 0x55a7dc691300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:11.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.611+0000 7f4296000700 1 -- 172.21.15.1:0/3221106309 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7dc67e930 msgr2=0x55a7dc691300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:11.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.611+0000 7f4296000700 1 --2- 172.21.15.1:0/3221106309 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7dc67e930 0x55a7dc691300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:11.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.611+0000 7f4296000700 1 -- 172.21.15.1:0/3221106309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a7dc67d130 msgr2=0x55a7dc677e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:11.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.611+0000 7f4296000700 1 --2- 172.21.15.1:0/3221106309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a7dc67d130 0x55a7dc677e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:11.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.612+0000 7f4296000700 1 -- 172.21.15.1:0/3221106309 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7dc593e50 con 0x55a7dc67da90 2024-02-02T15:26:11.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.612+0000 7f4296a00700 1 --2- 172.21.15.1:0/3221106309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a7dc67d130 0x55a7dc677e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:26:11.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.612+0000 7f4296000700 1 --2- 172.21.15.1:0/3221106309 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7dc67da90 0x55a7dc69a9b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f4280009e30 tx=0x7f4280021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a8f1906072d0348 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:11.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.612+0000 7f428cc00700 1 -- 172.21.15.1:0/3221106309 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4280016040 con 0x55a7dc67da90 2024-02-02T15:26:11.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.612+0000 7f428cc00700 1 -- 172.21.15.1:0/3221106309 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f428002b3b0 con 0x55a7dc67da90 2024-02-02T15:26:11.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.612+0000 7f428cc00700 1 -- 172.21.15.1:0/3221106309 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f428000f830 con 0x55a7dc67da90 2024-02-02T15:26:11.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.613+0000 7f42b14a1b80 1 -- 172.21.15.1:0/3221106309 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7dc67da90 msgr2=0x55a7dc69a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:11.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.613+0000 7f42b14a1b80 1 --2- 172.21.15.1:0/3221106309 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7dc67da90 0x55a7dc69a9b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f4280009e30 tx=0x7f4280021000 comp rx=0 tx=0).stop 2024-02-02T15:26:11.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.613+0000 7f42b14a1b80 1 -- 172.21.15.1:0/3221106309 shutdown_connections 2024-02-02T15:26:11.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.613+0000 7f42b14a1b80 1 --2- 172.21.15.1:0/3221106309 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7dc67e930 0x55a7dc691300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:11.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.613+0000 7f42b14a1b80 1 --2- 172.21.15.1:0/3221106309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a7dc67d130 0x55a7dc677e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:11.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.613+0000 7f42b14a1b80 1 --2- 172.21.15.1:0/3221106309 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7dc67da90 0x55a7dc69a9b0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:11.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.613+0000 7f42b14a1b80 1 -- 172.21.15.1:0/3221106309 >> 172.21.15.1:0/3221106309 conn(0x55a7dc5dbd60 msgr2=0x55a7dc685bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:11.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.613+0000 7f42b14a1b80 1 -- 172.21.15.1:0/3221106309 shutdown_connections 2024-02-02T15:26:11.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.613+0000 7f42b14a1b80 1 -- 172.21.15.1:0/3221106309 wait complete. 2024-02-02T15:26:11.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.613+0000 7f42b14a1b80 1 -- start start 2024-02-02T15:26:11.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.613+0000 7f42b14a1b80 1 Processor -- start 2024-02-02T15:26:11.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:11 smithi071 ceph-mon[44707]: pgmap v278: 129 pgs: 129 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-02-02T15:26:11.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:11 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/2534898477' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:26:11.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:11 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/3780855617' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-02T15:26:11.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.614+0000 7f42b14a1b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a7dc67d130 0x55a7dc695020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:11.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.614+0000 7f42b14a1b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7dc67da90 0x55a7dc695560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:11.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.614+0000 7f42b14a1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7dc67e930 0x55a7dc69a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:11.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.614+0000 7f42b14a1b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7dc69a8f0 con 0x55a7dc67da90 2024-02-02T15:26:11.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.614+0000 7f42b14a1b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7dc69aa60 con 0x55a7dc67d130 2024-02-02T15:26:11.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.614+0000 7f42b14a1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7dc69ac20 con 0x55a7dc67e930 2024-02-02T15:26:11.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.614+0000 7f42b14a1b80 10 client.0 _ll_register_callbacks cb 0x55a7dc56b240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:26:11.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.614+0000 7f4296000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7dc67da90 0x55a7dc695560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:11.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.614+0000 7f4296a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a7dc67d130 0x55a7dc695020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:11.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.614+0000 7f4296000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7dc67da90 0x55a7dc695560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55328/0 (socket says 172.21.15.1:55328) 2024-02-02T15:26:11.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.614+0000 7f4296000700 1 -- 172.21.15.1:0/267878653 learned_addr learned my addr 172.21.15.1:0/267878653 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:11.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.614+0000 7f4297400700 1 --2- 172.21.15.1:0/267878653 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7dc67e930 0x55a7dc69a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:11.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.614+0000 7f4296000700 1 -- 172.21.15.1:0/267878653 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7dc67e930 msgr2=0x55a7dc69a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:11.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.615+0000 7f4296000700 1 --2- 172.21.15.1:0/267878653 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7dc67e930 0x55a7dc69a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:11.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.615+0000 7f4296000700 1 -- 172.21.15.1:0/267878653 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a7dc67d130 msgr2=0x55a7dc695020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:11.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.615+0000 7f4296000700 1 --2- 172.21.15.1:0/267878653 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a7dc67d130 0x55a7dc695020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:11.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.615+0000 7f4296000700 1 -- 172.21.15.1:0/267878653 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a7dc3fbb10 con 0x55a7dc67da90 2024-02-02T15:26:11.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.615+0000 7f4297400700 1 --2- 172.21.15.1:0/267878653 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7dc67e930 0x55a7dc69a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:26:11.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.615+0000 7f4296a00700 1 --2- 172.21.15.1:0/267878653 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a7dc67d130 0x55a7dc695020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:26:11.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.615+0000 7f4296000700 1 --2- 172.21.15.1:0/267878653 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7dc67da90 0x55a7dc695560 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f428002e040 tx=0x7f428000f6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69fa3dbeaff00984 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:11.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.615+0000 7f4284c00700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:26:11.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.615+0000 7f4284c00700 1 -- 172.21.15.1:0/267878653 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4280016030 con 0x55a7dc67da90 2024-02-02T15:26:11.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.615+0000 7f4284c00700 1 -- 172.21.15.1:0/267878653 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f428002b8e0 con 0x55a7dc67da90 2024-02-02T15:26:11.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.615+0000 7f42b14a1b80 1 -- 172.21.15.1:0/267878653 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f428801e040 con 0x55a7dc67da90 2024-02-02T15:26:11.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.615+0000 7f4284c00700 1 -- 172.21.15.1:0/267878653 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f428002bac0 con 0x55a7dc67da90 2024-02-02T15:26:11.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.615+0000 7f42b14a1b80 10 client.15099 fetch_fsmap learned FSMap version 27 2024-02-02T15:26:11.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.615+0000 7f42b14a1b80 1 -- 172.21.15.1:0/267878653 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f4280025040 con 0x55a7dc67da90 2024-02-02T15:26:11.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.616+0000 7f4284c00700 1 -- 172.21.15.1:0/267878653 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f428801e040 con 0x55a7dc67da90 2024-02-02T15:26:11.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.616+0000 7f4284c00700 1 -- 172.21.15.1:0/267878653 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4280003680 con 0x55a7dc67da90 2024-02-02T15:26:11.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.616+0000 7f42b14a1b80 10 client.15099 fetch_fsmap finished waiting for FSMap version 27 2024-02-02T15:26:11.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.616+0000 7f42b14a1b80 10 client.15099 resolve_mds: resolved name 'f' to daemon mds.f 2024-02-02T15:26:11.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.616+0000 7f42b14a1b80 20 client.15099 populate_metadata read hostname 'smithi001' 2024-02-02T15:26:11.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.616+0000 7f42b14a1b80 1 --2- 172.21.15.1:0/267878653 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55a7dc6e80e0 0x55a7dc6e4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:11.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.616+0000 7f42b14a1b80 4 client.15099 mds_command: new command op to 24502 tid=0[{"prefix": "status"}] 2024-02-02T15:26:11.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.616+0000 7f42b14a1b80 1 -- 172.21.15.1:0/267878653 --> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a7dc58cf00 con 0x55a7dc6e80e0 2024-02-02T15:26:11.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.616+0000 7f4296a00700 1 --2- 172.21.15.1:0/267878653 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55a7dc6e80e0 0x55a7dc6e4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:11.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.617+0000 7f4296a00700 1 --2- 172.21.15.1:0/267878653 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55a7dc6e80e0 0x55a7dc6e4a90 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7d4e6c1c59ff9e84 server_cookie=6f1bd30a5b0b1ac0 in_seq=0 out_seq=0 2024-02-02T15:26:11.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.617+0000 7f4284c00700 10 client.15099 ms_handle_connect on v2:172.21.15.142:6834/1052606547 2024-02-02T15:26:11.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.617+0000 7f4284c00700 1 -- 172.21.15.1:0/267878653 <== mds.4 v2:172.21.15.142:6834/1052606547 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x55a7dc58cf00 con 0x55a7dc6e80e0 2024-02-02T15:26:11.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.617+0000 7f4284c00700 10 client.15099 handle_command_reply: tid=0 2024-02-02T15:26:11.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.617+0000 7f42b14a1b80 1 client.15099 shutdown 2024-02-02T15:26:11.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.617+0000 7f42b14a1b80 1 -- 172.21.15.1:0/267878653 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7dc67da90 msgr2=0x55a7dc695560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:11.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.617+0000 7f42b14a1b80 1 --2- 172.21.15.1:0/267878653 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7dc67da90 0x55a7dc695560 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f428002e040 tx=0x7f428000f6d0 comp rx=0 tx=0).stop 2024-02-02T15:26:11.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.617+0000 7f42b14a1b80 1 -- 172.21.15.1:0/267878653 shutdown_connections 2024-02-02T15:26:11.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.617+0000 7f42b14a1b80 1 --2- 172.21.15.1:0/267878653 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55a7dc6e80e0 0x55a7dc6e4a90 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:11.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.617+0000 7f42b14a1b80 1 --2- 172.21.15.1:0/267878653 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7dc67e930 0x55a7dc69a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:11.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.617+0000 7f42b14a1b80 1 --2- 172.21.15.1:0/267878653 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a7dc67d130 0x55a7dc695020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:11.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.617+0000 7f4284c00700 0 client.15099 ms_handle_reset on v2:172.21.15.142:6834/1052606547 2024-02-02T15:26:11.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.617+0000 7f42b14a1b80 1 --2- 172.21.15.1:0/267878653 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7dc67da90 0x55a7dc695560 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:11.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.617+0000 7f42b14a1b80 1 -- 172.21.15.1:0/267878653 >> 172.21.15.1:0/267878653 conn(0x55a7dc5dbd60 msgr2=0x55a7dc685bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:11.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.617+0000 7f42b14a1b80 1 -- 172.21.15.1:0/267878653 shutdown_connections 2024-02-02T15:26:11.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.617+0000 7f42b14a1b80 1 -- 172.21.15.1:0/267878653 wait complete. 2024-02-02T15:26:11.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.618+0000 7f42b14a1b80 20 client.15099 trim_cache size 0 max 16384 2024-02-02T15:26:11.622 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:26:11.623 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "371a409c-c1de-11ee-95b6-87774f69a715", 2024-02-02T15:26:11.623 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 4, 2024-02-02T15:26:11.623 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 24502, 2024-02-02T15:26:11.623 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2024-02-02T15:26:11.623 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2024-02-02T15:26:11.623 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2024-02-02T15:26:11.623 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 52.386639017, 2024-02-02T15:26:11.623 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 27, 2024-02-02T15:26:11.623 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 86, 2024-02-02T15:26:11.623 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 84, 2024-02-02T15:26:11.623 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 102.55963411099999 2024-02-02T15:26:11.623 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:26:11.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.618+0000 7f429da00700 1 -- 172.21.15.1:0/235270189 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f42780a05b0 msgr2=0x7f42780a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:11.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.618+0000 7f429da00700 1 --2- 172.21.15.1:0/235270189 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f42780a05b0 0x7f42780a2a70 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f42800119b0 tx=0x7f4280009bf0 comp rx=0 tx=0).stop 2024-02-02T15:26:11.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.618+0000 7f429da00700 1 -- 172.21.15.1:0/235270189 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4298102990 msgr2=0x7f42981a02b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:11.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.618+0000 7f429da00700 1 --2- 172.21.15.1:0/235270189 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4298102990 0x7f42981a02b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f429000ef30 tx=0x7f429000c3a0 comp rx=0 tx=0).stop 2024-02-02T15:26:11.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.618+0000 7f429da00700 1 -- 172.21.15.1:0/235270189 shutdown_connections 2024-02-02T15:26:11.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.618+0000 7f429da00700 1 --2- 172.21.15.1:0/235270189 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4298102990 0x7f42981a02b0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:11.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.618+0000 7f429da00700 1 --2- 172.21.15.1:0/235270189 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4298100e00 0x7f4298198d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:11.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.618+0000 7f429da00700 1 --2- 172.21.15.1:0/235270189 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f42780a05b0 0x7f42780a2a70 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:11.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.618+0000 7f429da00700 1 --2- 172.21.15.1:0/235270189 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4298101ff0 0x7f4298199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:11.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.618+0000 7f429da00700 1 -- 172.21.15.1:0/235270189 >> 172.21.15.1:0/235270189 conn(0x7f42980fc390 msgr2=0x7f42980fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:11.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.618+0000 7f429da00700 1 -- 172.21.15.1:0/235270189 shutdown_connections 2024-02-02T15:26:11.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.618+0000 7f429da00700 1 -- 172.21.15.1:0/235270189 wait complete. 2024-02-02T15:26:11.634 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:11 smithi001 ceph-mon[35470]: pgmap v278: 129 pgs: 129 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-02-02T15:26:11.635 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:11 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/2534898477' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:26:11.635 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:11 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/3780855617' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-02T15:26:11.637 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-02-02T15:26:11.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:11 smithi142 ceph-mon[45728]: pgmap v278: 129 pgs: 129 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-02-02T15:26:11.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:11 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/2534898477' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:26:11.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:11 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/3780855617' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-02T15:26:11.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.836+0000 7f2a22400700 1 -- 172.21.15.1:0/4241860843 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a1c100e00 msgr2=0x7f2a1c101220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:11.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.836+0000 7f2a22400700 1 --2- 172.21.15.1:0/4241860843 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a1c100e00 0x7f2a1c101220 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c0099c0 tx=0x7f2a0c009cd0 comp rx=0 tx=0).stop 2024-02-02T15:26:11.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.836+0000 7f2a22400700 1 -- 172.21.15.1:0/4241860843 shutdown_connections 2024-02-02T15:26:11.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.836+0000 7f2a22400700 1 --2- 172.21.15.1:0/4241860843 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a1c102990 0x7f2a1c109240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:11.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.836+0000 7f2a22400700 1 --2- 172.21.15.1:0/4241860843 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a1c100e00 0x7f2a1c101220 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:11.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.836+0000 7f2a22400700 1 --2- 172.21.15.1:0/4241860843 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a1c101ff0 0x7f2a1c102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:11.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.836+0000 7f2a22400700 1 -- 172.21.15.1:0/4241860843 >> 172.21.15.1:0/4241860843 conn(0x7f2a1c0fc370 msgr2=0x7f2a1c0fe7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:11.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.837+0000 7f2a22400700 1 -- 172.21.15.1:0/4241860843 shutdown_connections 2024-02-02T15:26:11.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.837+0000 7f2a22400700 1 -- 172.21.15.1:0/4241860843 wait complete. 2024-02-02T15:26:11.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.838+0000 7f2a22400700 1 Processor -- start 2024-02-02T15:26:11.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.838+0000 7f2a22400700 1 -- start start 2024-02-02T15:26:11.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.838+0000 7f2a22400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a1c100e00 0x7f2a1c198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:11.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.838+0000 7f2a22400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a1c101ff0 0x7f2a1c199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:11.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.838+0000 7f2a22400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a1c102990 0x7f2a1c1a02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:11.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.838+0000 7f2a22400700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a1c1a0880 con 0x7f2a1c101ff0 2024-02-02T15:26:11.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.838+0000 7f2a22400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a1c1a09f0 con 0x7f2a1c102990 2024-02-02T15:26:11.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.838+0000 7f2a22400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a1c1a0bb0 con 0x7f2a1c100e00 2024-02-02T15:26:11.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.838+0000 7f2a1aa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a1c101ff0 0x7f2a1c199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:11.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.838+0000 7f2a1aa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a1c101ff0 0x7f2a1c199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55360/0 (socket says 172.21.15.1:55360) 2024-02-02T15:26:11.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.838+0000 7f2a1aa00700 1 -- 172.21.15.1:0/3682277949 learned_addr learned my addr 172.21.15.1:0/3682277949 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:11.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.839+0000 7f2a1b400700 1 --2- 172.21.15.1:0/3682277949 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a1c100e00 0x7f2a1c198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:11.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.839+0000 7f2a1be00700 1 --2- 172.21.15.1:0/3682277949 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a1c102990 0x7f2a1c1a02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:11.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.839+0000 7f2a1aa00700 1 -- 172.21.15.1:0/3682277949 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a1c100e00 msgr2=0x7f2a1c198d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:11.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.839+0000 7f2a1aa00700 1 --2- 172.21.15.1:0/3682277949 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a1c100e00 0x7f2a1c198d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:11.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.839+0000 7f2a1aa00700 1 -- 172.21.15.1:0/3682277949 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a1c102990 msgr2=0x7f2a1c1a02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:11.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.839+0000 7f2a1aa00700 1 --2- 172.21.15.1:0/3682277949 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a1c102990 0x7f2a1c1a02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:11.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.839+0000 7f2a1aa00700 1 -- 172.21.15.1:0/3682277949 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a0c0095f0 con 0x7f2a1c101ff0 2024-02-02T15:26:11.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.839+0000 7f2a1aa00700 1 --2- 172.21.15.1:0/3682277949 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a1c101ff0 0x7f2a1c199280 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2a0400e9a0 tx=0x7f2a0400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:11.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.839+0000 7f2a13e00700 1 -- 172.21.15.1:0/3682277949 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a04009790 con 0x7f2a1c101ff0 2024-02-02T15:26:11.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.839+0000 7f2a13e00700 1 -- 172.21.15.1:0/3682277949 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a04010430 con 0x7f2a1c101ff0 2024-02-02T15:26:11.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.839+0000 7f2a22400700 1 -- 172.21.15.1:0/3682277949 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a1c1a0e90 con 0x7f2a1c101ff0 2024-02-02T15:26:11.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.839+0000 7f2a13e00700 1 -- 172.21.15.1:0/3682277949 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a04018680 con 0x7f2a1c101ff0 2024-02-02T15:26:11.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.840+0000 7f2a22400700 1 -- 172.21.15.1:0/3682277949 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a1c1a12c0 con 0x7f2a1c101ff0 2024-02-02T15:26:11.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.843+0000 7f2a13e00700 1 -- 172.21.15.1:0/3682277949 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) v1 ==== 135120+0+0 (secure 0 0 0) 0x7f2a040105a0 con 0x7f2a1c101ff0 2024-02-02T15:26:11.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.844+0000 7f2a35eb3b80 1 Processor -- start 2024-02-02T15:26:11.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.844+0000 7f2a35eb3b80 1 -- start start 2024-02-02T15:26:11.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.844+0000 7f2a13e00700 1 --2- 172.21.15.1:0/3682277949 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f29fc0a0630 0x7f29fc0a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:11.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.844+0000 7f2a13e00700 1 -- 172.21.15.1:0/3682277949 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f2a04014070 con 0x7f2a1c101ff0 2024-02-02T15:26:11.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.844+0000 7f2a35eb3b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f9dd27f130 0x55f9dd27f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:11.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.844+0000 7f2a1be00700 1 --2- 172.21.15.1:0/3682277949 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f29fc0a0630 0x7f29fc0a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:11.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.844+0000 7f2a35eb3b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f9dd27fa90 0x55f9dd2803a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:11.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.844+0000 7f2a35eb3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f9dd280930 0x55f9dd280dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:11.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.844+0000 7f2a35eb3b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9dd1f2470 con 0x55f9dd27fa90 2024-02-02T15:26:11.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.844+0000 7f2a35eb3b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9dd1d6a50 con 0x55f9dd27f130 2024-02-02T15:26:11.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.844+0000 7f2a35eb3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9dd1d4410 con 0x55f9dd280930 2024-02-02T15:26:11.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.845+0000 7f2a1aa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f9dd27fa90 0x55f9dd2803a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:11.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.845+0000 7f2a1aa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f9dd27fa90 0x55f9dd2803a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55362/0 (socket says 172.21.15.1:55362) 2024-02-02T15:26:11.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.845+0000 7f2a1aa00700 1 -- 172.21.15.1:0/634231652 learned_addr learned my addr 172.21.15.1:0/634231652 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:11.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.845+0000 7f2a1b400700 1 --2- 172.21.15.1:0/634231652 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f9dd27f130 0x55f9dd27f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:11.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.845+0000 7f2a1be00700 1 --2- 172.21.15.1:0/634231652 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f9dd280930 0x55f9dd280dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:11.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.845+0000 7f2a1aa00700 1 -- 172.21.15.1:0/634231652 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f9dd280930 msgr2=0x55f9dd280dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:11.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.845+0000 7f2a1aa00700 1 --2- 172.21.15.1:0/634231652 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f9dd280930 0x55f9dd280dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:11.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.845+0000 7f2a1aa00700 1 -- 172.21.15.1:0/634231652 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f9dd27f130 msgr2=0x55f9dd27f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:11.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.845+0000 7f2a1aa00700 1 --2- 172.21.15.1:0/634231652 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f9dd27f130 0x55f9dd27f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:11.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.845+0000 7f2a1aa00700 1 -- 172.21.15.1:0/634231652 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f9dd051360 con 0x55f9dd27fa90 2024-02-02T15:26:11.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.845+0000 7f2a1be00700 1 --2- 172.21.15.1:0/3682277949 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f29fc0a0630 0x7f29fc0a2af0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2a1400f540 tx=0x7f2a14009920 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:11.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.845+0000 7f2a1aa00700 1 --2- 172.21.15.1:0/634231652 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f9dd27fa90 0x55f9dd2803a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2a04095de0 tx=0x7f2a04099a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24566c333a5a0850 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:11.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.845+0000 7f2a10c00700 1 -- 172.21.15.1:0/634231652 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a0409d1a0 con 0x55f9dd27fa90 2024-02-02T15:26:11.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.846+0000 7f2a10c00700 1 -- 172.21.15.1:0/634231652 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a040a13e0 con 0x55f9dd27fa90 2024-02-02T15:26:11.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.846+0000 7f2a10c00700 1 -- 172.21.15.1:0/634231652 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a04098a60 con 0x55f9dd27fa90 2024-02-02T15:26:11.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.846+0000 7f2a35eb3b80 1 -- 172.21.15.1:0/634231652 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f9dd27fa90 msgr2=0x55f9dd2803a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:11.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.846+0000 7f2a35eb3b80 1 --2- 172.21.15.1:0/634231652 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f9dd27fa90 0x55f9dd2803a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2a04095de0 tx=0x7f2a04099a70 comp rx=0 tx=0).stop 2024-02-02T15:26:11.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.846+0000 7f2a35eb3b80 1 -- 172.21.15.1:0/634231652 shutdown_connections 2024-02-02T15:26:11.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.846+0000 7f2a35eb3b80 1 --2- 172.21.15.1:0/634231652 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f9dd280930 0x55f9dd280dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:11.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.846+0000 7f2a35eb3b80 1 --2- 172.21.15.1:0/634231652 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f9dd27f130 0x55f9dd27f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:11.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.846+0000 7f2a35eb3b80 1 --2- 172.21.15.1:0/634231652 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f9dd27fa90 0x55f9dd2803a0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:11.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.846+0000 7f2a35eb3b80 1 -- 172.21.15.1:0/634231652 >> 172.21.15.1:0/634231652 conn(0x55f9dd1ddd60 msgr2=0x55f9dd279e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:11.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.846+0000 7f2a35eb3b80 1 -- 172.21.15.1:0/634231652 shutdown_connections 2024-02-02T15:26:11.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.846+0000 7f2a35eb3b80 1 -- 172.21.15.1:0/634231652 wait complete. 2024-02-02T15:26:11.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.847+0000 7f2a35eb3b80 1 -- start start 2024-02-02T15:26:11.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.847+0000 7f2a35eb3b80 1 Processor -- start 2024-02-02T15:26:11.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.848+0000 7f2a35eb3b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f9dd27f130 0x55f9dd288ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:11.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.848+0000 7f2a35eb3b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f9dd27fa90 0x55f9dd289420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:11.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.848+0000 7f2a35eb3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f9dd280930 0x55f9dd290460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:11.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.848+0000 7f2a35eb3b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9dd1cfd00 con 0x55f9dd27fa90 2024-02-02T15:26:11.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.848+0000 7f2a35eb3b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9dd16bd30 con 0x55f9dd27f130 2024-02-02T15:26:11.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.848+0000 7f2a35eb3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9dd16b9d0 con 0x55f9dd280930 2024-02-02T15:26:11.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.848+0000 7f2a35eb3b80 10 client.0 _ll_register_callbacks cb 0x55f9dd16d240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:26:11.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.848+0000 7f2a1b400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f9dd27f130 0x55f9dd288ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:11.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.848+0000 7f2a1aa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f9dd27fa90 0x55f9dd289420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:11.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.848+0000 7f2a1b400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f9dd27f130 0x55f9dd288ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:56520/0 (socket says 172.21.15.1:56520) 2024-02-02T15:26:11.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.848+0000 7f2a1b400700 1 -- 172.21.15.1:0/2672624038 learned_addr learned my addr 172.21.15.1:0/2672624038 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:11.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.848+0000 7f2a1be00700 1 --2- 172.21.15.1:0/2672624038 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f9dd280930 0x55f9dd290460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:11.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.848+0000 7f2a1b400700 1 -- 172.21.15.1:0/2672624038 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f9dd280930 msgr2=0x55f9dd290460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:11.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.848+0000 7f2a1b400700 1 --2- 172.21.15.1:0/2672624038 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f9dd280930 0x55f9dd290460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:11.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.848+0000 7f2a1b400700 1 -- 172.21.15.1:0/2672624038 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f9dd27fa90 msgr2=0x55f9dd289420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:11.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.848+0000 7f2a1b400700 1 --2- 172.21.15.1:0/2672624038 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f9dd27fa90 0x55f9dd289420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:11.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.848+0000 7f2a1b400700 1 -- 172.21.15.1:0/2672624038 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a1c1a12c0 con 0x55f9dd27f130 2024-02-02T15:26:11.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.849+0000 7f2a1b400700 1 --2- 172.21.15.1:0/2672624038 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f9dd27f130 0x55f9dd288ee0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c000c00 tx=0x7f2a0c0057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=524b140fb4f915fd server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:11.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.849+0000 7f2a09600700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:26:11.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.849+0000 7f2a09600700 1 -- 172.21.15.1:0/2672624038 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a0c0055a0 con 0x55f9dd27f130 2024-02-02T15:26:11.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.849+0000 7f2a09600700 1 -- 172.21.15.1:0/2672624038 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f2a0c0037d0 con 0x55f9dd27f130 2024-02-02T15:26:11.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.849+0000 7f2a35eb3b80 1 -- 172.21.15.1:0/2672624038 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f9dd2720a0 con 0x55f9dd27f130 2024-02-02T15:26:11.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.849+0000 7f2a09600700 1 -- 172.21.15.1:0/2672624038 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a0c003990 con 0x55f9dd27f130 2024-02-02T15:26:11.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.849+0000 7f2a35eb3b80 10 client.25139 fetch_fsmap learned FSMap version 27 2024-02-02T15:26:11.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.849+0000 7f2a35eb3b80 1 -- 172.21.15.1:0/2672624038 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f9dcffdb10 con 0x55f9dd27f130 2024-02-02T15:26:11.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.849+0000 7f2a09600700 1 -- 172.21.15.1:0/2672624038 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f9dd2720a0 con 0x55f9dd27f130 2024-02-02T15:26:11.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.850+0000 7f2a09600700 1 -- 172.21.15.1:0/2672624038 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2a0c024ac0 con 0x55f9dd27f130 2024-02-02T15:26:11.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.850+0000 7f2a35eb3b80 10 client.25139 fetch_fsmap finished waiting for FSMap version 27 2024-02-02T15:26:11.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.850+0000 7f2a35eb3b80 10 client.25139 resolve_mds: resolved name 'c' to daemon mds.c 2024-02-02T15:26:11.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.850+0000 7f2a35eb3b80 20 client.25139 populate_metadata read hostname 'smithi001' 2024-02-02T15:26:11.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.850+0000 7f2a35eb3b80 1 --2- 172.21.15.1:0/2672624038 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55f9dd294180 0x55f9dd293560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:11.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.850+0000 7f2a35eb3b80 4 client.25139 mds_command: new command op to 24514 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:26:11.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.850+0000 7f2a35eb3b80 1 -- 172.21.15.1:0/2672624038 --> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f9dd1c8d80 con 0x55f9dd294180 2024-02-02T15:26:11.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.850+0000 7f2a1aa00700 1 --2- 172.21.15.1:0/2672624038 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55f9dd294180 0x55f9dd293560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:11.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.851+0000 7f2a1aa00700 1 --2- 172.21.15.1:0/2672624038 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55f9dd294180 0x55f9dd293560 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7cd5aa70ebfcdff5 server_cookie=a2f2df7ee2950a66 in_seq=0 out_seq=0 2024-02-02T15:26:11.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.851+0000 7f2a09600700 10 client.25139 ms_handle_connect on v2:172.21.15.142:6836/2870289019 2024-02-02T15:26:11.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.851+0000 7f2a09600700 1 -- 172.21.15.1:0/2672624038 <== mds.3 v2:172.21.15.142:6836/2870289019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55f9dd1c8d80 con 0x55f9dd294180 2024-02-02T15:26:11.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.851+0000 7f2a09600700 10 client.25139 handle_command_reply: tid=0 2024-02-02T15:26:11.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.852+0000 7f2a35eb3b80 1 client.25139 shutdown 2024-02-02T15:26:11.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.852+0000 7f2a35eb3b80 1 -- 172.21.15.1:0/2672624038 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f9dd27f130 msgr2=0x55f9dd288ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:11.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.852+0000 7f2a35eb3b80 1 --2- 172.21.15.1:0/2672624038 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f9dd27f130 0x55f9dd288ee0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c000c00 tx=0x7f2a0c0057f0 comp rx=0 tx=0).stop 2024-02-02T15:26:11.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.852+0000 7f2a35eb3b80 1 -- 172.21.15.1:0/2672624038 shutdown_connections 2024-02-02T15:26:11.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.852+0000 7f2a35eb3b80 1 --2- 172.21.15.1:0/2672624038 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f9dd280930 0x55f9dd290460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:11.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.852+0000 7f2a35eb3b80 1 --2- 172.21.15.1:0/2672624038 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f9dd27f130 0x55f9dd288ee0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:11.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.852+0000 7f2a35eb3b80 1 --2- 172.21.15.1:0/2672624038 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55f9dd294180 0x55f9dd293560 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:11.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.852+0000 7f2a35eb3b80 1 --2- 172.21.15.1:0/2672624038 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f9dd27fa90 0x55f9dd289420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:11.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.852+0000 7f2a35eb3b80 1 -- 172.21.15.1:0/2672624038 >> 172.21.15.1:0/2672624038 conn(0x55f9dd1ddd60 msgr2=0x55f9dd279570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:11.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.852+0000 7f2a09600700 0 client.25139 ms_handle_reset on v2:172.21.15.142:6836/2870289019 2024-02-02T15:26:11.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.852+0000 7f2a35eb3b80 1 -- 172.21.15.1:0/2672624038 shutdown_connections 2024-02-02T15:26:11.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.852+0000 7f2a35eb3b80 1 -- 172.21.15.1:0/2672624038 wait complete. 2024-02-02T15:26:11.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.852+0000 7f2a35eb3b80 20 client.25139 trim_cache size 0 max 16384 2024-02-02T15:26:11.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.871+0000 7f2a35eb3b80 1 Processor -- start 2024-02-02T15:26:11.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.871+0000 7f2a35eb3b80 1 -- start start 2024-02-02T15:26:11.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.871+0000 7f2a35eb3b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f9dd27f130 0x55f9dd293560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:11.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.871+0000 7f2a35eb3b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f9dd27fa90 0x55f9dd293aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:11.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.871+0000 7f2a35eb3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f9dd280930 0x55f9dd29ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:11.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.871+0000 7f2a35eb3b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9dd053650 con 0x55f9dd27fa90 2024-02-02T15:26:11.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.871+0000 7f2a35eb3b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9dd28a0d0 con 0x55f9dd27f130 2024-02-02T15:26:11.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.871+0000 7f2a35eb3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9dd05fb60 con 0x55f9dd280930 2024-02-02T15:26:11.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.871+0000 7f2a1aa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f9dd27fa90 0x55f9dd293aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:11.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.871+0000 7f2a1aa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f9dd27fa90 0x55f9dd293aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55384/0 (socket says 172.21.15.1:55384) 2024-02-02T15:26:11.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.871+0000 7f2a1aa00700 1 -- 172.21.15.1:0/59103448 learned_addr learned my addr 172.21.15.1:0/59103448 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:11.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.871+0000 7f2a1b400700 1 --2- 172.21.15.1:0/59103448 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f9dd27f130 0x55f9dd293560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:11.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.871+0000 7f2a1be00700 1 --2- 172.21.15.1:0/59103448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f9dd280930 0x55f9dd29ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:11.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.872+0000 7f2a1aa00700 1 -- 172.21.15.1:0/59103448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f9dd280930 msgr2=0x55f9dd29ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:11.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.872+0000 7f2a1aa00700 1 --2- 172.21.15.1:0/59103448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f9dd280930 0x55f9dd29ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:11.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.872+0000 7f2a1aa00700 1 -- 172.21.15.1:0/59103448 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f9dd27f130 msgr2=0x55f9dd293560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:11.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.872+0000 7f2a1aa00700 1 --2- 172.21.15.1:0/59103448 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f9dd27f130 0x55f9dd293560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:11.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.872+0000 7f2a1aa00700 1 -- 172.21.15.1:0/59103448 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f9dd195e50 con 0x55f9dd27fa90 2024-02-02T15:26:11.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.872+0000 7f2a1b400700 1 --2- 172.21.15.1:0/59103448 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f9dd27f130 0x55f9dd293560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:26:11.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.872+0000 7f2a1aa00700 1 --2- 172.21.15.1:0/59103448 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f9dd27fa90 0x55f9dd293aa0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2a04099070 tx=0x7f2a04010c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f61df1e0f771dca1 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:11.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.872+0000 7f2a11600700 1 -- 172.21.15.1:0/59103448 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a0409d1a0 con 0x55f9dd27fa90 2024-02-02T15:26:11.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.872+0000 7f2a11600700 1 -- 172.21.15.1:0/59103448 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a04097710 con 0x55f9dd27fa90 2024-02-02T15:26:11.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.872+0000 7f2a11600700 1 -- 172.21.15.1:0/59103448 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a040a11b0 con 0x55f9dd27fa90 2024-02-02T15:26:11.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.873+0000 7f2a35eb3b80 1 -- 172.21.15.1:0/59103448 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f9dd27fa90 msgr2=0x55f9dd293aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:11.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.873+0000 7f2a35eb3b80 1 --2- 172.21.15.1:0/59103448 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f9dd27fa90 0x55f9dd293aa0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2a04099070 tx=0x7f2a04010c80 comp rx=0 tx=0).stop 2024-02-02T15:26:11.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.873+0000 7f2a35eb3b80 1 -- 172.21.15.1:0/59103448 shutdown_connections 2024-02-02T15:26:11.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.873+0000 7f2a35eb3b80 1 --2- 172.21.15.1:0/59103448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f9dd280930 0x55f9dd29ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:11.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.873+0000 7f2a35eb3b80 1 --2- 172.21.15.1:0/59103448 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f9dd27f130 0x55f9dd293560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:11.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.873+0000 7f2a35eb3b80 1 --2- 172.21.15.1:0/59103448 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f9dd27fa90 0x55f9dd293aa0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:11.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.873+0000 7f2a35eb3b80 1 -- 172.21.15.1:0/59103448 >> 172.21.15.1:0/59103448 conn(0x55f9dd1ddd60 msgr2=0x55f9dd286dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:11.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.873+0000 7f2a35eb3b80 1 -- 172.21.15.1:0/59103448 shutdown_connections 2024-02-02T15:26:11.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.873+0000 7f2a35eb3b80 1 -- 172.21.15.1:0/59103448 wait complete. 2024-02-02T15:26:11.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.873+0000 7f2a35eb3b80 1 -- start start 2024-02-02T15:26:11.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.874+0000 7f2a35eb3b80 1 Processor -- start 2024-02-02T15:26:11.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.874+0000 7f2a35eb3b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f9dd27f130 0x55f9dd299f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:11.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.874+0000 7f2a35eb3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f9dd280930 0x55f9dd29a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:11.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.874+0000 7f2a35eb3b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f9dd29b460 0x55f9dd29b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:11.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.874+0000 7f2a35eb3b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9dd29be20 con 0x55f9dd27f130 2024-02-02T15:26:11.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.874+0000 7f2a35eb3b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9dd29bf90 con 0x55f9dd29b460 2024-02-02T15:26:11.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.874+0000 7f2a35eb3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9dd29c150 con 0x55f9dd280930 2024-02-02T15:26:11.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.874+0000 7f2a35eb3b80 10 client.0 _ll_register_callbacks cb 0x55f9dd16d240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:26:11.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.874+0000 7f2a1b400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f9dd27f130 0x55f9dd299f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:11.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.874+0000 7f2a1aa00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f9dd280930 0x55f9dd29a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:11.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.874+0000 7f2a1b400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f9dd27f130 0x55f9dd299f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55390/0 (socket says 172.21.15.1:55390) 2024-02-02T15:26:11.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.874+0000 7f2a1b400700 1 -- 172.21.15.1:0/2819268443 learned_addr learned my addr 172.21.15.1:0/2819268443 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:11.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.875+0000 7f2a1be00700 1 --2- 172.21.15.1:0/2819268443 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f9dd29b460 0x55f9dd29b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:11.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.875+0000 7f2a1b400700 1 -- 172.21.15.1:0/2819268443 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f9dd280930 msgr2=0x55f9dd29a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:11.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.875+0000 7f2a1b400700 1 --2- 172.21.15.1:0/2819268443 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f9dd280930 0x55f9dd29a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:11.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.875+0000 7f2a1b400700 1 -- 172.21.15.1:0/2819268443 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f9dd29b460 msgr2=0x55f9dd29b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:11.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.875+0000 7f2a1b400700 1 --2- 172.21.15.1:0/2819268443 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f9dd29b460 0x55f9dd29b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:11.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.875+0000 7f2a1b400700 1 -- 172.21.15.1:0/2819268443 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f9dcffdb10 con 0x55f9dd27f130 2024-02-02T15:26:11.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.875+0000 7f2a1be00700 1 --2- 172.21.15.1:0/2819268443 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f9dd29b460 0x55f9dd29b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:26:11.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.875+0000 7f2a1b400700 1 --2- 172.21.15.1:0/2819268443 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f9dd27f130 0x55f9dd299f10 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c025ff0 tx=0x7f2a0c009320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77d8c1780faa17dd server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:11.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.875+0000 7f2a09600700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:26:11.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.875+0000 7f2a09600700 1 -- 172.21.15.1:0/2819268443 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a0c03a070 con 0x55f9dd27f130 2024-02-02T15:26:11.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.875+0000 7f2a09600700 1 -- 172.21.15.1:0/2819268443 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f2a0c02c350 con 0x55f9dd27f130 2024-02-02T15:26:11.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.875+0000 7f2a35eb3b80 1 -- 172.21.15.1:0/2819268443 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2a040a13e0 con 0x55f9dd27f130 2024-02-02T15:26:11.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.876+0000 7f2a09600700 1 -- 172.21.15.1:0/2819268443 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a0c0053a0 con 0x55f9dd27f130 2024-02-02T15:26:11.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.876+0000 7f2a35eb3b80 10 client.15117 fetch_fsmap learned FSMap version 27 2024-02-02T15:26:11.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.876+0000 7f2a35eb3b80 1 -- 172.21.15.1:0/2819268443 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55f9dd051360 con 0x55f9dd27f130 2024-02-02T15:26:11.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.876+0000 7f2a09600700 1 -- 172.21.15.1:0/2819268443 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a0c047020 con 0x55f9dd27f130 2024-02-02T15:26:11.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.876+0000 7f2a09600700 1 -- 172.21.15.1:0/2819268443 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2a0c00f850 con 0x55f9dd27f130 2024-02-02T15:26:11.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.876+0000 7f2a35eb3b80 10 client.15117 fetch_fsmap finished waiting for FSMap version 27 2024-02-02T15:26:11.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.876+0000 7f2a35eb3b80 10 client.15117 resolve_mds: resolved name 'c' to daemon mds.c 2024-02-02T15:26:11.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.876+0000 7f2a35eb3b80 20 client.15117 populate_metadata read hostname 'smithi001' 2024-02-02T15:26:11.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.876+0000 7f2a35eb3b80 1 --2- 172.21.15.1:0/2819268443 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55f9dd2eb210 0x55f9dd2e7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:11.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.876+0000 7f2a35eb3b80 4 client.15117 mds_command: new command op to 24514 tid=0[{"prefix": "status"}] 2024-02-02T15:26:11.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.876+0000 7f2a35eb3b80 1 -- 172.21.15.1:0/2819268443 --> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f9dd18ef00 con 0x55f9dd2eb210 2024-02-02T15:26:11.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.876+0000 7f2a1aa00700 1 --2- 172.21.15.1:0/2819268443 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55f9dd2eb210 0x55f9dd2e7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:11.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.877+0000 7f2a1aa00700 1 --2- 172.21.15.1:0/2819268443 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55f9dd2eb210 0x55f9dd2e7c40 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=81fe91d9583a849c server_cookie=3f749e8584d52920 in_seq=0 out_seq=0 2024-02-02T15:26:11.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.877+0000 7f2a09600700 10 client.15117 ms_handle_connect on v2:172.21.15.142:6836/2870289019 2024-02-02T15:26:11.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.877+0000 7f2a09600700 1 -- 172.21.15.1:0/2819268443 <== mds.3 v2:172.21.15.142:6836/2870289019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55f9dd18ef00 con 0x55f9dd2eb210 2024-02-02T15:26:11.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.877+0000 7f2a09600700 10 client.15117 handle_command_reply: tid=0 2024-02-02T15:26:11.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.877+0000 7f2a35eb3b80 1 client.15117 shutdown 2024-02-02T15:26:11.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.877+0000 7f2a35eb3b80 1 -- 172.21.15.1:0/2819268443 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f9dd27f130 msgr2=0x55f9dd299f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:11.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.877+0000 7f2a35eb3b80 1 --2- 172.21.15.1:0/2819268443 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f9dd27f130 0x55f9dd299f10 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c025ff0 tx=0x7f2a0c009320 comp rx=0 tx=0).stop 2024-02-02T15:26:11.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.878+0000 7f2a35eb3b80 1 -- 172.21.15.1:0/2819268443 shutdown_connections 2024-02-02T15:26:11.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.878+0000 7f2a35eb3b80 1 --2- 172.21.15.1:0/2819268443 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f9dd29b460 0x55f9dd29b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:11.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.878+0000 7f2a35eb3b80 1 --2- 172.21.15.1:0/2819268443 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55f9dd2eb210 0x55f9dd2e7c40 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:11.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.878+0000 7f2a35eb3b80 1 --2- 172.21.15.1:0/2819268443 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f9dd27f130 0x55f9dd299f10 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:11.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.878+0000 7f2a35eb3b80 1 --2- 172.21.15.1:0/2819268443 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f9dd280930 0x55f9dd29a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:11.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.878+0000 7f2a35eb3b80 1 -- 172.21.15.1:0/2819268443 >> 172.21.15.1:0/2819268443 conn(0x55f9dd1ddd60 msgr2=0x55f9dd293560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:11.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.878+0000 7f2a09600700 0 client.15117 ms_handle_reset on v2:172.21.15.142:6836/2870289019 2024-02-02T15:26:11.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.878+0000 7f2a35eb3b80 1 -- 172.21.15.1:0/2819268443 shutdown_connections 2024-02-02T15:26:11.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.878+0000 7f2a35eb3b80 1 -- 172.21.15.1:0/2819268443 wait complete. 2024-02-02T15:26:11.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.878+0000 7f2a35eb3b80 20 client.15117 trim_cache size 0 max 16384 2024-02-02T15:26:11.885 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:26:11.885 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "371a409c-c1de-11ee-95b6-87774f69a715", 2024-02-02T15:26:11.885 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 3, 2024-02-02T15:26:11.885 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 24514, 2024-02-02T15:26:11.885 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2024-02-02T15:26:11.885 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2024-02-02T15:26:11.885 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2024-02-02T15:26:11.885 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 53.653498710000001, 2024-02-02T15:26:11.885 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 27, 2024-02-02T15:26:11.885 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 86, 2024-02-02T15:26:11.885 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 84, 2024-02-02T15:26:11.885 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 100.98636740000001 2024-02-02T15:26:11.885 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:26:11.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.878+0000 7f2a22400700 1 -- 172.21.15.1:0/3682277949 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f29fc0a0630 msgr2=0x7f29fc0a2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:11.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.878+0000 7f2a22400700 1 --2- 172.21.15.1:0/3682277949 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f29fc0a0630 0x7f29fc0a2af0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2a1400f540 tx=0x7f2a14009920 comp rx=0 tx=0).stop 2024-02-02T15:26:11.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.878+0000 7f2a22400700 1 -- 172.21.15.1:0/3682277949 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a1c101ff0 msgr2=0x7f2a1c199280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:11.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.878+0000 7f2a22400700 1 --2- 172.21.15.1:0/3682277949 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a1c101ff0 0x7f2a1c199280 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2a0400e9a0 tx=0x7f2a0400ecb0 comp rx=0 tx=0).stop 2024-02-02T15:26:11.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.878+0000 7f2a22400700 1 -- 172.21.15.1:0/3682277949 shutdown_connections 2024-02-02T15:26:11.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.879+0000 7f2a22400700 1 --2- 172.21.15.1:0/3682277949 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a1c102990 0x7f2a1c1a02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:11.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.879+0000 7f2a22400700 1 --2- 172.21.15.1:0/3682277949 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f29fc0a0630 0x7f29fc0a2af0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:11.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.879+0000 7f2a22400700 1 --2- 172.21.15.1:0/3682277949 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a1c100e00 0x7f2a1c198d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:11.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.879+0000 7f2a22400700 1 --2- 172.21.15.1:0/3682277949 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a1c101ff0 0x7f2a1c199280 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:11.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.879+0000 7f2a22400700 1 -- 172.21.15.1:0/3682277949 >> 172.21.15.1:0/3682277949 conn(0x7f2a1c0fc370 msgr2=0x7f2a1c107900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:11.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.879+0000 7f2a22400700 1 -- 172.21.15.1:0/3682277949 shutdown_connections 2024-02-02T15:26:11.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:11.879+0000 7f2a22400700 1 -- 172.21.15.1:0/3682277949 wait complete. 2024-02-02T15:26:11.897 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-02-02T15:26:12.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.097+0000 7faf95e00700 1 -- 172.21.15.1:0/1137691081 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf900737a0 msgr2=0x7faf90073c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:12.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.097+0000 7faf95e00700 1 --2- 172.21.15.1:0/1137691081 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf900737a0 0x7faf90073c20 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7faf78009990 tx=0x7faf78009ca0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.097+0000 7faf95e00700 1 -- 172.21.15.1:0/1137691081 shutdown_connections 2024-02-02T15:26:12.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.097+0000 7faf95e00700 1 --2- 172.21.15.1:0/1137691081 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf90108cc0 0x7faf9010b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.097+0000 7faf95e00700 1 --2- 172.21.15.1:0/1137691081 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf90074d80 0x7faf900731d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.097+0000 7faf95e00700 1 --2- 172.21.15.1:0/1137691081 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf900737a0 0x7faf90073c20 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.097+0000 7faf95e00700 1 -- 172.21.15.1:0/1137691081 >> 172.21.15.1:0/1137691081 conn(0x7faf900fc370 msgr2=0x7faf900fe7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:12.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.097+0000 7faf95e00700 1 -- 172.21.15.1:0/1137691081 shutdown_connections 2024-02-02T15:26:12.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.097+0000 7faf95e00700 1 -- 172.21.15.1:0/1137691081 wait complete. 2024-02-02T15:26:12.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.098+0000 7faf95e00700 1 Processor -- start 2024-02-02T15:26:12.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.098+0000 7faf95e00700 1 -- start start 2024-02-02T15:26:12.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.098+0000 7faf95e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf900737a0 0x7faf90198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:12.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.098+0000 7faf95e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf90074d80 0x7faf90199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:12.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.099+0000 7faf95e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf90108cc0 0x7faf901a02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:12.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.099+0000 7faf95e00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf901a0880 con 0x7faf90074d80 2024-02-02T15:26:12.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.099+0000 7faf95e00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf901a09f0 con 0x7faf900737a0 2024-02-02T15:26:12.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.099+0000 7faf95e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf901a0bb0 con 0x7faf90108cc0 2024-02-02T15:26:12.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.099+0000 7faf8e000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf90074d80 0x7faf90199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:12.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.099+0000 7faf8f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf90108cc0 0x7faf901a02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:12.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.099+0000 7faf8e000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf90074d80 0x7faf90199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55416/0 (socket says 172.21.15.1:55416) 2024-02-02T15:26:12.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.099+0000 7faf8e000700 1 -- 172.21.15.1:0/2708765926 learned_addr learned my addr 172.21.15.1:0/2708765926 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:12.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.099+0000 7faf8ea00700 1 --2- 172.21.15.1:0/2708765926 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf900737a0 0x7faf90198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:12.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.099+0000 7faf8e000700 1 -- 172.21.15.1:0/2708765926 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf90108cc0 msgr2=0x7faf901a02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:12.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.099+0000 7faf8e000700 1 --2- 172.21.15.1:0/2708765926 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf90108cc0 0x7faf901a02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.099+0000 7faf8e000700 1 -- 172.21.15.1:0/2708765926 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf900737a0 msgr2=0x7faf90198d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:12.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.099+0000 7faf8e000700 1 --2- 172.21.15.1:0/2708765926 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf900737a0 0x7faf90198d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.099+0000 7faf8e000700 1 -- 172.21.15.1:0/2708765926 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf780095f0 con 0x7faf90074d80 2024-02-02T15:26:12.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.099+0000 7faf8e000700 1 --2- 172.21.15.1:0/2708765926 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf90074d80 0x7faf90199280 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7faf78004710 tx=0x7faf78009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:12.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.100+0000 7faf94a00700 1 -- 172.21.15.1:0/2708765926 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf7801d070 con 0x7faf90074d80 2024-02-02T15:26:12.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.100+0000 7faf94a00700 1 -- 172.21.15.1:0/2708765926 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf78021410 con 0x7faf90074d80 2024-02-02T15:26:12.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.100+0000 7faf95e00700 1 -- 172.21.15.1:0/2708765926 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf901a0e30 con 0x7faf90074d80 2024-02-02T15:26:12.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.100+0000 7faf94a00700 1 -- 172.21.15.1:0/2708765926 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf78017650 con 0x7faf90074d80 2024-02-02T15:26:12.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.100+0000 7faf95e00700 1 -- 172.21.15.1:0/2708765926 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf901a11a0 con 0x7faf90074d80 2024-02-02T15:26:12.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.104+0000 7faf94a00700 1 -- 172.21.15.1:0/2708765926 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) v1 ==== 135120+0+0 (secure 0 0 0) 0x7faf78021580 con 0x7faf90074d80 2024-02-02T15:26:12.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.104+0000 7fafa9991b80 1 Processor -- start 2024-02-02T15:26:12.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.105+0000 7fafa9991b80 1 -- start start 2024-02-02T15:26:12.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.105+0000 7fafa9991b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56034c78e130 0x56034c78e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:12.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.105+0000 7fafa9991b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56034c78ea90 0x56034c78f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:12.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.105+0000 7fafa9991b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56034c78f930 0x56034c78fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:12.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.105+0000 7fafa9991b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56034c701470 con 0x56034c78ea90 2024-02-02T15:26:12.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.105+0000 7fafa9991b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56034c6e5a50 con 0x56034c78f930 2024-02-02T15:26:12.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.105+0000 7fafa9991b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56034c6e3410 con 0x56034c78e130 2024-02-02T15:26:12.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.105+0000 7faf8ea00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56034c78ea90 0x56034c78f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:12.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.105+0000 7faf8ea00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56034c78ea90 0x56034c78f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55418/0 (socket says 172.21.15.1:55418) 2024-02-02T15:26:12.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.105+0000 7faf8ea00700 1 -- 172.21.15.1:0/1759032288 learned_addr learned my addr 172.21.15.1:0/1759032288 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:12.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.105+0000 7faf8f400700 1 --2- 172.21.15.1:0/1759032288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56034c78e130 0x56034c78e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:12.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.105+0000 7faf8e000700 1 --2- 172.21.15.1:0/1759032288 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56034c78f930 0x56034c78fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:12.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.106+0000 7faf8ea00700 1 -- 172.21.15.1:0/1759032288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56034c78e130 msgr2=0x56034c78e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:12.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.106+0000 7faf8ea00700 1 --2- 172.21.15.1:0/1759032288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56034c78e130 0x56034c78e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.106+0000 7faf8ea00700 1 -- 172.21.15.1:0/1759032288 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56034c78f930 msgr2=0x56034c78fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:12.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.106+0000 7faf8ea00700 1 --2- 172.21.15.1:0/1759032288 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56034c78f930 0x56034c78fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.106+0000 7faf8ea00700 1 -- 172.21.15.1:0/1759032288 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56034c560360 con 0x56034c78ea90 2024-02-02T15:26:12.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.106+0000 7faf94a00700 1 --2- 172.21.15.1:0/2708765926 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7faf700a05b0 0x7faf700a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:12.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.106+0000 7faf94a00700 1 -- 172.21.15.1:0/2708765926 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7faf780cd140 con 0x7faf90074d80 2024-02-02T15:26:12.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.106+0000 7faf8ea00700 1 --2- 172.21.15.1:0/1759032288 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56034c78ea90 0x56034c78f3a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7faf80009d20 tx=0x7faf80011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3e68b18d9d6812 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:12.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.106+0000 7faf8f400700 1 --2- 172.21.15.1:0/2708765926 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7faf700a05b0 0x7faf700a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:12.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.106+0000 7faf7fe00700 1 -- 172.21.15.1:0/1759032288 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf80004a30 con 0x56034c78ea90 2024-02-02T15:26:12.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.106+0000 7faf7fe00700 1 -- 172.21.15.1:0/1759032288 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf800054e0 con 0x56034c78ea90 2024-02-02T15:26:12.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.106+0000 7faf7fe00700 1 -- 172.21.15.1:0/1759032288 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf800197a0 con 0x56034c78ea90 2024-02-02T15:26:12.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.107+0000 7fafa9991b80 1 -- 172.21.15.1:0/1759032288 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56034c78ea90 msgr2=0x56034c78f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:12.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.107+0000 7faf8f400700 1 --2- 172.21.15.1:0/2708765926 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7faf700a05b0 0x7faf700a2a70 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7faf88007500 tx=0x7faf8800b040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:12.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.107+0000 7fafa9991b80 1 --2- 172.21.15.1:0/1759032288 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56034c78ea90 0x56034c78f3a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7faf80009d20 tx=0x7faf80011720 comp rx=0 tx=0).stop 2024-02-02T15:26:12.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.107+0000 7fafa9991b80 1 -- 172.21.15.1:0/1759032288 shutdown_connections 2024-02-02T15:26:12.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.107+0000 7fafa9991b80 1 --2- 172.21.15.1:0/1759032288 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56034c78f930 0x56034c78fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.107+0000 7fafa9991b80 1 --2- 172.21.15.1:0/1759032288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56034c78e130 0x56034c78e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.107+0000 7fafa9991b80 1 --2- 172.21.15.1:0/1759032288 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56034c78ea90 0x56034c78f3a0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.107+0000 7fafa9991b80 1 -- 172.21.15.1:0/1759032288 >> 172.21.15.1:0/1759032288 conn(0x56034c6ecd60 msgr2=0x56034c788e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:12.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.107+0000 7fafa9991b80 1 -- 172.21.15.1:0/1759032288 shutdown_connections 2024-02-02T15:26:12.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.107+0000 7fafa9991b80 1 -- 172.21.15.1:0/1759032288 wait complete. 2024-02-02T15:26:12.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.108+0000 7fafa9991b80 1 -- start start 2024-02-02T15:26:12.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.108+0000 7fafa9991b80 1 Processor -- start 2024-02-02T15:26:12.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.109+0000 7fafa9991b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56034c78e130 0x56034c797eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:12.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.109+0000 7fafa9991b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56034c78ea90 0x56034c7983f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:12.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.109+0000 7fafa9991b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56034c78f930 0x56034c79f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:12.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.109+0000 7fafa9991b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56034c6ded00 con 0x56034c78e130 2024-02-02T15:26:12.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.109+0000 7fafa9991b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56034c67ad30 con 0x56034c78ea90 2024-02-02T15:26:12.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.109+0000 7fafa9991b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56034c67a9d0 con 0x56034c78f930 2024-02-02T15:26:12.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.109+0000 7fafa9991b80 10 client.0 _ll_register_callbacks cb 0x56034c67c240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:26:12.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.109+0000 7faf8ea00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56034c78e130 0x56034c797eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:12.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.109+0000 7faf8ea00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56034c78e130 0x56034c797eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55432/0 (socket says 172.21.15.1:55432) 2024-02-02T15:26:12.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.109+0000 7faf8ea00700 1 -- 172.21.15.1:0/2988495470 learned_addr learned my addr 172.21.15.1:0/2988495470 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:12.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.109+0000 7faf8e000700 1 --2- 172.21.15.1:0/2988495470 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56034c78ea90 0x56034c7983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:12.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.109+0000 7faf8f400700 1 --2- 172.21.15.1:0/2988495470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56034c78f930 0x56034c79f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:12.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.109+0000 7faf8ea00700 1 -- 172.21.15.1:0/2988495470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56034c78f930 msgr2=0x56034c79f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:12.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.109+0000 7faf8ea00700 1 --2- 172.21.15.1:0/2988495470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56034c78f930 0x56034c79f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.109+0000 7faf8ea00700 1 -- 172.21.15.1:0/2988495470 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56034c78ea90 msgr2=0x56034c7983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:12.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.109+0000 7faf8ea00700 1 --2- 172.21.15.1:0/2988495470 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56034c78ea90 0x56034c7983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.109+0000 7faf8ea00700 1 -- 172.21.15.1:0/2988495470 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf80009980 con 0x56034c78e130 2024-02-02T15:26:12.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.109+0000 7faf8e000700 1 --2- 172.21.15.1:0/2988495470 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56034c78ea90 0x56034c7983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:26:12.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.110+0000 7faf8ea00700 1 --2- 172.21.15.1:0/2988495470 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56034c78e130 0x56034c797eb0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7faf80005e90 tx=0x7faf80009fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f533c934e0f215f9 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:12.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.110+0000 7faf7cc00700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:26:12.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.110+0000 7faf7cc00700 1 -- 172.21.15.1:0/2988495470 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf80004960 con 0x56034c78e130 2024-02-02T15:26:12.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.110+0000 7faf7cc00700 1 -- 172.21.15.1:0/2988495470 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7faf800194a0 con 0x56034c78e130 2024-02-02T15:26:12.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.110+0000 7fafa9991b80 1 -- 172.21.15.1:0/2988495470 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56034c7810a0 con 0x56034c78e130 2024-02-02T15:26:12.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.110+0000 7faf7cc00700 1 -- 172.21.15.1:0/2988495470 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7faf80004ac0 con 0x56034c78e130 2024-02-02T15:26:12.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.110+0000 7fafa9991b80 10 client.15129 fetch_fsmap learned FSMap version 27 2024-02-02T15:26:12.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.110+0000 7fafa9991b80 1 -- 172.21.15.1:0/2988495470 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56034c50cb10 con 0x56034c78e130 2024-02-02T15:26:12.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.110+0000 7faf7cc00700 1 -- 172.21.15.1:0/2988495470 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56034c7810a0 con 0x56034c78e130 2024-02-02T15:26:12.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.111+0000 7faf7cc00700 1 -- 172.21.15.1:0/2988495470 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7faf800247a0 con 0x56034c78e130 2024-02-02T15:26:12.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.111+0000 7fafa9991b80 10 client.15129 fetch_fsmap finished waiting for FSMap version 27 2024-02-02T15:26:12.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.111+0000 7fafa9991b80 10 client.15129 resolve_mds: resolved name 'h' to daemon mds.h 2024-02-02T15:26:12.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.111+0000 7fafa9991b80 20 client.15129 populate_metadata read hostname 'smithi001' 2024-02-02T15:26:12.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.111+0000 7fafa9991b80 1 --2- 172.21.15.1:0/2988495470 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x56034c7a3020 0x56034c7a2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:12.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.111+0000 7fafa9991b80 4 client.15129 mds_command: new command op to 24518 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:26:12.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.111+0000 7fafa9991b80 1 -- 172.21.15.1:0/2988495470 --> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56034c6d7d80 con 0x56034c7a3020 2024-02-02T15:26:12.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.111+0000 7faf8e000700 1 --2- 172.21.15.1:0/2988495470 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x56034c7a3020 0x56034c7a2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:12.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.112+0000 7faf8e000700 1 --2- 172.21.15.1:0/2988495470 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x56034c7a3020 0x56034c7a2480 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8292fd97764425c6 server_cookie=5d7c1f0689c2b1de in_seq=0 out_seq=0 2024-02-02T15:26:12.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.112+0000 7faf7cc00700 10 client.15129 ms_handle_connect on v2:172.21.15.71:6836/628016199 2024-02-02T15:26:12.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.112+0000 7faf7cc00700 1 -- 172.21.15.1:0/2988495470 <== mds.2 v2:172.21.15.71:6836/628016199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x56034c6d7d80 con 0x56034c7a3020 2024-02-02T15:26:12.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.113+0000 7faf7cc00700 10 client.15129 handle_command_reply: tid=0 2024-02-02T15:26:12.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.113+0000 7fafa9991b80 1 client.15129 shutdown 2024-02-02T15:26:12.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.113+0000 7fafa9991b80 1 -- 172.21.15.1:0/2988495470 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56034c78e130 msgr2=0x56034c797eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:12.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.113+0000 7fafa9991b80 1 --2- 172.21.15.1:0/2988495470 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56034c78e130 0x56034c797eb0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7faf80005e90 tx=0x7faf80009fa0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.113+0000 7fafa9991b80 1 -- 172.21.15.1:0/2988495470 shutdown_connections 2024-02-02T15:26:12.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.113+0000 7fafa9991b80 1 --2- 172.21.15.1:0/2988495470 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x56034c7a3020 0x56034c7a2480 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.113+0000 7fafa9991b80 1 --2- 172.21.15.1:0/2988495470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56034c78f930 0x56034c79f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.113+0000 7fafa9991b80 1 --2- 172.21.15.1:0/2988495470 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56034c78e130 0x56034c797eb0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.113+0000 7fafa9991b80 1 --2- 172.21.15.1:0/2988495470 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56034c78ea90 0x56034c7983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.113+0000 7faf7cc00700 0 client.15129 ms_handle_reset on v2:172.21.15.71:6836/628016199 2024-02-02T15:26:12.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.113+0000 7fafa9991b80 1 -- 172.21.15.1:0/2988495470 >> 172.21.15.1:0/2988495470 conn(0x56034c6ecd60 msgr2=0x56034c788570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:12.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.113+0000 7fafa9991b80 1 -- 172.21.15.1:0/2988495470 shutdown_connections 2024-02-02T15:26:12.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.113+0000 7fafa9991b80 1 -- 172.21.15.1:0/2988495470 wait complete. 2024-02-02T15:26:12.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.113+0000 7fafa9991b80 20 client.15129 trim_cache size 0 max 16384 2024-02-02T15:26:12.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.131+0000 7fafa9991b80 1 Processor -- start 2024-02-02T15:26:12.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.131+0000 7fafa9991b80 1 -- start start 2024-02-02T15:26:12.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.132+0000 7fafa9991b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56034c78e130 0x56034c796c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:12.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.132+0000 7fafa9991b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56034c78ea90 0x56034c7abd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:12.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.132+0000 7fafa9991b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56034c78f930 0x56034c7a2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:12.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.132+0000 7fafa9991b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56034c562650 con 0x56034c78ea90 2024-02-02T15:26:12.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.132+0000 7fafa9991b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56034c6e3d80 con 0x56034c78f930 2024-02-02T15:26:12.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.132+0000 7fafa9991b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56034c56eb60 con 0x56034c78e130 2024-02-02T15:26:12.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.132+0000 7faf8e000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56034c78ea90 0x56034c7abd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:12.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.132+0000 7faf8e000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56034c78ea90 0x56034c7abd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55438/0 (socket says 172.21.15.1:55438) 2024-02-02T15:26:12.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.132+0000 7faf8e000700 1 -- 172.21.15.1:0/474446336 learned_addr learned my addr 172.21.15.1:0/474446336 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:12.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.132+0000 7faf8f400700 1 --2- 172.21.15.1:0/474446336 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56034c78f930 0x56034c7a2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:12.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.132+0000 7faf8ea00700 1 --2- 172.21.15.1:0/474446336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56034c78e130 0x56034c796c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:12.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.133+0000 7faf8e000700 1 -- 172.21.15.1:0/474446336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56034c78e130 msgr2=0x56034c796c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:12.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.133+0000 7faf8e000700 1 --2- 172.21.15.1:0/474446336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56034c78e130 0x56034c796c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.133+0000 7faf8e000700 1 -- 172.21.15.1:0/474446336 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56034c78f930 msgr2=0x56034c7a2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:12.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.133+0000 7faf8e000700 1 --2- 172.21.15.1:0/474446336 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56034c78f930 0x56034c7a2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.133+0000 7faf8e000700 1 -- 172.21.15.1:0/474446336 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56034c6a4e50 con 0x56034c78ea90 2024-02-02T15:26:12.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.133+0000 7faf8e000700 1 --2- 172.21.15.1:0/474446336 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56034c78ea90 0x56034c7abd90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7faf78021830 tx=0x7faf78009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5084ac1baa6d2ba3 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:12.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.133+0000 7faf85200700 1 -- 172.21.15.1:0/474446336 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf7809a440 con 0x56034c78ea90 2024-02-02T15:26:12.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.133+0000 7faf85200700 1 -- 172.21.15.1:0/474446336 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf7809a1b0 con 0x56034c78ea90 2024-02-02T15:26:12.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.133+0000 7faf85200700 1 -- 172.21.15.1:0/474446336 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf7809a8d0 con 0x56034c78ea90 2024-02-02T15:26:12.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.134+0000 7fafa9991b80 1 -- 172.21.15.1:0/474446336 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56034c78ea90 msgr2=0x56034c7abd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:12.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.134+0000 7fafa9991b80 1 --2- 172.21.15.1:0/474446336 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56034c78ea90 0x56034c7abd90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7faf78021830 tx=0x7faf78009ca0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.134+0000 7fafa9991b80 1 -- 172.21.15.1:0/474446336 shutdown_connections 2024-02-02T15:26:12.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.134+0000 7fafa9991b80 1 --2- 172.21.15.1:0/474446336 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56034c78f930 0x56034c7a2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.134+0000 7fafa9991b80 1 --2- 172.21.15.1:0/474446336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56034c78e130 0x56034c796c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.134+0000 7fafa9991b80 1 --2- 172.21.15.1:0/474446336 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56034c78ea90 0x56034c7abd90 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.134+0000 7fafa9991b80 1 -- 172.21.15.1:0/474446336 >> 172.21.15.1:0/474446336 conn(0x56034c6ecd60 msgr2=0x56034c788570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:12.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.134+0000 7fafa9991b80 1 -- 172.21.15.1:0/474446336 shutdown_connections 2024-02-02T15:26:12.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.134+0000 7fafa9991b80 1 -- 172.21.15.1:0/474446336 wait complete. 2024-02-02T15:26:12.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.134+0000 7fafa9991b80 1 -- start start 2024-02-02T15:26:12.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.135+0000 7fafa9991b80 1 Processor -- start 2024-02-02T15:26:12.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.135+0000 7fafa9991b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56034c78e130 0x56034c7a9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:12.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.135+0000 7fafa9991b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56034c78ea90 0x56034c7ab560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:12.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.135+0000 7fafa9991b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56034c78f930 0x56034c7abaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:12.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.135+0000 7fafa9991b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56034c7ac180 con 0x56034c78ea90 2024-02-02T15:26:12.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.135+0000 7fafa9991b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56034c7f4b40 con 0x56034c78f930 2024-02-02T15:26:12.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.135+0000 7fafa9991b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56034c7f4d00 con 0x56034c78e130 2024-02-02T15:26:12.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.135+0000 7fafa9991b80 10 client.0 _ll_register_callbacks cb 0x56034c67c240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:26:12.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.135+0000 7faf8e000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56034c78ea90 0x56034c7ab560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:12.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.136+0000 7faf8e000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56034c78ea90 0x56034c7ab560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55448/0 (socket says 172.21.15.1:55448) 2024-02-02T15:26:12.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.136+0000 7faf8e000700 1 -- 172.21.15.1:0/144331569 learned_addr learned my addr 172.21.15.1:0/144331569 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:12.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.136+0000 7faf8f400700 1 --2- 172.21.15.1:0/144331569 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56034c78f930 0x56034c7abaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:12.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.136+0000 7faf8ea00700 1 --2- 172.21.15.1:0/144331569 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56034c78e130 0x56034c7a9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:12.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.136+0000 7faf8e000700 1 -- 172.21.15.1:0/144331569 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56034c78e130 msgr2=0x56034c7a9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:12.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.136+0000 7faf8e000700 1 --2- 172.21.15.1:0/144331569 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56034c78e130 0x56034c7a9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.136+0000 7faf8e000700 1 -- 172.21.15.1:0/144331569 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56034c78f930 msgr2=0x56034c7abaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:12.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.136+0000 7faf8e000700 1 --2- 172.21.15.1:0/144331569 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56034c78f930 0x56034c7abaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.136+0000 7faf8e000700 1 -- 172.21.15.1:0/144331569 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf901a11a0 con 0x56034c78ea90 2024-02-02T15:26:12.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.136+0000 7faf8ea00700 1 --2- 172.21.15.1:0/144331569 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56034c78e130 0x56034c7a9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:26:12.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.136+0000 7faf8e000700 1 --2- 172.21.15.1:0/144331569 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56034c78ea90 0x56034c7ab560 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7faf780a2120 tx=0x7faf780a27d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f689eb82d765ebe4 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:12.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.136+0000 7faf7cc00700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:26:12.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.136+0000 7faf7cc00700 1 -- 172.21.15.1:0/144331569 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf78099fd0 con 0x56034c78ea90 2024-02-02T15:26:12.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.136+0000 7faf7cc00700 1 -- 172.21.15.1:0/144331569 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7faf7809a670 con 0x56034c78ea90 2024-02-02T15:26:12.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.136+0000 7fafa9991b80 1 -- 172.21.15.1:0/144331569 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faf800054e0 con 0x56034c78ea90 2024-02-02T15:26:12.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.137+0000 7faf7cc00700 1 -- 172.21.15.1:0/144331569 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7faf7809ad70 con 0x56034c78ea90 2024-02-02T15:26:12.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.137+0000 7fafa9991b80 10 client.15135 fetch_fsmap learned FSMap version 27 2024-02-02T15:26:12.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.137+0000 7fafa9991b80 1 -- 172.21.15.1:0/144331569 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56034c564c80 con 0x56034c78ea90 2024-02-02T15:26:12.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.137+0000 7faf7cc00700 1 -- 172.21.15.1:0/144331569 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf800054e0 con 0x56034c78ea90 2024-02-02T15:26:12.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.137+0000 7faf7cc00700 1 -- 172.21.15.1:0/144331569 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7faf78099c80 con 0x56034c78ea90 2024-02-02T15:26:12.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.137+0000 7fafa9991b80 10 client.15135 fetch_fsmap finished waiting for FSMap version 27 2024-02-02T15:26:12.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.137+0000 7fafa9991b80 10 client.15135 resolve_mds: resolved name 'h' to daemon mds.h 2024-02-02T15:26:12.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.137+0000 7fafa9991b80 20 client.15135 populate_metadata read hostname 'smithi001' 2024-02-02T15:26:12.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.137+0000 7fafa9991b80 1 --2- 172.21.15.1:0/144331569 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x56034c7f95a0 0x56034c7f5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:12.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.137+0000 7fafa9991b80 4 client.15135 mds_command: new command op to 24518 tid=0[{"prefix": "status"}] 2024-02-02T15:26:12.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.137+0000 7fafa9991b80 1 -- 172.21.15.1:0/144331569 --> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56034c69df00 con 0x56034c7f95a0 2024-02-02T15:26:12.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.137+0000 7faf8ea00700 1 --2- 172.21.15.1:0/144331569 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x56034c7f95a0 0x56034c7f5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:12.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.138+0000 7faf8ea00700 1 --2- 172.21.15.1:0/144331569 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x56034c7f95a0 0x56034c7f5fc0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bce07c3c7879adeb server_cookie=88c6c128be4de70c in_seq=0 out_seq=0 2024-02-02T15:26:12.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.138+0000 7faf7cc00700 10 client.15135 ms_handle_connect on v2:172.21.15.71:6836/628016199 2024-02-02T15:26:12.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.138+0000 7faf7cc00700 1 -- 172.21.15.1:0/144331569 <== mds.2 v2:172.21.15.71:6836/628016199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x56034c69df00 con 0x56034c7f95a0 2024-02-02T15:26:12.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.138+0000 7faf7cc00700 10 client.15135 handle_command_reply: tid=0 2024-02-02T15:26:12.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.138+0000 7fafa9991b80 1 client.15135 shutdown 2024-02-02T15:26:12.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.138+0000 7fafa9991b80 1 -- 172.21.15.1:0/144331569 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56034c78ea90 msgr2=0x56034c7ab560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:12.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.138+0000 7fafa9991b80 1 --2- 172.21.15.1:0/144331569 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56034c78ea90 0x56034c7ab560 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7faf780a2120 tx=0x7faf780a27d0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.138+0000 7fafa9991b80 1 -- 172.21.15.1:0/144331569 shutdown_connections 2024-02-02T15:26:12.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.138+0000 7fafa9991b80 1 --2- 172.21.15.1:0/144331569 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56034c78f930 0x56034c7abaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.138+0000 7fafa9991b80 1 --2- 172.21.15.1:0/144331569 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x56034c7f95a0 0x56034c7f5fc0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.138+0000 7fafa9991b80 1 --2- 172.21.15.1:0/144331569 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56034c78e130 0x56034c7a9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.138+0000 7faf7cc00700 0 client.15135 ms_handle_reset on v2:172.21.15.71:6836/628016199 2024-02-02T15:26:12.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.138+0000 7fafa9991b80 1 --2- 172.21.15.1:0/144331569 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56034c78ea90 0x56034c7ab560 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.138+0000 7fafa9991b80 1 -- 172.21.15.1:0/144331569 >> 172.21.15.1:0/144331569 conn(0x56034c6ecd60 msgr2=0x56034c796c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:12.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.139+0000 7fafa9991b80 1 -- 172.21.15.1:0/144331569 shutdown_connections 2024-02-02T15:26:12.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.139+0000 7fafa9991b80 1 -- 172.21.15.1:0/144331569 wait complete. 2024-02-02T15:26:12.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.139+0000 7fafa9991b80 20 client.15135 trim_cache size 0 max 16384 2024-02-02T15:26:12.143 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:26:12.143 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "371a409c-c1de-11ee-95b6-87774f69a715", 2024-02-02T15:26:12.144 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 2, 2024-02-02T15:26:12.144 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 24518, 2024-02-02T15:26:12.144 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2024-02-02T15:26:12.144 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2024-02-02T15:26:12.144 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2024-02-02T15:26:12.144 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 54.921130775000002, 2024-02-02T15:26:12.144 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 27, 2024-02-02T15:26:12.144 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 86, 2024-02-02T15:26:12.144 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 84, 2024-02-02T15:26:12.144 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 99.527081835999994 2024-02-02T15:26:12.144 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:26:12.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.139+0000 7faf95e00700 1 -- 172.21.15.1:0/2708765926 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7faf700a05b0 msgr2=0x7faf700a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:12.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.139+0000 7faf95e00700 1 --2- 172.21.15.1:0/2708765926 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7faf700a05b0 0x7faf700a2a70 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7faf88007500 tx=0x7faf8800b040 comp rx=0 tx=0).stop 2024-02-02T15:26:12.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.139+0000 7faf95e00700 1 -- 172.21.15.1:0/2708765926 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf90074d80 msgr2=0x7faf90199280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:12.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.139+0000 7faf95e00700 1 --2- 172.21.15.1:0/2708765926 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf90074d80 0x7faf90199280 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7faf78004710 tx=0x7faf78009580 comp rx=0 tx=0).stop 2024-02-02T15:26:12.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.139+0000 7faf95e00700 1 -- 172.21.15.1:0/2708765926 shutdown_connections 2024-02-02T15:26:12.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.139+0000 7faf95e00700 1 --2- 172.21.15.1:0/2708765926 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7faf700a05b0 0x7faf700a2a70 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.139+0000 7faf95e00700 1 --2- 172.21.15.1:0/2708765926 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf90108cc0 0x7faf901a02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.139+0000 7faf95e00700 1 --2- 172.21.15.1:0/2708765926 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf900737a0 0x7faf90198d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.139+0000 7faf95e00700 1 --2- 172.21.15.1:0/2708765926 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf90074d80 0x7faf90199280 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.139+0000 7faf95e00700 1 -- 172.21.15.1:0/2708765926 >> 172.21.15.1:0/2708765926 conn(0x7faf900fc370 msgr2=0x7faf900fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:12.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.140+0000 7faf95e00700 1 -- 172.21.15.1:0/2708765926 shutdown_connections 2024-02-02T15:26:12.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.140+0000 7faf95e00700 1 -- 172.21.15.1:0/2708765926 wait complete. 2024-02-02T15:26:12.157 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-02-02T15:26:12.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.358+0000 7f6269200700 1 -- 172.21.15.1:0/1785877465 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f62640737a0 msgr2=0x7f6264073c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:12.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.358+0000 7f6269200700 1 --2- 172.21.15.1:0/1785877465 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f62640737a0 0x7f6264073c20 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f624c009990 tx=0x7f624c009ca0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.358+0000 7f6269200700 1 -- 172.21.15.1:0/1785877465 shutdown_connections 2024-02-02T15:26:12.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.358+0000 7f6269200700 1 --2- 172.21.15.1:0/1785877465 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6264108cc0 0x7f626410b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.358+0000 7f6269200700 1 --2- 172.21.15.1:0/1785877465 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6264074d80 0x7f62640731d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.358+0000 7f6269200700 1 --2- 172.21.15.1:0/1785877465 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f62640737a0 0x7f6264073c20 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.358+0000 7f6269200700 1 -- 172.21.15.1:0/1785877465 >> 172.21.15.1:0/1785877465 conn(0x7f62640fc370 msgr2=0x7f62640fe7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:12.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.358+0000 7f6269200700 1 -- 172.21.15.1:0/1785877465 shutdown_connections 2024-02-02T15:26:12.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.358+0000 7f6269200700 1 -- 172.21.15.1:0/1785877465 wait complete. 2024-02-02T15:26:12.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.359+0000 7f6269200700 1 Processor -- start 2024-02-02T15:26:12.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.359+0000 7f6269200700 1 -- start start 2024-02-02T15:26:12.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.359+0000 7f6269200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f62640737a0 0x7f6264198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:12.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.359+0000 7f6269200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6264074d80 0x7f6264199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:12.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.359+0000 7f6269200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6264108cc0 0x7f62641a02c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:12.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.359+0000 7f6269200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62641a0890 con 0x7f62640737a0 2024-02-02T15:26:12.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.359+0000 7f6269200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62641a0a00 con 0x7f6264108cc0 2024-02-02T15:26:12.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.359+0000 7f6269200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62641a0bc0 con 0x7f6264074d80 2024-02-02T15:26:12.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.360+0000 7f6261e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f62640737a0 0x7f6264198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:12.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.360+0000 7f6261e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f62640737a0 0x7f6264198d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55466/0 (socket says 172.21.15.1:55466) 2024-02-02T15:26:12.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.360+0000 7f6261e00700 1 -- 172.21.15.1:0/3835665555 learned_addr learned my addr 172.21.15.1:0/3835665555 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:12.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.360+0000 7f6261400700 1 --2- 172.21.15.1:0/3835665555 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6264074d80 0x7f6264199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:12.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.360+0000 7f6262800700 1 --2- 172.21.15.1:0/3835665555 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6264108cc0 0x7f62641a02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:12.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.360+0000 7f6261e00700 1 -- 172.21.15.1:0/3835665555 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6264074d80 msgr2=0x7f6264199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:12.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.360+0000 7f6261e00700 1 --2- 172.21.15.1:0/3835665555 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6264074d80 0x7f6264199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.360+0000 7f6261e00700 1 -- 172.21.15.1:0/3835665555 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6264108cc0 msgr2=0x7f62641a02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:12.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.360+0000 7f6261e00700 1 --2- 172.21.15.1:0/3835665555 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6264108cc0 0x7f62641a02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.360+0000 7f6261e00700 1 -- 172.21.15.1:0/3835665555 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6254009520 con 0x7f62640737a0 2024-02-02T15:26:12.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.360+0000 7f6261400700 1 --2- 172.21.15.1:0/3835665555 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6264074d80 0x7f6264199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:26:12.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.360+0000 7f6262800700 1 --2- 172.21.15.1:0/3835665555 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6264108cc0 0x7f62641a02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:26:12.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.360+0000 7f6261e00700 1 --2- 172.21.15.1:0/3835665555 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f62640737a0 0x7f6264198d50 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f625400eb30 tx=0x7f625400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:12.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.361+0000 7f625aa00700 1 -- 172.21.15.1:0/3835665555 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6254009880 con 0x7f62640737a0 2024-02-02T15:26:12.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.361+0000 7f6269200700 1 -- 172.21.15.1:0/3835665555 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f624c0095f0 con 0x7f62640737a0 2024-02-02T15:26:12.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.361+0000 7f625aa00700 1 -- 172.21.15.1:0/3835665555 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f62540044e0 con 0x7f62640737a0 2024-02-02T15:26:12.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.361+0000 7f625aa00700 1 -- 172.21.15.1:0/3835665555 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6254003e90 con 0x7f62640737a0 2024-02-02T15:26:12.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.361+0000 7f6269200700 1 -- 172.21.15.1:0/3835665555 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62641a11c0 con 0x7f62640737a0 2024-02-02T15:26:12.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.364+0000 7f625aa00700 1 -- 172.21.15.1:0/3835665555 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) v1 ==== 135120+0+0 (secure 0 0 0) 0x7f6254004000 con 0x7f62640737a0 2024-02-02T15:26:12.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.365+0000 7f627ccacb80 1 Processor -- start 2024-02-02T15:26:12.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.365+0000 7f627ccacb80 1 -- start start 2024-02-02T15:26:12.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.366+0000 7f625aa00700 1 --2- 172.21.15.1:0/3835665555 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f62440a05b0 0x7f62440a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:12.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.366+0000 7f627ccacb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590ba4f1130 0x5590ba4f1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:12.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.366+0000 7f627ccacb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5590ba4f1a90 0x5590ba4f23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:12.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.366+0000 7f625aa00700 1 -- 172.21.15.1:0/3835665555 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f6254014070 con 0x7f62640737a0 2024-02-02T15:26:12.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.366+0000 7f627ccacb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5590ba4f2930 0x5590ba4f2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:12.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.366+0000 7f627ccacb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590ba464470 con 0x5590ba4f1a90 2024-02-02T15:26:12.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.366+0000 7f627ccacb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590ba448a50 con 0x5590ba4f2930 2024-02-02T15:26:12.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.366+0000 7f627ccacb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590ba446410 con 0x5590ba4f1130 2024-02-02T15:26:12.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.366+0000 7f6262800700 1 --2- 172.21.15.1:0/3835665555 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f62440a05b0 0x7f62440a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:12.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.366+0000 7f6261400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5590ba4f1a90 0x5590ba4f23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:12.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.366+0000 7f6261400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5590ba4f1a90 0x5590ba4f23a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55478/0 (socket says 172.21.15.1:55478) 2024-02-02T15:26:12.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.366+0000 7f6261e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590ba4f1130 0x5590ba4f1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:12.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.366+0000 7f6261400700 1 -- 172.21.15.1:0/1609128343 learned_addr learned my addr 172.21.15.1:0/1609128343 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:12.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.366+0000 7f6262800700 1 --2- 172.21.15.1:0/1609128343 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5590ba4f2930 0x5590ba4f2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:12.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.367+0000 7f6261e00700 1 -- 172.21.15.1:0/1609128343 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5590ba4f2930 msgr2=0x5590ba4f2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:12.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.367+0000 7f6261e00700 1 --2- 172.21.15.1:0/1609128343 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5590ba4f2930 0x5590ba4f2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.367+0000 7f6261e00700 1 -- 172.21.15.1:0/1609128343 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5590ba4f1a90 msgr2=0x5590ba4f23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:12.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.367+0000 7f6261e00700 1 --2- 172.21.15.1:0/1609128343 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5590ba4f1a90 0x5590ba4f23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.367+0000 7f6261e00700 1 -- 172.21.15.1:0/1609128343 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5590ba2c3360 con 0x5590ba4f1130 2024-02-02T15:26:12.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.367+0000 7f6262800700 1 --2- 172.21.15.1:0/3835665555 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f62440a05b0 0x7f62440a2a70 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f625c007770 tx=0x7f625c007b30 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:12.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.367+0000 7f6261e00700 1 --2- 172.21.15.1:0/1609128343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590ba4f1130 0x5590ba4f1550 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f62540c98f0 tx=0x7f62540994e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3dc1465440dc5492 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:12.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.367+0000 7f6253400700 1 -- 172.21.15.1:0/1609128343 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6254016330 con 0x5590ba4f1130 2024-02-02T15:26:12.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.367+0000 7f6253400700 1 -- 172.21.15.1:0/1609128343 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6254082e90 con 0x5590ba4f1130 2024-02-02T15:26:12.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.367+0000 7f6253400700 1 -- 172.21.15.1:0/1609128343 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62540a16b0 con 0x5590ba4f1130 2024-02-02T15:26:12.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.368+0000 7f627ccacb80 1 -- 172.21.15.1:0/1609128343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590ba4f1130 msgr2=0x5590ba4f1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:12.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.368+0000 7f627ccacb80 1 --2- 172.21.15.1:0/1609128343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590ba4f1130 0x5590ba4f1550 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f62540c98f0 tx=0x7f62540994e0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.368+0000 7f627ccacb80 1 -- 172.21.15.1:0/1609128343 shutdown_connections 2024-02-02T15:26:12.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.368+0000 7f627ccacb80 1 --2- 172.21.15.1:0/1609128343 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5590ba4f2930 0x5590ba4f2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.368+0000 7f627ccacb80 1 --2- 172.21.15.1:0/1609128343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590ba4f1130 0x5590ba4f1550 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.368+0000 7f627ccacb80 1 --2- 172.21.15.1:0/1609128343 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5590ba4f1a90 0x5590ba4f23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.368+0000 7f627ccacb80 1 -- 172.21.15.1:0/1609128343 >> 172.21.15.1:0/1609128343 conn(0x5590ba44fd60 msgr2=0x5590ba4ebe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:12.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.368+0000 7f627ccacb80 1 -- 172.21.15.1:0/1609128343 shutdown_connections 2024-02-02T15:26:12.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.368+0000 7f627ccacb80 1 -- 172.21.15.1:0/1609128343 wait complete. 2024-02-02T15:26:12.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.368+0000 7f627ccacb80 1 -- start start 2024-02-02T15:26:12.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.369+0000 7f627ccacb80 1 Processor -- start 2024-02-02T15:26:12.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.369+0000 7f627ccacb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5590ba4f1130 0x5590ba4faf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:12.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.369+0000 7f627ccacb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5590ba4f1a90 0x5590ba4fb480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:12.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.369+0000 7f627ccacb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590ba4f2930 0x5590ba5024c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:12.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.369+0000 7f627ccacb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590ba441d00 con 0x5590ba4f1a90 2024-02-02T15:26:12.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.369+0000 7f627ccacb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590ba3ddd30 con 0x5590ba4f1130 2024-02-02T15:26:12.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.369+0000 7f627ccacb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590ba3dd9d0 con 0x5590ba4f2930 2024-02-02T15:26:12.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.369+0000 7f627ccacb80 10 client.0 _ll_register_callbacks cb 0x5590ba3df240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:26:12.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.369+0000 7f6261400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5590ba4f1a90 0x5590ba4fb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:12.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.369+0000 7f6261400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5590ba4f1a90 0x5590ba4fb480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55484/0 (socket says 172.21.15.1:55484) 2024-02-02T15:26:12.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.369+0000 7f6261400700 1 -- 172.21.15.1:0/3290037310 learned_addr learned my addr 172.21.15.1:0/3290037310 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:12.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.369+0000 7f6261e00700 1 --2- 172.21.15.1:0/3290037310 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5590ba4f1130 0x5590ba4faf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:12.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.370+0000 7f6262800700 1 --2- 172.21.15.1:0/3290037310 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590ba4f2930 0x5590ba5024c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:12.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.370+0000 7f6261400700 1 -- 172.21.15.1:0/3290037310 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590ba4f2930 msgr2=0x5590ba5024c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:12.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.370+0000 7f6261400700 1 --2- 172.21.15.1:0/3290037310 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590ba4f2930 0x5590ba5024c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.370+0000 7f6261400700 1 -- 172.21.15.1:0/3290037310 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5590ba4f1130 msgr2=0x5590ba4faf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:12.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.370+0000 7f6261400700 1 --2- 172.21.15.1:0/3290037310 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5590ba4f1130 0x5590ba4faf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.370+0000 7f6261400700 1 -- 172.21.15.1:0/3290037310 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62641a11c0 con 0x5590ba4f1a90 2024-02-02T15:26:12.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.370+0000 7f6261e00700 1 --2- 172.21.15.1:0/3290037310 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5590ba4f1130 0x5590ba4faf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:26:12.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.370+0000 7f6261400700 1 --2- 172.21.15.1:0/3290037310 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5590ba4f1a90 0x5590ba4fb480 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6264074aa0 tx=0x7f624c0053b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b2100063a8eb974 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:12.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.370+0000 7f624be00700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:26:12.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.370+0000 7f624be00700 1 -- 172.21.15.1:0/3290037310 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f624c01d070 con 0x5590ba4f1a90 2024-02-02T15:26:12.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.370+0000 7f624be00700 1 -- 172.21.15.1:0/3290037310 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f624c0049d0 con 0x5590ba4f1a90 2024-02-02T15:26:12.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.370+0000 7f627ccacb80 1 -- 172.21.15.1:0/3290037310 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5590ba4e40a0 con 0x5590ba4f1a90 2024-02-02T15:26:12.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.371+0000 7f624be00700 1 -- 172.21.15.1:0/3290037310 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f624c004b90 con 0x5590ba4f1a90 2024-02-02T15:26:12.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.371+0000 7f627ccacb80 10 client.15147 fetch_fsmap learned FSMap version 27 2024-02-02T15:26:12.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.371+0000 7f627ccacb80 1 -- 172.21.15.1:0/3290037310 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5590ba26fb10 con 0x5590ba4f1a90 2024-02-02T15:26:12.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.371+0000 7f624be00700 1 -- 172.21.15.1:0/3290037310 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5590ba4e40a0 con 0x5590ba4f1a90 2024-02-02T15:26:12.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.371+0000 7f624be00700 1 -- 172.21.15.1:0/3290037310 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f624c024450 con 0x5590ba4f1a90 2024-02-02T15:26:12.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.371+0000 7f627ccacb80 10 client.15147 fetch_fsmap finished waiting for FSMap version 27 2024-02-02T15:26:12.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.371+0000 7f627ccacb80 10 client.15147 resolve_mds: resolved name 'i' to daemon mds.i 2024-02-02T15:26:12.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.371+0000 7f627ccacb80 20 client.15147 populate_metadata read hostname 'smithi001' 2024-02-02T15:26:12.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.371+0000 7f627ccacb80 1 --2- 172.21.15.1:0/3290037310 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x5590ba506190 0x5590ba5055c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:12.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.371+0000 7f627ccacb80 4 client.15147 mds_command: new command op to 24529 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:26:12.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.371+0000 7f627ccacb80 1 -- 172.21.15.1:0/3290037310 --> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5590ba43ad80 con 0x5590ba506190 2024-02-02T15:26:12.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.372+0000 7f6261e00700 1 --2- 172.21.15.1:0/3290037310 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x5590ba506190 0x5590ba5055c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:12.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.372+0000 7f6261e00700 1 --2- 172.21.15.1:0/3290037310 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x5590ba506190 0x5590ba5055c0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=626a57c8ddaaa6a6 server_cookie=c7f42a223e26f16b in_seq=0 out_seq=0 2024-02-02T15:26:12.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.372+0000 7f624be00700 10 client.15147 ms_handle_connect on v2:172.21.15.142:6838/321675421 2024-02-02T15:26:12.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.373+0000 7f624be00700 1 -- 172.21.15.1:0/3290037310 <== mds.1 v2:172.21.15.142:6838/321675421 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5590ba43ad80 con 0x5590ba506190 2024-02-02T15:26:12.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.373+0000 7f624be00700 10 client.15147 handle_command_reply: tid=0 2024-02-02T15:26:12.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.373+0000 7f627ccacb80 1 client.15147 shutdown 2024-02-02T15:26:12.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.373+0000 7f627ccacb80 1 -- 172.21.15.1:0/3290037310 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5590ba4f1a90 msgr2=0x5590ba4fb480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:12.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.373+0000 7f627ccacb80 1 --2- 172.21.15.1:0/3290037310 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5590ba4f1a90 0x5590ba4fb480 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6264074aa0 tx=0x7f624c0053b0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.373+0000 7f627ccacb80 1 -- 172.21.15.1:0/3290037310 shutdown_connections 2024-02-02T15:26:12.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.373+0000 7f627ccacb80 1 --2- 172.21.15.1:0/3290037310 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x5590ba506190 0x5590ba5055c0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.373+0000 7f627ccacb80 1 --2- 172.21.15.1:0/3290037310 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590ba4f2930 0x5590ba5024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.373+0000 7f627ccacb80 1 --2- 172.21.15.1:0/3290037310 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5590ba4f1130 0x5590ba4faf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.373+0000 7f627ccacb80 1 --2- 172.21.15.1:0/3290037310 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5590ba4f1a90 0x5590ba4fb480 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.373+0000 7f627ccacb80 1 -- 172.21.15.1:0/3290037310 >> 172.21.15.1:0/3290037310 conn(0x5590ba44fd60 msgr2=0x5590ba4eb570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:12.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.373+0000 7f624be00700 0 client.15147 ms_handle_reset on v2:172.21.15.142:6838/321675421 2024-02-02T15:26:12.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.373+0000 7f627ccacb80 1 -- 172.21.15.1:0/3290037310 shutdown_connections 2024-02-02T15:26:12.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.373+0000 7f627ccacb80 1 -- 172.21.15.1:0/3290037310 wait complete. 2024-02-02T15:26:12.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.373+0000 7f627ccacb80 20 client.15147 trim_cache size 0 max 16384 2024-02-02T15:26:12.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.391+0000 7f627ccacb80 1 Processor -- start 2024-02-02T15:26:12.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.391+0000 7f627ccacb80 1 -- start start 2024-02-02T15:26:12.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.392+0000 7f627ccacb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5590ba4f1130 0x5590ba4f9c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:12.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.392+0000 7f627ccacb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5590ba4f1a90 0x5590ba50ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:12.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.392+0000 7f627ccacb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590ba4f2930 0x5590ba4eb570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:12.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.392+0000 7f627ccacb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590ba2c5650 con 0x5590ba4f1a90 2024-02-02T15:26:12.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.392+0000 7f627ccacb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590ba2d1b60 con 0x5590ba4f1130 2024-02-02T15:26:12.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.392+0000 7f627ccacb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590ba3ffad0 con 0x5590ba4f2930 2024-02-02T15:26:12.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.392+0000 7f6261400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5590ba4f1a90 0x5590ba50ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:12.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.392+0000 7f6262800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590ba4f2930 0x5590ba4eb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:12.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.392+0000 7f6262800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590ba4f2930 0x5590ba4eb570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:57420/0 (socket says 172.21.15.1:57420) 2024-02-02T15:26:12.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.392+0000 7f6262800700 1 -- 172.21.15.1:0/788111809 learned_addr learned my addr 172.21.15.1:0/788111809 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:12.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.392+0000 7f6261400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5590ba4f1a90 0x5590ba50ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55492/0 (socket says 172.21.15.1:55492) 2024-02-02T15:26:12.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.392+0000 7f6261e00700 1 --2- 172.21.15.1:0/788111809 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5590ba4f1130 0x5590ba4f9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:12.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.392+0000 7f6261400700 1 -- 172.21.15.1:0/788111809 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590ba4f2930 msgr2=0x5590ba4eb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:12.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.392+0000 7f6261400700 1 --2- 172.21.15.1:0/788111809 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590ba4f2930 0x5590ba4eb570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.392+0000 7f6261400700 1 -- 172.21.15.1:0/788111809 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5590ba4f1130 msgr2=0x5590ba4f9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:12.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.392+0000 7f6261400700 1 --2- 172.21.15.1:0/788111809 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5590ba4f1130 0x5590ba4f9c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.392+0000 7f6261400700 1 -- 172.21.15.1:0/788111809 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5590ba407e50 con 0x5590ba4f1a90 2024-02-02T15:26:12.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.392+0000 7f6262800700 1 --2- 172.21.15.1:0/788111809 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590ba4f2930 0x5590ba4eb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:26:12.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.393+0000 7f6261e00700 1 --2- 172.21.15.1:0/788111809 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5590ba4f1130 0x5590ba4f9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:26:12.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.393+0000 7f6261400700 1 --2- 172.21.15.1:0/788111809 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5590ba4f1a90 0x5590ba50ef00 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f624c0229b0 tx=0x7f624c009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4df65cf21ac80a85 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:12.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.393+0000 7f6253e00700 1 -- 172.21.15.1:0/788111809 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f624c01d040 con 0x5590ba4f1a90 2024-02-02T15:26:12.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.393+0000 7f6253e00700 1 -- 172.21.15.1:0/788111809 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f624c01e040 con 0x5590ba4f1a90 2024-02-02T15:26:12.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.393+0000 7f6253e00700 1 -- 172.21.15.1:0/788111809 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f624c022c40 con 0x5590ba4f1a90 2024-02-02T15:26:12.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.394+0000 7f627ccacb80 1 -- 172.21.15.1:0/788111809 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5590ba4f1a90 msgr2=0x5590ba50ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:12.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.394+0000 7f627ccacb80 1 --2- 172.21.15.1:0/788111809 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5590ba4f1a90 0x5590ba50ef00 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f624c0229b0 tx=0x7f624c009ca0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.394+0000 7f627ccacb80 1 -- 172.21.15.1:0/788111809 shutdown_connections 2024-02-02T15:26:12.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.394+0000 7f627ccacb80 1 --2- 172.21.15.1:0/788111809 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590ba4f2930 0x5590ba4eb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.394+0000 7f627ccacb80 1 --2- 172.21.15.1:0/788111809 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5590ba4f1130 0x5590ba4f9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.394+0000 7f627ccacb80 1 --2- 172.21.15.1:0/788111809 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5590ba4f1a90 0x5590ba50ef00 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.394+0000 7f627ccacb80 1 -- 172.21.15.1:0/788111809 >> 172.21.15.1:0/788111809 conn(0x5590ba44fd60 msgr2=0x5590ba4e7db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:12.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.394+0000 7f627ccacb80 1 -- 172.21.15.1:0/788111809 shutdown_connections 2024-02-02T15:26:12.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.394+0000 7f627ccacb80 1 -- 172.21.15.1:0/788111809 wait complete. 2024-02-02T15:26:12.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.394+0000 7f627ccacb80 1 -- start start 2024-02-02T15:26:12.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.394+0000 7f627ccacb80 1 Processor -- start 2024-02-02T15:26:12.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.395+0000 7f627ccacb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590ba4f1130 0x5590ba50bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:12.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.395+0000 7f627ccacb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5590ba4f1a90 0x5590ba50e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:12.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.395+0000 7f627ccacb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5590ba4f2930 0x5590ba50e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:12.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.395+0000 7f627ccacb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590ba50ef90 con 0x5590ba4f2930 2024-02-02T15:26:12.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.395+0000 7f627ccacb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590ba50f100 con 0x5590ba4f1a90 2024-02-02T15:26:12.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.395+0000 7f627ccacb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590ba50f2c0 con 0x5590ba4f1130 2024-02-02T15:26:12.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.395+0000 7f627ccacb80 10 client.0 _ll_register_callbacks cb 0x5590ba3df240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:26:12.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.395+0000 7f6261400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5590ba4f1a90 0x5590ba50e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:12.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.395+0000 7f6261400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5590ba4f1a90 0x5590ba50e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:56642/0 (socket says 172.21.15.1:56642) 2024-02-02T15:26:12.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.395+0000 7f6261400700 1 -- 172.21.15.1:0/2549713109 learned_addr learned my addr 172.21.15.1:0/2549713109 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:12.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.395+0000 7f6261e00700 1 --2- 172.21.15.1:0/2549713109 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590ba4f1130 0x5590ba50bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:12.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.395+0000 7f6262800700 1 --2- 172.21.15.1:0/2549713109 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5590ba4f2930 0x5590ba50e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:12.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.396+0000 7f6261400700 1 -- 172.21.15.1:0/2549713109 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590ba4f1130 msgr2=0x5590ba50bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:12.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.396+0000 7f6261400700 1 --2- 172.21.15.1:0/2549713109 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590ba4f1130 0x5590ba50bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.396+0000 7f6261400700 1 -- 172.21.15.1:0/2549713109 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5590ba4f2930 msgr2=0x5590ba50e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:12.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.396+0000 7f6261400700 1 --2- 172.21.15.1:0/2549713109 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5590ba4f2930 0x5590ba50e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.396+0000 7f6261400700 1 -- 172.21.15.1:0/2549713109 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5590ba2c3360 con 0x5590ba4f1a90 2024-02-02T15:26:12.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.396+0000 7f6261e00700 1 --2- 172.21.15.1:0/2549713109 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590ba4f1130 0x5590ba50bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:26:12.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.396+0000 7f6262800700 1 --2- 172.21.15.1:0/2549713109 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5590ba4f2930 0x5590ba50e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:26:12.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.396+0000 7f6261400700 1 --2- 172.21.15.1:0/2549713109 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5590ba4f1a90 0x5590ba50e480 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f624c005b20 tx=0x7f624c009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbcb61489b96f2b server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:12.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.396+0000 7f624be00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:26:12.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.396+0000 7f624be00700 1 -- 172.21.15.1:0/2549713109 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f624c024a50 con 0x5590ba4f1a90 2024-02-02T15:26:12.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.396+0000 7f627ccacb80 1 -- 172.21.15.1:0/2549713109 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6254082e90 con 0x5590ba4f1a90 2024-02-02T15:26:12.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.396+0000 7f624be00700 1 -- 172.21.15.1:0/2549713109 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f624c000e40 con 0x5590ba4f1a90 2024-02-02T15:26:12.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.396+0000 7f624be00700 1 -- 172.21.15.1:0/2549713109 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f624c022bf0 con 0x5590ba4f1a90 2024-02-02T15:26:12.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.396+0000 7f627ccacb80 10 client.25181 fetch_fsmap learned FSMap version 27 2024-02-02T15:26:12.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.396+0000 7f627ccacb80 1 -- 172.21.15.1:0/2549713109 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f625c01a4f0 con 0x5590ba4f1a90 2024-02-02T15:26:12.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.397+0000 7f624be00700 1 -- 172.21.15.1:0/2549713109 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6254082e90 con 0x5590ba4f1a90 2024-02-02T15:26:12.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.397+0000 7f624be00700 1 -- 172.21.15.1:0/2549713109 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f624c0099c0 con 0x5590ba4f1a90 2024-02-02T15:26:12.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.397+0000 7f627ccacb80 10 client.25181 fetch_fsmap finished waiting for FSMap version 27 2024-02-02T15:26:12.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.397+0000 7f627ccacb80 10 client.25181 resolve_mds: resolved name 'i' to daemon mds.i 2024-02-02T15:26:12.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.397+0000 7f627ccacb80 20 client.25181 populate_metadata read hostname 'smithi001' 2024-02-02T15:26:12.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.397+0000 7f627ccacb80 1 --2- 172.21.15.1:0/2549713109 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x5590ba55c410 0x5590ba558d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:12.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.397+0000 7f627ccacb80 4 client.25181 mds_command: new command op to 24529 tid=0[{"prefix": "status"}] 2024-02-02T15:26:12.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.397+0000 7f627ccacb80 1 -- 172.21.15.1:0/2549713109 --> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5590ba400f00 con 0x5590ba55c410 2024-02-02T15:26:12.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.397+0000 7f6261e00700 1 --2- 172.21.15.1:0/2549713109 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x5590ba55c410 0x5590ba558d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:12.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.398+0000 7f6261e00700 1 --2- 172.21.15.1:0/2549713109 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x5590ba55c410 0x5590ba558d60 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=927eac52aace6928 server_cookie=6340fbc1f19e787c in_seq=0 out_seq=0 2024-02-02T15:26:12.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.398+0000 7f624be00700 10 client.25181 ms_handle_connect on v2:172.21.15.142:6838/321675421 2024-02-02T15:26:12.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.398+0000 7f624be00700 1 -- 172.21.15.1:0/2549713109 <== mds.1 v2:172.21.15.142:6838/321675421 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x5590ba400f00 con 0x5590ba55c410 2024-02-02T15:26:12.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.398+0000 7f624be00700 10 client.25181 handle_command_reply: tid=0 2024-02-02T15:26:12.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.398+0000 7f627ccacb80 1 client.25181 shutdown 2024-02-02T15:26:12.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.398+0000 7f627ccacb80 1 -- 172.21.15.1:0/2549713109 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5590ba4f1a90 msgr2=0x5590ba50e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:12.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.398+0000 7f627ccacb80 1 --2- 172.21.15.1:0/2549713109 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5590ba4f1a90 0x5590ba50e480 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f624c005b20 tx=0x7f624c009580 comp rx=0 tx=0).stop 2024-02-02T15:26:12.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.398+0000 7f627ccacb80 1 -- 172.21.15.1:0/2549713109 shutdown_connections 2024-02-02T15:26:12.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.398+0000 7f627ccacb80 1 --2- 172.21.15.1:0/2549713109 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x5590ba55c410 0x5590ba558d60 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.398+0000 7f627ccacb80 1 --2- 172.21.15.1:0/2549713109 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5590ba4f2930 0x5590ba50e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.398+0000 7f627ccacb80 1 --2- 172.21.15.1:0/2549713109 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5590ba4f1130 0x5590ba50bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.398+0000 7f624be00700 0 client.25181 ms_handle_reset on v2:172.21.15.142:6838/321675421 2024-02-02T15:26:12.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.398+0000 7f627ccacb80 1 --2- 172.21.15.1:0/2549713109 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5590ba4f1a90 0x5590ba50e480 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.398+0000 7f627ccacb80 1 -- 172.21.15.1:0/2549713109 >> 172.21.15.1:0/2549713109 conn(0x5590ba44fd60 msgr2=0x5590ba4f9c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:12.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.399+0000 7f627ccacb80 1 -- 172.21.15.1:0/2549713109 shutdown_connections 2024-02-02T15:26:12.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.399+0000 7f627ccacb80 1 -- 172.21.15.1:0/2549713109 wait complete. 2024-02-02T15:26:12.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.399+0000 7f627ccacb80 20 client.25181 trim_cache size 0 max 16384 2024-02-02T15:26:12.404 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:26:12.404 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "371a409c-c1de-11ee-95b6-87774f69a715", 2024-02-02T15:26:12.404 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 1, 2024-02-02T15:26:12.404 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 24529, 2024-02-02T15:26:12.404 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2024-02-02T15:26:12.404 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2024-02-02T15:26:12.404 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2024-02-02T15:26:12.404 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 56.191393179999999, 2024-02-02T15:26:12.405 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 27, 2024-02-02T15:26:12.405 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 86, 2024-02-02T15:26:12.405 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 84, 2024-02-02T15:26:12.405 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 97.882186262999994 2024-02-02T15:26:12.405 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:26:12.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.399+0000 7f6269200700 1 -- 172.21.15.1:0/3835665555 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f62440a05b0 msgr2=0x7f62440a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:12.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.399+0000 7f6269200700 1 --2- 172.21.15.1:0/3835665555 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f62440a05b0 0x7f62440a2a70 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f625c007770 tx=0x7f625c007b30 comp rx=0 tx=0).stop 2024-02-02T15:26:12.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.399+0000 7f6269200700 1 -- 172.21.15.1:0/3835665555 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f62640737a0 msgr2=0x7f6264198d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:12.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.399+0000 7f6269200700 1 --2- 172.21.15.1:0/3835665555 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f62640737a0 0x7f6264198d50 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f625400eb30 tx=0x7f625400ee40 comp rx=0 tx=0).stop 2024-02-02T15:26:12.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.399+0000 7f6269200700 1 -- 172.21.15.1:0/3835665555 shutdown_connections 2024-02-02T15:26:12.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.399+0000 7f6269200700 1 --2- 172.21.15.1:0/3835665555 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6264108cc0 0x7f62641a02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.399+0000 7f6269200700 1 --2- 172.21.15.1:0/3835665555 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f62640737a0 0x7f6264198d50 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.399+0000 7f6269200700 1 --2- 172.21.15.1:0/3835665555 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f62440a05b0 0x7f62440a2a70 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.399+0000 7f6269200700 1 --2- 172.21.15.1:0/3835665555 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6264074d80 0x7f6264199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.399+0000 7f6269200700 1 -- 172.21.15.1:0/3835665555 >> 172.21.15.1:0/3835665555 conn(0x7f62640fc370 msgr2=0x7f62640fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:12.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.399+0000 7f6269200700 1 -- 172.21.15.1:0/3835665555 shutdown_connections 2024-02-02T15:26:12.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.400+0000 7f6269200700 1 -- 172.21.15.1:0/3835665555 wait complete. 2024-02-02T15:26:12.415 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:12 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/1659083492' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:26:12.415 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:12 smithi001 ceph-mon[35470]: pgmap v279: 129 pgs: 129 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-02-02T15:26:12.417 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-02-02T15:26:12.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:12 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/1659083492' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:26:12.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:12 smithi071 ceph-mon[44707]: pgmap v279: 129 pgs: 129 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-02-02T15:26:12.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.615+0000 7f5aa9c00700 1 -- 172.21.15.1:0/3123288114 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5aa4101ff0 msgr2=0x7f5aa4102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:12.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.615+0000 7f5aa9c00700 1 --2- 172.21.15.1:0/3123288114 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5aa4101ff0 0x7f5aa4102450 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c0099c0 tx=0x7f5a8c009cd0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.615+0000 7f5aa9c00700 1 -- 172.21.15.1:0/3123288114 shutdown_connections 2024-02-02T15:26:12.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.615+0000 7f5aa9c00700 1 --2- 172.21.15.1:0/3123288114 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5aa4102990 0x7f5aa4109240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.615+0000 7f5aa9c00700 1 --2- 172.21.15.1:0/3123288114 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5aa4100e00 0x7f5aa4101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.615+0000 7f5aa9c00700 1 --2- 172.21.15.1:0/3123288114 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5aa4101ff0 0x7f5aa4102450 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.615+0000 7f5aa9c00700 1 -- 172.21.15.1:0/3123288114 >> 172.21.15.1:0/3123288114 conn(0x7f5aa40fc370 msgr2=0x7f5aa40fe7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:12.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.615+0000 7f5aa9c00700 1 -- 172.21.15.1:0/3123288114 shutdown_connections 2024-02-02T15:26:12.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.616+0000 7f5aa9c00700 1 -- 172.21.15.1:0/3123288114 wait complete. 2024-02-02T15:26:12.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.617+0000 7f5aa9c00700 1 Processor -- start 2024-02-02T15:26:12.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.617+0000 7f5aa9c00700 1 -- start start 2024-02-02T15:26:12.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.617+0000 7f5aa9c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5aa4100e00 0x7f5aa4198d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:12.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.617+0000 7f5aa9c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5aa4101ff0 0x7f5aa4199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:12.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.617+0000 7f5aa9c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5aa4102990 0x7f5aa41a02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:12.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.617+0000 7f5aa9c00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa41a08c0 con 0x7f5aa4102990 2024-02-02T15:26:12.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.617+0000 7f5aa9c00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa41a0a00 con 0x7f5aa4101ff0 2024-02-02T15:26:12.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.617+0000 7f5aa9c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa41a0b90 con 0x7f5aa4100e00 2024-02-02T15:26:12.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.617+0000 7f5aa3400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5aa4102990 0x7f5aa41a02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:12.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.617+0000 7f5aa3400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5aa4102990 0x7f5aa41a02f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55528/0 (socket says 172.21.15.1:55528) 2024-02-02T15:26:12.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.617+0000 7f5aa3400700 1 -- 172.21.15.1:0/1282482640 learned_addr learned my addr 172.21.15.1:0/1282482640 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:12.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.617+0000 7f5aa2a00700 1 --2- 172.21.15.1:0/1282482640 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5aa4100e00 0x7f5aa4198d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:12.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.618+0000 7f5aa2000700 1 --2- 172.21.15.1:0/1282482640 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5aa4101ff0 0x7f5aa4199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:12.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.618+0000 7f5aa3400700 1 -- 172.21.15.1:0/1282482640 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5aa4100e00 msgr2=0x7f5aa4198d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:12.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.618+0000 7f5aa3400700 1 --2- 172.21.15.1:0/1282482640 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5aa4100e00 0x7f5aa4198d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.618+0000 7f5aa3400700 1 -- 172.21.15.1:0/1282482640 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5aa4101ff0 msgr2=0x7f5aa4199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:12.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.618+0000 7f5aa3400700 1 --2- 172.21.15.1:0/1282482640 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5aa4101ff0 0x7f5aa4199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.618+0000 7f5aa3400700 1 -- 172.21.15.1:0/1282482640 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a8c0095f0 con 0x7f5aa4102990 2024-02-02T15:26:12.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.618+0000 7f5aa2000700 1 --2- 172.21.15.1:0/1282482640 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5aa4101ff0 0x7f5aa4199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:26:12.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.618+0000 7f5aa3400700 1 --2- 172.21.15.1:0/1282482640 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5aa4102990 0x7f5aa41a02f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c007ee0 tx=0x7f5a9c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:12.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.618+0000 7f5a9b400700 1 -- 172.21.15.1:0/1282482640 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9c00ddd0 con 0x7f5aa4102990 2024-02-02T15:26:12.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.618+0000 7f5a9b400700 1 -- 172.21.15.1:0/1282482640 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a9c00f040 con 0x7f5aa4102990 2024-02-02T15:26:12.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.618+0000 7f5aa9c00700 1 -- 172.21.15.1:0/1282482640 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5aa41a0e70 con 0x7f5aa4102990 2024-02-02T15:26:12.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.618+0000 7f5a9b400700 1 -- 172.21.15.1:0/1282482640 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9c0048e0 con 0x7f5aa4102990 2024-02-02T15:26:12.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.618+0000 7f5aa9c00700 1 -- 172.21.15.1:0/1282482640 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5aa41a1340 con 0x7f5aa4102990 2024-02-02T15:26:12.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.621+0000 7f5a9b400700 1 -- 172.21.15.1:0/1282482640 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) v1 ==== 135120+0+0 (secure 0 0 0) 0x7f5a9c003a00 con 0x7f5aa4102990 2024-02-02T15:26:12.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.623+0000 7f5a9b400700 1 --2- 172.21.15.1:0/1282482640 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f5a840a0630 0x7f5a840a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:12.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.623+0000 7f5abd6ccb80 1 Processor -- start 2024-02-02T15:26:12.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.623+0000 7f5a9b400700 1 -- 172.21.15.1:0/1282482640 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f5a9c0cb710 con 0x7f5aa4102990 2024-02-02T15:26:12.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.623+0000 7f5abd6ccb80 1 -- start start 2024-02-02T15:26:12.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.623+0000 7f5aa2a00700 1 --2- 172.21.15.1:0/1282482640 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f5a840a0630 0x7f5a840a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:12.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.623+0000 7f5abd6ccb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55792fce6130 0x55792fce6550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:12.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.623+0000 7f5abd6ccb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55792fce6a90 0x55792fce73a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:12.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.623+0000 7f5abd6ccb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55792fce7930 0x55792fce7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:12.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.623+0000 7f5abd6ccb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55792fc59470 con 0x55792fce6130 2024-02-02T15:26:12.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.623+0000 7f5abd6ccb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55792fc3da50 con 0x55792fce7930 2024-02-02T15:26:12.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.623+0000 7f5abd6ccb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55792fc3b410 con 0x55792fce6a90 2024-02-02T15:26:12.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.624+0000 7f5aa2a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55792fce6130 0x55792fce6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:12.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.624+0000 7f5aa3400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55792fce7930 0x55792fce7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:12.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.624+0000 7f5aa2a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55792fce6130 0x55792fce6550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55542/0 (socket says 172.21.15.1:55542) 2024-02-02T15:26:12.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.624+0000 7f5aa2a00700 1 -- 172.21.15.1:0/1756109379 learned_addr learned my addr 172.21.15.1:0/1756109379 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:12.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.624+0000 7f5aa2000700 1 --2- 172.21.15.1:0/1756109379 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55792fce6a90 0x55792fce73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:12.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.624+0000 7f5aa2a00700 1 --2- 172.21.15.1:0/1282482640 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f5a840a0630 0x7f5a840a2af0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5aa4101e50 tx=0x7f5a940106b0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:12.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.624+0000 7f5aa3400700 1 -- 172.21.15.1:0/1756109379 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55792fce6a90 msgr2=0x55792fce73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:12.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.624+0000 7f5aa3400700 1 --2- 172.21.15.1:0/1756109379 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55792fce6a90 0x55792fce73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.624+0000 7f5aa3400700 1 -- 172.21.15.1:0/1756109379 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55792fce6130 msgr2=0x55792fce6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:12.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.624+0000 7f5aa3400700 1 --2- 172.21.15.1:0/1756109379 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55792fce6130 0x55792fce6550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.624+0000 7f5aa3400700 1 -- 172.21.15.1:0/1756109379 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55792fab8360 con 0x55792fce7930 2024-02-02T15:26:12.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.624+0000 7f5aa3400700 1 --2- 172.21.15.1:0/1756109379 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55792fce7930 0x55792fce7dc0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c0c8d50 tx=0x7f5a9c00dfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=abf77d655ab7ed6d server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:12.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.625+0000 7f5a93e00700 1 -- 172.21.15.1:0/1756109379 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9c0991a0 con 0x55792fce7930 2024-02-02T15:26:12.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.625+0000 7f5a93e00700 1 -- 172.21.15.1:0/1756109379 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a9c098aa0 con 0x55792fce7930 2024-02-02T15:26:12.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.625+0000 7f5a93e00700 1 -- 172.21.15.1:0/1756109379 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9c003cb0 con 0x55792fce7930 2024-02-02T15:26:12.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.625+0000 7f5abd6ccb80 1 -- 172.21.15.1:0/1756109379 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55792fce7930 msgr2=0x55792fce7dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:12.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.625+0000 7f5abd6ccb80 1 --2- 172.21.15.1:0/1756109379 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55792fce7930 0x55792fce7dc0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c0c8d50 tx=0x7f5a9c00dfd0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.625+0000 7f5abd6ccb80 1 -- 172.21.15.1:0/1756109379 shutdown_connections 2024-02-02T15:26:12.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.625+0000 7f5abd6ccb80 1 --2- 172.21.15.1:0/1756109379 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55792fce7930 0x55792fce7dc0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.625+0000 7f5abd6ccb80 1 --2- 172.21.15.1:0/1756109379 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55792fce6130 0x55792fce6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.625+0000 7f5abd6ccb80 1 --2- 172.21.15.1:0/1756109379 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55792fce6a90 0x55792fce73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.625+0000 7f5abd6ccb80 1 -- 172.21.15.1:0/1756109379 >> 172.21.15.1:0/1756109379 conn(0x55792fc44d60 msgr2=0x55792fce0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:12.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.625+0000 7f5abd6ccb80 1 -- 172.21.15.1:0/1756109379 shutdown_connections 2024-02-02T15:26:12.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.626+0000 7f5abd6ccb80 1 -- 172.21.15.1:0/1756109379 wait complete. 2024-02-02T15:26:12.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.626+0000 7f5abd6ccb80 1 -- start start 2024-02-02T15:26:12.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.626+0000 7f5abd6ccb80 1 Processor -- start 2024-02-02T15:26:12.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.627+0000 7f5abd6ccb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55792fce6130 0x55792fceff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:12.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.627+0000 7f5abd6ccb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55792fce6a90 0x55792fcf04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:12.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.627+0000 7f5abd6ccb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55792fce7930 0x55792fcf7510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:12.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.627+0000 7f5abd6ccb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55792fc36d00 con 0x55792fce7930 2024-02-02T15:26:12.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.627+0000 7f5abd6ccb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55792fbd2d30 con 0x55792fce6a90 2024-02-02T15:26:12.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.627+0000 7f5abd6ccb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55792fbd29d0 con 0x55792fce6130 2024-02-02T15:26:12.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.627+0000 7f5abd6ccb80 10 client.0 _ll_register_callbacks cb 0x55792fbd4240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:26:12.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.627+0000 7f5aa3400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55792fce7930 0x55792fcf7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:12.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.627+0000 7f5aa2000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55792fce6a90 0x55792fcf04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:12.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.627+0000 7f5aa2a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55792fce6130 0x55792fceff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:12.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.627+0000 7f5aa2000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55792fce6a90 0x55792fcf04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:56680/0 (socket says 172.21.15.1:56680) 2024-02-02T15:26:12.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.627+0000 7f5aa2000700 1 -- 172.21.15.1:0/1803408372 learned_addr learned my addr 172.21.15.1:0/1803408372 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:12.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.627+0000 7f5aa3400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55792fce7930 0x55792fcf7510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55556/0 (socket says 172.21.15.1:55556) 2024-02-02T15:26:12.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.627+0000 7f5aa2000700 1 -- 172.21.15.1:0/1803408372 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55792fce6130 msgr2=0x55792fceff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:12.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.627+0000 7f5aa2000700 1 --2- 172.21.15.1:0/1803408372 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55792fce6130 0x55792fceff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.627+0000 7f5aa2000700 1 -- 172.21.15.1:0/1803408372 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55792fce7930 msgr2=0x55792fcf7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:12.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.627+0000 7f5aa2000700 1 --2- 172.21.15.1:0/1803408372 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55792fce7930 0x55792fcf7510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.627+0000 7f5aa2000700 1 -- 172.21.15.1:0/1803408372 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5aa41a1340 con 0x55792fce6a90 2024-02-02T15:26:12.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.628+0000 7f5aa2000700 1 --2- 172.21.15.1:0/1803408372 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55792fce6a90 0x55792fcf04d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c009990 tx=0x7f5a8c005800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8cee6b3d3dcd92bc server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:12.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.628+0000 7f5a90c00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:26:12.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.628+0000 7f5a90c00700 1 -- 172.21.15.1:0/1803408372 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a8c00bc60 con 0x55792fce6a90 2024-02-02T15:26:12.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.628+0000 7f5abd6ccb80 1 -- 172.21.15.1:0/1803408372 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55792fcd90a0 con 0x55792fce6a90 2024-02-02T15:26:12.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.628+0000 7f5a90c00700 1 -- 172.21.15.1:0/1803408372 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f5a8c00bdc0 con 0x55792fce6a90 2024-02-02T15:26:12.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.628+0000 7f5a90c00700 1 -- 172.21.15.1:0/1803408372 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a8c0038c0 con 0x55792fce6a90 2024-02-02T15:26:12.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.628+0000 7f5abd6ccb80 10 client.25193 fetch_fsmap learned FSMap version 27 2024-02-02T15:26:12.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.628+0000 7f5abd6ccb80 1 -- 172.21.15.1:0/1803408372 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55792fa64b10 con 0x55792fce6a90 2024-02-02T15:26:12.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.629+0000 7f5a90c00700 1 -- 172.21.15.1:0/1803408372 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55792fcd90a0 con 0x55792fce6a90 2024-02-02T15:26:12.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.629+0000 7f5a90c00700 1 -- 172.21.15.1:0/1803408372 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5a8c004730 con 0x55792fce6a90 2024-02-02T15:26:12.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.629+0000 7f5abd6ccb80 10 client.25193 fetch_fsmap finished waiting for FSMap version 27 2024-02-02T15:26:12.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.629+0000 7f5abd6ccb80 10 client.25193 resolve_mds: resolved name 'b' to daemon mds.b 2024-02-02T15:26:12.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.629+0000 7f5abd6ccb80 20 client.25193 populate_metadata read hostname 'smithi001' 2024-02-02T15:26:12.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.629+0000 7f5abd6ccb80 1 --2- 172.21.15.1:0/1803408372 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55792fcfb2b0 0x55792fcfa660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:12.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.629+0000 7f5abd6ccb80 4 client.25193 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:26:12.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.629+0000 7f5abd6ccb80 1 -- 172.21.15.1:0/1803408372 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55792fc2fd80 con 0x55792fcfb2b0 2024-02-02T15:26:12.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.629+0000 7f5aa2a00700 1 --2- 172.21.15.1:0/1803408372 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55792fcfb2b0 0x55792fcfa660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:12.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.630+0000 7f5aa2a00700 1 --2- 172.21.15.1:0/1803408372 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55792fcfb2b0 0x55792fcfa660 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ed19ddeac6120ba server_cookie=ca050ed2fc0fa1c in_seq=0 out_seq=0 2024-02-02T15:26:12.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.630+0000 7f5a90c00700 10 client.25193 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:26:12.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.630+0000 7f5a90c00700 1 -- 172.21.15.1:0/1803408372 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55792fc2fd80 con 0x55792fcfb2b0 2024-02-02T15:26:12.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.630+0000 7f5a90c00700 10 client.25193 handle_command_reply: tid=0 2024-02-02T15:26:12.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.630+0000 7f5abd6ccb80 1 client.25193 shutdown 2024-02-02T15:26:12.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.631+0000 7f5abd6ccb80 1 -- 172.21.15.1:0/1803408372 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55792fce6a90 msgr2=0x55792fcf04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:12.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.631+0000 7f5abd6ccb80 1 --2- 172.21.15.1:0/1803408372 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55792fce6a90 0x55792fcf04d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c009990 tx=0x7f5a8c005800 comp rx=0 tx=0).stop 2024-02-02T15:26:12.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.631+0000 7f5abd6ccb80 1 -- 172.21.15.1:0/1803408372 shutdown_connections 2024-02-02T15:26:12.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.631+0000 7f5abd6ccb80 1 --2- 172.21.15.1:0/1803408372 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55792fcfb2b0 0x55792fcfa660 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.631+0000 7f5abd6ccb80 1 --2- 172.21.15.1:0/1803408372 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55792fce7930 0x55792fcf7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.631+0000 7f5abd6ccb80 1 --2- 172.21.15.1:0/1803408372 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55792fce6130 0x55792fceff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.631+0000 7f5abd6ccb80 1 --2- 172.21.15.1:0/1803408372 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55792fce6a90 0x55792fcf04d0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.631+0000 7f5a90c00700 0 client.25193 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:26:12.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.631+0000 7f5abd6ccb80 1 -- 172.21.15.1:0/1803408372 >> 172.21.15.1:0/1803408372 conn(0x55792fc44d60 msgr2=0x55792fce0570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:12.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.631+0000 7f5abd6ccb80 1 -- 172.21.15.1:0/1803408372 shutdown_connections 2024-02-02T15:26:12.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.631+0000 7f5abd6ccb80 1 -- 172.21.15.1:0/1803408372 wait complete. 2024-02-02T15:26:12.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.631+0000 7f5abd6ccb80 20 client.25193 trim_cache size 0 max 16384 2024-02-02T15:26:12.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.649+0000 7f5abd6ccb80 1 Processor -- start 2024-02-02T15:26:12.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.649+0000 7f5abd6ccb80 1 -- start start 2024-02-02T15:26:12.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.649+0000 7f5abd6ccb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55792fce6130 0x55792fcfa660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:12.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.649+0000 7f5abd6ccb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55792fce6a90 0x55792fd03fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:12.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.649+0000 7f5abd6ccb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55792fce7930 0x55792fcfaba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:12.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.649+0000 7f5abd6ccb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55792faba650 con 0x55792fce7930 2024-02-02T15:26:12.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.649+0000 7f5abd6ccb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55792fcf1180 con 0x55792fce6130 2024-02-02T15:26:12.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.649+0000 7f5abd6ccb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55792fcee4a0 con 0x55792fce6a90 2024-02-02T15:26:12.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.649+0000 7f5aa3400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55792fce7930 0x55792fcfaba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:12.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.649+0000 7f5aa3400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55792fce7930 0x55792fcfaba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55570/0 (socket says 172.21.15.1:55570) 2024-02-02T15:26:12.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.649+0000 7f5aa3400700 1 -- 172.21.15.1:0/1315508549 learned_addr learned my addr 172.21.15.1:0/1315508549 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:12.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.650+0000 7f5aa2a00700 1 --2- 172.21.15.1:0/1315508549 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55792fce6130 0x55792fcfa660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:12.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.650+0000 7f5aa2000700 1 --2- 172.21.15.1:0/1315508549 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55792fce6a90 0x55792fd03fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:12.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.650+0000 7f5aa3400700 1 -- 172.21.15.1:0/1315508549 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55792fce6a90 msgr2=0x55792fd03fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:12.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.650+0000 7f5aa3400700 1 --2- 172.21.15.1:0/1315508549 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55792fce6a90 0x55792fd03fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.650+0000 7f5aa3400700 1 -- 172.21.15.1:0/1315508549 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55792fce6130 msgr2=0x55792fcfa660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:12.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.650+0000 7f5aa3400700 1 --2- 172.21.15.1:0/1315508549 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55792fce6130 0x55792fcfa660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.650+0000 7f5aa3400700 1 -- 172.21.15.1:0/1315508549 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55792fbfce50 con 0x55792fce7930 2024-02-02T15:26:12.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.650+0000 7f5aa3400700 1 --2- 172.21.15.1:0/1315508549 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55792fce7930 0x55792fcfaba0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c0c8d80 tx=0x7f5a9c000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87ed342e2c46f644 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:12.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.650+0000 7f5a98c00700 1 -- 172.21.15.1:0/1315508549 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9c0991a0 con 0x55792fce7930 2024-02-02T15:26:12.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.650+0000 7f5a98c00700 1 -- 172.21.15.1:0/1315508549 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a9c0a0b10 con 0x55792fce7930 2024-02-02T15:26:12.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.650+0000 7f5a98c00700 1 -- 172.21.15.1:0/1315508549 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9c0c8fa0 con 0x55792fce7930 2024-02-02T15:26:12.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.651+0000 7f5abd6ccb80 1 -- 172.21.15.1:0/1315508549 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55792fce7930 msgr2=0x55792fcfaba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:12.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.651+0000 7f5abd6ccb80 1 --2- 172.21.15.1:0/1315508549 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55792fce7930 0x55792fcfaba0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c0c8d80 tx=0x7f5a9c000f40 comp rx=0 tx=0).stop 2024-02-02T15:26:12.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.651+0000 7f5abd6ccb80 1 -- 172.21.15.1:0/1315508549 shutdown_connections 2024-02-02T15:26:12.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.651+0000 7f5abd6ccb80 1 --2- 172.21.15.1:0/1315508549 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55792fce7930 0x55792fcfaba0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.651+0000 7f5abd6ccb80 1 --2- 172.21.15.1:0/1315508549 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55792fce6130 0x55792fcfa660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.651+0000 7f5abd6ccb80 1 --2- 172.21.15.1:0/1315508549 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55792fce6a90 0x55792fd03fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.651+0000 7f5abd6ccb80 1 -- 172.21.15.1:0/1315508549 >> 172.21.15.1:0/1315508549 conn(0x55792fc44d60 msgr2=0x55792fcedd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:12.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.651+0000 7f5abd6ccb80 1 -- 172.21.15.1:0/1315508549 shutdown_connections 2024-02-02T15:26:12.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.651+0000 7f5abd6ccb80 1 -- 172.21.15.1:0/1315508549 wait complete. 2024-02-02T15:26:12.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.651+0000 7f5abd6ccb80 1 -- start start 2024-02-02T15:26:12.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.652+0000 7f5abd6ccb80 1 Processor -- start 2024-02-02T15:26:12.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.652+0000 7f5abd6ccb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55792fce6130 0x55792fd03f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:12.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.652+0000 7f5abd6ccb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55792fce6a90 0x55792fd00a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:12.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.652+0000 7f5abd6ccb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55792fce7930 0x55792fd01050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:12.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.652+0000 7f5abd6ccb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55792fd4cd60 con 0x55792fce6130 2024-02-02T15:26:12.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.652+0000 7f5abd6ccb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55792fd4cea0 con 0x55792fce7930 2024-02-02T15:26:12.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.652+0000 7f5abd6ccb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55792fd4cfe0 con 0x55792fce6a90 2024-02-02T15:26:12.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.652+0000 7f5abd6ccb80 10 client.0 _ll_register_callbacks cb 0x55792fbd4240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:26:12.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.652+0000 7f5aa2a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55792fce6130 0x55792fd03f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:12.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.652+0000 7f5aa2a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55792fce6130 0x55792fd03f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55584/0 (socket says 172.21.15.1:55584) 2024-02-02T15:26:12.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.652+0000 7f5aa2a00700 1 -- 172.21.15.1:0/247280321 learned_addr learned my addr 172.21.15.1:0/247280321 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:12.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.652+0000 7f5aa3400700 1 --2- 172.21.15.1:0/247280321 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55792fce7930 0x55792fd01050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:12.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.653+0000 7f5aa2000700 1 --2- 172.21.15.1:0/247280321 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55792fce6a90 0x55792fd00a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:12.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.653+0000 7f5aa2a00700 1 -- 172.21.15.1:0/247280321 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55792fce6a90 msgr2=0x55792fd00a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:12.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.653+0000 7f5aa2a00700 1 --2- 172.21.15.1:0/247280321 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55792fce6a90 0x55792fd00a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.653+0000 7f5aa2a00700 1 -- 172.21.15.1:0/247280321 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55792fce7930 msgr2=0x55792fd01050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:12.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.653+0000 7f5aa2a00700 1 --2- 172.21.15.1:0/247280321 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55792fce7930 0x55792fd01050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.653+0000 7f5aa2a00700 1 -- 172.21.15.1:0/247280321 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55792fab8360 con 0x55792fce6130 2024-02-02T15:26:12.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.653+0000 7f5aa2a00700 1 --2- 172.21.15.1:0/247280321 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55792fce6130 0x55792fd03f80 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5a9400ba60 tx=0x7f5a940047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6b91fe0ecd718d7 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:12.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.653+0000 7f5a90c00700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:26:12.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.653+0000 7f5a90c00700 1 -- 172.21.15.1:0/247280321 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a94024040 con 0x55792fce6130 2024-02-02T15:26:12.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.653+0000 7f5a90c00700 1 -- 172.21.15.1:0/247280321 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f5a94019040 con 0x55792fce6130 2024-02-02T15:26:12.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.653+0000 7f5abd6ccb80 1 -- 172.21.15.1:0/247280321 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5a9c098aa0 con 0x55792fce6130 2024-02-02T15:26:12.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.654+0000 7f5a90c00700 1 -- 172.21.15.1:0/247280321 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a94029430 con 0x55792fce6130 2024-02-02T15:26:12.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.654+0000 7f5abd6ccb80 10 client.15171 fetch_fsmap learned FSMap version 27 2024-02-02T15:26:12.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.654+0000 7f5abd6ccb80 1 -- 172.21.15.1:0/247280321 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55792fabcc80 con 0x55792fce6130 2024-02-02T15:26:12.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.654+0000 7f5a90c00700 1 -- 172.21.15.1:0/247280321 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a9c098aa0 con 0x55792fce6130 2024-02-02T15:26:12.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.654+0000 7f5a90c00700 1 -- 172.21.15.1:0/247280321 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5a94011650 con 0x55792fce6130 2024-02-02T15:26:12.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.654+0000 7f5abd6ccb80 10 client.15171 fetch_fsmap finished waiting for FSMap version 27 2024-02-02T15:26:12.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.654+0000 7f5abd6ccb80 10 client.15171 resolve_mds: resolved name 'b' to daemon mds.b 2024-02-02T15:26:12.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.654+0000 7f5abd6ccb80 20 client.15171 populate_metadata read hostname 'smithi001' 2024-02-02T15:26:12.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.654+0000 7f5abd6ccb80 1 --2- 172.21.15.1:0/247280321 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55792fd51e80 0x55792fd4e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:12.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.654+0000 7f5abd6ccb80 4 client.15171 mds_command: new command op to 24557 tid=0[{"prefix": "status"}] 2024-02-02T15:26:12.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.654+0000 7f5abd6ccb80 1 -- 172.21.15.1:0/247280321 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55792fbf5f00 con 0x55792fd51e80 2024-02-02T15:26:12.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.655+0000 7f5aa2000700 1 --2- 172.21.15.1:0/247280321 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55792fd51e80 0x55792fd4e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:12.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.655+0000 7f5aa2000700 1 --2- 172.21.15.1:0/247280321 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55792fd51e80 0x55792fd4e820 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36779abc72c1eb28 server_cookie=9d72718d1332a873 in_seq=0 out_seq=0 2024-02-02T15:26:12.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.655+0000 7f5a90c00700 10 client.15171 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:26:12.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.655+0000 7f5a90c00700 1 -- 172.21.15.1:0/247280321 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55792fbf5f00 con 0x55792fd51e80 2024-02-02T15:26:12.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.655+0000 7f5a90c00700 10 client.15171 handle_command_reply: tid=0 2024-02-02T15:26:12.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.655+0000 7f5abd6ccb80 1 client.15171 shutdown 2024-02-02T15:26:12.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.655+0000 7f5abd6ccb80 1 -- 172.21.15.1:0/247280321 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55792fce6130 msgr2=0x55792fd03f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:12.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.655+0000 7f5abd6ccb80 1 --2- 172.21.15.1:0/247280321 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55792fce6130 0x55792fd03f80 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5a9400ba60 tx=0x7f5a940047c0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.655+0000 7f5abd6ccb80 1 -- 172.21.15.1:0/247280321 shutdown_connections 2024-02-02T15:26:12.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.655+0000 7f5abd6ccb80 1 --2- 172.21.15.1:0/247280321 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55792fd51e80 0x55792fd4e820 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.655+0000 7f5abd6ccb80 1 --2- 172.21.15.1:0/247280321 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55792fce7930 0x55792fd01050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.655+0000 7f5abd6ccb80 1 --2- 172.21.15.1:0/247280321 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55792fce6130 0x55792fd03f80 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.655+0000 7f5a90c00700 0 client.15171 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:26:12.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.656+0000 7f5abd6ccb80 1 --2- 172.21.15.1:0/247280321 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55792fce6a90 0x55792fd00a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.656+0000 7f5abd6ccb80 1 -- 172.21.15.1:0/247280321 >> 172.21.15.1:0/247280321 conn(0x55792fc44d60 msgr2=0x55792fcfa660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:12.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.656+0000 7f5abd6ccb80 1 -- 172.21.15.1:0/247280321 shutdown_connections 2024-02-02T15:26:12.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.656+0000 7f5abd6ccb80 1 -- 172.21.15.1:0/247280321 wait complete. 2024-02-02T15:26:12.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.656+0000 7f5abd6ccb80 20 client.15171 trim_cache size 0 max 16384 2024-02-02T15:26:12.663 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:26:12.663 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "371a409c-c1de-11ee-95b6-87774f69a715", 2024-02-02T15:26:12.663 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 0, 2024-02-02T15:26:12.663 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 24557, 2024-02-02T15:26:12.663 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2024-02-02T15:26:12.664 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2024-02-02T15:26:12.664 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2024-02-02T15:26:12.664 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 84.890528450999994, 2024-02-02T15:26:12.664 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 27, 2024-02-02T15:26:12.664 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 86, 2024-02-02T15:26:12.664 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 84, 2024-02-02T15:26:12.664 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 91.221222432999994 2024-02-02T15:26:12.664 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:26:12.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.656+0000 7f5aa9c00700 1 -- 172.21.15.1:0/1282482640 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f5a840a0630 msgr2=0x7f5a840a2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:12.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.656+0000 7f5aa9c00700 1 --2- 172.21.15.1:0/1282482640 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f5a840a0630 0x7f5a840a2af0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5aa4101e50 tx=0x7f5a940106b0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.656+0000 7f5aa9c00700 1 -- 172.21.15.1:0/1282482640 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5aa4102990 msgr2=0x7f5aa41a02f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:12.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.656+0000 7f5aa9c00700 1 --2- 172.21.15.1:0/1282482640 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5aa4102990 0x7f5aa41a02f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c007ee0 tx=0x7f5a9c00d3b0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.656+0000 7f5aa9c00700 1 -- 172.21.15.1:0/1282482640 shutdown_connections 2024-02-02T15:26:12.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.656+0000 7f5aa9c00700 1 --2- 172.21.15.1:0/1282482640 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5aa4102990 0x7f5aa41a02f0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.656+0000 7f5aa9c00700 1 --2- 172.21.15.1:0/1282482640 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f5a840a0630 0x7f5a840a2af0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.656+0000 7f5aa9c00700 1 --2- 172.21.15.1:0/1282482640 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5aa4100e00 0x7f5aa4198d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.656+0000 7f5aa9c00700 1 --2- 172.21.15.1:0/1282482640 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5aa4101ff0 0x7f5aa4199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:12.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.656+0000 7f5aa9c00700 1 -- 172.21.15.1:0/1282482640 >> 172.21.15.1:0/1282482640 conn(0x7f5aa40fc370 msgr2=0x7f5aa4107930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:12.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.657+0000 7f5aa9c00700 1 -- 172.21.15.1:0/1282482640 shutdown_connections 2024-02-02T15:26:12.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:12.657+0000 7f5aa9c00700 1 -- 172.21.15.1:0/1282482640 wait complete. 2024-02-02T15:26:12.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:12 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/1659083492' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:26:12.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:12 smithi142 ceph-mon[45728]: pgmap v279: 129 pgs: 129 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-02-02T15:26:12.674 INFO:tasks.cephadm:Setup complete, yielding 2024-02-02T15:26:12.675 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-02T15:26:12.684 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi001.front.sepia.ceph.com 2024-02-02T15:26:12.685 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- bash -c 'ceph orch status' 2024-02-02T15:26:13.052 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.a/config 2024-02-02T15:26:14.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:14.196+0000 7fe854800700 1 -- 172.21.15.1:0/1184149873 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe84c102970 msgr2=0x7fe84c109220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:14.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:14.196+0000 7fe854800700 1 --2- 172.21.15.1:0/1184149873 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe84c102970 0x7fe84c109220 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe84400b0d0 tx=0x7fe84400b3e0 comp rx=0 tx=0).stop 2024-02-02T15:26:14.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:14.196+0000 7fe854800700 1 -- 172.21.15.1:0/1184149873 shutdown_connections 2024-02-02T15:26:14.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:14.196+0000 7fe854800700 1 --2- 172.21.15.1:0/1184149873 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe84c102970 0x7fe84c109220 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:14.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:14.196+0000 7fe854800700 1 --2- 172.21.15.1:0/1184149873 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe84c100dd0 0x7fe84c1011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:14.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:14.196+0000 7fe854800700 1 --2- 172.21.15.1:0/1184149873 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe84c101fd0 0x7fe84c102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:14.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:14.196+0000 7fe854800700 1 -- 172.21.15.1:0/1184149873 >> 172.21.15.1:0/1184149873 conn(0x7fe84c0fc330 msgr2=0x7fe84c0fe7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:14.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:14.196+0000 7fe854800700 1 -- 172.21.15.1:0/1184149873 shutdown_connections 2024-02-02T15:26:14.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:14.196+0000 7fe854800700 1 -- 172.21.15.1:0/1184149873 wait complete. 2024-02-02T15:26:14.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:14.197+0000 7fe854800700 1 Processor -- start 2024-02-02T15:26:14.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:14.197+0000 7fe854800700 1 -- start start 2024-02-02T15:26:14.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:14.198+0000 7fe854800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe84c100dd0 0x7fe84c074ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:14.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:14.198+0000 7fe854800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe84c101fd0 0x7fe84c073130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:14.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:14.198+0000 7fe854800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe84c102970 0x7fe84c073670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:14.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:14.198+0000 7fe854800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe84c073df0 con 0x7fe84c100dd0 2024-02-02T15:26:14.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:14.198+0000 7fe854800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe84c073f60 con 0x7fe84c102970 2024-02-02T15:26:14.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:14.198+0000 7fe854800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe84c1a0c20 con 0x7fe84c101fd0 2024-02-02T15:26:14.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:14.198+0000 7fe851e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe84c100dd0 0x7fe84c074ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:14.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:14.198+0000 7fe851e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe84c100dd0 0x7fe84c074ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52540/0 (socket says 172.21.15.1:52540) 2024-02-02T15:26:14.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:14.198+0000 7fe851e00700 1 -- 172.21.15.1:0/3778109764 learned_addr learned my addr 172.21.15.1:0/3778109764 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:14.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:14.198+0000 7fe852800700 1 --2- 172.21.15.1:0/3778109764 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe84c102970 0x7fe84c073670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:14.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:14.198+0000 7fe851400700 1 --2- 172.21.15.1:0/3778109764 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe84c101fd0 0x7fe84c073130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:14.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:14.198+0000 7fe851e00700 1 -- 172.21.15.1:0/3778109764 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe84c101fd0 msgr2=0x7fe84c073130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:14.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:14.198+0000 7fe851e00700 1 --2- 172.21.15.1:0/3778109764 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe84c101fd0 0x7fe84c073130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:14.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:14.198+0000 7fe851e00700 1 -- 172.21.15.1:0/3778109764 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe84c102970 msgr2=0x7fe84c073670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:14.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:14.198+0000 7fe851e00700 1 --2- 172.21.15.1:0/3778109764 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe84c102970 0x7fe84c073670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:14.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:14.198+0000 7fe851e00700 1 -- 172.21.15.1:0/3778109764 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe844009d00 con 0x7fe84c100dd0 2024-02-02T15:26:14.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:14.199+0000 7fe851e00700 1 --2- 172.21.15.1:0/3778109764 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe84c100dd0 0x7fe84c074ae0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fe83c00e970 tx=0x7fe83c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:14.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:14.199+0000 7fe84aa00700 1 -- 172.21.15.1:0/3778109764 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe83c009790 con 0x7fe84c100dd0 2024-02-02T15:26:14.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:14.199+0000 7fe84aa00700 1 -- 172.21.15.1:0/3778109764 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe83c00c7e0 con 0x7fe84c100dd0 2024-02-02T15:26:14.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:14.199+0000 7fe854800700 1 -- 172.21.15.1:0/3778109764 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe84c1a0f00 con 0x7fe84c100dd0 2024-02-02T15:26:14.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:14.199+0000 7fe84aa00700 1 -- 172.21.15.1:0/3778109764 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe83c003c70 con 0x7fe84c100dd0 2024-02-02T15:26:14.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:14.199+0000 7fe854800700 1 -- 172.21.15.1:0/3778109764 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe84c1a1320 con 0x7fe84c100dd0 2024-02-02T15:26:14.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:14.200+0000 7fe854800700 1 -- 172.21.15.1:0/3778109764 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe84c108310 con 0x7fe84c100dd0 2024-02-02T15:26:14.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:14.202+0000 7fe84aa00700 1 -- 172.21.15.1:0/3778109764 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) v1 ==== 135120+0+0 (secure 0 0 0) 0x7fe83c00cc50 con 0x7fe84c100dd0 2024-02-02T15:26:14.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:14.204+0000 7fe84aa00700 1 --2- 172.21.15.1:0/3778109764 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fe82c0a0630 0x7fe82c0a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:14.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:14.205+0000 7fe84aa00700 1 -- 172.21.15.1:0/3778109764 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fe83c014070 con 0x7fe84c100dd0 2024-02-02T15:26:14.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:14.205+0000 7fe851400700 1 --2- 172.21.15.1:0/3778109764 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fe82c0a0630 0x7fe82c0a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:14.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:14.205+0000 7fe851400700 1 --2- 172.21.15.1:0/3778109764 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fe82c0a0630 0x7fe82c0a2af0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe834009c60 tx=0x7fe834009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:14.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:14.207+0000 7fe84aa00700 1 -- 172.21.15.1:0/3778109764 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7fe83c095810 con 0x7fe84c100dd0 2024-02-02T15:26:14.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:14.433+0000 7fe854800700 1 -- 172.21.15.1:0/3778109764 --> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe84c061180 con 0x7fe82c0a0630 2024-02-02T15:26:14.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:14.434+0000 7fe84aa00700 1 -- 172.21.15.1:0/3778109764 <== mgr.14154 v2:172.21.15.1:6800/3768989981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fe84c061180 con 0x7fe82c0a0630 2024-02-02T15:26:14.434 INFO:teuthology.orchestra.run.smithi001.stdout:Backend: cephadm 2024-02-02T15:26:14.434 INFO:teuthology.orchestra.run.smithi001.stdout:Available: Yes 2024-02-02T15:26:14.434 INFO:teuthology.orchestra.run.smithi001.stdout:Paused: No 2024-02-02T15:26:14.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:14.436+0000 7fe854800700 1 -- 172.21.15.1:0/3778109764 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fe82c0a0630 msgr2=0x7fe82c0a2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:14.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:14.436+0000 7fe854800700 1 --2- 172.21.15.1:0/3778109764 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fe82c0a0630 0x7fe82c0a2af0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe834009c60 tx=0x7fe834009260 comp rx=0 tx=0).stop 2024-02-02T15:26:14.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:14.436+0000 7fe854800700 1 -- 172.21.15.1:0/3778109764 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe84c100dd0 msgr2=0x7fe84c074ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:14.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:14.436+0000 7fe854800700 1 --2- 172.21.15.1:0/3778109764 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe84c100dd0 0x7fe84c074ae0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fe83c00e970 tx=0x7fe83c00ec80 comp rx=0 tx=0).stop 2024-02-02T15:26:14.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:14.437+0000 7fe854800700 1 -- 172.21.15.1:0/3778109764 shutdown_connections 2024-02-02T15:26:14.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:14.437+0000 7fe854800700 1 --2- 172.21.15.1:0/3778109764 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe84c102970 0x7fe84c073670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:14.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:14.437+0000 7fe854800700 1 --2- 172.21.15.1:0/3778109764 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe84c100dd0 0x7fe84c074ae0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:14.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:14.437+0000 7fe854800700 1 --2- 172.21.15.1:0/3778109764 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fe82c0a0630 0x7fe82c0a2af0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:14.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:14.437+0000 7fe854800700 1 --2- 172.21.15.1:0/3778109764 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe84c101fd0 0x7fe84c073130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:14.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:14.437+0000 7fe854800700 1 -- 172.21.15.1:0/3778109764 >> 172.21.15.1:0/3778109764 conn(0x7fe84c0fc330 msgr2=0x7fe84c105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:14.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:14.437+0000 7fe854800700 1 -- 172.21.15.1:0/3778109764 shutdown_connections 2024-02-02T15:26:14.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:14.437+0000 7fe854800700 1 -- 172.21.15.1:0/3778109764 wait complete. 2024-02-02T15:26:14.805 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- bash -c 'ceph orch ps' 2024-02-02T15:26:15.173 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.a/config 2024-02-02T15:26:15.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:14 smithi001 ceph-mon[35470]: pgmap v280: 129 pgs: 129 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-02-02T15:26:15.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:14 smithi071 ceph-mon[44707]: pgmap v280: 129 pgs: 129 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-02-02T15:26:15.419 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:14 smithi142 ceph-mon[45728]: pgmap v280: 129 pgs: 129 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-02-02T15:26:16.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:16.256+0000 7f57da600700 1 -- 172.21.15.1:0/3990749350 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57d4101860 msgr2=0x7f57d4103c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:16.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:16.256+0000 7f57da600700 1 --2- 172.21.15.1:0/3990749350 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57d4101860 0x7f57d4103c50 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f57c4009990 tx=0x7f57c4009ca0 comp rx=0 tx=0).stop 2024-02-02T15:26:16.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:16.256+0000 7f57da600700 1 -- 172.21.15.1:0/3990749350 shutdown_connections 2024-02-02T15:26:16.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:16.256+0000 7f57da600700 1 --2- 172.21.15.1:0/3990749350 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f57d4106ac0 0x7f57d4108f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:16.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:16.256+0000 7f57da600700 1 --2- 172.21.15.1:0/3990749350 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57d4101860 0x7f57d4103c50 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:16.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:16.256+0000 7f57da600700 1 --2- 172.21.15.1:0/3990749350 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57d4104190 0x7f57d4106580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:16.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:16.256+0000 7f57da600700 1 -- 172.21.15.1:0/3990749350 >> 172.21.15.1:0/3990749350 conn(0x7f57d40fb250 msgr2=0x7f57d40fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:16.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:16.256+0000 7f57da600700 1 -- 172.21.15.1:0/3990749350 shutdown_connections 2024-02-02T15:26:16.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:16.256+0000 7f57da600700 1 -- 172.21.15.1:0/3990749350 wait complete. 2024-02-02T15:26:16.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:16.258+0000 7f57da600700 1 Processor -- start 2024-02-02T15:26:16.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:16.258+0000 7f57da600700 1 -- start start 2024-02-02T15:26:16.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:16.258+0000 7f57da600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57d4101860 0x7f57d4196ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:16.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:16.258+0000 7f57da600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f57d4104190 0x7f57d4197010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:16.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:16.258+0000 7f57da600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57d4106ac0 0x7f57d419e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:16.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:16.258+0000 7f57da600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d419e660 con 0x7f57d4106ac0 2024-02-02T15:26:16.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:16.258+0000 7f57da600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d419e7a0 con 0x7f57d4101860 2024-02-02T15:26:16.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:16.258+0000 7f57da600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d419e930 con 0x7f57d4104190 2024-02-02T15:26:16.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:16.259+0000 7f57d3e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57d4106ac0 0x7f57d419e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:16.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:16.259+0000 7f57d2a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f57d4104190 0x7f57d4197010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:16.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:16.259+0000 7f57d3400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57d4101860 0x7f57d4196ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:16.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:16.259+0000 7f57d2a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f57d4104190 0x7f57d4197010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:55228/0 (socket says 172.21.15.1:55228) 2024-02-02T15:26:16.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:16.259+0000 7f57d2a00700 1 -- 172.21.15.1:0/155240730 learned_addr learned my addr 172.21.15.1:0/155240730 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:16.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:16.259+0000 7f57d2a00700 1 -- 172.21.15.1:0/155240730 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57d4101860 msgr2=0x7f57d4196ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:16.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:16.259+0000 7f57d2a00700 1 --2- 172.21.15.1:0/155240730 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57d4101860 0x7f57d4196ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:16.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:16.259+0000 7f57d2a00700 1 -- 172.21.15.1:0/155240730 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57d4106ac0 msgr2=0x7f57d419e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:16.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:16.259+0000 7f57d2a00700 1 --2- 172.21.15.1:0/155240730 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57d4106ac0 0x7f57d419e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:16.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:16.259+0000 7f57d2a00700 1 -- 172.21.15.1:0/155240730 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57c40095f0 con 0x7f57d4104190 2024-02-02T15:26:16.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:16.259+0000 7f57d3e00700 1 --2- 172.21.15.1:0/155240730 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57d4106ac0 0x7f57d419e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:26:16.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:16.260+0000 7f57d2a00700 1 --2- 172.21.15.1:0/155240730 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f57d4104190 0x7f57d4197010 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f57bc00d9b0 tx=0x7f57bc00dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:16.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:16.260+0000 7f57cbe00700 1 -- 172.21.15.1:0/155240730 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57bc004220 con 0x7f57d4104190 2024-02-02T15:26:16.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:16.260+0000 7f57da600700 1 -- 172.21.15.1:0/155240730 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57d419ec10 con 0x7f57d4104190 2024-02-02T15:26:16.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:16.260+0000 7f57cbe00700 1 -- 172.21.15.1:0/155240730 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f57bc007640 con 0x7f57d4104190 2024-02-02T15:26:16.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:16.260+0000 7f57cbe00700 1 -- 172.21.15.1:0/155240730 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57bc005590 con 0x7f57d4104190 2024-02-02T15:26:16.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:16.260+0000 7f57da600700 1 -- 172.21.15.1:0/155240730 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57d419f160 con 0x7f57d4104190 2024-02-02T15:26:16.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:16.261+0000 7f57da600700 1 -- 172.21.15.1:0/155240730 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57d40fcdd0 con 0x7f57d4104190 2024-02-02T15:26:16.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:16.263+0000 7f57cbe00700 1 -- 172.21.15.1:0/155240730 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 17) v1 ==== 135120+0+0 (secure 0 0 0) 0x7f57bc004750 con 0x7f57d4104190 2024-02-02T15:26:16.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:16.264+0000 7f57cbe00700 1 --2- 172.21.15.1:0/155240730 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f57b40a0630 0x7f57b40a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:16.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:16.264+0000 7f57cbe00700 1 -- 172.21.15.1:0/155240730 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f57bc0cc060 con 0x7f57d4104190 2024-02-02T15:26:16.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:16.264+0000 7f57d3400700 1 --2- 172.21.15.1:0/155240730 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f57b40a0630 0x7f57b40a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:16.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:16.265+0000 7f57d3400700 1 --2- 172.21.15.1:0/155240730 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f57b40a0630 0x7f57b40a2af0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f57c400b2e0 tx=0x7f57c4005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:16.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:16.267+0000 7f57cbe00700 1 -- 172.21.15.1:0/155240730 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f57bc095730 con 0x7f57d4104190 2024-02-02T15:26:16.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:15 smithi071 ceph-mon[44707]: from='client.15177 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:26:16.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:15 smithi142 ceph-mon[45728]: from='client.15177 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:26:16.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:15 smithi001 ceph-mon[35470]: from='client.15177 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:26:16.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:16.487+0000 7f57da600700 1 -- 172.21.15.1:0/155240730 --> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f57d4061180 con 0x7f57b40a0630 2024-02-02T15:26:16.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:16.501+0000 7f57cbe00700 1 -- 172.21.15.1:0/155240730 <== mgr.14154 v2:172.21.15.1:6800/3768989981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4154 (secure 0 0 0) 0x7f57d4061180 con 0x7f57b40a0630 2024-02-02T15:26:16.501 INFO:teuthology.orchestra.run.smithi001.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-02T15:26:16.501 INFO:teuthology.orchestra.run.smithi001.stdout:mds.a smithi001 running (97s) 18s ago 96s 16.1M - 18.2.1-325-g0fb32788 0b0faa63c314 9bd8bce92f7c 2024-02-02T15:26:16.501 INFO:teuthology.orchestra.run.smithi001.stdout:mds.b smithi071 running (95s) 29s ago 95s 33.1M - 18.2.1-325-g0fb32788 0b0faa63c314 971da35dde5a 2024-02-02T15:26:16.501 INFO:teuthology.orchestra.run.smithi001.stdout:mds.c smithi142 running (105s) 28s ago 105s 20.9M - 18.2.1-325-g0fb32788 0b0faa63c314 cde6423bf9d3 2024-02-02T15:26:16.501 INFO:teuthology.orchestra.run.smithi001.stdout:mds.d smithi001 running (111s) 18s ago 111s 13.8M - 18.2.1-325-g0fb32788 0b0faa63c314 eb4b8787cfb6 2024-02-02T15:26:16.501 INFO:teuthology.orchestra.run.smithi001.stdout:mds.e smithi071 running (109s) 29s ago 109s 18.4M - 18.2.1-325-g0fb32788 0b0faa63c314 4ca66c9f3221 2024-02-02T15:26:16.502 INFO:teuthology.orchestra.run.smithi001.stdout:mds.f smithi142 running (107s) 28s ago 107s 20.8M - 18.2.1-325-g0fb32788 0b0faa63c314 8ef5a11decce 2024-02-02T15:26:16.502 INFO:teuthology.orchestra.run.smithi001.stdout:mds.g smithi001 running (99s) 18s ago 99s 16.2M - 18.2.1-325-g0fb32788 0b0faa63c314 ebb0bc4afb88 2024-02-02T15:26:16.502 INFO:teuthology.orchestra.run.smithi001.stdout:mds.h smithi071 running (104s) 29s ago 103s 20.5M - 18.2.1-325-g0fb32788 0b0faa63c314 4898fcbe16ed 2024-02-02T15:26:16.502 INFO:teuthology.orchestra.run.smithi001.stdout:mds.i smithi142 running (102s) 28s ago 101s 18.3M - 18.2.1-325-g0fb32788 0b0faa63c314 cda2f53d586d 2024-02-02T15:26:16.502 INFO:teuthology.orchestra.run.smithi001.stdout:mds.j smithi001 running (113s) 18s ago 113s 13.8M - 18.2.1-325-g0fb32788 0b0faa63c314 32b061251d8c 2024-02-02T15:26:16.502 INFO:teuthology.orchestra.run.smithi001.stdout:mds.k smithi071 running (116s) 29s ago 116s 16.3M - 18.2.1-325-g0fb32788 0b0faa63c314 98bbf5cf46e0 2024-02-02T15:26:16.502 INFO:teuthology.orchestra.run.smithi001.stdout:mds.l smithi142 running (118s) 28s ago 117s 17.3M - 18.2.1-325-g0fb32788 0b0faa63c314 9614f16ef39c 2024-02-02T15:26:16.502 INFO:teuthology.orchestra.run.smithi001.stdout:mgr.x smithi001 *:9283,8765 running (7m) 18s ago 7m 525M - 18.2.1-325-g0fb32788 0b0faa63c314 5ead66761a21 2024-02-02T15:26:16.502 INFO:teuthology.orchestra.run.smithi001.stdout:mgr.y smithi071 *:8443,8765 running (5m) 29s ago 5m 448M - 18.2.1-325-g0fb32788 0b0faa63c314 ee6fbe603556 2024-02-02T15:26:16.502 INFO:teuthology.orchestra.run.smithi001.stdout:mgr.z smithi142 *:8443,8765 running (5m) 28s ago 5m 448M - 18.2.1-325-g0fb32788 0b0faa63c314 47f3ec4035f2 2024-02-02T15:26:16.502 INFO:teuthology.orchestra.run.smithi001.stdout:mon.a smithi001 running (7m) 18s ago 7m 95.1M 2048M 18.2.1-325-g0fb32788 0b0faa63c314 5fecb94058f0 2024-02-02T15:26:16.502 INFO:teuthology.orchestra.run.smithi001.stdout:mon.b smithi071 running (6m) 29s ago 6m 67.2M 2048M 18.2.1-325-g0fb32788 0b0faa63c314 bc27b4caedde 2024-02-02T15:26:16.502 INFO:teuthology.orchestra.run.smithi001.stdout:mon.c smithi142 running (6m) 28s ago 6m 69.1M 2048M 18.2.1-325-g0fb32788 0b0faa63c314 7b91dcb0fe7a 2024-02-02T15:26:16.503 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0 smithi001 running (5m) 18s ago 5m 69.5M 4319M 18.2.1-325-g0fb32788 0b0faa63c314 394468afa3f6 2024-02-02T15:26:16.503 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1 smithi071 running (5m) 29s ago 5m 76.4M 4278M 18.2.1-325-g0fb32788 0b0faa63c314 74685efdd812 2024-02-02T15:26:16.503 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2 smithi142 running (5m) 28s ago 5m 74.7M 4278M 18.2.1-325-g0fb32788 0b0faa63c314 4587c17ed721 2024-02-02T15:26:16.503 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3 smithi001 running (4m) 18s ago 4m 64.9M 4319M 18.2.1-325-g0fb32788 0b0faa63c314 465f70ee4724 2024-02-02T15:26:16.503 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4 smithi071 running (4m) 29s ago 4m 63.3M 4278M 18.2.1-325-g0fb32788 0b0faa63c314 f994129b1de0 2024-02-02T15:26:16.503 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5 smithi142 running (4m) 28s ago 4m 68.5M 4278M 18.2.1-325-g0fb32788 0b0faa63c314 0634ce2dc9b9 2024-02-02T15:26:16.503 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6 smithi001 running (3m) 18s ago 3m 61.8M 4319M 18.2.1-325-g0fb32788 0b0faa63c314 abef8d5b04d6 2024-02-02T15:26:16.503 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7 smithi071 running (3m) 29s ago 3m 67.8M 4278M 18.2.1-325-g0fb32788 0b0faa63c314 240d9cbdafad 2024-02-02T15:26:16.503 INFO:teuthology.orchestra.run.smithi001.stdout:osd.8 smithi142 running (3m) 28s ago 3m 63.4M 4278M 18.2.1-325-g0fb32788 0b0faa63c314 dfb9bbb73970 2024-02-02T15:26:16.503 INFO:teuthology.orchestra.run.smithi001.stdout:osd.9 smithi001 running (2m) 18s ago 2m 60.5M 4319M 18.2.1-325-g0fb32788 0b0faa63c314 19f81834ba26 2024-02-02T15:26:16.503 INFO:teuthology.orchestra.run.smithi001.stdout:osd.10 smithi071 running (2m) 29s ago 2m 62.2M 4278M 18.2.1-325-g0fb32788 0b0faa63c314 60ca5c0de152 2024-02-02T15:26:16.503 INFO:teuthology.orchestra.run.smithi001.stdout:osd.11 smithi142 running (2m) 28s ago 2m 58.6M 4278M 18.2.1-325-g0fb32788 0b0faa63c314 f9c6912c8529 2024-02-02T15:26:16.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:16.503+0000 7f57da600700 1 -- 172.21.15.1:0/155240730 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f57b40a0630 msgr2=0x7f57b40a2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:16.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:16.503+0000 7f57da600700 1 --2- 172.21.15.1:0/155240730 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f57b40a0630 0x7f57b40a2af0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f57c400b2e0 tx=0x7f57c4005fb0 comp rx=0 tx=0).stop 2024-02-02T15:26:16.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:16.503+0000 7f57da600700 1 -- 172.21.15.1:0/155240730 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f57d4104190 msgr2=0x7f57d4197010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:16.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:16.503+0000 7f57da600700 1 --2- 172.21.15.1:0/155240730 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f57d4104190 0x7f57d4197010 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f57bc00d9b0 tx=0x7f57bc00dcc0 comp rx=0 tx=0).stop 2024-02-02T15:26:16.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:16.503+0000 7f57da600700 1 -- 172.21.15.1:0/155240730 shutdown_connections 2024-02-02T15:26:16.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:16.503+0000 7f57da600700 1 --2- 172.21.15.1:0/155240730 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57d4106ac0 0x7f57d419e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:16.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:16.503+0000 7f57da600700 1 --2- 172.21.15.1:0/155240730 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57d4101860 0x7f57d4196ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:16.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:16.503+0000 7f57da600700 1 --2- 172.21.15.1:0/155240730 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f57b40a0630 0x7f57b40a2af0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:16.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:16.503+0000 7f57da600700 1 --2- 172.21.15.1:0/155240730 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f57d4104190 0x7f57d4197010 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:16.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:16.503+0000 7f57da600700 1 -- 172.21.15.1:0/155240730 >> 172.21.15.1:0/155240730 conn(0x7f57d40fb250 msgr2=0x7f57d41024d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:16.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:16.504+0000 7f57da600700 1 -- 172.21.15.1:0/155240730 shutdown_connections 2024-02-02T15:26:16.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:16.504+0000 7f57da600700 1 -- 172.21.15.1:0/155240730 wait complete. 2024-02-02T15:26:16.892 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- bash -c 'ceph orch ls' 2024-02-02T15:26:17.261 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.a/config 2024-02-02T15:26:17.554 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:17 smithi001 ceph-mon[35470]: pgmap v281: 129 pgs: 129 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-02-02T15:26:17.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:17 smithi071 ceph-mon[44707]: pgmap v281: 129 pgs: 129 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-02-02T15:26:17.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:17 smithi142 ceph-mon[45728]: pgmap v281: 129 pgs: 129 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-02-02T15:26:18.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:18.406+0000 7fe6f7800700 1 -- 172.21.15.1:0/2705677917 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6f0101fd0 msgr2=0x7fe6f0102430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:18.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:18.406+0000 7fe6f7800700 1 --2- 172.21.15.1:0/2705677917 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6f0101fd0 0x7fe6f0102430 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fe6d80099c0 tx=0x7fe6d8009cd0 comp rx=0 tx=0).stop 2024-02-02T15:26:18.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:18.406+0000 7fe6f7800700 1 -- 172.21.15.1:0/2705677917 shutdown_connections 2024-02-02T15:26:18.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:18.406+0000 7fe6f7800700 1 --2- 172.21.15.1:0/2705677917 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6f0102970 0x7fe6f0109220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:18.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:18.406+0000 7fe6f7800700 1 --2- 172.21.15.1:0/2705677917 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6f0100dd0 0x7fe6f01011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:18.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:18.406+0000 7fe6f7800700 1 --2- 172.21.15.1:0/2705677917 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6f0101fd0 0x7fe6f0102430 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:18.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:18.406+0000 7fe6f7800700 1 -- 172.21.15.1:0/2705677917 >> 172.21.15.1:0/2705677917 conn(0x7fe6f00fc330 msgr2=0x7fe6f00fe7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:18.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:18.406+0000 7fe6f7800700 1 -- 172.21.15.1:0/2705677917 shutdown_connections 2024-02-02T15:26:18.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:18.406+0000 7fe6f7800700 1 -- 172.21.15.1:0/2705677917 wait complete. 2024-02-02T15:26:18.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:18.407+0000 7fe6f7800700 1 Processor -- start 2024-02-02T15:26:18.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:18.408+0000 7fe6f7800700 1 -- start start 2024-02-02T15:26:18.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:18.408+0000 7fe6f7800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6f0100dd0 0x7fe6f0198cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:18.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:18.408+0000 7fe6f7800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6f0101fd0 0x7fe6f0199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:18.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:18.408+0000 7fe6f7800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6f0102970 0x7fe6f01a0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:18.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:18.408+0000 7fe6f7800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f01a0840 con 0x7fe6f0100dd0 2024-02-02T15:26:18.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:18.408+0000 7fe6f7800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f01a0980 con 0x7fe6f0102970 2024-02-02T15:26:18.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:18.408+0000 7fe6f7800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f01a0b40 con 0x7fe6f0101fd0 2024-02-02T15:26:18.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:18.408+0000 7fe6f4e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6f0100dd0 0x7fe6f0198cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:18.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:18.408+0000 7fe6f4e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6f0100dd0 0x7fe6f0198cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52578/0 (socket says 172.21.15.1:52578) 2024-02-02T15:26:18.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:18.408+0000 7fe6f4e00700 1 -- 172.21.15.1:0/4121263426 learned_addr learned my addr 172.21.15.1:0/4121263426 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:18.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:18.408+0000 7fe6f5800700 1 --2- 172.21.15.1:0/4121263426 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6f0102970 0x7fe6f01a0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:18.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:18.408+0000 7fe6efe00700 1 --2- 172.21.15.1:0/4121263426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6f0101fd0 0x7fe6f0199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:18.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:18.409+0000 7fe6f4e00700 1 -- 172.21.15.1:0/4121263426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6f0101fd0 msgr2=0x7fe6f0199230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:18.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:18.409+0000 7fe6f4e00700 1 --2- 172.21.15.1:0/4121263426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6f0101fd0 0x7fe6f0199230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:18.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:18.409+0000 7fe6f4e00700 1 -- 172.21.15.1:0/4121263426 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6f0102970 msgr2=0x7fe6f01a0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:18.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:18.409+0000 7fe6f4e00700 1 --2- 172.21.15.1:0/4121263426 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6f0102970 0x7fe6f01a0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:18.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:18.409+0000 7fe6f4e00700 1 -- 172.21.15.1:0/4121263426 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6d80095f0 con 0x7fe6f0100dd0 2024-02-02T15:26:18.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:18.409+0000 7fe6f5800700 1 --2- 172.21.15.1:0/4121263426 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6f0102970 0x7fe6f01a0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:26:18.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:18.409+0000 7fe6f4e00700 1 --2- 172.21.15.1:0/4121263426 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6f0100dd0 0x7fe6f0198cf0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe6e000e970 tx=0x7fe6e000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:18.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:18.409+0000 7fe6ed600700 1 -- 172.21.15.1:0/4121263426 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e0009790 con 0x7fe6f0100dd0 2024-02-02T15:26:18.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:18.409+0000 7fe6f7800700 1 -- 172.21.15.1:0/4121263426 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6f01a0e20 con 0x7fe6f0100dd0 2024-02-02T15:26:18.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:18.409+0000 7fe6ed600700 1 -- 172.21.15.1:0/4121263426 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe6e000c7e0 con 0x7fe6f0100dd0 2024-02-02T15:26:18.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:18.409+0000 7fe6ed600700 1 -- 172.21.15.1:0/4121263426 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e0003c70 con 0x7fe6f0100dd0 2024-02-02T15:26:18.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:18.410+0000 7fe6f7800700 1 -- 172.21.15.1:0/4121263426 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6f01a1240 con 0x7fe6f0100dd0 2024-02-02T15:26:18.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:18.410+0000 7fe6f7800700 1 -- 172.21.15.1:0/4121263426 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6f0193410 con 0x7fe6f0100dd0 2024-02-02T15:26:18.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:18.412+0000 7fe6ed600700 1 -- 172.21.15.1:0/4121263426 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 17) v1 ==== 135120+0+0 (secure 0 0 0) 0x7fe6e000cc50 con 0x7fe6f0100dd0 2024-02-02T15:26:18.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:18.414+0000 7fe6ed600700 1 --2- 172.21.15.1:0/4121263426 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fe6d00a0630 0x7fe6d00a2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:18.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:18.414+0000 7fe6ed600700 1 -- 172.21.15.1:0/4121263426 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fe6e0014070 con 0x7fe6f0100dd0 2024-02-02T15:26:18.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:18.414+0000 7fe6efe00700 1 --2- 172.21.15.1:0/4121263426 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fe6d00a0630 0x7fe6d00a2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:18.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:18.415+0000 7fe6efe00700 1 --2- 172.21.15.1:0/4121263426 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fe6d00a0630 0x7fe6d00a2af0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe6d800b580 tx=0x7fe6d8005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:18.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:18.416+0000 7fe6ed600700 1 -- 172.21.15.1:0/4121263426 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7fe6e0095810 con 0x7fe6f0100dd0 2024-02-02T15:26:18.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:18 smithi001 ceph-mon[35470]: from='client.25219 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:26:18.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:18 smithi071 ceph-mon[44707]: from='client.25219 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:26:18.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:18.635+0000 7fe6f7800700 1 -- 172.21.15.1:0/4121263426 --> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fe6f0061180 con 0x7fe6d00a0630 2024-02-02T15:26:18.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:18.641+0000 7fe6ed600700 1 -- 172.21.15.1:0/4121263426 <== mgr.14154 v2:172.21.15.1:6800/3768989981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7fe6f0061180 con 0x7fe6d00a0630 2024-02-02T15:26:18.641 INFO:teuthology.orchestra.run.smithi001.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-02-02T15:26:18.641 INFO:teuthology.orchestra.run.smithi001.stdout:mds.all 12/12 31s ago 2m smithi001=a;smithi001=d;smithi001=g;smithi001=j;smithi071=b;smithi071=e;smithi071=h;smithi071=k;smithi142=c;smithi142=f;smithi142=i;smithi142=l;count:12 2024-02-02T15:26:18.641 INFO:teuthology.orchestra.run.smithi001.stdout:mgr 3/3 31s ago 6m smithi001=x;smithi071=y;smithi142=z;count:3 2024-02-02T15:26:18.641 INFO:teuthology.orchestra.run.smithi001.stdout:mon 3/3 31s ago 6m smithi001:172.21.15.1=a;smithi071:172.21.15.71=b;smithi142:172.21.15.142=c;count:3 2024-02-02T15:26:18.641 INFO:teuthology.orchestra.run.smithi001.stdout:osd 12 31s ago - 2024-02-02T15:26:18.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:18.643+0000 7fe6f7800700 1 -- 172.21.15.1:0/4121263426 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fe6d00a0630 msgr2=0x7fe6d00a2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:18.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:18.643+0000 7fe6f7800700 1 --2- 172.21.15.1:0/4121263426 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fe6d00a0630 0x7fe6d00a2af0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe6d800b580 tx=0x7fe6d8005fb0 comp rx=0 tx=0).stop 2024-02-02T15:26:18.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:18.643+0000 7fe6f7800700 1 -- 172.21.15.1:0/4121263426 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6f0100dd0 msgr2=0x7fe6f0198cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:18.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:18.643+0000 7fe6f7800700 1 --2- 172.21.15.1:0/4121263426 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6f0100dd0 0x7fe6f0198cf0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe6e000e970 tx=0x7fe6e000ec80 comp rx=0 tx=0).stop 2024-02-02T15:26:18.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:18.643+0000 7fe6f7800700 1 -- 172.21.15.1:0/4121263426 shutdown_connections 2024-02-02T15:26:18.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:18.643+0000 7fe6f7800700 1 --2- 172.21.15.1:0/4121263426 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6f0102970 0x7fe6f01a0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:18.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:18.643+0000 7fe6f7800700 1 --2- 172.21.15.1:0/4121263426 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6f0100dd0 0x7fe6f0198cf0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:18.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:18.643+0000 7fe6f7800700 1 --2- 172.21.15.1:0/4121263426 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fe6d00a0630 0x7fe6d00a2af0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:18.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:18.643+0000 7fe6f7800700 1 --2- 172.21.15.1:0/4121263426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6f0101fd0 0x7fe6f0199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:18.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:18.643+0000 7fe6f7800700 1 -- 172.21.15.1:0/4121263426 >> 172.21.15.1:0/4121263426 conn(0x7fe6f00fc330 msgr2=0x7fe6f01078f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:18.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:18.644+0000 7fe6f7800700 1 -- 172.21.15.1:0/4121263426 shutdown_connections 2024-02-02T15:26:18.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:18.644+0000 7fe6f7800700 1 -- 172.21.15.1:0/4121263426 wait complete. 2024-02-02T15:26:18.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:18 smithi142 ceph-mon[45728]: from='client.25219 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:26:19.019 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- bash -c 'ceph orch host ls' 2024-02-02T15:26:19.387 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.a/config 2024-02-02T15:26:19.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:19 smithi001 ceph-mon[35470]: pgmap v282: 129 pgs: 129 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-02-02T15:26:19.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:19 smithi071 ceph-mon[44707]: pgmap v282: 129 pgs: 129 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-02-02T15:26:19.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:19 smithi142 ceph-mon[45728]: pgmap v282: 129 pgs: 129 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-02-02T15:26:20.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:20.587+0000 7fba2bc00700 1 -- 172.21.15.1:0/227658456 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fba24101fd0 msgr2=0x7fba24102430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:20.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:20.587+0000 7fba2bc00700 1 --2- 172.21.15.1:0/227658456 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fba24101fd0 0x7fba24102430 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fba0c0099c0 tx=0x7fba0c009cd0 comp rx=0 tx=0).stop 2024-02-02T15:26:20.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:20.587+0000 7fba2bc00700 1 -- 172.21.15.1:0/227658456 shutdown_connections 2024-02-02T15:26:20.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:20.587+0000 7fba2bc00700 1 --2- 172.21.15.1:0/227658456 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fba24102970 0x7fba24109220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:20.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:20.587+0000 7fba2bc00700 1 --2- 172.21.15.1:0/227658456 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba24100dd0 0x7fba241011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:20.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:20.587+0000 7fba2bc00700 1 --2- 172.21.15.1:0/227658456 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fba24101fd0 0x7fba24102430 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:20.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:20.587+0000 7fba2bc00700 1 -- 172.21.15.1:0/227658456 >> 172.21.15.1:0/227658456 conn(0x7fba240fc330 msgr2=0x7fba240fe7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:20.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:20.587+0000 7fba2bc00700 1 -- 172.21.15.1:0/227658456 shutdown_connections 2024-02-02T15:26:20.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:20.587+0000 7fba2bc00700 1 -- 172.21.15.1:0/227658456 wait complete. 2024-02-02T15:26:20.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:20.588+0000 7fba2bc00700 1 Processor -- start 2024-02-02T15:26:20.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:20.588+0000 7fba2bc00700 1 -- start start 2024-02-02T15:26:20.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:20.589+0000 7fba2bc00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fba24100dd0 0x7fba24074ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:20.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:20.589+0000 7fba2bc00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fba24101fd0 0x7fba24073130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:20.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:20.589+0000 7fba2bc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba24102970 0x7fba24073670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:20.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:20.589+0000 7fba2bc00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba24073df0 con 0x7fba24101fd0 2024-02-02T15:26:20.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:20.589+0000 7fba2bc00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba24073f60 con 0x7fba24100dd0 2024-02-02T15:26:20.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:20.589+0000 7fba2bc00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba241a0b50 con 0x7fba24102970 2024-02-02T15:26:20.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:20.589+0000 7fba23e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fba24101fd0 0x7fba24073130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:20.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:20.589+0000 7fba23e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fba24101fd0 0x7fba24073130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52598/0 (socket says 172.21.15.1:52598) 2024-02-02T15:26:20.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:20.589+0000 7fba23e00700 1 -- 172.21.15.1:0/516896037 learned_addr learned my addr 172.21.15.1:0/516896037 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:20.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:20.589+0000 7fba29200700 1 --2- 172.21.15.1:0/516896037 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fba24100dd0 0x7fba24074ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:20.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:20.589+0000 7fba29c00700 1 --2- 172.21.15.1:0/516896037 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba24102970 0x7fba24073670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:20.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:20.590+0000 7fba29200700 1 -- 172.21.15.1:0/516896037 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba24102970 msgr2=0x7fba24073670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:20.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:20.590+0000 7fba29200700 1 --2- 172.21.15.1:0/516896037 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba24102970 0x7fba24073670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:20.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:20.590+0000 7fba29200700 1 -- 172.21.15.1:0/516896037 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fba24101fd0 msgr2=0x7fba24073130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:20.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:20.590+0000 7fba29200700 1 --2- 172.21.15.1:0/516896037 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fba24101fd0 0x7fba24073130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:20.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:20.590+0000 7fba29200700 1 -- 172.21.15.1:0/516896037 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba0c0095f0 con 0x7fba24100dd0 2024-02-02T15:26:20.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:20.590+0000 7fba23e00700 1 --2- 172.21.15.1:0/516896037 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fba24101fd0 0x7fba24073130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:26:20.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:20.590+0000 7fba29200700 1 --2- 172.21.15.1:0/516896037 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fba24100dd0 0x7fba24074ae0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fba1400e9c0 tx=0x7fba1400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:20.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:20.590+0000 7fba21600700 1 -- 172.21.15.1:0/516896037 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba1400cbd0 con 0x7fba24100dd0 2024-02-02T15:26:20.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:20.590+0000 7fba2bc00700 1 -- 172.21.15.1:0/516896037 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba241a0e30 con 0x7fba24100dd0 2024-02-02T15:26:20.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:20.590+0000 7fba21600700 1 -- 172.21.15.1:0/516896037 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba1400eeb0 con 0x7fba24100dd0 2024-02-02T15:26:20.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:20.590+0000 7fba21600700 1 -- 172.21.15.1:0/516896037 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba14003c70 con 0x7fba24100dd0 2024-02-02T15:26:20.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:20.590+0000 7fba2bc00700 1 -- 172.21.15.1:0/516896037 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba241a1250 con 0x7fba24100dd0 2024-02-02T15:26:20.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:20.591+0000 7fba2bc00700 1 -- 172.21.15.1:0/516896037 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba2410c620 con 0x7fba24100dd0 2024-02-02T15:26:20.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:20.593+0000 7fba21600700 1 -- 172.21.15.1:0/516896037 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 135120+0+0 (secure 0 0 0) 0x7fba14003e90 con 0x7fba24100dd0 2024-02-02T15:26:20.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:20.595+0000 7fba21600700 1 --2- 172.21.15.1:0/516896037 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fba040a05b0 0x7fba040a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:20.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:20.595+0000 7fba21600700 1 -- 172.21.15.1:0/516896037 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fba14014070 con 0x7fba24100dd0 2024-02-02T15:26:20.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:20.595+0000 7fba23e00700 1 --2- 172.21.15.1:0/516896037 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fba040a05b0 0x7fba040a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:20.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:20.595+0000 7fba23e00700 1 --2- 172.21.15.1:0/516896037 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fba040a05b0 0x7fba040a2a70 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fba0c000c00 tx=0x7fba0c005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:20.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:20.599+0000 7fba21600700 1 -- 172.21.15.1:0/516896037 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7fba14095800 con 0x7fba24100dd0 2024-02-02T15:26:20.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:20 smithi071 ceph-mon[44707]: from='client.15189 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:26:20.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:20 smithi142 ceph-mon[45728]: from='client.15189 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:26:20.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:20 smithi001 ceph-mon[35470]: from='client.15189 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:26:20.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:20.814+0000 7fba2bc00700 1 -- 172.21.15.1:0/516896037 --> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fba24061180 con 0x7fba040a05b0 2024-02-02T15:26:20.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:20.816+0000 7fba21600700 1 -- 172.21.15.1:0/516896037 <== mgr.14154 v2:172.21.15.1:6800/3768989981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7fba24061180 con 0x7fba040a05b0 2024-02-02T15:26:20.816 INFO:teuthology.orchestra.run.smithi001.stdout:HOST ADDR LABELS STATUS 2024-02-02T15:26:20.816 INFO:teuthology.orchestra.run.smithi001.stdout:smithi001 172.21.15.1 2024-02-02T15:26:20.816 INFO:teuthology.orchestra.run.smithi001.stdout:smithi071 172.21.15.71 2024-02-02T15:26:20.816 INFO:teuthology.orchestra.run.smithi001.stdout:smithi142 172.21.15.142 2024-02-02T15:26:20.816 INFO:teuthology.orchestra.run.smithi001.stdout:3 hosts in cluster 2024-02-02T15:26:20.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:20.818+0000 7fba2bc00700 1 -- 172.21.15.1:0/516896037 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fba040a05b0 msgr2=0x7fba040a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:20.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:20.818+0000 7fba2bc00700 1 --2- 172.21.15.1:0/516896037 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fba040a05b0 0x7fba040a2a70 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fba0c000c00 tx=0x7fba0c005fb0 comp rx=0 tx=0).stop 2024-02-02T15:26:20.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:20.818+0000 7fba2bc00700 1 -- 172.21.15.1:0/516896037 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fba24100dd0 msgr2=0x7fba24074ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:20.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:20.818+0000 7fba2bc00700 1 --2- 172.21.15.1:0/516896037 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fba24100dd0 0x7fba24074ae0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fba1400e9c0 tx=0x7fba1400ecd0 comp rx=0 tx=0).stop 2024-02-02T15:26:20.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:20.818+0000 7fba2bc00700 1 -- 172.21.15.1:0/516896037 shutdown_connections 2024-02-02T15:26:20.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:20.818+0000 7fba2bc00700 1 --2- 172.21.15.1:0/516896037 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fba040a05b0 0x7fba040a2a70 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:20.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:20.818+0000 7fba2bc00700 1 --2- 172.21.15.1:0/516896037 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba24102970 0x7fba24073670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:20.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:20.818+0000 7fba2bc00700 1 --2- 172.21.15.1:0/516896037 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fba24100dd0 0x7fba24074ae0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:20.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:20.818+0000 7fba2bc00700 1 --2- 172.21.15.1:0/516896037 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fba24101fd0 0x7fba24073130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:20.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:20.818+0000 7fba2bc00700 1 -- 172.21.15.1:0/516896037 >> 172.21.15.1:0/516896037 conn(0x7fba240fc330 msgr2=0x7fba240fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:20.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:20.818+0000 7fba2bc00700 1 -- 172.21.15.1:0/516896037 shutdown_connections 2024-02-02T15:26:20.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:20.818+0000 7fba2bc00700 1 -- 172.21.15.1:0/516896037 wait complete. 2024-02-02T15:26:21.194 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- bash -c 'ceph orch device ls' 2024-02-02T15:26:21.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:21 smithi001 ceph-mon[35470]: pgmap v283: 129 pgs: 129 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-02-02T15:26:21.563 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.a/config 2024-02-02T15:26:21.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:21 smithi071 ceph-mon[44707]: pgmap v283: 129 pgs: 129 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-02-02T15:26:21.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:21 smithi142 ceph-mon[45728]: pgmap v283: 129 pgs: 129 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-02-02T15:26:22.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:22 smithi001 ceph-mon[35470]: from='client.25223 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:26:22.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:22 smithi071 ceph-mon[44707]: from='client.25223 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:26:22.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:22 smithi142 ceph-mon[45728]: from='client.25223 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:26:22.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:22.739+0000 7f3b27000700 1 -- 172.21.15.1:0/4117611256 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b200fffb0 msgr2=0x7f3b20100430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:22.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:22.739+0000 7f3b27000700 1 --2- 172.21.15.1:0/4117611256 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b200fffb0 0x7f3b20100430 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3b08009990 tx=0x7f3b08009ca0 comp rx=0 tx=0).stop 2024-02-02T15:26:22.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:22.739+0000 7f3b27000700 1 -- 172.21.15.1:0/4117611256 shutdown_connections 2024-02-02T15:26:22.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:22.739+0000 7f3b27000700 1 --2- 172.21.15.1:0/4117611256 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3b20100970 0x7f3b20100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:22.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:22.739+0000 7f3b27000700 1 --2- 172.21.15.1:0/4117611256 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b200ff650 0x7f3b200ffa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:22.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:22.739+0000 7f3b27000700 1 --2- 172.21.15.1:0/4117611256 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b200fffb0 0x7f3b20100430 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:22.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:22.739+0000 7f3b27000700 1 -- 172.21.15.1:0/4117611256 >> 172.21.15.1:0/4117611256 conn(0x7f3b200fb250 msgr2=0x7f3b200fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:22.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:22.739+0000 7f3b27000700 1 -- 172.21.15.1:0/4117611256 shutdown_connections 2024-02-02T15:26:22.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:22.739+0000 7f3b27000700 1 -- 172.21.15.1:0/4117611256 wait complete. 2024-02-02T15:26:22.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:22.740+0000 7f3b27000700 1 Processor -- start 2024-02-02T15:26:22.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:22.740+0000 7f3b27000700 1 -- start start 2024-02-02T15:26:22.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:22.741+0000 7f3b27000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b200ff650 0x7f3b20198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:22.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:22.741+0000 7f3b27000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3b200fffb0 0x7f3b20199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:22.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:22.741+0000 7f3b27000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b20100970 0x7f3b201a0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:22.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:22.741+0000 7f3b27000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b201a0850 con 0x7f3b200fffb0 2024-02-02T15:26:22.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:22.741+0000 7f3b27000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b201a0990 con 0x7f3b200ff650 2024-02-02T15:26:22.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:22.741+0000 7f3b27000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b201a0b50 con 0x7f3b20100970 2024-02-02T15:26:22.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:22.741+0000 7f3b1fe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b200ff650 0x7f3b20198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:22.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:22.741+0000 7f3b1f400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3b200fffb0 0x7f3b20199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:22.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:22.741+0000 7f3b1fe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b200ff650 0x7f3b20198d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:53286/0 (socket says 172.21.15.1:53286) 2024-02-02T15:26:22.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:22.741+0000 7f3b1fe00700 1 -- 172.21.15.1:0/3643230885 learned_addr learned my addr 172.21.15.1:0/3643230885 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:22.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:22.741+0000 7f3b25000700 1 --2- 172.21.15.1:0/3643230885 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b20100970 0x7f3b201a0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:22.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:22.741+0000 7f3b1fe00700 1 -- 172.21.15.1:0/3643230885 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b20100970 msgr2=0x7f3b201a0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:22.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:22.741+0000 7f3b1fe00700 1 --2- 172.21.15.1:0/3643230885 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b20100970 0x7f3b201a0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:22.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:22.741+0000 7f3b1fe00700 1 -- 172.21.15.1:0/3643230885 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3b200fffb0 msgr2=0x7f3b20199240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:22.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:22.741+0000 7f3b1fe00700 1 --2- 172.21.15.1:0/3643230885 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3b200fffb0 0x7f3b20199240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:22.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:22.741+0000 7f3b1fe00700 1 -- 172.21.15.1:0/3643230885 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b18009c40 con 0x7f3b200ff650 2024-02-02T15:26:22.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:22.742+0000 7f3b1fe00700 1 --2- 172.21.15.1:0/3643230885 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b200ff650 0x7f3b20198d00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3b1000e9c0 tx=0x7f3b1000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:22.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:22.742+0000 7f3b1cc00700 1 -- 172.21.15.1:0/3643230885 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b100097b0 con 0x7f3b200ff650 2024-02-02T15:26:22.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:22.742+0000 7f3b1cc00700 1 -- 172.21.15.1:0/3643230885 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b1000ee50 con 0x7f3b200ff650 2024-02-02T15:26:22.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:22.742+0000 7f3b27000700 1 -- 172.21.15.1:0/3643230885 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b080095f0 con 0x7f3b200ff650 2024-02-02T15:26:22.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:22.742+0000 7f3b1cc00700 1 -- 172.21.15.1:0/3643230885 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b100054f0 con 0x7f3b200ff650 2024-02-02T15:26:22.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:22.742+0000 7f3b27000700 1 -- 172.21.15.1:0/3643230885 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b201a1150 con 0x7f3b200ff650 2024-02-02T15:26:22.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:22.743+0000 7f3b27000700 1 -- 172.21.15.1:0/3643230885 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b2010a430 con 0x7f3b200ff650 2024-02-02T15:26:22.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:22.745+0000 7f3b1cc00700 1 -- 172.21.15.1:0/3643230885 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 135120+0+0 (secure 0 0 0) 0x7f3b10004750 con 0x7f3b200ff650 2024-02-02T15:26:22.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:22.746+0000 7f3b1cc00700 1 --2- 172.21.15.1:0/3643230885 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f3b000a05b0 0x7f3b000a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:22.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:22.746+0000 7f3b1cc00700 1 -- 172.21.15.1:0/3643230885 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f3b10014070 con 0x7f3b200ff650 2024-02-02T15:26:22.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:22.746+0000 7f3b1f400700 1 --2- 172.21.15.1:0/3643230885 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f3b000a05b0 0x7f3b000a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:22.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:22.747+0000 7f3b1f400700 1 --2- 172.21.15.1:0/3643230885 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f3b000a05b0 0x7f3b000a2a70 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3b0800b2e0 tx=0x7f3b08005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:22.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:22.749+0000 7f3b1cc00700 1 -- 172.21.15.1:0/3643230885 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f3b10095890 con 0x7f3b200ff650 2024-02-02T15:26:22.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:22.967+0000 7f3b27000700 1 -- 172.21.15.1:0/3643230885 --> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f3b20061180 con 0x7f3b000a05b0 2024-02-02T15:26:22.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:22.969+0000 7f3b1cc00700 1 -- 172.21.15.1:0/3643230885 <== mgr.14154 v2:172.21.15.1:6800/3768989981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+532 (secure 0 0 0) 0x7f3b20061180 con 0x7f3b000a05b0 2024-02-02T15:26:22.969 INFO:teuthology.orchestra.run.smithi001.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-02-02T15:26:22.969 INFO:teuthology.orchestra.run.smithi001.stdout:smithi001 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7513000D480DGN 447G No 2m ago Has a FileSystem, LVM detected 2024-02-02T15:26:22.969 INFO:teuthology.orchestra.run.smithi001.stdout:smithi071 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT62040172400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-02-02T15:26:22.969 INFO:teuthology.orchestra.run.smithi001.stdout:smithi142 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6303009Y400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-02-02T15:26:22.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:22.971+0000 7f3b27000700 1 -- 172.21.15.1:0/3643230885 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f3b000a05b0 msgr2=0x7f3b000a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:22.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:22.971+0000 7f3b27000700 1 --2- 172.21.15.1:0/3643230885 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f3b000a05b0 0x7f3b000a2a70 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3b0800b2e0 tx=0x7f3b08005fb0 comp rx=0 tx=0).stop 2024-02-02T15:26:22.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:22.971+0000 7f3b27000700 1 -- 172.21.15.1:0/3643230885 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b200ff650 msgr2=0x7f3b20198d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:22.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:22.971+0000 7f3b27000700 1 --2- 172.21.15.1:0/3643230885 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b200ff650 0x7f3b20198d00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3b1000e9c0 tx=0x7f3b1000ecd0 comp rx=0 tx=0).stop 2024-02-02T15:26:22.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:22.972+0000 7f3b27000700 1 -- 172.21.15.1:0/3643230885 shutdown_connections 2024-02-02T15:26:22.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:22.972+0000 7f3b27000700 1 --2- 172.21.15.1:0/3643230885 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f3b000a05b0 0x7f3b000a2a70 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:22.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:22.972+0000 7f3b27000700 1 --2- 172.21.15.1:0/3643230885 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3b20100970 0x7f3b201a0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:22.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:22.972+0000 7f3b27000700 1 --2- 172.21.15.1:0/3643230885 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3b200ff650 0x7f3b20198d00 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:22.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:22.972+0000 7f3b27000700 1 --2- 172.21.15.1:0/3643230885 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3b200fffb0 0x7f3b20199240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:22.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:22.972+0000 7f3b27000700 1 -- 172.21.15.1:0/3643230885 >> 172.21.15.1:0/3643230885 conn(0x7f3b200fb250 msgr2=0x7f3b200fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:22.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:22.972+0000 7f3b27000700 1 -- 172.21.15.1:0/3643230885 shutdown_connections 2024-02-02T15:26:22.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:22.972+0000 7f3b27000700 1 -- 172.21.15.1:0/3643230885 wait complete. 2024-02-02T15:26:23.362 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-02T15:26:23.371 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi001.front.sepia.ceph.com 2024-02-02T15:26:23.372 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- bash -c 'ceph fs dump' 2024-02-02T15:26:23.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:23 smithi071 ceph-mon[44707]: pgmap v284: 129 pgs: 129 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-02-02T15:26:23.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:23 smithi142 ceph-mon[45728]: pgmap v284: 129 pgs: 129 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-02-02T15:26:23.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:23 smithi001 ceph-mon[35470]: pgmap v284: 129 pgs: 129 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-02-02T15:26:23.741 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.a/config 2024-02-02T15:26:24.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:24 smithi071 ceph-mon[44707]: from='client.25229 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:26:24.637 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:24 smithi001 ceph-mon[35470]: from='client.25229 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:26:24.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:24 smithi142 ceph-mon[45728]: from='client.25229 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:26:24.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:24.880+0000 7f39c0e00700 1 -- 172.21.15.1:0/1581845687 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f39bc102970 msgr2=0x7f39bc109220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:24.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:24.880+0000 7f39c0e00700 1 --2- 172.21.15.1:0/1581845687 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f39bc102970 0x7f39bc109220 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f39b400b0d0 tx=0x7f39b400b3e0 comp rx=0 tx=0).stop 2024-02-02T15:26:24.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:24.880+0000 7f39c0e00700 1 -- 172.21.15.1:0/1581845687 shutdown_connections 2024-02-02T15:26:24.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:24.880+0000 7f39c0e00700 1 --2- 172.21.15.1:0/1581845687 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f39bc102970 0x7f39bc109220 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:24.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:24.880+0000 7f39c0e00700 1 --2- 172.21.15.1:0/1581845687 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f39bc100dd0 0x7f39bc1011f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:24.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:24.880+0000 7f39c0e00700 1 --2- 172.21.15.1:0/1581845687 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39bc101fd0 0x7f39bc102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:24.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:24.880+0000 7f39c0e00700 1 -- 172.21.15.1:0/1581845687 >> 172.21.15.1:0/1581845687 conn(0x7f39bc0fc330 msgr2=0x7f39bc0fe7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:24.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:24.880+0000 7f39c0e00700 1 -- 172.21.15.1:0/1581845687 shutdown_connections 2024-02-02T15:26:24.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:24.880+0000 7f39c0e00700 1 -- 172.21.15.1:0/1581845687 wait complete. 2024-02-02T15:26:24.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:24.881+0000 7f39c0e00700 1 Processor -- start 2024-02-02T15:26:24.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:24.882+0000 7f39c0e00700 1 -- start start 2024-02-02T15:26:24.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:24.882+0000 7f39c0e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f39bc100dd0 0x7f39bc106b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:24.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:24.882+0000 7f39c0e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39bc101fd0 0x7f39bc107050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:24.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:24.882+0000 7f39c0e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f39bc102970 0x7f39bc103690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:24.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:24.882+0000 7f39c0e00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39bc103e30 con 0x7f39bc102970 2024-02-02T15:26:24.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:24.882+0000 7f39c0e00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39bc103fa0 con 0x7f39bc100dd0 2024-02-02T15:26:24.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:24.882+0000 7f39c0e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39bc104160 con 0x7f39bc101fd0 2024-02-02T15:26:24.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:24.882+0000 7f39b9e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f39bc100dd0 0x7f39bc106b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:24.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:24.882+0000 7f39ba800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f39bc102970 0x7f39bc103690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:24.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:24.882+0000 7f39b9400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39bc101fd0 0x7f39bc107050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:24.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:24.882+0000 7f39b9e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f39bc100dd0 0x7f39bc106b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:35930/0 (socket says 172.21.15.1:35930) 2024-02-02T15:26:24.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:24.882+0000 7f39b9e00700 1 -- 172.21.15.1:0/3552751534 learned_addr learned my addr 172.21.15.1:0/3552751534 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:24.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:24.883+0000 7f39b9e00700 1 -- 172.21.15.1:0/3552751534 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39bc101fd0 msgr2=0x7f39bc107050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:24.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:24.883+0000 7f39b9e00700 1 --2- 172.21.15.1:0/3552751534 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39bc101fd0 0x7f39bc107050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:24.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:24.883+0000 7f39b9e00700 1 -- 172.21.15.1:0/3552751534 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f39bc102970 msgr2=0x7f39bc103690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:24.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:24.883+0000 7f39b9e00700 1 --2- 172.21.15.1:0/3552751534 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f39bc102970 0x7f39bc103690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:24.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:24.883+0000 7f39b9e00700 1 -- 172.21.15.1:0/3552751534 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39b4009d00 con 0x7f39bc100dd0 2024-02-02T15:26:24.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:24.883+0000 7f39ba800700 1 --2- 172.21.15.1:0/3552751534 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f39bc102970 0x7f39bc103690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:26:24.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:24.883+0000 7f39b9e00700 1 --2- 172.21.15.1:0/3552751534 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f39bc100dd0 0x7f39bc106b10 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f39ac00e970 tx=0x7f39ac00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:24.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:24.883+0000 7f39b2a00700 1 -- 172.21.15.1:0/3552751534 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ac009790 con 0x7f39bc100dd0 2024-02-02T15:26:24.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:24.883+0000 7f39b2a00700 1 -- 172.21.15.1:0/3552751534 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39ac00c7e0 con 0x7f39bc100dd0 2024-02-02T15:26:24.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:24.883+0000 7f39c0e00700 1 -- 172.21.15.1:0/3552751534 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39bc1a52f0 con 0x7f39bc100dd0 2024-02-02T15:26:24.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:24.883+0000 7f39b2a00700 1 -- 172.21.15.1:0/3552751534 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ac003c70 con 0x7f39bc100dd0 2024-02-02T15:26:24.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:24.883+0000 7f39c0e00700 1 -- 172.21.15.1:0/3552751534 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39bc1a57c0 con 0x7f39bc100dd0 2024-02-02T15:26:24.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:24.884+0000 7f39c0e00700 1 -- 172.21.15.1:0/3552751534 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39bc193420 con 0x7f39bc100dd0 2024-02-02T15:26:24.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:24.887+0000 7f39b2a00700 1 -- 172.21.15.1:0/3552751534 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) v1 ==== 135120+0+0 (secure 0 0 0) 0x7f39ac00cc50 con 0x7f39bc100dd0 2024-02-02T15:26:24.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:24.888+0000 7f39b2a00700 1 --2- 172.21.15.1:0/3552751534 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f399c0a05b0 0x7f399c0a2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:24.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:24.888+0000 7f39b2a00700 1 -- 172.21.15.1:0/3552751534 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f39ac014070 con 0x7f39bc100dd0 2024-02-02T15:26:24.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:24.888+0000 7f39b9400700 1 --2- 172.21.15.1:0/3552751534 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f399c0a05b0 0x7f399c0a2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:24.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:24.889+0000 7f39b9400700 1 --2- 172.21.15.1:0/3552751534 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f399c0a05b0 0x7f399c0a2a70 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f39a4009c60 tx=0x7f39a4009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:24.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:24.890+0000 7f39b2a00700 1 -- 172.21.15.1:0/3552751534 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f39ac095870 con 0x7f39bc100dd0 2024-02-02T15:26:25.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:25.142+0000 7f39c0e00700 1 -- 172.21.15.1:0/3552751534 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f39bc02cc70 con 0x7f39bc100dd0 2024-02-02T15:26:25.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:25.142+0000 7f39b2a00700 1 -- 172.21.15.1:0/3552751534 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) v1 ==== 76+0+2911 (secure 0 0 0) 0x7f39ac003dd0 con 0x7f39bc100dd0 2024-02-02T15:26:25.142 INFO:teuthology.orchestra.run.smithi001.stdout:e27 2024-02-02T15:26:25.143 INFO:teuthology.orchestra.run.smithi001.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-02-02T15:26:25.143 INFO:teuthology.orchestra.run.smithi001.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-02T15:26:25.143 INFO:teuthology.orchestra.run.smithi001.stdout:legacy client fscid: 1 2024-02-02T15:26:25.143 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:26:25.143 INFO:teuthology.orchestra.run.smithi001.stdout:Filesystem 'cephfs' (1) 2024-02-02T15:26:25.143 INFO:teuthology.orchestra.run.smithi001.stdout:fs_name cephfs 2024-02-02T15:26:25.143 INFO:teuthology.orchestra.run.smithi001.stdout:epoch 27 2024-02-02T15:26:25.143 INFO:teuthology.orchestra.run.smithi001.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-02-02T15:26:25.143 INFO:teuthology.orchestra.run.smithi001.stdout:created 2024-02-02T15:24:47.761435+0000 2024-02-02T15:26:25.143 INFO:teuthology.orchestra.run.smithi001.stdout:modified 2024-02-02T15:25:20.251601+0000 2024-02-02T15:26:25.143 INFO:teuthology.orchestra.run.smithi001.stdout:tableserver 0 2024-02-02T15:26:25.143 INFO:teuthology.orchestra.run.smithi001.stdout:root 0 2024-02-02T15:26:25.143 INFO:teuthology.orchestra.run.smithi001.stdout:session_timeout 300 2024-02-02T15:26:25.143 INFO:teuthology.orchestra.run.smithi001.stdout:session_autoclose 300 2024-02-02T15:26:25.143 INFO:teuthology.orchestra.run.smithi001.stdout:max_file_size 1099511627776 2024-02-02T15:26:25.143 INFO:teuthology.orchestra.run.smithi001.stdout:max_xattr_size 65536 2024-02-02T15:26:25.143 INFO:teuthology.orchestra.run.smithi001.stdout:required_client_features {} 2024-02-02T15:26:25.144 INFO:teuthology.orchestra.run.smithi001.stdout:last_failure 0 2024-02-02T15:26:25.144 INFO:teuthology.orchestra.run.smithi001.stdout:last_failure_osd_epoch 0 2024-02-02T15:26:25.144 INFO:teuthology.orchestra.run.smithi001.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-02T15:26:25.144 INFO:teuthology.orchestra.run.smithi001.stdout:max_mds 5 2024-02-02T15:26:25.144 INFO:teuthology.orchestra.run.smithi001.stdout:in 0,1,2,3,4 2024-02-02T15:26:25.144 INFO:teuthology.orchestra.run.smithi001.stdout:up {0=24557,1=24529,2=24518,3=24514,4=24502} 2024-02-02T15:26:25.144 INFO:teuthology.orchestra.run.smithi001.stdout:failed 2024-02-02T15:26:25.144 INFO:teuthology.orchestra.run.smithi001.stdout:damaged 2024-02-02T15:26:25.144 INFO:teuthology.orchestra.run.smithi001.stdout:stopped 2024-02-02T15:26:25.144 INFO:teuthology.orchestra.run.smithi001.stdout:data_pools [3,4] 2024-02-02T15:26:25.144 INFO:teuthology.orchestra.run.smithi001.stdout:metadata_pool 2 2024-02-02T15:26:25.144 INFO:teuthology.orchestra.run.smithi001.stdout:inline_data disabled 2024-02-02T15:26:25.144 INFO:teuthology.orchestra.run.smithi001.stdout:balancer 2024-02-02T15:26:25.144 INFO:teuthology.orchestra.run.smithi001.stdout:bal_rank_mask -1 2024-02-02T15:26:25.145 INFO:teuthology.orchestra.run.smithi001.stdout:standby_count_wanted 0 2024-02-02T15:26:25.145 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.b{0:24557} state up:active seq 3 addr [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] compat {c=[1],r=[1],i=[7ff]}] 2024-02-02T15:26:25.145 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.e{0:24494} state up:standby-replay seq 1 addr [v2:172.21.15.71:6834/388207814,v1:172.21.15.71:6835/388207814] compat {c=[1],r=[1],i=[7ff]}] 2024-02-02T15:26:25.145 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.i{1:24529} state up:active seq 12 addr [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] compat {c=[1],r=[1],i=[7ff]}] 2024-02-02T15:26:25.145 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.k{1:24458} state up:standby-replay seq 1 addr [v2:172.21.15.71:6832/3482746094,v1:172.21.15.71:6833/3482746094] compat {c=[1],r=[1],i=[7ff]}] 2024-02-02T15:26:25.145 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.h{2:24518} state up:active seq 13 addr [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] compat {c=[1],r=[1],i=[7ff]}] 2024-02-02T15:26:25.145 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.l{2:24457} state up:standby-replay seq 1 addr [v2:172.21.15.142:6832/7648986,v1:172.21.15.142:6833/7648986] compat {c=[1],r=[1],i=[7ff]}] 2024-02-02T15:26:25.145 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.c{3:24514} state up:active seq 13 addr [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] compat {c=[1],r=[1],i=[7ff]}] 2024-02-02T15:26:25.145 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.a{3:14529} state up:standby-replay seq 1 addr [v2:172.21.15.1:6840/2059197496,v1:172.21.15.1:6841/2059197496] compat {c=[1],r=[1],i=[7ff]}] 2024-02-02T15:26:25.145 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.f{4:24502} state up:active seq 14 addr [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] compat {c=[1],r=[1],i=[7ff]}] 2024-02-02T15:26:25.145 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.g{4:14517} state up:standby-replay seq 1 addr [v2:172.21.15.1:6838/1218303683,v1:172.21.15.1:6839/1218303683] compat {c=[1],r=[1],i=[7ff]}] 2024-02-02T15:26:25.145 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:26:25.145 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:26:25.145 INFO:teuthology.orchestra.run.smithi001.stdout:Standby daemons: 2024-02-02T15:26:25.146 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:26:25.146 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.j{-1:14457} state up:standby seq 1 addr [v2:172.21.15.1:6834/2922240448,v1:172.21.15.1:6835/2922240448] compat {c=[1],r=[1],i=[7ff]}] 2024-02-02T15:26:25.146 INFO:teuthology.orchestra.run.smithi001.stdout:[mds.d{-1:14469} state up:standby seq 1 addr [v2:172.21.15.1:6836/2729497431,v1:172.21.15.1:6837/2729497431] compat {c=[1],r=[1],i=[7ff]}] 2024-02-02T15:26:25.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:25.145+0000 7f39c0e00700 1 -- 172.21.15.1:0/3552751534 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f399c0a05b0 msgr2=0x7f399c0a2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:25.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:25.145+0000 7f39c0e00700 1 --2- 172.21.15.1:0/3552751534 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f399c0a05b0 0x7f399c0a2a70 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f39a4009c60 tx=0x7f39a4009260 comp rx=0 tx=0).stop 2024-02-02T15:26:25.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:25.145+0000 7f39c0e00700 1 -- 172.21.15.1:0/3552751534 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f39bc100dd0 msgr2=0x7f39bc106b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:25.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:25.145+0000 7f39c0e00700 1 --2- 172.21.15.1:0/3552751534 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f39bc100dd0 0x7f39bc106b10 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f39ac00e970 tx=0x7f39ac00ec80 comp rx=0 tx=0).stop 2024-02-02T15:26:25.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:25.145+0000 7f39c0e00700 1 -- 172.21.15.1:0/3552751534 shutdown_connections 2024-02-02T15:26:25.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:25.145+0000 7f39c0e00700 1 --2- 172.21.15.1:0/3552751534 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f39bc102970 0x7f39bc103690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:25.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:25.145+0000 7f39c0e00700 1 --2- 172.21.15.1:0/3552751534 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f39bc100dd0 0x7f39bc106b10 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:25.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:25.145+0000 7f39c0e00700 1 --2- 172.21.15.1:0/3552751534 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f399c0a05b0 0x7f399c0a2a70 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:25.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:25.145+0000 7f39c0e00700 1 --2- 172.21.15.1:0/3552751534 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39bc101fd0 0x7f39bc107050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:25.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:25.145+0000 7f39c0e00700 1 -- 172.21.15.1:0/3552751534 >> 172.21.15.1:0/3552751534 conn(0x7f39bc0fc330 msgr2=0x7f39bc107880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:25.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:25.145+0000 7f39c0e00700 1 -- 172.21.15.1:0/3552751534 shutdown_connections 2024-02-02T15:26:25.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:25.145+0000 7f39c0e00700 1 -- 172.21.15.1:0/3552751534 wait complete. 2024-02-02T15:26:25.147 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 27 2024-02-02T15:26:25.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:25 smithi001 ceph-mon[35470]: pgmap v285: 129 pgs: 129 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-02-02T15:26:25.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:25 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/3552751534' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-02T15:26:25.510 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- bash -c 'ceph osd dump' 2024-02-02T15:26:25.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:25 smithi071 ceph-mon[44707]: pgmap v285: 129 pgs: 129 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-02-02T15:26:25.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:25 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/3552751534' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-02T15:26:25.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:25 smithi142 ceph-mon[45728]: pgmap v285: 129 pgs: 129 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-02-02T15:26:25.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:25 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/3552751534' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-02-02T15:26:25.876 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.a/config 2024-02-02T15:26:27.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:27.186+0000 7f4500c00700 1 -- 172.21.15.1:0/347844646 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f44fc106ac0 msgr2=0x7f44fc108f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:27.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:27.186+0000 7f4500c00700 1 --2- 172.21.15.1:0/347844646 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f44fc106ac0 0x7f44fc108f30 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f44f400b0a0 tx=0x7f44f400b3b0 comp rx=0 tx=0).stop 2024-02-02T15:26:27.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:27.186+0000 7f4500c00700 1 -- 172.21.15.1:0/347844646 shutdown_connections 2024-02-02T15:26:27.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:27.186+0000 7f4500c00700 1 --2- 172.21.15.1:0/347844646 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f44fc106ac0 0x7f44fc108f30 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:27.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:27.186+0000 7f4500c00700 1 --2- 172.21.15.1:0/347844646 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44fc101860 0x7f44fc103c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:27.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:27.186+0000 7f4500c00700 1 --2- 172.21.15.1:0/347844646 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44fc104190 0x7f44fc106580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:27.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:27.186+0000 7f4500c00700 1 -- 172.21.15.1:0/347844646 >> 172.21.15.1:0/347844646 conn(0x7f44fc0fb250 msgr2=0x7f44fc0fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:27.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:27.186+0000 7f4500c00700 1 -- 172.21.15.1:0/347844646 shutdown_connections 2024-02-02T15:26:27.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:27.186+0000 7f4500c00700 1 -- 172.21.15.1:0/347844646 wait complete. 2024-02-02T15:26:27.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:27.187+0000 7f4500c00700 1 Processor -- start 2024-02-02T15:26:27.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:27.187+0000 7f4500c00700 1 -- start start 2024-02-02T15:26:27.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:27.188+0000 7f4500c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44fc101860 0x7f44fc1011c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:27.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:27.188+0000 7f4500c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44fc104190 0x7f44fc0ff810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:27.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:27.188+0000 7f4500c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f44fc106ac0 0x7f44fc0ffd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:27.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:27.188+0000 7f4500c00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44fc19aa10 con 0x7f44fc106ac0 2024-02-02T15:26:27.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:27.188+0000 7f4500c00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44fc100290 con 0x7f44fc101860 2024-02-02T15:26:27.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:27.188+0000 7f4500c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44fc100450 con 0x7f44fc104190 2024-02-02T15:26:27.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:27.188+0000 7f44f9400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44fc104190 0x7f44fc0ff810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:27.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:27.188+0000 7f44f9400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44fc104190 0x7f44fc0ff810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:35688/0 (socket says 172.21.15.1:35688) 2024-02-02T15:26:27.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:27.188+0000 7f44f9400700 1 -- 172.21.15.1:0/2799108861 learned_addr learned my addr 172.21.15.1:0/2799108861 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:27.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:27.188+0000 7f44fa800700 1 --2- 172.21.15.1:0/2799108861 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f44fc106ac0 0x7f44fc0ffd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:27.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:27.188+0000 7f44f9e00700 1 --2- 172.21.15.1:0/2799108861 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44fc101860 0x7f44fc1011c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:27.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:27.188+0000 7f44f9400700 1 -- 172.21.15.1:0/2799108861 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44fc101860 msgr2=0x7f44fc1011c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:27.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:27.188+0000 7f44f9400700 1 --2- 172.21.15.1:0/2799108861 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44fc101860 0x7f44fc1011c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:27.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:27.188+0000 7f44f9400700 1 -- 172.21.15.1:0/2799108861 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f44fc106ac0 msgr2=0x7f44fc0ffd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:27.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:27.188+0000 7f44f9400700 1 --2- 172.21.15.1:0/2799108861 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f44fc106ac0 0x7f44fc0ffd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:27.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:27.188+0000 7f44f9400700 1 -- 172.21.15.1:0/2799108861 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44ec009520 con 0x7f44fc104190 2024-02-02T15:26:27.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:27.189+0000 7f44f9400700 1 --2- 172.21.15.1:0/2799108861 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44fc104190 0x7f44fc0ff810 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f44e400e9a0 tx=0x7f44e400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:27.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:27.189+0000 7f44f2a00700 1 -- 172.21.15.1:0/2799108861 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e4009790 con 0x7f44fc104190 2024-02-02T15:26:27.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:27.189+0000 7f4500c00700 1 -- 172.21.15.1:0/2799108861 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44f4009d00 con 0x7f44fc104190 2024-02-02T15:26:27.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:27.189+0000 7f44f2a00700 1 -- 172.21.15.1:0/2799108861 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f44e4010430 con 0x7f44fc104190 2024-02-02T15:26:27.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:27.189+0000 7f44f2a00700 1 -- 172.21.15.1:0/2799108861 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e4018680 con 0x7f44fc104190 2024-02-02T15:26:27.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:27.189+0000 7f4500c00700 1 -- 172.21.15.1:0/2799108861 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44fc1a56c0 con 0x7f44fc104190 2024-02-02T15:26:27.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:27.190+0000 7f4500c00700 1 -- 172.21.15.1:0/2799108861 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44fc0fcdd0 con 0x7f44fc104190 2024-02-02T15:26:27.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:27.192+0000 7f44f2a00700 1 -- 172.21.15.1:0/2799108861 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135069+0+0 (secure 0 0 0) 0x7f44e40105a0 con 0x7f44fc104190 2024-02-02T15:26:27.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:27.194+0000 7f44f2a00700 1 --2- 172.21.15.1:0/2799108861 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f44dc0a05a0 0x7f44dc0a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:27.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:27.194+0000 7f44f2a00700 1 -- 172.21.15.1:0/2799108861 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f44e4014070 con 0x7f44fc104190 2024-02-02T15:26:27.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:27.194+0000 7f44f9e00700 1 --2- 172.21.15.1:0/2799108861 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f44dc0a05a0 0x7f44dc0a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:27.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:27.194+0000 7f44f9e00700 1 --2- 172.21.15.1:0/2799108861 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f44dc0a05a0 0x7f44dc0a2a60 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f44ec009df0 tx=0x7f44ec009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:27.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:27.196+0000 7f44f2a00700 1 -- 172.21.15.1:0/2799108861 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f44e40958b0 con 0x7f44fc104190 2024-02-02T15:26:27.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:27.410+0000 7f4500c00700 1 -- 172.21.15.1:0/2799108861 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7f44fc02cec0 con 0x7f44fc104190 2024-02-02T15:26:27.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:27.411+0000 7f44f2a00700 1 -- 172.21.15.1:0/2799108861 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v87) v1 ==== 56+0+5632 (secure 0 0 0) 0x7f44e4026020 con 0x7f44fc104190 2024-02-02T15:26:27.411 INFO:teuthology.orchestra.run.smithi001.stdout:epoch 87 2024-02-02T15:26:27.411 INFO:teuthology.orchestra.run.smithi001.stdout:fsid 371a409c-c1de-11ee-95b6-87774f69a715 2024-02-02T15:26:27.411 INFO:teuthology.orchestra.run.smithi001.stdout:created 2024-02-02T15:18:26.746506+0000 2024-02-02T15:26:27.411 INFO:teuthology.orchestra.run.smithi001.stdout:modified 2024-02-02T15:26:27.264632+0000 2024-02-02T15:26:27.411 INFO:teuthology.orchestra.run.smithi001.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-02-02T15:26:27.411 INFO:teuthology.orchestra.run.smithi001.stdout:crush_version 27 2024-02-02T15:26:27.411 INFO:teuthology.orchestra.run.smithi001.stdout:full_ratio 0.95 2024-02-02T15:26:27.412 INFO:teuthology.orchestra.run.smithi001.stdout:backfillfull_ratio 0.9 2024-02-02T15:26:27.412 INFO:teuthology.orchestra.run.smithi001.stdout:nearfull_ratio 0.85 2024-02-02T15:26:27.412 INFO:teuthology.orchestra.run.smithi001.stdout:require_min_compat_client luminous 2024-02-02T15:26:27.412 INFO:teuthology.orchestra.run.smithi001.stdout:min_compat_client luminous 2024-02-02T15:26:27.412 INFO:teuthology.orchestra.run.smithi001.stdout:require_osd_release reef 2024-02-02T15:26:27.412 INFO:teuthology.orchestra.run.smithi001.stdout:stretch_mode_enabled false 2024-02-02T15:26:27.412 INFO:teuthology.orchestra.run.smithi001.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-02-02T15:26:27.412 INFO:teuthology.orchestra.run.smithi001.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 79 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2024-02-02T15:26:27.412 INFO:teuthology.orchestra.run.smithi001.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 80 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-02-02T15:26:27.412 INFO:teuthology.orchestra.run.smithi001.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 84 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-02-02T15:26:27.412 INFO:teuthology.orchestra.run.smithi001.stdout:max_osd 12 2024-02-02T15:26:27.412 INFO:teuthology.orchestra.run.smithi001.stdout:osd.0 up in weight 1 up_from 8 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.1:6802/2591906423,v1:172.21.15.1:6803/2591906423] [v2:172.21.15.1:6804/2591906423,v1:172.21.15.1:6805/2591906423] exists,up 73e8a07f-3560-4c19-85ed-030d4d8f0f23 2024-02-02T15:26:27.413 INFO:teuthology.orchestra.run.smithi001.stdout:osd.1 up in weight 1 up_from 13 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.71:6800/747581719,v1:172.21.15.71:6801/747581719] [v2:172.21.15.71:6802/747581719,v1:172.21.15.71:6803/747581719] exists,up e53cffec-0ff9-4ed9-8dcf-ff47d04db2bb 2024-02-02T15:26:27.413 INFO:teuthology.orchestra.run.smithi001.stdout:osd.2 up in weight 1 up_from 18 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.142:6800/377511414,v1:172.21.15.142:6801/377511414] [v2:172.21.15.142:6802/377511414,v1:172.21.15.142:6803/377511414] exists,up 856c2347-294c-482b-8595-ad77f4792db4 2024-02-02T15:26:27.413 INFO:teuthology.orchestra.run.smithi001.stdout:osd.3 up in weight 1 up_from 24 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.1:6810/1086124477,v1:172.21.15.1:6811/1086124477] [v2:172.21.15.1:6812/1086124477,v1:172.21.15.1:6813/1086124477] exists,up 4b946ad2-2da9-4e0b-9d61-d039d8669a42 2024-02-02T15:26:27.413 INFO:teuthology.orchestra.run.smithi001.stdout:osd.4 up in weight 1 up_from 31 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.71:6808/3095380305,v1:172.21.15.71:6809/3095380305] [v2:172.21.15.71:6810/3095380305,v1:172.21.15.71:6811/3095380305] exists,up 43e36c43-47a0-4634-a45b-1da14a0e19d0 2024-02-02T15:26:27.413 INFO:teuthology.orchestra.run.smithi001.stdout:osd.5 up in weight 1 up_from 36 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.142:6808/4195149262,v1:172.21.15.142:6809/4195149262] [v2:172.21.15.142:6810/4195149262,v1:172.21.15.142:6811/4195149262] exists,up eb48160e-ea7c-42f7-9477-be110bc8a34e 2024-02-02T15:26:27.413 INFO:teuthology.orchestra.run.smithi001.stdout:osd.6 up in weight 1 up_from 43 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.1:6818/1560153428,v1:172.21.15.1:6819/1560153428] [v2:172.21.15.1:6820/1560153428,v1:172.21.15.1:6821/1560153428] exists,up 0ec4501a-5853-4ca6-b325-231f5b72335e 2024-02-02T15:26:27.413 INFO:teuthology.orchestra.run.smithi001.stdout:osd.7 up in weight 1 up_from 48 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.71:6816/4080955538,v1:172.21.15.71:6817/4080955538] [v2:172.21.15.71:6818/4080955538,v1:172.21.15.71:6819/4080955538] exists,up 029debe8-9cdf-44f6-9aea-cabd1d185f93 2024-02-02T15:26:27.413 INFO:teuthology.orchestra.run.smithi001.stdout:osd.8 up in weight 1 up_from 55 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.142:6816/3942038209,v1:172.21.15.142:6817/3942038209] [v2:172.21.15.142:6818/3942038209,v1:172.21.15.142:6819/3942038209] exists,up 7c90896c-9955-4c1b-aaca-7531dd5457f6 2024-02-02T15:26:27.413 INFO:teuthology.orchestra.run.smithi001.stdout:osd.9 up in weight 1 up_from 62 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.1:6826/2716544850,v1:172.21.15.1:6827/2716544850] [v2:172.21.15.1:6828/2716544850,v1:172.21.15.1:6829/2716544850] exists,up 7e165e68-31a8-4598-a577-de3e90ae5c5b 2024-02-02T15:26:27.413 INFO:teuthology.orchestra.run.smithi001.stdout:osd.10 up in weight 1 up_from 67 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.71:6824/84691841,v1:172.21.15.71:6825/84691841] [v2:172.21.15.71:6826/84691841,v1:172.21.15.71:6827/84691841] exists,up 48cd5fdb-0887-444a-aab8-a1de24282bb8 2024-02-02T15:26:27.413 INFO:teuthology.orchestra.run.smithi001.stdout:osd.11 up in weight 1 up_from 75 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.142:6824/2290179934,v1:172.21.15.142:6825/2290179934] [v2:172.21.15.142:6826/2290179934,v1:172.21.15.142:6827/2290179934] exists,up f011dd27-11b0-4166-adfe-76756533163a 2024-02-02T15:26:27.413 INFO:teuthology.orchestra.run.smithi001.stdout:pg_upmap_items 2.0 [8,1] 2024-02-02T15:26:27.413 INFO:teuthology.orchestra.run.smithi001.stdout:pg_upmap_items 2.a [2,11] 2024-02-02T15:26:27.413 INFO:teuthology.orchestra.run.smithi001.stdout:pg_upmap_items 2.13 [2,11] 2024-02-02T15:26:27.413 INFO:teuthology.orchestra.run.smithi001.stdout:pg_upmap_items 3.0 [4,8] 2024-02-02T15:26:27.414 INFO:teuthology.orchestra.run.smithi001.stdout:pg_upmap_items 3.14 [4,7] 2024-02-02T15:26:27.414 INFO:teuthology.orchestra.run.smithi001.stdout:pg_upmap_items 3.1d [5,9] 2024-02-02T15:26:27.414 INFO:teuthology.orchestra.run.smithi001.stdout:pg_upmap_items 3.24 [4,9] 2024-02-02T15:26:27.414 INFO:teuthology.orchestra.run.smithi001.stdout:pg_upmap_items 3.28 [5,6] 2024-02-02T15:26:27.414 INFO:teuthology.orchestra.run.smithi001.stdout:pg_upmap_items 3.29 [3,9] 2024-02-02T15:26:27.414 INFO:teuthology.orchestra.run.smithi001.stdout:pg_upmap_items 4.e [9,11] 2024-02-02T15:26:27.414 INFO:teuthology.orchestra.run.smithi001.stdout:pg_upmap_items 4.f [9,8] 2024-02-02T15:26:27.414 INFO:teuthology.orchestra.run.smithi001.stdout:pg_upmap_items 4.12 [5,10] 2024-02-02T15:26:27.414 INFO:teuthology.orchestra.run.smithi001.stdout:pg_upmap_items 4.16 [9,8] 2024-02-02T15:26:27.414 INFO:teuthology.orchestra.run.smithi001.stdout:blocklist 172.21.15.1:0/4092532060 expires 2024-02-03T15:19:25.836171+0000 2024-02-02T15:26:27.414 INFO:teuthology.orchestra.run.smithi001.stdout:blocklist 172.21.15.1:6801/395498296 expires 2024-02-03T15:19:25.836171+0000 2024-02-02T15:26:27.414 INFO:teuthology.orchestra.run.smithi001.stdout:blocklist 172.21.15.1:6800/395498296 expires 2024-02-03T15:19:25.836171+0000 2024-02-02T15:26:27.414 INFO:teuthology.orchestra.run.smithi001.stdout:blocklist 172.21.15.1:0/1575373489 expires 2024-02-03T15:19:25.836171+0000 2024-02-02T15:26:27.414 INFO:teuthology.orchestra.run.smithi001.stdout:blocklist 172.21.15.1:0/865983374 expires 2024-02-03T15:19:01.431596+0000 2024-02-02T15:26:27.414 INFO:teuthology.orchestra.run.smithi001.stdout:blocklist 172.21.15.1:0/3165960227 expires 2024-02-03T15:19:01.431596+0000 2024-02-02T15:26:27.415 INFO:teuthology.orchestra.run.smithi001.stdout:blocklist 172.21.15.1:0/2350830293 expires 2024-02-03T15:19:01.431596+0000 2024-02-02T15:26:27.415 INFO:teuthology.orchestra.run.smithi001.stdout:blocklist 172.21.15.1:0/2959690250 expires 2024-02-03T15:19:25.836171+0000 2024-02-02T15:26:27.415 INFO:teuthology.orchestra.run.smithi001.stdout:blocklist 172.21.15.1:6800/1162530732 expires 2024-02-03T15:19:01.431596+0000 2024-02-02T15:26:27.415 INFO:teuthology.orchestra.run.smithi001.stdout:blocklist 172.21.15.1:6801/1162530732 expires 2024-02-03T15:19:01.431596+0000 2024-02-02T15:26:27.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:27.413+0000 7f4500c00700 1 -- 172.21.15.1:0/2799108861 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f44dc0a05a0 msgr2=0x7f44dc0a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:27.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:27.413+0000 7f4500c00700 1 --2- 172.21.15.1:0/2799108861 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f44dc0a05a0 0x7f44dc0a2a60 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f44ec009df0 tx=0x7f44ec009260 comp rx=0 tx=0).stop 2024-02-02T15:26:27.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:27.413+0000 7f4500c00700 1 -- 172.21.15.1:0/2799108861 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44fc104190 msgr2=0x7f44fc0ff810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:27.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:27.413+0000 7f4500c00700 1 --2- 172.21.15.1:0/2799108861 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44fc104190 0x7f44fc0ff810 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f44e400e9a0 tx=0x7f44e400ecb0 comp rx=0 tx=0).stop 2024-02-02T15:26:27.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:27.413+0000 7f4500c00700 1 -- 172.21.15.1:0/2799108861 shutdown_connections 2024-02-02T15:26:27.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:27.413+0000 7f4500c00700 1 --2- 172.21.15.1:0/2799108861 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f44fc106ac0 0x7f44fc0ffd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:27.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:27.413+0000 7f4500c00700 1 --2- 172.21.15.1:0/2799108861 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44fc101860 0x7f44fc1011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:27.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:27.413+0000 7f4500c00700 1 --2- 172.21.15.1:0/2799108861 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f44dc0a05a0 0x7f44dc0a2a60 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:27.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:27.413+0000 7f4500c00700 1 --2- 172.21.15.1:0/2799108861 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44fc104190 0x7f44fc0ff810 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:27.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:27.413+0000 7f4500c00700 1 -- 172.21.15.1:0/2799108861 >> 172.21.15.1:0/2799108861 conn(0x7f44fc0fb250 msgr2=0x7f44fc1024d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:27.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:27.413+0000 7f4500c00700 1 -- 172.21.15.1:0/2799108861 shutdown_connections 2024-02-02T15:26:27.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:27.414+0000 7f4500c00700 1 -- 172.21.15.1:0/2799108861 wait complete. 2024-02-02T15:26:27.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:27 smithi071 ceph-mon[44707]: pgmap v286: 129 pgs: 129 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-02-02T15:26:27.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:27 smithi071 ceph-mon[44707]: mgrmap e18: x(active, since 7m), standbys: z, y 2024-02-02T15:26:27.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:27 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 7]}]: dispatch 2024-02-02T15:26:27.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:27 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]: dispatch 2024-02-02T15:26:27.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:27 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]: dispatch 2024-02-02T15:26:27.616 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:27 smithi001 ceph-mon[35470]: pgmap v286: 129 pgs: 129 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-02-02T15:26:27.616 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:27 smithi001 ceph-mon[35470]: mgrmap e18: x(active, since 7m), standbys: z, y 2024-02-02T15:26:27.616 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:27 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 7]}]: dispatch 2024-02-02T15:26:27.616 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:27 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]: dispatch 2024-02-02T15:26:27.616 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:27 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]: dispatch 2024-02-02T15:26:27.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:27 smithi142 ceph-mon[45728]: pgmap v286: 129 pgs: 129 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-02-02T15:26:27.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:27 smithi142 ceph-mon[45728]: mgrmap e18: x(active, since 7m), standbys: z, y 2024-02-02T15:26:27.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:27 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 7]}]: dispatch 2024-02-02T15:26:27.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:27 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]: dispatch 2024-02-02T15:26:27.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:27 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]: dispatch 2024-02-02T15:26:27.813 INFO:teuthology.run_tasks:Running task fs.ready... 2024-02-02T15:26:27.834 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T15:26:28.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.036+0000 7f39c6600700 1 -- 172.21.15.1:0/2378569789 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f39c0101f30 msgr2=0x7f39c0102390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:28.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.036+0000 7f39c6600700 1 --2- 172.21.15.1:0/2378569789 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f39c0101f30 0x7f39c0102390 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f39a80099c0 tx=0x7f39a8009cd0 comp rx=0 tx=0).stop 2024-02-02T15:26:28.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.036+0000 7f39c6600700 1 -- 172.21.15.1:0/2378569789 shutdown_connections 2024-02-02T15:26:28.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.036+0000 7f39c6600700 1 --2- 172.21.15.1:0/2378569789 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f39c01028d0 0x7f39c0109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:28.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.036+0000 7f39c6600700 1 --2- 172.21.15.1:0/2378569789 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39c0100d40 0x7f39c0101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:28.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.036+0000 7f39c6600700 1 --2- 172.21.15.1:0/2378569789 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f39c0101f30 0x7f39c0102390 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:28.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.036+0000 7f39c6600700 1 -- 172.21.15.1:0/2378569789 >> 172.21.15.1:0/2378569789 conn(0x7f39c00fc290 msgr2=0x7f39c00fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:28.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.036+0000 7f39c6600700 1 -- 172.21.15.1:0/2378569789 shutdown_connections 2024-02-02T15:26:28.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.036+0000 7f39c6600700 1 -- 172.21.15.1:0/2378569789 wait complete. 2024-02-02T15:26:28.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.038+0000 7f39c6600700 1 Processor -- start 2024-02-02T15:26:28.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.038+0000 7f39c6600700 1 -- start start 2024-02-02T15:26:28.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.038+0000 7f39c6600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f39c0100d40 0x7f39c0074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:28.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.039+0000 7f39c6600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f39c0101f30 0x7f39c0073130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:28.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.039+0000 7f39c6600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39c01028d0 0x7f39c0073670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:28.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.039+0000 7f39c6600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39c0073e10 con 0x7f39c0101f30 2024-02-02T15:26:28.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.039+0000 7f39c6600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39c0073f80 con 0x7f39c0100d40 2024-02-02T15:26:28.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.039+0000 7f39c6600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39c01a0aa0 con 0x7f39c01028d0 2024-02-02T15:26:28.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.039+0000 7f39bea00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f39c0101f30 0x7f39c0073130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:28.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.039+0000 7f39bfe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39c01028d0 0x7f39c0073670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:28.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.039+0000 7f39bea00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f39c0101f30 0x7f39c0073130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56864/0 (socket says 172.21.15.1:56864) 2024-02-02T15:26:28.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.039+0000 7f39bea00700 1 -- 172.21.15.1:0/434312877 learned_addr learned my addr 172.21.15.1:0/434312877 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:28.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.039+0000 7f39bf400700 1 --2- 172.21.15.1:0/434312877 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f39c0100d40 0x7f39c0074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:28.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.039+0000 7f39bea00700 1 -- 172.21.15.1:0/434312877 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39c01028d0 msgr2=0x7f39c0073670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:28.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.039+0000 7f39bea00700 1 --2- 172.21.15.1:0/434312877 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39c01028d0 0x7f39c0073670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:28.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.039+0000 7f39bea00700 1 -- 172.21.15.1:0/434312877 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f39c0100d40 msgr2=0x7f39c0074af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:28.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.039+0000 7f39bea00700 1 --2- 172.21.15.1:0/434312877 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f39c0100d40 0x7f39c0074af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:28.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.039+0000 7f39bea00700 1 -- 172.21.15.1:0/434312877 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39a80095f0 con 0x7f39c0101f30 2024-02-02T15:26:28.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.040+0000 7f39bea00700 1 --2- 172.21.15.1:0/434312877 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f39c0101f30 0x7f39c0073130 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f39a8009990 tx=0x7f39a800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:28.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.040+0000 7f39b7e00700 1 -- 172.21.15.1:0/434312877 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a801d070 con 0x7f39c0101f30 2024-02-02T15:26:28.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.040+0000 7f39b7e00700 1 -- 172.21.15.1:0/434312877 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39a8005030 con 0x7f39c0101f30 2024-02-02T15:26:28.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.040+0000 7f39c6600700 1 -- 172.21.15.1:0/434312877 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39c01a0d20 con 0x7f39c0101f30 2024-02-02T15:26:28.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.040+0000 7f39b7e00700 1 -- 172.21.15.1:0/434312877 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a8004a70 con 0x7f39c0101f30 2024-02-02T15:26:28.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.040+0000 7f39c6600700 1 -- 172.21.15.1:0/434312877 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39c01a11a0 con 0x7f39c0101f30 2024-02-02T15:26:28.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.041+0000 7f39c6600700 1 -- 172.21.15.1:0/434312877 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39c0107c70 con 0x7f39c0101f30 2024-02-02T15:26:28.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.043+0000 7f39b7e00700 1 -- 172.21.15.1:0/434312877 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) v1 ==== 135069+0+0 (secure 0 0 0) 0x7f39a8005410 con 0x7f39c0101f30 2024-02-02T15:26:28.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.044+0000 7f39b7e00700 1 --2- 172.21.15.1:0/434312877 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f39a00a05a0 0x7f39a00a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:28.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.044+0000 7f39b7e00700 1 -- 172.21.15.1:0/434312877 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f39a80cc9f0 con 0x7f39c0101f30 2024-02-02T15:26:28.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.045+0000 7f39bf400700 1 --2- 172.21.15.1:0/434312877 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f39a00a05a0 0x7f39a00a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:28.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.045+0000 7f39bf400700 1 --2- 172.21.15.1:0/434312877 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f39a00a05a0 0x7f39a00a2a60 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f39c0101d90 tx=0x7f39b0009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:28.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.049+0000 7f39b7e00700 1 -- 172.21.15.1:0/434312877 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f39a8096250 con 0x7f39c0101f30 2024-02-02T15:26:28.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.264+0000 7f39c6600700 1 -- 172.21.15.1:0/434312877 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f39c002cc70 con 0x7f39c0101f30 2024-02-02T15:26:28.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.265+0000 7f39b7e00700 1 -- 172.21.15.1:0/434312877 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+24453 (secure 0 0 0) 0x7f39a8099870 con 0x7f39c0101f30 2024-02-02T15:26:28.266 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:26:28.266 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":87,"fsid":"371a409c-c1de-11ee-95b6-87774f69a715","created":"2024-02-02T15:18:26.746506+0000","modified":"2024-02-02T15:26:27.264632+0000","last_up_change":"2024-02-02T15:24:08.800951+0000","last_in_change":"2024-02-02T15:23:52.784280+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T15:21:22.471754+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-02T15:24:46.195779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-02T15:24:47.226403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-02T15:24:50.300462+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"73e8a07f-3560-4c19-85ed-030d4d8f0f23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6803","nonce":2591906423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6805","nonce":2591906423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6809","nonce":2591906423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6807","nonce":2591906423}]},"public_addr":"172.21.15.1:6803/2591906423","cluster_addr":"172.21.15.1:6805/2591906423","heartbeat_back_addr":"172.21.15.1:6809/2591906423","heartbeat_front_addr":"172.21.15.1:6807/2591906423","state":["exists","up"]},{"osd":1,"uuid":"e53cffec-0ff9-4ed9-8dcf-ff47d04db2bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6801","nonce":747581719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6803","nonce":747581719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6807","nonce":747581719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6805","nonce":747581719}]},"public_addr":"172.21.15.71:6801/747581719","cluster_addr":"172.21.15.71:6803/747581719","heartbeat_back_addr":"172.21.15.71:6807/747581719","heartbeat_front_addr":"172.21.15.71:6805/747581719","state":["exists","up"]},{"osd":2,"uuid":"856c2347-294c-482b-8595-ad77f4792db4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6801","nonce":377511414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6803","nonce":377511414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6807","nonce":377511414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6805","nonce":377511414}]},"public_addr":"172.21.15.142:6801/377511414","cluster_addr":"172.21.15.142:6803/377511414","heartbeat_back_addr":"172.21.15.142:6807/377511414","heartbeat_front_addr":"172.21.15.142:6805/377511414","state":["exists","up"]},{"osd":3,"uuid":"4b946ad2-2da9-4e0b-9d61-d039d8669a42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6811","nonce":1086124477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6813","nonce":1086124477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6817","nonce":1086124477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6815","nonce":1086124477}]},"public_addr":"172.21.15.1:6811/1086124477","cluster_addr":"172.21.15.1:6813/1086124477","heartbeat_back_addr":"172.21.15.1:6817/1086124477","heartbeat_front_addr":"172.21.15.1:6815/1086124477","state":["exists","up"]},{"osd":4,"uuid":"43e36c43-47a0-4634-a45b-1da14a0e19d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6809","nonce":3095380305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6811","nonce":3095380305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6815","nonce":3095380305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6813","nonce":3095380305}]},"public_addr":"172.21.15.71:6809/3095380305","cluster_addr":"172.21.15.71:6811/3095380305","heartbeat_back_addr":"172.21.15.71:6815/3095380305","heartbeat_front_addr":"172.21.15.71:6813/3095380305","state":["exists","up"]},{"osd":5,"uuid":"eb48160e-ea7c-42f7-9477-be110bc8a34e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6809","nonce":4195149262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6811","nonce":4195149262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6815","nonce":4195149262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6813","nonce":4195149262}]},"public_addr":"172.21.15.142:6809/4195149262","cluster_addr":"172.21.15.142:6811/4195149262","heartbeat_back_addr":"172.21.15.142:6815/4195149262","heartbeat_front_addr":"172.21.15.142:6813/4195149262","state":["exists","up"]},{"osd":6,"uuid":"0ec4501a-5853-4ca6-b325-231f5b72335e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6819","nonce":1560153428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6821","nonce":1560153428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6825","nonce":1560153428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6823","nonce":1560153428}]},"public_addr":"172.21.15.1:6819/1560153428","cluster_addr":"172.21.15.1:6821/1560153428","heartbeat_back_addr":"172.21.15.1:6825/1560153428","heartbeat_front_addr":"172.21.15.1:6823/1560153428","state":["exists","up"]},{"osd":7,"uuid":"029debe8-9cdf-44f6-9aea-cabd1d185f93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6817","nonce":4080955538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6819","nonce":4080955538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6823","nonce":4080955538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6821","nonce":4080955538}]},"public_addr":"172.21.15.71:6817/4080955538","cluster_addr":"172.21.15.71:6819/4080955538","heartbeat_back_addr":"172.21.15.71:6823/4080955538","heartbeat_front_addr":"172.21.15.71:6821/4080955538","state":["exists","up"]},{"osd":8,"uuid":"7c90896c-9955-4c1b-aaca-7531dd5457f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6817","nonce":3942038209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6819","nonce":3942038209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6823","nonce":3942038209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6821","nonce":3942038209}]},"public_addr":"172.21.15.142:6817/3942038209","cluster_addr":"172.21.15.142:6819/3942038209","heartbeat_back_addr":"172.21.15.142:6823/3942038209","heartbeat_front_addr":"172.21.15.142:6821/3942038209","state":["exists","up"]},{"osd":9,"uuid":"7e165e68-31a8-4598-a577-de3e90ae5c5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6827","nonce":2716544850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6829","nonce":2716544850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6833","nonce":2716544850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6831","nonce":2716544850}]},"public_addr":"172.21.15.1:6827/2716544850","cluster_addr":"172.21.15.1:6829/2716544850","heartbeat_back_addr":"172.21.15.1:6833/2716544850","heartbeat_front_addr":"172.21.15.1:6831/2716544850","state":["exists","up"]},{"osd":10,"uuid":"48cd5fdb-0887-444a-aab8-a1de24282bb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6825","nonce":84691841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6827","nonce":84691841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6831","nonce":84691841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6829","nonce":84691841}]},"public_addr":"172.21.15.71:6825/84691841","cluster_addr":"172.21.15.71:6827/84691841","heartbeat_back_addr":"172.21.15.71:6831/84691841","heartbeat_front_addr":"172.21.15.71:6829/84691841","state":["exists","up"]},{"osd":11,"uuid":"f011dd27-11b0-4166-adfe-76756533163a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6825","nonce":2290179934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6827","nonce":2290179934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6831","nonce":2290179934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6829","nonce":2290179934}]},"public_addr":"172.21.15.142:6825/2290179934","cluster_addr":"172.21.15.142:6827/2290179934","heartbeat_back_addr":"172.21.15.142:6831/2290179934","heartbeat_front_addr":"172.21.15.142:6829/2290179934","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:20:44.060692+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:01.903817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:19.368258+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:40.028037+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:57.513361+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:22:14.805944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:22:35.188439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:22:53.014412+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:23:10.507420+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:23:31.849765+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:23:50.070612+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:24:07.388625+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":5,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/4092532060":"2024-02-03T15:19:25.836171+0000","172.21.15.1:6801/395498296":"2024-02-03T15:19:25.836171+0000","172.21.15.1:6800/395498296":"2024-02-03T15:19:25.836171+0000","172.21.15.1:0/1575373489":"2024-02-03T15:19:25.836171+0000","172.21.15.1:0/865983374":"2024-02-03T15:19:01.431596+0000","172.21.15.1:0/3165960227":"2024-02-03T15:19:01.431596+0000","172.21.15.1:0/2350830293":"2024-02-03T15:19:01.431596+0000","172.21.15.1:0/2959690250":"2024-02-03T15:19:25.836171+0000","172.21.15.1:6800/1162530732":"2024-02-03T15:19:01.431596+0000","172.21.15.1:6801/1162530732":"2024-02-03T15:19:01.431596+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T15:26:28.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.268+0000 7f39c6600700 1 -- 172.21.15.1:0/434312877 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f39a00a05a0 msgr2=0x7f39a00a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:28.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.268+0000 7f39c6600700 1 --2- 172.21.15.1:0/434312877 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f39a00a05a0 0x7f39a00a2a60 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f39c0101d90 tx=0x7f39b0009260 comp rx=0 tx=0).stop 2024-02-02T15:26:28.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.268+0000 7f39c6600700 1 -- 172.21.15.1:0/434312877 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f39c0101f30 msgr2=0x7f39c0073130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:28.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.268+0000 7f39c6600700 1 --2- 172.21.15.1:0/434312877 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f39c0101f30 0x7f39c0073130 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f39a8009990 tx=0x7f39a800f690 comp rx=0 tx=0).stop 2024-02-02T15:26:28.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.268+0000 7f39c6600700 1 -- 172.21.15.1:0/434312877 shutdown_connections 2024-02-02T15:26:28.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.268+0000 7f39c6600700 1 --2- 172.21.15.1:0/434312877 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f39a00a05a0 0x7f39a00a2a60 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:28.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.268+0000 7f39c6600700 1 --2- 172.21.15.1:0/434312877 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f39c01028d0 0x7f39c0073670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:28.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.268+0000 7f39c6600700 1 --2- 172.21.15.1:0/434312877 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f39c0100d40 0x7f39c0074af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:28.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.268+0000 7f39c6600700 1 --2- 172.21.15.1:0/434312877 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f39c0101f30 0x7f39c0073130 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:28.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.268+0000 7f39c6600700 1 -- 172.21.15.1:0/434312877 >> 172.21.15.1:0/434312877 conn(0x7f39c00fc290 msgr2=0x7f39c00fe530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:28.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.268+0000 7f39c6600700 1 -- 172.21.15.1:0/434312877 shutdown_connections 2024-02-02T15:26:28.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.268+0000 7f39c6600700 1 -- 172.21.15.1:0/434312877 wait complete. 2024-02-02T15:26:28.292 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-02T15:21:22.471754+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-02T15:24:46.195779+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-02T15:24:47.226403+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-02-02T15:24:50.300462+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-02-02T15:26:28.293 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-02T15:26:28.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.496+0000 7f5995600700 1 -- 172.21.15.1:0/4171745944 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5990100e20 msgr2=0x7f5990101240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:28.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.496+0000 7f5995600700 1 --2- 172.21.15.1:0/4171745944 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5990100e20 0x7f5990101240 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f59800098f0 tx=0x7f5980009c00 comp rx=0 tx=0).stop 2024-02-02T15:26:28.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.496+0000 7f5995600700 1 -- 172.21.15.1:0/4171745944 shutdown_connections 2024-02-02T15:26:28.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.496+0000 7f5995600700 1 --2- 172.21.15.1:0/4171745944 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5990102930 0x7f5990109250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:28.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.496+0000 7f5995600700 1 --2- 172.21.15.1:0/4171745944 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5990100e20 0x7f5990101240 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:28.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.496+0000 7f5995600700 1 --2- 172.21.15.1:0/4171745944 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5990101f70 0x7f59901023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:28.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.496+0000 7f5995600700 1 -- 172.21.15.1:0/4171745944 >> 172.21.15.1:0/4171745944 conn(0x7f59900fc390 msgr2=0x7f59900fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:28.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.497+0000 7f5995600700 1 -- 172.21.15.1:0/4171745944 shutdown_connections 2024-02-02T15:26:28.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.497+0000 7f5995600700 1 -- 172.21.15.1:0/4171745944 wait complete. 2024-02-02T15:26:28.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.498+0000 7f5995600700 1 Processor -- start 2024-02-02T15:26:28.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.498+0000 7f5995600700 1 -- start start 2024-02-02T15:26:28.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.498+0000 7f5995600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5990100e20 0x7f5990074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:28.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.498+0000 7f5995600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5990101f70 0x7f5990073130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:28.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.498+0000 7f5995600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5990102930 0x7f5990073670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:28.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.498+0000 7f5995600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5990073e10 con 0x7f5990101f70 2024-02-02T15:26:28.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.498+0000 7f5995600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5990073f80 con 0x7f5990102930 2024-02-02T15:26:28.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.498+0000 7f5995600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59901a0b00 con 0x7f5990100e20 2024-02-02T15:26:28.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.498+0000 7f598e000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5990101f70 0x7f5990073130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:28.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.498+0000 7f598e000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5990101f70 0x7f5990073130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56880/0 (socket says 172.21.15.1:56880) 2024-02-02T15:26:28.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.498+0000 7f598e000700 1 -- 172.21.15.1:0/2516851472 learned_addr learned my addr 172.21.15.1:0/2516851472 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:28.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.499+0000 7f598ea00700 1 --2- 172.21.15.1:0/2516851472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5990100e20 0x7f5990074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:28.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.499+0000 7f598f400700 1 --2- 172.21.15.1:0/2516851472 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5990102930 0x7f5990073670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:28.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.499+0000 7f598e000700 1 -- 172.21.15.1:0/2516851472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5990100e20 msgr2=0x7f5990074af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:28.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.499+0000 7f598e000700 1 --2- 172.21.15.1:0/2516851472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5990100e20 0x7f5990074af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:28.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.499+0000 7f598e000700 1 -- 172.21.15.1:0/2516851472 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5990102930 msgr2=0x7f5990073670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:28.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.499+0000 7f598e000700 1 --2- 172.21.15.1:0/2516851472 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5990102930 0x7f5990073670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:28.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.499+0000 7f598e000700 1 -- 172.21.15.1:0/2516851472 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59780095f0 con 0x7f5990101f70 2024-02-02T15:26:28.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.499+0000 7f598f400700 1 --2- 172.21.15.1:0/2516851472 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5990102930 0x7f5990073670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:26:28.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.499+0000 7f598e000700 1 --2- 172.21.15.1:0/2516851472 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5990101f70 0x7f5990073130 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f597800ec10 tx=0x7f597800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:28.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.499+0000 7f5987400700 1 -- 172.21.15.1:0/2516851472 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f597800cd40 con 0x7f5990101f70 2024-02-02T15:26:28.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.499+0000 7f5995600700 1 -- 172.21.15.1:0/2516851472 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5988009c40 con 0x7f5990101f70 2024-02-02T15:26:28.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.499+0000 7f5987400700 1 -- 172.21.15.1:0/2516851472 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f59780044e0 con 0x7f5990101f70 2024-02-02T15:26:28.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.499+0000 7f5987400700 1 -- 172.21.15.1:0/2516851472 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5978010690 con 0x7f5990101f70 2024-02-02T15:26:28.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.500+0000 7f5995600700 1 -- 172.21.15.1:0/2516851472 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5980009520 con 0x7f5990101f70 2024-02-02T15:26:28.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.501+0000 7f5995600700 1 -- 172.21.15.1:0/2516851472 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f599010c5c0 con 0x7f5990101f70 2024-02-02T15:26:28.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.503+0000 7f5987400700 1 -- 172.21.15.1:0/2516851472 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) v1 ==== 135069+0+0 (secure 0 0 0) 0x7f5978003710 con 0x7f5990101f70 2024-02-02T15:26:28.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.504+0000 7f5987400700 1 --2- 172.21.15.1:0/2516851472 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f59700a0520 0x7f59700a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:28.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.504+0000 7f5987400700 1 -- 172.21.15.1:0/2516851472 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f5978014070 con 0x7f5990101f70 2024-02-02T15:26:28.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.504+0000 7f598ea00700 1 --2- 172.21.15.1:0/2516851472 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f59700a0520 0x7f59700a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:28.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.505+0000 7f598ea00700 1 --2- 172.21.15.1:0/2516851472 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f59700a0520 0x7f59700a29e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f598000fa40 tx=0x7f5980019040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:28.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.508+0000 7f5987400700 1 -- 172.21.15.1:0/2516851472 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f5978095af0 con 0x7f5990101f70 2024-02-02T15:26:28.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:28 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 7]}]': finished 2024-02-02T15:26:28.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:28 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]': finished 2024-02-02T15:26:28.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:28 smithi071 ceph-mon[44707]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]': finished 2024-02-02T15:26:28.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:28 smithi071 ceph-mon[44707]: osdmap e87: 12 total, 12 up, 12 in 2024-02-02T15:26:28.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:28 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/2799108861' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-02-02T15:26:28.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:28 smithi071 ceph-mon[44707]: pgmap v288: 129 pgs: 129 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 307 B/s wr, 10 op/s 2024-02-02T15:26:28.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:28 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/434312877' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-02T15:26:28.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:28 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 7]}]': finished 2024-02-02T15:26:28.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:28 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]': finished 2024-02-02T15:26:28.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:28 smithi142 ceph-mon[45728]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]': finished 2024-02-02T15:26:28.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:28 smithi142 ceph-mon[45728]: osdmap e87: 12 total, 12 up, 12 in 2024-02-02T15:26:28.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:28 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/2799108861' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-02-02T15:26:28.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:28 smithi142 ceph-mon[45728]: pgmap v288: 129 pgs: 129 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 307 B/s wr, 10 op/s 2024-02-02T15:26:28.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:28 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/434312877' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-02T15:26:28.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:28 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 7]}]': finished 2024-02-02T15:26:28.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:28 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [5, 9]}]': finished 2024-02-02T15:26:28.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:28 smithi001 ceph-mon[35470]: from='mgr.14154 172.21.15.1:0/1880599806' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.29", "id": [3, 9]}]': finished 2024-02-02T15:26:28.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:28 smithi001 ceph-mon[35470]: osdmap e87: 12 total, 12 up, 12 in 2024-02-02T15:26:28.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:28 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/2799108861' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-02-02T15:26:28.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:28 smithi001 ceph-mon[35470]: pgmap v288: 129 pgs: 129 active+clean; 642 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 307 B/s wr, 10 op/s 2024-02-02T15:26:28.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:28 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/434312877' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-02T15:26:28.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.720+0000 7f5995600700 1 -- 172.21.15.1:0/2516851472 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5990107de0 con 0x7f5990101f70 2024-02-02T15:26:28.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.721+0000 7f5987400700 1 -- 172.21.15.1:0/2516851472 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5978099110 con 0x7f5990101f70 2024-02-02T15:26:28.721 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-02-02T15:26:28.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.723+0000 7f5995600700 1 -- 172.21.15.1:0/2516851472 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f59700a0520 msgr2=0x7f59700a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:28.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.723+0000 7f5995600700 1 --2- 172.21.15.1:0/2516851472 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f59700a0520 0x7f59700a29e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f598000fa40 tx=0x7f5980019040 comp rx=0 tx=0).stop 2024-02-02T15:26:28.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.723+0000 7f5995600700 1 -- 172.21.15.1:0/2516851472 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5990101f70 msgr2=0x7f5990073130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:28.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.723+0000 7f5995600700 1 --2- 172.21.15.1:0/2516851472 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5990101f70 0x7f5990073130 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f597800ec10 tx=0x7f597800ef20 comp rx=0 tx=0).stop 2024-02-02T15:26:28.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.723+0000 7f5995600700 1 -- 172.21.15.1:0/2516851472 shutdown_connections 2024-02-02T15:26:28.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.723+0000 7f5995600700 1 --2- 172.21.15.1:0/2516851472 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5990102930 0x7f5990073670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:28.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.723+0000 7f5995600700 1 --2- 172.21.15.1:0/2516851472 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f59700a0520 0x7f59700a29e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:28.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.723+0000 7f5995600700 1 --2- 172.21.15.1:0/2516851472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5990100e20 0x7f5990074af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:28.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.723+0000 7f5995600700 1 --2- 172.21.15.1:0/2516851472 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5990101f70 0x7f5990073130 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:28.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.723+0000 7f5995600700 1 -- 172.21.15.1:0/2516851472 >> 172.21.15.1:0/2516851472 conn(0x7f59900fc390 msgr2=0x7f59900fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:28.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.723+0000 7f5995600700 1 -- 172.21.15.1:0/2516851472 shutdown_connections 2024-02-02T15:26:28.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.723+0000 7f5995600700 1 -- 172.21.15.1:0/2516851472 wait complete. 2024-02-02T15:26:28.742 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-02T15:26:28.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.943+0000 7f1229c00700 1 -- 172.21.15.1:0/1557783893 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1224100e20 msgr2=0x7f1224101240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:28.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.943+0000 7f1229c00700 1 --2- 172.21.15.1:0/1557783893 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1224100e20 0x7f1224101240 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f12140098f0 tx=0x7f1214009c00 comp rx=0 tx=0).stop 2024-02-02T15:26:28.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.943+0000 7f1229c00700 1 -- 172.21.15.1:0/1557783893 shutdown_connections 2024-02-02T15:26:28.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.943+0000 7f1229c00700 1 --2- 172.21.15.1:0/1557783893 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1224102930 0x7f1224109250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:28.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.943+0000 7f1229c00700 1 --2- 172.21.15.1:0/1557783893 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1224100e20 0x7f1224101240 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:28.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.943+0000 7f1229c00700 1 --2- 172.21.15.1:0/1557783893 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1224101f70 0x7f12241023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:28.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.943+0000 7f1229c00700 1 -- 172.21.15.1:0/1557783893 >> 172.21.15.1:0/1557783893 conn(0x7f12240fc390 msgr2=0x7f12240fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:28.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.943+0000 7f1229c00700 1 -- 172.21.15.1:0/1557783893 shutdown_connections 2024-02-02T15:26:28.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.943+0000 7f1229c00700 1 -- 172.21.15.1:0/1557783893 wait complete. 2024-02-02T15:26:28.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.945+0000 7f1229c00700 1 Processor -- start 2024-02-02T15:26:28.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.945+0000 7f1229c00700 1 -- start start 2024-02-02T15:26:28.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.945+0000 7f1229c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1224100e20 0x7f1224196b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:28.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.945+0000 7f1229c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1224101f70 0x7f12241970a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:28.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.946+0000 7f1229c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1224102930 0x7f122419e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:28.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.946+0000 7f1229c00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f122419e6e0 con 0x7f1224102930 2024-02-02T15:26:28.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.946+0000 7f1229c00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f122419e820 con 0x7f1224101f70 2024-02-02T15:26:28.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.946+0000 7f1229c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f122419e9b0 con 0x7f1224100e20 2024-02-02T15:26:28.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.946+0000 7f1223400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1224102930 0x7f122419e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:28.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.946+0000 7f1222a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1224100e20 0x7f1224196b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:28.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.946+0000 7f1223400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1224102930 0x7f122419e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56892/0 (socket says 172.21.15.1:56892) 2024-02-02T15:26:28.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.946+0000 7f1223400700 1 -- 172.21.15.1:0/1293797826 learned_addr learned my addr 172.21.15.1:0/1293797826 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:28.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.946+0000 7f1222000700 1 --2- 172.21.15.1:0/1293797826 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1224101f70 0x7f12241970a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:28.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.946+0000 7f1223400700 1 -- 172.21.15.1:0/1293797826 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1224100e20 msgr2=0x7f1224196b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:28.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.946+0000 7f1223400700 1 --2- 172.21.15.1:0/1293797826 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1224100e20 0x7f1224196b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:28.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.946+0000 7f1223400700 1 -- 172.21.15.1:0/1293797826 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1224101f70 msgr2=0x7f12241970a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:28.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.946+0000 7f1223400700 1 --2- 172.21.15.1:0/1293797826 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1224101f70 0x7f12241970a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:28.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.946+0000 7f1223400700 1 -- 172.21.15.1:0/1293797826 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f120c0095f0 con 0x7f1224102930 2024-02-02T15:26:28.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.947+0000 7f1223400700 1 --2- 172.21.15.1:0/1293797826 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1224102930 0x7f122419e110 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f121c00ef30 tx=0x7f121c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:28.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.947+0000 7f121b400700 1 -- 172.21.15.1:0/1293797826 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f121c018070 con 0x7f1224102930 2024-02-02T15:26:28.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.947+0000 7f1229c00700 1 -- 172.21.15.1:0/1293797826 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1214009520 con 0x7f1224102930 2024-02-02T15:26:28.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.947+0000 7f1229c00700 1 -- 172.21.15.1:0/1293797826 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f122419f000 con 0x7f1224102930 2024-02-02T15:26:28.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.947+0000 7f121b400700 1 -- 172.21.15.1:0/1293797826 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f121c00f040 con 0x7f1224102930 2024-02-02T15:26:28.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.947+0000 7f121b400700 1 -- 172.21.15.1:0/1293797826 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f121c00ccb0 con 0x7f1224102930 2024-02-02T15:26:28.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.948+0000 7f1229c00700 1 -- 172.21.15.1:0/1293797826 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11fc005310 con 0x7f1224102930 2024-02-02T15:26:28.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.950+0000 7f121b400700 1 -- 172.21.15.1:0/1293797826 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) v1 ==== 135069+0+0 (secure 0 0 0) 0x7f121c004ad0 con 0x7f1224102930 2024-02-02T15:26:28.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.951+0000 7f121b400700 1 --2- 172.21.15.1:0/1293797826 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f12040a05a0 0x7f12040a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:28.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.951+0000 7f121b400700 1 -- 172.21.15.1:0/1293797826 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f121c0cb650 con 0x7f1224102930 2024-02-02T15:26:28.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.951+0000 7f1222a00700 1 --2- 172.21.15.1:0/1293797826 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f12040a05a0 0x7f12040a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:28.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.952+0000 7f1222a00700 1 --2- 172.21.15.1:0/1293797826 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f12040a05a0 0x7f12040a2a60 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1214005e90 tx=0x7f1214019040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:28.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:28.955+0000 7f121b400700 1 -- 172.21.15.1:0/1293797826 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f121c098940 con 0x7f1224102930 2024-02-02T15:26:29.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.170+0000 7f1229c00700 1 -- 172.21.15.1:0/1293797826 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f11fc005f60 con 0x7f1224102930 2024-02-02T15:26:29.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.170+0000 7f121b400700 1 -- 172.21.15.1:0/1293797826 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f121c0092e0 con 0x7f1224102930 2024-02-02T15:26:29.170 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-02-02T15:26:29.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.173+0000 7f1229c00700 1 -- 172.21.15.1:0/1293797826 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f12040a05a0 msgr2=0x7f12040a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:29.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.173+0000 7f1229c00700 1 --2- 172.21.15.1:0/1293797826 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f12040a05a0 0x7f12040a2a60 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1214005e90 tx=0x7f1214019040 comp rx=0 tx=0).stop 2024-02-02T15:26:29.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.173+0000 7f1229c00700 1 -- 172.21.15.1:0/1293797826 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1224102930 msgr2=0x7f122419e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:29.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.173+0000 7f1229c00700 1 --2- 172.21.15.1:0/1293797826 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1224102930 0x7f122419e110 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f121c00ef30 tx=0x7f121c00c3a0 comp rx=0 tx=0).stop 2024-02-02T15:26:29.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.173+0000 7f1229c00700 1 -- 172.21.15.1:0/1293797826 shutdown_connections 2024-02-02T15:26:29.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.173+0000 7f1229c00700 1 --2- 172.21.15.1:0/1293797826 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1224102930 0x7f122419e110 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:29.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.173+0000 7f1229c00700 1 --2- 172.21.15.1:0/1293797826 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f12040a05a0 0x7f12040a2a60 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:29.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.173+0000 7f1229c00700 1 --2- 172.21.15.1:0/1293797826 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1224100e20 0x7f1224196b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:29.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.173+0000 7f1229c00700 1 --2- 172.21.15.1:0/1293797826 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1224101f70 0x7f12241970a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:29.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.173+0000 7f1229c00700 1 -- 172.21.15.1:0/1293797826 >> 172.21.15.1:0/1293797826 conn(0x7f12240fc390 msgr2=0x7f12240fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:29.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.173+0000 7f1229c00700 1 -- 172.21.15.1:0/1293797826 shutdown_connections 2024-02-02T15:26:29.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.173+0000 7f1229c00700 1 -- 172.21.15.1:0/1293797826 wait complete. 2024-02-02T15:26:29.191 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-02T15:26:29.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.394+0000 7f8cd2c00700 1 -- 172.21.15.1:0/2504714026 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8ccc102930 msgr2=0x7f8ccc109250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:29.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.394+0000 7f8cd2c00700 1 --2- 172.21.15.1:0/2504714026 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8ccc102930 0x7f8ccc109250 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8cc400b0a0 tx=0x7f8cc400b3b0 comp rx=0 tx=0).stop 2024-02-02T15:26:29.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.395+0000 7f8cd2c00700 1 -- 172.21.15.1:0/2504714026 shutdown_connections 2024-02-02T15:26:29.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.395+0000 7f8cd2c00700 1 --2- 172.21.15.1:0/2504714026 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8ccc102930 0x7f8ccc109250 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:29.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.395+0000 7f8cd2c00700 1 --2- 172.21.15.1:0/2504714026 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ccc100e20 0x7f8ccc101240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:29.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.395+0000 7f8cd2c00700 1 --2- 172.21.15.1:0/2504714026 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ccc101f70 0x7f8ccc1023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:29.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.395+0000 7f8cd2c00700 1 -- 172.21.15.1:0/2504714026 >> 172.21.15.1:0/2504714026 conn(0x7f8ccc0fc390 msgr2=0x7f8ccc0fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:29.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.395+0000 7f8cd2c00700 1 -- 172.21.15.1:0/2504714026 shutdown_connections 2024-02-02T15:26:29.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.395+0000 7f8cd2c00700 1 -- 172.21.15.1:0/2504714026 wait complete. 2024-02-02T15:26:29.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.396+0000 7f8cd2c00700 1 Processor -- start 2024-02-02T15:26:29.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.396+0000 7f8cd2c00700 1 -- start start 2024-02-02T15:26:29.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.396+0000 7f8cd2c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ccc100e20 0x7f8ccc196b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:29.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.396+0000 7f8cd2c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8ccc101f70 0x7f8ccc1970c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:29.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.396+0000 7f8cd2c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ccc102930 0x7f8ccc19e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:29.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.396+0000 7f8cd2c00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ccc19e700 con 0x7f8ccc101f70 2024-02-02T15:26:29.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.396+0000 7f8cd2c00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ccc19e840 con 0x7f8ccc100e20 2024-02-02T15:26:29.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.396+0000 7f8cd2c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ccc19e9d0 con 0x7f8ccc102930 2024-02-02T15:26:29.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.396+0000 7f8ccb400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8ccc101f70 0x7f8ccc1970c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:29.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.396+0000 7f8ccb400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8ccc101f70 0x7f8ccc1970c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56910/0 (socket says 172.21.15.1:56910) 2024-02-02T15:26:29.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.396+0000 7f8ccb400700 1 -- 172.21.15.1:0/2241273544 learned_addr learned my addr 172.21.15.1:0/2241273544 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:29.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.397+0000 7f8ccbe00700 1 --2- 172.21.15.1:0/2241273544 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ccc100e20 0x7f8ccc196b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:29.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.397+0000 7f8cd0c00700 1 --2- 172.21.15.1:0/2241273544 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ccc102930 0x7f8ccc19e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:29.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.397+0000 7f8ccb400700 1 -- 172.21.15.1:0/2241273544 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ccc102930 msgr2=0x7f8ccc19e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:29.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.397+0000 7f8ccb400700 1 --2- 172.21.15.1:0/2241273544 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ccc102930 0x7f8ccc19e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:29.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.397+0000 7f8ccb400700 1 -- 172.21.15.1:0/2241273544 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ccc100e20 msgr2=0x7f8ccc196b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:29.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.397+0000 7f8ccb400700 1 --2- 172.21.15.1:0/2241273544 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ccc100e20 0x7f8ccc196b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:29.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.397+0000 7f8ccb400700 1 -- 172.21.15.1:0/2241273544 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cc4009d00 con 0x7f8ccc101f70 2024-02-02T15:26:29.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.397+0000 7f8ccb400700 1 --2- 172.21.15.1:0/2241273544 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8ccc101f70 0x7f8ccc1970c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f8cb400e9a0 tx=0x7f8cb400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:29.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.397+0000 7f8cc8c00700 1 -- 172.21.15.1:0/2241273544 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb4009790 con 0x7f8ccc101f70 2024-02-02T15:26:29.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.397+0000 7f8cc8c00700 1 -- 172.21.15.1:0/2241273544 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8cb4010430 con 0x7f8ccc101f70 2024-02-02T15:26:29.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.397+0000 7f8cc8c00700 1 -- 172.21.15.1:0/2241273544 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb4018680 con 0x7f8ccc101f70 2024-02-02T15:26:29.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.397+0000 7f8cd2c00700 1 -- 172.21.15.1:0/2241273544 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ccc19ecb0 con 0x7f8ccc101f70 2024-02-02T15:26:29.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.398+0000 7f8cd2c00700 1 -- 172.21.15.1:0/2241273544 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ccc19f170 con 0x7f8ccc101f70 2024-02-02T15:26:29.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.399+0000 7f8cd2c00700 1 -- 172.21.15.1:0/2241273544 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ccc10c670 con 0x7f8ccc101f70 2024-02-02T15:26:29.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.401+0000 7f8cc8c00700 1 -- 172.21.15.1:0/2241273544 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) v1 ==== 135069+0+0 (secure 0 0 0) 0x7f8cb40105a0 con 0x7f8ccc101f70 2024-02-02T15:26:29.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.402+0000 7f8cc8c00700 1 --2- 172.21.15.1:0/2241273544 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f8cac0a0520 0x7f8cac0a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:29.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.403+0000 7f8cc8c00700 1 -- 172.21.15.1:0/2241273544 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f8cb4014070 con 0x7f8ccc101f70 2024-02-02T15:26:29.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.403+0000 7f8ccbe00700 1 --2- 172.21.15.1:0/2241273544 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f8cac0a0520 0x7f8cac0a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:29.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.404+0000 7f8ccbe00700 1 --2- 172.21.15.1:0/2241273544 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f8cac0a0520 0x7f8cac0a29e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc009c60 tx=0x7f8cbc009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:29.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.407+0000 7f8cc8c00700 1 -- 172.21.15.1:0/2241273544 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f8cb4095560 con 0x7f8ccc101f70 2024-02-02T15:26:29.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:29 smithi001 ceph-mon[35470]: osdmap e88: 12 total, 12 up, 12 in 2024-02-02T15:26:29.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:29 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/2516851472' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-02T15:26:29.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:29 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/1293797826' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-02T15:26:29.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:29 smithi071 ceph-mon[44707]: osdmap e88: 12 total, 12 up, 12 in 2024-02-02T15:26:29.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:29 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/2516851472' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-02T15:26:29.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:29 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/1293797826' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-02T15:26:29.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.618+0000 7f8cd2c00700 1 -- 172.21.15.1:0/2241273544 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8ccc107cc0 con 0x7f8ccc101f70 2024-02-02T15:26:29.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.619+0000 7f8cc8c00700 1 -- 172.21.15.1:0/2241273544 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8cb4098b80 con 0x7f8ccc101f70 2024-02-02T15:26:29.618 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 64 2024-02-02T15:26:29.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.621+0000 7f8cd2c00700 1 -- 172.21.15.1:0/2241273544 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f8cac0a0520 msgr2=0x7f8cac0a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:29.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.621+0000 7f8cd2c00700 1 --2- 172.21.15.1:0/2241273544 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f8cac0a0520 0x7f8cac0a29e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc009c60 tx=0x7f8cbc009260 comp rx=0 tx=0).stop 2024-02-02T15:26:29.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.621+0000 7f8cd2c00700 1 -- 172.21.15.1:0/2241273544 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8ccc101f70 msgr2=0x7f8ccc1970c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:29.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.621+0000 7f8cd2c00700 1 --2- 172.21.15.1:0/2241273544 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8ccc101f70 0x7f8ccc1970c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f8cb400e9a0 tx=0x7f8cb400ecb0 comp rx=0 tx=0).stop 2024-02-02T15:26:29.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.621+0000 7f8cd2c00700 1 -- 172.21.15.1:0/2241273544 shutdown_connections 2024-02-02T15:26:29.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.621+0000 7f8cd2c00700 1 --2- 172.21.15.1:0/2241273544 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f8cac0a0520 0x7f8cac0a29e0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:29.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.621+0000 7f8cd2c00700 1 --2- 172.21.15.1:0/2241273544 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ccc102930 0x7f8ccc19e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:29.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.621+0000 7f8cd2c00700 1 --2- 172.21.15.1:0/2241273544 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ccc100e20 0x7f8ccc196b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:29.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.621+0000 7f8cd2c00700 1 --2- 172.21.15.1:0/2241273544 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8ccc101f70 0x7f8ccc1970c0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:29.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.621+0000 7f8cd2c00700 1 -- 172.21.15.1:0/2241273544 >> 172.21.15.1:0/2241273544 conn(0x7f8ccc0fc390 msgr2=0x7f8ccc0fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:29.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.621+0000 7f8cd2c00700 1 -- 172.21.15.1:0/2241273544 shutdown_connections 2024-02-02T15:26:29.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.621+0000 7f8cd2c00700 1 -- 172.21.15.1:0/2241273544 wait complete. 2024-02-02T15:26:29.639 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-02-02T15:26:29.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:29 smithi142 ceph-mon[45728]: osdmap e88: 12 total, 12 up, 12 in 2024-02-02T15:26:29.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:29 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/2516851472' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-02T15:26:29.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:29 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/1293797826' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-02T15:26:29.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.841+0000 7fc23a200700 1 -- 172.21.15.1:0/44203496 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc234101f70 msgr2=0x7fc2341023f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:29.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.841+0000 7fc23a200700 1 --2- 172.21.15.1:0/44203496 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc234101f70 0x7fc2341023f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fc21c009990 tx=0x7fc21c009ca0 comp rx=0 tx=0).stop 2024-02-02T15:26:29.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.841+0000 7fc23a200700 1 -- 172.21.15.1:0/44203496 shutdown_connections 2024-02-02T15:26:29.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.841+0000 7fc23a200700 1 --2- 172.21.15.1:0/44203496 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc234102930 0x7fc234109250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:29.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.841+0000 7fc23a200700 1 --2- 172.21.15.1:0/44203496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc234100e20 0x7fc234101240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:29.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.841+0000 7fc23a200700 1 --2- 172.21.15.1:0/44203496 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc234101f70 0x7fc2341023f0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:29.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.841+0000 7fc23a200700 1 -- 172.21.15.1:0/44203496 >> 172.21.15.1:0/44203496 conn(0x7fc2340fc390 msgr2=0x7fc2340fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:29.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.841+0000 7fc23a200700 1 -- 172.21.15.1:0/44203496 shutdown_connections 2024-02-02T15:26:29.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.841+0000 7fc23a200700 1 -- 172.21.15.1:0/44203496 wait complete. 2024-02-02T15:26:29.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.842+0000 7fc23a200700 1 Processor -- start 2024-02-02T15:26:29.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.842+0000 7fc23a200700 1 -- start start 2024-02-02T15:26:29.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.842+0000 7fc23a200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc234100e20 0x7fc234198d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:29.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.842+0000 7fc23a200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc234101f70 0x7fc234199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:29.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.842+0000 7fc23a200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc234102930 0x7fc2341a0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:29.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.842+0000 7fc23a200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2341a0850 con 0x7fc234100e20 2024-02-02T15:26:29.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.842+0000 7fc23a200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2341a09c0 con 0x7fc234102930 2024-02-02T15:26:29.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.842+0000 7fc23a200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2341a0b80 con 0x7fc234101f70 2024-02-02T15:26:29.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.843+0000 7fc233400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc234100e20 0x7fc234198d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:29.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.843+0000 7fc233400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc234100e20 0x7fc234198d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56922/0 (socket says 172.21.15.1:56922) 2024-02-02T15:26:29.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.843+0000 7fc233400700 1 -- 172.21.15.1:0/700114500 learned_addr learned my addr 172.21.15.1:0/700114500 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:29.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.843+0000 7fc233e00700 1 --2- 172.21.15.1:0/700114500 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc234102930 0x7fc2341a0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:29.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.843+0000 7fc232a00700 1 --2- 172.21.15.1:0/700114500 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc234101f70 0x7fc234199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:29.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.843+0000 7fc233400700 1 -- 172.21.15.1:0/700114500 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc234101f70 msgr2=0x7fc234199250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:29.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.843+0000 7fc233400700 1 --2- 172.21.15.1:0/700114500 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc234101f70 0x7fc234199250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:29.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.843+0000 7fc233400700 1 -- 172.21.15.1:0/700114500 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc234102930 msgr2=0x7fc2341a0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:29.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.843+0000 7fc233400700 1 --2- 172.21.15.1:0/700114500 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc234102930 0x7fc2341a0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:29.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.843+0000 7fc233400700 1 -- 172.21.15.1:0/700114500 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc21c0095f0 con 0x7fc234100e20 2024-02-02T15:26:29.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.843+0000 7fc233400700 1 --2- 172.21.15.1:0/700114500 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc234100e20 0x7fc234198d10 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fc22400e9a0 tx=0x7fc22400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:29.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.844+0000 7fc22be00700 1 -- 172.21.15.1:0/700114500 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc224009790 con 0x7fc234100e20 2024-02-02T15:26:29.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.844+0000 7fc22be00700 1 -- 172.21.15.1:0/700114500 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc224010430 con 0x7fc234100e20 2024-02-02T15:26:29.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.844+0000 7fc22be00700 1 -- 172.21.15.1:0/700114500 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc224018680 con 0x7fc234100e20 2024-02-02T15:26:29.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.844+0000 7fc23a200700 1 -- 172.21.15.1:0/700114500 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2341a0e60 con 0x7fc234100e20 2024-02-02T15:26:29.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.844+0000 7fc23a200700 1 -- 172.21.15.1:0/700114500 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2341a1280 con 0x7fc234100e20 2024-02-02T15:26:29.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.845+0000 7fc23a200700 1 -- 172.21.15.1:0/700114500 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc23410c670 con 0x7fc234100e20 2024-02-02T15:26:29.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.847+0000 7fc22be00700 1 -- 172.21.15.1:0/700114500 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) v1 ==== 135069+0+0 (secure 0 0 0) 0x7fc2240105a0 con 0x7fc234100e20 2024-02-02T15:26:29.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.848+0000 7fc22be00700 1 --2- 172.21.15.1:0/700114500 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fc2140a05a0 0x7fc2140a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:29.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.848+0000 7fc22be00700 1 -- 172.21.15.1:0/700114500 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fc224014070 con 0x7fc234100e20 2024-02-02T15:26:29.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.848+0000 7fc232a00700 1 --2- 172.21.15.1:0/700114500 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fc2140a05a0 0x7fc2140a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:29.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.849+0000 7fc232a00700 1 --2- 172.21.15.1:0/700114500 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fc2140a05a0 0x7fc2140a2a60 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc21c00b2e0 tx=0x7fc21c005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:29.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:29.852+0000 7fc22be00700 1 -- 172.21.15.1:0/700114500 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7fc224052110 con 0x7fc234100e20 2024-02-02T15:26:30.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.064+0000 7fc23a200700 1 -- 172.21.15.1:0/700114500 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fc2341a1890 con 0x7fc234100e20 2024-02-02T15:26:30.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.065+0000 7fc22be00700 1 -- 172.21.15.1:0/700114500 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v88) v1 ==== 103+0+11 (secure 0 0 0) 0x7fc224095680 con 0x7fc234100e20 2024-02-02T15:26:30.065 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-02-02T15:26:30.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.067+0000 7fc23a200700 1 -- 172.21.15.1:0/700114500 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fc2140a05a0 msgr2=0x7fc2140a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:30.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.067+0000 7fc23a200700 1 --2- 172.21.15.1:0/700114500 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fc2140a05a0 0x7fc2140a2a60 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc21c00b2e0 tx=0x7fc21c005fb0 comp rx=0 tx=0).stop 2024-02-02T15:26:30.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.067+0000 7fc23a200700 1 -- 172.21.15.1:0/700114500 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc234100e20 msgr2=0x7fc234198d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:30.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.067+0000 7fc23a200700 1 --2- 172.21.15.1:0/700114500 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc234100e20 0x7fc234198d10 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fc22400e9a0 tx=0x7fc22400ecb0 comp rx=0 tx=0).stop 2024-02-02T15:26:30.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.067+0000 7fc23a200700 1 -- 172.21.15.1:0/700114500 shutdown_connections 2024-02-02T15:26:30.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.067+0000 7fc23a200700 1 --2- 172.21.15.1:0/700114500 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc234102930 0x7fc2341a0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:30.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.067+0000 7fc23a200700 1 --2- 172.21.15.1:0/700114500 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc234100e20 0x7fc234198d10 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:30.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.067+0000 7fc23a200700 1 --2- 172.21.15.1:0/700114500 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fc2140a05a0 0x7fc2140a2a60 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:30.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.067+0000 7fc23a200700 1 --2- 172.21.15.1:0/700114500 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc234101f70 0x7fc234199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:30.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.067+0000 7fc23a200700 1 -- 172.21.15.1:0/700114500 >> 172.21.15.1:0/700114500 conn(0x7fc2340fc390 msgr2=0x7fc2340fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:30.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.067+0000 7fc23a200700 1 -- 172.21.15.1:0/700114500 shutdown_connections 2024-02-02T15:26:30.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.067+0000 7fc23a200700 1 -- 172.21.15.1:0/700114500 wait complete. 2024-02-02T15:26:30.086 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T15:26:30.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.283+0000 7f2566a00700 1 -- 172.21.15.1:0/398496729 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f25600737a0 msgr2=0x7f2560073c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:30.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.283+0000 7f2566a00700 1 --2- 172.21.15.1:0/398496729 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f25600737a0 0x7f2560073c20 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f25480098f0 tx=0x7f2548009c00 comp rx=0 tx=0).stop 2024-02-02T15:26:30.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.283+0000 7f2566a00700 1 -- 172.21.15.1:0/398496729 shutdown_connections 2024-02-02T15:26:30.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.283+0000 7f2566a00700 1 --2- 172.21.15.1:0/398496729 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2560108c00 0x7f256010b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:30.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.283+0000 7f2566a00700 1 --2- 172.21.15.1:0/398496729 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2560074d80 0x7f25600731d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:30.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.283+0000 7f2566a00700 1 --2- 172.21.15.1:0/398496729 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f25600737a0 0x7f2560073c20 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:30.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.283+0000 7f2566a00700 1 -- 172.21.15.1:0/398496729 >> 172.21.15.1:0/398496729 conn(0x7f25600fc290 msgr2=0x7f25600fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:30.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.283+0000 7f2566a00700 1 -- 172.21.15.1:0/398496729 shutdown_connections 2024-02-02T15:26:30.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.283+0000 7f2566a00700 1 -- 172.21.15.1:0/398496729 wait complete. 2024-02-02T15:26:30.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.284+0000 7f2566a00700 1 Processor -- start 2024-02-02T15:26:30.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.284+0000 7f2566a00700 1 -- start start 2024-02-02T15:26:30.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.284+0000 7f2566a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25600737a0 0x7f2560198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:30.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.284+0000 7f2566a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2560074d80 0x7f25601992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:30.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.284+0000 7f2566a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2560108c00 0x7f25601a0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:30.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.284+0000 7f2566a00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25601a08e0 con 0x7f2560074d80 2024-02-02T15:26:30.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.284+0000 7f2566a00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25601a0a20 con 0x7f25600737a0 2024-02-02T15:26:30.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.284+0000 7f2566a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25601a0bb0 con 0x7f2560108c00 2024-02-02T15:26:30.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.285+0000 7f255f400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2560074d80 0x7f25601992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:30.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.285+0000 7f255f400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2560074d80 0x7f25601992a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56936/0 (socket says 172.21.15.1:56936) 2024-02-02T15:26:30.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.285+0000 7f255f400700 1 -- 172.21.15.1:0/3192970293 learned_addr learned my addr 172.21.15.1:0/3192970293 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:30.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.285+0000 7f255fe00700 1 --2- 172.21.15.1:0/3192970293 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25600737a0 0x7f2560198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:30.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.285+0000 7f2564a00700 1 --2- 172.21.15.1:0/3192970293 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2560108c00 0x7f25601a0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:30.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.285+0000 7f255f400700 1 -- 172.21.15.1:0/3192970293 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2560108c00 msgr2=0x7f25601a0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:30.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.285+0000 7f255f400700 1 --2- 172.21.15.1:0/3192970293 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2560108c00 0x7f25601a0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:30.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.285+0000 7f255f400700 1 -- 172.21.15.1:0/3192970293 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25600737a0 msgr2=0x7f2560198d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:30.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.285+0000 7f255f400700 1 --2- 172.21.15.1:0/3192970293 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25600737a0 0x7f2560198d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:30.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.285+0000 7f255f400700 1 -- 172.21.15.1:0/3192970293 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2548009520 con 0x7f2560074d80 2024-02-02T15:26:30.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.285+0000 7f2564a00700 1 --2- 172.21.15.1:0/3192970293 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2560108c00 0x7f25601a0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:26:30.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.285+0000 7f255f400700 1 --2- 172.21.15.1:0/3192970293 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2560074d80 0x7f25601992a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f2548009fd0 tx=0x7f254800b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:30.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.286+0000 7f255cc00700 1 -- 172.21.15.1:0/3192970293 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f254801c070 con 0x7f2560074d80 2024-02-02T15:26:30.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.286+0000 7f255cc00700 1 -- 172.21.15.1:0/3192970293 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f254800f7d0 con 0x7f2560074d80 2024-02-02T15:26:30.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.286+0000 7f2566a00700 1 -- 172.21.15.1:0/3192970293 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25601a0e30 con 0x7f2560074d80 2024-02-02T15:26:30.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.286+0000 7f2566a00700 1 -- 172.21.15.1:0/3192970293 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25601a11a0 con 0x7f2560074d80 2024-02-02T15:26:30.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.286+0000 7f255cc00700 1 -- 172.21.15.1:0/3192970293 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2548003b00 con 0x7f2560074d80 2024-02-02T15:26:30.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.287+0000 7f2566a00700 1 -- 172.21.15.1:0/3192970293 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25601072b0 con 0x7f2560074d80 2024-02-02T15:26:30.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.289+0000 7f255cc00700 1 -- 172.21.15.1:0/3192970293 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) v1 ==== 135069+0+0 (secure 0 0 0) 0x7f2548020030 con 0x7f2560074d80 2024-02-02T15:26:30.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.291+0000 7f255cc00700 1 --2- 172.21.15.1:0/3192970293 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f25400a0520 0x7f25400a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:30.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.291+0000 7f255cc00700 1 -- 172.21.15.1:0/3192970293 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f25480ccda0 con 0x7f2560074d80 2024-02-02T15:26:30.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.291+0000 7f255fe00700 1 --2- 172.21.15.1:0/3192970293 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f25400a0520 0x7f25400a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:30.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.292+0000 7f255fe00700 1 --2- 172.21.15.1:0/3192970293 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f25400a0520 0x7f25400a29e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2550009c60 tx=0x7f2550009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:30.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.295+0000 7f255cc00700 1 -- 172.21.15.1:0/3192970293 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f2548096600 con 0x7f2560074d80 2024-02-02T15:26:30.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:30 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/2241273544' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-02T15:26:30.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:30 smithi001 ceph-mon[35470]: pgmap v290: 129 pgs: 129 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 639 B/s wr, 9 op/s 2024-02-02T15:26:30.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:30 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/700114500' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-02T15:26:30.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.555+0000 7f2566a00700 1 -- 172.21.15.1:0/3192970293 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f25600619f0 con 0x7f2560074d80 2024-02-02T15:26:30.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.556+0000 7f255cc00700 1 -- 172.21.15.1:0/3192970293 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10866 (secure 0 0 0) 0x7f2548099c20 con 0x7f2560074d80 2024-02-02T15:26:30.556 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:26:30.556 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2922240448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2922240448},{"type":"v1","addr":"172.21.15.1:6835","nonce":2922240448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2729497431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2729497431},{"type":"v1","addr":"172.21.15.1:6837","nonce":2729497431}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-02T15:24:47.761435+0000","modified":"2024-02-02T15:25:20.251601+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24529,"mds_2":24518,"mds_3":24514,"mds_4":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14517":{"gid":14517,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/1218303683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":1218303683},{"type":"v1","addr":"172.21.15.1:6839","nonce":1218303683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14529":{"gid":14529,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/2059197496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2059197496},{"type":"v1","addr":"172.21.15.1:6841","nonce":2059197496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/7648986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":7648986},{"type":"v1","addr":"172.21.15.142:6833","nonce":7648986}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/3482746094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3482746094},{"type":"v1","addr":"172.21.15.71:6833","nonce":3482746094}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/388207814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":388207814},{"type":"v1","addr":"172.21.15.71:6835","nonce":388207814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6835/1052606547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1052606547},{"type":"v1","addr":"172.21.15.142:6835","nonce":1052606547}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.142:6837/2870289019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2870289019},{"type":"v1","addr":"172.21.15.142:6837","nonce":2870289019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.71:6837/628016199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":628016199},{"type":"v1","addr":"172.21.15.71:6837","nonce":628016199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.142:6839/321675421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":321675421},{"type":"v1","addr":"172.21.15.142:6839","nonce":321675421}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/1268086493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1268086493},{"type":"v1","addr":"172.21.15.71:6839","nonce":1268086493}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-02T15:26:30.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.558+0000 7f2566a00700 1 -- 172.21.15.1:0/3192970293 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f25400a0520 msgr2=0x7f25400a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:30.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.558+0000 7f2566a00700 1 --2- 172.21.15.1:0/3192970293 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f25400a0520 0x7f25400a29e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2550009c60 tx=0x7f2550009260 comp rx=0 tx=0).stop 2024-02-02T15:26:30.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.558+0000 7f2566a00700 1 -- 172.21.15.1:0/3192970293 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2560074d80 msgr2=0x7f25601992a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:30.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.558+0000 7f2566a00700 1 --2- 172.21.15.1:0/3192970293 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2560074d80 0x7f25601992a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f2548009fd0 tx=0x7f254800b790 comp rx=0 tx=0).stop 2024-02-02T15:26:30.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.558+0000 7f2566a00700 1 -- 172.21.15.1:0/3192970293 shutdown_connections 2024-02-02T15:26:30.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.558+0000 7f2566a00700 1 --2- 172.21.15.1:0/3192970293 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f25400a0520 0x7f25400a29e0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:30.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.558+0000 7f2566a00700 1 --2- 172.21.15.1:0/3192970293 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2560108c00 0x7f25601a0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:30.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.558+0000 7f2566a00700 1 --2- 172.21.15.1:0/3192970293 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25600737a0 0x7f2560198d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:30.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.558+0000 7f2566a00700 1 --2- 172.21.15.1:0/3192970293 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2560074d80 0x7f25601992a0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:30.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.558+0000 7f2566a00700 1 -- 172.21.15.1:0/3192970293 >> 172.21.15.1:0/3192970293 conn(0x7f25600fc290 msgr2=0x7f25600fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:30.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.558+0000 7f2566a00700 1 -- 172.21.15.1:0/3192970293 shutdown_connections 2024-02-02T15:26:30.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.558+0000 7f2566a00700 1 -- 172.21.15.1:0/3192970293 wait complete. 2024-02-02T15:26:30.559 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 27 2024-02-02T15:26:30.577 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T15:26:30.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:30 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/2241273544' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-02T15:26:30.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:30 smithi071 ceph-mon[44707]: pgmap v290: 129 pgs: 129 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 639 B/s wr, 9 op/s 2024-02-02T15:26:30.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:30 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/700114500' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-02T15:26:30.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:30 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/2241273544' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-02T15:26:30.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:30 smithi142 ceph-mon[45728]: pgmap v290: 129 pgs: 129 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 639 B/s wr, 9 op/s 2024-02-02T15:26:30.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:30 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/700114500' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-02T15:26:30.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.774+0000 7f0915c00700 1 -- 172.21.15.1:0/1321857451 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f09101028d0 msgr2=0x7f0910109180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:30.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.774+0000 7f0915c00700 1 --2- 172.21.15.1:0/1321857451 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f09101028d0 0x7f0910109180 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f090800b0d0 tx=0x7f090800b3e0 comp rx=0 tx=0).stop 2024-02-02T15:26:30.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.774+0000 7f0915c00700 1 -- 172.21.15.1:0/1321857451 shutdown_connections 2024-02-02T15:26:30.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.774+0000 7f0915c00700 1 --2- 172.21.15.1:0/1321857451 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f09101028d0 0x7f0910109180 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:30.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.774+0000 7f0915c00700 1 --2- 172.21.15.1:0/1321857451 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0910100d40 0x7f0910101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:30.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.774+0000 7f0915c00700 1 --2- 172.21.15.1:0/1321857451 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0910101f30 0x7f0910102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:30.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.774+0000 7f0915c00700 1 -- 172.21.15.1:0/1321857451 >> 172.21.15.1:0/1321857451 conn(0x7f09100fc290 msgr2=0x7f09100fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:30.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.774+0000 7f0915c00700 1 -- 172.21.15.1:0/1321857451 shutdown_connections 2024-02-02T15:26:30.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.775+0000 7f0915c00700 1 -- 172.21.15.1:0/1321857451 wait complete. 2024-02-02T15:26:30.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.776+0000 7f0915c00700 1 Processor -- start 2024-02-02T15:26:30.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.776+0000 7f0915c00700 1 -- start start 2024-02-02T15:26:30.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.777+0000 7f0915c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0910100d40 0x7f0910198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:30.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.777+0000 7f0915c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0910101f30 0x7f0910199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:30.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.777+0000 7f0915c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09101028d0 0x7f09101a02c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:30.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.777+0000 7f0915c00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09101a0890 con 0x7f0910101f30 2024-02-02T15:26:30.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.777+0000 7f0915c00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09101a0a00 con 0x7f0910100d40 2024-02-02T15:26:30.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.777+0000 7f0915c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09101a0bc0 con 0x7f09101028d0 2024-02-02T15:26:30.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.777+0000 7f090e000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0910101f30 0x7f0910199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:30.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.777+0000 7f090e000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0910101f30 0x7f0910199290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56952/0 (socket says 172.21.15.1:56952) 2024-02-02T15:26:30.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.777+0000 7f090e000700 1 -- 172.21.15.1:0/1342453051 learned_addr learned my addr 172.21.15.1:0/1342453051 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:30.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.777+0000 7f090ea00700 1 --2- 172.21.15.1:0/1342453051 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0910100d40 0x7f0910198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:30.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.777+0000 7f090f400700 1 --2- 172.21.15.1:0/1342453051 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09101028d0 0x7f09101a02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:30.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.777+0000 7f090e000700 1 -- 172.21.15.1:0/1342453051 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09101028d0 msgr2=0x7f09101a02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:30.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.777+0000 7f090e000700 1 --2- 172.21.15.1:0/1342453051 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09101028d0 0x7f09101a02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:30.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.777+0000 7f090e000700 1 -- 172.21.15.1:0/1342453051 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0910100d40 msgr2=0x7f0910198d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:30.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.777+0000 7f090e000700 1 --2- 172.21.15.1:0/1342453051 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0910100d40 0x7f0910198d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:30.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.777+0000 7f090e000700 1 -- 172.21.15.1:0/1342453051 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0908009d00 con 0x7f0910101f30 2024-02-02T15:26:30.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.777+0000 7f090f400700 1 --2- 172.21.15.1:0/1342453051 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09101028d0 0x7f09101a02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:26:30.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.778+0000 7f090e000700 1 --2- 172.21.15.1:0/1342453051 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0910101f30 0x7f0910199290 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f08f800e9a0 tx=0x7f08f800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:30.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.778+0000 7f0907400700 1 -- 172.21.15.1:0/1342453051 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f8009790 con 0x7f0910101f30 2024-02-02T15:26:30.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.778+0000 7f0907400700 1 -- 172.21.15.1:0/1342453051 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f08f8010430 con 0x7f0910101f30 2024-02-02T15:26:30.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.778+0000 7f0915c00700 1 -- 172.21.15.1:0/1342453051 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09101a0ea0 con 0x7f0910101f30 2024-02-02T15:26:30.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.778+0000 7f0907400700 1 -- 172.21.15.1:0/1342453051 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f8018680 con 0x7f0910101f30 2024-02-02T15:26:30.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.778+0000 7f0915c00700 1 -- 172.21.15.1:0/1342453051 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09101a12c0 con 0x7f0910101f30 2024-02-02T15:26:30.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.779+0000 7f0915c00700 1 -- 172.21.15.1:0/1342453051 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f091010c5b0 con 0x7f0910101f30 2024-02-02T15:26:30.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.781+0000 7f0907400700 1 -- 172.21.15.1:0/1342453051 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) v1 ==== 135069+0+0 (secure 0 0 0) 0x7f08f80105a0 con 0x7f0910101f30 2024-02-02T15:26:30.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.782+0000 7f0907400700 1 --2- 172.21.15.1:0/1342453051 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f08f00a0520 0x7f08f00a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:30.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.782+0000 7f0907400700 1 -- 172.21.15.1:0/1342453051 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f08f8014070 con 0x7f0910101f30 2024-02-02T15:26:30.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.783+0000 7f090ea00700 1 --2- 172.21.15.1:0/1342453051 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f08f00a0520 0x7f08f00a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:30.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.783+0000 7f090ea00700 1 --2- 172.21.15.1:0/1342453051 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f08f00a0520 0x7f08f00a29e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f091019a220 tx=0x7f0900009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:30.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:30.786+0000 7f0907400700 1 -- 172.21.15.1:0/1342453051 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f08f8052110 con 0x7f0910101f30 2024-02-02T15:26:31.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.000+0000 7f0915c00700 1 -- 172.21.15.1:0/1342453051 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f09101a1900 con 0x7f0910101f30 2024-02-02T15:26:31.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.001+0000 7f0907400700 1 -- 172.21.15.1:0/1342453051 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+24453 (secure 0 0 0) 0x7f08f80956f0 con 0x7f0910101f30 2024-02-02T15:26:31.002 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:26:31.002 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":88,"fsid":"371a409c-c1de-11ee-95b6-87774f69a715","created":"2024-02-02T15:18:26.746506+0000","modified":"2024-02-02T15:26:28.267447+0000","last_up_change":"2024-02-02T15:24:08.800951+0000","last_in_change":"2024-02-02T15:23:52.784280+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T15:21:22.471754+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-02T15:24:46.195779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-02T15:24:47.226403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-02T15:24:50.300462+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"73e8a07f-3560-4c19-85ed-030d4d8f0f23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6803","nonce":2591906423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6805","nonce":2591906423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6809","nonce":2591906423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6807","nonce":2591906423}]},"public_addr":"172.21.15.1:6803/2591906423","cluster_addr":"172.21.15.1:6805/2591906423","heartbeat_back_addr":"172.21.15.1:6809/2591906423","heartbeat_front_addr":"172.21.15.1:6807/2591906423","state":["exists","up"]},{"osd":1,"uuid":"e53cffec-0ff9-4ed9-8dcf-ff47d04db2bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6801","nonce":747581719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6803","nonce":747581719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6807","nonce":747581719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6805","nonce":747581719}]},"public_addr":"172.21.15.71:6801/747581719","cluster_addr":"172.21.15.71:6803/747581719","heartbeat_back_addr":"172.21.15.71:6807/747581719","heartbeat_front_addr":"172.21.15.71:6805/747581719","state":["exists","up"]},{"osd":2,"uuid":"856c2347-294c-482b-8595-ad77f4792db4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6801","nonce":377511414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6803","nonce":377511414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6807","nonce":377511414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6805","nonce":377511414}]},"public_addr":"172.21.15.142:6801/377511414","cluster_addr":"172.21.15.142:6803/377511414","heartbeat_back_addr":"172.21.15.142:6807/377511414","heartbeat_front_addr":"172.21.15.142:6805/377511414","state":["exists","up"]},{"osd":3,"uuid":"4b946ad2-2da9-4e0b-9d61-d039d8669a42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6811","nonce":1086124477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6813","nonce":1086124477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6817","nonce":1086124477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6815","nonce":1086124477}]},"public_addr":"172.21.15.1:6811/1086124477","cluster_addr":"172.21.15.1:6813/1086124477","heartbeat_back_addr":"172.21.15.1:6817/1086124477","heartbeat_front_addr":"172.21.15.1:6815/1086124477","state":["exists","up"]},{"osd":4,"uuid":"43e36c43-47a0-4634-a45b-1da14a0e19d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6809","nonce":3095380305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6811","nonce":3095380305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6815","nonce":3095380305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6813","nonce":3095380305}]},"public_addr":"172.21.15.71:6809/3095380305","cluster_addr":"172.21.15.71:6811/3095380305","heartbeat_back_addr":"172.21.15.71:6815/3095380305","heartbeat_front_addr":"172.21.15.71:6813/3095380305","state":["exists","up"]},{"osd":5,"uuid":"eb48160e-ea7c-42f7-9477-be110bc8a34e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6809","nonce":4195149262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6811","nonce":4195149262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6815","nonce":4195149262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6813","nonce":4195149262}]},"public_addr":"172.21.15.142:6809/4195149262","cluster_addr":"172.21.15.142:6811/4195149262","heartbeat_back_addr":"172.21.15.142:6815/4195149262","heartbeat_front_addr":"172.21.15.142:6813/4195149262","state":["exists","up"]},{"osd":6,"uuid":"0ec4501a-5853-4ca6-b325-231f5b72335e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6819","nonce":1560153428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6821","nonce":1560153428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6825","nonce":1560153428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6823","nonce":1560153428}]},"public_addr":"172.21.15.1:6819/1560153428","cluster_addr":"172.21.15.1:6821/1560153428","heartbeat_back_addr":"172.21.15.1:6825/1560153428","heartbeat_front_addr":"172.21.15.1:6823/1560153428","state":["exists","up"]},{"osd":7,"uuid":"029debe8-9cdf-44f6-9aea-cabd1d185f93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6817","nonce":4080955538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6819","nonce":4080955538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6823","nonce":4080955538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6821","nonce":4080955538}]},"public_addr":"172.21.15.71:6817/4080955538","cluster_addr":"172.21.15.71:6819/4080955538","heartbeat_back_addr":"172.21.15.71:6823/4080955538","heartbeat_front_addr":"172.21.15.71:6821/4080955538","state":["exists","up"]},{"osd":8,"uuid":"7c90896c-9955-4c1b-aaca-7531dd5457f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6817","nonce":3942038209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6819","nonce":3942038209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6823","nonce":3942038209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6821","nonce":3942038209}]},"public_addr":"172.21.15.142:6817/3942038209","cluster_addr":"172.21.15.142:6819/3942038209","heartbeat_back_addr":"172.21.15.142:6823/3942038209","heartbeat_front_addr":"172.21.15.142:6821/3942038209","state":["exists","up"]},{"osd":9,"uuid":"7e165e68-31a8-4598-a577-de3e90ae5c5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6827","nonce":2716544850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6829","nonce":2716544850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6833","nonce":2716544850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6831","nonce":2716544850}]},"public_addr":"172.21.15.1:6827/2716544850","cluster_addr":"172.21.15.1:6829/2716544850","heartbeat_back_addr":"172.21.15.1:6833/2716544850","heartbeat_front_addr":"172.21.15.1:6831/2716544850","state":["exists","up"]},{"osd":10,"uuid":"48cd5fdb-0887-444a-aab8-a1de24282bb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6825","nonce":84691841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6827","nonce":84691841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6831","nonce":84691841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6829","nonce":84691841}]},"public_addr":"172.21.15.71:6825/84691841","cluster_addr":"172.21.15.71:6827/84691841","heartbeat_back_addr":"172.21.15.71:6831/84691841","heartbeat_front_addr":"172.21.15.71:6829/84691841","state":["exists","up"]},{"osd":11,"uuid":"f011dd27-11b0-4166-adfe-76756533163a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6825","nonce":2290179934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6827","nonce":2290179934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6831","nonce":2290179934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6829","nonce":2290179934}]},"public_addr":"172.21.15.142:6825/2290179934","cluster_addr":"172.21.15.142:6827/2290179934","heartbeat_back_addr":"172.21.15.142:6831/2290179934","heartbeat_front_addr":"172.21.15.142:6829/2290179934","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:20:44.060692+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:01.903817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:19.368258+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:40.028037+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:57.513361+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:22:14.805944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:22:35.188439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:22:53.014412+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:23:10.507420+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:23:31.849765+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:23:50.070612+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:24:07.388625+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":5,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/4092532060":"2024-02-03T15:19:25.836171+0000","172.21.15.1:6801/395498296":"2024-02-03T15:19:25.836171+0000","172.21.15.1:6800/395498296":"2024-02-03T15:19:25.836171+0000","172.21.15.1:0/1575373489":"2024-02-03T15:19:25.836171+0000","172.21.15.1:0/865983374":"2024-02-03T15:19:01.431596+0000","172.21.15.1:0/3165960227":"2024-02-03T15:19:01.431596+0000","172.21.15.1:0/2350830293":"2024-02-03T15:19:01.431596+0000","172.21.15.1:0/2959690250":"2024-02-03T15:19:25.836171+0000","172.21.15.1:6800/1162530732":"2024-02-03T15:19:01.431596+0000","172.21.15.1:6801/1162530732":"2024-02-03T15:19:01.431596+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T15:26:31.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.004+0000 7f0915c00700 1 -- 172.21.15.1:0/1342453051 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f08f00a0520 msgr2=0x7f08f00a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:31.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.004+0000 7f0915c00700 1 --2- 172.21.15.1:0/1342453051 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f08f00a0520 0x7f08f00a29e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f091019a220 tx=0x7f0900009260 comp rx=0 tx=0).stop 2024-02-02T15:26:31.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.004+0000 7f0915c00700 1 -- 172.21.15.1:0/1342453051 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0910101f30 msgr2=0x7f0910199290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:31.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.004+0000 7f0915c00700 1 --2- 172.21.15.1:0/1342453051 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0910101f30 0x7f0910199290 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f08f800e9a0 tx=0x7f08f800ecb0 comp rx=0 tx=0).stop 2024-02-02T15:26:31.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.004+0000 7f0915c00700 1 -- 172.21.15.1:0/1342453051 shutdown_connections 2024-02-02T15:26:31.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.004+0000 7f0915c00700 1 --2- 172.21.15.1:0/1342453051 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f08f00a0520 0x7f08f00a29e0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:31.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.004+0000 7f0915c00700 1 --2- 172.21.15.1:0/1342453051 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f09101028d0 0x7f09101a02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:31.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.004+0000 7f0915c00700 1 --2- 172.21.15.1:0/1342453051 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0910100d40 0x7f0910198d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:31.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.004+0000 7f0915c00700 1 --2- 172.21.15.1:0/1342453051 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0910101f30 0x7f0910199290 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:31.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.004+0000 7f0915c00700 1 -- 172.21.15.1:0/1342453051 >> 172.21.15.1:0/1342453051 conn(0x7f09100fc290 msgr2=0x7f0910107760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:31.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.004+0000 7f0915c00700 1 -- 172.21.15.1:0/1342453051 shutdown_connections 2024-02-02T15:26:31.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.004+0000 7f0915c00700 1 -- 172.21.15.1:0/1342453051 wait complete. 2024-02-02T15:26:31.023 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-02T15:21:22.471754+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-02T15:24:46.195779+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-02T15:24:47.226403+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-02-02T15:24:50.300462+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-02-02T15:26:31.024 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-02T15:26:31.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.224+0000 7f6440c00700 1 -- 172.21.15.1:0/3431731571 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f643c0737a0 msgr2=0x7f643c073c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:31.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.224+0000 7f6440c00700 1 --2- 172.21.15.1:0/3431731571 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f643c0737a0 0x7f643c073c20 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f6424009990 tx=0x7f6424009ca0 comp rx=0 tx=0).stop 2024-02-02T15:26:31.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.224+0000 7f6440c00700 1 -- 172.21.15.1:0/3431731571 shutdown_connections 2024-02-02T15:26:31.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.224+0000 7f6440c00700 1 --2- 172.21.15.1:0/3431731571 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f643c108ce0 0x7f643c10b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:31.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.224+0000 7f6440c00700 1 --2- 172.21.15.1:0/3431731571 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f643c074d80 0x7f643c0731d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:31.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.224+0000 7f6440c00700 1 --2- 172.21.15.1:0/3431731571 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f643c0737a0 0x7f643c073c20 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:31.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.224+0000 7f6440c00700 1 -- 172.21.15.1:0/3431731571 >> 172.21.15.1:0/3431731571 conn(0x7f643c0fc390 msgr2=0x7f643c0fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:31.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.224+0000 7f6440c00700 1 -- 172.21.15.1:0/3431731571 shutdown_connections 2024-02-02T15:26:31.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.224+0000 7f6440c00700 1 -- 172.21.15.1:0/3431731571 wait complete. 2024-02-02T15:26:31.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.225+0000 7f6440c00700 1 Processor -- start 2024-02-02T15:26:31.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.225+0000 7f6440c00700 1 -- start start 2024-02-02T15:26:31.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.225+0000 7f6440c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f643c0737a0 0x7f643c198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:31.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.225+0000 7f6440c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f643c074d80 0x7f643c199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:31.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.225+0000 7f6440c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f643c108ce0 0x7f643c1a02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:31.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.225+0000 7f6440c00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f643c1a0870 con 0x7f643c0737a0 2024-02-02T15:26:31.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.225+0000 7f6440c00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f643c1a09e0 con 0x7f643c108ce0 2024-02-02T15:26:31.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.225+0000 7f6440c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f643c1a0ba0 con 0x7f643c074d80 2024-02-02T15:26:31.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.226+0000 7f6439e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f643c0737a0 0x7f643c198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:31.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.226+0000 7f6439400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f643c074d80 0x7f643c199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:31.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.226+0000 7f643a800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f643c108ce0 0x7f643c1a02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:31.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.226+0000 7f6439e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f643c0737a0 0x7f643c198d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56970/0 (socket says 172.21.15.1:56970) 2024-02-02T15:26:31.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.226+0000 7f6439400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f643c074d80 0x7f643c199270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:35848/0 (socket says 172.21.15.1:35848) 2024-02-02T15:26:31.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.226+0000 7f6439e00700 1 -- 172.21.15.1:0/2417006974 learned_addr learned my addr 172.21.15.1:0/2417006974 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:31.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.226+0000 7f643a800700 1 -- 172.21.15.1:0/2417006974 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f643c074d80 msgr2=0x7f643c199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:31.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.226+0000 7f643a800700 1 --2- 172.21.15.1:0/2417006974 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f643c074d80 0x7f643c199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:31.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.226+0000 7f643a800700 1 -- 172.21.15.1:0/2417006974 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f643c0737a0 msgr2=0x7f643c198d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:31.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.226+0000 7f643a800700 1 --2- 172.21.15.1:0/2417006974 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f643c0737a0 0x7f643c198d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:31.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.226+0000 7f643a800700 1 -- 172.21.15.1:0/2417006974 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64240095f0 con 0x7f643c108ce0 2024-02-02T15:26:31.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.226+0000 7f6439400700 1 --2- 172.21.15.1:0/2417006974 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f643c074d80 0x7f643c199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:26:31.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.226+0000 7f6439e00700 1 --2- 172.21.15.1:0/2417006974 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f643c0737a0 0x7f643c198d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:26:31.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.227+0000 7f643a800700 1 --2- 172.21.15.1:0/2417006974 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f643c108ce0 0x7f643c1a02a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f643400efc0 tx=0x7f643400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:31.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.227+0000 7f6432a00700 1 -- 172.21.15.1:0/2417006974 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6434018070 con 0x7f643c108ce0 2024-02-02T15:26:31.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.227+0000 7f6440c00700 1 -- 172.21.15.1:0/2417006974 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f643c1a0e20 con 0x7f643c108ce0 2024-02-02T15:26:31.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.227+0000 7f6432a00700 1 -- 172.21.15.1:0/2417006974 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f643400f040 con 0x7f643c108ce0 2024-02-02T15:26:31.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.227+0000 7f6432a00700 1 -- 172.21.15.1:0/2417006974 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f643400cd40 con 0x7f643c108ce0 2024-02-02T15:26:31.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.227+0000 7f6440c00700 1 -- 172.21.15.1:0/2417006974 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f643c1a1240 con 0x7f643c108ce0 2024-02-02T15:26:31.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.228+0000 7f6440c00700 1 -- 172.21.15.1:0/2417006974 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f643c107390 con 0x7f643c108ce0 2024-02-02T15:26:31.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.230+0000 7f6432a00700 1 -- 172.21.15.1:0/2417006974 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 135069+0+0 (secure 0 0 0) 0x7f6434003a00 con 0x7f643c108ce0 2024-02-02T15:26:31.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.231+0000 7f6432a00700 1 --2- 172.21.15.1:0/2417006974 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f641c0a0520 0x7f641c0a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:31.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.231+0000 7f6432a00700 1 -- 172.21.15.1:0/2417006974 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f64340cc270 con 0x7f643c108ce0 2024-02-02T15:26:31.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.232+0000 7f6439e00700 1 --2- 172.21.15.1:0/2417006974 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f641c0a0520 0x7f641c0a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:31.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.232+0000 7f6439e00700 1 --2- 172.21.15.1:0/2417006974 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f641c0a0520 0x7f641c0a29e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f643c19a200 tx=0x7f642c009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:31.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.234+0000 7f6432a00700 1 -- 172.21.15.1:0/2417006974 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f6434095ad0 con 0x7f643c108ce0 2024-02-02T15:26:31.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.447+0000 7f6440c00700 1 -- 172.21.15.1:0/2417006974 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f643c02ce00 con 0x7f643c108ce0 2024-02-02T15:26:31.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.448+0000 7f6432a00700 1 -- 172.21.15.1:0/2417006974 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f64340990f0 con 0x7f643c108ce0 2024-02-02T15:26:31.448 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-02-02T15:26:31.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.450+0000 7f6440c00700 1 -- 172.21.15.1:0/2417006974 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f641c0a0520 msgr2=0x7f641c0a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:31.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.450+0000 7f6440c00700 1 --2- 172.21.15.1:0/2417006974 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f641c0a0520 0x7f641c0a29e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f643c19a200 tx=0x7f642c009260 comp rx=0 tx=0).stop 2024-02-02T15:26:31.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.450+0000 7f6440c00700 1 -- 172.21.15.1:0/2417006974 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f643c108ce0 msgr2=0x7f643c1a02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:31.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.450+0000 7f6440c00700 1 --2- 172.21.15.1:0/2417006974 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f643c108ce0 0x7f643c1a02a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f643400efc0 tx=0x7f643400c3a0 comp rx=0 tx=0).stop 2024-02-02T15:26:31.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.451+0000 7f6440c00700 1 -- 172.21.15.1:0/2417006974 shutdown_connections 2024-02-02T15:26:31.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.451+0000 7f6440c00700 1 --2- 172.21.15.1:0/2417006974 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f643c108ce0 0x7f643c1a02a0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:31.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.451+0000 7f6440c00700 1 --2- 172.21.15.1:0/2417006974 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f643c0737a0 0x7f643c198d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:31.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.451+0000 7f6440c00700 1 --2- 172.21.15.1:0/2417006974 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f641c0a0520 0x7f641c0a29e0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:31.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.451+0000 7f6440c00700 1 --2- 172.21.15.1:0/2417006974 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f643c074d80 0x7f643c199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:31.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.451+0000 7f6440c00700 1 -- 172.21.15.1:0/2417006974 >> 172.21.15.1:0/2417006974 conn(0x7f643c0fc390 msgr2=0x7f643c0fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:31.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.451+0000 7f6440c00700 1 -- 172.21.15.1:0/2417006974 shutdown_connections 2024-02-02T15:26:31.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.451+0000 7f6440c00700 1 -- 172.21.15.1:0/2417006974 wait complete. 2024-02-02T15:26:31.469 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-02T15:26:31.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:31 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/3192970293' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:26:31.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:31 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/1342453051' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-02T15:26:31.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.669+0000 7fd394a00700 1 -- 172.21.15.1:0/3281232937 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd390100b20 msgr2=0x7fd390100f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:31.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.669+0000 7fd394a00700 1 --2- 172.21.15.1:0/3281232937 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd390100b20 0x7fd390100f40 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd3800099c0 tx=0x7fd380009cd0 comp rx=0 tx=0).stop 2024-02-02T15:26:31.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.669+0000 7fd394a00700 1 -- 172.21.15.1:0/3281232937 shutdown_connections 2024-02-02T15:26:31.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.669+0000 7fd394a00700 1 --2- 172.21.15.1:0/3281232937 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd3901026b0 0x7fd390108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:31.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.669+0000 7fd394a00700 1 --2- 172.21.15.1:0/3281232937 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd390100b20 0x7fd390100f40 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:31.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.669+0000 7fd394a00700 1 --2- 172.21.15.1:0/3281232937 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd390101d10 0x7fd390102170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:31.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.669+0000 7fd394a00700 1 -- 172.21.15.1:0/3281232937 >> 172.21.15.1:0/3281232937 conn(0x7fd3900fc050 msgr2=0x7fd3900fe4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:31.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.669+0000 7fd394a00700 1 -- 172.21.15.1:0/3281232937 shutdown_connections 2024-02-02T15:26:31.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.669+0000 7fd394a00700 1 -- 172.21.15.1:0/3281232937 wait complete. 2024-02-02T15:26:31.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:31 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/3192970293' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:26:31.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:31 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/1342453051' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-02T15:26:31.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.670+0000 7fd394a00700 1 Processor -- start 2024-02-02T15:26:31.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.671+0000 7fd394a00700 1 -- start start 2024-02-02T15:26:31.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.671+0000 7fd394a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd390100b20 0x7fd390196830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:31.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.671+0000 7fd394a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd390101d10 0x7fd390196d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:31.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.671+0000 7fd394a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd3901026b0 0x7fd39019dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:31.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.671+0000 7fd394a00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd39019e3b0 con 0x7fd3901026b0 2024-02-02T15:26:31.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.671+0000 7fd394a00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd39019e4f0 con 0x7fd390100b20 2024-02-02T15:26:31.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.671+0000 7fd394a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd39019e680 con 0x7fd390101d10 2024-02-02T15:26:31.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.671+0000 7fd38f200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd3901026b0 0x7fd39019dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:31.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.672+0000 7fd38f200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd3901026b0 0x7fd39019dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57002/0 (socket says 172.21.15.1:57002) 2024-02-02T15:26:31.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.672+0000 7fd38f200700 1 -- 172.21.15.1:0/1103801423 learned_addr learned my addr 172.21.15.1:0/1103801423 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:31.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.672+0000 7fd38e800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd390100b20 0x7fd390196830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:31.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.672+0000 7fd38de00700 1 --2- 172.21.15.1:0/1103801423 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd390101d10 0x7fd390196d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:31.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.672+0000 7fd38f200700 1 -- 172.21.15.1:0/1103801423 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd390101d10 msgr2=0x7fd390196d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:31.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.672+0000 7fd38f200700 1 --2- 172.21.15.1:0/1103801423 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd390101d10 0x7fd390196d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:31.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.672+0000 7fd38f200700 1 -- 172.21.15.1:0/1103801423 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd390100b20 msgr2=0x7fd390196830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:31.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.672+0000 7fd38f200700 1 --2- 172.21.15.1:0/1103801423 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd390100b20 0x7fd390196830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:31.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.672+0000 7fd38f200700 1 -- 172.21.15.1:0/1103801423 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3800095f0 con 0x7fd3901026b0 2024-02-02T15:26:31.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.672+0000 7fd38e800700 1 --2- 172.21.15.1:0/1103801423 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd390100b20 0x7fd390196830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:26:31.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.672+0000 7fd38f200700 1 --2- 172.21.15.1:0/1103801423 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd3901026b0 0x7fd39019dde0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd38800ef30 tx=0x7fd38800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:31.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.672+0000 7fd387400700 1 -- 172.21.15.1:0/1103801423 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd388018070 con 0x7fd3901026b0 2024-02-02T15:26:31.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.672+0000 7fd387400700 1 -- 172.21.15.1:0/1103801423 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd38800f040 con 0x7fd3901026b0 2024-02-02T15:26:31.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.673+0000 7fd394a00700 1 -- 172.21.15.1:0/1103801423 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd39019e900 con 0x7fd3901026b0 2024-02-02T15:26:31.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.673+0000 7fd387400700 1 -- 172.21.15.1:0/1103801423 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd38800cd00 con 0x7fd3901026b0 2024-02-02T15:26:31.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.673+0000 7fd394a00700 1 -- 172.21.15.1:0/1103801423 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd39019ee60 con 0x7fd3901026b0 2024-02-02T15:26:31.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.674+0000 7fd394a00700 1 -- 172.21.15.1:0/1103801423 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd390108110 con 0x7fd3901026b0 2024-02-02T15:26:31.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.676+0000 7fd387400700 1 -- 172.21.15.1:0/1103801423 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) v1 ==== 135069+0+0 (secure 0 0 0) 0x7fd388003a00 con 0x7fd3901026b0 2024-02-02T15:26:31.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.677+0000 7fd387400700 1 --2- 172.21.15.1:0/1103801423 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fd3700a05a0 0x7fd3700a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:31.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.677+0000 7fd387400700 1 -- 172.21.15.1:0/1103801423 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fd3880cb810 con 0x7fd3901026b0 2024-02-02T15:26:31.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.677+0000 7fd38e800700 1 --2- 172.21.15.1:0/1103801423 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fd3700a05a0 0x7fd3700a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:31.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.678+0000 7fd38e800700 1 --2- 172.21.15.1:0/1103801423 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fd3700a05a0 0x7fd3700a2a60 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd380009990 tx=0x7fd38001a040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:31.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.681+0000 7fd387400700 1 -- 172.21.15.1:0/1103801423 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7fd388095070 con 0x7fd3901026b0 2024-02-02T15:26:31.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:31 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/3192970293' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:26:31.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:31 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/1342453051' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-02T15:26:31.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.902+0000 7fd394a00700 1 -- 172.21.15.1:0/1103801423 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd3900619f0 con 0x7fd3901026b0 2024-02-02T15:26:31.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.902+0000 7fd387400700 1 -- 172.21.15.1:0/1103801423 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd388098690 con 0x7fd3901026b0 2024-02-02T15:26:31.902 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-02-02T15:26:31.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.905+0000 7fd394a00700 1 -- 172.21.15.1:0/1103801423 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fd3700a05a0 msgr2=0x7fd3700a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:31.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.905+0000 7fd394a00700 1 --2- 172.21.15.1:0/1103801423 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fd3700a05a0 0x7fd3700a2a60 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd380009990 tx=0x7fd38001a040 comp rx=0 tx=0).stop 2024-02-02T15:26:31.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.905+0000 7fd394a00700 1 -- 172.21.15.1:0/1103801423 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd3901026b0 msgr2=0x7fd39019dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:31.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.905+0000 7fd394a00700 1 --2- 172.21.15.1:0/1103801423 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd3901026b0 0x7fd39019dde0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd38800ef30 tx=0x7fd38800c3a0 comp rx=0 tx=0).stop 2024-02-02T15:26:31.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.905+0000 7fd394a00700 1 -- 172.21.15.1:0/1103801423 shutdown_connections 2024-02-02T15:26:31.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.905+0000 7fd394a00700 1 --2- 172.21.15.1:0/1103801423 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd3901026b0 0x7fd39019dde0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:31.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.905+0000 7fd394a00700 1 --2- 172.21.15.1:0/1103801423 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd390100b20 0x7fd390196830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:31.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.905+0000 7fd394a00700 1 --2- 172.21.15.1:0/1103801423 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fd3700a05a0 0x7fd3700a2a60 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:31.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.905+0000 7fd394a00700 1 --2- 172.21.15.1:0/1103801423 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd390101d10 0x7fd390196d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:31.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.905+0000 7fd394a00700 1 -- 172.21.15.1:0/1103801423 >> 172.21.15.1:0/1103801423 conn(0x7fd3900fc050 msgr2=0x7fd390105250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:31.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.905+0000 7fd394a00700 1 -- 172.21.15.1:0/1103801423 shutdown_connections 2024-02-02T15:26:31.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:31.905+0000 7fd394a00700 1 -- 172.21.15.1:0/1103801423 wait complete. 2024-02-02T15:26:31.923 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-02T15:26:32.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.125+0000 7fbde6800700 1 -- 172.21.15.1:0/965317067 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbde0102930 msgr2=0x7fbde0109250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:32.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.125+0000 7fbde6800700 1 --2- 172.21.15.1:0/965317067 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbde0102930 0x7fbde0109250 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fbdd800b0a0 tx=0x7fbdd800b3b0 comp rx=0 tx=0).stop 2024-02-02T15:26:32.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.125+0000 7fbde6800700 1 -- 172.21.15.1:0/965317067 shutdown_connections 2024-02-02T15:26:32.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.125+0000 7fbde6800700 1 --2- 172.21.15.1:0/965317067 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbde0102930 0x7fbde0109250 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:32.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.125+0000 7fbde6800700 1 --2- 172.21.15.1:0/965317067 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbde0100e20 0x7fbde0101240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:32.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.125+0000 7fbde6800700 1 --2- 172.21.15.1:0/965317067 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbde0101f70 0x7fbde01023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:32.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.125+0000 7fbde6800700 1 -- 172.21.15.1:0/965317067 >> 172.21.15.1:0/965317067 conn(0x7fbde00fc390 msgr2=0x7fbde00fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:32.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.126+0000 7fbde6800700 1 -- 172.21.15.1:0/965317067 shutdown_connections 2024-02-02T15:26:32.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.126+0000 7fbde6800700 1 -- 172.21.15.1:0/965317067 wait complete. 2024-02-02T15:26:32.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.127+0000 7fbde6800700 1 Processor -- start 2024-02-02T15:26:32.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.127+0000 7fbde6800700 1 -- start start 2024-02-02T15:26:32.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.127+0000 7fbde6800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbde0100e20 0x7fbde0198de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:32.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.127+0000 7fbde6800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbde0101f70 0x7fbde0199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:32.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.127+0000 7fbde6800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbde0102930 0x7fbde01a0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:32.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.127+0000 7fbde6800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde01a0960 con 0x7fbde0100e20 2024-02-02T15:26:32.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.127+0000 7fbde6800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde01a0aa0 con 0x7fbde0101f70 2024-02-02T15:26:32.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.127+0000 7fbde6800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde01a0c30 con 0x7fbde0102930 2024-02-02T15:26:32.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.128+0000 7fbddf400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbde0100e20 0x7fbde0198de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:32.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.128+0000 7fbddf400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbde0100e20 0x7fbde0198de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57026/0 (socket says 172.21.15.1:57026) 2024-02-02T15:26:32.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.128+0000 7fbddf400700 1 -- 172.21.15.1:0/1402033817 learned_addr learned my addr 172.21.15.1:0/1402033817 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:32.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.128+0000 7fbddea00700 1 --2- 172.21.15.1:0/1402033817 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbde0101f70 0x7fbde0199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:32.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.128+0000 7fbddfe00700 1 --2- 172.21.15.1:0/1402033817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbde0102930 0x7fbde01a0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:32.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.128+0000 7fbddf400700 1 -- 172.21.15.1:0/1402033817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbde0102930 msgr2=0x7fbde01a0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:32.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.128+0000 7fbddf400700 1 --2- 172.21.15.1:0/1402033817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbde0102930 0x7fbde01a0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:32.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.128+0000 7fbddf400700 1 -- 172.21.15.1:0/1402033817 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbde0101f70 msgr2=0x7fbde0199320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:32.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.128+0000 7fbddf400700 1 --2- 172.21.15.1:0/1402033817 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbde0101f70 0x7fbde0199320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:32.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.128+0000 7fbddf400700 1 -- 172.21.15.1:0/1402033817 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdd8009d00 con 0x7fbde0100e20 2024-02-02T15:26:32.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.128+0000 7fbddf400700 1 --2- 172.21.15.1:0/1402033817 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbde0100e20 0x7fbde0198de0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fbdd000e9a0 tx=0x7fbdd000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:32.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.128+0000 7fbdd7e00700 1 -- 172.21.15.1:0/1402033817 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd0009790 con 0x7fbde0100e20 2024-02-02T15:26:32.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.128+0000 7fbdd7e00700 1 -- 172.21.15.1:0/1402033817 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbdd0010430 con 0x7fbde0100e20 2024-02-02T15:26:32.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.129+0000 7fbde6800700 1 -- 172.21.15.1:0/1402033817 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbde01a0f10 con 0x7fbde0100e20 2024-02-02T15:26:32.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.129+0000 7fbdd7e00700 1 -- 172.21.15.1:0/1402033817 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd0018680 con 0x7fbde0100e20 2024-02-02T15:26:32.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.129+0000 7fbde6800700 1 -- 172.21.15.1:0/1402033817 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbde01a1340 con 0x7fbde0100e20 2024-02-02T15:26:32.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.130+0000 7fbde6800700 1 -- 172.21.15.1:0/1402033817 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbde010c670 con 0x7fbde0100e20 2024-02-02T15:26:32.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.132+0000 7fbdd7e00700 1 -- 172.21.15.1:0/1402033817 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) v1 ==== 135069+0+0 (secure 0 0 0) 0x7fbdd00105a0 con 0x7fbde0100e20 2024-02-02T15:26:32.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.133+0000 7fbdd7e00700 1 --2- 172.21.15.1:0/1402033817 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fbdc00a05a0 0x7fbdc00a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:32.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.133+0000 7fbdd7e00700 1 -- 172.21.15.1:0/1402033817 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fbdd0014070 con 0x7fbde0100e20 2024-02-02T15:26:32.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.133+0000 7fbddea00700 1 --2- 172.21.15.1:0/1402033817 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fbdc00a05a0 0x7fbdc00a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:32.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.134+0000 7fbddea00700 1 --2- 172.21.15.1:0/1402033817 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fbdc00a05a0 0x7fbdc00a2a60 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fbdc8009c60 tx=0x7fbdc8009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:32.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.137+0000 7fbdd7e00700 1 -- 172.21.15.1:0/1402033817 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7fbdd0052110 con 0x7fbde0100e20 2024-02-02T15:26:32.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.351+0000 7fbde6800700 1 -- 172.21.15.1:0/1402033817 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbde01a1a40 con 0x7fbde0100e20 2024-02-02T15:26:32.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.352+0000 7fbdd7e00700 1 -- 172.21.15.1:0/1402033817 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbdd0095670 con 0x7fbde0100e20 2024-02-02T15:26:32.352 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 64 2024-02-02T15:26:32.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.354+0000 7fbde6800700 1 -- 172.21.15.1:0/1402033817 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fbdc00a05a0 msgr2=0x7fbdc00a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:32.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.354+0000 7fbde6800700 1 --2- 172.21.15.1:0/1402033817 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fbdc00a05a0 0x7fbdc00a2a60 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fbdc8009c60 tx=0x7fbdc8009260 comp rx=0 tx=0).stop 2024-02-02T15:26:32.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.354+0000 7fbde6800700 1 -- 172.21.15.1:0/1402033817 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbde0100e20 msgr2=0x7fbde0198de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:32.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.354+0000 7fbde6800700 1 --2- 172.21.15.1:0/1402033817 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbde0100e20 0x7fbde0198de0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fbdd000e9a0 tx=0x7fbdd000ecb0 comp rx=0 tx=0).stop 2024-02-02T15:26:32.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.355+0000 7fbde6800700 1 -- 172.21.15.1:0/1402033817 shutdown_connections 2024-02-02T15:26:32.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.355+0000 7fbde6800700 1 --2- 172.21.15.1:0/1402033817 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fbdc00a05a0 0x7fbdc00a2a60 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:32.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.355+0000 7fbde6800700 1 --2- 172.21.15.1:0/1402033817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbde0102930 0x7fbde01a0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:32.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.355+0000 7fbde6800700 1 --2- 172.21.15.1:0/1402033817 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbde0100e20 0x7fbde0198de0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:32.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.355+0000 7fbde6800700 1 --2- 172.21.15.1:0/1402033817 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbde0101f70 0x7fbde0199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:32.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.355+0000 7fbde6800700 1 -- 172.21.15.1:0/1402033817 >> 172.21.15.1:0/1402033817 conn(0x7fbde00fc390 msgr2=0x7fbde00fe520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:32.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.355+0000 7fbde6800700 1 -- 172.21.15.1:0/1402033817 shutdown_connections 2024-02-02T15:26:32.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.355+0000 7fbde6800700 1 -- 172.21.15.1:0/1402033817 wait complete. 2024-02-02T15:26:32.373 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-02-02T15:26:32.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.573+0000 7ff725200700 1 -- 172.21.15.1:0/3844235772 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff720100e20 msgr2=0x7ff720101240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:32.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.573+0000 7ff725200700 1 --2- 172.21.15.1:0/3844235772 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff720100e20 0x7ff720101240 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff710009990 tx=0x7ff710009ca0 comp rx=0 tx=0).stop 2024-02-02T15:26:32.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.573+0000 7ff725200700 1 -- 172.21.15.1:0/3844235772 shutdown_connections 2024-02-02T15:26:32.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.573+0000 7ff725200700 1 --2- 172.21.15.1:0/3844235772 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff720102930 0x7ff720109250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:32.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.573+0000 7ff725200700 1 --2- 172.21.15.1:0/3844235772 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff720100e20 0x7ff720101240 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:32.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.573+0000 7ff725200700 1 --2- 172.21.15.1:0/3844235772 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff720101f70 0x7ff7201023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:32.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.573+0000 7ff725200700 1 -- 172.21.15.1:0/3844235772 >> 172.21.15.1:0/3844235772 conn(0x7ff7200fc390 msgr2=0x7ff7200fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:32.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.573+0000 7ff725200700 1 -- 172.21.15.1:0/3844235772 shutdown_connections 2024-02-02T15:26:32.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.573+0000 7ff725200700 1 -- 172.21.15.1:0/3844235772 wait complete. 2024-02-02T15:26:32.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.574+0000 7ff725200700 1 Processor -- start 2024-02-02T15:26:32.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.574+0000 7ff725200700 1 -- start start 2024-02-02T15:26:32.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.574+0000 7ff725200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff720100e20 0x7ff720198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:32.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.574+0000 7ff725200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff720101f70 0x7ff7201992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:32.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.574+0000 7ff725200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff720102930 0x7ff7201a0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:32.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.574+0000 7ff725200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7201a08f0 con 0x7ff720102930 2024-02-02T15:26:32.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.574+0000 7ff725200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7201a0a60 con 0x7ff720101f70 2024-02-02T15:26:32.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.574+0000 7ff725200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7201a0c20 con 0x7ff720100e20 2024-02-02T15:26:32.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.575+0000 7ff71d400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff720101f70 0x7ff7201992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:32.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.575+0000 7ff71e800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff720102930 0x7ff7201a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:32.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.575+0000 7ff71d400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff720101f70 0x7ff7201992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:36124/0 (socket says 172.21.15.1:36124) 2024-02-02T15:26:32.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.575+0000 7ff71d400700 1 -- 172.21.15.1:0/1780084262 learned_addr learned my addr 172.21.15.1:0/1780084262 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:32.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.575+0000 7ff71de00700 1 --2- 172.21.15.1:0/1780084262 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff720100e20 0x7ff720198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:32.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.575+0000 7ff71d400700 1 -- 172.21.15.1:0/1780084262 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff720100e20 msgr2=0x7ff720198db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:32.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.575+0000 7ff71d400700 1 --2- 172.21.15.1:0/1780084262 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff720100e20 0x7ff720198db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:32.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.575+0000 7ff71d400700 1 -- 172.21.15.1:0/1780084262 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff720102930 msgr2=0x7ff7201a0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:32.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.575+0000 7ff71d400700 1 --2- 172.21.15.1:0/1780084262 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff720102930 0x7ff7201a0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:32.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.575+0000 7ff71d400700 1 -- 172.21.15.1:0/1780084262 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7100095f0 con 0x7ff720101f70 2024-02-02T15:26:32.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.575+0000 7ff71de00700 1 --2- 172.21.15.1:0/1780084262 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff720100e20 0x7ff720198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:26:32.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.575+0000 7ff71e800700 1 --2- 172.21.15.1:0/1780084262 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff720102930 0x7ff7201a0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:26:32.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.575+0000 7ff71d400700 1 --2- 172.21.15.1:0/1780084262 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff720101f70 0x7ff7201992f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff70800e9a0 tx=0x7ff70800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:32.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.576+0000 7ff716a00700 1 -- 172.21.15.1:0/1780084262 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff708009790 con 0x7ff720101f70 2024-02-02T15:26:32.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.576+0000 7ff716a00700 1 -- 172.21.15.1:0/1780084262 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff70800cbd0 con 0x7ff720101f70 2024-02-02T15:26:32.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.576+0000 7ff725200700 1 -- 172.21.15.1:0/1780084262 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7201a0f00 con 0x7ff720101f70 2024-02-02T15:26:32.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.576+0000 7ff716a00700 1 -- 172.21.15.1:0/1780084262 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff708018680 con 0x7ff720101f70 2024-02-02T15:26:32.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.576+0000 7ff725200700 1 -- 172.21.15.1:0/1780084262 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7201a1320 con 0x7ff720101f70 2024-02-02T15:26:32.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.577+0000 7ff725200700 1 -- 172.21.15.1:0/1780084262 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff720193460 con 0x7ff720101f70 2024-02-02T15:26:32.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.579+0000 7ff716a00700 1 -- 172.21.15.1:0/1780084262 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 135069+0+0 (secure 0 0 0) 0x7ff70800cd40 con 0x7ff720101f70 2024-02-02T15:26:32.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.580+0000 7ff716a00700 1 --2- 172.21.15.1:0/1780084262 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7ff7000a0520 0x7ff7000a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:32.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.580+0000 7ff716a00700 1 -- 172.21.15.1:0/1780084262 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9107+0+0 (secure 0 0 0) 0x7ff708014070 con 0x7ff720101f70 2024-02-02T15:26:32.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.580+0000 7ff71de00700 1 --2- 172.21.15.1:0/1780084262 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7ff7000a0520 0x7ff7000a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:32.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.581+0000 7ff71de00700 1 --2- 172.21.15.1:0/1780084262 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7ff7000a0520 0x7ff7000a29e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff71000b2e0 tx=0x7ff710005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:32.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.582+0000 7ff716a00700 1 -- 172.21.15.1:0/1780084262 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7ff708095a60 con 0x7ff720101f70 2024-02-02T15:26:32.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:32 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/2417006974' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-02T15:26:32.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:32 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/1103801423' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-02T15:26:32.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:32 smithi071 ceph-mon[44707]: pgmap v291: 129 pgs: 129 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 639 B/s wr, 11 op/s 2024-02-02T15:26:32.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:32 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/2417006974' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-02T15:26:32.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:32 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/1103801423' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-02T15:26:32.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:32 smithi142 ceph-mon[45728]: pgmap v291: 129 pgs: 129 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 639 B/s wr, 11 op/s 2024-02-02T15:26:32.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:32 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/2417006974' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-02T15:26:32.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:32 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/1103801423' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-02T15:26:32.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:32 smithi001 ceph-mon[35470]: pgmap v291: 129 pgs: 129 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 639 B/s wr, 11 op/s 2024-02-02T15:26:32.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.795+0000 7ff725200700 1 -- 172.21.15.1:0/1780084262 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7ff72002ce00 con 0x7ff720101f70 2024-02-02T15:26:32.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.796+0000 7ff716a00700 1 -- 172.21.15.1:0/1780084262 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v88) v1 ==== 103+0+11 (secure 0 0 0) 0x7ff708099080 con 0x7ff720101f70 2024-02-02T15:26:32.796 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-02-02T15:26:32.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.798+0000 7ff725200700 1 -- 172.21.15.1:0/1780084262 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7ff7000a0520 msgr2=0x7ff7000a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:32.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.798+0000 7ff725200700 1 --2- 172.21.15.1:0/1780084262 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7ff7000a0520 0x7ff7000a29e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff71000b2e0 tx=0x7ff710005fb0 comp rx=0 tx=0).stop 2024-02-02T15:26:32.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.798+0000 7ff725200700 1 -- 172.21.15.1:0/1780084262 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff720101f70 msgr2=0x7ff7201992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:32.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.798+0000 7ff725200700 1 --2- 172.21.15.1:0/1780084262 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff720101f70 0x7ff7201992f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff70800e9a0 tx=0x7ff70800ecb0 comp rx=0 tx=0).stop 2024-02-02T15:26:32.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.798+0000 7ff725200700 1 -- 172.21.15.1:0/1780084262 shutdown_connections 2024-02-02T15:26:32.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.798+0000 7ff725200700 1 --2- 172.21.15.1:0/1780084262 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff720102930 0x7ff7201a0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:32.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.798+0000 7ff725200700 1 --2- 172.21.15.1:0/1780084262 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7ff7000a0520 0x7ff7000a29e0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:32.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.798+0000 7ff725200700 1 --2- 172.21.15.1:0/1780084262 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff720100e20 0x7ff720198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:32.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.798+0000 7ff725200700 1 --2- 172.21.15.1:0/1780084262 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff720101f70 0x7ff7201992f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:32.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.798+0000 7ff725200700 1 -- 172.21.15.1:0/1780084262 >> 172.21.15.1:0/1780084262 conn(0x7ff7200fc390 msgr2=0x7ff720107930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:32.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.798+0000 7ff725200700 1 -- 172.21.15.1:0/1780084262 shutdown_connections 2024-02-02T15:26:32.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:32.798+0000 7ff725200700 1 -- 172.21.15.1:0/1780084262 wait complete. 2024-02-02T15:26:32.818 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T15:26:33.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.016+0000 7f535ea00700 1 -- 172.21.15.1:0/1942186030 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f53580737a0 msgr2=0x7f5358073c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:33.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.016+0000 7f535ea00700 1 --2- 172.21.15.1:0/1942186030 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f53580737a0 0x7f5358073c20 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f5340009990 tx=0x7f5340009ca0 comp rx=0 tx=0).stop 2024-02-02T15:26:33.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.016+0000 7f535ea00700 1 -- 172.21.15.1:0/1942186030 shutdown_connections 2024-02-02T15:26:33.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.016+0000 7f535ea00700 1 --2- 172.21.15.1:0/1942186030 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5358108c00 0x7f535810b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:33.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.016+0000 7f535ea00700 1 --2- 172.21.15.1:0/1942186030 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5358074d80 0x7f53580731d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:33.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.016+0000 7f535ea00700 1 --2- 172.21.15.1:0/1942186030 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f53580737a0 0x7f5358073c20 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:33.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.016+0000 7f535ea00700 1 -- 172.21.15.1:0/1942186030 >> 172.21.15.1:0/1942186030 conn(0x7f53580fc290 msgr2=0x7f53580fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:33.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.016+0000 7f535ea00700 1 -- 172.21.15.1:0/1942186030 shutdown_connections 2024-02-02T15:26:33.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.016+0000 7f535ea00700 1 -- 172.21.15.1:0/1942186030 wait complete. 2024-02-02T15:26:33.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.017+0000 7f535ea00700 1 Processor -- start 2024-02-02T15:26:33.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.018+0000 7f535ea00700 1 -- start start 2024-02-02T15:26:33.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.018+0000 7f535ea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53580737a0 0x7f5358198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:33.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.018+0000 7f535ea00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5358074d80 0x7f53581991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:33.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.018+0000 7f535ea00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5358108c00 0x7f53581a01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:33.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.018+0000 7f535ea00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53581a07b0 con 0x7f5358108c00 2024-02-02T15:26:33.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.018+0000 7f535ea00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53581a0920 con 0x7f5358074d80 2024-02-02T15:26:33.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.018+0000 7f535ea00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53581a0ae0 con 0x7f53580737a0 2024-02-02T15:26:33.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.018+0000 7f535ca00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5358108c00 0x7f53581a01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:33.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.018+0000 7f5357400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5358074d80 0x7f53581991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:33.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.018+0000 7f535ca00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5358108c00 0x7f53581a01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52650/0 (socket says 172.21.15.1:52650) 2024-02-02T15:26:33.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.018+0000 7f5357400700 1 --2- 172.21.15.1:0/392594067 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5358074d80 0x7f53581991b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:57586/0 (socket says 172.21.15.1:57586) 2024-02-02T15:26:33.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.018+0000 7f5357e00700 1 --2- 172.21.15.1:0/392594067 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53580737a0 0x7f5358198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:33.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.018+0000 7f535ca00700 1 -- 172.21.15.1:0/392594067 learned_addr learned my addr 172.21.15.1:0/392594067 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:33.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.019+0000 7f5357400700 1 -- 172.21.15.1:0/392594067 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53580737a0 msgr2=0x7f5358198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:33.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.019+0000 7f5357400700 1 --2- 172.21.15.1:0/392594067 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53580737a0 0x7f5358198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:33.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.019+0000 7f5357400700 1 -- 172.21.15.1:0/392594067 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5358108c00 msgr2=0x7f53581a01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:33.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.019+0000 7f5357400700 1 --2- 172.21.15.1:0/392594067 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5358108c00 0x7f53581a01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:33.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.019+0000 7f5357400700 1 -- 172.21.15.1:0/392594067 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53400095f0 con 0x7f5358074d80 2024-02-02T15:26:33.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.019+0000 7f535ca00700 1 --2- 172.21.15.1:0/392594067 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5358108c00 0x7f53581a01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:26:33.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.019+0000 7f5357400700 1 --2- 172.21.15.1:0/392594067 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5358074d80 0x7f53581991b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f534000b680 tx=0x7f5340009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:33.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.019+0000 7f5354c00700 1 -- 172.21.15.1:0/392594067 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534001d070 con 0x7f5358074d80 2024-02-02T15:26:33.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.019+0000 7f535ea00700 1 -- 172.21.15.1:0/392594067 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53581a0d60 con 0x7f5358074d80 2024-02-02T15:26:33.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.020+0000 7f5354c00700 1 -- 172.21.15.1:0/392594067 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5340021410 con 0x7f5358074d80 2024-02-02T15:26:33.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.020+0000 7f5354c00700 1 -- 172.21.15.1:0/392594067 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5340017650 con 0x7f5358074d80 2024-02-02T15:26:33.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.020+0000 7f535ea00700 1 -- 172.21.15.1:0/392594067 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53581a1190 con 0x7f5358074d80 2024-02-02T15:26:33.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.020+0000 7f535ea00700 1 -- 172.21.15.1:0/392594067 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53581072b0 con 0x7f5358074d80 2024-02-02T15:26:33.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.023+0000 7f5354c00700 1 -- 172.21.15.1:0/392594067 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 135069+0+0 (secure 0 0 0) 0x7f5340021810 con 0x7f5358074d80 2024-02-02T15:26:33.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.024+0000 7f5354c00700 1 --2- 172.21.15.1:0/392594067 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f53380a0520 0x7f53380a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:33.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.024+0000 7f5354c00700 1 -- 172.21.15.1:0/392594067 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f53400cd780 con 0x7f5358074d80 2024-02-02T15:26:33.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.024+0000 7f5357e00700 1 --2- 172.21.15.1:0/392594067 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f53380a0520 0x7f53380a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:33.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.025+0000 7f5357e00700 1 --2- 172.21.15.1:0/392594067 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f53380a0520 0x7f53380a29e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f535819a140 tx=0x7f5348009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:33.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.026+0000 7f5354c00700 1 -- 172.21.15.1:0/392594067 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f5340096fe0 con 0x7f5358074d80 2024-02-02T15:26:33.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.289+0000 7f535ea00700 1 -- 172.21.15.1:0/392594067 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f535802cc70 con 0x7f5358074d80 2024-02-02T15:26:33.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.291+0000 7f5354c00700 1 -- 172.21.15.1:0/392594067 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10866 (secure 0 0 0) 0x7f534009a600 con 0x7f5358074d80 2024-02-02T15:26:33.291 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:26:33.291 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2922240448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2922240448},{"type":"v1","addr":"172.21.15.1:6835","nonce":2922240448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2729497431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2729497431},{"type":"v1","addr":"172.21.15.1:6837","nonce":2729497431}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-02T15:24:47.761435+0000","modified":"2024-02-02T15:25:20.251601+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24529,"mds_2":24518,"mds_3":24514,"mds_4":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14517":{"gid":14517,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/1218303683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":1218303683},{"type":"v1","addr":"172.21.15.1:6839","nonce":1218303683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14529":{"gid":14529,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/2059197496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2059197496},{"type":"v1","addr":"172.21.15.1:6841","nonce":2059197496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/7648986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":7648986},{"type":"v1","addr":"172.21.15.142:6833","nonce":7648986}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/3482746094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3482746094},{"type":"v1","addr":"172.21.15.71:6833","nonce":3482746094}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/388207814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":388207814},{"type":"v1","addr":"172.21.15.71:6835","nonce":388207814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6835/1052606547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1052606547},{"type":"v1","addr":"172.21.15.142:6835","nonce":1052606547}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.142:6837/2870289019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2870289019},{"type":"v1","addr":"172.21.15.142:6837","nonce":2870289019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.71:6837/628016199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":628016199},{"type":"v1","addr":"172.21.15.71:6837","nonce":628016199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.142:6839/321675421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":321675421},{"type":"v1","addr":"172.21.15.142:6839","nonce":321675421}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/1268086493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1268086493},{"type":"v1","addr":"172.21.15.71:6839","nonce":1268086493}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-02T15:26:33.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.294+0000 7f535ea00700 1 -- 172.21.15.1:0/392594067 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f53380a0520 msgr2=0x7f53380a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:33.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.294+0000 7f535ea00700 1 --2- 172.21.15.1:0/392594067 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f53380a0520 0x7f53380a29e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f535819a140 tx=0x7f5348009260 comp rx=0 tx=0).stop 2024-02-02T15:26:33.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.294+0000 7f535ea00700 1 -- 172.21.15.1:0/392594067 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5358074d80 msgr2=0x7f53581991b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:33.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.294+0000 7f535ea00700 1 --2- 172.21.15.1:0/392594067 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5358074d80 0x7f53581991b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f534000b680 tx=0x7f5340009580 comp rx=0 tx=0).stop 2024-02-02T15:26:33.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.294+0000 7f535ea00700 1 -- 172.21.15.1:0/392594067 shutdown_connections 2024-02-02T15:26:33.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.294+0000 7f535ea00700 1 --2- 172.21.15.1:0/392594067 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5358108c00 0x7f53581a01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:33.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.294+0000 7f535ea00700 1 --2- 172.21.15.1:0/392594067 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f53380a0520 0x7f53380a29e0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:33.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.294+0000 7f535ea00700 1 --2- 172.21.15.1:0/392594067 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f53580737a0 0x7f5358198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:33.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.294+0000 7f535ea00700 1 --2- 172.21.15.1:0/392594067 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5358074d80 0x7f53581991b0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:33.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.294+0000 7f535ea00700 1 -- 172.21.15.1:0/392594067 >> 172.21.15.1:0/392594067 conn(0x7f53580fc290 msgr2=0x7f53580fe680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:33.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.294+0000 7f535ea00700 1 -- 172.21.15.1:0/392594067 shutdown_connections 2024-02-02T15:26:33.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.294+0000 7f535ea00700 1 -- 172.21.15.1:0/392594067 wait complete. 2024-02-02T15:26:33.295 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 27 2024-02-02T15:26:33.313 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T15:26:33.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.510+0000 7fd377800700 1 -- 172.21.15.1:0/4146525916 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd370100d50 msgr2=0x7fd370101170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:33.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.510+0000 7fd377800700 1 --2- 172.21.15.1:0/4146525916 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd370100d50 0x7fd370101170 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd3600099c0 tx=0x7fd360009cd0 comp rx=0 tx=0).stop 2024-02-02T15:26:33.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.510+0000 7fd377800700 1 -- 172.21.15.1:0/4146525916 shutdown_connections 2024-02-02T15:26:33.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.510+0000 7fd377800700 1 --2- 172.21.15.1:0/4146525916 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd3701028e0 0x7fd370109190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:33.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.510+0000 7fd377800700 1 --2- 172.21.15.1:0/4146525916 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd370100d50 0x7fd370101170 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:33.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.510+0000 7fd377800700 1 --2- 172.21.15.1:0/4146525916 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd370101f40 0x7fd3701023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:33.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.510+0000 7fd377800700 1 -- 172.21.15.1:0/4146525916 >> 172.21.15.1:0/4146525916 conn(0x7fd3700fc2c0 msgr2=0x7fd3700fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:33.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.510+0000 7fd377800700 1 -- 172.21.15.1:0/4146525916 shutdown_connections 2024-02-02T15:26:33.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.510+0000 7fd377800700 1 -- 172.21.15.1:0/4146525916 wait complete. 2024-02-02T15:26:33.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.511+0000 7fd377800700 1 Processor -- start 2024-02-02T15:26:33.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.511+0000 7fd377800700 1 -- start start 2024-02-02T15:26:33.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.511+0000 7fd377800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd370100d50 0x7fd370198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:33.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.511+0000 7fd377800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd370101f40 0x7fd3701991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:33.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.511+0000 7fd377800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd3701028e0 0x7fd3701a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:33.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.511+0000 7fd377800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3701a07c0 con 0x7fd370101f40 2024-02-02T15:26:33.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.511+0000 7fd377800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3701a0930 con 0x7fd3701028e0 2024-02-02T15:26:33.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.511+0000 7fd377800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3701a0af0 con 0x7fd370100d50 2024-02-02T15:26:33.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.512+0000 7fd36fe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd370101f40 0x7fd3701991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:33.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.512+0000 7fd36fe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd370101f40 0x7fd3701991c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52670/0 (socket says 172.21.15.1:52670) 2024-02-02T15:26:33.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.512+0000 7fd36fe00700 1 -- 172.21.15.1:0/987838252 learned_addr learned my addr 172.21.15.1:0/987838252 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:33.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.512+0000 7fd374e00700 1 --2- 172.21.15.1:0/987838252 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd370100d50 0x7fd370198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:33.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.512+0000 7fd375800700 1 --2- 172.21.15.1:0/987838252 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd3701028e0 0x7fd3701a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:33.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.512+0000 7fd36fe00700 1 -- 172.21.15.1:0/987838252 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd370100d50 msgr2=0x7fd370198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:33.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.512+0000 7fd36fe00700 1 --2- 172.21.15.1:0/987838252 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd370100d50 0x7fd370198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:33.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.512+0000 7fd36fe00700 1 -- 172.21.15.1:0/987838252 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd3701028e0 msgr2=0x7fd3701a01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:33.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.512+0000 7fd36fe00700 1 --2- 172.21.15.1:0/987838252 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd3701028e0 0x7fd3701a01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:33.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.512+0000 7fd36fe00700 1 -- 172.21.15.1:0/987838252 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3600095f0 con 0x7fd370101f40 2024-02-02T15:26:33.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.512+0000 7fd374e00700 1 --2- 172.21.15.1:0/987838252 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd370100d50 0x7fd370198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:26:33.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.512+0000 7fd36fe00700 1 --2- 172.21.15.1:0/987838252 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd370101f40 0x7fd3701991c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd35800e9a0 tx=0x7fd35800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:33.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.513+0000 7fd36d600700 1 -- 172.21.15.1:0/987838252 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd358009790 con 0x7fd370101f40 2024-02-02T15:26:33.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.513+0000 7fd36d600700 1 -- 172.21.15.1:0/987838252 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd358010430 con 0x7fd370101f40 2024-02-02T15:26:33.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.513+0000 7fd377800700 1 -- 172.21.15.1:0/987838252 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3701a0dd0 con 0x7fd370101f40 2024-02-02T15:26:33.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.513+0000 7fd36d600700 1 -- 172.21.15.1:0/987838252 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd358018680 con 0x7fd370101f40 2024-02-02T15:26:33.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.513+0000 7fd377800700 1 -- 172.21.15.1:0/987838252 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3701a1250 con 0x7fd370101f40 2024-02-02T15:26:33.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.514+0000 7fd377800700 1 -- 172.21.15.1:0/987838252 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd37010c5b0 con 0x7fd370101f40 2024-02-02T15:26:33.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.516+0000 7fd36d600700 1 -- 172.21.15.1:0/987838252 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) v1 ==== 135069+0+0 (secure 0 0 0) 0x7fd3580105a0 con 0x7fd370101f40 2024-02-02T15:26:33.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.518+0000 7fd36d600700 1 --2- 172.21.15.1:0/987838252 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fd3500a05a0 0x7fd3500a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:33.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.518+0000 7fd36d600700 1 -- 172.21.15.1:0/987838252 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fd358014070 con 0x7fd370101f40 2024-02-02T15:26:33.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.518+0000 7fd374e00700 1 --2- 172.21.15.1:0/987838252 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fd3500a05a0 0x7fd3500a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:33.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.519+0000 7fd374e00700 1 --2- 172.21.15.1:0/987838252 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fd3500a05a0 0x7fd3500a2a60 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd37019a150 tx=0x7fd360005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:33.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.522+0000 7fd36d600700 1 -- 172.21.15.1:0/987838252 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7fd358052110 con 0x7fd370101f40 2024-02-02T15:26:33.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:33 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/1402033817' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-02T15:26:33.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:33 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/1780084262' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-02T15:26:33.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:33 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/1402033817' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-02T15:26:33.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:33 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/1780084262' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-02T15:26:33.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:33 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/1402033817' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-02T15:26:33.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:33 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/1780084262' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-02T15:26:33.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.734+0000 7fd377800700 1 -- 172.21.15.1:0/987838252 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd37010c740 con 0x7fd370101f40 2024-02-02T15:26:33.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.735+0000 7fd36d600700 1 -- 172.21.15.1:0/987838252 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+24453 (secure 0 0 0) 0x7fd358095670 con 0x7fd370101f40 2024-02-02T15:26:33.735 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:26:33.736 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":88,"fsid":"371a409c-c1de-11ee-95b6-87774f69a715","created":"2024-02-02T15:18:26.746506+0000","modified":"2024-02-02T15:26:28.267447+0000","last_up_change":"2024-02-02T15:24:08.800951+0000","last_in_change":"2024-02-02T15:23:52.784280+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T15:21:22.471754+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-02T15:24:46.195779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-02T15:24:47.226403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-02T15:24:50.300462+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"73e8a07f-3560-4c19-85ed-030d4d8f0f23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6803","nonce":2591906423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6805","nonce":2591906423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6809","nonce":2591906423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6807","nonce":2591906423}]},"public_addr":"172.21.15.1:6803/2591906423","cluster_addr":"172.21.15.1:6805/2591906423","heartbeat_back_addr":"172.21.15.1:6809/2591906423","heartbeat_front_addr":"172.21.15.1:6807/2591906423","state":["exists","up"]},{"osd":1,"uuid":"e53cffec-0ff9-4ed9-8dcf-ff47d04db2bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6801","nonce":747581719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6803","nonce":747581719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6807","nonce":747581719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6805","nonce":747581719}]},"public_addr":"172.21.15.71:6801/747581719","cluster_addr":"172.21.15.71:6803/747581719","heartbeat_back_addr":"172.21.15.71:6807/747581719","heartbeat_front_addr":"172.21.15.71:6805/747581719","state":["exists","up"]},{"osd":2,"uuid":"856c2347-294c-482b-8595-ad77f4792db4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6801","nonce":377511414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6803","nonce":377511414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6807","nonce":377511414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6805","nonce":377511414}]},"public_addr":"172.21.15.142:6801/377511414","cluster_addr":"172.21.15.142:6803/377511414","heartbeat_back_addr":"172.21.15.142:6807/377511414","heartbeat_front_addr":"172.21.15.142:6805/377511414","state":["exists","up"]},{"osd":3,"uuid":"4b946ad2-2da9-4e0b-9d61-d039d8669a42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6811","nonce":1086124477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6813","nonce":1086124477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6817","nonce":1086124477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6815","nonce":1086124477}]},"public_addr":"172.21.15.1:6811/1086124477","cluster_addr":"172.21.15.1:6813/1086124477","heartbeat_back_addr":"172.21.15.1:6817/1086124477","heartbeat_front_addr":"172.21.15.1:6815/1086124477","state":["exists","up"]},{"osd":4,"uuid":"43e36c43-47a0-4634-a45b-1da14a0e19d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6809","nonce":3095380305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6811","nonce":3095380305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6815","nonce":3095380305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6813","nonce":3095380305}]},"public_addr":"172.21.15.71:6809/3095380305","cluster_addr":"172.21.15.71:6811/3095380305","heartbeat_back_addr":"172.21.15.71:6815/3095380305","heartbeat_front_addr":"172.21.15.71:6813/3095380305","state":["exists","up"]},{"osd":5,"uuid":"eb48160e-ea7c-42f7-9477-be110bc8a34e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6809","nonce":4195149262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6811","nonce":4195149262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6815","nonce":4195149262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6813","nonce":4195149262}]},"public_addr":"172.21.15.142:6809/4195149262","cluster_addr":"172.21.15.142:6811/4195149262","heartbeat_back_addr":"172.21.15.142:6815/4195149262","heartbeat_front_addr":"172.21.15.142:6813/4195149262","state":["exists","up"]},{"osd":6,"uuid":"0ec4501a-5853-4ca6-b325-231f5b72335e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6819","nonce":1560153428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6821","nonce":1560153428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6825","nonce":1560153428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6823","nonce":1560153428}]},"public_addr":"172.21.15.1:6819/1560153428","cluster_addr":"172.21.15.1:6821/1560153428","heartbeat_back_addr":"172.21.15.1:6825/1560153428","heartbeat_front_addr":"172.21.15.1:6823/1560153428","state":["exists","up"]},{"osd":7,"uuid":"029debe8-9cdf-44f6-9aea-cabd1d185f93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6817","nonce":4080955538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6819","nonce":4080955538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6823","nonce":4080955538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6821","nonce":4080955538}]},"public_addr":"172.21.15.71:6817/4080955538","cluster_addr":"172.21.15.71:6819/4080955538","heartbeat_back_addr":"172.21.15.71:6823/4080955538","heartbeat_front_addr":"172.21.15.71:6821/4080955538","state":["exists","up"]},{"osd":8,"uuid":"7c90896c-9955-4c1b-aaca-7531dd5457f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6817","nonce":3942038209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6819","nonce":3942038209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6823","nonce":3942038209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6821","nonce":3942038209}]},"public_addr":"172.21.15.142:6817/3942038209","cluster_addr":"172.21.15.142:6819/3942038209","heartbeat_back_addr":"172.21.15.142:6823/3942038209","heartbeat_front_addr":"172.21.15.142:6821/3942038209","state":["exists","up"]},{"osd":9,"uuid":"7e165e68-31a8-4598-a577-de3e90ae5c5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6827","nonce":2716544850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6829","nonce":2716544850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6833","nonce":2716544850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6831","nonce":2716544850}]},"public_addr":"172.21.15.1:6827/2716544850","cluster_addr":"172.21.15.1:6829/2716544850","heartbeat_back_addr":"172.21.15.1:6833/2716544850","heartbeat_front_addr":"172.21.15.1:6831/2716544850","state":["exists","up"]},{"osd":10,"uuid":"48cd5fdb-0887-444a-aab8-a1de24282bb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6825","nonce":84691841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6827","nonce":84691841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6831","nonce":84691841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6829","nonce":84691841}]},"public_addr":"172.21.15.71:6825/84691841","cluster_addr":"172.21.15.71:6827/84691841","heartbeat_back_addr":"172.21.15.71:6831/84691841","heartbeat_front_addr":"172.21.15.71:6829/84691841","state":["exists","up"]},{"osd":11,"uuid":"f011dd27-11b0-4166-adfe-76756533163a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6825","nonce":2290179934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6827","nonce":2290179934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6831","nonce":2290179934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6829","nonce":2290179934}]},"public_addr":"172.21.15.142:6825/2290179934","cluster_addr":"172.21.15.142:6827/2290179934","heartbeat_back_addr":"172.21.15.142:6831/2290179934","heartbeat_front_addr":"172.21.15.142:6829/2290179934","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:20:44.060692+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:01.903817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:19.368258+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:40.028037+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:57.513361+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:22:14.805944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:22:35.188439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:22:53.014412+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:23:10.507420+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:23:31.849765+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:23:50.070612+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:24:07.388625+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":5,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/4092532060":"2024-02-03T15:19:25.836171+0000","172.21.15.1:6801/395498296":"2024-02-03T15:19:25.836171+0000","172.21.15.1:6800/395498296":"2024-02-03T15:19:25.836171+0000","172.21.15.1:0/1575373489":"2024-02-03T15:19:25.836171+0000","172.21.15.1:0/865983374":"2024-02-03T15:19:01.431596+0000","172.21.15.1:0/3165960227":"2024-02-03T15:19:01.431596+0000","172.21.15.1:0/2350830293":"2024-02-03T15:19:01.431596+0000","172.21.15.1:0/2959690250":"2024-02-03T15:19:25.836171+0000","172.21.15.1:6800/1162530732":"2024-02-03T15:19:01.431596+0000","172.21.15.1:6801/1162530732":"2024-02-03T15:19:01.431596+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T15:26:33.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.738+0000 7fd377800700 1 -- 172.21.15.1:0/987838252 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fd3500a05a0 msgr2=0x7fd3500a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:33.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.738+0000 7fd377800700 1 --2- 172.21.15.1:0/987838252 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fd3500a05a0 0x7fd3500a2a60 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd37019a150 tx=0x7fd360005fb0 comp rx=0 tx=0).stop 2024-02-02T15:26:33.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.738+0000 7fd377800700 1 -- 172.21.15.1:0/987838252 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd370101f40 msgr2=0x7fd3701991c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:33.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.738+0000 7fd377800700 1 --2- 172.21.15.1:0/987838252 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd370101f40 0x7fd3701991c0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd35800e9a0 tx=0x7fd35800ecb0 comp rx=0 tx=0).stop 2024-02-02T15:26:33.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.738+0000 7fd377800700 1 -- 172.21.15.1:0/987838252 shutdown_connections 2024-02-02T15:26:33.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.738+0000 7fd377800700 1 --2- 172.21.15.1:0/987838252 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd3701028e0 0x7fd3701a01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:33.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.738+0000 7fd377800700 1 --2- 172.21.15.1:0/987838252 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fd3500a05a0 0x7fd3500a2a60 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:33.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.738+0000 7fd377800700 1 --2- 172.21.15.1:0/987838252 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd370100d50 0x7fd370198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:33.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.738+0000 7fd377800700 1 --2- 172.21.15.1:0/987838252 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd370101f40 0x7fd3701991c0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:33.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.738+0000 7fd377800700 1 -- 172.21.15.1:0/987838252 >> 172.21.15.1:0/987838252 conn(0x7fd3700fc2c0 msgr2=0x7fd3700fe530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:33.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.738+0000 7fd377800700 1 -- 172.21.15.1:0/987838252 shutdown_connections 2024-02-02T15:26:33.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.738+0000 7fd377800700 1 -- 172.21.15.1:0/987838252 wait complete. 2024-02-02T15:26:33.757 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-02-02T15:24:47.761435+0000', 'modified': '2024-02-02T15:25:20.251601+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24557, 'mds_1': 24529, 'mds_2': 24518, 'mds_3': 24514, 'mds_4': 24502}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14517': {'gid': 14517, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.1:6839/1218303683', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6838', 'nonce': 1218303683}, {'type': 'v1', 'addr': '172.21.15.1:6839', 'nonce': 1218303683}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_14529': {'gid': 14529, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.1:6841/2059197496', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.1:6840', 'nonce': 2059197496}, {'type': 'v1', 'addr': '172.21.15.1:6841', 'nonce': 2059197496}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24457': {'gid': 24457, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.142:6833/7648986', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6832', 'nonce': 7648986}, {'type': 'v1', 'addr': '172.21.15.142:6833', 'nonce': 7648986}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24458': {'gid': 24458, 'name': 'k', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.71:6833/3482746094', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6832', 'nonce': 3482746094}, {'type': 'v1', 'addr': '172.21.15.71:6833', 'nonce': 3482746094}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24494': {'gid': 24494, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.71:6835/388207814', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6834', 'nonce': 388207814}, {'type': 'v1', 'addr': '172.21.15.71:6835', 'nonce': 388207814}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24502': {'gid': 24502, 'name': 'f', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.142:6835/1052606547', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6834', 'nonce': 1052606547}, {'type': 'v1', 'addr': '172.21.15.142:6835', 'nonce': 1052606547}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24514': {'gid': 24514, 'name': 'c', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.142:6837/2870289019', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6836', 'nonce': 2870289019}, {'type': 'v1', 'addr': '172.21.15.142:6837', 'nonce': 2870289019}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24518': {'gid': 24518, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.71:6837/628016199', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6836', 'nonce': 628016199}, {'type': 'v1', 'addr': '172.21.15.71:6837', 'nonce': 628016199}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24529': {'gid': 24529, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.142:6839/321675421', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.142:6838', 'nonce': 321675421}, {'type': 'v1', 'addr': '172.21.15.142:6839', 'nonce': 321675421}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24557': {'gid': 24557, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.71:6839/1268086493', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6838', 'nonce': 1268086493}, {'type': 'v1', 'addr': '172.21.15.71:6839', 'nonce': 1268086493}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-02-02T15:26:33.757 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-02-02T15:26:33.758 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2024-02-02T15:26:33.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.957+0000 7fc289200700 1 -- 172.21.15.1:0/1457230631 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc284074d20 msgr2=0x7fc284073170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:33.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.957+0000 7fc289200700 1 --2- 172.21.15.1:0/1457230631 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc284074d20 0x7fc284073170 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc2740098f0 tx=0x7fc274009c00 comp rx=0 tx=0).stop 2024-02-02T15:26:33.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.958+0000 7fc289200700 1 -- 172.21.15.1:0/1457230631 shutdown_connections 2024-02-02T15:26:33.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.958+0000 7fc289200700 1 --2- 172.21.15.1:0/1457230631 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc284108ce0 0x7fc28410b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:33.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.958+0000 7fc289200700 1 --2- 172.21.15.1:0/1457230631 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc284074d20 0x7fc284073170 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:33.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.958+0000 7fc289200700 1 --2- 172.21.15.1:0/1457230631 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc284073740 0x7fc284073bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:33.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.958+0000 7fc289200700 1 -- 172.21.15.1:0/1457230631 >> 172.21.15.1:0/1457230631 conn(0x7fc2840fc3a0 msgr2=0x7fc2840fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:33.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.958+0000 7fc289200700 1 -- 172.21.15.1:0/1457230631 shutdown_connections 2024-02-02T15:26:33.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.958+0000 7fc289200700 1 -- 172.21.15.1:0/1457230631 wait complete. 2024-02-02T15:26:33.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.959+0000 7fc289200700 1 Processor -- start 2024-02-02T15:26:33.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.959+0000 7fc289200700 1 -- start start 2024-02-02T15:26:33.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.960+0000 7fc289200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc284073740 0x7fc284198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:33.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.960+0000 7fc289200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc284074d20 0x7fc284199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:33.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.960+0000 7fc289200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc284108ce0 0x7fc2841a02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:33.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.960+0000 7fc289200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2841a0870 con 0x7fc284074d20 2024-02-02T15:26:33.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.960+0000 7fc289200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2841a09e0 con 0x7fc284108ce0 2024-02-02T15:26:33.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.960+0000 7fc289200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2841a0ba0 con 0x7fc284073740 2024-02-02T15:26:33.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.960+0000 7fc281400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc284074d20 0x7fc284199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:33.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.960+0000 7fc281e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc284073740 0x7fc284198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:33.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.960+0000 7fc281400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc284074d20 0x7fc284199270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52692/0 (socket says 172.21.15.1:52692) 2024-02-02T15:26:33.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.960+0000 7fc281400700 1 -- 172.21.15.1:0/4065558828 learned_addr learned my addr 172.21.15.1:0/4065558828 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:33.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.960+0000 7fc281e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc284073740 0x7fc284198d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:41938/0 (socket says 172.21.15.1:41938) 2024-02-02T15:26:33.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.960+0000 7fc282800700 1 --2- 172.21.15.1:0/4065558828 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc284108ce0 0x7fc2841a02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:33.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.961+0000 7fc281e00700 1 -- 172.21.15.1:0/4065558828 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc284108ce0 msgr2=0x7fc2841a02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:33.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.961+0000 7fc281e00700 1 --2- 172.21.15.1:0/4065558828 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc284108ce0 0x7fc2841a02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:33.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.961+0000 7fc281e00700 1 -- 172.21.15.1:0/4065558828 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc284074d20 msgr2=0x7fc284199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:33.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.961+0000 7fc281e00700 1 --2- 172.21.15.1:0/4065558828 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc284074d20 0x7fc284199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:33.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.961+0000 7fc281e00700 1 -- 172.21.15.1:0/4065558828 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc274009520 con 0x7fc284073740 2024-02-02T15:26:33.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.961+0000 7fc281400700 1 --2- 172.21.15.1:0/4065558828 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc284074d20 0x7fc284199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:26:33.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.961+0000 7fc281e00700 1 --2- 172.21.15.1:0/4065558828 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc284073740 0x7fc284198d30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc274009230 tx=0x7fc27400b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:33.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.961+0000 7fc27aa00700 1 -- 172.21.15.1:0/4065558828 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc27401c070 con 0x7fc284073740 2024-02-02T15:26:33.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.961+0000 7fc289200700 1 -- 172.21.15.1:0/4065558828 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2841a0e20 con 0x7fc284073740 2024-02-02T15:26:33.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.961+0000 7fc27aa00700 1 -- 172.21.15.1:0/4065558828 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc27400f7d0 con 0x7fc284073740 2024-02-02T15:26:33.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.961+0000 7fc289200700 1 -- 172.21.15.1:0/4065558828 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2841a11e0 con 0x7fc284073740 2024-02-02T15:26:33.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.961+0000 7fc27aa00700 1 -- 172.21.15.1:0/4065558828 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc274003a00 con 0x7fc284073740 2024-02-02T15:26:33.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.964+0000 7fc27aa00700 1 -- 172.21.15.1:0/4065558828 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 18) v1 ==== 135069+0+0 (secure 0 0 0) 0x7fc274020030 con 0x7fc284073740 2024-02-02T15:26:33.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.965+0000 7fc29cd2fb80 1 Processor -- start 2024-02-02T15:26:33.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.966+0000 7fc29cd2fb80 1 -- start start 2024-02-02T15:26:33.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.966+0000 7fc27aa00700 1 --2- 172.21.15.1:0/4065558828 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fc2640a05a0 0x7fc2640a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:33.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.966+0000 7fc29cd2fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560371ae3130 0x560371ae3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:33.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.966+0000 7fc29cd2fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560371ae3a90 0x560371ae43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:33.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.966+0000 7fc29cd2fb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560371ae4930 0x560371ae4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:33.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.966+0000 7fc29cd2fb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x560371a56470 con 0x560371ae4930 2024-02-02T15:26:33.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.966+0000 7fc29cd2fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x560371a3aa50 con 0x560371ae3130 2024-02-02T15:26:33.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.966+0000 7fc29cd2fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560371a38410 con 0x560371ae3a90 2024-02-02T15:26:33.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.966+0000 7fc27aa00700 1 -- 172.21.15.1:0/4065558828 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fc2740ccc00 con 0x7fc284073740 2024-02-02T15:26:33.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.966+0000 7fc282800700 1 --2- 172.21.15.1:0/4065558828 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fc2640a05a0 0x7fc2640a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:33.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.966+0000 7fc282800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560371ae4930 0x560371ae4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:33.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.966+0000 7fc282800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560371ae4930 0x560371ae4dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52708/0 (socket says 172.21.15.1:52708) 2024-02-02T15:26:33.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.966+0000 7fc282800700 1 -- 172.21.15.1:0/62303115 learned_addr learned my addr 172.21.15.1:0/62303115 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:33.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.967+0000 7fc281400700 1 --2- 172.21.15.1:0/62303115 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560371ae3a90 0x560371ae43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:33.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.967+0000 7fc281e00700 1 --2- 172.21.15.1:0/62303115 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560371ae3130 0x560371ae3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:33.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.967+0000 7fc282800700 1 -- 172.21.15.1:0/62303115 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560371ae3a90 msgr2=0x560371ae43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:33.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.967+0000 7fc282800700 1 --2- 172.21.15.1:0/62303115 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560371ae3a90 0x560371ae43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:33.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.967+0000 7fc282800700 1 -- 172.21.15.1:0/62303115 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560371ae3130 msgr2=0x560371ae3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:33.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.967+0000 7fc282800700 1 --2- 172.21.15.1:0/62303115 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560371ae3130 0x560371ae3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:33.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.967+0000 7fc282800700 1 -- 172.21.15.1:0/62303115 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5603718b5360 con 0x560371ae4930 2024-02-02T15:26:33.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.967+0000 7fc281e00700 1 --2- 172.21.15.1:0/62303115 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560371ae3130 0x560371ae3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:26:33.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.967+0000 7fc282800700 1 --2- 172.21.15.1:0/62303115 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560371ae4930 0x560371ae4dc0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fc27c008f80 tx=0x7fc27c01a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df94d3e44976a361 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:33.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.967+0000 7fc282800700 1 --2- 172.21.15.1:0/4065558828 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fc2640a05a0 0x7fc2640a2a60 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc27c009fd0 tx=0x7fc27c010000 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:33.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.967+0000 7fc273400700 1 -- 172.21.15.1:0/62303115 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc27c027070 con 0x560371ae4930 2024-02-02T15:26:33.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.968+0000 7fc273400700 1 -- 172.21.15.1:0/62303115 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc27c01f070 con 0x560371ae4930 2024-02-02T15:26:33.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.968+0000 7fc273400700 1 -- 172.21.15.1:0/62303115 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc27c00e040 con 0x560371ae4930 2024-02-02T15:26:33.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.968+0000 7fc29cd2fb80 1 -- 172.21.15.1:0/62303115 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560371ae4930 msgr2=0x560371ae4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:33.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.968+0000 7fc29cd2fb80 1 --2- 172.21.15.1:0/62303115 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560371ae4930 0x560371ae4dc0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fc27c008f80 tx=0x7fc27c01a6c0 comp rx=0 tx=0).stop 2024-02-02T15:26:33.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.968+0000 7fc29cd2fb80 1 -- 172.21.15.1:0/62303115 shutdown_connections 2024-02-02T15:26:33.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.968+0000 7fc29cd2fb80 1 --2- 172.21.15.1:0/62303115 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560371ae4930 0x560371ae4dc0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:33.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.968+0000 7fc29cd2fb80 1 --2- 172.21.15.1:0/62303115 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560371ae3130 0x560371ae3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:33.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.968+0000 7fc29cd2fb80 1 --2- 172.21.15.1:0/62303115 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560371ae3a90 0x560371ae43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:33.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.968+0000 7fc29cd2fb80 1 -- 172.21.15.1:0/62303115 >> 172.21.15.1:0/62303115 conn(0x560371a41d60 msgr2=0x560371adde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:33.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.968+0000 7fc29cd2fb80 1 -- 172.21.15.1:0/62303115 shutdown_connections 2024-02-02T15:26:33.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.968+0000 7fc29cd2fb80 1 -- 172.21.15.1:0/62303115 wait complete. 2024-02-02T15:26:33.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.969+0000 7fc29cd2fb80 1 -- start start 2024-02-02T15:26:33.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.969+0000 7fc29cd2fb80 1 Processor -- start 2024-02-02T15:26:33.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.970+0000 7fc29cd2fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560371ae3130 0x560371aecf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:33.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.970+0000 7fc29cd2fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560371ae3a90 0x560371aed490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:33.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.970+0000 7fc29cd2fb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560371ae4930 0x560371af44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:33.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.970+0000 7fc29cd2fb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x560371a33d00 con 0x560371ae4930 2024-02-02T15:26:33.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.970+0000 7fc29cd2fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603719cfd30 con 0x560371ae3130 2024-02-02T15:26:33.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.970+0000 7fc29cd2fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603719cf9d0 con 0x560371ae3a90 2024-02-02T15:26:33.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.970+0000 7fc29cd2fb80 10 client.0 _ll_register_callbacks cb 0x5603719d1240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:26:33.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.970+0000 7fc281e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560371ae3130 0x560371aecf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:33.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.970+0000 7fc281400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560371ae3a90 0x560371aed490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:33.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.970+0000 7fc281e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560371ae3130 0x560371aecf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:57630/0 (socket says 172.21.15.1:57630) 2024-02-02T15:26:33.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.970+0000 7fc281e00700 1 -- 172.21.15.1:0/1561813292 learned_addr learned my addr 172.21.15.1:0/1561813292 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:33.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.970+0000 7fc282800700 1 --2- 172.21.15.1:0/1561813292 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560371ae4930 0x560371af44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:33.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.970+0000 7fc281400700 1 -- 172.21.15.1:0/1561813292 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560371ae3130 msgr2=0x560371aecf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:33.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.970+0000 7fc281400700 1 --2- 172.21.15.1:0/1561813292 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560371ae3130 0x560371aecf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:33.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.970+0000 7fc281400700 1 -- 172.21.15.1:0/1561813292 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560371ae4930 msgr2=0x560371af44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:33.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.970+0000 7fc281400700 1 --2- 172.21.15.1:0/1561813292 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560371ae4930 0x560371af44d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:33.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.970+0000 7fc281400700 1 -- 172.21.15.1:0/1561813292 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2841a11e0 con 0x560371ae3a90 2024-02-02T15:26:33.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.970+0000 7fc282800700 1 --2- 172.21.15.1:0/1561813292 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560371ae4930 0x560371af44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:26:33.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.970+0000 7fc281e00700 1 --2- 172.21.15.1:0/1561813292 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560371ae3130 0x560371aecf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:26:33.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.971+0000 7fc281400700 1 --2- 172.21.15.1:0/1561813292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560371ae3a90 0x560371aed490 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc26c005950 tx=0x7fc26c009e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fad1be97d52cb35a server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:33.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.971+0000 7fc26be00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:26:33.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.971+0000 7fc26be00700 1 -- 172.21.15.1:0/1561813292 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc26c017070 con 0x560371ae3a90 2024-02-02T15:26:33.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.971+0000 7fc26be00700 1 -- 172.21.15.1:0/1561813292 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fc26c004050 con 0x560371ae3a90 2024-02-02T15:26:33.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.971+0000 7fc29cd2fb80 1 -- 172.21.15.1:0/1561813292 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560371ad60a0 con 0x560371ae3a90 2024-02-02T15:26:33.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.971+0000 7fc26be00700 1 -- 172.21.15.1:0/1561813292 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc26c004210 con 0x560371ae3a90 2024-02-02T15:26:33.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.971+0000 7fc29cd2fb80 10 client.25339 fetch_fsmap learned FSMap version 27 2024-02-02T15:26:33.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.971+0000 7fc29cd2fb80 1 -- 172.21.15.1:0/1561813292 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7fc27c008b30 con 0x560371ae3a90 2024-02-02T15:26:33.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.972+0000 7fc26be00700 1 -- 172.21.15.1:0/1561813292 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560371ad60a0 con 0x560371ae3a90 2024-02-02T15:26:33.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.972+0000 7fc26be00700 1 -- 172.21.15.1:0/1561813292 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc26c0038e0 con 0x560371ae3a90 2024-02-02T15:26:33.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.972+0000 7fc29cd2fb80 10 client.25339 fetch_fsmap finished waiting for FSMap version 27 2024-02-02T15:26:33.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.972+0000 7fc29cd2fb80 10 client.25339 resolve_mds: resolved name 'f' to daemon mds.f 2024-02-02T15:26:33.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.972+0000 7fc29cd2fb80 20 client.25339 populate_metadata read hostname 'smithi001' 2024-02-02T15:26:33.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.972+0000 7fc29cd2fb80 1 --2- 172.21.15.1:0/1561813292 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x560371af8230 0x560371af7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:33.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.972+0000 7fc29cd2fb80 4 client.25339 mds_command: new command op to 24502 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:26:33.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.972+0000 7fc29cd2fb80 1 -- 172.21.15.1:0/1561813292 --> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560371a2cd80 con 0x560371af8230 2024-02-02T15:26:33.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.972+0000 7fc281e00700 1 --2- 172.21.15.1:0/1561813292 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x560371af8230 0x560371af7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:33.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.973+0000 7fc281e00700 1 --2- 172.21.15.1:0/1561813292 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x560371af8230 0x560371af7630 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bab1fad40acfc9f9 server_cookie=f2a1f01980b03ee3 in_seq=0 out_seq=0 2024-02-02T15:26:33.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.973+0000 7fc26be00700 10 client.25339 ms_handle_connect on v2:172.21.15.142:6834/1052606547 2024-02-02T15:26:33.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.973+0000 7fc26be00700 1 -- 172.21.15.1:0/1561813292 <== mds.4 v2:172.21.15.142:6834/1052606547 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x560371a2cd80 con 0x560371af8230 2024-02-02T15:26:33.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.973+0000 7fc26be00700 10 client.25339 handle_command_reply: tid=0 2024-02-02T15:26:33.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.974+0000 7fc29cd2fb80 1 client.25339 shutdown 2024-02-02T15:26:33.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.974+0000 7fc29cd2fb80 1 -- 172.21.15.1:0/1561813292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560371ae3a90 msgr2=0x560371aed490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:33.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.974+0000 7fc29cd2fb80 1 --2- 172.21.15.1:0/1561813292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560371ae3a90 0x560371aed490 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc26c005950 tx=0x7fc26c009e80 comp rx=0 tx=0).stop 2024-02-02T15:26:33.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.974+0000 7fc29cd2fb80 1 -- 172.21.15.1:0/1561813292 shutdown_connections 2024-02-02T15:26:33.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.974+0000 7fc29cd2fb80 1 --2- 172.21.15.1:0/1561813292 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x560371af8230 0x560371af7630 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:33.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.974+0000 7fc29cd2fb80 1 --2- 172.21.15.1:0/1561813292 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560371ae4930 0x560371af44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:33.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.974+0000 7fc29cd2fb80 1 --2- 172.21.15.1:0/1561813292 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560371ae3130 0x560371aecf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:33.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.974+0000 7fc26be00700 0 client.25339 ms_handle_reset on v2:172.21.15.142:6834/1052606547 2024-02-02T15:26:33.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.974+0000 7fc29cd2fb80 1 --2- 172.21.15.1:0/1561813292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560371ae3a90 0x560371aed490 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:33.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.974+0000 7fc29cd2fb80 1 -- 172.21.15.1:0/1561813292 >> 172.21.15.1:0/1561813292 conn(0x560371a41d60 msgr2=0x560371add570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:33.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.974+0000 7fc29cd2fb80 1 -- 172.21.15.1:0/1561813292 shutdown_connections 2024-02-02T15:26:33.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.974+0000 7fc29cd2fb80 1 -- 172.21.15.1:0/1561813292 wait complete. 2024-02-02T15:26:33.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.974+0000 7fc29cd2fb80 20 client.25339 trim_cache size 0 max 16384 2024-02-02T15:26:33.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.992+0000 7fc29cd2fb80 1 Processor -- start 2024-02-02T15:26:33.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.992+0000 7fc29cd2fb80 1 -- start start 2024-02-02T15:26:33.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.992+0000 7fc29cd2fb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560371a41d60 0x560371aead40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:33.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.992+0000 7fc29cd2fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560371ae3130 0x560371b00f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:33.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.992+0000 7fc29cd2fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560371ae3a90 0x560371af7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:33.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.992+0000 7fc29cd2fb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603718b7650 con 0x560371a41d60 2024-02-02T15:26:33.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.992+0000 7fc29cd2fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x560371a2cd80 con 0x560371ae3a90 2024-02-02T15:26:33.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.992+0000 7fc29cd2fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560371ae4000 con 0x560371ae3130 2024-02-02T15:26:33.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.993+0000 7fc281e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560371a41d60 0x560371aead40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:33.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.993+0000 7fc282800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560371ae3a90 0x560371af7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:33.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.993+0000 7fc281e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560371a41d60 0x560371aead40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52718/0 (socket says 172.21.15.1:52718) 2024-02-02T15:26:33.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.993+0000 7fc281e00700 1 -- 172.21.15.1:0/379060022 learned_addr learned my addr 172.21.15.1:0/379060022 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:33.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.993+0000 7fc281400700 1 --2- 172.21.15.1:0/379060022 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560371ae3130 0x560371b00f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:33.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.993+0000 7fc282800700 1 -- 172.21.15.1:0/379060022 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560371ae3130 msgr2=0x560371b00f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:33.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.993+0000 7fc282800700 1 --2- 172.21.15.1:0/379060022 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560371ae3130 0x560371b00f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:33.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.993+0000 7fc282800700 1 -- 172.21.15.1:0/379060022 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560371a41d60 msgr2=0x560371aead40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:33.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.993+0000 7fc282800700 1 --2- 172.21.15.1:0/379060022 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560371a41d60 0x560371aead40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:33.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.993+0000 7fc282800700 1 -- 172.21.15.1:0/379060022 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560371861b10 con 0x560371ae3a90 2024-02-02T15:26:33.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.993+0000 7fc281400700 1 --2- 172.21.15.1:0/379060022 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560371ae3130 0x560371b00f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:26:33.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.993+0000 7fc281e00700 1 --2- 172.21.15.1:0/379060022 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560371a41d60 0x560371aead40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:26:33.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.993+0000 7fc282800700 1 --2- 172.21.15.1:0/379060022 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560371ae3a90 0x560371af7630 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc27c01ae40 tx=0x7fc27c006080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9c50740e5ad171f server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:33.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.994+0000 7fc273e00700 1 -- 172.21.15.1:0/379060022 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc27c00e030 con 0x560371ae3a90 2024-02-02T15:26:33.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.994+0000 7fc273e00700 1 -- 172.21.15.1:0/379060022 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc27c00f040 con 0x560371ae3a90 2024-02-02T15:26:33.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.994+0000 7fc273e00700 1 -- 172.21.15.1:0/379060022 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc27c027040 con 0x560371ae3a90 2024-02-02T15:26:33.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.994+0000 7fc29cd2fb80 1 -- 172.21.15.1:0/379060022 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560371ae3a90 msgr2=0x560371af7630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:33.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.994+0000 7fc29cd2fb80 1 --2- 172.21.15.1:0/379060022 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560371ae3a90 0x560371af7630 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc27c01ae40 tx=0x7fc27c006080 comp rx=0 tx=0).stop 2024-02-02T15:26:33.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.994+0000 7fc29cd2fb80 1 -- 172.21.15.1:0/379060022 shutdown_connections 2024-02-02T15:26:33.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.994+0000 7fc29cd2fb80 1 --2- 172.21.15.1:0/379060022 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560371ae3a90 0x560371af7630 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:33.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.994+0000 7fc29cd2fb80 1 --2- 172.21.15.1:0/379060022 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560371a41d60 0x560371aead40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:33.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.994+0000 7fc29cd2fb80 1 --2- 172.21.15.1:0/379060022 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560371ae3130 0x560371b00f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:33.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.994+0000 7fc29cd2fb80 1 -- 172.21.15.1:0/379060022 >> 172.21.15.1:0/379060022 conn(0x560371ade1c0 msgr2=0x560371af4d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:33.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.994+0000 7fc29cd2fb80 1 -- 172.21.15.1:0/379060022 shutdown_connections 2024-02-02T15:26:33.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.994+0000 7fc29cd2fb80 1 -- 172.21.15.1:0/379060022 wait complete. 2024-02-02T15:26:33.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.995+0000 7fc29cd2fb80 1 -- start start 2024-02-02T15:26:33.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.995+0000 7fc29cd2fb80 1 Processor -- start 2024-02-02T15:26:33.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.995+0000 7fc29cd2fb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560371a41d60 0x560371afc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:33.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.995+0000 7fc29cd2fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560371ae3130 0x560371afc730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:33.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.995+0000 7fc29cd2fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560371ae3a90 0x560371b49d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:34.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.995+0000 7fc29cd2fb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x560371afcce0 con 0x560371a41d60 2024-02-02T15:26:34.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.995+0000 7fc29cd2fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x560371afce50 con 0x560371ae3a90 2024-02-02T15:26:34.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.995+0000 7fc29cd2fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560371b4a2c0 con 0x560371ae3130 2024-02-02T15:26:34.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.995+0000 7fc29cd2fb80 10 client.0 _ll_register_callbacks cb 0x5603719d1240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:26:34.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.996+0000 7fc281e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560371a41d60 0x560371afc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:34.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.996+0000 7fc282800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560371ae3a90 0x560371b49d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:34.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.996+0000 7fc281e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560371a41d60 0x560371afc1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52732/0 (socket says 172.21.15.1:52732) 2024-02-02T15:26:34.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.996+0000 7fc281e00700 1 -- 172.21.15.1:0/642014003 learned_addr learned my addr 172.21.15.1:0/642014003 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:34.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.996+0000 7fc281400700 1 --2- 172.21.15.1:0/642014003 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560371ae3130 0x560371afc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:34.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.996+0000 7fc281e00700 1 -- 172.21.15.1:0/642014003 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560371ae3130 msgr2=0x560371afc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:34.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.996+0000 7fc281e00700 1 --2- 172.21.15.1:0/642014003 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560371ae3130 0x560371afc730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.996+0000 7fc281e00700 1 -- 172.21.15.1:0/642014003 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560371ae3a90 msgr2=0x560371b49d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:34.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.996+0000 7fc281e00700 1 --2- 172.21.15.1:0/642014003 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560371ae3a90 0x560371b49d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.996+0000 7fc281e00700 1 -- 172.21.15.1:0/642014003 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2841a0e20 con 0x560371a41d60 2024-02-02T15:26:34.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.996+0000 7fc281e00700 1 --2- 172.21.15.1:0/642014003 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560371a41d60 0x560371afc1f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fc2740ca330 tx=0x7fc274009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f37c95dc22b6b17c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:34.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.996+0000 7fc26be00700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:26:34.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.997+0000 7fc26be00700 1 -- 172.21.15.1:0/642014003 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc27409c040 con 0x560371a41d60 2024-02-02T15:26:34.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.997+0000 7fc26be00700 1 -- 172.21.15.1:0/642014003 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fc27409a120 con 0x560371a41d60 2024-02-02T15:26:34.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.997+0000 7fc29cd2fb80 1 -- 172.21.15.1:0/642014003 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc27c01f070 con 0x560371a41d60 2024-02-02T15:26:34.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.997+0000 7fc26be00700 1 -- 172.21.15.1:0/642014003 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc27409df50 con 0x560371a41d60 2024-02-02T15:26:34.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.997+0000 7fc29cd2fb80 10 client.15309 fetch_fsmap learned FSMap version 27 2024-02-02T15:26:34.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.997+0000 7fc29cd2fb80 1 -- 172.21.15.1:0/642014003 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7fc26c021040 con 0x560371a41d60 2024-02-02T15:26:34.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.997+0000 7fc26be00700 1 -- 172.21.15.1:0/642014003 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc27c01f070 con 0x560371a41d60 2024-02-02T15:26:34.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.997+0000 7fc26be00700 1 -- 172.21.15.1:0/642014003 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc27409a380 con 0x560371a41d60 2024-02-02T15:26:34.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.997+0000 7fc29cd2fb80 10 client.15309 fetch_fsmap finished waiting for FSMap version 27 2024-02-02T15:26:34.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.997+0000 7fc29cd2fb80 10 client.15309 resolve_mds: resolved name 'f' to daemon mds.f 2024-02-02T15:26:34.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.998+0000 7fc29cd2fb80 20 client.15309 populate_metadata read hostname 'smithi001' 2024-02-02T15:26:34.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.998+0000 7fc29cd2fb80 1 --2- 172.21.15.1:0/642014003 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x560371ae4930 0x560371b4b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:34.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.998+0000 7fc29cd2fb80 4 client.15309 mds_command: new command op to 24502 tid=0[{"prefix": "status"}] 2024-02-02T15:26:34.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.998+0000 7fc29cd2fb80 1 -- 172.21.15.1:0/642014003 --> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5603719f2f00 con 0x560371ae4930 2024-02-02T15:26:34.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.998+0000 7fc281400700 1 --2- 172.21.15.1:0/642014003 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x560371ae4930 0x560371b4b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:34.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.998+0000 7fc281400700 1 --2- 172.21.15.1:0/642014003 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x560371ae4930 0x560371b4b580 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b0bee3b4c5b633cf server_cookie=b617fd8469acddc6 in_seq=0 out_seq=0 2024-02-02T15:26:34.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.998+0000 7fc26be00700 10 client.15309 ms_handle_connect on v2:172.21.15.142:6834/1052606547 2024-02-02T15:26:34.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.999+0000 7fc26be00700 1 -- 172.21.15.1:0/642014003 <== mds.4 v2:172.21.15.142:6834/1052606547 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+321 (crc 0 0 0) 0x5603719f2f00 con 0x560371ae4930 2024-02-02T15:26:34.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.999+0000 7fc26be00700 10 client.15309 handle_command_reply: tid=0 2024-02-02T15:26:34.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.999+0000 7fc29cd2fb80 1 client.15309 shutdown 2024-02-02T15:26:34.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.999+0000 7fc29cd2fb80 1 -- 172.21.15.1:0/642014003 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560371a41d60 msgr2=0x560371afc1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:34.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.999+0000 7fc29cd2fb80 1 --2- 172.21.15.1:0/642014003 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560371a41d60 0x560371afc1f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fc2740ca330 tx=0x7fc274009c00 comp rx=0 tx=0).stop 2024-02-02T15:26:34.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.999+0000 7fc29cd2fb80 1 -- 172.21.15.1:0/642014003 shutdown_connections 2024-02-02T15:26:34.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.999+0000 7fc29cd2fb80 1 --2- 172.21.15.1:0/642014003 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x560371ae4930 0x560371b4b580 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.999+0000 7fc29cd2fb80 1 --2- 172.21.15.1:0/642014003 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560371ae3a90 0x560371b49d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.999+0000 7fc29cd2fb80 1 --2- 172.21.15.1:0/642014003 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560371a41d60 0x560371afc1f0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.999+0000 7fc26be00700 0 client.15309 ms_handle_reset on v2:172.21.15.142:6834/1052606547 2024-02-02T15:26:34.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.999+0000 7fc29cd2fb80 1 --2- 172.21.15.1:0/642014003 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560371ae3130 0x560371afc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.999+0000 7fc29cd2fb80 1 -- 172.21.15.1:0/642014003 >> 172.21.15.1:0/642014003 conn(0x560371ade1c0 msgr2=0x560371af4d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:34.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.999+0000 7fc29cd2fb80 1 -- 172.21.15.1:0/642014003 shutdown_connections 2024-02-02T15:26:34.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.999+0000 7fc29cd2fb80 1 -- 172.21.15.1:0/642014003 wait complete. 2024-02-02T15:26:34.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:33.999+0000 7fc29cd2fb80 20 client.15309 trim_cache size 0 max 16384 2024-02-02T15:26:34.003 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:26:34.003 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "371a409c-c1de-11ee-95b6-87774f69a715", 2024-02-02T15:26:34.003 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 4, 2024-02-02T15:26:34.004 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 24502, 2024-02-02T15:26:34.004 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2024-02-02T15:26:34.004 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2024-02-02T15:26:34.004 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2024-02-02T15:26:34.004 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 74.768176556, 2024-02-02T15:26:34.004 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 27, 2024-02-02T15:26:34.004 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 88, 2024-02-02T15:26:34.004 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 84, 2024-02-02T15:26:34.004 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 124.941172018 2024-02-02T15:26:34.004 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:26:34.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.000+0000 7fc289200700 1 -- 172.21.15.1:0/4065558828 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fc2640a05a0 msgr2=0x7fc2640a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:34.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.000+0000 7fc289200700 1 --2- 172.21.15.1:0/4065558828 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fc2640a05a0 0x7fc2640a2a60 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc27c009fd0 tx=0x7fc27c010000 comp rx=0 tx=0).stop 2024-02-02T15:26:34.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.000+0000 7fc289200700 1 -- 172.21.15.1:0/4065558828 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc284073740 msgr2=0x7fc284198d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:34.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.000+0000 7fc289200700 1 --2- 172.21.15.1:0/4065558828 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc284073740 0x7fc284198d30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc274009230 tx=0x7fc27400b790 comp rx=0 tx=0).stop 2024-02-02T15:26:34.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.000+0000 7fc289200700 1 -- 172.21.15.1:0/4065558828 shutdown_connections 2024-02-02T15:26:34.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.000+0000 7fc289200700 1 --2- 172.21.15.1:0/4065558828 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc284108ce0 0x7fc2841a02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.000+0000 7fc289200700 1 --2- 172.21.15.1:0/4065558828 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fc2640a05a0 0x7fc2640a2a60 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.000+0000 7fc289200700 1 --2- 172.21.15.1:0/4065558828 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc284073740 0x7fc284198d30 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.000+0000 7fc289200700 1 --2- 172.21.15.1:0/4065558828 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc284074d20 0x7fc284199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.000+0000 7fc289200700 1 -- 172.21.15.1:0/4065558828 >> 172.21.15.1:0/4065558828 conn(0x7fc2840fc3a0 msgr2=0x7fc2840fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:34.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.000+0000 7fc289200700 1 -- 172.21.15.1:0/4065558828 shutdown_connections 2024-02-02T15:26:34.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.000+0000 7fc289200700 1 -- 172.21.15.1:0/4065558828 wait complete. 2024-02-02T15:26:34.019 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-02-02T15:26:34.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.217+0000 7f6084400700 1 -- 172.21.15.1:0/3557449009 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f607c101ff0 msgr2=0x7f607c102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:34.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.217+0000 7f6084400700 1 --2- 172.21.15.1:0/3557449009 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f607c101ff0 0x7f607c102450 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f60640099c0 tx=0x7f6064009cd0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.218+0000 7f6084400700 1 -- 172.21.15.1:0/3557449009 shutdown_connections 2024-02-02T15:26:34.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.218+0000 7f6084400700 1 --2- 172.21.15.1:0/3557449009 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f607c102990 0x7f607c109240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.218+0000 7f6084400700 1 --2- 172.21.15.1:0/3557449009 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f607c100e00 0x7f607c101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.218+0000 7f6084400700 1 --2- 172.21.15.1:0/3557449009 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f607c101ff0 0x7f607c102450 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.218+0000 7f6084400700 1 -- 172.21.15.1:0/3557449009 >> 172.21.15.1:0/3557449009 conn(0x7f607c0fc370 msgr2=0x7f607c0fe7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:34.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.218+0000 7f6084400700 1 -- 172.21.15.1:0/3557449009 shutdown_connections 2024-02-02T15:26:34.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.218+0000 7f6084400700 1 -- 172.21.15.1:0/3557449009 wait complete. 2024-02-02T15:26:34.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.219+0000 7f6084400700 1 Processor -- start 2024-02-02T15:26:34.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.219+0000 7f6084400700 1 -- start start 2024-02-02T15:26:34.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.219+0000 7f6084400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f607c100e00 0x7f607c074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:34.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.219+0000 7f6084400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f607c101ff0 0x7f607c073130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:34.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.219+0000 7f6084400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f607c102990 0x7f607c073670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:34.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.219+0000 7f6084400700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f607c073e10 con 0x7f607c102990 2024-02-02T15:26:34.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.219+0000 7f6084400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f607c073f80 con 0x7f607c101ff0 2024-02-02T15:26:34.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.219+0000 7f6084400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f607c1a0b10 con 0x7f607c100e00 2024-02-02T15:26:34.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.219+0000 7f6081000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f607c101ff0 0x7f607c073130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:34.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.219+0000 7f6081000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f607c101ff0 0x7f607c073130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:57650/0 (socket says 172.21.15.1:57650) 2024-02-02T15:26:34.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.219+0000 7f6081000700 1 -- 172.21.15.1:0/652334293 learned_addr learned my addr 172.21.15.1:0/652334293 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:34.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.220+0000 7f6082400700 1 --2- 172.21.15.1:0/652334293 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f607c102990 0x7f607c073670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:34.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.220+0000 7f6081a00700 1 --2- 172.21.15.1:0/652334293 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f607c100e00 0x7f607c074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:34.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.220+0000 7f6081000700 1 -- 172.21.15.1:0/652334293 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f607c100e00 msgr2=0x7f607c074af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:34.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.220+0000 7f6081000700 1 --2- 172.21.15.1:0/652334293 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f607c100e00 0x7f607c074af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.220+0000 7f6081000700 1 -- 172.21.15.1:0/652334293 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f607c102990 msgr2=0x7f607c073670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:34.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.220+0000 7f6081000700 1 --2- 172.21.15.1:0/652334293 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f607c102990 0x7f607c073670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.220+0000 7f6081000700 1 -- 172.21.15.1:0/652334293 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60640095f0 con 0x7f607c101ff0 2024-02-02T15:26:34.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.220+0000 7f6081000700 1 --2- 172.21.15.1:0/652334293 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f607c101ff0 0x7f607c073130 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6064009fd0 tx=0x7f6064009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:34.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.220+0000 7f607a000700 1 -- 172.21.15.1:0/652334293 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606401d070 con 0x7f607c101ff0 2024-02-02T15:26:34.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.220+0000 7f607a000700 1 -- 172.21.15.1:0/652334293 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6064005030 con 0x7f607c101ff0 2024-02-02T15:26:34.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.220+0000 7f607a000700 1 -- 172.21.15.1:0/652334293 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6064004a90 con 0x7f607c101ff0 2024-02-02T15:26:34.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.220+0000 7f6084400700 1 -- 172.21.15.1:0/652334293 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f607c1a0d90 con 0x7f607c101ff0 2024-02-02T15:26:34.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.221+0000 7f6084400700 1 -- 172.21.15.1:0/652334293 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f607c1a1160 con 0x7f607c101ff0 2024-02-02T15:26:34.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.225+0000 7f607a000700 1 -- 172.21.15.1:0/652334293 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 135069+0+0 (secure 0 0 0) 0x7f60640051a0 con 0x7f607c101ff0 2024-02-02T15:26:34.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.226+0000 7f6097f03b80 1 Processor -- start 2024-02-02T15:26:34.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.226+0000 7f6097f03b80 1 -- start start 2024-02-02T15:26:34.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.226+0000 7f6097f03b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e2c0b0130 0x564e2c0b0550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:34.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.226+0000 7f6097f03b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564e2c0b0a90 0x564e2c0b13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:34.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.226+0000 7f6097f03b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e2c0b1930 0x564e2c0b1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:34.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.226+0000 7f6097f03b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e2c023470 con 0x564e2c0b0130 2024-02-02T15:26:34.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.226+0000 7f6097f03b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e2c007a50 con 0x564e2c0b0a90 2024-02-02T15:26:34.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.226+0000 7f6097f03b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e2c005410 con 0x564e2c0b1930 2024-02-02T15:26:34.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.227+0000 7f6082400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e2c0b0130 0x564e2c0b0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:34.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.227+0000 7f6081a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564e2c0b0a90 0x564e2c0b13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:34.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.227+0000 7f6081a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564e2c0b0a90 0x564e2c0b13a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:57656/0 (socket says 172.21.15.1:57656) 2024-02-02T15:26:34.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.227+0000 7f6081a00700 1 -- 172.21.15.1:0/1765362083 learned_addr learned my addr 172.21.15.1:0/1765362083 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:34.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.227+0000 7f6081000700 1 --2- 172.21.15.1:0/1765362083 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e2c0b1930 0x564e2c0b1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:34.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.227+0000 7f6081a00700 1 -- 172.21.15.1:0/1765362083 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e2c0b1930 msgr2=0x564e2c0b1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:34.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.227+0000 7f6081a00700 1 --2- 172.21.15.1:0/1765362083 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e2c0b1930 0x564e2c0b1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.227+0000 7f6081a00700 1 -- 172.21.15.1:0/1765362083 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e2c0b0130 msgr2=0x564e2c0b0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:34.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.227+0000 7f6081a00700 1 --2- 172.21.15.1:0/1765362083 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e2c0b0130 0x564e2c0b0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.227+0000 7f6081a00700 1 -- 172.21.15.1:0/1765362083 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e2be82360 con 0x564e2c0b0a90 2024-02-02T15:26:34.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.227+0000 7f607a000700 1 --2- 172.21.15.1:0/652334293 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f605c0a0520 0x7f605c0a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:34.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.227+0000 7f6081a00700 1 --2- 172.21.15.1:0/1765362083 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564e2c0b0a90 0x564e2c0b13a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f606c009d30 tx=0x7f606c011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4493aa3fe2cf01cb server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:34.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.227+0000 7f607a000700 1 -- 172.21.15.1:0/652334293 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f60640ccb40 con 0x7f607c101ff0 2024-02-02T15:26:34.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.227+0000 7f6082400700 1 --2- 172.21.15.1:0/652334293 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f605c0a0520 0x7f605c0a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:34.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.227+0000 7f6072a00700 1 -- 172.21.15.1:0/1765362083 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606c004220 con 0x564e2c0b0a90 2024-02-02T15:26:34.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.228+0000 7f6072a00700 1 -- 172.21.15.1:0/1765362083 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f606c004380 con 0x564e2c0b0a90 2024-02-02T15:26:34.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.228+0000 7f6072a00700 1 -- 172.21.15.1:0/1765362083 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606c0198d0 con 0x564e2c0b0a90 2024-02-02T15:26:34.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.228+0000 7f6097f03b80 1 -- 172.21.15.1:0/1765362083 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564e2c0b0a90 msgr2=0x564e2c0b13a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:34.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.228+0000 7f6097f03b80 1 --2- 172.21.15.1:0/1765362083 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564e2c0b0a90 0x564e2c0b13a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f606c009d30 tx=0x7f606c011720 comp rx=0 tx=0).stop 2024-02-02T15:26:34.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.228+0000 7f6082400700 1 --2- 172.21.15.1:0/652334293 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f605c0a0520 0x7f605c0a29e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f607c075150 tx=0x7f607400b040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:34.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.228+0000 7f6097f03b80 1 -- 172.21.15.1:0/1765362083 shutdown_connections 2024-02-02T15:26:34.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.228+0000 7f6097f03b80 1 --2- 172.21.15.1:0/1765362083 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e2c0b1930 0x564e2c0b1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.228+0000 7f6097f03b80 1 --2- 172.21.15.1:0/1765362083 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e2c0b0130 0x564e2c0b0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.228+0000 7f6097f03b80 1 --2- 172.21.15.1:0/1765362083 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564e2c0b0a90 0x564e2c0b13a0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.228+0000 7f6097f03b80 1 -- 172.21.15.1:0/1765362083 >> 172.21.15.1:0/1765362083 conn(0x564e2c00ed60 msgr2=0x564e2c0aae90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:34.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.228+0000 7f6097f03b80 1 -- 172.21.15.1:0/1765362083 shutdown_connections 2024-02-02T15:26:34.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.228+0000 7f6097f03b80 1 -- 172.21.15.1:0/1765362083 wait complete. 2024-02-02T15:26:34.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.229+0000 7f6097f03b80 1 -- start start 2024-02-02T15:26:34.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.229+0000 7f6097f03b80 1 Processor -- start 2024-02-02T15:26:34.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.229+0000 7f6097f03b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564e2c0b0130 0x564e2c0b9f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:34.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.230+0000 7f6097f03b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e2c0b0a90 0x564e2c0ba460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:34.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.230+0000 7f6097f03b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e2c0b1930 0x564e2c0c14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:34.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.230+0000 7f6097f03b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e2c000d00 con 0x564e2c0b1930 2024-02-02T15:26:34.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.230+0000 7f6097f03b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e2bf9cd30 con 0x564e2c0b0130 2024-02-02T15:26:34.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.230+0000 7f6097f03b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e2bf9c9d0 con 0x564e2c0b0a90 2024-02-02T15:26:34.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.230+0000 7f6097f03b80 10 client.0 _ll_register_callbacks cb 0x564e2bf9e240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:26:34.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.230+0000 7f6082400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e2c0b1930 0x564e2c0c14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:34.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.230+0000 7f6081000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e2c0b0a90 0x564e2c0ba460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:34.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.230+0000 7f6082400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e2c0b1930 0x564e2c0c14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52764/0 (socket says 172.21.15.1:52764) 2024-02-02T15:26:34.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.230+0000 7f6081a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564e2c0b0130 0x564e2c0b9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:34.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.230+0000 7f6082400700 1 -- 172.21.15.1:0/10521271 learned_addr learned my addr 172.21.15.1:0/10521271 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:34.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.230+0000 7f6081000700 1 -- 172.21.15.1:0/10521271 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564e2c0b0130 msgr2=0x564e2c0b9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:34.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.230+0000 7f6081000700 1 --2- 172.21.15.1:0/10521271 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564e2c0b0130 0x564e2c0b9f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.230+0000 7f6081000700 1 -- 172.21.15.1:0/10521271 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e2c0b1930 msgr2=0x564e2c0c14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:34.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.230+0000 7f6081000700 1 --2- 172.21.15.1:0/10521271 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e2c0b1930 0x564e2c0c14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.230+0000 7f6081000700 1 -- 172.21.15.1:0/10521271 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f606c009960 con 0x564e2c0b0a90 2024-02-02T15:26:34.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.230+0000 7f6081a00700 1 --2- 172.21.15.1:0/10521271 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564e2c0b0130 0x564e2c0b9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:26:34.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.231+0000 7f6081000700 1 --2- 172.21.15.1:0/10521271 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e2c0b0a90 0x564e2c0ba460 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f60640ca270 tx=0x7f6064099ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b31688084c450f7 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:34.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.231+0000 7f606b400700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:26:34.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.231+0000 7f606b400700 1 -- 172.21.15.1:0/10521271 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f606409a180 con 0x564e2c0b0a90 2024-02-02T15:26:34.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.231+0000 7f606b400700 1 -- 172.21.15.1:0/10521271 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f606409c070 con 0x564e2c0b0a90 2024-02-02T15:26:34.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.231+0000 7f6097f03b80 1 -- 172.21.15.1:0/10521271 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564e2c0a30a0 con 0x564e2c0b0a90 2024-02-02T15:26:34.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.231+0000 7f606b400700 1 -- 172.21.15.1:0/10521271 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f606409dfb0 con 0x564e2c0b0a90 2024-02-02T15:26:34.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.231+0000 7f6097f03b80 10 client.25357 fetch_fsmap learned FSMap version 27 2024-02-02T15:26:34.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.231+0000 7f6097f03b80 1 -- 172.21.15.1:0/10521271 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x564e2be2eb10 con 0x564e2c0b0a90 2024-02-02T15:26:34.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.231+0000 7f606b400700 1 -- 172.21.15.1:0/10521271 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f60640b6020 con 0x564e2c0b0a90 2024-02-02T15:26:34.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.231+0000 7f606b400700 1 -- 172.21.15.1:0/10521271 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6064099670 con 0x564e2c0b0a90 2024-02-02T15:26:34.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.231+0000 7f6097f03b80 10 client.25357 fetch_fsmap finished waiting for FSMap version 27 2024-02-02T15:26:34.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.232+0000 7f6097f03b80 10 client.25357 resolve_mds: resolved name 'c' to daemon mds.c 2024-02-02T15:26:34.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.232+0000 7f6097f03b80 20 client.25357 populate_metadata read hostname 'smithi001' 2024-02-02T15:26:34.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.232+0000 7f6097f03b80 1 --2- 172.21.15.1:0/10521271 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x564e2c0c4e50 0x564e2c0c43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:34.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.232+0000 7f6097f03b80 4 client.25357 mds_command: new command op to 24514 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:26:34.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.232+0000 7f6097f03b80 1 -- 172.21.15.1:0/10521271 --> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564e2bff9d80 con 0x564e2c0c4e50 2024-02-02T15:26:34.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.232+0000 7f6081a00700 1 --2- 172.21.15.1:0/10521271 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x564e2c0c4e50 0x564e2c0c43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:34.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.232+0000 7f6081a00700 1 --2- 172.21.15.1:0/10521271 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x564e2c0c4e50 0x564e2c0c43d0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b7a4a664d6c091c2 server_cookie=86d099bb36f976e2 in_seq=0 out_seq=0 2024-02-02T15:26:34.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.232+0000 7f606b400700 10 client.25357 ms_handle_connect on v2:172.21.15.142:6836/2870289019 2024-02-02T15:26:34.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.233+0000 7f606b400700 1 -- 172.21.15.1:0/10521271 <== mds.3 v2:172.21.15.142:6836/2870289019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x564e2bff9d80 con 0x564e2c0c4e50 2024-02-02T15:26:34.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.233+0000 7f606b400700 10 client.25357 handle_command_reply: tid=0 2024-02-02T15:26:34.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.233+0000 7f6097f03b80 1 client.25357 shutdown 2024-02-02T15:26:34.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.233+0000 7f6097f03b80 1 -- 172.21.15.1:0/10521271 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e2c0b0a90 msgr2=0x564e2c0ba460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:34.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.233+0000 7f6097f03b80 1 --2- 172.21.15.1:0/10521271 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e2c0b0a90 0x564e2c0ba460 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f60640ca270 tx=0x7f6064099ae0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.233+0000 7f6097f03b80 1 -- 172.21.15.1:0/10521271 shutdown_connections 2024-02-02T15:26:34.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.233+0000 7f6097f03b80 1 --2- 172.21.15.1:0/10521271 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x564e2c0c4e50 0x564e2c0c43d0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.233+0000 7f6097f03b80 1 --2- 172.21.15.1:0/10521271 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e2c0b1930 0x564e2c0c14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.233+0000 7f6097f03b80 1 --2- 172.21.15.1:0/10521271 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564e2c0b0130 0x564e2c0b9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.233+0000 7f6097f03b80 1 --2- 172.21.15.1:0/10521271 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e2c0b0a90 0x564e2c0ba460 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.233+0000 7f606b400700 0 client.25357 ms_handle_reset on v2:172.21.15.142:6836/2870289019 2024-02-02T15:26:34.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.233+0000 7f6097f03b80 1 -- 172.21.15.1:0/10521271 >> 172.21.15.1:0/10521271 conn(0x564e2c00ed60 msgr2=0x564e2c0aa570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:34.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.234+0000 7f6097f03b80 1 -- 172.21.15.1:0/10521271 shutdown_connections 2024-02-02T15:26:34.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.234+0000 7f6097f03b80 1 -- 172.21.15.1:0/10521271 wait complete. 2024-02-02T15:26:34.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.234+0000 7f6097f03b80 20 client.25357 trim_cache size 0 max 16384 2024-02-02T15:26:34.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.252+0000 7f6097f03b80 1 Processor -- start 2024-02-02T15:26:34.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.252+0000 7f6097f03b80 1 -- start start 2024-02-02T15:26:34.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.252+0000 7f6097f03b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e2c0b0130 0x564e2c0b7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:34.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.252+0000 7f6097f03b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564e2c0b0a90 0x564e2c0cdbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:34.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.252+0000 7f6097f03b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e2c0b1930 0x564e2c0b48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:34.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.252+0000 7f6097f03b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e2be84650 con 0x564e2c0b1930 2024-02-02T15:26:34.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.252+0000 7f6097f03b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e2c0b21b0 con 0x564e2c0b0a90 2024-02-02T15:26:34.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.252+0000 7f6097f03b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e2c0a5500 con 0x564e2c0b0130 2024-02-02T15:26:34.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.252+0000 7f6082400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e2c0b1930 0x564e2c0b48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:34.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.252+0000 7f6082400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e2c0b1930 0x564e2c0b48a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52778/0 (socket says 172.21.15.1:52778) 2024-02-02T15:26:34.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.252+0000 7f6082400700 1 -- 172.21.15.1:0/2766241008 learned_addr learned my addr 172.21.15.1:0/2766241008 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:34.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.253+0000 7f6081a00700 1 --2- 172.21.15.1:0/2766241008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e2c0b0130 0x564e2c0b7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:34.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.253+0000 7f6081000700 1 --2- 172.21.15.1:0/2766241008 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564e2c0b0a90 0x564e2c0cdbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:34.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.253+0000 7f6082400700 1 -- 172.21.15.1:0/2766241008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e2c0b0130 msgr2=0x564e2c0b7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:34.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.253+0000 7f6082400700 1 --2- 172.21.15.1:0/2766241008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e2c0b0130 0x564e2c0b7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.253+0000 7f6082400700 1 -- 172.21.15.1:0/2766241008 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564e2c0b0a90 msgr2=0x564e2c0cdbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:34.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.253+0000 7f6082400700 1 --2- 172.21.15.1:0/2766241008 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564e2c0b0a90 0x564e2c0cdbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.253+0000 7f6082400700 1 -- 172.21.15.1:0/2766241008 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e2bfc6e50 con 0x564e2c0b1930 2024-02-02T15:26:34.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.253+0000 7f6082400700 1 --2- 172.21.15.1:0/2766241008 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e2c0b1930 0x564e2c0b48a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f6074003f90 tx=0x7f6074017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b1e59235e65596b server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:34.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.253+0000 7f6073400700 1 -- 172.21.15.1:0/2766241008 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f607400e040 con 0x564e2c0b1930 2024-02-02T15:26:34.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.253+0000 7f6073400700 1 -- 172.21.15.1:0/2766241008 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f607400f040 con 0x564e2c0b1930 2024-02-02T15:26:34.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.254+0000 7f6073400700 1 -- 172.21.15.1:0/2766241008 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6074024990 con 0x564e2c0b1930 2024-02-02T15:26:34.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.254+0000 7f6097f03b80 1 -- 172.21.15.1:0/2766241008 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e2c0b1930 msgr2=0x564e2c0b48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:34.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.254+0000 7f6097f03b80 1 --2- 172.21.15.1:0/2766241008 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e2c0b1930 0x564e2c0b48a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f6074003f90 tx=0x7f6074017000 comp rx=0 tx=0).stop 2024-02-02T15:26:34.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.254+0000 7f6097f03b80 1 -- 172.21.15.1:0/2766241008 shutdown_connections 2024-02-02T15:26:34.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.254+0000 7f6097f03b80 1 --2- 172.21.15.1:0/2766241008 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e2c0b1930 0x564e2c0b48a0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.254+0000 7f6097f03b80 1 --2- 172.21.15.1:0/2766241008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e2c0b0130 0x564e2c0b7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.254+0000 7f6097f03b80 1 --2- 172.21.15.1:0/2766241008 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564e2c0b0a90 0x564e2c0cdbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.254+0000 7f6097f03b80 1 -- 172.21.15.1:0/2766241008 >> 172.21.15.1:0/2766241008 conn(0x564e2c00ed60 msgr2=0x564e2c0c43d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:34.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.254+0000 7f6097f03b80 1 -- 172.21.15.1:0/2766241008 shutdown_connections 2024-02-02T15:26:34.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.254+0000 7f6097f03b80 1 -- 172.21.15.1:0/2766241008 wait complete. 2024-02-02T15:26:34.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.255+0000 7f6097f03b80 1 -- start start 2024-02-02T15:26:34.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.255+0000 7f6097f03b80 1 Processor -- start 2024-02-02T15:26:34.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.255+0000 7f6097f03b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564e2c0b0130 0x564e2c0c8890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:34.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.255+0000 7f6097f03b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e2c0b0a90 0x564e2c0c8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:34.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.255+0000 7f6097f03b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e2c0b1930 0x564e2c116940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:34.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.255+0000 7f6097f03b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e2c0cde00 con 0x564e2c0b0a90 2024-02-02T15:26:34.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.255+0000 7f6097f03b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e2c0cdf70 con 0x564e2c0b0130 2024-02-02T15:26:34.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.255+0000 7f6097f03b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e2c116e80 con 0x564e2c0b1930 2024-02-02T15:26:34.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.255+0000 7f6097f03b80 10 client.0 _ll_register_callbacks cb 0x564e2bf9e240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:26:34.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.255+0000 7f6081000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e2c0b0a90 0x564e2c0c8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:34.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.255+0000 7f6081a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564e2c0b0130 0x564e2c0c8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:34.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.255+0000 7f6081000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e2c0b0a90 0x564e2c0c8dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52794/0 (socket says 172.21.15.1:52794) 2024-02-02T15:26:34.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.255+0000 7f6081000700 1 -- 172.21.15.1:0/1399753199 learned_addr learned my addr 172.21.15.1:0/1399753199 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:34.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.256+0000 7f6082400700 1 --2- 172.21.15.1:0/1399753199 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e2c0b1930 0x564e2c116940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:34.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.256+0000 7f6081000700 1 -- 172.21.15.1:0/1399753199 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e2c0b1930 msgr2=0x564e2c116940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:34.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.256+0000 7f6081000700 1 --2- 172.21.15.1:0/1399753199 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e2c0b1930 0x564e2c116940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.256+0000 7f6081000700 1 -- 172.21.15.1:0/1399753199 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564e2c0b0130 msgr2=0x564e2c0c8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:34.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.256+0000 7f6081000700 1 --2- 172.21.15.1:0/1399753199 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564e2c0b0130 0x564e2c0c8890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.256+0000 7f6081000700 1 -- 172.21.15.1:0/1399753199 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6074003c70 con 0x564e2c0b0a90 2024-02-02T15:26:34.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.256+0000 7f6081a00700 1 --2- 172.21.15.1:0/1399753199 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564e2c0b0130 0x564e2c0c8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:26:34.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.256+0000 7f6081000700 1 --2- 172.21.15.1:0/1399753199 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e2c0b0a90 0x564e2c0c8dd0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f6064000c00 tx=0x7f606400b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bf83bf5bc70a5dd server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:34.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.256+0000 7f606b400700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:26:34.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.256+0000 7f606b400700 1 -- 172.21.15.1:0/1399753199 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6064004df0 con 0x564e2c0b0a90 2024-02-02T15:26:34.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.256+0000 7f606b400700 1 -- 172.21.15.1:0/1399753199 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f606409c040 con 0x564e2c0b0a90 2024-02-02T15:26:34.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.256+0000 7f6097f03b80 1 -- 172.21.15.1:0/1399753199 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f606c004380 con 0x564e2c0b0a90 2024-02-02T15:26:34.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.257+0000 7f606b400700 1 -- 172.21.15.1:0/1399753199 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f60640ccd80 con 0x564e2c0b0a90 2024-02-02T15:26:34.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.257+0000 7f6097f03b80 10 client.15327 fetch_fsmap learned FSMap version 27 2024-02-02T15:26:34.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.257+0000 7f6097f03b80 1 -- 172.21.15.1:0/1399753199 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x564e2be86c80 con 0x564e2c0b0a90 2024-02-02T15:26:34.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.257+0000 7f606b400700 1 -- 172.21.15.1:0/1399753199 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f606c004380 con 0x564e2c0b0a90 2024-02-02T15:26:34.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.257+0000 7f606b400700 1 -- 172.21.15.1:0/1399753199 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f60640adbb0 con 0x564e2c0b0a90 2024-02-02T15:26:34.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.257+0000 7f6097f03b80 10 client.15327 fetch_fsmap finished waiting for FSMap version 27 2024-02-02T15:26:34.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.257+0000 7f6097f03b80 10 client.15327 resolve_mds: resolved name 'c' to daemon mds.c 2024-02-02T15:26:34.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.257+0000 7f6097f03b80 20 client.15327 populate_metadata read hostname 'smithi001' 2024-02-02T15:26:34.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.257+0000 7f6097f03b80 1 --2- 172.21.15.1:0/1399753199 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x564e2c11b660 0x564e2c118060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:34.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.257+0000 7f6097f03b80 4 client.15327 mds_command: new command op to 24514 tid=0[{"prefix": "status"}] 2024-02-02T15:26:34.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.257+0000 7f6097f03b80 1 -- 172.21.15.1:0/1399753199 --> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] -- command(tid 0: {"prefix": "status"}) v1 -- 0x564e2bfbff00 con 0x564e2c11b660 2024-02-02T15:26:34.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.257+0000 7f6081a00700 1 --2- 172.21.15.1:0/1399753199 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x564e2c11b660 0x564e2c118060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:34.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.258+0000 7f6081a00700 1 --2- 172.21.15.1:0/1399753199 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x564e2c11b660 0x564e2c118060 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b904994155c39c68 server_cookie=eb29b80bd340d92d in_seq=0 out_seq=0 2024-02-02T15:26:34.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.258+0000 7f606b400700 10 client.15327 ms_handle_connect on v2:172.21.15.142:6836/2870289019 2024-02-02T15:26:34.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.258+0000 7f606b400700 1 -- 172.21.15.1:0/1399753199 <== mds.3 v2:172.21.15.142:6836/2870289019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x564e2bfbff00 con 0x564e2c11b660 2024-02-02T15:26:34.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.258+0000 7f606b400700 10 client.15327 handle_command_reply: tid=0 2024-02-02T15:26:34.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.258+0000 7f6097f03b80 1 client.15327 shutdown 2024-02-02T15:26:34.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.258+0000 7f6097f03b80 1 -- 172.21.15.1:0/1399753199 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e2c0b0a90 msgr2=0x564e2c0c8dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:34.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.258+0000 7f6097f03b80 1 --2- 172.21.15.1:0/1399753199 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e2c0b0a90 0x564e2c0c8dd0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f6064000c00 tx=0x7f606400b5b0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.258+0000 7f6097f03b80 1 -- 172.21.15.1:0/1399753199 shutdown_connections 2024-02-02T15:26:34.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.258+0000 7f6097f03b80 1 --2- 172.21.15.1:0/1399753199 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564e2c0b1930 0x564e2c116940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.258+0000 7f6097f03b80 1 --2- 172.21.15.1:0/1399753199 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564e2c0b0130 0x564e2c0c8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.258+0000 7f6097f03b80 1 --2- 172.21.15.1:0/1399753199 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x564e2c11b660 0x564e2c118060 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.258+0000 7f6097f03b80 1 --2- 172.21.15.1:0/1399753199 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564e2c0b0a90 0x564e2c0c8dd0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.258+0000 7f6097f03b80 1 -- 172.21.15.1:0/1399753199 >> 172.21.15.1:0/1399753199 conn(0x564e2c00ed60 msgr2=0x564e2c0b7d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:34.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.258+0000 7f606b400700 0 client.15327 ms_handle_reset on v2:172.21.15.142:6836/2870289019 2024-02-02T15:26:34.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.258+0000 7f6097f03b80 1 -- 172.21.15.1:0/1399753199 shutdown_connections 2024-02-02T15:26:34.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.258+0000 7f6097f03b80 1 -- 172.21.15.1:0/1399753199 wait complete. 2024-02-02T15:26:34.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.259+0000 7f6097f03b80 20 client.15327 trim_cache size 0 max 16384 2024-02-02T15:26:34.263 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:26:34.263 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "371a409c-c1de-11ee-95b6-87774f69a715", 2024-02-02T15:26:34.263 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 3, 2024-02-02T15:26:34.263 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 24514, 2024-02-02T15:26:34.263 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2024-02-02T15:26:34.263 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2024-02-02T15:26:34.263 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2024-02-02T15:26:34.263 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 76.034386878000007, 2024-02-02T15:26:34.263 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 27, 2024-02-02T15:26:34.263 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 88, 2024-02-02T15:26:34.263 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 84, 2024-02-02T15:26:34.263 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 123.367255868 2024-02-02T15:26:34.263 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:26:34.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.259+0000 7f6084400700 1 -- 172.21.15.1:0/652334293 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f605c0a0520 msgr2=0x7f605c0a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:34.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.259+0000 7f6084400700 1 --2- 172.21.15.1:0/652334293 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f605c0a0520 0x7f605c0a29e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f607c075150 tx=0x7f607400b040 comp rx=0 tx=0).stop 2024-02-02T15:26:34.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.259+0000 7f6084400700 1 -- 172.21.15.1:0/652334293 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f607c101ff0 msgr2=0x7f607c073130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:34.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.259+0000 7f6084400700 1 --2- 172.21.15.1:0/652334293 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f607c101ff0 0x7f607c073130 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6064009fd0 tx=0x7f6064009580 comp rx=0 tx=0).stop 2024-02-02T15:26:34.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.259+0000 7f6084400700 1 -- 172.21.15.1:0/652334293 shutdown_connections 2024-02-02T15:26:34.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.259+0000 7f6084400700 1 --2- 172.21.15.1:0/652334293 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f607c102990 0x7f607c073670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.259+0000 7f6084400700 1 --2- 172.21.15.1:0/652334293 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f605c0a0520 0x7f605c0a29e0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.259+0000 7f6084400700 1 --2- 172.21.15.1:0/652334293 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f607c100e00 0x7f607c074af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.259+0000 7f6084400700 1 --2- 172.21.15.1:0/652334293 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f607c101ff0 0x7f607c073130 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.259+0000 7f6084400700 1 -- 172.21.15.1:0/652334293 >> 172.21.15.1:0/652334293 conn(0x7f607c0fc370 msgr2=0x7f607c0fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:34.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.259+0000 7f6084400700 1 -- 172.21.15.1:0/652334293 shutdown_connections 2024-02-02T15:26:34.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.259+0000 7f6084400700 1 -- 172.21.15.1:0/652334293 wait complete. 2024-02-02T15:26:34.277 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-02-02T15:26:34.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.477+0000 7fa6bf600700 1 -- 172.21.15.1:0/1750873358 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa6b8102990 msgr2=0x7fa6b8109240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:34.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.477+0000 7fa6bf600700 1 --2- 172.21.15.1:0/1750873358 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa6b8102990 0x7fa6b8109240 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa6b000b0d0 tx=0x7fa6b000b3e0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.478+0000 7fa6bf600700 1 -- 172.21.15.1:0/1750873358 shutdown_connections 2024-02-02T15:26:34.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.478+0000 7fa6bf600700 1 --2- 172.21.15.1:0/1750873358 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa6b8102990 0x7fa6b8109240 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.478+0000 7fa6bf600700 1 --2- 172.21.15.1:0/1750873358 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6b8100e00 0x7fa6b8101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.478+0000 7fa6bf600700 1 --2- 172.21.15.1:0/1750873358 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6b8101ff0 0x7fa6b8102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.478+0000 7fa6bf600700 1 -- 172.21.15.1:0/1750873358 >> 172.21.15.1:0/1750873358 conn(0x7fa6b80fc370 msgr2=0x7fa6b80fe7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:34.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.478+0000 7fa6bf600700 1 -- 172.21.15.1:0/1750873358 shutdown_connections 2024-02-02T15:26:34.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.478+0000 7fa6bf600700 1 -- 172.21.15.1:0/1750873358 wait complete. 2024-02-02T15:26:34.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.479+0000 7fa6bf600700 1 Processor -- start 2024-02-02T15:26:34.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.479+0000 7fa6bf600700 1 -- start start 2024-02-02T15:26:34.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.479+0000 7fa6bf600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6b8100e00 0x7fa6b8198d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:34.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.479+0000 7fa6bf600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6b8101ff0 0x7fa6b81992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:34.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.479+0000 7fa6bf600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa6b8102990 0x7fa6b81a0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:34.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.479+0000 7fa6bf600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b81a08e0 con 0x7fa6b8102990 2024-02-02T15:26:34.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.479+0000 7fa6bf600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b81a0a20 con 0x7fa6b8100e00 2024-02-02T15:26:34.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.479+0000 7fa6bf600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b81a0bb0 con 0x7fa6b8101ff0 2024-02-02T15:26:34.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.480+0000 7fa6bcc00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6b8100e00 0x7fa6b8198d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:34.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.480+0000 7fa6bd600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa6b8102990 0x7fa6b81a0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:34.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.480+0000 7fa6bcc00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6b8100e00 0x7fa6b8198d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:57716/0 (socket says 172.21.15.1:57716) 2024-02-02T15:26:34.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.480+0000 7fa6bcc00700 1 -- 172.21.15.1:0/988293293 learned_addr learned my addr 172.21.15.1:0/988293293 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:34.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.480+0000 7fa6b7e00700 1 --2- 172.21.15.1:0/988293293 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6b8101ff0 0x7fa6b81992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:34.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.480+0000 7fa6bd600700 1 -- 172.21.15.1:0/988293293 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6b8101ff0 msgr2=0x7fa6b81992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:34.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.480+0000 7fa6bd600700 1 --2- 172.21.15.1:0/988293293 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6b8101ff0 0x7fa6b81992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.480+0000 7fa6bd600700 1 -- 172.21.15.1:0/988293293 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6b8100e00 msgr2=0x7fa6b8198d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:34.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.480+0000 7fa6bd600700 1 --2- 172.21.15.1:0/988293293 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6b8100e00 0x7fa6b8198d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.480+0000 7fa6bd600700 1 -- 172.21.15.1:0/988293293 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6b0009d00 con 0x7fa6b8102990 2024-02-02T15:26:34.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.480+0000 7fa6bcc00700 1 --2- 172.21.15.1:0/988293293 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6b8100e00 0x7fa6b8198d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:26:34.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.480+0000 7fa6bd600700 1 --2- 172.21.15.1:0/988293293 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa6b8102990 0x7fa6b81a0310 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa6b000b770 tx=0x7fa6b0008ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:34.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.481+0000 7fa6b5600700 1 -- 172.21.15.1:0/988293293 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b000e040 con 0x7fa6b8102990 2024-02-02T15:26:34.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.481+0000 7fa6b5600700 1 -- 172.21.15.1:0/988293293 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa6b001d930 con 0x7fa6b8102990 2024-02-02T15:26:34.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.481+0000 7fa6bf600700 1 -- 172.21.15.1:0/988293293 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6b81a0e30 con 0x7fa6b8102990 2024-02-02T15:26:34.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.481+0000 7fa6b5600700 1 -- 172.21.15.1:0/988293293 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b001b8f0 con 0x7fa6b8102990 2024-02-02T15:26:34.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.481+0000 7fa6bf600700 1 -- 172.21.15.1:0/988293293 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6b81a1200 con 0x7fa6b8102990 2024-02-02T15:26:34.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.484+0000 7fa6b5600700 1 -- 172.21.15.1:0/988293293 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) v1 ==== 135069+0+0 (secure 0 0 0) 0x7fa6b001daa0 con 0x7fa6b8102990 2024-02-02T15:26:34.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.485+0000 7fa6b5600700 1 --2- 172.21.15.1:0/988293293 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fa6980a0520 0x7fa6980a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:34.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.485+0000 7fa6d3198b80 1 Processor -- start 2024-02-02T15:26:34.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.485+0000 7fa6b5600700 1 -- 172.21.15.1:0/988293293 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fa6b00cd260 con 0x7fa6b8102990 2024-02-02T15:26:34.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.485+0000 7fa6d3198b80 1 -- start start 2024-02-02T15:26:34.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.485+0000 7fa6bcc00700 1 --2- 172.21.15.1:0/988293293 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fa6980a0520 0x7fa6980a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:34.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.486+0000 7fa6d3198b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e51fe1130 0x556e51fe1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:34.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.486+0000 7fa6d3198b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556e51fe1a90 0x556e51fe23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:34.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.486+0000 7fa6d3198b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556e51fe2930 0x556e51fe2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:34.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.486+0000 7fa6d3198b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e51f54470 con 0x556e51fe1a90 2024-02-02T15:26:34.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.486+0000 7fa6d3198b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e51f38a50 con 0x556e51fe2930 2024-02-02T15:26:34.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.486+0000 7fa6d3198b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e51f36410 con 0x556e51fe1130 2024-02-02T15:26:34.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.486+0000 7fa6b7e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556e51fe1a90 0x556e51fe23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:34.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.486+0000 7fa6bcc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e51fe1130 0x556e51fe1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:34.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.486+0000 7fa6b7e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556e51fe1a90 0x556e51fe23a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52810/0 (socket says 172.21.15.1:52810) 2024-02-02T15:26:34.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.486+0000 7fa6b7e00700 1 -- 172.21.15.1:0/3121553677 learned_addr learned my addr 172.21.15.1:0/3121553677 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:34.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.486+0000 7fa6bd600700 1 --2- 172.21.15.1:0/3121553677 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556e51fe2930 0x556e51fe2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:34.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.486+0000 7fa6bcc00700 1 --2- 172.21.15.1:0/988293293 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fa6980a0520 0x7fa6980a29e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa6a8005950 tx=0x7fa6a8009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:34.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.486+0000 7fa6b7e00700 1 -- 172.21.15.1:0/3121553677 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e51fe1130 msgr2=0x556e51fe1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:34.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.486+0000 7fa6b7e00700 1 --2- 172.21.15.1:0/3121553677 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e51fe1130 0x556e51fe1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.486+0000 7fa6b7e00700 1 -- 172.21.15.1:0/3121553677 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556e51fe2930 msgr2=0x556e51fe2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:34.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.486+0000 7fa6b7e00700 1 --2- 172.21.15.1:0/3121553677 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556e51fe2930 0x556e51fe2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.486+0000 7fa6b7e00700 1 -- 172.21.15.1:0/3121553677 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556e51db3360 con 0x556e51fe1a90 2024-02-02T15:26:34.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.486+0000 7fa6bd600700 1 --2- 172.21.15.1:0/3121553677 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556e51fe2930 0x556e51fe2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:26:34.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.487+0000 7fa6b7e00700 1 --2- 172.21.15.1:0/3121553677 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556e51fe1a90 0x556e51fe23a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fa6a0009d80 tx=0x7fa6a0011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b861995fbdb54012 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:34.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.487+0000 7fa6adc00700 1 -- 172.21.15.1:0/3121553677 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a0004230 con 0x556e51fe1a90 2024-02-02T15:26:34.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.487+0000 7fa6adc00700 1 -- 172.21.15.1:0/3121553677 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa6a0009e10 con 0x556e51fe1a90 2024-02-02T15:26:34.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.487+0000 7fa6adc00700 1 -- 172.21.15.1:0/3121553677 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a0019880 con 0x556e51fe1a90 2024-02-02T15:26:34.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.488+0000 7fa6d3198b80 1 -- 172.21.15.1:0/3121553677 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556e51fe1a90 msgr2=0x556e51fe23a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:34.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.488+0000 7fa6d3198b80 1 --2- 172.21.15.1:0/3121553677 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556e51fe1a90 0x556e51fe23a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fa6a0009d80 tx=0x7fa6a0011720 comp rx=0 tx=0).stop 2024-02-02T15:26:34.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.488+0000 7fa6d3198b80 1 -- 172.21.15.1:0/3121553677 shutdown_connections 2024-02-02T15:26:34.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.488+0000 7fa6d3198b80 1 --2- 172.21.15.1:0/3121553677 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556e51fe2930 0x556e51fe2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.488+0000 7fa6d3198b80 1 --2- 172.21.15.1:0/3121553677 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e51fe1130 0x556e51fe1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.488+0000 7fa6d3198b80 1 --2- 172.21.15.1:0/3121553677 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556e51fe1a90 0x556e51fe23a0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.488+0000 7fa6d3198b80 1 -- 172.21.15.1:0/3121553677 >> 172.21.15.1:0/3121553677 conn(0x556e51f3fd60 msgr2=0x556e51fdbe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:34.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.488+0000 7fa6d3198b80 1 -- 172.21.15.1:0/3121553677 shutdown_connections 2024-02-02T15:26:34.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.488+0000 7fa6d3198b80 1 -- 172.21.15.1:0/3121553677 wait complete. 2024-02-02T15:26:34.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.488+0000 7fa6d3198b80 1 -- start start 2024-02-02T15:26:34.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.489+0000 7fa6d3198b80 1 Processor -- start 2024-02-02T15:26:34.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.489+0000 7fa6d3198b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556e51fe1130 0x556e51feaef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:34.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.489+0000 7fa6d3198b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556e51fe1a90 0x556e51feb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:34.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.489+0000 7fa6d3198b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e51fe2930 0x556e51ff2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:34.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.489+0000 7fa6d3198b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e51f31d00 con 0x556e51fe1a90 2024-02-02T15:26:34.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.489+0000 7fa6d3198b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e51ecdd30 con 0x556e51fe1130 2024-02-02T15:26:34.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.489+0000 7fa6d3198b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e51ecd9d0 con 0x556e51fe2930 2024-02-02T15:26:34.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.489+0000 7fa6d3198b80 10 client.0 _ll_register_callbacks cb 0x556e51ecf240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:26:34.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.489+0000 7fa6b7e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556e51fe1a90 0x556e51feb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:34.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.489+0000 7fa6b7e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556e51fe1a90 0x556e51feb430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52812/0 (socket says 172.21.15.1:52812) 2024-02-02T15:26:34.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.489+0000 7fa6b7e00700 1 -- 172.21.15.1:0/1713734570 learned_addr learned my addr 172.21.15.1:0/1713734570 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:34.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.489+0000 7fa6bcc00700 1 --2- 172.21.15.1:0/1713734570 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556e51fe1130 0x556e51feaef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:34.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.489+0000 7fa6bd600700 1 --2- 172.21.15.1:0/1713734570 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e51fe2930 0x556e51ff2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:34.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.490+0000 7fa6b7e00700 1 -- 172.21.15.1:0/1713734570 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e51fe2930 msgr2=0x556e51ff2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:34.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.490+0000 7fa6b7e00700 1 --2- 172.21.15.1:0/1713734570 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e51fe2930 0x556e51ff2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.490+0000 7fa6b7e00700 1 -- 172.21.15.1:0/1713734570 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556e51fe1130 msgr2=0x556e51feaef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:34.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.490+0000 7fa6b7e00700 1 --2- 172.21.15.1:0/1713734570 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556e51fe1130 0x556e51feaef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.490+0000 7fa6b7e00700 1 -- 172.21.15.1:0/1713734570 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6b81a1200 con 0x556e51fe1a90 2024-02-02T15:26:34.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.490+0000 7fa6b7e00700 1 --2- 172.21.15.1:0/1713734570 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556e51fe1a90 0x556e51feb430 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa6a001b040 tx=0x7fa6a00041d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ada35460c57ee15 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:34.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.490+0000 7fa6a6000700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:26:34.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.490+0000 7fa6a6000700 1 -- 172.21.15.1:0/1713734570 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a0005240 con 0x556e51fe1a90 2024-02-02T15:26:34.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.490+0000 7fa6a6000700 1 -- 172.21.15.1:0/1713734570 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fa6a00053a0 con 0x556e51fe1a90 2024-02-02T15:26:34.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.490+0000 7fa6d3198b80 1 -- 172.21.15.1:0/1713734570 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556e51fd40a0 con 0x556e51fe1a90 2024-02-02T15:26:34.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.491+0000 7fa6a6000700 1 -- 172.21.15.1:0/1713734570 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa6a0003d60 con 0x556e51fe1a90 2024-02-02T15:26:34.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.491+0000 7fa6d3198b80 10 client.15339 fetch_fsmap learned FSMap version 27 2024-02-02T15:26:34.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.491+0000 7fa6d3198b80 1 -- 172.21.15.1:0/1713734570 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7fa6a00099e0 con 0x556e51fe1a90 2024-02-02T15:26:34.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.491+0000 7fa6a6000700 1 -- 172.21.15.1:0/1713734570 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556e51fd40a0 con 0x556e51fe1a90 2024-02-02T15:26:34.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.491+0000 7fa6a6000700 1 -- 172.21.15.1:0/1713734570 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa6a0021040 con 0x556e51fe1a90 2024-02-02T15:26:34.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.491+0000 7fa6d3198b80 10 client.15339 fetch_fsmap finished waiting for FSMap version 27 2024-02-02T15:26:34.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.491+0000 7fa6d3198b80 10 client.15339 resolve_mds: resolved name 'h' to daemon mds.h 2024-02-02T15:26:34.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.491+0000 7fa6d3198b80 20 client.15339 populate_metadata read hostname 'smithi001' 2024-02-02T15:26:34.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.491+0000 7fa6d3198b80 1 --2- 172.21.15.1:0/1713734570 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x556e51ff60f0 0x556e51ff5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:34.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.491+0000 7fa6d3198b80 4 client.15339 mds_command: new command op to 24518 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:26:34.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.491+0000 7fa6d3198b80 1 -- 172.21.15.1:0/1713734570 --> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556e51f2ad80 con 0x556e51ff60f0 2024-02-02T15:26:34.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.491+0000 7fa6bcc00700 1 --2- 172.21.15.1:0/1713734570 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x556e51ff60f0 0x556e51ff5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:34.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.492+0000 7fa6bcc00700 1 --2- 172.21.15.1:0/1713734570 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x556e51ff60f0 0x556e51ff5520 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d020a0f4dbcf4bcc server_cookie=ef6077d20f36e061 in_seq=0 out_seq=0 2024-02-02T15:26:34.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.492+0000 7fa6a6000700 10 client.15339 ms_handle_connect on v2:172.21.15.71:6836/628016199 2024-02-02T15:26:34.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.493+0000 7fa6a6000700 1 -- 172.21.15.1:0/1713734570 <== mds.2 v2:172.21.15.71:6836/628016199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x556e51f2ad80 con 0x556e51ff60f0 2024-02-02T15:26:34.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.493+0000 7fa6a6000700 10 client.15339 handle_command_reply: tid=0 2024-02-02T15:26:34.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.493+0000 7fa6d3198b80 1 client.15339 shutdown 2024-02-02T15:26:34.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.493+0000 7fa6d3198b80 1 -- 172.21.15.1:0/1713734570 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556e51fe1a90 msgr2=0x556e51feb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:34.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.493+0000 7fa6d3198b80 1 --2- 172.21.15.1:0/1713734570 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556e51fe1a90 0x556e51feb430 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fa6a001b040 tx=0x7fa6a00041d0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.493+0000 7fa6d3198b80 1 -- 172.21.15.1:0/1713734570 shutdown_connections 2024-02-02T15:26:34.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.493+0000 7fa6d3198b80 1 --2- 172.21.15.1:0/1713734570 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x556e51ff60f0 0x556e51ff5520 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.493+0000 7fa6d3198b80 1 --2- 172.21.15.1:0/1713734570 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e51fe2930 0x556e51ff2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.493+0000 7fa6d3198b80 1 --2- 172.21.15.1:0/1713734570 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556e51fe1130 0x556e51feaef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.493+0000 7fa6d3198b80 1 --2- 172.21.15.1:0/1713734570 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556e51fe1a90 0x556e51feb430 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.493+0000 7fa6a6000700 0 client.15339 ms_handle_reset on v2:172.21.15.71:6836/628016199 2024-02-02T15:26:34.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.493+0000 7fa6d3198b80 1 -- 172.21.15.1:0/1713734570 >> 172.21.15.1:0/1713734570 conn(0x556e51f3fd60 msgr2=0x556e51fdb570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:34.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.493+0000 7fa6d3198b80 1 -- 172.21.15.1:0/1713734570 shutdown_connections 2024-02-02T15:26:34.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.493+0000 7fa6d3198b80 1 -- 172.21.15.1:0/1713734570 wait complete. 2024-02-02T15:26:34.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.494+0000 7fa6d3198b80 20 client.15339 trim_cache size 0 max 16384 2024-02-02T15:26:34.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.511+0000 7fa6d3198b80 1 Processor -- start 2024-02-02T15:26:34.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.511+0000 7fa6d3198b80 1 -- start start 2024-02-02T15:26:34.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.511+0000 7fa6d3198b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e51fe1130 0x556e51fe2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:34.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.511+0000 7fa6d3198b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556e51fe1a90 0x556e51ffee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:34.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.511+0000 7fa6d3198b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556e51fe2930 0x556e51fd7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:34.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.511+0000 7fa6d3198b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e51db5650 con 0x556e51fe2930 2024-02-02T15:26:34.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.511+0000 7fa6d3198b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e51fec0e0 con 0x556e51fe1a90 2024-02-02T15:26:34.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.511+0000 7fa6d3198b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e51dc1b60 con 0x556e51fe1130 2024-02-02T15:26:34.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.512+0000 7fa6bd600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556e51fe2930 0x556e51fd7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:34.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.512+0000 7fa6b7e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556e51fe1a90 0x556e51ffee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:34.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.512+0000 7fa6bd600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556e51fe2930 0x556e51fd7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52816/0 (socket says 172.21.15.1:52816) 2024-02-02T15:26:34.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.512+0000 7fa6bd600700 1 -- 172.21.15.1:0/3028824002 learned_addr learned my addr 172.21.15.1:0/3028824002 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:34.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.512+0000 7fa6bcc00700 1 --2- 172.21.15.1:0/3028824002 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e51fe1130 0x556e51fe2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:34.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.512+0000 7fa6b7e00700 1 -- 172.21.15.1:0/3028824002 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e51fe1130 msgr2=0x556e51fe2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:34.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.512+0000 7fa6b7e00700 1 --2- 172.21.15.1:0/3028824002 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e51fe1130 0x556e51fe2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.512+0000 7fa6b7e00700 1 -- 172.21.15.1:0/3028824002 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556e51fe2930 msgr2=0x556e51fd7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:34.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.512+0000 7fa6b7e00700 1 --2- 172.21.15.1:0/3028824002 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556e51fe2930 0x556e51fd7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.512+0000 7fa6b7e00700 1 -- 172.21.15.1:0/3028824002 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556e51d5fb10 con 0x556e51fe1a90 2024-02-02T15:26:34.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.512+0000 7fa6b7e00700 1 --2- 172.21.15.1:0/3028824002 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556e51fe1a90 0x556e51ffee40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa6a0009cd0 tx=0x7fa6a0011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0f1d222a471cc49 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:34.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.513+0000 7fa6ae600700 1 -- 172.21.15.1:0/3028824002 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a0019420 con 0x556e51fe1a90 2024-02-02T15:26:34.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.513+0000 7fa6ae600700 1 -- 172.21.15.1:0/3028824002 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa6a0004240 con 0x556e51fe1a90 2024-02-02T15:26:34.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.513+0000 7fa6ae600700 1 -- 172.21.15.1:0/3028824002 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a00049d0 con 0x556e51fe1a90 2024-02-02T15:26:34.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.513+0000 7fa6d3198b80 1 -- 172.21.15.1:0/3028824002 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556e51fe1a90 msgr2=0x556e51ffee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:34.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.513+0000 7fa6d3198b80 1 --2- 172.21.15.1:0/3028824002 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556e51fe1a90 0x556e51ffee40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa6a0009cd0 tx=0x7fa6a0011720 comp rx=0 tx=0).stop 2024-02-02T15:26:34.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.513+0000 7fa6d3198b80 1 -- 172.21.15.1:0/3028824002 shutdown_connections 2024-02-02T15:26:34.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.513+0000 7fa6d3198b80 1 --2- 172.21.15.1:0/3028824002 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556e51fe2930 0x556e51fd7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.513+0000 7fa6d3198b80 1 --2- 172.21.15.1:0/3028824002 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e51fe1130 0x556e51fe2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.513+0000 7fa6d3198b80 1 --2- 172.21.15.1:0/3028824002 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556e51fe1a90 0x556e51ffee40 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.513+0000 7fa6d3198b80 1 -- 172.21.15.1:0/3028824002 >> 172.21.15.1:0/3028824002 conn(0x556e51f3fd60 msgr2=0x556e51fdbe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:34.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.513+0000 7fa6d3198b80 1 -- 172.21.15.1:0/3028824002 shutdown_connections 2024-02-02T15:26:34.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.513+0000 7fa6d3198b80 1 -- 172.21.15.1:0/3028824002 wait complete. 2024-02-02T15:26:34.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.514+0000 7fa6d3198b80 1 -- start start 2024-02-02T15:26:34.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.514+0000 7fa6d3198b80 1 Processor -- start 2024-02-02T15:26:34.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.514+0000 7fa6d3198b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e51fe1130 0x556e51ff9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:34.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.514+0000 7fa6d3198b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556e51fe1a90 0x556e51ff95a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:34.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.514+0000 7fa6d3198b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556e51fe2930 0x556e51ffe3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:34.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.514+0000 7fa6d3198b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e51ffe920 con 0x556e51fe2930 2024-02-02T15:26:34.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.514+0000 7fa6d3198b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e51ffea90 con 0x556e51fe1a90 2024-02-02T15:26:34.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.514+0000 7fa6d3198b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556e51ffec50 con 0x556e51fe1130 2024-02-02T15:26:34.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.514+0000 7fa6d3198b80 10 client.0 _ll_register_callbacks cb 0x556e51ecf240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:26:34.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.515+0000 7fa6bd600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556e51fe2930 0x556e51ffe3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:34.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.515+0000 7fa6bd600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556e51fe2930 0x556e51ffe3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52828/0 (socket says 172.21.15.1:52828) 2024-02-02T15:26:34.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.515+0000 7fa6bd600700 1 -- 172.21.15.1:0/1270454381 learned_addr learned my addr 172.21.15.1:0/1270454381 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:34.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.515+0000 7fa6b7e00700 1 --2- 172.21.15.1:0/1270454381 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556e51fe1a90 0x556e51ff95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:34.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.515+0000 7fa6bcc00700 1 --2- 172.21.15.1:0/1270454381 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e51fe1130 0x556e51ff9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:34.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.515+0000 7fa6bd600700 1 -- 172.21.15.1:0/1270454381 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e51fe1130 msgr2=0x556e51ff9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:34.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.515+0000 7fa6bd600700 1 --2- 172.21.15.1:0/1270454381 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e51fe1130 0x556e51ff9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.515+0000 7fa6bd600700 1 -- 172.21.15.1:0/1270454381 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556e51fe1a90 msgr2=0x556e51ff95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:34.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.515+0000 7fa6bd600700 1 --2- 172.21.15.1:0/1270454381 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556e51fe1a90 0x556e51ff95a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.515+0000 7fa6bd600700 1 -- 172.21.15.1:0/1270454381 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6a00099e0 con 0x556e51fe2930 2024-02-02T15:26:34.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.515+0000 7fa6bd600700 1 --2- 172.21.15.1:0/1270454381 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556e51fe2930 0x556e51ffe3e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa6b000b0d0 tx=0x7fa6b00ca860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17b78949c64ee75f server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:34.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.515+0000 7fa6a6000700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:26:34.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.515+0000 7fa6a6000700 1 -- 172.21.15.1:0/1270454381 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b009ae80 con 0x556e51fe2930 2024-02-02T15:26:34.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.515+0000 7fa6a6000700 1 -- 172.21.15.1:0/1270454381 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fa6b009a780 con 0x556e51fe2930 2024-02-02T15:26:34.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.515+0000 7fa6d3198b80 1 -- 172.21.15.1:0/1270454381 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556e51ef0f00 con 0x556e51fe2930 2024-02-02T15:26:34.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.516+0000 7fa6a6000700 1 -- 172.21.15.1:0/1270454381 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa6b009e5b0 con 0x556e51fe2930 2024-02-02T15:26:34.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.516+0000 7fa6d3198b80 10 client.15345 fetch_fsmap learned FSMap version 27 2024-02-02T15:26:34.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.516+0000 7fa6d3198b80 1 -- 172.21.15.1:0/1270454381 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x556e51ef7e50 con 0x556e51fe2930 2024-02-02T15:26:34.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.516+0000 7fa6a6000700 1 -- 172.21.15.1:0/1270454381 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556e51ef0f00 con 0x556e51fe2930 2024-02-02T15:26:34.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.516+0000 7fa6a6000700 1 -- 172.21.15.1:0/1270454381 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa6b0099c70 con 0x556e51fe2930 2024-02-02T15:26:34.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.516+0000 7fa6d3198b80 10 client.15345 fetch_fsmap finished waiting for FSMap version 27 2024-02-02T15:26:34.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.516+0000 7fa6d3198b80 10 client.15345 resolve_mds: resolved name 'h' to daemon mds.h 2024-02-02T15:26:34.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.516+0000 7fa6d3198b80 20 client.15345 populate_metadata read hostname 'smithi001' 2024-02-02T15:26:34.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.516+0000 7fa6d3198b80 1 --2- 172.21.15.1:0/1270454381 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x556e5204c010 0x556e52048980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:34.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.516+0000 7fa6d3198b80 4 client.15345 mds_command: new command op to 24518 tid=0[{"prefix": "status"}] 2024-02-02T15:26:34.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.516+0000 7fa6d3198b80 1 -- 172.21.15.1:0/1270454381 --> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] -- command(tid 0: {"prefix": "status"}) v1 -- 0x556e51d80ca0 con 0x556e5204c010 2024-02-02T15:26:34.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.517+0000 7fa6bcc00700 1 --2- 172.21.15.1:0/1270454381 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x556e5204c010 0x556e52048980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:34.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.517+0000 7fa6a6000700 10 client.15345 ms_handle_connect on v2:172.21.15.71:6836/628016199 2024-02-02T15:26:34.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.517+0000 7fa6bcc00700 1 --2- 172.21.15.1:0/1270454381 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x556e5204c010 0x556e52048980 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ccc75e5b36ed39d8 server_cookie=326fa1b6340597a4 in_seq=0 out_seq=0 2024-02-02T15:26:34.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.517+0000 7fa6a6000700 1 -- 172.21.15.1:0/1270454381 <== mds.2 v2:172.21.15.71:6836/628016199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x556e51d80ca0 con 0x556e5204c010 2024-02-02T15:26:34.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.517+0000 7fa6a6000700 10 client.15345 handle_command_reply: tid=0 2024-02-02T15:26:34.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.517+0000 7fa6d3198b80 1 client.15345 shutdown 2024-02-02T15:26:34.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.517+0000 7fa6d3198b80 1 -- 172.21.15.1:0/1270454381 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556e51fe2930 msgr2=0x556e51ffe3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:34.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.517+0000 7fa6d3198b80 1 --2- 172.21.15.1:0/1270454381 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556e51fe2930 0x556e51ffe3e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa6b000b0d0 tx=0x7fa6b00ca860 comp rx=0 tx=0).stop 2024-02-02T15:26:34.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.518+0000 7fa6d3198b80 1 -- 172.21.15.1:0/1270454381 shutdown_connections 2024-02-02T15:26:34.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.518+0000 7fa6d3198b80 1 --2- 172.21.15.1:0/1270454381 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556e51fe2930 0x556e51ffe3e0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.518+0000 7fa6d3198b80 1 --2- 172.21.15.1:0/1270454381 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x556e5204c010 0x556e52048980 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.518+0000 7fa6d3198b80 1 --2- 172.21.15.1:0/1270454381 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556e51fe1130 0x556e51ff9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.518+0000 7fa6d3198b80 1 --2- 172.21.15.1:0/1270454381 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556e51fe1a90 0x556e51ff95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.518+0000 7fa6a6000700 0 client.15345 ms_handle_reset on v2:172.21.15.71:6836/628016199 2024-02-02T15:26:34.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.518+0000 7fa6d3198b80 1 -- 172.21.15.1:0/1270454381 >> 172.21.15.1:0/1270454381 conn(0x556e51f3fd60 msgr2=0x556e51ff5520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:34.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.518+0000 7fa6d3198b80 1 -- 172.21.15.1:0/1270454381 shutdown_connections 2024-02-02T15:26:34.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.518+0000 7fa6d3198b80 1 -- 172.21.15.1:0/1270454381 wait complete. 2024-02-02T15:26:34.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.518+0000 7fa6d3198b80 20 client.15345 trim_cache size 0 max 16384 2024-02-02T15:26:34.522 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:26:34.522 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "371a409c-c1de-11ee-95b6-87774f69a715", 2024-02-02T15:26:34.522 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 2, 2024-02-02T15:26:34.522 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 24518, 2024-02-02T15:26:34.522 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2024-02-02T15:26:34.522 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2024-02-02T15:26:34.522 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2024-02-02T15:26:34.522 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 77.300161825999993, 2024-02-02T15:26:34.522 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 27, 2024-02-02T15:26:34.522 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 88, 2024-02-02T15:26:34.522 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 84, 2024-02-02T15:26:34.522 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 121.90611188 2024-02-02T15:26:34.522 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:26:34.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.518+0000 7fa6bf600700 1 -- 172.21.15.1:0/988293293 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fa6980a0520 msgr2=0x7fa6980a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:34.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.518+0000 7fa6bf600700 1 --2- 172.21.15.1:0/988293293 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fa6980a0520 0x7fa6980a29e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa6a8005950 tx=0x7fa6a8009260 comp rx=0 tx=0).stop 2024-02-02T15:26:34.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.518+0000 7fa6bf600700 1 -- 172.21.15.1:0/988293293 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa6b8102990 msgr2=0x7fa6b81a0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:34.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.518+0000 7fa6bf600700 1 --2- 172.21.15.1:0/988293293 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa6b8102990 0x7fa6b81a0310 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa6b000b770 tx=0x7fa6b0008ec0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.518+0000 7fa6bf600700 1 -- 172.21.15.1:0/988293293 shutdown_connections 2024-02-02T15:26:34.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.518+0000 7fa6bf600700 1 --2- 172.21.15.1:0/988293293 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa6b8102990 0x7fa6b81a0310 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.518+0000 7fa6bf600700 1 --2- 172.21.15.1:0/988293293 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa6b8100e00 0x7fa6b8198d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.518+0000 7fa6bf600700 1 --2- 172.21.15.1:0/988293293 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fa6980a0520 0x7fa6980a29e0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.519+0000 7fa6bf600700 1 --2- 172.21.15.1:0/988293293 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa6b8101ff0 0x7fa6b81992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.519+0000 7fa6bf600700 1 -- 172.21.15.1:0/988293293 >> 172.21.15.1:0/988293293 conn(0x7fa6b80fc370 msgr2=0x7fa6b80fe520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:34.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.519+0000 7fa6bf600700 1 -- 172.21.15.1:0/988293293 shutdown_connections 2024-02-02T15:26:34.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.519+0000 7fa6bf600700 1 -- 172.21.15.1:0/988293293 wait complete. 2024-02-02T15:26:34.534 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:34 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/392594067' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:26:34.534 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:34 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/987838252' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-02T15:26:34.534 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:34 smithi001 ceph-mon[35470]: pgmap v292: 129 pgs: 129 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 639 B/s wr, 12 op/s 2024-02-02T15:26:34.537 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-02-02T15:26:34.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:34 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/392594067' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:26:34.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:34 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/987838252' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-02T15:26:34.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:34 smithi071 ceph-mon[44707]: pgmap v292: 129 pgs: 129 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 639 B/s wr, 12 op/s 2024-02-02T15:26:34.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:34 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/392594067' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:26:34.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:34 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/987838252' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-02T15:26:34.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:34 smithi142 ceph-mon[45728]: pgmap v292: 129 pgs: 129 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 639 B/s wr, 12 op/s 2024-02-02T15:26:34.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.736+0000 7f5889600700 1 -- 172.21.15.1:0/2093297614 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f58841041a0 msgr2=0x7f5884106590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:34.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.736+0000 7f5889600700 1 --2- 172.21.15.1:0/2093297614 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f58841041a0 0x7f5884106590 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f586c009990 tx=0x7f586c009ca0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.736+0000 7f5889600700 1 -- 172.21.15.1:0/2093297614 shutdown_connections 2024-02-02T15:26:34.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.736+0000 7f5889600700 1 --2- 172.21.15.1:0/2093297614 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5884106ad0 0x7f5884108f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.736+0000 7f5889600700 1 --2- 172.21.15.1:0/2093297614 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5884101870 0x7f5884103c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.736+0000 7f5889600700 1 --2- 172.21.15.1:0/2093297614 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f58841041a0 0x7f5884106590 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.736+0000 7f5889600700 1 -- 172.21.15.1:0/2093297614 >> 172.21.15.1:0/2093297614 conn(0x7f58840fb280 msgr2=0x7f58840fd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:34.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.736+0000 7f5889600700 1 -- 172.21.15.1:0/2093297614 shutdown_connections 2024-02-02T15:26:34.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.736+0000 7f5889600700 1 -- 172.21.15.1:0/2093297614 wait complete. 2024-02-02T15:26:34.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.737+0000 7f5889600700 1 Processor -- start 2024-02-02T15:26:34.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.738+0000 7f5889600700 1 -- start start 2024-02-02T15:26:34.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.738+0000 7f5889600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5884101870 0x7f58841011e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:34.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.738+0000 7f5889600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f58841041a0 0x7f58840ff820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:34.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.738+0000 7f5889600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5884106ad0 0x7f58840ffd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:34.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.738+0000 7f5889600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5884100500 con 0x7f5884106ad0 2024-02-02T15:26:34.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.738+0000 7f5889600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5884100670 con 0x7f58841041a0 2024-02-02T15:26:34.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.738+0000 7f5889600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58841a0b70 con 0x7f5884101870 2024-02-02T15:26:34.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.738+0000 7f5883400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5884106ad0 0x7f58840ffd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:34.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.738+0000 7f5883400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5884106ad0 0x7f58840ffd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52852/0 (socket says 172.21.15.1:52852) 2024-02-02T15:26:34.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.738+0000 7f5882a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5884101870 0x7f58841011e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:34.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.738+0000 7f5883400700 1 -- 172.21.15.1:0/1069639680 learned_addr learned my addr 172.21.15.1:0/1069639680 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:34.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.738+0000 7f5882000700 1 --2- 172.21.15.1:0/1069639680 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f58841041a0 0x7f58840ff820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:34.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.739+0000 7f5883400700 1 -- 172.21.15.1:0/1069639680 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5884101870 msgr2=0x7f58841011e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:34.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.739+0000 7f5883400700 1 --2- 172.21.15.1:0/1069639680 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5884101870 0x7f58841011e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.739+0000 7f5883400700 1 -- 172.21.15.1:0/1069639680 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f58841041a0 msgr2=0x7f58840ff820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:34.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.739+0000 7f5883400700 1 --2- 172.21.15.1:0/1069639680 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f58841041a0 0x7f58840ff820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.739+0000 7f5883400700 1 -- 172.21.15.1:0/1069639680 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f587c009c40 con 0x7f5884106ad0 2024-02-02T15:26:34.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.739+0000 7f5882a00700 1 --2- 172.21.15.1:0/1069639680 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5884101870 0x7f58841011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:26:34.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.739+0000 7f5883400700 1 --2- 172.21.15.1:0/1069639680 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5884106ad0 0x7f58840ffd60 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f587c00c370 tx=0x7f587c00c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:34.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.739+0000 7f587b400700 1 -- 172.21.15.1:0/1069639680 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f587c017070 con 0x7f5884106ad0 2024-02-02T15:26:34.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.739+0000 7f5889600700 1 -- 172.21.15.1:0/1069639680 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f586c0095f0 con 0x7f5884106ad0 2024-02-02T15:26:34.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.739+0000 7f587b400700 1 -- 172.21.15.1:0/1069639680 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f587c00f040 con 0x7f5884106ad0 2024-02-02T15:26:34.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.739+0000 7f587b400700 1 -- 172.21.15.1:0/1069639680 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f587c00e050 con 0x7f5884106ad0 2024-02-02T15:26:34.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.739+0000 7f5889600700 1 -- 172.21.15.1:0/1069639680 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58841a11c0 con 0x7f5884106ad0 2024-02-02T15:26:34.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.742+0000 7f587b400700 1 -- 172.21.15.1:0/1069639680 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) v1 ==== 135069+0+0 (secure 0 0 0) 0x7f587c00cb60 con 0x7f5884106ad0 2024-02-02T15:26:34.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.744+0000 7f589d0c6b80 1 Processor -- start 2024-02-02T15:26:34.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.744+0000 7f589d0c6b80 1 -- start start 2024-02-02T15:26:34.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.745+0000 7f589d0c6b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557cea8c4130 0x557cea8c4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:34.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.745+0000 7f587b400700 1 --2- 172.21.15.1:0/1069639680 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f58640a05a0 0x7f58640a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:34.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.745+0000 7f589d0c6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557cea8c4a90 0x557cea8c53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:34.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.745+0000 7f589d0c6b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557cea8c5930 0x557cea8c5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:34.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.745+0000 7f589d0c6b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cea837470 con 0x557cea8c4130 2024-02-02T15:26:34.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.745+0000 7f589d0c6b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cea81ba50 con 0x557cea8c5930 2024-02-02T15:26:34.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.745+0000 7f589d0c6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cea819410 con 0x557cea8c4a90 2024-02-02T15:26:34.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.745+0000 7f587b400700 1 -- 172.21.15.1:0/1069639680 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f587c0cb600 con 0x7f5884106ad0 2024-02-02T15:26:34.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.745+0000 7f5883400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557cea8c5930 0x557cea8c5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:34.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.745+0000 7f5883400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557cea8c5930 0x557cea8c5dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:57780/0 (socket says 172.21.15.1:57780) 2024-02-02T15:26:34.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.745+0000 7f5883400700 1 -- 172.21.15.1:0/1141638290 learned_addr learned my addr 172.21.15.1:0/1141638290 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:34.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.745+0000 7f5882a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557cea8c4130 0x557cea8c4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:34.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.745+0000 7f5882000700 1 --2- 172.21.15.1:0/1069639680 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f58640a05a0 0x7f58640a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:34.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.745+0000 7f5882000700 1 --2- 172.21.15.1:0/1141638290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557cea8c4a90 0x557cea8c53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:34.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.745+0000 7f5883400700 1 -- 172.21.15.1:0/1141638290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557cea8c4a90 msgr2=0x557cea8c53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:34.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.745+0000 7f5883400700 1 --2- 172.21.15.1:0/1141638290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557cea8c4a90 0x557cea8c53a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.745+0000 7f5883400700 1 -- 172.21.15.1:0/1141638290 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557cea8c4130 msgr2=0x557cea8c4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:34.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.745+0000 7f5883400700 1 --2- 172.21.15.1:0/1141638290 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557cea8c4130 0x557cea8c4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.745+0000 7f5883400700 1 -- 172.21.15.1:0/1141638290 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557cea696360 con 0x557cea8c5930 2024-02-02T15:26:34.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.746+0000 7f5883400700 1 --2- 172.21.15.1:0/1141638290 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557cea8c5930 0x557cea8c5dc0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f587c094f90 tx=0x7f587c003dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47fc6424f6353222 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:34.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.746+0000 7f5882000700 1 --2- 172.21.15.1:0/1069639680 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f58640a05a0 0x7f58640a2a60 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x557cea8c4ed0 tx=0x7f586c005bc0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:34.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.746+0000 7f5873400700 1 -- 172.21.15.1:0/1141638290 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f587c070b80 con 0x557cea8c5930 2024-02-02T15:26:34.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.746+0000 7f5873400700 1 -- 172.21.15.1:0/1141638290 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f587c0092e0 con 0x557cea8c5930 2024-02-02T15:26:34.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.746+0000 7f5873400700 1 -- 172.21.15.1:0/1141638290 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f587c09c8c0 con 0x557cea8c5930 2024-02-02T15:26:34.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.746+0000 7f589d0c6b80 1 -- 172.21.15.1:0/1141638290 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557cea8c5930 msgr2=0x557cea8c5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:34.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.746+0000 7f589d0c6b80 1 --2- 172.21.15.1:0/1141638290 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557cea8c5930 0x557cea8c5dc0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f587c094f90 tx=0x7f587c003dd0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.747+0000 7f589d0c6b80 1 -- 172.21.15.1:0/1141638290 shutdown_connections 2024-02-02T15:26:34.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.747+0000 7f589d0c6b80 1 --2- 172.21.15.1:0/1141638290 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557cea8c5930 0x557cea8c5dc0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.747+0000 7f589d0c6b80 1 --2- 172.21.15.1:0/1141638290 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557cea8c4130 0x557cea8c4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.747+0000 7f589d0c6b80 1 --2- 172.21.15.1:0/1141638290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557cea8c4a90 0x557cea8c53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.747+0000 7f589d0c6b80 1 -- 172.21.15.1:0/1141638290 >> 172.21.15.1:0/1141638290 conn(0x557cea822d60 msgr2=0x557cea8bee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:34.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.747+0000 7f589d0c6b80 1 -- 172.21.15.1:0/1141638290 shutdown_connections 2024-02-02T15:26:34.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.747+0000 7f589d0c6b80 1 -- 172.21.15.1:0/1141638290 wait complete. 2024-02-02T15:26:34.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.747+0000 7f589d0c6b80 1 -- start start 2024-02-02T15:26:34.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.747+0000 7f589d0c6b80 1 Processor -- start 2024-02-02T15:26:34.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.748+0000 7f589d0c6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557cea8c4130 0x557cea8cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:34.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.748+0000 7f589d0c6b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557cea8c4a90 0x557cea8ce3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:34.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.748+0000 7f589d0c6b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557cea8c5930 0x557cea8d5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:34.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.748+0000 7f589d0c6b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cea814d00 con 0x557cea8c5930 2024-02-02T15:26:34.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.748+0000 7f589d0c6b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cea7b0d30 con 0x557cea8c4a90 2024-02-02T15:26:34.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.748+0000 7f589d0c6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cea7b09d0 con 0x557cea8c4130 2024-02-02T15:26:34.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.748+0000 7f589d0c6b80 10 client.0 _ll_register_callbacks cb 0x557cea7b2240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:26:34.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.748+0000 7f5882000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557cea8c4a90 0x557cea8ce3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:34.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.748+0000 7f5882a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557cea8c4130 0x557cea8cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:34.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.748+0000 7f5882000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557cea8c4a90 0x557cea8ce3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:57794/0 (socket says 172.21.15.1:57794) 2024-02-02T15:26:34.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.748+0000 7f5882000700 1 -- 172.21.15.1:0/3067705249 learned_addr learned my addr 172.21.15.1:0/3067705249 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:34.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.748+0000 7f5883400700 1 --2- 172.21.15.1:0/3067705249 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557cea8c5930 0x557cea8d5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:34.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.748+0000 7f5882000700 1 -- 172.21.15.1:0/3067705249 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557cea8c4130 msgr2=0x557cea8cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:34.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.748+0000 7f5882000700 1 --2- 172.21.15.1:0/3067705249 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557cea8c4130 0x557cea8cde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.749+0000 7f5882000700 1 -- 172.21.15.1:0/3067705249 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557cea8c5930 msgr2=0x557cea8d5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:34.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.749+0000 7f5882000700 1 --2- 172.21.15.1:0/3067705249 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557cea8c5930 0x557cea8d5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.749+0000 7f5882000700 1 -- 172.21.15.1:0/3067705249 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58841a11c0 con 0x557cea8c4a90 2024-02-02T15:26:34.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.749+0000 7f5882000700 1 --2- 172.21.15.1:0/3067705249 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557cea8c4a90 0x557cea8ce3c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f586c009990 tx=0x7f586c005fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5b81dd03e7d175c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:34.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.749+0000 7f586be00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:26:34.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.749+0000 7f586be00700 1 -- 172.21.15.1:0/3067705249 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f586c02c070 con 0x557cea8c4a90 2024-02-02T15:26:34.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.749+0000 7f589d0c6b80 1 -- 172.21.15.1:0/3067705249 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557cea8b70a0 con 0x557cea8c4a90 2024-02-02T15:26:34.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.749+0000 7f586be00700 1 -- 172.21.15.1:0/3067705249 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f586c004af0 con 0x557cea8c4a90 2024-02-02T15:26:34.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.749+0000 7f586be00700 1 -- 172.21.15.1:0/3067705249 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f586c01d040 con 0x557cea8c4a90 2024-02-02T15:26:34.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.749+0000 7f589d0c6b80 10 client.25385 fetch_fsmap learned FSMap version 27 2024-02-02T15:26:34.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.749+0000 7f589d0c6b80 1 -- 172.21.15.1:0/3067705249 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x557cea642b10 con 0x557cea8c4a90 2024-02-02T15:26:34.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.750+0000 7f586be00700 1 -- 172.21.15.1:0/3067705249 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557cea8b70a0 con 0x557cea8c4a90 2024-02-02T15:26:34.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.750+0000 7f586be00700 1 -- 172.21.15.1:0/3067705249 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f586c030410 con 0x557cea8c4a90 2024-02-02T15:26:34.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.750+0000 7f589d0c6b80 10 client.25385 fetch_fsmap finished waiting for FSMap version 27 2024-02-02T15:26:34.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.750+0000 7f589d0c6b80 10 client.25385 resolve_mds: resolved name 'i' to daemon mds.i 2024-02-02T15:26:34.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.750+0000 7f589d0c6b80 20 client.25385 populate_metadata read hostname 'smithi001' 2024-02-02T15:26:34.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.750+0000 7f589d0c6b80 1 --2- 172.21.15.1:0/3067705249 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x557cea8d91b0 0x557cea8d8550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:34.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.750+0000 7f589d0c6b80 4 client.25385 mds_command: new command op to 24529 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:26:34.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.750+0000 7f589d0c6b80 1 -- 172.21.15.1:0/3067705249 --> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557cea80dd80 con 0x557cea8d91b0 2024-02-02T15:26:34.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.750+0000 7f5882a00700 1 --2- 172.21.15.1:0/3067705249 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x557cea8d91b0 0x557cea8d8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:34.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.751+0000 7f5882a00700 1 --2- 172.21.15.1:0/3067705249 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x557cea8d91b0 0x557cea8d8550 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f631d89bac443fc4 server_cookie=cf5842e099bf9b91 in_seq=0 out_seq=0 2024-02-02T15:26:34.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.751+0000 7f586be00700 10 client.25385 ms_handle_connect on v2:172.21.15.142:6838/321675421 2024-02-02T15:26:34.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.752+0000 7f586be00700 1 -- 172.21.15.1:0/3067705249 <== mds.1 v2:172.21.15.142:6838/321675421 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x557cea80dd80 con 0x557cea8d91b0 2024-02-02T15:26:34.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.752+0000 7f586be00700 10 client.25385 handle_command_reply: tid=0 2024-02-02T15:26:34.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.752+0000 7f589d0c6b80 1 client.25385 shutdown 2024-02-02T15:26:34.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.752+0000 7f589d0c6b80 1 -- 172.21.15.1:0/3067705249 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557cea8c4a90 msgr2=0x557cea8ce3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:34.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.752+0000 7f589d0c6b80 1 --2- 172.21.15.1:0/3067705249 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557cea8c4a90 0x557cea8ce3c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f586c009990 tx=0x7f586c005fb0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.752+0000 7f589d0c6b80 1 -- 172.21.15.1:0/3067705249 shutdown_connections 2024-02-02T15:26:34.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.752+0000 7f589d0c6b80 1 --2- 172.21.15.1:0/3067705249 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x557cea8d91b0 0x557cea8d8550 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.752+0000 7f589d0c6b80 1 --2- 172.21.15.1:0/3067705249 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557cea8c5930 0x557cea8d5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.752+0000 7f589d0c6b80 1 --2- 172.21.15.1:0/3067705249 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557cea8c4130 0x557cea8cde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.752+0000 7f589d0c6b80 1 --2- 172.21.15.1:0/3067705249 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557cea8c4a90 0x557cea8ce3c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.752+0000 7f586be00700 0 client.25385 ms_handle_reset on v2:172.21.15.142:6838/321675421 2024-02-02T15:26:34.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.752+0000 7f589d0c6b80 1 -- 172.21.15.1:0/3067705249 >> 172.21.15.1:0/3067705249 conn(0x557cea822d60 msgr2=0x557cea8be570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:34.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.752+0000 7f589d0c6b80 1 -- 172.21.15.1:0/3067705249 shutdown_connections 2024-02-02T15:26:34.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.752+0000 7f589d0c6b80 1 -- 172.21.15.1:0/3067705249 wait complete. 2024-02-02T15:26:34.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.753+0000 7f589d0c6b80 20 client.25385 trim_cache size 0 max 16384 2024-02-02T15:26:34.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.771+0000 7f589d0c6b80 1 Processor -- start 2024-02-02T15:26:34.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.771+0000 7f589d0c6b80 1 -- start start 2024-02-02T15:26:34.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.771+0000 7f589d0c6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557cea8c4130 0x557cea8d8550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:34.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.771+0000 7f589d0c6b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557cea8c4a90 0x557cea8e1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:34.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.771+0000 7f589d0c6b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557cea8c5930 0x557cea8d8a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:34.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.771+0000 7f589d0c6b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cea698650 con 0x557cea8c5930 2024-02-02T15:26:34.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.771+0000 7f589d0c6b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cea80ddb0 con 0x557cea8c4a90 2024-02-02T15:26:34.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.771+0000 7f589d0c6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cea819d80 con 0x557cea8c4130 2024-02-02T15:26:34.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.771+0000 7f5882000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557cea8c4a90 0x557cea8e1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:34.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.772+0000 7f5883400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557cea8c5930 0x557cea8d8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:34.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.772+0000 7f5882000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557cea8c4a90 0x557cea8e1f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:57800/0 (socket says 172.21.15.1:57800) 2024-02-02T15:26:34.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.772+0000 7f5882000700 1 -- 172.21.15.1:0/2153444571 learned_addr learned my addr 172.21.15.1:0/2153444571 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:34.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.772+0000 7f5882a00700 1 --2- 172.21.15.1:0/2153444571 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557cea8c4130 0x557cea8d8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:34.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.772+0000 7f5882000700 1 -- 172.21.15.1:0/2153444571 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557cea8c4130 msgr2=0x557cea8d8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:34.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.772+0000 7f5882000700 1 --2- 172.21.15.1:0/2153444571 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557cea8c4130 0x557cea8d8550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.772+0000 7f5882000700 1 -- 172.21.15.1:0/2153444571 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557cea8c5930 msgr2=0x557cea8d8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:34.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.772+0000 7f5882000700 1 --2- 172.21.15.1:0/2153444571 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557cea8c5930 0x557cea8d8a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.772+0000 7f5882000700 1 -- 172.21.15.1:0/2153444571 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557cea7dae50 con 0x557cea8c4a90 2024-02-02T15:26:34.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.772+0000 7f5883400700 1 --2- 172.21.15.1:0/2153444571 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557cea8c5930 0x557cea8d8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:26:34.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.772+0000 7f5882000700 1 --2- 172.21.15.1:0/2153444571 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557cea8c4a90 0x557cea8e1f20 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f586c0098e0 tx=0x7f586c005d40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da0344beb1230c54 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:34.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.772+0000 7f5873e00700 1 -- 172.21.15.1:0/2153444571 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f586c01d030 con 0x557cea8c4a90 2024-02-02T15:26:34.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.772+0000 7f5873e00700 1 -- 172.21.15.1:0/2153444571 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f586c030940 con 0x557cea8c4a90 2024-02-02T15:26:34.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.772+0000 7f5873e00700 1 -- 172.21.15.1:0/2153444571 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f586c02c070 con 0x557cea8c4a90 2024-02-02T15:26:34.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.773+0000 7f589d0c6b80 1 -- 172.21.15.1:0/2153444571 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557cea8c4a90 msgr2=0x557cea8e1f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:34.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.773+0000 7f589d0c6b80 1 --2- 172.21.15.1:0/2153444571 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557cea8c4a90 0x557cea8e1f20 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f586c0098e0 tx=0x7f586c005d40 comp rx=0 tx=0).stop 2024-02-02T15:26:34.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.773+0000 7f589d0c6b80 1 -- 172.21.15.1:0/2153444571 shutdown_connections 2024-02-02T15:26:34.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.773+0000 7f589d0c6b80 1 --2- 172.21.15.1:0/2153444571 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557cea8c5930 0x557cea8d8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.773+0000 7f589d0c6b80 1 --2- 172.21.15.1:0/2153444571 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557cea8c4130 0x557cea8d8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.773+0000 7f589d0c6b80 1 --2- 172.21.15.1:0/2153444571 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557cea8c4a90 0x557cea8e1f20 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.773+0000 7f589d0c6b80 1 -- 172.21.15.1:0/2153444571 >> 172.21.15.1:0/2153444571 conn(0x557cea822d60 msgr2=0x557cea8cbd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:34.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.773+0000 7f589d0c6b80 1 -- 172.21.15.1:0/2153444571 shutdown_connections 2024-02-02T15:26:34.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.773+0000 7f589d0c6b80 1 -- 172.21.15.1:0/2153444571 wait complete. 2024-02-02T15:26:34.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.774+0000 7f589d0c6b80 1 -- start start 2024-02-02T15:26:34.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.774+0000 7f589d0c6b80 1 Processor -- start 2024-02-02T15:26:34.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.774+0000 7f589d0c6b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557cea8c4130 0x557cea8dcad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:34.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.774+0000 7f589d0c6b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557cea8c4a90 0x557cea8dd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:34.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.774+0000 7f589d0c6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557cea8c5930 0x557cea92aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:34.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.774+0000 7f589d0c6b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cea8e2040 con 0x557cea8c4130 2024-02-02T15:26:34.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.774+0000 7f589d0c6b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cea8e21b0 con 0x557cea8c4a90 2024-02-02T15:26:34.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.774+0000 7f589d0c6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557cea92b1e0 con 0x557cea8c5930 2024-02-02T15:26:34.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.774+0000 7f589d0c6b80 10 client.0 _ll_register_callbacks cb 0x557cea7b2240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:26:34.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.775+0000 7f5882a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557cea8c4130 0x557cea8dcad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:34.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.775+0000 7f5882a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557cea8c4130 0x557cea8dcad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52892/0 (socket says 172.21.15.1:52892) 2024-02-02T15:26:34.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.775+0000 7f5882a00700 1 -- 172.21.15.1:0/1262594892 learned_addr learned my addr 172.21.15.1:0/1262594892 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:34.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.775+0000 7f5882000700 1 --2- 172.21.15.1:0/1262594892 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557cea8c4a90 0x557cea8dd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:34.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.775+0000 7f5883400700 1 --2- 172.21.15.1:0/1262594892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557cea8c5930 0x557cea92aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:34.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.775+0000 7f5882a00700 1 -- 172.21.15.1:0/1262594892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557cea8c5930 msgr2=0x557cea92aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:34.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.775+0000 7f5882a00700 1 --2- 172.21.15.1:0/1262594892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557cea8c5930 0x557cea92aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.775+0000 7f5882a00700 1 -- 172.21.15.1:0/1262594892 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557cea8c4a90 msgr2=0x557cea8dd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:34.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.775+0000 7f5882a00700 1 --2- 172.21.15.1:0/1262594892 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557cea8c4a90 0x557cea8dd010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.775+0000 7f5882a00700 1 -- 172.21.15.1:0/1262594892 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557cea696360 con 0x557cea8c4130 2024-02-02T15:26:34.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.775+0000 7f5882a00700 1 --2- 172.21.15.1:0/1262594892 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557cea8c4130 0x557cea8dcad0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x557cea8cde50 tx=0x7f5874020320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c040ee04ccaa3ca server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:34.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.775+0000 7f586be00700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:26:34.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.776+0000 7f586be00700 1 -- 172.21.15.1:0/1262594892 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5874017030 con 0x557cea8c4130 2024-02-02T15:26:34.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.776+0000 7f586be00700 1 -- 172.21.15.1:0/1262594892 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f5874004a30 con 0x557cea8c4130 2024-02-02T15:26:34.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.776+0000 7f589d0c6b80 1 -- 172.21.15.1:0/1262594892 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f587c0092e0 con 0x557cea8c4130 2024-02-02T15:26:34.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.776+0000 7f586be00700 1 -- 172.21.15.1:0/1262594892 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5874004bf0 con 0x557cea8c4130 2024-02-02T15:26:34.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.776+0000 7f589d0c6b80 10 client.15363 fetch_fsmap learned FSMap version 27 2024-02-02T15:26:34.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.776+0000 7f589d0c6b80 1 -- 172.21.15.1:0/1262594892 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x557cea642b10 con 0x557cea8c4130 2024-02-02T15:26:34.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.776+0000 7f586be00700 1 -- 172.21.15.1:0/1262594892 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f587c0092e0 con 0x557cea8c4130 2024-02-02T15:26:34.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.776+0000 7f586be00700 1 -- 172.21.15.1:0/1262594892 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f58740050f0 con 0x557cea8c4130 2024-02-02T15:26:34.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.776+0000 7f589d0c6b80 10 client.15363 fetch_fsmap finished waiting for FSMap version 27 2024-02-02T15:26:34.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.776+0000 7f589d0c6b80 10 client.15363 resolve_mds: resolved name 'i' to daemon mds.i 2024-02-02T15:26:34.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.776+0000 7f589d0c6b80 20 client.15363 populate_metadata read hostname 'smithi001' 2024-02-02T15:26:34.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.776+0000 7f589d0c6b80 1 --2- 172.21.15.1:0/1262594892 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x557cea92ff40 0x557cea92c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:34.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.776+0000 7f589d0c6b80 4 client.15363 mds_command: new command op to 24529 tid=0[{"prefix": "status"}] 2024-02-02T15:26:34.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.776+0000 7f589d0c6b80 1 -- 172.21.15.1:0/1262594892 --> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] -- command(tid 0: {"prefix": "status"}) v1 -- 0x557cea7d3f00 con 0x557cea92ff40 2024-02-02T15:26:34.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.777+0000 7f5882000700 1 --2- 172.21.15.1:0/1262594892 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x557cea92ff40 0x557cea92c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:34.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.777+0000 7f5882000700 1 --2- 172.21.15.1:0/1262594892 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x557cea92ff40 0x557cea92c8b0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9565246270dd50c2 server_cookie=4d7d20e1ee007d69 in_seq=0 out_seq=0 2024-02-02T15:26:34.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.777+0000 7f586be00700 10 client.15363 ms_handle_connect on v2:172.21.15.142:6838/321675421 2024-02-02T15:26:34.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.778+0000 7f586be00700 1 -- 172.21.15.1:0/1262594892 <== mds.1 v2:172.21.15.142:6838/321675421 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x557cea7d3f00 con 0x557cea92ff40 2024-02-02T15:26:34.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.778+0000 7f586be00700 10 client.15363 handle_command_reply: tid=0 2024-02-02T15:26:34.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.778+0000 7f589d0c6b80 1 client.15363 shutdown 2024-02-02T15:26:34.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.778+0000 7f589d0c6b80 1 -- 172.21.15.1:0/1262594892 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557cea8c4130 msgr2=0x557cea8dcad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:34.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.778+0000 7f589d0c6b80 1 --2- 172.21.15.1:0/1262594892 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557cea8c4130 0x557cea8dcad0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x557cea8cde50 tx=0x7f5874020320 comp rx=0 tx=0).stop 2024-02-02T15:26:34.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.778+0000 7f589d0c6b80 1 -- 172.21.15.1:0/1262594892 shutdown_connections 2024-02-02T15:26:34.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.778+0000 7f589d0c6b80 1 --2- 172.21.15.1:0/1262594892 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x557cea92ff40 0x557cea92c8b0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.778+0000 7f589d0c6b80 1 --2- 172.21.15.1:0/1262594892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557cea8c5930 0x557cea92aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.778+0000 7f589d0c6b80 1 --2- 172.21.15.1:0/1262594892 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557cea8c4130 0x557cea8dcad0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.778+0000 7f589d0c6b80 1 --2- 172.21.15.1:0/1262594892 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557cea8c4a90 0x557cea8dd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.778+0000 7f586be00700 0 client.15363 ms_handle_reset on v2:172.21.15.142:6838/321675421 2024-02-02T15:26:34.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.778+0000 7f589d0c6b80 1 -- 172.21.15.1:0/1262594892 >> 172.21.15.1:0/1262594892 conn(0x557cea822d60 msgr2=0x557cea8c88a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:34.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.778+0000 7f589d0c6b80 1 -- 172.21.15.1:0/1262594892 shutdown_connections 2024-02-02T15:26:34.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.778+0000 7f589d0c6b80 1 -- 172.21.15.1:0/1262594892 wait complete. 2024-02-02T15:26:34.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.778+0000 7f589d0c6b80 20 client.15363 trim_cache size 0 max 16384 2024-02-02T15:26:34.782 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:26:34.782 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "371a409c-c1de-11ee-95b6-87774f69a715", 2024-02-02T15:26:34.782 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 1, 2024-02-02T15:26:34.782 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 24529, 2024-02-02T15:26:34.782 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2024-02-02T15:26:34.782 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2024-02-02T15:26:34.782 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2024-02-02T15:26:34.782 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 78.570926682999996, 2024-02-02T15:26:34.783 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 27, 2024-02-02T15:26:34.783 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 88, 2024-02-02T15:26:34.783 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 84, 2024-02-02T15:26:34.783 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 120.261718676 2024-02-02T15:26:34.783 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:26:34.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.779+0000 7f5889600700 1 -- 172.21.15.1:0/1069639680 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f58640a05a0 msgr2=0x7f58640a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:34.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.779+0000 7f5889600700 1 --2- 172.21.15.1:0/1069639680 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f58640a05a0 0x7f58640a2a60 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x557cea8c4ed0 tx=0x7f586c005bc0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.779+0000 7f5889600700 1 -- 172.21.15.1:0/1069639680 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5884106ad0 msgr2=0x7f58840ffd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:34.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.779+0000 7f5889600700 1 --2- 172.21.15.1:0/1069639680 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5884106ad0 0x7f58840ffd60 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f587c00c370 tx=0x7f587c00c680 comp rx=0 tx=0).stop 2024-02-02T15:26:34.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.779+0000 7f5889600700 1 -- 172.21.15.1:0/1069639680 shutdown_connections 2024-02-02T15:26:34.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.779+0000 7f5889600700 1 --2- 172.21.15.1:0/1069639680 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5884106ad0 0x7f58840ffd60 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.779+0000 7f5889600700 1 --2- 172.21.15.1:0/1069639680 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f58640a05a0 0x7f58640a2a60 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.779+0000 7f5889600700 1 --2- 172.21.15.1:0/1069639680 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5884101870 0x7f58841011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.779+0000 7f5889600700 1 --2- 172.21.15.1:0/1069639680 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f58841041a0 0x7f58840ff820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.779+0000 7f5889600700 1 -- 172.21.15.1:0/1069639680 >> 172.21.15.1:0/1069639680 conn(0x7f58840fb280 msgr2=0x7f58841024e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:34.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.779+0000 7f5889600700 1 -- 172.21.15.1:0/1069639680 shutdown_connections 2024-02-02T15:26:34.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.779+0000 7f5889600700 1 -- 172.21.15.1:0/1069639680 wait complete. 2024-02-02T15:26:34.798 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-02-02T15:26:34.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.997+0000 7fccde800700 1 -- 172.21.15.1:0/1843781355 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fccd8100e00 msgr2=0x7fccd8101220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:34.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.997+0000 7fccde800700 1 --2- 172.21.15.1:0/1843781355 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fccd8100e00 0x7fccd8101220 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fccc80099c0 tx=0x7fccc8009cd0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.997+0000 7fccde800700 1 -- 172.21.15.1:0/1843781355 shutdown_connections 2024-02-02T15:26:34.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.997+0000 7fccde800700 1 --2- 172.21.15.1:0/1843781355 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fccd8102990 0x7fccd8109240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.997+0000 7fccde800700 1 --2- 172.21.15.1:0/1843781355 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fccd8100e00 0x7fccd8101220 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.997+0000 7fccde800700 1 --2- 172.21.15.1:0/1843781355 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fccd8101ff0 0x7fccd8102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:34.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.997+0000 7fccde800700 1 -- 172.21.15.1:0/1843781355 >> 172.21.15.1:0/1843781355 conn(0x7fccd80fc370 msgr2=0x7fccd80fe7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:34.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.997+0000 7fccde800700 1 -- 172.21.15.1:0/1843781355 shutdown_connections 2024-02-02T15:26:34.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.997+0000 7fccde800700 1 -- 172.21.15.1:0/1843781355 wait complete. 2024-02-02T15:26:35.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.998+0000 7fccde800700 1 Processor -- start 2024-02-02T15:26:35.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.998+0000 7fccde800700 1 -- start start 2024-02-02T15:26:35.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.999+0000 7fccde800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fccd8100e00 0x7fccd8198da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:35.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.999+0000 7fccde800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fccd8101ff0 0x7fccd81992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:35.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.999+0000 7fccde800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fccd8102990 0x7fccd81a0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:35.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.999+0000 7fccde800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd81a08e0 con 0x7fccd8102990 2024-02-02T15:26:35.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.999+0000 7fccde800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd81a0a50 con 0x7fccd8101ff0 2024-02-02T15:26:35.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.999+0000 7fccde800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd81a0c10 con 0x7fccd8100e00 2024-02-02T15:26:35.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.999+0000 7fccd7400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fccd8100e00 0x7fccd8198da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:35.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.999+0000 7fccd6a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fccd8101ff0 0x7fccd81992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:35.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.999+0000 7fccd7e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fccd8102990 0x7fccd81a0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:35.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.999+0000 7fccd7400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fccd8100e00 0x7fccd8198da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:42134/0 (socket says 172.21.15.1:42134) 2024-02-02T15:26:35.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.999+0000 7fccd7400700 1 -- 172.21.15.1:0/437177588 learned_addr learned my addr 172.21.15.1:0/437177588 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:35.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.999+0000 7fccd6a00700 1 -- 172.21.15.1:0/437177588 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fccd8100e00 msgr2=0x7fccd8198da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:35.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.999+0000 7fccd6a00700 1 --2- 172.21.15.1:0/437177588 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fccd8100e00 0x7fccd8198da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:35.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.999+0000 7fccd6a00700 1 -- 172.21.15.1:0/437177588 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fccd8102990 msgr2=0x7fccd81a0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:35.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.999+0000 7fccd6a00700 1 --2- 172.21.15.1:0/437177588 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fccd8102990 0x7fccd81a0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:35.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.999+0000 7fccd6a00700 1 -- 172.21.15.1:0/437177588 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccc0009520 con 0x7fccd8101ff0 2024-02-02T15:26:35.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:34.999+0000 7fccd7e00700 1 --2- 172.21.15.1:0/437177588 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fccd8102990 0x7fccd81a0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:26:35.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.000+0000 7fccd6a00700 1 --2- 172.21.15.1:0/437177588 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fccd8101ff0 0x7fccd81992e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fccc000e9f0 tx=0x7fccc000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:35.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.000+0000 7fcccfe00700 1 -- 172.21.15.1:0/437177588 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc000cb70 con 0x7fccd8101ff0 2024-02-02T15:26:35.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.000+0000 7fcccfe00700 1 -- 172.21.15.1:0/437177588 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fccc000ee90 con 0x7fccd8101ff0 2024-02-02T15:26:35.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.000+0000 7fcccfe00700 1 -- 172.21.15.1:0/437177588 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc0003cf0 con 0x7fccd8101ff0 2024-02-02T15:26:35.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.000+0000 7fccde800700 1 -- 172.21.15.1:0/437177588 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccc80095f0 con 0x7fccd8101ff0 2024-02-02T15:26:35.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.000+0000 7fccde800700 1 -- 172.21.15.1:0/437177588 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccd81a11c0 con 0x7fccd8101ff0 2024-02-02T15:26:35.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.003+0000 7fcccfe00700 1 -- 172.21.15.1:0/437177588 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 135069+0+0 (secure 0 0 0) 0x7fccc000ccd0 con 0x7fccd8101ff0 2024-02-02T15:26:35.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.004+0000 7fccf2229b80 1 Processor -- start 2024-02-02T15:26:35.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.004+0000 7fccf2229b80 1 -- start start 2024-02-02T15:26:35.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.005+0000 7fccf2229b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56473a01e130 0x56473a01e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:35.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.005+0000 7fccf2229b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56473a01ea90 0x56473a01f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:35.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.005+0000 7fcccfe00700 1 --2- 172.21.15.1:0/437177588 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fccb80a0520 0x7fccb80a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:35.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.005+0000 7fccf2229b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56473a01f930 0x56473a01fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:35.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.005+0000 7fccf2229b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x564739f91470 con 0x56473a01e130 2024-02-02T15:26:35.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.005+0000 7fccf2229b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x564739f75a50 con 0x56473a01ea90 2024-02-02T15:26:35.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.005+0000 7fccf2229b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564739f73410 con 0x56473a01f930 2024-02-02T15:26:35.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.005+0000 7fcccfe00700 1 -- 172.21.15.1:0/437177588 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fccc0014070 con 0x7fccd8101ff0 2024-02-02T15:26:35.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.005+0000 7fccd7e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56473a01e130 0x56473a01e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:35.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.005+0000 7fccd7400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56473a01ea90 0x56473a01f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:35.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.005+0000 7fccd7e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56473a01e130 0x56473a01e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52908/0 (socket says 172.21.15.1:52908) 2024-02-02T15:26:35.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.005+0000 7fccd7e00700 1 -- 172.21.15.1:0/1039010724 learned_addr learned my addr 172.21.15.1:0/1039010724 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:35.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.005+0000 7fccd6a00700 1 --2- 172.21.15.1:0/437177588 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fccb80a0520 0x7fccb80a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:35.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.005+0000 7fccd7e00700 1 --2- 172.21.15.1:0/1039010724 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56473a01f930 0x56473a01fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:35.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.005+0000 7fccd7400700 1 -- 172.21.15.1:0/1039010724 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56473a01f930 msgr2=0x56473a01fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:35.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.005+0000 7fccd7400700 1 --2- 172.21.15.1:0/1039010724 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56473a01f930 0x56473a01fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:35.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.005+0000 7fccd7400700 1 -- 172.21.15.1:0/1039010724 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56473a01e130 msgr2=0x56473a01e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:35.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.005+0000 7fccd7400700 1 --2- 172.21.15.1:0/1039010724 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56473a01e130 0x56473a01e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:35.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.005+0000 7fccd7400700 1 -- 172.21.15.1:0/1039010724 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564739df0360 con 0x56473a01ea90 2024-02-02T15:26:35.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.006+0000 7fccd7400700 1 --2- 172.21.15.1:0/1039010724 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56473a01ea90 0x56473a01f3a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fccc800b580 tx=0x7fccc8005fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9da539311e094a0f server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:35.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.006+0000 7fccd6a00700 1 --2- 172.21.15.1:0/437177588 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fccb80a0520 0x7fccb80a29e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fccc0000c00 tx=0x7fccc0003680 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:35.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.006+0000 7fccccc00700 1 -- 172.21.15.1:0/1039010724 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc800b990 con 0x56473a01ea90 2024-02-02T15:26:35.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.006+0000 7fccccc00700 1 -- 172.21.15.1:0/1039010724 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fccc801e040 con 0x56473a01ea90 2024-02-02T15:26:35.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.006+0000 7fccccc00700 1 -- 172.21.15.1:0/1039010724 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc800fe60 con 0x56473a01ea90 2024-02-02T15:26:35.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.007+0000 7fccf2229b80 1 -- 172.21.15.1:0/1039010724 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56473a01ea90 msgr2=0x56473a01f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:35.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.007+0000 7fccf2229b80 1 --2- 172.21.15.1:0/1039010724 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56473a01ea90 0x56473a01f3a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fccc800b580 tx=0x7fccc8005fb0 comp rx=0 tx=0).stop 2024-02-02T15:26:35.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.007+0000 7fccf2229b80 1 -- 172.21.15.1:0/1039010724 shutdown_connections 2024-02-02T15:26:35.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.007+0000 7fccf2229b80 1 --2- 172.21.15.1:0/1039010724 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56473a01f930 0x56473a01fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:35.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.007+0000 7fccf2229b80 1 --2- 172.21.15.1:0/1039010724 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56473a01e130 0x56473a01e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:35.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.007+0000 7fccf2229b80 1 --2- 172.21.15.1:0/1039010724 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56473a01ea90 0x56473a01f3a0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:35.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.007+0000 7fccf2229b80 1 -- 172.21.15.1:0/1039010724 >> 172.21.15.1:0/1039010724 conn(0x564739f7cd60 msgr2=0x56473a018e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:35.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.007+0000 7fccf2229b80 1 -- 172.21.15.1:0/1039010724 shutdown_connections 2024-02-02T15:26:35.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.007+0000 7fccf2229b80 1 -- 172.21.15.1:0/1039010724 wait complete. 2024-02-02T15:26:35.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.007+0000 7fccf2229b80 1 -- start start 2024-02-02T15:26:35.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.008+0000 7fccf2229b80 1 Processor -- start 2024-02-02T15:26:35.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.008+0000 7fccf2229b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56473a01e130 0x56473a027e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:35.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.008+0000 7fccf2229b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56473a01ea90 0x56473a0283a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:35.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.008+0000 7fccf2229b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56473a01f930 0x56473a02f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:35.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.008+0000 7fccf2229b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x564739f6ed00 con 0x56473a01f930 2024-02-02T15:26:35.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.008+0000 7fccf2229b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x564739f0ad30 con 0x56473a01e130 2024-02-02T15:26:35.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.008+0000 7fccf2229b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564739f0a9d0 con 0x56473a01ea90 2024-02-02T15:26:35.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.008+0000 7fccf2229b80 10 client.0 _ll_register_callbacks cb 0x564739f0c240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:26:35.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.008+0000 7fccd7e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56473a01e130 0x56473a027e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:35.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.008+0000 7fccd7e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56473a01e130 0x56473a027e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:57858/0 (socket says 172.21.15.1:57858) 2024-02-02T15:26:35.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.008+0000 7fccd7e00700 1 -- 172.21.15.1:0/105754035 learned_addr learned my addr 172.21.15.1:0/105754035 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:35.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.008+0000 7fccd7e00700 1 --2- 172.21.15.1:0/105754035 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56473a01f930 0x56473a02f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:35.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.008+0000 7fccd7400700 1 --2- 172.21.15.1:0/105754035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56473a01ea90 0x56473a0283a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:35.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.009+0000 7fccd7e00700 1 -- 172.21.15.1:0/105754035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56473a01ea90 msgr2=0x56473a0283a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:35.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.009+0000 7fccd7e00700 1 --2- 172.21.15.1:0/105754035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56473a01ea90 0x56473a0283a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:35.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.009+0000 7fccd7e00700 1 -- 172.21.15.1:0/105754035 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56473a01f930 msgr2=0x56473a02f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:35.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.009+0000 7fccd7e00700 1 --2- 172.21.15.1:0/105754035 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56473a01f930 0x56473a02f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:35.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.009+0000 7fccd7e00700 1 -- 172.21.15.1:0/105754035 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccd8109b90 con 0x56473a01e130 2024-02-02T15:26:35.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.009+0000 7fccd7e00700 1 --2- 172.21.15.1:0/105754035 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56473a01e130 0x56473a027e60 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fccd0004a30 tx=0x7fccd000b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efc08ccdd5b9e59c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:35.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.009+0000 7fccc5600700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:26:35.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.009+0000 7fccc5600700 1 -- 172.21.15.1:0/105754035 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd0009310 con 0x56473a01e130 2024-02-02T15:26:35.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.009+0000 7fccf2229b80 1 -- 172.21.15.1:0/105754035 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56473a0110a0 con 0x56473a01e130 2024-02-02T15:26:35.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.009+0000 7fccc5600700 1 -- 172.21.15.1:0/105754035 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fccd0022030 con 0x56473a01e130 2024-02-02T15:26:35.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.010+0000 7fccc5600700 1 -- 172.21.15.1:0/105754035 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fccd001a660 con 0x56473a01e130 2024-02-02T15:26:35.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.010+0000 7fccf2229b80 10 client.25403 fetch_fsmap learned FSMap version 27 2024-02-02T15:26:35.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.010+0000 7fccf2229b80 1 -- 172.21.15.1:0/105754035 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x564739d9cb10 con 0x56473a01e130 2024-02-02T15:26:35.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.010+0000 7fccc5600700 1 -- 172.21.15.1:0/105754035 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56473a0110a0 con 0x56473a01e130 2024-02-02T15:26:35.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.010+0000 7fccc5600700 1 -- 172.21.15.1:0/105754035 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fccd0007690 con 0x56473a01e130 2024-02-02T15:26:35.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.010+0000 7fccf2229b80 10 client.25403 fetch_fsmap finished waiting for FSMap version 27 2024-02-02T15:26:35.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.010+0000 7fccf2229b80 10 client.25403 resolve_mds: resolved name 'b' to daemon mds.b 2024-02-02T15:26:35.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.010+0000 7fccf2229b80 20 client.25403 populate_metadata read hostname 'smithi001' 2024-02-02T15:26:35.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.010+0000 7fccf2229b80 1 --2- 172.21.15.1:0/105754035 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56473a033110 0x56473a032540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:35.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.010+0000 7fccf2229b80 4 client.25403 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:26:35.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.010+0000 7fccf2229b80 1 -- 172.21.15.1:0/105754035 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564739f67d80 con 0x56473a033110 2024-02-02T15:26:35.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.010+0000 7fccd7400700 1 --2- 172.21.15.1:0/105754035 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56473a033110 0x56473a032540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:35.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.011+0000 7fccd7400700 1 --2- 172.21.15.1:0/105754035 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56473a033110 0x56473a032540 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=218e691d12f4df1a server_cookie=202d1fd5e4561884 in_seq=0 out_seq=0 2024-02-02T15:26:35.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.011+0000 7fccc5600700 10 client.25403 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:26:35.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.012+0000 7fccc5600700 1 -- 172.21.15.1:0/105754035 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x564739f67d80 con 0x56473a033110 2024-02-02T15:26:35.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.012+0000 7fccc5600700 10 client.25403 handle_command_reply: tid=0 2024-02-02T15:26:35.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.012+0000 7fccf2229b80 1 client.25403 shutdown 2024-02-02T15:26:35.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.012+0000 7fccf2229b80 1 -- 172.21.15.1:0/105754035 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56473a01e130 msgr2=0x56473a027e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:35.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.012+0000 7fccf2229b80 1 --2- 172.21.15.1:0/105754035 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56473a01e130 0x56473a027e60 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fccd0004a30 tx=0x7fccd000b040 comp rx=0 tx=0).stop 2024-02-02T15:26:35.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.012+0000 7fccf2229b80 1 -- 172.21.15.1:0/105754035 shutdown_connections 2024-02-02T15:26:35.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.012+0000 7fccf2229b80 1 --2- 172.21.15.1:0/105754035 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56473a033110 0x56473a032540 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:35.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.012+0000 7fccf2229b80 1 --2- 172.21.15.1:0/105754035 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56473a01f930 0x56473a02f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:35.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.012+0000 7fccf2229b80 1 --2- 172.21.15.1:0/105754035 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56473a01e130 0x56473a027e60 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:35.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.012+0000 7fccf2229b80 1 --2- 172.21.15.1:0/105754035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56473a01ea90 0x56473a0283a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:35.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.012+0000 7fccf2229b80 1 -- 172.21.15.1:0/105754035 >> 172.21.15.1:0/105754035 conn(0x564739f7cd60 msgr2=0x56473a018570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:35.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.012+0000 7fccc5600700 0 client.25403 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:26:35.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.012+0000 7fccf2229b80 1 -- 172.21.15.1:0/105754035 shutdown_connections 2024-02-02T15:26:35.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.012+0000 7fccf2229b80 1 -- 172.21.15.1:0/105754035 wait complete. 2024-02-02T15:26:35.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.012+0000 7fccf2229b80 20 client.25403 trim_cache size 0 max 16384 2024-02-02T15:26:35.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.030+0000 7fccf2229b80 1 Processor -- start 2024-02-02T15:26:35.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.030+0000 7fccf2229b80 1 -- start start 2024-02-02T15:26:35.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.031+0000 7fccf2229b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56473a01e130 0x56473a025cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:35.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.031+0000 7fccf2229b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56473a01ea90 0x56473a03be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:35.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.031+0000 7fccf2229b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56473a01f930 0x56473a022800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:35.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.031+0000 7fccf2229b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x564739df2650 con 0x56473a01e130 2024-02-02T15:26:35.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.031+0000 7fccf2229b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x564739f67db0 con 0x56473a01f930 2024-02-02T15:26:35.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.031+0000 7fccf2229b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564739dfeb60 con 0x56473a01ea90 2024-02-02T15:26:35.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.031+0000 7fccd7e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56473a01e130 0x56473a025cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:35.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.031+0000 7fccd7e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56473a01e130 0x56473a025cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52922/0 (socket says 172.21.15.1:52922) 2024-02-02T15:26:35.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.031+0000 7fccd7e00700 1 -- 172.21.15.1:0/1861289861 learned_addr learned my addr 172.21.15.1:0/1861289861 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:35.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.031+0000 7fccd7400700 1 --2- 172.21.15.1:0/1861289861 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56473a01ea90 0x56473a03be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:35.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.031+0000 7fccd7e00700 1 --2- 172.21.15.1:0/1861289861 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56473a01f930 0x56473a022800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:35.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.031+0000 7fccd7e00700 1 -- 172.21.15.1:0/1861289861 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56473a01ea90 msgr2=0x56473a03be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:35.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.031+0000 7fccd7e00700 1 --2- 172.21.15.1:0/1861289861 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56473a01ea90 0x56473a03be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:35.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.031+0000 7fccd7e00700 1 -- 172.21.15.1:0/1861289861 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56473a01f930 msgr2=0x56473a022800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:35.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.031+0000 7fccd7e00700 1 --2- 172.21.15.1:0/1861289861 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56473a01f930 0x56473a022800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:35.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.031+0000 7fccd7e00700 1 -- 172.21.15.1:0/1861289861 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564739f34e50 con 0x56473a01e130 2024-02-02T15:26:35.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.032+0000 7fccd7e00700 1 --2- 172.21.15.1:0/1861289861 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56473a01e130 0x56473a025cf0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fccd0003ea0 tx=0x7fccd0009ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29ce15da8ac1ced7 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:35.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.032+0000 7fcccd600700 1 -- 172.21.15.1:0/1861289861 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd00044f0 con 0x56473a01e130 2024-02-02T15:26:35.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.032+0000 7fcccd600700 1 -- 172.21.15.1:0/1861289861 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fccd0021020 con 0x56473a01e130 2024-02-02T15:26:35.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.032+0000 7fcccd600700 1 -- 172.21.15.1:0/1861289861 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd0004e80 con 0x56473a01e130 2024-02-02T15:26:35.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.032+0000 7fccf2229b80 1 -- 172.21.15.1:0/1861289861 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56473a01e130 msgr2=0x56473a025cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:35.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.032+0000 7fccf2229b80 1 --2- 172.21.15.1:0/1861289861 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56473a01e130 0x56473a025cf0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fccd0003ea0 tx=0x7fccd0009ac0 comp rx=0 tx=0).stop 2024-02-02T15:26:35.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.032+0000 7fccf2229b80 1 -- 172.21.15.1:0/1861289861 shutdown_connections 2024-02-02T15:26:35.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.032+0000 7fccf2229b80 1 --2- 172.21.15.1:0/1861289861 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56473a01f930 0x56473a022800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:35.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.033+0000 7fccf2229b80 1 --2- 172.21.15.1:0/1861289861 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56473a01e130 0x56473a025cf0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:35.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.033+0000 7fccf2229b80 1 --2- 172.21.15.1:0/1861289861 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56473a01ea90 0x56473a03be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:35.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.033+0000 7fccf2229b80 1 -- 172.21.15.1:0/1861289861 >> 172.21.15.1:0/1861289861 conn(0x564739f7cd60 msgr2=0x56473a032540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:35.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.033+0000 7fccf2229b80 1 -- 172.21.15.1:0/1861289861 shutdown_connections 2024-02-02T15:26:35.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.033+0000 7fccf2229b80 1 -- 172.21.15.1:0/1861289861 wait complete. 2024-02-02T15:26:35.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.033+0000 7fccf2229b80 1 -- start start 2024-02-02T15:26:35.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.033+0000 7fccf2229b80 1 Processor -- start 2024-02-02T15:26:35.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.034+0000 7fccf2229b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56473a01e130 0x56473a036ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:35.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.034+0000 7fccf2229b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56473a01ea90 0x56473a037220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:35.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.034+0000 7fccf2229b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56473a01f930 0x56473a084be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:35.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.034+0000 7fccf2229b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56473a037900 con 0x56473a01ea90 2024-02-02T15:26:35.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.034+0000 7fccf2229b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56473a085120 con 0x56473a01f930 2024-02-02T15:26:35.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.034+0000 7fccf2229b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56473a0852b0 con 0x56473a01e130 2024-02-02T15:26:35.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.034+0000 7fccf2229b80 10 client.0 _ll_register_callbacks cb 0x564739f0c240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:26:35.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.034+0000 7fccd7400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56473a01ea90 0x56473a037220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:35.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.034+0000 7fccd7e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56473a01e130 0x56473a036ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:35.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.034+0000 7fccd7400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56473a01ea90 0x56473a037220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52930/0 (socket says 172.21.15.1:52930) 2024-02-02T15:26:35.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.034+0000 7fccd7400700 1 -- 172.21.15.1:0/2582812891 learned_addr learned my addr 172.21.15.1:0/2582812891 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:35.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.034+0000 7fccd7e00700 1 --2- 172.21.15.1:0/2582812891 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56473a01f930 0x56473a084be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:35.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.034+0000 7fccd7400700 1 -- 172.21.15.1:0/2582812891 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56473a01e130 msgr2=0x56473a036ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:35.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.034+0000 7fccd7400700 1 --2- 172.21.15.1:0/2582812891 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56473a01e130 0x56473a036ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:35.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.034+0000 7fccd7400700 1 -- 172.21.15.1:0/2582812891 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56473a01f930 msgr2=0x56473a084be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:35.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.034+0000 7fccd7400700 1 --2- 172.21.15.1:0/2582812891 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56473a01f930 0x56473a084be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:35.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.034+0000 7fccd7400700 1 -- 172.21.15.1:0/2582812891 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564739d9cb10 con 0x56473a01ea90 2024-02-02T15:26:35.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.034+0000 7fccd7e00700 1 --2- 172.21.15.1:0/2582812891 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56473a01f930 0x56473a084be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:26:35.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.035+0000 7fccd7400700 1 --2- 172.21.15.1:0/2582812891 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56473a01ea90 0x56473a037220 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fccc8009960 tx=0x7fccc8009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=239d99ddf14b7a5b server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:35.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.035+0000 7fccc5600700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:26:35.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.035+0000 7fccc5600700 1 -- 172.21.15.1:0/2582812891 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc8034070 con 0x56473a01ea90 2024-02-02T15:26:35.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.035+0000 7fccc5600700 1 -- 172.21.15.1:0/2582812891 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fccc800fb00 con 0x56473a01ea90 2024-02-02T15:26:35.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.035+0000 7fccf2229b80 1 -- 172.21.15.1:0/2582812891 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fccc801e040 con 0x56473a01ea90 2024-02-02T15:26:35.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.035+0000 7fccc5600700 1 -- 172.21.15.1:0/2582812891 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fccc800fcc0 con 0x56473a01ea90 2024-02-02T15:26:35.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.035+0000 7fccf2229b80 10 client.15381 fetch_fsmap learned FSMap version 27 2024-02-02T15:26:35.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.035+0000 7fccf2229b80 1 -- 172.21.15.1:0/2582812891 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x564739df4c80 con 0x56473a01ea90 2024-02-02T15:26:35.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.035+0000 7fccc5600700 1 -- 172.21.15.1:0/2582812891 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fccc801e040 con 0x56473a01ea90 2024-02-02T15:26:35.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.035+0000 7fccc5600700 1 -- 172.21.15.1:0/2582812891 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fccc800b260 con 0x56473a01ea90 2024-02-02T15:26:35.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.035+0000 7fccf2229b80 10 client.15381 fetch_fsmap finished waiting for FSMap version 27 2024-02-02T15:26:35.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.036+0000 7fccf2229b80 10 client.15381 resolve_mds: resolved name 'b' to daemon mds.b 2024-02-02T15:26:35.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.036+0000 7fccf2229b80 20 client.15381 populate_metadata read hostname 'smithi001' 2024-02-02T15:26:35.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.036+0000 7fccf2229b80 1 --2- 172.21.15.1:0/2582812891 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56473a089c20 0x56473a086580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:35.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.036+0000 7fccf2229b80 4 client.15381 mds_command: new command op to 24557 tid=0[{"prefix": "status"}] 2024-02-02T15:26:35.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.036+0000 7fccf2229b80 1 -- 172.21.15.1:0/2582812891 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "status"}) v1 -- 0x564739f2df00 con 0x56473a089c20 2024-02-02T15:26:35.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.036+0000 7fccd7e00700 1 --2- 172.21.15.1:0/2582812891 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56473a089c20 0x56473a086580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:35.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.036+0000 7fccd7e00700 1 --2- 172.21.15.1:0/2582812891 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56473a089c20 0x56473a086580 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5ea505bfcdd1662 server_cookie=62499ea71578fe56 in_seq=0 out_seq=0 2024-02-02T15:26:35.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.036+0000 7fccc5600700 10 client.15381 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:26:35.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.037+0000 7fccc5600700 1 -- 172.21.15.1:0/2582812891 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x564739f2df00 con 0x56473a089c20 2024-02-02T15:26:35.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.037+0000 7fccc5600700 10 client.15381 handle_command_reply: tid=0 2024-02-02T15:26:35.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.037+0000 7fccf2229b80 1 client.15381 shutdown 2024-02-02T15:26:35.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.037+0000 7fccf2229b80 1 -- 172.21.15.1:0/2582812891 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56473a01ea90 msgr2=0x56473a037220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:35.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.037+0000 7fccf2229b80 1 --2- 172.21.15.1:0/2582812891 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56473a01ea90 0x56473a037220 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fccc8009960 tx=0x7fccc8009cd0 comp rx=0 tx=0).stop 2024-02-02T15:26:35.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.037+0000 7fccf2229b80 1 -- 172.21.15.1:0/2582812891 shutdown_connections 2024-02-02T15:26:35.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.037+0000 7fccf2229b80 1 --2- 172.21.15.1:0/2582812891 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56473a089c20 0x56473a086580 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:35.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.037+0000 7fccf2229b80 1 --2- 172.21.15.1:0/2582812891 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56473a01f930 0x56473a084be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:35.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.037+0000 7fccc5600700 0 client.15381 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:26:35.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.037+0000 7fccf2229b80 1 --2- 172.21.15.1:0/2582812891 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56473a01e130 0x56473a036ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:35.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.037+0000 7fccf2229b80 1 --2- 172.21.15.1:0/2582812891 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56473a01ea90 0x56473a037220 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:35.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.037+0000 7fccf2229b80 1 -- 172.21.15.1:0/2582812891 >> 172.21.15.1:0/2582812891 conn(0x564739f7cd60 msgr2=0x56473a025cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:35.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.037+0000 7fccf2229b80 1 -- 172.21.15.1:0/2582812891 shutdown_connections 2024-02-02T15:26:35.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.037+0000 7fccf2229b80 1 -- 172.21.15.1:0/2582812891 wait complete. 2024-02-02T15:26:35.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.037+0000 7fccf2229b80 20 client.15381 trim_cache size 0 max 16384 2024-02-02T15:26:35.043 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:26:35.043 INFO:teuthology.orchestra.run.smithi001.stdout: "cluster_fsid": "371a409c-c1de-11ee-95b6-87774f69a715", 2024-02-02T15:26:35.043 INFO:teuthology.orchestra.run.smithi001.stdout: "whoami": 0, 2024-02-02T15:26:35.043 INFO:teuthology.orchestra.run.smithi001.stdout: "id": 24557, 2024-02-02T15:26:35.043 INFO:teuthology.orchestra.run.smithi001.stdout: "want_state": "up:active", 2024-02-02T15:26:35.043 INFO:teuthology.orchestra.run.smithi001.stdout: "state": "up:active", 2024-02-02T15:26:35.043 INFO:teuthology.orchestra.run.smithi001.stdout: "fs_name": "cephfs", 2024-02-02T15:26:35.044 INFO:teuthology.orchestra.run.smithi001.stdout: "rank_uptime": 107.27184187100001, 2024-02-02T15:26:35.044 INFO:teuthology.orchestra.run.smithi001.stdout: "mdsmap_epoch": 27, 2024-02-02T15:26:35.044 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch": 88, 2024-02-02T15:26:35.044 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_epoch_barrier": 84, 2024-02-02T15:26:35.044 INFO:teuthology.orchestra.run.smithi001.stdout: "uptime": 113.602539249 2024-02-02T15:26:35.044 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:26:35.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.038+0000 7fccde800700 1 -- 172.21.15.1:0/437177588 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fccb80a0520 msgr2=0x7fccb80a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:35.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.038+0000 7fccde800700 1 --2- 172.21.15.1:0/437177588 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fccb80a0520 0x7fccb80a29e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fccc0000c00 tx=0x7fccc0003680 comp rx=0 tx=0).stop 2024-02-02T15:26:35.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.038+0000 7fccde800700 1 -- 172.21.15.1:0/437177588 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fccd8101ff0 msgr2=0x7fccd81992e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:35.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.038+0000 7fccde800700 1 --2- 172.21.15.1:0/437177588 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fccd8101ff0 0x7fccd81992e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fccc000e9f0 tx=0x7fccc000ed00 comp rx=0 tx=0).stop 2024-02-02T15:26:35.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.038+0000 7fccde800700 1 -- 172.21.15.1:0/437177588 shutdown_connections 2024-02-02T15:26:35.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.038+0000 7fccde800700 1 --2- 172.21.15.1:0/437177588 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fccd8102990 0x7fccd81a0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:35.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.038+0000 7fccde800700 1 --2- 172.21.15.1:0/437177588 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fccb80a0520 0x7fccb80a29e0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:35.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.038+0000 7fccde800700 1 --2- 172.21.15.1:0/437177588 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fccd8100e00 0x7fccd8198da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:35.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.038+0000 7fccde800700 1 --2- 172.21.15.1:0/437177588 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fccd8101ff0 0x7fccd81992e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:35.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.038+0000 7fccde800700 1 -- 172.21.15.1:0/437177588 >> 172.21.15.1:0/437177588 conn(0x7fccd80fc370 msgr2=0x7fccd8107920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:35.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.038+0000 7fccde800700 1 -- 172.21.15.1:0/437177588 shutdown_connections 2024-02-02T15:26:35.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:35.038+0000 7fccde800700 1 -- 172.21.15.1:0/437177588 wait complete. 2024-02-02T15:26:35.057 INFO:teuthology.run_tasks:Running task kclient... 2024-02-02T15:26:35.071 INFO:tasks.kclient:Mounting kernel clients... 2024-02-02T15:26:35.071 INFO:tasks.kclient:config is {'client.0': {}} 2024-02-02T15:26:35.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T15:26:35.072 DEBUG:teuthology.orchestra.run.smithi001:> ip netns list 2024-02-02T15:26:35.114 INFO:teuthology.orchestra.run.smithi001.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-02-02T15:26:35.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T15:26:35.114 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-02-02T15:26:35.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T15:26:35.221 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link delete ceph-brx 2024-02-02T15:26:35.336 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'wsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-02-02T15:26:35.336 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-02-02T15:26:35.336 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-02-02T15:26:35.337 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph fs ls 2024-02-02T15:26:35.709 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.a/config 2024-02-02T15:26:36.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:36.901+0000 7faeb9e00700 1 -- 172.21.15.1:0/3658950688 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faeb41060a0 msgr2=0x7faeb410dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:36.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:36.901+0000 7faeb9e00700 1 --2- 172.21.15.1:0/3658950688 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faeb41060a0 0x7faeb410dc70 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7faeac00b0a0 tx=0x7faeac00b3b0 comp rx=0 tx=0).stop 2024-02-02T15:26:36.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:36.901+0000 7faeb9e00700 1 -- 172.21.15.1:0/3658950688 shutdown_connections 2024-02-02T15:26:36.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:36.901+0000 7faeb9e00700 1 --2- 172.21.15.1:0/3658950688 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faeb41060a0 0x7faeb410dc70 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:36.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:36.901+0000 7faeb9e00700 1 --2- 172.21.15.1:0/3658950688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faeb4100fc0 0x7faeb41013a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:36.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:36.901+0000 7faeb9e00700 1 --2- 172.21.15.1:0/3658950688 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faeb4101970 0x7faeb41059c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:36.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:36.901+0000 7faeb9e00700 1 -- 172.21.15.1:0/3658950688 >> 172.21.15.1:0/3658950688 conn(0x7faeb40fc830 msgr2=0x7faeb40fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:36.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:36.901+0000 7faeb9e00700 1 -- 172.21.15.1:0/3658950688 shutdown_connections 2024-02-02T15:26:36.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:36.901+0000 7faeb9e00700 1 -- 172.21.15.1:0/3658950688 wait complete. 2024-02-02T15:26:36.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:36.902+0000 7faeb9e00700 1 Processor -- start 2024-02-02T15:26:36.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:36.902+0000 7faeb9e00700 1 -- start start 2024-02-02T15:26:36.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:36.903+0000 7faeb9e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faeb4100fc0 0x7faeb4199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:36.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:36.903+0000 7faeb9e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faeb4101970 0x7faeb41996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:36.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:36.903+0000 7faeb9e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faeb41060a0 0x7faeb419db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:36.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:36.903+0000 7faeb9e00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeb419e130 con 0x7faeb4101970 2024-02-02T15:26:36.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:36.903+0000 7faeb9e00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeb419e2a0 con 0x7faeb4100fc0 2024-02-02T15:26:36.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:36.903+0000 7faeb9e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeb419e460 con 0x7faeb41060a0 2024-02-02T15:26:36.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:36.903+0000 7faeb2a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faeb4100fc0 0x7faeb4199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:36.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:36.903+0000 7faeabe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faeb4101970 0x7faeb41996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:36.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:36.903+0000 7faeb2a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faeb4100fc0 0x7faeb4199170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:57902/0 (socket says 172.21.15.1:57902) 2024-02-02T15:26:36.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:36.903+0000 7faeb2a00700 1 -- 172.21.15.1:0/3081026214 learned_addr learned my addr 172.21.15.1:0/3081026214 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:36.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:36.903+0000 7faeb3400700 1 --2- 172.21.15.1:0/3081026214 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faeb41060a0 0x7faeb419db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:36.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:36.904+0000 7faeb2a00700 1 -- 172.21.15.1:0/3081026214 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faeb41060a0 msgr2=0x7faeb419db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:36.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:36.904+0000 7faeb2a00700 1 --2- 172.21.15.1:0/3081026214 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faeb41060a0 0x7faeb419db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:36.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:36.904+0000 7faeb2a00700 1 -- 172.21.15.1:0/3081026214 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faeb4101970 msgr2=0x7faeb41996b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:36.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:36.904+0000 7faeb2a00700 1 --2- 172.21.15.1:0/3081026214 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faeb4101970 0x7faeb41996b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:36.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:36.904+0000 7faeb2a00700 1 -- 172.21.15.1:0/3081026214 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeac009d00 con 0x7faeb4100fc0 2024-02-02T15:26:36.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:36.904+0000 7faeb2a00700 1 --2- 172.21.15.1:0/3081026214 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faeb4100fc0 0x7faeb4199170 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7faea400e9a0 tx=0x7faea400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:36.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:36.904+0000 7faeb8a00700 1 -- 172.21.15.1:0/3081026214 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea4009790 con 0x7faeb4100fc0 2024-02-02T15:26:36.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:36.904+0000 7faeb9e00700 1 -- 172.21.15.1:0/3081026214 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faeb419e740 con 0x7faeb4100fc0 2024-02-02T15:26:36.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:36.904+0000 7faeb8a00700 1 -- 172.21.15.1:0/3081026214 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faea4010430 con 0x7faeb4100fc0 2024-02-02T15:26:36.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:36.904+0000 7faeb8a00700 1 -- 172.21.15.1:0/3081026214 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea4018680 con 0x7faeb4100fc0 2024-02-02T15:26:36.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:36.904+0000 7faeb9e00700 1 -- 172.21.15.1:0/3081026214 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faeb41a5fa0 con 0x7faeb4100fc0 2024-02-02T15:26:36.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:36.905+0000 7faeb9e00700 1 -- 172.21.15.1:0/3081026214 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faeb404ea40 con 0x7faeb4100fc0 2024-02-02T15:26:36.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:36.907+0000 7faeb8a00700 1 -- 172.21.15.1:0/3081026214 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 135069+0+0 (secure 0 0 0) 0x7faea40105a0 con 0x7faeb4100fc0 2024-02-02T15:26:36.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:36.909+0000 7faeb8a00700 1 --2- 172.21.15.1:0/3081026214 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fae940a0520 0x7fae940a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:36.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:36.909+0000 7faeb8a00700 1 -- 172.21.15.1:0/3081026214 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9107+0+0 (secure 0 0 0) 0x7faea4014070 con 0x7faeb4100fc0 2024-02-02T15:26:36.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:36.909+0000 7faeabe00700 1 --2- 172.21.15.1:0/3081026214 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fae940a0520 0x7fae940a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:36.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:36.910+0000 7faeabe00700 1 --2- 172.21.15.1:0/3081026214 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fae940a0520 0x7fae940a29e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7faeb419a790 tx=0x7fae9c009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:36.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:36.911+0000 7faeb8a00700 1 -- 172.21.15.1:0/3081026214 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7faea40f9110 con 0x7faeb4100fc0 2024-02-02T15:26:37.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:37.164+0000 7faeb9e00700 1 -- 172.21.15.1:0/3081026214 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7faeb419a4d0 con 0x7faeb4100fc0 2024-02-02T15:26:37.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:37.165+0000 7faeb8a00700 1 -- 172.21.15.1:0/3081026214 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) v1 ==== 53+0+88 (secure 0 0 0) 0x7faea40958c0 con 0x7faeb4100fc0 2024-02-02T15:26:37.165 INFO:teuthology.orchestra.run.smithi001.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-02-02T15:26:37.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:37.167+0000 7faeb9e00700 1 -- 172.21.15.1:0/3081026214 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fae940a0520 msgr2=0x7fae940a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:37.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:37.167+0000 7faeb9e00700 1 --2- 172.21.15.1:0/3081026214 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fae940a0520 0x7fae940a29e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7faeb419a790 tx=0x7fae9c009260 comp rx=0 tx=0).stop 2024-02-02T15:26:37.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:37.167+0000 7faeb9e00700 1 -- 172.21.15.1:0/3081026214 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faeb4100fc0 msgr2=0x7faeb4199170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:37.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:37.167+0000 7faeb9e00700 1 --2- 172.21.15.1:0/3081026214 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faeb4100fc0 0x7faeb4199170 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7faea400e9a0 tx=0x7faea400ecb0 comp rx=0 tx=0).stop 2024-02-02T15:26:37.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:37.167+0000 7faeb9e00700 1 -- 172.21.15.1:0/3081026214 shutdown_connections 2024-02-02T15:26:37.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:37.167+0000 7faeb9e00700 1 --2- 172.21.15.1:0/3081026214 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fae940a0520 0x7fae940a29e0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:37.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:37.167+0000 7faeb9e00700 1 --2- 172.21.15.1:0/3081026214 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faeb41060a0 0x7faeb419db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:37.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:37.167+0000 7faeb9e00700 1 --2- 172.21.15.1:0/3081026214 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faeb4100fc0 0x7faeb4199170 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:37.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:37.167+0000 7faeb9e00700 1 --2- 172.21.15.1:0/3081026214 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faeb4101970 0x7faeb41996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:37.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:37.167+0000 7faeb9e00700 1 -- 172.21.15.1:0/3081026214 >> 172.21.15.1:0/3081026214 conn(0x7faeb40fc830 msgr2=0x7faeb40fd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:37.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:37.167+0000 7faeb9e00700 1 -- 172.21.15.1:0/3081026214 shutdown_connections 2024-02-02T15:26:37.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:37.167+0000 7faeb9e00700 1 -- 172.21.15.1:0/3081026214 wait complete. 2024-02-02T15:26:37.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:37 smithi001 ceph-mon[35470]: pgmap v293: 129 pgs: 129 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 236 B/s wr, 8 op/s 2024-02-02T15:26:37.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:37 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/3081026214' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-02-02T15:26:37.566 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-02-02T15:26:37.566 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-02-02T15:26:37.566 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi001.front.sepia.ceph.com 2024-02-02T15:26:37.566 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-02-02T15:26:37.566 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-02-02T15:26:37.566 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-02-02T15:26:37.566 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-02-02T15:26:37.567 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-02-02T15:26:37.567 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-02-02T15:26:37.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T15:26:37.567 DEBUG:teuthology.orchestra.run.smithi001:> ip addr 2024-02-02T15:26:37.610 INFO:teuthology.orchestra.run.smithi001.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-02-02T15:26:37.610 INFO:teuthology.orchestra.run.smithi001.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-02-02T15:26:37.610 INFO:teuthology.orchestra.run.smithi001.stdout: inet 127.0.0.1/8 scope host lo 2024-02-02T15:26:37.610 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-02-02T15:26:37.611 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 ::1/128 scope host 2024-02-02T15:26:37.611 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-02-02T15:26:37.611 INFO:teuthology.orchestra.run.smithi001.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-02T15:26:37.611 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:6b:f6:7a brd ff:ff:ff:ff:ff:ff 2024-02-02T15:26:37.611 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp7s0f0 2024-02-02T15:26:37.611 INFO:teuthology.orchestra.run.smithi001.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-02T15:26:37.611 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:6b:f6:7b brd ff:ff:ff:ff:ff:ff 2024-02-02T15:26:37.611 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp7s0f1 2024-02-02T15:26:37.611 INFO:teuthology.orchestra.run.smithi001.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-02-02T15:26:37.611 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:bd:15:e8 brd ff:ff:ff:ff:ff:ff 2024-02-02T15:26:37.611 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp1s0f0 2024-02-02T15:26:37.611 INFO:teuthology.orchestra.run.smithi001.stdout: inet 172.21.15.1/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-02-02T15:26:37.611 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft 41922sec preferred_lft 41922sec 2024-02-02T15:26:37.611 INFO:teuthology.orchestra.run.smithi001.stdout: inet6 fe80::ec4:7aff:febd:15e8/64 scope link 2024-02-02T15:26:37.611 INFO:teuthology.orchestra.run.smithi001.stdout: valid_lft forever preferred_lft forever 2024-02-02T15:26:37.612 INFO:teuthology.orchestra.run.smithi001.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-02-02T15:26:37.612 INFO:teuthology.orchestra.run.smithi001.stdout: link/ether 0c:c4:7a:bd:15:e9 brd ff:ff:ff:ff:ff:ff 2024-02-02T15:26:37.612 INFO:teuthology.orchestra.run.smithi001.stdout: altname enp1s0f1 2024-02-02T15:26:37.612 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-02-02T15:26:37.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T15:26:37.612 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec bash -c ' 2024-02-02T15:26:37.612 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2024-02-02T15:26:37.612 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link add name ceph-brx type bridge 2024-02-02T15:26:37.612 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip addr flush dev ceph-brx 2024-02-02T15:26:37.612 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link set ceph-brx up 2024-02-02T15:26:37.613 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-02-02T15:26:37.613 DEBUG:teuthology.orchestra.run.smithi001:> ') 2024-02-02T15:26:37.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:37 smithi071 ceph-mon[44707]: pgmap v293: 129 pgs: 129 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 236 B/s wr, 8 op/s 2024-02-02T15:26:37.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:37 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/3081026214' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-02-02T15:26:37.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:37 smithi142 ceph-mon[45728]: pgmap v293: 129 pgs: 129 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 236 B/s wr, 8 op/s 2024-02-02T15:26:37.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:37 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/3081026214' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-02-02T15:26:37.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T15:26:37.847 DEBUG:teuthology.orchestra.run.smithi001:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-02-02T15:26:37.914 INFO:teuthology.orchestra.run.smithi001.stdout:1 2024-02-02T15:26:37.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T15:26:37.918 DEBUG:teuthology.orchestra.run.smithi001:> route 2024-02-02T15:26:37.962 INFO:teuthology.orchestra.run.smithi001.stdout:Kernel IP routing table 2024-02-02T15:26:37.962 INFO:teuthology.orchestra.run.smithi001.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-02-02T15:26:37.962 INFO:teuthology.orchestra.run.smithi001.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-02-02T15:26:37.962 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-02-02T15:26:37.962 INFO:teuthology.orchestra.run.smithi001.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-02-02T15:26:37.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T15:26:37.963 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec bash -c ' 2024-02-02T15:26:37.963 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2024-02-02T15:26:37.963 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-02-02T15:26:37.963 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-02-02T15:26:37.963 DEBUG:teuthology.orchestra.run.smithi001:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-02-02T15:26:37.964 DEBUG:teuthology.orchestra.run.smithi001:> ') 2024-02-02T15:26:38.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T15:26:38.169 DEBUG:teuthology.orchestra.run.smithi001:> ip netns list 2024-02-02T15:26:38.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T15:26:38.211 DEBUG:teuthology.orchestra.run.smithi001:> ip netns list-id 2024-02-02T15:26:38.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T15:26:38.293 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec bash -c ' 2024-02-02T15:26:38.293 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2024-02-02T15:26:38.293 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-02T15:26:38.293 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-02-02T15:26:38.293 DEBUG:teuthology.orchestra.run.smithi001:> ') 2024-02-02T15:26:38.470 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-02-02T15:26:38.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T15:26:38.471 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec bash -c ' 2024-02-02T15:26:38.471 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2024-02-02T15:26:38.471 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-02-02T15:26:38.471 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-02-02T15:26:38.471 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-02-02T15:26:38.471 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-02-02T15:26:38.471 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-02-02T15:26:38.471 DEBUG:teuthology.orchestra.run.smithi001:> ') 2024-02-02T15:26:38.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T15:26:38.721 DEBUG:teuthology.orchestra.run.smithi001:> (cd / && exec bash -c ' 2024-02-02T15:26:38.721 DEBUG:teuthology.orchestra.run.smithi001:> set -e 2024-02-02T15:26:38.721 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link set brx.0 up 2024-02-02T15:26:38.721 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link set dev brx.0 master ceph-brx 2024-02-02T15:26:38.721 DEBUG:teuthology.orchestra.run.smithi001:> ') 2024-02-02T15:26:38.887 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-02T15:26:38.888 DEBUG:teuthology.orchestra.run.smithi001:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-02-02T15:26:38.929 INFO:teuthology.orchestra.run.smithi001.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-02-02T15:26:38.930 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-02-02T15:26:38.930 DEBUG:teuthology.orchestra.run.smithi001:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-02-02T15:26:39.010 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-02-02T15:26:39.011 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-02-02T15:26:39.011 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/_nogroup/sv_1/2234ea52-cd6c-42c8-816f-3ba5d3b919ec 2024-02-02T15:26:39.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T15:26:39.011 DEBUG:teuthology.orchestra.run.smithi001:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/_nogroup/sv_1/2234ea52-cd6c-42c8-816f-3ba5d3b919ec /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,wsync,nofallback 2024-02-02T15:26:39.151 INFO:teuthology.orchestra.run.smithi001.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,wsync,nofallback 2024-02-02T15:26:39.152 INFO:teuthology.orchestra.run.smithi001.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,wsync". 2024-02-02T15:26:39.409 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:39 smithi001 ceph-mon[35470]: pgmap v294: 129 pgs: 129 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-02-02T15:26:39.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:39 smithi071 ceph-mon[44707]: pgmap v294: 129 pgs: 129 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-02-02T15:26:39.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:39 smithi142 ceph-mon[45728]: pgmap v294: 129 pgs: 129 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-02-02T15:26:39.752 INFO:teuthology.orchestra.run.smithi001.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=legacy,wsync,nofallback 2024-02-02T15:26:39.752 INFO:teuthology.orchestra.run.smithi001.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,wsync". 2024-02-02T15:26:39.752 INFO:teuthology.orchestra.run.smithi001.stdout:mount.ceph: resolved to: "172.21.15.1:6789,172.21.15.71:6789,172.21.15.142:6789" 2024-02-02T15:26:39.752 INFO:teuthology.orchestra.run.smithi001.stdout:mount.ceph: trying mount with new device syntax: 0@371a409c-c1de-11ee-95b6-87774f69a715.cephfs=/volumes/_nogroup/sv_1/2234ea52-cd6c-42c8-816f-3ba5d3b919ec 2024-02-02T15:26:39.752 INFO:teuthology.orchestra.run.smithi001.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=legacy,wsync,name=0,key=0,mon_addr=172.21.15.1:6789/172.21.15.71:6789/172.21.15.142:6789" will pass to kernel 2024-02-02T15:26:39.758 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-02-02T15:26:39.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T15:26:39.759 DEBUG:teuthology.orchestra.run.smithi001:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-02-02T15:26:39.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T15:26:39.831 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-02T15:26:39.913 INFO:teuthology.orchestra.run.smithi001.stdout:client25423 2024-02-02T15:26:39.913 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-02-02T15:26:39.913 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T15:26:39.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T15:26:39.955 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-02T15:26:40.076 INFO:teuthology.orchestra.run.smithi001.stdout:client25423 2024-02-02T15:26:40.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T15:26:40.077 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-02-02T15:26:40.159 INFO:teuthology.orchestra.run.smithi001.stdout:client25423 2024-02-02T15:26:40.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T15:26:40.159 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/371a409c-c1de-11ee-95b6-87774f69a715.client25423/status') 2024-02-02T15:26:40.261 INFO:teuthology.orchestra.run.smithi001.stdout:instance: client.25423 (3)172.21.15.1:0/29645390 2024-02-02T15:26:40.261 INFO:teuthology.orchestra.run.smithi001.stdout:blocklisted: false 2024-02-02T15:26:40.262 INFO:teuthology.orchestra.run.smithi001.stderr:0+1 records in 2024-02-02T15:26:40.262 INFO:teuthology.orchestra.run.smithi001.stderr:0+1 records out 2024-02-02T15:26:40.262 INFO:teuthology.orchestra.run.smithi001.stderr:68 bytes copied, 5.3363e-05 s, 1.3 MB/s 2024-02-02T15:26:40.265 INFO:teuthology.run_tasks:Running task check-counter... 2024-02-02T15:26:40.314 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-02-02T15:26:40.327 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T15:26:40.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.525+0000 7fb9ada00700 1 -- 172.21.15.1:0/145576626 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9a8101f30 msgr2=0x7fb9a8102390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:40.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.525+0000 7fb9ada00700 1 --2- 172.21.15.1:0/145576626 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9a8101f30 0x7fb9a8102390 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb9900099c0 tx=0x7fb990009cd0 comp rx=0 tx=0).stop 2024-02-02T15:26:40.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.526+0000 7fb9ada00700 1 -- 172.21.15.1:0/145576626 shutdown_connections 2024-02-02T15:26:40.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.526+0000 7fb9ada00700 1 --2- 172.21.15.1:0/145576626 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9a81028d0 0x7fb9a8109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:40.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.526+0000 7fb9ada00700 1 --2- 172.21.15.1:0/145576626 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9a8100d40 0x7fb9a8101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:40.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.526+0000 7fb9ada00700 1 --2- 172.21.15.1:0/145576626 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9a8101f30 0x7fb9a8102390 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:40.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.526+0000 7fb9ada00700 1 -- 172.21.15.1:0/145576626 >> 172.21.15.1:0/145576626 conn(0x7fb9a80fc290 msgr2=0x7fb9a80fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:40.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.526+0000 7fb9ada00700 1 -- 172.21.15.1:0/145576626 shutdown_connections 2024-02-02T15:26:40.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.526+0000 7fb9ada00700 1 -- 172.21.15.1:0/145576626 wait complete. 2024-02-02T15:26:40.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.527+0000 7fb9ada00700 1 Processor -- start 2024-02-02T15:26:40.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.527+0000 7fb9ada00700 1 -- start start 2024-02-02T15:26:40.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.527+0000 7fb9ada00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9a8100d40 0x7fb9a8198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:40.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.527+0000 7fb9ada00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9a8101f30 0x7fb9a81991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:40.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.527+0000 7fb9ada00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9a81028d0 0x7fb9a81a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:40.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.527+0000 7fb9ada00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a81a07c0 con 0x7fb9a8100d40 2024-02-02T15:26:40.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.527+0000 7fb9ada00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a81a0930 con 0x7fb9a81028d0 2024-02-02T15:26:40.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.527+0000 7fb9ada00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a81a0af0 con 0x7fb9a8101f30 2024-02-02T15:26:40.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.527+0000 7fb9a6a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9a8100d40 0x7fb9a8198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:40.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.527+0000 7fb9a6a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9a8100d40 0x7fb9a8198c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52972/0 (socket says 172.21.15.1:52972) 2024-02-02T15:26:40.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.527+0000 7fb9a6a00700 1 -- 172.21.15.1:0/1272057546 learned_addr learned my addr 172.21.15.1:0/1272057546 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:40.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.528+0000 7fb9a6000700 1 --2- 172.21.15.1:0/1272057546 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9a8101f30 0x7fb9a81991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:40.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.528+0000 7fb9a7400700 1 --2- 172.21.15.1:0/1272057546 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9a81028d0 0x7fb9a81a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:40.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.528+0000 7fb9a6a00700 1 -- 172.21.15.1:0/1272057546 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9a8101f30 msgr2=0x7fb9a81991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:40.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.528+0000 7fb9a6a00700 1 --2- 172.21.15.1:0/1272057546 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9a8101f30 0x7fb9a81991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:40.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.528+0000 7fb9a6a00700 1 -- 172.21.15.1:0/1272057546 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9a81028d0 msgr2=0x7fb9a81a01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:40.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.528+0000 7fb9a6a00700 1 --2- 172.21.15.1:0/1272057546 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9a81028d0 0x7fb9a81a01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:40.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.528+0000 7fb9a6a00700 1 -- 172.21.15.1:0/1272057546 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9900095f0 con 0x7fb9a8100d40 2024-02-02T15:26:40.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.528+0000 7fb9a7400700 1 --2- 172.21.15.1:0/1272057546 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9a81028d0 0x7fb9a81a01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:26:40.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.528+0000 7fb9a6a00700 1 --2- 172.21.15.1:0/1272057546 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9a8100d40 0x7fb9a8198c80 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb99800e970 tx=0x7fb99800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:40.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.528+0000 7fb99f400700 1 -- 172.21.15.1:0/1272057546 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb998009790 con 0x7fb9a8100d40 2024-02-02T15:26:40.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.528+0000 7fb99f400700 1 -- 172.21.15.1:0/1272057546 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb99800eec0 con 0x7fb9a8100d40 2024-02-02T15:26:40.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.528+0000 7fb9ada00700 1 -- 172.21.15.1:0/1272057546 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9a81a0dd0 con 0x7fb9a8100d40 2024-02-02T15:26:40.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.528+0000 7fb99f400700 1 -- 172.21.15.1:0/1272057546 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb998003cf0 con 0x7fb9a8100d40 2024-02-02T15:26:40.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.528+0000 7fb9ada00700 1 -- 172.21.15.1:0/1272057546 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9a81a1250 con 0x7fb9a8100d40 2024-02-02T15:26:40.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.529+0000 7fb9ada00700 1 -- 172.21.15.1:0/1272057546 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9a810c5a0 con 0x7fb9a8100d40 2024-02-02T15:26:40.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.531+0000 7fb99f400700 1 -- 172.21.15.1:0/1272057546 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) v1 ==== 135069+0+0 (secure 0 0 0) 0x7fb99800ccd0 con 0x7fb9a8100d40 2024-02-02T15:26:40.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.533+0000 7fb99f400700 1 --2- 172.21.15.1:0/1272057546 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fb9880a05a0 0x7fb9880a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:40.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.533+0000 7fb99f400700 1 -- 172.21.15.1:0/1272057546 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fb998014070 con 0x7fb9a8100d40 2024-02-02T15:26:40.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.533+0000 7fb9a6000700 1 --2- 172.21.15.1:0/1272057546 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fb9880a05a0 0x7fb9880a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:40.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.534+0000 7fb9a6000700 1 --2- 172.21.15.1:0/1272057546 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fb9880a05a0 0x7fb9880a2a60 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb99000b580 tx=0x7fb990005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:40.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.537+0000 7fb99f400700 1 -- 172.21.15.1:0/1272057546 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7fb998095720 con 0x7fb9a8100d40 2024-02-02T15:26:40.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.753+0000 7fb9ada00700 1 -- 172.21.15.1:0/1272057546 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb9a81a18c0 con 0x7fb9a8100d40 2024-02-02T15:26:40.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.755+0000 7fb99f400700 1 -- 172.21.15.1:0/1272057546 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+24453 (secure 0 0 0) 0x7fb998098d40 con 0x7fb9a8100d40 2024-02-02T15:26:40.755 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:26:40.755 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":88,"fsid":"371a409c-c1de-11ee-95b6-87774f69a715","created":"2024-02-02T15:18:26.746506+0000","modified":"2024-02-02T15:26:28.267447+0000","last_up_change":"2024-02-02T15:24:08.800951+0000","last_in_change":"2024-02-02T15:23:52.784280+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T15:21:22.471754+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-02T15:24:46.195779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-02T15:24:47.226403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-02T15:24:50.300462+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"73e8a07f-3560-4c19-85ed-030d4d8f0f23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6803","nonce":2591906423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6805","nonce":2591906423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6809","nonce":2591906423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6807","nonce":2591906423}]},"public_addr":"172.21.15.1:6803/2591906423","cluster_addr":"172.21.15.1:6805/2591906423","heartbeat_back_addr":"172.21.15.1:6809/2591906423","heartbeat_front_addr":"172.21.15.1:6807/2591906423","state":["exists","up"]},{"osd":1,"uuid":"e53cffec-0ff9-4ed9-8dcf-ff47d04db2bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6801","nonce":747581719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6803","nonce":747581719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6807","nonce":747581719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6805","nonce":747581719}]},"public_addr":"172.21.15.71:6801/747581719","cluster_addr":"172.21.15.71:6803/747581719","heartbeat_back_addr":"172.21.15.71:6807/747581719","heartbeat_front_addr":"172.21.15.71:6805/747581719","state":["exists","up"]},{"osd":2,"uuid":"856c2347-294c-482b-8595-ad77f4792db4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6801","nonce":377511414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6803","nonce":377511414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6807","nonce":377511414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6805","nonce":377511414}]},"public_addr":"172.21.15.142:6801/377511414","cluster_addr":"172.21.15.142:6803/377511414","heartbeat_back_addr":"172.21.15.142:6807/377511414","heartbeat_front_addr":"172.21.15.142:6805/377511414","state":["exists","up"]},{"osd":3,"uuid":"4b946ad2-2da9-4e0b-9d61-d039d8669a42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6811","nonce":1086124477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6813","nonce":1086124477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6817","nonce":1086124477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6815","nonce":1086124477}]},"public_addr":"172.21.15.1:6811/1086124477","cluster_addr":"172.21.15.1:6813/1086124477","heartbeat_back_addr":"172.21.15.1:6817/1086124477","heartbeat_front_addr":"172.21.15.1:6815/1086124477","state":["exists","up"]},{"osd":4,"uuid":"43e36c43-47a0-4634-a45b-1da14a0e19d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6809","nonce":3095380305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6811","nonce":3095380305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6815","nonce":3095380305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6813","nonce":3095380305}]},"public_addr":"172.21.15.71:6809/3095380305","cluster_addr":"172.21.15.71:6811/3095380305","heartbeat_back_addr":"172.21.15.71:6815/3095380305","heartbeat_front_addr":"172.21.15.71:6813/3095380305","state":["exists","up"]},{"osd":5,"uuid":"eb48160e-ea7c-42f7-9477-be110bc8a34e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6809","nonce":4195149262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6811","nonce":4195149262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6815","nonce":4195149262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6813","nonce":4195149262}]},"public_addr":"172.21.15.142:6809/4195149262","cluster_addr":"172.21.15.142:6811/4195149262","heartbeat_back_addr":"172.21.15.142:6815/4195149262","heartbeat_front_addr":"172.21.15.142:6813/4195149262","state":["exists","up"]},{"osd":6,"uuid":"0ec4501a-5853-4ca6-b325-231f5b72335e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6819","nonce":1560153428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6821","nonce":1560153428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6825","nonce":1560153428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6823","nonce":1560153428}]},"public_addr":"172.21.15.1:6819/1560153428","cluster_addr":"172.21.15.1:6821/1560153428","heartbeat_back_addr":"172.21.15.1:6825/1560153428","heartbeat_front_addr":"172.21.15.1:6823/1560153428","state":["exists","up"]},{"osd":7,"uuid":"029debe8-9cdf-44f6-9aea-cabd1d185f93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6817","nonce":4080955538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6819","nonce":4080955538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6823","nonce":4080955538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6821","nonce":4080955538}]},"public_addr":"172.21.15.71:6817/4080955538","cluster_addr":"172.21.15.71:6819/4080955538","heartbeat_back_addr":"172.21.15.71:6823/4080955538","heartbeat_front_addr":"172.21.15.71:6821/4080955538","state":["exists","up"]},{"osd":8,"uuid":"7c90896c-9955-4c1b-aaca-7531dd5457f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6817","nonce":3942038209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6819","nonce":3942038209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6823","nonce":3942038209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6821","nonce":3942038209}]},"public_addr":"172.21.15.142:6817/3942038209","cluster_addr":"172.21.15.142:6819/3942038209","heartbeat_back_addr":"172.21.15.142:6823/3942038209","heartbeat_front_addr":"172.21.15.142:6821/3942038209","state":["exists","up"]},{"osd":9,"uuid":"7e165e68-31a8-4598-a577-de3e90ae5c5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6827","nonce":2716544850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6829","nonce":2716544850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6833","nonce":2716544850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6831","nonce":2716544850}]},"public_addr":"172.21.15.1:6827/2716544850","cluster_addr":"172.21.15.1:6829/2716544850","heartbeat_back_addr":"172.21.15.1:6833/2716544850","heartbeat_front_addr":"172.21.15.1:6831/2716544850","state":["exists","up"]},{"osd":10,"uuid":"48cd5fdb-0887-444a-aab8-a1de24282bb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6825","nonce":84691841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6827","nonce":84691841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6831","nonce":84691841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6829","nonce":84691841}]},"public_addr":"172.21.15.71:6825/84691841","cluster_addr":"172.21.15.71:6827/84691841","heartbeat_back_addr":"172.21.15.71:6831/84691841","heartbeat_front_addr":"172.21.15.71:6829/84691841","state":["exists","up"]},{"osd":11,"uuid":"f011dd27-11b0-4166-adfe-76756533163a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6825","nonce":2290179934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6827","nonce":2290179934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6831","nonce":2290179934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6829","nonce":2290179934}]},"public_addr":"172.21.15.142:6825/2290179934","cluster_addr":"172.21.15.142:6827/2290179934","heartbeat_back_addr":"172.21.15.142:6831/2290179934","heartbeat_front_addr":"172.21.15.142:6829/2290179934","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:20:44.060692+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:01.903817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:19.368258+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:40.028037+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:57.513361+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:22:14.805944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:22:35.188439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:22:53.014412+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:23:10.507420+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:23:31.849765+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:23:50.070612+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:24:07.388625+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":5,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/4092532060":"2024-02-03T15:19:25.836171+0000","172.21.15.1:6801/395498296":"2024-02-03T15:19:25.836171+0000","172.21.15.1:6800/395498296":"2024-02-03T15:19:25.836171+0000","172.21.15.1:0/1575373489":"2024-02-03T15:19:25.836171+0000","172.21.15.1:0/865983374":"2024-02-03T15:19:01.431596+0000","172.21.15.1:0/3165960227":"2024-02-03T15:19:01.431596+0000","172.21.15.1:0/2350830293":"2024-02-03T15:19:01.431596+0000","172.21.15.1:0/2959690250":"2024-02-03T15:19:25.836171+0000","172.21.15.1:6800/1162530732":"2024-02-03T15:19:01.431596+0000","172.21.15.1:6801/1162530732":"2024-02-03T15:19:01.431596+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T15:26:40.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.757+0000 7fb9ada00700 1 -- 172.21.15.1:0/1272057546 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fb9880a05a0 msgr2=0x7fb9880a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:40.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.757+0000 7fb9ada00700 1 --2- 172.21.15.1:0/1272057546 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fb9880a05a0 0x7fb9880a2a60 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb99000b580 tx=0x7fb990005fb0 comp rx=0 tx=0).stop 2024-02-02T15:26:40.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.757+0000 7fb9ada00700 1 -- 172.21.15.1:0/1272057546 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9a8100d40 msgr2=0x7fb9a8198c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:40.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.757+0000 7fb9ada00700 1 --2- 172.21.15.1:0/1272057546 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9a8100d40 0x7fb9a8198c80 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb99800e970 tx=0x7fb99800ec80 comp rx=0 tx=0).stop 2024-02-02T15:26:40.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.757+0000 7fb9ada00700 1 -- 172.21.15.1:0/1272057546 shutdown_connections 2024-02-02T15:26:40.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.757+0000 7fb9ada00700 1 --2- 172.21.15.1:0/1272057546 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb9a81028d0 0x7fb9a81a01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:40.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.757+0000 7fb9ada00700 1 --2- 172.21.15.1:0/1272057546 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb9a8100d40 0x7fb9a8198c80 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:40.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.757+0000 7fb9ada00700 1 --2- 172.21.15.1:0/1272057546 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fb9880a05a0 0x7fb9880a2a60 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:40.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.757+0000 7fb9ada00700 1 --2- 172.21.15.1:0/1272057546 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb9a8101f30 0x7fb9a81991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:40.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.757+0000 7fb9ada00700 1 -- 172.21.15.1:0/1272057546 >> 172.21.15.1:0/1272057546 conn(0x7fb9a80fc290 msgr2=0x7fb9a80fe510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:40.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.757+0000 7fb9ada00700 1 -- 172.21.15.1:0/1272057546 shutdown_connections 2024-02-02T15:26:40.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.757+0000 7fb9ada00700 1 -- 172.21.15.1:0/1272057546 wait complete. 2024-02-02T15:26:40.776 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-02T15:21:22.471754+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-02T15:24:46.195779+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-02T15:24:47.226403+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-02-02T15:24:50.300462+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-02-02T15:26:40.777 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-02T15:26:40.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.977+0000 7f2017800700 1 -- 172.21.15.1:0/1618580016 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2010101f80 msgr2=0x7f2010102400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:40.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.977+0000 7f2017800700 1 --2- 172.21.15.1:0/1618580016 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2010101f80 0x7f2010102400 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f1ff8009990 tx=0x7f1ff8009ca0 comp rx=0 tx=0).stop 2024-02-02T15:26:40.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.978+0000 7f2017800700 1 -- 172.21.15.1:0/1618580016 shutdown_connections 2024-02-02T15:26:40.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.978+0000 7f2017800700 1 --2- 172.21.15.1:0/1618580016 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2010102940 0x7f2010109260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:40.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.978+0000 7f2017800700 1 --2- 172.21.15.1:0/1618580016 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2010100e30 0x7f2010101250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:40.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.978+0000 7f2017800700 1 --2- 172.21.15.1:0/1618580016 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2010101f80 0x7f2010102400 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:40.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.978+0000 7f2017800700 1 -- 172.21.15.1:0/1618580016 >> 172.21.15.1:0/1618580016 conn(0x7f20100fc3a0 msgr2=0x7f20100fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:40.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.978+0000 7f2017800700 1 -- 172.21.15.1:0/1618580016 shutdown_connections 2024-02-02T15:26:40.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.978+0000 7f2017800700 1 -- 172.21.15.1:0/1618580016 wait complete. 2024-02-02T15:26:40.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.979+0000 7f2017800700 1 Processor -- start 2024-02-02T15:26:40.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.979+0000 7f2017800700 1 -- start start 2024-02-02T15:26:40.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.979+0000 7f2017800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2010100e30 0x7f2010196ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:40.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.979+0000 7f2017800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2010101f80 0x7f2010197020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:40.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.979+0000 7f2017800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2010102940 0x7f201019e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:40.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.979+0000 7f2017800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f201019e660 con 0x7f2010100e30 2024-02-02T15:26:40.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.979+0000 7f2017800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f201019e7a0 con 0x7f2010101f80 2024-02-02T15:26:40.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.979+0000 7f2017800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f201019e930 con 0x7f2010102940 2024-02-02T15:26:40.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.980+0000 7f2014e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2010100e30 0x7f2010196ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:40.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.980+0000 7f2014e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2010100e30 0x7f2010196ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52990/0 (socket says 172.21.15.1:52990) 2024-02-02T15:26:40.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.980+0000 7f2014e00700 1 -- 172.21.15.1:0/1953453191 learned_addr learned my addr 172.21.15.1:0/1953453191 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:40.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.980+0000 7f2015800700 1 --2- 172.21.15.1:0/1953453191 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2010102940 0x7f201019e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:40.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.980+0000 7f200fe00700 1 --2- 172.21.15.1:0/1953453191 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2010101f80 0x7f2010197020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:40.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.980+0000 7f2014e00700 1 -- 172.21.15.1:0/1953453191 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2010102940 msgr2=0x7f201019e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:40.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.980+0000 7f2014e00700 1 --2- 172.21.15.1:0/1953453191 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2010102940 0x7f201019e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:40.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.980+0000 7f2014e00700 1 -- 172.21.15.1:0/1953453191 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2010101f80 msgr2=0x7f2010197020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:40.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.980+0000 7f2014e00700 1 --2- 172.21.15.1:0/1953453191 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2010101f80 0x7f2010197020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:40.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.980+0000 7f2014e00700 1 -- 172.21.15.1:0/1953453191 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ff80095f0 con 0x7f2010100e30 2024-02-02T15:26:40.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.980+0000 7f200fe00700 1 --2- 172.21.15.1:0/1953453191 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2010101f80 0x7f2010197020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:26:40.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.980+0000 7f2014e00700 1 --2- 172.21.15.1:0/1953453191 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2010100e30 0x7f2010196ae0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f200000e9a0 tx=0x7f200000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:40.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.980+0000 7f200d600700 1 -- 172.21.15.1:0/1953453191 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2000009790 con 0x7f2010100e30 2024-02-02T15:26:40.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.981+0000 7f2017800700 1 -- 172.21.15.1:0/1953453191 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f201019ec10 con 0x7f2010100e30 2024-02-02T15:26:40.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.981+0000 7f200d600700 1 -- 172.21.15.1:0/1953453191 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f200000cbd0 con 0x7f2010100e30 2024-02-02T15:26:40.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.981+0000 7f200d600700 1 -- 172.21.15.1:0/1953453191 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2000018680 con 0x7f2010100e30 2024-02-02T15:26:40.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.981+0000 7f2017800700 1 -- 172.21.15.1:0/1953453191 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f201019f120 con 0x7f2010100e30 2024-02-02T15:26:40.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.982+0000 7f2017800700 1 -- 172.21.15.1:0/1953453191 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2010107d50 con 0x7f2010100e30 2024-02-02T15:26:40.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.983+0000 7f200d600700 1 -- 172.21.15.1:0/1953453191 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) v1 ==== 135069+0+0 (secure 0 0 0) 0x7f200000cd40 con 0x7f2010100e30 2024-02-02T15:26:40.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.985+0000 7f200d600700 1 --2- 172.21.15.1:0/1953453191 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f1ff00a0520 0x7f1ff00a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:40.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.985+0000 7f200d600700 1 -- 172.21.15.1:0/1953453191 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f2000014070 con 0x7f2010100e30 2024-02-02T15:26:40.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.985+0000 7f200fe00700 1 --2- 172.21.15.1:0/1953453191 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f1ff00a0520 0x7f1ff00a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:40.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.986+0000 7f200fe00700 1 --2- 172.21.15.1:0/1953453191 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f1ff00a0520 0x7f1ff00a29e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1ff8000c00 tx=0x7f1ff8005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:40.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:40.989+0000 7f200d600700 1 -- 172.21.15.1:0/1953453191 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f2000052110 con 0x7f2010100e30 2024-02-02T15:26:41.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.199+0000 7f2017800700 1 -- 172.21.15.1:0/1953453191 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2010061950 con 0x7f2010100e30 2024-02-02T15:26:41.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.199+0000 7f200d600700 1 -- 172.21.15.1:0/1953453191 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f20000cf030 con 0x7f2010100e30 2024-02-02T15:26:41.199 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-02-02T15:26:41.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.201+0000 7f2017800700 1 -- 172.21.15.1:0/1953453191 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f1ff00a0520 msgr2=0x7f1ff00a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:41.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.201+0000 7f2017800700 1 --2- 172.21.15.1:0/1953453191 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f1ff00a0520 0x7f1ff00a29e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f1ff8000c00 tx=0x7f1ff8005fb0 comp rx=0 tx=0).stop 2024-02-02T15:26:41.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.201+0000 7f2017800700 1 -- 172.21.15.1:0/1953453191 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2010100e30 msgr2=0x7f2010196ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:41.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.201+0000 7f2017800700 1 --2- 172.21.15.1:0/1953453191 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2010100e30 0x7f2010196ae0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f200000e9a0 tx=0x7f200000ecb0 comp rx=0 tx=0).stop 2024-02-02T15:26:41.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.202+0000 7f2017800700 1 -- 172.21.15.1:0/1953453191 shutdown_connections 2024-02-02T15:26:41.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.202+0000 7f2017800700 1 --2- 172.21.15.1:0/1953453191 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f1ff00a0520 0x7f1ff00a29e0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:41.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.202+0000 7f2017800700 1 --2- 172.21.15.1:0/1953453191 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2010102940 0x7f201019e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:41.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.202+0000 7f2017800700 1 --2- 172.21.15.1:0/1953453191 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2010100e30 0x7f2010196ae0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:41.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.202+0000 7f2017800700 1 --2- 172.21.15.1:0/1953453191 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2010101f80 0x7f2010197020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:41.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.202+0000 7f2017800700 1 -- 172.21.15.1:0/1953453191 >> 172.21.15.1:0/1953453191 conn(0x7f20100fc3a0 msgr2=0x7f20100fe610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:41.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.202+0000 7f2017800700 1 -- 172.21.15.1:0/1953453191 shutdown_connections 2024-02-02T15:26:41.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.202+0000 7f2017800700 1 -- 172.21.15.1:0/1953453191 wait complete. 2024-02-02T15:26:41.220 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-02T15:26:41.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.422+0000 7ff6a1a00700 1 -- 172.21.15.1:0/1031344812 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff69c101f70 msgr2=0x7ff69c1023f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:41.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.422+0000 7ff6a1a00700 1 --2- 172.21.15.1:0/1031344812 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff69c101f70 0x7ff69c1023f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff684009990 tx=0x7ff684009ca0 comp rx=0 tx=0).stop 2024-02-02T15:26:41.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.423+0000 7ff6a1a00700 1 -- 172.21.15.1:0/1031344812 shutdown_connections 2024-02-02T15:26:41.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.423+0000 7ff6a1a00700 1 --2- 172.21.15.1:0/1031344812 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff69c102930 0x7ff69c109250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:41.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.423+0000 7ff6a1a00700 1 --2- 172.21.15.1:0/1031344812 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff69c100e20 0x7ff69c101240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:41.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.423+0000 7ff6a1a00700 1 --2- 172.21.15.1:0/1031344812 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff69c101f70 0x7ff69c1023f0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:41.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.423+0000 7ff6a1a00700 1 -- 172.21.15.1:0/1031344812 >> 172.21.15.1:0/1031344812 conn(0x7ff69c0fc390 msgr2=0x7ff69c0fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:41.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.423+0000 7ff6a1a00700 1 -- 172.21.15.1:0/1031344812 shutdown_connections 2024-02-02T15:26:41.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.423+0000 7ff6a1a00700 1 -- 172.21.15.1:0/1031344812 wait complete. 2024-02-02T15:26:41.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.424+0000 7ff6a1a00700 1 Processor -- start 2024-02-02T15:26:41.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.424+0000 7ff6a1a00700 1 -- start start 2024-02-02T15:26:41.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.424+0000 7ff6a1a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff69c100e20 0x7ff69c074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:41.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.424+0000 7ff6a1a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff69c101f70 0x7ff69c073130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:41.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.424+0000 7ff6a1a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff69c102930 0x7ff69c073670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:41.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.424+0000 7ff6a1a00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff69c073e10 con 0x7ff69c102930 2024-02-02T15:26:41.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.424+0000 7ff6a1a00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff69c073f80 con 0x7ff69c101f70 2024-02-02T15:26:41.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.424+0000 7ff6a1a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff69c1a0b40 con 0x7ff69c100e20 2024-02-02T15:26:41.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.424+0000 7ff69b400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff69c102930 0x7ff69c073670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:41.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.425+0000 7ff69aa00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff69c100e20 0x7ff69c074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:41.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.425+0000 7ff69b400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff69c102930 0x7ff69c073670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53002/0 (socket says 172.21.15.1:53002) 2024-02-02T15:26:41.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.425+0000 7ff69b400700 1 -- 172.21.15.1:0/1452125694 learned_addr learned my addr 172.21.15.1:0/1452125694 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:41.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.425+0000 7ff69a000700 1 --2- 172.21.15.1:0/1452125694 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff69c101f70 0x7ff69c073130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:41.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.425+0000 7ff69b400700 1 -- 172.21.15.1:0/1452125694 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff69c100e20 msgr2=0x7ff69c074af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:41.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.425+0000 7ff69b400700 1 --2- 172.21.15.1:0/1452125694 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff69c100e20 0x7ff69c074af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:41.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.425+0000 7ff69b400700 1 -- 172.21.15.1:0/1452125694 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff69c101f70 msgr2=0x7ff69c073130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:41.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.425+0000 7ff69b400700 1 --2- 172.21.15.1:0/1452125694 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff69c101f70 0x7ff69c073130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:41.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.425+0000 7ff69b400700 1 -- 172.21.15.1:0/1452125694 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6840095f0 con 0x7ff69c102930 2024-02-02T15:26:41.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.425+0000 7ff69b400700 1 --2- 172.21.15.1:0/1452125694 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff69c102930 0x7ff69c073670 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ff694007ee0 tx=0x7ff69400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:41.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.425+0000 7ff693400700 1 -- 172.21.15.1:0/1452125694 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff694017070 con 0x7ff69c102930 2024-02-02T15:26:41.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.425+0000 7ff6a1a00700 1 -- 172.21.15.1:0/1452125694 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff69c1a0e20 con 0x7ff69c102930 2024-02-02T15:26:41.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.425+0000 7ff693400700 1 -- 172.21.15.1:0/1452125694 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff69400f040 con 0x7ff69c102930 2024-02-02T15:26:41.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.425+0000 7ff693400700 1 -- 172.21.15.1:0/1452125694 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff694005550 con 0x7ff69c102930 2024-02-02T15:26:41.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.426+0000 7ff6a1a00700 1 -- 172.21.15.1:0/1452125694 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff69c1a12a0 con 0x7ff69c102930 2024-02-02T15:26:41.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.427+0000 7ff6a1a00700 1 -- 172.21.15.1:0/1452125694 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff69c107d40 con 0x7ff69c102930 2024-02-02T15:26:41.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.428+0000 7ff693400700 1 -- 172.21.15.1:0/1452125694 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) v1 ==== 135069+0+0 (secure 0 0 0) 0x7ff694005750 con 0x7ff69c102930 2024-02-02T15:26:41.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.430+0000 7ff693400700 1 --2- 172.21.15.1:0/1452125694 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7ff67c0a0520 0x7ff67c0a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:41.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.430+0000 7ff693400700 1 -- 172.21.15.1:0/1452125694 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9107+0+0 (secure 0 0 0) 0x7ff6940cb480 con 0x7ff69c102930 2024-02-02T15:26:41.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.430+0000 7ff69aa00700 1 --2- 172.21.15.1:0/1452125694 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7ff67c0a0520 0x7ff67c0a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:41.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.431+0000 7ff69aa00700 1 --2- 172.21.15.1:0/1452125694 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7ff67c0a0520 0x7ff67c0a29e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff68c005950 tx=0x7ff68c00b410 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:41.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.434+0000 7ff693400700 1 -- 172.21.15.1:0/1452125694 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7ff694094c50 con 0x7ff69c102930 2024-02-02T15:26:41.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:41 smithi071 ceph-mon[44707]: pgmap v295: 129 pgs: 129 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 87 B/s wr, 9 op/s 2024-02-02T15:26:41.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:41 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/1272057546' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-02T15:26:41.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:41 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/1953453191' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-02T15:26:41.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.648+0000 7ff6a1a00700 1 -- 172.21.15.1:0/1452125694 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff69c02ced0 con 0x7ff69c102930 2024-02-02T15:26:41.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.649+0000 7ff693400700 1 -- 172.21.15.1:0/1452125694 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff694098270 con 0x7ff69c102930 2024-02-02T15:26:41.649 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-02-02T15:26:41.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.651+0000 7ff6a1a00700 1 -- 172.21.15.1:0/1452125694 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7ff67c0a0520 msgr2=0x7ff67c0a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:41.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.651+0000 7ff6a1a00700 1 --2- 172.21.15.1:0/1452125694 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7ff67c0a0520 0x7ff67c0a29e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff68c005950 tx=0x7ff68c00b410 comp rx=0 tx=0).stop 2024-02-02T15:26:41.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.651+0000 7ff6a1a00700 1 -- 172.21.15.1:0/1452125694 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff69c102930 msgr2=0x7ff69c073670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:41.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.651+0000 7ff6a1a00700 1 --2- 172.21.15.1:0/1452125694 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff69c102930 0x7ff69c073670 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ff694007ee0 tx=0x7ff69400d3b0 comp rx=0 tx=0).stop 2024-02-02T15:26:41.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.651+0000 7ff6a1a00700 1 -- 172.21.15.1:0/1452125694 shutdown_connections 2024-02-02T15:26:41.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.651+0000 7ff6a1a00700 1 --2- 172.21.15.1:0/1452125694 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff69c102930 0x7ff69c073670 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:41.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.651+0000 7ff6a1a00700 1 --2- 172.21.15.1:0/1452125694 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7ff67c0a0520 0x7ff67c0a29e0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:41.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.651+0000 7ff6a1a00700 1 --2- 172.21.15.1:0/1452125694 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff69c100e20 0x7ff69c074af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:41.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.651+0000 7ff6a1a00700 1 --2- 172.21.15.1:0/1452125694 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff69c101f70 0x7ff69c073130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:41.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.651+0000 7ff6a1a00700 1 -- 172.21.15.1:0/1452125694 >> 172.21.15.1:0/1452125694 conn(0x7ff69c0fc390 msgr2=0x7ff69c0fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:41.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.652+0000 7ff6a1a00700 1 -- 172.21.15.1:0/1452125694 shutdown_connections 2024-02-02T15:26:41.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.652+0000 7ff6a1a00700 1 -- 172.21.15.1:0/1452125694 wait complete. 2024-02-02T15:26:41.668 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:41 smithi001 ceph-mon[35470]: pgmap v295: 129 pgs: 129 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 87 B/s wr, 9 op/s 2024-02-02T15:26:41.668 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:41 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/1272057546' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-02T15:26:41.668 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:41 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/1953453191' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-02T15:26:41.670 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-02T15:26:41.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:41 smithi142 ceph-mon[45728]: pgmap v295: 129 pgs: 129 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 87 B/s wr, 9 op/s 2024-02-02T15:26:41.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:41 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/1272057546' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-02T15:26:41.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:41 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/1953453191' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-02T15:26:41.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.872+0000 7f0bd1800700 1 -- 172.21.15.1:0/391414926 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0bcc101f70 msgr2=0x7f0bcc1023f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:41.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.872+0000 7f0bd1800700 1 --2- 172.21.15.1:0/391414926 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0bcc101f70 0x7f0bcc1023f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f0bb4009990 tx=0x7f0bb4009ca0 comp rx=0 tx=0).stop 2024-02-02T15:26:41.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.872+0000 7f0bd1800700 1 -- 172.21.15.1:0/391414926 shutdown_connections 2024-02-02T15:26:41.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.872+0000 7f0bd1800700 1 --2- 172.21.15.1:0/391414926 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0bcc102930 0x7f0bcc109250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:41.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.872+0000 7f0bd1800700 1 --2- 172.21.15.1:0/391414926 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0bcc100e20 0x7f0bcc101240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:41.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.872+0000 7f0bd1800700 1 --2- 172.21.15.1:0/391414926 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0bcc101f70 0x7f0bcc1023f0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:41.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.872+0000 7f0bd1800700 1 -- 172.21.15.1:0/391414926 >> 172.21.15.1:0/391414926 conn(0x7f0bcc0fc390 msgr2=0x7f0bcc0fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:41.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.872+0000 7f0bd1800700 1 -- 172.21.15.1:0/391414926 shutdown_connections 2024-02-02T15:26:41.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.872+0000 7f0bd1800700 1 -- 172.21.15.1:0/391414926 wait complete. 2024-02-02T15:26:41.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.873+0000 7f0bd1800700 1 Processor -- start 2024-02-02T15:26:41.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.873+0000 7f0bd1800700 1 -- start start 2024-02-02T15:26:41.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.873+0000 7f0bd1800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0bcc100e20 0x7f0bcc074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:41.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.873+0000 7f0bd1800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0bcc101f70 0x7f0bcc073130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:41.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.873+0000 7f0bd1800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0bcc102930 0x7f0bcc073670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:41.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.873+0000 7f0bd1800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bcc073e10 con 0x7f0bcc102930 2024-02-02T15:26:41.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.873+0000 7f0bd1800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bcc073f80 con 0x7f0bcc101f70 2024-02-02T15:26:41.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.873+0000 7f0bd1800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bcc1a0b40 con 0x7f0bcc100e20 2024-02-02T15:26:41.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.874+0000 7f0bcb400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0bcc102930 0x7f0bcc073670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:41.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.874+0000 7f0bcb400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0bcc102930 0x7f0bcc073670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53024/0 (socket says 172.21.15.1:53024) 2024-02-02T15:26:41.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.874+0000 7f0bcb400700 1 -- 172.21.15.1:0/307245782 learned_addr learned my addr 172.21.15.1:0/307245782 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:41.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.874+0000 7f0bca000700 1 --2- 172.21.15.1:0/307245782 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0bcc101f70 0x7f0bcc073130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:41.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.874+0000 7f0bcaa00700 1 --2- 172.21.15.1:0/307245782 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0bcc100e20 0x7f0bcc074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:41.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.874+0000 7f0bcb400700 1 -- 172.21.15.1:0/307245782 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0bcc100e20 msgr2=0x7f0bcc074af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:41.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.874+0000 7f0bcb400700 1 --2- 172.21.15.1:0/307245782 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0bcc100e20 0x7f0bcc074af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:41.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.874+0000 7f0bcb400700 1 -- 172.21.15.1:0/307245782 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0bcc101f70 msgr2=0x7f0bcc073130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:41.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.874+0000 7f0bcb400700 1 --2- 172.21.15.1:0/307245782 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0bcc101f70 0x7f0bcc073130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:41.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.874+0000 7f0bcb400700 1 -- 172.21.15.1:0/307245782 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bb40095f0 con 0x7f0bcc102930 2024-02-02T15:26:41.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.874+0000 7f0bcaa00700 1 --2- 172.21.15.1:0/307245782 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0bcc100e20 0x7f0bcc074af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:26:41.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.874+0000 7f0bcb400700 1 --2- 172.21.15.1:0/307245782 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0bcc102930 0x7f0bcc073670 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f0bc400ef30 tx=0x7f0bc400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:41.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.875+0000 7f0bc3400700 1 -- 172.21.15.1:0/307245782 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc4018070 con 0x7f0bcc102930 2024-02-02T15:26:41.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.875+0000 7f0bc3400700 1 -- 172.21.15.1:0/307245782 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0bc400f040 con 0x7f0bcc102930 2024-02-02T15:26:41.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.875+0000 7f0bc3400700 1 -- 172.21.15.1:0/307245782 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc400cdb0 con 0x7f0bcc102930 2024-02-02T15:26:41.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.875+0000 7f0bd1800700 1 -- 172.21.15.1:0/307245782 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bcc1a0dc0 con 0x7f0bcc102930 2024-02-02T15:26:41.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.875+0000 7f0bd1800700 1 -- 172.21.15.1:0/307245782 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bcc1a1220 con 0x7f0bcc102930 2024-02-02T15:26:41.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.876+0000 7f0bd1800700 1 -- 172.21.15.1:0/307245782 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bcc107d40 con 0x7f0bcc102930 2024-02-02T15:26:41.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.878+0000 7f0bc3400700 1 -- 172.21.15.1:0/307245782 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) v1 ==== 135069+0+0 (secure 0 0 0) 0x7f0bc4003a00 con 0x7f0bcc102930 2024-02-02T15:26:41.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.879+0000 7f0bc3400700 1 --2- 172.21.15.1:0/307245782 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f0bac0a0520 0x7f0bac0a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:41.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.879+0000 7f0bc3400700 1 -- 172.21.15.1:0/307245782 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f0bc40cb870 con 0x7f0bcc102930 2024-02-02T15:26:41.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.879+0000 7f0bcaa00700 1 --2- 172.21.15.1:0/307245782 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f0bac0a0520 0x7f0bac0a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:41.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.880+0000 7f0bcaa00700 1 --2- 172.21.15.1:0/307245782 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f0bac0a0520 0x7f0bac0a29e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc005fd0 tx=0x7f0bbc005f60 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:41.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:41.883+0000 7f0bc3400700 1 -- 172.21.15.1:0/307245782 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f0bc40950d0 con 0x7f0bcc102930 2024-02-02T15:26:42.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.095+0000 7f0bd1800700 1 -- 172.21.15.1:0/307245782 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0bcc02ced0 con 0x7f0bcc102930 2024-02-02T15:26:42.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.095+0000 7f0bc3400700 1 -- 172.21.15.1:0/307245782 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0bc40986f0 con 0x7f0bcc102930 2024-02-02T15:26:42.096 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 64 2024-02-02T15:26:42.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.098+0000 7f0bd1800700 1 -- 172.21.15.1:0/307245782 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f0bac0a0520 msgr2=0x7f0bac0a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:42.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.098+0000 7f0bd1800700 1 --2- 172.21.15.1:0/307245782 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f0bac0a0520 0x7f0bac0a29e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc005fd0 tx=0x7f0bbc005f60 comp rx=0 tx=0).stop 2024-02-02T15:26:42.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.098+0000 7f0bd1800700 1 -- 172.21.15.1:0/307245782 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0bcc102930 msgr2=0x7f0bcc073670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:42.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.098+0000 7f0bd1800700 1 --2- 172.21.15.1:0/307245782 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0bcc102930 0x7f0bcc073670 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f0bc400ef30 tx=0x7f0bc400c3a0 comp rx=0 tx=0).stop 2024-02-02T15:26:42.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.098+0000 7f0bd1800700 1 -- 172.21.15.1:0/307245782 shutdown_connections 2024-02-02T15:26:42.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.098+0000 7f0bd1800700 1 --2- 172.21.15.1:0/307245782 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0bcc102930 0x7f0bcc073670 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:42.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.098+0000 7f0bd1800700 1 --2- 172.21.15.1:0/307245782 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f0bac0a0520 0x7f0bac0a29e0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:42.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.098+0000 7f0bd1800700 1 --2- 172.21.15.1:0/307245782 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0bcc100e20 0x7f0bcc074af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:42.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.098+0000 7f0bd1800700 1 --2- 172.21.15.1:0/307245782 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0bcc101f70 0x7f0bcc073130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:42.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.098+0000 7f0bd1800700 1 -- 172.21.15.1:0/307245782 >> 172.21.15.1:0/307245782 conn(0x7f0bcc0fc390 msgr2=0x7f0bcc0fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:42.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.098+0000 7f0bd1800700 1 -- 172.21.15.1:0/307245782 shutdown_connections 2024-02-02T15:26:42.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.098+0000 7f0bd1800700 1 -- 172.21.15.1:0/307245782 wait complete. 2024-02-02T15:26:42.116 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-02-02T15:26:42.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.319+0000 7f2688c00700 1 -- 172.21.15.1:0/3767374904 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2684101f70 msgr2=0x7f26841023f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:42.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.319+0000 7f2688c00700 1 --2- 172.21.15.1:0/3767374904 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2684101f70 0x7f26841023f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f266c009990 tx=0x7f266c009ca0 comp rx=0 tx=0).stop 2024-02-02T15:26:42.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.319+0000 7f2688c00700 1 -- 172.21.15.1:0/3767374904 shutdown_connections 2024-02-02T15:26:42.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.319+0000 7f2688c00700 1 --2- 172.21.15.1:0/3767374904 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2684102930 0x7f2684109250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:42.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.319+0000 7f2688c00700 1 --2- 172.21.15.1:0/3767374904 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2684100e20 0x7f2684101240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:42.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.319+0000 7f2688c00700 1 --2- 172.21.15.1:0/3767374904 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2684101f70 0x7f26841023f0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:42.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.319+0000 7f2688c00700 1 -- 172.21.15.1:0/3767374904 >> 172.21.15.1:0/3767374904 conn(0x7f26840fc390 msgr2=0x7f26840fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:42.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.319+0000 7f2688c00700 1 -- 172.21.15.1:0/3767374904 shutdown_connections 2024-02-02T15:26:42.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.319+0000 7f2688c00700 1 -- 172.21.15.1:0/3767374904 wait complete. 2024-02-02T15:26:42.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.320+0000 7f2688c00700 1 Processor -- start 2024-02-02T15:26:42.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.320+0000 7f2688c00700 1 -- start start 2024-02-02T15:26:42.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.320+0000 7f2688c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2684100e20 0x7f2684198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:42.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.320+0000 7f2688c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2684101f70 0x7f2684199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:42.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.320+0000 7f2688c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2684102930 0x7f26841a0290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:42.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.320+0000 7f2688c00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26841a0860 con 0x7f2684101f70 2024-02-02T15:26:42.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.320+0000 7f2688c00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26841a09d0 con 0x7f2684102930 2024-02-02T15:26:42.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.320+0000 7f2688c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26841a0b90 con 0x7f2684100e20 2024-02-02T15:26:42.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.321+0000 7f2681400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2684101f70 0x7f2684199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:42.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.321+0000 7f2681e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2684100e20 0x7f2684198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:42.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.321+0000 7f2681400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2684101f70 0x7f2684199260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53042/0 (socket says 172.21.15.1:53042) 2024-02-02T15:26:42.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.321+0000 7f2681400700 1 -- 172.21.15.1:0/4031181402 learned_addr learned my addr 172.21.15.1:0/4031181402 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:42.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.321+0000 7f2682800700 1 --2- 172.21.15.1:0/4031181402 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2684102930 0x7f26841a0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:42.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.321+0000 7f2681400700 1 -- 172.21.15.1:0/4031181402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2684100e20 msgr2=0x7f2684198d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:42.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.321+0000 7f2681400700 1 --2- 172.21.15.1:0/4031181402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2684100e20 0x7f2684198d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:42.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.321+0000 7f2681400700 1 -- 172.21.15.1:0/4031181402 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2684102930 msgr2=0x7f26841a0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:42.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.321+0000 7f2681400700 1 --2- 172.21.15.1:0/4031181402 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2684102930 0x7f26841a0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:42.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.321+0000 7f2681400700 1 -- 172.21.15.1:0/4031181402 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f266c0095f0 con 0x7f2684101f70 2024-02-02T15:26:42.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.321+0000 7f2681400700 1 --2- 172.21.15.1:0/4031181402 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2684101f70 0x7f2684199260 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f266c005910 tx=0x7f266c005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:42.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.322+0000 7f267aa00700 1 -- 172.21.15.1:0/4031181402 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f266c01d070 con 0x7f2684101f70 2024-02-02T15:26:42.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.322+0000 7f267aa00700 1 -- 172.21.15.1:0/4031181402 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f266c003f60 con 0x7f2684101f70 2024-02-02T15:26:42.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.322+0000 7f2688c00700 1 -- 172.21.15.1:0/4031181402 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26841a0e10 con 0x7f2684101f70 2024-02-02T15:26:42.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.322+0000 7f267aa00700 1 -- 172.21.15.1:0/4031181402 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f266c022c30 con 0x7f2684101f70 2024-02-02T15:26:42.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.322+0000 7f2688c00700 1 -- 172.21.15.1:0/4031181402 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26841a1180 con 0x7f2684101f70 2024-02-02T15:26:42.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.323+0000 7f2688c00700 1 -- 172.21.15.1:0/4031181402 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f268410c670 con 0x7f2684101f70 2024-02-02T15:26:42.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.325+0000 7f267aa00700 1 -- 172.21.15.1:0/4031181402 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) v1 ==== 135069+0+0 (secure 0 0 0) 0x7f266c0040d0 con 0x7f2684101f70 2024-02-02T15:26:42.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.326+0000 7f267aa00700 1 --2- 172.21.15.1:0/4031181402 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f26640a0520 0x7f26640a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:42.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.326+0000 7f267aa00700 1 -- 172.21.15.1:0/4031181402 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f266c0cd970 con 0x7f2684101f70 2024-02-02T15:26:42.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.326+0000 7f2681e00700 1 --2- 172.21.15.1:0/4031181402 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f26640a0520 0x7f26640a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:42.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.327+0000 7f2681e00700 1 --2- 172.21.15.1:0/4031181402 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f26640a0520 0x7f26640a29e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f268419a1f0 tx=0x7f267400b410 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:42.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.328+0000 7f267aa00700 1 -- 172.21.15.1:0/4031181402 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f266c0971d0 con 0x7f2684101f70 2024-02-02T15:26:42.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:42 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/1452125694' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-02T15:26:42.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:42 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/307245782' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-02T15:26:42.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.539+0000 7f2688c00700 1 -- 172.21.15.1:0/4031181402 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f26841a1790 con 0x7f2684101f70 2024-02-02T15:26:42.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.539+0000 7f267aa00700 1 -- 172.21.15.1:0/4031181402 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v88) v1 ==== 103+0+11 (secure 0 0 0) 0x7f266c09a7f0 con 0x7f2684101f70 2024-02-02T15:26:42.540 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-02-02T15:26:42.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.542+0000 7f2688c00700 1 -- 172.21.15.1:0/4031181402 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f26640a0520 msgr2=0x7f26640a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:42.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.542+0000 7f2688c00700 1 --2- 172.21.15.1:0/4031181402 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f26640a0520 0x7f26640a29e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f268419a1f0 tx=0x7f267400b410 comp rx=0 tx=0).stop 2024-02-02T15:26:42.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.542+0000 7f2688c00700 1 -- 172.21.15.1:0/4031181402 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2684101f70 msgr2=0x7f2684199260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:42.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.542+0000 7f2688c00700 1 --2- 172.21.15.1:0/4031181402 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2684101f70 0x7f2684199260 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f266c005910 tx=0x7f266c005740 comp rx=0 tx=0).stop 2024-02-02T15:26:42.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.542+0000 7f2688c00700 1 -- 172.21.15.1:0/4031181402 shutdown_connections 2024-02-02T15:26:42.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.542+0000 7f2688c00700 1 --2- 172.21.15.1:0/4031181402 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2684102930 0x7f26841a0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:42.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.542+0000 7f2688c00700 1 --2- 172.21.15.1:0/4031181402 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f26640a0520 0x7f26640a29e0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:42.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.542+0000 7f2688c00700 1 --2- 172.21.15.1:0/4031181402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2684100e20 0x7f2684198d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:42.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.542+0000 7f2688c00700 1 --2- 172.21.15.1:0/4031181402 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2684101f70 0x7f2684199260 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:42.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.542+0000 7f2688c00700 1 -- 172.21.15.1:0/4031181402 >> 172.21.15.1:0/4031181402 conn(0x7f26840fc390 msgr2=0x7f26840fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:42.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.542+0000 7f2688c00700 1 -- 172.21.15.1:0/4031181402 shutdown_connections 2024-02-02T15:26:42.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.542+0000 7f2688c00700 1 -- 172.21.15.1:0/4031181402 wait complete. 2024-02-02T15:26:42.561 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T15:26:42.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:42 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/1452125694' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-02T15:26:42.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:42 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/307245782' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-02T15:26:42.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:42 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/1452125694' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-02T15:26:42.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:42 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/307245782' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-02T15:26:42.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.759+0000 7f7577c00700 1 -- 172.21.15.1:0/1748401640 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7570101f30 msgr2=0x7f7570102390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:42.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.759+0000 7f7577c00700 1 --2- 172.21.15.1:0/1748401640 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7570101f30 0x7f7570102390 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f75580099c0 tx=0x7f7558009cd0 comp rx=0 tx=0).stop 2024-02-02T15:26:42.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.760+0000 7f7577c00700 1 -- 172.21.15.1:0/1748401640 shutdown_connections 2024-02-02T15:26:42.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.760+0000 7f7577c00700 1 --2- 172.21.15.1:0/1748401640 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75701028d0 0x7f7570109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:42.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.760+0000 7f7577c00700 1 --2- 172.21.15.1:0/1748401640 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7570100d40 0x7f7570101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:42.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.760+0000 7f7577c00700 1 --2- 172.21.15.1:0/1748401640 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7570101f30 0x7f7570102390 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:42.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.760+0000 7f7577c00700 1 -- 172.21.15.1:0/1748401640 >> 172.21.15.1:0/1748401640 conn(0x7f75700fc290 msgr2=0x7f75700fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:42.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.760+0000 7f7577c00700 1 -- 172.21.15.1:0/1748401640 shutdown_connections 2024-02-02T15:26:42.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.760+0000 7f7577c00700 1 -- 172.21.15.1:0/1748401640 wait complete. 2024-02-02T15:26:42.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.761+0000 7f7577c00700 1 Processor -- start 2024-02-02T15:26:42.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.761+0000 7f7577c00700 1 -- start start 2024-02-02T15:26:42.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.761+0000 7f7577c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7570100d40 0x7f7570198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:42.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.761+0000 7f7577c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7570101f30 0x7f75701991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:42.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.761+0000 7f7577c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75701028d0 0x7f75701a01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:42.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.761+0000 7f7577c00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75701a07b0 con 0x7f7570100d40 2024-02-02T15:26:42.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.761+0000 7f7577c00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75701a0920 con 0x7f7570101f30 2024-02-02T15:26:42.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.761+0000 7f7577c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75701a0ae0 con 0x7f75701028d0 2024-02-02T15:26:42.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.762+0000 7f756fe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7570101f30 0x7f75701991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:42.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.762+0000 7f7575c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75701028d0 0x7f75701a01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:42.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.762+0000 7f7575200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7570100d40 0x7f7570198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:42.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.762+0000 7f756fe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7570101f30 0x7f75701991b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:58028/0 (socket says 172.21.15.1:58028) 2024-02-02T15:26:42.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.762+0000 7f7575c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75701028d0 0x7f75701a01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:42266/0 (socket says 172.21.15.1:42266) 2024-02-02T15:26:42.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.762+0000 7f756fe00700 1 -- 172.21.15.1:0/3077564406 learned_addr learned my addr 172.21.15.1:0/3077564406 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:42.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.762+0000 7f756fe00700 1 -- 172.21.15.1:0/3077564406 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75701028d0 msgr2=0x7f75701a01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:42.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.762+0000 7f756fe00700 1 --2- 172.21.15.1:0/3077564406 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75701028d0 0x7f75701a01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:42.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.762+0000 7f756fe00700 1 -- 172.21.15.1:0/3077564406 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7570100d40 msgr2=0x7f7570198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:42.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.762+0000 7f756fe00700 1 --2- 172.21.15.1:0/3077564406 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7570100d40 0x7f7570198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:42.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.762+0000 7f756fe00700 1 -- 172.21.15.1:0/3077564406 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75580095f0 con 0x7f7570101f30 2024-02-02T15:26:42.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.762+0000 7f756fe00700 1 --2- 172.21.15.1:0/3077564406 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7570101f30 0x7f75701991b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7558009fd0 tx=0x7f755800f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:42.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.763+0000 7f756d600700 1 -- 172.21.15.1:0/3077564406 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f755801d070 con 0x7f7570101f30 2024-02-02T15:26:42.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.763+0000 7f756d600700 1 -- 172.21.15.1:0/3077564406 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7558005050 con 0x7f7570101f30 2024-02-02T15:26:42.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.763+0000 7f7577c00700 1 -- 172.21.15.1:0/3077564406 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75701a0d60 con 0x7f7570101f30 2024-02-02T15:26:42.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.763+0000 7f756d600700 1 -- 172.21.15.1:0/3077564406 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7558004b30 con 0x7f7570101f30 2024-02-02T15:26:42.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.763+0000 7f7577c00700 1 -- 172.21.15.1:0/3077564406 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75701a1190 con 0x7f7570101f30 2024-02-02T15:26:42.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.764+0000 7f7577c00700 1 -- 172.21.15.1:0/3077564406 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f757010c5a0 con 0x7f7570101f30 2024-02-02T15:26:42.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.766+0000 7f756d600700 1 -- 172.21.15.1:0/3077564406 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 135069+0+0 (secure 0 0 0) 0x7f7558004c90 con 0x7f7570101f30 2024-02-02T15:26:42.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.768+0000 7f756d600700 1 --2- 172.21.15.1:0/3077564406 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f75500a05a0 0x7f75500a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:42.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.768+0000 7f756d600700 1 -- 172.21.15.1:0/3077564406 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f75580cce40 con 0x7f7570101f30 2024-02-02T15:26:42.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.768+0000 7f7575200700 1 --2- 172.21.15.1:0/3077564406 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f75500a05a0 0x7f75500a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:42.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.768+0000 7f7575200700 1 --2- 172.21.15.1:0/3077564406 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f75500a05a0 0x7f75500a2a60 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f757019a140 tx=0x7f7560009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:42.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.769+0000 7f756d600700 1 -- 172.21.15.1:0/3077564406 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f7558096700 con 0x7f7570101f30 2024-02-02T15:26:42.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.984+0000 7f7577c00700 1 -- 172.21.15.1:0/3077564406 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f757010c730 con 0x7f7570101f30 2024-02-02T15:26:42.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.987+0000 7f756d600700 1 -- 172.21.15.1:0/3077564406 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+24453 (secure 0 0 0) 0x7f7558099d20 con 0x7f7570101f30 2024-02-02T15:26:42.986 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:26:42.987 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":88,"fsid":"371a409c-c1de-11ee-95b6-87774f69a715","created":"2024-02-02T15:18:26.746506+0000","modified":"2024-02-02T15:26:28.267447+0000","last_up_change":"2024-02-02T15:24:08.800951+0000","last_in_change":"2024-02-02T15:23:52.784280+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T15:21:22.471754+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-02T15:24:46.195779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-02T15:24:47.226403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-02T15:24:50.300462+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"73e8a07f-3560-4c19-85ed-030d4d8f0f23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6803","nonce":2591906423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6805","nonce":2591906423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6809","nonce":2591906423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6807","nonce":2591906423}]},"public_addr":"172.21.15.1:6803/2591906423","cluster_addr":"172.21.15.1:6805/2591906423","heartbeat_back_addr":"172.21.15.1:6809/2591906423","heartbeat_front_addr":"172.21.15.1:6807/2591906423","state":["exists","up"]},{"osd":1,"uuid":"e53cffec-0ff9-4ed9-8dcf-ff47d04db2bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6801","nonce":747581719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6803","nonce":747581719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6807","nonce":747581719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6805","nonce":747581719}]},"public_addr":"172.21.15.71:6801/747581719","cluster_addr":"172.21.15.71:6803/747581719","heartbeat_back_addr":"172.21.15.71:6807/747581719","heartbeat_front_addr":"172.21.15.71:6805/747581719","state":["exists","up"]},{"osd":2,"uuid":"856c2347-294c-482b-8595-ad77f4792db4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6801","nonce":377511414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6803","nonce":377511414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6807","nonce":377511414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6805","nonce":377511414}]},"public_addr":"172.21.15.142:6801/377511414","cluster_addr":"172.21.15.142:6803/377511414","heartbeat_back_addr":"172.21.15.142:6807/377511414","heartbeat_front_addr":"172.21.15.142:6805/377511414","state":["exists","up"]},{"osd":3,"uuid":"4b946ad2-2da9-4e0b-9d61-d039d8669a42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6811","nonce":1086124477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6813","nonce":1086124477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6817","nonce":1086124477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6815","nonce":1086124477}]},"public_addr":"172.21.15.1:6811/1086124477","cluster_addr":"172.21.15.1:6813/1086124477","heartbeat_back_addr":"172.21.15.1:6817/1086124477","heartbeat_front_addr":"172.21.15.1:6815/1086124477","state":["exists","up"]},{"osd":4,"uuid":"43e36c43-47a0-4634-a45b-1da14a0e19d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6809","nonce":3095380305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6811","nonce":3095380305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6815","nonce":3095380305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6813","nonce":3095380305}]},"public_addr":"172.21.15.71:6809/3095380305","cluster_addr":"172.21.15.71:6811/3095380305","heartbeat_back_addr":"172.21.15.71:6815/3095380305","heartbeat_front_addr":"172.21.15.71:6813/3095380305","state":["exists","up"]},{"osd":5,"uuid":"eb48160e-ea7c-42f7-9477-be110bc8a34e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6809","nonce":4195149262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6811","nonce":4195149262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6815","nonce":4195149262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6813","nonce":4195149262}]},"public_addr":"172.21.15.142:6809/4195149262","cluster_addr":"172.21.15.142:6811/4195149262","heartbeat_back_addr":"172.21.15.142:6815/4195149262","heartbeat_front_addr":"172.21.15.142:6813/4195149262","state":["exists","up"]},{"osd":6,"uuid":"0ec4501a-5853-4ca6-b325-231f5b72335e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6819","nonce":1560153428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6821","nonce":1560153428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6825","nonce":1560153428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6823","nonce":1560153428}]},"public_addr":"172.21.15.1:6819/1560153428","cluster_addr":"172.21.15.1:6821/1560153428","heartbeat_back_addr":"172.21.15.1:6825/1560153428","heartbeat_front_addr":"172.21.15.1:6823/1560153428","state":["exists","up"]},{"osd":7,"uuid":"029debe8-9cdf-44f6-9aea-cabd1d185f93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6817","nonce":4080955538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6819","nonce":4080955538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6823","nonce":4080955538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6821","nonce":4080955538}]},"public_addr":"172.21.15.71:6817/4080955538","cluster_addr":"172.21.15.71:6819/4080955538","heartbeat_back_addr":"172.21.15.71:6823/4080955538","heartbeat_front_addr":"172.21.15.71:6821/4080955538","state":["exists","up"]},{"osd":8,"uuid":"7c90896c-9955-4c1b-aaca-7531dd5457f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6817","nonce":3942038209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6819","nonce":3942038209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6823","nonce":3942038209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6821","nonce":3942038209}]},"public_addr":"172.21.15.142:6817/3942038209","cluster_addr":"172.21.15.142:6819/3942038209","heartbeat_back_addr":"172.21.15.142:6823/3942038209","heartbeat_front_addr":"172.21.15.142:6821/3942038209","state":["exists","up"]},{"osd":9,"uuid":"7e165e68-31a8-4598-a577-de3e90ae5c5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6827","nonce":2716544850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6829","nonce":2716544850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6833","nonce":2716544850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6831","nonce":2716544850}]},"public_addr":"172.21.15.1:6827/2716544850","cluster_addr":"172.21.15.1:6829/2716544850","heartbeat_back_addr":"172.21.15.1:6833/2716544850","heartbeat_front_addr":"172.21.15.1:6831/2716544850","state":["exists","up"]},{"osd":10,"uuid":"48cd5fdb-0887-444a-aab8-a1de24282bb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6825","nonce":84691841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6827","nonce":84691841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6831","nonce":84691841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6829","nonce":84691841}]},"public_addr":"172.21.15.71:6825/84691841","cluster_addr":"172.21.15.71:6827/84691841","heartbeat_back_addr":"172.21.15.71:6831/84691841","heartbeat_front_addr":"172.21.15.71:6829/84691841","state":["exists","up"]},{"osd":11,"uuid":"f011dd27-11b0-4166-adfe-76756533163a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6825","nonce":2290179934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6827","nonce":2290179934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6831","nonce":2290179934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6829","nonce":2290179934}]},"public_addr":"172.21.15.142:6825/2290179934","cluster_addr":"172.21.15.142:6827/2290179934","heartbeat_back_addr":"172.21.15.142:6831/2290179934","heartbeat_front_addr":"172.21.15.142:6829/2290179934","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:20:44.060692+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:01.903817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:19.368258+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:40.028037+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:57.513361+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:22:14.805944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:22:35.188439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:22:53.014412+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:23:10.507420+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:23:31.849765+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:23:50.070612+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:24:07.388625+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":5,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/4092532060":"2024-02-03T15:19:25.836171+0000","172.21.15.1:6801/395498296":"2024-02-03T15:19:25.836171+0000","172.21.15.1:6800/395498296":"2024-02-03T15:19:25.836171+0000","172.21.15.1:0/1575373489":"2024-02-03T15:19:25.836171+0000","172.21.15.1:0/865983374":"2024-02-03T15:19:01.431596+0000","172.21.15.1:0/3165960227":"2024-02-03T15:19:01.431596+0000","172.21.15.1:0/2350830293":"2024-02-03T15:19:01.431596+0000","172.21.15.1:0/2959690250":"2024-02-03T15:19:25.836171+0000","172.21.15.1:6800/1162530732":"2024-02-03T15:19:01.431596+0000","172.21.15.1:6801/1162530732":"2024-02-03T15:19:01.431596+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T15:26:42.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.989+0000 7f7577c00700 1 -- 172.21.15.1:0/3077564406 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f75500a05a0 msgr2=0x7f75500a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:42.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.989+0000 7f7577c00700 1 --2- 172.21.15.1:0/3077564406 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f75500a05a0 0x7f75500a2a60 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f757019a140 tx=0x7f7560009260 comp rx=0 tx=0).stop 2024-02-02T15:26:42.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.989+0000 7f7577c00700 1 -- 172.21.15.1:0/3077564406 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7570101f30 msgr2=0x7f75701991b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:42.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.989+0000 7f7577c00700 1 --2- 172.21.15.1:0/3077564406 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7570101f30 0x7f75701991b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7558009fd0 tx=0x7f755800f690 comp rx=0 tx=0).stop 2024-02-02T15:26:42.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.989+0000 7f7577c00700 1 -- 172.21.15.1:0/3077564406 shutdown_connections 2024-02-02T15:26:42.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.989+0000 7f7577c00700 1 --2- 172.21.15.1:0/3077564406 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f75500a05a0 0x7f75500a2a60 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:42.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.989+0000 7f7577c00700 1 --2- 172.21.15.1:0/3077564406 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f75701028d0 0x7f75701a01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:42.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.989+0000 7f7577c00700 1 --2- 172.21.15.1:0/3077564406 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7570100d40 0x7f7570198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:42.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.989+0000 7f7577c00700 1 --2- 172.21.15.1:0/3077564406 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7570101f30 0x7f75701991b0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:42.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.989+0000 7f7577c00700 1 -- 172.21.15.1:0/3077564406 >> 172.21.15.1:0/3077564406 conn(0x7f75700fc290 msgr2=0x7f75700fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:42.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.989+0000 7f7577c00700 1 -- 172.21.15.1:0/3077564406 shutdown_connections 2024-02-02T15:26:42.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:42.989+0000 7f7577c00700 1 -- 172.21.15.1:0/3077564406 wait complete. 2024-02-02T15:26:43.008 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-02T15:21:22.471754+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-02T15:24:46.195779+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-02T15:24:47.226403+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-02-02T15:24:50.300462+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-02-02T15:26:43.008 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-02T15:26:43.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.209+0000 7efd45200700 1 -- 172.21.15.1:0/4224757027 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd40100e20 msgr2=0x7efd40101240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:43.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.209+0000 7efd45200700 1 --2- 172.21.15.1:0/4224757027 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd40100e20 0x7efd40101240 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7efd300098f0 tx=0x7efd30009c00 comp rx=0 tx=0).stop 2024-02-02T15:26:43.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.210+0000 7efd45200700 1 -- 172.21.15.1:0/4224757027 shutdown_connections 2024-02-02T15:26:43.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.210+0000 7efd45200700 1 --2- 172.21.15.1:0/4224757027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd40102930 0x7efd40109250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:43.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.210+0000 7efd45200700 1 --2- 172.21.15.1:0/4224757027 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd40100e20 0x7efd40101240 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:43.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.210+0000 7efd45200700 1 --2- 172.21.15.1:0/4224757027 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efd40101f70 0x7efd401023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:43.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.210+0000 7efd45200700 1 -- 172.21.15.1:0/4224757027 >> 172.21.15.1:0/4224757027 conn(0x7efd400fc390 msgr2=0x7efd400fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:43.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.210+0000 7efd45200700 1 -- 172.21.15.1:0/4224757027 shutdown_connections 2024-02-02T15:26:43.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.210+0000 7efd45200700 1 -- 172.21.15.1:0/4224757027 wait complete. 2024-02-02T15:26:43.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.211+0000 7efd45200700 1 Processor -- start 2024-02-02T15:26:43.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.211+0000 7efd45200700 1 -- start start 2024-02-02T15:26:43.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.211+0000 7efd45200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd40100e20 0x7efd40196b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:43.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.211+0000 7efd45200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efd40101f70 0x7efd401970a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:43.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.211+0000 7efd45200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd40102930 0x7efd4019e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:43.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.211+0000 7efd45200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd4019e6e0 con 0x7efd40101f70 2024-02-02T15:26:43.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.211+0000 7efd45200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd4019e820 con 0x7efd40102930 2024-02-02T15:26:43.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.211+0000 7efd45200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd4019e9b0 con 0x7efd40100e20 2024-02-02T15:26:43.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.211+0000 7efd3d400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efd40101f70 0x7efd401970a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:43.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.211+0000 7efd3d400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efd40101f70 0x7efd401970a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53312/0 (socket says 172.21.15.1:53312) 2024-02-02T15:26:43.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.211+0000 7efd3d400700 1 -- 172.21.15.1:0/7712011 learned_addr learned my addr 172.21.15.1:0/7712011 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:43.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.211+0000 7efd3de00700 1 --2- 172.21.15.1:0/7712011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd40100e20 0x7efd40196b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:43.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.212+0000 7efd3e800700 1 --2- 172.21.15.1:0/7712011 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd40102930 0x7efd4019e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:43.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.212+0000 7efd3d400700 1 -- 172.21.15.1:0/7712011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd40100e20 msgr2=0x7efd40196b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:43.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.212+0000 7efd3d400700 1 --2- 172.21.15.1:0/7712011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd40100e20 0x7efd40196b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:43.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.212+0000 7efd3d400700 1 -- 172.21.15.1:0/7712011 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd40102930 msgr2=0x7efd4019e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:43.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.212+0000 7efd3d400700 1 --2- 172.21.15.1:0/7712011 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd40102930 0x7efd4019e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:43.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.212+0000 7efd3d400700 1 -- 172.21.15.1:0/7712011 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd280095f0 con 0x7efd40101f70 2024-02-02T15:26:43.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.212+0000 7efd3d400700 1 --2- 172.21.15.1:0/7712011 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efd40101f70 0x7efd401970a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7efd2800ec10 tx=0x7efd2800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:43.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.212+0000 7efd36a00700 1 -- 172.21.15.1:0/7712011 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd2800cc80 con 0x7efd40101f70 2024-02-02T15:26:43.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.212+0000 7efd45200700 1 -- 172.21.15.1:0/7712011 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd38009c40 con 0x7efd40101f70 2024-02-02T15:26:43.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.212+0000 7efd36a00700 1 -- 172.21.15.1:0/7712011 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd280044e0 con 0x7efd40101f70 2024-02-02T15:26:43.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.212+0000 7efd36a00700 1 -- 172.21.15.1:0/7712011 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd28010430 con 0x7efd40101f70 2024-02-02T15:26:43.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.213+0000 7efd45200700 1 -- 172.21.15.1:0/7712011 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd30009520 con 0x7efd40101f70 2024-02-02T15:26:43.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.214+0000 7efd45200700 1 -- 172.21.15.1:0/7712011 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd4010c670 con 0x7efd40101f70 2024-02-02T15:26:43.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.215+0000 7efd36a00700 1 -- 172.21.15.1:0/7712011 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) v1 ==== 135069+0+0 (secure 0 0 0) 0x7efd28003f90 con 0x7efd40101f70 2024-02-02T15:26:43.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.217+0000 7efd36a00700 1 --2- 172.21.15.1:0/7712011 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7efd200a05a0 0x7efd200a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:43.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.217+0000 7efd36a00700 1 -- 172.21.15.1:0/7712011 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9107+0+0 (secure 0 0 0) 0x7efd28014070 con 0x7efd40101f70 2024-02-02T15:26:43.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.217+0000 7efd3de00700 1 --2- 172.21.15.1:0/7712011 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7efd200a05a0 0x7efd200a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:43.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.219+0000 7efd3de00700 1 --2- 172.21.15.1:0/7712011 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7efd200a05a0 0x7efd200a2a60 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7efd30005e90 tx=0x7efd30019040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:43.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.222+0000 7efd36a00700 1 -- 172.21.15.1:0/7712011 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7efd28095980 con 0x7efd40101f70 2024-02-02T15:26:43.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.431+0000 7efd45200700 1 -- 172.21.15.1:0/7712011 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efd40107d90 con 0x7efd40101f70 2024-02-02T15:26:43.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.432+0000 7efd36a00700 1 -- 172.21.15.1:0/7712011 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7efd28098fa0 con 0x7efd40101f70 2024-02-02T15:26:43.432 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-02-02T15:26:43.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.435+0000 7efd45200700 1 -- 172.21.15.1:0/7712011 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7efd200a05a0 msgr2=0x7efd200a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:43.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.435+0000 7efd45200700 1 --2- 172.21.15.1:0/7712011 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7efd200a05a0 0x7efd200a2a60 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7efd30005e90 tx=0x7efd30019040 comp rx=0 tx=0).stop 2024-02-02T15:26:43.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.435+0000 7efd45200700 1 -- 172.21.15.1:0/7712011 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efd40101f70 msgr2=0x7efd401970a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:43.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.435+0000 7efd45200700 1 --2- 172.21.15.1:0/7712011 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efd40101f70 0x7efd401970a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7efd2800ec10 tx=0x7efd2800ef20 comp rx=0 tx=0).stop 2024-02-02T15:26:43.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.435+0000 7efd45200700 1 -- 172.21.15.1:0/7712011 shutdown_connections 2024-02-02T15:26:43.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.435+0000 7efd45200700 1 --2- 172.21.15.1:0/7712011 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd40102930 0x7efd4019e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:43.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.435+0000 7efd45200700 1 --2- 172.21.15.1:0/7712011 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7efd200a05a0 0x7efd200a2a60 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:43.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.435+0000 7efd45200700 1 --2- 172.21.15.1:0/7712011 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd40100e20 0x7efd40196b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:43.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.435+0000 7efd45200700 1 --2- 172.21.15.1:0/7712011 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efd40101f70 0x7efd401970a0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:43.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.435+0000 7efd45200700 1 -- 172.21.15.1:0/7712011 >> 172.21.15.1:0/7712011 conn(0x7efd400fc390 msgr2=0x7efd400fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:43.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.435+0000 7efd45200700 1 -- 172.21.15.1:0/7712011 shutdown_connections 2024-02-02T15:26:43.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.435+0000 7efd45200700 1 -- 172.21.15.1:0/7712011 wait complete. 2024-02-02T15:26:43.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:43 smithi001 ceph-mon[35470]: pgmap v296: 129 pgs: 129 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 85 B/s wr, 11 op/s 2024-02-02T15:26:43.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:43 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/4031181402' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-02T15:26:43.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:43 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/3077564406' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-02T15:26:43.452 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-02T15:26:43.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:43 smithi071 ceph-mon[44707]: pgmap v296: 129 pgs: 129 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 85 B/s wr, 11 op/s 2024-02-02T15:26:43.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:43 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/4031181402' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-02T15:26:43.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:43 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/3077564406' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-02T15:26:43.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.653+0000 7f8ef1600700 1 -- 172.21.15.1:0/3609272196 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8eec108ce0 msgr2=0x7f8eec10b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:43.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.653+0000 7f8ef1600700 1 --2- 172.21.15.1:0/3609272196 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8eec108ce0 0x7f8eec10b170 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f8ee400b0a0 tx=0x7f8ee400b3b0 comp rx=0 tx=0).stop 2024-02-02T15:26:43.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.653+0000 7f8ef1600700 1 -- 172.21.15.1:0/3609272196 shutdown_connections 2024-02-02T15:26:43.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.653+0000 7f8ef1600700 1 --2- 172.21.15.1:0/3609272196 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8eec108ce0 0x7f8eec10b170 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:43.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.653+0000 7f8ef1600700 1 --2- 172.21.15.1:0/3609272196 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8eec074d80 0x7f8eec0731d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:43.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.653+0000 7f8ef1600700 1 --2- 172.21.15.1:0/3609272196 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8eec0737a0 0x7f8eec073c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:43.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.653+0000 7f8ef1600700 1 -- 172.21.15.1:0/3609272196 >> 172.21.15.1:0/3609272196 conn(0x7f8eec0fc390 msgr2=0x7f8eec0fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:43.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.653+0000 7f8ef1600700 1 -- 172.21.15.1:0/3609272196 shutdown_connections 2024-02-02T15:26:43.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.654+0000 7f8ef1600700 1 -- 172.21.15.1:0/3609272196 wait complete. 2024-02-02T15:26:43.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.656+0000 7f8ef1600700 1 Processor -- start 2024-02-02T15:26:43.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.656+0000 7f8ef1600700 1 -- start start 2024-02-02T15:26:43.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.656+0000 7f8ef1600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8eec0737a0 0x7f8eec198df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:43.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.656+0000 7f8ef1600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8eec074d80 0x7f8eec199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:43.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.656+0000 7f8ef1600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8eec108ce0 0x7f8eec1a03a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:43.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.656+0000 7f8ef1600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eec1a0970 con 0x7f8eec108ce0 2024-02-02T15:26:43.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.656+0000 7f8ef1600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eec1a0ab0 con 0x7f8eec0737a0 2024-02-02T15:26:43.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.656+0000 7f8ef1600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eec1a0c40 con 0x7f8eec074d80 2024-02-02T15:26:43.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.656+0000 7f8eeb400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8eec108ce0 0x7f8eec1a03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:43.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.656+0000 7f8eeaa00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8eec0737a0 0x7f8eec198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:43.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.656+0000 7f8eeb400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8eec108ce0 0x7f8eec1a03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53326/0 (socket says 172.21.15.1:53326) 2024-02-02T15:26:43.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.656+0000 7f8eeb400700 1 -- 172.21.15.1:0/3132040401 learned_addr learned my addr 172.21.15.1:0/3132040401 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:43.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.656+0000 7f8eea000700 1 --2- 172.21.15.1:0/3132040401 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8eec074d80 0x7f8eec199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:43.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.656+0000 7f8eeb400700 1 -- 172.21.15.1:0/3132040401 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8eec074d80 msgr2=0x7f8eec199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:43.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.656+0000 7f8eeb400700 1 --2- 172.21.15.1:0/3132040401 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8eec074d80 0x7f8eec199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:43.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.656+0000 7f8eeb400700 1 -- 172.21.15.1:0/3132040401 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8eec0737a0 msgr2=0x7f8eec198df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:43.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.656+0000 7f8eeb400700 1 --2- 172.21.15.1:0/3132040401 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8eec0737a0 0x7f8eec198df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:43.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.656+0000 7f8eeb400700 1 -- 172.21.15.1:0/3132040401 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ee4009d00 con 0x7f8eec108ce0 2024-02-02T15:26:43.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.656+0000 7f8eeb400700 1 --2- 172.21.15.1:0/3132040401 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8eec108ce0 0x7f8eec1a03a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f8ee4009fd0 tx=0x7f8ee400bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:43.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.656+0000 7f8ee3400700 1 -- 172.21.15.1:0/3132040401 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee400e040 con 0x7f8eec108ce0 2024-02-02T15:26:43.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.656+0000 7f8ee3400700 1 -- 172.21.15.1:0/3132040401 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ee40094a0 con 0x7f8eec108ce0 2024-02-02T15:26:43.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.656+0000 7f8ef1600700 1 -- 172.21.15.1:0/3132040401 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8eec1a0ec0 con 0x7f8eec108ce0 2024-02-02T15:26:43.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.656+0000 7f8ee3400700 1 -- 172.21.15.1:0/3132040401 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee401b6c0 con 0x7f8eec108ce0 2024-02-02T15:26:43.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.656+0000 7f8ef1600700 1 -- 172.21.15.1:0/3132040401 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8eec1a1230 con 0x7f8eec108ce0 2024-02-02T15:26:43.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.657+0000 7f8ef1600700 1 -- 172.21.15.1:0/3132040401 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8eec107aa0 con 0x7f8eec108ce0 2024-02-02T15:26:43.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.660+0000 7f8ee3400700 1 -- 172.21.15.1:0/3132040401 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) v1 ==== 135069+0+0 (secure 0 0 0) 0x7f8ee401b820 con 0x7f8eec108ce0 2024-02-02T15:26:43.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.662+0000 7f8ee3400700 1 --2- 172.21.15.1:0/3132040401 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f8ecc0a0520 0x7f8ecc0a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:43.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.662+0000 7f8ee3400700 1 -- 172.21.15.1:0/3132040401 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f8ee40cd5c0 con 0x7f8eec108ce0 2024-02-02T15:26:43.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.662+0000 7f8eeaa00700 1 --2- 172.21.15.1:0/3132040401 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f8ecc0a0520 0x7f8ecc0a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:43.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.662+0000 7f8eeaa00700 1 --2- 172.21.15.1:0/3132040401 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f8ecc0a0520 0x7f8ecc0a29e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8edc009c60 tx=0x7f8edc009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:43.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.663+0000 7f8ee3400700 1 -- 172.21.15.1:0/3132040401 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f8ee4096e20 con 0x7f8eec108ce0 2024-02-02T15:26:43.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:43 smithi142 ceph-mon[45728]: pgmap v296: 129 pgs: 129 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 85 B/s wr, 11 op/s 2024-02-02T15:26:43.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:43 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/4031181402' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-02T15:26:43.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:43 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/3077564406' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-02T15:26:43.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.881+0000 7f8ef1600700 1 -- 172.21.15.1:0/3132040401 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8eec02d0c0 con 0x7f8eec108ce0 2024-02-02T15:26:43.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.882+0000 7f8ee3400700 1 -- 172.21.15.1:0/3132040401 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8ee409a440 con 0x7f8eec108ce0 2024-02-02T15:26:43.881 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-02-02T15:26:43.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.884+0000 7f8ef1600700 1 -- 172.21.15.1:0/3132040401 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f8ecc0a0520 msgr2=0x7f8ecc0a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:43.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.884+0000 7f8ef1600700 1 --2- 172.21.15.1:0/3132040401 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f8ecc0a0520 0x7f8ecc0a29e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8edc009c60 tx=0x7f8edc009260 comp rx=0 tx=0).stop 2024-02-02T15:26:43.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.884+0000 7f8ef1600700 1 -- 172.21.15.1:0/3132040401 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8eec108ce0 msgr2=0x7f8eec1a03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:43.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.884+0000 7f8ef1600700 1 --2- 172.21.15.1:0/3132040401 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8eec108ce0 0x7f8eec1a03a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f8ee4009fd0 tx=0x7f8ee400bce0 comp rx=0 tx=0).stop 2024-02-02T15:26:43.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.884+0000 7f8ef1600700 1 -- 172.21.15.1:0/3132040401 shutdown_connections 2024-02-02T15:26:43.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.884+0000 7f8ef1600700 1 --2- 172.21.15.1:0/3132040401 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8eec108ce0 0x7f8eec1a03a0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:43.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.884+0000 7f8ef1600700 1 --2- 172.21.15.1:0/3132040401 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8eec0737a0 0x7f8eec198df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:43.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.884+0000 7f8ef1600700 1 --2- 172.21.15.1:0/3132040401 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f8ecc0a0520 0x7f8ecc0a29e0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:43.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.884+0000 7f8ef1600700 1 --2- 172.21.15.1:0/3132040401 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8eec074d80 0x7f8eec199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:43.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.884+0000 7f8ef1600700 1 -- 172.21.15.1:0/3132040401 >> 172.21.15.1:0/3132040401 conn(0x7f8eec0fc390 msgr2=0x7f8eec104be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:43.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.884+0000 7f8ef1600700 1 -- 172.21.15.1:0/3132040401 shutdown_connections 2024-02-02T15:26:43.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:43.885+0000 7f8ef1600700 1 -- 172.21.15.1:0/3132040401 wait complete. 2024-02-02T15:26:43.902 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-02T15:26:44.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.104+0000 7feee5000700 1 -- 172.21.15.1:0/1078682013 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feee0100e20 msgr2=0x7feee0101240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:44.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.104+0000 7feee5000700 1 --2- 172.21.15.1:0/1078682013 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feee0100e20 0x7feee0101240 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7feed0009990 tx=0x7feed0009ca0 comp rx=0 tx=0).stop 2024-02-02T15:26:44.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.105+0000 7feee5000700 1 -- 172.21.15.1:0/1078682013 shutdown_connections 2024-02-02T15:26:44.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.105+0000 7feee5000700 1 --2- 172.21.15.1:0/1078682013 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feee0102930 0x7feee0109250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:44.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.105+0000 7feee5000700 1 --2- 172.21.15.1:0/1078682013 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feee0100e20 0x7feee0101240 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:44.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.105+0000 7feee5000700 1 --2- 172.21.15.1:0/1078682013 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feee0101f70 0x7feee01023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:44.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.105+0000 7feee5000700 1 -- 172.21.15.1:0/1078682013 >> 172.21.15.1:0/1078682013 conn(0x7feee00fc390 msgr2=0x7feee00fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:44.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.105+0000 7feee5000700 1 -- 172.21.15.1:0/1078682013 shutdown_connections 2024-02-02T15:26:44.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.105+0000 7feee5000700 1 -- 172.21.15.1:0/1078682013 wait complete. 2024-02-02T15:26:44.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.106+0000 7feee5000700 1 Processor -- start 2024-02-02T15:26:44.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.106+0000 7feee5000700 1 -- start start 2024-02-02T15:26:44.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.106+0000 7feee5000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feee0100e20 0x7feee0198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:44.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.106+0000 7feee5000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feee0101f70 0x7feee01992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:44.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.106+0000 7feee5000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feee0102930 0x7feee01a0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:44.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.106+0000 7feee5000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee01a08f0 con 0x7feee0101f70 2024-02-02T15:26:44.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.106+0000 7feee5000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee01a0a60 con 0x7feee0102930 2024-02-02T15:26:44.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.106+0000 7feee5000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee01a0c20 con 0x7feee0100e20 2024-02-02T15:26:44.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.106+0000 7feedd400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feee0101f70 0x7feee01992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:44.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.106+0000 7feedd400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feee0101f70 0x7feee01992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53340/0 (socket says 172.21.15.1:53340) 2024-02-02T15:26:44.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.106+0000 7feedd400700 1 -- 172.21.15.1:0/1004252218 learned_addr learned my addr 172.21.15.1:0/1004252218 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:44.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.107+0000 7feedde00700 1 --2- 172.21.15.1:0/1004252218 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feee0100e20 0x7feee0198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:44.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.107+0000 7feede800700 1 --2- 172.21.15.1:0/1004252218 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feee0102930 0x7feee01a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:44.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.107+0000 7feedd400700 1 -- 172.21.15.1:0/1004252218 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feee0100e20 msgr2=0x7feee0198db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:44.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.107+0000 7feedd400700 1 --2- 172.21.15.1:0/1004252218 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feee0100e20 0x7feee0198db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:44.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.107+0000 7feedd400700 1 -- 172.21.15.1:0/1004252218 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feee0102930 msgr2=0x7feee01a0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:44.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.107+0000 7feedd400700 1 --2- 172.21.15.1:0/1004252218 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feee0102930 0x7feee01a0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:44.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.107+0000 7feedd400700 1 -- 172.21.15.1:0/1004252218 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feed00095f0 con 0x7feee0101f70 2024-02-02T15:26:44.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.107+0000 7feedd400700 1 --2- 172.21.15.1:0/1004252218 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feee0101f70 0x7feee01992f0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7feec800e9a0 tx=0x7feec800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:44.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.107+0000 7feed6a00700 1 -- 172.21.15.1:0/1004252218 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec8009790 con 0x7feee0101f70 2024-02-02T15:26:44.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.107+0000 7feed6a00700 1 -- 172.21.15.1:0/1004252218 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feec8010430 con 0x7feee0101f70 2024-02-02T15:26:44.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.107+0000 7feee5000700 1 -- 172.21.15.1:0/1004252218 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feee01a0f00 con 0x7feee0101f70 2024-02-02T15:26:44.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.107+0000 7feed6a00700 1 -- 172.21.15.1:0/1004252218 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec8018680 con 0x7feee0101f70 2024-02-02T15:26:44.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.108+0000 7feee5000700 1 -- 172.21.15.1:0/1004252218 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feee01a1380 con 0x7feee0101f70 2024-02-02T15:26:44.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.108+0000 7feee5000700 1 -- 172.21.15.1:0/1004252218 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feee0193460 con 0x7feee0101f70 2024-02-02T15:26:44.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.110+0000 7feed6a00700 1 -- 172.21.15.1:0/1004252218 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) v1 ==== 135069+0+0 (secure 0 0 0) 0x7feec80105a0 con 0x7feee0101f70 2024-02-02T15:26:44.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.112+0000 7feed6a00700 1 --2- 172.21.15.1:0/1004252218 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7feec00a05a0 0x7feec00a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:44.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.112+0000 7feed6a00700 1 -- 172.21.15.1:0/1004252218 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9107+0+0 (secure 0 0 0) 0x7feec8014070 con 0x7feee0101f70 2024-02-02T15:26:44.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.112+0000 7feedde00700 1 --2- 172.21.15.1:0/1004252218 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7feec00a05a0 0x7feec00a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:44.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.113+0000 7feedde00700 1 --2- 172.21.15.1:0/1004252218 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7feec00a05a0 0x7feec00a2a60 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7feed000b2e0 tx=0x7feed0005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:44.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.114+0000 7feed6a00700 1 -- 172.21.15.1:0/1004252218 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7feec80958d0 con 0x7feee0101f70 2024-02-02T15:26:44.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.326+0000 7feee5000700 1 -- 172.21.15.1:0/1004252218 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7feee002ce00 con 0x7feee0101f70 2024-02-02T15:26:44.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.326+0000 7feed6a00700 1 -- 172.21.15.1:0/1004252218 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7feec8026070 con 0x7feee0101f70 2024-02-02T15:26:44.326 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 64 2024-02-02T15:26:44.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.329+0000 7feee5000700 1 -- 172.21.15.1:0/1004252218 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7feec00a05a0 msgr2=0x7feec00a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:44.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.329+0000 7feee5000700 1 --2- 172.21.15.1:0/1004252218 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7feec00a05a0 0x7feec00a2a60 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7feed000b2e0 tx=0x7feed0005fb0 comp rx=0 tx=0).stop 2024-02-02T15:26:44.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.329+0000 7feee5000700 1 -- 172.21.15.1:0/1004252218 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feee0101f70 msgr2=0x7feee01992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:44.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.329+0000 7feee5000700 1 --2- 172.21.15.1:0/1004252218 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feee0101f70 0x7feee01992f0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7feec800e9a0 tx=0x7feec800ecb0 comp rx=0 tx=0).stop 2024-02-02T15:26:44.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.329+0000 7feee5000700 1 -- 172.21.15.1:0/1004252218 shutdown_connections 2024-02-02T15:26:44.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.329+0000 7feee5000700 1 --2- 172.21.15.1:0/1004252218 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feee0102930 0x7feee01a0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:44.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.329+0000 7feee5000700 1 --2- 172.21.15.1:0/1004252218 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7feec00a05a0 0x7feec00a2a60 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:44.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.329+0000 7feee5000700 1 --2- 172.21.15.1:0/1004252218 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feee0100e20 0x7feee0198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:44.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.329+0000 7feee5000700 1 --2- 172.21.15.1:0/1004252218 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feee0101f70 0x7feee01992f0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:44.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.329+0000 7feee5000700 1 -- 172.21.15.1:0/1004252218 >> 172.21.15.1:0/1004252218 conn(0x7feee00fc390 msgr2=0x7feee0107990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:44.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.329+0000 7feee5000700 1 -- 172.21.15.1:0/1004252218 shutdown_connections 2024-02-02T15:26:44.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.329+0000 7feee5000700 1 -- 172.21.15.1:0/1004252218 wait complete. 2024-02-02T15:26:44.347 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-02-02T15:26:44.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.549+0000 7f0f9f200700 1 -- 172.21.15.1:0/3180196332 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0f98101f70 msgr2=0x7f0f981023f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:44.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.549+0000 7f0f9f200700 1 --2- 172.21.15.1:0/3180196332 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0f98101f70 0x7f0f981023f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f0f80009990 tx=0x7f0f80009ca0 comp rx=0 tx=0).stop 2024-02-02T15:26:44.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.549+0000 7f0f9f200700 1 -- 172.21.15.1:0/3180196332 shutdown_connections 2024-02-02T15:26:44.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.549+0000 7f0f9f200700 1 --2- 172.21.15.1:0/3180196332 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0f98102930 0x7f0f98109250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:44.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.549+0000 7f0f9f200700 1 --2- 172.21.15.1:0/3180196332 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f98100e20 0x7f0f98101240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:44.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.549+0000 7f0f9f200700 1 --2- 172.21.15.1:0/3180196332 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0f98101f70 0x7f0f981023f0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:44.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.549+0000 7f0f9f200700 1 -- 172.21.15.1:0/3180196332 >> 172.21.15.1:0/3180196332 conn(0x7f0f980fc390 msgr2=0x7f0f980fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:44.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.549+0000 7f0f9f200700 1 -- 172.21.15.1:0/3180196332 shutdown_connections 2024-02-02T15:26:44.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.549+0000 7f0f9f200700 1 -- 172.21.15.1:0/3180196332 wait complete. 2024-02-02T15:26:44.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.550+0000 7f0f9f200700 1 Processor -- start 2024-02-02T15:26:44.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.550+0000 7f0f9f200700 1 -- start start 2024-02-02T15:26:44.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.550+0000 7f0f9f200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0f98100e20 0x7f0f98074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:44.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.550+0000 7f0f9f200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0f98101f70 0x7f0f98073130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:44.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.551+0000 7f0f9f200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f98102930 0x7f0f98073670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:44.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.551+0000 7f0f9f200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f98073e10 con 0x7f0f98100e20 2024-02-02T15:26:44.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.551+0000 7f0f9f200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f98073f80 con 0x7f0f98101f70 2024-02-02T15:26:44.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.551+0000 7f0f9f200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f981a0b40 con 0x7f0f98102930 2024-02-02T15:26:44.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.551+0000 7f0f97e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0f98100e20 0x7f0f98074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:44.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.551+0000 7f0f97e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0f98100e20 0x7f0f98074af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53360/0 (socket says 172.21.15.1:53360) 2024-02-02T15:26:44.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.551+0000 7f0f97e00700 1 -- 172.21.15.1:0/1707433336 learned_addr learned my addr 172.21.15.1:0/1707433336 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:44.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.551+0000 7f0f9d200700 1 --2- 172.21.15.1:0/1707433336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f98102930 0x7f0f98073670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:44.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.551+0000 7f0f97400700 1 --2- 172.21.15.1:0/1707433336 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0f98101f70 0x7f0f98073130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:44.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.551+0000 7f0f97e00700 1 -- 172.21.15.1:0/1707433336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f98102930 msgr2=0x7f0f98073670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:44.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.551+0000 7f0f97e00700 1 --2- 172.21.15.1:0/1707433336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f98102930 0x7f0f98073670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:44.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.551+0000 7f0f97e00700 1 -- 172.21.15.1:0/1707433336 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0f98101f70 msgr2=0x7f0f98073130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:44.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.551+0000 7f0f97e00700 1 --2- 172.21.15.1:0/1707433336 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0f98101f70 0x7f0f98073130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:44.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.551+0000 7f0f97e00700 1 -- 172.21.15.1:0/1707433336 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f800095f0 con 0x7f0f98100e20 2024-02-02T15:26:44.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.551+0000 7f0f97400700 1 --2- 172.21.15.1:0/1707433336 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0f98101f70 0x7f0f98073130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:26:44.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.551+0000 7f0f97e00700 1 --2- 172.21.15.1:0/1707433336 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0f98100e20 0x7f0f98074af0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f0f8800e9a0 tx=0x7f0f8800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:44.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.552+0000 7f0f94c00700 1 -- 172.21.15.1:0/1707433336 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f88009790 con 0x7f0f98100e20 2024-02-02T15:26:44.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.552+0000 7f0f94c00700 1 -- 172.21.15.1:0/1707433336 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f8800cbd0 con 0x7f0f98100e20 2024-02-02T15:26:44.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.552+0000 7f0f9f200700 1 -- 172.21.15.1:0/1707433336 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f981a0e20 con 0x7f0f98100e20 2024-02-02T15:26:44.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.552+0000 7f0f94c00700 1 -- 172.21.15.1:0/1707433336 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f88018680 con 0x7f0f98100e20 2024-02-02T15:26:44.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.552+0000 7f0f9f200700 1 -- 172.21.15.1:0/1707433336 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f981a1250 con 0x7f0f98100e20 2024-02-02T15:26:44.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.553+0000 7f0f9f200700 1 -- 172.21.15.1:0/1707433336 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f98107d40 con 0x7f0f98100e20 2024-02-02T15:26:44.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.555+0000 7f0f94c00700 1 -- 172.21.15.1:0/1707433336 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) v1 ==== 135069+0+0 (secure 0 0 0) 0x7f0f8800cd40 con 0x7f0f98100e20 2024-02-02T15:26:44.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.556+0000 7f0f94c00700 1 --2- 172.21.15.1:0/1707433336 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f0f780a05a0 0x7f0f780a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:44.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.556+0000 7f0f94c00700 1 -- 172.21.15.1:0/1707433336 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f0f88014070 con 0x7f0f98100e20 2024-02-02T15:26:44.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.556+0000 7f0f97400700 1 --2- 172.21.15.1:0/1707433336 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f0f780a05a0 0x7f0f780a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:44.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.557+0000 7f0f97400700 1 --2- 172.21.15.1:0/1707433336 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f0f780a05a0 0x7f0f780a2a60 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0f8000b2e0 tx=0x7f0f80005fd0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:44.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.560+0000 7f0f94c00700 1 -- 172.21.15.1:0/1707433336 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f0f88052110 con 0x7f0f98100e20 2024-02-02T15:26:44.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:44 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/7712011' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-02T15:26:44.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:44 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/3132040401' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-02T15:26:44.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:44 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/7712011' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-02T15:26:44.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:44 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/3132040401' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-02T15:26:44.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:44 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/7712011' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-02T15:26:44.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:44 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/3132040401' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-02T15:26:44.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.771+0000 7f0f9f200700 1 -- 172.21.15.1:0/1707433336 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f0f980619f0 con 0x7f0f98100e20 2024-02-02T15:26:44.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.771+0000 7f0f94c00700 1 -- 172.21.15.1:0/1707433336 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v88) v1 ==== 103+0+11 (secure 0 0 0) 0x7f0f88095720 con 0x7f0f98100e20 2024-02-02T15:26:44.770 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-02-02T15:26:44.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.774+0000 7f0f9f200700 1 -- 172.21.15.1:0/1707433336 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f0f780a05a0 msgr2=0x7f0f780a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:44.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.774+0000 7f0f9f200700 1 --2- 172.21.15.1:0/1707433336 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f0f780a05a0 0x7f0f780a2a60 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0f8000b2e0 tx=0x7f0f80005fd0 comp rx=0 tx=0).stop 2024-02-02T15:26:44.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.774+0000 7f0f9f200700 1 -- 172.21.15.1:0/1707433336 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0f98100e20 msgr2=0x7f0f98074af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:44.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.774+0000 7f0f9f200700 1 --2- 172.21.15.1:0/1707433336 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0f98100e20 0x7f0f98074af0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f0f8800e9a0 tx=0x7f0f8800ecb0 comp rx=0 tx=0).stop 2024-02-02T15:26:44.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.774+0000 7f0f9f200700 1 -- 172.21.15.1:0/1707433336 shutdown_connections 2024-02-02T15:26:44.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.774+0000 7f0f9f200700 1 --2- 172.21.15.1:0/1707433336 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f0f780a05a0 0x7f0f780a2a60 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:44.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.774+0000 7f0f9f200700 1 --2- 172.21.15.1:0/1707433336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0f98102930 0x7f0f98073670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:44.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.774+0000 7f0f9f200700 1 --2- 172.21.15.1:0/1707433336 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0f98100e20 0x7f0f98074af0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:44.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.774+0000 7f0f9f200700 1 --2- 172.21.15.1:0/1707433336 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0f98101f70 0x7f0f98073130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:44.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.774+0000 7f0f9f200700 1 -- 172.21.15.1:0/1707433336 >> 172.21.15.1:0/1707433336 conn(0x7f0f980fc390 msgr2=0x7f0f980fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:44.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.774+0000 7f0f9f200700 1 -- 172.21.15.1:0/1707433336 shutdown_connections 2024-02-02T15:26:44.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.774+0000 7f0f9f200700 1 -- 172.21.15.1:0/1707433336 wait complete. 2024-02-02T15:26:44.792 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-02-02T15:26:44.792 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T15:26:44.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.991+0000 7f2832e00700 1 -- 172.21.15.1:0/4163247087 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f282c100d50 msgr2=0x7f282c101170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:44.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.991+0000 7f2832e00700 1 --2- 172.21.15.1:0/4163247087 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f282c100d50 0x7f282c101170 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f281c009920 tx=0x7f281c009c30 comp rx=0 tx=0).stop 2024-02-02T15:26:44.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.991+0000 7f2832e00700 1 -- 172.21.15.1:0/4163247087 shutdown_connections 2024-02-02T15:26:44.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.991+0000 7f2832e00700 1 --2- 172.21.15.1:0/4163247087 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f282c1028e0 0x7f282c109190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:44.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.991+0000 7f2832e00700 1 --2- 172.21.15.1:0/4163247087 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f282c100d50 0x7f282c101170 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:44.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.991+0000 7f2832e00700 1 --2- 172.21.15.1:0/4163247087 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f282c101f40 0x7f282c1023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:44.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.991+0000 7f2832e00700 1 -- 172.21.15.1:0/4163247087 >> 172.21.15.1:0/4163247087 conn(0x7f282c0fc2c0 msgr2=0x7f282c0fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:44.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.991+0000 7f2832e00700 1 -- 172.21.15.1:0/4163247087 shutdown_connections 2024-02-02T15:26:44.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.991+0000 7f2832e00700 1 -- 172.21.15.1:0/4163247087 wait complete. 2024-02-02T15:26:44.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.992+0000 7f2832e00700 1 Processor -- start 2024-02-02T15:26:44.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.993+0000 7f2832e00700 1 -- start start 2024-02-02T15:26:44.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.993+0000 7f2832e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f282c100d50 0x7f282c198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:44.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.993+0000 7f2832e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f282c101f40 0x7f282c199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:44.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.993+0000 7f2832e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f282c1028e0 0x7f282c1a02c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:44.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.993+0000 7f2832e00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f282c1a0890 con 0x7f282c101f40 2024-02-02T15:26:44.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.993+0000 7f2832e00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f282c1a0a00 con 0x7f282c100d50 2024-02-02T15:26:44.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.993+0000 7f2832e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f282c1a0bc0 con 0x7f282c1028e0 2024-02-02T15:26:44.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.993+0000 7f282b400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f282c101f40 0x7f282c199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:44.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.994+0000 7f282b400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f282c101f40 0x7f282c199290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53380/0 (socket says 172.21.15.1:53380) 2024-02-02T15:26:44.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.994+0000 7f282b400700 1 -- 172.21.15.1:0/3609866608 learned_addr learned my addr 172.21.15.1:0/3609866608 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:44.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.994+0000 7f282be00700 1 --2- 172.21.15.1:0/3609866608 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f282c100d50 0x7f282c198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:44.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.994+0000 7f2830e00700 1 --2- 172.21.15.1:0/3609866608 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f282c1028e0 0x7f282c1a02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:44.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.994+0000 7f282b400700 1 -- 172.21.15.1:0/3609866608 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f282c1028e0 msgr2=0x7f282c1a02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:44.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.994+0000 7f282b400700 1 --2- 172.21.15.1:0/3609866608 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f282c1028e0 0x7f282c1a02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:44.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.994+0000 7f282b400700 1 -- 172.21.15.1:0/3609866608 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f282c100d50 msgr2=0x7f282c198d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:44.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.994+0000 7f282b400700 1 --2- 172.21.15.1:0/3609866608 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f282c100d50 0x7f282c198d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:44.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.994+0000 7f282b400700 1 -- 172.21.15.1:0/3609866608 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28140095f0 con 0x7f282c101f40 2024-02-02T15:26:44.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.994+0000 7f282be00700 1 --2- 172.21.15.1:0/3609866608 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f282c100d50 0x7f282c198d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:26:44.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.994+0000 7f282b400700 1 --2- 172.21.15.1:0/3609866608 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f282c101f40 0x7f282c199290 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f281400ec10 tx=0x7f281400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:44.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.994+0000 7f2828c00700 1 -- 172.21.15.1:0/3609866608 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f281400cc90 con 0x7f282c101f40 2024-02-02T15:26:44.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.994+0000 7f2832e00700 1 -- 172.21.15.1:0/3609866608 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f281c009520 con 0x7f282c101f40 2024-02-02T15:26:44.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.994+0000 7f2828c00700 1 -- 172.21.15.1:0/3609866608 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f28140044e0 con 0x7f282c101f40 2024-02-02T15:26:44.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.995+0000 7f2832e00700 1 -- 172.21.15.1:0/3609866608 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f282c1a11c0 con 0x7f282c101f40 2024-02-02T15:26:44.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.995+0000 7f2828c00700 1 -- 172.21.15.1:0/3609866608 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2814018430 con 0x7f282c101f40 2024-02-02T15:26:44.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.996+0000 7f2832e00700 1 -- 172.21.15.1:0/3609866608 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f282c10c5b0 con 0x7f282c101f40 2024-02-02T15:26:44.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:44.998+0000 7f2828c00700 1 -- 172.21.15.1:0/3609866608 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) v1 ==== 135069+0+0 (secure 0 0 0) 0x7f2814003f90 con 0x7f282c101f40 2024-02-02T15:26:44.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.000+0000 7f2828c00700 1 --2- 172.21.15.1:0/3609866608 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f280c0a0520 0x7f280c0a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:44.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.000+0000 7f2828c00700 1 -- 172.21.15.1:0/3609866608 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f2814014070 con 0x7f282c101f40 2024-02-02T15:26:44.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.000+0000 7f282be00700 1 --2- 172.21.15.1:0/3609866608 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f280c0a0520 0x7f280c0a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:44.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.001+0000 7f282be00700 1 --2- 172.21.15.1:0/3609866608 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f280c0a0520 0x7f280c0a29e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f282c19a220 tx=0x7f281c019040 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:45.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.004+0000 7f2828c00700 1 -- 172.21.15.1:0/3609866608 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f28140959d0 con 0x7f282c101f40 2024-02-02T15:26:45.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.263+0000 7f2832e00700 1 -- 172.21.15.1:0/3609866608 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f282c1a1860 con 0x7f282c101f40 2024-02-02T15:26:45.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.264+0000 7f2828c00700 1 -- 172.21.15.1:0/3609866608 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10866 (secure 0 0 0) 0x7f2814098ff0 con 0x7f282c101f40 2024-02-02T15:26:45.263 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:26:45.263 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2922240448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2922240448},{"type":"v1","addr":"172.21.15.1:6835","nonce":2922240448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2729497431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2729497431},{"type":"v1","addr":"172.21.15.1:6837","nonce":2729497431}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-02T15:24:47.761435+0000","modified":"2024-02-02T15:25:20.251601+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24529,"mds_2":24518,"mds_3":24514,"mds_4":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14517":{"gid":14517,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/1218303683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":1218303683},{"type":"v1","addr":"172.21.15.1:6839","nonce":1218303683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14529":{"gid":14529,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/2059197496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2059197496},{"type":"v1","addr":"172.21.15.1:6841","nonce":2059197496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/7648986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":7648986},{"type":"v1","addr":"172.21.15.142:6833","nonce":7648986}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/3482746094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3482746094},{"type":"v1","addr":"172.21.15.71:6833","nonce":3482746094}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/388207814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":388207814},{"type":"v1","addr":"172.21.15.71:6835","nonce":388207814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6835/1052606547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1052606547},{"type":"v1","addr":"172.21.15.142:6835","nonce":1052606547}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.142:6837/2870289019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2870289019},{"type":"v1","addr":"172.21.15.142:6837","nonce":2870289019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.71:6837/628016199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":628016199},{"type":"v1","addr":"172.21.15.71:6837","nonce":628016199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.142:6839/321675421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":321675421},{"type":"v1","addr":"172.21.15.142:6839","nonce":321675421}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/1268086493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1268086493},{"type":"v1","addr":"172.21.15.71:6839","nonce":1268086493}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-02T15:26:45.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.267+0000 7f2832e00700 1 -- 172.21.15.1:0/3609866608 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f280c0a0520 msgr2=0x7f280c0a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:45.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.267+0000 7f2832e00700 1 --2- 172.21.15.1:0/3609866608 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f280c0a0520 0x7f280c0a29e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f282c19a220 tx=0x7f281c019040 comp rx=0 tx=0).stop 2024-02-02T15:26:45.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.267+0000 7f2832e00700 1 -- 172.21.15.1:0/3609866608 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f282c101f40 msgr2=0x7f282c199290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:45.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.267+0000 7f2832e00700 1 --2- 172.21.15.1:0/3609866608 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f282c101f40 0x7f282c199290 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f281400ec10 tx=0x7f281400ef20 comp rx=0 tx=0).stop 2024-02-02T15:26:45.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.267+0000 7f2832e00700 1 -- 172.21.15.1:0/3609866608 shutdown_connections 2024-02-02T15:26:45.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.267+0000 7f2832e00700 1 --2- 172.21.15.1:0/3609866608 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f280c0a0520 0x7f280c0a29e0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:45.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.267+0000 7f2832e00700 1 --2- 172.21.15.1:0/3609866608 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f282c1028e0 0x7f282c1a02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:45.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.267+0000 7f2832e00700 1 --2- 172.21.15.1:0/3609866608 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f282c100d50 0x7f282c198d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:45.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.267+0000 7f2832e00700 1 --2- 172.21.15.1:0/3609866608 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f282c101f40 0x7f282c199290 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:45.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.267+0000 7f2832e00700 1 -- 172.21.15.1:0/3609866608 >> 172.21.15.1:0/3609866608 conn(0x7f282c0fc2c0 msgr2=0x7f282c0fe650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:45.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.267+0000 7f2832e00700 1 -- 172.21.15.1:0/3609866608 shutdown_connections 2024-02-02T15:26:45.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.267+0000 7f2832e00700 1 -- 172.21.15.1:0/3609866608 wait complete. 2024-02-02T15:26:45.267 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 27 2024-02-02T15:26:45.284 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-02-02T15:26:45.284 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-02-02T15:26:45.284 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-02T15:26:45.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.484+0000 7f07cb000700 1 -- 172.21.15.1:0/387997545 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f07c41017c0 msgr2=0x7f07c4103bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:45.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.484+0000 7f07cb000700 1 --2- 172.21.15.1:0/387997545 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f07c41017c0 0x7f07c4103bb0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f07b4009990 tx=0x7f07b4009ca0 comp rx=0 tx=0).stop 2024-02-02T15:26:45.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.484+0000 7f07cb000700 1 -- 172.21.15.1:0/387997545 shutdown_connections 2024-02-02T15:26:45.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.484+0000 7f07cb000700 1 --2- 172.21.15.1:0/387997545 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f07c4106a20 0x7f07c4108e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:45.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.484+0000 7f07cb000700 1 --2- 172.21.15.1:0/387997545 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f07c41017c0 0x7f07c4103bb0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:45.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.484+0000 7f07cb000700 1 --2- 172.21.15.1:0/387997545 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07c41040f0 0x7f07c41064e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:45.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.484+0000 7f07cb000700 1 -- 172.21.15.1:0/387997545 >> 172.21.15.1:0/387997545 conn(0x7f07c40fb170 msgr2=0x7f07c40fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:45.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.484+0000 7f07cb000700 1 -- 172.21.15.1:0/387997545 shutdown_connections 2024-02-02T15:26:45.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.484+0000 7f07cb000700 1 -- 172.21.15.1:0/387997545 wait complete. 2024-02-02T15:26:45.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.485+0000 7f07cb000700 1 Processor -- start 2024-02-02T15:26:45.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.485+0000 7f07cb000700 1 -- start start 2024-02-02T15:26:45.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.486+0000 7f07cb000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f07c41017c0 0x7f07c4198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:45.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.486+0000 7f07cb000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07c41040f0 0x7f07c41991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:45.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.486+0000 7f07cb000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f07c4106a20 0x7f07c41a01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:45.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.486+0000 7f07cb000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07c41a07a0 con 0x7f07c4106a20 2024-02-02T15:26:45.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.486+0000 7f07cb000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07c41a0910 con 0x7f07c41017c0 2024-02-02T15:26:45.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.486+0000 7f07cb000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07c41a0ad0 con 0x7f07c41040f0 2024-02-02T15:26:45.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.486+0000 7f07c9000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f07c4106a20 0x7f07c41a01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:45.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.486+0000 7f07c3e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f07c41017c0 0x7f07c4198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:45.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.486+0000 7f07bbe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07c41040f0 0x7f07c41991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:45.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.486+0000 7f07c3e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f07c41017c0 0x7f07c4198c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42170/0 (socket says 172.21.15.1:42170) 2024-02-02T15:26:45.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.486+0000 7f07c3e00700 1 -- 172.21.15.1:0/2529172490 learned_addr learned my addr 172.21.15.1:0/2529172490 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:45.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.486+0000 7f07bbe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07c41040f0 0x7f07c41991a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:41436/0 (socket says 172.21.15.1:41436) 2024-02-02T15:26:45.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.487+0000 7f07c9000700 1 -- 172.21.15.1:0/2529172490 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07c41040f0 msgr2=0x7f07c41991a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:45.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.487+0000 7f07c9000700 1 --2- 172.21.15.1:0/2529172490 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07c41040f0 0x7f07c41991a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:45.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.487+0000 7f07c9000700 1 -- 172.21.15.1:0/2529172490 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f07c41017c0 msgr2=0x7f07c4198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:45.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.487+0000 7f07c9000700 1 --2- 172.21.15.1:0/2529172490 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f07c41017c0 0x7f07c4198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:45.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.487+0000 7f07c9000700 1 -- 172.21.15.1:0/2529172490 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07b40095f0 con 0x7f07c4106a20 2024-02-02T15:26:45.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.487+0000 7f07c3e00700 1 --2- 172.21.15.1:0/2529172490 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f07c41017c0 0x7f07c4198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:26:45.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.487+0000 7f07c9000700 1 --2- 172.21.15.1:0/2529172490 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f07c4106a20 0x7f07c41a01d0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f07bc007f10 tx=0x7f07bc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:45.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.487+0000 7f07c1600700 1 -- 172.21.15.1:0/2529172490 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07bc004db0 con 0x7f07c4106a20 2024-02-02T15:26:45.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.487+0000 7f07cb000700 1 -- 172.21.15.1:0/2529172490 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07c41a0db0 con 0x7f07c4106a20 2024-02-02T15:26:45.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.487+0000 7f07c1600700 1 -- 172.21.15.1:0/2529172490 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f07bc00f040 con 0x7f07c4106a20 2024-02-02T15:26:45.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.487+0000 7f07c1600700 1 -- 172.21.15.1:0/2529172490 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07bc005990 con 0x7f07c4106a20 2024-02-02T15:26:45.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.487+0000 7f07cb000700 1 -- 172.21.15.1:0/2529172490 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07c41a1200 con 0x7f07c4106a20 2024-02-02T15:26:45.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.488+0000 7f07cb000700 1 -- 172.21.15.1:0/2529172490 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07c41085f0 con 0x7f07c4106a20 2024-02-02T15:26:45.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.490+0000 7f07c1600700 1 -- 172.21.15.1:0/2529172490 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) v1 ==== 135069+0+0 (secure 0 0 0) 0x7f07bc00db20 con 0x7f07c4106a20 2024-02-02T15:26:45.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.491+0000 7f07c1600700 1 --2- 172.21.15.1:0/2529172490 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f07a40a0520 0x7f07a40a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:45.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.491+0000 7f07c1600700 1 -- 172.21.15.1:0/2529172490 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f07bc0cb5d0 con 0x7f07c4106a20 2024-02-02T15:26:45.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.492+0000 7f07c3e00700 1 --2- 172.21.15.1:0/2529172490 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f07a40a0520 0x7f07a40a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:45.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.492+0000 7f07c3e00700 1 --2- 172.21.15.1:0/2529172490 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f07a40a0520 0x7f07a40a29e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f07b400b2e0 tx=0x7f07b4005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:45.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.495+0000 7f07c1600700 1 -- 172.21.15.1:0/2529172490 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f07bc094da0 con 0x7f07c4106a20 2024-02-02T15:26:45.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:45 smithi071 ceph-mon[44707]: pgmap v297: 129 pgs: 129 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 170 B/s wr, 9 op/s 2024-02-02T15:26:45.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:45 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/1004252218' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-02T15:26:45.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:45 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/1707433336' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-02T15:26:45.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:45 smithi142 ceph-mon[45728]: pgmap v297: 129 pgs: 129 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 170 B/s wr, 9 op/s 2024-02-02T15:26:45.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:45 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/1004252218' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-02T15:26:45.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:45 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/1707433336' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-02T15:26:45.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:45 smithi001 ceph-mon[35470]: pgmap v297: 129 pgs: 129 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 170 B/s wr, 9 op/s 2024-02-02T15:26:45.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:45 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/1004252218' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-02T15:26:45.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:45 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/1707433336' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-02T15:26:45.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.706+0000 7f07cb000700 1 -- 172.21.15.1:0/2529172490 --> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f07c4061180 con 0x7f07a40a0520 2024-02-02T15:26:45.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.720+0000 7f07c1600700 1 -- 172.21.15.1:0/2529172490 <== mgr.14154 v2:172.21.15.1:6800/3768989981 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427971 (secure 0 0 0) 0x7f07c4061180 con 0x7f07a40a0520 2024-02-02T15:26:45.726 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:26:45.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.725+0000 7f07cb000700 1 -- 172.21.15.1:0/2529172490 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f07a40a0520 msgr2=0x7f07a40a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:45.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.725+0000 7f07cb000700 1 --2- 172.21.15.1:0/2529172490 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f07a40a0520 0x7f07a40a29e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f07b400b2e0 tx=0x7f07b4005fb0 comp rx=0 tx=0).stop 2024-02-02T15:26:45.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.725+0000 7f07cb000700 1 -- 172.21.15.1:0/2529172490 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f07c4106a20 msgr2=0x7f07c41a01d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:45.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.725+0000 7f07cb000700 1 --2- 172.21.15.1:0/2529172490 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f07c4106a20 0x7f07c41a01d0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f07bc007f10 tx=0x7f07bc00d3b0 comp rx=0 tx=0).stop 2024-02-02T15:26:45.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.725+0000 7f07cb000700 1 -- 172.21.15.1:0/2529172490 shutdown_connections 2024-02-02T15:26:45.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.725+0000 7f07cb000700 1 --2- 172.21.15.1:0/2529172490 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f07c4106a20 0x7f07c41a01d0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:45.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.725+0000 7f07cb000700 1 --2- 172.21.15.1:0/2529172490 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f07c41017c0 0x7f07c4198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:45.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.725+0000 7f07cb000700 1 --2- 172.21.15.1:0/2529172490 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f07a40a0520 0x7f07a40a29e0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:45.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.725+0000 7f07cb000700 1 --2- 172.21.15.1:0/2529172490 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f07c41040f0 0x7f07c41991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:45.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.725+0000 7f07cb000700 1 -- 172.21.15.1:0/2529172490 >> 172.21.15.1:0/2529172490 conn(0x7f07c40fb170 msgr2=0x7f07c40ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:45.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.725+0000 7f07cb000700 1 -- 172.21.15.1:0/2529172490 shutdown_connections 2024-02-02T15:26:45.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.725+0000 7f07cb000700 1 -- 172.21.15.1:0/2529172490 wait complete. 2024-02-02T15:26:45.731 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-02-02T15:26:45.743 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":297,"stamp":"2024-02-02T15:26:43.934695+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658334,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":990,"num_read_kb":624,"num_write":431,"num_write_kb":3084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1180804,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":855,"ondisk_log_size":855,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":336228,"kb_used_data":11972,"kb_used_omap":20,"kb_used_meta":324075,"kb_avail":1124605596,"statfs":{"total":1151940427776,"available":1151596130304,"internally_reserved":0,"allocated":12259328,"data_stored":7915039,"data_compressed":392601,"data_compressed_allocated":528384,"data_compressed_original":2555904,"omap_allocated":21372,"internal_metadata":331852932},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":312,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":116,"num_read_kb":61,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002937"},"pg_stats":[{"pgid":"3.3f","version":"86'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.379934+0000","last_change":"2024-02-02T15:24:47.793308+0000","last_active":"2024-02-02T15:26:27.379934+0000","last_peered":"2024-02-02T15:26:27.379934+0000","last_clean":"2024-02-02T15:26:27.379934+0000","last_became_active":"2024-02-02T15:24:47.792420+0000","last_became_peered":"2024-02-02T15:24:47.792420+0000","last_unstale":"2024-02-02T15:26:27.379934+0000","last_undegraded":"2024-02-02T15:26:27.379934+0000","last_fullsized":"2024-02-02T15:26:27.379934+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:39:09.871317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"86'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.486648+0000","last_change":"2024-02-02T15:24:47.791272+0000","last_active":"2024-02-02T15:26:27.486648+0000","last_peered":"2024-02-02T15:26:27.486648+0000","last_clean":"2024-02-02T15:26:27.486648+0000","last_became_active":"2024-02-02T15:24:47.790902+0000","last_became_peered":"2024-02-02T15:24:47.790902+0000","last_unstale":"2024-02-02T15:26:27.486648+0000","last_undegraded":"2024-02-02T15:26:27.486648+0000","last_fullsized":"2024-02-02T15:26:27.486648+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:31:11.620873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"86'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.486605+0000","last_change":"2024-02-02T15:24:47.798712+0000","last_active":"2024-02-02T15:26:27.486605+0000","last_peered":"2024-02-02T15:26:27.486605+0000","last_clean":"2024-02-02T15:26:27.486605+0000","last_became_active":"2024-02-02T15:24:47.798405+0000","last_became_peered":"2024-02-02T15:24:47.798405+0000","last_unstale":"2024-02-02T15:26:27.486605+0000","last_undegraded":"2024-02-02T15:26:27.486605+0000","last_fullsized":"2024-02-02T15:26:27.486605+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:37:15.238070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"86'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.280412+0000","last_change":"2024-02-02T15:24:47.786456+0000","last_active":"2024-02-02T15:26:27.280412+0000","last_peered":"2024-02-02T15:26:27.280412+0000","last_clean":"2024-02-02T15:26:27.280412+0000","last_became_active":"2024-02-02T15:24:47.785520+0000","last_became_peered":"2024-02-02T15:24:47.785520+0000","last_unstale":"2024-02-02T15:26:27.280412+0000","last_undegraded":"2024-02-02T15:26:27.280412+0000","last_fullsized":"2024-02-02T15:26:27.280412+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:16:04.003459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"86'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.274132+0000","last_change":"2024-02-02T15:24:47.780084+0000","last_active":"2024-02-02T15:26:27.274132+0000","last_peered":"2024-02-02T15:26:27.274132+0000","last_clean":"2024-02-02T15:26:27.274132+0000","last_became_active":"2024-02-02T15:24:47.779667+0000","last_became_peered":"2024-02-02T15:24:47.779667+0000","last_unstale":"2024-02-02T15:26:27.274132+0000","last_undegraded":"2024-02-02T15:26:27.274132+0000","last_fullsized":"2024-02-02T15:26:27.274132+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:30:15.377691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"86'6","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.362466+0000","last_change":"2024-02-02T15:24:47.786173+0000","last_active":"2024-02-02T15:26:27.362466+0000","last_peered":"2024-02-02T15:26:27.362466+0000","last_clean":"2024-02-02T15:26:27.362466+0000","last_became_active":"2024-02-02T15:24:47.785301+0000","last_became_peered":"2024-02-02T15:24:47.785301+0000","last_unstale":"2024-02-02T15:26:27.362466+0000","last_undegraded":"2024-02-02T15:26:27.362466+0000","last_fullsized":"2024-02-02T15:26:27.362466+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:14:58.524317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"86'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.273751+0000","last_change":"2024-02-02T15:24:47.790913+0000","last_active":"2024-02-02T15:26:27.273751+0000","last_peered":"2024-02-02T15:26:27.273751+0000","last_clean":"2024-02-02T15:26:27.273751+0000","last_became_active":"2024-02-02T15:24:47.790566+0000","last_became_peered":"2024-02-02T15:24:47.790566+0000","last_unstale":"2024-02-02T15:26:27.273751+0000","last_undegraded":"2024-02-02T15:26:27.273751+0000","last_fullsized":"2024-02-02T15:26:27.273751+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:34:51.954830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"86'9","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.375616+0000","last_change":"2024-02-02T15:24:47.786465+0000","last_active":"2024-02-02T15:26:27.375616+0000","last_peered":"2024-02-02T15:26:27.375616+0000","last_clean":"2024-02-02T15:26:27.375616+0000","last_became_active":"2024-02-02T15:24:47.786175+0000","last_became_peered":"2024-02-02T15:24:47.786175+0000","last_unstale":"2024-02-02T15:26:27.375616+0000","last_undegraded":"2024-02-02T15:26:27.375616+0000","last_fullsized":"2024-02-02T15:26:27.375616+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:42:13.748850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"86'11","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.277236+0000","last_change":"2024-02-02T15:24:47.787115+0000","last_active":"2024-02-02T15:26:27.277236+0000","last_peered":"2024-02-02T15:26:27.277236+0000","last_clean":"2024-02-02T15:26:27.277236+0000","last_became_active":"2024-02-02T15:24:47.786804+0000","last_became_peered":"2024-02-02T15:24:47.786804+0000","last_unstale":"2024-02-02T15:26:27.277236+0000","last_undegraded":"2024-02-02T15:26:27.277236+0000","last_fullsized":"2024-02-02T15:26:27.277236+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T03:06:17.873243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"86'16","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.362900+0000","last_change":"2024-02-02T15:24:47.787211+0000","last_active":"2024-02-02T15:26:27.362900+0000","last_peered":"2024-02-02T15:26:27.362900+0000","last_clean":"2024-02-02T15:26:27.362900+0000","last_became_active":"2024-02-02T15:24:47.786888+0000","last_became_peered":"2024-02-02T15:24:47.786888+0000","last_unstale":"2024-02-02T15:26:27.362900+0000","last_undegraded":"2024-02-02T15:26:27.362900+0000","last_fullsized":"2024-02-02T15:26:27.362900+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:14:21.442787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"86'6","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.274294+0000","last_change":"2024-02-02T15:24:47.798962+0000","last_active":"2024-02-02T15:26:27.274294+0000","last_peered":"2024-02-02T15:26:27.274294+0000","last_clean":"2024-02-02T15:26:27.274294+0000","last_became_active":"2024-02-02T15:24:47.798657+0000","last_became_peered":"2024-02-02T15:24:47.798657+0000","last_unstale":"2024-02-02T15:26:27.274294+0000","last_undegraded":"2024-02-02T15:26:27.274294+0000","last_fullsized":"2024-02-02T15:26:27.274294+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:53:37.316027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"86'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.280289+0000","last_change":"2024-02-02T15:24:47.782647+0000","last_active":"2024-02-02T15:26:27.280289+0000","last_peered":"2024-02-02T15:26:27.280289+0000","last_clean":"2024-02-02T15:26:27.280289+0000","last_became_active":"2024-02-02T15:24:47.782344+0000","last_became_peered":"2024-02-02T15:24:47.782344+0000","last_unstale":"2024-02-02T15:26:27.280289+0000","last_undegraded":"2024-02-02T15:26:27.280289+0000","last_fullsized":"2024-02-02T15:26:27.280289+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:06:54.874657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"86'10","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.269115+0000","last_change":"2024-02-02T15:24:47.787032+0000","last_active":"2024-02-02T15:26:27.269115+0000","last_peered":"2024-02-02T15:26:27.269115+0000","last_clean":"2024-02-02T15:26:27.269115+0000","last_became_active":"2024-02-02T15:24:47.786710+0000","last_became_peered":"2024-02-02T15:24:47.786710+0000","last_unstale":"2024-02-02T15:26:27.269115+0000","last_undegraded":"2024-02-02T15:26:27.269115+0000","last_fullsized":"2024-02-02T15:26:27.269115+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:26:15.429316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"86'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.486412+0000","last_change":"2024-02-02T15:24:47.785708+0000","last_active":"2024-02-02T15:26:27.486412+0000","last_peered":"2024-02-02T15:26:27.486412+0000","last_clean":"2024-02-02T15:26:27.486412+0000","last_became_active":"2024-02-02T15:24:47.785280+0000","last_became_peered":"2024-02-02T15:24:47.785280+0000","last_unstale":"2024-02-02T15:26:27.486412+0000","last_undegraded":"2024-02-02T15:26:27.486412+0000","last_fullsized":"2024-02-02T15:26:27.486412+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T02:21:23.224933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"86'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.486347+0000","last_change":"2024-02-02T15:24:47.783904+0000","last_active":"2024-02-02T15:26:27.486347+0000","last_peered":"2024-02-02T15:26:27.486347+0000","last_clean":"2024-02-02T15:26:27.486347+0000","last_became_active":"2024-02-02T15:24:47.783599+0000","last_became_peered":"2024-02-02T15:24:47.783599+0000","last_unstale":"2024-02-02T15:26:27.486347+0000","last_undegraded":"2024-02-02T15:26:27.486347+0000","last_fullsized":"2024-02-02T15:26:27.486347+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:35:45.805527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"86'9","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.276951+0000","last_change":"2024-02-02T15:24:47.795770+0000","last_active":"2024-02-02T15:26:27.276951+0000","last_peered":"2024-02-02T15:26:27.276951+0000","last_clean":"2024-02-02T15:26:27.276951+0000","last_became_active":"2024-02-02T15:24:47.795433+0000","last_became_peered":"2024-02-02T15:24:47.795433+0000","last_unstale":"2024-02-02T15:26:27.276951+0000","last_undegraded":"2024-02-02T15:26:27.276951+0000","last_fullsized":"2024-02-02T15:26:27.276951+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:50:07.496998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"86'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.379835+0000","last_change":"2024-02-02T15:24:47.793684+0000","last_active":"2024-02-02T15:26:27.379835+0000","last_peered":"2024-02-02T15:26:27.379835+0000","last_clean":"2024-02-02T15:26:27.379835+0000","last_became_active":"2024-02-02T15:24:47.792853+0000","last_became_peered":"2024-02-02T15:24:47.792853+0000","last_unstale":"2024-02-02T15:26:27.379835+0000","last_undegraded":"2024-02-02T15:26:27.379835+0000","last_fullsized":"2024-02-02T15:26:27.379835+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:30:40.379379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"86'10","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.280363+0000","last_change":"2024-02-02T15:24:47.780935+0000","last_active":"2024-02-02T15:26:27.280363+0000","last_peered":"2024-02-02T15:26:27.280363+0000","last_clean":"2024-02-02T15:26:27.280363+0000","last_became_active":"2024-02-02T15:24:47.780642+0000","last_became_peered":"2024-02-02T15:24:47.780642+0000","last_unstale":"2024-02-02T15:26:27.280363+0000","last_undegraded":"2024-02-02T15:26:27.280363+0000","last_fullsized":"2024-02-02T15:26:27.280363+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:36:37.615368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"86'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.380235+0000","last_change":"2024-02-02T15:24:47.791051+0000","last_active":"2024-02-02T15:26:27.380235+0000","last_peered":"2024-02-02T15:26:27.380235+0000","last_clean":"2024-02-02T15:26:27.380235+0000","last_became_active":"2024-02-02T15:24:47.790732+0000","last_became_peered":"2024-02-02T15:24:47.790732+0000","last_unstale":"2024-02-02T15:26:27.380235+0000","last_undegraded":"2024-02-02T15:26:27.380235+0000","last_fullsized":"2024-02-02T15:26:27.380235+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:45:05.155144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.380085+0000","last_change":"2024-02-02T15:24:51.814699+0000","last_active":"2024-02-02T15:26:27.380085+0000","last_peered":"2024-02-02T15:26:27.380085+0000","last_clean":"2024-02-02T15:26:27.380085+0000","last_became_active":"2024-02-02T15:24:51.814251+0000","last_became_peered":"2024-02-02T15:24:51.814251+0000","last_unstale":"2024-02-02T15:26:27.380085+0000","last_undegraded":"2024-02-02T15:26:27.380085+0000","last_fullsized":"2024-02-02T15:26:27.380085+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:27:21.674922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"84'13","reported_seq":118,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-02T15:26:42.393747+0000","last_change":"2024-02-02T15:24:47.269429+0000","last_active":"2024-02-02T15:26:42.393747+0000","last_peered":"2024-02-02T15:26:42.393747+0000","last_clean":"2024-02-02T15:26:42.393747+0000","last_became_active":"2024-02-02T15:24:47.269023+0000","last_became_peered":"2024-02-02T15:24:47.269023+0000","last_unstale":"2024-02-02T15:26:42.393747+0000","last_undegraded":"2024-02-02T15:26:42.393747+0000","last_fullsized":"2024-02-02T15:26:42.393747+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T03:11:46.344438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":85,"num_read_kb":85,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"86'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.376226+0000","last_change":"2024-02-02T15:24:47.785900+0000","last_active":"2024-02-02T15:26:27.376226+0000","last_peered":"2024-02-02T15:26:27.376226+0000","last_clean":"2024-02-02T15:26:27.376226+0000","last_became_active":"2024-02-02T15:24:47.781994+0000","last_became_peered":"2024-02-02T15:24:47.781994+0000","last_unstale":"2024-02-02T15:26:27.376226+0000","last_undegraded":"2024-02-02T15:26:27.376226+0000","last_fullsized":"2024-02-02T15:26:27.376226+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:00:52.432637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.279297+0000","last_change":"2024-02-02T15:24:51.825655+0000","last_active":"2024-02-02T15:26:27.279297+0000","last_peered":"2024-02-02T15:26:27.279297+0000","last_clean":"2024-02-02T15:26:27.279297+0000","last_became_active":"2024-02-02T15:24:51.825199+0000","last_became_peered":"2024-02-02T15:24:51.825199+0000","last_unstale":"2024-02-02T15:26:27.279297+0000","last_undegraded":"2024-02-02T15:26:27.279297+0000","last_fullsized":"2024-02-02T15:26:27.279297+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:50:32.283394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"84'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.376399+0000","last_change":"2024-02-02T15:24:47.281869+0000","last_active":"2024-02-02T15:26:27.376399+0000","last_peered":"2024-02-02T15:26:27.376399+0000","last_clean":"2024-02-02T15:26:27.376399+0000","last_became_active":"2024-02-02T15:24:47.280825+0000","last_became_peered":"2024-02-02T15:24:47.280825+0000","last_unstale":"2024-02-02T15:26:27.376399+0000","last_undegraded":"2024-02-02T15:26:27.376399+0000","last_fullsized":"2024-02-02T15:26:27.376399+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:24:19.688389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"86'6","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.486649+0000","last_change":"2024-02-02T15:24:47.795893+0000","last_active":"2024-02-02T15:26:27.486649+0000","last_peered":"2024-02-02T15:26:27.486649+0000","last_clean":"2024-02-02T15:26:27.486649+0000","last_became_active":"2024-02-02T15:24:47.795491+0000","last_became_peered":"2024-02-02T15:24:47.795491+0000","last_unstale":"2024-02-02T15:26:27.486649+0000","last_undegraded":"2024-02-02T15:26:27.486649+0000","last_fullsized":"2024-02-02T15:26:27.486649+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:54:38.923036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.279462+0000","last_change":"2024-02-02T15:24:51.826980+0000","last_active":"2024-02-02T15:26:27.279462+0000","last_peered":"2024-02-02T15:26:27.279462+0000","last_clean":"2024-02-02T15:26:27.279462+0000","last_became_active":"2024-02-02T15:24:51.826221+0000","last_became_peered":"2024-02-02T15:24:51.826221+0000","last_unstale":"2024-02-02T15:26:27.279462+0000","last_undegraded":"2024-02-02T15:26:27.279462+0000","last_fullsized":"2024-02-02T15:26:27.279462+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:45:17.126174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"84'12","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.274623+0000","last_change":"2024-02-02T15:24:47.272219+0000","last_active":"2024-02-02T15:26:27.274623+0000","last_peered":"2024-02-02T15:26:27.274623+0000","last_clean":"2024-02-02T15:26:27.274623+0000","last_became_active":"2024-02-02T15:24:47.271899+0000","last_became_peered":"2024-02-02T15:24:47.271899+0000","last_unstale":"2024-02-02T15:26:27.274623+0000","last_undegraded":"2024-02-02T15:26:27.274623+0000","last_fullsized":"2024-02-02T15:26:27.274623+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:44:27.370783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"86'9","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.277362+0000","last_change":"2024-02-02T15:24:47.783434+0000","last_active":"2024-02-02T15:26:27.277362+0000","last_peered":"2024-02-02T15:26:27.277362+0000","last_clean":"2024-02-02T15:26:27.277362+0000","last_became_active":"2024-02-02T15:24:47.782130+0000","last_became_peered":"2024-02-02T15:24:47.782130+0000","last_unstale":"2024-02-02T15:26:27.277362+0000","last_undegraded":"2024-02-02T15:26:27.277362+0000","last_fullsized":"2024-02-02T15:26:27.277362+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:32:25.981913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.279554+0000","last_change":"2024-02-02T15:24:51.810791+0000","last_active":"2024-02-02T15:26:27.279554+0000","last_peered":"2024-02-02T15:26:27.279554+0000","last_clean":"2024-02-02T15:26:27.279554+0000","last_became_active":"2024-02-02T15:24:51.810379+0000","last_became_peered":"2024-02-02T15:24:51.810379+0000","last_unstale":"2024-02-02T15:26:27.279554+0000","last_undegraded":"2024-02-02T15:26:27.279554+0000","last_fullsized":"2024-02-02T15:26:27.279554+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:49:44.635820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'5","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.274488+0000","last_change":"2024-02-02T15:24:47.271225+0000","last_active":"2024-02-02T15:26:27.274488+0000","last_peered":"2024-02-02T15:26:27.274488+0000","last_clean":"2024-02-02T15:26:27.274488+0000","last_became_active":"2024-02-02T15:24:47.270870+0000","last_became_peered":"2024-02-02T15:24:47.270870+0000","last_unstale":"2024-02-02T15:26:27.274488+0000","last_undegraded":"2024-02-02T15:26:27.274488+0000","last_fullsized":"2024-02-02T15:26:27.274488+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:16:55.659950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"86'15","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-02T15:26:28.281479+0000","last_change":"2024-02-02T15:26:28.281073+0000","last_active":"2024-02-02T15:26:28.281479+0000","last_peered":"2024-02-02T15:26:28.281479+0000","last_clean":"2024-02-02T15:26:28.281479+0000","last_became_active":"2024-02-02T15:26:28.280680+0000","last_became_peered":"2024-02-02T15:26:28.280680+0000","last_unstale":"2024-02-02T15:26:28.281479+0000","last_undegraded":"2024-02-02T15:26:28.281479+0000","last_fullsized":"2024-02-02T15:26:28.281479+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:53:34.526394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.379201+0000","last_change":"2024-02-02T15:24:51.807328+0000","last_active":"2024-02-02T15:26:27.379201+0000","last_peered":"2024-02-02T15:26:27.379201+0000","last_clean":"2024-02-02T15:26:27.379201+0000","last_became_active":"2024-02-02T15:24:51.806853+0000","last_became_peered":"2024-02-02T15:24:51.806853+0000","last_unstale":"2024-02-02T15:26:27.379201+0000","last_undegraded":"2024-02-02T15:26:27.379201+0000","last_fullsized":"2024-02-02T15:26:27.379201+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:46:13.898450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"84'5","reported_seq":108,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-02T15:26:39.385542+0000","last_change":"2024-02-02T15:24:47.268192+0000","last_active":"2024-02-02T15:26:39.385542+0000","last_peered":"2024-02-02T15:26:39.385542+0000","last_clean":"2024-02-02T15:26:39.385542+0000","last_became_active":"2024-02-02T15:24:47.267796+0000","last_became_peered":"2024-02-02T15:24:47.267796+0000","last_unstale":"2024-02-02T15:26:39.385542+0000","last_undegraded":"2024-02-02T15:26:39.385542+0000","last_fullsized":"2024-02-02T15:26:39.385542+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:12:13.161690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":83,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"86'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.274477+0000","last_change":"2024-02-02T15:24:47.782976+0000","last_active":"2024-02-02T15:26:27.274477+0000","last_peered":"2024-02-02T15:26:27.274477+0000","last_clean":"2024-02-02T15:26:27.274477+0000","last_became_active":"2024-02-02T15:24:47.782640+0000","last_became_peered":"2024-02-02T15:24:47.782640+0000","last_unstale":"2024-02-02T15:26:27.274477+0000","last_undegraded":"2024-02-02T15:26:27.274477+0000","last_fullsized":"2024-02-02T15:26:27.274477+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T02:09:54.851418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.379834+0000","last_change":"2024-02-02T15:24:51.814799+0000","last_active":"2024-02-02T15:26:27.379834+0000","last_peered":"2024-02-02T15:26:27.379834+0000","last_clean":"2024-02-02T15:26:27.379834+0000","last_became_active":"2024-02-02T15:24:51.814263+0000","last_became_peered":"2024-02-02T15:24:51.814263+0000","last_unstale":"2024-02-02T15:26:27.379834+0000","last_undegraded":"2024-02-02T15:26:27.379834+0000","last_fullsized":"2024-02-02T15:26:27.379834+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:52:02.786496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"84'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.362424+0000","last_change":"2024-02-02T15:24:47.273204+0000","last_active":"2024-02-02T15:26:27.362424+0000","last_peered":"2024-02-02T15:26:27.362424+0000","last_clean":"2024-02-02T15:26:27.362424+0000","last_became_active":"2024-02-02T15:24:47.272888+0000","last_became_peered":"2024-02-02T15:24:47.272888+0000","last_unstale":"2024-02-02T15:26:27.362424+0000","last_undegraded":"2024-02-02T15:26:27.362424+0000","last_fullsized":"2024-02-02T15:26:27.362424+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:26:25.188915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"86'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.281811+0000","last_change":"2024-02-02T15:24:47.788645+0000","last_active":"2024-02-02T15:26:27.281811+0000","last_peered":"2024-02-02T15:26:27.281811+0000","last_clean":"2024-02-02T15:26:27.281811+0000","last_became_active":"2024-02-02T15:24:47.787478+0000","last_became_peered":"2024-02-02T15:24:47.787478+0000","last_unstale":"2024-02-02T15:26:27.281811+0000","last_undegraded":"2024-02-02T15:26:27.281811+0000","last_fullsized":"2024-02-02T15:26:27.281811+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:24:45.327030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.379732+0000","last_change":"2024-02-02T15:24:51.822681+0000","last_active":"2024-02-02T15:26:27.379732+0000","last_peered":"2024-02-02T15:26:27.379732+0000","last_clean":"2024-02-02T15:26:27.379732+0000","last_became_active":"2024-02-02T15:24:51.822202+0000","last_became_peered":"2024-02-02T15:24:51.822202+0000","last_unstale":"2024-02-02T15:26:27.379732+0000","last_undegraded":"2024-02-02T15:26:27.379732+0000","last_fullsized":"2024-02-02T15:26:27.379732+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:05:43.751558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"84'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.268759+0000","last_change":"2024-02-02T15:24:47.271095+0000","last_active":"2024-02-02T15:26:27.268759+0000","last_peered":"2024-02-02T15:26:27.268759+0000","last_clean":"2024-02-02T15:26:27.268759+0000","last_became_active":"2024-02-02T15:24:47.270696+0000","last_became_peered":"2024-02-02T15:24:47.270696+0000","last_unstale":"2024-02-02T15:26:27.268759+0000","last_undegraded":"2024-02-02T15:26:27.268759+0000","last_fullsized":"2024-02-02T15:26:27.268759+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T02:20:59.370217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"86'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.375968+0000","last_change":"2024-02-02T15:24:47.791699+0000","last_active":"2024-02-02T15:26:27.375968+0000","last_peered":"2024-02-02T15:26:27.375968+0000","last_clean":"2024-02-02T15:26:27.375968+0000","last_became_active":"2024-02-02T15:24:47.787280+0000","last_became_peered":"2024-02-02T15:24:47.787280+0000","last_unstale":"2024-02-02T15:26:27.375968+0000","last_undegraded":"2024-02-02T15:26:27.375968+0000","last_fullsized":"2024-02-02T15:26:27.375968+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:18:10.316393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.486617+0000","last_change":"2024-02-02T15:24:51.810050+0000","last_active":"2024-02-02T15:26:27.486617+0000","last_peered":"2024-02-02T15:26:27.486617+0000","last_clean":"2024-02-02T15:26:27.486617+0000","last_became_active":"2024-02-02T15:24:51.809621+0000","last_became_peered":"2024-02-02T15:24:51.809621+0000","last_unstale":"2024-02-02T15:26:27.486617+0000","last_undegraded":"2024-02-02T15:26:27.486617+0000","last_fullsized":"2024-02-02T15:26:27.486617+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:52:35.885973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"84'3","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.281444+0000","last_change":"2024-02-02T15:24:47.267973+0000","last_active":"2024-02-02T15:26:27.281444+0000","last_peered":"2024-02-02T15:26:27.281444+0000","last_clean":"2024-02-02T15:26:27.281444+0000","last_became_active":"2024-02-02T15:24:47.267649+0000","last_became_peered":"2024-02-02T15:24:47.267649+0000","last_unstale":"2024-02-02T15:26:27.281444+0000","last_undegraded":"2024-02-02T15:26:27.281444+0000","last_fullsized":"2024-02-02T15:26:27.281444+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T02:46:56.844570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"86'11","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.279736+0000","last_change":"2024-02-02T15:24:47.798636+0000","last_active":"2024-02-02T15:26:27.279736+0000","last_peered":"2024-02-02T15:26:27.279736+0000","last_clean":"2024-02-02T15:26:27.279736+0000","last_became_active":"2024-02-02T15:24:47.798328+0000","last_became_peered":"2024-02-02T15:24:47.798328+0000","last_unstale":"2024-02-02T15:26:27.279736+0000","last_undegraded":"2024-02-02T15:26:27.279736+0000","last_fullsized":"2024-02-02T15:26:27.279736+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:37:43.687336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.375711+0000","last_change":"2024-02-02T15:24:51.811312+0000","last_active":"2024-02-02T15:26:27.375711+0000","last_peered":"2024-02-02T15:26:27.375711+0000","last_clean":"2024-02-02T15:26:27.375711+0000","last_became_active":"2024-02-02T15:24:51.810932+0000","last_became_peered":"2024-02-02T15:24:51.810932+0000","last_unstale":"2024-02-02T15:26:27.375711+0000","last_undegraded":"2024-02-02T15:26:27.375711+0000","last_fullsized":"2024-02-02T15:26:27.375711+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:10:31.530518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"84'4","reported_seq":186,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-02T15:26:40.386290+0000","last_change":"2024-02-02T15:24:47.271483+0000","last_active":"2024-02-02T15:26:40.386290+0000","last_peered":"2024-02-02T15:26:40.386290+0000","last_clean":"2024-02-02T15:26:40.386290+0000","last_became_active":"2024-02-02T15:24:47.271160+0000","last_became_peered":"2024-02-02T15:24:47.271160+0000","last_unstale":"2024-02-02T15:26:40.386290+0000","last_undegraded":"2024-02-02T15:26:40.386290+0000","last_fullsized":"2024-02-02T15:26:40.386290+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:29:20.135351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":162,"num_read_kb":162,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"86'10","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.277538+0000","last_change":"2024-02-02T15:24:47.786492+0000","last_active":"2024-02-02T15:26:27.277538+0000","last_peered":"2024-02-02T15:26:27.277538+0000","last_clean":"2024-02-02T15:26:27.277538+0000","last_became_active":"2024-02-02T15:24:47.786103+0000","last_became_peered":"2024-02-02T15:24:47.786103+0000","last_unstale":"2024-02-02T15:26:27.277538+0000","last_undegraded":"2024-02-02T15:26:27.277538+0000","last_fullsized":"2024-02-02T15:26:27.277538+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:24:23.666596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.279580+0000","last_change":"2024-02-02T15:24:51.819941+0000","last_active":"2024-02-02T15:26:27.279580+0000","last_peered":"2024-02-02T15:26:27.279580+0000","last_clean":"2024-02-02T15:26:27.279580+0000","last_became_active":"2024-02-02T15:24:51.818066+0000","last_became_peered":"2024-02-02T15:24:51.818066+0000","last_unstale":"2024-02-02T15:26:27.279580+0000","last_undegraded":"2024-02-02T15:26:27.279580+0000","last_fullsized":"2024-02-02T15:26:27.279580+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:55:51.550988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"84'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.274167+0000","last_change":"2024-02-02T15:24:47.268541+0000","last_active":"2024-02-02T15:26:27.274167+0000","last_peered":"2024-02-02T15:26:27.274167+0000","last_clean":"2024-02-02T15:26:27.274167+0000","last_became_active":"2024-02-02T15:24:47.268184+0000","last_became_peered":"2024-02-02T15:24:47.268184+0000","last_unstale":"2024-02-02T15:26:27.274167+0000","last_undegraded":"2024-02-02T15:26:27.274167+0000","last_fullsized":"2024-02-02T15:26:27.274167+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:30:12.187297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"86'9","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.486650+0000","last_change":"2024-02-02T15:24:47.787792+0000","last_active":"2024-02-02T15:26:27.486650+0000","last_peered":"2024-02-02T15:26:27.486650+0000","last_clean":"2024-02-02T15:26:27.486650+0000","last_became_active":"2024-02-02T15:24:47.787479+0000","last_became_peered":"2024-02-02T15:24:47.787479+0000","last_unstale":"2024-02-02T15:26:27.486650+0000","last_undegraded":"2024-02-02T15:26:27.486650+0000","last_fullsized":"2024-02-02T15:26:27.486650+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T03:24:06.544232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.279743+0000","last_change":"2024-02-02T15:24:51.826639+0000","last_active":"2024-02-02T15:26:27.279743+0000","last_peered":"2024-02-02T15:26:27.279743+0000","last_clean":"2024-02-02T15:26:27.279743+0000","last_became_active":"2024-02-02T15:24:51.825089+0000","last_became_peered":"2024-02-02T15:24:51.825089+0000","last_unstale":"2024-02-02T15:26:27.279743+0000","last_undegraded":"2024-02-02T15:26:27.279743+0000","last_fullsized":"2024-02-02T15:26:27.279743+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:41:44.592603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"84'5","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.281688+0000","last_change":"2024-02-02T15:24:47.281490+0000","last_active":"2024-02-02T15:26:27.281688+0000","last_peered":"2024-02-02T15:26:27.281688+0000","last_clean":"2024-02-02T15:26:27.281688+0000","last_became_active":"2024-02-02T15:24:47.281137+0000","last_became_peered":"2024-02-02T15:24:47.281137+0000","last_unstale":"2024-02-02T15:26:27.281688+0000","last_undegraded":"2024-02-02T15:26:27.281688+0000","last_fullsized":"2024-02-02T15:26:27.281688+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:42:08.508881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"86'11","reported_seq":39,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-02T15:26:28.376134+0000","last_change":"2024-02-02T15:26:28.376134+0000","last_active":"2024-02-02T15:26:28.376134+0000","last_peered":"2024-02-02T15:26:28.376134+0000","last_clean":"2024-02-02T15:26:28.376134+0000","last_became_active":"2024-02-02T15:26:28.375708+0000","last_became_peered":"2024-02-02T15:26:28.375708+0000","last_unstale":"2024-02-02T15:26:28.376134+0000","last_undegraded":"2024-02-02T15:26:28.376134+0000","last_fullsized":"2024-02-02T15:26:28.376134+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:03:10.254243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,7],"acting":[1,8,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.279858+0000","last_change":"2024-02-02T15:25:27.286403+0000","last_active":"2024-02-02T15:26:27.279858+0000","last_peered":"2024-02-02T15:26:27.279858+0000","last_clean":"2024-02-02T15:26:27.279858+0000","last_became_active":"2024-02-02T15:25:27.285824+0000","last_became_peered":"2024-02-02T15:25:27.285824+0000","last_unstale":"2024-02-02T15:26:27.279858+0000","last_undegraded":"2024-02-02T15:26:27.279858+0000","last_fullsized":"2024-02-02T15:26:27.279858+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:11:25.986947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,10],"acting":[9,8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"88'39","reported_seq":231,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-02T15:26:40.386763+0000","last_change":"2024-02-02T15:24:47.270604+0000","last_active":"2024-02-02T15:26:40.386763+0000","last_peered":"2024-02-02T15:26:40.386763+0000","last_clean":"2024-02-02T15:26:40.386763+0000","last_became_active":"2024-02-02T15:24:47.270261+0000","last_became_peered":"2024-02-02T15:24:47.270261+0000","last_unstale":"2024-02-02T15:26:40.386763+0000","last_undegraded":"2024-02-02T15:26:40.386763+0000","last_fullsized":"2024-02-02T15:26:40.386763+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:56:33.767423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60957,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":172,"num_read_kb":147,"num_write":41,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"86'6","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.274315+0000","last_change":"2024-02-02T15:24:47.789663+0000","last_active":"2024-02-02T15:26:27.274315+0000","last_peered":"2024-02-02T15:26:27.274315+0000","last_clean":"2024-02-02T15:26:27.274315+0000","last_became_active":"2024-02-02T15:24:47.789352+0000","last_became_peered":"2024-02-02T15:24:47.789352+0000","last_unstale":"2024-02-02T15:26:27.274315+0000","last_undegraded":"2024-02-02T15:26:27.274315+0000","last_fullsized":"2024-02-02T15:26:27.274315+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:13:37.991350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.282133+0000","last_change":"2024-02-02T15:24:51.816021+0000","last_active":"2024-02-02T15:26:27.282133+0000","last_peered":"2024-02-02T15:26:27.282133+0000","last_clean":"2024-02-02T15:26:27.282133+0000","last_became_active":"2024-02-02T15:24:51.815698+0000","last_became_peered":"2024-02-02T15:24:51.815698+0000","last_unstale":"2024-02-02T15:26:27.282133+0000","last_undegraded":"2024-02-02T15:26:27.282133+0000","last_fullsized":"2024-02-02T15:26:27.282133+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:11:10.670195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"84'9","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.379371+0000","last_change":"2024-02-02T15:25:27.295339+0000","last_active":"2024-02-02T15:26:27.379371+0000","last_peered":"2024-02-02T15:26:27.379371+0000","last_clean":"2024-02-02T15:26:27.379371+0000","last_became_active":"2024-02-02T15:25:27.281605+0000","last_became_peered":"2024-02-02T15:25:27.281605+0000","last_unstale":"2024-02-02T15:26:27.379371+0000","last_undegraded":"2024-02-02T15:26:27.379371+0000","last_fullsized":"2024-02-02T15:26:27.379371+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T02:35:36.952261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"86'9","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.268962+0000","last_change":"2024-02-02T15:24:47.798089+0000","last_active":"2024-02-02T15:26:27.268962+0000","last_peered":"2024-02-02T15:26:27.268962+0000","last_clean":"2024-02-02T15:26:27.268962+0000","last_became_active":"2024-02-02T15:24:47.793914+0000","last_became_peered":"2024-02-02T15:24:47.793914+0000","last_unstale":"2024-02-02T15:26:27.268962+0000","last_undegraded":"2024-02-02T15:26:27.268962+0000","last_fullsized":"2024-02-02T15:26:27.268962+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:33:39.927550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.379433+0000","last_change":"2024-02-02T15:24:51.823098+0000","last_active":"2024-02-02T15:26:27.379433+0000","last_peered":"2024-02-02T15:26:27.379433+0000","last_clean":"2024-02-02T15:26:27.379433+0000","last_became_active":"2024-02-02T15:24:51.822634+0000","last_became_peered":"2024-02-02T15:24:51.822634+0000","last_unstale":"2024-02-02T15:26:27.379433+0000","last_undegraded":"2024-02-02T15:26:27.379433+0000","last_fullsized":"2024-02-02T15:26:27.379433+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T03:18:07.420196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"84'5","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.281450+0000","last_change":"2024-02-02T15:24:47.268962+0000","last_active":"2024-02-02T15:26:27.281450+0000","last_peered":"2024-02-02T15:26:27.281450+0000","last_clean":"2024-02-02T15:26:27.281450+0000","last_became_active":"2024-02-02T15:24:47.268607+0000","last_became_peered":"2024-02-02T15:24:47.268607+0000","last_unstale":"2024-02-02T15:26:27.281450+0000","last_undegraded":"2024-02-02T15:26:27.281450+0000","last_fullsized":"2024-02-02T15:26:27.281450+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:31:06.330437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"86'15","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.273962+0000","last_change":"2024-02-02T15:24:47.792205+0000","last_active":"2024-02-02T15:26:27.273962+0000","last_peered":"2024-02-02T15:26:27.273962+0000","last_clean":"2024-02-02T15:26:27.273962+0000","last_became_active":"2024-02-02T15:24:47.791874+0000","last_became_peered":"2024-02-02T15:24:47.791874+0000","last_unstale":"2024-02-02T15:26:27.273962+0000","last_undegraded":"2024-02-02T15:26:27.273962+0000","last_fullsized":"2024-02-02T15:26:27.273962+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:33:38.795921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.280422+0000","last_change":"2024-02-02T15:24:51.825499+0000","last_active":"2024-02-02T15:26:27.280422+0000","last_peered":"2024-02-02T15:26:27.280422+0000","last_clean":"2024-02-02T15:26:27.280422+0000","last_became_active":"2024-02-02T15:24:51.825049+0000","last_became_peered":"2024-02-02T15:24:51.825049+0000","last_unstale":"2024-02-02T15:26:27.280422+0000","last_undegraded":"2024-02-02T15:26:27.280422+0000","last_fullsized":"2024-02-02T15:26:27.280422+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:41:05.430891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"86'7","reported_seq":110,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-02T15:26:38.396064+0000","last_change":"2024-02-02T15:24:47.267683+0000","last_active":"2024-02-02T15:26:38.396064+0000","last_peered":"2024-02-02T15:26:38.396064+0000","last_clean":"2024-02-02T15:26:38.396064+0000","last_became_active":"2024-02-02T15:24:47.267324+0000","last_became_peered":"2024-02-02T15:24:47.267324+0000","last_unstale":"2024-02-02T15:26:38.396064+0000","last_undegraded":"2024-02-02T15:26:38.396064+0000","last_fullsized":"2024-02-02T15:26:38.396064+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:03:31.326791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1073,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":84,"num_read_kb":4,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"86'6","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.274103+0000","last_change":"2024-02-02T15:24:47.792509+0000","last_active":"2024-02-02T15:26:27.274103+0000","last_peered":"2024-02-02T15:26:27.274103+0000","last_clean":"2024-02-02T15:26:27.274103+0000","last_became_active":"2024-02-02T15:24:47.791662+0000","last_became_peered":"2024-02-02T15:24:47.791662+0000","last_unstale":"2024-02-02T15:26:27.274103+0000","last_undegraded":"2024-02-02T15:26:27.274103+0000","last_fullsized":"2024-02-02T15:26:27.274103+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:42:50.009971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.269435+0000","last_change":"2024-02-02T15:25:27.282514+0000","last_active":"2024-02-02T15:26:27.269435+0000","last_peered":"2024-02-02T15:26:27.269435+0000","last_clean":"2024-02-02T15:26:27.269435+0000","last_became_active":"2024-02-02T15:25:27.281336+0000","last_became_peered":"2024-02-02T15:25:27.281336+0000","last_unstale":"2024-02-02T15:26:27.269435+0000","last_undegraded":"2024-02-02T15:26:27.269435+0000","last_fullsized":"2024-02-02T15:26:27.269435+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:50:39.728813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"84'17","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.274050+0000","last_change":"2024-02-02T15:24:47.274152+0000","last_active":"2024-02-02T15:26:27.274050+0000","last_peered":"2024-02-02T15:26:27.274050+0000","last_clean":"2024-02-02T15:26:27.274050+0000","last_became_active":"2024-02-02T15:24:47.273720+0000","last_became_peered":"2024-02-02T15:24:47.273720+0000","last_unstale":"2024-02-02T15:26:27.274050+0000","last_undegraded":"2024-02-02T15:26:27.274050+0000","last_fullsized":"2024-02-02T15:26:27.274050+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:38:05.280309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"86'6","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.281387+0000","last_change":"2024-02-02T15:24:47.790681+0000","last_active":"2024-02-02T15:26:27.281387+0000","last_peered":"2024-02-02T15:26:27.281387+0000","last_clean":"2024-02-02T15:26:27.281387+0000","last_became_active":"2024-02-02T15:24:47.789952+0000","last_became_peered":"2024-02-02T15:24:47.789952+0000","last_unstale":"2024-02-02T15:26:27.281387+0000","last_undegraded":"2024-02-02T15:26:27.281387+0000","last_fullsized":"2024-02-02T15:26:27.281387+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:21:03.103398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.281438+0000","last_change":"2024-02-02T15:24:51.812676+0000","last_active":"2024-02-02T15:26:27.281438+0000","last_peered":"2024-02-02T15:26:27.281438+0000","last_clean":"2024-02-02T15:26:27.281438+0000","last_became_active":"2024-02-02T15:24:51.812212+0000","last_became_peered":"2024-02-02T15:24:51.812212+0000","last_unstale":"2024-02-02T15:26:27.281438+0000","last_undegraded":"2024-02-02T15:26:27.281438+0000","last_fullsized":"2024-02-02T15:26:27.281438+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:48:01.944379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'2","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.274629+0000","last_change":"2024-02-02T15:24:47.271707+0000","last_active":"2024-02-02T15:26:27.274629+0000","last_peered":"2024-02-02T15:26:27.274629+0000","last_clean":"2024-02-02T15:26:27.274629+0000","last_became_active":"2024-02-02T15:24:47.271379+0000","last_became_peered":"2024-02-02T15:24:47.271379+0000","last_unstale":"2024-02-02T15:26:27.274629+0000","last_undegraded":"2024-02-02T15:26:27.274629+0000","last_fullsized":"2024-02-02T15:26:27.274629+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:00:24.571385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"86'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.379740+0000","last_change":"2024-02-02T15:24:47.788360+0000","last_active":"2024-02-02T15:26:27.379740+0000","last_peered":"2024-02-02T15:26:27.379740+0000","last_clean":"2024-02-02T15:26:27.379740+0000","last_became_active":"2024-02-02T15:24:47.787945+0000","last_became_peered":"2024-02-02T15:24:47.787945+0000","last_unstale":"2024-02-02T15:26:27.379740+0000","last_undegraded":"2024-02-02T15:26:27.379740+0000","last_fullsized":"2024-02-02T15:26:27.379740+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:37:54.531487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.274398+0000","last_change":"2024-02-02T15:24:51.821320+0000","last_active":"2024-02-02T15:26:27.274398+0000","last_peered":"2024-02-02T15:26:27.274398+0000","last_clean":"2024-02-02T15:26:27.274398+0000","last_became_active":"2024-02-02T15:24:51.820853+0000","last_became_peered":"2024-02-02T15:24:51.820853+0000","last_unstale":"2024-02-02T15:26:27.274398+0000","last_undegraded":"2024-02-02T15:26:27.274398+0000","last_fullsized":"2024-02-02T15:26:27.274398+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:20:12.304517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"84'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.280230+0000","last_change":"2024-02-02T15:24:47.278291+0000","last_active":"2024-02-02T15:26:27.280230+0000","last_peered":"2024-02-02T15:26:27.280230+0000","last_clean":"2024-02-02T15:26:27.280230+0000","last_became_active":"2024-02-02T15:24:47.276163+0000","last_became_peered":"2024-02-02T15:24:47.276163+0000","last_unstale":"2024-02-02T15:26:27.280230+0000","last_undegraded":"2024-02-02T15:26:27.280230+0000","last_fullsized":"2024-02-02T15:26:27.280230+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:53:50.312627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.362700+0000","last_change":"2024-02-02T15:24:47.791290+0000","last_active":"2024-02-02T15:26:27.362700+0000","last_peered":"2024-02-02T15:26:27.362700+0000","last_clean":"2024-02-02T15:26:27.362700+0000","last_became_active":"2024-02-02T15:24:47.790987+0000","last_became_peered":"2024-02-02T15:24:47.790987+0000","last_unstale":"2024-02-02T15:26:27.362700+0000","last_undegraded":"2024-02-02T15:26:27.362700+0000","last_fullsized":"2024-02-02T15:26:27.362700+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:38:08.803338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.268999+0000","last_change":"2024-02-02T15:24:51.813050+0000","last_active":"2024-02-02T15:26:27.268999+0000","last_peered":"2024-02-02T15:26:27.268999+0000","last_clean":"2024-02-02T15:26:27.268999+0000","last_became_active":"2024-02-02T15:24:51.812634+0000","last_became_peered":"2024-02-02T15:24:51.812634+0000","last_unstale":"2024-02-02T15:26:27.268999+0000","last_undegraded":"2024-02-02T15:26:27.268999+0000","last_fullsized":"2024-02-02T15:26:27.268999+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:06:56.428431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"84'5","reported_seq":107,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-02T15:26:39.390566+0000","last_change":"2024-02-02T15:24:47.274033+0000","last_active":"2024-02-02T15:26:39.390566+0000","last_peered":"2024-02-02T15:26:39.390566+0000","last_clean":"2024-02-02T15:26:39.390566+0000","last_became_active":"2024-02-02T15:24:47.273672+0000","last_became_peered":"2024-02-02T15:24:47.273672+0000","last_unstale":"2024-02-02T15:26:39.390566+0000","last_undegraded":"2024-02-02T15:26:39.390566+0000","last_fullsized":"2024-02-02T15:26:39.390566+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:51:46.926397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":82,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"86'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.362550+0000","last_change":"2024-02-02T15:24:47.786533+0000","last_active":"2024-02-02T15:26:27.362550+0000","last_peered":"2024-02-02T15:26:27.362550+0000","last_clean":"2024-02-02T15:26:27.362550+0000","last_became_active":"2024-02-02T15:24:47.785436+0000","last_became_peered":"2024-02-02T15:24:47.785436+0000","last_unstale":"2024-02-02T15:26:27.362550+0000","last_undegraded":"2024-02-02T15:26:27.362550+0000","last_fullsized":"2024-02-02T15:26:27.362550+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:23:45.910800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.269411+0000","last_change":"2024-02-02T15:24:51.814326+0000","last_active":"2024-02-02T15:26:27.269411+0000","last_peered":"2024-02-02T15:26:27.269411+0000","last_clean":"2024-02-02T15:26:27.269411+0000","last_became_active":"2024-02-02T15:24:51.813909+0000","last_became_peered":"2024-02-02T15:24:51.813909+0000","last_unstale":"2024-02-02T15:26:27.269411+0000","last_undegraded":"2024-02-02T15:26:27.269411+0000","last_fullsized":"2024-02-02T15:26:27.269411+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:16:12.902718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"86'8","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.281562+0000","last_change":"2024-02-02T15:25:27.287818+0000","last_active":"2024-02-02T15:26:27.281562+0000","last_peered":"2024-02-02T15:26:27.281562+0000","last_clean":"2024-02-02T15:26:27.281562+0000","last_became_active":"2024-02-02T15:25:27.287481+0000","last_became_peered":"2024-02-02T15:25:27.287481+0000","last_unstale":"2024-02-02T15:26:27.281562+0000","last_undegraded":"2024-02-02T15:26:27.281562+0000","last_fullsized":"2024-02-02T15:26:27.281562+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:14:32.820969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'6","reported_seq":110,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-02T15:26:41.399825+0000","last_change":"2024-02-02T15:24:47.274437+0000","last_active":"2024-02-02T15:26:41.399825+0000","last_peered":"2024-02-02T15:26:41.399825+0000","last_clean":"2024-02-02T15:26:41.399825+0000","last_became_active":"2024-02-02T15:24:47.274010+0000","last_became_peered":"2024-02-02T15:24:47.274010+0000","last_unstale":"2024-02-02T15:26:41.399825+0000","last_undegraded":"2024-02-02T15:26:41.399825+0000","last_fullsized":"2024-02-02T15:26:41.399825+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:10:36.862190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":84,"num_read_kb":84,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"84'1","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.274175+0000","last_change":"2024-02-02T15:24:51.825698+0000","last_active":"2024-02-02T15:26:27.274175+0000","last_peered":"2024-02-02T15:26:27.274175+0000","last_clean":"2024-02-02T15:26:27.274175+0000","last_became_active":"2024-02-02T15:24:51.825295+0000","last_became_peered":"2024-02-02T15:24:51.825295+0000","last_unstale":"2024-02-02T15:26:27.274175+0000","last_undegraded":"2024-02-02T15:26:27.274175+0000","last_fullsized":"2024-02-02T15:26:27.274175+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:17:09.838124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"86'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.375702+0000","last_change":"2024-02-02T15:24:47.781836+0000","last_active":"2024-02-02T15:26:27.375702+0000","last_peered":"2024-02-02T15:26:27.375702+0000","last_clean":"2024-02-02T15:26:27.375702+0000","last_became_active":"2024-02-02T15:24:47.781472+0000","last_became_peered":"2024-02-02T15:24:47.781472+0000","last_unstale":"2024-02-02T15:26:27.375702+0000","last_undegraded":"2024-02-02T15:26:27.375702+0000","last_fullsized":"2024-02-02T15:26:27.375702+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:46:33.714191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"84'6","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.279642+0000","last_change":"2024-02-02T15:24:47.269087+0000","last_active":"2024-02-02T15:26:27.279642+0000","last_peered":"2024-02-02T15:26:27.279642+0000","last_clean":"2024-02-02T15:26:27.279642+0000","last_became_active":"2024-02-02T15:24:47.268743+0000","last_became_peered":"2024-02-02T15:24:47.268743+0000","last_unstale":"2024-02-02T15:26:27.279642+0000","last_undegraded":"2024-02-02T15:26:27.279642+0000","last_fullsized":"2024-02-02T15:26:27.279642+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:03:04.323079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":338,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.273687+0000","last_change":"2024-02-02T15:23:55.191735+0000","last_active":"2024-02-02T15:26:27.273687+0000","last_peered":"2024-02-02T15:26:27.273687+0000","last_clean":"2024-02-02T15:26:27.273687+0000","last_became_active":"2024-02-02T15:23:55.191119+0000","last_became_peered":"2024-02-02T15:23:55.191119+0000","last_unstale":"2024-02-02T15:26:27.273687+0000","last_undegraded":"2024-02-02T15:26:27.273687+0000","last_fullsized":"2024-02-02T15:26:27.273687+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:21:22.472631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:21:22.472631+0000","last_clean_scrub_stamp":"2024-02-02T15:21:22.472631+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:24:44.153461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.281781+0000","last_change":"2024-02-02T15:24:51.822842+0000","last_active":"2024-02-02T15:26:27.281781+0000","last_peered":"2024-02-02T15:26:27.281781+0000","last_clean":"2024-02-02T15:26:27.281781+0000","last_became_active":"2024-02-02T15:24:51.822372+0000","last_became_peered":"2024-02-02T15:24:51.822372+0000","last_unstale":"2024-02-02T15:26:27.281781+0000","last_undegraded":"2024-02-02T15:26:27.281781+0000","last_fullsized":"2024-02-02T15:26:27.281781+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:45:29.631422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"86'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.268664+0000","last_change":"2024-02-02T15:24:47.786507+0000","last_active":"2024-02-02T15:26:27.268664+0000","last_peered":"2024-02-02T15:26:27.268664+0000","last_clean":"2024-02-02T15:26:27.268664+0000","last_became_active":"2024-02-02T15:24:47.786125+0000","last_became_peered":"2024-02-02T15:24:47.786125+0000","last_unstale":"2024-02-02T15:26:27.268664+0000","last_undegraded":"2024-02-02T15:26:27.268664+0000","last_fullsized":"2024-02-02T15:26:27.268664+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:33:09.014258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"84'4","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.274171+0000","last_change":"2024-02-02T15:25:27.628074+0000","last_active":"2024-02-02T15:26:27.274171+0000","last_peered":"2024-02-02T15:26:27.274171+0000","last_clean":"2024-02-02T15:26:27.274171+0000","last_became_active":"2024-02-02T15:25:27.625103+0000","last_became_peered":"2024-02-02T15:25:27.625103+0000","last_unstale":"2024-02-02T15:26:27.274171+0000","last_undegraded":"2024-02-02T15:26:27.274171+0000","last_fullsized":"2024-02-02T15:26:27.274171+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:30:35.540456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.379470+0000","last_change":"2024-02-02T15:24:51.808979+0000","last_active":"2024-02-02T15:26:27.379470+0000","last_peered":"2024-02-02T15:26:27.379470+0000","last_clean":"2024-02-02T15:26:27.379470+0000","last_became_active":"2024-02-02T15:24:51.808557+0000","last_became_peered":"2024-02-02T15:24:51.808557+0000","last_unstale":"2024-02-02T15:26:27.379470+0000","last_undegraded":"2024-02-02T15:26:27.379470+0000","last_fullsized":"2024-02-02T15:26:27.379470+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T02:38:34.697137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"84'4","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.274530+0000","last_change":"2024-02-02T15:24:47.281396+0000","last_active":"2024-02-02T15:26:27.274530+0000","last_peered":"2024-02-02T15:26:27.274530+0000","last_clean":"2024-02-02T15:26:27.274530+0000","last_became_active":"2024-02-02T15:24:47.281036+0000","last_became_peered":"2024-02-02T15:24:47.281036+0000","last_unstale":"2024-02-02T15:26:27.274530+0000","last_undegraded":"2024-02-02T15:26:27.274530+0000","last_fullsized":"2024-02-02T15:26:27.274530+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:54:47.139151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"86'9","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.269092+0000","last_change":"2024-02-02T15:24:47.791057+0000","last_active":"2024-02-02T15:26:27.269092+0000","last_peered":"2024-02-02T15:26:27.269092+0000","last_clean":"2024-02-02T15:26:27.269092+0000","last_became_active":"2024-02-02T15:24:47.790647+0000","last_became_peered":"2024-02-02T15:24:47.790647+0000","last_unstale":"2024-02-02T15:26:27.269092+0000","last_undegraded":"2024-02-02T15:26:27.269092+0000","last_fullsized":"2024-02-02T15:26:27.269092+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T02:49:42.067700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.375624+0000","last_change":"2024-02-02T15:24:51.817792+0000","last_active":"2024-02-02T15:26:27.375624+0000","last_peered":"2024-02-02T15:26:27.375624+0000","last_clean":"2024-02-02T15:26:27.375624+0000","last_became_active":"2024-02-02T15:24:51.815800+0000","last_became_peered":"2024-02-02T15:24:51.815800+0000","last_unstale":"2024-02-02T15:26:27.375624+0000","last_undegraded":"2024-02-02T15:26:27.375624+0000","last_fullsized":"2024-02-02T15:26:27.375624+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:28:10.440251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'12","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-02T15:26:28.276379+0000","last_change":"2024-02-02T15:24:47.277098+0000","last_active":"2024-02-02T15:26:28.276379+0000","last_peered":"2024-02-02T15:26:28.276379+0000","last_clean":"2024-02-02T15:26:28.276379+0000","last_became_active":"2024-02-02T15:24:47.275867+0000","last_became_peered":"2024-02-02T15:24:47.275867+0000","last_unstale":"2024-02-02T15:26:28.276379+0000","last_undegraded":"2024-02-02T15:26:28.276379+0000","last_fullsized":"2024-02-02T15:26:28.276379+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:35:21.015918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"86'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.376351+0000","last_change":"2024-02-02T15:24:47.784873+0000","last_active":"2024-02-02T15:26:27.376351+0000","last_peered":"2024-02-02T15:26:27.376351+0000","last_clean":"2024-02-02T15:26:27.376351+0000","last_became_active":"2024-02-02T15:24:47.784550+0000","last_became_peered":"2024-02-02T15:24:47.784550+0000","last_unstale":"2024-02-02T15:26:27.376351+0000","last_undegraded":"2024-02-02T15:26:27.376351+0000","last_fullsized":"2024-02-02T15:26:27.376351+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:13:37.513754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.379656+0000","last_change":"2024-02-02T15:24:51.805131+0000","last_active":"2024-02-02T15:26:27.379656+0000","last_peered":"2024-02-02T15:26:27.379656+0000","last_clean":"2024-02-02T15:26:27.379656+0000","last_became_active":"2024-02-02T15:24:51.804705+0000","last_became_peered":"2024-02-02T15:24:51.804705+0000","last_unstale":"2024-02-02T15:26:27.379656+0000","last_undegraded":"2024-02-02T15:26:27.379656+0000","last_fullsized":"2024-02-02T15:26:27.379656+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T02:12:59.945050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"84'7","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.280301+0000","last_change":"2024-02-02T15:24:47.266469+0000","last_active":"2024-02-02T15:26:27.280301+0000","last_peered":"2024-02-02T15:26:27.280301+0000","last_clean":"2024-02-02T15:26:27.280301+0000","last_became_active":"2024-02-02T15:24:47.266094+0000","last_became_peered":"2024-02-02T15:24:47.266094+0000","last_unstale":"2024-02-02T15:26:27.280301+0000","last_undegraded":"2024-02-02T15:26:27.280301+0000","last_fullsized":"2024-02-02T15:26:27.280301+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:51:41.324698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"86'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.486689+0000","last_change":"2024-02-02T15:24:47.793051+0000","last_active":"2024-02-02T15:26:27.486689+0000","last_peered":"2024-02-02T15:26:27.486689+0000","last_clean":"2024-02-02T15:26:27.486689+0000","last_became_active":"2024-02-02T15:24:47.792645+0000","last_became_peered":"2024-02-02T15:24:47.792645+0000","last_unstale":"2024-02-02T15:26:27.486689+0000","last_undegraded":"2024-02-02T15:26:27.486689+0000","last_fullsized":"2024-02-02T15:26:27.486689+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:47:17.229407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.280008+0000","last_change":"2024-02-02T15:25:27.282505+0000","last_active":"2024-02-02T15:26:27.280008+0000","last_peered":"2024-02-02T15:26:27.280008+0000","last_clean":"2024-02-02T15:26:27.280008+0000","last_became_active":"2024-02-02T15:25:27.282039+0000","last_became_peered":"2024-02-02T15:25:27.282039+0000","last_unstale":"2024-02-02T15:26:27.280008+0000","last_undegraded":"2024-02-02T15:26:27.280008+0000","last_fullsized":"2024-02-02T15:26:27.280008+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:31:29.700848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"84'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.277078+0000","last_change":"2024-02-02T15:24:47.273716+0000","last_active":"2024-02-02T15:26:27.277078+0000","last_peered":"2024-02-02T15:26:27.277078+0000","last_clean":"2024-02-02T15:26:27.277078+0000","last_became_active":"2024-02-02T15:24:47.273438+0000","last_became_peered":"2024-02-02T15:24:47.273438+0000","last_unstale":"2024-02-02T15:26:27.277078+0000","last_undegraded":"2024-02-02T15:26:27.277078+0000","last_fullsized":"2024-02-02T15:26:27.277078+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:52:16.498119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"86'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.274074+0000","last_change":"2024-02-02T15:24:47.782950+0000","last_active":"2024-02-02T15:26:27.274074+0000","last_peered":"2024-02-02T15:26:27.274074+0000","last_clean":"2024-02-02T15:26:27.274074+0000","last_became_active":"2024-02-02T15:24:47.782605+0000","last_became_peered":"2024-02-02T15:24:47.782605+0000","last_unstale":"2024-02-02T15:26:27.274074+0000","last_undegraded":"2024-02-02T15:26:27.274074+0000","last_fullsized":"2024-02-02T15:26:27.274074+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:57:34.125985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.274573+0000","last_change":"2024-02-02T15:25:27.288179+0000","last_active":"2024-02-02T15:26:27.274573+0000","last_peered":"2024-02-02T15:26:27.274573+0000","last_clean":"2024-02-02T15:26:27.274573+0000","last_became_active":"2024-02-02T15:25:27.287753+0000","last_became_peered":"2024-02-02T15:25:27.287753+0000","last_unstale":"2024-02-02T15:26:27.274573+0000","last_undegraded":"2024-02-02T15:26:27.274573+0000","last_fullsized":"2024-02-02T15:26:27.274573+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:02:46.854950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"84'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.375810+0000","last_change":"2024-02-02T15:24:47.278478+0000","last_active":"2024-02-02T15:26:27.375810+0000","last_peered":"2024-02-02T15:26:27.375810+0000","last_clean":"2024-02-02T15:26:27.375810+0000","last_became_active":"2024-02-02T15:24:47.278084+0000","last_became_peered":"2024-02-02T15:24:47.278084+0000","last_unstale":"2024-02-02T15:26:27.375810+0000","last_undegraded":"2024-02-02T15:26:27.375810+0000","last_fullsized":"2024-02-02T15:26:27.375810+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:10:23.044968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"86'9","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.378859+0000","last_change":"2024-02-02T15:24:47.795291+0000","last_active":"2024-02-02T15:26:27.378859+0000","last_peered":"2024-02-02T15:26:27.378859+0000","last_clean":"2024-02-02T15:26:27.378859+0000","last_became_active":"2024-02-02T15:24:47.794514+0000","last_became_peered":"2024-02-02T15:24:47.794514+0000","last_unstale":"2024-02-02T15:26:27.378859+0000","last_undegraded":"2024-02-02T15:26:27.378859+0000","last_fullsized":"2024-02-02T15:26:27.378859+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:32:12.722258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"84'1","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.280163+0000","last_change":"2024-02-02T15:24:51.825985+0000","last_active":"2024-02-02T15:26:27.280163+0000","last_peered":"2024-02-02T15:26:27.280163+0000","last_clean":"2024-02-02T15:26:27.280163+0000","last_became_active":"2024-02-02T15:24:51.825588+0000","last_became_peered":"2024-02-02T15:24:51.825588+0000","last_unstale":"2024-02-02T15:26:27.280163+0000","last_undegraded":"2024-02-02T15:26:27.280163+0000","last_fullsized":"2024-02-02T15:26:27.280163+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:32:39.910258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"84'3","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.378872+0000","last_change":"2024-02-02T15:25:27.298553+0000","last_active":"2024-02-02T15:26:27.378872+0000","last_peered":"2024-02-02T15:26:27.378872+0000","last_clean":"2024-02-02T15:26:27.378872+0000","last_became_active":"2024-02-02T15:25:27.282469+0000","last_became_peered":"2024-02-02T15:25:27.282469+0000","last_unstale":"2024-02-02T15:26:27.378872+0000","last_undegraded":"2024-02-02T15:26:27.378872+0000","last_fullsized":"2024-02-02T15:26:27.378872+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:54:14.740951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"86'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.375881+0000","last_change":"2024-02-02T15:24:47.784188+0000","last_active":"2024-02-02T15:26:27.375881+0000","last_peered":"2024-02-02T15:26:27.375881+0000","last_clean":"2024-02-02T15:26:27.375881+0000","last_became_active":"2024-02-02T15:24:47.783872+0000","last_became_peered":"2024-02-02T15:24:47.783872+0000","last_unstale":"2024-02-02T15:26:27.375881+0000","last_undegraded":"2024-02-02T15:26:27.375881+0000","last_fullsized":"2024-02-02T15:26:27.375881+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:45:45.213255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.280221+0000","last_change":"2024-02-02T15:24:51.815263+0000","last_active":"2024-02-02T15:26:27.280221+0000","last_peered":"2024-02-02T15:26:27.280221+0000","last_clean":"2024-02-02T15:26:27.280221+0000","last_became_active":"2024-02-02T15:24:51.814849+0000","last_became_peered":"2024-02-02T15:24:51.814849+0000","last_unstale":"2024-02-02T15:26:27.280221+0000","last_undegraded":"2024-02-02T15:26:27.280221+0000","last_fullsized":"2024-02-02T15:26:27.280221+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:05:32.987359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"84'4","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.274712+0000","last_change":"2024-02-02T15:24:47.270534+0000","last_active":"2024-02-02T15:26:27.274712+0000","last_peered":"2024-02-02T15:26:27.274712+0000","last_clean":"2024-02-02T15:26:27.274712+0000","last_became_active":"2024-02-02T15:24:47.270110+0000","last_became_peered":"2024-02-02T15:24:47.270110+0000","last_unstale":"2024-02-02T15:26:27.274712+0000","last_undegraded":"2024-02-02T15:26:27.274712+0000","last_fullsized":"2024-02-02T15:26:27.274712+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T02:59:52.297315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"86'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.273855+0000","last_change":"2024-02-02T15:24:47.791826+0000","last_active":"2024-02-02T15:26:27.273855+0000","last_peered":"2024-02-02T15:26:27.273855+0000","last_clean":"2024-02-02T15:26:27.273855+0000","last_became_active":"2024-02-02T15:24:47.791016+0000","last_became_peered":"2024-02-02T15:24:47.791016+0000","last_unstale":"2024-02-02T15:26:27.273855+0000","last_undegraded":"2024-02-02T15:26:27.273855+0000","last_fullsized":"2024-02-02T15:26:27.273855+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:47:14.296876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.276753+0000","last_change":"2024-02-02T15:24:51.806741+0000","last_active":"2024-02-02T15:26:27.276753+0000","last_peered":"2024-02-02T15:26:27.276753+0000","last_clean":"2024-02-02T15:26:27.276753+0000","last_became_active":"2024-02-02T15:24:51.806085+0000","last_became_peered":"2024-02-02T15:24:51.806085+0000","last_unstale":"2024-02-02T15:26:27.276753+0000","last_undegraded":"2024-02-02T15:26:27.276753+0000","last_fullsized":"2024-02-02T15:26:27.276753+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:53:39.944222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"84'5","reported_seq":109,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-02T15:26:40.400639+0000","last_change":"2024-02-02T15:24:47.278515+0000","last_active":"2024-02-02T15:26:40.400639+0000","last_peered":"2024-02-02T15:26:40.400639+0000","last_clean":"2024-02-02T15:26:40.400639+0000","last_became_active":"2024-02-02T15:24:47.278145+0000","last_became_peered":"2024-02-02T15:24:47.278145+0000","last_unstale":"2024-02-02T15:26:40.400639+0000","last_undegraded":"2024-02-02T15:26:40.400639+0000","last_fullsized":"2024-02-02T15:26:40.400639+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:48:55.118003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":84,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"86'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.274330+0000","last_change":"2024-02-02T15:24:47.791996+0000","last_active":"2024-02-02T15:26:27.274330+0000","last_peered":"2024-02-02T15:26:27.274330+0000","last_clean":"2024-02-02T15:26:27.274330+0000","last_became_active":"2024-02-02T15:24:47.791624+0000","last_became_peered":"2024-02-02T15:24:47.791624+0000","last_unstale":"2024-02-02T15:26:27.274330+0000","last_undegraded":"2024-02-02T15:26:27.274330+0000","last_fullsized":"2024-02-02T15:26:27.274330+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:33:27.963740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.486271+0000","last_change":"2024-02-02T15:24:51.826835+0000","last_active":"2024-02-02T15:26:27.486271+0000","last_peered":"2024-02-02T15:26:27.486271+0000","last_clean":"2024-02-02T15:26:27.486271+0000","last_became_active":"2024-02-02T15:24:51.826310+0000","last_became_peered":"2024-02-02T15:24:51.826310+0000","last_unstale":"2024-02-02T15:26:27.486271+0000","last_undegraded":"2024-02-02T15:26:27.486271+0000","last_fullsized":"2024-02-02T15:26:27.486271+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:48:16.687048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'3","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.282058+0000","last_change":"2024-02-02T15:24:47.268865+0000","last_active":"2024-02-02T15:26:27.282058+0000","last_peered":"2024-02-02T15:26:27.282058+0000","last_clean":"2024-02-02T15:26:27.282058+0000","last_became_active":"2024-02-02T15:24:47.268523+0000","last_became_peered":"2024-02-02T15:24:47.268523+0000","last_unstale":"2024-02-02T15:26:27.282058+0000","last_undegraded":"2024-02-02T15:26:27.282058+0000","last_fullsized":"2024-02-02T15:26:27.282058+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:45:16.957512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'6","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.274101+0000","last_change":"2024-02-02T15:24:47.792601+0000","last_active":"2024-02-02T15:26:27.274101+0000","last_peered":"2024-02-02T15:26:27.274101+0000","last_clean":"2024-02-02T15:26:27.274101+0000","last_became_active":"2024-02-02T15:24:47.792240+0000","last_became_peered":"2024-02-02T15:24:47.792240+0000","last_unstale":"2024-02-02T15:26:27.274101+0000","last_undegraded":"2024-02-02T15:26:27.274101+0000","last_fullsized":"2024-02-02T15:26:27.274101+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:51:25.205859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.281132+0000","last_change":"2024-02-02T15:24:51.806267+0000","last_active":"2024-02-02T15:26:27.281132+0000","last_peered":"2024-02-02T15:26:27.281132+0000","last_clean":"2024-02-02T15:26:27.281132+0000","last_became_active":"2024-02-02T15:24:51.805877+0000","last_became_peered":"2024-02-02T15:24:51.805877+0000","last_unstale":"2024-02-02T15:26:27.281132+0000","last_undegraded":"2024-02-02T15:26:27.281132+0000","last_fullsized":"2024-02-02T15:26:27.281132+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:00:47.747377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.269402+0000","last_change":"2024-02-02T15:24:47.271057+0000","last_active":"2024-02-02T15:26:27.269402+0000","last_peered":"2024-02-02T15:26:27.269402+0000","last_clean":"2024-02-02T15:26:27.269402+0000","last_became_active":"2024-02-02T15:24:47.270737+0000","last_became_peered":"2024-02-02T15:24:47.270737+0000","last_unstale":"2024-02-02T15:26:27.269402+0000","last_undegraded":"2024-02-02T15:26:27.269402+0000","last_fullsized":"2024-02-02T15:26:27.269402+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:17:00.212426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"86'9","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.274263+0000","last_change":"2024-02-02T15:24:47.794108+0000","last_active":"2024-02-02T15:26:27.274263+0000","last_peered":"2024-02-02T15:26:27.274263+0000","last_clean":"2024-02-02T15:26:27.274263+0000","last_became_active":"2024-02-02T15:24:47.793890+0000","last_became_peered":"2024-02-02T15:24:47.793890+0000","last_unstale":"2024-02-02T15:26:27.274263+0000","last_undegraded":"2024-02-02T15:26:27.274263+0000","last_fullsized":"2024-02-02T15:26:27.274263+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T03:05:51.704385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"86'9","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.378946+0000","last_change":"2024-02-02T15:24:47.796234+0000","last_active":"2024-02-02T15:26:27.378946+0000","last_peered":"2024-02-02T15:26:27.378946+0000","last_clean":"2024-02-02T15:26:27.378946+0000","last_became_active":"2024-02-02T15:24:47.795474+0000","last_became_peered":"2024-02-02T15:24:47.795474+0000","last_unstale":"2024-02-02T15:26:27.378946+0000","last_undegraded":"2024-02-02T15:26:27.378946+0000","last_fullsized":"2024-02-02T15:26:27.378946+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:25:28.059500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"86'12","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.379845+0000","last_change":"2024-02-02T15:24:47.786930+0000","last_active":"2024-02-02T15:26:27.379845+0000","last_peered":"2024-02-02T15:26:27.379845+0000","last_clean":"2024-02-02T15:26:27.379845+0000","last_became_active":"2024-02-02T15:24:47.786615+0000","last_became_peered":"2024-02-02T15:24:47.786615+0000","last_unstale":"2024-02-02T15:26:27.379845+0000","last_undegraded":"2024-02-02T15:26:27.379845+0000","last_fullsized":"2024-02-02T15:26:27.379845+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:17:23.231511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"86'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.379038+0000","last_change":"2024-02-02T15:24:47.797877+0000","last_active":"2024-02-02T15:26:27.379038+0000","last_peered":"2024-02-02T15:26:27.379038+0000","last_clean":"2024-02-02T15:26:27.379038+0000","last_became_active":"2024-02-02T15:24:47.797559+0000","last_became_peered":"2024-02-02T15:24:47.797559+0000","last_unstale":"2024-02-02T15:26:27.379038+0000","last_undegraded":"2024-02-02T15:26:27.379038+0000","last_fullsized":"2024-02-02T15:26:27.379038+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:13:55.529796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"86'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.486450+0000","last_change":"2024-02-02T15:24:47.790270+0000","last_active":"2024-02-02T15:26:27.486450+0000","last_peered":"2024-02-02T15:26:27.486450+0000","last_clean":"2024-02-02T15:26:27.486450+0000","last_became_active":"2024-02-02T15:24:47.789928+0000","last_became_peered":"2024-02-02T15:24:47.789928+0000","last_unstale":"2024-02-02T15:26:27.486450+0000","last_undegraded":"2024-02-02T15:26:27.486450+0000","last_fullsized":"2024-02-02T15:26:27.486450+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T02:34:44.885803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"86'4","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.269112+0000","last_change":"2024-02-02T15:25:27.298415+0000","last_active":"2024-02-02T15:26:27.269112+0000","last_peered":"2024-02-02T15:26:27.269112+0000","last_clean":"2024-02-02T15:26:27.269112+0000","last_became_active":"2024-02-02T15:25:27.297990+0000","last_became_peered":"2024-02-02T15:25:27.297990+0000","last_unstale":"2024-02-02T15:26:27.269112+0000","last_undegraded":"2024-02-02T15:26:27.269112+0000","last_fullsized":"2024-02-02T15:26:27.269112+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:38:02.040613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"86'6","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.380026+0000","last_change":"2024-02-02T15:24:47.797861+0000","last_active":"2024-02-02T15:26:27.380026+0000","last_peered":"2024-02-02T15:26:27.380026+0000","last_clean":"2024-02-02T15:26:27.380026+0000","last_became_active":"2024-02-02T15:24:47.797543+0000","last_became_peered":"2024-02-02T15:24:47.797543+0000","last_unstale":"2024-02-02T15:26:27.380026+0000","last_undegraded":"2024-02-02T15:26:27.380026+0000","last_fullsized":"2024-02-02T15:26:27.380026+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:09:37.710909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"86'15","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.380423+0000","last_change":"2024-02-02T15:24:47.796221+0000","last_active":"2024-02-02T15:26:27.380423+0000","last_peered":"2024-02-02T15:26:27.380423+0000","last_clean":"2024-02-02T15:26:27.380423+0000","last_became_active":"2024-02-02T15:24:47.795845+0000","last_became_peered":"2024-02-02T15:24:47.795845+0000","last_unstale":"2024-02-02T15:26:27.380423+0000","last_undegraded":"2024-02-02T15:26:27.380423+0000","last_fullsized":"2024-02-02T15:26:27.380423+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:54:55.637660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"86'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.376469+0000","last_change":"2024-02-02T15:24:47.783086+0000","last_active":"2024-02-02T15:26:27.376469+0000","last_peered":"2024-02-02T15:26:27.376469+0000","last_clean":"2024-02-02T15:26:27.376469+0000","last_became_active":"2024-02-02T15:24:47.781826+0000","last_became_peered":"2024-02-02T15:24:47.781826+0000","last_unstale":"2024-02-02T15:26:27.376469+0000","last_undegraded":"2024-02-02T15:26:27.376469+0000","last_fullsized":"2024-02-02T15:26:27.376469+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:32:05.623591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"86'6","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.268677+0000","last_change":"2024-02-02T15:25:27.284461+0000","last_active":"2024-02-02T15:26:27.268677+0000","last_peered":"2024-02-02T15:26:27.268677+0000","last_clean":"2024-02-02T15:26:27.268677+0000","last_became_active":"2024-02-02T15:25:27.284080+0000","last_became_peered":"2024-02-02T15:25:27.284080+0000","last_unstale":"2024-02-02T15:26:27.268677+0000","last_undegraded":"2024-02-02T15:26:27.268677+0000","last_fullsized":"2024-02-02T15:26:27.268677+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:43:40.124825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.29","version":"86'8","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-02T15:26:28.282109+0000","last_change":"2024-02-02T15:26:28.281754+0000","last_active":"2024-02-02T15:26:28.282109+0000","last_peered":"2024-02-02T15:26:28.282109+0000","last_clean":"2024-02-02T15:26:28.282109+0000","last_became_active":"2024-02-02T15:26:28.281396+0000","last_became_peered":"2024-02-02T15:26:28.281396+0000","last_unstale":"2024-02-02T15:26:28.282109+0000","last_undegraded":"2024-02-02T15:26:28.282109+0000","last_fullsized":"2024-02-02T15:26:28.282109+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:26:26.973936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2a","version":"86'10","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.277193+0000","last_change":"2024-02-02T15:24:47.795928+0000","last_active":"2024-02-02T15:26:27.277193+0000","last_peered":"2024-02-02T15:26:27.277193+0000","last_clean":"2024-02-02T15:26:27.277193+0000","last_became_active":"2024-02-02T15:24:47.795511+0000","last_became_peered":"2024-02-02T15:24:47.795511+0000","last_unstale":"2024-02-02T15:26:27.277193+0000","last_undegraded":"2024-02-02T15:26:27.277193+0000","last_fullsized":"2024-02-02T15:26:27.277193+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:55:31.444331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"86'11","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.268957+0000","last_change":"2024-02-02T15:24:47.794368+0000","last_active":"2024-02-02T15:26:27.268957+0000","last_peered":"2024-02-02T15:26:27.268957+0000","last_clean":"2024-02-02T15:26:27.268957+0000","last_became_active":"2024-02-02T15:24:47.793808+0000","last_became_peered":"2024-02-02T15:24:47.793808+0000","last_unstale":"2024-02-02T15:26:27.268957+0000","last_undegraded":"2024-02-02T15:26:27.268957+0000","last_fullsized":"2024-02-02T15:26:27.268957+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T02:39:00.099808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"86'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.379545+0000","last_change":"2024-02-02T15:24:47.796358+0000","last_active":"2024-02-02T15:26:27.379545+0000","last_peered":"2024-02-02T15:26:27.379545+0000","last_clean":"2024-02-02T15:26:27.379545+0000","last_became_active":"2024-02-02T15:24:47.795800+0000","last_became_peered":"2024-02-02T15:24:47.795800+0000","last_unstale":"2024-02-02T15:26:27.379545+0000","last_undegraded":"2024-02-02T15:26:27.379545+0000","last_fullsized":"2024-02-02T15:26:27.379545+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:49:40.019787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":250199,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67720,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":854,"num_read_kb":505,"num_write":248,"num_write_kb":144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":581632,"data_stored":213918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":216,"ondisk_log_size":216,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547234,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":983040,"data_stored":496773,"data_compressed":23664,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 15:26:15 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.53600000000000003,"15min":0.53300000000000003},"min":{"1min":0.42999999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.745,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.57099999999999995,"15min":0.55900000000000005},"min":{"1min":0.5,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.42499999999999999}]},{"osd":1,"last update":"Fri Feb 2 15:26:15 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.59599999999999997,"15min":0.58899999999999997},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.80200000000000005,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.61199999999999999,"15min":0.60299999999999998},"min":{"1min":0.39300000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.64300000000000002}]},{"osd":2,"last update":"Fri Feb 2 15:26:15 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.45500000000000002,"15min":0.44400000000000001},"min":{"1min":0.39300000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.46000000000000002,"15min":0.45100000000000001},"min":{"1min":0.376,"5min":0.316,"15min":0.316},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.44400000000000001}]},{"osd":3,"last update":"Fri Feb 2 15:26:27 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.58799999999999997}]},{"osd":4,"last update":"Fri Feb 2 15:26:15 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.64100000000000001,"15min":0.629},"min":{"1min":0.52400000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.63600000000000001,"15min":0.63400000000000001},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.76300000000000001,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.68799999999999994}]},{"osd":5,"last update":"Fri Feb 2 15:26:15 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.57999999999999996,"15min":0.56299999999999994},"min":{"1min":0.53200000000000003,"5min":0.377,"15min":0.377},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.621},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.57899999999999996,"15min":0.56599999999999995},"min":{"1min":0.504,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.53500000000000003}]},{"osd":6,"last update":"Fri Feb 2 15:26:15 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.70399999999999996,"15min":0.70599999999999996},"min":{"1min":0.57499999999999996,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.93999999999999995,"5min":1.325,"15min":1.325},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.72699999999999998,"15min":0.71499999999999997},"min":{"1min":0.64100000000000001,"5min":0.498,"15min":0.498},"max":{"1min":0.97099999999999997,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.71599999999999997}]},{"osd":7,"last update":"Fri Feb 2 15:26:15 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.71699999999999997,"15min":0.71199999999999997},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.96799999999999997,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.73799999999999999,"15min":0.73099999999999998},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.92600000000000005,"5min":1.27,"15min":1.27},"last":0.72699999999999998}]},{"osd":8,"last update":"Fri Feb 2 15:25:50 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.75800000000000001}]},{"osd":9,"last update":"Fri Feb 2 15:26:15 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.79000000000000004,"15min":0.78100000000000003},"min":{"1min":0.66900000000000004,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.0109999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.77000000000000002,"15min":0.76000000000000001},"min":{"1min":0.71999999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.006,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.79600000000000004}]},{"osd":10,"last update":"Fri Feb 2 15:26:15 2024","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.79700000000000004,"15min":0.79100000000000004},"min":{"1min":0.63,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":0.95999999999999996,"5min":1.21,"15min":1.21},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.77200000000000002,"15min":0.75800000000000001},"min":{"1min":0.71899999999999997,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.98999999999999999,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.81799999999999995}]}]},{"osd":10,"up_from":67,"seq":287762808868,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1122304,"data_stored":1145011,"data_compressed":59875,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 15:25:48 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.45300000000000001}]},{"osd":1,"last update":"Fri Feb 2 15:25:53 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.439,"15min":0.441},"min":{"1min":0.32200000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.63300000000000001,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.432},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.45900000000000002,"15min":0.46100000000000002},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.36699999999999999}]},{"osd":2,"last update":"Fri Feb 2 15:25:53 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.58799999999999997,"15min":0.59199999999999997},"min":{"1min":0.40000000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.72499999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.59399999999999997,"15min":0.59499999999999997},"min":{"1min":0.42599999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.74399999999999999,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.59299999999999997}]},{"osd":3,"last update":"Fri Feb 2 15:25:53 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.58999999999999997,"15min":0.59799999999999998},"min":{"1min":0.39900000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.73299999999999998,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.78700000000000003,"15min":0.82399999999999995},"min":{"1min":0.40600000000000003,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.70399999999999996,"5min":4.4109999999999996,"15min":4.4109999999999996},"last":0.56100000000000005}]},{"osd":4,"last update":"Fri Feb 2 15:25:53 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.36899999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.78400000000000003,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.52000000000000002,"15min":0.52500000000000002},"min":{"1min":0.375,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.65200000000000002,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.40200000000000002}]},{"osd":5,"last update":"Fri Feb 2 15:25:53 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.65600000000000003,"15min":0.65800000000000003},"min":{"1min":0.45500000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.77200000000000002,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.66300000000000003,"15min":0.66000000000000003},"min":{"1min":0.47099999999999997,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.86499999999999999,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.51700000000000002}]},{"osd":6,"last update":"Fri Feb 2 15:25:53 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.80100000000000005,"15min":0.82699999999999996},"min":{"1min":0.52300000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.82799999999999996,"5min":4.4459999999999997,"15min":4.4459999999999997},"last":0.627},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.71999999999999997,"15min":0.72899999999999998},"min":{"1min":0.44500000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.83899999999999997,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.64700000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40300000000000002}]},{"osd":8,"last update":"Fri Feb 2 15:25:53 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.71999999999999997,"15min":0.72399999999999998},"min":{"1min":0.496,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.92100000000000004,"5min":1.413,"15min":1.413},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.71899999999999997,"15min":0.72199999999999998},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.89800000000000002,"5min":1.036,"15min":1.036},"last":0.54400000000000004}]},{"osd":9,"last update":"Fri Feb 2 15:25:53 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.77600000000000002,"15min":0.78400000000000003},"min":{"1min":0.58899999999999997,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.88300000000000001,"5min":1.506,"15min":1.506},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.89800000000000002,"15min":0.93200000000000005},"min":{"1min":0.55400000000000005,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.86799999999999999,"5min":4.4269999999999996,"15min":4.4269999999999996},"last":0.63900000000000001}]},{"osd":11,"last update":"Fri Feb 2 15:26:14 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.73899999999999999,"15min":0.73299999999999998},"min":{"1min":0.67500000000000004,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.95999999999999996,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.74099999999999999,"15min":0.73799999999999999},"min":{"1min":0.64900000000000002,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.93899999999999995,"5min":0.999,"15min":0.999},"last":0.60499999999999998}]}]},{"osd":9,"up_from":62,"seq":266287972393,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1003520,"data_stored":500779,"data_compressed":23664,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 15:26:38 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.53600000000000003,"15min":0.54300000000000004},"min":{"1min":0.33500000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.069,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.55200000000000005,"15min":0.56499999999999995},"min":{"1min":0.32700000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.70799999999999996,"5min":1.196,"15min":1.196},"last":0.53400000000000003}]},{"osd":1,"last update":"Fri Feb 2 15:26:38 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.72799999999999998,"15min":0.72199999999999998},"min":{"1min":0.51600000000000001,"5min":0.38,"15min":0.38},"max":{"1min":1.0309999999999999,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.748},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.69299999999999995,"15min":0.70299999999999996},"min":{"1min":0.46800000000000003,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.80100000000000005,"5min":1.198,"15min":1.198},"last":0.67300000000000004}]},{"osd":2,"last update":"Fri Feb 2 15:26:38 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.71299999999999997,"15min":0.72999999999999998},"min":{"1min":0.50900000000000001,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.90600000000000003,"5min":1.135,"15min":1.135},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.68700000000000006,"15min":0.68300000000000005},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.66000000000000003}]},{"osd":3,"last update":"Fri Feb 2 15:26:38 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.71499999999999997,"15min":0.71899999999999997},"min":{"1min":0.48499999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.98499999999999999,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.67900000000000005,"15min":0.71499999999999997},"min":{"1min":0.46100000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.89100000000000001,"5min":1.296,"15min":1.296},"last":0.61899999999999999}]},{"osd":4,"last update":"Fri Feb 2 15:26:38 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.69899999999999995,"15min":0.71499999999999997},"min":{"1min":0.45300000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.302,"5min":1.381,"15min":1.381},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.81699999999999995,"15min":0.83199999999999996},"min":{"1min":0.55900000000000005,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.1000000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.79400000000000004}]},{"osd":5,"last update":"Fri Feb 2 15:26:38 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.84899999999999998,"15min":0.91500000000000004},"min":{"1min":0.497,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.3680000000000001,"5min":3.895,"15min":3.895},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.84399999999999997,"15min":0.85699999999999998},"min":{"1min":0.55400000000000005,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.1679999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.83899999999999997}]},{"osd":6,"last update":"Fri Feb 2 15:24:35 2024","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.6739999999999999,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":0.64500000000000002}]},{"osd":7,"last update":"Fri Feb 2 15:26:38 2024","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.89500000000000002,"15min":0.91300000000000003},"min":{"1min":0.69499999999999995,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.1359999999999999,"5min":1.347,"15min":1.347},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.84199999999999997,"15min":0.88600000000000001},"min":{"1min":0.55200000000000005,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.86399999999999999,"5min":1.413,"15min":1.413},"last":0.77900000000000003}]},{"osd":8,"last update":"Fri Feb 2 15:26:38 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.86799999999999999,"15min":0.91300000000000003},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.474,"5min":1.7789999999999999,"15min":1.7789999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.93600000000000005,"15min":0.96699999999999997},"min":{"1min":0.66900000000000004,"5min":0.625,"15min":0.625},"max":{"1min":1.1970000000000001,"5min":1.538,"15min":1.538},"last":0.86699999999999999}]},{"osd":10,"last update":"Fri Feb 2 15:25:56 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.93200000000000005,"15min":0.92700000000000005},"min":{"1min":0.71399999999999997,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.389,"5min":1.389,"15min":1.389},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.874,"5min":0.86199999999999999,"15min":0.85999999999999999},"min":{"1min":0.70499999999999996,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":0.78700000000000003}]},{"osd":11,"last update":"Fri Feb 2 15:25:51 2024","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.474,"5min":1.474,"15min":1.474},"last":0.91000000000000003}]}]},{"osd":8,"up_from":55,"seq":236223201325,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":991232,"data_stored":496730,"data_compressed":23664,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 15:26:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.57899999999999996,"15min":0.58199999999999996},"min":{"1min":0.44900000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.71099999999999997,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.58599999999999997,"15min":0.59799999999999998},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.73799999999999999,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.50600000000000001}]},{"osd":1,"last update":"Fri Feb 2 15:26:16 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.59399999999999997,"15min":0.57299999999999995},"min":{"1min":0.45900000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.83099999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.626},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.59399999999999997,"15min":0.56999999999999995},"min":{"1min":0.505,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.79500000000000004,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.64100000000000001}]},{"osd":2,"last update":"Fri Feb 2 15:26:16 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.44700000000000001,"15min":0.42599999999999999},"min":{"1min":0.34999999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.499,"5min":0.44,"15min":0.42199999999999999},"min":{"1min":0.38500000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.59699999999999998}]},{"osd":3,"last update":"Fri Feb 2 15:26:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.61399999999999999,"15min":0.625},"min":{"1min":0.40999999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.751,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.61299999999999999,"15min":0.62},"min":{"1min":0.435,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.58499999999999996}]},{"osd":4,"last update":"Fri Feb 2 15:26:16 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.621,"15min":0.60499999999999998},"min":{"1min":0.42199999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.84599999999999997,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.61899999999999999,"15min":0.59899999999999998},"min":{"1min":0.45400000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.67800000000000005}]},{"osd":5,"last update":"Fri Feb 2 15:26:16 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.53400000000000003,"15min":0.51100000000000001},"min":{"1min":0.49399999999999999,"5min":0.311,"15min":0.311},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.53400000000000003,"15min":0.51000000000000001},"min":{"1min":0.47399999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.71499999999999997}]},{"osd":6,"last update":"Fri Feb 2 15:26:16 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.68300000000000005,"15min":0.67900000000000005},"min":{"1min":0.54800000000000004,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.70599999999999996,"15min":0.69799999999999995},"min":{"1min":0.56000000000000005,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.76100000000000001}]},{"osd":7,"last update":"Fri Feb 2 15:26:16 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.67200000000000004,"15min":0.64900000000000002},"min":{"1min":0.62,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.69799999999999995,"15min":0.67800000000000005},"min":{"1min":0.61299999999999999,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.96099999999999997,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.752}]},{"osd":9,"last update":"Fri Feb 2 15:26:40 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.72199999999999998,"15min":0.69799999999999995},"min":{"1min":0.53600000000000003,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.86799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.74399999999999999,"15min":0.71999999999999997},"min":{"1min":0.61899999999999999,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.90300000000000002,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.83299999999999996}]},{"osd":10,"last update":"Fri Feb 2 15:25:56 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.73099999999999998,"15min":0.71699999999999997},"min":{"1min":0.56200000000000006,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.94599999999999995,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.72899999999999998,"15min":0.71299999999999997},"min":{"1min":0.63600000000000001,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.80900000000000005}]},{"osd":11,"last update":"Fri Feb 2 15:25:48 2024","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.78500000000000003}]}]},{"osd":7,"up_from":48,"seq":206158430256,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1048576,"data_stored":1093556,"data_compressed":59875,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 15:23:58 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.439},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.38500000000000001}]},{"osd":1,"last update":"Fri Feb 2 15:26:01 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.432,"15min":0.42799999999999999},"min":{"1min":0.34399999999999997,"5min":0.247,"15min":0.247},"max":{"1min":0.64300000000000002,"5min":0.749,"15min":0.749},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.42199999999999999,"15min":0.41099999999999998},"min":{"1min":0.37,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.65500000000000003,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.53100000000000003}]},{"osd":2,"last update":"Fri Feb 2 15:26:01 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.57499999999999996,"15min":0.57199999999999995},"min":{"1min":0.434,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.77000000000000002,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.59599999999999997,"15min":0.58799999999999997},"min":{"1min":0.48599999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.76500000000000001,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.75600000000000001}]},{"osd":3,"last update":"Fri Feb 2 15:26:01 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.64300000000000002,"15min":0.61099999999999999},"min":{"1min":0.51000000000000001,"5min":0.374,"15min":0.374},"max":{"1min":4.3890000000000002,"5min":4.3890000000000002,"15min":4.3890000000000002},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.623,"15min":0.60499999999999998},"min":{"1min":0.47599999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":4.1280000000000001,"5min":4.1280000000000001,"15min":4.1280000000000001},"last":0.60199999999999998}]},{"osd":4,"last update":"Fri Feb 2 15:26:01 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.50600000000000001,"15min":0.501},"min":{"1min":0.38800000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.78500000000000003,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.5,"15min":0.496},"min":{"1min":0.35999999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.437}]},{"osd":5,"last update":"Fri Feb 2 15:26:01 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.64000000000000001,"15min":0.622},"min":{"1min":0.44900000000000001,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.81299999999999994,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.63100000000000001,"15min":0.625},"min":{"1min":0.54700000000000004,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.88600000000000001,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.63500000000000001}]},{"osd":6,"last update":"Fri Feb 2 15:26:01 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.68400000000000005,"15min":0.68999999999999995},"min":{"1min":0.54400000000000004,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.96399999999999997,"5min":0.998,"15min":0.998},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.67300000000000004,"15min":0.66700000000000004},"min":{"1min":0.46400000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.95599999999999996,"5min":1.778,"15min":1.778},"last":0.66100000000000003}]},{"osd":8,"last update":"Fri Feb 2 15:26:17 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.71199999999999997,"15min":0.70299999999999996},"min":{"1min":0.53600000000000003,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.97699999999999998,"5min":2.069,"15min":2.069},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.73599999999999999,"15min":0.73499999999999999},"min":{"1min":0.65900000000000003,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.93100000000000005,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.74099999999999999}]},{"osd":9,"last update":"Fri Feb 2 15:25:40 2024","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.76100000000000001,"15min":0.747},"min":{"1min":0.497,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":4.9550000000000001,"5min":4.9550000000000001,"15min":4.9550000000000001},"last":0.62},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.753,"15min":0.73899999999999999},"min":{"1min":0.54500000000000004,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":4.0759999999999996,"5min":4.0759999999999996,"15min":4.0759999999999996},"last":0.67600000000000005}]},{"osd":10,"last update":"Fri Feb 2 15:25:55 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.47099999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.82499999999999996,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.495,"5min":0.436,"15min":0.436},"max":{"1min":0.88800000000000001,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.72499999999999998}]},{"osd":11,"last update":"Fri Feb 2 15:25:51 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.85199999999999998}]}]},{"osd":6,"up_from":43,"seq":184683593780,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":974848,"data_stored":492139,"data_compressed":23664,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 15:25:44 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.497,"15min":0.48499999999999999},"min":{"1min":0.35599999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.67900000000000005,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51900000000000002,"15min":0.51200000000000001},"min":{"1min":0.371,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.67200000000000004,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.442}]},{"osd":1,"last update":"Fri Feb 2 15:25:44 2024","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":0.72199999999999998,"15min":0.66400000000000003},"min":{"1min":0.47999999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":6.8650000000000002,"5min":6.8650000000000002,"15min":6.8650000000000002},"last":0.626},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.66400000000000003,"15min":0.65200000000000002},"min":{"1min":0.50600000000000001,"5min":0.501,"15min":0.501},"max":{"1min":0.88900000000000001,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.501}]},{"osd":2,"last update":"Fri Feb 2 15:25:44 2024","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":0.79700000000000004,"15min":0.72699999999999998},"min":{"1min":0.57199999999999995,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":6.9320000000000004,"5min":6.9320000000000004,"15min":6.9320000000000004},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.753,"15min":0.72499999999999998},"min":{"1min":0.52600000000000002,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":3.008,"5min":3.008,"15min":3.008},"last":0.68700000000000006}]},{"osd":3,"last update":"Fri Feb 2 15:25:44 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.628,"15min":0.63300000000000001},"min":{"1min":0.432,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.875,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.67400000000000004,"15min":0.66400000000000003},"min":{"1min":0.45100000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.73299999999999998}]},{"osd":4,"last update":"Fri Feb 2 15:25:44 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.72999999999999998,"15min":0.71999999999999997},"min":{"1min":0.53100000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.95599999999999996,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76200000000000001,"15min":0.75600000000000001},"min":{"1min":0.55700000000000005,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.0489999999999999,"5min":1.228,"15min":1.228},"last":0.58999999999999997}]},{"osd":5,"last update":"Fri Feb 2 15:25:44 2024","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.79100000000000004,"15min":0.77900000000000003},"min":{"1min":0.55100000000000005,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.63700000000000001},{"interface":"front","average":{"1min":1.268,"5min":0.90900000000000003,"15min":0.84499999999999997},"min":{"1min":0.61199999999999999,"5min":0.442,"15min":0.442},"max":{"1min":6.9820000000000002,"5min":6.9820000000000002,"15min":6.9820000000000002},"last":0.84199999999999997}]},{"osd":7,"last update":"Fri Feb 2 15:26:02 2024","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":0.88900000000000001,"15min":0.84999999999999998},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":6.9530000000000003,"5min":6.9530000000000003,"15min":6.9530000000000003},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.876,"15min":0.878},"min":{"1min":0.60899999999999999,"5min":0.501,"15min":0.501},"max":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.72099999999999997}]},{"osd":8,"last update":"Fri Feb 2 15:26:15 2024","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.88200000000000001,"15min":0.89500000000000002},"min":{"1min":0.60699999999999998,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.0429999999999999,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.90900000000000003,"15min":0.91000000000000003},"min":{"1min":0.625,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.0620000000000001,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.749}]},{"osd":9,"last update":"Fri Feb 2 15:26:41 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.93300000000000005,"15min":0.94299999999999995},"min":{"1min":0.69799999999999995,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.8280000000000001,"5min":1.8280000000000001,"15min":1.8280000000000001},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.95799999999999996,"15min":0.97399999999999998},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.6299999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":0.74099999999999999}]},{"osd":10,"last update":"Fri Feb 2 15:25:56 2024","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1,"15min":0.96399999999999997},"min":{"1min":0.68300000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":6.9960000000000004,"5min":6.9960000000000004,"15min":6.9960000000000004},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.97799999999999998,"15min":0.98699999999999999},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.1879999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.88700000000000001}]},{"osd":11,"last update":"Fri Feb 2 15:25:51 2024","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.92300000000000004},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":3.032,"5min":3.032,"15min":3.032},"last":0.78500000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574918,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1048576,"data_stored":559505,"data_compressed":23664,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 15:26:16 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51100000000000001,"15min":0.50800000000000001},"min":{"1min":0.41999999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.66100000000000003,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.53400000000000003,"15min":0.52400000000000002},"min":{"1min":0.45700000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.72999999999999998,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.68999999999999995}]},{"osd":2,"last update":"Fri Feb 2 15:26:37 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.58199999999999996,"15min":0.55600000000000005},"min":{"1min":0.42699999999999999,"5min":0.31,"15min":0.31},"max":{"1min":0.80600000000000005,"5min":0.871,"15min":0.871},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.59199999999999997,"15min":0.55600000000000005},"min":{"1min":0.44900000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.51500000000000001}]},{"osd":3,"last update":"Fri Feb 2 15:25:50 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.56899999999999995,"15min":0.53700000000000003},"min":{"1min":0.434,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.57999999999999996,"15min":0.55400000000000005},"min":{"1min":0.45800000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.621}]},{"osd":4,"last update":"Fri Feb 2 15:26:05 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.43099999999999999,"15min":0.38100000000000001},"min":{"1min":0.35299999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.754,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.433,"15min":0.38600000000000001},"min":{"1min":0.376,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.69799999999999995,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.47499999999999998}]},{"osd":5,"last update":"Fri Feb 2 15:26:22 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.59499999999999997,"15min":0.55300000000000005},"min":{"1min":0.52800000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.438},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.626,"15min":0.56999999999999995},"min":{"1min":0.58399999999999996,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.85699999999999998,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.64300000000000002}]},{"osd":6,"last update":"Fri Feb 2 15:25:46 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.621,"15min":0.59499999999999997},"min":{"1min":0.50800000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.60599999999999998,"15min":0.58799999999999997},"min":{"1min":0.48099999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.71499999999999997}]},{"osd":7,"last update":"Fri Feb 2 15:26:01 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.59599999999999997,"15min":0.57999999999999996},"min":{"1min":0.45000000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.90500000000000003,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.55800000000000005,"15min":0.54000000000000004},"min":{"1min":0.434,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.79100000000000004,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.48599999999999999}]},{"osd":8,"last update":"Fri Feb 2 15:26:16 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.65800000000000003,"15min":0.63800000000000001},"min":{"1min":0.51000000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.68600000000000005,"15min":0.65600000000000003},"min":{"1min":0.55100000000000005,"5min":0.442,"15min":0.442},"max":{"1min":0.94099999999999995,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.67400000000000004}]},{"osd":9,"last update":"Fri Feb 2 15:26:39 2024","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.73899999999999999,"15min":0.71399999999999997},"min":{"1min":0.47499999999999998,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":2.6840000000000002,"5min":2.6840000000000002,"15min":2.6840000000000002},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.76300000000000001,"15min":0.747},"min":{"1min":0.70999999999999996,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.96199999999999997,"5min":1.121,"15min":1.121},"last":0.79100000000000004}]},{"osd":10,"last update":"Fri Feb 2 15:25:56 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.69499999999999995,"15min":0.68500000000000005},"min":{"1min":0.57399999999999995,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.72699999999999998,"15min":0.71499999999999997},"min":{"1min":0.57999999999999996,"5min":0.5,"15min":0.5},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.73099999999999998}]},{"osd":11,"last update":"Fri Feb 2 15:25:50 2024","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.77600000000000002}]}]},{"osd":0,"up_from":8,"seq":34359738441,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":999424,"data_stored":493595,"data_compressed":23664,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Feb 2 15:26:11 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.56599999999999995,"15min":0.54600000000000004},"min":{"1min":0.374,"5min":0.314,"15min":0.314},"max":{"1min":0.79500000000000004,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.63},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.59999999999999998,"15min":0.55400000000000005},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.411,"5min":1.411,"15min":1.411},"last":0.78400000000000003}]},{"osd":2,"last update":"Fri Feb 2 15:26:32 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.70499999999999996,"15min":0.59999999999999998},"min":{"1min":0.52900000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.369,"5min":1.369,"15min":1.369},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.745,"5min":0.74299999999999999,"15min":0.628},"min":{"1min":0.373,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.9079999999999999,"5min":4.6550000000000002,"15min":4.6550000000000002},"last":0.57299999999999995}]},{"osd":3,"last update":"Fri Feb 2 15:25:48 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.495},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":0.44600000000000001}]},{"osd":4,"last update":"Fri Feb 2 15:26:05 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.68400000000000005,"15min":0.64700000000000002},"min":{"1min":0.44400000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.66000000000000003,"15min":0.63200000000000001},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.8049999999999999,"5min":1.8049999999999999,"15min":1.8049999999999999},"last":0.64900000000000002}]},{"osd":5,"last update":"Fri Feb 2 15:26:22 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.69999999999999996,"15min":0.68899999999999995},"min":{"1min":0.52300000000000002,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.9450000000000001,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":0.748},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.71699999999999997,"15min":0.67800000000000005},"min":{"1min":0.44400000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.98599999999999999,"5min":4.5890000000000004,"15min":4.5890000000000004},"last":0.47999999999999998}]},{"osd":6,"last update":"Fri Feb 2 15:25:43 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.72899999999999998,"15min":0.70499999999999996},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.494,"5min":1.494,"15min":1.494},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.74099999999999999,"15min":0.73099999999999998},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.872}]},{"osd":7,"last update":"Fri Feb 2 15:25:55 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.79800000000000004}]},{"osd":8,"last update":"Fri Feb 2 15:26:22 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.84899999999999998,"15min":0.84299999999999997},"min":{"1min":0.65900000000000003,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":2.0169999999999999,"5min":2.0169999999999999,"15min":2.0169999999999999},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.79200000000000004,"15min":0.77400000000000002},"min":{"1min":0.57199999999999995,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.1479999999999999,"5min":1.155,"15min":1.155},"last":0.76800000000000002}]},{"osd":9,"last update":"Fri Feb 2 15:25:40 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.96399999999999997,"15min":0.98699999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0980000000000001,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.95399999999999996,"15min":0.96799999999999997},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.411,"5min":1.411,"15min":1.411},"last":0.92000000000000004}]},{"osd":10,"last update":"Fri Feb 2 15:26:01 2024","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.879,"15min":0.878},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.84399999999999997},{"interface":"front","average":{"1min":1,"5min":0.97199999999999998,"15min":0.96799999999999997},"min":{"1min":0.66100000000000003,"5min":0.63,"15min":0.63},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.96099999999999997}]},{"osd":11,"last update":"Fri Feb 2 15:26:11 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.93500000000000005,"15min":0.93899999999999995},"min":{"1min":0.73299999999999998,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":0.88400000000000001},{"interface":"front","average":{"1min":1.008,"5min":0.97499999999999998,"15min":0.96899999999999997},"min":{"1min":0.81100000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.6000000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.98299999999999998}]}]},{"osd":2,"up_from":18,"seq":77309411395,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1060864,"data_stored":559609,"data_compressed":23664,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 15:26:29 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.55500000000000005,"15min":0.54600000000000004},"min":{"1min":0.45200000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.79500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.53800000000000003,"15min":0.52400000000000002},"min":{"1min":0.40899999999999997,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.85999999999999999,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.53300000000000003}]},{"osd":1,"last update":"Fri Feb 2 15:26:29 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.55100000000000005,"15min":0.51300000000000001},"min":{"1min":0.35299999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.496},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.59199999999999997,"15min":0.54300000000000004},"min":{"1min":0.41199999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.86299999999999999,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.76200000000000001}]},{"osd":3,"last update":"Fri Feb 2 15:25:46 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.60099999999999998,"15min":0.57199999999999995},"min":{"1min":0.46100000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.75600000000000001,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.623,"5min":0.58799999999999997,"15min":0.55600000000000005},"min":{"1min":0.42799999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.81000000000000005,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.60899999999999999}]},{"osd":4,"last update":"Fri Feb 2 15:25:50 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.65600000000000003}]},{"osd":5,"last update":"Fri Feb 2 15:26:29 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.45700000000000002,"15min":0.41799999999999998},"min":{"1min":0.36699999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.78900000000000003,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.44700000000000001,"15min":0.41499999999999998},"min":{"1min":0.36799999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.63600000000000001}]},{"osd":6,"last update":"Fri Feb 2 15:25:40 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.625,"15min":0.61899999999999999},"min":{"1min":0.46100000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.79500000000000004,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.628,"5min":0.625,"15min":0.627},"min":{"1min":0.437,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.746}]},{"osd":7,"last update":"Fri Feb 2 15:25:52 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.78900000000000003}]},{"osd":8,"last update":"Fri Feb 2 15:26:18 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.59099999999999997,"15min":0.57699999999999996},"min":{"1min":0.40500000000000003,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.55700000000000005,"15min":0.54300000000000004},"min":{"1min":0.41999999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.70799999999999996}]},{"osd":9,"last update":"Fri Feb 2 15:25:40 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.66900000000000004,"15min":0.66700000000000004},"min":{"1min":0.501,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.751,"15min":0.746},"min":{"1min":0.60299999999999998,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.92500000000000004,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.81399999999999995}]},{"osd":10,"last update":"Fri Feb 2 15:25:52 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.68300000000000005,"15min":0.67200000000000004},"min":{"1min":0.628,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.68100000000000005,"15min":0.67100000000000004},"min":{"1min":0.56299999999999994,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.86799999999999999,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.86599999999999999}]},{"osd":11,"last update":"Fri Feb 2 15:25:46 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.89900000000000002}]}]},{"osd":3,"up_from":24,"seq":103079215166,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":987136,"data_stored":491895,"data_compressed":23664,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 15:25:44 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.442,"15min":0.41899999999999998},"min":{"1min":0.34899999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.69699999999999995,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.46600000000000003,"15min":0.45300000000000001},"min":{"1min":0.33300000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.73099999999999998,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.442}]},{"osd":1,"last update":"Fri Feb 2 15:25:44 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.65000000000000002,"15min":0.59599999999999997},"min":{"1min":0.52900000000000003,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.0649999999999999,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.622,"15min":0.59899999999999998},"min":{"1min":0.49099999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.879,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.52400000000000002}]},{"osd":2,"last update":"Fri Feb 2 15:25:44 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.63300000000000001,"15min":0.623},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.625},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.70299999999999996,"15min":0.65700000000000003},"min":{"1min":0.56799999999999995,"5min":0.443,"15min":0.443},"max":{"1min":1.1319999999999999,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":0.504}]},{"osd":4,"last update":"Fri Feb 2 15:26:01 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.62,"15min":0.624},"min":{"1min":0.495,"5min":0.376,"15min":0.376},"max":{"1min":0.873,"5min":1.022,"15min":1.022},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.72599999999999998,"15min":0.69799999999999995},"min":{"1min":0.49199999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.75900000000000001}]},{"osd":5,"last update":"Fri Feb 2 15:26:25 2024","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.748,"15min":0.72599999999999998},"min":{"1min":0.57699999999999996,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.88600000000000001,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.66400000000000003,"15min":0.64700000000000002},"min":{"1min":0.50900000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.84399999999999997,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.69699999999999995}]},{"osd":6,"last update":"Fri Feb 2 15:25:44 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.74199999999999999,"15min":0.71599999999999997},"min":{"1min":0.57099999999999995,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.1100000000000001,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.68999999999999995,"15min":0.68799999999999994},"min":{"1min":0.40500000000000003,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.90600000000000003,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.58199999999999996}]},{"osd":7,"last update":"Fri Feb 2 15:26:01 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.751,"15min":0.74099999999999999},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.008,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.501,"5min":0.44,"15min":0.44},"max":{"1min":0.89900000000000002,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.74199999999999999}]},{"osd":8,"last update":"Fri Feb 2 15:24:14 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.65800000000000003}]},{"osd":9,"last update":"Fri Feb 2 15:26:38 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.91100000000000003,"15min":0.94199999999999995},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.93700000000000006,"5min":1.2,"15min":1.2},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.81999999999999995,"15min":0.84299999999999997},"min":{"1min":0.55400000000000005,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0629999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.77300000000000002}]},{"osd":10,"last update":"Fri Feb 2 15:25:59 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.87,"15min":0.877},"min":{"1min":0.65500000000000003,"5min":0.624,"15min":0.624},"max":{"1min":1.0840000000000001,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.95699999999999996,"15min":0.96699999999999997},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.83299999999999996}]},{"osd":11,"last update":"Fri Feb 2 15:25:49 2024","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.68300000000000005}]}]},{"osd":4,"up_from":31,"seq":133143986236,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":496746,"data_compressed":23664,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 15:24:03 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.52300000000000002,"15min":0.52100000000000002},"min":{"1min":0.371,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.78600000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.53800000000000003,"15min":0.53600000000000003},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.47799999999999998}]},{"osd":1,"last update":"Fri Feb 2 15:26:06 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.376,"15min":0.34300000000000003},"min":{"1min":0.33300000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.55000000000000004,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.38700000000000001,"15min":0.35499999999999998},"min":{"1min":0.29899999999999999,"5min":0.255,"15min":0.255},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.54400000000000004}]},{"osd":2,"last update":"Fri Feb 2 15:26:06 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.54600000000000004,"15min":0.51400000000000001},"min":{"1min":0.438,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.54700000000000004,"15min":0.51700000000000002},"min":{"1min":0.52500000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.67900000000000005,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.63300000000000001}]},{"osd":3,"last update":"Fri Feb 2 15:26:06 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.61099999999999999,"15min":0.57899999999999996},"min":{"1min":0.40200000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":4.3220000000000001,"5min":4.3220000000000001,"15min":4.3220000000000001},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.60699999999999998,"15min":0.57599999999999996},"min":{"1min":0.436,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":4.3959999999999999,"5min":4.3959999999999999,"15min":4.3959999999999999},"last":0.90700000000000003}]},{"osd":5,"last update":"Fri Feb 2 15:26:24 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.63200000000000001,"15min":0.59599999999999997},"min":{"1min":0.59099999999999997,"5min":0.374,"15min":0.374},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.626,"15min":0.60799999999999998},"min":{"1min":0.53600000000000003,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.81599999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.69699999999999995}]},{"osd":6,"last update":"Fri Feb 2 15:25:41 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.63600000000000001,"15min":0.63200000000000001},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.752,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.64200000000000002},"min":{"1min":0.49199999999999999,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.84499999999999997,"5min":1.335,"15min":1.335},"last":0.82799999999999996}]},{"osd":7,"last update":"Fri Feb 2 15:26:00 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.497,"15min":0.48499999999999999},"min":{"1min":0.34000000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.747,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.54500000000000004,"15min":0.53500000000000003},"min":{"1min":0.46200000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76000000000000001,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.61399999999999999}]},{"osd":8,"last update":"Fri Feb 2 15:26:19 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.66200000000000003,"15min":0.65400000000000003},"min":{"1min":0.55100000000000005,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.877,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.753,"5min":0.67700000000000005,"15min":0.65800000000000003},"min":{"1min":0.5,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.78100000000000003}]},{"osd":9,"last update":"Fri Feb 2 15:26:39 2024","interfaces":[{"interface":"back","average":{"1min":1.78,"5min":0.93000000000000005,"15min":0.79300000000000004},"min":{"1min":0.48799999999999999,"5min":0.441,"15min":0.441},"max":{"1min":20.126999999999999,"5min":20.126999999999999,"15min":20.126999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":1.5409999999999999,"5min":0.89200000000000002,"15min":0.77900000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":14.965999999999999,"5min":14.965999999999999,"15min":14.965999999999999},"last":0.84599999999999997}]},{"osd":10,"last update":"Fri Feb 2 15:25:57 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.67700000000000005,"15min":0.67300000000000004},"min":{"1min":0.48199999999999998,"5min":0.379,"15min":0.379},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.66800000000000004,"15min":0.66700000000000004},"min":{"1min":0.55200000000000005,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.79100000000000004,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.75900000000000001}]},{"osd":11,"last update":"Fri Feb 2 15:25:48 2024","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.79400000000000004}]}]},{"osd":5,"up_from":36,"seq":154618822711,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":1088701,"data_compressed":59875,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 15:26:23 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.53800000000000003,"15min":0.51800000000000002},"min":{"1min":0.30399999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.72399999999999998,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.504},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.54000000000000004,"15min":0.52600000000000002},"min":{"1min":0.33500000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.67100000000000004,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.372}]},{"osd":1,"last update":"Fri Feb 2 15:26:23 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.56200000000000006,"15min":0.55000000000000004},"min":{"1min":0.318,"5min":0.252,"15min":0.252},"max":{"1min":0.69099999999999995,"5min":1.018,"15min":1.018},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.57699999999999996,"15min":0.55100000000000005},"min":{"1min":0.40100000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.78000000000000003,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.60299999999999998}]},{"osd":2,"last update":"Fri Feb 2 15:26:23 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.40699999999999997,"15min":0.38700000000000001},"min":{"1min":0.30599999999999999,"5min":0.215,"15min":0.215},"max":{"1min":0.59099999999999997,"5min":0.63,"15min":0.63},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.42199999999999999,"15min":0.40400000000000003},"min":{"1min":0.32900000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.64900000000000002,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.28100000000000003}]},{"osd":3,"last update":"Fri Feb 2 15:26:23 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.56000000000000005,"15min":0.52700000000000002},"min":{"1min":0.434,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.69899999999999995,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.58499999999999996,"15min":0.56699999999999995},"min":{"1min":0.44700000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.74399999999999999,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.42399999999999999}]},{"osd":4,"last update":"Fri Feb 2 15:26:23 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.59099999999999997,"15min":0.56699999999999995},"min":{"1min":0.45600000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.77200000000000002,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.437},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.59899999999999998,"15min":0.57999999999999996},"min":{"1min":0.42199999999999999,"5min":0.316,"15min":0.316},"max":{"1min":0.79900000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.56499999999999995}]},{"osd":6,"last update":"Fri Feb 2 15:25:42 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.63300000000000001,"15min":0.63200000000000001},"min":{"1min":0.51700000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.84599999999999997,"5min":0.878,"15min":0.878},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.63400000000000001,"15min":0.63300000000000001},"min":{"1min":0.57599999999999996,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.81999999999999995,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.46500000000000002}]},{"osd":7,"last update":"Fri Feb 2 15:26:01 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.64200000000000002,"15min":0.626},"min":{"1min":0.52700000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.621,"15min":0.60899999999999999},"min":{"1min":0.49299999999999999,"5min":0.373,"15min":0.373},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.66700000000000004}]},{"osd":8,"last update":"Fri Feb 2 15:26:18 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.57299999999999995,"15min":0.56100000000000005},"min":{"1min":0.43099999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.84499999999999997,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.60199999999999998,"15min":0.59699999999999998},"min":{"1min":0.379,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.92700000000000005,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.53300000000000003}]},{"osd":9,"last update":"Fri Feb 2 15:26:38 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.69999999999999996,"15min":0.68600000000000005},"min":{"1min":0.52400000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.86499999999999999,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.62},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71599999999999997,"15min":0.70599999999999996},"min":{"1min":0.54500000000000004,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.86599999999999999,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.63200000000000001}]},{"osd":10,"last update":"Fri Feb 2 15:25:54 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.70899999999999996,"15min":0.69999999999999996},"min":{"1min":0.57699999999999996,"5min":0.497,"15min":0.497},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.72799999999999998,"15min":0.72299999999999998},"min":{"1min":0.54000000000000004,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1,"5min":1,"15min":1},"last":0.58099999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34572,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34572,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36211,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36211,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36211,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36211,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36211,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":68040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":68144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":63178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-02T15:26:45.751 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-02T15:26:45.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.982+0000 7f0579a00700 1 -- 172.21.15.1:0/3619281069 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0574100b20 msgr2=0x7f0574100f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:45.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.982+0000 7f0579a00700 1 --2- 172.21.15.1:0/3619281069 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0574100b20 0x7f0574100f40 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f05640099c0 tx=0x7f0564009cd0 comp rx=0 tx=0).stop 2024-02-02T15:26:45.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.982+0000 7f0579a00700 1 -- 172.21.15.1:0/3619281069 shutdown_connections 2024-02-02T15:26:45.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.982+0000 7f0579a00700 1 --2- 172.21.15.1:0/3619281069 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05741026b0 0x7f0574108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:45.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.982+0000 7f0579a00700 1 --2- 172.21.15.1:0/3619281069 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0574100b20 0x7f0574100f40 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:45.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.982+0000 7f0579a00700 1 --2- 172.21.15.1:0/3619281069 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0574101d10 0x7f0574102170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:45.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.982+0000 7f0579a00700 1 -- 172.21.15.1:0/3619281069 >> 172.21.15.1:0/3619281069 conn(0x7f05740fc050 msgr2=0x7f05740fe4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:45.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.982+0000 7f0579a00700 1 -- 172.21.15.1:0/3619281069 shutdown_connections 2024-02-02T15:26:45.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.982+0000 7f0579a00700 1 -- 172.21.15.1:0/3619281069 wait complete. 2024-02-02T15:26:45.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.983+0000 7f0579a00700 1 Processor -- start 2024-02-02T15:26:45.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.983+0000 7f0579a00700 1 -- start start 2024-02-02T15:26:45.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.983+0000 7f0579a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0574100b20 0x7f0574198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:45.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.983+0000 7f0579a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0574101d10 0x7f0574198f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:45.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.983+0000 7f0579a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05741026b0 0x7f057419fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:45.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.983+0000 7f0579a00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05741a05c0 con 0x7f0574100b20 2024-02-02T15:26:45.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.983+0000 7f0579a00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05741a0700 con 0x7f0574101d10 2024-02-02T15:26:45.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.983+0000 7f0579a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05741a0890 con 0x7f05741026b0 2024-02-02T15:26:45.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.984+0000 7f0573e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0574100b20 0x7f0574198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:45.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.984+0000 7f0573e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0574100b20 0x7f0574198a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53408/0 (socket says 172.21.15.1:53408) 2024-02-02T15:26:45.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.984+0000 7f0573e00700 1 -- 172.21.15.1:0/4272017835 learned_addr learned my addr 172.21.15.1:0/4272017835 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:45.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.984+0000 7f0573400700 1 --2- 172.21.15.1:0/4272017835 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0574101d10 0x7f0574198f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:45.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.984+0000 7f0579000700 1 --2- 172.21.15.1:0/4272017835 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05741026b0 0x7f057419fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:45.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.984+0000 7f0573e00700 1 -- 172.21.15.1:0/4272017835 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05741026b0 msgr2=0x7f057419fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:45.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.984+0000 7f0573e00700 1 --2- 172.21.15.1:0/4272017835 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05741026b0 0x7f057419fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:45.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.984+0000 7f0573e00700 1 -- 172.21.15.1:0/4272017835 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0574101d10 msgr2=0x7f0574198f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:45.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.984+0000 7f0573e00700 1 --2- 172.21.15.1:0/4272017835 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0574101d10 0x7f0574198f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:45.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.984+0000 7f0573e00700 1 -- 172.21.15.1:0/4272017835 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05640095f0 con 0x7f0574100b20 2024-02-02T15:26:45.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.984+0000 7f0579000700 1 --2- 172.21.15.1:0/4272017835 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05741026b0 0x7f057419fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:26:45.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.984+0000 7f0573e00700 1 --2- 172.21.15.1:0/4272017835 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0574100b20 0x7f0574198a40 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f05640098e0 tx=0x7f056400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:45.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.985+0000 7f0570c00700 1 -- 172.21.15.1:0/4272017835 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f056401d070 con 0x7f0574100b20 2024-02-02T15:26:45.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.985+0000 7f0570c00700 1 -- 172.21.15.1:0/4272017835 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0564005510 con 0x7f0574100b20 2024-02-02T15:26:45.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.985+0000 7f0579a00700 1 -- 172.21.15.1:0/4272017835 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05741a0b10 con 0x7f0574100b20 2024-02-02T15:26:45.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.985+0000 7f0570c00700 1 -- 172.21.15.1:0/4272017835 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0564004c00 con 0x7f0574100b20 2024-02-02T15:26:45.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.985+0000 7f0579a00700 1 -- 172.21.15.1:0/4272017835 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05741a0f30 con 0x7f0574100b20 2024-02-02T15:26:45.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.986+0000 7f0579a00700 1 -- 172.21.15.1:0/4272017835 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0574193170 con 0x7f0574100b20 2024-02-02T15:26:45.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.988+0000 7f0570c00700 1 -- 172.21.15.1:0/4272017835 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) v1 ==== 135069+0+0 (secure 0 0 0) 0x7f056400fb80 con 0x7f0574100b20 2024-02-02T15:26:45.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.989+0000 7f0570c00700 1 --2- 172.21.15.1:0/4272017835 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f05540a05a0 0x7f05540a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:45.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.990+0000 7f0570c00700 1 -- 172.21.15.1:0/4272017835 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f05640ccc30 con 0x7f0574100b20 2024-02-02T15:26:45.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.990+0000 7f0573400700 1 --2- 172.21.15.1:0/4272017835 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f05540a05a0 0x7f05540a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:45.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.990+0000 7f0573400700 1 --2- 172.21.15.1:0/4272017835 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f05540a05a0 0x7f05540a2a60 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f055c007c60 tx=0x7f055c0073d0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:45.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:45.993+0000 7f0570c00700 1 -- 172.21.15.1:0/4272017835 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f0564096490 con 0x7f0574100b20 2024-02-02T15:26:46.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.199+0000 7f0579a00700 1 -- 172.21.15.1:0/4272017835 --> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0574061180 con 0x7f05540a05a0 2024-02-02T15:26:46.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.212+0000 7f0570c00700 1 -- 172.21.15.1:0/4272017835 <== mgr.14154 v2:172.21.15.1:6800/3768989981 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427911 (secure 0 0 0) 0x7f0574061180 con 0x7f05540a05a0 2024-02-02T15:26:46.216 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:26:46.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.217+0000 7f0579a00700 1 -- 172.21.15.1:0/4272017835 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f05540a05a0 msgr2=0x7f05540a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:46.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.217+0000 7f0579a00700 1 --2- 172.21.15.1:0/4272017835 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f05540a05a0 0x7f05540a2a60 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f055c007c60 tx=0x7f055c0073d0 comp rx=0 tx=0).stop 2024-02-02T15:26:46.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.217+0000 7f0579a00700 1 -- 172.21.15.1:0/4272017835 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0574100b20 msgr2=0x7f0574198a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:46.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.217+0000 7f0579a00700 1 --2- 172.21.15.1:0/4272017835 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0574100b20 0x7f0574198a40 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f05640098e0 tx=0x7f056400f690 comp rx=0 tx=0).stop 2024-02-02T15:26:46.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.217+0000 7f0579a00700 1 -- 172.21.15.1:0/4272017835 shutdown_connections 2024-02-02T15:26:46.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.218+0000 7f0579a00700 1 --2- 172.21.15.1:0/4272017835 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f05540a05a0 0x7f05540a2a60 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:46.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.218+0000 7f0579a00700 1 --2- 172.21.15.1:0/4272017835 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05741026b0 0x7f057419fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:46.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.218+0000 7f0579a00700 1 --2- 172.21.15.1:0/4272017835 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0574100b20 0x7f0574198a40 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:46.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.218+0000 7f0579a00700 1 --2- 172.21.15.1:0/4272017835 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0574101d10 0x7f0574198f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:46.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.218+0000 7f0579a00700 1 -- 172.21.15.1:0/4272017835 >> 172.21.15.1:0/4272017835 conn(0x7f05740fc050 msgr2=0x7f0574107640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:46.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.218+0000 7f0579a00700 1 -- 172.21.15.1:0/4272017835 shutdown_connections 2024-02-02T15:26:46.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.218+0000 7f0579a00700 1 -- 172.21.15.1:0/4272017835 wait complete. 2024-02-02T15:26:46.220 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-02-02T15:26:46.235 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":298,"stamp":"2024-02-02T15:26:45.935234+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658334,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1000,"num_read_kb":624,"num_write":431,"num_write_kb":3084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1180804,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":855,"ondisk_log_size":855,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":336228,"kb_used_data":11972,"kb_used_omap":20,"kb_used_meta":324075,"kb_avail":1124605596,"statfs":{"total":1151940427776,"available":1151596130304,"internally_reserved":0,"allocated":12259328,"data_stored":7915039,"data_compressed":392601,"data_compressed_allocated":528384,"data_compressed_original":2555904,"omap_allocated":21372,"internal_metadata":331852932},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":312,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":111,"num_read_kb":51,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002994"},"pg_stats":[{"pgid":"3.3f","version":"86'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.379934+0000","last_change":"2024-02-02T15:24:47.793308+0000","last_active":"2024-02-02T15:26:27.379934+0000","last_peered":"2024-02-02T15:26:27.379934+0000","last_clean":"2024-02-02T15:26:27.379934+0000","last_became_active":"2024-02-02T15:24:47.792420+0000","last_became_peered":"2024-02-02T15:24:47.792420+0000","last_unstale":"2024-02-02T15:26:27.379934+0000","last_undegraded":"2024-02-02T15:26:27.379934+0000","last_fullsized":"2024-02-02T15:26:27.379934+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:39:09.871317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"86'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.486648+0000","last_change":"2024-02-02T15:24:47.791272+0000","last_active":"2024-02-02T15:26:27.486648+0000","last_peered":"2024-02-02T15:26:27.486648+0000","last_clean":"2024-02-02T15:26:27.486648+0000","last_became_active":"2024-02-02T15:24:47.790902+0000","last_became_peered":"2024-02-02T15:24:47.790902+0000","last_unstale":"2024-02-02T15:26:27.486648+0000","last_undegraded":"2024-02-02T15:26:27.486648+0000","last_fullsized":"2024-02-02T15:26:27.486648+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:31:11.620873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"86'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.486605+0000","last_change":"2024-02-02T15:24:47.798712+0000","last_active":"2024-02-02T15:26:27.486605+0000","last_peered":"2024-02-02T15:26:27.486605+0000","last_clean":"2024-02-02T15:26:27.486605+0000","last_became_active":"2024-02-02T15:24:47.798405+0000","last_became_peered":"2024-02-02T15:24:47.798405+0000","last_unstale":"2024-02-02T15:26:27.486605+0000","last_undegraded":"2024-02-02T15:26:27.486605+0000","last_fullsized":"2024-02-02T15:26:27.486605+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:37:15.238070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"86'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.280412+0000","last_change":"2024-02-02T15:24:47.786456+0000","last_active":"2024-02-02T15:26:27.280412+0000","last_peered":"2024-02-02T15:26:27.280412+0000","last_clean":"2024-02-02T15:26:27.280412+0000","last_became_active":"2024-02-02T15:24:47.785520+0000","last_became_peered":"2024-02-02T15:24:47.785520+0000","last_unstale":"2024-02-02T15:26:27.280412+0000","last_undegraded":"2024-02-02T15:26:27.280412+0000","last_fullsized":"2024-02-02T15:26:27.280412+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:16:04.003459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"86'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.274132+0000","last_change":"2024-02-02T15:24:47.780084+0000","last_active":"2024-02-02T15:26:27.274132+0000","last_peered":"2024-02-02T15:26:27.274132+0000","last_clean":"2024-02-02T15:26:27.274132+0000","last_became_active":"2024-02-02T15:24:47.779667+0000","last_became_peered":"2024-02-02T15:24:47.779667+0000","last_unstale":"2024-02-02T15:26:27.274132+0000","last_undegraded":"2024-02-02T15:26:27.274132+0000","last_fullsized":"2024-02-02T15:26:27.274132+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:30:15.377691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"86'6","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.362466+0000","last_change":"2024-02-02T15:24:47.786173+0000","last_active":"2024-02-02T15:26:27.362466+0000","last_peered":"2024-02-02T15:26:27.362466+0000","last_clean":"2024-02-02T15:26:27.362466+0000","last_became_active":"2024-02-02T15:24:47.785301+0000","last_became_peered":"2024-02-02T15:24:47.785301+0000","last_unstale":"2024-02-02T15:26:27.362466+0000","last_undegraded":"2024-02-02T15:26:27.362466+0000","last_fullsized":"2024-02-02T15:26:27.362466+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:14:58.524317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"86'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.273751+0000","last_change":"2024-02-02T15:24:47.790913+0000","last_active":"2024-02-02T15:26:27.273751+0000","last_peered":"2024-02-02T15:26:27.273751+0000","last_clean":"2024-02-02T15:26:27.273751+0000","last_became_active":"2024-02-02T15:24:47.790566+0000","last_became_peered":"2024-02-02T15:24:47.790566+0000","last_unstale":"2024-02-02T15:26:27.273751+0000","last_undegraded":"2024-02-02T15:26:27.273751+0000","last_fullsized":"2024-02-02T15:26:27.273751+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:34:51.954830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"86'9","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.375616+0000","last_change":"2024-02-02T15:24:47.786465+0000","last_active":"2024-02-02T15:26:27.375616+0000","last_peered":"2024-02-02T15:26:27.375616+0000","last_clean":"2024-02-02T15:26:27.375616+0000","last_became_active":"2024-02-02T15:24:47.786175+0000","last_became_peered":"2024-02-02T15:24:47.786175+0000","last_unstale":"2024-02-02T15:26:27.375616+0000","last_undegraded":"2024-02-02T15:26:27.375616+0000","last_fullsized":"2024-02-02T15:26:27.375616+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:42:13.748850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"86'11","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.277236+0000","last_change":"2024-02-02T15:24:47.787115+0000","last_active":"2024-02-02T15:26:27.277236+0000","last_peered":"2024-02-02T15:26:27.277236+0000","last_clean":"2024-02-02T15:26:27.277236+0000","last_became_active":"2024-02-02T15:24:47.786804+0000","last_became_peered":"2024-02-02T15:24:47.786804+0000","last_unstale":"2024-02-02T15:26:27.277236+0000","last_undegraded":"2024-02-02T15:26:27.277236+0000","last_fullsized":"2024-02-02T15:26:27.277236+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T03:06:17.873243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"86'16","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.362900+0000","last_change":"2024-02-02T15:24:47.787211+0000","last_active":"2024-02-02T15:26:27.362900+0000","last_peered":"2024-02-02T15:26:27.362900+0000","last_clean":"2024-02-02T15:26:27.362900+0000","last_became_active":"2024-02-02T15:24:47.786888+0000","last_became_peered":"2024-02-02T15:24:47.786888+0000","last_unstale":"2024-02-02T15:26:27.362900+0000","last_undegraded":"2024-02-02T15:26:27.362900+0000","last_fullsized":"2024-02-02T15:26:27.362900+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:14:21.442787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"86'6","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.274294+0000","last_change":"2024-02-02T15:24:47.798962+0000","last_active":"2024-02-02T15:26:27.274294+0000","last_peered":"2024-02-02T15:26:27.274294+0000","last_clean":"2024-02-02T15:26:27.274294+0000","last_became_active":"2024-02-02T15:24:47.798657+0000","last_became_peered":"2024-02-02T15:24:47.798657+0000","last_unstale":"2024-02-02T15:26:27.274294+0000","last_undegraded":"2024-02-02T15:26:27.274294+0000","last_fullsized":"2024-02-02T15:26:27.274294+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:53:37.316027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"86'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.280289+0000","last_change":"2024-02-02T15:24:47.782647+0000","last_active":"2024-02-02T15:26:27.280289+0000","last_peered":"2024-02-02T15:26:27.280289+0000","last_clean":"2024-02-02T15:26:27.280289+0000","last_became_active":"2024-02-02T15:24:47.782344+0000","last_became_peered":"2024-02-02T15:24:47.782344+0000","last_unstale":"2024-02-02T15:26:27.280289+0000","last_undegraded":"2024-02-02T15:26:27.280289+0000","last_fullsized":"2024-02-02T15:26:27.280289+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:06:54.874657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"86'10","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.269115+0000","last_change":"2024-02-02T15:24:47.787032+0000","last_active":"2024-02-02T15:26:27.269115+0000","last_peered":"2024-02-02T15:26:27.269115+0000","last_clean":"2024-02-02T15:26:27.269115+0000","last_became_active":"2024-02-02T15:24:47.786710+0000","last_became_peered":"2024-02-02T15:24:47.786710+0000","last_unstale":"2024-02-02T15:26:27.269115+0000","last_undegraded":"2024-02-02T15:26:27.269115+0000","last_fullsized":"2024-02-02T15:26:27.269115+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:26:15.429316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"86'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.486412+0000","last_change":"2024-02-02T15:24:47.785708+0000","last_active":"2024-02-02T15:26:27.486412+0000","last_peered":"2024-02-02T15:26:27.486412+0000","last_clean":"2024-02-02T15:26:27.486412+0000","last_became_active":"2024-02-02T15:24:47.785280+0000","last_became_peered":"2024-02-02T15:24:47.785280+0000","last_unstale":"2024-02-02T15:26:27.486412+0000","last_undegraded":"2024-02-02T15:26:27.486412+0000","last_fullsized":"2024-02-02T15:26:27.486412+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T02:21:23.224933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"86'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.486347+0000","last_change":"2024-02-02T15:24:47.783904+0000","last_active":"2024-02-02T15:26:27.486347+0000","last_peered":"2024-02-02T15:26:27.486347+0000","last_clean":"2024-02-02T15:26:27.486347+0000","last_became_active":"2024-02-02T15:24:47.783599+0000","last_became_peered":"2024-02-02T15:24:47.783599+0000","last_unstale":"2024-02-02T15:26:27.486347+0000","last_undegraded":"2024-02-02T15:26:27.486347+0000","last_fullsized":"2024-02-02T15:26:27.486347+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:35:45.805527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"86'9","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.276951+0000","last_change":"2024-02-02T15:24:47.795770+0000","last_active":"2024-02-02T15:26:27.276951+0000","last_peered":"2024-02-02T15:26:27.276951+0000","last_clean":"2024-02-02T15:26:27.276951+0000","last_became_active":"2024-02-02T15:24:47.795433+0000","last_became_peered":"2024-02-02T15:24:47.795433+0000","last_unstale":"2024-02-02T15:26:27.276951+0000","last_undegraded":"2024-02-02T15:26:27.276951+0000","last_fullsized":"2024-02-02T15:26:27.276951+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:50:07.496998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"86'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.379835+0000","last_change":"2024-02-02T15:24:47.793684+0000","last_active":"2024-02-02T15:26:27.379835+0000","last_peered":"2024-02-02T15:26:27.379835+0000","last_clean":"2024-02-02T15:26:27.379835+0000","last_became_active":"2024-02-02T15:24:47.792853+0000","last_became_peered":"2024-02-02T15:24:47.792853+0000","last_unstale":"2024-02-02T15:26:27.379835+0000","last_undegraded":"2024-02-02T15:26:27.379835+0000","last_fullsized":"2024-02-02T15:26:27.379835+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:30:40.379379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"86'10","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.280363+0000","last_change":"2024-02-02T15:24:47.780935+0000","last_active":"2024-02-02T15:26:27.280363+0000","last_peered":"2024-02-02T15:26:27.280363+0000","last_clean":"2024-02-02T15:26:27.280363+0000","last_became_active":"2024-02-02T15:24:47.780642+0000","last_became_peered":"2024-02-02T15:24:47.780642+0000","last_unstale":"2024-02-02T15:26:27.280363+0000","last_undegraded":"2024-02-02T15:26:27.280363+0000","last_fullsized":"2024-02-02T15:26:27.280363+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:36:37.615368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"86'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.380235+0000","last_change":"2024-02-02T15:24:47.791051+0000","last_active":"2024-02-02T15:26:27.380235+0000","last_peered":"2024-02-02T15:26:27.380235+0000","last_clean":"2024-02-02T15:26:27.380235+0000","last_became_active":"2024-02-02T15:24:47.790732+0000","last_became_peered":"2024-02-02T15:24:47.790732+0000","last_unstale":"2024-02-02T15:26:27.380235+0000","last_undegraded":"2024-02-02T15:26:27.380235+0000","last_fullsized":"2024-02-02T15:26:27.380235+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:45:05.155144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.380085+0000","last_change":"2024-02-02T15:24:51.814699+0000","last_active":"2024-02-02T15:26:27.380085+0000","last_peered":"2024-02-02T15:26:27.380085+0000","last_clean":"2024-02-02T15:26:27.380085+0000","last_became_active":"2024-02-02T15:24:51.814251+0000","last_became_peered":"2024-02-02T15:24:51.814251+0000","last_unstale":"2024-02-02T15:26:27.380085+0000","last_undegraded":"2024-02-02T15:26:27.380085+0000","last_fullsized":"2024-02-02T15:26:27.380085+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:27:21.674922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"84'13","reported_seq":118,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-02T15:26:42.393747+0000","last_change":"2024-02-02T15:24:47.269429+0000","last_active":"2024-02-02T15:26:42.393747+0000","last_peered":"2024-02-02T15:26:42.393747+0000","last_clean":"2024-02-02T15:26:42.393747+0000","last_became_active":"2024-02-02T15:24:47.269023+0000","last_became_peered":"2024-02-02T15:24:47.269023+0000","last_unstale":"2024-02-02T15:26:42.393747+0000","last_undegraded":"2024-02-02T15:26:42.393747+0000","last_fullsized":"2024-02-02T15:26:42.393747+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T03:11:46.344438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":85,"num_read_kb":85,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"86'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.376226+0000","last_change":"2024-02-02T15:24:47.785900+0000","last_active":"2024-02-02T15:26:27.376226+0000","last_peered":"2024-02-02T15:26:27.376226+0000","last_clean":"2024-02-02T15:26:27.376226+0000","last_became_active":"2024-02-02T15:24:47.781994+0000","last_became_peered":"2024-02-02T15:24:47.781994+0000","last_unstale":"2024-02-02T15:26:27.376226+0000","last_undegraded":"2024-02-02T15:26:27.376226+0000","last_fullsized":"2024-02-02T15:26:27.376226+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:00:52.432637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.279297+0000","last_change":"2024-02-02T15:24:51.825655+0000","last_active":"2024-02-02T15:26:27.279297+0000","last_peered":"2024-02-02T15:26:27.279297+0000","last_clean":"2024-02-02T15:26:27.279297+0000","last_became_active":"2024-02-02T15:24:51.825199+0000","last_became_peered":"2024-02-02T15:24:51.825199+0000","last_unstale":"2024-02-02T15:26:27.279297+0000","last_undegraded":"2024-02-02T15:26:27.279297+0000","last_fullsized":"2024-02-02T15:26:27.279297+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:50:32.283394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"84'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.376399+0000","last_change":"2024-02-02T15:24:47.281869+0000","last_active":"2024-02-02T15:26:27.376399+0000","last_peered":"2024-02-02T15:26:27.376399+0000","last_clean":"2024-02-02T15:26:27.376399+0000","last_became_active":"2024-02-02T15:24:47.280825+0000","last_became_peered":"2024-02-02T15:24:47.280825+0000","last_unstale":"2024-02-02T15:26:27.376399+0000","last_undegraded":"2024-02-02T15:26:27.376399+0000","last_fullsized":"2024-02-02T15:26:27.376399+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:24:19.688389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"86'6","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.486649+0000","last_change":"2024-02-02T15:24:47.795893+0000","last_active":"2024-02-02T15:26:27.486649+0000","last_peered":"2024-02-02T15:26:27.486649+0000","last_clean":"2024-02-02T15:26:27.486649+0000","last_became_active":"2024-02-02T15:24:47.795491+0000","last_became_peered":"2024-02-02T15:24:47.795491+0000","last_unstale":"2024-02-02T15:26:27.486649+0000","last_undegraded":"2024-02-02T15:26:27.486649+0000","last_fullsized":"2024-02-02T15:26:27.486649+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:54:38.923036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.279462+0000","last_change":"2024-02-02T15:24:51.826980+0000","last_active":"2024-02-02T15:26:27.279462+0000","last_peered":"2024-02-02T15:26:27.279462+0000","last_clean":"2024-02-02T15:26:27.279462+0000","last_became_active":"2024-02-02T15:24:51.826221+0000","last_became_peered":"2024-02-02T15:24:51.826221+0000","last_unstale":"2024-02-02T15:26:27.279462+0000","last_undegraded":"2024-02-02T15:26:27.279462+0000","last_fullsized":"2024-02-02T15:26:27.279462+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:45:17.126174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"84'12","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.274623+0000","last_change":"2024-02-02T15:24:47.272219+0000","last_active":"2024-02-02T15:26:27.274623+0000","last_peered":"2024-02-02T15:26:27.274623+0000","last_clean":"2024-02-02T15:26:27.274623+0000","last_became_active":"2024-02-02T15:24:47.271899+0000","last_became_peered":"2024-02-02T15:24:47.271899+0000","last_unstale":"2024-02-02T15:26:27.274623+0000","last_undegraded":"2024-02-02T15:26:27.274623+0000","last_fullsized":"2024-02-02T15:26:27.274623+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:44:27.370783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"86'9","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.277362+0000","last_change":"2024-02-02T15:24:47.783434+0000","last_active":"2024-02-02T15:26:27.277362+0000","last_peered":"2024-02-02T15:26:27.277362+0000","last_clean":"2024-02-02T15:26:27.277362+0000","last_became_active":"2024-02-02T15:24:47.782130+0000","last_became_peered":"2024-02-02T15:24:47.782130+0000","last_unstale":"2024-02-02T15:26:27.277362+0000","last_undegraded":"2024-02-02T15:26:27.277362+0000","last_fullsized":"2024-02-02T15:26:27.277362+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:32:25.981913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.279554+0000","last_change":"2024-02-02T15:24:51.810791+0000","last_active":"2024-02-02T15:26:27.279554+0000","last_peered":"2024-02-02T15:26:27.279554+0000","last_clean":"2024-02-02T15:26:27.279554+0000","last_became_active":"2024-02-02T15:24:51.810379+0000","last_became_peered":"2024-02-02T15:24:51.810379+0000","last_unstale":"2024-02-02T15:26:27.279554+0000","last_undegraded":"2024-02-02T15:26:27.279554+0000","last_fullsized":"2024-02-02T15:26:27.279554+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:49:44.635820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'5","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.274488+0000","last_change":"2024-02-02T15:24:47.271225+0000","last_active":"2024-02-02T15:26:27.274488+0000","last_peered":"2024-02-02T15:26:27.274488+0000","last_clean":"2024-02-02T15:26:27.274488+0000","last_became_active":"2024-02-02T15:24:47.270870+0000","last_became_peered":"2024-02-02T15:24:47.270870+0000","last_unstale":"2024-02-02T15:26:27.274488+0000","last_undegraded":"2024-02-02T15:26:27.274488+0000","last_fullsized":"2024-02-02T15:26:27.274488+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:16:55.659950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"86'15","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-02T15:26:28.281479+0000","last_change":"2024-02-02T15:26:28.281073+0000","last_active":"2024-02-02T15:26:28.281479+0000","last_peered":"2024-02-02T15:26:28.281479+0000","last_clean":"2024-02-02T15:26:28.281479+0000","last_became_active":"2024-02-02T15:26:28.280680+0000","last_became_peered":"2024-02-02T15:26:28.280680+0000","last_unstale":"2024-02-02T15:26:28.281479+0000","last_undegraded":"2024-02-02T15:26:28.281479+0000","last_fullsized":"2024-02-02T15:26:28.281479+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:53:34.526394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.379201+0000","last_change":"2024-02-02T15:24:51.807328+0000","last_active":"2024-02-02T15:26:27.379201+0000","last_peered":"2024-02-02T15:26:27.379201+0000","last_clean":"2024-02-02T15:26:27.379201+0000","last_became_active":"2024-02-02T15:24:51.806853+0000","last_became_peered":"2024-02-02T15:24:51.806853+0000","last_unstale":"2024-02-02T15:26:27.379201+0000","last_undegraded":"2024-02-02T15:26:27.379201+0000","last_fullsized":"2024-02-02T15:26:27.379201+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:46:13.898450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"84'5","reported_seq":113,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-02T15:26:44.392961+0000","last_change":"2024-02-02T15:24:47.268192+0000","last_active":"2024-02-02T15:26:44.392961+0000","last_peered":"2024-02-02T15:26:44.392961+0000","last_clean":"2024-02-02T15:26:44.392961+0000","last_became_active":"2024-02-02T15:24:47.267796+0000","last_became_peered":"2024-02-02T15:24:47.267796+0000","last_unstale":"2024-02-02T15:26:44.392961+0000","last_undegraded":"2024-02-02T15:26:44.392961+0000","last_fullsized":"2024-02-02T15:26:44.392961+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:12:13.161690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":88,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"86'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.274477+0000","last_change":"2024-02-02T15:24:47.782976+0000","last_active":"2024-02-02T15:26:27.274477+0000","last_peered":"2024-02-02T15:26:27.274477+0000","last_clean":"2024-02-02T15:26:27.274477+0000","last_became_active":"2024-02-02T15:24:47.782640+0000","last_became_peered":"2024-02-02T15:24:47.782640+0000","last_unstale":"2024-02-02T15:26:27.274477+0000","last_undegraded":"2024-02-02T15:26:27.274477+0000","last_fullsized":"2024-02-02T15:26:27.274477+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T02:09:54.851418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.379834+0000","last_change":"2024-02-02T15:24:51.814799+0000","last_active":"2024-02-02T15:26:27.379834+0000","last_peered":"2024-02-02T15:26:27.379834+0000","last_clean":"2024-02-02T15:26:27.379834+0000","last_became_active":"2024-02-02T15:24:51.814263+0000","last_became_peered":"2024-02-02T15:24:51.814263+0000","last_unstale":"2024-02-02T15:26:27.379834+0000","last_undegraded":"2024-02-02T15:26:27.379834+0000","last_fullsized":"2024-02-02T15:26:27.379834+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:52:02.786496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"84'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.362424+0000","last_change":"2024-02-02T15:24:47.273204+0000","last_active":"2024-02-02T15:26:27.362424+0000","last_peered":"2024-02-02T15:26:27.362424+0000","last_clean":"2024-02-02T15:26:27.362424+0000","last_became_active":"2024-02-02T15:24:47.272888+0000","last_became_peered":"2024-02-02T15:24:47.272888+0000","last_unstale":"2024-02-02T15:26:27.362424+0000","last_undegraded":"2024-02-02T15:26:27.362424+0000","last_fullsized":"2024-02-02T15:26:27.362424+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:26:25.188915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"86'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.281811+0000","last_change":"2024-02-02T15:24:47.788645+0000","last_active":"2024-02-02T15:26:27.281811+0000","last_peered":"2024-02-02T15:26:27.281811+0000","last_clean":"2024-02-02T15:26:27.281811+0000","last_became_active":"2024-02-02T15:24:47.787478+0000","last_became_peered":"2024-02-02T15:24:47.787478+0000","last_unstale":"2024-02-02T15:26:27.281811+0000","last_undegraded":"2024-02-02T15:26:27.281811+0000","last_fullsized":"2024-02-02T15:26:27.281811+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:24:45.327030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.379732+0000","last_change":"2024-02-02T15:24:51.822681+0000","last_active":"2024-02-02T15:26:27.379732+0000","last_peered":"2024-02-02T15:26:27.379732+0000","last_clean":"2024-02-02T15:26:27.379732+0000","last_became_active":"2024-02-02T15:24:51.822202+0000","last_became_peered":"2024-02-02T15:24:51.822202+0000","last_unstale":"2024-02-02T15:26:27.379732+0000","last_undegraded":"2024-02-02T15:26:27.379732+0000","last_fullsized":"2024-02-02T15:26:27.379732+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:05:43.751558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"84'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.268759+0000","last_change":"2024-02-02T15:24:47.271095+0000","last_active":"2024-02-02T15:26:27.268759+0000","last_peered":"2024-02-02T15:26:27.268759+0000","last_clean":"2024-02-02T15:26:27.268759+0000","last_became_active":"2024-02-02T15:24:47.270696+0000","last_became_peered":"2024-02-02T15:24:47.270696+0000","last_unstale":"2024-02-02T15:26:27.268759+0000","last_undegraded":"2024-02-02T15:26:27.268759+0000","last_fullsized":"2024-02-02T15:26:27.268759+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T02:20:59.370217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"86'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.375968+0000","last_change":"2024-02-02T15:24:47.791699+0000","last_active":"2024-02-02T15:26:27.375968+0000","last_peered":"2024-02-02T15:26:27.375968+0000","last_clean":"2024-02-02T15:26:27.375968+0000","last_became_active":"2024-02-02T15:24:47.787280+0000","last_became_peered":"2024-02-02T15:24:47.787280+0000","last_unstale":"2024-02-02T15:26:27.375968+0000","last_undegraded":"2024-02-02T15:26:27.375968+0000","last_fullsized":"2024-02-02T15:26:27.375968+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:18:10.316393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.486617+0000","last_change":"2024-02-02T15:24:51.810050+0000","last_active":"2024-02-02T15:26:27.486617+0000","last_peered":"2024-02-02T15:26:27.486617+0000","last_clean":"2024-02-02T15:26:27.486617+0000","last_became_active":"2024-02-02T15:24:51.809621+0000","last_became_peered":"2024-02-02T15:24:51.809621+0000","last_unstale":"2024-02-02T15:26:27.486617+0000","last_undegraded":"2024-02-02T15:26:27.486617+0000","last_fullsized":"2024-02-02T15:26:27.486617+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:52:35.885973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"84'3","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.281444+0000","last_change":"2024-02-02T15:24:47.267973+0000","last_active":"2024-02-02T15:26:27.281444+0000","last_peered":"2024-02-02T15:26:27.281444+0000","last_clean":"2024-02-02T15:26:27.281444+0000","last_became_active":"2024-02-02T15:24:47.267649+0000","last_became_peered":"2024-02-02T15:24:47.267649+0000","last_unstale":"2024-02-02T15:26:27.281444+0000","last_undegraded":"2024-02-02T15:26:27.281444+0000","last_fullsized":"2024-02-02T15:26:27.281444+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T02:46:56.844570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"86'11","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.279736+0000","last_change":"2024-02-02T15:24:47.798636+0000","last_active":"2024-02-02T15:26:27.279736+0000","last_peered":"2024-02-02T15:26:27.279736+0000","last_clean":"2024-02-02T15:26:27.279736+0000","last_became_active":"2024-02-02T15:24:47.798328+0000","last_became_peered":"2024-02-02T15:24:47.798328+0000","last_unstale":"2024-02-02T15:26:27.279736+0000","last_undegraded":"2024-02-02T15:26:27.279736+0000","last_fullsized":"2024-02-02T15:26:27.279736+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:37:43.687336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.375711+0000","last_change":"2024-02-02T15:24:51.811312+0000","last_active":"2024-02-02T15:26:27.375711+0000","last_peered":"2024-02-02T15:26:27.375711+0000","last_clean":"2024-02-02T15:26:27.375711+0000","last_became_active":"2024-02-02T15:24:51.810932+0000","last_became_peered":"2024-02-02T15:24:51.810932+0000","last_unstale":"2024-02-02T15:26:27.375711+0000","last_undegraded":"2024-02-02T15:26:27.375711+0000","last_fullsized":"2024-02-02T15:26:27.375711+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:10:31.530518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"84'4","reported_seq":186,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-02T15:26:40.386290+0000","last_change":"2024-02-02T15:24:47.271483+0000","last_active":"2024-02-02T15:26:40.386290+0000","last_peered":"2024-02-02T15:26:40.386290+0000","last_clean":"2024-02-02T15:26:40.386290+0000","last_became_active":"2024-02-02T15:24:47.271160+0000","last_became_peered":"2024-02-02T15:24:47.271160+0000","last_unstale":"2024-02-02T15:26:40.386290+0000","last_undegraded":"2024-02-02T15:26:40.386290+0000","last_fullsized":"2024-02-02T15:26:40.386290+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:29:20.135351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":162,"num_read_kb":162,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"86'10","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.277538+0000","last_change":"2024-02-02T15:24:47.786492+0000","last_active":"2024-02-02T15:26:27.277538+0000","last_peered":"2024-02-02T15:26:27.277538+0000","last_clean":"2024-02-02T15:26:27.277538+0000","last_became_active":"2024-02-02T15:24:47.786103+0000","last_became_peered":"2024-02-02T15:24:47.786103+0000","last_unstale":"2024-02-02T15:26:27.277538+0000","last_undegraded":"2024-02-02T15:26:27.277538+0000","last_fullsized":"2024-02-02T15:26:27.277538+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:24:23.666596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.279580+0000","last_change":"2024-02-02T15:24:51.819941+0000","last_active":"2024-02-02T15:26:27.279580+0000","last_peered":"2024-02-02T15:26:27.279580+0000","last_clean":"2024-02-02T15:26:27.279580+0000","last_became_active":"2024-02-02T15:24:51.818066+0000","last_became_peered":"2024-02-02T15:24:51.818066+0000","last_unstale":"2024-02-02T15:26:27.279580+0000","last_undegraded":"2024-02-02T15:26:27.279580+0000","last_fullsized":"2024-02-02T15:26:27.279580+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:55:51.550988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"84'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.274167+0000","last_change":"2024-02-02T15:24:47.268541+0000","last_active":"2024-02-02T15:26:27.274167+0000","last_peered":"2024-02-02T15:26:27.274167+0000","last_clean":"2024-02-02T15:26:27.274167+0000","last_became_active":"2024-02-02T15:24:47.268184+0000","last_became_peered":"2024-02-02T15:24:47.268184+0000","last_unstale":"2024-02-02T15:26:27.274167+0000","last_undegraded":"2024-02-02T15:26:27.274167+0000","last_fullsized":"2024-02-02T15:26:27.274167+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:30:12.187297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"86'9","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.486650+0000","last_change":"2024-02-02T15:24:47.787792+0000","last_active":"2024-02-02T15:26:27.486650+0000","last_peered":"2024-02-02T15:26:27.486650+0000","last_clean":"2024-02-02T15:26:27.486650+0000","last_became_active":"2024-02-02T15:24:47.787479+0000","last_became_peered":"2024-02-02T15:24:47.787479+0000","last_unstale":"2024-02-02T15:26:27.486650+0000","last_undegraded":"2024-02-02T15:26:27.486650+0000","last_fullsized":"2024-02-02T15:26:27.486650+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T03:24:06.544232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.279743+0000","last_change":"2024-02-02T15:24:51.826639+0000","last_active":"2024-02-02T15:26:27.279743+0000","last_peered":"2024-02-02T15:26:27.279743+0000","last_clean":"2024-02-02T15:26:27.279743+0000","last_became_active":"2024-02-02T15:24:51.825089+0000","last_became_peered":"2024-02-02T15:24:51.825089+0000","last_unstale":"2024-02-02T15:26:27.279743+0000","last_undegraded":"2024-02-02T15:26:27.279743+0000","last_fullsized":"2024-02-02T15:26:27.279743+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:41:44.592603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"84'5","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.281688+0000","last_change":"2024-02-02T15:24:47.281490+0000","last_active":"2024-02-02T15:26:27.281688+0000","last_peered":"2024-02-02T15:26:27.281688+0000","last_clean":"2024-02-02T15:26:27.281688+0000","last_became_active":"2024-02-02T15:24:47.281137+0000","last_became_peered":"2024-02-02T15:24:47.281137+0000","last_unstale":"2024-02-02T15:26:27.281688+0000","last_undegraded":"2024-02-02T15:26:27.281688+0000","last_fullsized":"2024-02-02T15:26:27.281688+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:42:08.508881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"86'11","reported_seq":39,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-02T15:26:28.376134+0000","last_change":"2024-02-02T15:26:28.376134+0000","last_active":"2024-02-02T15:26:28.376134+0000","last_peered":"2024-02-02T15:26:28.376134+0000","last_clean":"2024-02-02T15:26:28.376134+0000","last_became_active":"2024-02-02T15:26:28.375708+0000","last_became_peered":"2024-02-02T15:26:28.375708+0000","last_unstale":"2024-02-02T15:26:28.376134+0000","last_undegraded":"2024-02-02T15:26:28.376134+0000","last_fullsized":"2024-02-02T15:26:28.376134+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:03:10.254243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,7],"acting":[1,8,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.279858+0000","last_change":"2024-02-02T15:25:27.286403+0000","last_active":"2024-02-02T15:26:27.279858+0000","last_peered":"2024-02-02T15:26:27.279858+0000","last_clean":"2024-02-02T15:26:27.279858+0000","last_became_active":"2024-02-02T15:25:27.285824+0000","last_became_peered":"2024-02-02T15:25:27.285824+0000","last_unstale":"2024-02-02T15:26:27.279858+0000","last_undegraded":"2024-02-02T15:26:27.279858+0000","last_fullsized":"2024-02-02T15:26:27.279858+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:11:25.986947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,10],"acting":[9,8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"88'39","reported_seq":231,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-02T15:26:40.386763+0000","last_change":"2024-02-02T15:24:47.270604+0000","last_active":"2024-02-02T15:26:40.386763+0000","last_peered":"2024-02-02T15:26:40.386763+0000","last_clean":"2024-02-02T15:26:40.386763+0000","last_became_active":"2024-02-02T15:24:47.270261+0000","last_became_peered":"2024-02-02T15:24:47.270261+0000","last_unstale":"2024-02-02T15:26:40.386763+0000","last_undegraded":"2024-02-02T15:26:40.386763+0000","last_fullsized":"2024-02-02T15:26:40.386763+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:56:33.767423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60957,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":172,"num_read_kb":147,"num_write":41,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"86'6","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.274315+0000","last_change":"2024-02-02T15:24:47.789663+0000","last_active":"2024-02-02T15:26:27.274315+0000","last_peered":"2024-02-02T15:26:27.274315+0000","last_clean":"2024-02-02T15:26:27.274315+0000","last_became_active":"2024-02-02T15:24:47.789352+0000","last_became_peered":"2024-02-02T15:24:47.789352+0000","last_unstale":"2024-02-02T15:26:27.274315+0000","last_undegraded":"2024-02-02T15:26:27.274315+0000","last_fullsized":"2024-02-02T15:26:27.274315+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:13:37.991350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.282133+0000","last_change":"2024-02-02T15:24:51.816021+0000","last_active":"2024-02-02T15:26:27.282133+0000","last_peered":"2024-02-02T15:26:27.282133+0000","last_clean":"2024-02-02T15:26:27.282133+0000","last_became_active":"2024-02-02T15:24:51.815698+0000","last_became_peered":"2024-02-02T15:24:51.815698+0000","last_unstale":"2024-02-02T15:26:27.282133+0000","last_undegraded":"2024-02-02T15:26:27.282133+0000","last_fullsized":"2024-02-02T15:26:27.282133+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:11:10.670195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"84'9","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.379371+0000","last_change":"2024-02-02T15:25:27.295339+0000","last_active":"2024-02-02T15:26:27.379371+0000","last_peered":"2024-02-02T15:26:27.379371+0000","last_clean":"2024-02-02T15:26:27.379371+0000","last_became_active":"2024-02-02T15:25:27.281605+0000","last_became_peered":"2024-02-02T15:25:27.281605+0000","last_unstale":"2024-02-02T15:26:27.379371+0000","last_undegraded":"2024-02-02T15:26:27.379371+0000","last_fullsized":"2024-02-02T15:26:27.379371+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T02:35:36.952261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"86'9","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.268962+0000","last_change":"2024-02-02T15:24:47.798089+0000","last_active":"2024-02-02T15:26:27.268962+0000","last_peered":"2024-02-02T15:26:27.268962+0000","last_clean":"2024-02-02T15:26:27.268962+0000","last_became_active":"2024-02-02T15:24:47.793914+0000","last_became_peered":"2024-02-02T15:24:47.793914+0000","last_unstale":"2024-02-02T15:26:27.268962+0000","last_undegraded":"2024-02-02T15:26:27.268962+0000","last_fullsized":"2024-02-02T15:26:27.268962+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:33:39.927550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.379433+0000","last_change":"2024-02-02T15:24:51.823098+0000","last_active":"2024-02-02T15:26:27.379433+0000","last_peered":"2024-02-02T15:26:27.379433+0000","last_clean":"2024-02-02T15:26:27.379433+0000","last_became_active":"2024-02-02T15:24:51.822634+0000","last_became_peered":"2024-02-02T15:24:51.822634+0000","last_unstale":"2024-02-02T15:26:27.379433+0000","last_undegraded":"2024-02-02T15:26:27.379433+0000","last_fullsized":"2024-02-02T15:26:27.379433+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T03:18:07.420196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"84'5","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.281450+0000","last_change":"2024-02-02T15:24:47.268962+0000","last_active":"2024-02-02T15:26:27.281450+0000","last_peered":"2024-02-02T15:26:27.281450+0000","last_clean":"2024-02-02T15:26:27.281450+0000","last_became_active":"2024-02-02T15:24:47.268607+0000","last_became_peered":"2024-02-02T15:24:47.268607+0000","last_unstale":"2024-02-02T15:26:27.281450+0000","last_undegraded":"2024-02-02T15:26:27.281450+0000","last_fullsized":"2024-02-02T15:26:27.281450+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:31:06.330437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"86'15","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.273962+0000","last_change":"2024-02-02T15:24:47.792205+0000","last_active":"2024-02-02T15:26:27.273962+0000","last_peered":"2024-02-02T15:26:27.273962+0000","last_clean":"2024-02-02T15:26:27.273962+0000","last_became_active":"2024-02-02T15:24:47.791874+0000","last_became_peered":"2024-02-02T15:24:47.791874+0000","last_unstale":"2024-02-02T15:26:27.273962+0000","last_undegraded":"2024-02-02T15:26:27.273962+0000","last_fullsized":"2024-02-02T15:26:27.273962+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:33:38.795921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.280422+0000","last_change":"2024-02-02T15:24:51.825499+0000","last_active":"2024-02-02T15:26:27.280422+0000","last_peered":"2024-02-02T15:26:27.280422+0000","last_clean":"2024-02-02T15:26:27.280422+0000","last_became_active":"2024-02-02T15:24:51.825049+0000","last_became_peered":"2024-02-02T15:24:51.825049+0000","last_unstale":"2024-02-02T15:26:27.280422+0000","last_undegraded":"2024-02-02T15:26:27.280422+0000","last_fullsized":"2024-02-02T15:26:27.280422+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:41:05.430891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"86'7","reported_seq":115,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-02T15:26:43.404328+0000","last_change":"2024-02-02T15:24:47.267683+0000","last_active":"2024-02-02T15:26:43.404328+0000","last_peered":"2024-02-02T15:26:43.404328+0000","last_clean":"2024-02-02T15:26:43.404328+0000","last_became_active":"2024-02-02T15:24:47.267324+0000","last_became_peered":"2024-02-02T15:24:47.267324+0000","last_unstale":"2024-02-02T15:26:43.404328+0000","last_undegraded":"2024-02-02T15:26:43.404328+0000","last_fullsized":"2024-02-02T15:26:43.404328+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:03:31.326791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1073,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":89,"num_read_kb":4,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"86'6","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.274103+0000","last_change":"2024-02-02T15:24:47.792509+0000","last_active":"2024-02-02T15:26:27.274103+0000","last_peered":"2024-02-02T15:26:27.274103+0000","last_clean":"2024-02-02T15:26:27.274103+0000","last_became_active":"2024-02-02T15:24:47.791662+0000","last_became_peered":"2024-02-02T15:24:47.791662+0000","last_unstale":"2024-02-02T15:26:27.274103+0000","last_undegraded":"2024-02-02T15:26:27.274103+0000","last_fullsized":"2024-02-02T15:26:27.274103+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:42:50.009971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.269435+0000","last_change":"2024-02-02T15:25:27.282514+0000","last_active":"2024-02-02T15:26:27.269435+0000","last_peered":"2024-02-02T15:26:27.269435+0000","last_clean":"2024-02-02T15:26:27.269435+0000","last_became_active":"2024-02-02T15:25:27.281336+0000","last_became_peered":"2024-02-02T15:25:27.281336+0000","last_unstale":"2024-02-02T15:26:27.269435+0000","last_undegraded":"2024-02-02T15:26:27.269435+0000","last_fullsized":"2024-02-02T15:26:27.269435+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:50:39.728813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"84'17","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.274050+0000","last_change":"2024-02-02T15:24:47.274152+0000","last_active":"2024-02-02T15:26:27.274050+0000","last_peered":"2024-02-02T15:26:27.274050+0000","last_clean":"2024-02-02T15:26:27.274050+0000","last_became_active":"2024-02-02T15:24:47.273720+0000","last_became_peered":"2024-02-02T15:24:47.273720+0000","last_unstale":"2024-02-02T15:26:27.274050+0000","last_undegraded":"2024-02-02T15:26:27.274050+0000","last_fullsized":"2024-02-02T15:26:27.274050+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:38:05.280309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"86'6","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.281387+0000","last_change":"2024-02-02T15:24:47.790681+0000","last_active":"2024-02-02T15:26:27.281387+0000","last_peered":"2024-02-02T15:26:27.281387+0000","last_clean":"2024-02-02T15:26:27.281387+0000","last_became_active":"2024-02-02T15:24:47.789952+0000","last_became_peered":"2024-02-02T15:24:47.789952+0000","last_unstale":"2024-02-02T15:26:27.281387+0000","last_undegraded":"2024-02-02T15:26:27.281387+0000","last_fullsized":"2024-02-02T15:26:27.281387+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:21:03.103398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.281438+0000","last_change":"2024-02-02T15:24:51.812676+0000","last_active":"2024-02-02T15:26:27.281438+0000","last_peered":"2024-02-02T15:26:27.281438+0000","last_clean":"2024-02-02T15:26:27.281438+0000","last_became_active":"2024-02-02T15:24:51.812212+0000","last_became_peered":"2024-02-02T15:24:51.812212+0000","last_unstale":"2024-02-02T15:26:27.281438+0000","last_undegraded":"2024-02-02T15:26:27.281438+0000","last_fullsized":"2024-02-02T15:26:27.281438+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:48:01.944379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'2","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.274629+0000","last_change":"2024-02-02T15:24:47.271707+0000","last_active":"2024-02-02T15:26:27.274629+0000","last_peered":"2024-02-02T15:26:27.274629+0000","last_clean":"2024-02-02T15:26:27.274629+0000","last_became_active":"2024-02-02T15:24:47.271379+0000","last_became_peered":"2024-02-02T15:24:47.271379+0000","last_unstale":"2024-02-02T15:26:27.274629+0000","last_undegraded":"2024-02-02T15:26:27.274629+0000","last_fullsized":"2024-02-02T15:26:27.274629+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:00:24.571385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"86'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.379740+0000","last_change":"2024-02-02T15:24:47.788360+0000","last_active":"2024-02-02T15:26:27.379740+0000","last_peered":"2024-02-02T15:26:27.379740+0000","last_clean":"2024-02-02T15:26:27.379740+0000","last_became_active":"2024-02-02T15:24:47.787945+0000","last_became_peered":"2024-02-02T15:24:47.787945+0000","last_unstale":"2024-02-02T15:26:27.379740+0000","last_undegraded":"2024-02-02T15:26:27.379740+0000","last_fullsized":"2024-02-02T15:26:27.379740+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:37:54.531487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.274398+0000","last_change":"2024-02-02T15:24:51.821320+0000","last_active":"2024-02-02T15:26:27.274398+0000","last_peered":"2024-02-02T15:26:27.274398+0000","last_clean":"2024-02-02T15:26:27.274398+0000","last_became_active":"2024-02-02T15:24:51.820853+0000","last_became_peered":"2024-02-02T15:24:51.820853+0000","last_unstale":"2024-02-02T15:26:27.274398+0000","last_undegraded":"2024-02-02T15:26:27.274398+0000","last_fullsized":"2024-02-02T15:26:27.274398+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:20:12.304517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"84'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.280230+0000","last_change":"2024-02-02T15:24:47.278291+0000","last_active":"2024-02-02T15:26:27.280230+0000","last_peered":"2024-02-02T15:26:27.280230+0000","last_clean":"2024-02-02T15:26:27.280230+0000","last_became_active":"2024-02-02T15:24:47.276163+0000","last_became_peered":"2024-02-02T15:24:47.276163+0000","last_unstale":"2024-02-02T15:26:27.280230+0000","last_undegraded":"2024-02-02T15:26:27.280230+0000","last_fullsized":"2024-02-02T15:26:27.280230+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:53:50.312627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.362700+0000","last_change":"2024-02-02T15:24:47.791290+0000","last_active":"2024-02-02T15:26:27.362700+0000","last_peered":"2024-02-02T15:26:27.362700+0000","last_clean":"2024-02-02T15:26:27.362700+0000","last_became_active":"2024-02-02T15:24:47.790987+0000","last_became_peered":"2024-02-02T15:24:47.790987+0000","last_unstale":"2024-02-02T15:26:27.362700+0000","last_undegraded":"2024-02-02T15:26:27.362700+0000","last_fullsized":"2024-02-02T15:26:27.362700+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:38:08.803338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.268999+0000","last_change":"2024-02-02T15:24:51.813050+0000","last_active":"2024-02-02T15:26:27.268999+0000","last_peered":"2024-02-02T15:26:27.268999+0000","last_clean":"2024-02-02T15:26:27.268999+0000","last_became_active":"2024-02-02T15:24:51.812634+0000","last_became_peered":"2024-02-02T15:24:51.812634+0000","last_unstale":"2024-02-02T15:26:27.268999+0000","last_undegraded":"2024-02-02T15:26:27.268999+0000","last_fullsized":"2024-02-02T15:26:27.268999+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:06:56.428431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"84'5","reported_seq":107,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-02T15:26:39.390566+0000","last_change":"2024-02-02T15:24:47.274033+0000","last_active":"2024-02-02T15:26:39.390566+0000","last_peered":"2024-02-02T15:26:39.390566+0000","last_clean":"2024-02-02T15:26:39.390566+0000","last_became_active":"2024-02-02T15:24:47.273672+0000","last_became_peered":"2024-02-02T15:24:47.273672+0000","last_unstale":"2024-02-02T15:26:39.390566+0000","last_undegraded":"2024-02-02T15:26:39.390566+0000","last_fullsized":"2024-02-02T15:26:39.390566+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:51:46.926397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":82,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"86'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.362550+0000","last_change":"2024-02-02T15:24:47.786533+0000","last_active":"2024-02-02T15:26:27.362550+0000","last_peered":"2024-02-02T15:26:27.362550+0000","last_clean":"2024-02-02T15:26:27.362550+0000","last_became_active":"2024-02-02T15:24:47.785436+0000","last_became_peered":"2024-02-02T15:24:47.785436+0000","last_unstale":"2024-02-02T15:26:27.362550+0000","last_undegraded":"2024-02-02T15:26:27.362550+0000","last_fullsized":"2024-02-02T15:26:27.362550+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:23:45.910800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.269411+0000","last_change":"2024-02-02T15:24:51.814326+0000","last_active":"2024-02-02T15:26:27.269411+0000","last_peered":"2024-02-02T15:26:27.269411+0000","last_clean":"2024-02-02T15:26:27.269411+0000","last_became_active":"2024-02-02T15:24:51.813909+0000","last_became_peered":"2024-02-02T15:24:51.813909+0000","last_unstale":"2024-02-02T15:26:27.269411+0000","last_undegraded":"2024-02-02T15:26:27.269411+0000","last_fullsized":"2024-02-02T15:26:27.269411+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:16:12.902718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"86'8","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.281562+0000","last_change":"2024-02-02T15:25:27.287818+0000","last_active":"2024-02-02T15:26:27.281562+0000","last_peered":"2024-02-02T15:26:27.281562+0000","last_clean":"2024-02-02T15:26:27.281562+0000","last_became_active":"2024-02-02T15:25:27.287481+0000","last_became_peered":"2024-02-02T15:25:27.287481+0000","last_unstale":"2024-02-02T15:26:27.281562+0000","last_undegraded":"2024-02-02T15:26:27.281562+0000","last_fullsized":"2024-02-02T15:26:27.281562+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:14:32.820969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'6","reported_seq":110,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-02T15:26:41.399825+0000","last_change":"2024-02-02T15:24:47.274437+0000","last_active":"2024-02-02T15:26:41.399825+0000","last_peered":"2024-02-02T15:26:41.399825+0000","last_clean":"2024-02-02T15:26:41.399825+0000","last_became_active":"2024-02-02T15:24:47.274010+0000","last_became_peered":"2024-02-02T15:24:47.274010+0000","last_unstale":"2024-02-02T15:26:41.399825+0000","last_undegraded":"2024-02-02T15:26:41.399825+0000","last_fullsized":"2024-02-02T15:26:41.399825+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:10:36.862190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":84,"num_read_kb":84,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"84'1","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.274175+0000","last_change":"2024-02-02T15:24:51.825698+0000","last_active":"2024-02-02T15:26:27.274175+0000","last_peered":"2024-02-02T15:26:27.274175+0000","last_clean":"2024-02-02T15:26:27.274175+0000","last_became_active":"2024-02-02T15:24:51.825295+0000","last_became_peered":"2024-02-02T15:24:51.825295+0000","last_unstale":"2024-02-02T15:26:27.274175+0000","last_undegraded":"2024-02-02T15:26:27.274175+0000","last_fullsized":"2024-02-02T15:26:27.274175+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:17:09.838124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"86'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.375702+0000","last_change":"2024-02-02T15:24:47.781836+0000","last_active":"2024-02-02T15:26:27.375702+0000","last_peered":"2024-02-02T15:26:27.375702+0000","last_clean":"2024-02-02T15:26:27.375702+0000","last_became_active":"2024-02-02T15:24:47.781472+0000","last_became_peered":"2024-02-02T15:24:47.781472+0000","last_unstale":"2024-02-02T15:26:27.375702+0000","last_undegraded":"2024-02-02T15:26:27.375702+0000","last_fullsized":"2024-02-02T15:26:27.375702+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:46:33.714191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"84'6","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.279642+0000","last_change":"2024-02-02T15:24:47.269087+0000","last_active":"2024-02-02T15:26:27.279642+0000","last_peered":"2024-02-02T15:26:27.279642+0000","last_clean":"2024-02-02T15:26:27.279642+0000","last_became_active":"2024-02-02T15:24:47.268743+0000","last_became_peered":"2024-02-02T15:24:47.268743+0000","last_unstale":"2024-02-02T15:26:27.279642+0000","last_undegraded":"2024-02-02T15:26:27.279642+0000","last_fullsized":"2024-02-02T15:26:27.279642+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:03:04.323079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":338,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.273687+0000","last_change":"2024-02-02T15:23:55.191735+0000","last_active":"2024-02-02T15:26:27.273687+0000","last_peered":"2024-02-02T15:26:27.273687+0000","last_clean":"2024-02-02T15:26:27.273687+0000","last_became_active":"2024-02-02T15:23:55.191119+0000","last_became_peered":"2024-02-02T15:23:55.191119+0000","last_unstale":"2024-02-02T15:26:27.273687+0000","last_undegraded":"2024-02-02T15:26:27.273687+0000","last_fullsized":"2024-02-02T15:26:27.273687+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:21:22.472631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:21:22.472631+0000","last_clean_scrub_stamp":"2024-02-02T15:21:22.472631+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:24:44.153461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.281781+0000","last_change":"2024-02-02T15:24:51.822842+0000","last_active":"2024-02-02T15:26:27.281781+0000","last_peered":"2024-02-02T15:26:27.281781+0000","last_clean":"2024-02-02T15:26:27.281781+0000","last_became_active":"2024-02-02T15:24:51.822372+0000","last_became_peered":"2024-02-02T15:24:51.822372+0000","last_unstale":"2024-02-02T15:26:27.281781+0000","last_undegraded":"2024-02-02T15:26:27.281781+0000","last_fullsized":"2024-02-02T15:26:27.281781+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:45:29.631422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"86'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.268664+0000","last_change":"2024-02-02T15:24:47.786507+0000","last_active":"2024-02-02T15:26:27.268664+0000","last_peered":"2024-02-02T15:26:27.268664+0000","last_clean":"2024-02-02T15:26:27.268664+0000","last_became_active":"2024-02-02T15:24:47.786125+0000","last_became_peered":"2024-02-02T15:24:47.786125+0000","last_unstale":"2024-02-02T15:26:27.268664+0000","last_undegraded":"2024-02-02T15:26:27.268664+0000","last_fullsized":"2024-02-02T15:26:27.268664+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:33:09.014258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"84'4","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.274171+0000","last_change":"2024-02-02T15:25:27.628074+0000","last_active":"2024-02-02T15:26:27.274171+0000","last_peered":"2024-02-02T15:26:27.274171+0000","last_clean":"2024-02-02T15:26:27.274171+0000","last_became_active":"2024-02-02T15:25:27.625103+0000","last_became_peered":"2024-02-02T15:25:27.625103+0000","last_unstale":"2024-02-02T15:26:27.274171+0000","last_undegraded":"2024-02-02T15:26:27.274171+0000","last_fullsized":"2024-02-02T15:26:27.274171+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:30:35.540456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.379470+0000","last_change":"2024-02-02T15:24:51.808979+0000","last_active":"2024-02-02T15:26:27.379470+0000","last_peered":"2024-02-02T15:26:27.379470+0000","last_clean":"2024-02-02T15:26:27.379470+0000","last_became_active":"2024-02-02T15:24:51.808557+0000","last_became_peered":"2024-02-02T15:24:51.808557+0000","last_unstale":"2024-02-02T15:26:27.379470+0000","last_undegraded":"2024-02-02T15:26:27.379470+0000","last_fullsized":"2024-02-02T15:26:27.379470+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T02:38:34.697137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"84'4","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.274530+0000","last_change":"2024-02-02T15:24:47.281396+0000","last_active":"2024-02-02T15:26:27.274530+0000","last_peered":"2024-02-02T15:26:27.274530+0000","last_clean":"2024-02-02T15:26:27.274530+0000","last_became_active":"2024-02-02T15:24:47.281036+0000","last_became_peered":"2024-02-02T15:24:47.281036+0000","last_unstale":"2024-02-02T15:26:27.274530+0000","last_undegraded":"2024-02-02T15:26:27.274530+0000","last_fullsized":"2024-02-02T15:26:27.274530+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:54:47.139151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"86'9","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.269092+0000","last_change":"2024-02-02T15:24:47.791057+0000","last_active":"2024-02-02T15:26:27.269092+0000","last_peered":"2024-02-02T15:26:27.269092+0000","last_clean":"2024-02-02T15:26:27.269092+0000","last_became_active":"2024-02-02T15:24:47.790647+0000","last_became_peered":"2024-02-02T15:24:47.790647+0000","last_unstale":"2024-02-02T15:26:27.269092+0000","last_undegraded":"2024-02-02T15:26:27.269092+0000","last_fullsized":"2024-02-02T15:26:27.269092+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T02:49:42.067700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.375624+0000","last_change":"2024-02-02T15:24:51.817792+0000","last_active":"2024-02-02T15:26:27.375624+0000","last_peered":"2024-02-02T15:26:27.375624+0000","last_clean":"2024-02-02T15:26:27.375624+0000","last_became_active":"2024-02-02T15:24:51.815800+0000","last_became_peered":"2024-02-02T15:24:51.815800+0000","last_unstale":"2024-02-02T15:26:27.375624+0000","last_undegraded":"2024-02-02T15:26:27.375624+0000","last_fullsized":"2024-02-02T15:26:27.375624+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:28:10.440251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'12","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-02T15:26:28.276379+0000","last_change":"2024-02-02T15:24:47.277098+0000","last_active":"2024-02-02T15:26:28.276379+0000","last_peered":"2024-02-02T15:26:28.276379+0000","last_clean":"2024-02-02T15:26:28.276379+0000","last_became_active":"2024-02-02T15:24:47.275867+0000","last_became_peered":"2024-02-02T15:24:47.275867+0000","last_unstale":"2024-02-02T15:26:28.276379+0000","last_undegraded":"2024-02-02T15:26:28.276379+0000","last_fullsized":"2024-02-02T15:26:28.276379+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:35:21.015918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"86'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.376351+0000","last_change":"2024-02-02T15:24:47.784873+0000","last_active":"2024-02-02T15:26:27.376351+0000","last_peered":"2024-02-02T15:26:27.376351+0000","last_clean":"2024-02-02T15:26:27.376351+0000","last_became_active":"2024-02-02T15:24:47.784550+0000","last_became_peered":"2024-02-02T15:24:47.784550+0000","last_unstale":"2024-02-02T15:26:27.376351+0000","last_undegraded":"2024-02-02T15:26:27.376351+0000","last_fullsized":"2024-02-02T15:26:27.376351+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:13:37.513754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.379656+0000","last_change":"2024-02-02T15:24:51.805131+0000","last_active":"2024-02-02T15:26:27.379656+0000","last_peered":"2024-02-02T15:26:27.379656+0000","last_clean":"2024-02-02T15:26:27.379656+0000","last_became_active":"2024-02-02T15:24:51.804705+0000","last_became_peered":"2024-02-02T15:24:51.804705+0000","last_unstale":"2024-02-02T15:26:27.379656+0000","last_undegraded":"2024-02-02T15:26:27.379656+0000","last_fullsized":"2024-02-02T15:26:27.379656+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T02:12:59.945050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"84'7","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.280301+0000","last_change":"2024-02-02T15:24:47.266469+0000","last_active":"2024-02-02T15:26:27.280301+0000","last_peered":"2024-02-02T15:26:27.280301+0000","last_clean":"2024-02-02T15:26:27.280301+0000","last_became_active":"2024-02-02T15:24:47.266094+0000","last_became_peered":"2024-02-02T15:24:47.266094+0000","last_unstale":"2024-02-02T15:26:27.280301+0000","last_undegraded":"2024-02-02T15:26:27.280301+0000","last_fullsized":"2024-02-02T15:26:27.280301+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:51:41.324698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"86'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.486689+0000","last_change":"2024-02-02T15:24:47.793051+0000","last_active":"2024-02-02T15:26:27.486689+0000","last_peered":"2024-02-02T15:26:27.486689+0000","last_clean":"2024-02-02T15:26:27.486689+0000","last_became_active":"2024-02-02T15:24:47.792645+0000","last_became_peered":"2024-02-02T15:24:47.792645+0000","last_unstale":"2024-02-02T15:26:27.486689+0000","last_undegraded":"2024-02-02T15:26:27.486689+0000","last_fullsized":"2024-02-02T15:26:27.486689+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:47:17.229407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.280008+0000","last_change":"2024-02-02T15:25:27.282505+0000","last_active":"2024-02-02T15:26:27.280008+0000","last_peered":"2024-02-02T15:26:27.280008+0000","last_clean":"2024-02-02T15:26:27.280008+0000","last_became_active":"2024-02-02T15:25:27.282039+0000","last_became_peered":"2024-02-02T15:25:27.282039+0000","last_unstale":"2024-02-02T15:26:27.280008+0000","last_undegraded":"2024-02-02T15:26:27.280008+0000","last_fullsized":"2024-02-02T15:26:27.280008+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:31:29.700848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"84'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.277078+0000","last_change":"2024-02-02T15:24:47.273716+0000","last_active":"2024-02-02T15:26:27.277078+0000","last_peered":"2024-02-02T15:26:27.277078+0000","last_clean":"2024-02-02T15:26:27.277078+0000","last_became_active":"2024-02-02T15:24:47.273438+0000","last_became_peered":"2024-02-02T15:24:47.273438+0000","last_unstale":"2024-02-02T15:26:27.277078+0000","last_undegraded":"2024-02-02T15:26:27.277078+0000","last_fullsized":"2024-02-02T15:26:27.277078+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:52:16.498119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"86'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.274074+0000","last_change":"2024-02-02T15:24:47.782950+0000","last_active":"2024-02-02T15:26:27.274074+0000","last_peered":"2024-02-02T15:26:27.274074+0000","last_clean":"2024-02-02T15:26:27.274074+0000","last_became_active":"2024-02-02T15:24:47.782605+0000","last_became_peered":"2024-02-02T15:24:47.782605+0000","last_unstale":"2024-02-02T15:26:27.274074+0000","last_undegraded":"2024-02-02T15:26:27.274074+0000","last_fullsized":"2024-02-02T15:26:27.274074+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:57:34.125985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.274573+0000","last_change":"2024-02-02T15:25:27.288179+0000","last_active":"2024-02-02T15:26:27.274573+0000","last_peered":"2024-02-02T15:26:27.274573+0000","last_clean":"2024-02-02T15:26:27.274573+0000","last_became_active":"2024-02-02T15:25:27.287753+0000","last_became_peered":"2024-02-02T15:25:27.287753+0000","last_unstale":"2024-02-02T15:26:27.274573+0000","last_undegraded":"2024-02-02T15:26:27.274573+0000","last_fullsized":"2024-02-02T15:26:27.274573+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:02:46.854950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"84'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.375810+0000","last_change":"2024-02-02T15:24:47.278478+0000","last_active":"2024-02-02T15:26:27.375810+0000","last_peered":"2024-02-02T15:26:27.375810+0000","last_clean":"2024-02-02T15:26:27.375810+0000","last_became_active":"2024-02-02T15:24:47.278084+0000","last_became_peered":"2024-02-02T15:24:47.278084+0000","last_unstale":"2024-02-02T15:26:27.375810+0000","last_undegraded":"2024-02-02T15:26:27.375810+0000","last_fullsized":"2024-02-02T15:26:27.375810+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:10:23.044968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"86'9","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.378859+0000","last_change":"2024-02-02T15:24:47.795291+0000","last_active":"2024-02-02T15:26:27.378859+0000","last_peered":"2024-02-02T15:26:27.378859+0000","last_clean":"2024-02-02T15:26:27.378859+0000","last_became_active":"2024-02-02T15:24:47.794514+0000","last_became_peered":"2024-02-02T15:24:47.794514+0000","last_unstale":"2024-02-02T15:26:27.378859+0000","last_undegraded":"2024-02-02T15:26:27.378859+0000","last_fullsized":"2024-02-02T15:26:27.378859+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:32:12.722258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"84'1","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.280163+0000","last_change":"2024-02-02T15:24:51.825985+0000","last_active":"2024-02-02T15:26:27.280163+0000","last_peered":"2024-02-02T15:26:27.280163+0000","last_clean":"2024-02-02T15:26:27.280163+0000","last_became_active":"2024-02-02T15:24:51.825588+0000","last_became_peered":"2024-02-02T15:24:51.825588+0000","last_unstale":"2024-02-02T15:26:27.280163+0000","last_undegraded":"2024-02-02T15:26:27.280163+0000","last_fullsized":"2024-02-02T15:26:27.280163+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:32:39.910258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"84'3","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.378872+0000","last_change":"2024-02-02T15:25:27.298553+0000","last_active":"2024-02-02T15:26:27.378872+0000","last_peered":"2024-02-02T15:26:27.378872+0000","last_clean":"2024-02-02T15:26:27.378872+0000","last_became_active":"2024-02-02T15:25:27.282469+0000","last_became_peered":"2024-02-02T15:25:27.282469+0000","last_unstale":"2024-02-02T15:26:27.378872+0000","last_undegraded":"2024-02-02T15:26:27.378872+0000","last_fullsized":"2024-02-02T15:26:27.378872+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:54:14.740951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"86'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.375881+0000","last_change":"2024-02-02T15:24:47.784188+0000","last_active":"2024-02-02T15:26:27.375881+0000","last_peered":"2024-02-02T15:26:27.375881+0000","last_clean":"2024-02-02T15:26:27.375881+0000","last_became_active":"2024-02-02T15:24:47.783872+0000","last_became_peered":"2024-02-02T15:24:47.783872+0000","last_unstale":"2024-02-02T15:26:27.375881+0000","last_undegraded":"2024-02-02T15:26:27.375881+0000","last_fullsized":"2024-02-02T15:26:27.375881+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:45:45.213255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.280221+0000","last_change":"2024-02-02T15:24:51.815263+0000","last_active":"2024-02-02T15:26:27.280221+0000","last_peered":"2024-02-02T15:26:27.280221+0000","last_clean":"2024-02-02T15:26:27.280221+0000","last_became_active":"2024-02-02T15:24:51.814849+0000","last_became_peered":"2024-02-02T15:24:51.814849+0000","last_unstale":"2024-02-02T15:26:27.280221+0000","last_undegraded":"2024-02-02T15:26:27.280221+0000","last_fullsized":"2024-02-02T15:26:27.280221+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:05:32.987359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"84'4","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.274712+0000","last_change":"2024-02-02T15:24:47.270534+0000","last_active":"2024-02-02T15:26:27.274712+0000","last_peered":"2024-02-02T15:26:27.274712+0000","last_clean":"2024-02-02T15:26:27.274712+0000","last_became_active":"2024-02-02T15:24:47.270110+0000","last_became_peered":"2024-02-02T15:24:47.270110+0000","last_unstale":"2024-02-02T15:26:27.274712+0000","last_undegraded":"2024-02-02T15:26:27.274712+0000","last_fullsized":"2024-02-02T15:26:27.274712+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T02:59:52.297315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"86'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.273855+0000","last_change":"2024-02-02T15:24:47.791826+0000","last_active":"2024-02-02T15:26:27.273855+0000","last_peered":"2024-02-02T15:26:27.273855+0000","last_clean":"2024-02-02T15:26:27.273855+0000","last_became_active":"2024-02-02T15:24:47.791016+0000","last_became_peered":"2024-02-02T15:24:47.791016+0000","last_unstale":"2024-02-02T15:26:27.273855+0000","last_undegraded":"2024-02-02T15:26:27.273855+0000","last_fullsized":"2024-02-02T15:26:27.273855+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:47:14.296876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.276753+0000","last_change":"2024-02-02T15:24:51.806741+0000","last_active":"2024-02-02T15:26:27.276753+0000","last_peered":"2024-02-02T15:26:27.276753+0000","last_clean":"2024-02-02T15:26:27.276753+0000","last_became_active":"2024-02-02T15:24:51.806085+0000","last_became_peered":"2024-02-02T15:24:51.806085+0000","last_unstale":"2024-02-02T15:26:27.276753+0000","last_undegraded":"2024-02-02T15:26:27.276753+0000","last_fullsized":"2024-02-02T15:26:27.276753+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:53:39.944222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"84'5","reported_seq":109,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-02T15:26:40.400639+0000","last_change":"2024-02-02T15:24:47.278515+0000","last_active":"2024-02-02T15:26:40.400639+0000","last_peered":"2024-02-02T15:26:40.400639+0000","last_clean":"2024-02-02T15:26:40.400639+0000","last_became_active":"2024-02-02T15:24:47.278145+0000","last_became_peered":"2024-02-02T15:24:47.278145+0000","last_unstale":"2024-02-02T15:26:40.400639+0000","last_undegraded":"2024-02-02T15:26:40.400639+0000","last_fullsized":"2024-02-02T15:26:40.400639+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:48:55.118003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":84,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"86'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.274330+0000","last_change":"2024-02-02T15:24:47.791996+0000","last_active":"2024-02-02T15:26:27.274330+0000","last_peered":"2024-02-02T15:26:27.274330+0000","last_clean":"2024-02-02T15:26:27.274330+0000","last_became_active":"2024-02-02T15:24:47.791624+0000","last_became_peered":"2024-02-02T15:24:47.791624+0000","last_unstale":"2024-02-02T15:26:27.274330+0000","last_undegraded":"2024-02-02T15:26:27.274330+0000","last_fullsized":"2024-02-02T15:26:27.274330+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:33:27.963740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.486271+0000","last_change":"2024-02-02T15:24:51.826835+0000","last_active":"2024-02-02T15:26:27.486271+0000","last_peered":"2024-02-02T15:26:27.486271+0000","last_clean":"2024-02-02T15:26:27.486271+0000","last_became_active":"2024-02-02T15:24:51.826310+0000","last_became_peered":"2024-02-02T15:24:51.826310+0000","last_unstale":"2024-02-02T15:26:27.486271+0000","last_undegraded":"2024-02-02T15:26:27.486271+0000","last_fullsized":"2024-02-02T15:26:27.486271+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:48:16.687048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'3","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.282058+0000","last_change":"2024-02-02T15:24:47.268865+0000","last_active":"2024-02-02T15:26:27.282058+0000","last_peered":"2024-02-02T15:26:27.282058+0000","last_clean":"2024-02-02T15:26:27.282058+0000","last_became_active":"2024-02-02T15:24:47.268523+0000","last_became_peered":"2024-02-02T15:24:47.268523+0000","last_unstale":"2024-02-02T15:26:27.282058+0000","last_undegraded":"2024-02-02T15:26:27.282058+0000","last_fullsized":"2024-02-02T15:26:27.282058+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:45:16.957512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'6","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.274101+0000","last_change":"2024-02-02T15:24:47.792601+0000","last_active":"2024-02-02T15:26:27.274101+0000","last_peered":"2024-02-02T15:26:27.274101+0000","last_clean":"2024-02-02T15:26:27.274101+0000","last_became_active":"2024-02-02T15:24:47.792240+0000","last_became_peered":"2024-02-02T15:24:47.792240+0000","last_unstale":"2024-02-02T15:26:27.274101+0000","last_undegraded":"2024-02-02T15:26:27.274101+0000","last_fullsized":"2024-02-02T15:26:27.274101+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:51:25.205859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.281132+0000","last_change":"2024-02-02T15:24:51.806267+0000","last_active":"2024-02-02T15:26:27.281132+0000","last_peered":"2024-02-02T15:26:27.281132+0000","last_clean":"2024-02-02T15:26:27.281132+0000","last_became_active":"2024-02-02T15:24:51.805877+0000","last_became_peered":"2024-02-02T15:24:51.805877+0000","last_unstale":"2024-02-02T15:26:27.281132+0000","last_undegraded":"2024-02-02T15:26:27.281132+0000","last_fullsized":"2024-02-02T15:26:27.281132+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:00:47.747377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.269402+0000","last_change":"2024-02-02T15:24:47.271057+0000","last_active":"2024-02-02T15:26:27.269402+0000","last_peered":"2024-02-02T15:26:27.269402+0000","last_clean":"2024-02-02T15:26:27.269402+0000","last_became_active":"2024-02-02T15:24:47.270737+0000","last_became_peered":"2024-02-02T15:24:47.270737+0000","last_unstale":"2024-02-02T15:26:27.269402+0000","last_undegraded":"2024-02-02T15:26:27.269402+0000","last_fullsized":"2024-02-02T15:26:27.269402+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:17:00.212426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"86'9","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.274263+0000","last_change":"2024-02-02T15:24:47.794108+0000","last_active":"2024-02-02T15:26:27.274263+0000","last_peered":"2024-02-02T15:26:27.274263+0000","last_clean":"2024-02-02T15:26:27.274263+0000","last_became_active":"2024-02-02T15:24:47.793890+0000","last_became_peered":"2024-02-02T15:24:47.793890+0000","last_unstale":"2024-02-02T15:26:27.274263+0000","last_undegraded":"2024-02-02T15:26:27.274263+0000","last_fullsized":"2024-02-02T15:26:27.274263+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T03:05:51.704385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"86'9","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.378946+0000","last_change":"2024-02-02T15:24:47.796234+0000","last_active":"2024-02-02T15:26:27.378946+0000","last_peered":"2024-02-02T15:26:27.378946+0000","last_clean":"2024-02-02T15:26:27.378946+0000","last_became_active":"2024-02-02T15:24:47.795474+0000","last_became_peered":"2024-02-02T15:24:47.795474+0000","last_unstale":"2024-02-02T15:26:27.378946+0000","last_undegraded":"2024-02-02T15:26:27.378946+0000","last_fullsized":"2024-02-02T15:26:27.378946+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:25:28.059500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"86'12","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.379845+0000","last_change":"2024-02-02T15:24:47.786930+0000","last_active":"2024-02-02T15:26:27.379845+0000","last_peered":"2024-02-02T15:26:27.379845+0000","last_clean":"2024-02-02T15:26:27.379845+0000","last_became_active":"2024-02-02T15:24:47.786615+0000","last_became_peered":"2024-02-02T15:24:47.786615+0000","last_unstale":"2024-02-02T15:26:27.379845+0000","last_undegraded":"2024-02-02T15:26:27.379845+0000","last_fullsized":"2024-02-02T15:26:27.379845+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:17:23.231511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"86'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.379038+0000","last_change":"2024-02-02T15:24:47.797877+0000","last_active":"2024-02-02T15:26:27.379038+0000","last_peered":"2024-02-02T15:26:27.379038+0000","last_clean":"2024-02-02T15:26:27.379038+0000","last_became_active":"2024-02-02T15:24:47.797559+0000","last_became_peered":"2024-02-02T15:24:47.797559+0000","last_unstale":"2024-02-02T15:26:27.379038+0000","last_undegraded":"2024-02-02T15:26:27.379038+0000","last_fullsized":"2024-02-02T15:26:27.379038+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:13:55.529796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"86'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.486450+0000","last_change":"2024-02-02T15:24:47.790270+0000","last_active":"2024-02-02T15:26:27.486450+0000","last_peered":"2024-02-02T15:26:27.486450+0000","last_clean":"2024-02-02T15:26:27.486450+0000","last_became_active":"2024-02-02T15:24:47.789928+0000","last_became_peered":"2024-02-02T15:24:47.789928+0000","last_unstale":"2024-02-02T15:26:27.486450+0000","last_undegraded":"2024-02-02T15:26:27.486450+0000","last_fullsized":"2024-02-02T15:26:27.486450+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T02:34:44.885803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"86'4","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.269112+0000","last_change":"2024-02-02T15:25:27.298415+0000","last_active":"2024-02-02T15:26:27.269112+0000","last_peered":"2024-02-02T15:26:27.269112+0000","last_clean":"2024-02-02T15:26:27.269112+0000","last_became_active":"2024-02-02T15:25:27.297990+0000","last_became_peered":"2024-02-02T15:25:27.297990+0000","last_unstale":"2024-02-02T15:26:27.269112+0000","last_undegraded":"2024-02-02T15:26:27.269112+0000","last_fullsized":"2024-02-02T15:26:27.269112+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:38:02.040613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"86'6","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.380026+0000","last_change":"2024-02-02T15:24:47.797861+0000","last_active":"2024-02-02T15:26:27.380026+0000","last_peered":"2024-02-02T15:26:27.380026+0000","last_clean":"2024-02-02T15:26:27.380026+0000","last_became_active":"2024-02-02T15:24:47.797543+0000","last_became_peered":"2024-02-02T15:24:47.797543+0000","last_unstale":"2024-02-02T15:26:27.380026+0000","last_undegraded":"2024-02-02T15:26:27.380026+0000","last_fullsized":"2024-02-02T15:26:27.380026+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:09:37.710909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"86'15","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.380423+0000","last_change":"2024-02-02T15:24:47.796221+0000","last_active":"2024-02-02T15:26:27.380423+0000","last_peered":"2024-02-02T15:26:27.380423+0000","last_clean":"2024-02-02T15:26:27.380423+0000","last_became_active":"2024-02-02T15:24:47.795845+0000","last_became_peered":"2024-02-02T15:24:47.795845+0000","last_unstale":"2024-02-02T15:26:27.380423+0000","last_undegraded":"2024-02-02T15:26:27.380423+0000","last_fullsized":"2024-02-02T15:26:27.380423+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:54:55.637660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"86'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.376469+0000","last_change":"2024-02-02T15:24:47.783086+0000","last_active":"2024-02-02T15:26:27.376469+0000","last_peered":"2024-02-02T15:26:27.376469+0000","last_clean":"2024-02-02T15:26:27.376469+0000","last_became_active":"2024-02-02T15:24:47.781826+0000","last_became_peered":"2024-02-02T15:24:47.781826+0000","last_unstale":"2024-02-02T15:26:27.376469+0000","last_undegraded":"2024-02-02T15:26:27.376469+0000","last_fullsized":"2024-02-02T15:26:27.376469+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:32:05.623591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"86'6","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.268677+0000","last_change":"2024-02-02T15:25:27.284461+0000","last_active":"2024-02-02T15:26:27.268677+0000","last_peered":"2024-02-02T15:26:27.268677+0000","last_clean":"2024-02-02T15:26:27.268677+0000","last_became_active":"2024-02-02T15:25:27.284080+0000","last_became_peered":"2024-02-02T15:25:27.284080+0000","last_unstale":"2024-02-02T15:26:27.268677+0000","last_undegraded":"2024-02-02T15:26:27.268677+0000","last_fullsized":"2024-02-02T15:26:27.268677+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:43:40.124825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.29","version":"86'8","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-02T15:26:28.282109+0000","last_change":"2024-02-02T15:26:28.281754+0000","last_active":"2024-02-02T15:26:28.282109+0000","last_peered":"2024-02-02T15:26:28.282109+0000","last_clean":"2024-02-02T15:26:28.282109+0000","last_became_active":"2024-02-02T15:26:28.281396+0000","last_became_peered":"2024-02-02T15:26:28.281396+0000","last_unstale":"2024-02-02T15:26:28.282109+0000","last_undegraded":"2024-02-02T15:26:28.282109+0000","last_fullsized":"2024-02-02T15:26:28.282109+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:26:26.973936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2a","version":"86'10","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.277193+0000","last_change":"2024-02-02T15:24:47.795928+0000","last_active":"2024-02-02T15:26:27.277193+0000","last_peered":"2024-02-02T15:26:27.277193+0000","last_clean":"2024-02-02T15:26:27.277193+0000","last_became_active":"2024-02-02T15:24:47.795511+0000","last_became_peered":"2024-02-02T15:24:47.795511+0000","last_unstale":"2024-02-02T15:26:27.277193+0000","last_undegraded":"2024-02-02T15:26:27.277193+0000","last_fullsized":"2024-02-02T15:26:27.277193+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:55:31.444331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"86'11","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.268957+0000","last_change":"2024-02-02T15:24:47.794368+0000","last_active":"2024-02-02T15:26:27.268957+0000","last_peered":"2024-02-02T15:26:27.268957+0000","last_clean":"2024-02-02T15:26:27.268957+0000","last_became_active":"2024-02-02T15:24:47.793808+0000","last_became_peered":"2024-02-02T15:24:47.793808+0000","last_unstale":"2024-02-02T15:26:27.268957+0000","last_undegraded":"2024-02-02T15:26:27.268957+0000","last_fullsized":"2024-02-02T15:26:27.268957+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T02:39:00.099808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"86'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.379545+0000","last_change":"2024-02-02T15:24:47.796358+0000","last_active":"2024-02-02T15:26:27.379545+0000","last_peered":"2024-02-02T15:26:27.379545+0000","last_clean":"2024-02-02T15:26:27.379545+0000","last_became_active":"2024-02-02T15:24:47.795800+0000","last_became_peered":"2024-02-02T15:24:47.795800+0000","last_unstale":"2024-02-02T15:26:27.379545+0000","last_undegraded":"2024-02-02T15:26:27.379545+0000","last_fullsized":"2024-02-02T15:26:27.379545+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:49:40.019787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":250199,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67720,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":864,"num_read_kb":505,"num_write":248,"num_write_kb":144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":581632,"data_stored":213918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":216,"ondisk_log_size":216,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547235,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":983040,"data_stored":496773,"data_compressed":23664,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 15:26:15 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.53600000000000003,"15min":0.53300000000000003},"min":{"1min":0.42999999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.745,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.57099999999999995,"15min":0.55900000000000005},"min":{"1min":0.5,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.45200000000000001}]},{"osd":1,"last update":"Fri Feb 2 15:26:15 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.59599999999999997,"15min":0.58899999999999997},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.80200000000000005,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.61199999999999999,"15min":0.60299999999999998},"min":{"1min":0.39300000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.67200000000000004}]},{"osd":2,"last update":"Fri Feb 2 15:26:15 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.45500000000000002,"15min":0.44400000000000001},"min":{"1min":0.39300000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.496},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.46000000000000002,"15min":0.45100000000000001},"min":{"1min":0.376,"5min":0.316,"15min":0.316},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.63600000000000001}]},{"osd":3,"last update":"Fri Feb 2 15:26:27 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.56699999999999995}]},{"osd":4,"last update":"Fri Feb 2 15:26:15 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.64100000000000001,"15min":0.629},"min":{"1min":0.52400000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.63600000000000001,"15min":0.63400000000000001},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.76300000000000001,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.71299999999999997}]},{"osd":5,"last update":"Fri Feb 2 15:26:15 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.57999999999999996,"15min":0.56299999999999994},"min":{"1min":0.53200000000000003,"5min":0.377,"15min":0.377},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.622},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.57899999999999996,"15min":0.56599999999999995},"min":{"1min":0.504,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.73399999999999999}]},{"osd":6,"last update":"Fri Feb 2 15:26:15 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.70399999999999996,"15min":0.70599999999999996},"min":{"1min":0.57499999999999996,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.93999999999999995,"5min":1.325,"15min":1.325},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.72699999999999998,"15min":0.71499999999999997},"min":{"1min":0.64100000000000001,"5min":0.498,"15min":0.498},"max":{"1min":0.97099999999999997,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.76700000000000002}]},{"osd":7,"last update":"Fri Feb 2 15:26:15 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.71699999999999997,"15min":0.71199999999999997},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.96799999999999997,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.874},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.73799999999999999,"15min":0.73099999999999998},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.92600000000000005,"5min":1.27,"15min":1.27},"last":0.78400000000000003}]},{"osd":8,"last update":"Fri Feb 2 15:25:50 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.82199999999999995}]},{"osd":9,"last update":"Fri Feb 2 15:26:15 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.79000000000000004,"15min":0.78100000000000003},"min":{"1min":0.66900000000000004,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.0109999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.77000000000000002,"15min":0.76000000000000001},"min":{"1min":0.71999999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.006,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.80200000000000005}]},{"osd":10,"last update":"Fri Feb 2 15:26:15 2024","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.79700000000000004,"15min":0.79100000000000004},"min":{"1min":0.63,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":0.95999999999999996,"5min":1.21,"15min":1.21},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.77200000000000002,"15min":0.75800000000000001},"min":{"1min":0.71899999999999997,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.98999999999999999,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.86799999999999999}]}]},{"osd":10,"up_from":67,"seq":287762808868,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1122304,"data_stored":1145011,"data_compressed":59875,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 15:25:48 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.45300000000000001}]},{"osd":1,"last update":"Fri Feb 2 15:25:53 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.439,"15min":0.441},"min":{"1min":0.32200000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.63300000000000001,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.432},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.45900000000000002,"15min":0.46100000000000002},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.36699999999999999}]},{"osd":2,"last update":"Fri Feb 2 15:25:53 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.58799999999999997,"15min":0.59199999999999997},"min":{"1min":0.40000000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.72499999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.59399999999999997,"15min":0.59499999999999997},"min":{"1min":0.42599999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.74399999999999999,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.59299999999999997}]},{"osd":3,"last update":"Fri Feb 2 15:25:53 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.58999999999999997,"15min":0.59799999999999998},"min":{"1min":0.39900000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.73299999999999998,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.78700000000000003,"15min":0.82399999999999995},"min":{"1min":0.40600000000000003,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.70399999999999996,"5min":4.4109999999999996,"15min":4.4109999999999996},"last":0.56100000000000005}]},{"osd":4,"last update":"Fri Feb 2 15:25:53 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.36899999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.78400000000000003,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.52000000000000002,"15min":0.52500000000000002},"min":{"1min":0.375,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.65200000000000002,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.40200000000000002}]},{"osd":5,"last update":"Fri Feb 2 15:25:53 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.65600000000000003,"15min":0.65800000000000003},"min":{"1min":0.45500000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.77200000000000002,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.66300000000000003,"15min":0.66000000000000003},"min":{"1min":0.47099999999999997,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.86499999999999999,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.51700000000000002}]},{"osd":6,"last update":"Fri Feb 2 15:25:53 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.80100000000000005,"15min":0.82699999999999996},"min":{"1min":0.52300000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.82799999999999996,"5min":4.4459999999999997,"15min":4.4459999999999997},"last":0.627},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.71999999999999997,"15min":0.72899999999999998},"min":{"1min":0.44500000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.83899999999999997,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.64700000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40300000000000002}]},{"osd":8,"last update":"Fri Feb 2 15:25:53 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.71999999999999997,"15min":0.72399999999999998},"min":{"1min":0.496,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.92100000000000004,"5min":1.413,"15min":1.413},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.71899999999999997,"15min":0.72199999999999998},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.89800000000000002,"5min":1.036,"15min":1.036},"last":0.54400000000000004}]},{"osd":9,"last update":"Fri Feb 2 15:25:53 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.77600000000000002,"15min":0.78400000000000003},"min":{"1min":0.58899999999999997,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.88300000000000001,"5min":1.506,"15min":1.506},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.89800000000000002,"15min":0.93200000000000005},"min":{"1min":0.55400000000000005,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.86799999999999999,"5min":4.4269999999999996,"15min":4.4269999999999996},"last":0.63900000000000001}]},{"osd":11,"last update":"Fri Feb 2 15:26:14 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.73899999999999999,"15min":0.73299999999999998},"min":{"1min":0.67500000000000004,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.95999999999999996,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.74099999999999999,"15min":0.73799999999999999},"min":{"1min":0.64900000000000002,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.93899999999999995,"5min":0.999,"15min":0.999},"last":0.60499999999999998}]}]},{"osd":9,"up_from":62,"seq":266287972393,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1003520,"data_stored":500779,"data_compressed":23664,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 15:26:38 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.53600000000000003,"15min":0.54300000000000004},"min":{"1min":0.33500000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.069,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.55200000000000005,"15min":0.56499999999999995},"min":{"1min":0.32700000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.70799999999999996,"5min":1.196,"15min":1.196},"last":0.53400000000000003}]},{"osd":1,"last update":"Fri Feb 2 15:26:38 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.72799999999999998,"15min":0.72199999999999998},"min":{"1min":0.51600000000000001,"5min":0.38,"15min":0.38},"max":{"1min":1.0309999999999999,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.748},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.69299999999999995,"15min":0.70299999999999996},"min":{"1min":0.46800000000000003,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.80100000000000005,"5min":1.198,"15min":1.198},"last":0.67300000000000004}]},{"osd":2,"last update":"Fri Feb 2 15:26:38 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.71299999999999997,"15min":0.72999999999999998},"min":{"1min":0.50900000000000001,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.90600000000000003,"5min":1.135,"15min":1.135},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.68700000000000006,"15min":0.68300000000000005},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.66000000000000003}]},{"osd":3,"last update":"Fri Feb 2 15:26:38 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.71499999999999997,"15min":0.71899999999999997},"min":{"1min":0.48499999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.98499999999999999,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.67900000000000005,"15min":0.71499999999999997},"min":{"1min":0.46100000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.89100000000000001,"5min":1.296,"15min":1.296},"last":0.61899999999999999}]},{"osd":4,"last update":"Fri Feb 2 15:26:38 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.69899999999999995,"15min":0.71499999999999997},"min":{"1min":0.45300000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.302,"5min":1.381,"15min":1.381},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.81699999999999995,"15min":0.83199999999999996},"min":{"1min":0.55900000000000005,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.1000000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.79400000000000004}]},{"osd":5,"last update":"Fri Feb 2 15:26:38 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.84899999999999998,"15min":0.91500000000000004},"min":{"1min":0.497,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.3680000000000001,"5min":3.895,"15min":3.895},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.84399999999999997,"15min":0.85699999999999998},"min":{"1min":0.55400000000000005,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.1679999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.83899999999999997}]},{"osd":6,"last update":"Fri Feb 2 15:24:35 2024","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.6739999999999999,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":0.64500000000000002}]},{"osd":7,"last update":"Fri Feb 2 15:26:38 2024","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.89500000000000002,"15min":0.91300000000000003},"min":{"1min":0.69499999999999995,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.1359999999999999,"5min":1.347,"15min":1.347},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.84199999999999997,"15min":0.88600000000000001},"min":{"1min":0.55200000000000005,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.86399999999999999,"5min":1.413,"15min":1.413},"last":0.77900000000000003}]},{"osd":8,"last update":"Fri Feb 2 15:26:38 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.86799999999999999,"15min":0.91300000000000003},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.474,"5min":1.7789999999999999,"15min":1.7789999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.93600000000000005,"15min":0.96699999999999997},"min":{"1min":0.66900000000000004,"5min":0.625,"15min":0.625},"max":{"1min":1.1970000000000001,"5min":1.538,"15min":1.538},"last":0.86699999999999999}]},{"osd":10,"last update":"Fri Feb 2 15:25:56 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.93200000000000005,"15min":0.92700000000000005},"min":{"1min":0.71399999999999997,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.389,"5min":1.389,"15min":1.389},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.874,"5min":0.86199999999999999,"15min":0.85999999999999999},"min":{"1min":0.70499999999999996,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":0.78700000000000003}]},{"osd":11,"last update":"Fri Feb 2 15:25:51 2024","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.474,"5min":1.474,"15min":1.474},"last":0.91000000000000003}]}]},{"osd":8,"up_from":55,"seq":236223201325,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":991232,"data_stored":496730,"data_compressed":23664,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 15:26:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.57899999999999996,"15min":0.58199999999999996},"min":{"1min":0.44900000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.71099999999999997,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.58599999999999997,"15min":0.59799999999999998},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.73799999999999999,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.50600000000000001}]},{"osd":1,"last update":"Fri Feb 2 15:26:16 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.59399999999999997,"15min":0.57299999999999995},"min":{"1min":0.45900000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.83099999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.626},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.59399999999999997,"15min":0.56999999999999995},"min":{"1min":0.505,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.79500000000000004,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.64100000000000001}]},{"osd":2,"last update":"Fri Feb 2 15:26:16 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.44700000000000001,"15min":0.42599999999999999},"min":{"1min":0.34999999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.499,"5min":0.44,"15min":0.42199999999999999},"min":{"1min":0.38500000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.59699999999999998}]},{"osd":3,"last update":"Fri Feb 2 15:26:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.61399999999999999,"15min":0.625},"min":{"1min":0.40999999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.751,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.61299999999999999,"15min":0.62},"min":{"1min":0.435,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.58499999999999996}]},{"osd":4,"last update":"Fri Feb 2 15:26:16 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.621,"15min":0.60499999999999998},"min":{"1min":0.42199999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.84599999999999997,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.61899999999999999,"15min":0.59899999999999998},"min":{"1min":0.45400000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.67800000000000005}]},{"osd":5,"last update":"Fri Feb 2 15:26:16 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.53400000000000003,"15min":0.51100000000000001},"min":{"1min":0.49399999999999999,"5min":0.311,"15min":0.311},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.53400000000000003,"15min":0.51000000000000001},"min":{"1min":0.47399999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.71499999999999997}]},{"osd":6,"last update":"Fri Feb 2 15:26:16 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.68300000000000005,"15min":0.67900000000000005},"min":{"1min":0.54800000000000004,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.70599999999999996,"15min":0.69799999999999995},"min":{"1min":0.56000000000000005,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.76100000000000001}]},{"osd":7,"last update":"Fri Feb 2 15:26:16 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.67200000000000004,"15min":0.64900000000000002},"min":{"1min":0.62,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.69799999999999995,"15min":0.67800000000000005},"min":{"1min":0.61299999999999999,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.96099999999999997,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.752}]},{"osd":9,"last update":"Fri Feb 2 15:26:40 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.72199999999999998,"15min":0.69799999999999995},"min":{"1min":0.53600000000000003,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.86799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.74399999999999999,"15min":0.71999999999999997},"min":{"1min":0.61899999999999999,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.90300000000000002,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.83299999999999996}]},{"osd":10,"last update":"Fri Feb 2 15:25:56 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.73099999999999998,"15min":0.71699999999999997},"min":{"1min":0.56200000000000006,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.94599999999999995,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.72899999999999998,"15min":0.71299999999999997},"min":{"1min":0.63600000000000001,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.80900000000000005}]},{"osd":11,"last update":"Fri Feb 2 15:25:48 2024","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.78500000000000003}]}]},{"osd":7,"up_from":48,"seq":206158430256,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1048576,"data_stored":1093556,"data_compressed":59875,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 15:23:58 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.439},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.38500000000000001}]},{"osd":1,"last update":"Fri Feb 2 15:26:01 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.432,"15min":0.42799999999999999},"min":{"1min":0.34399999999999997,"5min":0.247,"15min":0.247},"max":{"1min":0.64300000000000002,"5min":0.749,"15min":0.749},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.42199999999999999,"15min":0.41099999999999998},"min":{"1min":0.37,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.65500000000000003,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.53100000000000003}]},{"osd":2,"last update":"Fri Feb 2 15:26:01 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.57499999999999996,"15min":0.57199999999999995},"min":{"1min":0.434,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.77000000000000002,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.59599999999999997,"15min":0.58799999999999997},"min":{"1min":0.48599999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.76500000000000001,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.75600000000000001}]},{"osd":3,"last update":"Fri Feb 2 15:26:01 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.64300000000000002,"15min":0.61099999999999999},"min":{"1min":0.51000000000000001,"5min":0.374,"15min":0.374},"max":{"1min":4.3890000000000002,"5min":4.3890000000000002,"15min":4.3890000000000002},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.623,"15min":0.60499999999999998},"min":{"1min":0.47599999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":4.1280000000000001,"5min":4.1280000000000001,"15min":4.1280000000000001},"last":0.60199999999999998}]},{"osd":4,"last update":"Fri Feb 2 15:26:01 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.50600000000000001,"15min":0.501},"min":{"1min":0.38800000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.78500000000000003,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.5,"15min":0.496},"min":{"1min":0.35999999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.437}]},{"osd":5,"last update":"Fri Feb 2 15:26:01 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.64000000000000001,"15min":0.622},"min":{"1min":0.44900000000000001,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.81299999999999994,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.63100000000000001,"15min":0.625},"min":{"1min":0.54700000000000004,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.88600000000000001,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.63500000000000001}]},{"osd":6,"last update":"Fri Feb 2 15:26:01 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.68400000000000005,"15min":0.68999999999999995},"min":{"1min":0.54400000000000004,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.96399999999999997,"5min":0.998,"15min":0.998},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.67300000000000004,"15min":0.66700000000000004},"min":{"1min":0.46400000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.95599999999999996,"5min":1.778,"15min":1.778},"last":0.66100000000000003}]},{"osd":8,"last update":"Fri Feb 2 15:26:17 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.71199999999999997,"15min":0.70299999999999996},"min":{"1min":0.53600000000000003,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.97699999999999998,"5min":2.069,"15min":2.069},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.73599999999999999,"15min":0.73499999999999999},"min":{"1min":0.65900000000000003,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.93100000000000005,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.74099999999999999}]},{"osd":9,"last update":"Fri Feb 2 15:25:40 2024","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.76100000000000001,"15min":0.747},"min":{"1min":0.497,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":4.9550000000000001,"5min":4.9550000000000001,"15min":4.9550000000000001},"last":0.62},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.753,"15min":0.73899999999999999},"min":{"1min":0.54500000000000004,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":4.0759999999999996,"5min":4.0759999999999996,"15min":4.0759999999999996},"last":0.67600000000000005}]},{"osd":10,"last update":"Fri Feb 2 15:25:55 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.47099999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.82499999999999996,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.495,"5min":0.436,"15min":0.436},"max":{"1min":0.88800000000000001,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.72499999999999998}]},{"osd":11,"last update":"Fri Feb 2 15:25:51 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.85199999999999998}]}]},{"osd":6,"up_from":43,"seq":184683593780,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":974848,"data_stored":492139,"data_compressed":23664,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 15:25:44 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.497,"15min":0.48499999999999999},"min":{"1min":0.35599999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.67900000000000005,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51900000000000002,"15min":0.51200000000000001},"min":{"1min":0.371,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.67200000000000004,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.442}]},{"osd":1,"last update":"Fri Feb 2 15:25:44 2024","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":0.72199999999999998,"15min":0.66400000000000003},"min":{"1min":0.47999999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":6.8650000000000002,"5min":6.8650000000000002,"15min":6.8650000000000002},"last":0.626},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.66400000000000003,"15min":0.65200000000000002},"min":{"1min":0.50600000000000001,"5min":0.501,"15min":0.501},"max":{"1min":0.88900000000000001,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.501}]},{"osd":2,"last update":"Fri Feb 2 15:25:44 2024","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":0.79700000000000004,"15min":0.72699999999999998},"min":{"1min":0.57199999999999995,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":6.9320000000000004,"5min":6.9320000000000004,"15min":6.9320000000000004},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.753,"15min":0.72499999999999998},"min":{"1min":0.52600000000000002,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":3.008,"5min":3.008,"15min":3.008},"last":0.68700000000000006}]},{"osd":3,"last update":"Fri Feb 2 15:25:44 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.628,"15min":0.63300000000000001},"min":{"1min":0.432,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.875,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.67400000000000004,"15min":0.66400000000000003},"min":{"1min":0.45100000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.73299999999999998}]},{"osd":4,"last update":"Fri Feb 2 15:25:44 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.72999999999999998,"15min":0.71999999999999997},"min":{"1min":0.53100000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.95599999999999996,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76200000000000001,"15min":0.75600000000000001},"min":{"1min":0.55700000000000005,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.0489999999999999,"5min":1.228,"15min":1.228},"last":0.58999999999999997}]},{"osd":5,"last update":"Fri Feb 2 15:25:44 2024","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.79100000000000004,"15min":0.77900000000000003},"min":{"1min":0.55100000000000005,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.63700000000000001},{"interface":"front","average":{"1min":1.268,"5min":0.90900000000000003,"15min":0.84499999999999997},"min":{"1min":0.61199999999999999,"5min":0.442,"15min":0.442},"max":{"1min":6.9820000000000002,"5min":6.9820000000000002,"15min":6.9820000000000002},"last":0.84199999999999997}]},{"osd":7,"last update":"Fri Feb 2 15:26:02 2024","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":0.88900000000000001,"15min":0.84999999999999998},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":6.9530000000000003,"5min":6.9530000000000003,"15min":6.9530000000000003},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.876,"15min":0.878},"min":{"1min":0.60899999999999999,"5min":0.501,"15min":0.501},"max":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.72099999999999997}]},{"osd":8,"last update":"Fri Feb 2 15:26:15 2024","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.88200000000000001,"15min":0.89500000000000002},"min":{"1min":0.60699999999999998,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.0429999999999999,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.90900000000000003,"15min":0.91000000000000003},"min":{"1min":0.625,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.0620000000000001,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.749}]},{"osd":9,"last update":"Fri Feb 2 15:26:41 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.93300000000000005,"15min":0.94299999999999995},"min":{"1min":0.69799999999999995,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.8280000000000001,"5min":1.8280000000000001,"15min":1.8280000000000001},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.95799999999999996,"15min":0.97399999999999998},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.6299999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":0.74099999999999999}]},{"osd":10,"last update":"Fri Feb 2 15:25:56 2024","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1,"15min":0.96399999999999997},"min":{"1min":0.68300000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":6.9960000000000004,"5min":6.9960000000000004,"15min":6.9960000000000004},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.97799999999999998,"15min":0.98699999999999999},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.1879999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.88700000000000001}]},{"osd":11,"last update":"Fri Feb 2 15:25:51 2024","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.92300000000000004},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":3.032,"5min":3.032,"15min":3.032},"last":0.78500000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574918,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1048576,"data_stored":559505,"data_compressed":23664,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 15:26:16 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51100000000000001,"15min":0.50800000000000001},"min":{"1min":0.41999999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.66100000000000003,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.53400000000000003,"15min":0.52400000000000002},"min":{"1min":0.45700000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.72999999999999998,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.68999999999999995}]},{"osd":2,"last update":"Fri Feb 2 15:26:37 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.58199999999999996,"15min":0.55600000000000005},"min":{"1min":0.42699999999999999,"5min":0.31,"15min":0.31},"max":{"1min":0.80600000000000005,"5min":0.871,"15min":0.871},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.59199999999999997,"15min":0.55600000000000005},"min":{"1min":0.44900000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.51500000000000001}]},{"osd":3,"last update":"Fri Feb 2 15:25:50 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.56899999999999995,"15min":0.53700000000000003},"min":{"1min":0.434,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.57999999999999996,"15min":0.55400000000000005},"min":{"1min":0.45800000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.621}]},{"osd":4,"last update":"Fri Feb 2 15:26:05 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.43099999999999999,"15min":0.38100000000000001},"min":{"1min":0.35299999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.754,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.433,"15min":0.38600000000000001},"min":{"1min":0.376,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.69799999999999995,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.47499999999999998}]},{"osd":5,"last update":"Fri Feb 2 15:26:22 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.59499999999999997,"15min":0.55300000000000005},"min":{"1min":0.52800000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.438},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.626,"15min":0.56999999999999995},"min":{"1min":0.58399999999999996,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.85699999999999998,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.64300000000000002}]},{"osd":6,"last update":"Fri Feb 2 15:25:46 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.621,"15min":0.59499999999999997},"min":{"1min":0.50800000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.60599999999999998,"15min":0.58799999999999997},"min":{"1min":0.48099999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.71499999999999997}]},{"osd":7,"last update":"Fri Feb 2 15:26:01 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.59599999999999997,"15min":0.57999999999999996},"min":{"1min":0.45000000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.90500000000000003,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.55800000000000005,"15min":0.54000000000000004},"min":{"1min":0.434,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.79100000000000004,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.48599999999999999}]},{"osd":8,"last update":"Fri Feb 2 15:26:16 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.65800000000000003,"15min":0.63800000000000001},"min":{"1min":0.51000000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.68600000000000005,"15min":0.65600000000000003},"min":{"1min":0.55100000000000005,"5min":0.442,"15min":0.442},"max":{"1min":0.94099999999999995,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.67400000000000004}]},{"osd":9,"last update":"Fri Feb 2 15:26:39 2024","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.73899999999999999,"15min":0.71399999999999997},"min":{"1min":0.47499999999999998,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":2.6840000000000002,"5min":2.6840000000000002,"15min":2.6840000000000002},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.76300000000000001,"15min":0.747},"min":{"1min":0.70999999999999996,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.96199999999999997,"5min":1.121,"15min":1.121},"last":0.79100000000000004}]},{"osd":10,"last update":"Fri Feb 2 15:25:56 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.69499999999999995,"15min":0.68500000000000005},"min":{"1min":0.57399999999999995,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.72699999999999998,"15min":0.71499999999999997},"min":{"1min":0.57999999999999996,"5min":0.5,"15min":0.5},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.73099999999999998}]},{"osd":11,"last update":"Fri Feb 2 15:25:50 2024","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.77600000000000002}]}]},{"osd":0,"up_from":8,"seq":34359738441,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":999424,"data_stored":493595,"data_compressed":23664,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Feb 2 15:26:11 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.56599999999999995,"15min":0.54600000000000004},"min":{"1min":0.374,"5min":0.314,"15min":0.314},"max":{"1min":0.79500000000000004,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.63},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.59999999999999998,"15min":0.55400000000000005},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.411,"5min":1.411,"15min":1.411},"last":0.78400000000000003}]},{"osd":2,"last update":"Fri Feb 2 15:26:32 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.70499999999999996,"15min":0.59999999999999998},"min":{"1min":0.52900000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.369,"5min":1.369,"15min":1.369},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.745,"5min":0.74299999999999999,"15min":0.628},"min":{"1min":0.373,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.9079999999999999,"5min":4.6550000000000002,"15min":4.6550000000000002},"last":0.57299999999999995}]},{"osd":3,"last update":"Fri Feb 2 15:25:48 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.495},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":0.44600000000000001}]},{"osd":4,"last update":"Fri Feb 2 15:26:05 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.68400000000000005,"15min":0.64700000000000002},"min":{"1min":0.44400000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.66000000000000003,"15min":0.63200000000000001},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.8049999999999999,"5min":1.8049999999999999,"15min":1.8049999999999999},"last":0.64900000000000002}]},{"osd":5,"last update":"Fri Feb 2 15:26:22 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.69999999999999996,"15min":0.68899999999999995},"min":{"1min":0.52300000000000002,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.9450000000000001,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":0.748},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.71699999999999997,"15min":0.67800000000000005},"min":{"1min":0.44400000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.98599999999999999,"5min":4.5890000000000004,"15min":4.5890000000000004},"last":0.47999999999999998}]},{"osd":6,"last update":"Fri Feb 2 15:25:43 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.72899999999999998,"15min":0.70499999999999996},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.494,"5min":1.494,"15min":1.494},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.74099999999999999,"15min":0.73099999999999998},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.872}]},{"osd":7,"last update":"Fri Feb 2 15:25:55 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.79800000000000004}]},{"osd":8,"last update":"Fri Feb 2 15:26:22 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.84899999999999998,"15min":0.84299999999999997},"min":{"1min":0.65900000000000003,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":2.0169999999999999,"5min":2.0169999999999999,"15min":2.0169999999999999},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.79200000000000004,"15min":0.77400000000000002},"min":{"1min":0.57199999999999995,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.1479999999999999,"5min":1.155,"15min":1.155},"last":0.76800000000000002}]},{"osd":9,"last update":"Fri Feb 2 15:25:40 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.96399999999999997,"15min":0.98699999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0980000000000001,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.95399999999999996,"15min":0.96799999999999997},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.411,"5min":1.411,"15min":1.411},"last":0.92000000000000004}]},{"osd":10,"last update":"Fri Feb 2 15:26:01 2024","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.879,"15min":0.878},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.84399999999999997},{"interface":"front","average":{"1min":1,"5min":0.97199999999999998,"15min":0.96799999999999997},"min":{"1min":0.66100000000000003,"5min":0.63,"15min":0.63},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.96099999999999997}]},{"osd":11,"last update":"Fri Feb 2 15:26:11 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.93500000000000005,"15min":0.93899999999999995},"min":{"1min":0.73299999999999998,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":0.88400000000000001},{"interface":"front","average":{"1min":1.008,"5min":0.97499999999999998,"15min":0.96899999999999997},"min":{"1min":0.81100000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.6000000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.98299999999999998}]}]},{"osd":2,"up_from":18,"seq":77309411395,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1060864,"data_stored":559609,"data_compressed":23664,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 15:26:29 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.55500000000000005,"15min":0.54600000000000004},"min":{"1min":0.45200000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.79500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.53800000000000003,"15min":0.52400000000000002},"min":{"1min":0.40899999999999997,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.85999999999999999,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.53300000000000003}]},{"osd":1,"last update":"Fri Feb 2 15:26:29 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.55100000000000005,"15min":0.51300000000000001},"min":{"1min":0.35299999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.496},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.59199999999999997,"15min":0.54300000000000004},"min":{"1min":0.41199999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.86299999999999999,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.76200000000000001}]},{"osd":3,"last update":"Fri Feb 2 15:25:46 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.60099999999999998,"15min":0.57199999999999995},"min":{"1min":0.46100000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.75600000000000001,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.623,"5min":0.58799999999999997,"15min":0.55600000000000005},"min":{"1min":0.42799999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.81000000000000005,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.60899999999999999}]},{"osd":4,"last update":"Fri Feb 2 15:25:50 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.65600000000000003}]},{"osd":5,"last update":"Fri Feb 2 15:26:29 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.45700000000000002,"15min":0.41799999999999998},"min":{"1min":0.36699999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.78900000000000003,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.44700000000000001,"15min":0.41499999999999998},"min":{"1min":0.36799999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.63600000000000001}]},{"osd":6,"last update":"Fri Feb 2 15:25:40 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.625,"15min":0.61899999999999999},"min":{"1min":0.46100000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.79500000000000004,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.628,"5min":0.625,"15min":0.627},"min":{"1min":0.437,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.746}]},{"osd":7,"last update":"Fri Feb 2 15:25:52 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.78900000000000003}]},{"osd":8,"last update":"Fri Feb 2 15:26:18 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.59099999999999997,"15min":0.57699999999999996},"min":{"1min":0.40500000000000003,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.55700000000000005,"15min":0.54300000000000004},"min":{"1min":0.41999999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.70799999999999996}]},{"osd":9,"last update":"Fri Feb 2 15:25:40 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.66900000000000004,"15min":0.66700000000000004},"min":{"1min":0.501,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.751,"15min":0.746},"min":{"1min":0.60299999999999998,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.92500000000000004,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.81399999999999995}]},{"osd":10,"last update":"Fri Feb 2 15:25:52 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.68300000000000005,"15min":0.67200000000000004},"min":{"1min":0.628,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.68100000000000005,"15min":0.67100000000000004},"min":{"1min":0.56299999999999994,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.86799999999999999,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.86599999999999999}]},{"osd":11,"last update":"Fri Feb 2 15:25:46 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.89900000000000002}]}]},{"osd":3,"up_from":24,"seq":103079215166,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":987136,"data_stored":491895,"data_compressed":23664,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 15:25:44 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.442,"15min":0.41899999999999998},"min":{"1min":0.34899999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.69699999999999995,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.46600000000000003,"15min":0.45300000000000001},"min":{"1min":0.33300000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.73099999999999998,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.442}]},{"osd":1,"last update":"Fri Feb 2 15:25:44 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.65000000000000002,"15min":0.59599999999999997},"min":{"1min":0.52900000000000003,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.0649999999999999,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.622,"15min":0.59899999999999998},"min":{"1min":0.49099999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.879,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.52400000000000002}]},{"osd":2,"last update":"Fri Feb 2 15:25:44 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.63300000000000001,"15min":0.623},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.625},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.70299999999999996,"15min":0.65700000000000003},"min":{"1min":0.56799999999999995,"5min":0.443,"15min":0.443},"max":{"1min":1.1319999999999999,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":0.504}]},{"osd":4,"last update":"Fri Feb 2 15:26:01 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.62,"15min":0.624},"min":{"1min":0.495,"5min":0.376,"15min":0.376},"max":{"1min":0.873,"5min":1.022,"15min":1.022},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.72599999999999998,"15min":0.69799999999999995},"min":{"1min":0.49199999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.75900000000000001}]},{"osd":5,"last update":"Fri Feb 2 15:26:25 2024","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.748,"15min":0.72599999999999998},"min":{"1min":0.57699999999999996,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.88600000000000001,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.66400000000000003,"15min":0.64700000000000002},"min":{"1min":0.50900000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.84399999999999997,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.69699999999999995}]},{"osd":6,"last update":"Fri Feb 2 15:25:44 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.74199999999999999,"15min":0.71599999999999997},"min":{"1min":0.57099999999999995,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.1100000000000001,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.68999999999999995,"15min":0.68799999999999994},"min":{"1min":0.40500000000000003,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.90600000000000003,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.58199999999999996}]},{"osd":7,"last update":"Fri Feb 2 15:26:01 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.751,"15min":0.74099999999999999},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.008,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.501,"5min":0.44,"15min":0.44},"max":{"1min":0.89900000000000002,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.74199999999999999}]},{"osd":8,"last update":"Fri Feb 2 15:24:14 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.65800000000000003}]},{"osd":9,"last update":"Fri Feb 2 15:26:38 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.91100000000000003,"15min":0.94199999999999995},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.93700000000000006,"5min":1.2,"15min":1.2},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.81999999999999995,"15min":0.84299999999999997},"min":{"1min":0.55400000000000005,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0629999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.77300000000000002}]},{"osd":10,"last update":"Fri Feb 2 15:25:59 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.87,"15min":0.877},"min":{"1min":0.65500000000000003,"5min":0.624,"15min":0.624},"max":{"1min":1.0840000000000001,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.95699999999999996,"15min":0.96699999999999997},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.83299999999999996}]},{"osd":11,"last update":"Fri Feb 2 15:25:49 2024","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.68300000000000005}]}]},{"osd":4,"up_from":31,"seq":133143986237,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":496746,"data_compressed":23664,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 15:24:03 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.52300000000000002,"15min":0.52100000000000002},"min":{"1min":0.371,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.78600000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.53800000000000003,"15min":0.53600000000000003},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.47799999999999998}]},{"osd":1,"last update":"Fri Feb 2 15:26:06 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.376,"15min":0.34300000000000003},"min":{"1min":0.33300000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.55000000000000004,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.439},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.38700000000000001,"15min":0.35499999999999998},"min":{"1min":0.29899999999999999,"5min":0.255,"15min":0.255},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.47999999999999998}]},{"osd":2,"last update":"Fri Feb 2 15:26:06 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.54600000000000004,"15min":0.51400000000000001},"min":{"1min":0.438,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.54700000000000004,"15min":0.51700000000000002},"min":{"1min":0.52500000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.67900000000000005,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.63300000000000001}]},{"osd":3,"last update":"Fri Feb 2 15:26:06 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.61099999999999999,"15min":0.57899999999999996},"min":{"1min":0.40200000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":4.3220000000000001,"5min":4.3220000000000001,"15min":4.3220000000000001},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.60699999999999998,"15min":0.57599999999999996},"min":{"1min":0.436,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":4.3959999999999999,"5min":4.3959999999999999,"15min":4.3959999999999999},"last":0.58699999999999997}]},{"osd":5,"last update":"Fri Feb 2 15:26:24 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.63200000000000001,"15min":0.59599999999999997},"min":{"1min":0.59099999999999997,"5min":0.374,"15min":0.374},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.626,"15min":0.60799999999999998},"min":{"1min":0.53600000000000003,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.81599999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.78300000000000003}]},{"osd":6,"last update":"Fri Feb 2 15:26:44 2024","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.69899999999999995,"15min":0.65300000000000002},"min":{"1min":0.45000000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":5.6840000000000002,"5min":5.6840000000000002,"15min":5.6840000000000002},"last":0.66500000000000004},{"interface":"front","average":{"1min":1.2490000000000001,"5min":0.77800000000000002,"15min":0.68300000000000005},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":5.9100000000000001,"5min":5.9100000000000001,"15min":5.9100000000000001},"last":0.67400000000000004}]},{"osd":7,"last update":"Fri Feb 2 15:26:00 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.497,"15min":0.48499999999999999},"min":{"1min":0.34000000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.747,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.54500000000000004,"15min":0.53500000000000003},"min":{"1min":0.46200000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76000000000000001,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.60899999999999999}]},{"osd":8,"last update":"Fri Feb 2 15:26:19 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.66200000000000003,"15min":0.65400000000000003},"min":{"1min":0.55100000000000005,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.877,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.753,"5min":0.67700000000000005,"15min":0.65800000000000003},"min":{"1min":0.5,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.83499999999999996}]},{"osd":9,"last update":"Fri Feb 2 15:26:39 2024","interfaces":[{"interface":"back","average":{"1min":1.78,"5min":0.93000000000000005,"15min":0.79300000000000004},"min":{"1min":0.48799999999999999,"5min":0.441,"15min":0.441},"max":{"1min":20.126999999999999,"5min":20.126999999999999,"15min":20.126999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":1.5409999999999999,"5min":0.89200000000000002,"15min":0.77900000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":14.965999999999999,"5min":14.965999999999999,"15min":14.965999999999999},"last":0.76700000000000002}]},{"osd":10,"last update":"Fri Feb 2 15:25:57 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.67700000000000005,"15min":0.67300000000000004},"min":{"1min":0.48199999999999998,"5min":0.379,"15min":0.379},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.751},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.66800000000000004,"15min":0.66700000000000004},"min":{"1min":0.55200000000000005,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.79100000000000004,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.69999999999999996}]},{"osd":11,"last update":"Fri Feb 2 15:25:48 2024","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.82499999999999996}]}]},{"osd":5,"up_from":36,"seq":154618822712,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":1088701,"data_compressed":59875,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 15:26:23 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.53800000000000003,"15min":0.51800000000000002},"min":{"1min":0.30399999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.72399999999999998,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.505},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.54000000000000004,"15min":0.52600000000000002},"min":{"1min":0.33500000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.67100000000000004,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.52700000000000002}]},{"osd":1,"last update":"Fri Feb 2 15:26:23 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.56200000000000006,"15min":0.55000000000000004},"min":{"1min":0.318,"5min":0.252,"15min":0.252},"max":{"1min":0.69099999999999995,"5min":1.018,"15min":1.018},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.57699999999999996,"15min":0.55100000000000005},"min":{"1min":0.40100000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.78000000000000003,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.65100000000000002}]},{"osd":2,"last update":"Fri Feb 2 15:26:23 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.40699999999999997,"15min":0.38700000000000001},"min":{"1min":0.30599999999999999,"5min":0.215,"15min":0.215},"max":{"1min":0.59099999999999997,"5min":0.63,"15min":0.63},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.42199999999999999,"15min":0.40400000000000003},"min":{"1min":0.32900000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.64900000000000002,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.59999999999999998}]},{"osd":3,"last update":"Fri Feb 2 15:26:23 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.56000000000000005,"15min":0.52700000000000002},"min":{"1min":0.434,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.69899999999999995,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.58499999999999996,"15min":0.56699999999999995},"min":{"1min":0.44700000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.74399999999999999,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.76500000000000001}]},{"osd":4,"last update":"Fri Feb 2 15:26:23 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.59099999999999997,"15min":0.56699999999999995},"min":{"1min":0.45600000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.77200000000000002,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.59899999999999998,"15min":0.57999999999999996},"min":{"1min":0.42199999999999999,"5min":0.316,"15min":0.316},"max":{"1min":0.79900000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.89700000000000002}]},{"osd":6,"last update":"Fri Feb 2 15:26:42 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.628,"15min":0.63},"min":{"1min":0.47799999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.84299999999999997,"5min":0.878,"15min":0.878},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.629,"15min":0.63100000000000001},"min":{"1min":0.46500000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.75600000000000001,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.86499999999999999}]},{"osd":7,"last update":"Fri Feb 2 15:26:01 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.64200000000000002,"15min":0.626},"min":{"1min":0.52700000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.621,"15min":0.60899999999999999},"min":{"1min":0.49299999999999999,"5min":0.373,"15min":0.373},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.94999999999999996}]},{"osd":8,"last update":"Fri Feb 2 15:26:18 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.57299999999999995,"15min":0.56100000000000005},"min":{"1min":0.43099999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.84499999999999997,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.60199999999999998,"15min":0.59699999999999998},"min":{"1min":0.379,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.92700000000000005,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.83499999999999996}]},{"osd":9,"last update":"Fri Feb 2 15:26:38 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.69999999999999996,"15min":0.68600000000000005},"min":{"1min":0.52400000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.86499999999999999,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71599999999999997,"15min":0.70599999999999996},"min":{"1min":0.54500000000000004,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.86599999999999999,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.91400000000000003}]},{"osd":10,"last update":"Fri Feb 2 15:25:54 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.70899999999999996,"15min":0.69999999999999996},"min":{"1min":0.57699999999999996,"5min":0.497,"15min":0.497},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.72799999999999998,"15min":0.72299999999999998},"min":{"1min":0.54000000000000004,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1,"5min":1,"15min":1},"last":0.98399999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34572,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34572,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36211,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36211,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36211,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36211,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36211,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":68040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":68144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":63178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-02T15:26:46.241 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-02-02T15:26:46.241 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-02-02T15:26:46.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.475+0000 7fb3c7a00700 1 -- 172.21.15.1:0/3239199498 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3c0101f40 msgr2=0x7fb3c01023a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:46.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.475+0000 7fb3c7a00700 1 --2- 172.21.15.1:0/3239199498 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3c0101f40 0x7fb3c01023a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fb3a80099c0 tx=0x7fb3a8009cd0 comp rx=0 tx=0).stop 2024-02-02T15:26:46.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.476+0000 7fb3c7a00700 1 -- 172.21.15.1:0/3239199498 shutdown_connections 2024-02-02T15:26:46.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.476+0000 7fb3c7a00700 1 --2- 172.21.15.1:0/3239199498 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb3c01028e0 0x7fb3c0109190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:46.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.476+0000 7fb3c7a00700 1 --2- 172.21.15.1:0/3239199498 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3c0100d50 0x7fb3c0101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:46.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.476+0000 7fb3c7a00700 1 --2- 172.21.15.1:0/3239199498 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3c0101f40 0x7fb3c01023a0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:46.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.476+0000 7fb3c7a00700 1 -- 172.21.15.1:0/3239199498 >> 172.21.15.1:0/3239199498 conn(0x7fb3c00fc2c0 msgr2=0x7fb3c00fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:46.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.476+0000 7fb3c7a00700 1 -- 172.21.15.1:0/3239199498 shutdown_connections 2024-02-02T15:26:46.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.476+0000 7fb3c7a00700 1 -- 172.21.15.1:0/3239199498 wait complete. 2024-02-02T15:26:46.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.477+0000 7fb3c7a00700 1 Processor -- start 2024-02-02T15:26:46.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.477+0000 7fb3c7a00700 1 -- start start 2024-02-02T15:26:46.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.478+0000 7fb3c7a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3c0100d50 0x7fb3c0196a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:46.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.478+0000 7fb3c7a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb3c0101f40 0x7fb3c0196f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:46.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.478+0000 7fb3c7a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3c01028e0 0x7fb3c019dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:46.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.478+0000 7fb3c7a00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c019e590 con 0x7fb3c01028e0 2024-02-02T15:26:46.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.478+0000 7fb3c7a00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c019e6d0 con 0x7fb3c0100d50 2024-02-02T15:26:46.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.478+0000 7fb3c7a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c019e860 con 0x7fb3c0101f40 2024-02-02T15:26:46.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.478+0000 7fb3c5a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3c01028e0 0x7fb3c019dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:46.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.478+0000 7fb3c5000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3c0100d50 0x7fb3c0196a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:46.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.478+0000 7fb3c5a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3c01028e0 0x7fb3c019dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53418/0 (socket says 172.21.15.1:53418) 2024-02-02T15:26:46.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.478+0000 7fb3c5a00700 1 -- 172.21.15.1:0/2781930454 learned_addr learned my addr 172.21.15.1:0/2781930454 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:46.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.478+0000 7fb3bfe00700 1 --2- 172.21.15.1:0/2781930454 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb3c0101f40 0x7fb3c0196f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:46.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.478+0000 7fb3c5a00700 1 -- 172.21.15.1:0/2781930454 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb3c0101f40 msgr2=0x7fb3c0196f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:46.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.479+0000 7fb3c5a00700 1 --2- 172.21.15.1:0/2781930454 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb3c0101f40 0x7fb3c0196f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:46.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.479+0000 7fb3c5a00700 1 -- 172.21.15.1:0/2781930454 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3c0100d50 msgr2=0x7fb3c0196a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:46.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.479+0000 7fb3c5a00700 1 --2- 172.21.15.1:0/2781930454 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3c0100d50 0x7fb3c0196a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:46.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.479+0000 7fb3c5a00700 1 -- 172.21.15.1:0/2781930454 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3a80095f0 con 0x7fb3c01028e0 2024-02-02T15:26:46.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.479+0000 7fb3c5a00700 1 --2- 172.21.15.1:0/2781930454 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3c01028e0 0x7fb3c019dfc0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fb3b800ef30 tx=0x7fb3b800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:46.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.479+0000 7fb3bd600700 1 -- 172.21.15.1:0/2781930454 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b8018070 con 0x7fb3c01028e0 2024-02-02T15:26:46.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.479+0000 7fb3c7a00700 1 -- 172.21.15.1:0/2781930454 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3c019eae0 con 0x7fb3c01028e0 2024-02-02T15:26:46.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.479+0000 7fb3bd600700 1 -- 172.21.15.1:0/2781930454 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb3b800f040 con 0x7fb3c01028e0 2024-02-02T15:26:46.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.479+0000 7fb3bd600700 1 -- 172.21.15.1:0/2781930454 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b800cca0 con 0x7fb3c01028e0 2024-02-02T15:26:46.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.479+0000 7fb3c7a00700 1 -- 172.21.15.1:0/2781930454 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3c019eff0 con 0x7fb3c01028e0 2024-02-02T15:26:46.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.480+0000 7fb3c7a00700 1 -- 172.21.15.1:0/2781930454 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3c0107cd0 con 0x7fb3c01028e0 2024-02-02T15:26:46.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.482+0000 7fb3bd600700 1 -- 172.21.15.1:0/2781930454 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) v1 ==== 135069+0+0 (secure 0 0 0) 0x7fb3b8003a00 con 0x7fb3c01028e0 2024-02-02T15:26:46.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.484+0000 7fb3bd600700 1 --2- 172.21.15.1:0/2781930454 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fb3a00a0520 0x7fb3a00a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:46.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.484+0000 7fb3bd600700 1 -- 172.21.15.1:0/2781930454 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fb3b80cb830 con 0x7fb3c01028e0 2024-02-02T15:26:46.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.484+0000 7fb3c5000700 1 --2- 172.21.15.1:0/2781930454 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fb3a00a0520 0x7fb3a00a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:46.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.484+0000 7fb3c5000700 1 --2- 172.21.15.1:0/2781930454 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fb3a00a0520 0x7fb3a00a29e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0101da0 tx=0x7fb3b0009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:46.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.488+0000 7fb3bd600700 1 -- 172.21.15.1:0/2781930454 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7fb3b8095090 con 0x7fb3c01028e0 2024-02-02T15:26:46.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:46 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/3609866608' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:26:46.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:46 smithi071 ceph-mon[44707]: from='client.15459 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-02T15:26:46.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:46 smithi071 ceph-mon[44707]: pgmap v298: 129 pgs: 129 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 9 op/s 2024-02-02T15:26:46.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:46 smithi071 ceph-mon[44707]: from='client.15465 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-02T15:26:46.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:46 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/3609866608' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:26:46.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:46 smithi142 ceph-mon[45728]: from='client.15459 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-02T15:26:46.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:46 smithi142 ceph-mon[45728]: pgmap v298: 129 pgs: 129 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 9 op/s 2024-02-02T15:26:46.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:46 smithi142 ceph-mon[45728]: from='client.15465 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-02T15:26:46.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:46 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/3609866608' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:26:46.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:46 smithi001 ceph-mon[35470]: from='client.15459 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-02T15:26:46.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:46 smithi001 ceph-mon[35470]: pgmap v298: 129 pgs: 129 active+clean; 643 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 9 op/s 2024-02-02T15:26:46.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:46 smithi001 ceph-mon[35470]: from='client.15465 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-02T15:26:46.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.698+0000 7fb3c7a00700 1 -- 172.21.15.1:0/2781930454 --> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb3c0061180 con 0x7fb3a00a0520 2024-02-02T15:26:46.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.712+0000 7fb3bd600700 1 -- 172.21.15.1:0/2781930454 <== mgr.14154 v2:172.21.15.1:6800/3768989981 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427911 (secure 0 0 0) 0x7fb3c0061180 con 0x7fb3a00a0520 2024-02-02T15:26:46.716 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:26:46.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.717+0000 7fb3c7a00700 1 -- 172.21.15.1:0/2781930454 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fb3a00a0520 msgr2=0x7fb3a00a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:46.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.717+0000 7fb3c7a00700 1 --2- 172.21.15.1:0/2781930454 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fb3a00a0520 0x7fb3a00a29e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0101da0 tx=0x7fb3b0009260 comp rx=0 tx=0).stop 2024-02-02T15:26:46.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.717+0000 7fb3c7a00700 1 -- 172.21.15.1:0/2781930454 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3c01028e0 msgr2=0x7fb3c019dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:46.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.717+0000 7fb3c7a00700 1 --2- 172.21.15.1:0/2781930454 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3c01028e0 0x7fb3c019dfc0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fb3b800ef30 tx=0x7fb3b800c3a0 comp rx=0 tx=0).stop 2024-02-02T15:26:46.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.717+0000 7fb3c7a00700 1 -- 172.21.15.1:0/2781930454 shutdown_connections 2024-02-02T15:26:46.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.717+0000 7fb3c7a00700 1 --2- 172.21.15.1:0/2781930454 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb3c01028e0 0x7fb3c019dfc0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:46.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.717+0000 7fb3c7a00700 1 --2- 172.21.15.1:0/2781930454 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb3c0100d50 0x7fb3c0196a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:46.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.717+0000 7fb3c7a00700 1 --2- 172.21.15.1:0/2781930454 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fb3a00a0520 0x7fb3a00a29e0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:46.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.717+0000 7fb3c7a00700 1 --2- 172.21.15.1:0/2781930454 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb3c0101f40 0x7fb3c0196f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:46.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.717+0000 7fb3c7a00700 1 -- 172.21.15.1:0/2781930454 >> 172.21.15.1:0/2781930454 conn(0x7fb3c00fc2c0 msgr2=0x7fb3c00fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:46.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.717+0000 7fb3c7a00700 1 -- 172.21.15.1:0/2781930454 shutdown_connections 2024-02-02T15:26:46.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.717+0000 7fb3c7a00700 1 -- 172.21.15.1:0/2781930454 wait complete. 2024-02-02T15:26:46.721 INFO:teuthology.orchestra.run.smithi001.stderr:dumped all 2024-02-02T15:26:46.735 INFO:teuthology.orchestra.run.smithi001.stdout:{"pg_ready":true,"pg_map":{"version":298,"stamp":"2024-02-02T15:26:45.935234+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658334,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1000,"num_read_kb":624,"num_write":431,"num_write_kb":3084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":1180804,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":855,"ondisk_log_size":855,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":336228,"kb_used_data":11972,"kb_used_omap":20,"kb_used_meta":324075,"kb_avail":1124605596,"statfs":{"total":1151940427776,"available":1151596130304,"internally_reserved":0,"allocated":12259328,"data_stored":7915039,"data_compressed":392601,"data_compressed_allocated":528384,"data_compressed_original":2555904,"omap_allocated":21372,"internal_metadata":331852932},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":312,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":111,"num_read_kb":51,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002994"},"pg_stats":[{"pgid":"3.3f","version":"86'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.379934+0000","last_change":"2024-02-02T15:24:47.793308+0000","last_active":"2024-02-02T15:26:27.379934+0000","last_peered":"2024-02-02T15:26:27.379934+0000","last_clean":"2024-02-02T15:26:27.379934+0000","last_became_active":"2024-02-02T15:24:47.792420+0000","last_became_peered":"2024-02-02T15:24:47.792420+0000","last_unstale":"2024-02-02T15:26:27.379934+0000","last_undegraded":"2024-02-02T15:26:27.379934+0000","last_fullsized":"2024-02-02T15:26:27.379934+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:39:09.871317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"86'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.486648+0000","last_change":"2024-02-02T15:24:47.791272+0000","last_active":"2024-02-02T15:26:27.486648+0000","last_peered":"2024-02-02T15:26:27.486648+0000","last_clean":"2024-02-02T15:26:27.486648+0000","last_became_active":"2024-02-02T15:24:47.790902+0000","last_became_peered":"2024-02-02T15:24:47.790902+0000","last_unstale":"2024-02-02T15:26:27.486648+0000","last_undegraded":"2024-02-02T15:26:27.486648+0000","last_fullsized":"2024-02-02T15:26:27.486648+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:31:11.620873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"86'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.486605+0000","last_change":"2024-02-02T15:24:47.798712+0000","last_active":"2024-02-02T15:26:27.486605+0000","last_peered":"2024-02-02T15:26:27.486605+0000","last_clean":"2024-02-02T15:26:27.486605+0000","last_became_active":"2024-02-02T15:24:47.798405+0000","last_became_peered":"2024-02-02T15:24:47.798405+0000","last_unstale":"2024-02-02T15:26:27.486605+0000","last_undegraded":"2024-02-02T15:26:27.486605+0000","last_fullsized":"2024-02-02T15:26:27.486605+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:37:15.238070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"86'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.280412+0000","last_change":"2024-02-02T15:24:47.786456+0000","last_active":"2024-02-02T15:26:27.280412+0000","last_peered":"2024-02-02T15:26:27.280412+0000","last_clean":"2024-02-02T15:26:27.280412+0000","last_became_active":"2024-02-02T15:24:47.785520+0000","last_became_peered":"2024-02-02T15:24:47.785520+0000","last_unstale":"2024-02-02T15:26:27.280412+0000","last_undegraded":"2024-02-02T15:26:27.280412+0000","last_fullsized":"2024-02-02T15:26:27.280412+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:16:04.003459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"86'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.274132+0000","last_change":"2024-02-02T15:24:47.780084+0000","last_active":"2024-02-02T15:26:27.274132+0000","last_peered":"2024-02-02T15:26:27.274132+0000","last_clean":"2024-02-02T15:26:27.274132+0000","last_became_active":"2024-02-02T15:24:47.779667+0000","last_became_peered":"2024-02-02T15:24:47.779667+0000","last_unstale":"2024-02-02T15:26:27.274132+0000","last_undegraded":"2024-02-02T15:26:27.274132+0000","last_fullsized":"2024-02-02T15:26:27.274132+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:30:15.377691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"86'6","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.362466+0000","last_change":"2024-02-02T15:24:47.786173+0000","last_active":"2024-02-02T15:26:27.362466+0000","last_peered":"2024-02-02T15:26:27.362466+0000","last_clean":"2024-02-02T15:26:27.362466+0000","last_became_active":"2024-02-02T15:24:47.785301+0000","last_became_peered":"2024-02-02T15:24:47.785301+0000","last_unstale":"2024-02-02T15:26:27.362466+0000","last_undegraded":"2024-02-02T15:26:27.362466+0000","last_fullsized":"2024-02-02T15:26:27.362466+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:14:58.524317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"86'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.273751+0000","last_change":"2024-02-02T15:24:47.790913+0000","last_active":"2024-02-02T15:26:27.273751+0000","last_peered":"2024-02-02T15:26:27.273751+0000","last_clean":"2024-02-02T15:26:27.273751+0000","last_became_active":"2024-02-02T15:24:47.790566+0000","last_became_peered":"2024-02-02T15:24:47.790566+0000","last_unstale":"2024-02-02T15:26:27.273751+0000","last_undegraded":"2024-02-02T15:26:27.273751+0000","last_fullsized":"2024-02-02T15:26:27.273751+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:34:51.954830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"86'9","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.375616+0000","last_change":"2024-02-02T15:24:47.786465+0000","last_active":"2024-02-02T15:26:27.375616+0000","last_peered":"2024-02-02T15:26:27.375616+0000","last_clean":"2024-02-02T15:26:27.375616+0000","last_became_active":"2024-02-02T15:24:47.786175+0000","last_became_peered":"2024-02-02T15:24:47.786175+0000","last_unstale":"2024-02-02T15:26:27.375616+0000","last_undegraded":"2024-02-02T15:26:27.375616+0000","last_fullsized":"2024-02-02T15:26:27.375616+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:42:13.748850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"86'11","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.277236+0000","last_change":"2024-02-02T15:24:47.787115+0000","last_active":"2024-02-02T15:26:27.277236+0000","last_peered":"2024-02-02T15:26:27.277236+0000","last_clean":"2024-02-02T15:26:27.277236+0000","last_became_active":"2024-02-02T15:24:47.786804+0000","last_became_peered":"2024-02-02T15:24:47.786804+0000","last_unstale":"2024-02-02T15:26:27.277236+0000","last_undegraded":"2024-02-02T15:26:27.277236+0000","last_fullsized":"2024-02-02T15:26:27.277236+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T03:06:17.873243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"86'16","reported_seq":35,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.362900+0000","last_change":"2024-02-02T15:24:47.787211+0000","last_active":"2024-02-02T15:26:27.362900+0000","last_peered":"2024-02-02T15:26:27.362900+0000","last_clean":"2024-02-02T15:26:27.362900+0000","last_became_active":"2024-02-02T15:24:47.786888+0000","last_became_peered":"2024-02-02T15:24:47.786888+0000","last_unstale":"2024-02-02T15:26:27.362900+0000","last_undegraded":"2024-02-02T15:26:27.362900+0000","last_fullsized":"2024-02-02T15:26:27.362900+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:14:21.442787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"86'6","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.274294+0000","last_change":"2024-02-02T15:24:47.798962+0000","last_active":"2024-02-02T15:26:27.274294+0000","last_peered":"2024-02-02T15:26:27.274294+0000","last_clean":"2024-02-02T15:26:27.274294+0000","last_became_active":"2024-02-02T15:24:47.798657+0000","last_became_peered":"2024-02-02T15:24:47.798657+0000","last_unstale":"2024-02-02T15:26:27.274294+0000","last_undegraded":"2024-02-02T15:26:27.274294+0000","last_fullsized":"2024-02-02T15:26:27.274294+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:53:37.316027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"86'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.280289+0000","last_change":"2024-02-02T15:24:47.782647+0000","last_active":"2024-02-02T15:26:27.280289+0000","last_peered":"2024-02-02T15:26:27.280289+0000","last_clean":"2024-02-02T15:26:27.280289+0000","last_became_active":"2024-02-02T15:24:47.782344+0000","last_became_peered":"2024-02-02T15:24:47.782344+0000","last_unstale":"2024-02-02T15:26:27.280289+0000","last_undegraded":"2024-02-02T15:26:27.280289+0000","last_fullsized":"2024-02-02T15:26:27.280289+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:06:54.874657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"86'10","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.269115+0000","last_change":"2024-02-02T15:24:47.787032+0000","last_active":"2024-02-02T15:26:27.269115+0000","last_peered":"2024-02-02T15:26:27.269115+0000","last_clean":"2024-02-02T15:26:27.269115+0000","last_became_active":"2024-02-02T15:24:47.786710+0000","last_became_peered":"2024-02-02T15:24:47.786710+0000","last_unstale":"2024-02-02T15:26:27.269115+0000","last_undegraded":"2024-02-02T15:26:27.269115+0000","last_fullsized":"2024-02-02T15:26:27.269115+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:26:15.429316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"86'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.486412+0000","last_change":"2024-02-02T15:24:47.785708+0000","last_active":"2024-02-02T15:26:27.486412+0000","last_peered":"2024-02-02T15:26:27.486412+0000","last_clean":"2024-02-02T15:26:27.486412+0000","last_became_active":"2024-02-02T15:24:47.785280+0000","last_became_peered":"2024-02-02T15:24:47.785280+0000","last_unstale":"2024-02-02T15:26:27.486412+0000","last_undegraded":"2024-02-02T15:26:27.486412+0000","last_fullsized":"2024-02-02T15:26:27.486412+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T02:21:23.224933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"86'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.486347+0000","last_change":"2024-02-02T15:24:47.783904+0000","last_active":"2024-02-02T15:26:27.486347+0000","last_peered":"2024-02-02T15:26:27.486347+0000","last_clean":"2024-02-02T15:26:27.486347+0000","last_became_active":"2024-02-02T15:24:47.783599+0000","last_became_peered":"2024-02-02T15:24:47.783599+0000","last_unstale":"2024-02-02T15:26:27.486347+0000","last_undegraded":"2024-02-02T15:26:27.486347+0000","last_fullsized":"2024-02-02T15:26:27.486347+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:35:45.805527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"86'9","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.276951+0000","last_change":"2024-02-02T15:24:47.795770+0000","last_active":"2024-02-02T15:26:27.276951+0000","last_peered":"2024-02-02T15:26:27.276951+0000","last_clean":"2024-02-02T15:26:27.276951+0000","last_became_active":"2024-02-02T15:24:47.795433+0000","last_became_peered":"2024-02-02T15:24:47.795433+0000","last_unstale":"2024-02-02T15:26:27.276951+0000","last_undegraded":"2024-02-02T15:26:27.276951+0000","last_fullsized":"2024-02-02T15:26:27.276951+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:50:07.496998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"86'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.379835+0000","last_change":"2024-02-02T15:24:47.793684+0000","last_active":"2024-02-02T15:26:27.379835+0000","last_peered":"2024-02-02T15:26:27.379835+0000","last_clean":"2024-02-02T15:26:27.379835+0000","last_became_active":"2024-02-02T15:24:47.792853+0000","last_became_peered":"2024-02-02T15:24:47.792853+0000","last_unstale":"2024-02-02T15:26:27.379835+0000","last_undegraded":"2024-02-02T15:26:27.379835+0000","last_fullsized":"2024-02-02T15:26:27.379835+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:30:40.379379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"86'10","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.280363+0000","last_change":"2024-02-02T15:24:47.780935+0000","last_active":"2024-02-02T15:26:27.280363+0000","last_peered":"2024-02-02T15:26:27.280363+0000","last_clean":"2024-02-02T15:26:27.280363+0000","last_became_active":"2024-02-02T15:24:47.780642+0000","last_became_peered":"2024-02-02T15:24:47.780642+0000","last_unstale":"2024-02-02T15:26:27.280363+0000","last_undegraded":"2024-02-02T15:26:27.280363+0000","last_fullsized":"2024-02-02T15:26:27.280363+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:36:37.615368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"86'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.380235+0000","last_change":"2024-02-02T15:24:47.791051+0000","last_active":"2024-02-02T15:26:27.380235+0000","last_peered":"2024-02-02T15:26:27.380235+0000","last_clean":"2024-02-02T15:26:27.380235+0000","last_became_active":"2024-02-02T15:24:47.790732+0000","last_became_peered":"2024-02-02T15:24:47.790732+0000","last_unstale":"2024-02-02T15:26:27.380235+0000","last_undegraded":"2024-02-02T15:26:27.380235+0000","last_fullsized":"2024-02-02T15:26:27.380235+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:45:05.155144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.380085+0000","last_change":"2024-02-02T15:24:51.814699+0000","last_active":"2024-02-02T15:26:27.380085+0000","last_peered":"2024-02-02T15:26:27.380085+0000","last_clean":"2024-02-02T15:26:27.380085+0000","last_became_active":"2024-02-02T15:24:51.814251+0000","last_became_peered":"2024-02-02T15:24:51.814251+0000","last_unstale":"2024-02-02T15:26:27.380085+0000","last_undegraded":"2024-02-02T15:26:27.380085+0000","last_fullsized":"2024-02-02T15:26:27.380085+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:27:21.674922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"84'13","reported_seq":118,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-02T15:26:42.393747+0000","last_change":"2024-02-02T15:24:47.269429+0000","last_active":"2024-02-02T15:26:42.393747+0000","last_peered":"2024-02-02T15:26:42.393747+0000","last_clean":"2024-02-02T15:26:42.393747+0000","last_became_active":"2024-02-02T15:24:47.269023+0000","last_became_peered":"2024-02-02T15:24:47.269023+0000","last_unstale":"2024-02-02T15:26:42.393747+0000","last_undegraded":"2024-02-02T15:26:42.393747+0000","last_fullsized":"2024-02-02T15:26:42.393747+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T03:11:46.344438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":85,"num_read_kb":85,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"86'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.376226+0000","last_change":"2024-02-02T15:24:47.785900+0000","last_active":"2024-02-02T15:26:27.376226+0000","last_peered":"2024-02-02T15:26:27.376226+0000","last_clean":"2024-02-02T15:26:27.376226+0000","last_became_active":"2024-02-02T15:24:47.781994+0000","last_became_peered":"2024-02-02T15:24:47.781994+0000","last_unstale":"2024-02-02T15:26:27.376226+0000","last_undegraded":"2024-02-02T15:26:27.376226+0000","last_fullsized":"2024-02-02T15:26:27.376226+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:00:52.432637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.279297+0000","last_change":"2024-02-02T15:24:51.825655+0000","last_active":"2024-02-02T15:26:27.279297+0000","last_peered":"2024-02-02T15:26:27.279297+0000","last_clean":"2024-02-02T15:26:27.279297+0000","last_became_active":"2024-02-02T15:24:51.825199+0000","last_became_peered":"2024-02-02T15:24:51.825199+0000","last_unstale":"2024-02-02T15:26:27.279297+0000","last_undegraded":"2024-02-02T15:26:27.279297+0000","last_fullsized":"2024-02-02T15:26:27.279297+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:50:32.283394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"84'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.376399+0000","last_change":"2024-02-02T15:24:47.281869+0000","last_active":"2024-02-02T15:26:27.376399+0000","last_peered":"2024-02-02T15:26:27.376399+0000","last_clean":"2024-02-02T15:26:27.376399+0000","last_became_active":"2024-02-02T15:24:47.280825+0000","last_became_peered":"2024-02-02T15:24:47.280825+0000","last_unstale":"2024-02-02T15:26:27.376399+0000","last_undegraded":"2024-02-02T15:26:27.376399+0000","last_fullsized":"2024-02-02T15:26:27.376399+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:24:19.688389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"86'6","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.486649+0000","last_change":"2024-02-02T15:24:47.795893+0000","last_active":"2024-02-02T15:26:27.486649+0000","last_peered":"2024-02-02T15:26:27.486649+0000","last_clean":"2024-02-02T15:26:27.486649+0000","last_became_active":"2024-02-02T15:24:47.795491+0000","last_became_peered":"2024-02-02T15:24:47.795491+0000","last_unstale":"2024-02-02T15:26:27.486649+0000","last_undegraded":"2024-02-02T15:26:27.486649+0000","last_fullsized":"2024-02-02T15:26:27.486649+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:54:38.923036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.279462+0000","last_change":"2024-02-02T15:24:51.826980+0000","last_active":"2024-02-02T15:26:27.279462+0000","last_peered":"2024-02-02T15:26:27.279462+0000","last_clean":"2024-02-02T15:26:27.279462+0000","last_became_active":"2024-02-02T15:24:51.826221+0000","last_became_peered":"2024-02-02T15:24:51.826221+0000","last_unstale":"2024-02-02T15:26:27.279462+0000","last_undegraded":"2024-02-02T15:26:27.279462+0000","last_fullsized":"2024-02-02T15:26:27.279462+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:45:17.126174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"84'12","reported_seq":33,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.274623+0000","last_change":"2024-02-02T15:24:47.272219+0000","last_active":"2024-02-02T15:26:27.274623+0000","last_peered":"2024-02-02T15:26:27.274623+0000","last_clean":"2024-02-02T15:26:27.274623+0000","last_became_active":"2024-02-02T15:24:47.271899+0000","last_became_peered":"2024-02-02T15:24:47.271899+0000","last_unstale":"2024-02-02T15:26:27.274623+0000","last_undegraded":"2024-02-02T15:26:27.274623+0000","last_fullsized":"2024-02-02T15:26:27.274623+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:44:27.370783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"86'9","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.277362+0000","last_change":"2024-02-02T15:24:47.783434+0000","last_active":"2024-02-02T15:26:27.277362+0000","last_peered":"2024-02-02T15:26:27.277362+0000","last_clean":"2024-02-02T15:26:27.277362+0000","last_became_active":"2024-02-02T15:24:47.782130+0000","last_became_peered":"2024-02-02T15:24:47.782130+0000","last_unstale":"2024-02-02T15:26:27.277362+0000","last_undegraded":"2024-02-02T15:26:27.277362+0000","last_fullsized":"2024-02-02T15:26:27.277362+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:32:25.981913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.279554+0000","last_change":"2024-02-02T15:24:51.810791+0000","last_active":"2024-02-02T15:26:27.279554+0000","last_peered":"2024-02-02T15:26:27.279554+0000","last_clean":"2024-02-02T15:26:27.279554+0000","last_became_active":"2024-02-02T15:24:51.810379+0000","last_became_peered":"2024-02-02T15:24:51.810379+0000","last_unstale":"2024-02-02T15:26:27.279554+0000","last_undegraded":"2024-02-02T15:26:27.279554+0000","last_fullsized":"2024-02-02T15:26:27.279554+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:49:44.635820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'5","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.274488+0000","last_change":"2024-02-02T15:24:47.271225+0000","last_active":"2024-02-02T15:26:27.274488+0000","last_peered":"2024-02-02T15:26:27.274488+0000","last_clean":"2024-02-02T15:26:27.274488+0000","last_became_active":"2024-02-02T15:24:47.270870+0000","last_became_peered":"2024-02-02T15:24:47.270870+0000","last_unstale":"2024-02-02T15:26:27.274488+0000","last_undegraded":"2024-02-02T15:26:27.274488+0000","last_fullsized":"2024-02-02T15:26:27.274488+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:16:55.659950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"86'15","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-02T15:26:28.281479+0000","last_change":"2024-02-02T15:26:28.281073+0000","last_active":"2024-02-02T15:26:28.281479+0000","last_peered":"2024-02-02T15:26:28.281479+0000","last_clean":"2024-02-02T15:26:28.281479+0000","last_became_active":"2024-02-02T15:26:28.280680+0000","last_became_peered":"2024-02-02T15:26:28.280680+0000","last_unstale":"2024-02-02T15:26:28.281479+0000","last_undegraded":"2024-02-02T15:26:28.281479+0000","last_fullsized":"2024-02-02T15:26:28.281479+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:53:34.526394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.379201+0000","last_change":"2024-02-02T15:24:51.807328+0000","last_active":"2024-02-02T15:26:27.379201+0000","last_peered":"2024-02-02T15:26:27.379201+0000","last_clean":"2024-02-02T15:26:27.379201+0000","last_became_active":"2024-02-02T15:24:51.806853+0000","last_became_peered":"2024-02-02T15:24:51.806853+0000","last_unstale":"2024-02-02T15:26:27.379201+0000","last_undegraded":"2024-02-02T15:26:27.379201+0000","last_fullsized":"2024-02-02T15:26:27.379201+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:46:13.898450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"84'5","reported_seq":113,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-02T15:26:44.392961+0000","last_change":"2024-02-02T15:24:47.268192+0000","last_active":"2024-02-02T15:26:44.392961+0000","last_peered":"2024-02-02T15:26:44.392961+0000","last_clean":"2024-02-02T15:26:44.392961+0000","last_became_active":"2024-02-02T15:24:47.267796+0000","last_became_peered":"2024-02-02T15:24:47.267796+0000","last_unstale":"2024-02-02T15:26:44.392961+0000","last_undegraded":"2024-02-02T15:26:44.392961+0000","last_fullsized":"2024-02-02T15:26:44.392961+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:12:13.161690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":88,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"86'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.274477+0000","last_change":"2024-02-02T15:24:47.782976+0000","last_active":"2024-02-02T15:26:27.274477+0000","last_peered":"2024-02-02T15:26:27.274477+0000","last_clean":"2024-02-02T15:26:27.274477+0000","last_became_active":"2024-02-02T15:24:47.782640+0000","last_became_peered":"2024-02-02T15:24:47.782640+0000","last_unstale":"2024-02-02T15:26:27.274477+0000","last_undegraded":"2024-02-02T15:26:27.274477+0000","last_fullsized":"2024-02-02T15:26:27.274477+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T02:09:54.851418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.379834+0000","last_change":"2024-02-02T15:24:51.814799+0000","last_active":"2024-02-02T15:26:27.379834+0000","last_peered":"2024-02-02T15:26:27.379834+0000","last_clean":"2024-02-02T15:26:27.379834+0000","last_became_active":"2024-02-02T15:24:51.814263+0000","last_became_peered":"2024-02-02T15:24:51.814263+0000","last_unstale":"2024-02-02T15:26:27.379834+0000","last_undegraded":"2024-02-02T15:26:27.379834+0000","last_fullsized":"2024-02-02T15:26:27.379834+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:52:02.786496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"84'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.362424+0000","last_change":"2024-02-02T15:24:47.273204+0000","last_active":"2024-02-02T15:26:27.362424+0000","last_peered":"2024-02-02T15:26:27.362424+0000","last_clean":"2024-02-02T15:26:27.362424+0000","last_became_active":"2024-02-02T15:24:47.272888+0000","last_became_peered":"2024-02-02T15:24:47.272888+0000","last_unstale":"2024-02-02T15:26:27.362424+0000","last_undegraded":"2024-02-02T15:26:27.362424+0000","last_fullsized":"2024-02-02T15:26:27.362424+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:26:25.188915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"86'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.281811+0000","last_change":"2024-02-02T15:24:47.788645+0000","last_active":"2024-02-02T15:26:27.281811+0000","last_peered":"2024-02-02T15:26:27.281811+0000","last_clean":"2024-02-02T15:26:27.281811+0000","last_became_active":"2024-02-02T15:24:47.787478+0000","last_became_peered":"2024-02-02T15:24:47.787478+0000","last_unstale":"2024-02-02T15:26:27.281811+0000","last_undegraded":"2024-02-02T15:26:27.281811+0000","last_fullsized":"2024-02-02T15:26:27.281811+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:24:45.327030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.379732+0000","last_change":"2024-02-02T15:24:51.822681+0000","last_active":"2024-02-02T15:26:27.379732+0000","last_peered":"2024-02-02T15:26:27.379732+0000","last_clean":"2024-02-02T15:26:27.379732+0000","last_became_active":"2024-02-02T15:24:51.822202+0000","last_became_peered":"2024-02-02T15:24:51.822202+0000","last_unstale":"2024-02-02T15:26:27.379732+0000","last_undegraded":"2024-02-02T15:26:27.379732+0000","last_fullsized":"2024-02-02T15:26:27.379732+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:05:43.751558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"84'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.268759+0000","last_change":"2024-02-02T15:24:47.271095+0000","last_active":"2024-02-02T15:26:27.268759+0000","last_peered":"2024-02-02T15:26:27.268759+0000","last_clean":"2024-02-02T15:26:27.268759+0000","last_became_active":"2024-02-02T15:24:47.270696+0000","last_became_peered":"2024-02-02T15:24:47.270696+0000","last_unstale":"2024-02-02T15:26:27.268759+0000","last_undegraded":"2024-02-02T15:26:27.268759+0000","last_fullsized":"2024-02-02T15:26:27.268759+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T02:20:59.370217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"86'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.375968+0000","last_change":"2024-02-02T15:24:47.791699+0000","last_active":"2024-02-02T15:26:27.375968+0000","last_peered":"2024-02-02T15:26:27.375968+0000","last_clean":"2024-02-02T15:26:27.375968+0000","last_became_active":"2024-02-02T15:24:47.787280+0000","last_became_peered":"2024-02-02T15:24:47.787280+0000","last_unstale":"2024-02-02T15:26:27.375968+0000","last_undegraded":"2024-02-02T15:26:27.375968+0000","last_fullsized":"2024-02-02T15:26:27.375968+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:18:10.316393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.486617+0000","last_change":"2024-02-02T15:24:51.810050+0000","last_active":"2024-02-02T15:26:27.486617+0000","last_peered":"2024-02-02T15:26:27.486617+0000","last_clean":"2024-02-02T15:26:27.486617+0000","last_became_active":"2024-02-02T15:24:51.809621+0000","last_became_peered":"2024-02-02T15:24:51.809621+0000","last_unstale":"2024-02-02T15:26:27.486617+0000","last_undegraded":"2024-02-02T15:26:27.486617+0000","last_fullsized":"2024-02-02T15:26:27.486617+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:52:35.885973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"84'3","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.281444+0000","last_change":"2024-02-02T15:24:47.267973+0000","last_active":"2024-02-02T15:26:27.281444+0000","last_peered":"2024-02-02T15:26:27.281444+0000","last_clean":"2024-02-02T15:26:27.281444+0000","last_became_active":"2024-02-02T15:24:47.267649+0000","last_became_peered":"2024-02-02T15:24:47.267649+0000","last_unstale":"2024-02-02T15:26:27.281444+0000","last_undegraded":"2024-02-02T15:26:27.281444+0000","last_fullsized":"2024-02-02T15:26:27.281444+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T02:46:56.844570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"86'11","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.279736+0000","last_change":"2024-02-02T15:24:47.798636+0000","last_active":"2024-02-02T15:26:27.279736+0000","last_peered":"2024-02-02T15:26:27.279736+0000","last_clean":"2024-02-02T15:26:27.279736+0000","last_became_active":"2024-02-02T15:24:47.798328+0000","last_became_peered":"2024-02-02T15:24:47.798328+0000","last_unstale":"2024-02-02T15:26:27.279736+0000","last_undegraded":"2024-02-02T15:26:27.279736+0000","last_fullsized":"2024-02-02T15:26:27.279736+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:37:43.687336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.375711+0000","last_change":"2024-02-02T15:24:51.811312+0000","last_active":"2024-02-02T15:26:27.375711+0000","last_peered":"2024-02-02T15:26:27.375711+0000","last_clean":"2024-02-02T15:26:27.375711+0000","last_became_active":"2024-02-02T15:24:51.810932+0000","last_became_peered":"2024-02-02T15:24:51.810932+0000","last_unstale":"2024-02-02T15:26:27.375711+0000","last_undegraded":"2024-02-02T15:26:27.375711+0000","last_fullsized":"2024-02-02T15:26:27.375711+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:10:31.530518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"84'4","reported_seq":186,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-02T15:26:40.386290+0000","last_change":"2024-02-02T15:24:47.271483+0000","last_active":"2024-02-02T15:26:40.386290+0000","last_peered":"2024-02-02T15:26:40.386290+0000","last_clean":"2024-02-02T15:26:40.386290+0000","last_became_active":"2024-02-02T15:24:47.271160+0000","last_became_peered":"2024-02-02T15:24:47.271160+0000","last_unstale":"2024-02-02T15:26:40.386290+0000","last_undegraded":"2024-02-02T15:26:40.386290+0000","last_fullsized":"2024-02-02T15:26:40.386290+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:29:20.135351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":162,"num_read_kb":162,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"86'10","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.277538+0000","last_change":"2024-02-02T15:24:47.786492+0000","last_active":"2024-02-02T15:26:27.277538+0000","last_peered":"2024-02-02T15:26:27.277538+0000","last_clean":"2024-02-02T15:26:27.277538+0000","last_became_active":"2024-02-02T15:24:47.786103+0000","last_became_peered":"2024-02-02T15:24:47.786103+0000","last_unstale":"2024-02-02T15:26:27.277538+0000","last_undegraded":"2024-02-02T15:26:27.277538+0000","last_fullsized":"2024-02-02T15:26:27.277538+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:24:23.666596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.279580+0000","last_change":"2024-02-02T15:24:51.819941+0000","last_active":"2024-02-02T15:26:27.279580+0000","last_peered":"2024-02-02T15:26:27.279580+0000","last_clean":"2024-02-02T15:26:27.279580+0000","last_became_active":"2024-02-02T15:24:51.818066+0000","last_became_peered":"2024-02-02T15:24:51.818066+0000","last_unstale":"2024-02-02T15:26:27.279580+0000","last_undegraded":"2024-02-02T15:26:27.279580+0000","last_fullsized":"2024-02-02T15:26:27.279580+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:55:51.550988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"84'4","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.274167+0000","last_change":"2024-02-02T15:24:47.268541+0000","last_active":"2024-02-02T15:26:27.274167+0000","last_peered":"2024-02-02T15:26:27.274167+0000","last_clean":"2024-02-02T15:26:27.274167+0000","last_became_active":"2024-02-02T15:24:47.268184+0000","last_became_peered":"2024-02-02T15:24:47.268184+0000","last_unstale":"2024-02-02T15:26:27.274167+0000","last_undegraded":"2024-02-02T15:26:27.274167+0000","last_fullsized":"2024-02-02T15:26:27.274167+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:30:12.187297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"86'9","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.486650+0000","last_change":"2024-02-02T15:24:47.787792+0000","last_active":"2024-02-02T15:26:27.486650+0000","last_peered":"2024-02-02T15:26:27.486650+0000","last_clean":"2024-02-02T15:26:27.486650+0000","last_became_active":"2024-02-02T15:24:47.787479+0000","last_became_peered":"2024-02-02T15:24:47.787479+0000","last_unstale":"2024-02-02T15:26:27.486650+0000","last_undegraded":"2024-02-02T15:26:27.486650+0000","last_fullsized":"2024-02-02T15:26:27.486650+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T03:24:06.544232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.279743+0000","last_change":"2024-02-02T15:24:51.826639+0000","last_active":"2024-02-02T15:26:27.279743+0000","last_peered":"2024-02-02T15:26:27.279743+0000","last_clean":"2024-02-02T15:26:27.279743+0000","last_became_active":"2024-02-02T15:24:51.825089+0000","last_became_peered":"2024-02-02T15:24:51.825089+0000","last_unstale":"2024-02-02T15:26:27.279743+0000","last_undegraded":"2024-02-02T15:26:27.279743+0000","last_fullsized":"2024-02-02T15:26:27.279743+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:41:44.592603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"84'5","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.281688+0000","last_change":"2024-02-02T15:24:47.281490+0000","last_active":"2024-02-02T15:26:27.281688+0000","last_peered":"2024-02-02T15:26:27.281688+0000","last_clean":"2024-02-02T15:26:27.281688+0000","last_became_active":"2024-02-02T15:24:47.281137+0000","last_became_peered":"2024-02-02T15:24:47.281137+0000","last_unstale":"2024-02-02T15:26:27.281688+0000","last_undegraded":"2024-02-02T15:26:27.281688+0000","last_fullsized":"2024-02-02T15:26:27.281688+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:42:08.508881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"86'11","reported_seq":39,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-02T15:26:28.376134+0000","last_change":"2024-02-02T15:26:28.376134+0000","last_active":"2024-02-02T15:26:28.376134+0000","last_peered":"2024-02-02T15:26:28.376134+0000","last_clean":"2024-02-02T15:26:28.376134+0000","last_became_active":"2024-02-02T15:26:28.375708+0000","last_became_peered":"2024-02-02T15:26:28.375708+0000","last_unstale":"2024-02-02T15:26:28.376134+0000","last_undegraded":"2024-02-02T15:26:28.376134+0000","last_fullsized":"2024-02-02T15:26:28.376134+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:03:10.254243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,7],"acting":[1,8,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.279858+0000","last_change":"2024-02-02T15:25:27.286403+0000","last_active":"2024-02-02T15:26:27.279858+0000","last_peered":"2024-02-02T15:26:27.279858+0000","last_clean":"2024-02-02T15:26:27.279858+0000","last_became_active":"2024-02-02T15:25:27.285824+0000","last_became_peered":"2024-02-02T15:25:27.285824+0000","last_unstale":"2024-02-02T15:26:27.279858+0000","last_undegraded":"2024-02-02T15:26:27.279858+0000","last_fullsized":"2024-02-02T15:26:27.279858+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:11:25.986947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,10],"acting":[9,8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"88'39","reported_seq":231,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-02T15:26:40.386763+0000","last_change":"2024-02-02T15:24:47.270604+0000","last_active":"2024-02-02T15:26:40.386763+0000","last_peered":"2024-02-02T15:26:40.386763+0000","last_clean":"2024-02-02T15:26:40.386763+0000","last_became_active":"2024-02-02T15:24:47.270261+0000","last_became_peered":"2024-02-02T15:24:47.270261+0000","last_unstale":"2024-02-02T15:26:40.386763+0000","last_undegraded":"2024-02-02T15:26:40.386763+0000","last_fullsized":"2024-02-02T15:26:40.386763+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:56:33.767423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60957,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":172,"num_read_kb":147,"num_write":41,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"86'6","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.274315+0000","last_change":"2024-02-02T15:24:47.789663+0000","last_active":"2024-02-02T15:26:27.274315+0000","last_peered":"2024-02-02T15:26:27.274315+0000","last_clean":"2024-02-02T15:26:27.274315+0000","last_became_active":"2024-02-02T15:24:47.789352+0000","last_became_peered":"2024-02-02T15:24:47.789352+0000","last_unstale":"2024-02-02T15:26:27.274315+0000","last_undegraded":"2024-02-02T15:26:27.274315+0000","last_fullsized":"2024-02-02T15:26:27.274315+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:13:37.991350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.282133+0000","last_change":"2024-02-02T15:24:51.816021+0000","last_active":"2024-02-02T15:26:27.282133+0000","last_peered":"2024-02-02T15:26:27.282133+0000","last_clean":"2024-02-02T15:26:27.282133+0000","last_became_active":"2024-02-02T15:24:51.815698+0000","last_became_peered":"2024-02-02T15:24:51.815698+0000","last_unstale":"2024-02-02T15:26:27.282133+0000","last_undegraded":"2024-02-02T15:26:27.282133+0000","last_fullsized":"2024-02-02T15:26:27.282133+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:11:10.670195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"84'9","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.379371+0000","last_change":"2024-02-02T15:25:27.295339+0000","last_active":"2024-02-02T15:26:27.379371+0000","last_peered":"2024-02-02T15:26:27.379371+0000","last_clean":"2024-02-02T15:26:27.379371+0000","last_became_active":"2024-02-02T15:25:27.281605+0000","last_became_peered":"2024-02-02T15:25:27.281605+0000","last_unstale":"2024-02-02T15:26:27.379371+0000","last_undegraded":"2024-02-02T15:26:27.379371+0000","last_fullsized":"2024-02-02T15:26:27.379371+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T02:35:36.952261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"86'9","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.268962+0000","last_change":"2024-02-02T15:24:47.798089+0000","last_active":"2024-02-02T15:26:27.268962+0000","last_peered":"2024-02-02T15:26:27.268962+0000","last_clean":"2024-02-02T15:26:27.268962+0000","last_became_active":"2024-02-02T15:24:47.793914+0000","last_became_peered":"2024-02-02T15:24:47.793914+0000","last_unstale":"2024-02-02T15:26:27.268962+0000","last_undegraded":"2024-02-02T15:26:27.268962+0000","last_fullsized":"2024-02-02T15:26:27.268962+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:33:39.927550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.379433+0000","last_change":"2024-02-02T15:24:51.823098+0000","last_active":"2024-02-02T15:26:27.379433+0000","last_peered":"2024-02-02T15:26:27.379433+0000","last_clean":"2024-02-02T15:26:27.379433+0000","last_became_active":"2024-02-02T15:24:51.822634+0000","last_became_peered":"2024-02-02T15:24:51.822634+0000","last_unstale":"2024-02-02T15:26:27.379433+0000","last_undegraded":"2024-02-02T15:26:27.379433+0000","last_fullsized":"2024-02-02T15:26:27.379433+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T03:18:07.420196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"84'5","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.281450+0000","last_change":"2024-02-02T15:24:47.268962+0000","last_active":"2024-02-02T15:26:27.281450+0000","last_peered":"2024-02-02T15:26:27.281450+0000","last_clean":"2024-02-02T15:26:27.281450+0000","last_became_active":"2024-02-02T15:24:47.268607+0000","last_became_peered":"2024-02-02T15:24:47.268607+0000","last_unstale":"2024-02-02T15:26:27.281450+0000","last_undegraded":"2024-02-02T15:26:27.281450+0000","last_fullsized":"2024-02-02T15:26:27.281450+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:31:06.330437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"86'15","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.273962+0000","last_change":"2024-02-02T15:24:47.792205+0000","last_active":"2024-02-02T15:26:27.273962+0000","last_peered":"2024-02-02T15:26:27.273962+0000","last_clean":"2024-02-02T15:26:27.273962+0000","last_became_active":"2024-02-02T15:24:47.791874+0000","last_became_peered":"2024-02-02T15:24:47.791874+0000","last_unstale":"2024-02-02T15:26:27.273962+0000","last_undegraded":"2024-02-02T15:26:27.273962+0000","last_fullsized":"2024-02-02T15:26:27.273962+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:33:38.795921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.280422+0000","last_change":"2024-02-02T15:24:51.825499+0000","last_active":"2024-02-02T15:26:27.280422+0000","last_peered":"2024-02-02T15:26:27.280422+0000","last_clean":"2024-02-02T15:26:27.280422+0000","last_became_active":"2024-02-02T15:24:51.825049+0000","last_became_peered":"2024-02-02T15:24:51.825049+0000","last_unstale":"2024-02-02T15:26:27.280422+0000","last_undegraded":"2024-02-02T15:26:27.280422+0000","last_fullsized":"2024-02-02T15:26:27.280422+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:41:05.430891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"86'7","reported_seq":115,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-02T15:26:43.404328+0000","last_change":"2024-02-02T15:24:47.267683+0000","last_active":"2024-02-02T15:26:43.404328+0000","last_peered":"2024-02-02T15:26:43.404328+0000","last_clean":"2024-02-02T15:26:43.404328+0000","last_became_active":"2024-02-02T15:24:47.267324+0000","last_became_peered":"2024-02-02T15:24:47.267324+0000","last_unstale":"2024-02-02T15:26:43.404328+0000","last_undegraded":"2024-02-02T15:26:43.404328+0000","last_fullsized":"2024-02-02T15:26:43.404328+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:03:31.326791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1073,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":89,"num_read_kb":4,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"86'6","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.274103+0000","last_change":"2024-02-02T15:24:47.792509+0000","last_active":"2024-02-02T15:26:27.274103+0000","last_peered":"2024-02-02T15:26:27.274103+0000","last_clean":"2024-02-02T15:26:27.274103+0000","last_became_active":"2024-02-02T15:24:47.791662+0000","last_became_peered":"2024-02-02T15:24:47.791662+0000","last_unstale":"2024-02-02T15:26:27.274103+0000","last_undegraded":"2024-02-02T15:26:27.274103+0000","last_fullsized":"2024-02-02T15:26:27.274103+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:42:50.009971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.269435+0000","last_change":"2024-02-02T15:25:27.282514+0000","last_active":"2024-02-02T15:26:27.269435+0000","last_peered":"2024-02-02T15:26:27.269435+0000","last_clean":"2024-02-02T15:26:27.269435+0000","last_became_active":"2024-02-02T15:25:27.281336+0000","last_became_peered":"2024-02-02T15:25:27.281336+0000","last_unstale":"2024-02-02T15:26:27.269435+0000","last_undegraded":"2024-02-02T15:26:27.269435+0000","last_fullsized":"2024-02-02T15:26:27.269435+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:50:39.728813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,8,11],"acting":[6,3,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"84'17","reported_seq":38,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.274050+0000","last_change":"2024-02-02T15:24:47.274152+0000","last_active":"2024-02-02T15:26:27.274050+0000","last_peered":"2024-02-02T15:26:27.274050+0000","last_clean":"2024-02-02T15:26:27.274050+0000","last_became_active":"2024-02-02T15:24:47.273720+0000","last_became_peered":"2024-02-02T15:24:47.273720+0000","last_unstale":"2024-02-02T15:26:27.274050+0000","last_undegraded":"2024-02-02T15:26:27.274050+0000","last_fullsized":"2024-02-02T15:26:27.274050+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:38:05.280309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"86'6","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.281387+0000","last_change":"2024-02-02T15:24:47.790681+0000","last_active":"2024-02-02T15:26:27.281387+0000","last_peered":"2024-02-02T15:26:27.281387+0000","last_clean":"2024-02-02T15:26:27.281387+0000","last_became_active":"2024-02-02T15:24:47.789952+0000","last_became_peered":"2024-02-02T15:24:47.789952+0000","last_unstale":"2024-02-02T15:26:27.281387+0000","last_undegraded":"2024-02-02T15:26:27.281387+0000","last_fullsized":"2024-02-02T15:26:27.281387+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:21:03.103398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.281438+0000","last_change":"2024-02-02T15:24:51.812676+0000","last_active":"2024-02-02T15:26:27.281438+0000","last_peered":"2024-02-02T15:26:27.281438+0000","last_clean":"2024-02-02T15:26:27.281438+0000","last_became_active":"2024-02-02T15:24:51.812212+0000","last_became_peered":"2024-02-02T15:24:51.812212+0000","last_unstale":"2024-02-02T15:26:27.281438+0000","last_undegraded":"2024-02-02T15:26:27.281438+0000","last_fullsized":"2024-02-02T15:26:27.281438+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:48:01.944379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'2","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.274629+0000","last_change":"2024-02-02T15:24:47.271707+0000","last_active":"2024-02-02T15:26:27.274629+0000","last_peered":"2024-02-02T15:26:27.274629+0000","last_clean":"2024-02-02T15:26:27.274629+0000","last_became_active":"2024-02-02T15:24:47.271379+0000","last_became_peered":"2024-02-02T15:24:47.271379+0000","last_unstale":"2024-02-02T15:26:27.274629+0000","last_undegraded":"2024-02-02T15:26:27.274629+0000","last_fullsized":"2024-02-02T15:26:27.274629+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:00:24.571385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"86'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.379740+0000","last_change":"2024-02-02T15:24:47.788360+0000","last_active":"2024-02-02T15:26:27.379740+0000","last_peered":"2024-02-02T15:26:27.379740+0000","last_clean":"2024-02-02T15:26:27.379740+0000","last_became_active":"2024-02-02T15:24:47.787945+0000","last_became_peered":"2024-02-02T15:24:47.787945+0000","last_unstale":"2024-02-02T15:26:27.379740+0000","last_undegraded":"2024-02-02T15:26:27.379740+0000","last_fullsized":"2024-02-02T15:26:27.379740+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:37:54.531487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.274398+0000","last_change":"2024-02-02T15:24:51.821320+0000","last_active":"2024-02-02T15:26:27.274398+0000","last_peered":"2024-02-02T15:26:27.274398+0000","last_clean":"2024-02-02T15:26:27.274398+0000","last_became_active":"2024-02-02T15:24:51.820853+0000","last_became_peered":"2024-02-02T15:24:51.820853+0000","last_unstale":"2024-02-02T15:26:27.274398+0000","last_undegraded":"2024-02-02T15:26:27.274398+0000","last_fullsized":"2024-02-02T15:26:27.274398+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:20:12.304517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"84'1","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.280230+0000","last_change":"2024-02-02T15:24:47.278291+0000","last_active":"2024-02-02T15:26:27.280230+0000","last_peered":"2024-02-02T15:26:27.280230+0000","last_clean":"2024-02-02T15:26:27.280230+0000","last_became_active":"2024-02-02T15:24:47.276163+0000","last_became_peered":"2024-02-02T15:24:47.276163+0000","last_unstale":"2024-02-02T15:26:27.280230+0000","last_undegraded":"2024-02-02T15:26:27.280230+0000","last_fullsized":"2024-02-02T15:26:27.280230+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:53:50.312627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.362700+0000","last_change":"2024-02-02T15:24:47.791290+0000","last_active":"2024-02-02T15:26:27.362700+0000","last_peered":"2024-02-02T15:26:27.362700+0000","last_clean":"2024-02-02T15:26:27.362700+0000","last_became_active":"2024-02-02T15:24:47.790987+0000","last_became_peered":"2024-02-02T15:24:47.790987+0000","last_unstale":"2024-02-02T15:26:27.362700+0000","last_undegraded":"2024-02-02T15:26:27.362700+0000","last_fullsized":"2024-02-02T15:26:27.362700+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:38:08.803338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.268999+0000","last_change":"2024-02-02T15:24:51.813050+0000","last_active":"2024-02-02T15:26:27.268999+0000","last_peered":"2024-02-02T15:26:27.268999+0000","last_clean":"2024-02-02T15:26:27.268999+0000","last_became_active":"2024-02-02T15:24:51.812634+0000","last_became_peered":"2024-02-02T15:24:51.812634+0000","last_unstale":"2024-02-02T15:26:27.268999+0000","last_undegraded":"2024-02-02T15:26:27.268999+0000","last_fullsized":"2024-02-02T15:26:27.268999+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:06:56.428431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"84'5","reported_seq":107,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-02T15:26:39.390566+0000","last_change":"2024-02-02T15:24:47.274033+0000","last_active":"2024-02-02T15:26:39.390566+0000","last_peered":"2024-02-02T15:26:39.390566+0000","last_clean":"2024-02-02T15:26:39.390566+0000","last_became_active":"2024-02-02T15:24:47.273672+0000","last_became_peered":"2024-02-02T15:24:47.273672+0000","last_unstale":"2024-02-02T15:26:39.390566+0000","last_undegraded":"2024-02-02T15:26:39.390566+0000","last_fullsized":"2024-02-02T15:26:39.390566+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:51:46.926397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":82,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"86'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.362550+0000","last_change":"2024-02-02T15:24:47.786533+0000","last_active":"2024-02-02T15:26:27.362550+0000","last_peered":"2024-02-02T15:26:27.362550+0000","last_clean":"2024-02-02T15:26:27.362550+0000","last_became_active":"2024-02-02T15:24:47.785436+0000","last_became_peered":"2024-02-02T15:24:47.785436+0000","last_unstale":"2024-02-02T15:26:27.362550+0000","last_undegraded":"2024-02-02T15:26:27.362550+0000","last_fullsized":"2024-02-02T15:26:27.362550+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:23:45.910800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.269411+0000","last_change":"2024-02-02T15:24:51.814326+0000","last_active":"2024-02-02T15:26:27.269411+0000","last_peered":"2024-02-02T15:26:27.269411+0000","last_clean":"2024-02-02T15:26:27.269411+0000","last_became_active":"2024-02-02T15:24:51.813909+0000","last_became_peered":"2024-02-02T15:24:51.813909+0000","last_unstale":"2024-02-02T15:26:27.269411+0000","last_undegraded":"2024-02-02T15:26:27.269411+0000","last_fullsized":"2024-02-02T15:26:27.269411+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:16:12.902718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"86'8","reported_seq":36,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.281562+0000","last_change":"2024-02-02T15:25:27.287818+0000","last_active":"2024-02-02T15:26:27.281562+0000","last_peered":"2024-02-02T15:26:27.281562+0000","last_clean":"2024-02-02T15:26:27.281562+0000","last_became_active":"2024-02-02T15:25:27.287481+0000","last_became_peered":"2024-02-02T15:25:27.287481+0000","last_unstale":"2024-02-02T15:26:27.281562+0000","last_undegraded":"2024-02-02T15:26:27.281562+0000","last_fullsized":"2024-02-02T15:26:27.281562+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:14:32.820969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'6","reported_seq":110,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-02T15:26:41.399825+0000","last_change":"2024-02-02T15:24:47.274437+0000","last_active":"2024-02-02T15:26:41.399825+0000","last_peered":"2024-02-02T15:26:41.399825+0000","last_clean":"2024-02-02T15:26:41.399825+0000","last_became_active":"2024-02-02T15:24:47.274010+0000","last_became_peered":"2024-02-02T15:24:47.274010+0000","last_unstale":"2024-02-02T15:26:41.399825+0000","last_undegraded":"2024-02-02T15:26:41.399825+0000","last_fullsized":"2024-02-02T15:26:41.399825+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:10:36.862190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":84,"num_read_kb":84,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"84'1","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.274175+0000","last_change":"2024-02-02T15:24:51.825698+0000","last_active":"2024-02-02T15:26:27.274175+0000","last_peered":"2024-02-02T15:26:27.274175+0000","last_clean":"2024-02-02T15:26:27.274175+0000","last_became_active":"2024-02-02T15:24:51.825295+0000","last_became_peered":"2024-02-02T15:24:51.825295+0000","last_unstale":"2024-02-02T15:26:27.274175+0000","last_undegraded":"2024-02-02T15:26:27.274175+0000","last_fullsized":"2024-02-02T15:26:27.274175+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:17:09.838124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"86'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.375702+0000","last_change":"2024-02-02T15:24:47.781836+0000","last_active":"2024-02-02T15:26:27.375702+0000","last_peered":"2024-02-02T15:26:27.375702+0000","last_clean":"2024-02-02T15:26:27.375702+0000","last_became_active":"2024-02-02T15:24:47.781472+0000","last_became_peered":"2024-02-02T15:24:47.781472+0000","last_unstale":"2024-02-02T15:26:27.375702+0000","last_undegraded":"2024-02-02T15:26:27.375702+0000","last_fullsized":"2024-02-02T15:26:27.375702+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:46:33.714191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"84'6","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.279642+0000","last_change":"2024-02-02T15:24:47.269087+0000","last_active":"2024-02-02T15:26:27.279642+0000","last_peered":"2024-02-02T15:26:27.279642+0000","last_clean":"2024-02-02T15:26:27.279642+0000","last_became_active":"2024-02-02T15:24:47.268743+0000","last_became_peered":"2024-02-02T15:24:47.268743+0000","last_unstale":"2024-02-02T15:26:27.279642+0000","last_undegraded":"2024-02-02T15:26:27.279642+0000","last_fullsized":"2024-02-02T15:26:27.279642+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:03:04.323079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":338,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.273687+0000","last_change":"2024-02-02T15:23:55.191735+0000","last_active":"2024-02-02T15:26:27.273687+0000","last_peered":"2024-02-02T15:26:27.273687+0000","last_clean":"2024-02-02T15:26:27.273687+0000","last_became_active":"2024-02-02T15:23:55.191119+0000","last_became_peered":"2024-02-02T15:23:55.191119+0000","last_unstale":"2024-02-02T15:26:27.273687+0000","last_undegraded":"2024-02-02T15:26:27.273687+0000","last_fullsized":"2024-02-02T15:26:27.273687+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:21:22.472631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:21:22.472631+0000","last_clean_scrub_stamp":"2024-02-02T15:21:22.472631+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:24:44.153461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.281781+0000","last_change":"2024-02-02T15:24:51.822842+0000","last_active":"2024-02-02T15:26:27.281781+0000","last_peered":"2024-02-02T15:26:27.281781+0000","last_clean":"2024-02-02T15:26:27.281781+0000","last_became_active":"2024-02-02T15:24:51.822372+0000","last_became_peered":"2024-02-02T15:24:51.822372+0000","last_unstale":"2024-02-02T15:26:27.281781+0000","last_undegraded":"2024-02-02T15:26:27.281781+0000","last_fullsized":"2024-02-02T15:26:27.281781+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:45:29.631422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"86'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.268664+0000","last_change":"2024-02-02T15:24:47.786507+0000","last_active":"2024-02-02T15:26:27.268664+0000","last_peered":"2024-02-02T15:26:27.268664+0000","last_clean":"2024-02-02T15:26:27.268664+0000","last_became_active":"2024-02-02T15:24:47.786125+0000","last_became_peered":"2024-02-02T15:24:47.786125+0000","last_unstale":"2024-02-02T15:26:27.268664+0000","last_undegraded":"2024-02-02T15:26:27.268664+0000","last_fullsized":"2024-02-02T15:26:27.268664+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:33:09.014258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"84'4","reported_seq":37,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.274171+0000","last_change":"2024-02-02T15:25:27.628074+0000","last_active":"2024-02-02T15:26:27.274171+0000","last_peered":"2024-02-02T15:26:27.274171+0000","last_clean":"2024-02-02T15:26:27.274171+0000","last_became_active":"2024-02-02T15:25:27.625103+0000","last_became_peered":"2024-02-02T15:25:27.625103+0000","last_unstale":"2024-02-02T15:26:27.274171+0000","last_undegraded":"2024-02-02T15:26:27.274171+0000","last_fullsized":"2024-02-02T15:26:27.274171+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:30:35.540456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.379470+0000","last_change":"2024-02-02T15:24:51.808979+0000","last_active":"2024-02-02T15:26:27.379470+0000","last_peered":"2024-02-02T15:26:27.379470+0000","last_clean":"2024-02-02T15:26:27.379470+0000","last_became_active":"2024-02-02T15:24:51.808557+0000","last_became_peered":"2024-02-02T15:24:51.808557+0000","last_unstale":"2024-02-02T15:26:27.379470+0000","last_undegraded":"2024-02-02T15:26:27.379470+0000","last_fullsized":"2024-02-02T15:26:27.379470+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T02:38:34.697137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"84'4","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.274530+0000","last_change":"2024-02-02T15:24:47.281396+0000","last_active":"2024-02-02T15:26:27.274530+0000","last_peered":"2024-02-02T15:26:27.274530+0000","last_clean":"2024-02-02T15:26:27.274530+0000","last_became_active":"2024-02-02T15:24:47.281036+0000","last_became_peered":"2024-02-02T15:24:47.281036+0000","last_unstale":"2024-02-02T15:26:27.274530+0000","last_undegraded":"2024-02-02T15:26:27.274530+0000","last_fullsized":"2024-02-02T15:26:27.274530+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:54:47.139151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"86'9","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.269092+0000","last_change":"2024-02-02T15:24:47.791057+0000","last_active":"2024-02-02T15:26:27.269092+0000","last_peered":"2024-02-02T15:26:27.269092+0000","last_clean":"2024-02-02T15:26:27.269092+0000","last_became_active":"2024-02-02T15:24:47.790647+0000","last_became_peered":"2024-02-02T15:24:47.790647+0000","last_unstale":"2024-02-02T15:26:27.269092+0000","last_undegraded":"2024-02-02T15:26:27.269092+0000","last_fullsized":"2024-02-02T15:26:27.269092+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T02:49:42.067700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.375624+0000","last_change":"2024-02-02T15:24:51.817792+0000","last_active":"2024-02-02T15:26:27.375624+0000","last_peered":"2024-02-02T15:26:27.375624+0000","last_clean":"2024-02-02T15:26:27.375624+0000","last_became_active":"2024-02-02T15:24:51.815800+0000","last_became_peered":"2024-02-02T15:24:51.815800+0000","last_unstale":"2024-02-02T15:26:27.375624+0000","last_undegraded":"2024-02-02T15:26:27.375624+0000","last_fullsized":"2024-02-02T15:26:27.375624+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:28:10.440251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'12","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-02T15:26:28.276379+0000","last_change":"2024-02-02T15:24:47.277098+0000","last_active":"2024-02-02T15:26:28.276379+0000","last_peered":"2024-02-02T15:26:28.276379+0000","last_clean":"2024-02-02T15:26:28.276379+0000","last_became_active":"2024-02-02T15:24:47.275867+0000","last_became_peered":"2024-02-02T15:24:47.275867+0000","last_unstale":"2024-02-02T15:26:28.276379+0000","last_undegraded":"2024-02-02T15:26:28.276379+0000","last_fullsized":"2024-02-02T15:26:28.276379+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:35:21.015918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"86'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.376351+0000","last_change":"2024-02-02T15:24:47.784873+0000","last_active":"2024-02-02T15:26:27.376351+0000","last_peered":"2024-02-02T15:26:27.376351+0000","last_clean":"2024-02-02T15:26:27.376351+0000","last_became_active":"2024-02-02T15:24:47.784550+0000","last_became_peered":"2024-02-02T15:24:47.784550+0000","last_unstale":"2024-02-02T15:26:27.376351+0000","last_undegraded":"2024-02-02T15:26:27.376351+0000","last_fullsized":"2024-02-02T15:26:27.376351+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:13:37.513754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.379656+0000","last_change":"2024-02-02T15:24:51.805131+0000","last_active":"2024-02-02T15:26:27.379656+0000","last_peered":"2024-02-02T15:26:27.379656+0000","last_clean":"2024-02-02T15:26:27.379656+0000","last_became_active":"2024-02-02T15:24:51.804705+0000","last_became_peered":"2024-02-02T15:24:51.804705+0000","last_unstale":"2024-02-02T15:26:27.379656+0000","last_undegraded":"2024-02-02T15:26:27.379656+0000","last_fullsized":"2024-02-02T15:26:27.379656+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T02:12:59.945050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"84'7","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.280301+0000","last_change":"2024-02-02T15:24:47.266469+0000","last_active":"2024-02-02T15:26:27.280301+0000","last_peered":"2024-02-02T15:26:27.280301+0000","last_clean":"2024-02-02T15:26:27.280301+0000","last_became_active":"2024-02-02T15:24:47.266094+0000","last_became_peered":"2024-02-02T15:24:47.266094+0000","last_unstale":"2024-02-02T15:26:27.280301+0000","last_undegraded":"2024-02-02T15:26:27.280301+0000","last_fullsized":"2024-02-02T15:26:27.280301+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:51:41.324698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"86'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.486689+0000","last_change":"2024-02-02T15:24:47.793051+0000","last_active":"2024-02-02T15:26:27.486689+0000","last_peered":"2024-02-02T15:26:27.486689+0000","last_clean":"2024-02-02T15:26:27.486689+0000","last_became_active":"2024-02-02T15:24:47.792645+0000","last_became_peered":"2024-02-02T15:24:47.792645+0000","last_unstale":"2024-02-02T15:26:27.486689+0000","last_undegraded":"2024-02-02T15:26:27.486689+0000","last_fullsized":"2024-02-02T15:26:27.486689+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:47:17.229407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.280008+0000","last_change":"2024-02-02T15:25:27.282505+0000","last_active":"2024-02-02T15:26:27.280008+0000","last_peered":"2024-02-02T15:26:27.280008+0000","last_clean":"2024-02-02T15:26:27.280008+0000","last_became_active":"2024-02-02T15:25:27.282039+0000","last_became_peered":"2024-02-02T15:25:27.282039+0000","last_unstale":"2024-02-02T15:26:27.280008+0000","last_undegraded":"2024-02-02T15:26:27.280008+0000","last_fullsized":"2024-02-02T15:26:27.280008+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T21:31:29.700848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,11],"acting":[4,5,10,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"84'2","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.277078+0000","last_change":"2024-02-02T15:24:47.273716+0000","last_active":"2024-02-02T15:26:27.277078+0000","last_peered":"2024-02-02T15:26:27.277078+0000","last_clean":"2024-02-02T15:26:27.277078+0000","last_became_active":"2024-02-02T15:24:47.273438+0000","last_became_peered":"2024-02-02T15:24:47.273438+0000","last_unstale":"2024-02-02T15:26:27.277078+0000","last_undegraded":"2024-02-02T15:26:27.277078+0000","last_fullsized":"2024-02-02T15:26:27.277078+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:52:16.498119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"86'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.274074+0000","last_change":"2024-02-02T15:24:47.782950+0000","last_active":"2024-02-02T15:26:27.274074+0000","last_peered":"2024-02-02T15:26:27.274074+0000","last_clean":"2024-02-02T15:26:27.274074+0000","last_became_active":"2024-02-02T15:24:47.782605+0000","last_became_peered":"2024-02-02T15:24:47.782605+0000","last_unstale":"2024-02-02T15:26:27.274074+0000","last_undegraded":"2024-02-02T15:26:27.274074+0000","last_fullsized":"2024-02-02T15:26:27.274074+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:57:34.125985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.274573+0000","last_change":"2024-02-02T15:25:27.288179+0000","last_active":"2024-02-02T15:26:27.274573+0000","last_peered":"2024-02-02T15:26:27.274573+0000","last_clean":"2024-02-02T15:26:27.274573+0000","last_became_active":"2024-02-02T15:25:27.287753+0000","last_became_peered":"2024-02-02T15:25:27.287753+0000","last_unstale":"2024-02-02T15:26:27.274573+0000","last_undegraded":"2024-02-02T15:26:27.274573+0000","last_fullsized":"2024-02-02T15:26:27.274573+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:02:46.854950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"84'7","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.375810+0000","last_change":"2024-02-02T15:24:47.278478+0000","last_active":"2024-02-02T15:26:27.375810+0000","last_peered":"2024-02-02T15:26:27.375810+0000","last_clean":"2024-02-02T15:26:27.375810+0000","last_became_active":"2024-02-02T15:24:47.278084+0000","last_became_peered":"2024-02-02T15:24:47.278084+0000","last_unstale":"2024-02-02T15:26:27.375810+0000","last_undegraded":"2024-02-02T15:26:27.375810+0000","last_fullsized":"2024-02-02T15:26:27.375810+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:10:23.044968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"86'9","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.378859+0000","last_change":"2024-02-02T15:24:47.795291+0000","last_active":"2024-02-02T15:26:27.378859+0000","last_peered":"2024-02-02T15:26:27.378859+0000","last_clean":"2024-02-02T15:26:27.378859+0000","last_became_active":"2024-02-02T15:24:47.794514+0000","last_became_peered":"2024-02-02T15:24:47.794514+0000","last_unstale":"2024-02-02T15:26:27.378859+0000","last_undegraded":"2024-02-02T15:26:27.378859+0000","last_fullsized":"2024-02-02T15:26:27.378859+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:32:12.722258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"84'1","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.280163+0000","last_change":"2024-02-02T15:24:51.825985+0000","last_active":"2024-02-02T15:26:27.280163+0000","last_peered":"2024-02-02T15:26:27.280163+0000","last_clean":"2024-02-02T15:26:27.280163+0000","last_became_active":"2024-02-02T15:24:51.825588+0000","last_became_peered":"2024-02-02T15:24:51.825588+0000","last_unstale":"2024-02-02T15:26:27.280163+0000","last_undegraded":"2024-02-02T15:26:27.280163+0000","last_fullsized":"2024-02-02T15:26:27.280163+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T01:32:39.910258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"84'3","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.378872+0000","last_change":"2024-02-02T15:25:27.298553+0000","last_active":"2024-02-02T15:26:27.378872+0000","last_peered":"2024-02-02T15:26:27.378872+0000","last_clean":"2024-02-02T15:26:27.378872+0000","last_became_active":"2024-02-02T15:25:27.282469+0000","last_became_peered":"2024-02-02T15:25:27.282469+0000","last_unstale":"2024-02-02T15:26:27.378872+0000","last_undegraded":"2024-02-02T15:26:27.378872+0000","last_fullsized":"2024-02-02T15:26:27.378872+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T00:54:14.740951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"86'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.375881+0000","last_change":"2024-02-02T15:24:47.784188+0000","last_active":"2024-02-02T15:26:27.375881+0000","last_peered":"2024-02-02T15:26:27.375881+0000","last_clean":"2024-02-02T15:26:27.375881+0000","last_became_active":"2024-02-02T15:24:47.783872+0000","last_became_peered":"2024-02-02T15:24:47.783872+0000","last_unstale":"2024-02-02T15:26:27.375881+0000","last_undegraded":"2024-02-02T15:26:27.375881+0000","last_fullsized":"2024-02-02T15:26:27.375881+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:45:45.213255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.280221+0000","last_change":"2024-02-02T15:24:51.815263+0000","last_active":"2024-02-02T15:26:27.280221+0000","last_peered":"2024-02-02T15:26:27.280221+0000","last_clean":"2024-02-02T15:26:27.280221+0000","last_became_active":"2024-02-02T15:24:51.814849+0000","last_became_peered":"2024-02-02T15:24:51.814849+0000","last_unstale":"2024-02-02T15:26:27.280221+0000","last_undegraded":"2024-02-02T15:26:27.280221+0000","last_fullsized":"2024-02-02T15:26:27.280221+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:05:32.987359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"84'4","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.274712+0000","last_change":"2024-02-02T15:24:47.270534+0000","last_active":"2024-02-02T15:26:27.274712+0000","last_peered":"2024-02-02T15:26:27.274712+0000","last_clean":"2024-02-02T15:26:27.274712+0000","last_became_active":"2024-02-02T15:24:47.270110+0000","last_became_peered":"2024-02-02T15:24:47.270110+0000","last_unstale":"2024-02-02T15:26:27.274712+0000","last_undegraded":"2024-02-02T15:26:27.274712+0000","last_fullsized":"2024-02-02T15:26:27.274712+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T02:59:52.297315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"86'5","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.273855+0000","last_change":"2024-02-02T15:24:47.791826+0000","last_active":"2024-02-02T15:26:27.273855+0000","last_peered":"2024-02-02T15:26:27.273855+0000","last_clean":"2024-02-02T15:26:27.273855+0000","last_became_active":"2024-02-02T15:24:47.791016+0000","last_became_peered":"2024-02-02T15:24:47.791016+0000","last_unstale":"2024-02-02T15:26:27.273855+0000","last_undegraded":"2024-02-02T15:26:27.273855+0000","last_fullsized":"2024-02-02T15:26:27.273855+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:47:14.296876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.276753+0000","last_change":"2024-02-02T15:24:51.806741+0000","last_active":"2024-02-02T15:26:27.276753+0000","last_peered":"2024-02-02T15:26:27.276753+0000","last_clean":"2024-02-02T15:26:27.276753+0000","last_became_active":"2024-02-02T15:24:51.806085+0000","last_became_peered":"2024-02-02T15:24:51.806085+0000","last_unstale":"2024-02-02T15:26:27.276753+0000","last_undegraded":"2024-02-02T15:26:27.276753+0000","last_fullsized":"2024-02-02T15:26:27.276753+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:53:39.944222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"84'5","reported_seq":109,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-02T15:26:40.400639+0000","last_change":"2024-02-02T15:24:47.278515+0000","last_active":"2024-02-02T15:26:40.400639+0000","last_peered":"2024-02-02T15:26:40.400639+0000","last_clean":"2024-02-02T15:26:40.400639+0000","last_became_active":"2024-02-02T15:24:47.278145+0000","last_became_peered":"2024-02-02T15:24:47.278145+0000","last_unstale":"2024-02-02T15:26:40.400639+0000","last_undegraded":"2024-02-02T15:26:40.400639+0000","last_fullsized":"2024-02-02T15:26:40.400639+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:48:55.118003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1122,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":84,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"86'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.274330+0000","last_change":"2024-02-02T15:24:47.791996+0000","last_active":"2024-02-02T15:26:27.274330+0000","last_peered":"2024-02-02T15:26:27.274330+0000","last_clean":"2024-02-02T15:26:27.274330+0000","last_became_active":"2024-02-02T15:24:47.791624+0000","last_became_peered":"2024-02-02T15:24:47.791624+0000","last_unstale":"2024-02-02T15:26:27.274330+0000","last_undegraded":"2024-02-02T15:26:27.274330+0000","last_fullsized":"2024-02-02T15:26:27.274330+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:33:27.963740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.486271+0000","last_change":"2024-02-02T15:24:51.826835+0000","last_active":"2024-02-02T15:26:27.486271+0000","last_peered":"2024-02-02T15:26:27.486271+0000","last_clean":"2024-02-02T15:26:27.486271+0000","last_became_active":"2024-02-02T15:24:51.826310+0000","last_became_peered":"2024-02-02T15:24:51.826310+0000","last_unstale":"2024-02-02T15:26:27.486271+0000","last_undegraded":"2024-02-02T15:26:27.486271+0000","last_fullsized":"2024-02-02T15:26:27.486271+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:48:16.687048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'3","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.282058+0000","last_change":"2024-02-02T15:24:47.268865+0000","last_active":"2024-02-02T15:26:27.282058+0000","last_peered":"2024-02-02T15:26:27.282058+0000","last_clean":"2024-02-02T15:26:27.282058+0000","last_became_active":"2024-02-02T15:24:47.268523+0000","last_became_peered":"2024-02-02T15:24:47.268523+0000","last_unstale":"2024-02-02T15:26:27.282058+0000","last_undegraded":"2024-02-02T15:26:27.282058+0000","last_fullsized":"2024-02-02T15:26:27.282058+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T15:45:16.957512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'6","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.274101+0000","last_change":"2024-02-02T15:24:47.792601+0000","last_active":"2024-02-02T15:26:27.274101+0000","last_peered":"2024-02-02T15:26:27.274101+0000","last_clean":"2024-02-02T15:26:27.274101+0000","last_became_active":"2024-02-02T15:24:47.792240+0000","last_became_peered":"2024-02-02T15:24:47.792240+0000","last_unstale":"2024-02-02T15:26:27.274101+0000","last_undegraded":"2024-02-02T15:26:27.274101+0000","last_fullsized":"2024-02-02T15:26:27.274101+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:51:25.205859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.281132+0000","last_change":"2024-02-02T15:24:51.806267+0000","last_active":"2024-02-02T15:26:27.281132+0000","last_peered":"2024-02-02T15:26:27.281132+0000","last_clean":"2024-02-02T15:26:27.281132+0000","last_became_active":"2024-02-02T15:24:51.805877+0000","last_became_peered":"2024-02-02T15:24:51.805877+0000","last_unstale":"2024-02-02T15:26:27.281132+0000","last_undegraded":"2024-02-02T15:26:27.281132+0000","last_fullsized":"2024-02-02T15:26:27.281132+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:50.778822+0000","last_clean_scrub_stamp":"2024-02-02T15:24:50.778822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:00:47.747377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'5","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.269402+0000","last_change":"2024-02-02T15:24:47.271057+0000","last_active":"2024-02-02T15:26:27.269402+0000","last_peered":"2024-02-02T15:26:27.269402+0000","last_clean":"2024-02-02T15:26:27.269402+0000","last_became_active":"2024-02-02T15:24:47.270737+0000","last_became_peered":"2024-02-02T15:24:47.270737+0000","last_unstale":"2024-02-02T15:26:27.269402+0000","last_undegraded":"2024-02-02T15:26:27.269402+0000","last_fullsized":"2024-02-02T15:26:27.269402+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:46.245921+0000","last_clean_scrub_stamp":"2024-02-02T15:24:46.245921+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:17:00.212426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"86'9","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.274263+0000","last_change":"2024-02-02T15:24:47.794108+0000","last_active":"2024-02-02T15:26:27.274263+0000","last_peered":"2024-02-02T15:26:27.274263+0000","last_clean":"2024-02-02T15:26:27.274263+0000","last_became_active":"2024-02-02T15:24:47.793890+0000","last_became_peered":"2024-02-02T15:24:47.793890+0000","last_unstale":"2024-02-02T15:26:27.274263+0000","last_undegraded":"2024-02-02T15:26:27.274263+0000","last_fullsized":"2024-02-02T15:26:27.274263+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T03:05:51.704385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"86'9","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.378946+0000","last_change":"2024-02-02T15:24:47.796234+0000","last_active":"2024-02-02T15:26:27.378946+0000","last_peered":"2024-02-02T15:26:27.378946+0000","last_clean":"2024-02-02T15:26:27.378946+0000","last_became_active":"2024-02-02T15:24:47.795474+0000","last_became_peered":"2024-02-02T15:24:47.795474+0000","last_unstale":"2024-02-02T15:26:27.378946+0000","last_undegraded":"2024-02-02T15:26:27.378946+0000","last_fullsized":"2024-02-02T15:26:27.378946+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:25:28.059500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"86'12","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.379845+0000","last_change":"2024-02-02T15:24:47.786930+0000","last_active":"2024-02-02T15:26:27.379845+0000","last_peered":"2024-02-02T15:26:27.379845+0000","last_clean":"2024-02-02T15:26:27.379845+0000","last_became_active":"2024-02-02T15:24:47.786615+0000","last_became_peered":"2024-02-02T15:24:47.786615+0000","last_unstale":"2024-02-02T15:26:27.379845+0000","last_undegraded":"2024-02-02T15:26:27.379845+0000","last_fullsized":"2024-02-02T15:26:27.379845+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T22:17:23.231511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"86'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.379038+0000","last_change":"2024-02-02T15:24:47.797877+0000","last_active":"2024-02-02T15:26:27.379038+0000","last_peered":"2024-02-02T15:26:27.379038+0000","last_clean":"2024-02-02T15:26:27.379038+0000","last_became_active":"2024-02-02T15:24:47.797559+0000","last_became_peered":"2024-02-02T15:24:47.797559+0000","last_unstale":"2024-02-02T15:26:27.379038+0000","last_undegraded":"2024-02-02T15:26:27.379038+0000","last_fullsized":"2024-02-02T15:26:27.379038+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:13:55.529796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"86'7","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.486450+0000","last_change":"2024-02-02T15:24:47.790270+0000","last_active":"2024-02-02T15:26:27.486450+0000","last_peered":"2024-02-02T15:26:27.486450+0000","last_clean":"2024-02-02T15:26:27.486450+0000","last_became_active":"2024-02-02T15:24:47.789928+0000","last_became_peered":"2024-02-02T15:24:47.789928+0000","last_unstale":"2024-02-02T15:26:27.486450+0000","last_undegraded":"2024-02-02T15:26:27.486450+0000","last_fullsized":"2024-02-02T15:26:27.486450+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T02:34:44.885803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"86'4","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.269112+0000","last_change":"2024-02-02T15:25:27.298415+0000","last_active":"2024-02-02T15:26:27.269112+0000","last_peered":"2024-02-02T15:26:27.269112+0000","last_clean":"2024-02-02T15:26:27.269112+0000","last_became_active":"2024-02-02T15:25:27.297990+0000","last_became_peered":"2024-02-02T15:25:27.297990+0000","last_unstale":"2024-02-02T15:26:27.269112+0000","last_undegraded":"2024-02-02T15:26:27.269112+0000","last_fullsized":"2024-02-02T15:26:27.269112+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T23:38:02.040613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"86'6","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.380026+0000","last_change":"2024-02-02T15:24:47.797861+0000","last_active":"2024-02-02T15:26:27.380026+0000","last_peered":"2024-02-02T15:26:27.380026+0000","last_clean":"2024-02-02T15:26:27.380026+0000","last_became_active":"2024-02-02T15:24:47.797543+0000","last_became_peered":"2024-02-02T15:24:47.797543+0000","last_unstale":"2024-02-02T15:26:27.380026+0000","last_undegraded":"2024-02-02T15:26:27.380026+0000","last_fullsized":"2024-02-02T15:26:27.380026+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:09:37.710909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"86'15","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.380423+0000","last_change":"2024-02-02T15:24:47.796221+0000","last_active":"2024-02-02T15:26:27.380423+0000","last_peered":"2024-02-02T15:26:27.380423+0000","last_clean":"2024-02-02T15:26:27.380423+0000","last_became_active":"2024-02-02T15:24:47.795845+0000","last_became_peered":"2024-02-02T15:24:47.795845+0000","last_unstale":"2024-02-02T15:26:27.380423+0000","last_undegraded":"2024-02-02T15:26:27.380423+0000","last_fullsized":"2024-02-02T15:26:27.380423+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T17:54:55.637660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"86'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.376469+0000","last_change":"2024-02-02T15:24:47.783086+0000","last_active":"2024-02-02T15:26:27.376469+0000","last_peered":"2024-02-02T15:26:27.376469+0000","last_clean":"2024-02-02T15:26:27.376469+0000","last_became_active":"2024-02-02T15:24:47.781826+0000","last_became_peered":"2024-02-02T15:24:47.781826+0000","last_unstale":"2024-02-02T15:26:27.376469+0000","last_undegraded":"2024-02-02T15:26:27.376469+0000","last_fullsized":"2024-02-02T15:26:27.376469+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:32:05.623591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"86'6","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.268677+0000","last_change":"2024-02-02T15:25:27.284461+0000","last_active":"2024-02-02T15:26:27.268677+0000","last_peered":"2024-02-02T15:26:27.268677+0000","last_clean":"2024-02-02T15:26:27.268677+0000","last_became_active":"2024-02-02T15:25:27.284080+0000","last_became_peered":"2024-02-02T15:25:27.284080+0000","last_unstale":"2024-02-02T15:26:27.268677+0000","last_undegraded":"2024-02-02T15:26:27.268677+0000","last_fullsized":"2024-02-02T15:26:27.268677+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T19:43:40.124825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,3],"acting":[6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.29","version":"86'8","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-02-02T15:26:28.282109+0000","last_change":"2024-02-02T15:26:28.281754+0000","last_active":"2024-02-02T15:26:28.282109+0000","last_peered":"2024-02-02T15:26:28.282109+0000","last_clean":"2024-02-02T15:26:28.282109+0000","last_became_active":"2024-02-02T15:26:28.281396+0000","last_became_peered":"2024-02-02T15:26:28.281396+0000","last_unstale":"2024-02-02T15:26:28.282109+0000","last_undegraded":"2024-02-02T15:26:28.282109+0000","last_fullsized":"2024-02-02T15:26:28.282109+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T16:26:26.973936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,11],"acting":[9,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2a","version":"86'10","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.277193+0000","last_change":"2024-02-02T15:24:47.795928+0000","last_active":"2024-02-02T15:26:27.277193+0000","last_peered":"2024-02-02T15:26:27.277193+0000","last_clean":"2024-02-02T15:26:27.277193+0000","last_became_active":"2024-02-02T15:24:47.795511+0000","last_became_peered":"2024-02-02T15:24:47.795511+0000","last_unstale":"2024-02-02T15:26:27.277193+0000","last_undegraded":"2024-02-02T15:26:27.277193+0000","last_fullsized":"2024-02-02T15:26:27.277193+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T20:55:31.444331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"86'11","reported_seq":30,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.268957+0000","last_change":"2024-02-02T15:24:47.794368+0000","last_active":"2024-02-02T15:26:27.268957+0000","last_peered":"2024-02-02T15:26:27.268957+0000","last_clean":"2024-02-02T15:26:27.268957+0000","last_became_active":"2024-02-02T15:24:47.793808+0000","last_became_peered":"2024-02-02T15:24:47.793808+0000","last_unstale":"2024-02-02T15:26:27.268957+0000","last_undegraded":"2024-02-02T15:26:27.268957+0000","last_fullsized":"2024-02-02T15:26:27.268957+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-04T02:39:00.099808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"86'8","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2024-02-02T15:26:27.379545+0000","last_change":"2024-02-02T15:24:47.796358+0000","last_active":"2024-02-02T15:26:27.379545+0000","last_peered":"2024-02-02T15:26:27.379545+0000","last_clean":"2024-02-02T15:26:27.379545+0000","last_became_active":"2024-02-02T15:24:47.795800+0000","last_became_peered":"2024-02-02T15:24:47.795800+0000","last_unstale":"2024-02-02T15:26:27.379545+0000","last_undegraded":"2024-02-02T15:26:27.379545+0000","last_fullsized":"2024-02-02T15:26:27.379545+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-02T15:24:47.247814+0000","last_clean_scrub_stamp":"2024-02-02T15:24:47.247814+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-03T18:49:40.019787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":250199,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67720,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":864,"num_read_kb":505,"num_write":248,"num_write_kb":144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":68,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":581632,"data_stored":213918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":216,"ondisk_log_size":216,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547235,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":983040,"data_stored":496773,"data_compressed":23664,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 15:26:15 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.53600000000000003,"15min":0.53300000000000003},"min":{"1min":0.42999999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.745,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.57099999999999995,"15min":0.55900000000000005},"min":{"1min":0.5,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.45200000000000001}]},{"osd":1,"last update":"Fri Feb 2 15:26:15 2024","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.59599999999999997,"15min":0.58899999999999997},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.80200000000000005,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.61199999999999999,"15min":0.60299999999999998},"min":{"1min":0.39300000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.67200000000000004}]},{"osd":2,"last update":"Fri Feb 2 15:26:15 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.45500000000000002,"15min":0.44400000000000001},"min":{"1min":0.39300000000000002,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.496},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.46000000000000002,"15min":0.45100000000000001},"min":{"1min":0.376,"5min":0.316,"15min":0.316},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.63600000000000001}]},{"osd":3,"last update":"Fri Feb 2 15:26:27 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.56699999999999995}]},{"osd":4,"last update":"Fri Feb 2 15:26:15 2024","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.64100000000000001,"15min":0.629},"min":{"1min":0.52400000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.63600000000000001,"15min":0.63400000000000001},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.76300000000000001,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.71299999999999997}]},{"osd":5,"last update":"Fri Feb 2 15:26:15 2024","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.57999999999999996,"15min":0.56299999999999994},"min":{"1min":0.53200000000000003,"5min":0.377,"15min":0.377},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.622},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.57899999999999996,"15min":0.56599999999999995},"min":{"1min":0.504,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.73399999999999999}]},{"osd":6,"last update":"Fri Feb 2 15:26:15 2024","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.70399999999999996,"15min":0.70599999999999996},"min":{"1min":0.57499999999999996,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.93999999999999995,"5min":1.325,"15min":1.325},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.72699999999999998,"15min":0.71499999999999997},"min":{"1min":0.64100000000000001,"5min":0.498,"15min":0.498},"max":{"1min":0.97099999999999997,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.76700000000000002}]},{"osd":7,"last update":"Fri Feb 2 15:26:15 2024","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.71699999999999997,"15min":0.71199999999999997},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.96799999999999997,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.874},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.73799999999999999,"15min":0.73099999999999998},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.92600000000000005,"5min":1.27,"15min":1.27},"last":0.78400000000000003}]},{"osd":8,"last update":"Fri Feb 2 15:25:50 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.82199999999999995}]},{"osd":9,"last update":"Fri Feb 2 15:26:15 2024","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.79000000000000004,"15min":0.78100000000000003},"min":{"1min":0.66900000000000004,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.0109999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.77000000000000002,"15min":0.76000000000000001},"min":{"1min":0.71999999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.006,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.80200000000000005}]},{"osd":10,"last update":"Fri Feb 2 15:26:15 2024","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.79700000000000004,"15min":0.79100000000000004},"min":{"1min":0.63,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":0.95999999999999996,"5min":1.21,"15min":1.21},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.85599999999999998,"5min":0.77200000000000002,"15min":0.75800000000000001},"min":{"1min":0.71899999999999997,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.98999999999999999,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.86799999999999999}]}]},{"osd":10,"up_from":67,"seq":287762808868,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1122304,"data_stored":1145011,"data_compressed":59875,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 15:25:48 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.45300000000000001}]},{"osd":1,"last update":"Fri Feb 2 15:25:53 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.439,"15min":0.441},"min":{"1min":0.32200000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.63300000000000001,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.432},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.45900000000000002,"15min":0.46100000000000002},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.36699999999999999}]},{"osd":2,"last update":"Fri Feb 2 15:25:53 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.58799999999999997,"15min":0.59199999999999997},"min":{"1min":0.40000000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.72499999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.59399999999999997,"15min":0.59499999999999997},"min":{"1min":0.42599999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.74399999999999999,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.59299999999999997}]},{"osd":3,"last update":"Fri Feb 2 15:25:53 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.58999999999999997,"15min":0.59799999999999998},"min":{"1min":0.39900000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.73299999999999998,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.78700000000000003,"15min":0.82399999999999995},"min":{"1min":0.40600000000000003,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.70399999999999996,"5min":4.4109999999999996,"15min":4.4109999999999996},"last":0.56100000000000005}]},{"osd":4,"last update":"Fri Feb 2 15:25:53 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.36899999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.78400000000000003,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.52000000000000002,"15min":0.52500000000000002},"min":{"1min":0.375,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.65200000000000002,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.40200000000000002}]},{"osd":5,"last update":"Fri Feb 2 15:25:53 2024","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.65600000000000003,"15min":0.65800000000000003},"min":{"1min":0.45500000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.77200000000000002,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.66300000000000003,"15min":0.66000000000000003},"min":{"1min":0.47099999999999997,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.86499999999999999,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.51700000000000002}]},{"osd":6,"last update":"Fri Feb 2 15:25:53 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.80100000000000005,"15min":0.82699999999999996},"min":{"1min":0.52300000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.82799999999999996,"5min":4.4459999999999997,"15min":4.4459999999999997},"last":0.627},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.71999999999999997,"15min":0.72899999999999998},"min":{"1min":0.44500000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.83899999999999997,"5min":1.3740000000000001,"15min":1.3740000000000001},"last":0.64700000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40300000000000002}]},{"osd":8,"last update":"Fri Feb 2 15:25:53 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.71999999999999997,"15min":0.72399999999999998},"min":{"1min":0.496,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.92100000000000004,"5min":1.413,"15min":1.413},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.71899999999999997,"15min":0.72199999999999998},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":0.89800000000000002,"5min":1.036,"15min":1.036},"last":0.54400000000000004}]},{"osd":9,"last update":"Fri Feb 2 15:25:53 2024","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.77600000000000002,"15min":0.78400000000000003},"min":{"1min":0.58899999999999997,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.88300000000000001,"5min":1.506,"15min":1.506},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.89800000000000002,"15min":0.93200000000000005},"min":{"1min":0.55400000000000005,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":0.86799999999999999,"5min":4.4269999999999996,"15min":4.4269999999999996},"last":0.63900000000000001}]},{"osd":11,"last update":"Fri Feb 2 15:26:14 2024","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.73899999999999999,"15min":0.73299999999999998},"min":{"1min":0.67500000000000004,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.95999999999999996,"5min":1.5680000000000001,"15min":1.5680000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.74099999999999999,"15min":0.73799999999999999},"min":{"1min":0.64900000000000002,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":0.93899999999999995,"5min":0.999,"15min":0.999},"last":0.60499999999999998}]}]},{"osd":9,"up_from":62,"seq":266287972393,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1003520,"data_stored":500779,"data_compressed":23664,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 15:26:38 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.53600000000000003,"15min":0.54300000000000004},"min":{"1min":0.33500000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.069,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.55200000000000005,"15min":0.56499999999999995},"min":{"1min":0.32700000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.70799999999999996,"5min":1.196,"15min":1.196},"last":0.53400000000000003}]},{"osd":1,"last update":"Fri Feb 2 15:26:38 2024","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.72799999999999998,"15min":0.72199999999999998},"min":{"1min":0.51600000000000001,"5min":0.38,"15min":0.38},"max":{"1min":1.0309999999999999,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.748},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.69299999999999995,"15min":0.70299999999999996},"min":{"1min":0.46800000000000003,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.80100000000000005,"5min":1.198,"15min":1.198},"last":0.67300000000000004}]},{"osd":2,"last update":"Fri Feb 2 15:26:38 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.71299999999999997,"15min":0.72999999999999998},"min":{"1min":0.50900000000000001,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.90600000000000003,"5min":1.135,"15min":1.135},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.68700000000000006,"15min":0.68300000000000005},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"last":0.66000000000000003}]},{"osd":3,"last update":"Fri Feb 2 15:26:38 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.71499999999999997,"15min":0.71899999999999997},"min":{"1min":0.48499999999999999,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.98499999999999999,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.67900000000000005,"15min":0.71499999999999997},"min":{"1min":0.46100000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.89100000000000001,"5min":1.296,"15min":1.296},"last":0.61899999999999999}]},{"osd":4,"last update":"Fri Feb 2 15:26:38 2024","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.69899999999999995,"15min":0.71499999999999997},"min":{"1min":0.45300000000000001,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.302,"5min":1.381,"15min":1.381},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.81699999999999995,"15min":0.83199999999999996},"min":{"1min":0.55900000000000005,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.1000000000000001,"5min":1.3080000000000001,"15min":1.3080000000000001},"last":0.79400000000000004}]},{"osd":5,"last update":"Fri Feb 2 15:26:38 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.84899999999999998,"15min":0.91500000000000004},"min":{"1min":0.497,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.3680000000000001,"5min":3.895,"15min":3.895},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.84399999999999997,"15min":0.85699999999999998},"min":{"1min":0.55400000000000005,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.1679999999999999,"5min":1.5089999999999999,"15min":1.5089999999999999},"last":0.83899999999999997}]},{"osd":6,"last update":"Fri Feb 2 15:24:35 2024","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.6739999999999999,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":0.64500000000000002}]},{"osd":7,"last update":"Fri Feb 2 15:26:38 2024","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.89500000000000002,"15min":0.91300000000000003},"min":{"1min":0.69499999999999995,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.1359999999999999,"5min":1.347,"15min":1.347},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.84199999999999997,"15min":0.88600000000000001},"min":{"1min":0.55200000000000005,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.86399999999999999,"5min":1.413,"15min":1.413},"last":0.77900000000000003}]},{"osd":8,"last update":"Fri Feb 2 15:26:38 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.86799999999999999,"15min":0.91300000000000003},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.474,"5min":1.7789999999999999,"15min":1.7789999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.93600000000000005,"15min":0.96699999999999997},"min":{"1min":0.66900000000000004,"5min":0.625,"15min":0.625},"max":{"1min":1.1970000000000001,"5min":1.538,"15min":1.538},"last":0.86699999999999999}]},{"osd":10,"last update":"Fri Feb 2 15:25:56 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.93200000000000005,"15min":0.92700000000000005},"min":{"1min":0.71399999999999997,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.389,"5min":1.389,"15min":1.389},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.874,"5min":0.86199999999999999,"15min":0.85999999999999999},"min":{"1min":0.70499999999999996,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":0.78700000000000003}]},{"osd":11,"last update":"Fri Feb 2 15:25:51 2024","interfaces":[{"interface":"back","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.474,"5min":1.474,"15min":1.474},"last":0.91000000000000003}]}]},{"osd":8,"up_from":55,"seq":236223201325,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":991232,"data_stored":496730,"data_compressed":23664,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 15:26:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.57899999999999996,"15min":0.58199999999999996},"min":{"1min":0.44900000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.71099999999999997,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.58599999999999997,"15min":0.59799999999999998},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.73799999999999999,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.50600000000000001}]},{"osd":1,"last update":"Fri Feb 2 15:26:16 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.59399999999999997,"15min":0.57299999999999995},"min":{"1min":0.45900000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.83099999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.626},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.59399999999999997,"15min":0.56999999999999995},"min":{"1min":0.505,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.79500000000000004,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.64100000000000001}]},{"osd":2,"last update":"Fri Feb 2 15:26:16 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.44700000000000001,"15min":0.42599999999999999},"min":{"1min":0.34999999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.499,"5min":0.44,"15min":0.42199999999999999},"min":{"1min":0.38500000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.59699999999999998}]},{"osd":3,"last update":"Fri Feb 2 15:26:16 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.61399999999999999,"15min":0.625},"min":{"1min":0.40999999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.751,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.61299999999999999,"15min":0.62},"min":{"1min":0.435,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.58499999999999996}]},{"osd":4,"last update":"Fri Feb 2 15:26:16 2024","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.621,"15min":0.60499999999999998},"min":{"1min":0.42199999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.84599999999999997,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.61899999999999999,"15min":0.59899999999999998},"min":{"1min":0.45400000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.67800000000000005}]},{"osd":5,"last update":"Fri Feb 2 15:26:16 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.53400000000000003,"15min":0.51100000000000001},"min":{"1min":0.49399999999999999,"5min":0.311,"15min":0.311},"max":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.53400000000000003,"15min":0.51000000000000001},"min":{"1min":0.47399999999999998,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.71499999999999997}]},{"osd":6,"last update":"Fri Feb 2 15:26:16 2024","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.68300000000000005,"15min":0.67900000000000005},"min":{"1min":0.54800000000000004,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.70599999999999996,"15min":0.69799999999999995},"min":{"1min":0.56000000000000005,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.76100000000000001}]},{"osd":7,"last update":"Fri Feb 2 15:26:16 2024","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.67200000000000004,"15min":0.64900000000000002},"min":{"1min":0.62,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.69799999999999995,"15min":0.67800000000000005},"min":{"1min":0.61299999999999999,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.96099999999999997,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.752}]},{"osd":9,"last update":"Fri Feb 2 15:26:40 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.72199999999999998,"15min":0.69799999999999995},"min":{"1min":0.53600000000000003,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.86799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.74399999999999999,"15min":0.71999999999999997},"min":{"1min":0.61899999999999999,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.90300000000000002,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.83299999999999996}]},{"osd":10,"last update":"Fri Feb 2 15:25:56 2024","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.73099999999999998,"15min":0.71699999999999997},"min":{"1min":0.56200000000000006,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.94599999999999995,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.72899999999999998,"15min":0.71299999999999997},"min":{"1min":0.63600000000000001,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.80900000000000005}]},{"osd":11,"last update":"Fri Feb 2 15:25:48 2024","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.78500000000000003}]}]},{"osd":7,"up_from":48,"seq":206158430256,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1048576,"data_stored":1093556,"data_compressed":59875,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 15:23:58 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.439},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.38500000000000001}]},{"osd":1,"last update":"Fri Feb 2 15:26:01 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.432,"15min":0.42799999999999999},"min":{"1min":0.34399999999999997,"5min":0.247,"15min":0.247},"max":{"1min":0.64300000000000002,"5min":0.749,"15min":0.749},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.42199999999999999,"15min":0.41099999999999998},"min":{"1min":0.37,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.65500000000000003,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.53100000000000003}]},{"osd":2,"last update":"Fri Feb 2 15:26:01 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.57499999999999996,"15min":0.57199999999999995},"min":{"1min":0.434,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.77000000000000002,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.59599999999999997,"15min":0.58799999999999997},"min":{"1min":0.48599999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.76500000000000001,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.75600000000000001}]},{"osd":3,"last update":"Fri Feb 2 15:26:01 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.64300000000000002,"15min":0.61099999999999999},"min":{"1min":0.51000000000000001,"5min":0.374,"15min":0.374},"max":{"1min":4.3890000000000002,"5min":4.3890000000000002,"15min":4.3890000000000002},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.623,"15min":0.60499999999999998},"min":{"1min":0.47599999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":4.1280000000000001,"5min":4.1280000000000001,"15min":4.1280000000000001},"last":0.60199999999999998}]},{"osd":4,"last update":"Fri Feb 2 15:26:01 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.50600000000000001,"15min":0.501},"min":{"1min":0.38800000000000001,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.78500000000000003,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.5,"15min":0.496},"min":{"1min":0.35999999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.437}]},{"osd":5,"last update":"Fri Feb 2 15:26:01 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.64000000000000001,"15min":0.622},"min":{"1min":0.44900000000000001,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.81299999999999994,"5min":2.3370000000000002,"15min":2.3370000000000002},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.63100000000000001,"15min":0.625},"min":{"1min":0.54700000000000004,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.88600000000000001,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.63500000000000001}]},{"osd":6,"last update":"Fri Feb 2 15:26:01 2024","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.68400000000000005,"15min":0.68999999999999995},"min":{"1min":0.54400000000000004,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.96399999999999997,"5min":0.998,"15min":0.998},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.67300000000000004,"15min":0.66700000000000004},"min":{"1min":0.46400000000000002,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.95599999999999996,"5min":1.778,"15min":1.778},"last":0.66100000000000003}]},{"osd":8,"last update":"Fri Feb 2 15:26:17 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.71199999999999997,"15min":0.70299999999999996},"min":{"1min":0.53600000000000003,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.97699999999999998,"5min":2.069,"15min":2.069},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.73599999999999999,"15min":0.73499999999999999},"min":{"1min":0.65900000000000003,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.93100000000000005,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.74099999999999999}]},{"osd":9,"last update":"Fri Feb 2 15:25:40 2024","interfaces":[{"interface":"back","average":{"1min":0.84699999999999998,"5min":0.76100000000000001,"15min":0.747},"min":{"1min":0.497,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":4.9550000000000001,"5min":4.9550000000000001,"15min":4.9550000000000001},"last":0.62},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.753,"15min":0.73899999999999999},"min":{"1min":0.54500000000000004,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":4.0759999999999996,"5min":4.0759999999999996,"15min":4.0759999999999996},"last":0.67600000000000005}]},{"osd":10,"last update":"Fri Feb 2 15:25:55 2024","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"min":{"1min":0.47099999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.82499999999999996,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.495,"5min":0.436,"15min":0.436},"max":{"1min":0.88800000000000001,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.72499999999999998}]},{"osd":11,"last update":"Fri Feb 2 15:25:51 2024","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.85199999999999998}]}]},{"osd":6,"up_from":43,"seq":184683593780,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":974848,"data_stored":492139,"data_compressed":23664,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 15:25:44 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.497,"15min":0.48499999999999999},"min":{"1min":0.35599999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.67900000000000005,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51900000000000002,"15min":0.51200000000000001},"min":{"1min":0.371,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.67200000000000004,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.442}]},{"osd":1,"last update":"Fri Feb 2 15:25:44 2024","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":0.72199999999999998,"15min":0.66400000000000003},"min":{"1min":0.47999999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":6.8650000000000002,"5min":6.8650000000000002,"15min":6.8650000000000002},"last":0.626},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.66400000000000003,"15min":0.65200000000000002},"min":{"1min":0.50600000000000001,"5min":0.501,"15min":0.501},"max":{"1min":0.88900000000000001,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.501}]},{"osd":2,"last update":"Fri Feb 2 15:25:44 2024","interfaces":[{"interface":"back","average":{"1min":1.175,"5min":0.79700000000000004,"15min":0.72699999999999998},"min":{"1min":0.57199999999999995,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":6.9320000000000004,"5min":6.9320000000000004,"15min":6.9320000000000004},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.753,"15min":0.72499999999999998},"min":{"1min":0.52600000000000002,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":3.008,"5min":3.008,"15min":3.008},"last":0.68700000000000006}]},{"osd":3,"last update":"Fri Feb 2 15:25:44 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.628,"15min":0.63300000000000001},"min":{"1min":0.432,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.875,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.67400000000000004,"15min":0.66400000000000003},"min":{"1min":0.45100000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.73299999999999998}]},{"osd":4,"last update":"Fri Feb 2 15:25:44 2024","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.72999999999999998,"15min":0.71999999999999997},"min":{"1min":0.53100000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.95599999999999996,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.76200000000000001,"15min":0.75600000000000001},"min":{"1min":0.55700000000000005,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.0489999999999999,"5min":1.228,"15min":1.228},"last":0.58999999999999997}]},{"osd":5,"last update":"Fri Feb 2 15:25:44 2024","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.79100000000000004,"15min":0.77900000000000003},"min":{"1min":0.55100000000000005,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.63700000000000001},{"interface":"front","average":{"1min":1.268,"5min":0.90900000000000003,"15min":0.84499999999999997},"min":{"1min":0.61199999999999999,"5min":0.442,"15min":0.442},"max":{"1min":6.9820000000000002,"5min":6.9820000000000002,"15min":6.9820000000000002},"last":0.84199999999999997}]},{"osd":7,"last update":"Fri Feb 2 15:26:02 2024","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":0.88900000000000001,"15min":0.84999999999999998},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":6.9530000000000003,"5min":6.9530000000000003,"15min":6.9530000000000003},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.876,"15min":0.878},"min":{"1min":0.60899999999999999,"5min":0.501,"15min":0.501},"max":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":0.72099999999999997}]},{"osd":8,"last update":"Fri Feb 2 15:26:15 2024","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.88200000000000001,"15min":0.89500000000000002},"min":{"1min":0.60699999999999998,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.0429999999999999,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.90900000000000003,"15min":0.91000000000000003},"min":{"1min":0.625,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.0620000000000001,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.749}]},{"osd":9,"last update":"Fri Feb 2 15:26:41 2024","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.93300000000000005,"15min":0.94299999999999995},"min":{"1min":0.69799999999999995,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.8280000000000001,"5min":1.8280000000000001,"15min":1.8280000000000001},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.95799999999999996,"15min":0.97399999999999998},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.6299999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":0.74099999999999999}]},{"osd":10,"last update":"Fri Feb 2 15:25:56 2024","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1,"15min":0.96399999999999997},"min":{"1min":0.68300000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":6.9960000000000004,"5min":6.9960000000000004,"15min":6.9960000000000004},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.97799999999999998,"15min":0.98699999999999999},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.1879999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"last":0.88700000000000001}]},{"osd":11,"last update":"Fri Feb 2 15:25:51 2024","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.92300000000000004},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":3.032,"5min":3.032,"15min":3.032},"last":0.78500000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574918,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1048576,"data_stored":559505,"data_compressed":23664,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 15:26:16 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51100000000000001,"15min":0.50800000000000001},"min":{"1min":0.41999999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.66100000000000003,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.53400000000000003,"15min":0.52400000000000002},"min":{"1min":0.45700000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.72999999999999998,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.68999999999999995}]},{"osd":2,"last update":"Fri Feb 2 15:26:37 2024","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.58199999999999996,"15min":0.55600000000000005},"min":{"1min":0.42699999999999999,"5min":0.31,"15min":0.31},"max":{"1min":0.80600000000000005,"5min":0.871,"15min":0.871},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.59199999999999997,"15min":0.55600000000000005},"min":{"1min":0.44900000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.51500000000000001}]},{"osd":3,"last update":"Fri Feb 2 15:25:50 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.56899999999999995,"15min":0.53700000000000003},"min":{"1min":0.434,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.57999999999999996,"15min":0.55400000000000005},"min":{"1min":0.45800000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.621}]},{"osd":4,"last update":"Fri Feb 2 15:26:05 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.43099999999999999,"15min":0.38100000000000001},"min":{"1min":0.35299999999999998,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.754,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.433,"15min":0.38600000000000001},"min":{"1min":0.376,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.69799999999999995,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.47499999999999998}]},{"osd":5,"last update":"Fri Feb 2 15:26:22 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.59499999999999997,"15min":0.55300000000000005},"min":{"1min":0.52800000000000002,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.438},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.626,"15min":0.56999999999999995},"min":{"1min":0.58399999999999996,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.85699999999999998,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.64300000000000002}]},{"osd":6,"last update":"Fri Feb 2 15:25:46 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.621,"15min":0.59499999999999997},"min":{"1min":0.50800000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.60599999999999998,"15min":0.58799999999999997},"min":{"1min":0.48099999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.71499999999999997}]},{"osd":7,"last update":"Fri Feb 2 15:26:01 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.59599999999999997,"15min":0.57999999999999996},"min":{"1min":0.45000000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.90500000000000003,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.55800000000000005,"15min":0.54000000000000004},"min":{"1min":0.434,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.79100000000000004,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.48599999999999999}]},{"osd":8,"last update":"Fri Feb 2 15:26:16 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.65800000000000003,"15min":0.63800000000000001},"min":{"1min":0.51000000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.68600000000000005,"15min":0.65600000000000003},"min":{"1min":0.55100000000000005,"5min":0.442,"15min":0.442},"max":{"1min":0.94099999999999995,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.67400000000000004}]},{"osd":9,"last update":"Fri Feb 2 15:26:39 2024","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.73899999999999999,"15min":0.71399999999999997},"min":{"1min":0.47499999999999998,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":2.6840000000000002,"5min":2.6840000000000002,"15min":2.6840000000000002},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.76300000000000001,"15min":0.747},"min":{"1min":0.70999999999999996,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":0.96199999999999997,"5min":1.121,"15min":1.121},"last":0.79100000000000004}]},{"osd":10,"last update":"Fri Feb 2 15:25:56 2024","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.69499999999999995,"15min":0.68500000000000005},"min":{"1min":0.57399999999999995,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.72699999999999998,"15min":0.71499999999999997},"min":{"1min":0.57999999999999996,"5min":0.5,"15min":0.5},"max":{"1min":1.008,"5min":1.008,"15min":1.008},"last":0.73099999999999998}]},{"osd":11,"last update":"Fri Feb 2 15:25:50 2024","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.77600000000000002}]}]},{"osd":0,"up_from":8,"seq":34359738441,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":999424,"data_stored":493595,"data_compressed":23664,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Feb 2 15:26:11 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.56599999999999995,"15min":0.54600000000000004},"min":{"1min":0.374,"5min":0.314,"15min":0.314},"max":{"1min":0.79500000000000004,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.63},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.59999999999999998,"15min":0.55400000000000005},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.411,"5min":1.411,"15min":1.411},"last":0.78400000000000003}]},{"osd":2,"last update":"Fri Feb 2 15:26:32 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.70499999999999996,"15min":0.59999999999999998},"min":{"1min":0.52900000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.369,"5min":1.369,"15min":1.369},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.745,"5min":0.74299999999999999,"15min":0.628},"min":{"1min":0.373,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.9079999999999999,"5min":4.6550000000000002,"15min":4.6550000000000002},"last":0.57299999999999995}]},{"osd":3,"last update":"Fri Feb 2 15:25:48 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.495},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.7549999999999999,"5min":1.7549999999999999,"15min":1.7549999999999999},"last":0.44600000000000001}]},{"osd":4,"last update":"Fri Feb 2 15:26:05 2024","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.68400000000000005,"15min":0.64700000000000002},"min":{"1min":0.44400000000000001,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.66000000000000003,"15min":0.63200000000000001},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.8049999999999999,"5min":1.8049999999999999,"15min":1.8049999999999999},"last":0.64900000000000002}]},{"osd":5,"last update":"Fri Feb 2 15:26:22 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.69999999999999996,"15min":0.68899999999999995},"min":{"1min":0.52300000000000002,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.9450000000000001,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":0.748},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.71699999999999997,"15min":0.67800000000000005},"min":{"1min":0.44400000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.98599999999999999,"5min":4.5890000000000004,"15min":4.5890000000000004},"last":0.47999999999999998}]},{"osd":6,"last update":"Fri Feb 2 15:25:43 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.72899999999999998,"15min":0.70499999999999996},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.494,"5min":1.494,"15min":1.494},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.74099999999999999,"15min":0.73099999999999998},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.872}]},{"osd":7,"last update":"Fri Feb 2 15:25:55 2024","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.79800000000000004}]},{"osd":8,"last update":"Fri Feb 2 15:26:22 2024","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.84899999999999998,"15min":0.84299999999999997},"min":{"1min":0.65900000000000003,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":2.0169999999999999,"5min":2.0169999999999999,"15min":2.0169999999999999},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.79200000000000004,"15min":0.77400000000000002},"min":{"1min":0.57199999999999995,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.1479999999999999,"5min":1.155,"15min":1.155},"last":0.76800000000000002}]},{"osd":9,"last update":"Fri Feb 2 15:25:40 2024","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.96399999999999997,"15min":0.98699999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0980000000000001,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.95399999999999996,"15min":0.96799999999999997},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.411,"5min":1.411,"15min":1.411},"last":0.92000000000000004}]},{"osd":10,"last update":"Fri Feb 2 15:26:01 2024","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.879,"15min":0.878},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.3460000000000001,"5min":1.3460000000000001,"15min":1.3460000000000001},"last":0.84399999999999997},{"interface":"front","average":{"1min":1,"5min":0.97199999999999998,"15min":0.96799999999999997},"min":{"1min":0.66100000000000003,"5min":0.63,"15min":0.63},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":0.96099999999999997}]},{"osd":11,"last update":"Fri Feb 2 15:26:11 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.93500000000000005,"15min":0.93899999999999995},"min":{"1min":0.73299999999999998,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":0.88400000000000001},{"interface":"front","average":{"1min":1.008,"5min":0.97499999999999998,"15min":0.96899999999999997},"min":{"1min":0.81100000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.6000000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.98299999999999998}]}]},{"osd":2,"up_from":18,"seq":77309411395,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1060864,"data_stored":559609,"data_compressed":23664,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 15:26:29 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.55500000000000005,"15min":0.54600000000000004},"min":{"1min":0.45200000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.79500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.53800000000000003,"15min":0.52400000000000002},"min":{"1min":0.40899999999999997,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.85999999999999999,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.53300000000000003}]},{"osd":1,"last update":"Fri Feb 2 15:26:29 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.55100000000000005,"15min":0.51300000000000001},"min":{"1min":0.35299999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.496},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.59199999999999997,"15min":0.54300000000000004},"min":{"1min":0.41199999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.86299999999999999,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.76200000000000001}]},{"osd":3,"last update":"Fri Feb 2 15:25:46 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.60099999999999998,"15min":0.57199999999999995},"min":{"1min":0.46100000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.75600000000000001,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.623,"5min":0.58799999999999997,"15min":0.55600000000000005},"min":{"1min":0.42799999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.81000000000000005,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.60899999999999999}]},{"osd":4,"last update":"Fri Feb 2 15:25:50 2024","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.65600000000000003}]},{"osd":5,"last update":"Fri Feb 2 15:26:29 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.45700000000000002,"15min":0.41799999999999998},"min":{"1min":0.36699999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.78900000000000003,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.44700000000000001,"15min":0.41499999999999998},"min":{"1min":0.36799999999999999,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.747,"5min":0.747,"15min":0.747},"last":0.63600000000000001}]},{"osd":6,"last update":"Fri Feb 2 15:25:40 2024","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.625,"15min":0.61899999999999999},"min":{"1min":0.46100000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.79500000000000004,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.628,"5min":0.625,"15min":0.627},"min":{"1min":0.437,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.746}]},{"osd":7,"last update":"Fri Feb 2 15:25:52 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.78900000000000003}]},{"osd":8,"last update":"Fri Feb 2 15:26:18 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.59099999999999997,"15min":0.57699999999999996},"min":{"1min":0.40500000000000003,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.55700000000000005,"15min":0.54300000000000004},"min":{"1min":0.41999999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.70799999999999996}]},{"osd":9,"last update":"Fri Feb 2 15:25:40 2024","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.66900000000000004,"15min":0.66700000000000004},"min":{"1min":0.501,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.751,"15min":0.746},"min":{"1min":0.60299999999999998,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":0.92500000000000004,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.81399999999999995}]},{"osd":10,"last update":"Fri Feb 2 15:25:52 2024","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.68300000000000005,"15min":0.67200000000000004},"min":{"1min":0.628,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.68100000000000005,"15min":0.67100000000000004},"min":{"1min":0.56299999999999994,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.86799999999999999,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.86599999999999999}]},{"osd":11,"last update":"Fri Feb 2 15:25:46 2024","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.89900000000000002}]}]},{"osd":3,"up_from":24,"seq":103079215166,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":987136,"data_stored":491895,"data_compressed":23664,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 15:25:44 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.442,"15min":0.41899999999999998},"min":{"1min":0.34899999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.69699999999999995,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.46600000000000003,"15min":0.45300000000000001},"min":{"1min":0.33300000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.73099999999999998,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.442}]},{"osd":1,"last update":"Fri Feb 2 15:25:44 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.65000000000000002,"15min":0.59599999999999997},"min":{"1min":0.52900000000000003,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.0649999999999999,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.622,"15min":0.59899999999999998},"min":{"1min":0.49099999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.879,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.52400000000000002}]},{"osd":2,"last update":"Fri Feb 2 15:25:44 2024","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.63300000000000001,"15min":0.623},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.625},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.70299999999999996,"15min":0.65700000000000003},"min":{"1min":0.56799999999999995,"5min":0.443,"15min":0.443},"max":{"1min":1.1319999999999999,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":0.504}]},{"osd":4,"last update":"Fri Feb 2 15:26:01 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.62,"15min":0.624},"min":{"1min":0.495,"5min":0.376,"15min":0.376},"max":{"1min":0.873,"5min":1.022,"15min":1.022},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.72599999999999998,"15min":0.69799999999999995},"min":{"1min":0.49199999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.75900000000000001}]},{"osd":5,"last update":"Fri Feb 2 15:26:25 2024","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.748,"15min":0.72599999999999998},"min":{"1min":0.57699999999999996,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.88600000000000001,"5min":1.9099999999999999,"15min":1.9099999999999999},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.66400000000000003,"15min":0.64700000000000002},"min":{"1min":0.50900000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.84399999999999997,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.69699999999999995}]},{"osd":6,"last update":"Fri Feb 2 15:25:44 2024","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.74199999999999999,"15min":0.71599999999999997},"min":{"1min":0.57099999999999995,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.1100000000000001,"5min":1.1539999999999999,"15min":1.1539999999999999},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.68999999999999995,"15min":0.68799999999999994},"min":{"1min":0.40500000000000003,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.90600000000000003,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.58199999999999996}]},{"osd":7,"last update":"Fri Feb 2 15:26:01 2024","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.751,"15min":0.74099999999999999},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.008,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.501,"5min":0.44,"15min":0.44},"max":{"1min":0.89900000000000002,"5min":1.0920000000000001,"15min":1.0920000000000001},"last":0.74199999999999999}]},{"osd":8,"last update":"Fri Feb 2 15:24:14 2024","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.65800000000000003}]},{"osd":9,"last update":"Fri Feb 2 15:26:38 2024","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.91100000000000003,"15min":0.94199999999999995},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":0.93700000000000006,"5min":1.2,"15min":1.2},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.81999999999999995,"15min":0.84299999999999997},"min":{"1min":0.55400000000000005,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0629999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.77300000000000002}]},{"osd":10,"last update":"Fri Feb 2 15:25:59 2024","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.87,"15min":0.877},"min":{"1min":0.65500000000000003,"5min":0.624,"15min":0.624},"max":{"1min":1.0840000000000001,"5min":1.1379999999999999,"15min":1.1379999999999999},"last":0.83799999999999997},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.95699999999999996,"15min":0.96699999999999997},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.83299999999999996}]},{"osd":11,"last update":"Fri Feb 2 15:25:49 2024","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.256,"5min":1.256,"15min":1.256},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.68300000000000005}]}]},{"osd":4,"up_from":31,"seq":133143986237,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":496746,"data_compressed":23664,"data_compressed_allocated":32768,"data_compressed_original":65536,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 15:24:03 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.52300000000000002,"15min":0.52100000000000002},"min":{"1min":0.371,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.78600000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.53800000000000003,"15min":0.53600000000000003},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.47799999999999998}]},{"osd":1,"last update":"Fri Feb 2 15:26:06 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.376,"15min":0.34300000000000003},"min":{"1min":0.33300000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.55000000000000004,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.439},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.38700000000000001,"15min":0.35499999999999998},"min":{"1min":0.29899999999999999,"5min":0.255,"15min":0.255},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.47999999999999998}]},{"osd":2,"last update":"Fri Feb 2 15:26:06 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.54600000000000004,"15min":0.51400000000000001},"min":{"1min":0.438,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.54700000000000004,"15min":0.51700000000000002},"min":{"1min":0.52500000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.67900000000000005,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.63300000000000001}]},{"osd":3,"last update":"Fri Feb 2 15:26:06 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.61099999999999999,"15min":0.57899999999999996},"min":{"1min":0.40200000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":4.3220000000000001,"5min":4.3220000000000001,"15min":4.3220000000000001},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.60699999999999998,"15min":0.57599999999999996},"min":{"1min":0.436,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":4.3959999999999999,"5min":4.3959999999999999,"15min":4.3959999999999999},"last":0.58699999999999997}]},{"osd":5,"last update":"Fri Feb 2 15:26:24 2024","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.63200000000000001,"15min":0.59599999999999997},"min":{"1min":0.59099999999999997,"5min":0.374,"15min":0.374},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.626,"15min":0.60799999999999998},"min":{"1min":0.53600000000000003,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.81599999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.78300000000000003}]},{"osd":6,"last update":"Fri Feb 2 15:26:44 2024","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.69899999999999995,"15min":0.65300000000000002},"min":{"1min":0.45000000000000001,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":5.6840000000000002,"5min":5.6840000000000002,"15min":5.6840000000000002},"last":0.66500000000000004},{"interface":"front","average":{"1min":1.2490000000000001,"5min":0.77800000000000002,"15min":0.68300000000000005},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":5.9100000000000001,"5min":5.9100000000000001,"15min":5.9100000000000001},"last":0.67400000000000004}]},{"osd":7,"last update":"Fri Feb 2 15:26:00 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.497,"15min":0.48499999999999999},"min":{"1min":0.34000000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.747,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.54500000000000004,"15min":0.53500000000000003},"min":{"1min":0.46200000000000002,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.76000000000000001,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.60899999999999999}]},{"osd":8,"last update":"Fri Feb 2 15:26:19 2024","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.66200000000000003,"15min":0.65400000000000003},"min":{"1min":0.55100000000000005,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.877,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.753,"5min":0.67700000000000005,"15min":0.65800000000000003},"min":{"1min":0.5,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.83499999999999996}]},{"osd":9,"last update":"Fri Feb 2 15:26:39 2024","interfaces":[{"interface":"back","average":{"1min":1.78,"5min":0.93000000000000005,"15min":0.79300000000000004},"min":{"1min":0.48799999999999999,"5min":0.441,"15min":0.441},"max":{"1min":20.126999999999999,"5min":20.126999999999999,"15min":20.126999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":1.5409999999999999,"5min":0.89200000000000002,"15min":0.77900000000000003},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":14.965999999999999,"5min":14.965999999999999,"15min":14.965999999999999},"last":0.76700000000000002}]},{"osd":10,"last update":"Fri Feb 2 15:25:57 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.67700000000000005,"15min":0.67300000000000004},"min":{"1min":0.48199999999999998,"5min":0.379,"15min":0.379},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.751},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.66800000000000004,"15min":0.66700000000000004},"min":{"1min":0.55200000000000005,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.79100000000000004,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.69999999999999996}]},{"osd":11,"last update":"Fri Feb 2 15:25:48 2024","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.82499999999999996}]}]},{"osd":5,"up_from":36,"seq":154618822712,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":1088701,"data_compressed":59875,"data_compressed_allocated":77824,"data_compressed_original":655360,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Feb 2 15:26:23 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.53800000000000003,"15min":0.51800000000000002},"min":{"1min":0.30399999999999999,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.72399999999999998,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.505},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.54000000000000004,"15min":0.52600000000000002},"min":{"1min":0.33500000000000002,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.67100000000000004,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.52700000000000002}]},{"osd":1,"last update":"Fri Feb 2 15:26:23 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.56200000000000006,"15min":0.55000000000000004},"min":{"1min":0.318,"5min":0.252,"15min":0.252},"max":{"1min":0.69099999999999995,"5min":1.018,"15min":1.018},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.57699999999999996,"15min":0.55100000000000005},"min":{"1min":0.40100000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.78000000000000003,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.65100000000000002}]},{"osd":2,"last update":"Fri Feb 2 15:26:23 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.40699999999999997,"15min":0.38700000000000001},"min":{"1min":0.30599999999999999,"5min":0.215,"15min":0.215},"max":{"1min":0.59099999999999997,"5min":0.63,"15min":0.63},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.42199999999999999,"15min":0.40400000000000003},"min":{"1min":0.32900000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.64900000000000002,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.59999999999999998}]},{"osd":3,"last update":"Fri Feb 2 15:26:23 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.56000000000000005,"15min":0.52700000000000002},"min":{"1min":0.434,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.69899999999999995,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.58499999999999996,"15min":0.56699999999999995},"min":{"1min":0.44700000000000001,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.74399999999999999,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.76500000000000001}]},{"osd":4,"last update":"Fri Feb 2 15:26:23 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.59099999999999997,"15min":0.56699999999999995},"min":{"1min":0.45600000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.77200000000000002,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.59899999999999998,"15min":0.57999999999999996},"min":{"1min":0.42199999999999999,"5min":0.316,"15min":0.316},"max":{"1min":0.79900000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.89700000000000002}]},{"osd":6,"last update":"Fri Feb 2 15:26:42 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.628,"15min":0.63},"min":{"1min":0.47799999999999998,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.84299999999999997,"5min":0.878,"15min":0.878},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.629,"15min":0.63100000000000001},"min":{"1min":0.46500000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.75600000000000001,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.86499999999999999}]},{"osd":7,"last update":"Fri Feb 2 15:26:01 2024","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.64200000000000002,"15min":0.626},"min":{"1min":0.52700000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.621,"15min":0.60899999999999999},"min":{"1min":0.49299999999999999,"5min":0.373,"15min":0.373},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.94999999999999996}]},{"osd":8,"last update":"Fri Feb 2 15:26:18 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.57299999999999995,"15min":0.56100000000000005},"min":{"1min":0.43099999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.84499999999999997,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.60199999999999998,"15min":0.59699999999999998},"min":{"1min":0.379,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.92700000000000005,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.83499999999999996}]},{"osd":9,"last update":"Fri Feb 2 15:26:38 2024","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.69999999999999996,"15min":0.68600000000000005},"min":{"1min":0.52400000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.86499999999999999,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.96999999999999997},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71599999999999997,"15min":0.70599999999999996},"min":{"1min":0.54500000000000004,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.86599999999999999,"5min":0.92700000000000005,"15min":0.92700000000000005},"last":0.91400000000000003}]},{"osd":10,"last update":"Fri Feb 2 15:25:54 2024","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.70899999999999996,"15min":0.69999999999999996},"min":{"1min":0.57699999999999996,"5min":0.497,"15min":0.497},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.72799999999999998,"15min":0.72299999999999998},"min":{"1min":0.54000000000000004,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1,"5min":1,"15min":1},"last":0.98399999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34572,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34572,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36211,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36211,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36211,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36211,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36211,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":68040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":68144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":63178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-02T15:26:46.742 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T15:26:46.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.975+0000 7f3d89c00700 1 -- 172.21.15.1:0/1457161390 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3d84100d50 msgr2=0x7f3d84101170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:46.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.975+0000 7f3d89c00700 1 --2- 172.21.15.1:0/1457161390 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3d84100d50 0x7f3d84101170 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f3d740099c0 tx=0x7f3d74009cd0 comp rx=0 tx=0).stop 2024-02-02T15:26:46.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.975+0000 7f3d89c00700 1 -- 172.21.15.1:0/1457161390 shutdown_connections 2024-02-02T15:26:46.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.975+0000 7f3d89c00700 1 --2- 172.21.15.1:0/1457161390 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d841028e0 0x7f3d84109190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:46.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.975+0000 7f3d89c00700 1 --2- 172.21.15.1:0/1457161390 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3d84100d50 0x7f3d84101170 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:46.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.975+0000 7f3d89c00700 1 --2- 172.21.15.1:0/1457161390 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d84101f40 0x7f3d841023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:46.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.975+0000 7f3d89c00700 1 -- 172.21.15.1:0/1457161390 >> 172.21.15.1:0/1457161390 conn(0x7f3d840fc2c0 msgr2=0x7f3d840fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:46.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.975+0000 7f3d89c00700 1 -- 172.21.15.1:0/1457161390 shutdown_connections 2024-02-02T15:26:46.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.975+0000 7f3d89c00700 1 -- 172.21.15.1:0/1457161390 wait complete. 2024-02-02T15:26:46.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.976+0000 7f3d89c00700 1 Processor -- start 2024-02-02T15:26:46.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.976+0000 7f3d89c00700 1 -- start start 2024-02-02T15:26:46.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.977+0000 7f3d89c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d84100d50 0x7f3d84196a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:46.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.977+0000 7f3d89c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3d84101f40 0x7f3d84196f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:46.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.977+0000 7f3d89c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d841028e0 0x7f3d8419dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:46.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.977+0000 7f3d89c00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8419e5c0 con 0x7f3d84101f40 2024-02-02T15:26:46.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.977+0000 7f3d89c00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8419e700 con 0x7f3d84100d50 2024-02-02T15:26:46.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.977+0000 7f3d89c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d8419e890 con 0x7f3d841028e0 2024-02-02T15:26:46.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.977+0000 7f3d82a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d84100d50 0x7f3d84196a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:46.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.977+0000 7f3d82a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d84100d50 0x7f3d84196a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42216/0 (socket says 172.21.15.1:42216) 2024-02-02T15:26:46.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.977+0000 7f3d82a00700 1 -- 172.21.15.1:0/2625229927 learned_addr learned my addr 172.21.15.1:0/2625229927 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:46.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.977+0000 7f3d82000700 1 --2- 172.21.15.1:0/2625229927 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3d84101f40 0x7f3d84196f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:46.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.977+0000 7f3d83400700 1 --2- 172.21.15.1:0/2625229927 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d841028e0 0x7f3d8419dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:46.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.977+0000 7f3d82a00700 1 -- 172.21.15.1:0/2625229927 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d841028e0 msgr2=0x7f3d8419dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:46.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.977+0000 7f3d82a00700 1 --2- 172.21.15.1:0/2625229927 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d841028e0 0x7f3d8419dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:46.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.977+0000 7f3d82a00700 1 -- 172.21.15.1:0/2625229927 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3d84101f40 msgr2=0x7f3d84196f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:46.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.977+0000 7f3d82a00700 1 --2- 172.21.15.1:0/2625229927 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3d84101f40 0x7f3d84196f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:46.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.977+0000 7f3d82a00700 1 -- 172.21.15.1:0/2625229927 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d740095f0 con 0x7f3d84100d50 2024-02-02T15:26:46.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.977+0000 7f3d83400700 1 --2- 172.21.15.1:0/2625229927 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d841028e0 0x7f3d8419dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:26:46.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.977+0000 7f3d82000700 1 --2- 172.21.15.1:0/2625229927 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3d84101f40 0x7f3d84196f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:26:46.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.978+0000 7f3d82a00700 1 --2- 172.21.15.1:0/2625229927 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d84100d50 0x7f3d84196a40 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3d74009990 tx=0x7f3d7400f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:46.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.978+0000 7f3d7b400700 1 -- 172.21.15.1:0/2625229927 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d7401d070 con 0x7f3d84100d50 2024-02-02T15:26:46.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.978+0000 7f3d7b400700 1 -- 172.21.15.1:0/2625229927 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d740054f0 con 0x7f3d84100d50 2024-02-02T15:26:46.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.978+0000 7f3d89c00700 1 -- 172.21.15.1:0/2625229927 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d8419eb10 con 0x7f3d84100d50 2024-02-02T15:26:46.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.978+0000 7f3d7b400700 1 -- 172.21.15.1:0/2625229927 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d74004b00 con 0x7f3d84100d50 2024-02-02T15:26:46.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.978+0000 7f3d89c00700 1 -- 172.21.15.1:0/2625229927 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d8419ef30 con 0x7f3d84100d50 2024-02-02T15:26:46.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.979+0000 7f3d89c00700 1 -- 172.21.15.1:0/2625229927 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d84107cd0 con 0x7f3d84100d50 2024-02-02T15:26:46.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.981+0000 7f3d7b400700 1 -- 172.21.15.1:0/2625229927 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 135069+0+0 (secure 0 0 0) 0x7f3d7400fb80 con 0x7f3d84100d50 2024-02-02T15:26:46.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.982+0000 7f3d7b400700 1 --2- 172.21.15.1:0/2625229927 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f3d640a0520 0x7f3d640a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:46.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.982+0000 7f3d7b400700 1 -- 172.21.15.1:0/2625229927 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f3d740ccb30 con 0x7f3d84100d50 2024-02-02T15:26:46.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.983+0000 7f3d82000700 1 --2- 172.21.15.1:0/2625229927 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f3d640a0520 0x7f3d640a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:46.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.983+0000 7f3d82000700 1 --2- 172.21.15.1:0/2625229927 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f3d640a0520 0x7f3d640a29e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c011490 tx=0x7f3d6c009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:46.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:46.987+0000 7f3d7b400700 1 -- 172.21.15.1:0/2625229927 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f3d74096390 con 0x7f3d84100d50 2024-02-02T15:26:47.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.202+0000 7f3d89c00700 1 -- 172.21.15.1:0/2625229927 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3d8410c5b0 con 0x7f3d84100d50 2024-02-02T15:26:47.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.204+0000 7f3d7b400700 1 -- 172.21.15.1:0/2625229927 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+24453 (secure 0 0 0) 0x7f3d740999b0 con 0x7f3d84100d50 2024-02-02T15:26:47.203 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:26:47.203 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":88,"fsid":"371a409c-c1de-11ee-95b6-87774f69a715","created":"2024-02-02T15:18:26.746506+0000","modified":"2024-02-02T15:26:28.267447+0000","last_up_change":"2024-02-02T15:24:08.800951+0000","last_in_change":"2024-02-02T15:23:52.784280+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T15:21:22.471754+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-02T15:24:46.195779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-02T15:24:47.226403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-02T15:24:50.300462+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"73e8a07f-3560-4c19-85ed-030d4d8f0f23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6803","nonce":2591906423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6805","nonce":2591906423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6809","nonce":2591906423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6807","nonce":2591906423}]},"public_addr":"172.21.15.1:6803/2591906423","cluster_addr":"172.21.15.1:6805/2591906423","heartbeat_back_addr":"172.21.15.1:6809/2591906423","heartbeat_front_addr":"172.21.15.1:6807/2591906423","state":["exists","up"]},{"osd":1,"uuid":"e53cffec-0ff9-4ed9-8dcf-ff47d04db2bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6801","nonce":747581719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6803","nonce":747581719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6807","nonce":747581719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6805","nonce":747581719}]},"public_addr":"172.21.15.71:6801/747581719","cluster_addr":"172.21.15.71:6803/747581719","heartbeat_back_addr":"172.21.15.71:6807/747581719","heartbeat_front_addr":"172.21.15.71:6805/747581719","state":["exists","up"]},{"osd":2,"uuid":"856c2347-294c-482b-8595-ad77f4792db4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6801","nonce":377511414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6803","nonce":377511414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6807","nonce":377511414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6805","nonce":377511414}]},"public_addr":"172.21.15.142:6801/377511414","cluster_addr":"172.21.15.142:6803/377511414","heartbeat_back_addr":"172.21.15.142:6807/377511414","heartbeat_front_addr":"172.21.15.142:6805/377511414","state":["exists","up"]},{"osd":3,"uuid":"4b946ad2-2da9-4e0b-9d61-d039d8669a42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6811","nonce":1086124477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6813","nonce":1086124477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6817","nonce":1086124477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6815","nonce":1086124477}]},"public_addr":"172.21.15.1:6811/1086124477","cluster_addr":"172.21.15.1:6813/1086124477","heartbeat_back_addr":"172.21.15.1:6817/1086124477","heartbeat_front_addr":"172.21.15.1:6815/1086124477","state":["exists","up"]},{"osd":4,"uuid":"43e36c43-47a0-4634-a45b-1da14a0e19d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6809","nonce":3095380305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6811","nonce":3095380305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6815","nonce":3095380305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6813","nonce":3095380305}]},"public_addr":"172.21.15.71:6809/3095380305","cluster_addr":"172.21.15.71:6811/3095380305","heartbeat_back_addr":"172.21.15.71:6815/3095380305","heartbeat_front_addr":"172.21.15.71:6813/3095380305","state":["exists","up"]},{"osd":5,"uuid":"eb48160e-ea7c-42f7-9477-be110bc8a34e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6809","nonce":4195149262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6811","nonce":4195149262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6815","nonce":4195149262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6813","nonce":4195149262}]},"public_addr":"172.21.15.142:6809/4195149262","cluster_addr":"172.21.15.142:6811/4195149262","heartbeat_back_addr":"172.21.15.142:6815/4195149262","heartbeat_front_addr":"172.21.15.142:6813/4195149262","state":["exists","up"]},{"osd":6,"uuid":"0ec4501a-5853-4ca6-b325-231f5b72335e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6819","nonce":1560153428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6821","nonce":1560153428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6825","nonce":1560153428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6823","nonce":1560153428}]},"public_addr":"172.21.15.1:6819/1560153428","cluster_addr":"172.21.15.1:6821/1560153428","heartbeat_back_addr":"172.21.15.1:6825/1560153428","heartbeat_front_addr":"172.21.15.1:6823/1560153428","state":["exists","up"]},{"osd":7,"uuid":"029debe8-9cdf-44f6-9aea-cabd1d185f93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6817","nonce":4080955538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6819","nonce":4080955538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6823","nonce":4080955538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6821","nonce":4080955538}]},"public_addr":"172.21.15.71:6817/4080955538","cluster_addr":"172.21.15.71:6819/4080955538","heartbeat_back_addr":"172.21.15.71:6823/4080955538","heartbeat_front_addr":"172.21.15.71:6821/4080955538","state":["exists","up"]},{"osd":8,"uuid":"7c90896c-9955-4c1b-aaca-7531dd5457f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6817","nonce":3942038209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6819","nonce":3942038209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6823","nonce":3942038209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6821","nonce":3942038209}]},"public_addr":"172.21.15.142:6817/3942038209","cluster_addr":"172.21.15.142:6819/3942038209","heartbeat_back_addr":"172.21.15.142:6823/3942038209","heartbeat_front_addr":"172.21.15.142:6821/3942038209","state":["exists","up"]},{"osd":9,"uuid":"7e165e68-31a8-4598-a577-de3e90ae5c5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6827","nonce":2716544850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6829","nonce":2716544850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6833","nonce":2716544850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6831","nonce":2716544850}]},"public_addr":"172.21.15.1:6827/2716544850","cluster_addr":"172.21.15.1:6829/2716544850","heartbeat_back_addr":"172.21.15.1:6833/2716544850","heartbeat_front_addr":"172.21.15.1:6831/2716544850","state":["exists","up"]},{"osd":10,"uuid":"48cd5fdb-0887-444a-aab8-a1de24282bb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6825","nonce":84691841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6827","nonce":84691841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6831","nonce":84691841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6829","nonce":84691841}]},"public_addr":"172.21.15.71:6825/84691841","cluster_addr":"172.21.15.71:6827/84691841","heartbeat_back_addr":"172.21.15.71:6831/84691841","heartbeat_front_addr":"172.21.15.71:6829/84691841","state":["exists","up"]},{"osd":11,"uuid":"f011dd27-11b0-4166-adfe-76756533163a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6825","nonce":2290179934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6827","nonce":2290179934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6831","nonce":2290179934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6829","nonce":2290179934}]},"public_addr":"172.21.15.142:6825/2290179934","cluster_addr":"172.21.15.142:6827/2290179934","heartbeat_back_addr":"172.21.15.142:6831/2290179934","heartbeat_front_addr":"172.21.15.142:6829/2290179934","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:20:44.060692+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:01.903817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:19.368258+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:40.028037+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:57.513361+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:22:14.805944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:22:35.188439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:22:53.014412+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:23:10.507420+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:23:31.849765+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:23:50.070612+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:24:07.388625+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":5,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/4092532060":"2024-02-03T15:19:25.836171+0000","172.21.15.1:6801/395498296":"2024-02-03T15:19:25.836171+0000","172.21.15.1:6800/395498296":"2024-02-03T15:19:25.836171+0000","172.21.15.1:0/1575373489":"2024-02-03T15:19:25.836171+0000","172.21.15.1:0/865983374":"2024-02-03T15:19:01.431596+0000","172.21.15.1:0/3165960227":"2024-02-03T15:19:01.431596+0000","172.21.15.1:0/2350830293":"2024-02-03T15:19:01.431596+0000","172.21.15.1:0/2959690250":"2024-02-03T15:19:25.836171+0000","172.21.15.1:6800/1162530732":"2024-02-03T15:19:01.431596+0000","172.21.15.1:6801/1162530732":"2024-02-03T15:19:01.431596+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T15:26:47.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.206+0000 7f3d89c00700 1 -- 172.21.15.1:0/2625229927 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f3d640a0520 msgr2=0x7f3d640a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:47.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.206+0000 7f3d89c00700 1 --2- 172.21.15.1:0/2625229927 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f3d640a0520 0x7f3d640a29e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c011490 tx=0x7f3d6c009260 comp rx=0 tx=0).stop 2024-02-02T15:26:47.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.206+0000 7f3d89c00700 1 -- 172.21.15.1:0/2625229927 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d84100d50 msgr2=0x7f3d84196a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:47.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.206+0000 7f3d89c00700 1 --2- 172.21.15.1:0/2625229927 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d84100d50 0x7f3d84196a40 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3d74009990 tx=0x7f3d7400f690 comp rx=0 tx=0).stop 2024-02-02T15:26:47.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.206+0000 7f3d89c00700 1 -- 172.21.15.1:0/2625229927 shutdown_connections 2024-02-02T15:26:47.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.206+0000 7f3d89c00700 1 --2- 172.21.15.1:0/2625229927 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f3d640a0520 0x7f3d640a29e0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:47.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.206+0000 7f3d89c00700 1 --2- 172.21.15.1:0/2625229927 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3d841028e0 0x7f3d8419dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:47.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.206+0000 7f3d89c00700 1 --2- 172.21.15.1:0/2625229927 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3d84100d50 0x7f3d84196a40 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:47.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.206+0000 7f3d89c00700 1 --2- 172.21.15.1:0/2625229927 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3d84101f40 0x7f3d84196f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:47.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.206+0000 7f3d89c00700 1 -- 172.21.15.1:0/2625229927 >> 172.21.15.1:0/2625229927 conn(0x7f3d840fc2c0 msgr2=0x7f3d840fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:47.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.207+0000 7f3d89c00700 1 -- 172.21.15.1:0/2625229927 shutdown_connections 2024-02-02T15:26:47.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.207+0000 7f3d89c00700 1 -- 172.21.15.1:0/2625229927 wait complete. 2024-02-02T15:26:47.224 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-02T15:21:22.471754+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-02T15:24:46.195779+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-02T15:24:47.226403+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-02-02T15:24:50.300462+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-02-02T15:26:47.224 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-02T15:26:47.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.430+0000 7f0e6da00700 1 -- 172.21.15.1:0/3031395237 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e68102930 msgr2=0x7f0e68109250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:47.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.430+0000 7f0e6da00700 1 --2- 172.21.15.1:0/3031395237 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e68102930 0x7f0e68109250 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f0e6000b0a0 tx=0x7f0e6000b3b0 comp rx=0 tx=0).stop 2024-02-02T15:26:47.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.430+0000 7f0e6da00700 1 -- 172.21.15.1:0/3031395237 shutdown_connections 2024-02-02T15:26:47.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.430+0000 7f0e6da00700 1 --2- 172.21.15.1:0/3031395237 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e68102930 0x7f0e68109250 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:47.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.430+0000 7f0e6da00700 1 --2- 172.21.15.1:0/3031395237 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e68100e20 0x7f0e68101240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:47.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.430+0000 7f0e6da00700 1 --2- 172.21.15.1:0/3031395237 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e68101f70 0x7f0e681023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:47.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.430+0000 7f0e6da00700 1 -- 172.21.15.1:0/3031395237 >> 172.21.15.1:0/3031395237 conn(0x7f0e680fc390 msgr2=0x7f0e680fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:47.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.430+0000 7f0e6da00700 1 -- 172.21.15.1:0/3031395237 shutdown_connections 2024-02-02T15:26:47.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.430+0000 7f0e6da00700 1 -- 172.21.15.1:0/3031395237 wait complete. 2024-02-02T15:26:47.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.431+0000 7f0e6da00700 1 Processor -- start 2024-02-02T15:26:47.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.432+0000 7f0e6da00700 1 -- start start 2024-02-02T15:26:47.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.432+0000 7f0e6da00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e68100e20 0x7f0e68196be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:47.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.432+0000 7f0e6da00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e68101f70 0x7f0e68197120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:47.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.432+0000 7f0e6da00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e68102930 0x7f0e6819e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:47.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.432+0000 7f0e6da00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6819e760 con 0x7f0e68100e20 2024-02-02T15:26:47.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.432+0000 7f0e6da00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6819e8a0 con 0x7f0e68102930 2024-02-02T15:26:47.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.432+0000 7f0e6da00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6819ea30 con 0x7f0e68101f70 2024-02-02T15:26:47.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.432+0000 7f0e66a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e68100e20 0x7f0e68196be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:47.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.433+0000 7f0e66a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e68100e20 0x7f0e68196be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53448/0 (socket says 172.21.15.1:53448) 2024-02-02T15:26:47.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.433+0000 7f0e66a00700 1 -- 172.21.15.1:0/376667784 learned_addr learned my addr 172.21.15.1:0/376667784 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:47.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.433+0000 7f0e66000700 1 --2- 172.21.15.1:0/376667784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e68101f70 0x7f0e68197120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:47.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.433+0000 7f0e67400700 1 --2- 172.21.15.1:0/376667784 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e68102930 0x7f0e6819e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:47.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.433+0000 7f0e66a00700 1 -- 172.21.15.1:0/376667784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e68101f70 msgr2=0x7f0e68197120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:47.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.433+0000 7f0e66a00700 1 --2- 172.21.15.1:0/376667784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e68101f70 0x7f0e68197120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:47.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.433+0000 7f0e66a00700 1 -- 172.21.15.1:0/376667784 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e68102930 msgr2=0x7f0e6819e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:47.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.433+0000 7f0e66a00700 1 --2- 172.21.15.1:0/376667784 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e68102930 0x7f0e6819e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:47.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.433+0000 7f0e66a00700 1 -- 172.21.15.1:0/376667784 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e60009d00 con 0x7f0e68100e20 2024-02-02T15:26:47.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.433+0000 7f0e66a00700 1 --2- 172.21.15.1:0/376667784 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e68100e20 0x7f0e68196be0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0e5800e9a0 tx=0x7f0e5800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:47.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.433+0000 7f0e5f400700 1 -- 172.21.15.1:0/376667784 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e58009790 con 0x7f0e68100e20 2024-02-02T15:26:47.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.433+0000 7f0e5f400700 1 -- 172.21.15.1:0/376667784 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e58010430 con 0x7f0e68100e20 2024-02-02T15:26:47.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.433+0000 7f0e6da00700 1 -- 172.21.15.1:0/376667784 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e6819ed10 con 0x7f0e68100e20 2024-02-02T15:26:47.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.433+0000 7f0e5f400700 1 -- 172.21.15.1:0/376667784 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e58018680 con 0x7f0e68100e20 2024-02-02T15:26:47.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.434+0000 7f0e6da00700 1 -- 172.21.15.1:0/376667784 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e6819f1c0 con 0x7f0e68100e20 2024-02-02T15:26:47.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.435+0000 7f0e6da00700 1 -- 172.21.15.1:0/376667784 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e68107ca0 con 0x7f0e68100e20 2024-02-02T15:26:47.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.436+0000 7f0e5f400700 1 -- 172.21.15.1:0/376667784 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) v1 ==== 135069+0+0 (secure 0 0 0) 0x7f0e580105a0 con 0x7f0e68100e20 2024-02-02T15:26:47.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.438+0000 7f0e5f400700 1 --2- 172.21.15.1:0/376667784 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f0e480a05a0 0x7f0e480a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:47.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.438+0000 7f0e5f400700 1 -- 172.21.15.1:0/376667784 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f0e58014070 con 0x7f0e68100e20 2024-02-02T15:26:47.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.438+0000 7f0e66000700 1 --2- 172.21.15.1:0/376667784 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f0e480a05a0 0x7f0e480a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:47.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.439+0000 7f0e66000700 1 --2- 172.21.15.1:0/376667784 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f0e480a05a0 0x7f0e480a2a60 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0e50009c60 tx=0x7f0e50009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:47.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.442+0000 7f0e5f400700 1 -- 172.21.15.1:0/376667784 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f0e58052110 con 0x7f0e68100e20 2024-02-02T15:26:47.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:47 smithi071 ceph-mon[44707]: from='client.15471 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-02T15:26:47.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:47 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/2625229927' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-02T15:26:47.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.651+0000 7f0e6da00700 1 -- 172.21.15.1:0/376667784 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0e680619f0 con 0x7f0e68100e20 2024-02-02T15:26:47.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.652+0000 7f0e5f400700 1 -- 172.21.15.1:0/376667784 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0e58095670 con 0x7f0e68100e20 2024-02-02T15:26:47.651 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-02-02T15:26:47.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.655+0000 7f0e6da00700 1 -- 172.21.15.1:0/376667784 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f0e480a05a0 msgr2=0x7f0e480a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:47.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.655+0000 7f0e6da00700 1 --2- 172.21.15.1:0/376667784 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f0e480a05a0 0x7f0e480a2a60 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0e50009c60 tx=0x7f0e50009260 comp rx=0 tx=0).stop 2024-02-02T15:26:47.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.655+0000 7f0e6da00700 1 -- 172.21.15.1:0/376667784 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e68100e20 msgr2=0x7f0e68196be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:47.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.655+0000 7f0e6da00700 1 --2- 172.21.15.1:0/376667784 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e68100e20 0x7f0e68196be0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0e5800e9a0 tx=0x7f0e5800ecb0 comp rx=0 tx=0).stop 2024-02-02T15:26:47.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.655+0000 7f0e6da00700 1 -- 172.21.15.1:0/376667784 shutdown_connections 2024-02-02T15:26:47.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.655+0000 7f0e6da00700 1 --2- 172.21.15.1:0/376667784 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e68102930 0x7f0e6819e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:47.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.655+0000 7f0e6da00700 1 --2- 172.21.15.1:0/376667784 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e68100e20 0x7f0e68196be0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:47.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.655+0000 7f0e6da00700 1 --2- 172.21.15.1:0/376667784 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f0e480a05a0 0x7f0e480a2a60 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:47.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.655+0000 7f0e6da00700 1 --2- 172.21.15.1:0/376667784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e68101f70 0x7f0e68197120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:47.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.655+0000 7f0e6da00700 1 -- 172.21.15.1:0/376667784 >> 172.21.15.1:0/376667784 conn(0x7f0e680fc390 msgr2=0x7f0e680fe540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:47.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.655+0000 7f0e6da00700 1 -- 172.21.15.1:0/376667784 shutdown_connections 2024-02-02T15:26:47.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.655+0000 7f0e6da00700 1 -- 172.21.15.1:0/376667784 wait complete. 2024-02-02T15:26:47.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:47 smithi142 ceph-mon[45728]: from='client.15471 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-02T15:26:47.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:47 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/2625229927' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-02T15:26:47.670 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:47 smithi001 ceph-mon[35470]: from='client.15471 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-02T15:26:47.671 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:47 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/2625229927' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-02T15:26:47.673 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-02T15:26:47.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.878+0000 7f588a600700 1 -- 172.21.15.1:0/3403611736 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5884100e20 msgr2=0x7f5884101240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:47.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.878+0000 7f588a600700 1 --2- 172.21.15.1:0/3403611736 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5884100e20 0x7f5884101240 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f5874009990 tx=0x7f5874009ca0 comp rx=0 tx=0).stop 2024-02-02T15:26:47.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.878+0000 7f588a600700 1 -- 172.21.15.1:0/3403611736 shutdown_connections 2024-02-02T15:26:47.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.878+0000 7f588a600700 1 --2- 172.21.15.1:0/3403611736 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5884102930 0x7f5884109250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:47.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.878+0000 7f588a600700 1 --2- 172.21.15.1:0/3403611736 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5884100e20 0x7f5884101240 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:47.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.878+0000 7f588a600700 1 --2- 172.21.15.1:0/3403611736 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5884101f70 0x7f58841023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:47.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.878+0000 7f588a600700 1 -- 172.21.15.1:0/3403611736 >> 172.21.15.1:0/3403611736 conn(0x7f58840fc390 msgr2=0x7f58840fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:47.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.878+0000 7f588a600700 1 -- 172.21.15.1:0/3403611736 shutdown_connections 2024-02-02T15:26:47.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.878+0000 7f588a600700 1 -- 172.21.15.1:0/3403611736 wait complete. 2024-02-02T15:26:47.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.880+0000 7f588a600700 1 Processor -- start 2024-02-02T15:26:47.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.880+0000 7f588a600700 1 -- start start 2024-02-02T15:26:47.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.881+0000 7f588a600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5884100e20 0x7f5884196ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:47.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.881+0000 7f588a600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5884101f70 0x7f5884197020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:47.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.881+0000 7f588a600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5884102930 0x7f588419e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:47.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.881+0000 7f588a600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f588419e660 con 0x7f5884100e20 2024-02-02T15:26:47.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.881+0000 7f588a600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f588419e7a0 con 0x7f5884102930 2024-02-02T15:26:47.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.881+0000 7f588a600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f588419e930 con 0x7f5884101f70 2024-02-02T15:26:47.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.881+0000 7f5883400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5884100e20 0x7f5884196ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:47.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.881+0000 7f5882a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5884101f70 0x7f5884197020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:47.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.881+0000 7f5882a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5884101f70 0x7f5884197020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:41502/0 (socket says 172.21.15.1:41502) 2024-02-02T15:26:47.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.881+0000 7f5882a00700 1 -- 172.21.15.1:0/2829076107 learned_addr learned my addr 172.21.15.1:0/2829076107 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:47.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.881+0000 7f5883e00700 1 --2- 172.21.15.1:0/2829076107 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5884102930 0x7f588419e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:47.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.881+0000 7f5883400700 1 -- 172.21.15.1:0/2829076107 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5884101f70 msgr2=0x7f5884197020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:47.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.881+0000 7f5883400700 1 --2- 172.21.15.1:0/2829076107 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5884101f70 0x7f5884197020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:47.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.881+0000 7f5883400700 1 -- 172.21.15.1:0/2829076107 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5884102930 msgr2=0x7f588419e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:47.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.881+0000 7f5883400700 1 --2- 172.21.15.1:0/2829076107 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5884102930 0x7f588419e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:47.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.881+0000 7f5883400700 1 -- 172.21.15.1:0/2829076107 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58740095f0 con 0x7f5884100e20 2024-02-02T15:26:47.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.882+0000 7f5883400700 1 --2- 172.21.15.1:0/2829076107 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5884100e20 0x7f5884196ae0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f587400b620 tx=0x7f5874005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:47.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.882+0000 7f587be00700 1 -- 172.21.15.1:0/2829076107 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f587401d070 con 0x7f5884100e20 2024-02-02T15:26:47.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.882+0000 7f587be00700 1 -- 172.21.15.1:0/2829076107 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5874021410 con 0x7f5884100e20 2024-02-02T15:26:47.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.882+0000 7f587be00700 1 -- 172.21.15.1:0/2829076107 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5874017650 con 0x7f5884100e20 2024-02-02T15:26:47.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.882+0000 7f588a600700 1 -- 172.21.15.1:0/2829076107 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f588419ebb0 con 0x7f5884100e20 2024-02-02T15:26:47.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.882+0000 7f588a600700 1 -- 172.21.15.1:0/2829076107 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f588419ef80 con 0x7f5884100e20 2024-02-02T15:26:47.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.883+0000 7f588a600700 1 -- 172.21.15.1:0/2829076107 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5884107d60 con 0x7f5884100e20 2024-02-02T15:26:47.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.885+0000 7f587be00700 1 -- 172.21.15.1:0/2829076107 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) v1 ==== 135069+0+0 (secure 0 0 0) 0x7f5874021580 con 0x7f5884100e20 2024-02-02T15:26:47.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.886+0000 7f587be00700 1 --2- 172.21.15.1:0/2829076107 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f58640a05a0 0x7f58640a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:47.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.886+0000 7f587be00700 1 -- 172.21.15.1:0/2829076107 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f58740cd110 con 0x7f5884100e20 2024-02-02T15:26:47.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.887+0000 7f5882a00700 1 --2- 172.21.15.1:0/2829076107 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f58640a05a0 0x7f58640a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:47.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.887+0000 7f5882a00700 1 --2- 172.21.15.1:0/2829076107 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f58640a05a0 0x7f58640a2a60 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f586c009d20 tx=0x7f586c009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:47.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:47.890+0000 7f587be00700 1 -- 172.21.15.1:0/2829076107 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f5874096970 con 0x7f5884100e20 2024-02-02T15:26:48.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.106+0000 7f588a600700 1 -- 172.21.15.1:0/2829076107 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5884061950 con 0x7f5884100e20 2024-02-02T15:26:48.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.106+0000 7f587be00700 1 -- 172.21.15.1:0/2829076107 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5874099f90 con 0x7f5884100e20 2024-02-02T15:26:48.105 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-02-02T15:26:48.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.108+0000 7f588a600700 1 -- 172.21.15.1:0/2829076107 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f58640a05a0 msgr2=0x7f58640a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:48.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.108+0000 7f588a600700 1 --2- 172.21.15.1:0/2829076107 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f58640a05a0 0x7f58640a2a60 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f586c009d20 tx=0x7f586c009260 comp rx=0 tx=0).stop 2024-02-02T15:26:48.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.108+0000 7f588a600700 1 -- 172.21.15.1:0/2829076107 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5884100e20 msgr2=0x7f5884196ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:48.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.108+0000 7f588a600700 1 --2- 172.21.15.1:0/2829076107 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5884100e20 0x7f5884196ae0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f587400b620 tx=0x7f5874005740 comp rx=0 tx=0).stop 2024-02-02T15:26:48.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.109+0000 7f588a600700 1 -- 172.21.15.1:0/2829076107 shutdown_connections 2024-02-02T15:26:48.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.109+0000 7f588a600700 1 --2- 172.21.15.1:0/2829076107 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5884102930 0x7f588419e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:48.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.109+0000 7f588a600700 1 --2- 172.21.15.1:0/2829076107 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5884100e20 0x7f5884196ae0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:48.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.109+0000 7f588a600700 1 --2- 172.21.15.1:0/2829076107 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f58640a05a0 0x7f58640a2a60 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:48.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.109+0000 7f588a600700 1 --2- 172.21.15.1:0/2829076107 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5884101f70 0x7f5884197020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:48.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.109+0000 7f588a600700 1 -- 172.21.15.1:0/2829076107 >> 172.21.15.1:0/2829076107 conn(0x7f58840fc390 msgr2=0x7f58840fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:48.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.109+0000 7f588a600700 1 -- 172.21.15.1:0/2829076107 shutdown_connections 2024-02-02T15:26:48.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.109+0000 7f588a600700 1 -- 172.21.15.1:0/2829076107 wait complete. 2024-02-02T15:26:48.127 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-02T15:26:48.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.329+0000 7f40a7400700 1 -- 172.21.15.1:0/1712027506 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40a0101f80 msgr2=0x7f40a0102400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:48.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.329+0000 7f40a7400700 1 --2- 172.21.15.1:0/1712027506 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40a0101f80 0x7f40a0102400 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f4088009990 tx=0x7f4088009ca0 comp rx=0 tx=0).stop 2024-02-02T15:26:48.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.329+0000 7f40a7400700 1 -- 172.21.15.1:0/1712027506 shutdown_connections 2024-02-02T15:26:48.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.329+0000 7f40a7400700 1 --2- 172.21.15.1:0/1712027506 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40a0102940 0x7f40a0109260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:48.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.329+0000 7f40a7400700 1 --2- 172.21.15.1:0/1712027506 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40a0100e30 0x7f40a0101250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:48.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.329+0000 7f40a7400700 1 --2- 172.21.15.1:0/1712027506 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40a0101f80 0x7f40a0102400 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:48.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.329+0000 7f40a7400700 1 -- 172.21.15.1:0/1712027506 >> 172.21.15.1:0/1712027506 conn(0x7f40a00fc3a0 msgr2=0x7f40a00fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:48.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.330+0000 7f40a7400700 1 -- 172.21.15.1:0/1712027506 shutdown_connections 2024-02-02T15:26:48.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.330+0000 7f40a7400700 1 -- 172.21.15.1:0/1712027506 wait complete. 2024-02-02T15:26:48.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.330+0000 7f40a7400700 1 Processor -- start 2024-02-02T15:26:48.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.331+0000 7f40a7400700 1 -- start start 2024-02-02T15:26:48.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.331+0000 7f40a7400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40a0100e30 0x7f40a0198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:48.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.331+0000 7f40a7400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40a0101f80 0x7f40a0199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:48.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.331+0000 7f40a7400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40a0102940 0x7f40a01a0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:48.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.331+0000 7f40a7400700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a01a0900 con 0x7f40a0100e30 2024-02-02T15:26:48.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.331+0000 7f40a7400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a01a0a70 con 0x7f40a0102940 2024-02-02T15:26:48.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.331+0000 7f40a7400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a01a0c30 con 0x7f40a0101f80 2024-02-02T15:26:48.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.331+0000 7f40a4a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40a0100e30 0x7f40a0198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:48.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.331+0000 7f40a4a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40a0100e30 0x7f40a0198dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53498/0 (socket says 172.21.15.1:53498) 2024-02-02T15:26:48.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.331+0000 7f40a4a00700 1 -- 172.21.15.1:0/1073446709 learned_addr learned my addr 172.21.15.1:0/1073446709 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:48.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.331+0000 7f409fe00700 1 --2- 172.21.15.1:0/1073446709 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40a0101f80 0x7f40a0199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:48.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.331+0000 7f40a5400700 1 --2- 172.21.15.1:0/1073446709 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40a0102940 0x7f40a01a0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:48.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.332+0000 7f40a4a00700 1 -- 172.21.15.1:0/1073446709 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40a0101f80 msgr2=0x7f40a0199300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:48.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.332+0000 7f40a4a00700 1 --2- 172.21.15.1:0/1073446709 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40a0101f80 0x7f40a0199300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:48.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.332+0000 7f40a4a00700 1 -- 172.21.15.1:0/1073446709 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40a0102940 msgr2=0x7f40a01a0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:48.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.332+0000 7f40a4a00700 1 --2- 172.21.15.1:0/1073446709 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40a0102940 0x7f40a01a0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:48.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.332+0000 7f40a4a00700 1 -- 172.21.15.1:0/1073446709 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40880095f0 con 0x7f40a0100e30 2024-02-02T15:26:48.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.332+0000 7f40a5400700 1 --2- 172.21.15.1:0/1073446709 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40a0102940 0x7f40a01a0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:26:48.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.332+0000 7f40a4a00700 1 --2- 172.21.15.1:0/1073446709 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40a0100e30 0x7f40a0198dc0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f409000e9a0 tx=0x7f409000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:48.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.332+0000 7f409d600700 1 -- 172.21.15.1:0/1073446709 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4090009790 con 0x7f40a0100e30 2024-02-02T15:26:48.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.332+0000 7f409d600700 1 -- 172.21.15.1:0/1073446709 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f409000cbd0 con 0x7f40a0100e30 2024-02-02T15:26:48.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.332+0000 7f40a7400700 1 -- 172.21.15.1:0/1073446709 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40a01a0f10 con 0x7f40a0100e30 2024-02-02T15:26:48.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.332+0000 7f409d600700 1 -- 172.21.15.1:0/1073446709 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4090018680 con 0x7f40a0100e30 2024-02-02T15:26:48.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.332+0000 7f40a7400700 1 -- 172.21.15.1:0/1073446709 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40a01a1340 con 0x7f40a0100e30 2024-02-02T15:26:48.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.333+0000 7f40a7400700 1 -- 172.21.15.1:0/1073446709 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40a0193470 con 0x7f40a0100e30 2024-02-02T15:26:48.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.335+0000 7f409d600700 1 -- 172.21.15.1:0/1073446709 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) v1 ==== 135069+0+0 (secure 0 0 0) 0x7f409000cd40 con 0x7f40a0100e30 2024-02-02T15:26:48.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.336+0000 7f409d600700 1 --2- 172.21.15.1:0/1073446709 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f40800a05a0 0x7f40800a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:48.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.337+0000 7f409d600700 1 -- 172.21.15.1:0/1073446709 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f4090014070 con 0x7f40a0100e30 2024-02-02T15:26:48.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.337+0000 7f409fe00700 1 --2- 172.21.15.1:0/1073446709 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f40800a05a0 0x7f40800a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:48.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.337+0000 7f409fe00700 1 --2- 172.21.15.1:0/1073446709 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f40800a05a0 0x7f40800a2a60 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f408800b2e0 tx=0x7f4088005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:48.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.339+0000 7f409d600700 1 -- 172.21.15.1:0/1073446709 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f40900959a0 con 0x7f40a0100e30 2024-02-02T15:26:48.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:48 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/376667784' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-02T15:26:48.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:48 smithi001 ceph-mon[35470]: pgmap v299: 129 pgs: 129 active+clean; 648 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 767 B/s wr, 11 op/s 2024-02-02T15:26:48.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:48 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/2829076107' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-02T15:26:48.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.551+0000 7f40a7400700 1 -- 172.21.15.1:0/1073446709 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f40a002ce20 con 0x7f40a0100e30 2024-02-02T15:26:48.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.551+0000 7f409d600700 1 -- 172.21.15.1:0/1073446709 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f40900260a0 con 0x7f40a0100e30 2024-02-02T15:26:48.551 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 64 2024-02-02T15:26:48.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.554+0000 7f40a7400700 1 -- 172.21.15.1:0/1073446709 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f40800a05a0 msgr2=0x7f40800a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:48.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.554+0000 7f40a7400700 1 --2- 172.21.15.1:0/1073446709 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f40800a05a0 0x7f40800a2a60 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f408800b2e0 tx=0x7f4088005fb0 comp rx=0 tx=0).stop 2024-02-02T15:26:48.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.554+0000 7f40a7400700 1 -- 172.21.15.1:0/1073446709 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40a0100e30 msgr2=0x7f40a0198dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:48.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.554+0000 7f40a7400700 1 --2- 172.21.15.1:0/1073446709 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40a0100e30 0x7f40a0198dc0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f409000e9a0 tx=0x7f409000ecb0 comp rx=0 tx=0).stop 2024-02-02T15:26:48.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.554+0000 7f40a7400700 1 -- 172.21.15.1:0/1073446709 shutdown_connections 2024-02-02T15:26:48.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.554+0000 7f40a7400700 1 --2- 172.21.15.1:0/1073446709 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40a0102940 0x7f40a01a0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:48.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.554+0000 7f40a7400700 1 --2- 172.21.15.1:0/1073446709 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40a0100e30 0x7f40a0198dc0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:48.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.554+0000 7f40a7400700 1 --2- 172.21.15.1:0/1073446709 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f40800a05a0 0x7f40800a2a60 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:48.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.554+0000 7f40a7400700 1 --2- 172.21.15.1:0/1073446709 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40a0101f80 0x7f40a0199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:48.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.554+0000 7f40a7400700 1 -- 172.21.15.1:0/1073446709 >> 172.21.15.1:0/1073446709 conn(0x7f40a00fc3a0 msgr2=0x7f40a0107940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:48.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.554+0000 7f40a7400700 1 -- 172.21.15.1:0/1073446709 shutdown_connections 2024-02-02T15:26:48.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.554+0000 7f40a7400700 1 -- 172.21.15.1:0/1073446709 wait complete. 2024-02-02T15:26:48.572 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-02-02T15:26:48.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:48 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/376667784' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-02T15:26:48.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:48 smithi071 ceph-mon[44707]: pgmap v299: 129 pgs: 129 active+clean; 648 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 767 B/s wr, 11 op/s 2024-02-02T15:26:48.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:48 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/2829076107' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-02T15:26:48.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:48 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/376667784' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-02T15:26:48.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:48 smithi142 ceph-mon[45728]: pgmap v299: 129 pgs: 129 active+clean; 648 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 767 B/s wr, 11 op/s 2024-02-02T15:26:48.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:48 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/2829076107' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-02T15:26:48.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.775+0000 7f246d800700 1 -- 172.21.15.1:0/1040866554 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2468101f70 msgr2=0x7f24681023f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:48.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.775+0000 7f246d800700 1 --2- 172.21.15.1:0/1040866554 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2468101f70 0x7f24681023f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f2450009990 tx=0x7f2450009ca0 comp rx=0 tx=0).stop 2024-02-02T15:26:48.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.775+0000 7f246d800700 1 -- 172.21.15.1:0/1040866554 shutdown_connections 2024-02-02T15:26:48.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.775+0000 7f246d800700 1 --2- 172.21.15.1:0/1040866554 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2468102930 0x7f2468109250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:48.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.775+0000 7f246d800700 1 --2- 172.21.15.1:0/1040866554 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2468100e20 0x7f2468101240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:48.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.775+0000 7f246d800700 1 --2- 172.21.15.1:0/1040866554 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2468101f70 0x7f24681023f0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:48.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.775+0000 7f246d800700 1 -- 172.21.15.1:0/1040866554 >> 172.21.15.1:0/1040866554 conn(0x7f24680fc390 msgr2=0x7f24680fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:48.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.775+0000 7f246d800700 1 -- 172.21.15.1:0/1040866554 shutdown_connections 2024-02-02T15:26:48.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.775+0000 7f246d800700 1 -- 172.21.15.1:0/1040866554 wait complete. 2024-02-02T15:26:48.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.777+0000 7f246d800700 1 Processor -- start 2024-02-02T15:26:48.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.777+0000 7f246d800700 1 -- start start 2024-02-02T15:26:48.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.777+0000 7f246d800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2468100e20 0x7f2468074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:48.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.777+0000 7f246d800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2468101f70 0x7f2468073130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:48.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.777+0000 7f246d800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2468102930 0x7f2468073670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:48.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.777+0000 7f246d800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2468073e10 con 0x7f2468102930 2024-02-02T15:26:48.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.777+0000 7f246d800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2468073f80 con 0x7f2468101f70 2024-02-02T15:26:48.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.777+0000 7f246d800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24681a0b40 con 0x7f2468100e20 2024-02-02T15:26:48.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.778+0000 7f2467400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2468102930 0x7f2468073670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:48.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.778+0000 7f2466a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2468100e20 0x7f2468074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:48.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.778+0000 7f2467400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2468102930 0x7f2468073670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53528/0 (socket says 172.21.15.1:53528) 2024-02-02T15:26:48.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.778+0000 7f2467400700 1 -- 172.21.15.1:0/3989499469 learned_addr learned my addr 172.21.15.1:0/3989499469 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:48.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.778+0000 7f2466000700 1 --2- 172.21.15.1:0/3989499469 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2468101f70 0x7f2468073130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:48.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.778+0000 7f2467400700 1 -- 172.21.15.1:0/3989499469 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2468100e20 msgr2=0x7f2468074af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:48.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.778+0000 7f2467400700 1 --2- 172.21.15.1:0/3989499469 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2468100e20 0x7f2468074af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:48.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.778+0000 7f2467400700 1 -- 172.21.15.1:0/3989499469 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2468101f70 msgr2=0x7f2468073130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:48.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.778+0000 7f2467400700 1 --2- 172.21.15.1:0/3989499469 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2468101f70 0x7f2468073130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:48.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.778+0000 7f2467400700 1 -- 172.21.15.1:0/3989499469 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24500095f0 con 0x7f2468102930 2024-02-02T15:26:48.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.778+0000 7f2467400700 1 --2- 172.21.15.1:0/3989499469 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2468102930 0x7f2468073670 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f246000ef30 tx=0x7f246000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:48.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.778+0000 7f245f400700 1 -- 172.21.15.1:0/3989499469 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2460018070 con 0x7f2468102930 2024-02-02T15:26:48.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.779+0000 7f246d800700 1 -- 172.21.15.1:0/3989499469 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24681a0dc0 con 0x7f2468102930 2024-02-02T15:26:48.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.779+0000 7f245f400700 1 -- 172.21.15.1:0/3989499469 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f246000f040 con 0x7f2468102930 2024-02-02T15:26:48.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.779+0000 7f245f400700 1 -- 172.21.15.1:0/3989499469 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f246000cd00 con 0x7f2468102930 2024-02-02T15:26:48.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.779+0000 7f246d800700 1 -- 172.21.15.1:0/3989499469 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24681a1250 con 0x7f2468102930 2024-02-02T15:26:48.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.780+0000 7f246d800700 1 -- 172.21.15.1:0/3989499469 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2468107d40 con 0x7f2468102930 2024-02-02T15:26:48.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.781+0000 7f245f400700 1 -- 172.21.15.1:0/3989499469 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) v1 ==== 135069+0+0 (secure 0 0 0) 0x7f2460003a00 con 0x7f2468102930 2024-02-02T15:26:48.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.783+0000 7f245f400700 1 --2- 172.21.15.1:0/3989499469 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f24480a0520 0x7f24480a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:48.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.783+0000 7f245f400700 1 -- 172.21.15.1:0/3989499469 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f24600cb770 con 0x7f2468102930 2024-02-02T15:26:48.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.783+0000 7f2466a00700 1 --2- 172.21.15.1:0/3989499469 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f24480a0520 0x7f24480a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:48.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.784+0000 7f2466a00700 1 --2- 172.21.15.1:0/3989499469 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f24480a0520 0x7f24480a29e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2458007c60 tx=0x7f24580073d0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:48.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:48.787+0000 7f245f400700 1 -- 172.21.15.1:0/3989499469 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f2460095030 con 0x7f2468102930 2024-02-02T15:26:48.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.000+0000 7f246d800700 1 -- 172.21.15.1:0/3989499469 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f2468061980 con 0x7f2468102930 2024-02-02T15:26:48.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.001+0000 7f245f400700 1 -- 172.21.15.1:0/3989499469 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v88) v1 ==== 103+0+11 (secure 0 0 0) 0x7f2460098650 con 0x7f2468102930 2024-02-02T15:26:49.000 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-02-02T15:26:49.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.003+0000 7f246d800700 1 -- 172.21.15.1:0/3989499469 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f24480a0520 msgr2=0x7f24480a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:49.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.003+0000 7f246d800700 1 --2- 172.21.15.1:0/3989499469 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f24480a0520 0x7f24480a29e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2458007c60 tx=0x7f24580073d0 comp rx=0 tx=0).stop 2024-02-02T15:26:49.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.003+0000 7f246d800700 1 -- 172.21.15.1:0/3989499469 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2468102930 msgr2=0x7f2468073670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:49.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.003+0000 7f246d800700 1 --2- 172.21.15.1:0/3989499469 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2468102930 0x7f2468073670 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f246000ef30 tx=0x7f246000c3a0 comp rx=0 tx=0).stop 2024-02-02T15:26:49.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.003+0000 7f246d800700 1 -- 172.21.15.1:0/3989499469 shutdown_connections 2024-02-02T15:26:49.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.003+0000 7f246d800700 1 --2- 172.21.15.1:0/3989499469 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2468102930 0x7f2468073670 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:49.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.003+0000 7f246d800700 1 --2- 172.21.15.1:0/3989499469 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f24480a0520 0x7f24480a29e0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:49.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.003+0000 7f246d800700 1 --2- 172.21.15.1:0/3989499469 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2468100e20 0x7f2468074af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:49.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.003+0000 7f246d800700 1 --2- 172.21.15.1:0/3989499469 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2468101f70 0x7f2468073130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:49.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.003+0000 7f246d800700 1 -- 172.21.15.1:0/3989499469 >> 172.21.15.1:0/3989499469 conn(0x7f24680fc390 msgr2=0x7f24680fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:49.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.004+0000 7f246d800700 1 -- 172.21.15.1:0/3989499469 shutdown_connections 2024-02-02T15:26:49.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.004+0000 7f246d800700 1 -- 172.21.15.1:0/3989499469 wait complete. 2024-02-02T15:26:49.021 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T15:26:49.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.220+0000 7f84c7000700 1 -- 172.21.15.1:0/175943105 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84c01040f0 msgr2=0x7f84c01064e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:49.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.220+0000 7f84c7000700 1 --2- 172.21.15.1:0/175943105 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84c01040f0 0x7f84c01064e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f84a8009990 tx=0x7f84a8009ca0 comp rx=0 tx=0).stop 2024-02-02T15:26:49.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.220+0000 7f84c7000700 1 -- 172.21.15.1:0/175943105 shutdown_connections 2024-02-02T15:26:49.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.220+0000 7f84c7000700 1 --2- 172.21.15.1:0/175943105 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84c0106a20 0x7f84c0108e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:49.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.220+0000 7f84c7000700 1 --2- 172.21.15.1:0/175943105 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84c01017c0 0x7f84c0103bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:49.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.220+0000 7f84c7000700 1 --2- 172.21.15.1:0/175943105 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84c01040f0 0x7f84c01064e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:49.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.220+0000 7f84c7000700 1 -- 172.21.15.1:0/175943105 >> 172.21.15.1:0/175943105 conn(0x7f84c00fb170 msgr2=0x7f84c00fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:49.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.220+0000 7f84c7000700 1 -- 172.21.15.1:0/175943105 shutdown_connections 2024-02-02T15:26:49.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.220+0000 7f84c7000700 1 -- 172.21.15.1:0/175943105 wait complete. 2024-02-02T15:26:49.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.221+0000 7f84c7000700 1 Processor -- start 2024-02-02T15:26:49.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.221+0000 7f84c7000700 1 -- start start 2024-02-02T15:26:49.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.222+0000 7f84c7000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84c01017c0 0x7f84c0198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:49.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.222+0000 7f84c7000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84c01040f0 0x7f84c0199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:49.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.222+0000 7f84c7000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84c0106a20 0x7f84c01a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:49.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.222+0000 7f84c7000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c01a0760 con 0x7f84c01040f0 2024-02-02T15:26:49.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.222+0000 7f84c7000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c01a08d0 con 0x7f84c0106a20 2024-02-02T15:26:49.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.222+0000 7f84c7000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c01a0a90 con 0x7f84c01017c0 2024-02-02T15:26:49.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.222+0000 7f84bf400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84c01040f0 0x7f84c0199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:49.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.222+0000 7f84bf400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84c01040f0 0x7f84c0199160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53544/0 (socket says 172.21.15.1:53544) 2024-02-02T15:26:49.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.222+0000 7f84bf400700 1 -- 172.21.15.1:0/1724999362 learned_addr learned my addr 172.21.15.1:0/1724999362 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:49.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.222+0000 7f84c5000700 1 --2- 172.21.15.1:0/1724999362 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84c0106a20 0x7f84c01a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:49.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.222+0000 7f84bfe00700 1 --2- 172.21.15.1:0/1724999362 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84c01017c0 0x7f84c0198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:49.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.222+0000 7f84bf400700 1 -- 172.21.15.1:0/1724999362 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84c01017c0 msgr2=0x7f84c0198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:49.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.222+0000 7f84bf400700 1 --2- 172.21.15.1:0/1724999362 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84c01017c0 0x7f84c0198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:49.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.222+0000 7f84bf400700 1 -- 172.21.15.1:0/1724999362 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84c0106a20 msgr2=0x7f84c01a0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:49.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.222+0000 7f84bf400700 1 --2- 172.21.15.1:0/1724999362 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84c0106a20 0x7f84c01a0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:49.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.222+0000 7f84bf400700 1 -- 172.21.15.1:0/1724999362 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84a80095f0 con 0x7f84c01040f0 2024-02-02T15:26:49.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.222+0000 7f84c5000700 1 --2- 172.21.15.1:0/1724999362 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84c0106a20 0x7f84c01a0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:26:49.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.222+0000 7f84bfe00700 1 --2- 172.21.15.1:0/1724999362 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84c01017c0 0x7f84c0198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:26:49.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.223+0000 7f84bf400700 1 --2- 172.21.15.1:0/1724999362 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84c01040f0 0x7f84c0199160 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f84a800b680 tx=0x7f84a8005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:49.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.223+0000 7f84bcc00700 1 -- 172.21.15.1:0/1724999362 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a801d070 con 0x7f84c01040f0 2024-02-02T15:26:49.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.223+0000 7f84bcc00700 1 -- 172.21.15.1:0/1724999362 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f84a8022470 con 0x7f84c01040f0 2024-02-02T15:26:49.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.223+0000 7f84bcc00700 1 -- 172.21.15.1:0/1724999362 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a8017650 con 0x7f84c01040f0 2024-02-02T15:26:49.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.223+0000 7f84c7000700 1 -- 172.21.15.1:0/1724999362 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84c01a0d10 con 0x7f84c01040f0 2024-02-02T15:26:49.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.223+0000 7f84c7000700 1 -- 172.21.15.1:0/1724999362 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84c01a10e0 con 0x7f84c01040f0 2024-02-02T15:26:49.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.224+0000 7f84c7000700 1 -- 172.21.15.1:0/1724999362 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84c01085f0 con 0x7f84c01040f0 2024-02-02T15:26:49.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.226+0000 7f84bcc00700 1 -- 172.21.15.1:0/1724999362 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) v1 ==== 135069+0+0 (secure 0 0 0) 0x7f84a80225e0 con 0x7f84c01040f0 2024-02-02T15:26:49.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.227+0000 7f84bcc00700 1 --2- 172.21.15.1:0/1724999362 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f84a00a0520 0x7f84a00a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:49.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.227+0000 7f84bcc00700 1 -- 172.21.15.1:0/1724999362 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f84a80cd200 con 0x7f84c01040f0 2024-02-02T15:26:49.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.228+0000 7f84bfe00700 1 --2- 172.21.15.1:0/1724999362 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f84a00a0520 0x7f84a00a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:49.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.228+0000 7f84bfe00700 1 --2- 172.21.15.1:0/1724999362 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f84a00a0520 0x7f84a00a29e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f84c019a0f0 tx=0x7f84b0009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:49.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.231+0000 7f84bcc00700 1 -- 172.21.15.1:0/1724999362 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f84a8096b80 con 0x7f84c01040f0 2024-02-02T15:26:49.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:49 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/1073446709' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-02T15:26:49.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:49 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/3989499469' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-02T15:26:49.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.488+0000 7f84c7000700 1 -- 172.21.15.1:0/1724999362 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f84c002cc70 con 0x7f84c01040f0 2024-02-02T15:26:49.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.488+0000 7f84bcc00700 1 -- 172.21.15.1:0/1724999362 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10866 (secure 0 0 0) 0x7f84a8027020 con 0x7f84c01040f0 2024-02-02T15:26:49.487 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:26:49.487 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2922240448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2922240448},{"type":"v1","addr":"172.21.15.1:6835","nonce":2922240448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2729497431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2729497431},{"type":"v1","addr":"172.21.15.1:6837","nonce":2729497431}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-02T15:24:47.761435+0000","modified":"2024-02-02T15:25:20.251601+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24529,"mds_2":24518,"mds_3":24514,"mds_4":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14517":{"gid":14517,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/1218303683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":1218303683},{"type":"v1","addr":"172.21.15.1:6839","nonce":1218303683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14529":{"gid":14529,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/2059197496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2059197496},{"type":"v1","addr":"172.21.15.1:6841","nonce":2059197496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/7648986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":7648986},{"type":"v1","addr":"172.21.15.142:6833","nonce":7648986}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/3482746094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3482746094},{"type":"v1","addr":"172.21.15.71:6833","nonce":3482746094}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/388207814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":388207814},{"type":"v1","addr":"172.21.15.71:6835","nonce":388207814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6835/1052606547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1052606547},{"type":"v1","addr":"172.21.15.142:6835","nonce":1052606547}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.142:6837/2870289019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2870289019},{"type":"v1","addr":"172.21.15.142:6837","nonce":2870289019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.71:6837/628016199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":628016199},{"type":"v1","addr":"172.21.15.71:6837","nonce":628016199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.142:6839/321675421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":321675421},{"type":"v1","addr":"172.21.15.142:6839","nonce":321675421}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/1268086493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1268086493},{"type":"v1","addr":"172.21.15.71:6839","nonce":1268086493}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-02T15:26:49.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.491+0000 7f84c7000700 1 -- 172.21.15.1:0/1724999362 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f84a00a0520 msgr2=0x7f84a00a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:49.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.491+0000 7f84c7000700 1 --2- 172.21.15.1:0/1724999362 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f84a00a0520 0x7f84a00a29e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f84c019a0f0 tx=0x7f84b0009260 comp rx=0 tx=0).stop 2024-02-02T15:26:49.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.491+0000 7f84c7000700 1 -- 172.21.15.1:0/1724999362 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84c01040f0 msgr2=0x7f84c0199160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:49.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.491+0000 7f84c7000700 1 --2- 172.21.15.1:0/1724999362 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84c01040f0 0x7f84c0199160 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f84a800b680 tx=0x7f84a8005740 comp rx=0 tx=0).stop 2024-02-02T15:26:49.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.491+0000 7f84c7000700 1 -- 172.21.15.1:0/1724999362 shutdown_connections 2024-02-02T15:26:49.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.491+0000 7f84c7000700 1 --2- 172.21.15.1:0/1724999362 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f84c0106a20 0x7f84c01a0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:49.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.491+0000 7f84c7000700 1 --2- 172.21.15.1:0/1724999362 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f84a00a0520 0x7f84a00a29e0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:49.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.491+0000 7f84c7000700 1 --2- 172.21.15.1:0/1724999362 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84c01017c0 0x7f84c0198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:49.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.491+0000 7f84c7000700 1 --2- 172.21.15.1:0/1724999362 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f84c01040f0 0x7f84c0199160 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:49.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.491+0000 7f84c7000700 1 -- 172.21.15.1:0/1724999362 >> 172.21.15.1:0/1724999362 conn(0x7f84c00fb170 msgr2=0x7f84c00ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:49.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.491+0000 7f84c7000700 1 -- 172.21.15.1:0/1724999362 shutdown_connections 2024-02-02T15:26:49.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.491+0000 7f84c7000700 1 -- 172.21.15.1:0/1724999362 wait complete. 2024-02-02T15:26:49.491 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 27 2024-02-02T15:26:49.509 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T15:26:49.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:49 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/1073446709' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-02T15:26:49.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:49 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/3989499469' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-02T15:26:49.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:49 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/1073446709' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-02T15:26:49.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:49 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/3989499469' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-02T15:26:49.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.707+0000 7fe6d3800700 1 -- 172.21.15.1:0/3672038897 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6cc101f30 msgr2=0x7fe6cc102390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:49.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.707+0000 7fe6d3800700 1 --2- 172.21.15.1:0/3672038897 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6cc101f30 0x7fe6cc102390 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe6b40099c0 tx=0x7fe6b4009cd0 comp rx=0 tx=0).stop 2024-02-02T15:26:49.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.707+0000 7fe6d3800700 1 -- 172.21.15.1:0/3672038897 shutdown_connections 2024-02-02T15:26:49.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.707+0000 7fe6d3800700 1 --2- 172.21.15.1:0/3672038897 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6cc1028d0 0x7fe6cc109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:49.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.707+0000 7fe6d3800700 1 --2- 172.21.15.1:0/3672038897 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6cc100d40 0x7fe6cc101160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:49.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.707+0000 7fe6d3800700 1 --2- 172.21.15.1:0/3672038897 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6cc101f30 0x7fe6cc102390 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:49.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.707+0000 7fe6d3800700 1 -- 172.21.15.1:0/3672038897 >> 172.21.15.1:0/3672038897 conn(0x7fe6cc0fc290 msgr2=0x7fe6cc0fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:49.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.707+0000 7fe6d3800700 1 -- 172.21.15.1:0/3672038897 shutdown_connections 2024-02-02T15:26:49.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.707+0000 7fe6d3800700 1 -- 172.21.15.1:0/3672038897 wait complete. 2024-02-02T15:26:49.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.708+0000 7fe6d3800700 1 Processor -- start 2024-02-02T15:26:49.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.709+0000 7fe6d3800700 1 -- start start 2024-02-02T15:26:49.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.709+0000 7fe6d3800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6cc100d40 0x7fe6cc074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:49.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.709+0000 7fe6d3800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6cc101f30 0x7fe6cc073130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:49.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.709+0000 7fe6d3800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6cc1028d0 0x7fe6cc073670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:49.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.709+0000 7fe6d3800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6cc073e10 con 0x7fe6cc101f30 2024-02-02T15:26:49.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.709+0000 7fe6d3800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6cc073f80 con 0x7fe6cc100d40 2024-02-02T15:26:49.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.709+0000 7fe6d3800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6cc1a0ac0 con 0x7fe6cc1028d0 2024-02-02T15:26:49.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.709+0000 7fe6cbe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6cc101f30 0x7fe6cc073130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:49.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.709+0000 7fe6cbe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6cc101f30 0x7fe6cc073130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53556/0 (socket says 172.21.15.1:53556) 2024-02-02T15:26:49.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.709+0000 7fe6cbe00700 1 -- 172.21.15.1:0/2892118238 learned_addr learned my addr 172.21.15.1:0/2892118238 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:49.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.709+0000 7fe6d0e00700 1 --2- 172.21.15.1:0/2892118238 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6cc100d40 0x7fe6cc074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:49.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.709+0000 7fe6d1800700 1 --2- 172.21.15.1:0/2892118238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6cc1028d0 0x7fe6cc073670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:49.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.710+0000 7fe6cbe00700 1 -- 172.21.15.1:0/2892118238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6cc1028d0 msgr2=0x7fe6cc073670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:49.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.710+0000 7fe6cbe00700 1 --2- 172.21.15.1:0/2892118238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6cc1028d0 0x7fe6cc073670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:49.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.710+0000 7fe6cbe00700 1 -- 172.21.15.1:0/2892118238 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6cc100d40 msgr2=0x7fe6cc074af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:49.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.710+0000 7fe6cbe00700 1 --2- 172.21.15.1:0/2892118238 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6cc100d40 0x7fe6cc074af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:49.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.710+0000 7fe6cbe00700 1 -- 172.21.15.1:0/2892118238 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6b40095f0 con 0x7fe6cc101f30 2024-02-02T15:26:49.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.710+0000 7fe6d1800700 1 --2- 172.21.15.1:0/2892118238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6cc1028d0 0x7fe6cc073670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:26:49.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.710+0000 7fe6cbe00700 1 --2- 172.21.15.1:0/2892118238 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6cc101f30 0x7fe6cc073130 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4009fd0 tx=0x7fe6b400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:49.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.710+0000 7fe6c9600700 1 -- 172.21.15.1:0/2892118238 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b401d070 con 0x7fe6cc101f30 2024-02-02T15:26:49.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.710+0000 7fe6c9600700 1 -- 172.21.15.1:0/2892118238 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe6b4005510 con 0x7fe6cc101f30 2024-02-02T15:26:49.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.710+0000 7fe6d3800700 1 -- 172.21.15.1:0/2892118238 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6cc1a0d40 con 0x7fe6cc101f30 2024-02-02T15:26:49.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.710+0000 7fe6c9600700 1 -- 172.21.15.1:0/2892118238 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b4004ae0 con 0x7fe6cc101f30 2024-02-02T15:26:49.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.710+0000 7fe6d3800700 1 -- 172.21.15.1:0/2892118238 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6cc1a1160 con 0x7fe6cc101f30 2024-02-02T15:26:49.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.711+0000 7fe6d3800700 1 -- 172.21.15.1:0/2892118238 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6cc10c5a0 con 0x7fe6cc101f30 2024-02-02T15:26:49.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.713+0000 7fe6c9600700 1 -- 172.21.15.1:0/2892118238 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) v1 ==== 135069+0+0 (secure 0 0 0) 0x7fe6b400fb80 con 0x7fe6cc101f30 2024-02-02T15:26:49.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.715+0000 7fe6c9600700 1 --2- 172.21.15.1:0/2892118238 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fe6ac0a0520 0x7fe6ac0a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:49.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.715+0000 7fe6c9600700 1 -- 172.21.15.1:0/2892118238 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fe6b40ccf90 con 0x7fe6cc101f30 2024-02-02T15:26:49.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.715+0000 7fe6d0e00700 1 --2- 172.21.15.1:0/2892118238 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fe6ac0a0520 0x7fe6ac0a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:49.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.715+0000 7fe6d0e00700 1 --2- 172.21.15.1:0/2892118238 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fe6ac0a0520 0x7fe6ac0a29e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc101d90 tx=0x7fe6bc009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:49.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.717+0000 7fe6c9600700 1 -- 172.21.15.1:0/2892118238 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7fe6b40967f0 con 0x7fe6cc101f30 2024-02-02T15:26:49.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.929+0000 7fe6d3800700 1 -- 172.21.15.1:0/2892118238 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe6cc107c90 con 0x7fe6cc101f30 2024-02-02T15:26:49.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.931+0000 7fe6c9600700 1 -- 172.21.15.1:0/2892118238 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+24453 (secure 0 0 0) 0x7fe6b4099e10 con 0x7fe6cc101f30 2024-02-02T15:26:49.930 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:26:49.931 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":88,"fsid":"371a409c-c1de-11ee-95b6-87774f69a715","created":"2024-02-02T15:18:26.746506+0000","modified":"2024-02-02T15:26:28.267447+0000","last_up_change":"2024-02-02T15:24:08.800951+0000","last_in_change":"2024-02-02T15:23:52.784280+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T15:21:22.471754+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-02T15:24:46.195779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-02T15:24:47.226403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-02T15:24:50.300462+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"73e8a07f-3560-4c19-85ed-030d4d8f0f23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6803","nonce":2591906423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6805","nonce":2591906423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6809","nonce":2591906423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6807","nonce":2591906423}]},"public_addr":"172.21.15.1:6803/2591906423","cluster_addr":"172.21.15.1:6805/2591906423","heartbeat_back_addr":"172.21.15.1:6809/2591906423","heartbeat_front_addr":"172.21.15.1:6807/2591906423","state":["exists","up"]},{"osd":1,"uuid":"e53cffec-0ff9-4ed9-8dcf-ff47d04db2bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6801","nonce":747581719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6803","nonce":747581719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6807","nonce":747581719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6805","nonce":747581719}]},"public_addr":"172.21.15.71:6801/747581719","cluster_addr":"172.21.15.71:6803/747581719","heartbeat_back_addr":"172.21.15.71:6807/747581719","heartbeat_front_addr":"172.21.15.71:6805/747581719","state":["exists","up"]},{"osd":2,"uuid":"856c2347-294c-482b-8595-ad77f4792db4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6801","nonce":377511414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6803","nonce":377511414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6807","nonce":377511414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6805","nonce":377511414}]},"public_addr":"172.21.15.142:6801/377511414","cluster_addr":"172.21.15.142:6803/377511414","heartbeat_back_addr":"172.21.15.142:6807/377511414","heartbeat_front_addr":"172.21.15.142:6805/377511414","state":["exists","up"]},{"osd":3,"uuid":"4b946ad2-2da9-4e0b-9d61-d039d8669a42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6811","nonce":1086124477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6813","nonce":1086124477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6817","nonce":1086124477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6815","nonce":1086124477}]},"public_addr":"172.21.15.1:6811/1086124477","cluster_addr":"172.21.15.1:6813/1086124477","heartbeat_back_addr":"172.21.15.1:6817/1086124477","heartbeat_front_addr":"172.21.15.1:6815/1086124477","state":["exists","up"]},{"osd":4,"uuid":"43e36c43-47a0-4634-a45b-1da14a0e19d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6809","nonce":3095380305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6811","nonce":3095380305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6815","nonce":3095380305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6813","nonce":3095380305}]},"public_addr":"172.21.15.71:6809/3095380305","cluster_addr":"172.21.15.71:6811/3095380305","heartbeat_back_addr":"172.21.15.71:6815/3095380305","heartbeat_front_addr":"172.21.15.71:6813/3095380305","state":["exists","up"]},{"osd":5,"uuid":"eb48160e-ea7c-42f7-9477-be110bc8a34e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6809","nonce":4195149262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6811","nonce":4195149262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6815","nonce":4195149262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6813","nonce":4195149262}]},"public_addr":"172.21.15.142:6809/4195149262","cluster_addr":"172.21.15.142:6811/4195149262","heartbeat_back_addr":"172.21.15.142:6815/4195149262","heartbeat_front_addr":"172.21.15.142:6813/4195149262","state":["exists","up"]},{"osd":6,"uuid":"0ec4501a-5853-4ca6-b325-231f5b72335e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6819","nonce":1560153428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6821","nonce":1560153428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6825","nonce":1560153428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6823","nonce":1560153428}]},"public_addr":"172.21.15.1:6819/1560153428","cluster_addr":"172.21.15.1:6821/1560153428","heartbeat_back_addr":"172.21.15.1:6825/1560153428","heartbeat_front_addr":"172.21.15.1:6823/1560153428","state":["exists","up"]},{"osd":7,"uuid":"029debe8-9cdf-44f6-9aea-cabd1d185f93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6817","nonce":4080955538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6819","nonce":4080955538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6823","nonce":4080955538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6821","nonce":4080955538}]},"public_addr":"172.21.15.71:6817/4080955538","cluster_addr":"172.21.15.71:6819/4080955538","heartbeat_back_addr":"172.21.15.71:6823/4080955538","heartbeat_front_addr":"172.21.15.71:6821/4080955538","state":["exists","up"]},{"osd":8,"uuid":"7c90896c-9955-4c1b-aaca-7531dd5457f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6817","nonce":3942038209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6819","nonce":3942038209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6823","nonce":3942038209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6821","nonce":3942038209}]},"public_addr":"172.21.15.142:6817/3942038209","cluster_addr":"172.21.15.142:6819/3942038209","heartbeat_back_addr":"172.21.15.142:6823/3942038209","heartbeat_front_addr":"172.21.15.142:6821/3942038209","state":["exists","up"]},{"osd":9,"uuid":"7e165e68-31a8-4598-a577-de3e90ae5c5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6827","nonce":2716544850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6829","nonce":2716544850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6833","nonce":2716544850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6831","nonce":2716544850}]},"public_addr":"172.21.15.1:6827/2716544850","cluster_addr":"172.21.15.1:6829/2716544850","heartbeat_back_addr":"172.21.15.1:6833/2716544850","heartbeat_front_addr":"172.21.15.1:6831/2716544850","state":["exists","up"]},{"osd":10,"uuid":"48cd5fdb-0887-444a-aab8-a1de24282bb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6825","nonce":84691841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6827","nonce":84691841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6831","nonce":84691841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6829","nonce":84691841}]},"public_addr":"172.21.15.71:6825/84691841","cluster_addr":"172.21.15.71:6827/84691841","heartbeat_back_addr":"172.21.15.71:6831/84691841","heartbeat_front_addr":"172.21.15.71:6829/84691841","state":["exists","up"]},{"osd":11,"uuid":"f011dd27-11b0-4166-adfe-76756533163a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6825","nonce":2290179934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6827","nonce":2290179934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6831","nonce":2290179934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6829","nonce":2290179934}]},"public_addr":"172.21.15.142:6825/2290179934","cluster_addr":"172.21.15.142:6827/2290179934","heartbeat_back_addr":"172.21.15.142:6831/2290179934","heartbeat_front_addr":"172.21.15.142:6829/2290179934","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:20:44.060692+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:01.903817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:19.368258+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:40.028037+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:57.513361+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:22:14.805944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:22:35.188439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:22:53.014412+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:23:10.507420+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:23:31.849765+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:23:50.070612+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:24:07.388625+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":5,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/4092532060":"2024-02-03T15:19:25.836171+0000","172.21.15.1:6801/395498296":"2024-02-03T15:19:25.836171+0000","172.21.15.1:6800/395498296":"2024-02-03T15:19:25.836171+0000","172.21.15.1:0/1575373489":"2024-02-03T15:19:25.836171+0000","172.21.15.1:0/865983374":"2024-02-03T15:19:01.431596+0000","172.21.15.1:0/3165960227":"2024-02-03T15:19:01.431596+0000","172.21.15.1:0/2350830293":"2024-02-03T15:19:01.431596+0000","172.21.15.1:0/2959690250":"2024-02-03T15:19:25.836171+0000","172.21.15.1:6800/1162530732":"2024-02-03T15:19:01.431596+0000","172.21.15.1:6801/1162530732":"2024-02-03T15:19:01.431596+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T15:26:49.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.933+0000 7fe6d3800700 1 -- 172.21.15.1:0/2892118238 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fe6ac0a0520 msgr2=0x7fe6ac0a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:49.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.933+0000 7fe6d3800700 1 --2- 172.21.15.1:0/2892118238 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fe6ac0a0520 0x7fe6ac0a29e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc101d90 tx=0x7fe6bc009260 comp rx=0 tx=0).stop 2024-02-02T15:26:49.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.933+0000 7fe6d3800700 1 -- 172.21.15.1:0/2892118238 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6cc101f30 msgr2=0x7fe6cc073130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:49.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.933+0000 7fe6d3800700 1 --2- 172.21.15.1:0/2892118238 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6cc101f30 0x7fe6cc073130 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4009fd0 tx=0x7fe6b400f690 comp rx=0 tx=0).stop 2024-02-02T15:26:49.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.933+0000 7fe6d3800700 1 -- 172.21.15.1:0/2892118238 shutdown_connections 2024-02-02T15:26:49.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.933+0000 7fe6d3800700 1 --2- 172.21.15.1:0/2892118238 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fe6ac0a0520 0x7fe6ac0a29e0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:49.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.933+0000 7fe6d3800700 1 --2- 172.21.15.1:0/2892118238 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe6cc1028d0 0x7fe6cc073670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:49.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.934+0000 7fe6d3800700 1 --2- 172.21.15.1:0/2892118238 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6cc100d40 0x7fe6cc074af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:49.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.934+0000 7fe6d3800700 1 --2- 172.21.15.1:0/2892118238 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe6cc101f30 0x7fe6cc073130 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:49.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.934+0000 7fe6d3800700 1 -- 172.21.15.1:0/2892118238 >> 172.21.15.1:0/2892118238 conn(0x7fe6cc0fc290 msgr2=0x7fe6cc0fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:49.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.934+0000 7fe6d3800700 1 -- 172.21.15.1:0/2892118238 shutdown_connections 2024-02-02T15:26:49.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:49.934+0000 7fe6d3800700 1 -- 172.21.15.1:0/2892118238 wait complete. 2024-02-02T15:26:49.953 DEBUG:tasks.fwd_scrub:Yielding 2024-02-02T15:26:49.953 INFO:teuthology.run_tasks:Running task exec... 2024-02-02T15:26:49.963 INFO:teuthology.task.exec:Executing custom commands... 2024-02-02T15:26:49.963 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi001.front.sepia.ceph.com 2024-02-02T15:26:49.963 DEBUG:teuthology.orchestra.run.smithi001:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-02-02T15:26:49.963 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-02-02T15:26:49.963 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-02T15:26:49.964 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-02T15:26:50.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.164+0000 7fb445200700 1 -- 172.21.15.1:0/1967529000 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb440101f60 msgr2=0x7fb4401023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:50.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.164+0000 7fb445200700 1 --2- 172.21.15.1:0/1967529000 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb440101f60 0x7fb4401023c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb4280099c0 tx=0x7fb428009cd0 comp rx=0 tx=0).stop 2024-02-02T15:26:50.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.164+0000 7fb445200700 1 -- 172.21.15.1:0/1967529000 shutdown_connections 2024-02-02T15:26:50.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.164+0000 7fb445200700 1 --2- 172.21.15.1:0/1967529000 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb440102900 0x7fb4401091b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:50.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.164+0000 7fb445200700 1 --2- 172.21.15.1:0/1967529000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb440100d70 0x7fb440101190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:50.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.164+0000 7fb445200700 1 --2- 172.21.15.1:0/1967529000 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb440101f60 0x7fb4401023c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:50.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.164+0000 7fb445200700 1 -- 172.21.15.1:0/1967529000 >> 172.21.15.1:0/1967529000 conn(0x7fb4400fc2c0 msgr2=0x7fb4400fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:50.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.164+0000 7fb445200700 1 -- 172.21.15.1:0/1967529000 shutdown_connections 2024-02-02T15:26:50.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.164+0000 7fb445200700 1 -- 172.21.15.1:0/1967529000 wait complete. 2024-02-02T15:26:50.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.165+0000 7fb445200700 1 Processor -- start 2024-02-02T15:26:50.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.165+0000 7fb445200700 1 -- start start 2024-02-02T15:26:50.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.165+0000 7fb445200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb440100d70 0x7fb440196a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:50.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.165+0000 7fb445200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb440101f60 0x7fb440196f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:50.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.165+0000 7fb445200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb440102900 0x7fb44019dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:50.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.165+0000 7fb445200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb44019e5a0 con 0x7fb440100d70 2024-02-02T15:26:50.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.165+0000 7fb445200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb44019e6e0 con 0x7fb440102900 2024-02-02T15:26:50.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.165+0000 7fb445200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb44019e870 con 0x7fb440101f60 2024-02-02T15:26:50.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.166+0000 7fb43de00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb440100d70 0x7fb440196a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:50.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.166+0000 7fb43de00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb440100d70 0x7fb440196a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53576/0 (socket says 172.21.15.1:53576) 2024-02-02T15:26:50.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.166+0000 7fb43de00700 1 -- 172.21.15.1:0/3616006573 learned_addr learned my addr 172.21.15.1:0/3616006573 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:50.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.166+0000 7fb43e800700 1 --2- 172.21.15.1:0/3616006573 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb440102900 0x7fb44019dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:50.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.166+0000 7fb43d400700 1 --2- 172.21.15.1:0/3616006573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb440101f60 0x7fb440196f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:50.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.166+0000 7fb43de00700 1 -- 172.21.15.1:0/3616006573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb440101f60 msgr2=0x7fb440196f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:50.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.166+0000 7fb43de00700 1 --2- 172.21.15.1:0/3616006573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb440101f60 0x7fb440196f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:50.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.166+0000 7fb43de00700 1 -- 172.21.15.1:0/3616006573 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb440102900 msgr2=0x7fb44019dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:50.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.166+0000 7fb43de00700 1 --2- 172.21.15.1:0/3616006573 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb440102900 0x7fb44019dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:50.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.166+0000 7fb43de00700 1 -- 172.21.15.1:0/3616006573 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4280095f0 con 0x7fb440100d70 2024-02-02T15:26:50.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.166+0000 7fb43e800700 1 --2- 172.21.15.1:0/3616006573 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb440102900 0x7fb44019dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:26:50.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.166+0000 7fb43de00700 1 --2- 172.21.15.1:0/3616006573 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb440100d70 0x7fb440196a20 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb43000d980 tx=0x7fb43000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:50.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.167+0000 7fb436a00700 1 -- 172.21.15.1:0/3616006573 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb430004020 con 0x7fb440100d70 2024-02-02T15:26:50.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.167+0000 7fb436a00700 1 -- 172.21.15.1:0/3616006573 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb43000f690 con 0x7fb440100d70 2024-02-02T15:26:50.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.167+0000 7fb436a00700 1 -- 172.21.15.1:0/3616006573 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb430003a60 con 0x7fb440100d70 2024-02-02T15:26:50.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.167+0000 7fb445200700 1 -- 172.21.15.1:0/3616006573 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb44019eb50 con 0x7fb440100d70 2024-02-02T15:26:50.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.167+0000 7fb445200700 1 -- 172.21.15.1:0/3616006573 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb44019f010 con 0x7fb440100d70 2024-02-02T15:26:50.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.168+0000 7fb445200700 1 -- 172.21.15.1:0/3616006573 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb44010c580 con 0x7fb440100d70 2024-02-02T15:26:50.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.170+0000 7fb436a00700 1 -- 172.21.15.1:0/3616006573 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) v1 ==== 135069+0+0 (secure 0 0 0) 0x7fb43000fb00 con 0x7fb440100d70 2024-02-02T15:26:50.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.171+0000 7fb436a00700 1 --2- 172.21.15.1:0/3616006573 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fb4200a0520 0x7fb4200a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:50.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.171+0000 7fb436a00700 1 -- 172.21.15.1:0/3616006573 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fb4300cb900 con 0x7fb440100d70 2024-02-02T15:26:50.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.171+0000 7fb43d400700 1 --2- 172.21.15.1:0/3616006573 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fb4200a0520 0x7fb4200a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:50.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.172+0000 7fb43d400700 1 --2- 172.21.15.1:0/3616006573 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fb4200a0520 0x7fb4200a29e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb42800b580 tx=0x7fb428005fb0 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:50.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.175+0000 7fb436a00700 1 -- 172.21.15.1:0/3616006573 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7fb4300950d0 con 0x7fb440100d70 2024-02-02T15:26:50.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.176+0000 7f088da00700 1 -- 172.21.15.1:0/1068583983 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0888101ff0 msgr2=0x7f0888104480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:50.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.176+0000 7f088da00700 1 --2- 172.21.15.1:0/1068583983 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0888101ff0 0x7f0888104480 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f0870009990 tx=0x7f0870009ca0 comp rx=0 tx=0).stop 2024-02-02T15:26:50.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.176+0000 7f088da00700 1 -- 172.21.15.1:0/1068583983 shutdown_connections 2024-02-02T15:26:50.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.176+0000 7f088da00700 1 --2- 172.21.15.1:0/1068583983 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08881049c0 0x7f0888104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:50.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.176+0000 7f088da00700 1 --2- 172.21.15.1:0/1068583983 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f08880ff680 0x7f0888101ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:50.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.176+0000 7f088da00700 1 --2- 172.21.15.1:0/1068583983 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0888101ff0 0x7f0888104480 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:50.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.176+0000 7f088da00700 1 -- 172.21.15.1:0/1068583983 >> 172.21.15.1:0/1068583983 conn(0x7f08880fb260 msgr2=0x7f08880fd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:50.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.176+0000 7f088da00700 1 -- 172.21.15.1:0/1068583983 shutdown_connections 2024-02-02T15:26:50.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.176+0000 7f088da00700 1 -- 172.21.15.1:0/1068583983 wait complete. 2024-02-02T15:26:50.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.178+0000 7f088da00700 1 Processor -- start 2024-02-02T15:26:50.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.178+0000 7f088da00700 1 -- start start 2024-02-02T15:26:50.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.178+0000 7f088da00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08880ff680 0x7f088819d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:50.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.178+0000 7f088da00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0888101ff0 0x7f088819d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:50.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.178+0000 7f088da00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f08881049c0 0x7f08881a4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:50.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.178+0000 7f088da00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08881a4c50 con 0x7f08881049c0 2024-02-02T15:26:50.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.178+0000 7f088da00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08881a4d90 con 0x7f0888101ff0 2024-02-02T15:26:50.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.178+0000 7f088da00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08881a4f20 con 0x7f08880ff680 2024-02-02T15:26:50.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.179+0000 7f0887400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f08881049c0 0x7f08881a4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:50.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.179+0000 7f0887400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f08881049c0 0x7f08881a4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53590/0 (socket says 172.21.15.1:53590) 2024-02-02T15:26:50.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.179+0000 7f0887400700 1 -- 172.21.15.1:0/453363197 learned_addr learned my addr 172.21.15.1:0/453363197 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:50.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.179+0000 7f0886000700 1 --2- 172.21.15.1:0/453363197 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0888101ff0 0x7f088819d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:50.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.179+0000 7f0886a00700 1 --2- 172.21.15.1:0/453363197 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08880ff680 0x7f088819d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:50.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.179+0000 7f0887400700 1 -- 172.21.15.1:0/453363197 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08880ff680 msgr2=0x7f088819d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:50.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.179+0000 7f0887400700 1 --2- 172.21.15.1:0/453363197 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08880ff680 0x7f088819d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:50.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.179+0000 7f0887400700 1 -- 172.21.15.1:0/453363197 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0888101ff0 msgr2=0x7f088819d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:50.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.179+0000 7f0887400700 1 --2- 172.21.15.1:0/453363197 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0888101ff0 0x7f088819d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:50.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.179+0000 7f0887400700 1 -- 172.21.15.1:0/453363197 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08700095f0 con 0x7f08881049c0 2024-02-02T15:26:50.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.179+0000 7f0886000700 1 --2- 172.21.15.1:0/453363197 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0888101ff0 0x7f088819d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:26:50.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.179+0000 7f0887400700 1 --2- 172.21.15.1:0/453363197 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f08881049c0 0x7f08881a4680 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f088000ef30 tx=0x7f088000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:50.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.180+0000 7f087f400700 1 -- 172.21.15.1:0/453363197 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0880018070 con 0x7f08881049c0 2024-02-02T15:26:50.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.180+0000 7f087f400700 1 -- 172.21.15.1:0/453363197 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f088000f040 con 0x7f08881049c0 2024-02-02T15:26:50.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.180+0000 7f087f400700 1 -- 172.21.15.1:0/453363197 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f088000cd00 con 0x7f08881049c0 2024-02-02T15:26:50.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.180+0000 7f088da00700 1 -- 172.21.15.1:0/453363197 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08881a51a0 con 0x7f08881049c0 2024-02-02T15:26:50.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.180+0000 7f088da00700 1 -- 172.21.15.1:0/453363197 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08881a5660 con 0x7f08881049c0 2024-02-02T15:26:50.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.183+0000 7f087f400700 1 -- 172.21.15.1:0/453363197 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 18) v1 ==== 135069+0+0 (secure 0 0 0) 0x7f0880003a00 con 0x7f08881049c0 2024-02-02T15:26:50.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.184+0000 7f087f400700 1 --2- 172.21.15.1:0/453363197 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f08680a05a0 0x7f08680a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:50.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.184+0000 7f087f400700 1 -- 172.21.15.1:0/453363197 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f08800cb5b0 con 0x7f08881049c0 2024-02-02T15:26:50.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.185+0000 7f0886a00700 1 --2- 172.21.15.1:0/453363197 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f08680a05a0 0x7f08680a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:50.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.185+0000 7f0886a00700 1 --2- 172.21.15.1:0/453363197 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f08680a05a0 0x7f08680a2a60 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0878009c60 tx=0x7f0878009260 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:50.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.186+0000 7f08a1594b80 1 Processor -- start 2024-02-02T15:26:50.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.187+0000 7f08a1594b80 1 -- start start 2024-02-02T15:26:50.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.187+0000 7f08a1594b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559e0c2da200 0x559e0c2da620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:50.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.187+0000 7f08a1594b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e0c2dab60 0x559e0c2db470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:50.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.187+0000 7f08a1594b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559e0c2dba00 0x559e0c2dbe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:50.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.187+0000 7f08a1594b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e0c24d540 con 0x559e0c2dba00 2024-02-02T15:26:50.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.187+0000 7f08a1594b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e0c231b20 con 0x559e0c2da200 2024-02-02T15:26:50.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.187+0000 7f08a1594b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e0c22f4e0 con 0x559e0c2dab60 2024-02-02T15:26:50.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.187+0000 7f0887400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559e0c2dba00 0x559e0c2dbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:50.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.187+0000 7f0886a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559e0c2da200 0x559e0c2da620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:50.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.188+0000 7f0886a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559e0c2da200 0x559e0c2da620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42368/0 (socket says 172.21.15.1:42368) 2024-02-02T15:26:50.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.188+0000 7f0886a00700 1 -- 172.21.15.1:0/3966306440 learned_addr learned my addr 172.21.15.1:0/3966306440 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:50.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.188+0000 7f0886000700 1 --2- 172.21.15.1:0/3966306440 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e0c2dab60 0x559e0c2db470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:50.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.188+0000 7f0886a00700 1 -- 172.21.15.1:0/3966306440 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e0c2dab60 msgr2=0x559e0c2db470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:50.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.188+0000 7f0886a00700 1 --2- 172.21.15.1:0/3966306440 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e0c2dab60 0x559e0c2db470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:50.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.188+0000 7f0886a00700 1 -- 172.21.15.1:0/3966306440 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559e0c2dba00 msgr2=0x559e0c2dbe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:50.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.188+0000 7f0886a00700 1 --2- 172.21.15.1:0/3966306440 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559e0c2dba00 0x559e0c2dbe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:50.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.188+0000 7f0886a00700 1 -- 172.21.15.1:0/3966306440 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559e0c0ac430 con 0x559e0c2da200 2024-02-02T15:26:50.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.188+0000 7f0887400700 1 --2- 172.21.15.1:0/3966306440 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559e0c2dba00 0x559e0c2dbe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:26:50.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.188+0000 7f0886a00700 1 --2- 172.21.15.1:0/3966306440 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559e0c2da200 0x559e0c2da620 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f087800e4e0 tx=0x7f0878014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29cfb1f75e1bc63b server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:50.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.188+0000 7f0877e00700 1 -- 172.21.15.1:0/3966306440 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087800ee00 con 0x559e0c2da200 2024-02-02T15:26:50.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.188+0000 7f0877e00700 1 -- 172.21.15.1:0/3966306440 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0878021070 con 0x559e0c2da200 2024-02-02T15:26:50.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.188+0000 7f0877e00700 1 -- 172.21.15.1:0/3966306440 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087801d7b0 con 0x559e0c2da200 2024-02-02T15:26:50.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.189+0000 7f08a1594b80 1 -- 172.21.15.1:0/3966306440 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559e0c2da200 msgr2=0x559e0c2da620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:50.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.189+0000 7f08a1594b80 1 --2- 172.21.15.1:0/3966306440 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559e0c2da200 0x559e0c2da620 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f087800e4e0 tx=0x7f0878014000 comp rx=0 tx=0).stop 2024-02-02T15:26:50.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.190+0000 7f08a1594b80 1 -- 172.21.15.1:0/3966306440 shutdown_connections 2024-02-02T15:26:50.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.190+0000 7f08a1594b80 1 --2- 172.21.15.1:0/3966306440 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559e0c2dba00 0x559e0c2dbe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:50.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.190+0000 7f08a1594b80 1 --2- 172.21.15.1:0/3966306440 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559e0c2da200 0x559e0c2da620 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:50.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.190+0000 7f08a1594b80 1 --2- 172.21.15.1:0/3966306440 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e0c2dab60 0x559e0c2db470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:50.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.190+0000 7f08a1594b80 1 -- 172.21.15.1:0/3966306440 >> 172.21.15.1:0/3966306440 conn(0x559e0c238e30 msgr2=0x559e0c2d4f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:50.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.190+0000 7f08a1594b80 1 -- 172.21.15.1:0/3966306440 shutdown_connections 2024-02-02T15:26:50.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.190+0000 7f08a1594b80 1 -- 172.21.15.1:0/3966306440 wait complete. 2024-02-02T15:26:50.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.190+0000 7f08a1594b80 1 -- start start 2024-02-02T15:26:50.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.191+0000 7f08a1594b80 1 Processor -- start 2024-02-02T15:26:50.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.191+0000 7f08a1594b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559e0c2da200 0x559e0c2e3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:50.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.191+0000 7f08a1594b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559e0c2dab60 0x559e0c2e44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:50.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.191+0000 7f08a1594b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e0c2dba00 0x559e0c2eb500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:50.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.191+0000 7f08a1594b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e0c22add0 con 0x559e0c2dab60 2024-02-02T15:26:50.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.191+0000 7f08a1594b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e0c1c6e00 con 0x559e0c2da200 2024-02-02T15:26:50.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.191+0000 7f08a1594b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e0c1c6aa0 con 0x559e0c2dba00 2024-02-02T15:26:50.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.191+0000 7f08a1594b80 10 client.0 _ll_register_callbacks cb 0x559e0c1c8310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:26:50.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.192+0000 7f0886000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559e0c2dab60 0x559e0c2e44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:50.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.192+0000 7f0886a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559e0c2da200 0x559e0c2e3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:50.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.192+0000 7f0886000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559e0c2dab60 0x559e0c2e44c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53620/0 (socket says 172.21.15.1:53620) 2024-02-02T15:26:50.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.192+0000 7f0886000700 1 -- 172.21.15.1:0/1369722552 learned_addr learned my addr 172.21.15.1:0/1369722552 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:50.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.192+0000 7f0887400700 1 --2- 172.21.15.1:0/1369722552 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e0c2dba00 0x559e0c2eb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:50.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.192+0000 7f0886a00700 1 -- 172.21.15.1:0/1369722552 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e0c2dba00 msgr2=0x559e0c2eb500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:50.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.192+0000 7f0886a00700 1 --2- 172.21.15.1:0/1369722552 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e0c2dba00 0x559e0c2eb500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:50.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.192+0000 7f0886a00700 1 -- 172.21.15.1:0/1369722552 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559e0c2dab60 msgr2=0x559e0c2e44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:50.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.192+0000 7f0886a00700 1 --2- 172.21.15.1:0/1369722552 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559e0c2dab60 0x559e0c2e44c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:50.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.192+0000 7f0886a00700 1 -- 172.21.15.1:0/1369722552 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08881a5660 con 0x559e0c2da200 2024-02-02T15:26:50.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.192+0000 7f0886a00700 1 --2- 172.21.15.1:0/1369722552 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559e0c2da200 0x559e0c2e3f80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f087800ead0 tx=0x7f087800ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14012d79629ca7cb server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:50.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.192+0000 7f0874c00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:26:50.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.193+0000 7f0874c00700 1 -- 172.21.15.1:0/1369722552 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0878005350 con 0x559e0c2da200 2024-02-02T15:26:50.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.193+0000 7f08a1594b80 1 -- 172.21.15.1:0/1369722552 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559e0c2cd170 con 0x559e0c2da200 2024-02-02T15:26:50.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.193+0000 7f0874c00700 1 -- 172.21.15.1:0/1369722552 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f08780054b0 con 0x559e0c2da200 2024-02-02T15:26:50.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.193+0000 7f0874c00700 1 -- 172.21.15.1:0/1369722552 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0878005670 con 0x559e0c2da200 2024-02-02T15:26:50.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.193+0000 7f08a1594b80 10 client.25559 fetch_fsmap learned FSMap version 27 2024-02-02T15:26:50.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.193+0000 7f08a1594b80 1 -- 172.21.15.1:0/1369722552 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f0878017070 con 0x559e0c2da200 2024-02-02T15:26:50.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.193+0000 7f0874c00700 1 -- 172.21.15.1:0/1369722552 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559e0c2cd170 con 0x559e0c2da200 2024-02-02T15:26:50.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.193+0000 7f0874c00700 1 -- 172.21.15.1:0/1369722552 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f087801c860 con 0x559e0c2da200 2024-02-02T15:26:50.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.193+0000 7f08a1594b80 10 client.25559 fetch_fsmap finished waiting for FSMap version 27 2024-02-02T15:26:50.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.193+0000 7f08a1594b80 10 client.25559 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:26:50.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.193+0000 7f08a1594b80 20 client.25559 populate_metadata read hostname 'smithi001' 2024-02-02T15:26:50.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.194+0000 7f08a1594b80 1 --2- 172.21.15.1:0/1369722552 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x559e0c2efcb0 0x559e0c2f20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:50.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.194+0000 7f08a1594b80 4 client.25559 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:26:50.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.194+0000 7f08a1594b80 1 -- 172.21.15.1:0/1369722552 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559e0c223e50 con 0x559e0c2efcb0 2024-02-02T15:26:50.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.194+0000 7f0886000700 1 --2- 172.21.15.1:0/1369722552 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x559e0c2efcb0 0x559e0c2f20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:50.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.195+0000 7f0886000700 1 --2- 172.21.15.1:0/1369722552 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x559e0c2efcb0 0x559e0c2f20d0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dab3e01443acb440 server_cookie=629c85f219cb3b59 in_seq=0 out_seq=0 2024-02-02T15:26:50.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.195+0000 7f0874c00700 10 client.25559 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:26:50.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.195+0000 7f0874c00700 1 -- 172.21.15.1:0/1369722552 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x559e0c223e50 con 0x559e0c2efcb0 2024-02-02T15:26:50.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.195+0000 7f0874c00700 10 client.25559 handle_command_reply: tid=0 2024-02-02T15:26:50.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.195+0000 7f08a1594b80 1 client.25559 shutdown 2024-02-02T15:26:50.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.196+0000 7f08a1594b80 1 -- 172.21.15.1:0/1369722552 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559e0c2da200 msgr2=0x559e0c2e3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:50.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.196+0000 7f08a1594b80 1 --2- 172.21.15.1:0/1369722552 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559e0c2da200 0x559e0c2e3f80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f087800ead0 tx=0x7f087800ee90 comp rx=0 tx=0).stop 2024-02-02T15:26:50.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.196+0000 7f08a1594b80 1 -- 172.21.15.1:0/1369722552 shutdown_connections 2024-02-02T15:26:50.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.196+0000 7f08a1594b80 1 --2- 172.21.15.1:0/1369722552 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x559e0c2efcb0 0x559e0c2f20d0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:50.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.196+0000 7f08a1594b80 1 --2- 172.21.15.1:0/1369722552 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e0c2dba00 0x559e0c2eb500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:50.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.196+0000 7f08a1594b80 1 --2- 172.21.15.1:0/1369722552 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559e0c2da200 0x559e0c2e3f80 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:50.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.196+0000 7f08a1594b80 1 --2- 172.21.15.1:0/1369722552 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559e0c2dab60 0x559e0c2e44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:50.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.196+0000 7f0874c00700 0 client.25559 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:26:50.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.196+0000 7f08a1594b80 1 -- 172.21.15.1:0/1369722552 >> 172.21.15.1:0/1369722552 conn(0x559e0c238e30 msgr2=0x559e0c2d4640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:50.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.196+0000 7f08a1594b80 1 -- 172.21.15.1:0/1369722552 shutdown_connections 2024-02-02T15:26:50.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.196+0000 7f08a1594b80 1 -- 172.21.15.1:0/1369722552 wait complete. 2024-02-02T15:26:50.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.196+0000 7f08a1594b80 20 client.25559 trim_cache size 0 max 16384 2024-02-02T15:26:50.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.213+0000 7f08a1594b80 1 Processor -- start 2024-02-02T15:26:50.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.213+0000 7f08a1594b80 1 -- start start 2024-02-02T15:26:50.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.214+0000 7f08a1594b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559e0c2da200 0x559e0c2e94b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:50.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.214+0000 7f08a1594b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e0c2dab60 0x559e0c2e3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:50.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.214+0000 7f08a1594b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559e0c2dba00 0x559e0c2e4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:50.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.214+0000 7f08a1594b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e0c0ae720 con 0x559e0c2da200 2024-02-02T15:26:50.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.214+0000 7f08a1594b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e0c2e26e0 con 0x559e0c2dba00 2024-02-02T15:26:50.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.214+0000 7f08a1594b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e0c2d5dc0 con 0x559e0c2dab60 2024-02-02T15:26:50.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.214+0000 7f0886000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e0c2dab60 0x559e0c2e3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:50.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.214+0000 7f0886a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559e0c2da200 0x559e0c2e94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:50.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.214+0000 7f0886000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e0c2dab60 0x559e0c2e3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:41646/0 (socket says 172.21.15.1:41646) 2024-02-02T15:26:50.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.214+0000 7f0886000700 1 -- 172.21.15.1:0/3980538922 learned_addr learned my addr 172.21.15.1:0/3980538922 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:50.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.214+0000 7f0887400700 1 --2- 172.21.15.1:0/3980538922 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559e0c2dba00 0x559e0c2e4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:50.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.215+0000 7f0886000700 1 -- 172.21.15.1:0/3980538922 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559e0c2dba00 msgr2=0x559e0c2e4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:50.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.215+0000 7f0886000700 1 --2- 172.21.15.1:0/3980538922 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559e0c2dba00 0x559e0c2e4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:50.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.215+0000 7f0886000700 1 -- 172.21.15.1:0/3980538922 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559e0c2da200 msgr2=0x559e0c2e94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:50.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.215+0000 7f0886000700 1 --2- 172.21.15.1:0/3980538922 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559e0c2da200 0x559e0c2e94b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:50.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.215+0000 7f0886000700 1 -- 172.21.15.1:0/3980538922 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559e0c2d1150 con 0x559e0c2dab60 2024-02-02T15:26:50.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.215+0000 7f0886000700 1 --2- 172.21.15.1:0/3980538922 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e0c2dab60 0x559e0c2e3d30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0870009990 tx=0x7f0870005fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d8da301455e779 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:50.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.215+0000 7f087cc00700 1 -- 172.21.15.1:0/3980538922 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087001d040 con 0x559e0c2dab60 2024-02-02T15:26:50.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.215+0000 7f087cc00700 1 -- 172.21.15.1:0/3980538922 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f087001e040 con 0x559e0c2dab60 2024-02-02T15:26:50.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.215+0000 7f087cc00700 1 -- 172.21.15.1:0/3980538922 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0870005530 con 0x559e0c2dab60 2024-02-02T15:26:50.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.216+0000 7f08a1594b80 1 -- 172.21.15.1:0/3980538922 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e0c2dab60 msgr2=0x559e0c2e3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:50.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.216+0000 7f08a1594b80 1 --2- 172.21.15.1:0/3980538922 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e0c2dab60 0x559e0c2e3d30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0870009990 tx=0x7f0870005fb0 comp rx=0 tx=0).stop 2024-02-02T15:26:50.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.216+0000 7f08a1594b80 1 -- 172.21.15.1:0/3980538922 shutdown_connections 2024-02-02T15:26:50.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.216+0000 7f08a1594b80 1 --2- 172.21.15.1:0/3980538922 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559e0c2dba00 0x559e0c2e4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:50.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.216+0000 7f08a1594b80 1 --2- 172.21.15.1:0/3980538922 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559e0c2da200 0x559e0c2e94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:50.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.216+0000 7f08a1594b80 1 --2- 172.21.15.1:0/3980538922 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e0c2dab60 0x559e0c2e3d30 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:50.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.216+0000 7f08a1594b80 1 -- 172.21.15.1:0/3980538922 >> 172.21.15.1:0/3980538922 conn(0x559e0c238e30 msgr2=0x559e0c2de9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:50.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.216+0000 7f08a1594b80 1 -- 172.21.15.1:0/3980538922 shutdown_connections 2024-02-02T15:26:50.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.216+0000 7f08a1594b80 1 -- 172.21.15.1:0/3980538922 wait complete. 2024-02-02T15:26:50.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.216+0000 7f08a1594b80 1 -- start start 2024-02-02T15:26:50.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.217+0000 7f08a1594b80 1 Processor -- start 2024-02-02T15:26:50.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.217+0000 7f08a1594b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e0c2da200 0x559e0c33a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:50.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.217+0000 7f08a1594b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559e0c2dab60 0x559e0c33a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:50.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.217+0000 7f08a1594b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559e0c2dba00 0x559e0c3418c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:50.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.217+0000 7f08a1594b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e0c341e00 con 0x559e0c2dba00 2024-02-02T15:26:50.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.217+0000 7f08a1594b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e0c341f40 con 0x559e0c2dab60 2024-02-02T15:26:50.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.217+0000 7f08a1594b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e0c3420d0 con 0x559e0c2da200 2024-02-02T15:26:50.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.217+0000 7f08a1594b80 10 client.0 _ll_register_callbacks cb 0x559e0c1c8310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:26:50.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.217+0000 7f0886000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559e0c2dab60 0x559e0c33a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:50.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.217+0000 7f0886000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559e0c2dab60 0x559e0c33a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42400/0 (socket says 172.21.15.1:42400) 2024-02-02T15:26:50.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.217+0000 7f0886000700 1 -- 172.21.15.1:0/812700122 learned_addr learned my addr 172.21.15.1:0/812700122 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:50.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.217+0000 7f0887400700 1 --2- 172.21.15.1:0/812700122 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559e0c2dba00 0x559e0c3418c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:50.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.218+0000 7f0886a00700 1 --2- 172.21.15.1:0/812700122 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e0c2da200 0x559e0c33a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:50.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.218+0000 7f0886000700 1 -- 172.21.15.1:0/812700122 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e0c2da200 msgr2=0x559e0c33a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:50.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.218+0000 7f0886000700 1 --2- 172.21.15.1:0/812700122 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e0c2da200 0x559e0c33a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:50.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.218+0000 7f0886000700 1 -- 172.21.15.1:0/812700122 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559e0c2dba00 msgr2=0x559e0c3418c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:50.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.218+0000 7f0886000700 1 --2- 172.21.15.1:0/812700122 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559e0c2dba00 0x559e0c3418c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:50.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.218+0000 7f0886000700 1 -- 172.21.15.1:0/812700122 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0888105970 con 0x559e0c2dab60 2024-02-02T15:26:50.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.218+0000 7f0886000700 1 --2- 172.21.15.1:0/812700122 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559e0c2dab60 0x559e0c33a880 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f087000b2e0 tx=0x7f0870009540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=973267577f760c6d server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:50.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.218+0000 7f0874c00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:26:50.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.218+0000 7f0874c00700 1 -- 172.21.15.1:0/812700122 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087001d040 con 0x559e0c2dab60 2024-02-02T15:26:50.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.219+0000 7f0874c00700 1 -- 172.21.15.1:0/812700122 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f08700054b0 con 0x559e0c2dab60 2024-02-02T15:26:50.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.219+0000 7f08a1594b80 1 -- 172.21.15.1:0/812700122 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0878021070 con 0x559e0c2dab60 2024-02-02T15:26:50.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.219+0000 7f0874c00700 1 -- 172.21.15.1:0/812700122 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0870005670 con 0x559e0c2dab60 2024-02-02T15:26:50.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.219+0000 7f08a1594b80 10 client.25565 fetch_fsmap learned FSMap version 27 2024-02-02T15:26:50.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.219+0000 7f08a1594b80 1 -- 172.21.15.1:0/812700122 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x559e0c058be0 con 0x559e0c2dab60 2024-02-02T15:26:50.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.219+0000 7f0874c00700 1 -- 172.21.15.1:0/812700122 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0878021070 con 0x559e0c2dab60 2024-02-02T15:26:50.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.219+0000 7f0874c00700 1 -- 172.21.15.1:0/812700122 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0870004b70 con 0x559e0c2dab60 2024-02-02T15:26:50.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.219+0000 7f08a1594b80 10 client.25565 fetch_fsmap finished waiting for FSMap version 27 2024-02-02T15:26:50.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.219+0000 7f08a1594b80 10 client.25565 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:26:50.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.219+0000 7f08a1594b80 20 client.25565 populate_metadata read hostname 'smithi001' 2024-02-02T15:26:50.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.219+0000 7f08a1594b80 1 --2- 172.21.15.1:0/812700122 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x559e0c3436d0 0x559e0c345af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:50.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.219+0000 7f08a1594b80 4 client.25565 mds_command: new command op to 24557 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-02T15:26:50.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.219+0000 7f08a1594b80 1 -- 172.21.15.1:0/812700122 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x559e0c1e9fd0 con 0x559e0c3436d0 2024-02-02T15:26:50.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.220+0000 7f0886a00700 1 --2- 172.21.15.1:0/812700122 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x559e0c3436d0 0x559e0c345af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:50.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.220+0000 7f0886a00700 1 --2- 172.21.15.1:0/812700122 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x559e0c3436d0 0x559e0c345af0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a80d191647a23bed server_cookie=bcf837a3a43bd06f in_seq=0 out_seq=0 2024-02-02T15:26:50.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.220+0000 7f0874c00700 10 client.25565 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:26:50.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.221+0000 7f0874c00700 1 -- 172.21.15.1:0/812700122 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x559e0c1e9fd0 con 0x559e0c3436d0 2024-02-02T15:26:50.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.221+0000 7f0874c00700 10 client.25565 handle_command_reply: tid=0 2024-02-02T15:26:50.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.221+0000 7f08a1594b80 1 client.25565 shutdown 2024-02-02T15:26:50.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.221+0000 7f08a1594b80 1 -- 172.21.15.1:0/812700122 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559e0c2dab60 msgr2=0x559e0c33a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:50.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.221+0000 7f08a1594b80 1 --2- 172.21.15.1:0/812700122 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559e0c2dab60 0x559e0c33a880 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f087000b2e0 tx=0x7f0870009540 comp rx=0 tx=0).stop 2024-02-02T15:26:50.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.221+0000 7f08a1594b80 1 -- 172.21.15.1:0/812700122 shutdown_connections 2024-02-02T15:26:50.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.221+0000 7f08a1594b80 1 --2- 172.21.15.1:0/812700122 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x559e0c3436d0 0x559e0c345af0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:50.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.221+0000 7f08a1594b80 1 --2- 172.21.15.1:0/812700122 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559e0c2dba00 0x559e0c3418c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:50.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.221+0000 7f0874c00700 0 client.25565 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:26:50.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.221+0000 7f08a1594b80 1 --2- 172.21.15.1:0/812700122 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559e0c2da200 0x559e0c33a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:50.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.221+0000 7f08a1594b80 1 --2- 172.21.15.1:0/812700122 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559e0c2dab60 0x559e0c33a880 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:50.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.221+0000 7f08a1594b80 1 -- 172.21.15.1:0/812700122 >> 172.21.15.1:0/812700122 conn(0x559e0c238e30 msgr2=0x559e0c2e3d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:50.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.222+0000 7f08a1594b80 1 -- 172.21.15.1:0/812700122 shutdown_connections 2024-02-02T15:26:50.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.222+0000 7f08a1594b80 1 -- 172.21.15.1:0/812700122 wait complete. 2024-02-02T15:26:50.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.222+0000 7f08a1594b80 20 client.25565 trim_cache size 0 max 16384 2024-02-02T15:26:50.225 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:26:50.225 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2024-02-02T15:26:50.225 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "f6829dab-4e53-49fb-af6f-f5a676339bda", 2024-02-02T15:26:50.225 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2024-02-02T15:26:50.225 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:26:50.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.222+0000 7f088da00700 1 -- 172.21.15.1:0/453363197 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f08680a05a0 msgr2=0x7f08680a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:50.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.222+0000 7f088da00700 1 --2- 172.21.15.1:0/453363197 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f08680a05a0 0x7f08680a2a60 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f0878009c60 tx=0x7f0878009260 comp rx=0 tx=0).stop 2024-02-02T15:26:50.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.222+0000 7f088da00700 1 -- 172.21.15.1:0/453363197 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f08881049c0 msgr2=0x7f08881a4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:50.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.222+0000 7f088da00700 1 --2- 172.21.15.1:0/453363197 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f08881049c0 0x7f08881a4680 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f088000ef30 tx=0x7f088000c3a0 comp rx=0 tx=0).stop 2024-02-02T15:26:50.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.223+0000 7f088da00700 1 -- 172.21.15.1:0/453363197 shutdown_connections 2024-02-02T15:26:50.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.223+0000 7f088da00700 1 --2- 172.21.15.1:0/453363197 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f08881049c0 0x7f08881a4680 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:50.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.223+0000 7f088da00700 1 --2- 172.21.15.1:0/453363197 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f08680a05a0 0x7f08680a2a60 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:50.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.223+0000 7f088da00700 1 --2- 172.21.15.1:0/453363197 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08880ff680 0x7f088819d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:50.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.223+0000 7f088da00700 1 --2- 172.21.15.1:0/453363197 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0888101ff0 0x7f088819d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:50.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.223+0000 7f088da00700 1 -- 172.21.15.1:0/453363197 >> 172.21.15.1:0/453363197 conn(0x7f08880fb260 msgr2=0x7f0888100390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:50.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.223+0000 7f088da00700 1 -- 172.21.15.1:0/453363197 shutdown_connections 2024-02-02T15:26:50.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.223+0000 7f088da00700 1 -- 172.21.15.1:0/453363197 wait complete. 2024-02-02T15:26:50.239 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-02T15:26:50.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.442+0000 7f101e600700 1 -- 172.21.15.1:0/4229022968 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1018074d80 msgr2=0x7f10180731d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:50.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.442+0000 7f101e600700 1 --2- 172.21.15.1:0/4229022968 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1018074d80 0x7f10180731d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f10080098f0 tx=0x7f1008009c00 comp rx=0 tx=0).stop 2024-02-02T15:26:50.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.442+0000 7f101e600700 1 -- 172.21.15.1:0/4229022968 shutdown_connections 2024-02-02T15:26:50.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.442+0000 7f101e600700 1 --2- 172.21.15.1:0/4229022968 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1018108ce0 0x7f101810b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:50.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.442+0000 7f101e600700 1 --2- 172.21.15.1:0/4229022968 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1018074d80 0x7f10180731d0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:50.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.442+0000 7f101e600700 1 --2- 172.21.15.1:0/4229022968 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f10180737a0 0x7f1018073c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:50.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.442+0000 7f101e600700 1 -- 172.21.15.1:0/4229022968 >> 172.21.15.1:0/4229022968 conn(0x7f10180fc390 msgr2=0x7f10180fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:50.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.442+0000 7f101e600700 1 -- 172.21.15.1:0/4229022968 shutdown_connections 2024-02-02T15:26:50.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.442+0000 7f101e600700 1 -- 172.21.15.1:0/4229022968 wait complete. 2024-02-02T15:26:50.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.443+0000 7f101e600700 1 Processor -- start 2024-02-02T15:26:50.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.443+0000 7f101e600700 1 -- start start 2024-02-02T15:26:50.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.444+0000 7f101e600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f10180737a0 0x7f1018198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:50.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.444+0000 7f101e600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1018074d80 0x7f10181992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:50.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.444+0000 7f101e600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1018108ce0 0x7f10181a0320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:50.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.444+0000 7f101e600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10181a08f0 con 0x7f1018108ce0 2024-02-02T15:26:50.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.444+0000 7f101e600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10181a0a30 con 0x7f10180737a0 2024-02-02T15:26:50.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.444+0000 7f101e600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10181a0bc0 con 0x7f1018074d80 2024-02-02T15:26:50.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.444+0000 7f1016a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1018074d80 0x7f10181992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:50.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.444+0000 7f1017e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1018108ce0 0x7f10181a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:50.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.444+0000 7f1016a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1018074d80 0x7f10181992b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:41666/0 (socket says 172.21.15.1:41666) 2024-02-02T15:26:50.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.444+0000 7f1017400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f10180737a0 0x7f1018198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:50.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.444+0000 7f1016a00700 1 -- 172.21.15.1:0/33652797 learned_addr learned my addr 172.21.15.1:0/33652797 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:50.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.444+0000 7f1017400700 1 -- 172.21.15.1:0/33652797 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1018074d80 msgr2=0x7f10181992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:50.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.444+0000 7f1017400700 1 --2- 172.21.15.1:0/33652797 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1018074d80 0x7f10181992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:50.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.445+0000 7f1017400700 1 -- 172.21.15.1:0/33652797 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1018108ce0 msgr2=0x7f10181a0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:50.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.445+0000 7f1017400700 1 --2- 172.21.15.1:0/33652797 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1018108ce0 0x7f10181a0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:50.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.445+0000 7f1017400700 1 -- 172.21.15.1:0/33652797 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10000095f0 con 0x7f10180737a0 2024-02-02T15:26:50.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.445+0000 7f1016a00700 1 --2- 172.21.15.1:0/33652797 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1018074d80 0x7f10181992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:26:50.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.445+0000 7f1017e00700 1 --2- 172.21.15.1:0/33652797 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1018108ce0 0x7f10181a0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:26:50.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.445+0000 7f1017400700 1 --2- 172.21.15.1:0/33652797 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f10180737a0 0x7f1018198d70 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1008009230 tx=0x7f100800b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:50.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.445+0000 7f100fe00700 1 -- 172.21.15.1:0/33652797 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f100801c070 con 0x7f10180737a0 2024-02-02T15:26:50.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.445+0000 7f101e600700 1 -- 172.21.15.1:0/33652797 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1008009520 con 0x7f10180737a0 2024-02-02T15:26:50.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.445+0000 7f100fe00700 1 -- 172.21.15.1:0/33652797 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f100800f7d0 con 0x7f10180737a0 2024-02-02T15:26:50.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.445+0000 7f100fe00700 1 -- 172.21.15.1:0/33652797 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1008003a00 con 0x7f10180737a0 2024-02-02T15:26:50.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.445+0000 7f101e600700 1 -- 172.21.15.1:0/33652797 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10181a1130 con 0x7f10180737a0 2024-02-02T15:26:50.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.448+0000 7f100fe00700 1 -- 172.21.15.1:0/33652797 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) v1 ==== 135069+0+0 (secure 0 0 0) 0x7f1008020030 con 0x7f10180737a0 2024-02-02T15:26:50.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.450+0000 7f100fe00700 1 --2- 172.21.15.1:0/33652797 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f0ff80a05a0 0x7f0ff80a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:50.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.450+0000 7f100fe00700 1 -- 172.21.15.1:0/33652797 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f10080ccaf0 con 0x7f10180737a0 2024-02-02T15:26:50.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.450+0000 7f1016a00700 1 --2- 172.21.15.1:0/33652797 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f0ff80a05a0 0x7f0ff80a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:50.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.450+0000 7f1016a00700 1 --2- 172.21.15.1:0/33652797 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f0ff80a05a0 0x7f0ff80a2a60 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1000005950 tx=0x7f1000009310 comp rx=0 tx=0).ready entity=mgr.14154 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:50.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.452+0000 7f10320b6b80 1 Processor -- start 2024-02-02T15:26:50.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.452+0000 7f10320b6b80 1 -- start start 2024-02-02T15:26:50.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.452+0000 7f10320b6b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561a78b0a180 0x561a78b0a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:50.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.452+0000 7f10320b6b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561a78b0aae0 0x561a78b0b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:50.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.452+0000 7f10320b6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a78b0b980 0x561a78b0be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:50.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.453+0000 7f10320b6b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a78a7d4c0 con 0x561a78b0aae0 2024-02-02T15:26:50.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.453+0000 7f10320b6b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a78a61aa0 con 0x561a78b0a180 2024-02-02T15:26:50.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.453+0000 7f10320b6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a78a5f460 con 0x561a78b0b980 2024-02-02T15:26:50.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.453+0000 7f1016a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561a78b0aae0 0x561a78b0b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:50.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.453+0000 7f1017400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561a78b0a180 0x561a78b0a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:50.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.453+0000 7f1017400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561a78b0a180 0x561a78b0a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42428/0 (socket says 172.21.15.1:42428) 2024-02-02T15:26:50.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.453+0000 7f1017400700 1 -- 172.21.15.1:0/201410868 learned_addr learned my addr 172.21.15.1:0/201410868 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:50.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.453+0000 7f1017e00700 1 --2- 172.21.15.1:0/201410868 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a78b0b980 0x561a78b0be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:50.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.453+0000 7f1017400700 1 -- 172.21.15.1:0/201410868 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a78b0b980 msgr2=0x561a78b0be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:50.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.453+0000 7f1017400700 1 --2- 172.21.15.1:0/201410868 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a78b0b980 0x561a78b0be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:50.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.453+0000 7f1017400700 1 -- 172.21.15.1:0/201410868 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561a78b0aae0 msgr2=0x561a78b0b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:50.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.453+0000 7f1017400700 1 --2- 172.21.15.1:0/201410868 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561a78b0aae0 0x561a78b0b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:50.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.453+0000 7f1017400700 1 -- 172.21.15.1:0/201410868 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561a788dc3b0 con 0x561a78b0a180 2024-02-02T15:26:50.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.453+0000 7f1017400700 1 --2- 172.21.15.1:0/201410868 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561a78b0a180 0x561a78b0a5a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f10080a1ac0 tx=0x7f1008009f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e03bb5b75f44afa server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:50.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.454+0000 7f100cc00700 1 -- 172.21.15.1:0/201410868 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10080b1070 con 0x561a78b0a180 2024-02-02T15:26:50.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.454+0000 7f100cc00700 1 -- 172.21.15.1:0/201410868 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f100809a710 con 0x561a78b0a180 2024-02-02T15:26:50.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.454+0000 7f100cc00700 1 -- 172.21.15.1:0/201410868 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10080983b0 con 0x561a78b0a180 2024-02-02T15:26:50.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.455+0000 7f10320b6b80 1 -- 172.21.15.1:0/201410868 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561a78b0a180 msgr2=0x561a78b0a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:50.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.455+0000 7f10320b6b80 1 --2- 172.21.15.1:0/201410868 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561a78b0a180 0x561a78b0a5a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f10080a1ac0 tx=0x7f1008009f50 comp rx=0 tx=0).stop 2024-02-02T15:26:50.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.455+0000 7f10320b6b80 1 -- 172.21.15.1:0/201410868 shutdown_connections 2024-02-02T15:26:50.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.455+0000 7f10320b6b80 1 --2- 172.21.15.1:0/201410868 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a78b0b980 0x561a78b0be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:50.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.455+0000 7f10320b6b80 1 --2- 172.21.15.1:0/201410868 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561a78b0a180 0x561a78b0a5a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:50.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.455+0000 7f10320b6b80 1 --2- 172.21.15.1:0/201410868 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561a78b0aae0 0x561a78b0b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:50.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.455+0000 7f10320b6b80 1 -- 172.21.15.1:0/201410868 >> 172.21.15.1:0/201410868 conn(0x561a78a68db0 msgr2=0x561a78b04ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:50.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.455+0000 7f10320b6b80 1 -- 172.21.15.1:0/201410868 shutdown_connections 2024-02-02T15:26:50.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.455+0000 7f10320b6b80 1 -- 172.21.15.1:0/201410868 wait complete. 2024-02-02T15:26:50.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.455+0000 7f10320b6b80 1 -- start start 2024-02-02T15:26:50.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.456+0000 7f10320b6b80 1 Processor -- start 2024-02-02T15:26:50.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.456+0000 7f10320b6b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561a78b0a180 0x561a78b13c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:50.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.456+0000 7f10320b6b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561a78b0aae0 0x561a78b14190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:50.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.456+0000 7f10320b6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a78b0b980 0x561a78b1b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:50.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.456+0000 7f10320b6b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a78a5ad50 con 0x561a78b0aae0 2024-02-02T15:26:50.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.456+0000 7f10320b6b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a789f6d80 con 0x561a78b0a180 2024-02-02T15:26:50.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.456+0000 7f10320b6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a789f6a20 con 0x561a78b0b980 2024-02-02T15:26:50.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.456+0000 7f10320b6b80 10 client.0 _ll_register_callbacks cb 0x561a789f8290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:26:50.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.456+0000 7f1016a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561a78b0aae0 0x561a78b14190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:50.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.456+0000 7f1016a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561a78b0aae0 0x561a78b14190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53670/0 (socket says 172.21.15.1:53670) 2024-02-02T15:26:50.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.456+0000 7f1016a00700 1 -- 172.21.15.1:0/2287737728 learned_addr learned my addr 172.21.15.1:0/2287737728 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:50.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.456+0000 7f1017400700 1 --2- 172.21.15.1:0/2287737728 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561a78b0a180 0x561a78b13c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:50.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.456+0000 7f1017e00700 1 --2- 172.21.15.1:0/2287737728 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a78b0b980 0x561a78b1b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:50.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.457+0000 7f1017400700 1 -- 172.21.15.1:0/2287737728 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a78b0b980 msgr2=0x561a78b1b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:50.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.457+0000 7f1017400700 1 --2- 172.21.15.1:0/2287737728 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a78b0b980 0x561a78b1b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:50.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.457+0000 7f1017400700 1 -- 172.21.15.1:0/2287737728 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561a78b0aae0 msgr2=0x561a78b14190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:50.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.457+0000 7f1017400700 1 --2- 172.21.15.1:0/2287737728 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561a78b0aae0 0x561a78b14190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:50.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.457+0000 7f1017400700 1 -- 172.21.15.1:0/2287737728 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10080b9040 con 0x561a78b0a180 2024-02-02T15:26:50.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.457+0000 7f1017400700 1 --2- 172.21.15.1:0/2287737728 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561a78b0a180 0x561a78b13c50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f100801f040 tx=0x7f100800b400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97b048f88e4ea878 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:50.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.457+0000 7f1005600700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:26:50.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.457+0000 7f1005600700 1 -- 172.21.15.1:0/2287737728 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10080b1050 con 0x561a78b0a180 2024-02-02T15:26:50.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.457+0000 7f1005600700 1 -- 172.21.15.1:0/2287737728 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f100809de40 con 0x561a78b0a180 2024-02-02T15:26:50.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.457+0000 7f10320b6b80 1 -- 172.21.15.1:0/2287737728 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561a78afd0f0 con 0x561a78b0a180 2024-02-02T15:26:50.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.458+0000 7f1005600700 1 -- 172.21.15.1:0/2287737728 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f100809a010 con 0x561a78b0a180 2024-02-02T15:26:50.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.458+0000 7f10320b6b80 10 client.25577 fetch_fsmap learned FSMap version 27 2024-02-02T15:26:50.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.458+0000 7f10320b6b80 1 -- 172.21.15.1:0/2287737728 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x561a78888b60 con 0x561a78b0a180 2024-02-02T15:26:50.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.458+0000 7f1005600700 1 -- 172.21.15.1:0/2287737728 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561a78afd0f0 con 0x561a78b0a180 2024-02-02T15:26:50.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.458+0000 7f1005600700 1 -- 172.21.15.1:0/2287737728 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1008099620 con 0x561a78b0a180 2024-02-02T15:26:50.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.458+0000 7f10320b6b80 10 client.25577 fetch_fsmap finished waiting for FSMap version 27 2024-02-02T15:26:50.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.458+0000 7f10320b6b80 10 client.25577 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:26:50.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.458+0000 7f10320b6b80 20 client.25577 populate_metadata read hostname 'smithi001' 2024-02-02T15:26:50.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.458+0000 7f10320b6b80 1 --2- 172.21.15.1:0/2287737728 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x561a78b1f450 0x561a78b21840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:50.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.458+0000 7f10320b6b80 4 client.25577 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:26:50.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.458+0000 7f10320b6b80 1 -- 172.21.15.1:0/2287737728 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561a78a53dd0 con 0x561a78b1f450 2024-02-02T15:26:50.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.459+0000 7f1016a00700 1 --2- 172.21.15.1:0/2287737728 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x561a78b1f450 0x561a78b21840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:50.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.459+0000 7f1016a00700 1 --2- 172.21.15.1:0/2287737728 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x561a78b1f450 0x561a78b21840 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=509c9dc5f2034685 server_cookie=bad613418dc02a9 in_seq=0 out_seq=0 2024-02-02T15:26:50.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.459+0000 7f1005600700 10 client.25577 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:26:50.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.460+0000 7f1005600700 1 -- 172.21.15.1:0/2287737728 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x561a78a53dd0 con 0x561a78b1f450 2024-02-02T15:26:50.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.460+0000 7f1005600700 10 client.25577 handle_command_reply: tid=0 2024-02-02T15:26:50.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.460+0000 7f10320b6b80 1 client.25577 shutdown 2024-02-02T15:26:50.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.460+0000 7f10320b6b80 1 -- 172.21.15.1:0/2287737728 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561a78b0a180 msgr2=0x561a78b13c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:50.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.460+0000 7f10320b6b80 1 --2- 172.21.15.1:0/2287737728 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561a78b0a180 0x561a78b13c50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f100801f040 tx=0x7f100800b400 comp rx=0 tx=0).stop 2024-02-02T15:26:50.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.460+0000 7f10320b6b80 1 -- 172.21.15.1:0/2287737728 shutdown_connections 2024-02-02T15:26:50.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.460+0000 7f10320b6b80 1 --2- 172.21.15.1:0/2287737728 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x561a78b1f450 0x561a78b21840 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:50.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.460+0000 7f10320b6b80 1 --2- 172.21.15.1:0/2287737728 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a78b0b980 0x561a78b1b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:50.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.460+0000 7f10320b6b80 1 --2- 172.21.15.1:0/2287737728 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561a78b0a180 0x561a78b13c50 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:50.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.460+0000 7f10320b6b80 1 --2- 172.21.15.1:0/2287737728 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561a78b0aae0 0x561a78b14190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:50.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.460+0000 7f1005600700 0 client.25577 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:26:50.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.460+0000 7f10320b6b80 1 -- 172.21.15.1:0/2287737728 >> 172.21.15.1:0/2287737728 conn(0x561a78a68db0 msgr2=0x561a78b045c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:50.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.460+0000 7f10320b6b80 1 -- 172.21.15.1:0/2287737728 shutdown_connections 2024-02-02T15:26:50.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.461+0000 7f10320b6b80 1 -- 172.21.15.1:0/2287737728 wait complete. 2024-02-02T15:26:50.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.461+0000 7f10320b6b80 20 client.25577 trim_cache size 0 max 16384 2024-02-02T15:26:50.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.477+0000 7f10320b6b80 1 Processor -- start 2024-02-02T15:26:50.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.478+0000 7f10320b6b80 1 -- start start 2024-02-02T15:26:50.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.478+0000 7f10320b6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a78a68db0 0x561a78b19150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:50.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.478+0000 7f10320b6b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561a78b0a180 0x561a78b1ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:50.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.478+0000 7f10320b6b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561a78b0aae0 0x561a78b0e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:50.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.478+0000 7f10320b6b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a78b13820 con 0x561a78b0aae0 2024-02-02T15:26:50.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.478+0000 7f10320b6b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a788de6a0 con 0x561a78b0a180 2024-02-02T15:26:50.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.478+0000 7f10320b6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a78b14e70 con 0x561a78a68db0 2024-02-02T15:26:50.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.478+0000 7fb445200700 1 -- 172.21.15.1:0/3616006573 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fb44010c770 con 0x7fb440100d70 2024-02-02T15:26:50.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.478+0000 7f1017e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561a78b0aae0 0x561a78b0e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:50.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.478+0000 7f1017e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561a78b0aae0 0x561a78b0e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53674/0 (socket says 172.21.15.1:53674) 2024-02-02T15:26:50.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.478+0000 7f1017e00700 1 -- 172.21.15.1:0/3889976135 learned_addr learned my addr 172.21.15.1:0/3889976135 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:50.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.478+0000 7f1016a00700 1 --2- 172.21.15.1:0/3889976135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561a78b0a180 0x561a78b1ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:50.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.478+0000 7f1017400700 1 --2- 172.21.15.1:0/3889976135 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a78a68db0 0x561a78b19150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:50.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.479+0000 7f1017e00700 1 -- 172.21.15.1:0/3889976135 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a78a68db0 msgr2=0x561a78b19150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:50.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.479+0000 7f1017e00700 1 --2- 172.21.15.1:0/3889976135 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a78a68db0 0x561a78b19150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:50.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.479+0000 7f1017e00700 1 -- 172.21.15.1:0/3889976135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561a78b0a180 msgr2=0x561a78b1ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:50.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.479+0000 7f1017e00700 1 --2- 172.21.15.1:0/3889976135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561a78b0a180 0x561a78b1ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:50.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.479+0000 7f1017e00700 1 -- 172.21.15.1:0/3889976135 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561a78a20ea0 con 0x561a78b0aae0 2024-02-02T15:26:50.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.479+0000 7f1017e00700 1 --2- 172.21.15.1:0/3889976135 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561a78b0aae0 0x561a78b0e930 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f101000eb70 tx=0x7f1010009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=844e2f57c0d5c90c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:50.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.479+0000 7f100d600700 1 -- 172.21.15.1:0/3889976135 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1010017820 con 0x561a78b0aae0 2024-02-02T15:26:50.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.479+0000 7f100d600700 1 -- 172.21.15.1:0/3889976135 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f101000f040 con 0x561a78b0aae0 2024-02-02T15:26:50.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.479+0000 7f100d600700 1 -- 172.21.15.1:0/3889976135 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1010007dd0 con 0x561a78b0aae0 2024-02-02T15:26:50.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.480+0000 7f10320b6b80 1 -- 172.21.15.1:0/3889976135 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561a78b0aae0 msgr2=0x561a78b0e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:50.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.480+0000 7f10320b6b80 1 --2- 172.21.15.1:0/3889976135 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561a78b0aae0 0x561a78b0e930 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f101000eb70 tx=0x7f1010009920 comp rx=0 tx=0).stop 2024-02-02T15:26:50.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.480+0000 7f10320b6b80 1 -- 172.21.15.1:0/3889976135 shutdown_connections 2024-02-02T15:26:50.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.480+0000 7f10320b6b80 1 --2- 172.21.15.1:0/3889976135 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561a78b0aae0 0x561a78b0e930 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:50.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.480+0000 7f10320b6b80 1 --2- 172.21.15.1:0/3889976135 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a78a68db0 0x561a78b19150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:50.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.480+0000 7f10320b6b80 1 --2- 172.21.15.1:0/3889976135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561a78b0a180 0x561a78b1ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:50.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.480+0000 7f10320b6b80 1 -- 172.21.15.1:0/3889976135 >> 172.21.15.1:0/3889976135 conn(0x561a78b05210 msgr2=0x561a78b00e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:50.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.480+0000 7f10320b6b80 1 -- 172.21.15.1:0/3889976135 shutdown_connections 2024-02-02T15:26:50.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.480+0000 7f10320b6b80 1 -- 172.21.15.1:0/3889976135 wait complete. 2024-02-02T15:26:50.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.480+0000 7f10320b6b80 1 -- start start 2024-02-02T15:26:50.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.481+0000 7f10320b6b80 1 Processor -- start 2024-02-02T15:26:50.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.481+0000 7f10320b6b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561a78a68db0 0x561a78b69140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:50.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.481+0000 7f10320b6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a78b0a180 0x561a78b69680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:50.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.481+0000 7f10320b6b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561a78b0aae0 0x561a78b70530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:50.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.481+0000 7f10320b6b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a78b70a70 con 0x561a78b0aae0 2024-02-02T15:26:50.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.481+0000 7f10320b6b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a78b70bb0 con 0x561a78a68db0 2024-02-02T15:26:50.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.481+0000 7f10320b6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a78b70d40 con 0x561a78b0a180 2024-02-02T15:26:50.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.481+0000 7f10320b6b80 10 client.0 _ll_register_callbacks cb 0x561a789f8290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:26:50.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.481+0000 7f1017e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561a78b0aae0 0x561a78b70530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:50.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.481+0000 7f1017400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561a78a68db0 0x561a78b69140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:50.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.481+0000 7f1017e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561a78b0aae0 0x561a78b70530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53684/0 (socket says 172.21.15.1:53684) 2024-02-02T15:26:50.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.481+0000 7f1017e00700 1 -- 172.21.15.1:0/1038491392 learned_addr learned my addr 172.21.15.1:0/1038491392 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:50.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.481+0000 7f1017400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561a78a68db0 0x561a78b69140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42468/0 (socket says 172.21.15.1:42468) 2024-02-02T15:26:50.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.482+0000 7f1016a00700 1 --2- 172.21.15.1:0/1038491392 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a78b0a180 0x561a78b69680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:50.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.482+0000 7f1017e00700 1 -- 172.21.15.1:0/1038491392 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a78b0a180 msgr2=0x561a78b69680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:50.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.482+0000 7f1017e00700 1 --2- 172.21.15.1:0/1038491392 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a78b0a180 0x561a78b69680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:50.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.482+0000 7f1017e00700 1 -- 172.21.15.1:0/1038491392 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561a78a68db0 msgr2=0x561a78b69140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:26:50.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.482+0000 7f1017e00700 1 --2- 172.21.15.1:0/1038491392 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561a78a68db0 0x561a78b69140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:50.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.482+0000 7f1017e00700 1 -- 172.21.15.1:0/1038491392 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f101000e820 con 0x561a78b0aae0 2024-02-02T15:26:50.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.482+0000 7f1016a00700 1 --2- 172.21.15.1:0/1038491392 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a78b0a180 0x561a78b69680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:26:50.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.482+0000 7f1017e00700 1 --2- 172.21.15.1:0/1038491392 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561a78b0aae0 0x561a78b70530 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1010009b00 tx=0x7f1010017a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e82ad3f306efa66 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:50.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.482+0000 7f1005600700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:26:50.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.482+0000 7f1005600700 1 -- 172.21.15.1:0/1038491392 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10100076f0 con 0x561a78b0aae0 2024-02-02T15:26:50.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.482+0000 7f1005600700 1 -- 172.21.15.1:0/1038491392 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f1010007500 con 0x561a78b0aae0 2024-02-02T15:26:50.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.482+0000 7f10320b6b80 1 -- 172.21.15.1:0/1038491392 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f100809a710 con 0x561a78b0aae0 2024-02-02T15:26:50.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.483+0000 7f1005600700 1 -- 172.21.15.1:0/1038491392 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f10100092e0 con 0x561a78b0aae0 2024-02-02T15:26:50.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.483+0000 7f10320b6b80 10 client.15555 fetch_fsmap learned FSMap version 27 2024-02-02T15:26:50.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.483+0000 7f10320b6b80 1 -- 172.21.15.1:0/1038491392 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x561a788e0cd0 con 0x561a78b0aae0 2024-02-02T15:26:50.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.483+0000 7f1005600700 1 -- 172.21.15.1:0/1038491392 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f100809a710 con 0x561a78b0aae0 2024-02-02T15:26:50.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.483+0000 7f1005600700 1 -- 172.21.15.1:0/1038491392 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1010005710 con 0x561a78b0aae0 2024-02-02T15:26:50.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.483+0000 7f10320b6b80 10 client.15555 fetch_fsmap finished waiting for FSMap version 27 2024-02-02T15:26:50.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.483+0000 7f10320b6b80 10 client.15555 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:26:50.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.483+0000 7f10320b6b80 20 client.15555 populate_metadata read hostname 'smithi001' 2024-02-02T15:26:50.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.483+0000 7f10320b6b80 1 --2- 172.21.15.1:0/1038491392 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x561a78b0b980 0x561a78b74740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:50.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.483+0000 7f10320b6b80 4 client.15555 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2024-02-02T15:26:50.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.483+0000 7f10320b6b80 1 -- 172.21.15.1:0/1038491392 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561a78a19f50 con 0x561a78b0b980 2024-02-02T15:26:50.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.484+0000 7f1017400700 1 --2- 172.21.15.1:0/1038491392 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x561a78b0b980 0x561a78b74740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:50.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.484+0000 7f1017400700 1 --2- 172.21.15.1:0/1038491392 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x561a78b0b980 0x561a78b74740 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26c7ffa98da44e93 server_cookie=58a0f60afc7021b0 in_seq=0 out_seq=0 2024-02-02T15:26:50.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.484+0000 7f1005600700 10 client.15555 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:26:50.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.484+0000 7f1005600700 1 -- 172.21.15.1:0/1038491392 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x561a78a19f50 con 0x561a78b0b980 2024-02-02T15:26:50.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.484+0000 7f1005600700 10 client.15555 handle_command_reply: tid=0 2024-02-02T15:26:50.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.484+0000 7f10320b6b80 1 client.15555 shutdown 2024-02-02T15:26:50.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.484+0000 7f10320b6b80 1 -- 172.21.15.1:0/1038491392 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561a78b0aae0 msgr2=0x561a78b70530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:50.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.484+0000 7f10320b6b80 1 --2- 172.21.15.1:0/1038491392 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561a78b0aae0 0x561a78b70530 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1010009b00 tx=0x7f1010017a50 comp rx=0 tx=0).stop 2024-02-02T15:26:50.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.484+0000 7f10320b6b80 1 -- 172.21.15.1:0/1038491392 shutdown_connections 2024-02-02T15:26:50.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.484+0000 7f10320b6b80 1 --2- 172.21.15.1:0/1038491392 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x561a78b0b980 0x561a78b74740 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:50.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.484+0000 7f10320b6b80 1 --2- 172.21.15.1:0/1038491392 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561a78b0aae0 0x561a78b70530 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:50.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.484+0000 7f10320b6b80 1 --2- 172.21.15.1:0/1038491392 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561a78a68db0 0x561a78b69140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:50.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.484+0000 7f10320b6b80 1 --2- 172.21.15.1:0/1038491392 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561a78b0a180 0x561a78b69680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:50.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.484+0000 7f1005600700 0 client.15555 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:26:50.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.485+0000 7f10320b6b80 1 -- 172.21.15.1:0/1038491392 >> 172.21.15.1:0/1038491392 conn(0x561a78b05210 msgr2=0x561a78b1ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:50.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.485+0000 7f10320b6b80 1 -- 172.21.15.1:0/1038491392 shutdown_connections 2024-02-02T15:26:50.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.485+0000 7f10320b6b80 1 -- 172.21.15.1:0/1038491392 wait complete. 2024-02-02T15:26:50.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.485+0000 7f10320b6b80 20 client.15555 trim_cache size 0 max 16384 2024-02-02T15:26:50.488 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:26:50.489 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-02T15:26:50.489 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2024-02-02T15:26:50.489 INFO:teuthology.orchestra.run.smithi001.stdout: "f6829dab-4e53-49fb-af6f-f5a676339bda": { 2024-02-02T15:26:50.489 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2024-02-02T15:26:50.489 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "f6829dab-4e53-49fb-af6f-f5a676339bda", 2024-02-02T15:26:50.489 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,force" 2024-02-02T15:26:50.489 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:26:50.489 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:26:50.489 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:26:50.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.485+0000 7f101e600700 1 -- 172.21.15.1:0/33652797 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f0ff80a05a0 msgr2=0x7f0ff80a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:50.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.485+0000 7f101e600700 1 --2- 172.21.15.1:0/33652797 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f0ff80a05a0 0x7f0ff80a2a60 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1000005950 tx=0x7f1000009310 comp rx=0 tx=0).stop 2024-02-02T15:26:50.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.485+0000 7f101e600700 1 -- 172.21.15.1:0/33652797 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f10180737a0 msgr2=0x7f1018198d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:50.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.485+0000 7f101e600700 1 --2- 172.21.15.1:0/33652797 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f10180737a0 0x7f1018198d70 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1008009230 tx=0x7f100800b790 comp rx=0 tx=0).stop 2024-02-02T15:26:50.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.485+0000 7f101e600700 1 -- 172.21.15.1:0/33652797 shutdown_connections 2024-02-02T15:26:50.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.485+0000 7f101e600700 1 --2- 172.21.15.1:0/33652797 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1018108ce0 0x7f10181a0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:50.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.485+0000 7f101e600700 1 --2- 172.21.15.1:0/33652797 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f10180737a0 0x7f1018198d70 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:50.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.485+0000 7f101e600700 1 --2- 172.21.15.1:0/33652797 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f0ff80a05a0 0x7f0ff80a2a60 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:50.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.485+0000 7f101e600700 1 --2- 172.21.15.1:0/33652797 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1018074d80 0x7f10181992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:50.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.485+0000 7f101e600700 1 -- 172.21.15.1:0/33652797 >> 172.21.15.1:0/33652797 conn(0x7f10180fc390 msgr2=0x7f10180fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:50.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.486+0000 7f101e600700 1 -- 172.21.15.1:0/33652797 shutdown_connections 2024-02-02T15:26:50.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:50.486+0000 7f101e600700 1 -- 172.21.15.1:0/33652797 wait complete. 2024-02-02T15:26:50.499 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:50 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/1724999362' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:26:50.499 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:50 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/2892118238' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-02T15:26:50.499 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:50 smithi001 ceph-mon[35470]: pgmap v300: 129 pgs: 129 active+clean; 648 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 767 B/s wr, 10 op/s 2024-02-02T15:26:50.501 INFO:tasks.cephfs.filesystem:scrub status for tag:f6829dab-4e53-49fb-af6f-f5a676339bda - {'path': '/', 'tag': 'f6829dab-4e53-49fb-af6f-f5a676339bda', 'options': 'recursive,force'} 2024-02-02T15:26:50.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:50 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/1724999362' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:26:50.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:50 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/2892118238' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-02T15:26:50.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:50 smithi071 ceph-mon[44707]: pgmap v300: 129 pgs: 129 active+clean; 648 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 767 B/s wr, 10 op/s 2024-02-02T15:26:50.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:50 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/1724999362' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:26:50.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:50 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/2892118238' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-02T15:26:50.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:50 smithi142 ceph-mon[45728]: pgmap v300: 129 pgs: 129 active+clean; 648 KiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 767 B/s wr, 10 op/s 2024-02-02T15:26:51.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.290+0000 7fb436a00700 1 -- 172.21.15.1:0/3616006573 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mgrmap(e 19) v1 ==== 135086+0+0 (secure 0 0 0) 0x7fb43000f800 con 0x7fb440100d70 2024-02-02T15:26:51.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.292+0000 7fb436a00700 1 -- 172.21.15.1:0/3616006573 <== mon.0 v2:172.21.15.1:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) v1 ==== 92+0+0 (secure 0 0 0) 0x7fb4300950d0 con 0x7fb440100d70 2024-02-02T15:26:51.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.295+0000 7fb445200700 1 -- 172.21.15.1:0/3616006573 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fb4200a0520 msgr2=0x7fb4200a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:51.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.295+0000 7fb445200700 1 --2- 172.21.15.1:0/3616006573 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fb4200a0520 0x7fb4200a29e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb42800b580 tx=0x7fb428005fb0 comp rx=0 tx=0).stop 2024-02-02T15:26:51.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.295+0000 7fb445200700 1 -- 172.21.15.1:0/3616006573 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb440100d70 msgr2=0x7fb440196a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:51.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.295+0000 7fb445200700 1 --2- 172.21.15.1:0/3616006573 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb440100d70 0x7fb440196a20 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb43000d980 tx=0x7fb43000dc90 comp rx=0 tx=0).stop 2024-02-02T15:26:51.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.295+0000 7fb445200700 1 -- 172.21.15.1:0/3616006573 shutdown_connections 2024-02-02T15:26:51.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.295+0000 7fb445200700 1 --2- 172.21.15.1:0/3616006573 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb440102900 0x7fb44019dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:51.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.295+0000 7fb445200700 1 --2- 172.21.15.1:0/3616006573 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb440100d70 0x7fb440196a20 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:51.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.295+0000 7fb445200700 1 --2- 172.21.15.1:0/3616006573 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fb4200a0520 0x7fb4200a29e0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:51.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.295+0000 7fb445200700 1 --2- 172.21.15.1:0/3616006573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb440101f60 0x7fb440196f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:51.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.295+0000 7fb445200700 1 -- 172.21.15.1:0/3616006573 >> 172.21.15.1:0/3616006573 conn(0x7fb4400fc2c0 msgr2=0x7fb4400fe530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:51.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.295+0000 7fb445200700 1 -- 172.21.15.1:0/3616006573 shutdown_connections 2024-02-02T15:26:51.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.296+0000 7fb445200700 1 -- 172.21.15.1:0/3616006573 wait complete. 2024-02-02T15:26:51.317 DEBUG:teuthology.orchestra.run.smithi001:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-02-02T15:26:51.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.523+0000 7f91af400700 1 -- 172.21.15.1:0/1334223739 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f91a80ff650 msgr2=0x7f91a80ffa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:51.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.523+0000 7f91af400700 1 --2- 172.21.15.1:0/1334223739 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f91a80ff650 0x7f91a80ffa70 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9198009990 tx=0x7f9198009ca0 comp rx=0 tx=0).stop 2024-02-02T15:26:51.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.523+0000 7f91af400700 1 -- 172.21.15.1:0/1334223739 shutdown_connections 2024-02-02T15:26:51.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.523+0000 7f91af400700 1 --2- 172.21.15.1:0/1334223739 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f91a8100970 0x7f91a8100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:51.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.523+0000 7f91af400700 1 --2- 172.21.15.1:0/1334223739 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f91a80ff650 0x7f91a80ffa70 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:51.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.523+0000 7f91af400700 1 --2- 172.21.15.1:0/1334223739 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91a80fffb0 0x7f91a8100430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:51.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.523+0000 7f91af400700 1 -- 172.21.15.1:0/1334223739 >> 172.21.15.1:0/1334223739 conn(0x7f91a80fb1d0 msgr2=0x7f91a80fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:51.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.524+0000 7f91af400700 1 -- 172.21.15.1:0/1334223739 shutdown_connections 2024-02-02T15:26:51.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.524+0000 7f91af400700 1 -- 172.21.15.1:0/1334223739 wait complete. 2024-02-02T15:26:51.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.525+0000 7f91af400700 1 Processor -- start 2024-02-02T15:26:51.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.525+0000 7f91af400700 1 -- start start 2024-02-02T15:26:51.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.525+0000 7f91af400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f91a80ff650 0x7f91a8198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:51.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.525+0000 7f91af400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91a80fffb0 0x7f91a81991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:51.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.525+0000 7f91af400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f91a8100970 0x7f91a81a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:51.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.525+0000 7f91af400700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91a81a07c0 con 0x7f91a80ff650 2024-02-02T15:26:51.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.525+0000 7f91af400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91a81a0930 con 0x7f91a8100970 2024-02-02T15:26:51.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.525+0000 7f91af400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91a81a0af0 con 0x7f91a80fffb0 2024-02-02T15:26:51.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.525+0000 7f91aca00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f91a80ff650 0x7f91a8198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:51.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.525+0000 7f91aca00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f91a80ff650 0x7f91a8198c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53740/0 (socket says 172.21.15.1:53740) 2024-02-02T15:26:51.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.525+0000 7f91aca00700 1 -- 172.21.15.1:0/1831650909 learned_addr learned my addr 172.21.15.1:0/1831650909 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:51.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.525+0000 7f91a7e00700 1 --2- 172.21.15.1:0/1831650909 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91a80fffb0 0x7f91a81991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:51.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.525+0000 7f91ad400700 1 --2- 172.21.15.1:0/1831650909 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f91a8100970 0x7f91a81a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:51.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.526+0000 7f91a7e00700 1 -- 172.21.15.1:0/1831650909 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f91a8100970 msgr2=0x7f91a81a01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:51.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.526+0000 7f91a7e00700 1 --2- 172.21.15.1:0/1831650909 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f91a8100970 0x7f91a81a01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:51.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.526+0000 7f91a7e00700 1 -- 172.21.15.1:0/1831650909 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f91a80ff650 msgr2=0x7f91a8198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:51.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.526+0000 7f91a7e00700 1 --2- 172.21.15.1:0/1831650909 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f91a80ff650 0x7f91a8198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:51.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.526+0000 7f91a7e00700 1 -- 172.21.15.1:0/1831650909 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91980095f0 con 0x7f91a80fffb0 2024-02-02T15:26:51.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.526+0000 7f91aca00700 1 --2- 172.21.15.1:0/1831650909 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f91a80ff650 0x7f91a8198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:26:51.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.526+0000 7f91a7e00700 1 --2- 172.21.15.1:0/1831650909 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91a80fffb0 0x7f91a81991c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f919000e9f0 tx=0x7f919000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:51.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.526+0000 7f91a5600700 1 -- 172.21.15.1:0/1831650909 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f919000c9c0 con 0x7f91a80fffb0 2024-02-02T15:26:51.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.526+0000 7f91af400700 1 -- 172.21.15.1:0/1831650909 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91a81a0dd0 con 0x7f91a80fffb0 2024-02-02T15:26:51.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.526+0000 7f91a5600700 1 -- 172.21.15.1:0/1831650909 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9190010430 con 0x7f91a80fffb0 2024-02-02T15:26:51.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.526+0000 7f91a5600700 1 -- 172.21.15.1:0/1831650909 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9190018680 con 0x7f91a80fffb0 2024-02-02T15:26:51.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.527+0000 7f91af400700 1 -- 172.21.15.1:0/1831650909 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91a81a1200 con 0x7f91a80fffb0 2024-02-02T15:26:51.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.528+0000 7f91af400700 1 -- 172.21.15.1:0/1831650909 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91a810a430 con 0x7f91a80fffb0 2024-02-02T15:26:51.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.530+0000 7f91a5600700 1 -- 172.21.15.1:0/1831650909 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 19) v1 ==== 135086+0+0 (secure 0 0 0) 0x7f9190010860 con 0x7f91a80fffb0 2024-02-02T15:26:51.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.531+0000 7f91a5600700 1 --2- 172.21.15.1:0/1831650909 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f91880a0570 0x7f91880a2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:51.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.531+0000 7f91aca00700 1 -- 172.21.15.1:0/1831650909 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f91880a0570 msgr2=0x7f91880a2a30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/3768989981 2024-02-02T15:26:51.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.531+0000 7f91a5600700 1 -- 172.21.15.1:0/1831650909 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f9190014070 con 0x7f91a80fffb0 2024-02-02T15:26:51.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.531+0000 7f91aca00700 1 --2- 172.21.15.1:0/1831650909 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f91880a0570 0x7f91880a2a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-02T15:26:51.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.535+0000 7f91a5600700 1 -- 172.21.15.1:0/1831650909 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f9190052110 con 0x7f91a80fffb0 2024-02-02T15:26:51.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:51 smithi071 ceph-mon[44707]: scrub queued for path: / 2024-02-02T15:26:51.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:51 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:26:51.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:51 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:26:51.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:51 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:26:51.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:51 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/3616006573' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-02-02T15:26:51.615 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Feb 02 15:26:51 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-y[46187]: ignoring --setuser ceph since I am not root 2024-02-02T15:26:51.615 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Feb 02 15:26:51 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-y[46187]: ignoring --setgroup ceph since I am not root 2024-02-02T15:26:51.615 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Feb 02 15:26:51 smithi071 ceph-mgr[46210]: -- 172.21.15.71:0/1999112229 <== mon.1 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b67c4e11e0 con 0x55b67c4b6c00 2024-02-02T15:26:51.670 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Feb 02 15:26:51 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-z[47590]: ignoring --setuser ceph since I am not root 2024-02-02T15:26:51.670 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Feb 02 15:26:51 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-z[47590]: ignoring --setgroup ceph since I am not root 2024-02-02T15:26:51.670 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Feb 02 15:26:51 smithi142 ceph-mgr[47613]: -- 172.21.15.142:0/3986800262 <== mon.1 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555e7ac651e0 con 0x555e7ac3a800 2024-02-02T15:26:51.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:51 smithi142 ceph-mon[45728]: scrub queued for path: / 2024-02-02T15:26:51.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:51 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:26:51.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:51 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:26:51.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:51 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:26:51.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:51 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/3616006573' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-02-02T15:26:51.690 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:26:51 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: ignoring --setuser ceph since I am not root 2024-02-02T15:26:51.690 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:26:51 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: ignoring --setgroup ceph since I am not root 2024-02-02T15:26:51.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:51 smithi001 ceph-mon[35470]: scrub queued for path: / 2024-02-02T15:26:51.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:51 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:26:51.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:51 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:26:51.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:51 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:26:51.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:51 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/3616006573' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-02-02T15:26:51.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.732+0000 7f91aca00700 1 -- 172.21.15.1:0/1831650909 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f91880a0570 msgr2=0x7f91880a2a30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/3768989981 2024-02-02T15:26:51.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.732+0000 7f91aca00700 1 --2- 172.21.15.1:0/1831650909 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f91880a0570 0x7f91880a2a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-02T15:26:51.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.753+0000 7f91af400700 1 -- 172.21.15.1:0/1831650909 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f91a802cc70 con 0x7f91a80fffb0 2024-02-02T15:26:51.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.758+0000 7f91a5600700 1 -- 172.21.15.1:0/1831650909 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v23)=0 v23) v1 ==== 138+0+0 (secure 0 0 0) 0x7f91900956b0 con 0x7f91a80fffb0 2024-02-02T15:26:51.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.761+0000 7f91af400700 1 -- 172.21.15.1:0/1831650909 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f91880a0570 msgr2=0x7f91880a2a30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-02T15:26:51.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.761+0000 7f91af400700 1 --2- 172.21.15.1:0/1831650909 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f91880a0570 0x7f91880a2a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:51.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.761+0000 7f91af400700 1 -- 172.21.15.1:0/1831650909 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91a80fffb0 msgr2=0x7f91a81991c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:51.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.761+0000 7f91af400700 1 --2- 172.21.15.1:0/1831650909 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91a80fffb0 0x7f91a81991c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f919000e9f0 tx=0x7f919000ed00 comp rx=0 tx=0).stop 2024-02-02T15:26:51.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.761+0000 7f91af400700 1 -- 172.21.15.1:0/1831650909 shutdown_connections 2024-02-02T15:26:51.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.761+0000 7f91af400700 1 --2- 172.21.15.1:0/1831650909 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f91a8100970 0x7f91a81a01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:51.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.761+0000 7f91af400700 1 --2- 172.21.15.1:0/1831650909 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f91a80ff650 0x7f91a8198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:51.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.761+0000 7f91af400700 1 --2- 172.21.15.1:0/1831650909 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f91880a0570 0x7f91880a2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:51.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.761+0000 7f91af400700 1 --2- 172.21.15.1:0/1831650909 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91a80fffb0 0x7f91a81991c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:51.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.761+0000 7f91af400700 1 -- 172.21.15.1:0/1831650909 >> 172.21.15.1:0/1831650909 conn(0x7f91a80fb1d0 msgr2=0x7f91a80fced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:51.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.761+0000 7f91af400700 1 -- 172.21.15.1:0/1831650909 shutdown_connections 2024-02-02T15:26:51.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.761+0000 7f91af400700 1 -- 172.21.15.1:0/1831650909 wait complete. 2024-02-02T15:26:51.778 DEBUG:teuthology.orchestra.run.smithi001:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-02-02T15:26:51.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.980+0000 7f6a4d000700 1 -- 172.21.15.1:0/432386029 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a480737a0 msgr2=0x7f6a48073c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:51.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.980+0000 7f6a4d000700 1 --2- 172.21.15.1:0/432386029 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a480737a0 0x7f6a48073c20 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6a30009990 tx=0x7f6a30009ca0 comp rx=0 tx=0).stop 2024-02-02T15:26:51.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.980+0000 7f6a4d000700 1 -- 172.21.15.1:0/432386029 shutdown_connections 2024-02-02T15:26:51.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.980+0000 7f6a4d000700 1 --2- 172.21.15.1:0/432386029 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a48108c30 0x7f6a4810b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:51.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.980+0000 7f6a4d000700 1 --2- 172.21.15.1:0/432386029 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a48074d80 0x7f6a480731d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:51.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.980+0000 7f6a4d000700 1 --2- 172.21.15.1:0/432386029 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a480737a0 0x7f6a48073c20 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:51.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.980+0000 7f6a4d000700 1 -- 172.21.15.1:0/432386029 >> 172.21.15.1:0/432386029 conn(0x7f6a480fc2c0 msgr2=0x7f6a480fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:51.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.980+0000 7f6a4d000700 1 -- 172.21.15.1:0/432386029 shutdown_connections 2024-02-02T15:26:51.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.981+0000 7f6a4d000700 1 -- 172.21.15.1:0/432386029 wait complete. 2024-02-02T15:26:51.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.981+0000 7f6a4d000700 1 Processor -- start 2024-02-02T15:26:51.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.982+0000 7f6a4d000700 1 -- start start 2024-02-02T15:26:51.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.982+0000 7f6a4d000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a480737a0 0x7f6a48198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:51.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.982+0000 7f6a4d000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a48074d80 0x7f6a481991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:51.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.982+0000 7f6a4d000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a48108c30 0x7f6a481a0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:51.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.982+0000 7f6a4d000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a481a07f0 con 0x7f6a48108c30 2024-02-02T15:26:51.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.982+0000 7f6a4d000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a481a0960 con 0x7f6a480737a0 2024-02-02T15:26:51.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.982+0000 7f6a4d000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a481a0b20 con 0x7f6a48074d80 2024-02-02T15:26:51.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.982+0000 7f6a45400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a48074d80 0x7f6a481991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:51.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.982+0000 7f6a45e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a480737a0 0x7f6a48198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:51.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.982+0000 7f6a45400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a48074d80 0x7f6a481991c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:41758/0 (socket says 172.21.15.1:41758) 2024-02-02T15:26:51.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.982+0000 7f6a45e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a480737a0 0x7f6a48198c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42524/0 (socket says 172.21.15.1:42524) 2024-02-02T15:26:51.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.982+0000 7f6a45400700 1 -- 172.21.15.1:0/1000595794 learned_addr learned my addr 172.21.15.1:0/1000595794 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:51.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.982+0000 7f6a46800700 1 --2- 172.21.15.1:0/1000595794 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a48108c30 0x7f6a481a0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:51.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.983+0000 7f6a45400700 1 -- 172.21.15.1:0/1000595794 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a480737a0 msgr2=0x7f6a48198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:51.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.983+0000 7f6a45400700 1 --2- 172.21.15.1:0/1000595794 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a480737a0 0x7f6a48198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:51.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.983+0000 7f6a45400700 1 -- 172.21.15.1:0/1000595794 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a48108c30 msgr2=0x7f6a481a0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:51.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.983+0000 7f6a45400700 1 --2- 172.21.15.1:0/1000595794 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a48108c30 0x7f6a481a0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:51.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.983+0000 7f6a45400700 1 -- 172.21.15.1:0/1000595794 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a300095f0 con 0x7f6a48074d80 2024-02-02T15:26:51.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.983+0000 7f6a45400700 1 --2- 172.21.15.1:0/1000595794 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a48074d80 0x7f6a481991c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6a30004710 tx=0x7f6a30009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:51.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.983+0000 7f6a3ea00700 1 -- 172.21.15.1:0/1000595794 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a3001d070 con 0x7f6a48074d80 2024-02-02T15:26:51.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.983+0000 7f6a3ea00700 1 -- 172.21.15.1:0/1000595794 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a30021410 con 0x7f6a48074d80 2024-02-02T15:26:51.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.983+0000 7f6a3ea00700 1 -- 172.21.15.1:0/1000595794 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a30017650 con 0x7f6a48074d80 2024-02-02T15:26:51.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.983+0000 7f6a4d000700 1 -- 172.21.15.1:0/1000595794 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a481a0da0 con 0x7f6a48074d80 2024-02-02T15:26:51.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.984+0000 7f6a4d000700 1 -- 172.21.15.1:0/1000595794 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a481a1170 con 0x7f6a48074d80 2024-02-02T15:26:51.985 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Feb 02 15:26:51 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-z[47590]: 2024-02-02T15:26:51.719+0000 7efc0707d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-02T15:26:51.985 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Feb 02 15:26:51 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-z[47590]: 2024-02-02T15:26:51.813+0000 7efc0707d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-02T15:26:51.985 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Feb 02 15:26:51 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-z[47590]: 2024-02-02T15:26:51.901+0000 7efc0707d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-02T15:26:51.985 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Feb 02 15:26:51 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-y[46187]: 2024-02-02T15:26:51.718+0000 7f9872778200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-02T15:26:51.986 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Feb 02 15:26:51 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-y[46187]: 2024-02-02T15:26:51.811+0000 7f9872778200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-02T15:26:51.986 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Feb 02 15:26:51 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-y[46187]: 2024-02-02T15:26:51.899+0000 7f9872778200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-02T15:26:51.986 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Feb 02 15:26:51 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-y[46187]: 2024-02-02T15:26:51.982+0000 7f9872778200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-02T15:26:51.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.985+0000 7f6a4d000700 1 -- 172.21.15.1:0/1000595794 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a481072e0 con 0x7f6a48074d80 2024-02-02T15:26:51.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.986+0000 7f6a3ea00700 1 -- 172.21.15.1:0/1000595794 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 19) v1 ==== 135086+0+0 (secure 0 0 0) 0x7f6a30021580 con 0x7f6a48074d80 2024-02-02T15:26:51.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.987+0000 7f6a3ea00700 1 --2- 172.21.15.1:0/1000595794 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f6a280a05f0 0x7f6a280a2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:51.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.988+0000 7f6a45e00700 1 -- 172.21.15.1:0/1000595794 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f6a280a05f0 msgr2=0x7f6a280a2ab0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/3768989981 2024-02-02T15:26:51.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.988+0000 7f6a3ea00700 1 -- 172.21.15.1:0/1000595794 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9107+0+0 (secure 0 0 0) 0x7f6a300cd120 con 0x7f6a48074d80 2024-02-02T15:26:51.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.988+0000 7f6a45e00700 1 --2- 172.21.15.1:0/1000595794 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f6a280a05f0 0x7f6a280a2ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-02T15:26:51.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:51.991+0000 7f6a3ea00700 1 -- 172.21.15.1:0/1000595794 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f6a30096980 con 0x7f6a48074d80 2024-02-02T15:26:52.042 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:26:51 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:26:51.915+0000 7f1a305df200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-02T15:26:52.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:52.189+0000 7f6a45e00700 1 -- 172.21.15.1:0/1000595794 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f6a280a05f0 msgr2=0x7f6a280a2ab0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/3768989981 2024-02-02T15:26:52.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:52.189+0000 7f6a45e00700 1 --2- 172.21.15.1:0/1000595794 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f6a280a05f0 0x7f6a280a2ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-02T15:26:52.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:52.206+0000 7f6a4d000700 1 -- 172.21.15.1:0/1000595794 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f6a4802cc70 con 0x7f6a48074d80 2024-02-02T15:26:52.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:52.212+0000 7f6a3ea00700 1 -- 172.21.15.1:0/1000595794 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v24)=0 v24) v1 ==== 133+0+0 (secure 0 0 0) 0x7f6a30099fa0 con 0x7f6a48074d80 2024-02-02T15:26:52.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:52.216+0000 7f6a4d000700 1 -- 172.21.15.1:0/1000595794 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f6a280a05f0 msgr2=0x7f6a280a2ab0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-02T15:26:52.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:52.216+0000 7f6a4d000700 1 --2- 172.21.15.1:0/1000595794 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f6a280a05f0 0x7f6a280a2ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:52.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:52.216+0000 7f6a4d000700 1 -- 172.21.15.1:0/1000595794 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a48074d80 msgr2=0x7f6a481991c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:52.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:52.216+0000 7f6a4d000700 1 --2- 172.21.15.1:0/1000595794 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a48074d80 0x7f6a481991c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6a30004710 tx=0x7f6a30009580 comp rx=0 tx=0).stop 2024-02-02T15:26:52.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:52.216+0000 7f6a4d000700 1 -- 172.21.15.1:0/1000595794 shutdown_connections 2024-02-02T15:26:52.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:52.216+0000 7f6a4d000700 1 --2- 172.21.15.1:0/1000595794 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a48108c30 0x7f6a481a0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:52.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:52.216+0000 7f6a4d000700 1 --2- 172.21.15.1:0/1000595794 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a480737a0 0x7f6a48198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:52.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:52.216+0000 7f6a4d000700 1 --2- 172.21.15.1:0/1000595794 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7f6a280a05f0 0x7f6a280a2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:52.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:52.216+0000 7f6a4d000700 1 --2- 172.21.15.1:0/1000595794 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a48074d80 0x7f6a481991c0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:52.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:52.216+0000 7f6a4d000700 1 -- 172.21.15.1:0/1000595794 >> 172.21.15.1:0/1000595794 conn(0x7f6a480fc2c0 msgr2=0x7f6a480fe6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:52.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:52.216+0000 7f6a4d000700 1 -- 172.21.15.1:0/1000595794 shutdown_connections 2024-02-02T15:26:52.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:52.216+0000 7f6a4d000700 1 -- 172.21.15.1:0/1000595794 wait complete. 2024-02-02T15:26:52.243 DEBUG:teuthology.orchestra.run.smithi001:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-02-02T15:26:52.293 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:26:52 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:26:52.043+0000 7f1a305df200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-02T15:26:52.294 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:26:52 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:26:52.161+0000 7f1a305df200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-02T15:26:52.294 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:26:52 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:26:52.288+0000 7f1a305df200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-02T15:26:52.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:52.443+0000 7fdf15e00700 1 -- 172.21.15.1:0/2817007015 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf100737a0 msgr2=0x7fdf10073c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:52.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:52.443+0000 7fdf15e00700 1 --2- 172.21.15.1:0/2817007015 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf100737a0 0x7fdf10073c20 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fdef80098f0 tx=0x7fdef8009c00 comp rx=0 tx=0).stop 2024-02-02T15:26:52.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:52.444+0000 7fdf15e00700 1 -- 172.21.15.1:0/2817007015 shutdown_connections 2024-02-02T15:26:52.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:52.444+0000 7fdf15e00700 1 --2- 172.21.15.1:0/2817007015 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf10108c30 0x7fdf1010b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:52.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:52.444+0000 7fdf15e00700 1 --2- 172.21.15.1:0/2817007015 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf10074d80 0x7fdf100731d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:52.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:52.444+0000 7fdf15e00700 1 --2- 172.21.15.1:0/2817007015 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf100737a0 0x7fdf10073c20 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:52.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:52.444+0000 7fdf15e00700 1 -- 172.21.15.1:0/2817007015 >> 172.21.15.1:0/2817007015 conn(0x7fdf100fc300 msgr2=0x7fdf100fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:26:52.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:52.444+0000 7fdf15e00700 1 -- 172.21.15.1:0/2817007015 shutdown_connections 2024-02-02T15:26:52.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:52.444+0000 7fdf15e00700 1 -- 172.21.15.1:0/2817007015 wait complete. 2024-02-02T15:26:52.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:52.445+0000 7fdf15e00700 1 Processor -- start 2024-02-02T15:26:52.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:52.446+0000 7fdf15e00700 1 -- start start 2024-02-02T15:26:52.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:52.446+0000 7fdf15e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf100737a0 0x7fdf10198d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:52.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:52.446+0000 7fdf15e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf10074d80 0x7fdf10199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:52.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:52.446+0000 7fdf15e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf10108c30 0x7fdf101a0290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:52.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:52.446+0000 7fdf15e00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf101a0860 con 0x7fdf100737a0 2024-02-02T15:26:52.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:52.446+0000 7fdf15e00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf101a09d0 con 0x7fdf10108c30 2024-02-02T15:26:52.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:52.446+0000 7fdf15e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf101a0b90 con 0x7fdf10074d80 2024-02-02T15:26:52.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:52.446+0000 7fdf0ea00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf100737a0 0x7fdf10198d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:52.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:52.446+0000 7fdf0ea00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf100737a0 0x7fdf10198d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53788/0 (socket says 172.21.15.1:53788) 2024-02-02T15:26:52.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:52.446+0000 7fdf0ea00700 1 -- 172.21.15.1:0/1848261795 learned_addr learned my addr 172.21.15.1:0/1848261795 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:26:52.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:52.446+0000 7fdf0e000700 1 --2- 172.21.15.1:0/1848261795 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf10074d80 0x7fdf10199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:52.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:52.446+0000 7fdf0f400700 1 --2- 172.21.15.1:0/1848261795 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf10108c30 0x7fdf101a0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:26:52.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:52.447+0000 7fdf0ea00700 1 -- 172.21.15.1:0/1848261795 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf10074d80 msgr2=0x7fdf10199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:52.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:52.447+0000 7fdf0ea00700 1 --2- 172.21.15.1:0/1848261795 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf10074d80 0x7fdf10199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:52.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:52.447+0000 7fdf0ea00700 1 -- 172.21.15.1:0/1848261795 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf10108c30 msgr2=0x7fdf101a0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:26:52.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:52.447+0000 7fdf0ea00700 1 --2- 172.21.15.1:0/1848261795 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf10108c30 0x7fdf101a0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:26:52.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:52.447+0000 7fdf0ea00700 1 -- 172.21.15.1:0/1848261795 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf000095f0 con 0x7fdf100737a0 2024-02-02T15:26:52.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:52.447+0000 7fdf0e000700 1 --2- 172.21.15.1:0/1848261795 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf10074d80 0x7fdf10199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:26:52.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:52.447+0000 7fdf0f400700 1 --2- 172.21.15.1:0/1848261795 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf10108c30 0x7fdf101a0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:26:52.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:52.447+0000 7fdf0ea00700 1 --2- 172.21.15.1:0/1848261795 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf100737a0 0x7fdf10198d20 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fdf0000ec10 tx=0x7fdf0000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:26:52.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:52.447+0000 7fdf14a00700 1 -- 172.21.15.1:0/1848261795 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf0000cc90 con 0x7fdf100737a0 2024-02-02T15:26:52.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:52.447+0000 7fdf14a00700 1 -- 172.21.15.1:0/1848261795 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf000044e0 con 0x7fdf100737a0 2024-02-02T15:26:52.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:52.447+0000 7fdf15e00700 1 -- 172.21.15.1:0/1848261795 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdef8009520 con 0x7fdf100737a0 2024-02-02T15:26:52.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:52.447+0000 7fdf14a00700 1 -- 172.21.15.1:0/1848261795 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf00010430 con 0x7fdf100737a0 2024-02-02T15:26:52.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:52.447+0000 7fdf15e00700 1 -- 172.21.15.1:0/1848261795 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf101a1190 con 0x7fdf100737a0 2024-02-02T15:26:52.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:52.448+0000 7fdf15e00700 1 -- 172.21.15.1:0/1848261795 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf101072e0 con 0x7fdf100737a0 2024-02-02T15:26:52.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:52.450+0000 7fdf14a00700 1 -- 172.21.15.1:0/1848261795 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 19) v1 ==== 135086+0+0 (secure 0 0 0) 0x7fdf00003f90 con 0x7fdf100737a0 2024-02-02T15:26:52.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:52.452+0000 7fdf14a00700 1 --2- 172.21.15.1:0/1848261795 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fdef00a05f0 0x7fdef00a2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:26:52.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:52.452+0000 7fdf0e000700 1 -- 172.21.15.1:0/1848261795 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fdef00a05f0 msgr2=0x7fdef00a2ab0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/3768989981 2024-02-02T15:26:52.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:52.452+0000 7fdf14a00700 1 -- 172.21.15.1:0/1848261795 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 9107+0+0 (secure 0 0 0) 0x7fdf00014070 con 0x7fdf100737a0 2024-02-02T15:26:52.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:52.452+0000 7fdf0e000700 1 --2- 172.21.15.1:0/1848261795 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fdef00a05f0 0x7fdef00a2ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-02-02T15:26:52.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:52.459+0000 7fdf14a00700 1 -- 172.21.15.1:0/1848261795 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7fdf00095bb0 con 0x7fdf100737a0 2024-02-02T15:26:52.588 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:52 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/3616006573' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-02-02T15:26:52.588 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:52 smithi071 ceph-mon[44707]: mgrmap e19: x(active, since 7m), standbys: z, y 2024-02-02T15:26:52.588 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:52 smithi071 ceph-mon[44707]: from='client.? ' entity='client.admin' 2024-02-02T15:26:52.588 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:52 smithi071 ceph-mon[44707]: from='client.? ' entity='client.admin' 2024-02-02T15:26:52.596 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Feb 02 15:26:51 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-z[47590]: 2024-02-02T15:26:51.985+0000 7efc0707d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-02T15:26:52.597 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:52 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/3616006573' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-02-02T15:26:52.597 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:52 smithi142 ceph-mon[45728]: mgrmap e19: x(active, since 7m), standbys: z, y 2024-02-02T15:26:52.597 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:52 smithi142 ceph-mon[45728]: from='client.? ' entity='client.admin' 2024-02-02T15:26:52.597 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:52 smithi142 ceph-mon[45728]: from='client.? ' entity='client.admin' 2024-02-02T15:26:52.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:52.653+0000 7fdf0e000700 1 -- 172.21.15.1:0/1848261795 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fdef00a05f0 msgr2=0x7fdef00a2ab0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/3768989981 2024-02-02T15:26:52.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:52.653+0000 7fdf0e000700 1 --2- 172.21.15.1:0/1848261795 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fdef00a05f0 0x7fdef00a2ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-02-02T15:26:52.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:52 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/3616006573' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-02-02T15:26:52.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:52 smithi001 ceph-mon[35470]: mgrmap e19: x(active, since 7m), standbys: z, y 2024-02-02T15:26:52.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:52 smithi001 ceph-mon[35470]: from='client.? ' entity='client.admin' 2024-02-02T15:26:52.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:52 smithi001 ceph-mon[35470]: from='client.? ' entity='client.admin' 2024-02-02T15:26:52.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:52.698+0000 7fdf15e00700 1 -- 172.21.15.1:0/1848261795 --> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fdf10061180 con 0x7fdef00a05f0 2024-02-02T15:26:52.864 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Feb 02 15:26:52 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-y[46187]: 2024-02-02T15:26:52.588+0000 7f9872778200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-02T15:26:52.864 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Feb 02 15:26:52 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-y[46187]: 2024-02-02T15:26:52.822+0000 7f9872778200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-02T15:26:52.917 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Feb 02 15:26:52 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-z[47590]: 2024-02-02T15:26:52.597+0000 7efc0707d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-02T15:26:52.918 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Feb 02 15:26:52 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-z[47590]: 2024-02-02T15:26:52.834+0000 7efc0707d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-02T15:26:53.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:53.054+0000 7fdf0e000700 1 -- 172.21.15.1:0/1848261795 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fdef00a05f0 msgr2=0x7fdef00a2ab0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/3768989981 2024-02-02T15:26:53.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:53.054+0000 7fdf0e000700 1 --2- 172.21.15.1:0/1848261795 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fdef00a05f0 0x7fdef00a2ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-02-02T15:26:53.169 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Feb 02 15:26:52 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-y[46187]: 2024-02-02T15:26:52.907+0000 7f9872778200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-02T15:26:53.169 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Feb 02 15:26:53 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-y[46187]: 2024-02-02T15:26:53.013+0000 7f9872778200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-02T15:26:53.170 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Feb 02 15:26:52 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-z[47590]: 2024-02-02T15:26:52.918+0000 7efc0707d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-02T15:26:53.170 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Feb 02 15:26:53 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-z[47590]: 2024-02-02T15:26:53.026+0000 7efc0707d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-02T15:26:53.435 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:26:53 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:26:53.131+0000 7f1a305df200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-02T15:26:53.614 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Feb 02 15:26:53 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-y[46187]: 2024-02-02T15:26:53.169+0000 7f9872778200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-02T15:26:53.614 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Feb 02 15:26:53 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-y[46187]: 2024-02-02T15:26:53.295+0000 7f9872778200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-02T15:26:53.614 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Feb 02 15:26:53 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-y[46187]: 2024-02-02T15:26:53.379+0000 7f9872778200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-02T15:26:53.670 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Feb 02 15:26:53 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-z[47590]: 2024-02-02T15:26:53.184+0000 7efc0707d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-02T15:26:53.670 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Feb 02 15:26:53 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-z[47590]: 2024-02-02T15:26:53.311+0000 7efc0707d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-02T15:26:53.671 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Feb 02 15:26:53 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-z[47590]: 2024-02-02T15:26:53.396+0000 7efc0707d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-02T15:26:53.690 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:26:53 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:26:53.436+0000 7f1a305df200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-02T15:26:53.690 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:26:53 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:26:53.550+0000 7f1a305df200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-02T15:26:53.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:53.855+0000 7fdf0e000700 1 -- 172.21.15.1:0/1848261795 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fdef00a05f0 msgr2=0x7fdef00a2ab0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/3768989981 2024-02-02T15:26:53.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:53.855+0000 7fdf0e000700 1 --2- 172.21.15.1:0/1848261795 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fdef00a05f0 0x7fdef00a2ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-02-02T15:26:54.009 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Feb 02 15:26:53 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-y[46187]: 2024-02-02T15:26:53.745+0000 7f9872778200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-02T15:26:54.029 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Feb 02 15:26:53 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-z[47590]: 2024-02-02T15:26:53.764+0000 7efc0707d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-02T15:26:54.068 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:26:53 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:26:53.692+0000 7f1a305df200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-02T15:26:54.068 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:26:53 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:26:53.894+0000 7f1a305df200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-02T15:26:54.289 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Feb 02 15:26:54 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-y[46187]: 2024-02-02T15:26:54.009+0000 7f9872778200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-02T15:26:54.313 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Feb 02 15:26:54 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-z[47590]: 2024-02-02T15:26:54.030+0000 7efc0707d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-02T15:26:54.440 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:26:54 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:26:54.069+0000 7f1a305df200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-02T15:26:54.440 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:26:54 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:26:54.184+0000 7f1a305df200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-02T15:26:54.614 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Feb 02 15:26:54 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-y[46187]: 2024-02-02T15:26:54.289+0000 7f9872778200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-02T15:26:54.614 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Feb 02 15:26:54 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-y[46187]: 2024-02-02T15:26:54.382+0000 7f9872778200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-02T15:26:54.670 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Feb 02 15:26:54 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-z[47590]: 2024-02-02T15:26:54.314+0000 7efc0707d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-02T15:26:54.670 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Feb 02 15:26:54 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-z[47590]: 2024-02-02T15:26:54.408+0000 7efc0707d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-02T15:26:54.940 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:26:54 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:26:54.665+0000 7f1a305df200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-02T15:26:55.382 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:26:55 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:26:55.018+0000 7f1a305df200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-02T15:26:55.419 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Feb 02 15:26:55 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-z[47590]: 2024-02-02T15:26:55.158+0000 7efc0707d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-02T15:26:55.420 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Feb 02 15:26:55 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-z[47590]: 2024-02-02T15:26:55.252+0000 7efc0707d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-02T15:26:55.420 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Feb 02 15:26:55 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-z[47590]: 2024-02-02T15:26:55.402+0000 7efc0707d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-02T15:26:55.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:55.455+0000 7fdf0e000700 1 -- 172.21.15.1:0/1848261795 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fdef00a05f0 msgr2=0x7fdef00a2ab0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/3768989981 2024-02-02T15:26:55.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:55.455+0000 7fdf0e000700 1 --2- 172.21.15.1:0/1848261795 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fdef00a05f0 0x7fdef00a2ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-02-02T15:26:55.614 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Feb 02 15:26:55 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-y[46187]: 2024-02-02T15:26:55.127+0000 7f9872778200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-02T15:26:55.614 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Feb 02 15:26:55 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-y[46187]: 2024-02-02T15:26:55.221+0000 7f9872778200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-02T15:26:55.614 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Feb 02 15:26:55 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-y[46187]: 2024-02-02T15:26:55.369+0000 7f9872778200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-02T15:26:55.690 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:26:55 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:26:55.382+0000 7f1a305df200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-02T15:26:55.690 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:26:55 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:26:55.510+0000 7f1a305df200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-02T15:26:56.359 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Feb 02 15:26:55 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-y[46187]: 2024-02-02T15:26:55.988+0000 7f9872778200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-02T15:26:56.359 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Feb 02 15:26:56 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-y[46187]: 2024-02-02T15:26:56.167+0000 7f9872778200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-02T15:26:56.359 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Feb 02 15:26:56 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-y[46187]: 2024-02-02T15:26:56.358+0000 7f9872778200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-02T15:26:56.402 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Feb 02 15:26:56 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-z[47590]: 2024-02-02T15:26:56.030+0000 7efc0707d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-02T15:26:56.402 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Feb 02 15:26:56 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-z[47590]: 2024-02-02T15:26:56.210+0000 7efc0707d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-02T15:26:56.669 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Feb 02 15:26:56 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-z[47590]: 2024-02-02T15:26:56.403+0000 7efc0707d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-02T15:26:56.818 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:26:56 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:26:56.491+0000 7f1a305df200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-02T15:26:56.818 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:26:56 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:26:56.620+0000 7f1a305df200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-02T15:26:57.534 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Feb 02 15:26:57 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-y[46187]: 2024-02-02T15:26:57.268+0000 7f9872778200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-02T15:26:57.587 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Feb 02 15:26:57 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-z[47590]: 2024-02-02T15:26:57.321+0000 7efc0707d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-02T15:26:57.810 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Feb 02 15:26:57 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-y[46187]: 2024-02-02T15:26:57.534+0000 7f9872778200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-02T15:26:57.810 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Feb 02 15:26:57 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-y[46187]: 2024-02-02T15:26:57.621+0000 7f9872778200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-02T15:26:57.810 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Feb 02 15:26:57 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-y[46187]: 2024-02-02T15:26:57.809+0000 7f9872778200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-02T15:26:57.863 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Feb 02 15:26:57 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-z[47590]: 2024-02-02T15:26:57.588+0000 7efc0707d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-02T15:26:57.863 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Feb 02 15:26:57 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-z[47590]: 2024-02-02T15:26:57.675+0000 7efc0707d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-02T15:26:57.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:57.884+0000 7fdf14a00700 1 -- 172.21.15.1:0/1848261795 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mgrmap(e 20) v1 ==== 135086+0+0 (secure 0 0 0) 0x7fdf00095f80 con 0x7fdf100737a0 2024-02-02T15:26:57.915 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:57 smithi001 ceph-mon[35470]: Standby manager daemon y restarted 2024-02-02T15:26:57.915 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:57 smithi001 ceph-mon[35470]: Standby manager daemon y started 2024-02-02T15:26:57.915 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:57 smithi001 ceph-mon[35470]: from='mgr.? 172.21.15.71:0/2356064772' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-02-02T15:26:57.915 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:57 smithi001 ceph-mon[35470]: from='mgr.? 172.21.15.71:0/2356064772' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-02T15:26:57.915 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:57 smithi001 ceph-mon[35470]: from='mgr.? 172.21.15.71:0/2356064772' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-02-02T15:26:57.915 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:57 smithi001 ceph-mon[35470]: from='mgr.? 172.21.15.71:0/2356064772' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-02T15:26:57.916 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:26:56 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:26:56.817+0000 7f1a305df200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-02T15:26:57.916 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:26:57 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:26:57.662+0000 7f1a305df200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-02T15:26:57.916 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:26:57 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:26:57.915+0000 7f1a305df200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-02T15:26:58.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:57 smithi071 ceph-mon[44707]: Standby manager daemon y restarted 2024-02-02T15:26:58.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:57 smithi071 ceph-mon[44707]: Standby manager daemon y started 2024-02-02T15:26:58.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:57 smithi071 ceph-mon[44707]: from='mgr.? 172.21.15.71:0/2356064772' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-02-02T15:26:58.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:57 smithi071 ceph-mon[44707]: from='mgr.? 172.21.15.71:0/2356064772' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-02T15:26:58.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:57 smithi071 ceph-mon[44707]: from='mgr.? 172.21.15.71:0/2356064772' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-02-02T15:26:58.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:57 smithi071 ceph-mon[44707]: from='mgr.? 172.21.15.71:0/2356064772' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-02T15:26:58.170 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Feb 02 15:26:57 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-z[47590]: 2024-02-02T15:26:57.864+0000 7efc0707d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-02T15:26:58.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:57 smithi142 ceph-mon[45728]: Standby manager daemon y restarted 2024-02-02T15:26:58.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:57 smithi142 ceph-mon[45728]: Standby manager daemon y started 2024-02-02T15:26:58.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:57 smithi142 ceph-mon[45728]: from='mgr.? 172.21.15.71:0/2356064772' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-02-02T15:26:58.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:57 smithi142 ceph-mon[45728]: from='mgr.? 172.21.15.71:0/2356064772' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-02T15:26:58.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:57 smithi142 ceph-mon[45728]: from='mgr.? 172.21.15.71:0/2356064772' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-02-02T15:26:58.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:57 smithi142 ceph-mon[45728]: from='mgr.? 172.21.15.71:0/2356064772' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-02T15:26:58.440 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:26:58 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:26:58.179+0000 7f1a305df200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-02T15:26:58.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:58.658+0000 7fdf0e000700 1 -- 172.21.15.1:0/1848261795 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fdef00a05f0 msgr2=0x7fdef00a2ab0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.1:6800/3768989981 2024-02-02T15:26:58.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:58.658+0000 7fdf0e000700 1 --2- 172.21.15.1:0/1848261795 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fdef00a05f0 0x7fdef00a2ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-02-02T15:26:58.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:26:58.889+0000 7fdf14a00700 1 -- 172.21.15.1:0/1848261795 <== mon.0 v2:172.21.15.1:3300/0 8 ==== mgrmap(e 21) v1 ==== 135086+0+0 (secure 0 0 0) 0x7fdf000c9b00 con 0x7fdf100737a0 2024-02-02T15:26:59.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:58 smithi142 ceph-mon[45728]: from='mgr.? 172.21.15.142:0/1802625356' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-02T15:26:59.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:58 smithi142 ceph-mon[45728]: Standby manager daemon z restarted 2024-02-02T15:26:59.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:58 smithi142 ceph-mon[45728]: Standby manager daemon z started 2024-02-02T15:26:59.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:58 smithi142 ceph-mon[45728]: mgrmap e20: x(active, since 7m), standbys: z, y 2024-02-02T15:26:59.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:58 smithi142 ceph-mon[45728]: from='mgr.? 172.21.15.142:0/1802625356' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-02T15:26:59.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:58 smithi142 ceph-mon[45728]: from='mgr.? 172.21.15.142:0/1802625356' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-02T15:26:59.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:58 smithi142 ceph-mon[45728]: from='mgr.? 172.21.15.142:0/1802625356' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-02T15:26:59.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:58 smithi001 ceph-mon[35470]: from='mgr.? 172.21.15.142:0/1802625356' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-02T15:26:59.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:58 smithi001 ceph-mon[35470]: Standby manager daemon z restarted 2024-02-02T15:26:59.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:58 smithi001 ceph-mon[35470]: Standby manager daemon z started 2024-02-02T15:26:59.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:58 smithi001 ceph-mon[35470]: mgrmap e20: x(active, since 7m), standbys: z, y 2024-02-02T15:26:59.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:58 smithi001 ceph-mon[35470]: from='mgr.? 172.21.15.142:0/1802625356' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-02T15:26:59.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:58 smithi001 ceph-mon[35470]: from='mgr.? 172.21.15.142:0/1802625356' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-02T15:26:59.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:58 smithi001 ceph-mon[35470]: from='mgr.? 172.21.15.142:0/1802625356' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-02T15:26:59.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:58 smithi071 ceph-mon[44707]: from='mgr.? 172.21.15.142:0/1802625356' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-02T15:26:59.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:58 smithi071 ceph-mon[44707]: Standby manager daemon z restarted 2024-02-02T15:26:59.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:58 smithi071 ceph-mon[44707]: Standby manager daemon z started 2024-02-02T15:26:59.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:58 smithi071 ceph-mon[44707]: mgrmap e20: x(active, since 7m), standbys: z, y 2024-02-02T15:26:59.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:58 smithi071 ceph-mon[44707]: from='mgr.? 172.21.15.142:0/1802625356' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-02T15:26:59.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:58 smithi071 ceph-mon[44707]: from='mgr.? 172.21.15.142:0/1802625356' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-02T15:26:59.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:58 smithi071 ceph-mon[44707]: from='mgr.? 172.21.15.142:0/1802625356' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-02T15:26:59.690 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:26:59 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:26:59.355+0000 7f1a305df200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-02T15:27:00.111 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:26:59 smithi001 ceph-mon[35470]: mgrmap e21: x(active, since 7m), standbys: y, z 2024-02-02T15:27:00.111 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:26:59 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:26:59.723+0000 7f1a305df200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-02T15:27:00.111 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:26:59 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:26:59.846+0000 7f1a305df200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-02T15:27:00.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:00.125+0000 7fdf14a00700 1 -- 172.21.15.1:0/1848261795 <== mon.0 v2:172.21.15.1:3300/0 9 ==== mgrmap(e 22) v1 ==== 134755+0+0 (secure 0 0 0) 0x7fdf00080110 con 0x7fdf100737a0 2024-02-02T15:27:00.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:00.126+0000 7fdf14a00700 1 -- 172.21.15.1:0/1848261795 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fdef00a05f0 msgr2=0x7fdef00a2ab0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-02T15:27:00.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:00.126+0000 7fdf14a00700 1 --2- 172.21.15.1:0/1848261795 >> [v2:172.21.15.1:6800/3768989981,v1:172.21.15.1:6801/3768989981] conn(0x7fdef00a05f0 0x7fdef00a2ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:00.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:26:59 smithi142 ceph-mon[45728]: mgrmap e21: x(active, since 7m), standbys: y, z 2024-02-02T15:27:00.293 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:26:59 smithi071 ceph-mon[44707]: mgrmap e21: x(active, since 7m), standbys: y, z 2024-02-02T15:27:00.440 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:27:00 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:27:00.111+0000 7f1a305df200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-02T15:27:00.440 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:27:00 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:27:00.342+0000 7f1986000700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-02T15:27:00.440 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:27:00 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:27:00.342+0000 7f1986000700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-02T15:27:00.440 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:27:00 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:27:00.342+0000 7f1986000700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-02T15:27:00.440 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:27:00 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:27:00.342+0000 7f1986000700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-02T15:27:00.440 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:27:00 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:27:00.342+0000 7f1986000700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-02T15:27:00.440 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:27:00 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:27:00.342+0000 7f1983800700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-02T15:27:00.440 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:27:00 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:27:00.342+0000 7f1983800700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-02T15:27:00.441 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:27:00 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:27:00.342+0000 7f1983800700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-02T15:27:00.441 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:27:00 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:27:00.342+0000 7f1983800700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-02T15:27:00.441 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:27:00 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:27:00.342+0000 7f1983800700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-02T15:27:01.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:01.133+0000 7fdf14a00700 1 -- 172.21.15.1:0/1848261795 <== mon.0 v2:172.21.15.1:3300/0 10 ==== mgrmap(e 23) v1 ==== 134882+0+0 (secure 0 0 0) 0x7fdf00096850 con 0x7fdf100737a0 2024-02-02T15:27:01.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:01.133+0000 7fdf14a00700 1 --2- 172.21.15.1:0/1848261795 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fdef00671d0 0x7fdef00675b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:01.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:01.133+0000 7fdf14a00700 1 -- 172.21.15.1:0/1848261795 --> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) v1 -- 0x7fdf10061180 con 0x7fdef00671d0 2024-02-02T15:27:01.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:01.141+0000 7fdf0e000700 1 --2- 172.21.15.1:0/1848261795 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fdef00671d0 0x7fdef00675b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:01.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:01.146+0000 7fdf0e000700 1 --2- 172.21.15.1:0/1848261795 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fdef00671d0 0x7fdef00675b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fdef8009fd0 tx=0x7fdef8005dc0 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:01.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:00 smithi142 ceph-mon[45728]: Active manager daemon x restarted 2024-02-02T15:27:01.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:00 smithi142 ceph-mon[45728]: Activating manager daemon x 2024-02-02T15:27:01.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:00 smithi142 ceph-mon[45728]: osdmap e89: 12 total, 12 up, 12 in 2024-02-02T15:27:01.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:00 smithi142 ceph-mon[45728]: mgrmap e22: x(active, starting, since 0.0124561s), standbys: y, z 2024-02-02T15:27:01.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:00 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-02T15:27:01.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:00 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-02T15:27:01.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:00 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-02T15:27:01.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:00 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-02-02T15:27:01.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:00 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-02-02T15:27:01.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:00 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-02-02T15:27:01.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:00 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-02-02T15:27:01.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:00 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-02-02T15:27:01.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:00 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-02-02T15:27:01.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:00 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-02-02T15:27:01.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:00 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-02-02T15:27:01.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:00 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-02-02T15:27:01.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:00 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-02T15:27:01.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:00 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-02T15:27:01.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:00 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-02-02T15:27:01.172 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:00 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-02T15:27:01.172 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:00 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-02T15:27:01.172 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:00 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-02T15:27:01.172 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:00 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-02T15:27:01.172 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:00 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-02T15:27:01.172 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:00 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-02T15:27:01.172 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:00 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-02T15:27:01.172 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:00 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-02T15:27:01.172 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:00 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-02T15:27:01.172 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:00 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-02T15:27:01.172 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:00 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-02T15:27:01.172 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:00 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-02T15:27:01.172 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:00 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-02T15:27:01.172 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:00 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-02T15:27:01.173 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:00 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-02T15:27:01.173 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:00 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-02T15:27:01.173 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:00 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-02T15:27:01.173 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:00 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-02T15:27:01.173 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:00 smithi142 ceph-mon[45728]: Manager daemon x is now available 2024-02-02T15:27:01.173 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:00 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:27:01.173 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:00 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:27:01.173 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:00 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-02-02T15:27:01.173 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:00 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-02-02T15:27:01.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:00 smithi001 ceph-mon[35470]: Active manager daemon x restarted 2024-02-02T15:27:01.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:00 smithi001 ceph-mon[35470]: Activating manager daemon x 2024-02-02T15:27:01.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:00 smithi001 ceph-mon[35470]: osdmap e89: 12 total, 12 up, 12 in 2024-02-02T15:27:01.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:00 smithi001 ceph-mon[35470]: mgrmap e22: x(active, starting, since 0.0124561s), standbys: y, z 2024-02-02T15:27:01.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:00 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-02T15:27:01.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:00 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-02T15:27:01.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:00 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-02T15:27:01.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:00 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-02-02T15:27:01.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:00 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-02-02T15:27:01.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:00 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-02-02T15:27:01.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:00 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-02-02T15:27:01.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:00 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-02-02T15:27:01.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:00 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-02-02T15:27:01.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:00 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-02-02T15:27:01.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:00 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-02-02T15:27:01.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:00 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-02-02T15:27:01.192 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:00 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-02T15:27:01.192 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:00 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-02T15:27:01.192 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:00 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-02-02T15:27:01.192 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:00 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-02T15:27:01.192 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:00 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-02T15:27:01.192 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:00 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-02T15:27:01.192 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:00 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-02T15:27:01.192 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:00 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-02T15:27:01.192 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:00 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-02T15:27:01.192 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:00 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-02T15:27:01.192 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:00 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-02T15:27:01.192 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:00 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-02T15:27:01.192 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:00 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-02T15:27:01.192 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:00 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-02T15:27:01.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:00 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-02T15:27:01.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:00 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-02T15:27:01.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:00 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-02T15:27:01.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:00 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-02T15:27:01.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:00 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-02T15:27:01.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:00 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-02T15:27:01.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:00 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-02T15:27:01.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:00 smithi001 ceph-mon[35470]: Manager daemon x is now available 2024-02-02T15:27:01.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:00 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:27:01.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:00 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:27:01.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:00 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-02-02T15:27:01.193 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:00 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-02-02T15:27:01.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:01.218+0000 7fdf14a00700 1 -- 172.21.15.1:0/1848261795 <== mgr.25603 v2:172.21.15.1:6800/167617173 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fdf10061180 con 0x7fdef00671d0 2024-02-02T15:27:01.218 INFO:teuthology.orchestra.run.smithi001.stdout:Schedule set for path / 2024-02-02T15:27:01.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:01.221+0000 7fdf15e00700 1 -- 172.21.15.1:0/1848261795 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fdef00671d0 msgr2=0x7fdef00675b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:01.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:01.221+0000 7fdf15e00700 1 --2- 172.21.15.1:0/1848261795 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fdef00671d0 0x7fdef00675b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fdef8009fd0 tx=0x7fdef8005dc0 comp rx=0 tx=0).stop 2024-02-02T15:27:01.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:01.221+0000 7fdf15e00700 1 -- 172.21.15.1:0/1848261795 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf100737a0 msgr2=0x7fdf10198d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:01.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:01.221+0000 7fdf15e00700 1 --2- 172.21.15.1:0/1848261795 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf100737a0 0x7fdf10198d20 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fdf0000ec10 tx=0x7fdf0000ef20 comp rx=0 tx=0).stop 2024-02-02T15:27:01.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:01.221+0000 7fdf0f400700 1 -- 172.21.15.1:0/1848261795 reap_dead start 2024-02-02T15:27:01.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:01.221+0000 7fdf15e00700 1 -- 172.21.15.1:0/1848261795 shutdown_connections 2024-02-02T15:27:01.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:01.221+0000 7fdf15e00700 1 -- 172.21.15.1:0/1848261795 >> 172.21.15.1:0/1848261795 conn(0x7fdf100fc300 msgr2=0x7fdf100fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:01.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:01.221+0000 7fdf15e00700 1 -- 172.21.15.1:0/1848261795 shutdown_connections 2024-02-02T15:27:01.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:01.221+0000 7fdf15e00700 1 -- 172.21.15.1:0/1848261795 wait complete. 2024-02-02T15:27:01.241 DEBUG:teuthology.orchestra.run.smithi001:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-02-02T15:27:01.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:00 smithi071 ceph-mon[44707]: Active manager daemon x restarted 2024-02-02T15:27:01.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:00 smithi071 ceph-mon[44707]: Activating manager daemon x 2024-02-02T15:27:01.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:00 smithi071 ceph-mon[44707]: osdmap e89: 12 total, 12 up, 12 in 2024-02-02T15:27:01.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:00 smithi071 ceph-mon[44707]: mgrmap e22: x(active, starting, since 0.0124561s), standbys: y, z 2024-02-02T15:27:01.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:00 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-02T15:27:01.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:00 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-02T15:27:01.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:00 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-02T15:27:01.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:00 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-02-02T15:27:01.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:00 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-02-02T15:27:01.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:00 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-02-02T15:27:01.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:00 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-02-02T15:27:01.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:00 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-02-02T15:27:01.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:00 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-02-02T15:27:01.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:00 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-02-02T15:27:01.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:00 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-02-02T15:27:01.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:00 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-02-02T15:27:01.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:00 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-02T15:27:01.366 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:00 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-02T15:27:01.366 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:00 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-02-02T15:27:01.366 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:00 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-02T15:27:01.366 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:00 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-02T15:27:01.366 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:00 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-02T15:27:01.366 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:00 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-02T15:27:01.366 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:00 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-02T15:27:01.366 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:00 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-02T15:27:01.366 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:00 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-02T15:27:01.366 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:00 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-02T15:27:01.366 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:00 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-02T15:27:01.366 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:00 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-02T15:27:01.366 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:00 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-02T15:27:01.366 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:00 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-02-02T15:27:01.367 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:00 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-02-02T15:27:01.367 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:00 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-02-02T15:27:01.367 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:00 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-02-02T15:27:01.367 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:00 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-02T15:27:01.367 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:00 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-02T15:27:01.367 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:00 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-02T15:27:01.367 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:00 smithi071 ceph-mon[44707]: Manager daemon x is now available 2024-02-02T15:27:01.367 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:00 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:27:01.367 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:00 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:27:01.367 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:00 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-02-02T15:27:01.367 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:00 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-02-02T15:27:01.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:01.465+0000 7f1cb2a00700 1 -- 172.21.15.1:0/3314858001 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1cac100c80 msgr2=0x7f1cac1010e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:01.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:01.465+0000 7f1cb2a00700 1 --2- 172.21.15.1:0/3314858001 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1cac100c80 0x7f1cac1010e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1c940099c0 tx=0x7f1c94009cd0 comp rx=0 tx=0).stop 2024-02-02T15:27:01.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:01.465+0000 7f1cb2a00700 1 -- 172.21.15.1:0/3314858001 shutdown_connections 2024-02-02T15:27:01.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:01.465+0000 7f1cb2a00700 1 --2- 172.21.15.1:0/3314858001 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1cac101620 0x7f1cac107ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:01.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:01.465+0000 7f1cb2a00700 1 --2- 172.21.15.1:0/3314858001 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1cac0ffa90 0x7f1cac0ffeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:01.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:01.465+0000 7f1cb2a00700 1 --2- 172.21.15.1:0/3314858001 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1cac100c80 0x7f1cac1010e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:01.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:01.465+0000 7f1cb2a00700 1 -- 172.21.15.1:0/3314858001 >> 172.21.15.1:0/3314858001 conn(0x7f1cac0fb020 msgr2=0x7f1cac0fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:01.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:01.465+0000 7f1cb2a00700 1 -- 172.21.15.1:0/3314858001 shutdown_connections 2024-02-02T15:27:01.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:01.465+0000 7f1cb2a00700 1 -- 172.21.15.1:0/3314858001 wait complete. 2024-02-02T15:27:01.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:01.466+0000 7f1cb2a00700 1 Processor -- start 2024-02-02T15:27:01.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:01.466+0000 7f1cb2a00700 1 -- start start 2024-02-02T15:27:01.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:01.466+0000 7f1cb2a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1cac0ffa90 0x7f1cac198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:01.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:01.466+0000 7f1cb2a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1cac100c80 0x7f1cac198fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:01.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:01.466+0000 7f1cb2a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1cac101620 0x7f1cac1a0030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:01.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:01.466+0000 7f1cb2a00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cac1a0600 con 0x7f1cac100c80 2024-02-02T15:27:01.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:01.466+0000 7f1cb2a00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cac1a0740 con 0x7f1cac101620 2024-02-02T15:27:01.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:01.466+0000 7f1cb2a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cac1a08d0 con 0x7f1cac0ffa90 2024-02-02T15:27:01.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:01.467+0000 7f1cb0c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1cac100c80 0x7f1cac198fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:01.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:01.467+0000 7f1cb0c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1cac100c80 0x7f1cac198fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46450/0 (socket says 172.21.15.1:46450) 2024-02-02T15:27:01.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:01.467+0000 7f1cb0c00700 1 -- 172.21.15.1:0/598553733 learned_addr learned my addr 172.21.15.1:0/598553733 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:01.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:01.467+0000 7f1cb2000700 1 --2- 172.21.15.1:0/598553733 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1cac101620 0x7f1cac1a0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:01.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:01.467+0000 7f1cb1600700 1 --2- 172.21.15.1:0/598553733 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1cac0ffa90 0x7f1cac198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:01.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:01.467+0000 7f1cb2000700 1 -- 172.21.15.1:0/598553733 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1cac0ffa90 msgr2=0x7f1cac198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:01.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:01.467+0000 7f1cb2000700 1 --2- 172.21.15.1:0/598553733 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1cac0ffa90 0x7f1cac198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:01.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:01.467+0000 7f1cb2000700 1 -- 172.21.15.1:0/598553733 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1cac100c80 msgr2=0x7f1cac198fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:01.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:01.467+0000 7f1cb2000700 1 --2- 172.21.15.1:0/598553733 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1cac100c80 0x7f1cac198fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:01.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:01.467+0000 7f1cb2000700 1 -- 172.21.15.1:0/598553733 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c940095f0 con 0x7f1cac101620 2024-02-02T15:27:01.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:01.467+0000 7f1cb2000700 1 --2- 172.21.15.1:0/598553733 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1cac101620 0x7f1cac1a0030 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1ca400efc0 tx=0x7f1ca400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:01.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:01.468+0000 7f1caa000700 1 -- 172.21.15.1:0/598553733 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca4018070 con 0x7f1cac101620 2024-02-02T15:27:01.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:01.468+0000 7f1caa000700 1 -- 172.21.15.1:0/598553733 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ca400f040 con 0x7f1cac101620 2024-02-02T15:27:01.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:01.468+0000 7f1caa000700 1 -- 172.21.15.1:0/598553733 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ca400cd40 con 0x7f1cac101620 2024-02-02T15:27:01.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:01.468+0000 7f1cb2a00700 1 -- 172.21.15.1:0/598553733 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cac1a0b50 con 0x7f1cac101620 2024-02-02T15:27:01.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:01.468+0000 7f1cb2a00700 1 -- 172.21.15.1:0/598553733 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cac1a1010 con 0x7f1cac101620 2024-02-02T15:27:01.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:01.469+0000 7f1cb2a00700 1 -- 172.21.15.1:0/598553733 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cac10b2f0 con 0x7f1cac101620 2024-02-02T15:27:01.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:01.471+0000 7f1caa000700 1 -- 172.21.15.1:0/598553733 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 23) v1 ==== 134882+0+0 (secure 0 0 0) 0x7f1ca4003a00 con 0x7f1cac101620 2024-02-02T15:27:01.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:01.472+0000 7f1caa000700 1 --2- 172.21.15.1:0/598553733 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f1c8c0a0320 0x7f1c8c0a27e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:01.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:01.472+0000 7f1caa000700 1 -- 172.21.15.1:0/598553733 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f1ca40cb5d0 con 0x7f1cac101620 2024-02-02T15:27:01.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:01.473+0000 7f1cb1600700 1 --2- 172.21.15.1:0/598553733 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f1c8c0a0320 0x7f1c8c0a27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:01.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:01.473+0000 7f1cb1600700 1 --2- 172.21.15.1:0/598553733 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f1c8c0a0320 0x7f1c8c0a27e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1cac100ae0 tx=0x7f1c9c009260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:01.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:01.476+0000 7f1caa000700 1 -- 172.21.15.1:0/598553733 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f1ca4095040 con 0x7f1cac101620 2024-02-02T15:27:01.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:01.734+0000 7f1cb2a00700 1 -- 172.21.15.1:0/598553733 --> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) v1 -- 0x7f1cac061180 con 0x7f1c8c0a0320 2024-02-02T15:27:01.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:01.799+0000 7f1caa000700 1 -- 172.21.15.1:0/598553733 <== mgr.25603 v2:172.21.15.1:6800/167617173 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7f1cac061180 con 0x7f1c8c0a0320 2024-02-02T15:27:01.800 INFO:teuthology.orchestra.run.smithi001.stdout:Retention added to path / 2024-02-02T15:27:01.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:01.802+0000 7f1cb2a00700 1 -- 172.21.15.1:0/598553733 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f1c8c0a0320 msgr2=0x7f1c8c0a27e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:01.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:01.802+0000 7f1cb2a00700 1 --2- 172.21.15.1:0/598553733 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f1c8c0a0320 0x7f1c8c0a27e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1cac100ae0 tx=0x7f1c9c009260 comp rx=0 tx=0).stop 2024-02-02T15:27:01.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:01.802+0000 7f1cb2a00700 1 -- 172.21.15.1:0/598553733 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1cac101620 msgr2=0x7f1cac1a0030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:01.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:01.802+0000 7f1cb2a00700 1 --2- 172.21.15.1:0/598553733 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1cac101620 0x7f1cac1a0030 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1ca400efc0 tx=0x7f1ca400c3a0 comp rx=0 tx=0).stop 2024-02-02T15:27:01.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:01.802+0000 7f1cb2a00700 1 -- 172.21.15.1:0/598553733 shutdown_connections 2024-02-02T15:27:01.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:01.803+0000 7f1cb2a00700 1 --2- 172.21.15.1:0/598553733 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f1c8c0a0320 0x7f1c8c0a27e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:01.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:01.803+0000 7f1cb2a00700 1 --2- 172.21.15.1:0/598553733 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1cac101620 0x7f1cac1a0030 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:01.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:01.803+0000 7f1cb2a00700 1 --2- 172.21.15.1:0/598553733 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1cac0ffa90 0x7f1cac198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:01.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:01.803+0000 7f1cb2a00700 1 --2- 172.21.15.1:0/598553733 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1cac100c80 0x7f1cac198fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:01.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:01.803+0000 7f1cb2a00700 1 -- 172.21.15.1:0/598553733 >> 172.21.15.1:0/598553733 conn(0x7f1cac0fb020 msgr2=0x7f1cac0fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:01.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:01.803+0000 7f1cb2a00700 1 -- 172.21.15.1:0/598553733 shutdown_connections 2024-02-02T15:27:01.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:01.803+0000 7f1cb2a00700 1 -- 172.21.15.1:0/598553733 wait complete. 2024-02-02T15:27:01.830 DEBUG:teuthology.orchestra.run.smithi001:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-02-02T15:27:02.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.053+0000 7fa55f600700 1 -- 172.21.15.1:0/692733639 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa558108c70 msgr2=0x7fa55810b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:02.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.053+0000 7fa55f600700 1 --2- 172.21.15.1:0/692733639 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa558108c70 0x7fa55810b100 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa550009f90 tx=0x7fa55000b330 comp rx=0 tx=0).stop 2024-02-02T15:27:02.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.053+0000 7fa55f600700 1 -- 172.21.15.1:0/692733639 shutdown_connections 2024-02-02T15:27:02.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.053+0000 7fa55f600700 1 --2- 172.21.15.1:0/692733639 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa558108c70 0x7fa55810b100 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:02.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.053+0000 7fa55f600700 1 --2- 172.21.15.1:0/692733639 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa558074d00 0x7fa558073150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:02.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.053+0000 7fa55f600700 1 --2- 172.21.15.1:0/692733639 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa558073720 0x7fa558073ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:02.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.053+0000 7fa55f600700 1 -- 172.21.15.1:0/692733639 >> 172.21.15.1:0/692733639 conn(0x7fa5580fc2f0 msgr2=0x7fa5580fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:02.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.053+0000 7fa55f600700 1 -- 172.21.15.1:0/692733639 shutdown_connections 2024-02-02T15:27:02.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.053+0000 7fa55f600700 1 -- 172.21.15.1:0/692733639 wait complete. 2024-02-02T15:27:02.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.055+0000 7fa55f600700 1 Processor -- start 2024-02-02T15:27:02.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.055+0000 7fa55f600700 1 -- start start 2024-02-02T15:27:02.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.055+0000 7fa55f600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa558073720 0x7fa558198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:02.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.055+0000 7fa55f600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa558074d00 0x7fa558199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:02.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.055+0000 7fa55f600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa558108c70 0x7fa5581a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:02.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.055+0000 7fa55f600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5581a0770 con 0x7fa558073720 2024-02-02T15:27:02.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.055+0000 7fa55f600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5581a08e0 con 0x7fa558108c70 2024-02-02T15:27:02.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.055+0000 7fa55f600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5581a0aa0 con 0x7fa558074d00 2024-02-02T15:27:02.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.056+0000 7fa55cc00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa558073720 0x7fa558198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:02.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.056+0000 7fa557e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa558074d00 0x7fa558199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:02.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.056+0000 7fa557e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa558074d00 0x7fa558199170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:40882/0 (socket says 172.21.15.1:40882) 2024-02-02T15:27:02.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.056+0000 7fa55cc00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa558073720 0x7fa558198c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46476/0 (socket says 172.21.15.1:46476) 2024-02-02T15:27:02.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.056+0000 7fa557e00700 1 -- 172.21.15.1:0/2009786180 learned_addr learned my addr 172.21.15.1:0/2009786180 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:02.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.056+0000 7fa55d600700 1 --2- 172.21.15.1:0/2009786180 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa558108c70 0x7fa5581a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:02.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.056+0000 7fa557e00700 1 -- 172.21.15.1:0/2009786180 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa558108c70 msgr2=0x7fa5581a01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:02.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.056+0000 7fa557e00700 1 --2- 172.21.15.1:0/2009786180 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa558108c70 0x7fa5581a01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:02.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.056+0000 7fa557e00700 1 -- 172.21.15.1:0/2009786180 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa558073720 msgr2=0x7fa558198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:02.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.056+0000 7fa557e00700 1 --2- 172.21.15.1:0/2009786180 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa558073720 0x7fa558198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:02.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.056+0000 7fa557e00700 1 -- 172.21.15.1:0/2009786180 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5400095f0 con 0x7fa558074d00 2024-02-02T15:27:02.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.056+0000 7fa55cc00700 1 --2- 172.21.15.1:0/2009786180 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa558073720 0x7fa558198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:27:02.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.056+0000 7fa557e00700 1 --2- 172.21.15.1:0/2009786180 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa558074d00 0x7fa558199170 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa54000ec10 tx=0x7fa54000ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:02.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.057+0000 7fa555600700 1 -- 172.21.15.1:0/2009786180 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa54000cc90 con 0x7fa558074d00 2024-02-02T15:27:02.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.057+0000 7fa555600700 1 -- 172.21.15.1:0/2009786180 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa5400044e0 con 0x7fa558074d00 2024-02-02T15:27:02.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.057+0000 7fa555600700 1 -- 172.21.15.1:0/2009786180 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa540010430 con 0x7fa558074d00 2024-02-02T15:27:02.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.057+0000 7fa55f600700 1 -- 172.21.15.1:0/2009786180 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa550009c40 con 0x7fa558074d00 2024-02-02T15:27:02.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.057+0000 7fa55f600700 1 -- 172.21.15.1:0/2009786180 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5581a10b0 con 0x7fa558074d00 2024-02-02T15:27:02.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.058+0000 7fa55f600700 1 -- 172.21.15.1:0/2009786180 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa558107370 con 0x7fa558074d00 2024-02-02T15:27:02.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.060+0000 7fa555600700 1 -- 172.21.15.1:0/2009786180 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 23) v1 ==== 134882+0+0 (secure 0 0 0) 0x7fa540003fb0 con 0x7fa558074d00 2024-02-02T15:27:02.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.061+0000 7fa555600700 1 --2- 172.21.15.1:0/2009786180 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fa5380a0320 0x7fa5380a27e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:02.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.061+0000 7fa555600700 1 -- 172.21.15.1:0/2009786180 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fa540014070 con 0x7fa558074d00 2024-02-02T15:27:02.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.062+0000 7fa55cc00700 1 --2- 172.21.15.1:0/2009786180 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fa5380a0320 0x7fa5380a27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:02.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.062+0000 7fa55cc00700 1 --2- 172.21.15.1:0/2009786180 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fa5380a0320 0x7fa5380a27e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa55819a100 tx=0x7fa548009260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:02.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.063+0000 7fa555600700 1 -- 172.21.15.1:0/2009786180 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7fa540095a40 con 0x7fa558074d00 2024-02-02T15:27:02.131 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:02 smithi001 ceph-mon[35470]: mgrmap e23: x(active, since 1.02459s), standbys: y, z 2024-02-02T15:27:02.131 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:02 smithi001 ceph-mon[35470]: from='client.15585 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:27:02.131 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:02 smithi001 ceph-mon[35470]: pgmap v3: 129 pgs: 129 active+clean; 1.0 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-02T15:27:02.131 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:02 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:27:02.132 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:02 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:27:02.132 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:02 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:27:02.132 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:02 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:27:02.132 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:02 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:27:02.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.345+0000 7fa55f600700 1 -- 172.21.15.1:0/2009786180 --> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7fa558061180 con 0x7fa5380a0320 2024-02-02T15:27:02.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.353+0000 7fa555600700 1 -- 172.21.15.1:0/2009786180 <== mgr.25603 v2:172.21.15.1:6800/167617173 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+283 (secure 0 0 0) 0x7fa558061180 con 0x7fa5380a0320 2024-02-02T15:27:02.353 INFO:teuthology.orchestra.run.smithi001.stdout:{"fs": "cephfs", "subvol": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-02-02T00:00:00", "created": "2024-02-02T15:27:01", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-02-02T15:27:02.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.356+0000 7fa55f600700 1 -- 172.21.15.1:0/2009786180 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fa5380a0320 msgr2=0x7fa5380a27e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:02.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.356+0000 7fa55f600700 1 --2- 172.21.15.1:0/2009786180 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fa5380a0320 0x7fa5380a27e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa55819a100 tx=0x7fa548009260 comp rx=0 tx=0).stop 2024-02-02T15:27:02.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.356+0000 7fa55f600700 1 -- 172.21.15.1:0/2009786180 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa558074d00 msgr2=0x7fa558199170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:02.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.356+0000 7fa55f600700 1 --2- 172.21.15.1:0/2009786180 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa558074d00 0x7fa558199170 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa54000ec10 tx=0x7fa54000ef20 comp rx=0 tx=0).stop 2024-02-02T15:27:02.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.356+0000 7fa55f600700 1 -- 172.21.15.1:0/2009786180 shutdown_connections 2024-02-02T15:27:02.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.356+0000 7fa55f600700 1 --2- 172.21.15.1:0/2009786180 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fa5380a0320 0x7fa5380a27e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:02.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.356+0000 7fa55f600700 1 --2- 172.21.15.1:0/2009786180 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa558108c70 0x7fa5581a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:02.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.356+0000 7fa55f600700 1 --2- 172.21.15.1:0/2009786180 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa558073720 0x7fa558198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:02.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.356+0000 7fa55f600700 1 --2- 172.21.15.1:0/2009786180 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa558074d00 0x7fa558199170 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:02.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.356+0000 7fa55f600700 1 -- 172.21.15.1:0/2009786180 >> 172.21.15.1:0/2009786180 conn(0x7fa5580fc2f0 msgr2=0x7fa5580fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:02.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.357+0000 7fa55f600700 1 -- 172.21.15.1:0/2009786180 shutdown_connections 2024-02-02T15:27:02.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.357+0000 7fa55f600700 1 -- 172.21.15.1:0/2009786180 wait complete. 2024-02-02T15:27:02.380 DEBUG:teuthology.orchestra.run.smithi001:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-02-02T15:27:02.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:02 smithi142 ceph-mon[45728]: mgrmap e23: x(active, since 1.02459s), standbys: y, z 2024-02-02T15:27:02.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:02 smithi142 ceph-mon[45728]: from='client.15585 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:27:02.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:02 smithi142 ceph-mon[45728]: pgmap v3: 129 pgs: 129 active+clean; 1.0 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-02T15:27:02.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:02 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:27:02.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:02 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:27:02.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:02 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:27:02.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:02 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:27:02.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:02 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:27:02.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.602+0000 7f355f000700 1 -- 172.21.15.1:0/3974476027 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3558101f70 msgr2=0x7f35581023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:02.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.602+0000 7f355f000700 1 --2- 172.21.15.1:0/3974476027 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3558101f70 0x7f35581023d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f35400099c0 tx=0x7f3540009cd0 comp rx=0 tx=0).stop 2024-02-02T15:27:02.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.603+0000 7f355f000700 1 -- 172.21.15.1:0/3974476027 shutdown_connections 2024-02-02T15:27:02.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.603+0000 7f355f000700 1 --2- 172.21.15.1:0/3974476027 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3558102910 0x7f35581091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:02.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.603+0000 7f355f000700 1 --2- 172.21.15.1:0/3974476027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3558100d80 0x7f35581011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:02.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.603+0000 7f355f000700 1 --2- 172.21.15.1:0/3974476027 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3558101f70 0x7f35581023d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:02.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.603+0000 7f355f000700 1 -- 172.21.15.1:0/3974476027 >> 172.21.15.1:0/3974476027 conn(0x7f35580fc2f0 msgr2=0x7f35580fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:02.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.603+0000 7f355f000700 1 -- 172.21.15.1:0/3974476027 shutdown_connections 2024-02-02T15:27:02.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.603+0000 7f355f000700 1 -- 172.21.15.1:0/3974476027 wait complete. 2024-02-02T15:27:02.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.604+0000 7f355f000700 1 Processor -- start 2024-02-02T15:27:02.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.604+0000 7f355f000700 1 -- start start 2024-02-02T15:27:02.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.604+0000 7f355f000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3558100d80 0x7f3558196a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:02.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.604+0000 7f355f000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3558101f70 0x7f3558196fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:02.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.604+0000 7f355f000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3558102910 0x7f355819e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:02.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.604+0000 7f355f000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f355819e5e0 con 0x7f3558100d80 2024-02-02T15:27:02.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.604+0000 7f355f000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f355819e720 con 0x7f3558101f70 2024-02-02T15:27:02.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.604+0000 7f355f000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f355819e8b0 con 0x7f3558102910 2024-02-02T15:27:02.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.604+0000 7f3557e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3558100d80 0x7f3558196a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:02.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.604+0000 7f3557e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3558100d80 0x7f3558196a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46508/0 (socket says 172.21.15.1:46508) 2024-02-02T15:27:02.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.604+0000 7f3557e00700 1 -- 172.21.15.1:0/2588242805 learned_addr learned my addr 172.21.15.1:0/2588242805 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:02.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.605+0000 7f355d000700 1 --2- 172.21.15.1:0/2588242805 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3558102910 0x7f355819e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:02.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.605+0000 7f3557400700 1 --2- 172.21.15.1:0/2588242805 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3558101f70 0x7f3558196fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:02.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.605+0000 7f3557e00700 1 -- 172.21.15.1:0/2588242805 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3558102910 msgr2=0x7f355819e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:02.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.605+0000 7f3557e00700 1 --2- 172.21.15.1:0/2588242805 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3558102910 0x7f355819e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:02.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.605+0000 7f3557e00700 1 -- 172.21.15.1:0/2588242805 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3558101f70 msgr2=0x7f3558196fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:02.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.605+0000 7f3557e00700 1 --2- 172.21.15.1:0/2588242805 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3558101f70 0x7f3558196fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:02.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.605+0000 7f3557e00700 1 -- 172.21.15.1:0/2588242805 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35400095f0 con 0x7f3558100d80 2024-02-02T15:27:02.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.605+0000 7f355d000700 1 --2- 172.21.15.1:0/2588242805 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3558102910 0x7f355819e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:27:02.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.605+0000 7f3557400700 1 --2- 172.21.15.1:0/2588242805 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3558101f70 0x7f3558196fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:27:02.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.606+0000 7f3557e00700 1 --2- 172.21.15.1:0/2588242805 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3558100d80 0x7f3558196a60 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f354800e950 tx=0x7f354800ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:02.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.606+0000 7f3554c00700 1 -- 172.21.15.1:0/2588242805 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3548009770 con 0x7f3558100d80 2024-02-02T15:27:02.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.606+0000 7f3554c00700 1 -- 172.21.15.1:0/2588242805 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f354800c7e0 con 0x7f3558100d80 2024-02-02T15:27:02.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.606+0000 7f355f000700 1 -- 172.21.15.1:0/2588242805 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f355819eb90 con 0x7f3558100d80 2024-02-02T15:27:02.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.606+0000 7f3554c00700 1 -- 172.21.15.1:0/2588242805 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3548003cf0 con 0x7f3558100d80 2024-02-02T15:27:02.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.606+0000 7f355f000700 1 -- 172.21.15.1:0/2588242805 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f355819f0a0 con 0x7f3558100d80 2024-02-02T15:27:02.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.607+0000 7f355f000700 1 -- 172.21.15.1:0/2588242805 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f355810c5b0 con 0x7f3558100d80 2024-02-02T15:27:02.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.609+0000 7f3554c00700 1 -- 172.21.15.1:0/2588242805 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 23) v1 ==== 134882+0+0 (secure 0 0 0) 0x7f354800c950 con 0x7f3558100d80 2024-02-02T15:27:02.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.610+0000 7f3554c00700 1 --2- 172.21.15.1:0/2588242805 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f35380a03a0 0x7f35380a2860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:02.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.611+0000 7f3554c00700 1 -- 172.21.15.1:0/2588242805 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f3548014070 con 0x7f3558100d80 2024-02-02T15:27:02.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.611+0000 7f3557400700 1 --2- 172.21.15.1:0/2588242805 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f35380a03a0 0x7f35380a2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:02.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.611+0000 7f3557400700 1 --2- 172.21.15.1:0/2588242805 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f35380a03a0 0x7f35380a2860 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f354000b580 tx=0x7f354001a040 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:02.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.613+0000 7f3554c00700 1 -- 172.21.15.1:0/2588242805 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f3548095680 con 0x7f3558100d80 2024-02-02T15:27:02.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:02 smithi071 ceph-mon[44707]: mgrmap e23: x(active, since 1.02459s), standbys: y, z 2024-02-02T15:27:02.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:02 smithi071 ceph-mon[44707]: from='client.15585 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:27:02.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:02 smithi071 ceph-mon[44707]: pgmap v3: 129 pgs: 129 active+clean; 1.0 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-02T15:27:02.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:02 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:27:02.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:02 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:27:02.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:02 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:27:02.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:02 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:27:02.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:02 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:27:02.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.739+0000 7f3554c00700 1 -- 172.21.15.1:0/2588242805 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mgrmap(e 24) v1 ==== 135189+0+0 (secure 0 0 0) 0x7f354800cc30 con 0x7f3558100d80 2024-02-02T15:27:02.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.869+0000 7f355f000700 1 -- 172.21.15.1:0/2588242805 --> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f3558061180 con 0x7f35380a03a0 2024-02-02T15:27:02.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.877+0000 7f3554c00700 1 -- 172.21.15.1:0/2588242805 <== mgr.25603 v2:172.21.15.1:6800/167617173 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f3558061180 con 0x7f35380a03a0 2024-02-02T15:27:02.877 INFO:teuthology.orchestra.run.smithi001.stdout:/ 1m 6m3h 2024-02-02T15:27:02.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.879+0000 7f355f000700 1 -- 172.21.15.1:0/2588242805 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f35380a03a0 msgr2=0x7f35380a2860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:02.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.879+0000 7f355f000700 1 --2- 172.21.15.1:0/2588242805 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f35380a03a0 0x7f35380a2860 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f354000b580 tx=0x7f354001a040 comp rx=0 tx=0).stop 2024-02-02T15:27:02.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.879+0000 7f355f000700 1 -- 172.21.15.1:0/2588242805 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3558100d80 msgr2=0x7f3558196a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:02.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.879+0000 7f355f000700 1 --2- 172.21.15.1:0/2588242805 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3558100d80 0x7f3558196a60 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f354800e950 tx=0x7f354800ec60 comp rx=0 tx=0).stop 2024-02-02T15:27:02.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.879+0000 7f355f000700 1 -- 172.21.15.1:0/2588242805 shutdown_connections 2024-02-02T15:27:02.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.879+0000 7f355f000700 1 --2- 172.21.15.1:0/2588242805 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f35380a03a0 0x7f35380a2860 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:02.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.879+0000 7f355f000700 1 --2- 172.21.15.1:0/2588242805 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3558102910 0x7f355819e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:02.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.879+0000 7f355f000700 1 --2- 172.21.15.1:0/2588242805 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3558100d80 0x7f3558196a60 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:02.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.879+0000 7f355f000700 1 --2- 172.21.15.1:0/2588242805 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3558101f70 0x7f3558196fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:02.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.879+0000 7f355f000700 1 -- 172.21.15.1:0/2588242805 >> 172.21.15.1:0/2588242805 conn(0x7f35580fc2f0 msgr2=0x7f35580fe570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:02.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.879+0000 7f355f000700 1 -- 172.21.15.1:0/2588242805 shutdown_connections 2024-02-02T15:27:02.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:02.879+0000 7f355f000700 1 -- 172.21.15.1:0/2588242805 wait complete. 2024-02-02T15:27:02.898 DEBUG:teuthology.orchestra.run.smithi001:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-02-02T15:27:02.982 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-02-02T15:27:02.997 INFO:teuthology.run_tasks:Running task workunit... 2024-02-02T15:27:03.038 INFO:tasks.workunit:Pulling workunits from ref 0fb327883c124fd83155c94db0804f823b805a12 2024-02-02T15:27:03.038 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-02-02T15:27:03.038 INFO:tasks.workunit:timeout=3h 2024-02-02T15:27:03.038 INFO:tasks.workunit:cleanup=True 2024-02-02T15:27:03.039 DEBUG:teuthology.orchestra.run.smithi001:> stat -- /home/ubuntu/cephtest/mnt.0 2024-02-02T15:27:03.081 INFO:teuthology.orchestra.run.smithi001.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-02-02T15:27:03.081 INFO:teuthology.orchestra.run.smithi001.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-02-02T15:27:03.081 INFO:teuthology.orchestra.run.smithi001.stdout:Device: 79h/121d Inode: 1099511628283 Links: 2 2024-02-02T15:27:03.082 INFO:teuthology.orchestra.run.smithi001.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-02T15:27:03.082 INFO:teuthology.orchestra.run.smithi001.stdout:Context: system_u:object_r:cephfs_t:s0 2024-02-02T15:27:03.082 INFO:teuthology.orchestra.run.smithi001.stdout:Access: 2024-02-02 15:25:20.243782230 +0000 2024-02-02T15:27:03.082 INFO:teuthology.orchestra.run.smithi001.stdout:Modify: 2024-02-02 15:25:20.243782230 +0000 2024-02-02T15:27:03.082 INFO:teuthology.orchestra.run.smithi001.stdout:Change: 2024-02-02 15:26:39.826194354 +0000 2024-02-02T15:27:03.082 INFO:teuthology.orchestra.run.smithi001.stdout: Birth: 2024-02-02 15:25:20.243782230 +0000 2024-02-02T15:27:03.082 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-02-02T15:27:03.082 DEBUG:teuthology.orchestra.run.smithi001:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-02-02T15:27:03.199 DEBUG:teuthology.orchestra.run.smithi001:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 0fb327883c124fd83155c94db0804f823b805a12 2024-02-02T15:27:03.244 INFO:tasks.workunit.client.0.smithi001.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-02-02T15:27:03.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:03 smithi001 ceph-mon[35470]: [02/Feb/2024:15:27:02] ENGINE Bus STARTING 2024-02-02T15:27:03.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:03 smithi001 ceph-mon[35470]: pgmap v4: 129 pgs: 129 active+clean; 1.0 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-02T15:27:03.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:03 smithi001 ceph-mon[35470]: [02/Feb/2024:15:27:02] ENGINE Serving on http://172.21.15.1:8765 2024-02-02T15:27:03.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:03 smithi001 ceph-mon[35470]: from='client.25663 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:27:03.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:03 smithi001 ceph-mon[35470]: [02/Feb/2024:15:27:02] ENGINE Serving on https://172.21.15.1:7150 2024-02-02T15:27:03.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:03 smithi001 ceph-mon[35470]: [02/Feb/2024:15:27:02] ENGINE Bus STARTED 2024-02-02T15:27:03.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:03 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:27:03.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:03 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:27:03.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:03 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:27:03.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:03 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:27:03.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:03 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:27:03.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:03 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:27:03.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:03 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:27:03.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:03 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:27:03.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:03 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:27:03.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:03 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:27:03.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:03 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:27:03.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:03 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:27:03.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:03 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:27:03.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:03 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:27:03.442 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:03 smithi001 ceph-mon[35470]: mgrmap e24: x(active, since 2s), standbys: y, z 2024-02-02T15:27:03.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:03 smithi142 ceph-mon[45728]: [02/Feb/2024:15:27:02] ENGINE Bus STARTING 2024-02-02T15:27:03.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:03 smithi142 ceph-mon[45728]: pgmap v4: 129 pgs: 129 active+clean; 1.0 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-02T15:27:03.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:03 smithi142 ceph-mon[45728]: [02/Feb/2024:15:27:02] ENGINE Serving on http://172.21.15.1:8765 2024-02-02T15:27:03.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:03 smithi142 ceph-mon[45728]: from='client.25663 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:27:03.672 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:03 smithi142 ceph-mon[45728]: [02/Feb/2024:15:27:02] ENGINE Serving on https://172.21.15.1:7150 2024-02-02T15:27:03.672 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:03 smithi142 ceph-mon[45728]: [02/Feb/2024:15:27:02] ENGINE Bus STARTED 2024-02-02T15:27:03.672 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:03 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:27:03.672 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:03 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:27:03.672 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:03 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:27:03.672 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:03 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:27:03.672 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:03 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:27:03.672 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:03 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:27:03.672 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:03 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:27:03.672 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:03 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:27:03.672 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:03 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:27:03.672 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:03 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:27:03.672 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:03 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:27:03.672 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:03 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:27:03.672 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:03 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:27:03.673 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:03 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:27:03.673 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:03 smithi142 ceph-mon[45728]: mgrmap e24: x(active, since 2s), standbys: y, z 2024-02-02T15:27:03.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:03 smithi071 ceph-mon[44707]: [02/Feb/2024:15:27:02] ENGINE Bus STARTING 2024-02-02T15:27:03.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:03 smithi071 ceph-mon[44707]: pgmap v4: 129 pgs: 129 active+clean; 1.0 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-02T15:27:03.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:03 smithi071 ceph-mon[44707]: [02/Feb/2024:15:27:02] ENGINE Serving on http://172.21.15.1:8765 2024-02-02T15:27:03.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:03 smithi071 ceph-mon[44707]: from='client.25663 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:27:03.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:03 smithi071 ceph-mon[44707]: [02/Feb/2024:15:27:02] ENGINE Serving on https://172.21.15.1:7150 2024-02-02T15:27:03.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:03 smithi071 ceph-mon[44707]: [02/Feb/2024:15:27:02] ENGINE Bus STARTED 2024-02-02T15:27:03.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:03 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:27:03.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:03 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:27:03.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:03 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:27:03.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:03 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:27:03.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:03 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:27:03.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:03 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:27:03.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:03 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:27:03.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:03 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:27:03.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:03 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:27:03.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:03 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:27:03.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:03 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:27:03.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:03 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:27:03.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:03 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:27:03.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:03 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:27:03.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:03 smithi071 ceph-mon[44707]: mgrmap e24: x(active, since 2s), standbys: y, z 2024-02-02T15:27:04.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:04 smithi142 ceph-mon[45728]: Adjusting osd_memory_target on smithi142 to 182.4M 2024-02-02T15:27:04.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:04 smithi142 ceph-mon[45728]: Unable to set osd_memory_target on smithi142 to 191358361: error parsing value: Value '191358361' is below minimum 939524096 2024-02-02T15:27:04.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:04 smithi142 ceph-mon[45728]: Adjusting osd_memory_target on smithi071 to 182.4M 2024-02-02T15:27:04.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:04 smithi142 ceph-mon[45728]: Unable to set osd_memory_target on smithi071 to 191358361: error parsing value: Value '191358361' is below minimum 939524096 2024-02-02T15:27:04.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:04 smithi142 ceph-mon[45728]: from='client.15639 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:27:04.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:04 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:27:04.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:04 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:27:04.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:04 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:27:04.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:04 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:27:04.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:04 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:27:04.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:04 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:27:04.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:04 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:27:04.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:04 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:27:04.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:04 smithi001 ceph-mon[35470]: Adjusting osd_memory_target on smithi142 to 182.4M 2024-02-02T15:27:04.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:04 smithi001 ceph-mon[35470]: Unable to set osd_memory_target on smithi142 to 191358361: error parsing value: Value '191358361' is below minimum 939524096 2024-02-02T15:27:04.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:04 smithi001 ceph-mon[35470]: Adjusting osd_memory_target on smithi071 to 182.4M 2024-02-02T15:27:04.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:04 smithi001 ceph-mon[35470]: Unable to set osd_memory_target on smithi071 to 191358361: error parsing value: Value '191358361' is below minimum 939524096 2024-02-02T15:27:04.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:04 smithi001 ceph-mon[35470]: from='client.15639 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:27:04.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:04 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:27:04.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:04 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:27:04.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:04 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:27:04.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:04 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:27:04.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:04 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:27:04.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:04 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:27:04.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:04 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:27:04.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:04 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:27:04.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:04 smithi071 ceph-mon[44707]: Adjusting osd_memory_target on smithi142 to 182.4M 2024-02-02T15:27:04.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:04 smithi071 ceph-mon[44707]: Unable to set osd_memory_target on smithi142 to 191358361: error parsing value: Value '191358361' is below minimum 939524096 2024-02-02T15:27:04.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:04 smithi071 ceph-mon[44707]: Adjusting osd_memory_target on smithi071 to 182.4M 2024-02-02T15:27:04.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:04 smithi071 ceph-mon[44707]: Unable to set osd_memory_target on smithi071 to 191358361: error parsing value: Value '191358361' is below minimum 939524096 2024-02-02T15:27:04.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:04 smithi071 ceph-mon[44707]: from='client.15639 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:27:04.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:04 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:27:04.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:04 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:27:04.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:04 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:27:04.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:04 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:27:04.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:04 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:27:04.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:04 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:27:04.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:04 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:27:04.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:04 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:27:05.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:05 smithi142 ceph-mon[45728]: Adjusting osd_memory_target on smithi001 to 223.0M 2024-02-02T15:27:05.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:05 smithi142 ceph-mon[45728]: Unable to set osd_memory_target on smithi001 to 233866752: error parsing value: Value '233866752' is below minimum 939524096 2024-02-02T15:27:05.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:05 smithi142 ceph-mon[45728]: Updating smithi001:/etc/ceph/ceph.conf 2024-02-02T15:27:05.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:05 smithi142 ceph-mon[45728]: Updating smithi071:/etc/ceph/ceph.conf 2024-02-02T15:27:05.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:05 smithi142 ceph-mon[45728]: Updating smithi142:/etc/ceph/ceph.conf 2024-02-02T15:27:05.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:05 smithi142 ceph-mon[45728]: Updating smithi071:/var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-02T15:27:05.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:05 smithi142 ceph-mon[45728]: Updating smithi142:/var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-02T15:27:05.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:05 smithi142 ceph-mon[45728]: pgmap v5: 129 pgs: 129 active+clean; 1.0 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-02T15:27:05.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:05 smithi142 ceph-mon[45728]: Updating smithi001:/var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-02T15:27:05.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:05 smithi142 ceph-mon[45728]: Updating smithi071:/etc/ceph/ceph.client.admin.keyring 2024-02-02T15:27:05.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:05 smithi142 ceph-mon[45728]: Updating smithi142:/etc/ceph/ceph.client.admin.keyring 2024-02-02T15:27:05.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:05 smithi142 ceph-mon[45728]: mgrmap e25: x(active, since 4s), standbys: y, z 2024-02-02T15:27:05.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:05 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:27:05.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:05 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:27:05.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:05 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:27:05.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:05 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:27:05.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:05 smithi001 ceph-mon[35470]: Adjusting osd_memory_target on smithi001 to 223.0M 2024-02-02T15:27:05.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:05 smithi001 ceph-mon[35470]: Unable to set osd_memory_target on smithi001 to 233866752: error parsing value: Value '233866752' is below minimum 939524096 2024-02-02T15:27:05.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:05 smithi001 ceph-mon[35470]: Updating smithi001:/etc/ceph/ceph.conf 2024-02-02T15:27:05.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:05 smithi001 ceph-mon[35470]: Updating smithi071:/etc/ceph/ceph.conf 2024-02-02T15:27:05.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:05 smithi001 ceph-mon[35470]: Updating smithi142:/etc/ceph/ceph.conf 2024-02-02T15:27:05.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:05 smithi001 ceph-mon[35470]: Updating smithi071:/var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-02T15:27:05.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:05 smithi001 ceph-mon[35470]: Updating smithi142:/var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-02T15:27:05.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:05 smithi001 ceph-mon[35470]: pgmap v5: 129 pgs: 129 active+clean; 1.0 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-02T15:27:05.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:05 smithi001 ceph-mon[35470]: Updating smithi001:/var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-02T15:27:05.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:05 smithi001 ceph-mon[35470]: Updating smithi071:/etc/ceph/ceph.client.admin.keyring 2024-02-02T15:27:05.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:05 smithi001 ceph-mon[35470]: Updating smithi142:/etc/ceph/ceph.client.admin.keyring 2024-02-02T15:27:05.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:05 smithi001 ceph-mon[35470]: mgrmap e25: x(active, since 4s), standbys: y, z 2024-02-02T15:27:05.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:05 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:27:05.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:05 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:27:05.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:05 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:27:05.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:05 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:27:05.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:05 smithi071 ceph-mon[44707]: Adjusting osd_memory_target on smithi001 to 223.0M 2024-02-02T15:27:05.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:05 smithi071 ceph-mon[44707]: Unable to set osd_memory_target on smithi001 to 233866752: error parsing value: Value '233866752' is below minimum 939524096 2024-02-02T15:27:05.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:05 smithi071 ceph-mon[44707]: Updating smithi001:/etc/ceph/ceph.conf 2024-02-02T15:27:05.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:05 smithi071 ceph-mon[44707]: Updating smithi071:/etc/ceph/ceph.conf 2024-02-02T15:27:05.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:05 smithi071 ceph-mon[44707]: Updating smithi142:/etc/ceph/ceph.conf 2024-02-02T15:27:05.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:05 smithi071 ceph-mon[44707]: Updating smithi071:/var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-02T15:27:05.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:05 smithi071 ceph-mon[44707]: Updating smithi142:/var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-02T15:27:05.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:05 smithi071 ceph-mon[44707]: pgmap v5: 129 pgs: 129 active+clean; 1.0 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail 2024-02-02T15:27:05.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:05 smithi071 ceph-mon[44707]: Updating smithi001:/var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-02T15:27:05.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:05 smithi071 ceph-mon[44707]: Updating smithi071:/etc/ceph/ceph.client.admin.keyring 2024-02-02T15:27:05.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:05 smithi071 ceph-mon[44707]: Updating smithi142:/etc/ceph/ceph.client.admin.keyring 2024-02-02T15:27:05.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:05 smithi071 ceph-mon[44707]: mgrmap e25: x(active, since 4s), standbys: y, z 2024-02-02T15:27:05.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:05 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:27:05.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:05 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:27:05.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:05 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:27:05.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:05 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:27:06.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:06 smithi142 ceph-mon[45728]: Updating smithi071:/var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-02T15:27:06.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:06 smithi142 ceph-mon[45728]: Updating smithi142:/var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-02T15:27:06.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:06 smithi142 ceph-mon[45728]: Updating smithi001:/etc/ceph/ceph.client.admin.keyring 2024-02-02T15:27:06.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:06 smithi142 ceph-mon[45728]: Updating smithi001:/var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-02T15:27:06.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:06 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:27:06.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:06 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:27:06.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:06 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:27:06.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:06 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:27:06.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:06 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:27:06.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:06 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:27:06.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:06 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:27:06.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:06 smithi001 ceph-mon[35470]: Updating smithi071:/var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-02T15:27:06.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:06 smithi001 ceph-mon[35470]: Updating smithi142:/var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-02T15:27:06.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:06 smithi001 ceph-mon[35470]: Updating smithi001:/etc/ceph/ceph.client.admin.keyring 2024-02-02T15:27:06.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:06 smithi001 ceph-mon[35470]: Updating smithi001:/var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-02T15:27:06.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:06 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:27:06.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:06 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:27:06.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:06 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:27:06.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:06 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:27:06.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:06 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:27:06.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:06 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:27:06.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:06 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:27:06.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:06 smithi071 ceph-mon[44707]: Updating smithi071:/var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-02T15:27:06.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:06 smithi071 ceph-mon[44707]: Updating smithi142:/var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-02T15:27:06.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:06 smithi071 ceph-mon[44707]: Updating smithi001:/etc/ceph/ceph.client.admin.keyring 2024-02-02T15:27:06.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:06 smithi071 ceph-mon[44707]: Updating smithi001:/var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-02T15:27:06.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:06 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:27:06.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:06 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:27:06.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:06 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:27:06.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:06 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:27:06.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:06 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:27:06.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:06 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:27:06.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:06 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:27:07.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:07 smithi142 ceph-mon[45728]: pgmap v6: 129 pgs: 129 active+clean; 1.0 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-02T15:27:07.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:07 smithi001 ceph-mon[35470]: pgmap v6: 129 pgs: 129 active+clean; 1.0 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-02T15:27:07.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:07 smithi071 ceph-mon[44707]: pgmap v6: 129 pgs: 129 active+clean; 1.0 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-02T15:27:08.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:08 smithi071 ceph-mon[44707]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 132 KiB/s wr, 70 op/s 2024-02-02T15:27:08.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:08 smithi142 ceph-mon[45728]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 132 KiB/s wr, 70 op/s 2024-02-02T15:27:08.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:08 smithi001 ceph-mon[35470]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 132 KiB/s wr, 70 op/s 2024-02-02T15:27:11.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:11 smithi071 ceph-mon[44707]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 103 KiB/s wr, 55 op/s 2024-02-02T15:27:11.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:11 smithi142 ceph-mon[45728]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 103 KiB/s wr, 55 op/s 2024-02-02T15:27:11.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:11 smithi001 ceph-mon[35470]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 103 KiB/s wr, 55 op/s 2024-02-02T15:27:13.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:13 smithi071 ceph-mon[44707]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 84 KiB/s wr, 50 op/s 2024-02-02T15:27:13.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:13 smithi142 ceph-mon[45728]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 84 KiB/s wr, 50 op/s 2024-02-02T15:27:13.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:13 smithi001 ceph-mon[35470]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 84 KiB/s wr, 50 op/s 2024-02-02T15:27:15.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:15 smithi071 ceph-mon[44707]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 45 op/s 2024-02-02T15:27:15.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:15 smithi142 ceph-mon[45728]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 45 op/s 2024-02-02T15:27:15.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:15 smithi001 ceph-mon[35470]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 77 KiB/s wr, 45 op/s 2024-02-02T15:27:17.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:17 smithi071 ceph-mon[44707]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 48 op/s 2024-02-02T15:27:17.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:17 smithi142 ceph-mon[45728]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 48 op/s 2024-02-02T15:27:17.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:17 smithi001 ceph-mon[35470]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 48 op/s 2024-02-02T15:27:19.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:19 smithi071 ceph-mon[44707]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 49 op/s 2024-02-02T15:27:19.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:19 smithi142 ceph-mon[45728]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 49 op/s 2024-02-02T15:27:19.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:19 smithi001 ceph-mon[35470]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 49 op/s 2024-02-02T15:27:20.503 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-02T15:27:20.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.716+0000 7f303a400700 1 -- 172.21.15.1:0/3191279645 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3034106110 msgr2=0x7f303410dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:20.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.716+0000 7f303a400700 1 --2- 172.21.15.1:0/3191279645 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3034106110 0x7f303410dc90 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f302c00b0a0 tx=0x7f302c00b3b0 comp rx=0 tx=0).stop 2024-02-02T15:27:20.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.717+0000 7f303a400700 1 -- 172.21.15.1:0/3191279645 shutdown_connections 2024-02-02T15:27:20.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.717+0000 7f303a400700 1 --2- 172.21.15.1:0/3191279645 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3034106110 0x7f303410dc90 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:20.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.717+0000 7f303a400700 1 --2- 172.21.15.1:0/3191279645 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3034100ff0 0x7f30341013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:20.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.717+0000 7f303a400700 1 --2- 172.21.15.1:0/3191279645 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f30341019a0 0x7f30341059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:20.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.717+0000 7f303a400700 1 -- 172.21.15.1:0/3191279645 >> 172.21.15.1:0/3191279645 conn(0x7f30340fc810 msgr2=0x7f30340fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:20.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.717+0000 7f303a400700 1 -- 172.21.15.1:0/3191279645 shutdown_connections 2024-02-02T15:27:20.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.717+0000 7f303a400700 1 -- 172.21.15.1:0/3191279645 wait complete. 2024-02-02T15:27:20.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.718+0000 7f303a400700 1 Processor -- start 2024-02-02T15:27:20.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.718+0000 7f303a400700 1 -- start start 2024-02-02T15:27:20.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.718+0000 7f303a400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3034100ff0 0x7f30341991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:20.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.718+0000 7f303a400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3034106110 0x7f3034199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:20.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.718+0000 7f303a400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f303419db60 0x7f303419dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:20.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.718+0000 7f303a400700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f303419e5e0 con 0x7f3034100ff0 2024-02-02T15:27:20.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.718+0000 7f303a400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f303419e750 con 0x7f3034106110 2024-02-02T15:27:20.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.718+0000 7f303a400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f303419e910 con 0x7f303419db60 2024-02-02T15:27:20.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.718+0000 7f3033400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3034100ff0 0x7f30341991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:20.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.719+0000 7f3033400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3034100ff0 0x7f30341991c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43468/0 (socket says 172.21.15.1:43468) 2024-02-02T15:27:20.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.719+0000 7f3033400700 1 -- 172.21.15.1:0/2379111865 learned_addr learned my addr 172.21.15.1:0/2379111865 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:20.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.719+0000 7f3032a00700 1 --2- 172.21.15.1:0/2379111865 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3034106110 0x7f3034199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:20.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.719+0000 7f3033e00700 1 --2- 172.21.15.1:0/2379111865 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f303419db60 0x7f303419dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:20.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.719+0000 7f3033400700 1 -- 172.21.15.1:0/2379111865 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f303419db60 msgr2=0x7f303419dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:20.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.719+0000 7f3033400700 1 --2- 172.21.15.1:0/2379111865 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f303419db60 0x7f303419dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:20.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.719+0000 7f3033400700 1 -- 172.21.15.1:0/2379111865 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3034106110 msgr2=0x7f3034199700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:20.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.719+0000 7f3033400700 1 --2- 172.21.15.1:0/2379111865 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3034106110 0x7f3034199700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:20.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.719+0000 7f3033400700 1 -- 172.21.15.1:0/2379111865 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f302c009d00 con 0x7f3034100ff0 2024-02-02T15:27:20.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.719+0000 7f3033e00700 1 --2- 172.21.15.1:0/2379111865 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f303419db60 0x7f303419dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:27:20.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.719+0000 7f3033400700 1 --2- 172.21.15.1:0/2379111865 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3034100ff0 0x7f30341991c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f302400e9a0 tx=0x7f302400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:20.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.719+0000 7f302be00700 1 -- 172.21.15.1:0/2379111865 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3024009790 con 0x7f3034100ff0 2024-02-02T15:27:20.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.720+0000 7f302be00700 1 -- 172.21.15.1:0/2379111865 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f302400cbd0 con 0x7f3034100ff0 2024-02-02T15:27:20.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.720+0000 7f303a400700 1 -- 172.21.15.1:0/2379111865 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30341a5e70 con 0x7f3034100ff0 2024-02-02T15:27:20.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.720+0000 7f302be00700 1 -- 172.21.15.1:0/2379111865 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3024018680 con 0x7f3034100ff0 2024-02-02T15:27:20.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.720+0000 7f303a400700 1 -- 172.21.15.1:0/2379111865 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30341a62f0 con 0x7f3034100ff0 2024-02-02T15:27:20.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.723+0000 7f302be00700 1 -- 172.21.15.1:0/2379111865 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135245+0+0 (secure 0 0 0) 0x7f302400cd40 con 0x7f3034100ff0 2024-02-02T15:27:20.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.724+0000 7f302be00700 1 --2- 172.21.15.1:0/2379111865 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f30140a0720 0x7f30140a2be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:20.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.724+0000 7f304df24b80 1 Processor -- start 2024-02-02T15:27:20.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.724+0000 7f302be00700 1 -- 172.21.15.1:0/2379111865 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f3024014070 con 0x7f3034100ff0 2024-02-02T15:27:20.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.724+0000 7f304df24b80 1 -- start start 2024-02-02T15:27:20.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.724+0000 7f3032a00700 1 --2- 172.21.15.1:0/2379111865 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f30140a0720 0x7f30140a2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:20.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.724+0000 7f304df24b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5585b8b0a010 0x5585b8b0a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:20.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.724+0000 7f304df24b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585b8b0a930 0x5585b8b152a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:20.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.725+0000 7f304df24b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5585b8b157e0 0x5585b8b17bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:20.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.725+0000 7f304df24b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585b8a7e4c0 con 0x5585b8b0a010 2024-02-02T15:27:20.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.725+0000 7f304df24b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585b8a62aa0 con 0x5585b8b157e0 2024-02-02T15:27:20.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.725+0000 7f304df24b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585b8a60460 con 0x5585b8b0a930 2024-02-02T15:27:20.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.725+0000 7f3033e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5585b8b157e0 0x5585b8b17bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:20.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.725+0000 7f3033400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5585b8b0a010 0x5585b8b0a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:20.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.725+0000 7f3033e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5585b8b157e0 0x5585b8b17bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59298/0 (socket says 172.21.15.1:59298) 2024-02-02T15:27:20.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.725+0000 7f3033e00700 1 -- 172.21.15.1:0/2903876572 learned_addr learned my addr 172.21.15.1:0/2903876572 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:20.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.725+0000 7f3033400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5585b8b0a010 0x5585b8b0a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43472/0 (socket says 172.21.15.1:43472) 2024-02-02T15:27:20.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.725+0000 7f3032a00700 1 --2- 172.21.15.1:0/2903876572 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585b8b0a930 0x5585b8b152a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:20.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.725+0000 7f3032a00700 1 --2- 172.21.15.1:0/2379111865 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f30140a0720 0x7f30140a2be0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f303419a7d0 tx=0x7f301c009f80 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:20.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.725+0000 7f3033e00700 1 -- 172.21.15.1:0/2903876572 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585b8b0a930 msgr2=0x5585b8b152a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:20.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.725+0000 7f3033e00700 1 --2- 172.21.15.1:0/2903876572 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585b8b0a930 0x5585b8b152a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:20.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.725+0000 7f3033e00700 1 -- 172.21.15.1:0/2903876572 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5585b8b0a010 msgr2=0x5585b8b0a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:20.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.725+0000 7f3033e00700 1 --2- 172.21.15.1:0/2903876572 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5585b8b0a010 0x5585b8b0a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:20.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.725+0000 7f3033e00700 1 -- 172.21.15.1:0/2903876572 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5585b88dd3b0 con 0x5585b8b157e0 2024-02-02T15:27:20.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.725+0000 7f3033400700 1 --2- 172.21.15.1:0/2903876572 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5585b8b0a010 0x5585b8b0a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:27:20.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.726+0000 7f3033e00700 1 --2- 172.21.15.1:0/2903876572 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5585b8b157e0 0x5585b8b17bd0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f302c009cc0 tx=0x7f302c0092e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=518540afc0a01d18 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:20.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.726+0000 7f3028c00700 1 -- 172.21.15.1:0/2903876572 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f302c00e070 con 0x5585b8b157e0 2024-02-02T15:27:20.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.726+0000 7f3028c00700 1 -- 172.21.15.1:0/2903876572 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f302c007bc0 con 0x5585b8b157e0 2024-02-02T15:27:20.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.726+0000 7f3028c00700 1 -- 172.21.15.1:0/2903876572 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f302c0052a0 con 0x5585b8b157e0 2024-02-02T15:27:20.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.726+0000 7f304df24b80 1 -- 172.21.15.1:0/2903876572 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5585b8b157e0 msgr2=0x5585b8b17bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:20.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.726+0000 7f304df24b80 1 --2- 172.21.15.1:0/2903876572 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5585b8b157e0 0x5585b8b17bd0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f302c009cc0 tx=0x7f302c0092e0 comp rx=0 tx=0).stop 2024-02-02T15:27:20.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.727+0000 7f304df24b80 1 -- 172.21.15.1:0/2903876572 shutdown_connections 2024-02-02T15:27:20.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.727+0000 7f304df24b80 1 --2- 172.21.15.1:0/2903876572 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5585b8b157e0 0x5585b8b17bd0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:20.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.727+0000 7f304df24b80 1 --2- 172.21.15.1:0/2903876572 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5585b8b0a010 0x5585b8b0a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:20.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.727+0000 7f304df24b80 1 --2- 172.21.15.1:0/2903876572 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585b8b0a930 0x5585b8b152a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:20.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.727+0000 7f304df24b80 1 -- 172.21.15.1:0/2903876572 >> 172.21.15.1:0/2903876572 conn(0x5585b8a69db0 msgr2=0x5585b8b063b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:20.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.727+0000 7f304df24b80 1 -- 172.21.15.1:0/2903876572 shutdown_connections 2024-02-02T15:27:20.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.727+0000 7f304df24b80 1 -- 172.21.15.1:0/2903876572 wait complete. 2024-02-02T15:27:20.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.727+0000 7f304df24b80 1 -- start start 2024-02-02T15:27:20.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.727+0000 7f304df24b80 1 Processor -- start 2024-02-02T15:27:20.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.728+0000 7f304df24b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5585b8b0a010 0x5585b8b15170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:20.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.728+0000 7f304df24b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585b8b0a930 0x5585b8b123f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:20.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.728+0000 7f304df24b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5585b8b157e0 0x5585b8b12990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:20.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.728+0000 7f304df24b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585b8a5bd50 con 0x5585b8b0a010 2024-02-02T15:27:20.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.728+0000 7f304df24b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585b89f7d80 con 0x5585b8b157e0 2024-02-02T15:27:20.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.728+0000 7f304df24b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585b89f7a20 con 0x5585b8b0a930 2024-02-02T15:27:20.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.728+0000 7f304df24b80 10 client.0 _ll_register_callbacks cb 0x5585b89f9290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:27:20.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.728+0000 7f3033400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5585b8b0a010 0x5585b8b15170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:20.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.728+0000 7f3033400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5585b8b0a010 0x5585b8b15170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43474/0 (socket says 172.21.15.1:43474) 2024-02-02T15:27:20.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.728+0000 7f3033400700 1 -- 172.21.15.1:0/3674308978 learned_addr learned my addr 172.21.15.1:0/3674308978 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:20.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.728+0000 7f3032a00700 1 --2- 172.21.15.1:0/3674308978 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585b8b0a930 0x5585b8b123f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:20.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.728+0000 7f3033e00700 1 --2- 172.21.15.1:0/3674308978 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5585b8b157e0 0x5585b8b12990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:20.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.729+0000 7f3033400700 1 -- 172.21.15.1:0/3674308978 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585b8b0a930 msgr2=0x5585b8b123f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:20.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.729+0000 7f3033400700 1 --2- 172.21.15.1:0/3674308978 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585b8b0a930 0x5585b8b123f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:20.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.729+0000 7f3033400700 1 -- 172.21.15.1:0/3674308978 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5585b8b157e0 msgr2=0x5585b8b12990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:20.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.729+0000 7f3033400700 1 --2- 172.21.15.1:0/3674308978 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5585b8b157e0 0x5585b8b12990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:20.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.729+0000 7f3033400700 1 -- 172.21.15.1:0/3674308978 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30341a62f0 con 0x5585b8b0a010 2024-02-02T15:27:20.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.729+0000 7f3033400700 1 --2- 172.21.15.1:0/3674308978 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5585b8b0a010 0x5585b8b15170 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f302401e000 tx=0x7f30240958f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee5c381997610a81 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:20.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.729+0000 7f3021600700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:27:20.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.729+0000 7f3021600700 1 -- 172.21.15.1:0/3674308978 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3024099990 con 0x5585b8b0a010 2024-02-02T15:27:20.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.729+0000 7f3021600700 1 -- 172.21.15.1:0/3674308978 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f30240cc310 con 0x5585b8b0a010 2024-02-02T15:27:20.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.729+0000 7f304df24b80 1 -- 172.21.15.1:0/3674308978 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5585b8afe0f0 con 0x5585b8b0a010 2024-02-02T15:27:20.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.729+0000 7f3021600700 1 -- 172.21.15.1:0/3674308978 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f302409d170 con 0x5585b8b0a010 2024-02-02T15:27:20.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.729+0000 7f304df24b80 10 client.15651 fetch_fsmap learned FSMap version 27 2024-02-02T15:27:20.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.729+0000 7f304df24b80 1 -- 172.21.15.1:0/3674308978 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f303410e500 con 0x5585b8b0a010 2024-02-02T15:27:20.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.730+0000 7f3021600700 1 -- 172.21.15.1:0/3674308978 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3024063020 con 0x5585b8b0a010 2024-02-02T15:27:20.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.730+0000 7f3021600700 1 -- 172.21.15.1:0/3674308978 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3024098b40 con 0x5585b8b0a010 2024-02-02T15:27:20.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.730+0000 7f304df24b80 10 client.15651 fetch_fsmap finished waiting for FSMap version 27 2024-02-02T15:27:20.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.730+0000 7f304df24b80 10 client.15651 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:27:20.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.730+0000 7f304df24b80 20 client.15651 populate_metadata read hostname 'smithi001' 2024-02-02T15:27:20.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.730+0000 7f304df24b80 1 --2- 172.21.15.1:0/3674308978 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5585b8b25190 0x5585b8b275b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:20.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.730+0000 7f304df24b80 4 client.15651 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:27:20.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.730+0000 7f304df24b80 1 -- 172.21.15.1:0/3674308978 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5585b8a54dd0 con 0x5585b8b25190 2024-02-02T15:27:20.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.730+0000 7f3032a00700 1 --2- 172.21.15.1:0/3674308978 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5585b8b25190 0x5585b8b275b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:20.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.731+0000 7f3032a00700 1 --2- 172.21.15.1:0/3674308978 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5585b8b25190 0x5585b8b275b0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc3fd696d546bf0f server_cookie=a0409219dd6ff970 in_seq=0 out_seq=0 2024-02-02T15:27:20.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.731+0000 7f3021600700 10 client.15651 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:27:20.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.731+0000 7f3021600700 1 -- 172.21.15.1:0/3674308978 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5585b8a54dd0 con 0x5585b8b25190 2024-02-02T15:27:20.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.731+0000 7f3021600700 10 client.15651 handle_command_reply: tid=0 2024-02-02T15:27:20.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.732+0000 7f304df24b80 1 client.15651 shutdown 2024-02-02T15:27:20.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.732+0000 7f304df24b80 1 -- 172.21.15.1:0/3674308978 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5585b8b0a010 msgr2=0x5585b8b15170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:20.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.732+0000 7f304df24b80 1 --2- 172.21.15.1:0/3674308978 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5585b8b0a010 0x5585b8b15170 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f302401e000 tx=0x7f30240958f0 comp rx=0 tx=0).stop 2024-02-02T15:27:20.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.732+0000 7f304df24b80 1 -- 172.21.15.1:0/3674308978 shutdown_connections 2024-02-02T15:27:20.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.732+0000 7f304df24b80 1 --2- 172.21.15.1:0/3674308978 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5585b8b25190 0x5585b8b275b0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:20.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.732+0000 7f304df24b80 1 --2- 172.21.15.1:0/3674308978 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5585b8b157e0 0x5585b8b12990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:20.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.732+0000 7f304df24b80 1 --2- 172.21.15.1:0/3674308978 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5585b8b0a010 0x5585b8b15170 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:20.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.732+0000 7f304df24b80 1 --2- 172.21.15.1:0/3674308978 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585b8b0a930 0x5585b8b123f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:20.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.732+0000 7f3021600700 0 client.15651 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:27:20.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.732+0000 7f304df24b80 1 -- 172.21.15.1:0/3674308978 >> 172.21.15.1:0/3674308978 conn(0x5585b8a69db0 msgr2=0x5585b8b05fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:20.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.732+0000 7f304df24b80 1 -- 172.21.15.1:0/3674308978 shutdown_connections 2024-02-02T15:27:20.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.732+0000 7f304df24b80 1 -- 172.21.15.1:0/3674308978 wait complete. 2024-02-02T15:27:20.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.732+0000 7f304df24b80 20 client.15651 trim_cache size 0 max 16384 2024-02-02T15:27:20.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.749+0000 7f304df24b80 1 Processor -- start 2024-02-02T15:27:20.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.749+0000 7f304df24b80 1 -- start start 2024-02-02T15:27:20.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.749+0000 7f304df24b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5585b8a69db0 0x5585b8b1c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:20.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.750+0000 7f304df24b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5585b8b0a010 0x5585b8b14ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:20.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.750+0000 7f304df24b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585b8b0a930 0x5585b8b0c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:20.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.750+0000 7f304df24b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585b88df6a0 con 0x5585b8a69db0 2024-02-02T15:27:20.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.750+0000 7f304df24b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585b8b21b00 con 0x5585b8b0a010 2024-02-02T15:27:20.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.750+0000 7f304df24b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585b8b0dff0 con 0x5585b8b0a930 2024-02-02T15:27:20.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.750+0000 7f3033400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5585b8a69db0 0x5585b8b1c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:20.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.750+0000 7f3033400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5585b8a69db0 0x5585b8b1c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43484/0 (socket says 172.21.15.1:43484) 2024-02-02T15:27:20.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.750+0000 7f3033400700 1 -- 172.21.15.1:0/20377782 learned_addr learned my addr 172.21.15.1:0/20377782 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:20.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.750+0000 7f3032a00700 1 --2- 172.21.15.1:0/20377782 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5585b8b0a010 0x5585b8b14ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:20.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.750+0000 7f3033e00700 1 --2- 172.21.15.1:0/20377782 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585b8b0a930 0x5585b8b0c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:20.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.750+0000 7f3032a00700 1 -- 172.21.15.1:0/20377782 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585b8b0a930 msgr2=0x5585b8b0c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:20.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.750+0000 7f3032a00700 1 --2- 172.21.15.1:0/20377782 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585b8b0a930 0x5585b8b0c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:20.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.750+0000 7f3032a00700 1 -- 172.21.15.1:0/20377782 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5585b8a69db0 msgr2=0x5585b8b1c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:20.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.750+0000 7f3032a00700 1 --2- 172.21.15.1:0/20377782 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5585b8a69db0 0x5585b8b1c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:20.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.750+0000 7f3032a00700 1 -- 172.21.15.1:0/20377782 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5585b8889b60 con 0x5585b8b0a010 2024-02-02T15:27:20.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.750+0000 7f3033400700 1 --2- 172.21.15.1:0/20377782 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5585b8a69db0 0x5585b8b1c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:27:20.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.751+0000 7f3032a00700 1 --2- 172.21.15.1:0/20377782 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5585b8b0a010 0x5585b8b14ca0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x5585b8b27e60 tx=0x7f301c01d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bcf36a5badcca19a server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:20.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.751+0000 7f3029600700 1 -- 172.21.15.1:0/20377782 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f301c023040 con 0x5585b8b0a010 2024-02-02T15:27:20.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.751+0000 7f3029600700 1 -- 172.21.15.1:0/20377782 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f301c025040 con 0x5585b8b0a010 2024-02-02T15:27:20.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.751+0000 7f3029600700 1 -- 172.21.15.1:0/20377782 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f301c00f8c0 con 0x5585b8b0a010 2024-02-02T15:27:20.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.751+0000 7f304df24b80 1 -- 172.21.15.1:0/20377782 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5585b8b0a010 msgr2=0x5585b8b14ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:20.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.751+0000 7f304df24b80 1 --2- 172.21.15.1:0/20377782 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5585b8b0a010 0x5585b8b14ca0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x5585b8b27e60 tx=0x7f301c01d000 comp rx=0 tx=0).stop 2024-02-02T15:27:20.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.752+0000 7f304df24b80 1 -- 172.21.15.1:0/20377782 shutdown_connections 2024-02-02T15:27:20.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.752+0000 7f304df24b80 1 --2- 172.21.15.1:0/20377782 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585b8b0a930 0x5585b8b0c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:20.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.752+0000 7f304df24b80 1 --2- 172.21.15.1:0/20377782 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5585b8a69db0 0x5585b8b1c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:20.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.752+0000 7f304df24b80 1 --2- 172.21.15.1:0/20377782 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5585b8b0a010 0x5585b8b14ca0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:20.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.752+0000 7f304df24b80 1 -- 172.21.15.1:0/20377782 >> 172.21.15.1:0/20377782 conn(0x5585b8b17d90 msgr2=0x5585b8b208a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:20.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.752+0000 7f304df24b80 1 -- 172.21.15.1:0/20377782 shutdown_connections 2024-02-02T15:27:20.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.752+0000 7f304df24b80 1 -- 172.21.15.1:0/20377782 wait complete. 2024-02-02T15:27:20.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.752+0000 7f304df24b80 1 -- start start 2024-02-02T15:27:20.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.752+0000 7f304df24b80 1 Processor -- start 2024-02-02T15:27:20.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.753+0000 7f304df24b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585b8a69db0 0x5585b8b1c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:20.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.753+0000 7f304df24b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5585b8b0a010 0x5585b8b186f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:20.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.753+0000 7f304df24b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5585b8b0a930 0x5585b8b18d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:20.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.753+0000 7f304df24b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585b8b19420 con 0x5585b8b0a010 2024-02-02T15:27:20.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.753+0000 7f304df24b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585b8b19590 con 0x5585b8b0a930 2024-02-02T15:27:20.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.753+0000 7f304df24b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585b8b76d00 con 0x5585b8a69db0 2024-02-02T15:27:20.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.753+0000 7f304df24b80 10 client.0 _ll_register_callbacks cb 0x5585b89f9290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:27:20.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.753+0000 7f3033400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585b8a69db0 0x5585b8b1c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:20.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.753+0000 7f3033400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585b8a69db0 0x5585b8b1c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:53716/0 (socket says 172.21.15.1:53716) 2024-02-02T15:27:20.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.753+0000 7f3033400700 1 -- 172.21.15.1:0/2147694569 learned_addr learned my addr 172.21.15.1:0/2147694569 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:20.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.753+0000 7f3033e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5585b8b0a930 0x5585b8b18d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:20.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.753+0000 7f3032a00700 1 --2- 172.21.15.1:0/2147694569 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5585b8b0a010 0x5585b8b186f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:20.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.753+0000 7f3033400700 1 -- 172.21.15.1:0/2147694569 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5585b8b0a930 msgr2=0x5585b8b18d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:20.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.753+0000 7f3033400700 1 --2- 172.21.15.1:0/2147694569 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5585b8b0a930 0x5585b8b18d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:20.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.753+0000 7f3033400700 1 -- 172.21.15.1:0/2147694569 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5585b8b0a010 msgr2=0x5585b8b186f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:20.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.753+0000 7f3033400700 1 --2- 172.21.15.1:0/2147694569 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5585b8b0a010 0x5585b8b186f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:20.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.753+0000 7f3033400700 1 -- 172.21.15.1:0/2147694569 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f303410e500 con 0x5585b8a69db0 2024-02-02T15:27:20.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.753+0000 7f3033400700 1 --2- 172.21.15.1:0/2147694569 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585b8a69db0 0x5585b8b1c1f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3024000c00 tx=0x7f3024005b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5271be1c0952a47 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:20.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.753+0000 7f3021600700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:27:20.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.754+0000 7f3021600700 1 -- 172.21.15.1:0/2147694569 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30240c9750 con 0x5585b8a69db0 2024-02-02T15:27:20.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.754+0000 7f3021600700 1 -- 172.21.15.1:0/2147694569 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f3024064080 con 0x5585b8a69db0 2024-02-02T15:27:20.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.754+0000 7f304df24b80 1 -- 172.21.15.1:0/2147694569 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f302c007bc0 con 0x5585b8a69db0 2024-02-02T15:27:20.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.754+0000 7f3021600700 1 -- 172.21.15.1:0/2147694569 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f302409d170 con 0x5585b8a69db0 2024-02-02T15:27:20.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.754+0000 7f304df24b80 10 client.25693 fetch_fsmap learned FSMap version 27 2024-02-02T15:27:20.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.754+0000 7f304df24b80 1 -- 172.21.15.1:0/2147694569 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f301c01f500 con 0x5585b8a69db0 2024-02-02T15:27:20.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.754+0000 7f3021600700 1 -- 172.21.15.1:0/2147694569 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3024080070 con 0x5585b8a69db0 2024-02-02T15:27:20.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.754+0000 7f3021600700 1 -- 172.21.15.1:0/2147694569 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3024098a20 con 0x5585b8a69db0 2024-02-02T15:27:20.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.754+0000 7f304df24b80 10 client.25693 fetch_fsmap finished waiting for FSMap version 27 2024-02-02T15:27:20.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.754+0000 7f304df24b80 10 client.25693 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:27:20.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.754+0000 7f304df24b80 20 client.25693 populate_metadata read hostname 'smithi001' 2024-02-02T15:27:20.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.754+0000 7f304df24b80 1 --2- 172.21.15.1:0/2147694569 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5585b8b157e0 0x5585b8b7a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:20.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.754+0000 7f304df24b80 4 client.25693 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2024-02-02T15:27:20.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.754+0000 7f304df24b80 1 -- 172.21.15.1:0/2147694569 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5585b8a1af50 con 0x5585b8b157e0 2024-02-02T15:27:20.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.755+0000 7f3032a00700 1 --2- 172.21.15.1:0/2147694569 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5585b8b157e0 0x5585b8b7a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:20.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.755+0000 7f3032a00700 1 --2- 172.21.15.1:0/2147694569 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5585b8b157e0 0x5585b8b7a050 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=706690f279d3b923 server_cookie=75148f4c5d4751f4 in_seq=0 out_seq=0 2024-02-02T15:27:20.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.755+0000 7f3021600700 10 client.25693 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:27:20.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.755+0000 7f3021600700 1 -- 172.21.15.1:0/2147694569 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5585b8a1af50 con 0x5585b8b157e0 2024-02-02T15:27:20.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.755+0000 7f3021600700 10 client.25693 handle_command_reply: tid=0 2024-02-02T15:27:20.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.755+0000 7f304df24b80 1 client.25693 shutdown 2024-02-02T15:27:20.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.755+0000 7f304df24b80 1 -- 172.21.15.1:0/2147694569 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585b8a69db0 msgr2=0x5585b8b1c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:20.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.755+0000 7f304df24b80 1 --2- 172.21.15.1:0/2147694569 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585b8a69db0 0x5585b8b1c1f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3024000c00 tx=0x7f3024005b40 comp rx=0 tx=0).stop 2024-02-02T15:27:20.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.756+0000 7f304df24b80 1 -- 172.21.15.1:0/2147694569 shutdown_connections 2024-02-02T15:27:20.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.756+0000 7f304df24b80 1 --2- 172.21.15.1:0/2147694569 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5585b8b157e0 0x5585b8b7a050 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:20.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.756+0000 7f304df24b80 1 --2- 172.21.15.1:0/2147694569 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5585b8b0a930 0x5585b8b18d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:20.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.756+0000 7f304df24b80 1 --2- 172.21.15.1:0/2147694569 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585b8a69db0 0x5585b8b1c1f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:20.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.756+0000 7f3021600700 0 client.25693 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:27:20.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.756+0000 7f304df24b80 1 --2- 172.21.15.1:0/2147694569 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5585b8b0a010 0x5585b8b186f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:20.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.756+0000 7f304df24b80 1 -- 172.21.15.1:0/2147694569 >> 172.21.15.1:0/2147694569 conn(0x5585b8b17d90 msgr2=0x5585b8b11c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:20.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.756+0000 7f304df24b80 1 -- 172.21.15.1:0/2147694569 shutdown_connections 2024-02-02T15:27:20.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.756+0000 7f304df24b80 1 -- 172.21.15.1:0/2147694569 wait complete. 2024-02-02T15:27:20.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.756+0000 7f304df24b80 20 client.25693 trim_cache size 0 max 16384 2024-02-02T15:27:20.764 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:27:20.764 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2024-02-02T15:27:20.764 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2024-02-02T15:27:20.764 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:27:20.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.756+0000 7f303a400700 1 -- 172.21.15.1:0/2379111865 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f30140a0720 msgr2=0x7f30140a2be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:20.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.756+0000 7f303a400700 1 --2- 172.21.15.1:0/2379111865 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f30140a0720 0x7f30140a2be0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f303419a7d0 tx=0x7f301c009f80 comp rx=0 tx=0).stop 2024-02-02T15:27:20.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.756+0000 7f303a400700 1 -- 172.21.15.1:0/2379111865 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3034100ff0 msgr2=0x7f30341991c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:20.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.756+0000 7f303a400700 1 --2- 172.21.15.1:0/2379111865 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3034100ff0 0x7f30341991c0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f302400e9a0 tx=0x7f302400ecb0 comp rx=0 tx=0).stop 2024-02-02T15:27:20.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.756+0000 7f303a400700 1 -- 172.21.15.1:0/2379111865 shutdown_connections 2024-02-02T15:27:20.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.756+0000 7f303a400700 1 --2- 172.21.15.1:0/2379111865 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f30140a0720 0x7f30140a2be0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:20.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.756+0000 7f303a400700 1 --2- 172.21.15.1:0/2379111865 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f303419db60 0x7f303419dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:20.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.756+0000 7f303a400700 1 --2- 172.21.15.1:0/2379111865 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3034100ff0 0x7f30341991c0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:20.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.756+0000 7f303a400700 1 --2- 172.21.15.1:0/2379111865 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3034106110 0x7f3034199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:20.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.756+0000 7f303a400700 1 -- 172.21.15.1:0/2379111865 >> 172.21.15.1:0/2379111865 conn(0x7f30340fc810 msgr2=0x7f30340fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:20.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.757+0000 7f303a400700 1 -- 172.21.15.1:0/2379111865 shutdown_connections 2024-02-02T15:27:20.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.757+0000 7f303a400700 1 -- 172.21.15.1:0/2379111865 wait complete. 2024-02-02T15:27:20.774 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-02T15:27:20.775 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T15:27:20.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.974+0000 7fc109e00700 1 -- 172.21.15.1:0/3308432326 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc104106e20 msgr2=0x7fc104107200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:20.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.974+0000 7fc109e00700 1 --2- 172.21.15.1:0/3308432326 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc104106e20 0x7fc104107200 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc0f40098f0 tx=0x7fc0f4009c00 comp rx=0 tx=0).stop 2024-02-02T15:27:20.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.974+0000 7fc109e00700 1 -- 172.21.15.1:0/3308432326 shutdown_connections 2024-02-02T15:27:20.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.974+0000 7fc109e00700 1 --2- 172.21.15.1:0/3308432326 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc104068a30 0x7fc10410dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:20.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.974+0000 7fc109e00700 1 --2- 172.21.15.1:0/3308432326 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc104106e20 0x7fc104107200 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:20.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.974+0000 7fc109e00700 1 --2- 172.21.15.1:0/3308432326 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc104068070 0x7fc1040684f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:20.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.974+0000 7fc109e00700 1 -- 172.21.15.1:0/3308432326 >> 172.21.15.1:0/3308432326 conn(0x7fc1040769c0 msgr2=0x7fc104076dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:20.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.974+0000 7fc109e00700 1 -- 172.21.15.1:0/3308432326 shutdown_connections 2024-02-02T15:27:20.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.975+0000 7fc109e00700 1 -- 172.21.15.1:0/3308432326 wait complete. 2024-02-02T15:27:20.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.975+0000 7fc109e00700 1 Processor -- start 2024-02-02T15:27:20.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.976+0000 7fc109e00700 1 -- start start 2024-02-02T15:27:20.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.976+0000 7fc109e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc104068070 0x7fc1040ffc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:20.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.976+0000 7fc109e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc104068a30 0x7fc104100170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:20.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.976+0000 7fc109e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc104106e20 0x7fc1041a4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:20.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.976+0000 7fc109e00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1041009a0 con 0x7fc104068070 2024-02-02T15:27:20.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.976+0000 7fc109e00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1041a5460 con 0x7fc104106e20 2024-02-02T15:27:20.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.976+0000 7fc109e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1041a55a0 con 0x7fc104068a30 2024-02-02T15:27:20.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.976+0000 7fc102a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc104068070 0x7fc1040ffc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:20.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.976+0000 7fc102a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc104068070 0x7fc1040ffc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43512/0 (socket says 172.21.15.1:43512) 2024-02-02T15:27:20.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.976+0000 7fc102a00700 1 -- 172.21.15.1:0/2685187689 learned_addr learned my addr 172.21.15.1:0/2685187689 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:20.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.977+0000 7fc103400700 1 --2- 172.21.15.1:0/2685187689 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc104106e20 0x7fc1041a4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:20.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.977+0000 7fc102000700 1 --2- 172.21.15.1:0/2685187689 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc104068a30 0x7fc104100170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:20.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.977+0000 7fc102a00700 1 -- 172.21.15.1:0/2685187689 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc104068a30 msgr2=0x7fc104100170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:20.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.977+0000 7fc102a00700 1 --2- 172.21.15.1:0/2685187689 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc104068a30 0x7fc104100170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:20.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.977+0000 7fc102a00700 1 -- 172.21.15.1:0/2685187689 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc104106e20 msgr2=0x7fc1041a4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:20.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.977+0000 7fc102a00700 1 --2- 172.21.15.1:0/2685187689 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc104106e20 0x7fc1041a4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:20.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.977+0000 7fc102a00700 1 -- 172.21.15.1:0/2685187689 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0f4009520 con 0x7fc104068070 2024-02-02T15:27:20.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.977+0000 7fc103400700 1 --2- 172.21.15.1:0/2685187689 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc104106e20 0x7fc1041a4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:27:20.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.977+0000 7fc102a00700 1 --2- 172.21.15.1:0/2685187689 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc104068070 0x7fc1040ffc30 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fc0f4009230 tx=0x7fc0f400b560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:20.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.977+0000 7fc108a00700 1 -- 172.21.15.1:0/2685187689 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f401c070 con 0x7fc104068070 2024-02-02T15:27:20.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.977+0000 7fc109e00700 1 -- 172.21.15.1:0/2685187689 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1041a57a0 con 0x7fc104068070 2024-02-02T15:27:20.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.977+0000 7fc108a00700 1 -- 172.21.15.1:0/2685187689 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc0f40058e0 con 0x7fc104068070 2024-02-02T15:27:20.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.977+0000 7fc108a00700 1 -- 172.21.15.1:0/2685187689 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f4003970 con 0x7fc104068070 2024-02-02T15:27:20.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.977+0000 7fc109e00700 1 -- 172.21.15.1:0/2685187689 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1041a5c70 con 0x7fc104068070 2024-02-02T15:27:20.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.978+0000 7fc109e00700 1 -- 172.21.15.1:0/2685187689 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc104067bc0 con 0x7fc104068070 2024-02-02T15:27:20.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.980+0000 7fc108a00700 1 -- 172.21.15.1:0/2685187689 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135245+0+0 (secure 0 0 0) 0x7fc0f4020030 con 0x7fc104068070 2024-02-02T15:27:20.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.981+0000 7fc108a00700 1 --2- 172.21.15.1:0/2685187689 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fc0e40a0720 0x7fc0e40a2be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:20.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.982+0000 7fc108a00700 1 -- 172.21.15.1:0/2685187689 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fc0f40cd3f0 con 0x7fc104068070 2024-02-02T15:27:20.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.982+0000 7fc102000700 1 --2- 172.21.15.1:0/2685187689 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fc0e40a0720 0x7fc0e40a2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:20.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.982+0000 7fc102000700 1 --2- 172.21.15.1:0/2685187689 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fc0e40a0720 0x7fc0e40a2be0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc104101420 tx=0x7fc0ec009260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:20.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:20.986+0000 7fc108a00700 1 -- 172.21.15.1:0/2685187689 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7fc0f40967c0 con 0x7fc104068070 2024-02-02T15:27:21.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.246+0000 7fc109e00700 1 -- 172.21.15.1:0/2685187689 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc104101160 con 0x7fc104068070 2024-02-02T15:27:21.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.249+0000 7fc108a00700 1 -- 172.21.15.1:0/2685187689 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10866 (secure 0 0 0) 0x7fc0f4099de0 con 0x7fc104068070 2024-02-02T15:27:21.249 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:27:21.249 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2922240448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2922240448},{"type":"v1","addr":"172.21.15.1:6835","nonce":2922240448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2729497431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2729497431},{"type":"v1","addr":"172.21.15.1:6837","nonce":2729497431}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-02T15:24:47.761435+0000","modified":"2024-02-02T15:25:20.251601+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24529,"mds_2":24518,"mds_3":24514,"mds_4":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14517":{"gid":14517,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/1218303683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":1218303683},{"type":"v1","addr":"172.21.15.1:6839","nonce":1218303683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14529":{"gid":14529,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/2059197496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2059197496},{"type":"v1","addr":"172.21.15.1:6841","nonce":2059197496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/7648986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":7648986},{"type":"v1","addr":"172.21.15.142:6833","nonce":7648986}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/3482746094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3482746094},{"type":"v1","addr":"172.21.15.71:6833","nonce":3482746094}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/388207814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":388207814},{"type":"v1","addr":"172.21.15.71:6835","nonce":388207814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6835/1052606547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1052606547},{"type":"v1","addr":"172.21.15.142:6835","nonce":1052606547}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.142:6837/2870289019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2870289019},{"type":"v1","addr":"172.21.15.142:6837","nonce":2870289019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.71:6837/628016199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":628016199},{"type":"v1","addr":"172.21.15.71:6837","nonce":628016199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.142:6839/321675421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":321675421},{"type":"v1","addr":"172.21.15.142:6839","nonce":321675421}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/1268086493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1268086493},{"type":"v1","addr":"172.21.15.71:6839","nonce":1268086493}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-02T15:27:21.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.252+0000 7fc109e00700 1 -- 172.21.15.1:0/2685187689 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fc0e40a0720 msgr2=0x7fc0e40a2be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:21.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.252+0000 7fc109e00700 1 --2- 172.21.15.1:0/2685187689 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fc0e40a0720 0x7fc0e40a2be0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc104101420 tx=0x7fc0ec009260 comp rx=0 tx=0).stop 2024-02-02T15:27:21.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.252+0000 7fc109e00700 1 -- 172.21.15.1:0/2685187689 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc104068070 msgr2=0x7fc1040ffc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:21.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.252+0000 7fc109e00700 1 --2- 172.21.15.1:0/2685187689 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc104068070 0x7fc1040ffc30 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fc0f4009230 tx=0x7fc0f400b560 comp rx=0 tx=0).stop 2024-02-02T15:27:21.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.252+0000 7fc109e00700 1 -- 172.21.15.1:0/2685187689 shutdown_connections 2024-02-02T15:27:21.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.252+0000 7fc109e00700 1 --2- 172.21.15.1:0/2685187689 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fc0e40a0720 0x7fc0e40a2be0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.252+0000 7fc109e00700 1 --2- 172.21.15.1:0/2685187689 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc104106e20 0x7fc1041a4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.252+0000 7fc109e00700 1 --2- 172.21.15.1:0/2685187689 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc104068070 0x7fc1040ffc30 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.252+0000 7fc109e00700 1 --2- 172.21.15.1:0/2685187689 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc104068a30 0x7fc104100170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.252+0000 7fc109e00700 1 -- 172.21.15.1:0/2685187689 >> 172.21.15.1:0/2685187689 conn(0x7fc1040769c0 msgr2=0x7fc1040fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:21.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.252+0000 7fc109e00700 1 -- 172.21.15.1:0/2685187689 shutdown_connections 2024-02-02T15:27:21.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.252+0000 7fc109e00700 1 -- 172.21.15.1:0/2685187689 wait complete. 2024-02-02T15:27:21.253 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 27 2024-02-02T15:27:21.272 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-02-02T15:27:21.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.474+0000 7f9c64600700 1 -- 172.21.15.1:0/125785370 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9c5c100ff0 msgr2=0x7f9c5c1013d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:21.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.474+0000 7f9c64600700 1 --2- 172.21.15.1:0/125785370 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9c5c100ff0 0x7f9c5c1013d0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c009990 tx=0x7f9c4c009ca0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.474+0000 7f9c64600700 1 -- 172.21.15.1:0/125785370 shutdown_connections 2024-02-02T15:27:21.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.474+0000 7f9c64600700 1 --2- 172.21.15.1:0/125785370 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9c5c106110 0x7f9c5c10dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.474+0000 7f9c64600700 1 --2- 172.21.15.1:0/125785370 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9c5c100ff0 0x7f9c5c1013d0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.474+0000 7f9c64600700 1 --2- 172.21.15.1:0/125785370 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9c5c1019a0 0x7f9c5c1059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.474+0000 7f9c64600700 1 -- 172.21.15.1:0/125785370 >> 172.21.15.1:0/125785370 conn(0x7f9c5c0fc810 msgr2=0x7f9c5c0fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:21.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.474+0000 7f9c64600700 1 -- 172.21.15.1:0/125785370 shutdown_connections 2024-02-02T15:27:21.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.474+0000 7f9c64600700 1 -- 172.21.15.1:0/125785370 wait complete. 2024-02-02T15:27:21.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.475+0000 7f9c64600700 1 Processor -- start 2024-02-02T15:27:21.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.475+0000 7f9c64600700 1 -- start start 2024-02-02T15:27:21.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.475+0000 7f9c64600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9c5c100ff0 0x7f9c5c196ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:21.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.475+0000 7f9c64600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9c5c106110 0x7f9c5c197410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:21.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.475+0000 7f9c64600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9c5c19b870 0x7f9c5c19bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:21.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.475+0000 7f9c64600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5c19c2f0 con 0x7f9c5c106110 2024-02-02T15:27:21.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.475+0000 7f9c64600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5c19c460 con 0x7f9c5c19b870 2024-02-02T15:27:21.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.475+0000 7f9c64600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5c19c620 con 0x7f9c5c100ff0 2024-02-02T15:27:21.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.476+0000 7f9c61200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9c5c106110 0x7f9c5c197410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:21.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.476+0000 7f9c61200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9c5c106110 0x7f9c5c197410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43530/0 (socket says 172.21.15.1:43530) 2024-02-02T15:27:21.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.476+0000 7f9c61c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9c5c100ff0 0x7f9c5c196ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:21.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.476+0000 7f9c61200700 1 -- 172.21.15.1:0/1057935357 learned_addr learned my addr 172.21.15.1:0/1057935357 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:21.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.476+0000 7f9c62600700 1 --2- 172.21.15.1:0/1057935357 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9c5c19b870 0x7f9c5c19bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:21.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.476+0000 7f9c61200700 1 -- 172.21.15.1:0/1057935357 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9c5c100ff0 msgr2=0x7f9c5c196ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:21.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.476+0000 7f9c61200700 1 --2- 172.21.15.1:0/1057935357 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9c5c100ff0 0x7f9c5c196ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.476+0000 7f9c61200700 1 -- 172.21.15.1:0/1057935357 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9c5c19b870 msgr2=0x7f9c5c19bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:21.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.476+0000 7f9c61200700 1 --2- 172.21.15.1:0/1057935357 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9c5c19b870 0x7f9c5c19bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.476+0000 7f9c61200700 1 -- 172.21.15.1:0/1057935357 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c4c0095f0 con 0x7f9c5c106110 2024-02-02T15:27:21.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.476+0000 7f9c62600700 1 --2- 172.21.15.1:0/1057935357 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9c5c19b870 0x7f9c5c19bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:27:21.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.476+0000 7f9c61200700 1 --2- 172.21.15.1:0/1057935357 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9c5c106110 0x7f9c5c197410 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f9c4400e970 tx=0x7f9c4400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:21.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.477+0000 7f9c5a000700 1 -- 172.21.15.1:0/1057935357 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c44009790 con 0x7f9c5c106110 2024-02-02T15:27:21.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.477+0000 7f9c64600700 1 -- 172.21.15.1:0/1057935357 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c5c072fd0 con 0x7f9c5c106110 2024-02-02T15:27:21.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.477+0000 7f9c5a000700 1 -- 172.21.15.1:0/1057935357 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c4400eec0 con 0x7f9c5c106110 2024-02-02T15:27:21.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.477+0000 7f9c5a000700 1 -- 172.21.15.1:0/1057935357 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c44005590 con 0x7f9c5c106110 2024-02-02T15:27:21.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.477+0000 7f9c64600700 1 -- 172.21.15.1:0/1057935357 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c5c073530 con 0x7f9c5c106110 2024-02-02T15:27:21.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.480+0000 7f9c5a000700 1 -- 172.21.15.1:0/1057935357 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135245+0+0 (secure 0 0 0) 0x7f9c4400cbd0 con 0x7f9c5c106110 2024-02-02T15:27:21.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.481+0000 7f9c780a3b80 1 Processor -- start 2024-02-02T15:27:21.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.481+0000 7f9c780a3b80 1 -- start start 2024-02-02T15:27:21.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.481+0000 7f9c780a3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561fff7ad010 0x561fff7ad3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:21.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.481+0000 7f9c780a3b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561fff7ad930 0x561fff7b82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:21.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.482+0000 7f9c780a3b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561fff7b87e0 0x561fff7babd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:21.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.482+0000 7f9c5a000700 1 --2- 172.21.15.1:0/1057935357 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f9c3c0a07a0 0x7f9c3c0a2c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:21.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.482+0000 7f9c780a3b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fff7214c0 con 0x561fff7ad930 2024-02-02T15:27:21.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.482+0000 7f9c780a3b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fff705aa0 con 0x561fff7b87e0 2024-02-02T15:27:21.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.482+0000 7f9c780a3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fff703460 con 0x561fff7ad010 2024-02-02T15:27:21.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.482+0000 7f9c5a000700 1 -- 172.21.15.1:0/1057935357 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f9c44014070 con 0x7f9c5c106110 2024-02-02T15:27:21.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.482+0000 7f9c62600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561fff7ad010 0x561fff7ad3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:21.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.482+0000 7f9c61c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561fff7ad930 0x561fff7b82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:21.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.482+0000 7f9c62600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561fff7ad010 0x561fff7ad3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:53748/0 (socket says 172.21.15.1:53748) 2024-02-02T15:27:21.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.482+0000 7f9c62600700 1 -- 172.21.15.1:0/1814157343 learned_addr learned my addr 172.21.15.1:0/1814157343 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:21.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.482+0000 7f9c62600700 1 --2- 172.21.15.1:0/1057935357 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f9c3c0a07a0 0x7f9c3c0a2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:21.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.482+0000 7f9c61200700 1 --2- 172.21.15.1:0/1814157343 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561fff7b87e0 0x561fff7babd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:21.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.482+0000 7f9c61c00700 1 -- 172.21.15.1:0/1814157343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561fff7ad010 msgr2=0x561fff7ad3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:21.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.482+0000 7f9c61c00700 1 --2- 172.21.15.1:0/1814157343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561fff7ad010 0x561fff7ad3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.482+0000 7f9c61c00700 1 -- 172.21.15.1:0/1814157343 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561fff7b87e0 msgr2=0x561fff7babd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:21.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.482+0000 7f9c61c00700 1 --2- 172.21.15.1:0/1814157343 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561fff7b87e0 0x561fff7babd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.482+0000 7f9c61c00700 1 -- 172.21.15.1:0/1814157343 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561fff5803b0 con 0x561fff7ad930 2024-02-02T15:27:21.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.483+0000 7f9c61c00700 1 --2- 172.21.15.1:0/1814157343 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561fff7ad930 0x561fff7b82a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c000c00 tx=0x7f9c4c00b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f3092f49f3fa488 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:21.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.483+0000 7f9c62600700 1 --2- 172.21.15.1:0/1057935357 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f9c3c0a07a0 0x7f9c3c0a2c60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9c54007ce0 tx=0x7f9c54006040 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:21.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.483+0000 7f9c52a00700 1 -- 172.21.15.1:0/1814157343 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c01d070 con 0x561fff7ad930 2024-02-02T15:27:21.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.483+0000 7f9c52a00700 1 -- 172.21.15.1:0/1814157343 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c4c01e040 con 0x561fff7ad930 2024-02-02T15:27:21.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.483+0000 7f9c52a00700 1 -- 172.21.15.1:0/1814157343 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c004820 con 0x561fff7ad930 2024-02-02T15:27:21.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.484+0000 7f9c780a3b80 1 -- 172.21.15.1:0/1814157343 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561fff7ad930 msgr2=0x561fff7b82a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:21.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.484+0000 7f9c780a3b80 1 --2- 172.21.15.1:0/1814157343 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561fff7ad930 0x561fff7b82a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c000c00 tx=0x7f9c4c00b5b0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.484+0000 7f9c780a3b80 1 -- 172.21.15.1:0/1814157343 shutdown_connections 2024-02-02T15:27:21.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.484+0000 7f9c780a3b80 1 --2- 172.21.15.1:0/1814157343 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561fff7b87e0 0x561fff7babd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.484+0000 7f9c780a3b80 1 --2- 172.21.15.1:0/1814157343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561fff7ad010 0x561fff7ad3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.484+0000 7f9c780a3b80 1 --2- 172.21.15.1:0/1814157343 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561fff7ad930 0x561fff7b82a0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.484+0000 7f9c780a3b80 1 -- 172.21.15.1:0/1814157343 >> 172.21.15.1:0/1814157343 conn(0x561fff70cdb0 msgr2=0x561fff7a93b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:21.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.484+0000 7f9c780a3b80 1 -- 172.21.15.1:0/1814157343 shutdown_connections 2024-02-02T15:27:21.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.484+0000 7f9c780a3b80 1 -- 172.21.15.1:0/1814157343 wait complete. 2024-02-02T15:27:21.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.485+0000 7f9c780a3b80 1 -- start start 2024-02-02T15:27:21.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.485+0000 7f9c780a3b80 1 Processor -- start 2024-02-02T15:27:21.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.486+0000 7f9c780a3b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561fff7ad010 0x561fff7b8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:21.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.486+0000 7f9c780a3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561fff7ad930 0x561fff7b53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:21.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.486+0000 7f9c780a3b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561fff7b87e0 0x561fff7b5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:21.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.486+0000 7f9c780a3b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fff6fed50 con 0x561fff7b87e0 2024-02-02T15:27:21.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.486+0000 7f9c780a3b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fff69ad80 con 0x561fff7ad010 2024-02-02T15:27:21.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.486+0000 7f9c780a3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fff69aa20 con 0x561fff7ad930 2024-02-02T15:27:21.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.486+0000 7f9c780a3b80 10 client.0 _ll_register_callbacks cb 0x561fff69c290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:27:21.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.486+0000 7f9c62600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561fff7b87e0 0x561fff7b5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:21.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.486+0000 7f9c61c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561fff7ad010 0x561fff7b8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:21.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.486+0000 7f9c61c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561fff7ad010 0x561fff7b8100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59376/0 (socket says 172.21.15.1:59376) 2024-02-02T15:27:21.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.486+0000 7f9c61c00700 1 -- 172.21.15.1:0/3855971471 learned_addr learned my addr 172.21.15.1:0/3855971471 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:21.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.486+0000 7f9c61200700 1 --2- 172.21.15.1:0/3855971471 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561fff7ad930 0x561fff7b53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:21.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.486+0000 7f9c61c00700 1 -- 172.21.15.1:0/3855971471 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561fff7ad930 msgr2=0x561fff7b53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:21.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.486+0000 7f9c61c00700 1 --2- 172.21.15.1:0/3855971471 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561fff7ad930 0x561fff7b53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.486+0000 7f9c61c00700 1 -- 172.21.15.1:0/3855971471 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561fff7b87e0 msgr2=0x561fff7b5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:21.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.486+0000 7f9c61c00700 1 --2- 172.21.15.1:0/3855971471 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561fff7b87e0 0x561fff7b5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.486+0000 7f9c61c00700 1 -- 172.21.15.1:0/3855971471 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c5c10e500 con 0x561fff7ad010 2024-02-02T15:27:21.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.486+0000 7f9c62600700 1 --2- 172.21.15.1:0/3855971471 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561fff7b87e0 0x561fff7b5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:27:21.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.487+0000 7f9c61c00700 1 --2- 172.21.15.1:0/3855971471 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561fff7ad010 0x561fff7b8100 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c00b2e0 tx=0x7f9c4c00bfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dac00427fda51c6f server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:21.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.487+0000 7f9c4b400700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:27:21.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.487+0000 7f9c4b400700 1 -- 172.21.15.1:0/3855971471 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c01d030 con 0x561fff7ad010 2024-02-02T15:27:21.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.487+0000 7f9c780a3b80 1 -- 172.21.15.1:0/3855971471 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561fff7a10f0 con 0x561fff7ad010 2024-02-02T15:27:21.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.487+0000 7f9c4b400700 1 -- 172.21.15.1:0/3855971471 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f9c4c005980 con 0x561fff7ad010 2024-02-02T15:27:21.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.487+0000 7f9c4b400700 1 -- 172.21.15.1:0/3855971471 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9c4c00bd10 con 0x561fff7ad010 2024-02-02T15:27:21.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.487+0000 7f9c780a3b80 10 client.25712 fetch_fsmap learned FSMap version 27 2024-02-02T15:27:21.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.487+0000 7f9c780a3b80 1 -- 172.21.15.1:0/3855971471 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x561fff52cb60 con 0x561fff7ad010 2024-02-02T15:27:21.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.488+0000 7f9c4b400700 1 -- 172.21.15.1:0/3855971471 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561fff7a10f0 con 0x561fff7ad010 2024-02-02T15:27:21.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.488+0000 7f9c4b400700 1 -- 172.21.15.1:0/3855971471 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9c4c00f990 con 0x561fff7ad010 2024-02-02T15:27:21.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.488+0000 7f9c780a3b80 10 client.25712 fetch_fsmap finished waiting for FSMap version 27 2024-02-02T15:27:21.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.488+0000 7f9c780a3b80 10 client.25712 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-02-02T15:27:21.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.488+0000 7f9c780a3b80 20 client.25712 populate_metadata read hostname 'smithi001' 2024-02-02T15:27:21.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.488+0000 7f9c780a3b80 1 --2- 172.21.15.1:0/3855971471 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x561fff7c7c50 0x561fff7ca040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:21.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.488+0000 7f9c780a3b80 4 client.25712 mds_command: new command op to 24502 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:27:21.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.488+0000 7f9c780a3b80 1 -- 172.21.15.1:0/3855971471 --> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561fff6f7dd0 con 0x561fff7c7c50 2024-02-02T15:27:21.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.488+0000 7f9c61200700 1 --2- 172.21.15.1:0/3855971471 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x561fff7c7c50 0x561fff7ca040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:21.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.489+0000 7f9c61200700 1 --2- 172.21.15.1:0/3855971471 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x561fff7c7c50 0x561fff7ca040 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e9bf750c0a40bb5f server_cookie=37b64f75e55d168e in_seq=0 out_seq=0 2024-02-02T15:27:21.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.489+0000 7f9c4b400700 10 client.25712 ms_handle_connect on v2:172.21.15.142:6834/1052606547 2024-02-02T15:27:21.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.489+0000 7f9c4b400700 1 -- 172.21.15.1:0/3855971471 <== mds.4 v2:172.21.15.142:6834/1052606547 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x561fff6f7dd0 con 0x561fff7c7c50 2024-02-02T15:27:21.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.489+0000 7f9c4b400700 10 client.25712 handle_command_reply: tid=0 2024-02-02T15:27:21.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.489+0000 7f9c780a3b80 1 client.25712 shutdown 2024-02-02T15:27:21.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.490+0000 7f9c780a3b80 1 -- 172.21.15.1:0/3855971471 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561fff7ad010 msgr2=0x561fff7b8100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:21.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.490+0000 7f9c780a3b80 1 --2- 172.21.15.1:0/3855971471 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561fff7ad010 0x561fff7b8100 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c00b2e0 tx=0x7f9c4c00bfd0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.490+0000 7f9c780a3b80 1 -- 172.21.15.1:0/3855971471 shutdown_connections 2024-02-02T15:27:21.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.490+0000 7f9c780a3b80 1 --2- 172.21.15.1:0/3855971471 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x561fff7c7c50 0x561fff7ca040 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.490+0000 7f9c780a3b80 1 --2- 172.21.15.1:0/3855971471 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561fff7b87e0 0x561fff7b5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.490+0000 7f9c4b400700 0 client.25712 ms_handle_reset on v2:172.21.15.142:6834/1052606547 2024-02-02T15:27:21.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.490+0000 7f9c780a3b80 1 --2- 172.21.15.1:0/3855971471 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561fff7ad010 0x561fff7b8100 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.490+0000 7f9c780a3b80 1 --2- 172.21.15.1:0/3855971471 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561fff7ad930 0x561fff7b53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.490+0000 7f9c780a3b80 1 -- 172.21.15.1:0/3855971471 >> 172.21.15.1:0/3855971471 conn(0x561fff70cdb0 msgr2=0x561fff7a8fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:21.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.490+0000 7f9c780a3b80 1 -- 172.21.15.1:0/3855971471 shutdown_connections 2024-02-02T15:27:21.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.490+0000 7f9c780a3b80 1 -- 172.21.15.1:0/3855971471 wait complete. 2024-02-02T15:27:21.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.490+0000 7f9c780a3b80 20 client.25712 trim_cache size 0 max 16384 2024-02-02T15:27:21.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.503+0000 7f9c780a3b80 1 Processor -- start 2024-02-02T15:27:21.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.503+0000 7f9c780a3b80 1 -- start start 2024-02-02T15:27:21.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.504+0000 7f9c780a3b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561fff7ad010 0x561fff7bf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:21.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.504+0000 7f9c780a3b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561fff7ad930 0x561fff7b7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:21.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.504+0000 7f9c780a3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561fff7b87e0 0x561fff7b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:21.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.504+0000 7f9c780a3b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fff5826a0 con 0x561fff7ad930 2024-02-02T15:27:21.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.504+0000 7f9c780a3b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fff7adf10 con 0x561fff7ad010 2024-02-02T15:27:21.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.504+0000 7f9c780a3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fff7b6960 con 0x561fff7b87e0 2024-02-02T15:27:21.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.504+0000 7f9c61200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561fff7ad930 0x561fff7b7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:21.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.504+0000 7f9c61200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561fff7ad930 0x561fff7b7ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43552/0 (socket says 172.21.15.1:43552) 2024-02-02T15:27:21.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.504+0000 7f9c61200700 1 -- 172.21.15.1:0/1401334784 learned_addr learned my addr 172.21.15.1:0/1401334784 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:21.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.504+0000 7f9c61c00700 1 --2- 172.21.15.1:0/1401334784 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561fff7ad010 0x561fff7bf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:21.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.504+0000 7f9c62600700 1 --2- 172.21.15.1:0/1401334784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561fff7b87e0 0x561fff7b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:21.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.504+0000 7f9c61200700 1 -- 172.21.15.1:0/1401334784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561fff7b87e0 msgr2=0x561fff7b4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:21.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.504+0000 7f9c61200700 1 --2- 172.21.15.1:0/1401334784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561fff7b87e0 0x561fff7b4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.504+0000 7f9c61200700 1 -- 172.21.15.1:0/1401334784 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561fff7ad010 msgr2=0x561fff7bf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:21.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.504+0000 7f9c61200700 1 --2- 172.21.15.1:0/1401334784 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561fff7ad010 0x561fff7bf2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.504+0000 7f9c61200700 1 -- 172.21.15.1:0/1401334784 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561fff6c4ea0 con 0x561fff7ad930 2024-02-02T15:27:21.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.504+0000 7f9c61200700 1 --2- 172.21.15.1:0/1401334784 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561fff7ad930 0x561fff7b7ca0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f9c44018000 tx=0x7f9c4400ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82dbc2c4c8332c12 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:21.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.505+0000 7f9c53400700 1 -- 172.21.15.1:0/1401334784 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c4409d2e0 con 0x561fff7ad930 2024-02-02T15:27:21.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.505+0000 7f9c53400700 1 -- 172.21.15.1:0/1401334784 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c440a1520 con 0x561fff7ad930 2024-02-02T15:27:21.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.505+0000 7f9c53400700 1 -- 172.21.15.1:0/1401334784 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c44099710 con 0x561fff7ad930 2024-02-02T15:27:21.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.505+0000 7f9c780a3b80 1 -- 172.21.15.1:0/1401334784 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561fff7ad930 msgr2=0x561fff7b7ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:21.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.505+0000 7f9c780a3b80 1 --2- 172.21.15.1:0/1401334784 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561fff7ad930 0x561fff7b7ca0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f9c44018000 tx=0x7f9c4400ced0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.505+0000 7f9c780a3b80 1 -- 172.21.15.1:0/1401334784 shutdown_connections 2024-02-02T15:27:21.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.505+0000 7f9c780a3b80 1 --2- 172.21.15.1:0/1401334784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561fff7b87e0 0x561fff7b4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.505+0000 7f9c780a3b80 1 --2- 172.21.15.1:0/1401334784 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561fff7ad010 0x561fff7bf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.505+0000 7f9c780a3b80 1 --2- 172.21.15.1:0/1401334784 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561fff7ad930 0x561fff7b7ca0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.505+0000 7f9c780a3b80 1 -- 172.21.15.1:0/1401334784 >> 172.21.15.1:0/1401334784 conn(0x561fff70cdb0 msgr2=0x561fff7c44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:21.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.505+0000 7f9c780a3b80 1 -- 172.21.15.1:0/1401334784 shutdown_connections 2024-02-02T15:27:21.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.505+0000 7f9c780a3b80 1 -- 172.21.15.1:0/1401334784 wait complete. 2024-02-02T15:27:21.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.506+0000 7f9c780a3b80 1 -- start start 2024-02-02T15:27:21.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.506+0000 7f9c780a3b80 1 Processor -- start 2024-02-02T15:27:21.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.506+0000 7f9c780a3b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561fff7ad010 0x561fff7beed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:21.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.506+0000 7f9c780a3b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561fff7ad930 0x561fff7bb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:21.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.506+0000 7f9c780a3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561fff7b87e0 0x561fff7bb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:21.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.506+0000 7f9c780a3b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fff7bbd20 con 0x561fff7ad010 2024-02-02T15:27:21.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.507+0000 7f9c780a3b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fff7bbe90 con 0x561fff7ad930 2024-02-02T15:27:21.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.507+0000 7f9c780a3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fff7bc050 con 0x561fff7b87e0 2024-02-02T15:27:21.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.507+0000 7f9c780a3b80 10 client.0 _ll_register_callbacks cb 0x561fff7aab40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:27:21.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.507+0000 7f9c61c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561fff7ad010 0x561fff7beed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:21.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.507+0000 7f9c61c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561fff7ad010 0x561fff7beed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43568/0 (socket says 172.21.15.1:43568) 2024-02-02T15:27:21.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.507+0000 7f9c61c00700 1 -- 172.21.15.1:0/1645732125 learned_addr learned my addr 172.21.15.1:0/1645732125 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:21.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.507+0000 7f9c61200700 1 --2- 172.21.15.1:0/1645732125 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561fff7ad930 0x561fff7bb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:21.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.507+0000 7f9c62600700 1 --2- 172.21.15.1:0/1645732125 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561fff7b87e0 0x561fff7bb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:21.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.507+0000 7f9c61c00700 1 -- 172.21.15.1:0/1645732125 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561fff7b87e0 msgr2=0x561fff7bb7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:21.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.507+0000 7f9c61c00700 1 --2- 172.21.15.1:0/1645732125 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561fff7b87e0 0x561fff7bb7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.507+0000 7f9c61c00700 1 -- 172.21.15.1:0/1645732125 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561fff7ad930 msgr2=0x561fff7bb2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:21.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.507+0000 7f9c61c00700 1 --2- 172.21.15.1:0/1645732125 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561fff7ad930 0x561fff7bb2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.507+0000 7f9c61c00700 1 -- 172.21.15.1:0/1645732125 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c5c073530 con 0x561fff7ad010 2024-02-02T15:27:21.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.507+0000 7f9c62600700 1 --2- 172.21.15.1:0/1645732125 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561fff7b87e0 0x561fff7bb7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:27:21.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.507+0000 7f9c61c00700 1 --2- 172.21.15.1:0/1645732125 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561fff7ad010 0x561fff7beed0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c009990 tx=0x7f9c4c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f13f521323513c16 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:21.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.507+0000 7f9c4b400700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:27:21.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.508+0000 7f9c4b400700 1 -- 172.21.15.1:0/1645732125 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c01d040 con 0x561fff7ad010 2024-02-02T15:27:21.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.508+0000 7f9c4b400700 1 -- 172.21.15.1:0/1645732125 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f9c4c000e40 con 0x561fff7ad010 2024-02-02T15:27:21.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.508+0000 7f9c780a3b80 1 -- 172.21.15.1:0/1645732125 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9c4c01e040 con 0x561fff7ad010 2024-02-02T15:27:21.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.508+0000 7f9c4b400700 1 -- 172.21.15.1:0/1645732125 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9c4c00f460 con 0x561fff7ad010 2024-02-02T15:27:21.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.508+0000 7f9c780a3b80 10 client.15681 fetch_fsmap learned FSMap version 27 2024-02-02T15:27:21.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.508+0000 7f9c780a3b80 1 -- 172.21.15.1:0/1645732125 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x561fff584cd0 con 0x561fff7ad010 2024-02-02T15:27:21.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.508+0000 7f9c4b400700 1 -- 172.21.15.1:0/1645732125 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c4c01e040 con 0x561fff7ad010 2024-02-02T15:27:21.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.508+0000 7f9c4b400700 1 -- 172.21.15.1:0/1645732125 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9c4c004c70 con 0x561fff7ad010 2024-02-02T15:27:21.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.508+0000 7f9c780a3b80 10 client.15681 fetch_fsmap finished waiting for FSMap version 27 2024-02-02T15:27:21.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.508+0000 7f9c780a3b80 10 client.15681 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-02-02T15:27:21.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.508+0000 7f9c780a3b80 20 client.15681 populate_metadata read hostname 'smithi001' 2024-02-02T15:27:21.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.508+0000 7f9c780a3b80 1 --2- 172.21.15.1:0/1645732125 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x561fff81a700 0x561fff81cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:21.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.508+0000 7f9c780a3b80 4 client.15681 mds_command: new command op to 24502 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:27:21.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.508+0000 7f9c780a3b80 1 -- 172.21.15.1:0/1645732125 --> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561fff6bdf50 con 0x561fff81a700 2024-02-02T15:27:21.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.509+0000 7f9c61200700 1 --2- 172.21.15.1:0/1645732125 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x561fff81a700 0x561fff81cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:21.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.509+0000 7f9c61200700 1 --2- 172.21.15.1:0/1645732125 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x561fff81a700 0x561fff81cb20 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=55a218adb37d8abc server_cookie=8f16fd55204a17e3 in_seq=0 out_seq=0 2024-02-02T15:27:21.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.509+0000 7f9c4b400700 10 client.15681 ms_handle_connect on v2:172.21.15.142:6834/1052606547 2024-02-02T15:27:21.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.509+0000 7f9c4b400700 1 -- 172.21.15.1:0/1645732125 <== mds.4 v2:172.21.15.142:6834/1052606547 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561fff6bdf50 con 0x561fff81a700 2024-02-02T15:27:21.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.509+0000 7f9c4b400700 10 client.15681 handle_command_reply: tid=0 2024-02-02T15:27:21.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.509+0000 7f9c780a3b80 1 client.15681 shutdown 2024-02-02T15:27:21.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.510+0000 7f9c780a3b80 1 -- 172.21.15.1:0/1645732125 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561fff7ad010 msgr2=0x561fff7beed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:21.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.510+0000 7f9c780a3b80 1 --2- 172.21.15.1:0/1645732125 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561fff7ad010 0x561fff7beed0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c009990 tx=0x7f9c4c000bc0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.510+0000 7f9c780a3b80 1 -- 172.21.15.1:0/1645732125 shutdown_connections 2024-02-02T15:27:21.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.510+0000 7f9c780a3b80 1 --2- 172.21.15.1:0/1645732125 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x561fff81a700 0x561fff81cb20 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.510+0000 7f9c780a3b80 1 --2- 172.21.15.1:0/1645732125 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561fff7b87e0 0x561fff7bb7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.510+0000 7f9c780a3b80 1 --2- 172.21.15.1:0/1645732125 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561fff7ad010 0x561fff7beed0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.510+0000 7f9c780a3b80 1 --2- 172.21.15.1:0/1645732125 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561fff7ad930 0x561fff7bb2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.510+0000 7f9c4b400700 0 client.15681 ms_handle_reset on v2:172.21.15.142:6834/1052606547 2024-02-02T15:27:21.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.510+0000 7f9c780a3b80 1 -- 172.21.15.1:0/1645732125 >> 172.21.15.1:0/1645732125 conn(0x561fff70cdb0 msgr2=0x561fff7a85c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:21.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.510+0000 7f9c780a3b80 1 -- 172.21.15.1:0/1645732125 shutdown_connections 2024-02-02T15:27:21.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.510+0000 7f9c780a3b80 1 -- 172.21.15.1:0/1645732125 wait complete. 2024-02-02T15:27:21.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.510+0000 7f9c780a3b80 20 client.15681 trim_cache size 0 max 16384 2024-02-02T15:27:21.516 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:27:21.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.510+0000 7f9c64600700 1 -- 172.21.15.1:0/1057935357 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f9c3c0a07a0 msgr2=0x7f9c3c0a2c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:21.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.511+0000 7f9c64600700 1 --2- 172.21.15.1:0/1057935357 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f9c3c0a07a0 0x7f9c3c0a2c60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9c54007ce0 tx=0x7f9c54006040 comp rx=0 tx=0).stop 2024-02-02T15:27:21.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.511+0000 7f9c64600700 1 -- 172.21.15.1:0/1057935357 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9c5c106110 msgr2=0x7f9c5c197410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:21.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.511+0000 7f9c64600700 1 --2- 172.21.15.1:0/1057935357 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9c5c106110 0x7f9c5c197410 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f9c4400e970 tx=0x7f9c4400ec80 comp rx=0 tx=0).stop 2024-02-02T15:27:21.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.511+0000 7f9c64600700 1 -- 172.21.15.1:0/1057935357 shutdown_connections 2024-02-02T15:27:21.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.511+0000 7f9c64600700 1 --2- 172.21.15.1:0/1057935357 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f9c3c0a07a0 0x7f9c3c0a2c60 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.511+0000 7f9c64600700 1 --2- 172.21.15.1:0/1057935357 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9c5c19b870 0x7f9c5c19bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.511+0000 7f9c64600700 1 --2- 172.21.15.1:0/1057935357 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9c5c100ff0 0x7f9c5c196ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.511+0000 7f9c64600700 1 --2- 172.21.15.1:0/1057935357 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9c5c106110 0x7f9c5c197410 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.511+0000 7f9c64600700 1 -- 172.21.15.1:0/1057935357 >> 172.21.15.1:0/1057935357 conn(0x7f9c5c0fc810 msgr2=0x7f9c5c0fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:21.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.511+0000 7f9c64600700 1 -- 172.21.15.1:0/1057935357 shutdown_connections 2024-02-02T15:27:21.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.511+0000 7f9c64600700 1 -- 172.21.15.1:0/1057935357 wait complete. 2024-02-02T15:27:21.527 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:21 smithi001 ceph-mon[35470]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-02-02T15:27:21.527 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:21 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/2685187689' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:27:21.529 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-02-02T15:27:21.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:21 smithi071 ceph-mon[44707]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-02-02T15:27:21.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:21 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/2685187689' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:27:21.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:21 smithi142 ceph-mon[45728]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-02-02T15:27:21.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:21 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/2685187689' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:27:21.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.731+0000 7ff358400700 1 -- 172.21.15.1:0/1427417402 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff350106110 msgr2=0x7ff35010dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:21.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.731+0000 7ff358400700 1 --2- 172.21.15.1:0/1427417402 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff350106110 0x7ff35010dc90 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ff34800b0a0 tx=0x7ff34800b3b0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.731+0000 7ff358400700 1 -- 172.21.15.1:0/1427417402 shutdown_connections 2024-02-02T15:27:21.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.731+0000 7ff358400700 1 --2- 172.21.15.1:0/1427417402 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff350106110 0x7ff35010dc90 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.731+0000 7ff358400700 1 --2- 172.21.15.1:0/1427417402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff350100ff0 0x7ff3501013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.731+0000 7ff358400700 1 --2- 172.21.15.1:0/1427417402 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff3501019a0 0x7ff3501059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.731+0000 7ff358400700 1 -- 172.21.15.1:0/1427417402 >> 172.21.15.1:0/1427417402 conn(0x7ff3500fc810 msgr2=0x7ff3500fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:21.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.731+0000 7ff358400700 1 -- 172.21.15.1:0/1427417402 shutdown_connections 2024-02-02T15:27:21.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.731+0000 7ff358400700 1 -- 172.21.15.1:0/1427417402 wait complete. 2024-02-02T15:27:21.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.732+0000 7ff358400700 1 Processor -- start 2024-02-02T15:27:21.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.732+0000 7ff358400700 1 -- start start 2024-02-02T15:27:21.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.732+0000 7ff358400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff350100ff0 0x7ff350073040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:21.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.732+0000 7ff358400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff350106110 0x7ff350073580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:21.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.733+0000 7ff358400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff350073ac0 0x7ff3501a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:21.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.733+0000 7ff358400700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3501a5880 con 0x7ff350073ac0 2024-02-02T15:27:21.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.733+0000 7ff358400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3501a59f0 con 0x7ff350100ff0 2024-02-02T15:27:21.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.733+0000 7ff358400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3501a5bb0 con 0x7ff350106110 2024-02-02T15:27:21.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.733+0000 7ff356400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff350073ac0 0x7ff3501a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:21.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.733+0000 7ff356400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff350073ac0 0x7ff3501a51f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43578/0 (socket says 172.21.15.1:43578) 2024-02-02T15:27:21.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.733+0000 7ff356400700 1 -- 172.21.15.1:0/2982756108 learned_addr learned my addr 172.21.15.1:0/2982756108 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:21.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.733+0000 7ff355a00700 1 --2- 172.21.15.1:0/2982756108 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff350100ff0 0x7ff350073040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:21.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.733+0000 7ff355000700 1 --2- 172.21.15.1:0/2982756108 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff350106110 0x7ff350073580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:21.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.733+0000 7ff356400700 1 -- 172.21.15.1:0/2982756108 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff350106110 msgr2=0x7ff350073580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:21.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.733+0000 7ff356400700 1 --2- 172.21.15.1:0/2982756108 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff350106110 0x7ff350073580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.733+0000 7ff356400700 1 -- 172.21.15.1:0/2982756108 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff350100ff0 msgr2=0x7ff350073040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:21.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.733+0000 7ff356400700 1 --2- 172.21.15.1:0/2982756108 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff350100ff0 0x7ff350073040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.733+0000 7ff356400700 1 -- 172.21.15.1:0/2982756108 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff348009d00 con 0x7ff350073ac0 2024-02-02T15:27:21.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.734+0000 7ff356400700 1 --2- 172.21.15.1:0/2982756108 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff350073ac0 0x7ff3501a51f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff34800b070 tx=0x7ff34800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:21.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.734+0000 7ff34e000700 1 -- 172.21.15.1:0/2982756108 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff34800e040 con 0x7ff350073ac0 2024-02-02T15:27:21.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.734+0000 7ff34e000700 1 -- 172.21.15.1:0/2982756108 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff348009480 con 0x7ff350073ac0 2024-02-02T15:27:21.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.734+0000 7ff358400700 1 -- 172.21.15.1:0/2982756108 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3501a5e30 con 0x7ff350073ac0 2024-02-02T15:27:21.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.734+0000 7ff34e000700 1 -- 172.21.15.1:0/2982756108 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3480052a0 con 0x7ff350073ac0 2024-02-02T15:27:21.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.734+0000 7ff358400700 1 -- 172.21.15.1:0/2982756108 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3501a6200 con 0x7ff350073ac0 2024-02-02T15:27:21.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.738+0000 7ff34e000700 1 -- 172.21.15.1:0/2982756108 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135245+0+0 (secure 0 0 0) 0x7ff34801d480 con 0x7ff350073ac0 2024-02-02T15:27:21.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.739+0000 7ff36bf55b80 1 Processor -- start 2024-02-02T15:27:21.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.739+0000 7ff36bf55b80 1 -- start start 2024-02-02T15:27:21.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.740+0000 7ff34e000700 1 --2- 172.21.15.1:0/2982756108 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff3300a0720 0x7ff3300a2be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:21.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.740+0000 7ff36bf55b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad5f6b3010 0x55ad5f6b33f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:21.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.740+0000 7ff36bf55b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad5f6b3930 0x55ad5f6be2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:21.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.740+0000 7ff36bf55b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ad5f6be7e0 0x55ad5f6c0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:21.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.740+0000 7ff36bf55b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad5f6274c0 con 0x55ad5f6b3930 2024-02-02T15:27:21.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.740+0000 7ff36bf55b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad5f60baa0 con 0x55ad5f6b3010 2024-02-02T15:27:21.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.740+0000 7ff36bf55b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad5f609460 con 0x55ad5f6be7e0 2024-02-02T15:27:21.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.740+0000 7ff34e000700 1 -- 172.21.15.1:0/2982756108 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7ff3480cd450 con 0x7ff350073ac0 2024-02-02T15:27:21.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.740+0000 7ff355000700 1 --2- 172.21.15.1:0/2982756108 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff3300a0720 0x7ff3300a2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:21.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.740+0000 7ff355a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad5f6b3010 0x55ad5f6b33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:21.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.740+0000 7ff355000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad5f6b3930 0x55ad5f6be2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:21.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.740+0000 7ff355a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad5f6b3010 0x55ad5f6b33f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59424/0 (socket says 172.21.15.1:59424) 2024-02-02T15:27:21.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.740+0000 7ff355a00700 1 -- 172.21.15.1:0/1135771888 learned_addr learned my addr 172.21.15.1:0/1135771888 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:21.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.740+0000 7ff356400700 1 --2- 172.21.15.1:0/1135771888 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ad5f6be7e0 0x55ad5f6c0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:21.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.741+0000 7ff355a00700 1 -- 172.21.15.1:0/1135771888 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ad5f6be7e0 msgr2=0x55ad5f6c0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:21.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.741+0000 7ff355a00700 1 --2- 172.21.15.1:0/1135771888 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ad5f6be7e0 0x55ad5f6c0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.741+0000 7ff355a00700 1 -- 172.21.15.1:0/1135771888 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad5f6b3930 msgr2=0x55ad5f6be2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:21.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.741+0000 7ff355a00700 1 --2- 172.21.15.1:0/1135771888 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad5f6b3930 0x55ad5f6be2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.741+0000 7ff355a00700 1 -- 172.21.15.1:0/1135771888 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ad5f4863b0 con 0x55ad5f6b3010 2024-02-02T15:27:21.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.741+0000 7ff356400700 1 --2- 172.21.15.1:0/1135771888 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ad5f6be7e0 0x55ad5f6c0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:27:21.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.741+0000 7ff355000700 1 --2- 172.21.15.1:0/2982756108 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff3300a0720 0x7ff3300a2be0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff338009fd0 tx=0x7ff338009b40 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:21.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.741+0000 7ff355a00700 1 --2- 172.21.15.1:0/1135771888 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad5f6b3010 0x55ad5f6b33f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff340009db0 tx=0x7ff340011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edc81846a046ddeb server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:21.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.741+0000 7ff346a00700 1 -- 172.21.15.1:0/1135771888 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff340004230 con 0x55ad5f6b3010 2024-02-02T15:27:21.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.741+0000 7ff346a00700 1 -- 172.21.15.1:0/1135771888 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff340004390 con 0x55ad5f6b3010 2024-02-02T15:27:21.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.741+0000 7ff346a00700 1 -- 172.21.15.1:0/1135771888 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff340019880 con 0x55ad5f6b3010 2024-02-02T15:27:21.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.742+0000 7ff36bf55b80 1 -- 172.21.15.1:0/1135771888 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad5f6b3010 msgr2=0x55ad5f6b33f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:21.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.742+0000 7ff36bf55b80 1 --2- 172.21.15.1:0/1135771888 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad5f6b3010 0x55ad5f6b33f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff340009db0 tx=0x7ff340011720 comp rx=0 tx=0).stop 2024-02-02T15:27:21.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.742+0000 7ff36bf55b80 1 -- 172.21.15.1:0/1135771888 shutdown_connections 2024-02-02T15:27:21.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.742+0000 7ff36bf55b80 1 --2- 172.21.15.1:0/1135771888 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ad5f6be7e0 0x55ad5f6c0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.742+0000 7ff36bf55b80 1 --2- 172.21.15.1:0/1135771888 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad5f6b3010 0x55ad5f6b33f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.742+0000 7ff36bf55b80 1 --2- 172.21.15.1:0/1135771888 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad5f6b3930 0x55ad5f6be2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.742+0000 7ff36bf55b80 1 -- 172.21.15.1:0/1135771888 >> 172.21.15.1:0/1135771888 conn(0x55ad5f612db0 msgr2=0x55ad5f6af3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:21.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.742+0000 7ff36bf55b80 1 -- 172.21.15.1:0/1135771888 shutdown_connections 2024-02-02T15:27:21.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.742+0000 7ff36bf55b80 1 -- 172.21.15.1:0/1135771888 wait complete. 2024-02-02T15:27:21.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.743+0000 7ff36bf55b80 1 -- start start 2024-02-02T15:27:21.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.743+0000 7ff36bf55b80 1 Processor -- start 2024-02-02T15:27:21.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.743+0000 7ff36bf55b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad5f6b3010 0x55ad5f6be0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:21.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.744+0000 7ff36bf55b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad5f6b3930 0x55ad5f6bb350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:21.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.744+0000 7ff36bf55b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ad5f6be7e0 0x55ad5f6bb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:21.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.744+0000 7ff36bf55b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad5f604d50 con 0x55ad5f6b3930 2024-02-02T15:27:21.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.744+0000 7ff36bf55b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad5f5a0d80 con 0x55ad5f6b3010 2024-02-02T15:27:21.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.744+0000 7ff36bf55b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad5f5a0a20 con 0x55ad5f6be7e0 2024-02-02T15:27:21.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.744+0000 7ff36bf55b80 10 client.0 _ll_register_callbacks cb 0x55ad5f5a2290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:27:21.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.744+0000 7ff355000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad5f6b3930 0x55ad5f6bb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:21.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.744+0000 7ff356400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ad5f6be7e0 0x55ad5f6bb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:21.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.744+0000 7ff355000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad5f6b3930 0x55ad5f6bb350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43596/0 (socket says 172.21.15.1:43596) 2024-02-02T15:27:21.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.744+0000 7ff355000700 1 -- 172.21.15.1:0/2883940058 learned_addr learned my addr 172.21.15.1:0/2883940058 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:21.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.744+0000 7ff355a00700 1 --2- 172.21.15.1:0/2883940058 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad5f6b3010 0x55ad5f6be0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:21.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.744+0000 7ff356400700 1 -- 172.21.15.1:0/2883940058 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad5f6b3010 msgr2=0x55ad5f6be0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:21.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.744+0000 7ff356400700 1 --2- 172.21.15.1:0/2883940058 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad5f6b3010 0x55ad5f6be0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.744+0000 7ff356400700 1 -- 172.21.15.1:0/2883940058 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad5f6b3930 msgr2=0x55ad5f6bb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:21.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.744+0000 7ff356400700 1 --2- 172.21.15.1:0/2883940058 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad5f6b3930 0x55ad5f6bb350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.744+0000 7ff356400700 1 -- 172.21.15.1:0/2883940058 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3501a6200 con 0x55ad5f6be7e0 2024-02-02T15:27:21.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.745+0000 7ff356400700 1 --2- 172.21.15.1:0/2883940058 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ad5f6be7e0 0x55ad5f6bb8d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff3480ca730 tx=0x7ff3480961f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=854db40ccb9ac564 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:21.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.745+0000 7ff33f400700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:27:21.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.745+0000 7ff33f400700 1 -- 172.21.15.1:0/2883940058 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3480cabe0 con 0x55ad5f6be7e0 2024-02-02T15:27:21.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.745+0000 7ff33f400700 1 -- 172.21.15.1:0/2883940058 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7ff348003e10 con 0x55ad5f6be7e0 2024-02-02T15:27:21.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.745+0000 7ff36bf55b80 1 -- 172.21.15.1:0/2883940058 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ad5f6a70f0 con 0x55ad5f6be7e0 2024-02-02T15:27:21.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.745+0000 7ff33f400700 1 -- 172.21.15.1:0/2883940058 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff34809e120 con 0x55ad5f6be7e0 2024-02-02T15:27:21.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.745+0000 7ff36bf55b80 10 client.25729 fetch_fsmap learned FSMap version 27 2024-02-02T15:27:21.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.745+0000 7ff36bf55b80 1 -- 172.21.15.1:0/2883940058 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7ff3400099e0 con 0x55ad5f6be7e0 2024-02-02T15:27:21.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.745+0000 7ff33f400700 1 -- 172.21.15.1:0/2883940058 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ad5f6a70f0 con 0x55ad5f6be7e0 2024-02-02T15:27:21.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.745+0000 7ff33f400700 1 -- 172.21.15.1:0/2883940058 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff3480cd020 con 0x55ad5f6be7e0 2024-02-02T15:27:21.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.746+0000 7ff36bf55b80 10 client.25729 fetch_fsmap finished waiting for FSMap version 27 2024-02-02T15:27:21.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.746+0000 7ff36bf55b80 10 client.25729 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-02-02T15:27:21.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.746+0000 7ff36bf55b80 20 client.25729 populate_metadata read hostname 'smithi001' 2024-02-02T15:27:21.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.746+0000 7ff36bf55b80 1 --2- 172.21.15.1:0/2883940058 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55ad5f6cdfa0 0x55ad5f6d0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:21.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.746+0000 7ff36bf55b80 4 client.25729 mds_command: new command op to 24514 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:27:21.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.746+0000 7ff36bf55b80 1 -- 172.21.15.1:0/2883940058 --> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ad5f5fddd0 con 0x55ad5f6cdfa0 2024-02-02T15:27:21.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.746+0000 7ff355a00700 1 --2- 172.21.15.1:0/2883940058 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55ad5f6cdfa0 0x55ad5f6d0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:21.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.746+0000 7ff355a00700 1 --2- 172.21.15.1:0/2883940058 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55ad5f6cdfa0 0x55ad5f6d0390 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bedc3478da83e6f1 server_cookie=47b70c86e654042f in_seq=0 out_seq=0 2024-02-02T15:27:21.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.746+0000 7ff33f400700 10 client.25729 ms_handle_connect on v2:172.21.15.142:6836/2870289019 2024-02-02T15:27:21.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.747+0000 7ff33f400700 1 -- 172.21.15.1:0/2883940058 <== mds.3 v2:172.21.15.142:6836/2870289019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55ad5f5fddd0 con 0x55ad5f6cdfa0 2024-02-02T15:27:21.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.747+0000 7ff33f400700 10 client.25729 handle_command_reply: tid=0 2024-02-02T15:27:21.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.747+0000 7ff36bf55b80 1 client.25729 shutdown 2024-02-02T15:27:21.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.747+0000 7ff36bf55b80 1 -- 172.21.15.1:0/2883940058 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ad5f6be7e0 msgr2=0x55ad5f6bb8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:21.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.747+0000 7ff36bf55b80 1 --2- 172.21.15.1:0/2883940058 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ad5f6be7e0 0x55ad5f6bb8d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff3480ca730 tx=0x7ff3480961f0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.748+0000 7ff36bf55b80 1 -- 172.21.15.1:0/2883940058 shutdown_connections 2024-02-02T15:27:21.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.748+0000 7ff36bf55b80 1 --2- 172.21.15.1:0/2883940058 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ad5f6be7e0 0x55ad5f6bb8d0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.748+0000 7ff36bf55b80 1 --2- 172.21.15.1:0/2883940058 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad5f6b3010 0x55ad5f6be0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.748+0000 7ff36bf55b80 1 --2- 172.21.15.1:0/2883940058 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55ad5f6cdfa0 0x55ad5f6d0390 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.748+0000 7ff36bf55b80 1 --2- 172.21.15.1:0/2883940058 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad5f6b3930 0x55ad5f6bb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.748+0000 7ff33f400700 0 client.25729 ms_handle_reset on v2:172.21.15.142:6836/2870289019 2024-02-02T15:27:21.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.748+0000 7ff36bf55b80 1 -- 172.21.15.1:0/2883940058 >> 172.21.15.1:0/2883940058 conn(0x55ad5f612db0 msgr2=0x55ad5f6aefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:21.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.748+0000 7ff36bf55b80 1 -- 172.21.15.1:0/2883940058 shutdown_connections 2024-02-02T15:27:21.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.748+0000 7ff36bf55b80 1 -- 172.21.15.1:0/2883940058 wait complete. 2024-02-02T15:27:21.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.748+0000 7ff36bf55b80 20 client.25729 trim_cache size 0 max 16384 2024-02-02T15:27:21.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.761+0000 7ff36bf55b80 1 Processor -- start 2024-02-02T15:27:21.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.761+0000 7ff36bf55b80 1 -- start start 2024-02-02T15:27:21.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.761+0000 7ff36bf55b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad5f6b3010 0x55ad5f6c5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:21.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.761+0000 7ff36bf55b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad5f6b3930 0x55ad5f6bdc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:21.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.761+0000 7ff36bf55b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ad5f6be7e0 0x55ad5f6b85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:21.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.761+0000 7ff36bf55b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad5f4886a0 con 0x55ad5f6b3010 2024-02-02T15:27:21.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.761+0000 7ff36bf55b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad5f6b6f50 con 0x55ad5f6b3930 2024-02-02T15:27:21.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.761+0000 7ff36bf55b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad5f6ae1a0 con 0x55ad5f6be7e0 2024-02-02T15:27:21.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.762+0000 7ff355a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad5f6b3010 0x55ad5f6c5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:21.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.762+0000 7ff355000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad5f6b3930 0x55ad5f6bdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:21.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.762+0000 7ff355a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad5f6b3010 0x55ad5f6c5370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43612/0 (socket says 172.21.15.1:43612) 2024-02-02T15:27:21.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.762+0000 7ff355a00700 1 -- 172.21.15.1:0/965158291 learned_addr learned my addr 172.21.15.1:0/965158291 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:21.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.762+0000 7ff356400700 1 --2- 172.21.15.1:0/965158291 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ad5f6be7e0 0x55ad5f6b85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:21.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.762+0000 7ff355000700 1 -- 172.21.15.1:0/965158291 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ad5f6be7e0 msgr2=0x55ad5f6b85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:21.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.762+0000 7ff355000700 1 --2- 172.21.15.1:0/965158291 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ad5f6be7e0 0x55ad5f6b85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.762+0000 7ff355000700 1 -- 172.21.15.1:0/965158291 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad5f6b3010 msgr2=0x55ad5f6c5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:21.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.762+0000 7ff355000700 1 --2- 172.21.15.1:0/965158291 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad5f6b3010 0x55ad5f6c5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.762+0000 7ff355000700 1 -- 172.21.15.1:0/965158291 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ad5f432b60 con 0x55ad5f6b3930 2024-02-02T15:27:21.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.762+0000 7ff355000700 1 --2- 172.21.15.1:0/965158291 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad5f6b3930 0x55ad5f6bdc70 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x55ad5f6bc900 tx=0x7ff3380047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=915830052412d113 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:21.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.763+0000 7ff347400700 1 -- 172.21.15.1:0/965158291 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3380044d0 con 0x55ad5f6b3930 2024-02-02T15:27:21.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.763+0000 7ff347400700 1 -- 172.21.15.1:0/965158291 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff338004cf0 con 0x55ad5f6b3930 2024-02-02T15:27:21.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.763+0000 7ff347400700 1 -- 172.21.15.1:0/965158291 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff338011e40 con 0x55ad5f6b3930 2024-02-02T15:27:21.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.763+0000 7ff36bf55b80 1 -- 172.21.15.1:0/965158291 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad5f6b3930 msgr2=0x55ad5f6bdc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:21.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.763+0000 7ff36bf55b80 1 --2- 172.21.15.1:0/965158291 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad5f6b3930 0x55ad5f6bdc70 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x55ad5f6bc900 tx=0x7ff3380047c0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.763+0000 7ff36bf55b80 1 -- 172.21.15.1:0/965158291 shutdown_connections 2024-02-02T15:27:21.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.763+0000 7ff36bf55b80 1 --2- 172.21.15.1:0/965158291 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ad5f6be7e0 0x55ad5f6b85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.763+0000 7ff36bf55b80 1 --2- 172.21.15.1:0/965158291 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad5f6b3010 0x55ad5f6c5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.763+0000 7ff36bf55b80 1 --2- 172.21.15.1:0/965158291 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad5f6b3930 0x55ad5f6bdc70 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.763+0000 7ff36bf55b80 1 -- 172.21.15.1:0/965158291 >> 172.21.15.1:0/965158291 conn(0x55ad5f612db0 msgr2=0x55ad5f6c9940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:21.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.763+0000 7ff36bf55b80 1 -- 172.21.15.1:0/965158291 shutdown_connections 2024-02-02T15:27:21.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.763+0000 7ff36bf55b80 1 -- 172.21.15.1:0/965158291 wait complete. 2024-02-02T15:27:21.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.764+0000 7ff36bf55b80 1 -- start start 2024-02-02T15:27:21.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.764+0000 7ff36bf55b80 1 Processor -- start 2024-02-02T15:27:21.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.764+0000 7ff36bf55b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad5f6b3010 0x55ad5f713bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:21.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.764+0000 7ff36bf55b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ad5f6b3930 0x55ad5f714110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:21.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.764+0000 7ff36bf55b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad5f6be7e0 0x55ad5f718a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:21.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.764+0000 7ff36bf55b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad5f718f70 con 0x55ad5f6b3010 2024-02-02T15:27:21.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.764+0000 7ff36bf55b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad5f7190e0 con 0x55ad5f6be7e0 2024-02-02T15:27:21.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.764+0000 7ff36bf55b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad5f7192a0 con 0x55ad5f6b3930 2024-02-02T15:27:21.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.765+0000 7ff36bf55b80 10 client.0 _ll_register_callbacks cb 0x55ad5f5a2290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:27:21.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.765+0000 7ff355a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad5f6b3010 0x55ad5f713bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:21.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.765+0000 7ff356400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad5f6be7e0 0x55ad5f718a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:21.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.765+0000 7ff356400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad5f6be7e0 0x55ad5f718a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59454/0 (socket says 172.21.15.1:59454) 2024-02-02T15:27:21.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.765+0000 7ff356400700 1 -- 172.21.15.1:0/3998159573 learned_addr learned my addr 172.21.15.1:0/3998159573 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:21.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.765+0000 7ff355000700 1 --2- 172.21.15.1:0/3998159573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ad5f6b3930 0x55ad5f714110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:21.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.765+0000 7ff356400700 1 -- 172.21.15.1:0/3998159573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ad5f6b3930 msgr2=0x55ad5f714110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:21.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.765+0000 7ff356400700 1 --2- 172.21.15.1:0/3998159573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ad5f6b3930 0x55ad5f714110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.765+0000 7ff356400700 1 -- 172.21.15.1:0/3998159573 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad5f6b3010 msgr2=0x55ad5f713bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:21.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.765+0000 7ff356400700 1 --2- 172.21.15.1:0/3998159573 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad5f6b3010 0x55ad5f713bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.765+0000 7ff356400700 1 -- 172.21.15.1:0/3998159573 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff338013e70 con 0x55ad5f6be7e0 2024-02-02T15:27:21.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.765+0000 7ff355a00700 1 --2- 172.21.15.1:0/3998159573 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad5f6b3010 0x55ad5f713bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:27:21.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.765+0000 7ff356400700 1 --2- 172.21.15.1:0/3998159573 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad5f6be7e0 0x55ad5f718a30 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff34800b780 tx=0x7ff348007550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55bcdfa0ef096615 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:21.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.765+0000 7ff33f400700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:27:21.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.766+0000 7ff33f400700 1 -- 172.21.15.1:0/3998159573 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff348009960 con 0x55ad5f6be7e0 2024-02-02T15:27:21.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.766+0000 7ff36bf55b80 1 -- 172.21.15.1:0/3998159573 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff340004390 con 0x55ad5f6be7e0 2024-02-02T15:27:21.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.766+0000 7ff33f400700 1 -- 172.21.15.1:0/3998159573 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7ff348003e10 con 0x55ad5f6be7e0 2024-02-02T15:27:21.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.766+0000 7ff33f400700 1 -- 172.21.15.1:0/3998159573 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff34809e120 con 0x55ad5f6be7e0 2024-02-02T15:27:21.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.766+0000 7ff36bf55b80 10 client.25736 fetch_fsmap learned FSMap version 27 2024-02-02T15:27:21.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.766+0000 7ff36bf55b80 1 -- 172.21.15.1:0/3998159573 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55ad5f5caea0 con 0x55ad5f6be7e0 2024-02-02T15:27:21.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.766+0000 7ff33f400700 1 -- 172.21.15.1:0/3998159573 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff348066020 con 0x55ad5f6be7e0 2024-02-02T15:27:21.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.766+0000 7ff33f400700 1 -- 172.21.15.1:0/3998159573 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff348066190 con 0x55ad5f6be7e0 2024-02-02T15:27:21.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.766+0000 7ff36bf55b80 10 client.25736 fetch_fsmap finished waiting for FSMap version 27 2024-02-02T15:27:21.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.766+0000 7ff36bf55b80 10 client.25736 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-02-02T15:27:21.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.766+0000 7ff36bf55b80 20 client.25736 populate_metadata read hostname 'smithi001' 2024-02-02T15:27:21.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.766+0000 7ff36bf55b80 1 --2- 172.21.15.1:0/3998159573 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55ad5f720540 0x55ad5f722930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:21.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.766+0000 7ff36bf55b80 4 client.25736 mds_command: new command op to 24514 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:27:21.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.766+0000 7ff36bf55b80 1 -- 172.21.15.1:0/3998159573 --> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ad5f5c3f50 con 0x55ad5f720540 2024-02-02T15:27:21.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.767+0000 7ff355a00700 1 --2- 172.21.15.1:0/3998159573 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55ad5f720540 0x55ad5f722930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:21.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.767+0000 7ff355a00700 1 --2- 172.21.15.1:0/3998159573 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55ad5f720540 0x55ad5f722930 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ad95fb0decd88f2 server_cookie=52b4de10f5ec3f59 in_seq=0 out_seq=0 2024-02-02T15:27:21.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.767+0000 7ff33f400700 10 client.25736 ms_handle_connect on v2:172.21.15.142:6836/2870289019 2024-02-02T15:27:21.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.768+0000 7ff33f400700 1 -- 172.21.15.1:0/3998159573 <== mds.3 v2:172.21.15.142:6836/2870289019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ad5f5c3f50 con 0x55ad5f720540 2024-02-02T15:27:21.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.768+0000 7ff33f400700 10 client.25736 handle_command_reply: tid=0 2024-02-02T15:27:21.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.768+0000 7ff36bf55b80 1 client.25736 shutdown 2024-02-02T15:27:21.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.768+0000 7ff36bf55b80 1 -- 172.21.15.1:0/3998159573 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad5f6be7e0 msgr2=0x55ad5f718a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:21.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.768+0000 7ff36bf55b80 1 --2- 172.21.15.1:0/3998159573 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad5f6be7e0 0x55ad5f718a30 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff34800b780 tx=0x7ff348007550 comp rx=0 tx=0).stop 2024-02-02T15:27:21.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.768+0000 7ff36bf55b80 1 -- 172.21.15.1:0/3998159573 shutdown_connections 2024-02-02T15:27:21.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.768+0000 7ff36bf55b80 1 --2- 172.21.15.1:0/3998159573 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad5f6be7e0 0x55ad5f718a30 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.768+0000 7ff36bf55b80 1 --2- 172.21.15.1:0/3998159573 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55ad5f720540 0x55ad5f722930 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.768+0000 7ff36bf55b80 1 --2- 172.21.15.1:0/3998159573 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ad5f6b3010 0x55ad5f713bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.768+0000 7ff36bf55b80 1 --2- 172.21.15.1:0/3998159573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ad5f6b3930 0x55ad5f714110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.768+0000 7ff36bf55b80 1 -- 172.21.15.1:0/3998159573 >> 172.21.15.1:0/3998159573 conn(0x55ad5f612db0 msgr2=0x55ad5f6ae5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:21.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.768+0000 7ff33f400700 0 client.25736 ms_handle_reset on v2:172.21.15.142:6836/2870289019 2024-02-02T15:27:21.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.768+0000 7ff36bf55b80 1 -- 172.21.15.1:0/3998159573 shutdown_connections 2024-02-02T15:27:21.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.768+0000 7ff36bf55b80 1 -- 172.21.15.1:0/3998159573 wait complete. 2024-02-02T15:27:21.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.768+0000 7ff36bf55b80 20 client.25736 trim_cache size 0 max 16384 2024-02-02T15:27:21.775 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:27:21.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.769+0000 7ff358400700 1 -- 172.21.15.1:0/2982756108 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff3300a0720 msgr2=0x7ff3300a2be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:21.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.769+0000 7ff358400700 1 --2- 172.21.15.1:0/2982756108 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff3300a0720 0x7ff3300a2be0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff338009fd0 tx=0x7ff338009b40 comp rx=0 tx=0).stop 2024-02-02T15:27:21.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.769+0000 7ff358400700 1 -- 172.21.15.1:0/2982756108 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff350073ac0 msgr2=0x7ff3501a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:21.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.769+0000 7ff358400700 1 --2- 172.21.15.1:0/2982756108 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff350073ac0 0x7ff3501a51f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff34800b070 tx=0x7ff34800bc80 comp rx=0 tx=0).stop 2024-02-02T15:27:21.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.769+0000 7ff358400700 1 -- 172.21.15.1:0/2982756108 shutdown_connections 2024-02-02T15:27:21.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.769+0000 7ff358400700 1 --2- 172.21.15.1:0/2982756108 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff3300a0720 0x7ff3300a2be0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.769+0000 7ff358400700 1 --2- 172.21.15.1:0/2982756108 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff350073ac0 0x7ff3501a51f0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.769+0000 7ff358400700 1 --2- 172.21.15.1:0/2982756108 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff350100ff0 0x7ff350073040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.769+0000 7ff358400700 1 --2- 172.21.15.1:0/2982756108 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff350106110 0x7ff350073580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.769+0000 7ff358400700 1 -- 172.21.15.1:0/2982756108 >> 172.21.15.1:0/2982756108 conn(0x7ff3500fc810 msgr2=0x7ff3500fd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:21.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.769+0000 7ff358400700 1 -- 172.21.15.1:0/2982756108 shutdown_connections 2024-02-02T15:27:21.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.769+0000 7ff358400700 1 -- 172.21.15.1:0/2982756108 wait complete. 2024-02-02T15:27:21.787 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-02T15:27:21.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.987+0000 7f90ba800700 1 -- 172.21.15.1:0/1623161412 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90b41019a0 msgr2=0x7f90b41059e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:21.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.987+0000 7f90ba800700 1 --2- 172.21.15.1:0/1623161412 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90b41019a0 0x7f90b41059e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f909c0098f0 tx=0x7f909c009c00 comp rx=0 tx=0).stop 2024-02-02T15:27:21.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.987+0000 7f90ba800700 1 -- 172.21.15.1:0/1623161412 shutdown_connections 2024-02-02T15:27:21.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.987+0000 7f90ba800700 1 --2- 172.21.15.1:0/1623161412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90b4106110 0x7f90b410dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.988+0000 7f90ba800700 1 --2- 172.21.15.1:0/1623161412 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90b4100ff0 0x7f90b41013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.988+0000 7f90ba800700 1 --2- 172.21.15.1:0/1623161412 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90b41019a0 0x7f90b41059e0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.988+0000 7f90ba800700 1 -- 172.21.15.1:0/1623161412 >> 172.21.15.1:0/1623161412 conn(0x7f90b40fc810 msgr2=0x7f90b40fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:21.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.988+0000 7f90ba800700 1 -- 172.21.15.1:0/1623161412 shutdown_connections 2024-02-02T15:27:21.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.988+0000 7f90ba800700 1 -- 172.21.15.1:0/1623161412 wait complete. 2024-02-02T15:27:21.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.989+0000 7f90ba800700 1 Processor -- start 2024-02-02T15:27:21.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.989+0000 7f90ba800700 1 -- start start 2024-02-02T15:27:21.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.989+0000 7f90ba800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90b4100ff0 0x7f90b4199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:21.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.989+0000 7f90ba800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90b4106110 0x7f90b4199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:21.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.989+0000 7f90ba800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90b419dad0 0x7f90b419df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:21.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.989+0000 7f90ba800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90b419e550 con 0x7f90b4106110 2024-02-02T15:27:21.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.989+0000 7f90ba800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90b419e6c0 con 0x7f90b4100ff0 2024-02-02T15:27:21.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.989+0000 7f90ba800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90b419e880 con 0x7f90b419dad0 2024-02-02T15:27:21.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.989+0000 7f90b2a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90b4106110 0x7f90b4199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:21.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.990+0000 7f90b3400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90b4100ff0 0x7f90b4199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:21.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.990+0000 7f90b2a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90b4106110 0x7f90b4199670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43636/0 (socket says 172.21.15.1:43636) 2024-02-02T15:27:21.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.990+0000 7f90b2a00700 1 -- 172.21.15.1:0/1859583157 learned_addr learned my addr 172.21.15.1:0/1859583157 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:21.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.990+0000 7f90b3e00700 1 --2- 172.21.15.1:0/1859583157 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90b419dad0 0x7f90b419df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:21.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.990+0000 7f90b3400700 1 -- 172.21.15.1:0/1859583157 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90b419dad0 msgr2=0x7f90b419df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:21.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.990+0000 7f90b3400700 1 --2- 172.21.15.1:0/1859583157 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90b419dad0 0x7f90b419df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.990+0000 7f90b3400700 1 -- 172.21.15.1:0/1859583157 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90b4106110 msgr2=0x7f90b4199670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:21.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.990+0000 7f90b3400700 1 --2- 172.21.15.1:0/1859583157 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90b4106110 0x7f90b4199670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.990+0000 7f90b3400700 1 -- 172.21.15.1:0/1859583157 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f909c009520 con 0x7f90b4100ff0 2024-02-02T15:27:21.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.990+0000 7f90b2a00700 1 --2- 172.21.15.1:0/1859583157 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90b4106110 0x7f90b4199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:27:21.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.990+0000 7f90b3e00700 1 --2- 172.21.15.1:0/1859583157 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90b419dad0 0x7f90b419df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:27:21.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.990+0000 7f90b3400700 1 --2- 172.21.15.1:0/1859583157 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90b4100ff0 0x7f90b4199130 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f90a400e9c0 tx=0x7f90a400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:21.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.990+0000 7f90abe00700 1 -- 172.21.15.1:0/1859583157 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a400c9c0 con 0x7f90b4100ff0 2024-02-02T15:27:21.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.991+0000 7f90ba800700 1 -- 172.21.15.1:0/1859583157 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90b41a5de0 con 0x7f90b4100ff0 2024-02-02T15:27:21.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.991+0000 7f90abe00700 1 -- 172.21.15.1:0/1859583157 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f90a4010430 con 0x7f90b4100ff0 2024-02-02T15:27:21.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.991+0000 7f90abe00700 1 -- 172.21.15.1:0/1859583157 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a4018680 con 0x7f90b4100ff0 2024-02-02T15:27:21.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.991+0000 7f90ba800700 1 -- 172.21.15.1:0/1859583157 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90b41a6270 con 0x7f90b4100ff0 2024-02-02T15:27:21.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.994+0000 7f90abe00700 1 -- 172.21.15.1:0/1859583157 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 25) v1 ==== 135245+0+0 (secure 0 0 0) 0x7f90a4010860 con 0x7f90b4100ff0 2024-02-02T15:27:21.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.995+0000 7f90ce22bb80 1 Processor -- start 2024-02-02T15:27:21.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.995+0000 7f90ce22bb80 1 -- start start 2024-02-02T15:27:21.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.995+0000 7f90abe00700 1 --2- 172.21.15.1:0/1859583157 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f90940a07a0 0x7f90940a2c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:21.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.995+0000 7f90ce22bb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5564f5635010 0x5564f56353f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:21.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.995+0000 7f90ce22bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5564f5635930 0x5564f56402a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:21.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.995+0000 7f90ce22bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5564f56407e0 0x5564f5642bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:21.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.995+0000 7f90abe00700 1 -- 172.21.15.1:0/1859583157 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f90a4014070 con 0x7f90b4100ff0 2024-02-02T15:27:21.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.995+0000 7f90ce22bb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564f55a94c0 con 0x5564f5635010 2024-02-02T15:27:21.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.995+0000 7f90ce22bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564f558daa0 con 0x5564f56407e0 2024-02-02T15:27:21.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.995+0000 7f90ce22bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564f558b460 con 0x5564f5635930 2024-02-02T15:27:21.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.996+0000 7f90b3400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5564f5635010 0x5564f56353f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:21.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.996+0000 7f90b3e00700 1 --2- 172.21.15.1:0/1859583157 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f90940a07a0 0x7f90940a2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:21.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.996+0000 7f90b3400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5564f5635010 0x5564f56353f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43648/0 (socket says 172.21.15.1:43648) 2024-02-02T15:27:21.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.996+0000 7f90b2a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5564f5635930 0x5564f56402a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:21.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.996+0000 7f90b3400700 1 -- 172.21.15.1:0/3137261125 learned_addr learned my addr 172.21.15.1:0/3137261125 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:21.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.996+0000 7f90b3e00700 1 --2- 172.21.15.1:0/3137261125 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5564f56407e0 0x5564f5642bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:21.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.996+0000 7f90b3400700 1 -- 172.21.15.1:0/3137261125 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5564f5635930 msgr2=0x5564f56402a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:21.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.996+0000 7f90b3400700 1 --2- 172.21.15.1:0/3137261125 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5564f5635930 0x5564f56402a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.996+0000 7f90b3400700 1 -- 172.21.15.1:0/3137261125 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5564f56407e0 msgr2=0x5564f5642bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:21.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.996+0000 7f90b3400700 1 --2- 172.21.15.1:0/3137261125 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5564f56407e0 0x5564f5642bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.996+0000 7f90b3400700 1 -- 172.21.15.1:0/3137261125 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5564f54083b0 con 0x5564f5635010 2024-02-02T15:27:21.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.996+0000 7f90b3e00700 1 --2- 172.21.15.1:0/3137261125 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5564f56407e0 0x5564f5642bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:27:21.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.996+0000 7f90b3400700 1 --2- 172.21.15.1:0/3137261125 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5564f5635010 0x5564f56353f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f90a400c5b0 tx=0x7f90a4098dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b427622e3aa2f2ed server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:21.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.996+0000 7f90b3e00700 1 --2- 172.21.15.1:0/1859583157 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f90940a07a0 0x7f90940a2c60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f90ac009fd0 tx=0x7f90ac009920 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:21.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.997+0000 7f90a8c00700 1 -- 172.21.15.1:0/3137261125 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a4093ea0 con 0x5564f5635010 2024-02-02T15:27:21.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.997+0000 7f90a8c00700 1 -- 172.21.15.1:0/3137261125 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f90a4099990 con 0x5564f5635010 2024-02-02T15:27:21.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.997+0000 7f90a8c00700 1 -- 172.21.15.1:0/3137261125 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a4099710 con 0x5564f5635010 2024-02-02T15:27:21.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.997+0000 7f90ce22bb80 1 -- 172.21.15.1:0/3137261125 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5564f5635010 msgr2=0x5564f56353f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:21.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.997+0000 7f90ce22bb80 1 --2- 172.21.15.1:0/3137261125 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5564f5635010 0x5564f56353f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f90a400c5b0 tx=0x7f90a4098dd0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.997+0000 7f90ce22bb80 1 -- 172.21.15.1:0/3137261125 shutdown_connections 2024-02-02T15:27:21.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.997+0000 7f90ce22bb80 1 --2- 172.21.15.1:0/3137261125 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5564f56407e0 0x5564f5642bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.998+0000 7f90ce22bb80 1 --2- 172.21.15.1:0/3137261125 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5564f5635010 0x5564f56353f0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.998+0000 7f90ce22bb80 1 --2- 172.21.15.1:0/3137261125 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5564f5635930 0x5564f56402a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:21.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.998+0000 7f90ce22bb80 1 -- 172.21.15.1:0/3137261125 >> 172.21.15.1:0/3137261125 conn(0x5564f5594db0 msgr2=0x5564f56313b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:21.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.998+0000 7f90ce22bb80 1 -- 172.21.15.1:0/3137261125 shutdown_connections 2024-02-02T15:27:21.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.998+0000 7f90ce22bb80 1 -- 172.21.15.1:0/3137261125 wait complete. 2024-02-02T15:27:22.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.998+0000 7f90ce22bb80 1 -- start start 2024-02-02T15:27:22.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.999+0000 7f90ce22bb80 1 Processor -- start 2024-02-02T15:27:22.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.999+0000 7f90ce22bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5564f5635010 0x5564f5640130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:22.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.999+0000 7f90ce22bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5564f5635930 0x5564f563d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:22.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.999+0000 7f90ce22bb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5564f56407e0 0x5564f563d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:22.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.999+0000 7f90ce22bb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564f5586d50 con 0x5564f56407e0 2024-02-02T15:27:22.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.999+0000 7f90ce22bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564f5522d80 con 0x5564f5635930 2024-02-02T15:27:22.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.999+0000 7f90ce22bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564f5522a20 con 0x5564f5635010 2024-02-02T15:27:22.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:21.999+0000 7f90ce22bb80 10 client.0 _ll_register_callbacks cb 0x5564f5524290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:27:22.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.000+0000 7f90b3e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5564f56407e0 0x5564f563d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:22.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.000+0000 7f90b3e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5564f56407e0 0x5564f563d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43654/0 (socket says 172.21.15.1:43654) 2024-02-02T15:27:22.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.000+0000 7f90b3e00700 1 -- 172.21.15.1:0/2465531947 learned_addr learned my addr 172.21.15.1:0/2465531947 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:22.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.000+0000 7f90b3400700 1 --2- 172.21.15.1:0/2465531947 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5564f5635010 0x5564f5640130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:22.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.000+0000 7f90b2a00700 1 --2- 172.21.15.1:0/2465531947 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5564f5635930 0x5564f563d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:22.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.000+0000 7f90b3e00700 1 -- 172.21.15.1:0/2465531947 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5564f5635010 msgr2=0x5564f5640130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:22.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.000+0000 7f90b3e00700 1 --2- 172.21.15.1:0/2465531947 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5564f5635010 0x5564f5640130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.000+0000 7f90b3e00700 1 -- 172.21.15.1:0/2465531947 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5564f5635930 msgr2=0x5564f563d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:22.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.000+0000 7f90b3e00700 1 --2- 172.21.15.1:0/2465531947 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5564f5635930 0x5564f563d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.000+0000 7f90b3e00700 1 -- 172.21.15.1:0/2465531947 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90b41a6270 con 0x5564f56407e0 2024-02-02T15:27:22.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.000+0000 7f90b3e00700 1 --2- 172.21.15.1:0/2465531947 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5564f56407e0 0x5564f563d990 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f90ac00ebf0 tx=0x7f90ac004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c95e2b8d65855e72 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:22.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.000+0000 7f90a1600700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:27:22.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.000+0000 7f90a1600700 1 -- 172.21.15.1:0/2465531947 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90ac01aad0 con 0x5564f56407e0 2024-02-02T15:27:22.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.001+0000 7f90a1600700 1 -- 172.21.15.1:0/2465531947 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f90ac01f040 con 0x5564f56407e0 2024-02-02T15:27:22.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.001+0000 7f90ce22bb80 1 -- 172.21.15.1:0/2465531947 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5564f56290f0 con 0x5564f56407e0 2024-02-02T15:27:22.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.001+0000 7f90a1600700 1 -- 172.21.15.1:0/2465531947 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f90ac01ac30 con 0x5564f56407e0 2024-02-02T15:27:22.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.001+0000 7f90ce22bb80 10 client.15711 fetch_fsmap learned FSMap version 27 2024-02-02T15:27:22.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.001+0000 7f90ce22bb80 1 -- 172.21.15.1:0/2465531947 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5564f53b4b60 con 0x5564f56407e0 2024-02-02T15:27:22.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.001+0000 7f90a1600700 1 -- 172.21.15.1:0/2465531947 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5564f56290f0 con 0x5564f56407e0 2024-02-02T15:27:22.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.001+0000 7f90a1600700 1 -- 172.21.15.1:0/2465531947 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f90ac023410 con 0x5564f56407e0 2024-02-02T15:27:22.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.001+0000 7f90ce22bb80 10 client.15711 fetch_fsmap finished waiting for FSMap version 27 2024-02-02T15:27:22.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.001+0000 7f90ce22bb80 10 client.15711 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-02-02T15:27:22.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.001+0000 7f90ce22bb80 20 client.15711 populate_metadata read hostname 'smithi001' 2024-02-02T15:27:22.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.001+0000 7f90ce22bb80 1 --2- 172.21.15.1:0/2465531947 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x5564f5650240 0x5564f5652660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:22.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.001+0000 7f90ce22bb80 4 client.15711 mds_command: new command op to 24518 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:27:22.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.001+0000 7f90ce22bb80 1 -- 172.21.15.1:0/2465531947 --> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5564f557fdd0 con 0x5564f5650240 2024-02-02T15:27:22.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.002+0000 7f90b3400700 1 --2- 172.21.15.1:0/2465531947 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x5564f5650240 0x5564f5652660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:22.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.002+0000 7f90b3400700 1 --2- 172.21.15.1:0/2465531947 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x5564f5650240 0x5564f5652660 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=179ec8ff94c1323c server_cookie=52e272f1d64a78d in_seq=0 out_seq=0 2024-02-02T15:27:22.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.002+0000 7f90a1600700 10 client.15711 ms_handle_connect on v2:172.21.15.71:6836/628016199 2024-02-02T15:27:22.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.003+0000 7f90a1600700 1 -- 172.21.15.1:0/2465531947 <== mds.2 v2:172.21.15.71:6836/628016199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5564f557fdd0 con 0x5564f5650240 2024-02-02T15:27:22.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.003+0000 7f90a1600700 10 client.15711 handle_command_reply: tid=0 2024-02-02T15:27:22.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.003+0000 7f90ce22bb80 1 client.15711 shutdown 2024-02-02T15:27:22.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.003+0000 7f90ce22bb80 1 -- 172.21.15.1:0/2465531947 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5564f56407e0 msgr2=0x5564f563d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:22.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.003+0000 7f90ce22bb80 1 --2- 172.21.15.1:0/2465531947 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5564f56407e0 0x5564f563d990 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f90ac00ebf0 tx=0x7f90ac004b40 comp rx=0 tx=0).stop 2024-02-02T15:27:22.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.003+0000 7f90ce22bb80 1 -- 172.21.15.1:0/2465531947 shutdown_connections 2024-02-02T15:27:22.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.003+0000 7f90ce22bb80 1 --2- 172.21.15.1:0/2465531947 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5564f56407e0 0x5564f563d990 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.003+0000 7f90ce22bb80 1 --2- 172.21.15.1:0/2465531947 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x5564f5650240 0x5564f5652660 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.003+0000 7f90ce22bb80 1 --2- 172.21.15.1:0/2465531947 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5564f5635010 0x5564f5640130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.003+0000 7f90ce22bb80 1 --2- 172.21.15.1:0/2465531947 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5564f5635930 0x5564f563d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.003+0000 7f90a1600700 0 client.15711 ms_handle_reset on v2:172.21.15.71:6836/628016199 2024-02-02T15:27:22.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.003+0000 7f90ce22bb80 1 -- 172.21.15.1:0/2465531947 >> 172.21.15.1:0/2465531947 conn(0x5564f5594db0 msgr2=0x5564f5630fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:22.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.003+0000 7f90ce22bb80 1 -- 172.21.15.1:0/2465531947 shutdown_connections 2024-02-02T15:27:22.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.004+0000 7f90ce22bb80 1 -- 172.21.15.1:0/2465531947 wait complete. 2024-02-02T15:27:22.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.004+0000 7f90ce22bb80 20 client.15711 trim_cache size 0 max 16384 2024-02-02T15:27:22.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.017+0000 7f90ce22bb80 1 Processor -- start 2024-02-02T15:27:22.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.017+0000 7f90ce22bb80 1 -- start start 2024-02-02T15:27:22.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.017+0000 7f90ce22bb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5564f5635010 0x5564f5637b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:22.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.017+0000 7f90ce22bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5564f5635930 0x5564f564b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:22.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.017+0000 7f90ce22bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5564f56407e0 0x5564f564bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:22.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.017+0000 7f90ce22bb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564f540a6a0 con 0x5564f5635010 2024-02-02T15:27:22.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.017+0000 7f90ce22bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564f5636260 con 0x5564f5635930 2024-02-02T15:27:22.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.017+0000 7f90ce22bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564f5639240 con 0x5564f56407e0 2024-02-02T15:27:22.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.017+0000 7f90b3400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5564f5635010 0x5564f5637b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:22.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.017+0000 7f90b3400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5564f5635010 0x5564f5637b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43666/0 (socket says 172.21.15.1:43666) 2024-02-02T15:27:22.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.017+0000 7f90b3400700 1 -- 172.21.15.1:0/1848989344 learned_addr learned my addr 172.21.15.1:0/1848989344 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:22.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.017+0000 7f90b2a00700 1 --2- 172.21.15.1:0/1848989344 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5564f5635930 0x5564f564b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:22.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.018+0000 7f90b3e00700 1 --2- 172.21.15.1:0/1848989344 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5564f56407e0 0x5564f564bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:22.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.018+0000 7f90b3400700 1 -- 172.21.15.1:0/1848989344 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5564f56407e0 msgr2=0x5564f564bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:22.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.018+0000 7f90b3400700 1 --2- 172.21.15.1:0/1848989344 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5564f56407e0 0x5564f564bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.018+0000 7f90b3400700 1 -- 172.21.15.1:0/1848989344 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5564f5635930 msgr2=0x5564f564b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:22.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.018+0000 7f90b3400700 1 --2- 172.21.15.1:0/1848989344 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5564f5635930 0x5564f564b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.018+0000 7f90b3400700 1 -- 172.21.15.1:0/1848989344 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5564f554cea0 con 0x5564f5635010 2024-02-02T15:27:22.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.018+0000 7f90b3400700 1 --2- 172.21.15.1:0/1848989344 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5564f5635010 0x5564f5637b10 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f90a4095d00 tx=0x7f90a4010c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=effa58137973386e server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:22.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.018+0000 7f90a9600700 1 -- 172.21.15.1:0/1848989344 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a4093ea0 con 0x5564f5635010 2024-02-02T15:27:22.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.018+0000 7f90a9600700 1 -- 172.21.15.1:0/1848989344 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f90a4099290 con 0x5564f5635010 2024-02-02T15:27:22.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.018+0000 7f90a9600700 1 -- 172.21.15.1:0/1848989344 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90a40989e0 con 0x5564f5635010 2024-02-02T15:27:22.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.019+0000 7f90ce22bb80 1 -- 172.21.15.1:0/1848989344 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5564f5635010 msgr2=0x5564f5637b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:22.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.019+0000 7f90ce22bb80 1 --2- 172.21.15.1:0/1848989344 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5564f5635010 0x5564f5637b10 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f90a4095d00 tx=0x7f90a4010c60 comp rx=0 tx=0).stop 2024-02-02T15:27:22.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.019+0000 7f90ce22bb80 1 -- 172.21.15.1:0/1848989344 shutdown_connections 2024-02-02T15:27:22.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.019+0000 7f90ce22bb80 1 --2- 172.21.15.1:0/1848989344 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5564f56407e0 0x5564f564bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.019+0000 7f90ce22bb80 1 --2- 172.21.15.1:0/1848989344 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5564f5635010 0x5564f5637b10 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.019+0000 7f90ce22bb80 1 --2- 172.21.15.1:0/1848989344 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5564f5635930 0x5564f564b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.019+0000 7f90ce22bb80 1 -- 172.21.15.1:0/1848989344 >> 172.21.15.1:0/1848989344 conn(0x5564f5594db0 msgr2=0x5564f564c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:22.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.019+0000 7f90ce22bb80 1 -- 172.21.15.1:0/1848989344 shutdown_connections 2024-02-02T15:27:22.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.019+0000 7f90ce22bb80 1 -- 172.21.15.1:0/1848989344 wait complete. 2024-02-02T15:27:22.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.019+0000 7f90ce22bb80 1 -- start start 2024-02-02T15:27:22.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.020+0000 7f90ce22bb80 1 Processor -- start 2024-02-02T15:27:22.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.020+0000 7f90ce22bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5564f5635010 0x5564f5644f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:22.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.020+0000 7f90ce22bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5564f5635930 0x5564f56454a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:22.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.020+0000 7f90ce22bb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5564f56407e0 0x5564f56a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:22.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.020+0000 7f90ce22bb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564f56a12a0 con 0x5564f56407e0 2024-02-02T15:27:22.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.020+0000 7f90ce22bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564f56a13e0 con 0x5564f5635010 2024-02-02T15:27:22.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.020+0000 7f90ce22bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564f56a15a0 con 0x5564f5635930 2024-02-02T15:27:22.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.020+0000 7f90ce22bb80 10 client.0 _ll_register_callbacks cb 0x5564f5524290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:27:22.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.020+0000 7f90b3e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5564f56407e0 0x5564f56a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:22.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.020+0000 7f90b3e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5564f56407e0 0x5564f56a0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43676/0 (socket says 172.21.15.1:43676) 2024-02-02T15:27:22.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.020+0000 7f90b3e00700 1 -- 172.21.15.1:0/3643516507 learned_addr learned my addr 172.21.15.1:0/3643516507 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:22.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.020+0000 7f90b2a00700 1 --2- 172.21.15.1:0/3643516507 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5564f5635930 0x5564f56454a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:22.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.020+0000 7f90b3400700 1 --2- 172.21.15.1:0/3643516507 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5564f5635010 0x5564f5644f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:22.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.021+0000 7f90b3e00700 1 -- 172.21.15.1:0/3643516507 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5564f5635930 msgr2=0x5564f56454a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:22.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.021+0000 7f90b3e00700 1 --2- 172.21.15.1:0/3643516507 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5564f5635930 0x5564f56454a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.021+0000 7f90b3e00700 1 -- 172.21.15.1:0/3643516507 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5564f5635010 msgr2=0x5564f5644f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:22.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.021+0000 7f90b3e00700 1 --2- 172.21.15.1:0/3643516507 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5564f5635010 0x5564f5644f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.021+0000 7f90b3e00700 1 -- 172.21.15.1:0/3643516507 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5564f54083b0 con 0x5564f56407e0 2024-02-02T15:27:22.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.021+0000 7f90b3e00700 1 --2- 172.21.15.1:0/3643516507 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5564f56407e0 0x5564f56a0d60 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f90ac00ebf0 tx=0x7f90ac006060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1abb3e448afd6a5f server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:22.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.021+0000 7f90a1600700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:27:22.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.022+0000 7f90a1600700 1 -- 172.21.15.1:0/3643516507 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90ac008240 con 0x5564f56407e0 2024-02-02T15:27:22.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.022+0000 7f90a1600700 1 -- 172.21.15.1:0/3643516507 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f90ac01aad0 con 0x5564f56407e0 2024-02-02T15:27:22.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.022+0000 7f90ce22bb80 1 -- 172.21.15.1:0/3643516507 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f90a4099990 con 0x5564f56407e0 2024-02-02T15:27:22.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.022+0000 7f90a1600700 1 -- 172.21.15.1:0/3643516507 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f90ac0083a0 con 0x5564f56407e0 2024-02-02T15:27:22.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.022+0000 7f90ce22bb80 10 client.15717 fetch_fsmap learned FSMap version 27 2024-02-02T15:27:22.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.022+0000 7f90ce22bb80 1 -- 172.21.15.1:0/3643516507 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5564f540ccd0 con 0x5564f56407e0 2024-02-02T15:27:22.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.022+0000 7f90a1600700 1 -- 172.21.15.1:0/3643516507 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f90a4099990 con 0x5564f56407e0 2024-02-02T15:27:22.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.022+0000 7f90a1600700 1 -- 172.21.15.1:0/3643516507 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f90ac00ec20 con 0x5564f56407e0 2024-02-02T15:27:22.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.022+0000 7f90ce22bb80 10 client.15717 fetch_fsmap finished waiting for FSMap version 27 2024-02-02T15:27:22.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.022+0000 7f90ce22bb80 10 client.15717 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-02-02T15:27:22.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.022+0000 7f90ce22bb80 20 client.15717 populate_metadata read hostname 'smithi001' 2024-02-02T15:27:22.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.022+0000 7f90ce22bb80 1 --2- 172.21.15.1:0/3643516507 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x5564f56a2c10 0x5564f56a5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:22.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.022+0000 7f90ce22bb80 4 client.15717 mds_command: new command op to 24518 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:27:22.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.022+0000 7f90ce22bb80 1 -- 172.21.15.1:0/3643516507 --> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5564f5545f50 con 0x5564f56a2c10 2024-02-02T15:27:22.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.023+0000 7f90b3400700 1 --2- 172.21.15.1:0/3643516507 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x5564f56a2c10 0x5564f56a5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:22.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.023+0000 7f90b3400700 1 --2- 172.21.15.1:0/3643516507 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x5564f56a2c10 0x5564f56a5030 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c6ff223894462a71 server_cookie=7d51ad9e8576141b in_seq=0 out_seq=0 2024-02-02T15:27:22.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.023+0000 7f90a1600700 10 client.15717 ms_handle_connect on v2:172.21.15.71:6836/628016199 2024-02-02T15:27:22.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.024+0000 7f90a1600700 1 -- 172.21.15.1:0/3643516507 <== mds.2 v2:172.21.15.71:6836/628016199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5564f5545f50 con 0x5564f56a2c10 2024-02-02T15:27:22.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.024+0000 7f90a1600700 10 client.15717 handle_command_reply: tid=0 2024-02-02T15:27:22.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.024+0000 7f90ce22bb80 1 client.15717 shutdown 2024-02-02T15:27:22.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.024+0000 7f90ce22bb80 1 -- 172.21.15.1:0/3643516507 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5564f56407e0 msgr2=0x5564f56a0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:22.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.024+0000 7f90ce22bb80 1 --2- 172.21.15.1:0/3643516507 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5564f56407e0 0x5564f56a0d60 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f90ac00ebf0 tx=0x7f90ac006060 comp rx=0 tx=0).stop 2024-02-02T15:27:22.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.024+0000 7f90ce22bb80 1 -- 172.21.15.1:0/3643516507 shutdown_connections 2024-02-02T15:27:22.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.024+0000 7f90ce22bb80 1 --2- 172.21.15.1:0/3643516507 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5564f56407e0 0x5564f56a0d60 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.024+0000 7f90ce22bb80 1 --2- 172.21.15.1:0/3643516507 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5564f5635010 0x5564f5644f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.024+0000 7f90ce22bb80 1 --2- 172.21.15.1:0/3643516507 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x5564f56a2c10 0x5564f56a5030 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.024+0000 7f90ce22bb80 1 --2- 172.21.15.1:0/3643516507 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5564f5635930 0x5564f56454a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.024+0000 7f90ce22bb80 1 -- 172.21.15.1:0/3643516507 >> 172.21.15.1:0/3643516507 conn(0x5564f5594db0 msgr2=0x5564f5638700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:22.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.024+0000 7f90a1600700 0 client.15717 ms_handle_reset on v2:172.21.15.71:6836/628016199 2024-02-02T15:27:22.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.024+0000 7f90ce22bb80 1 -- 172.21.15.1:0/3643516507 shutdown_connections 2024-02-02T15:27:22.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.024+0000 7f90ce22bb80 1 -- 172.21.15.1:0/3643516507 wait complete. 2024-02-02T15:27:22.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.024+0000 7f90ce22bb80 20 client.15717 trim_cache size 0 max 16384 2024-02-02T15:27:22.028 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:27:22.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.025+0000 7f90ba800700 1 -- 172.21.15.1:0/1859583157 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f90940a07a0 msgr2=0x7f90940a2c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:22.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.025+0000 7f90ba800700 1 --2- 172.21.15.1:0/1859583157 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f90940a07a0 0x7f90940a2c60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f90ac009fd0 tx=0x7f90ac009920 comp rx=0 tx=0).stop 2024-02-02T15:27:22.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.025+0000 7f90ba800700 1 -- 172.21.15.1:0/1859583157 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90b4100ff0 msgr2=0x7f90b4199130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:22.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.025+0000 7f90ba800700 1 --2- 172.21.15.1:0/1859583157 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90b4100ff0 0x7f90b4199130 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f90a400e9c0 tx=0x7f90a400ecd0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.025+0000 7f90ba800700 1 -- 172.21.15.1:0/1859583157 shutdown_connections 2024-02-02T15:27:22.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.025+0000 7f90ba800700 1 --2- 172.21.15.1:0/1859583157 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f90940a07a0 0x7f90940a2c60 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.025+0000 7f90ba800700 1 --2- 172.21.15.1:0/1859583157 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90b419dad0 0x7f90b419df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.025+0000 7f90ba800700 1 --2- 172.21.15.1:0/1859583157 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90b4100ff0 0x7f90b4199130 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.025+0000 7f90ba800700 1 --2- 172.21.15.1:0/1859583157 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90b4106110 0x7f90b4199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.025+0000 7f90ba800700 1 -- 172.21.15.1:0/1859583157 >> 172.21.15.1:0/1859583157 conn(0x7f90b40fc810 msgr2=0x7f90b40fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:22.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.025+0000 7f90ba800700 1 -- 172.21.15.1:0/1859583157 shutdown_connections 2024-02-02T15:27:22.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.025+0000 7f90ba800700 1 -- 172.21.15.1:0/1859583157 wait complete. 2024-02-02T15:27:22.043 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-02T15:27:22.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.243+0000 7fefcb600700 1 -- 172.21.15.1:0/4259076932 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fefc4073a30 msgr2=0x7fefc410d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:22.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.243+0000 7fefcb600700 1 --2- 172.21.15.1:0/4259076932 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fefc4073a30 0x7fefc410d0f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fefac009990 tx=0x7fefac009ca0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.243+0000 7fefcb600700 1 -- 172.21.15.1:0/4259076932 shutdown_connections 2024-02-02T15:27:22.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.243+0000 7fefcb600700 1 --2- 172.21.15.1:0/4259076932 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fefc410d630 0x7fefc410fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.243+0000 7fefcb600700 1 --2- 172.21.15.1:0/4259076932 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fefc4073110 0x7fefc40734f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.243+0000 7fefcb600700 1 --2- 172.21.15.1:0/4259076932 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fefc4073a30 0x7fefc410d0f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.243+0000 7fefcb600700 1 -- 172.21.15.1:0/4259076932 >> 172.21.15.1:0/4259076932 conn(0x7fefc40fc840 msgr2=0x7fefc40fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:22.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.243+0000 7fefcb600700 1 -- 172.21.15.1:0/4259076932 shutdown_connections 2024-02-02T15:27:22.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.243+0000 7fefcb600700 1 -- 172.21.15.1:0/4259076932 wait complete. 2024-02-02T15:27:22.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.244+0000 7fefcb600700 1 Processor -- start 2024-02-02T15:27:22.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.244+0000 7fefcb600700 1 -- start start 2024-02-02T15:27:22.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.245+0000 7fefcb600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fefc4073110 0x7fefc4199140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:22.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.245+0000 7fefcb600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fefc4073a30 0x7fefc4199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:22.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.245+0000 7fefcb600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fefc410d630 0x7fefc419dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:22.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.245+0000 7fefcb600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefc419e120 con 0x7fefc410d630 2024-02-02T15:27:22.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.245+0000 7fefcb600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefc419e290 con 0x7fefc4073110 2024-02-02T15:27:22.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.245+0000 7fefcb600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefc419e450 con 0x7fefc4073a30 2024-02-02T15:27:22.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.245+0000 7fefc3e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fefc4073a30 0x7fefc4199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:22.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.245+0000 7fefc9600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fefc410d630 0x7fefc419dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:22.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.245+0000 7fefc3e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fefc4073a30 0x7fefc4199680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:53884/0 (socket says 172.21.15.1:53884) 2024-02-02T15:27:22.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.245+0000 7fefc8c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fefc4073110 0x7fefc4199140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:22.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.245+0000 7fefc3e00700 1 -- 172.21.15.1:0/3081364877 learned_addr learned my addr 172.21.15.1:0/3081364877 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:22.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.246+0000 7fefc3e00700 1 -- 172.21.15.1:0/3081364877 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fefc4073110 msgr2=0x7fefc4199140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:22.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.246+0000 7fefc3e00700 1 --2- 172.21.15.1:0/3081364877 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fefc4073110 0x7fefc4199140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.246+0000 7fefc3e00700 1 -- 172.21.15.1:0/3081364877 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fefc410d630 msgr2=0x7fefc419dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:22.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.246+0000 7fefc3e00700 1 --2- 172.21.15.1:0/3081364877 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fefc410d630 0x7fefc419dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.246+0000 7fefc3e00700 1 -- 172.21.15.1:0/3081364877 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefac0095f0 con 0x7fefc4073a30 2024-02-02T15:27:22.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.246+0000 7fefc8c00700 1 --2- 172.21.15.1:0/3081364877 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fefc4073110 0x7fefc4199140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:27:22.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.246+0000 7fefc9600700 1 --2- 172.21.15.1:0/3081364877 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fefc410d630 0x7fefc419dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:27:22.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.246+0000 7fefc3e00700 1 --2- 172.21.15.1:0/3081364877 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fefc4073a30 0x7fefc4199680 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fefac005910 tx=0x7fefac005720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:22.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.246+0000 7fefc1600700 1 -- 172.21.15.1:0/3081364877 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefac01d070 con 0x7fefc4073a30 2024-02-02T15:27:22.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.246+0000 7fefc1600700 1 -- 172.21.15.1:0/3081364877 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fefac022470 con 0x7fefc4073a30 2024-02-02T15:27:22.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.246+0000 7fefc1600700 1 -- 172.21.15.1:0/3081364877 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefac017650 con 0x7fefc4073a30 2024-02-02T15:27:22.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.246+0000 7fefcb600700 1 -- 172.21.15.1:0/3081364877 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefc419e6d0 con 0x7fefc4073a30 2024-02-02T15:27:22.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.246+0000 7fefcb600700 1 -- 172.21.15.1:0/3081364877 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefc41a5f70 con 0x7fefc4073a30 2024-02-02T15:27:22.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.250+0000 7fefc1600700 1 -- 172.21.15.1:0/3081364877 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) v1 ==== 135245+0+0 (secure 0 0 0) 0x7fefac0225e0 con 0x7fefc4073a30 2024-02-02T15:27:22.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.251+0000 7fefdf036b80 1 Processor -- start 2024-02-02T15:27:22.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.251+0000 7fefdf036b80 1 -- start start 2024-02-02T15:27:22.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.251+0000 7fefdf036b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555d0a83f010 0x555d0a83f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:22.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.251+0000 7fefdf036b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555d0a83f930 0x555d0a84a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:22.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.251+0000 7fefdf036b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555d0a84a7e0 0x555d0a84cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:22.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.251+0000 7fefdf036b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d0a7b34c0 con 0x555d0a84a7e0 2024-02-02T15:27:22.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.251+0000 7fefdf036b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d0a797aa0 con 0x555d0a83f930 2024-02-02T15:27:22.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.251+0000 7fefdf036b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d0a795460 con 0x555d0a83f010 2024-02-02T15:27:22.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.252+0000 7fefc9600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555d0a83f010 0x555d0a83f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:22.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.252+0000 7fefc9600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555d0a83f010 0x555d0a83f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:53900/0 (socket says 172.21.15.1:53900) 2024-02-02T15:27:22.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.252+0000 7fefc9600700 1 -- 172.21.15.1:0/916675040 learned_addr learned my addr 172.21.15.1:0/916675040 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:22.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.252+0000 7fefc8c00700 1 --2- 172.21.15.1:0/916675040 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555d0a83f930 0x555d0a84a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:22.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.252+0000 7fefc3e00700 1 --2- 172.21.15.1:0/916675040 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555d0a84a7e0 0x555d0a84cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:22.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.252+0000 7fefc9600700 1 -- 172.21.15.1:0/916675040 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555d0a83f930 msgr2=0x555d0a84a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:22.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.252+0000 7fefc9600700 1 --2- 172.21.15.1:0/916675040 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555d0a83f930 0x555d0a84a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.252+0000 7fefc9600700 1 -- 172.21.15.1:0/916675040 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555d0a84a7e0 msgr2=0x555d0a84cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:22.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.252+0000 7fefc9600700 1 --2- 172.21.15.1:0/916675040 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555d0a84a7e0 0x555d0a84cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.252+0000 7fefc9600700 1 -- 172.21.15.1:0/916675040 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d0a6123b0 con 0x555d0a83f010 2024-02-02T15:27:22.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.252+0000 7fefc1600700 1 --2- 172.21.15.1:0/3081364877 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fefa40a07a0 0x7fefa40a2c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:22.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.252+0000 7fefc9600700 1 --2- 172.21.15.1:0/916675040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555d0a83f010 0x555d0a83f3f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fefbc003e70 tx=0x7fefbc009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39a663a44f16b97c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:22.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.252+0000 7fefc1600700 1 -- 172.21.15.1:0/3081364877 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fefac0cd510 con 0x7fefc4073a30 2024-02-02T15:27:22.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.253+0000 7fefc8c00700 1 --2- 172.21.15.1:0/3081364877 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fefa40a07a0 0x7fefa40a2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:22.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.253+0000 7fefb9c00700 1 -- 172.21.15.1:0/916675040 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefbc0092d0 con 0x555d0a83f010 2024-02-02T15:27:22.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.253+0000 7fefb9c00700 1 -- 172.21.15.1:0/916675040 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fefbc00f040 con 0x555d0a83f010 2024-02-02T15:27:22.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.253+0000 7fefb9c00700 1 -- 172.21.15.1:0/916675040 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefbc00e7e0 con 0x555d0a83f010 2024-02-02T15:27:22.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.253+0000 7fefdf036b80 1 -- 172.21.15.1:0/916675040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555d0a83f010 msgr2=0x555d0a83f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:22.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.253+0000 7fefc8c00700 1 --2- 172.21.15.1:0/3081364877 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fefa40a07a0 0x7fefa40a2c60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fefb4012680 tx=0x7fefb4009260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:22.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.253+0000 7fefdf036b80 1 --2- 172.21.15.1:0/916675040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555d0a83f010 0x555d0a83f3f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fefbc003e70 tx=0x7fefbc009920 comp rx=0 tx=0).stop 2024-02-02T15:27:22.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.253+0000 7fefdf036b80 1 -- 172.21.15.1:0/916675040 shutdown_connections 2024-02-02T15:27:22.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.253+0000 7fefdf036b80 1 --2- 172.21.15.1:0/916675040 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555d0a84a7e0 0x555d0a84cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.254+0000 7fefdf036b80 1 --2- 172.21.15.1:0/916675040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555d0a83f010 0x555d0a83f3f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.254+0000 7fefdf036b80 1 --2- 172.21.15.1:0/916675040 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555d0a83f930 0x555d0a84a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.254+0000 7fefdf036b80 1 -- 172.21.15.1:0/916675040 >> 172.21.15.1:0/916675040 conn(0x555d0a79edb0 msgr2=0x555d0a83b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:22.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.254+0000 7fefdf036b80 1 -- 172.21.15.1:0/916675040 shutdown_connections 2024-02-02T15:27:22.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.254+0000 7fefdf036b80 1 -- 172.21.15.1:0/916675040 wait complete. 2024-02-02T15:27:22.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.254+0000 7fefdf036b80 1 -- start start 2024-02-02T15:27:22.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.254+0000 7fefdf036b80 1 Processor -- start 2024-02-02T15:27:22.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.255+0000 7fefdf036b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555d0a83f010 0x555d0a84a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:22.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.255+0000 7fefdf036b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555d0a83f930 0x555d0a847360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:22.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.255+0000 7fefdf036b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555d0a84a7e0 0x555d0a8478e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:22.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.255+0000 7fefdf036b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d0a790d50 con 0x555d0a83f010 2024-02-02T15:27:22.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.255+0000 7fefdf036b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d0a72cd80 con 0x555d0a83f930 2024-02-02T15:27:22.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.255+0000 7fefdf036b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d0a72ca20 con 0x555d0a84a7e0 2024-02-02T15:27:22.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.255+0000 7fefdf036b80 10 client.0 _ll_register_callbacks cb 0x555d0a72e290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:27:22.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.255+0000 7fefc8c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555d0a83f010 0x555d0a84a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:22.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.255+0000 7fefc9600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555d0a84a7e0 0x555d0a8478e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:22.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.255+0000 7fefc8c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555d0a83f010 0x555d0a84a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43716/0 (socket says 172.21.15.1:43716) 2024-02-02T15:27:22.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.255+0000 7fefc8c00700 1 -- 172.21.15.1:0/2511092060 learned_addr learned my addr 172.21.15.1:0/2511092060 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:22.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.255+0000 7fefc3e00700 1 --2- 172.21.15.1:0/2511092060 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555d0a83f930 0x555d0a847360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:22.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.255+0000 7fefc9600700 1 -- 172.21.15.1:0/2511092060 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555d0a83f930 msgr2=0x555d0a847360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:22.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.255+0000 7fefc9600700 1 --2- 172.21.15.1:0/2511092060 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555d0a83f930 0x555d0a847360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.255+0000 7fefc9600700 1 -- 172.21.15.1:0/2511092060 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555d0a83f010 msgr2=0x555d0a84a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:22.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.255+0000 7fefc9600700 1 --2- 172.21.15.1:0/2511092060 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555d0a83f010 0x555d0a84a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.255+0000 7fefc9600700 1 -- 172.21.15.1:0/2511092060 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefbc003b20 con 0x555d0a84a7e0 2024-02-02T15:27:22.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.255+0000 7fefc8c00700 1 --2- 172.21.15.1:0/2511092060 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555d0a83f010 0x555d0a84a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:27:22.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.256+0000 7fefc9600700 1 --2- 172.21.15.1:0/2511092060 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555d0a84a7e0 0x555d0a8478e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fefbc004600 tx=0x7fefbc004d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7594468add24a9d server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:22.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.256+0000 7fefb2000700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:27:22.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.256+0000 7fefb2000700 1 -- 172.21.15.1:0/2511092060 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefbc007ea0 con 0x555d0a84a7e0 2024-02-02T15:27:22.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.256+0000 7fefdf036b80 1 -- 172.21.15.1:0/2511092060 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555d0a8330f0 con 0x555d0a84a7e0 2024-02-02T15:27:22.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.256+0000 7fefb2000700 1 -- 172.21.15.1:0/2511092060 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fefbc007910 con 0x555d0a84a7e0 2024-02-02T15:27:22.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.256+0000 7fefb2000700 1 -- 172.21.15.1:0/2511092060 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fefbc007ad0 con 0x555d0a84a7e0 2024-02-02T15:27:22.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.256+0000 7fefdf036b80 10 client.25765 fetch_fsmap learned FSMap version 27 2024-02-02T15:27:22.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.256+0000 7fefdf036b80 1 -- 172.21.15.1:0/2511092060 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x555d0a5beb60 con 0x555d0a84a7e0 2024-02-02T15:27:22.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.257+0000 7fefb2000700 1 -- 172.21.15.1:0/2511092060 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555d0a8330f0 con 0x555d0a84a7e0 2024-02-02T15:27:22.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.257+0000 7fefb2000700 1 -- 172.21.15.1:0/2511092060 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fefbc015610 con 0x555d0a84a7e0 2024-02-02T15:27:22.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.257+0000 7fefdf036b80 10 client.25765 fetch_fsmap finished waiting for FSMap version 27 2024-02-02T15:27:22.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.257+0000 7fefdf036b80 10 client.25765 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-02T15:27:22.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.257+0000 7fefdf036b80 20 client.25765 populate_metadata read hostname 'smithi001' 2024-02-02T15:27:22.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.257+0000 7fefdf036b80 1 --2- 172.21.15.1:0/2511092060 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x555d0a85a190 0x555d0a85c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:22.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.257+0000 7fefdf036b80 4 client.25765 mds_command: new command op to 24529 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:27:22.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.257+0000 7fefdf036b80 1 -- 172.21.15.1:0/2511092060 --> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555d0a789dd0 con 0x555d0a85a190 2024-02-02T15:27:22.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.257+0000 7fefc8c00700 1 --2- 172.21.15.1:0/2511092060 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x555d0a85a190 0x555d0a85c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:22.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.258+0000 7fefc8c00700 1 --2- 172.21.15.1:0/2511092060 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x555d0a85a190 0x555d0a85c5b0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c888e6fa85c85219 server_cookie=7eb31b7446d45762 in_seq=0 out_seq=0 2024-02-02T15:27:22.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.258+0000 7fefb2000700 10 client.25765 ms_handle_connect on v2:172.21.15.142:6838/321675421 2024-02-02T15:27:22.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.259+0000 7fefb2000700 1 -- 172.21.15.1:0/2511092060 <== mds.1 v2:172.21.15.142:6838/321675421 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x555d0a789dd0 con 0x555d0a85a190 2024-02-02T15:27:22.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.259+0000 7fefb2000700 10 client.25765 handle_command_reply: tid=0 2024-02-02T15:27:22.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.259+0000 7fefdf036b80 1 client.25765 shutdown 2024-02-02T15:27:22.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.259+0000 7fefdf036b80 1 -- 172.21.15.1:0/2511092060 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555d0a84a7e0 msgr2=0x555d0a8478e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:22.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.259+0000 7fefdf036b80 1 --2- 172.21.15.1:0/2511092060 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555d0a84a7e0 0x555d0a8478e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fefbc004600 tx=0x7fefbc004d80 comp rx=0 tx=0).stop 2024-02-02T15:27:22.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.259+0000 7fefdf036b80 1 -- 172.21.15.1:0/2511092060 shutdown_connections 2024-02-02T15:27:22.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.259+0000 7fefdf036b80 1 --2- 172.21.15.1:0/2511092060 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x555d0a85a190 0x555d0a85c5b0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.259+0000 7fefdf036b80 1 --2- 172.21.15.1:0/2511092060 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555d0a84a7e0 0x555d0a8478e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.259+0000 7fefdf036b80 1 --2- 172.21.15.1:0/2511092060 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555d0a83f010 0x555d0a84a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.259+0000 7fefb2000700 0 client.25765 ms_handle_reset on v2:172.21.15.142:6838/321675421 2024-02-02T15:27:22.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.259+0000 7fefdf036b80 1 --2- 172.21.15.1:0/2511092060 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555d0a83f930 0x555d0a847360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.259+0000 7fefdf036b80 1 -- 172.21.15.1:0/2511092060 >> 172.21.15.1:0/2511092060 conn(0x555d0a79edb0 msgr2=0x555d0a83afa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:22.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.259+0000 7fefdf036b80 1 -- 172.21.15.1:0/2511092060 shutdown_connections 2024-02-02T15:27:22.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.259+0000 7fefdf036b80 1 -- 172.21.15.1:0/2511092060 wait complete. 2024-02-02T15:27:22.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.259+0000 7fefdf036b80 20 client.25765 trim_cache size 0 max 16384 2024-02-02T15:27:22.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.273+0000 7fefdf036b80 1 Processor -- start 2024-02-02T15:27:22.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.273+0000 7fefdf036b80 1 -- start start 2024-02-02T15:27:22.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.273+0000 7fefdf036b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555d0a83f010 0x555d0a85c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:22.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.273+0000 7fefdf036b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555d0a83f930 0x555d0a8445d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:22.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.273+0000 7fefdf036b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555d0a84a7e0 0x555d0a844b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:22.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.273+0000 7fefdf036b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d0a6146a0 con 0x555d0a84a7e0 2024-02-02T15:27:22.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.274+0000 7fefdf036b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d0a845940 con 0x555d0a83f010 2024-02-02T15:27:22.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.274+0000 7fefdf036b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d0a8431e0 con 0x555d0a83f930 2024-02-02T15:27:22.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.274+0000 7fefc9600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555d0a84a7e0 0x555d0a844b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:22.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.274+0000 7fefc9600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555d0a84a7e0 0x555d0a844b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43724/0 (socket says 172.21.15.1:43724) 2024-02-02T15:27:22.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.274+0000 7fefc9600700 1 -- 172.21.15.1:0/3845872089 learned_addr learned my addr 172.21.15.1:0/3845872089 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:22.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.274+0000 7fefc8c00700 1 --2- 172.21.15.1:0/3845872089 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555d0a83f010 0x555d0a85c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:22.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.274+0000 7fefc3e00700 1 --2- 172.21.15.1:0/3845872089 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555d0a83f930 0x555d0a8445d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:22.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.274+0000 7fefc9600700 1 -- 172.21.15.1:0/3845872089 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555d0a83f930 msgr2=0x555d0a8445d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:22.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.274+0000 7fefc9600700 1 --2- 172.21.15.1:0/3845872089 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555d0a83f930 0x555d0a8445d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.274+0000 7fefc9600700 1 -- 172.21.15.1:0/3845872089 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555d0a83f010 msgr2=0x555d0a85c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:22.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.274+0000 7fefc9600700 1 --2- 172.21.15.1:0/3845872089 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555d0a83f010 0x555d0a85c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.274+0000 7fefc9600700 1 -- 172.21.15.1:0/3845872089 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d0a756ea0 con 0x555d0a84a7e0 2024-02-02T15:27:22.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.274+0000 7fefc8c00700 1 --2- 172.21.15.1:0/3845872089 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555d0a83f010 0x555d0a85c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:27:22.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.274+0000 7fefc9600700 1 --2- 172.21.15.1:0/3845872089 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555d0a84a7e0 0x555d0a844b10 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fefbc0159f0 tx=0x7fefbc003e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0b59fac85382fd0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:22.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.275+0000 7fefba600700 1 -- 172.21.15.1:0/3845872089 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefbc015ea0 con 0x555d0a84a7e0 2024-02-02T15:27:22.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.275+0000 7fefba600700 1 -- 172.21.15.1:0/3845872089 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fefbc015b10 con 0x555d0a84a7e0 2024-02-02T15:27:22.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.275+0000 7fefba600700 1 -- 172.21.15.1:0/3845872089 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefbc00e750 con 0x555d0a84a7e0 2024-02-02T15:27:22.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.275+0000 7fefdf036b80 1 -- 172.21.15.1:0/3845872089 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555d0a84a7e0 msgr2=0x555d0a844b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:22.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.275+0000 7fefdf036b80 1 --2- 172.21.15.1:0/3845872089 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555d0a84a7e0 0x555d0a844b10 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fefbc0159f0 tx=0x7fefbc003e70 comp rx=0 tx=0).stop 2024-02-02T15:27:22.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.275+0000 7fefdf036b80 1 -- 172.21.15.1:0/3845872089 shutdown_connections 2024-02-02T15:27:22.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.275+0000 7fefdf036b80 1 --2- 172.21.15.1:0/3845872089 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555d0a84a7e0 0x555d0a844b10 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.276+0000 7fefdf036b80 1 --2- 172.21.15.1:0/3845872089 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555d0a83f010 0x555d0a85c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.276+0000 7fefdf036b80 1 --2- 172.21.15.1:0/3845872089 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555d0a83f930 0x555d0a8445d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.276+0000 7fefdf036b80 1 -- 172.21.15.1:0/3845872089 >> 172.21.15.1:0/3845872089 conn(0x555d0a79edb0 msgr2=0x555d0a839b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:22.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.276+0000 7fefdf036b80 1 -- 172.21.15.1:0/3845872089 shutdown_connections 2024-02-02T15:27:22.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.276+0000 7fefdf036b80 1 -- 172.21.15.1:0/3845872089 wait complete. 2024-02-02T15:27:22.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.276+0000 7fefdf036b80 1 -- start start 2024-02-02T15:27:22.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.276+0000 7fefdf036b80 1 Processor -- start 2024-02-02T15:27:22.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.277+0000 7fefdf036b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555d0a83f010 0x555d0a850390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:22.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.277+0000 7fefdf036b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555d0a83f930 0x555d0a8508d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:22.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.277+0000 7fefdf036b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555d0a84a7e0 0x555d0a8a2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:22.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.277+0000 7fefdf036b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d0a8a3250 con 0x555d0a83f930 2024-02-02T15:27:22.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.277+0000 7fefdf036b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d0a8a33c0 con 0x555d0a84a7e0 2024-02-02T15:27:22.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.277+0000 7fefdf036b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d0a8a3580 con 0x555d0a83f010 2024-02-02T15:27:22.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.277+0000 7fefdf036b80 10 client.0 _ll_register_callbacks cb 0x555d0a72e290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:27:22.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.277+0000 7fefc3e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555d0a83f930 0x555d0a8508d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:22.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.277+0000 7fefc8c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555d0a83f010 0x555d0a850390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:22.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.277+0000 7fefc3e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555d0a83f930 0x555d0a8508d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43728/0 (socket says 172.21.15.1:43728) 2024-02-02T15:27:22.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.277+0000 7fefc9600700 1 --2- 172.21.15.1:0/2803732402 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555d0a84a7e0 0x555d0a8a2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:22.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.277+0000 7fefc3e00700 1 -- 172.21.15.1:0/2803732402 learned_addr learned my addr 172.21.15.1:0/2803732402 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:22.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.277+0000 7fefc8c00700 1 -- 172.21.15.1:0/2803732402 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555d0a84a7e0 msgr2=0x555d0a8a2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:22.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.277+0000 7fefc8c00700 1 --2- 172.21.15.1:0/2803732402 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555d0a84a7e0 0x555d0a8a2ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.277+0000 7fefc8c00700 1 -- 172.21.15.1:0/2803732402 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555d0a83f930 msgr2=0x555d0a8508d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:22.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.278+0000 7fefc8c00700 1 --2- 172.21.15.1:0/2803732402 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555d0a83f930 0x555d0a8508d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.278+0000 7fefc8c00700 1 -- 172.21.15.1:0/2803732402 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefb40038a0 con 0x555d0a83f010 2024-02-02T15:27:22.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.278+0000 7fefc9600700 1 --2- 172.21.15.1:0/2803732402 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555d0a84a7e0 0x555d0a8a2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:27:22.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.278+0000 7fefc3e00700 1 --2- 172.21.15.1:0/2803732402 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555d0a83f930 0x555d0a8508d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:27:22.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.278+0000 7fefc8c00700 1 --2- 172.21.15.1:0/2803732402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555d0a83f010 0x555d0a850390 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x555d0a85a570 tx=0x7fefb4004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8cdaab389d0f352 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:22.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.278+0000 7fefb2000700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:27:22.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.278+0000 7fefb2000700 1 -- 172.21.15.1:0/2803732402 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefb4024040 con 0x555d0a83f010 2024-02-02T15:27:22.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.278+0000 7fefb2000700 1 -- 172.21.15.1:0/2803732402 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fefb4012020 con 0x555d0a83f010 2024-02-02T15:27:22.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.278+0000 7fefdf036b80 1 -- 172.21.15.1:0/2803732402 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fefbc00f040 con 0x555d0a83f010 2024-02-02T15:27:22.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.278+0000 7fefb2000700 1 -- 172.21.15.1:0/2803732402 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fefb4017070 con 0x555d0a83f010 2024-02-02T15:27:22.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.278+0000 7fefdf036b80 10 client.25771 fetch_fsmap learned FSMap version 27 2024-02-02T15:27:22.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.278+0000 7fefdf036b80 1 -- 172.21.15.1:0/2803732402 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x555d0a5beb60 con 0x555d0a83f010 2024-02-02T15:27:22.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.279+0000 7fefb2000700 1 -- 172.21.15.1:0/2803732402 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fefbc00f040 con 0x555d0a83f010 2024-02-02T15:27:22.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.279+0000 7fefb2000700 1 -- 172.21.15.1:0/2803732402 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fefb40116e0 con 0x555d0a83f010 2024-02-02T15:27:22.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.279+0000 7fefdf036b80 10 client.25771 fetch_fsmap finished waiting for FSMap version 27 2024-02-02T15:27:22.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.279+0000 7fefdf036b80 10 client.25771 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-02T15:27:22.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.279+0000 7fefdf036b80 20 client.25771 populate_metadata read hostname 'smithi001' 2024-02-02T15:27:22.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.279+0000 7fefdf036b80 1 --2- 172.21.15.1:0/2803732402 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x555d0a8abf00 0x555d0a8ae320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:22.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.279+0000 7fefdf036b80 4 client.25771 mds_command: new command op to 24529 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:27:22.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.279+0000 7fefdf036b80 1 -- 172.21.15.1:0/2803732402 --> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555d0a74ff50 con 0x555d0a8abf00 2024-02-02T15:27:22.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.279+0000 7fefc3e00700 1 --2- 172.21.15.1:0/2803732402 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x555d0a8abf00 0x555d0a8ae320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:22.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.280+0000 7fefc3e00700 1 --2- 172.21.15.1:0/2803732402 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x555d0a8abf00 0x555d0a8ae320 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e33d2cc2fd9a7498 server_cookie=5e135f9e9b89680f in_seq=0 out_seq=0 2024-02-02T15:27:22.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.280+0000 7fefb2000700 10 client.25771 ms_handle_connect on v2:172.21.15.142:6838/321675421 2024-02-02T15:27:22.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.280+0000 7fefb2000700 1 -- 172.21.15.1:0/2803732402 <== mds.1 v2:172.21.15.142:6838/321675421 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555d0a74ff50 con 0x555d0a8abf00 2024-02-02T15:27:22.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.280+0000 7fefb2000700 10 client.25771 handle_command_reply: tid=0 2024-02-02T15:27:22.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.280+0000 7fefdf036b80 1 client.25771 shutdown 2024-02-02T15:27:22.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.280+0000 7fefdf036b80 1 -- 172.21.15.1:0/2803732402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555d0a83f010 msgr2=0x555d0a850390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:22.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.280+0000 7fefdf036b80 1 --2- 172.21.15.1:0/2803732402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555d0a83f010 0x555d0a850390 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x555d0a85a570 tx=0x7fefb4004750 comp rx=0 tx=0).stop 2024-02-02T15:27:22.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.280+0000 7fefdf036b80 1 -- 172.21.15.1:0/2803732402 shutdown_connections 2024-02-02T15:27:22.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.280+0000 7fefdf036b80 1 --2- 172.21.15.1:0/2803732402 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x555d0a8abf00 0x555d0a8ae320 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.280+0000 7fefdf036b80 1 --2- 172.21.15.1:0/2803732402 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555d0a84a7e0 0x555d0a8a2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.280+0000 7fefdf036b80 1 --2- 172.21.15.1:0/2803732402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555d0a83f010 0x555d0a850390 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.280+0000 7fefdf036b80 1 --2- 172.21.15.1:0/2803732402 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555d0a83f930 0x555d0a8508d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.280+0000 7fefb2000700 0 client.25771 ms_handle_reset on v2:172.21.15.142:6838/321675421 2024-02-02T15:27:22.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.280+0000 7fefdf036b80 1 -- 172.21.15.1:0/2803732402 >> 172.21.15.1:0/2803732402 conn(0x555d0a79edb0 msgr2=0x555d0a848c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:22.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.280+0000 7fefdf036b80 1 -- 172.21.15.1:0/2803732402 shutdown_connections 2024-02-02T15:27:22.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.281+0000 7fefdf036b80 1 -- 172.21.15.1:0/2803732402 wait complete. 2024-02-02T15:27:22.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.281+0000 7fefdf036b80 20 client.25771 trim_cache size 0 max 16384 2024-02-02T15:27:22.285 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:27:22.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.281+0000 7fefcb600700 1 -- 172.21.15.1:0/3081364877 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fefa40a07a0 msgr2=0x7fefa40a2c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:22.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.281+0000 7fefcb600700 1 --2- 172.21.15.1:0/3081364877 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fefa40a07a0 0x7fefa40a2c60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fefb4012680 tx=0x7fefb4009260 comp rx=0 tx=0).stop 2024-02-02T15:27:22.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.281+0000 7fefcb600700 1 -- 172.21.15.1:0/3081364877 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fefc4073a30 msgr2=0x7fefc4199680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:22.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.281+0000 7fefcb600700 1 --2- 172.21.15.1:0/3081364877 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fefc4073a30 0x7fefc4199680 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fefac005910 tx=0x7fefac005720 comp rx=0 tx=0).stop 2024-02-02T15:27:22.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.281+0000 7fefcb600700 1 -- 172.21.15.1:0/3081364877 shutdown_connections 2024-02-02T15:27:22.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.281+0000 7fefcb600700 1 --2- 172.21.15.1:0/3081364877 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fefa40a07a0 0x7fefa40a2c60 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.281+0000 7fefcb600700 1 --2- 172.21.15.1:0/3081364877 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fefc410d630 0x7fefc419dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.281+0000 7fefcb600700 1 --2- 172.21.15.1:0/3081364877 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fefc4073110 0x7fefc4199140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.281+0000 7fefcb600700 1 --2- 172.21.15.1:0/3081364877 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fefc4073a30 0x7fefc4199680 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.281+0000 7fefcb600700 1 -- 172.21.15.1:0/3081364877 >> 172.21.15.1:0/3081364877 conn(0x7fefc40fc840 msgr2=0x7fefc4105090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:22.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.282+0000 7fefcb600700 1 -- 172.21.15.1:0/3081364877 shutdown_connections 2024-02-02T15:27:22.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.282+0000 7fefcb600700 1 -- 172.21.15.1:0/3081364877 wait complete. 2024-02-02T15:27:22.299 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-02T15:27:22.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.500+0000 7f2e83e00700 1 -- 172.21.15.1:0/2338153480 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e84106ab0 msgr2=0x7f2e84106e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:22.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.500+0000 7f2e83e00700 1 --2- 172.21.15.1:0/2338153480 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e84106ab0 0x7f2e84106e90 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f2e74009990 tx=0x7f2e74009ca0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.500+0000 7f2e83e00700 1 -- 172.21.15.1:0/2338153480 shutdown_connections 2024-02-02T15:27:22.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.500+0000 7f2e83e00700 1 --2- 172.21.15.1:0/2338153480 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e840689d0 0x7f2e8410d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.500+0000 7f2e83e00700 1 --2- 172.21.15.1:0/2338153480 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e84106ab0 0x7f2e84106e90 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.500+0000 7f2e83e00700 1 --2- 172.21.15.1:0/2338153480 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e84068010 0x7f2e84068490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.500+0000 7f2e83e00700 1 -- 172.21.15.1:0/2338153480 >> 172.21.15.1:0/2338153480 conn(0x7f2e84076770 msgr2=0x7f2e84076b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:22.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.500+0000 7f2e83e00700 1 -- 172.21.15.1:0/2338153480 shutdown_connections 2024-02-02T15:27:22.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.501+0000 7f2e83e00700 1 -- 172.21.15.1:0/2338153480 wait complete. 2024-02-02T15:27:22.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.501+0000 7f2e83e00700 1 Processor -- start 2024-02-02T15:27:22.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.502+0000 7f2e83e00700 1 -- start start 2024-02-02T15:27:22.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.502+0000 7f2e83e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e84068010 0x7f2e8419c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:22.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.502+0000 7f2e83e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e840689d0 0x7f2e8419cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:22.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.502+0000 7f2e83e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e84106ab0 0x7f2e84196a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:22.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.502+0000 7f2e83e00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e84197090 con 0x7f2e84068010 2024-02-02T15:27:22.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.502+0000 7f2e83e00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e84197200 con 0x7f2e840689d0 2024-02-02T15:27:22.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.502+0000 7f2e83e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e841973c0 con 0x7f2e84106ab0 2024-02-02T15:27:22.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.502+0000 7f2e82a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e84068010 0x7f2e8419c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:22.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.502+0000 7f2e82a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e84068010 0x7f2e8419c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43744/0 (socket says 172.21.15.1:43744) 2024-02-02T15:27:22.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.502+0000 7f2e82a00700 1 -- 172.21.15.1:0/2711752913 learned_addr learned my addr 172.21.15.1:0/2711752913 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:22.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.502+0000 7f2e82000700 1 --2- 172.21.15.1:0/2711752913 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e840689d0 0x7f2e8419cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:22.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.502+0000 7f2e83400700 1 --2- 172.21.15.1:0/2711752913 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e84106ab0 0x7f2e84196a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:22.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.503+0000 7f2e82000700 1 -- 172.21.15.1:0/2711752913 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e84106ab0 msgr2=0x7f2e84196a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:22.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.503+0000 7f2e82000700 1 --2- 172.21.15.1:0/2711752913 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e84106ab0 0x7f2e84196a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.503+0000 7f2e82000700 1 -- 172.21.15.1:0/2711752913 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e84068010 msgr2=0x7f2e8419c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:22.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.503+0000 7f2e82000700 1 --2- 172.21.15.1:0/2711752913 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e84068010 0x7f2e8419c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.503+0000 7f2e82000700 1 -- 172.21.15.1:0/2711752913 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e740095f0 con 0x7f2e840689d0 2024-02-02T15:27:22.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.503+0000 7f2e82a00700 1 --2- 172.21.15.1:0/2711752913 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e84068010 0x7f2e8419c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:27:22.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.503+0000 7f2e82000700 1 --2- 172.21.15.1:0/2711752913 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e840689d0 0x7f2e8419cf00 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c00e9f0 tx=0x7f2e6c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:22.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.503+0000 7f2e7b400700 1 -- 172.21.15.1:0/2711752913 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e6c00c9c0 con 0x7f2e840689d0 2024-02-02T15:27:22.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.503+0000 7f2e7b400700 1 -- 172.21.15.1:0/2711752913 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e6c010430 con 0x7f2e840689d0 2024-02-02T15:27:22.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.503+0000 7f2e83e00700 1 -- 172.21.15.1:0/2711752913 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e841976a0 con 0x7f2e840689d0 2024-02-02T15:27:22.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.503+0000 7f2e7b400700 1 -- 172.21.15.1:0/2711752913 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e6c018680 con 0x7f2e840689d0 2024-02-02T15:27:22.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.504+0000 7f2e83e00700 1 -- 172.21.15.1:0/2711752913 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e841a39b0 con 0x7f2e840689d0 2024-02-02T15:27:22.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.507+0000 7f2e7b400700 1 -- 172.21.15.1:0/2711752913 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 25) v1 ==== 135245+0+0 (secure 0 0 0) 0x7f2e6c010860 con 0x7f2e840689d0 2024-02-02T15:27:22.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.508+0000 7f2e9d521b80 1 Processor -- start 2024-02-02T15:27:22.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.508+0000 7f2e9d521b80 1 -- start start 2024-02-02T15:27:22.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.508+0000 7f2e7b400700 1 --2- 172.21.15.1:0/2711752913 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f2e640a0720 0x7f2e640a2be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:22.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.508+0000 7f2e7b400700 1 -- 172.21.15.1:0/2711752913 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f2e6c014070 con 0x7f2e840689d0 2024-02-02T15:27:22.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.508+0000 7f2e9d521b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55858c3c8dd0 0x55858c3c71f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:22.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.508+0000 7f2e83400700 1 --2- 172.21.15.1:0/2711752913 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f2e640a0720 0x7f2e640a2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:22.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.508+0000 7f2e9d521b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55858c3c7730 0x55858c3c7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:22.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.508+0000 7f2e9d521b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55858c3db090 0x55858c3dd480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:22.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.508+0000 7f2e9d521b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55858c33b4c0 con 0x55858c3db090 2024-02-02T15:27:22.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.508+0000 7f2e9d521b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55858c31faa0 con 0x55858c3c7730 2024-02-02T15:27:22.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.508+0000 7f2e9d521b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55858c31d460 con 0x55858c3c8dd0 2024-02-02T15:27:22.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.509+0000 7f2e82000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55858c3c7730 0x55858c3c7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:22.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.509+0000 7f2e82a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55858c3c8dd0 0x55858c3c71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:22.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.509+0000 7f2e82000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55858c3c7730 0x55858c3c7bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59564/0 (socket says 172.21.15.1:59564) 2024-02-02T15:27:22.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.509+0000 7f2e82000700 1 -- 172.21.15.1:0/565546187 learned_addr learned my addr 172.21.15.1:0/565546187 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:22.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.509+0000 7f2e82a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55858c3c8dd0 0x55858c3c71f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:53964/0 (socket says 172.21.15.1:53964) 2024-02-02T15:27:22.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.509+0000 7f2e83400700 1 --2- 172.21.15.1:0/565546187 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55858c3db090 0x55858c3dd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:22.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.509+0000 7f2e82000700 1 -- 172.21.15.1:0/565546187 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55858c3c8dd0 msgr2=0x55858c3c71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:22.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.509+0000 7f2e82000700 1 --2- 172.21.15.1:0/565546187 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55858c3c8dd0 0x55858c3c71f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.509+0000 7f2e82000700 1 -- 172.21.15.1:0/565546187 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55858c3db090 msgr2=0x55858c3dd480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:22.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.509+0000 7f2e82000700 1 --2- 172.21.15.1:0/565546187 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55858c3db090 0x55858c3dd480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.509+0000 7f2e82000700 1 -- 172.21.15.1:0/565546187 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55858c19a3b0 con 0x55858c3c7730 2024-02-02T15:27:22.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.509+0000 7f2e82000700 1 --2- 172.21.15.1:0/565546187 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55858c3c7730 0x55858c3c7bb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c00c5b0 tx=0x7f2e6c098fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3603ae347f35f361 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:22.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.509+0000 7f2e83400700 1 --2- 172.21.15.1:0/2711752913 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f2e640a0720 0x7f2e640a2be0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c009bf0 tx=0x7f2e7c004040 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:22.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.509+0000 7f2e73e00700 1 -- 172.21.15.1:0/565546187 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e6c099b10 con 0x55858c3c7730 2024-02-02T15:27:22.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.510+0000 7f2e73e00700 1 -- 172.21.15.1:0/565546187 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e6c099410 con 0x55858c3c7730 2024-02-02T15:27:22.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.510+0000 7f2e73e00700 1 -- 172.21.15.1:0/565546187 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e6c099670 con 0x55858c3c7730 2024-02-02T15:27:22.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.510+0000 7f2e9d521b80 1 -- 172.21.15.1:0/565546187 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55858c3c7730 msgr2=0x55858c3c7bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:22.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.510+0000 7f2e9d521b80 1 --2- 172.21.15.1:0/565546187 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55858c3c7730 0x55858c3c7bb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c00c5b0 tx=0x7f2e6c098fd0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.510+0000 7f2e9d521b80 1 -- 172.21.15.1:0/565546187 shutdown_connections 2024-02-02T15:27:22.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.510+0000 7f2e9d521b80 1 --2- 172.21.15.1:0/565546187 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55858c3db090 0x55858c3dd480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.510+0000 7f2e9d521b80 1 --2- 172.21.15.1:0/565546187 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55858c3c8dd0 0x55858c3c71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.510+0000 7f2e9d521b80 1 --2- 172.21.15.1:0/565546187 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55858c3c7730 0x55858c3c7bb0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.510+0000 7f2e9d521b80 1 -- 172.21.15.1:0/565546187 >> 172.21.15.1:0/565546187 conn(0x55858c326db0 msgr2=0x55858c3c35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:22.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.510+0000 7f2e9d521b80 1 -- 172.21.15.1:0/565546187 shutdown_connections 2024-02-02T15:27:22.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.510+0000 7f2e9d521b80 1 -- 172.21.15.1:0/565546187 wait complete. 2024-02-02T15:27:22.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.511+0000 7f2e9d521b80 1 -- start start 2024-02-02T15:27:22.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.511+0000 7f2e9d521b80 1 Processor -- start 2024-02-02T15:27:22.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.512+0000 7f2e9d521b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55858c3c7730 0x55858c3cdef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:22.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.512+0000 7f2e9d521b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55858c3c8dd0 0x55858c3cb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:22.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.512+0000 7f2e9d521b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55858c3db090 0x55858c3cb860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:22.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.512+0000 7f2e9d521b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55858c318d50 con 0x55858c3db090 2024-02-02T15:27:22.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.512+0000 7f2e9d521b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55858c2b4d80 con 0x55858c3c8dd0 2024-02-02T15:27:22.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.512+0000 7f2e9d521b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55858c2b4a20 con 0x55858c3c7730 2024-02-02T15:27:22.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.512+0000 7f2e9d521b80 10 client.0 _ll_register_callbacks cb 0x55858c2b6290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:27:22.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.512+0000 7f2e82000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55858c3c8dd0 0x55858c3cb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:22.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.512+0000 7f2e82000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55858c3c8dd0 0x55858c3cb1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59576/0 (socket says 172.21.15.1:59576) 2024-02-02T15:27:22.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.512+0000 7f2e82000700 1 -- 172.21.15.1:0/570872033 learned_addr learned my addr 172.21.15.1:0/570872033 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:22.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.512+0000 7f2e82a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55858c3c7730 0x55858c3cdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:22.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.512+0000 7f2e83400700 1 --2- 172.21.15.1:0/570872033 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55858c3db090 0x55858c3cb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:22.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.512+0000 7f2e82000700 1 -- 172.21.15.1:0/570872033 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55858c3c7730 msgr2=0x55858c3cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:22.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.512+0000 7f2e82000700 1 --2- 172.21.15.1:0/570872033 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55858c3c7730 0x55858c3cdef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.512+0000 7f2e82000700 1 -- 172.21.15.1:0/570872033 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55858c3db090 msgr2=0x55858c3cb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:22.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.512+0000 7f2e82000700 1 --2- 172.21.15.1:0/570872033 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55858c3db090 0x55858c3cb860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.512+0000 7f2e82000700 1 -- 172.21.15.1:0/570872033 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e841a39b0 con 0x55858c3c8dd0 2024-02-02T15:27:22.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.513+0000 7f2e82000700 1 --2- 172.21.15.1:0/570872033 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55858c3c8dd0 0x55858c3cb1e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c00e9c0 tx=0x7f2e6c010b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c4d5a6ce3a29b40 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:22.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.513+0000 7f2e70c00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:27:22.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.513+0000 7f2e70c00700 1 -- 172.21.15.1:0/570872033 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e6c099b10 con 0x55858c3c8dd0 2024-02-02T15:27:22.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.513+0000 7f2e70c00700 1 -- 172.21.15.1:0/570872033 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f2e6c09d240 con 0x55858c3c8dd0 2024-02-02T15:27:22.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.513+0000 7f2e9d521b80 1 -- 172.21.15.1:0/570872033 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55858c3bb340 con 0x55858c3c8dd0 2024-02-02T15:27:22.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.513+0000 7f2e70c00700 1 -- 172.21.15.1:0/570872033 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2e6c0a1480 con 0x55858c3c8dd0 2024-02-02T15:27:22.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.513+0000 7f2e9d521b80 10 client.25784 fetch_fsmap learned FSMap version 27 2024-02-02T15:27:22.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.513+0000 7f2e9d521b80 1 -- 172.21.15.1:0/570872033 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55858c146b60 con 0x55858c3c8dd0 2024-02-02T15:27:22.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.513+0000 7f2e70c00700 1 -- 172.21.15.1:0/570872033 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e6c065020 con 0x55858c3c8dd0 2024-02-02T15:27:22.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.513+0000 7f2e70c00700 1 -- 172.21.15.1:0/570872033 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2e6c099670 con 0x55858c3c8dd0 2024-02-02T15:27:22.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.514+0000 7f2e9d521b80 10 client.25784 fetch_fsmap finished waiting for FSMap version 27 2024-02-02T15:27:22.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.514+0000 7f2e9d521b80 10 client.25784 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:27:22.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.514+0000 7f2e9d521b80 20 client.25784 populate_metadata read hostname 'smithi001' 2024-02-02T15:27:22.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.514+0000 7f2e9d521b80 1 --2- 172.21.15.1:0/570872033 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55858c3ea750 0x55858c3ecb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:22.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.514+0000 7f2e9d521b80 4 client.25784 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:27:22.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.514+0000 7f2e9d521b80 1 -- 172.21.15.1:0/570872033 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55858c311dd0 con 0x55858c3ea750 2024-02-02T15:27:22.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.514+0000 7f2e82a00700 1 --2- 172.21.15.1:0/570872033 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55858c3ea750 0x55858c3ecb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:22.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.514+0000 7f2e82a00700 1 --2- 172.21.15.1:0/570872033 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55858c3ea750 0x55858c3ecb40 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=749ea875e612d3fa server_cookie=f8a6faea52ae587a in_seq=0 out_seq=0 2024-02-02T15:27:22.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.514+0000 7f2e70c00700 10 client.25784 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:27:22.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.515+0000 7f2e70c00700 1 -- 172.21.15.1:0/570872033 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55858c311dd0 con 0x55858c3ea750 2024-02-02T15:27:22.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.515+0000 7f2e70c00700 10 client.25784 handle_command_reply: tid=0 2024-02-02T15:27:22.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.515+0000 7f2e9d521b80 1 client.25784 shutdown 2024-02-02T15:27:22.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.516+0000 7f2e9d521b80 1 -- 172.21.15.1:0/570872033 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55858c3c8dd0 msgr2=0x55858c3cb1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:22.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.516+0000 7f2e9d521b80 1 --2- 172.21.15.1:0/570872033 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55858c3c8dd0 0x55858c3cb1e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c00e9c0 tx=0x7f2e6c010b80 comp rx=0 tx=0).stop 2024-02-02T15:27:22.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.516+0000 7f2e9d521b80 1 -- 172.21.15.1:0/570872033 shutdown_connections 2024-02-02T15:27:22.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.516+0000 7f2e9d521b80 1 --2- 172.21.15.1:0/570872033 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55858c3ea750 0x55858c3ecb40 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.516+0000 7f2e9d521b80 1 --2- 172.21.15.1:0/570872033 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55858c3db090 0x55858c3cb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.516+0000 7f2e9d521b80 1 --2- 172.21.15.1:0/570872033 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55858c3c7730 0x55858c3cdef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.516+0000 7f2e9d521b80 1 --2- 172.21.15.1:0/570872033 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55858c3c8dd0 0x55858c3cb1e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.516+0000 7f2e70c00700 0 client.25784 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:27:22.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.516+0000 7f2e9d521b80 1 -- 172.21.15.1:0/570872033 >> 172.21.15.1:0/570872033 conn(0x55858c326db0 msgr2=0x55858c3c2810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:22.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.516+0000 7f2e9d521b80 1 -- 172.21.15.1:0/570872033 shutdown_connections 2024-02-02T15:27:22.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.516+0000 7f2e9d521b80 1 -- 172.21.15.1:0/570872033 wait complete. 2024-02-02T15:27:22.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.516+0000 7f2e9d521b80 20 client.25784 trim_cache size 0 max 16384 2024-02-02T15:27:22.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.529+0000 7f2e9d521b80 1 Processor -- start 2024-02-02T15:27:22.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.529+0000 7f2e9d521b80 1 -- start start 2024-02-02T15:27:22.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.530+0000 7f2e9d521b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55858c3c7730 0x55858c3c71f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:22.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.530+0000 7f2e9d521b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55858c3c8dd0 0x55858c3e1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:22.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.530+0000 7f2e9d521b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55858c3db090 0x55858c3e6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:22.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.530+0000 7f2e9d521b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55858c19c6a0 con 0x55858c3c7730 2024-02-02T15:27:22.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.530+0000 7f2e9d521b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55858c3cc750 con 0x55858c3db090 2024-02-02T15:27:22.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.530+0000 7f2e9d521b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55858c3ce230 con 0x55858c3c8dd0 2024-02-02T15:27:22.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.530+0000 7f2e82a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55858c3c7730 0x55858c3c71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:22.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.530+0000 7f2e82a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55858c3c7730 0x55858c3c71f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43762/0 (socket says 172.21.15.1:43762) 2024-02-02T15:27:22.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.530+0000 7f2e82a00700 1 -- 172.21.15.1:0/1870555808 learned_addr learned my addr 172.21.15.1:0/1870555808 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:22.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.530+0000 7f2e82000700 1 --2- 172.21.15.1:0/1870555808 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55858c3c8dd0 0x55858c3e1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:22.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.530+0000 7f2e83400700 1 --2- 172.21.15.1:0/1870555808 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55858c3db090 0x55858c3e6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:22.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.530+0000 7f2e82a00700 1 -- 172.21.15.1:0/1870555808 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55858c3c8dd0 msgr2=0x55858c3e1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:22.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.530+0000 7f2e82a00700 1 --2- 172.21.15.1:0/1870555808 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55858c3c8dd0 0x55858c3e1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.530+0000 7f2e82a00700 1 -- 172.21.15.1:0/1870555808 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55858c3db090 msgr2=0x55858c3e6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:22.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.530+0000 7f2e82a00700 1 --2- 172.21.15.1:0/1870555808 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55858c3db090 0x55858c3e6250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.530+0000 7f2e82a00700 1 -- 172.21.15.1:0/1870555808 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55858c2deea0 con 0x55858c3c7730 2024-02-02T15:27:22.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.530+0000 7f2e82a00700 1 --2- 172.21.15.1:0/1870555808 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55858c3c7730 0x55858c3c71f0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f2e74000c00 tx=0x7f2e7400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1c62ebd4d373ed2 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:22.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.531+0000 7f2e78c00700 1 -- 172.21.15.1:0/1870555808 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e7401d040 con 0x55858c3c7730 2024-02-02T15:27:22.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.531+0000 7f2e78c00700 1 -- 172.21.15.1:0/1870555808 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e7401e040 con 0x55858c3c7730 2024-02-02T15:27:22.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.531+0000 7f2e78c00700 1 -- 172.21.15.1:0/1870555808 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e74004500 con 0x55858c3c7730 2024-02-02T15:27:22.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.531+0000 7f2e9d521b80 1 -- 172.21.15.1:0/1870555808 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55858c3c7730 msgr2=0x55858c3c71f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:22.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.531+0000 7f2e9d521b80 1 --2- 172.21.15.1:0/1870555808 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55858c3c7730 0x55858c3c71f0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f2e74000c00 tx=0x7f2e7400b260 comp rx=0 tx=0).stop 2024-02-02T15:27:22.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.531+0000 7f2e9d521b80 1 -- 172.21.15.1:0/1870555808 shutdown_connections 2024-02-02T15:27:22.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.531+0000 7f2e9d521b80 1 --2- 172.21.15.1:0/1870555808 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55858c3db090 0x55858c3e6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.531+0000 7f2e9d521b80 1 --2- 172.21.15.1:0/1870555808 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55858c3c7730 0x55858c3c71f0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.531+0000 7f2e9d521b80 1 --2- 172.21.15.1:0/1870555808 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55858c3c8dd0 0x55858c3e1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.531+0000 7f2e9d521b80 1 -- 172.21.15.1:0/1870555808 >> 172.21.15.1:0/1870555808 conn(0x55858c326db0 msgr2=0x55858c3cdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:22.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.532+0000 7f2e9d521b80 1 -- 172.21.15.1:0/1870555808 shutdown_connections 2024-02-02T15:27:22.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.532+0000 7f2e9d521b80 1 -- 172.21.15.1:0/1870555808 wait complete. 2024-02-02T15:27:22.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.532+0000 7f2e9d521b80 1 -- start start 2024-02-02T15:27:22.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.532+0000 7f2e9d521b80 1 Processor -- start 2024-02-02T15:27:22.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.533+0000 7f2e9d521b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55858c3c7730 0x55858c434530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:22.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.533+0000 7f2e9d521b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55858c3c8dd0 0x55858c434a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:22.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.533+0000 7f2e9d521b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55858c3db090 0x55858c3ddc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:22.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.533+0000 7f2e9d521b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55858c3de250 con 0x55858c3c8dd0 2024-02-02T15:27:22.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.533+0000 7f2e9d521b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55858c3de3c0 con 0x55858c3db090 2024-02-02T15:27:22.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.533+0000 7f2e9d521b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55858c3de580 con 0x55858c3c7730 2024-02-02T15:27:22.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.533+0000 7f2e9d521b80 10 client.0 _ll_register_callbacks cb 0x55858c2b6290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:27:22.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.533+0000 7f2e82000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55858c3c8dd0 0x55858c434a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:22.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.533+0000 7f2e82a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55858c3c7730 0x55858c434530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:22.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.533+0000 7f2e83400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55858c3db090 0x55858c3ddc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:22.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.533+0000 7f2e82a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55858c3c7730 0x55858c434530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:53986/0 (socket says 172.21.15.1:53986) 2024-02-02T15:27:22.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.533+0000 7f2e82a00700 1 -- 172.21.15.1:0/562510730 learned_addr learned my addr 172.21.15.1:0/562510730 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:22.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.533+0000 7f2e83400700 1 -- 172.21.15.1:0/562510730 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55858c3c7730 msgr2=0x55858c434530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:22.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.533+0000 7f2e83400700 1 --2- 172.21.15.1:0/562510730 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55858c3c7730 0x55858c434530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.533+0000 7f2e83400700 1 -- 172.21.15.1:0/562510730 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55858c3c8dd0 msgr2=0x55858c434a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:22.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.533+0000 7f2e83400700 1 --2- 172.21.15.1:0/562510730 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55858c3c8dd0 0x55858c434a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.533+0000 7f2e83400700 1 -- 172.21.15.1:0/562510730 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e8410e270 con 0x55858c3db090 2024-02-02T15:27:22.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.533+0000 7f2e82000700 1 --2- 172.21.15.1:0/562510730 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55858c3c8dd0 0x55858c434a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:27:22.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.533+0000 7f2e82a00700 1 --2- 172.21.15.1:0/562510730 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55858c3c7730 0x55858c434530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:27:22.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.534+0000 7f2e83400700 1 --2- 172.21.15.1:0/562510730 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55858c3db090 0x55858c3ddc50 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c004440 tx=0x7f2e7c00b050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d8a225579806fa3 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:22.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.534+0000 7f2e70c00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:27:22.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.534+0000 7f2e70c00700 1 -- 172.21.15.1:0/562510730 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e7c007e70 con 0x55858c3db090 2024-02-02T15:27:22.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.534+0000 7f2e9d521b80 1 -- 172.21.15.1:0/562510730 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2e6c099410 con 0x55858c3db090 2024-02-02T15:27:22.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.534+0000 7f2e70c00700 1 -- 172.21.15.1:0/562510730 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f2e7c01a470 con 0x55858c3db090 2024-02-02T15:27:22.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.534+0000 7f2e70c00700 1 -- 172.21.15.1:0/562510730 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2e7c01a630 con 0x55858c3db090 2024-02-02T15:27:22.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.534+0000 7f2e9d521b80 10 client.25790 fetch_fsmap learned FSMap version 27 2024-02-02T15:27:22.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.534+0000 7f2e9d521b80 1 -- 172.21.15.1:0/562510730 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55858c19ecd0 con 0x55858c3db090 2024-02-02T15:27:22.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.535+0000 7f2e70c00700 1 -- 172.21.15.1:0/562510730 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e6c099410 con 0x55858c3db090 2024-02-02T15:27:22.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.535+0000 7f2e70c00700 1 -- 172.21.15.1:0/562510730 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2e7c027820 con 0x55858c3db090 2024-02-02T15:27:22.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.535+0000 7f2e9d521b80 10 client.25790 fetch_fsmap finished waiting for FSMap version 27 2024-02-02T15:27:22.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.535+0000 7f2e9d521b80 10 client.25790 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:27:22.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.535+0000 7f2e9d521b80 20 client.25790 populate_metadata read hostname 'smithi001' 2024-02-02T15:27:22.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.535+0000 7f2e9d521b80 1 --2- 172.21.15.1:0/562510730 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55858c43d520 0x55858c43f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:22.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.535+0000 7f2e9d521b80 4 client.25790 mds_command: new command op to 24557 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:27:22.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.535+0000 7f2e9d521b80 1 -- 172.21.15.1:0/562510730 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55858c2d7f50 con 0x55858c43d520 2024-02-02T15:27:22.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.535+0000 7f2e82a00700 1 --2- 172.21.15.1:0/562510730 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55858c43d520 0x55858c43f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:22.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.535+0000 7f2e82a00700 1 --2- 172.21.15.1:0/562510730 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55858c43d520 0x55858c43f940 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d01b2d859029df0b server_cookie=539600236b6f9119 in_seq=0 out_seq=0 2024-02-02T15:27:22.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.535+0000 7f2e70c00700 10 client.25790 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:27:22.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.536+0000 7f2e70c00700 1 -- 172.21.15.1:0/562510730 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55858c2d7f50 con 0x55858c43d520 2024-02-02T15:27:22.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.536+0000 7f2e70c00700 10 client.25790 handle_command_reply: tid=0 2024-02-02T15:27:22.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.536+0000 7f2e9d521b80 1 client.25790 shutdown 2024-02-02T15:27:22.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.536+0000 7f2e9d521b80 1 -- 172.21.15.1:0/562510730 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55858c3db090 msgr2=0x55858c3ddc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:22.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.536+0000 7f2e9d521b80 1 --2- 172.21.15.1:0/562510730 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55858c3db090 0x55858c3ddc50 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c004440 tx=0x7f2e7c00b050 comp rx=0 tx=0).stop 2024-02-02T15:27:22.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.536+0000 7f2e9d521b80 1 -- 172.21.15.1:0/562510730 shutdown_connections 2024-02-02T15:27:22.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.536+0000 7f2e9d521b80 1 --2- 172.21.15.1:0/562510730 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55858c43d520 0x55858c43f940 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.536+0000 7f2e9d521b80 1 --2- 172.21.15.1:0/562510730 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55858c3db090 0x55858c3ddc50 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.536+0000 7f2e9d521b80 1 --2- 172.21.15.1:0/562510730 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55858c3c7730 0x55858c434530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.536+0000 7f2e9d521b80 1 --2- 172.21.15.1:0/562510730 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55858c3c8dd0 0x55858c434a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.536+0000 7f2e9d521b80 1 -- 172.21.15.1:0/562510730 >> 172.21.15.1:0/562510730 conn(0x55858c326db0 msgr2=0x55858c3e6250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:22.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.536+0000 7f2e70c00700 0 client.25790 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:27:22.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.536+0000 7f2e9d521b80 1 -- 172.21.15.1:0/562510730 shutdown_connections 2024-02-02T15:27:22.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.536+0000 7f2e9d521b80 1 -- 172.21.15.1:0/562510730 wait complete. 2024-02-02T15:27:22.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.536+0000 7f2e9d521b80 20 client.25790 trim_cache size 0 max 16384 2024-02-02T15:27:22.541 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:27:22.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.537+0000 7f2e83e00700 1 -- 172.21.15.1:0/2711752913 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f2e640a0720 msgr2=0x7f2e640a2be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:22.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.537+0000 7f2e83e00700 1 --2- 172.21.15.1:0/2711752913 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f2e640a0720 0x7f2e640a2be0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c009bf0 tx=0x7f2e7c004040 comp rx=0 tx=0).stop 2024-02-02T15:27:22.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.537+0000 7f2e83e00700 1 -- 172.21.15.1:0/2711752913 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e840689d0 msgr2=0x7f2e8419cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:22.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.537+0000 7f2e83e00700 1 --2- 172.21.15.1:0/2711752913 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e840689d0 0x7f2e8419cf00 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2e6c00e9f0 tx=0x7f2e6c00ed00 comp rx=0 tx=0).stop 2024-02-02T15:27:22.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.537+0000 7f2e83e00700 1 -- 172.21.15.1:0/2711752913 shutdown_connections 2024-02-02T15:27:22.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.537+0000 7f2e83e00700 1 --2- 172.21.15.1:0/2711752913 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f2e640a0720 0x7f2e640a2be0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.537+0000 7f2e83e00700 1 --2- 172.21.15.1:0/2711752913 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e84106ab0 0x7f2e84196a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.537+0000 7f2e83e00700 1 --2- 172.21.15.1:0/2711752913 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e84068010 0x7f2e8419c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.537+0000 7f2e83e00700 1 --2- 172.21.15.1:0/2711752913 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e840689d0 0x7f2e8419cf00 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:22.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.537+0000 7f2e83e00700 1 -- 172.21.15.1:0/2711752913 >> 172.21.15.1:0/2711752913 conn(0x7f2e84076770 msgr2=0x7f2e840fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:22.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.537+0000 7f2e83e00700 1 -- 172.21.15.1:0/2711752913 shutdown_connections 2024-02-02T15:27:22.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:22.537+0000 7f2e83e00700 1 -- 172.21.15.1:0/2711752913 wait complete. 2024-02-02T15:27:23.555 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-02T15:27:23.555 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-02T15:27:23.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:23 smithi071 ceph-mon[44707]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 170 B/s wr, 12 op/s 2024-02-02T15:27:23.619 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:23 smithi001 ceph-mon[35470]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 170 B/s wr, 12 op/s 2024-02-02T15:27:23.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:23 smithi142 ceph-mon[45728]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 170 B/s wr, 12 op/s 2024-02-02T15:27:23.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.769+0000 7fb00ce00700 1 -- 172.21.15.1:0/2877305747 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb008100a90 msgr2=0x7fb008100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:23.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.769+0000 7fb00ce00700 1 --2- 172.21.15.1:0/2877305747 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb008100a90 0x7fb008100f10 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7faff0009990 tx=0x7faff0009ca0 comp rx=0 tx=0).stop 2024-02-02T15:27:23.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.769+0000 7fb00ce00700 1 -- 172.21.15.1:0/2877305747 shutdown_connections 2024-02-02T15:27:23.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.769+0000 7fb00ce00700 1 --2- 172.21.15.1:0/2877305747 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb008101450 0x7fb00810d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:23.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.770+0000 7fb00ce00700 1 --2- 172.21.15.1:0/2877305747 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb008106aa0 0x7fb008106e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:23.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.770+0000 7fb00ce00700 1 --2- 172.21.15.1:0/2877305747 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb008100a90 0x7fb008100f10 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:23.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.770+0000 7fb00ce00700 1 -- 172.21.15.1:0/2877305747 >> 172.21.15.1:0/2877305747 conn(0x7fb0080fc580 msgr2=0x7fb0080fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:23.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.770+0000 7fb00ce00700 1 -- 172.21.15.1:0/2877305747 shutdown_connections 2024-02-02T15:27:23.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.770+0000 7fb00ce00700 1 -- 172.21.15.1:0/2877305747 wait complete. 2024-02-02T15:27:23.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.771+0000 7fb00ce00700 1 Processor -- start 2024-02-02T15:27:23.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.771+0000 7fb00ce00700 1 -- start start 2024-02-02T15:27:23.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.771+0000 7fb00ce00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb008100a90 0x7fb008198d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:23.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.771+0000 7fb00ce00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb008101450 0x7fb008199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:23.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.771+0000 7fb00ce00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb008106aa0 0x7fb00819d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:23.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.771+0000 7fb00ce00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb00819dd30 con 0x7fb008100a90 2024-02-02T15:27:23.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.771+0000 7fb00ce00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb00819dea0 con 0x7fb008101450 2024-02-02T15:27:23.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.771+0000 7fb00ce00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb00819e060 con 0x7fb008106aa0 2024-02-02T15:27:23.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.771+0000 7fb007400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb008100a90 0x7fb008198d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:23.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.771+0000 7fb007400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb008100a90 0x7fb008198d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33184/0 (socket says 172.21.15.1:33184) 2024-02-02T15:27:23.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.771+0000 7fb007400700 1 -- 172.21.15.1:0/4183924983 learned_addr learned my addr 172.21.15.1:0/4183924983 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:23.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.771+0000 7fb006a00700 1 --2- 172.21.15.1:0/4183924983 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb008101450 0x7fb008199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:23.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.772+0000 7fb007e00700 1 --2- 172.21.15.1:0/4183924983 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb008106aa0 0x7fb00819d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:23.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.772+0000 7fb007400700 1 -- 172.21.15.1:0/4183924983 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb008106aa0 msgr2=0x7fb00819d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:23.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.772+0000 7fb007400700 1 --2- 172.21.15.1:0/4183924983 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb008106aa0 0x7fb00819d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:23.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.772+0000 7fb007400700 1 -- 172.21.15.1:0/4183924983 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb008101450 msgr2=0x7fb008199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:23.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.772+0000 7fb007400700 1 --2- 172.21.15.1:0/4183924983 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb008101450 0x7fb008199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:23.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.772+0000 7fb007400700 1 -- 172.21.15.1:0/4183924983 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faff00095f0 con 0x7fb008100a90 2024-02-02T15:27:23.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.772+0000 7fb007e00700 1 --2- 172.21.15.1:0/4183924983 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb008106aa0 0x7fb00819d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:27:23.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.772+0000 7fb007400700 1 --2- 172.21.15.1:0/4183924983 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb008100a90 0x7fb008198d50 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7faff800e9a0 tx=0x7faff800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:23.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.772+0000 7fafffe00700 1 -- 172.21.15.1:0/4183924983 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff8009790 con 0x7fb008100a90 2024-02-02T15:27:23.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.772+0000 7fafffe00700 1 -- 172.21.15.1:0/4183924983 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faff800cbd0 con 0x7fb008100a90 2024-02-02T15:27:23.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.772+0000 7fb00ce00700 1 -- 172.21.15.1:0/4183924983 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb00819e340 con 0x7fb008100a90 2024-02-02T15:27:23.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.772+0000 7fafffe00700 1 -- 172.21.15.1:0/4183924983 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff8018680 con 0x7fb008100a90 2024-02-02T15:27:23.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.772+0000 7fb00ce00700 1 -- 172.21.15.1:0/4183924983 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0081a5b80 con 0x7fb008100a90 2024-02-02T15:27:23.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.775+0000 7fafffe00700 1 -- 172.21.15.1:0/4183924983 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135245+0+0 (secure 0 0 0) 0x7faff800cd40 con 0x7fb008100a90 2024-02-02T15:27:23.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.777+0000 7fb021e89b80 1 Processor -- start 2024-02-02T15:27:23.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.777+0000 7fafffe00700 1 --2- 172.21.15.1:0/4183924983 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fafe80a07a0 0x7fafe80a2c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:23.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.777+0000 7fafffe00700 1 -- 172.21.15.1:0/4183924983 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7faff8014070 con 0x7fb008100a90 2024-02-02T15:27:23.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.777+0000 7fb021e89b80 1 -- start start 2024-02-02T15:27:23.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.777+0000 7fb006a00700 1 --2- 172.21.15.1:0/4183924983 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fafe80a07a0 0x7fafe80a2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:23.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.777+0000 7fb021e89b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560df01b3270 0x560df01b3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:23.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.777+0000 7fb021e89b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560df01b3b90 0x560df01be530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:23.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.777+0000 7fb021e89b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560df01bea70 0x560df01c0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:23.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.777+0000 7fb021e89b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x560df0127540 con 0x560df01b3b90 2024-02-02T15:27:23.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.777+0000 7fb021e89b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x560df010bb20 con 0x560df01bea70 2024-02-02T15:27:23.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.777+0000 7fb021e89b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560df01094e0 con 0x560df01b3270 2024-02-02T15:27:23.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.778+0000 7fb007e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560df01bea70 0x560df01c0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:23.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.778+0000 7fb007e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560df01bea70 0x560df01c0e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:43780/0 (socket says 172.21.15.1:43780) 2024-02-02T15:27:23.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.778+0000 7fb007400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560df01b3270 0x560df01b3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:23.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.778+0000 7fb007e00700 1 -- 172.21.15.1:0/1687718638 learned_addr learned my addr 172.21.15.1:0/1687718638 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:23.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.778+0000 7fb006a00700 1 --2- 172.21.15.1:0/1687718638 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560df01b3b90 0x560df01be530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:23.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.778+0000 7fb007e00700 1 -- 172.21.15.1:0/1687718638 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560df01b3270 msgr2=0x560df01b3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:23.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.778+0000 7fb007e00700 1 --2- 172.21.15.1:0/1687718638 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560df01b3270 0x560df01b3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:23.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.778+0000 7fb007e00700 1 -- 172.21.15.1:0/1687718638 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560df01b3b90 msgr2=0x560df01be530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:23.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.778+0000 7fb007e00700 1 --2- 172.21.15.1:0/1687718638 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560df01b3b90 0x560df01be530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:23.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.778+0000 7fb007e00700 1 -- 172.21.15.1:0/1687718638 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560deff86430 con 0x560df01bea70 2024-02-02T15:27:23.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.778+0000 7fb007400700 1 --2- 172.21.15.1:0/1687718638 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560df01b3270 0x560df01b3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:27:23.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.778+0000 7fb006a00700 1 --2- 172.21.15.1:0/4183924983 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fafe80a07a0 0x7fafe80a2c60 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb00819a360 tx=0x7faff000b850 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:23.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.778+0000 7fb007e00700 1 --2- 172.21.15.1:0/1687718638 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560df01bea70 0x560df01c0e60 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb000009e50 tx=0x7fb0000077e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b5ea43f37690b1e server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:23.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.779+0000 7faffcc00700 1 -- 172.21.15.1:0/1687718638 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb000014610 con 0x560df01bea70 2024-02-02T15:27:23.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.779+0000 7faffcc00700 1 -- 172.21.15.1:0/1687718638 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb00000f050 con 0x560df01bea70 2024-02-02T15:27:23.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.779+0000 7faffcc00700 1 -- 172.21.15.1:0/1687718638 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb000013a10 con 0x560df01bea70 2024-02-02T15:27:23.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.779+0000 7fb021e89b80 1 -- 172.21.15.1:0/1687718638 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560df01bea70 msgr2=0x560df01c0e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:23.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.779+0000 7fb021e89b80 1 --2- 172.21.15.1:0/1687718638 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560df01bea70 0x560df01c0e60 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb000009e50 tx=0x7fb0000077e0 comp rx=0 tx=0).stop 2024-02-02T15:27:23.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.780+0000 7fb021e89b80 1 -- 172.21.15.1:0/1687718638 shutdown_connections 2024-02-02T15:27:23.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.780+0000 7fb021e89b80 1 --2- 172.21.15.1:0/1687718638 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560df01bea70 0x560df01c0e60 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:23.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.780+0000 7fb021e89b80 1 --2- 172.21.15.1:0/1687718638 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560df01b3270 0x560df01b3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:23.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.780+0000 7fb021e89b80 1 --2- 172.21.15.1:0/1687718638 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560df01b3b90 0x560df01be530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:23.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.780+0000 7fb021e89b80 1 -- 172.21.15.1:0/1687718638 >> 172.21.15.1:0/1687718638 conn(0x560df0112e30 msgr2=0x560df01af640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:23.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.780+0000 7fb021e89b80 1 -- 172.21.15.1:0/1687718638 shutdown_connections 2024-02-02T15:27:23.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.780+0000 7fb021e89b80 1 -- 172.21.15.1:0/1687718638 wait complete. 2024-02-02T15:27:23.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.780+0000 7fb021e89b80 1 -- start start 2024-02-02T15:27:23.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.780+0000 7fb021e89b80 1 Processor -- start 2024-02-02T15:27:23.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.781+0000 7fb021e89b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560df01b3270 0x560df01be3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:23.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.781+0000 7fb021e89b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560df01b3b90 0x560df01bb670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:23.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.781+0000 7fb021e89b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560df01bea70 0x560df01bbca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:23.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.781+0000 7fb021e89b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x560df0104dd0 con 0x560df01b3270 2024-02-02T15:27:23.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.781+0000 7fb021e89b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x560df00a0e00 con 0x560df01bea70 2024-02-02T15:27:23.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.781+0000 7fb021e89b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560df00a0aa0 con 0x560df01b3b90 2024-02-02T15:27:23.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.781+0000 7fb021e89b80 10 client.0 _ll_register_callbacks cb 0x560df00a2310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:27:23.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.781+0000 7fb007400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560df01b3270 0x560df01be3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:23.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.781+0000 7fb007e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560df01bea70 0x560df01bbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:23.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.781+0000 7fb007400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560df01b3270 0x560df01be3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33198/0 (socket says 172.21.15.1:33198) 2024-02-02T15:27:23.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.781+0000 7fb007400700 1 -- 172.21.15.1:0/4094119210 learned_addr learned my addr 172.21.15.1:0/4094119210 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:23.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.781+0000 7fb006a00700 1 --2- 172.21.15.1:0/4094119210 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560df01b3b90 0x560df01bb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:23.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.782+0000 7fb007400700 1 -- 172.21.15.1:0/4094119210 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560df01b3b90 msgr2=0x560df01bb670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:23.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.782+0000 7fb007400700 1 --2- 172.21.15.1:0/4094119210 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560df01b3b90 0x560df01bb670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:23.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.782+0000 7fb007400700 1 -- 172.21.15.1:0/4094119210 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560df01bea70 msgr2=0x560df01bbca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:23.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.782+0000 7fb007400700 1 --2- 172.21.15.1:0/4094119210 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560df01bea70 0x560df01bbca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:23.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.782+0000 7fb007400700 1 -- 172.21.15.1:0/4094119210 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0081a5b80 con 0x560df01b3270 2024-02-02T15:27:23.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.782+0000 7fb007e00700 1 --2- 172.21.15.1:0/4094119210 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560df01bea70 0x560df01bbca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:27:23.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.782+0000 7fb007400700 1 --2- 172.21.15.1:0/4094119210 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560df01b3270 0x560df01be3f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7faff8095ef0 tx=0x7faff80c9cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70c0bed2b95e9c3f server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:23.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.782+0000 7faff5600700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:27:23.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.782+0000 7faff5600700 1 -- 172.21.15.1:0/4094119210 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff8099480 con 0x560df01b3270 2024-02-02T15:27:23.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.782+0000 7faff5600700 1 -- 172.21.15.1:0/4094119210 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7faff809d2b0 con 0x560df01b3270 2024-02-02T15:27:23.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.782+0000 7fb021e89b80 1 -- 172.21.15.1:0/4094119210 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560df01a73c0 con 0x560df01b3270 2024-02-02T15:27:23.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.782+0000 7faff5600700 1 -- 172.21.15.1:0/4094119210 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7faff80a14f0 con 0x560df01b3270 2024-02-02T15:27:23.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.782+0000 7fb021e89b80 10 client.15765 fetch_fsmap learned FSMap version 27 2024-02-02T15:27:23.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.782+0000 7fb021e89b80 1 -- 172.21.15.1:0/4094119210 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7fb000009a80 con 0x560df01b3270 2024-02-02T15:27:23.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.783+0000 7faff5600700 1 -- 172.21.15.1:0/4094119210 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560df01a73c0 con 0x560df01b3270 2024-02-02T15:27:23.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.783+0000 7faff5600700 1 -- 172.21.15.1:0/4094119210 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7faff80a10f0 con 0x560df01b3270 2024-02-02T15:27:23.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.783+0000 7fb021e89b80 10 client.15765 fetch_fsmap finished waiting for FSMap version 27 2024-02-02T15:27:23.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.783+0000 7fb021e89b80 10 client.15765 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:27:23.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.783+0000 7fb021e89b80 20 client.15765 populate_metadata read hostname 'smithi001' 2024-02-02T15:27:23.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.783+0000 7fb021e89b80 1 --2- 172.21.15.1:0/4094119210 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x560df01ce3f0 0x560df01d0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:23.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.783+0000 7fb021e89b80 4 client.15765 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:27:23.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.783+0000 7fb021e89b80 1 -- 172.21.15.1:0/4094119210 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560df00fde50 con 0x560df01ce3f0 2024-02-02T15:27:23.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.783+0000 7fb006a00700 1 --2- 172.21.15.1:0/4094119210 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x560df01ce3f0 0x560df01d0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:23.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.784+0000 7fb006a00700 1 --2- 172.21.15.1:0/4094119210 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x560df01ce3f0 0x560df01d0810 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d84f869938e6c93 server_cookie=cc3643334c15e7b3 in_seq=0 out_seq=0 2024-02-02T15:27:23.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.784+0000 7faff5600700 10 client.15765 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:27:23.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.784+0000 7faff5600700 1 -- 172.21.15.1:0/4094119210 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x560df00fde50 con 0x560df01ce3f0 2024-02-02T15:27:23.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.784+0000 7faff5600700 10 client.15765 handle_command_reply: tid=0 2024-02-02T15:27:23.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.785+0000 7fb021e89b80 1 client.15765 shutdown 2024-02-02T15:27:23.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.785+0000 7fb021e89b80 1 -- 172.21.15.1:0/4094119210 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560df01b3270 msgr2=0x560df01be3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:23.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.785+0000 7fb021e89b80 1 --2- 172.21.15.1:0/4094119210 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560df01b3270 0x560df01be3f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7faff8095ef0 tx=0x7faff80c9cf0 comp rx=0 tx=0).stop 2024-02-02T15:27:23.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.785+0000 7fb021e89b80 1 -- 172.21.15.1:0/4094119210 shutdown_connections 2024-02-02T15:27:23.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.785+0000 7fb021e89b80 1 --2- 172.21.15.1:0/4094119210 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x560df01ce3f0 0x560df01d0810 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:23.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.785+0000 7fb021e89b80 1 --2- 172.21.15.1:0/4094119210 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560df01bea70 0x560df01bbca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:23.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.785+0000 7fb021e89b80 1 --2- 172.21.15.1:0/4094119210 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560df01b3270 0x560df01be3f0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:23.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.785+0000 7faff5600700 0 client.15765 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:27:23.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.785+0000 7fb021e89b80 1 --2- 172.21.15.1:0/4094119210 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560df01b3b90 0x560df01bb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:23.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.785+0000 7fb021e89b80 1 -- 172.21.15.1:0/4094119210 >> 172.21.15.1:0/4094119210 conn(0x560df0112e30 msgr2=0x560df01af1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:23.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.785+0000 7fb021e89b80 1 -- 172.21.15.1:0/4094119210 shutdown_connections 2024-02-02T15:27:23.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.785+0000 7fb021e89b80 1 -- 172.21.15.1:0/4094119210 wait complete. 2024-02-02T15:27:23.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.785+0000 7fb021e89b80 20 client.15765 trim_cache size 0 max 16384 2024-02-02T15:27:23.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.802+0000 7fb021e89b80 1 Processor -- start 2024-02-02T15:27:23.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.803+0000 7fb021e89b80 1 -- start start 2024-02-02T15:27:23.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.803+0000 7fb021e89b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560df01b3270 0x560df01c54d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:23.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.803+0000 7fb021e89b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560df01b3b90 0x560df01b59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:23.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.803+0000 7fb021e89b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560df01bea70 0x560df01b5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:23.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.803+0000 7fb021e89b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x560deff88720 con 0x560df01b3270 2024-02-02T15:27:23.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.803+0000 7fb021e89b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x560df01afbc0 con 0x560df01bea70 2024-02-02T15:27:23.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.803+0000 7fb021e89b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560df01b44e0 con 0x560df01b3b90 2024-02-02T15:27:23.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.803+0000 7fb007400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560df01b3270 0x560df01c54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:23.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.803+0000 7fb007400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560df01b3270 0x560df01c54d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33212/0 (socket says 172.21.15.1:33212) 2024-02-02T15:27:23.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.803+0000 7fb007400700 1 -- 172.21.15.1:0/2483101273 learned_addr learned my addr 172.21.15.1:0/2483101273 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:23.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.803+0000 7fb007e00700 1 --2- 172.21.15.1:0/2483101273 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560df01bea70 0x560df01b5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:23.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.803+0000 7fb006a00700 1 --2- 172.21.15.1:0/2483101273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560df01b3b90 0x560df01b59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:23.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.803+0000 7fb007400700 1 -- 172.21.15.1:0/2483101273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560df01b3b90 msgr2=0x560df01b59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:23.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.803+0000 7fb007400700 1 --2- 172.21.15.1:0/2483101273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560df01b3b90 0x560df01b59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:23.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.804+0000 7fb007400700 1 -- 172.21.15.1:0/2483101273 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560df01bea70 msgr2=0x560df01b5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:23.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.804+0000 7fb007400700 1 --2- 172.21.15.1:0/2483101273 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560df01bea70 0x560df01b5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:23.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.804+0000 7fb007400700 1 -- 172.21.15.1:0/2483101273 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560deff32be0 con 0x560df01b3270 2024-02-02T15:27:23.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.804+0000 7fb007400700 1 --2- 172.21.15.1:0/2483101273 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560df01b3270 0x560df01c54d0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7faff8099410 tx=0x7faff8097a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3507162ef4673047 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:23.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.804+0000 7faffd600700 1 -- 172.21.15.1:0/2483101273 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff80c96a0 con 0x560df01b3270 2024-02-02T15:27:23.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.804+0000 7faffd600700 1 -- 172.21.15.1:0/2483101273 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faff809d2b0 con 0x560df01b3270 2024-02-02T15:27:23.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.804+0000 7faffd600700 1 -- 172.21.15.1:0/2483101273 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff80a1060 con 0x560df01b3270 2024-02-02T15:27:23.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.804+0000 7fb021e89b80 1 -- 172.21.15.1:0/2483101273 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560df01b3270 msgr2=0x560df01c54d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:23.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.804+0000 7fb021e89b80 1 --2- 172.21.15.1:0/2483101273 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560df01b3270 0x560df01c54d0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7faff8099410 tx=0x7faff8097a30 comp rx=0 tx=0).stop 2024-02-02T15:27:23.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.805+0000 7fb021e89b80 1 -- 172.21.15.1:0/2483101273 shutdown_connections 2024-02-02T15:27:23.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.805+0000 7fb021e89b80 1 --2- 172.21.15.1:0/2483101273 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560df01bea70 0x560df01b5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:23.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.805+0000 7fb021e89b80 1 --2- 172.21.15.1:0/2483101273 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560df01b3270 0x560df01c54d0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:23.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.805+0000 7fb021e89b80 1 --2- 172.21.15.1:0/2483101273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560df01b3b90 0x560df01b59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:23.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.805+0000 7fb021e89b80 1 -- 172.21.15.1:0/2483101273 >> 172.21.15.1:0/2483101273 conn(0x560df0112e30 msgr2=0x560df01b52a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:23.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.805+0000 7fb021e89b80 1 -- 172.21.15.1:0/2483101273 shutdown_connections 2024-02-02T15:27:23.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.805+0000 7fb021e89b80 1 -- 172.21.15.1:0/2483101273 wait complete. 2024-02-02T15:27:23.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.805+0000 7fb021e89b80 1 -- start start 2024-02-02T15:27:23.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.805+0000 7fb021e89b80 1 Processor -- start 2024-02-02T15:27:23.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.806+0000 7fb021e89b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560df01b3270 0x560df01c3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:23.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.806+0000 7fb021e89b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560df01b3b90 0x560df01c3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:23.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.806+0000 7fb021e89b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560df01bea70 0x560df01c4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:23.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.806+0000 7fb021e89b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x560df021ff20 con 0x560df01b3270 2024-02-02T15:27:23.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.806+0000 7fb021e89b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x560df0220060 con 0x560df01b3b90 2024-02-02T15:27:23.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.806+0000 7fb021e89b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560df02201a0 con 0x560df01bea70 2024-02-02T15:27:23.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.806+0000 7fb021e89b80 10 client.0 _ll_register_callbacks cb 0x560df00a2310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:27:23.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.806+0000 7fb007400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560df01b3270 0x560df01c3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:23.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.806+0000 7fb007400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560df01b3270 0x560df01c3a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33216/0 (socket says 172.21.15.1:33216) 2024-02-02T15:27:23.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.806+0000 7fb007400700 1 -- 172.21.15.1:0/298124223 learned_addr learned my addr 172.21.15.1:0/298124223 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:23.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.806+0000 7fb007e00700 1 --2- 172.21.15.1:0/298124223 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560df01bea70 0x560df01c4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:23.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.806+0000 7fb006a00700 1 --2- 172.21.15.1:0/298124223 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560df01b3b90 0x560df01c3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:23.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.806+0000 7fb007400700 1 -- 172.21.15.1:0/298124223 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560df01bea70 msgr2=0x560df01c4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:23.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.806+0000 7fb007400700 1 --2- 172.21.15.1:0/298124223 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560df01bea70 0x560df01c4510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:23.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.806+0000 7fb007400700 1 -- 172.21.15.1:0/298124223 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560df01b3b90 msgr2=0x560df01c3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:23.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.806+0000 7fb007400700 1 --2- 172.21.15.1:0/298124223 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560df01b3b90 0x560df01c3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:23.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.806+0000 7fb007400700 1 -- 172.21.15.1:0/298124223 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb000009a80 con 0x560df01b3270 2024-02-02T15:27:23.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.806+0000 7fb006a00700 1 --2- 172.21.15.1:0/298124223 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560df01b3b90 0x560df01c3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:27:23.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.806+0000 7fb007e00700 1 --2- 172.21.15.1:0/298124223 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560df01bea70 0x560df01c4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:27:23.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.807+0000 7fb007400700 1 --2- 172.21.15.1:0/298124223 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560df01b3270 0x560df01c3a90 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7faff800c5b0 tx=0x7faff80988b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=872a994e5ec590bc server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:23.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.807+0000 7faff5600700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:27:23.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.807+0000 7faff5600700 1 -- 172.21.15.1:0/298124223 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff80c96a0 con 0x560df01b3270 2024-02-02T15:27:23.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.807+0000 7faff5600700 1 -- 172.21.15.1:0/298124223 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7faff80a14f0 con 0x560df01b3270 2024-02-02T15:27:23.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.807+0000 7fb021e89b80 1 -- 172.21.15.1:0/298124223 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb00000f050 con 0x560df01b3270 2024-02-02T15:27:23.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.807+0000 7faff5600700 1 -- 172.21.15.1:0/298124223 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7faff8099480 con 0x560df01b3270 2024-02-02T15:27:23.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.807+0000 7fb021e89b80 10 client.15771 fetch_fsmap learned FSMap version 27 2024-02-02T15:27:23.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.807+0000 7fb021e89b80 1 -- 172.21.15.1:0/298124223 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x560df00caf20 con 0x560df01b3270 2024-02-02T15:27:23.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.807+0000 7faff5600700 1 -- 172.21.15.1:0/298124223 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb00000f050 con 0x560df01b3270 2024-02-02T15:27:23.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.807+0000 7faff5600700 1 -- 172.21.15.1:0/298124223 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7faff809e080 con 0x560df01b3270 2024-02-02T15:27:23.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.807+0000 7fb021e89b80 10 client.15771 fetch_fsmap finished waiting for FSMap version 27 2024-02-02T15:27:23.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.808+0000 7fb021e89b80 10 client.15771 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:27:23.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.808+0000 7fb021e89b80 20 client.15771 populate_metadata read hostname 'smithi001' 2024-02-02T15:27:23.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.808+0000 7fb021e89b80 1 --2- 172.21.15.1:0/298124223 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x560df0221400 0x560df02237f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:23.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.808+0000 7fb021e89b80 4 client.15771 mds_command: new command op to 24557 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-02T15:27:23.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.808+0000 7fb021e89b80 1 -- 172.21.15.1:0/298124223 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x560df00c3fd0 con 0x560df0221400 2024-02-02T15:27:23.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.808+0000 7fb006a00700 1 --2- 172.21.15.1:0/298124223 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x560df0221400 0x560df02237f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:23.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.808+0000 7fb006a00700 1 --2- 172.21.15.1:0/298124223 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x560df0221400 0x560df02237f0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f8e14473377628 server_cookie=767e5636e6a4d2b0 in_seq=0 out_seq=0 2024-02-02T15:27:23.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.808+0000 7faff5600700 10 client.15771 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:27:23.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.810+0000 7faff5600700 1 -- 172.21.15.1:0/298124223 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x560df00c3fd0 con 0x560df0221400 2024-02-02T15:27:23.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.810+0000 7faff5600700 10 client.15771 handle_command_reply: tid=0 2024-02-02T15:27:23.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.810+0000 7fb021e89b80 1 client.15771 shutdown 2024-02-02T15:27:23.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.810+0000 7fb021e89b80 1 -- 172.21.15.1:0/298124223 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560df01b3270 msgr2=0x560df01c3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:23.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.810+0000 7fb021e89b80 1 --2- 172.21.15.1:0/298124223 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560df01b3270 0x560df01c3a90 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7faff800c5b0 tx=0x7faff80988b0 comp rx=0 tx=0).stop 2024-02-02T15:27:23.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.811+0000 7fb021e89b80 1 -- 172.21.15.1:0/298124223 shutdown_connections 2024-02-02T15:27:23.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.811+0000 7fb021e89b80 1 --2- 172.21.15.1:0/298124223 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x560df0221400 0x560df02237f0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:23.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.811+0000 7fb021e89b80 1 --2- 172.21.15.1:0/298124223 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560df01bea70 0x560df01c4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:23.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.811+0000 7fb021e89b80 1 --2- 172.21.15.1:0/298124223 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560df01b3270 0x560df01c3a90 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:23.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.811+0000 7fb021e89b80 1 --2- 172.21.15.1:0/298124223 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560df01b3b90 0x560df01c3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:23.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.811+0000 7faff5600700 0 client.15771 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:27:23.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.811+0000 7fb021e89b80 1 -- 172.21.15.1:0/298124223 >> 172.21.15.1:0/298124223 conn(0x560df0112e30 msgr2=0x560df01b5b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:23.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.811+0000 7fb021e89b80 1 -- 172.21.15.1:0/298124223 shutdown_connections 2024-02-02T15:27:23.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.811+0000 7fb021e89b80 1 -- 172.21.15.1:0/298124223 wait complete. 2024-02-02T15:27:23.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.811+0000 7fb021e89b80 20 client.15771 trim_cache size 0 max 16384 2024-02-02T15:27:23.815 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:27:23.815 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2024-02-02T15:27:23.815 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "49338c4a-0237-41cd-86c5-863b43c5f7be", 2024-02-02T15:27:23.815 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2024-02-02T15:27:23.815 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:27:23.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.811+0000 7fb00ce00700 1 -- 172.21.15.1:0/4183924983 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fafe80a07a0 msgr2=0x7fafe80a2c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:23.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.811+0000 7fb00ce00700 1 --2- 172.21.15.1:0/4183924983 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fafe80a07a0 0x7fafe80a2c60 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb00819a360 tx=0x7faff000b850 comp rx=0 tx=0).stop 2024-02-02T15:27:23.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.811+0000 7fb00ce00700 1 -- 172.21.15.1:0/4183924983 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb008100a90 msgr2=0x7fb008198d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:23.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.811+0000 7fb00ce00700 1 --2- 172.21.15.1:0/4183924983 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb008100a90 0x7fb008198d50 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7faff800e9a0 tx=0x7faff800ecb0 comp rx=0 tx=0).stop 2024-02-02T15:27:23.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.812+0000 7fb00ce00700 1 -- 172.21.15.1:0/4183924983 shutdown_connections 2024-02-02T15:27:23.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.812+0000 7fb00ce00700 1 --2- 172.21.15.1:0/4183924983 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fafe80a07a0 0x7fafe80a2c60 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:23.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.812+0000 7fb00ce00700 1 --2- 172.21.15.1:0/4183924983 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb008106aa0 0x7fb00819d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:23.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.812+0000 7fb00ce00700 1 --2- 172.21.15.1:0/4183924983 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb008100a90 0x7fb008198d50 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:23.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.812+0000 7fb00ce00700 1 --2- 172.21.15.1:0/4183924983 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb008101450 0x7fb008199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:23.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.812+0000 7fb00ce00700 1 -- 172.21.15.1:0/4183924983 >> 172.21.15.1:0/4183924983 conn(0x7fb0080fc580 msgr2=0x7fb00810bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:23.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.812+0000 7fb00ce00700 1 -- 172.21.15.1:0/4183924983 shutdown_connections 2024-02-02T15:27:23.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:23.812+0000 7fb00ce00700 1 -- 172.21.15.1:0/4183924983 wait complete. 2024-02-02T15:27:23.831 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-02T15:27:24.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.031+0000 7ff691e00700 1 -- 172.21.15.1:0/2402880396 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff68c106110 msgr2=0x7ff68c10dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:24.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.031+0000 7ff691e00700 1 --2- 172.21.15.1:0/2402880396 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff68c106110 0x7ff68c10dc90 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7ff68400b0a0 tx=0x7ff68400b3b0 comp rx=0 tx=0).stop 2024-02-02T15:27:24.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.031+0000 7ff691e00700 1 -- 172.21.15.1:0/2402880396 shutdown_connections 2024-02-02T15:27:24.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.031+0000 7ff691e00700 1 --2- 172.21.15.1:0/2402880396 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff68c106110 0x7ff68c10dc90 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:24.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.031+0000 7ff691e00700 1 --2- 172.21.15.1:0/2402880396 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff68c100ff0 0x7ff68c1013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:24.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.031+0000 7ff691e00700 1 --2- 172.21.15.1:0/2402880396 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff68c1019a0 0x7ff68c1059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:24.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.031+0000 7ff691e00700 1 -- 172.21.15.1:0/2402880396 >> 172.21.15.1:0/2402880396 conn(0x7ff68c0fc810 msgr2=0x7ff68c0fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:24.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.031+0000 7ff691e00700 1 -- 172.21.15.1:0/2402880396 shutdown_connections 2024-02-02T15:27:24.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.032+0000 7ff691e00700 1 -- 172.21.15.1:0/2402880396 wait complete. 2024-02-02T15:27:24.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.033+0000 7ff691e00700 1 Processor -- start 2024-02-02T15:27:24.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.033+0000 7ff691e00700 1 -- start start 2024-02-02T15:27:24.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.033+0000 7ff691e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff68c100ff0 0x7ff68c1991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:24.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.033+0000 7ff691e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff68c106110 0x7ff68c199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:24.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.033+0000 7ff691e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff68c19db60 0x7ff68c19dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:24.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.033+0000 7ff691e00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff68c19e5e0 con 0x7ff68c100ff0 2024-02-02T15:27:24.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.033+0000 7ff691e00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff68c19e750 con 0x7ff68c19db60 2024-02-02T15:27:24.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.033+0000 7ff691e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff68c19e910 con 0x7ff68c106110 2024-02-02T15:27:24.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.033+0000 7ff68aa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff68c100ff0 0x7ff68c1991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:24.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.033+0000 7ff68a000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff68c106110 0x7ff68c199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:24.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.033+0000 7ff68aa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff68c100ff0 0x7ff68c1991c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33240/0 (socket says 172.21.15.1:33240) 2024-02-02T15:27:24.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.033+0000 7ff68aa00700 1 -- 172.21.15.1:0/1178738892 learned_addr learned my addr 172.21.15.1:0/1178738892 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:24.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.034+0000 7ff68b400700 1 --2- 172.21.15.1:0/1178738892 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff68c19db60 0x7ff68c19dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:24.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.034+0000 7ff68aa00700 1 -- 172.21.15.1:0/1178738892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff68c106110 msgr2=0x7ff68c199700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:24.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.034+0000 7ff68aa00700 1 --2- 172.21.15.1:0/1178738892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff68c106110 0x7ff68c199700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:24.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.034+0000 7ff68aa00700 1 -- 172.21.15.1:0/1178738892 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff68c19db60 msgr2=0x7ff68c19dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:24.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.034+0000 7ff68aa00700 1 --2- 172.21.15.1:0/1178738892 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff68c19db60 0x7ff68c19dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:24.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.034+0000 7ff68aa00700 1 -- 172.21.15.1:0/1178738892 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff684009d00 con 0x7ff68c100ff0 2024-02-02T15:27:24.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.034+0000 7ff68a000700 1 --2- 172.21.15.1:0/1178738892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff68c106110 0x7ff68c199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:27:24.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.034+0000 7ff68aa00700 1 --2- 172.21.15.1:0/1178738892 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff68c100ff0 0x7ff68c1991c0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7ff67c00e9a0 tx=0x7ff67c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:24.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.034+0000 7ff690a00700 1 -- 172.21.15.1:0/1178738892 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff67c009790 con 0x7ff68c100ff0 2024-02-02T15:27:24.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.034+0000 7ff690a00700 1 -- 172.21.15.1:0/1178738892 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff67c010430 con 0x7ff68c100ff0 2024-02-02T15:27:24.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.034+0000 7ff691e00700 1 -- 172.21.15.1:0/1178738892 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff68c1a5e70 con 0x7ff68c100ff0 2024-02-02T15:27:24.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.035+0000 7ff690a00700 1 -- 172.21.15.1:0/1178738892 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff67c018680 con 0x7ff68c100ff0 2024-02-02T15:27:24.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.035+0000 7ff691e00700 1 -- 172.21.15.1:0/1178738892 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff68c1a62f0 con 0x7ff68c100ff0 2024-02-02T15:27:24.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.038+0000 7ff690a00700 1 -- 172.21.15.1:0/1178738892 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135245+0+0 (secure 0 0 0) 0x7ff67c0105a0 con 0x7ff68c100ff0 2024-02-02T15:27:24.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.039+0000 7ff6a5965b80 1 Processor -- start 2024-02-02T15:27:24.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.039+0000 7ff690a00700 1 --2- 172.21.15.1:0/1178738892 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff66c0a0720 0x7ff66c0a2be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:24.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.039+0000 7ff6a5965b80 1 -- start start 2024-02-02T15:27:24.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.039+0000 7ff690a00700 1 -- 172.21.15.1:0/1178738892 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7ff67c014070 con 0x7ff68c100ff0 2024-02-02T15:27:24.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.039+0000 7ff68b400700 1 --2- 172.21.15.1:0/1178738892 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff66c0a0720 0x7ff66c0a2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:24.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.039+0000 7ff6a5965b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5638f98c9010 0x5638f98c93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:24.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.039+0000 7ff6a5965b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5638f98c9930 0x5638f98d42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:24.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.039+0000 7ff6a5965b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5638f98d47e0 0x5638f98d6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:24.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.039+0000 7ff6a5965b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638f983d4c0 con 0x5638f98c9930 2024-02-02T15:27:24.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.039+0000 7ff6a5965b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638f9821aa0 con 0x5638f98d47e0 2024-02-02T15:27:24.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.039+0000 7ff6a5965b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638f981f460 con 0x5638f98c9010 2024-02-02T15:27:24.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.040+0000 7ff68a000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5638f98c9930 0x5638f98d42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:24.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.040+0000 7ff68aa00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5638f98c9010 0x5638f98c93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:24.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.040+0000 7ff68b400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5638f98d47e0 0x5638f98d6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:24.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.040+0000 7ff68aa00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5638f98c9010 0x5638f98c93f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:48498/0 (socket says 172.21.15.1:48498) 2024-02-02T15:27:24.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.040+0000 7ff68a000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5638f98c9930 0x5638f98d42a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33252/0 (socket says 172.21.15.1:33252) 2024-02-02T15:27:24.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.040+0000 7ff68aa00700 1 -- 172.21.15.1:0/2773842651 learned_addr learned my addr 172.21.15.1:0/2773842651 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:24.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.040+0000 7ff68b400700 1 --2- 172.21.15.1:0/1178738892 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff66c0a0720 0x7ff66c0a2be0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff6840062a0 tx=0x7ff684006210 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:24.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.040+0000 7ff68aa00700 1 -- 172.21.15.1:0/2773842651 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5638f98d47e0 msgr2=0x5638f98d6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:24.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.040+0000 7ff68aa00700 1 --2- 172.21.15.1:0/2773842651 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5638f98d47e0 0x5638f98d6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:24.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.040+0000 7ff68aa00700 1 -- 172.21.15.1:0/2773842651 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5638f98c9930 msgr2=0x5638f98d42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:24.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.040+0000 7ff68aa00700 1 --2- 172.21.15.1:0/2773842651 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5638f98c9930 0x5638f98d42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:24.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.040+0000 7ff68aa00700 1 -- 172.21.15.1:0/2773842651 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5638f969c3b0 con 0x5638f98c9010 2024-02-02T15:27:24.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.041+0000 7ff68aa00700 1 --2- 172.21.15.1:0/2773842651 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5638f98c9010 0x5638f98c93f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff67c00c2d0 tx=0x7ff67c099090 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9679f44f6747b360 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:24.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.041+0000 7ff67be00700 1 -- 172.21.15.1:0/2773842651 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff67c099460 con 0x5638f98c9010 2024-02-02T15:27:24.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.041+0000 7ff67be00700 1 -- 172.21.15.1:0/2773842651 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff67c09d290 con 0x5638f98c9010 2024-02-02T15:27:24.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.041+0000 7ff67be00700 1 -- 172.21.15.1:0/2773842651 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff67c0996c0 con 0x5638f98c9010 2024-02-02T15:27:24.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.042+0000 7ff6a5965b80 1 -- 172.21.15.1:0/2773842651 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5638f98c9010 msgr2=0x5638f98c93f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:24.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.042+0000 7ff6a5965b80 1 --2- 172.21.15.1:0/2773842651 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5638f98c9010 0x5638f98c93f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff67c00c2d0 tx=0x7ff67c099090 comp rx=0 tx=0).stop 2024-02-02T15:27:24.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.042+0000 7ff6a5965b80 1 -- 172.21.15.1:0/2773842651 shutdown_connections 2024-02-02T15:27:24.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.042+0000 7ff6a5965b80 1 --2- 172.21.15.1:0/2773842651 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5638f98d47e0 0x5638f98d6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:24.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.042+0000 7ff6a5965b80 1 --2- 172.21.15.1:0/2773842651 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5638f98c9010 0x5638f98c93f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:24.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.042+0000 7ff6a5965b80 1 --2- 172.21.15.1:0/2773842651 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5638f98c9930 0x5638f98d42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:24.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.042+0000 7ff6a5965b80 1 -- 172.21.15.1:0/2773842651 >> 172.21.15.1:0/2773842651 conn(0x5638f9828db0 msgr2=0x5638f98c53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:24.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.042+0000 7ff6a5965b80 1 -- 172.21.15.1:0/2773842651 shutdown_connections 2024-02-02T15:27:24.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.042+0000 7ff6a5965b80 1 -- 172.21.15.1:0/2773842651 wait complete. 2024-02-02T15:27:24.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.042+0000 7ff6a5965b80 1 -- start start 2024-02-02T15:27:24.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.042+0000 7ff6a5965b80 1 Processor -- start 2024-02-02T15:27:24.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.043+0000 7ff6a5965b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5638f98c9010 0x5638f98d4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:24.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.043+0000 7ff6a5965b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5638f98c9930 0x5638f98d14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:24.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.043+0000 7ff6a5965b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5638f98d47e0 0x5638f98d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:24.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.043+0000 7ff6a5965b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638f981ad50 con 0x5638f98c9930 2024-02-02T15:27:24.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.043+0000 7ff6a5965b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638f97b6d80 con 0x5638f98d47e0 2024-02-02T15:27:24.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.043+0000 7ff6a5965b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638f97b6a20 con 0x5638f98c9010 2024-02-02T15:27:24.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.043+0000 7ff6a5965b80 10 client.0 _ll_register_callbacks cb 0x5638f97b8290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:27:24.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.043+0000 7ff68b400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5638f98d47e0 0x5638f98d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:24.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.043+0000 7ff68aa00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5638f98c9010 0x5638f98d4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:24.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.043+0000 7ff68a000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5638f98c9930 0x5638f98d14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:24.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.043+0000 7ff68b400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5638f98d47e0 0x5638f98d1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:43834/0 (socket says 172.21.15.1:43834) 2024-02-02T15:27:24.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.043+0000 7ff68aa00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5638f98c9010 0x5638f98d4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:48508/0 (socket says 172.21.15.1:48508) 2024-02-02T15:27:24.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.043+0000 7ff68b400700 1 -- 172.21.15.1:0/3207370287 learned_addr learned my addr 172.21.15.1:0/3207370287 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:24.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.044+0000 7ff68aa00700 1 -- 172.21.15.1:0/3207370287 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5638f98d47e0 msgr2=0x5638f98d1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:24.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.044+0000 7ff68aa00700 1 --2- 172.21.15.1:0/3207370287 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5638f98d47e0 0x5638f98d1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:24.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.044+0000 7ff68aa00700 1 -- 172.21.15.1:0/3207370287 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5638f98c9930 msgr2=0x5638f98d14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:24.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.044+0000 7ff68aa00700 1 --2- 172.21.15.1:0/3207370287 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5638f98c9930 0x5638f98d14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:24.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.044+0000 7ff68aa00700 1 -- 172.21.15.1:0/3207370287 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff68c1a62f0 con 0x5638f98c9010 2024-02-02T15:27:24.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.044+0000 7ff68b400700 1 --2- 172.21.15.1:0/3207370287 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5638f98d47e0 0x5638f98d1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:27:24.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.044+0000 7ff68a000700 1 --2- 172.21.15.1:0/3207370287 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5638f98c9930 0x5638f98d14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:27:24.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.044+0000 7ff68aa00700 1 --2- 172.21.15.1:0/3207370287 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5638f98c9010 0x5638f98d4230 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff67c00c2d0 tx=0x7ff67c00c400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=acfd876bbd97e635 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:24.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.044+0000 7ff678c00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:27:24.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.044+0000 7ff678c00700 1 -- 172.21.15.1:0/3207370287 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff67c099460 con 0x5638f98c9010 2024-02-02T15:27:24.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.044+0000 7ff678c00700 1 -- 172.21.15.1:0/3207370287 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7ff67c0a14d0 con 0x5638f98c9010 2024-02-02T15:27:24.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.044+0000 7ff6a5965b80 1 -- 172.21.15.1:0/3207370287 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5638f98bd0f0 con 0x5638f98c9010 2024-02-02T15:27:24.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.045+0000 7ff678c00700 1 -- 172.21.15.1:0/3207370287 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff67c010880 con 0x5638f98c9010 2024-02-02T15:27:24.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.045+0000 7ff6a5965b80 10 client.25819 fetch_fsmap learned FSMap version 27 2024-02-02T15:27:24.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.045+0000 7ff6a5965b80 1 -- 172.21.15.1:0/3207370287 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5638f9648b60 con 0x5638f98c9010 2024-02-02T15:27:24.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.045+0000 7ff678c00700 1 -- 172.21.15.1:0/3207370287 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5638f98bd0f0 con 0x5638f98c9010 2024-02-02T15:27:24.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.045+0000 7ff678c00700 1 -- 172.21.15.1:0/3207370287 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff67c0983c0 con 0x5638f98c9010 2024-02-02T15:27:24.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.045+0000 7ff6a5965b80 10 client.25819 fetch_fsmap finished waiting for FSMap version 27 2024-02-02T15:27:24.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.045+0000 7ff6a5965b80 10 client.25819 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:27:24.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.045+0000 7ff6a5965b80 20 client.25819 populate_metadata read hostname 'smithi001' 2024-02-02T15:27:24.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.045+0000 7ff6a5965b80 1 --2- 172.21.15.1:0/3207370287 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5638f98e3d10 0x5638f98e6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:24.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.045+0000 7ff6a5965b80 4 client.25819 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:27:24.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.045+0000 7ff6a5965b80 1 -- 172.21.15.1:0/3207370287 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5638f9813dd0 con 0x5638f98e3d10 2024-02-02T15:27:24.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.045+0000 7ff68a000700 1 --2- 172.21.15.1:0/3207370287 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5638f98e3d10 0x5638f98e6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:24.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.046+0000 7ff68a000700 1 --2- 172.21.15.1:0/3207370287 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5638f98e3d10 0x5638f98e6100 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de9449e6b8b20693 server_cookie=dfa9020422e658b6 in_seq=0 out_seq=0 2024-02-02T15:27:24.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.046+0000 7ff678c00700 10 client.25819 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:27:24.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.047+0000 7ff678c00700 1 -- 172.21.15.1:0/3207370287 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5638f9813dd0 con 0x5638f98e3d10 2024-02-02T15:27:24.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.047+0000 7ff678c00700 10 client.25819 handle_command_reply: tid=0 2024-02-02T15:27:24.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.047+0000 7ff6a5965b80 1 client.25819 shutdown 2024-02-02T15:27:24.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.047+0000 7ff6a5965b80 1 -- 172.21.15.1:0/3207370287 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5638f98c9010 msgr2=0x5638f98d4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:24.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.047+0000 7ff6a5965b80 1 --2- 172.21.15.1:0/3207370287 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5638f98c9010 0x5638f98d4230 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff67c00c2d0 tx=0x7ff67c00c400 comp rx=0 tx=0).stop 2024-02-02T15:27:24.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.047+0000 7ff6a5965b80 1 -- 172.21.15.1:0/3207370287 shutdown_connections 2024-02-02T15:27:24.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.047+0000 7ff6a5965b80 1 --2- 172.21.15.1:0/3207370287 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5638f98e3d10 0x5638f98e6100 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:24.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.047+0000 7ff6a5965b80 1 --2- 172.21.15.1:0/3207370287 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5638f98d47e0 0x5638f98d1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:24.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.047+0000 7ff6a5965b80 1 --2- 172.21.15.1:0/3207370287 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5638f98c9010 0x5638f98d4230 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:24.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.047+0000 7ff6a5965b80 1 --2- 172.21.15.1:0/3207370287 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5638f98c9930 0x5638f98d14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:24.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.047+0000 7ff678c00700 0 client.25819 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:27:24.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.047+0000 7ff6a5965b80 1 -- 172.21.15.1:0/3207370287 >> 172.21.15.1:0/3207370287 conn(0x5638f9828db0 msgr2=0x5638f98c4fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:24.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.047+0000 7ff6a5965b80 1 -- 172.21.15.1:0/3207370287 shutdown_connections 2024-02-02T15:27:24.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.047+0000 7ff6a5965b80 1 -- 172.21.15.1:0/3207370287 wait complete. 2024-02-02T15:27:24.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.047+0000 7ff6a5965b80 20 client.25819 trim_cache size 0 max 16384 2024-02-02T15:27:24.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.064+0000 7ff6a5965b80 1 Processor -- start 2024-02-02T15:27:24.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.064+0000 7ff6a5965b80 1 -- start start 2024-02-02T15:27:24.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.065+0000 7ff6a5965b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5638f98c9010 0x5638f98db2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:24.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.065+0000 7ff6a5965b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5638f98c9930 0x5638f98d0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:24.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.065+0000 7ff6a5965b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5638f98d47e0 0x5638f98c3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:24.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.065+0000 7ff6a5965b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638f969e6a0 con 0x5638f98c9930 2024-02-02T15:27:24.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.065+0000 7ff6a5965b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638f98d21f0 con 0x5638f98d47e0 2024-02-02T15:27:24.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.065+0000 7ff6a5965b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638f98bcf30 con 0x5638f98c9010 2024-02-02T15:27:24.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.065+0000 7ff68a000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5638f98c9930 0x5638f98d0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:24.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.065+0000 7ff68aa00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5638f98c9010 0x5638f98db2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:24.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.065+0000 7ff68aa00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5638f98c9010 0x5638f98db2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:48518/0 (socket says 172.21.15.1:48518) 2024-02-02T15:27:24.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.065+0000 7ff68b400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5638f98d47e0 0x5638f98c3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:24.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.065+0000 7ff68aa00700 1 -- 172.21.15.1:0/1184032225 learned_addr learned my addr 172.21.15.1:0/1184032225 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:24.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.065+0000 7ff68a000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5638f98c9930 0x5638f98d0790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33270/0 (socket says 172.21.15.1:33270) 2024-02-02T15:27:24.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.065+0000 7ff68a000700 1 -- 172.21.15.1:0/1184032225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5638f98c9010 msgr2=0x5638f98db2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:24.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.065+0000 7ff68a000700 1 --2- 172.21.15.1:0/1184032225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5638f98c9010 0x5638f98db2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:24.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.065+0000 7ff68a000700 1 -- 172.21.15.1:0/1184032225 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5638f98d47e0 msgr2=0x5638f98c3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:24.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.065+0000 7ff68a000700 1 --2- 172.21.15.1:0/1184032225 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5638f98d47e0 0x5638f98c3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:24.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.065+0000 7ff68a000700 1 -- 172.21.15.1:0/1184032225 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5638f97e0ea0 con 0x5638f98c9930 2024-02-02T15:27:24.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.065+0000 7ff68aa00700 1 --2- 172.21.15.1:0/1184032225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5638f98c9010 0x5638f98db2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:27:24.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.066+0000 7ff68a000700 1 --2- 172.21.15.1:0/1184032225 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5638f98c9930 0x5638f98d0790 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x5638f98e6940 tx=0x7ff674009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=671113ec882379d4 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:24.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.066+0000 7ff681200700 1 -- 172.21.15.1:0/1184032225 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff674017050 con 0x5638f98c9930 2024-02-02T15:27:24.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.066+0000 7ff681200700 1 -- 172.21.15.1:0/1184032225 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff674018040 con 0x5638f98c9930 2024-02-02T15:27:24.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.066+0000 7ff681200700 1 -- 172.21.15.1:0/1184032225 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff674004a90 con 0x5638f98c9930 2024-02-02T15:27:24.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.066+0000 7ff6a5965b80 1 -- 172.21.15.1:0/1184032225 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5638f98c9930 msgr2=0x5638f98d0790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:24.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.066+0000 7ff6a5965b80 1 --2- 172.21.15.1:0/1184032225 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5638f98c9930 0x5638f98d0790 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x5638f98e6940 tx=0x7ff674009260 comp rx=0 tx=0).stop 2024-02-02T15:27:24.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.067+0000 7ff6a5965b80 1 -- 172.21.15.1:0/1184032225 shutdown_connections 2024-02-02T15:27:24.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.067+0000 7ff6a5965b80 1 --2- 172.21.15.1:0/1184032225 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5638f98d47e0 0x5638f98c3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:24.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.067+0000 7ff6a5965b80 1 --2- 172.21.15.1:0/1184032225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5638f98c9010 0x5638f98db2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:24.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.067+0000 7ff6a5965b80 1 --2- 172.21.15.1:0/1184032225 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5638f98c9930 0x5638f98d0790 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:24.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.067+0000 7ff6a5965b80 1 -- 172.21.15.1:0/1184032225 >> 172.21.15.1:0/1184032225 conn(0x5638f9828db0 msgr2=0x5638f98e05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:24.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.067+0000 7ff6a5965b80 1 -- 172.21.15.1:0/1184032225 shutdown_connections 2024-02-02T15:27:24.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.067+0000 7ff6a5965b80 1 -- 172.21.15.1:0/1184032225 wait complete. 2024-02-02T15:27:24.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.067+0000 7ff6a5965b80 1 -- start start 2024-02-02T15:27:24.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.067+0000 7ff6a5965b80 1 Processor -- start 2024-02-02T15:27:24.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.068+0000 7ff6a5965b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5638f98c9010 0x5638f98d94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:24.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.068+0000 7ff6a5965b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5638f98c9930 0x5638f98d99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:24.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.068+0000 7ff6a5965b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5638f98d47e0 0x5638f9934760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:24.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.068+0000 7ff6a5965b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638f9934ca0 con 0x5638f98c9930 2024-02-02T15:27:24.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.068+0000 7ff6a5965b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638f9934de0 con 0x5638f98c9010 2024-02-02T15:27:24.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.068+0000 7ff6a5965b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638f9934f20 con 0x5638f98d47e0 2024-02-02T15:27:24.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.068+0000 7ff6a5965b80 10 client.0 _ll_register_callbacks cb 0x5638f97b8290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:27:24.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.068+0000 7ff68aa00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5638f98c9010 0x5638f98d94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:24.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.068+0000 7ff68aa00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5638f98c9010 0x5638f98d94a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:43850/0 (socket says 172.21.15.1:43850) 2024-02-02T15:27:24.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.068+0000 7ff68a000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5638f98c9930 0x5638f98d99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:24.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.068+0000 7ff68aa00700 1 -- 172.21.15.1:0/2372586651 learned_addr learned my addr 172.21.15.1:0/2372586651 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:24.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.068+0000 7ff68b400700 1 --2- 172.21.15.1:0/2372586651 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5638f98d47e0 0x5638f9934760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:24.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.068+0000 7ff68aa00700 1 -- 172.21.15.1:0/2372586651 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5638f98d47e0 msgr2=0x5638f9934760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:24.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.068+0000 7ff68aa00700 1 --2- 172.21.15.1:0/2372586651 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5638f98d47e0 0x5638f9934760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:24.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.068+0000 7ff68aa00700 1 -- 172.21.15.1:0/2372586651 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5638f98c9930 msgr2=0x5638f98d99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:24.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.068+0000 7ff68aa00700 1 --2- 172.21.15.1:0/2372586651 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5638f98c9930 0x5638f98d99e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:24.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.068+0000 7ff68aa00700 1 -- 172.21.15.1:0/2372586651 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5638f9648b60 con 0x5638f98c9010 2024-02-02T15:27:24.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.069+0000 7ff68aa00700 1 --2- 172.21.15.1:0/2372586651 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5638f98c9010 0x5638f98d94a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff67c00c510 tx=0x7ff67c09d170 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c6d256591b166e4 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:24.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.069+0000 7ff678c00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:27:24.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.069+0000 7ff678c00700 1 -- 172.21.15.1:0/2372586651 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff67c098dc0 con 0x5638f98c9010 2024-02-02T15:27:24.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.069+0000 7ff6a5965b80 1 -- 172.21.15.1:0/2372586651 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff67c09d290 con 0x5638f98c9010 2024-02-02T15:27:24.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.069+0000 7ff678c00700 1 -- 172.21.15.1:0/2372586651 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7ff67c0a14d0 con 0x5638f98c9010 2024-02-02T15:27:24.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.069+0000 7ff678c00700 1 -- 172.21.15.1:0/2372586651 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff67c0c9680 con 0x5638f98c9010 2024-02-02T15:27:24.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.069+0000 7ff6a5965b80 10 client.25826 fetch_fsmap learned FSMap version 27 2024-02-02T15:27:24.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.069+0000 7ff6a5965b80 1 -- 172.21.15.1:0/2372586651 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7ff674012050 con 0x5638f98c9010 2024-02-02T15:27:24.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.069+0000 7ff678c00700 1 -- 172.21.15.1:0/2372586651 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff67c09d290 con 0x5638f98c9010 2024-02-02T15:27:24.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.069+0000 7ff678c00700 1 -- 172.21.15.1:0/2372586651 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff67c0997e0 con 0x5638f98c9010 2024-02-02T15:27:24.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.069+0000 7ff6a5965b80 10 client.25826 fetch_fsmap finished waiting for FSMap version 27 2024-02-02T15:27:24.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.069+0000 7ff6a5965b80 10 client.25826 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:27:24.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.069+0000 7ff6a5965b80 20 client.25826 populate_metadata read hostname 'smithi001' 2024-02-02T15:27:24.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.069+0000 7ff6a5965b80 1 --2- 172.21.15.1:0/2372586651 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5638f99360f0 0x5638f99384e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:24.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.069+0000 7ff6a5965b80 4 client.25826 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2024-02-02T15:27:24.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.070+0000 7ff6a5965b80 1 -- 172.21.15.1:0/2372586651 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5638f97d9f50 con 0x5638f99360f0 2024-02-02T15:27:24.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.070+0000 7ff68a000700 1 --2- 172.21.15.1:0/2372586651 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5638f99360f0 0x5638f99384e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:24.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.070+0000 7ff68a000700 1 --2- 172.21.15.1:0/2372586651 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5638f99360f0 0x5638f99384e0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=427c5a095a05ec65 server_cookie=116b4f3c72913082 in_seq=0 out_seq=0 2024-02-02T15:27:24.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.070+0000 7ff678c00700 10 client.25826 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:27:24.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.071+0000 7ff678c00700 1 -- 172.21.15.1:0/2372586651 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5638f97d9f50 con 0x5638f99360f0 2024-02-02T15:27:24.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.071+0000 7ff678c00700 10 client.25826 handle_command_reply: tid=0 2024-02-02T15:27:24.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.071+0000 7ff6a5965b80 1 client.25826 shutdown 2024-02-02T15:27:24.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.071+0000 7ff6a5965b80 1 -- 172.21.15.1:0/2372586651 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5638f98c9010 msgr2=0x5638f98d94a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:24.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.071+0000 7ff6a5965b80 1 --2- 172.21.15.1:0/2372586651 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5638f98c9010 0x5638f98d94a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff67c00c510 tx=0x7ff67c09d170 comp rx=0 tx=0).stop 2024-02-02T15:27:24.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.071+0000 7ff6a5965b80 1 -- 172.21.15.1:0/2372586651 shutdown_connections 2024-02-02T15:27:24.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.071+0000 7ff6a5965b80 1 --2- 172.21.15.1:0/2372586651 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5638f99360f0 0x5638f99384e0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:24.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.071+0000 7ff6a5965b80 1 --2- 172.21.15.1:0/2372586651 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5638f98d47e0 0x5638f9934760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:24.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.071+0000 7ff6a5965b80 1 --2- 172.21.15.1:0/2372586651 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5638f98c9010 0x5638f98d94a0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:24.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.071+0000 7ff678c00700 0 client.25826 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:27:24.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.071+0000 7ff6a5965b80 1 --2- 172.21.15.1:0/2372586651 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5638f98c9930 0x5638f98d99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:24.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.071+0000 7ff6a5965b80 1 -- 172.21.15.1:0/2372586651 >> 172.21.15.1:0/2372586651 conn(0x5638f9828db0 msgr2=0x5638f98df8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:24.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.071+0000 7ff6a5965b80 1 -- 172.21.15.1:0/2372586651 shutdown_connections 2024-02-02T15:27:24.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.071+0000 7ff6a5965b80 1 -- 172.21.15.1:0/2372586651 wait complete. 2024-02-02T15:27:24.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.071+0000 7ff6a5965b80 20 client.25826 trim_cache size 0 max 16384 2024-02-02T15:27:24.076 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:27:24.076 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-02T15:27:24.076 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2024-02-02T15:27:24.076 INFO:teuthology.orchestra.run.smithi001.stdout: "49338c4a-0237-41cd-86c5-863b43c5f7be": { 2024-02-02T15:27:24.076 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2024-02-02T15:27:24.076 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "49338c4a-0237-41cd-86c5-863b43c5f7be", 2024-02-02T15:27:24.076 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,force" 2024-02-02T15:27:24.076 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:27:24.076 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:27:24.076 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:27:24.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.072+0000 7ff691e00700 1 -- 172.21.15.1:0/1178738892 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff66c0a0720 msgr2=0x7ff66c0a2be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:24.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.072+0000 7ff691e00700 1 --2- 172.21.15.1:0/1178738892 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff66c0a0720 0x7ff66c0a2be0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff6840062a0 tx=0x7ff684006210 comp rx=0 tx=0).stop 2024-02-02T15:27:24.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.072+0000 7ff691e00700 1 -- 172.21.15.1:0/1178738892 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff68c100ff0 msgr2=0x7ff68c1991c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:24.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.072+0000 7ff691e00700 1 --2- 172.21.15.1:0/1178738892 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff68c100ff0 0x7ff68c1991c0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7ff67c00e9a0 tx=0x7ff67c00ecb0 comp rx=0 tx=0).stop 2024-02-02T15:27:24.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.072+0000 7ff691e00700 1 -- 172.21.15.1:0/1178738892 shutdown_connections 2024-02-02T15:27:24.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.072+0000 7ff691e00700 1 --2- 172.21.15.1:0/1178738892 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff66c0a0720 0x7ff66c0a2be0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:24.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.072+0000 7ff691e00700 1 --2- 172.21.15.1:0/1178738892 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff68c19db60 0x7ff68c19dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:24.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.072+0000 7ff691e00700 1 --2- 172.21.15.1:0/1178738892 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff68c100ff0 0x7ff68c1991c0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:24.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.072+0000 7ff691e00700 1 --2- 172.21.15.1:0/1178738892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff68c106110 0x7ff68c199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:24.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.072+0000 7ff691e00700 1 -- 172.21.15.1:0/1178738892 >> 172.21.15.1:0/1178738892 conn(0x7ff68c0fc810 msgr2=0x7ff68c0fd260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:24.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.072+0000 7ff691e00700 1 -- 172.21.15.1:0/1178738892 shutdown_connections 2024-02-02T15:27:24.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:24.072+0000 7ff691e00700 1 -- 172.21.15.1:0/1178738892 wait complete. 2024-02-02T15:27:24.090 INFO:tasks.cephfs.filesystem:scrub status for tag:49338c4a-0237-41cd-86c5-863b43c5f7be - {'path': '/', 'tag': '49338c4a-0237-41cd-86c5-863b43c5f7be', 'options': 'recursive,force'} 2024-02-02T15:27:25.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:25 smithi071 ceph-mon[44707]: scrub queued for path: / 2024-02-02T15:27:25.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:25 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:27:25.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:25 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:27:25.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:25 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:27:25.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:25 smithi071 ceph-mon[44707]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-02T15:27:25.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:25 smithi142 ceph-mon[45728]: scrub queued for path: / 2024-02-02T15:27:25.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:25 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:27:25.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:25 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:27:25.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:25 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:27:25.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:25 smithi142 ceph-mon[45728]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-02T15:27:25.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:25 smithi001 ceph-mon[35470]: scrub queued for path: / 2024-02-02T15:27:25.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:25 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:27:25.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:25 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:27:25.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:25 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:27:25.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:25 smithi001 ceph-mon[35470]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-02T15:27:27.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:27 smithi071 ceph-mon[44707]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 11 op/s 2024-02-02T15:27:27.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:27 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:27:27.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:27 smithi142 ceph-mon[45728]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 11 op/s 2024-02-02T15:27:27.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:27 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:27:27.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:27 smithi001 ceph-mon[35470]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 11 op/s 2024-02-02T15:27:27.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:27 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:27:29.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:29 smithi071 ceph-mon[44707]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-02-02T15:27:29.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:29 smithi142 ceph-mon[45728]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-02-02T15:27:29.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:29 smithi001 ceph-mon[35470]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-02-02T15:27:31.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:31 smithi071 ceph-mon[44707]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-02-02T15:27:31.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:31 smithi142 ceph-mon[45728]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-02-02T15:27:31.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:31 smithi001 ceph-mon[35470]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-02-02T15:27:33.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:33 smithi071 ceph-mon[44707]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-02-02T15:27:33.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:33 smithi142 ceph-mon[45728]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-02-02T15:27:33.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:33 smithi001 ceph-mon[35470]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 12 op/s 2024-02-02T15:27:35.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:35 smithi071 ceph-mon[44707]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-02-02T15:27:35.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:35 smithi142 ceph-mon[45728]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-02-02T15:27:35.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:35 smithi001 ceph-mon[35470]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-02-02T15:27:37.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:37 smithi071 ceph-mon[44707]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 597 B/s wr, 11 op/s 2024-02-02T15:27:37.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:37 smithi142 ceph-mon[45728]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 597 B/s wr, 11 op/s 2024-02-02T15:27:37.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:37 smithi001 ceph-mon[35470]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 597 B/s wr, 11 op/s 2024-02-02T15:27:39.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:39 smithi071 ceph-mon[44707]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 9 op/s 2024-02-02T15:27:39.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:39 smithi142 ceph-mon[45728]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 9 op/s 2024-02-02T15:27:39.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:39 smithi001 ceph-mon[35470]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 9 op/s 2024-02-02T15:27:41.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:41 smithi071 ceph-mon[44707]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 8 op/s 2024-02-02T15:27:41.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:41 smithi142 ceph-mon[45728]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 8 op/s 2024-02-02T15:27:41.689 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:41 smithi001 ceph-mon[35470]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 8 op/s 2024-02-02T15:27:43.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:43 smithi071 ceph-mon[44707]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 682 B/s wr, 12 op/s 2024-02-02T15:27:43.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:43 smithi142 ceph-mon[45728]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 682 B/s wr, 12 op/s 2024-02-02T15:27:43.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:43 smithi001 ceph-mon[35470]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 682 B/s wr, 12 op/s 2024-02-02T15:27:45.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:45 smithi071 ceph-mon[44707]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 8 op/s 2024-02-02T15:27:45.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:45 smithi142 ceph-mon[45728]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 8 op/s 2024-02-02T15:27:45.689 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:45 smithi001 ceph-mon[35470]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 8 op/s 2024-02-02T15:27:47.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:47 smithi071 ceph-mon[44707]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 682 B/s wr, 12 op/s 2024-02-02T15:27:47.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:47 smithi142 ceph-mon[45728]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 682 B/s wr, 12 op/s 2024-02-02T15:27:47.689 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:47 smithi001 ceph-mon[35470]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 682 B/s wr, 12 op/s 2024-02-02T15:27:49.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:49 smithi071 ceph-mon[44707]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-02-02T15:27:49.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:49 smithi142 ceph-mon[45728]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-02-02T15:27:49.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:49 smithi001 ceph-mon[35470]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 9 op/s 2024-02-02T15:27:51.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:51 smithi071 ceph-mon[44707]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 8 op/s 2024-02-02T15:27:51.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:51 smithi142 ceph-mon[45728]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 8 op/s 2024-02-02T15:27:51.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:51 smithi001 ceph-mon[35470]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 8 op/s 2024-02-02T15:27:53.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:53 smithi071 ceph-mon[44707]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 85 B/s wr, 12 op/s 2024-02-02T15:27:53.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:53 smithi142 ceph-mon[45728]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 85 B/s wr, 12 op/s 2024-02-02T15:27:53.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:53 smithi001 ceph-mon[35470]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 85 B/s wr, 12 op/s 2024-02-02T15:27:54.091 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-02T15:27:54.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.312+0000 7fadce200700 1 -- 172.21.15.1:0/73044678 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fadc81059e0 msgr2=0x7fadc8105dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:54.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.312+0000 7fadce200700 1 --2- 172.21.15.1:0/73044678 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fadc81059e0 0x7fadc8105dc0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fadb8009990 tx=0x7fadb8009ca0 comp rx=0 tx=0).stop 2024-02-02T15:27:54.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.312+0000 7fadce200700 1 -- 172.21.15.1:0/73044678 shutdown_connections 2024-02-02T15:27:54.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.312+0000 7fadce200700 1 --2- 172.21.15.1:0/73044678 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fadc8100390 0x7fadc810c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:54.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.312+0000 7fadce200700 1 --2- 172.21.15.1:0/73044678 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fadc81059e0 0x7fadc8105dc0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:54.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.312+0000 7fadce200700 1 --2- 172.21.15.1:0/73044678 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fadc80ff9d0 0x7fadc80ffe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:54.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.312+0000 7fadce200700 1 -- 172.21.15.1:0/73044678 >> 172.21.15.1:0/73044678 conn(0x7fadc80fb4e0 msgr2=0x7fadc80fd900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:54.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.312+0000 7fadce200700 1 -- 172.21.15.1:0/73044678 shutdown_connections 2024-02-02T15:27:54.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.313+0000 7fadce200700 1 -- 172.21.15.1:0/73044678 wait complete. 2024-02-02T15:27:54.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.314+0000 7fadce200700 1 Processor -- start 2024-02-02T15:27:54.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.314+0000 7fadce200700 1 -- start start 2024-02-02T15:27:54.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.315+0000 7fadce200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fadc80ff9d0 0x7fadc8198de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:54.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.315+0000 7fadce200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fadc8100390 0x7fadc8199320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:54.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.315+0000 7fadce200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fadc81059e0 0x7fadc819d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:54.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.315+0000 7fadce200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc819ddc0 con 0x7fadc80ff9d0 2024-02-02T15:27:54.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.315+0000 7fadce200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc819df30 con 0x7fadc8100390 2024-02-02T15:27:54.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.315+0000 7fadce200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc819e0f0 con 0x7fadc81059e0 2024-02-02T15:27:54.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.315+0000 7fadcce00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fadc80ff9d0 0x7fadc8198de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:54.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.315+0000 7fadcce00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fadc80ff9d0 0x7fadc8198de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33058/0 (socket says 172.21.15.1:33058) 2024-02-02T15:27:54.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.315+0000 7fadcce00700 1 -- 172.21.15.1:0/1357109311 learned_addr learned my addr 172.21.15.1:0/1357109311 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:54.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.315+0000 7fadcd800700 1 --2- 172.21.15.1:0/1357109311 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fadc81059e0 0x7fadc819d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:54.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.316+0000 7fadc7e00700 1 --2- 172.21.15.1:0/1357109311 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fadc8100390 0x7fadc8199320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:54.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.316+0000 7fadcce00700 1 -- 172.21.15.1:0/1357109311 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fadc81059e0 msgr2=0x7fadc819d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:54.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.316+0000 7fadcce00700 1 --2- 172.21.15.1:0/1357109311 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fadc81059e0 0x7fadc819d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:54.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.316+0000 7fadcce00700 1 -- 172.21.15.1:0/1357109311 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fadc8100390 msgr2=0x7fadc8199320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:54.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.316+0000 7fadcce00700 1 --2- 172.21.15.1:0/1357109311 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fadc8100390 0x7fadc8199320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:54.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.316+0000 7fadcce00700 1 -- 172.21.15.1:0/1357109311 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadb80095f0 con 0x7fadc80ff9d0 2024-02-02T15:27:54.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.316+0000 7fadcd800700 1 --2- 172.21.15.1:0/1357109311 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fadc81059e0 0x7fadc819d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:27:54.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.316+0000 7fadcce00700 1 --2- 172.21.15.1:0/1357109311 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fadc80ff9d0 0x7fadc8198de0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fadb8005910 tx=0x7fadb8005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:54.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.317+0000 7fadc5600700 1 -- 172.21.15.1:0/1357109311 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb801d070 con 0x7fadc80ff9d0 2024-02-02T15:27:54.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.317+0000 7fadc5600700 1 -- 172.21.15.1:0/1357109311 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fadb8022470 con 0x7fadc80ff9d0 2024-02-02T15:27:54.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.317+0000 7fadce200700 1 -- 172.21.15.1:0/1357109311 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadc819e370 con 0x7fadc80ff9d0 2024-02-02T15:27:54.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.317+0000 7fadc5600700 1 -- 172.21.15.1:0/1357109311 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb8017650 con 0x7fadc80ff9d0 2024-02-02T15:27:54.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.317+0000 7fadce200700 1 -- 172.21.15.1:0/1357109311 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadc81a5c10 con 0x7fadc80ff9d0 2024-02-02T15:27:54.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.320+0000 7fadc5600700 1 -- 172.21.15.1:0/1357109311 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135245+0+0 (secure 0 0 0) 0x7fadb800fad0 con 0x7fadc80ff9d0 2024-02-02T15:27:54.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.321+0000 7fadc5600700 1 --2- 172.21.15.1:0/1357109311 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fada80a0720 0x7fada80a2be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:54.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.322+0000 7fadc5600700 1 -- 172.21.15.1:0/1357109311 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fadb80cd780 con 0x7fadc80ff9d0 2024-02-02T15:27:54.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.322+0000 7fadc7e00700 1 --2- 172.21.15.1:0/1357109311 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fada80a0720 0x7fada80a2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:54.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.323+0000 7fadc7e00700 1 --2- 172.21.15.1:0/1357109311 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fada80a0720 0x7fada80a2be0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fadc819a3f0 tx=0x7fadb0009260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:54.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.323+0000 7fade33aeb80 1 Processor -- start 2024-02-02T15:27:54.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.323+0000 7fade33aeb80 1 -- start start 2024-02-02T15:27:54.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.324+0000 7fade33aeb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559cf24851f0 0x559cf24855d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:54.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.324+0000 7fade33aeb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559cf2485b10 0x559cf24904b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:54.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.324+0000 7fade33aeb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559cf24909f0 0x559cf2492de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:54.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.324+0000 7fade33aeb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cf23f94c0 con 0x559cf24851f0 2024-02-02T15:27:54.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.324+0000 7fade33aeb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cf23ddaa0 con 0x559cf24909f0 2024-02-02T15:27:54.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.324+0000 7fade33aeb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cf23db460 con 0x559cf2485b10 2024-02-02T15:27:54.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.324+0000 7fadcd800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559cf24909f0 0x559cf2492de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:54.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.324+0000 7fadc7e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559cf2485b10 0x559cf24904b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:54.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.324+0000 7fadcd800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559cf24909f0 0x559cf2492de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:36782/0 (socket says 172.21.15.1:36782) 2024-02-02T15:27:54.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.324+0000 7fadcd800700 1 -- 172.21.15.1:0/2218871362 learned_addr learned my addr 172.21.15.1:0/2218871362 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:54.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.324+0000 7fadcd800700 1 -- 172.21.15.1:0/2218871362 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559cf2485b10 msgr2=0x559cf24904b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:54.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.324+0000 7fadcd800700 1 --2- 172.21.15.1:0/2218871362 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559cf2485b10 0x559cf24904b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:54.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.324+0000 7fadcd800700 1 -- 172.21.15.1:0/2218871362 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559cf24851f0 msgr2=0x559cf24855d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:54.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.324+0000 7fadcd800700 1 --2- 172.21.15.1:0/2218871362 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559cf24851f0 0x559cf24855d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:54.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.324+0000 7fadcd800700 1 -- 172.21.15.1:0/2218871362 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559cf22583b0 con 0x559cf24909f0 2024-02-02T15:27:54.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.325+0000 7fadcd800700 1 --2- 172.21.15.1:0/2218871362 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559cf24909f0 0x559cf2492de0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fadc0007740 tx=0x7fadc0009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8948a4f85b6d31b2 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:54.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.325+0000 7fadbe000700 1 -- 172.21.15.1:0/2218871362 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc0015850 con 0x559cf24909f0 2024-02-02T15:27:54.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.325+0000 7fadbe000700 1 -- 172.21.15.1:0/2218871362 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fadc000f040 con 0x559cf24909f0 2024-02-02T15:27:54.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.325+0000 7fadbe000700 1 -- 172.21.15.1:0/2218871362 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc0014d50 con 0x559cf24909f0 2024-02-02T15:27:54.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.326+0000 7fade33aeb80 1 -- 172.21.15.1:0/2218871362 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559cf24909f0 msgr2=0x559cf2492de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:54.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.326+0000 7fade33aeb80 1 --2- 172.21.15.1:0/2218871362 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559cf24909f0 0x559cf2492de0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fadc0007740 tx=0x7fadc0009f90 comp rx=0 tx=0).stop 2024-02-02T15:27:54.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.326+0000 7fade33aeb80 1 -- 172.21.15.1:0/2218871362 shutdown_connections 2024-02-02T15:27:54.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.326+0000 7fade33aeb80 1 --2- 172.21.15.1:0/2218871362 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559cf24909f0 0x559cf2492de0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:54.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.326+0000 7fade33aeb80 1 --2- 172.21.15.1:0/2218871362 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559cf24851f0 0x559cf24855d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:54.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.326+0000 7fade33aeb80 1 --2- 172.21.15.1:0/2218871362 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559cf2485b10 0x559cf24904b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:54.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.326+0000 7fade33aeb80 1 -- 172.21.15.1:0/2218871362 >> 172.21.15.1:0/2218871362 conn(0x559cf23e4db0 msgr2=0x559cf24815c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:54.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.326+0000 7fade33aeb80 1 -- 172.21.15.1:0/2218871362 shutdown_connections 2024-02-02T15:27:54.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.326+0000 7fade33aeb80 1 -- 172.21.15.1:0/2218871362 wait complete. 2024-02-02T15:27:54.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.327+0000 7fade33aeb80 1 -- start start 2024-02-02T15:27:54.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.327+0000 7fade33aeb80 1 Processor -- start 2024-02-02T15:27:54.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.328+0000 7fade33aeb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559cf24851f0 0x559cf2490430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:54.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.328+0000 7fade33aeb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559cf2485b10 0x559cf248d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:54.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.328+0000 7fade33aeb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559cf24909f0 0x559cf248de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:54.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.328+0000 7fade33aeb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cf23d6d50 con 0x559cf24851f0 2024-02-02T15:27:54.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.328+0000 7fade33aeb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cf2372d80 con 0x559cf2485b10 2024-02-02T15:27:54.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.328+0000 7fade33aeb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cf2372a20 con 0x559cf24909f0 2024-02-02T15:27:54.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.328+0000 7fade33aeb80 10 client.0 _ll_register_callbacks cb 0x559cf2374290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:27:54.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.328+0000 7fadcce00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559cf24851f0 0x559cf2490430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:54.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.328+0000 7fadc7e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559cf2485b10 0x559cf248d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:54.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.328+0000 7fadcce00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559cf24851f0 0x559cf2490430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33072/0 (socket says 172.21.15.1:33072) 2024-02-02T15:27:54.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.328+0000 7fadcce00700 1 -- 172.21.15.1:0/1798064415 learned_addr learned my addr 172.21.15.1:0/1798064415 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:54.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.328+0000 7fadcd800700 1 --2- 172.21.15.1:0/1798064415 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559cf24909f0 0x559cf248de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:54.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.329+0000 7fadc7e00700 1 -- 172.21.15.1:0/1798064415 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559cf24909f0 msgr2=0x559cf248de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:54.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.329+0000 7fadc7e00700 1 --2- 172.21.15.1:0/1798064415 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559cf24909f0 0x559cf248de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:54.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.329+0000 7fadc7e00700 1 -- 172.21.15.1:0/1798064415 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559cf24851f0 msgr2=0x559cf2490430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:54.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.329+0000 7fadc7e00700 1 --2- 172.21.15.1:0/1798064415 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559cf24851f0 0x559cf2490430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:54.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.329+0000 7fadc7e00700 1 -- 172.21.15.1:0/1798064415 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadc0009e00 con 0x559cf2485b10 2024-02-02T15:27:54.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.329+0000 7fadcce00700 1 --2- 172.21.15.1:0/1798064415 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559cf24851f0 0x559cf2490430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:27:54.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.329+0000 7fadcd800700 1 --2- 172.21.15.1:0/1798064415 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559cf24909f0 0x559cf248de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:27:54.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.329+0000 7fadc7e00700 1 --2- 172.21.15.1:0/1798064415 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559cf2485b10 0x559cf248d720 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fadb00117d0 tx=0x7fadb00047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2efa6297b509d0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:54.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.329+0000 7fadb6a00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:27:54.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.329+0000 7fadb6a00700 1 -- 172.21.15.1:0/1798064415 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb0022070 con 0x559cf2485b10 2024-02-02T15:27:54.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.330+0000 7fade33aeb80 1 -- 172.21.15.1:0/1798064415 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559cf2479340 con 0x559cf2485b10 2024-02-02T15:27:54.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.330+0000 7fadb6a00700 1 -- 172.21.15.1:0/1798064415 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fadb0003e20 con 0x559cf2485b10 2024-02-02T15:27:54.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.330+0000 7fadb6a00700 1 -- 172.21.15.1:0/1798064415 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fadb0027420 con 0x559cf2485b10 2024-02-02T15:27:54.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.330+0000 7fade33aeb80 10 client.25838 fetch_fsmap learned FSMap version 27 2024-02-02T15:27:54.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.330+0000 7fade33aeb80 1 -- 172.21.15.1:0/1798064415 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x559cf2204b60 con 0x559cf2485b10 2024-02-02T15:27:54.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.330+0000 7fadb6a00700 1 -- 172.21.15.1:0/1798064415 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559cf2479340 con 0x559cf2485b10 2024-02-02T15:27:54.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.330+0000 7fadb6a00700 1 -- 172.21.15.1:0/1798064415 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fadb001c5d0 con 0x559cf2485b10 2024-02-02T15:27:54.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.331+0000 7fade33aeb80 10 client.25838 fetch_fsmap finished waiting for FSMap version 27 2024-02-02T15:27:54.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.331+0000 7fade33aeb80 10 client.25838 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:27:54.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.331+0000 7fade33aeb80 20 client.25838 populate_metadata read hostname 'smithi001' 2024-02-02T15:27:54.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.331+0000 7fade33aeb80 1 --2- 172.21.15.1:0/1798064415 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x559cf24a0460 0x559cf24a2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:54.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.331+0000 7fade33aeb80 4 client.25838 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:27:54.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.331+0000 7fade33aeb80 1 -- 172.21.15.1:0/1798064415 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559cf23cfdd0 con 0x559cf24a0460 2024-02-02T15:27:54.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.331+0000 7fadcce00700 1 --2- 172.21.15.1:0/1798064415 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x559cf24a0460 0x559cf24a2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:54.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.332+0000 7fadcce00700 1 --2- 172.21.15.1:0/1798064415 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x559cf24a0460 0x559cf24a2880 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=916a4e1e6eb69105 server_cookie=2ef7a1124b01ba53 in_seq=0 out_seq=0 2024-02-02T15:27:54.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.332+0000 7fadb6a00700 10 client.25838 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:27:54.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.332+0000 7fadb6a00700 1 -- 172.21.15.1:0/1798064415 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x559cf23cfdd0 con 0x559cf24a0460 2024-02-02T15:27:54.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.332+0000 7fadb6a00700 10 client.25838 handle_command_reply: tid=0 2024-02-02T15:27:54.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.332+0000 7fade33aeb80 1 client.25838 shutdown 2024-02-02T15:27:54.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.333+0000 7fade33aeb80 1 -- 172.21.15.1:0/1798064415 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559cf2485b10 msgr2=0x559cf248d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:54.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.333+0000 7fade33aeb80 1 --2- 172.21.15.1:0/1798064415 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559cf2485b10 0x559cf248d720 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fadb00117d0 tx=0x7fadb00047c0 comp rx=0 tx=0).stop 2024-02-02T15:27:54.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.333+0000 7fade33aeb80 1 -- 172.21.15.1:0/1798064415 shutdown_connections 2024-02-02T15:27:54.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.333+0000 7fade33aeb80 1 --2- 172.21.15.1:0/1798064415 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x559cf24a0460 0x559cf24a2880 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:54.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.333+0000 7fade33aeb80 1 --2- 172.21.15.1:0/1798064415 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559cf24909f0 0x559cf248de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:54.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.333+0000 7fade33aeb80 1 --2- 172.21.15.1:0/1798064415 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559cf24851f0 0x559cf2490430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:54.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.333+0000 7fade33aeb80 1 --2- 172.21.15.1:0/1798064415 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559cf2485b10 0x559cf248d720 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:54.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.333+0000 7fade33aeb80 1 -- 172.21.15.1:0/1798064415 >> 172.21.15.1:0/1798064415 conn(0x559cf23e4db0 msgr2=0x559cf24811b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:54.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.333+0000 7fadb6a00700 0 client.25838 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:27:54.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.333+0000 7fade33aeb80 1 -- 172.21.15.1:0/1798064415 shutdown_connections 2024-02-02T15:27:54.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.333+0000 7fade33aeb80 1 -- 172.21.15.1:0/1798064415 wait complete. 2024-02-02T15:27:54.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.333+0000 7fade33aeb80 20 client.25838 trim_cache size 0 max 16384 2024-02-02T15:27:54.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.359+0000 7fade33aeb80 1 Processor -- start 2024-02-02T15:27:54.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.359+0000 7fade33aeb80 1 -- start start 2024-02-02T15:27:54.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.360+0000 7fade33aeb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559cf24851f0 0x559cf248ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:54.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.360+0000 7fade33aeb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559cf2485b10 0x559cf2488a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:54.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.360+0000 7fade33aeb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559cf24909f0 0x559cf2488f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:54.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.360+0000 7fade33aeb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cf225a6a0 con 0x559cf2485b10 2024-02-02T15:27:54.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.360+0000 7fade33aeb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cf24881a0 con 0x559cf24851f0 2024-02-02T15:27:54.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.360+0000 7fade33aeb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cf248bcb0 con 0x559cf24909f0 2024-02-02T15:27:54.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.360+0000 7fadc7e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559cf2485b10 0x559cf2488a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:54.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.360+0000 7fadc7e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559cf2485b10 0x559cf2488a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33078/0 (socket says 172.21.15.1:33078) 2024-02-02T15:27:54.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.360+0000 7fadc7e00700 1 -- 172.21.15.1:0/3866822944 learned_addr learned my addr 172.21.15.1:0/3866822944 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:54.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.360+0000 7fadcce00700 1 --2- 172.21.15.1:0/3866822944 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559cf24851f0 0x559cf248ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:54.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.360+0000 7fadcd800700 1 --2- 172.21.15.1:0/3866822944 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559cf24909f0 0x559cf2488f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:54.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.361+0000 7fadc7e00700 1 -- 172.21.15.1:0/3866822944 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559cf24909f0 msgr2=0x559cf2488f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:54.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.361+0000 7fadc7e00700 1 --2- 172.21.15.1:0/3866822944 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559cf24909f0 0x559cf2488f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:54.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.361+0000 7fadc7e00700 1 -- 172.21.15.1:0/3866822944 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559cf24851f0 msgr2=0x559cf248ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:54.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.361+0000 7fadc7e00700 1 --2- 172.21.15.1:0/3866822944 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559cf24851f0 0x559cf248ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:54.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.361+0000 7fadc7e00700 1 -- 172.21.15.1:0/3866822944 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559cf239cea0 con 0x559cf2485b10 2024-02-02T15:27:54.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.361+0000 7fadcce00700 1 --2- 172.21.15.1:0/3866822944 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559cf24851f0 0x559cf248ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:27:54.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.361+0000 7fadc7e00700 1 --2- 172.21.15.1:0/3866822944 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559cf2485b10 0x559cf2488a10 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fadb0009b20 tx=0x7fadb0009350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9848349cf7cdf421 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:54.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.361+0000 7fadbea00700 1 -- 172.21.15.1:0/3866822944 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb0022070 con 0x559cf2485b10 2024-02-02T15:27:54.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.361+0000 7fadbea00700 1 -- 172.21.15.1:0/3866822944 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fadb0027420 con 0x559cf2485b10 2024-02-02T15:27:54.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.361+0000 7fadbea00700 1 -- 172.21.15.1:0/3866822944 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb0027920 con 0x559cf2485b10 2024-02-02T15:27:54.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.362+0000 7fade33aeb80 1 -- 172.21.15.1:0/3866822944 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559cf2485b10 msgr2=0x559cf2488a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:54.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.362+0000 7fade33aeb80 1 --2- 172.21.15.1:0/3866822944 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559cf2485b10 0x559cf2488a10 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fadb0009b20 tx=0x7fadb0009350 comp rx=0 tx=0).stop 2024-02-02T15:27:54.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.362+0000 7fade33aeb80 1 -- 172.21.15.1:0/3866822944 shutdown_connections 2024-02-02T15:27:54.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.362+0000 7fade33aeb80 1 --2- 172.21.15.1:0/3866822944 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559cf24909f0 0x559cf2488f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:54.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.362+0000 7fade33aeb80 1 --2- 172.21.15.1:0/3866822944 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559cf24851f0 0x559cf248ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:54.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.362+0000 7fade33aeb80 1 --2- 172.21.15.1:0/3866822944 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559cf2485b10 0x559cf2488a10 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:54.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.362+0000 7fade33aeb80 1 -- 172.21.15.1:0/3866822944 >> 172.21.15.1:0/3866822944 conn(0x559cf23e4db0 msgr2=0x559cf247fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:54.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.362+0000 7fade33aeb80 1 -- 172.21.15.1:0/3866822944 shutdown_connections 2024-02-02T15:27:54.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.362+0000 7fade33aeb80 1 -- 172.21.15.1:0/3866822944 wait complete. 2024-02-02T15:27:54.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.363+0000 7fade33aeb80 1 -- start start 2024-02-02T15:27:54.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.363+0000 7fade33aeb80 1 Processor -- start 2024-02-02T15:27:54.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.364+0000 7fade33aeb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559cf24851f0 0x559cf2496d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:54.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.364+0000 7fade33aeb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559cf2485b10 0x559cf2497270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:54.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.364+0000 7fade33aeb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559cf24909f0 0x559cf24951d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:54.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.364+0000 7fade33aeb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cf24958e0 con 0x559cf24909f0 2024-02-02T15:27:54.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.364+0000 7fade33aeb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cf2495a50 con 0x559cf24851f0 2024-02-02T15:27:54.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.364+0000 7fade33aeb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cf2495c10 con 0x559cf2485b10 2024-02-02T15:27:54.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.364+0000 7fade33aeb80 10 client.0 _ll_register_callbacks cb 0x559cf2374290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:27:54.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.364+0000 7fadcce00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559cf24851f0 0x559cf2496d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:54.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.364+0000 7fadcce00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559cf24851f0 0x559cf2496d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:36816/0 (socket says 172.21.15.1:36816) 2024-02-02T15:27:54.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.364+0000 7fadcce00700 1 -- 172.21.15.1:0/917886804 learned_addr learned my addr 172.21.15.1:0/917886804 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:54.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.365+0000 7fadc7e00700 1 --2- 172.21.15.1:0/917886804 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559cf2485b10 0x559cf2497270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:54.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.365+0000 7fadcce00700 1 -- 172.21.15.1:0/917886804 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559cf2485b10 msgr2=0x559cf2497270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:54.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.365+0000 7fadcce00700 1 --2- 172.21.15.1:0/917886804 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559cf2485b10 0x559cf2497270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:54.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.365+0000 7fadcce00700 1 -- 172.21.15.1:0/917886804 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559cf24909f0 msgr2=0x559cf24951d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:54.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.365+0000 7fadcce00700 1 --2- 172.21.15.1:0/917886804 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559cf24909f0 0x559cf24951d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:54.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.365+0000 7fadcce00700 1 -- 172.21.15.1:0/917886804 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadc81a5c10 con 0x559cf24851f0 2024-02-02T15:27:54.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.365+0000 7fadcce00700 1 --2- 172.21.15.1:0/917886804 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559cf24851f0 0x559cf2496d30 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fadb8096e30 tx=0x7fadb801a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2632f1b89c6bf350 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:54.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.365+0000 7fadb6a00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:27:54.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.365+0000 7fadb6a00700 1 -- 172.21.15.1:0/917886804 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb809a930 con 0x559cf24851f0 2024-02-02T15:27:54.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.365+0000 7fadb6a00700 1 -- 172.21.15.1:0/917886804 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fadb809e760 con 0x559cf24851f0 2024-02-02T15:27:54.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.365+0000 7fade33aeb80 1 -- 172.21.15.1:0/917886804 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fadc000f040 con 0x559cf24851f0 2024-02-02T15:27:54.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.365+0000 7fadb6a00700 1 -- 172.21.15.1:0/917886804 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fadb80a2910 con 0x559cf24851f0 2024-02-02T15:27:54.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.366+0000 7fade33aeb80 10 client.25844 fetch_fsmap learned FSMap version 27 2024-02-02T15:27:54.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.366+0000 7fade33aeb80 1 -- 172.21.15.1:0/917886804 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x559cf2204b60 con 0x559cf24851f0 2024-02-02T15:27:54.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.366+0000 7fadb6a00700 1 -- 172.21.15.1:0/917886804 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fadc000f040 con 0x559cf24851f0 2024-02-02T15:27:54.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.366+0000 7fadb6a00700 1 -- 172.21.15.1:0/917886804 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fadb809ab90 con 0x559cf24851f0 2024-02-02T15:27:54.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.366+0000 7fade33aeb80 10 client.25844 fetch_fsmap finished waiting for FSMap version 27 2024-02-02T15:27:54.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.366+0000 7fade33aeb80 10 client.25844 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:27:54.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.366+0000 7fade33aeb80 20 client.25844 populate_metadata read hostname 'smithi001' 2024-02-02T15:27:54.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.366+0000 7fade33aeb80 1 --2- 172.21.15.1:0/917886804 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x559cf24f2f30 0x559cf24f5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:54.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.366+0000 7fade33aeb80 4 client.25844 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2024-02-02T15:27:54.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.366+0000 7fade33aeb80 1 -- 172.21.15.1:0/917886804 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559cf2395f50 con 0x559cf24f2f30 2024-02-02T15:27:54.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.366+0000 7fadc7e00700 1 --2- 172.21.15.1:0/917886804 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x559cf24f2f30 0x559cf24f5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:54.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.367+0000 7fadc7e00700 1 --2- 172.21.15.1:0/917886804 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x559cf24f2f30 0x559cf24f5320 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a5c9e6183d30ff8 server_cookie=18dfdf163fdf5b2c in_seq=0 out_seq=0 2024-02-02T15:27:54.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.367+0000 7fadb6a00700 10 client.25844 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:27:54.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.367+0000 7fadb6a00700 1 -- 172.21.15.1:0/917886804 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x559cf2395f50 con 0x559cf24f2f30 2024-02-02T15:27:54.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.367+0000 7fadb6a00700 10 client.25844 handle_command_reply: tid=0 2024-02-02T15:27:54.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.367+0000 7fade33aeb80 1 client.25844 shutdown 2024-02-02T15:27:54.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.367+0000 7fade33aeb80 1 -- 172.21.15.1:0/917886804 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559cf24851f0 msgr2=0x559cf2496d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:54.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.367+0000 7fade33aeb80 1 --2- 172.21.15.1:0/917886804 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559cf24851f0 0x559cf2496d30 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fadb8096e30 tx=0x7fadb801a040 comp rx=0 tx=0).stop 2024-02-02T15:27:54.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.367+0000 7fade33aeb80 1 -- 172.21.15.1:0/917886804 shutdown_connections 2024-02-02T15:27:54.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.367+0000 7fade33aeb80 1 --2- 172.21.15.1:0/917886804 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x559cf24f2f30 0x559cf24f5320 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:54.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.367+0000 7fade33aeb80 1 --2- 172.21.15.1:0/917886804 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559cf24909f0 0x559cf24951d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:54.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.367+0000 7fade33aeb80 1 --2- 172.21.15.1:0/917886804 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559cf24851f0 0x559cf2496d30 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:54.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.367+0000 7fadb6a00700 0 client.25844 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:27:54.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.367+0000 7fade33aeb80 1 --2- 172.21.15.1:0/917886804 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559cf2485b10 0x559cf2497270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:54.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.367+0000 7fade33aeb80 1 -- 172.21.15.1:0/917886804 >> 172.21.15.1:0/917886804 conn(0x559cf23e4db0 msgr2=0x559cf2488a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:54.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.368+0000 7fade33aeb80 1 -- 172.21.15.1:0/917886804 shutdown_connections 2024-02-02T15:27:54.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.368+0000 7fade33aeb80 1 -- 172.21.15.1:0/917886804 wait complete. 2024-02-02T15:27:54.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.368+0000 7fade33aeb80 20 client.25844 trim_cache size 0 max 16384 2024-02-02T15:27:54.371 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:27:54.372 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2024-02-02T15:27:54.372 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2024-02-02T15:27:54.372 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:27:54.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.368+0000 7fadce200700 1 -- 172.21.15.1:0/1357109311 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fada80a0720 msgr2=0x7fada80a2be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:54.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.368+0000 7fadce200700 1 --2- 172.21.15.1:0/1357109311 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fada80a0720 0x7fada80a2be0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fadc819a3f0 tx=0x7fadb0009260 comp rx=0 tx=0).stop 2024-02-02T15:27:54.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.368+0000 7fadce200700 1 -- 172.21.15.1:0/1357109311 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fadc80ff9d0 msgr2=0x7fadc8198de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:54.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.368+0000 7fadce200700 1 --2- 172.21.15.1:0/1357109311 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fadc80ff9d0 0x7fadc8198de0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fadb8005910 tx=0x7fadb8005740 comp rx=0 tx=0).stop 2024-02-02T15:27:54.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.369+0000 7fadce200700 1 -- 172.21.15.1:0/1357109311 shutdown_connections 2024-02-02T15:27:54.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.369+0000 7fadce200700 1 --2- 172.21.15.1:0/1357109311 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fada80a0720 0x7fada80a2be0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:54.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.369+0000 7fadce200700 1 --2- 172.21.15.1:0/1357109311 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fadc81059e0 0x7fadc819d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:54.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.369+0000 7fadce200700 1 --2- 172.21.15.1:0/1357109311 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fadc80ff9d0 0x7fadc8198de0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:54.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.369+0000 7fadce200700 1 --2- 172.21.15.1:0/1357109311 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fadc8100390 0x7fadc8199320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:54.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.369+0000 7fadce200700 1 -- 172.21.15.1:0/1357109311 >> 172.21.15.1:0/1357109311 conn(0x7fadc80fb4e0 msgr2=0x7fadc810b120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:54.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.369+0000 7fadce200700 1 -- 172.21.15.1:0/1357109311 shutdown_connections 2024-02-02T15:27:54.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.369+0000 7fadce200700 1 -- 172.21.15.1:0/1357109311 wait complete. 2024-02-02T15:27:54.391 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-02T15:27:54.391 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T15:27:54.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.613+0000 7f54f8c00700 1 -- 172.21.15.1:0/2612143625 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f54f40689d0 msgr2=0x7f54f410d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:54.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.613+0000 7f54f8c00700 1 --2- 172.21.15.1:0/2612143625 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f54f40689d0 0x7f54f410d970 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f54ec009f90 tx=0x7f54ec00b330 comp rx=0 tx=0).stop 2024-02-02T15:27:54.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.613+0000 7f54f8c00700 1 -- 172.21.15.1:0/2612143625 shutdown_connections 2024-02-02T15:27:54.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.613+0000 7f54f8c00700 1 --2- 172.21.15.1:0/2612143625 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f54f40689d0 0x7f54f410d970 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:54.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.613+0000 7f54f8c00700 1 --2- 172.21.15.1:0/2612143625 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54f4106ab0 0x7f54f4106e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:54.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.613+0000 7f54f8c00700 1 --2- 172.21.15.1:0/2612143625 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54f4068010 0x7f54f4068490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:54.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.613+0000 7f54f8c00700 1 -- 172.21.15.1:0/2612143625 >> 172.21.15.1:0/2612143625 conn(0x7f54f4076730 msgr2=0x7f54f4076b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:54.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.613+0000 7f54f8c00700 1 -- 172.21.15.1:0/2612143625 shutdown_connections 2024-02-02T15:27:54.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.613+0000 7f54f8c00700 1 -- 172.21.15.1:0/2612143625 wait complete. 2024-02-02T15:27:54.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.615+0000 7f54f8c00700 1 Processor -- start 2024-02-02T15:27:54.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.615+0000 7f54f8c00700 1 -- start start 2024-02-02T15:27:54.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.615+0000 7f54f8c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f54f4068010 0x7f54f40ffc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:54.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.615+0000 7f54f8c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54f40689d0 0x7f54f41001a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:54.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.615+0000 7f54f8c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54f4106ab0 0x7f54f41a4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:54.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.615+0000 7f54f8c00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54f41009d0 con 0x7f54f40689d0 2024-02-02T15:27:54.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.615+0000 7f54f8c00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54f41a5220 con 0x7f54f4106ab0 2024-02-02T15:27:54.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.615+0000 7f54f8c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54f41a5360 con 0x7f54f4068010 2024-02-02T15:27:54.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.616+0000 7f54f3400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f54f4068010 0x7f54f40ffc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:54.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.616+0000 7f54f3400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f54f4068010 0x7f54f40ffc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:60756/0 (socket says 172.21.15.1:60756) 2024-02-02T15:27:54.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.616+0000 7f54f3400700 1 -- 172.21.15.1:0/2407352987 learned_addr learned my addr 172.21.15.1:0/2407352987 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:54.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.616+0000 7f54f3e00700 1 --2- 172.21.15.1:0/2407352987 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54f4106ab0 0x7f54f41a4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:54.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.616+0000 7f54f2a00700 1 --2- 172.21.15.1:0/2407352987 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54f40689d0 0x7f54f41001a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:54.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.616+0000 7f54f3e00700 1 -- 172.21.15.1:0/2407352987 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f54f4068010 msgr2=0x7f54f40ffc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:54.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.616+0000 7f54f3e00700 1 --2- 172.21.15.1:0/2407352987 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f54f4068010 0x7f54f40ffc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:54.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.616+0000 7f54f3e00700 1 -- 172.21.15.1:0/2407352987 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54f40689d0 msgr2=0x7f54f41001a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:54.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.616+0000 7f54f3e00700 1 --2- 172.21.15.1:0/2407352987 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54f40689d0 0x7f54f41001a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:54.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.616+0000 7f54f3e00700 1 -- 172.21.15.1:0/2407352987 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54ec009c40 con 0x7f54f4106ab0 2024-02-02T15:27:54.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.616+0000 7f54f2a00700 1 --2- 172.21.15.1:0/2407352987 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54f40689d0 0x7f54f41001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:27:54.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.617+0000 7f54f3e00700 1 --2- 172.21.15.1:0/2407352987 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54f4106ab0 0x7f54f41a4ce0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f54ec00b6c0 tx=0x7f54ec009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:54.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.617+0000 7f54ebe00700 1 -- 172.21.15.1:0/2407352987 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54ec00e040 con 0x7f54f4106ab0 2024-02-02T15:27:54.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.617+0000 7f54ebe00700 1 -- 172.21.15.1:0/2407352987 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f54ec004e70 con 0x7f54f4106ab0 2024-02-02T15:27:54.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.617+0000 7f54ebe00700 1 -- 172.21.15.1:0/2407352987 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54ec005040 con 0x7f54f4106ab0 2024-02-02T15:27:54.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.617+0000 7f54f8c00700 1 -- 172.21.15.1:0/2407352987 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54f41a5560 con 0x7f54f4106ab0 2024-02-02T15:27:54.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.617+0000 7f54f8c00700 1 -- 172.21.15.1:0/2407352987 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54f41a5990 con 0x7f54f4106ab0 2024-02-02T15:27:54.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.618+0000 7f54e9600700 1 -- 172.21.15.1:0/2407352987 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54c40052e0 con 0x7f54f4106ab0 2024-02-02T15:27:54.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.622+0000 7f54ebe00700 1 -- 172.21.15.1:0/2407352987 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 25) v1 ==== 135245+0+0 (secure 0 0 0) 0x7f54ec01d470 con 0x7f54f4106ab0 2024-02-02T15:27:54.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.623+0000 7f54ebe00700 1 --2- 172.21.15.1:0/2407352987 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f54d40a0720 0x7f54d40a2be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:54.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.623+0000 7f54ebe00700 1 -- 172.21.15.1:0/2407352987 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f54ec0cd9a0 con 0x7f54f4106ab0 2024-02-02T15:27:54.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.623+0000 7f54f3400700 1 --2- 172.21.15.1:0/2407352987 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f54d40a0720 0x7f54d40a2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:54.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.624+0000 7f54f3400700 1 --2- 172.21.15.1:0/2407352987 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f54d40a0720 0x7f54d40a2be0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f54e4005950 tx=0x7f54e4009310 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:54.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.624+0000 7f54ebe00700 1 -- 172.21.15.1:0/2407352987 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f54ec096dd0 con 0x7f54f4106ab0 2024-02-02T15:27:54.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.941+0000 7f54e9600700 1 -- 172.21.15.1:0/2407352987 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f54c4005150 con 0x7f54f4106ab0 2024-02-02T15:27:54.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.941+0000 7f54ebe00700 1 -- 172.21.15.1:0/2407352987 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10866 (secure 0 0 0) 0x7f54ec09a3f0 con 0x7f54f4106ab0 2024-02-02T15:27:54.941 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:27:54.942 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2922240448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2922240448},{"type":"v1","addr":"172.21.15.1:6835","nonce":2922240448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2729497431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2729497431},{"type":"v1","addr":"172.21.15.1:6837","nonce":2729497431}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-02T15:24:47.761435+0000","modified":"2024-02-02T15:25:20.251601+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24529,"mds_2":24518,"mds_3":24514,"mds_4":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14517":{"gid":14517,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/1218303683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":1218303683},{"type":"v1","addr":"172.21.15.1:6839","nonce":1218303683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14529":{"gid":14529,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/2059197496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2059197496},{"type":"v1","addr":"172.21.15.1:6841","nonce":2059197496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/7648986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":7648986},{"type":"v1","addr":"172.21.15.142:6833","nonce":7648986}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/3482746094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3482746094},{"type":"v1","addr":"172.21.15.71:6833","nonce":3482746094}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/388207814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":388207814},{"type":"v1","addr":"172.21.15.71:6835","nonce":388207814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6835/1052606547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1052606547},{"type":"v1","addr":"172.21.15.142:6835","nonce":1052606547}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.142:6837/2870289019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2870289019},{"type":"v1","addr":"172.21.15.142:6837","nonce":2870289019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.71:6837/628016199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":628016199},{"type":"v1","addr":"172.21.15.71:6837","nonce":628016199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.142:6839/321675421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":321675421},{"type":"v1","addr":"172.21.15.142:6839","nonce":321675421}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/1268086493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1268086493},{"type":"v1","addr":"172.21.15.71:6839","nonce":1268086493}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-02T15:27:54.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.944+0000 7f54e9600700 1 -- 172.21.15.1:0/2407352987 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f54d40a0720 msgr2=0x7f54d40a2be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:54.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.944+0000 7f54e9600700 1 --2- 172.21.15.1:0/2407352987 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f54d40a0720 0x7f54d40a2be0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f54e4005950 tx=0x7f54e4009310 comp rx=0 tx=0).stop 2024-02-02T15:27:54.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.944+0000 7f54e9600700 1 -- 172.21.15.1:0/2407352987 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54f4106ab0 msgr2=0x7f54f41a4ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:54.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.944+0000 7f54e9600700 1 --2- 172.21.15.1:0/2407352987 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54f4106ab0 0x7f54f41a4ce0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f54ec00b6c0 tx=0x7f54ec009500 comp rx=0 tx=0).stop 2024-02-02T15:27:54.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.944+0000 7f54e9600700 1 -- 172.21.15.1:0/2407352987 shutdown_connections 2024-02-02T15:27:54.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.944+0000 7f54e9600700 1 --2- 172.21.15.1:0/2407352987 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f54d40a0720 0x7f54d40a2be0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:54.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.944+0000 7f54e9600700 1 --2- 172.21.15.1:0/2407352987 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54f4106ab0 0x7f54f41a4ce0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:54.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.944+0000 7f54e9600700 1 --2- 172.21.15.1:0/2407352987 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f54f4068010 0x7f54f40ffc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:54.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.944+0000 7f54e9600700 1 --2- 172.21.15.1:0/2407352987 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54f40689d0 0x7f54f41001a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:54.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.944+0000 7f54e9600700 1 -- 172.21.15.1:0/2407352987 >> 172.21.15.1:0/2407352987 conn(0x7f54f4076730 msgr2=0x7f54f40fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:54.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.944+0000 7f54e9600700 1 -- 172.21.15.1:0/2407352987 shutdown_connections 2024-02-02T15:27:54.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:54.944+0000 7f54e9600700 1 -- 172.21.15.1:0/2407352987 wait complete. 2024-02-02T15:27:54.945 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 27 2024-02-02T15:27:54.963 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-02-02T15:27:55.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.189+0000 7f49cde00700 1 -- 172.21.15.1:0/1800275493 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49c8068010 msgr2=0x7f49c8068490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:55.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.189+0000 7f49cde00700 1 --2- 172.21.15.1:0/1800275493 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49c8068010 0x7f49c8068490 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f49b0009990 tx=0x7f49b0009ca0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.189+0000 7f49cde00700 1 -- 172.21.15.1:0/1800275493 shutdown_connections 2024-02-02T15:27:55.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.189+0000 7f49cde00700 1 --2- 172.21.15.1:0/1800275493 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49c80689d0 0x7f49c810d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.189+0000 7f49cde00700 1 --2- 172.21.15.1:0/1800275493 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f49c8106ab0 0x7f49c8106e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.189+0000 7f49cde00700 1 --2- 172.21.15.1:0/1800275493 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49c8068010 0x7f49c8068490 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.189+0000 7f49cde00700 1 -- 172.21.15.1:0/1800275493 >> 172.21.15.1:0/1800275493 conn(0x7f49c8076770 msgr2=0x7f49c8076b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:55.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.189+0000 7f49cde00700 1 -- 172.21.15.1:0/1800275493 shutdown_connections 2024-02-02T15:27:55.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.189+0000 7f49cde00700 1 -- 172.21.15.1:0/1800275493 wait complete. 2024-02-02T15:27:55.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.190+0000 7f49cde00700 1 Processor -- start 2024-02-02T15:27:55.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.191+0000 7f49cde00700 1 -- start start 2024-02-02T15:27:55.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.191+0000 7f49cde00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f49c8068010 0x7f49c8198ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:55.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.191+0000 7f49cde00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49c80689d0 0x7f49c8199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:55.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.191+0000 7f49cde00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49c8106ab0 0x7f49c819c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:55.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.191+0000 7f49cde00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c819cbd0 con 0x7f49c8068010 2024-02-02T15:27:55.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.191+0000 7f49cde00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c819cd40 con 0x7f49c8106ab0 2024-02-02T15:27:55.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.191+0000 7f49cde00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c819cf00 con 0x7f49c80689d0 2024-02-02T15:27:55.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.191+0000 7f49cca00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f49c8068010 0x7f49c8198ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:55.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.192+0000 7f49cca00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f49c8068010 0x7f49c8198ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33112/0 (socket says 172.21.15.1:33112) 2024-02-02T15:27:55.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.192+0000 7f49cca00700 1 -- 172.21.15.1:0/3519403655 learned_addr learned my addr 172.21.15.1:0/3519403655 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:55.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.192+0000 7f49c7e00700 1 --2- 172.21.15.1:0/3519403655 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49c80689d0 0x7f49c8199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:55.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.192+0000 7f49cd400700 1 --2- 172.21.15.1:0/3519403655 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49c8106ab0 0x7f49c819c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:55.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.192+0000 7f49c7e00700 1 -- 172.21.15.1:0/3519403655 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49c8106ab0 msgr2=0x7f49c819c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:55.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.192+0000 7f49c7e00700 1 --2- 172.21.15.1:0/3519403655 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49c8106ab0 0x7f49c819c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.192+0000 7f49c7e00700 1 -- 172.21.15.1:0/3519403655 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f49c8068010 msgr2=0x7f49c8198ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:55.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.192+0000 7f49c7e00700 1 --2- 172.21.15.1:0/3519403655 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f49c8068010 0x7f49c8198ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.192+0000 7f49c7e00700 1 -- 172.21.15.1:0/3519403655 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49b00095f0 con 0x7f49c80689d0 2024-02-02T15:27:55.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.192+0000 7f49c7e00700 1 --2- 172.21.15.1:0/3519403655 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49c80689d0 0x7f49c8199220 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f49b00059d0 tx=0x7f49b0005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:55.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.193+0000 7f49c5600700 1 -- 172.21.15.1:0/3519403655 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b001d070 con 0x7f49c80689d0 2024-02-02T15:27:55.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.193+0000 7f49c5600700 1 -- 172.21.15.1:0/3519403655 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f49b0022470 con 0x7f49c80689d0 2024-02-02T15:27:55.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.193+0000 7f49cde00700 1 -- 172.21.15.1:0/3519403655 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49c819d180 con 0x7f49c80689d0 2024-02-02T15:27:55.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.193+0000 7f49c5600700 1 -- 172.21.15.1:0/3519403655 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b0017650 con 0x7f49c80689d0 2024-02-02T15:27:55.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.193+0000 7f49cde00700 1 -- 172.21.15.1:0/3519403655 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49c81a5b40 con 0x7f49c80689d0 2024-02-02T15:27:55.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.197+0000 7f49c5600700 1 -- 172.21.15.1:0/3519403655 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) v1 ==== 135245+0+0 (secure 0 0 0) 0x7f49b0022890 con 0x7f49c80689d0 2024-02-02T15:27:55.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.199+0000 7f49e300fb80 1 Processor -- start 2024-02-02T15:27:55.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.199+0000 7f49e300fb80 1 -- start start 2024-02-02T15:27:55.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.199+0000 7f49c5600700 1 --2- 172.21.15.1:0/3519403655 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f49a80a8fb0 0x7f49a80ab470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:55.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.200+0000 7f49c5600700 1 -- 172.21.15.1:0/3519403655 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f49b00cd680 con 0x7f49c80689d0 2024-02-02T15:27:55.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.200+0000 7f49e300fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5631a6e641f0 0x5631a6e645d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:55.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.200+0000 7f49e300fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5631a6e64b10 0x5631a6e6f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:55.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.200+0000 7f49e300fb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5631a6e6f9f0 0x5631a6e71de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:55.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.200+0000 7f49e300fb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631a6dd84c0 con 0x5631a6e6f9f0 2024-02-02T15:27:55.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.200+0000 7f49e300fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631a6dbcaa0 con 0x5631a6e64b10 2024-02-02T15:27:55.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.200+0000 7f49e300fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631a6dba460 con 0x5631a6e641f0 2024-02-02T15:27:55.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.200+0000 7f49c7e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5631a6e64b10 0x5631a6e6f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:55.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.200+0000 7f49c7e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5631a6e64b10 0x5631a6e6f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:36864/0 (socket says 172.21.15.1:36864) 2024-02-02T15:27:55.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.200+0000 7f49c7e00700 1 -- 172.21.15.1:0/3684164243 learned_addr learned my addr 172.21.15.1:0/3684164243 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:55.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.200+0000 7f49cca00700 1 --2- 172.21.15.1:0/3684164243 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5631a6e641f0 0x5631a6e645d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:55.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.200+0000 7f49cd400700 1 --2- 172.21.15.1:0/3684164243 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5631a6e6f9f0 0x5631a6e71de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:55.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.200+0000 7f49c7e00700 1 -- 172.21.15.1:0/3684164243 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5631a6e641f0 msgr2=0x5631a6e645d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:55.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.200+0000 7f49c7e00700 1 --2- 172.21.15.1:0/3684164243 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5631a6e641f0 0x5631a6e645d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.200+0000 7f49c7e00700 1 -- 172.21.15.1:0/3684164243 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5631a6e6f9f0 msgr2=0x5631a6e71de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:55.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.200+0000 7f49c7e00700 1 --2- 172.21.15.1:0/3684164243 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5631a6e6f9f0 0x5631a6e71de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.200+0000 7f49c7e00700 1 -- 172.21.15.1:0/3684164243 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5631a6c373b0 con 0x5631a6e64b10 2024-02-02T15:27:55.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.200+0000 7f49cd400700 1 --2- 172.21.15.1:0/3684164243 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5631a6e6f9f0 0x5631a6e71de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:27:55.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.200+0000 7f49cd400700 1 --2- 172.21.15.1:0/3519403655 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f49a80a8fb0 0x7f49a80ab470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:55.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.201+0000 7f49c7e00700 1 --2- 172.21.15.1:0/3684164243 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5631a6e64b10 0x5631a6e6f4b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f49b0000c00 tx=0x7f49b009a4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=374cf2aedc6e6f9c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:55.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.201+0000 7f49bdc00700 1 -- 172.21.15.1:0/3684164243 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b0011110 con 0x5631a6e64b10 2024-02-02T15:27:55.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.201+0000 7f49bdc00700 1 -- 172.21.15.1:0/3684164243 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f49b00af070 con 0x5631a6e64b10 2024-02-02T15:27:55.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.201+0000 7f49bdc00700 1 -- 172.21.15.1:0/3684164243 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b00cd290 con 0x5631a6e64b10 2024-02-02T15:27:55.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.202+0000 7f49cd400700 1 --2- 172.21.15.1:0/3519403655 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f49a80a8fb0 0x7f49a80ab470 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f49c0009fa0 tx=0x7f49c0009050 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:55.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.202+0000 7f49e300fb80 1 -- 172.21.15.1:0/3684164243 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5631a6e64b10 msgr2=0x5631a6e6f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:55.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.202+0000 7f49e300fb80 1 --2- 172.21.15.1:0/3684164243 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5631a6e64b10 0x5631a6e6f4b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f49b0000c00 tx=0x7f49b009a4e0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.202+0000 7f49e300fb80 1 -- 172.21.15.1:0/3684164243 shutdown_connections 2024-02-02T15:27:55.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.202+0000 7f49e300fb80 1 --2- 172.21.15.1:0/3684164243 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5631a6e6f9f0 0x5631a6e71de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.202+0000 7f49e300fb80 1 --2- 172.21.15.1:0/3684164243 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5631a6e641f0 0x5631a6e645d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.202+0000 7f49e300fb80 1 --2- 172.21.15.1:0/3684164243 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5631a6e64b10 0x5631a6e6f4b0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.202+0000 7f49e300fb80 1 -- 172.21.15.1:0/3684164243 >> 172.21.15.1:0/3684164243 conn(0x5631a6dc3db0 msgr2=0x5631a6e605c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:55.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.202+0000 7f49e300fb80 1 -- 172.21.15.1:0/3684164243 shutdown_connections 2024-02-02T15:27:55.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.202+0000 7f49e300fb80 1 -- 172.21.15.1:0/3684164243 wait complete. 2024-02-02T15:27:55.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.203+0000 7f49e300fb80 1 -- start start 2024-02-02T15:27:55.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.203+0000 7f49e300fb80 1 Processor -- start 2024-02-02T15:27:55.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.204+0000 7f49e300fb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5631a6e641f0 0x5631a6e6fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:55.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.204+0000 7f49e300fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5631a6e64b10 0x5631a6e70310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:55.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.204+0000 7f49e300fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5631a6e6f9f0 0x5631a6e70850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:55.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.204+0000 7f49e300fb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631a6db5d50 con 0x5631a6e641f0 2024-02-02T15:27:55.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.204+0000 7f49e300fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631a6d51d80 con 0x5631a6e6f9f0 2024-02-02T15:27:55.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.204+0000 7f49e300fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631a6d51a20 con 0x5631a6e64b10 2024-02-02T15:27:55.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.204+0000 7f49e300fb80 10 client.0 _ll_register_callbacks cb 0x5631a6d53290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:27:55.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.204+0000 7f49c7e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5631a6e64b10 0x5631a6e70310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:55.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.204+0000 7f49c7e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5631a6e64b10 0x5631a6e70310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:60796/0 (socket says 172.21.15.1:60796) 2024-02-02T15:27:55.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.204+0000 7f49c7e00700 1 -- 172.21.15.1:0/1111013248 learned_addr learned my addr 172.21.15.1:0/1111013248 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:55.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.204+0000 7f49cd400700 1 --2- 172.21.15.1:0/1111013248 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5631a6e6f9f0 0x5631a6e70850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:55.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.204+0000 7f49cca00700 1 --2- 172.21.15.1:0/1111013248 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5631a6e641f0 0x5631a6e6fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:55.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.205+0000 7f49c7e00700 1 -- 172.21.15.1:0/1111013248 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5631a6e6f9f0 msgr2=0x5631a6e70850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:55.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.205+0000 7f49c7e00700 1 --2- 172.21.15.1:0/1111013248 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5631a6e6f9f0 0x5631a6e70850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.205+0000 7f49c7e00700 1 -- 172.21.15.1:0/1111013248 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5631a6e641f0 msgr2=0x5631a6e6fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:55.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.205+0000 7f49c7e00700 1 --2- 172.21.15.1:0/1111013248 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5631a6e641f0 0x5631a6e6fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.205+0000 7f49c7e00700 1 -- 172.21.15.1:0/1111013248 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49c81a5b40 con 0x5631a6e64b10 2024-02-02T15:27:55.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.205+0000 7f49cca00700 1 --2- 172.21.15.1:0/1111013248 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5631a6e641f0 0x5631a6e6fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:27:55.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.205+0000 7f49c7e00700 1 --2- 172.21.15.1:0/1111013248 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5631a6e64b10 0x5631a6e70310 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f49b0006010 tx=0x7f49b0009540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f70344b0131bba62 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:55.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.205+0000 7f49b6000700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:27:55.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.206+0000 7f49b6000700 1 -- 172.21.15.1:0/1111013248 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b009e540 con 0x5631a6e64b10 2024-02-02T15:27:55.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.206+0000 7f49b6000700 1 -- 172.21.15.1:0/1111013248 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f49b009ae10 con 0x5631a6e64b10 2024-02-02T15:27:55.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.206+0000 7f49e300fb80 1 -- 172.21.15.1:0/1111013248 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5631a6e58340 con 0x5631a6e64b10 2024-02-02T15:27:55.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.206+0000 7f49b6000700 1 -- 172.21.15.1:0/1111013248 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f49b00a26f0 con 0x5631a6e64b10 2024-02-02T15:27:55.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.206+0000 7f49e300fb80 10 client.25858 fetch_fsmap learned FSMap version 27 2024-02-02T15:27:55.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.206+0000 7f49e300fb80 1 -- 172.21.15.1:0/1111013248 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5631a6be3b60 con 0x5631a6e64b10 2024-02-02T15:27:55.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.206+0000 7f49b6000700 1 -- 172.21.15.1:0/1111013248 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5631a6e58340 con 0x5631a6e64b10 2024-02-02T15:27:55.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.206+0000 7f49b6000700 1 -- 172.21.15.1:0/1111013248 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f49b00a2260 con 0x5631a6e64b10 2024-02-02T15:27:55.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.206+0000 7f49e300fb80 10 client.25858 fetch_fsmap finished waiting for FSMap version 27 2024-02-02T15:27:55.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.206+0000 7f49e300fb80 10 client.25858 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-02-02T15:27:55.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.206+0000 7f49e300fb80 20 client.25858 populate_metadata read hostname 'smithi001' 2024-02-02T15:27:55.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.206+0000 7f49e300fb80 1 --2- 172.21.15.1:0/1111013248 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x5631a6e7ea40 0x5631a6e80e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:55.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.206+0000 7f49e300fb80 4 client.25858 mds_command: new command op to 24502 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:27:55.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.207+0000 7f49e300fb80 1 -- 172.21.15.1:0/1111013248 --> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5631a6daedd0 con 0x5631a6e7ea40 2024-02-02T15:27:55.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.207+0000 7f49cca00700 1 --2- 172.21.15.1:0/1111013248 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x5631a6e7ea40 0x5631a6e80e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:55.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.208+0000 7f49b6000700 10 client.25858 ms_handle_connect on v2:172.21.15.142:6834/1052606547 2024-02-02T15:27:55.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.208+0000 7f49cca00700 1 --2- 172.21.15.1:0/1111013248 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x5631a6e7ea40 0x5631a6e80e30 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bb427745a4421982 server_cookie=7316481c2b58ea63 in_seq=0 out_seq=0 2024-02-02T15:27:55.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.208+0000 7f49b6000700 1 -- 172.21.15.1:0/1111013248 <== mds.4 v2:172.21.15.142:6834/1052606547 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5631a6daedd0 con 0x5631a6e7ea40 2024-02-02T15:27:55.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.208+0000 7f49b6000700 10 client.25858 handle_command_reply: tid=0 2024-02-02T15:27:55.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.208+0000 7f49e300fb80 1 client.25858 shutdown 2024-02-02T15:27:55.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.209+0000 7f49e300fb80 1 -- 172.21.15.1:0/1111013248 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5631a6e64b10 msgr2=0x5631a6e70310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:55.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.209+0000 7f49e300fb80 1 --2- 172.21.15.1:0/1111013248 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5631a6e64b10 0x5631a6e70310 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f49b0006010 tx=0x7f49b0009540 comp rx=0 tx=0).stop 2024-02-02T15:27:55.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.209+0000 7f49e300fb80 1 -- 172.21.15.1:0/1111013248 shutdown_connections 2024-02-02T15:27:55.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.209+0000 7f49e300fb80 1 --2- 172.21.15.1:0/1111013248 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x5631a6e7ea40 0x5631a6e80e30 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.209+0000 7f49e300fb80 1 --2- 172.21.15.1:0/1111013248 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5631a6e6f9f0 0x5631a6e70850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.209+0000 7f49e300fb80 1 --2- 172.21.15.1:0/1111013248 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5631a6e641f0 0x5631a6e6fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.209+0000 7f49b6000700 0 client.25858 ms_handle_reset on v2:172.21.15.142:6834/1052606547 2024-02-02T15:27:55.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.209+0000 7f49e300fb80 1 --2- 172.21.15.1:0/1111013248 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5631a6e64b10 0x5631a6e70310 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.209+0000 7f49e300fb80 1 -- 172.21.15.1:0/1111013248 >> 172.21.15.1:0/1111013248 conn(0x5631a6dc3db0 msgr2=0x5631a6e601b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:55.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.209+0000 7f49e300fb80 1 -- 172.21.15.1:0/1111013248 shutdown_connections 2024-02-02T15:27:55.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.209+0000 7f49e300fb80 1 -- 172.21.15.1:0/1111013248 wait complete. 2024-02-02T15:27:55.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.209+0000 7f49e300fb80 20 client.25858 trim_cache size 0 max 16384 2024-02-02T15:27:55.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.227+0000 7f49e300fb80 1 Processor -- start 2024-02-02T15:27:55.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.227+0000 7f49e300fb80 1 -- start start 2024-02-02T15:27:55.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.227+0000 7f49e300fb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5631a6dc3db0 0x5631a6e7a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:55.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.227+0000 7f49e300fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5631a6e641f0 0x5631a6e7a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:55.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.227+0000 7f49e300fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5631a6e64b10 0x5631a6e6bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:55.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.227+0000 7f49e300fb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631a6c396a0 con 0x5631a6dc3db0 2024-02-02T15:27:55.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.227+0000 7f49e300fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631a6e61270 con 0x5631a6e641f0 2024-02-02T15:27:55.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.227+0000 7f49e300fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631a6e5a750 con 0x5631a6e64b10 2024-02-02T15:27:55.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.228+0000 7f49cca00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5631a6dc3db0 0x5631a6e7a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:55.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.228+0000 7f49cca00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5631a6dc3db0 0x5631a6e7a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33130/0 (socket says 172.21.15.1:33130) 2024-02-02T15:27:55.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.228+0000 7f49cca00700 1 -- 172.21.15.1:0/3568982818 learned_addr learned my addr 172.21.15.1:0/3568982818 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:55.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.228+0000 7f49cd400700 1 --2- 172.21.15.1:0/3568982818 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5631a6e64b10 0x5631a6e6bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:55.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.228+0000 7f49c7e00700 1 --2- 172.21.15.1:0/3568982818 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5631a6e641f0 0x5631a6e7a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:55.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.228+0000 7f49cca00700 1 -- 172.21.15.1:0/3568982818 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5631a6e64b10 msgr2=0x5631a6e6bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:55.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.228+0000 7f49cca00700 1 --2- 172.21.15.1:0/3568982818 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5631a6e64b10 0x5631a6e6bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.228+0000 7f49cca00700 1 -- 172.21.15.1:0/3568982818 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5631a6e641f0 msgr2=0x5631a6e7a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:55.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.228+0000 7f49cca00700 1 --2- 172.21.15.1:0/3568982818 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5631a6e641f0 0x5631a6e7a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.228+0000 7f49cca00700 1 -- 172.21.15.1:0/3568982818 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5631a6d7bea0 con 0x5631a6dc3db0 2024-02-02T15:27:55.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.229+0000 7f49cca00700 1 --2- 172.21.15.1:0/3568982818 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5631a6dc3db0 0x5631a6e7a3c0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f49b8009a30 tx=0x7f49b8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d4ac79cbbc36a15 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:55.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.229+0000 7f49be600700 1 -- 172.21.15.1:0/3568982818 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b8018040 con 0x5631a6dc3db0 2024-02-02T15:27:55.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.229+0000 7f49be600700 1 -- 172.21.15.1:0/3568982818 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f49b801a430 con 0x5631a6dc3db0 2024-02-02T15:27:55.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.229+0000 7f49be600700 1 -- 172.21.15.1:0/3568982818 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b8012450 con 0x5631a6dc3db0 2024-02-02T15:27:55.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.230+0000 7f49e300fb80 1 -- 172.21.15.1:0/3568982818 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5631a6dc3db0 msgr2=0x5631a6e7a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:55.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.230+0000 7f49e300fb80 1 --2- 172.21.15.1:0/3568982818 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5631a6dc3db0 0x5631a6e7a3c0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f49b8009a30 tx=0x7f49b8009260 comp rx=0 tx=0).stop 2024-02-02T15:27:55.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.230+0000 7f49e300fb80 1 -- 172.21.15.1:0/3568982818 shutdown_connections 2024-02-02T15:27:55.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.230+0000 7f49e300fb80 1 --2- 172.21.15.1:0/3568982818 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5631a6e64b10 0x5631a6e6bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.230+0000 7f49e300fb80 1 --2- 172.21.15.1:0/3568982818 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5631a6dc3db0 0x5631a6e7a3c0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.230+0000 7f49e300fb80 1 --2- 172.21.15.1:0/3568982818 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5631a6e641f0 0x5631a6e7a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.230+0000 7f49e300fb80 1 -- 172.21.15.1:0/3568982818 >> 172.21.15.1:0/3568982818 conn(0x5631a6e6f360 msgr2=0x5631a6e69270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:55.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.230+0000 7f49e300fb80 1 -- 172.21.15.1:0/3568982818 shutdown_connections 2024-02-02T15:27:55.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.230+0000 7f49e300fb80 1 -- 172.21.15.1:0/3568982818 wait complete. 2024-02-02T15:27:55.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.231+0000 7f49e300fb80 1 -- start start 2024-02-02T15:27:55.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.231+0000 7f49e300fb80 1 Processor -- start 2024-02-02T15:27:55.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.232+0000 7f49e300fb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5631a6dc3db0 0x5631a6e749d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:55.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.232+0000 7f49e300fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5631a6e641f0 0x5631a6e72da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:55.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.232+0000 7f49e300fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5631a6e64b10 0x5631a6e732e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:55.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.232+0000 7f49e300fb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631a6e73820 con 0x5631a6dc3db0 2024-02-02T15:27:55.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.232+0000 7f49e300fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631a6e73990 con 0x5631a6e641f0 2024-02-02T15:27:55.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.232+0000 7f49e300fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631a6e73b50 con 0x5631a6e64b10 2024-02-02T15:27:55.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.232+0000 7f49e300fb80 10 client.0 _ll_register_callbacks cb 0x5631a6d53290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:27:55.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.232+0000 7f49cd400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5631a6e64b10 0x5631a6e732e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:55.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.232+0000 7f49c7e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5631a6e641f0 0x5631a6e72da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:55.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.232+0000 7f49cd400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5631a6e64b10 0x5631a6e732e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:60806/0 (socket says 172.21.15.1:60806) 2024-02-02T15:27:55.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.232+0000 7f49cd400700 1 -- 172.21.15.1:0/1271776301 learned_addr learned my addr 172.21.15.1:0/1271776301 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:55.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.232+0000 7f49cca00700 1 --2- 172.21.15.1:0/1271776301 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5631a6dc3db0 0x5631a6e749d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:55.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.232+0000 7f49cd400700 1 -- 172.21.15.1:0/1271776301 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5631a6e641f0 msgr2=0x5631a6e72da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:55.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.232+0000 7f49cd400700 1 --2- 172.21.15.1:0/1271776301 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5631a6e641f0 0x5631a6e72da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.233+0000 7f49cd400700 1 -- 172.21.15.1:0/1271776301 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5631a6dc3db0 msgr2=0x5631a6e749d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:55.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.233+0000 7f49cd400700 1 --2- 172.21.15.1:0/1271776301 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5631a6dc3db0 0x5631a6e749d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.233+0000 7f49cd400700 1 -- 172.21.15.1:0/1271776301 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5631a6c3bcd0 con 0x5631a6e64b10 2024-02-02T15:27:55.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.233+0000 7f49cd400700 1 --2- 172.21.15.1:0/1271776301 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5631a6e64b10 0x5631a6e732e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f49c0017c00 tx=0x7f49c0004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad6c4adf0ba0808a server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:55.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.233+0000 7f49b6000700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:27:55.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.233+0000 7f49b6000700 1 -- 172.21.15.1:0/1271776301 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c0015980 con 0x5631a6e64b10 2024-02-02T15:27:55.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.233+0000 7f49e300fb80 1 -- 172.21.15.1:0/1271776301 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5631a6d74f50 con 0x5631a6e64b10 2024-02-02T15:27:55.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.233+0000 7f49b6000700 1 -- 172.21.15.1:0/1271776301 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f49c0015ae0 con 0x5631a6e64b10 2024-02-02T15:27:55.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.233+0000 7f49b6000700 1 -- 172.21.15.1:0/1271776301 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f49c0015ca0 con 0x5631a6e64b10 2024-02-02T15:27:55.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.234+0000 7f49e300fb80 10 client.25864 fetch_fsmap learned FSMap version 27 2024-02-02T15:27:55.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.234+0000 7f49e300fb80 1 -- 172.21.15.1:0/1271776301 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x5631a6be2030 con 0x5631a6e64b10 2024-02-02T15:27:55.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.234+0000 7f49b6000700 1 -- 172.21.15.1:0/1271776301 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5631a6d74f50 con 0x5631a6e64b10 2024-02-02T15:27:55.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.234+0000 7f49b6000700 1 -- 172.21.15.1:0/1271776301 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f49c00243d0 con 0x5631a6e64b10 2024-02-02T15:27:55.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.234+0000 7f49e300fb80 10 client.25864 fetch_fsmap finished waiting for FSMap version 27 2024-02-02T15:27:55.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.234+0000 7f49e300fb80 10 client.25864 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-02-02T15:27:55.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.234+0000 7f49e300fb80 20 client.25864 populate_metadata read hostname 'smithi001' 2024-02-02T15:27:55.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.234+0000 7f49e300fb80 1 --2- 172.21.15.1:0/1271776301 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x5631a6e6f9f0 0x5631a6ed39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:55.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.234+0000 7f49e300fb80 4 client.25864 mds_command: new command op to 24502 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:27:55.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.234+0000 7f49e300fb80 1 -- 172.21.15.1:0/1271776301 --> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5631a6c04cf0 con 0x5631a6e6f9f0 2024-02-02T15:27:55.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.234+0000 7f49cca00700 1 --2- 172.21.15.1:0/1271776301 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x5631a6e6f9f0 0x5631a6ed39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:55.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.235+0000 7f49cca00700 1 --2- 172.21.15.1:0/1271776301 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x5631a6e6f9f0 0x5631a6ed39c0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3dfcbff3c14a1b6a server_cookie=9b1afb3124595b74 in_seq=0 out_seq=0 2024-02-02T15:27:55.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.235+0000 7f49b6000700 10 client.25864 ms_handle_connect on v2:172.21.15.142:6834/1052606547 2024-02-02T15:27:55.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.235+0000 7f49b6000700 1 -- 172.21.15.1:0/1271776301 <== mds.4 v2:172.21.15.142:6834/1052606547 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5631a6c04cf0 con 0x5631a6e6f9f0 2024-02-02T15:27:55.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.235+0000 7f49b6000700 10 client.25864 handle_command_reply: tid=0 2024-02-02T15:27:55.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.236+0000 7f49e300fb80 1 client.25864 shutdown 2024-02-02T15:27:55.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.236+0000 7f49e300fb80 1 -- 172.21.15.1:0/1271776301 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5631a6e64b10 msgr2=0x5631a6e732e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:55.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.236+0000 7f49e300fb80 1 --2- 172.21.15.1:0/1271776301 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5631a6e64b10 0x5631a6e732e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f49c0017c00 tx=0x7f49c0004b40 comp rx=0 tx=0).stop 2024-02-02T15:27:55.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.236+0000 7f49e300fb80 1 -- 172.21.15.1:0/1271776301 shutdown_connections 2024-02-02T15:27:55.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.236+0000 7f49e300fb80 1 --2- 172.21.15.1:0/1271776301 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x5631a6e6f9f0 0x5631a6ed39c0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.236+0000 7f49e300fb80 1 --2- 172.21.15.1:0/1271776301 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5631a6e64b10 0x5631a6e732e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.236+0000 7f49e300fb80 1 --2- 172.21.15.1:0/1271776301 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5631a6dc3db0 0x5631a6e749d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.236+0000 7f49e300fb80 1 --2- 172.21.15.1:0/1271776301 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5631a6e641f0 0x5631a6e72da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.236+0000 7f49e300fb80 1 -- 172.21.15.1:0/1271776301 >> 172.21.15.1:0/1271776301 conn(0x5631a6e6f360 msgr2=0x5631a6e6d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:55.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.236+0000 7f49b6000700 0 client.25864 ms_handle_reset on v2:172.21.15.142:6834/1052606547 2024-02-02T15:27:55.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.236+0000 7f49e300fb80 1 -- 172.21.15.1:0/1271776301 shutdown_connections 2024-02-02T15:27:55.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.236+0000 7f49e300fb80 1 -- 172.21.15.1:0/1271776301 wait complete. 2024-02-02T15:27:55.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.236+0000 7f49e300fb80 20 client.25864 trim_cache size 0 max 16384 2024-02-02T15:27:55.239 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:27:55.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.237+0000 7f49cde00700 1 -- 172.21.15.1:0/3519403655 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f49a80a8fb0 msgr2=0x7f49a80ab470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:55.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.237+0000 7f49cde00700 1 --2- 172.21.15.1:0/3519403655 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f49a80a8fb0 0x7f49a80ab470 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f49c0009fa0 tx=0x7f49c0009050 comp rx=0 tx=0).stop 2024-02-02T15:27:55.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.237+0000 7f49cde00700 1 -- 172.21.15.1:0/3519403655 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49c80689d0 msgr2=0x7f49c8199220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:55.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.237+0000 7f49cde00700 1 --2- 172.21.15.1:0/3519403655 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49c80689d0 0x7f49c8199220 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f49b00059d0 tx=0x7f49b0005740 comp rx=0 tx=0).stop 2024-02-02T15:27:55.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.237+0000 7f49cde00700 1 -- 172.21.15.1:0/3519403655 shutdown_connections 2024-02-02T15:27:55.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.237+0000 7f49cde00700 1 --2- 172.21.15.1:0/3519403655 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f49a80a8fb0 0x7f49a80ab470 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.237+0000 7f49cde00700 1 --2- 172.21.15.1:0/3519403655 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f49c8106ab0 0x7f49c819c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.237+0000 7f49cde00700 1 --2- 172.21.15.1:0/3519403655 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f49c8068010 0x7f49c8198ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.237+0000 7f49cde00700 1 --2- 172.21.15.1:0/3519403655 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49c80689d0 0x7f49c8199220 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.237+0000 7f49cde00700 1 -- 172.21.15.1:0/3519403655 >> 172.21.15.1:0/3519403655 conn(0x7f49c8076770 msgr2=0x7f49c810bd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:55.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.237+0000 7f49cde00700 1 -- 172.21.15.1:0/3519403655 shutdown_connections 2024-02-02T15:27:55.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.237+0000 7f49cde00700 1 -- 172.21.15.1:0/3519403655 wait complete. 2024-02-02T15:27:55.260 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-02-02T15:27:55.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.521+0000 7f3352c00700 1 -- 172.21.15.1:0/2170392724 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f334c100a90 msgr2=0x7f334c1048f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:55.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.521+0000 7f3352c00700 1 --2- 172.21.15.1:0/2170392724 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f334c100a90 0x7f334c1048f0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f33340099c0 tx=0x7f3334009cd0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.521+0000 7f3352c00700 1 -- 172.21.15.1:0/2170392724 shutdown_connections 2024-02-02T15:27:55.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.521+0000 7f3352c00700 1 --2- 172.21.15.1:0/2170392724 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f334c105020 0x7f334c10cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.521+0000 7f3352c00700 1 --2- 172.21.15.1:0/2170392724 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f334c1000e0 0x7f334c1004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.521+0000 7f3352c00700 1 --2- 172.21.15.1:0/2170392724 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f334c100a90 0x7f334c1048f0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.521+0000 7f3352c00700 1 -- 172.21.15.1:0/2170392724 >> 172.21.15.1:0/2170392724 conn(0x7f334c0757d0 msgr2=0x7f334c075be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:55.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.522+0000 7f3352c00700 1 -- 172.21.15.1:0/2170392724 shutdown_connections 2024-02-02T15:27:55.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.522+0000 7f3352c00700 1 -- 172.21.15.1:0/2170392724 wait complete. 2024-02-02T15:27:55.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.523+0000 7f3352c00700 1 Processor -- start 2024-02-02T15:27:55.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.523+0000 7f3352c00700 1 -- start start 2024-02-02T15:27:55.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.524+0000 7f3352c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f334c1000e0 0x7f334c101990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:55.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.524+0000 7f3352c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f334c105020 0x7f334c101ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:55.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.524+0000 7f3352c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f334c102410 0x7f334c1a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:55.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.524+0000 7f3352c00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f334c1a5770 con 0x7f334c105020 2024-02-02T15:27:55.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.524+0000 7f3352c00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f334c1a58e0 con 0x7f334c1000e0 2024-02-02T15:27:55.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.524+0000 7f3352c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f334c1a5aa0 con 0x7f334c102410 2024-02-02T15:27:55.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.524+0000 7f334be00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f334c1000e0 0x7f334c101990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:55.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.524+0000 7f334be00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f334c1000e0 0x7f334c101990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:36918/0 (socket says 172.21.15.1:36918) 2024-02-02T15:27:55.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.524+0000 7f334be00700 1 -- 172.21.15.1:0/2664941010 learned_addr learned my addr 172.21.15.1:0/2664941010 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:55.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.524+0000 7f3350c00700 1 --2- 172.21.15.1:0/2664941010 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f334c102410 0x7f334c1a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:55.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.524+0000 7f334b400700 1 --2- 172.21.15.1:0/2664941010 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f334c105020 0x7f334c101ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:55.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.525+0000 7f334be00700 1 -- 172.21.15.1:0/2664941010 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f334c102410 msgr2=0x7f334c1a50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:55.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.525+0000 7f334be00700 1 --2- 172.21.15.1:0/2664941010 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f334c102410 0x7f334c1a50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.525+0000 7f334be00700 1 -- 172.21.15.1:0/2664941010 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f334c105020 msgr2=0x7f334c101ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:55.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.525+0000 7f334be00700 1 --2- 172.21.15.1:0/2664941010 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f334c105020 0x7f334c101ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.525+0000 7f334be00700 1 -- 172.21.15.1:0/2664941010 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33340095f0 con 0x7f334c1000e0 2024-02-02T15:27:55.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.525+0000 7f334be00700 1 --2- 172.21.15.1:0/2664941010 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f334c1000e0 0x7f334c101990 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f333c00e9a0 tx=0x7f333c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:55.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.525+0000 7f3348c00700 1 -- 172.21.15.1:0/2664941010 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f333c009790 con 0x7f334c1000e0 2024-02-02T15:27:55.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.525+0000 7f3348c00700 1 -- 172.21.15.1:0/2664941010 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f333c010430 con 0x7f334c1000e0 2024-02-02T15:27:55.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.525+0000 7f3348c00700 1 -- 172.21.15.1:0/2664941010 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f333c018680 con 0x7f334c1000e0 2024-02-02T15:27:55.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.526+0000 7f3352c00700 1 -- 172.21.15.1:0/2664941010 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f334c1a5d80 con 0x7f334c1000e0 2024-02-02T15:27:55.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.526+0000 7f3352c00700 1 -- 172.21.15.1:0/2664941010 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f334c1a61b0 con 0x7f334c1000e0 2024-02-02T15:27:55.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.530+0000 7f3348c00700 1 -- 172.21.15.1:0/2664941010 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 25) v1 ==== 135245+0+0 (secure 0 0 0) 0x7f333c0105a0 con 0x7f334c1000e0 2024-02-02T15:27:55.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.530+0000 7f33667eab80 1 Processor -- start 2024-02-02T15:27:55.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.530+0000 7f33667eab80 1 -- start start 2024-02-02T15:27:55.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.531+0000 7f33667eab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d15abda010 0x55d15abda3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:55.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.531+0000 7f33667eab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d15abda930 0x55d15abe52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:55.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.531+0000 7f33667eab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d15abe57e0 0x55d15abe7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:55.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.531+0000 7f33667eab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d15ab4e4c0 con 0x55d15abda010 2024-02-02T15:27:55.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.531+0000 7f33667eab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d15ab32aa0 con 0x55d15abe57e0 2024-02-02T15:27:55.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.531+0000 7f33667eab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d15ab30460 con 0x55d15abda930 2024-02-02T15:27:55.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.531+0000 7f3350c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d15abda010 0x55d15abda3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:55.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.531+0000 7f3350c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d15abda010 0x55d15abda3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33178/0 (socket says 172.21.15.1:33178) 2024-02-02T15:27:55.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.531+0000 7f3350c00700 1 -- 172.21.15.1:0/3189573259 learned_addr learned my addr 172.21.15.1:0/3189573259 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:55.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.532+0000 7f334be00700 1 --2- 172.21.15.1:0/3189573259 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d15abda930 0x55d15abe52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:55.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.532+0000 7f334b400700 1 --2- 172.21.15.1:0/3189573259 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d15abe57e0 0x55d15abe7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:55.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.532+0000 7f3350c00700 1 -- 172.21.15.1:0/3189573259 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d15abda930 msgr2=0x55d15abe52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:55.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.532+0000 7f3350c00700 1 --2- 172.21.15.1:0/3189573259 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d15abda930 0x55d15abe52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.532+0000 7f3350c00700 1 -- 172.21.15.1:0/3189573259 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d15abe57e0 msgr2=0x55d15abe7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:55.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.532+0000 7f3350c00700 1 --2- 172.21.15.1:0/3189573259 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d15abe57e0 0x55d15abe7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.532+0000 7f3350c00700 1 -- 172.21.15.1:0/3189573259 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d15a9ad3b0 con 0x55d15abda010 2024-02-02T15:27:55.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.532+0000 7f334b400700 1 --2- 172.21.15.1:0/3189573259 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d15abe57e0 0x55d15abe7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:27:55.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.532+0000 7f3348c00700 1 --2- 172.21.15.1:0/2664941010 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f332c0a07a0 0x7f332c0a2c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:55.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.532+0000 7f3350c00700 1 --2- 172.21.15.1:0/3189573259 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d15abda010 0x55d15abda3f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f3344011680 tx=0x7f3344009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee60d2b073d0ff2c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:55.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.532+0000 7f3348c00700 1 -- 172.21.15.1:0/2664941010 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f333c014070 con 0x7f334c1000e0 2024-02-02T15:27:55.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.533+0000 7f3341200700 1 -- 172.21.15.1:0/3189573259 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f334400e040 con 0x55d15abda010 2024-02-02T15:27:55.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.533+0000 7f3341200700 1 -- 172.21.15.1:0/3189573259 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f334400f040 con 0x55d15abda010 2024-02-02T15:27:55.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.533+0000 7f3341200700 1 -- 172.21.15.1:0/3189573259 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33440048b0 con 0x55d15abda010 2024-02-02T15:27:55.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.533+0000 7f334be00700 1 --2- 172.21.15.1:0/2664941010 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f332c0a07a0 0x7f332c0a2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:55.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.534+0000 7f33667eab80 1 -- 172.21.15.1:0/3189573259 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d15abda010 msgr2=0x55d15abda3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:55.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.534+0000 7f33667eab80 1 --2- 172.21.15.1:0/3189573259 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d15abda010 0x55d15abda3f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f3344011680 tx=0x7f3344009920 comp rx=0 tx=0).stop 2024-02-02T15:27:55.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.534+0000 7f334be00700 1 --2- 172.21.15.1:0/2664941010 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f332c0a07a0 0x7f332c0a2c60 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f333c0c9770 tx=0x7f333c0cfc30 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:55.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.534+0000 7f33667eab80 1 -- 172.21.15.1:0/3189573259 shutdown_connections 2024-02-02T15:27:55.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.534+0000 7f33667eab80 1 --2- 172.21.15.1:0/3189573259 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d15abe57e0 0x55d15abe7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.534+0000 7f33667eab80 1 --2- 172.21.15.1:0/3189573259 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d15abda010 0x55d15abda3f0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.534+0000 7f33667eab80 1 --2- 172.21.15.1:0/3189573259 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d15abda930 0x55d15abe52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.534+0000 7f33667eab80 1 -- 172.21.15.1:0/3189573259 >> 172.21.15.1:0/3189573259 conn(0x55d15ab39db0 msgr2=0x55d15abd63b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:55.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.534+0000 7f33667eab80 1 -- 172.21.15.1:0/3189573259 shutdown_connections 2024-02-02T15:27:55.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.534+0000 7f33667eab80 1 -- 172.21.15.1:0/3189573259 wait complete. 2024-02-02T15:27:55.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.535+0000 7f33667eab80 1 -- start start 2024-02-02T15:27:55.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.535+0000 7f33667eab80 1 Processor -- start 2024-02-02T15:27:55.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.536+0000 7f33667eab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d15abda010 0x55d15abe51f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:55.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.536+0000 7f33667eab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d15abda930 0x55d15abe2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:55.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.536+0000 7f33667eab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d15abe57e0 0x55d15abe2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:55.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.536+0000 7f33667eab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d15ab2bd50 con 0x55d15abda010 2024-02-02T15:27:55.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.536+0000 7f33667eab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d15aac7d80 con 0x55d15abe57e0 2024-02-02T15:27:55.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.536+0000 7f33667eab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d15aac7a20 con 0x55d15abda930 2024-02-02T15:27:55.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.536+0000 7f33667eab80 10 client.0 _ll_register_callbacks cb 0x55d15aac9290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:27:55.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.536+0000 7f334b400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d15abda930 0x55d15abe2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:55.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.536+0000 7f334b400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d15abda930 0x55d15abe2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:60856/0 (socket says 172.21.15.1:60856) 2024-02-02T15:27:55.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.536+0000 7f334b400700 1 -- 172.21.15.1:0/3168191106 learned_addr learned my addr 172.21.15.1:0/3168191106 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:55.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.537+0000 7f3350c00700 1 --2- 172.21.15.1:0/3168191106 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d15abda010 0x55d15abe51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:55.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.537+0000 7f334b400700 1 -- 172.21.15.1:0/3168191106 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d15abe57e0 msgr2=0x55d15abe2a30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-02T15:27:55.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.537+0000 7f334b400700 1 --2- 172.21.15.1:0/3168191106 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d15abe57e0 0x55d15abe2a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.537+0000 7f334b400700 1 -- 172.21.15.1:0/3168191106 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d15abda010 msgr2=0x55d15abe51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:55.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.537+0000 7f334b400700 1 --2- 172.21.15.1:0/3168191106 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d15abda010 0x55d15abe51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.537+0000 7f334b400700 1 -- 172.21.15.1:0/3168191106 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3344011330 con 0x55d15abda930 2024-02-02T15:27:55.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.537+0000 7f3350c00700 1 --2- 172.21.15.1:0/3168191106 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d15abda010 0x55d15abe51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:27:55.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.538+0000 7f334b400700 1 --2- 172.21.15.1:0/3168191106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d15abda930 0x55d15abe2400 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f334c073cc0 tx=0x7f3334003ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3746ade8256cfab5 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:55.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.538+0000 7f3339600700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:27:55.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.538+0000 7f3339600700 1 -- 172.21.15.1:0/3168191106 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33340054d0 con 0x55d15abda930 2024-02-02T15:27:55.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.538+0000 7f33667eab80 1 -- 172.21.15.1:0/3168191106 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d15abce0f0 con 0x55d15abda930 2024-02-02T15:27:55.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.538+0000 7f3339600700 1 -- 172.21.15.1:0/3168191106 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f3334005630 con 0x55d15abda930 2024-02-02T15:27:55.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.538+0000 7f3339600700 1 -- 172.21.15.1:0/3168191106 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f333402b7b0 con 0x55d15abda930 2024-02-02T15:27:55.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.538+0000 7f33667eab80 10 client.25876 fetch_fsmap learned FSMap version 27 2024-02-02T15:27:55.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.538+0000 7f33667eab80 1 -- 172.21.15.1:0/3168191106 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55d15a959b60 con 0x55d15abda930 2024-02-02T15:27:55.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.539+0000 7f3339600700 1 -- 172.21.15.1:0/3168191106 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d15abce0f0 con 0x55d15abda930 2024-02-02T15:27:55.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.539+0000 7f3339600700 1 -- 172.21.15.1:0/3168191106 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f333400b7d0 con 0x55d15abda930 2024-02-02T15:27:55.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.539+0000 7f33667eab80 10 client.25876 fetch_fsmap finished waiting for FSMap version 27 2024-02-02T15:27:55.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.539+0000 7f33667eab80 10 client.25876 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-02-02T15:27:55.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.539+0000 7f33667eab80 20 client.25876 populate_metadata read hostname 'smithi001' 2024-02-02T15:27:55.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.539+0000 7f33667eab80 1 --2- 172.21.15.1:0/3168191106 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55d15abf5260 0x55d15abf7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:55.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.539+0000 7f33667eab80 4 client.25876 mds_command: new command op to 24514 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:27:55.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.539+0000 7f33667eab80 1 -- 172.21.15.1:0/3168191106 --> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d15ab24dd0 con 0x55d15abf5260 2024-02-02T15:27:55.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.540+0000 7f3350c00700 1 --2- 172.21.15.1:0/3168191106 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55d15abf5260 0x55d15abf7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:55.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.540+0000 7f3350c00700 1 --2- 172.21.15.1:0/3168191106 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55d15abf5260 0x55d15abf7680 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2e1b0ffb2d00c225 server_cookie=44b906fe8c114c9 in_seq=0 out_seq=0 2024-02-02T15:27:55.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.540+0000 7f3339600700 10 client.25876 ms_handle_connect on v2:172.21.15.142:6836/2870289019 2024-02-02T15:27:55.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.541+0000 7f3339600700 1 -- 172.21.15.1:0/3168191106 <== mds.3 v2:172.21.15.142:6836/2870289019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55d15ab24dd0 con 0x55d15abf5260 2024-02-02T15:27:55.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.541+0000 7f3339600700 10 client.25876 handle_command_reply: tid=0 2024-02-02T15:27:55.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.541+0000 7f33667eab80 1 client.25876 shutdown 2024-02-02T15:27:55.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.542+0000 7f33667eab80 1 -- 172.21.15.1:0/3168191106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d15abda930 msgr2=0x55d15abe2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:55.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.542+0000 7f33667eab80 1 --2- 172.21.15.1:0/3168191106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d15abda930 0x55d15abe2400 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f334c073cc0 tx=0x7f3334003ea0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.542+0000 7f33667eab80 1 -- 172.21.15.1:0/3168191106 shutdown_connections 2024-02-02T15:27:55.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.542+0000 7f33667eab80 1 --2- 172.21.15.1:0/3168191106 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d15abe57e0 0x55d15abe2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.542+0000 7f33667eab80 1 --2- 172.21.15.1:0/3168191106 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55d15abf5260 0x55d15abf7680 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.542+0000 7f3339600700 0 client.25876 ms_handle_reset on v2:172.21.15.142:6836/2870289019 2024-02-02T15:27:55.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.542+0000 7f33667eab80 1 --2- 172.21.15.1:0/3168191106 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d15abda010 0x55d15abe51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.542+0000 7f33667eab80 1 --2- 172.21.15.1:0/3168191106 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d15abda930 0x55d15abe2400 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.542+0000 7f33667eab80 1 -- 172.21.15.1:0/3168191106 >> 172.21.15.1:0/3168191106 conn(0x55d15ab39db0 msgr2=0x55d15abd5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:55.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.542+0000 7f33667eab80 1 -- 172.21.15.1:0/3168191106 shutdown_connections 2024-02-02T15:27:55.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.542+0000 7f33667eab80 1 -- 172.21.15.1:0/3168191106 wait complete. 2024-02-02T15:27:55.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.542+0000 7f33667eab80 20 client.25876 trim_cache size 0 max 16384 2024-02-02T15:27:55.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.567+0000 7f33667eab80 1 Processor -- start 2024-02-02T15:27:55.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.567+0000 7f33667eab80 1 -- start start 2024-02-02T15:27:55.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.567+0000 7f33667eab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d15abda010 0x55d15abdcb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:55.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.567+0000 7f33667eab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d15abda930 0x55d15abe4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:55.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.567+0000 7f33667eab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d15abe57e0 0x55d15abe23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:55.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.567+0000 7f33667eab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d15a9af6a0 con 0x55d15abe57e0 2024-02-02T15:27:55.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.567+0000 7f33667eab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d15abde2d0 con 0x55d15abda010 2024-02-02T15:27:55.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.567+0000 7f33667eab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d15abd7100 con 0x55d15abda930 2024-02-02T15:27:55.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.568+0000 7f3350c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d15abda010 0x55d15abdcb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:55.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.568+0000 7f3350c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d15abe57e0 0x55d15abe23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:55.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.568+0000 7f3350c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d15abe57e0 0x55d15abe23c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33188/0 (socket says 172.21.15.1:33188) 2024-02-02T15:27:55.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.568+0000 7f3350c00700 1 -- 172.21.15.1:0/719156882 learned_addr learned my addr 172.21.15.1:0/719156882 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:55.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.568+0000 7f334b400700 1 --2- 172.21.15.1:0/719156882 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d15abda930 0x55d15abe4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:55.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.568+0000 7f3350c00700 1 -- 172.21.15.1:0/719156882 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d15abda930 msgr2=0x55d15abe4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:55.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.568+0000 7f3350c00700 1 --2- 172.21.15.1:0/719156882 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d15abda930 0x55d15abe4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.568+0000 7f3350c00700 1 -- 172.21.15.1:0/719156882 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d15abda010 msgr2=0x55d15abdcb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:55.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.568+0000 7f3350c00700 1 --2- 172.21.15.1:0/719156882 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d15abda010 0x55d15abdcb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.568+0000 7f3350c00700 1 -- 172.21.15.1:0/719156882 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d15aaf1ea0 con 0x55d15abe57e0 2024-02-02T15:27:55.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.568+0000 7f334b400700 1 --2- 172.21.15.1:0/719156882 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d15abda930 0x55d15abe4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:27:55.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.569+0000 7f3350c00700 1 --2- 172.21.15.1:0/719156882 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d15abe57e0 0x55d15abe23c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f3344006090 tx=0x7f3344009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ac1786d9c8f3206 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:55.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.569+0000 7f3341c00700 1 -- 172.21.15.1:0/719156882 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f334400e040 con 0x55d15abe57e0 2024-02-02T15:27:55.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.569+0000 7f3341c00700 1 -- 172.21.15.1:0/719156882 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3344025350 con 0x55d15abe57e0 2024-02-02T15:27:55.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.569+0000 7f3341c00700 1 -- 172.21.15.1:0/719156882 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3344004d70 con 0x55d15abe57e0 2024-02-02T15:27:55.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.570+0000 7f33667eab80 1 -- 172.21.15.1:0/719156882 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d15abe57e0 msgr2=0x55d15abe23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:55.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.570+0000 7f33667eab80 1 --2- 172.21.15.1:0/719156882 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d15abe57e0 0x55d15abe23c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f3344006090 tx=0x7f3344009920 comp rx=0 tx=0).stop 2024-02-02T15:27:55.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.570+0000 7f33667eab80 1 -- 172.21.15.1:0/719156882 shutdown_connections 2024-02-02T15:27:55.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.570+0000 7f33667eab80 1 --2- 172.21.15.1:0/719156882 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d15abe57e0 0x55d15abe23c0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.570+0000 7f33667eab80 1 --2- 172.21.15.1:0/719156882 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d15abda010 0x55d15abdcb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.570+0000 7f33667eab80 1 --2- 172.21.15.1:0/719156882 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d15abda930 0x55d15abe4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.570+0000 7f33667eab80 1 -- 172.21.15.1:0/719156882 >> 172.21.15.1:0/719156882 conn(0x55d15ab39db0 msgr2=0x55d15abf0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:55.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.570+0000 7f33667eab80 1 -- 172.21.15.1:0/719156882 shutdown_connections 2024-02-02T15:27:55.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.570+0000 7f33667eab80 1 -- 172.21.15.1:0/719156882 wait complete. 2024-02-02T15:27:55.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.571+0000 7f33667eab80 1 -- start start 2024-02-02T15:27:55.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.571+0000 7f33667eab80 1 Processor -- start 2024-02-02T15:27:55.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.571+0000 7f33667eab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d15abda010 0x55d15abe9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:55.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.572+0000 7f33667eab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d15abda930 0x55d15abe9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:55.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.572+0000 7f33667eab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d15abe57e0 0x55d15abea570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:55.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.572+0000 7f33667eab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d15ac45f30 con 0x55d15abda010 2024-02-02T15:27:55.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.572+0000 7f33667eab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d15ac460a0 con 0x55d15abda930 2024-02-02T15:27:55.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.572+0000 7f33667eab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d15ac46260 con 0x55d15abe57e0 2024-02-02T15:27:55.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.572+0000 7f33667eab80 10 client.0 _ll_register_callbacks cb 0x55d15aac9290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:27:55.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.572+0000 7f3350c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d15abda010 0x55d15abe9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:55.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.572+0000 7f3350c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d15abda010 0x55d15abe9ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33200/0 (socket says 172.21.15.1:33200) 2024-02-02T15:27:55.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.572+0000 7f3350c00700 1 -- 172.21.15.1:0/2573341817 learned_addr learned my addr 172.21.15.1:0/2573341817 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:55.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.572+0000 7f334b400700 1 --2- 172.21.15.1:0/2573341817 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d15abda930 0x55d15abe9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:55.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.572+0000 7f3350c00700 1 --2- 172.21.15.1:0/2573341817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d15abe57e0 0x55d15abea570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:55.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.572+0000 7f3350c00700 1 -- 172.21.15.1:0/2573341817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d15abe57e0 msgr2=0x55d15abea570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:55.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.572+0000 7f3350c00700 1 --2- 172.21.15.1:0/2573341817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d15abe57e0 0x55d15abea570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.572+0000 7f3350c00700 1 -- 172.21.15.1:0/2573341817 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d15abda930 msgr2=0x55d15abe9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:55.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.572+0000 7f3350c00700 1 --2- 172.21.15.1:0/2573341817 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d15abda930 0x55d15abe9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.572+0000 7f3350c00700 1 -- 172.21.15.1:0/2573341817 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d15a9ad3b0 con 0x55d15abda010 2024-02-02T15:27:55.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.573+0000 7f3350c00700 1 --2- 172.21.15.1:0/2573341817 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d15abda010 0x55d15abe9ab0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f3344011c70 tx=0x7f33440021f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4be12d782987f61 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:55.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.573+0000 7f3339600700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:27:55.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.573+0000 7f3339600700 1 -- 172.21.15.1:0/2573341817 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f334400e070 con 0x55d15abda010 2024-02-02T15:27:55.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.573+0000 7f3339600700 1 -- 172.21.15.1:0/2573341817 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f3344003c10 con 0x55d15abda010 2024-02-02T15:27:55.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.573+0000 7f33667eab80 1 -- 172.21.15.1:0/2573341817 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f334400f040 con 0x55d15abda010 2024-02-02T15:27:55.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.573+0000 7f3339600700 1 -- 172.21.15.1:0/2573341817 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3344003dd0 con 0x55d15abda010 2024-02-02T15:27:55.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.573+0000 7f33667eab80 10 client.15834 fetch_fsmap learned FSMap version 27 2024-02-02T15:27:55.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.573+0000 7f33667eab80 1 -- 172.21.15.1:0/2573341817 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55d15a9b1cd0 con 0x55d15abda010 2024-02-02T15:27:55.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.574+0000 7f3339600700 1 -- 172.21.15.1:0/2573341817 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f334400f040 con 0x55d15abda010 2024-02-02T15:27:55.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.574+0000 7f3339600700 1 -- 172.21.15.1:0/2573341817 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f33440154e0 con 0x55d15abda010 2024-02-02T15:27:55.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.574+0000 7f33667eab80 10 client.15834 fetch_fsmap finished waiting for FSMap version 27 2024-02-02T15:27:55.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.574+0000 7f33667eab80 10 client.15834 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-02-02T15:27:55.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.574+0000 7f33667eab80 20 client.15834 populate_metadata read hostname 'smithi001' 2024-02-02T15:27:55.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.574+0000 7f33667eab80 1 --2- 172.21.15.1:0/2573341817 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55d15abf5260 0x55d15ac495b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:55.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.574+0000 7f33667eab80 4 client.15834 mds_command: new command op to 24514 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:27:55.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.574+0000 7f33667eab80 1 -- 172.21.15.1:0/2573341817 --> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d15aaeaf50 con 0x55d15abf5260 2024-02-02T15:27:55.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.575+0000 7f334b400700 1 --2- 172.21.15.1:0/2573341817 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55d15abf5260 0x55d15ac495b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:55.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.575+0000 7f334b400700 1 --2- 172.21.15.1:0/2573341817 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55d15abf5260 0x55d15ac495b0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8e42141cef66c34 server_cookie=ec678511cf0871d5 in_seq=0 out_seq=0 2024-02-02T15:27:55.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.575+0000 7f3339600700 10 client.15834 ms_handle_connect on v2:172.21.15.142:6836/2870289019 2024-02-02T15:27:55.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.575+0000 7f3339600700 1 -- 172.21.15.1:0/2573341817 <== mds.3 v2:172.21.15.142:6836/2870289019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d15aaeaf50 con 0x55d15abf5260 2024-02-02T15:27:55.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.575+0000 7f3339600700 10 client.15834 handle_command_reply: tid=0 2024-02-02T15:27:55.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.575+0000 7f33667eab80 1 client.15834 shutdown 2024-02-02T15:27:55.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.576+0000 7f33667eab80 1 -- 172.21.15.1:0/2573341817 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d15abda010 msgr2=0x55d15abe9ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:55.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.576+0000 7f33667eab80 1 --2- 172.21.15.1:0/2573341817 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d15abda010 0x55d15abe9ab0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f3344011c70 tx=0x7f33440021f0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.576+0000 7f33667eab80 1 -- 172.21.15.1:0/2573341817 shutdown_connections 2024-02-02T15:27:55.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.576+0000 7f33667eab80 1 --2- 172.21.15.1:0/2573341817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d15abe57e0 0x55d15abea570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.576+0000 7f33667eab80 1 --2- 172.21.15.1:0/2573341817 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55d15abf5260 0x55d15ac495b0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.576+0000 7f33667eab80 1 --2- 172.21.15.1:0/2573341817 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d15abda010 0x55d15abe9ab0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.576+0000 7f33667eab80 1 --2- 172.21.15.1:0/2573341817 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d15abda930 0x55d15abe9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.576+0000 7f33667eab80 1 -- 172.21.15.1:0/2573341817 >> 172.21.15.1:0/2573341817 conn(0x55d15ab39db0 msgr2=0x55d15abe2350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:55.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.576+0000 7f3339600700 0 client.15834 ms_handle_reset on v2:172.21.15.142:6836/2870289019 2024-02-02T15:27:55.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.576+0000 7f33667eab80 1 -- 172.21.15.1:0/2573341817 shutdown_connections 2024-02-02T15:27:55.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.576+0000 7f33667eab80 1 -- 172.21.15.1:0/2573341817 wait complete. 2024-02-02T15:27:55.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.577+0000 7f33667eab80 20 client.15834 trim_cache size 0 max 16384 2024-02-02T15:27:55.582 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:27:55.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.577+0000 7f3352c00700 1 -- 172.21.15.1:0/2664941010 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f332c0a07a0 msgr2=0x7f332c0a2c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:55.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.577+0000 7f3352c00700 1 --2- 172.21.15.1:0/2664941010 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f332c0a07a0 0x7f332c0a2c60 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f333c0c9770 tx=0x7f333c0cfc30 comp rx=0 tx=0).stop 2024-02-02T15:27:55.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.577+0000 7f3352c00700 1 -- 172.21.15.1:0/2664941010 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f334c1000e0 msgr2=0x7f334c101990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:55.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.577+0000 7f3352c00700 1 --2- 172.21.15.1:0/2664941010 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f334c1000e0 0x7f334c101990 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f333c00e9a0 tx=0x7f333c00ecb0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.577+0000 7f3352c00700 1 -- 172.21.15.1:0/2664941010 shutdown_connections 2024-02-02T15:27:55.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.577+0000 7f3352c00700 1 --2- 172.21.15.1:0/2664941010 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f332c0a07a0 0x7f332c0a2c60 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.577+0000 7f3352c00700 1 --2- 172.21.15.1:0/2664941010 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f334c102410 0x7f334c1a50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.577+0000 7f3352c00700 1 --2- 172.21.15.1:0/2664941010 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f334c1000e0 0x7f334c101990 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.577+0000 7f3352c00700 1 --2- 172.21.15.1:0/2664941010 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f334c105020 0x7f334c101ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.577+0000 7f3352c00700 1 -- 172.21.15.1:0/2664941010 >> 172.21.15.1:0/2664941010 conn(0x7f334c0757d0 msgr2=0x7f334c0681b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:55.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.578+0000 7f3352c00700 1 -- 172.21.15.1:0/2664941010 shutdown_connections 2024-02-02T15:27:55.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.578+0000 7f3352c00700 1 -- 172.21.15.1:0/2664941010 wait complete. 2024-02-02T15:27:55.599 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:55 smithi001 ceph-mon[35470]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-02T15:27:55.600 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:55 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/2407352987' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:27:55.602 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-02T15:27:55.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:55 smithi071 ceph-mon[44707]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-02T15:27:55.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:55 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/2407352987' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:27:55.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:55 smithi142 ceph-mon[45728]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-02T15:27:55.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:55 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/2407352987' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:27:55.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.862+0000 7f3ed3000700 1 -- 172.21.15.1:0/68184619 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ecc073a30 msgr2=0x7f3ecc10ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:55.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.862+0000 7f3ed3000700 1 --2- 172.21.15.1:0/68184619 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ecc073a30 0x7f3ecc10ce20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3eb40098f0 tx=0x7f3eb4009c00 comp rx=0 tx=0).stop 2024-02-02T15:27:55.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.862+0000 7f3ed3000700 1 -- 172.21.15.1:0/68184619 shutdown_connections 2024-02-02T15:27:55.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.862+0000 7f3ed3000700 1 --2- 172.21.15.1:0/68184619 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ecc10d360 0x7f3ecc10f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.862+0000 7f3ed3000700 1 --2- 172.21.15.1:0/68184619 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ecc073110 0x7f3ecc0734f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.862+0000 7f3ed3000700 1 --2- 172.21.15.1:0/68184619 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ecc073a30 0x7f3ecc10ce20 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.862+0000 7f3ed3000700 1 -- 172.21.15.1:0/68184619 >> 172.21.15.1:0/68184619 conn(0x7f3ecc0fc5d0 msgr2=0x7f3ecc0fe9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:55.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.862+0000 7f3ed3000700 1 -- 172.21.15.1:0/68184619 shutdown_connections 2024-02-02T15:27:55.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.862+0000 7f3ed3000700 1 -- 172.21.15.1:0/68184619 wait complete. 2024-02-02T15:27:55.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.864+0000 7f3ed3000700 1 Processor -- start 2024-02-02T15:27:55.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.864+0000 7f3ed3000700 1 -- start start 2024-02-02T15:27:55.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.864+0000 7f3ed3000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ecc073110 0x7f3ecc198e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:55.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.864+0000 7f3ed3000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ecc073a30 0x7f3ecc199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:55.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.864+0000 7f3ed3000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ecc10d360 0x7f3ecc19d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:55.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.864+0000 7f3ed3000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ecc19de00 con 0x7f3ecc073110 2024-02-02T15:27:55.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.864+0000 7f3ed3000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ecc19df70 con 0x7f3ecc10d360 2024-02-02T15:27:55.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.864+0000 7f3ed3000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ecc19e130 con 0x7f3ecc073a30 2024-02-02T15:27:55.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.865+0000 7f3ed1c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ecc073110 0x7f3ecc198e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:55.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.865+0000 7f3ed1200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ecc073a30 0x7f3ecc199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:55.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.865+0000 7f3ed1200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ecc073a30 0x7f3ecc199360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:60904/0 (socket says 172.21.15.1:60904) 2024-02-02T15:27:55.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.865+0000 7f3ed1c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ecc073110 0x7f3ecc198e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33230/0 (socket says 172.21.15.1:33230) 2024-02-02T15:27:55.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.865+0000 7f3ed1200700 1 -- 172.21.15.1:0/2545565703 learned_addr learned my addr 172.21.15.1:0/2545565703 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:55.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.865+0000 7f3ed2600700 1 --2- 172.21.15.1:0/2545565703 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ecc10d360 0x7f3ecc19d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:55.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.865+0000 7f3ed1200700 1 -- 172.21.15.1:0/2545565703 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ecc10d360 msgr2=0x7f3ecc19d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:55.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.865+0000 7f3ed1200700 1 --2- 172.21.15.1:0/2545565703 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ecc10d360 0x7f3ecc19d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.865+0000 7f3ed1200700 1 -- 172.21.15.1:0/2545565703 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ecc073110 msgr2=0x7f3ecc198e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:55.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.865+0000 7f3ed1200700 1 --2- 172.21.15.1:0/2545565703 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ecc073110 0x7f3ecc198e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.865+0000 7f3ed1200700 1 -- 172.21.15.1:0/2545565703 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ebc0095f0 con 0x7f3ecc073a30 2024-02-02T15:27:55.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.865+0000 7f3ed2600700 1 --2- 172.21.15.1:0/2545565703 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ecc10d360 0x7f3ecc19d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:27:55.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.866+0000 7f3ed1200700 1 --2- 172.21.15.1:0/2545565703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ecc073a30 0x7f3ecc199360 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f3eb4009810 tx=0x7f3eb400fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:55.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.866+0000 7f3eca000700 1 -- 172.21.15.1:0/2545565703 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb401c070 con 0x7f3ecc073a30 2024-02-02T15:27:55.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.866+0000 7f3ed3000700 1 -- 172.21.15.1:0/2545565703 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3eb4009520 con 0x7f3ecc073a30 2024-02-02T15:27:55.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.866+0000 7f3eca000700 1 -- 172.21.15.1:0/2545565703 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3eb400bbd0 con 0x7f3ecc073a30 2024-02-02T15:27:55.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.866+0000 7f3eca000700 1 -- 172.21.15.1:0/2545565703 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb40049c0 con 0x7f3ecc073a30 2024-02-02T15:27:55.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.866+0000 7f3ed3000700 1 -- 172.21.15.1:0/2545565703 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ecc1a5a70 con 0x7f3ecc073a30 2024-02-02T15:27:55.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.871+0000 7f3ee81a2b80 1 Processor -- start 2024-02-02T15:27:55.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.871+0000 7f3ee81a2b80 1 -- start start 2024-02-02T15:27:55.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.871+0000 7f3eca000700 1 -- 172.21.15.1:0/2545565703 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) v1 ==== 135245+0+0 (secure 0 0 0) 0x7f3eb4020070 con 0x7f3ecc073a30 2024-02-02T15:27:55.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.871+0000 7f3ee81a2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ab2e761f0 0x559ab2e765d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:55.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.871+0000 7f3ee81a2b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559ab2e76b10 0x559ab2e814b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:55.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.871+0000 7f3ee81a2b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ab2e819f0 0x559ab2e83de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:55.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.871+0000 7f3ee81a2b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ab2dea4c0 con 0x559ab2e819f0 2024-02-02T15:27:55.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.871+0000 7f3ee81a2b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ab2dceaa0 con 0x559ab2e76b10 2024-02-02T15:27:55.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.871+0000 7f3ee81a2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ab2dcc460 con 0x559ab2e761f0 2024-02-02T15:27:55.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.872+0000 7f3ed2600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ab2e761f0 0x559ab2e765d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:55.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.872+0000 7f3ed2600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ab2e761f0 0x559ab2e765d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:60916/0 (socket says 172.21.15.1:60916) 2024-02-02T15:27:55.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.872+0000 7f3ed2600700 1 -- 172.21.15.1:0/973857290 learned_addr learned my addr 172.21.15.1:0/973857290 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:55.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.872+0000 7f3ed1200700 1 --2- 172.21.15.1:0/973857290 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ab2e819f0 0x559ab2e83de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:55.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.872+0000 7f3ed1c00700 1 --2- 172.21.15.1:0/973857290 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559ab2e76b10 0x559ab2e814b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:55.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.872+0000 7f3ed1c00700 1 -- 172.21.15.1:0/973857290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ab2e761f0 msgr2=0x559ab2e765d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:55.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.872+0000 7f3ed1c00700 1 --2- 172.21.15.1:0/973857290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ab2e761f0 0x559ab2e765d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.872+0000 7f3ed1c00700 1 -- 172.21.15.1:0/973857290 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ab2e819f0 msgr2=0x559ab2e83de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:55.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.873+0000 7f3ed1c00700 1 --2- 172.21.15.1:0/973857290 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ab2e819f0 0x559ab2e83de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.873+0000 7f3ed1c00700 1 -- 172.21.15.1:0/973857290 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ab2c493b0 con 0x559ab2e76b10 2024-02-02T15:27:55.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.873+0000 7f3ed1200700 1 --2- 172.21.15.1:0/973857290 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ab2e819f0 0x559ab2e83de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:27:55.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.873+0000 7f3ed1c00700 1 --2- 172.21.15.1:0/973857290 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559ab2e76b10 0x559ab2e814b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc009f20 tx=0x7f3ebc011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f377258e58afa66 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:55.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.873+0000 7f3eca000700 1 --2- 172.21.15.1:0/2545565703 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f3eac0a0720 0x7f3eac0a2be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:55.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.873+0000 7f3eca000700 1 -- 172.21.15.1:0/2545565703 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f3eb40ccfd0 con 0x7f3ecc073a30 2024-02-02T15:27:55.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.873+0000 7f3ed2600700 1 --2- 172.21.15.1:0/2545565703 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f3eac0a0720 0x7f3eac0a2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:55.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.873+0000 7f3ec2a00700 1 -- 172.21.15.1:0/973857290 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc004310 con 0x559ab2e76b10 2024-02-02T15:27:55.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.873+0000 7f3ec2a00700 1 -- 172.21.15.1:0/973857290 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ebc004470 con 0x559ab2e76b10 2024-02-02T15:27:55.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.873+0000 7f3ec2a00700 1 -- 172.21.15.1:0/973857290 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc0199d0 con 0x559ab2e76b10 2024-02-02T15:27:55.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.874+0000 7f3ee81a2b80 1 -- 172.21.15.1:0/973857290 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559ab2e76b10 msgr2=0x559ab2e814b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:55.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.874+0000 7f3ee81a2b80 1 --2- 172.21.15.1:0/973857290 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559ab2e76b10 0x559ab2e814b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc009f20 tx=0x7f3ebc011720 comp rx=0 tx=0).stop 2024-02-02T15:27:55.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.874+0000 7f3ed2600700 1 --2- 172.21.15.1:0/2545565703 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f3eac0a0720 0x7f3eac0a2be0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3ec4009fd0 tx=0x7f3ec400b040 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:55.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.874+0000 7f3ee81a2b80 1 -- 172.21.15.1:0/973857290 shutdown_connections 2024-02-02T15:27:55.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.874+0000 7f3ee81a2b80 1 --2- 172.21.15.1:0/973857290 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ab2e819f0 0x559ab2e83de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.874+0000 7f3ee81a2b80 1 --2- 172.21.15.1:0/973857290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ab2e761f0 0x559ab2e765d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.874+0000 7f3ee81a2b80 1 --2- 172.21.15.1:0/973857290 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559ab2e76b10 0x559ab2e814b0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.874+0000 7f3ee81a2b80 1 -- 172.21.15.1:0/973857290 >> 172.21.15.1:0/973857290 conn(0x559ab2dd5db0 msgr2=0x559ab2e725c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:55.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.874+0000 7f3ee81a2b80 1 -- 172.21.15.1:0/973857290 shutdown_connections 2024-02-02T15:27:55.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.874+0000 7f3ee81a2b80 1 -- 172.21.15.1:0/973857290 wait complete. 2024-02-02T15:27:55.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.875+0000 7f3ee81a2b80 1 -- start start 2024-02-02T15:27:55.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.875+0000 7f3ee81a2b80 1 Processor -- start 2024-02-02T15:27:55.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.876+0000 7f3ee81a2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ab2e761f0 0x559ab2e81440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:55.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.876+0000 7f3ee81a2b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ab2e76b10 0x559ab2e7e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:55.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.876+0000 7f3ee81a2b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559ab2e819f0 0x559ab2e7ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:55.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.876+0000 7f3ee81a2b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ab2dc7d50 con 0x559ab2e76b10 2024-02-02T15:27:55.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.876+0000 7f3ee81a2b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ab2d63d80 con 0x559ab2e819f0 2024-02-02T15:27:55.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.876+0000 7f3ee81a2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ab2d63a20 con 0x559ab2e761f0 2024-02-02T15:27:55.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.876+0000 7f3ee81a2b80 10 client.0 _ll_register_callbacks cb 0x559ab2d65290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:27:55.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.876+0000 7f3ed1200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ab2e76b10 0x559ab2e7e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:55.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.877+0000 7f3ed1200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ab2e76b10 0x559ab2e7e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33254/0 (socket says 172.21.15.1:33254) 2024-02-02T15:27:55.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.877+0000 7f3ed1200700 1 -- 172.21.15.1:0/2395767010 learned_addr learned my addr 172.21.15.1:0/2395767010 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:55.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.877+0000 7f3ed1c00700 1 --2- 172.21.15.1:0/2395767010 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ab2e761f0 0x559ab2e81440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:55.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.877+0000 7f3ed2600700 1 --2- 172.21.15.1:0/2395767010 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559ab2e819f0 0x559ab2e7ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:55.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.877+0000 7f3ed1200700 1 -- 172.21.15.1:0/2395767010 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ab2e761f0 msgr2=0x559ab2e81440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:55.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.877+0000 7f3ed1200700 1 --2- 172.21.15.1:0/2395767010 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ab2e761f0 0x559ab2e81440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.877+0000 7f3ed1200700 1 -- 172.21.15.1:0/2395767010 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559ab2e819f0 msgr2=0x559ab2e7ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:55.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.877+0000 7f3ed1200700 1 --2- 172.21.15.1:0/2395767010 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559ab2e819f0 0x559ab2e7ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.877+0000 7f3ed1200700 1 -- 172.21.15.1:0/2395767010 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ecc1a5a70 con 0x559ab2e76b10 2024-02-02T15:27:55.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.877+0000 7f3ed1200700 1 --2- 172.21.15.1:0/2395767010 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ab2e76b10 0x559ab2e7e6c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f3eb40ca340 tx=0x7f3eb4005600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c72ec1e280d5228 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:55.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.877+0000 7f3ebb400700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:27:55.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.878+0000 7f3ebb400700 1 -- 172.21.15.1:0/2395767010 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb409a4c0 con 0x559ab2e76b10 2024-02-02T15:27:55.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.878+0000 7f3ebb400700 1 -- 172.21.15.1:0/2395767010 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f3eb409a760 con 0x559ab2e76b10 2024-02-02T15:27:55.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.878+0000 7f3ee81a2b80 1 -- 172.21.15.1:0/2395767010 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559ab2e6a340 con 0x559ab2e76b10 2024-02-02T15:27:55.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.878+0000 7f3ebb400700 1 -- 172.21.15.1:0/2395767010 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3eb4005470 con 0x559ab2e76b10 2024-02-02T15:27:55.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.878+0000 7f3ee81a2b80 10 client.15846 fetch_fsmap learned FSMap version 27 2024-02-02T15:27:55.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.878+0000 7f3ee81a2b80 1 -- 172.21.15.1:0/2395767010 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f3ebc009b80 con 0x559ab2e76b10 2024-02-02T15:27:55.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.878+0000 7f3ebb400700 1 -- 172.21.15.1:0/2395767010 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559ab2e6a340 con 0x559ab2e76b10 2024-02-02T15:27:55.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.879+0000 7f3ebb400700 1 -- 172.21.15.1:0/2395767010 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3eb4099550 con 0x559ab2e76b10 2024-02-02T15:27:55.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.879+0000 7f3ee81a2b80 10 client.15846 fetch_fsmap finished waiting for FSMap version 27 2024-02-02T15:27:55.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.879+0000 7f3ee81a2b80 10 client.15846 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-02-02T15:27:55.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.879+0000 7f3ee81a2b80 20 client.15846 populate_metadata read hostname 'smithi001' 2024-02-02T15:27:55.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.879+0000 7f3ee81a2b80 1 --2- 172.21.15.1:0/2395767010 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x559ab2e910b0 0x559ab2e934a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:55.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.879+0000 7f3ee81a2b80 4 client.15846 mds_command: new command op to 24518 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:27:55.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.879+0000 7f3ee81a2b80 1 -- 172.21.15.1:0/2395767010 --> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559ab2dc0dd0 con 0x559ab2e910b0 2024-02-02T15:27:55.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.879+0000 7f3ed1c00700 1 --2- 172.21.15.1:0/2395767010 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x559ab2e910b0 0x559ab2e934a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:55.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.880+0000 7f3ed1c00700 1 --2- 172.21.15.1:0/2395767010 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x559ab2e910b0 0x559ab2e934a0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dc4936bca42ffc15 server_cookie=37ac6fb15500da37 in_seq=0 out_seq=0 2024-02-02T15:27:55.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.880+0000 7f3ebb400700 10 client.15846 ms_handle_connect on v2:172.21.15.71:6836/628016199 2024-02-02T15:27:55.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.881+0000 7f3ebb400700 1 -- 172.21.15.1:0/2395767010 <== mds.2 v2:172.21.15.71:6836/628016199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x559ab2dc0dd0 con 0x559ab2e910b0 2024-02-02T15:27:55.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.881+0000 7f3ebb400700 10 client.15846 handle_command_reply: tid=0 2024-02-02T15:27:55.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.881+0000 7f3ee81a2b80 1 client.15846 shutdown 2024-02-02T15:27:55.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.881+0000 7f3ee81a2b80 1 -- 172.21.15.1:0/2395767010 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ab2e76b10 msgr2=0x559ab2e7e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:55.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.881+0000 7f3ee81a2b80 1 --2- 172.21.15.1:0/2395767010 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ab2e76b10 0x559ab2e7e6c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f3eb40ca340 tx=0x7f3eb4005600 comp rx=0 tx=0).stop 2024-02-02T15:27:55.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.882+0000 7f3ee81a2b80 1 -- 172.21.15.1:0/2395767010 shutdown_connections 2024-02-02T15:27:55.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.882+0000 7f3ee81a2b80 1 --2- 172.21.15.1:0/2395767010 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559ab2e819f0 0x559ab2e7ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.882+0000 7f3ee81a2b80 1 --2- 172.21.15.1:0/2395767010 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x559ab2e910b0 0x559ab2e934a0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.882+0000 7f3ee81a2b80 1 --2- 172.21.15.1:0/2395767010 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ab2e761f0 0x559ab2e81440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.882+0000 7f3ee81a2b80 1 --2- 172.21.15.1:0/2395767010 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ab2e76b10 0x559ab2e7e6c0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.882+0000 7f3ebb400700 0 client.15846 ms_handle_reset on v2:172.21.15.71:6836/628016199 2024-02-02T15:27:55.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.882+0000 7f3ee81a2b80 1 -- 172.21.15.1:0/2395767010 >> 172.21.15.1:0/2395767010 conn(0x559ab2dd5db0 msgr2=0x559ab2e721b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:55.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.882+0000 7f3ee81a2b80 1 -- 172.21.15.1:0/2395767010 shutdown_connections 2024-02-02T15:27:55.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.882+0000 7f3ee81a2b80 1 -- 172.21.15.1:0/2395767010 wait complete. 2024-02-02T15:27:55.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.882+0000 7f3ee81a2b80 20 client.15846 trim_cache size 0 max 16384 2024-02-02T15:27:55.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.896+0000 7f3ee81a2b80 1 Processor -- start 2024-02-02T15:27:55.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.896+0000 7f3ee81a2b80 1 -- start start 2024-02-02T15:27:55.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.896+0000 7f3ee81a2b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559ab2e761f0 0x559ab2e884e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:55.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.896+0000 7f3ee81a2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ab2e76b10 0x559ab2e7cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:55.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.896+0000 7f3ee81a2b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ab2e819f0 0x559ab2e8cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:55.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.896+0000 7f3ee81a2b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ab2c4b6a0 con 0x559ab2e819f0 2024-02-02T15:27:55.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.896+0000 7f3ee81a2b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ab2e7dd80 con 0x559ab2e761f0 2024-02-02T15:27:55.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.896+0000 7f3ee81a2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ab2e73260 con 0x559ab2e76b10 2024-02-02T15:27:55.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.896+0000 7f3ed1c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559ab2e761f0 0x559ab2e884e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:55.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.897+0000 7f3ed2600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ab2e819f0 0x559ab2e8cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:55.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.897+0000 7f3ed1c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559ab2e761f0 0x559ab2e884e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:36992/0 (socket says 172.21.15.1:36992) 2024-02-02T15:27:55.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.897+0000 7f3ed1c00700 1 -- 172.21.15.1:0/346177716 learned_addr learned my addr 172.21.15.1:0/346177716 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:55.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.897+0000 7f3ed1200700 1 --2- 172.21.15.1:0/346177716 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ab2e76b10 0x559ab2e7cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:55.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.897+0000 7f3ed1c00700 1 -- 172.21.15.1:0/346177716 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ab2e76b10 msgr2=0x559ab2e7cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:55.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.897+0000 7f3ed1c00700 1 --2- 172.21.15.1:0/346177716 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ab2e76b10 0x559ab2e7cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.897+0000 7f3ed1c00700 1 -- 172.21.15.1:0/346177716 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ab2e819f0 msgr2=0x559ab2e8cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:55.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.897+0000 7f3ed1c00700 1 --2- 172.21.15.1:0/346177716 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ab2e819f0 0x559ab2e8cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.897+0000 7f3ed1c00700 1 -- 172.21.15.1:0/346177716 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ab2bf5b60 con 0x559ab2e761f0 2024-02-02T15:27:55.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.897+0000 7f3ed2600700 1 --2- 172.21.15.1:0/346177716 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ab2e819f0 0x559ab2e8cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:27:55.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.897+0000 7f3ed1c00700 1 --2- 172.21.15.1:0/346177716 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559ab2e761f0 0x559ab2e884e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc009f20 tx=0x7f3ebc009e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0a68370a56741ff server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:55.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.897+0000 7f3ec3400700 1 -- 172.21.15.1:0/346177716 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc004110 con 0x559ab2e761f0 2024-02-02T15:27:55.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.898+0000 7f3ec3400700 1 -- 172.21.15.1:0/346177716 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ebc004270 con 0x559ab2e761f0 2024-02-02T15:27:55.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.898+0000 7f3ec3400700 1 -- 172.21.15.1:0/346177716 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc004df0 con 0x559ab2e761f0 2024-02-02T15:27:55.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.898+0000 7f3ee81a2b80 1 -- 172.21.15.1:0/346177716 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559ab2e761f0 msgr2=0x559ab2e884e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:55.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.898+0000 7f3ee81a2b80 1 --2- 172.21.15.1:0/346177716 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559ab2e761f0 0x559ab2e884e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc009f20 tx=0x7f3ebc009e50 comp rx=0 tx=0).stop 2024-02-02T15:27:55.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.898+0000 7f3ee81a2b80 1 -- 172.21.15.1:0/346177716 shutdown_connections 2024-02-02T15:27:55.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.898+0000 7f3ee81a2b80 1 --2- 172.21.15.1:0/346177716 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ab2e819f0 0x559ab2e8cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.898+0000 7f3ee81a2b80 1 --2- 172.21.15.1:0/346177716 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559ab2e761f0 0x559ab2e884e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.898+0000 7f3ee81a2b80 1 --2- 172.21.15.1:0/346177716 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ab2e76b10 0x559ab2e7cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.899+0000 7f3ee81a2b80 1 -- 172.21.15.1:0/346177716 >> 172.21.15.1:0/346177716 conn(0x559ab2dd5db0 msgr2=0x559ab2e70d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:55.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.899+0000 7f3ee81a2b80 1 -- 172.21.15.1:0/346177716 shutdown_connections 2024-02-02T15:27:55.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.899+0000 7f3ee81a2b80 1 -- 172.21.15.1:0/346177716 wait complete. 2024-02-02T15:27:55.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.899+0000 7f3ee81a2b80 1 -- start start 2024-02-02T15:27:55.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.899+0000 7f3ee81a2b80 1 Processor -- start 2024-02-02T15:27:55.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.900+0000 7f3ee81a2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ab2e761f0 0x559ab2e85d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:55.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.900+0000 7f3ee81a2b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559ab2e76b10 0x559ab2e86260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:55.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.900+0000 7f3ee81a2b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ab2e819f0 0x559ab2ee1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:55.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.900+0000 7f3ee81a2b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ab2e86b70 con 0x559ab2e819f0 2024-02-02T15:27:55.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.900+0000 7f3ee81a2b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ab2ee20e0 con 0x559ab2e76b10 2024-02-02T15:27:55.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.900+0000 7f3ee81a2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ab2ee2220 con 0x559ab2e761f0 2024-02-02T15:27:55.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.900+0000 7f3ee81a2b80 10 client.0 _ll_register_callbacks cb 0x559ab2d65290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:27:55.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.900+0000 7f3ed2600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ab2e819f0 0x559ab2ee1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:55.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.901+0000 7f3ed1c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ab2e761f0 0x559ab2e85d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:55.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.901+0000 7f3ed2600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ab2e819f0 0x559ab2ee1ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33272/0 (socket says 172.21.15.1:33272) 2024-02-02T15:27:55.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.901+0000 7f3ed2600700 1 -- 172.21.15.1:0/3743361010 learned_addr learned my addr 172.21.15.1:0/3743361010 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:55.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.901+0000 7f3ed1c00700 1 -- 172.21.15.1:0/3743361010 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559ab2e76b10 msgr2=0x559ab2e86260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:55.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.901+0000 7f3ed1c00700 1 --2- 172.21.15.1:0/3743361010 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559ab2e76b10 0x559ab2e86260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.901+0000 7f3ed1c00700 1 -- 172.21.15.1:0/3743361010 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ab2e819f0 msgr2=0x559ab2ee1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:55.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.901+0000 7f3ed1c00700 1 --2- 172.21.15.1:0/3743361010 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ab2e819f0 0x559ab2ee1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.901+0000 7f3ed1c00700 1 -- 172.21.15.1:0/3743361010 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ebc01e040 con 0x559ab2e761f0 2024-02-02T15:27:55.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.901+0000 7f3ed1c00700 1 --2- 172.21.15.1:0/3743361010 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ab2e761f0 0x559ab2e85d20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc011a70 tx=0x7f3ebc004200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66f8bc7c0c5ba1b6 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:55.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.901+0000 7f3ebb400700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:27:55.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.902+0000 7f3ebb400700 1 -- 172.21.15.1:0/3743361010 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc004a70 con 0x559ab2e761f0 2024-02-02T15:27:55.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.902+0000 7f3ebb400700 1 -- 172.21.15.1:0/3743361010 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f3ebc004bd0 con 0x559ab2e761f0 2024-02-02T15:27:55.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.902+0000 7f3ee81a2b80 1 -- 172.21.15.1:0/3743361010 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3ebc004470 con 0x559ab2e761f0 2024-02-02T15:27:55.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.902+0000 7f3ebb400700 1 -- 172.21.15.1:0/3743361010 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ebc004d90 con 0x559ab2e761f0 2024-02-02T15:27:55.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.902+0000 7f3ee81a2b80 10 client.25900 fetch_fsmap learned FSMap version 27 2024-02-02T15:27:55.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.902+0000 7f3ee81a2b80 1 -- 172.21.15.1:0/3743361010 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x559ab2d8dea0 con 0x559ab2e761f0 2024-02-02T15:27:55.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.902+0000 7f3ebb400700 1 -- 172.21.15.1:0/3743361010 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ebc004470 con 0x559ab2e761f0 2024-02-02T15:27:55.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.903+0000 7f3ebb400700 1 -- 172.21.15.1:0/3743361010 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3ebc003a90 con 0x559ab2e761f0 2024-02-02T15:27:55.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.903+0000 7f3ee81a2b80 10 client.25900 fetch_fsmap finished waiting for FSMap version 27 2024-02-02T15:27:55.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.903+0000 7f3ee81a2b80 10 client.25900 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-02-02T15:27:55.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.903+0000 7f3ee81a2b80 20 client.25900 populate_metadata read hostname 'smithi001' 2024-02-02T15:27:55.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.903+0000 7f3ee81a2b80 1 --2- 172.21.15.1:0/3743361010 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x559ab2eebdc0 0x559ab2eee1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:55.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.903+0000 7f3ee81a2b80 4 client.25900 mds_command: new command op to 24518 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:27:55.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.903+0000 7f3ee81a2b80 1 -- 172.21.15.1:0/3743361010 --> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559ab2d86f50 con 0x559ab2eebdc0 2024-02-02T15:27:55.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.903+0000 7f3ed1200700 1 --2- 172.21.15.1:0/3743361010 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x559ab2eebdc0 0x559ab2eee1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:55.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.904+0000 7f3ed1200700 1 --2- 172.21.15.1:0/3743361010 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x559ab2eebdc0 0x559ab2eee1b0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b13d4d601a5340be server_cookie=3ee153c8fb02d6e6 in_seq=0 out_seq=0 2024-02-02T15:27:55.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.904+0000 7f3ebb400700 10 client.25900 ms_handle_connect on v2:172.21.15.71:6836/628016199 2024-02-02T15:27:55.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.904+0000 7f3ebb400700 1 -- 172.21.15.1:0/3743361010 <== mds.2 v2:172.21.15.71:6836/628016199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559ab2d86f50 con 0x559ab2eebdc0 2024-02-02T15:27:55.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.904+0000 7f3ebb400700 10 client.25900 handle_command_reply: tid=0 2024-02-02T15:27:55.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.904+0000 7f3ee81a2b80 1 client.25900 shutdown 2024-02-02T15:27:55.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.904+0000 7f3ee81a2b80 1 -- 172.21.15.1:0/3743361010 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ab2e761f0 msgr2=0x559ab2e85d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:55.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.904+0000 7f3ee81a2b80 1 --2- 172.21.15.1:0/3743361010 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ab2e761f0 0x559ab2e85d20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc011a70 tx=0x7f3ebc004200 comp rx=0 tx=0).stop 2024-02-02T15:27:55.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.904+0000 7f3ee81a2b80 1 -- 172.21.15.1:0/3743361010 shutdown_connections 2024-02-02T15:27:55.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.904+0000 7f3ee81a2b80 1 --2- 172.21.15.1:0/3743361010 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ab2e819f0 0x559ab2ee1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.904+0000 7f3ee81a2b80 1 --2- 172.21.15.1:0/3743361010 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x559ab2eebdc0 0x559ab2eee1b0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.904+0000 7f3ee81a2b80 1 --2- 172.21.15.1:0/3743361010 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ab2e761f0 0x559ab2e85d20 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.904+0000 7f3ee81a2b80 1 --2- 172.21.15.1:0/3743361010 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559ab2e76b10 0x559ab2e86260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.904+0000 7f3ebb400700 0 client.25900 ms_handle_reset on v2:172.21.15.71:6836/628016199 2024-02-02T15:27:55.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.904+0000 7f3ee81a2b80 1 -- 172.21.15.1:0/3743361010 >> 172.21.15.1:0/3743361010 conn(0x559ab2dd5db0 msgr2=0x559ab2e8d6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:55.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.905+0000 7f3ee81a2b80 1 -- 172.21.15.1:0/3743361010 shutdown_connections 2024-02-02T15:27:55.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.905+0000 7f3ee81a2b80 1 -- 172.21.15.1:0/3743361010 wait complete. 2024-02-02T15:27:55.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.905+0000 7f3ee81a2b80 20 client.25900 trim_cache size 0 max 16384 2024-02-02T15:27:55.908 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:27:55.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.905+0000 7f3ed3000700 1 -- 172.21.15.1:0/2545565703 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f3eac0a0720 msgr2=0x7f3eac0a2be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:55.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.905+0000 7f3ed3000700 1 --2- 172.21.15.1:0/2545565703 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f3eac0a0720 0x7f3eac0a2be0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3ec4009fd0 tx=0x7f3ec400b040 comp rx=0 tx=0).stop 2024-02-02T15:27:55.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.905+0000 7f3ed3000700 1 -- 172.21.15.1:0/2545565703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ecc073a30 msgr2=0x7f3ecc199360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:55.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.905+0000 7f3ed3000700 1 --2- 172.21.15.1:0/2545565703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ecc073a30 0x7f3ecc199360 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f3eb4009810 tx=0x7f3eb400fa00 comp rx=0 tx=0).stop 2024-02-02T15:27:55.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.906+0000 7f3ed3000700 1 -- 172.21.15.1:0/2545565703 shutdown_connections 2024-02-02T15:27:55.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.906+0000 7f3ed3000700 1 --2- 172.21.15.1:0/2545565703 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f3eac0a0720 0x7f3eac0a2be0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.906+0000 7f3ed3000700 1 --2- 172.21.15.1:0/2545565703 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ecc10d360 0x7f3ecc19d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.906+0000 7f3ed3000700 1 --2- 172.21.15.1:0/2545565703 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ecc073110 0x7f3ecc198e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.906+0000 7f3ed3000700 1 --2- 172.21.15.1:0/2545565703 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ecc073a30 0x7f3ecc199360 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:55.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.906+0000 7f3ed3000700 1 -- 172.21.15.1:0/2545565703 >> 172.21.15.1:0/2545565703 conn(0x7f3ecc0fc5d0 msgr2=0x7f3ecc10f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:55.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.906+0000 7f3ed3000700 1 -- 172.21.15.1:0/2545565703 shutdown_connections 2024-02-02T15:27:55.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:55.906+0000 7f3ed3000700 1 -- 172.21.15.1:0/2545565703 wait complete. 2024-02-02T15:27:55.925 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-02T15:27:56.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.139+0000 7f792d000700 1 -- 172.21.15.1:0/1854159719 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7928068010 msgr2=0x7f7928068490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:56.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.139+0000 7f792d000700 1 --2- 172.21.15.1:0/1854159719 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7928068010 0x7f7928068490 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7910009990 tx=0x7f7910009ca0 comp rx=0 tx=0).stop 2024-02-02T15:27:56.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.139+0000 7f792d000700 1 -- 172.21.15.1:0/1854159719 shutdown_connections 2024-02-02T15:27:56.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.139+0000 7f792d000700 1 --2- 172.21.15.1:0/1854159719 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79280689d0 0x7f792810d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:56.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.139+0000 7f792d000700 1 --2- 172.21.15.1:0/1854159719 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7928106ab0 0x7f7928106e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:56.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.139+0000 7f792d000700 1 --2- 172.21.15.1:0/1854159719 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7928068010 0x7f7928068490 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:56.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.139+0000 7f792d000700 1 -- 172.21.15.1:0/1854159719 >> 172.21.15.1:0/1854159719 conn(0x7f7928076770 msgr2=0x7f7928076b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:56.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.139+0000 7f792d000700 1 -- 172.21.15.1:0/1854159719 shutdown_connections 2024-02-02T15:27:56.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.139+0000 7f792d000700 1 -- 172.21.15.1:0/1854159719 wait complete. 2024-02-02T15:27:56.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.140+0000 7f792d000700 1 Processor -- start 2024-02-02T15:27:56.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.141+0000 7f792d000700 1 -- start start 2024-02-02T15:27:56.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.141+0000 7f792d000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7928068010 0x7f7928198db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:56.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.141+0000 7f792d000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f79280689d0 0x7f79281992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:56.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.141+0000 7f792d000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7928106ab0 0x7f792819d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:56.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.141+0000 7f792d000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f792819dd90 con 0x7f79280689d0 2024-02-02T15:27:56.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.141+0000 7f792d000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f792819df00 con 0x7f7928106ab0 2024-02-02T15:27:56.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.141+0000 7f792d000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f792819e0c0 con 0x7f7928068010 2024-02-02T15:27:56.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.142+0000 7f7926a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f79280689d0 0x7f79281992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:56.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.142+0000 7f7926a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f79280689d0 0x7f79281992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33286/0 (socket says 172.21.15.1:33286) 2024-02-02T15:27:56.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.142+0000 7f7926a00700 1 -- 172.21.15.1:0/1163866214 learned_addr learned my addr 172.21.15.1:0/1163866214 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:56.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.142+0000 7f7927400700 1 --2- 172.21.15.1:0/1163866214 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7928068010 0x7f7928198db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:56.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.142+0000 7f7927e00700 1 --2- 172.21.15.1:0/1163866214 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7928106ab0 0x7f792819d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:56.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.142+0000 7f7926a00700 1 -- 172.21.15.1:0/1163866214 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7928068010 msgr2=0x7f7928198db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:56.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.142+0000 7f7926a00700 1 --2- 172.21.15.1:0/1163866214 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7928068010 0x7f7928198db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:56.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.142+0000 7f7926a00700 1 -- 172.21.15.1:0/1163866214 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7928106ab0 msgr2=0x7f792819d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:56.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.142+0000 7f7926a00700 1 --2- 172.21.15.1:0/1163866214 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7928106ab0 0x7f792819d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:56.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.142+0000 7f7926a00700 1 -- 172.21.15.1:0/1163866214 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79100095f0 con 0x7f79280689d0 2024-02-02T15:27:56.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.142+0000 7f7927e00700 1 --2- 172.21.15.1:0/1163866214 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7928106ab0 0x7f792819d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:27:56.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.142+0000 7f7927400700 1 --2- 172.21.15.1:0/1163866214 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7928068010 0x7f7928198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:27:56.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.142+0000 7f7926a00700 1 --2- 172.21.15.1:0/1163866214 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f79280689d0 0x7f79281992f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f7910006010 tx=0x7f7910005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:56.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.143+0000 7f791fe00700 1 -- 172.21.15.1:0/1163866214 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f791001d070 con 0x7f79280689d0 2024-02-02T15:27:56.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.143+0000 7f791fe00700 1 -- 172.21.15.1:0/1163866214 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7910022470 con 0x7f79280689d0 2024-02-02T15:27:56.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.143+0000 7f791fe00700 1 -- 172.21.15.1:0/1163866214 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7910017650 con 0x7f79280689d0 2024-02-02T15:27:56.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.143+0000 7f792d000700 1 -- 172.21.15.1:0/1163866214 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f792819e340 con 0x7f79280689d0 2024-02-02T15:27:56.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.143+0000 7f792d000700 1 -- 172.21.15.1:0/1163866214 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79281a5be0 con 0x7f79280689d0 2024-02-02T15:27:56.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.147+0000 7f791fe00700 1 -- 172.21.15.1:0/1163866214 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135245+0+0 (secure 0 0 0) 0x7f7910022850 con 0x7f79280689d0 2024-02-02T15:27:56.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.148+0000 7f791fe00700 1 --2- 172.21.15.1:0/1163866214 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f79080a0720 0x7f79080a2be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:56.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.148+0000 7f791fe00700 1 -- 172.21.15.1:0/1163866214 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f791009aeb0 con 0x7f79280689d0 2024-02-02T15:27:56.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.149+0000 7f7927400700 1 --2- 172.21.15.1:0/1163866214 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f79080a0720 0x7f79080a2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:56.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.150+0000 7f7942123b80 1 Processor -- start 2024-02-02T15:27:56.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.150+0000 7f7942123b80 1 -- start start 2024-02-02T15:27:56.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.151+0000 7f7927400700 1 --2- 172.21.15.1:0/1163866214 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f79080a0720 0x7f79080a2be0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7918009fd0 tx=0x7f7918009260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:56.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.151+0000 7f7942123b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e97ca251f0 0x55e97ca255d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:56.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.151+0000 7f7942123b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e97ca25b10 0x55e97ca304b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:56.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.151+0000 7f7942123b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e97ca309f0 0x55e97ca32de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:56.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.151+0000 7f7942123b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e97c9994c0 con 0x55e97ca251f0 2024-02-02T15:27:56.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.151+0000 7f7942123b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e97c97daa0 con 0x55e97ca25b10 2024-02-02T15:27:56.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.151+0000 7f7942123b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e97c97b460 con 0x55e97ca309f0 2024-02-02T15:27:56.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.151+0000 7f7926a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e97ca25b10 0x55e97ca304b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:56.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.151+0000 7f7926a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e97ca25b10 0x55e97ca304b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:37012/0 (socket says 172.21.15.1:37012) 2024-02-02T15:27:56.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.151+0000 7f7926a00700 1 -- 172.21.15.1:0/132269751 learned_addr learned my addr 172.21.15.1:0/132269751 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:56.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.152+0000 7f7927e00700 1 --2- 172.21.15.1:0/132269751 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e97ca309f0 0x55e97ca32de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:56.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.152+0000 7f7927400700 1 --2- 172.21.15.1:0/132269751 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e97ca251f0 0x55e97ca255d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:56.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.152+0000 7f7926a00700 1 -- 172.21.15.1:0/132269751 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e97ca309f0 msgr2=0x55e97ca32de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:56.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.152+0000 7f7926a00700 1 --2- 172.21.15.1:0/132269751 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e97ca309f0 0x55e97ca32de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:56.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.152+0000 7f7926a00700 1 -- 172.21.15.1:0/132269751 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e97ca251f0 msgr2=0x55e97ca255d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:56.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.152+0000 7f7926a00700 1 --2- 172.21.15.1:0/132269751 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e97ca251f0 0x55e97ca255d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:56.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.152+0000 7f7926a00700 1 -- 172.21.15.1:0/132269751 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e97c7f83b0 con 0x55e97ca25b10 2024-02-02T15:27:56.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.152+0000 7f7927e00700 1 --2- 172.21.15.1:0/132269751 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e97ca309f0 0x55e97ca32de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:27:56.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.152+0000 7f7926a00700 1 --2- 172.21.15.1:0/132269751 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e97ca25b10 0x55e97ca304b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7910097ca0 tx=0x7f791009e7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5cbe758e45374c6c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:56.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.152+0000 7f791cc00700 1 -- 172.21.15.1:0/132269751 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f791009a110 con 0x55e97ca25b10 2024-02-02T15:27:56.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.152+0000 7f791cc00700 1 -- 172.21.15.1:0/132269751 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f791009a580 con 0x55e97ca25b10 2024-02-02T15:27:56.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.152+0000 7f791cc00700 1 -- 172.21.15.1:0/132269751 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7910098b00 con 0x55e97ca25b10 2024-02-02T15:27:56.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.153+0000 7f7942123b80 1 -- 172.21.15.1:0/132269751 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e97ca25b10 msgr2=0x55e97ca304b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:56.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.153+0000 7f7942123b80 1 --2- 172.21.15.1:0/132269751 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e97ca25b10 0x55e97ca304b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7910097ca0 tx=0x7f791009e7d0 comp rx=0 tx=0).stop 2024-02-02T15:27:56.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.153+0000 7f7942123b80 1 -- 172.21.15.1:0/132269751 shutdown_connections 2024-02-02T15:27:56.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.153+0000 7f7942123b80 1 --2- 172.21.15.1:0/132269751 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e97ca309f0 0x55e97ca32de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:56.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.153+0000 7f7942123b80 1 --2- 172.21.15.1:0/132269751 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e97ca251f0 0x55e97ca255d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:56.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.153+0000 7f7942123b80 1 --2- 172.21.15.1:0/132269751 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e97ca25b10 0x55e97ca304b0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:56.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.153+0000 7f7942123b80 1 -- 172.21.15.1:0/132269751 >> 172.21.15.1:0/132269751 conn(0x55e97c984db0 msgr2=0x55e97ca215c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:56.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.154+0000 7f7942123b80 1 -- 172.21.15.1:0/132269751 shutdown_connections 2024-02-02T15:27:56.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.154+0000 7f7942123b80 1 -- 172.21.15.1:0/132269751 wait complete. 2024-02-02T15:27:56.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.154+0000 7f7942123b80 1 -- start start 2024-02-02T15:27:56.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.155+0000 7f7942123b80 1 Processor -- start 2024-02-02T15:27:56.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.156+0000 7f7942123b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e97ca251f0 0x55e97ca35a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:56.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.156+0000 7f7942123b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e97ca25b10 0x55e97ca35fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:56.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.156+0000 7f7942123b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e97ca309f0 0x55e97ca2f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:56.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.156+0000 7f7942123b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e97c976d50 con 0x55e97ca25b10 2024-02-02T15:27:56.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.156+0000 7f7942123b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e97c912d80 con 0x55e97ca309f0 2024-02-02T15:27:56.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.156+0000 7f7942123b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e97c912a20 con 0x55e97ca251f0 2024-02-02T15:27:56.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.156+0000 7f7942123b80 10 client.0 _ll_register_callbacks cb 0x55e97c914290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:27:56.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.156+0000 7f7927400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e97ca251f0 0x55e97ca35a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:56.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.156+0000 7f7927400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e97ca251f0 0x55e97ca35a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:60988/0 (socket says 172.21.15.1:60988) 2024-02-02T15:27:56.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.156+0000 7f7927400700 1 -- 172.21.15.1:0/4096306640 learned_addr learned my addr 172.21.15.1:0/4096306640 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:56.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.156+0000 7f7927e00700 1 --2- 172.21.15.1:0/4096306640 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e97ca309f0 0x55e97ca2f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:56.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.157+0000 7f7926a00700 1 --2- 172.21.15.1:0/4096306640 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e97ca25b10 0x55e97ca35fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:56.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.157+0000 7f7927e00700 1 -- 172.21.15.1:0/4096306640 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e97ca251f0 msgr2=0x55e97ca35a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:56.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.157+0000 7f7927e00700 1 --2- 172.21.15.1:0/4096306640 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e97ca251f0 0x55e97ca35a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:56.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.157+0000 7f7927e00700 1 -- 172.21.15.1:0/4096306640 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e97ca25b10 msgr2=0x55e97ca35fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:56.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.157+0000 7f7927e00700 1 --2- 172.21.15.1:0/4096306640 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e97ca25b10 0x55e97ca35fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:56.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.157+0000 7f7927e00700 1 -- 172.21.15.1:0/4096306640 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79281a5be0 con 0x55e97ca309f0 2024-02-02T15:27:56.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.157+0000 7f7926a00700 1 --2- 172.21.15.1:0/4096306640 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e97ca25b10 0x55e97ca35fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:27:56.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.157+0000 7f7927e00700 1 --2- 172.21.15.1:0/4096306640 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e97ca309f0 0x55e97ca2f950 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7920007a60 tx=0x7f7920007d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7d35ee1acaa919d server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:56.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.157+0000 7f7915600700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:27:56.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.158+0000 7f7915600700 1 -- 172.21.15.1:0/4096306640 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f792000e030 con 0x55e97ca309f0 2024-02-02T15:27:56.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.158+0000 7f7942123b80 1 -- 172.21.15.1:0/4096306640 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e97ca19340 con 0x55e97ca309f0 2024-02-02T15:27:56.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.158+0000 7f7915600700 1 -- 172.21.15.1:0/4096306640 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f792000f070 con 0x55e97ca309f0 2024-02-02T15:27:56.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.158+0000 7f7915600700 1 -- 172.21.15.1:0/4096306640 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7920003a00 con 0x55e97ca309f0 2024-02-02T15:27:56.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.158+0000 7f7942123b80 10 client.25907 fetch_fsmap learned FSMap version 27 2024-02-02T15:27:56.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.158+0000 7f7942123b80 1 -- 172.21.15.1:0/4096306640 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e97c7a4b60 con 0x55e97ca309f0 2024-02-02T15:27:56.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.158+0000 7f7915600700 1 -- 172.21.15.1:0/4096306640 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e97ca19340 con 0x55e97ca309f0 2024-02-02T15:27:56.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.158+0000 7f7915600700 1 -- 172.21.15.1:0/4096306640 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7920014070 con 0x55e97ca309f0 2024-02-02T15:27:56.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.158+0000 7f7942123b80 10 client.25907 fetch_fsmap finished waiting for FSMap version 27 2024-02-02T15:27:56.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.158+0000 7f7942123b80 10 client.25907 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-02T15:27:56.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.158+0000 7f7942123b80 20 client.25907 populate_metadata read hostname 'smithi001' 2024-02-02T15:27:56.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.159+0000 7f7942123b80 1 --2- 172.21.15.1:0/4096306640 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55e97ca3ff80 0x55e97ca423a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:56.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.159+0000 7f7942123b80 4 client.25907 mds_command: new command op to 24529 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:27:56.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.159+0000 7f7942123b80 1 -- 172.21.15.1:0/4096306640 --> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e97c96fdd0 con 0x55e97ca3ff80 2024-02-02T15:27:56.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.159+0000 7f7927400700 1 --2- 172.21.15.1:0/4096306640 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55e97ca3ff80 0x55e97ca423a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:56.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.160+0000 7f7915600700 10 client.25907 ms_handle_connect on v2:172.21.15.142:6838/321675421 2024-02-02T15:27:56.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.160+0000 7f7927400700 1 --2- 172.21.15.1:0/4096306640 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55e97ca3ff80 0x55e97ca423a0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=87d3f1c537fe08d0 server_cookie=5d8c1ee1be52072 in_seq=0 out_seq=0 2024-02-02T15:27:56.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.160+0000 7f7915600700 1 -- 172.21.15.1:0/4096306640 <== mds.1 v2:172.21.15.142:6838/321675421 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55e97c96fdd0 con 0x55e97ca3ff80 2024-02-02T15:27:56.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.160+0000 7f7915600700 10 client.25907 handle_command_reply: tid=0 2024-02-02T15:27:56.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.160+0000 7f7942123b80 1 client.25907 shutdown 2024-02-02T15:27:56.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.161+0000 7f7942123b80 1 -- 172.21.15.1:0/4096306640 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e97ca309f0 msgr2=0x55e97ca2f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:56.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.161+0000 7f7942123b80 1 --2- 172.21.15.1:0/4096306640 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e97ca309f0 0x55e97ca2f950 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7920007a60 tx=0x7f7920007d70 comp rx=0 tx=0).stop 2024-02-02T15:27:56.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.161+0000 7f7942123b80 1 -- 172.21.15.1:0/4096306640 shutdown_connections 2024-02-02T15:27:56.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.161+0000 7f7942123b80 1 --2- 172.21.15.1:0/4096306640 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55e97ca3ff80 0x55e97ca423a0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:56.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.161+0000 7f7942123b80 1 --2- 172.21.15.1:0/4096306640 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e97ca309f0 0x55e97ca2f950 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:56.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.161+0000 7f7942123b80 1 --2- 172.21.15.1:0/4096306640 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e97ca251f0 0x55e97ca35a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:56.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.161+0000 7f7915600700 0 client.25907 ms_handle_reset on v2:172.21.15.142:6838/321675421 2024-02-02T15:27:56.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.161+0000 7f7942123b80 1 --2- 172.21.15.1:0/4096306640 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e97ca25b10 0x55e97ca35fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:56.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.161+0000 7f7942123b80 1 -- 172.21.15.1:0/4096306640 >> 172.21.15.1:0/4096306640 conn(0x55e97c984db0 msgr2=0x55e97ca21150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:56.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.161+0000 7f7942123b80 1 -- 172.21.15.1:0/4096306640 shutdown_connections 2024-02-02T15:27:56.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.161+0000 7f7942123b80 1 -- 172.21.15.1:0/4096306640 wait complete. 2024-02-02T15:27:56.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.162+0000 7f7942123b80 20 client.25907 trim_cache size 0 max 16384 2024-02-02T15:27:56.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.187+0000 7f7942123b80 1 Processor -- start 2024-02-02T15:27:56.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.187+0000 7f7942123b80 1 -- start start 2024-02-02T15:27:56.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.187+0000 7f7942123b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e97ca251f0 0x55e97ca2f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:56.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.187+0000 7f7942123b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e97ca25b10 0x55e97ca2f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:56.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.187+0000 7f7942123b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e97ca309f0 0x55e97ca2cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:56.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.187+0000 7f7942123b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e97c7fa6a0 con 0x55e97ca309f0 2024-02-02T15:27:56.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.187+0000 7f7942123b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e97ca22480 con 0x55e97ca25b10 2024-02-02T15:27:56.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.187+0000 7f7942123b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e97ca1b750 con 0x55e97ca251f0 2024-02-02T15:27:56.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.187+0000 7f7927e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e97ca309f0 0x55e97ca2cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:56.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.187+0000 7f7927400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e97ca251f0 0x55e97ca2f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:56.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.188+0000 7f7926a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e97ca25b10 0x55e97ca2f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:56.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.188+0000 7f7927400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e97ca251f0 0x55e97ca2f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:60998/0 (socket says 172.21.15.1:60998) 2024-02-02T15:27:56.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.188+0000 7f7927400700 1 -- 172.21.15.1:0/3127927057 learned_addr learned my addr 172.21.15.1:0/3127927057 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:56.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.188+0000 7f7927400700 1 -- 172.21.15.1:0/3127927057 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e97ca25b10 msgr2=0x55e97ca2f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:56.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.188+0000 7f7927400700 1 --2- 172.21.15.1:0/3127927057 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e97ca25b10 0x55e97ca2f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:56.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.188+0000 7f7927400700 1 -- 172.21.15.1:0/3127927057 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e97ca309f0 msgr2=0x55e97ca2cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:56.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.188+0000 7f7927400700 1 --2- 172.21.15.1:0/3127927057 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e97ca309f0 0x55e97ca2cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:56.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.188+0000 7f7927400700 1 -- 172.21.15.1:0/3127927057 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e97c93cea0 con 0x55e97ca251f0 2024-02-02T15:27:56.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.188+0000 7f7927e00700 1 --2- 172.21.15.1:0/3127927057 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e97ca309f0 0x55e97ca2cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:27:56.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.188+0000 7f7927400700 1 --2- 172.21.15.1:0/3127927057 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e97ca251f0 0x55e97ca2f130 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7918011b60 tx=0x7f7918014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4fea449dfad6083e server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:56.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.189+0000 7f791d600700 1 -- 172.21.15.1:0/3127927057 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7918024040 con 0x55e97ca251f0 2024-02-02T15:27:56.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.189+0000 7f791d600700 1 -- 172.21.15.1:0/3127927057 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f791802a7f0 con 0x55e97ca251f0 2024-02-02T15:27:56.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.189+0000 7f791d600700 1 -- 172.21.15.1:0/3127927057 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7918003950 con 0x55e97ca251f0 2024-02-02T15:27:56.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.190+0000 7f7942123b80 1 -- 172.21.15.1:0/3127927057 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e97ca251f0 msgr2=0x55e97ca2f130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:56.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.190+0000 7f7942123b80 1 --2- 172.21.15.1:0/3127927057 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e97ca251f0 0x55e97ca2f130 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7918011b60 tx=0x7f7918014000 comp rx=0 tx=0).stop 2024-02-02T15:27:56.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.190+0000 7f7942123b80 1 -- 172.21.15.1:0/3127927057 shutdown_connections 2024-02-02T15:27:56.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.190+0000 7f7942123b80 1 --2- 172.21.15.1:0/3127927057 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e97ca309f0 0x55e97ca2cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:56.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.190+0000 7f7942123b80 1 --2- 172.21.15.1:0/3127927057 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e97ca251f0 0x55e97ca2f130 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:56.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.190+0000 7f7942123b80 1 --2- 172.21.15.1:0/3127927057 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e97ca25b10 0x55e97ca2f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:56.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.190+0000 7f7942123b80 1 -- 172.21.15.1:0/3127927057 >> 172.21.15.1:0/3127927057 conn(0x55e97c984db0 msgr2=0x55e97ca26a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:56.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.190+0000 7f7942123b80 1 -- 172.21.15.1:0/3127927057 shutdown_connections 2024-02-02T15:27:56.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.190+0000 7f7942123b80 1 -- 172.21.15.1:0/3127927057 wait complete. 2024-02-02T15:27:56.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.190+0000 7f7942123b80 1 -- start start 2024-02-02T15:27:56.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.191+0000 7f7942123b80 1 Processor -- start 2024-02-02T15:27:56.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.191+0000 7f7942123b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e97ca251f0 0x55e97ca3bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:56.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.191+0000 7f7942123b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e97ca25b10 0x55e97ca3c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:56.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.192+0000 7f7942123b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e97ca309f0 0x55e97ca3a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:56.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.192+0000 7f7942123b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e97ca3a930 con 0x55e97ca25b10 2024-02-02T15:27:56.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.192+0000 7f7942123b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e97ca3aaa0 con 0x55e97ca251f0 2024-02-02T15:27:56.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.192+0000 7f7942123b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e97ca3ac60 con 0x55e97ca309f0 2024-02-02T15:27:56.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.192+0000 7f7942123b80 10 client.0 _ll_register_callbacks cb 0x55e97c914290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:27:56.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.192+0000 7f7927400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e97ca251f0 0x55e97ca3bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:56.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.192+0000 7f7927400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e97ca251f0 0x55e97ca3bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:37040/0 (socket says 172.21.15.1:37040) 2024-02-02T15:27:56.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.192+0000 7f7927400700 1 -- 172.21.15.1:0/2557133785 learned_addr learned my addr 172.21.15.1:0/2557133785 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:56.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.192+0000 7f7926a00700 1 --2- 172.21.15.1:0/2557133785 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e97ca25b10 0x55e97ca3c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:56.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.192+0000 7f7927e00700 1 --2- 172.21.15.1:0/2557133785 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e97ca309f0 0x55e97ca3a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:56.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.192+0000 7f7927400700 1 -- 172.21.15.1:0/2557133785 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e97ca309f0 msgr2=0x55e97ca3a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:56.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.192+0000 7f7927400700 1 --2- 172.21.15.1:0/2557133785 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e97ca309f0 0x55e97ca3a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:56.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.192+0000 7f7927400700 1 -- 172.21.15.1:0/2557133785 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e97ca25b10 msgr2=0x55e97ca3c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:56.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.193+0000 7f7927400700 1 --2- 172.21.15.1:0/2557133785 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e97ca25b10 0x55e97ca3c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:56.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.193+0000 7f7927400700 1 -- 172.21.15.1:0/2557133785 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7918017070 con 0x55e97ca251f0 2024-02-02T15:27:56.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.193+0000 7f7926a00700 1 --2- 172.21.15.1:0/2557133785 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e97ca25b10 0x55e97ca3c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:27:56.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.193+0000 7f7927400700 1 --2- 172.21.15.1:0/2557133785 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e97ca251f0 0x55e97ca3bdd0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7918009e90 tx=0x7f79180057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13bb56c4118db68c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:56.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.193+0000 7f7915600700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:27:56.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.193+0000 7f7915600700 1 -- 172.21.15.1:0/2557133785 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7918024050 con 0x55e97ca251f0 2024-02-02T15:27:56.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.193+0000 7f7915600700 1 -- 172.21.15.1:0/2557133785 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f791800e900 con 0x55e97ca251f0 2024-02-02T15:27:56.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.193+0000 7f7942123b80 1 -- 172.21.15.1:0/2557133785 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f791009a580 con 0x55e97ca251f0 2024-02-02T15:27:56.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.194+0000 7f7915600700 1 -- 172.21.15.1:0/2557133785 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f791800eac0 con 0x55e97ca251f0 2024-02-02T15:27:56.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.194+0000 7f7942123b80 10 client.25913 fetch_fsmap learned FSMap version 27 2024-02-02T15:27:56.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.194+0000 7f7942123b80 1 -- 172.21.15.1:0/2557133785 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e97c7fccd0 con 0x55e97ca251f0 2024-02-02T15:27:56.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.194+0000 7f7915600700 1 -- 172.21.15.1:0/2557133785 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f791009a580 con 0x55e97ca251f0 2024-02-02T15:27:56.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.194+0000 7f7915600700 1 -- 172.21.15.1:0/2557133785 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7918033450 con 0x55e97ca251f0 2024-02-02T15:27:56.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.194+0000 7f7942123b80 10 client.25913 fetch_fsmap finished waiting for FSMap version 27 2024-02-02T15:27:56.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.194+0000 7f7942123b80 10 client.25913 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-02T15:27:56.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.194+0000 7f7942123b80 20 client.25913 populate_metadata read hostname 'smithi001' 2024-02-02T15:27:56.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.194+0000 7f7942123b80 1 --2- 172.21.15.1:0/2557133785 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55e97ca92bc0 0x55e97ca94fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:56.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.195+0000 7f7942123b80 4 client.25913 mds_command: new command op to 24529 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:27:56.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.195+0000 7f7942123b80 1 -- 172.21.15.1:0/2557133785 --> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e97c935f50 con 0x55e97ca92bc0 2024-02-02T15:27:56.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.195+0000 7f7926a00700 1 --2- 172.21.15.1:0/2557133785 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55e97ca92bc0 0x55e97ca94fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:56.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.195+0000 7f7926a00700 1 --2- 172.21.15.1:0/2557133785 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55e97ca92bc0 0x55e97ca94fe0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=df97f719e563315b server_cookie=acf116625c279e8f in_seq=0 out_seq=0 2024-02-02T15:27:56.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.195+0000 7f7915600700 10 client.25913 ms_handle_connect on v2:172.21.15.142:6838/321675421 2024-02-02T15:27:56.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.196+0000 7f7915600700 1 -- 172.21.15.1:0/2557133785 <== mds.1 v2:172.21.15.142:6838/321675421 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e97c935f50 con 0x55e97ca92bc0 2024-02-02T15:27:56.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.196+0000 7f7915600700 10 client.25913 handle_command_reply: tid=0 2024-02-02T15:27:56.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.196+0000 7f7942123b80 1 client.25913 shutdown 2024-02-02T15:27:56.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.196+0000 7f7942123b80 1 -- 172.21.15.1:0/2557133785 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e97ca251f0 msgr2=0x55e97ca3bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:56.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.196+0000 7f7942123b80 1 --2- 172.21.15.1:0/2557133785 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e97ca251f0 0x55e97ca3bdd0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7918009e90 tx=0x7f79180057f0 comp rx=0 tx=0).stop 2024-02-02T15:27:56.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.196+0000 7f7942123b80 1 -- 172.21.15.1:0/2557133785 shutdown_connections 2024-02-02T15:27:56.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.196+0000 7f7942123b80 1 --2- 172.21.15.1:0/2557133785 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55e97ca92bc0 0x55e97ca94fe0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:56.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.196+0000 7f7942123b80 1 --2- 172.21.15.1:0/2557133785 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e97ca309f0 0x55e97ca3a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:56.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.196+0000 7f7942123b80 1 --2- 172.21.15.1:0/2557133785 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e97ca251f0 0x55e97ca3bdd0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:56.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.196+0000 7f7942123b80 1 --2- 172.21.15.1:0/2557133785 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e97ca25b10 0x55e97ca3c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:56.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.196+0000 7f7915600700 0 client.25913 ms_handle_reset on v2:172.21.15.142:6838/321675421 2024-02-02T15:27:56.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.196+0000 7f7942123b80 1 -- 172.21.15.1:0/2557133785 >> 172.21.15.1:0/2557133785 conn(0x55e97c984db0 msgr2=0x55e97ca20810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:56.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.196+0000 7f7942123b80 1 -- 172.21.15.1:0/2557133785 shutdown_connections 2024-02-02T15:27:56.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.196+0000 7f7942123b80 1 -- 172.21.15.1:0/2557133785 wait complete. 2024-02-02T15:27:56.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.197+0000 7f7942123b80 20 client.25913 trim_cache size 0 max 16384 2024-02-02T15:27:56.199 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:27:56.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.197+0000 7f792d000700 1 -- 172.21.15.1:0/1163866214 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f79080a0720 msgr2=0x7f79080a2be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:56.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.197+0000 7f792d000700 1 --2- 172.21.15.1:0/1163866214 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f79080a0720 0x7f79080a2be0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7918009fd0 tx=0x7f7918009260 comp rx=0 tx=0).stop 2024-02-02T15:27:56.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.197+0000 7f792d000700 1 -- 172.21.15.1:0/1163866214 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f79280689d0 msgr2=0x7f79281992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:56.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.197+0000 7f792d000700 1 --2- 172.21.15.1:0/1163866214 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f79280689d0 0x7f79281992f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f7910006010 tx=0x7f7910005720 comp rx=0 tx=0).stop 2024-02-02T15:27:56.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.198+0000 7f792d000700 1 -- 172.21.15.1:0/1163866214 shutdown_connections 2024-02-02T15:27:56.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.198+0000 7f792d000700 1 --2- 172.21.15.1:0/1163866214 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f79080a0720 0x7f79080a2be0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:56.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.198+0000 7f792d000700 1 --2- 172.21.15.1:0/1163866214 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7928106ab0 0x7f792819d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:56.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.198+0000 7f792d000700 1 --2- 172.21.15.1:0/1163866214 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7928068010 0x7f7928198db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:56.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.198+0000 7f792d000700 1 --2- 172.21.15.1:0/1163866214 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f79280689d0 0x7f79281992f0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:56.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.198+0000 7f792d000700 1 -- 172.21.15.1:0/1163866214 >> 172.21.15.1:0/1163866214 conn(0x7f7928076770 msgr2=0x7f792810bd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:56.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.198+0000 7f792d000700 1 -- 172.21.15.1:0/1163866214 shutdown_connections 2024-02-02T15:27:56.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.198+0000 7f792d000700 1 -- 172.21.15.1:0/1163866214 wait complete. 2024-02-02T15:27:56.219 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-02T15:27:56.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.492+0000 7f0fb3200700 1 -- 172.21.15.1:0/2160347945 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0fb4106ab0 msgr2=0x7f0fb4106e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:56.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.492+0000 7f0fb3200700 1 --2- 172.21.15.1:0/2160347945 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0fb4106ab0 0x7f0fb4106e90 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0fa40098f0 tx=0x7f0fa4009c00 comp rx=0 tx=0).stop 2024-02-02T15:27:56.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.492+0000 7f0fb3200700 1 -- 172.21.15.1:0/2160347945 shutdown_connections 2024-02-02T15:27:56.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.492+0000 7f0fb3200700 1 --2- 172.21.15.1:0/2160347945 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0fb40689d0 0x7f0fb410d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:56.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.492+0000 7f0fb3200700 1 --2- 172.21.15.1:0/2160347945 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0fb4106ab0 0x7f0fb4106e90 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:56.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.492+0000 7f0fb3200700 1 --2- 172.21.15.1:0/2160347945 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0fb4068010 0x7f0fb4068490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:56.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.492+0000 7f0fb3200700 1 -- 172.21.15.1:0/2160347945 >> 172.21.15.1:0/2160347945 conn(0x7f0fb4076770 msgr2=0x7f0fb4076b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:56.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.492+0000 7f0fb3200700 1 -- 172.21.15.1:0/2160347945 shutdown_connections 2024-02-02T15:27:56.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.492+0000 7f0fb3200700 1 -- 172.21.15.1:0/2160347945 wait complete. 2024-02-02T15:27:56.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.494+0000 7f0fb3200700 1 Processor -- start 2024-02-02T15:27:56.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.494+0000 7f0fb3200700 1 -- start start 2024-02-02T15:27:56.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.494+0000 7f0fb3200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0fb4068010 0x7f0fb4198ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:56.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.494+0000 7f0fb3200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0fb40689d0 0x7f0fb4199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:56.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.494+0000 7f0fb3200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0fb4106ab0 0x7f0fb419d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:56.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.494+0000 7f0fb3200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb419deb0 con 0x7f0fb4068010 2024-02-02T15:27:56.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.494+0000 7f0fb3200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb419e020 con 0x7f0fb4106ab0 2024-02-02T15:27:56.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.494+0000 7f0fb3200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb419e1e0 con 0x7f0fb40689d0 2024-02-02T15:27:56.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.495+0000 7f0fb1400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0fb40689d0 0x7f0fb4199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:56.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.495+0000 7f0fb1400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0fb40689d0 0x7f0fb4199410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:32800/0 (socket says 172.21.15.1:32800) 2024-02-02T15:27:56.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.495+0000 7f0fb1400700 1 -- 172.21.15.1:0/3652919274 learned_addr learned my addr 172.21.15.1:0/3652919274 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:56.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.495+0000 7f0fb2800700 1 --2- 172.21.15.1:0/3652919274 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0fb4106ab0 0x7f0fb419d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:56.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.495+0000 7f0fb1e00700 1 --2- 172.21.15.1:0/3652919274 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0fb4068010 0x7f0fb4198ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:56.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.495+0000 7f0fb1400700 1 -- 172.21.15.1:0/3652919274 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0fb4106ab0 msgr2=0x7f0fb419d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:56.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.495+0000 7f0fb1400700 1 --2- 172.21.15.1:0/3652919274 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0fb4106ab0 0x7f0fb419d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:56.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.495+0000 7f0fb1400700 1 -- 172.21.15.1:0/3652919274 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0fb4068010 msgr2=0x7f0fb4198ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:56.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.495+0000 7f0fb1400700 1 --2- 172.21.15.1:0/3652919274 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0fb4068010 0x7f0fb4198ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:56.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.495+0000 7f0fb1400700 1 -- 172.21.15.1:0/3652919274 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fa4009520 con 0x7f0fb40689d0 2024-02-02T15:27:56.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.496+0000 7f0fb1400700 1 --2- 172.21.15.1:0/3652919274 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0fb40689d0 0x7f0fb4199410 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c00e9a0 tx=0x7f0f9c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:56.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.496+0000 7f0faaa00700 1 -- 172.21.15.1:0/3652919274 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f9c009790 con 0x7f0fb40689d0 2024-02-02T15:27:56.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.496+0000 7f0faaa00700 1 -- 172.21.15.1:0/3652919274 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0f9c00cbd0 con 0x7f0fb40689d0 2024-02-02T15:27:56.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.496+0000 7f0fb3200700 1 -- 172.21.15.1:0/3652919274 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fb419e4c0 con 0x7f0fb40689d0 2024-02-02T15:27:56.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.496+0000 7f0faaa00700 1 -- 172.21.15.1:0/3652919274 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f9c018680 con 0x7f0fb40689d0 2024-02-02T15:27:56.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.496+0000 7f0fb3200700 1 -- 172.21.15.1:0/3652919274 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fb41a5d00 con 0x7f0fb40689d0 2024-02-02T15:27:56.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.501+0000 7f0faaa00700 1 -- 172.21.15.1:0/3652919274 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 25) v1 ==== 135245+0+0 (secure 0 0 0) 0x7f0f9c00cd40 con 0x7f0fb40689d0 2024-02-02T15:27:56.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.501+0000 7f0fcc84ab80 1 Processor -- start 2024-02-02T15:27:56.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.501+0000 7f0fcc84ab80 1 -- start start 2024-02-02T15:27:56.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.502+0000 7f0fcc84ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56198978e1f0 0x56198978e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:56.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.502+0000 7f0fcc84ab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56198978eb10 0x5619897994b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:56.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.502+0000 7f0fcc84ab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5619897999f0 0x56198979bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:56.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.502+0000 7f0fcc84ab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619897024c0 con 0x5619897999f0 2024-02-02T15:27:56.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.502+0000 7f0fcc84ab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619896e6aa0 con 0x56198978eb10 2024-02-02T15:27:56.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.502+0000 7f0fcc84ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619896e4460 con 0x56198978e1f0 2024-02-02T15:27:56.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.502+0000 7f0fb2800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56198978e1f0 0x56198978e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:56.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.502+0000 7f0fb1e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56198978eb10 0x5619897994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:56.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.502+0000 7f0fb1e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56198978eb10 0x5619897994b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:37074/0 (socket says 172.21.15.1:37074) 2024-02-02T15:27:56.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.502+0000 7f0fb1e00700 1 -- 172.21.15.1:0/3446933226 learned_addr learned my addr 172.21.15.1:0/3446933226 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:56.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.502+0000 7f0fb2800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56198978e1f0 0x56198978e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:32802/0 (socket says 172.21.15.1:32802) 2024-02-02T15:27:56.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.503+0000 7f0fb1400700 1 --2- 172.21.15.1:0/3446933226 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5619897999f0 0x56198979bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:56.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.503+0000 7f0fb1e00700 1 -- 172.21.15.1:0/3446933226 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56198978e1f0 msgr2=0x56198978e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:56.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.503+0000 7f0fb1e00700 1 --2- 172.21.15.1:0/3446933226 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56198978e1f0 0x56198978e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:56.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.503+0000 7f0fb1e00700 1 -- 172.21.15.1:0/3446933226 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5619897999f0 msgr2=0x56198979bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:56.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.503+0000 7f0fb1e00700 1 --2- 172.21.15.1:0/3446933226 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5619897999f0 0x56198979bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:56.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.503+0000 7f0faaa00700 1 --2- 172.21.15.1:0/3652919274 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f0f940a0720 0x7f0f940a2be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:56.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.503+0000 7f0fb1e00700 1 -- 172.21.15.1:0/3446933226 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619895613b0 con 0x56198978eb10 2024-02-02T15:27:56.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.503+0000 7f0fb1400700 1 --2- 172.21.15.1:0/3446933226 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5619897999f0 0x56198979bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:27:56.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.503+0000 7f0faaa00700 1 -- 172.21.15.1:0/3652919274 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f0f9c014070 con 0x7f0fb40689d0 2024-02-02T15:27:56.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.503+0000 7f0fb1e00700 1 --2- 172.21.15.1:0/3446933226 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56198978eb10 0x5619897994b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0fa4016000 tx=0x7f0fa4009480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfa3a61b2350e837 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:56.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.503+0000 7f0fa3400700 1 -- 172.21.15.1:0/3446933226 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa401c040 con 0x56198978eb10 2024-02-02T15:27:56.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.503+0000 7f0fa3400700 1 -- 172.21.15.1:0/3446933226 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0fa400ba20 con 0x56198978eb10 2024-02-02T15:27:56.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.503+0000 7f0fa3400700 1 -- 172.21.15.1:0/3446933226 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa4005620 con 0x56198978eb10 2024-02-02T15:27:56.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.504+0000 7f0fb2800700 1 --2- 172.21.15.1:0/3652919274 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f0f940a0720 0x7f0f940a2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:56.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.504+0000 7f0fcc84ab80 1 -- 172.21.15.1:0/3446933226 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56198978eb10 msgr2=0x5619897994b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:56.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.504+0000 7f0fcc84ab80 1 --2- 172.21.15.1:0/3446933226 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56198978eb10 0x5619897994b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0fa4016000 tx=0x7f0fa4009480 comp rx=0 tx=0).stop 2024-02-02T15:27:56.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.504+0000 7f0fb2800700 1 --2- 172.21.15.1:0/3652919274 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f0f940a0720 0x7f0f940a2be0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0fac009fd0 tx=0x7f0fac00b040 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:56.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.504+0000 7f0fcc84ab80 1 -- 172.21.15.1:0/3446933226 shutdown_connections 2024-02-02T15:27:56.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.504+0000 7f0fcc84ab80 1 --2- 172.21.15.1:0/3446933226 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5619897999f0 0x56198979bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:56.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.504+0000 7f0fcc84ab80 1 --2- 172.21.15.1:0/3446933226 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56198978e1f0 0x56198978e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:56.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.504+0000 7f0fcc84ab80 1 --2- 172.21.15.1:0/3446933226 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56198978eb10 0x5619897994b0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:56.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.504+0000 7f0fcc84ab80 1 -- 172.21.15.1:0/3446933226 >> 172.21.15.1:0/3446933226 conn(0x5619896eddb0 msgr2=0x56198978a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:56.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.505+0000 7f0fcc84ab80 1 -- 172.21.15.1:0/3446933226 shutdown_connections 2024-02-02T15:27:56.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.505+0000 7f0fcc84ab80 1 -- 172.21.15.1:0/3446933226 wait complete. 2024-02-02T15:27:56.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.505+0000 7f0fcc84ab80 1 -- start start 2024-02-02T15:27:56.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.505+0000 7f0fcc84ab80 1 Processor -- start 2024-02-02T15:27:56.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.506+0000 7f0fcc84ab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56198978e1f0 0x561989799370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:56.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.506+0000 7f0fcc84ab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56198978eb10 0x5619897965f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:56.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.506+0000 7f0fcc84ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5619897999f0 0x561989796c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:56.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.506+0000 7f0fcc84ab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619896dfd50 con 0x56198978e1f0 2024-02-02T15:27:56.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.506+0000 7f0fcc84ab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56198967bd80 con 0x56198978eb10 2024-02-02T15:27:56.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.506+0000 7f0fcc84ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56198967ba20 con 0x5619897999f0 2024-02-02T15:27:56.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.506+0000 7f0fcc84ab80 10 client.0 _ll_register_callbacks cb 0x56198967d290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:27:56.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.506+0000 7f0fb1e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56198978e1f0 0x561989799370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:56.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.506+0000 7f0fb1e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56198978e1f0 0x561989799370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33320/0 (socket says 172.21.15.1:33320) 2024-02-02T15:27:56.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.506+0000 7f0fb1e00700 1 -- 172.21.15.1:0/839994141 learned_addr learned my addr 172.21.15.1:0/839994141 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:56.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.507+0000 7f0fb1400700 1 --2- 172.21.15.1:0/839994141 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56198978eb10 0x5619897965f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:56.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.507+0000 7f0fb2800700 1 --2- 172.21.15.1:0/839994141 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5619897999f0 0x561989796c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:56.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.507+0000 7f0fb1e00700 1 -- 172.21.15.1:0/839994141 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5619897999f0 msgr2=0x561989796c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:56.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.507+0000 7f0fb1e00700 1 --2- 172.21.15.1:0/839994141 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5619897999f0 0x561989796c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:56.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.507+0000 7f0fb1e00700 1 -- 172.21.15.1:0/839994141 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56198978eb10 msgr2=0x5619897965f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:56.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.507+0000 7f0fb1e00700 1 --2- 172.21.15.1:0/839994141 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56198978eb10 0x5619897965f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:56.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.507+0000 7f0fb1e00700 1 -- 172.21.15.1:0/839994141 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fb410e270 con 0x56198978e1f0 2024-02-02T15:27:56.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.507+0000 7f0fb2800700 1 --2- 172.21.15.1:0/839994141 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5619897999f0 0x561989796c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:27:56.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.507+0000 7f0fb1e00700 1 --2- 172.21.15.1:0/839994141 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56198978e1f0 0x561989799370 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f0fa400b7c0 tx=0x7f0fa400f9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9403ab28a9a79693 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:56.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.507+0000 7f0f9be00700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:27:56.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.508+0000 7f0f9be00700 1 -- 172.21.15.1:0/839994141 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa401c070 con 0x56198978e1f0 2024-02-02T15:27:56.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.508+0000 7f0f9be00700 1 -- 172.21.15.1:0/839994141 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f0fa4005630 con 0x56198978e1f0 2024-02-02T15:27:56.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.508+0000 7f0fcc84ab80 1 -- 172.21.15.1:0/839994141 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561989782340 con 0x56198978e1f0 2024-02-02T15:27:56.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.508+0000 7f0f9be00700 1 -- 172.21.15.1:0/839994141 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0fa400bb90 con 0x56198978e1f0 2024-02-02T15:27:56.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.508+0000 7f0fcc84ab80 10 client.15876 fetch_fsmap learned FSMap version 27 2024-02-02T15:27:56.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.508+0000 7f0fcc84ab80 1 -- 172.21.15.1:0/839994141 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56198950db60 con 0x56198978e1f0 2024-02-02T15:27:56.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.509+0000 7f0f9be00700 1 -- 172.21.15.1:0/839994141 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561989782340 con 0x56198978e1f0 2024-02-02T15:27:56.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.509+0000 7f0f9be00700 1 -- 172.21.15.1:0/839994141 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0fa4015020 con 0x56198978e1f0 2024-02-02T15:27:56.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.509+0000 7f0fcc84ab80 10 client.15876 fetch_fsmap finished waiting for FSMap version 27 2024-02-02T15:27:56.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.509+0000 7f0fcc84ab80 10 client.15876 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:27:56.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.509+0000 7f0fcc84ab80 20 client.15876 populate_metadata read hostname 'smithi001' 2024-02-02T15:27:56.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.509+0000 7f0fcc84ab80 1 --2- 172.21.15.1:0/839994141 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5619897b1560 0x5619897b3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:56.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.509+0000 7f0fcc84ab80 4 client.15876 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:27:56.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.509+0000 7f0fcc84ab80 1 -- 172.21.15.1:0/839994141 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5619896d8dd0 con 0x5619897b1560 2024-02-02T15:27:56.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.510+0000 7f0fb1400700 1 --2- 172.21.15.1:0/839994141 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5619897b1560 0x5619897b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:56.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.511+0000 7f0f9be00700 10 client.15876 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:27:56.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.511+0000 7f0fb1400700 1 --2- 172.21.15.1:0/839994141 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5619897b1560 0x5619897b3950 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8b31ac04a79abea server_cookie=6be42774f025fdbc in_seq=0 out_seq=0 2024-02-02T15:27:56.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.512+0000 7f0f9be00700 1 -- 172.21.15.1:0/839994141 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5619896d8dd0 con 0x5619897b1560 2024-02-02T15:27:56.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.512+0000 7f0f9be00700 10 client.15876 handle_command_reply: tid=0 2024-02-02T15:27:56.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.512+0000 7f0fcc84ab80 1 client.15876 shutdown 2024-02-02T15:27:56.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.512+0000 7f0fcc84ab80 1 -- 172.21.15.1:0/839994141 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56198978e1f0 msgr2=0x561989799370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:56.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.512+0000 7f0fcc84ab80 1 --2- 172.21.15.1:0/839994141 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56198978e1f0 0x561989799370 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f0fa400b7c0 tx=0x7f0fa400f9f0 comp rx=0 tx=0).stop 2024-02-02T15:27:56.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.512+0000 7f0fcc84ab80 1 -- 172.21.15.1:0/839994141 shutdown_connections 2024-02-02T15:27:56.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.512+0000 7f0fcc84ab80 1 --2- 172.21.15.1:0/839994141 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5619897b1560 0x5619897b3950 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:56.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.512+0000 7f0fcc84ab80 1 --2- 172.21.15.1:0/839994141 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5619897999f0 0x561989796c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:56.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.512+0000 7f0fcc84ab80 1 --2- 172.21.15.1:0/839994141 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56198978e1f0 0x561989799370 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:56.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.512+0000 7f0fcc84ab80 1 --2- 172.21.15.1:0/839994141 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56198978eb10 0x5619897965f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:56.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.512+0000 7f0fcc84ab80 1 -- 172.21.15.1:0/839994141 >> 172.21.15.1:0/839994141 conn(0x5619896eddb0 msgr2=0x56198978a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:56.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.512+0000 7f0f9be00700 0 client.15876 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:27:56.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.513+0000 7f0fcc84ab80 1 -- 172.21.15.1:0/839994141 shutdown_connections 2024-02-02T15:27:56.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.513+0000 7f0fcc84ab80 1 -- 172.21.15.1:0/839994141 wait complete. 2024-02-02T15:27:56.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.513+0000 7f0fcc84ab80 20 client.15876 trim_cache size 0 max 16384 2024-02-02T15:27:56.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.526+0000 7f0fcc84ab80 1 Processor -- start 2024-02-02T15:27:56.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.527+0000 7f0fcc84ab80 1 -- start start 2024-02-02T15:27:56.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.527+0000 7f0fcc84ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56198978e1f0 0x56198979bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:56.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.527+0000 7f0fcc84ab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56198978eb10 0x561989788d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:56.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.527+0000 7f0fcc84ab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5619897999f0 0x561989798f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:56.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.527+0000 7f0fcc84ab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619895636a0 con 0x5619897999f0 2024-02-02T15:27:56.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.527+0000 7f0fcc84ab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619897892b0 con 0x56198978eb10 2024-02-02T15:27:56.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.527+0000 7f0fcc84ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56198978f200 con 0x56198978e1f0 2024-02-02T15:27:56.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.527+0000 7f0fb1e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56198978e1f0 0x56198979bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:56.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.527+0000 7f0fb1e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56198978e1f0 0x56198979bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:32816/0 (socket says 172.21.15.1:32816) 2024-02-02T15:27:56.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.527+0000 7f0fb1400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56198978eb10 0x561989788d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:56.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.527+0000 7f0fb1e00700 1 -- 172.21.15.1:0/2615990151 learned_addr learned my addr 172.21.15.1:0/2615990151 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:56.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.527+0000 7f0fb2800700 1 --2- 172.21.15.1:0/2615990151 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5619897999f0 0x561989798f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:56.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.528+0000 7f0fb1e00700 1 -- 172.21.15.1:0/2615990151 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56198978eb10 msgr2=0x561989788d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:56.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.528+0000 7f0fb1e00700 1 --2- 172.21.15.1:0/2615990151 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56198978eb10 0x561989788d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:56.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.528+0000 7f0fb1e00700 1 -- 172.21.15.1:0/2615990151 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5619897999f0 msgr2=0x561989798f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:56.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.528+0000 7f0fb1e00700 1 --2- 172.21.15.1:0/2615990151 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5619897999f0 0x561989798f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:56.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.528+0000 7f0fb1e00700 1 -- 172.21.15.1:0/2615990151 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5619896a5ea0 con 0x56198978e1f0 2024-02-02T15:27:56.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.528+0000 7f0fb1e00700 1 --2- 172.21.15.1:0/2615990151 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56198978e1f0 0x56198979bde0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0fa4009810 tx=0x7f0fa4017950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3611ecf9cd732907 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:56.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.528+0000 7f0fa3e00700 1 -- 172.21.15.1:0/2615990151 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa401c040 con 0x56198978e1f0 2024-02-02T15:27:56.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.529+0000 7f0fa3e00700 1 -- 172.21.15.1:0/2615990151 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0fa400bb90 con 0x56198978e1f0 2024-02-02T15:27:56.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.529+0000 7f0fa3e00700 1 -- 172.21.15.1:0/2615990151 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa4005690 con 0x56198978e1f0 2024-02-02T15:27:56.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.529+0000 7f0fcc84ab80 1 -- 172.21.15.1:0/2615990151 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56198978e1f0 msgr2=0x56198979bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:56.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.529+0000 7f0fcc84ab80 1 --2- 172.21.15.1:0/2615990151 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56198978e1f0 0x56198979bde0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0fa4009810 tx=0x7f0fa4017950 comp rx=0 tx=0).stop 2024-02-02T15:27:56.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.530+0000 7f0fcc84ab80 1 -- 172.21.15.1:0/2615990151 shutdown_connections 2024-02-02T15:27:56.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.530+0000 7f0fcc84ab80 1 --2- 172.21.15.1:0/2615990151 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5619897999f0 0x561989798f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:56.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.530+0000 7f0fcc84ab80 1 --2- 172.21.15.1:0/2615990151 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56198978e1f0 0x56198979bde0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:56.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.530+0000 7f0fcc84ab80 1 --2- 172.21.15.1:0/2615990151 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56198978eb10 0x561989788d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:56.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.530+0000 7f0fcc84ab80 1 -- 172.21.15.1:0/2615990151 >> 172.21.15.1:0/2615990151 conn(0x5619896eddb0 msgr2=0x5619897ade90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:56.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.530+0000 7f0fcc84ab80 1 -- 172.21.15.1:0/2615990151 shutdown_connections 2024-02-02T15:27:56.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.530+0000 7f0fcc84ab80 1 -- 172.21.15.1:0/2615990151 wait complete. 2024-02-02T15:27:56.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.530+0000 7f0fcc84ab80 1 -- start start 2024-02-02T15:27:56.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.531+0000 7f0fcc84ab80 1 Processor -- start 2024-02-02T15:27:56.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.531+0000 7f0fcc84ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56198978e1f0 0x5619897a4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:56.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.531+0000 7f0fcc84ab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56198978eb10 0x5619897a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:56.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.531+0000 7f0fcc84ab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5619897999f0 0x5619897a5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:56.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.531+0000 7f0fcc84ab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x561989801fc0 con 0x56198978eb10 2024-02-02T15:27:56.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.531+0000 7f0fcc84ab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x561989802100 con 0x5619897999f0 2024-02-02T15:27:56.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.531+0000 7f0fcc84ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5619898022c0 con 0x56198978e1f0 2024-02-02T15:27:56.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.531+0000 7f0fcc84ab80 10 client.0 _ll_register_callbacks cb 0x56198967d290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:27:56.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.532+0000 7f0fb1400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56198978eb10 0x5619897a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:56.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.532+0000 7f0fb1400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56198978eb10 0x5619897a51f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33334/0 (socket says 172.21.15.1:33334) 2024-02-02T15:27:56.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.532+0000 7f0fb1e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56198978e1f0 0x5619897a4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:56.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.532+0000 7f0fb1400700 1 -- 172.21.15.1:0/2897675721 learned_addr learned my addr 172.21.15.1:0/2897675721 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:56.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.532+0000 7f0fb2800700 1 --2- 172.21.15.1:0/2897675721 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5619897999f0 0x5619897a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:56.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.532+0000 7f0fb1e00700 1 -- 172.21.15.1:0/2897675721 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5619897999f0 msgr2=0x5619897a5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:56.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.532+0000 7f0fb1e00700 1 --2- 172.21.15.1:0/2897675721 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5619897999f0 0x5619897a5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:56.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.532+0000 7f0fb1e00700 1 -- 172.21.15.1:0/2897675721 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56198978eb10 msgr2=0x5619897a51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:56.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.532+0000 7f0fb1e00700 1 --2- 172.21.15.1:0/2897675721 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56198978eb10 0x5619897a51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:56.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.532+0000 7f0fb1e00700 1 -- 172.21.15.1:0/2897675721 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56198950db60 con 0x56198978e1f0 2024-02-02T15:27:56.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.532+0000 7f0fb2800700 1 --2- 172.21.15.1:0/2897675721 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5619897999f0 0x5619897a5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:27:56.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.533+0000 7f0fb1e00700 1 --2- 172.21.15.1:0/2897675721 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56198978e1f0 0x5619897a4cb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0fa40051f0 tx=0x7f0fa4017630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf87b32a6debef26 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:56.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.533+0000 7f0f9be00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:27:56.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.533+0000 7f0f9be00700 1 -- 172.21.15.1:0/2897675721 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa4017a20 con 0x56198978e1f0 2024-02-02T15:27:56.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.533+0000 7f0fcc84ab80 1 -- 172.21.15.1:0/2897675721 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0fa400ba20 con 0x56198978e1f0 2024-02-02T15:27:56.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.533+0000 7f0f9be00700 1 -- 172.21.15.1:0/2897675721 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f0fa4000e40 con 0x56198978e1f0 2024-02-02T15:27:56.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.533+0000 7f0f9be00700 1 -- 172.21.15.1:0/2897675721 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0fa4035350 con 0x56198978e1f0 2024-02-02T15:27:56.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.533+0000 7f0fcc84ab80 10 client.25930 fetch_fsmap learned FSMap version 27 2024-02-02T15:27:56.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.533+0000 7f0fcc84ab80 1 -- 172.21.15.1:0/2897675721 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x561989565cd0 con 0x56198978e1f0 2024-02-02T15:27:56.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.534+0000 7f0f9be00700 1 -- 172.21.15.1:0/2897675721 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0fa400ba20 con 0x56198978e1f0 2024-02-02T15:27:56.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.534+0000 7f0f9be00700 1 -- 172.21.15.1:0/2897675721 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0fa4015070 con 0x56198978e1f0 2024-02-02T15:27:56.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.534+0000 7f0fcc84ab80 10 client.25930 fetch_fsmap finished waiting for FSMap version 27 2024-02-02T15:27:56.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.534+0000 7f0fcc84ab80 10 client.25930 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:27:56.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.534+0000 7f0fcc84ab80 20 client.25930 populate_metadata read hostname 'smithi001' 2024-02-02T15:27:56.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.534+0000 7f0fcc84ab80 1 --2- 172.21.15.1:0/2897675721 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x561989803510 0x561989805900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:56.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.534+0000 7f0fcc84ab80 4 client.25930 mds_command: new command op to 24557 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:27:56.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.534+0000 7f0fcc84ab80 1 -- 172.21.15.1:0/2897675721 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56198969ef50 con 0x561989803510 2024-02-02T15:27:56.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.534+0000 7f0fb1400700 1 --2- 172.21.15.1:0/2897675721 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x561989803510 0x561989805900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:56.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.535+0000 7f0fb1400700 1 --2- 172.21.15.1:0/2897675721 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x561989803510 0x561989805900 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee76ce2f39dd6d53 server_cookie=16dc2dc6c71269a0 in_seq=0 out_seq=0 2024-02-02T15:27:56.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.535+0000 7f0f9be00700 10 client.25930 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:27:56.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.535+0000 7f0f9be00700 1 -- 172.21.15.1:0/2897675721 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56198969ef50 con 0x561989803510 2024-02-02T15:27:56.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.535+0000 7f0f9be00700 10 client.25930 handle_command_reply: tid=0 2024-02-02T15:27:56.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.535+0000 7f0fcc84ab80 1 client.25930 shutdown 2024-02-02T15:27:56.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.536+0000 7f0fcc84ab80 1 -- 172.21.15.1:0/2897675721 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56198978e1f0 msgr2=0x5619897a4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:56.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.536+0000 7f0fcc84ab80 1 --2- 172.21.15.1:0/2897675721 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56198978e1f0 0x5619897a4cb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0fa40051f0 tx=0x7f0fa4017630 comp rx=0 tx=0).stop 2024-02-02T15:27:56.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.536+0000 7f0fcc84ab80 1 -- 172.21.15.1:0/2897675721 shutdown_connections 2024-02-02T15:27:56.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.536+0000 7f0fcc84ab80 1 --2- 172.21.15.1:0/2897675721 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x561989803510 0x561989805900 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:56.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.536+0000 7f0fcc84ab80 1 --2- 172.21.15.1:0/2897675721 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5619897999f0 0x5619897a5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:56.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.536+0000 7f0fcc84ab80 1 --2- 172.21.15.1:0/2897675721 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56198978e1f0 0x5619897a4cb0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:56.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.536+0000 7f0f9be00700 0 client.25930 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:27:56.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.536+0000 7f0fcc84ab80 1 --2- 172.21.15.1:0/2897675721 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56198978eb10 0x5619897a51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:56.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.536+0000 7f0fcc84ab80 1 -- 172.21.15.1:0/2897675721 >> 172.21.15.1:0/2897675721 conn(0x5619896eddb0 msgr2=0x561989789810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:56.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.536+0000 7f0fcc84ab80 1 -- 172.21.15.1:0/2897675721 shutdown_connections 2024-02-02T15:27:56.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.536+0000 7f0fcc84ab80 1 -- 172.21.15.1:0/2897675721 wait complete. 2024-02-02T15:27:56.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.536+0000 7f0fcc84ab80 20 client.25930 trim_cache size 0 max 16384 2024-02-02T15:27:56.539 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:27:56.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.537+0000 7f0fb3200700 1 -- 172.21.15.1:0/3652919274 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f0f940a0720 msgr2=0x7f0f940a2be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:56.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.537+0000 7f0fb3200700 1 --2- 172.21.15.1:0/3652919274 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f0f940a0720 0x7f0f940a2be0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0fac009fd0 tx=0x7f0fac00b040 comp rx=0 tx=0).stop 2024-02-02T15:27:56.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.537+0000 7f0fb3200700 1 -- 172.21.15.1:0/3652919274 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0fb40689d0 msgr2=0x7f0fb4199410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:56.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.537+0000 7f0fb3200700 1 --2- 172.21.15.1:0/3652919274 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0fb40689d0 0x7f0fb4199410 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c00e9a0 tx=0x7f0f9c00ecb0 comp rx=0 tx=0).stop 2024-02-02T15:27:56.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.537+0000 7f0fb3200700 1 -- 172.21.15.1:0/3652919274 shutdown_connections 2024-02-02T15:27:56.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.537+0000 7f0fb3200700 1 --2- 172.21.15.1:0/3652919274 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f0f940a0720 0x7f0f940a2be0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:56.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.537+0000 7f0fb3200700 1 --2- 172.21.15.1:0/3652919274 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0fb4106ab0 0x7f0fb419d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:56.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.537+0000 7f0fb3200700 1 --2- 172.21.15.1:0/3652919274 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0fb4068010 0x7f0fb4198ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:56.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.537+0000 7f0fb3200700 1 --2- 172.21.15.1:0/3652919274 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0fb40689d0 0x7f0fb4199410 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:56.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.537+0000 7f0fb3200700 1 -- 172.21.15.1:0/3652919274 >> 172.21.15.1:0/3652919274 conn(0x7f0fb4076770 msgr2=0x7f0fb410bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:56.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.537+0000 7f0fb3200700 1 -- 172.21.15.1:0/3652919274 shutdown_connections 2024-02-02T15:27:56.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:56.537+0000 7f0fb3200700 1 -- 172.21.15.1:0/3652919274 wait complete. 2024-02-02T15:27:57.562 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-02T15:27:57.563 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-02T15:27:57.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:57 smithi071 ceph-mon[44707]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-02T15:27:57.636 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:57 smithi001 ceph-mon[35470]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-02T15:27:57.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:57 smithi142 ceph-mon[45728]: pgmap v31: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-02T15:27:57.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.832+0000 7f1aca800700 1 -- 172.21.15.1:0/639002390 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ac41038f0 msgr2=0x7f1ac4107930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:57.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.832+0000 7f1aca800700 1 --2- 172.21.15.1:0/639002390 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ac41038f0 0x7f1ac4107930 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f1aac009990 tx=0x7f1aac009ca0 comp rx=0 tx=0).stop 2024-02-02T15:27:57.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.832+0000 7f1aca800700 1 -- 172.21.15.1:0/639002390 shutdown_connections 2024-02-02T15:27:57.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.832+0000 7f1aca800700 1 --2- 172.21.15.1:0/639002390 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ac4108060 0x7f1ac410fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:57.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.832+0000 7f1aca800700 1 --2- 172.21.15.1:0/639002390 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ac4102f40 0x7f1ac4103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:57.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.832+0000 7f1aca800700 1 --2- 172.21.15.1:0/639002390 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ac41038f0 0x7f1ac4107930 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:57.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.832+0000 7f1aca800700 1 -- 172.21.15.1:0/639002390 >> 172.21.15.1:0/639002390 conn(0x7f1ac40fe7e0 msgr2=0x7f1ac4100c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:57.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.832+0000 7f1aca800700 1 -- 172.21.15.1:0/639002390 shutdown_connections 2024-02-02T15:27:57.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.832+0000 7f1aca800700 1 -- 172.21.15.1:0/639002390 wait complete. 2024-02-02T15:27:57.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.833+0000 7f1aca800700 1 Processor -- start 2024-02-02T15:27:57.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.833+0000 7f1aca800700 1 -- start start 2024-02-02T15:27:57.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.834+0000 7f1aca800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ac4102f40 0x7f1ac406b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:57.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.834+0000 7f1aca800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ac4108060 0x7f1ac406b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:57.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.834+0000 7f1aca800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ac41ab2f0 0x7f1ac41ad6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:57.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.834+0000 7f1aca800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac41adce0 con 0x7f1ac4102f40 2024-02-02T15:27:57.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.834+0000 7f1aca800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac41ade50 con 0x7f1ac41ab2f0 2024-02-02T15:27:57.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.834+0000 7f1aca800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac41ae010 con 0x7f1ac4108060 2024-02-02T15:27:57.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.834+0000 7f1ac9400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ac4102f40 0x7f1ac406b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:57.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.834+0000 7f1ac9400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ac4102f40 0x7f1ac406b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33352/0 (socket says 172.21.15.1:33352) 2024-02-02T15:27:57.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.834+0000 7f1ac9400700 1 -- 172.21.15.1:0/3915872281 learned_addr learned my addr 172.21.15.1:0/3915872281 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:57.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.834+0000 7f1ac8a00700 1 --2- 172.21.15.1:0/3915872281 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ac4108060 0x7f1ac406b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:57.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.834+0000 7f1ac9e00700 1 --2- 172.21.15.1:0/3915872281 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ac41ab2f0 0x7f1ac41ad6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:57.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.835+0000 7f1ac9400700 1 -- 172.21.15.1:0/3915872281 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ac4108060 msgr2=0x7f1ac406b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:57.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.835+0000 7f1ac9400700 1 --2- 172.21.15.1:0/3915872281 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ac4108060 0x7f1ac406b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:57.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.835+0000 7f1ac9400700 1 -- 172.21.15.1:0/3915872281 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ac41ab2f0 msgr2=0x7f1ac41ad6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:57.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.835+0000 7f1ac9400700 1 --2- 172.21.15.1:0/3915872281 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ac41ab2f0 0x7f1ac41ad6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:57.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.835+0000 7f1ac9400700 1 -- 172.21.15.1:0/3915872281 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aac0095f0 con 0x7f1ac4102f40 2024-02-02T15:27:57.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.835+0000 7f1ac9400700 1 --2- 172.21.15.1:0/3915872281 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ac4102f40 0x7f1ac406b2a0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f1ab400e9a0 tx=0x7f1ab400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:57.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.835+0000 7f1ac2000700 1 -- 172.21.15.1:0/3915872281 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab4009790 con 0x7f1ac4102f40 2024-02-02T15:27:57.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.835+0000 7f1ac2000700 1 -- 172.21.15.1:0/3915872281 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ab400cbd0 con 0x7f1ac4102f40 2024-02-02T15:27:57.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.835+0000 7f1ac2000700 1 -- 172.21.15.1:0/3915872281 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab4018680 con 0x7f1ac4102f40 2024-02-02T15:27:57.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.835+0000 7f1aca800700 1 -- 172.21.15.1:0/3915872281 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ac41ae2f0 con 0x7f1ac4102f40 2024-02-02T15:27:57.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.835+0000 7f1aca800700 1 -- 172.21.15.1:0/3915872281 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ac41ae7b0 con 0x7f1ac4102f40 2024-02-02T15:27:57.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.839+0000 7f1ac2000700 1 -- 172.21.15.1:0/3915872281 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135245+0+0 (secure 0 0 0) 0x7f1ab400cd40 con 0x7f1ac4102f40 2024-02-02T15:27:57.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.841+0000 7f1adf8c1b80 1 Processor -- start 2024-02-02T15:27:57.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.841+0000 7f1ac2000700 1 --2- 172.21.15.1:0/3915872281 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f1aa40a07a0 0x7f1aa40a2c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:57.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.842+0000 7f1ac2000700 1 -- 172.21.15.1:0/3915872281 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f1ab4014070 con 0x7f1ac4102f40 2024-02-02T15:27:57.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.842+0000 7f1ac9e00700 1 --2- 172.21.15.1:0/3915872281 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f1aa40a07a0 0x7f1aa40a2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:57.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.842+0000 7f1adf8c1b80 1 -- start start 2024-02-02T15:27:57.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.842+0000 7f1adf8c1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56000f5f0270 0x56000f5f0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:57.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.842+0000 7f1adf8c1b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56000f5f0b90 0x56000f5fb530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:57.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.843+0000 7f1adf8c1b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56000f5fba70 0x56000f5fde60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:57.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.843+0000 7f1adf8c1b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56000f564540 con 0x56000f5fba70 2024-02-02T15:27:57.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.843+0000 7f1adf8c1b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56000f548b20 con 0x56000f5f0b90 2024-02-02T15:27:57.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.843+0000 7f1adf8c1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56000f5464e0 con 0x56000f5f0270 2024-02-02T15:27:57.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.843+0000 7f1ac9400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56000f5f0270 0x56000f5f0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:57.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.843+0000 7f1ac8a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56000f5f0b90 0x56000f5fb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:57.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.843+0000 7f1ac9e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56000f5fba70 0x56000f5fde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:57.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.843+0000 7f1ac9400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56000f5f0270 0x56000f5f0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:32850/0 (socket says 172.21.15.1:32850) 2024-02-02T15:27:57.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.843+0000 7f1ac8a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56000f5f0b90 0x56000f5fb530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:37142/0 (socket says 172.21.15.1:37142) 2024-02-02T15:27:57.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.843+0000 7f1ac9400700 1 -- 172.21.15.1:0/3758484324 learned_addr learned my addr 172.21.15.1:0/3758484324 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:57.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.843+0000 7f1ac9e00700 1 --2- 172.21.15.1:0/3915872281 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f1aa40a07a0 0x7f1aa40a2c60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1ac4070cb0 tx=0x7f1abc00b040 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:57.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.843+0000 7f1ac9400700 1 -- 172.21.15.1:0/3758484324 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56000f5f0b90 msgr2=0x56000f5fb530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:57.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.843+0000 7f1ac9400700 1 --2- 172.21.15.1:0/3758484324 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56000f5f0b90 0x56000f5fb530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:57.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.843+0000 7f1ac9400700 1 -- 172.21.15.1:0/3758484324 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56000f5fba70 msgr2=0x56000f5fde60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:57.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.843+0000 7f1ac9400700 1 --2- 172.21.15.1:0/3758484324 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56000f5fba70 0x56000f5fde60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:57.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.843+0000 7f1ac9400700 1 -- 172.21.15.1:0/3758484324 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56000f3c3430 con 0x56000f5f0270 2024-02-02T15:27:57.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.844+0000 7f1ac9400700 1 --2- 172.21.15.1:0/3758484324 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56000f5f0270 0x56000f5f0650 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1ab400e970 tx=0x7f1ab4098fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7450f1f58b385105 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:57.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.844+0000 7f1aba600700 1 -- 172.21.15.1:0/3758484324 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab409d400 con 0x56000f5f0270 2024-02-02T15:27:57.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.844+0000 7f1aba600700 1 -- 172.21.15.1:0/3758484324 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ab40a1640 con 0x56000f5f0270 2024-02-02T15:27:57.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.844+0000 7f1aba600700 1 -- 172.21.15.1:0/3758484324 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab40cc1c0 con 0x56000f5f0270 2024-02-02T15:27:57.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.845+0000 7f1adf8c1b80 1 -- 172.21.15.1:0/3758484324 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56000f5f0270 msgr2=0x56000f5f0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:57.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.845+0000 7f1adf8c1b80 1 --2- 172.21.15.1:0/3758484324 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56000f5f0270 0x56000f5f0650 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1ab400e970 tx=0x7f1ab4098fb0 comp rx=0 tx=0).stop 2024-02-02T15:27:57.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.845+0000 7f1adf8c1b80 1 -- 172.21.15.1:0/3758484324 shutdown_connections 2024-02-02T15:27:57.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.845+0000 7f1adf8c1b80 1 --2- 172.21.15.1:0/3758484324 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56000f5fba70 0x56000f5fde60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:57.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.845+0000 7f1adf8c1b80 1 --2- 172.21.15.1:0/3758484324 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56000f5f0270 0x56000f5f0650 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:57.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.845+0000 7f1adf8c1b80 1 --2- 172.21.15.1:0/3758484324 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56000f5f0b90 0x56000f5fb530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:57.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.845+0000 7f1adf8c1b80 1 -- 172.21.15.1:0/3758484324 >> 172.21.15.1:0/3758484324 conn(0x56000f54fe30 msgr2=0x56000f5ec640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:57.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.845+0000 7f1adf8c1b80 1 -- 172.21.15.1:0/3758484324 shutdown_connections 2024-02-02T15:27:57.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.845+0000 7f1adf8c1b80 1 -- 172.21.15.1:0/3758484324 wait complete. 2024-02-02T15:27:57.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.846+0000 7f1adf8c1b80 1 -- start start 2024-02-02T15:27:57.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.846+0000 7f1adf8c1b80 1 Processor -- start 2024-02-02T15:27:57.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.846+0000 7f1adf8c1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56000f5f0270 0x56000f5fb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:57.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.846+0000 7f1adf8c1b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56000f5f0b90 0x56000f5f8770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:57.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.846+0000 7f1adf8c1b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56000f5fba70 0x56000f5f8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:57.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.846+0000 7f1adf8c1b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56000f541dd0 con 0x56000f5f0b90 2024-02-02T15:27:57.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.846+0000 7f1adf8c1b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56000f4dde00 con 0x56000f5fba70 2024-02-02T15:27:57.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.846+0000 7f1adf8c1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56000f4ddaa0 con 0x56000f5f0270 2024-02-02T15:27:57.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.847+0000 7f1adf8c1b80 10 client.0 _ll_register_callbacks cb 0x56000f4df310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:27:57.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.847+0000 7f1ac9400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56000f5f0270 0x56000f5fb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:57.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.847+0000 7f1ac9400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56000f5f0270 0x56000f5fb4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:32866/0 (socket says 172.21.15.1:32866) 2024-02-02T15:27:57.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.847+0000 7f1ac9400700 1 -- 172.21.15.1:0/3245527476 learned_addr learned my addr 172.21.15.1:0/3245527476 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:57.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.847+0000 7f1ac8a00700 1 --2- 172.21.15.1:0/3245527476 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56000f5f0b90 0x56000f5f8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:57.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.847+0000 7f1ac9e00700 1 --2- 172.21.15.1:0/3245527476 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56000f5fba70 0x56000f5f8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:57.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.847+0000 7f1ac9400700 1 -- 172.21.15.1:0/3245527476 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56000f5fba70 msgr2=0x56000f5f8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:57.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.847+0000 7f1ac9400700 1 --2- 172.21.15.1:0/3245527476 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56000f5fba70 0x56000f5f8e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:57.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.847+0000 7f1ac9400700 1 -- 172.21.15.1:0/3245527476 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56000f5f0b90 msgr2=0x56000f5f8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:57.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.847+0000 7f1ac9400700 1 --2- 172.21.15.1:0/3245527476 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56000f5f0b90 0x56000f5f8770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:57.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.847+0000 7f1ac9400700 1 -- 172.21.15.1:0/3245527476 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ac41ae7b0 con 0x56000f5f0270 2024-02-02T15:27:57.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.847+0000 7f1ac9e00700 1 --2- 172.21.15.1:0/3245527476 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56000f5fba70 0x56000f5f8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:27:57.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.848+0000 7f1ac9400700 1 --2- 172.21.15.1:0/3245527476 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56000f5f0270 0x56000f5fb4f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1ab400c5b0 tx=0x7f1ab4099690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2922943e9607e04 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:57.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.848+0000 7f1ab2a00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:27:57.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.848+0000 7f1ab2a00700 1 -- 172.21.15.1:0/3245527476 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab40af030 con 0x56000f5f0270 2024-02-02T15:27:57.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.848+0000 7f1ab2a00700 1 -- 172.21.15.1:0/3245527476 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f1ab40b0070 con 0x56000f5f0270 2024-02-02T15:27:57.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.848+0000 7f1adf8c1b80 1 -- 172.21.15.1:0/3245527476 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56000f5e43c0 con 0x56000f5f0270 2024-02-02T15:27:57.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.848+0000 7f1ab2a00700 1 -- 172.21.15.1:0/3245527476 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ab409d400 con 0x56000f5f0270 2024-02-02T15:27:57.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.848+0000 7f1adf8c1b80 10 client.25942 fetch_fsmap learned FSMap version 27 2024-02-02T15:27:57.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.848+0000 7f1adf8c1b80 1 -- 172.21.15.1:0/3245527476 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56000f36fbe0 con 0x56000f5f0270 2024-02-02T15:27:57.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.849+0000 7f1ab2a00700 1 -- 172.21.15.1:0/3245527476 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56000f5e43c0 con 0x56000f5f0270 2024-02-02T15:27:57.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.849+0000 7f1ab2a00700 1 -- 172.21.15.1:0/3245527476 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1ab4098ac0 con 0x56000f5f0270 2024-02-02T15:27:57.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.849+0000 7f1adf8c1b80 10 client.25942 fetch_fsmap finished waiting for FSMap version 27 2024-02-02T15:27:57.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.849+0000 7f1adf8c1b80 10 client.25942 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:27:57.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.849+0000 7f1adf8c1b80 20 client.25942 populate_metadata read hostname 'smithi001' 2024-02-02T15:27:57.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.849+0000 7f1adf8c1b80 1 --2- 172.21.15.1:0/3245527476 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56000f613990 0x56000f615d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:57.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.849+0000 7f1adf8c1b80 4 client.25942 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:27:57.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.849+0000 7f1adf8c1b80 1 -- 172.21.15.1:0/3245527476 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56000f53ae50 con 0x56000f613990 2024-02-02T15:27:57.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.850+0000 7f1ac8a00700 1 --2- 172.21.15.1:0/3245527476 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56000f613990 0x56000f615d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:57.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.850+0000 7f1ac8a00700 1 --2- 172.21.15.1:0/3245527476 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56000f613990 0x56000f615d80 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50994d53bee3af8e server_cookie=ea474812327fcda6 in_seq=0 out_seq=0 2024-02-02T15:27:57.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.850+0000 7f1ab2a00700 10 client.25942 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:27:57.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.851+0000 7f1ab2a00700 1 -- 172.21.15.1:0/3245527476 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x56000f53ae50 con 0x56000f613990 2024-02-02T15:27:57.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.851+0000 7f1ab2a00700 10 client.25942 handle_command_reply: tid=0 2024-02-02T15:27:57.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.851+0000 7f1adf8c1b80 1 client.25942 shutdown 2024-02-02T15:27:57.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.851+0000 7f1adf8c1b80 1 -- 172.21.15.1:0/3245527476 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56000f5f0270 msgr2=0x56000f5fb4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:57.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.851+0000 7f1adf8c1b80 1 --2- 172.21.15.1:0/3245527476 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56000f5f0270 0x56000f5fb4f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1ab400c5b0 tx=0x7f1ab4099690 comp rx=0 tx=0).stop 2024-02-02T15:27:57.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.852+0000 7f1adf8c1b80 1 -- 172.21.15.1:0/3245527476 shutdown_connections 2024-02-02T15:27:57.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.852+0000 7f1adf8c1b80 1 --2- 172.21.15.1:0/3245527476 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56000f613990 0x56000f615d80 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:57.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.852+0000 7f1adf8c1b80 1 --2- 172.21.15.1:0/3245527476 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56000f5fba70 0x56000f5f8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:57.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.852+0000 7f1adf8c1b80 1 --2- 172.21.15.1:0/3245527476 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56000f5f0270 0x56000f5fb4f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:57.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.852+0000 7f1ab2a00700 0 client.25942 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:27:57.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.852+0000 7f1adf8c1b80 1 --2- 172.21.15.1:0/3245527476 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56000f5f0b90 0x56000f5f8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:57.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.852+0000 7f1adf8c1b80 1 -- 172.21.15.1:0/3245527476 >> 172.21.15.1:0/3245527476 conn(0x56000f54fe30 msgr2=0x56000f5ec230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:57.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.852+0000 7f1adf8c1b80 1 -- 172.21.15.1:0/3245527476 shutdown_connections 2024-02-02T15:27:57.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.852+0000 7f1adf8c1b80 1 -- 172.21.15.1:0/3245527476 wait complete. 2024-02-02T15:27:57.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.852+0000 7f1adf8c1b80 20 client.25942 trim_cache size 0 max 16384 2024-02-02T15:27:57.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.875+0000 7f1adf8c1b80 1 Processor -- start 2024-02-02T15:27:57.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.875+0000 7f1adf8c1b80 1 -- start start 2024-02-02T15:27:57.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.876+0000 7f1adf8c1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56000f5f0270 0x56000f602560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:57.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.876+0000 7f1adf8c1b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56000f5f0b90 0x56000f5f2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:57.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.876+0000 7f1adf8c1b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56000f5fba70 0x56000f5f7f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:57.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.876+0000 7f1adf8c1b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56000f3c5720 con 0x56000f5f0b90 2024-02-02T15:27:57.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.876+0000 7f1adf8c1b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56000f5edca0 con 0x56000f5fba70 2024-02-02T15:27:57.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.876+0000 7f1adf8c1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56000f5f97b0 con 0x56000f5f0270 2024-02-02T15:27:57.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.876+0000 7f1ac8a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56000f5f0b90 0x56000f5f2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:57.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.876+0000 7f1ac9e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56000f5fba70 0x56000f5f7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:57.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.876+0000 7f1ac8a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56000f5f0b90 0x56000f5f2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33376/0 (socket says 172.21.15.1:33376) 2024-02-02T15:27:57.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.876+0000 7f1ac8a00700 1 -- 172.21.15.1:0/4094340183 learned_addr learned my addr 172.21.15.1:0/4094340183 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:57.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.876+0000 7f1ac9400700 1 --2- 172.21.15.1:0/4094340183 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56000f5f0270 0x56000f602560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:57.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.877+0000 7f1ac9e00700 1 -- 172.21.15.1:0/4094340183 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56000f5f0270 msgr2=0x56000f602560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:57.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.877+0000 7f1ac9e00700 1 --2- 172.21.15.1:0/4094340183 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56000f5f0270 0x56000f602560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:57.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.877+0000 7f1ac9e00700 1 -- 172.21.15.1:0/4094340183 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56000f5f0b90 msgr2=0x56000f5f2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:57.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.877+0000 7f1ac9e00700 1 --2- 172.21.15.1:0/4094340183 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56000f5f0b90 0x56000f5f2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:57.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.877+0000 7f1ac9e00700 1 -- 172.21.15.1:0/4094340183 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56000f507f20 con 0x56000f5fba70 2024-02-02T15:27:57.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.877+0000 7f1ac8a00700 1 --2- 172.21.15.1:0/4094340183 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56000f5f0b90 0x56000f5f2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:27:57.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.877+0000 7f1ac9e00700 1 --2- 172.21.15.1:0/4094340183 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56000f5fba70 0x56000f5f7f30 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1abc015850 tx=0x7f1abc013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8125268357a67c6 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:57.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.877+0000 7f1abb000700 1 -- 172.21.15.1:0/4094340183 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1abc00e050 con 0x56000f5fba70 2024-02-02T15:27:57.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.877+0000 7f1abb000700 1 -- 172.21.15.1:0/4094340183 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1abc00f040 con 0x56000f5fba70 2024-02-02T15:27:57.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.877+0000 7f1abb000700 1 -- 172.21.15.1:0/4094340183 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1abc016ad0 con 0x56000f5fba70 2024-02-02T15:27:57.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.878+0000 7f1adf8c1b80 1 -- 172.21.15.1:0/4094340183 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56000f5fba70 msgr2=0x56000f5f7f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:57.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.878+0000 7f1adf8c1b80 1 --2- 172.21.15.1:0/4094340183 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56000f5fba70 0x56000f5f7f30 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1abc015850 tx=0x7f1abc013000 comp rx=0 tx=0).stop 2024-02-02T15:27:57.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.878+0000 7f1adf8c1b80 1 -- 172.21.15.1:0/4094340183 shutdown_connections 2024-02-02T15:27:57.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.878+0000 7f1adf8c1b80 1 --2- 172.21.15.1:0/4094340183 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56000f5fba70 0x56000f5f7f30 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:57.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.878+0000 7f1adf8c1b80 1 --2- 172.21.15.1:0/4094340183 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56000f5f0270 0x56000f602560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:57.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.878+0000 7f1adf8c1b80 1 --2- 172.21.15.1:0/4094340183 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56000f5f0b90 0x56000f5f2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:57.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.878+0000 7f1adf8c1b80 1 -- 172.21.15.1:0/4094340183 >> 172.21.15.1:0/4094340183 conn(0x56000f54fe30 msgr2=0x56000f5f2380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:57.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.878+0000 7f1adf8c1b80 1 -- 172.21.15.1:0/4094340183 shutdown_connections 2024-02-02T15:27:57.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.878+0000 7f1adf8c1b80 1 -- 172.21.15.1:0/4094340183 wait complete. 2024-02-02T15:27:57.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.879+0000 7f1adf8c1b80 1 -- start start 2024-02-02T15:27:57.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.879+0000 7f1adf8c1b80 1 Processor -- start 2024-02-02T15:27:57.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.880+0000 7f1adf8c1b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56000f5f0270 0x56000f600cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:57.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.880+0000 7f1adf8c1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56000f5f0b90 0x56000f601230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:57.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.880+0000 7f1adf8c1b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56000f5fba70 0x56000f665460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:57.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.880+0000 7f1adf8c1b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56000f6019c0 con 0x56000f5f0270 2024-02-02T15:27:57.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.880+0000 7f1adf8c1b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56000f601b30 con 0x56000f5fba70 2024-02-02T15:27:57.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.880+0000 7f1adf8c1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56000f6659a0 con 0x56000f5f0b90 2024-02-02T15:27:57.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.880+0000 7f1adf8c1b80 10 client.0 _ll_register_callbacks cb 0x56000f4df310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:27:57.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.880+0000 7f1ac9400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56000f5f0270 0x56000f600cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:57.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.880+0000 7f1ac9e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56000f5fba70 0x56000f665460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:57.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.880+0000 7f1ac9e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56000f5fba70 0x56000f665460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:37172/0 (socket says 172.21.15.1:37172) 2024-02-02T15:27:57.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.880+0000 7f1ac9e00700 1 -- 172.21.15.1:0/1756011280 learned_addr learned my addr 172.21.15.1:0/1756011280 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:57.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.880+0000 7f1ac8a00700 1 --2- 172.21.15.1:0/1756011280 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56000f5f0b90 0x56000f601230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:57.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.881+0000 7f1ac9400700 1 -- 172.21.15.1:0/1756011280 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56000f5f0b90 msgr2=0x56000f601230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:57.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.881+0000 7f1ac9400700 1 --2- 172.21.15.1:0/1756011280 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56000f5f0b90 0x56000f601230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:57.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.881+0000 7f1ac9400700 1 -- 172.21.15.1:0/1756011280 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56000f5fba70 msgr2=0x56000f665460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:57.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.881+0000 7f1ac9400700 1 --2- 172.21.15.1:0/1756011280 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56000f5fba70 0x56000f665460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:57.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.881+0000 7f1ac9400700 1 -- 172.21.15.1:0/1756011280 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ac4110450 con 0x56000f5f0270 2024-02-02T15:27:57.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.881+0000 7f1ac8a00700 1 --2- 172.21.15.1:0/1756011280 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56000f5f0b90 0x56000f601230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:27:57.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.881+0000 7f1ac9400700 1 --2- 172.21.15.1:0/1756011280 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56000f5f0270 0x56000f600cf0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f1ab401e000 tx=0x7f1ab40c9990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea7a7cf7755e88b2 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:57.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.881+0000 7f1ab2a00700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:27:57.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.881+0000 7f1ab2a00700 1 -- 172.21.15.1:0/1756011280 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab40af020 con 0x56000f5f0270 2024-02-02T15:27:57.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.881+0000 7f1ab2a00700 1 -- 172.21.15.1:0/1756011280 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f1ab409d400 con 0x56000f5f0270 2024-02-02T15:27:57.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.881+0000 7f1adf8c1b80 1 -- 172.21.15.1:0/1756011280 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1ab40a1640 con 0x56000f5f0270 2024-02-02T15:27:57.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.881+0000 7f1ab2a00700 1 -- 172.21.15.1:0/1756011280 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ab40a1470 con 0x56000f5f0270 2024-02-02T15:27:57.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.882+0000 7f1adf8c1b80 10 client.15900 fetch_fsmap learned FSMap version 27 2024-02-02T15:27:57.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.882+0000 7f1adf8c1b80 1 -- 172.21.15.1:0/1756011280 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f1abc007e20 con 0x56000f5f0270 2024-02-02T15:27:57.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.882+0000 7f1ab2a00700 1 -- 172.21.15.1:0/1756011280 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ab40a1640 con 0x56000f5f0270 2024-02-02T15:27:57.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.882+0000 7f1ab2a00700 1 -- 172.21.15.1:0/1756011280 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1ab40b72c0 con 0x56000f5f0270 2024-02-02T15:27:57.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.882+0000 7f1adf8c1b80 10 client.15900 fetch_fsmap finished waiting for FSMap version 27 2024-02-02T15:27:57.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.882+0000 7f1adf8c1b80 10 client.15900 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:27:57.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.882+0000 7f1adf8c1b80 20 client.15900 populate_metadata read hostname 'smithi001' 2024-02-02T15:27:57.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.882+0000 7f1adf8c1b80 1 --2- 172.21.15.1:0/1756011280 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56000f666c30 0x56000f669020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:57.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.882+0000 7f1adf8c1b80 4 client.15900 mds_command: new command op to 24557 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-02T15:27:57.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.882+0000 7f1adf8c1b80 1 -- 172.21.15.1:0/1756011280 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x56000f500fd0 con 0x56000f666c30 2024-02-02T15:27:57.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.883+0000 7f1ac8a00700 1 --2- 172.21.15.1:0/1756011280 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56000f666c30 0x56000f669020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:57.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.883+0000 7f1ac8a00700 1 --2- 172.21.15.1:0/1756011280 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56000f666c30 0x56000f669020 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80b4283f64ae2ec5 server_cookie=44ae0a4269900a1f in_seq=0 out_seq=0 2024-02-02T15:27:57.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.883+0000 7f1ab2a00700 10 client.15900 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:27:57.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.884+0000 7f1ab2a00700 1 -- 172.21.15.1:0/1756011280 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x56000f500fd0 con 0x56000f666c30 2024-02-02T15:27:57.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.884+0000 7f1ab2a00700 10 client.15900 handle_command_reply: tid=0 2024-02-02T15:27:57.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.884+0000 7f1adf8c1b80 1 client.15900 shutdown 2024-02-02T15:27:57.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.885+0000 7f1adf8c1b80 1 -- 172.21.15.1:0/1756011280 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56000f5f0270 msgr2=0x56000f600cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:57.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.885+0000 7f1adf8c1b80 1 --2- 172.21.15.1:0/1756011280 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56000f5f0270 0x56000f600cf0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f1ab401e000 tx=0x7f1ab40c9990 comp rx=0 tx=0).stop 2024-02-02T15:27:57.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.885+0000 7f1adf8c1b80 1 -- 172.21.15.1:0/1756011280 shutdown_connections 2024-02-02T15:27:57.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.885+0000 7f1adf8c1b80 1 --2- 172.21.15.1:0/1756011280 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56000f666c30 0x56000f669020 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:57.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.885+0000 7f1adf8c1b80 1 --2- 172.21.15.1:0/1756011280 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56000f5fba70 0x56000f665460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:57.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.885+0000 7f1adf8c1b80 1 --2- 172.21.15.1:0/1756011280 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56000f5f0270 0x56000f600cf0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:57.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.885+0000 7f1ab2a00700 0 client.15900 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:27:57.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.885+0000 7f1adf8c1b80 1 --2- 172.21.15.1:0/1756011280 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56000f5f0b90 0x56000f601230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:57.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.885+0000 7f1adf8c1b80 1 -- 172.21.15.1:0/1756011280 >> 172.21.15.1:0/1756011280 conn(0x56000f54fe30 msgr2=0x56000f5f2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:57.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.885+0000 7f1adf8c1b80 1 -- 172.21.15.1:0/1756011280 shutdown_connections 2024-02-02T15:27:57.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.885+0000 7f1adf8c1b80 1 -- 172.21.15.1:0/1756011280 wait complete. 2024-02-02T15:27:57.887 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:27:57.887 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2024-02-02T15:27:57.887 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "3b84f3cd-f721-41d1-821e-797a6ec8b02f", 2024-02-02T15:27:57.888 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2024-02-02T15:27:57.888 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:27:57.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.886+0000 7f1adf8c1b80 20 client.15900 trim_cache size 0 max 16384 2024-02-02T15:27:57.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.886+0000 7f1aca800700 1 -- 172.21.15.1:0/3915872281 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f1aa40a07a0 msgr2=0x7f1aa40a2c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:57.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.886+0000 7f1aca800700 1 --2- 172.21.15.1:0/3915872281 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f1aa40a07a0 0x7f1aa40a2c60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1ac4070cb0 tx=0x7f1abc00b040 comp rx=0 tx=0).stop 2024-02-02T15:27:57.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.886+0000 7f1aca800700 1 -- 172.21.15.1:0/3915872281 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ac4102f40 msgr2=0x7f1ac406b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:57.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.886+0000 7f1aca800700 1 --2- 172.21.15.1:0/3915872281 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ac4102f40 0x7f1ac406b2a0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f1ab400e9a0 tx=0x7f1ab400ed60 comp rx=0 tx=0).stop 2024-02-02T15:27:57.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.887+0000 7f1aca800700 1 -- 172.21.15.1:0/3915872281 shutdown_connections 2024-02-02T15:27:57.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.887+0000 7f1aca800700 1 --2- 172.21.15.1:0/3915872281 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f1aa40a07a0 0x7f1aa40a2c60 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:57.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.887+0000 7f1aca800700 1 --2- 172.21.15.1:0/3915872281 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ac41ab2f0 0x7f1ac41ad6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:57.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.887+0000 7f1aca800700 1 --2- 172.21.15.1:0/3915872281 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ac4102f40 0x7f1ac406b2a0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:57.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.887+0000 7f1aca800700 1 --2- 172.21.15.1:0/3915872281 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ac4108060 0x7f1ac406b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:57.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.887+0000 7f1aca800700 1 -- 172.21.15.1:0/3915872281 >> 172.21.15.1:0/3915872281 conn(0x7f1ac40fe7e0 msgr2=0x7f1ac40ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:57.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.887+0000 7f1aca800700 1 -- 172.21.15.1:0/3915872281 shutdown_connections 2024-02-02T15:27:57.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:57.887+0000 7f1aca800700 1 -- 172.21.15.1:0/3915872281 wait complete. 2024-02-02T15:27:57.910 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-02T15:27:58.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.192+0000 7f5123e00700 1 -- 172.21.15.1:0/1207943047 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5124068010 msgr2=0x7f5124068490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:58.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.192+0000 7f5123e00700 1 --2- 172.21.15.1:0/1207943047 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5124068010 0x7f5124068490 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f510c009990 tx=0x7f510c009ca0 comp rx=0 tx=0).stop 2024-02-02T15:27:58.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.192+0000 7f5123e00700 1 -- 172.21.15.1:0/1207943047 shutdown_connections 2024-02-02T15:27:58.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.192+0000 7f5123e00700 1 --2- 172.21.15.1:0/1207943047 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51240689d0 0x7f512410d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:58.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.192+0000 7f5123e00700 1 --2- 172.21.15.1:0/1207943047 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5124106ab0 0x7f5124106e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:58.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.192+0000 7f5123e00700 1 --2- 172.21.15.1:0/1207943047 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5124068010 0x7f5124068490 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:58.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.192+0000 7f5123e00700 1 -- 172.21.15.1:0/1207943047 >> 172.21.15.1:0/1207943047 conn(0x7f5124076770 msgr2=0x7f5124076b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:58.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.192+0000 7f5123e00700 1 -- 172.21.15.1:0/1207943047 shutdown_connections 2024-02-02T15:27:58.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.192+0000 7f5123e00700 1 -- 172.21.15.1:0/1207943047 wait complete. 2024-02-02T15:27:58.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.193+0000 7f5123e00700 1 Processor -- start 2024-02-02T15:27:58.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.194+0000 7f5123e00700 1 -- start start 2024-02-02T15:27:58.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.194+0000 7f5123e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5124068010 0x7f5124198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:58.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.194+0000 7f5123e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51240689d0 0x7f51241992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:58.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.194+0000 7f5123e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5124106ab0 0x7f512419d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:58.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.194+0000 7f5123e00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f512419dd50 con 0x7f5124068010 2024-02-02T15:27:58.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.194+0000 7f5123e00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f512419dec0 con 0x7f5124106ab0 2024-02-02T15:27:58.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.194+0000 7f5123e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f512419e080 con 0x7f51240689d0 2024-02-02T15:27:58.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.195+0000 7f5122a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5124068010 0x7f5124198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:58.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.195+0000 7f5122a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5124068010 0x7f5124198d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33400/0 (socket says 172.21.15.1:33400) 2024-02-02T15:27:58.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.195+0000 7f5122a00700 1 -- 172.21.15.1:0/3625066717 learned_addr learned my addr 172.21.15.1:0/3625066717 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:58.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.195+0000 7f5122000700 1 --2- 172.21.15.1:0/3625066717 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51240689d0 0x7f51241992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:58.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.195+0000 7f5123400700 1 --2- 172.21.15.1:0/3625066717 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5124106ab0 0x7f512419d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:58.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.195+0000 7f5122a00700 1 -- 172.21.15.1:0/3625066717 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51240689d0 msgr2=0x7f51241992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:58.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.195+0000 7f5122a00700 1 --2- 172.21.15.1:0/3625066717 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51240689d0 0x7f51241992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:58.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.195+0000 7f5122a00700 1 -- 172.21.15.1:0/3625066717 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5124106ab0 msgr2=0x7f512419d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:58.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.195+0000 7f5122a00700 1 --2- 172.21.15.1:0/3625066717 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5124106ab0 0x7f512419d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:58.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.195+0000 7f5122a00700 1 -- 172.21.15.1:0/3625066717 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f511c009c40 con 0x7f5124068010 2024-02-02T15:27:58.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.195+0000 7f5123400700 1 --2- 172.21.15.1:0/3625066717 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5124106ab0 0x7f512419d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:27:58.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.195+0000 7f5122a00700 1 --2- 172.21.15.1:0/3625066717 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5124068010 0x7f5124198d70 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f511400d950 tx=0x7f511400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:58.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.196+0000 7f511b400700 1 -- 172.21.15.1:0/3625066717 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5114009940 con 0x7f5124068010 2024-02-02T15:27:58.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.196+0000 7f5123e00700 1 -- 172.21.15.1:0/3625066717 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f510c0095f0 con 0x7f5124068010 2024-02-02T15:27:58.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.196+0000 7f5123e00700 1 -- 172.21.15.1:0/3625066717 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f512419e550 con 0x7f5124068010 2024-02-02T15:27:58.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.196+0000 7f511b400700 1 -- 172.21.15.1:0/3625066717 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5114010460 con 0x7f5124068010 2024-02-02T15:27:58.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.196+0000 7f511b400700 1 -- 172.21.15.1:0/3625066717 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f511400f620 con 0x7f5124068010 2024-02-02T15:27:58.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.199+0000 7f511b400700 1 -- 172.21.15.1:0/3625066717 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 25) v1 ==== 135245+0+0 (secure 0 0 0) 0x7f5114009aa0 con 0x7f5124068010 2024-02-02T15:27:58.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.201+0000 7f511b400700 1 --2- 172.21.15.1:0/3625066717 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f51040a07a0 0x7f51040a2c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:58.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.201+0000 7f511b400700 1 -- 172.21.15.1:0/3625066717 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f51140cbd20 con 0x7f5124068010 2024-02-02T15:27:58.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.201+0000 7f5122000700 1 --2- 172.21.15.1:0/3625066717 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f51040a07a0 0x7f51040a2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:58.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.202+0000 7f5122000700 1 --2- 172.21.15.1:0/3625066717 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f51040a07a0 0x7f51040a2c60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f512419a380 tx=0x7f510c005fd0 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:58.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.202+0000 7f513d643b80 1 Processor -- start 2024-02-02T15:27:58.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.203+0000 7f513d643b80 1 -- start start 2024-02-02T15:27:58.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.203+0000 7f513d643b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c86aa41f0 0x560c86aa45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:58.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.203+0000 7f513d643b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560c86aa4b10 0x560c86aaf4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:58.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.203+0000 7f513d643b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560c86aaf9f0 0x560c86ab1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:58.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.203+0000 7f513d643b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c86a184c0 con 0x560c86aa4b10 2024-02-02T15:27:58.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.203+0000 7f513d643b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c869fcaa0 con 0x560c86aa41f0 2024-02-02T15:27:58.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.203+0000 7f513d643b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c869fa460 con 0x560c86aaf9f0 2024-02-02T15:27:58.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.204+0000 7f5122000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560c86aa4b10 0x560c86aaf4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:58.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.204+0000 7f5122a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c86aa41f0 0x560c86aa45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:58.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.204+0000 7f5122a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c86aa41f0 0x560c86aa45d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:37198/0 (socket says 172.21.15.1:37198) 2024-02-02T15:27:58.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.204+0000 7f5122a00700 1 -- 172.21.15.1:0/3109300064 learned_addr learned my addr 172.21.15.1:0/3109300064 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:58.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.204+0000 7f5123400700 1 --2- 172.21.15.1:0/3109300064 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560c86aaf9f0 0x560c86ab1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:58.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.204+0000 7f5122a00700 1 -- 172.21.15.1:0/3109300064 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560c86aaf9f0 msgr2=0x560c86ab1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:58.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.204+0000 7f5122a00700 1 --2- 172.21.15.1:0/3109300064 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560c86aaf9f0 0x560c86ab1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:58.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.204+0000 7f5122a00700 1 -- 172.21.15.1:0/3109300064 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560c86aa4b10 msgr2=0x560c86aaf4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:58.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.204+0000 7f5122a00700 1 --2- 172.21.15.1:0/3109300064 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560c86aa4b10 0x560c86aaf4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:58.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.204+0000 7f5122a00700 1 -- 172.21.15.1:0/3109300064 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c868773b0 con 0x560c86aa41f0 2024-02-02T15:27:58.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.204+0000 7f5122a00700 1 --2- 172.21.15.1:0/3109300064 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c86aa41f0 0x560c86aa45d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5114010fd0 tx=0x7f5114099420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f80a73528c0c5de9 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:58.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.205+0000 7f5113e00700 1 -- 172.21.15.1:0/3109300064 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f511409cb50 con 0x560c86aa41f0 2024-02-02T15:27:58.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.205+0000 7f5113e00700 1 -- 172.21.15.1:0/3109300064 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51140a0d90 con 0x560c86aa41f0 2024-02-02T15:27:58.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.205+0000 7f5113e00700 1 -- 172.21.15.1:0/3109300064 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5114098470 con 0x560c86aa41f0 2024-02-02T15:27:58.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.205+0000 7f513d643b80 1 -- 172.21.15.1:0/3109300064 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c86aa41f0 msgr2=0x560c86aa45d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:58.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.205+0000 7f513d643b80 1 --2- 172.21.15.1:0/3109300064 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c86aa41f0 0x560c86aa45d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f5114010fd0 tx=0x7f5114099420 comp rx=0 tx=0).stop 2024-02-02T15:27:58.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.206+0000 7f513d643b80 1 -- 172.21.15.1:0/3109300064 shutdown_connections 2024-02-02T15:27:58.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.206+0000 7f513d643b80 1 --2- 172.21.15.1:0/3109300064 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560c86aaf9f0 0x560c86ab1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:58.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.206+0000 7f513d643b80 1 --2- 172.21.15.1:0/3109300064 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c86aa41f0 0x560c86aa45d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:58.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.206+0000 7f513d643b80 1 --2- 172.21.15.1:0/3109300064 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560c86aa4b10 0x560c86aaf4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:58.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.206+0000 7f513d643b80 1 -- 172.21.15.1:0/3109300064 >> 172.21.15.1:0/3109300064 conn(0x560c86a03db0 msgr2=0x560c86aa05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:58.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.206+0000 7f513d643b80 1 -- 172.21.15.1:0/3109300064 shutdown_connections 2024-02-02T15:27:58.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.206+0000 7f513d643b80 1 -- 172.21.15.1:0/3109300064 wait complete. 2024-02-02T15:27:58.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.206+0000 7f513d643b80 1 -- start start 2024-02-02T15:27:58.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.207+0000 7f513d643b80 1 Processor -- start 2024-02-02T15:27:58.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.207+0000 7f513d643b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560c86aa41f0 0x560c86aaf3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:58.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.207+0000 7f513d643b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c86aa4b10 0x560c86aac660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:58.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.207+0000 7f513d643b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560c86aaf9f0 0x560c86aacc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:58.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.207+0000 7f513d643b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c869f5d50 con 0x560c86aaf9f0 2024-02-02T15:27:58.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.207+0000 7f513d643b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c86991d80 con 0x560c86aa4b10 2024-02-02T15:27:58.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.207+0000 7f513d643b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c86991a20 con 0x560c86aa41f0 2024-02-02T15:27:58.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.207+0000 7f513d643b80 10 client.0 _ll_register_callbacks cb 0x560c86993290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:27:58.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.208+0000 7f5122000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c86aa4b10 0x560c86aac660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:58.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.208+0000 7f5122000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c86aa4b10 0x560c86aac660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:37208/0 (socket says 172.21.15.1:37208) 2024-02-02T15:27:58.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.208+0000 7f5122000700 1 -- 172.21.15.1:0/1320519600 learned_addr learned my addr 172.21.15.1:0/1320519600 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:58.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.208+0000 7f5123400700 1 --2- 172.21.15.1:0/1320519600 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560c86aaf9f0 0x560c86aacc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:58.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.208+0000 7f5122a00700 1 --2- 172.21.15.1:0/1320519600 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560c86aa41f0 0x560c86aaf3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:58.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.208+0000 7f5122000700 1 -- 172.21.15.1:0/1320519600 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560c86aa41f0 msgr2=0x560c86aaf3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:58.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.208+0000 7f5122000700 1 --2- 172.21.15.1:0/1320519600 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560c86aa41f0 0x560c86aaf3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:58.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.208+0000 7f5122000700 1 -- 172.21.15.1:0/1320519600 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560c86aaf9f0 msgr2=0x560c86aacc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:58.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.208+0000 7f5122000700 1 --2- 172.21.15.1:0/1320519600 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560c86aaf9f0 0x560c86aacc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:58.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.208+0000 7f5122000700 1 -- 172.21.15.1:0/1320519600 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f512419e550 con 0x560c86aa4b10 2024-02-02T15:27:58.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.209+0000 7f5123400700 1 --2- 172.21.15.1:0/1320519600 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560c86aaf9f0 0x560c86aacc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:27:58.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.209+0000 7f5122000700 1 --2- 172.21.15.1:0/1320519600 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c86aa4b10 0x560c86aac660 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f510c009990 tx=0x7f510c020000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16041ccaf7ec8bea server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:58.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.209+0000 7f5110c00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:27:58.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.209+0000 7f5110c00700 1 -- 172.21.15.1:0/1320519600 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f510c02d040 con 0x560c86aa4b10 2024-02-02T15:27:58.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.209+0000 7f5110c00700 1 -- 172.21.15.1:0/1320519600 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f510c00f660 con 0x560c86aa4b10 2024-02-02T15:27:58.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.209+0000 7f513d643b80 1 -- 172.21.15.1:0/1320519600 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560c86a98340 con 0x560c86aa4b10 2024-02-02T15:27:58.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.210+0000 7f5110c00700 1 -- 172.21.15.1:0/1320519600 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f510c004e80 con 0x560c86aa4b10 2024-02-02T15:27:58.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.210+0000 7f513d643b80 10 client.25958 fetch_fsmap learned FSMap version 27 2024-02-02T15:27:58.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.210+0000 7f513d643b80 1 -- 172.21.15.1:0/1320519600 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x560c86823b60 con 0x560c86aa4b10 2024-02-02T15:27:58.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.210+0000 7f5110c00700 1 -- 172.21.15.1:0/1320519600 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560c86a98340 con 0x560c86aa4b10 2024-02-02T15:27:58.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.211+0000 7f5110c00700 1 -- 172.21.15.1:0/1320519600 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f510c03c7f0 con 0x560c86aa4b10 2024-02-02T15:27:58.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.211+0000 7f513d643b80 10 client.25958 fetch_fsmap finished waiting for FSMap version 27 2024-02-02T15:27:58.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.211+0000 7f513d643b80 10 client.25958 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:27:58.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.211+0000 7f513d643b80 20 client.25958 populate_metadata read hostname 'smithi001' 2024-02-02T15:27:58.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.211+0000 7f513d643b80 1 --2- 172.21.15.1:0/1320519600 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x560c86abf2f0 0x560c86ac1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:58.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.211+0000 7f513d643b80 4 client.25958 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:27:58.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.211+0000 7f513d643b80 1 -- 172.21.15.1:0/1320519600 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560c869eedd0 con 0x560c86abf2f0 2024-02-02T15:27:58.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.211+0000 7f5122a00700 1 --2- 172.21.15.1:0/1320519600 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x560c86abf2f0 0x560c86ac1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:58.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.212+0000 7f5122a00700 1 --2- 172.21.15.1:0/1320519600 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x560c86abf2f0 0x560c86ac1710 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8c29e68ff54500f server_cookie=4b19854a4496b905 in_seq=0 out_seq=0 2024-02-02T15:27:58.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.212+0000 7f5110c00700 10 client.25958 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:27:58.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.212+0000 7f5110c00700 1 -- 172.21.15.1:0/1320519600 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x560c869eedd0 con 0x560c86abf2f0 2024-02-02T15:27:58.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.212+0000 7f5110c00700 10 client.25958 handle_command_reply: tid=0 2024-02-02T15:27:58.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.212+0000 7f513d643b80 1 client.25958 shutdown 2024-02-02T15:27:58.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.213+0000 7f513d643b80 1 -- 172.21.15.1:0/1320519600 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c86aa4b10 msgr2=0x560c86aac660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:58.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.213+0000 7f513d643b80 1 --2- 172.21.15.1:0/1320519600 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c86aa4b10 0x560c86aac660 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f510c009990 tx=0x7f510c020000 comp rx=0 tx=0).stop 2024-02-02T15:27:58.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.213+0000 7f513d643b80 1 -- 172.21.15.1:0/1320519600 shutdown_connections 2024-02-02T15:27:58.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.213+0000 7f513d643b80 1 --2- 172.21.15.1:0/1320519600 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x560c86abf2f0 0x560c86ac1710 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:58.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.213+0000 7f5110c00700 0 client.25958 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:27:58.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.213+0000 7f513d643b80 1 --2- 172.21.15.1:0/1320519600 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560c86aaf9f0 0x560c86aacc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:58.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.213+0000 7f513d643b80 1 --2- 172.21.15.1:0/1320519600 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560c86aa41f0 0x560c86aaf3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:58.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.213+0000 7f513d643b80 1 --2- 172.21.15.1:0/1320519600 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c86aa4b10 0x560c86aac660 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:58.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.213+0000 7f513d643b80 1 -- 172.21.15.1:0/1320519600 >> 172.21.15.1:0/1320519600 conn(0x560c86a03db0 msgr2=0x560c86aa0150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:58.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.213+0000 7f513d643b80 1 -- 172.21.15.1:0/1320519600 shutdown_connections 2024-02-02T15:27:58.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.213+0000 7f513d643b80 1 -- 172.21.15.1:0/1320519600 wait complete. 2024-02-02T15:27:58.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.213+0000 7f513d643b80 20 client.25958 trim_cache size 0 max 16384 2024-02-02T15:27:58.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.238+0000 7f513d643b80 1 Processor -- start 2024-02-02T15:27:58.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.238+0000 7f513d643b80 1 -- start start 2024-02-02T15:27:58.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.238+0000 7f513d643b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560c86aa41f0 0x560c86aac610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:58.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.238+0000 7f513d643b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c86aa4b10 0x560c86aaef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:58.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.238+0000 7f513d643b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560c86aaf9f0 0x560c86aa98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:58.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.238+0000 7f513d643b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c868796a0 con 0x560c86aa41f0 2024-02-02T15:27:58.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.238+0000 7f513d643b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c86aa5c80 con 0x560c86aa4b10 2024-02-02T15:27:58.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.238+0000 7f513d643b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c86aad270 con 0x560c86aaf9f0 2024-02-02T15:27:58.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.238+0000 7f5122a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560c86aa41f0 0x560c86aac610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:58.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.238+0000 7f5122000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c86aa4b10 0x560c86aaef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:58.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.238+0000 7f5122a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560c86aa41f0 0x560c86aac610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33420/0 (socket says 172.21.15.1:33420) 2024-02-02T15:27:58.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.238+0000 7f5122000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c86aa4b10 0x560c86aaef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:37220/0 (socket says 172.21.15.1:37220) 2024-02-02T15:27:58.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.238+0000 7f5122a00700 1 -- 172.21.15.1:0/2523482823 learned_addr learned my addr 172.21.15.1:0/2523482823 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:58.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.238+0000 7f5123400700 1 --2- 172.21.15.1:0/2523482823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560c86aaf9f0 0x560c86aa98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:58.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.239+0000 7f5122000700 1 -- 172.21.15.1:0/2523482823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560c86aaf9f0 msgr2=0x560c86aa98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:58.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.239+0000 7f5122000700 1 --2- 172.21.15.1:0/2523482823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560c86aaf9f0 0x560c86aa98d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:58.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.239+0000 7f5122000700 1 -- 172.21.15.1:0/2523482823 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560c86aa41f0 msgr2=0x560c86aac610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:58.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.239+0000 7f5122000700 1 --2- 172.21.15.1:0/2523482823 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560c86aa41f0 0x560c86aac610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:58.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.239+0000 7f5122000700 1 -- 172.21.15.1:0/2523482823 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c869bbea0 con 0x560c86aa4b10 2024-02-02T15:27:58.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.239+0000 7f5123400700 1 --2- 172.21.15.1:0/2523482823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560c86aaf9f0 0x560c86aa98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:27:58.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.239+0000 7f5122a00700 1 --2- 172.21.15.1:0/2523482823 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560c86aa41f0 0x560c86aac610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:27:58.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.239+0000 7f5122000700 1 --2- 172.21.15.1:0/2523482823 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c86aa4b10 0x560c86aaef10 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f510c000c00 tx=0x7f510c009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb1361cddeaa72d0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:58.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.239+0000 7f5118c00700 1 -- 172.21.15.1:0/2523482823 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f510c02d070 con 0x560c86aa4b10 2024-02-02T15:27:58.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.240+0000 7f5118c00700 1 -- 172.21.15.1:0/2523482823 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f510c01e040 con 0x560c86aa4b10 2024-02-02T15:27:58.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.240+0000 7f5118c00700 1 -- 172.21.15.1:0/2523482823 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f510c032b30 con 0x560c86aa4b10 2024-02-02T15:27:58.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.240+0000 7f513d643b80 1 -- 172.21.15.1:0/2523482823 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c86aa4b10 msgr2=0x560c86aaef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:58.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.240+0000 7f513d643b80 1 --2- 172.21.15.1:0/2523482823 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c86aa4b10 0x560c86aaef10 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f510c000c00 tx=0x7f510c009310 comp rx=0 tx=0).stop 2024-02-02T15:27:58.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.240+0000 7f513d643b80 1 -- 172.21.15.1:0/2523482823 shutdown_connections 2024-02-02T15:27:58.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.240+0000 7f513d643b80 1 --2- 172.21.15.1:0/2523482823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560c86aaf9f0 0x560c86aa98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:58.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.240+0000 7f513d643b80 1 --2- 172.21.15.1:0/2523482823 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560c86aa41f0 0x560c86aac610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:58.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.240+0000 7f513d643b80 1 --2- 172.21.15.1:0/2523482823 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c86aa4b10 0x560c86aaef10 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:58.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.240+0000 7f513d643b80 1 -- 172.21.15.1:0/2523482823 >> 172.21.15.1:0/2523482823 conn(0x560c86a03db0 msgr2=0x560c86aabf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:58.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.240+0000 7f513d643b80 1 -- 172.21.15.1:0/2523482823 shutdown_connections 2024-02-02T15:27:58.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.241+0000 7f513d643b80 1 -- 172.21.15.1:0/2523482823 wait complete. 2024-02-02T15:27:58.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.241+0000 7f513d643b80 1 -- start start 2024-02-02T15:27:58.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.241+0000 7f513d643b80 1 Processor -- start 2024-02-02T15:27:58.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.242+0000 7f513d643b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c86aa41f0 0x560c86ab5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:58.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.242+0000 7f513d643b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560c86aa4b10 0x560c86ab60e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:58.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.242+0000 7f513d643b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560c86aaf9f0 0x560c86ab4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:58.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.242+0000 7f513d643b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c86ab4720 con 0x560c86aaf9f0 2024-02-02T15:27:58.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.243+0000 7f5122a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c86aa41f0 0x560c86ab5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:58.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.243+0000 7f513d643b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c86ab4890 con 0x560c86aa41f0 2024-02-02T15:27:58.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.243+0000 7f5122a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c86aa41f0 0x560c86ab5ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:37230/0 (socket says 172.21.15.1:37230) 2024-02-02T15:27:58.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.243+0000 7f5122a00700 1 -- 172.21.15.1:0/2579637954 learned_addr learned my addr 172.21.15.1:0/2579637954 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:27:58.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.243+0000 7f5123400700 1 --2- 172.21.15.1:0/2579637954 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560c86aaf9f0 0x560c86ab4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:58.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.243+0000 7f513d643b80 1 -- 172.21.15.1:0/2579637954 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c86ab4a50 con 0x560c86aa4b10 2024-02-02T15:27:58.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.243+0000 7f5122000700 1 --2- 172.21.15.1:0/2579637954 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560c86aa4b10 0x560c86ab60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:58.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.243+0000 7f513d643b80 10 client.0 _ll_register_callbacks cb 0x560c86993290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:27:58.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.243+0000 7f5123400700 1 -- 172.21.15.1:0/2579637954 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560c86aa4b10 msgr2=0x560c86ab60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:58.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.243+0000 7f5123400700 1 --2- 172.21.15.1:0/2579637954 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560c86aa4b10 0x560c86ab60e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:58.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.243+0000 7f5123400700 1 -- 172.21.15.1:0/2579637954 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c86aa41f0 msgr2=0x560c86ab5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:27:58.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.243+0000 7f5123400700 1 --2- 172.21.15.1:0/2579637954 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c86aa41f0 0x560c86ab5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:58.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.243+0000 7f5123400700 1 -- 172.21.15.1:0/2579637954 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560c868773b0 con 0x560c86aaf9f0 2024-02-02T15:27:58.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.243+0000 7f5122a00700 1 --2- 172.21.15.1:0/2579637954 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c86aa41f0 0x560c86ab5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:27:58.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.244+0000 7f5123400700 1 --2- 172.21.15.1:0/2579637954 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560c86aaf9f0 0x560c86ab4190 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x560c86a9a820 tx=0x7f511c0179a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f91a375c6c05ccc server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:27:58.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.244+0000 7f5110c00700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:27:58.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.244+0000 7f5110c00700 1 -- 172.21.15.1:0/2579637954 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f511c004db0 con 0x560c86aaf9f0 2024-02-02T15:27:58.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.244+0000 7f513d643b80 1 -- 172.21.15.1:0/2579637954 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f51140a0d90 con 0x560c86aaf9f0 2024-02-02T15:27:58.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.244+0000 7f5110c00700 1 -- 172.21.15.1:0/2579637954 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f511c01e070 con 0x560c86aaf9f0 2024-02-02T15:27:58.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.244+0000 7f5110c00700 1 -- 172.21.15.1:0/2579637954 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f511c003a00 con 0x560c86aaf9f0 2024-02-02T15:27:58.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.244+0000 7f513d643b80 10 client.15918 fetch_fsmap learned FSMap version 27 2024-02-02T15:27:58.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.244+0000 7f513d643b80 1 -- 172.21.15.1:0/2579637954 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f511c007690 con 0x560c86aaf9f0 2024-02-02T15:27:58.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.245+0000 7f5110c00700 1 -- 172.21.15.1:0/2579637954 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f51140a0d90 con 0x560c86aaf9f0 2024-02-02T15:27:58.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.245+0000 7f5110c00700 1 -- 172.21.15.1:0/2579637954 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f511c004020 con 0x560c86aaf9f0 2024-02-02T15:27:58.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.245+0000 7f513d643b80 10 client.15918 fetch_fsmap finished waiting for FSMap version 27 2024-02-02T15:27:58.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.245+0000 7f513d643b80 10 client.15918 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:27:58.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.245+0000 7f513d643b80 20 client.15918 populate_metadata read hostname 'smithi001' 2024-02-02T15:27:58.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.245+0000 7f513d643b80 1 --2- 172.21.15.1:0/2579637954 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x560c86b1a870 0x560c86b1cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:27:58.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.245+0000 7f513d643b80 4 client.15918 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2024-02-02T15:27:58.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.245+0000 7f513d643b80 1 -- 172.21.15.1:0/2579637954 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560c869b4f50 con 0x560c86b1a870 2024-02-02T15:27:58.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.246+0000 7f5122a00700 1 --2- 172.21.15.1:0/2579637954 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x560c86b1a870 0x560c86b1cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:27:58.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.246+0000 7f5122a00700 1 --2- 172.21.15.1:0/2579637954 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x560c86b1a870 0x560c86b1cc90 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2d7e70dcda80ac0 server_cookie=d043f544fa2a9b10 in_seq=0 out_seq=0 2024-02-02T15:27:58.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.246+0000 7f5110c00700 10 client.15918 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:27:58.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.246+0000 7f5110c00700 1 -- 172.21.15.1:0/2579637954 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x560c869b4f50 con 0x560c86b1a870 2024-02-02T15:27:58.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.246+0000 7f5110c00700 10 client.15918 handle_command_reply: tid=0 2024-02-02T15:27:58.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.246+0000 7f513d643b80 1 client.15918 shutdown 2024-02-02T15:27:58.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.247+0000 7f513d643b80 1 -- 172.21.15.1:0/2579637954 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560c86aaf9f0 msgr2=0x560c86ab4190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:58.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.247+0000 7f513d643b80 1 --2- 172.21.15.1:0/2579637954 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560c86aaf9f0 0x560c86ab4190 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x560c86a9a820 tx=0x7f511c0179a0 comp rx=0 tx=0).stop 2024-02-02T15:27:58.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.247+0000 7f513d643b80 1 -- 172.21.15.1:0/2579637954 shutdown_connections 2024-02-02T15:27:58.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.247+0000 7f513d643b80 1 --2- 172.21.15.1:0/2579637954 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x560c86b1a870 0x560c86b1cc90 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:58.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.247+0000 7f513d643b80 1 --2- 172.21.15.1:0/2579637954 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560c86aaf9f0 0x560c86ab4190 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:58.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.247+0000 7f513d643b80 1 --2- 172.21.15.1:0/2579637954 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c86aa41f0 0x560c86ab5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:58.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.247+0000 7f513d643b80 1 --2- 172.21.15.1:0/2579637954 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560c86aa4b10 0x560c86ab60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:58.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.247+0000 7f513d643b80 1 -- 172.21.15.1:0/2579637954 >> 172.21.15.1:0/2579637954 conn(0x560c86a03db0 msgr2=0x560c86aaae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:58.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.247+0000 7f5110c00700 0 client.15918 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:27:58.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.247+0000 7f513d643b80 1 -- 172.21.15.1:0/2579637954 shutdown_connections 2024-02-02T15:27:58.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.247+0000 7f513d643b80 1 -- 172.21.15.1:0/2579637954 wait complete. 2024-02-02T15:27:58.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.247+0000 7f513d643b80 20 client.15918 trim_cache size 0 max 16384 2024-02-02T15:27:58.250 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:27:58.250 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-02T15:27:58.250 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2024-02-02T15:27:58.250 INFO:teuthology.orchestra.run.smithi001.stdout: "3b84f3cd-f721-41d1-821e-797a6ec8b02f": { 2024-02-02T15:27:58.250 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2024-02-02T15:27:58.250 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "3b84f3cd-f721-41d1-821e-797a6ec8b02f", 2024-02-02T15:27:58.251 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,force" 2024-02-02T15:27:58.251 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:27:58.251 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:27:58.251 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:27:58.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.248+0000 7f5123e00700 1 -- 172.21.15.1:0/3625066717 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f51040a07a0 msgr2=0x7f51040a2c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:58.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.248+0000 7f5123e00700 1 --2- 172.21.15.1:0/3625066717 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f51040a07a0 0x7f51040a2c60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f512419a380 tx=0x7f510c005fd0 comp rx=0 tx=0).stop 2024-02-02T15:27:58.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.248+0000 7f5123e00700 1 -- 172.21.15.1:0/3625066717 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5124068010 msgr2=0x7f5124198d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:27:58.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.248+0000 7f5123e00700 1 --2- 172.21.15.1:0/3625066717 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5124068010 0x7f5124198d70 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f511400d950 tx=0x7f511400dc60 comp rx=0 tx=0).stop 2024-02-02T15:27:58.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.248+0000 7f5123e00700 1 -- 172.21.15.1:0/3625066717 shutdown_connections 2024-02-02T15:27:58.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.248+0000 7f5123e00700 1 --2- 172.21.15.1:0/3625066717 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f51040a07a0 0x7f51040a2c60 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:58.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.248+0000 7f5123e00700 1 --2- 172.21.15.1:0/3625066717 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5124106ab0 0x7f512419d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:58.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.248+0000 7f5123e00700 1 --2- 172.21.15.1:0/3625066717 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5124068010 0x7f5124198d70 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:58.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.248+0000 7f5123e00700 1 --2- 172.21.15.1:0/3625066717 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f51240689d0 0x7f51241992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:27:58.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.248+0000 7f5123e00700 1 -- 172.21.15.1:0/3625066717 >> 172.21.15.1:0/3625066717 conn(0x7f5124076770 msgr2=0x7f512410bd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:27:58.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.248+0000 7f5123e00700 1 -- 172.21.15.1:0/3625066717 shutdown_connections 2024-02-02T15:27:58.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:27:58.248+0000 7f5123e00700 1 -- 172.21.15.1:0/3625066717 wait complete. 2024-02-02T15:27:58.272 INFO:tasks.cephfs.filesystem:scrub status for tag:3b84f3cd-f721-41d1-821e-797a6ec8b02f - {'path': '/', 'tag': '3b84f3cd-f721-41d1-821e-797a6ec8b02f', 'options': 'recursive,force'} 2024-02-02T15:27:59.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:59 smithi071 ceph-mon[44707]: scrub queued for path: / 2024-02-02T15:27:59.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:59 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:27:59.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:59 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:27:59.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:59 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:27:59.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:27:59 smithi071 ceph-mon[44707]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-02T15:27:59.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:59 smithi142 ceph-mon[45728]: scrub queued for path: / 2024-02-02T15:27:59.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:59 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:27:59.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:59 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:27:59.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:59 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:27:59.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:27:59 smithi142 ceph-mon[45728]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-02T15:27:59.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:59 smithi001 ceph-mon[35470]: scrub queued for path: / 2024-02-02T15:27:59.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:59 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:27:59.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:59 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:27:59.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:59 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:27:59.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:27:59 smithi001 ceph-mon[35470]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-02T15:28:01.190 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:28:00 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:28:00.813+0000 7f196a800700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-02T15:28:01.190 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:28:00 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:28:00.813+0000 7f196a800700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-02T15:28:01.190 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:28:00 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:28:00.813+0000 7f196a800700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-02T15:28:01.190 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:28:00 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:28:00.813+0000 7f196a800700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-02T15:28:01.190 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:28:00 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:28:00.813+0000 7f196a800700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-02T15:28:01.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:28:01 smithi071 ceph-mon[44707]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-02T15:28:01.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:28:01 smithi142 ceph-mon[45728]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-02T15:28:01.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:28:01 smithi001 ceph-mon[35470]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-02T15:28:02.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:28:02 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:28:02.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:28:02 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:28:02.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:28:02 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:28:03.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:28:03 smithi071 ceph-mon[44707]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-02-02T15:28:03.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:28:03 smithi071 ceph-mon[44707]: mgrmap e26: x(active, since 62s), standbys: y, z 2024-02-02T15:28:03.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:28:03 smithi142 ceph-mon[45728]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-02-02T15:28:03.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:28:03 smithi142 ceph-mon[45728]: mgrmap e26: x(active, since 62s), standbys: y, z 2024-02-02T15:28:03.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:28:03 smithi001 ceph-mon[35470]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 25 op/s 2024-02-02T15:28:03.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:28:03 smithi001 ceph-mon[35470]: mgrmap e26: x(active, since 62s), standbys: y, z 2024-02-02T15:28:05.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:28:05 smithi071 ceph-mon[44707]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-02-02T15:28:05.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:28:05 smithi142 ceph-mon[45728]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-02-02T15:28:05.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:28:05 smithi001 ceph-mon[35470]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-02-02T15:28:07.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:28:07 smithi071 ceph-mon[44707]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-02-02T15:28:07.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:28:07 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:28:07.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:28:07 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:28:07.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:28:07 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:28:07.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:28:07 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:28:07.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:28:07 smithi142 ceph-mon[45728]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-02-02T15:28:07.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:28:07 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:28:07.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:28:07 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:28:07.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:28:07 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:28:07.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:28:07 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:28:07.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:28:07 smithi001 ceph-mon[35470]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-02-02T15:28:07.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:28:07 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:28:07.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:28:07 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:28:07.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:28:07 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:28:07.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:28:07 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:28:09.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:28:09 smithi071 ceph-mon[44707]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-02-02T15:28:09.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:28:09 smithi142 ceph-mon[45728]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-02-02T15:28:09.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:28:09 smithi001 ceph-mon[35470]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-02-02T15:28:11.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:28:11 smithi071 ceph-mon[44707]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-02-02T15:28:11.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:28:11 smithi142 ceph-mon[45728]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-02-02T15:28:11.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:28:11 smithi001 ceph-mon[35470]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-02-02T15:28:13.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:28:13 smithi071 ceph-mon[44707]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-02-02T15:28:13.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:28:13 smithi142 ceph-mon[45728]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-02-02T15:28:13.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:28:13 smithi001 ceph-mon[35470]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-02-02T15:28:15.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:28:15 smithi071 ceph-mon[44707]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 170 B/s wr, 8 op/s 2024-02-02T15:28:15.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:28:15 smithi142 ceph-mon[45728]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 170 B/s wr, 8 op/s 2024-02-02T15:28:15.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:28:15 smithi001 ceph-mon[35470]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 170 B/s wr, 8 op/s 2024-02-02T15:28:17.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:28:17 smithi071 ceph-mon[44707]: pgmap v41: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 170 B/s wr, 12 op/s 2024-02-02T15:28:17.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:28:17 smithi142 ceph-mon[45728]: pgmap v41: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 170 B/s wr, 12 op/s 2024-02-02T15:28:17.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:28:17 smithi001 ceph-mon[35470]: pgmap v41: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 170 B/s wr, 12 op/s 2024-02-02T15:28:19.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:28:19 smithi071 ceph-mon[44707]: pgmap v42: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-02-02T15:28:19.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:28:19 smithi142 ceph-mon[45728]: pgmap v42: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-02-02T15:28:19.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:28:19 smithi001 ceph-mon[35470]: pgmap v42: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-02-02T15:28:21.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:28:21 smithi142 ceph-mon[45728]: pgmap v43: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-02-02T15:28:21.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:28:21 smithi001 ceph-mon[35470]: pgmap v43: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-02-02T15:28:21.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:28:21 smithi071 ceph-mon[44707]: pgmap v43: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-02-02T15:28:23.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:28:23 smithi142 ceph-mon[45728]: pgmap v44: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-02-02T15:28:23.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:28:23 smithi001 ceph-mon[35470]: pgmap v44: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-02-02T15:28:23.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:28:23 smithi071 ceph-mon[44707]: pgmap v44: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-02-02T15:28:24.254 INFO:tasks.workunit.client.0.smithi001.stderr:Updating files: 62% (7641/12223) Updating files: 63% (7701/12223) Updating files: 64% (7823/12223) Updating files: 65% (7945/12223) Updating files: 66% (8068/12223) Updating files: 67% (8190/12223) Updating files: 68% (8312/12223) Updating files: 69% (8434/12223) Updating files: 70% (8557/12223) Updating files: 71% (8679/12223) Updating files: 72% (8801/12223) Updating files: 73% (8923/12223) Updating files: 74% (9046/12223) Updating files: 75% (9168/12223) Updating files: 76% (9290/12223) Updating files: 77% (9412/12223) Updating files: 78% (9534/12223) Updating files: 79% (9657/12223) Updating files: 80% (9779/12223) Updating files: 81% (9901/12223) Updating files: 82% (10023/12223) Updating files: 83% (10146/12223) Updating files: 84% (10268/12223) Updating files: 85% (10390/12223) Updating files: 86% (10512/12223) Updating files: 87% (10635/12223) Updating files: 88% (10757/12223) Updating files: 89% (10879/12223) Updating files: 90% (11001/12223) Updating files: 91% (11123/12223) Updating files: 92% (11246/12223) Updating files: 93% (11368/12223) Updating files: 94% (11490/12223) Updating files: 95% (11612/12223) Updating files: 96% (11735/12223) Updating files: 97% (11857/12223) Updating files: 98% (11979/12223) Updating files: 99% (12101/12223) Updating files: 100% (12223/12223) Updating files: 100% (12223/12223), done. 2024-02-02T15:28:25.078 INFO:tasks.workunit.client.0.smithi001.stderr:Note: switching to '0fb327883c124fd83155c94db0804f823b805a12'. 2024-02-02T15:28:25.078 INFO:tasks.workunit.client.0.smithi001.stderr: 2024-02-02T15:28:25.078 INFO:tasks.workunit.client.0.smithi001.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-02-02T15:28:25.079 INFO:tasks.workunit.client.0.smithi001.stderr:changes and commit them, and you can discard any commits you make in this 2024-02-02T15:28:25.079 INFO:tasks.workunit.client.0.smithi001.stderr:state without impacting any branches by switching back to a branch. 2024-02-02T15:28:25.079 INFO:tasks.workunit.client.0.smithi001.stderr: 2024-02-02T15:28:25.079 INFO:tasks.workunit.client.0.smithi001.stderr:If you want to create a new branch to retain commits you create, you may 2024-02-02T15:28:25.079 INFO:tasks.workunit.client.0.smithi001.stderr:do so (now or later) by using -c with the switch command. Example: 2024-02-02T15:28:25.079 INFO:tasks.workunit.client.0.smithi001.stderr: 2024-02-02T15:28:25.079 INFO:tasks.workunit.client.0.smithi001.stderr: git switch -c 2024-02-02T15:28:25.079 INFO:tasks.workunit.client.0.smithi001.stderr: 2024-02-02T15:28:25.079 INFO:tasks.workunit.client.0.smithi001.stderr:Or undo this operation with: 2024-02-02T15:28:25.079 INFO:tasks.workunit.client.0.smithi001.stderr: 2024-02-02T15:28:25.079 INFO:tasks.workunit.client.0.smithi001.stderr: git switch - 2024-02-02T15:28:25.079 INFO:tasks.workunit.client.0.smithi001.stderr: 2024-02-02T15:28:25.079 INFO:tasks.workunit.client.0.smithi001.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-02-02T15:28:25.080 INFO:tasks.workunit.client.0.smithi001.stderr: 2024-02-02T15:28:25.080 INFO:tasks.workunit.client.0.smithi001.stderr:HEAD is now at 0fb327883c1 Merge branch 'revert-54865-wip-63782-reef' of https://github.com/ceph/ceph into wip-vshankar-testing1-reef-2024-02-02-1152 2024-02-02T15:28:25.094 DEBUG:teuthology.orchestra.run.smithi001:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-02-02T15:28:25.143 INFO:tasks.workunit.client.0.smithi001.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-02-02T15:28:25.148 INFO:tasks.workunit.client.0.smithi001.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-02-02T15:28:25.148 INFO:tasks.workunit.client.0.smithi001.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-02-02T15:28:25.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:28:25 smithi142 ceph-mon[45728]: pgmap v45: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-02T15:28:25.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:28:25 smithi001 ceph-mon[35470]: pgmap v45: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-02T15:28:25.716 INFO:tasks.workunit.client.0.smithi001.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-02-02T15:28:25.821 INFO:tasks.workunit.client.0.smithi001.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-02-02T15:28:25.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:28:25 smithi071 ceph-mon[44707]: pgmap v45: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-02T15:28:25.886 INFO:tasks.workunit.client.0.smithi001.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-02-02T15:28:25.890 INFO:tasks.workunit.client.0.smithi001.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-02-02T15:28:25.890 INFO:tasks.workunit.client.0.smithi001.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-02-02T15:28:25.954 INFO:tasks.workunit.client.0.smithi001.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-02-02T15:28:25.961 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-02-02T15:28:25.961 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-02-02T15:28:26.002 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2024-02-02T15:28:26.004 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2024-02-02T15:28:26.004 DEBUG:teuthology.orchestra.run.smithi001:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=0fb327883c124fd83155c94db0804f823b805a12 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2024-02-02T15:28:26.110 INFO:tasks.workunit.client.0.smithi001.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-02-02T15:28:26.110 INFO:tasks.workunit.client.0.smithi001.stdout: 2024-02-02T15:28:26.110 INFO:tasks.workunit.client.0.smithi001.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-02-02T15:28:26.116 INFO:tasks.workunit.client.0.smithi001.stdout:0 of 1 processes prepared for launch 0 sec 2024-02-02T15:28:26.216 INFO:tasks.workunit.client.0.smithi001.stdout:1 of 1 processes prepared for launch 0 sec 2024-02-02T15:28:26.216 INFO:tasks.workunit.client.0.smithi001.stdout:releasing clients 2024-02-02T15:28:27.216 INFO:tasks.workunit.client.0.smithi001.stdout: 1 2602 29.01 MB/sec warmup 1 sec latency 16.036 ms 2024-02-02T15:28:27.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:28:27 smithi142 ceph-mon[45728]: pgmap v46: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-02T15:28:27.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:28:27 smithi001 ceph-mon[35470]: pgmap v46: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-02T15:28:27.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:28:27 smithi071 ceph-mon[44707]: pgmap v46: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-02-02T15:28:28.216 INFO:tasks.workunit.client.0.smithi001.stdout: 1 4810 21.10 MB/sec warmup 2 sec latency 7.242 ms 2024-02-02T15:28:28.273 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-02T15:28:28.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.510+0000 7f2e3ec00700 1 -- 172.21.15.1:0/2763876846 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e38101000 msgr2=0x7f2e381013e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:28:28.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.510+0000 7f2e3ec00700 1 --2- 172.21.15.1:0/2763876846 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e38101000 0x7f2e381013e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2e28009990 tx=0x7f2e28009ca0 comp rx=0 tx=0).stop 2024-02-02T15:28:28.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.510+0000 7f2e3ec00700 1 -- 172.21.15.1:0/2763876846 shutdown_connections 2024-02-02T15:28:28.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.510+0000 7f2e3ec00700 1 --2- 172.21.15.1:0/2763876846 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e38106120 0x7f2e3810dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:28.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.510+0000 7f2e3ec00700 1 --2- 172.21.15.1:0/2763876846 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e38101000 0x7f2e381013e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:28.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.510+0000 7f2e3ec00700 1 --2- 172.21.15.1:0/2763876846 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e381019b0 0x7f2e381059f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:28.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.510+0000 7f2e3ec00700 1 -- 172.21.15.1:0/2763876846 >> 172.21.15.1:0/2763876846 conn(0x7f2e380fc860 msgr2=0x7f2e380fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:28:28.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.510+0000 7f2e3ec00700 1 -- 172.21.15.1:0/2763876846 shutdown_connections 2024-02-02T15:28:28.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.510+0000 7f2e3ec00700 1 -- 172.21.15.1:0/2763876846 wait complete. 2024-02-02T15:28:28.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.511+0000 7f2e3ec00700 1 Processor -- start 2024-02-02T15:28:28.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.511+0000 7f2e3ec00700 1 -- start start 2024-02-02T15:28:28.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.512+0000 7f2e3ec00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e38101000 0x7f2e38073040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:28.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.512+0000 7f2e3ec00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e38106120 0x7f2e38073580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:28.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.512+0000 7f2e3ec00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e38073ac0 0x7f2e381a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:28.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.512+0000 7f2e3ec00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e381a5760 con 0x7f2e38101000 2024-02-02T15:28:28.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.512+0000 7f2e3ec00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e381a58d0 con 0x7f2e38073ac0 2024-02-02T15:28:28.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.512+0000 7f2e3ec00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e381a5a90 con 0x7f2e38106120 2024-02-02T15:28:28.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.512+0000 7f2e37400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e38106120 0x7f2e38073580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:28.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.512+0000 7f2e3cc00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e38073ac0 0x7f2e381a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:28.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.512+0000 7f2e37400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e38106120 0x7f2e38073580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:47488/0 (socket says 172.21.15.1:47488) 2024-02-02T15:28:28.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.512+0000 7f2e37400700 1 -- 172.21.15.1:0/2519361272 learned_addr learned my addr 172.21.15.1:0/2519361272 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:28:28.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.512+0000 7f2e37e00700 1 --2- 172.21.15.1:0/2519361272 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e38101000 0x7f2e38073040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:28.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.513+0000 7f2e37400700 1 -- 172.21.15.1:0/2519361272 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e38073ac0 msgr2=0x7f2e381a50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:28:28.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.513+0000 7f2e37400700 1 --2- 172.21.15.1:0/2519361272 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e38073ac0 0x7f2e381a50d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:28.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.513+0000 7f2e37400700 1 -- 172.21.15.1:0/2519361272 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e38101000 msgr2=0x7f2e38073040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:28:28.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.513+0000 7f2e37400700 1 --2- 172.21.15.1:0/2519361272 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e38101000 0x7f2e38073040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:28.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.513+0000 7f2e37400700 1 -- 172.21.15.1:0/2519361272 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e20009520 con 0x7f2e38106120 2024-02-02T15:28:28.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.513+0000 7f2e37400700 1 --2- 172.21.15.1:0/2519361272 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e38106120 0x7f2e38073580 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2e2000ea30 tx=0x7f2e2000ed40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:28:28.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.513+0000 7f2e34c00700 1 -- 172.21.15.1:0/2519361272 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e2000cbd0 con 0x7f2e38106120 2024-02-02T15:28:28.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.513+0000 7f2e3ec00700 1 -- 172.21.15.1:0/2519361272 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e280095f0 con 0x7f2e38106120 2024-02-02T15:28:28.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.513+0000 7f2e34c00700 1 -- 172.21.15.1:0/2519361272 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e2000eed0 con 0x7f2e38106120 2024-02-02T15:28:28.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.513+0000 7f2e34c00700 1 -- 172.21.15.1:0/2519361272 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e20003d20 con 0x7f2e38106120 2024-02-02T15:28:28.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.513+0000 7f2e3ec00700 1 -- 172.21.15.1:0/2519361272 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e381a6140 con 0x7f2e38106120 2024-02-02T15:28:28.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.516+0000 7f2e34c00700 1 -- 172.21.15.1:0/2519361272 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 26) v1 ==== 135302+0+0 (secure 0 0 0) 0x7f2e20003ee0 con 0x7f2e38106120 2024-02-02T15:28:28.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.518+0000 7f2e527a3b80 1 Processor -- start 2024-02-02T15:28:28.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.518+0000 7f2e527a3b80 1 -- start start 2024-02-02T15:28:28.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.518+0000 7f2e34c00700 1 --2- 172.21.15.1:0/2519361272 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f2e180a07b0 0x7f2e180a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:28.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.518+0000 7f2e34c00700 1 -- 172.21.15.1:0/2519361272 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f2e20014070 con 0x7f2e38106120 2024-02-02T15:28:28.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.518+0000 7f2e3cc00700 1 --2- 172.21.15.1:0/2519361272 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f2e180a07b0 0x7f2e180a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:28.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.518+0000 7f2e527a3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56487dc50010 0x56487dc503f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:28.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.518+0000 7f2e527a3b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56487dc50930 0x56487dc5b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:28.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.518+0000 7f2e527a3b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56487dc5b7e0 0x56487dc5dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:28.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.518+0000 7f2e527a3b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56487dbc44c0 con 0x56487dc50930 2024-02-02T15:28:28.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.518+0000 7f2e527a3b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56487dba8aa0 con 0x56487dc5b7e0 2024-02-02T15:28:28.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.518+0000 7f2e527a3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56487dba6460 con 0x56487dc50010 2024-02-02T15:28:28.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.519+0000 7f2e3cc00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56487dc5b7e0 0x56487dc5dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:28.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.519+0000 7f2e37e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56487dc50010 0x56487dc503f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:28.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.519+0000 7f2e3cc00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56487dc5b7e0 0x56487dc5dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:51146/0 (socket says 172.21.15.1:51146) 2024-02-02T15:28:28.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.519+0000 7f2e3cc00700 1 -- 172.21.15.1:0/1137379268 learned_addr learned my addr 172.21.15.1:0/1137379268 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:28:28.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.519+0000 7f2e37e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56487dc50010 0x56487dc503f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:47492/0 (socket says 172.21.15.1:47492) 2024-02-02T15:28:28.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.519+0000 7f2e37400700 1 --2- 172.21.15.1:0/1137379268 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56487dc50930 0x56487dc5b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:28.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.519+0000 7f2e3cc00700 1 --2- 172.21.15.1:0/2519361272 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f2e180a07b0 0x7f2e180a2c70 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2e3000f930 tx=0x7f2e3000fcf0 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:28:28.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.519+0000 7f2e3cc00700 1 -- 172.21.15.1:0/1137379268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56487dc50010 msgr2=0x56487dc503f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:28:28.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.519+0000 7f2e3cc00700 1 --2- 172.21.15.1:0/1137379268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56487dc50010 0x56487dc503f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:28.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.519+0000 7f2e3cc00700 1 -- 172.21.15.1:0/1137379268 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56487dc50930 msgr2=0x56487dc5b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:28:28.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.519+0000 7f2e3cc00700 1 --2- 172.21.15.1:0/1137379268 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56487dc50930 0x56487dc5b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:28.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.519+0000 7f2e3cc00700 1 -- 172.21.15.1:0/1137379268 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56487da233b0 con 0x56487dc5b7e0 2024-02-02T15:28:28.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.520+0000 7f2e3cc00700 1 --2- 172.21.15.1:0/1137379268 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56487dc5b7e0 0x56487dc5dbd0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2e30009120 tx=0x7f2e3001d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a24cd7e53cb34add server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:28:28.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.520+0000 7f2e2d200700 1 -- 172.21.15.1:0/1137379268 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e3000e040 con 0x56487dc5b7e0 2024-02-02T15:28:28.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.520+0000 7f2e2d200700 1 -- 172.21.15.1:0/1137379268 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e3000f040 con 0x56487dc5b7e0 2024-02-02T15:28:28.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.520+0000 7f2e2d200700 1 -- 172.21.15.1:0/1137379268 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e3001a440 con 0x56487dc5b7e0 2024-02-02T15:28:28.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.521+0000 7f2e527a3b80 1 -- 172.21.15.1:0/1137379268 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56487dc5b7e0 msgr2=0x56487dc5dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:28:28.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.521+0000 7f2e527a3b80 1 --2- 172.21.15.1:0/1137379268 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56487dc5b7e0 0x56487dc5dbd0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2e30009120 tx=0x7f2e3001d000 comp rx=0 tx=0).stop 2024-02-02T15:28:28.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.521+0000 7f2e527a3b80 1 -- 172.21.15.1:0/1137379268 shutdown_connections 2024-02-02T15:28:28.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.521+0000 7f2e527a3b80 1 --2- 172.21.15.1:0/1137379268 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56487dc5b7e0 0x56487dc5dbd0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:28.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.521+0000 7f2e527a3b80 1 --2- 172.21.15.1:0/1137379268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56487dc50010 0x56487dc503f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:28.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.521+0000 7f2e527a3b80 1 --2- 172.21.15.1:0/1137379268 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56487dc50930 0x56487dc5b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:28.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.521+0000 7f2e527a3b80 1 -- 172.21.15.1:0/1137379268 >> 172.21.15.1:0/1137379268 conn(0x56487dbafdb0 msgr2=0x56487dc4c3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:28:28.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.521+0000 7f2e527a3b80 1 -- 172.21.15.1:0/1137379268 shutdown_connections 2024-02-02T15:28:28.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.521+0000 7f2e527a3b80 1 -- 172.21.15.1:0/1137379268 wait complete. 2024-02-02T15:28:28.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.521+0000 7f2e527a3b80 1 -- start start 2024-02-02T15:28:28.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.521+0000 7f2e527a3b80 1 Processor -- start 2024-02-02T15:28:28.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.522+0000 7f2e527a3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56487dc50010 0x56487dc5b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:28.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.522+0000 7f2e527a3b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56487dc50930 0x56487dc58380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:28.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.522+0000 7f2e527a3b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56487dc5b7e0 0x56487dc58920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:28.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.522+0000 7f2e527a3b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56487dba1d50 con 0x56487dc5b7e0 2024-02-02T15:28:28.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.522+0000 7f2e527a3b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56487db3dd80 con 0x56487dc50930 2024-02-02T15:28:28.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.522+0000 7f2e527a3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56487db3da20 con 0x56487dc50010 2024-02-02T15:28:28.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.522+0000 7f2e527a3b80 10 client.0 _ll_register_callbacks cb 0x56487db3f290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:28:28.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.522+0000 7f2e37e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56487dc50010 0x56487dc5b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:28.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.522+0000 7f2e37e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56487dc50010 0x56487dc5b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:47502/0 (socket says 172.21.15.1:47502) 2024-02-02T15:28:28.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.522+0000 7f2e37e00700 1 -- 172.21.15.1:0/3648655953 learned_addr learned my addr 172.21.15.1:0/3648655953 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:28:28.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.522+0000 7f2e37400700 1 --2- 172.21.15.1:0/3648655953 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56487dc50930 0x56487dc58380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:28.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.523+0000 7f2e3cc00700 1 --2- 172.21.15.1:0/3648655953 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56487dc5b7e0 0x56487dc58920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:28.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.523+0000 7f2e37e00700 1 -- 172.21.15.1:0/3648655953 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56487dc50930 msgr2=0x56487dc58380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:28:28.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.523+0000 7f2e37e00700 1 --2- 172.21.15.1:0/3648655953 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56487dc50930 0x56487dc58380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:28.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.523+0000 7f2e37e00700 1 -- 172.21.15.1:0/3648655953 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56487dc5b7e0 msgr2=0x56487dc58920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:28:28.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.523+0000 7f2e37e00700 1 --2- 172.21.15.1:0/3648655953 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56487dc5b7e0 0x56487dc58920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:28.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.523+0000 7f2e37e00700 1 -- 172.21.15.1:0/3648655953 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e3000fe70 con 0x56487dc50010 2024-02-02T15:28:28.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.523+0000 7f2e3cc00700 1 --2- 172.21.15.1:0/3648655953 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56487dc5b7e0 0x56487dc58920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:28:28.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.523+0000 7f2e37e00700 1 --2- 172.21.15.1:0/3648655953 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56487dc50010 0x56487dc5b170 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2e2800c010 tx=0x7f2e28009530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f959084122f9b957 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:28:28.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.523+0000 7f2e25600700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:28:28.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.523+0000 7f2e25600700 1 -- 172.21.15.1:0/3648655953 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e2801d030 con 0x56487dc50010 2024-02-02T15:28:28.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.523+0000 7f2e25600700 1 -- 172.21.15.1:0/3648655953 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f2e28004500 con 0x56487dc50010 2024-02-02T15:28:28.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.523+0000 7f2e527a3b80 1 -- 172.21.15.1:0/3648655953 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56487dc440f0 con 0x56487dc50010 2024-02-02T15:28:28.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.524+0000 7f2e25600700 1 -- 172.21.15.1:0/3648655953 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2e280046c0 con 0x56487dc50010 2024-02-02T15:28:28.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.524+0000 7f2e527a3b80 10 client.25978 fetch_fsmap learned FSMap version 27 2024-02-02T15:28:28.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.524+0000 7f2e527a3b80 1 -- 172.21.15.1:0/3648655953 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56487d9cfb60 con 0x56487dc50010 2024-02-02T15:28:28.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.524+0000 7f2e25600700 1 -- 172.21.15.1:0/3648655953 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56487dc440f0 con 0x56487dc50010 2024-02-02T15:28:28.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.524+0000 7f2e25600700 1 -- 172.21.15.1:0/3648655953 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2e2800f9c0 con 0x56487dc50010 2024-02-02T15:28:28.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.524+0000 7f2e527a3b80 10 client.25978 fetch_fsmap finished waiting for FSMap version 27 2024-02-02T15:28:28.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.524+0000 7f2e527a3b80 10 client.25978 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:28:28.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.524+0000 7f2e527a3b80 20 client.25978 populate_metadata read hostname 'smithi001' 2024-02-02T15:28:28.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.524+0000 7f2e527a3b80 1 --2- 172.21.15.1:0/3648655953 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56487dc6b110 0x56487dc6d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:28.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.524+0000 7f2e527a3b80 4 client.25978 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:28:28.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.524+0000 7f2e527a3b80 1 -- 172.21.15.1:0/3648655953 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56487db9add0 con 0x56487dc6b110 2024-02-02T15:28:28.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.524+0000 7f2e37400700 1 --2- 172.21.15.1:0/3648655953 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56487dc6b110 0x56487dc6d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:28.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.525+0000 7f2e37400700 1 --2- 172.21.15.1:0/3648655953 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56487dc6b110 0x56487dc6d530 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7b401fbb70f5078 server_cookie=52fd41009579675e in_seq=0 out_seq=0 2024-02-02T15:28:28.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.525+0000 7f2e25600700 10 client.25978 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:28:28.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.526+0000 7f2e25600700 1 -- 172.21.15.1:0/3648655953 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x56487db9add0 con 0x56487dc6b110 2024-02-02T15:28:28.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.526+0000 7f2e25600700 10 client.25978 handle_command_reply: tid=0 2024-02-02T15:28:28.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.526+0000 7f2e527a3b80 1 client.25978 shutdown 2024-02-02T15:28:28.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.526+0000 7f2e527a3b80 1 -- 172.21.15.1:0/3648655953 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56487dc50010 msgr2=0x56487dc5b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:28:28.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.526+0000 7f2e527a3b80 1 --2- 172.21.15.1:0/3648655953 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56487dc50010 0x56487dc5b170 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2e2800c010 tx=0x7f2e28009530 comp rx=0 tx=0).stop 2024-02-02T15:28:28.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.526+0000 7f2e527a3b80 1 -- 172.21.15.1:0/3648655953 shutdown_connections 2024-02-02T15:28:28.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.526+0000 7f2e527a3b80 1 --2- 172.21.15.1:0/3648655953 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56487dc6b110 0x56487dc6d530 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:28.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.526+0000 7f2e527a3b80 1 --2- 172.21.15.1:0/3648655953 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56487dc5b7e0 0x56487dc58920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:28.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.526+0000 7f2e527a3b80 1 --2- 172.21.15.1:0/3648655953 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56487dc50010 0x56487dc5b170 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:28.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.526+0000 7f2e527a3b80 1 --2- 172.21.15.1:0/3648655953 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56487dc50930 0x56487dc58380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:28.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.526+0000 7f2e527a3b80 1 -- 172.21.15.1:0/3648655953 >> 172.21.15.1:0/3648655953 conn(0x56487dbafdb0 msgr2=0x56487dc4bfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:28:28.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.526+0000 7f2e25600700 0 client.25978 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:28:28.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.526+0000 7f2e527a3b80 1 -- 172.21.15.1:0/3648655953 shutdown_connections 2024-02-02T15:28:28.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.526+0000 7f2e527a3b80 1 -- 172.21.15.1:0/3648655953 wait complete. 2024-02-02T15:28:28.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.527+0000 7f2e527a3b80 20 client.25978 trim_cache size 0 max 16384 2024-02-02T15:28:28.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.547+0000 7f2e527a3b80 1 Processor -- start 2024-02-02T15:28:28.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.547+0000 7f2e527a3b80 1 -- start start 2024-02-02T15:28:28.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.547+0000 7f2e527a3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56487dc50010 0x56487dc622a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:28.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.547+0000 7f2e527a3b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56487dc50930 0x56487dc5aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:28.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.547+0000 7f2e527a3b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56487dc5b7e0 0x56487dc57b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:28.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.547+0000 7f2e527a3b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56487da256a0 con 0x56487dc50930 2024-02-02T15:28:28.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.547+0000 7f2e527a3b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56487dc489b0 con 0x56487dc5b7e0 2024-02-02T15:28:28.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.547+0000 7f2e527a3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56487dc541d0 con 0x56487dc50010 2024-02-02T15:28:28.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.547+0000 7f2e37e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56487dc50010 0x56487dc622a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:28.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.547+0000 7f2e3cc00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56487dc5b7e0 0x56487dc57b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:28.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.547+0000 7f2e37e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56487dc50010 0x56487dc622a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:47518/0 (socket says 172.21.15.1:47518) 2024-02-02T15:28:28.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.547+0000 7f2e37e00700 1 -- 172.21.15.1:0/3098925720 learned_addr learned my addr 172.21.15.1:0/3098925720 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:28:28.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.547+0000 7f2e37400700 1 --2- 172.21.15.1:0/3098925720 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56487dc50930 0x56487dc5aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:28.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.548+0000 7f2e37e00700 1 -- 172.21.15.1:0/3098925720 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56487dc5b7e0 msgr2=0x56487dc57b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:28:28.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.548+0000 7f2e37e00700 1 --2- 172.21.15.1:0/3098925720 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56487dc5b7e0 0x56487dc57b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:28.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.548+0000 7f2e37e00700 1 -- 172.21.15.1:0/3098925720 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56487dc50930 msgr2=0x56487dc5aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:28:28.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.548+0000 7f2e37e00700 1 --2- 172.21.15.1:0/3098925720 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56487dc50930 0x56487dc5aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:28.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.548+0000 7f2e37e00700 1 -- 172.21.15.1:0/3098925720 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56487db67ea0 con 0x56487dc50010 2024-02-02T15:28:28.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.548+0000 7f2e37400700 1 --2- 172.21.15.1:0/3098925720 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56487dc50930 0x56487dc5aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:28:28.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.548+0000 7f2e37e00700 1 --2- 172.21.15.1:0/3098925720 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56487dc50010 0x56487dc622a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2e280098e0 tx=0x7f2e28005f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b269b3bda98ab179 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:28:28.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.548+0000 7f2e2dc00700 1 -- 172.21.15.1:0/3098925720 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e2801d070 con 0x56487dc50010 2024-02-02T15:28:28.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.548+0000 7f2e2dc00700 1 -- 172.21.15.1:0/3098925720 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e2803b070 con 0x56487dc50010 2024-02-02T15:28:28.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.548+0000 7f2e2dc00700 1 -- 172.21.15.1:0/3098925720 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e28024940 con 0x56487dc50010 2024-02-02T15:28:28.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.549+0000 7f2e527a3b80 1 -- 172.21.15.1:0/3098925720 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56487dc50010 msgr2=0x56487dc622a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:28:28.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.549+0000 7f2e527a3b80 1 --2- 172.21.15.1:0/3098925720 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56487dc50010 0x56487dc622a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2e280098e0 tx=0x7f2e28005f00 comp rx=0 tx=0).stop 2024-02-02T15:28:28.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.549+0000 7f2e527a3b80 1 -- 172.21.15.1:0/3098925720 shutdown_connections 2024-02-02T15:28:28.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.549+0000 7f2e527a3b80 1 --2- 172.21.15.1:0/3098925720 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56487dc5b7e0 0x56487dc57b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:28.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.549+0000 7f2e527a3b80 1 --2- 172.21.15.1:0/3098925720 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56487dc50010 0x56487dc622a0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:28.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.549+0000 7f2e527a3b80 1 --2- 172.21.15.1:0/3098925720 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56487dc50930 0x56487dc5aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:28.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.549+0000 7f2e527a3b80 1 -- 172.21.15.1:0/3098925720 >> 172.21.15.1:0/3098925720 conn(0x56487dbafdb0 msgr2=0x56487dc67420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:28:28.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.549+0000 7f2e527a3b80 1 -- 172.21.15.1:0/3098925720 shutdown_connections 2024-02-02T15:28:28.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.549+0000 7f2e527a3b80 1 -- 172.21.15.1:0/3098925720 wait complete. 2024-02-02T15:28:28.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.550+0000 7f2e527a3b80 1 -- start start 2024-02-02T15:28:28.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.550+0000 7f2e527a3b80 1 Processor -- start 2024-02-02T15:28:28.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.550+0000 7f2e527a3b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56487dc50010 0x56487dc5e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:28.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.550+0000 7f2e527a3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56487dc50930 0x56487dc5e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:28.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.550+0000 7f2e527a3b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56487dc5b7e0 0x56487dcbbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:28.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.550+0000 7f2e527a3b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56487dcbc140 con 0x56487dc50010 2024-02-02T15:28:28.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.550+0000 7f2e527a3b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56487dcbc280 con 0x56487dc5b7e0 2024-02-02T15:28:28.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.550+0000 7f2e527a3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56487dcbc3c0 con 0x56487dc50930 2024-02-02T15:28:28.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.551+0000 7f2e527a3b80 10 client.0 _ll_register_callbacks cb 0x56487db3f290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:28:28.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.551+0000 7f2e37e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56487dc50010 0x56487dc5e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:28.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.551+0000 7f2e3cc00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56487dc5b7e0 0x56487dcbbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:28.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.551+0000 7f2e3cc00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56487dc5b7e0 0x56487dcbbc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:51174/0 (socket says 172.21.15.1:51174) 2024-02-02T15:28:28.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.551+0000 7f2e3cc00700 1 -- 172.21.15.1:0/1685764982 learned_addr learned my addr 172.21.15.1:0/1685764982 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:28:28.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.551+0000 7f2e37400700 1 --2- 172.21.15.1:0/1685764982 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56487dc50930 0x56487dc5e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:28.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.551+0000 7f2e37e00700 1 -- 172.21.15.1:0/1685764982 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56487dc50930 msgr2=0x56487dc5e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:28:28.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.551+0000 7f2e37e00700 1 --2- 172.21.15.1:0/1685764982 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56487dc50930 0x56487dc5e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:28.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.551+0000 7f2e37e00700 1 -- 172.21.15.1:0/1685764982 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56487dc5b7e0 msgr2=0x56487dcbbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:28:28.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.551+0000 7f2e37e00700 1 --2- 172.21.15.1:0/1685764982 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56487dc5b7e0 0x56487dcbbc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:28.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.551+0000 7f2e37e00700 1 -- 172.21.15.1:0/1685764982 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56487d9cfb60 con 0x56487dc50010 2024-02-02T15:28:28.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.551+0000 7f2e3cc00700 1 --2- 172.21.15.1:0/1685764982 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56487dc5b7e0 0x56487dcbbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:28:28.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.552+0000 7f2e37e00700 1 --2- 172.21.15.1:0/1685764982 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56487dc50010 0x56487dc5e470 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f2e2800fcf0 tx=0x7f2e28005870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d40e362c91eda48 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:28:28.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.552+0000 7f2e25600700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:28:28.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.552+0000 7f2e25600700 1 -- 172.21.15.1:0/1685764982 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e2801d040 con 0x56487dc50010 2024-02-02T15:28:28.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.552+0000 7f2e25600700 1 -- 172.21.15.1:0/1685764982 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f2e28000e40 con 0x56487dc50010 2024-02-02T15:28:28.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.552+0000 7f2e527a3b80 1 -- 172.21.15.1:0/1685764982 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2e3000f040 con 0x56487dc50010 2024-02-02T15:28:28.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.552+0000 7f2e25600700 1 -- 172.21.15.1:0/1685764982 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2e28009290 con 0x56487dc50010 2024-02-02T15:28:28.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.552+0000 7f2e527a3b80 10 client.15936 fetch_fsmap learned FSMap version 27 2024-02-02T15:28:28.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.552+0000 7f2e527a3b80 1 -- 172.21.15.1:0/1685764982 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56487da27cd0 con 0x56487dc50010 2024-02-02T15:28:28.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.552+0000 7f2e25600700 1 -- 172.21.15.1:0/1685764982 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e3000f040 con 0x56487dc50010 2024-02-02T15:28:28.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.552+0000 7f2e25600700 1 -- 172.21.15.1:0/1685764982 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2e28004500 con 0x56487dc50010 2024-02-02T15:28:28.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.553+0000 7f2e527a3b80 10 client.15936 fetch_fsmap finished waiting for FSMap version 27 2024-02-02T15:28:28.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.553+0000 7f2e527a3b80 10 client.15936 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:28:28.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.553+0000 7f2e527a3b80 20 client.15936 populate_metadata read hostname 'smithi001' 2024-02-02T15:28:28.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.553+0000 7f2e527a3b80 1 --2- 172.21.15.1:0/1685764982 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56487dcbd620 0x56487dcbfa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:28.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.553+0000 7f2e527a3b80 4 client.15936 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2024-02-02T15:28:28.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.553+0000 7f2e527a3b80 1 -- 172.21.15.1:0/1685764982 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56487db60f50 con 0x56487dcbd620 2024-02-02T15:28:28.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.553+0000 7f2e37400700 1 --2- 172.21.15.1:0/1685764982 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56487dcbd620 0x56487dcbfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:28.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.553+0000 7f2e37400700 1 --2- 172.21.15.1:0/1685764982 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56487dcbd620 0x56487dcbfa10 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df61a3b4890266f2 server_cookie=27d85003b57997ae in_seq=0 out_seq=0 2024-02-02T15:28:28.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.553+0000 7f2e25600700 10 client.15936 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:28:28.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.554+0000 7f2e25600700 1 -- 172.21.15.1:0/1685764982 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56487db60f50 con 0x56487dcbd620 2024-02-02T15:28:28.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.554+0000 7f2e25600700 10 client.15936 handle_command_reply: tid=0 2024-02-02T15:28:28.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.554+0000 7f2e527a3b80 1 client.15936 shutdown 2024-02-02T15:28:28.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.554+0000 7f2e527a3b80 1 -- 172.21.15.1:0/1685764982 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56487dc50010 msgr2=0x56487dc5e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:28:28.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.554+0000 7f2e527a3b80 1 --2- 172.21.15.1:0/1685764982 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56487dc50010 0x56487dc5e470 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f2e2800fcf0 tx=0x7f2e28005870 comp rx=0 tx=0).stop 2024-02-02T15:28:28.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.554+0000 7f2e527a3b80 1 -- 172.21.15.1:0/1685764982 shutdown_connections 2024-02-02T15:28:28.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.554+0000 7f2e527a3b80 1 --2- 172.21.15.1:0/1685764982 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56487dcbd620 0x56487dcbfa10 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:28.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.554+0000 7f2e527a3b80 1 --2- 172.21.15.1:0/1685764982 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56487dc5b7e0 0x56487dcbbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:28.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.554+0000 7f2e527a3b80 1 --2- 172.21.15.1:0/1685764982 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56487dc50010 0x56487dc5e470 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:28.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.554+0000 7f2e527a3b80 1 --2- 172.21.15.1:0/1685764982 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56487dc50930 0x56487dc5e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:28.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.554+0000 7f2e25600700 0 client.15936 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:28:28.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.554+0000 7f2e527a3b80 1 -- 172.21.15.1:0/1685764982 >> 172.21.15.1:0/1685764982 conn(0x56487dbafdb0 msgr2=0x56487dc51fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:28:28.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.554+0000 7f2e527a3b80 1 -- 172.21.15.1:0/1685764982 shutdown_connections 2024-02-02T15:28:28.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.554+0000 7f2e527a3b80 1 -- 172.21.15.1:0/1685764982 wait complete. 2024-02-02T15:28:28.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.555+0000 7f2e527a3b80 20 client.15936 trim_cache size 0 max 16384 2024-02-02T15:28:28.558 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:28:28.559 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2024-02-02T15:28:28.559 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2024-02-02T15:28:28.559 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:28:28.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.555+0000 7f2e3ec00700 1 -- 172.21.15.1:0/2519361272 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f2e180a07b0 msgr2=0x7f2e180a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:28:28.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.555+0000 7f2e3ec00700 1 --2- 172.21.15.1:0/2519361272 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f2e180a07b0 0x7f2e180a2c70 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2e3000f930 tx=0x7f2e3000fcf0 comp rx=0 tx=0).stop 2024-02-02T15:28:28.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.555+0000 7f2e3ec00700 1 -- 172.21.15.1:0/2519361272 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e38106120 msgr2=0x7f2e38073580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:28:28.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.555+0000 7f2e3ec00700 1 --2- 172.21.15.1:0/2519361272 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e38106120 0x7f2e38073580 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2e2000ea30 tx=0x7f2e2000ed40 comp rx=0 tx=0).stop 2024-02-02T15:28:28.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.555+0000 7f2e3ec00700 1 -- 172.21.15.1:0/2519361272 shutdown_connections 2024-02-02T15:28:28.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.555+0000 7f2e3ec00700 1 --2- 172.21.15.1:0/2519361272 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f2e180a07b0 0x7f2e180a2c70 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:28.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.555+0000 7f2e3ec00700 1 --2- 172.21.15.1:0/2519361272 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e38073ac0 0x7f2e381a50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:28.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.555+0000 7f2e3ec00700 1 --2- 172.21.15.1:0/2519361272 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e38101000 0x7f2e38073040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:28.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.555+0000 7f2e3ec00700 1 --2- 172.21.15.1:0/2519361272 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e38106120 0x7f2e38073580 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:28.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.555+0000 7f2e3ec00700 1 -- 172.21.15.1:0/2519361272 >> 172.21.15.1:0/2519361272 conn(0x7f2e380fc860 msgr2=0x7f2e38104a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:28:28.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.555+0000 7f2e3ec00700 1 -- 172.21.15.1:0/2519361272 shutdown_connections 2024-02-02T15:28:28.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.556+0000 7f2e3ec00700 1 -- 172.21.15.1:0/2519361272 wait complete. 2024-02-02T15:28:28.576 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-02T15:28:28.576 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T15:28:28.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.779+0000 7fc4d7600700 1 -- 172.21.15.1:0/2696968470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4d00730f0 msgr2=0x7fc4d00734d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:28:28.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.779+0000 7fc4d7600700 1 --2- 172.21.15.1:0/2696968470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4d00730f0 0x7fc4d00734d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc4c0009990 tx=0x7fc4c0009ca0 comp rx=0 tx=0).stop 2024-02-02T15:28:28.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.780+0000 7fc4d7600700 1 -- 172.21.15.1:0/2696968470 shutdown_connections 2024-02-02T15:28:28.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.780+0000 7fc4d7600700 1 --2- 172.21.15.1:0/2696968470 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4d010d5c0 0x7fc4d010f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:28.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.780+0000 7fc4d7600700 1 --2- 172.21.15.1:0/2696968470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4d00730f0 0x7fc4d00734d0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:28.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.780+0000 7fc4d7600700 1 --2- 172.21.15.1:0/2696968470 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4d0073a10 0x7fc4d010d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:28.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.780+0000 7fc4d7600700 1 -- 172.21.15.1:0/2696968470 >> 172.21.15.1:0/2696968470 conn(0x7fc4d00fc810 msgr2=0x7fc4d00fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:28:28.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.780+0000 7fc4d7600700 1 -- 172.21.15.1:0/2696968470 shutdown_connections 2024-02-02T15:28:28.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.780+0000 7fc4d7600700 1 -- 172.21.15.1:0/2696968470 wait complete. 2024-02-02T15:28:28.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.781+0000 7fc4d7600700 1 Processor -- start 2024-02-02T15:28:28.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.781+0000 7fc4d7600700 1 -- start start 2024-02-02T15:28:28.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.781+0000 7fc4d7600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4d00730f0 0x7fc4d0199070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:28.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.781+0000 7fc4d7600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4d0073a10 0x7fc4d01995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:28.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.781+0000 7fc4d7600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4d010d5c0 0x7fc4d019da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:28.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.781+0000 7fc4d7600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d019e050 con 0x7fc4d010d5c0 2024-02-02T15:28:28.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.781+0000 7fc4d7600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d019e1c0 con 0x7fc4d00730f0 2024-02-02T15:28:28.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.781+0000 7fc4d7600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d019e380 con 0x7fc4d0073a10 2024-02-02T15:28:28.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.781+0000 7fc4d4c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4d00730f0 0x7fc4d0199070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:28.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.781+0000 7fc4d4c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4d00730f0 0x7fc4d0199070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:51180/0 (socket says 172.21.15.1:51180) 2024-02-02T15:28:28.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.781+0000 7fc4d4c00700 1 -- 172.21.15.1:0/3033400681 learned_addr learned my addr 172.21.15.1:0/3033400681 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:28:28.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.782+0000 7fc4d5600700 1 --2- 172.21.15.1:0/3033400681 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4d010d5c0 0x7fc4d019da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:28.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.782+0000 7fc4cfe00700 1 --2- 172.21.15.1:0/3033400681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4d0073a10 0x7fc4d01995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:28.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.782+0000 7fc4d4c00700 1 -- 172.21.15.1:0/3033400681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4d0073a10 msgr2=0x7fc4d01995b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:28:28.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.782+0000 7fc4d4c00700 1 --2- 172.21.15.1:0/3033400681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4d0073a10 0x7fc4d01995b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:28.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.782+0000 7fc4d4c00700 1 -- 172.21.15.1:0/3033400681 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4d010d5c0 msgr2=0x7fc4d019da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:28:28.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.782+0000 7fc4d4c00700 1 --2- 172.21.15.1:0/3033400681 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4d010d5c0 0x7fc4d019da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:28.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.782+0000 7fc4d4c00700 1 -- 172.21.15.1:0/3033400681 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4c00095f0 con 0x7fc4d00730f0 2024-02-02T15:28:28.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.782+0000 7fc4d4c00700 1 --2- 172.21.15.1:0/3033400681 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4d00730f0 0x7fc4d0199070 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc4c0005910 tx=0x7fc4c0004710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:28:28.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.782+0000 7fc4cd600700 1 -- 172.21.15.1:0/3033400681 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c001d070 con 0x7fc4d00730f0 2024-02-02T15:28:28.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.782+0000 7fc4d7600700 1 -- 172.21.15.1:0/3033400681 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4d019e600 con 0x7fc4d00730f0 2024-02-02T15:28:28.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.782+0000 7fc4cd600700 1 -- 172.21.15.1:0/3033400681 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc4c0022470 con 0x7fc4d00730f0 2024-02-02T15:28:28.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.782+0000 7fc4cd600700 1 -- 172.21.15.1:0/3033400681 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c0017650 con 0x7fc4d00730f0 2024-02-02T15:28:28.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.783+0000 7fc4d7600700 1 -- 172.21.15.1:0/3033400681 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4d01a5ea0 con 0x7fc4d00730f0 2024-02-02T15:28:28.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.783+0000 7fc4d7600700 1 -- 172.21.15.1:0/3033400681 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4d0067bc0 con 0x7fc4d00730f0 2024-02-02T15:28:28.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.786+0000 7fc4cd600700 1 -- 172.21.15.1:0/3033400681 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) v1 ==== 135302+0+0 (secure 0 0 0) 0x7fc4c00225e0 con 0x7fc4d00730f0 2024-02-02T15:28:28.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.787+0000 7fc4cd600700 1 --2- 172.21.15.1:0/3033400681 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fc4b00a07b0 0x7fc4b00a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:28.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.787+0000 7fc4cd600700 1 -- 172.21.15.1:0/3033400681 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fc4c00cd810 con 0x7fc4d00730f0 2024-02-02T15:28:28.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.787+0000 7fc4cfe00700 1 --2- 172.21.15.1:0/3033400681 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fc4b00a07b0 0x7fc4b00a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:28.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.788+0000 7fc4cfe00700 1 --2- 172.21.15.1:0/3033400681 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fc4b00a07b0 0x7fc4b00a2c70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc4d019a680 tx=0x7fc4b8009260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:28:28.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:28.792+0000 7fc4cd600700 1 -- 172.21.15.1:0/3033400681 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7fc4c0096b00 con 0x7fc4d00730f0 2024-02-02T15:28:29.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.069+0000 7fc4d7600700 1 -- 172.21.15.1:0/3033400681 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc4d019a3c0 con 0x7fc4d00730f0 2024-02-02T15:28:29.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.070+0000 7fc4cd600700 1 -- 172.21.15.1:0/3033400681 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+10866 (secure 0 0 0) 0x7fc4c009a120 con 0x7fc4d00730f0 2024-02-02T15:28:29.070 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:28:29.070 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2922240448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2922240448},{"type":"v1","addr":"172.21.15.1:6835","nonce":2922240448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2729497431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2729497431},{"type":"v1","addr":"172.21.15.1:6837","nonce":2729497431}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-02T15:24:47.761435+0000","modified":"2024-02-02T15:25:20.251601+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24529,"mds_2":24518,"mds_3":24514,"mds_4":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14517":{"gid":14517,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/1218303683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":1218303683},{"type":"v1","addr":"172.21.15.1:6839","nonce":1218303683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14529":{"gid":14529,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/2059197496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2059197496},{"type":"v1","addr":"172.21.15.1:6841","nonce":2059197496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/7648986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":7648986},{"type":"v1","addr":"172.21.15.142:6833","nonce":7648986}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/3482746094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3482746094},{"type":"v1","addr":"172.21.15.71:6833","nonce":3482746094}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/388207814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":388207814},{"type":"v1","addr":"172.21.15.71:6835","nonce":388207814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6835/1052606547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1052606547},{"type":"v1","addr":"172.21.15.142:6835","nonce":1052606547}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.142:6837/2870289019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2870289019},{"type":"v1","addr":"172.21.15.142:6837","nonce":2870289019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.71:6837/628016199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":628016199},{"type":"v1","addr":"172.21.15.71:6837","nonce":628016199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.142:6839/321675421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":321675421},{"type":"v1","addr":"172.21.15.142:6839","nonce":321675421}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/1268086493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1268086493},{"type":"v1","addr":"172.21.15.71:6839","nonce":1268086493}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-02T15:28:29.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.072+0000 7fc4d7600700 1 -- 172.21.15.1:0/3033400681 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fc4b00a07b0 msgr2=0x7fc4b00a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:28:29.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.072+0000 7fc4d7600700 1 --2- 172.21.15.1:0/3033400681 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fc4b00a07b0 0x7fc4b00a2c70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc4d019a680 tx=0x7fc4b8009260 comp rx=0 tx=0).stop 2024-02-02T15:28:29.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.072+0000 7fc4d7600700 1 -- 172.21.15.1:0/3033400681 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4d00730f0 msgr2=0x7fc4d0199070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:28:29.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.072+0000 7fc4d7600700 1 --2- 172.21.15.1:0/3033400681 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4d00730f0 0x7fc4d0199070 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc4c0005910 tx=0x7fc4c0004710 comp rx=0 tx=0).stop 2024-02-02T15:28:29.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.073+0000 7fc4d7600700 1 -- 172.21.15.1:0/3033400681 shutdown_connections 2024-02-02T15:28:29.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.073+0000 7fc4d7600700 1 --2- 172.21.15.1:0/3033400681 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fc4b00a07b0 0x7fc4b00a2c70 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.073+0000 7fc4d7600700 1 --2- 172.21.15.1:0/3033400681 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc4d010d5c0 0x7fc4d019da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.073+0000 7fc4d7600700 1 --2- 172.21.15.1:0/3033400681 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc4d00730f0 0x7fc4d0199070 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.073+0000 7fc4d7600700 1 --2- 172.21.15.1:0/3033400681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc4d0073a10 0x7fc4d01995b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.073+0000 7fc4d7600700 1 -- 172.21.15.1:0/3033400681 >> 172.21.15.1:0/3033400681 conn(0x7fc4d00fc810 msgr2=0x7fc4d010f690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:28:29.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.073+0000 7fc4d7600700 1 -- 172.21.15.1:0/3033400681 shutdown_connections 2024-02-02T15:28:29.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.073+0000 7fc4d7600700 1 -- 172.21.15.1:0/3033400681 wait complete. 2024-02-02T15:28:29.074 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 27 2024-02-02T15:28:29.092 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-02-02T15:28:29.216 INFO:tasks.workunit.client.0.smithi001.stdout: 1 7637 19.74 MB/sec warmup 3 sec latency 7.504 ms 2024-02-02T15:28:29.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.296+0000 7f345d200700 1 -- 172.21.15.1:0/2901082354 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3458073110 msgr2=0x7f34580734f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:28:29.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.296+0000 7f345d200700 1 --2- 172.21.15.1:0/2901082354 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3458073110 0x7f34580734f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3448009990 tx=0x7f3448009ca0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.296+0000 7f345d200700 1 -- 172.21.15.1:0/2901082354 shutdown_connections 2024-02-02T15:28:29.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.296+0000 7f345d200700 1 --2- 172.21.15.1:0/2901082354 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f345810d620 0x7f345810fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.296+0000 7f345d200700 1 --2- 172.21.15.1:0/2901082354 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3458073110 0x7f34580734f0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.296+0000 7f345d200700 1 --2- 172.21.15.1:0/2901082354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3458073a30 0x7f345810d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.296+0000 7f345d200700 1 -- 172.21.15.1:0/2901082354 >> 172.21.15.1:0/2901082354 conn(0x7f34580fc810 msgr2=0x7f34580fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:28:29.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.296+0000 7f345d200700 1 -- 172.21.15.1:0/2901082354 shutdown_connections 2024-02-02T15:28:29.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.296+0000 7f345d200700 1 -- 172.21.15.1:0/2901082354 wait complete. 2024-02-02T15:28:29.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.298+0000 7f345d200700 1 Processor -- start 2024-02-02T15:28:29.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.298+0000 7f345d200700 1 -- start start 2024-02-02T15:28:29.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.298+0000 7f345d200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3458073110 0x7f34581990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:29.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.298+0000 7f345d200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3458073a30 0x7f3458199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:29.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.298+0000 7f345d200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f345810d620 0x7f345819da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:29.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.298+0000 7f345d200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f345819e0a0 con 0x7f3458073110 2024-02-02T15:28:29.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.298+0000 7f345d200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f345819e210 con 0x7f345810d620 2024-02-02T15:28:29.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.298+0000 7f345d200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f345819e3d0 con 0x7f3458073a30 2024-02-02T15:28:29.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.298+0000 7f3456800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f345810d620 0x7f345819da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:29.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.298+0000 7f3455e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3458073110 0x7f34581990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:29.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.298+0000 7f3456800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f345810d620 0x7f345819da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:51208/0 (socket says 172.21.15.1:51208) 2024-02-02T15:28:29.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.298+0000 7f3456800700 1 -- 172.21.15.1:0/3837620778 learned_addr learned my addr 172.21.15.1:0/3837620778 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:28:29.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.298+0000 7f3455400700 1 --2- 172.21.15.1:0/3837620778 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3458073a30 0x7f3458199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:29.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.299+0000 7f3456800700 1 -- 172.21.15.1:0/3837620778 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3458073a30 msgr2=0x7f3458199600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:28:29.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.299+0000 7f3456800700 1 --2- 172.21.15.1:0/3837620778 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3458073a30 0x7f3458199600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.299+0000 7f3456800700 1 -- 172.21.15.1:0/3837620778 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3458073110 msgr2=0x7f34581990c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:28:29.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.299+0000 7f3456800700 1 --2- 172.21.15.1:0/3837620778 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3458073110 0x7f34581990c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.299+0000 7f3456800700 1 -- 172.21.15.1:0/3837620778 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34480095f0 con 0x7f345810d620 2024-02-02T15:28:29.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.299+0000 7f3456800700 1 --2- 172.21.15.1:0/3837620778 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f345810d620 0x7f345819da60 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f345000ef90 tx=0x7f345000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:28:29.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.299+0000 7f344ea00700 1 -- 172.21.15.1:0/3837620778 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3450018070 con 0x7f345810d620 2024-02-02T15:28:29.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.299+0000 7f344ea00700 1 -- 172.21.15.1:0/3837620778 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f345000f040 con 0x7f345810d620 2024-02-02T15:28:29.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.299+0000 7f344ea00700 1 -- 172.21.15.1:0/3837620778 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f345000cda0 con 0x7f345810d620 2024-02-02T15:28:29.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.299+0000 7f345d200700 1 -- 172.21.15.1:0/3837620778 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f345819e650 con 0x7f345810d620 2024-02-02T15:28:29.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.299+0000 7f345d200700 1 -- 172.21.15.1:0/3837620778 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34581a5ef0 con 0x7f345810d620 2024-02-02T15:28:29.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.302+0000 7f344ea00700 1 -- 172.21.15.1:0/3837620778 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) v1 ==== 135302+0+0 (secure 0 0 0) 0x7f3450003a00 con 0x7f345810d620 2024-02-02T15:28:29.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.304+0000 7f344ea00700 1 --2- 172.21.15.1:0/3837620778 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f34380a0830 0x7f34380a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:29.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.304+0000 7f3470dd7b80 1 Processor -- start 2024-02-02T15:28:29.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.304+0000 7f344ea00700 1 -- 172.21.15.1:0/3837620778 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f34500cbde0 con 0x7f345810d620 2024-02-02T15:28:29.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.304+0000 7f3470dd7b80 1 -- start start 2024-02-02T15:28:29.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.304+0000 7f3455e00700 1 --2- 172.21.15.1:0/3837620778 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f34380a0830 0x7f34380a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:29.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.304+0000 7f3470dd7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9975a7010 0x55d9975a73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:29.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.304+0000 7f3470dd7b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d9975a7930 0x55d9975b22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:29.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.304+0000 7f3470dd7b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9975b27e0 0x55d9975b4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:29.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.304+0000 7f3470dd7b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d99751b4c0 con 0x55d9975b27e0 2024-02-02T15:28:29.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.304+0000 7f3470dd7b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9974ffaa0 con 0x55d9975a7930 2024-02-02T15:28:29.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.304+0000 7f3470dd7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9974fd460 con 0x55d9975a7010 2024-02-02T15:28:29.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.305+0000 7f3455400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d9975a7930 0x55d9975b22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:29.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.305+0000 7f3455400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d9975a7930 0x55d9975b22a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:51218/0 (socket says 172.21.15.1:51218) 2024-02-02T15:28:29.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.305+0000 7f3455400700 1 -- 172.21.15.1:0/1974968162 learned_addr learned my addr 172.21.15.1:0/1974968162 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:28:29.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.305+0000 7f3455e00700 1 --2- 172.21.15.1:0/1974968162 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9975a7010 0x55d9975a73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:29.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.305+0000 7f3456800700 1 --2- 172.21.15.1:0/1974968162 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9975b27e0 0x55d9975b4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:29.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.305+0000 7f3455e00700 1 --2- 172.21.15.1:0/3837620778 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f34380a0830 0x7f34380a2cf0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f344800b2e0 tx=0x7f3448005960 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:28:29.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.305+0000 7f3455400700 1 -- 172.21.15.1:0/1974968162 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9975a7010 msgr2=0x55d9975a73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:28:29.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.305+0000 7f3455400700 1 --2- 172.21.15.1:0/1974968162 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9975a7010 0x55d9975a73f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.305+0000 7f3455400700 1 -- 172.21.15.1:0/1974968162 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9975b27e0 msgr2=0x55d9975b4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:28:29.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.305+0000 7f3455400700 1 --2- 172.21.15.1:0/1974968162 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9975b27e0 0x55d9975b4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.305+0000 7f3455400700 1 -- 172.21.15.1:0/1974968162 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d99737a3b0 con 0x55d9975a7930 2024-02-02T15:28:29.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.305+0000 7f3456800700 1 --2- 172.21.15.1:0/1974968162 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9975b27e0 0x55d9975b4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:28:29.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.305+0000 7f3455400700 1 --2- 172.21.15.1:0/1974968162 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d9975a7930 0x55d9975b22a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3440009d50 tx=0x7f3440011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d20b80589a25e65 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:28:29.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.306+0000 7f3447400700 1 -- 172.21.15.1:0/1974968162 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3440019630 con 0x55d9975a7930 2024-02-02T15:28:29.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.306+0000 7f3447400700 1 -- 172.21.15.1:0/1974968162 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3440019c50 con 0x55d9975a7930 2024-02-02T15:28:29.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.306+0000 7f3447400700 1 -- 172.21.15.1:0/1974968162 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f344000ec10 con 0x55d9975a7930 2024-02-02T15:28:29.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.306+0000 7f3470dd7b80 1 -- 172.21.15.1:0/1974968162 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d9975a7930 msgr2=0x55d9975b22a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:28:29.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.306+0000 7f3470dd7b80 1 --2- 172.21.15.1:0/1974968162 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d9975a7930 0x55d9975b22a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3440009d50 tx=0x7f3440011720 comp rx=0 tx=0).stop 2024-02-02T15:28:29.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.306+0000 7f3470dd7b80 1 -- 172.21.15.1:0/1974968162 shutdown_connections 2024-02-02T15:28:29.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.306+0000 7f3470dd7b80 1 --2- 172.21.15.1:0/1974968162 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9975b27e0 0x55d9975b4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.306+0000 7f3470dd7b80 1 --2- 172.21.15.1:0/1974968162 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9975a7010 0x55d9975a73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.306+0000 7f3470dd7b80 1 --2- 172.21.15.1:0/1974968162 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d9975a7930 0x55d9975b22a0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.306+0000 7f3470dd7b80 1 -- 172.21.15.1:0/1974968162 >> 172.21.15.1:0/1974968162 conn(0x55d997506db0 msgr2=0x55d9975a33b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:28:29.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.307+0000 7f3470dd7b80 1 -- 172.21.15.1:0/1974968162 shutdown_connections 2024-02-02T15:28:29.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.307+0000 7f3470dd7b80 1 -- 172.21.15.1:0/1974968162 wait complete. 2024-02-02T15:28:29.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.307+0000 7f3470dd7b80 1 -- start start 2024-02-02T15:28:29.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.307+0000 7f3470dd7b80 1 Processor -- start 2024-02-02T15:28:29.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.308+0000 7f3470dd7b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d9975a7010 0x55d9975b2150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:29.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.308+0000 7f3470dd7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9975a7930 0x55d9975af360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:29.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.308+0000 7f3470dd7b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9975b27e0 0x55d9975af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:29.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.308+0000 7f3470dd7b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9974f8d50 con 0x55d9975b27e0 2024-02-02T15:28:29.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.308+0000 7f3470dd7b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d997494d80 con 0x55d9975a7010 2024-02-02T15:28:29.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.308+0000 7f3470dd7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d997494a20 con 0x55d9975a7930 2024-02-02T15:28:29.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.308+0000 7f3470dd7b80 10 client.0 _ll_register_callbacks cb 0x55d997496290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:28:29.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.308+0000 7f3455e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d9975a7010 0x55d9975b2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:29.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.308+0000 7f3455400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9975a7930 0x55d9975af360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:29.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.308+0000 7f3455e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d9975a7010 0x55d9975b2150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:51220/0 (socket says 172.21.15.1:51220) 2024-02-02T15:28:29.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.308+0000 7f3455400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9975a7930 0x55d9975af360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:47574/0 (socket says 172.21.15.1:47574) 2024-02-02T15:28:29.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.308+0000 7f3455e00700 1 -- 172.21.15.1:0/2830506091 learned_addr learned my addr 172.21.15.1:0/2830506091 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:28:29.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.308+0000 7f3456800700 1 --2- 172.21.15.1:0/2830506091 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9975b27e0 0x55d9975af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:29.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.309+0000 7f3455e00700 1 -- 172.21.15.1:0/2830506091 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9975a7930 msgr2=0x55d9975af360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:28:29.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.309+0000 7f3455e00700 1 --2- 172.21.15.1:0/2830506091 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9975a7930 0x55d9975af360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.309+0000 7f3455e00700 1 -- 172.21.15.1:0/2830506091 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9975b27e0 msgr2=0x55d9975af8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:28:29.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.309+0000 7f3455e00700 1 --2- 172.21.15.1:0/2830506091 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9975b27e0 0x55d9975af8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.309+0000 7f3455e00700 1 -- 172.21.15.1:0/2830506091 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3440009980 con 0x55d9975a7010 2024-02-02T15:28:29.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.309+0000 7f3455e00700 1 --2- 172.21.15.1:0/2830506091 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d9975a7010 0x55d9975b2150 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3448009990 tx=0x7f3448009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5eff03b2f49d12c8 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:28:29.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.309+0000 7f343fe00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:28:29.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.309+0000 7f343fe00700 1 -- 172.21.15.1:0/2830506091 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3448004e80 con 0x55d9975a7010 2024-02-02T15:28:29.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.309+0000 7f343fe00700 1 -- 172.21.15.1:0/2830506091 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f344800f460 con 0x55d9975a7010 2024-02-02T15:28:29.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.309+0000 7f3470dd7b80 1 -- 172.21.15.1:0/2830506091 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d99759b0f0 con 0x55d9975a7010 2024-02-02T15:28:29.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.310+0000 7f343fe00700 1 -- 172.21.15.1:0/2830506091 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f344800f620 con 0x55d9975a7010 2024-02-02T15:28:29.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.310+0000 7f3470dd7b80 10 client.26006 fetch_fsmap learned FSMap version 27 2024-02-02T15:28:29.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.310+0000 7f3470dd7b80 1 -- 172.21.15.1:0/2830506091 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55d997326b60 con 0x55d9975a7010 2024-02-02T15:28:29.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.310+0000 7f343fe00700 1 -- 172.21.15.1:0/2830506091 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d99759b0f0 con 0x55d9975a7010 2024-02-02T15:28:29.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.310+0000 7f343fe00700 1 -- 172.21.15.1:0/2830506091 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f34480306b0 con 0x55d9975a7010 2024-02-02T15:28:29.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.310+0000 7f3470dd7b80 10 client.26006 fetch_fsmap finished waiting for FSMap version 27 2024-02-02T15:28:29.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.310+0000 7f3470dd7b80 10 client.26006 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-02-02T15:28:29.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.310+0000 7f3470dd7b80 20 client.26006 populate_metadata read hostname 'smithi001' 2024-02-02T15:28:29.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.310+0000 7f3470dd7b80 1 --2- 172.21.15.1:0/2830506091 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55d9975c21a0 0x55d9975c45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:29.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.310+0000 7f3470dd7b80 4 client.26006 mds_command: new command op to 24502 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:28:29.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.310+0000 7f3470dd7b80 1 -- 172.21.15.1:0/2830506091 --> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d9974f1dd0 con 0x55d9975c21a0 2024-02-02T15:28:29.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.310+0000 7f3455400700 1 --2- 172.21.15.1:0/2830506091 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55d9975c21a0 0x55d9975c45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:29.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.311+0000 7f3455400700 1 --2- 172.21.15.1:0/2830506091 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55d9975c21a0 0x55d9975c45c0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=72ca6d1ec45c25f4 server_cookie=2bc5a5bacaf3568 in_seq=0 out_seq=0 2024-02-02T15:28:29.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.311+0000 7f343fe00700 10 client.26006 ms_handle_connect on v2:172.21.15.142:6834/1052606547 2024-02-02T15:28:29.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.312+0000 7f343fe00700 1 -- 172.21.15.1:0/2830506091 <== mds.4 v2:172.21.15.142:6834/1052606547 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55d9974f1dd0 con 0x55d9975c21a0 2024-02-02T15:28:29.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.312+0000 7f343fe00700 10 client.26006 handle_command_reply: tid=0 2024-02-02T15:28:29.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.312+0000 7f3470dd7b80 1 client.26006 shutdown 2024-02-02T15:28:29.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.312+0000 7f3470dd7b80 1 -- 172.21.15.1:0/2830506091 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d9975a7010 msgr2=0x55d9975b2150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:28:29.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.312+0000 7f3470dd7b80 1 --2- 172.21.15.1:0/2830506091 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d9975a7010 0x55d9975b2150 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3448009990 tx=0x7f3448009ca0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.312+0000 7f3470dd7b80 1 -- 172.21.15.1:0/2830506091 shutdown_connections 2024-02-02T15:28:29.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.312+0000 7f3470dd7b80 1 --2- 172.21.15.1:0/2830506091 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55d9975c21a0 0x55d9975c45c0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.312+0000 7f3470dd7b80 1 --2- 172.21.15.1:0/2830506091 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9975b27e0 0x55d9975af8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.312+0000 7f3470dd7b80 1 --2- 172.21.15.1:0/2830506091 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d9975a7010 0x55d9975b2150 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.312+0000 7f3470dd7b80 1 --2- 172.21.15.1:0/2830506091 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9975a7930 0x55d9975af360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.312+0000 7f343fe00700 0 client.26006 ms_handle_reset on v2:172.21.15.142:6834/1052606547 2024-02-02T15:28:29.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.312+0000 7f3470dd7b80 1 -- 172.21.15.1:0/2830506091 >> 172.21.15.1:0/2830506091 conn(0x55d997506db0 msgr2=0x55d9975a2fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:28:29.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.312+0000 7f3470dd7b80 1 -- 172.21.15.1:0/2830506091 shutdown_connections 2024-02-02T15:28:29.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.312+0000 7f3470dd7b80 1 -- 172.21.15.1:0/2830506091 wait complete. 2024-02-02T15:28:29.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.312+0000 7f3470dd7b80 20 client.26006 trim_cache size 0 max 16384 2024-02-02T15:28:29.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.326+0000 7f3470dd7b80 1 Processor -- start 2024-02-02T15:28:29.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.326+0000 7f3470dd7b80 1 -- start start 2024-02-02T15:28:29.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.327+0000 7f3470dd7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9975a7010 0x55d9975b9370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:29.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.327+0000 7f3470dd7b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d9975a7930 0x55d9975a2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:29.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.327+0000 7f3470dd7b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9975b27e0 0x55d9975a33f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:29.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.327+0000 7f3470dd7b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d99737c6a0 con 0x55d9975b27e0 2024-02-02T15:28:29.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.327+0000 7f3470dd7b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d99759afc0 con 0x55d9975a7930 2024-02-02T15:28:29.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.327+0000 7f3470dd7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9975beb60 con 0x55d9975a7010 2024-02-02T15:28:29.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.327+0000 7f3455e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9975a7010 0x55d9975b9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:29.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.327+0000 7f3455e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9975a7010 0x55d9975b9370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:47590/0 (socket says 172.21.15.1:47590) 2024-02-02T15:28:29.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.327+0000 7f3455e00700 1 -- 172.21.15.1:0/711734306 learned_addr learned my addr 172.21.15.1:0/711734306 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:28:29.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.327+0000 7f3455400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d9975a7930 0x55d9975a2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:29.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.327+0000 7f3456800700 1 --2- 172.21.15.1:0/711734306 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9975b27e0 0x55d9975a33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:29.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.327+0000 7f3455e00700 1 -- 172.21.15.1:0/711734306 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d9975a7930 msgr2=0x55d9975a2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:28:29.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.327+0000 7f3455e00700 1 --2- 172.21.15.1:0/711734306 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d9975a7930 0x55d9975a2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.327+0000 7f3455e00700 1 -- 172.21.15.1:0/711734306 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9975b27e0 msgr2=0x55d9975a33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:28:29.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.327+0000 7f3455e00700 1 --2- 172.21.15.1:0/711734306 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9975b27e0 0x55d9975a33f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.327+0000 7f3455e00700 1 -- 172.21.15.1:0/711734306 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d9974beea0 con 0x55d9975a7010 2024-02-02T15:28:29.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.327+0000 7f3455e00700 1 --2- 172.21.15.1:0/711734306 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9975a7010 0x55d9975b9370 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3448004bf0 tx=0x7f344800b310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=176de68a02bd9b93 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:28:29.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.328+0000 7f3447e00700 1 -- 172.21.15.1:0/711734306 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3448009290 con 0x55d9975a7010 2024-02-02T15:28:29.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.328+0000 7f3447e00700 1 -- 172.21.15.1:0/711734306 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f344801e040 con 0x55d9975a7010 2024-02-02T15:28:29.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.328+0000 7f3447e00700 1 -- 172.21.15.1:0/711734306 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f344800fa30 con 0x55d9975a7010 2024-02-02T15:28:29.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.328+0000 7f3470dd7b80 1 -- 172.21.15.1:0/711734306 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9975a7010 msgr2=0x55d9975b9370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:28:29.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.328+0000 7f3470dd7b80 1 --2- 172.21.15.1:0/711734306 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9975a7010 0x55d9975b9370 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3448004bf0 tx=0x7f344800b310 comp rx=0 tx=0).stop 2024-02-02T15:28:29.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.329+0000 7f3470dd7b80 1 -- 172.21.15.1:0/711734306 shutdown_connections 2024-02-02T15:28:29.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.329+0000 7f3470dd7b80 1 --2- 172.21.15.1:0/711734306 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9975b27e0 0x55d9975a33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.329+0000 7f3470dd7b80 1 --2- 172.21.15.1:0/711734306 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9975a7010 0x55d9975b9370 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.329+0000 7f3470dd7b80 1 --2- 172.21.15.1:0/711734306 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d9975a7930 0x55d9975a2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.329+0000 7f3470dd7b80 1 -- 172.21.15.1:0/711734306 >> 172.21.15.1:0/711734306 conn(0x55d997506db0 msgr2=0x55d9975bd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:28:29.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.329+0000 7f3470dd7b80 1 -- 172.21.15.1:0/711734306 shutdown_connections 2024-02-02T15:28:29.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.329+0000 7f3470dd7b80 1 -- 172.21.15.1:0/711734306 wait complete. 2024-02-02T15:28:29.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.329+0000 7f3470dd7b80 1 -- start start 2024-02-02T15:28:29.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.330+0000 7f3470dd7b80 1 Processor -- start 2024-02-02T15:28:29.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.330+0000 7f3470dd7b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9975a7010 0x55d99760d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:29.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.330+0000 7f3470dd7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9975a7930 0x55d99760d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:29.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.330+0000 7f3470dd7b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d9975b27e0 0x55d997607c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:29.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.330+0000 7f3470dd7b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d997608260 con 0x55d9975a7010 2024-02-02T15:28:29.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.330+0000 7f3470dd7b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9976083d0 con 0x55d9975b27e0 2024-02-02T15:28:29.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.330+0000 7f3470dd7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d997608590 con 0x55d9975a7930 2024-02-02T15:28:29.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.330+0000 7f3470dd7b80 10 client.0 _ll_register_callbacks cb 0x55d997496290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:28:29.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.330+0000 7f3455e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9975a7010 0x55d99760d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:29.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.330+0000 7f3455e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9975a7010 0x55d99760d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49606/0 (socket says 172.21.15.1:49606) 2024-02-02T15:28:29.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.330+0000 7f3455400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9975a7930 0x55d99760d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:29.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.330+0000 7f3455e00700 1 -- 172.21.15.1:0/4268293267 learned_addr learned my addr 172.21.15.1:0/4268293267 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:28:29.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.330+0000 7f3456800700 1 --2- 172.21.15.1:0/4268293267 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d9975b27e0 0x55d997607c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:29.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.331+0000 7f3455400700 1 -- 172.21.15.1:0/4268293267 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d9975b27e0 msgr2=0x55d997607c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:28:29.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.331+0000 7f3455400700 1 --2- 172.21.15.1:0/4268293267 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d9975b27e0 0x55d997607c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.331+0000 7f3455400700 1 -- 172.21.15.1:0/4268293267 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9975a7010 msgr2=0x55d99760d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:28:29.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.331+0000 7f3455400700 1 --2- 172.21.15.1:0/4268293267 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9975a7010 0x55d99760d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.331+0000 7f3455400700 1 -- 172.21.15.1:0/4268293267 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d997326b60 con 0x55d9975a7930 2024-02-02T15:28:29.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.331+0000 7f3456800700 1 --2- 172.21.15.1:0/4268293267 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d9975b27e0 0x55d997607c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:28:29.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.331+0000 7f3455e00700 1 --2- 172.21.15.1:0/4268293267 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9975a7010 0x55d99760d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:28:29.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.331+0000 7f3455400700 1 --2- 172.21.15.1:0/4268293267 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9975a7930 0x55d99760d970 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3440000c00 tx=0x7f3440009f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=daca4e3e44880a7b server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:28:29.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.331+0000 7f343fe00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:28:29.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.331+0000 7f343fe00700 1 -- 172.21.15.1:0/4268293267 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3440029040 con 0x55d9975a7930 2024-02-02T15:28:29.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.331+0000 7f343fe00700 1 -- 172.21.15.1:0/4268293267 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f3440005040 con 0x55d9975a7930 2024-02-02T15:28:29.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.331+0000 7f3470dd7b80 1 -- 172.21.15.1:0/4268293267 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3440019c50 con 0x55d9975a7930 2024-02-02T15:28:29.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.331+0000 7f343fe00700 1 -- 172.21.15.1:0/4268293267 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3440009260 con 0x55d9975a7930 2024-02-02T15:28:29.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.331+0000 7f3470dd7b80 10 client.26008 fetch_fsmap learned FSMap version 27 2024-02-02T15:28:29.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.331+0000 7f3470dd7b80 1 -- 172.21.15.1:0/4268293267 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55d99737ecd0 con 0x55d9975a7930 2024-02-02T15:28:29.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.332+0000 7f343fe00700 1 -- 172.21.15.1:0/4268293267 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3440019c50 con 0x55d9975a7930 2024-02-02T15:28:29.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.332+0000 7f343fe00700 1 -- 172.21.15.1:0/4268293267 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f344002c070 con 0x55d9975a7930 2024-02-02T15:28:29.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.332+0000 7f3470dd7b80 10 client.26008 fetch_fsmap finished waiting for FSMap version 27 2024-02-02T15:28:29.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.332+0000 7f3470dd7b80 10 client.26008 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-02-02T15:28:29.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.332+0000 7f3470dd7b80 20 client.26008 populate_metadata read hostname 'smithi001' 2024-02-02T15:28:29.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.332+0000 7f3470dd7b80 1 --2- 172.21.15.1:0/4268293267 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55d997615210 0x55d997617600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:29.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.332+0000 7f3470dd7b80 4 client.26008 mds_command: new command op to 24502 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:28:29.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.332+0000 7f3470dd7b80 1 -- 172.21.15.1:0/4268293267 --> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d9974b7f50 con 0x55d997615210 2024-02-02T15:28:29.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.332+0000 7f3455e00700 1 --2- 172.21.15.1:0/4268293267 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55d997615210 0x55d997617600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:29.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.333+0000 7f3455e00700 1 --2- 172.21.15.1:0/4268293267 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55d997615210 0x55d997617600 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f68e58cd9c9aa2f1 server_cookie=383398ba2612525c in_seq=0 out_seq=0 2024-02-02T15:28:29.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.333+0000 7f343fe00700 10 client.26008 ms_handle_connect on v2:172.21.15.142:6834/1052606547 2024-02-02T15:28:29.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.333+0000 7f343fe00700 1 -- 172.21.15.1:0/4268293267 <== mds.4 v2:172.21.15.142:6834/1052606547 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d9974b7f50 con 0x55d997615210 2024-02-02T15:28:29.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.333+0000 7f343fe00700 10 client.26008 handle_command_reply: tid=0 2024-02-02T15:28:29.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.333+0000 7f3470dd7b80 1 client.26008 shutdown 2024-02-02T15:28:29.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.333+0000 7f3470dd7b80 1 -- 172.21.15.1:0/4268293267 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9975a7930 msgr2=0x55d99760d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:28:29.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.333+0000 7f3470dd7b80 1 --2- 172.21.15.1:0/4268293267 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9975a7930 0x55d99760d970 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3440000c00 tx=0x7f3440009f60 comp rx=0 tx=0).stop 2024-02-02T15:28:29.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.333+0000 7f3470dd7b80 1 -- 172.21.15.1:0/4268293267 shutdown_connections 2024-02-02T15:28:29.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.333+0000 7f3470dd7b80 1 --2- 172.21.15.1:0/4268293267 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55d997615210 0x55d997617600 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.333+0000 7f3470dd7b80 1 --2- 172.21.15.1:0/4268293267 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d9975b27e0 0x55d997607c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.333+0000 7f3470dd7b80 1 --2- 172.21.15.1:0/4268293267 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9975a7010 0x55d99760d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.333+0000 7f3470dd7b80 1 --2- 172.21.15.1:0/4268293267 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9975a7930 0x55d99760d970 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.333+0000 7f343fe00700 0 client.26008 ms_handle_reset on v2:172.21.15.142:6834/1052606547 2024-02-02T15:28:29.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.333+0000 7f3470dd7b80 1 -- 172.21.15.1:0/4268293267 >> 172.21.15.1:0/4268293267 conn(0x55d997506db0 msgr2=0x55d9975a2e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:28:29.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.333+0000 7f3470dd7b80 1 -- 172.21.15.1:0/4268293267 shutdown_connections 2024-02-02T15:28:29.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.334+0000 7f3470dd7b80 1 -- 172.21.15.1:0/4268293267 wait complete. 2024-02-02T15:28:29.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.334+0000 7f3470dd7b80 20 client.26008 trim_cache size 0 max 16384 2024-02-02T15:28:29.338 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:28:29.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.334+0000 7f345d200700 1 -- 172.21.15.1:0/3837620778 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f34380a0830 msgr2=0x7f34380a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:28:29.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.334+0000 7f345d200700 1 --2- 172.21.15.1:0/3837620778 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f34380a0830 0x7f34380a2cf0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f344800b2e0 tx=0x7f3448005960 comp rx=0 tx=0).stop 2024-02-02T15:28:29.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.334+0000 7f345d200700 1 -- 172.21.15.1:0/3837620778 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f345810d620 msgr2=0x7f345819da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:28:29.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.334+0000 7f345d200700 1 --2- 172.21.15.1:0/3837620778 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f345810d620 0x7f345819da60 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f345000ef90 tx=0x7f345000c3a0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.334+0000 7f345d200700 1 -- 172.21.15.1:0/3837620778 shutdown_connections 2024-02-02T15:28:29.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.334+0000 7f345d200700 1 --2- 172.21.15.1:0/3837620778 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f34380a0830 0x7f34380a2cf0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.334+0000 7f345d200700 1 --2- 172.21.15.1:0/3837620778 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f345810d620 0x7f345819da60 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.334+0000 7f345d200700 1 --2- 172.21.15.1:0/3837620778 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3458073110 0x7f34581990c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.334+0000 7f345d200700 1 --2- 172.21.15.1:0/3837620778 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3458073a30 0x7f3458199600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.334+0000 7f345d200700 1 -- 172.21.15.1:0/3837620778 >> 172.21.15.1:0/3837620778 conn(0x7f34580fc810 msgr2=0x7f345810f7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:28:29.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.335+0000 7f345d200700 1 -- 172.21.15.1:0/3837620778 shutdown_connections 2024-02-02T15:28:29.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.335+0000 7f345d200700 1 -- 172.21.15.1:0/3837620778 wait complete. 2024-02-02T15:28:29.353 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-02-02T15:28:29.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.571+0000 7fecec000700 1 -- 172.21.15.1:0/2927850759 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fece4101950 msgr2=0x7fece4105990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:28:29.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.571+0000 7fecec000700 1 --2- 172.21.15.1:0/2927850759 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fece4101950 0x7fece4105990 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7feccc0098f0 tx=0x7feccc009c00 comp rx=0 tx=0).stop 2024-02-02T15:28:29.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.571+0000 7fecec000700 1 -- 172.21.15.1:0/2927850759 shutdown_connections 2024-02-02T15:28:29.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.571+0000 7fecec000700 1 --2- 172.21.15.1:0/2927850759 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fece41060c0 0x7fece410dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.571+0000 7fecec000700 1 --2- 172.21.15.1:0/2927850759 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fece4100fa0 0x7fece4101380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.571+0000 7fecec000700 1 --2- 172.21.15.1:0/2927850759 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fece4101950 0x7fece4105990 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.571+0000 7fecec000700 1 -- 172.21.15.1:0/2927850759 >> 172.21.15.1:0/2927850759 conn(0x7fece40fc800 msgr2=0x7fece40fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:28:29.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.571+0000 7fecec000700 1 -- 172.21.15.1:0/2927850759 shutdown_connections 2024-02-02T15:28:29.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.571+0000 7fecec000700 1 -- 172.21.15.1:0/2927850759 wait complete. 2024-02-02T15:28:29.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.572+0000 7fecec000700 1 Processor -- start 2024-02-02T15:28:29.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.573+0000 7fecec000700 1 -- start start 2024-02-02T15:28:29.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.573+0000 7fecec000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fece4100fa0 0x7fece4199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:29.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.573+0000 7fecec000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fece41060c0 0x7fece4199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:29.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.573+0000 7fecec000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fece419dab0 0x7fece419df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:29.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.573+0000 7fecec000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fece419e530 con 0x7fece4100fa0 2024-02-02T15:28:29.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.573+0000 7fecec000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fece419e6a0 con 0x7fece419dab0 2024-02-02T15:28:29.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.573+0000 7fecec000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fece419e860 con 0x7fece41060c0 2024-02-02T15:28:29.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.573+0000 7fecea000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fece419dab0 0x7fece419df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:29.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.573+0000 7fece9600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fece4100fa0 0x7fece4199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:29.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.573+0000 7fece8c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fece41060c0 0x7fece4199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:29.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.573+0000 7fecea000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fece419dab0 0x7fece419df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:51240/0 (socket says 172.21.15.1:51240) 2024-02-02T15:28:29.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.573+0000 7fecea000700 1 -- 172.21.15.1:0/1726427232 learned_addr learned my addr 172.21.15.1:0/1726427232 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:28:29.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.573+0000 7fece9600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fece4100fa0 0x7fece4199110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49622/0 (socket says 172.21.15.1:49622) 2024-02-02T15:28:29.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.573+0000 7fece8c00700 1 --2- 172.21.15.1:0/1726427232 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fece41060c0 0x7fece4199650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:47624/0 (socket says 172.21.15.1:47624) 2024-02-02T15:28:29.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.574+0000 7fecea000700 1 -- 172.21.15.1:0/1726427232 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fece41060c0 msgr2=0x7fece4199650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:28:29.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.574+0000 7fecea000700 1 --2- 172.21.15.1:0/1726427232 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fece41060c0 0x7fece4199650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.574+0000 7fecea000700 1 -- 172.21.15.1:0/1726427232 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fece4100fa0 msgr2=0x7fece4199110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:28:29.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.574+0000 7fecea000700 1 --2- 172.21.15.1:0/1726427232 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fece4100fa0 0x7fece4199110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.574+0000 7fecea000700 1 -- 172.21.15.1:0/1726427232 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecd40095f0 con 0x7fece419dab0 2024-02-02T15:28:29.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.574+0000 7fece8c00700 1 --2- 172.21.15.1:0/1726427232 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fece41060c0 0x7fece4199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:28:29.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.574+0000 7fece9600700 1 --2- 172.21.15.1:0/1726427232 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fece4100fa0 0x7fece4199110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:28:29.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.574+0000 7fecea000700 1 --2- 172.21.15.1:0/1726427232 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fece419dab0 0x7fece419df30 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fecdc00ef30 tx=0x7fecdc00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:28:29.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.574+0000 7fece2000700 1 -- 172.21.15.1:0/1726427232 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecdc018070 con 0x7fece419dab0 2024-02-02T15:28:29.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.574+0000 7fecec000700 1 -- 172.21.15.1:0/1726427232 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feccc009520 con 0x7fece419dab0 2024-02-02T15:28:29.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.574+0000 7fece2000700 1 -- 172.21.15.1:0/1726427232 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fecdc00f040 con 0x7fece419dab0 2024-02-02T15:28:29.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.574+0000 7fece2000700 1 -- 172.21.15.1:0/1726427232 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecdc00cd00 con 0x7fece419dab0 2024-02-02T15:28:29.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.574+0000 7fecec000700 1 -- 172.21.15.1:0/1726427232 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fece41a6100 con 0x7fece419dab0 2024-02-02T15:28:29.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.577+0000 7fece2000700 1 -- 172.21.15.1:0/1726427232 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) v1 ==== 135302+0+0 (secure 0 0 0) 0x7fecdc003a00 con 0x7fece419dab0 2024-02-02T15:28:29.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.579+0000 7fece2000700 1 --2- 172.21.15.1:0/1726427232 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fecc40a07b0 0x7fecc40a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:29.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.579+0000 7fecffb19b80 1 Processor -- start 2024-02-02T15:28:29.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.579+0000 7fece2000700 1 -- 172.21.15.1:0/1726427232 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fecdc0cbe80 con 0x7fece419dab0 2024-02-02T15:28:29.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.579+0000 7fecffb19b80 1 -- start start 2024-02-02T15:28:29.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.579+0000 7fece9600700 1 --2- 172.21.15.1:0/1726427232 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fecc40a07b0 0x7fecc40a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:29.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.579+0000 7fecffb19b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b164bd0010 0x55b164bd03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:29.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.579+0000 7fecffb19b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b164bd0930 0x55b164bdb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:29.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.579+0000 7fecffb19b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b164bdb7e0 0x55b164bddbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:29.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.579+0000 7fecffb19b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b164b444c0 con 0x55b164bd0930 2024-02-02T15:28:29.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.579+0000 7fecffb19b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b164b28aa0 con 0x55b164bdb7e0 2024-02-02T15:28:29.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.579+0000 7fecffb19b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b164b26460 con 0x55b164bd0010 2024-02-02T15:28:29.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.580+0000 7fece8c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b164bd0930 0x55b164bdb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:29.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.580+0000 7fecea000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b164bdb7e0 0x55b164bddbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:29.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.580+0000 7fece9600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b164bd0010 0x55b164bd03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:29.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.580+0000 7fecea000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b164bdb7e0 0x55b164bddbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:51242/0 (socket says 172.21.15.1:51242) 2024-02-02T15:28:29.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.580+0000 7fecea000700 1 -- 172.21.15.1:0/2237175199 learned_addr learned my addr 172.21.15.1:0/2237175199 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:28:29.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.580+0000 7fece9600700 1 --2- 172.21.15.1:0/2237175199 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b164bd0010 0x55b164bd03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:47628/0 (socket says 172.21.15.1:47628) 2024-02-02T15:28:29.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.580+0000 7fecea000700 1 -- 172.21.15.1:0/2237175199 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b164bd0010 msgr2=0x55b164bd03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:28:29.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.580+0000 7fecea000700 1 --2- 172.21.15.1:0/2237175199 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b164bd0010 0x55b164bd03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.580+0000 7fecea000700 1 -- 172.21.15.1:0/2237175199 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b164bd0930 msgr2=0x55b164bdb2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:28:29.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.580+0000 7fecea000700 1 --2- 172.21.15.1:0/2237175199 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b164bd0930 0x55b164bdb2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.580+0000 7fecea000700 1 -- 172.21.15.1:0/2237175199 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b1649a33b0 con 0x55b164bdb7e0 2024-02-02T15:28:29.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.580+0000 7fece9600700 1 --2- 172.21.15.1:0/2237175199 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b164bd0010 0x55b164bd03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:28:29.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.580+0000 7fece9600700 1 --2- 172.21.15.1:0/1726427232 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fecc40a07b0 0x7fecc40a2c70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fecd4005fd0 tx=0x7fecd400fe10 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:28:29.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.580+0000 7fecea000700 1 --2- 172.21.15.1:0/2237175199 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b164bdb7e0 0x55b164bddbd0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fecdc009c10 tx=0x7fecdc098d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b77fae7b6155805 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:28:29.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.581+0000 7fecda600700 1 -- 172.21.15.1:0/2237175199 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecdc004ad0 con 0x55b164bdb7e0 2024-02-02T15:28:29.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.581+0000 7fecda600700 1 -- 172.21.15.1:0/2237175199 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fecdc01b320 con 0x55b164bdb7e0 2024-02-02T15:28:29.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.581+0000 7fecda600700 1 -- 172.21.15.1:0/2237175199 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecdc0992f0 con 0x55b164bdb7e0 2024-02-02T15:28:29.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.581+0000 7fecffb19b80 1 -- 172.21.15.1:0/2237175199 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b164bdb7e0 msgr2=0x55b164bddbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:28:29.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.581+0000 7fecffb19b80 1 --2- 172.21.15.1:0/2237175199 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b164bdb7e0 0x55b164bddbd0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fecdc009c10 tx=0x7fecdc098d30 comp rx=0 tx=0).stop 2024-02-02T15:28:29.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.581+0000 7fecffb19b80 1 -- 172.21.15.1:0/2237175199 shutdown_connections 2024-02-02T15:28:29.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.581+0000 7fecffb19b80 1 --2- 172.21.15.1:0/2237175199 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b164bdb7e0 0x55b164bddbd0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.581+0000 7fecffb19b80 1 --2- 172.21.15.1:0/2237175199 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b164bd0010 0x55b164bd03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.581+0000 7fecffb19b80 1 --2- 172.21.15.1:0/2237175199 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b164bd0930 0x55b164bdb2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.581+0000 7fecffb19b80 1 -- 172.21.15.1:0/2237175199 >> 172.21.15.1:0/2237175199 conn(0x55b164b2fdb0 msgr2=0x55b164bcc3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:28:29.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.581+0000 7fecffb19b80 1 -- 172.21.15.1:0/2237175199 shutdown_connections 2024-02-02T15:28:29.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.582+0000 7fecffb19b80 1 -- 172.21.15.1:0/2237175199 wait complete. 2024-02-02T15:28:29.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.582+0000 7fecffb19b80 1 -- start start 2024-02-02T15:28:29.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.582+0000 7fecffb19b80 1 Processor -- start 2024-02-02T15:28:29.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.583+0000 7fecffb19b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b164bd0010 0x55b164bdb290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:29.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.583+0000 7fecffb19b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b164bd0930 0x55b164bd84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:29.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.583+0000 7fecffb19b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b164bdb7e0 0x55b164bd8be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:29.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.583+0000 7fecffb19b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b164b21d50 con 0x55b164bdb7e0 2024-02-02T15:28:29.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.583+0000 7fecffb19b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b164abdd80 con 0x55b164bd0930 2024-02-02T15:28:29.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.583+0000 7fecffb19b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b164abda20 con 0x55b164bd0010 2024-02-02T15:28:29.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.583+0000 7fecffb19b80 10 client.0 _ll_register_callbacks cb 0x55b164abf290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:28:29.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.583+0000 7fece9600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b164bd0010 0x55b164bdb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:29.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.583+0000 7fece9600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b164bd0010 0x55b164bdb290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:47630/0 (socket says 172.21.15.1:47630) 2024-02-02T15:28:29.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.583+0000 7fece8c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b164bd0930 0x55b164bd84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:29.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.583+0000 7fece9600700 1 -- 172.21.15.1:0/1627567811 learned_addr learned my addr 172.21.15.1:0/1627567811 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:28:29.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.583+0000 7fecea000700 1 --2- 172.21.15.1:0/1627567811 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b164bdb7e0 0x55b164bd8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:29.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.583+0000 7fece9600700 1 -- 172.21.15.1:0/1627567811 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b164bd0930 msgr2=0x55b164bd84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:28:29.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.583+0000 7fece9600700 1 --2- 172.21.15.1:0/1627567811 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b164bd0930 0x55b164bd84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.583+0000 7fece9600700 1 -- 172.21.15.1:0/1627567811 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b164bdb7e0 msgr2=0x55b164bd8be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:28:29.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.583+0000 7fece9600700 1 --2- 172.21.15.1:0/1627567811 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b164bdb7e0 0x55b164bd8be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.583+0000 7fece9600700 1 -- 172.21.15.1:0/1627567811 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fece41a6100 con 0x55b164bd0010 2024-02-02T15:28:29.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.584+0000 7fece9600700 1 --2- 172.21.15.1:0/1627567811 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b164bd0010 0x55b164bdb290 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fecd4012a60 tx=0x7fecd40047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a07b9fce26854d5a server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:28:29.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.584+0000 7fecd2a00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:28:29.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.584+0000 7fecd2a00700 1 -- 172.21.15.1:0/1627567811 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd400ebd0 con 0x55b164bd0010 2024-02-02T15:28:29.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.584+0000 7fecd2a00700 1 -- 172.21.15.1:0/1627567811 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fecd4012e30 con 0x55b164bd0010 2024-02-02T15:28:29.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.584+0000 7fecffb19b80 1 -- 172.21.15.1:0/1627567811 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b164bc40f0 con 0x55b164bd0010 2024-02-02T15:28:29.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.585+0000 7fecd2a00700 1 -- 172.21.15.1:0/1627567811 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fecd400ed30 con 0x55b164bd0010 2024-02-02T15:28:29.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.585+0000 7fecffb19b80 10 client.26020 fetch_fsmap learned FSMap version 27 2024-02-02T15:28:29.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.585+0000 7fecffb19b80 1 -- 172.21.15.1:0/1627567811 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55b16494fb60 con 0x55b164bd0010 2024-02-02T15:28:29.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.585+0000 7fecd2a00700 1 -- 172.21.15.1:0/1627567811 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b164bc40f0 con 0x55b164bd0010 2024-02-02T15:28:29.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.585+0000 7fecd2a00700 1 -- 172.21.15.1:0/1627567811 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fecd401f3b0 con 0x55b164bd0010 2024-02-02T15:28:29.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.585+0000 7fecffb19b80 10 client.26020 fetch_fsmap finished waiting for FSMap version 27 2024-02-02T15:28:29.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.585+0000 7fecffb19b80 10 client.26020 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-02-02T15:28:29.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.585+0000 7fecffb19b80 20 client.26020 populate_metadata read hostname 'smithi001' 2024-02-02T15:28:29.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.585+0000 7fecffb19b80 1 --2- 172.21.15.1:0/1627567811 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55b164beb1d0 0x55b164bed5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:29.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.585+0000 7fecffb19b80 4 client.26020 mds_command: new command op to 24514 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:28:29.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.585+0000 7fecffb19b80 1 -- 172.21.15.1:0/1627567811 --> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b164b1add0 con 0x55b164beb1d0 2024-02-02T15:28:29.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.585+0000 7fece8c00700 1 --2- 172.21.15.1:0/1627567811 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55b164beb1d0 0x55b164bed5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:29.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.586+0000 7fece8c00700 1 --2- 172.21.15.1:0/1627567811 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55b164beb1d0 0x55b164bed5f0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=770cfcf50f34402e server_cookie=65832efc78f87adc in_seq=0 out_seq=0 2024-02-02T15:28:29.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.586+0000 7fecd2a00700 10 client.26020 ms_handle_connect on v2:172.21.15.142:6836/2870289019 2024-02-02T15:28:29.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.587+0000 7fecd2a00700 1 -- 172.21.15.1:0/1627567811 <== mds.3 v2:172.21.15.142:6836/2870289019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55b164b1add0 con 0x55b164beb1d0 2024-02-02T15:28:29.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.587+0000 7fecd2a00700 10 client.26020 handle_command_reply: tid=0 2024-02-02T15:28:29.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.587+0000 7fecffb19b80 1 client.26020 shutdown 2024-02-02T15:28:29.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.587+0000 7fecffb19b80 1 -- 172.21.15.1:0/1627567811 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b164bd0010 msgr2=0x55b164bdb290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:28:29.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.587+0000 7fecffb19b80 1 --2- 172.21.15.1:0/1627567811 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b164bd0010 0x55b164bdb290 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fecd4012a60 tx=0x7fecd40047c0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.587+0000 7fecffb19b80 1 -- 172.21.15.1:0/1627567811 shutdown_connections 2024-02-02T15:28:29.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.587+0000 7fecffb19b80 1 --2- 172.21.15.1:0/1627567811 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55b164beb1d0 0x55b164bed5f0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.587+0000 7fecffb19b80 1 --2- 172.21.15.1:0/1627567811 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b164bdb7e0 0x55b164bd8be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.587+0000 7fecffb19b80 1 --2- 172.21.15.1:0/1627567811 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b164bd0010 0x55b164bdb290 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.587+0000 7fecffb19b80 1 --2- 172.21.15.1:0/1627567811 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b164bd0930 0x55b164bd84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.587+0000 7fecd2a00700 0 client.26020 ms_handle_reset on v2:172.21.15.142:6836/2870289019 2024-02-02T15:28:29.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.587+0000 7fecffb19b80 1 -- 172.21.15.1:0/1627567811 >> 172.21.15.1:0/1627567811 conn(0x55b164b2fdb0 msgr2=0x55b164bcbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:28:29.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.587+0000 7fecffb19b80 1 -- 172.21.15.1:0/1627567811 shutdown_connections 2024-02-02T15:28:29.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.587+0000 7fecffb19b80 1 -- 172.21.15.1:0/1627567811 wait complete. 2024-02-02T15:28:29.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.587+0000 7fecffb19b80 20 client.26020 trim_cache size 0 max 16384 2024-02-02T15:28:29.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.600+0000 7fecffb19b80 1 Processor -- start 2024-02-02T15:28:29.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.601+0000 7fecffb19b80 1 -- start start 2024-02-02T15:28:29.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.601+0000 7fecffb19b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b164bd0010 0x55b164bd8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:29.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.601+0000 7fecffb19b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b164bd0930 0x55b164be2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:29.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.601+0000 7fecffb19b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b164bdb7e0 0x55b164bd1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:29.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.601+0000 7fecffb19b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1649a56a0 con 0x55b164bd0010 2024-02-02T15:28:29.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.601+0000 7fecffb19b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b164bd1dd0 con 0x55b164bdb7e0 2024-02-02T15:28:29.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.601+0000 7fecffb19b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b164bc6580 con 0x55b164bd0930 2024-02-02T15:28:29.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.601+0000 7fece8c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b164bd0930 0x55b164be2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:29.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.601+0000 7fece9600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b164bd0010 0x55b164bd8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:29.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.601+0000 7fece8c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b164bd0930 0x55b164be2330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:47636/0 (socket says 172.21.15.1:47636) 2024-02-02T15:28:29.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.601+0000 7fece8c00700 1 -- 172.21.15.1:0/3680162705 learned_addr learned my addr 172.21.15.1:0/3680162705 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:28:29.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.601+0000 7fecea000700 1 --2- 172.21.15.1:0/3680162705 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b164bdb7e0 0x55b164bd1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:29.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.602+0000 7fece8c00700 1 -- 172.21.15.1:0/3680162705 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b164bdb7e0 msgr2=0x55b164bd1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:28:29.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.602+0000 7fece8c00700 1 --2- 172.21.15.1:0/3680162705 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b164bdb7e0 0x55b164bd1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.602+0000 7fece8c00700 1 -- 172.21.15.1:0/3680162705 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b164bd0010 msgr2=0x55b164bd8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:28:29.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.602+0000 7fece8c00700 1 --2- 172.21.15.1:0/3680162705 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b164bd0010 0x55b164bd8ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.602+0000 7fece8c00700 1 -- 172.21.15.1:0/3680162705 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b164ae7ea0 con 0x55b164bd0930 2024-02-02T15:28:29.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.602+0000 7fece9600700 1 --2- 172.21.15.1:0/3680162705 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b164bd0010 0x55b164bd8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:28:29.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.602+0000 7fece8c00700 1 --2- 172.21.15.1:0/3680162705 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b164bd0930 0x55b164be2330 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7feccc0098c0 tx=0x7feccc005dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab6e9bc527b192fb server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:28:29.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.602+0000 7fecdb000700 1 -- 172.21.15.1:0/3680162705 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feccc01c040 con 0x55b164bd0930 2024-02-02T15:28:29.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.602+0000 7fecdb000700 1 -- 172.21.15.1:0/3680162705 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feccc01d040 con 0x55b164bd0930 2024-02-02T15:28:29.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.602+0000 7fecdb000700 1 -- 172.21.15.1:0/3680162705 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feccc00b900 con 0x55b164bd0930 2024-02-02T15:28:29.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.603+0000 7fecffb19b80 1 -- 172.21.15.1:0/3680162705 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b164bd0930 msgr2=0x55b164be2330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:28:29.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.603+0000 7fecffb19b80 1 --2- 172.21.15.1:0/3680162705 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b164bd0930 0x55b164be2330 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7feccc0098c0 tx=0x7feccc005dc0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.603+0000 7fecffb19b80 1 -- 172.21.15.1:0/3680162705 shutdown_connections 2024-02-02T15:28:29.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.603+0000 7fecffb19b80 1 --2- 172.21.15.1:0/3680162705 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b164bdb7e0 0x55b164bd1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.603+0000 7fecffb19b80 1 --2- 172.21.15.1:0/3680162705 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b164bd0010 0x55b164bd8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.603+0000 7fecffb19b80 1 --2- 172.21.15.1:0/3680162705 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b164bd0930 0x55b164be2330 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.603+0000 7fecffb19b80 1 -- 172.21.15.1:0/3680162705 >> 172.21.15.1:0/3680162705 conn(0x55b164b2fdb0 msgr2=0x55b164bcab20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:28:29.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.603+0000 7fecffb19b80 1 -- 172.21.15.1:0/3680162705 shutdown_connections 2024-02-02T15:28:29.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.603+0000 7fecffb19b80 1 -- 172.21.15.1:0/3680162705 wait complete. 2024-02-02T15:28:29.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.603+0000 7fecffb19b80 1 -- start start 2024-02-02T15:28:29.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.603+0000 7fecffb19b80 1 Processor -- start 2024-02-02T15:28:29.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.604+0000 7fecffb19b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b164bd0010 0x55b164be0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:29.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.604+0000 7fecffb19b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b164bd0930 0x55b164be0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:29.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.604+0000 7fecffb19b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b164bdb7e0 0x55b164be0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:29.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.604+0000 7fecffb19b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b164c3bcf0 con 0x55b164bdb7e0 2024-02-02T15:28:29.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.604+0000 7fecffb19b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b164c3be60 con 0x55b164bd0010 2024-02-02T15:28:29.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.604+0000 7fecffb19b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b164c3c020 con 0x55b164bd0930 2024-02-02T15:28:29.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.604+0000 7fecffb19b80 10 client.0 _ll_register_callbacks cb 0x55b164abf290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:28:29.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.604+0000 7fece9600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b164bd0010 0x55b164be0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:29.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.604+0000 7fece9600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b164bd0010 0x55b164be0310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:51278/0 (socket says 172.21.15.1:51278) 2024-02-02T15:28:29.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.604+0000 7fece9600700 1 -- 172.21.15.1:0/2474830517 learned_addr learned my addr 172.21.15.1:0/2474830517 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:28:29.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.604+0000 7fecea000700 1 --2- 172.21.15.1:0/2474830517 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b164bdb7e0 0x55b164be0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:29.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.604+0000 7fece8c00700 1 --2- 172.21.15.1:0/2474830517 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b164bd0930 0x55b164be0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:29.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.605+0000 7fece9600700 1 -- 172.21.15.1:0/2474830517 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b164bd0930 msgr2=0x55b164be0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:28:29.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.605+0000 7fece9600700 1 --2- 172.21.15.1:0/2474830517 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b164bd0930 0x55b164be0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.605+0000 7fece9600700 1 -- 172.21.15.1:0/2474830517 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b164bdb7e0 msgr2=0x55b164be0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:28:29.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.605+0000 7fece9600700 1 --2- 172.21.15.1:0/2474830517 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b164bdb7e0 0x55b164be0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.605+0000 7fece9600700 1 -- 172.21.15.1:0/2474830517 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b16494fb60 con 0x55b164bd0010 2024-02-02T15:28:29.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.605+0000 7fecea000700 1 --2- 172.21.15.1:0/2474830517 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b164bdb7e0 0x55b164be0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:28:29.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.605+0000 7fece9600700 1 --2- 172.21.15.1:0/2474830517 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b164bd0010 0x55b164be0310 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fecd401fe50 tx=0x7fecd401fe80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=200c5749943c7beb server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:28:29.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.605+0000 7fecd2a00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:28:29.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.605+0000 7fecd2a00700 1 -- 172.21.15.1:0/2474830517 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd400ebd0 con 0x55b164bd0010 2024-02-02T15:28:29.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.605+0000 7fecd2a00700 1 -- 172.21.15.1:0/2474830517 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fecd4005da0 con 0x55b164bd0010 2024-02-02T15:28:29.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.605+0000 7fecffb19b80 1 -- 172.21.15.1:0/2474830517 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fecdc01b320 con 0x55b164bd0010 2024-02-02T15:28:29.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.606+0000 7fecd2a00700 1 -- 172.21.15.1:0/2474830517 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fecd401f990 con 0x55b164bd0010 2024-02-02T15:28:29.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.606+0000 7fecffb19b80 10 client.26030 fetch_fsmap learned FSMap version 27 2024-02-02T15:28:29.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.606+0000 7fecffb19b80 1 -- 172.21.15.1:0/2474830517 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7fece410e4b0 con 0x55b164bd0010 2024-02-02T15:28:29.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.606+0000 7fecd2a00700 1 -- 172.21.15.1:0/2474830517 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fecdc01b320 con 0x55b164bd0010 2024-02-02T15:28:29.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.606+0000 7fecd2a00700 1 -- 172.21.15.1:0/2474830517 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fecd4029be0 con 0x55b164bd0010 2024-02-02T15:28:29.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.606+0000 7fecffb19b80 10 client.26030 fetch_fsmap finished waiting for FSMap version 27 2024-02-02T15:28:29.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.606+0000 7fecffb19b80 10 client.26030 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-02-02T15:28:29.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.606+0000 7fecffb19b80 20 client.26030 populate_metadata read hostname 'smithi001' 2024-02-02T15:28:29.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.606+0000 7fecffb19b80 1 --2- 172.21.15.1:0/2474830517 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55b164c3d680 0x55b164c3faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:29.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.606+0000 7fecffb19b80 4 client.26030 mds_command: new command op to 24514 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:28:29.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.606+0000 7fecffb19b80 1 -- 172.21.15.1:0/2474830517 --> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b164ae0f50 con 0x55b164c3d680 2024-02-02T15:28:29.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.606+0000 7fece8c00700 1 --2- 172.21.15.1:0/2474830517 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55b164c3d680 0x55b164c3faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:29.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.607+0000 7fece8c00700 1 --2- 172.21.15.1:0/2474830517 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55b164c3d680 0x55b164c3faa0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e7523f48742c23df server_cookie=74c33805261a8075 in_seq=0 out_seq=0 2024-02-02T15:28:29.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.607+0000 7fecd2a00700 10 client.26030 ms_handle_connect on v2:172.21.15.142:6836/2870289019 2024-02-02T15:28:29.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.607+0000 7fecd2a00700 1 -- 172.21.15.1:0/2474830517 <== mds.3 v2:172.21.15.142:6836/2870289019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b164ae0f50 con 0x55b164c3d680 2024-02-02T15:28:29.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.607+0000 7fecd2a00700 10 client.26030 handle_command_reply: tid=0 2024-02-02T15:28:29.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.607+0000 7fecffb19b80 1 client.26030 shutdown 2024-02-02T15:28:29.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.607+0000 7fecffb19b80 1 -- 172.21.15.1:0/2474830517 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b164bd0010 msgr2=0x55b164be0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:28:29.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.607+0000 7fecffb19b80 1 --2- 172.21.15.1:0/2474830517 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b164bd0010 0x55b164be0310 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fecd401fe50 tx=0x7fecd401fe80 comp rx=0 tx=0).stop 2024-02-02T15:28:29.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.607+0000 7fecffb19b80 1 -- 172.21.15.1:0/2474830517 shutdown_connections 2024-02-02T15:28:29.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.607+0000 7fecffb19b80 1 --2- 172.21.15.1:0/2474830517 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55b164c3d680 0x55b164c3faa0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.607+0000 7fecffb19b80 1 --2- 172.21.15.1:0/2474830517 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b164bdb7e0 0x55b164be0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.607+0000 7fecffb19b80 1 --2- 172.21.15.1:0/2474830517 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b164bd0010 0x55b164be0310 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.607+0000 7fecffb19b80 1 --2- 172.21.15.1:0/2474830517 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b164bd0930 0x55b164be0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.607+0000 7fecd2a00700 0 client.26030 ms_handle_reset on v2:172.21.15.142:6836/2870289019 2024-02-02T15:28:29.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.607+0000 7fecffb19b80 1 -- 172.21.15.1:0/2474830517 >> 172.21.15.1:0/2474830517 conn(0x55b164b2fdb0 msgr2=0x55b164be6900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:28:29.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.608+0000 7fecffb19b80 1 -- 172.21.15.1:0/2474830517 shutdown_connections 2024-02-02T15:28:29.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.608+0000 7fecffb19b80 1 -- 172.21.15.1:0/2474830517 wait complete. 2024-02-02T15:28:29.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.608+0000 7fecffb19b80 20 client.26030 trim_cache size 0 max 16384 2024-02-02T15:28:29.613 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:28:29.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.608+0000 7fecec000700 1 -- 172.21.15.1:0/1726427232 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fecc40a07b0 msgr2=0x7fecc40a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:28:29.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.608+0000 7fecec000700 1 --2- 172.21.15.1:0/1726427232 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fecc40a07b0 0x7fecc40a2c70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fecd4005fd0 tx=0x7fecd400fe10 comp rx=0 tx=0).stop 2024-02-02T15:28:29.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.608+0000 7fecec000700 1 -- 172.21.15.1:0/1726427232 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fece419dab0 msgr2=0x7fece419df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:28:29.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.608+0000 7fecec000700 1 --2- 172.21.15.1:0/1726427232 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fece419dab0 0x7fece419df30 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fecdc00ef30 tx=0x7fecdc00c3a0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.608+0000 7fecec000700 1 -- 172.21.15.1:0/1726427232 shutdown_connections 2024-02-02T15:28:29.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.608+0000 7fecec000700 1 --2- 172.21.15.1:0/1726427232 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fecc40a07b0 0x7fecc40a2c70 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.608+0000 7fecec000700 1 --2- 172.21.15.1:0/1726427232 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fece419dab0 0x7fece419df30 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.608+0000 7fecec000700 1 --2- 172.21.15.1:0/1726427232 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fece4100fa0 0x7fece4199110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.608+0000 7fecec000700 1 --2- 172.21.15.1:0/1726427232 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fece41060c0 0x7fece4199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.608+0000 7fecec000700 1 -- 172.21.15.1:0/1726427232 >> 172.21.15.1:0/1726427232 conn(0x7fece40fc800 msgr2=0x7fece40fd520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:28:29.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.609+0000 7fecec000700 1 -- 172.21.15.1:0/1726427232 shutdown_connections 2024-02-02T15:28:29.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.609+0000 7fecec000700 1 -- 172.21.15.1:0/1726427232 wait complete. 2024-02-02T15:28:29.624 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:28:29 smithi001 ceph-mon[35470]: pgmap v47: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-02T15:28:29.624 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:28:29 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/3033400681' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:28:29.626 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-02T15:28:29.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:28:29 smithi142 ceph-mon[45728]: pgmap v47: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-02T15:28:29.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:28:29 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/3033400681' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:28:29.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.841+0000 7f0e9b400700 1 -- 172.21.15.1:0/1189769489 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e94100ff0 msgr2=0x7f0e941013d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:28:29.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.841+0000 7f0e9b400700 1 --2- 172.21.15.1:0/1189769489 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e94100ff0 0x7f0e941013d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0e84009990 tx=0x7f0e84009ca0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.841+0000 7f0e9b400700 1 -- 172.21.15.1:0/1189769489 shutdown_connections 2024-02-02T15:28:29.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.841+0000 7f0e9b400700 1 --2- 172.21.15.1:0/1189769489 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e94106110 0x7f0e9410dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.841+0000 7f0e9b400700 1 --2- 172.21.15.1:0/1189769489 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e94100ff0 0x7f0e941013d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.841+0000 7f0e9b400700 1 --2- 172.21.15.1:0/1189769489 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e941019a0 0x7f0e941059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.841+0000 7f0e9b400700 1 -- 172.21.15.1:0/1189769489 >> 172.21.15.1:0/1189769489 conn(0x7f0e940fc810 msgr2=0x7f0e940fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:28:29.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.841+0000 7f0e9b400700 1 -- 172.21.15.1:0/1189769489 shutdown_connections 2024-02-02T15:28:29.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.841+0000 7f0e9b400700 1 -- 172.21.15.1:0/1189769489 wait complete. 2024-02-02T15:28:29.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.842+0000 7f0e9b400700 1 Processor -- start 2024-02-02T15:28:29.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.842+0000 7f0e9b400700 1 -- start start 2024-02-02T15:28:29.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.843+0000 7f0e9b400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e94100ff0 0x7f0e94073040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:29.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.843+0000 7f0e9b400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e94106110 0x7f0e94073580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:29.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.843+0000 7f0e9b400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e94073ac0 0x7f0e941a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:29.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.843+0000 7f0e9b400700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e941a5700 con 0x7f0e94106110 2024-02-02T15:28:29.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.843+0000 7f0e9b400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e941a5870 con 0x7f0e94073ac0 2024-02-02T15:28:29.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.843+0000 7f0e9b400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e941a5a30 con 0x7f0e94100ff0 2024-02-02T15:28:29.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.843+0000 7f0e93e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e94106110 0x7f0e94073580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:29.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.843+0000 7f0e99400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e94073ac0 0x7f0e941a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:29.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.843+0000 7f0e93e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e94106110 0x7f0e94073580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49686/0 (socket says 172.21.15.1:49686) 2024-02-02T15:28:29.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.843+0000 7f0e93e00700 1 -- 172.21.15.1:0/1063413568 learned_addr learned my addr 172.21.15.1:0/1063413568 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:28:29.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.843+0000 7f0e98a00700 1 --2- 172.21.15.1:0/1063413568 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e94100ff0 0x7f0e94073040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:29.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.843+0000 7f0e93e00700 1 -- 172.21.15.1:0/1063413568 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e94100ff0 msgr2=0x7f0e94073040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:28:29.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.843+0000 7f0e93e00700 1 --2- 172.21.15.1:0/1063413568 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e94100ff0 0x7f0e94073040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.843+0000 7f0e93e00700 1 -- 172.21.15.1:0/1063413568 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e94073ac0 msgr2=0x7f0e941a5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:28:29.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.843+0000 7f0e93e00700 1 --2- 172.21.15.1:0/1063413568 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e94073ac0 0x7f0e941a5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.843+0000 7f0e93e00700 1 -- 172.21.15.1:0/1063413568 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e840095f0 con 0x7f0e94106110 2024-02-02T15:28:29.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.843+0000 7f0e99400700 1 --2- 172.21.15.1:0/1063413568 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e94073ac0 0x7f0e941a5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:28:29.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.844+0000 7f0e93e00700 1 --2- 172.21.15.1:0/1063413568 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e94106110 0x7f0e94073580 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c00e970 tx=0x7f0e7c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:28:29.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.844+0000 7f0e91600700 1 -- 172.21.15.1:0/1063413568 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c009790 con 0x7f0e94106110 2024-02-02T15:28:29.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.844+0000 7f0e91600700 1 -- 172.21.15.1:0/1063413568 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e7c00c7e0 con 0x7f0e94106110 2024-02-02T15:28:29.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.844+0000 7f0e9b400700 1 -- 172.21.15.1:0/1063413568 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e941a5d10 con 0x7f0e94106110 2024-02-02T15:28:29.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.844+0000 7f0e91600700 1 -- 172.21.15.1:0/1063413568 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c0054e0 con 0x7f0e94106110 2024-02-02T15:28:29.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.844+0000 7f0e9b400700 1 -- 172.21.15.1:0/1063413568 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e941a61d0 con 0x7f0e94106110 2024-02-02T15:28:29.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.848+0000 7f0e91600700 1 -- 172.21.15.1:0/1063413568 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 26) v1 ==== 135302+0+0 (secure 0 0 0) 0x7f0e7c00c950 con 0x7f0e94106110 2024-02-02T15:28:29.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.848+0000 7f0eaef09b80 1 Processor -- start 2024-02-02T15:28:29.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.849+0000 7f0eaef09b80 1 -- start start 2024-02-02T15:28:29.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.849+0000 7f0eaef09b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56171c558010 0x56171c5583f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:29.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.849+0000 7f0eaef09b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56171c558930 0x56171c5632a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:29.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.849+0000 7f0eaef09b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56171c5637e0 0x56171c565bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:29.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.849+0000 7f0eaef09b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56171c4cc4c0 con 0x56171c558010 2024-02-02T15:28:29.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.849+0000 7f0eaef09b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56171c4b0aa0 con 0x56171c558930 2024-02-02T15:28:29.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.849+0000 7f0eaef09b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56171c4ae460 con 0x56171c5637e0 2024-02-02T15:28:29.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.849+0000 7f0e99400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56171c558010 0x56171c5583f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:29.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.849+0000 7f0e98a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56171c558930 0x56171c5632a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:29.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.849+0000 7f0e99400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56171c558010 0x56171c5583f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49698/0 (socket says 172.21.15.1:49698) 2024-02-02T15:28:29.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.849+0000 7f0e99400700 1 -- 172.21.15.1:0/530361643 learned_addr learned my addr 172.21.15.1:0/530361643 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:28:29.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.849+0000 7f0e93e00700 1 --2- 172.21.15.1:0/530361643 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56171c5637e0 0x56171c565bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:29.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.850+0000 7f0e98a00700 1 -- 172.21.15.1:0/530361643 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56171c5637e0 msgr2=0x56171c565bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:28:29.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.850+0000 7f0e98a00700 1 --2- 172.21.15.1:0/530361643 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56171c5637e0 0x56171c565bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.850+0000 7f0e98a00700 1 -- 172.21.15.1:0/530361643 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56171c558010 msgr2=0x56171c5583f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:28:29.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.850+0000 7f0e98a00700 1 --2- 172.21.15.1:0/530361643 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56171c558010 0x56171c5583f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.850+0000 7f0e98a00700 1 -- 172.21.15.1:0/530361643 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56171c32b3b0 con 0x56171c558930 2024-02-02T15:28:29.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.850+0000 7f0e98a00700 1 --2- 172.21.15.1:0/530361643 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56171c558930 0x56171c5632a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0e840098e0 tx=0x7f0e84005ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43409b50002820b5 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:28:29.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.850+0000 7f0e91600700 1 --2- 172.21.15.1:0/1063413568 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f0e740a07b0 0x7f0e740a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:29.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.850+0000 7f0e91600700 1 -- 172.21.15.1:0/1063413568 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f0e7c014070 con 0x7f0e94106110 2024-02-02T15:28:29.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.850+0000 7f0e89c00700 1 -- 172.21.15.1:0/530361643 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e8401d030 con 0x56171c558930 2024-02-02T15:28:29.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.850+0000 7f0e99400700 1 --2- 172.21.15.1:0/1063413568 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f0e740a07b0 0x7f0e740a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:29.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.850+0000 7f0e89c00700 1 -- 172.21.15.1:0/530361643 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e8401e040 con 0x56171c558930 2024-02-02T15:28:29.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.850+0000 7f0e89c00700 1 -- 172.21.15.1:0/530361643 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e84004880 con 0x56171c558930 2024-02-02T15:28:29.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.851+0000 7f0eaef09b80 1 -- 172.21.15.1:0/530361643 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56171c558930 msgr2=0x56171c5632a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:28:29.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.851+0000 7f0eaef09b80 1 --2- 172.21.15.1:0/530361643 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56171c558930 0x56171c5632a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0e840098e0 tx=0x7f0e84005ea0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.851+0000 7f0e99400700 1 --2- 172.21.15.1:0/1063413568 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f0e740a07b0 0x7f0e740a2c70 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c009fd0 tx=0x7f0e8c00b040 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:28:29.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.851+0000 7f0eaef09b80 1 -- 172.21.15.1:0/530361643 shutdown_connections 2024-02-02T15:28:29.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.851+0000 7f0eaef09b80 1 --2- 172.21.15.1:0/530361643 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56171c5637e0 0x56171c565bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.851+0000 7f0eaef09b80 1 --2- 172.21.15.1:0/530361643 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56171c558010 0x56171c5583f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.851+0000 7f0eaef09b80 1 --2- 172.21.15.1:0/530361643 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56171c558930 0x56171c5632a0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.851+0000 7f0eaef09b80 1 -- 172.21.15.1:0/530361643 >> 172.21.15.1:0/530361643 conn(0x56171c4b7db0 msgr2=0x56171c5543b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:28:29.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.851+0000 7f0eaef09b80 1 -- 172.21.15.1:0/530361643 shutdown_connections 2024-02-02T15:28:29.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.851+0000 7f0eaef09b80 1 -- 172.21.15.1:0/530361643 wait complete. 2024-02-02T15:28:29.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.852+0000 7f0eaef09b80 1 -- start start 2024-02-02T15:28:29.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.852+0000 7f0eaef09b80 1 Processor -- start 2024-02-02T15:28:29.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.852+0000 7f0eaef09b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56171c558010 0x56171c563220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:29.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.852+0000 7f0eaef09b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56171c558930 0x56171c5604a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:29.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.852+0000 7f0eaef09b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56171c5637e0 0x56171c560be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:29.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.853+0000 7f0eaef09b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56171c4a9d50 con 0x56171c558010 2024-02-02T15:28:29.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.853+0000 7f0eaef09b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56171c445d80 con 0x56171c5637e0 2024-02-02T15:28:29.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.853+0000 7f0eaef09b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56171c445a20 con 0x56171c558930 2024-02-02T15:28:29.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.853+0000 7f0eaef09b80 10 client.0 _ll_register_callbacks cb 0x56171c447290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:28:29.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.853+0000 7f0e98a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56171c558010 0x56171c563220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:29.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.853+0000 7f0e93e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56171c558930 0x56171c5604a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:29.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.853+0000 7f0e99400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56171c5637e0 0x56171c560be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:29.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.853+0000 7f0e98a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56171c558010 0x56171c563220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49708/0 (socket says 172.21.15.1:49708) 2024-02-02T15:28:29.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.853+0000 7f0e93e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56171c558930 0x56171c5604a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:47666/0 (socket says 172.21.15.1:47666) 2024-02-02T15:28:29.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.853+0000 7f0e98a00700 1 -- 172.21.15.1:0/1299807979 learned_addr learned my addr 172.21.15.1:0/1299807979 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:28:29.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.853+0000 7f0e99400700 1 -- 172.21.15.1:0/1299807979 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56171c558930 msgr2=0x56171c5604a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:28:29.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.853+0000 7f0e99400700 1 --2- 172.21.15.1:0/1299807979 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56171c558930 0x56171c5604a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.853+0000 7f0e99400700 1 -- 172.21.15.1:0/1299807979 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56171c558010 msgr2=0x56171c563220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:28:29.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.853+0000 7f0e99400700 1 --2- 172.21.15.1:0/1299807979 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56171c558010 0x56171c563220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.853+0000 7f0e99400700 1 -- 172.21.15.1:0/1299807979 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e9410e500 con 0x56171c5637e0 2024-02-02T15:28:29.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.854+0000 7f0e99400700 1 --2- 172.21.15.1:0/1299807979 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56171c5637e0 0x56171c560be0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c004950 tx=0x7f0e8c004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4cc9c6837321a7a server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:28:29.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.854+0000 7f0e82000700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:28:29.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.854+0000 7f0e82000700 1 -- 172.21.15.1:0/1299807979 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e8c004db0 con 0x56171c5637e0 2024-02-02T15:28:29.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.854+0000 7f0eaef09b80 1 -- 172.21.15.1:0/1299807979 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56171c54c0f0 con 0x56171c5637e0 2024-02-02T15:28:29.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.854+0000 7f0e82000700 1 -- 172.21.15.1:0/1299807979 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f0e8c016e30 con 0x56171c5637e0 2024-02-02T15:28:29.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.854+0000 7f0e82000700 1 -- 172.21.15.1:0/1299807979 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0e8c003db0 con 0x56171c5637e0 2024-02-02T15:28:29.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.854+0000 7f0eaef09b80 10 client.26042 fetch_fsmap learned FSMap version 27 2024-02-02T15:28:29.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.854+0000 7f0eaef09b80 1 -- 172.21.15.1:0/1299807979 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56171c2d7b60 con 0x56171c5637e0 2024-02-02T15:28:29.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.854+0000 7f0e82000700 1 -- 172.21.15.1:0/1299807979 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56171c54c0f0 con 0x56171c5637e0 2024-02-02T15:28:29.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.854+0000 7f0e82000700 1 -- 172.21.15.1:0/1299807979 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0e8c01e070 con 0x56171c5637e0 2024-02-02T15:28:29.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.855+0000 7f0eaef09b80 10 client.26042 fetch_fsmap finished waiting for FSMap version 27 2024-02-02T15:28:29.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.855+0000 7f0eaef09b80 10 client.26042 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-02-02T15:28:29.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.855+0000 7f0eaef09b80 20 client.26042 populate_metadata read hostname 'smithi001' 2024-02-02T15:28:29.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.855+0000 7f0eaef09b80 1 --2- 172.21.15.1:0/1299807979 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x56171c573150 0x56171c575570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:29.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.855+0000 7f0eaef09b80 4 client.26042 mds_command: new command op to 24518 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:28:29.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.855+0000 7f0eaef09b80 1 -- 172.21.15.1:0/1299807979 --> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56171c4a2dd0 con 0x56171c573150 2024-02-02T15:28:29.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.855+0000 7f0e98a00700 1 --2- 172.21.15.1:0/1299807979 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x56171c573150 0x56171c575570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:29.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.855+0000 7f0e98a00700 1 --2- 172.21.15.1:0/1299807979 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x56171c573150 0x56171c575570 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2991bc8c179cd268 server_cookie=e134211646038b7f in_seq=0 out_seq=0 2024-02-02T15:28:29.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.855+0000 7f0e82000700 10 client.26042 ms_handle_connect on v2:172.21.15.71:6836/628016199 2024-02-02T15:28:29.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.856+0000 7f0e82000700 1 -- 172.21.15.1:0/1299807979 <== mds.2 v2:172.21.15.71:6836/628016199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x56171c4a2dd0 con 0x56171c573150 2024-02-02T15:28:29.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.856+0000 7f0e82000700 10 client.26042 handle_command_reply: tid=0 2024-02-02T15:28:29.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.856+0000 7f0eaef09b80 1 client.26042 shutdown 2024-02-02T15:28:29.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.856+0000 7f0eaef09b80 1 -- 172.21.15.1:0/1299807979 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56171c5637e0 msgr2=0x56171c560be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:28:29.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.856+0000 7f0eaef09b80 1 --2- 172.21.15.1:0/1299807979 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56171c5637e0 0x56171c560be0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c004950 tx=0x7f0e8c004b40 comp rx=0 tx=0).stop 2024-02-02T15:28:29.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.856+0000 7f0eaef09b80 1 -- 172.21.15.1:0/1299807979 shutdown_connections 2024-02-02T15:28:29.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.856+0000 7f0eaef09b80 1 --2- 172.21.15.1:0/1299807979 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56171c5637e0 0x56171c560be0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.856+0000 7f0eaef09b80 1 --2- 172.21.15.1:0/1299807979 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56171c558010 0x56171c563220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.856+0000 7f0eaef09b80 1 --2- 172.21.15.1:0/1299807979 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x56171c573150 0x56171c575570 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.856+0000 7f0eaef09b80 1 --2- 172.21.15.1:0/1299807979 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56171c558930 0x56171c5604a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.856+0000 7f0e82000700 0 client.26042 ms_handle_reset on v2:172.21.15.71:6836/628016199 2024-02-02T15:28:29.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.856+0000 7f0eaef09b80 1 -- 172.21.15.1:0/1299807979 >> 172.21.15.1:0/1299807979 conn(0x56171c4b7db0 msgr2=0x56171c553fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:28:29.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.856+0000 7f0eaef09b80 1 -- 172.21.15.1:0/1299807979 shutdown_connections 2024-02-02T15:28:29.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.856+0000 7f0eaef09b80 1 -- 172.21.15.1:0/1299807979 wait complete. 2024-02-02T15:28:29.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.857+0000 7f0eaef09b80 20 client.26042 trim_cache size 0 max 16384 2024-02-02T15:28:29.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:28:29 smithi071 ceph-mon[44707]: pgmap v47: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-02T15:28:29.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:28:29 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/3033400681' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:28:29.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.870+0000 7f0eaef09b80 1 Processor -- start 2024-02-02T15:28:29.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.870+0000 7f0eaef09b80 1 -- start start 2024-02-02T15:28:29.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.871+0000 7f0eaef09b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56171c558010 0x56171c55b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:29.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.871+0000 7f0eaef09b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56171c558930 0x56171c562d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:29.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.871+0000 7f0eaef09b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56171c5637e0 0x56171c55fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:29.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.871+0000 7f0eaef09b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56171c32d6a0 con 0x56171c558930 2024-02-02T15:28:29.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.871+0000 7f0eaef09b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56171c5611e0 con 0x56171c558010 2024-02-02T15:28:29.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.871+0000 7f0eaef09b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56171c5617f0 con 0x56171c5637e0 2024-02-02T15:28:29.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.871+0000 7f0e98a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56171c558010 0x56171c55b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:29.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.871+0000 7f0e99400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56171c5637e0 0x56171c55fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:29.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.871+0000 7f0e93e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56171c558930 0x56171c562d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:29.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.871+0000 7f0e98a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56171c558010 0x56171c55b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:51322/0 (socket says 172.21.15.1:51322) 2024-02-02T15:28:29.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.871+0000 7f0e98a00700 1 -- 172.21.15.1:0/3215113778 learned_addr learned my addr 172.21.15.1:0/3215113778 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:28:29.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.871+0000 7f0e99400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56171c5637e0 0x56171c55fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:47676/0 (socket says 172.21.15.1:47676) 2024-02-02T15:28:29.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.871+0000 7f0e98a00700 1 -- 172.21.15.1:0/3215113778 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56171c5637e0 msgr2=0x56171c55fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:28:29.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.871+0000 7f0e98a00700 1 --2- 172.21.15.1:0/3215113778 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56171c5637e0 0x56171c55fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.871+0000 7f0e98a00700 1 -- 172.21.15.1:0/3215113778 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56171c558930 msgr2=0x56171c562d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:28:29.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.871+0000 7f0e98a00700 1 --2- 172.21.15.1:0/3215113778 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56171c558930 0x56171c562d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.871+0000 7f0e98a00700 1 -- 172.21.15.1:0/3215113778 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56171c46fea0 con 0x56171c558010 2024-02-02T15:28:29.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.872+0000 7f0e99400700 1 --2- 172.21.15.1:0/3215113778 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56171c5637e0 0x56171c55fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:28:29.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.872+0000 7f0e98a00700 1 --2- 172.21.15.1:0/3215113778 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56171c558010 0x56171c55b050 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0e84009ce0 tx=0x7f0e84005fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4d37b417d37f2b3 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:28:29.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.872+0000 7f0e8a600700 1 -- 172.21.15.1:0/3215113778 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e8401d040 con 0x56171c558010 2024-02-02T15:28:29.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.872+0000 7f0e8a600700 1 -- 172.21.15.1:0/3215113778 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e84005950 con 0x56171c558010 2024-02-02T15:28:29.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.872+0000 7f0e8a600700 1 -- 172.21.15.1:0/3215113778 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e84004760 con 0x56171c558010 2024-02-02T15:28:29.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.873+0000 7f0eaef09b80 1 -- 172.21.15.1:0/3215113778 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56171c558010 msgr2=0x56171c55b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:28:29.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.873+0000 7f0eaef09b80 1 --2- 172.21.15.1:0/3215113778 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56171c558010 0x56171c55b050 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f0e84009ce0 tx=0x7f0e84005fb0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.873+0000 7f0eaef09b80 1 -- 172.21.15.1:0/3215113778 shutdown_connections 2024-02-02T15:28:29.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.873+0000 7f0eaef09b80 1 --2- 172.21.15.1:0/3215113778 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56171c5637e0 0x56171c55fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.873+0000 7f0eaef09b80 1 --2- 172.21.15.1:0/3215113778 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56171c558010 0x56171c55b050 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.873+0000 7f0eaef09b80 1 --2- 172.21.15.1:0/3215113778 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56171c558930 0x56171c562d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.873+0000 7f0eaef09b80 1 -- 172.21.15.1:0/3215113778 >> 172.21.15.1:0/3215113778 conn(0x56171c4b7db0 msgr2=0x56171c56e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:28:29.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.873+0000 7f0eaef09b80 1 -- 172.21.15.1:0/3215113778 shutdown_connections 2024-02-02T15:28:29.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.873+0000 7f0eaef09b80 1 -- 172.21.15.1:0/3215113778 wait complete. 2024-02-02T15:28:29.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.873+0000 7f0eaef09b80 1 -- start start 2024-02-02T15:28:29.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.874+0000 7f0eaef09b80 1 Processor -- start 2024-02-02T15:28:29.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.874+0000 7f0eaef09b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56171c558010 0x56171c569a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:29.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.874+0000 7f0eaef09b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56171c558930 0x56171c569f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:29.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.874+0000 7f0eaef09b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56171c5637e0 0x56171c567e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:29.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.874+0000 7f0eaef09b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56171c5684b0 con 0x56171c558930 2024-02-02T15:28:29.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.874+0000 7f0eaef09b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56171c568620 con 0x56171c558010 2024-02-02T15:28:29.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.874+0000 7f0eaef09b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56171c5687e0 con 0x56171c5637e0 2024-02-02T15:28:29.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.874+0000 7f0eaef09b80 10 client.0 _ll_register_callbacks cb 0x56171c447290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:28:29.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.874+0000 7f0e98a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56171c558010 0x56171c569a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:29.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.874+0000 7f0e98a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56171c558010 0x56171c569a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:51330/0 (socket says 172.21.15.1:51330) 2024-02-02T15:28:29.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.874+0000 7f0e98a00700 1 -- 172.21.15.1:0/3882053411 learned_addr learned my addr 172.21.15.1:0/3882053411 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:28:29.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.874+0000 7f0e93e00700 1 --2- 172.21.15.1:0/3882053411 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56171c558930 0x56171c569f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:29.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.875+0000 7f0e99400700 1 --2- 172.21.15.1:0/3882053411 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56171c5637e0 0x56171c567e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:29.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.875+0000 7f0e98a00700 1 -- 172.21.15.1:0/3882053411 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56171c5637e0 msgr2=0x56171c567e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:28:29.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.875+0000 7f0e98a00700 1 --2- 172.21.15.1:0/3882053411 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56171c5637e0 0x56171c567e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.875+0000 7f0e98a00700 1 -- 172.21.15.1:0/3882053411 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56171c558930 msgr2=0x56171c569f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:28:29.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.875+0000 7f0e98a00700 1 --2- 172.21.15.1:0/3882053411 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56171c558930 0x56171c569f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.875+0000 7f0e98a00700 1 -- 172.21.15.1:0/3882053411 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56171c2d7b60 con 0x56171c558010 2024-02-02T15:28:29.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.875+0000 7f0e98a00700 1 --2- 172.21.15.1:0/3882053411 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56171c558010 0x56171c569a10 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0e84005ac0 tx=0x7f0e84005590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0db84223414fe8d server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:28:29.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.875+0000 7f0e82000700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:28:29.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.875+0000 7f0e82000700 1 -- 172.21.15.1:0/3882053411 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e8401d040 con 0x56171c558010 2024-02-02T15:28:29.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.875+0000 7f0eaef09b80 1 -- 172.21.15.1:0/3882053411 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0e8401e040 con 0x56171c558010 2024-02-02T15:28:29.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.875+0000 7f0e82000700 1 -- 172.21.15.1:0/3882053411 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f0e8400bc30 con 0x56171c558010 2024-02-02T15:28:29.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.876+0000 7f0e82000700 1 -- 172.21.15.1:0/3882053411 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0e84032960 con 0x56171c558010 2024-02-02T15:28:29.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.876+0000 7f0eaef09b80 10 client.26048 fetch_fsmap learned FSMap version 27 2024-02-02T15:28:29.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.876+0000 7f0eaef09b80 1 -- 172.21.15.1:0/3882053411 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56171c32b3b0 con 0x56171c558010 2024-02-02T15:28:29.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.876+0000 7f0e82000700 1 -- 172.21.15.1:0/3882053411 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e8401e040 con 0x56171c558010 2024-02-02T15:28:29.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.876+0000 7f0e82000700 1 -- 172.21.15.1:0/3882053411 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0e8400fca0 con 0x56171c558010 2024-02-02T15:28:29.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.876+0000 7f0eaef09b80 10 client.26048 fetch_fsmap finished waiting for FSMap version 27 2024-02-02T15:28:29.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.876+0000 7f0eaef09b80 10 client.26048 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-02-02T15:28:29.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.876+0000 7f0eaef09b80 20 client.26048 populate_metadata read hostname 'smithi001' 2024-02-02T15:28:29.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.876+0000 7f0eaef09b80 1 --2- 172.21.15.1:0/3882053411 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x56171c5c5850 0x56171c5c7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:29.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.876+0000 7f0eaef09b80 4 client.26048 mds_command: new command op to 24518 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:28:29.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.876+0000 7f0eaef09b80 1 -- 172.21.15.1:0/3882053411 --> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56171c468f50 con 0x56171c5c5850 2024-02-02T15:28:29.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.876+0000 7f0e93e00700 1 --2- 172.21.15.1:0/3882053411 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x56171c5c5850 0x56171c5c7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:29.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.877+0000 7f0e93e00700 1 --2- 172.21.15.1:0/3882053411 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x56171c5c5850 0x56171c5c7c40 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=605d6706a6b9c678 server_cookie=50414bc61ca79227 in_seq=0 out_seq=0 2024-02-02T15:28:29.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.877+0000 7f0e82000700 10 client.26048 ms_handle_connect on v2:172.21.15.71:6836/628016199 2024-02-02T15:28:29.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.877+0000 7f0e82000700 1 -- 172.21.15.1:0/3882053411 <== mds.2 v2:172.21.15.71:6836/628016199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56171c468f50 con 0x56171c5c5850 2024-02-02T15:28:29.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.877+0000 7f0e82000700 10 client.26048 handle_command_reply: tid=0 2024-02-02T15:28:29.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.877+0000 7f0eaef09b80 1 client.26048 shutdown 2024-02-02T15:28:29.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.878+0000 7f0eaef09b80 1 -- 172.21.15.1:0/3882053411 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56171c558010 msgr2=0x56171c569a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:28:29.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.878+0000 7f0eaef09b80 1 --2- 172.21.15.1:0/3882053411 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56171c558010 0x56171c569a10 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f0e84005ac0 tx=0x7f0e84005590 comp rx=0 tx=0).stop 2024-02-02T15:28:29.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.878+0000 7f0eaef09b80 1 -- 172.21.15.1:0/3882053411 shutdown_connections 2024-02-02T15:28:29.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.878+0000 7f0eaef09b80 1 --2- 172.21.15.1:0/3882053411 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x56171c5c5850 0x56171c5c7c40 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.878+0000 7f0eaef09b80 1 --2- 172.21.15.1:0/3882053411 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56171c5637e0 0x56171c567e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.878+0000 7f0eaef09b80 1 --2- 172.21.15.1:0/3882053411 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56171c558010 0x56171c569a10 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.878+0000 7f0eaef09b80 1 --2- 172.21.15.1:0/3882053411 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56171c558930 0x56171c569f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.878+0000 7f0eaef09b80 1 -- 172.21.15.1:0/3882053411 >> 172.21.15.1:0/3882053411 conn(0x56171c4b7db0 msgr2=0x56171c5535c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:28:29.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.878+0000 7f0e82000700 0 client.26048 ms_handle_reset on v2:172.21.15.71:6836/628016199 2024-02-02T15:28:29.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.878+0000 7f0eaef09b80 1 -- 172.21.15.1:0/3882053411 shutdown_connections 2024-02-02T15:28:29.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.878+0000 7f0eaef09b80 1 -- 172.21.15.1:0/3882053411 wait complete. 2024-02-02T15:28:29.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.878+0000 7f0eaef09b80 20 client.26048 trim_cache size 0 max 16384 2024-02-02T15:28:29.884 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:28:29.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.878+0000 7f0e9b400700 1 -- 172.21.15.1:0/1063413568 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f0e740a07b0 msgr2=0x7f0e740a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:28:29.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.878+0000 7f0e9b400700 1 --2- 172.21.15.1:0/1063413568 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f0e740a07b0 0x7f0e740a2c70 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c009fd0 tx=0x7f0e8c00b040 comp rx=0 tx=0).stop 2024-02-02T15:28:29.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.878+0000 7f0e9b400700 1 -- 172.21.15.1:0/1063413568 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e94106110 msgr2=0x7f0e94073580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:28:29.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.878+0000 7f0e9b400700 1 --2- 172.21.15.1:0/1063413568 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e94106110 0x7f0e94073580 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c00e970 tx=0x7f0e7c00ec80 comp rx=0 tx=0).stop 2024-02-02T15:28:29.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.879+0000 7f0e9b400700 1 -- 172.21.15.1:0/1063413568 shutdown_connections 2024-02-02T15:28:29.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.879+0000 7f0e9b400700 1 --2- 172.21.15.1:0/1063413568 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f0e740a07b0 0x7f0e740a2c70 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.879+0000 7f0e9b400700 1 --2- 172.21.15.1:0/1063413568 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e94073ac0 0x7f0e941a5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.879+0000 7f0e9b400700 1 --2- 172.21.15.1:0/1063413568 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e94100ff0 0x7f0e94073040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.879+0000 7f0e9b400700 1 --2- 172.21.15.1:0/1063413568 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e94106110 0x7f0e94073580 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:29.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.879+0000 7f0e9b400700 1 -- 172.21.15.1:0/1063413568 >> 172.21.15.1:0/1063413568 conn(0x7f0e940fc810 msgr2=0x7f0e94104a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:28:29.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.879+0000 7f0e9b400700 1 -- 172.21.15.1:0/1063413568 shutdown_connections 2024-02-02T15:28:29.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:29.879+0000 7f0e9b400700 1 -- 172.21.15.1:0/1063413568 wait complete. 2024-02-02T15:28:29.898 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-02T15:28:30.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.109+0000 7fb82dc00700 1 -- 172.21.15.1:0/901717439 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb8281019a0 msgr2=0x7fb8281059e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:28:30.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.109+0000 7fb82dc00700 1 --2- 172.21.15.1:0/901717439 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb8281019a0 0x7fb8281059e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb810009990 tx=0x7fb810009ca0 comp rx=0 tx=0).stop 2024-02-02T15:28:30.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.109+0000 7fb82dc00700 1 -- 172.21.15.1:0/901717439 shutdown_connections 2024-02-02T15:28:30.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.109+0000 7fb82dc00700 1 --2- 172.21.15.1:0/901717439 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb828106110 0x7fb82810dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:30.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.109+0000 7fb82dc00700 1 --2- 172.21.15.1:0/901717439 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb828100ff0 0x7fb8281013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:30.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.109+0000 7fb82dc00700 1 --2- 172.21.15.1:0/901717439 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb8281019a0 0x7fb8281059e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:30.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.109+0000 7fb82dc00700 1 -- 172.21.15.1:0/901717439 >> 172.21.15.1:0/901717439 conn(0x7fb8280fc810 msgr2=0x7fb8280fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:28:30.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.110+0000 7fb82dc00700 1 -- 172.21.15.1:0/901717439 shutdown_connections 2024-02-02T15:28:30.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.110+0000 7fb82dc00700 1 -- 172.21.15.1:0/901717439 wait complete. 2024-02-02T15:28:30.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.111+0000 7fb82dc00700 1 Processor -- start 2024-02-02T15:28:30.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.111+0000 7fb82dc00700 1 -- start start 2024-02-02T15:28:30.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.112+0000 7fb82dc00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb828100ff0 0x7fb828073040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:30.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.112+0000 7fb82dc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb828106110 0x7fb828073580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:30.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.112+0000 7fb82dc00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb828073ac0 0x7fb8281a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:30.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.112+0000 7fb82dc00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8281a5700 con 0x7fb828073ac0 2024-02-02T15:28:30.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.112+0000 7fb82dc00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8281a5870 con 0x7fb828100ff0 2024-02-02T15:28:30.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.112+0000 7fb82dc00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8281a5a30 con 0x7fb828106110 2024-02-02T15:28:30.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.112+0000 7fb826a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb828100ff0 0x7fb828073040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:30.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.112+0000 7fb827400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb828073ac0 0x7fb8281a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:30.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.112+0000 7fb826a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb828100ff0 0x7fb828073040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:51340/0 (socket says 172.21.15.1:51340) 2024-02-02T15:28:30.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.112+0000 7fb826a00700 1 -- 172.21.15.1:0/3153412231 learned_addr learned my addr 172.21.15.1:0/3153412231 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:28:30.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.112+0000 7fb826000700 1 --2- 172.21.15.1:0/3153412231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb828106110 0x7fb828073580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:30.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.112+0000 7fb826a00700 1 -- 172.21.15.1:0/3153412231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb828106110 msgr2=0x7fb828073580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:28:30.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.112+0000 7fb826a00700 1 --2- 172.21.15.1:0/3153412231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb828106110 0x7fb828073580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:30.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.112+0000 7fb826a00700 1 -- 172.21.15.1:0/3153412231 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb828073ac0 msgr2=0x7fb8281a5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:28:30.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.112+0000 7fb826a00700 1 --2- 172.21.15.1:0/3153412231 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb828073ac0 0x7fb8281a5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:30.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.113+0000 7fb826a00700 1 -- 172.21.15.1:0/3153412231 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb820009c40 con 0x7fb828100ff0 2024-02-02T15:28:30.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.113+0000 7fb827400700 1 --2- 172.21.15.1:0/3153412231 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb828073ac0 0x7fb8281a5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:28:30.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.113+0000 7fb826a00700 1 --2- 172.21.15.1:0/3153412231 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb828100ff0 0x7fb828073040 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb81800e9a0 tx=0x7fb81800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:28:30.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.113+0000 7fb81f400700 1 -- 172.21.15.1:0/3153412231 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb818009790 con 0x7fb828100ff0 2024-02-02T15:28:30.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.113+0000 7fb81f400700 1 -- 172.21.15.1:0/3153412231 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb81800ce80 con 0x7fb828100ff0 2024-02-02T15:28:30.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.113+0000 7fb81f400700 1 -- 172.21.15.1:0/3153412231 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb818018680 con 0x7fb828100ff0 2024-02-02T15:28:30.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.113+0000 7fb82dc00700 1 -- 172.21.15.1:0/3153412231 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8100095f0 con 0x7fb828100ff0 2024-02-02T15:28:30.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.113+0000 7fb82dc00700 1 -- 172.21.15.1:0/3153412231 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8281a6040 con 0x7fb828100ff0 2024-02-02T15:28:30.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.116+0000 7fb81f400700 1 -- 172.21.15.1:0/3153412231 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) v1 ==== 135302+0+0 (secure 0 0 0) 0x7fb818010430 con 0x7fb828100ff0 2024-02-02T15:28:30.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.118+0000 7fb841745b80 1 Processor -- start 2024-02-02T15:28:30.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.118+0000 7fb841745b80 1 -- start start 2024-02-02T15:28:30.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.119+0000 7fb841745b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56304aa57010 0x56304aa573f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:30.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.119+0000 7fb841745b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56304aa57930 0x56304aa622a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:30.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.119+0000 7fb841745b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56304aa627e0 0x56304aa64bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:30.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.119+0000 7fb841745b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56304a9cb4c0 con 0x56304aa57010 2024-02-02T15:28:30.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.119+0000 7fb81f400700 1 --2- 172.21.15.1:0/3153412231 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fb8080a07b0 0x7fb8080a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:30.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.119+0000 7fb841745b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56304a9afaa0 con 0x56304aa57930 2024-02-02T15:28:30.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.119+0000 7fb841745b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56304a9ad460 con 0x56304aa627e0 2024-02-02T15:28:30.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.119+0000 7fb81f400700 1 -- 172.21.15.1:0/3153412231 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fb818014070 con 0x7fb828100ff0 2024-02-02T15:28:30.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.119+0000 7fb826a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56304aa57930 0x56304aa622a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:30.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.119+0000 7fb826000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56304aa627e0 0x56304aa64bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:30.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.119+0000 7fb826000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56304aa627e0 0x56304aa64bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:47694/0 (socket says 172.21.15.1:47694) 2024-02-02T15:28:30.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.119+0000 7fb826000700 1 -- 172.21.15.1:0/1774289519 learned_addr learned my addr 172.21.15.1:0/1774289519 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:28:30.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.119+0000 7fb826a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56304aa57930 0x56304aa622a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:51348/0 (socket says 172.21.15.1:51348) 2024-02-02T15:28:30.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.119+0000 7fb827400700 1 --2- 172.21.15.1:0/1774289519 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56304aa57010 0x56304aa573f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:30.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.119+0000 7fb827400700 1 --2- 172.21.15.1:0/3153412231 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fb8080a07b0 0x7fb8080a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:30.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.119+0000 7fb826000700 1 -- 172.21.15.1:0/1774289519 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56304aa57930 msgr2=0x56304aa622a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:28:30.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.120+0000 7fb826000700 1 --2- 172.21.15.1:0/1774289519 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56304aa57930 0x56304aa622a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:30.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.120+0000 7fb826000700 1 -- 172.21.15.1:0/1774289519 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56304aa57010 msgr2=0x56304aa573f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:28:30.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.120+0000 7fb826000700 1 --2- 172.21.15.1:0/1774289519 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56304aa57010 0x56304aa573f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:30.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.120+0000 7fb826000700 1 -- 172.21.15.1:0/1774289519 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56304a82a3b0 con 0x56304aa627e0 2024-02-02T15:28:30.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.120+0000 7fb826000700 1 --2- 172.21.15.1:0/1774289519 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56304aa627e0 0x56304aa64bd0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb810015000 tx=0x7fb810005ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7cdacaa88cf4d82b server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:28:30.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.120+0000 7fb817e00700 1 -- 172.21.15.1:0/1774289519 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81001d030 con 0x56304aa627e0 2024-02-02T15:28:30.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.120+0000 7fb817e00700 1 -- 172.21.15.1:0/1774289519 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb81001e040 con 0x56304aa627e0 2024-02-02T15:28:30.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.120+0000 7fb817e00700 1 -- 172.21.15.1:0/1774289519 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb810004840 con 0x56304aa627e0 2024-02-02T15:28:30.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.121+0000 7fb841745b80 1 -- 172.21.15.1:0/1774289519 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56304aa627e0 msgr2=0x56304aa64bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:28:30.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.121+0000 7fb841745b80 1 --2- 172.21.15.1:0/1774289519 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56304aa627e0 0x56304aa64bd0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb810015000 tx=0x7fb810005ea0 comp rx=0 tx=0).stop 2024-02-02T15:28:30.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.121+0000 7fb827400700 1 --2- 172.21.15.1:0/3153412231 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fb8080a07b0 0x7fb8080a2c70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb820009550 tx=0x7fb82000b040 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:28:30.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.121+0000 7fb841745b80 1 -- 172.21.15.1:0/1774289519 shutdown_connections 2024-02-02T15:28:30.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.121+0000 7fb841745b80 1 --2- 172.21.15.1:0/1774289519 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56304aa627e0 0x56304aa64bd0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:30.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.121+0000 7fb841745b80 1 --2- 172.21.15.1:0/1774289519 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56304aa57010 0x56304aa573f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:30.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.121+0000 7fb841745b80 1 --2- 172.21.15.1:0/1774289519 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56304aa57930 0x56304aa622a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:30.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.121+0000 7fb841745b80 1 -- 172.21.15.1:0/1774289519 >> 172.21.15.1:0/1774289519 conn(0x56304a9b6db0 msgr2=0x56304aa533b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:28:30.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.121+0000 7fb841745b80 1 -- 172.21.15.1:0/1774289519 shutdown_connections 2024-02-02T15:28:30.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.121+0000 7fb841745b80 1 -- 172.21.15.1:0/1774289519 wait complete. 2024-02-02T15:28:30.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.122+0000 7fb841745b80 1 -- start start 2024-02-02T15:28:30.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.122+0000 7fb841745b80 1 Processor -- start 2024-02-02T15:28:30.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.122+0000 7fb841745b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56304aa57010 0x56304aa62170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:30.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.122+0000 7fb841745b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56304aa57930 0x56304aa5f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:30.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.122+0000 7fb841745b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56304aa627e0 0x56304aa5f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:30.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.122+0000 7fb841745b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56304a9a8d50 con 0x56304aa57930 2024-02-02T15:28:30.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.122+0000 7fb841745b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56304a944d80 con 0x56304aa57010 2024-02-02T15:28:30.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.122+0000 7fb841745b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56304a944a20 con 0x56304aa627e0 2024-02-02T15:28:30.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.122+0000 7fb841745b80 10 client.0 _ll_register_callbacks cb 0x56304a946290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:28:30.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.123+0000 7fb827400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56304aa627e0 0x56304aa5f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:30.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.123+0000 7fb826000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56304aa57930 0x56304aa5f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:30.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.123+0000 7fb827400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56304aa627e0 0x56304aa5f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:47706/0 (socket says 172.21.15.1:47706) 2024-02-02T15:28:30.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.123+0000 7fb827400700 1 -- 172.21.15.1:0/2964925359 learned_addr learned my addr 172.21.15.1:0/2964925359 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:28:30.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.123+0000 7fb826a00700 1 --2- 172.21.15.1:0/2964925359 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56304aa57010 0x56304aa62170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:30.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.123+0000 7fb827400700 1 -- 172.21.15.1:0/2964925359 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56304aa57010 msgr2=0x56304aa62170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:28:30.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.123+0000 7fb827400700 1 --2- 172.21.15.1:0/2964925359 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56304aa57010 0x56304aa62170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:30.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.123+0000 7fb827400700 1 -- 172.21.15.1:0/2964925359 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56304aa57930 msgr2=0x56304aa5f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:28:30.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.123+0000 7fb827400700 1 --2- 172.21.15.1:0/2964925359 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56304aa57930 0x56304aa5f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:30.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.123+0000 7fb827400700 1 -- 172.21.15.1:0/2964925359 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb82810e500 con 0x56304aa627e0 2024-02-02T15:28:30.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.123+0000 7fb827400700 1 --2- 172.21.15.1:0/2964925359 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56304aa627e0 0x56304aa5f920 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb820004330 tx=0x7fb820004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8ff9d38675aba7f server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:28:30.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.123+0000 7fb814c00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:28:30.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.124+0000 7fb814c00700 1 -- 172.21.15.1:0/2964925359 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb820020070 con 0x56304aa627e0 2024-02-02T15:28:30.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.124+0000 7fb814c00700 1 -- 172.21.15.1:0/2964925359 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fb820004bb0 con 0x56304aa627e0 2024-02-02T15:28:30.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.124+0000 7fb841745b80 1 -- 172.21.15.1:0/2964925359 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56304aa4b0f0 con 0x56304aa627e0 2024-02-02T15:28:30.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.124+0000 7fb814c00700 1 -- 172.21.15.1:0/2964925359 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb820004d70 con 0x56304aa627e0 2024-02-02T15:28:30.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.124+0000 7fb841745b80 10 client.26056 fetch_fsmap learned FSMap version 27 2024-02-02T15:28:30.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.124+0000 7fb841745b80 1 -- 172.21.15.1:0/2964925359 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56304a7d6b60 con 0x56304aa627e0 2024-02-02T15:28:30.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.124+0000 7fb814c00700 1 -- 172.21.15.1:0/2964925359 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56304aa4b0f0 con 0x56304aa627e0 2024-02-02T15:28:30.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.124+0000 7fb814c00700 1 -- 172.21.15.1:0/2964925359 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb8200198c0 con 0x56304aa627e0 2024-02-02T15:28:30.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.125+0000 7fb841745b80 10 client.26056 fetch_fsmap finished waiting for FSMap version 27 2024-02-02T15:28:30.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.125+0000 7fb841745b80 10 client.26056 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-02T15:28:30.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.125+0000 7fb841745b80 20 client.26056 populate_metadata read hostname 'smithi001' 2024-02-02T15:28:30.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.125+0000 7fb841745b80 1 --2- 172.21.15.1:0/2964925359 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x56304aa72190 0x56304aa745b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:30.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.125+0000 7fb841745b80 4 client.26056 mds_command: new command op to 24529 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:28:30.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.125+0000 7fb841745b80 1 -- 172.21.15.1:0/2964925359 --> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56304a9a1dd0 con 0x56304aa72190 2024-02-02T15:28:30.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.125+0000 7fb826a00700 1 --2- 172.21.15.1:0/2964925359 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x56304aa72190 0x56304aa745b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:30.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.125+0000 7fb826a00700 1 --2- 172.21.15.1:0/2964925359 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x56304aa72190 0x56304aa745b0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d3ac8bd87b739fb2 server_cookie=196ef56721a74229 in_seq=0 out_seq=0 2024-02-02T15:28:30.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.125+0000 7fb814c00700 10 client.26056 ms_handle_connect on v2:172.21.15.142:6838/321675421 2024-02-02T15:28:30.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.126+0000 7fb814c00700 1 -- 172.21.15.1:0/2964925359 <== mds.1 v2:172.21.15.142:6838/321675421 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x56304a9a1dd0 con 0x56304aa72190 2024-02-02T15:28:30.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.126+0000 7fb814c00700 10 client.26056 handle_command_reply: tid=0 2024-02-02T15:28:30.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.126+0000 7fb841745b80 1 client.26056 shutdown 2024-02-02T15:28:30.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.126+0000 7fb841745b80 1 -- 172.21.15.1:0/2964925359 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56304aa627e0 msgr2=0x56304aa5f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:28:30.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.126+0000 7fb841745b80 1 --2- 172.21.15.1:0/2964925359 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56304aa627e0 0x56304aa5f920 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb820004330 tx=0x7fb820004ad0 comp rx=0 tx=0).stop 2024-02-02T15:28:30.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.126+0000 7fb841745b80 1 -- 172.21.15.1:0/2964925359 shutdown_connections 2024-02-02T15:28:30.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.126+0000 7fb841745b80 1 --2- 172.21.15.1:0/2964925359 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x56304aa72190 0x56304aa745b0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:30.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.126+0000 7fb841745b80 1 --2- 172.21.15.1:0/2964925359 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56304aa627e0 0x56304aa5f920 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:30.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.126+0000 7fb841745b80 1 --2- 172.21.15.1:0/2964925359 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56304aa57010 0x56304aa62170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:30.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.126+0000 7fb814c00700 0 client.26056 ms_handle_reset on v2:172.21.15.142:6838/321675421 2024-02-02T15:28:30.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.126+0000 7fb841745b80 1 --2- 172.21.15.1:0/2964925359 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56304aa57930 0x56304aa5f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:30.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.126+0000 7fb841745b80 1 -- 172.21.15.1:0/2964925359 >> 172.21.15.1:0/2964925359 conn(0x56304a9b6db0 msgr2=0x56304aa52fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:28:30.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.126+0000 7fb841745b80 1 -- 172.21.15.1:0/2964925359 shutdown_connections 2024-02-02T15:28:30.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.126+0000 7fb841745b80 1 -- 172.21.15.1:0/2964925359 wait complete. 2024-02-02T15:28:30.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.127+0000 7fb841745b80 20 client.26056 trim_cache size 0 max 16384 2024-02-02T15:28:30.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.142+0000 7fb841745b80 1 Processor -- start 2024-02-02T15:28:30.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.142+0000 7fb841745b80 1 -- start start 2024-02-02T15:28:30.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.142+0000 7fb841745b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56304aa57010 0x56304aa5e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:30.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.142+0000 7fb841745b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56304aa57930 0x56304aa61ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:30.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.142+0000 7fb841745b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56304aa627e0 0x56304aa59840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:30.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.142+0000 7fb841745b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56304a82c6a0 con 0x56304aa627e0 2024-02-02T15:28:30.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.142+0000 7fb841745b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56304aa6eb20 con 0x56304aa57930 2024-02-02T15:28:30.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.142+0000 7fb841745b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56304aa5fd00 con 0x56304aa57010 2024-02-02T15:28:30.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.142+0000 7fb826a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56304aa57010 0x56304aa5e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:30.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.142+0000 7fb826000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56304aa57930 0x56304aa61ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:30.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.143+0000 7fb827400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56304aa627e0 0x56304aa59840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:30.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.143+0000 7fb826a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56304aa57010 0x56304aa5e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:47710/0 (socket says 172.21.15.1:47710) 2024-02-02T15:28:30.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.143+0000 7fb826000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56304aa57930 0x56304aa61ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:51372/0 (socket says 172.21.15.1:51372) 2024-02-02T15:28:30.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.143+0000 7fb826a00700 1 -- 172.21.15.1:0/1972127427 learned_addr learned my addr 172.21.15.1:0/1972127427 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:28:30.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.143+0000 7fb826a00700 1 -- 172.21.15.1:0/1972127427 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56304aa57930 msgr2=0x56304aa61ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:28:30.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.143+0000 7fb826a00700 1 --2- 172.21.15.1:0/1972127427 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56304aa57930 0x56304aa61ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:30.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.143+0000 7fb826a00700 1 -- 172.21.15.1:0/1972127427 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56304aa627e0 msgr2=0x56304aa59840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:28:30.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.143+0000 7fb826a00700 1 --2- 172.21.15.1:0/1972127427 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56304aa627e0 0x56304aa59840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:30.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.143+0000 7fb826a00700 1 -- 172.21.15.1:0/1972127427 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56304a96eea0 con 0x56304aa57010 2024-02-02T15:28:30.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.143+0000 7fb827400700 1 --2- 172.21.15.1:0/1972127427 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56304aa627e0 0x56304aa59840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:28:30.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.143+0000 7fb826a00700 1 --2- 172.21.15.1:0/1972127427 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56304aa57010 0x56304aa5e290 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb818095a00 tx=0x7fb818084040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2bf3c075eff2b47e server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:28:30.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.143+0000 7fb81cc00700 1 -- 172.21.15.1:0/1972127427 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81809d330 con 0x56304aa57010 2024-02-02T15:28:30.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.143+0000 7fb81cc00700 1 -- 172.21.15.1:0/1972127427 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb8180cc040 con 0x56304aa57010 2024-02-02T15:28:30.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.143+0000 7fb81cc00700 1 -- 172.21.15.1:0/1972127427 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8180cc1b0 con 0x56304aa57010 2024-02-02T15:28:30.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.144+0000 7fb841745b80 1 -- 172.21.15.1:0/1972127427 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56304aa57010 msgr2=0x56304aa5e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:28:30.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.144+0000 7fb841745b80 1 --2- 172.21.15.1:0/1972127427 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56304aa57010 0x56304aa5e290 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb818095a00 tx=0x7fb818084040 comp rx=0 tx=0).stop 2024-02-02T15:28:30.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.144+0000 7fb841745b80 1 -- 172.21.15.1:0/1972127427 shutdown_connections 2024-02-02T15:28:30.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.144+0000 7fb841745b80 1 --2- 172.21.15.1:0/1972127427 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56304aa627e0 0x56304aa59840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:30.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.144+0000 7fb841745b80 1 --2- 172.21.15.1:0/1972127427 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56304aa57010 0x56304aa5e290 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:30.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.144+0000 7fb841745b80 1 --2- 172.21.15.1:0/1972127427 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56304aa57930 0x56304aa61ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:30.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.144+0000 7fb841745b80 1 -- 172.21.15.1:0/1972127427 >> 172.21.15.1:0/1972127427 conn(0x56304a9b6db0 msgr2=0x56304aa5db80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:28:30.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.144+0000 7fb841745b80 1 -- 172.21.15.1:0/1972127427 shutdown_connections 2024-02-02T15:28:30.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.144+0000 7fb841745b80 1 -- 172.21.15.1:0/1972127427 wait complete. 2024-02-02T15:28:30.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.145+0000 7fb841745b80 1 -- start start 2024-02-02T15:28:30.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.145+0000 7fb841745b80 1 Processor -- start 2024-02-02T15:28:30.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.146+0000 7fb841745b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56304aa57010 0x56304aa68fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:30.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.146+0000 7fb841745b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56304aa57930 0x56304aa672f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:30.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.146+0000 7fb841745b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56304aa627e0 0x56304aa67830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:30.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.146+0000 7fb841745b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56304aa67d70 con 0x56304aa627e0 2024-02-02T15:28:30.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.146+0000 7fb841745b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56304aa67eb0 con 0x56304aa57010 2024-02-02T15:28:30.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.146+0000 7fb841745b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56304aa68070 con 0x56304aa57930 2024-02-02T15:28:30.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.146+0000 7fb841745b80 10 client.0 _ll_register_callbacks cb 0x56304a946290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:28:30.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.146+0000 7fb826000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56304aa57930 0x56304aa672f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:30.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.146+0000 7fb826a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56304aa57010 0x56304aa68fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:30.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.146+0000 7fb826000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56304aa57930 0x56304aa672f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:47720/0 (socket says 172.21.15.1:47720) 2024-02-02T15:28:30.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.146+0000 7fb826a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56304aa57010 0x56304aa68fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:51380/0 (socket says 172.21.15.1:51380) 2024-02-02T15:28:30.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.146+0000 7fb826000700 1 -- 172.21.15.1:0/3481459483 learned_addr learned my addr 172.21.15.1:0/3481459483 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:28:30.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.146+0000 7fb827400700 1 --2- 172.21.15.1:0/3481459483 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56304aa627e0 0x56304aa67830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:30.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.147+0000 7fb826000700 1 -- 172.21.15.1:0/3481459483 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56304aa57010 msgr2=0x56304aa68fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:28:30.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.147+0000 7fb826000700 1 --2- 172.21.15.1:0/3481459483 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56304aa57010 0x56304aa68fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:30.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.147+0000 7fb826000700 1 -- 172.21.15.1:0/3481459483 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56304aa627e0 msgr2=0x56304aa67830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:28:30.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.147+0000 7fb826000700 1 --2- 172.21.15.1:0/3481459483 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56304aa627e0 0x56304aa67830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:30.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.147+0000 7fb826000700 1 -- 172.21.15.1:0/3481459483 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56304a7d6b60 con 0x56304aa57930 2024-02-02T15:28:30.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.147+0000 7fb826000700 1 --2- 172.21.15.1:0/3481459483 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56304aa57930 0x56304aa672f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb810005f50 tx=0x7fb810005280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99c50f1549b0c5fd server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:28:30.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.147+0000 7fb814c00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:28:30.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.147+0000 7fb814c00700 1 -- 172.21.15.1:0/3481459483 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81001d040 con 0x56304aa57930 2024-02-02T15:28:30.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.147+0000 7fb814c00700 1 -- 172.21.15.1:0/3481459483 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fb810004500 con 0x56304aa57930 2024-02-02T15:28:30.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.147+0000 7fb841745b80 1 -- 172.21.15.1:0/3481459483 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb81001e040 con 0x56304aa57930 2024-02-02T15:28:30.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.148+0000 7fb814c00700 1 -- 172.21.15.1:0/3481459483 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb8100046e0 con 0x56304aa57930 2024-02-02T15:28:30.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.148+0000 7fb841745b80 10 client.26062 fetch_fsmap learned FSMap version 27 2024-02-02T15:28:30.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.148+0000 7fb841745b80 1 -- 172.21.15.1:0/3481459483 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7fb8281a6040 con 0x56304aa57930 2024-02-02T15:28:30.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.148+0000 7fb814c00700 1 -- 172.21.15.1:0/3481459483 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb81001e040 con 0x56304aa57930 2024-02-02T15:28:30.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.148+0000 7fb814c00700 1 -- 172.21.15.1:0/3481459483 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb8100237d0 con 0x56304aa57930 2024-02-02T15:28:30.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.148+0000 7fb841745b80 10 client.26062 fetch_fsmap finished waiting for FSMap version 27 2024-02-02T15:28:30.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.148+0000 7fb841745b80 10 client.26062 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-02T15:28:30.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.148+0000 7fb841745b80 20 client.26062 populate_metadata read hostname 'smithi001' 2024-02-02T15:28:30.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.148+0000 7fb841745b80 1 --2- 172.21.15.1:0/3481459483 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x56304aac4c30 0x56304aac7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:30.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.148+0000 7fb841745b80 4 client.26062 mds_command: new command op to 24529 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:28:30.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.148+0000 7fb841745b80 1 -- 172.21.15.1:0/3481459483 --> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56304a967f50 con 0x56304aac4c30 2024-02-02T15:28:30.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.149+0000 7fb826a00700 1 --2- 172.21.15.1:0/3481459483 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x56304aac4c30 0x56304aac7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:30.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.149+0000 7fb826a00700 1 --2- 172.21.15.1:0/3481459483 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x56304aac4c30 0x56304aac7050 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6fe68fe35448bd4a server_cookie=3df90df6dc243144 in_seq=0 out_seq=0 2024-02-02T15:28:30.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.149+0000 7fb814c00700 10 client.26062 ms_handle_connect on v2:172.21.15.142:6838/321675421 2024-02-02T15:28:30.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.149+0000 7fb814c00700 1 -- 172.21.15.1:0/3481459483 <== mds.1 v2:172.21.15.142:6838/321675421 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56304a967f50 con 0x56304aac4c30 2024-02-02T15:28:30.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.149+0000 7fb814c00700 10 client.26062 handle_command_reply: tid=0 2024-02-02T15:28:30.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.149+0000 7fb841745b80 1 client.26062 shutdown 2024-02-02T15:28:30.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.149+0000 7fb841745b80 1 -- 172.21.15.1:0/3481459483 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56304aa57930 msgr2=0x56304aa672f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:28:30.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.149+0000 7fb841745b80 1 --2- 172.21.15.1:0/3481459483 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56304aa57930 0x56304aa672f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb810005f50 tx=0x7fb810005280 comp rx=0 tx=0).stop 2024-02-02T15:28:30.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.150+0000 7fb841745b80 1 -- 172.21.15.1:0/3481459483 shutdown_connections 2024-02-02T15:28:30.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.150+0000 7fb841745b80 1 --2- 172.21.15.1:0/3481459483 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x56304aac4c30 0x56304aac7050 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:30.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.150+0000 7fb841745b80 1 --2- 172.21.15.1:0/3481459483 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56304aa627e0 0x56304aa67830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:30.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.150+0000 7fb814c00700 0 client.26062 ms_handle_reset on v2:172.21.15.142:6838/321675421 2024-02-02T15:28:30.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.150+0000 7fb841745b80 1 --2- 172.21.15.1:0/3481459483 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56304aa57010 0x56304aa68fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:30.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.150+0000 7fb841745b80 1 --2- 172.21.15.1:0/3481459483 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56304aa57930 0x56304aa672f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:30.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.150+0000 7fb841745b80 1 -- 172.21.15.1:0/3481459483 >> 172.21.15.1:0/3481459483 conn(0x56304a9b6db0 msgr2=0x56304aa5f500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:28:30.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.150+0000 7fb841745b80 1 -- 172.21.15.1:0/3481459483 shutdown_connections 2024-02-02T15:28:30.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.150+0000 7fb841745b80 1 -- 172.21.15.1:0/3481459483 wait complete. 2024-02-02T15:28:30.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.150+0000 7fb841745b80 20 client.26062 trim_cache size 0 max 16384 2024-02-02T15:28:30.154 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:28:30.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.151+0000 7fb82dc00700 1 -- 172.21.15.1:0/3153412231 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fb8080a07b0 msgr2=0x7fb8080a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:28:30.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.151+0000 7fb82dc00700 1 --2- 172.21.15.1:0/3153412231 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fb8080a07b0 0x7fb8080a2c70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb820009550 tx=0x7fb82000b040 comp rx=0 tx=0).stop 2024-02-02T15:28:30.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.151+0000 7fb82dc00700 1 -- 172.21.15.1:0/3153412231 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb828100ff0 msgr2=0x7fb828073040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:28:30.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.151+0000 7fb82dc00700 1 --2- 172.21.15.1:0/3153412231 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb828100ff0 0x7fb828073040 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb81800e9a0 tx=0x7fb81800ecb0 comp rx=0 tx=0).stop 2024-02-02T15:28:30.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.151+0000 7fb82dc00700 1 -- 172.21.15.1:0/3153412231 shutdown_connections 2024-02-02T15:28:30.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.151+0000 7fb82dc00700 1 --2- 172.21.15.1:0/3153412231 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fb8080a07b0 0x7fb8080a2c70 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:30.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.151+0000 7fb82dc00700 1 --2- 172.21.15.1:0/3153412231 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb828073ac0 0x7fb8281a5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:30.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.151+0000 7fb82dc00700 1 --2- 172.21.15.1:0/3153412231 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb828100ff0 0x7fb828073040 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:30.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.151+0000 7fb82dc00700 1 --2- 172.21.15.1:0/3153412231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb828106110 0x7fb828073580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:30.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.151+0000 7fb82dc00700 1 -- 172.21.15.1:0/3153412231 >> 172.21.15.1:0/3153412231 conn(0x7fb8280fc810 msgr2=0x7fb8280fd2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:28:30.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.151+0000 7fb82dc00700 1 -- 172.21.15.1:0/3153412231 shutdown_connections 2024-02-02T15:28:30.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.151+0000 7fb82dc00700 1 -- 172.21.15.1:0/3153412231 wait complete. 2024-02-02T15:28:30.169 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-02T15:28:30.216 INFO:tasks.workunit.client.0.smithi001.stdout: 1 10636 19.15 MB/sec warmup 4 sec latency 13.105 ms 2024-02-02T15:28:30.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.379+0000 7f9e03200700 1 -- 172.21.15.1:0/155966818 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9dfc1019a0 msgr2=0x7f9dfc1059e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:28:30.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.379+0000 7f9e03200700 1 --2- 172.21.15.1:0/155966818 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9dfc1019a0 0x7f9dfc1059e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f9de4009990 tx=0x7f9de4009ca0 comp rx=0 tx=0).stop 2024-02-02T15:28:30.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.379+0000 7f9e03200700 1 -- 172.21.15.1:0/155966818 shutdown_connections 2024-02-02T15:28:30.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.379+0000 7f9e03200700 1 --2- 172.21.15.1:0/155966818 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9dfc106110 0x7f9dfc10dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:30.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.379+0000 7f9e03200700 1 --2- 172.21.15.1:0/155966818 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9dfc100ff0 0x7f9dfc1013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:30.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.379+0000 7f9e03200700 1 --2- 172.21.15.1:0/155966818 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9dfc1019a0 0x7f9dfc1059e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:30.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.379+0000 7f9e03200700 1 -- 172.21.15.1:0/155966818 >> 172.21.15.1:0/155966818 conn(0x7f9dfc0fc810 msgr2=0x7f9dfc0fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:28:30.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.380+0000 7f9e03200700 1 -- 172.21.15.1:0/155966818 shutdown_connections 2024-02-02T15:28:30.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.380+0000 7f9e03200700 1 -- 172.21.15.1:0/155966818 wait complete. 2024-02-02T15:28:30.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.381+0000 7f9e03200700 1 Processor -- start 2024-02-02T15:28:30.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.381+0000 7f9e03200700 1 -- start start 2024-02-02T15:28:30.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.382+0000 7f9e03200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9dfc100ff0 0x7f9dfc196f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:30.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.382+0000 7f9e03200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9dfc106110 0x7f9dfc197440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:30.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.382+0000 7f9e03200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9dfc19b8a0 0x7f9dfc19bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:30.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.382+0000 7f9e03200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dfc19c320 con 0x7f9dfc106110 2024-02-02T15:28:30.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.382+0000 7f9e03200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dfc19c490 con 0x7f9dfc19b8a0 2024-02-02T15:28:30.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.382+0000 7f9e03200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dfc19c650 con 0x7f9dfc100ff0 2024-02-02T15:28:30.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.382+0000 7f9dfbe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9dfc100ff0 0x7f9dfc196f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:30.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.382+0000 7f9dfbe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9dfc100ff0 0x7f9dfc196f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:47744/0 (socket says 172.21.15.1:47744) 2024-02-02T15:28:30.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.382+0000 7f9dfbe00700 1 -- 172.21.15.1:0/2171146312 learned_addr learned my addr 172.21.15.1:0/2171146312 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:28:30.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.382+0000 7f9dfb400700 1 --2- 172.21.15.1:0/2171146312 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9dfc106110 0x7f9dfc197440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:30.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.382+0000 7f9e01200700 1 --2- 172.21.15.1:0/2171146312 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9dfc19b8a0 0x7f9dfc19bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:30.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.383+0000 7f9dfbe00700 1 -- 172.21.15.1:0/2171146312 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9dfc19b8a0 msgr2=0x7f9dfc19bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:28:30.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.383+0000 7f9dfbe00700 1 --2- 172.21.15.1:0/2171146312 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9dfc19b8a0 0x7f9dfc19bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:30.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.383+0000 7f9dfbe00700 1 -- 172.21.15.1:0/2171146312 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9dfc106110 msgr2=0x7f9dfc197440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:28:30.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.383+0000 7f9dfbe00700 1 --2- 172.21.15.1:0/2171146312 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9dfc106110 0x7f9dfc197440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:30.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.383+0000 7f9dfbe00700 1 -- 172.21.15.1:0/2171146312 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9de40095f0 con 0x7f9dfc100ff0 2024-02-02T15:28:30.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.383+0000 7f9dfb400700 1 --2- 172.21.15.1:0/2171146312 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9dfc106110 0x7f9dfc197440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:28:30.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.383+0000 7f9dfbe00700 1 --2- 172.21.15.1:0/2171146312 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9dfc100ff0 0x7f9dfc196f00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9dec00e9a0 tx=0x7f9dec00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:28:30.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.383+0000 7f9df8c00700 1 -- 172.21.15.1:0/2171146312 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dec009790 con 0x7f9dfc100ff0 2024-02-02T15:28:30.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.383+0000 7f9df8c00700 1 -- 172.21.15.1:0/2171146312 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9dec010430 con 0x7f9dfc100ff0 2024-02-02T15:28:30.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.383+0000 7f9e03200700 1 -- 172.21.15.1:0/2171146312 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dfc1a3bb0 con 0x7f9dfc100ff0 2024-02-02T15:28:30.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.383+0000 7f9df8c00700 1 -- 172.21.15.1:0/2171146312 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dec018680 con 0x7f9dfc100ff0 2024-02-02T15:28:30.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.383+0000 7f9e03200700 1 -- 172.21.15.1:0/2171146312 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dfc1a4090 con 0x7f9dfc100ff0 2024-02-02T15:28:30.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.387+0000 7f9df8c00700 1 -- 172.21.15.1:0/2171146312 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 26) v1 ==== 135302+0+0 (secure 0 0 0) 0x7f9dec0105a0 con 0x7f9dfc100ff0 2024-02-02T15:28:30.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.388+0000 7f9e16d39b80 1 Processor -- start 2024-02-02T15:28:30.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.388+0000 7f9df8c00700 1 --2- 172.21.15.1:0/2171146312 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f9ddc0a07b0 0x7f9ddc0a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:30.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.388+0000 7f9e16d39b80 1 -- start start 2024-02-02T15:28:30.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.388+0000 7f9df8c00700 1 -- 172.21.15.1:0/2171146312 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f9dec014070 con 0x7f9dfc100ff0 2024-02-02T15:28:30.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.388+0000 7f9e01200700 1 --2- 172.21.15.1:0/2171146312 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f9ddc0a07b0 0x7f9ddc0a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:30.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.388+0000 7f9e16d39b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557f64859010 0x557f648593f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:30.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.388+0000 7f9e16d39b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557f64859930 0x557f648686b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:30.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.388+0000 7f9e16d39b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557f64868bf0 0x557f6486afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:30.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.388+0000 7f9e16d39b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f647cd4c0 con 0x557f64859930 2024-02-02T15:28:30.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.388+0000 7f9e16d39b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f647b1aa0 con 0x557f64868bf0 2024-02-02T15:28:30.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.388+0000 7f9e16d39b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f647af460 con 0x557f64859010 2024-02-02T15:28:30.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.389+0000 7f9dfbe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557f64859010 0x557f648593f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:30.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.389+0000 7f9e01200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557f64868bf0 0x557f6486afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:30.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.389+0000 7f9dfbe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557f64859010 0x557f648593f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:47758/0 (socket says 172.21.15.1:47758) 2024-02-02T15:28:30.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.389+0000 7f9dfbe00700 1 -- 172.21.15.1:0/2885584899 learned_addr learned my addr 172.21.15.1:0/2885584899 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:28:30.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.389+0000 7f9dfb400700 1 --2- 172.21.15.1:0/2885584899 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557f64859930 0x557f648686b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:30.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.389+0000 7f9dfbe00700 1 -- 172.21.15.1:0/2885584899 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557f64868bf0 msgr2=0x557f6486afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:28:30.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.389+0000 7f9dfbe00700 1 --2- 172.21.15.1:0/2885584899 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557f64868bf0 0x557f6486afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:30.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.389+0000 7f9dfbe00700 1 -- 172.21.15.1:0/2885584899 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557f64859930 msgr2=0x557f648686b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:28:30.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.389+0000 7f9dfbe00700 1 --2- 172.21.15.1:0/2885584899 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557f64859930 0x557f648686b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:30.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.389+0000 7f9dfbe00700 1 -- 172.21.15.1:0/2885584899 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f6462c3b0 con 0x557f64859010 2024-02-02T15:28:30.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.389+0000 7f9e01200700 1 --2- 172.21.15.1:0/2171146312 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f9ddc0a07b0 0x7f9ddc0a2c70 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9df400f870 tx=0x7f9df400fc30 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:28:30.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.389+0000 7f9dfbe00700 1 --2- 172.21.15.1:0/2885584899 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557f64859010 0x557f648593f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9dec0991a0 tx=0x7f9dec098e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f64283b338c30ae server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:28:30.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.390+0000 7f9df1600700 1 -- 172.21.15.1:0/2885584899 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dec0994a0 con 0x557f64859010 2024-02-02T15:28:30.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.390+0000 7f9df1600700 1 -- 172.21.15.1:0/2885584899 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9dec09d2d0 con 0x557f64859010 2024-02-02T15:28:30.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.390+0000 7f9df1600700 1 -- 172.21.15.1:0/2885584899 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dec0cc190 con 0x557f64859010 2024-02-02T15:28:30.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.390+0000 7f9e16d39b80 1 -- 172.21.15.1:0/2885584899 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557f64859010 msgr2=0x557f648593f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:28:30.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.390+0000 7f9e16d39b80 1 --2- 172.21.15.1:0/2885584899 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557f64859010 0x557f648593f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9dec0991a0 tx=0x7f9dec098e00 comp rx=0 tx=0).stop 2024-02-02T15:28:30.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.390+0000 7f9e16d39b80 1 -- 172.21.15.1:0/2885584899 shutdown_connections 2024-02-02T15:28:30.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.390+0000 7f9e16d39b80 1 --2- 172.21.15.1:0/2885584899 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557f64868bf0 0x557f6486afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:30.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.390+0000 7f9e16d39b80 1 --2- 172.21.15.1:0/2885584899 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557f64859010 0x557f648593f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:30.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.390+0000 7f9e16d39b80 1 --2- 172.21.15.1:0/2885584899 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557f64859930 0x557f648686b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:30.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.390+0000 7f9e16d39b80 1 -- 172.21.15.1:0/2885584899 >> 172.21.15.1:0/2885584899 conn(0x557f647b8db0 msgr2=0x557f648553b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:28:30.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.390+0000 7f9e16d39b80 1 -- 172.21.15.1:0/2885584899 shutdown_connections 2024-02-02T15:28:30.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.391+0000 7f9e16d39b80 1 -- 172.21.15.1:0/2885584899 wait complete. 2024-02-02T15:28:30.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.391+0000 7f9e16d39b80 1 -- start start 2024-02-02T15:28:30.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.391+0000 7f9e16d39b80 1 Processor -- start 2024-02-02T15:28:30.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.392+0000 7f9e16d39b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557f64859010 0x557f64863c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:30.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.392+0000 7f9e16d39b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557f64859930 0x557f64864180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:30.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.392+0000 7f9e16d39b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557f64868bf0 0x557f648646c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:30.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.392+0000 7f9e16d39b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f647aad50 con 0x557f64859930 2024-02-02T15:28:30.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.392+0000 7f9e16d39b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f64746d80 con 0x557f64868bf0 2024-02-02T15:28:30.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.392+0000 7f9e16d39b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f64746a20 con 0x557f64859010 2024-02-02T15:28:30.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.392+0000 7f9dfbe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557f64859010 0x557f64863c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:30.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.392+0000 7f9e16d39b80 10 client.0 _ll_register_callbacks cb 0x557f64748290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:28:30.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.392+0000 7f9dfbe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557f64859010 0x557f64863c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:47766/0 (socket says 172.21.15.1:47766) 2024-02-02T15:28:30.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.392+0000 7f9dfbe00700 1 -- 172.21.15.1:0/2186909021 learned_addr learned my addr 172.21.15.1:0/2186909021 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:28:30.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.392+0000 7f9dfb400700 1 --2- 172.21.15.1:0/2186909021 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557f64859930 0x557f64864180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:30.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.393+0000 7f9e01200700 1 --2- 172.21.15.1:0/2186909021 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557f64868bf0 0x557f648646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:30.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.393+0000 7f9dfbe00700 1 -- 172.21.15.1:0/2186909021 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557f64868bf0 msgr2=0x557f648646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:28:30.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.393+0000 7f9dfbe00700 1 --2- 172.21.15.1:0/2186909021 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557f64868bf0 0x557f648646c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:30.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.393+0000 7f9dfbe00700 1 -- 172.21.15.1:0/2186909021 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557f64859930 msgr2=0x557f64864180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:28:30.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.393+0000 7f9dfbe00700 1 --2- 172.21.15.1:0/2186909021 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557f64859930 0x557f64864180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:30.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.393+0000 7f9dfbe00700 1 -- 172.21.15.1:0/2186909021 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dfc1a4090 con 0x557f64859010 2024-02-02T15:28:30.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.393+0000 7f9dfbe00700 1 --2- 172.21.15.1:0/2186909021 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557f64859010 0x557f64863c40 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9dec0a0fe0 tx=0x7f9dec099ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37614068c6131575 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:28:30.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.393+0000 7f9dea000700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:28:30.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.393+0000 7f9dea000700 1 -- 172.21.15.1:0/2186909021 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dec0a1510 con 0x557f64859010 2024-02-02T15:28:30.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.393+0000 7f9dea000700 1 -- 172.21.15.1:0/2186909021 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f9dec0994a0 con 0x557f64859010 2024-02-02T15:28:30.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.393+0000 7f9e16d39b80 1 -- 172.21.15.1:0/2186909021 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557f6484d0f0 con 0x557f64859010 2024-02-02T15:28:30.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.393+0000 7f9dea000700 1 -- 172.21.15.1:0/2186909021 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9dec0651a0 con 0x557f64859010 2024-02-02T15:28:30.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.393+0000 7f9e16d39b80 10 client.26074 fetch_fsmap learned FSMap version 27 2024-02-02T15:28:30.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.393+0000 7f9e16d39b80 1 -- 172.21.15.1:0/2186909021 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x557f645d8b60 con 0x557f64859010 2024-02-02T15:28:30.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.394+0000 7f9dea000700 1 -- 172.21.15.1:0/2186909021 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557f6484d0f0 con 0x557f64859010 2024-02-02T15:28:30.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.394+0000 7f9dea000700 1 -- 172.21.15.1:0/2186909021 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9dec0a1080 con 0x557f64859010 2024-02-02T15:28:30.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.394+0000 7f9e16d39b80 10 client.26074 fetch_fsmap finished waiting for FSMap version 27 2024-02-02T15:28:30.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.394+0000 7f9e16d39b80 10 client.26074 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:28:30.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.394+0000 7f9e16d39b80 20 client.26074 populate_metadata read hostname 'smithi001' 2024-02-02T15:28:30.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.394+0000 7f9e16d39b80 1 --2- 172.21.15.1:0/2186909021 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x557f64873d70 0x557f64876160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:30.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.394+0000 7f9e16d39b80 4 client.26074 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:28:30.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.394+0000 7f9e16d39b80 1 -- 172.21.15.1:0/2186909021 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557f647a3dd0 con 0x557f64873d70 2024-02-02T15:28:30.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.394+0000 7f9dfb400700 1 --2- 172.21.15.1:0/2186909021 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x557f64873d70 0x557f64876160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:30.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.395+0000 7f9dfb400700 1 --2- 172.21.15.1:0/2186909021 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x557f64873d70 0x557f64876160 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e97ecc06e11250ab server_cookie=7522be36a3629fa6 in_seq=0 out_seq=0 2024-02-02T15:28:30.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.395+0000 7f9dea000700 10 client.26074 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:28:30.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.395+0000 7f9dea000700 1 -- 172.21.15.1:0/2186909021 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x557f647a3dd0 con 0x557f64873d70 2024-02-02T15:28:30.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.395+0000 7f9dea000700 10 client.26074 handle_command_reply: tid=0 2024-02-02T15:28:30.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.395+0000 7f9e16d39b80 1 client.26074 shutdown 2024-02-02T15:28:30.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.396+0000 7f9e16d39b80 1 -- 172.21.15.1:0/2186909021 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557f64859010 msgr2=0x557f64863c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:28:30.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.396+0000 7f9e16d39b80 1 --2- 172.21.15.1:0/2186909021 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557f64859010 0x557f64863c40 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9dec0a0fe0 tx=0x7f9dec099ce0 comp rx=0 tx=0).stop 2024-02-02T15:28:30.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.396+0000 7f9e16d39b80 1 -- 172.21.15.1:0/2186909021 shutdown_connections 2024-02-02T15:28:30.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.396+0000 7f9e16d39b80 1 --2- 172.21.15.1:0/2186909021 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x557f64873d70 0x557f64876160 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:30.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.396+0000 7f9e16d39b80 1 --2- 172.21.15.1:0/2186909021 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557f64868bf0 0x557f648646c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:30.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.396+0000 7f9dea000700 0 client.26074 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:28:30.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.396+0000 7f9e16d39b80 1 --2- 172.21.15.1:0/2186909021 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557f64859010 0x557f64863c40 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:30.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.396+0000 7f9e16d39b80 1 --2- 172.21.15.1:0/2186909021 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557f64859930 0x557f64864180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:30.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.396+0000 7f9e16d39b80 1 -- 172.21.15.1:0/2186909021 >> 172.21.15.1:0/2186909021 conn(0x557f647b8db0 msgr2=0x557f64854fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:28:30.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.396+0000 7f9e16d39b80 1 -- 172.21.15.1:0/2186909021 shutdown_connections 2024-02-02T15:28:30.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.396+0000 7f9e16d39b80 1 -- 172.21.15.1:0/2186909021 wait complete. 2024-02-02T15:28:30.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.396+0000 7f9e16d39b80 20 client.26074 trim_cache size 0 max 16384 2024-02-02T15:28:30.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.411+0000 7f9e16d39b80 1 Processor -- start 2024-02-02T15:28:30.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.411+0000 7f9e16d39b80 1 -- start start 2024-02-02T15:28:30.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.411+0000 7f9e16d39b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557f64859010 0x557f6486d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:30.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.411+0000 7f9e16d39b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557f64859930 0x557f6486f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:30.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.411+0000 7f9e16d39b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557f64868bf0 0x557f6485ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:30.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.412+0000 7f9e16d39b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f6462e6a0 con 0x557f64868bf0 2024-02-02T15:28:30.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.412+0000 7f9e16d39b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f64864db0 con 0x557f64859010 2024-02-02T15:28:30.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.412+0000 7f9e16d39b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f647a3dd0 con 0x557f64859930 2024-02-02T15:28:30.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.412+0000 7f9dfbe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557f64859010 0x557f6486d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:30.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.412+0000 7f9dfbe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557f64859010 0x557f6486d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:51446/0 (socket says 172.21.15.1:51446) 2024-02-02T15:28:30.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.412+0000 7f9dfbe00700 1 -- 172.21.15.1:0/337399157 learned_addr learned my addr 172.21.15.1:0/337399157 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:28:30.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.412+0000 7f9dfb400700 1 --2- 172.21.15.1:0/337399157 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557f64859930 0x557f6486f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:30.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.412+0000 7f9e01200700 1 --2- 172.21.15.1:0/337399157 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557f64868bf0 0x557f6485ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:30.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.412+0000 7f9dfbe00700 1 -- 172.21.15.1:0/337399157 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557f64859930 msgr2=0x557f6486f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:28:30.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.412+0000 7f9dfbe00700 1 --2- 172.21.15.1:0/337399157 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557f64859930 0x557f6486f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:30.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.412+0000 7f9dfbe00700 1 -- 172.21.15.1:0/337399157 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557f64868bf0 msgr2=0x557f6485ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:28:30.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.412+0000 7f9dfbe00700 1 --2- 172.21.15.1:0/337399157 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557f64868bf0 0x557f6485ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:30.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.412+0000 7f9dfbe00700 1 -- 172.21.15.1:0/337399157 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f648510d0 con 0x557f64859010 2024-02-02T15:28:30.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.412+0000 7f9dfbe00700 1 --2- 172.21.15.1:0/337399157 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557f64859010 0x557f6486d000 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9dec099430 tx=0x7f9dec099090 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe2c12bc06577532 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:28:30.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.413+0000 7f9df2000700 1 -- 172.21.15.1:0/337399157 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dec0957e0 con 0x557f64859010 2024-02-02T15:28:30.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.413+0000 7f9df2000700 1 -- 172.21.15.1:0/337399157 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9dec09f670 con 0x557f64859010 2024-02-02T15:28:30.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.413+0000 7f9df2000700 1 -- 172.21.15.1:0/337399157 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dec098a90 con 0x557f64859010 2024-02-02T15:28:30.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.413+0000 7f9e16d39b80 1 -- 172.21.15.1:0/337399157 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557f64859010 msgr2=0x557f6486d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:28:30.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.413+0000 7f9e16d39b80 1 --2- 172.21.15.1:0/337399157 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557f64859010 0x557f6486d000 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9dec099430 tx=0x7f9dec099090 comp rx=0 tx=0).stop 2024-02-02T15:28:30.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.413+0000 7f9e16d39b80 1 -- 172.21.15.1:0/337399157 shutdown_connections 2024-02-02T15:28:30.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.413+0000 7f9e16d39b80 1 --2- 172.21.15.1:0/337399157 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557f64868bf0 0x557f6485ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:30.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.413+0000 7f9e16d39b80 1 --2- 172.21.15.1:0/337399157 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557f64859010 0x557f6486d000 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:30.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.413+0000 7f9e16d39b80 1 --2- 172.21.15.1:0/337399157 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557f64859930 0x557f6486f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:30.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.413+0000 7f9e16d39b80 1 -- 172.21.15.1:0/337399157 >> 172.21.15.1:0/337399157 conn(0x557f647b8db0 msgr2=0x557f64864400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:28:30.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.413+0000 7f9e16d39b80 1 -- 172.21.15.1:0/337399157 shutdown_connections 2024-02-02T15:28:30.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.413+0000 7f9e16d39b80 1 -- 172.21.15.1:0/337399157 wait complete. 2024-02-02T15:28:30.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.414+0000 7f9e16d39b80 1 -- start start 2024-02-02T15:28:30.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.414+0000 7f9e16d39b80 1 Processor -- start 2024-02-02T15:28:30.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.414+0000 7f9e16d39b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557f64859010 0x557f648be2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:30.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.414+0000 7f9e16d39b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557f64859930 0x557f648be7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:30.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.414+0000 7f9e16d39b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557f64868bf0 0x557f6486c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:30.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.414+0000 7f9e16d39b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f6486ca70 con 0x557f64859930 2024-02-02T15:28:30.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.414+0000 7f9e16d39b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f6486cbe0 con 0x557f64859010 2024-02-02T15:28:30.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.414+0000 7f9e16d39b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f6486cda0 con 0x557f64868bf0 2024-02-02T15:28:30.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.414+0000 7f9e16d39b80 10 client.0 _ll_register_callbacks cb 0x557f64748290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:28:30.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.415+0000 7f9dfbe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557f64859010 0x557f648be2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:30.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.415+0000 7f9dfb400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557f64859930 0x557f648be7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:30.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.415+0000 7f9dfbe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557f64859010 0x557f648be2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:51456/0 (socket says 172.21.15.1:51456) 2024-02-02T15:28:30.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.415+0000 7f9dfbe00700 1 -- 172.21.15.1:0/2480006665 learned_addr learned my addr 172.21.15.1:0/2480006665 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:28:30.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.415+0000 7f9e01200700 1 --2- 172.21.15.1:0/2480006665 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557f64868bf0 0x557f6486c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:30.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.415+0000 7f9dfbe00700 1 -- 172.21.15.1:0/2480006665 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557f64868bf0 msgr2=0x557f6486c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:28:30.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.415+0000 7f9dfbe00700 1 --2- 172.21.15.1:0/2480006665 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557f64868bf0 0x557f6486c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:30.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.415+0000 7f9dfbe00700 1 -- 172.21.15.1:0/2480006665 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557f64859930 msgr2=0x557f648be7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:28:30.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.415+0000 7f9dfbe00700 1 --2- 172.21.15.1:0/2480006665 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557f64859930 0x557f648be7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:30.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.415+0000 7f9dfbe00700 1 -- 172.21.15.1:0/2480006665 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557f645d8b60 con 0x557f64859010 2024-02-02T15:28:30.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.415+0000 7f9dfbe00700 1 --2- 172.21.15.1:0/2480006665 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557f64859010 0x557f648be2b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9dec000c00 tx=0x7f9dec0c98e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c97aec48248c9ef server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:28:30.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.415+0000 7f9dea000700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:28:30.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.415+0000 7f9dea000700 1 -- 172.21.15.1:0/2480006665 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dec0957e0 con 0x557f64859010 2024-02-02T15:28:30.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.415+0000 7f9dea000700 1 -- 172.21.15.1:0/2480006665 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f9dec0994a0 con 0x557f64859010 2024-02-02T15:28:30.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.415+0000 7f9e16d39b80 1 -- 172.21.15.1:0/2480006665 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9dec09d2d0 con 0x557f64859010 2024-02-02T15:28:30.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.415+0000 7f9dea000700 1 -- 172.21.15.1:0/2480006665 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9dec0b7250 con 0x557f64859010 2024-02-02T15:28:30.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.416+0000 7f9e16d39b80 10 client.26084 fetch_fsmap learned FSMap version 27 2024-02-02T15:28:30.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.416+0000 7f9e16d39b80 1 -- 172.21.15.1:0/2480006665 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x557f64770ea0 con 0x557f64859010 2024-02-02T15:28:30.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.416+0000 7f9dea000700 1 -- 172.21.15.1:0/2480006665 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9dec0a5020 con 0x557f64859010 2024-02-02T15:28:30.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.416+0000 7f9dea000700 1 -- 172.21.15.1:0/2480006665 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9dec098400 con 0x557f64859010 2024-02-02T15:28:30.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.416+0000 7f9e16d39b80 10 client.26084 fetch_fsmap finished waiting for FSMap version 27 2024-02-02T15:28:30.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.416+0000 7f9e16d39b80 10 client.26084 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:28:30.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.416+0000 7f9e16d39b80 20 client.26084 populate_metadata read hostname 'smithi001' 2024-02-02T15:28:30.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.416+0000 7f9e16d39b80 1 --2- 172.21.15.1:0/2480006665 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x557f648c5c30 0x557f648c8020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:30.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.416+0000 7f9e16d39b80 4 client.26084 mds_command: new command op to 24557 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:28:30.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.416+0000 7f9e16d39b80 1 -- 172.21.15.1:0/2480006665 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557f64769f50 con 0x557f648c5c30 2024-02-02T15:28:30.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.416+0000 7f9dfb400700 1 --2- 172.21.15.1:0/2480006665 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x557f648c5c30 0x557f648c8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:30.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.417+0000 7f9dfb400700 1 --2- 172.21.15.1:0/2480006665 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x557f648c5c30 0x557f648c8020 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=738f55acc04809ca server_cookie=1582aed5173de049 in_seq=0 out_seq=0 2024-02-02T15:28:30.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.417+0000 7f9dea000700 10 client.26084 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:28:30.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.417+0000 7f9dea000700 1 -- 172.21.15.1:0/2480006665 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557f64769f50 con 0x557f648c5c30 2024-02-02T15:28:30.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.417+0000 7f9dea000700 10 client.26084 handle_command_reply: tid=0 2024-02-02T15:28:30.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.417+0000 7f9e16d39b80 1 client.26084 shutdown 2024-02-02T15:28:30.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.417+0000 7f9e16d39b80 1 -- 172.21.15.1:0/2480006665 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557f64859010 msgr2=0x557f648be2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:28:30.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.417+0000 7f9e16d39b80 1 --2- 172.21.15.1:0/2480006665 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557f64859010 0x557f648be2b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9dec000c00 tx=0x7f9dec0c98e0 comp rx=0 tx=0).stop 2024-02-02T15:28:30.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.417+0000 7f9e16d39b80 1 -- 172.21.15.1:0/2480006665 shutdown_connections 2024-02-02T15:28:30.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.417+0000 7f9e16d39b80 1 --2- 172.21.15.1:0/2480006665 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x557f648c5c30 0x557f648c8020 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:30.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.417+0000 7f9e16d39b80 1 --2- 172.21.15.1:0/2480006665 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557f64868bf0 0x557f6486c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:30.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.417+0000 7f9e16d39b80 1 --2- 172.21.15.1:0/2480006665 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557f64859010 0x557f648be2b0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:30.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.417+0000 7f9e16d39b80 1 --2- 172.21.15.1:0/2480006665 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557f64859930 0x557f648be7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:30.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.417+0000 7f9e16d39b80 1 -- 172.21.15.1:0/2480006665 >> 172.21.15.1:0/2480006665 conn(0x557f647b8db0 msgr2=0x557f6486f5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:28:30.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.417+0000 7f9dea000700 0 client.26084 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:28:30.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.417+0000 7f9e16d39b80 1 -- 172.21.15.1:0/2480006665 shutdown_connections 2024-02-02T15:28:30.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.417+0000 7f9e16d39b80 1 -- 172.21.15.1:0/2480006665 wait complete. 2024-02-02T15:28:30.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.417+0000 7f9e16d39b80 20 client.26084 trim_cache size 0 max 16384 2024-02-02T15:28:30.426 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:28:30.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.418+0000 7f9e03200700 1 -- 172.21.15.1:0/2171146312 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f9ddc0a07b0 msgr2=0x7f9ddc0a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:28:30.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.418+0000 7f9e03200700 1 --2- 172.21.15.1:0/2171146312 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f9ddc0a07b0 0x7f9ddc0a2c70 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9df400f870 tx=0x7f9df400fc30 comp rx=0 tx=0).stop 2024-02-02T15:28:30.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.418+0000 7f9e03200700 1 -- 172.21.15.1:0/2171146312 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9dfc100ff0 msgr2=0x7f9dfc196f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:28:30.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.418+0000 7f9e03200700 1 --2- 172.21.15.1:0/2171146312 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9dfc100ff0 0x7f9dfc196f00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9dec00e9a0 tx=0x7f9dec00ecb0 comp rx=0 tx=0).stop 2024-02-02T15:28:30.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.418+0000 7f9e03200700 1 -- 172.21.15.1:0/2171146312 shutdown_connections 2024-02-02T15:28:30.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.418+0000 7f9e03200700 1 --2- 172.21.15.1:0/2171146312 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f9ddc0a07b0 0x7f9ddc0a2c70 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:30.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.418+0000 7f9e03200700 1 --2- 172.21.15.1:0/2171146312 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9dfc19b8a0 0x7f9dfc19bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:30.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.418+0000 7f9e03200700 1 --2- 172.21.15.1:0/2171146312 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9dfc100ff0 0x7f9dfc196f00 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:30.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.418+0000 7f9e03200700 1 --2- 172.21.15.1:0/2171146312 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9dfc106110 0x7f9dfc197440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:30.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.418+0000 7f9e03200700 1 -- 172.21.15.1:0/2171146312 >> 172.21.15.1:0/2171146312 conn(0x7f9dfc0fc810 msgr2=0x7f9dfc0fd2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:28:30.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.418+0000 7f9e03200700 1 -- 172.21.15.1:0/2171146312 shutdown_connections 2024-02-02T15:28:30.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:30.418+0000 7f9e03200700 1 -- 172.21.15.1:0/2171146312 wait complete. 2024-02-02T15:28:31.216 INFO:tasks.workunit.client.0.smithi001.stdout: 1 12687 16.79 MB/sec warmup 5 sec latency 12.063 ms 2024-02-02T15:28:31.437 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-02T15:28:31.438 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-02T15:28:31.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.658+0000 7f3ec9c00700 1 -- 172.21.15.1:0/506887645 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ec41019a0 msgr2=0x7f3ec41059e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:28:31.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.658+0000 7f3ec9c00700 1 --2- 172.21.15.1:0/506887645 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ec41019a0 0x7f3ec41059e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3eac009990 tx=0x7f3eac009ca0 comp rx=0 tx=0).stop 2024-02-02T15:28:31.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.658+0000 7f3ec9c00700 1 -- 172.21.15.1:0/506887645 shutdown_connections 2024-02-02T15:28:31.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.658+0000 7f3ec9c00700 1 --2- 172.21.15.1:0/506887645 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ec4106110 0x7f3ec410dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:31.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.658+0000 7f3ec9c00700 1 --2- 172.21.15.1:0/506887645 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ec4100ff0 0x7f3ec41013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:31.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.658+0000 7f3ec9c00700 1 --2- 172.21.15.1:0/506887645 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ec41019a0 0x7f3ec41059e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:31.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.658+0000 7f3ec9c00700 1 -- 172.21.15.1:0/506887645 >> 172.21.15.1:0/506887645 conn(0x7f3ec40fc810 msgr2=0x7f3ec40fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:28:31.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.658+0000 7f3ec9c00700 1 -- 172.21.15.1:0/506887645 shutdown_connections 2024-02-02T15:28:31.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.658+0000 7f3ec9c00700 1 -- 172.21.15.1:0/506887645 wait complete. 2024-02-02T15:28:31.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.659+0000 7f3ec9c00700 1 Processor -- start 2024-02-02T15:28:31.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.659+0000 7f3ec9c00700 1 -- start start 2024-02-02T15:28:31.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.659+0000 7f3ec9c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ec4100ff0 0x7f3ec4199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:31.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.659+0000 7f3ec9c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ec4106110 0x7f3ec41996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:31.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.660+0000 7f3ec9c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ec419db10 0x7f3ec419df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:31.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.660+0000 7f3ec9c00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec419e590 con 0x7f3ec419db10 2024-02-02T15:28:31.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.660+0000 7f3ec9c00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec419e700 con 0x7f3ec4106110 2024-02-02T15:28:31.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.660+0000 7f3ec9c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec419e8c0 con 0x7f3ec4100ff0 2024-02-02T15:28:31.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.660+0000 7f3ec2000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ec4106110 0x7f3ec41996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:31.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.660+0000 7f3ec2a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ec4100ff0 0x7f3ec4199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:31.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.660+0000 7f3ec2000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ec4106110 0x7f3ec41996b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:51472/0 (socket says 172.21.15.1:51472) 2024-02-02T15:28:31.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.660+0000 7f3ec2000700 1 -- 172.21.15.1:0/3733254540 learned_addr learned my addr 172.21.15.1:0/3733254540 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:28:31.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.660+0000 7f3ec3400700 1 --2- 172.21.15.1:0/3733254540 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ec419db10 0x7f3ec419df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:31.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.660+0000 7f3ec2000700 1 -- 172.21.15.1:0/3733254540 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ec4100ff0 msgr2=0x7f3ec4199170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:28:31.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.660+0000 7f3ec2000700 1 --2- 172.21.15.1:0/3733254540 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ec4100ff0 0x7f3ec4199170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:31.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.660+0000 7f3ec2000700 1 -- 172.21.15.1:0/3733254540 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ec419db10 msgr2=0x7f3ec419df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:28:31.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.660+0000 7f3ec2000700 1 --2- 172.21.15.1:0/3733254540 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ec419db10 0x7f3ec419df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:31.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.660+0000 7f3ec2000700 1 -- 172.21.15.1:0/3733254540 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eac0095f0 con 0x7f3ec4106110 2024-02-02T15:28:31.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.660+0000 7f3ec2000700 1 --2- 172.21.15.1:0/3733254540 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ec4106110 0x7f3ec41996b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3eac004760 tx=0x7f3eac004790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:28:31.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.661+0000 7f3ebb400700 1 -- 172.21.15.1:0/3733254540 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eac01d070 con 0x7f3ec4106110 2024-02-02T15:28:31.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.661+0000 7f3ebb400700 1 -- 172.21.15.1:0/3733254540 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3eac022470 con 0x7f3ec4106110 2024-02-02T15:28:31.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.661+0000 7f3ec9c00700 1 -- 172.21.15.1:0/3733254540 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ec41a5dc0 con 0x7f3ec4106110 2024-02-02T15:28:31.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.661+0000 7f3ebb400700 1 -- 172.21.15.1:0/3733254540 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eac017650 con 0x7f3ec4106110 2024-02-02T15:28:31.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.661+0000 7f3ec9c00700 1 -- 172.21.15.1:0/3733254540 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ec41a6190 con 0x7f3ec4106110 2024-02-02T15:28:31.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.664+0000 7f3ebb400700 1 -- 172.21.15.1:0/3733254540 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) v1 ==== 135302+0+0 (secure 0 0 0) 0x7f3eac0225e0 con 0x7f3ec4106110 2024-02-02T15:28:31.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.665+0000 7f3ebb400700 1 --2- 172.21.15.1:0/3733254540 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f3ea40a07b0 0x7f3ea40a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:31.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.665+0000 7f3ebb400700 1 -- 172.21.15.1:0/3733254540 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f3eac0cd690 con 0x7f3ec4106110 2024-02-02T15:28:31.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.665+0000 7f3edd703b80 1 Processor -- start 2024-02-02T15:28:31.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.666+0000 7f3ec2a00700 1 --2- 172.21.15.1:0/3733254540 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f3ea40a07b0 0x7f3ea40a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:31.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.666+0000 7f3edd703b80 1 -- start start 2024-02-02T15:28:31.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.666+0000 7f3edd703b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c50eaff090 0x55c50eaff470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:31.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.666+0000 7f3edd703b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c50eaff9b0 0x55c50eb0a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:31.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.666+0000 7f3edd703b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c50eb0a860 0x55c50eb0cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:31.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.666+0000 7f3edd703b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c50ea73540 con 0x55c50eaff9b0 2024-02-02T15:28:31.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.666+0000 7f3edd703b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c50ea57b20 con 0x55c50eb0a860 2024-02-02T15:28:31.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.666+0000 7f3edd703b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c50ea554e0 con 0x55c50eaff090 2024-02-02T15:28:31.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.666+0000 7f3ec2000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c50eaff9b0 0x55c50eb0a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:31.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.666+0000 7f3ec2a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c50eaff090 0x55c50eaff470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:31.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.666+0000 7f3ec2000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c50eaff9b0 0x55c50eb0a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49854/0 (socket says 172.21.15.1:49854) 2024-02-02T15:28:31.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.666+0000 7f3ec2000700 1 -- 172.21.15.1:0/2687878140 learned_addr learned my addr 172.21.15.1:0/2687878140 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:28:31.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.666+0000 7f3ec3400700 1 --2- 172.21.15.1:0/2687878140 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c50eb0a860 0x55c50eb0cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:31.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.667+0000 7f3ec2a00700 1 --2- 172.21.15.1:0/3733254540 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f3ea40a07b0 0x7f3ea40a2c70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3eb4009c60 tx=0x7f3eb4009260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:28:31.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.667+0000 7f3ec2000700 1 -- 172.21.15.1:0/2687878140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c50eaff090 msgr2=0x55c50eaff470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:28:31.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.667+0000 7f3ec2000700 1 --2- 172.21.15.1:0/2687878140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c50eaff090 0x55c50eaff470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:31.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.667+0000 7f3ec2000700 1 -- 172.21.15.1:0/2687878140 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c50eb0a860 msgr2=0x55c50eb0cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:28:31.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.667+0000 7f3ec2000700 1 --2- 172.21.15.1:0/2687878140 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c50eb0a860 0x55c50eb0cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:31.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.667+0000 7f3ec2000700 1 -- 172.21.15.1:0/2687878140 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c50e8d2430 con 0x55c50eaff9b0 2024-02-02T15:28:31.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.667+0000 7f3ec2a00700 1 --2- 172.21.15.1:0/2687878140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c50eaff090 0x55c50eaff470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:28:31.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.667+0000 7f3ec2000700 1 --2- 172.21.15.1:0/2687878140 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c50eaff9b0 0x55c50eb0a320 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f3eac0a2210 tx=0x7f3eac022f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f92897964b51a6b7 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:28:31.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.667+0000 7f3eb3e00700 1 -- 172.21.15.1:0/2687878140 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eac09a2a0 con 0x55c50eaff9b0 2024-02-02T15:28:31.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.667+0000 7f3eb3e00700 1 -- 172.21.15.1:0/2687878140 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3eac09a0c0 con 0x55c50eaff9b0 2024-02-02T15:28:31.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.667+0000 7f3eb3e00700 1 -- 172.21.15.1:0/2687878140 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eac09e170 con 0x55c50eaff9b0 2024-02-02T15:28:31.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.668+0000 7f3edd703b80 1 -- 172.21.15.1:0/2687878140 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c50eaff9b0 msgr2=0x55c50eb0a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:28:31.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.668+0000 7f3edd703b80 1 --2- 172.21.15.1:0/2687878140 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c50eaff9b0 0x55c50eb0a320 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f3eac0a2210 tx=0x7f3eac022f90 comp rx=0 tx=0).stop 2024-02-02T15:28:31.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.668+0000 7f3edd703b80 1 -- 172.21.15.1:0/2687878140 shutdown_connections 2024-02-02T15:28:31.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.668+0000 7f3edd703b80 1 --2- 172.21.15.1:0/2687878140 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c50eb0a860 0x55c50eb0cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:31.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.668+0000 7f3edd703b80 1 --2- 172.21.15.1:0/2687878140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c50eaff090 0x55c50eaff470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:31.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.668+0000 7f3edd703b80 1 --2- 172.21.15.1:0/2687878140 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c50eaff9b0 0x55c50eb0a320 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:31.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.668+0000 7f3edd703b80 1 -- 172.21.15.1:0/2687878140 >> 172.21.15.1:0/2687878140 conn(0x55c50ea5ee30 msgr2=0x55c50eafb430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:28:31.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.668+0000 7f3edd703b80 1 -- 172.21.15.1:0/2687878140 shutdown_connections 2024-02-02T15:28:31.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.668+0000 7f3edd703b80 1 -- 172.21.15.1:0/2687878140 wait complete. 2024-02-02T15:28:31.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.669+0000 7f3edd703b80 1 -- start start 2024-02-02T15:28:31.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:28:31 smithi142 ceph-mon[45728]: pgmap v48: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-02T15:28:31.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.669+0000 7f3edd703b80 1 Processor -- start 2024-02-02T15:28:31.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.670+0000 7f3edd703b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c50eaff090 0x55c50eb0ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:31.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.670+0000 7f3edd703b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c50eaff9b0 0x55c50eb0b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:31.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.670+0000 7f3edd703b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c50eb0a860 0x55c50eb0b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:31.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.670+0000 7f3edd703b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c50ea50dd0 con 0x55c50eb0a860 2024-02-02T15:28:31.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.670+0000 7f3edd703b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c50e9ece00 con 0x55c50eaff9b0 2024-02-02T15:28:31.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.670+0000 7f3edd703b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c50e9ecaa0 con 0x55c50eaff090 2024-02-02T15:28:31.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.670+0000 7f3edd703b80 10 client.0 _ll_register_callbacks cb 0x55c50e9ee310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:28:31.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.670+0000 7f3ec2a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c50eaff090 0x55c50eb0ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:31.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.670+0000 7f3ec2a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c50eaff090 0x55c50eb0ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:47830/0 (socket says 172.21.15.1:47830) 2024-02-02T15:28:31.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.670+0000 7f3ec2a00700 1 -- 172.21.15.1:0/1468939125 learned_addr learned my addr 172.21.15.1:0/1468939125 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:28:31.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.670+0000 7f3ec2000700 1 --2- 172.21.15.1:0/1468939125 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c50eaff9b0 0x55c50eb0b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:31.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.670+0000 7f3ec3400700 1 --2- 172.21.15.1:0/1468939125 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c50eb0a860 0x55c50eb0b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:31.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.671+0000 7f3ec2a00700 1 -- 172.21.15.1:0/1468939125 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c50eaff9b0 msgr2=0x55c50eb0b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:28:31.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.671+0000 7f3ec2a00700 1 --2- 172.21.15.1:0/1468939125 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c50eaff9b0 0x55c50eb0b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:31.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.671+0000 7f3ec2a00700 1 -- 172.21.15.1:0/1468939125 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c50eb0a860 msgr2=0x55c50eb0b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:28:31.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.671+0000 7f3ec2a00700 1 --2- 172.21.15.1:0/1468939125 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c50eb0a860 0x55c50eb0b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:31.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.671+0000 7f3ec2a00700 1 -- 172.21.15.1:0/1468939125 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3eb4017070 con 0x55c50eaff090 2024-02-02T15:28:31.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.671+0000 7f3ec3400700 1 --2- 172.21.15.1:0/1468939125 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c50eb0a860 0x55c50eb0b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:28:31.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.671+0000 7f3ec2a00700 1 --2- 172.21.15.1:0/1468939125 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c50eaff090 0x55c50eb0ac40 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3eb401dc00 tx=0x7f3eb40047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49853dacef14ad05 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:28:31.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.671+0000 7f3eb0c00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:28:31.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.671+0000 7f3eb0c00700 1 -- 172.21.15.1:0/1468939125 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb4022070 con 0x55c50eaff090 2024-02-02T15:28:31.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.671+0000 7f3eb0c00700 1 -- 172.21.15.1:0/1468939125 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f3eb4004240 con 0x55c50eaff090 2024-02-02T15:28:31.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.671+0000 7f3edd703b80 1 -- 172.21.15.1:0/1468939125 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c50eaf3170 con 0x55c50eaff090 2024-02-02T15:28:31.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.672+0000 7f3eb0c00700 1 -- 172.21.15.1:0/1468939125 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3eb4004400 con 0x55c50eaff090 2024-02-02T15:28:31.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.672+0000 7f3edd703b80 10 client.26092 fetch_fsmap learned FSMap version 27 2024-02-02T15:28:31.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.672+0000 7f3edd703b80 1 -- 172.21.15.1:0/1468939125 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x7f3ec41a6190 con 0x55c50eaff090 2024-02-02T15:28:31.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.672+0000 7f3eb0c00700 1 -- 172.21.15.1:0/1468939125 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c50eaf3170 con 0x55c50eaff090 2024-02-02T15:28:31.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.672+0000 7f3eb0c00700 1 -- 172.21.15.1:0/1468939125 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3eb4003ad0 con 0x55c50eaff090 2024-02-02T15:28:31.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.672+0000 7f3edd703b80 10 client.26092 fetch_fsmap finished waiting for FSMap version 27 2024-02-02T15:28:31.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.672+0000 7f3edd703b80 10 client.26092 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:28:31.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.672+0000 7f3edd703b80 20 client.26092 populate_metadata read hostname 'smithi001' 2024-02-02T15:28:31.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.672+0000 7f3edd703b80 1 --2- 172.21.15.1:0/1468939125 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55c50eb19f20 0x55c50eb1c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:31.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.672+0000 7f3edd703b80 4 client.26092 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:28:31.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.672+0000 7f3edd703b80 1 -- 172.21.15.1:0/1468939125 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c50ea49e50 con 0x55c50eb19f20 2024-02-02T15:28:31.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.673+0000 7f3ec2000700 1 --2- 172.21.15.1:0/1468939125 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55c50eb19f20 0x55c50eb1c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:31.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.673+0000 7f3eb0c00700 10 client.26092 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:28:31.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.673+0000 7f3ec2000700 1 --2- 172.21.15.1:0/1468939125 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55c50eb19f20 0x55c50eb1c340 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0782cf9c1c4d7f server_cookie=a41d05f6ca739248 in_seq=0 out_seq=0 2024-02-02T15:28:31.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.674+0000 7f3eb0c00700 1 -- 172.21.15.1:0/1468939125 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55c50ea49e50 con 0x55c50eb19f20 2024-02-02T15:28:31.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.674+0000 7f3eb0c00700 10 client.26092 handle_command_reply: tid=0 2024-02-02T15:28:31.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.674+0000 7f3edd703b80 1 client.26092 shutdown 2024-02-02T15:28:31.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.674+0000 7f3edd703b80 1 -- 172.21.15.1:0/1468939125 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c50eaff090 msgr2=0x55c50eb0ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:28:31.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.674+0000 7f3edd703b80 1 --2- 172.21.15.1:0/1468939125 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c50eaff090 0x55c50eb0ac40 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3eb401dc00 tx=0x7f3eb40047c0 comp rx=0 tx=0).stop 2024-02-02T15:28:31.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.674+0000 7f3edd703b80 1 -- 172.21.15.1:0/1468939125 shutdown_connections 2024-02-02T15:28:31.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.674+0000 7f3edd703b80 1 --2- 172.21.15.1:0/1468939125 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55c50eb19f20 0x55c50eb1c340 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:31.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.674+0000 7f3edd703b80 1 --2- 172.21.15.1:0/1468939125 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c50eb0a860 0x55c50eb0b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:31.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.674+0000 7f3edd703b80 1 --2- 172.21.15.1:0/1468939125 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c50eaff090 0x55c50eb0ac40 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:31.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.674+0000 7f3eb0c00700 0 client.26092 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:28:31.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.674+0000 7f3edd703b80 1 --2- 172.21.15.1:0/1468939125 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c50eaff9b0 0x55c50eb0b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:31.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.674+0000 7f3edd703b80 1 -- 172.21.15.1:0/1468939125 >> 172.21.15.1:0/1468939125 conn(0x55c50ea5ee30 msgr2=0x55c50eafb020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:28:31.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.674+0000 7f3edd703b80 1 -- 172.21.15.1:0/1468939125 shutdown_connections 2024-02-02T15:28:31.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.674+0000 7f3edd703b80 1 -- 172.21.15.1:0/1468939125 wait complete. 2024-02-02T15:28:31.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.675+0000 7f3edd703b80 20 client.26092 trim_cache size 0 max 16384 2024-02-02T15:28:31.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:28:31 smithi001 ceph-mon[35470]: pgmap v48: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-02T15:28:31.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.692+0000 7f3edd703b80 1 Processor -- start 2024-02-02T15:28:31.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.692+0000 7f3edd703b80 1 -- start start 2024-02-02T15:28:31.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.693+0000 7f3edd703b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c50eaff090 0x55c50eb09040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:31.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.693+0000 7f3edd703b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c50eaff9b0 0x55c50eb15230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:31.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.693+0000 7f3edd703b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c50eb0a860 0x55c50eb15770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:31.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.693+0000 7f3edd703b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c50e8d4720 con 0x55c50eaff9b0 2024-02-02T15:28:31.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.693+0000 7f3edd703b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c50eb056d0 con 0x55c50eaff090 2024-02-02T15:28:31.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.693+0000 7f3edd703b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c50eb099e0 con 0x55c50eb0a860 2024-02-02T15:28:31.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.693+0000 7f3ec2a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c50eaff090 0x55c50eb09040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:31.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.693+0000 7f3ec2000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c50eaff9b0 0x55c50eb15230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:31.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.693+0000 7f3ec2a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c50eaff090 0x55c50eb09040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:51500/0 (socket says 172.21.15.1:51500) 2024-02-02T15:28:31.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.693+0000 7f3ec2a00700 1 -- 172.21.15.1:0/43232888 learned_addr learned my addr 172.21.15.1:0/43232888 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:28:31.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.693+0000 7f3ec3400700 1 --2- 172.21.15.1:0/43232888 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c50eb0a860 0x55c50eb15770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:31.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.694+0000 7f3ec2000700 1 -- 172.21.15.1:0/43232888 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c50eb0a860 msgr2=0x55c50eb15770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:28:31.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.694+0000 7f3ec2000700 1 --2- 172.21.15.1:0/43232888 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c50eb0a860 0x55c50eb15770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:31.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.694+0000 7f3ec2000700 1 -- 172.21.15.1:0/43232888 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c50eaff090 msgr2=0x55c50eb09040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:28:31.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.694+0000 7f3ec2000700 1 --2- 172.21.15.1:0/43232888 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c50eaff090 0x55c50eb09040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:31.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.694+0000 7f3ec2000700 1 -- 172.21.15.1:0/43232888 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c50e87ebe0 con 0x55c50eaff9b0 2024-02-02T15:28:31.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.694+0000 7f3ec2a00700 1 --2- 172.21.15.1:0/43232888 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c50eaff090 0x55c50eb09040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:28:31.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.694+0000 7f3ec2000700 1 --2- 172.21.15.1:0/43232888 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c50eaff9b0 0x55c50eb15230 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f3eac004730 tx=0x7f3eac022f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66fda9b9932eae02 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:28:31.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.694+0000 7f3eb8c00700 1 -- 172.21.15.1:0/43232888 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eac09ae60 con 0x55c50eaff9b0 2024-02-02T15:28:31.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.694+0000 7f3eb8c00700 1 -- 172.21.15.1:0/43232888 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3eac064070 con 0x55c50eaff9b0 2024-02-02T15:28:31.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.694+0000 7f3eb8c00700 1 -- 172.21.15.1:0/43232888 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eac0a2740 con 0x55c50eaff9b0 2024-02-02T15:28:31.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.695+0000 7f3edd703b80 1 -- 172.21.15.1:0/43232888 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c50eaff9b0 msgr2=0x55c50eb15230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:28:31.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.695+0000 7f3edd703b80 1 --2- 172.21.15.1:0/43232888 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c50eaff9b0 0x55c50eb15230 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f3eac004730 tx=0x7f3eac022f90 comp rx=0 tx=0).stop 2024-02-02T15:28:31.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.695+0000 7f3edd703b80 1 -- 172.21.15.1:0/43232888 shutdown_connections 2024-02-02T15:28:31.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.695+0000 7f3edd703b80 1 --2- 172.21.15.1:0/43232888 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c50eb0a860 0x55c50eb15770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:31.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.695+0000 7f3edd703b80 1 --2- 172.21.15.1:0/43232888 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c50eaff090 0x55c50eb09040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:31.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.695+0000 7f3edd703b80 1 --2- 172.21.15.1:0/43232888 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c50eaff9b0 0x55c50eb15230 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:31.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.695+0000 7f3edd703b80 1 -- 172.21.15.1:0/43232888 >> 172.21.15.1:0/43232888 conn(0x55c50ea5ee30 msgr2=0x55c50eb0b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:28:31.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.695+0000 7f3edd703b80 1 -- 172.21.15.1:0/43232888 shutdown_connections 2024-02-02T15:28:31.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.695+0000 7f3edd703b80 1 -- 172.21.15.1:0/43232888 wait complete. 2024-02-02T15:28:31.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.695+0000 7f3edd703b80 1 -- start start 2024-02-02T15:28:31.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.695+0000 7f3edd703b80 1 Processor -- start 2024-02-02T15:28:31.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.696+0000 7f3edd703b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c50eaff090 0x55c50eb0e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:31.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.696+0000 7f3edd703b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c50eaff9b0 0x55c50eb0e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:31.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.696+0000 7f3edd703b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c50eb0a860 0x55c50eb0ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:31.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.696+0000 7f3edd703b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c50eb6bb80 con 0x55c50eb0a860 2024-02-02T15:28:31.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.696+0000 7f3edd703b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c50eb6bcc0 con 0x55c50eaff9b0 2024-02-02T15:28:31.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.696+0000 7f3edd703b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c50eb6be80 con 0x55c50eaff090 2024-02-02T15:28:31.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.696+0000 7f3edd703b80 10 client.0 _ll_register_callbacks cb 0x55c50e9ee310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:28:31.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.696+0000 7f3ec2a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c50eaff090 0x55c50eb0e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:31.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.696+0000 7f3ec2000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c50eaff9b0 0x55c50eb0e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:31.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.696+0000 7f3ec2a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c50eaff090 0x55c50eb0e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:47844/0 (socket says 172.21.15.1:47844) 2024-02-02T15:28:31.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.696+0000 7f3ec3400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c50eb0a860 0x55c50eb0ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:31.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.696+0000 7f3ec2a00700 1 -- 172.21.15.1:0/3244969048 learned_addr learned my addr 172.21.15.1:0/3244969048 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:28:31.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.697+0000 7f3ec2a00700 1 -- 172.21.15.1:0/3244969048 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c50eaff9b0 msgr2=0x55c50eb0e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:28:31.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.697+0000 7f3ec2a00700 1 --2- 172.21.15.1:0/3244969048 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c50eaff9b0 0x55c50eb0e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:31.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.697+0000 7f3ec2a00700 1 -- 172.21.15.1:0/3244969048 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c50eb0a860 msgr2=0x55c50eb0ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:28:31.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.697+0000 7f3ec2a00700 1 --2- 172.21.15.1:0/3244969048 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c50eb0a860 0x55c50eb0ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:31.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.697+0000 7f3ec2a00700 1 -- 172.21.15.1:0/3244969048 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3eb402c040 con 0x55c50eaff090 2024-02-02T15:28:31.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.697+0000 7f3ec2a00700 1 --2- 172.21.15.1:0/3244969048 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c50eaff090 0x55c50eb0e2f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3eb401dc00 tx=0x7f3eb4004560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d407955b2bad6fd server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:28:31.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.697+0000 7f3eb0c00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:28:31.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.697+0000 7f3eb0c00700 1 -- 172.21.15.1:0/3244969048 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb4022040 con 0x55c50eaff090 2024-02-02T15:28:31.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.697+0000 7f3eb0c00700 1 -- 172.21.15.1:0/3244969048 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f3eb4003890 con 0x55c50eaff090 2024-02-02T15:28:31.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.697+0000 7f3edd703b80 1 -- 172.21.15.1:0/3244969048 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3eac09a0c0 con 0x55c50eaff090 2024-02-02T15:28:31.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.697+0000 7f3eb0c00700 1 -- 172.21.15.1:0/3244969048 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3eb4003a50 con 0x55c50eaff090 2024-02-02T15:28:31.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.697+0000 7f3edd703b80 10 client.26098 fetch_fsmap learned FSMap version 27 2024-02-02T15:28:31.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.697+0000 7f3edd703b80 1 -- 172.21.15.1:0/3244969048 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55c50e8d2430 con 0x55c50eaff090 2024-02-02T15:28:31.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.698+0000 7f3eb0c00700 1 -- 172.21.15.1:0/3244969048 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3eac09a0c0 con 0x55c50eaff090 2024-02-02T15:28:31.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.698+0000 7f3eb0c00700 1 -- 172.21.15.1:0/3244969048 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3eb4027a90 con 0x55c50eaff090 2024-02-02T15:28:31.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.698+0000 7f3edd703b80 10 client.26098 fetch_fsmap finished waiting for FSMap version 27 2024-02-02T15:28:31.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.698+0000 7f3edd703b80 10 client.26098 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:28:31.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.698+0000 7f3edd703b80 20 client.26098 populate_metadata read hostname 'smithi001' 2024-02-02T15:28:31.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.698+0000 7f3edd703b80 1 --2- 172.21.15.1:0/3244969048 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55c50eb6d540 0x55c50eb6f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:31.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.698+0000 7f3edd703b80 4 client.26098 mds_command: new command op to 24557 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-02T15:28:31.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.698+0000 7f3edd703b80 1 -- 172.21.15.1:0/3244969048 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55c50ea0ffd0 con 0x55c50eb6d540 2024-02-02T15:28:31.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.698+0000 7f3ec2000700 1 --2- 172.21.15.1:0/3244969048 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55c50eb6d540 0x55c50eb6f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:31.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.699+0000 7f3ec2000700 1 --2- 172.21.15.1:0/3244969048 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55c50eb6d540 0x55c50eb6f960 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19abb3ed48b4e723 server_cookie=4eb122ec6414752e in_seq=0 out_seq=0 2024-02-02T15:28:31.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.699+0000 7f3eb0c00700 10 client.26098 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:28:31.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.700+0000 7f3eb0c00700 1 -- 172.21.15.1:0/3244969048 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55c50ea0ffd0 con 0x55c50eb6d540 2024-02-02T15:28:31.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.700+0000 7f3eb0c00700 10 client.26098 handle_command_reply: tid=0 2024-02-02T15:28:31.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.700+0000 7f3edd703b80 1 client.26098 shutdown 2024-02-02T15:28:31.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.700+0000 7f3edd703b80 1 -- 172.21.15.1:0/3244969048 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c50eaff090 msgr2=0x55c50eb0e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:28:31.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.700+0000 7f3edd703b80 1 --2- 172.21.15.1:0/3244969048 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c50eaff090 0x55c50eb0e2f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3eb401dc00 tx=0x7f3eb4004560 comp rx=0 tx=0).stop 2024-02-02T15:28:31.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.700+0000 7f3edd703b80 1 -- 172.21.15.1:0/3244969048 shutdown_connections 2024-02-02T15:28:31.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.700+0000 7f3edd703b80 1 --2- 172.21.15.1:0/3244969048 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55c50eb6d540 0x55c50eb6f960 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:31.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.701+0000 7f3edd703b80 1 --2- 172.21.15.1:0/3244969048 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c50eb0a860 0x55c50eb0ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:31.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.701+0000 7f3edd703b80 1 --2- 172.21.15.1:0/3244969048 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c50eaff090 0x55c50eb0e2f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:31.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.701+0000 7f3eb0c00700 0 client.26098 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:28:31.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.701+0000 7f3edd703b80 1 --2- 172.21.15.1:0/3244969048 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c50eaff9b0 0x55c50eb0e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:31.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.701+0000 7f3edd703b80 1 -- 172.21.15.1:0/3244969048 >> 172.21.15.1:0/3244969048 conn(0x55c50ea5ee30 msgr2=0x55c50eb008b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:28:31.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.701+0000 7f3edd703b80 1 -- 172.21.15.1:0/3244969048 shutdown_connections 2024-02-02T15:28:31.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.701+0000 7f3edd703b80 1 -- 172.21.15.1:0/3244969048 wait complete. 2024-02-02T15:28:31.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.701+0000 7f3edd703b80 20 client.26098 trim_cache size 0 max 16384 2024-02-02T15:28:31.704 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:28:31.704 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2024-02-02T15:28:31.704 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "9490b50f-8ec9-4055-998a-01b17cbfcd58", 2024-02-02T15:28:31.705 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2024-02-02T15:28:31.705 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:28:31.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.701+0000 7f3ec9c00700 1 -- 172.21.15.1:0/3733254540 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f3ea40a07b0 msgr2=0x7f3ea40a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:28:31.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.701+0000 7f3ec9c00700 1 --2- 172.21.15.1:0/3733254540 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f3ea40a07b0 0x7f3ea40a2c70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3eb4009c60 tx=0x7f3eb4009260 comp rx=0 tx=0).stop 2024-02-02T15:28:31.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.701+0000 7f3ec9c00700 1 -- 172.21.15.1:0/3733254540 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ec4106110 msgr2=0x7f3ec41996b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:28:31.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.701+0000 7f3ec9c00700 1 --2- 172.21.15.1:0/3733254540 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ec4106110 0x7f3ec41996b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f3eac004760 tx=0x7f3eac004790 comp rx=0 tx=0).stop 2024-02-02T15:28:31.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.701+0000 7f3ec9c00700 1 -- 172.21.15.1:0/3733254540 shutdown_connections 2024-02-02T15:28:31.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.702+0000 7f3ec9c00700 1 --2- 172.21.15.1:0/3733254540 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f3ea40a07b0 0x7f3ea40a2c70 secure :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3eb4009c60 tx=0x7f3eb4009260 comp rx=0 tx=0).stop 2024-02-02T15:28:31.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.702+0000 7f3ec9c00700 1 --2- 172.21.15.1:0/3733254540 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ec419db10 0x7f3ec419df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:31.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.702+0000 7f3ec9c00700 1 --2- 172.21.15.1:0/3733254540 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ec4100ff0 0x7f3ec4199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:31.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.702+0000 7f3ec9c00700 1 --2- 172.21.15.1:0/3733254540 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ec4106110 0x7f3ec41996b0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:31.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.702+0000 7f3ec9c00700 1 -- 172.21.15.1:0/3733254540 >> 172.21.15.1:0/3733254540 conn(0x7f3ec40fc810 msgr2=0x7f3ec40fcf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:28:31.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.702+0000 7f3ec9c00700 1 -- 172.21.15.1:0/3733254540 shutdown_connections 2024-02-02T15:28:31.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.702+0000 7f3ec9c00700 1 -- 172.21.15.1:0/3733254540 wait complete. 2024-02-02T15:28:31.720 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-02T15:28:31.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:28:31 smithi071 ceph-mon[44707]: pgmap v48: 129 pgs: 129 active+clean; 1.3 MiB data, 329 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-02-02T15:28:31.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.933+0000 7f3ed3800700 1 -- 172.21.15.1:0/141970748 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ecc106ed0 msgr2=0x7f3ecc1072b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:28:31.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.933+0000 7f3ed3800700 1 --2- 172.21.15.1:0/141970748 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ecc106ed0 0x7f3ecc1072b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc0098f0 tx=0x7f3ebc009c00 comp rx=0 tx=0).stop 2024-02-02T15:28:31.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.933+0000 7f3ed3800700 1 -- 172.21.15.1:0/141970748 shutdown_connections 2024-02-02T15:28:31.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.933+0000 7f3ed3800700 1 --2- 172.21.15.1:0/141970748 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ecc0689b0 0x7f3ecc10dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:31.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.933+0000 7f3ed3800700 1 --2- 172.21.15.1:0/141970748 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ecc106ed0 0x7f3ecc1072b0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:31.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.933+0000 7f3ed3800700 1 --2- 172.21.15.1:0/141970748 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ecc068010 0x7f3ecc068470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:31.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.933+0000 7f3ed3800700 1 -- 172.21.15.1:0/141970748 >> 172.21.15.1:0/141970748 conn(0x7f3ecc0757d0 msgr2=0x7f3ecc075be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:28:31.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.934+0000 7f3ed3800700 1 -- 172.21.15.1:0/141970748 shutdown_connections 2024-02-02T15:28:31.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.934+0000 7f3ed3800700 1 -- 172.21.15.1:0/141970748 wait complete. 2024-02-02T15:28:31.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.935+0000 7f3ed3800700 1 Processor -- start 2024-02-02T15:28:31.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.935+0000 7f3ed3800700 1 -- start start 2024-02-02T15:28:31.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.935+0000 7f3ed3800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ecc068010 0x7f3ecc0fff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:31.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.935+0000 7f3ed3800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ecc0689b0 0x7f3ecc100480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:31.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.935+0000 7f3ed3800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ecc106ed0 0x7f3ecc1a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:31.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.935+0000 7f3ed3800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ecc100cb0 con 0x7f3ecc0689b0 2024-02-02T15:28:31.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.935+0000 7f3ed3800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ecc1a56d0 con 0x7f3ecc068010 2024-02-02T15:28:31.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.935+0000 7f3ed3800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ecc1a5810 con 0x7f3ecc106ed0 2024-02-02T15:28:31.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.935+0000 7f3ed0e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ecc068010 0x7f3ecc0fff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:31.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.935+0000 7f3ed1800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ecc106ed0 0x7f3ecc1a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:31.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.935+0000 7f3ecbe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ecc0689b0 0x7f3ecc100480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:31.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.935+0000 7f3ed0e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ecc068010 0x7f3ecc0fff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:51518/0 (socket says 172.21.15.1:51518) 2024-02-02T15:28:31.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.935+0000 7f3ed1800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ecc106ed0 0x7f3ecc1a5190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:47858/0 (socket says 172.21.15.1:47858) 2024-02-02T15:28:31.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.935+0000 7f3ed0e00700 1 -- 172.21.15.1:0/2113901234 learned_addr learned my addr 172.21.15.1:0/2113901234 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:28:31.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.936+0000 7f3ed0e00700 1 -- 172.21.15.1:0/2113901234 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ecc106ed0 msgr2=0x7f3ecc1a5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:28:31.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.936+0000 7f3ed0e00700 1 --2- 172.21.15.1:0/2113901234 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ecc106ed0 0x7f3ecc1a5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:31.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.936+0000 7f3ed0e00700 1 -- 172.21.15.1:0/2113901234 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ecc0689b0 msgr2=0x7f3ecc100480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:28:31.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.936+0000 7f3ed0e00700 1 --2- 172.21.15.1:0/2113901234 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ecc0689b0 0x7f3ecc100480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:31.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.936+0000 7f3ed0e00700 1 -- 172.21.15.1:0/2113901234 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ebc009520 con 0x7f3ecc068010 2024-02-02T15:28:31.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.936+0000 7f3ecbe00700 1 --2- 172.21.15.1:0/2113901234 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ecc0689b0 0x7f3ecc100480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:28:31.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.936+0000 7f3ed0e00700 1 --2- 172.21.15.1:0/2113901234 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ecc068010 0x7f3ecc0fff40 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc009810 tx=0x7f3ebc00fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:28:31.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.936+0000 7f3ec9600700 1 -- 172.21.15.1:0/2113901234 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc01c070 con 0x7f3ecc068010 2024-02-02T15:28:31.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.936+0000 7f3ed3800700 1 -- 172.21.15.1:0/2113901234 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ecc1a5a10 con 0x7f3ecc068010 2024-02-02T15:28:31.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.936+0000 7f3ec9600700 1 -- 172.21.15.1:0/2113901234 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ebc005040 con 0x7f3ecc068010 2024-02-02T15:28:31.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.936+0000 7f3ec9600700 1 -- 172.21.15.1:0/2113901234 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc004930 con 0x7f3ecc068010 2024-02-02T15:28:31.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.937+0000 7f3ed3800700 1 -- 172.21.15.1:0/2113901234 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ecc1a5ee0 con 0x7f3ecc068010 2024-02-02T15:28:31.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.939+0000 7f3ec9600700 1 -- 172.21.15.1:0/2113901234 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) v1 ==== 135302+0+0 (secure 0 0 0) 0x7f3ebc020070 con 0x7f3ecc068010 2024-02-02T15:28:31.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.941+0000 7f3ee73d7b80 1 Processor -- start 2024-02-02T15:28:31.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.941+0000 7f3ee73d7b80 1 -- start start 2024-02-02T15:28:31.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.942+0000 7f3ec9600700 1 --2- 172.21.15.1:0/2113901234 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f3eac0a0830 0x7f3eac0a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:31.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.942+0000 7f3ee73d7b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56331a4df010 0x56331a4df3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:31.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.942+0000 7f3ee73d7b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56331a4df930 0x56331a4f2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:31.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.942+0000 7f3ee73d7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56331a4f2ff0 0x56331a4f53e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:31.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.942+0000 7f3ee73d7b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56331a4534c0 con 0x56331a4df930 2024-02-02T15:28:31.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.942+0000 7f3ee73d7b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56331a437aa0 con 0x56331a4df010 2024-02-02T15:28:31.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.942+0000 7f3ee73d7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56331a435460 con 0x56331a4f2ff0 2024-02-02T15:28:31.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.942+0000 7f3ec9600700 1 -- 172.21.15.1:0/2113901234 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f3ebc0cd0c0 con 0x7f3ecc068010 2024-02-02T15:28:31.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.942+0000 7f3ed1800700 1 --2- 172.21.15.1:0/2113901234 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f3eac0a0830 0x7f3eac0a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:31.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.942+0000 7f3ecbe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56331a4df930 0x56331a4f2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:31.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.942+0000 7f3ed0e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56331a4df010 0x56331a4df3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:31.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.942+0000 7f3ecbe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56331a4df930 0x56331a4f2ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49892/0 (socket says 172.21.15.1:49892) 2024-02-02T15:28:31.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.942+0000 7f3ecbe00700 1 -- 172.21.15.1:0/3755404756 learned_addr learned my addr 172.21.15.1:0/3755404756 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:28:31.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.943+0000 7f3ed1800700 1 --2- 172.21.15.1:0/3755404756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56331a4f2ff0 0x56331a4f53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:31.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.943+0000 7f3ed0e00700 1 -- 172.21.15.1:0/3755404756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56331a4f2ff0 msgr2=0x56331a4f53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:28:31.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.943+0000 7f3ed0e00700 1 --2- 172.21.15.1:0/3755404756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56331a4f2ff0 0x56331a4f53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:31.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.943+0000 7f3ed0e00700 1 -- 172.21.15.1:0/3755404756 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56331a4df930 msgr2=0x56331a4f2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:28:31.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.943+0000 7f3ed0e00700 1 --2- 172.21.15.1:0/3755404756 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56331a4df930 0x56331a4f2ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:31.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.943+0000 7f3ed0e00700 1 -- 172.21.15.1:0/3755404756 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56331a2b23b0 con 0x56331a4df010 2024-02-02T15:28:31.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.943+0000 7f3ed1800700 1 --2- 172.21.15.1:0/3755404756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56331a4f2ff0 0x56331a4f53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:28:31.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.943+0000 7f3ed0e00700 1 --2- 172.21.15.1:0/3755404756 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56331a4df010 0x56331a4df3f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc0ca2b0 tx=0x7f3ebc099fe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90f34a14653e47a7 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:28:31.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.943+0000 7f3ed1800700 1 --2- 172.21.15.1:0/2113901234 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f3eac0a0830 0x7f3eac0a2cf0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3ec4009ec0 tx=0x7f3ec4009920 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:28:31.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.943+0000 7f3ec2000700 1 -- 172.21.15.1:0/3755404756 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc09a890 con 0x56331a4df010 2024-02-02T15:28:31.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.943+0000 7f3ec2000700 1 -- 172.21.15.1:0/3755404756 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ebc09a190 con 0x56331a4df010 2024-02-02T15:28:31.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.943+0000 7f3ec2000700 1 -- 172.21.15.1:0/3755404756 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc09a630 con 0x56331a4df010 2024-02-02T15:28:31.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.944+0000 7f3ee73d7b80 1 -- 172.21.15.1:0/3755404756 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56331a4df010 msgr2=0x56331a4df3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:28:31.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.944+0000 7f3ee73d7b80 1 --2- 172.21.15.1:0/3755404756 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56331a4df010 0x56331a4df3f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc0ca2b0 tx=0x7f3ebc099fe0 comp rx=0 tx=0).stop 2024-02-02T15:28:31.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.944+0000 7f3ee73d7b80 1 -- 172.21.15.1:0/3755404756 shutdown_connections 2024-02-02T15:28:31.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.944+0000 7f3ee73d7b80 1 --2- 172.21.15.1:0/3755404756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56331a4f2ff0 0x56331a4f53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:31.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.944+0000 7f3ee73d7b80 1 --2- 172.21.15.1:0/3755404756 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56331a4df010 0x56331a4df3f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:31.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.944+0000 7f3ee73d7b80 1 --2- 172.21.15.1:0/3755404756 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56331a4df930 0x56331a4f2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:31.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.944+0000 7f3ee73d7b80 1 -- 172.21.15.1:0/3755404756 >> 172.21.15.1:0/3755404756 conn(0x56331a43edb0 msgr2=0x56331a4db3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:28:31.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.944+0000 7f3ee73d7b80 1 -- 172.21.15.1:0/3755404756 shutdown_connections 2024-02-02T15:28:31.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.944+0000 7f3ee73d7b80 1 -- 172.21.15.1:0/3755404756 wait complete. 2024-02-02T15:28:31.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.945+0000 7f3ee73d7b80 1 -- start start 2024-02-02T15:28:31.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.945+0000 7f3ee73d7b80 1 Processor -- start 2024-02-02T15:28:31.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.945+0000 7f3ee73d7b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56331a4df010 0x56331a4f33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:31.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.945+0000 7f3ee73d7b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56331a4df930 0x56331a4f3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:31.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.945+0000 7f3ee73d7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56331a4f2ff0 0x56331a4f3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:31.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.945+0000 7f3ee73d7b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56331a430d50 con 0x56331a4df930 2024-02-02T15:28:31.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.945+0000 7f3ee73d7b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56331a3ccd80 con 0x56331a4df010 2024-02-02T15:28:31.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.945+0000 7f3ee73d7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56331a3cca20 con 0x56331a4f2ff0 2024-02-02T15:28:31.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.945+0000 7f3ee73d7b80 10 client.0 _ll_register_callbacks cb 0x56331a3ce290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:28:31.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.946+0000 7f3ed0e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56331a4df010 0x56331a4f33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:31.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.946+0000 7f3ed0e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56331a4df010 0x56331a4f33d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:51540/0 (socket says 172.21.15.1:51540) 2024-02-02T15:28:31.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.946+0000 7f3ed1800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56331a4f2ff0 0x56331a4f3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:31.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.946+0000 7f3ed0e00700 1 -- 172.21.15.1:0/2879567026 learned_addr learned my addr 172.21.15.1:0/2879567026 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:28:31.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.946+0000 7f3ecbe00700 1 --2- 172.21.15.1:0/2879567026 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56331a4df930 0x56331a4f3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:31.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.946+0000 7f3ed0e00700 1 -- 172.21.15.1:0/2879567026 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56331a4f2ff0 msgr2=0x56331a4f3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:28:31.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.946+0000 7f3ed0e00700 1 --2- 172.21.15.1:0/2879567026 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56331a4f2ff0 0x56331a4f3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:31.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.946+0000 7f3ed0e00700 1 -- 172.21.15.1:0/2879567026 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56331a4df930 msgr2=0x56331a4f3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:28:31.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.946+0000 7f3ed0e00700 1 --2- 172.21.15.1:0/2879567026 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56331a4df930 0x56331a4f3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:31.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.946+0000 7f3ed0e00700 1 -- 172.21.15.1:0/2879567026 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ecc1a5ee0 con 0x56331a4df010 2024-02-02T15:28:31.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.946+0000 7f3ecbe00700 1 --2- 172.21.15.1:0/2879567026 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56331a4df930 0x56331a4f3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:28:31.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.946+0000 7f3ed0e00700 1 --2- 172.21.15.1:0/2879567026 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56331a4df010 0x56331a4f33d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc009fd0 tx=0x7f3ebc0998b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7774e187575d58d server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:28:31.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.946+0000 7f3ebaa00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:28:31.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.946+0000 7f3ebaa00700 1 -- 172.21.15.1:0/2879567026 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc09dfc0 con 0x56331a4df010 2024-02-02T15:28:31.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.946+0000 7f3ebaa00700 1 -- 172.21.15.1:0/2879567026 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f3ebc0a2170 con 0x56331a4df010 2024-02-02T15:28:31.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.946+0000 7f3ee73d7b80 1 -- 172.21.15.1:0/2879567026 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56331a4d30f0 con 0x56331a4df010 2024-02-02T15:28:31.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.947+0000 7f3ebaa00700 1 -- 172.21.15.1:0/2879567026 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ebc09a600 con 0x56331a4df010 2024-02-02T15:28:31.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.947+0000 7f3ee73d7b80 10 client.26114 fetch_fsmap learned FSMap version 27 2024-02-02T15:28:31.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.947+0000 7f3ee73d7b80 1 -- 172.21.15.1:0/2879567026 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56331a25eb60 con 0x56331a4df010 2024-02-02T15:28:31.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.947+0000 7f3ebaa00700 1 -- 172.21.15.1:0/2879567026 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ebc0a7020 con 0x56331a4df010 2024-02-02T15:28:31.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.947+0000 7f3ebaa00700 1 -- 172.21.15.1:0/2879567026 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3ebc0cabb0 con 0x56331a4df010 2024-02-02T15:28:31.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.947+0000 7f3ee73d7b80 10 client.26114 fetch_fsmap finished waiting for FSMap version 27 2024-02-02T15:28:31.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.947+0000 7f3ee73d7b80 10 client.26114 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:28:31.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.947+0000 7f3ee73d7b80 20 client.26114 populate_metadata read hostname 'smithi001' 2024-02-02T15:28:31.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.947+0000 7f3ee73d7b80 1 --2- 172.21.15.1:0/2879567026 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56331a502190 0x56331a5045b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:31.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.947+0000 7f3ee73d7b80 4 client.26114 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:28:31.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.947+0000 7f3ee73d7b80 1 -- 172.21.15.1:0/2879567026 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56331a429dd0 con 0x56331a502190 2024-02-02T15:28:31.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.947+0000 7f3ecbe00700 1 --2- 172.21.15.1:0/2879567026 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56331a502190 0x56331a5045b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:31.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.948+0000 7f3ecbe00700 1 --2- 172.21.15.1:0/2879567026 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56331a502190 0x56331a5045b0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e94fe5a153a36af server_cookie=9b921fdc3e8d765c in_seq=0 out_seq=0 2024-02-02T15:28:31.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.948+0000 7f3ebaa00700 10 client.26114 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:28:31.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.948+0000 7f3ebaa00700 1 -- 172.21.15.1:0/2879567026 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x56331a429dd0 con 0x56331a502190 2024-02-02T15:28:31.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.948+0000 7f3ebaa00700 10 client.26114 handle_command_reply: tid=0 2024-02-02T15:28:31.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.949+0000 7f3ee73d7b80 1 client.26114 shutdown 2024-02-02T15:28:31.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.949+0000 7f3ee73d7b80 1 -- 172.21.15.1:0/2879567026 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56331a4df010 msgr2=0x56331a4f33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:28:31.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.949+0000 7f3ee73d7b80 1 --2- 172.21.15.1:0/2879567026 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56331a4df010 0x56331a4f33d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc009fd0 tx=0x7f3ebc0998b0 comp rx=0 tx=0).stop 2024-02-02T15:28:31.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.949+0000 7f3ee73d7b80 1 -- 172.21.15.1:0/2879567026 shutdown_connections 2024-02-02T15:28:31.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.949+0000 7f3ee73d7b80 1 --2- 172.21.15.1:0/2879567026 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56331a502190 0x56331a5045b0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:31.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.949+0000 7f3ee73d7b80 1 --2- 172.21.15.1:0/2879567026 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56331a4f2ff0 0x56331a4f3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:31.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.949+0000 7f3ee73d7b80 1 --2- 172.21.15.1:0/2879567026 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56331a4df010 0x56331a4f33d0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:31.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.949+0000 7f3ebaa00700 0 client.26114 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:28:31.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.949+0000 7f3ee73d7b80 1 --2- 172.21.15.1:0/2879567026 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56331a4df930 0x56331a4f3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:31.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.949+0000 7f3ee73d7b80 1 -- 172.21.15.1:0/2879567026 >> 172.21.15.1:0/2879567026 conn(0x56331a43edb0 msgr2=0x56331a4daf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:28:31.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.949+0000 7f3ee73d7b80 1 -- 172.21.15.1:0/2879567026 shutdown_connections 2024-02-02T15:28:31.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.949+0000 7f3ee73d7b80 1 -- 172.21.15.1:0/2879567026 wait complete. 2024-02-02T15:28:31.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.949+0000 7f3ee73d7b80 20 client.26114 trim_cache size 0 max 16384 2024-02-02T15:28:31.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.967+0000 7f3ee73d7b80 1 Processor -- start 2024-02-02T15:28:31.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.967+0000 7f3ee73d7b80 1 -- start start 2024-02-02T15:28:31.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.967+0000 7f3ee73d7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56331a4df010 0x56331a4f7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:31.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.967+0000 7f3ee73d7b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56331a4df930 0x56331a4e1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:31.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.967+0000 7f3ee73d7b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56331a4f2ff0 0x56331a4f18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:31.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.967+0000 7f3ee73d7b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56331a2b46a0 con 0x56331a4f2ff0 2024-02-02T15:28:31.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.967+0000 7f3ee73d7b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56331a4f2680 con 0x56331a4df930 2024-02-02T15:28:31.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.967+0000 7f3ee73d7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56331a4f05b0 con 0x56331a4df010 2024-02-02T15:28:31.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.967+0000 7f3ecbe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56331a4df930 0x56331a4e1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:31.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.967+0000 7f3ed0e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56331a4df010 0x56331a4f7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:31.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.967+0000 7f3ecbe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56331a4df930 0x56331a4e1ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:51556/0 (socket says 172.21.15.1:51556) 2024-02-02T15:28:31.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.967+0000 7f3ecbe00700 1 -- 172.21.15.1:0/372988642 learned_addr learned my addr 172.21.15.1:0/372988642 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:28:31.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.967+0000 7f3ed1800700 1 --2- 172.21.15.1:0/372988642 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56331a4f2ff0 0x56331a4f18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:31.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.968+0000 7f3ecbe00700 1 -- 172.21.15.1:0/372988642 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56331a4df010 msgr2=0x56331a4f7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:28:31.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.968+0000 7f3ecbe00700 1 --2- 172.21.15.1:0/372988642 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56331a4df010 0x56331a4f7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:31.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.968+0000 7f3ecbe00700 1 -- 172.21.15.1:0/372988642 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56331a4f2ff0 msgr2=0x56331a4f18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:28:31.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.968+0000 7f3ecbe00700 1 --2- 172.21.15.1:0/372988642 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56331a4f2ff0 0x56331a4f18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:31.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.968+0000 7f3ecbe00700 1 -- 172.21.15.1:0/372988642 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56331a3f6ea0 con 0x56331a4df930 2024-02-02T15:28:31.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.968+0000 7f3ed1800700 1 --2- 172.21.15.1:0/372988642 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56331a4f2ff0 0x56331a4f18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:28:31.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.968+0000 7f3ecbe00700 1 --2- 172.21.15.1:0/372988642 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56331a4df930 0x56331a4e1ab0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x56331a4f25a0 tx=0x7f3eb4009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42ef651f59f36337 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:28:31.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.968+0000 7f3ec2a00700 1 -- 172.21.15.1:0/372988642 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb4011e20 con 0x56331a4df930 2024-02-02T15:28:31.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.968+0000 7f3ec2a00700 1 -- 172.21.15.1:0/372988642 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3eb4018040 con 0x56331a4df930 2024-02-02T15:28:31.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.968+0000 7f3ec2a00700 1 -- 172.21.15.1:0/372988642 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eb4004a70 con 0x56331a4df930 2024-02-02T15:28:31.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.969+0000 7f3ee73d7b80 1 -- 172.21.15.1:0/372988642 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56331a4df930 msgr2=0x56331a4e1ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:28:31.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.969+0000 7f3ee73d7b80 1 --2- 172.21.15.1:0/372988642 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56331a4df930 0x56331a4e1ab0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x56331a4f25a0 tx=0x7f3eb4009310 comp rx=0 tx=0).stop 2024-02-02T15:28:31.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.969+0000 7f3ee73d7b80 1 -- 172.21.15.1:0/372988642 shutdown_connections 2024-02-02T15:28:31.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.969+0000 7f3ee73d7b80 1 --2- 172.21.15.1:0/372988642 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56331a4f2ff0 0x56331a4f18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:31.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.969+0000 7f3ee73d7b80 1 --2- 172.21.15.1:0/372988642 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56331a4df010 0x56331a4f7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:31.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.969+0000 7f3ee73d7b80 1 --2- 172.21.15.1:0/372988642 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56331a4df930 0x56331a4e1ab0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:31.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.969+0000 7f3ee73d7b80 1 -- 172.21.15.1:0/372988642 >> 172.21.15.1:0/372988642 conn(0x56331a43edb0 msgr2=0x56331a4fe960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:28:31.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.969+0000 7f3ee73d7b80 1 -- 172.21.15.1:0/372988642 shutdown_connections 2024-02-02T15:28:31.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.969+0000 7f3ee73d7b80 1 -- 172.21.15.1:0/372988642 wait complete. 2024-02-02T15:28:31.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.969+0000 7f3ee73d7b80 1 -- start start 2024-02-02T15:28:31.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.970+0000 7f3ee73d7b80 1 Processor -- start 2024-02-02T15:28:31.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.970+0000 7f3ee73d7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56331a4df010 0x56331a547e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:31.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.970+0000 7f3ee73d7b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56331a4df930 0x56331a548370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:31.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.970+0000 7f3ee73d7b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56331a4f2ff0 0x56331a552cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:31.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.970+0000 7f3ee73d7b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56331a553200 con 0x56331a4df930 2024-02-02T15:28:31.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.970+0000 7f3ee73d7b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56331a553340 con 0x56331a4f2ff0 2024-02-02T15:28:31.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.970+0000 7f3ee73d7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56331a553480 con 0x56331a4df010 2024-02-02T15:28:31.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.971+0000 7f3ed0e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56331a4df010 0x56331a547e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:31.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.971+0000 7f3ee73d7b80 10 client.0 _ll_register_callbacks cb 0x56331a3ce290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:28:31.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.971+0000 7f3ed0e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56331a4df010 0x56331a547e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:47884/0 (socket says 172.21.15.1:47884) 2024-02-02T15:28:31.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.971+0000 7f3ed0e00700 1 -- 172.21.15.1:0/3129838630 learned_addr learned my addr 172.21.15.1:0/3129838630 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:28:31.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.971+0000 7f3ed1800700 1 --2- 172.21.15.1:0/3129838630 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56331a4f2ff0 0x56331a552cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:31.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.971+0000 7f3ecbe00700 1 --2- 172.21.15.1:0/3129838630 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56331a4df930 0x56331a548370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:31.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.971+0000 7f3ed0e00700 1 -- 172.21.15.1:0/3129838630 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56331a4f2ff0 msgr2=0x56331a552cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:28:31.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.971+0000 7f3ed0e00700 1 --2- 172.21.15.1:0/3129838630 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56331a4f2ff0 0x56331a552cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:31.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.971+0000 7f3ed0e00700 1 -- 172.21.15.1:0/3129838630 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56331a4df930 msgr2=0x56331a548370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:28:31.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.971+0000 7f3ed0e00700 1 --2- 172.21.15.1:0/3129838630 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56331a4df930 0x56331a548370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:31.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.971+0000 7f3ed0e00700 1 -- 172.21.15.1:0/3129838630 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3eb401f870 con 0x56331a4df010 2024-02-02T15:28:31.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.971+0000 7f3ecbe00700 1 --2- 172.21.15.1:0/3129838630 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56331a4df930 0x56331a548370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:28:31.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.971+0000 7f3ed0e00700 1 --2- 172.21.15.1:0/3129838630 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56331a4df010 0x56331a547e30 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc09a120 tx=0x7f3ebc098740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b06388f3ebdd34a7 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:28:31.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.971+0000 7f3ebaa00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:28:31.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.971+0000 7f3ebaa00700 1 -- 172.21.15.1:0/3129838630 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc0addc0 con 0x56331a4df010 2024-02-02T15:28:31.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.972+0000 7f3ebaa00700 1 -- 172.21.15.1:0/3129838630 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f3ebc0a8080 con 0x56331a4df010 2024-02-02T15:28:31.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.972+0000 7f3ee73d7b80 1 -- 172.21.15.1:0/3129838630 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3ebc09a190 con 0x56331a4df010 2024-02-02T15:28:31.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.972+0000 7f3ebaa00700 1 -- 172.21.15.1:0/3129838630 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ebc095fd0 con 0x56331a4df010 2024-02-02T15:28:31.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.972+0000 7f3ee73d7b80 10 client.26116 fetch_fsmap learned FSMap version 27 2024-02-02T15:28:31.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.972+0000 7f3ee73d7b80 1 -- 172.21.15.1:0/3129838630 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x56331a2b6cd0 con 0x56331a4df010 2024-02-02T15:28:31.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.972+0000 7f3ebaa00700 1 -- 172.21.15.1:0/3129838630 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ebc09a190 con 0x56331a4df010 2024-02-02T15:28:31.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.972+0000 7f3ebaa00700 1 -- 172.21.15.1:0/3129838630 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 27) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3ebc0a1e00 con 0x56331a4df010 2024-02-02T15:28:31.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.972+0000 7f3ee73d7b80 10 client.26116 fetch_fsmap finished waiting for FSMap version 27 2024-02-02T15:28:31.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.972+0000 7f3ee73d7b80 10 client.26116 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:28:31.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.972+0000 7f3ee73d7b80 20 client.26116 populate_metadata read hostname 'smithi001' 2024-02-02T15:28:31.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.972+0000 7f3ee73d7b80 1 --2- 172.21.15.1:0/3129838630 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56331a5546e0 0x56331a556ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:28:31.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.972+0000 7f3ee73d7b80 4 client.26116 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2024-02-02T15:28:31.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.972+0000 7f3ee73d7b80 1 -- 172.21.15.1:0/3129838630 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56331a3eff50 con 0x56331a5546e0 2024-02-02T15:28:31.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.972+0000 7f3ecbe00700 1 --2- 172.21.15.1:0/3129838630 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56331a5546e0 0x56331a556ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:28:31.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.973+0000 7f3ebaa00700 10 client.26116 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:28:31.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.973+0000 7f3ecbe00700 1 --2- 172.21.15.1:0/3129838630 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56331a5546e0 0x56331a556ad0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2097d7251f29e69 server_cookie=af459c63492921f in_seq=0 out_seq=0 2024-02-02T15:28:31.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.973+0000 7f3ebaa00700 1 -- 172.21.15.1:0/3129838630 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x56331a3eff50 con 0x56331a5546e0 2024-02-02T15:28:31.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.973+0000 7f3ebaa00700 10 client.26116 handle_command_reply: tid=0 2024-02-02T15:28:31.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.973+0000 7f3ee73d7b80 1 client.26116 shutdown 2024-02-02T15:28:31.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.973+0000 7f3ee73d7b80 1 -- 172.21.15.1:0/3129838630 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56331a4df010 msgr2=0x56331a547e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:28:31.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.973+0000 7f3ee73d7b80 1 --2- 172.21.15.1:0/3129838630 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56331a4df010 0x56331a547e30 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc09a120 tx=0x7f3ebc098740 comp rx=0 tx=0).stop 2024-02-02T15:28:31.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.973+0000 7f3ee73d7b80 1 -- 172.21.15.1:0/3129838630 shutdown_connections 2024-02-02T15:28:31.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.973+0000 7f3ee73d7b80 1 --2- 172.21.15.1:0/3129838630 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56331a5546e0 0x56331a556ad0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:31.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.973+0000 7f3ee73d7b80 1 --2- 172.21.15.1:0/3129838630 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56331a4f2ff0 0x56331a552cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:31.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.973+0000 7f3ee73d7b80 1 --2- 172.21.15.1:0/3129838630 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56331a4df010 0x56331a547e30 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:31.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.973+0000 7f3ee73d7b80 1 --2- 172.21.15.1:0/3129838630 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56331a4df930 0x56331a548370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:31.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.973+0000 7f3ebaa00700 0 client.26116 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:28:31.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.973+0000 7f3ee73d7b80 1 -- 172.21.15.1:0/3129838630 >> 172.21.15.1:0/3129838630 conn(0x56331a43edb0 msgr2=0x56331a4f18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:28:31.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.974+0000 7f3ee73d7b80 1 -- 172.21.15.1:0/3129838630 shutdown_connections 2024-02-02T15:28:31.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.974+0000 7f3ee73d7b80 1 -- 172.21.15.1:0/3129838630 wait complete. 2024-02-02T15:28:31.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.974+0000 7f3ee73d7b80 20 client.26116 trim_cache size 0 max 16384 2024-02-02T15:28:31.979 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:28:31.979 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-02T15:28:31.979 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2024-02-02T15:28:31.979 INFO:teuthology.orchestra.run.smithi001.stdout: "9490b50f-8ec9-4055-998a-01b17cbfcd58": { 2024-02-02T15:28:31.979 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2024-02-02T15:28:31.979 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "9490b50f-8ec9-4055-998a-01b17cbfcd58", 2024-02-02T15:28:31.979 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,force" 2024-02-02T15:28:31.979 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:28:31.979 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:28:31.979 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:28:31.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.974+0000 7f3ed3800700 1 -- 172.21.15.1:0/2113901234 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f3eac0a0830 msgr2=0x7f3eac0a2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:28:31.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.974+0000 7f3ed3800700 1 --2- 172.21.15.1:0/2113901234 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f3eac0a0830 0x7f3eac0a2cf0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3ec4009ec0 tx=0x7f3ec4009920 comp rx=0 tx=0).stop 2024-02-02T15:28:31.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.974+0000 7f3ed3800700 1 -- 172.21.15.1:0/2113901234 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ecc068010 msgr2=0x7f3ecc0fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:28:31.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.974+0000 7f3ed3800700 1 --2- 172.21.15.1:0/2113901234 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ecc068010 0x7f3ecc0fff40 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc009810 tx=0x7f3ebc00fa00 comp rx=0 tx=0).stop 2024-02-02T15:28:31.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.974+0000 7f3ed3800700 1 -- 172.21.15.1:0/2113901234 shutdown_connections 2024-02-02T15:28:31.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.974+0000 7f3ed3800700 1 --2- 172.21.15.1:0/2113901234 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f3eac0a0830 0x7f3eac0a2cf0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:31.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.974+0000 7f3ed3800700 1 --2- 172.21.15.1:0/2113901234 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3ecc106ed0 0x7f3ecc1a5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:31.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.974+0000 7f3ed3800700 1 --2- 172.21.15.1:0/2113901234 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3ecc068010 0x7f3ecc0fff40 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:31.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.974+0000 7f3ed3800700 1 --2- 172.21.15.1:0/2113901234 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3ecc0689b0 0x7f3ecc100480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:28:31.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.974+0000 7f3ed3800700 1 -- 172.21.15.1:0/2113901234 >> 172.21.15.1:0/2113901234 conn(0x7f3ecc0757d0 msgr2=0x7f3ecc0feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:28:31.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.974+0000 7f3ed3800700 1 -- 172.21.15.1:0/2113901234 shutdown_connections 2024-02-02T15:28:31.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:28:31.974+0000 7f3ed3800700 1 -- 172.21.15.1:0/2113901234 wait complete. 2024-02-02T15:28:31.992 INFO:tasks.cephfs.filesystem:scrub status for tag:9490b50f-8ec9-4055-998a-01b17cbfcd58 - {'path': '/', 'tag': '9490b50f-8ec9-4055-998a-01b17cbfcd58', 'options': 'recursive,force'} 2024-02-02T15:28:32.217 INFO:tasks.workunit.client.0.smithi001.stdout: 1 14862 16.26 MB/sec warmup 6 sec latency 11.325 ms 2024-02-02T15:28:33.217 INFO:tasks.workunit.client.0.smithi001.stdout: 1 17829 16.44 MB/sec warmup 7 sec latency 8.233 ms 2024-02-02T15:28:33.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:28:33 smithi142 ceph-mon[45728]: scrub queued for path: / 2024-02-02T15:28:33.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:28:33 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:28:33.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:28:33 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:28:33.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:28:33 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:28:33.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:28:33 smithi142 ceph-mon[45728]: pgmap v49: 129 pgs: 129 active+clean; 7.5 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 637 KiB/s wr, 88 op/s 2024-02-02T15:28:33.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:28:33 smithi142 ceph-mon[45728]: mgrmap e27: x(active, since 92s), standbys: y, z 2024-02-02T15:28:33.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:28:33 smithi001 ceph-mon[35470]: scrub queued for path: / 2024-02-02T15:28:33.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:28:33 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:28:33.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:28:33 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:28:33.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:28:33 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:28:33.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:28:33 smithi001 ceph-mon[35470]: pgmap v49: 129 pgs: 129 active+clean; 7.5 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 637 KiB/s wr, 88 op/s 2024-02-02T15:28:33.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:28:33 smithi001 ceph-mon[35470]: mgrmap e27: x(active, since 92s), standbys: y, z 2024-02-02T15:28:33.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:28:33 smithi071 ceph-mon[44707]: scrub queued for path: / 2024-02-02T15:28:33.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:28:33 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:28:33.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:28:33 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:28:33.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:28:33 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:28:33.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:28:33 smithi071 ceph-mon[44707]: pgmap v49: 129 pgs: 129 active+clean; 7.5 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 637 KiB/s wr, 88 op/s 2024-02-02T15:28:33.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:28:33 smithi071 ceph-mon[44707]: mgrmap e27: x(active, since 92s), standbys: y, z 2024-02-02T15:28:34.217 INFO:tasks.workunit.client.0.smithi001.stdout: 1 19747 15.26 MB/sec warmup 8 sec latency 14.523 ms 2024-02-02T15:28:35.217 INFO:tasks.workunit.client.0.smithi001.stdout: 1 22093 15.12 MB/sec warmup 9 sec latency 7.970 ms 2024-02-02T15:28:35.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:28:35 smithi142 ceph-mon[45728]: pgmap v50: 129 pgs: 129 active+clean; 7.5 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 637 KiB/s wr, 84 op/s 2024-02-02T15:28:35.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:28:35 smithi001 ceph-mon[35470]: pgmap v50: 129 pgs: 129 active+clean; 7.5 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 637 KiB/s wr, 84 op/s 2024-02-02T15:28:35.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:28:35 smithi071 ceph-mon[44707]: pgmap v50: 129 pgs: 129 active+clean; 7.5 MiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 637 KiB/s wr, 84 op/s 2024-02-02T15:28:36.217 INFO:tasks.workunit.client.0.smithi001.stdout: 1 25100 15.45 MB/sec warmup 10 sec latency 7.171 ms 2024-02-02T15:28:37.217 INFO:tasks.workunit.client.0.smithi001.stdout: 1 27299 14.73 MB/sec warmup 11 sec latency 22.991 ms 2024-02-02T15:28:37.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:28:37 smithi142 ceph-mon[45728]: pgmap v51: 129 pgs: 129 active+clean; 13 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 1.2 MiB/s wr, 160 op/s 2024-02-02T15:28:37.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:28:37 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:28:37.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:28:37 smithi001 ceph-mon[35470]: pgmap v51: 129 pgs: 129 active+clean; 13 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 1.2 MiB/s wr, 160 op/s 2024-02-02T15:28:37.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:28:37 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:28:37.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:28:37 smithi071 ceph-mon[44707]: pgmap v51: 129 pgs: 129 active+clean; 13 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 596 KiB/s rd, 1.2 MiB/s wr, 160 op/s 2024-02-02T15:28:37.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:28:37 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:28:38.217 INFO:tasks.workunit.client.0.smithi001.stdout: 1 29340 14.57 MB/sec warmup 12 sec latency 6.960 ms 2024-02-02T15:28:39.217 INFO:tasks.workunit.client.0.smithi001.stdout: 1 32338 14.85 MB/sec warmup 13 sec latency 10.139 ms 2024-02-02T15:28:39.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:28:39 smithi142 ceph-mon[45728]: pgmap v52: 129 pgs: 129 active+clean; 13 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 1.2 MiB/s wr, 156 op/s 2024-02-02T15:28:39.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:28:39 smithi001 ceph-mon[35470]: pgmap v52: 129 pgs: 129 active+clean; 13 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 1.2 MiB/s wr, 156 op/s 2024-02-02T15:28:39.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:28:39 smithi071 ceph-mon[44707]: pgmap v52: 129 pgs: 129 active+clean; 13 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 1.2 MiB/s wr, 156 op/s 2024-02-02T15:28:40.217 INFO:tasks.workunit.client.0.smithi001.stdout: 1 34733 14.58 MB/sec warmup 14 sec latency 18.158 ms 2024-02-02T15:28:41.217 INFO:tasks.workunit.client.0.smithi001.stdout: 1 36788 14.29 MB/sec warmup 15 sec latency 6.541 ms 2024-02-02T15:28:41.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:28:41 smithi142 ceph-mon[45728]: pgmap v53: 129 pgs: 129 active+clean; 13 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 1.2 MiB/s wr, 156 op/s 2024-02-02T15:28:41.689 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:28:41 smithi001 ceph-mon[35470]: pgmap v53: 129 pgs: 129 active+clean; 13 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 1.2 MiB/s wr, 156 op/s 2024-02-02T15:28:41.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:28:41 smithi071 ceph-mon[44707]: pgmap v53: 129 pgs: 129 active+clean; 13 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 1.2 MiB/s wr, 156 op/s 2024-02-02T15:28:42.217 INFO:tasks.workunit.client.0.smithi001.stdout: 1 39447 14.46 MB/sec warmup 16 sec latency 14.086 ms 2024-02-02T15:28:43.217 INFO:tasks.workunit.client.0.smithi001.stdout: 1 41956 14.31 MB/sec warmup 17 sec latency 12.638 ms 2024-02-02T15:28:43.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:28:43 smithi142 ceph-mon[45728]: pgmap v54: 129 pgs: 129 active+clean; 19 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 932 KiB/s rd, 1.8 MiB/s wr, 235 op/s 2024-02-02T15:28:43.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:28:43 smithi001 ceph-mon[35470]: pgmap v54: 129 pgs: 129 active+clean; 19 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 932 KiB/s rd, 1.8 MiB/s wr, 235 op/s 2024-02-02T15:28:43.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:28:43 smithi071 ceph-mon[44707]: pgmap v54: 129 pgs: 129 active+clean; 19 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 932 KiB/s rd, 1.8 MiB/s wr, 235 op/s 2024-02-02T15:28:44.217 INFO:tasks.workunit.client.0.smithi001.stdout: 1 43781 14.03 MB/sec warmup 18 sec latency 7.744 ms 2024-02-02T15:28:45.217 INFO:tasks.workunit.client.0.smithi001.stdout: 1 46527 14.19 MB/sec warmup 19 sec latency 25.673 ms 2024-02-02T15:28:45.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:28:45 smithi142 ceph-mon[45728]: pgmap v55: 129 pgs: 129 active+clean; 19 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 1.2 MiB/s wr, 155 op/s 2024-02-02T15:28:45.689 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:28:45 smithi001 ceph-mon[35470]: pgmap v55: 129 pgs: 129 active+clean; 19 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 1.2 MiB/s wr, 155 op/s 2024-02-02T15:28:45.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:28:45 smithi071 ceph-mon[44707]: pgmap v55: 129 pgs: 129 active+clean; 19 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 1.2 MiB/s wr, 155 op/s 2024-02-02T15:28:46.217 INFO:tasks.workunit.client.0.smithi001.stdout: 1 48908 14.03 MB/sec warmup 20 sec latency 26.321 ms 2024-02-02T15:28:47.218 INFO:tasks.workunit.client.0.smithi001.stdout: 1 50796 13.84 MB/sec warmup 21 sec latency 8.281 ms 2024-02-02T15:28:47.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:28:47 smithi142 ceph-mon[45728]: pgmap v56: 129 pgs: 129 active+clean; 24 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 1.8 MiB/s wr, 230 op/s 2024-02-02T15:28:47.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:28:47 smithi001 ceph-mon[35470]: pgmap v56: 129 pgs: 129 active+clean; 24 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 1.8 MiB/s wr, 230 op/s 2024-02-02T15:28:47.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:28:47 smithi071 ceph-mon[44707]: pgmap v56: 129 pgs: 129 active+clean; 24 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 906 KiB/s rd, 1.8 MiB/s wr, 230 op/s 2024-02-02T15:28:48.219 INFO:tasks.workunit.client.0.smithi001.stdout: 1 53393 13.93 MB/sec warmup 22 sec latency 90.238 ms 2024-02-02T15:28:48.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:28:48 smithi142 ceph-mon[45728]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:28:48.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:28:48 smithi001 ceph-mon[35470]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:28:48.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:28:48 smithi071 ceph-mon[44707]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:28:49.218 INFO:tasks.workunit.client.0.smithi001.stdout: 1 53393 13.33 MB/sec warmup 23 sec latency 1090.363 ms 2024-02-02T15:28:49.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:28:49 smithi142 ceph-mon[45728]: pgmap v57: 129 pgs: 129 active+clean; 24 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 578 KiB/s rd, 1.2 MiB/s wr, 155 op/s 2024-02-02T15:28:49.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:28:49 smithi001 ceph-mon[35470]: pgmap v57: 129 pgs: 129 active+clean; 24 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 578 KiB/s rd, 1.2 MiB/s wr, 155 op/s 2024-02-02T15:28:49.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:28:49 smithi071 ceph-mon[44707]: pgmap v57: 129 pgs: 129 active+clean; 24 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 578 KiB/s rd, 1.2 MiB/s wr, 155 op/s 2024-02-02T15:28:50.218 INFO:tasks.workunit.client.0.smithi001.stdout: 1 53479 12.81 MB/sec warmup 24 sec latency 1407.061 ms 2024-02-02T15:28:51.218 INFO:tasks.workunit.client.0.smithi001.stdout: 1 53479 12.30 MB/sec warmup 25 sec latency 1591.033 ms 2024-02-02T15:28:51.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:28:51 smithi142 ceph-mon[45728]: pgmap v58: 129 pgs: 129 active+clean; 24 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 578 KiB/s rd, 1.2 MiB/s wr, 155 op/s 2024-02-02T15:28:51.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:28:51 smithi001 ceph-mon[35470]: pgmap v58: 129 pgs: 129 active+clean; 24 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 578 KiB/s rd, 1.2 MiB/s wr, 155 op/s 2024-02-02T15:28:51.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:28:51 smithi071 ceph-mon[44707]: pgmap v58: 129 pgs: 129 active+clean; 24 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 578 KiB/s rd, 1.2 MiB/s wr, 155 op/s 2024-02-02T15:28:52.218 INFO:tasks.workunit.client.0.smithi001.stdout: 1 53479 11.83 MB/sec warmup 26 sec latency 2591.181 ms 2024-02-02T15:28:53.218 INFO:tasks.workunit.client.0.smithi001.stdout: 1 53479 11.39 MB/sec warmup 27 sec latency 3591.325 ms 2024-02-02T15:28:53.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:28:53 smithi001 ceph-mon[35470]: pgmap v59: 129 pgs: 129 active+clean; 26 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 768 KiB/s rd, 1.4 MiB/s wr, 190 op/s 2024-02-02T15:28:53.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:28:53 smithi001 ceph-mon[35470]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:28:53.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:28:53 smithi071 ceph-mon[44707]: pgmap v59: 129 pgs: 129 active+clean; 26 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 768 KiB/s rd, 1.4 MiB/s wr, 190 op/s 2024-02-02T15:28:53.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:28:53 smithi071 ceph-mon[44707]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:28:53.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:28:53 smithi142 ceph-mon[45728]: pgmap v59: 129 pgs: 129 active+clean; 26 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 768 KiB/s rd, 1.4 MiB/s wr, 190 op/s 2024-02-02T15:28:53.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:28:53 smithi142 ceph-mon[45728]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:28:54.218 INFO:tasks.workunit.client.0.smithi001.stdout: 1 53479 10.98 MB/sec warmup 28 sec latency 4591.460 ms 2024-02-02T15:28:55.218 INFO:tasks.workunit.client.0.smithi001.stdout: 1 54111 10.63 MB/sec warmup 29 sec latency 4911.688 ms 2024-02-02T15:28:55.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:28:55 smithi001 ceph-mon[35470]: pgmap v60: 129 pgs: 129 active+clean; 26 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 813 KiB/s wr, 110 op/s 2024-02-02T15:28:55.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:28:55 smithi071 ceph-mon[44707]: pgmap v60: 129 pgs: 129 active+clean; 26 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 813 KiB/s wr, 110 op/s 2024-02-02T15:28:55.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:28:55 smithi142 ceph-mon[45728]: pgmap v60: 129 pgs: 129 active+clean; 26 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 813 KiB/s wr, 110 op/s 2024-02-02T15:28:56.219 INFO:tasks.workunit.client.0.smithi001.stdout: 1 56746 10.84 MB/sec warmup 30 sec latency 10.496 ms 2024-02-02T15:28:57.219 INFO:tasks.workunit.client.0.smithi001.stdout: 1 58031 10.61 MB/sec warmup 31 sec latency 9.532 ms 2024-02-02T15:28:57.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:28:57 smithi001 ceph-mon[35470]: pgmap v61: 129 pgs: 129 active+clean; 28 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 994 KiB/s wr, 147 op/s 2024-02-02T15:28:57.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:28:57 smithi071 ceph-mon[44707]: pgmap v61: 129 pgs: 129 active+clean; 28 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 994 KiB/s wr, 147 op/s 2024-02-02T15:28:57.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:28:57 smithi142 ceph-mon[45728]: pgmap v61: 129 pgs: 129 active+clean; 28 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 994 KiB/s wr, 147 op/s 2024-02-02T15:28:58.219 INFO:tasks.workunit.client.0.smithi001.stdout: 1 60402 10.77 MB/sec warmup 32 sec latency 126.923 ms 2024-02-02T15:28:59.219 INFO:tasks.workunit.client.0.smithi001.stdout: 1 61587 10.55 MB/sec warmup 33 sec latency 16.618 ms 2024-02-02T15:28:59.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:28:59 smithi001 ceph-mon[35470]: pgmap v62: 129 pgs: 129 active+clean; 28 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 400 KiB/s wr, 72 op/s 2024-02-02T15:28:59.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:28:59 smithi071 ceph-mon[44707]: pgmap v62: 129 pgs: 129 active+clean; 28 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 400 KiB/s wr, 72 op/s 2024-02-02T15:28:59.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:28:59 smithi142 ceph-mon[45728]: pgmap v62: 129 pgs: 129 active+clean; 28 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 400 KiB/s wr, 72 op/s 2024-02-02T15:29:00.219 INFO:tasks.workunit.client.0.smithi001.stdout: 1 64111 10.73 MB/sec warmup 34 sec latency 14.429 ms 2024-02-02T15:29:00.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:29:00 smithi071 ceph-mon[44707]: pgmap v63: 129 pgs: 129 active+clean; 28 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 400 KiB/s wr, 72 op/s 2024-02-02T15:29:00.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:29:00 smithi142 ceph-mon[45728]: pgmap v63: 129 pgs: 129 active+clean; 28 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 400 KiB/s wr, 72 op/s 2024-02-02T15:29:00.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:29:00 smithi001 ceph-mon[35470]: pgmap v63: 129 pgs: 129 active+clean; 28 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 400 KiB/s wr, 72 op/s 2024-02-02T15:29:01.219 INFO:tasks.workunit.client.0.smithi001.stdout: 1 64886 10.47 MB/sec warmup 35 sec latency 52.450 ms 2024-02-02T15:29:01.993 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-02T15:29:02.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.210+0000 7f2587800700 1 -- 172.21.15.1:0/1458420454 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2580111d90 msgr2=0x7f2580114180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:02.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.210+0000 7f2587800700 1 --2- 172.21.15.1:0/1458420454 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2580111d90 0x7f2580114180 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f257800b0a0 tx=0x7f257800b3b0 comp rx=0 tx=0).stop 2024-02-02T15:29:02.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.210+0000 7f2587800700 1 -- 172.21.15.1:0/1458420454 shutdown_connections 2024-02-02T15:29:02.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.210+0000 7f2587800700 1 --2- 172.21.15.1:0/1458420454 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2580111d90 0x7f2580114180 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:02.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.210+0000 7f2587800700 1 --2- 172.21.15.1:0/1458420454 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f25800730e0 0x7f25800734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:02.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.210+0000 7f2587800700 1 --2- 172.21.15.1:0/1458420454 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2580073a00 0x7f2580111850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:02.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.210+0000 7f2587800700 1 -- 172.21.15.1:0/1458420454 >> 172.21.15.1:0/1458420454 conn(0x7f25800fc810 msgr2=0x7f25800fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:02.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.210+0000 7f2587800700 1 -- 172.21.15.1:0/1458420454 shutdown_connections 2024-02-02T15:29:02.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.210+0000 7f2587800700 1 -- 172.21.15.1:0/1458420454 wait complete. 2024-02-02T15:29:02.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.211+0000 7f2587800700 1 Processor -- start 2024-02-02T15:29:02.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.212+0000 7f2587800700 1 -- start start 2024-02-02T15:29:02.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.212+0000 7f2587800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25800730e0 0x7f258019d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:02.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.212+0000 7f2587800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2580073a00 0x7f258019db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:02.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.212+0000 7f2587800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2580111d90 0x7f25801a1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:02.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.212+0000 7f2587800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25801a25a0 con 0x7f2580073a00 2024-02-02T15:29:02.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.212+0000 7f2587800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25801a2710 con 0x7f25800730e0 2024-02-02T15:29:02.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.212+0000 7f2587800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25801a28d0 con 0x7f2580111d90 2024-02-02T15:29:02.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.212+0000 7f2584e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25800730e0 0x7f258019d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:02.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.212+0000 7f2584e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25800730e0 0x7f258019d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:60790/0 (socket says 172.21.15.1:60790) 2024-02-02T15:29:02.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.212+0000 7f2584e00700 1 -- 172.21.15.1:0/2711133743 learned_addr learned my addr 172.21.15.1:0/2711133743 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:29:02.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.212+0000 7f257fe00700 1 --2- 172.21.15.1:0/2711133743 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2580073a00 0x7f258019db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:02.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.213+0000 7f2585800700 1 --2- 172.21.15.1:0/2711133743 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2580111d90 0x7f25801a1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:02.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.213+0000 7f2584e00700 1 -- 172.21.15.1:0/2711133743 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2580111d90 msgr2=0x7f25801a1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:02.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.213+0000 7f2584e00700 1 --2- 172.21.15.1:0/2711133743 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2580111d90 0x7f25801a1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:02.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.213+0000 7f2584e00700 1 -- 172.21.15.1:0/2711133743 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2580073a00 msgr2=0x7f258019db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:02.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.213+0000 7f2584e00700 1 --2- 172.21.15.1:0/2711133743 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2580073a00 0x7f258019db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:02.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.213+0000 7f2584e00700 1 -- 172.21.15.1:0/2711133743 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2578009d00 con 0x7f25800730e0 2024-02-02T15:29:02.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.213+0000 7f2585800700 1 --2- 172.21.15.1:0/2711133743 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2580111d90 0x7f25801a1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:29:02.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.213+0000 7f2584e00700 1 --2- 172.21.15.1:0/2711133743 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25800730e0 0x7f258019d5c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f257000e9a0 tx=0x7f257000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:02.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.213+0000 7f257d600700 1 -- 172.21.15.1:0/2711133743 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2570009790 con 0x7f25800730e0 2024-02-02T15:29:02.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.213+0000 7f257d600700 1 -- 172.21.15.1:0/2711133743 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f257000cbd0 con 0x7f25800730e0 2024-02-02T15:29:02.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.213+0000 7f2587800700 1 -- 172.21.15.1:0/2711133743 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25801a2bb0 con 0x7f25800730e0 2024-02-02T15:29:02.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.213+0000 7f257d600700 1 -- 172.21.15.1:0/2711133743 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2570018680 con 0x7f25800730e0 2024-02-02T15:29:02.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.214+0000 7f2587800700 1 -- 172.21.15.1:0/2711133743 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25801aa3f0 con 0x7f25800730e0 2024-02-02T15:29:02.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.217+0000 7f257d600700 1 -- 172.21.15.1:0/2711133743 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 27) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f257000cd40 con 0x7f25800730e0 2024-02-02T15:29:02.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.218+0000 7f257d600700 1 --2- 172.21.15.1:0/2711133743 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f25600a0690 0x7f25600a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:02.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.218+0000 7f257d600700 1 -- 172.21.15.1:0/2711133743 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f2570014070 con 0x7f25800730e0 2024-02-02T15:29:02.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.218+0000 7f257fe00700 1 --2- 172.21.15.1:0/2711133743 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f25600a0690 0x7f25600a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:02.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.219+0000 7f259b358b80 1 Processor -- start 2024-02-02T15:29:02.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.219+0000 7f259b358b80 1 -- start start 2024-02-02T15:29:02.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.219+0000 7f257fe00700 1 --2- 172.21.15.1:0/2711133743 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f25600a0690 0x7f25600a2b50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f258019ebd0 tx=0x7f2568009260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:02.219 INFO:tasks.workunit.client.0.smithi001.stdout: 1 66349 10.36 MB/sec warmup 36 sec latency 53.751 ms 2024-02-02T15:29:02.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.219+0000 7f259b358b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba17ff2010 0x55ba17ff23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:02.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.219+0000 7f259b358b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba17ff2930 0x55ba17ffd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:02.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.219+0000 7f259b358b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba17ffd7e0 0x55ba17fffbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:02.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.219+0000 7f259b358b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba17f664c0 con 0x55ba17ff2010 2024-02-02T15:29:02.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.219+0000 7f259b358b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba17f4aaa0 con 0x55ba17ffd7e0 2024-02-02T15:29:02.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.219+0000 7f259b358b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba17f48460 con 0x55ba17ff2930 2024-02-02T15:29:02.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.219+0000 7f257fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba17ff2930 0x55ba17ffd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:02.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.220+0000 7f257fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba17ff2930 0x55ba17ffd2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:51292/0 (socket says 172.21.15.1:51292) 2024-02-02T15:29:02.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.220+0000 7f257fe00700 1 -- 172.21.15.1:0/3217142536 learned_addr learned my addr 172.21.15.1:0/3217142536 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:29:02.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.220+0000 7f2585800700 1 --2- 172.21.15.1:0/3217142536 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba17ffd7e0 0x55ba17fffbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:02.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.220+0000 7f2584e00700 1 --2- 172.21.15.1:0/3217142536 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba17ff2010 0x55ba17ff23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:02.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.220+0000 7f2585800700 1 -- 172.21.15.1:0/3217142536 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba17ff2930 msgr2=0x55ba17ffd2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:02.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.220+0000 7f2585800700 1 --2- 172.21.15.1:0/3217142536 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba17ff2930 0x55ba17ffd2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:02.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.220+0000 7f2585800700 1 -- 172.21.15.1:0/3217142536 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba17ff2010 msgr2=0x55ba17ff23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:02.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.220+0000 7f2585800700 1 --2- 172.21.15.1:0/3217142536 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba17ff2010 0x55ba17ff23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:02.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.220+0000 7f2585800700 1 -- 172.21.15.1:0/3217142536 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba17dc53b0 con 0x55ba17ffd7e0 2024-02-02T15:29:02.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.220+0000 7f257fe00700 1 --2- 172.21.15.1:0/3217142536 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba17ff2930 0x55ba17ffd2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:29:02.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.220+0000 7f2585800700 1 --2- 172.21.15.1:0/3217142536 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba17ffd7e0 0x55ba17fffbd0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f258006a330 tx=0x7f2578009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c6c33f7f5e042ad server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:02.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.221+0000 7f2576000700 1 -- 172.21.15.1:0/3217142536 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257800e040 con 0x55ba17ffd7e0 2024-02-02T15:29:02.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.221+0000 7f2576000700 1 -- 172.21.15.1:0/3217142536 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2578009460 con 0x55ba17ffd7e0 2024-02-02T15:29:02.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.221+0000 7f2576000700 1 -- 172.21.15.1:0/3217142536 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2578007950 con 0x55ba17ffd7e0 2024-02-02T15:29:02.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.221+0000 7f259b358b80 1 -- 172.21.15.1:0/3217142536 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba17ffd7e0 msgr2=0x55ba17fffbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:02.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.221+0000 7f259b358b80 1 --2- 172.21.15.1:0/3217142536 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba17ffd7e0 0x55ba17fffbd0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f258006a330 tx=0x7f2578009c20 comp rx=0 tx=0).stop 2024-02-02T15:29:02.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.221+0000 7f259b358b80 1 -- 172.21.15.1:0/3217142536 shutdown_connections 2024-02-02T15:29:02.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.221+0000 7f259b358b80 1 --2- 172.21.15.1:0/3217142536 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba17ffd7e0 0x55ba17fffbd0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:02.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.222+0000 7f259b358b80 1 --2- 172.21.15.1:0/3217142536 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba17ff2010 0x55ba17ff23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:02.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.222+0000 7f259b358b80 1 --2- 172.21.15.1:0/3217142536 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba17ff2930 0x55ba17ffd2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:02.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.222+0000 7f259b358b80 1 -- 172.21.15.1:0/3217142536 >> 172.21.15.1:0/3217142536 conn(0x55ba17f51db0 msgr2=0x55ba17fee3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:02.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.222+0000 7f259b358b80 1 -- 172.21.15.1:0/3217142536 shutdown_connections 2024-02-02T15:29:02.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.222+0000 7f259b358b80 1 -- 172.21.15.1:0/3217142536 wait complete. 2024-02-02T15:29:02.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.222+0000 7f259b358b80 1 -- start start 2024-02-02T15:29:02.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.223+0000 7f259b358b80 1 Processor -- start 2024-02-02T15:29:02.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.223+0000 7f259b358b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba17ff2010 0x55ba17ffd190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:02.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.223+0000 7f259b358b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba17ff2930 0x55ba17ffa3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:02.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.223+0000 7f259b358b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba17ffd7e0 0x55ba17ffa960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:02.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.223+0000 7f259b358b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba17f43d50 con 0x55ba17ff2930 2024-02-02T15:29:02.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.223+0000 7f259b358b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba17edfd80 con 0x55ba17ffd7e0 2024-02-02T15:29:02.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.223+0000 7f259b358b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba17edfa20 con 0x55ba17ff2010 2024-02-02T15:29:02.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.223+0000 7f259b358b80 10 client.0 _ll_register_callbacks cb 0x55ba17ee1290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:29:02.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.224+0000 7f2584e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba17ff2010 0x55ba17ffd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:02.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.224+0000 7f2584e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba17ff2010 0x55ba17ffd190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:51302/0 (socket says 172.21.15.1:51302) 2024-02-02T15:29:02.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.224+0000 7f2584e00700 1 -- 172.21.15.1:0/589251482 learned_addr learned my addr 172.21.15.1:0/589251482 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:29:02.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.224+0000 7f257fe00700 1 --2- 172.21.15.1:0/589251482 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba17ff2930 0x55ba17ffa3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:02.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.224+0000 7f2585800700 1 --2- 172.21.15.1:0/589251482 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba17ffd7e0 0x55ba17ffa960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:02.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.224+0000 7f2584e00700 1 -- 172.21.15.1:0/589251482 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba17ffd7e0 msgr2=0x55ba17ffa960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:02.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.224+0000 7f2584e00700 1 --2- 172.21.15.1:0/589251482 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba17ffd7e0 0x55ba17ffa960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:02.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.224+0000 7f2584e00700 1 -- 172.21.15.1:0/589251482 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba17ff2930 msgr2=0x55ba17ffa3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:02.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.224+0000 7f2584e00700 1 --2- 172.21.15.1:0/589251482 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba17ff2930 0x55ba17ffa3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:02.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.224+0000 7f2584e00700 1 -- 172.21.15.1:0/589251482 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2568017070 con 0x55ba17ff2010 2024-02-02T15:29:02.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.224+0000 7f257fe00700 1 --2- 172.21.15.1:0/589251482 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba17ff2930 0x55ba17ffa3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:29:02.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.224+0000 7f2584e00700 1 --2- 172.21.15.1:0/589251482 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba17ff2010 0x55ba17ffd190 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f257000e970 tx=0x7f2570099c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5516edeb5461a674 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:02.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.224+0000 7f256ea00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:29:02.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.224+0000 7f256ea00700 1 -- 172.21.15.1:0/589251482 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25700999d0 con 0x55ba17ff2010 2024-02-02T15:29:02.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.224+0000 7f256ea00700 1 -- 172.21.15.1:0/589251482 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f257009d170 con 0x55ba17ff2010 2024-02-02T15:29:02.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.224+0000 7f259b358b80 1 -- 172.21.15.1:0/589251482 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ba17fe60f0 con 0x55ba17ff2010 2024-02-02T15:29:02.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.225+0000 7f256ea00700 1 -- 172.21.15.1:0/589251482 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f257009d3a0 con 0x55ba17ff2010 2024-02-02T15:29:02.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.225+0000 7f259b358b80 10 client.26125 fetch_fsmap learned FSMap version 29 2024-02-02T15:29:02.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.225+0000 7f259b358b80 1 -- 172.21.15.1:0/589251482 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f2580114b30 con 0x55ba17ff2010 2024-02-02T15:29:02.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.225+0000 7f256ea00700 1 -- 172.21.15.1:0/589251482 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2570063020 con 0x55ba17ff2010 2024-02-02T15:29:02.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.225+0000 7f256ea00700 1 -- 172.21.15.1:0/589251482 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 29) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f25700a1150 con 0x55ba17ff2010 2024-02-02T15:29:02.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.225+0000 7f259b358b80 10 client.26125 fetch_fsmap finished waiting for FSMap version 29 2024-02-02T15:29:02.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.225+0000 7f259b358b80 10 client.26125 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:29:02.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.225+0000 7f259b358b80 20 client.26125 populate_metadata read hostname 'smithi001' 2024-02-02T15:29:02.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.225+0000 7f259b358b80 1 --2- 172.21.15.1:0/589251482 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55ba1800d1c0 0x55ba1800f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:02.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.225+0000 7f259b358b80 4 client.26125 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:29:02.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.225+0000 7f259b358b80 1 -- 172.21.15.1:0/589251482 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ba17f3cdd0 con 0x55ba1800d1c0 2024-02-02T15:29:02.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.226+0000 7f257fe00700 1 --2- 172.21.15.1:0/589251482 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55ba1800d1c0 0x55ba1800f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:02.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.226+0000 7f257fe00700 1 --2- 172.21.15.1:0/589251482 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55ba1800d1c0 0x55ba1800f5e0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b12a05d23722e79e server_cookie=9a9df2b604ddb4c0 in_seq=0 out_seq=0 2024-02-02T15:29:02.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.226+0000 7f256ea00700 10 client.26125 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:29:02.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.227+0000 7f256ea00700 1 -- 172.21.15.1:0/589251482 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55ba17f3cdd0 con 0x55ba1800d1c0 2024-02-02T15:29:02.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.227+0000 7f256ea00700 10 client.26125 handle_command_reply: tid=0 2024-02-02T15:29:02.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.227+0000 7f259b358b80 1 client.26125 shutdown 2024-02-02T15:29:02.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.227+0000 7f259b358b80 1 -- 172.21.15.1:0/589251482 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba17ff2010 msgr2=0x55ba17ffd190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:02.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.227+0000 7f259b358b80 1 --2- 172.21.15.1:0/589251482 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba17ff2010 0x55ba17ffd190 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f257000e970 tx=0x7f2570099c70 comp rx=0 tx=0).stop 2024-02-02T15:29:02.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.227+0000 7f259b358b80 1 -- 172.21.15.1:0/589251482 shutdown_connections 2024-02-02T15:29:02.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.227+0000 7f259b358b80 1 --2- 172.21.15.1:0/589251482 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55ba1800d1c0 0x55ba1800f5e0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:02.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.227+0000 7f259b358b80 1 --2- 172.21.15.1:0/589251482 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba17ffd7e0 0x55ba17ffa960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:02.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.227+0000 7f259b358b80 1 --2- 172.21.15.1:0/589251482 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba17ff2010 0x55ba17ffd190 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:02.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.227+0000 7f259b358b80 1 --2- 172.21.15.1:0/589251482 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba17ff2930 0x55ba17ffa3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:02.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.227+0000 7f256ea00700 0 client.26125 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:29:02.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.227+0000 7f259b358b80 1 -- 172.21.15.1:0/589251482 >> 172.21.15.1:0/589251482 conn(0x55ba17f51db0 msgr2=0x55ba17fedfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:02.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.227+0000 7f259b358b80 1 -- 172.21.15.1:0/589251482 shutdown_connections 2024-02-02T15:29:02.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.227+0000 7f259b358b80 1 -- 172.21.15.1:0/589251482 wait complete. 2024-02-02T15:29:02.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.228+0000 7f259b358b80 20 client.26125 trim_cache size 0 max 16384 2024-02-02T15:29:02.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.245+0000 7f259b358b80 1 Processor -- start 2024-02-02T15:29:02.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.245+0000 7f259b358b80 1 -- start start 2024-02-02T15:29:02.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.245+0000 7f259b358b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba17ff2010 0x55ba180088a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:02.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.245+0000 7f259b358b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba17ff2930 0x55ba18008de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:02.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.245+0000 7f259b358b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba17ffd7e0 0x55ba17fedeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:02.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.245+0000 7f259b358b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba17dc76a0 con 0x55ba17ffd7e0 2024-02-02T15:29:02.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.245+0000 7f259b358b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba17fef100 con 0x55ba17ff2930 2024-02-02T15:29:02.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.245+0000 7f259b358b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba17ffffd0 con 0x55ba17ff2010 2024-02-02T15:29:02.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.246+0000 7f2584e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba17ff2010 0x55ba180088a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:02.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.246+0000 7f257fe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba17ff2930 0x55ba18008de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:02.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.246+0000 7f257fe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba17ff2930 0x55ba18008de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:60808/0 (socket says 172.21.15.1:60808) 2024-02-02T15:29:02.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.246+0000 7f257fe00700 1 -- 172.21.15.1:0/3529807826 learned_addr learned my addr 172.21.15.1:0/3529807826 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:29:02.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.246+0000 7f2584e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba17ff2010 0x55ba180088a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:51308/0 (socket says 172.21.15.1:51308) 2024-02-02T15:29:02.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.246+0000 7f2585800700 1 --2- 172.21.15.1:0/3529807826 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba17ffd7e0 0x55ba17fedeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:02.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.246+0000 7f257fe00700 1 -- 172.21.15.1:0/3529807826 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba17ff2010 msgr2=0x55ba180088a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:02.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.246+0000 7f257fe00700 1 --2- 172.21.15.1:0/3529807826 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba17ff2010 0x55ba180088a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:02.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.246+0000 7f257fe00700 1 -- 172.21.15.1:0/3529807826 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba17ffd7e0 msgr2=0x55ba17fedeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:02.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.246+0000 7f257fe00700 1 --2- 172.21.15.1:0/3529807826 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba17ffd7e0 0x55ba17fedeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:02.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.246+0000 7f257fe00700 1 -- 172.21.15.1:0/3529807826 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba17d71b60 con 0x55ba17ff2930 2024-02-02T15:29:02.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.246+0000 7f2585800700 1 --2- 172.21.15.1:0/3529807826 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba17ffd7e0 0x55ba17fedeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:29:02.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.246+0000 7f2584e00700 1 --2- 172.21.15.1:0/3529807826 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba17ff2010 0x55ba180088a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:29:02.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.246+0000 7f257fe00700 1 --2- 172.21.15.1:0/3529807826 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba17ff2930 0x55ba18008de0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2568009fd0 tx=0x7f2568014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=639b793cd9fcaf67 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:02.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.247+0000 7f2576a00700 1 -- 172.21.15.1:0/3529807826 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2568024040 con 0x55ba17ff2930 2024-02-02T15:29:02.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.247+0000 7f2576a00700 1 -- 172.21.15.1:0/3529807826 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2568026040 con 0x55ba17ff2930 2024-02-02T15:29:02.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.247+0000 7f2576a00700 1 -- 172.21.15.1:0/3529807826 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2568003e30 con 0x55ba17ff2930 2024-02-02T15:29:02.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.247+0000 7f259b358b80 1 -- 172.21.15.1:0/3529807826 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba17ff2930 msgr2=0x55ba18008de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:02.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.247+0000 7f259b358b80 1 --2- 172.21.15.1:0/3529807826 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba17ff2930 0x55ba18008de0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2568009fd0 tx=0x7f2568014000 comp rx=0 tx=0).stop 2024-02-02T15:29:02.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.248+0000 7f259b358b80 1 -- 172.21.15.1:0/3529807826 shutdown_connections 2024-02-02T15:29:02.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.248+0000 7f259b358b80 1 --2- 172.21.15.1:0/3529807826 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba17ffd7e0 0x55ba17fedeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:02.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.248+0000 7f259b358b80 1 --2- 172.21.15.1:0/3529807826 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba17ff2010 0x55ba180088a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:02.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.248+0000 7f259b358b80 1 --2- 172.21.15.1:0/3529807826 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba17ff2930 0x55ba18008de0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:02.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.248+0000 7f259b358b80 1 -- 172.21.15.1:0/3529807826 >> 172.21.15.1:0/3529807826 conn(0x55ba17f51db0 msgr2=0x55ba180094a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:02.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.248+0000 7f259b358b80 1 -- 172.21.15.1:0/3529807826 shutdown_connections 2024-02-02T15:29:02.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.248+0000 7f259b358b80 1 -- 172.21.15.1:0/3529807826 wait complete. 2024-02-02T15:29:02.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.248+0000 7f259b358b80 1 -- start start 2024-02-02T15:29:02.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.249+0000 7f259b358b80 1 Processor -- start 2024-02-02T15:29:02.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.249+0000 7f259b358b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba17ff2010 0x55ba180042d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:02.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.249+0000 7f259b358b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba17ff2930 0x55ba17ff6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:02.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.249+0000 7f259b358b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba17ffd7e0 0x55ba17ff71c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:02.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.249+0000 7f259b358b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba17ff7900 con 0x55ba17ffd7e0 2024-02-02T15:29:02.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.249+0000 7f259b358b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba1805eca0 con 0x55ba17ff2010 2024-02-02T15:29:02.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.249+0000 7f259b358b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba1805ede0 con 0x55ba17ff2930 2024-02-02T15:29:02.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.249+0000 7f259b358b80 10 client.0 _ll_register_callbacks cb 0x55ba17ee1290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:29:02.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.249+0000 7f257fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba17ff2930 0x55ba17ff6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:02.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.249+0000 7f2584e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba17ff2010 0x55ba180042d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:02.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.249+0000 7f257fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba17ff2930 0x55ba17ff6ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:51310/0 (socket says 172.21.15.1:51310) 2024-02-02T15:29:02.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.249+0000 7f257fe00700 1 -- 172.21.15.1:0/2973113418 learned_addr learned my addr 172.21.15.1:0/2973113418 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:29:02.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.250+0000 7f257fe00700 1 -- 172.21.15.1:0/2973113418 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba17ff2010 msgr2=0x55ba180042d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:02.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.250+0000 7f257fe00700 1 --2- 172.21.15.1:0/2973113418 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba17ff2010 0x55ba180042d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:02.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.250+0000 7f257fe00700 1 -- 172.21.15.1:0/2973113418 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba17ffd7e0 msgr2=0x55ba17ff71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:02.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.250+0000 7f257fe00700 1 --2- 172.21.15.1:0/2973113418 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba17ffd7e0 0x55ba17ff71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:02.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.250+0000 7f257fe00700 1 -- 172.21.15.1:0/2973113418 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2580114b30 con 0x55ba17ff2930 2024-02-02T15:29:02.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.250+0000 7f2584e00700 1 --2- 172.21.15.1:0/2973113418 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba17ff2010 0x55ba180042d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:29:02.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.250+0000 7f257fe00700 1 --2- 172.21.15.1:0/2973113418 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba17ff2930 0x55ba17ff6ad0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f256802be40 tx=0x7f2568003c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87ffb743aab4db0e server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:02.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.250+0000 7f256ea00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:29:02.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.250+0000 7f256ea00700 1 -- 172.21.15.1:0/2973113418 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2568024040 con 0x55ba17ff2930 2024-02-02T15:29:02.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.250+0000 7f256ea00700 1 -- 172.21.15.1:0/2973113418 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f2568011d10 con 0x55ba17ff2930 2024-02-02T15:29:02.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.250+0000 7f259b358b80 1 -- 172.21.15.1:0/2973113418 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2578009460 con 0x55ba17ff2930 2024-02-02T15:29:02.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.251+0000 7f256ea00700 1 -- 172.21.15.1:0/2973113418 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2568004560 con 0x55ba17ff2930 2024-02-02T15:29:02.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.251+0000 7f259b358b80 10 client.26131 fetch_fsmap learned FSMap version 29 2024-02-02T15:29:02.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.251+0000 7f259b358b80 1 -- 172.21.15.1:0/2973113418 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f2568004040 con 0x55ba17ff2930 2024-02-02T15:29:02.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.251+0000 7f256ea00700 1 -- 172.21.15.1:0/2973113418 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2578009460 con 0x55ba17ff2930 2024-02-02T15:29:02.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.251+0000 7f256ea00700 1 -- 172.21.15.1:0/2973113418 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 29) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f2568005480 con 0x55ba17ff2930 2024-02-02T15:29:02.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.251+0000 7f259b358b80 10 client.26131 fetch_fsmap finished waiting for FSMap version 29 2024-02-02T15:29:02.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.251+0000 7f259b358b80 10 client.26131 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:29:02.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.251+0000 7f259b358b80 20 client.26131 populate_metadata read hostname 'smithi001' 2024-02-02T15:29:02.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.251+0000 7f259b358b80 1 --2- 172.21.15.1:0/2973113418 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55ba18060540 0x55ba18062960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:02.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.251+0000 7f259b358b80 4 client.26131 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2024-02-02T15:29:02.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.251+0000 7f259b358b80 1 -- 172.21.15.1:0/2973113418 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ba17f02f50 con 0x55ba18060540 2024-02-02T15:29:02.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.252+0000 7f2584e00700 1 --2- 172.21.15.1:0/2973113418 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55ba18060540 0x55ba18062960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:02.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.252+0000 7f2584e00700 1 --2- 172.21.15.1:0/2973113418 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55ba18060540 0x55ba18062960 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55ec707453e57de9 server_cookie=f66635dd871fb968 in_seq=0 out_seq=0 2024-02-02T15:29:02.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.252+0000 7f256ea00700 10 client.26131 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:29:02.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.252+0000 7f256ea00700 1 -- 172.21.15.1:0/2973113418 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55ba17f02f50 con 0x55ba18060540 2024-02-02T15:29:02.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.252+0000 7f256ea00700 10 client.26131 handle_command_reply: tid=0 2024-02-02T15:29:02.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.252+0000 7f259b358b80 1 client.26131 shutdown 2024-02-02T15:29:02.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.252+0000 7f259b358b80 1 -- 172.21.15.1:0/2973113418 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba17ff2930 msgr2=0x55ba17ff6ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:02.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.252+0000 7f259b358b80 1 --2- 172.21.15.1:0/2973113418 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba17ff2930 0x55ba17ff6ad0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f256802be40 tx=0x7f2568003c00 comp rx=0 tx=0).stop 2024-02-02T15:29:02.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.252+0000 7f259b358b80 1 -- 172.21.15.1:0/2973113418 shutdown_connections 2024-02-02T15:29:02.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.252+0000 7f259b358b80 1 --2- 172.21.15.1:0/2973113418 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55ba18060540 0x55ba18062960 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:02.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.252+0000 7f259b358b80 1 --2- 172.21.15.1:0/2973113418 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba17ffd7e0 0x55ba17ff71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:02.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.252+0000 7f259b358b80 1 --2- 172.21.15.1:0/2973113418 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba17ff2010 0x55ba180042d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:02.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.252+0000 7f259b358b80 1 --2- 172.21.15.1:0/2973113418 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba17ff2930 0x55ba17ff6ad0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:02.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.252+0000 7f256ea00700 0 client.26131 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:29:02.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.252+0000 7f259b358b80 1 -- 172.21.15.1:0/2973113418 >> 172.21.15.1:0/2973113418 conn(0x55ba17f51db0 msgr2=0x55ba180088a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:02.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.253+0000 7f259b358b80 1 -- 172.21.15.1:0/2973113418 shutdown_connections 2024-02-02T15:29:02.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.253+0000 7f259b358b80 1 -- 172.21.15.1:0/2973113418 wait complete. 2024-02-02T15:29:02.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.253+0000 7f259b358b80 20 client.26131 trim_cache size 0 max 16384 2024-02-02T15:29:02.273 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:29:02.274 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2024-02-02T15:29:02.274 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2024-02-02T15:29:02.274 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:29:02.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.253+0000 7f2587800700 1 -- 172.21.15.1:0/2711133743 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f25600a0690 msgr2=0x7f25600a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:02.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.253+0000 7f2587800700 1 --2- 172.21.15.1:0/2711133743 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f25600a0690 0x7f25600a2b50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f258019ebd0 tx=0x7f2568009260 comp rx=0 tx=0).stop 2024-02-02T15:29:02.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.253+0000 7f2587800700 1 -- 172.21.15.1:0/2711133743 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25800730e0 msgr2=0x7f258019d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:02.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.253+0000 7f2587800700 1 --2- 172.21.15.1:0/2711133743 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25800730e0 0x7f258019d5c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f257000e9a0 tx=0x7f257000ecb0 comp rx=0 tx=0).stop 2024-02-02T15:29:02.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.253+0000 7f2587800700 1 -- 172.21.15.1:0/2711133743 shutdown_connections 2024-02-02T15:29:02.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.253+0000 7f2587800700 1 --2- 172.21.15.1:0/2711133743 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f25600a0690 0x7f25600a2b50 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:02.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.253+0000 7f2587800700 1 --2- 172.21.15.1:0/2711133743 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2580111d90 0x7f25801a1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:02.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.253+0000 7f2587800700 1 --2- 172.21.15.1:0/2711133743 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f25800730e0 0x7f258019d5c0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:02.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.253+0000 7f2587800700 1 --2- 172.21.15.1:0/2711133743 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2580073a00 0x7f258019db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:02.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.254+0000 7f2587800700 1 -- 172.21.15.1:0/2711133743 >> 172.21.15.1:0/2711133743 conn(0x7f25800fc810 msgr2=0x7f25800fd480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:02.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.254+0000 7f2587800700 1 -- 172.21.15.1:0/2711133743 shutdown_connections 2024-02-02T15:29:02.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.254+0000 7f2587800700 1 -- 172.21.15.1:0/2711133743 wait complete. 2024-02-02T15:29:02.278 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-02T15:29:02.278 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T15:29:02.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.534+0000 7fcf98c00700 1 -- 172.21.15.1:0/38678569 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcf940ff140 msgr2=0x7fcf940ff520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:02.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.534+0000 7fcf98c00700 1 --2- 172.21.15.1:0/38678569 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcf940ff140 0x7fcf940ff520 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fcf840098f0 tx=0x7fcf84009c00 comp rx=0 tx=0).stop 2024-02-02T15:29:02.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.535+0000 7fcf98c00700 1 -- 172.21.15.1:0/38678569 shutdown_connections 2024-02-02T15:29:02.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.535+0000 7fcf98c00700 1 --2- 172.21.15.1:0/38678569 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf94104260 0x7fcf9410bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:02.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.535+0000 7fcf98c00700 1 --2- 172.21.15.1:0/38678569 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcf940ff140 0x7fcf940ff520 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:02.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.535+0000 7fcf98c00700 1 --2- 172.21.15.1:0/38678569 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf940ffaf0 0x7fcf94103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:02.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.535+0000 7fcf98c00700 1 -- 172.21.15.1:0/38678569 >> 172.21.15.1:0/38678569 conn(0x7fcf94075cc0 msgr2=0x7fcf940fd040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:02.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.535+0000 7fcf98c00700 1 -- 172.21.15.1:0/38678569 shutdown_connections 2024-02-02T15:29:02.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.535+0000 7fcf98c00700 1 -- 172.21.15.1:0/38678569 wait complete. 2024-02-02T15:29:02.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.540+0000 7fcf98c00700 1 Processor -- start 2024-02-02T15:29:02.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.543+0000 7fcf98c00700 1 -- start start 2024-02-02T15:29:02.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.546+0000 7fcf98c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf940ff140 0x7fcf94107e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:02.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.546+0000 7fcf98c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcf94104260 0x7fcf941083d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:02.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.546+0000 7fcf98c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf94108910 0x7fcf941a9810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:02.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.546+0000 7fcf98c00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf9410c150 con 0x7fcf94104260 2024-02-02T15:29:02.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.546+0000 7fcf98c00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf941a9d50 con 0x7fcf940ff140 2024-02-02T15:29:02.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.546+0000 7fcf98c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf941a9ec0 con 0x7fcf94108910 2024-02-02T15:29:02.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.546+0000 7fcf91e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf940ff140 0x7fcf94107e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:02.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.547+0000 7fcf91e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf940ff140 0x7fcf94107e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:60834/0 (socket says 172.21.15.1:60834) 2024-02-02T15:29:02.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.547+0000 7fcf91e00700 1 -- 172.21.15.1:0/3797650113 learned_addr learned my addr 172.21.15.1:0/3797650113 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:29:02.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.547+0000 7fcf91400700 1 --2- 172.21.15.1:0/3797650113 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcf94104260 0x7fcf941083d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:02.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.547+0000 7fcf91e00700 1 -- 172.21.15.1:0/3797650113 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf94108910 msgr2=0x7fcf941a9810 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-02-02T15:29:02.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.548+0000 7fcf92800700 1 --2- 172.21.15.1:0/3797650113 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf94108910 0x7fcf941a9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:02.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.548+0000 7fcf91e00700 1 --2- 172.21.15.1:0/3797650113 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf94108910 0x7fcf941a9810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:02.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.548+0000 7fcf91e00700 1 -- 172.21.15.1:0/3797650113 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcf94104260 msgr2=0x7fcf941083d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:02.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.548+0000 7fcf91e00700 1 --2- 172.21.15.1:0/3797650113 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcf94104260 0x7fcf941083d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:02.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.548+0000 7fcf91e00700 1 -- 172.21.15.1:0/3797650113 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf8c009c40 con 0x7fcf940ff140 2024-02-02T15:29:02.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.548+0000 7fcf91400700 1 --2- 172.21.15.1:0/3797650113 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcf94104260 0x7fcf941083d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:29:02.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.548+0000 7fcf92800700 1 --2- 172.21.15.1:0/3797650113 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf94108910 0x7fcf941a9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:29:02.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.548+0000 7fcf91e00700 1 --2- 172.21.15.1:0/3797650113 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf940ff140 0x7fcf94107e90 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fcf84009810 tx=0x7fcf8400fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:02.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.550+0000 7fcf8aa00700 1 -- 172.21.15.1:0/3797650113 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf8401c070 con 0x7fcf940ff140 2024-02-02T15:29:02.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.550+0000 7fcf8aa00700 1 -- 172.21.15.1:0/3797650113 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf84005040 con 0x7fcf940ff140 2024-02-02T15:29:02.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.550+0000 7fcf98c00700 1 -- 172.21.15.1:0/3797650113 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf84009520 con 0x7fcf940ff140 2024-02-02T15:29:02.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.550+0000 7fcf8aa00700 1 -- 172.21.15.1:0/3797650113 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf84004a10 con 0x7fcf940ff140 2024-02-02T15:29:02.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.551+0000 7fcf98c00700 1 -- 172.21.15.1:0/3797650113 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf941aa3d0 con 0x7fcf940ff140 2024-02-02T15:29:02.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.554+0000 7fcf98c00700 1 -- 172.21.15.1:0/3797650113 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf94068010 con 0x7fcf940ff140 2024-02-02T15:29:02.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.557+0000 7fcf8aa00700 1 -- 172.21.15.1:0/3797650113 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 27) v1 ==== 135200+0+0 (secure 0 0 0) 0x7fcf84020070 con 0x7fcf940ff140 2024-02-02T15:29:02.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.559+0000 7fcf8aa00700 1 --2- 172.21.15.1:0/3797650113 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fcf740a0710 0x7fcf740a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:02.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.559+0000 7fcf8aa00700 1 -- 172.21.15.1:0/3797650113 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fcf840cce20 con 0x7fcf940ff140 2024-02-02T15:29:02.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.559+0000 7fcf91400700 1 --2- 172.21.15.1:0/3797650113 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fcf740a0710 0x7fcf740a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:02.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.560+0000 7fcf91400700 1 --2- 172.21.15.1:0/3797650113 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fcf740a0710 0x7fcf740a2bd0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fcf94109680 tx=0x7fcf7c00c040 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:02.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.567+0000 7fcf8aa00700 1 -- 172.21.15.1:0/3797650113 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7fcf840964e0 con 0x7fcf940ff140 2024-02-02T15:29:02.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.844+0000 7fcf98c00700 1 -- 172.21.15.1:0/3797650113 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcf940ffa60 con 0x7fcf940ff140 2024-02-02T15:29:02.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.844+0000 7fcf8aa00700 1 -- 172.21.15.1:0/3797650113 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+10873 (secure 0 0 0) 0x7fcf84099b00 con 0x7fcf940ff140 2024-02-02T15:29:02.845 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:29:02.845 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2922240448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2922240448},{"type":"v1","addr":"172.21.15.1:6835","nonce":2922240448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2729497431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2729497431},{"type":"v1","addr":"172.21.15.1:6837","nonce":2729497431}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-02T15:24:47.761435+0000","modified":"2024-02-02T15:28:52.416969+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24529,"mds_2":24518,"mds_3":24514,"mds_4":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14517":{"gid":14517,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/1218303683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":1218303683},{"type":"v1","addr":"172.21.15.1:6839","nonce":1218303683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14529":{"gid":14529,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/2059197496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2059197496},{"type":"v1","addr":"172.21.15.1:6841","nonce":2059197496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/7648986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":7648986},{"type":"v1","addr":"172.21.15.142:6833","nonce":7648986}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/3482746094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3482746094},{"type":"v1","addr":"172.21.15.71:6833","nonce":3482746094}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/388207814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":388207814},{"type":"v1","addr":"172.21.15.71:6835","nonce":388207814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6835/1052606547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1052606547},{"type":"v1","addr":"172.21.15.142:6835","nonce":1052606547}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.142:6837/2870289019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2870289019},{"type":"v1","addr":"172.21.15.142:6837","nonce":2870289019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.71:6837/628016199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":628016199},{"type":"v1","addr":"172.21.15.71:6837","nonce":628016199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.142:6839/321675421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":321675421},{"type":"v1","addr":"172.21.15.142:6839","nonce":321675421}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/1268086493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1268086493},{"type":"v1","addr":"172.21.15.71:6839","nonce":1268086493}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-02T15:29:02.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.847+0000 7fcf98c00700 1 -- 172.21.15.1:0/3797650113 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fcf740a0710 msgr2=0x7fcf740a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:02.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.847+0000 7fcf98c00700 1 --2- 172.21.15.1:0/3797650113 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fcf740a0710 0x7fcf740a2bd0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fcf94109680 tx=0x7fcf7c00c040 comp rx=0 tx=0).stop 2024-02-02T15:29:02.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.847+0000 7fcf98c00700 1 -- 172.21.15.1:0/3797650113 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf940ff140 msgr2=0x7fcf94107e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:02.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.847+0000 7fcf98c00700 1 --2- 172.21.15.1:0/3797650113 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf940ff140 0x7fcf94107e90 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fcf84009810 tx=0x7fcf8400fa00 comp rx=0 tx=0).stop 2024-02-02T15:29:02.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.847+0000 7fcf98c00700 1 -- 172.21.15.1:0/3797650113 shutdown_connections 2024-02-02T15:29:02.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.847+0000 7fcf98c00700 1 --2- 172.21.15.1:0/3797650113 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fcf740a0710 0x7fcf740a2bd0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:02.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.847+0000 7fcf98c00700 1 --2- 172.21.15.1:0/3797650113 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcf94108910 0x7fcf941a9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:02.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.847+0000 7fcf98c00700 1 --2- 172.21.15.1:0/3797650113 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcf940ff140 0x7fcf94107e90 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:02.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.847+0000 7fcf98c00700 1 --2- 172.21.15.1:0/3797650113 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcf94104260 0x7fcf941083d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:02.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.847+0000 7fcf98c00700 1 -- 172.21.15.1:0/3797650113 >> 172.21.15.1:0/3797650113 conn(0x7fcf94075cc0 msgr2=0x7fcf94102140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:02.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.847+0000 7fcf98c00700 1 -- 172.21.15.1:0/3797650113 shutdown_connections 2024-02-02T15:29:02.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:02.847+0000 7fcf98c00700 1 -- 172.21.15.1:0/3797650113 wait complete. 2024-02-02T15:29:02.849 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 29 2024-02-02T15:29:02.867 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-02-02T15:29:03.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.131+0000 7f186b400700 1 -- 172.21.15.1:0/1119165257 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1864106110 msgr2=0x7f186410dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:03.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.131+0000 7f186b400700 1 --2- 172.21.15.1:0/1119165257 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1864106110 0x7f186410dc90 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f185c009f90 tx=0x7f185c00b330 comp rx=0 tx=0).stop 2024-02-02T15:29:03.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.131+0000 7f186b400700 1 -- 172.21.15.1:0/1119165257 shutdown_connections 2024-02-02T15:29:03.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.131+0000 7f186b400700 1 --2- 172.21.15.1:0/1119165257 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1864106110 0x7f186410dc90 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.132+0000 7f186b400700 1 --2- 172.21.15.1:0/1119165257 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1864100ff0 0x7f18641013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.132+0000 7f186b400700 1 --2- 172.21.15.1:0/1119165257 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f18641019a0 0x7f18641059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.132+0000 7f186b400700 1 -- 172.21.15.1:0/1119165257 >> 172.21.15.1:0/1119165257 conn(0x7f18640fc810 msgr2=0x7f18640fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:03.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.132+0000 7f186b400700 1 -- 172.21.15.1:0/1119165257 shutdown_connections 2024-02-02T15:29:03.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.132+0000 7f186b400700 1 -- 172.21.15.1:0/1119165257 wait complete. 2024-02-02T15:29:03.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.133+0000 7f186b400700 1 Processor -- start 2024-02-02T15:29:03.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.133+0000 7f186b400700 1 -- start start 2024-02-02T15:29:03.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.133+0000 7f186b400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1864100ff0 0x7f1864196e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:03.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.133+0000 7f186b400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1864106110 0x7f1864197390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:03.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.133+0000 7f186b400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f186419b7f0 0x7f186419bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:03.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.133+0000 7f186b400700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f186419c270 con 0x7f186419b7f0 2024-02-02T15:29:03.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.133+0000 7f186b400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f186419c3e0 con 0x7f1864100ff0 2024-02-02T15:29:03.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.133+0000 7f186b400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f186419c5a0 con 0x7f1864106110 2024-02-02T15:29:03.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.134+0000 7f1863e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1864106110 0x7f1864197390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:03.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.134+0000 7f1868a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1864100ff0 0x7f1864196e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:03.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.134+0000 7f1863e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1864106110 0x7f1864197390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:54444/0 (socket says 172.21.15.1:54444) 2024-02-02T15:29:03.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.134+0000 7f1863e00700 1 -- 172.21.15.1:0/105052634 learned_addr learned my addr 172.21.15.1:0/105052634 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:29:03.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.134+0000 7f1869400700 1 --2- 172.21.15.1:0/105052634 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f186419b7f0 0x7f186419bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:03.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.134+0000 7f1868a00700 1 -- 172.21.15.1:0/105052634 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1864106110 msgr2=0x7f1864197390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:03.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.134+0000 7f1868a00700 1 --2- 172.21.15.1:0/105052634 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1864106110 0x7f1864197390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.134+0000 7f1868a00700 1 -- 172.21.15.1:0/105052634 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f186419b7f0 msgr2=0x7f186419bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:03.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.134+0000 7f1868a00700 1 --2- 172.21.15.1:0/105052634 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f186419b7f0 0x7f186419bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.134+0000 7f1868a00700 1 -- 172.21.15.1:0/105052634 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f185c009c40 con 0x7f1864100ff0 2024-02-02T15:29:03.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.134+0000 7f1863e00700 1 --2- 172.21.15.1:0/105052634 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1864106110 0x7f1864197390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:29:03.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.134+0000 7f1868a00700 1 --2- 172.21.15.1:0/105052634 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1864100ff0 0x7f1864196e50 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f185400e9f0 tx=0x7f185400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:03.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.134+0000 7f1861600700 1 -- 172.21.15.1:0/105052634 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185400c9c0 con 0x7f1864100ff0 2024-02-02T15:29:03.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.135+0000 7f1861600700 1 -- 172.21.15.1:0/105052634 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1854010430 con 0x7f1864100ff0 2024-02-02T15:29:03.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.135+0000 7f1861600700 1 -- 172.21.15.1:0/105052634 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1854018680 con 0x7f1864100ff0 2024-02-02T15:29:03.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.135+0000 7f186b400700 1 -- 172.21.15.1:0/105052634 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18641a3b00 con 0x7f1864100ff0 2024-02-02T15:29:03.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.135+0000 7f186b400700 1 -- 172.21.15.1:0/105052634 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18641a3fe0 con 0x7f1864100ff0 2024-02-02T15:29:03.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.138+0000 7f1861600700 1 -- 172.21.15.1:0/105052634 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 27) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f1854010860 con 0x7f1864100ff0 2024-02-02T15:29:03.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.139+0000 7f187ee8fb80 1 Processor -- start 2024-02-02T15:29:03.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.139+0000 7f187ee8fb80 1 -- start start 2024-02-02T15:29:03.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.139+0000 7f1861600700 1 --2- 172.21.15.1:0/105052634 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f18440a0710 0x7f18440a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:03.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.140+0000 7f1861600700 1 -- 172.21.15.1:0/105052634 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f1854014070 con 0x7f1864100ff0 2024-02-02T15:29:03.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.140+0000 7f187ee8fb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555f83907010 0x555f839073f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:03.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.140+0000 7f187ee8fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555f83907930 0x555f839166b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:03.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.140+0000 7f187ee8fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555f83916bf0 0x555f83918fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:03.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.140+0000 7f187ee8fb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f8387b4c0 con 0x555f83907010 2024-02-02T15:29:03.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.140+0000 7f187ee8fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f8385faa0 con 0x555f83916bf0 2024-02-02T15:29:03.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.140+0000 7f187ee8fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f8385d460 con 0x555f83907930 2024-02-02T15:29:03.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.140+0000 7f1869400700 1 --2- 172.21.15.1:0/105052634 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f18440a0710 0x7f18440a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:03.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.140+0000 7f1868a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555f83907010 0x555f839073f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:03.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.140+0000 7f1868a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555f83907010 0x555f839073f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37712/0 (socket says 172.21.15.1:37712) 2024-02-02T15:29:03.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.140+0000 7f1868a00700 1 -- 172.21.15.1:0/4248067320 learned_addr learned my addr 172.21.15.1:0/4248067320 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:29:03.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.140+0000 7f1869400700 1 --2- 172.21.15.1:0/4248067320 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555f83916bf0 0x555f83918fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:03.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.140+0000 7f1863e00700 1 --2- 172.21.15.1:0/4248067320 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555f83907930 0x555f839166b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:03.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.141+0000 7f1863e00700 1 -- 172.21.15.1:0/4248067320 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555f83916bf0 msgr2=0x555f83918fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:03.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.141+0000 7f1863e00700 1 --2- 172.21.15.1:0/4248067320 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555f83916bf0 0x555f83918fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.141+0000 7f1863e00700 1 -- 172.21.15.1:0/4248067320 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555f83907010 msgr2=0x555f839073f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:03.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.141+0000 7f1863e00700 1 --2- 172.21.15.1:0/4248067320 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555f83907010 0x555f839073f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.141+0000 7f1863e00700 1 -- 172.21.15.1:0/4248067320 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f836da3b0 con 0x555f83907930 2024-02-02T15:29:03.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.141+0000 7f1869400700 1 --2- 172.21.15.1:0/105052634 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f18440a0710 0x7f18440a2bd0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f185c009790 tx=0x7f185c004b40 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:03.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.141+0000 7f1863e00700 1 --2- 172.21.15.1:0/4248067320 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555f83907930 0x555f839166b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f184c0114f0 tx=0x7f184c011800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ec3304ae624108a server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:03.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.141+0000 7f1859c00700 1 -- 172.21.15.1:0/4248067320 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f184c004030 con 0x555f83907930 2024-02-02T15:29:03.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.141+0000 7f1859c00700 1 -- 172.21.15.1:0/4248067320 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f184c003680 con 0x555f83907930 2024-02-02T15:29:03.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.141+0000 7f1859c00700 1 -- 172.21.15.1:0/4248067320 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f184c00e6d0 con 0x555f83907930 2024-02-02T15:29:03.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.142+0000 7f187ee8fb80 1 -- 172.21.15.1:0/4248067320 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555f83907930 msgr2=0x555f839166b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:03.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.142+0000 7f187ee8fb80 1 --2- 172.21.15.1:0/4248067320 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555f83907930 0x555f839166b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f184c0114f0 tx=0x7f184c011800 comp rx=0 tx=0).stop 2024-02-02T15:29:03.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.142+0000 7f187ee8fb80 1 -- 172.21.15.1:0/4248067320 shutdown_connections 2024-02-02T15:29:03.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.142+0000 7f187ee8fb80 1 --2- 172.21.15.1:0/4248067320 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555f83916bf0 0x555f83918fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.142+0000 7f187ee8fb80 1 --2- 172.21.15.1:0/4248067320 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555f83907010 0x555f839073f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.142+0000 7f187ee8fb80 1 --2- 172.21.15.1:0/4248067320 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555f83907930 0x555f839166b0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.142+0000 7f187ee8fb80 1 -- 172.21.15.1:0/4248067320 >> 172.21.15.1:0/4248067320 conn(0x555f83866db0 msgr2=0x555f839033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:03.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.142+0000 7f187ee8fb80 1 -- 172.21.15.1:0/4248067320 shutdown_connections 2024-02-02T15:29:03.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.142+0000 7f187ee8fb80 1 -- 172.21.15.1:0/4248067320 wait complete. 2024-02-02T15:29:03.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.143+0000 7f187ee8fb80 1 -- start start 2024-02-02T15:29:03.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.143+0000 7f187ee8fb80 1 Processor -- start 2024-02-02T15:29:03.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.144+0000 7f187ee8fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555f83907010 0x555f83916040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:03.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.144+0000 7f187ee8fb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555f83907930 0x555f83916580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:03.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.144+0000 7f187ee8fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555f83916bf0 0x555f8390f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:03.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.144+0000 7f187ee8fb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f83858d50 con 0x555f83907930 2024-02-02T15:29:03.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.144+0000 7f187ee8fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f837f4d80 con 0x555f83907010 2024-02-02T15:29:03.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.144+0000 7f187ee8fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f837f4a20 con 0x555f83916bf0 2024-02-02T15:29:03.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.144+0000 7f187ee8fb80 10 client.0 _ll_register_callbacks cb 0x555f837f6290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:29:03.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.144+0000 7f1869400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555f83916bf0 0x555f8390f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:03.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.144+0000 7f1869400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555f83916bf0 0x555f8390f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:54450/0 (socket says 172.21.15.1:54450) 2024-02-02T15:29:03.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.144+0000 7f1869400700 1 -- 172.21.15.1:0/1649214516 learned_addr learned my addr 172.21.15.1:0/1649214516 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:29:03.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.144+0000 7f1863e00700 1 --2- 172.21.15.1:0/1649214516 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555f83907930 0x555f83916580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:03.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.144+0000 7f1868a00700 1 --2- 172.21.15.1:0/1649214516 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555f83907010 0x555f83916040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:03.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.144+0000 7f1869400700 1 -- 172.21.15.1:0/1649214516 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555f83907010 msgr2=0x555f83916040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:03.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.144+0000 7f1869400700 1 --2- 172.21.15.1:0/1649214516 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555f83907010 0x555f83916040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.145+0000 7f1869400700 1 -- 172.21.15.1:0/1649214516 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555f83907930 msgr2=0x555f83916580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:03.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.145+0000 7f1869400700 1 --2- 172.21.15.1:0/1649214516 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555f83907930 0x555f83916580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.145+0000 7f1869400700 1 -- 172.21.15.1:0/1649214516 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f184c009cc0 con 0x555f83916bf0 2024-02-02T15:29:03.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.145+0000 7f1869400700 1 --2- 172.21.15.1:0/1649214516 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555f83916bf0 0x555f8390f350 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f185c00bf40 tx=0x7f185c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c127e5f0c035dfa3 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:03.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.145+0000 7f1852000700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:29:03.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.145+0000 7f1852000700 1 -- 172.21.15.1:0/1649214516 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185c00e040 con 0x555f83916bf0 2024-02-02T15:29:03.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.145+0000 7f1852000700 1 -- 172.21.15.1:0/1649214516 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f185c01fad0 con 0x555f83916bf0 2024-02-02T15:29:03.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.145+0000 7f187ee8fb80 1 -- 172.21.15.1:0/1649214516 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555f838fb0f0 con 0x555f83916bf0 2024-02-02T15:29:03.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.146+0000 7f1852000700 1 -- 172.21.15.1:0/1649214516 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f185c01fc90 con 0x555f83916bf0 2024-02-02T15:29:03.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.146+0000 7f187ee8fb80 10 client.26146 fetch_fsmap learned FSMap version 29 2024-02-02T15:29:03.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.146+0000 7f187ee8fb80 1 -- 172.21.15.1:0/1649214516 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x555f83686b60 con 0x555f83916bf0 2024-02-02T15:29:03.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.146+0000 7f1852000700 1 -- 172.21.15.1:0/1649214516 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555f838fb0f0 con 0x555f83916bf0 2024-02-02T15:29:03.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.146+0000 7f1852000700 1 -- 172.21.15.1:0/1649214516 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 29) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f185c034070 con 0x555f83916bf0 2024-02-02T15:29:03.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.146+0000 7f187ee8fb80 10 client.26146 fetch_fsmap finished waiting for FSMap version 29 2024-02-02T15:29:03.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.146+0000 7f187ee8fb80 10 client.26146 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-02-02T15:29:03.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.146+0000 7f187ee8fb80 20 client.26146 populate_metadata read hostname 'smithi001' 2024-02-02T15:29:03.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.146+0000 7f187ee8fb80 1 --2- 172.21.15.1:0/1649214516 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x555f8392a7d0 0x555f8392cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:03.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.146+0000 7f187ee8fb80 4 client.26146 mds_command: new command op to 24502 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:29:03.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.146+0000 7f187ee8fb80 1 -- 172.21.15.1:0/1649214516 --> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555f83851dd0 con 0x555f8392a7d0 2024-02-02T15:29:03.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.147+0000 7f1868a00700 1 --2- 172.21.15.1:0/1649214516 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x555f8392a7d0 0x555f8392cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:03.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.147+0000 7f1868a00700 1 --2- 172.21.15.1:0/1649214516 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x555f8392a7d0 0x555f8392cbf0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b44482bf2ab84e4f server_cookie=36da49240e874bf8 in_seq=0 out_seq=0 2024-02-02T15:29:03.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.147+0000 7f1852000700 10 client.26146 ms_handle_connect on v2:172.21.15.142:6834/1052606547 2024-02-02T15:29:03.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.148+0000 7f1852000700 1 -- 172.21.15.1:0/1649214516 <== mds.4 v2:172.21.15.142:6834/1052606547 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x555f83851dd0 con 0x555f8392a7d0 2024-02-02T15:29:03.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.148+0000 7f1852000700 10 client.26146 handle_command_reply: tid=0 2024-02-02T15:29:03.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.148+0000 7f187ee8fb80 1 client.26146 shutdown 2024-02-02T15:29:03.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.148+0000 7f187ee8fb80 1 -- 172.21.15.1:0/1649214516 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555f83916bf0 msgr2=0x555f8390f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:03.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.148+0000 7f187ee8fb80 1 --2- 172.21.15.1:0/1649214516 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555f83916bf0 0x555f8390f350 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f185c00bf40 tx=0x7f185c00b330 comp rx=0 tx=0).stop 2024-02-02T15:29:03.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.148+0000 7f187ee8fb80 1 -- 172.21.15.1:0/1649214516 shutdown_connections 2024-02-02T15:29:03.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.148+0000 7f187ee8fb80 1 --2- 172.21.15.1:0/1649214516 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x555f8392a7d0 0x555f8392cbf0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.148+0000 7f187ee8fb80 1 --2- 172.21.15.1:0/1649214516 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555f83916bf0 0x555f8390f350 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.148+0000 7f187ee8fb80 1 --2- 172.21.15.1:0/1649214516 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555f83907010 0x555f83916040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.148+0000 7f187ee8fb80 1 --2- 172.21.15.1:0/1649214516 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555f83907930 0x555f83916580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.148+0000 7f187ee8fb80 1 -- 172.21.15.1:0/1649214516 >> 172.21.15.1:0/1649214516 conn(0x555f83866db0 msgr2=0x555f83902fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:03.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.148+0000 7f1852000700 0 client.26146 ms_handle_reset on v2:172.21.15.142:6834/1052606547 2024-02-02T15:29:03.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.148+0000 7f187ee8fb80 1 -- 172.21.15.1:0/1649214516 shutdown_connections 2024-02-02T15:29:03.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.148+0000 7f187ee8fb80 1 -- 172.21.15.1:0/1649214516 wait complete. 2024-02-02T15:29:03.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.149+0000 7f187ee8fb80 20 client.26146 trim_cache size 0 max 16384 2024-02-02T15:29:03.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.164+0000 7f187ee8fb80 1 Processor -- start 2024-02-02T15:29:03.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.164+0000 7f187ee8fb80 1 -- start start 2024-02-02T15:29:03.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.164+0000 7f187ee8fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555f83907010 0x555f8390ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:03.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.164+0000 7f187ee8fb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555f83907930 0x555f83902eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:03.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.165+0000 7f187ee8fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555f83916bf0 0x555f839033f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:03.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.165+0000 7f187ee8fb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f836dc6a0 con 0x555f83907930 2024-02-02T15:29:03.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.165+0000 7f187ee8fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f83910860 con 0x555f83907010 2024-02-02T15:29:03.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.165+0000 7f187ee8fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f839021a0 con 0x555f83916bf0 2024-02-02T15:29:03.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.165+0000 7f1868a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555f83907010 0x555f8390ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:03.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.165+0000 7f1868a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555f83907010 0x555f8390ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:45570/0 (socket says 172.21.15.1:45570) 2024-02-02T15:29:03.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.165+0000 7f1868a00700 1 -- 172.21.15.1:0/2571126020 learned_addr learned my addr 172.21.15.1:0/2571126020 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:29:03.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.165+0000 7f1863e00700 1 --2- 172.21.15.1:0/2571126020 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555f83907930 0x555f83902eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:03.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.165+0000 7f1869400700 1 --2- 172.21.15.1:0/2571126020 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555f83916bf0 0x555f839033f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:03.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.165+0000 7f1868a00700 1 -- 172.21.15.1:0/2571126020 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555f83916bf0 msgr2=0x555f839033f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:03.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.165+0000 7f1868a00700 1 --2- 172.21.15.1:0/2571126020 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555f83916bf0 0x555f839033f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.165+0000 7f1868a00700 1 -- 172.21.15.1:0/2571126020 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555f83907930 msgr2=0x555f83902eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:03.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.165+0000 7f1868a00700 1 --2- 172.21.15.1:0/2571126020 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555f83907930 0x555f83902eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.165+0000 7f1868a00700 1 -- 172.21.15.1:0/2571126020 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555f8381eea0 con 0x555f83907010 2024-02-02T15:29:03.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.165+0000 7f1868a00700 1 --2- 172.21.15.1:0/2571126020 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555f83907010 0x555f8390ea40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1854095a30 tx=0x7f18540cb040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=396230f8d5725123 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:03.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.166+0000 7f185a600700 1 -- 172.21.15.1:0/2571126020 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185409d170 con 0x555f83907010 2024-02-02T15:29:03.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.166+0000 7f185a600700 1 -- 172.21.15.1:0/2571126020 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1854095040 con 0x555f83907010 2024-02-02T15:29:03.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.166+0000 7f185a600700 1 -- 172.21.15.1:0/2571126020 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1854097960 con 0x555f83907010 2024-02-02T15:29:03.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.166+0000 7f187ee8fb80 1 -- 172.21.15.1:0/2571126020 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555f83907010 msgr2=0x555f8390ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:03.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.166+0000 7f187ee8fb80 1 --2- 172.21.15.1:0/2571126020 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555f83907010 0x555f8390ea40 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1854095a30 tx=0x7f18540cb040 comp rx=0 tx=0).stop 2024-02-02T15:29:03.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.166+0000 7f187ee8fb80 1 -- 172.21.15.1:0/2571126020 shutdown_connections 2024-02-02T15:29:03.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.166+0000 7f187ee8fb80 1 --2- 172.21.15.1:0/2571126020 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555f83916bf0 0x555f839033f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.166+0000 7f187ee8fb80 1 --2- 172.21.15.1:0/2571126020 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555f83907010 0x555f8390ea40 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.166+0000 7f187ee8fb80 1 --2- 172.21.15.1:0/2571126020 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555f83907930 0x555f83902eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.166+0000 7f187ee8fb80 1 -- 172.21.15.1:0/2571126020 >> 172.21.15.1:0/2571126020 conn(0x555f83866db0 msgr2=0x555f8390a600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:03.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.167+0000 7f187ee8fb80 1 -- 172.21.15.1:0/2571126020 shutdown_connections 2024-02-02T15:29:03.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.167+0000 7f187ee8fb80 1 -- 172.21.15.1:0/2571126020 wait complete. 2024-02-02T15:29:03.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.167+0000 7f187ee8fb80 1 -- start start 2024-02-02T15:29:03.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.167+0000 7f187ee8fb80 1 Processor -- start 2024-02-02T15:29:03.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.168+0000 7f187ee8fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555f83907010 0x555f839237f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:03.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.168+0000 7f187ee8fb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555f83907930 0x555f83923d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:03.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.168+0000 7f187ee8fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555f83916bf0 0x555f8397b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:03.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.168+0000 7f187ee8fb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f8397b820 con 0x555f83907930 2024-02-02T15:29:03.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.168+0000 7f187ee8fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f8397b960 con 0x555f83916bf0 2024-02-02T15:29:03.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.168+0000 7f187ee8fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x555f8397bad0 con 0x555f83907010 2024-02-02T15:29:03.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.168+0000 7f187ee8fb80 10 client.0 _ll_register_callbacks cb 0x555f837f6290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:29:03.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.168+0000 7f1868a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555f83907010 0x555f839237f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:03.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.168+0000 7f1869400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555f83916bf0 0x555f8397b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:03.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.168+0000 7f1863e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555f83907930 0x555f83923d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:03.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.168+0000 7f1869400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555f83916bf0 0x555f8397b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:45580/0 (socket says 172.21.15.1:45580) 2024-02-02T15:29:03.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.168+0000 7f1868a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555f83907010 0x555f839237f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:54458/0 (socket says 172.21.15.1:54458) 2024-02-02T15:29:03.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.168+0000 7f1869400700 1 -- 172.21.15.1:0/408194173 learned_addr learned my addr 172.21.15.1:0/408194173 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:29:03.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.168+0000 7f1869400700 1 -- 172.21.15.1:0/408194173 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555f83907010 msgr2=0x555f839237f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:03.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.168+0000 7f1869400700 1 --2- 172.21.15.1:0/408194173 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555f83907010 0x555f839237f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.168+0000 7f1869400700 1 -- 172.21.15.1:0/408194173 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555f83907930 msgr2=0x555f83923d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:03.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.168+0000 7f1869400700 1 --2- 172.21.15.1:0/408194173 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555f83907930 0x555f83923d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.168+0000 7f1869400700 1 -- 172.21.15.1:0/408194173 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18641a3fe0 con 0x555f83916bf0 2024-02-02T15:29:03.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.168+0000 7f1869400700 1 --2- 172.21.15.1:0/408194173 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555f83916bf0 0x555f8397b2e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f185c02cec0 tx=0x7f185c01b650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aae6ace7712b2de9 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:03.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.169+0000 7f1852000700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:29:03.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.169+0000 7f1852000700 1 -- 172.21.15.1:0/408194173 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185c00e070 con 0x555f83916bf0 2024-02-02T15:29:03.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.169+0000 7f1852000700 1 -- 172.21.15.1:0/408194173 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f185c01c560 con 0x555f83916bf0 2024-02-02T15:29:03.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.169+0000 7f187ee8fb80 1 -- 172.21.15.1:0/408194173 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f184c003680 con 0x555f83916bf0 2024-02-02T15:29:03.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.169+0000 7f1852000700 1 -- 172.21.15.1:0/408194173 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f185c003e10 con 0x555f83916bf0 2024-02-02T15:29:03.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.169+0000 7f187ee8fb80 10 client.26162 fetch_fsmap learned FSMap version 29 2024-02-02T15:29:03.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.169+0000 7f187ee8fb80 1 -- 172.21.15.1:0/408194173 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x555f836decd0 con 0x555f83916bf0 2024-02-02T15:29:03.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.169+0000 7f1852000700 1 -- 172.21.15.1:0/408194173 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f184c003680 con 0x555f83916bf0 2024-02-02T15:29:03.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.169+0000 7f1852000700 1 -- 172.21.15.1:0/408194173 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 29) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f185c007420 con 0x555f83916bf0 2024-02-02T15:29:03.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.169+0000 7f187ee8fb80 10 client.26162 fetch_fsmap finished waiting for FSMap version 29 2024-02-02T15:29:03.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.169+0000 7f187ee8fb80 10 client.26162 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-02-02T15:29:03.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.169+0000 7f187ee8fb80 20 client.26162 populate_metadata read hostname 'smithi001' 2024-02-02T15:29:03.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.169+0000 7f187ee8fb80 1 --2- 172.21.15.1:0/408194173 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x555f8397cee0 0x555f8397f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:03.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.169+0000 7f187ee8fb80 4 client.26162 mds_command: new command op to 24502 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:29:03.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.169+0000 7f187ee8fb80 1 -- 172.21.15.1:0/408194173 --> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555f83817f50 con 0x555f8397cee0 2024-02-02T15:29:03.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.170+0000 7f1868a00700 1 --2- 172.21.15.1:0/408194173 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x555f8397cee0 0x555f8397f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:03.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.170+0000 7f1868a00700 1 --2- 172.21.15.1:0/408194173 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x555f8397cee0 0x555f8397f300 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d71774702d60a724 server_cookie=7b12495c6ed550b9 in_seq=0 out_seq=0 2024-02-02T15:29:03.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.170+0000 7f1852000700 10 client.26162 ms_handle_connect on v2:172.21.15.142:6834/1052606547 2024-02-02T15:29:03.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.170+0000 7f1852000700 1 -- 172.21.15.1:0/408194173 <== mds.4 v2:172.21.15.142:6834/1052606547 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555f83817f50 con 0x555f8397cee0 2024-02-02T15:29:03.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.170+0000 7f1852000700 10 client.26162 handle_command_reply: tid=0 2024-02-02T15:29:03.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.170+0000 7f187ee8fb80 1 client.26162 shutdown 2024-02-02T15:29:03.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.170+0000 7f187ee8fb80 1 -- 172.21.15.1:0/408194173 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555f83916bf0 msgr2=0x555f8397b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:03.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.171+0000 7f187ee8fb80 1 --2- 172.21.15.1:0/408194173 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555f83916bf0 0x555f8397b2e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f185c02cec0 tx=0x7f185c01b650 comp rx=0 tx=0).stop 2024-02-02T15:29:03.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.171+0000 7f187ee8fb80 1 -- 172.21.15.1:0/408194173 shutdown_connections 2024-02-02T15:29:03.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.171+0000 7f187ee8fb80 1 --2- 172.21.15.1:0/408194173 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x555f8397cee0 0x555f8397f300 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.171+0000 7f187ee8fb80 1 --2- 172.21.15.1:0/408194173 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555f83916bf0 0x555f8397b2e0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.171+0000 7f187ee8fb80 1 --2- 172.21.15.1:0/408194173 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555f83907010 0x555f839237f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.171+0000 7f1852000700 0 client.26162 ms_handle_reset on v2:172.21.15.142:6834/1052606547 2024-02-02T15:29:03.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.171+0000 7f187ee8fb80 1 --2- 172.21.15.1:0/408194173 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555f83907930 0x555f83923d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.171+0000 7f187ee8fb80 1 -- 172.21.15.1:0/408194173 >> 172.21.15.1:0/408194173 conn(0x555f83866db0 msgr2=0x555f83902e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:03.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.171+0000 7f187ee8fb80 1 -- 172.21.15.1:0/408194173 shutdown_connections 2024-02-02T15:29:03.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.171+0000 7f187ee8fb80 1 -- 172.21.15.1:0/408194173 wait complete. 2024-02-02T15:29:03.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.171+0000 7f187ee8fb80 20 client.26162 trim_cache size 0 max 16384 2024-02-02T15:29:03.176 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:29:03.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.171+0000 7f186b400700 1 -- 172.21.15.1:0/105052634 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f18440a0710 msgr2=0x7f18440a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:03.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.171+0000 7f186b400700 1 --2- 172.21.15.1:0/105052634 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f18440a0710 0x7f18440a2bd0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f185c009790 tx=0x7f185c004b40 comp rx=0 tx=0).stop 2024-02-02T15:29:03.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.171+0000 7f186b400700 1 -- 172.21.15.1:0/105052634 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1864100ff0 msgr2=0x7f1864196e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:03.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.171+0000 7f186b400700 1 --2- 172.21.15.1:0/105052634 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1864100ff0 0x7f1864196e50 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f185400e9f0 tx=0x7f185400ed00 comp rx=0 tx=0).stop 2024-02-02T15:29:03.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.172+0000 7f186b400700 1 -- 172.21.15.1:0/105052634 shutdown_connections 2024-02-02T15:29:03.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.172+0000 7f186b400700 1 --2- 172.21.15.1:0/105052634 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f18440a0710 0x7f18440a2bd0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.172+0000 7f186b400700 1 --2- 172.21.15.1:0/105052634 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f186419b7f0 0x7f186419bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.172+0000 7f186b400700 1 --2- 172.21.15.1:0/105052634 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1864100ff0 0x7f1864196e50 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.172+0000 7f186b400700 1 --2- 172.21.15.1:0/105052634 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1864106110 0x7f1864197390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.172+0000 7f186b400700 1 -- 172.21.15.1:0/105052634 >> 172.21.15.1:0/105052634 conn(0x7f18640fc810 msgr2=0x7f18640fd280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:03.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.172+0000 7f186b400700 1 -- 172.21.15.1:0/105052634 shutdown_connections 2024-02-02T15:29:03.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.172+0000 7f186b400700 1 -- 172.21.15.1:0/105052634 wait complete. 2024-02-02T15:29:03.190 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-02-02T15:29:03.219 INFO:tasks.workunit.client.0.smithi001.stdout: 1 67739 10.38 MB/sec warmup 37 sec latency 95.167 ms 2024-02-02T15:29:03.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.419+0000 7f2cdca00700 1 -- 172.21.15.1:0/4162450674 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2cd4106230 msgr2=0x7f2cd410dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:03.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.419+0000 7f2cdca00700 1 --2- 172.21.15.1:0/4162450674 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2cd4106230 0x7f2cd410dd50 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc00b0a0 tx=0x7f2ccc00b3b0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.420+0000 7f2cdca00700 1 -- 172.21.15.1:0/4162450674 shutdown_connections 2024-02-02T15:29:03.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.420+0000 7f2cdca00700 1 --2- 172.21.15.1:0/4162450674 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2cd4106230 0x7f2cd410dd50 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.420+0000 7f2cdca00700 1 --2- 172.21.15.1:0/4162450674 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2cd4068f30 0x7f2cd4069310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.420+0000 7f2cdca00700 1 --2- 172.21.15.1:0/4162450674 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2cd40698e0 0x7f2cd4105b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.420+0000 7f2cdca00700 1 -- 172.21.15.1:0/4162450674 >> 172.21.15.1:0/4162450674 conn(0x7f2cd4077bf0 msgr2=0x7f2cd4078000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:03.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.420+0000 7f2cdca00700 1 -- 172.21.15.1:0/4162450674 shutdown_connections 2024-02-02T15:29:03.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.420+0000 7f2cdca00700 1 -- 172.21.15.1:0/4162450674 wait complete. 2024-02-02T15:29:03.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.421+0000 7f2cdca00700 1 Processor -- start 2024-02-02T15:29:03.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.421+0000 7f2cdca00700 1 -- start start 2024-02-02T15:29:03.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.422+0000 7f2cdca00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2cd4068f30 0x7f2cd4197140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:03.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.422+0000 7f2cdca00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2cd4106230 0x7f2cd4197680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:03.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.422+0000 7f2cdca00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2cd419bae0 0x7f2cd419bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:03.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.422+0000 7f2cdca00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd419c560 con 0x7f2cd4106230 2024-02-02T15:29:03.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.422+0000 7f2cdca00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd419c6d0 con 0x7f2cd4068f30 2024-02-02T15:29:03.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.422+0000 7f2cdca00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd419c890 con 0x7f2cd419bae0 2024-02-02T15:29:03.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.422+0000 7f2cd9600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2cd4106230 0x7f2cd4197680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:03.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.422+0000 7f2cdaa00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2cd419bae0 0x7f2cd419bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:03.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.422+0000 7f2cd9600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2cd4106230 0x7f2cd4197680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37768/0 (socket says 172.21.15.1:37768) 2024-02-02T15:29:03.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.422+0000 7f2cdaa00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2cd419bae0 0x7f2cd419bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:54476/0 (socket says 172.21.15.1:54476) 2024-02-02T15:29:03.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.422+0000 7f2cda000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2cd4068f30 0x7f2cd4197140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:03.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.422+0000 7f2cd9600700 1 -- 172.21.15.1:0/3492413855 learned_addr learned my addr 172.21.15.1:0/3492413855 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:29:03.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.422+0000 7f2cda000700 1 -- 172.21.15.1:0/3492413855 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2cd419bae0 msgr2=0x7f2cd419bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:03.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.422+0000 7f2cda000700 1 --2- 172.21.15.1:0/3492413855 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2cd419bae0 0x7f2cd419bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.422+0000 7f2cda000700 1 -- 172.21.15.1:0/3492413855 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2cd4106230 msgr2=0x7f2cd4197680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:03.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.422+0000 7f2cda000700 1 --2- 172.21.15.1:0/3492413855 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2cd4106230 0x7f2cd4197680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.422+0000 7f2cda000700 1 -- 172.21.15.1:0/3492413855 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ccc009d00 con 0x7f2cd4068f30 2024-02-02T15:29:03.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.422+0000 7f2cdaa00700 1 --2- 172.21.15.1:0/3492413855 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2cd419bae0 0x7f2cd419bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:29:03.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.422+0000 7f2cd9600700 1 --2- 172.21.15.1:0/3492413855 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2cd4106230 0x7f2cd4197680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:29:03.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.423+0000 7f2cda000700 1 --2- 172.21.15.1:0/3492413855 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2cd4068f30 0x7f2cd4197140 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2cc400e9f0 tx=0x7f2cc400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:03.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.423+0000 7f2cd2a00700 1 -- 172.21.15.1:0/3492413855 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc400c9c0 con 0x7f2cd4068f30 2024-02-02T15:29:03.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.423+0000 7f2cd2a00700 1 -- 172.21.15.1:0/3492413855 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2cc4010430 con 0x7f2cd4068f30 2024-02-02T15:29:03.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.423+0000 7f2cd2a00700 1 -- 172.21.15.1:0/3492413855 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc4018680 con 0x7f2cd4068f30 2024-02-02T15:29:03.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.423+0000 7f2cdca00700 1 -- 172.21.15.1:0/3492413855 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cd41a3df0 con 0x7f2cd4068f30 2024-02-02T15:29:03.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.423+0000 7f2cdca00700 1 -- 172.21.15.1:0/3492413855 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cd41a4220 con 0x7f2cd4068f30 2024-02-02T15:29:03.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.426+0000 7f2cd2a00700 1 -- 172.21.15.1:0/3492413855 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 27) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f2cc4010860 con 0x7f2cd4068f30 2024-02-02T15:29:03.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.427+0000 7f2cd2a00700 1 --2- 172.21.15.1:0/3492413855 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f2cb40a0690 0x7f2cb40a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:03.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.428+0000 7f2cd2a00700 1 -- 172.21.15.1:0/3492413855 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f2cc4014070 con 0x7f2cd4068f30 2024-02-02T15:29:03.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.428+0000 7f2cd9600700 1 --2- 172.21.15.1:0/3492413855 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f2cb40a0690 0x7f2cb40a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:03.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.428+0000 7f2cf053eb80 1 Processor -- start 2024-02-02T15:29:03.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.428+0000 7f2cf053eb80 1 -- start start 2024-02-02T15:29:03.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.428+0000 7f2cf053eb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a28c55010 0x560a28c553f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:03.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.428+0000 7f2cd9600700 1 --2- 172.21.15.1:0/3492413855 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f2cb40a0690 0x7f2cb40a2b50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2cd4198750 tx=0x7f2cbc009260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:03.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.428+0000 7f2cf053eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560a28c55930 0x560a28c646b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:03.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.428+0000 7f2cf053eb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560a28c64bf0 0x560a28c66fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:03.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.428+0000 7f2cf053eb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a28bc94c0 con 0x560a28c55010 2024-02-02T15:29:03.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.428+0000 7f2cf053eb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a28badaa0 con 0x560a28c64bf0 2024-02-02T15:29:03.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.428+0000 7f2cf053eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a28bab460 con 0x560a28c55930 2024-02-02T15:29:03.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.429+0000 7f2cda000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a28c55010 0x560a28c553f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:03.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.429+0000 7f2cd9600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560a28c55930 0x560a28c646b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:03.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.429+0000 7f2cdaa00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560a28c64bf0 0x560a28c66fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:03.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.429+0000 7f2cd9600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560a28c55930 0x560a28c646b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:54480/0 (socket says 172.21.15.1:54480) 2024-02-02T15:29:03.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.429+0000 7f2cda000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a28c55010 0x560a28c553f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37778/0 (socket says 172.21.15.1:37778) 2024-02-02T15:29:03.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.429+0000 7f2cd9600700 1 -- 172.21.15.1:0/2585095501 learned_addr learned my addr 172.21.15.1:0/2585095501 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:29:03.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.429+0000 7f2cda000700 1 -- 172.21.15.1:0/2585095501 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560a28c55930 msgr2=0x560a28c646b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:03.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.429+0000 7f2cda000700 1 --2- 172.21.15.1:0/2585095501 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560a28c55930 0x560a28c646b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.429+0000 7f2cda000700 1 -- 172.21.15.1:0/2585095501 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560a28c64bf0 msgr2=0x560a28c66fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:03.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.429+0000 7f2cda000700 1 --2- 172.21.15.1:0/2585095501 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560a28c64bf0 0x560a28c66fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.429+0000 7f2cda000700 1 -- 172.21.15.1:0/2585095501 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a28a283b0 con 0x560a28c55010 2024-02-02T15:29:03.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.429+0000 7f2cd9600700 1 --2- 172.21.15.1:0/2585095501 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560a28c55930 0x560a28c646b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:29:03.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.429+0000 7f2cdaa00700 1 --2- 172.21.15.1:0/2585095501 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560a28c64bf0 0x560a28c66fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:29:03.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.429+0000 7f2cda000700 1 --2- 172.21.15.1:0/2585095501 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a28c55010 0x560a28c553f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f2cc40c95a0 tx=0x7f2cc4098cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35af687547290cd5 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:03.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.430+0000 7f2ccb000700 1 -- 172.21.15.1:0/2585095501 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc4099370 con 0x560a28c55010 2024-02-02T15:29:03.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.430+0000 7f2ccb000700 1 -- 172.21.15.1:0/2585095501 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2cc409d1a0 con 0x560a28c55010 2024-02-02T15:29:03.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.430+0000 7f2ccb000700 1 -- 172.21.15.1:0/2585095501 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc4098860 con 0x560a28c55010 2024-02-02T15:29:03.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.430+0000 7f2cf053eb80 1 -- 172.21.15.1:0/2585095501 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a28c55010 msgr2=0x560a28c553f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:03.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.430+0000 7f2cf053eb80 1 --2- 172.21.15.1:0/2585095501 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a28c55010 0x560a28c553f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f2cc40c95a0 tx=0x7f2cc4098cd0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.430+0000 7f2cf053eb80 1 -- 172.21.15.1:0/2585095501 shutdown_connections 2024-02-02T15:29:03.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.430+0000 7f2cf053eb80 1 --2- 172.21.15.1:0/2585095501 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560a28c64bf0 0x560a28c66fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.430+0000 7f2cf053eb80 1 --2- 172.21.15.1:0/2585095501 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a28c55010 0x560a28c553f0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.430+0000 7f2cf053eb80 1 --2- 172.21.15.1:0/2585095501 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560a28c55930 0x560a28c646b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.430+0000 7f2cf053eb80 1 -- 172.21.15.1:0/2585095501 >> 172.21.15.1:0/2585095501 conn(0x560a28bb4db0 msgr2=0x560a28c513b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:03.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.431+0000 7f2cf053eb80 1 -- 172.21.15.1:0/2585095501 shutdown_connections 2024-02-02T15:29:03.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.431+0000 7f2cf053eb80 1 -- 172.21.15.1:0/2585095501 wait complete. 2024-02-02T15:29:03.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.431+0000 7f2cf053eb80 1 -- start start 2024-02-02T15:29:03.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.431+0000 7f2cf053eb80 1 Processor -- start 2024-02-02T15:29:03.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.432+0000 7f2cf053eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560a28c55010 0x560a28c645e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:03.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.432+0000 7f2cf053eb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a28c55930 0x560a28c61860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:03.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.432+0000 7f2cf053eb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560a28c64bf0 0x560a28c61e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:03.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.432+0000 7f2cf053eb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a28ba6d50 con 0x560a28c55930 2024-02-02T15:29:03.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.432+0000 7f2cf053eb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a28b42d80 con 0x560a28c64bf0 2024-02-02T15:29:03.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.432+0000 7f2cf053eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a28b42a20 con 0x560a28c55010 2024-02-02T15:29:03.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.432+0000 7f2cf053eb80 10 client.0 _ll_register_callbacks cb 0x560a28b44290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:29:03.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.432+0000 7f2cda000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560a28c55010 0x560a28c645e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:03.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.432+0000 7f2cda000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560a28c55010 0x560a28c645e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:54482/0 (socket says 172.21.15.1:54482) 2024-02-02T15:29:03.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.432+0000 7f2cda000700 1 -- 172.21.15.1:0/1393284364 learned_addr learned my addr 172.21.15.1:0/1393284364 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:29:03.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.432+0000 7f2cd9600700 1 --2- 172.21.15.1:0/1393284364 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a28c55930 0x560a28c61860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:03.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.432+0000 7f2cdaa00700 1 --2- 172.21.15.1:0/1393284364 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560a28c64bf0 0x560a28c61e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:03.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.432+0000 7f2cda000700 1 -- 172.21.15.1:0/1393284364 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560a28c64bf0 msgr2=0x560a28c61e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:03.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.432+0000 7f2cda000700 1 --2- 172.21.15.1:0/1393284364 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560a28c64bf0 0x560a28c61e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.432+0000 7f2cda000700 1 -- 172.21.15.1:0/1393284364 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a28c55930 msgr2=0x560a28c61860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:03.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.432+0000 7f2cda000700 1 --2- 172.21.15.1:0/1393284364 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a28c55930 0x560a28c61860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.432+0000 7f2cda000700 1 -- 172.21.15.1:0/1393284364 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cd410e640 con 0x560a28c55010 2024-02-02T15:29:03.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.433+0000 7f2cda000700 1 --2- 172.21.15.1:0/1393284364 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560a28c55010 0x560a28c645e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2cc4099070 tx=0x7f2cc40107d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4283a3de971fa08 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:03.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.433+0000 7f2cc3400700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:29:03.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.433+0000 7f2cc3400700 1 -- 172.21.15.1:0/1393284364 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc40a13e0 con 0x560a28c55010 2024-02-02T15:29:03.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.433+0000 7f2cc3400700 1 -- 172.21.15.1:0/1393284364 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f2cc4098a70 con 0x560a28c55010 2024-02-02T15:29:03.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.433+0000 7f2cf053eb80 1 -- 172.21.15.1:0/1393284364 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560a28c490f0 con 0x560a28c55010 2024-02-02T15:29:03.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.433+0000 7f2cc3400700 1 -- 172.21.15.1:0/1393284364 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2cc40b7120 con 0x560a28c55010 2024-02-02T15:29:03.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.433+0000 7f2cf053eb80 10 client.26164 fetch_fsmap learned FSMap version 29 2024-02-02T15:29:03.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.433+0000 7f2cf053eb80 1 -- 172.21.15.1:0/1393284364 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x7f2cd41a4220 con 0x560a28c55010 2024-02-02T15:29:03.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.433+0000 7f2cc3400700 1 -- 172.21.15.1:0/1393284364 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2cc4063020 con 0x560a28c55010 2024-02-02T15:29:03.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.434+0000 7f2cc3400700 1 -- 172.21.15.1:0/1393284364 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 29) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f2cc40a0fe0 con 0x560a28c55010 2024-02-02T15:29:03.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.434+0000 7f2cf053eb80 10 client.26164 fetch_fsmap finished waiting for FSMap version 29 2024-02-02T15:29:03.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.434+0000 7f2cf053eb80 10 client.26164 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-02-02T15:29:03.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.434+0000 7f2cf053eb80 20 client.26164 populate_metadata read hostname 'smithi001' 2024-02-02T15:29:03.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.434+0000 7f2cf053eb80 1 --2- 172.21.15.1:0/1393284364 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x560a28c74110 0x560a28c76530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:03.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.434+0000 7f2cf053eb80 4 client.26164 mds_command: new command op to 24514 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:29:03.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.434+0000 7f2cf053eb80 1 -- 172.21.15.1:0/1393284364 --> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560a28b9fdd0 con 0x560a28c74110 2024-02-02T15:29:03.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.434+0000 7f2cd9600700 1 --2- 172.21.15.1:0/1393284364 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x560a28c74110 0x560a28c76530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:03.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.434+0000 7f2cd9600700 1 --2- 172.21.15.1:0/1393284364 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x560a28c74110 0x560a28c76530 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=90d387669b70f73a server_cookie=9798f9f61a238a27 in_seq=0 out_seq=0 2024-02-02T15:29:03.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.434+0000 7f2cc3400700 10 client.26164 ms_handle_connect on v2:172.21.15.142:6836/2870289019 2024-02-02T15:29:03.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.435+0000 7f2cc3400700 1 -- 172.21.15.1:0/1393284364 <== mds.3 v2:172.21.15.142:6836/2870289019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x560a28b9fdd0 con 0x560a28c74110 2024-02-02T15:29:03.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.435+0000 7f2cc3400700 10 client.26164 handle_command_reply: tid=0 2024-02-02T15:29:03.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.435+0000 7f2cf053eb80 1 client.26164 shutdown 2024-02-02T15:29:03.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.435+0000 7f2cf053eb80 1 -- 172.21.15.1:0/1393284364 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560a28c55010 msgr2=0x560a28c645e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:03.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.435+0000 7f2cf053eb80 1 --2- 172.21.15.1:0/1393284364 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560a28c55010 0x560a28c645e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2cc4099070 tx=0x7f2cc40107d0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.435+0000 7f2cf053eb80 1 -- 172.21.15.1:0/1393284364 shutdown_connections 2024-02-02T15:29:03.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.435+0000 7f2cf053eb80 1 --2- 172.21.15.1:0/1393284364 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560a28c64bf0 0x560a28c61e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.435+0000 7f2cf053eb80 1 --2- 172.21.15.1:0/1393284364 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560a28c55010 0x560a28c645e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.435+0000 7f2cf053eb80 1 --2- 172.21.15.1:0/1393284364 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x560a28c74110 0x560a28c76530 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.435+0000 7f2cf053eb80 1 --2- 172.21.15.1:0/1393284364 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a28c55930 0x560a28c61860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.435+0000 7f2cf053eb80 1 -- 172.21.15.1:0/1393284364 >> 172.21.15.1:0/1393284364 conn(0x560a28bb4db0 msgr2=0x560a28c50fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:03.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.435+0000 7f2cc3400700 0 client.26164 ms_handle_reset on v2:172.21.15.142:6836/2870289019 2024-02-02T15:29:03.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.436+0000 7f2cf053eb80 1 -- 172.21.15.1:0/1393284364 shutdown_connections 2024-02-02T15:29:03.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.436+0000 7f2cf053eb80 1 -- 172.21.15.1:0/1393284364 wait complete. 2024-02-02T15:29:03.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.436+0000 7f2cf053eb80 20 client.26164 trim_cache size 0 max 16384 2024-02-02T15:29:03.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:29:03 smithi001 ceph-mon[35470]: pgmap v64: 129 pgs: 129 active+clean; 52 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 2.7 MiB/s wr, 191 op/s 2024-02-02T15:29:03.443 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:29:03 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/3797650113' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:29:03.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.449+0000 7f2cf053eb80 1 Processor -- start 2024-02-02T15:29:03.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.450+0000 7f2cf053eb80 1 -- start start 2024-02-02T15:29:03.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.450+0000 7f2cf053eb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560a28c55010 0x560a28c6b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:03.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.450+0000 7f2cf053eb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a28c55930 0x560a28c64110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:03.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.450+0000 7f2cf053eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560a28c64bf0 0x560a28c6fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:03.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.450+0000 7f2cf053eb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a28a2a6a0 con 0x560a28c55930 2024-02-02T15:29:03.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.450+0000 7f2cf053eb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a28c521e0 con 0x560a28c55010 2024-02-02T15:29:03.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.450+0000 7f2cf053eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a28c5fe10 con 0x560a28c64bf0 2024-02-02T15:29:03.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.450+0000 7f2cda000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560a28c55010 0x560a28c6b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:03.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.450+0000 7f2cda000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560a28c55010 0x560a28c6b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:45632/0 (socket says 172.21.15.1:45632) 2024-02-02T15:29:03.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.450+0000 7f2cda000700 1 -- 172.21.15.1:0/2032921347 learned_addr learned my addr 172.21.15.1:0/2032921347 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:29:03.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.450+0000 7f2cd9600700 1 --2- 172.21.15.1:0/2032921347 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a28c55930 0x560a28c64110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:03.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.450+0000 7f2cdaa00700 1 --2- 172.21.15.1:0/2032921347 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560a28c64bf0 0x560a28c6fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:03.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.451+0000 7f2cda000700 1 -- 172.21.15.1:0/2032921347 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560a28c64bf0 msgr2=0x560a28c6fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:03.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.451+0000 7f2cda000700 1 --2- 172.21.15.1:0/2032921347 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560a28c64bf0 0x560a28c6fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.451+0000 7f2cda000700 1 -- 172.21.15.1:0/2032921347 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a28c55930 msgr2=0x560a28c64110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:03.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.451+0000 7f2cda000700 1 --2- 172.21.15.1:0/2032921347 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a28c55930 0x560a28c64110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.451+0000 7f2cda000700 1 -- 172.21.15.1:0/2032921347 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a289d4b60 con 0x560a28c55010 2024-02-02T15:29:03.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.451+0000 7f2cda000700 1 --2- 172.21.15.1:0/2032921347 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560a28c55010 0x560a28c6b740 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2cc4099300 tx=0x7f2cc4098f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a110bb949ed5e8d4 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:03.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.451+0000 7f2ccba00700 1 -- 172.21.15.1:0/2032921347 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc4082c10 con 0x560a28c55010 2024-02-02T15:29:03.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.451+0000 7f2ccba00700 1 -- 172.21.15.1:0/2032921347 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2cc4063020 con 0x560a28c55010 2024-02-02T15:29:03.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.451+0000 7f2ccba00700 1 -- 172.21.15.1:0/2032921347 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc40995d0 con 0x560a28c55010 2024-02-02T15:29:03.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.452+0000 7f2cf053eb80 1 -- 172.21.15.1:0/2032921347 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560a28c55010 msgr2=0x560a28c6b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:03.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.452+0000 7f2cf053eb80 1 --2- 172.21.15.1:0/2032921347 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560a28c55010 0x560a28c6b740 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2cc4099300 tx=0x7f2cc4098f60 comp rx=0 tx=0).stop 2024-02-02T15:29:03.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.452+0000 7f2cf053eb80 1 -- 172.21.15.1:0/2032921347 shutdown_connections 2024-02-02T15:29:03.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.452+0000 7f2cf053eb80 1 --2- 172.21.15.1:0/2032921347 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560a28c64bf0 0x560a28c6fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.452+0000 7f2cf053eb80 1 --2- 172.21.15.1:0/2032921347 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560a28c55010 0x560a28c6b740 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.452+0000 7f2cf053eb80 1 --2- 172.21.15.1:0/2032921347 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a28c55930 0x560a28c64110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.452+0000 7f2cf053eb80 1 -- 172.21.15.1:0/2032921347 >> 172.21.15.1:0/2032921347 conn(0x560a28bb4db0 msgr2=0x560a28c5cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:03.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.452+0000 7f2cf053eb80 1 -- 172.21.15.1:0/2032921347 shutdown_connections 2024-02-02T15:29:03.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.452+0000 7f2cf053eb80 1 -- 172.21.15.1:0/2032921347 wait complete. 2024-02-02T15:29:03.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.452+0000 7f2cf053eb80 1 -- start start 2024-02-02T15:29:03.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.453+0000 7f2cf053eb80 1 Processor -- start 2024-02-02T15:29:03.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.453+0000 7f2cf053eb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560a28c55010 0x560a28c68dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:03.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.453+0000 7f2cf053eb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a28c55930 0x560a28c69300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:03.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.453+0000 7f2cf053eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560a28c64bf0 0x560a28c69840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:03.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.453+0000 7f2cf053eb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a28cc4d10 con 0x560a28c55930 2024-02-02T15:29:03.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.453+0000 7f2cf053eb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a28cc4e80 con 0x560a28c55010 2024-02-02T15:29:03.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.453+0000 7f2cf053eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a28cc5040 con 0x560a28c64bf0 2024-02-02T15:29:03.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.453+0000 7f2cf053eb80 10 client.0 _ll_register_callbacks cb 0x560a28b44290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:29:03.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.453+0000 7f2cda000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560a28c55010 0x560a28c68dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:03.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.453+0000 7f2cda000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560a28c55010 0x560a28c68dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:45648/0 (socket says 172.21.15.1:45648) 2024-02-02T15:29:03.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.453+0000 7f2cda000700 1 -- 172.21.15.1:0/3905459794 learned_addr learned my addr 172.21.15.1:0/3905459794 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:29:03.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.453+0000 7f2cdaa00700 1 --2- 172.21.15.1:0/3905459794 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560a28c64bf0 0x560a28c69840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:03.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.453+0000 7f2cd9600700 1 --2- 172.21.15.1:0/3905459794 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a28c55930 0x560a28c69300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:03.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.454+0000 7f2cda000700 1 -- 172.21.15.1:0/3905459794 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560a28c64bf0 msgr2=0x560a28c69840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:03.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.454+0000 7f2cda000700 1 --2- 172.21.15.1:0/3905459794 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560a28c64bf0 0x560a28c69840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.454+0000 7f2cda000700 1 -- 172.21.15.1:0/3905459794 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a28c55930 msgr2=0x560a28c69300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:03.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.454+0000 7f2cda000700 1 --2- 172.21.15.1:0/3905459794 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a28c55930 0x560a28c69300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.454+0000 7f2cda000700 1 -- 172.21.15.1:0/3905459794 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cd41a4220 con 0x560a28c55010 2024-02-02T15:29:03.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.454+0000 7f2cda000700 1 --2- 172.21.15.1:0/3905459794 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560a28c55010 0x560a28c68dc0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2cc4095de0 tx=0x7f2cc4098cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba9b54236cf78b1 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:03.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.454+0000 7f2cc3400700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:29:03.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.454+0000 7f2cc3400700 1 -- 172.21.15.1:0/3905459794 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc4082c10 con 0x560a28c55010 2024-02-02T15:29:03.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.454+0000 7f2cc3400700 1 -- 172.21.15.1:0/3905459794 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f2cc4064080 con 0x560a28c55010 2024-02-02T15:29:03.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.454+0000 7f2cf053eb80 1 -- 172.21.15.1:0/3905459794 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2cc409d1a0 con 0x560a28c55010 2024-02-02T15:29:03.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.455+0000 7f2cc3400700 1 -- 172.21.15.1:0/3905459794 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2cc40a13e0 con 0x560a28c55010 2024-02-02T15:29:03.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.455+0000 7f2cf053eb80 10 client.26180 fetch_fsmap learned FSMap version 29 2024-02-02T15:29:03.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.455+0000 7f2cf053eb80 1 -- 172.21.15.1:0/3905459794 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x560a28b6cea0 con 0x560a28c55010 2024-02-02T15:29:03.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.455+0000 7f2cc3400700 1 -- 172.21.15.1:0/3905459794 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2cc409d1a0 con 0x560a28c55010 2024-02-02T15:29:03.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.455+0000 7f2cc3400700 1 -- 172.21.15.1:0/3905459794 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 29) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f2cc40951f0 con 0x560a28c55010 2024-02-02T15:29:03.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.455+0000 7f2cf053eb80 10 client.26180 fetch_fsmap finished waiting for FSMap version 29 2024-02-02T15:29:03.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.455+0000 7f2cf053eb80 10 client.26180 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-02-02T15:29:03.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.455+0000 7f2cf053eb80 20 client.26180 populate_metadata read hostname 'smithi001' 2024-02-02T15:29:03.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.455+0000 7f2cf053eb80 1 --2- 172.21.15.1:0/3905459794 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x560a28cc6270 0x560a28cc8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:03.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.455+0000 7f2cf053eb80 4 client.26180 mds_command: new command op to 24514 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:29:03.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.455+0000 7f2cf053eb80 1 -- 172.21.15.1:0/3905459794 --> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560a28b65f50 con 0x560a28cc6270 2024-02-02T15:29:03.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.455+0000 7f2cd9600700 1 --2- 172.21.15.1:0/3905459794 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x560a28cc6270 0x560a28cc8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:03.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.456+0000 7f2cd9600700 1 --2- 172.21.15.1:0/3905459794 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x560a28cc6270 0x560a28cc8660 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5382cc641268a3d server_cookie=3c864b63b766b84a in_seq=0 out_seq=0 2024-02-02T15:29:03.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.456+0000 7f2cc3400700 10 client.26180 ms_handle_connect on v2:172.21.15.142:6836/2870289019 2024-02-02T15:29:03.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.456+0000 7f2cc3400700 1 -- 172.21.15.1:0/3905459794 <== mds.3 v2:172.21.15.142:6836/2870289019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560a28b65f50 con 0x560a28cc6270 2024-02-02T15:29:03.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.456+0000 7f2cc3400700 10 client.26180 handle_command_reply: tid=0 2024-02-02T15:29:03.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.456+0000 7f2cf053eb80 1 client.26180 shutdown 2024-02-02T15:29:03.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.456+0000 7f2cf053eb80 1 -- 172.21.15.1:0/3905459794 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560a28c55010 msgr2=0x560a28c68dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:03.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.456+0000 7f2cf053eb80 1 --2- 172.21.15.1:0/3905459794 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560a28c55010 0x560a28c68dc0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2cc4095de0 tx=0x7f2cc4098cd0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.456+0000 7f2cf053eb80 1 -- 172.21.15.1:0/3905459794 shutdown_connections 2024-02-02T15:29:03.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.456+0000 7f2cf053eb80 1 --2- 172.21.15.1:0/3905459794 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560a28c64bf0 0x560a28c69840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.456+0000 7f2cf053eb80 1 --2- 172.21.15.1:0/3905459794 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560a28c55010 0x560a28c68dc0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.456+0000 7f2cf053eb80 1 --2- 172.21.15.1:0/3905459794 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x560a28cc6270 0x560a28cc8660 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.456+0000 7f2cf053eb80 1 --2- 172.21.15.1:0/3905459794 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560a28c55930 0x560a28c69300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.456+0000 7f2cf053eb80 1 -- 172.21.15.1:0/3905459794 >> 172.21.15.1:0/3905459794 conn(0x560a28bb4db0 msgr2=0x560a28c6fd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:03.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.456+0000 7f2cc3400700 0 client.26180 ms_handle_reset on v2:172.21.15.142:6836/2870289019 2024-02-02T15:29:03.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.457+0000 7f2cf053eb80 1 -- 172.21.15.1:0/3905459794 shutdown_connections 2024-02-02T15:29:03.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.457+0000 7f2cf053eb80 1 -- 172.21.15.1:0/3905459794 wait complete. 2024-02-02T15:29:03.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.457+0000 7f2cf053eb80 20 client.26180 trim_cache size 0 max 16384 2024-02-02T15:29:03.463 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:29:03.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.457+0000 7f2cdca00700 1 -- 172.21.15.1:0/3492413855 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f2cb40a0690 msgr2=0x7f2cb40a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:03.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.457+0000 7f2cdca00700 1 --2- 172.21.15.1:0/3492413855 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f2cb40a0690 0x7f2cb40a2b50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2cd4198750 tx=0x7f2cbc009260 comp rx=0 tx=0).stop 2024-02-02T15:29:03.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.457+0000 7f2cdca00700 1 -- 172.21.15.1:0/3492413855 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2cd4068f30 msgr2=0x7f2cd4197140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:03.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.457+0000 7f2cdca00700 1 --2- 172.21.15.1:0/3492413855 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2cd4068f30 0x7f2cd4197140 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2cc400e9f0 tx=0x7f2cc400ed00 comp rx=0 tx=0).stop 2024-02-02T15:29:03.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.457+0000 7f2cdca00700 1 -- 172.21.15.1:0/3492413855 shutdown_connections 2024-02-02T15:29:03.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.457+0000 7f2cdca00700 1 --2- 172.21.15.1:0/3492413855 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f2cb40a0690 0x7f2cb40a2b50 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.457+0000 7f2cdca00700 1 --2- 172.21.15.1:0/3492413855 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2cd419bae0 0x7f2cd419bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.457+0000 7f2cdca00700 1 --2- 172.21.15.1:0/3492413855 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2cd4068f30 0x7f2cd4197140 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.457+0000 7f2cdca00700 1 --2- 172.21.15.1:0/3492413855 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2cd4106230 0x7f2cd4197680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.457+0000 7f2cdca00700 1 -- 172.21.15.1:0/3492413855 >> 172.21.15.1:0/3492413855 conn(0x7f2cd4077bf0 msgr2=0x7f2cd40fdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:03.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.458+0000 7f2cdca00700 1 -- 172.21.15.1:0/3492413855 shutdown_connections 2024-02-02T15:29:03.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.458+0000 7f2cdca00700 1 -- 172.21.15.1:0/3492413855 wait complete. 2024-02-02T15:29:03.476 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-02T15:29:03.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:29:03 smithi071 ceph-mon[44707]: pgmap v64: 129 pgs: 129 active+clean; 52 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 2.7 MiB/s wr, 191 op/s 2024-02-02T15:29:03.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:29:03 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/3797650113' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:29:03.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:29:03 smithi142 ceph-mon[45728]: pgmap v64: 129 pgs: 129 active+clean; 52 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 2.7 MiB/s wr, 191 op/s 2024-02-02T15:29:03.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:29:03 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/3797650113' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:29:03.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.728+0000 7f1ddac00700 1 -- 172.21.15.1:0/1134265089 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1dd4102950 msgr2=0x7f1dd410ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:03.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.728+0000 7f1ddac00700 1 --2- 172.21.15.1:0/1134265089 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1dd4102950 0x7f1dd410ae30 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc009990 tx=0x7f1dbc009ca0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.728+0000 7f1ddac00700 1 -- 172.21.15.1:0/1134265089 shutdown_connections 2024-02-02T15:29:03.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.728+0000 7f1ddac00700 1 --2- 172.21.15.1:0/1134265089 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1dd410b400 0x7f1dd410d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.728+0000 7f1ddac00700 1 --2- 172.21.15.1:0/1134265089 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1dd4102030 0x7f1dd4102410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.728+0000 7f1ddac00700 1 --2- 172.21.15.1:0/1134265089 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1dd4102950 0x7f1dd410ae30 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.728+0000 7f1ddac00700 1 -- 172.21.15.1:0/1134265089 >> 172.21.15.1:0/1134265089 conn(0x7f1dd4076d70 msgr2=0x7f1dd4077180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:03.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.729+0000 7f1ddac00700 1 -- 172.21.15.1:0/1134265089 shutdown_connections 2024-02-02T15:29:03.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.729+0000 7f1ddac00700 1 -- 172.21.15.1:0/1134265089 wait complete. 2024-02-02T15:29:03.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.730+0000 7f1ddac00700 1 Processor -- start 2024-02-02T15:29:03.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.730+0000 7f1ddac00700 1 -- start start 2024-02-02T15:29:03.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.730+0000 7f1ddac00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1dd4102030 0x7f1dd41990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:03.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.730+0000 7f1ddac00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1dd4102950 0x7f1dd4199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:03.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.730+0000 7f1ddac00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1dd410b400 0x7f1dd419da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:03.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.730+0000 7f1ddac00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd419e0b0 con 0x7f1dd410b400 2024-02-02T15:29:03.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.730+0000 7f1ddac00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd419e220 con 0x7f1dd4102030 2024-02-02T15:29:03.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.730+0000 7f1ddac00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd419e3e0 con 0x7f1dd4102950 2024-02-02T15:29:03.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.730+0000 7f1dd3e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1dd4102030 0x7f1dd41990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:03.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.730+0000 7f1dd3e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1dd4102030 0x7f1dd41990d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:45658/0 (socket says 172.21.15.1:45658) 2024-02-02T15:29:03.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.731+0000 7f1dd3e00700 1 -- 172.21.15.1:0/3504534144 learned_addr learned my addr 172.21.15.1:0/3504534144 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:29:03.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.731+0000 7f1dd3400700 1 --2- 172.21.15.1:0/3504534144 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1dd4102950 0x7f1dd4199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:03.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.731+0000 7f1dd8c00700 1 --2- 172.21.15.1:0/3504534144 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1dd410b400 0x7f1dd419da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:03.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.731+0000 7f1dd3e00700 1 -- 172.21.15.1:0/3504534144 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1dd4102950 msgr2=0x7f1dd4199610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:03.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.731+0000 7f1dd3e00700 1 --2- 172.21.15.1:0/3504534144 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1dd4102950 0x7f1dd4199610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.731+0000 7f1dd3e00700 1 -- 172.21.15.1:0/3504534144 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1dd410b400 msgr2=0x7f1dd419da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:03.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.731+0000 7f1dd3e00700 1 --2- 172.21.15.1:0/3504534144 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1dd410b400 0x7f1dd419da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.731+0000 7f1dd3e00700 1 -- 172.21.15.1:0/3504534144 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dbc0095f0 con 0x7f1dd4102030 2024-02-02T15:29:03.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.731+0000 7f1dd8c00700 1 --2- 172.21.15.1:0/3504534144 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1dd410b400 0x7f1dd419da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:29:03.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.731+0000 7f1dd3400700 1 --2- 172.21.15.1:0/3504534144 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1dd4102950 0x7f1dd4199610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:29:03.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.731+0000 7f1dd3e00700 1 --2- 172.21.15.1:0/3504534144 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1dd4102030 0x7f1dd41990d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1dc400e9a0 tx=0x7f1dc400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:03.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.731+0000 7f1dd0c00700 1 -- 172.21.15.1:0/3504534144 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc4009790 con 0x7f1dd4102030 2024-02-02T15:29:03.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.732+0000 7f1ddac00700 1 -- 172.21.15.1:0/3504534144 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dd419e6c0 con 0x7f1dd4102030 2024-02-02T15:29:03.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.732+0000 7f1dd0c00700 1 -- 172.21.15.1:0/3504534144 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1dc400cbd0 con 0x7f1dd4102030 2024-02-02T15:29:03.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.732+0000 7f1dd0c00700 1 -- 172.21.15.1:0/3504534144 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc4018680 con 0x7f1dd4102030 2024-02-02T15:29:03.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.732+0000 7f1ddac00700 1 -- 172.21.15.1:0/3504534144 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dd41a5f00 con 0x7f1dd4102030 2024-02-02T15:29:03.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.735+0000 7f1dd0c00700 1 -- 172.21.15.1:0/3504534144 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 27) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f1dc400cd40 con 0x7f1dd4102030 2024-02-02T15:29:03.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.736+0000 7f1dd0c00700 1 --2- 172.21.15.1:0/3504534144 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f1db40a0710 0x7f1db40a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:03.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.736+0000 7f1dee7e3b80 1 Processor -- start 2024-02-02T15:29:03.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.736+0000 7f1dd0c00700 1 -- 172.21.15.1:0/3504534144 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f1dc4014070 con 0x7f1dd4102030 2024-02-02T15:29:03.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.736+0000 7f1dee7e3b80 1 -- start start 2024-02-02T15:29:03.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.737+0000 7f1dd3400700 1 --2- 172.21.15.1:0/3504534144 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f1db40a0710 0x7f1db40a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:03.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.737+0000 7f1dee7e3b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565239ed3010 0x565239ed33f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:03.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.737+0000 7f1dee7e3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565239ed3930 0x565239ede2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:03.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.737+0000 7f1dee7e3b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565239ede7e0 0x565239ee0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:03.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.737+0000 7f1dee7e3b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x565239e474c0 con 0x565239ed3010 2024-02-02T15:29:03.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.737+0000 7f1dee7e3b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x565239e2baa0 con 0x565239ede7e0 2024-02-02T15:29:03.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.737+0000 7f1dee7e3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x565239e29460 con 0x565239ed3930 2024-02-02T15:29:03.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.737+0000 7f1dd3e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565239ed3010 0x565239ed33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:03.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.737+0000 7f1dd8c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565239ede7e0 0x565239ee0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:03.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.737+0000 7f1dd8c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565239ede7e0 0x565239ee0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:45672/0 (socket says 172.21.15.1:45672) 2024-02-02T15:29:03.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.737+0000 7f1dd3400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565239ed3930 0x565239ede2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:03.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.737+0000 7f1dd3e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565239ed3010 0x565239ed33f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37838/0 (socket says 172.21.15.1:37838) 2024-02-02T15:29:03.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.737+0000 7f1dd8c00700 1 -- 172.21.15.1:0/1205538436 learned_addr learned my addr 172.21.15.1:0/1205538436 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:29:03.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.737+0000 7f1dd3400700 1 --2- 172.21.15.1:0/3504534144 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f1db40a0710 0x7f1db40a2bd0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1dd419a6e0 tx=0x7f1dbc005fd0 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:03.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.738+0000 7f1dd3e00700 1 -- 172.21.15.1:0/1205538436 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565239ed3930 msgr2=0x565239ede2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:03.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.738+0000 7f1dd3e00700 1 --2- 172.21.15.1:0/1205538436 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565239ed3930 0x565239ede2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.738+0000 7f1dd3e00700 1 -- 172.21.15.1:0/1205538436 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565239ede7e0 msgr2=0x565239ee0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:03.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.738+0000 7f1dd3e00700 1 --2- 172.21.15.1:0/1205538436 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565239ede7e0 0x565239ee0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.738+0000 7f1dd3e00700 1 -- 172.21.15.1:0/1205538436 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565239ca63b0 con 0x565239ed3010 2024-02-02T15:29:03.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.738+0000 7f1dd3e00700 1 --2- 172.21.15.1:0/1205538436 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565239ed3010 0x565239ed33f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f1dc40c96e0 tx=0x7f1dc4095920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a791b345fef808c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:03.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.738+0000 7f1dc9200700 1 -- 172.21.15.1:0/1205538436 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc409d120 con 0x565239ed3010 2024-02-02T15:29:03.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.738+0000 7f1dc9200700 1 -- 172.21.15.1:0/1205538436 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1dc40cc170 con 0x565239ed3010 2024-02-02T15:29:03.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.738+0000 7f1dc9200700 1 -- 172.21.15.1:0/1205538436 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc40999c0 con 0x565239ed3010 2024-02-02T15:29:03.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.739+0000 7f1dee7e3b80 1 -- 172.21.15.1:0/1205538436 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565239ed3010 msgr2=0x565239ed33f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:03.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.739+0000 7f1dee7e3b80 1 --2- 172.21.15.1:0/1205538436 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565239ed3010 0x565239ed33f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f1dc40c96e0 tx=0x7f1dc4095920 comp rx=0 tx=0).stop 2024-02-02T15:29:03.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.739+0000 7f1dee7e3b80 1 -- 172.21.15.1:0/1205538436 shutdown_connections 2024-02-02T15:29:03.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.739+0000 7f1dee7e3b80 1 --2- 172.21.15.1:0/1205538436 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565239ede7e0 0x565239ee0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.739+0000 7f1dee7e3b80 1 --2- 172.21.15.1:0/1205538436 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565239ed3010 0x565239ed33f0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.739+0000 7f1dee7e3b80 1 --2- 172.21.15.1:0/1205538436 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565239ed3930 0x565239ede2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.739+0000 7f1dee7e3b80 1 -- 172.21.15.1:0/1205538436 >> 172.21.15.1:0/1205538436 conn(0x565239e32db0 msgr2=0x565239ecf3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:03.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.739+0000 7f1dee7e3b80 1 -- 172.21.15.1:0/1205538436 shutdown_connections 2024-02-02T15:29:03.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.739+0000 7f1dee7e3b80 1 -- 172.21.15.1:0/1205538436 wait complete. 2024-02-02T15:29:03.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.740+0000 7f1dee7e3b80 1 -- start start 2024-02-02T15:29:03.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.740+0000 7f1dee7e3b80 1 Processor -- start 2024-02-02T15:29:03.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.740+0000 7f1dee7e3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565239ed3010 0x565239ede1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:03.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.741+0000 7f1dee7e3b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565239ed3930 0x565239edb450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:03.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.741+0000 7f1dee7e3b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565239ede7e0 0x565239edba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:03.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.741+0000 7f1dee7e3b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x565239e24d50 con 0x565239ede7e0 2024-02-02T15:29:03.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.741+0000 7f1dee7e3b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x565239dc0d80 con 0x565239ed3930 2024-02-02T15:29:03.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.741+0000 7f1dee7e3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x565239dc0a20 con 0x565239ed3010 2024-02-02T15:29:03.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.741+0000 7f1dee7e3b80 10 client.0 _ll_register_callbacks cb 0x565239dc2290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:29:03.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.741+0000 7f1dd3400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565239ed3930 0x565239edb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:03.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.741+0000 7f1dd3e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565239ed3010 0x565239ede1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:03.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.741+0000 7f1dd3400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565239ed3930 0x565239edb450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:45680/0 (socket says 172.21.15.1:45680) 2024-02-02T15:29:03.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.741+0000 7f1dd3400700 1 -- 172.21.15.1:0/1308118045 learned_addr learned my addr 172.21.15.1:0/1308118045 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:29:03.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.741+0000 7f1dd8c00700 1 --2- 172.21.15.1:0/1308118045 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565239ede7e0 0x565239edba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:03.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.741+0000 7f1dd3e00700 1 -- 172.21.15.1:0/1308118045 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565239ed3930 msgr2=0x565239edb450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:03.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.741+0000 7f1dd3e00700 1 --2- 172.21.15.1:0/1308118045 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565239ed3930 0x565239edb450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.741+0000 7f1dd3e00700 1 -- 172.21.15.1:0/1308118045 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565239ede7e0 msgr2=0x565239edba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:03.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.741+0000 7f1dd3e00700 1 --2- 172.21.15.1:0/1308118045 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565239ede7e0 0x565239edba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.741+0000 7f1dd3e00700 1 -- 172.21.15.1:0/1308118045 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dd41a5f00 con 0x565239ed3010 2024-02-02T15:29:03.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.741+0000 7f1dd3400700 1 --2- 172.21.15.1:0/1308118045 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565239ed3930 0x565239edb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:29:03.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.742+0000 7f1dd3e00700 1 --2- 172.21.15.1:0/1308118045 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565239ed3010 0x565239ede1d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1dc40c9710 tx=0x7f1dc409d120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab757410aa5abb55 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:03.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.742+0000 7f1dc1600700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:29:03.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.742+0000 7f1dc1600700 1 -- 172.21.15.1:0/1308118045 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc4099910 con 0x565239ed3010 2024-02-02T15:29:03.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.742+0000 7f1dc1600700 1 -- 172.21.15.1:0/1308118045 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f1dc40a1520 con 0x565239ed3010 2024-02-02T15:29:03.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.742+0000 7f1dee7e3b80 1 -- 172.21.15.1:0/1308118045 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x565239ec70f0 con 0x565239ed3010 2024-02-02T15:29:03.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.742+0000 7f1dc1600700 1 -- 172.21.15.1:0/1308118045 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1dc40a1090 con 0x565239ed3010 2024-02-02T15:29:03.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.742+0000 7f1dee7e3b80 10 client.26182 fetch_fsmap learned FSMap version 29 2024-02-02T15:29:03.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.742+0000 7f1dee7e3b80 1 -- 172.21.15.1:0/1308118045 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x565239c52b60 con 0x565239ed3010 2024-02-02T15:29:03.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.742+0000 7f1dc1600700 1 -- 172.21.15.1:0/1308118045 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x565239ec70f0 con 0x565239ed3010 2024-02-02T15:29:03.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.742+0000 7f1dc1600700 1 -- 172.21.15.1:0/1308118045 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 29) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f1dc40954a0 con 0x565239ed3010 2024-02-02T15:29:03.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.743+0000 7f1dee7e3b80 10 client.26182 fetch_fsmap finished waiting for FSMap version 29 2024-02-02T15:29:03.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.743+0000 7f1dee7e3b80 10 client.26182 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-02-02T15:29:03.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.743+0000 7f1dee7e3b80 20 client.26182 populate_metadata read hostname 'smithi001' 2024-02-02T15:29:03.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.743+0000 7f1dee7e3b80 1 --2- 172.21.15.1:0/1308118045 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x565239eedda0 0x565239ef0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:03.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.743+0000 7f1dee7e3b80 4 client.26182 mds_command: new command op to 24518 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:29:03.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.743+0000 7f1dee7e3b80 1 -- 172.21.15.1:0/1308118045 --> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x565239e1ddd0 con 0x565239eedda0 2024-02-02T15:29:03.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.743+0000 7f1dd3400700 1 --2- 172.21.15.1:0/1308118045 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x565239eedda0 0x565239ef0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:03.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.743+0000 7f1dd3400700 1 --2- 172.21.15.1:0/1308118045 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x565239eedda0 0x565239ef0190 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8523436587b0bff3 server_cookie=3a5e4fdea222f2a2 in_seq=0 out_seq=0 2024-02-02T15:29:03.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.743+0000 7f1dc1600700 10 client.26182 ms_handle_connect on v2:172.21.15.71:6836/628016199 2024-02-02T15:29:03.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.744+0000 7f1dc1600700 1 -- 172.21.15.1:0/1308118045 <== mds.2 v2:172.21.15.71:6836/628016199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x565239e1ddd0 con 0x565239eedda0 2024-02-02T15:29:03.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.744+0000 7f1dc1600700 10 client.26182 handle_command_reply: tid=0 2024-02-02T15:29:03.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.744+0000 7f1dee7e3b80 1 client.26182 shutdown 2024-02-02T15:29:03.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.744+0000 7f1dee7e3b80 1 -- 172.21.15.1:0/1308118045 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565239ed3010 msgr2=0x565239ede1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:03.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.744+0000 7f1dee7e3b80 1 --2- 172.21.15.1:0/1308118045 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565239ed3010 0x565239ede1d0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1dc40c9710 tx=0x7f1dc409d120 comp rx=0 tx=0).stop 2024-02-02T15:29:03.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.744+0000 7f1dee7e3b80 1 -- 172.21.15.1:0/1308118045 shutdown_connections 2024-02-02T15:29:03.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.744+0000 7f1dee7e3b80 1 --2- 172.21.15.1:0/1308118045 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565239ede7e0 0x565239edba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.744+0000 7f1dee7e3b80 1 --2- 172.21.15.1:0/1308118045 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x565239eedda0 0x565239ef0190 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.744+0000 7f1dee7e3b80 1 --2- 172.21.15.1:0/1308118045 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565239ed3010 0x565239ede1d0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.744+0000 7f1dee7e3b80 1 --2- 172.21.15.1:0/1308118045 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565239ed3930 0x565239edb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.744+0000 7f1dee7e3b80 1 -- 172.21.15.1:0/1308118045 >> 172.21.15.1:0/1308118045 conn(0x565239e32db0 msgr2=0x565239ecefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:03.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.744+0000 7f1dc1600700 0 client.26182 ms_handle_reset on v2:172.21.15.71:6836/628016199 2024-02-02T15:29:03.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.745+0000 7f1dee7e3b80 1 -- 172.21.15.1:0/1308118045 shutdown_connections 2024-02-02T15:29:03.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.745+0000 7f1dee7e3b80 1 -- 172.21.15.1:0/1308118045 wait complete. 2024-02-02T15:29:03.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.745+0000 7f1dee7e3b80 20 client.26182 trim_cache size 0 max 16384 2024-02-02T15:29:03.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.759+0000 7f1dee7e3b80 1 Processor -- start 2024-02-02T15:29:03.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.759+0000 7f1dee7e3b80 1 -- start start 2024-02-02T15:29:03.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.759+0000 7f1dee7e3b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565239ed3010 0x565239edb420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:03.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.759+0000 7f1dee7e3b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565239ed3930 0x565239ed5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:03.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.759+0000 7f1dee7e3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565239ede7e0 0x565239eddd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:03.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.759+0000 7f1dee7e3b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x565239ca86a0 con 0x565239ed3010 2024-02-02T15:29:03.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.759+0000 7f1dee7e3b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x565239ed01e0 con 0x565239ed3930 2024-02-02T15:29:03.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.759+0000 7f1dee7e3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x565239ed9a00 con 0x565239ede7e0 2024-02-02T15:29:03.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.759+0000 7f1dd3400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565239ed3930 0x565239ed5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:03.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.759+0000 7f1dd3e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565239ed3010 0x565239edb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:03.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.759+0000 7f1dd3400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565239ed3930 0x565239ed5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:45688/0 (socket says 172.21.15.1:45688) 2024-02-02T15:29:03.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.759+0000 7f1dd3400700 1 -- 172.21.15.1:0/2899427908 learned_addr learned my addr 172.21.15.1:0/2899427908 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:29:03.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.759+0000 7f1dd8c00700 1 --2- 172.21.15.1:0/2899427908 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565239ede7e0 0x565239eddd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:03.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.760+0000 7f1dd3400700 1 -- 172.21.15.1:0/2899427908 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565239ede7e0 msgr2=0x565239eddd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:03.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.760+0000 7f1dd3400700 1 --2- 172.21.15.1:0/2899427908 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565239ede7e0 0x565239eddd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.760+0000 7f1dd3400700 1 -- 172.21.15.1:0/2899427908 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565239ed3010 msgr2=0x565239edb420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:03.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.760+0000 7f1dd3400700 1 --2- 172.21.15.1:0/2899427908 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565239ed3010 0x565239edb420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.760+0000 7f1dd3400700 1 -- 172.21.15.1:0/2899427908 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565239deaea0 con 0x565239ed3930 2024-02-02T15:29:03.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.760+0000 7f1dd3400700 1 --2- 172.21.15.1:0/2899427908 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565239ed3930 0x565239ed5ab0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc000c00 tx=0x7f1dbc025000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b890ebf902a76b7c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:03.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.760+0000 7f1dc9c00700 1 -- 172.21.15.1:0/2899427908 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dbc036350 con 0x565239ed3930 2024-02-02T15:29:03.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.760+0000 7f1dc9c00700 1 -- 172.21.15.1:0/2899427908 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1dbc01e040 con 0x565239ed3930 2024-02-02T15:29:03.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.760+0000 7f1dc9c00700 1 -- 172.21.15.1:0/2899427908 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dbc025690 con 0x565239ed3930 2024-02-02T15:29:03.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.761+0000 7f1dee7e3b80 1 -- 172.21.15.1:0/2899427908 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565239ed3930 msgr2=0x565239ed5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:03.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.761+0000 7f1dee7e3b80 1 --2- 172.21.15.1:0/2899427908 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565239ed3930 0x565239ed5ab0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc000c00 tx=0x7f1dbc025000 comp rx=0 tx=0).stop 2024-02-02T15:29:03.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.761+0000 7f1dee7e3b80 1 -- 172.21.15.1:0/2899427908 shutdown_connections 2024-02-02T15:29:03.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.761+0000 7f1dee7e3b80 1 --2- 172.21.15.1:0/2899427908 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565239ede7e0 0x565239eddd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.761+0000 7f1dee7e3b80 1 --2- 172.21.15.1:0/2899427908 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565239ed3010 0x565239edb420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.761+0000 7f1dee7e3b80 1 --2- 172.21.15.1:0/2899427908 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565239ed3930 0x565239ed5ab0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.761+0000 7f1dee7e3b80 1 -- 172.21.15.1:0/2899427908 >> 172.21.15.1:0/2899427908 conn(0x565239e32db0 msgr2=0x565239edad10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:03.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.761+0000 7f1dee7e3b80 1 -- 172.21.15.1:0/2899427908 shutdown_connections 2024-02-02T15:29:03.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.761+0000 7f1dee7e3b80 1 -- 172.21.15.1:0/2899427908 wait complete. 2024-02-02T15:29:03.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.761+0000 7f1dee7e3b80 1 -- start start 2024-02-02T15:29:03.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.762+0000 7f1dee7e3b80 1 Processor -- start 2024-02-02T15:29:03.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.762+0000 7f1dee7e3b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565239ed3010 0x565239ee2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:03.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.762+0000 7f1dee7e3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565239ed3930 0x565239ee3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:03.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.762+0000 7f1dee7e3b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565239ede7e0 0x565239f3e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:03.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.762+0000 7f1dee7e3b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x565239f3edd0 con 0x565239ede7e0 2024-02-02T15:29:03.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.762+0000 7f1dee7e3b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x565239f3ef10 con 0x565239ed3010 2024-02-02T15:29:03.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.762+0000 7f1dee7e3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x565239f3f0d0 con 0x565239ed3930 2024-02-02T15:29:03.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.762+0000 7f1dee7e3b80 10 client.0 _ll_register_callbacks cb 0x565239dc2290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:29:03.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.763+0000 7f1dd3400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565239ed3930 0x565239ee3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:03.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.763+0000 7f1dd3400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565239ed3930 0x565239ee3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:54548/0 (socket says 172.21.15.1:54548) 2024-02-02T15:29:03.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.763+0000 7f1dd3400700 1 -- 172.21.15.1:0/2826891431 learned_addr learned my addr 172.21.15.1:0/2826891431 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:29:03.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.763+0000 7f1dd3e00700 1 --2- 172.21.15.1:0/2826891431 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565239ed3010 0x565239ee2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:03.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.763+0000 7f1dd8c00700 1 --2- 172.21.15.1:0/2826891431 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565239ede7e0 0x565239f3e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:03.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.763+0000 7f1dd3400700 1 -- 172.21.15.1:0/2826891431 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565239ed3010 msgr2=0x565239ee2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:03.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.763+0000 7f1dd3400700 1 --2- 172.21.15.1:0/2826891431 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565239ed3010 0x565239ee2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.763+0000 7f1dd3400700 1 -- 172.21.15.1:0/2826891431 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565239ede7e0 msgr2=0x565239f3e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:03.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.763+0000 7f1dd3400700 1 --2- 172.21.15.1:0/2826891431 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565239ede7e0 0x565239f3e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.763+0000 7f1dd3400700 1 -- 172.21.15.1:0/2826891431 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dd410e560 con 0x565239ed3930 2024-02-02T15:29:03.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.763+0000 7f1dd3400700 1 --2- 172.21.15.1:0/2826891431 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565239ed3930 0x565239ee3520 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc00b6f0 tx=0x7f1dbc005660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=768565868584ee32 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:03.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.763+0000 7f1dc1600700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:29:03.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.763+0000 7f1dc1600700 1 -- 172.21.15.1:0/2826891431 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dbc004780 con 0x565239ed3930 2024-02-02T15:29:03.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.764+0000 7f1dee7e3b80 1 -- 172.21.15.1:0/2826891431 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1dc40cc170 con 0x565239ed3930 2024-02-02T15:29:03.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.764+0000 7f1dc1600700 1 -- 172.21.15.1:0/2826891431 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f1dbc0048e0 con 0x565239ed3930 2024-02-02T15:29:03.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.764+0000 7f1dc1600700 1 -- 172.21.15.1:0/2826891431 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1dbc004aa0 con 0x565239ed3930 2024-02-02T15:29:03.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.764+0000 7f1dee7e3b80 10 client.26188 fetch_fsmap learned FSMap version 29 2024-02-02T15:29:03.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.764+0000 7f1dee7e3b80 1 -- 172.21.15.1:0/2826891431 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x565239caacd0 con 0x565239ed3930 2024-02-02T15:29:03.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.764+0000 7f1dc1600700 1 -- 172.21.15.1:0/2826891431 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1dc40cc170 con 0x565239ed3930 2024-02-02T15:29:03.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.764+0000 7f1dc1600700 1 -- 172.21.15.1:0/2826891431 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 29) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f1dbc00f620 con 0x565239ed3930 2024-02-02T15:29:03.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.765+0000 7f1dee7e3b80 10 client.26188 fetch_fsmap finished waiting for FSMap version 29 2024-02-02T15:29:03.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.765+0000 7f1dee7e3b80 10 client.26188 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-02-02T15:29:03.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.765+0000 7f1dee7e3b80 20 client.26188 populate_metadata read hostname 'smithi001' 2024-02-02T15:29:03.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.765+0000 7f1dee7e3b80 1 --2- 172.21.15.1:0/2826891431 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x565239f40460 0x565239f42880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:03.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.765+0000 7f1dee7e3b80 4 client.26188 mds_command: new command op to 24518 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:29:03.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.765+0000 7f1dee7e3b80 1 -- 172.21.15.1:0/2826891431 --> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x565239de3f50 con 0x565239f40460 2024-02-02T15:29:03.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.765+0000 7f1dd3e00700 1 --2- 172.21.15.1:0/2826891431 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x565239f40460 0x565239f42880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:03.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.765+0000 7f1dd3e00700 1 --2- 172.21.15.1:0/2826891431 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x565239f40460 0x565239f42880 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f6fb4f1dfe250826 server_cookie=9cd6b26b335f01ff in_seq=0 out_seq=0 2024-02-02T15:29:03.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.765+0000 7f1dc1600700 10 client.26188 ms_handle_connect on v2:172.21.15.71:6836/628016199 2024-02-02T15:29:03.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.766+0000 7f1dc1600700 1 -- 172.21.15.1:0/2826891431 <== mds.2 v2:172.21.15.71:6836/628016199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x565239de3f50 con 0x565239f40460 2024-02-02T15:29:03.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.766+0000 7f1dc1600700 10 client.26188 handle_command_reply: tid=0 2024-02-02T15:29:03.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.766+0000 7f1dee7e3b80 1 client.26188 shutdown 2024-02-02T15:29:03.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.766+0000 7f1dee7e3b80 1 -- 172.21.15.1:0/2826891431 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565239ed3930 msgr2=0x565239ee3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:03.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.766+0000 7f1dee7e3b80 1 --2- 172.21.15.1:0/2826891431 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565239ed3930 0x565239ee3520 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc00b6f0 tx=0x7f1dbc005660 comp rx=0 tx=0).stop 2024-02-02T15:29:03.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.766+0000 7f1dee7e3b80 1 -- 172.21.15.1:0/2826891431 shutdown_connections 2024-02-02T15:29:03.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.766+0000 7f1dee7e3b80 1 --2- 172.21.15.1:0/2826891431 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565239ede7e0 0x565239f3e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.766+0000 7f1dee7e3b80 1 --2- 172.21.15.1:0/2826891431 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565239ed3010 0x565239ee2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.766+0000 7f1dee7e3b80 1 --2- 172.21.15.1:0/2826891431 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x565239f40460 0x565239f42880 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.766+0000 7f1dee7e3b80 1 --2- 172.21.15.1:0/2826891431 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565239ed3930 0x565239ee3520 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.766+0000 7f1dc1600700 0 client.26188 ms_handle_reset on v2:172.21.15.71:6836/628016199 2024-02-02T15:29:03.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.766+0000 7f1dee7e3b80 1 -- 172.21.15.1:0/2826891431 >> 172.21.15.1:0/2826891431 conn(0x565239e32db0 msgr2=0x565239ed9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:03.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.766+0000 7f1dee7e3b80 1 -- 172.21.15.1:0/2826891431 shutdown_connections 2024-02-02T15:29:03.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.766+0000 7f1dee7e3b80 1 -- 172.21.15.1:0/2826891431 wait complete. 2024-02-02T15:29:03.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.766+0000 7f1dee7e3b80 20 client.26188 trim_cache size 0 max 16384 2024-02-02T15:29:03.770 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:29:03.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.767+0000 7f1ddac00700 1 -- 172.21.15.1:0/3504534144 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f1db40a0710 msgr2=0x7f1db40a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:03.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.767+0000 7f1ddac00700 1 --2- 172.21.15.1:0/3504534144 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f1db40a0710 0x7f1db40a2bd0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1dd419a6e0 tx=0x7f1dbc005fd0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.767+0000 7f1ddac00700 1 -- 172.21.15.1:0/3504534144 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1dd4102030 msgr2=0x7f1dd41990d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:03.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.767+0000 7f1ddac00700 1 --2- 172.21.15.1:0/3504534144 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1dd4102030 0x7f1dd41990d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1dc400e9a0 tx=0x7f1dc400ecb0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.767+0000 7f1ddac00700 1 -- 172.21.15.1:0/3504534144 shutdown_connections 2024-02-02T15:29:03.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.767+0000 7f1ddac00700 1 --2- 172.21.15.1:0/3504534144 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f1db40a0710 0x7f1db40a2bd0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.767+0000 7f1ddac00700 1 --2- 172.21.15.1:0/3504534144 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1dd410b400 0x7f1dd419da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.767+0000 7f1ddac00700 1 --2- 172.21.15.1:0/3504534144 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1dd4102030 0x7f1dd41990d0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.767+0000 7f1ddac00700 1 --2- 172.21.15.1:0/3504534144 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1dd4102950 0x7f1dd4199610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:03.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.767+0000 7f1ddac00700 1 -- 172.21.15.1:0/3504534144 >> 172.21.15.1:0/3504534144 conn(0x7f1dd4076d70 msgr2=0x7f1dd4108270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:03.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.767+0000 7f1ddac00700 1 -- 172.21.15.1:0/3504534144 shutdown_connections 2024-02-02T15:29:03.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:03.767+0000 7f1ddac00700 1 -- 172.21.15.1:0/3504534144 wait complete. 2024-02-02T15:29:03.786 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-02T15:29:04.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.000+0000 7fe955600700 1 -- 172.21.15.1:0/1537286496 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe950073110 msgr2=0x7fe9500734f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:04.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.000+0000 7fe955600700 1 --2- 172.21.15.1:0/1537286496 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe950073110 0x7fe9500734f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe940009990 tx=0x7fe940009ca0 comp rx=0 tx=0).stop 2024-02-02T15:29:04.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.000+0000 7fe955600700 1 -- 172.21.15.1:0/1537286496 shutdown_connections 2024-02-02T15:29:04.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.000+0000 7fe955600700 1 --2- 172.21.15.1:0/1537286496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe95010d640 0x7fe95010fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:04.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.000+0000 7fe955600700 1 --2- 172.21.15.1:0/1537286496 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe950073110 0x7fe9500734f0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:04.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.000+0000 7fe955600700 1 --2- 172.21.15.1:0/1537286496 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe950073a30 0x7fe95010d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:04.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.000+0000 7fe955600700 1 -- 172.21.15.1:0/1537286496 >> 172.21.15.1:0/1537286496 conn(0x7fe9500fc8b0 msgr2=0x7fe9500fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:04.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.000+0000 7fe955600700 1 -- 172.21.15.1:0/1537286496 shutdown_connections 2024-02-02T15:29:04.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.000+0000 7fe955600700 1 -- 172.21.15.1:0/1537286496 wait complete. 2024-02-02T15:29:04.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.001+0000 7fe955600700 1 Processor -- start 2024-02-02T15:29:04.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.001+0000 7fe955600700 1 -- start start 2024-02-02T15:29:04.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.001+0000 7fe955600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe950073110 0x7fe950100d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:04.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.001+0000 7fe955600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe950073a30 0x7fe9501012a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:04.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.001+0000 7fe955600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe95010d640 0x7fe9501017e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:04.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.001+0000 7fe955600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9501a97c0 con 0x7fe95010d640 2024-02-02T15:29:04.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.001+0000 7fe955600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9501a9930 con 0x7fe950073110 2024-02-02T15:29:04.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.001+0000 7fe955600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9501a9af0 con 0x7fe950073a30 2024-02-02T15:29:04.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.002+0000 7fe94ea00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe950073110 0x7fe950100d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:04.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.002+0000 7fe94e000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe950073a30 0x7fe9501012a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:04.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.002+0000 7fe94ea00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe950073110 0x7fe950100d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:45722/0 (socket says 172.21.15.1:45722) 2024-02-02T15:29:04.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.002+0000 7fe94ea00700 1 -- 172.21.15.1:0/2825456604 learned_addr learned my addr 172.21.15.1:0/2825456604 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:29:04.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.002+0000 7fe94f400700 1 --2- 172.21.15.1:0/2825456604 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe95010d640 0x7fe9501017e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:04.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.002+0000 7fe94ea00700 1 -- 172.21.15.1:0/2825456604 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe950073a30 msgr2=0x7fe9501012a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:04.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.002+0000 7fe94ea00700 1 --2- 172.21.15.1:0/2825456604 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe950073a30 0x7fe9501012a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:04.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.002+0000 7fe94ea00700 1 -- 172.21.15.1:0/2825456604 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe95010d640 msgr2=0x7fe9501017e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:04.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.002+0000 7fe94ea00700 1 --2- 172.21.15.1:0/2825456604 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe95010d640 0x7fe9501017e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:04.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.002+0000 7fe94ea00700 1 -- 172.21.15.1:0/2825456604 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9400095f0 con 0x7fe950073110 2024-02-02T15:29:04.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.002+0000 7fe94e000700 1 --2- 172.21.15.1:0/2825456604 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe950073a30 0x7fe9501012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:29:04.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.002+0000 7fe94ea00700 1 --2- 172.21.15.1:0/2825456604 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe950073110 0x7fe950100d60 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe940005910 tx=0x7fe940004710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:04.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.003+0000 7fe947400700 1 -- 172.21.15.1:0/2825456604 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94001d070 con 0x7fe950073110 2024-02-02T15:29:04.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.003+0000 7fe947400700 1 -- 172.21.15.1:0/2825456604 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe940022470 con 0x7fe950073110 2024-02-02T15:29:04.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.003+0000 7fe955600700 1 -- 172.21.15.1:0/2825456604 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9501a9d70 con 0x7fe950073110 2024-02-02T15:29:04.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.003+0000 7fe947400700 1 -- 172.21.15.1:0/2825456604 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe940017650 con 0x7fe950073110 2024-02-02T15:29:04.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.003+0000 7fe955600700 1 -- 172.21.15.1:0/2825456604 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9501aa1d0 con 0x7fe950073110 2024-02-02T15:29:04.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.006+0000 7fe947400700 1 -- 172.21.15.1:0/2825456604 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 27) v1 ==== 135200+0+0 (secure 0 0 0) 0x7fe9400225e0 con 0x7fe950073110 2024-02-02T15:29:04.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.008+0000 7fe969065b80 1 Processor -- start 2024-02-02T15:29:04.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.008+0000 7fe969065b80 1 -- start start 2024-02-02T15:29:04.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.008+0000 7fe947400700 1 --2- 172.21.15.1:0/2825456604 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fe9300a0690 0x7fe9300a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:04.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.008+0000 7fe947400700 1 -- 172.21.15.1:0/2825456604 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fe9400cd310 con 0x7fe950073110 2024-02-02T15:29:04.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.008+0000 7fe969065b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556d76705010 0x556d767053f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:04.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.008+0000 7fe969065b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556d76705930 0x556d767102a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:04.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.009+0000 7fe969065b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556d767107e0 0x556d76712bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:04.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.009+0000 7fe94f400700 1 --2- 172.21.15.1:0/2825456604 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fe9300a0690 0x7fe9300a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:04.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.009+0000 7fe969065b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d766794c0 con 0x556d76705930 2024-02-02T15:29:04.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.009+0000 7fe969065b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d7665daa0 con 0x556d767107e0 2024-02-02T15:29:04.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.009+0000 7fe969065b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d7665b460 con 0x556d76705010 2024-02-02T15:29:04.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.009+0000 7fe94ea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556d76705010 0x556d767053f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:04.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.009+0000 7fe94ea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556d76705010 0x556d767053f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:54580/0 (socket says 172.21.15.1:54580) 2024-02-02T15:29:04.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.009+0000 7fe94ea00700 1 -- 172.21.15.1:0/1319849615 learned_addr learned my addr 172.21.15.1:0/1319849615 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:29:04.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.009+0000 7fe94f400700 1 --2- 172.21.15.1:0/1319849615 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556d767107e0 0x556d76712bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:04.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.009+0000 7fe94e000700 1 --2- 172.21.15.1:0/1319849615 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556d76705930 0x556d767102a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:04.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.009+0000 7fe94ea00700 1 -- 172.21.15.1:0/1319849615 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556d767107e0 msgr2=0x556d76712bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:04.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.009+0000 7fe94ea00700 1 --2- 172.21.15.1:0/1319849615 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556d767107e0 0x556d76712bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:04.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.009+0000 7fe94ea00700 1 -- 172.21.15.1:0/1319849615 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556d76705930 msgr2=0x556d767102a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:04.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.009+0000 7fe94ea00700 1 --2- 172.21.15.1:0/1319849615 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556d76705930 0x556d767102a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:04.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.009+0000 7fe94ea00700 1 -- 172.21.15.1:0/1319849615 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d764d83b0 con 0x556d76705010 2024-02-02T15:29:04.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.009+0000 7fe94ea00700 1 --2- 172.21.15.1:0/1319849615 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556d76705010 0x556d767053f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe940024000 tx=0x7fe94009a390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca41d4cca6ef8891 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:04.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.010+0000 7fe94f400700 1 --2- 172.21.15.1:0/2825456604 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fe9300a0690 0x7fe9300a2b50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe948009ec0 tx=0x7fe948009920 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:04.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.010+0000 7fe93f400700 1 -- 172.21.15.1:0/1319849615 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94009ac10 con 0x556d76705010 2024-02-02T15:29:04.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.010+0000 7fe93f400700 1 -- 172.21.15.1:0/1319849615 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe94009a510 con 0x556d76705010 2024-02-02T15:29:04.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.010+0000 7fe93f400700 1 -- 172.21.15.1:0/1319849615 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe940085200 con 0x556d76705010 2024-02-02T15:29:04.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.011+0000 7fe969065b80 1 -- 172.21.15.1:0/1319849615 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556d76705010 msgr2=0x556d767053f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:04.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.011+0000 7fe969065b80 1 --2- 172.21.15.1:0/1319849615 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556d76705010 0x556d767053f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe940024000 tx=0x7fe94009a390 comp rx=0 tx=0).stop 2024-02-02T15:29:04.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.011+0000 7fe969065b80 1 -- 172.21.15.1:0/1319849615 shutdown_connections 2024-02-02T15:29:04.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.011+0000 7fe969065b80 1 --2- 172.21.15.1:0/1319849615 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556d767107e0 0x556d76712bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:04.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.011+0000 7fe969065b80 1 --2- 172.21.15.1:0/1319849615 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556d76705010 0x556d767053f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:04.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.011+0000 7fe969065b80 1 --2- 172.21.15.1:0/1319849615 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556d76705930 0x556d767102a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:04.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.011+0000 7fe969065b80 1 -- 172.21.15.1:0/1319849615 >> 172.21.15.1:0/1319849615 conn(0x556d76664db0 msgr2=0x556d767013b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:04.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.011+0000 7fe969065b80 1 -- 172.21.15.1:0/1319849615 shutdown_connections 2024-02-02T15:29:04.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.011+0000 7fe969065b80 1 -- 172.21.15.1:0/1319849615 wait complete. 2024-02-02T15:29:04.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.011+0000 7fe969065b80 1 -- start start 2024-02-02T15:29:04.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.011+0000 7fe969065b80 1 Processor -- start 2024-02-02T15:29:04.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.012+0000 7fe969065b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556d76705010 0x556d76710bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:04.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.012+0000 7fe969065b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556d76705930 0x556d76711100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:04.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.012+0000 7fe969065b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556d767107e0 0x556d76711640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:04.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.012+0000 7fe969065b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d76656d50 con 0x556d76705010 2024-02-02T15:29:04.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.012+0000 7fe969065b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d765f2d80 con 0x556d767107e0 2024-02-02T15:29:04.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.012+0000 7fe969065b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d765f2a20 con 0x556d76705930 2024-02-02T15:29:04.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.012+0000 7fe969065b80 10 client.0 _ll_register_callbacks cb 0x556d765f4290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:29:04.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.012+0000 7fe94f400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556d767107e0 0x556d76711640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:04.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.012+0000 7fe94e000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556d76705930 0x556d76711100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:04.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.012+0000 7fe94f400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556d767107e0 0x556d76711640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:45732/0 (socket says 172.21.15.1:45732) 2024-02-02T15:29:04.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.012+0000 7fe94ea00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556d76705010 0x556d76710bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:04.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.012+0000 7fe94f400700 1 -- 172.21.15.1:0/1745987849 learned_addr learned my addr 172.21.15.1:0/1745987849 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:29:04.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.013+0000 7fe94e000700 1 -- 172.21.15.1:0/1745987849 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556d767107e0 msgr2=0x556d76711640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:04.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.013+0000 7fe94e000700 1 --2- 172.21.15.1:0/1745987849 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556d767107e0 0x556d76711640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:04.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.013+0000 7fe94e000700 1 -- 172.21.15.1:0/1745987849 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556d76705010 msgr2=0x556d76710bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:04.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.013+0000 7fe94e000700 1 --2- 172.21.15.1:0/1745987849 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556d76705010 0x556d76710bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:04.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.013+0000 7fe94e000700 1 -- 172.21.15.1:0/1745987849 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9501aa1d0 con 0x556d76705930 2024-02-02T15:29:04.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.013+0000 7fe94ea00700 1 --2- 172.21.15.1:0/1745987849 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556d76705010 0x556d76710bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:29:04.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.013+0000 7fe94e000700 1 --2- 172.21.15.1:0/1745987849 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556d76705930 0x556d76711100 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe95010eb40 tx=0x7fe9380058e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=244251546aca26c5 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:04.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.013+0000 7fe937e00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:29:04.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.013+0000 7fe937e00700 1 -- 172.21.15.1:0/1745987849 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe938017070 con 0x556d76705930 2024-02-02T15:29:04.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.013+0000 7fe937e00700 1 -- 172.21.15.1:0/1745987849 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fe938004050 con 0x556d76705930 2024-02-02T15:29:04.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.013+0000 7fe969065b80 1 -- 172.21.15.1:0/1745987849 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556d766f90f0 con 0x556d76705930 2024-02-02T15:29:04.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.014+0000 7fe937e00700 1 -- 172.21.15.1:0/1745987849 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe938004210 con 0x556d76705930 2024-02-02T15:29:04.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.014+0000 7fe969065b80 10 client.26200 fetch_fsmap learned FSMap version 29 2024-02-02T15:29:04.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.014+0000 7fe969065b80 1 -- 172.21.15.1:0/1745987849 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x556d76484b60 con 0x556d76705930 2024-02-02T15:29:04.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.014+0000 7fe937e00700 1 -- 172.21.15.1:0/1745987849 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556d766f90f0 con 0x556d76705930 2024-02-02T15:29:04.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.014+0000 7fe937e00700 1 -- 172.21.15.1:0/1745987849 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 29) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fe938003710 con 0x556d76705930 2024-02-02T15:29:04.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.014+0000 7fe969065b80 10 client.26200 fetch_fsmap finished waiting for FSMap version 29 2024-02-02T15:29:04.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.014+0000 7fe969065b80 10 client.26200 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-02T15:29:04.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.014+0000 7fe969065b80 20 client.26200 populate_metadata read hostname 'smithi001' 2024-02-02T15:29:04.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.014+0000 7fe969065b80 1 --2- 172.21.15.1:0/1745987849 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x556d7671ff80 0x556d767223a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:04.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.014+0000 7fe969065b80 4 client.26200 mds_command: new command op to 24529 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:29:04.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.014+0000 7fe969065b80 1 -- 172.21.15.1:0/1745987849 --> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556d7664fdd0 con 0x556d7671ff80 2024-02-02T15:29:04.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.015+0000 7fe94ea00700 1 --2- 172.21.15.1:0/1745987849 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x556d7671ff80 0x556d767223a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:04.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.015+0000 7fe94ea00700 1 --2- 172.21.15.1:0/1745987849 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x556d7671ff80 0x556d767223a0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=65dae600f4fab003 server_cookie=2ef1c18ac7ecc6a2 in_seq=0 out_seq=0 2024-02-02T15:29:04.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.015+0000 7fe937e00700 10 client.26200 ms_handle_connect on v2:172.21.15.142:6838/321675421 2024-02-02T15:29:04.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.016+0000 7fe937e00700 1 -- 172.21.15.1:0/1745987849 <== mds.1 v2:172.21.15.142:6838/321675421 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x556d7664fdd0 con 0x556d7671ff80 2024-02-02T15:29:04.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.016+0000 7fe937e00700 10 client.26200 handle_command_reply: tid=0 2024-02-02T15:29:04.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.016+0000 7fe969065b80 1 client.26200 shutdown 2024-02-02T15:29:04.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.016+0000 7fe969065b80 1 -- 172.21.15.1:0/1745987849 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556d76705930 msgr2=0x556d76711100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:04.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.016+0000 7fe969065b80 1 --2- 172.21.15.1:0/1745987849 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556d76705930 0x556d76711100 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe95010eb40 tx=0x7fe9380058e0 comp rx=0 tx=0).stop 2024-02-02T15:29:04.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.016+0000 7fe969065b80 1 -- 172.21.15.1:0/1745987849 shutdown_connections 2024-02-02T15:29:04.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.016+0000 7fe969065b80 1 --2- 172.21.15.1:0/1745987849 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x556d7671ff80 0x556d767223a0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:04.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.016+0000 7fe969065b80 1 --2- 172.21.15.1:0/1745987849 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556d767107e0 0x556d76711640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:04.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.016+0000 7fe937e00700 0 client.26200 ms_handle_reset on v2:172.21.15.142:6838/321675421 2024-02-02T15:29:04.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.016+0000 7fe969065b80 1 --2- 172.21.15.1:0/1745987849 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556d76705010 0x556d76710bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:04.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.016+0000 7fe969065b80 1 --2- 172.21.15.1:0/1745987849 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556d76705930 0x556d76711100 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:04.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.016+0000 7fe969065b80 1 -- 172.21.15.1:0/1745987849 >> 172.21.15.1:0/1745987849 conn(0x556d76664db0 msgr2=0x556d76700fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:04.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.016+0000 7fe969065b80 1 -- 172.21.15.1:0/1745987849 shutdown_connections 2024-02-02T15:29:04.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.016+0000 7fe969065b80 1 -- 172.21.15.1:0/1745987849 wait complete. 2024-02-02T15:29:04.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.016+0000 7fe969065b80 20 client.26200 trim_cache size 0 max 16384 2024-02-02T15:29:04.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.031+0000 7fe969065b80 1 Processor -- start 2024-02-02T15:29:04.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.032+0000 7fe969065b80 1 -- start start 2024-02-02T15:29:04.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.032+0000 7fe969065b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556d76705010 0x556d76714bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:04.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.032+0000 7fe969065b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556d76705930 0x556d7670ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:04.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.032+0000 7fe969065b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556d767107e0 0x556d7670b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:04.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.032+0000 7fe969065b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d764da6a0 con 0x556d767107e0 2024-02-02T15:29:04.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.032+0000 7fe969065b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d7664fdd0 con 0x556d76705010 2024-02-02T15:29:04.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.032+0000 7fe969065b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d766fd9b0 con 0x556d76705930 2024-02-02T15:29:04.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.032+0000 7fe94e000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556d76705930 0x556d7670ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:04.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.032+0000 7fe94e000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556d76705930 0x556d7670ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:54602/0 (socket says 172.21.15.1:54602) 2024-02-02T15:29:04.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.032+0000 7fe94e000700 1 -- 172.21.15.1:0/3610021766 learned_addr learned my addr 172.21.15.1:0/3610021766 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:29:04.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.032+0000 7fe94ea00700 1 --2- 172.21.15.1:0/3610021766 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556d76705010 0x556d76714bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:04.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.032+0000 7fe94f400700 1 --2- 172.21.15.1:0/3610021766 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556d767107e0 0x556d7670b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:04.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.033+0000 7fe94e000700 1 -- 172.21.15.1:0/3610021766 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556d76705010 msgr2=0x556d76714bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:04.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.033+0000 7fe94e000700 1 --2- 172.21.15.1:0/3610021766 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556d76705010 0x556d76714bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:04.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.033+0000 7fe94e000700 1 -- 172.21.15.1:0/3610021766 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556d767107e0 msgr2=0x556d7670b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:04.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.033+0000 7fe94e000700 1 --2- 172.21.15.1:0/3610021766 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556d767107e0 0x556d7670b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:04.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.033+0000 7fe94e000700 1 -- 172.21.15.1:0/3610021766 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d7661cea0 con 0x556d76705930 2024-02-02T15:29:04.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.033+0000 7fe94e000700 1 --2- 172.21.15.1:0/3610021766 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556d76705930 0x556d7670ab20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe938014000 tx=0x7fe938009fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=308fe85d2d8176fe server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:04.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.033+0000 7fe93fe00700 1 -- 172.21.15.1:0/3610021766 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe938017040 con 0x556d76705930 2024-02-02T15:29:04.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.033+0000 7fe93fe00700 1 -- 172.21.15.1:0/3610021766 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe9380092d0 con 0x556d76705930 2024-02-02T15:29:04.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.033+0000 7fe93fe00700 1 -- 172.21.15.1:0/3610021766 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe93800fbc0 con 0x556d76705930 2024-02-02T15:29:04.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.034+0000 7fe969065b80 1 -- 172.21.15.1:0/3610021766 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556d76705930 msgr2=0x556d7670ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:04.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.034+0000 7fe969065b80 1 --2- 172.21.15.1:0/3610021766 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556d76705930 0x556d7670ab20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe938014000 tx=0x7fe938009fd0 comp rx=0 tx=0).stop 2024-02-02T15:29:04.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.034+0000 7fe969065b80 1 -- 172.21.15.1:0/3610021766 shutdown_connections 2024-02-02T15:29:04.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.034+0000 7fe969065b80 1 --2- 172.21.15.1:0/3610021766 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556d767107e0 0x556d7670b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:04.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.034+0000 7fe969065b80 1 --2- 172.21.15.1:0/3610021766 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556d76705010 0x556d76714bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:04.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.034+0000 7fe969065b80 1 --2- 172.21.15.1:0/3610021766 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556d76705930 0x556d7670ab20 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:04.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.034+0000 7fe969065b80 1 -- 172.21.15.1:0/3610021766 >> 172.21.15.1:0/3610021766 conn(0x556d76664db0 msgr2=0x556d766ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:04.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.034+0000 7fe969065b80 1 -- 172.21.15.1:0/3610021766 shutdown_connections 2024-02-02T15:29:04.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.034+0000 7fe969065b80 1 -- 172.21.15.1:0/3610021766 wait complete. 2024-02-02T15:29:04.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.035+0000 7fe969065b80 1 -- start start 2024-02-02T15:29:04.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.035+0000 7fe969065b80 1 Processor -- start 2024-02-02T15:29:04.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.035+0000 7fe969065b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556d76705010 0x556d76765a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:04.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.035+0000 7fe969065b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556d76705930 0x556d76765f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:04.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.035+0000 7fe969065b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556d767107e0 0x556d767664b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:04.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.035+0000 7fe969065b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d76770b90 con 0x556d76705930 2024-02-02T15:29:04.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.035+0000 7fe969065b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d76770cd0 con 0x556d76705010 2024-02-02T15:29:04.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.035+0000 7fe969065b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d76770e90 con 0x556d767107e0 2024-02-02T15:29:04.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.035+0000 7fe969065b80 10 client.0 _ll_register_callbacks cb 0x556d765f4290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:29:04.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.035+0000 7fe94ea00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556d76705010 0x556d76765a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:04.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.036+0000 7fe94ea00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556d76705010 0x556d76765a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:45750/0 (socket says 172.21.15.1:45750) 2024-02-02T15:29:04.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.036+0000 7fe94ea00700 1 -- 172.21.15.1:0/682586931 learned_addr learned my addr 172.21.15.1:0/682586931 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:29:04.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.036+0000 7fe94e000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556d76705930 0x556d76765f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:04.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.036+0000 7fe94f400700 1 --2- 172.21.15.1:0/682586931 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556d767107e0 0x556d767664b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:04.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.036+0000 7fe94ea00700 1 -- 172.21.15.1:0/682586931 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556d767107e0 msgr2=0x556d767664b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:04.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.036+0000 7fe94ea00700 1 --2- 172.21.15.1:0/682586931 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556d767107e0 0x556d767664b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:04.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.036+0000 7fe94ea00700 1 -- 172.21.15.1:0/682586931 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556d76705930 msgr2=0x556d76765f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:04.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.036+0000 7fe94ea00700 1 --2- 172.21.15.1:0/682586931 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556d76705930 0x556d76765f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:04.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.036+0000 7fe94ea00700 1 -- 172.21.15.1:0/682586931 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556d76484b60 con 0x556d76705010 2024-02-02T15:29:04.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.036+0000 7fe94ea00700 1 --2- 172.21.15.1:0/682586931 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556d76705010 0x556d76765a30 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe94009a210 tx=0x7fe940085040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9676d27a4c422ff server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:04.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.036+0000 7fe937e00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:29:04.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.036+0000 7fe937e00700 1 -- 172.21.15.1:0/682586931 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94009e3c0 con 0x556d76705010 2024-02-02T15:29:04.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.036+0000 7fe937e00700 1 -- 172.21.15.1:0/682586931 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fe94009e120 con 0x556d76705010 2024-02-02T15:29:04.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.036+0000 7fe969065b80 1 -- 172.21.15.1:0/682586931 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe94009a510 con 0x556d76705010 2024-02-02T15:29:04.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.036+0000 7fe937e00700 1 -- 172.21.15.1:0/682586931 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9400a24f0 con 0x556d76705010 2024-02-02T15:29:04.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.036+0000 7fe969065b80 10 client.26216 fetch_fsmap learned FSMap version 29 2024-02-02T15:29:04.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.036+0000 7fe969065b80 1 -- 172.21.15.1:0/682586931 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x556d764dccd0 con 0x556d76705010 2024-02-02T15:29:04.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.037+0000 7fe937e00700 1 -- 172.21.15.1:0/682586931 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe94007f080 con 0x556d76705010 2024-02-02T15:29:04.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.037+0000 7fe937e00700 1 -- 172.21.15.1:0/682586931 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 29) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fe9400b82b0 con 0x556d76705010 2024-02-02T15:29:04.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.037+0000 7fe969065b80 10 client.26216 fetch_fsmap finished waiting for FSMap version 29 2024-02-02T15:29:04.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.037+0000 7fe969065b80 10 client.26216 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-02T15:29:04.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.037+0000 7fe969065b80 20 client.26216 populate_metadata read hostname 'smithi001' 2024-02-02T15:29:04.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.037+0000 7fe969065b80 1 --2- 172.21.15.1:0/682586931 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x556d767720c0 0x556d767744b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:04.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.037+0000 7fe969065b80 4 client.26216 mds_command: new command op to 24529 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:29:04.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.037+0000 7fe969065b80 1 -- 172.21.15.1:0/682586931 --> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556d76615f50 con 0x556d767720c0 2024-02-02T15:29:04.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.037+0000 7fe94e000700 1 --2- 172.21.15.1:0/682586931 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x556d767720c0 0x556d767744b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:04.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.038+0000 7fe94e000700 1 --2- 172.21.15.1:0/682586931 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x556d767720c0 0x556d767744b0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f0170a3130e4929 server_cookie=3783cddafe714e8c in_seq=0 out_seq=0 2024-02-02T15:29:04.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.038+0000 7fe937e00700 10 client.26216 ms_handle_connect on v2:172.21.15.142:6838/321675421 2024-02-02T15:29:04.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.038+0000 7fe937e00700 1 -- 172.21.15.1:0/682586931 <== mds.1 v2:172.21.15.142:6838/321675421 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556d76615f50 con 0x556d767720c0 2024-02-02T15:29:04.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.038+0000 7fe937e00700 10 client.26216 handle_command_reply: tid=0 2024-02-02T15:29:04.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.038+0000 7fe969065b80 1 client.26216 shutdown 2024-02-02T15:29:04.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.038+0000 7fe969065b80 1 -- 172.21.15.1:0/682586931 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556d76705010 msgr2=0x556d76765a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:04.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.038+0000 7fe969065b80 1 --2- 172.21.15.1:0/682586931 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556d76705010 0x556d76765a30 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe94009a210 tx=0x7fe940085040 comp rx=0 tx=0).stop 2024-02-02T15:29:04.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.038+0000 7fe969065b80 1 -- 172.21.15.1:0/682586931 shutdown_connections 2024-02-02T15:29:04.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.038+0000 7fe969065b80 1 --2- 172.21.15.1:0/682586931 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x556d767720c0 0x556d767744b0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:04.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.038+0000 7fe969065b80 1 --2- 172.21.15.1:0/682586931 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556d767107e0 0x556d767664b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:04.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.038+0000 7fe969065b80 1 --2- 172.21.15.1:0/682586931 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556d76705010 0x556d76765a30 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:04.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.038+0000 7fe969065b80 1 --2- 172.21.15.1:0/682586931 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556d76705930 0x556d76765f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:04.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.038+0000 7fe937e00700 0 client.26216 ms_handle_reset on v2:172.21.15.142:6838/321675421 2024-02-02T15:29:04.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.038+0000 7fe969065b80 1 -- 172.21.15.1:0/682586931 >> 172.21.15.1:0/682586931 conn(0x556d76664db0 msgr2=0x556d7670ab20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:04.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.038+0000 7fe969065b80 1 -- 172.21.15.1:0/682586931 shutdown_connections 2024-02-02T15:29:04.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.038+0000 7fe969065b80 1 -- 172.21.15.1:0/682586931 wait complete. 2024-02-02T15:29:04.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.039+0000 7fe969065b80 20 client.26216 trim_cache size 0 max 16384 2024-02-02T15:29:04.043 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:29:04.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.039+0000 7fe955600700 1 -- 172.21.15.1:0/2825456604 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fe9300a0690 msgr2=0x7fe9300a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:04.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.039+0000 7fe955600700 1 --2- 172.21.15.1:0/2825456604 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fe9300a0690 0x7fe9300a2b50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe948009ec0 tx=0x7fe948009920 comp rx=0 tx=0).stop 2024-02-02T15:29:04.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.039+0000 7fe955600700 1 -- 172.21.15.1:0/2825456604 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe950073110 msgr2=0x7fe950100d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:04.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.039+0000 7fe955600700 1 --2- 172.21.15.1:0/2825456604 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe950073110 0x7fe950100d60 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe940005910 tx=0x7fe940004710 comp rx=0 tx=0).stop 2024-02-02T15:29:04.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.039+0000 7fe955600700 1 -- 172.21.15.1:0/2825456604 shutdown_connections 2024-02-02T15:29:04.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.039+0000 7fe955600700 1 --2- 172.21.15.1:0/2825456604 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fe9300a0690 0x7fe9300a2b50 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:04.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.039+0000 7fe955600700 1 --2- 172.21.15.1:0/2825456604 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe95010d640 0x7fe9501017e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:04.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.039+0000 7fe955600700 1 --2- 172.21.15.1:0/2825456604 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe950073110 0x7fe950100d60 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:04.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.039+0000 7fe955600700 1 --2- 172.21.15.1:0/2825456604 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe950073a30 0x7fe9501012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:04.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.039+0000 7fe955600700 1 -- 172.21.15.1:0/2825456604 >> 172.21.15.1:0/2825456604 conn(0x7fe9500fc8b0 msgr2=0x7fe95010f770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:04.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.039+0000 7fe955600700 1 -- 172.21.15.1:0/2825456604 shutdown_connections 2024-02-02T15:29:04.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.039+0000 7fe955600700 1 -- 172.21.15.1:0/2825456604 wait complete. 2024-02-02T15:29:04.058 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-02T15:29:04.219 INFO:tasks.workunit.client.0.smithi001.stdout: 1 69197 10.25 MB/sec warmup 38 sec latency 16.422 ms 2024-02-02T15:29:04.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.281+0000 7fd94b800700 1 -- 172.21.15.1:0/226044481 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd944106120 msgr2=0x7fd94410dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:04.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.281+0000 7fd94b800700 1 --2- 172.21.15.1:0/226044481 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd944106120 0x7fd94410dca0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd93c00b0a0 tx=0x7fd93c00b3b0 comp rx=0 tx=0).stop 2024-02-02T15:29:04.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.282+0000 7fd94b800700 1 -- 172.21.15.1:0/226044481 shutdown_connections 2024-02-02T15:29:04.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.282+0000 7fd94b800700 1 --2- 172.21.15.1:0/226044481 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd944106120 0x7fd94410dca0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:04.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.282+0000 7fd94b800700 1 --2- 172.21.15.1:0/226044481 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd944101000 0x7fd9441013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:04.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.282+0000 7fd94b800700 1 --2- 172.21.15.1:0/226044481 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd9441019b0 0x7fd9441059f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:04.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.282+0000 7fd94b800700 1 -- 172.21.15.1:0/226044481 >> 172.21.15.1:0/226044481 conn(0x7fd9440fc860 msgr2=0x7fd9440fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:04.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.282+0000 7fd94b800700 1 -- 172.21.15.1:0/226044481 shutdown_connections 2024-02-02T15:29:04.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.282+0000 7fd94b800700 1 -- 172.21.15.1:0/226044481 wait complete. 2024-02-02T15:29:04.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.283+0000 7fd94b800700 1 Processor -- start 2024-02-02T15:29:04.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.284+0000 7fd94b800700 1 -- start start 2024-02-02T15:29:04.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.284+0000 7fd94b800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd944101000 0x7fd944073040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:04.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.284+0000 7fd94b800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd944106120 0x7fd944073580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:04.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.284+0000 7fd94b800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd944073ac0 0x7fd9441a4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:04.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.284+0000 7fd94b800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9441a5690 con 0x7fd944101000 2024-02-02T15:29:04.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.284+0000 7fd94b800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9441a5800 con 0x7fd944106120 2024-02-02T15:29:04.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.284+0000 7fd94b800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9441a59c0 con 0x7fd944073ac0 2024-02-02T15:29:04.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.284+0000 7fd943e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd944106120 0x7fd944073580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:04.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.284+0000 7fd943e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd944106120 0x7fd944073580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:45768/0 (socket says 172.21.15.1:45768) 2024-02-02T15:29:04.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.284+0000 7fd943e00700 1 -- 172.21.15.1:0/779068713 learned_addr learned my addr 172.21.15.1:0/779068713 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:29:04.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.284+0000 7fd948e00700 1 --2- 172.21.15.1:0/779068713 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd944101000 0x7fd944073040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:04.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.284+0000 7fd949800700 1 --2- 172.21.15.1:0/779068713 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd944073ac0 0x7fd9441a4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:04.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.285+0000 7fd943e00700 1 -- 172.21.15.1:0/779068713 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd944073ac0 msgr2=0x7fd9441a4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:04.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.285+0000 7fd943e00700 1 --2- 172.21.15.1:0/779068713 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd944073ac0 0x7fd9441a4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:04.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.285+0000 7fd943e00700 1 -- 172.21.15.1:0/779068713 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd944101000 msgr2=0x7fd944073040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:04.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.285+0000 7fd943e00700 1 --2- 172.21.15.1:0/779068713 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd944101000 0x7fd944073040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:04.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.285+0000 7fd943e00700 1 -- 172.21.15.1:0/779068713 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd93c009d00 con 0x7fd944106120 2024-02-02T15:29:04.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.285+0000 7fd943e00700 1 --2- 172.21.15.1:0/779068713 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd944106120 0x7fd944073580 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd92c00e970 tx=0x7fd92c00ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:04.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.285+0000 7fd941600700 1 -- 172.21.15.1:0/779068713 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd92c009790 con 0x7fd944106120 2024-02-02T15:29:04.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.285+0000 7fd941600700 1 -- 172.21.15.1:0/779068713 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd92c00c7e0 con 0x7fd944106120 2024-02-02T15:29:04.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.285+0000 7fd94b800700 1 -- 172.21.15.1:0/779068713 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9441a5ca0 con 0x7fd944106120 2024-02-02T15:29:04.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.285+0000 7fd941600700 1 -- 172.21.15.1:0/779068713 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd92c0054e0 con 0x7fd944106120 2024-02-02T15:29:04.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.286+0000 7fd94b800700 1 -- 172.21.15.1:0/779068713 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9441a61b0 con 0x7fd944106120 2024-02-02T15:29:04.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.289+0000 7fd941600700 1 -- 172.21.15.1:0/779068713 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 27) v1 ==== 135200+0+0 (secure 0 0 0) 0x7fd92c00c950 con 0x7fd944106120 2024-02-02T15:29:04.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.290+0000 7fd95f3d9b80 1 Processor -- start 2024-02-02T15:29:04.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.290+0000 7fd95f3d9b80 1 -- start start 2024-02-02T15:29:04.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.291+0000 7fd95f3d9b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56468b75a010 0x56468b75a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:04.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.291+0000 7fd95f3d9b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56468b75a930 0x56468b7652a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:04.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.291+0000 7fd95f3d9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56468b7657e0 0x56468b767bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:04.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.291+0000 7fd95f3d9b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56468b6ce4c0 con 0x56468b75a930 2024-02-02T15:29:04.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.291+0000 7fd95f3d9b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56468b6b2aa0 con 0x56468b75a010 2024-02-02T15:29:04.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.291+0000 7fd95f3d9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56468b6b0460 con 0x56468b7657e0 2024-02-02T15:29:04.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.291+0000 7fd949800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56468b75a010 0x56468b75a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:04.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.291+0000 7fd949800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56468b75a010 0x56468b75a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:45772/0 (socket says 172.21.15.1:45772) 2024-02-02T15:29:04.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.291+0000 7fd949800700 1 -- 172.21.15.1:0/3131740834 learned_addr learned my addr 172.21.15.1:0/3131740834 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:29:04.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.291+0000 7fd948e00700 1 --2- 172.21.15.1:0/3131740834 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56468b75a930 0x56468b7652a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:04.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.291+0000 7fd943e00700 1 --2- 172.21.15.1:0/3131740834 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56468b7657e0 0x56468b767bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:04.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.291+0000 7fd941600700 1 --2- 172.21.15.1:0/779068713 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fd9240a0710 0x7fd9240a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:04.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.291+0000 7fd941600700 1 -- 172.21.15.1:0/779068713 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fd92c014070 con 0x7fd944106120 2024-02-02T15:29:04.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.291+0000 7fd949800700 1 -- 172.21.15.1:0/3131740834 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56468b7657e0 msgr2=0x56468b767bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:04.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.291+0000 7fd949800700 1 --2- 172.21.15.1:0/3131740834 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56468b7657e0 0x56468b767bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:04.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.291+0000 7fd949800700 1 -- 172.21.15.1:0/3131740834 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56468b75a930 msgr2=0x56468b7652a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:04.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.292+0000 7fd949800700 1 --2- 172.21.15.1:0/3131740834 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56468b75a930 0x56468b7652a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:04.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.292+0000 7fd949800700 1 -- 172.21.15.1:0/3131740834 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56468b52d3b0 con 0x56468b75a010 2024-02-02T15:29:04.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.292+0000 7fd943e00700 1 --2- 172.21.15.1:0/3131740834 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56468b7657e0 0x56468b767bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:29:04.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.292+0000 7fd949800700 1 --2- 172.21.15.1:0/779068713 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fd9240a0710 0x7fd9240a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:04.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.292+0000 7fd949800700 1 --2- 172.21.15.1:0/3131740834 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56468b75a010 0x56468b75a3f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd93c009550 tx=0x7fd93c009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bef95fcbd01de60d server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:04.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.292+0000 7fd93a000700 1 -- 172.21.15.1:0/3131740834 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93c033070 con 0x56468b75a010 2024-02-02T15:29:04.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.292+0000 7fd93a000700 1 -- 172.21.15.1:0/3131740834 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd93c0079a0 con 0x56468b75a010 2024-02-02T15:29:04.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.292+0000 7fd93a000700 1 -- 172.21.15.1:0/3131740834 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93c00e040 con 0x56468b75a010 2024-02-02T15:29:04.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.293+0000 7fd95f3d9b80 1 -- 172.21.15.1:0/3131740834 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56468b75a010 msgr2=0x56468b75a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:04.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.293+0000 7fd95f3d9b80 1 --2- 172.21.15.1:0/3131740834 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56468b75a010 0x56468b75a3f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd93c009550 tx=0x7fd93c009580 comp rx=0 tx=0).stop 2024-02-02T15:29:04.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.293+0000 7fd949800700 1 --2- 172.21.15.1:0/779068713 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fd9240a0710 0x7fd9240a2bd0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd93c032000 tx=0x7fd93c02f000 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:04.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.293+0000 7fd95f3d9b80 1 -- 172.21.15.1:0/3131740834 shutdown_connections 2024-02-02T15:29:04.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.293+0000 7fd95f3d9b80 1 --2- 172.21.15.1:0/3131740834 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56468b7657e0 0x56468b767bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:04.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.293+0000 7fd95f3d9b80 1 --2- 172.21.15.1:0/3131740834 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56468b75a010 0x56468b75a3f0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:04.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.293+0000 7fd95f3d9b80 1 --2- 172.21.15.1:0/3131740834 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56468b75a930 0x56468b7652a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:04.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.293+0000 7fd95f3d9b80 1 -- 172.21.15.1:0/3131740834 >> 172.21.15.1:0/3131740834 conn(0x56468b6b9db0 msgr2=0x56468b7563b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:04.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.293+0000 7fd95f3d9b80 1 -- 172.21.15.1:0/3131740834 shutdown_connections 2024-02-02T15:29:04.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.293+0000 7fd95f3d9b80 1 -- 172.21.15.1:0/3131740834 wait complete. 2024-02-02T15:29:04.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.294+0000 7fd95f3d9b80 1 -- start start 2024-02-02T15:29:04.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.294+0000 7fd95f3d9b80 1 Processor -- start 2024-02-02T15:29:04.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.295+0000 7fd95f3d9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56468b75a010 0x56468b7651a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:04.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.295+0000 7fd95f3d9b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56468b75a930 0x56468b762420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:04.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.295+0000 7fd95f3d9b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56468b7657e0 0x56468b762aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:04.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.295+0000 7fd95f3d9b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56468b6abd50 con 0x56468b75a930 2024-02-02T15:29:04.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.295+0000 7fd95f3d9b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56468b647d80 con 0x56468b7657e0 2024-02-02T15:29:04.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.295+0000 7fd95f3d9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56468b647a20 con 0x56468b75a010 2024-02-02T15:29:04.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.295+0000 7fd95f3d9b80 10 client.0 _ll_register_callbacks cb 0x56468b649290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:29:04.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.295+0000 7fd949800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56468b7657e0 0x56468b762aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:04.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.295+0000 7fd943e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56468b75a930 0x56468b762420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:04.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.295+0000 7fd949800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56468b7657e0 0x56468b762aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:45780/0 (socket says 172.21.15.1:45780) 2024-02-02T15:29:04.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.295+0000 7fd943e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56468b75a930 0x56468b762420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37946/0 (socket says 172.21.15.1:37946) 2024-02-02T15:29:04.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.295+0000 7fd949800700 1 -- 172.21.15.1:0/799465515 learned_addr learned my addr 172.21.15.1:0/799465515 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:29:04.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.295+0000 7fd948e00700 1 --2- 172.21.15.1:0/799465515 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56468b75a010 0x56468b7651a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:04.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.295+0000 7fd949800700 1 -- 172.21.15.1:0/799465515 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56468b75a010 msgr2=0x56468b7651a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:04.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.295+0000 7fd949800700 1 --2- 172.21.15.1:0/799465515 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56468b75a010 0x56468b7651a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:04.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.295+0000 7fd949800700 1 -- 172.21.15.1:0/799465515 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56468b75a930 msgr2=0x56468b762420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:04.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.295+0000 7fd949800700 1 --2- 172.21.15.1:0/799465515 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56468b75a930 0x56468b762420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:04.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.295+0000 7fd949800700 1 -- 172.21.15.1:0/799465515 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd94410e510 con 0x56468b7657e0 2024-02-02T15:29:04.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.295+0000 7fd948e00700 1 --2- 172.21.15.1:0/799465515 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56468b75a010 0x56468b7651a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:29:04.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.295+0000 7fd943e00700 1 --2- 172.21.15.1:0/799465515 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56468b75a930 0x56468b762420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:29:04.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.296+0000 7fd949800700 1 --2- 172.21.15.1:0/799465515 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56468b7657e0 0x56468b762aa0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd93c02e520 tx=0x7fd93c024da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=733810fefeec7aa server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:04.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.296+0000 7fd932a00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:29:04.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.296+0000 7fd932a00700 1 -- 172.21.15.1:0/799465515 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93c033070 con 0x56468b7657e0 2024-02-02T15:29:04.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.296+0000 7fd932a00700 1 -- 172.21.15.1:0/799465515 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fd93c02e960 con 0x56468b7657e0 2024-02-02T15:29:04.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.296+0000 7fd95f3d9b80 1 -- 172.21.15.1:0/799465515 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56468b74e0f0 con 0x56468b7657e0 2024-02-02T15:29:04.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.296+0000 7fd932a00700 1 -- 172.21.15.1:0/799465515 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd93c02eb20 con 0x56468b7657e0 2024-02-02T15:29:04.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.296+0000 7fd95f3d9b80 10 client.26228 fetch_fsmap learned FSMap version 29 2024-02-02T15:29:04.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.296+0000 7fd95f3d9b80 1 -- 172.21.15.1:0/799465515 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x56468b4d9b60 con 0x56468b7657e0 2024-02-02T15:29:04.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.296+0000 7fd932a00700 1 -- 172.21.15.1:0/799465515 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56468b74e0f0 con 0x56468b7657e0 2024-02-02T15:29:04.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.297+0000 7fd932a00700 1 -- 172.21.15.1:0/799465515 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 29) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fd93c038ca0 con 0x56468b7657e0 2024-02-02T15:29:04.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.297+0000 7fd95f3d9b80 10 client.26228 fetch_fsmap finished waiting for FSMap version 29 2024-02-02T15:29:04.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.297+0000 7fd95f3d9b80 10 client.26228 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:29:04.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.297+0000 7fd95f3d9b80 20 client.26228 populate_metadata read hostname 'smithi001' 2024-02-02T15:29:04.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.297+0000 7fd95f3d9b80 1 --2- 172.21.15.1:0/799465515 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56468b774f60 0x56468b777380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:04.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.297+0000 7fd95f3d9b80 4 client.26228 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:29:04.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.297+0000 7fd95f3d9b80 1 -- 172.21.15.1:0/799465515 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56468b6a4dd0 con 0x56468b774f60 2024-02-02T15:29:04.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.297+0000 7fd948e00700 1 --2- 172.21.15.1:0/799465515 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56468b774f60 0x56468b777380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:04.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.298+0000 7fd948e00700 1 --2- 172.21.15.1:0/799465515 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56468b774f60 0x56468b777380 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd3806c9fdaeb762 server_cookie=d025a341ed6f5d13 in_seq=0 out_seq=0 2024-02-02T15:29:04.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.298+0000 7fd932a00700 10 client.26228 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:29:04.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.298+0000 7fd932a00700 1 -- 172.21.15.1:0/799465515 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x56468b6a4dd0 con 0x56468b774f60 2024-02-02T15:29:04.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.298+0000 7fd932a00700 10 client.26228 handle_command_reply: tid=0 2024-02-02T15:29:04.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.298+0000 7fd95f3d9b80 1 client.26228 shutdown 2024-02-02T15:29:04.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.299+0000 7fd95f3d9b80 1 -- 172.21.15.1:0/799465515 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56468b7657e0 msgr2=0x56468b762aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:04.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.299+0000 7fd95f3d9b80 1 --2- 172.21.15.1:0/799465515 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56468b7657e0 0x56468b762aa0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd93c02e520 tx=0x7fd93c024da0 comp rx=0 tx=0).stop 2024-02-02T15:29:04.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.299+0000 7fd95f3d9b80 1 -- 172.21.15.1:0/799465515 shutdown_connections 2024-02-02T15:29:04.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.299+0000 7fd95f3d9b80 1 --2- 172.21.15.1:0/799465515 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56468b774f60 0x56468b777380 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:04.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.299+0000 7fd95f3d9b80 1 --2- 172.21.15.1:0/799465515 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56468b7657e0 0x56468b762aa0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:04.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.299+0000 7fd95f3d9b80 1 --2- 172.21.15.1:0/799465515 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56468b75a010 0x56468b7651a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:04.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.299+0000 7fd95f3d9b80 1 --2- 172.21.15.1:0/799465515 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56468b75a930 0x56468b762420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:04.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.299+0000 7fd932a00700 0 client.26228 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:29:04.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.299+0000 7fd95f3d9b80 1 -- 172.21.15.1:0/799465515 >> 172.21.15.1:0/799465515 conn(0x56468b6b9db0 msgr2=0x56468b755fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:04.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.299+0000 7fd95f3d9b80 1 -- 172.21.15.1:0/799465515 shutdown_connections 2024-02-02T15:29:04.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.299+0000 7fd95f3d9b80 1 -- 172.21.15.1:0/799465515 wait complete. 2024-02-02T15:29:04.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.299+0000 7fd95f3d9b80 20 client.26228 trim_cache size 0 max 16384 2024-02-02T15:29:04.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.313+0000 7fd95f3d9b80 1 Processor -- start 2024-02-02T15:29:04.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.313+0000 7fd95f3d9b80 1 -- start start 2024-02-02T15:29:04.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.313+0000 7fd95f3d9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56468b75a010 0x56468b75a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:04.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.313+0000 7fd95f3d9b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56468b75a930 0x56468b76c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:04.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.313+0000 7fd95f3d9b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56468b7657e0 0x56468b754b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:04.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.313+0000 7fd95f3d9b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56468b52f6a0 con 0x56468b75a930 2024-02-02T15:29:04.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.313+0000 7fd95f3d9b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56468b7609b0 con 0x56468b7657e0 2024-02-02T15:29:04.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.313+0000 7fd95f3d9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56468b75e270 con 0x56468b75a010 2024-02-02T15:29:04.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.313+0000 7fd943e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56468b75a930 0x56468b76c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:04.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.314+0000 7fd949800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56468b7657e0 0x56468b754b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:04.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.314+0000 7fd948e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56468b75a010 0x56468b75a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:04.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.314+0000 7fd949800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56468b7657e0 0x56468b754b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:45790/0 (socket says 172.21.15.1:45790) 2024-02-02T15:29:04.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.314+0000 7fd949800700 1 -- 172.21.15.1:0/3198333185 learned_addr learned my addr 172.21.15.1:0/3198333185 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:29:04.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.314+0000 7fd948e00700 1 -- 172.21.15.1:0/3198333185 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56468b7657e0 msgr2=0x56468b754b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:04.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.314+0000 7fd948e00700 1 --2- 172.21.15.1:0/3198333185 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56468b7657e0 0x56468b754b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:04.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.314+0000 7fd948e00700 1 -- 172.21.15.1:0/3198333185 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56468b75a930 msgr2=0x56468b76c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:04.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.314+0000 7fd948e00700 1 --2- 172.21.15.1:0/3198333185 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56468b75a930 0x56468b76c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:04.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.314+0000 7fd948e00700 1 -- 172.21.15.1:0/3198333185 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56468b671ea0 con 0x56468b75a010 2024-02-02T15:29:04.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.314+0000 7fd949800700 1 --2- 172.21.15.1:0/3198333185 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56468b7657e0 0x56468b754b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:29:04.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.314+0000 7fd943e00700 1 --2- 172.21.15.1:0/3198333185 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56468b75a930 0x56468b76c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:29:04.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.314+0000 7fd948e00700 1 --2- 172.21.15.1:0/3198333185 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56468b75a010 0x56468b75a3f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x56468b777960 tx=0x7fd934004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24fe3370c5d16ec server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:04.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.314+0000 7fd93aa00700 1 -- 172.21.15.1:0/3198333185 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd934016070 con 0x56468b75a010 2024-02-02T15:29:04.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.315+0000 7fd93aa00700 1 -- 172.21.15.1:0/3198333185 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd934017040 con 0x56468b75a010 2024-02-02T15:29:04.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.315+0000 7fd93aa00700 1 -- 172.21.15.1:0/3198333185 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd934003680 con 0x56468b75a010 2024-02-02T15:29:04.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.315+0000 7fd95f3d9b80 1 -- 172.21.15.1:0/3198333185 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56468b75a010 msgr2=0x56468b75a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:04.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.315+0000 7fd95f3d9b80 1 --2- 172.21.15.1:0/3198333185 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56468b75a010 0x56468b75a3f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x56468b777960 tx=0x7fd934004750 comp rx=0 tx=0).stop 2024-02-02T15:29:04.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.315+0000 7fd95f3d9b80 1 -- 172.21.15.1:0/3198333185 shutdown_connections 2024-02-02T15:29:04.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.315+0000 7fd95f3d9b80 1 --2- 172.21.15.1:0/3198333185 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56468b7657e0 0x56468b754b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:04.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.315+0000 7fd95f3d9b80 1 --2- 172.21.15.1:0/3198333185 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56468b75a010 0x56468b75a3f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:04.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.315+0000 7fd95f3d9b80 1 --2- 172.21.15.1:0/3198333185 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56468b75a930 0x56468b76c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:04.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.315+0000 7fd95f3d9b80 1 -- 172.21.15.1:0/3198333185 >> 172.21.15.1:0/3198333185 conn(0x56468b6b9db0 msgr2=0x56468b767a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:04.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.315+0000 7fd95f3d9b80 1 -- 172.21.15.1:0/3198333185 shutdown_connections 2024-02-02T15:29:04.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.315+0000 7fd95f3d9b80 1 -- 172.21.15.1:0/3198333185 wait complete. 2024-02-02T15:29:04.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.316+0000 7fd95f3d9b80 1 -- start start 2024-02-02T15:29:04.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.316+0000 7fd95f3d9b80 1 Processor -- start 2024-02-02T15:29:04.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.317+0000 7fd95f3d9b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56468b75a010 0x56468b762660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:04.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.317+0000 7fd95f3d9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56468b75a930 0x56468b768300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:04.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.317+0000 7fd95f3d9b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56468b7657e0 0x56468b768840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:04.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.317+0000 7fd95f3d9b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56468b768e90 con 0x56468b75a010 2024-02-02T15:29:04.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.317+0000 7fd95f3d9b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56468b769000 con 0x56468b7657e0 2024-02-02T15:29:04.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.317+0000 7fd95f3d9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56468b7691c0 con 0x56468b75a930 2024-02-02T15:29:04.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.317+0000 7fd95f3d9b80 10 client.0 _ll_register_callbacks cb 0x56468b649290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:29:04.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.317+0000 7fd949800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56468b7657e0 0x56468b768840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:04.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.317+0000 7fd948e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56468b75a010 0x56468b762660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:04.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.317+0000 7fd943e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56468b75a930 0x56468b768300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:04.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.317+0000 7fd949800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56468b7657e0 0x56468b768840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:45802/0 (socket says 172.21.15.1:45802) 2024-02-02T15:29:04.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.317+0000 7fd949800700 1 -- 172.21.15.1:0/3161473566 learned_addr learned my addr 172.21.15.1:0/3161473566 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:29:04.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.318+0000 7fd943e00700 1 -- 172.21.15.1:0/3161473566 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56468b7657e0 msgr2=0x56468b768840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:04.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.318+0000 7fd943e00700 1 --2- 172.21.15.1:0/3161473566 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56468b7657e0 0x56468b768840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:04.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.318+0000 7fd943e00700 1 -- 172.21.15.1:0/3161473566 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56468b75a010 msgr2=0x56468b762660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:04.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.318+0000 7fd943e00700 1 --2- 172.21.15.1:0/3161473566 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56468b75a010 0x56468b762660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:04.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.318+0000 7fd943e00700 1 -- 172.21.15.1:0/3161473566 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9441a61b0 con 0x56468b75a930 2024-02-02T15:29:04.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.318+0000 7fd948e00700 1 --2- 172.21.15.1:0/3161473566 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56468b75a010 0x56468b762660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:29:04.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.318+0000 7fd949800700 1 --2- 172.21.15.1:0/3161473566 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56468b7657e0 0x56468b768840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:29:04.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.318+0000 7fd943e00700 1 --2- 172.21.15.1:0/3161473566 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56468b75a930 0x56468b768300 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd92c0c95e0 tx=0x7fd92c098d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54fc24bb17afacd3 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:04.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.318+0000 7fd932a00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:29:04.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.318+0000 7fd932a00700 1 -- 172.21.15.1:0/3161473566 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd92c099ab0 con 0x56468b75a930 2024-02-02T15:29:04.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.318+0000 7fd932a00700 1 -- 172.21.15.1:0/3161473566 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fd92c0993b0 con 0x56468b75a930 2024-02-02T15:29:04.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.318+0000 7fd95f3d9b80 1 -- 172.21.15.1:0/3161473566 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd93c0079a0 con 0x56468b75a930 2024-02-02T15:29:04.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.318+0000 7fd932a00700 1 -- 172.21.15.1:0/3161473566 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd92c09d1e0 con 0x56468b75a930 2024-02-02T15:29:04.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.318+0000 7fd95f3d9b80 10 client.26224 fetch_fsmap learned FSMap version 29 2024-02-02T15:29:04.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.318+0000 7fd95f3d9b80 1 -- 172.21.15.1:0/3161473566 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x56468b4d9b60 con 0x56468b75a930 2024-02-02T15:29:04.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.319+0000 7fd932a00700 1 -- 172.21.15.1:0/3161473566 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd93c0079a0 con 0x56468b75a930 2024-02-02T15:29:04.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.319+0000 7fd932a00700 1 -- 172.21.15.1:0/3161473566 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 29) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fd92c098930 con 0x56468b75a930 2024-02-02T15:29:04.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.319+0000 7fd95f3d9b80 10 client.26224 fetch_fsmap finished waiting for FSMap version 29 2024-02-02T15:29:04.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.319+0000 7fd95f3d9b80 10 client.26224 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:29:04.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.319+0000 7fd95f3d9b80 20 client.26224 populate_metadata read hostname 'smithi001' 2024-02-02T15:29:04.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.319+0000 7fd95f3d9b80 1 --2- 172.21.15.1:0/3161473566 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56468b7c71a0 0x56468b7c95c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:04.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.319+0000 7fd95f3d9b80 4 client.26224 mds_command: new command op to 24557 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:29:04.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.319+0000 7fd95f3d9b80 1 -- 172.21.15.1:0/3161473566 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56468b66af50 con 0x56468b7c71a0 2024-02-02T15:29:04.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.319+0000 7fd948e00700 1 --2- 172.21.15.1:0/3161473566 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56468b7c71a0 0x56468b7c95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:04.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.320+0000 7fd948e00700 1 --2- 172.21.15.1:0/3161473566 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56468b7c71a0 0x56468b7c95c0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bb55fc0926e3156 server_cookie=2ea30486e6711729 in_seq=0 out_seq=0 2024-02-02T15:29:04.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.320+0000 7fd932a00700 10 client.26224 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:29:04.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.320+0000 7fd932a00700 1 -- 172.21.15.1:0/3161473566 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56468b66af50 con 0x56468b7c71a0 2024-02-02T15:29:04.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.320+0000 7fd932a00700 10 client.26224 handle_command_reply: tid=0 2024-02-02T15:29:04.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.320+0000 7fd95f3d9b80 1 client.26224 shutdown 2024-02-02T15:29:04.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.321+0000 7fd95f3d9b80 1 -- 172.21.15.1:0/3161473566 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56468b75a930 msgr2=0x56468b768300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:04.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.321+0000 7fd95f3d9b80 1 --2- 172.21.15.1:0/3161473566 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56468b75a930 0x56468b768300 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd92c0c95e0 tx=0x7fd92c098d10 comp rx=0 tx=0).stop 2024-02-02T15:29:04.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.321+0000 7fd95f3d9b80 1 -- 172.21.15.1:0/3161473566 shutdown_connections 2024-02-02T15:29:04.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.321+0000 7fd95f3d9b80 1 --2- 172.21.15.1:0/3161473566 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56468b7c71a0 0x56468b7c95c0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:04.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.321+0000 7fd95f3d9b80 1 --2- 172.21.15.1:0/3161473566 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56468b7657e0 0x56468b768840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:04.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.321+0000 7fd95f3d9b80 1 --2- 172.21.15.1:0/3161473566 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56468b75a010 0x56468b762660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:04.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.321+0000 7fd95f3d9b80 1 --2- 172.21.15.1:0/3161473566 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56468b75a930 0x56468b768300 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:04.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.321+0000 7fd95f3d9b80 1 -- 172.21.15.1:0/3161473566 >> 172.21.15.1:0/3161473566 conn(0x56468b6b9db0 msgr2=0x56468b770900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:04.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.321+0000 7fd932a00700 0 client.26224 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:29:04.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.321+0000 7fd95f3d9b80 1 -- 172.21.15.1:0/3161473566 shutdown_connections 2024-02-02T15:29:04.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.321+0000 7fd95f3d9b80 1 -- 172.21.15.1:0/3161473566 wait complete. 2024-02-02T15:29:04.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.321+0000 7fd95f3d9b80 20 client.26224 trim_cache size 0 max 16384 2024-02-02T15:29:04.326 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:29:04.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.322+0000 7fd94b800700 1 -- 172.21.15.1:0/779068713 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fd9240a0710 msgr2=0x7fd9240a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:04.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.322+0000 7fd94b800700 1 --2- 172.21.15.1:0/779068713 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fd9240a0710 0x7fd9240a2bd0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd93c032000 tx=0x7fd93c02f000 comp rx=0 tx=0).stop 2024-02-02T15:29:04.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.322+0000 7fd94b800700 1 -- 172.21.15.1:0/779068713 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd944106120 msgr2=0x7fd944073580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:04.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.322+0000 7fd94b800700 1 --2- 172.21.15.1:0/779068713 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd944106120 0x7fd944073580 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd92c00e970 tx=0x7fd92c00ec80 comp rx=0 tx=0).stop 2024-02-02T15:29:04.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.322+0000 7fd94b800700 1 -- 172.21.15.1:0/779068713 shutdown_connections 2024-02-02T15:29:04.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.322+0000 7fd94b800700 1 --2- 172.21.15.1:0/779068713 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fd9240a0710 0x7fd9240a2bd0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:04.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.322+0000 7fd94b800700 1 --2- 172.21.15.1:0/779068713 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd944073ac0 0x7fd9441a4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:04.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.322+0000 7fd94b800700 1 --2- 172.21.15.1:0/779068713 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd944101000 0x7fd944073040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:04.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.322+0000 7fd94b800700 1 --2- 172.21.15.1:0/779068713 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd944106120 0x7fd944073580 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:04.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.322+0000 7fd94b800700 1 -- 172.21.15.1:0/779068713 >> 172.21.15.1:0/779068713 conn(0x7fd9440fc860 msgr2=0x7fd9440fd250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:04.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.323+0000 7fd94b800700 1 -- 172.21.15.1:0/779068713 shutdown_connections 2024-02-02T15:29:04.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:04.323+0000 7fd94b800700 1 -- 172.21.15.1:0/779068713 wait complete. 2024-02-02T15:29:05.219 INFO:tasks.workunit.client.0.smithi001.stdout: 1 71285 10.34 MB/sec warmup 39 sec latency 20.101 ms 2024-02-02T15:29:05.356 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-02T15:29:05.356 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-02T15:29:05.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.570+0000 7f223a600700 1 -- 172.21.15.1:0/3761775785 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2234103290 msgr2=0x7f2234103670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:05.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.570+0000 7f223a600700 1 --2- 172.21.15.1:0/3761775785 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2234103290 0x7f2234103670 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f22240098f0 tx=0x7f2224009c00 comp rx=0 tx=0).stop 2024-02-02T15:29:05.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.570+0000 7f223a600700 1 -- 172.21.15.1:0/3761775785 shutdown_connections 2024-02-02T15:29:05.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.570+0000 7f223a600700 1 --2- 172.21.15.1:0/3761775785 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f22341083b0 0x7f223410ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:05.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.570+0000 7f223a600700 1 --2- 172.21.15.1:0/3761775785 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2234103290 0x7f2234103670 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:05.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.570+0000 7f223a600700 1 --2- 172.21.15.1:0/3761775785 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2234103c40 0x7f2234107c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:05.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.570+0000 7f223a600700 1 -- 172.21.15.1:0/3761775785 >> 172.21.15.1:0/3761775785 conn(0x7f22340feb30 msgr2=0x7f2234100f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:05.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.570+0000 7f223a600700 1 -- 172.21.15.1:0/3761775785 shutdown_connections 2024-02-02T15:29:05.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.571+0000 7f223a600700 1 -- 172.21.15.1:0/3761775785 wait complete. 2024-02-02T15:29:05.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.571+0000 7f223a600700 1 Processor -- start 2024-02-02T15:29:05.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.572+0000 7f223a600700 1 -- start start 2024-02-02T15:29:05.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.572+0000 7f223a600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2234103290 0x7f22341992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:05.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.572+0000 7f223a600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f22341083b0 0x7f2234199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:05.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.572+0000 7f223a600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f223419dc70 0x7f223419e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:05.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.572+0000 7f223a600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f223419e6f0 con 0x7f2234103290 2024-02-02T15:29:05.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.572+0000 7f223a600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f223419e860 con 0x7f223419dc70 2024-02-02T15:29:05.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.572+0000 7f223a600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f223419ea20 con 0x7f22341083b0 2024-02-02T15:29:05.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.572+0000 7f2233400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2234103290 0x7f22341992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:05.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.572+0000 7f222be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f22341083b0 0x7f2234199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:05.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.572+0000 7f222be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f22341083b0 0x7f2234199810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:54692/0 (socket says 172.21.15.1:54692) 2024-02-02T15:29:05.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.572+0000 7f2233400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2234103290 0x7f22341992d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37980/0 (socket says 172.21.15.1:37980) 2024-02-02T15:29:05.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.572+0000 7f222be00700 1 -- 172.21.15.1:0/2207565424 learned_addr learned my addr 172.21.15.1:0/2207565424 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:29:05.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.573+0000 7f2233e00700 1 --2- 172.21.15.1:0/2207565424 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f223419dc70 0x7f223419e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:05.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.573+0000 7f222be00700 1 -- 172.21.15.1:0/2207565424 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f223419dc70 msgr2=0x7f223419e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:05.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.573+0000 7f222be00700 1 --2- 172.21.15.1:0/2207565424 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f223419dc70 0x7f223419e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:05.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.573+0000 7f222be00700 1 -- 172.21.15.1:0/2207565424 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2234103290 msgr2=0x7f22341992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:05.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.573+0000 7f222be00700 1 --2- 172.21.15.1:0/2207565424 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2234103290 0x7f22341992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:05.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.573+0000 7f222be00700 1 -- 172.21.15.1:0/2207565424 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f221c009520 con 0x7f22341083b0 2024-02-02T15:29:05.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.573+0000 7f2233400700 1 --2- 172.21.15.1:0/2207565424 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2234103290 0x7f22341992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:29:05.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.573+0000 7f222be00700 1 --2- 172.21.15.1:0/2207565424 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f22341083b0 0x7f2234199810 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f221c00ea30 tx=0x7f221c00ed40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:05.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.573+0000 7f2230c00700 1 -- 172.21.15.1:0/2207565424 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f221c00cbd0 con 0x7f22341083b0 2024-02-02T15:29:05.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.573+0000 7f2230c00700 1 -- 172.21.15.1:0/2207565424 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f221c00eed0 con 0x7f22341083b0 2024-02-02T15:29:05.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.573+0000 7f223a600700 1 -- 172.21.15.1:0/2207565424 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2224009520 con 0x7f22341083b0 2024-02-02T15:29:05.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.573+0000 7f2230c00700 1 -- 172.21.15.1:0/2207565424 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f221c003d20 con 0x7f22341083b0 2024-02-02T15:29:05.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.574+0000 7f223a600700 1 -- 172.21.15.1:0/2207565424 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22341a6270 con 0x7f22341083b0 2024-02-02T15:29:05.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.580+0000 7f2230c00700 1 -- 172.21.15.1:0/2207565424 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 27) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f221c003fa0 con 0x7f22341083b0 2024-02-02T15:29:05.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.580+0000 7f224e072b80 1 Processor -- start 2024-02-02T15:29:05.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.580+0000 7f224e072b80 1 -- start start 2024-02-02T15:29:05.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.581+0000 7f224e072b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c3f081090 0x560c3f081470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:05.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.581+0000 7f224e072b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560c3f0819b0 0x560c3f08c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:05.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.581+0000 7f224e072b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560c3f08c860 0x560c3f08ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:05.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.581+0000 7f224e072b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c3eff5540 con 0x560c3f08c860 2024-02-02T15:29:05.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.581+0000 7f224e072b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c3efd9b20 con 0x560c3f081090 2024-02-02T15:29:05.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.581+0000 7f224e072b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c3efd74e0 con 0x560c3f0819b0 2024-02-02T15:29:05.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.581+0000 7f2233e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c3f081090 0x560c3f081470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:05.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.581+0000 7f222be00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560c3f08c860 0x560c3f08ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:05.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.581+0000 7f2233e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c3f081090 0x560c3f081470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:45820/0 (socket says 172.21.15.1:45820) 2024-02-02T15:29:05.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.581+0000 7f2233e00700 1 -- 172.21.15.1:0/1128527317 learned_addr learned my addr 172.21.15.1:0/1128527317 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:29:05.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.581+0000 7f2233400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560c3f0819b0 0x560c3f08c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:05.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.581+0000 7f2233e00700 1 -- 172.21.15.1:0/1128527317 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560c3f0819b0 msgr2=0x560c3f08c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:05.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.581+0000 7f2233e00700 1 --2- 172.21.15.1:0/1128527317 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560c3f0819b0 0x560c3f08c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:05.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.581+0000 7f2233e00700 1 -- 172.21.15.1:0/1128527317 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560c3f08c860 msgr2=0x560c3f08ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:05.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.581+0000 7f2233e00700 1 --2- 172.21.15.1:0/1128527317 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560c3f08c860 0x560c3f08ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:05.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.582+0000 7f2233e00700 1 -- 172.21.15.1:0/1128527317 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c3ee54430 con 0x560c3f081090 2024-02-02T15:29:05.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.582+0000 7f2230c00700 1 --2- 172.21.15.1:0/2207565424 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f22140a0690 0x7f22140a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:05.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.582+0000 7f2233e00700 1 --2- 172.21.15.1:0/1128527317 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c3f081090 0x560c3f081470 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f222c0116a0 tx=0x7f222c0099e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d4726acbf5cc93a server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:05.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.582+0000 7f2230c00700 1 -- 172.21.15.1:0/2207565424 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f221c014070 con 0x7f22341083b0 2024-02-02T15:29:05.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.582+0000 7f2233400700 1 --2- 172.21.15.1:0/2207565424 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f22140a0690 0x7f22140a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:05.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.582+0000 7f2228c00700 1 -- 172.21.15.1:0/1128527317 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f222c00e040 con 0x560c3f081090 2024-02-02T15:29:05.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.582+0000 7f2228c00700 1 -- 172.21.15.1:0/1128527317 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f222c00f040 con 0x560c3f081090 2024-02-02T15:29:05.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.582+0000 7f2228c00700 1 -- 172.21.15.1:0/1128527317 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f222c004720 con 0x560c3f081090 2024-02-02T15:29:05.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.583+0000 7f2233400700 1 --2- 172.21.15.1:0/2207565424 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f22140a0690 0x7f22140a2b50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2224005e90 tx=0x7f2224019040 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:05.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.583+0000 7f224e072b80 1 -- 172.21.15.1:0/1128527317 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c3f081090 msgr2=0x560c3f081470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:05.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.583+0000 7f224e072b80 1 --2- 172.21.15.1:0/1128527317 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c3f081090 0x560c3f081470 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f222c0116a0 tx=0x7f222c0099e0 comp rx=0 tx=0).stop 2024-02-02T15:29:05.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.583+0000 7f224e072b80 1 -- 172.21.15.1:0/1128527317 shutdown_connections 2024-02-02T15:29:05.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.583+0000 7f224e072b80 1 --2- 172.21.15.1:0/1128527317 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560c3f08c860 0x560c3f08ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:05.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.583+0000 7f224e072b80 1 --2- 172.21.15.1:0/1128527317 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c3f081090 0x560c3f081470 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:05.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.583+0000 7f224e072b80 1 --2- 172.21.15.1:0/1128527317 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560c3f0819b0 0x560c3f08c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:05.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.583+0000 7f224e072b80 1 -- 172.21.15.1:0/1128527317 >> 172.21.15.1:0/1128527317 conn(0x560c3efe0e30 msgr2=0x560c3f07d430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:05.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.583+0000 7f224e072b80 1 -- 172.21.15.1:0/1128527317 shutdown_connections 2024-02-02T15:29:05.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.583+0000 7f224e072b80 1 -- 172.21.15.1:0/1128527317 wait complete. 2024-02-02T15:29:05.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.584+0000 7f224e072b80 1 -- start start 2024-02-02T15:29:05.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.584+0000 7f224e072b80 1 Processor -- start 2024-02-02T15:29:05.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.585+0000 7f224e072b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560c3f081090 0x560c3f08c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:05.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.585+0000 7f224e072b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c3f0819b0 0x560c3f089480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:05.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.585+0000 7f224e072b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560c3f08c860 0x560c3f089ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:05.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.585+0000 7f224e072b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c3efd2dd0 con 0x560c3f081090 2024-02-02T15:29:05.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.585+0000 7f224e072b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c3ef6ee00 con 0x560c3f0819b0 2024-02-02T15:29:05.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.585+0000 7f224e072b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c3ef6eaa0 con 0x560c3f08c860 2024-02-02T15:29:05.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.585+0000 7f224e072b80 10 client.0 _ll_register_callbacks cb 0x560c3ef70310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:29:05.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.585+0000 7f2233400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560c3f081090 0x560c3f08c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:05.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.585+0000 7f222be00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c3f0819b0 0x560c3f089480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:05.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.585+0000 7f222be00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c3f0819b0 0x560c3f089480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:45830/0 (socket says 172.21.15.1:45830) 2024-02-02T15:29:05.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.585+0000 7f222be00700 1 -- 172.21.15.1:0/1210141474 learned_addr learned my addr 172.21.15.1:0/1210141474 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:29:05.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.585+0000 7f2233e00700 1 --2- 172.21.15.1:0/1210141474 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560c3f08c860 0x560c3f089ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:05.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.586+0000 7f222be00700 1 -- 172.21.15.1:0/1210141474 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560c3f08c860 msgr2=0x560c3f089ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:05.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.586+0000 7f222be00700 1 --2- 172.21.15.1:0/1210141474 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560c3f08c860 0x560c3f089ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:05.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.586+0000 7f222be00700 1 -- 172.21.15.1:0/1210141474 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560c3f081090 msgr2=0x560c3f08c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:05.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.586+0000 7f222be00700 1 --2- 172.21.15.1:0/1210141474 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560c3f081090 0x560c3f08c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:05.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.586+0000 7f222be00700 1 -- 172.21.15.1:0/1210141474 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f222c011350 con 0x560c3f0819b0 2024-02-02T15:29:05.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.586+0000 7f2233400700 1 --2- 172.21.15.1:0/1210141474 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560c3f081090 0x560c3f08c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:29:05.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.586+0000 7f222be00700 1 --2- 172.21.15.1:0/1210141474 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c3f0819b0 0x560c3f089480 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f221c0c9860 tx=0x7f221c0cfef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4091fb085da8f124 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:05.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.586+0000 7f2221600700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:29:05.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.586+0000 7f2221600700 1 -- 172.21.15.1:0/1210141474 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f221c099110 con 0x560c3f0819b0 2024-02-02T15:29:05.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.586+0000 7f224e072b80 1 -- 172.21.15.1:0/1210141474 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560c3f075170 con 0x560c3f0819b0 2024-02-02T15:29:05.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.586+0000 7f2221600700 1 -- 172.21.15.1:0/1210141474 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f221c099cd0 con 0x560c3f0819b0 2024-02-02T15:29:05.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.586+0000 7f2221600700 1 -- 172.21.15.1:0/1210141474 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f221c098f30 con 0x560c3f0819b0 2024-02-02T15:29:05.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.587+0000 7f224e072b80 10 client.26246 fetch_fsmap learned FSMap version 29 2024-02-02T15:29:05.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.587+0000 7f224e072b80 1 -- 172.21.15.1:0/1210141474 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x560c3ee00be0 con 0x560c3f0819b0 2024-02-02T15:29:05.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.587+0000 7f2221600700 1 -- 172.21.15.1:0/1210141474 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560c3f075170 con 0x560c3f0819b0 2024-02-02T15:29:05.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.587+0000 7f2221600700 1 -- 172.21.15.1:0/1210141474 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 29) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f221c0840b0 con 0x560c3f0819b0 2024-02-02T15:29:05.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.587+0000 7f224e072b80 10 client.26246 fetch_fsmap finished waiting for FSMap version 29 2024-02-02T15:29:05.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.587+0000 7f224e072b80 10 client.26246 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:29:05.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.587+0000 7f224e072b80 20 client.26246 populate_metadata read hostname 'smithi001' 2024-02-02T15:29:05.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.587+0000 7f224e072b80 1 --2- 172.21.15.1:0/1210141474 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x560c3f09c200 0x560c3f09e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:05.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.587+0000 7f224e072b80 4 client.26246 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:29:05.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.587+0000 7f224e072b80 1 -- 172.21.15.1:0/1210141474 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560c3efcbe50 con 0x560c3f09c200 2024-02-02T15:29:05.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.588+0000 7f2233400700 1 --2- 172.21.15.1:0/1210141474 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x560c3f09c200 0x560c3f09e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:05.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.588+0000 7f2233400700 1 --2- 172.21.15.1:0/1210141474 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x560c3f09c200 0x560c3f09e620 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=810be3b5e2f802e0 server_cookie=afaf66b38a444bc9 in_seq=0 out_seq=0 2024-02-02T15:29:05.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.588+0000 7f2221600700 10 client.26246 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:29:05.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.589+0000 7f2221600700 1 -- 172.21.15.1:0/1210141474 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x560c3efcbe50 con 0x560c3f09c200 2024-02-02T15:29:05.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.589+0000 7f2221600700 10 client.26246 handle_command_reply: tid=0 2024-02-02T15:29:05.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.589+0000 7f224e072b80 1 client.26246 shutdown 2024-02-02T15:29:05.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.589+0000 7f224e072b80 1 -- 172.21.15.1:0/1210141474 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c3f0819b0 msgr2=0x560c3f089480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:05.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.589+0000 7f224e072b80 1 --2- 172.21.15.1:0/1210141474 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c3f0819b0 0x560c3f089480 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f221c0c9860 tx=0x7f221c0cfef0 comp rx=0 tx=0).stop 2024-02-02T15:29:05.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.589+0000 7f224e072b80 1 -- 172.21.15.1:0/1210141474 shutdown_connections 2024-02-02T15:29:05.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.589+0000 7f224e072b80 1 --2- 172.21.15.1:0/1210141474 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x560c3f09c200 0x560c3f09e620 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:05.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.589+0000 7f224e072b80 1 --2- 172.21.15.1:0/1210141474 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560c3f08c860 0x560c3f089ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:05.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.589+0000 7f224e072b80 1 --2- 172.21.15.1:0/1210141474 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560c3f081090 0x560c3f08c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:05.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.589+0000 7f224e072b80 1 --2- 172.21.15.1:0/1210141474 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c3f0819b0 0x560c3f089480 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:05.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.589+0000 7f2221600700 0 client.26246 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:29:05.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.589+0000 7f224e072b80 1 -- 172.21.15.1:0/1210141474 >> 172.21.15.1:0/1210141474 conn(0x560c3efe0e30 msgr2=0x560c3f07d020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:05.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.589+0000 7f224e072b80 1 -- 172.21.15.1:0/1210141474 shutdown_connections 2024-02-02T15:29:05.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.589+0000 7f224e072b80 1 -- 172.21.15.1:0/1210141474 wait complete. 2024-02-02T15:29:05.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.589+0000 7f224e072b80 20 client.26246 trim_cache size 0 max 16384 2024-02-02T15:29:05.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.609+0000 7f224e072b80 1 Processor -- start 2024-02-02T15:29:05.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.609+0000 7f224e072b80 1 -- start start 2024-02-02T15:29:05.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.610+0000 7f224e072b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c3f081090 0x560c3f08bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:05.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.610+0000 7f224e072b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560c3f0819b0 0x560c3f086750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:05.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.610+0000 7f224e072b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560c3f08c860 0x560c3f086c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:05.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.610+0000 7f224e072b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c3ee56720 con 0x560c3f08c860 2024-02-02T15:29:05.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.610+0000 7f224e072b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c3f0822e0 con 0x560c3f081090 2024-02-02T15:29:05.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.610+0000 7f224e072b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c3f08a090 con 0x560c3f0819b0 2024-02-02T15:29:05.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.610+0000 7f222be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560c3f0819b0 0x560c3f086750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:05.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.610+0000 7f2233400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c3f081090 0x560c3f08bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:05.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.610+0000 7f222be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560c3f0819b0 0x560c3f086750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:54716/0 (socket says 172.21.15.1:54716) 2024-02-02T15:29:05.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.610+0000 7f2233400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c3f081090 0x560c3f08bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:45832/0 (socket says 172.21.15.1:45832) 2024-02-02T15:29:05.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.610+0000 7f222be00700 1 -- 172.21.15.1:0/2210324588 learned_addr learned my addr 172.21.15.1:0/2210324588 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:29:05.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.610+0000 7f2233e00700 1 --2- 172.21.15.1:0/2210324588 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560c3f08c860 0x560c3f086c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:05.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.611+0000 7f2233400700 1 -- 172.21.15.1:0/2210324588 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560c3f0819b0 msgr2=0x560c3f086750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:05.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.611+0000 7f2233400700 1 --2- 172.21.15.1:0/2210324588 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560c3f0819b0 0x560c3f086750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:05.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.611+0000 7f2233400700 1 -- 172.21.15.1:0/2210324588 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560c3f08c860 msgr2=0x560c3f086c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:05.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.611+0000 7f2233400700 1 --2- 172.21.15.1:0/2210324588 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560c3f08c860 0x560c3f086c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:05.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.611+0000 7f2233400700 1 -- 172.21.15.1:0/2210324588 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c3f088ff0 con 0x560c3f081090 2024-02-02T15:29:05.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.611+0000 7f222be00700 1 --2- 172.21.15.1:0/2210324588 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560c3f0819b0 0x560c3f086750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:29:05.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.611+0000 7f2233e00700 1 --2- 172.21.15.1:0/2210324588 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560c3f08c860 0x560c3f086c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:29:05.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.611+0000 7f2233400700 1 --2- 172.21.15.1:0/2210324588 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c3f081090 0x560c3f08bda0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2224009810 tx=0x7f2224021000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1fccf57812dc009b server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:05.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.612+0000 7f2229600700 1 -- 172.21.15.1:0/2210324588 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f222401c040 con 0x560c3f081090 2024-02-02T15:29:05.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.612+0000 7f2229600700 1 -- 172.21.15.1:0/2210324588 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f222401d040 con 0x560c3f081090 2024-02-02T15:29:05.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.612+0000 7f2229600700 1 -- 172.21.15.1:0/2210324588 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2224005610 con 0x560c3f081090 2024-02-02T15:29:05.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.612+0000 7f224e072b80 1 -- 172.21.15.1:0/2210324588 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c3f081090 msgr2=0x560c3f08bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:05.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.612+0000 7f224e072b80 1 --2- 172.21.15.1:0/2210324588 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c3f081090 0x560c3f08bda0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2224009810 tx=0x7f2224021000 comp rx=0 tx=0).stop 2024-02-02T15:29:05.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.612+0000 7f224e072b80 1 -- 172.21.15.1:0/2210324588 shutdown_connections 2024-02-02T15:29:05.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.612+0000 7f224e072b80 1 --2- 172.21.15.1:0/2210324588 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560c3f08c860 0x560c3f086c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:05.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.613+0000 7f224e072b80 1 --2- 172.21.15.1:0/2210324588 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c3f081090 0x560c3f08bda0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:05.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.613+0000 7f224e072b80 1 --2- 172.21.15.1:0/2210324588 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560c3f0819b0 0x560c3f086750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:05.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.613+0000 7f224e072b80 1 -- 172.21.15.1:0/2210324588 >> 172.21.15.1:0/2210324588 conn(0x560c3efe0e30 msgr2=0x560c3f084820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:05.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:29:05 smithi071 ceph-mon[44707]: pgmap v65: 129 pgs: 129 active+clean; 52 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 2.5 MiB/s wr, 156 op/s 2024-02-02T15:29:05.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.613+0000 7f224e072b80 1 -- 172.21.15.1:0/2210324588 shutdown_connections 2024-02-02T15:29:05.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.613+0000 7f224e072b80 1 -- 172.21.15.1:0/2210324588 wait complete. 2024-02-02T15:29:05.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.614+0000 7f224e072b80 1 -- start start 2024-02-02T15:29:05.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.614+0000 7f224e072b80 1 Processor -- start 2024-02-02T15:29:05.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.614+0000 7f224e072b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560c3f081090 0x560c3f091aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:05.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.614+0000 7f224e072b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560c3f0819b0 0x560c3f091fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:05.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.614+0000 7f224e072b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c3f08c860 0x560c3f092520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:05.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.614+0000 7f224e072b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c3f0ede20 con 0x560c3f081090 2024-02-02T15:29:05.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.614+0000 7f224e072b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c3f0edf90 con 0x560c3f08c860 2024-02-02T15:29:05.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.614+0000 7f224e072b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c3f0ee150 con 0x560c3f0819b0 2024-02-02T15:29:05.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.614+0000 7f224e072b80 10 client.0 _ll_register_callbacks cb 0x560c3ef70310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:29:05.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.615+0000 7f222be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560c3f0819b0 0x560c3f091fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:05.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.615+0000 7f2233400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560c3f081090 0x560c3f091aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:05.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.615+0000 7f222be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560c3f0819b0 0x560c3f091fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:54726/0 (socket says 172.21.15.1:54726) 2024-02-02T15:29:05.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.615+0000 7f222be00700 1 -- 172.21.15.1:0/439220771 learned_addr learned my addr 172.21.15.1:0/439220771 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:29:05.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.615+0000 7f2233e00700 1 --2- 172.21.15.1:0/439220771 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c3f08c860 0x560c3f092520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:05.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.615+0000 7f222be00700 1 -- 172.21.15.1:0/439220771 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c3f08c860 msgr2=0x560c3f092520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:05.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.615+0000 7f222be00700 1 --2- 172.21.15.1:0/439220771 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c3f08c860 0x560c3f092520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:05.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.615+0000 7f222be00700 1 -- 172.21.15.1:0/439220771 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560c3f081090 msgr2=0x560c3f091aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:05.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.615+0000 7f222be00700 1 --2- 172.21.15.1:0/439220771 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560c3f081090 0x560c3f091aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:05.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.615+0000 7f222be00700 1 -- 172.21.15.1:0/439220771 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560c3ee00be0 con 0x560c3f0819b0 2024-02-02T15:29:05.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.615+0000 7f222be00700 1 --2- 172.21.15.1:0/439220771 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560c3f0819b0 0x560c3f091fe0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f221c009950 tx=0x7f221c000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d95a461485f82edd server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:05.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.615+0000 7f2221600700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:29:05.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.616+0000 7f2221600700 1 -- 172.21.15.1:0/439220771 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f221c099110 con 0x560c3f0819b0 2024-02-02T15:29:05.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.616+0000 7f2221600700 1 -- 172.21.15.1:0/439220771 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f221c099cd0 con 0x560c3f0819b0 2024-02-02T15:29:05.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.616+0000 7f224e072b80 1 -- 172.21.15.1:0/439220771 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f222c00f040 con 0x560c3f0819b0 2024-02-02T15:29:05.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.616+0000 7f2221600700 1 -- 172.21.15.1:0/439220771 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f221c098f30 con 0x560c3f0819b0 2024-02-02T15:29:05.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.616+0000 7f224e072b80 10 client.26242 fetch_fsmap learned FSMap version 29 2024-02-02T15:29:05.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.616+0000 7f224e072b80 1 -- 172.21.15.1:0/439220771 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x560c3ee54430 con 0x560c3f0819b0 2024-02-02T15:29:05.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.616+0000 7f2221600700 1 -- 172.21.15.1:0/439220771 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f222c00f040 con 0x560c3f0819b0 2024-02-02T15:29:05.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.616+0000 7f2221600700 1 -- 172.21.15.1:0/439220771 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 29) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f221c0840b0 con 0x560c3f0819b0 2024-02-02T15:29:05.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.617+0000 7f224e072b80 10 client.26242 fetch_fsmap finished waiting for FSMap version 29 2024-02-02T15:29:05.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.617+0000 7f224e072b80 10 client.26242 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:29:05.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.617+0000 7f224e072b80 20 client.26242 populate_metadata read hostname 'smithi001' 2024-02-02T15:29:05.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.617+0000 7f224e072b80 1 --2- 172.21.15.1:0/439220771 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x560c3f0ef410 0x560c3f0f1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:05.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.617+0000 7f224e072b80 4 client.26242 mds_command: new command op to 24557 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-02T15:29:05.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.617+0000 7f224e072b80 1 -- 172.21.15.1:0/439220771 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x560c3ef91fd0 con 0x560c3f0ef410 2024-02-02T15:29:05.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.617+0000 7f2233400700 1 --2- 172.21.15.1:0/439220771 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x560c3f0ef410 0x560c3f0f1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:05.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.618+0000 7f2233400700 1 --2- 172.21.15.1:0/439220771 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x560c3f0ef410 0x560c3f0f1800 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4f55a2bbc130978 server_cookie=469d8296035bd064 in_seq=0 out_seq=0 2024-02-02T15:29:05.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.618+0000 7f2221600700 10 client.26242 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:29:05.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.619+0000 7f2221600700 1 -- 172.21.15.1:0/439220771 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x560c3ef91fd0 con 0x560c3f0ef410 2024-02-02T15:29:05.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.619+0000 7f2221600700 10 client.26242 handle_command_reply: tid=0 2024-02-02T15:29:05.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.619+0000 7f224e072b80 1 client.26242 shutdown 2024-02-02T15:29:05.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.619+0000 7f224e072b80 1 -- 172.21.15.1:0/439220771 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560c3f0819b0 msgr2=0x560c3f091fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:05.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.619+0000 7f224e072b80 1 --2- 172.21.15.1:0/439220771 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560c3f0819b0 0x560c3f091fe0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f221c009950 tx=0x7f221c000bc0 comp rx=0 tx=0).stop 2024-02-02T15:29:05.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.619+0000 7f224e072b80 1 -- 172.21.15.1:0/439220771 shutdown_connections 2024-02-02T15:29:05.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.619+0000 7f224e072b80 1 --2- 172.21.15.1:0/439220771 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x560c3f0ef410 0x560c3f0f1800 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:05.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.619+0000 7f224e072b80 1 --2- 172.21.15.1:0/439220771 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560c3f08c860 0x560c3f092520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:05.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.619+0000 7f224e072b80 1 --2- 172.21.15.1:0/439220771 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560c3f081090 0x560c3f091aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:05.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.619+0000 7f224e072b80 1 --2- 172.21.15.1:0/439220771 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560c3f0819b0 0x560c3f091fe0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:05.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.619+0000 7f224e072b80 1 -- 172.21.15.1:0/439220771 >> 172.21.15.1:0/439220771 conn(0x560c3efe0e30 msgr2=0x560c3f087c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:05.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.619+0000 7f2221600700 0 client.26242 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:29:05.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.619+0000 7f224e072b80 1 -- 172.21.15.1:0/439220771 shutdown_connections 2024-02-02T15:29:05.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.620+0000 7f224e072b80 1 -- 172.21.15.1:0/439220771 wait complete. 2024-02-02T15:29:05.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.620+0000 7f224e072b80 20 client.26242 trim_cache size 0 max 16384 2024-02-02T15:29:05.624 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:29:05.624 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2024-02-02T15:29:05.624 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "8ec6fa0b-3ac6-4ac6-9a64-5e6170e2f035", 2024-02-02T15:29:05.624 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2024-02-02T15:29:05.625 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:29:05.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.620+0000 7f223a600700 1 -- 172.21.15.1:0/2207565424 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f22140a0690 msgr2=0x7f22140a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:05.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.620+0000 7f223a600700 1 --2- 172.21.15.1:0/2207565424 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f22140a0690 0x7f22140a2b50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2224005e90 tx=0x7f2224019040 comp rx=0 tx=0).stop 2024-02-02T15:29:05.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.620+0000 7f223a600700 1 -- 172.21.15.1:0/2207565424 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f22341083b0 msgr2=0x7f2234199810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:05.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.620+0000 7f223a600700 1 --2- 172.21.15.1:0/2207565424 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f22341083b0 0x7f2234199810 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f221c00ea30 tx=0x7f221c00ed40 comp rx=0 tx=0).stop 2024-02-02T15:29:05.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.620+0000 7f223a600700 1 -- 172.21.15.1:0/2207565424 shutdown_connections 2024-02-02T15:29:05.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.620+0000 7f223a600700 1 --2- 172.21.15.1:0/2207565424 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f22140a0690 0x7f22140a2b50 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:05.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.620+0000 7f223a600700 1 --2- 172.21.15.1:0/2207565424 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f223419dc70 0x7f223419e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:05.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.620+0000 7f223a600700 1 --2- 172.21.15.1:0/2207565424 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2234103290 0x7f22341992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:05.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.620+0000 7f223a600700 1 --2- 172.21.15.1:0/2207565424 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f22341083b0 0x7f2234199810 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:05.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.620+0000 7f223a600700 1 -- 172.21.15.1:0/2207565424 >> 172.21.15.1:0/2207565424 conn(0x7f22340feb30 msgr2=0x7f2234106c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:05.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.621+0000 7f223a600700 1 -- 172.21.15.1:0/2207565424 shutdown_connections 2024-02-02T15:29:05.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.621+0000 7f223a600700 1 -- 172.21.15.1:0/2207565424 wait complete. 2024-02-02T15:29:05.637 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:29:05 smithi001 ceph-mon[35470]: pgmap v65: 129 pgs: 129 active+clean; 52 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 2.5 MiB/s wr, 156 op/s 2024-02-02T15:29:05.641 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-02T15:29:05.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:29:05 smithi142 ceph-mon[45728]: pgmap v65: 129 pgs: 129 active+clean; 52 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 2.5 MiB/s wr, 156 op/s 2024-02-02T15:29:05.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.854+0000 7f5185600700 1 -- 172.21.15.1:0/365235057 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5180073110 msgr2=0x7f51800734f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:05.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.854+0000 7f5185600700 1 --2- 172.21.15.1:0/365235057 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5180073110 0x7f51800734f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5170009990 tx=0x7f5170009ca0 comp rx=0 tx=0).stop 2024-02-02T15:29:05.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.854+0000 7f5185600700 1 -- 172.21.15.1:0/365235057 shutdown_connections 2024-02-02T15:29:05.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.854+0000 7f5185600700 1 --2- 172.21.15.1:0/365235057 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f518010d320 0x7f518010f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:05.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.854+0000 7f5185600700 1 --2- 172.21.15.1:0/365235057 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5180073110 0x7f51800734f0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:05.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.854+0000 7f5185600700 1 --2- 172.21.15.1:0/365235057 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5180073a30 0x7f518010cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:05.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.854+0000 7f5185600700 1 -- 172.21.15.1:0/365235057 >> 172.21.15.1:0/365235057 conn(0x7f51800fc570 msgr2=0x7f51800fe990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:05.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.854+0000 7f5185600700 1 -- 172.21.15.1:0/365235057 shutdown_connections 2024-02-02T15:29:05.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.854+0000 7f5185600700 1 -- 172.21.15.1:0/365235057 wait complete. 2024-02-02T15:29:05.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.856+0000 7f5185600700 1 Processor -- start 2024-02-02T15:29:05.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.856+0000 7f5185600700 1 -- start start 2024-02-02T15:29:05.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.856+0000 7f5185600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5180073110 0x7f5180198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:05.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.856+0000 7f5185600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5180073a30 0x7f5180199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:05.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.856+0000 7f5185600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f518010d320 0x7f518019d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:05.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.856+0000 7f5185600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f518019dec0 con 0x7f518010d320 2024-02-02T15:29:05.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.856+0000 7f5185600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f518019e030 con 0x7f5180073a30 2024-02-02T15:29:05.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.856+0000 7f5185600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f518019e1f0 con 0x7f5180073110 2024-02-02T15:29:05.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.856+0000 7f517f400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5180073a30 0x7f5180199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:05.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.856+0000 7f517f400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5180073a30 0x7f5180199420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:45868/0 (socket says 172.21.15.1:45868) 2024-02-02T15:29:05.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.856+0000 7f517fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5180073110 0x7f5180198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:05.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.857+0000 7f517f400700 1 -- 172.21.15.1:0/3283633548 learned_addr learned my addr 172.21.15.1:0/3283633548 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:29:05.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.857+0000 7f5184c00700 1 --2- 172.21.15.1:0/3283633548 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f518010d320 0x7f518019d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:05.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.857+0000 7f517f400700 1 -- 172.21.15.1:0/3283633548 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5180073110 msgr2=0x7f5180198ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:05.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.857+0000 7f517f400700 1 --2- 172.21.15.1:0/3283633548 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5180073110 0x7f5180198ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:05.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.857+0000 7f517f400700 1 -- 172.21.15.1:0/3283633548 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f518010d320 msgr2=0x7f518019d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:05.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.857+0000 7f517f400700 1 --2- 172.21.15.1:0/3283633548 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f518010d320 0x7f518019d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:05.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.857+0000 7f517f400700 1 -- 172.21.15.1:0/3283633548 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51700095f0 con 0x7f5180073a30 2024-02-02T15:29:05.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.857+0000 7f517fe00700 1 --2- 172.21.15.1:0/3283633548 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5180073110 0x7f5180198ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:29:05.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.857+0000 7f517f400700 1 --2- 172.21.15.1:0/3283633548 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5180073a30 0x7f5180199420 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f516800e9a0 tx=0x7f516800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:05.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.857+0000 7f517cc00700 1 -- 172.21.15.1:0/3283633548 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5168009790 con 0x7f5180073a30 2024-02-02T15:29:05.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.857+0000 7f517cc00700 1 -- 172.21.15.1:0/3283633548 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5168010430 con 0x7f5180073a30 2024-02-02T15:29:05.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.857+0000 7f5185600700 1 -- 172.21.15.1:0/3283633548 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f518019e4d0 con 0x7f5180073a30 2024-02-02T15:29:05.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.857+0000 7f517cc00700 1 -- 172.21.15.1:0/3283633548 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5168018680 con 0x7f5180073a30 2024-02-02T15:29:05.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.858+0000 7f5185600700 1 -- 172.21.15.1:0/3283633548 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51801a5d10 con 0x7f5180073a30 2024-02-02T15:29:05.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.861+0000 7f517cc00700 1 -- 172.21.15.1:0/3283633548 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 27) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f51680105a0 con 0x7f5180073a30 2024-02-02T15:29:05.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.862+0000 7f519a62bb80 1 Processor -- start 2024-02-02T15:29:05.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.863+0000 7f519a62bb80 1 -- start start 2024-02-02T15:29:05.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.863+0000 7f517cc00700 1 --2- 172.21.15.1:0/3283633548 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f51600a0710 0x7f51600a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:05.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.863+0000 7f519a62bb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c067ea21f0 0x55c067ea25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:05.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.863+0000 7f517cc00700 1 -- 172.21.15.1:0/3283633548 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f5168014070 con 0x7f5180073a30 2024-02-02T15:29:05.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.863+0000 7f519a62bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c067ea2b10 0x55c067ead4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:05.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.863+0000 7f519a62bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c067ead9f0 0x55c067eafde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:05.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.863+0000 7f519a62bb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c067e164c0 con 0x55c067ea21f0 2024-02-02T15:29:05.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.863+0000 7f519a62bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c067dfaaa0 con 0x55c067ea2b10 2024-02-02T15:29:05.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.863+0000 7f5184c00700 1 --2- 172.21.15.1:0/3283633548 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f51600a0710 0x7f51600a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:05.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.863+0000 7f519a62bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c067df8460 con 0x55c067ead9f0 2024-02-02T15:29:05.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.863+0000 7f517f400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c067ea2b10 0x55c067ead4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:05.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.863+0000 7f517f400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c067ea2b10 0x55c067ead4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:45882/0 (socket says 172.21.15.1:45882) 2024-02-02T15:29:05.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.863+0000 7f517f400700 1 -- 172.21.15.1:0/4161210036 learned_addr learned my addr 172.21.15.1:0/4161210036 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:29:05.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.863+0000 7f5184c00700 1 --2- 172.21.15.1:0/4161210036 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c067ead9f0 0x55c067eafde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:05.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.864+0000 7f517fe00700 1 --2- 172.21.15.1:0/4161210036 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c067ea21f0 0x55c067ea25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:05.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.864+0000 7f517f400700 1 -- 172.21.15.1:0/4161210036 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c067ead9f0 msgr2=0x55c067eafde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:05.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.864+0000 7f517f400700 1 --2- 172.21.15.1:0/4161210036 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c067ead9f0 0x55c067eafde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:05.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.864+0000 7f517f400700 1 -- 172.21.15.1:0/4161210036 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c067ea21f0 msgr2=0x55c067ea25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:05.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.864+0000 7f517f400700 1 --2- 172.21.15.1:0/4161210036 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c067ea21f0 0x55c067ea25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:05.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.864+0000 7f517f400700 1 -- 172.21.15.1:0/4161210036 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c067c753b0 con 0x55c067ea2b10 2024-02-02T15:29:05.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.864+0000 7f517fe00700 1 --2- 172.21.15.1:0/4161210036 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c067ea21f0 0x55c067ea25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:29:05.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.864+0000 7f517f400700 1 --2- 172.21.15.1:0/4161210036 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c067ea2b10 0x55c067ead4b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5168093fd0 tx=0x7f5168098e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ca7337837ada481 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:05.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.864+0000 7f5184c00700 1 --2- 172.21.15.1:0/3283633548 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f51600a0710 0x7f51600a2bd0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5178009ec0 tx=0x7f5178009920 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:05.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.864+0000 7f5175200700 1 -- 172.21.15.1:0/4161210036 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51680993c0 con 0x55c067ea2b10 2024-02-02T15:29:05.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.864+0000 7f5175200700 1 -- 172.21.15.1:0/4161210036 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f516809d1f0 con 0x55c067ea2b10 2024-02-02T15:29:05.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.864+0000 7f5175200700 1 -- 172.21.15.1:0/4161210036 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5168099860 con 0x55c067ea2b10 2024-02-02T15:29:05.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.865+0000 7f519a62bb80 1 -- 172.21.15.1:0/4161210036 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c067ea2b10 msgr2=0x55c067ead4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:05.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.865+0000 7f519a62bb80 1 --2- 172.21.15.1:0/4161210036 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c067ea2b10 0x55c067ead4b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5168093fd0 tx=0x7f5168098e10 comp rx=0 tx=0).stop 2024-02-02T15:29:05.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.865+0000 7f519a62bb80 1 -- 172.21.15.1:0/4161210036 shutdown_connections 2024-02-02T15:29:05.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.865+0000 7f519a62bb80 1 --2- 172.21.15.1:0/4161210036 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c067ead9f0 0x55c067eafde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:05.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.865+0000 7f519a62bb80 1 --2- 172.21.15.1:0/4161210036 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c067ea21f0 0x55c067ea25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:05.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.865+0000 7f519a62bb80 1 --2- 172.21.15.1:0/4161210036 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c067ea2b10 0x55c067ead4b0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:05.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.865+0000 7f519a62bb80 1 -- 172.21.15.1:0/4161210036 >> 172.21.15.1:0/4161210036 conn(0x55c067e01db0 msgr2=0x55c067e9e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:05.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.865+0000 7f519a62bb80 1 -- 172.21.15.1:0/4161210036 shutdown_connections 2024-02-02T15:29:05.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.865+0000 7f519a62bb80 1 -- 172.21.15.1:0/4161210036 wait complete. 2024-02-02T15:29:05.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.866+0000 7f519a62bb80 1 -- start start 2024-02-02T15:29:05.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.866+0000 7f519a62bb80 1 Processor -- start 2024-02-02T15:29:05.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.866+0000 7f519a62bb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c067ea21f0 0x55c067ead420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:05.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.866+0000 7f519a62bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c067ea2b10 0x55c067eaa6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:05.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.866+0000 7f519a62bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c067ead9f0 0x55c067eaade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:05.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.866+0000 7f519a62bb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c067df3d50 con 0x55c067ea21f0 2024-02-02T15:29:05.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.866+0000 7f519a62bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c067d8fd80 con 0x55c067ea2b10 2024-02-02T15:29:05.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.866+0000 7f519a62bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c067d8fa20 con 0x55c067ead9f0 2024-02-02T15:29:05.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.866+0000 7f519a62bb80 10 client.0 _ll_register_callbacks cb 0x55c067d91290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:29:05.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.867+0000 7f5184c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c067ead9f0 0x55c067eaade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:05.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.867+0000 7f517fe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c067ea21f0 0x55c067ead420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:05.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.867+0000 7f5184c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c067ead9f0 0x55c067eaade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:54766/0 (socket says 172.21.15.1:54766) 2024-02-02T15:29:05.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.867+0000 7f5184c00700 1 -- 172.21.15.1:0/1420189501 learned_addr learned my addr 172.21.15.1:0/1420189501 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:29:05.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.867+0000 7f517f400700 1 --2- 172.21.15.1:0/1420189501 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c067ea2b10 0x55c067eaa6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:05.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.867+0000 7f517f400700 1 -- 172.21.15.1:0/1420189501 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c067ead9f0 msgr2=0x55c067eaade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:05.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.867+0000 7f517f400700 1 --2- 172.21.15.1:0/1420189501 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c067ead9f0 0x55c067eaade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:05.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.867+0000 7f517f400700 1 -- 172.21.15.1:0/1420189501 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c067ea21f0 msgr2=0x55c067ead420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:05.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.867+0000 7f517f400700 1 --2- 172.21.15.1:0/1420189501 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c067ea21f0 0x55c067ead420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:05.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.867+0000 7f517f400700 1 -- 172.21.15.1:0/1420189501 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51801a5d10 con 0x55c067ea2b10 2024-02-02T15:29:05.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.868+0000 7f517f400700 1 --2- 172.21.15.1:0/1420189501 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c067ea2b10 0x55c067eaa6a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x55c067ea2f90 tx=0x7f5168010ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a287ef948b0ca4b server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:05.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.868+0000 7f516d600700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:29:05.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.868+0000 7f516d600700 1 -- 172.21.15.1:0/1420189501 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51680993c0 con 0x55c067ea2b10 2024-02-02T15:29:05.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.868+0000 7f516d600700 1 -- 172.21.15.1:0/1420189501 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f5168099830 con 0x55c067ea2b10 2024-02-02T15:29:05.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.868+0000 7f519a62bb80 1 -- 172.21.15.1:0/1420189501 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c067e96340 con 0x55c067ea2b10 2024-02-02T15:29:05.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.868+0000 7f516d600700 1 -- 172.21.15.1:0/1420189501 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f51680b7280 con 0x55c067ea2b10 2024-02-02T15:29:05.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.868+0000 7f519a62bb80 10 client.26264 fetch_fsmap learned FSMap version 29 2024-02-02T15:29:05.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.868+0000 7f519a62bb80 1 -- 172.21.15.1:0/1420189501 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55c067c21b60 con 0x55c067ea2b10 2024-02-02T15:29:05.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.868+0000 7f516d600700 1 -- 172.21.15.1:0/1420189501 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c067e96340 con 0x55c067ea2b10 2024-02-02T15:29:05.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.869+0000 7f516d600700 1 -- 172.21.15.1:0/1420189501 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 29) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f5168098320 con 0x55c067ea2b10 2024-02-02T15:29:05.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.869+0000 7f519a62bb80 10 client.26264 fetch_fsmap finished waiting for FSMap version 29 2024-02-02T15:29:05.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.869+0000 7f519a62bb80 10 client.26264 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:29:05.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.869+0000 7f519a62bb80 20 client.26264 populate_metadata read hostname 'smithi001' 2024-02-02T15:29:05.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.869+0000 7f519a62bb80 1 --2- 172.21.15.1:0/1420189501 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55c067ebcff0 0x55c067ebf410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:05.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.869+0000 7f519a62bb80 4 client.26264 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:29:05.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.869+0000 7f519a62bb80 1 -- 172.21.15.1:0/1420189501 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c067decdd0 con 0x55c067ebcff0 2024-02-02T15:29:05.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.869+0000 7f517fe00700 1 --2- 172.21.15.1:0/1420189501 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55c067ebcff0 0x55c067ebf410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:05.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.870+0000 7f516d600700 10 client.26264 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:29:05.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.870+0000 7f517fe00700 1 --2- 172.21.15.1:0/1420189501 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55c067ebcff0 0x55c067ebf410 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db44ab6605922ee3 server_cookie=bf454950be8e75cd in_seq=0 out_seq=0 2024-02-02T15:29:05.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.871+0000 7f516d600700 1 -- 172.21.15.1:0/1420189501 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55c067decdd0 con 0x55c067ebcff0 2024-02-02T15:29:05.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.871+0000 7f516d600700 10 client.26264 handle_command_reply: tid=0 2024-02-02T15:29:05.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.871+0000 7f519a62bb80 1 client.26264 shutdown 2024-02-02T15:29:05.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.871+0000 7f519a62bb80 1 -- 172.21.15.1:0/1420189501 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c067ea2b10 msgr2=0x55c067eaa6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:05.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.871+0000 7f519a62bb80 1 --2- 172.21.15.1:0/1420189501 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c067ea2b10 0x55c067eaa6a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x55c067ea2f90 tx=0x7f5168010ab0 comp rx=0 tx=0).stop 2024-02-02T15:29:05.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.871+0000 7f519a62bb80 1 -- 172.21.15.1:0/1420189501 shutdown_connections 2024-02-02T15:29:05.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.871+0000 7f519a62bb80 1 --2- 172.21.15.1:0/1420189501 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55c067ebcff0 0x55c067ebf410 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:05.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.871+0000 7f519a62bb80 1 --2- 172.21.15.1:0/1420189501 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c067ead9f0 0x55c067eaade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:05.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.871+0000 7f519a62bb80 1 --2- 172.21.15.1:0/1420189501 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c067ea21f0 0x55c067ead420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:05.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.871+0000 7f516d600700 0 client.26264 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:29:05.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.871+0000 7f519a62bb80 1 --2- 172.21.15.1:0/1420189501 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c067ea2b10 0x55c067eaa6a0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:05.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.871+0000 7f519a62bb80 1 -- 172.21.15.1:0/1420189501 >> 172.21.15.1:0/1420189501 conn(0x55c067e01db0 msgr2=0x55c067e9e150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:05.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.871+0000 7f519a62bb80 1 -- 172.21.15.1:0/1420189501 shutdown_connections 2024-02-02T15:29:05.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.871+0000 7f519a62bb80 1 -- 172.21.15.1:0/1420189501 wait complete. 2024-02-02T15:29:05.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.871+0000 7f519a62bb80 20 client.26264 trim_cache size 0 max 16384 2024-02-02T15:29:05.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.891+0000 7f519a62bb80 1 Processor -- start 2024-02-02T15:29:05.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.891+0000 7f519a62bb80 1 -- start start 2024-02-02T15:29:05.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.891+0000 7f519a62bb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c067ea21f0 0x55c067ea7950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:05.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.891+0000 7f519a62bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c067ea2b10 0x55c067ea7e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:05.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.891+0000 7f519a62bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c067ead9f0 0x55c067ea9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:05.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.891+0000 7f519a62bb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c067c776a0 con 0x55c067ea21f0 2024-02-02T15:29:05.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.892+0000 7f519a62bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c067ea3200 con 0x55c067ea2b10 2024-02-02T15:29:05.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.892+0000 7f519a62bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c067ea62e0 con 0x55c067ead9f0 2024-02-02T15:29:05.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.892+0000 7f5184c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c067ead9f0 0x55c067ea9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:05.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.892+0000 7f517fe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c067ea21f0 0x55c067ea7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:05.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.892+0000 7f517f400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c067ea2b10 0x55c067ea7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:05.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.892+0000 7f5184c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c067ead9f0 0x55c067ea9c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:54768/0 (socket says 172.21.15.1:54768) 2024-02-02T15:29:05.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.892+0000 7f5184c00700 1 -- 172.21.15.1:0/460189647 learned_addr learned my addr 172.21.15.1:0/460189647 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:29:05.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.892+0000 7f5184c00700 1 -- 172.21.15.1:0/460189647 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c067ea2b10 msgr2=0x55c067ea7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:05.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.892+0000 7f5184c00700 1 --2- 172.21.15.1:0/460189647 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c067ea2b10 0x55c067ea7e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:05.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.892+0000 7f5184c00700 1 -- 172.21.15.1:0/460189647 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c067ea21f0 msgr2=0x55c067ea7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:05.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.892+0000 7f5184c00700 1 --2- 172.21.15.1:0/460189647 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c067ea21f0 0x55c067ea7950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:05.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.892+0000 7f5184c00700 1 -- 172.21.15.1:0/460189647 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c067db9ea0 con 0x55c067ead9f0 2024-02-02T15:29:05.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.893+0000 7f5184c00700 1 --2- 172.21.15.1:0/460189647 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c067ead9f0 0x55c067ea9c30 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5178003cf0 tx=0x7f517800c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f31aebc50c71c172 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:05.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.893+0000 7f5175c00700 1 -- 172.21.15.1:0/460189647 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5178022050 con 0x55c067ead9f0 2024-02-02T15:29:05.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.893+0000 7f5175c00700 1 -- 172.21.15.1:0/460189647 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5178023040 con 0x55c067ead9f0 2024-02-02T15:29:05.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.893+0000 7f5175c00700 1 -- 172.21.15.1:0/460189647 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f517800e040 con 0x55c067ead9f0 2024-02-02T15:29:05.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.894+0000 7f519a62bb80 1 -- 172.21.15.1:0/460189647 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c067ead9f0 msgr2=0x55c067ea9c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:05.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.894+0000 7f519a62bb80 1 --2- 172.21.15.1:0/460189647 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c067ead9f0 0x55c067ea9c30 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5178003cf0 tx=0x7f517800c000 comp rx=0 tx=0).stop 2024-02-02T15:29:05.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.894+0000 7f519a62bb80 1 -- 172.21.15.1:0/460189647 shutdown_connections 2024-02-02T15:29:05.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.894+0000 7f519a62bb80 1 --2- 172.21.15.1:0/460189647 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c067ead9f0 0x55c067ea9c30 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:05.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.894+0000 7f519a62bb80 1 --2- 172.21.15.1:0/460189647 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c067ea21f0 0x55c067ea7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:05.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.894+0000 7f519a62bb80 1 --2- 172.21.15.1:0/460189647 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c067ea2b10 0x55c067ea7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:05.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.894+0000 7f519a62bb80 1 -- 172.21.15.1:0/460189647 >> 172.21.15.1:0/460189647 conn(0x55c067e01db0 msgr2=0x55c067eb8a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:05.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.894+0000 7f519a62bb80 1 -- 172.21.15.1:0/460189647 shutdown_connections 2024-02-02T15:29:05.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.894+0000 7f519a62bb80 1 -- 172.21.15.1:0/460189647 wait complete. 2024-02-02T15:29:05.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.894+0000 7f519a62bb80 1 -- start start 2024-02-02T15:29:05.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.894+0000 7f519a62bb80 1 Processor -- start 2024-02-02T15:29:05.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.895+0000 7f519a62bb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c067ea21f0 0x55c067eb43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:05.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.895+0000 7f519a62bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c067ea2b10 0x55c067eb2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:05.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.895+0000 7f519a62bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c067ead9f0 0x55c067eb2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:05.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.895+0000 7f519a62bb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c067eb33f0 con 0x55c067ea21f0 2024-02-02T15:29:05.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.895+0000 7f519a62bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c067eb3560 con 0x55c067ea2b10 2024-02-02T15:29:05.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.895+0000 7f519a62bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c067f0eb60 con 0x55c067ead9f0 2024-02-02T15:29:05.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.895+0000 7f519a62bb80 10 client.0 _ll_register_callbacks cb 0x55c067d91290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:29:05.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.895+0000 7f517fe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c067ea21f0 0x55c067eb43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:05.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.895+0000 7f517fe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c067ea21f0 0x55c067eb43d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:38062/0 (socket says 172.21.15.1:38062) 2024-02-02T15:29:05.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.895+0000 7f517fe00700 1 -- 172.21.15.1:0/1439609094 learned_addr learned my addr 172.21.15.1:0/1439609094 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:29:05.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.895+0000 7f5184c00700 1 --2- 172.21.15.1:0/1439609094 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c067ead9f0 0x55c067eb2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:05.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.895+0000 7f517f400700 1 --2- 172.21.15.1:0/1439609094 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c067ea2b10 0x55c067eb2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:05.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.895+0000 7f517fe00700 1 -- 172.21.15.1:0/1439609094 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c067ead9f0 msgr2=0x55c067eb2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:05.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.895+0000 7f517fe00700 1 --2- 172.21.15.1:0/1439609094 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c067ead9f0 0x55c067eb2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:05.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.895+0000 7f517fe00700 1 -- 172.21.15.1:0/1439609094 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c067ea2b10 msgr2=0x55c067eb2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:05.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.895+0000 7f517fe00700 1 --2- 172.21.15.1:0/1439609094 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c067ea2b10 0x55c067eb2690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:05.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.896+0000 7f517fe00700 1 -- 172.21.15.1:0/1439609094 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5178003a00 con 0x55c067ea21f0 2024-02-02T15:29:05.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.896+0000 7f5184c00700 1 --2- 172.21.15.1:0/1439609094 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c067ead9f0 0x55c067eb2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:29:05.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.896+0000 7f517fe00700 1 --2- 172.21.15.1:0/1439609094 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c067ea21f0 0x55c067eb43d0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f51700098e0 tx=0x7f517000be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bf53a7dd1d329d1 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:05.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.896+0000 7f516d600700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:29:05.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.896+0000 7f516d600700 1 -- 172.21.15.1:0/1439609094 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f517001d040 con 0x55c067ea21f0 2024-02-02T15:29:05.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.896+0000 7f516d600700 1 -- 172.21.15.1:0/1439609094 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f5170028900 con 0x55c067ea21f0 2024-02-02T15:29:05.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.896+0000 7f519a62bb80 1 -- 172.21.15.1:0/1439609094 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f516809d1f0 con 0x55c067ea21f0 2024-02-02T15:29:05.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.896+0000 7f516d600700 1 -- 172.21.15.1:0/1439609094 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5170028450 con 0x55c067ea21f0 2024-02-02T15:29:05.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.896+0000 7f519a62bb80 10 client.16215 fetch_fsmap learned FSMap version 29 2024-02-02T15:29:05.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.896+0000 7f519a62bb80 1 -- 172.21.15.1:0/1439609094 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55c067c79cd0 con 0x55c067ea21f0 2024-02-02T15:29:05.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.897+0000 7f516d600700 1 -- 172.21.15.1:0/1439609094 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f516809d1f0 con 0x55c067ea21f0 2024-02-02T15:29:05.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.897+0000 7f516d600700 1 -- 172.21.15.1:0/1439609094 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 29) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f517002e020 con 0x55c067ea21f0 2024-02-02T15:29:05.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.897+0000 7f519a62bb80 10 client.16215 fetch_fsmap finished waiting for FSMap version 29 2024-02-02T15:29:05.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.897+0000 7f519a62bb80 10 client.16215 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:29:05.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.897+0000 7f519a62bb80 20 client.16215 populate_metadata read hostname 'smithi001' 2024-02-02T15:29:05.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.897+0000 7f519a62bb80 1 --2- 172.21.15.1:0/1439609094 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55c067f10270 0x55c067f12690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:05.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.897+0000 7f519a62bb80 4 client.16215 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2024-02-02T15:29:05.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.897+0000 7f519a62bb80 1 -- 172.21.15.1:0/1439609094 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c067db2f50 con 0x55c067f10270 2024-02-02T15:29:05.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.897+0000 7f517f400700 1 --2- 172.21.15.1:0/1439609094 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55c067f10270 0x55c067f12690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:05.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.898+0000 7f517f400700 1 --2- 172.21.15.1:0/1439609094 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55c067f10270 0x55c067f12690 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8c8c650a7879126 server_cookie=6bc719da09c9bdd0 in_seq=0 out_seq=0 2024-02-02T15:29:05.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.898+0000 7f516d600700 10 client.16215 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:29:05.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.898+0000 7f516d600700 1 -- 172.21.15.1:0/1439609094 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55c067db2f50 con 0x55c067f10270 2024-02-02T15:29:05.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.898+0000 7f516d600700 10 client.16215 handle_command_reply: tid=0 2024-02-02T15:29:05.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.898+0000 7f519a62bb80 1 client.16215 shutdown 2024-02-02T15:29:05.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.898+0000 7f519a62bb80 1 -- 172.21.15.1:0/1439609094 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c067ea21f0 msgr2=0x55c067eb43d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:05.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.898+0000 7f519a62bb80 1 --2- 172.21.15.1:0/1439609094 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c067ea21f0 0x55c067eb43d0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f51700098e0 tx=0x7f517000be10 comp rx=0 tx=0).stop 2024-02-02T15:29:05.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.899+0000 7f519a62bb80 1 -- 172.21.15.1:0/1439609094 shutdown_connections 2024-02-02T15:29:05.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.899+0000 7f519a62bb80 1 --2- 172.21.15.1:0/1439609094 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55c067f10270 0x55c067f12690 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:05.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.899+0000 7f519a62bb80 1 --2- 172.21.15.1:0/1439609094 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c067ead9f0 0x55c067eb2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:05.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.899+0000 7f519a62bb80 1 --2- 172.21.15.1:0/1439609094 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c067ea21f0 0x55c067eb43d0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:05.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.899+0000 7f519a62bb80 1 --2- 172.21.15.1:0/1439609094 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c067ea2b10 0x55c067eb2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:05.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.899+0000 7f519a62bb80 1 -- 172.21.15.1:0/1439609094 >> 172.21.15.1:0/1439609094 conn(0x55c067e01db0 msgr2=0x55c067eabfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:05.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.899+0000 7f516d600700 0 client.16215 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:29:05.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.899+0000 7f519a62bb80 1 -- 172.21.15.1:0/1439609094 shutdown_connections 2024-02-02T15:29:05.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.899+0000 7f519a62bb80 1 -- 172.21.15.1:0/1439609094 wait complete. 2024-02-02T15:29:05.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.899+0000 7f519a62bb80 20 client.16215 trim_cache size 0 max 16384 2024-02-02T15:29:05.902 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:29:05.903 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-02T15:29:05.903 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2024-02-02T15:29:05.903 INFO:teuthology.orchestra.run.smithi001.stdout: "8ec6fa0b-3ac6-4ac6-9a64-5e6170e2f035": { 2024-02-02T15:29:05.903 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2024-02-02T15:29:05.903 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "8ec6fa0b-3ac6-4ac6-9a64-5e6170e2f035", 2024-02-02T15:29:05.903 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,force" 2024-02-02T15:29:05.903 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:29:05.903 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:29:05.903 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:29:05.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.900+0000 7f5185600700 1 -- 172.21.15.1:0/3283633548 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f51600a0710 msgr2=0x7f51600a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:05.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.900+0000 7f5185600700 1 --2- 172.21.15.1:0/3283633548 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f51600a0710 0x7f51600a2bd0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5178009ec0 tx=0x7f5178009920 comp rx=0 tx=0).stop 2024-02-02T15:29:05.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.900+0000 7f5185600700 1 -- 172.21.15.1:0/3283633548 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5180073a30 msgr2=0x7f5180199420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:05.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.900+0000 7f5185600700 1 --2- 172.21.15.1:0/3283633548 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5180073a30 0x7f5180199420 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f516800e9a0 tx=0x7f516800ecb0 comp rx=0 tx=0).stop 2024-02-02T15:29:05.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.900+0000 7f5185600700 1 -- 172.21.15.1:0/3283633548 shutdown_connections 2024-02-02T15:29:05.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.900+0000 7f5185600700 1 --2- 172.21.15.1:0/3283633548 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f51600a0710 0x7f51600a2bd0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:05.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.900+0000 7f5185600700 1 --2- 172.21.15.1:0/3283633548 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f518010d320 0x7f518019d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:05.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.900+0000 7f5185600700 1 --2- 172.21.15.1:0/3283633548 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5180073110 0x7f5180198ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:05.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.900+0000 7f5185600700 1 --2- 172.21.15.1:0/3283633548 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5180073a30 0x7f5180199420 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:05.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.900+0000 7f5185600700 1 -- 172.21.15.1:0/3283633548 >> 172.21.15.1:0/3283633548 conn(0x7f51800fc570 msgr2=0x7f518010f490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:05.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.900+0000 7f5185600700 1 -- 172.21.15.1:0/3283633548 shutdown_connections 2024-02-02T15:29:05.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:05.900+0000 7f5185600700 1 -- 172.21.15.1:0/3283633548 wait complete. 2024-02-02T15:29:05.919 INFO:tasks.cephfs.filesystem:scrub status for tag:8ec6fa0b-3ac6-4ac6-9a64-5e6170e2f035 - {'path': '/', 'tag': '8ec6fa0b-3ac6-4ac6-9a64-5e6170e2f035', 'options': 'recursive,force'} 2024-02-02T15:29:06.219 INFO:tasks.workunit.client.0.smithi001.stdout: 1 73294 10.26 MB/sec warmup 40 sec latency 22.860 ms 2024-02-02T15:29:06.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:29:06 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:29:06.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:29:06 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:29:06.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:29:06 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:29:06.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:29:06 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:29:06.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:29:06 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:29:06.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:29:06 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:29:06.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:29:06 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:29:06.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:29:06 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:29:06.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:29:06 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:29:06.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:29:06 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:29:06.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:29:06 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:29:06.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:29:06 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:29:06.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:29:06 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:29:06.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:29:06 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:29:06.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:29:06 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:29:06.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:29:06 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:29:06.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:29:06 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:29:06.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:29:06 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:29:06.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:29:06 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:29:06.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:29:06 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:29:06.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:29:06 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:29:06.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:29:06 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:29:06.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:29:06 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:29:06.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:29:06 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:29:07.219 INFO:tasks.workunit.client.0.smithi001.stdout: 1 74916 10.31 MB/sec warmup 41 sec latency 22.039 ms 2024-02-02T15:29:07.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:29:07 smithi071 ceph-mon[44707]: scrub queued for path: / 2024-02-02T15:29:07.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:29:07 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:29:07.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:29:07 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:29:07.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:29:07 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:29:07.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:29:07 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:29:07.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:29:07 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:29:07.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:29:07 smithi071 ceph-mon[44707]: pgmap v66: 129 pgs: 129 active+clean; 55 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 855 KiB/s rd, 3.8 MiB/s wr, 283 op/s 2024-02-02T15:29:07.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:29:07 smithi142 ceph-mon[45728]: scrub queued for path: / 2024-02-02T15:29:07.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:29:07 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:29:07.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:29:07 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:29:07.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:29:07 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:29:07.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:29:07 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:29:07.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:29:07 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:29:07.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:29:07 smithi142 ceph-mon[45728]: pgmap v66: 129 pgs: 129 active+clean; 55 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 855 KiB/s rd, 3.8 MiB/s wr, 283 op/s 2024-02-02T15:29:07.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:29:07 smithi001 ceph-mon[35470]: scrub queued for path: / 2024-02-02T15:29:07.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:29:07 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:29:07.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:29:07 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:29:07.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:29:07 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:29:07.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:29:07 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:29:07.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:29:07 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:29:07.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:29:07 smithi001 ceph-mon[35470]: pgmap v66: 129 pgs: 129 active+clean; 55 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 855 KiB/s rd, 3.8 MiB/s wr, 283 op/s 2024-02-02T15:29:08.219 INFO:tasks.workunit.client.0.smithi001.stdout: 1 75800 10.11 MB/sec warmup 42 sec latency 82.711 ms 2024-02-02T15:29:08.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:29:08 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:29:08.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:29:08 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:29:08.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:29:08 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:29:09.219 INFO:tasks.workunit.client.0.smithi001.stdout: 1 78316 10.26 MB/sec warmup 43 sec latency 12.137 ms 2024-02-02T15:29:09.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:29:09 smithi071 ceph-mon[44707]: pgmap v67: 129 pgs: 129 active+clean; 55 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 798 KiB/s rd, 3.7 MiB/s wr, 246 op/s 2024-02-02T15:29:09.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:29:09 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:29:09.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:29:09 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:29:09.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:29:09 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:29:09.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:29:09 smithi142 ceph-mon[45728]: pgmap v67: 129 pgs: 129 active+clean; 55 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 798 KiB/s rd, 3.7 MiB/s wr, 246 op/s 2024-02-02T15:29:09.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:29:09 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:29:09.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:29:09 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:29:09.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:29:09 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:29:09.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:29:09 smithi001 ceph-mon[35470]: pgmap v67: 129 pgs: 129 active+clean; 55 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 798 KiB/s rd, 3.7 MiB/s wr, 246 op/s 2024-02-02T15:29:09.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:29:09 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:29:09.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:29:09 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:29:09.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:29:09 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:29:10.219 INFO:tasks.workunit.client.0.smithi001.stdout: 1 79555 10.15 MB/sec warmup 44 sec latency 18.917 ms 2024-02-02T15:29:11.219 INFO:tasks.workunit.client.0.smithi001.stdout: 1 81879 10.23 MB/sec warmup 45 sec latency 23.629 ms 2024-02-02T15:29:11.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:29:11 smithi071 ceph-mon[44707]: pgmap v68: 129 pgs: 129 active+clean; 55 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 798 KiB/s rd, 3.7 MiB/s wr, 246 op/s 2024-02-02T15:29:11.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:29:11 smithi142 ceph-mon[45728]: pgmap v68: 129 pgs: 129 active+clean; 55 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 798 KiB/s rd, 3.7 MiB/s wr, 246 op/s 2024-02-02T15:29:11.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:29:11 smithi001 ceph-mon[35470]: pgmap v68: 129 pgs: 129 active+clean; 55 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 798 KiB/s rd, 3.7 MiB/s wr, 246 op/s 2024-02-02T15:29:12.219 INFO:tasks.workunit.client.0.smithi001.stdout: 1 83294 10.13 MB/sec warmup 46 sec latency 13.885 ms 2024-02-02T15:29:12.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:29:12 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:29:12.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:29:12 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:29:12.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:29:12 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:29:12.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:29:12 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:29:12.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:29:12 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:29:12.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:29:12 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:29:12.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:29:12 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:29:12.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:29:12 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:29:12.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:29:12 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:29:12.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:29:12 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:29:12.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:29:12 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:29:12.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:29:12 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:29:12.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:29:12 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:29:12.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:29:12 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:29:12.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:29:12 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:29:13.219 INFO:tasks.workunit.client.0.smithi001.stdout: 1 85296 10.18 MB/sec warmup 47 sec latency 56.551 ms 2024-02-02T15:29:13.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:29:13 smithi071 ceph-mon[44707]: pgmap v69: 129 pgs: 129 active+clean; 58 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.1 MiB/s wr, 366 op/s 2024-02-02T15:29:13.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:29:13 smithi142 ceph-mon[45728]: pgmap v69: 129 pgs: 129 active+clean; 58 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.1 MiB/s wr, 366 op/s 2024-02-02T15:29:13.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:29:13 smithi001 ceph-mon[35470]: pgmap v69: 129 pgs: 129 active+clean; 58 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.1 MiB/s wr, 366 op/s 2024-02-02T15:29:14.220 INFO:tasks.workunit.client.0.smithi001.stdout: 1 86408 10.04 MB/sec warmup 48 sec latency 15.158 ms 2024-02-02T15:29:15.220 INFO:tasks.workunit.client.0.smithi001.stdout: 1 88934 10.18 MB/sec warmup 49 sec latency 12.647 ms 2024-02-02T15:29:15.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:29:15 smithi071 ceph-mon[44707]: pgmap v70: 129 pgs: 129 active+clean; 58 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 940 KiB/s rd, 2.7 MiB/s wr, 247 op/s 2024-02-02T15:29:15.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:29:15 smithi142 ceph-mon[45728]: pgmap v70: 129 pgs: 129 active+clean; 58 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 940 KiB/s rd, 2.7 MiB/s wr, 247 op/s 2024-02-02T15:29:15.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:29:15 smithi001 ceph-mon[35470]: pgmap v70: 129 pgs: 129 active+clean; 58 MiB data, 522 MiB used, 1.0 TiB / 1.0 TiB avail; 940 KiB/s rd, 2.7 MiB/s wr, 247 op/s 2024-02-02T15:29:16.220 INFO:tasks.workunit.client.0.smithi001.stdout: 1 90152 10.08 MB/sec warmup 50 sec latency 10.766 ms 2024-02-02T15:29:16.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:29:16 smithi071 ceph-mon[44707]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:29:16.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:29:16 smithi142 ceph-mon[45728]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:29:16.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:29:16 smithi001 ceph-mon[35470]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:29:17.220 INFO:tasks.workunit.client.0.smithi001.stdout: 1 92496 10.15 MB/sec warmup 51 sec latency 11.435 ms 2024-02-02T15:29:17.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:29:17 smithi071 ceph-mon[44707]: pgmap v71: 129 pgs: 129 active+clean; 62 MiB data, 568 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.1 MiB/s wr, 367 op/s 2024-02-02T15:29:17.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:29:17 smithi142 ceph-mon[45728]: pgmap v71: 129 pgs: 129 active+clean; 62 MiB data, 568 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.1 MiB/s wr, 367 op/s 2024-02-02T15:29:17.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:29:17 smithi001 ceph-mon[35470]: pgmap v71: 129 pgs: 129 active+clean; 62 MiB data, 568 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.1 MiB/s wr, 367 op/s 2024-02-02T15:29:18.220 INFO:tasks.workunit.client.0.smithi001.stdout: 1 93448 10.00 MB/sec warmup 52 sec latency 66.170 ms 2024-02-02T15:29:19.220 INFO:tasks.workunit.client.0.smithi001.stdout: 1 95822 10.11 MB/sec warmup 53 sec latency 26.637 ms 2024-02-02T15:29:19.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:29:19 smithi071 ceph-mon[44707]: pgmap v72: 129 pgs: 129 active+clean; 62 MiB data, 568 MiB used, 1.0 TiB / 1.0 TiB avail; 1022 KiB/s rd, 2.7 MiB/s wr, 240 op/s 2024-02-02T15:29:19.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:29:19 smithi142 ceph-mon[45728]: pgmap v72: 129 pgs: 129 active+clean; 62 MiB data, 568 MiB used, 1.0 TiB / 1.0 TiB avail; 1022 KiB/s rd, 2.7 MiB/s wr, 240 op/s 2024-02-02T15:29:19.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:29:19 smithi001 ceph-mon[35470]: pgmap v72: 129 pgs: 129 active+clean; 62 MiB data, 568 MiB used, 1.0 TiB / 1.0 TiB avail; 1022 KiB/s rd, 2.7 MiB/s wr, 240 op/s 2024-02-02T15:29:20.220 INFO:tasks.workunit.client.0.smithi001.stdout: 1 95822 9.92 MB/sec warmup 54 sec latency 1002.764 ms 2024-02-02T15:29:21.220 INFO:tasks.workunit.client.0.smithi001.stdout: 1 95822 9.74 MB/sec warmup 55 sec latency 2002.874 ms 2024-02-02T15:29:21.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:29:21 smithi071 ceph-mon[44707]: pgmap v73: 129 pgs: 129 active+clean; 62 MiB data, 568 MiB used, 1.0 TiB / 1.0 TiB avail; 1022 KiB/s rd, 2.7 MiB/s wr, 240 op/s 2024-02-02T15:29:21.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:29:21 smithi071 ceph-mon[44707]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:29:21.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:29:21 smithi142 ceph-mon[45728]: pgmap v73: 129 pgs: 129 active+clean; 62 MiB data, 568 MiB used, 1.0 TiB / 1.0 TiB avail; 1022 KiB/s rd, 2.7 MiB/s wr, 240 op/s 2024-02-02T15:29:21.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:29:21 smithi142 ceph-mon[45728]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:29:21.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:29:21 smithi001 ceph-mon[35470]: pgmap v73: 129 pgs: 129 active+clean; 62 MiB data, 568 MiB used, 1.0 TiB / 1.0 TiB avail; 1022 KiB/s rd, 2.7 MiB/s wr, 240 op/s 2024-02-02T15:29:21.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:29:21 smithi001 ceph-mon[35470]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:29:22.220 INFO:tasks.workunit.client.0.smithi001.stdout: 1 95822 9.57 MB/sec warmup 56 sec latency 3003.002 ms 2024-02-02T15:29:23.220 INFO:tasks.workunit.client.0.smithi001.stdout: 1 95822 9.40 MB/sec warmup 57 sec latency 4003.054 ms 2024-02-02T15:29:23.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:29:23 smithi071 ceph-mon[44707]: pgmap v74: 129 pgs: 129 active+clean; 63 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.0 MiB/s wr, 329 op/s 2024-02-02T15:29:23.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:29:23 smithi142 ceph-mon[45728]: pgmap v74: 129 pgs: 129 active+clean; 63 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.0 MiB/s wr, 329 op/s 2024-02-02T15:29:23.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:29:23 smithi001 ceph-mon[35470]: pgmap v74: 129 pgs: 129 active+clean; 63 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.0 MiB/s wr, 329 op/s 2024-02-02T15:29:24.220 INFO:tasks.workunit.client.0.smithi001.stdout: 1 96026 9.26 MB/sec warmup 58 sec latency 4858.710 ms 2024-02-02T15:29:25.221 INFO:tasks.workunit.client.0.smithi001.stdout: 1 97241 9.19 MB/sec warmup 59 sec latency 12.321 ms 2024-02-02T15:29:25.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:29:25 smithi071 ceph-mon[44707]: pgmap v75: 129 pgs: 129 active+clean; 63 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 209 op/s 2024-02-02T15:29:25.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:29:25 smithi142 ceph-mon[45728]: pgmap v75: 129 pgs: 129 active+clean; 63 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 209 op/s 2024-02-02T15:29:25.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:29:25 smithi001 ceph-mon[35470]: pgmap v75: 129 pgs: 129 active+clean; 63 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 209 op/s 2024-02-02T15:29:26.221 INFO:tasks.workunit.client.0.smithi001.stdout: 1 99368 9.25 MB/sec warmup 60 sec latency 213.208 ms 2024-02-02T15:29:27.221 INFO:tasks.workunit.client.0.smithi001.stdout: 1 99368 9.09 MB/sec warmup 61 sec latency 1213.336 ms 2024-02-02T15:29:27.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:29:27 smithi071 ceph-mon[44707]: pgmap v76: 129 pgs: 129 active+clean; 65 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 254 op/s 2024-02-02T15:29:27.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:29:27 smithi142 ceph-mon[45728]: pgmap v76: 129 pgs: 129 active+clean; 65 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 254 op/s 2024-02-02T15:29:27.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:29:27 smithi001 ceph-mon[35470]: pgmap v76: 129 pgs: 129 active+clean; 65 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 254 op/s 2024-02-02T15:29:28.221 INFO:tasks.workunit.client.0.smithi001.stdout: 1 99368 8.95 MB/sec warmup 62 sec latency 2213.471 ms 2024-02-02T15:29:29.221 INFO:tasks.workunit.client.0.smithi001.stdout: 1 99434 8.81 MB/sec warmup 63 sec latency 3077.610 ms 2024-02-02T15:29:29.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:29:29 smithi071 ceph-mon[44707]: pgmap v77: 129 pgs: 129 active+clean; 65 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 782 KiB/s rd, 1.5 MiB/s wr, 134 op/s 2024-02-02T15:29:29.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:29:29 smithi142 ceph-mon[45728]: pgmap v77: 129 pgs: 129 active+clean; 65 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 782 KiB/s rd, 1.5 MiB/s wr, 134 op/s 2024-02-02T15:29:29.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:29:29 smithi001 ceph-mon[35470]: pgmap v77: 129 pgs: 129 active+clean; 65 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 782 KiB/s rd, 1.5 MiB/s wr, 134 op/s 2024-02-02T15:29:30.221 INFO:tasks.workunit.client.0.smithi001.stdout: 1 99434 8.67 MB/sec warmup 64 sec latency 1117.148 ms 2024-02-02T15:29:31.221 INFO:tasks.workunit.client.0.smithi001.stdout: 1 99434 8.54 MB/sec warmup 65 sec latency 2117.248 ms 2024-02-02T15:29:31.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:29:31 smithi071 ceph-mon[44707]: pgmap v78: 129 pgs: 129 active+clean; 65 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 782 KiB/s rd, 1.5 MiB/s wr, 134 op/s 2024-02-02T15:29:31.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:29:31 smithi142 ceph-mon[45728]: pgmap v78: 129 pgs: 129 active+clean; 65 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 782 KiB/s rd, 1.5 MiB/s wr, 134 op/s 2024-02-02T15:29:31.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:29:31 smithi001 ceph-mon[35470]: pgmap v78: 129 pgs: 129 active+clean; 65 MiB data, 672 MiB used, 1.0 TiB / 1.0 TiB avail; 782 KiB/s rd, 1.5 MiB/s wr, 134 op/s 2024-02-02T15:29:32.221 INFO:tasks.workunit.client.0.smithi001.stdout: 1 99434 8.41 MB/sec warmup 66 sec latency 3117.379 ms 2024-02-02T15:29:33.222 INFO:tasks.workunit.client.0.smithi001.stdout: 1 99434 8.28 MB/sec warmup 67 sec latency 4117.462 ms 2024-02-02T15:29:33.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:29:33 smithi071 ceph-mon[44707]: pgmap v79: 129 pgs: 129 active+clean; 71 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 820 KiB/s rd, 2.0 MiB/s wr, 157 op/s 2024-02-02T15:29:33.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:29:33 smithi142 ceph-mon[45728]: pgmap v79: 129 pgs: 129 active+clean; 71 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 820 KiB/s rd, 2.0 MiB/s wr, 157 op/s 2024-02-02T15:29:33.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:29:33 smithi001 ceph-mon[35470]: pgmap v79: 129 pgs: 129 active+clean; 71 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 820 KiB/s rd, 2.0 MiB/s wr, 157 op/s 2024-02-02T15:29:34.222 INFO:tasks.workunit.client.0.smithi001.stdout: 1 99573 8.18 MB/sec warmup 68 sec latency 4978.097 ms 2024-02-02T15:29:35.222 INFO:tasks.workunit.client.0.smithi001.stdout: 1 100764 8.14 MB/sec warmup 69 sec latency 15.419 ms 2024-02-02T15:29:35.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:29:35 smithi071 ceph-mon[44707]: pgmap v80: 129 pgs: 129 active+clean; 71 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 823 KiB/s wr, 68 op/s 2024-02-02T15:29:35.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:29:35 smithi142 ceph-mon[45728]: pgmap v80: 129 pgs: 129 active+clean; 71 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 823 KiB/s wr, 68 op/s 2024-02-02T15:29:35.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:29:35 smithi001 ceph-mon[35470]: pgmap v80: 129 pgs: 129 active+clean; 71 MiB data, 630 MiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 823 KiB/s wr, 68 op/s 2024-02-02T15:29:35.920 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-02T15:29:36.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.128+0000 7f595f400700 1 -- 172.21.15.1:0/1490895090 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59581019b0 msgr2=0x7f59581059f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:36.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.128+0000 7f595f400700 1 --2- 172.21.15.1:0/1490895090 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59581019b0 0x7f59581059f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5940009990 tx=0x7f5940009ca0 comp rx=0 tx=0).stop 2024-02-02T15:29:36.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.128+0000 7f595f400700 1 -- 172.21.15.1:0/1490895090 shutdown_connections 2024-02-02T15:29:36.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.128+0000 7f595f400700 1 --2- 172.21.15.1:0/1490895090 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5958106120 0x7f595810dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:36.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.128+0000 7f595f400700 1 --2- 172.21.15.1:0/1490895090 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5958101000 0x7f59581013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:36.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.128+0000 7f595f400700 1 --2- 172.21.15.1:0/1490895090 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f59581019b0 0x7f59581059f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:36.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.128+0000 7f595f400700 1 -- 172.21.15.1:0/1490895090 >> 172.21.15.1:0/1490895090 conn(0x7f59580fc860 msgr2=0x7f59580fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:36.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.128+0000 7f595f400700 1 -- 172.21.15.1:0/1490895090 shutdown_connections 2024-02-02T15:29:36.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.128+0000 7f595f400700 1 -- 172.21.15.1:0/1490895090 wait complete. 2024-02-02T15:29:36.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.129+0000 7f595f400700 1 Processor -- start 2024-02-02T15:29:36.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.129+0000 7f595f400700 1 -- start start 2024-02-02T15:29:36.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.130+0000 7f595f400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5958101000 0x7f59581990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:36.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.130+0000 7f595f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5958106120 0x7f5958199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:36.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.130+0000 7f595f400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f595819da60 0x7f595819dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:36.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.130+0000 7f595f400700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f595819e4e0 con 0x7f5958101000 2024-02-02T15:29:36.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.130+0000 7f595f400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f595819e650 con 0x7f595819da60 2024-02-02T15:29:36.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.130+0000 7f595f400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f595819e810 con 0x7f5958106120 2024-02-02T15:29:36.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.130+0000 7f5957e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5958106120 0x7f5958199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:36.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.130+0000 7f595d400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f595819da60 0x7f595819dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:36.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.130+0000 7f595ca00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5958101000 0x7f59581990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:36.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.130+0000 7f5957e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5958106120 0x7f5958199600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:40338/0 (socket says 172.21.15.1:40338) 2024-02-02T15:29:36.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.130+0000 7f595d400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f595819da60 0x7f595819dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:48600/0 (socket says 172.21.15.1:48600) 2024-02-02T15:29:36.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.130+0000 7f5957e00700 1 -- 172.21.15.1:0/3806126336 learned_addr learned my addr 172.21.15.1:0/3806126336 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:29:36.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.130+0000 7f595d400700 1 -- 172.21.15.1:0/3806126336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5958106120 msgr2=0x7f5958199600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:36.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.130+0000 7f595d400700 1 --2- 172.21.15.1:0/3806126336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5958106120 0x7f5958199600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:36.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.130+0000 7f595d400700 1 -- 172.21.15.1:0/3806126336 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5958101000 msgr2=0x7f59581990c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:36.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.130+0000 7f595d400700 1 --2- 172.21.15.1:0/3806126336 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5958101000 0x7f59581990c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:36.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.130+0000 7f595d400700 1 -- 172.21.15.1:0/3806126336 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59400095f0 con 0x7f595819da60 2024-02-02T15:29:36.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.130+0000 7f5957e00700 1 --2- 172.21.15.1:0/3806126336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5958106120 0x7f5958199600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:29:36.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.131+0000 7f595d400700 1 --2- 172.21.15.1:0/3806126336 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f595819da60 0x7f595819dee0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f595000efc0 tx=0x7f595000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:36.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.131+0000 7f5955600700 1 -- 172.21.15.1:0/3806126336 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5950018070 con 0x7f595819da60 2024-02-02T15:29:36.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.131+0000 7f595f400700 1 -- 172.21.15.1:0/3806126336 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59581a5d10 con 0x7f595819da60 2024-02-02T15:29:36.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.131+0000 7f5955600700 1 -- 172.21.15.1:0/3806126336 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f595000f040 con 0x7f595819da60 2024-02-02T15:29:36.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.131+0000 7f5955600700 1 -- 172.21.15.1:0/3806126336 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f595000cd40 con 0x7f595819da60 2024-02-02T15:29:36.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.131+0000 7f595f400700 1 -- 172.21.15.1:0/3806126336 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59581a61f0 con 0x7f595819da60 2024-02-02T15:29:36.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.134+0000 7f5955600700 1 -- 172.21.15.1:0/3806126336 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 27) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f5950003a00 con 0x7f595819da60 2024-02-02T15:29:36.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.135+0000 7f5972f51b80 1 Processor -- start 2024-02-02T15:29:36.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.136+0000 7f5972f51b80 1 -- start start 2024-02-02T15:29:36.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.136+0000 7f5955600700 1 --2- 172.21.15.1:0/3806126336 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f59380a0690 0x7f59380a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:36.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.136+0000 7f5972f51b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af3bbe3010 0x55af3bbe33f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:36.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.136+0000 7f5972f51b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af3bbe3930 0x55af3bbee2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:36.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.136+0000 7f5972f51b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af3bbee7e0 0x55af3bbf0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:36.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.136+0000 7f5972f51b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af3bb574c0 con 0x55af3bbe3010 2024-02-02T15:29:36.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.136+0000 7f5972f51b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af3bb3baa0 con 0x55af3bbe3930 2024-02-02T15:29:36.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.136+0000 7f5972f51b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af3bb39460 con 0x55af3bbee7e0 2024-02-02T15:29:36.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.136+0000 7f5955600700 1 -- 172.21.15.1:0/3806126336 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f59500cbc00 con 0x7f595819da60 2024-02-02T15:29:36.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.136+0000 7f5957e00700 1 --2- 172.21.15.1:0/3806126336 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f59380a0690 0x7f59380a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:36.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.136+0000 7f5957e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af3bbe3930 0x55af3bbee2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:36.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.136+0000 7f595ca00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af3bbe3010 0x55af3bbe33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:36.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.136+0000 7f5957e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af3bbe3930 0x55af3bbee2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:48616/0 (socket says 172.21.15.1:48616) 2024-02-02T15:29:36.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.136+0000 7f5957e00700 1 -- 172.21.15.1:0/3839224111 learned_addr learned my addr 172.21.15.1:0/3839224111 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:29:36.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.136+0000 7f595d400700 1 --2- 172.21.15.1:0/3839224111 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af3bbee7e0 0x55af3bbf0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:36.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.137+0000 7f595d400700 1 -- 172.21.15.1:0/3839224111 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af3bbe3930 msgr2=0x55af3bbee2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:36.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.137+0000 7f595d400700 1 --2- 172.21.15.1:0/3839224111 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af3bbe3930 0x55af3bbee2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:36.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.137+0000 7f595d400700 1 -- 172.21.15.1:0/3839224111 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af3bbe3010 msgr2=0x55af3bbe33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:36.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.137+0000 7f595d400700 1 --2- 172.21.15.1:0/3839224111 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af3bbe3010 0x55af3bbe33f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:36.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.137+0000 7f595d400700 1 -- 172.21.15.1:0/3839224111 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af3b9b63b0 con 0x55af3bbee7e0 2024-02-02T15:29:36.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.137+0000 7f595d400700 1 --2- 172.21.15.1:0/3839224111 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af3bbee7e0 0x55af3bbf0bd0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f59500c8f70 tx=0x7f59500951c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86ef23d7e079f8be server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:36.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.137+0000 7f5957e00700 1 --2- 172.21.15.1:0/3806126336 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f59380a0690 0x7f59380a2b50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f594000bef0 tx=0x7f5940004500 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:36.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.137+0000 7f594dc00700 1 -- 172.21.15.1:0/3839224111 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f595001b320 con 0x55af3bbee7e0 2024-02-02T15:29:36.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.138+0000 7f594dc00700 1 -- 172.21.15.1:0/3839224111 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5950004ad0 con 0x55af3bbee7e0 2024-02-02T15:29:36.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.138+0000 7f594dc00700 1 -- 172.21.15.1:0/3839224111 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59500a0de0 con 0x55af3bbee7e0 2024-02-02T15:29:36.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.138+0000 7f5972f51b80 1 -- 172.21.15.1:0/3839224111 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af3bbee7e0 msgr2=0x55af3bbf0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:36.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.138+0000 7f5972f51b80 1 --2- 172.21.15.1:0/3839224111 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af3bbee7e0 0x55af3bbf0bd0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f59500c8f70 tx=0x7f59500951c0 comp rx=0 tx=0).stop 2024-02-02T15:29:36.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.138+0000 7f5972f51b80 1 -- 172.21.15.1:0/3839224111 shutdown_connections 2024-02-02T15:29:36.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.138+0000 7f5972f51b80 1 --2- 172.21.15.1:0/3839224111 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af3bbee7e0 0x55af3bbf0bd0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:36.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.138+0000 7f5972f51b80 1 --2- 172.21.15.1:0/3839224111 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af3bbe3010 0x55af3bbe33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:36.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.138+0000 7f5972f51b80 1 --2- 172.21.15.1:0/3839224111 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af3bbe3930 0x55af3bbee2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:36.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.138+0000 7f5972f51b80 1 -- 172.21.15.1:0/3839224111 >> 172.21.15.1:0/3839224111 conn(0x55af3bb42db0 msgr2=0x55af3bbdf3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:36.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.138+0000 7f5972f51b80 1 -- 172.21.15.1:0/3839224111 shutdown_connections 2024-02-02T15:29:36.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.138+0000 7f5972f51b80 1 -- 172.21.15.1:0/3839224111 wait complete. 2024-02-02T15:29:36.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.139+0000 7f5972f51b80 1 -- start start 2024-02-02T15:29:36.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.139+0000 7f5972f51b80 1 Processor -- start 2024-02-02T15:29:36.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.140+0000 7f5972f51b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af3bbe3010 0x55af3bbee200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:36.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.140+0000 7f5972f51b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af3bbe3930 0x55af3bbeb410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:36.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.140+0000 7f5972f51b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af3bbee7e0 0x55af3bbeba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:36.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.140+0000 7f5972f51b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af3bb34d50 con 0x55af3bbee7e0 2024-02-02T15:29:36.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.140+0000 7f5972f51b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af3bad0d80 con 0x55af3bbe3930 2024-02-02T15:29:36.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.140+0000 7f5972f51b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af3bad0a20 con 0x55af3bbe3010 2024-02-02T15:29:36.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.140+0000 7f5972f51b80 10 client.0 _ll_register_callbacks cb 0x55af3bad2290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:29:36.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.140+0000 7f595ca00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af3bbe3010 0x55af3bbee200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:36.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.140+0000 7f5957e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af3bbe3930 0x55af3bbeb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:36.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.140+0000 7f595ca00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af3bbe3010 0x55af3bbee200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:40358/0 (socket says 172.21.15.1:40358) 2024-02-02T15:29:36.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.140+0000 7f595ca00700 1 -- 172.21.15.1:0/652472420 learned_addr learned my addr 172.21.15.1:0/652472420 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:29:36.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.140+0000 7f595d400700 1 --2- 172.21.15.1:0/652472420 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af3bbee7e0 0x55af3bbeba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:36.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.140+0000 7f5957e00700 1 -- 172.21.15.1:0/652472420 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af3bbe3010 msgr2=0x55af3bbee200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:36.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.140+0000 7f5957e00700 1 --2- 172.21.15.1:0/652472420 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af3bbe3010 0x55af3bbee200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:36.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.140+0000 7f5957e00700 1 -- 172.21.15.1:0/652472420 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af3bbee7e0 msgr2=0x55af3bbeba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:36.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.140+0000 7f5957e00700 1 --2- 172.21.15.1:0/652472420 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af3bbee7e0 0x55af3bbeba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:36.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.140+0000 7f5957e00700 1 -- 172.21.15.1:0/652472420 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59581a61f0 con 0x55af3bbe3930 2024-02-02T15:29:36.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.140+0000 7f595d400700 1 --2- 172.21.15.1:0/652472420 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af3bbee7e0 0x55af3bbeba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:29:36.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.141+0000 7f5957e00700 1 --2- 172.21.15.1:0/652472420 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af3bbe3930 0x55af3bbeb410 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f594000f740 tx=0x7f5940029000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b7f55d15bb9c363 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:36.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.141+0000 7f5946000700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:29:36.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.141+0000 7f5946000700 1 -- 172.21.15.1:0/652472420 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59400056e0 con 0x55af3bbe3930 2024-02-02T15:29:36.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.141+0000 7f5946000700 1 -- 172.21.15.1:0/652472420 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f5940004690 con 0x55af3bbe3930 2024-02-02T15:29:36.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.141+0000 7f5972f51b80 1 -- 172.21.15.1:0/652472420 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55af3bbd70f0 con 0x55af3bbe3930 2024-02-02T15:29:36.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.141+0000 7f5946000700 1 -- 172.21.15.1:0/652472420 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5940004850 con 0x55af3bbe3930 2024-02-02T15:29:36.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.141+0000 7f5972f51b80 10 client.26282 fetch_fsmap learned FSMap version 31 2024-02-02T15:29:36.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.141+0000 7f5972f51b80 1 -- 172.21.15.1:0/652472420 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55af3b962b60 con 0x55af3bbe3930 2024-02-02T15:29:36.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.142+0000 7f5946000700 1 -- 172.21.15.1:0/652472420 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55af3bbd70f0 con 0x55af3bbe3930 2024-02-02T15:29:36.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.142+0000 7f5946000700 1 -- 172.21.15.1:0/652472420 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 31) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f5940031070 con 0x55af3bbe3930 2024-02-02T15:29:36.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.142+0000 7f5972f51b80 10 client.26282 fetch_fsmap finished waiting for FSMap version 31 2024-02-02T15:29:36.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.142+0000 7f5972f51b80 10 client.26282 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:29:36.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.142+0000 7f5972f51b80 20 client.26282 populate_metadata read hostname 'smithi001' 2024-02-02T15:29:36.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.142+0000 7f5972f51b80 1 --2- 172.21.15.1:0/652472420 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55af3bbfe140 0x55af3bc00560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:36.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.142+0000 7f5972f51b80 4 client.26282 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:29:36.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.142+0000 7f5972f51b80 1 -- 172.21.15.1:0/652472420 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55af3bb2ddd0 con 0x55af3bbfe140 2024-02-02T15:29:36.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.142+0000 7f595ca00700 1 --2- 172.21.15.1:0/652472420 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55af3bbfe140 0x55af3bc00560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:36.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.143+0000 7f595ca00700 1 --2- 172.21.15.1:0/652472420 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55af3bbfe140 0x55af3bc00560 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=159849722b9b2a67 server_cookie=1f348d7ee999455f in_seq=0 out_seq=0 2024-02-02T15:29:36.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.143+0000 7f5946000700 10 client.26282 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:29:36.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.144+0000 7f5946000700 1 -- 172.21.15.1:0/652472420 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55af3bb2ddd0 con 0x55af3bbfe140 2024-02-02T15:29:36.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.144+0000 7f5946000700 10 client.26282 handle_command_reply: tid=0 2024-02-02T15:29:36.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.144+0000 7f5972f51b80 1 client.26282 shutdown 2024-02-02T15:29:36.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.144+0000 7f5972f51b80 1 -- 172.21.15.1:0/652472420 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af3bbe3930 msgr2=0x55af3bbeb410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:36.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.144+0000 7f5972f51b80 1 --2- 172.21.15.1:0/652472420 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af3bbe3930 0x55af3bbeb410 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f594000f740 tx=0x7f5940029000 comp rx=0 tx=0).stop 2024-02-02T15:29:36.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.144+0000 7f5972f51b80 1 -- 172.21.15.1:0/652472420 shutdown_connections 2024-02-02T15:29:36.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.144+0000 7f5972f51b80 1 --2- 172.21.15.1:0/652472420 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55af3bbfe140 0x55af3bc00560 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:36.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.144+0000 7f5972f51b80 1 --2- 172.21.15.1:0/652472420 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af3bbee7e0 0x55af3bbeba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:36.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.144+0000 7f5972f51b80 1 --2- 172.21.15.1:0/652472420 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af3bbe3010 0x55af3bbee200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:36.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.144+0000 7f5946000700 0 client.26282 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:29:36.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.144+0000 7f5972f51b80 1 --2- 172.21.15.1:0/652472420 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af3bbe3930 0x55af3bbeb410 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:36.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.144+0000 7f5972f51b80 1 -- 172.21.15.1:0/652472420 >> 172.21.15.1:0/652472420 conn(0x55af3bb42db0 msgr2=0x55af3bbdefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:36.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.144+0000 7f5972f51b80 1 -- 172.21.15.1:0/652472420 shutdown_connections 2024-02-02T15:29:36.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.144+0000 7f5972f51b80 1 -- 172.21.15.1:0/652472420 wait complete. 2024-02-02T15:29:36.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.145+0000 7f5972f51b80 20 client.26282 trim_cache size 0 max 16384 2024-02-02T15:29:36.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.164+0000 7f5972f51b80 1 Processor -- start 2024-02-02T15:29:36.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.164+0000 7f5972f51b80 1 -- start start 2024-02-02T15:29:36.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.165+0000 7f5972f51b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af3bbe3010 0x55af3bbf5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:36.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.165+0000 7f5972f51b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af3bbe3930 0x55af3bbf0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:36.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.165+0000 7f5972f51b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af3bbee7e0 0x55af3bbdae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:36.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.165+0000 7f5972f51b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af3b9b86a0 con 0x55af3bbe3010 2024-02-02T15:29:36.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.165+0000 7f5972f51b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af3bbe9970 con 0x55af3bbee7e0 2024-02-02T15:29:36.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.165+0000 7f5972f51b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af3bbe7260 con 0x55af3bbe3930 2024-02-02T15:29:36.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.165+0000 7f595d400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af3bbee7e0 0x55af3bbdae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:36.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.165+0000 7f5957e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af3bbe3930 0x55af3bbf0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:36.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.165+0000 7f595d400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af3bbee7e0 0x55af3bbdae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:48638/0 (socket says 172.21.15.1:48638) 2024-02-02T15:29:36.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.165+0000 7f595d400700 1 -- 172.21.15.1:0/1821572494 learned_addr learned my addr 172.21.15.1:0/1821572494 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:29:36.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.165+0000 7f595ca00700 1 --2- 172.21.15.1:0/1821572494 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af3bbe3010 0x55af3bbf5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:36.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.165+0000 7f595d400700 1 -- 172.21.15.1:0/1821572494 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af3bbe3930 msgr2=0x55af3bbf0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:36.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.165+0000 7f595d400700 1 --2- 172.21.15.1:0/1821572494 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af3bbe3930 0x55af3bbf0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:36.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.165+0000 7f595d400700 1 -- 172.21.15.1:0/1821572494 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af3bbe3010 msgr2=0x55af3bbf5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:36.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.165+0000 7f595d400700 1 --2- 172.21.15.1:0/1821572494 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af3bbe3010 0x55af3bbf5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:36.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.165+0000 7f595d400700 1 -- 172.21.15.1:0/1821572494 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af3bafaea0 con 0x55af3bbee7e0 2024-02-02T15:29:36.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.166+0000 7f595d400700 1 --2- 172.21.15.1:0/1821572494 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af3bbee7e0 0x55af3bbdae00 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5950098b00 tx=0x7f595000cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=380a0ac6566b6da9 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:36.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.166+0000 7f594e600700 1 -- 172.21.15.1:0/1821572494 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f595001b320 con 0x55af3bbee7e0 2024-02-02T15:29:36.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.166+0000 7f594e600700 1 -- 172.21.15.1:0/1821572494 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5950099500 con 0x55af3bbee7e0 2024-02-02T15:29:36.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.166+0000 7f594e600700 1 -- 172.21.15.1:0/1821572494 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59500992c0 con 0x55af3bbee7e0 2024-02-02T15:29:36.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.167+0000 7f5972f51b80 1 -- 172.21.15.1:0/1821572494 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af3bbee7e0 msgr2=0x55af3bbdae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:36.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.167+0000 7f5972f51b80 1 --2- 172.21.15.1:0/1821572494 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af3bbee7e0 0x55af3bbdae00 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5950098b00 tx=0x7f595000cf90 comp rx=0 tx=0).stop 2024-02-02T15:29:36.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.167+0000 7f5972f51b80 1 -- 172.21.15.1:0/1821572494 shutdown_connections 2024-02-02T15:29:36.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.167+0000 7f5972f51b80 1 --2- 172.21.15.1:0/1821572494 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af3bbee7e0 0x55af3bbdae00 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:36.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.167+0000 7f5972f51b80 1 --2- 172.21.15.1:0/1821572494 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af3bbe3010 0x55af3bbf5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:36.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.167+0000 7f5972f51b80 1 --2- 172.21.15.1:0/1821572494 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af3bbe3930 0x55af3bbf0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:36.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.167+0000 7f5972f51b80 1 -- 172.21.15.1:0/1821572494 >> 172.21.15.1:0/1821572494 conn(0x55af3bb42db0 msgr2=0x55af3bbe6760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:36.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.167+0000 7f5972f51b80 1 -- 172.21.15.1:0/1821572494 shutdown_connections 2024-02-02T15:29:36.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.167+0000 7f5972f51b80 1 -- 172.21.15.1:0/1821572494 wait complete. 2024-02-02T15:29:36.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.167+0000 7f5972f51b80 1 -- start start 2024-02-02T15:29:36.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.167+0000 7f5972f51b80 1 Processor -- start 2024-02-02T15:29:36.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.168+0000 7f5972f51b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af3bbe3010 0x55af3bbf2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:36.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.168+0000 7f5972f51b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af3bbe3930 0x55af3bbf2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:36.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.168+0000 7f5972f51b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af3bbee7e0 0x55af3bbf33e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:36.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.168+0000 7f5972f51b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af3bc4ee10 con 0x55af3bbe3930 2024-02-02T15:29:36.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.168+0000 7f5972f51b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af3bc4ef80 con 0x55af3bbee7e0 2024-02-02T15:29:36.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.168+0000 7f5972f51b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af3bc4f140 con 0x55af3bbe3010 2024-02-02T15:29:36.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.168+0000 7f5972f51b80 10 client.0 _ll_register_callbacks cb 0x55af3bad2290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:29:36.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.168+0000 7f5957e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af3bbe3930 0x55af3bbf2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:36.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.168+0000 7f595ca00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af3bbe3010 0x55af3bbf2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:36.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.168+0000 7f595ca00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af3bbe3010 0x55af3bbf2960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:40378/0 (socket says 172.21.15.1:40378) 2024-02-02T15:29:36.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.168+0000 7f595d400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af3bbee7e0 0x55af3bbf33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:36.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.168+0000 7f595ca00700 1 -- 172.21.15.1:0/3547655271 learned_addr learned my addr 172.21.15.1:0/3547655271 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:29:36.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.168+0000 7f5957e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af3bbe3930 0x55af3bbf2ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:34802/0 (socket says 172.21.15.1:34802) 2024-02-02T15:29:36.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.169+0000 7f595ca00700 1 -- 172.21.15.1:0/3547655271 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af3bbee7e0 msgr2=0x55af3bbf33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:36.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.169+0000 7f595ca00700 1 --2- 172.21.15.1:0/3547655271 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af3bbee7e0 0x55af3bbf33e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:36.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.169+0000 7f595ca00700 1 -- 172.21.15.1:0/3547655271 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af3bbe3930 msgr2=0x55af3bbf2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:36.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.169+0000 7f595ca00700 1 --2- 172.21.15.1:0/3547655271 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af3bbe3930 0x55af3bbf2ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:36.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.169+0000 7f595ca00700 1 -- 172.21.15.1:0/3547655271 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55af3b9b63b0 con 0x55af3bbe3010 2024-02-02T15:29:36.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.169+0000 7f5957e00700 1 --2- 172.21.15.1:0/3547655271 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af3bbe3930 0x55af3bbf2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:29:36.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.169+0000 7f595d400700 1 --2- 172.21.15.1:0/3547655271 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af3bbee7e0 0x55af3bbf33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:29:36.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.169+0000 7f595ca00700 1 --2- 172.21.15.1:0/3547655271 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af3bbe3010 0x55af3bbf2960 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x55af3bc00e40 tx=0x7f594801fb50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a32b1178b4666ae4 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:36.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.169+0000 7f5946000700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:29:36.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.169+0000 7f5946000700 1 -- 172.21.15.1:0/3547655271 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5948004500 con 0x55af3bbe3010 2024-02-02T15:29:36.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.169+0000 7f5972f51b80 1 -- 172.21.15.1:0/3547655271 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5950004ad0 con 0x55af3bbe3010 2024-02-02T15:29:36.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.169+0000 7f5946000700 1 -- 172.21.15.1:0/3547655271 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f5948003680 con 0x55af3bbe3010 2024-02-02T15:29:36.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.169+0000 7f5946000700 1 -- 172.21.15.1:0/3547655271 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5948003840 con 0x55af3bbe3010 2024-02-02T15:29:36.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.169+0000 7f5972f51b80 10 client.26278 fetch_fsmap learned FSMap version 31 2024-02-02T15:29:36.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.169+0000 7f5972f51b80 1 -- 172.21.15.1:0/3547655271 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55af3b9bacd0 con 0x55af3bbe3010 2024-02-02T15:29:36.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.170+0000 7f5946000700 1 -- 172.21.15.1:0/3547655271 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5950004ad0 con 0x55af3bbe3010 2024-02-02T15:29:36.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.170+0000 7f5946000700 1 -- 172.21.15.1:0/3547655271 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 31) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f5948010c70 con 0x55af3bbe3010 2024-02-02T15:29:36.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.170+0000 7f5972f51b80 10 client.26278 fetch_fsmap finished waiting for FSMap version 31 2024-02-02T15:29:36.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.170+0000 7f5972f51b80 10 client.26278 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:29:36.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.170+0000 7f5972f51b80 20 client.26278 populate_metadata read hostname 'smithi001' 2024-02-02T15:29:36.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.170+0000 7f5972f51b80 1 --2- 172.21.15.1:0/3547655271 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55af3bc50870 0x55af3bc52c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:36.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.170+0000 7f5972f51b80 4 client.26278 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2024-02-02T15:29:36.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.170+0000 7f5972f51b80 1 -- 172.21.15.1:0/3547655271 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55af3baf3f50 con 0x55af3bc50870 2024-02-02T15:29:36.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.170+0000 7f5957e00700 1 --2- 172.21.15.1:0/3547655271 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55af3bc50870 0x55af3bc52c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:36.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.171+0000 7f5946000700 10 client.26278 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:29:36.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.171+0000 7f5957e00700 1 --2- 172.21.15.1:0/3547655271 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55af3bc50870 0x55af3bc52c90 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4594f251d50f0e07 server_cookie=50564c76621416c2 in_seq=0 out_seq=0 2024-02-02T15:29:36.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.171+0000 7f5946000700 1 -- 172.21.15.1:0/3547655271 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55af3baf3f50 con 0x55af3bc50870 2024-02-02T15:29:36.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.171+0000 7f5946000700 10 client.26278 handle_command_reply: tid=0 2024-02-02T15:29:36.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.171+0000 7f5972f51b80 1 client.26278 shutdown 2024-02-02T15:29:36.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.171+0000 7f5972f51b80 1 -- 172.21.15.1:0/3547655271 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af3bbe3010 msgr2=0x55af3bbf2960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:36.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.171+0000 7f5972f51b80 1 --2- 172.21.15.1:0/3547655271 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af3bbe3010 0x55af3bbf2960 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x55af3bc00e40 tx=0x7f594801fb50 comp rx=0 tx=0).stop 2024-02-02T15:29:36.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.171+0000 7f5972f51b80 1 -- 172.21.15.1:0/3547655271 shutdown_connections 2024-02-02T15:29:36.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.171+0000 7f5972f51b80 1 --2- 172.21.15.1:0/3547655271 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55af3bc50870 0x55af3bc52c90 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:36.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.171+0000 7f5972f51b80 1 --2- 172.21.15.1:0/3547655271 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55af3bbee7e0 0x55af3bbf33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:36.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.171+0000 7f5972f51b80 1 --2- 172.21.15.1:0/3547655271 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55af3bbe3010 0x55af3bbf2960 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:36.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.171+0000 7f5946000700 0 client.26278 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:29:36.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.171+0000 7f5972f51b80 1 --2- 172.21.15.1:0/3547655271 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55af3bbe3930 0x55af3bbf2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:36.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.171+0000 7f5972f51b80 1 -- 172.21.15.1:0/3547655271 >> 172.21.15.1:0/3547655271 conn(0x55af3bb42db0 msgr2=0x55af3bbe6760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:36.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.171+0000 7f5972f51b80 1 -- 172.21.15.1:0/3547655271 shutdown_connections 2024-02-02T15:29:36.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.171+0000 7f5972f51b80 1 -- 172.21.15.1:0/3547655271 wait complete. 2024-02-02T15:29:36.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.171+0000 7f5972f51b80 20 client.26278 trim_cache size 0 max 16384 2024-02-02T15:29:36.176 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:29:36.176 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2024-02-02T15:29:36.176 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2024-02-02T15:29:36.177 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:29:36.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.172+0000 7f595f400700 1 -- 172.21.15.1:0/3806126336 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f59380a0690 msgr2=0x7f59380a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:36.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.172+0000 7f595f400700 1 --2- 172.21.15.1:0/3806126336 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f59380a0690 0x7f59380a2b50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f594000bef0 tx=0x7f5940004500 comp rx=0 tx=0).stop 2024-02-02T15:29:36.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.172+0000 7f595f400700 1 -- 172.21.15.1:0/3806126336 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f595819da60 msgr2=0x7f595819dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:36.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.172+0000 7f595f400700 1 --2- 172.21.15.1:0/3806126336 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f595819da60 0x7f595819dee0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f595000efc0 tx=0x7f595000c3a0 comp rx=0 tx=0).stop 2024-02-02T15:29:36.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.172+0000 7f595f400700 1 -- 172.21.15.1:0/3806126336 shutdown_connections 2024-02-02T15:29:36.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.172+0000 7f595f400700 1 --2- 172.21.15.1:0/3806126336 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f59380a0690 0x7f59380a2b50 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:36.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.172+0000 7f595f400700 1 --2- 172.21.15.1:0/3806126336 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f595819da60 0x7f595819dee0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:36.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.172+0000 7f595f400700 1 --2- 172.21.15.1:0/3806126336 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5958101000 0x7f59581990c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:36.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.172+0000 7f595f400700 1 --2- 172.21.15.1:0/3806126336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5958106120 0x7f5958199600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:36.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.172+0000 7f595f400700 1 -- 172.21.15.1:0/3806126336 >> 172.21.15.1:0/3806126336 conn(0x7f59580fc860 msgr2=0x7f59580fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:36.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.172+0000 7f595f400700 1 -- 172.21.15.1:0/3806126336 shutdown_connections 2024-02-02T15:29:36.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.172+0000 7f595f400700 1 -- 172.21.15.1:0/3806126336 wait complete. 2024-02-02T15:29:36.190 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-02T15:29:36.190 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T15:29:36.221 INFO:tasks.workunit.client.0.smithi001.stdout: 1 103063 8.20 MB/sec warmup 70 sec latency 54.581 ms 2024-02-02T15:29:36.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.390+0000 7f0629e00700 1 -- 172.21.15.1:0/3324446766 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0624100f10 msgr2=0x7f06241012f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:36.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.390+0000 7f0629e00700 1 --2- 172.21.15.1:0/3324446766 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0624100f10 0x7f06241012f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0614009990 tx=0x7f0614009ca0 comp rx=0 tx=0).stop 2024-02-02T15:29:36.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.390+0000 7f0629e00700 1 -- 172.21.15.1:0/3324446766 shutdown_connections 2024-02-02T15:29:36.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.390+0000 7f0629e00700 1 --2- 172.21.15.1:0/3324446766 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0624106030 0x7f062410dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:36.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.390+0000 7f0629e00700 1 --2- 172.21.15.1:0/3324446766 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0624100f10 0x7f06241012f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:36.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.390+0000 7f0629e00700 1 --2- 172.21.15.1:0/3324446766 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f06241018c0 0x7f0624105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:36.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.390+0000 7f0629e00700 1 -- 172.21.15.1:0/3324446766 >> 172.21.15.1:0/3324446766 conn(0x7f06240fc790 msgr2=0x7f06240febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:36.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.390+0000 7f0629e00700 1 -- 172.21.15.1:0/3324446766 shutdown_connections 2024-02-02T15:29:36.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.390+0000 7f0629e00700 1 -- 172.21.15.1:0/3324446766 wait complete. 2024-02-02T15:29:36.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.391+0000 7f0629e00700 1 Processor -- start 2024-02-02T15:29:36.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.391+0000 7f0629e00700 1 -- start start 2024-02-02T15:29:36.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.392+0000 7f0629e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0624100f10 0x7f0624199070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:36.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.392+0000 7f0629e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0624106030 0x7f06241995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:36.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.392+0000 7f0629e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f062419da10 0x7f062419de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:36.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.392+0000 7f0629e00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f062419e490 con 0x7f0624100f10 2024-02-02T15:29:36.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.392+0000 7f0629e00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f062419e600 con 0x7f062419da10 2024-02-02T15:29:36.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.392+0000 7f0629e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f062419e7c0 con 0x7f0624106030 2024-02-02T15:29:36.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.392+0000 7f0622a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0624100f10 0x7f0624199070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:36.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.392+0000 7f0622a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0624100f10 0x7f0624199070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:34834/0 (socket says 172.21.15.1:34834) 2024-02-02T15:29:36.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.392+0000 7f0622a00700 1 -- 172.21.15.1:0/3075213107 learned_addr learned my addr 172.21.15.1:0/3075213107 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:29:36.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.392+0000 7f0623400700 1 --2- 172.21.15.1:0/3075213107 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f062419da10 0x7f062419de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:36.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.392+0000 7f0622000700 1 --2- 172.21.15.1:0/3075213107 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0624106030 0x7f06241995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:36.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.392+0000 7f0622a00700 1 -- 172.21.15.1:0/3075213107 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0624106030 msgr2=0x7f06241995b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:36.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.392+0000 7f0622a00700 1 --2- 172.21.15.1:0/3075213107 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0624106030 0x7f06241995b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:36.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.393+0000 7f0622a00700 1 -- 172.21.15.1:0/3075213107 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f062419da10 msgr2=0x7f062419de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:36.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.393+0000 7f0622a00700 1 --2- 172.21.15.1:0/3075213107 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f062419da10 0x7f062419de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:36.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.393+0000 7f0622a00700 1 -- 172.21.15.1:0/3075213107 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06140095f0 con 0x7f0624100f10 2024-02-02T15:29:36.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.393+0000 7f0622a00700 1 --2- 172.21.15.1:0/3075213107 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0624100f10 0x7f0624199070 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f0614005910 tx=0x7f0614004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:36.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.393+0000 7f0628a00700 1 -- 172.21.15.1:0/3075213107 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061401d070 con 0x7f0624100f10 2024-02-02T15:29:36.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.393+0000 7f0628a00700 1 -- 172.21.15.1:0/3075213107 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0614022470 con 0x7f0624100f10 2024-02-02T15:29:36.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.393+0000 7f0629e00700 1 -- 172.21.15.1:0/3075213107 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06241a5cc0 con 0x7f0624100f10 2024-02-02T15:29:36.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.393+0000 7f0628a00700 1 -- 172.21.15.1:0/3075213107 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0614017650 con 0x7f0624100f10 2024-02-02T15:29:36.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.393+0000 7f0629e00700 1 -- 172.21.15.1:0/3075213107 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06241a6140 con 0x7f0624100f10 2024-02-02T15:29:36.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.394+0000 7f0629e00700 1 -- 172.21.15.1:0/3075213107 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0624067bc0 con 0x7f0624100f10 2024-02-02T15:29:36.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.396+0000 7f0628a00700 1 -- 172.21.15.1:0/3075213107 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 27) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f06140225e0 con 0x7f0624100f10 2024-02-02T15:29:36.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.397+0000 7f0628a00700 1 --2- 172.21.15.1:0/3075213107 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f06040a0710 0x7f06040a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:36.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.398+0000 7f0628a00700 1 -- 172.21.15.1:0/3075213107 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f06140cd250 con 0x7f0624100f10 2024-02-02T15:29:36.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.398+0000 7f0622000700 1 --2- 172.21.15.1:0/3075213107 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f06040a0710 0x7f06040a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:36.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.398+0000 7f0622000700 1 --2- 172.21.15.1:0/3075213107 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f06040a0710 0x7f06040a2bd0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f062419a680 tx=0x7f060c009260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:36.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.402+0000 7f0628a00700 1 -- 172.21.15.1:0/3075213107 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f061405d110 con 0x7f0624100f10 2024-02-02T15:29:36.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.660+0000 7f0629e00700 1 -- 172.21.15.1:0/3075213107 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0624101960 con 0x7f0624100f10 2024-02-02T15:29:36.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.661+0000 7f0628a00700 1 -- 172.21.15.1:0/3075213107 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+10880 (secure 0 0 0) 0x7f0614096790 con 0x7f0624100f10 2024-02-02T15:29:36.661 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:29:36.661 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2922240448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2922240448},{"type":"v1","addr":"172.21.15.1:6835","nonce":2922240448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2729497431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2729497431},{"type":"v1","addr":"172.21.15.1:6837","nonce":2729497431}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-02T15:24:47.761435+0000","modified":"2024-02-02T15:29:20.322035+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24529,"mds_2":24518,"mds_3":24514,"mds_4":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14517":{"gid":14517,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/1218303683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":1218303683},{"type":"v1","addr":"172.21.15.1:6839","nonce":1218303683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14529":{"gid":14529,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/2059197496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2059197496},{"type":"v1","addr":"172.21.15.1:6841","nonce":2059197496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/7648986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":7648986},{"type":"v1","addr":"172.21.15.142:6833","nonce":7648986}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/3482746094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3482746094},{"type":"v1","addr":"172.21.15.71:6833","nonce":3482746094}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/388207814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":388207814},{"type":"v1","addr":"172.21.15.71:6835","nonce":388207814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6835/1052606547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1052606547},{"type":"v1","addr":"172.21.15.142:6835","nonce":1052606547}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.142:6837/2870289019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2870289019},{"type":"v1","addr":"172.21.15.142:6837","nonce":2870289019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.71:6837/628016199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":628016199},{"type":"v1","addr":"172.21.15.71:6837","nonce":628016199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.142:6839/321675421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":321675421},{"type":"v1","addr":"172.21.15.142:6839","nonce":321675421}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/1268086493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1268086493},{"type":"v1","addr":"172.21.15.71:6839","nonce":1268086493}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-02T15:29:36.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.664+0000 7f0629e00700 1 -- 172.21.15.1:0/3075213107 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f06040a0710 msgr2=0x7f06040a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:36.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.664+0000 7f0629e00700 1 --2- 172.21.15.1:0/3075213107 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f06040a0710 0x7f06040a2bd0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f062419a680 tx=0x7f060c009260 comp rx=0 tx=0).stop 2024-02-02T15:29:36.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.664+0000 7f0629e00700 1 -- 172.21.15.1:0/3075213107 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0624100f10 msgr2=0x7f0624199070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:36.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.664+0000 7f0629e00700 1 --2- 172.21.15.1:0/3075213107 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0624100f10 0x7f0624199070 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f0614005910 tx=0x7f0614004730 comp rx=0 tx=0).stop 2024-02-02T15:29:36.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.664+0000 7f0629e00700 1 -- 172.21.15.1:0/3075213107 shutdown_connections 2024-02-02T15:29:36.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.664+0000 7f0629e00700 1 --2- 172.21.15.1:0/3075213107 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f06040a0710 0x7f06040a2bd0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:36.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.664+0000 7f0629e00700 1 --2- 172.21.15.1:0/3075213107 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f062419da10 0x7f062419de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:36.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.664+0000 7f0629e00700 1 --2- 172.21.15.1:0/3075213107 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0624100f10 0x7f0624199070 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:36.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.664+0000 7f0629e00700 1 --2- 172.21.15.1:0/3075213107 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0624106030 0x7f06241995b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:36.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.664+0000 7f0629e00700 1 -- 172.21.15.1:0/3075213107 >> 172.21.15.1:0/3075213107 conn(0x7f06240fc790 msgr2=0x7f062410ce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:36.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.664+0000 7f0629e00700 1 -- 172.21.15.1:0/3075213107 shutdown_connections 2024-02-02T15:29:36.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.664+0000 7f0629e00700 1 -- 172.21.15.1:0/3075213107 wait complete. 2024-02-02T15:29:36.665 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 31 2024-02-02T15:29:36.683 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-02-02T15:29:36.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.926+0000 7f46b9000700 1 -- 172.21.15.1:0/2906487260 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46b41060c0 msgr2=0x7f46b410dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:36.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.926+0000 7f46b9000700 1 --2- 172.21.15.1:0/2906487260 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46b41060c0 0x7f46b410dc40 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f46ac00b0a0 tx=0x7f46ac00b3b0 comp rx=0 tx=0).stop 2024-02-02T15:29:36.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.926+0000 7f46b9000700 1 -- 172.21.15.1:0/2906487260 shutdown_connections 2024-02-02T15:29:36.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.926+0000 7f46b9000700 1 --2- 172.21.15.1:0/2906487260 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46b41060c0 0x7f46b410dc40 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:36.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.926+0000 7f46b9000700 1 --2- 172.21.15.1:0/2906487260 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46b4100fa0 0x7f46b4101380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:36.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.926+0000 7f46b9000700 1 --2- 172.21.15.1:0/2906487260 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46b4101950 0x7f46b4105990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:36.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.926+0000 7f46b9000700 1 -- 172.21.15.1:0/2906487260 >> 172.21.15.1:0/2906487260 conn(0x7f46b40fc800 msgr2=0x7f46b40fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:36.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.926+0000 7f46b9000700 1 -- 172.21.15.1:0/2906487260 shutdown_connections 2024-02-02T15:29:36.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.926+0000 7f46b9000700 1 -- 172.21.15.1:0/2906487260 wait complete. 2024-02-02T15:29:36.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.927+0000 7f46b9000700 1 Processor -- start 2024-02-02T15:29:36.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.927+0000 7f46b9000700 1 -- start start 2024-02-02T15:29:36.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.927+0000 7f46b9000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46b4100fa0 0x7f46b4073040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:36.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.927+0000 7f46b9000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46b41060c0 0x7f46b4073580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:36.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.927+0000 7f46b9000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46b4073ac0 0x7f46b41a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:36.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.927+0000 7f46b9000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46b41a5880 con 0x7f46b4073ac0 2024-02-02T15:29:36.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.927+0000 7f46b9000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46b41a59f0 con 0x7f46b4100fa0 2024-02-02T15:29:36.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.927+0000 7f46b9000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46b41a5bb0 con 0x7f46b41060c0 2024-02-02T15:29:36.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.928+0000 7f46b2800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46b4073ac0 0x7f46b41a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:36.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.928+0000 7f46b2800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46b4073ac0 0x7f46b41a51f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:34848/0 (socket says 172.21.15.1:34848) 2024-02-02T15:29:36.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.928+0000 7f46b2800700 1 -- 172.21.15.1:0/3771395397 learned_addr learned my addr 172.21.15.1:0/3771395397 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:29:36.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.928+0000 7f46b1e00700 1 --2- 172.21.15.1:0/3771395397 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46b4100fa0 0x7f46b4073040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:36.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.928+0000 7f46b1400700 1 --2- 172.21.15.1:0/3771395397 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46b41060c0 0x7f46b4073580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:36.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.928+0000 7f46b2800700 1 -- 172.21.15.1:0/3771395397 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46b41060c0 msgr2=0x7f46b4073580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:36.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.928+0000 7f46b2800700 1 --2- 172.21.15.1:0/3771395397 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46b41060c0 0x7f46b4073580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:36.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.928+0000 7f46b2800700 1 -- 172.21.15.1:0/3771395397 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46b4100fa0 msgr2=0x7f46b4073040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:36.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.928+0000 7f46b2800700 1 --2- 172.21.15.1:0/3771395397 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46b4100fa0 0x7f46b4073040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:36.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.928+0000 7f46b2800700 1 -- 172.21.15.1:0/3771395397 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46ac009d00 con 0x7f46b4073ac0 2024-02-02T15:29:36.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.928+0000 7f46b2800700 1 --2- 172.21.15.1:0/3771395397 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46b4073ac0 0x7f46b41a51f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f46ac00b070 tx=0x7f46ac00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:36.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.929+0000 7f46aaa00700 1 -- 172.21.15.1:0/3771395397 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ac00e040 con 0x7f46b4073ac0 2024-02-02T15:29:36.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.929+0000 7f46aaa00700 1 -- 172.21.15.1:0/3771395397 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f46ac009480 con 0x7f46b4073ac0 2024-02-02T15:29:36.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.929+0000 7f46b9000700 1 -- 172.21.15.1:0/3771395397 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46b41a5e30 con 0x7f46b4073ac0 2024-02-02T15:29:36.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.929+0000 7f46aaa00700 1 -- 172.21.15.1:0/3771395397 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ac0052a0 con 0x7f46b4073ac0 2024-02-02T15:29:36.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.929+0000 7f46b9000700 1 -- 172.21.15.1:0/3771395397 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46b41a6250 con 0x7f46b4073ac0 2024-02-02T15:29:36.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.932+0000 7f46aaa00700 1 -- 172.21.15.1:0/3771395397 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 27) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f46ac01d480 con 0x7f46b4073ac0 2024-02-02T15:29:36.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.933+0000 7f46aaa00700 1 --2- 172.21.15.1:0/3771395397 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f46940a0710 0x7f46940a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:36.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.933+0000 7f46ccbe7b80 1 Processor -- start 2024-02-02T15:29:36.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.933+0000 7f46aaa00700 1 -- 172.21.15.1:0/3771395397 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f46ac0cd180 con 0x7f46b4073ac0 2024-02-02T15:29:36.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.933+0000 7f46ccbe7b80 1 -- start start 2024-02-02T15:29:36.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.933+0000 7f46b1e00700 1 --2- 172.21.15.1:0/3771395397 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f46940a0710 0x7f46940a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:36.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.934+0000 7f46ccbe7b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d9ee22e010 0x55d9ee22e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:36.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.934+0000 7f46ccbe7b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9ee22e930 0x55d9ee2392a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:36.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.934+0000 7f46ccbe7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9ee2397e0 0x55d9ee23bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:36.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.934+0000 7f46ccbe7b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9ee1a24c0 con 0x55d9ee22e930 2024-02-02T15:29:36.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.934+0000 7f46ccbe7b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9ee186aa0 con 0x55d9ee22e010 2024-02-02T15:29:36.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.934+0000 7f46ccbe7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9ee184460 con 0x55d9ee2397e0 2024-02-02T15:29:36.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.934+0000 7f46b1e00700 1 --2- 172.21.15.1:0/3771395397 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f46940a0710 0x7f46940a2bd0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f46a4009c60 tx=0x7f46a4009260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:36.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.934+0000 7f46b1e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d9ee22e010 0x55d9ee22e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:36.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.934+0000 7f46b1400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9ee22e930 0x55d9ee2392a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:36.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.934+0000 7f46b1e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d9ee22e010 0x55d9ee22e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:48692/0 (socket says 172.21.15.1:48692) 2024-02-02T15:29:36.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.934+0000 7f46b1e00700 1 -- 172.21.15.1:0/4286788693 learned_addr learned my addr 172.21.15.1:0/4286788693 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:29:36.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.934+0000 7f46b2800700 1 --2- 172.21.15.1:0/4286788693 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9ee2397e0 0x55d9ee23bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:36.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.934+0000 7f46b1e00700 1 -- 172.21.15.1:0/4286788693 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9ee2397e0 msgr2=0x55d9ee23bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:36.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.934+0000 7f46b1e00700 1 --2- 172.21.15.1:0/4286788693 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9ee2397e0 0x55d9ee23bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:36.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.934+0000 7f46b1e00700 1 -- 172.21.15.1:0/4286788693 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9ee22e930 msgr2=0x55d9ee2392a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:36.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.935+0000 7f46b1e00700 1 --2- 172.21.15.1:0/4286788693 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9ee22e930 0x55d9ee2392a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:36.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.935+0000 7f46b1e00700 1 -- 172.21.15.1:0/4286788693 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d9ee0013b0 con 0x55d9ee22e010 2024-02-02T15:29:36.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.935+0000 7f46b1400700 1 --2- 172.21.15.1:0/4286788693 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9ee22e930 0x55d9ee2392a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:29:36.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.935+0000 7f46b2800700 1 --2- 172.21.15.1:0/4286788693 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9ee2397e0 0x55d9ee23bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:29:36.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.935+0000 7f46b1e00700 1 --2- 172.21.15.1:0/4286788693 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d9ee22e010 0x55d9ee22e3f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f46a400e510 tx=0x7f46a4014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d59a79714acf9ae7 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:36.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.935+0000 7f46a3400700 1 -- 172.21.15.1:0/4286788693 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a4026430 con 0x55d9ee22e010 2024-02-02T15:29:36.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.935+0000 7f46a3400700 1 -- 172.21.15.1:0/4286788693 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f46a4021070 con 0x55d9ee22e010 2024-02-02T15:29:36.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.935+0000 7f46a3400700 1 -- 172.21.15.1:0/4286788693 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a4025610 con 0x55d9ee22e010 2024-02-02T15:29:36.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.936+0000 7f46ccbe7b80 1 -- 172.21.15.1:0/4286788693 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d9ee22e010 msgr2=0x55d9ee22e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:36.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.936+0000 7f46ccbe7b80 1 --2- 172.21.15.1:0/4286788693 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d9ee22e010 0x55d9ee22e3f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f46a400e510 tx=0x7f46a4014000 comp rx=0 tx=0).stop 2024-02-02T15:29:36.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.936+0000 7f46ccbe7b80 1 -- 172.21.15.1:0/4286788693 shutdown_connections 2024-02-02T15:29:36.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.936+0000 7f46ccbe7b80 1 --2- 172.21.15.1:0/4286788693 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9ee2397e0 0x55d9ee23bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:36.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.936+0000 7f46ccbe7b80 1 --2- 172.21.15.1:0/4286788693 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d9ee22e010 0x55d9ee22e3f0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:36.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.936+0000 7f46ccbe7b80 1 --2- 172.21.15.1:0/4286788693 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9ee22e930 0x55d9ee2392a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:36.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.936+0000 7f46ccbe7b80 1 -- 172.21.15.1:0/4286788693 >> 172.21.15.1:0/4286788693 conn(0x55d9ee18ddb0 msgr2=0x55d9ee22a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:36.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.936+0000 7f46ccbe7b80 1 -- 172.21.15.1:0/4286788693 shutdown_connections 2024-02-02T15:29:36.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.936+0000 7f46ccbe7b80 1 -- 172.21.15.1:0/4286788693 wait complete. 2024-02-02T15:29:36.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.937+0000 7f46ccbe7b80 1 -- start start 2024-02-02T15:29:36.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.937+0000 7f46ccbe7b80 1 Processor -- start 2024-02-02T15:29:36.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.937+0000 7f46ccbe7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9ee22e010 0x55d9ee239110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:36.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.937+0000 7f46ccbe7b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9ee22e930 0x55d9ee236390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:36.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.937+0000 7f46ccbe7b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d9ee2397e0 0x55d9ee236950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:36.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.937+0000 7f46ccbe7b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9ee17fd50 con 0x55d9ee22e930 2024-02-02T15:29:36.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.937+0000 7f46ccbe7b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9ee11bd80 con 0x55d9ee2397e0 2024-02-02T15:29:36.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.937+0000 7f46ccbe7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9ee11ba20 con 0x55d9ee22e010 2024-02-02T15:29:36.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.937+0000 7f46ccbe7b80 10 client.0 _ll_register_callbacks cb 0x55d9ee11d290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:29:36.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.937+0000 7f46b1400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9ee22e930 0x55d9ee236390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:36.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.937+0000 7f46b1400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9ee22e930 0x55d9ee236390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:34862/0 (socket says 172.21.15.1:34862) 2024-02-02T15:29:36.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.937+0000 7f46b1400700 1 -- 172.21.15.1:0/1050176441 learned_addr learned my addr 172.21.15.1:0/1050176441 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:29:36.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.938+0000 7f46b2800700 1 --2- 172.21.15.1:0/1050176441 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d9ee2397e0 0x55d9ee236950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:36.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.938+0000 7f46b1e00700 1 --2- 172.21.15.1:0/1050176441 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9ee22e010 0x55d9ee239110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:36.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.938+0000 7f46b1400700 1 -- 172.21.15.1:0/1050176441 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9ee22e010 msgr2=0x55d9ee239110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:36.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.938+0000 7f46b1400700 1 --2- 172.21.15.1:0/1050176441 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9ee22e010 0x55d9ee239110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:36.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.938+0000 7f46b1400700 1 -- 172.21.15.1:0/1050176441 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d9ee2397e0 msgr2=0x55d9ee236950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:36.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.938+0000 7f46b1400700 1 --2- 172.21.15.1:0/1050176441 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d9ee2397e0 0x55d9ee236950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:36.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.938+0000 7f46b1400700 1 -- 172.21.15.1:0/1050176441 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46b41a6250 con 0x55d9ee22e930 2024-02-02T15:29:36.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.938+0000 7f46b2800700 1 --2- 172.21.15.1:0/1050176441 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d9ee2397e0 0x55d9ee236950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:29:36.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.938+0000 7f46b1400700 1 --2- 172.21.15.1:0/1050176441 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9ee22e930 0x55d9ee236390 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f469c011830 tx=0x7f469c011b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d026bc79fdcec69 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:36.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.938+0000 7f469be00700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:29:36.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.938+0000 7f469be00700 1 -- 172.21.15.1:0/1050176441 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f469c016070 con 0x55d9ee22e930 2024-02-02T15:29:36.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.938+0000 7f469be00700 1 -- 172.21.15.1:0/1050176441 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f469c004300 con 0x55d9ee22e930 2024-02-02T15:29:36.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.938+0000 7f46ccbe7b80 1 -- 172.21.15.1:0/1050176441 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d9ee2220f0 con 0x55d9ee22e930 2024-02-02T15:29:36.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.939+0000 7f469be00700 1 -- 172.21.15.1:0/1050176441 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f469c0044c0 con 0x55d9ee22e930 2024-02-02T15:29:36.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.939+0000 7f46ccbe7b80 10 client.16251 fetch_fsmap learned FSMap version 31 2024-02-02T15:29:36.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.939+0000 7f46ccbe7b80 1 -- 172.21.15.1:0/1050176441 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f46a4017070 con 0x55d9ee22e930 2024-02-02T15:29:36.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.939+0000 7f469be00700 1 -- 172.21.15.1:0/1050176441 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d9ee2220f0 con 0x55d9ee22e930 2024-02-02T15:29:36.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.939+0000 7f469be00700 1 -- 172.21.15.1:0/1050176441 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 31) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f469c00f8e0 con 0x55d9ee22e930 2024-02-02T15:29:36.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.939+0000 7f46ccbe7b80 10 client.16251 fetch_fsmap finished waiting for FSMap version 31 2024-02-02T15:29:36.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.939+0000 7f46ccbe7b80 10 client.16251 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-02-02T15:29:36.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.939+0000 7f46ccbe7b80 20 client.16251 populate_metadata read hostname 'smithi001' 2024-02-02T15:29:36.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.939+0000 7f46ccbe7b80 1 --2- 172.21.15.1:0/1050176441 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55d9ee2492f0 0x55d9ee24b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:36.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.939+0000 7f46ccbe7b80 4 client.16251 mds_command: new command op to 24502 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:29:36.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.939+0000 7f46ccbe7b80 1 -- 172.21.15.1:0/1050176441 --> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d9ee178dd0 con 0x55d9ee2492f0 2024-02-02T15:29:36.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.940+0000 7f46b1e00700 1 --2- 172.21.15.1:0/1050176441 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55d9ee2492f0 0x55d9ee24b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:36.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.940+0000 7f46b1e00700 1 --2- 172.21.15.1:0/1050176441 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55d9ee2492f0 0x55d9ee24b710 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=69aa08798d7e5440 server_cookie=4bb356e2a74e5938 in_seq=0 out_seq=0 2024-02-02T15:29:36.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.940+0000 7f469be00700 10 client.16251 ms_handle_connect on v2:172.21.15.142:6834/1052606547 2024-02-02T15:29:36.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.941+0000 7f469be00700 1 -- 172.21.15.1:0/1050176441 <== mds.4 v2:172.21.15.142:6834/1052606547 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55d9ee178dd0 con 0x55d9ee2492f0 2024-02-02T15:29:36.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.941+0000 7f469be00700 10 client.16251 handle_command_reply: tid=0 2024-02-02T15:29:36.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.941+0000 7f46ccbe7b80 1 client.16251 shutdown 2024-02-02T15:29:36.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.941+0000 7f46ccbe7b80 1 -- 172.21.15.1:0/1050176441 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9ee22e930 msgr2=0x55d9ee236390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:36.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.941+0000 7f46ccbe7b80 1 --2- 172.21.15.1:0/1050176441 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9ee22e930 0x55d9ee236390 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f469c011830 tx=0x7f469c011b40 comp rx=0 tx=0).stop 2024-02-02T15:29:36.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.941+0000 7f46ccbe7b80 1 -- 172.21.15.1:0/1050176441 shutdown_connections 2024-02-02T15:29:36.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.941+0000 7f46ccbe7b80 1 --2- 172.21.15.1:0/1050176441 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55d9ee2492f0 0x55d9ee24b710 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:36.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.941+0000 7f46ccbe7b80 1 --2- 172.21.15.1:0/1050176441 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d9ee2397e0 0x55d9ee236950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:36.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.941+0000 7f46ccbe7b80 1 --2- 172.21.15.1:0/1050176441 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9ee22e010 0x55d9ee239110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:36.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.941+0000 7f46ccbe7b80 1 --2- 172.21.15.1:0/1050176441 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9ee22e930 0x55d9ee236390 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:36.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.941+0000 7f469be00700 0 client.16251 ms_handle_reset on v2:172.21.15.142:6834/1052606547 2024-02-02T15:29:36.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.941+0000 7f46ccbe7b80 1 -- 172.21.15.1:0/1050176441 >> 172.21.15.1:0/1050176441 conn(0x55d9ee18ddb0 msgr2=0x55d9ee229fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:36.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.942+0000 7f46ccbe7b80 1 -- 172.21.15.1:0/1050176441 shutdown_connections 2024-02-02T15:29:36.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.942+0000 7f46ccbe7b80 1 -- 172.21.15.1:0/1050176441 wait complete. 2024-02-02T15:29:36.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.942+0000 7f46ccbe7b80 20 client.16251 trim_cache size 0 max 16384 2024-02-02T15:29:36.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.956+0000 7f46ccbe7b80 1 Processor -- start 2024-02-02T15:29:36.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.956+0000 7f46ccbe7b80 1 -- start start 2024-02-02T15:29:36.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.956+0000 7f46ccbe7b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d9ee22e010 0x55d9ee244900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:36.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.956+0000 7f46ccbe7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9ee22e930 0x55d9ee244e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:36.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.956+0000 7f46ccbe7b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9ee2397e0 0x55d9ee23abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:36.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.956+0000 7f46ccbe7b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9ee0036a0 con 0x55d9ee2397e0 2024-02-02T15:29:36.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.956+0000 7f46ccbe7b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9ee245bf0 con 0x55d9ee22e010 2024-02-02T15:29:36.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.956+0000 7f46ccbe7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9ee22f260 con 0x55d9ee22e930 2024-02-02T15:29:36.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.957+0000 7f46b2800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9ee2397e0 0x55d9ee23abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:36.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.957+0000 7f46b2800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9ee2397e0 0x55d9ee23abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:34876/0 (socket says 172.21.15.1:34876) 2024-02-02T15:29:36.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.957+0000 7f46b1e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d9ee22e010 0x55d9ee244900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:36.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.957+0000 7f46b2800700 1 -- 172.21.15.1:0/1185899879 learned_addr learned my addr 172.21.15.1:0/1185899879 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:29:36.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.957+0000 7f46b1400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9ee22e930 0x55d9ee244e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:36.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.957+0000 7f46b2800700 1 -- 172.21.15.1:0/1185899879 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9ee22e930 msgr2=0x55d9ee244e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:36.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.957+0000 7f46b2800700 1 --2- 172.21.15.1:0/1185899879 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9ee22e930 0x55d9ee244e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:36.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.957+0000 7f46b2800700 1 -- 172.21.15.1:0/1185899879 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d9ee22e010 msgr2=0x55d9ee244900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:36.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.957+0000 7f46b2800700 1 --2- 172.21.15.1:0/1185899879 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d9ee22e010 0x55d9ee244900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:36.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.957+0000 7f46b2800700 1 -- 172.21.15.1:0/1185899879 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d9edfadb60 con 0x55d9ee2397e0 2024-02-02T15:29:36.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.957+0000 7f46b2800700 1 --2- 172.21.15.1:0/1185899879 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9ee2397e0 0x55d9ee23abd0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f46ac0079e0 tx=0x7f46ac009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=183d6d25709c17db server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:36.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.957+0000 7f46a3e00700 1 -- 172.21.15.1:0/1185899879 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ac003e10 con 0x55d9ee2397e0 2024-02-02T15:29:36.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.957+0000 7f46a3e00700 1 -- 172.21.15.1:0/1185899879 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f46ac085070 con 0x55d9ee2397e0 2024-02-02T15:29:36.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.958+0000 7f46a3e00700 1 -- 172.21.15.1:0/1185899879 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ac0a23c0 con 0x55d9ee2397e0 2024-02-02T15:29:36.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.958+0000 7f46ccbe7b80 1 -- 172.21.15.1:0/1185899879 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9ee2397e0 msgr2=0x55d9ee23abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:36.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.958+0000 7f46ccbe7b80 1 --2- 172.21.15.1:0/1185899879 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9ee2397e0 0x55d9ee23abd0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f46ac0079e0 tx=0x7f46ac009040 comp rx=0 tx=0).stop 2024-02-02T15:29:36.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.958+0000 7f46ccbe7b80 1 -- 172.21.15.1:0/1185899879 shutdown_connections 2024-02-02T15:29:36.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.958+0000 7f46ccbe7b80 1 --2- 172.21.15.1:0/1185899879 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9ee2397e0 0x55d9ee23abd0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:36.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.958+0000 7f46ccbe7b80 1 --2- 172.21.15.1:0/1185899879 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d9ee22e010 0x55d9ee244900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:36.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.958+0000 7f46ccbe7b80 1 --2- 172.21.15.1:0/1185899879 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9ee22e930 0x55d9ee244e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:36.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.958+0000 7f46ccbe7b80 1 -- 172.21.15.1:0/1185899879 >> 172.21.15.1:0/1185899879 conn(0x55d9ee18ddb0 msgr2=0x55d9ee2454e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:36.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.958+0000 7f46ccbe7b80 1 -- 172.21.15.1:0/1185899879 shutdown_connections 2024-02-02T15:29:36.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.958+0000 7f46ccbe7b80 1 -- 172.21.15.1:0/1185899879 wait complete. 2024-02-02T15:29:36.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.959+0000 7f46ccbe7b80 1 -- start start 2024-02-02T15:29:36.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.959+0000 7f46ccbe7b80 1 Processor -- start 2024-02-02T15:29:36.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.959+0000 7f46ccbe7b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9ee22e010 0x55d9ee23db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:36.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.959+0000 7f46ccbe7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9ee22e930 0x55d9ee23e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:36.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.959+0000 7f46ccbe7b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d9ee2397e0 0x55d9ee299e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:36.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.959+0000 7f46ccbe7b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9ee23e850 con 0x55d9ee22e010 2024-02-02T15:29:36.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.959+0000 7f46ccbe7b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9ee23e9c0 con 0x55d9ee2397e0 2024-02-02T15:29:36.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.959+0000 7f46ccbe7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9ee29a3a0 con 0x55d9ee22e930 2024-02-02T15:29:36.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.959+0000 7f46ccbe7b80 10 client.0 _ll_register_callbacks cb 0x55d9ee11d290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:29:36.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.960+0000 7f46b1e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9ee22e010 0x55d9ee23db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:36.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.960+0000 7f46b1e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9ee22e010 0x55d9ee23db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:34890/0 (socket says 172.21.15.1:34890) 2024-02-02T15:29:36.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.960+0000 7f46b1e00700 1 -- 172.21.15.1:0/1254928048 learned_addr learned my addr 172.21.15.1:0/1254928048 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:29:36.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.960+0000 7f46b2800700 1 --2- 172.21.15.1:0/1254928048 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d9ee2397e0 0x55d9ee299e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:36.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.960+0000 7f46b1400700 1 --2- 172.21.15.1:0/1254928048 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9ee22e930 0x55d9ee23e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:36.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.960+0000 7f46b1e00700 1 -- 172.21.15.1:0/1254928048 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9ee22e930 msgr2=0x55d9ee23e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:36.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.960+0000 7f46b1e00700 1 --2- 172.21.15.1:0/1254928048 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9ee22e930 0x55d9ee23e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:36.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.960+0000 7f46b1e00700 1 -- 172.21.15.1:0/1254928048 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d9ee2397e0 msgr2=0x55d9ee299e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:36.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.960+0000 7f46b1e00700 1 --2- 172.21.15.1:0/1254928048 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d9ee2397e0 0x55d9ee299e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:36.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.960+0000 7f46b1e00700 1 -- 172.21.15.1:0/1254928048 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46ac02c070 con 0x55d9ee22e010 2024-02-02T15:29:36.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.960+0000 7f46b1e00700 1 --2- 172.21.15.1:0/1254928048 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9ee22e010 0x55d9ee23db30 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f46a400e510 tx=0x7f46a400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d12c680a301697f0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:36.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.960+0000 7f469be00700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:29:36.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.960+0000 7f469be00700 1 -- 172.21.15.1:0/1254928048 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a4035310 con 0x55d9ee22e010 2024-02-02T15:29:36.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.961+0000 7f46ccbe7b80 1 -- 172.21.15.1:0/1254928048 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f46a4021070 con 0x55d9ee22e010 2024-02-02T15:29:36.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.961+0000 7f469be00700 1 -- 172.21.15.1:0/1254928048 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f46a4035920 con 0x55d9ee22e010 2024-02-02T15:29:36.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.961+0000 7f469be00700 1 -- 172.21.15.1:0/1254928048 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f46a4035ae0 con 0x55d9ee22e010 2024-02-02T15:29:36.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.961+0000 7f46ccbe7b80 10 client.16257 fetch_fsmap learned FSMap version 31 2024-02-02T15:29:36.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.961+0000 7f46ccbe7b80 1 -- 172.21.15.1:0/1254928048 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55d9ee145ea0 con 0x55d9ee22e010 2024-02-02T15:29:36.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.961+0000 7f469be00700 1 -- 172.21.15.1:0/1254928048 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f46a4021070 con 0x55d9ee22e010 2024-02-02T15:29:36.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.961+0000 7f469be00700 1 -- 172.21.15.1:0/1254928048 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 31) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f46a401d520 con 0x55d9ee22e010 2024-02-02T15:29:36.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.961+0000 7f46ccbe7b80 10 client.16257 fetch_fsmap finished waiting for FSMap version 31 2024-02-02T15:29:36.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.961+0000 7f46ccbe7b80 10 client.16257 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-02-02T15:29:36.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.961+0000 7f46ccbe7b80 20 client.16257 populate_metadata read hostname 'smithi001' 2024-02-02T15:29:36.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.961+0000 7f46ccbe7b80 1 --2- 172.21.15.1:0/1254928048 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55d9ee29ba30 0x55d9ee29de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:36.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.961+0000 7f46ccbe7b80 4 client.16257 mds_command: new command op to 24502 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:29:36.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.961+0000 7f46ccbe7b80 1 -- 172.21.15.1:0/1254928048 --> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d9ee13ef50 con 0x55d9ee29ba30 2024-02-02T15:29:36.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.962+0000 7f46b1400700 1 --2- 172.21.15.1:0/1254928048 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55d9ee29ba30 0x55d9ee29de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:36.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.962+0000 7f46b1400700 1 --2- 172.21.15.1:0/1254928048 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55d9ee29ba30 0x55d9ee29de50 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f74cee0a5a0ffe57 server_cookie=aff0cf7695c85d6a in_seq=0 out_seq=0 2024-02-02T15:29:36.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.962+0000 7f469be00700 10 client.16257 ms_handle_connect on v2:172.21.15.142:6834/1052606547 2024-02-02T15:29:36.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.962+0000 7f469be00700 1 -- 172.21.15.1:0/1254928048 <== mds.4 v2:172.21.15.142:6834/1052606547 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d9ee13ef50 con 0x55d9ee29ba30 2024-02-02T15:29:36.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.962+0000 7f469be00700 10 client.16257 handle_command_reply: tid=0 2024-02-02T15:29:36.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.962+0000 7f46ccbe7b80 1 client.16257 shutdown 2024-02-02T15:29:36.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.962+0000 7f46ccbe7b80 1 -- 172.21.15.1:0/1254928048 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9ee22e010 msgr2=0x55d9ee23db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:36.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.962+0000 7f46ccbe7b80 1 --2- 172.21.15.1:0/1254928048 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9ee22e010 0x55d9ee23db30 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f46a400e510 tx=0x7f46a400eca0 comp rx=0 tx=0).stop 2024-02-02T15:29:36.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.962+0000 7f46ccbe7b80 1 -- 172.21.15.1:0/1254928048 shutdown_connections 2024-02-02T15:29:36.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.962+0000 7f46ccbe7b80 1 --2- 172.21.15.1:0/1254928048 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55d9ee29ba30 0x55d9ee29de50 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:36.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.962+0000 7f46ccbe7b80 1 --2- 172.21.15.1:0/1254928048 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d9ee2397e0 0x55d9ee299e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:36.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.962+0000 7f469be00700 0 client.16257 ms_handle_reset on v2:172.21.15.142:6834/1052606547 2024-02-02T15:29:36.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.962+0000 7f46ccbe7b80 1 --2- 172.21.15.1:0/1254928048 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9ee22e010 0x55d9ee23db30 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:36.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.962+0000 7f46ccbe7b80 1 --2- 172.21.15.1:0/1254928048 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9ee22e930 0x55d9ee23e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:36.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.962+0000 7f46ccbe7b80 1 -- 172.21.15.1:0/1254928048 >> 172.21.15.1:0/1254928048 conn(0x55d9ee18ddb0 msgr2=0x55d9ee2316a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:36.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.963+0000 7f46ccbe7b80 1 -- 172.21.15.1:0/1254928048 shutdown_connections 2024-02-02T15:29:36.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.963+0000 7f46ccbe7b80 1 -- 172.21.15.1:0/1254928048 wait complete. 2024-02-02T15:29:36.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.963+0000 7f46ccbe7b80 20 client.16257 trim_cache size 0 max 16384 2024-02-02T15:29:36.968 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:29:36.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.963+0000 7f46b9000700 1 -- 172.21.15.1:0/3771395397 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f46940a0710 msgr2=0x7f46940a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:36.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.963+0000 7f46b9000700 1 --2- 172.21.15.1:0/3771395397 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f46940a0710 0x7f46940a2bd0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f46a4009c60 tx=0x7f46a4009260 comp rx=0 tx=0).stop 2024-02-02T15:29:36.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.963+0000 7f46b9000700 1 -- 172.21.15.1:0/3771395397 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46b4073ac0 msgr2=0x7f46b41a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:36.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.963+0000 7f46b9000700 1 --2- 172.21.15.1:0/3771395397 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46b4073ac0 0x7f46b41a51f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f46ac00b070 tx=0x7f46ac00bc80 comp rx=0 tx=0).stop 2024-02-02T15:29:36.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.963+0000 7f46b9000700 1 -- 172.21.15.1:0/3771395397 shutdown_connections 2024-02-02T15:29:36.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.963+0000 7f46b9000700 1 --2- 172.21.15.1:0/3771395397 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f46940a0710 0x7f46940a2bd0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:36.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.963+0000 7f46b9000700 1 --2- 172.21.15.1:0/3771395397 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f46b4073ac0 0x7f46b41a51f0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:36.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.963+0000 7f46b9000700 1 --2- 172.21.15.1:0/3771395397 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f46b4100fa0 0x7f46b4073040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:36.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.963+0000 7f46b9000700 1 --2- 172.21.15.1:0/3771395397 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f46b41060c0 0x7f46b4073580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:36.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.963+0000 7f46b9000700 1 -- 172.21.15.1:0/3771395397 >> 172.21.15.1:0/3771395397 conn(0x7f46b40fc800 msgr2=0x7f46b40fd290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:36.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.964+0000 7f46b9000700 1 -- 172.21.15.1:0/3771395397 shutdown_connections 2024-02-02T15:29:36.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:36.964+0000 7f46b9000700 1 -- 172.21.15.1:0/3771395397 wait complete. 2024-02-02T15:29:36.982 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-02-02T15:29:37.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.181+0000 7ff00a600700 1 -- 172.21.15.1:0/788460858 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff004100ff0 msgr2=0x7ff0041013d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:37.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.181+0000 7ff00a600700 1 --2- 172.21.15.1:0/788460858 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff004100ff0 0x7ff0041013d0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7feff4009990 tx=0x7feff4009ca0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.182+0000 7ff00a600700 1 -- 172.21.15.1:0/788460858 shutdown_connections 2024-02-02T15:29:37.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.182+0000 7ff00a600700 1 --2- 172.21.15.1:0/788460858 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff004106110 0x7ff00410dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.182+0000 7ff00a600700 1 --2- 172.21.15.1:0/788460858 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff004100ff0 0x7ff0041013d0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.182+0000 7ff00a600700 1 --2- 172.21.15.1:0/788460858 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0041019a0 0x7ff0041059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.182+0000 7ff00a600700 1 -- 172.21.15.1:0/788460858 >> 172.21.15.1:0/788460858 conn(0x7ff0040fc810 msgr2=0x7ff0040fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:37.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.182+0000 7ff00a600700 1 -- 172.21.15.1:0/788460858 shutdown_connections 2024-02-02T15:29:37.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.182+0000 7ff00a600700 1 -- 172.21.15.1:0/788460858 wait complete. 2024-02-02T15:29:37.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.183+0000 7ff00a600700 1 Processor -- start 2024-02-02T15:29:37.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.183+0000 7ff00a600700 1 -- start start 2024-02-02T15:29:37.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.183+0000 7ff00a600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff004100ff0 0x7ff004199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:37.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.183+0000 7ff00a600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff004106110 0x7ff004199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:37.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.183+0000 7ff00a600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff00419dac0 0x7ff00419df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:37.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.183+0000 7ff00a600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff00419e540 con 0x7ff004100ff0 2024-02-02T15:29:37.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.183+0000 7ff00a600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff00419e6b0 con 0x7ff00419dac0 2024-02-02T15:29:37.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.183+0000 7ff00a600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff00419e870 con 0x7ff004106110 2024-02-02T15:29:37.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.183+0000 7ff003400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff004100ff0 0x7ff004199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:37.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.183+0000 7ff003400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff004100ff0 0x7ff004199120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:34918/0 (socket says 172.21.15.1:34918) 2024-02-02T15:29:37.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.183+0000 7ff003400700 1 -- 172.21.15.1:0/2439652449 learned_addr learned my addr 172.21.15.1:0/2439652449 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:29:37.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.184+0000 7ff003e00700 1 --2- 172.21.15.1:0/2439652449 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff00419dac0 0x7ff00419df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:37.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.184+0000 7ff002a00700 1 --2- 172.21.15.1:0/2439652449 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff004106110 0x7ff004199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:37.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.184+0000 7ff003400700 1 -- 172.21.15.1:0/2439652449 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff004106110 msgr2=0x7ff004199660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:37.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.184+0000 7ff003400700 1 --2- 172.21.15.1:0/2439652449 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff004106110 0x7ff004199660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.184+0000 7ff003400700 1 -- 172.21.15.1:0/2439652449 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff00419dac0 msgr2=0x7ff00419df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:37.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.184+0000 7ff003400700 1 --2- 172.21.15.1:0/2439652449 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff00419dac0 0x7ff00419df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.184+0000 7ff003400700 1 -- 172.21.15.1:0/2439652449 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feff40095f0 con 0x7ff004100ff0 2024-02-02T15:29:37.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.184+0000 7ff002a00700 1 --2- 172.21.15.1:0/2439652449 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff004106110 0x7ff004199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:29:37.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.184+0000 7ff003400700 1 --2- 172.21.15.1:0/2439652449 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff004100ff0 0x7ff004199120 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7feff4005910 tx=0x7feff4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:37.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.184+0000 7feffbe00700 1 -- 172.21.15.1:0/2439652449 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff401d070 con 0x7ff004100ff0 2024-02-02T15:29:37.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.184+0000 7feffbe00700 1 -- 172.21.15.1:0/2439652449 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feff4022470 con 0x7ff004100ff0 2024-02-02T15:29:37.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.184+0000 7ff00a600700 1 -- 172.21.15.1:0/2439652449 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0041a5d70 con 0x7ff004100ff0 2024-02-02T15:29:37.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.184+0000 7feffbe00700 1 -- 172.21.15.1:0/2439652449 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff4017650 con 0x7ff004100ff0 2024-02-02T15:29:37.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.185+0000 7ff00a600700 1 -- 172.21.15.1:0/2439652449 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0041a6190 con 0x7ff004100ff0 2024-02-02T15:29:37.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.187+0000 7feffbe00700 1 -- 172.21.15.1:0/2439652449 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 27) v1 ==== 135200+0+0 (secure 0 0 0) 0x7feff40225e0 con 0x7ff004100ff0 2024-02-02T15:29:37.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.189+0000 7ff01e165b80 1 Processor -- start 2024-02-02T15:29:37.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.189+0000 7feffbe00700 1 --2- 172.21.15.1:0/2439652449 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fefe40a0690 0x7fefe40a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:37.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.189+0000 7feffbe00700 1 -- 172.21.15.1:0/2439652449 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7feff40cd370 con 0x7ff004100ff0 2024-02-02T15:29:37.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.189+0000 7ff01e165b80 1 -- start start 2024-02-02T15:29:37.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.189+0000 7ff003e00700 1 --2- 172.21.15.1:0/2439652449 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fefe40a0690 0x7fefe40a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:37.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.189+0000 7ff01e165b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55671090d010 0x55671090d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:37.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.189+0000 7ff01e165b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55671090d930 0x5567109182a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:37.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.189+0000 7ff01e165b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567109187e0 0x55671091abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:37.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.189+0000 7ff01e165b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567108814c0 con 0x55671090d010 2024-02-02T15:29:37.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.189+0000 7ff01e165b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x556710865aa0 con 0x55671090d930 2024-02-02T15:29:37.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.189+0000 7ff01e165b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556710863460 con 0x5567109187e0 2024-02-02T15:29:37.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.190+0000 7ff003400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55671090d010 0x55671090d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:37.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.190+0000 7ff003400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55671090d010 0x55671090d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:34922/0 (socket says 172.21.15.1:34922) 2024-02-02T15:29:37.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.190+0000 7ff003400700 1 -- 172.21.15.1:0/4188492310 learned_addr learned my addr 172.21.15.1:0/4188492310 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:29:37.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.190+0000 7ff002a00700 1 --2- 172.21.15.1:0/4188492310 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55671090d930 0x5567109182a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:37.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.190+0000 7ff003e00700 1 --2- 172.21.15.1:0/4188492310 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567109187e0 0x55671091abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:37.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.190+0000 7ff003e00700 1 --2- 172.21.15.1:0/2439652449 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fefe40a0690 0x7fefe40a2b50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7feffc00f590 tx=0x7feffc00b040 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:37.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.190+0000 7ff003400700 1 -- 172.21.15.1:0/4188492310 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567109187e0 msgr2=0x55671091abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:37.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.190+0000 7ff003400700 1 --2- 172.21.15.1:0/4188492310 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567109187e0 0x55671091abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.190+0000 7ff003400700 1 -- 172.21.15.1:0/4188492310 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55671090d930 msgr2=0x5567109182a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:37.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.190+0000 7ff003400700 1 --2- 172.21.15.1:0/4188492310 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55671090d930 0x5567109182a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.190+0000 7ff003400700 1 -- 172.21.15.1:0/4188492310 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5567106e03b0 con 0x55671090d010 2024-02-02T15:29:37.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.190+0000 7ff003e00700 1 --2- 172.21.15.1:0/4188492310 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567109187e0 0x55671091abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:29:37.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.190+0000 7ff002a00700 1 --2- 172.21.15.1:0/4188492310 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55671090d930 0x5567109182a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:29:37.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.190+0000 7ff003400700 1 --2- 172.21.15.1:0/4188492310 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55671090d010 0x55671090d3f0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7feff40a2020 tx=0x7feff409a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82b7f57f6bcf4556 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:37.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.190+0000 7feff8c00700 1 -- 172.21.15.1:0/4188492310 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff409a570 con 0x55671090d010 2024-02-02T15:29:37.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.191+0000 7feff8c00700 1 -- 172.21.15.1:0/4188492310 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feff409e3a0 con 0x55671090d010 2024-02-02T15:29:37.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.191+0000 7feff8c00700 1 -- 172.21.15.1:0/4188492310 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff409a7d0 con 0x55671090d010 2024-02-02T15:29:37.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.191+0000 7ff01e165b80 1 -- 172.21.15.1:0/4188492310 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55671090d010 msgr2=0x55671090d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:37.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.191+0000 7ff01e165b80 1 --2- 172.21.15.1:0/4188492310 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55671090d010 0x55671090d3f0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7feff40a2020 tx=0x7feff409a130 comp rx=0 tx=0).stop 2024-02-02T15:29:37.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.191+0000 7ff01e165b80 1 -- 172.21.15.1:0/4188492310 shutdown_connections 2024-02-02T15:29:37.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.191+0000 7ff01e165b80 1 --2- 172.21.15.1:0/4188492310 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567109187e0 0x55671091abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.191+0000 7ff01e165b80 1 --2- 172.21.15.1:0/4188492310 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55671090d010 0x55671090d3f0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.191+0000 7ff01e165b80 1 --2- 172.21.15.1:0/4188492310 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55671090d930 0x5567109182a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.191+0000 7ff01e165b80 1 -- 172.21.15.1:0/4188492310 >> 172.21.15.1:0/4188492310 conn(0x55671086cdb0 msgr2=0x5567109093b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:37.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.191+0000 7ff01e165b80 1 -- 172.21.15.1:0/4188492310 shutdown_connections 2024-02-02T15:29:37.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.191+0000 7ff01e165b80 1 -- 172.21.15.1:0/4188492310 wait complete. 2024-02-02T15:29:37.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.192+0000 7ff01e165b80 1 -- start start 2024-02-02T15:29:37.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.192+0000 7ff01e165b80 1 Processor -- start 2024-02-02T15:29:37.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.193+0000 7ff01e165b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55671090d010 0x556710918bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:37.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.193+0000 7ff01e165b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55671090d930 0x556710919100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:37.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.193+0000 7ff01e165b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5567109187e0 0x556710919640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:37.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.193+0000 7ff01e165b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55671085ed50 con 0x5567109187e0 2024-02-02T15:29:37.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.193+0000 7ff01e165b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567107fad80 con 0x55671090d930 2024-02-02T15:29:37.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.193+0000 7ff01e165b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567107faa20 con 0x55671090d010 2024-02-02T15:29:37.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.193+0000 7ff01e165b80 10 client.0 _ll_register_callbacks cb 0x5567107fc290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:29:37.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.193+0000 7ff003400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55671090d010 0x556710918bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:37.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.193+0000 7ff003400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55671090d010 0x556710918bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:40506/0 (socket says 172.21.15.1:40506) 2024-02-02T15:29:37.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.193+0000 7ff003400700 1 -- 172.21.15.1:0/208972609 learned_addr learned my addr 172.21.15.1:0/208972609 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:29:37.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.193+0000 7ff003e00700 1 --2- 172.21.15.1:0/208972609 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5567109187e0 0x556710919640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:37.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.193+0000 7ff002a00700 1 --2- 172.21.15.1:0/208972609 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55671090d930 0x556710919100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:37.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.193+0000 7ff003400700 1 -- 172.21.15.1:0/208972609 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55671090d930 msgr2=0x556710919100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:37.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.193+0000 7ff003400700 1 --2- 172.21.15.1:0/208972609 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55671090d930 0x556710919100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.193+0000 7ff003400700 1 -- 172.21.15.1:0/208972609 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5567109187e0 msgr2=0x556710919640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:37.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.193+0000 7ff003400700 1 --2- 172.21.15.1:0/208972609 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5567109187e0 0x556710919640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.193+0000 7ff003400700 1 -- 172.21.15.1:0/208972609 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0041a6190 con 0x55671090d010 2024-02-02T15:29:37.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.194+0000 7ff003400700 1 --2- 172.21.15.1:0/208972609 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55671090d010 0x556710918bc0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7feff4000c00 tx=0x7feff4096210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a772cb2c253f88c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:37.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.194+0000 7feff1600700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:29:37.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.194+0000 7feff1600700 1 -- 172.21.15.1:0/208972609 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff40a2550 con 0x55671090d010 2024-02-02T15:29:37.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.194+0000 7feff1600700 1 -- 172.21.15.1:0/208972609 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7feff409a570 con 0x55671090d010 2024-02-02T15:29:37.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.194+0000 7ff01e165b80 1 -- 172.21.15.1:0/208972609 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5567109010f0 con 0x55671090d010 2024-02-02T15:29:37.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.194+0000 7feff1600700 1 -- 172.21.15.1:0/208972609 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7feff4099a60 con 0x55671090d010 2024-02-02T15:29:37.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.194+0000 7ff01e165b80 10 client.26314 fetch_fsmap learned FSMap version 31 2024-02-02T15:29:37.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.194+0000 7ff01e165b80 1 -- 172.21.15.1:0/208972609 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55671068cb60 con 0x55671090d010 2024-02-02T15:29:37.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.194+0000 7feff1600700 1 -- 172.21.15.1:0/208972609 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5567109010f0 con 0x55671090d010 2024-02-02T15:29:37.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.194+0000 7feff1600700 1 -- 172.21.15.1:0/208972609 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 31) v1 ==== 7502+0+0 (secure 0 0 0) 0x7feff409a8f0 con 0x55671090d010 2024-02-02T15:29:37.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.195+0000 7ff01e165b80 10 client.26314 fetch_fsmap finished waiting for FSMap version 31 2024-02-02T15:29:37.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.195+0000 7ff01e165b80 10 client.26314 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-02-02T15:29:37.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.195+0000 7ff01e165b80 20 client.26314 populate_metadata read hostname 'smithi001' 2024-02-02T15:29:37.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.195+0000 7ff01e165b80 1 --2- 172.21.15.1:0/208972609 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x5567109279a0 0x556710929dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:37.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.195+0000 7ff01e165b80 4 client.26314 mds_command: new command op to 24514 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:29:37.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.195+0000 7ff01e165b80 1 -- 172.21.15.1:0/208972609 --> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556710857dd0 con 0x5567109279a0 2024-02-02T15:29:37.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.195+0000 7ff002a00700 1 --2- 172.21.15.1:0/208972609 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x5567109279a0 0x556710929dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:37.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.196+0000 7ff002a00700 1 --2- 172.21.15.1:0/208972609 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x5567109279a0 0x556710929dc0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7c757c159876e845 server_cookie=3b6c56e21dc7c53a in_seq=0 out_seq=0 2024-02-02T15:29:37.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.196+0000 7feff1600700 10 client.26314 ms_handle_connect on v2:172.21.15.142:6836/2870289019 2024-02-02T15:29:37.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.196+0000 7feff1600700 1 -- 172.21.15.1:0/208972609 <== mds.3 v2:172.21.15.142:6836/2870289019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x556710857dd0 con 0x5567109279a0 2024-02-02T15:29:37.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.196+0000 7feff1600700 10 client.26314 handle_command_reply: tid=0 2024-02-02T15:29:37.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.196+0000 7ff01e165b80 1 client.26314 shutdown 2024-02-02T15:29:37.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.196+0000 7ff01e165b80 1 -- 172.21.15.1:0/208972609 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55671090d010 msgr2=0x556710918bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:37.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.196+0000 7ff01e165b80 1 --2- 172.21.15.1:0/208972609 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55671090d010 0x556710918bc0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7feff4000c00 tx=0x7feff4096210 comp rx=0 tx=0).stop 2024-02-02T15:29:37.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.197+0000 7ff01e165b80 1 -- 172.21.15.1:0/208972609 shutdown_connections 2024-02-02T15:29:37.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.197+0000 7ff01e165b80 1 --2- 172.21.15.1:0/208972609 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x5567109279a0 0x556710929dc0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.197+0000 7ff01e165b80 1 --2- 172.21.15.1:0/208972609 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5567109187e0 0x556710919640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.197+0000 7ff01e165b80 1 --2- 172.21.15.1:0/208972609 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55671090d010 0x556710918bc0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.197+0000 7ff01e165b80 1 --2- 172.21.15.1:0/208972609 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55671090d930 0x556710919100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.197+0000 7feff1600700 0 client.26314 ms_handle_reset on v2:172.21.15.142:6836/2870289019 2024-02-02T15:29:37.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.197+0000 7ff01e165b80 1 -- 172.21.15.1:0/208972609 >> 172.21.15.1:0/208972609 conn(0x55671086cdb0 msgr2=0x556710908fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:37.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.197+0000 7ff01e165b80 1 -- 172.21.15.1:0/208972609 shutdown_connections 2024-02-02T15:29:37.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.197+0000 7ff01e165b80 1 -- 172.21.15.1:0/208972609 wait complete. 2024-02-02T15:29:37.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.197+0000 7ff01e165b80 20 client.26314 trim_cache size 0 max 16384 2024-02-02T15:29:37.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.210+0000 7ff01e165b80 1 Processor -- start 2024-02-02T15:29:37.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.210+0000 7ff01e165b80 1 -- start start 2024-02-02T15:29:37.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.210+0000 7ff01e165b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55671090d010 0x55671091dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:37.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.211+0000 7ff01e165b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55671090d930 0x556710907b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:37.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.211+0000 7ff01e165b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5567109187e0 0x5567109231b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:37.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.211+0000 7ff01e165b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567106e26a0 con 0x55671090d010 2024-02-02T15:29:37.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.211+0000 7ff01e165b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x556710917c30 con 0x5567109187e0 2024-02-02T15:29:37.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.211+0000 7ff01e165b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556710917e60 con 0x55671090d930 2024-02-02T15:29:37.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.211+0000 7ff003400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55671090d010 0x55671091dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:37.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.211+0000 7ff003400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55671090d010 0x55671091dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:34946/0 (socket says 172.21.15.1:34946) 2024-02-02T15:29:37.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.211+0000 7ff003400700 1 -- 172.21.15.1:0/3716247323 learned_addr learned my addr 172.21.15.1:0/3716247323 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:29:37.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.211+0000 7ff002a00700 1 --2- 172.21.15.1:0/3716247323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55671090d930 0x556710907b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:37.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.211+0000 7ff003e00700 1 --2- 172.21.15.1:0/3716247323 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5567109187e0 0x5567109231b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:37.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.211+0000 7ff003400700 1 -- 172.21.15.1:0/3716247323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55671090d930 msgr2=0x556710907b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:37.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.211+0000 7ff003400700 1 --2- 172.21.15.1:0/3716247323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55671090d930 0x556710907b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.211+0000 7ff003400700 1 -- 172.21.15.1:0/3716247323 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5567109187e0 msgr2=0x5567109231b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:37.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.211+0000 7ff003400700 1 --2- 172.21.15.1:0/3716247323 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5567109187e0 0x5567109231b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.211+0000 7ff003400700 1 -- 172.21.15.1:0/3716247323 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556710824ea0 con 0x55671090d010 2024-02-02T15:29:37.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.211+0000 7ff003400700 1 --2- 172.21.15.1:0/3716247323 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55671090d010 0x55671091dbd0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7feff40988c0 tx=0x7feff4022ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67004c226d0d5b5f server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:37.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.212+0000 7feff9600700 1 -- 172.21.15.1:0/3716247323 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff40a2300 con 0x55671090d010 2024-02-02T15:29:37.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.212+0000 7feff9600700 1 -- 172.21.15.1:0/3716247323 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feff4099ed0 con 0x55671090d010 2024-02-02T15:29:37.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.212+0000 7feff9600700 1 -- 172.21.15.1:0/3716247323 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff4099cc0 con 0x55671090d010 2024-02-02T15:29:37.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.212+0000 7ff01e165b80 1 -- 172.21.15.1:0/3716247323 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55671090d010 msgr2=0x55671091dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:37.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.212+0000 7ff01e165b80 1 --2- 172.21.15.1:0/3716247323 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55671090d010 0x55671091dbd0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7feff40988c0 tx=0x7feff4022ad0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.212+0000 7ff01e165b80 1 -- 172.21.15.1:0/3716247323 shutdown_connections 2024-02-02T15:29:37.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.212+0000 7ff01e165b80 1 --2- 172.21.15.1:0/3716247323 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5567109187e0 0x5567109231b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.212+0000 7ff01e165b80 1 --2- 172.21.15.1:0/3716247323 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55671090d010 0x55671091dbd0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.212+0000 7ff01e165b80 1 --2- 172.21.15.1:0/3716247323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55671090d930 0x556710907b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.212+0000 7ff01e165b80 1 -- 172.21.15.1:0/3716247323 >> 172.21.15.1:0/3716247323 conn(0x55671086cdb0 msgr2=0x556710924140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:37.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.212+0000 7ff01e165b80 1 -- 172.21.15.1:0/3716247323 shutdown_connections 2024-02-02T15:29:37.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.213+0000 7ff01e165b80 1 -- 172.21.15.1:0/3716247323 wait complete. 2024-02-02T15:29:37.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.213+0000 7ff01e165b80 1 -- start start 2024-02-02T15:29:37.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.213+0000 7ff01e165b80 1 Processor -- start 2024-02-02T15:29:37.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.214+0000 7ff01e165b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55671090d010 0x55671091b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:37.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.214+0000 7ff01e165b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55671090d930 0x55671091b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:37.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.214+0000 7ff01e165b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5567109187e0 0x55671091baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:37.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.214+0000 7ff01e165b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55671091dfe0 con 0x55671090d930 2024-02-02T15:29:37.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.214+0000 7ff01e165b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567109784c0 con 0x5567109187e0 2024-02-02T15:29:37.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.214+0000 7ff01e165b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556710978600 con 0x55671090d010 2024-02-02T15:29:37.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.214+0000 7ff01e165b80 10 client.0 _ll_register_callbacks cb 0x5567107fc290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:29:37.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.214+0000 7ff002a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55671090d930 0x55671091b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:37.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.214+0000 7ff002a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55671090d930 0x55671091b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:34954/0 (socket says 172.21.15.1:34954) 2024-02-02T15:29:37.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.214+0000 7ff002a00700 1 -- 172.21.15.1:0/465526367 learned_addr learned my addr 172.21.15.1:0/465526367 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:29:37.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.214+0000 7ff003e00700 1 --2- 172.21.15.1:0/465526367 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5567109187e0 0x55671091baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:37.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.214+0000 7ff003400700 1 --2- 172.21.15.1:0/465526367 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55671090d010 0x55671091b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:37.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.214+0000 7ff002a00700 1 -- 172.21.15.1:0/465526367 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55671090d010 msgr2=0x55671091b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:37.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.214+0000 7ff002a00700 1 --2- 172.21.15.1:0/465526367 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55671090d010 0x55671091b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.214+0000 7ff002a00700 1 -- 172.21.15.1:0/465526367 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5567109187e0 msgr2=0x55671091baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:37.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.214+0000 7ff002a00700 1 --2- 172.21.15.1:0/465526367 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5567109187e0 0x55671091baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.214+0000 7ff002a00700 1 -- 172.21.15.1:0/465526367 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55671068cb60 con 0x55671090d930 2024-02-02T15:29:37.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.214+0000 7ff002a00700 1 --2- 172.21.15.1:0/465526367 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55671090d930 0x55671091b560 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fefec005d90 tx=0x7fefec009b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a324ed148e00b9f6 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:37.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.214+0000 7feff1600700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:29:37.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.215+0000 7feff1600700 1 -- 172.21.15.1:0/465526367 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefec018040 con 0x55671090d930 2024-02-02T15:29:37.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.215+0000 7feff1600700 1 -- 172.21.15.1:0/465526367 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fefec004230 con 0x55671090d930 2024-02-02T15:29:37.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.215+0000 7ff01e165b80 1 -- 172.21.15.1:0/465526367 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feff409e3a0 con 0x55671090d930 2024-02-02T15:29:37.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.215+0000 7feff1600700 1 -- 172.21.15.1:0/465526367 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fefec0043f0 con 0x55671090d930 2024-02-02T15:29:37.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.215+0000 7ff01e165b80 10 client.16275 fetch_fsmap learned FSMap version 31 2024-02-02T15:29:37.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.215+0000 7ff01e165b80 1 -- 172.21.15.1:0/465526367 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5567106e4cd0 con 0x55671090d930 2024-02-02T15:29:37.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.215+0000 7feff1600700 1 -- 172.21.15.1:0/465526367 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feff409e3a0 con 0x55671090d930 2024-02-02T15:29:37.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.215+0000 7feff1600700 1 -- 172.21.15.1:0/465526367 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 31) v1 ==== 7502+0+0 (secure 0 0 0) 0x7fefec003980 con 0x55671090d930 2024-02-02T15:29:37.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.216+0000 7ff01e165b80 10 client.16275 fetch_fsmap finished waiting for FSMap version 31 2024-02-02T15:29:37.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.216+0000 7ff01e165b80 10 client.16275 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-02-02T15:29:37.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.216+0000 7ff01e165b80 20 client.16275 populate_metadata read hostname 'smithi001' 2024-02-02T15:29:37.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.216+0000 7ff01e165b80 1 --2- 172.21.15.1:0/465526367 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x556710979e40 0x55671097c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:37.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.216+0000 7ff01e165b80 4 client.16275 mds_command: new command op to 24514 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:29:37.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.216+0000 7ff01e165b80 1 -- 172.21.15.1:0/465526367 --> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55671081df50 con 0x556710979e40 2024-02-02T15:29:37.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.216+0000 7ff003400700 1 --2- 172.21.15.1:0/465526367 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x556710979e40 0x55671097c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:37.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.216+0000 7ff003400700 1 --2- 172.21.15.1:0/465526367 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x556710979e40 0x55671097c260 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5eb06dfa274861a3 server_cookie=82bd8720c6f1028b in_seq=0 out_seq=0 2024-02-02T15:29:37.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.216+0000 7feff1600700 10 client.16275 ms_handle_connect on v2:172.21.15.142:6836/2870289019 2024-02-02T15:29:37.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.217+0000 7feff1600700 1 -- 172.21.15.1:0/465526367 <== mds.3 v2:172.21.15.142:6836/2870289019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55671081df50 con 0x556710979e40 2024-02-02T15:29:37.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.217+0000 7feff1600700 10 client.16275 handle_command_reply: tid=0 2024-02-02T15:29:37.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.217+0000 7ff01e165b80 1 client.16275 shutdown 2024-02-02T15:29:37.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.217+0000 7ff01e165b80 1 -- 172.21.15.1:0/465526367 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55671090d930 msgr2=0x55671091b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:37.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.217+0000 7ff01e165b80 1 --2- 172.21.15.1:0/465526367 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55671090d930 0x55671091b560 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fefec005d90 tx=0x7fefec009b10 comp rx=0 tx=0).stop 2024-02-02T15:29:37.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.217+0000 7ff01e165b80 1 -- 172.21.15.1:0/465526367 shutdown_connections 2024-02-02T15:29:37.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.217+0000 7ff01e165b80 1 --2- 172.21.15.1:0/465526367 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5567109187e0 0x55671091baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.217+0000 7ff01e165b80 1 --2- 172.21.15.1:0/465526367 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55671090d010 0x55671091b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.217+0000 7ff01e165b80 1 --2- 172.21.15.1:0/465526367 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x556710979e40 0x55671097c260 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.217+0000 7ff01e165b80 1 --2- 172.21.15.1:0/465526367 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55671090d930 0x55671091b560 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.217+0000 7feff1600700 0 client.16275 ms_handle_reset on v2:172.21.15.142:6836/2870289019 2024-02-02T15:29:37.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.217+0000 7ff01e165b80 1 -- 172.21.15.1:0/465526367 >> 172.21.15.1:0/465526367 conn(0x55671086cdb0 msgr2=0x556710912b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:37.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.217+0000 7ff01e165b80 1 -- 172.21.15.1:0/465526367 shutdown_connections 2024-02-02T15:29:37.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.217+0000 7ff01e165b80 1 -- 172.21.15.1:0/465526367 wait complete. 2024-02-02T15:29:37.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.217+0000 7ff01e165b80 20 client.16275 trim_cache size 0 max 16384 2024-02-02T15:29:37.222 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:29:37.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.218+0000 7ff00a600700 1 -- 172.21.15.1:0/2439652449 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fefe40a0690 msgr2=0x7fefe40a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:37.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.218+0000 7ff00a600700 1 --2- 172.21.15.1:0/2439652449 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fefe40a0690 0x7fefe40a2b50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7feffc00f590 tx=0x7feffc00b040 comp rx=0 tx=0).stop 2024-02-02T15:29:37.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.218+0000 7ff00a600700 1 -- 172.21.15.1:0/2439652449 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff004100ff0 msgr2=0x7ff004199120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:37.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.218+0000 7ff00a600700 1 --2- 172.21.15.1:0/2439652449 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff004100ff0 0x7ff004199120 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7feff4005910 tx=0x7feff4004730 comp rx=0 tx=0).stop 2024-02-02T15:29:37.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.218+0000 7ff00a600700 1 -- 172.21.15.1:0/2439652449 shutdown_connections 2024-02-02T15:29:37.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.218+0000 7ff00a600700 1 --2- 172.21.15.1:0/2439652449 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fefe40a0690 0x7fefe40a2b50 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.218+0000 7ff00a600700 1 --2- 172.21.15.1:0/2439652449 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff00419dac0 0x7ff00419df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.218+0000 7ff00a600700 1 --2- 172.21.15.1:0/2439652449 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff004100ff0 0x7ff004199120 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.218+0000 7ff00a600700 1 --2- 172.21.15.1:0/2439652449 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff004106110 0x7ff004199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.218+0000 7ff00a600700 1 -- 172.21.15.1:0/2439652449 >> 172.21.15.1:0/2439652449 conn(0x7ff0040fc810 msgr2=0x7ff0040fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:37.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.218+0000 7ff00a600700 1 -- 172.21.15.1:0/2439652449 shutdown_connections 2024-02-02T15:29:37.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.218+0000 7ff00a600700 1 -- 172.21.15.1:0/2439652449 wait complete. 2024-02-02T15:29:37.224 INFO:tasks.workunit.client.0.smithi001.stdout: 1 103063 8.09 MB/sec warmup 71 sec latency 1054.620 ms 2024-02-02T15:29:37.236 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-02T15:29:37.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.457+0000 7f73d4400700 1 -- 172.21.15.1:0/4233691781 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73cc100ff0 msgr2=0x7f73cc1013d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:37.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.457+0000 7f73d4400700 1 --2- 172.21.15.1:0/4233691781 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73cc100ff0 0x7f73cc1013d0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f73bc009990 tx=0x7f73bc009ca0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.457+0000 7f73d4400700 1 -- 172.21.15.1:0/4233691781 shutdown_connections 2024-02-02T15:29:37.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.457+0000 7f73d4400700 1 --2- 172.21.15.1:0/4233691781 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f73cc106110 0x7f73cc10dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.457+0000 7f73d4400700 1 --2- 172.21.15.1:0/4233691781 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73cc100ff0 0x7f73cc1013d0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.457+0000 7f73d4400700 1 --2- 172.21.15.1:0/4233691781 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f73cc1019a0 0x7f73cc1059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.457+0000 7f73d4400700 1 -- 172.21.15.1:0/4233691781 >> 172.21.15.1:0/4233691781 conn(0x7f73cc0fc810 msgr2=0x7f73cc0fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:37.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.457+0000 7f73d4400700 1 -- 172.21.15.1:0/4233691781 shutdown_connections 2024-02-02T15:29:37.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.457+0000 7f73d4400700 1 -- 172.21.15.1:0/4233691781 wait complete. 2024-02-02T15:29:37.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.458+0000 7f73d4400700 1 Processor -- start 2024-02-02T15:29:37.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.458+0000 7f73d4400700 1 -- start start 2024-02-02T15:29:37.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.458+0000 7f73d4400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f73cc100ff0 0x7f73cc196f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:37.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.458+0000 7f73d4400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f73cc106110 0x7f73cc197440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:37.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.458+0000 7f73d4400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73cc19b8a0 0x7f73cc19bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:37.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.458+0000 7f73d4400700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73cc19c320 con 0x7f73cc106110 2024-02-02T15:29:37.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.458+0000 7f73d4400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73cc19c490 con 0x7f73cc100ff0 2024-02-02T15:29:37.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.458+0000 7f73d4400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73cc19c650 con 0x7f73cc19b8a0 2024-02-02T15:29:37.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.459+0000 7f73d1000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f73cc106110 0x7f73cc197440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:37.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.459+0000 7f73d1000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f73cc106110 0x7f73cc197440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:34968/0 (socket says 172.21.15.1:34968) 2024-02-02T15:29:37.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.459+0000 7f73d1000700 1 -- 172.21.15.1:0/4225671869 learned_addr learned my addr 172.21.15.1:0/4225671869 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:29:37.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.459+0000 7f73d2400700 1 --2- 172.21.15.1:0/4225671869 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73cc19b8a0 0x7f73cc19bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:37.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.459+0000 7f73d1a00700 1 --2- 172.21.15.1:0/4225671869 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f73cc100ff0 0x7f73cc196f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:37.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.459+0000 7f73d1000700 1 -- 172.21.15.1:0/4225671869 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73cc19b8a0 msgr2=0x7f73cc19bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:37.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.459+0000 7f73d1000700 1 --2- 172.21.15.1:0/4225671869 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73cc19b8a0 0x7f73cc19bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.459+0000 7f73d1000700 1 -- 172.21.15.1:0/4225671869 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f73cc100ff0 msgr2=0x7f73cc196f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:37.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.459+0000 7f73d1000700 1 --2- 172.21.15.1:0/4225671869 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f73cc100ff0 0x7f73cc196f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.459+0000 7f73d1000700 1 -- 172.21.15.1:0/4225671869 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73bc0095f0 con 0x7f73cc106110 2024-02-02T15:29:37.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.459+0000 7f73d1000700 1 --2- 172.21.15.1:0/4225671869 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f73cc106110 0x7f73cc197440 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f73b400e970 tx=0x7f73b400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:37.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.459+0000 7f73ca000700 1 -- 172.21.15.1:0/4225671869 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b4009790 con 0x7f73cc106110 2024-02-02T15:29:37.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.459+0000 7f73ca000700 1 -- 172.21.15.1:0/4225671869 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f73b400c7e0 con 0x7f73cc106110 2024-02-02T15:29:37.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.460+0000 7f73ca000700 1 -- 172.21.15.1:0/4225671869 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b40054e0 con 0x7f73cc106110 2024-02-02T15:29:37.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.460+0000 7f73d4400700 1 -- 172.21.15.1:0/4225671869 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73cc1a3bb0 con 0x7f73cc106110 2024-02-02T15:29:37.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.460+0000 7f73d4400700 1 -- 172.21.15.1:0/4225671869 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73cc1a4040 con 0x7f73cc106110 2024-02-02T15:29:37.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.463+0000 7f73ca000700 1 -- 172.21.15.1:0/4225671869 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 27) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f73b400c950 con 0x7f73cc106110 2024-02-02T15:29:37.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.464+0000 7f73e7ebdb80 1 Processor -- start 2024-02-02T15:29:37.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.464+0000 7f73ca000700 1 --2- 172.21.15.1:0/4225671869 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f73ac0a0710 0x7f73ac0a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:37.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.464+0000 7f73e7ebdb80 1 -- start start 2024-02-02T15:29:37.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.464+0000 7f73ca000700 1 -- 172.21.15.1:0/4225671869 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f73b4014070 con 0x7f73cc106110 2024-02-02T15:29:37.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.464+0000 7f73d2400700 1 --2- 172.21.15.1:0/4225671869 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f73ac0a0710 0x7f73ac0a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:37.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.464+0000 7f73e7ebdb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556deb369010 0x556deb3693f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:37.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.464+0000 7f73e7ebdb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556deb369930 0x556deb3742a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:37.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.464+0000 7f73e7ebdb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556deb3747e0 0x556deb376bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:37.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.464+0000 7f73e7ebdb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x556deb2dd4c0 con 0x556deb3747e0 2024-02-02T15:29:37.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.464+0000 7f73e7ebdb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x556deb2c1aa0 con 0x556deb369010 2024-02-02T15:29:37.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.464+0000 7f73e7ebdb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556deb2bf460 con 0x556deb369930 2024-02-02T15:29:37.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.465+0000 7f73d1000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556deb369930 0x556deb3742a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:37.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.465+0000 7f73d1000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556deb369930 0x556deb3742a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:40550/0 (socket says 172.21.15.1:40550) 2024-02-02T15:29:37.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.465+0000 7f73d1000700 1 -- 172.21.15.1:0/2189466755 learned_addr learned my addr 172.21.15.1:0/2189466755 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:29:37.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.465+0000 7f73d1a00700 1 --2- 172.21.15.1:0/2189466755 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556deb369010 0x556deb3693f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:37.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.465+0000 7f73d2400700 1 --2- 172.21.15.1:0/2189466755 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556deb3747e0 0x556deb376bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:37.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.465+0000 7f73d1000700 1 -- 172.21.15.1:0/2189466755 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556deb369010 msgr2=0x556deb3693f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:37.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.465+0000 7f73d1000700 1 --2- 172.21.15.1:0/2189466755 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556deb369010 0x556deb3693f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.465+0000 7f73d1000700 1 -- 172.21.15.1:0/2189466755 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556deb3747e0 msgr2=0x556deb376bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:37.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.465+0000 7f73d1000700 1 --2- 172.21.15.1:0/2189466755 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556deb3747e0 0x556deb376bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.465+0000 7f73d2400700 1 --2- 172.21.15.1:0/4225671869 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f73ac0a0710 0x7f73ac0a2bd0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f73c400f590 tx=0x7f73c4009920 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:37.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.465+0000 7f73d1000700 1 -- 172.21.15.1:0/2189466755 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556deb13c3b0 con 0x556deb369930 2024-02-02T15:29:37.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.465+0000 7f73d1000700 1 --2- 172.21.15.1:0/2189466755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556deb369930 0x556deb3742a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f73b400efd0 tx=0x7f73b4099060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=211a2ad2f57abd99 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:37.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.466+0000 7f73c2a00700 1 -- 172.21.15.1:0/2189466755 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b40840c0 con 0x556deb369930 2024-02-02T15:29:37.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.466+0000 7f73c2a00700 1 -- 172.21.15.1:0/2189466755 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f73b4099410 con 0x556deb369930 2024-02-02T15:29:37.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.466+0000 7f73c2a00700 1 -- 172.21.15.1:0/2189466755 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b4095b20 con 0x556deb369930 2024-02-02T15:29:37.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.466+0000 7f73e7ebdb80 1 -- 172.21.15.1:0/2189466755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556deb369930 msgr2=0x556deb3742a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:37.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.466+0000 7f73e7ebdb80 1 --2- 172.21.15.1:0/2189466755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556deb369930 0x556deb3742a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f73b400efd0 tx=0x7f73b4099060 comp rx=0 tx=0).stop 2024-02-02T15:29:37.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.466+0000 7f73e7ebdb80 1 -- 172.21.15.1:0/2189466755 shutdown_connections 2024-02-02T15:29:37.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.466+0000 7f73e7ebdb80 1 --2- 172.21.15.1:0/2189466755 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556deb3747e0 0x556deb376bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.466+0000 7f73e7ebdb80 1 --2- 172.21.15.1:0/2189466755 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556deb369010 0x556deb3693f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.466+0000 7f73e7ebdb80 1 --2- 172.21.15.1:0/2189466755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556deb369930 0x556deb3742a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.466+0000 7f73e7ebdb80 1 -- 172.21.15.1:0/2189466755 >> 172.21.15.1:0/2189466755 conn(0x556deb2c8db0 msgr2=0x556deb3653b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:37.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.467+0000 7f73e7ebdb80 1 -- 172.21.15.1:0/2189466755 shutdown_connections 2024-02-02T15:29:37.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.467+0000 7f73e7ebdb80 1 -- 172.21.15.1:0/2189466755 wait complete. 2024-02-02T15:29:37.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.467+0000 7f73e7ebdb80 1 -- start start 2024-02-02T15:29:37.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.467+0000 7f73e7ebdb80 1 Processor -- start 2024-02-02T15:29:37.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.468+0000 7f73e7ebdb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556deb369010 0x556deb371550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:37.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.468+0000 7f73e7ebdb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556deb369930 0x556deb371a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:37.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.468+0000 7f73e7ebdb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556deb3747e0 0x556deb371fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:37.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.468+0000 7f73e7ebdb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x556deb2bad50 con 0x556deb3747e0 2024-02-02T15:29:37.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.468+0000 7f73e7ebdb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x556deb256d80 con 0x556deb369010 2024-02-02T15:29:37.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.468+0000 7f73e7ebdb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556deb256a20 con 0x556deb369930 2024-02-02T15:29:37.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.468+0000 7f73e7ebdb80 10 client.0 _ll_register_callbacks cb 0x556deb258290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:29:37.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.468+0000 7f73d2400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556deb3747e0 0x556deb371fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:37.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.468+0000 7f73d1000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556deb369930 0x556deb371a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:37.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.468+0000 7f73d1000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556deb369930 0x556deb371a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:40556/0 (socket says 172.21.15.1:40556) 2024-02-02T15:29:37.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.468+0000 7f73d1a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556deb369010 0x556deb371550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:37.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.468+0000 7f73d1000700 1 -- 172.21.15.1:0/2724328622 learned_addr learned my addr 172.21.15.1:0/2724328622 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:29:37.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.468+0000 7f73d2400700 1 --2- 172.21.15.1:0/2724328622 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556deb3747e0 0x556deb371fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:34972/0 (socket says 172.21.15.1:34972) 2024-02-02T15:29:37.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.469+0000 7f73d1000700 1 -- 172.21.15.1:0/2724328622 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556deb369010 msgr2=0x556deb371550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:37.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.469+0000 7f73d1000700 1 --2- 172.21.15.1:0/2724328622 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556deb369010 0x556deb371550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.469+0000 7f73d1000700 1 -- 172.21.15.1:0/2724328622 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556deb3747e0 msgr2=0x556deb371fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:37.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.469+0000 7f73d1000700 1 --2- 172.21.15.1:0/2724328622 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556deb3747e0 0x556deb371fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.469+0000 7f73d1000700 1 -- 172.21.15.1:0/2724328622 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73cc1a4040 con 0x556deb369930 2024-02-02T15:29:37.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.469+0000 7f73d1000700 1 --2- 172.21.15.1:0/2724328622 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556deb369930 0x556deb371a90 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f73b40a0f50 tx=0x7f73b4004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c78ee105cc80399 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:37.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.469+0000 7f73bb400700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:29:37.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.469+0000 7f73bb400700 1 -- 172.21.15.1:0/2724328622 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b40b71c0 con 0x556deb369930 2024-02-02T15:29:37.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.469+0000 7f73e7ebdb80 1 -- 172.21.15.1:0/2724328622 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556deb35d0f0 con 0x556deb369930 2024-02-02T15:29:37.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.469+0000 7f73bb400700 1 -- 172.21.15.1:0/2724328622 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f73b40b7320 con 0x556deb369930 2024-02-02T15:29:37.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.470+0000 7f73bb400700 1 -- 172.21.15.1:0/2724328622 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f73b4099670 con 0x556deb369930 2024-02-02T15:29:37.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.470+0000 7f73e7ebdb80 10 client.26332 fetch_fsmap learned FSMap version 31 2024-02-02T15:29:37.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.470+0000 7f73e7ebdb80 1 -- 172.21.15.1:0/2724328622 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x556deb0e8b60 con 0x556deb369930 2024-02-02T15:29:37.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.470+0000 7f73bb400700 1 -- 172.21.15.1:0/2724328622 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556deb35d0f0 con 0x556deb369930 2024-02-02T15:29:37.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.470+0000 7f73bb400700 1 -- 172.21.15.1:0/2724328622 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 31) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f73b40a0ff0 con 0x556deb369930 2024-02-02T15:29:37.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.470+0000 7f73e7ebdb80 10 client.26332 fetch_fsmap finished waiting for FSMap version 31 2024-02-02T15:29:37.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.470+0000 7f73e7ebdb80 10 client.26332 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-02-02T15:29:37.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.470+0000 7f73e7ebdb80 20 client.26332 populate_metadata read hostname 'smithi001' 2024-02-02T15:29:37.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.470+0000 7f73e7ebdb80 1 --2- 172.21.15.1:0/2724328622 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x556deb383ea0 0x556deb3862c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:37.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.470+0000 7f73e7ebdb80 4 client.26332 mds_command: new command op to 24518 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:29:37.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.470+0000 7f73e7ebdb80 1 -- 172.21.15.1:0/2724328622 --> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556deb2b3dd0 con 0x556deb383ea0 2024-02-02T15:29:37.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.470+0000 7f73d1a00700 1 --2- 172.21.15.1:0/2724328622 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x556deb383ea0 0x556deb3862c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:37.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.471+0000 7f73d1a00700 1 --2- 172.21.15.1:0/2724328622 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x556deb383ea0 0x556deb3862c0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f38aaa195b2403d2 server_cookie=9561ef2f4bb6d701 in_seq=0 out_seq=0 2024-02-02T15:29:37.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.471+0000 7f73bb400700 10 client.26332 ms_handle_connect on v2:172.21.15.71:6836/628016199 2024-02-02T15:29:37.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.472+0000 7f73bb400700 1 -- 172.21.15.1:0/2724328622 <== mds.2 v2:172.21.15.71:6836/628016199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x556deb2b3dd0 con 0x556deb383ea0 2024-02-02T15:29:37.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.472+0000 7f73bb400700 10 client.26332 handle_command_reply: tid=0 2024-02-02T15:29:37.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.472+0000 7f73e7ebdb80 1 client.26332 shutdown 2024-02-02T15:29:37.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.472+0000 7f73e7ebdb80 1 -- 172.21.15.1:0/2724328622 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556deb369930 msgr2=0x556deb371a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:37.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.472+0000 7f73e7ebdb80 1 --2- 172.21.15.1:0/2724328622 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556deb369930 0x556deb371a90 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f73b40a0f50 tx=0x7f73b4004750 comp rx=0 tx=0).stop 2024-02-02T15:29:37.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.472+0000 7f73e7ebdb80 1 -- 172.21.15.1:0/2724328622 shutdown_connections 2024-02-02T15:29:37.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.472+0000 7f73e7ebdb80 1 --2- 172.21.15.1:0/2724328622 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556deb3747e0 0x556deb371fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.472+0000 7f73e7ebdb80 1 --2- 172.21.15.1:0/2724328622 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556deb369010 0x556deb371550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.472+0000 7f73e7ebdb80 1 --2- 172.21.15.1:0/2724328622 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x556deb383ea0 0x556deb3862c0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.472+0000 7f73e7ebdb80 1 --2- 172.21.15.1:0/2724328622 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556deb369930 0x556deb371a90 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.472+0000 7f73bb400700 0 client.26332 ms_handle_reset on v2:172.21.15.71:6836/628016199 2024-02-02T15:29:37.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.472+0000 7f73e7ebdb80 1 -- 172.21.15.1:0/2724328622 >> 172.21.15.1:0/2724328622 conn(0x556deb2c8db0 msgr2=0x556deb364f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:37.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.472+0000 7f73e7ebdb80 1 -- 172.21.15.1:0/2724328622 shutdown_connections 2024-02-02T15:29:37.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.472+0000 7f73e7ebdb80 1 -- 172.21.15.1:0/2724328622 wait complete. 2024-02-02T15:29:37.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.472+0000 7f73e7ebdb80 20 client.26332 trim_cache size 0 max 16384 2024-02-02T15:29:37.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.485+0000 7f73e7ebdb80 1 Processor -- start 2024-02-02T15:29:37.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.485+0000 7f73e7ebdb80 1 -- start start 2024-02-02T15:29:37.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.486+0000 7f73e7ebdb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556deb369010 0x556deb37b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:37.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.486+0000 7f73e7ebdb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556deb369930 0x556deb371b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:37.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.486+0000 7f73e7ebdb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556deb3747e0 0x556deb372050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:37.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.486+0000 7f73e7ebdb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x556deb13e6a0 con 0x556deb3747e0 2024-02-02T15:29:37.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.486+0000 7f73e7ebdb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x556deb380190 con 0x556deb369010 2024-02-02T15:29:37.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.486+0000 7f73e7ebdb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556deb36d3f0 con 0x556deb369930 2024-02-02T15:29:37.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.486+0000 7f73d2400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556deb3747e0 0x556deb372050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:37.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.486+0000 7f73d2400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556deb3747e0 0x556deb372050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:34984/0 (socket says 172.21.15.1:34984) 2024-02-02T15:29:37.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.486+0000 7f73d2400700 1 -- 172.21.15.1:0/2366045381 learned_addr learned my addr 172.21.15.1:0/2366045381 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:29:37.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.486+0000 7f73d1a00700 1 --2- 172.21.15.1:0/2366045381 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556deb369010 0x556deb37b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:37.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.486+0000 7f73d1000700 1 --2- 172.21.15.1:0/2366045381 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556deb369930 0x556deb371b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:37.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.486+0000 7f73d2400700 1 -- 172.21.15.1:0/2366045381 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556deb369930 msgr2=0x556deb371b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:37.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.486+0000 7f73d2400700 1 --2- 172.21.15.1:0/2366045381 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556deb369930 0x556deb371b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.486+0000 7f73d2400700 1 -- 172.21.15.1:0/2366045381 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556deb369010 msgr2=0x556deb37b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:37.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.486+0000 7f73d2400700 1 --2- 172.21.15.1:0/2366045381 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556deb369010 0x556deb37b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.486+0000 7f73d2400700 1 -- 172.21.15.1:0/2366045381 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556deb280ea0 con 0x556deb3747e0 2024-02-02T15:29:37.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.486+0000 7f73d1000700 1 --2- 172.21.15.1:0/2366045381 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556deb369930 0x556deb371b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:29:37.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.486+0000 7f73d1a00700 1 --2- 172.21.15.1:0/2366045381 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556deb369010 0x556deb37b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:29:37.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.487+0000 7f73d2400700 1 --2- 172.21.15.1:0/2366045381 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556deb3747e0 0x556deb372050 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f73c4022a30 tx=0x7f73c401c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a5371604f271b3 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:37.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.487+0000 7f73c3400700 1 -- 172.21.15.1:0/2366045381 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c4027070 con 0x556deb3747e0 2024-02-02T15:29:37.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.487+0000 7f73c3400700 1 -- 172.21.15.1:0/2366045381 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f73c400f040 con 0x556deb3747e0 2024-02-02T15:29:37.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.487+0000 7f73c3400700 1 -- 172.21.15.1:0/2366045381 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c400e040 con 0x556deb3747e0 2024-02-02T15:29:37.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.487+0000 7f73e7ebdb80 1 -- 172.21.15.1:0/2366045381 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556deb3747e0 msgr2=0x556deb372050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:37.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.487+0000 7f73e7ebdb80 1 --2- 172.21.15.1:0/2366045381 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556deb3747e0 0x556deb372050 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f73c4022a30 tx=0x7f73c401c000 comp rx=0 tx=0).stop 2024-02-02T15:29:37.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.488+0000 7f73e7ebdb80 1 -- 172.21.15.1:0/2366045381 shutdown_connections 2024-02-02T15:29:37.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.488+0000 7f73e7ebdb80 1 --2- 172.21.15.1:0/2366045381 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556deb3747e0 0x556deb372050 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.488+0000 7f73e7ebdb80 1 --2- 172.21.15.1:0/2366045381 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556deb369010 0x556deb37b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.488+0000 7f73e7ebdb80 1 --2- 172.21.15.1:0/2366045381 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556deb369930 0x556deb371b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.488+0000 7f73e7ebdb80 1 -- 172.21.15.1:0/2366045381 >> 172.21.15.1:0/2366045381 conn(0x556deb2c8db0 msgr2=0x556deb36c880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:37.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.488+0000 7f73e7ebdb80 1 -- 172.21.15.1:0/2366045381 shutdown_connections 2024-02-02T15:29:37.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.488+0000 7f73e7ebdb80 1 -- 172.21.15.1:0/2366045381 wait complete. 2024-02-02T15:29:37.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.488+0000 7f73e7ebdb80 1 -- start start 2024-02-02T15:29:37.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.488+0000 7f73e7ebdb80 1 Processor -- start 2024-02-02T15:29:37.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.489+0000 7f73e7ebdb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556deb369010 0x556deb3cf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:37.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.489+0000 7f73e7ebdb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556deb369930 0x556deb3cf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:37.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.489+0000 7f73e7ebdb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556deb3747e0 0x556deb3c9910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:37.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.489+0000 7f73e7ebdb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x556deb3ca020 con 0x556deb369930 2024-02-02T15:29:37.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.489+0000 7f73e7ebdb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x556deb3ca190 con 0x556deb3747e0 2024-02-02T15:29:37.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.489+0000 7f73e7ebdb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556deb3ca350 con 0x556deb369010 2024-02-02T15:29:37.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.489+0000 7f73e7ebdb80 10 client.0 _ll_register_callbacks cb 0x556deb258290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:29:37.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.489+0000 7f73d1000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556deb369930 0x556deb3cf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:37.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.489+0000 7f73d1000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556deb369930 0x556deb3cf620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35000/0 (socket says 172.21.15.1:35000) 2024-02-02T15:29:37.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.489+0000 7f73d1000700 1 -- 172.21.15.1:0/1847052064 learned_addr learned my addr 172.21.15.1:0/1847052064 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:29:37.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.489+0000 7f73d1a00700 1 --2- 172.21.15.1:0/1847052064 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556deb369010 0x556deb3cf0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:37.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.489+0000 7f73d2400700 1 --2- 172.21.15.1:0/1847052064 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556deb3747e0 0x556deb3c9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:37.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.489+0000 7f73d1000700 1 -- 172.21.15.1:0/1847052064 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556deb369010 msgr2=0x556deb3cf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:37.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.489+0000 7f73d1000700 1 --2- 172.21.15.1:0/1847052064 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556deb369010 0x556deb3cf0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.489+0000 7f73d1000700 1 -- 172.21.15.1:0/1847052064 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556deb3747e0 msgr2=0x556deb3c9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:37.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.489+0000 7f73d1000700 1 --2- 172.21.15.1:0/1847052064 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556deb3747e0 0x556deb3c9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.489+0000 7f73d1000700 1 -- 172.21.15.1:0/1847052064 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73cc10e500 con 0x556deb369930 2024-02-02T15:29:37.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.489+0000 7f73d2400700 1 --2- 172.21.15.1:0/1847052064 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556deb3747e0 0x556deb3c9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:29:37.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.490+0000 7f73d1000700 1 --2- 172.21.15.1:0/1847052064 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556deb369930 0x556deb3cf620 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f73b4098d00 tx=0x7f73b4016fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3253cb8fed643e8c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:37.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.490+0000 7f73bb400700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:29:37.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.490+0000 7f73bb400700 1 -- 172.21.15.1:0/1847052064 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73b40b73c0 con 0x556deb369930 2024-02-02T15:29:37.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.490+0000 7f73bb400700 1 -- 172.21.15.1:0/1847052064 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f73b400cc50 con 0x556deb369930 2024-02-02T15:29:37.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.490+0000 7f73e7ebdb80 1 -- 172.21.15.1:0/1847052064 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f73b4099410 con 0x556deb369930 2024-02-02T15:29:37.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.490+0000 7f73bb400700 1 -- 172.21.15.1:0/1847052064 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f73b40b1bb0 con 0x556deb369930 2024-02-02T15:29:37.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.490+0000 7f73e7ebdb80 10 client.16293 fetch_fsmap learned FSMap version 31 2024-02-02T15:29:37.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.490+0000 7f73e7ebdb80 1 -- 172.21.15.1:0/1847052064 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f73c40226b0 con 0x556deb369930 2024-02-02T15:29:37.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.490+0000 7f73bb400700 1 -- 172.21.15.1:0/1847052064 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f73b4099410 con 0x556deb369930 2024-02-02T15:29:37.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.490+0000 7f73bb400700 1 -- 172.21.15.1:0/1847052064 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 31) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f73b4004a30 con 0x556deb369930 2024-02-02T15:29:37.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.490+0000 7f73e7ebdb80 10 client.16293 fetch_fsmap finished waiting for FSMap version 31 2024-02-02T15:29:37.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.490+0000 7f73e7ebdb80 10 client.16293 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-02-02T15:29:37.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.490+0000 7f73e7ebdb80 20 client.16293 populate_metadata read hostname 'smithi001' 2024-02-02T15:29:37.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.490+0000 7f73e7ebdb80 1 --2- 172.21.15.1:0/1847052064 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x556deb3d6a80 0x556deb3d8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:37.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.490+0000 7f73e7ebdb80 4 client.16293 mds_command: new command op to 24518 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:29:37.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.490+0000 7f73e7ebdb80 1 -- 172.21.15.1:0/1847052064 --> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556deb279f50 con 0x556deb3d6a80 2024-02-02T15:29:37.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.491+0000 7f73d1a00700 1 --2- 172.21.15.1:0/1847052064 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x556deb3d6a80 0x556deb3d8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:37.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.491+0000 7f73d1a00700 1 --2- 172.21.15.1:0/1847052064 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x556deb3d6a80 0x556deb3d8ea0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c5e4c73c4618a6d2 server_cookie=3b0ed55d29794568 in_seq=0 out_seq=0 2024-02-02T15:29:37.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.491+0000 7f73bb400700 10 client.16293 ms_handle_connect on v2:172.21.15.71:6836/628016199 2024-02-02T15:29:37.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.491+0000 7f73bb400700 1 -- 172.21.15.1:0/1847052064 <== mds.2 v2:172.21.15.71:6836/628016199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556deb279f50 con 0x556deb3d6a80 2024-02-02T15:29:37.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.491+0000 7f73bb400700 10 client.16293 handle_command_reply: tid=0 2024-02-02T15:29:37.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.491+0000 7f73e7ebdb80 1 client.16293 shutdown 2024-02-02T15:29:37.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.492+0000 7f73e7ebdb80 1 -- 172.21.15.1:0/1847052064 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556deb369930 msgr2=0x556deb3cf620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:37.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.492+0000 7f73e7ebdb80 1 --2- 172.21.15.1:0/1847052064 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556deb369930 0x556deb3cf620 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f73b4098d00 tx=0x7f73b4016fe0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.492+0000 7f73e7ebdb80 1 -- 172.21.15.1:0/1847052064 shutdown_connections 2024-02-02T15:29:37.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.492+0000 7f73e7ebdb80 1 --2- 172.21.15.1:0/1847052064 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556deb3747e0 0x556deb3c9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.492+0000 7f73e7ebdb80 1 --2- 172.21.15.1:0/1847052064 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x556deb3d6a80 0x556deb3d8ea0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.492+0000 7f73e7ebdb80 1 --2- 172.21.15.1:0/1847052064 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556deb369010 0x556deb3cf0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.492+0000 7f73e7ebdb80 1 --2- 172.21.15.1:0/1847052064 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556deb369930 0x556deb3cf620 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.492+0000 7f73bb400700 0 client.16293 ms_handle_reset on v2:172.21.15.71:6836/628016199 2024-02-02T15:29:37.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.492+0000 7f73e7ebdb80 1 -- 172.21.15.1:0/1847052064 >> 172.21.15.1:0/1847052064 conn(0x556deb2c8db0 msgr2=0x556deb371b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:37.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.492+0000 7f73e7ebdb80 1 -- 172.21.15.1:0/1847052064 shutdown_connections 2024-02-02T15:29:37.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.492+0000 7f73e7ebdb80 1 -- 172.21.15.1:0/1847052064 wait complete. 2024-02-02T15:29:37.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.492+0000 7f73e7ebdb80 20 client.16293 trim_cache size 0 max 16384 2024-02-02T15:29:37.497 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:29:37.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.492+0000 7f73d4400700 1 -- 172.21.15.1:0/4225671869 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f73ac0a0710 msgr2=0x7f73ac0a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:37.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.492+0000 7f73d4400700 1 --2- 172.21.15.1:0/4225671869 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f73ac0a0710 0x7f73ac0a2bd0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f73c400f590 tx=0x7f73c4009920 comp rx=0 tx=0).stop 2024-02-02T15:29:37.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.492+0000 7f73d4400700 1 -- 172.21.15.1:0/4225671869 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f73cc106110 msgr2=0x7f73cc197440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:37.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.492+0000 7f73d4400700 1 --2- 172.21.15.1:0/4225671869 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f73cc106110 0x7f73cc197440 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f73b400e970 tx=0x7f73b400ec80 comp rx=0 tx=0).stop 2024-02-02T15:29:37.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.493+0000 7f73d4400700 1 -- 172.21.15.1:0/4225671869 shutdown_connections 2024-02-02T15:29:37.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.493+0000 7f73d4400700 1 --2- 172.21.15.1:0/4225671869 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f73ac0a0710 0x7f73ac0a2bd0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.493+0000 7f73d4400700 1 --2- 172.21.15.1:0/4225671869 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f73cc19b8a0 0x7f73cc19bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.493+0000 7f73d4400700 1 --2- 172.21.15.1:0/4225671869 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f73cc100ff0 0x7f73cc196f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.493+0000 7f73d4400700 1 --2- 172.21.15.1:0/4225671869 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f73cc106110 0x7f73cc197440 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.493+0000 7f73d4400700 1 -- 172.21.15.1:0/4225671869 >> 172.21.15.1:0/4225671869 conn(0x7f73cc0fc810 msgr2=0x7f73cc1049c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:37.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.493+0000 7f73d4400700 1 -- 172.21.15.1:0/4225671869 shutdown_connections 2024-02-02T15:29:37.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.493+0000 7f73d4400700 1 -- 172.21.15.1:0/4225671869 wait complete. 2024-02-02T15:29:37.508 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:29:37 smithi001 ceph-mon[35470]: pgmap v81: 129 pgs: 129 active+clean; 67 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 1.5 MiB/s wr, 136 op/s 2024-02-02T15:29:37.508 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:29:37 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/3075213107' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:29:37.511 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-02T15:29:37.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:29:37 smithi071 ceph-mon[44707]: pgmap v81: 129 pgs: 129 active+clean; 67 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 1.5 MiB/s wr, 136 op/s 2024-02-02T15:29:37.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:29:37 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/3075213107' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:29:37.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:29:37 smithi142 ceph-mon[45728]: pgmap v81: 129 pgs: 129 active+clean; 67 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 568 KiB/s rd, 1.5 MiB/s wr, 136 op/s 2024-02-02T15:29:37.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:29:37 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/3075213107' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:29:37.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.710+0000 7f2a5e200700 1 -- 172.21.15.1:0/2828746859 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a58102950 msgr2=0x7f2a5810ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:37.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.710+0000 7f2a5e200700 1 --2- 172.21.15.1:0/2828746859 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a58102950 0x7f2a5810ae30 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f2a40009990 tx=0x7f2a40009ca0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.710+0000 7f2a5e200700 1 -- 172.21.15.1:0/2828746859 shutdown_connections 2024-02-02T15:29:37.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.710+0000 7f2a5e200700 1 --2- 172.21.15.1:0/2828746859 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a5810b400 0x7f2a5810d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.710+0000 7f2a5e200700 1 --2- 172.21.15.1:0/2828746859 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a58102030 0x7f2a58102410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.710+0000 7f2a5e200700 1 --2- 172.21.15.1:0/2828746859 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a58102950 0x7f2a5810ae30 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.710+0000 7f2a5e200700 1 -- 172.21.15.1:0/2828746859 >> 172.21.15.1:0/2828746859 conn(0x7f2a58076d70 msgr2=0x7f2a58077180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:37.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.710+0000 7f2a5e200700 1 -- 172.21.15.1:0/2828746859 shutdown_connections 2024-02-02T15:29:37.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.710+0000 7f2a5e200700 1 -- 172.21.15.1:0/2828746859 wait complete. 2024-02-02T15:29:37.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.711+0000 7f2a5e200700 1 Processor -- start 2024-02-02T15:29:37.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.711+0000 7f2a5e200700 1 -- start start 2024-02-02T15:29:37.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.712+0000 7f2a5e200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a58102030 0x7f2a581990a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:37.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.712+0000 7f2a5e200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a58102950 0x7f2a581995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:37.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.712+0000 7f2a5e200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a5810b400 0x7f2a5819da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:37.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.712+0000 7f2a5e200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5819e080 con 0x7f2a58102030 2024-02-02T15:29:37.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.712+0000 7f2a5e200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5819e1f0 con 0x7f2a58102950 2024-02-02T15:29:37.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.712+0000 7f2a5e200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5819e3b0 con 0x7f2a5810b400 2024-02-02T15:29:37.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.712+0000 7f2a57400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a58102030 0x7f2a581990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:37.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.712+0000 7f2a56a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a58102950 0x7f2a581995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:37.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.712+0000 7f2a56a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a58102950 0x7f2a581995e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:48868/0 (socket says 172.21.15.1:48868) 2024-02-02T15:29:37.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.712+0000 7f2a56a00700 1 -- 172.21.15.1:0/3727214198 learned_addr learned my addr 172.21.15.1:0/3727214198 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:29:37.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.712+0000 7f2a57e00700 1 --2- 172.21.15.1:0/3727214198 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a5810b400 0x7f2a5819da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:37.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.712+0000 7f2a57400700 1 -- 172.21.15.1:0/3727214198 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a5810b400 msgr2=0x7f2a5819da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:37.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.712+0000 7f2a57400700 1 --2- 172.21.15.1:0/3727214198 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a5810b400 0x7f2a5819da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.713+0000 7f2a57400700 1 -- 172.21.15.1:0/3727214198 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a58102950 msgr2=0x7f2a581995e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:37.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.713+0000 7f2a57400700 1 --2- 172.21.15.1:0/3727214198 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a58102950 0x7f2a581995e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.713+0000 7f2a57400700 1 -- 172.21.15.1:0/3727214198 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a400095f0 con 0x7f2a58102030 2024-02-02T15:29:37.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.713+0000 7f2a56a00700 1 --2- 172.21.15.1:0/3727214198 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a58102950 0x7f2a581995e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:29:37.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.713+0000 7f2a57400700 1 --2- 172.21.15.1:0/3727214198 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a58102030 0x7f2a581990a0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f2a4800e9f0 tx=0x7f2a4800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:37.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.713+0000 7f2a4fe00700 1 -- 172.21.15.1:0/3727214198 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a4800c9c0 con 0x7f2a58102030 2024-02-02T15:29:37.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.713+0000 7f2a4fe00700 1 -- 172.21.15.1:0/3727214198 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a48010430 con 0x7f2a58102030 2024-02-02T15:29:37.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.713+0000 7f2a5e200700 1 -- 172.21.15.1:0/3727214198 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a5819e690 con 0x7f2a58102030 2024-02-02T15:29:37.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.713+0000 7f2a4fe00700 1 -- 172.21.15.1:0/3727214198 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a48018680 con 0x7f2a58102030 2024-02-02T15:29:37.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.713+0000 7f2a5e200700 1 -- 172.21.15.1:0/3727214198 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a581a5ed0 con 0x7f2a58102030 2024-02-02T15:29:37.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.716+0000 7f2a4fe00700 1 -- 172.21.15.1:0/3727214198 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 27) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f2a48010860 con 0x7f2a58102030 2024-02-02T15:29:37.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.717+0000 7f2a4fe00700 1 --2- 172.21.15.1:0/3727214198 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f2a380a0710 0x7f2a380a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:37.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.717+0000 7f2a71cefb80 1 Processor -- start 2024-02-02T15:29:37.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.717+0000 7f2a4fe00700 1 -- 172.21.15.1:0/3727214198 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f2a48014070 con 0x7f2a58102030 2024-02-02T15:29:37.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.718+0000 7f2a71cefb80 1 -- start start 2024-02-02T15:29:37.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.718+0000 7f2a56a00700 1 --2- 172.21.15.1:0/3727214198 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f2a380a0710 0x7f2a380a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:37.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.718+0000 7f2a71cefb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559fbead1010 0x559fbead13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:37.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.718+0000 7f2a71cefb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559fbead1930 0x559fbeadc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:37.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.718+0000 7f2a71cefb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559fbeadc7e0 0x559fbeadebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:37.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.718+0000 7f2a71cefb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fbea454c0 con 0x559fbead1930 2024-02-02T15:29:37.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.718+0000 7f2a71cefb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fbea29aa0 con 0x559fbeadc7e0 2024-02-02T15:29:37.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.718+0000 7f2a71cefb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fbea27460 con 0x559fbead1010 2024-02-02T15:29:37.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.718+0000 7f2a56a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559fbead1930 0x559fbeadc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:37.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.718+0000 7f2a57400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559fbead1010 0x559fbead13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:37.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.719+0000 7f2a57400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559fbead1010 0x559fbead13f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:40604/0 (socket says 172.21.15.1:40604) 2024-02-02T15:29:37.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.719+0000 7f2a57400700 1 -- 172.21.15.1:0/3936576222 learned_addr learned my addr 172.21.15.1:0/3936576222 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:29:37.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.719+0000 7f2a57e00700 1 --2- 172.21.15.1:0/3936576222 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559fbeadc7e0 0x559fbeadebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:37.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.719+0000 7f2a56a00700 1 --2- 172.21.15.1:0/3727214198 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f2a380a0710 0x7f2a380a2bd0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2a5819a6b0 tx=0x7f2a40005c60 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:37.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.719+0000 7f2a57400700 1 -- 172.21.15.1:0/3936576222 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559fbeadc7e0 msgr2=0x559fbeadebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:37.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.719+0000 7f2a57400700 1 --2- 172.21.15.1:0/3936576222 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559fbeadc7e0 0x559fbeadebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.719+0000 7f2a57400700 1 -- 172.21.15.1:0/3936576222 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559fbead1930 msgr2=0x559fbeadc2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:37.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.719+0000 7f2a57400700 1 --2- 172.21.15.1:0/3936576222 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559fbead1930 0x559fbeadc2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.719+0000 7f2a57400700 1 -- 172.21.15.1:0/3936576222 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559fbe8a43b0 con 0x559fbead1010 2024-02-02T15:29:37.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.719+0000 7f2a57400700 1 --2- 172.21.15.1:0/3936576222 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559fbead1010 0x559fbead13f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2a48099040 tx=0x7f2a48098ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6fc5e3aa92b753a server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:37.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.719+0000 7f2a47e00700 1 -- 172.21.15.1:0/3936576222 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a48099340 con 0x559fbead1010 2024-02-02T15:29:37.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.719+0000 7f2a47e00700 1 -- 172.21.15.1:0/3936576222 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a4809d170 con 0x559fbead1010 2024-02-02T15:29:37.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.719+0000 7f2a47e00700 1 -- 172.21.15.1:0/3936576222 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a48099800 con 0x559fbead1010 2024-02-02T15:29:37.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.720+0000 7f2a71cefb80 1 -- 172.21.15.1:0/3936576222 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559fbead1010 msgr2=0x559fbead13f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:37.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.720+0000 7f2a71cefb80 1 --2- 172.21.15.1:0/3936576222 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559fbead1010 0x559fbead13f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2a48099040 tx=0x7f2a48098ca0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.720+0000 7f2a71cefb80 1 -- 172.21.15.1:0/3936576222 shutdown_connections 2024-02-02T15:29:37.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.720+0000 7f2a71cefb80 1 --2- 172.21.15.1:0/3936576222 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559fbeadc7e0 0x559fbeadebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.720+0000 7f2a71cefb80 1 --2- 172.21.15.1:0/3936576222 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559fbead1010 0x559fbead13f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.720+0000 7f2a71cefb80 1 --2- 172.21.15.1:0/3936576222 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559fbead1930 0x559fbeadc2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.720+0000 7f2a71cefb80 1 -- 172.21.15.1:0/3936576222 >> 172.21.15.1:0/3936576222 conn(0x559fbea30db0 msgr2=0x559fbeacd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:37.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.720+0000 7f2a71cefb80 1 -- 172.21.15.1:0/3936576222 shutdown_connections 2024-02-02T15:29:37.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.720+0000 7f2a71cefb80 1 -- 172.21.15.1:0/3936576222 wait complete. 2024-02-02T15:29:37.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.721+0000 7f2a71cefb80 1 -- start start 2024-02-02T15:29:37.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.721+0000 7f2a71cefb80 1 Processor -- start 2024-02-02T15:29:37.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.721+0000 7f2a71cefb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559fbead1010 0x559fbeadc200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:37.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.721+0000 7f2a71cefb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559fbead1930 0x559fbead9480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:37.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.721+0000 7f2a71cefb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559fbeadc7e0 0x559fbead9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:37.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.721+0000 7f2a71cefb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fbea22d50 con 0x559fbeadc7e0 2024-02-02T15:29:37.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.721+0000 7f2a71cefb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fbe9bed80 con 0x559fbead1010 2024-02-02T15:29:37.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.721+0000 7f2a71cefb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fbe9bea20 con 0x559fbead1930 2024-02-02T15:29:37.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.721+0000 7f2a71cefb80 10 client.0 _ll_register_callbacks cb 0x559fbe9c0290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:29:37.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.722+0000 7f2a57400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559fbead1010 0x559fbeadc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:37.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.722+0000 7f2a57e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559fbeadc7e0 0x559fbead9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:37.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.722+0000 7f2a57400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559fbead1010 0x559fbeadc200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:48894/0 (socket says 172.21.15.1:48894) 2024-02-02T15:29:37.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.722+0000 7f2a57400700 1 -- 172.21.15.1:0/1126943742 learned_addr learned my addr 172.21.15.1:0/1126943742 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:29:37.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.722+0000 7f2a56a00700 1 --2- 172.21.15.1:0/1126943742 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559fbead1930 0x559fbead9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:37.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.722+0000 7f2a57400700 1 -- 172.21.15.1:0/1126943742 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559fbead1930 msgr2=0x559fbead9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:37.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.722+0000 7f2a57400700 1 --2- 172.21.15.1:0/1126943742 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559fbead1930 0x559fbead9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.722+0000 7f2a57400700 1 -- 172.21.15.1:0/1126943742 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559fbeadc7e0 msgr2=0x559fbead9ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:37.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.722+0000 7f2a57400700 1 --2- 172.21.15.1:0/1126943742 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559fbeadc7e0 0x559fbead9ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.722+0000 7f2a57400700 1 -- 172.21.15.1:0/1126943742 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a581a5ed0 con 0x559fbead1010 2024-02-02T15:29:37.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.722+0000 7f2a57400700 1 --2- 172.21.15.1:0/1126943742 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559fbead1010 0x559fbeadc200 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2a480992d0 tx=0x7f2a48010bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcb4d6e7d64249f1 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:37.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.722+0000 7f2a44c00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:29:37.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.722+0000 7f2a44c00700 1 -- 172.21.15.1:0/1126943742 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a480a13b0 con 0x559fbead1010 2024-02-02T15:29:37.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.722+0000 7f2a44c00700 1 -- 172.21.15.1:0/1126943742 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f2a480995a0 con 0x559fbead1010 2024-02-02T15:29:37.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.722+0000 7f2a71cefb80 1 -- 172.21.15.1:0/1126943742 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559fbeac50f0 con 0x559fbead1010 2024-02-02T15:29:37.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.723+0000 7f2a44c00700 1 -- 172.21.15.1:0/1126943742 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a48099760 con 0x559fbead1010 2024-02-02T15:29:37.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.723+0000 7f2a71cefb80 10 client.26360 fetch_fsmap learned FSMap version 31 2024-02-02T15:29:37.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.723+0000 7f2a71cefb80 1 -- 172.21.15.1:0/1126943742 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x559fbe850b60 con 0x559fbead1010 2024-02-02T15:29:37.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.723+0000 7f2a44c00700 1 -- 172.21.15.1:0/1126943742 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559fbeac50f0 con 0x559fbead1010 2024-02-02T15:29:37.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.723+0000 7f2a44c00700 1 -- 172.21.15.1:0/1126943742 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 31) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f2a4800cb20 con 0x559fbead1010 2024-02-02T15:29:37.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.723+0000 7f2a71cefb80 10 client.26360 fetch_fsmap finished waiting for FSMap version 31 2024-02-02T15:29:37.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.723+0000 7f2a71cefb80 10 client.26360 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-02T15:29:37.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.723+0000 7f2a71cefb80 20 client.26360 populate_metadata read hostname 'smithi001' 2024-02-02T15:29:37.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.723+0000 7f2a71cefb80 1 --2- 172.21.15.1:0/1126943742 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x559fbeaebfe0 0x559fbeaee400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:37.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.723+0000 7f2a71cefb80 4 client.26360 mds_command: new command op to 24529 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:29:37.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.723+0000 7f2a71cefb80 1 -- 172.21.15.1:0/1126943742 --> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559fbea1bdd0 con 0x559fbeaebfe0 2024-02-02T15:29:37.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.724+0000 7f2a56a00700 1 --2- 172.21.15.1:0/1126943742 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x559fbeaebfe0 0x559fbeaee400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:37.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.725+0000 7f2a56a00700 1 --2- 172.21.15.1:0/1126943742 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x559fbeaebfe0 0x559fbeaee400 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dd751547edbf7276 server_cookie=1dc6631fa0126772 in_seq=0 out_seq=0 2024-02-02T15:29:37.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.725+0000 7f2a44c00700 10 client.26360 ms_handle_connect on v2:172.21.15.142:6838/321675421 2024-02-02T15:29:37.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.725+0000 7f2a44c00700 1 -- 172.21.15.1:0/1126943742 <== mds.1 v2:172.21.15.142:6838/321675421 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x559fbea1bdd0 con 0x559fbeaebfe0 2024-02-02T15:29:37.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.725+0000 7f2a44c00700 10 client.26360 handle_command_reply: tid=0 2024-02-02T15:29:37.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.726+0000 7f2a71cefb80 1 client.26360 shutdown 2024-02-02T15:29:37.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.726+0000 7f2a71cefb80 1 -- 172.21.15.1:0/1126943742 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559fbead1010 msgr2=0x559fbeadc200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:37.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.726+0000 7f2a71cefb80 1 --2- 172.21.15.1:0/1126943742 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559fbead1010 0x559fbeadc200 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2a480992d0 tx=0x7f2a48010bd0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.726+0000 7f2a71cefb80 1 -- 172.21.15.1:0/1126943742 shutdown_connections 2024-02-02T15:29:37.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.726+0000 7f2a71cefb80 1 --2- 172.21.15.1:0/1126943742 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x559fbeaebfe0 0x559fbeaee400 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.726+0000 7f2a71cefb80 1 --2- 172.21.15.1:0/1126943742 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559fbeadc7e0 0x559fbead9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.726+0000 7f2a71cefb80 1 --2- 172.21.15.1:0/1126943742 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559fbead1010 0x559fbeadc200 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.726+0000 7f2a71cefb80 1 --2- 172.21.15.1:0/1126943742 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559fbead1930 0x559fbead9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.726+0000 7f2a44c00700 0 client.26360 ms_handle_reset on v2:172.21.15.142:6838/321675421 2024-02-02T15:29:37.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.726+0000 7f2a71cefb80 1 -- 172.21.15.1:0/1126943742 >> 172.21.15.1:0/1126943742 conn(0x559fbea30db0 msgr2=0x559fbeaccfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:37.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.726+0000 7f2a71cefb80 1 -- 172.21.15.1:0/1126943742 shutdown_connections 2024-02-02T15:29:37.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.726+0000 7f2a71cefb80 1 -- 172.21.15.1:0/1126943742 wait complete. 2024-02-02T15:29:37.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.726+0000 7f2a71cefb80 20 client.26360 trim_cache size 0 max 16384 2024-02-02T15:29:37.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.739+0000 7f2a71cefb80 1 Processor -- start 2024-02-02T15:29:37.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.739+0000 7f2a71cefb80 1 -- start start 2024-02-02T15:29:37.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.740+0000 7f2a71cefb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559fbead1010 0x559fbead3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:37.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.740+0000 7f2a71cefb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559fbead1930 0x559fbeadbd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:37.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.740+0000 7f2a71cefb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559fbeadc7e0 0x559fbead66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:37.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.740+0000 7f2a71cefb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fbe8a66a0 con 0x559fbead1930 2024-02-02T15:29:37.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.740+0000 7f2a71cefb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fbeada090 con 0x559fbead1010 2024-02-02T15:29:37.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.740+0000 7f2a71cefb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fbead2830 con 0x559fbeadc7e0 2024-02-02T15:29:37.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.740+0000 7f2a56a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559fbead1930 0x559fbeadbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:37.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.740+0000 7f2a57400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559fbead1010 0x559fbead3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:37.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.740+0000 7f2a57400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559fbead1010 0x559fbead3a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:48904/0 (socket says 172.21.15.1:48904) 2024-02-02T15:29:37.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.740+0000 7f2a57400700 1 -- 172.21.15.1:0/3727933835 learned_addr learned my addr 172.21.15.1:0/3727933835 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:29:37.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.740+0000 7f2a57e00700 1 --2- 172.21.15.1:0/3727933835 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559fbeadc7e0 0x559fbead66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:37.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.740+0000 7f2a57400700 1 -- 172.21.15.1:0/3727933835 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559fbeadc7e0 msgr2=0x559fbead66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:37.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.740+0000 7f2a57400700 1 --2- 172.21.15.1:0/3727933835 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559fbeadc7e0 0x559fbead66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.740+0000 7f2a57400700 1 -- 172.21.15.1:0/3727933835 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559fbead1930 msgr2=0x559fbeadbd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:37.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.740+0000 7f2a57400700 1 --2- 172.21.15.1:0/3727933835 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559fbead1930 0x559fbeadbd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.740+0000 7f2a57400700 1 -- 172.21.15.1:0/3727933835 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559fbe9e8ea0 con 0x559fbead1010 2024-02-02T15:29:37.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.741+0000 7f2a57400700 1 --2- 172.21.15.1:0/3727933835 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559fbead1010 0x559fbead3a60 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f2a48097690 tx=0x7f2a48010c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d22f1bdc5ce34b53 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:37.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.741+0000 7f2a4d200700 1 -- 172.21.15.1:0/3727933835 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a480b0070 con 0x559fbead1010 2024-02-02T15:29:37.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.741+0000 7f2a4d200700 1 -- 172.21.15.1:0/3727933835 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a48098ca0 con 0x559fbead1010 2024-02-02T15:29:37.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.741+0000 7f2a4d200700 1 -- 172.21.15.1:0/3727933835 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a4800cc40 con 0x559fbead1010 2024-02-02T15:29:37.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.741+0000 7f2a71cefb80 1 -- 172.21.15.1:0/3727933835 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559fbead1010 msgr2=0x559fbead3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:37.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.741+0000 7f2a71cefb80 1 --2- 172.21.15.1:0/3727933835 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559fbead1010 0x559fbead3a60 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f2a48097690 tx=0x7f2a48010c60 comp rx=0 tx=0).stop 2024-02-02T15:29:37.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.742+0000 7f2a71cefb80 1 -- 172.21.15.1:0/3727933835 shutdown_connections 2024-02-02T15:29:37.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.742+0000 7f2a71cefb80 1 --2- 172.21.15.1:0/3727933835 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559fbeadc7e0 0x559fbead66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.742+0000 7f2a71cefb80 1 --2- 172.21.15.1:0/3727933835 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559fbead1010 0x559fbead3a60 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.742+0000 7f2a71cefb80 1 --2- 172.21.15.1:0/3727933835 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559fbead1930 0x559fbeadbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.742+0000 7f2a71cefb80 1 -- 172.21.15.1:0/3727933835 >> 172.21.15.1:0/3727933835 conn(0x559fbea30db0 msgr2=0x559fbeae78a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:37.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.742+0000 7f2a71cefb80 1 -- 172.21.15.1:0/3727933835 shutdown_connections 2024-02-02T15:29:37.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.742+0000 7f2a71cefb80 1 -- 172.21.15.1:0/3727933835 wait complete. 2024-02-02T15:29:37.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.742+0000 7f2a71cefb80 1 -- start start 2024-02-02T15:29:37.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.742+0000 7f2a71cefb80 1 Processor -- start 2024-02-02T15:29:37.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.743+0000 7f2a71cefb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559fbead1010 0x559fbeae14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:37.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.743+0000 7f2a71cefb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559fbead1930 0x559fbeae19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:37.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.743+0000 7f2a71cefb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559fbeadc7e0 0x559fbeb36a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:37.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.743+0000 7f2a71cefb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fbeb36fd0 con 0x559fbead1930 2024-02-02T15:29:37.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.743+0000 7f2a71cefb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fbeb37140 con 0x559fbeadc7e0 2024-02-02T15:29:37.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.743+0000 7f2a71cefb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fbeb37300 con 0x559fbead1010 2024-02-02T15:29:37.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.743+0000 7f2a71cefb80 10 client.0 _ll_register_callbacks cb 0x559fbe9c0290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:29:37.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.743+0000 7f2a56a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559fbead1930 0x559fbeae19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:37.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.743+0000 7f2a56a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559fbead1930 0x559fbeae19e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35060/0 (socket says 172.21.15.1:35060) 2024-02-02T15:29:37.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.743+0000 7f2a56a00700 1 -- 172.21.15.1:0/3161306945 learned_addr learned my addr 172.21.15.1:0/3161306945 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:29:37.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.744+0000 7f2a57400700 1 --2- 172.21.15.1:0/3161306945 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559fbead1010 0x559fbeae14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:37.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.744+0000 7f2a57e00700 1 --2- 172.21.15.1:0/3161306945 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559fbeadc7e0 0x559fbeb36a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:37.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.744+0000 7f2a56a00700 1 -- 172.21.15.1:0/3161306945 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559fbead1010 msgr2=0x559fbeae14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:37.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.744+0000 7f2a56a00700 1 --2- 172.21.15.1:0/3161306945 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559fbead1010 0x559fbeae14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.744+0000 7f2a56a00700 1 -- 172.21.15.1:0/3161306945 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559fbeadc7e0 msgr2=0x559fbeb36a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:37.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.744+0000 7f2a56a00700 1 --2- 172.21.15.1:0/3161306945 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559fbeadc7e0 0x559fbeb36a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.744+0000 7f2a56a00700 1 -- 172.21.15.1:0/3161306945 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559fbe850b60 con 0x559fbead1930 2024-02-02T15:29:37.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.744+0000 7f2a56a00700 1 --2- 172.21.15.1:0/3161306945 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559fbead1930 0x559fbeae19e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f2a40006010 tx=0x7f2a40009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e860ce60b5b2619 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:37.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.744+0000 7f2a44c00700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:29:37.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.744+0000 7f2a44c00700 1 -- 172.21.15.1:0/3161306945 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a4003b070 con 0x559fbead1930 2024-02-02T15:29:37.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.744+0000 7f2a71cefb80 1 -- 172.21.15.1:0/3161306945 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2a4809d170 con 0x559fbead1930 2024-02-02T15:29:37.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.744+0000 7f2a44c00700 1 -- 172.21.15.1:0/3161306945 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f2a4002c040 con 0x559fbead1930 2024-02-02T15:29:37.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.745+0000 7f2a44c00700 1 -- 172.21.15.1:0/3161306945 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a4000f700 con 0x559fbead1930 2024-02-02T15:29:37.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.745+0000 7f2a71cefb80 10 client.16311 fetch_fsmap learned FSMap version 31 2024-02-02T15:29:37.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.745+0000 7f2a71cefb80 1 -- 172.21.15.1:0/3161306945 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x559fbe8a8cd0 con 0x559fbead1930 2024-02-02T15:29:37.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.745+0000 7f2a44c00700 1 -- 172.21.15.1:0/3161306945 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a4809d170 con 0x559fbead1930 2024-02-02T15:29:37.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.745+0000 7f2a44c00700 1 -- 172.21.15.1:0/3161306945 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 31) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f2a40033820 con 0x559fbead1930 2024-02-02T15:29:37.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.745+0000 7f2a71cefb80 10 client.16311 fetch_fsmap finished waiting for FSMap version 31 2024-02-02T15:29:37.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.745+0000 7f2a71cefb80 10 client.16311 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-02T15:29:37.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.745+0000 7f2a71cefb80 20 client.16311 populate_metadata read hostname 'smithi001' 2024-02-02T15:29:37.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.745+0000 7f2a71cefb80 1 --2- 172.21.15.1:0/3161306945 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x559fbeb3ea40 0x559fbeb40e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:37.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.745+0000 7f2a71cefb80 4 client.16311 mds_command: new command op to 24529 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:29:37.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.745+0000 7f2a71cefb80 1 -- 172.21.15.1:0/3161306945 --> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559fbe9e1f50 con 0x559fbeb3ea40 2024-02-02T15:29:37.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.745+0000 7f2a57400700 1 --2- 172.21.15.1:0/3161306945 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x559fbeb3ea40 0x559fbeb40e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:37.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.746+0000 7f2a57400700 1 --2- 172.21.15.1:0/3161306945 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x559fbeb3ea40 0x559fbeb40e60 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=51da5eb8e7996709 server_cookie=aa7ee0b6b5ad1454 in_seq=0 out_seq=0 2024-02-02T15:29:37.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.746+0000 7f2a44c00700 10 client.16311 ms_handle_connect on v2:172.21.15.142:6838/321675421 2024-02-02T15:29:37.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.746+0000 7f2a44c00700 1 -- 172.21.15.1:0/3161306945 <== mds.1 v2:172.21.15.142:6838/321675421 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559fbe9e1f50 con 0x559fbeb3ea40 2024-02-02T15:29:37.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.746+0000 7f2a44c00700 10 client.16311 handle_command_reply: tid=0 2024-02-02T15:29:37.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.746+0000 7f2a71cefb80 1 client.16311 shutdown 2024-02-02T15:29:37.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.746+0000 7f2a71cefb80 1 -- 172.21.15.1:0/3161306945 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559fbead1930 msgr2=0x559fbeae19e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:37.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.746+0000 7f2a71cefb80 1 --2- 172.21.15.1:0/3161306945 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559fbead1930 0x559fbeae19e0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f2a40006010 tx=0x7f2a40009ca0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.746+0000 7f2a71cefb80 1 -- 172.21.15.1:0/3161306945 shutdown_connections 2024-02-02T15:29:37.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.746+0000 7f2a71cefb80 1 --2- 172.21.15.1:0/3161306945 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x559fbeb3ea40 0x559fbeb40e60 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.746+0000 7f2a71cefb80 1 --2- 172.21.15.1:0/3161306945 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559fbeadc7e0 0x559fbeb36a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.746+0000 7f2a44c00700 0 client.16311 ms_handle_reset on v2:172.21.15.142:6838/321675421 2024-02-02T15:29:37.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.746+0000 7f2a71cefb80 1 --2- 172.21.15.1:0/3161306945 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559fbead1010 0x559fbeae14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.746+0000 7f2a71cefb80 1 --2- 172.21.15.1:0/3161306945 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559fbead1930 0x559fbeae19e0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.746+0000 7f2a71cefb80 1 -- 172.21.15.1:0/3161306945 >> 172.21.15.1:0/3161306945 conn(0x559fbea30db0 msgr2=0x559fbeacc5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:37.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.747+0000 7f2a71cefb80 1 -- 172.21.15.1:0/3161306945 shutdown_connections 2024-02-02T15:29:37.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.747+0000 7f2a71cefb80 1 -- 172.21.15.1:0/3161306945 wait complete. 2024-02-02T15:29:37.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.747+0000 7f2a71cefb80 20 client.16311 trim_cache size 0 max 16384 2024-02-02T15:29:37.751 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:29:37.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.747+0000 7f2a5e200700 1 -- 172.21.15.1:0/3727214198 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f2a380a0710 msgr2=0x7f2a380a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:37.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.747+0000 7f2a5e200700 1 --2- 172.21.15.1:0/3727214198 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f2a380a0710 0x7f2a380a2bd0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2a5819a6b0 tx=0x7f2a40005c60 comp rx=0 tx=0).stop 2024-02-02T15:29:37.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.747+0000 7f2a5e200700 1 -- 172.21.15.1:0/3727214198 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a58102030 msgr2=0x7f2a581990a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:37.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.747+0000 7f2a5e200700 1 --2- 172.21.15.1:0/3727214198 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a58102030 0x7f2a581990a0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f2a4800e9f0 tx=0x7f2a4800ed00 comp rx=0 tx=0).stop 2024-02-02T15:29:37.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.747+0000 7f2a5e200700 1 -- 172.21.15.1:0/3727214198 shutdown_connections 2024-02-02T15:29:37.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.747+0000 7f2a5e200700 1 --2- 172.21.15.1:0/3727214198 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f2a380a0710 0x7f2a380a2bd0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.747+0000 7f2a5e200700 1 --2- 172.21.15.1:0/3727214198 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a5810b400 0x7f2a5819da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.747+0000 7f2a5e200700 1 --2- 172.21.15.1:0/3727214198 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a58102030 0x7f2a581990a0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.747+0000 7f2a5e200700 1 --2- 172.21.15.1:0/3727214198 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a58102950 0x7f2a581995e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.747+0000 7f2a5e200700 1 -- 172.21.15.1:0/3727214198 >> 172.21.15.1:0/3727214198 conn(0x7f2a58076d70 msgr2=0x7f2a580fdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:37.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.748+0000 7f2a5e200700 1 -- 172.21.15.1:0/3727214198 shutdown_connections 2024-02-02T15:29:37.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.748+0000 7f2a5e200700 1 -- 172.21.15.1:0/3727214198 wait complete. 2024-02-02T15:29:37.765 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-02T15:29:37.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.965+0000 7fe14e200700 1 -- 172.21.15.1:0/2677143005 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe148100ff0 msgr2=0x7fe1481013d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:37.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.965+0000 7fe14e200700 1 --2- 172.21.15.1:0/2677143005 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe148100ff0 0x7fe1481013d0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fe138009990 tx=0x7fe138009ca0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.965+0000 7fe14e200700 1 -- 172.21.15.1:0/2677143005 shutdown_connections 2024-02-02T15:29:37.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.965+0000 7fe14e200700 1 --2- 172.21.15.1:0/2677143005 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe148106110 0x7fe14810dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.965+0000 7fe14e200700 1 --2- 172.21.15.1:0/2677143005 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe148100ff0 0x7fe1481013d0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.965+0000 7fe14e200700 1 --2- 172.21.15.1:0/2677143005 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe1481019a0 0x7fe1481059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.965+0000 7fe14e200700 1 -- 172.21.15.1:0/2677143005 >> 172.21.15.1:0/2677143005 conn(0x7fe1480fc810 msgr2=0x7fe1480fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:37.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.966+0000 7fe14e200700 1 -- 172.21.15.1:0/2677143005 shutdown_connections 2024-02-02T15:29:37.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.966+0000 7fe14e200700 1 -- 172.21.15.1:0/2677143005 wait complete. 2024-02-02T15:29:37.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.966+0000 7fe14e200700 1 Processor -- start 2024-02-02T15:29:37.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.967+0000 7fe14e200700 1 -- start start 2024-02-02T15:29:37.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.967+0000 7fe14e200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe148100ff0 0x7fe148073040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:37.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.967+0000 7fe14e200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe148106110 0x7fe148073580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:37.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.967+0000 7fe14e200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe148073ac0 0x7fe1481a5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:37.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.967+0000 7fe14e200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1481a56f0 con 0x7fe148073ac0 2024-02-02T15:29:37.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.967+0000 7fe14e200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1481a5860 con 0x7fe148100ff0 2024-02-02T15:29:37.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.967+0000 7fe14e200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1481a5a20 con 0x7fe148106110 2024-02-02T15:29:37.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.967+0000 7fe147e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe148073ac0 0x7fe1481a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:37.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.967+0000 7fe147e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe148073ac0 0x7fe1481a5060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35074/0 (socket says 172.21.15.1:35074) 2024-02-02T15:29:37.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.967+0000 7fe147e00700 1 -- 172.21.15.1:0/714814464 learned_addr learned my addr 172.21.15.1:0/714814464 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:29:37.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.967+0000 7fe147400700 1 --2- 172.21.15.1:0/714814464 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe148100ff0 0x7fe148073040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:37.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.968+0000 7fe146a00700 1 --2- 172.21.15.1:0/714814464 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe148106110 0x7fe148073580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:37.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.968+0000 7fe147e00700 1 -- 172.21.15.1:0/714814464 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe148106110 msgr2=0x7fe148073580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:37.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.968+0000 7fe147e00700 1 --2- 172.21.15.1:0/714814464 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe148106110 0x7fe148073580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.968+0000 7fe147e00700 1 -- 172.21.15.1:0/714814464 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe148100ff0 msgr2=0x7fe148073040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:37.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.968+0000 7fe147e00700 1 --2- 172.21.15.1:0/714814464 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe148100ff0 0x7fe148073040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.968+0000 7fe147e00700 1 -- 172.21.15.1:0/714814464 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe130009520 con 0x7fe148073ac0 2024-02-02T15:29:37.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.968+0000 7fe147e00700 1 --2- 172.21.15.1:0/714814464 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe148073ac0 0x7fe1481a5060 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fe14000ef30 tx=0x7fe14000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:37.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.968+0000 7fe13fe00700 1 -- 172.21.15.1:0/714814464 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe140018070 con 0x7fe148073ac0 2024-02-02T15:29:37.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.968+0000 7fe13fe00700 1 -- 172.21.15.1:0/714814464 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe14000f040 con 0x7fe148073ac0 2024-02-02T15:29:37.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.968+0000 7fe14e200700 1 -- 172.21.15.1:0/714814464 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1380095f0 con 0x7fe148073ac0 2024-02-02T15:29:37.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.968+0000 7fe13fe00700 1 -- 172.21.15.1:0/714814464 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe14000cd00 con 0x7fe148073ac0 2024-02-02T15:29:37.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.968+0000 7fe14e200700 1 -- 172.21.15.1:0/714814464 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1481a6070 con 0x7fe148073ac0 2024-02-02T15:29:37.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.971+0000 7fe13fe00700 1 -- 172.21.15.1:0/714814464 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 27) v1 ==== 135200+0+0 (secure 0 0 0) 0x7fe140003a00 con 0x7fe148073ac0 2024-02-02T15:29:37.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.973+0000 7fe161e18b80 1 Processor -- start 2024-02-02T15:29:37.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.974+0000 7fe13fe00700 1 --2- 172.21.15.1:0/714814464 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fe1280a0710 0x7fe1280a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:37.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.974+0000 7fe161e18b80 1 -- start start 2024-02-02T15:29:37.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.974+0000 7fe13fe00700 1 -- 172.21.15.1:0/714814464 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fe1400cbb10 con 0x7fe148073ac0 2024-02-02T15:29:37.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.974+0000 7fe146a00700 1 --2- 172.21.15.1:0/714814464 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fe1280a0710 0x7fe1280a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:37.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.974+0000 7fe161e18b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5601e8b07010 0x5601e8b073f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:37.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.974+0000 7fe161e18b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5601e8b07930 0x5601e8b122a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:37.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.974+0000 7fe161e18b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5601e8b127e0 0x5601e8b14bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:37.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.974+0000 7fe161e18b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601e8a7b4c0 con 0x5601e8b127e0 2024-02-02T15:29:37.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.974+0000 7fe161e18b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601e8a5faa0 con 0x5601e8b07930 2024-02-02T15:29:37.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.974+0000 7fe161e18b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601e8a5d460 con 0x5601e8b07010 2024-02-02T15:29:37.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.975+0000 7fe147e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5601e8b127e0 0x5601e8b14bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:37.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.975+0000 7fe147400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5601e8b07010 0x5601e8b073f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:37.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.975+0000 7fe147e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5601e8b127e0 0x5601e8b14bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35084/0 (socket says 172.21.15.1:35084) 2024-02-02T15:29:37.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.975+0000 7fe147400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5601e8b07010 0x5601e8b073f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:40656/0 (socket says 172.21.15.1:40656) 2024-02-02T15:29:37.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.975+0000 7fe147e00700 1 -- 172.21.15.1:0/2067225110 learned_addr learned my addr 172.21.15.1:0/2067225110 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:29:37.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.975+0000 7fe146a00700 1 --2- 172.21.15.1:0/2067225110 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5601e8b07930 0x5601e8b122a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:37.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.975+0000 7fe146a00700 1 --2- 172.21.15.1:0/714814464 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fe1280a0710 0x7fe1280a2bd0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe148074830 tx=0x7fe130011720 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:37.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.975+0000 7fe146a00700 1 -- 172.21.15.1:0/2067225110 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5601e8b07010 msgr2=0x5601e8b073f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:37.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.975+0000 7fe146a00700 1 --2- 172.21.15.1:0/2067225110 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5601e8b07010 0x5601e8b073f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.975+0000 7fe146a00700 1 -- 172.21.15.1:0/2067225110 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5601e8b127e0 msgr2=0x5601e8b14bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:37.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.975+0000 7fe146a00700 1 --2- 172.21.15.1:0/2067225110 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5601e8b127e0 0x5601e8b14bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.975+0000 7fe146a00700 1 -- 172.21.15.1:0/2067225110 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5601e88da3b0 con 0x5601e8b07930 2024-02-02T15:29:37.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.975+0000 7fe147400700 1 --2- 172.21.15.1:0/2067225110 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5601e8b07010 0x5601e8b073f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:29:37.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.975+0000 7fe146a00700 1 --2- 172.21.15.1:0/2067225110 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5601e8b07930 0x5601e8b122a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe130009fd0 tx=0x7fe13001c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d893147dde03d510 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:37.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.976+0000 7fe137e00700 1 -- 172.21.15.1:0/2067225110 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13001ddf0 con 0x5601e8b07930 2024-02-02T15:29:37.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.976+0000 7fe137e00700 1 -- 172.21.15.1:0/2067225110 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe130026900 con 0x5601e8b07930 2024-02-02T15:29:37.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.976+0000 7fe137e00700 1 -- 172.21.15.1:0/2067225110 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13000f900 con 0x5601e8b07930 2024-02-02T15:29:37.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.976+0000 7fe161e18b80 1 -- 172.21.15.1:0/2067225110 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5601e8b07930 msgr2=0x5601e8b122a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:37.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.976+0000 7fe161e18b80 1 --2- 172.21.15.1:0/2067225110 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5601e8b07930 0x5601e8b122a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe130009fd0 tx=0x7fe13001c000 comp rx=0 tx=0).stop 2024-02-02T15:29:37.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.976+0000 7fe161e18b80 1 -- 172.21.15.1:0/2067225110 shutdown_connections 2024-02-02T15:29:37.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.976+0000 7fe161e18b80 1 --2- 172.21.15.1:0/2067225110 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5601e8b127e0 0x5601e8b14bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.976+0000 7fe161e18b80 1 --2- 172.21.15.1:0/2067225110 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5601e8b07010 0x5601e8b073f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.976+0000 7fe161e18b80 1 --2- 172.21.15.1:0/2067225110 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5601e8b07930 0x5601e8b122a0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.976+0000 7fe161e18b80 1 -- 172.21.15.1:0/2067225110 >> 172.21.15.1:0/2067225110 conn(0x5601e8a66db0 msgr2=0x5601e8b033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:37.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.976+0000 7fe161e18b80 1 -- 172.21.15.1:0/2067225110 shutdown_connections 2024-02-02T15:29:37.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.977+0000 7fe161e18b80 1 -- 172.21.15.1:0/2067225110 wait complete. 2024-02-02T15:29:37.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.977+0000 7fe161e18b80 1 -- start start 2024-02-02T15:29:37.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.977+0000 7fe161e18b80 1 Processor -- start 2024-02-02T15:29:37.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.978+0000 7fe161e18b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5601e8b07010 0x5601e8b0f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:37.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.978+0000 7fe161e18b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5601e8b07930 0x5601e8b0fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:37.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.978+0000 7fe161e18b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5601e8b127e0 0x5601e8b0ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:37.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.978+0000 7fe161e18b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601e8a58d50 con 0x5601e8b127e0 2024-02-02T15:29:37.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.978+0000 7fe161e18b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601e89f4d80 con 0x5601e8b07930 2024-02-02T15:29:37.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.978+0000 7fe161e18b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601e89f4a20 con 0x5601e8b07010 2024-02-02T15:29:37.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.978+0000 7fe161e18b80 10 client.0 _ll_register_callbacks cb 0x5601e89f6290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:29:37.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.978+0000 7fe147e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5601e8b127e0 0x5601e8b0ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:37.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.978+0000 7fe147e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5601e8b127e0 0x5601e8b0ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35098/0 (socket says 172.21.15.1:35098) 2024-02-02T15:29:37.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.978+0000 7fe147e00700 1 -- 172.21.15.1:0/4202911496 learned_addr learned my addr 172.21.15.1:0/4202911496 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:29:37.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.978+0000 7fe147400700 1 --2- 172.21.15.1:0/4202911496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5601e8b07010 0x5601e8b0f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:37.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.978+0000 7fe146a00700 1 --2- 172.21.15.1:0/4202911496 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5601e8b07930 0x5601e8b0fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:37.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.978+0000 7fe147e00700 1 -- 172.21.15.1:0/4202911496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5601e8b07010 msgr2=0x5601e8b0f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:37.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.978+0000 7fe147e00700 1 --2- 172.21.15.1:0/4202911496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5601e8b07010 0x5601e8b0f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.978+0000 7fe147e00700 1 -- 172.21.15.1:0/4202911496 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5601e8b07930 msgr2=0x5601e8b0fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:37.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.978+0000 7fe147e00700 1 --2- 172.21.15.1:0/4202911496 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5601e8b07930 0x5601e8b0fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.978+0000 7fe147e00700 1 -- 172.21.15.1:0/4202911496 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe14810e500 con 0x5601e8b127e0 2024-02-02T15:29:37.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.979+0000 7fe147e00700 1 --2- 172.21.15.1:0/4202911496 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5601e8b127e0 0x5601e8b0ff80 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fe140098ca0 tx=0x7fe1400986f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd011a0652a63c27 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:37.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.979+0000 7fe134c00700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:29:37.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.979+0000 7fe134c00700 1 -- 172.21.15.1:0/4202911496 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe14001b320 con 0x5601e8b127e0 2024-02-02T15:29:37.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.979+0000 7fe134c00700 1 -- 172.21.15.1:0/4202911496 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fe140099410 con 0x5601e8b127e0 2024-02-02T15:29:37.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.979+0000 7fe161e18b80 1 -- 172.21.15.1:0/4202911496 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5601e8afb0f0 con 0x5601e8b127e0 2024-02-02T15:29:37.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.979+0000 7fe134c00700 1 -- 172.21.15.1:0/4202911496 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe1400b1200 con 0x5601e8b127e0 2024-02-02T15:29:37.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.979+0000 7fe161e18b80 10 client.16323 fetch_fsmap learned FSMap version 31 2024-02-02T15:29:37.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.979+0000 7fe161e18b80 1 -- 172.21.15.1:0/4202911496 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7fe130011cf0 con 0x5601e8b127e0 2024-02-02T15:29:37.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.979+0000 7fe134c00700 1 -- 172.21.15.1:0/4202911496 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5601e8afb0f0 con 0x5601e8b127e0 2024-02-02T15:29:37.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.980+0000 7fe134c00700 1 -- 172.21.15.1:0/4202911496 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 31) v1 ==== 7502+0+0 (secure 0 0 0) 0x7fe140099d20 con 0x5601e8b127e0 2024-02-02T15:29:37.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.980+0000 7fe161e18b80 10 client.16323 fetch_fsmap finished waiting for FSMap version 31 2024-02-02T15:29:37.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.980+0000 7fe161e18b80 10 client.16323 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:29:37.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.980+0000 7fe161e18b80 20 client.16323 populate_metadata read hostname 'smithi001' 2024-02-02T15:29:37.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.980+0000 7fe161e18b80 1 --2- 172.21.15.1:0/4202911496 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5601e8b22220 0x5601e8b24640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:37.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.980+0000 7fe161e18b80 4 client.16323 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:29:37.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.980+0000 7fe161e18b80 1 -- 172.21.15.1:0/4202911496 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5601e8a51dd0 con 0x5601e8b22220 2024-02-02T15:29:37.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.980+0000 7fe147400700 1 --2- 172.21.15.1:0/4202911496 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5601e8b22220 0x5601e8b24640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:37.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.981+0000 7fe147400700 1 --2- 172.21.15.1:0/4202911496 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5601e8b22220 0x5601e8b24640 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fcbd1d8bd4e7f0f server_cookie=eb5f4955cc15b6c8 in_seq=0 out_seq=0 2024-02-02T15:29:37.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.981+0000 7fe134c00700 10 client.16323 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:29:37.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.981+0000 7fe134c00700 1 -- 172.21.15.1:0/4202911496 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5601e8a51dd0 con 0x5601e8b22220 2024-02-02T15:29:37.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.981+0000 7fe134c00700 10 client.16323 handle_command_reply: tid=0 2024-02-02T15:29:37.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.981+0000 7fe161e18b80 1 client.16323 shutdown 2024-02-02T15:29:37.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.981+0000 7fe161e18b80 1 -- 172.21.15.1:0/4202911496 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5601e8b127e0 msgr2=0x5601e8b0ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:37.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.981+0000 7fe161e18b80 1 --2- 172.21.15.1:0/4202911496 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5601e8b127e0 0x5601e8b0ff80 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fe140098ca0 tx=0x7fe1400986f0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.982+0000 7fe161e18b80 1 -- 172.21.15.1:0/4202911496 shutdown_connections 2024-02-02T15:29:37.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.982+0000 7fe161e18b80 1 --2- 172.21.15.1:0/4202911496 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5601e8b22220 0x5601e8b24640 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.982+0000 7fe161e18b80 1 --2- 172.21.15.1:0/4202911496 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5601e8b127e0 0x5601e8b0ff80 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.982+0000 7fe161e18b80 1 --2- 172.21.15.1:0/4202911496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5601e8b07010 0x5601e8b0f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.982+0000 7fe161e18b80 1 --2- 172.21.15.1:0/4202911496 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5601e8b07930 0x5601e8b0fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.982+0000 7fe134c00700 0 client.16323 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:29:37.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.982+0000 7fe161e18b80 1 -- 172.21.15.1:0/4202911496 >> 172.21.15.1:0/4202911496 conn(0x5601e8a66db0 msgr2=0x5601e8b02fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:37.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.982+0000 7fe161e18b80 1 -- 172.21.15.1:0/4202911496 shutdown_connections 2024-02-02T15:29:37.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.982+0000 7fe161e18b80 1 -- 172.21.15.1:0/4202911496 wait complete. 2024-02-02T15:29:37.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.982+0000 7fe161e18b80 20 client.16323 trim_cache size 0 max 16384 2024-02-02T15:29:37.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.995+0000 7fe161e18b80 1 Processor -- start 2024-02-02T15:29:37.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.995+0000 7fe161e18b80 1 -- start start 2024-02-02T15:29:37.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.996+0000 7fe161e18b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5601e8b07010 0x5601e8b19320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:37.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.996+0000 7fe161e18b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5601e8b07930 0x5601e8b09f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:37.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.996+0000 7fe161e18b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5601e8b127e0 0x5601e8b0fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:37.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.996+0000 7fe161e18b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601e88dc6a0 con 0x5601e8b127e0 2024-02-02T15:29:37.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.996+0000 7fe161e18b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601e8b0b0d0 con 0x5601e8b07930 2024-02-02T15:29:37.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.996+0000 7fe161e18b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601e8b0ea70 con 0x5601e8b07010 2024-02-02T15:29:37.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.996+0000 7fe147400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5601e8b07010 0x5601e8b19320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:37.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.996+0000 7fe147400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5601e8b07010 0x5601e8b19320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:40682/0 (socket says 172.21.15.1:40682) 2024-02-02T15:29:37.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.996+0000 7fe147400700 1 -- 172.21.15.1:0/3775725669 learned_addr learned my addr 172.21.15.1:0/3775725669 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:29:37.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.996+0000 7fe146a00700 1 --2- 172.21.15.1:0/3775725669 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5601e8b07930 0x5601e8b09f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:37.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.996+0000 7fe147e00700 1 --2- 172.21.15.1:0/3775725669 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5601e8b127e0 0x5601e8b0fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:37.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.996+0000 7fe147400700 1 -- 172.21.15.1:0/3775725669 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5601e8b07930 msgr2=0x5601e8b09f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:37.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.996+0000 7fe147400700 1 --2- 172.21.15.1:0/3775725669 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5601e8b07930 0x5601e8b09f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.996+0000 7fe147400700 1 -- 172.21.15.1:0/3775725669 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5601e8b127e0 msgr2=0x5601e8b0fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:37.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.996+0000 7fe147400700 1 --2- 172.21.15.1:0/3775725669 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5601e8b127e0 0x5601e8b0fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:37.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.996+0000 7fe147400700 1 -- 172.21.15.1:0/3775725669 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5601e8886b60 con 0x5601e8b07010 2024-02-02T15:29:37.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.997+0000 7fe147400700 1 --2- 172.21.15.1:0/3775725669 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5601e8b07010 0x5601e8b19320 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe138009990 tx=0x7fe138005960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71114c5d8915a7c0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:37.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.997+0000 7fe13d200700 1 -- 172.21.15.1:0/3775725669 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13802a070 con 0x5601e8b07010 2024-02-02T15:29:37.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.997+0000 7fe13d200700 1 -- 172.21.15.1:0/3775725669 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe13801e040 con 0x5601e8b07010 2024-02-02T15:29:37.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.997+0000 7fe13d200700 1 -- 172.21.15.1:0/3775725669 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13801d040 con 0x5601e8b07010 2024-02-02T15:29:37.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.998+0000 7fe161e18b80 1 -- 172.21.15.1:0/3775725669 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5601e8b07010 msgr2=0x5601e8b19320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:37.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.998+0000 7fe161e18b80 1 --2- 172.21.15.1:0/3775725669 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5601e8b07010 0x5601e8b19320 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe138009990 tx=0x7fe138005960 comp rx=0 tx=0).stop 2024-02-02T15:29:38.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.998+0000 7fe161e18b80 1 -- 172.21.15.1:0/3775725669 shutdown_connections 2024-02-02T15:29:38.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.998+0000 7fe161e18b80 1 --2- 172.21.15.1:0/3775725669 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5601e8b127e0 0x5601e8b0fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:38.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.998+0000 7fe161e18b80 1 --2- 172.21.15.1:0/3775725669 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5601e8b07010 0x5601e8b19320 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:38.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.998+0000 7fe161e18b80 1 --2- 172.21.15.1:0/3775725669 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5601e8b07930 0x5601e8b09f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:38.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.998+0000 7fe161e18b80 1 -- 172.21.15.1:0/3775725669 >> 172.21.15.1:0/3775725669 conn(0x5601e8a66db0 msgr2=0x5601e8b1d900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:38.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.998+0000 7fe161e18b80 1 -- 172.21.15.1:0/3775725669 shutdown_connections 2024-02-02T15:29:38.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.998+0000 7fe161e18b80 1 -- 172.21.15.1:0/3775725669 wait complete. 2024-02-02T15:29:38.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.998+0000 7fe161e18b80 1 -- start start 2024-02-02T15:29:38.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.998+0000 7fe161e18b80 1 Processor -- start 2024-02-02T15:29:38.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.999+0000 7fe161e18b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5601e8b07010 0x5601e8b18ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:38.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.999+0000 7fe161e18b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5601e8b07930 0x5601e8b17120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:38.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.999+0000 7fe161e18b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5601e8b127e0 0x5601e8b17660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:38.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.999+0000 7fe161e18b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601e8b19770 con 0x5601e8b07010 2024-02-02T15:29:38.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.999+0000 7fe161e18b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601e8b17ba0 con 0x5601e8b07930 2024-02-02T15:29:38.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.999+0000 7fe161e18b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601e8b17d60 con 0x5601e8b127e0 2024-02-02T15:29:38.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.999+0000 7fe161e18b80 10 client.0 _ll_register_callbacks cb 0x5601e89f6290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:29:38.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.999+0000 7fe147400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5601e8b07010 0x5601e8b18ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:38.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.999+0000 7fe147400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5601e8b07010 0x5601e8b18ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35108/0 (socket says 172.21.15.1:35108) 2024-02-02T15:29:38.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.999+0000 7fe147400700 1 -- 172.21.15.1:0/190587368 learned_addr learned my addr 172.21.15.1:0/190587368 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:29:38.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.999+0000 7fe146a00700 1 --2- 172.21.15.1:0/190587368 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5601e8b07930 0x5601e8b17120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:38.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:37.999+0000 7fe147e00700 1 --2- 172.21.15.1:0/190587368 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5601e8b127e0 0x5601e8b17660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:38.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:38.000+0000 7fe147400700 1 -- 172.21.15.1:0/190587368 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5601e8b127e0 msgr2=0x5601e8b17660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:38.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:38.000+0000 7fe147400700 1 --2- 172.21.15.1:0/190587368 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5601e8b127e0 0x5601e8b17660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:38.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:38.000+0000 7fe147400700 1 -- 172.21.15.1:0/190587368 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5601e8b07930 msgr2=0x5601e8b17120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:38.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:38.000+0000 7fe147400700 1 --2- 172.21.15.1:0/190587368 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5601e8b07930 0x5601e8b17120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:38.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:38.000+0000 7fe147400700 1 -- 172.21.15.1:0/190587368 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1380053e0 con 0x5601e8b07010 2024-02-02T15:29:38.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:38.000+0000 7fe147400700 1 --2- 172.21.15.1:0/190587368 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5601e8b07010 0x5601e8b18ea0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fe13800b2e0 tx=0x7fe13800b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b38f41c13611cbcf server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:38.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:38.000+0000 7fe134c00700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:29:38.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:38.000+0000 7fe134c00700 1 -- 172.21.15.1:0/190587368 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13801d070 con 0x5601e8b07010 2024-02-02T15:29:38.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:38.000+0000 7fe161e18b80 1 -- 172.21.15.1:0/190587368 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe130026900 con 0x5601e8b07010 2024-02-02T15:29:38.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:38.000+0000 7fe134c00700 1 -- 172.21.15.1:0/190587368 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fe13800bc00 con 0x5601e8b07010 2024-02-02T15:29:38.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:38.000+0000 7fe134c00700 1 -- 172.21.15.1:0/190587368 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe13802a040 con 0x5601e8b07010 2024-02-02T15:29:38.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:38.001+0000 7fe161e18b80 10 client.16329 fetch_fsmap learned FSMap version 31 2024-02-02T15:29:38.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:38.001+0000 7fe161e18b80 1 -- 172.21.15.1:0/190587368 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5601e8a1eea0 con 0x5601e8b07010 2024-02-02T15:29:38.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:38.001+0000 7fe134c00700 1 -- 172.21.15.1:0/190587368 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe130026900 con 0x5601e8b07010 2024-02-02T15:29:38.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:38.001+0000 7fe134c00700 1 -- 172.21.15.1:0/190587368 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 31) v1 ==== 7502+0+0 (secure 0 0 0) 0x7fe1380376c0 con 0x5601e8b07010 2024-02-02T15:29:38.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:38.001+0000 7fe161e18b80 10 client.16329 fetch_fsmap finished waiting for FSMap version 31 2024-02-02T15:29:38.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:38.001+0000 7fe161e18b80 10 client.16329 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:29:38.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:38.001+0000 7fe161e18b80 20 client.16329 populate_metadata read hostname 'smithi001' 2024-02-02T15:29:38.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:38.001+0000 7fe161e18b80 1 --2- 172.21.15.1:0/190587368 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5601e8b74c90 0x5601e8b770b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:38.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:38.001+0000 7fe161e18b80 4 client.16329 mds_command: new command op to 24557 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:29:38.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:38.001+0000 7fe161e18b80 1 -- 172.21.15.1:0/190587368 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5601e8a17f50 con 0x5601e8b74c90 2024-02-02T15:29:38.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:38.002+0000 7fe146a00700 1 --2- 172.21.15.1:0/190587368 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5601e8b74c90 0x5601e8b770b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:38.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:38.002+0000 7fe146a00700 1 --2- 172.21.15.1:0/190587368 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5601e8b74c90 0x5601e8b770b0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff24f565aa45f956 server_cookie=af4893a4ce8a5a7 in_seq=0 out_seq=0 2024-02-02T15:29:38.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:38.002+0000 7fe134c00700 10 client.16329 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:29:38.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:38.002+0000 7fe134c00700 1 -- 172.21.15.1:0/190587368 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5601e8a17f50 con 0x5601e8b74c90 2024-02-02T15:29:38.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:38.002+0000 7fe134c00700 10 client.16329 handle_command_reply: tid=0 2024-02-02T15:29:38.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:38.002+0000 7fe161e18b80 1 client.16329 shutdown 2024-02-02T15:29:38.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:38.002+0000 7fe161e18b80 1 -- 172.21.15.1:0/190587368 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5601e8b07010 msgr2=0x5601e8b18ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:38.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:38.002+0000 7fe161e18b80 1 --2- 172.21.15.1:0/190587368 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5601e8b07010 0x5601e8b18ea0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fe13800b2e0 tx=0x7fe13800b630 comp rx=0 tx=0).stop 2024-02-02T15:29:38.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:38.003+0000 7fe161e18b80 1 -- 172.21.15.1:0/190587368 shutdown_connections 2024-02-02T15:29:38.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:38.003+0000 7fe161e18b80 1 --2- 172.21.15.1:0/190587368 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5601e8b74c90 0x5601e8b770b0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:38.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:38.003+0000 7fe161e18b80 1 --2- 172.21.15.1:0/190587368 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5601e8b127e0 0x5601e8b17660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:38.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:38.003+0000 7fe161e18b80 1 --2- 172.21.15.1:0/190587368 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5601e8b07010 0x5601e8b18ea0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:38.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:38.003+0000 7fe134c00700 0 client.16329 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:29:38.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:38.003+0000 7fe161e18b80 1 --2- 172.21.15.1:0/190587368 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5601e8b07930 0x5601e8b17120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:38.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:38.003+0000 7fe161e18b80 1 -- 172.21.15.1:0/190587368 >> 172.21.15.1:0/190587368 conn(0x5601e8a66db0 msgr2=0x5601e8b0da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:38.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:38.003+0000 7fe161e18b80 1 -- 172.21.15.1:0/190587368 shutdown_connections 2024-02-02T15:29:38.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:38.003+0000 7fe161e18b80 1 -- 172.21.15.1:0/190587368 wait complete. 2024-02-02T15:29:38.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:38.003+0000 7fe161e18b80 20 client.16329 trim_cache size 0 max 16384 2024-02-02T15:29:38.007 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:29:38.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:38.003+0000 7fe14e200700 1 -- 172.21.15.1:0/714814464 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fe1280a0710 msgr2=0x7fe1280a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:38.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:38.003+0000 7fe14e200700 1 --2- 172.21.15.1:0/714814464 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fe1280a0710 0x7fe1280a2bd0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe148074830 tx=0x7fe130011720 comp rx=0 tx=0).stop 2024-02-02T15:29:38.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:38.003+0000 7fe14e200700 1 -- 172.21.15.1:0/714814464 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe148073ac0 msgr2=0x7fe1481a5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:38.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:38.003+0000 7fe14e200700 1 --2- 172.21.15.1:0/714814464 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe148073ac0 0x7fe1481a5060 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fe14000ef30 tx=0x7fe14000c3a0 comp rx=0 tx=0).stop 2024-02-02T15:29:38.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:38.003+0000 7fe14e200700 1 -- 172.21.15.1:0/714814464 shutdown_connections 2024-02-02T15:29:38.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:38.003+0000 7fe14e200700 1 --2- 172.21.15.1:0/714814464 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fe1280a0710 0x7fe1280a2bd0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:38.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:38.003+0000 7fe14e200700 1 --2- 172.21.15.1:0/714814464 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe148073ac0 0x7fe1481a5060 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:38.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:38.004+0000 7fe14e200700 1 --2- 172.21.15.1:0/714814464 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe148100ff0 0x7fe148073040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:38.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:38.004+0000 7fe14e200700 1 --2- 172.21.15.1:0/714814464 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe148106110 0x7fe148073580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:38.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:38.004+0000 7fe14e200700 1 -- 172.21.15.1:0/714814464 >> 172.21.15.1:0/714814464 conn(0x7fe1480fc810 msgr2=0x7fe1480fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:38.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:38.004+0000 7fe14e200700 1 -- 172.21.15.1:0/714814464 shutdown_connections 2024-02-02T15:29:38.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:38.004+0000 7fe14e200700 1 -- 172.21.15.1:0/714814464 wait complete. 2024-02-02T15:29:38.222 INFO:tasks.workunit.client.0.smithi001.stdout: 1 103063 7.98 MB/sec warmup 72 sec latency 2054.688 ms 2024-02-02T15:29:39.022 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-02T15:29:39.022 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-02T15:29:39.222 INFO:tasks.workunit.client.0.smithi001.stdout: 1 103183 7.88 MB/sec warmup 73 sec latency 2916.581 ms 2024-02-02T15:29:39.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.239+0000 7fd5af000700 1 -- 172.21.15.1:0/3055421274 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5a8106110 msgr2=0x7fd5a810dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:39.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.239+0000 7fd5af000700 1 --2- 172.21.15.1:0/3055421274 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5a8106110 0x7fd5a810dc90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd5a0009f90 tx=0x7fd5a000b330 comp rx=0 tx=0).stop 2024-02-02T15:29:39.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.239+0000 7fd5af000700 1 -- 172.21.15.1:0/3055421274 shutdown_connections 2024-02-02T15:29:39.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.239+0000 7fd5af000700 1 --2- 172.21.15.1:0/3055421274 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5a8106110 0x7fd5a810dc90 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:39.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.239+0000 7fd5af000700 1 --2- 172.21.15.1:0/3055421274 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd5a8100ff0 0x7fd5a81013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:39.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.239+0000 7fd5af000700 1 --2- 172.21.15.1:0/3055421274 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5a81019a0 0x7fd5a81059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:39.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.239+0000 7fd5af000700 1 -- 172.21.15.1:0/3055421274 >> 172.21.15.1:0/3055421274 conn(0x7fd5a80fc810 msgr2=0x7fd5a80fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:39.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.239+0000 7fd5af000700 1 -- 172.21.15.1:0/3055421274 shutdown_connections 2024-02-02T15:29:39.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.239+0000 7fd5af000700 1 -- 172.21.15.1:0/3055421274 wait complete. 2024-02-02T15:29:39.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.240+0000 7fd5af000700 1 Processor -- start 2024-02-02T15:29:39.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.240+0000 7fd5af000700 1 -- start start 2024-02-02T15:29:39.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.241+0000 7fd5af000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5a8100ff0 0x7fd5a81991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:39.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.241+0000 7fd5af000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd5a8106110 0x7fd5a8199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:39.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.241+0000 7fd5af000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5a819db70 0x7fd5a819dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:39.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.241+0000 7fd5af000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a819e5f0 con 0x7fd5a8106110 2024-02-02T15:29:39.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.241+0000 7fd5af000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a819e760 con 0x7fd5a8100ff0 2024-02-02T15:29:39.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.241+0000 7fd5af000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a819e920 con 0x7fd5a819db70 2024-02-02T15:29:39.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.241+0000 7fd5a7400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd5a8106110 0x7fd5a8199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:39.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.241+0000 7fd5a7e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5a8100ff0 0x7fd5a81991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:39.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.241+0000 7fd5ad000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5a819db70 0x7fd5a819dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:39.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.241+0000 7fd5a7400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd5a8106110 0x7fd5a8199710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35122/0 (socket says 172.21.15.1:35122) 2024-02-02T15:29:39.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.241+0000 7fd5a7400700 1 -- 172.21.15.1:0/3969084662 learned_addr learned my addr 172.21.15.1:0/3969084662 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:29:39.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.242+0000 7fd5a7400700 1 -- 172.21.15.1:0/3969084662 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5a819db70 msgr2=0x7fd5a819dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:39.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.242+0000 7fd5a7400700 1 --2- 172.21.15.1:0/3969084662 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5a819db70 0x7fd5a819dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:39.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.242+0000 7fd5a7400700 1 -- 172.21.15.1:0/3969084662 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5a8100ff0 msgr2=0x7fd5a81991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:39.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.242+0000 7fd5a7400700 1 --2- 172.21.15.1:0/3969084662 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5a8100ff0 0x7fd5a81991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:39.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.242+0000 7fd5a7400700 1 -- 172.21.15.1:0/3969084662 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5980095f0 con 0x7fd5a8106110 2024-02-02T15:29:39.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.242+0000 7fd5ad000700 1 --2- 172.21.15.1:0/3969084662 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5a819db70 0x7fd5a819dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:29:39.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.242+0000 7fd5a7e00700 1 --2- 172.21.15.1:0/3969084662 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5a8100ff0 0x7fd5a81991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:29:39.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.242+0000 7fd5a7400700 1 --2- 172.21.15.1:0/3969084662 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd5a8106110 0x7fd5a8199710 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fd59000e970 tx=0x7fd59000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:39.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.242+0000 7fd5a4c00700 1 -- 172.21.15.1:0/3969084662 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd590009790 con 0x7fd5a8106110 2024-02-02T15:29:39.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.242+0000 7fd5af000700 1 -- 172.21.15.1:0/3969084662 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5a0009c40 con 0x7fd5a8106110 2024-02-02T15:29:39.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.242+0000 7fd5a4c00700 1 -- 172.21.15.1:0/3969084662 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd59000c7e0 con 0x7fd5a8106110 2024-02-02T15:29:39.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.242+0000 7fd5a4c00700 1 -- 172.21.15.1:0/3969084662 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5900054c0 con 0x7fd5a8106110 2024-02-02T15:29:39.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.242+0000 7fd5af000700 1 -- 172.21.15.1:0/3969084662 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5a81a61d0 con 0x7fd5a8106110 2024-02-02T15:29:39.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.246+0000 7fd5a4c00700 1 -- 172.21.15.1:0/3969084662 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 27) v1 ==== 135200+0+0 (secure 0 0 0) 0x7fd59000c950 con 0x7fd5a8106110 2024-02-02T15:29:39.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.247+0000 7fd5a4c00700 1 --2- 172.21.15.1:0/3969084662 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fd5880a0690 0x7fd5880a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:39.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.247+0000 7fd5c2ba1b80 1 Processor -- start 2024-02-02T15:29:39.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.247+0000 7fd5a4c00700 1 -- 172.21.15.1:0/3969084662 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fd590014070 con 0x7fd5a8106110 2024-02-02T15:29:39.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.247+0000 7fd5c2ba1b80 1 -- start start 2024-02-02T15:29:39.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.247+0000 7fd5a7e00700 1 --2- 172.21.15.1:0/3969084662 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fd5880a0690 0x7fd5880a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:39.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.248+0000 7fd5c2ba1b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5570f13e2090 0x5570f13e2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:39.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.248+0000 7fd5c2ba1b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5570f13e29b0 0x5570f13ed320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:39.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.248+0000 7fd5c2ba1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5570f13ed860 0x5570f13efc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:39.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.248+0000 7fd5c2ba1b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570f1356540 con 0x5570f13e2090 2024-02-02T15:29:39.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.248+0000 7fd5c2ba1b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570f133ab20 con 0x5570f13e29b0 2024-02-02T15:29:39.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.248+0000 7fd5c2ba1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570f13384e0 con 0x5570f13ed860 2024-02-02T15:29:39.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.248+0000 7fd5a7400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5570f13e29b0 0x5570f13ed320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:39.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.248+0000 7fd5a7400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5570f13e29b0 0x5570f13ed320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:49024/0 (socket says 172.21.15.1:49024) 2024-02-02T15:29:39.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.248+0000 7fd5a7400700 1 -- 172.21.15.1:0/1481505401 learned_addr learned my addr 172.21.15.1:0/1481505401 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:29:39.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.248+0000 7fd5ad000700 1 --2- 172.21.15.1:0/1481505401 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5570f13ed860 0x5570f13efc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:39.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.248+0000 7fd5a7e00700 1 --2- 172.21.15.1:0/1481505401 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5570f13e2090 0x5570f13e2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:39.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.248+0000 7fd5a7400700 1 -- 172.21.15.1:0/1481505401 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5570f13ed860 msgr2=0x5570f13efc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:39.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.248+0000 7fd5a7400700 1 --2- 172.21.15.1:0/1481505401 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5570f13ed860 0x5570f13efc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:39.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.248+0000 7fd5a7400700 1 -- 172.21.15.1:0/1481505401 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5570f13e2090 msgr2=0x5570f13e2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:39.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.248+0000 7fd5a7400700 1 --2- 172.21.15.1:0/1481505401 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5570f13e2090 0x5570f13e2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:39.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.248+0000 7fd5a7400700 1 -- 172.21.15.1:0/1481505401 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5570f11b5430 con 0x5570f13e29b0 2024-02-02T15:29:39.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.248+0000 7fd5a7e00700 1 --2- 172.21.15.1:0/1481505401 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5570f13e2090 0x5570f13e2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:29:39.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.248+0000 7fd5a7e00700 1 --2- 172.21.15.1:0/3969084662 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fd5880a0690 0x7fd5880a2b50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd598005fd0 tx=0x7fd59800fa90 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:39.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.249+0000 7fd5a7400700 1 --2- 172.21.15.1:0/1481505401 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5570f13e29b0 0x5570f13ed320 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd590099540 tx=0x7fd590098fe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86ae591125fbc79b server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:39.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.249+0000 7fd59d600700 1 -- 172.21.15.1:0/1481505401 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5900995b0 con 0x5570f13e29b0 2024-02-02T15:29:39.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.249+0000 7fd59d600700 1 -- 172.21.15.1:0/1481505401 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd59009d3e0 con 0x5570f13e29b0 2024-02-02T15:29:39.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.249+0000 7fd59d600700 1 -- 172.21.15.1:0/1481505401 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd590095cc0 con 0x5570f13e29b0 2024-02-02T15:29:39.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.249+0000 7fd5c2ba1b80 1 -- 172.21.15.1:0/1481505401 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5570f13e29b0 msgr2=0x5570f13ed320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:39.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.249+0000 7fd5c2ba1b80 1 --2- 172.21.15.1:0/1481505401 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5570f13e29b0 0x5570f13ed320 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd590099540 tx=0x7fd590098fe0 comp rx=0 tx=0).stop 2024-02-02T15:29:39.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.249+0000 7fd5c2ba1b80 1 -- 172.21.15.1:0/1481505401 shutdown_connections 2024-02-02T15:29:39.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.249+0000 7fd5c2ba1b80 1 --2- 172.21.15.1:0/1481505401 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5570f13ed860 0x5570f13efc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:39.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.249+0000 7fd5c2ba1b80 1 --2- 172.21.15.1:0/1481505401 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5570f13e2090 0x5570f13e2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:39.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.249+0000 7fd5c2ba1b80 1 --2- 172.21.15.1:0/1481505401 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5570f13e29b0 0x5570f13ed320 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:39.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.249+0000 7fd5c2ba1b80 1 -- 172.21.15.1:0/1481505401 >> 172.21.15.1:0/1481505401 conn(0x5570f1341e30 msgr2=0x5570f13de430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:39.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.250+0000 7fd5c2ba1b80 1 -- 172.21.15.1:0/1481505401 shutdown_connections 2024-02-02T15:29:39.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.250+0000 7fd5c2ba1b80 1 -- 172.21.15.1:0/1481505401 wait complete. 2024-02-02T15:29:39.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.250+0000 7fd5c2ba1b80 1 -- start start 2024-02-02T15:29:39.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.250+0000 7fd5c2ba1b80 1 Processor -- start 2024-02-02T15:29:39.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.251+0000 7fd5c2ba1b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5570f13e2090 0x5570f13ed2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:39.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.251+0000 7fd5c2ba1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5570f13e29b0 0x5570f13ea4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:39.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.251+0000 7fd5c2ba1b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5570f13ed860 0x5570f13eac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:39.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.251+0000 7fd5c2ba1b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570f1333dd0 con 0x5570f13e2090 2024-02-02T15:29:39.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.251+0000 7fd5c2ba1b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570f12cfe00 con 0x5570f13ed860 2024-02-02T15:29:39.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.251+0000 7fd5c2ba1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570f12cfaa0 con 0x5570f13e29b0 2024-02-02T15:29:39.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.251+0000 7fd5c2ba1b80 10 client.0 _ll_register_callbacks cb 0x5570f12d1310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:29:39.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.251+0000 7fd5a7400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5570f13e29b0 0x5570f13ea4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:39.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.251+0000 7fd5ad000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5570f13ed860 0x5570f13eac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:39.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.251+0000 7fd5a7e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5570f13e2090 0x5570f13ed2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:39.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.251+0000 7fd5a7400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5570f13e29b0 0x5570f13ea4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:40724/0 (socket says 172.21.15.1:40724) 2024-02-02T15:29:39.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.251+0000 7fd5a7400700 1 -- 172.21.15.1:0/1169580500 learned_addr learned my addr 172.21.15.1:0/1169580500 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:29:39.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.251+0000 7fd5a7400700 1 -- 172.21.15.1:0/1169580500 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5570f13ed860 msgr2=0x5570f13eac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:39.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.251+0000 7fd5a7400700 1 --2- 172.21.15.1:0/1169580500 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5570f13ed860 0x5570f13eac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:39.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.251+0000 7fd5a7400700 1 -- 172.21.15.1:0/1169580500 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5570f13e2090 msgr2=0x5570f13ed2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:39.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.251+0000 7fd5a7400700 1 --2- 172.21.15.1:0/1169580500 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5570f13e2090 0x5570f13ed2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:39.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.251+0000 7fd5a7400700 1 -- 172.21.15.1:0/1169580500 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5a81a61d0 con 0x5570f13e29b0 2024-02-02T15:29:39.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.252+0000 7fd5a7400700 1 --2- 172.21.15.1:0/1169580500 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5570f13e29b0 0x5570f13ea4c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd5900c9d60 tx=0x7fd5900a16a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=620966dd265c7fb0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:39.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.252+0000 7fd596000700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:29:39.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.252+0000 7fd596000700 1 -- 172.21.15.1:0/1169580500 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5900cc230 con 0x5570f13e29b0 2024-02-02T15:29:39.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.252+0000 7fd596000700 1 -- 172.21.15.1:0/1169580500 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fd590065270 con 0x5570f13e29b0 2024-02-02T15:29:39.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.252+0000 7fd5c2ba1b80 1 -- 172.21.15.1:0/1169580500 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5570f13d6170 con 0x5570f13e29b0 2024-02-02T15:29:39.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.252+0000 7fd596000700 1 -- 172.21.15.1:0/1169580500 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd5900840c0 con 0x5570f13e29b0 2024-02-02T15:29:39.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.252+0000 7fd5c2ba1b80 10 client.26386 fetch_fsmap learned FSMap version 31 2024-02-02T15:29:39.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.252+0000 7fd5c2ba1b80 1 -- 172.21.15.1:0/1169580500 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5570f1161be0 con 0x5570f13e29b0 2024-02-02T15:29:39.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.252+0000 7fd596000700 1 -- 172.21.15.1:0/1169580500 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5570f13d6170 con 0x5570f13e29b0 2024-02-02T15:29:39.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.252+0000 7fd596000700 1 -- 172.21.15.1:0/1169580500 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 31) v1 ==== 7502+0+0 (secure 0 0 0) 0x7fd5900998a0 con 0x5570f13e29b0 2024-02-02T15:29:39.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.253+0000 7fd5c2ba1b80 10 client.26386 fetch_fsmap finished waiting for FSMap version 31 2024-02-02T15:29:39.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.253+0000 7fd5c2ba1b80 10 client.26386 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:29:39.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.253+0000 7fd5c2ba1b80 20 client.26386 populate_metadata read hostname 'smithi001' 2024-02-02T15:29:39.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.253+0000 7fd5c2ba1b80 1 --2- 172.21.15.1:0/1169580500 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5570f13fd080 0x5570f13ff4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:39.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.253+0000 7fd5c2ba1b80 4 client.26386 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:29:39.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.253+0000 7fd5c2ba1b80 1 -- 172.21.15.1:0/1169580500 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5570f132ce50 con 0x5570f13fd080 2024-02-02T15:29:39.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.253+0000 7fd5a7e00700 1 --2- 172.21.15.1:0/1169580500 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5570f13fd080 0x5570f13ff4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:39.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.253+0000 7fd5a7e00700 1 --2- 172.21.15.1:0/1169580500 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5570f13fd080 0x5570f13ff4a0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11b1a0a9028f3fa7 server_cookie=66943fe6838f1d98 in_seq=0 out_seq=0 2024-02-02T15:29:39.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.253+0000 7fd596000700 10 client.26386 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:29:39.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.254+0000 7fd596000700 1 -- 172.21.15.1:0/1169580500 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5570f132ce50 con 0x5570f13fd080 2024-02-02T15:29:39.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.254+0000 7fd596000700 10 client.26386 handle_command_reply: tid=0 2024-02-02T15:29:39.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.254+0000 7fd5c2ba1b80 1 client.26386 shutdown 2024-02-02T15:29:39.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.255+0000 7fd5c2ba1b80 1 -- 172.21.15.1:0/1169580500 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5570f13e29b0 msgr2=0x5570f13ea4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:39.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.255+0000 7fd5c2ba1b80 1 --2- 172.21.15.1:0/1169580500 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5570f13e29b0 0x5570f13ea4c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd5900c9d60 tx=0x7fd5900a16a0 comp rx=0 tx=0).stop 2024-02-02T15:29:39.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.255+0000 7fd5c2ba1b80 1 -- 172.21.15.1:0/1169580500 shutdown_connections 2024-02-02T15:29:39.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.255+0000 7fd5c2ba1b80 1 --2- 172.21.15.1:0/1169580500 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5570f13fd080 0x5570f13ff4a0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:39.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.255+0000 7fd5c2ba1b80 1 --2- 172.21.15.1:0/1169580500 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5570f13ed860 0x5570f13eac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:39.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.255+0000 7fd5c2ba1b80 1 --2- 172.21.15.1:0/1169580500 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5570f13e2090 0x5570f13ed2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:39.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.255+0000 7fd5c2ba1b80 1 --2- 172.21.15.1:0/1169580500 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5570f13e29b0 0x5570f13ea4c0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:39.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.255+0000 7fd596000700 0 client.26386 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:29:39.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.255+0000 7fd5c2ba1b80 1 -- 172.21.15.1:0/1169580500 >> 172.21.15.1:0/1169580500 conn(0x5570f1341e30 msgr2=0x5570f13de020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:39.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.255+0000 7fd5c2ba1b80 1 -- 172.21.15.1:0/1169580500 shutdown_connections 2024-02-02T15:29:39.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.255+0000 7fd5c2ba1b80 1 -- 172.21.15.1:0/1169580500 wait complete. 2024-02-02T15:29:39.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.255+0000 7fd5c2ba1b80 20 client.26386 trim_cache size 0 max 16384 2024-02-02T15:29:39.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.273+0000 7fd5c2ba1b80 1 Processor -- start 2024-02-02T15:29:39.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.273+0000 7fd5c2ba1b80 1 -- start start 2024-02-02T15:29:39.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.273+0000 7fd5c2ba1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5570f13e2090 0x5570f13f43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:39.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.273+0000 7fd5c2ba1b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5570f13e29b0 0x5570f13e7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:39.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.273+0000 7fd5c2ba1b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5570f13ed860 0x5570f13e7c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:39.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.273+0000 7fd5c2ba1b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570f11b7720 con 0x5570f13e29b0 2024-02-02T15:29:39.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.273+0000 7fd5c2ba1b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570f13de970 con 0x5570f13ed860 2024-02-02T15:29:39.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.273+0000 7fd5c2ba1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570f13ffb00 con 0x5570f13e2090 2024-02-02T15:29:39.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.273+0000 7fd5a7e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5570f13e2090 0x5570f13f43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:39.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.273+0000 7fd5a7400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5570f13e29b0 0x5570f13e7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:39.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.273+0000 7fd5a7e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5570f13e2090 0x5570f13f43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:40726/0 (socket says 172.21.15.1:40726) 2024-02-02T15:29:39.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.273+0000 7fd5a7e00700 1 -- 172.21.15.1:0/1578572628 learned_addr learned my addr 172.21.15.1:0/1578572628 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:29:39.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.274+0000 7fd5ad000700 1 --2- 172.21.15.1:0/1578572628 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5570f13ed860 0x5570f13e7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:39.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.274+0000 7fd5a7e00700 1 -- 172.21.15.1:0/1578572628 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5570f13ed860 msgr2=0x5570f13e7c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:39.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.274+0000 7fd5a7e00700 1 --2- 172.21.15.1:0/1578572628 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5570f13ed860 0x5570f13e7c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:39.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.274+0000 7fd5a7e00700 1 -- 172.21.15.1:0/1578572628 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5570f13e29b0 msgr2=0x5570f13e7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:39.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.274+0000 7fd5a7e00700 1 --2- 172.21.15.1:0/1578572628 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5570f13e29b0 0x5570f13e7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:39.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.274+0000 7fd5a7e00700 1 -- 172.21.15.1:0/1578572628 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5570f12f9f20 con 0x5570f13e2090 2024-02-02T15:29:39.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.274+0000 7fd5a7e00700 1 --2- 172.21.15.1:0/1578572628 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5570f13e2090 0x5570f13f43b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x5570f13ffd70 tx=0x7fd59801c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=276ea9218689b3ee server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:39.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.274+0000 7fd59e000700 1 -- 172.21.15.1:0/1578572628 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd598023040 con 0x5570f13e2090 2024-02-02T15:29:39.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.274+0000 7fd59e000700 1 -- 172.21.15.1:0/1578572628 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd598011e50 con 0x5570f13e2090 2024-02-02T15:29:39.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.274+0000 7fd59e000700 1 -- 172.21.15.1:0/1578572628 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd598028de0 con 0x5570f13e2090 2024-02-02T15:29:39.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.275+0000 7fd5c2ba1b80 1 -- 172.21.15.1:0/1578572628 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5570f13e2090 msgr2=0x5570f13f43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:39.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.275+0000 7fd5c2ba1b80 1 --2- 172.21.15.1:0/1578572628 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5570f13e2090 0x5570f13f43b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x5570f13ffd70 tx=0x7fd59801c000 comp rx=0 tx=0).stop 2024-02-02T15:29:39.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.275+0000 7fd5c2ba1b80 1 -- 172.21.15.1:0/1578572628 shutdown_connections 2024-02-02T15:29:39.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.275+0000 7fd5c2ba1b80 1 --2- 172.21.15.1:0/1578572628 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5570f13ed860 0x5570f13e7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:39.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.275+0000 7fd5c2ba1b80 1 --2- 172.21.15.1:0/1578572628 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5570f13e2090 0x5570f13f43b0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:39.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.275+0000 7fd5c2ba1b80 1 --2- 172.21.15.1:0/1578572628 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5570f13e29b0 0x5570f13e7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:39.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.275+0000 7fd5c2ba1b80 1 -- 172.21.15.1:0/1578572628 >> 172.21.15.1:0/1578572628 conn(0x5570f1341e30 msgr2=0x5570f13ebdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:39.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.275+0000 7fd5c2ba1b80 1 -- 172.21.15.1:0/1578572628 shutdown_connections 2024-02-02T15:29:39.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.275+0000 7fd5c2ba1b80 1 -- 172.21.15.1:0/1578572628 wait complete. 2024-02-02T15:29:39.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.276+0000 7fd5c2ba1b80 1 -- start start 2024-02-02T15:29:39.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.276+0000 7fd5c2ba1b80 1 Processor -- start 2024-02-02T15:29:39.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.276+0000 7fd5c2ba1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5570f13e2090 0x5570f1448ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:39.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.276+0000 7fd5c2ba1b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5570f13e29b0 0x5570f1449430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:39.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.276+0000 7fd5c2ba1b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5570f13ed860 0x5570f1443290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:39.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.276+0000 7fd5c2ba1b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570f1443870 con 0x5570f13e29b0 2024-02-02T15:29:39.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.276+0000 7fd5c2ba1b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570f14439e0 con 0x5570f13ed860 2024-02-02T15:29:39.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.276+0000 7fd5c2ba1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5570f1443ba0 con 0x5570f13e2090 2024-02-02T15:29:39.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.276+0000 7fd5c2ba1b80 10 client.0 _ll_register_callbacks cb 0x5570f12d1310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:29:39.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.276+0000 7fd5a7e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5570f13e2090 0x5570f1448ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:39.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.276+0000 7fd5a7e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5570f13e2090 0x5570f1448ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:40728/0 (socket says 172.21.15.1:40728) 2024-02-02T15:29:39.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.276+0000 7fd5a7e00700 1 -- 172.21.15.1:0/1405674942 learned_addr learned my addr 172.21.15.1:0/1405674942 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:29:39.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.276+0000 7fd5ad000700 1 --2- 172.21.15.1:0/1405674942 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5570f13ed860 0x5570f1443290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:39.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.276+0000 7fd5a7400700 1 --2- 172.21.15.1:0/1405674942 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5570f13e29b0 0x5570f1449430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:39.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.277+0000 7fd5a7e00700 1 -- 172.21.15.1:0/1405674942 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5570f13ed860 msgr2=0x5570f1443290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:39.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.277+0000 7fd5a7e00700 1 --2- 172.21.15.1:0/1405674942 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5570f13ed860 0x5570f1443290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:39.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.277+0000 7fd5a7e00700 1 -- 172.21.15.1:0/1405674942 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5570f13e29b0 msgr2=0x5570f1449430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:39.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.277+0000 7fd5a7e00700 1 --2- 172.21.15.1:0/1405674942 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5570f13e29b0 0x5570f1449430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:39.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.277+0000 7fd5a7e00700 1 -- 172.21.15.1:0/1405674942 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd598011930 con 0x5570f13e2090 2024-02-02T15:29:39.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.277+0000 7fd5ad000700 1 --2- 172.21.15.1:0/1405674942 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5570f13ed860 0x5570f1443290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:29:39.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.277+0000 7fd5a7e00700 1 --2- 172.21.15.1:0/1405674942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5570f13e2090 0x5570f1448ef0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd5980094e0 tx=0x7fd598005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15f6f6cbb9afdec server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:39.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.277+0000 7fd596000700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:29:39.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.277+0000 7fd596000700 1 -- 172.21.15.1:0/1405674942 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd598023040 con 0x5570f13e2090 2024-02-02T15:29:39.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.277+0000 7fd5c2ba1b80 1 -- 172.21.15.1:0/1405674942 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd59009d3e0 con 0x5570f13e2090 2024-02-02T15:29:39.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.277+0000 7fd596000700 1 -- 172.21.15.1:0/1405674942 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fd598036030 con 0x5570f13e2090 2024-02-02T15:29:39.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.278+0000 7fd596000700 1 -- 172.21.15.1:0/1405674942 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd5980289c0 con 0x5570f13e2090 2024-02-02T15:29:39.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.278+0000 7fd5c2ba1b80 10 client.26392 fetch_fsmap learned FSMap version 31 2024-02-02T15:29:39.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.278+0000 7fd5c2ba1b80 1 -- 172.21.15.1:0/1405674942 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5570f11b9d50 con 0x5570f13e2090 2024-02-02T15:29:39.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.278+0000 7fd596000700 1 -- 172.21.15.1:0/1405674942 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd59009d3e0 con 0x5570f13e2090 2024-02-02T15:29:39.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.278+0000 7fd596000700 1 -- 172.21.15.1:0/1405674942 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 31) v1 ==== 7502+0+0 (secure 0 0 0) 0x7fd598030480 con 0x5570f13e2090 2024-02-02T15:29:39.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.278+0000 7fd5c2ba1b80 10 client.26392 fetch_fsmap finished waiting for FSMap version 31 2024-02-02T15:29:39.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.278+0000 7fd5c2ba1b80 10 client.26392 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:29:39.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.278+0000 7fd5c2ba1b80 20 client.26392 populate_metadata read hostname 'smithi001' 2024-02-02T15:29:39.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.278+0000 7fd5c2ba1b80 1 --2- 172.21.15.1:0/1405674942 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5570f1450d70 0x5570f1453190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:39.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.278+0000 7fd5c2ba1b80 4 client.26392 mds_command: new command op to 24557 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-02T15:29:39.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.278+0000 7fd5c2ba1b80 1 -- 172.21.15.1:0/1405674942 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5570f12f2fd0 con 0x5570f1450d70 2024-02-02T15:29:39.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.279+0000 7fd5a7400700 1 --2- 172.21.15.1:0/1405674942 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5570f1450d70 0x5570f1453190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:39.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.279+0000 7fd5a7400700 1 --2- 172.21.15.1:0/1405674942 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5570f1450d70 0x5570f1453190 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef7b09292f452da server_cookie=ebac1ae1b184ed8f in_seq=0 out_seq=0 2024-02-02T15:29:39.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.279+0000 7fd596000700 10 client.26392 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:29:39.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.280+0000 7fd596000700 1 -- 172.21.15.1:0/1405674942 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5570f12f2fd0 con 0x5570f1450d70 2024-02-02T15:29:39.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.280+0000 7fd596000700 10 client.26392 handle_command_reply: tid=0 2024-02-02T15:29:39.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.280+0000 7fd5c2ba1b80 1 client.26392 shutdown 2024-02-02T15:29:39.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.280+0000 7fd5c2ba1b80 1 -- 172.21.15.1:0/1405674942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5570f13e2090 msgr2=0x5570f1448ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:39.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.280+0000 7fd5c2ba1b80 1 --2- 172.21.15.1:0/1405674942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5570f13e2090 0x5570f1448ef0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd5980094e0 tx=0x7fd598005e80 comp rx=0 tx=0).stop 2024-02-02T15:29:39.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.280+0000 7fd5c2ba1b80 1 -- 172.21.15.1:0/1405674942 shutdown_connections 2024-02-02T15:29:39.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.280+0000 7fd5c2ba1b80 1 --2- 172.21.15.1:0/1405674942 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5570f1450d70 0x5570f1453190 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:39.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.280+0000 7fd5c2ba1b80 1 --2- 172.21.15.1:0/1405674942 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5570f13ed860 0x5570f1443290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:39.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.280+0000 7fd5c2ba1b80 1 --2- 172.21.15.1:0/1405674942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5570f13e2090 0x5570f1448ef0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:39.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.280+0000 7fd5c2ba1b80 1 --2- 172.21.15.1:0/1405674942 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5570f13e29b0 0x5570f1449430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:39.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.280+0000 7fd5c2ba1b80 1 -- 172.21.15.1:0/1405674942 >> 172.21.15.1:0/1405674942 conn(0x5570f1341e30 msgr2=0x5570f13ec660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:39.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.280+0000 7fd596000700 0 client.26392 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:29:39.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.280+0000 7fd5c2ba1b80 1 -- 172.21.15.1:0/1405674942 shutdown_connections 2024-02-02T15:29:39.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.280+0000 7fd5c2ba1b80 1 -- 172.21.15.1:0/1405674942 wait complete. 2024-02-02T15:29:39.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.281+0000 7fd5c2ba1b80 20 client.26392 trim_cache size 0 max 16384 2024-02-02T15:29:39.290 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:29:39.291 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2024-02-02T15:29:39.291 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "2ca29b9a-0a1d-43d3-b629-cefb164c30ca", 2024-02-02T15:29:39.291 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2024-02-02T15:29:39.291 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:29:39.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.281+0000 7fd5af000700 1 -- 172.21.15.1:0/3969084662 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fd5880a0690 msgr2=0x7fd5880a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:39.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.281+0000 7fd5af000700 1 --2- 172.21.15.1:0/3969084662 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fd5880a0690 0x7fd5880a2b50 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd598005fd0 tx=0x7fd59800fa90 comp rx=0 tx=0).stop 2024-02-02T15:29:39.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.281+0000 7fd5af000700 1 -- 172.21.15.1:0/3969084662 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd5a8106110 msgr2=0x7fd5a8199710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:39.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.281+0000 7fd5af000700 1 --2- 172.21.15.1:0/3969084662 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd5a8106110 0x7fd5a8199710 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fd59000e970 tx=0x7fd59000ec80 comp rx=0 tx=0).stop 2024-02-02T15:29:39.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.281+0000 7fd5af000700 1 -- 172.21.15.1:0/3969084662 shutdown_connections 2024-02-02T15:29:39.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.281+0000 7fd5af000700 1 --2- 172.21.15.1:0/3969084662 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fd5880a0690 0x7fd5880a2b50 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:39.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.281+0000 7fd5af000700 1 --2- 172.21.15.1:0/3969084662 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd5a819db70 0x7fd5a819dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:39.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.281+0000 7fd5af000700 1 --2- 172.21.15.1:0/3969084662 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd5a8100ff0 0x7fd5a81991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:39.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.281+0000 7fd5af000700 1 --2- 172.21.15.1:0/3969084662 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd5a8106110 0x7fd5a8199710 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:39.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.281+0000 7fd5af000700 1 -- 172.21.15.1:0/3969084662 >> 172.21.15.1:0/3969084662 conn(0x7fd5a80fc810 msgr2=0x7fd5a80fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:39.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.281+0000 7fd5af000700 1 -- 172.21.15.1:0/3969084662 shutdown_connections 2024-02-02T15:29:39.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.281+0000 7fd5af000700 1 -- 172.21.15.1:0/3969084662 wait complete. 2024-02-02T15:29:39.302 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-02T15:29:39.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.515+0000 7f024e400700 1 -- 172.21.15.1:0/1945039593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0248103c40 msgr2=0x7f0248107c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:39.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.515+0000 7f024e400700 1 --2- 172.21.15.1:0/1945039593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0248103c40 0x7f0248107c80 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0230009990 tx=0x7f0230009ca0 comp rx=0 tx=0).stop 2024-02-02T15:29:39.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.515+0000 7f024e400700 1 -- 172.21.15.1:0/1945039593 shutdown_connections 2024-02-02T15:29:39.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.515+0000 7f024e400700 1 --2- 172.21.15.1:0/1945039593 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f02481083b0 0x7f024810ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:39.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.515+0000 7f024e400700 1 --2- 172.21.15.1:0/1945039593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0248103290 0x7f0248103670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:39.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.515+0000 7f024e400700 1 --2- 172.21.15.1:0/1945039593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0248103c40 0x7f0248107c80 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:39.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.515+0000 7f024e400700 1 -- 172.21.15.1:0/1945039593 >> 172.21.15.1:0/1945039593 conn(0x7f02480feb30 msgr2=0x7f0248100f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:39.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.515+0000 7f024e400700 1 -- 172.21.15.1:0/1945039593 shutdown_connections 2024-02-02T15:29:39.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.516+0000 7f024e400700 1 -- 172.21.15.1:0/1945039593 wait complete. 2024-02-02T15:29:39.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.517+0000 7f024e400700 1 Processor -- start 2024-02-02T15:29:39.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.517+0000 7f024e400700 1 -- start start 2024-02-02T15:29:39.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.517+0000 7f024e400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0248103290 0x7f02481990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:39.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.517+0000 7f024e400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f02481083b0 0x7f0248199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:39.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.517+0000 7f024e400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f024819da80 0x7f024819df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:39.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.517+0000 7f024e400700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f024819e500 con 0x7f02481083b0 2024-02-02T15:29:39.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.517+0000 7f024e400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f024819e670 con 0x7f024819da80 2024-02-02T15:29:39.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.517+0000 7f024e400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f024819e830 con 0x7f0248103290 2024-02-02T15:29:39.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.517+0000 7f0247400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0248103290 0x7f02481990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:39.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.517+0000 7f0246a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f02481083b0 0x7f0248199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:39.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.517+0000 7f0247e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f024819da80 0x7f024819df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:39.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.518+0000 7f0247400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0248103290 0x7f02481990e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:40756/0 (socket says 172.21.15.1:40756) 2024-02-02T15:29:39.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.518+0000 7f0247400700 1 -- 172.21.15.1:0/84187275 learned_addr learned my addr 172.21.15.1:0/84187275 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:29:39.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.518+0000 7f0247400700 1 -- 172.21.15.1:0/84187275 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f024819da80 msgr2=0x7f024819df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:39.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.518+0000 7f0247400700 1 --2- 172.21.15.1:0/84187275 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f024819da80 0x7f024819df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:39.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.518+0000 7f0247400700 1 -- 172.21.15.1:0/84187275 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f02481083b0 msgr2=0x7f0248199620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:39.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.518+0000 7f0247400700 1 --2- 172.21.15.1:0/84187275 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f02481083b0 0x7f0248199620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:39.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.518+0000 7f0247400700 1 -- 172.21.15.1:0/84187275 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02300095f0 con 0x7f0248103290 2024-02-02T15:29:39.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.518+0000 7f0246a00700 1 --2- 172.21.15.1:0/84187275 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f02481083b0 0x7f0248199620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:29:39.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.518+0000 7f0247400700 1 --2- 172.21.15.1:0/84187275 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0248103290 0x7f02481990e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f023800e9a0 tx=0x7f023800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:39.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.518+0000 7f023fe00700 1 -- 172.21.15.1:0/84187275 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0238009790 con 0x7f0248103290 2024-02-02T15:29:39.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.518+0000 7f023fe00700 1 -- 172.21.15.1:0/84187275 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0238010430 con 0x7f0248103290 2024-02-02T15:29:39.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.518+0000 7f023fe00700 1 -- 172.21.15.1:0/84187275 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0238018680 con 0x7f0248103290 2024-02-02T15:29:39.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.518+0000 7f024e400700 1 -- 172.21.15.1:0/84187275 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02481a5d90 con 0x7f0248103290 2024-02-02T15:29:39.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.519+0000 7f024e400700 1 -- 172.21.15.1:0/84187275 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02481a61c0 con 0x7f0248103290 2024-02-02T15:29:39.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.522+0000 7f023fe00700 1 -- 172.21.15.1:0/84187275 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 27) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f02380105a0 con 0x7f0248103290 2024-02-02T15:29:39.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.524+0000 7f0261fd3b80 1 Processor -- start 2024-02-02T15:29:39.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.524+0000 7f0261fd3b80 1 -- start start 2024-02-02T15:29:39.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.524+0000 7f023fe00700 1 --2- 172.21.15.1:0/84187275 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f02280a0710 0x7f02280a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:39.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.524+0000 7f023fe00700 1 -- 172.21.15.1:0/84187275 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f0238014070 con 0x7f0248103290 2024-02-02T15:29:39.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.524+0000 7f0247e00700 1 --2- 172.21.15.1:0/84187275 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f02280a0710 0x7f02280a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:39.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.524+0000 7f0261fd3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d5f6239010 0x55d5f62393f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:39.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.524+0000 7f0261fd3b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d5f6239930 0x55d5f62442a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:39.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.524+0000 7f0261fd3b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d5f62447e0 0x55d5f6246bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:39.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.524+0000 7f0261fd3b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5f61ad4c0 con 0x55d5f62447e0 2024-02-02T15:29:39.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.524+0000 7f0261fd3b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5f6191aa0 con 0x55d5f6239930 2024-02-02T15:29:39.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.524+0000 7f0261fd3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5f618f460 con 0x55d5f6239010 2024-02-02T15:29:39.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.525+0000 7f0246a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d5f6239930 0x55d5f62442a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:39.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.525+0000 7f0247400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d5f6239010 0x55d5f62393f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:39.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.525+0000 7f0246a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d5f6239930 0x55d5f62442a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:49086/0 (socket says 172.21.15.1:49086) 2024-02-02T15:29:39.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.525+0000 7f0247400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d5f6239010 0x55d5f62393f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:40764/0 (socket says 172.21.15.1:40764) 2024-02-02T15:29:39.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.525+0000 7f0246a00700 1 -- 172.21.15.1:0/1601942605 learned_addr learned my addr 172.21.15.1:0/1601942605 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:29:39.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.525+0000 7f0246a00700 1 -- 172.21.15.1:0/1601942605 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d5f6239010 msgr2=0x55d5f62393f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:39.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.525+0000 7f0246a00700 1 --2- 172.21.15.1:0/1601942605 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d5f6239010 0x55d5f62393f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:39.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.525+0000 7f0246a00700 1 -- 172.21.15.1:0/1601942605 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d5f62447e0 msgr2=0x55d5f6246bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:39.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.525+0000 7f0246a00700 1 --2- 172.21.15.1:0/1601942605 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d5f62447e0 0x55d5f6246bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:39.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.525+0000 7f0246a00700 1 -- 172.21.15.1:0/1601942605 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d5f600c3b0 con 0x55d5f6239930 2024-02-02T15:29:39.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.525+0000 7f0247e00700 1 --2- 172.21.15.1:0/84187275 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f02280a0710 0x7f02280a2bd0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f024000f5e0 tx=0x7f0240009920 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:39.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.525+0000 7f0246a00700 1 --2- 172.21.15.1:0/1601942605 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d5f6239930 0x55d5f62442a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0230015000 tx=0x7f0230005ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f08b4dbed110ff16 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:39.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.526+0000 7f023cc00700 1 -- 172.21.15.1:0/1601942605 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f023001d030 con 0x55d5f6239930 2024-02-02T15:29:39.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.526+0000 7f023cc00700 1 -- 172.21.15.1:0/1601942605 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f023001e040 con 0x55d5f6239930 2024-02-02T15:29:39.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.526+0000 7f023cc00700 1 -- 172.21.15.1:0/1601942605 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0230004d20 con 0x55d5f6239930 2024-02-02T15:29:39.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.526+0000 7f0261fd3b80 1 -- 172.21.15.1:0/1601942605 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d5f6239930 msgr2=0x55d5f62442a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:39.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.526+0000 7f0261fd3b80 1 --2- 172.21.15.1:0/1601942605 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d5f6239930 0x55d5f62442a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f0230015000 tx=0x7f0230005ea0 comp rx=0 tx=0).stop 2024-02-02T15:29:39.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.526+0000 7f0261fd3b80 1 -- 172.21.15.1:0/1601942605 shutdown_connections 2024-02-02T15:29:39.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.526+0000 7f0261fd3b80 1 --2- 172.21.15.1:0/1601942605 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d5f62447e0 0x55d5f6246bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:39.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.526+0000 7f0261fd3b80 1 --2- 172.21.15.1:0/1601942605 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d5f6239010 0x55d5f62393f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:39.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.526+0000 7f0261fd3b80 1 --2- 172.21.15.1:0/1601942605 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d5f6239930 0x55d5f62442a0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:39.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.526+0000 7f0261fd3b80 1 -- 172.21.15.1:0/1601942605 >> 172.21.15.1:0/1601942605 conn(0x55d5f6198db0 msgr2=0x55d5f62353b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:39.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.527+0000 7f0261fd3b80 1 -- 172.21.15.1:0/1601942605 shutdown_connections 2024-02-02T15:29:39.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.527+0000 7f0261fd3b80 1 -- 172.21.15.1:0/1601942605 wait complete. 2024-02-02T15:29:39.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.527+0000 7f0261fd3b80 1 -- start start 2024-02-02T15:29:39.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.527+0000 7f0261fd3b80 1 Processor -- start 2024-02-02T15:29:39.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.528+0000 7f0261fd3b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d5f6239010 0x55d5f6244140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:39.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.528+0000 7f0261fd3b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d5f6239930 0x55d5f6241350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:39.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.528+0000 7f0261fd3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d5f62447e0 0x55d5f62418d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:39.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.528+0000 7f0261fd3b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5f618ad50 con 0x55d5f6239010 2024-02-02T15:29:39.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.528+0000 7f0261fd3b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5f6126d80 con 0x55d5f6239930 2024-02-02T15:29:39.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.528+0000 7f0261fd3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5f6126a20 con 0x55d5f62447e0 2024-02-02T15:29:39.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.528+0000 7f0261fd3b80 10 client.0 _ll_register_callbacks cb 0x55d5f6128290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:29:39.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.528+0000 7f0246a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d5f6239930 0x55d5f6241350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:39.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.528+0000 7f0246a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d5f6239930 0x55d5f6241350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:49100/0 (socket says 172.21.15.1:49100) 2024-02-02T15:29:39.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.528+0000 7f0246a00700 1 -- 172.21.15.1:0/4127806919 learned_addr learned my addr 172.21.15.1:0/4127806919 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:29:39.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.528+0000 7f0247e00700 1 --2- 172.21.15.1:0/4127806919 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d5f62447e0 0x55d5f62418d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:39.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.528+0000 7f0247400700 1 --2- 172.21.15.1:0/4127806919 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d5f6239010 0x55d5f6244140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:39.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.528+0000 7f0246a00700 1 -- 172.21.15.1:0/4127806919 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d5f62447e0 msgr2=0x55d5f62418d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:39.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.529+0000 7f0246a00700 1 --2- 172.21.15.1:0/4127806919 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d5f62447e0 0x55d5f62418d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:39.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.529+0000 7f0246a00700 1 -- 172.21.15.1:0/4127806919 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d5f6239010 msgr2=0x55d5f6244140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:39.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.529+0000 7f0246a00700 1 --2- 172.21.15.1:0/4127806919 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d5f6239010 0x55d5f6244140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:39.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.529+0000 7f0246a00700 1 -- 172.21.15.1:0/4127806919 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02481107a0 con 0x55d5f6239930 2024-02-02T15:29:39.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.529+0000 7f0247400700 1 --2- 172.21.15.1:0/4127806919 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d5f6239010 0x55d5f6244140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:29:39.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.529+0000 7f0246a00700 1 --2- 172.21.15.1:0/4127806919 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d5f6239930 0x55d5f6241350 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0230005cd0 tx=0x7f0230005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52547131825667db server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:39.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.529+0000 7f0235600700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:29:39.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.529+0000 7f0235600700 1 -- 172.21.15.1:0/4127806919 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f023001d070 con 0x55d5f6239930 2024-02-02T15:29:39.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.529+0000 7f0235600700 1 -- 172.21.15.1:0/4127806919 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f0230004e20 con 0x55d5f6239930 2024-02-02T15:29:39.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.529+0000 7f0261fd3b80 1 -- 172.21.15.1:0/4127806919 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d5f622d0f0 con 0x55d5f6239930 2024-02-02T15:29:39.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.529+0000 7f0235600700 1 -- 172.21.15.1:0/4127806919 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0230005520 con 0x55d5f6239930 2024-02-02T15:29:39.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.530+0000 7f0261fd3b80 10 client.26414 fetch_fsmap learned FSMap version 31 2024-02-02T15:29:39.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.530+0000 7f0261fd3b80 1 -- 172.21.15.1:0/4127806919 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55d5f5fb8b60 con 0x55d5f6239930 2024-02-02T15:29:39.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.530+0000 7f0235600700 1 -- 172.21.15.1:0/4127806919 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d5f622d0f0 con 0x55d5f6239930 2024-02-02T15:29:39.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.530+0000 7f0235600700 1 -- 172.21.15.1:0/4127806919 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 31) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f023000f460 con 0x55d5f6239930 2024-02-02T15:29:39.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.530+0000 7f0261fd3b80 10 client.26414 fetch_fsmap finished waiting for FSMap version 31 2024-02-02T15:29:39.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.530+0000 7f0261fd3b80 10 client.26414 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:29:39.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.530+0000 7f0261fd3b80 20 client.26414 populate_metadata read hostname 'smithi001' 2024-02-02T15:29:39.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.530+0000 7f0261fd3b80 1 --2- 172.21.15.1:0/4127806919 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55d5f6253d40 0x55d5f6256160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:39.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.530+0000 7f0261fd3b80 4 client.26414 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:29:39.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.530+0000 7f0261fd3b80 1 -- 172.21.15.1:0/4127806919 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d5f6183dd0 con 0x55d5f6253d40 2024-02-02T15:29:39.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.530+0000 7f0247400700 1 --2- 172.21.15.1:0/4127806919 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55d5f6253d40 0x55d5f6256160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:39.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.531+0000 7f0235600700 10 client.26414 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:29:39.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.531+0000 7f0247400700 1 --2- 172.21.15.1:0/4127806919 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55d5f6253d40 0x55d5f6256160 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb55034da746abe5 server_cookie=f42a4443352a129a in_seq=0 out_seq=0 2024-02-02T15:29:39.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.531+0000 7f0235600700 1 -- 172.21.15.1:0/4127806919 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55d5f6183dd0 con 0x55d5f6253d40 2024-02-02T15:29:39.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.531+0000 7f0235600700 10 client.26414 handle_command_reply: tid=0 2024-02-02T15:29:39.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.531+0000 7f0261fd3b80 1 client.26414 shutdown 2024-02-02T15:29:39.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.532+0000 7f0261fd3b80 1 -- 172.21.15.1:0/4127806919 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d5f6239930 msgr2=0x55d5f6241350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:39.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.532+0000 7f0261fd3b80 1 --2- 172.21.15.1:0/4127806919 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d5f6239930 0x55d5f6241350 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0230005cd0 tx=0x7f0230005f10 comp rx=0 tx=0).stop 2024-02-02T15:29:39.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.532+0000 7f0261fd3b80 1 -- 172.21.15.1:0/4127806919 shutdown_connections 2024-02-02T15:29:39.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.532+0000 7f0261fd3b80 1 --2- 172.21.15.1:0/4127806919 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55d5f6253d40 0x55d5f6256160 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:39.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.532+0000 7f0261fd3b80 1 --2- 172.21.15.1:0/4127806919 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d5f62447e0 0x55d5f62418d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:39.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.532+0000 7f0261fd3b80 1 --2- 172.21.15.1:0/4127806919 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d5f6239010 0x55d5f6244140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:39.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.532+0000 7f0261fd3b80 1 --2- 172.21.15.1:0/4127806919 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d5f6239930 0x55d5f6241350 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:39.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.532+0000 7f0235600700 0 client.26414 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:29:39.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.532+0000 7f0261fd3b80 1 -- 172.21.15.1:0/4127806919 >> 172.21.15.1:0/4127806919 conn(0x55d5f6198db0 msgr2=0x55d5f6234fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:39.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.532+0000 7f0261fd3b80 1 -- 172.21.15.1:0/4127806919 shutdown_connections 2024-02-02T15:29:39.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.532+0000 7f0261fd3b80 1 -- 172.21.15.1:0/4127806919 wait complete. 2024-02-02T15:29:39.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.532+0000 7f0261fd3b80 20 client.26414 trim_cache size 0 max 16384 2024-02-02T15:29:39.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.549+0000 7f0261fd3b80 1 Processor -- start 2024-02-02T15:29:39.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.549+0000 7f0261fd3b80 1 -- start start 2024-02-02T15:29:39.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.550+0000 7f0261fd3b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d5f6239010 0x55d5f624b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:39.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.550+0000 7f0261fd3b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d5f6239930 0x55d5f6243c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:39.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.550+0000 7f0261fd3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d5f62447e0 0x55d5f6234cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:39.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.550+0000 7f0261fd3b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5f600e6a0 con 0x55d5f6239010 2024-02-02T15:29:39.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.550+0000 7f0261fd3b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5f6239f90 con 0x55d5f6239930 2024-02-02T15:29:39.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.550+0000 7f0261fd3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5f6250aa0 con 0x55d5f62447e0 2024-02-02T15:29:39.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.550+0000 7f0247400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d5f6239010 0x55d5f624b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:39.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.550+0000 7f0247400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d5f6239010 0x55d5f624b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35194/0 (socket says 172.21.15.1:35194) 2024-02-02T15:29:39.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.550+0000 7f0247400700 1 -- 172.21.15.1:0/1456474693 learned_addr learned my addr 172.21.15.1:0/1456474693 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:29:39.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.550+0000 7f0246a00700 1 --2- 172.21.15.1:0/1456474693 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d5f6239930 0x55d5f6243c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:39.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.550+0000 7f0247e00700 1 --2- 172.21.15.1:0/1456474693 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d5f62447e0 0x55d5f6234cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:39.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.550+0000 7f0247400700 1 -- 172.21.15.1:0/1456474693 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d5f62447e0 msgr2=0x55d5f6234cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:39.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.550+0000 7f0247400700 1 --2- 172.21.15.1:0/1456474693 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d5f62447e0 0x55d5f6234cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:39.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.550+0000 7f0247400700 1 -- 172.21.15.1:0/1456474693 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d5f6239930 msgr2=0x55d5f6243c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:39.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.550+0000 7f0247400700 1 --2- 172.21.15.1:0/1456474693 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d5f6239930 0x55d5f6243c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:39.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.550+0000 7f0247400700 1 -- 172.21.15.1:0/1456474693 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d5f6150ea0 con 0x55d5f6239010 2024-02-02T15:29:39.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.550+0000 7f0246a00700 1 --2- 172.21.15.1:0/1456474693 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d5f6239930 0x55d5f6243c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:29:39.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.551+0000 7f0247400700 1 --2- 172.21.15.1:0/1456474693 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d5f6239010 0x55d5f624b310 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f0238095e50 tx=0x7f023800c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a405bf685ab1bed server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:39.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.551+0000 7f023d600700 1 -- 172.21.15.1:0/1456474693 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02380acea0 con 0x55d5f6239010 2024-02-02T15:29:39.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.551+0000 7f023d600700 1 -- 172.21.15.1:0/1456474693 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f02380993e0 con 0x55d5f6239010 2024-02-02T15:29:39.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.551+0000 7f023d600700 1 -- 172.21.15.1:0/1456474693 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02380978a0 con 0x55d5f6239010 2024-02-02T15:29:39.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.551+0000 7f0261fd3b80 1 -- 172.21.15.1:0/1456474693 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d5f6239010 msgr2=0x55d5f624b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:39.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.551+0000 7f0261fd3b80 1 --2- 172.21.15.1:0/1456474693 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d5f6239010 0x55d5f624b310 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f0238095e50 tx=0x7f023800c480 comp rx=0 tx=0).stop 2024-02-02T15:29:39.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.552+0000 7f0261fd3b80 1 -- 172.21.15.1:0/1456474693 shutdown_connections 2024-02-02T15:29:39.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.552+0000 7f0261fd3b80 1 --2- 172.21.15.1:0/1456474693 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d5f62447e0 0x55d5f6234cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:39.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.552+0000 7f0261fd3b80 1 --2- 172.21.15.1:0/1456474693 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d5f6239010 0x55d5f624b310 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:39.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.552+0000 7f0261fd3b80 1 --2- 172.21.15.1:0/1456474693 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d5f6239930 0x55d5f6243c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:39.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.552+0000 7f0261fd3b80 1 -- 172.21.15.1:0/1456474693 >> 172.21.15.1:0/1456474693 conn(0x55d5f6198db0 msgr2=0x55d5f62345c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:39.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.552+0000 7f0261fd3b80 1 -- 172.21.15.1:0/1456474693 shutdown_connections 2024-02-02T15:29:39.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.552+0000 7f0261fd3b80 1 -- 172.21.15.1:0/1456474693 wait complete. 2024-02-02T15:29:39.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.552+0000 7f0261fd3b80 1 -- start start 2024-02-02T15:29:39.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.552+0000 7f0261fd3b80 1 Processor -- start 2024-02-02T15:29:39.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.553+0000 7f0261fd3b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d5f6239010 0x55d5f629f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:39.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.553+0000 7f0261fd3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d5f6239930 0x55d5f629f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:39.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.553+0000 7f0261fd3b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d5f62447e0 0x55d5f62997b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:39.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.553+0000 7f0261fd3b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5f6299dc0 con 0x55d5f6239010 2024-02-02T15:29:39.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.553+0000 7f0261fd3b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5f6299f30 con 0x55d5f62447e0 2024-02-02T15:29:39.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.553+0000 7f0261fd3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5f629a0f0 con 0x55d5f6239930 2024-02-02T15:29:39.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.553+0000 7f0261fd3b80 10 client.0 _ll_register_callbacks cb 0x55d5f6128290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:29:39.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.553+0000 7f0246a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d5f6239930 0x55d5f629f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:39.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.553+0000 7f0247400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d5f6239010 0x55d5f629f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:39.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.553+0000 7f0246a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d5f6239930 0x55d5f629f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:40782/0 (socket says 172.21.15.1:40782) 2024-02-02T15:29:39.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.553+0000 7f0246a00700 1 -- 172.21.15.1:0/1903768077 learned_addr learned my addr 172.21.15.1:0/1903768077 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:29:39.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.553+0000 7f0247e00700 1 --2- 172.21.15.1:0/1903768077 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d5f62447e0 0x55d5f62997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:39.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.553+0000 7f0246a00700 1 -- 172.21.15.1:0/1903768077 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d5f62447e0 msgr2=0x55d5f62997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:39.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.553+0000 7f0246a00700 1 --2- 172.21.15.1:0/1903768077 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d5f62447e0 0x55d5f62997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:39.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.553+0000 7f0246a00700 1 -- 172.21.15.1:0/1903768077 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d5f6239010 msgr2=0x55d5f629f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:29:39.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.553+0000 7f0246a00700 1 --2- 172.21.15.1:0/1903768077 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d5f6239010 0x55d5f629f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:39.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.553+0000 7f0246a00700 1 -- 172.21.15.1:0/1903768077 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d5f5fb8b60 con 0x55d5f6239930 2024-02-02T15:29:39.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.554+0000 7f0246a00700 1 --2- 172.21.15.1:0/1903768077 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d5f6239930 0x55d5f629f690 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0230000c00 tx=0x7f0230004880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5b1bbb81872c676 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:29:39.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.554+0000 7f0235600700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:29:39.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.554+0000 7f0235600700 1 -- 172.21.15.1:0/1903768077 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f023001d040 con 0x55d5f6239930 2024-02-02T15:29:39.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.554+0000 7f0261fd3b80 1 -- 172.21.15.1:0/1903768077 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f023001e040 con 0x55d5f6239930 2024-02-02T15:29:39.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.554+0000 7f0235600700 1 -- 172.21.15.1:0/1903768077 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f0230000e40 con 0x55d5f6239930 2024-02-02T15:29:39.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.554+0000 7f0235600700 1 -- 172.21.15.1:0/1903768077 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0230004be0 con 0x55d5f6239930 2024-02-02T15:29:39.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.554+0000 7f0261fd3b80 10 client.26410 fetch_fsmap learned FSMap version 31 2024-02-02T15:29:39.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.554+0000 7f0261fd3b80 1 -- 172.21.15.1:0/1903768077 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f02481a61c0 con 0x55d5f6239930 2024-02-02T15:29:39.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.555+0000 7f0235600700 1 -- 172.21.15.1:0/1903768077 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f023001e040 con 0x55d5f6239930 2024-02-02T15:29:39.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.555+0000 7f0235600700 1 -- 172.21.15.1:0/1903768077 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 31) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f023000bc00 con 0x55d5f6239930 2024-02-02T15:29:39.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.555+0000 7f0261fd3b80 10 client.26410 fetch_fsmap finished waiting for FSMap version 31 2024-02-02T15:29:39.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.555+0000 7f0261fd3b80 10 client.26410 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:29:39.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.555+0000 7f0261fd3b80 20 client.26410 populate_metadata read hostname 'smithi001' 2024-02-02T15:29:39.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.555+0000 7f0261fd3b80 1 --2- 172.21.15.1:0/1903768077 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55d5f62a6c20 0x55d5f62a9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:29:39.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.555+0000 7f0261fd3b80 4 client.26410 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2024-02-02T15:29:39.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.555+0000 7f0261fd3b80 1 -- 172.21.15.1:0/1903768077 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d5f6149f50 con 0x55d5f62a6c20 2024-02-02T15:29:39.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.555+0000 7f0247400700 1 --2- 172.21.15.1:0/1903768077 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55d5f62a6c20 0x55d5f62a9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:29:39.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.555+0000 7f0247400700 1 --2- 172.21.15.1:0/1903768077 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55d5f62a6c20 0x55d5f62a9040 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba2f608410183e53 server_cookie=2c552692e35b1044 in_seq=0 out_seq=0 2024-02-02T15:29:39.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.555+0000 7f0235600700 10 client.26410 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:29:39.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.556+0000 7f0235600700 1 -- 172.21.15.1:0/1903768077 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55d5f6149f50 con 0x55d5f62a6c20 2024-02-02T15:29:39.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.556+0000 7f0235600700 10 client.26410 handle_command_reply: tid=0 2024-02-02T15:29:39.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.556+0000 7f0261fd3b80 1 client.26410 shutdown 2024-02-02T15:29:39.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.556+0000 7f0261fd3b80 1 -- 172.21.15.1:0/1903768077 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d5f6239930 msgr2=0x55d5f629f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:39.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.556+0000 7f0261fd3b80 1 --2- 172.21.15.1:0/1903768077 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d5f6239930 0x55d5f629f690 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0230000c00 tx=0x7f0230004880 comp rx=0 tx=0).stop 2024-02-02T15:29:39.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.556+0000 7f0261fd3b80 1 -- 172.21.15.1:0/1903768077 shutdown_connections 2024-02-02T15:29:39.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.556+0000 7f0261fd3b80 1 --2- 172.21.15.1:0/1903768077 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55d5f62a6c20 0x55d5f62a9040 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:39.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.556+0000 7f0261fd3b80 1 --2- 172.21.15.1:0/1903768077 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d5f62447e0 0x55d5f62997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:39.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.556+0000 7f0261fd3b80 1 --2- 172.21.15.1:0/1903768077 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d5f6239010 0x55d5f629f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:39.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.556+0000 7f0261fd3b80 1 --2- 172.21.15.1:0/1903768077 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d5f6239930 0x55d5f629f690 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:39.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.556+0000 7f0235600700 0 client.26410 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:29:39.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.556+0000 7f0261fd3b80 1 -- 172.21.15.1:0/1903768077 >> 172.21.15.1:0/1903768077 conn(0x55d5f6198db0 msgr2=0x55d5f6234e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:39.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.556+0000 7f0261fd3b80 1 -- 172.21.15.1:0/1903768077 shutdown_connections 2024-02-02T15:29:39.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.556+0000 7f0261fd3b80 1 -- 172.21.15.1:0/1903768077 wait complete. 2024-02-02T15:29:39.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.556+0000 7f0261fd3b80 20 client.26410 trim_cache size 0 max 16384 2024-02-02T15:29:39.561 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:29:39.561 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-02T15:29:39.561 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2024-02-02T15:29:39.562 INFO:teuthology.orchestra.run.smithi001.stdout: "2ca29b9a-0a1d-43d3-b629-cefb164c30ca": { 2024-02-02T15:29:39.562 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2024-02-02T15:29:39.562 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "2ca29b9a-0a1d-43d3-b629-cefb164c30ca", 2024-02-02T15:29:39.562 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,force" 2024-02-02T15:29:39.562 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:29:39.562 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:29:39.562 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:29:39.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.557+0000 7f024e400700 1 -- 172.21.15.1:0/84187275 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f02280a0710 msgr2=0x7f02280a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:39.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.557+0000 7f024e400700 1 --2- 172.21.15.1:0/84187275 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f02280a0710 0x7f02280a2bd0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f024000f5e0 tx=0x7f0240009920 comp rx=0 tx=0).stop 2024-02-02T15:29:39.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.557+0000 7f024e400700 1 -- 172.21.15.1:0/84187275 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0248103290 msgr2=0x7f02481990e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:29:39.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.557+0000 7f024e400700 1 --2- 172.21.15.1:0/84187275 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0248103290 0x7f02481990e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f023800e9a0 tx=0x7f023800ecb0 comp rx=0 tx=0).stop 2024-02-02T15:29:39.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.557+0000 7f024e400700 1 -- 172.21.15.1:0/84187275 shutdown_connections 2024-02-02T15:29:39.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.557+0000 7f024e400700 1 --2- 172.21.15.1:0/84187275 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f02280a0710 0x7f02280a2bd0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:39.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.557+0000 7f024e400700 1 --2- 172.21.15.1:0/84187275 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f024819da80 0x7f024819df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:39.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.557+0000 7f024e400700 1 --2- 172.21.15.1:0/84187275 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0248103290 0x7f02481990e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:39.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.557+0000 7f024e400700 1 --2- 172.21.15.1:0/84187275 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f02481083b0 0x7f0248199620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:29:39.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.557+0000 7f024e400700 1 -- 172.21.15.1:0/84187275 >> 172.21.15.1:0/84187275 conn(0x7f02480feb30 msgr2=0x7f02480ff5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:29:39.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.557+0000 7f024e400700 1 -- 172.21.15.1:0/84187275 shutdown_connections 2024-02-02T15:29:39.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:29:39.557+0000 7f024e400700 1 -- 172.21.15.1:0/84187275 wait complete. 2024-02-02T15:29:39.576 INFO:tasks.cephfs.filesystem:scrub status for tag:2ca29b9a-0a1d-43d3-b629-cefb164c30ca - {'path': '/', 'tag': '2ca29b9a-0a1d-43d3-b629-cefb164c30ca', 'options': 'recursive,force'} 2024-02-02T15:29:39.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:29:39 smithi142 ceph-mon[45728]: pgmap v82: 129 pgs: 129 active+clean; 67 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 1.3 MiB/s wr, 90 op/s 2024-02-02T15:29:39.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:29:39 smithi001 ceph-mon[35470]: pgmap v82: 129 pgs: 129 active+clean; 67 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 1.3 MiB/s wr, 90 op/s 2024-02-02T15:29:39.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:29:39 smithi071 ceph-mon[44707]: pgmap v82: 129 pgs: 129 active+clean; 67 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 1.3 MiB/s wr, 90 op/s 2024-02-02T15:29:40.222 INFO:tasks.workunit.client.0.smithi001.stdout: 1 104602 7.85 MB/sec warmup 74 sec latency 18.857 ms 2024-02-02T15:29:40.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:29:40 smithi142 ceph-mon[45728]: scrub queued for path: / 2024-02-02T15:29:40.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:29:40 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:29:40.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:29:40 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:29:40.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:29:40 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:29:40.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:29:40 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:29:40.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:29:40 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:29:40.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:29:40 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:29:40.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:29:40 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:29:40.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:29:40 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:29:40.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:29:40 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:29:40.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:29:40 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:29:40.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:29:40 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:29:40.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:29:40 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:29:40.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:29:40 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:29:40.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:29:40 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:29:40.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:29:40 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:29:40.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:29:40 smithi001 ceph-mon[35470]: scrub queued for path: / 2024-02-02T15:29:40.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:29:40 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:29:40.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:29:40 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:29:40.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:29:40 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:29:40.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:29:40 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:29:40.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:29:40 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:29:40.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:29:40 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:29:40.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:29:40 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:29:40.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:29:40 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:29:40.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:29:40 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:29:40.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:29:40 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:29:40.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:29:40 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:29:40.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:29:40 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:29:40.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:29:40 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:29:40.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:29:40 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:29:40.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:29:40 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:29:40.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:29:40 smithi071 ceph-mon[44707]: scrub queued for path: / 2024-02-02T15:29:40.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:29:40 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:29:40.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:29:40 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:29:40.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:29:40 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:29:40.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:29:40 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:29:40.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:29:40 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:29:40.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:29:40 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:29:40.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:29:40 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:29:40.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:29:40 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:29:40.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:29:40 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:29:40.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:29:40 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:29:40.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:29:40 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:29:40.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:29:40 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:29:40.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:29:40 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:29:40.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:29:40 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:29:40.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:29:40 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:29:41.222 INFO:tasks.workunit.client.0.smithi001.stdout: 1 106525 7.91 MB/sec warmup 75 sec latency 229.562 ms 2024-02-02T15:29:41.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:29:41 smithi142 ceph-mon[45728]: pgmap v83: 129 pgs: 129 active+clean; 67 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 1.3 MiB/s wr, 90 op/s 2024-02-02T15:29:41.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:29:41 smithi001 ceph-mon[35470]: pgmap v83: 129 pgs: 129 active+clean; 67 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 1.3 MiB/s wr, 90 op/s 2024-02-02T15:29:41.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:29:41 smithi071 ceph-mon[44707]: pgmap v83: 129 pgs: 129 active+clean; 67 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 1.3 MiB/s wr, 90 op/s 2024-02-02T15:29:42.222 INFO:tasks.workunit.client.0.smithi001.stdout: 1 106525 7.81 MB/sec warmup 76 sec latency 1229.661 ms 2024-02-02T15:29:43.222 INFO:tasks.workunit.client.0.smithi001.stdout: 1 106525 7.70 MB/sec warmup 77 sec latency 2229.821 ms 2024-02-02T15:29:43.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:29:43 smithi142 ceph-mon[45728]: pgmap v84: 129 pgs: 129 active+clean; 68 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 2.0 MiB/s wr, 152 op/s 2024-02-02T15:29:43.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:29:43 smithi001 ceph-mon[35470]: pgmap v84: 129 pgs: 129 active+clean; 68 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 2.0 MiB/s wr, 152 op/s 2024-02-02T15:29:43.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:29:43 smithi071 ceph-mon[44707]: pgmap v84: 129 pgs: 129 active+clean; 68 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 620 KiB/s rd, 2.0 MiB/s wr, 152 op/s 2024-02-02T15:29:44.222 INFO:tasks.workunit.client.0.smithi001.stdout: 1 106541 7.61 MB/sec warmup 78 sec latency 3093.966 ms 2024-02-02T15:29:45.223 INFO:tasks.workunit.client.0.smithi001.stdout: 1 106541 7.51 MB/sec warmup 79 sec latency 1117.976 ms 2024-02-02T15:29:45.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:29:45 smithi142 ceph-mon[45728]: pgmap v85: 129 pgs: 129 active+clean; 68 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 1.5 MiB/s wr, 129 op/s 2024-02-02T15:29:45.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:29:45 smithi001 ceph-mon[35470]: pgmap v85: 129 pgs: 129 active+clean; 68 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 1.5 MiB/s wr, 129 op/s 2024-02-02T15:29:45.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:29:45 smithi071 ceph-mon[44707]: pgmap v85: 129 pgs: 129 active+clean; 68 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 582 KiB/s rd, 1.5 MiB/s wr, 129 op/s 2024-02-02T15:29:46.223 INFO:tasks.workunit.client.0.smithi001.stdout: 1 106541 7.42 MB/sec warmup 80 sec latency 2118.115 ms 2024-02-02T15:29:47.223 INFO:tasks.workunit.client.0.smithi001.stdout: 1 106541 7.32 MB/sec warmup 81 sec latency 3118.279 ms 2024-02-02T15:29:47.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:29:47 smithi142 ceph-mon[45728]: pgmap v86: 129 pgs: 129 active+clean; 75 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 2.2 MiB/s wr, 158 op/s 2024-02-02T15:29:47.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:29:47 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:29:47.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:29:47 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:29:47.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:29:47 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:29:47.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:29:47 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:29:47.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:29:47 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:29:47.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:29:47 smithi001 ceph-mon[35470]: pgmap v86: 129 pgs: 129 active+clean; 75 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 2.2 MiB/s wr, 158 op/s 2024-02-02T15:29:47.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:29:47 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:29:47.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:29:47 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:29:47.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:29:47 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:29:47.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:29:47 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:29:47.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:29:47 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:29:47.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:29:47 smithi071 ceph-mon[44707]: pgmap v86: 129 pgs: 129 active+clean; 75 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 2.2 MiB/s wr, 158 op/s 2024-02-02T15:29:47.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:29:47 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:29:47.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:29:47 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:29:47.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:29:47 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:29:47.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:29:47 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:29:47.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:29:47 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:29:48.223 INFO:tasks.workunit.client.0.smithi001.stdout: 1 106541 7.23 MB/sec warmup 82 sec latency 4118.438 ms 2024-02-02T15:29:49.223 INFO:tasks.workunit.client.0.smithi001.stdout: 1 106683 7.16 MB/sec warmup 83 sec latency 4980.005 ms 2024-02-02T15:29:49.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:29:49 smithi142 ceph-mon[45728]: pgmap v87: 129 pgs: 129 active+clean; 75 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 1.4 MiB/s wr, 90 op/s 2024-02-02T15:29:49.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:29:49 smithi001 ceph-mon[35470]: pgmap v87: 129 pgs: 129 active+clean; 75 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 1.4 MiB/s wr, 90 op/s 2024-02-02T15:29:49.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:29:49 smithi071 ceph-mon[44707]: pgmap v87: 129 pgs: 129 active+clean; 75 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 1.4 MiB/s wr, 90 op/s 2024-02-02T15:29:50.223 INFO:tasks.workunit.client.0.smithi001.stdout: 1 107903 7.14 MB/sec warmup 84 sec latency 21.868 ms 2024-02-02T15:29:51.224 INFO:tasks.workunit.client.0.smithi001.stdout: 1 110005 7.20 MB/sec warmup 85 sec latency 195.230 ms 2024-02-02T15:29:51.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:29:51 smithi142 ceph-mon[45728]: pgmap v88: 129 pgs: 129 active+clean; 75 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 1.4 MiB/s wr, 90 op/s 2024-02-02T15:29:51.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:29:51 smithi001 ceph-mon[35470]: pgmap v88: 129 pgs: 129 active+clean; 75 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 1.4 MiB/s wr, 90 op/s 2024-02-02T15:29:51.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:29:51 smithi071 ceph-mon[44707]: pgmap v88: 129 pgs: 129 active+clean; 75 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 1.4 MiB/s wr, 90 op/s 2024-02-02T15:29:52.224 INFO:tasks.workunit.client.0.smithi001.stdout: 1 110005 7.12 MB/sec warmup 86 sec latency 1195.408 ms 2024-02-02T15:29:53.224 INFO:tasks.workunit.client.0.smithi001.stdout: 1 110005 7.04 MB/sec warmup 87 sec latency 2195.561 ms 2024-02-02T15:29:53.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:29:53 smithi142 ceph-mon[45728]: pgmap v89: 129 pgs: 129 active+clean; 68 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 2.2 MiB/s wr, 158 op/s 2024-02-02T15:29:53.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:29:53 smithi001 ceph-mon[35470]: pgmap v89: 129 pgs: 129 active+clean; 68 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 2.2 MiB/s wr, 158 op/s 2024-02-02T15:29:53.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:29:53 smithi071 ceph-mon[44707]: pgmap v89: 129 pgs: 129 active+clean; 68 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 2.2 MiB/s wr, 158 op/s 2024-02-02T15:29:54.224 INFO:tasks.workunit.client.0.smithi001.stdout: 1 110156 6.96 MB/sec warmup 88 sec latency 3050.474 ms 2024-02-02T15:29:55.224 INFO:tasks.workunit.client.0.smithi001.stdout: 1 110156 6.88 MB/sec warmup 89 sec latency 1022.718 ms 2024-02-02T15:29:55.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:29:55 smithi142 ceph-mon[45728]: pgmap v90: 129 pgs: 129 active+clean; 68 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 1.4 MiB/s wr, 96 op/s 2024-02-02T15:29:55.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:29:55 smithi001 ceph-mon[35470]: pgmap v90: 129 pgs: 129 active+clean; 68 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 1.4 MiB/s wr, 96 op/s 2024-02-02T15:29:55.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:29:55 smithi071 ceph-mon[44707]: pgmap v90: 129 pgs: 129 active+clean; 68 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 1.4 MiB/s wr, 96 op/s 2024-02-02T15:29:56.224 INFO:tasks.workunit.client.0.smithi001.stdout: 1 110156 6.81 MB/sec warmup 90 sec latency 2022.829 ms 2024-02-02T15:29:57.224 INFO:tasks.workunit.client.0.smithi001.stdout: 1 110156 6.73 MB/sec warmup 91 sec latency 3022.977 ms 2024-02-02T15:29:57.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:29:57 smithi142 ceph-mon[45728]: pgmap v91: 129 pgs: 129 active+clean; 70 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 1.6 MiB/s wr, 110 op/s 2024-02-02T15:29:57.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:29:57 smithi001 ceph-mon[35470]: pgmap v91: 129 pgs: 129 active+clean; 70 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 1.6 MiB/s wr, 110 op/s 2024-02-02T15:29:57.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:29:57 smithi071 ceph-mon[44707]: pgmap v91: 129 pgs: 129 active+clean; 70 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 1.6 MiB/s wr, 110 op/s 2024-02-02T15:29:58.224 INFO:tasks.workunit.client.0.smithi001.stdout: 1 110156 6.66 MB/sec warmup 92 sec latency 4023.120 ms 2024-02-02T15:29:59.224 INFO:tasks.workunit.client.0.smithi001.stdout: 1 110276 6.60 MB/sec warmup 93 sec latency 4878.112 ms 2024-02-02T15:29:59.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:29:59 smithi142 ceph-mon[45728]: pgmap v92: 129 pgs: 129 active+clean; 70 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 972 KiB/s wr, 81 op/s 2024-02-02T15:29:59.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:29:59 smithi001 ceph-mon[35470]: pgmap v92: 129 pgs: 129 active+clean; 70 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 972 KiB/s wr, 81 op/s 2024-02-02T15:29:59.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:29:59 smithi071 ceph-mon[44707]: pgmap v92: 129 pgs: 129 active+clean; 70 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 972 KiB/s wr, 81 op/s 2024-02-02T15:30:00.224 INFO:tasks.workunit.client.0.smithi001.stdout: 1 111652 6.59 MB/sec warmup 94 sec latency 13.370 ms 2024-02-02T15:30:00.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:00 smithi142 ceph-mon[45728]: overall HEALTH_OK 2024-02-02T15:30:00.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:00 smithi001 ceph-mon[35470]: overall HEALTH_OK 2024-02-02T15:30:00.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:00 smithi071 ceph-mon[44707]: overall HEALTH_OK 2024-02-02T15:30:01.225 INFO:tasks.workunit.client.0.smithi001.stdout: 1 113549 6.64 MB/sec warmup 95 sec latency 231.191 ms 2024-02-02T15:30:01.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:01 smithi142 ceph-mon[45728]: pgmap v93: 129 pgs: 129 active+clean; 70 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 972 KiB/s wr, 81 op/s 2024-02-02T15:30:01.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:01 smithi001 ceph-mon[35470]: pgmap v93: 129 pgs: 129 active+clean; 70 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 972 KiB/s wr, 81 op/s 2024-02-02T15:30:01.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:01 smithi071 ceph-mon[44707]: pgmap v93: 129 pgs: 129 active+clean; 70 MiB data, 607 MiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 972 KiB/s wr, 81 op/s 2024-02-02T15:30:02.225 INFO:tasks.workunit.client.0.smithi001.stdout: 1 113549 6.58 MB/sec warmup 96 sec latency 1231.329 ms 2024-02-02T15:30:03.225 INFO:tasks.workunit.client.0.smithi001.stdout: 1 113549 6.51 MB/sec warmup 97 sec latency 2231.430 ms 2024-02-02T15:30:03.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:03 smithi071 ceph-mon[44707]: pgmap v94: 129 pgs: 129 active+clean; 69 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 1.4 MiB/s wr, 143 op/s 2024-02-02T15:30:03.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:03 smithi001 ceph-mon[35470]: pgmap v94: 129 pgs: 129 active+clean; 69 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 1.4 MiB/s wr, 143 op/s 2024-02-02T15:30:03.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:03 smithi142 ceph-mon[45728]: pgmap v94: 129 pgs: 129 active+clean; 69 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 1.4 MiB/s wr, 143 op/s 2024-02-02T15:30:04.225 INFO:tasks.workunit.client.0.smithi001.stdout: 1 113630 6.44 MB/sec warmup 98 sec latency 3086.327 ms 2024-02-02T15:30:05.225 INFO:tasks.workunit.client.0.smithi001.stdout: 1 113630 6.38 MB/sec warmup 99 sec latency 1101.104 ms 2024-02-02T15:30:05.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:05 smithi001 ceph-mon[35470]: pgmap v95: 129 pgs: 129 active+clean; 69 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 693 KiB/s wr, 76 op/s 2024-02-02T15:30:05.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:05 smithi071 ceph-mon[44707]: pgmap v95: 129 pgs: 129 active+clean; 69 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 693 KiB/s wr, 76 op/s 2024-02-02T15:30:05.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:05 smithi142 ceph-mon[45728]: pgmap v95: 129 pgs: 129 active+clean; 69 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 693 KiB/s wr, 76 op/s 2024-02-02T15:30:06.225 INFO:tasks.workunit.client.0.smithi001.stdout: 1 113630 6.31 MB/sec warmup 100 sec latency 2101.236 ms 2024-02-02T15:30:07.225 INFO:tasks.workunit.client.0.smithi001.stdout: 1 113630 6.25 MB/sec warmup 101 sec latency 3101.393 ms 2024-02-02T15:30:07.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:07 smithi001 ceph-mon[35470]: pgmap v96: 129 pgs: 129 active+clean; 69 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 708 KiB/s wr, 87 op/s 2024-02-02T15:30:07.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:07 smithi071 ceph-mon[44707]: pgmap v96: 129 pgs: 129 active+clean; 69 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 708 KiB/s wr, 87 op/s 2024-02-02T15:30:07.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:07 smithi142 ceph-mon[45728]: pgmap v96: 129 pgs: 129 active+clean; 69 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 708 KiB/s wr, 87 op/s 2024-02-02T15:30:08.226 INFO:tasks.workunit.client.0.smithi001.stdout: 1 113630 6.19 MB/sec warmup 102 sec latency 4101.540 ms 2024-02-02T15:30:09.226 INFO:tasks.workunit.client.0.smithi001.stdout: 1 113757 6.14 MB/sec warmup 103 sec latency 4973.020 ms 2024-02-02T15:30:09.577 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-02T15:30:09.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:09 smithi001 ceph-mon[35470]: pgmap v97: 129 pgs: 129 active+clean; 69 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 497 KiB/s wr, 73 op/s 2024-02-02T15:30:09.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:09 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:30:09.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:09 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:30:09.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:09 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:30:09.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:09 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:30:09.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:09 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:30:09.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:09 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:30:09.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:09 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:30:09.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.795+0000 7f85cfc00700 1 -- 172.21.15.1:0/633822051 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85c8101000 msgr2=0x7f85c81013e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:09.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.795+0000 7f85cfc00700 1 --2- 172.21.15.1:0/633822051 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85c8101000 0x7f85c81013e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f85b8009990 tx=0x7f85b8009ca0 comp rx=0 tx=0).stop 2024-02-02T15:30:09.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.795+0000 7f85cfc00700 1 -- 172.21.15.1:0/633822051 shutdown_connections 2024-02-02T15:30:09.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.795+0000 7f85cfc00700 1 --2- 172.21.15.1:0/633822051 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f85c8106120 0x7f85c810dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:09.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.795+0000 7f85cfc00700 1 --2- 172.21.15.1:0/633822051 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85c8101000 0x7f85c81013e0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:09.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.795+0000 7f85cfc00700 1 --2- 172.21.15.1:0/633822051 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85c81019b0 0x7f85c81059f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:09.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.795+0000 7f85cfc00700 1 -- 172.21.15.1:0/633822051 >> 172.21.15.1:0/633822051 conn(0x7f85c80fc840 msgr2=0x7f85c80fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:09.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.795+0000 7f85cfc00700 1 -- 172.21.15.1:0/633822051 shutdown_connections 2024-02-02T15:30:09.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.795+0000 7f85cfc00700 1 -- 172.21.15.1:0/633822051 wait complete. 2024-02-02T15:30:09.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.797+0000 7f85cfc00700 1 Processor -- start 2024-02-02T15:30:09.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.797+0000 7f85cfc00700 1 -- start start 2024-02-02T15:30:09.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.798+0000 7f85cfc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85c8101000 0x7f85c8073040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:09.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.798+0000 7f85cfc00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f85c8106120 0x7f85c8073580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:09.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.798+0000 7f85cfc00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85c8074d50 0x7f85c81a9800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:09.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.798+0000 7f85cfc00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c8073dd0 con 0x7f85c8106120 2024-02-02T15:30:09.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.798+0000 7f85cfc00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c81a9d40 con 0x7f85c8074d50 2024-02-02T15:30:09.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.798+0000 7f85cfc00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c81a9e80 con 0x7f85c8101000 2024-02-02T15:30:09.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.798+0000 7f85cd200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85c8101000 0x7f85c8073040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:09.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.798+0000 7f85cd200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85c8101000 0x7f85c8073040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:43052/0 (socket says 172.21.15.1:43052) 2024-02-02T15:30:09.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.798+0000 7f85cd200700 1 -- 172.21.15.1:0/3181920675 learned_addr learned my addr 172.21.15.1:0/3181920675 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:30:09.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.798+0000 7f85c7e00700 1 --2- 172.21.15.1:0/3181920675 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f85c8106120 0x7f85c8073580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:09.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.798+0000 7f85cdc00700 1 --2- 172.21.15.1:0/3181920675 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85c8074d50 0x7f85c81a9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:09.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.798+0000 7f85cd200700 1 -- 172.21.15.1:0/3181920675 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85c8074d50 msgr2=0x7f85c81a9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:09.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.798+0000 7f85cd200700 1 --2- 172.21.15.1:0/3181920675 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85c8074d50 0x7f85c81a9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:09.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.798+0000 7f85cd200700 1 -- 172.21.15.1:0/3181920675 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f85c8106120 msgr2=0x7f85c8073580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:09.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.798+0000 7f85cd200700 1 --2- 172.21.15.1:0/3181920675 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f85c8106120 0x7f85c8073580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:09.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.798+0000 7f85cd200700 1 -- 172.21.15.1:0/3181920675 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85b80095f0 con 0x7f85c8101000 2024-02-02T15:30:09.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.798+0000 7f85cdc00700 1 --2- 172.21.15.1:0/3181920675 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85c8074d50 0x7f85c81a9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:30:09.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.799+0000 7f85cd200700 1 --2- 172.21.15.1:0/3181920675 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85c8101000 0x7f85c8073040 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f85b800f690 tx=0x7f85b800f6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:09.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.799+0000 7f85c5600700 1 -- 172.21.15.1:0/3181920675 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b801d070 con 0x7f85c8101000 2024-02-02T15:30:09.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.799+0000 7f85c5600700 1 -- 172.21.15.1:0/3181920675 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f85b8005a00 con 0x7f85c8101000 2024-02-02T15:30:09.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.799+0000 7f85c5600700 1 -- 172.21.15.1:0/3181920675 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b8005560 con 0x7f85c8101000 2024-02-02T15:30:09.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.799+0000 7f85cfc00700 1 -- 172.21.15.1:0/3181920675 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85c81aa080 con 0x7f85c8101000 2024-02-02T15:30:09.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.799+0000 7f85cfc00700 1 -- 172.21.15.1:0/3181920675 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85c81aa530 con 0x7f85c8101000 2024-02-02T15:30:09.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.803+0000 7f85c5600700 1 -- 172.21.15.1:0/3181920675 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 27) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f85b8004800 con 0x7f85c8101000 2024-02-02T15:30:09.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.804+0000 7f85e374fb80 1 Processor -- start 2024-02-02T15:30:09.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.804+0000 7f85e374fb80 1 -- start start 2024-02-02T15:30:09.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.804+0000 7f85c5600700 1 --2- 172.21.15.1:0/3181920675 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f85a80a07e0 0x7f85a80a2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:09.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.805+0000 7f85c5600700 1 -- 172.21.15.1:0/3181920675 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f85b80cd100 con 0x7f85c8101000 2024-02-02T15:30:09.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.805+0000 7f85e374fb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb69696010 0x55bb696963f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:09.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.805+0000 7f85e374fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bb69696930 0x55bb696a12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:09.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.805+0000 7f85cdc00700 1 --2- 172.21.15.1:0/3181920675 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f85a80a07e0 0x7f85a80a2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:09.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.805+0000 7f85e374fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bb696a17e0 0x55bb696a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:09.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.805+0000 7f85e374fb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb6960a4c0 con 0x55bb69696010 2024-02-02T15:30:09.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.805+0000 7f85e374fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb695eeaa0 con 0x55bb69696930 2024-02-02T15:30:09.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.805+0000 7f85e374fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb695ec460 con 0x55bb696a17e0 2024-02-02T15:30:09.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.805+0000 7f85cdc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bb696a17e0 0x55bb696a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:09.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.805+0000 7f85cdc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bb696a17e0 0x55bb696a3bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:43064/0 (socket says 172.21.15.1:43064) 2024-02-02T15:30:09.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.805+0000 7f85cdc00700 1 -- 172.21.15.1:0/2345857346 learned_addr learned my addr 172.21.15.1:0/2345857346 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:30:09.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.805+0000 7f85cd200700 1 --2- 172.21.15.1:0/2345857346 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb69696010 0x55bb696963f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:09.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.805+0000 7f85c7e00700 1 --2- 172.21.15.1:0/2345857346 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bb69696930 0x55bb696a12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:09.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.806+0000 7f85cd200700 1 -- 172.21.15.1:0/2345857346 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bb696a17e0 msgr2=0x55bb696a3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:09.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.806+0000 7f85cd200700 1 --2- 172.21.15.1:0/2345857346 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bb696a17e0 0x55bb696a3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:09.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.806+0000 7f85cd200700 1 -- 172.21.15.1:0/2345857346 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bb69696930 msgr2=0x55bb696a12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:09.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.806+0000 7f85cd200700 1 --2- 172.21.15.1:0/2345857346 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bb69696930 0x55bb696a12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:09.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.806+0000 7f85cd200700 1 -- 172.21.15.1:0/2345857346 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb694693b0 con 0x55bb69696010 2024-02-02T15:30:09.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.806+0000 7f85cdc00700 1 --2- 172.21.15.1:0/2345857346 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bb696a17e0 0x55bb696a3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:30:09.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.806+0000 7f85c7e00700 1 --2- 172.21.15.1:0/2345857346 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bb69696930 0x55bb696a12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:30:09.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.806+0000 7f85cdc00700 1 --2- 172.21.15.1:0/3181920675 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f85a80a07e0 0x7f85a80a2ca0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f85c0007c50 tx=0x7f85c0009d10 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:09.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.806+0000 7f85cd200700 1 --2- 172.21.15.1:0/2345857346 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb69696010 0x55bb696963f0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f85b80ca4a0 tx=0x7f85b8099e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64710efb635a74d3 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:09.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.806+0000 7f85be000700 1 -- 172.21.15.1:0/2345857346 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b809aa00 con 0x55bb69696010 2024-02-02T15:30:09.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.807+0000 7f85be000700 1 -- 172.21.15.1:0/2345857346 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f85b809a300 con 0x55bb69696010 2024-02-02T15:30:09.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.807+0000 7f85be000700 1 -- 172.21.15.1:0/2345857346 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b8004ab0 con 0x55bb69696010 2024-02-02T15:30:09.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.807+0000 7f85e374fb80 1 -- 172.21.15.1:0/2345857346 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb69696010 msgr2=0x55bb696963f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:09.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.808+0000 7f85e374fb80 1 --2- 172.21.15.1:0/2345857346 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb69696010 0x55bb696963f0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f85b80ca4a0 tx=0x7f85b8099e40 comp rx=0 tx=0).stop 2024-02-02T15:30:09.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.808+0000 7f85e374fb80 1 -- 172.21.15.1:0/2345857346 shutdown_connections 2024-02-02T15:30:09.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.808+0000 7f85e374fb80 1 --2- 172.21.15.1:0/2345857346 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bb696a17e0 0x55bb696a3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:09.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.808+0000 7f85e374fb80 1 --2- 172.21.15.1:0/2345857346 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb69696010 0x55bb696963f0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:09.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.808+0000 7f85e374fb80 1 --2- 172.21.15.1:0/2345857346 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bb69696930 0x55bb696a12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:09.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.808+0000 7f85e374fb80 1 -- 172.21.15.1:0/2345857346 >> 172.21.15.1:0/2345857346 conn(0x55bb695f5db0 msgr2=0x55bb696923b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:09.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.808+0000 7f85e374fb80 1 -- 172.21.15.1:0/2345857346 shutdown_connections 2024-02-02T15:30:09.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.808+0000 7f85e374fb80 1 -- 172.21.15.1:0/2345857346 wait complete. 2024-02-02T15:30:09.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.809+0000 7f85e374fb80 1 -- start start 2024-02-02T15:30:09.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.809+0000 7f85e374fb80 1 Processor -- start 2024-02-02T15:30:09.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.809+0000 7f85e374fb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb69696010 0x55bb696a1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:09.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.810+0000 7f85e374fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bb69696930 0x55bb696a2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:09.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.810+0000 7f85e374fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bb696a17e0 0x55bb696a2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:09.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.810+0000 7f85e374fb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb695e7d50 con 0x55bb69696010 2024-02-02T15:30:09.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.810+0000 7f85e374fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb69583d80 con 0x55bb696a17e0 2024-02-02T15:30:09.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.810+0000 7f85e374fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb69583a20 con 0x55bb69696930 2024-02-02T15:30:09.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.810+0000 7f85e374fb80 10 client.0 _ll_register_callbacks cb 0x55bb69585290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:30:09.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.810+0000 7f85cdc00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bb696a17e0 0x55bb696a2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:09.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.810+0000 7f85c7e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bb69696930 0x55bb696a2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:09.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.810+0000 7f85cdc00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bb696a17e0 0x55bb696a2640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59842/0 (socket says 172.21.15.1:59842) 2024-02-02T15:30:09.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.810+0000 7f85cdc00700 1 -- 172.21.15.1:0/1851329621 learned_addr learned my addr 172.21.15.1:0/1851329621 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:30:09.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.810+0000 7f85cd200700 1 --2- 172.21.15.1:0/1851329621 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb69696010 0x55bb696a1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:09.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.810+0000 7f85cdc00700 1 -- 172.21.15.1:0/1851329621 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bb69696930 msgr2=0x55bb696a2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:09.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.810+0000 7f85cdc00700 1 --2- 172.21.15.1:0/1851329621 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bb69696930 0x55bb696a2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:09.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.810+0000 7f85cdc00700 1 -- 172.21.15.1:0/1851329621 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb69696010 msgr2=0x55bb696a1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:09.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.810+0000 7f85cdc00700 1 --2- 172.21.15.1:0/1851329621 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb69696010 0x55bb696a1bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:09.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.810+0000 7f85cdc00700 1 -- 172.21.15.1:0/1851329621 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85c0007d90 con 0x55bb696a17e0 2024-02-02T15:30:09.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.811+0000 7f85cdc00700 1 --2- 172.21.15.1:0/1851329621 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bb696a17e0 0x55bb696a2640 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f85c0008fd0 tx=0x7f85c0004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2caf1c5a0620f126 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:09.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.811+0000 7f85b6a00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:30:09.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.811+0000 7f85b6a00700 1 -- 172.21.15.1:0/1851329621 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c0004850 con 0x55bb696a17e0 2024-02-02T15:30:09.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.811+0000 7f85b6a00700 1 -- 172.21.15.1:0/1851329621 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f85c0032030 con 0x55bb696a17e0 2024-02-02T15:30:09.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.811+0000 7f85e374fb80 1 -- 172.21.15.1:0/1851329621 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bb6968a0f0 con 0x55bb696a17e0 2024-02-02T15:30:09.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.811+0000 7f85b6a00700 1 -- 172.21.15.1:0/1851329621 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f85c001b480 con 0x55bb696a17e0 2024-02-02T15:30:09.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.811+0000 7f85e374fb80 10 client.26432 fetch_fsmap learned FSMap version 31 2024-02-02T15:30:09.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.812+0000 7f85e374fb80 1 -- 172.21.15.1:0/1851329621 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f85c81aa530 con 0x55bb696a17e0 2024-02-02T15:30:09.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.812+0000 7f85b6a00700 1 -- 172.21.15.1:0/1851329621 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bb6968a0f0 con 0x55bb696a17e0 2024-02-02T15:30:09.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.812+0000 7f85b6a00700 1 -- 172.21.15.1:0/1851329621 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 31) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f85c0036430 con 0x55bb696a17e0 2024-02-02T15:30:09.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.812+0000 7f85e374fb80 10 client.26432 fetch_fsmap finished waiting for FSMap version 31 2024-02-02T15:30:09.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.812+0000 7f85e374fb80 10 client.26432 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:30:09.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.812+0000 7f85e374fb80 20 client.26432 populate_metadata read hostname 'smithi001' 2024-02-02T15:30:09.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.812+0000 7f85e374fb80 1 --2- 172.21.15.1:0/1851329621 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55bb696b0ef0 0x55bb696b3310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:09.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.812+0000 7f85e374fb80 4 client.26432 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:30:09.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.812+0000 7f85e374fb80 1 -- 172.21.15.1:0/1851329621 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bb695e0dd0 con 0x55bb696b0ef0 2024-02-02T15:30:09.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.812+0000 7f85cd200700 1 --2- 172.21.15.1:0/1851329621 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55bb696b0ef0 0x55bb696b3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:09.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.813+0000 7f85cd200700 1 --2- 172.21.15.1:0/1851329621 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55bb696b0ef0 0x55bb696b3310 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d037362c70c7a09 server_cookie=3f649dfc4feea4d0 in_seq=0 out_seq=0 2024-02-02T15:30:09.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.813+0000 7f85b6a00700 10 client.26432 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:30:09.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.813+0000 7f85b6a00700 1 -- 172.21.15.1:0/1851329621 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55bb695e0dd0 con 0x55bb696b0ef0 2024-02-02T15:30:09.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.813+0000 7f85b6a00700 10 client.26432 handle_command_reply: tid=0 2024-02-02T15:30:09.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.813+0000 7f85e374fb80 1 client.26432 shutdown 2024-02-02T15:30:09.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.814+0000 7f85e374fb80 1 -- 172.21.15.1:0/1851329621 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bb696a17e0 msgr2=0x55bb696a2640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:09.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.814+0000 7f85e374fb80 1 --2- 172.21.15.1:0/1851329621 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bb696a17e0 0x55bb696a2640 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f85c0008fd0 tx=0x7f85c0004b40 comp rx=0 tx=0).stop 2024-02-02T15:30:09.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.814+0000 7f85e374fb80 1 -- 172.21.15.1:0/1851329621 shutdown_connections 2024-02-02T15:30:09.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.814+0000 7f85e374fb80 1 --2- 172.21.15.1:0/1851329621 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55bb696b0ef0 0x55bb696b3310 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:09.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.814+0000 7f85e374fb80 1 --2- 172.21.15.1:0/1851329621 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bb696a17e0 0x55bb696a2640 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:09.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.814+0000 7f85e374fb80 1 --2- 172.21.15.1:0/1851329621 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb69696010 0x55bb696a1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:09.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.814+0000 7f85e374fb80 1 --2- 172.21.15.1:0/1851329621 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bb69696930 0x55bb696a2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:09.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.814+0000 7f85b6a00700 0 client.26432 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:30:09.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.814+0000 7f85e374fb80 1 -- 172.21.15.1:0/1851329621 >> 172.21.15.1:0/1851329621 conn(0x55bb695f5db0 msgr2=0x55bb69691fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:09.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.814+0000 7f85e374fb80 1 -- 172.21.15.1:0/1851329621 shutdown_connections 2024-02-02T15:30:09.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.814+0000 7f85e374fb80 1 -- 172.21.15.1:0/1851329621 wait complete. 2024-02-02T15:30:09.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.814+0000 7f85e374fb80 20 client.26432 trim_cache size 0 max 16384 2024-02-02T15:30:09.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.838+0000 7f85e374fb80 1 Processor -- start 2024-02-02T15:30:09.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.838+0000 7f85e374fb80 1 -- start start 2024-02-02T15:30:09.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.839+0000 7f85e374fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bb69696010 0x55bb6969b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:09.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.839+0000 7f85e374fb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb69696930 0x55bb696ad150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:09.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.839+0000 7f85e374fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bb696a17e0 0x55bb69690b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:09.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.839+0000 7f85e374fb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb6946b6a0 con 0x55bb69696930 2024-02-02T15:30:09.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.839+0000 7f85e374fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb69697830 con 0x55bb69696010 2024-02-02T15:30:09.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.839+0000 7f85e374fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb696937e0 con 0x55bb696a17e0 2024-02-02T15:30:09.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.839+0000 7f85cd200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bb69696010 0x55bb6969b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:09.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.839+0000 7f85cd200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bb69696010 0x55bb6969b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59846/0 (socket says 172.21.15.1:59846) 2024-02-02T15:30:09.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.839+0000 7f85cd200700 1 -- 172.21.15.1:0/2692306938 learned_addr learned my addr 172.21.15.1:0/2692306938 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:30:09.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.839+0000 7f85cdc00700 1 --2- 172.21.15.1:0/2692306938 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bb696a17e0 0x55bb69690b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:09.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.839+0000 7f85c7e00700 1 --2- 172.21.15.1:0/2692306938 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb69696930 0x55bb696ad150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:09.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.839+0000 7f85cd200700 1 -- 172.21.15.1:0/2692306938 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bb696a17e0 msgr2=0x55bb69690b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:09.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.839+0000 7f85cd200700 1 --2- 172.21.15.1:0/2692306938 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bb696a17e0 0x55bb69690b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:09.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.839+0000 7f85cd200700 1 -- 172.21.15.1:0/2692306938 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb69696930 msgr2=0x55bb696ad150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:09.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.839+0000 7f85cd200700 1 --2- 172.21.15.1:0/2692306938 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb69696930 0x55bb696ad150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:09.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.839+0000 7f85cd200700 1 -- 172.21.15.1:0/2692306938 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb69415b60 con 0x55bb69696010 2024-02-02T15:30:09.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.839+0000 7f85cd200700 1 --2- 172.21.15.1:0/2692306938 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bb69696010 0x55bb6969b250 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f85b809a290 tx=0x7f85b800fec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd702eb86a595e76 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:09.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.840+0000 7f85bea00700 1 -- 172.21.15.1:0/2692306938 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b809e130 con 0x55bb69696010 2024-02-02T15:30:09.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.840+0000 7f85bea00700 1 -- 172.21.15.1:0/2692306938 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f85b80a22e0 con 0x55bb69696010 2024-02-02T15:30:09.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.840+0000 7f85bea00700 1 -- 172.21.15.1:0/2692306938 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b8004ab0 con 0x55bb69696010 2024-02-02T15:30:09.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.841+0000 7f85e374fb80 1 -- 172.21.15.1:0/2692306938 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bb69696010 msgr2=0x55bb6969b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:09.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.841+0000 7f85e374fb80 1 --2- 172.21.15.1:0/2692306938 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bb69696010 0x55bb6969b250 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f85b809a290 tx=0x7f85b800fec0 comp rx=0 tx=0).stop 2024-02-02T15:30:09.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.841+0000 7f85e374fb80 1 -- 172.21.15.1:0/2692306938 shutdown_connections 2024-02-02T15:30:09.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.841+0000 7f85e374fb80 1 --2- 172.21.15.1:0/2692306938 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bb696a17e0 0x55bb69690b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:09.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.841+0000 7f85e374fb80 1 --2- 172.21.15.1:0/2692306938 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bb69696010 0x55bb6969b250 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:09.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.841+0000 7f85e374fb80 1 --2- 172.21.15.1:0/2692306938 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb69696930 0x55bb696ad150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:09.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.841+0000 7f85e374fb80 1 -- 172.21.15.1:0/2692306938 >> 172.21.15.1:0/2692306938 conn(0x55bb695f5db0 msgr2=0x55bb696a2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:09.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.841+0000 7f85e374fb80 1 -- 172.21.15.1:0/2692306938 shutdown_connections 2024-02-02T15:30:09.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.841+0000 7f85e374fb80 1 -- 172.21.15.1:0/2692306938 wait complete. 2024-02-02T15:30:09.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.841+0000 7f85e374fb80 1 -- start start 2024-02-02T15:30:09.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.842+0000 7f85e374fb80 1 Processor -- start 2024-02-02T15:30:09.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.842+0000 7f85e374fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bb69696010 0x55bb696a2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:09.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.842+0000 7f85e374fb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb69696930 0x55bb696a32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:09.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.842+0000 7f85e374fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bb696a17e0 0x55bb69701a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:09.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.842+0000 7f85e374fb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb696a3a10 con 0x55bb69696930 2024-02-02T15:30:09.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.842+0000 7f85e374fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb69701f40 con 0x55bb696a17e0 2024-02-02T15:30:09.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.842+0000 7f85e374fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb697020d0 con 0x55bb69696010 2024-02-02T15:30:09.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.842+0000 7f85e374fb80 10 client.0 _ll_register_callbacks cb 0x55bb69585290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:30:09.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.843+0000 7f85cd200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bb69696010 0x55bb696a2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:09.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.843+0000 7f85cdc00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bb696a17e0 0x55bb69701a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:09.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.843+0000 7f85cd200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bb69696010 0x55bb696a2d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:43092/0 (socket says 172.21.15.1:43092) 2024-02-02T15:30:09.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.843+0000 7f85cd200700 1 -- 172.21.15.1:0/2150347360 learned_addr learned my addr 172.21.15.1:0/2150347360 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:30:09.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.843+0000 7f85c7e00700 1 --2- 172.21.15.1:0/2150347360 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb69696930 0x55bb696a32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:09.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.843+0000 7f85cd200700 1 -- 172.21.15.1:0/2150347360 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bb696a17e0 msgr2=0x55bb69701a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:09.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.843+0000 7f85cd200700 1 --2- 172.21.15.1:0/2150347360 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bb696a17e0 0x55bb69701a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:09.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.843+0000 7f85cd200700 1 -- 172.21.15.1:0/2150347360 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb69696930 msgr2=0x55bb696a32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:09.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.843+0000 7f85cd200700 1 --2- 172.21.15.1:0/2150347360 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb69696930 0x55bb696a32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:09.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.843+0000 7f85cd200700 1 -- 172.21.15.1:0/2150347360 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85b80b1040 con 0x55bb69696010 2024-02-02T15:30:09.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.843+0000 7f85cdc00700 1 --2- 172.21.15.1:0/2150347360 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bb696a17e0 0x55bb69701a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:30:09.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.843+0000 7f85cd200700 1 --2- 172.21.15.1:0/2150347360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bb69696010 0x55bb696a2d90 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f85b809a000 tx=0x7f85b80ca4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a5f4900c1e8b51f server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:09.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.843+0000 7f85b6a00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:30:09.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.843+0000 7f85b6a00700 1 -- 172.21.15.1:0/2150347360 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b8099e40 con 0x55bb69696010 2024-02-02T15:30:09.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.843+0000 7f85b6a00700 1 -- 172.21.15.1:0/2150347360 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f85b809e130 con 0x55bb69696010 2024-02-02T15:30:09.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.844+0000 7f85e374fb80 1 -- 172.21.15.1:0/2150347360 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f85b809a300 con 0x55bb69696010 2024-02-02T15:30:09.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.844+0000 7f85b6a00700 1 -- 172.21.15.1:0/2150347360 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f85b80a00c0 con 0x55bb69696010 2024-02-02T15:30:09.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.844+0000 7f85e374fb80 10 client.26428 fetch_fsmap learned FSMap version 31 2024-02-02T15:30:09.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.844+0000 7f85e374fb80 1 -- 172.21.15.1:0/2150347360 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55bb695adea0 con 0x55bb69696010 2024-02-02T15:30:09.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.844+0000 7f85b6a00700 1 -- 172.21.15.1:0/2150347360 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f85b809a300 con 0x55bb69696010 2024-02-02T15:30:09.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.844+0000 7f85b6a00700 1 -- 172.21.15.1:0/2150347360 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 31) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f85b80997f0 con 0x55bb69696010 2024-02-02T15:30:09.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.844+0000 7f85e374fb80 10 client.26428 fetch_fsmap finished waiting for FSMap version 31 2024-02-02T15:30:09.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.844+0000 7f85e374fb80 10 client.26428 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:30:09.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.844+0000 7f85e374fb80 20 client.26428 populate_metadata read hostname 'smithi001' 2024-02-02T15:30:09.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.844+0000 7f85e374fb80 1 --2- 172.21.15.1:0/2150347360 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55bb69703360 0x55bb69705780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:09.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.844+0000 7f85e374fb80 4 client.26428 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2024-02-02T15:30:09.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.844+0000 7f85e374fb80 1 -- 172.21.15.1:0/2150347360 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55bb695a6f50 con 0x55bb69703360 2024-02-02T15:30:09.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.844+0000 7f85c7e00700 1 --2- 172.21.15.1:0/2150347360 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55bb69703360 0x55bb69705780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:09.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.845+0000 7f85c7e00700 1 --2- 172.21.15.1:0/2150347360 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55bb69703360 0x55bb69705780 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82a4e8cda6cec574 server_cookie=c68d2bce223c6fb3 in_seq=0 out_seq=0 2024-02-02T15:30:09.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.845+0000 7f85b6a00700 10 client.26428 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:30:09.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.845+0000 7f85b6a00700 1 -- 172.21.15.1:0/2150347360 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55bb695a6f50 con 0x55bb69703360 2024-02-02T15:30:09.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.845+0000 7f85b6a00700 10 client.26428 handle_command_reply: tid=0 2024-02-02T15:30:09.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.845+0000 7f85e374fb80 1 client.26428 shutdown 2024-02-02T15:30:09.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.846+0000 7f85e374fb80 1 -- 172.21.15.1:0/2150347360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bb69696010 msgr2=0x55bb696a2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:09.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.846+0000 7f85e374fb80 1 --2- 172.21.15.1:0/2150347360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bb69696010 0x55bb696a2d90 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f85b809a000 tx=0x7f85b80ca4a0 comp rx=0 tx=0).stop 2024-02-02T15:30:09.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.846+0000 7f85e374fb80 1 -- 172.21.15.1:0/2150347360 shutdown_connections 2024-02-02T15:30:09.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.846+0000 7f85e374fb80 1 --2- 172.21.15.1:0/2150347360 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55bb69703360 0x55bb69705780 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:09.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.846+0000 7f85e374fb80 1 --2- 172.21.15.1:0/2150347360 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bb696a17e0 0x55bb69701a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:09.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.846+0000 7f85e374fb80 1 --2- 172.21.15.1:0/2150347360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bb69696010 0x55bb696a2d90 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:09.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.846+0000 7f85b6a00700 0 client.26428 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:30:09.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.846+0000 7f85e374fb80 1 --2- 172.21.15.1:0/2150347360 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bb69696930 0x55bb696a32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:09.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.846+0000 7f85e374fb80 1 -- 172.21.15.1:0/2150347360 >> 172.21.15.1:0/2150347360 conn(0x55bb695f5db0 msgr2=0x55bb696a1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:09.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.846+0000 7f85e374fb80 1 -- 172.21.15.1:0/2150347360 shutdown_connections 2024-02-02T15:30:09.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.846+0000 7f85e374fb80 1 -- 172.21.15.1:0/2150347360 wait complete. 2024-02-02T15:30:09.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.846+0000 7f85e374fb80 20 client.26428 trim_cache size 0 max 16384 2024-02-02T15:30:09.851 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:30:09.852 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2024-02-02T15:30:09.852 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2024-02-02T15:30:09.852 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:30:09.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.847+0000 7f85cfc00700 1 -- 172.21.15.1:0/3181920675 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f85a80a07e0 msgr2=0x7f85a80a2ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:09.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.847+0000 7f85cfc00700 1 --2- 172.21.15.1:0/3181920675 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f85a80a07e0 0x7f85a80a2ca0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f85c0007c50 tx=0x7f85c0009d10 comp rx=0 tx=0).stop 2024-02-02T15:30:09.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.847+0000 7f85cfc00700 1 -- 172.21.15.1:0/3181920675 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85c8101000 msgr2=0x7f85c8073040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:09.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.847+0000 7f85cfc00700 1 --2- 172.21.15.1:0/3181920675 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85c8101000 0x7f85c8073040 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f85b800f690 tx=0x7f85b800f6c0 comp rx=0 tx=0).stop 2024-02-02T15:30:09.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.847+0000 7f85cfc00700 1 -- 172.21.15.1:0/3181920675 shutdown_connections 2024-02-02T15:30:09.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.847+0000 7f85cfc00700 1 --2- 172.21.15.1:0/3181920675 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f85a80a07e0 0x7f85a80a2ca0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:09.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.847+0000 7f85cfc00700 1 --2- 172.21.15.1:0/3181920675 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f85c8074d50 0x7f85c81a9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:09.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.847+0000 7f85cfc00700 1 --2- 172.21.15.1:0/3181920675 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f85c8101000 0x7f85c8073040 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:09.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.847+0000 7f85cfc00700 1 --2- 172.21.15.1:0/3181920675 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f85c8106120 0x7f85c8073580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:09.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.847+0000 7f85cfc00700 1 -- 172.21.15.1:0/3181920675 >> 172.21.15.1:0/3181920675 conn(0x7f85c80fc840 msgr2=0x7f85c80fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:09.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.847+0000 7f85cfc00700 1 -- 172.21.15.1:0/3181920675 shutdown_connections 2024-02-02T15:30:09.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:09.847+0000 7f85cfc00700 1 -- 172.21.15.1:0/3181920675 wait complete. 2024-02-02T15:30:09.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:09 smithi071 ceph-mon[44707]: pgmap v97: 129 pgs: 129 active+clean; 69 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 497 KiB/s wr, 73 op/s 2024-02-02T15:30:09.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:09 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:30:09.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:09 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:30:09.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:09 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:30:09.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:09 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:30:09.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:09 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:30:09.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:09 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:30:09.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:09 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:30:09.873 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-02T15:30:09.873 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T15:30:09.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:09 smithi142 ceph-mon[45728]: pgmap v97: 129 pgs: 129 active+clean; 69 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 497 KiB/s wr, 73 op/s 2024-02-02T15:30:09.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:09 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:30:09.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:09 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:30:09.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:09 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:30:09.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:09 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:30:09.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:09 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:30:09.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:09 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:30:09.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:09 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:30:10.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.117+0000 7f47dc800700 1 -- 172.21.15.1:0/2100935726 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47d4073170 msgr2=0x7f47d4073550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:10.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.117+0000 7f47dc800700 1 --2- 172.21.15.1:0/2100935726 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47d4073170 0x7f47d4073550 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f47c4009990 tx=0x7f47c4009ca0 comp rx=0 tx=0).stop 2024-02-02T15:30:10.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.118+0000 7f47dc800700 1 -- 172.21.15.1:0/2100935726 shutdown_connections 2024-02-02T15:30:10.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.118+0000 7f47dc800700 1 --2- 172.21.15.1:0/2100935726 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f47d410d4e0 0x7f47d410f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:10.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.118+0000 7f47dc800700 1 --2- 172.21.15.1:0/2100935726 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47d4073170 0x7f47d4073550 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:10.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.118+0000 7f47dc800700 1 --2- 172.21.15.1:0/2100935726 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f47d4073a90 0x7f47d410cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:10.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.118+0000 7f47dc800700 1 -- 172.21.15.1:0/2100935726 >> 172.21.15.1:0/2100935726 conn(0x7f47d40fc790 msgr2=0x7f47d40febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:10.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.118+0000 7f47dc800700 1 -- 172.21.15.1:0/2100935726 shutdown_connections 2024-02-02T15:30:10.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.118+0000 7f47dc800700 1 -- 172.21.15.1:0/2100935726 wait complete. 2024-02-02T15:30:10.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.119+0000 7f47dc800700 1 Processor -- start 2024-02-02T15:30:10.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.119+0000 7f47dc800700 1 -- start start 2024-02-02T15:30:10.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.119+0000 7f47dc800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47d4073170 0x7f47d4199000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:10.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.119+0000 7f47dc800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f47d4073a90 0x7f47d4199540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:10.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.119+0000 7f47dc800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f47d410d4e0 0x7f47d419d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:10.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.119+0000 7f47dc800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d419dfe0 con 0x7f47d4073a90 2024-02-02T15:30:10.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.119+0000 7f47dc800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d419e150 con 0x7f47d410d4e0 2024-02-02T15:30:10.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.119+0000 7f47dc800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d419e310 con 0x7f47d4073170 2024-02-02T15:30:10.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.120+0000 7f47d9e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47d4073170 0x7f47d4199000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:10.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.120+0000 7f47d9e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47d4073170 0x7f47d4199000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:43108/0 (socket says 172.21.15.1:43108) 2024-02-02T15:30:10.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.120+0000 7f47d9e00700 1 -- 172.21.15.1:0/1238693356 learned_addr learned my addr 172.21.15.1:0/1238693356 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:30:10.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.120+0000 7f47da800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f47d410d4e0 0x7f47d419d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:10.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.120+0000 7f47d9400700 1 --2- 172.21.15.1:0/1238693356 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f47d4073a90 0x7f47d4199540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:10.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.120+0000 7f47d9e00700 1 -- 172.21.15.1:0/1238693356 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f47d410d4e0 msgr2=0x7f47d419d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:10.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.120+0000 7f47d9e00700 1 --2- 172.21.15.1:0/1238693356 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f47d410d4e0 0x7f47d419d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:10.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.120+0000 7f47d9e00700 1 -- 172.21.15.1:0/1238693356 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f47d4073a90 msgr2=0x7f47d4199540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:10.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.120+0000 7f47d9e00700 1 --2- 172.21.15.1:0/1238693356 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f47d4073a90 0x7f47d4199540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:10.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.120+0000 7f47d9e00700 1 -- 172.21.15.1:0/1238693356 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47c40095f0 con 0x7f47d4073170 2024-02-02T15:30:10.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.120+0000 7f47d9e00700 1 --2- 172.21.15.1:0/1238693356 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47d4073170 0x7f47d4199000 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f47c4005910 tx=0x7f47c4004710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:10.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.120+0000 7f47d2a00700 1 -- 172.21.15.1:0/1238693356 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c401d070 con 0x7f47d4073170 2024-02-02T15:30:10.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.120+0000 7f47dc800700 1 -- 172.21.15.1:0/1238693356 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47d419e590 con 0x7f47d4073170 2024-02-02T15:30:10.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.120+0000 7f47d2a00700 1 -- 172.21.15.1:0/1238693356 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f47c4022470 con 0x7f47d4073170 2024-02-02T15:30:10.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.121+0000 7f47d2a00700 1 -- 172.21.15.1:0/1238693356 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c4017650 con 0x7f47d4073170 2024-02-02T15:30:10.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.121+0000 7f47dc800700 1 -- 172.21.15.1:0/1238693356 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47d41a5e30 con 0x7f47d4073170 2024-02-02T15:30:10.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.122+0000 7f47dc800700 1 -- 172.21.15.1:0/1238693356 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47d4067bc0 con 0x7f47d4073170 2024-02-02T15:30:10.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.123+0000 7f47d2a00700 1 -- 172.21.15.1:0/1238693356 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 27) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f47c40225e0 con 0x7f47d4073170 2024-02-02T15:30:10.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.125+0000 7f47d2a00700 1 --2- 172.21.15.1:0/1238693356 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f47b40a0690 0x7f47b40a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:10.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.125+0000 7f47d2a00700 1 -- 172.21.15.1:0/1238693356 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f47c40cd320 con 0x7f47d4073170 2024-02-02T15:30:10.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.125+0000 7f47d9400700 1 --2- 172.21.15.1:0/1238693356 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f47b40a0690 0x7f47b40a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:10.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.126+0000 7f47d9400700 1 --2- 172.21.15.1:0/1238693356 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f47b40a0690 0x7f47b40a2b50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f47d419a610 tx=0x7f47bc009260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:10.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.131+0000 7f47d2a00700 1 -- 172.21.15.1:0/1238693356 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f47c405d110 con 0x7f47d4073170 2024-02-02T15:30:10.226 INFO:tasks.workunit.client.0.smithi001.stdout: 1 114776 6.11 MB/sec warmup 104 sec latency 16.745 ms 2024-02-02T15:30:10.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.449+0000 7f47dc800700 1 -- 172.21.15.1:0/1238693356 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f47d419a350 con 0x7f47d4073170 2024-02-02T15:30:10.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.450+0000 7f47d2a00700 1 -- 172.21.15.1:0/1238693356 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+10880 (secure 0 0 0) 0x7f47c4096860 con 0x7f47d4073170 2024-02-02T15:30:10.450 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:30:10.450 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2922240448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2922240448},{"type":"v1","addr":"172.21.15.1:6835","nonce":2922240448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2729497431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2729497431},{"type":"v1","addr":"172.21.15.1:6837","nonce":2729497431}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-02T15:24:47.761435+0000","modified":"2024-02-02T15:29:20.322035+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24529,"mds_2":24518,"mds_3":24514,"mds_4":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14517":{"gid":14517,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/1218303683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":1218303683},{"type":"v1","addr":"172.21.15.1:6839","nonce":1218303683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14529":{"gid":14529,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/2059197496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2059197496},{"type":"v1","addr":"172.21.15.1:6841","nonce":2059197496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/7648986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":7648986},{"type":"v1","addr":"172.21.15.142:6833","nonce":7648986}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/3482746094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3482746094},{"type":"v1","addr":"172.21.15.71:6833","nonce":3482746094}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/388207814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":388207814},{"type":"v1","addr":"172.21.15.71:6835","nonce":388207814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6835/1052606547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1052606547},{"type":"v1","addr":"172.21.15.142:6835","nonce":1052606547}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.142:6837/2870289019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2870289019},{"type":"v1","addr":"172.21.15.142:6837","nonce":2870289019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.71:6837/628016199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":628016199},{"type":"v1","addr":"172.21.15.71:6837","nonce":628016199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.142:6839/321675421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":321675421},{"type":"v1","addr":"172.21.15.142:6839","nonce":321675421}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/1268086493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1268086493},{"type":"v1","addr":"172.21.15.71:6839","nonce":1268086493}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-02T15:30:10.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.452+0000 7f47dc800700 1 -- 172.21.15.1:0/1238693356 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f47b40a0690 msgr2=0x7f47b40a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:10.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.452+0000 7f47dc800700 1 --2- 172.21.15.1:0/1238693356 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f47b40a0690 0x7f47b40a2b50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f47d419a610 tx=0x7f47bc009260 comp rx=0 tx=0).stop 2024-02-02T15:30:10.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.452+0000 7f47dc800700 1 -- 172.21.15.1:0/1238693356 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47d4073170 msgr2=0x7f47d4199000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:10.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.452+0000 7f47dc800700 1 --2- 172.21.15.1:0/1238693356 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47d4073170 0x7f47d4199000 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f47c4005910 tx=0x7f47c4004710 comp rx=0 tx=0).stop 2024-02-02T15:30:10.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.452+0000 7f47dc800700 1 -- 172.21.15.1:0/1238693356 shutdown_connections 2024-02-02T15:30:10.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.452+0000 7f47dc800700 1 --2- 172.21.15.1:0/1238693356 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f47b40a0690 0x7f47b40a2b50 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:10.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.452+0000 7f47dc800700 1 --2- 172.21.15.1:0/1238693356 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f47d410d4e0 0x7f47d419d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:10.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.452+0000 7f47dc800700 1 --2- 172.21.15.1:0/1238693356 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f47d4073170 0x7f47d4199000 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:10.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.452+0000 7f47dc800700 1 --2- 172.21.15.1:0/1238693356 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f47d4073a90 0x7f47d4199540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:10.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.452+0000 7f47dc800700 1 -- 172.21.15.1:0/1238693356 >> 172.21.15.1:0/1238693356 conn(0x7f47d40fc790 msgr2=0x7f47d410f640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:10.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.452+0000 7f47dc800700 1 -- 172.21.15.1:0/1238693356 shutdown_connections 2024-02-02T15:30:10.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.453+0000 7f47dc800700 1 -- 172.21.15.1:0/1238693356 wait complete. 2024-02-02T15:30:10.454 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 31 2024-02-02T15:30:10.472 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-02-02T15:30:10.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:10 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:30:10.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:10 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:30:10.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:10 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:30:10.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.691+0000 7f186b800700 1 -- 172.21.15.1:0/1651342450 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18640730e0 msgr2=0x7f18640734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:10.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.691+0000 7f186b800700 1 --2- 172.21.15.1:0/1651342450 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18640730e0 0x7f18640734c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f1854009990 tx=0x7f1854009ca0 comp rx=0 tx=0).stop 2024-02-02T15:30:10.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.691+0000 7f186b800700 1 -- 172.21.15.1:0/1651342450 shutdown_connections 2024-02-02T15:30:10.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.691+0000 7f186b800700 1 --2- 172.21.15.1:0/1651342450 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1864111d90 0x7f1864114180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:10.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.691+0000 7f186b800700 1 --2- 172.21.15.1:0/1651342450 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f18640730e0 0x7f18640734c0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:10.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.691+0000 7f186b800700 1 --2- 172.21.15.1:0/1651342450 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1864073a00 0x7f1864111850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:10.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.691+0000 7f186b800700 1 -- 172.21.15.1:0/1651342450 >> 172.21.15.1:0/1651342450 conn(0x7f18640fc810 msgr2=0x7f18640fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:10.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.691+0000 7f186b800700 1 -- 172.21.15.1:0/1651342450 shutdown_connections 2024-02-02T15:30:10.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.691+0000 7f186b800700 1 -- 172.21.15.1:0/1651342450 wait complete. 2024-02-02T15:30:10.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.692+0000 7f186b800700 1 Processor -- start 2024-02-02T15:30:10.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.692+0000 7f186b800700 1 -- start start 2024-02-02T15:30:10.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.693+0000 7f186b800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f18640730e0 0x7f186419d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:10.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.693+0000 7f186b800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1864073a00 0x7f186419daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:10.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.693+0000 7f186b800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1864111d90 0x7f18641a1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:10.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.693+0000 7f186b800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18641a2540 con 0x7f18640730e0 2024-02-02T15:30:10.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.693+0000 7f186b800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18641a26b0 con 0x7f1864111d90 2024-02-02T15:30:10.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.693+0000 7f186b800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18641a2870 con 0x7f1864073a00 2024-02-02T15:30:10.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.693+0000 7f1869800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1864111d90 0x7f18641a1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:10.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.693+0000 7f1868e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f18640730e0 0x7f186419d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:10.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.693+0000 7f185be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1864073a00 0x7f186419daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:10.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.693+0000 7f1869800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1864111d90 0x7f18641a1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59890/0 (socket says 172.21.15.1:59890) 2024-02-02T15:30:10.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.693+0000 7f1869800700 1 -- 172.21.15.1:0/876411790 learned_addr learned my addr 172.21.15.1:0/876411790 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:30:10.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.693+0000 7f1869800700 1 -- 172.21.15.1:0/876411790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1864073a00 msgr2=0x7f186419daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:10.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.693+0000 7f1869800700 1 --2- 172.21.15.1:0/876411790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1864073a00 0x7f186419daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:10.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.693+0000 7f1869800700 1 -- 172.21.15.1:0/876411790 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f18640730e0 msgr2=0x7f186419d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:10.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.693+0000 7f1869800700 1 --2- 172.21.15.1:0/876411790 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f18640730e0 0x7f186419d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:10.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.693+0000 7f1869800700 1 -- 172.21.15.1:0/876411790 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18540095f0 con 0x7f1864111d90 2024-02-02T15:30:10.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.694+0000 7f1869800700 1 --2- 172.21.15.1:0/876411790 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1864111d90 0x7f18641a1f00 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f185c00ef90 tx=0x7f185c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:10.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.694+0000 7f1862000700 1 -- 172.21.15.1:0/876411790 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185c018070 con 0x7f1864111d90 2024-02-02T15:30:10.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.694+0000 7f1862000700 1 -- 172.21.15.1:0/876411790 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f185c00f040 con 0x7f1864111d90 2024-02-02T15:30:10.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.694+0000 7f1862000700 1 -- 172.21.15.1:0/876411790 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185c00cda0 con 0x7f1864111d90 2024-02-02T15:30:10.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.694+0000 7f186b800700 1 -- 172.21.15.1:0/876411790 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18641a2af0 con 0x7f1864111d90 2024-02-02T15:30:10.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.694+0000 7f186b800700 1 -- 172.21.15.1:0/876411790 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18641aa390 con 0x7f1864111d90 2024-02-02T15:30:10.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.697+0000 7f1862000700 1 -- 172.21.15.1:0/876411790 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 27) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f185c003a00 con 0x7f1864111d90 2024-02-02T15:30:10.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.699+0000 7f1862000700 1 --2- 172.21.15.1:0/876411790 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f18440a0710 0x7f18440a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:10.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.699+0000 7f187f2a1b80 1 Processor -- start 2024-02-02T15:30:10.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.699+0000 7f1862000700 1 -- 172.21.15.1:0/876411790 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f185c0cbb30 con 0x7f1864111d90 2024-02-02T15:30:10.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.699+0000 7f187f2a1b80 1 -- start start 2024-02-02T15:30:10.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.699+0000 7f1868e00700 1 --2- 172.21.15.1:0/876411790 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f18440a0710 0x7f18440a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:10.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.699+0000 7f187f2a1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564b5d90a010 0x564b5d90a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:10.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.699+0000 7f187f2a1b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564b5d90a930 0x564b5d9152a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:10.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.699+0000 7f187f2a1b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564b5d9157e0 0x564b5d917bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:10.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.699+0000 7f187f2a1b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b5d87e4c0 con 0x564b5d90a930 2024-02-02T15:30:10.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.699+0000 7f187f2a1b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b5d862aa0 con 0x564b5d9157e0 2024-02-02T15:30:10.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.699+0000 7f187f2a1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b5d860460 con 0x564b5d90a010 2024-02-02T15:30:10.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.700+0000 7f1868e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564b5d90a010 0x564b5d90a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:10.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.700+0000 7f1869800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564b5d9157e0 0x564b5d917bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:10.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.700+0000 7f1868e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564b5d90a010 0x564b5d90a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:43120/0 (socket says 172.21.15.1:43120) 2024-02-02T15:30:10.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.700+0000 7f1868e00700 1 -- 172.21.15.1:0/1133419323 learned_addr learned my addr 172.21.15.1:0/1133419323 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:30:10.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.700+0000 7f185be00700 1 --2- 172.21.15.1:0/1133419323 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564b5d90a930 0x564b5d9152a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:10.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.700+0000 7f1869800700 1 -- 172.21.15.1:0/1133419323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564b5d90a010 msgr2=0x564b5d90a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:10.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.700+0000 7f1869800700 1 --2- 172.21.15.1:0/1133419323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564b5d90a010 0x564b5d90a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:10.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.700+0000 7f1869800700 1 -- 172.21.15.1:0/1133419323 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564b5d90a930 msgr2=0x564b5d9152a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:10.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.700+0000 7f1869800700 1 --2- 172.21.15.1:0/1133419323 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564b5d90a930 0x564b5d9152a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:10.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.700+0000 7f1869800700 1 -- 172.21.15.1:0/1133419323 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564b5d6dd3b0 con 0x564b5d9157e0 2024-02-02T15:30:10.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.700+0000 7f1868e00700 1 --2- 172.21.15.1:0/1133419323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564b5d90a010 0x564b5d90a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:30:10.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.700+0000 7f185be00700 1 --2- 172.21.15.1:0/1133419323 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564b5d90a930 0x564b5d9152a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:30:10.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.700+0000 7f1868e00700 1 --2- 172.21.15.1:0/876411790 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f18440a0710 0x7f18440a2bd0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1854015000 tx=0x7f185400bb20 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:10.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.700+0000 7f1869800700 1 --2- 172.21.15.1:0/1133419323 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564b5d9157e0 0x564b5d917bd0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f185c0c8ea0 tx=0x7f185c095070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc84aa596187d8ef server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:10.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.700+0000 7f185a000700 1 -- 172.21.15.1:0/1133419323 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185c098690 con 0x564b5d9157e0 2024-02-02T15:30:10.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.701+0000 7f185a000700 1 -- 172.21.15.1:0/1133419323 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f185c099430 con 0x564b5d9157e0 2024-02-02T15:30:10.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.701+0000 7f185a000700 1 -- 172.21.15.1:0/1133419323 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185c07c1e0 con 0x564b5d9157e0 2024-02-02T15:30:10.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.701+0000 7f187f2a1b80 1 -- 172.21.15.1:0/1133419323 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564b5d9157e0 msgr2=0x564b5d917bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:10.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.701+0000 7f187f2a1b80 1 --2- 172.21.15.1:0/1133419323 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564b5d9157e0 0x564b5d917bd0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f185c0c8ea0 tx=0x7f185c095070 comp rx=0 tx=0).stop 2024-02-02T15:30:10.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.701+0000 7f187f2a1b80 1 -- 172.21.15.1:0/1133419323 shutdown_connections 2024-02-02T15:30:10.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.701+0000 7f187f2a1b80 1 --2- 172.21.15.1:0/1133419323 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564b5d9157e0 0x564b5d917bd0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:10.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.701+0000 7f187f2a1b80 1 --2- 172.21.15.1:0/1133419323 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564b5d90a010 0x564b5d90a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:10.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.701+0000 7f187f2a1b80 1 --2- 172.21.15.1:0/1133419323 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564b5d90a930 0x564b5d9152a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:10.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.701+0000 7f187f2a1b80 1 -- 172.21.15.1:0/1133419323 >> 172.21.15.1:0/1133419323 conn(0x564b5d869db0 msgr2=0x564b5d9063b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:10.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.701+0000 7f187f2a1b80 1 -- 172.21.15.1:0/1133419323 shutdown_connections 2024-02-02T15:30:10.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.701+0000 7f187f2a1b80 1 -- 172.21.15.1:0/1133419323 wait complete. 2024-02-02T15:30:10.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.702+0000 7f187f2a1b80 1 -- start start 2024-02-02T15:30:10.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.702+0000 7f187f2a1b80 1 Processor -- start 2024-02-02T15:30:10.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.703+0000 7f187f2a1b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564b5d90a010 0x564b5d915140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:10.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.703+0000 7f187f2a1b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564b5d90a930 0x564b5d9123c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:10.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.703+0000 7f187f2a1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564b5d9157e0 0x564b5d912940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:10.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.703+0000 7f187f2a1b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b5d85bd50 con 0x564b5d90a930 2024-02-02T15:30:10.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.703+0000 7f187f2a1b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b5d7f7d80 con 0x564b5d90a010 2024-02-02T15:30:10.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.703+0000 7f187f2a1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b5d7f7a20 con 0x564b5d9157e0 2024-02-02T15:30:10.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.703+0000 7f187f2a1b80 10 client.0 _ll_register_callbacks cb 0x564b5d7f9290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:30:10.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.703+0000 7f185be00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564b5d90a930 0x564b5d9123c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:10.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.703+0000 7f1868e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564b5d90a010 0x564b5d915140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:10.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.703+0000 7f1868e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564b5d90a010 0x564b5d915140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59902/0 (socket says 172.21.15.1:59902) 2024-02-02T15:30:10.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.703+0000 7f1868e00700 1 -- 172.21.15.1:0/2633417946 learned_addr learned my addr 172.21.15.1:0/2633417946 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:30:10.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.703+0000 7f185be00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564b5d90a930 0x564b5d9123c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54446/0 (socket says 172.21.15.1:54446) 2024-02-02T15:30:10.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.703+0000 7f1869800700 1 --2- 172.21.15.1:0/2633417946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564b5d9157e0 0x564b5d912940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:10.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.703+0000 7f1868e00700 1 -- 172.21.15.1:0/2633417946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564b5d9157e0 msgr2=0x564b5d912940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:10.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.703+0000 7f1868e00700 1 --2- 172.21.15.1:0/2633417946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564b5d9157e0 0x564b5d912940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:10.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.703+0000 7f1868e00700 1 -- 172.21.15.1:0/2633417946 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564b5d90a930 msgr2=0x564b5d9123c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:10.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.703+0000 7f1868e00700 1 --2- 172.21.15.1:0/2633417946 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564b5d90a930 0x564b5d9123c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:10.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.703+0000 7f1868e00700 1 -- 172.21.15.1:0/2633417946 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f184c0098e0 con 0x564b5d90a010 2024-02-02T15:30:10.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.704+0000 7f1868e00700 1 --2- 172.21.15.1:0/2633417946 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564b5d90a010 0x564b5d915140 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f18540050d0 tx=0x7f1854009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90c85df0e1ffdab8 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:10.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.704+0000 7f1852a00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:30:10.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.704+0000 7f1852a00700 1 -- 172.21.15.1:0/2633417946 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1854004910 con 0x564b5d90a010 2024-02-02T15:30:10.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.704+0000 7f187f2a1b80 1 -- 172.21.15.1:0/2633417946 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564b5d8fe0f0 con 0x564b5d90a010 2024-02-02T15:30:10.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.704+0000 7f1852a00700 1 -- 172.21.15.1:0/2633417946 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f1854004a70 con 0x564b5d90a010 2024-02-02T15:30:10.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.704+0000 7f1852a00700 1 -- 172.21.15.1:0/2633417946 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1854004c30 con 0x564b5d90a010 2024-02-02T15:30:10.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.704+0000 7f187f2a1b80 10 client.26456 fetch_fsmap learned FSMap version 31 2024-02-02T15:30:10.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.704+0000 7f187f2a1b80 1 -- 172.21.15.1:0/2633417946 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f18641aa390 con 0x564b5d90a010 2024-02-02T15:30:10.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.705+0000 7f1852a00700 1 -- 172.21.15.1:0/2633417946 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564b5d8fe0f0 con 0x564b5d90a010 2024-02-02T15:30:10.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.705+0000 7f1852a00700 1 -- 172.21.15.1:0/2633417946 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 31) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f185402f030 con 0x564b5d90a010 2024-02-02T15:30:10.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.705+0000 7f187f2a1b80 10 client.26456 fetch_fsmap finished waiting for FSMap version 31 2024-02-02T15:30:10.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.705+0000 7f187f2a1b80 10 client.26456 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-02-02T15:30:10.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.705+0000 7f187f2a1b80 20 client.26456 populate_metadata read hostname 'smithi001' 2024-02-02T15:30:10.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.705+0000 7f187f2a1b80 1 --2- 172.21.15.1:0/2633417946 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x564b5d924f70 0x564b5d927390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:10.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.705+0000 7f187f2a1b80 4 client.26456 mds_command: new command op to 24502 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:30:10.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.705+0000 7f187f2a1b80 1 -- 172.21.15.1:0/2633417946 --> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564b5d854dd0 con 0x564b5d924f70 2024-02-02T15:30:10.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.705+0000 7f185be00700 1 --2- 172.21.15.1:0/2633417946 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x564b5d924f70 0x564b5d927390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:10.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.706+0000 7f185be00700 1 --2- 172.21.15.1:0/2633417946 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x564b5d924f70 0x564b5d927390 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c4c35bfdd88eaa18 server_cookie=b408c04c84f18179 in_seq=0 out_seq=0 2024-02-02T15:30:10.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.706+0000 7f1852a00700 10 client.26456 ms_handle_connect on v2:172.21.15.142:6834/1052606547 2024-02-02T15:30:10.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.706+0000 7f1852a00700 1 -- 172.21.15.1:0/2633417946 <== mds.4 v2:172.21.15.142:6834/1052606547 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x564b5d854dd0 con 0x564b5d924f70 2024-02-02T15:30:10.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.706+0000 7f1852a00700 10 client.26456 handle_command_reply: tid=0 2024-02-02T15:30:10.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.706+0000 7f187f2a1b80 1 client.26456 shutdown 2024-02-02T15:30:10.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.706+0000 7f187f2a1b80 1 -- 172.21.15.1:0/2633417946 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564b5d90a010 msgr2=0x564b5d915140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:10.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.706+0000 7f187f2a1b80 1 --2- 172.21.15.1:0/2633417946 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564b5d90a010 0x564b5d915140 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f18540050d0 tx=0x7f1854009ca0 comp rx=0 tx=0).stop 2024-02-02T15:30:10.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.706+0000 7f187f2a1b80 1 -- 172.21.15.1:0/2633417946 shutdown_connections 2024-02-02T15:30:10.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.706+0000 7f187f2a1b80 1 --2- 172.21.15.1:0/2633417946 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x564b5d924f70 0x564b5d927390 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:10.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.707+0000 7f187f2a1b80 1 --2- 172.21.15.1:0/2633417946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564b5d9157e0 0x564b5d912940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:10.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.707+0000 7f187f2a1b80 1 --2- 172.21.15.1:0/2633417946 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564b5d90a010 0x564b5d915140 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:10.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.707+0000 7f187f2a1b80 1 --2- 172.21.15.1:0/2633417946 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564b5d90a930 0x564b5d9123c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:10.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.707+0000 7f187f2a1b80 1 -- 172.21.15.1:0/2633417946 >> 172.21.15.1:0/2633417946 conn(0x564b5d869db0 msgr2=0x564b5d905fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:10.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.707+0000 7f1852a00700 0 client.26456 ms_handle_reset on v2:172.21.15.142:6834/1052606547 2024-02-02T15:30:10.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.707+0000 7f187f2a1b80 1 -- 172.21.15.1:0/2633417946 shutdown_connections 2024-02-02T15:30:10.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.707+0000 7f187f2a1b80 1 -- 172.21.15.1:0/2633417946 wait complete. 2024-02-02T15:30:10.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.707+0000 7f187f2a1b80 20 client.26456 trim_cache size 0 max 16384 2024-02-02T15:30:10.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.721+0000 7f187f2a1b80 1 Processor -- start 2024-02-02T15:30:10.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.721+0000 7f187f2a1b80 1 -- start start 2024-02-02T15:30:10.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.721+0000 7f187f2a1b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564b5d90a010 0x564b5d91c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:10.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.721+0000 7f187f2a1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564b5d90a930 0x564b5d914c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:10.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.721+0000 7f187f2a1b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564b5d9157e0 0x564b5d911c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:10.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.721+0000 7f187f2a1b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b5d6df6a0 con 0x564b5d9157e0 2024-02-02T15:30:10.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.721+0000 7f187f2a1b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b5d90e060 con 0x564b5d90a010 2024-02-02T15:30:10.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.721+0000 7f187f2a1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b5d910970 con 0x564b5d90a930 2024-02-02T15:30:10.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.721+0000 7f185be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564b5d90a930 0x564b5d914c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:10.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.722+0000 7f185be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564b5d90a930 0x564b5d914c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:43152/0 (socket says 172.21.15.1:43152) 2024-02-02T15:30:10.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.722+0000 7f185be00700 1 -- 172.21.15.1:0/989510006 learned_addr learned my addr 172.21.15.1:0/989510006 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:30:10.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.722+0000 7f1868e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564b5d90a010 0x564b5d91c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:10.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.722+0000 7f1869800700 1 --2- 172.21.15.1:0/989510006 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564b5d9157e0 0x564b5d911c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:10.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.722+0000 7f1868e00700 1 -- 172.21.15.1:0/989510006 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564b5d90a930 msgr2=0x564b5d914c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:10.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.722+0000 7f1868e00700 1 --2- 172.21.15.1:0/989510006 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564b5d90a930 0x564b5d914c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:10.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.722+0000 7f1868e00700 1 -- 172.21.15.1:0/989510006 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564b5d9157e0 msgr2=0x564b5d911c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:10.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.722+0000 7f1868e00700 1 --2- 172.21.15.1:0/989510006 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564b5d9157e0 0x564b5d911c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:10.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.722+0000 7f1868e00700 1 -- 172.21.15.1:0/989510006 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564b5d689b60 con 0x564b5d90a010 2024-02-02T15:30:10.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.722+0000 7f185be00700 1 --2- 172.21.15.1:0/989510006 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564b5d90a930 0x564b5d914c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:30:10.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.722+0000 7f1868e00700 1 --2- 172.21.15.1:0/989510006 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564b5d90a010 0x564b5d91c260 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f185400fef0 tx=0x7f1854009510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=831b01f937d34f00 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:10.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.722+0000 7f185aa00700 1 -- 172.21.15.1:0/989510006 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18540319e0 con 0x564b5d90a010 2024-02-02T15:30:10.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.722+0000 7f185aa00700 1 -- 172.21.15.1:0/989510006 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f185401e040 con 0x564b5d90a010 2024-02-02T15:30:10.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.722+0000 7f185aa00700 1 -- 172.21.15.1:0/989510006 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1854004dc0 con 0x564b5d90a010 2024-02-02T15:30:10.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.723+0000 7f187f2a1b80 1 -- 172.21.15.1:0/989510006 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564b5d90a010 msgr2=0x564b5d91c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:10.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.723+0000 7f187f2a1b80 1 --2- 172.21.15.1:0/989510006 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564b5d90a010 0x564b5d91c260 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f185400fef0 tx=0x7f1854009510 comp rx=0 tx=0).stop 2024-02-02T15:30:10.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.723+0000 7f187f2a1b80 1 -- 172.21.15.1:0/989510006 shutdown_connections 2024-02-02T15:30:10.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.723+0000 7f187f2a1b80 1 --2- 172.21.15.1:0/989510006 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564b5d9157e0 0x564b5d911c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:10.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.723+0000 7f187f2a1b80 1 --2- 172.21.15.1:0/989510006 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564b5d90a010 0x564b5d91c260 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:10.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.723+0000 7f187f2a1b80 1 --2- 172.21.15.1:0/989510006 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564b5d90a930 0x564b5d914c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:10.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.723+0000 7f187f2a1b80 1 -- 172.21.15.1:0/989510006 >> 172.21.15.1:0/989510006 conn(0x564b5d869db0 msgr2=0x564b5d920840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:10.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.723+0000 7f187f2a1b80 1 -- 172.21.15.1:0/989510006 shutdown_connections 2024-02-02T15:30:10.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.723+0000 7f187f2a1b80 1 -- 172.21.15.1:0/989510006 wait complete. 2024-02-02T15:30:10.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.724+0000 7f187f2a1b80 1 -- start start 2024-02-02T15:30:10.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.724+0000 7f187f2a1b80 1 Processor -- start 2024-02-02T15:30:10.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.724+0000 7f187f2a1b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564b5d90a010 0x564b5d918210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:10.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.724+0000 7f187f2a1b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564b5d90a930 0x564b5d918750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:10.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.724+0000 7f187f2a1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564b5d9157e0 0x564b5d975a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:10.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.724+0000 7f187f2a1b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b5d918ef0 con 0x564b5d90a930 2024-02-02T15:30:10.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.724+0000 7f187f2a1b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b5d919060 con 0x564b5d90a010 2024-02-02T15:30:10.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.724+0000 7f187f2a1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b5d975f80 con 0x564b5d9157e0 2024-02-02T15:30:10.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.724+0000 7f187f2a1b80 10 client.0 _ll_register_callbacks cb 0x564b5d7f9290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:30:10.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.725+0000 7f1868e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564b5d90a010 0x564b5d918210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:10.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.725+0000 7f1868e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564b5d90a010 0x564b5d918210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59928/0 (socket says 172.21.15.1:59928) 2024-02-02T15:30:10.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.725+0000 7f1868e00700 1 -- 172.21.15.1:0/2931760417 learned_addr learned my addr 172.21.15.1:0/2931760417 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:30:10.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.725+0000 7f185be00700 1 --2- 172.21.15.1:0/2931760417 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564b5d90a930 0x564b5d918750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:10.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.725+0000 7f1869800700 1 --2- 172.21.15.1:0/2931760417 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564b5d9157e0 0x564b5d975a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:10.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.725+0000 7f1868e00700 1 -- 172.21.15.1:0/2931760417 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564b5d9157e0 msgr2=0x564b5d975a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:10.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.725+0000 7f1868e00700 1 --2- 172.21.15.1:0/2931760417 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564b5d9157e0 0x564b5d975a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:10.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.725+0000 7f1868e00700 1 -- 172.21.15.1:0/2931760417 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564b5d90a930 msgr2=0x564b5d918750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:10.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.725+0000 7f1868e00700 1 --2- 172.21.15.1:0/2931760417 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564b5d90a930 0x564b5d918750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:10.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.725+0000 7f1868e00700 1 -- 172.21.15.1:0/2931760417 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f184c004040 con 0x564b5d90a010 2024-02-02T15:30:10.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.725+0000 7f1868e00700 1 --2- 172.21.15.1:0/2931760417 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564b5d90a010 0x564b5d918210 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1854000c00 tx=0x7f18540056c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b36a8008dc5062d4 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:10.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.725+0000 7f1852a00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:30:10.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.726+0000 7f1852a00700 1 -- 172.21.15.1:0/2931760417 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1854031a70 con 0x564b5d90a010 2024-02-02T15:30:10.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.726+0000 7f1852a00700 1 -- 172.21.15.1:0/2931760417 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f1854031bd0 con 0x564b5d90a010 2024-02-02T15:30:10.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.726+0000 7f187f2a1b80 1 -- 172.21.15.1:0/2931760417 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f185c099430 con 0x564b5d90a010 2024-02-02T15:30:10.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.726+0000 7f1852a00700 1 -- 172.21.15.1:0/2931760417 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1854031d90 con 0x564b5d90a010 2024-02-02T15:30:10.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.726+0000 7f187f2a1b80 10 client.26462 fetch_fsmap learned FSMap version 31 2024-02-02T15:30:10.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.726+0000 7f187f2a1b80 1 -- 172.21.15.1:0/2931760417 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f18641aa390 con 0x564b5d90a010 2024-02-02T15:30:10.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.726+0000 7f1852a00700 1 -- 172.21.15.1:0/2931760417 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f185c099430 con 0x564b5d90a010 2024-02-02T15:30:10.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.726+0000 7f1852a00700 1 -- 172.21.15.1:0/2931760417 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 31) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f1854005100 con 0x564b5d90a010 2024-02-02T15:30:10.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.726+0000 7f187f2a1b80 10 client.26462 fetch_fsmap finished waiting for FSMap version 31 2024-02-02T15:30:10.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.726+0000 7f187f2a1b80 10 client.26462 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-02-02T15:30:10.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.726+0000 7f187f2a1b80 20 client.26462 populate_metadata read hostname 'smithi001' 2024-02-02T15:30:10.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.726+0000 7f187f2a1b80 1 --2- 172.21.15.1:0/2931760417 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x564b5d977200 0x564b5d9795f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:10.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.726+0000 7f187f2a1b80 4 client.26462 mds_command: new command op to 24502 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:30:10.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.726+0000 7f187f2a1b80 1 -- 172.21.15.1:0/2931760417 --> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564b5d81af50 con 0x564b5d977200 2024-02-02T15:30:10.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.727+0000 7f185be00700 1 --2- 172.21.15.1:0/2931760417 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x564b5d977200 0x564b5d9795f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:10.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.727+0000 7f185be00700 1 --2- 172.21.15.1:0/2931760417 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x564b5d977200 0x564b5d9795f0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=100e1f80de347069 server_cookie=ae358214bd0ad74f in_seq=0 out_seq=0 2024-02-02T15:30:10.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.727+0000 7f1852a00700 10 client.26462 ms_handle_connect on v2:172.21.15.142:6834/1052606547 2024-02-02T15:30:10.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.727+0000 7f1852a00700 1 -- 172.21.15.1:0/2931760417 <== mds.4 v2:172.21.15.142:6834/1052606547 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564b5d81af50 con 0x564b5d977200 2024-02-02T15:30:10.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.727+0000 7f1852a00700 10 client.26462 handle_command_reply: tid=0 2024-02-02T15:30:10.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.727+0000 7f187f2a1b80 1 client.26462 shutdown 2024-02-02T15:30:10.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.728+0000 7f187f2a1b80 1 -- 172.21.15.1:0/2931760417 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564b5d90a010 msgr2=0x564b5d918210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:10.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.728+0000 7f187f2a1b80 1 --2- 172.21.15.1:0/2931760417 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564b5d90a010 0x564b5d918210 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1854000c00 tx=0x7f18540056c0 comp rx=0 tx=0).stop 2024-02-02T15:30:10.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.728+0000 7f187f2a1b80 1 -- 172.21.15.1:0/2931760417 shutdown_connections 2024-02-02T15:30:10.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.728+0000 7f187f2a1b80 1 --2- 172.21.15.1:0/2931760417 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x564b5d977200 0x564b5d9795f0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:10.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.728+0000 7f187f2a1b80 1 --2- 172.21.15.1:0/2931760417 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564b5d9157e0 0x564b5d975a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:10.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.728+0000 7f187f2a1b80 1 --2- 172.21.15.1:0/2931760417 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564b5d90a010 0x564b5d918210 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:10.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.728+0000 7f1852a00700 0 client.26462 ms_handle_reset on v2:172.21.15.142:6834/1052606547 2024-02-02T15:30:10.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.728+0000 7f187f2a1b80 1 --2- 172.21.15.1:0/2931760417 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564b5d90a930 0x564b5d918750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:10.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.728+0000 7f187f2a1b80 1 -- 172.21.15.1:0/2931760417 >> 172.21.15.1:0/2931760417 conn(0x564b5d869db0 msgr2=0x564b5d911c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:10.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.728+0000 7f187f2a1b80 1 -- 172.21.15.1:0/2931760417 shutdown_connections 2024-02-02T15:30:10.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.728+0000 7f187f2a1b80 1 -- 172.21.15.1:0/2931760417 wait complete. 2024-02-02T15:30:10.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.728+0000 7f187f2a1b80 20 client.26462 trim_cache size 0 max 16384 2024-02-02T15:30:10.732 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:30:10.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.729+0000 7f186b800700 1 -- 172.21.15.1:0/876411790 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f18440a0710 msgr2=0x7f18440a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:10.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.729+0000 7f186b800700 1 --2- 172.21.15.1:0/876411790 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f18440a0710 0x7f18440a2bd0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1854015000 tx=0x7f185400bb20 comp rx=0 tx=0).stop 2024-02-02T15:30:10.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.729+0000 7f186b800700 1 -- 172.21.15.1:0/876411790 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1864111d90 msgr2=0x7f18641a1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:10.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.729+0000 7f186b800700 1 --2- 172.21.15.1:0/876411790 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1864111d90 0x7f18641a1f00 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f185c00ef90 tx=0x7f185c00c3a0 comp rx=0 tx=0).stop 2024-02-02T15:30:10.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.729+0000 7f186b800700 1 -- 172.21.15.1:0/876411790 shutdown_connections 2024-02-02T15:30:10.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.729+0000 7f186b800700 1 --2- 172.21.15.1:0/876411790 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f18440a0710 0x7f18440a2bd0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:10.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.729+0000 7f186b800700 1 --2- 172.21.15.1:0/876411790 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1864111d90 0x7f18641a1f00 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:10.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.729+0000 7f186b800700 1 --2- 172.21.15.1:0/876411790 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f18640730e0 0x7f186419d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:10.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.729+0000 7f186b800700 1 --2- 172.21.15.1:0/876411790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1864073a00 0x7f186419daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:10.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.729+0000 7f186b800700 1 -- 172.21.15.1:0/876411790 >> 172.21.15.1:0/876411790 conn(0x7f18640fc810 msgr2=0x7f18640fd480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:10.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.729+0000 7f186b800700 1 -- 172.21.15.1:0/876411790 shutdown_connections 2024-02-02T15:30:10.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.729+0000 7f186b800700 1 -- 172.21.15.1:0/876411790 wait complete. 2024-02-02T15:30:10.748 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-02-02T15:30:10.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:10 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:30:10.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:10 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:30:10.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:10 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:30:10.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:10 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:30:10.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:10 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:30:10.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:10 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:30:10.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.960+0000 7f7c27c00700 1 -- 172.21.15.1:0/1730320195 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7c20106ed0 msgr2=0x7f7c201072b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:10.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.960+0000 7f7c27c00700 1 --2- 172.21.15.1:0/1730320195 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7c20106ed0 0x7f7c201072b0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f7c100099c0 tx=0x7f7c10009cd0 comp rx=0 tx=0).stop 2024-02-02T15:30:10.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.961+0000 7f7c27c00700 1 -- 172.21.15.1:0/1730320195 shutdown_connections 2024-02-02T15:30:10.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.961+0000 7f7c27c00700 1 --2- 172.21.15.1:0/1730320195 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c200689b0 0x7f7c2010dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:10.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.961+0000 7f7c27c00700 1 --2- 172.21.15.1:0/1730320195 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7c20106ed0 0x7f7c201072b0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:10.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.961+0000 7f7c27c00700 1 --2- 172.21.15.1:0/1730320195 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c20068010 0x7f7c20068470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:10.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.961+0000 7f7c27c00700 1 -- 172.21.15.1:0/1730320195 >> 172.21.15.1:0/1730320195 conn(0x7f7c200757d0 msgr2=0x7f7c20075be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:10.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.961+0000 7f7c27c00700 1 -- 172.21.15.1:0/1730320195 shutdown_connections 2024-02-02T15:30:10.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.961+0000 7f7c27c00700 1 -- 172.21.15.1:0/1730320195 wait complete. 2024-02-02T15:30:10.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.962+0000 7f7c27c00700 1 Processor -- start 2024-02-02T15:30:10.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.962+0000 7f7c27c00700 1 -- start start 2024-02-02T15:30:10.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.962+0000 7f7c27c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7c20068010 0x7f7c20199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:10.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.963+0000 7f7c27c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c200689b0 0x7f7c20199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:10.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.963+0000 7f7c27c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c20106ed0 0x7f7c2019dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:10.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.963+0000 7f7c27c00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c2019e1e0 con 0x7f7c20068010 2024-02-02T15:30:10.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.963+0000 7f7c27c00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c2019e350 con 0x7f7c200689b0 2024-02-02T15:30:10.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.963+0000 7f7c27c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c2019e510 con 0x7f7c20106ed0 2024-02-02T15:30:10.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.963+0000 7f7c1fe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c200689b0 0x7f7c20199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:10.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.963+0000 7f7c1fe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c200689b0 0x7f7c20199740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59938/0 (socket says 172.21.15.1:59938) 2024-02-02T15:30:10.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.963+0000 7f7c1fe00700 1 -- 172.21.15.1:0/2430713505 learned_addr learned my addr 172.21.15.1:0/2430713505 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:30:10.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.963+0000 7f7c25c00700 1 --2- 172.21.15.1:0/2430713505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c20106ed0 0x7f7c2019dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:10.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.963+0000 7f7c25200700 1 --2- 172.21.15.1:0/2430713505 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7c20068010 0x7f7c20199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:10.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.963+0000 7f7c1fe00700 1 -- 172.21.15.1:0/2430713505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c20106ed0 msgr2=0x7f7c2019dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:10.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.963+0000 7f7c1fe00700 1 --2- 172.21.15.1:0/2430713505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c20106ed0 0x7f7c2019dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:10.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.963+0000 7f7c1fe00700 1 -- 172.21.15.1:0/2430713505 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7c20068010 msgr2=0x7f7c20199200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:10.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.963+0000 7f7c1fe00700 1 --2- 172.21.15.1:0/2430713505 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7c20068010 0x7f7c20199200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:10.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.963+0000 7f7c1fe00700 1 -- 172.21.15.1:0/2430713505 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c100095f0 con 0x7f7c200689b0 2024-02-02T15:30:10.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.964+0000 7f7c1fe00700 1 --2- 172.21.15.1:0/2430713505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c200689b0 0x7f7c20199740 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f7c0800b570 tx=0x7f7c0800b880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:10.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.964+0000 7f7c1d600700 1 -- 172.21.15.1:0/2430713505 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c08019070 con 0x7f7c200689b0 2024-02-02T15:30:10.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.964+0000 7f7c1d600700 1 -- 172.21.15.1:0/2430713505 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c08014e40 con 0x7f7c200689b0 2024-02-02T15:30:10.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.964+0000 7f7c27c00700 1 -- 172.21.15.1:0/2430713505 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c2019e7f0 con 0x7f7c200689b0 2024-02-02T15:30:10.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.964+0000 7f7c1d600700 1 -- 172.21.15.1:0/2430713505 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c0801d8d0 con 0x7f7c200689b0 2024-02-02T15:30:10.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.964+0000 7f7c27c00700 1 -- 172.21.15.1:0/2430713505 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c20107d40 con 0x7f7c200689b0 2024-02-02T15:30:10.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.968+0000 7f7c1d600700 1 -- 172.21.15.1:0/2430713505 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 27) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f7c0801b520 con 0x7f7c200689b0 2024-02-02T15:30:10.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.969+0000 7f7c3b7a8b80 1 Processor -- start 2024-02-02T15:30:10.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.969+0000 7f7c3b7a8b80 1 -- start start 2024-02-02T15:30:10.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.969+0000 7f7c3b7a8b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c6c73c1010 0x55c6c73c13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:10.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.969+0000 7f7c3b7a8b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c6c73c1930 0x55c6c73cc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:10.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.969+0000 7f7c3b7a8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6c73cc7e0 0x55c6c73cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:10.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.969+0000 7f7c3b7a8b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6c73354c0 con 0x55c6c73c1930 2024-02-02T15:30:10.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.969+0000 7f7c3b7a8b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6c7319aa0 con 0x55c6c73c1010 2024-02-02T15:30:10.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.969+0000 7f7c3b7a8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6c7317460 con 0x55c6c73cc7e0 2024-02-02T15:30:10.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.970+0000 7f7c25c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c6c73c1010 0x55c6c73c13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:10.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.970+0000 7f7c1fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6c73cc7e0 0x55c6c73cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:10.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.970+0000 7f7c25c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c6c73c1010 0x55c6c73c13f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59948/0 (socket says 172.21.15.1:59948) 2024-02-02T15:30:10.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.970+0000 7f7c1d600700 1 --2- 172.21.15.1:0/2430713505 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f7c000a0710 0x7f7c000a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:10.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.970+0000 7f7c25200700 1 --2- 172.21.15.1:0/2040780900 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c6c73c1930 0x55c6c73cc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:10.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.970+0000 7f7c25c00700 1 -- 172.21.15.1:0/2040780900 learned_addr learned my addr 172.21.15.1:0/2040780900 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:30:10.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.970+0000 7f7c1d600700 1 -- 172.21.15.1:0/2430713505 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f7c080d3d80 con 0x7f7c200689b0 2024-02-02T15:30:10.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.970+0000 7f7c25c00700 1 --2- 172.21.15.1:0/2430713505 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f7c000a0710 0x7f7c000a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:10.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.970+0000 7f7c1fe00700 1 -- 172.21.15.1:0/2040780900 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c6c73c1010 msgr2=0x55c6c73c13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:10.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.970+0000 7f7c1fe00700 1 --2- 172.21.15.1:0/2040780900 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c6c73c1010 0x55c6c73c13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:10.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.970+0000 7f7c1fe00700 1 -- 172.21.15.1:0/2040780900 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c6c73c1930 msgr2=0x55c6c73cc2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:10.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.970+0000 7f7c1fe00700 1 --2- 172.21.15.1:0/2040780900 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c6c73c1930 0x55c6c73cc2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:10.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.970+0000 7f7c1fe00700 1 -- 172.21.15.1:0/2040780900 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6c71943b0 con 0x55c6c73cc7e0 2024-02-02T15:30:10.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.970+0000 7f7c25200700 1 --2- 172.21.15.1:0/2040780900 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c6c73c1930 0x55c6c73cc2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:30:10.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.971+0000 7f7c1fe00700 1 --2- 172.21.15.1:0/2040780900 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6c73cc7e0 0x55c6c73cebd0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7c080d10f0 tx=0x7f7c0801c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bed851b6dc5b14d9 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:10.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.971+0000 7f7c16000700 1 -- 172.21.15.1:0/2040780900 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c080d5070 con 0x55c6c73cc7e0 2024-02-02T15:30:10.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.971+0000 7f7c16000700 1 -- 172.21.15.1:0/2040780900 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c080a0850 con 0x55c6c73cc7e0 2024-02-02T15:30:10.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.971+0000 7f7c16000700 1 -- 172.21.15.1:0/2040780900 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c080a4d20 con 0x55c6c73cc7e0 2024-02-02T15:30:10.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.972+0000 7f7c25c00700 1 --2- 172.21.15.1:0/2430713505 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f7c000a0710 0x7f7c000a2bd0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7c1800fb80 tx=0x7f7c18004b40 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:10.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.972+0000 7f7c3b7a8b80 1 -- 172.21.15.1:0/2040780900 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6c73cc7e0 msgr2=0x55c6c73cebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:10.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.972+0000 7f7c3b7a8b80 1 --2- 172.21.15.1:0/2040780900 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6c73cc7e0 0x55c6c73cebd0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7c080d10f0 tx=0x7f7c0801c000 comp rx=0 tx=0).stop 2024-02-02T15:30:10.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.972+0000 7f7c3b7a8b80 1 -- 172.21.15.1:0/2040780900 shutdown_connections 2024-02-02T15:30:10.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.972+0000 7f7c3b7a8b80 1 --2- 172.21.15.1:0/2040780900 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6c73cc7e0 0x55c6c73cebd0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:10.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.972+0000 7f7c3b7a8b80 1 --2- 172.21.15.1:0/2040780900 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c6c73c1010 0x55c6c73c13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:10.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.972+0000 7f7c3b7a8b80 1 --2- 172.21.15.1:0/2040780900 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c6c73c1930 0x55c6c73cc2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:10.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.972+0000 7f7c3b7a8b80 1 -- 172.21.15.1:0/2040780900 >> 172.21.15.1:0/2040780900 conn(0x55c6c7320db0 msgr2=0x55c6c73bd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:10.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.972+0000 7f7c3b7a8b80 1 -- 172.21.15.1:0/2040780900 shutdown_connections 2024-02-02T15:30:10.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.972+0000 7f7c3b7a8b80 1 -- 172.21.15.1:0/2040780900 wait complete. 2024-02-02T15:30:10.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.973+0000 7f7c3b7a8b80 1 -- start start 2024-02-02T15:30:10.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.973+0000 7f7c3b7a8b80 1 Processor -- start 2024-02-02T15:30:10.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.974+0000 7f7c3b7a8b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c6c73c1010 0x55c6c73cc100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:10.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.974+0000 7f7c3b7a8b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c6c73c1930 0x55c6c73c9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:10.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.974+0000 7f7c3b7a8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6c73cc7e0 0x55c6c73c9920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:10.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.974+0000 7f7c3b7a8b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6c7312d50 con 0x55c6c73c1010 2024-02-02T15:30:10.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.974+0000 7f7c3b7a8b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6c72aed80 con 0x55c6c73c1930 2024-02-02T15:30:10.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.974+0000 7f7c3b7a8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6c72aea20 con 0x55c6c73cc7e0 2024-02-02T15:30:10.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.974+0000 7f7c3b7a8b80 10 client.0 _ll_register_callbacks cb 0x55c6c72b0290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:30:10.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.974+0000 7f7c25200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c6c73c1010 0x55c6c73cc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:10.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.974+0000 7f7c25200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c6c73c1010 0x55c6c73cc100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54490/0 (socket says 172.21.15.1:54490) 2024-02-02T15:30:10.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.974+0000 7f7c25200700 1 -- 172.21.15.1:0/1202008802 learned_addr learned my addr 172.21.15.1:0/1202008802 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:30:10.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.974+0000 7f7c1fe00700 1 --2- 172.21.15.1:0/1202008802 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c6c73c1930 0x55c6c73c9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:10.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.975+0000 7f7c25c00700 1 --2- 172.21.15.1:0/1202008802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6c73cc7e0 0x55c6c73c9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:10.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.975+0000 7f7c1fe00700 1 -- 172.21.15.1:0/1202008802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6c73cc7e0 msgr2=0x55c6c73c9920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:10.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.975+0000 7f7c1fe00700 1 --2- 172.21.15.1:0/1202008802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6c73cc7e0 0x55c6c73c9920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:10.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.975+0000 7f7c1fe00700 1 -- 172.21.15.1:0/1202008802 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c6c73c1010 msgr2=0x55c6c73cc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:10.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.975+0000 7f7c1fe00700 1 --2- 172.21.15.1:0/1202008802 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c6c73c1010 0x55c6c73cc100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:10.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.975+0000 7f7c1fe00700 1 -- 172.21.15.1:0/1202008802 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c2010e5a0 con 0x55c6c73c1930 2024-02-02T15:30:10.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.975+0000 7f7c25200700 1 --2- 172.21.15.1:0/1202008802 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c6c73c1010 0x55c6c73cc100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:30:10.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.975+0000 7f7c1fe00700 1 --2- 172.21.15.1:0/1202008802 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c6c73c1930 0x55c6c73c9380 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f7c0809f240 tx=0x7f7c0809cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16c4a24eb3e5b0d8 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:10.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.975+0000 7f7c0ea00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:30:10.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.975+0000 7f7c0ea00700 1 -- 172.21.15.1:0/1202008802 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c080d5040 con 0x55c6c73c1930 2024-02-02T15:30:10.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.975+0000 7f7c0ea00700 1 -- 172.21.15.1:0/1202008802 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f7c080a15f0 con 0x55c6c73c1930 2024-02-02T15:30:10.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.975+0000 7f7c3b7a8b80 1 -- 172.21.15.1:0/1202008802 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c6c73b50f0 con 0x55c6c73c1930 2024-02-02T15:30:10.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.976+0000 7f7c0ea00700 1 -- 172.21.15.1:0/1202008802 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c080a8f60 con 0x55c6c73c1930 2024-02-02T15:30:10.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.976+0000 7f7c3b7a8b80 10 client.26474 fetch_fsmap learned FSMap version 31 2024-02-02T15:30:10.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.976+0000 7f7c3b7a8b80 1 -- 172.21.15.1:0/1202008802 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f7c20107d40 con 0x55c6c73c1930 2024-02-02T15:30:10.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.976+0000 7f7c0ea00700 1 -- 172.21.15.1:0/1202008802 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c08083020 con 0x55c6c73c1930 2024-02-02T15:30:10.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.976+0000 7f7c0ea00700 1 -- 172.21.15.1:0/1202008802 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 31) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f7c080a03e0 con 0x55c6c73c1930 2024-02-02T15:30:10.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.976+0000 7f7c3b7a8b80 10 client.26474 fetch_fsmap finished waiting for FSMap version 31 2024-02-02T15:30:10.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.976+0000 7f7c3b7a8b80 10 client.26474 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-02-02T15:30:10.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.976+0000 7f7c3b7a8b80 20 client.26474 populate_metadata read hostname 'smithi001' 2024-02-02T15:30:10.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.976+0000 7f7c3b7a8b80 1 --2- 172.21.15.1:0/1202008802 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55c6c73dbd90 0x55c6c73de1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:10.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.976+0000 7f7c3b7a8b80 4 client.26474 mds_command: new command op to 24514 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:30:10.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.976+0000 7f7c3b7a8b80 1 -- 172.21.15.1:0/1202008802 --> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c6c730bdd0 con 0x55c6c73dbd90 2024-02-02T15:30:10.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.977+0000 7f7c25200700 1 --2- 172.21.15.1:0/1202008802 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55c6c73dbd90 0x55c6c73de1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:10.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.977+0000 7f7c25200700 1 --2- 172.21.15.1:0/1202008802 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55c6c73dbd90 0x55c6c73de1b0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1528f1b791871ce6 server_cookie=66f97e5fefcb599f in_seq=0 out_seq=0 2024-02-02T15:30:10.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.977+0000 7f7c0ea00700 10 client.26474 ms_handle_connect on v2:172.21.15.142:6836/2870289019 2024-02-02T15:30:10.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.978+0000 7f7c0ea00700 1 -- 172.21.15.1:0/1202008802 <== mds.3 v2:172.21.15.142:6836/2870289019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55c6c730bdd0 con 0x55c6c73dbd90 2024-02-02T15:30:10.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.978+0000 7f7c0ea00700 10 client.26474 handle_command_reply: tid=0 2024-02-02T15:30:10.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.978+0000 7f7c3b7a8b80 1 client.26474 shutdown 2024-02-02T15:30:10.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.978+0000 7f7c3b7a8b80 1 -- 172.21.15.1:0/1202008802 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c6c73c1930 msgr2=0x55c6c73c9380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:10.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.978+0000 7f7c3b7a8b80 1 --2- 172.21.15.1:0/1202008802 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c6c73c1930 0x55c6c73c9380 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f7c0809f240 tx=0x7f7c0809cf90 comp rx=0 tx=0).stop 2024-02-02T15:30:10.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.978+0000 7f7c3b7a8b80 1 -- 172.21.15.1:0/1202008802 shutdown_connections 2024-02-02T15:30:10.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.978+0000 7f7c3b7a8b80 1 --2- 172.21.15.1:0/1202008802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6c73cc7e0 0x55c6c73c9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:10.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.978+0000 7f7c3b7a8b80 1 --2- 172.21.15.1:0/1202008802 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55c6c73dbd90 0x55c6c73de1b0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:10.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.978+0000 7f7c3b7a8b80 1 --2- 172.21.15.1:0/1202008802 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c6c73c1010 0x55c6c73cc100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:10.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.978+0000 7f7c3b7a8b80 1 --2- 172.21.15.1:0/1202008802 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c6c73c1930 0x55c6c73c9380 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:10.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.978+0000 7f7c0ea00700 0 client.26474 ms_handle_reset on v2:172.21.15.142:6836/2870289019 2024-02-02T15:30:10.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.978+0000 7f7c3b7a8b80 1 -- 172.21.15.1:0/1202008802 >> 172.21.15.1:0/1202008802 conn(0x55c6c7320db0 msgr2=0x55c6c73bcf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:10.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.978+0000 7f7c3b7a8b80 1 -- 172.21.15.1:0/1202008802 shutdown_connections 2024-02-02T15:30:10.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.978+0000 7f7c3b7a8b80 1 -- 172.21.15.1:0/1202008802 wait complete. 2024-02-02T15:30:10.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.979+0000 7f7c3b7a8b80 20 client.26474 trim_cache size 0 max 16384 2024-02-02T15:30:10.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.993+0000 7f7c3b7a8b80 1 Processor -- start 2024-02-02T15:30:10.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.994+0000 7f7c3b7a8b80 1 -- start start 2024-02-02T15:30:10.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.994+0000 7f7c3b7a8b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c6c73c1010 0x55c6c73cbca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:10.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.994+0000 7f7c3b7a8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6c73c1930 0x55c6c73c8b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:10.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.994+0000 7f7c3b7a8b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c6c73cc7e0 0x55c6c73c9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:10.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.994+0000 7f7c3b7a8b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6c71966a0 con 0x55c6c73c1010 2024-02-02T15:30:10.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.994+0000 7f7c3b7a8b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6c73bc1a0 con 0x55c6c73cc7e0 2024-02-02T15:30:10.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.994+0000 7f7c3b7a8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6c73c51d0 con 0x55c6c73c1930 2024-02-02T15:30:10.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.994+0000 7f7c1fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6c73c1930 0x55c6c73c8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:10.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.994+0000 7f7c25c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c6c73cc7e0 0x55c6c73c9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:10.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.994+0000 7f7c25200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c6c73c1010 0x55c6c73cbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:10.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.994+0000 7f7c1fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6c73c1930 0x55c6c73c8b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:43218/0 (socket says 172.21.15.1:43218) 2024-02-02T15:30:10.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.994+0000 7f7c1fe00700 1 -- 172.21.15.1:0/2129758252 learned_addr learned my addr 172.21.15.1:0/2129758252 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:30:10.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.994+0000 7f7c25c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c6c73cc7e0 0x55c6c73c9080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59956/0 (socket says 172.21.15.1:59956) 2024-02-02T15:30:10.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.994+0000 7f7c1fe00700 1 -- 172.21.15.1:0/2129758252 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c6c73cc7e0 msgr2=0x55c6c73c9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:10.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.994+0000 7f7c1fe00700 1 --2- 172.21.15.1:0/2129758252 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c6c73cc7e0 0x55c6c73c9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:10.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.994+0000 7f7c1fe00700 1 -- 172.21.15.1:0/2129758252 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c6c73c1010 msgr2=0x55c6c73cbca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:10.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.994+0000 7f7c1fe00700 1 --2- 172.21.15.1:0/2129758252 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c6c73c1010 0x55c6c73cbca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:10.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.995+0000 7f7c1fe00700 1 -- 172.21.15.1:0/2129758252 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6c7140b60 con 0x55c6c73c1930 2024-02-02T15:30:10.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.995+0000 7f7c1fe00700 1 --2- 172.21.15.1:0/2129758252 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6c73c1930 0x55c6c73c8b40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7c080d1180 tx=0x7f7c0801c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80afbcbb20628e99 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:10.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.995+0000 7f7c16a00700 1 -- 172.21.15.1:0/2129758252 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c080a0cc0 con 0x55c6c73c1930 2024-02-02T15:30:10.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.995+0000 7f7c16a00700 1 -- 172.21.15.1:0/2129758252 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c08083070 con 0x55c6c73c1930 2024-02-02T15:30:10.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.995+0000 7f7c16a00700 1 -- 172.21.15.1:0/2129758252 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c080a1360 con 0x55c6c73c1930 2024-02-02T15:30:10.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.996+0000 7f7c3b7a8b80 1 -- 172.21.15.1:0/2129758252 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6c73c1930 msgr2=0x55c6c73c8b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:10.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.996+0000 7f7c3b7a8b80 1 --2- 172.21.15.1:0/2129758252 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6c73c1930 0x55c6c73c8b40 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7c080d1180 tx=0x7f7c0801c000 comp rx=0 tx=0).stop 2024-02-02T15:30:10.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.996+0000 7f7c3b7a8b80 1 -- 172.21.15.1:0/2129758252 shutdown_connections 2024-02-02T15:30:10.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.996+0000 7f7c3b7a8b80 1 --2- 172.21.15.1:0/2129758252 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c6c73cc7e0 0x55c6c73c9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:10.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.996+0000 7f7c3b7a8b80 1 --2- 172.21.15.1:0/2129758252 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c6c73c1010 0x55c6c73cbca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:10.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.996+0000 7f7c3b7a8b80 1 --2- 172.21.15.1:0/2129758252 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6c73c1930 0x55c6c73c8b40 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:10.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.996+0000 7f7c3b7a8b80 1 -- 172.21.15.1:0/2129758252 >> 172.21.15.1:0/2129758252 conn(0x55c6c7320db0 msgr2=0x55c6c73c4660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:10.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.996+0000 7f7c3b7a8b80 1 -- 172.21.15.1:0/2129758252 shutdown_connections 2024-02-02T15:30:10.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.996+0000 7f7c3b7a8b80 1 -- 172.21.15.1:0/2129758252 wait complete. 2024-02-02T15:30:10.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.996+0000 7f7c3b7a8b80 1 -- start start 2024-02-02T15:30:10.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.997+0000 7f7c3b7a8b80 1 Processor -- start 2024-02-02T15:30:10.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.997+0000 7f7c3b7a8b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c6c73c1010 0x55c6c73d0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:10.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.997+0000 7f7c3b7a8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6c73c1930 0x55c6c73d0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:10.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.997+0000 7f7c3b7a8b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c6c73cc7e0 0x55c6c73d1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:10.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.997+0000 7f7c3b7a8b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6c742c870 con 0x55c6c73c1010 2024-02-02T15:30:10.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.997+0000 7f7c3b7a8b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6c742c9b0 con 0x55c6c73cc7e0 2024-02-02T15:30:10.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.997+0000 7f7c3b7a8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6c742cb40 con 0x55c6c73c1930 2024-02-02T15:30:10.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.997+0000 7f7c3b7a8b80 10 client.0 _ll_register_callbacks cb 0x55c6c72b0290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:30:10.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.997+0000 7f7c1fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6c73c1930 0x55c6c73d0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:10.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.997+0000 7f7c25200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c6c73c1010 0x55c6c73d0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:10.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.997+0000 7f7c1fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6c73c1930 0x55c6c73d0d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:43228/0 (socket says 172.21.15.1:43228) 2024-02-02T15:30:10.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.997+0000 7f7c1fe00700 1 -- 172.21.15.1:0/2003977643 learned_addr learned my addr 172.21.15.1:0/2003977643 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:30:10.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.998+0000 7f7c25c00700 1 --2- 172.21.15.1:0/2003977643 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c6c73cc7e0 0x55c6c73d1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:10.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.998+0000 7f7c1fe00700 1 -- 172.21.15.1:0/2003977643 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c6c73cc7e0 msgr2=0x55c6c73d1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:10.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.998+0000 7f7c1fe00700 1 --2- 172.21.15.1:0/2003977643 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c6c73cc7e0 0x55c6c73d1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:10.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.998+0000 7f7c1fe00700 1 -- 172.21.15.1:0/2003977643 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c6c73c1010 msgr2=0x55c6c73d0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:11.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.998+0000 7f7c1fe00700 1 --2- 172.21.15.1:0/2003977643 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c6c73c1010 0x55c6c73d0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.998+0000 7f7c1fe00700 1 -- 172.21.15.1:0/2003977643 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c20107d40 con 0x55c6c73c1930 2024-02-02T15:30:11.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.998+0000 7f7c1fe00700 1 --2- 172.21.15.1:0/2003977643 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6c73c1930 0x55c6c73d0d40 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7c0809f240 tx=0x7f7c0809cf70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3a1063c32c03cf8 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:11.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.998+0000 7f7c0ea00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:30:11.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.998+0000 7f7c0ea00700 1 -- 172.21.15.1:0/2003977643 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c080a8f60 con 0x55c6c73c1930 2024-02-02T15:30:11.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.999+0000 7f7c0ea00700 1 -- 172.21.15.1:0/2003977643 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f7c08084080 con 0x55c6c73c1930 2024-02-02T15:30:11.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.999+0000 7f7c3b7a8b80 1 -- 172.21.15.1:0/2003977643 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7c080a0850 con 0x55c6c73c1930 2024-02-02T15:30:11.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.999+0000 7f7c0ea00700 1 -- 172.21.15.1:0/2003977643 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c080a4d20 con 0x55c6c73c1930 2024-02-02T15:30:11.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.999+0000 7f7c3b7a8b80 10 client.26467 fetch_fsmap learned FSMap version 31 2024-02-02T15:30:11.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.999+0000 7f7c3b7a8b80 1 -- 172.21.15.1:0/2003977643 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55c6c72d8ea0 con 0x55c6c73c1930 2024-02-02T15:30:11.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:10.999+0000 7f7c0ea00700 1 -- 172.21.15.1:0/2003977643 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c080a0850 con 0x55c6c73c1930 2024-02-02T15:30:11.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.000+0000 7f7c0ea00700 1 -- 172.21.15.1:0/2003977643 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 31) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f7c0809fe50 con 0x55c6c73c1930 2024-02-02T15:30:11.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.000+0000 7f7c3b7a8b80 10 client.26467 fetch_fsmap finished waiting for FSMap version 31 2024-02-02T15:30:11.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.000+0000 7f7c3b7a8b80 10 client.26467 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-02-02T15:30:11.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.000+0000 7f7c3b7a8b80 20 client.26467 populate_metadata read hostname 'smithi001' 2024-02-02T15:30:11.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.000+0000 7f7c3b7a8b80 1 --2- 172.21.15.1:0/2003977643 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55c6c742dd70 0x55c6c7430160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:11.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.000+0000 7f7c3b7a8b80 4 client.26467 mds_command: new command op to 24514 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:30:11.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.000+0000 7f7c3b7a8b80 1 -- 172.21.15.1:0/2003977643 --> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c6c72d1f50 con 0x55c6c742dd70 2024-02-02T15:30:11.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.000+0000 7f7c25200700 1 --2- 172.21.15.1:0/2003977643 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55c6c742dd70 0x55c6c7430160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:11.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.001+0000 7f7c25200700 1 --2- 172.21.15.1:0/2003977643 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55c6c742dd70 0x55c6c7430160 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1e1fc119f4fe8011 server_cookie=409eb20762fa53e7 in_seq=0 out_seq=0 2024-02-02T15:30:11.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.001+0000 7f7c0ea00700 10 client.26467 ms_handle_connect on v2:172.21.15.142:6836/2870289019 2024-02-02T15:30:11.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.001+0000 7f7c0ea00700 1 -- 172.21.15.1:0/2003977643 <== mds.3 v2:172.21.15.142:6836/2870289019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c6c72d1f50 con 0x55c6c742dd70 2024-02-02T15:30:11.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.001+0000 7f7c0ea00700 10 client.26467 handle_command_reply: tid=0 2024-02-02T15:30:11.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.001+0000 7f7c3b7a8b80 1 client.26467 shutdown 2024-02-02T15:30:11.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.001+0000 7f7c3b7a8b80 1 -- 172.21.15.1:0/2003977643 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6c73c1930 msgr2=0x55c6c73d0d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:11.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.001+0000 7f7c3b7a8b80 1 --2- 172.21.15.1:0/2003977643 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6c73c1930 0x55c6c73d0d40 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7c0809f240 tx=0x7f7c0809cf70 comp rx=0 tx=0).stop 2024-02-02T15:30:11.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.001+0000 7f7c3b7a8b80 1 -- 172.21.15.1:0/2003977643 shutdown_connections 2024-02-02T15:30:11.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.001+0000 7f7c3b7a8b80 1 --2- 172.21.15.1:0/2003977643 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c6c73cc7e0 0x55c6c73d1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.001+0000 7f7c3b7a8b80 1 --2- 172.21.15.1:0/2003977643 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55c6c742dd70 0x55c6c7430160 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.001+0000 7f7c3b7a8b80 1 --2- 172.21.15.1:0/2003977643 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c6c73c1010 0x55c6c73d0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.001+0000 7f7c3b7a8b80 1 --2- 172.21.15.1:0/2003977643 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c6c73c1930 0x55c6c73d0d40 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.001+0000 7f7c0ea00700 0 client.26467 ms_handle_reset on v2:172.21.15.142:6836/2870289019 2024-02-02T15:30:11.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.001+0000 7f7c3b7a8b80 1 -- 172.21.15.1:0/2003977643 >> 172.21.15.1:0/2003977643 conn(0x55c6c7320db0 msgr2=0x55c6c73c2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:11.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.001+0000 7f7c3b7a8b80 1 -- 172.21.15.1:0/2003977643 shutdown_connections 2024-02-02T15:30:11.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.002+0000 7f7c3b7a8b80 1 -- 172.21.15.1:0/2003977643 wait complete. 2024-02-02T15:30:11.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.002+0000 7f7c3b7a8b80 20 client.26467 trim_cache size 0 max 16384 2024-02-02T15:30:11.004 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:30:11.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.002+0000 7f7c27c00700 1 -- 172.21.15.1:0/2430713505 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f7c000a0710 msgr2=0x7f7c000a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:11.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.002+0000 7f7c27c00700 1 --2- 172.21.15.1:0/2430713505 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f7c000a0710 0x7f7c000a2bd0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7c1800fb80 tx=0x7f7c18004b40 comp rx=0 tx=0).stop 2024-02-02T15:30:11.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.002+0000 7f7c27c00700 1 -- 172.21.15.1:0/2430713505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c200689b0 msgr2=0x7f7c20199740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:11.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.002+0000 7f7c27c00700 1 --2- 172.21.15.1:0/2430713505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c200689b0 0x7f7c20199740 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f7c0800b570 tx=0x7f7c0800b880 comp rx=0 tx=0).stop 2024-02-02T15:30:11.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.003+0000 7f7c27c00700 1 -- 172.21.15.1:0/2430713505 shutdown_connections 2024-02-02T15:30:11.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.003+0000 7f7c27c00700 1 --2- 172.21.15.1:0/2430713505 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f7c000a0710 0x7f7c000a2bd0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.003+0000 7f7c27c00700 1 --2- 172.21.15.1:0/2430713505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c20106ed0 0x7f7c2019dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.003+0000 7f7c27c00700 1 --2- 172.21.15.1:0/2430713505 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7c20068010 0x7f7c20199200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.003+0000 7f7c27c00700 1 --2- 172.21.15.1:0/2430713505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c200689b0 0x7f7c20199740 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.003+0000 7f7c27c00700 1 -- 172.21.15.1:0/2430713505 >> 172.21.15.1:0/2430713505 conn(0x7f7c200757d0 msgr2=0x7f7c2010bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:11.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.003+0000 7f7c27c00700 1 -- 172.21.15.1:0/2430713505 shutdown_connections 2024-02-02T15:30:11.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.003+0000 7f7c27c00700 1 -- 172.21.15.1:0/2430713505 wait complete. 2024-02-02T15:30:11.024 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-02T15:30:11.226 INFO:tasks.workunit.client.0.smithi001.stdout: 1 117085 6.19 MB/sec warmup 105 sec latency 12.494 ms 2024-02-02T15:30:11.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.241+0000 7f6cb9200700 1 -- 172.21.15.1:0/1804544502 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6cb4106110 msgr2=0x7f6cb410dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:11.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.241+0000 7f6cb9200700 1 --2- 172.21.15.1:0/1804544502 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6cb4106110 0x7f6cb410dc90 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6cac00b0a0 tx=0x7f6cac00b3b0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.241+0000 7f6cb9200700 1 -- 172.21.15.1:0/1804544502 shutdown_connections 2024-02-02T15:30:11.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.241+0000 7f6cb9200700 1 --2- 172.21.15.1:0/1804544502 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6cb4106110 0x7f6cb410dc90 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.241+0000 7f6cb9200700 1 --2- 172.21.15.1:0/1804544502 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6cb4100ff0 0x7f6cb41013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.241+0000 7f6cb9200700 1 --2- 172.21.15.1:0/1804544502 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6cb41019a0 0x7f6cb41059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.241+0000 7f6cb9200700 1 -- 172.21.15.1:0/1804544502 >> 172.21.15.1:0/1804544502 conn(0x7f6cb40fc810 msgr2=0x7f6cb40fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:11.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.241+0000 7f6cb9200700 1 -- 172.21.15.1:0/1804544502 shutdown_connections 2024-02-02T15:30:11.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.241+0000 7f6cb9200700 1 -- 172.21.15.1:0/1804544502 wait complete. 2024-02-02T15:30:11.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.242+0000 7f6cb9200700 1 Processor -- start 2024-02-02T15:30:11.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.242+0000 7f6cb9200700 1 -- start start 2024-02-02T15:30:11.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.242+0000 7f6cb9200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6cb4100ff0 0x7f6cb4198fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:11.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.243+0000 7f6cb9200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6cb4106110 0x7f6cb41994f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:11.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.243+0000 7f6cb9200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6cb419d950 0x7f6cb419ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:11.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.243+0000 7f6cb9200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb419e3d0 con 0x7f6cb4106110 2024-02-02T15:30:11.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.243+0000 7f6cb9200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb419e540 con 0x7f6cb4100ff0 2024-02-02T15:30:11.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.243+0000 7f6cb9200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb419e700 con 0x7f6cb419d950 2024-02-02T15:30:11.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.243+0000 7f6cb1e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6cb4100ff0 0x7f6cb4198fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:11.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.243+0000 7f6cb1400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6cb4106110 0x7f6cb41994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:11.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.243+0000 7f6cb1e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6cb4100ff0 0x7f6cb4198fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59980/0 (socket says 172.21.15.1:59980) 2024-02-02T15:30:11.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.243+0000 7f6cb2800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6cb419d950 0x7f6cb419ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:11.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.243+0000 7f6cb1e00700 1 -- 172.21.15.1:0/3349534465 learned_addr learned my addr 172.21.15.1:0/3349534465 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:30:11.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.243+0000 7f6cb1e00700 1 -- 172.21.15.1:0/3349534465 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6cb419d950 msgr2=0x7f6cb419ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:11.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.243+0000 7f6cb1e00700 1 --2- 172.21.15.1:0/3349534465 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6cb419d950 0x7f6cb419ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.243+0000 7f6cb1e00700 1 -- 172.21.15.1:0/3349534465 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6cb4106110 msgr2=0x7f6cb41994f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:11.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.243+0000 7f6cb1e00700 1 --2- 172.21.15.1:0/3349534465 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6cb4106110 0x7f6cb41994f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.243+0000 7f6cb1e00700 1 -- 172.21.15.1:0/3349534465 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cac009d00 con 0x7f6cb4100ff0 2024-02-02T15:30:11.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.243+0000 7f6cb2800700 1 --2- 172.21.15.1:0/3349534465 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6cb419d950 0x7f6cb419ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:30:11.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.243+0000 7f6cb1e00700 1 --2- 172.21.15.1:0/3349534465 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6cb4100ff0 0x7f6cb4198fb0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6ca400d950 tx=0x7f6ca400dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:11.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.244+0000 7f6caaa00700 1 -- 172.21.15.1:0/3349534465 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca400f890 con 0x7f6cb4100ff0 2024-02-02T15:30:11.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.244+0000 7f6cb9200700 1 -- 172.21.15.1:0/3349534465 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cb41a5c60 con 0x7f6cb4100ff0 2024-02-02T15:30:11.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.244+0000 7f6caaa00700 1 -- 172.21.15.1:0/3349534465 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ca4010460 con 0x7f6cb4100ff0 2024-02-02T15:30:11.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.244+0000 7f6caaa00700 1 -- 172.21.15.1:0/3349534465 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca400e610 con 0x7f6cb4100ff0 2024-02-02T15:30:11.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.244+0000 7f6cb9200700 1 -- 172.21.15.1:0/3349534465 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cb41a6190 con 0x7f6cb4100ff0 2024-02-02T15:30:11.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.247+0000 7f6caaa00700 1 -- 172.21.15.1:0/3349534465 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 27) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f6ca40108d0 con 0x7f6cb4100ff0 2024-02-02T15:30:11.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.248+0000 7f6cccc44b80 1 Processor -- start 2024-02-02T15:30:11.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.249+0000 7f6caaa00700 1 --2- 172.21.15.1:0/3349534465 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f6c940a0710 0x7f6c940a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:11.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.249+0000 7f6cccc44b80 1 -- start start 2024-02-02T15:30:11.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.249+0000 7f6caaa00700 1 -- 172.21.15.1:0/3349534465 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f6ca40cbe30 con 0x7f6cb4100ff0 2024-02-02T15:30:11.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.249+0000 7f6cb2800700 1 --2- 172.21.15.1:0/3349534465 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f6c940a0710 0x7f6c940a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:11.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.249+0000 7f6cccc44b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557a56682010 0x557a566823f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:11.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.249+0000 7f6cccc44b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557a56682930 0x557a5668d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:11.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.249+0000 7f6cccc44b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a5668d7e0 0x557a5668fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:11.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.249+0000 7f6cccc44b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a565f64c0 con 0x557a56682010 2024-02-02T15:30:11.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.249+0000 7f6cccc44b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a565daaa0 con 0x557a56682930 2024-02-02T15:30:11.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.249+0000 7f6cccc44b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a565d8460 con 0x557a5668d7e0 2024-02-02T15:30:11.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.249+0000 7f6cb1400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557a56682930 0x557a5668d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:11.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.249+0000 7f6cb1e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557a56682010 0x557a566823f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:11.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.249+0000 7f6cb1400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557a56682930 0x557a5668d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59986/0 (socket says 172.21.15.1:59986) 2024-02-02T15:30:11.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.249+0000 7f6cb2800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a5668d7e0 0x557a5668fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:11.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.249+0000 7f6cb1400700 1 -- 172.21.15.1:0/2673362438 learned_addr learned my addr 172.21.15.1:0/2673362438 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:30:11.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.250+0000 7f6cb1400700 1 -- 172.21.15.1:0/2673362438 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a5668d7e0 msgr2=0x557a5668fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:11.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.250+0000 7f6cb1400700 1 --2- 172.21.15.1:0/2673362438 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a5668d7e0 0x557a5668fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.250+0000 7f6cb1400700 1 -- 172.21.15.1:0/2673362438 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557a56682010 msgr2=0x557a566823f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:11.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.250+0000 7f6cb1400700 1 --2- 172.21.15.1:0/2673362438 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557a56682010 0x557a566823f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.250+0000 7f6cb1400700 1 -- 172.21.15.1:0/2673362438 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a564553b0 con 0x557a56682930 2024-02-02T15:30:11.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.250+0000 7f6cb2800700 1 --2- 172.21.15.1:0/2673362438 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a5668d7e0 0x557a5668fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:30:11.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.250+0000 7f6cb1e00700 1 --2- 172.21.15.1:0/2673362438 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557a56682010 0x557a566823f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:30:11.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.250+0000 7f6cb2800700 1 --2- 172.21.15.1:0/3349534465 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f6c940a0710 0x7f6c940a2bd0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6cac009040 tx=0x7f6cac00bf40 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:11.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.250+0000 7f6cb1400700 1 --2- 172.21.15.1:0/2673362438 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557a56682930 0x557a5668d2a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c009ce0 tx=0x7f6c9c011440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48655d5ccc14a318 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:11.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.250+0000 7f6ca3400700 1 -- 172.21.15.1:0/2673362438 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c011d20 con 0x557a56682930 2024-02-02T15:30:11.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.250+0000 7f6ca3400700 1 -- 172.21.15.1:0/2673362438 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c9c011e80 con 0x557a56682930 2024-02-02T15:30:11.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.250+0000 7f6ca3400700 1 -- 172.21.15.1:0/2673362438 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c005240 con 0x557a56682930 2024-02-02T15:30:11.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.251+0000 7f6cccc44b80 1 -- 172.21.15.1:0/2673362438 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557a56682930 msgr2=0x557a5668d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:11.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.251+0000 7f6cccc44b80 1 --2- 172.21.15.1:0/2673362438 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557a56682930 0x557a5668d2a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c009ce0 tx=0x7f6c9c011440 comp rx=0 tx=0).stop 2024-02-02T15:30:11.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.251+0000 7f6cccc44b80 1 -- 172.21.15.1:0/2673362438 shutdown_connections 2024-02-02T15:30:11.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.251+0000 7f6cccc44b80 1 --2- 172.21.15.1:0/2673362438 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a5668d7e0 0x557a5668fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.251+0000 7f6cccc44b80 1 --2- 172.21.15.1:0/2673362438 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557a56682010 0x557a566823f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.251+0000 7f6cccc44b80 1 --2- 172.21.15.1:0/2673362438 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557a56682930 0x557a5668d2a0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.251+0000 7f6cccc44b80 1 -- 172.21.15.1:0/2673362438 >> 172.21.15.1:0/2673362438 conn(0x557a565e1db0 msgr2=0x557a5667e3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:11.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.251+0000 7f6cccc44b80 1 -- 172.21.15.1:0/2673362438 shutdown_connections 2024-02-02T15:30:11.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.251+0000 7f6cccc44b80 1 -- 172.21.15.1:0/2673362438 wait complete. 2024-02-02T15:30:11.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.252+0000 7f6cccc44b80 1 -- start start 2024-02-02T15:30:11.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.252+0000 7f6cccc44b80 1 Processor -- start 2024-02-02T15:30:11.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.252+0000 7f6cccc44b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557a56682010 0x557a5668d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:11.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.253+0000 7f6cccc44b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557a56682930 0x557a5668a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:11.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.253+0000 7f6cccc44b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a5668d7e0 0x557a5668a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:11.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.253+0000 7f6cccc44b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a565d3d50 con 0x557a56682930 2024-02-02T15:30:11.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.253+0000 7f6cccc44b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a5656fd80 con 0x557a56682010 2024-02-02T15:30:11.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.253+0000 7f6cccc44b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a5656fa20 con 0x557a5668d7e0 2024-02-02T15:30:11.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.253+0000 7f6cccc44b80 10 client.0 _ll_register_callbacks cb 0x557a56571290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:30:11.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.253+0000 7f6cb1e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557a56682010 0x557a5668d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:11.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.253+0000 7f6cb1e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557a56682010 0x557a5668d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59994/0 (socket says 172.21.15.1:59994) 2024-02-02T15:30:11.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.253+0000 7f6cb1e00700 1 -- 172.21.15.1:0/3812766192 learned_addr learned my addr 172.21.15.1:0/3812766192 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:30:11.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.253+0000 7f6cb1400700 1 --2- 172.21.15.1:0/3812766192 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557a56682930 0x557a5668a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:11.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.253+0000 7f6cb2800700 1 --2- 172.21.15.1:0/3812766192 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a5668d7e0 0x557a5668a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:11.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.253+0000 7f6cb1e00700 1 -- 172.21.15.1:0/3812766192 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a5668d7e0 msgr2=0x557a5668a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:11.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.253+0000 7f6cb1e00700 1 --2- 172.21.15.1:0/3812766192 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a5668d7e0 0x557a5668a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.253+0000 7f6cb1e00700 1 -- 172.21.15.1:0/3812766192 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557a56682930 msgr2=0x557a5668a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:11.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.253+0000 7f6cb1e00700 1 --2- 172.21.15.1:0/3812766192 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557a56682930 0x557a5668a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.253+0000 7f6cb1e00700 1 -- 172.21.15.1:0/3812766192 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cb41a6190 con 0x557a56682010 2024-02-02T15:30:11.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.253+0000 7f6cb1e00700 1 --2- 172.21.15.1:0/3812766192 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557a56682010 0x557a5668d130 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f6ca400b2f0 tx=0x7f6ca4099020 comp rx=0 tx=0).ready entity=mon.2 client_cookie=415b99a4441a9f0b server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:11.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.253+0000 7f6c9be00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:30:11.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.254+0000 7f6c9be00700 1 -- 172.21.15.1:0/3812766192 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca4099400 con 0x557a56682010 2024-02-02T15:30:11.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.254+0000 7f6c9be00700 1 -- 172.21.15.1:0/3812766192 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f6ca4098490 con 0x557a56682010 2024-02-02T15:30:11.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.254+0000 7f6cccc44b80 1 -- 172.21.15.1:0/3812766192 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557a565ccdd0 con 0x557a56682010 2024-02-02T15:30:11.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.254+0000 7f6c9be00700 1 -- 172.21.15.1:0/3812766192 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ca4098650 con 0x557a56682010 2024-02-02T15:30:11.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.254+0000 7f6cccc44b80 10 client.26492 fetch_fsmap learned FSMap version 31 2024-02-02T15:30:11.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.254+0000 7f6cccc44b80 1 -- 172.21.15.1:0/3812766192 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f6c9c0098e0 con 0x557a56682010 2024-02-02T15:30:11.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.254+0000 7f6c9be00700 1 -- 172.21.15.1:0/3812766192 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557a565ccdd0 con 0x557a56682010 2024-02-02T15:30:11.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.254+0000 7f6c9be00700 1 -- 172.21.15.1:0/3812766192 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 31) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f6ca4093bb0 con 0x557a56682010 2024-02-02T15:30:11.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.255+0000 7f6cccc44b80 10 client.26492 fetch_fsmap finished waiting for FSMap version 31 2024-02-02T15:30:11.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.255+0000 7f6cccc44b80 10 client.26492 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-02-02T15:30:11.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.255+0000 7f6cccc44b80 20 client.26492 populate_metadata read hostname 'smithi001' 2024-02-02T15:30:11.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.255+0000 7f6cccc44b80 1 --2- 172.21.15.1:0/3812766192 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x557a5669cf30 0x557a5669f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:11.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.255+0000 7f6cccc44b80 4 client.26492 mds_command: new command op to 24518 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:30:11.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.255+0000 7f6cccc44b80 1 -- 172.21.15.1:0/3812766192 --> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557a56592f50 con 0x557a5669cf30 2024-02-02T15:30:11.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.255+0000 7f6cb1400700 1 --2- 172.21.15.1:0/3812766192 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x557a5669cf30 0x557a5669f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:11.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.255+0000 7f6cb1400700 1 --2- 172.21.15.1:0/3812766192 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x557a5669cf30 0x557a5669f350 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=76c477bb5e1150c5 server_cookie=2a1bba155c3d7935 in_seq=0 out_seq=0 2024-02-02T15:30:11.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.255+0000 7f6c9be00700 10 client.26492 ms_handle_connect on v2:172.21.15.71:6836/628016199 2024-02-02T15:30:11.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.256+0000 7f6c9be00700 1 -- 172.21.15.1:0/3812766192 <== mds.2 v2:172.21.15.71:6836/628016199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x557a56592f50 con 0x557a5669cf30 2024-02-02T15:30:11.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.256+0000 7f6c9be00700 10 client.26492 handle_command_reply: tid=0 2024-02-02T15:30:11.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.256+0000 7f6cccc44b80 1 client.26492 shutdown 2024-02-02T15:30:11.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.257+0000 7f6cccc44b80 1 -- 172.21.15.1:0/3812766192 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557a56682010 msgr2=0x557a5668d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:11.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.257+0000 7f6cccc44b80 1 --2- 172.21.15.1:0/3812766192 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557a56682010 0x557a5668d130 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f6ca400b2f0 tx=0x7f6ca4099020 comp rx=0 tx=0).stop 2024-02-02T15:30:11.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.257+0000 7f6cccc44b80 1 -- 172.21.15.1:0/3812766192 shutdown_connections 2024-02-02T15:30:11.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.257+0000 7f6cccc44b80 1 --2- 172.21.15.1:0/3812766192 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x557a5669cf30 0x557a5669f350 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.257+0000 7f6cccc44b80 1 --2- 172.21.15.1:0/3812766192 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a5668d7e0 0x557a5668a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.257+0000 7f6cccc44b80 1 --2- 172.21.15.1:0/3812766192 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557a56682010 0x557a5668d130 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.257+0000 7f6cccc44b80 1 --2- 172.21.15.1:0/3812766192 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557a56682930 0x557a5668a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.257+0000 7f6c9be00700 0 client.26492 ms_handle_reset on v2:172.21.15.71:6836/628016199 2024-02-02T15:30:11.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.257+0000 7f6cccc44b80 1 -- 172.21.15.1:0/3812766192 >> 172.21.15.1:0/3812766192 conn(0x557a565e1db0 msgr2=0x557a5667dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:11.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.257+0000 7f6cccc44b80 1 -- 172.21.15.1:0/3812766192 shutdown_connections 2024-02-02T15:30:11.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.257+0000 7f6cccc44b80 1 -- 172.21.15.1:0/3812766192 wait complete. 2024-02-02T15:30:11.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.257+0000 7f6cccc44b80 20 client.26492 trim_cache size 0 max 16384 2024-02-02T15:30:11.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.271+0000 7f6cccc44b80 1 Processor -- start 2024-02-02T15:30:11.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.271+0000 7f6cccc44b80 1 -- start start 2024-02-02T15:30:11.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.272+0000 7f6cccc44b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557a56682010 0x557a56694320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:11.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.272+0000 7f6cccc44b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a56682930 0x557a5667cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:11.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.272+0000 7f6cccc44b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557a5668d7e0 0x557a56689380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:11.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.272+0000 7f6cccc44b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a564576a0 con 0x557a56682010 2024-02-02T15:30:11.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.272+0000 7f6cccc44b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a5667f060 con 0x557a5668d7e0 2024-02-02T15:30:11.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.272+0000 7f6cccc44b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a566861b0 con 0x557a56682930 2024-02-02T15:30:11.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.272+0000 7f6cb1e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557a56682010 0x557a56694320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:11.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.272+0000 7f6cb2800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557a5668d7e0 0x557a56689380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:11.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.272+0000 7f6cb1400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a56682930 0x557a5667cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:11.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.272+0000 7f6cb2800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557a5668d7e0 0x557a56689380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:60004/0 (socket says 172.21.15.1:60004) 2024-02-02T15:30:11.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.272+0000 7f6cb2800700 1 -- 172.21.15.1:0/1870038993 learned_addr learned my addr 172.21.15.1:0/1870038993 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:30:11.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.272+0000 7f6cb1400700 1 -- 172.21.15.1:0/1870038993 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557a5668d7e0 msgr2=0x557a56689380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:11.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.272+0000 7f6cb1400700 1 --2- 172.21.15.1:0/1870038993 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557a5668d7e0 0x557a56689380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.272+0000 7f6cb1400700 1 -- 172.21.15.1:0/1870038993 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557a56682010 msgr2=0x557a56694320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:11.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.272+0000 7f6cb1400700 1 --2- 172.21.15.1:0/1870038993 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557a56682010 0x557a56694320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.272+0000 7f6cb1400700 1 -- 172.21.15.1:0/1870038993 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a56401b60 con 0x557a56682930 2024-02-02T15:30:11.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.272+0000 7f6cb2800700 1 --2- 172.21.15.1:0/1870038993 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557a5668d7e0 0x557a56689380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:30:11.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.272+0000 7f6cb1e00700 1 --2- 172.21.15.1:0/1870038993 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557a56682010 0x557a56694320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:30:11.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.273+0000 7f6cb1400700 1 --2- 172.21.15.1:0/1870038993 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a56682930 0x557a5667cb20 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c004ce0 tx=0x7f6c9c009730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87eaaed2e14ed8f4 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:11.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.273+0000 7f6ca3e00700 1 -- 172.21.15.1:0/1870038993 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c004870 con 0x557a56682930 2024-02-02T15:30:11.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.273+0000 7f6ca3e00700 1 -- 172.21.15.1:0/1870038993 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c9c004ab0 con 0x557a56682930 2024-02-02T15:30:11.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.273+0000 7f6ca3e00700 1 -- 172.21.15.1:0/1870038993 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c0038e0 con 0x557a56682930 2024-02-02T15:30:11.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.274+0000 7f6cccc44b80 1 -- 172.21.15.1:0/1870038993 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a56682930 msgr2=0x557a5667cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:11.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.274+0000 7f6cccc44b80 1 --2- 172.21.15.1:0/1870038993 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a56682930 0x557a5667cb20 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c004ce0 tx=0x7f6c9c009730 comp rx=0 tx=0).stop 2024-02-02T15:30:11.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.274+0000 7f6cccc44b80 1 -- 172.21.15.1:0/1870038993 shutdown_connections 2024-02-02T15:30:11.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.274+0000 7f6cccc44b80 1 --2- 172.21.15.1:0/1870038993 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557a5668d7e0 0x557a56689380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.274+0000 7f6cccc44b80 1 --2- 172.21.15.1:0/1870038993 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557a56682010 0x557a56694320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.274+0000 7f6cccc44b80 1 --2- 172.21.15.1:0/1870038993 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a56682930 0x557a5667cb20 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.274+0000 7f6cccc44b80 1 -- 172.21.15.1:0/1870038993 >> 172.21.15.1:0/1870038993 conn(0x557a565e1db0 msgr2=0x557a56683830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:11.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.274+0000 7f6cccc44b80 1 -- 172.21.15.1:0/1870038993 shutdown_connections 2024-02-02T15:30:11.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.274+0000 7f6cccc44b80 1 -- 172.21.15.1:0/1870038993 wait complete. 2024-02-02T15:30:11.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.274+0000 7f6cccc44b80 1 -- start start 2024-02-02T15:30:11.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.275+0000 7f6cccc44b80 1 Processor -- start 2024-02-02T15:30:11.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.275+0000 7f6cccc44b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557a56682010 0x557a56687bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:11.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.275+0000 7f6cccc44b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557a56682930 0x557a566865c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:11.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.275+0000 7f6cccc44b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a5668d7e0 0x557a56686b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:11.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.275+0000 7f6cccc44b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a566883c0 con 0x557a56682930 2024-02-02T15:30:11.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.275+0000 7f6cccc44b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a56687040 con 0x557a56682010 2024-02-02T15:30:11.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.275+0000 7f6cccc44b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a56687200 con 0x557a5668d7e0 2024-02-02T15:30:11.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.275+0000 7f6cccc44b80 10 client.0 _ll_register_callbacks cb 0x557a5667fb40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:30:11.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.276+0000 7f6cb1e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557a56682010 0x557a56687bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:11.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.276+0000 7f6cb1400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557a56682930 0x557a566865c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:11.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.276+0000 7f6cb1e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557a56682010 0x557a56687bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:60014/0 (socket says 172.21.15.1:60014) 2024-02-02T15:30:11.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.276+0000 7f6cb1e00700 1 -- 172.21.15.1:0/3341966717 learned_addr learned my addr 172.21.15.1:0/3341966717 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:30:11.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.276+0000 7f6cb2800700 1 --2- 172.21.15.1:0/3341966717 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a5668d7e0 0x557a56686b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:11.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.276+0000 7f6cb1e00700 1 -- 172.21.15.1:0/3341966717 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a5668d7e0 msgr2=0x557a56686b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:11.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.276+0000 7f6cb1e00700 1 --2- 172.21.15.1:0/3341966717 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a5668d7e0 0x557a56686b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.276+0000 7f6cb1e00700 1 -- 172.21.15.1:0/3341966717 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557a56682930 msgr2=0x557a566865c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:11.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.276+0000 7f6cb1e00700 1 --2- 172.21.15.1:0/3341966717 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557a56682930 0x557a566865c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.276+0000 7f6cb1e00700 1 -- 172.21.15.1:0/3341966717 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c9c0098e0 con 0x557a56682010 2024-02-02T15:30:11.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.276+0000 7f6cb1e00700 1 --2- 172.21.15.1:0/3341966717 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557a56682010 0x557a56687bd0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6ca4098ca0 tx=0x7f6ca4005c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f52667ae042bcfe5 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:11.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.276+0000 7f6c9be00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:30:11.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.276+0000 7f6c9be00700 1 -- 172.21.15.1:0/3341966717 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca407a070 con 0x557a56682010 2024-02-02T15:30:11.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.276+0000 7f6c9be00700 1 -- 172.21.15.1:0/3341966717 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f6ca4082510 con 0x557a56682010 2024-02-02T15:30:11.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.276+0000 7f6cccc44b80 1 -- 172.21.15.1:0/3341966717 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557a566760f0 con 0x557a56682010 2024-02-02T15:30:11.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.277+0000 7f6c9be00700 1 -- 172.21.15.1:0/3341966717 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ca4094de0 con 0x557a56682010 2024-02-02T15:30:11.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.277+0000 7f6cccc44b80 10 client.26498 fetch_fsmap learned FSMap version 31 2024-02-02T15:30:11.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.277+0000 7f6cccc44b80 1 -- 172.21.15.1:0/3341966717 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f6cac027050 con 0x557a56682010 2024-02-02T15:30:11.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.277+0000 7f6c9be00700 1 -- 172.21.15.1:0/3341966717 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ca40a9080 con 0x557a56682010 2024-02-02T15:30:11.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.277+0000 7f6c9be00700 1 -- 172.21.15.1:0/3341966717 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 31) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f6ca40a3030 con 0x557a56682010 2024-02-02T15:30:11.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.277+0000 7f6cccc44b80 10 client.26498 fetch_fsmap finished waiting for FSMap version 31 2024-02-02T15:30:11.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.277+0000 7f6cccc44b80 10 client.26498 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-02-02T15:30:11.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.277+0000 7f6cccc44b80 20 client.26498 populate_metadata read hostname 'smithi001' 2024-02-02T15:30:11.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.277+0000 7f6cccc44b80 1 --2- 172.21.15.1:0/3341966717 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x557a566ef830 0x557a566f1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:11.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.277+0000 7f6cccc44b80 4 client.26498 mds_command: new command op to 24518 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:30:11.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.277+0000 7f6cccc44b80 1 -- 172.21.15.1:0/3341966717 --> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557a56422cf0 con 0x557a566ef830 2024-02-02T15:30:11.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.277+0000 7f6cb1400700 1 --2- 172.21.15.1:0/3341966717 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x557a566ef830 0x557a566f1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:11.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.278+0000 7f6cb1400700 1 --2- 172.21.15.1:0/3341966717 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x557a566ef830 0x557a566f1c20 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3504de21d1145583 server_cookie=e86e931d45421def in_seq=0 out_seq=0 2024-02-02T15:30:11.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.278+0000 7f6c9be00700 10 client.26498 ms_handle_connect on v2:172.21.15.71:6836/628016199 2024-02-02T15:30:11.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.278+0000 7f6c9be00700 1 -- 172.21.15.1:0/3341966717 <== mds.2 v2:172.21.15.71:6836/628016199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557a56422cf0 con 0x557a566ef830 2024-02-02T15:30:11.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.278+0000 7f6c9be00700 10 client.26498 handle_command_reply: tid=0 2024-02-02T15:30:11.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.278+0000 7f6cccc44b80 1 client.26498 shutdown 2024-02-02T15:30:11.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.278+0000 7f6cccc44b80 1 -- 172.21.15.1:0/3341966717 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557a56682010 msgr2=0x557a56687bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:11.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.278+0000 7f6cccc44b80 1 --2- 172.21.15.1:0/3341966717 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557a56682010 0x557a56687bd0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6ca4098ca0 tx=0x7f6ca4005c80 comp rx=0 tx=0).stop 2024-02-02T15:30:11.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.278+0000 7f6cccc44b80 1 -- 172.21.15.1:0/3341966717 shutdown_connections 2024-02-02T15:30:11.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.278+0000 7f6cccc44b80 1 --2- 172.21.15.1:0/3341966717 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x557a566ef830 0x557a566f1c20 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.278+0000 7f6cccc44b80 1 --2- 172.21.15.1:0/3341966717 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557a5668d7e0 0x557a56686b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.278+0000 7f6cccc44b80 1 --2- 172.21.15.1:0/3341966717 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557a56682010 0x557a56687bd0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.278+0000 7f6c9be00700 0 client.26498 ms_handle_reset on v2:172.21.15.71:6836/628016199 2024-02-02T15:30:11.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.278+0000 7f6cccc44b80 1 --2- 172.21.15.1:0/3341966717 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557a56682930 0x557a566865c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.278+0000 7f6cccc44b80 1 -- 172.21.15.1:0/3341966717 >> 172.21.15.1:0/3341966717 conn(0x557a565e1db0 msgr2=0x557a566848d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:11.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.279+0000 7f6cccc44b80 1 -- 172.21.15.1:0/3341966717 shutdown_connections 2024-02-02T15:30:11.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.279+0000 7f6cccc44b80 1 -- 172.21.15.1:0/3341966717 wait complete. 2024-02-02T15:30:11.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.279+0000 7f6cccc44b80 20 client.26498 trim_cache size 0 max 16384 2024-02-02T15:30:11.284 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:30:11.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.279+0000 7f6cb9200700 1 -- 172.21.15.1:0/3349534465 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f6c940a0710 msgr2=0x7f6c940a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:11.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.279+0000 7f6cb9200700 1 --2- 172.21.15.1:0/3349534465 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f6c940a0710 0x7f6c940a2bd0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6cac009040 tx=0x7f6cac00bf40 comp rx=0 tx=0).stop 2024-02-02T15:30:11.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.279+0000 7f6cb9200700 1 -- 172.21.15.1:0/3349534465 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6cb4100ff0 msgr2=0x7f6cb4198fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:11.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.279+0000 7f6cb9200700 1 --2- 172.21.15.1:0/3349534465 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6cb4100ff0 0x7f6cb4198fb0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f6ca400d950 tx=0x7f6ca400dc60 comp rx=0 tx=0).stop 2024-02-02T15:30:11.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.279+0000 7f6cb9200700 1 -- 172.21.15.1:0/3349534465 shutdown_connections 2024-02-02T15:30:11.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.279+0000 7f6cb9200700 1 --2- 172.21.15.1:0/3349534465 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f6c940a0710 0x7f6c940a2bd0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.279+0000 7f6cb9200700 1 --2- 172.21.15.1:0/3349534465 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6cb419d950 0x7f6cb419ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.279+0000 7f6cb9200700 1 --2- 172.21.15.1:0/3349534465 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6cb4100ff0 0x7f6cb4198fb0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.279+0000 7f6cb9200700 1 --2- 172.21.15.1:0/3349534465 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6cb4106110 0x7f6cb41994f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.279+0000 7f6cb9200700 1 -- 172.21.15.1:0/3349534465 >> 172.21.15.1:0/3349534465 conn(0x7f6cb40fc810 msgr2=0x7f6cb40fd240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:11.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.280+0000 7f6cb9200700 1 -- 172.21.15.1:0/3349534465 shutdown_connections 2024-02-02T15:30:11.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.280+0000 7f6cb9200700 1 -- 172.21.15.1:0/3349534465 wait complete. 2024-02-02T15:30:11.298 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-02T15:30:11.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.501+0000 7f0649a00700 1 -- 172.21.15.1:0/300299253 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0644106ed0 msgr2=0x7f06441072b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:11.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.501+0000 7f0649a00700 1 --2- 172.21.15.1:0/300299253 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0644106ed0 0x7f06441072b0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f06340099c0 tx=0x7f0634009cd0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.501+0000 7f0649a00700 1 -- 172.21.15.1:0/300299253 shutdown_connections 2024-02-02T15:30:11.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.501+0000 7f0649a00700 1 --2- 172.21.15.1:0/300299253 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06440689b0 0x7f064410dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.501+0000 7f0649a00700 1 --2- 172.21.15.1:0/300299253 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0644106ed0 0x7f06441072b0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.502+0000 7f0649a00700 1 --2- 172.21.15.1:0/300299253 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0644068010 0x7f0644068470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.502+0000 7f0649a00700 1 -- 172.21.15.1:0/300299253 >> 172.21.15.1:0/300299253 conn(0x7f06440757d0 msgr2=0x7f0644075be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:11.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.502+0000 7f0649a00700 1 -- 172.21.15.1:0/300299253 shutdown_connections 2024-02-02T15:30:11.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.502+0000 7f0649a00700 1 -- 172.21.15.1:0/300299253 wait complete. 2024-02-02T15:30:11.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.503+0000 7f0649a00700 1 Processor -- start 2024-02-02T15:30:11.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.503+0000 7f0649a00700 1 -- start start 2024-02-02T15:30:11.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.503+0000 7f0649a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0644068010 0x7f0644199150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:11.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.503+0000 7f0649a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f06440689b0 0x7f0644199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:11.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.503+0000 7f0649a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0644106ed0 0x7f064419daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:11.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.503+0000 7f0649a00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f064419e130 con 0x7f0644106ed0 2024-02-02T15:30:11.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.503+0000 7f0649a00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f064419e2a0 con 0x7f06440689b0 2024-02-02T15:30:11.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.503+0000 7f0649a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f064419e460 con 0x7f0644068010 2024-02-02T15:30:11.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.504+0000 7f0643400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0644106ed0 0x7f064419daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:11.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.504+0000 7f0643400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0644106ed0 0x7f064419daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54572/0 (socket says 172.21.15.1:54572) 2024-02-02T15:30:11.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.504+0000 7f0643400700 1 -- 172.21.15.1:0/2281459742 learned_addr learned my addr 172.21.15.1:0/2281459742 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:30:11.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.504+0000 7f0642000700 1 --2- 172.21.15.1:0/2281459742 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f06440689b0 0x7f0644199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:11.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.504+0000 7f0642a00700 1 --2- 172.21.15.1:0/2281459742 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0644068010 0x7f0644199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:11.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.504+0000 7f0643400700 1 -- 172.21.15.1:0/2281459742 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0644068010 msgr2=0x7f0644199150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:11.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.504+0000 7f0643400700 1 --2- 172.21.15.1:0/2281459742 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0644068010 0x7f0644199150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.504+0000 7f0643400700 1 -- 172.21.15.1:0/2281459742 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f06440689b0 msgr2=0x7f0644199690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:11.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.504+0000 7f0643400700 1 --2- 172.21.15.1:0/2281459742 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f06440689b0 0x7f0644199690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.504+0000 7f0643400700 1 -- 172.21.15.1:0/2281459742 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f063c009c40 con 0x7f0644106ed0 2024-02-02T15:30:11.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.504+0000 7f0642000700 1 --2- 172.21.15.1:0/2281459742 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f06440689b0 0x7f0644199690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:30:11.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.504+0000 7f0643400700 1 --2- 172.21.15.1:0/2281459742 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0644106ed0 0x7f064419daf0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f063c00c370 tx=0x7f063c00c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:11.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.505+0000 7f063b400700 1 -- 172.21.15.1:0/2281459742 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063c017070 con 0x7f0644106ed0 2024-02-02T15:30:11.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.505+0000 7f0649a00700 1 -- 172.21.15.1:0/2281459742 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06340095f0 con 0x7f0644106ed0 2024-02-02T15:30:11.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.505+0000 7f063b400700 1 -- 172.21.15.1:0/2281459742 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f063c00f040 con 0x7f0644106ed0 2024-02-02T15:30:11.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.505+0000 7f063b400700 1 -- 172.21.15.1:0/2281459742 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063c00e050 con 0x7f0644106ed0 2024-02-02T15:30:11.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.505+0000 7f0649a00700 1 -- 172.21.15.1:0/2281459742 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06441a5da0 con 0x7f0644106ed0 2024-02-02T15:30:11.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.508+0000 7f063b400700 1 -- 172.21.15.1:0/2281459742 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 27) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f063c00cb60 con 0x7f0644106ed0 2024-02-02T15:30:11.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.509+0000 7f063b400700 1 --2- 172.21.15.1:0/2281459742 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f06240a0690 0x7f06240a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:11.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.509+0000 7f065d472b80 1 Processor -- start 2024-02-02T15:30:11.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.509+0000 7f063b400700 1 -- 172.21.15.1:0/2281459742 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f063c0cb9e0 con 0x7f0644106ed0 2024-02-02T15:30:11.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.510+0000 7f065d472b80 1 -- start start 2024-02-02T15:30:11.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.510+0000 7f0642a00700 1 --2- 172.21.15.1:0/2281459742 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f06240a0690 0x7f06240a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:11.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.510+0000 7f065d472b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eba3b5f010 0x55eba3b5f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:11.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.510+0000 7f065d472b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eba3b5f930 0x55eba3b6a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:11.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.510+0000 7f065d472b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eba3b6a7e0 0x55eba3b6cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:11.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.510+0000 7f065d472b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eba3ad34c0 con 0x55eba3b5f010 2024-02-02T15:30:11.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.510+0000 7f065d472b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eba3ab7aa0 con 0x55eba3b5f930 2024-02-02T15:30:11.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.510+0000 7f065d472b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eba3ab5460 con 0x55eba3b6a7e0 2024-02-02T15:30:11.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.510+0000 7f0643400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eba3b6a7e0 0x55eba3b6cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:11.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.510+0000 7f0642a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eba3b5f010 0x55eba3b5f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:11.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.510+0000 7f0642000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eba3b5f930 0x55eba3b6a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:11.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.510+0000 7f0643400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eba3b6a7e0 0x55eba3b6cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:43316/0 (socket says 172.21.15.1:43316) 2024-02-02T15:30:11.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.510+0000 7f0643400700 1 -- 172.21.15.1:0/1912194986 learned_addr learned my addr 172.21.15.1:0/1912194986 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:30:11.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.511+0000 7f0642a00700 1 --2- 172.21.15.1:0/2281459742 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f06240a0690 0x7f06240a2b50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0634009990 tx=0x7f0634004960 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:11.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.511+0000 7f0643400700 1 -- 172.21.15.1:0/1912194986 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eba3b5f930 msgr2=0x55eba3b6a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:11.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.511+0000 7f0643400700 1 --2- 172.21.15.1:0/1912194986 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eba3b5f930 0x55eba3b6a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.511+0000 7f0643400700 1 -- 172.21.15.1:0/1912194986 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eba3b5f010 msgr2=0x55eba3b5f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:11.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.511+0000 7f0643400700 1 --2- 172.21.15.1:0/1912194986 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eba3b5f010 0x55eba3b5f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.511+0000 7f0643400700 1 -- 172.21.15.1:0/1912194986 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eba39323b0 con 0x55eba3b6a7e0 2024-02-02T15:30:11.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.511+0000 7f0643400700 1 --2- 172.21.15.1:0/1912194986 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eba3b6a7e0 0x55eba3b6cbd0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f063c098850 tx=0x7f063c0989c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6b678edd0a3d978 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:11.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.511+0000 7f0633e00700 1 -- 172.21.15.1:0/1912194986 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063c0092e0 con 0x55eba3b6a7e0 2024-02-02T15:30:11.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.511+0000 7f0633e00700 1 -- 172.21.15.1:0/1912194986 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f063c099250 con 0x55eba3b6a7e0 2024-02-02T15:30:11.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.511+0000 7f0633e00700 1 -- 172.21.15.1:0/1912194986 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063c0b80c0 con 0x55eba3b6a7e0 2024-02-02T15:30:11.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.512+0000 7f065d472b80 1 -- 172.21.15.1:0/1912194986 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eba3b6a7e0 msgr2=0x55eba3b6cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:11.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.512+0000 7f065d472b80 1 --2- 172.21.15.1:0/1912194986 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eba3b6a7e0 0x55eba3b6cbd0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f063c098850 tx=0x7f063c0989c0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.512+0000 7f065d472b80 1 -- 172.21.15.1:0/1912194986 shutdown_connections 2024-02-02T15:30:11.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.512+0000 7f065d472b80 1 --2- 172.21.15.1:0/1912194986 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eba3b6a7e0 0x55eba3b6cbd0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.512+0000 7f065d472b80 1 --2- 172.21.15.1:0/1912194986 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eba3b5f010 0x55eba3b5f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.512+0000 7f065d472b80 1 --2- 172.21.15.1:0/1912194986 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eba3b5f930 0x55eba3b6a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.512+0000 7f065d472b80 1 -- 172.21.15.1:0/1912194986 >> 172.21.15.1:0/1912194986 conn(0x55eba3abedb0 msgr2=0x55eba3b5b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:11.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.512+0000 7f065d472b80 1 -- 172.21.15.1:0/1912194986 shutdown_connections 2024-02-02T15:30:11.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.512+0000 7f065d472b80 1 -- 172.21.15.1:0/1912194986 wait complete. 2024-02-02T15:30:11.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.513+0000 7f065d472b80 1 -- start start 2024-02-02T15:30:11.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.513+0000 7f065d472b80 1 Processor -- start 2024-02-02T15:30:11.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.513+0000 7f065d472b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eba3b5f010 0x55eba3b6a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:11.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.513+0000 7f065d472b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eba3b5f930 0x55eba3b67470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:11.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.513+0000 7f065d472b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eba3b6a7e0 0x55eba3b67aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:11.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.513+0000 7f065d472b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eba3ab0d50 con 0x55eba3b5f010 2024-02-02T15:30:11.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.513+0000 7f065d472b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eba3a4cd80 con 0x55eba3b6a7e0 2024-02-02T15:30:11.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.513+0000 7f065d472b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eba3a4ca20 con 0x55eba3b5f930 2024-02-02T15:30:11.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.513+0000 7f065d472b80 10 client.0 _ll_register_callbacks cb 0x55eba3a4e290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:30:11.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.514+0000 7f0642a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eba3b5f010 0x55eba3b6a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:11.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.514+0000 7f0642a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eba3b5f010 0x55eba3b6a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54594/0 (socket says 172.21.15.1:54594) 2024-02-02T15:30:11.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.514+0000 7f0642a00700 1 -- 172.21.15.1:0/444571627 learned_addr learned my addr 172.21.15.1:0/444571627 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:30:11.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.514+0000 7f0643400700 1 --2- 172.21.15.1:0/444571627 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eba3b6a7e0 0x55eba3b67aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:11.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.514+0000 7f0642000700 1 --2- 172.21.15.1:0/444571627 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eba3b5f930 0x55eba3b67470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:11.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.514+0000 7f0642a00700 1 -- 172.21.15.1:0/444571627 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eba3b5f930 msgr2=0x55eba3b67470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:11.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.514+0000 7f0642a00700 1 --2- 172.21.15.1:0/444571627 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eba3b5f930 0x55eba3b67470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.514+0000 7f0642a00700 1 -- 172.21.15.1:0/444571627 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eba3b6a7e0 msgr2=0x55eba3b67aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:11.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.514+0000 7f0642a00700 1 --2- 172.21.15.1:0/444571627 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eba3b6a7e0 0x55eba3b67aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.514+0000 7f0642a00700 1 -- 172.21.15.1:0/444571627 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06441a5da0 con 0x55eba3b5f010 2024-02-02T15:30:11.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.514+0000 7f0643400700 1 --2- 172.21.15.1:0/444571627 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eba3b6a7e0 0x55eba3b67aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:30:11.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.514+0000 7f0642a00700 1 --2- 172.21.15.1:0/444571627 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eba3b5f010 0x55eba3b6a1f0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f0634000c00 tx=0x7f0634009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10a98e6714e23184 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:11.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.514+0000 7f0630c00700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:30:11.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.515+0000 7f0630c00700 1 -- 172.21.15.1:0/444571627 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063400bae0 con 0x55eba3b5f010 2024-02-02T15:30:11.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.515+0000 7f0630c00700 1 -- 172.21.15.1:0/444571627 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f063400bc40 con 0x55eba3b5f010 2024-02-02T15:30:11.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.515+0000 7f065d472b80 1 -- 172.21.15.1:0/444571627 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55eba3b530f0 con 0x55eba3b5f010 2024-02-02T15:30:11.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.515+0000 7f0630c00700 1 -- 172.21.15.1:0/444571627 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f063400be00 con 0x55eba3b5f010 2024-02-02T15:30:11.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.515+0000 7f065d472b80 10 client.16449 fetch_fsmap learned FSMap version 31 2024-02-02T15:30:11.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.515+0000 7f065d472b80 1 -- 172.21.15.1:0/444571627 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55eba38deb60 con 0x55eba3b5f010 2024-02-02T15:30:11.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.515+0000 7f0630c00700 1 -- 172.21.15.1:0/444571627 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55eba3b530f0 con 0x55eba3b5f010 2024-02-02T15:30:11.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.515+0000 7f0630c00700 1 -- 172.21.15.1:0/444571627 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 31) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f063400f560 con 0x55eba3b5f010 2024-02-02T15:30:11.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.515+0000 7f065d472b80 10 client.16449 fetch_fsmap finished waiting for FSMap version 31 2024-02-02T15:30:11.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.515+0000 7f065d472b80 10 client.16449 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-02T15:30:11.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.516+0000 7f065d472b80 20 client.16449 populate_metadata read hostname 'smithi001' 2024-02-02T15:30:11.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.516+0000 7f065d472b80 1 --2- 172.21.15.1:0/444571627 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55eba3b7a160 0x55eba3b7c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:11.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.516+0000 7f065d472b80 4 client.16449 mds_command: new command op to 24529 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:30:11.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.516+0000 7f065d472b80 1 -- 172.21.15.1:0/444571627 --> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eba3aa9dd0 con 0x55eba3b7a160 2024-02-02T15:30:11.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.516+0000 7f0642000700 1 --2- 172.21.15.1:0/444571627 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55eba3b7a160 0x55eba3b7c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:11.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.516+0000 7f0642000700 1 --2- 172.21.15.1:0/444571627 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55eba3b7a160 0x55eba3b7c580 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d0c311ef2a93e2b6 server_cookie=9aaf1c3f52151311 in_seq=0 out_seq=0 2024-02-02T15:30:11.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.516+0000 7f0630c00700 10 client.16449 ms_handle_connect on v2:172.21.15.142:6838/321675421 2024-02-02T15:30:11.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.517+0000 7f0630c00700 1 -- 172.21.15.1:0/444571627 <== mds.1 v2:172.21.15.142:6838/321675421 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55eba3aa9dd0 con 0x55eba3b7a160 2024-02-02T15:30:11.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.517+0000 7f0630c00700 10 client.16449 handle_command_reply: tid=0 2024-02-02T15:30:11.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.517+0000 7f065d472b80 1 client.16449 shutdown 2024-02-02T15:30:11.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.517+0000 7f065d472b80 1 -- 172.21.15.1:0/444571627 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eba3b5f010 msgr2=0x55eba3b6a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:11.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.517+0000 7f065d472b80 1 --2- 172.21.15.1:0/444571627 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eba3b5f010 0x55eba3b6a1f0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f0634000c00 tx=0x7f0634009cd0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.517+0000 7f065d472b80 1 -- 172.21.15.1:0/444571627 shutdown_connections 2024-02-02T15:30:11.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.517+0000 7f065d472b80 1 --2- 172.21.15.1:0/444571627 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55eba3b7a160 0x55eba3b7c580 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.517+0000 7f065d472b80 1 --2- 172.21.15.1:0/444571627 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eba3b6a7e0 0x55eba3b67aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.517+0000 7f065d472b80 1 --2- 172.21.15.1:0/444571627 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eba3b5f010 0x55eba3b6a1f0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.517+0000 7f0630c00700 0 client.16449 ms_handle_reset on v2:172.21.15.142:6838/321675421 2024-02-02T15:30:11.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.517+0000 7f065d472b80 1 --2- 172.21.15.1:0/444571627 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eba3b5f930 0x55eba3b67470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.517+0000 7f065d472b80 1 -- 172.21.15.1:0/444571627 >> 172.21.15.1:0/444571627 conn(0x55eba3abedb0 msgr2=0x55eba3b5afa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:11.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.517+0000 7f065d472b80 1 -- 172.21.15.1:0/444571627 shutdown_connections 2024-02-02T15:30:11.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.517+0000 7f065d472b80 1 -- 172.21.15.1:0/444571627 wait complete. 2024-02-02T15:30:11.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.518+0000 7f065d472b80 20 client.16449 trim_cache size 0 max 16384 2024-02-02T15:30:11.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.531+0000 7f065d472b80 1 Processor -- start 2024-02-02T15:30:11.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.531+0000 7f065d472b80 1 -- start start 2024-02-02T15:30:11.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.531+0000 7f065d472b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eba3b5f010 0x55eba3b71330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:11.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.531+0000 7f065d472b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eba3b5f930 0x55eba3b69d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:11.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.531+0000 7f065d472b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eba3b6a7e0 0x55eba3b75900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:11.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.531+0000 7f065d472b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eba39346a0 con 0x55eba3b6a7e0 2024-02-02T15:30:11.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.531+0000 7f065d472b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eba3b680a0 con 0x55eba3b5f930 2024-02-02T15:30:11.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.531+0000 7f065d472b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eba3b52cc0 con 0x55eba3b5f010 2024-02-02T15:30:11.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.531+0000 7f0643400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eba3b6a7e0 0x55eba3b75900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:11.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.532+0000 7f0642000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eba3b5f930 0x55eba3b69d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:11.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.532+0000 7f0642a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eba3b5f010 0x55eba3b71330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:11.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.532+0000 7f0642000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eba3b5f930 0x55eba3b69d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:60056/0 (socket says 172.21.15.1:60056) 2024-02-02T15:30:11.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.532+0000 7f0642000700 1 -- 172.21.15.1:0/1508606858 learned_addr learned my addr 172.21.15.1:0/1508606858 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:30:11.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.532+0000 7f0642a00700 1 -- 172.21.15.1:0/1508606858 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eba3b5f930 msgr2=0x55eba3b69d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:11.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.532+0000 7f0642a00700 1 --2- 172.21.15.1:0/1508606858 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eba3b5f930 0x55eba3b69d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.532+0000 7f0642a00700 1 -- 172.21.15.1:0/1508606858 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eba3b6a7e0 msgr2=0x55eba3b75900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:11.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.532+0000 7f0642a00700 1 --2- 172.21.15.1:0/1508606858 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eba3b6a7e0 0x55eba3b75900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.532+0000 7f0642a00700 1 -- 172.21.15.1:0/1508606858 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eba3a76ea0 con 0x55eba3b5f010 2024-02-02T15:30:11.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.532+0000 7f0643400700 1 --2- 172.21.15.1:0/1508606858 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eba3b6a7e0 0x55eba3b75900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:30:11.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.532+0000 7f0642a00700 1 --2- 172.21.15.1:0/1508606858 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eba3b5f010 0x55eba3b71330 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0634005f30 tx=0x7f0634005d40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71222a812fa20894 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:11.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.532+0000 7f0638c00700 1 -- 172.21.15.1:0/1508606858 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0634005980 con 0x55eba3b5f010 2024-02-02T15:30:11.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.532+0000 7f0638c00700 1 -- 172.21.15.1:0/1508606858 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f063401e040 con 0x55eba3b5f010 2024-02-02T15:30:11.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.532+0000 7f0638c00700 1 -- 172.21.15.1:0/1508606858 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06340308a0 con 0x55eba3b5f010 2024-02-02T15:30:11.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.533+0000 7f065d472b80 1 -- 172.21.15.1:0/1508606858 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eba3b5f010 msgr2=0x55eba3b71330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:11.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.533+0000 7f065d472b80 1 --2- 172.21.15.1:0/1508606858 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eba3b5f010 0x55eba3b71330 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0634005f30 tx=0x7f0634005d40 comp rx=0 tx=0).stop 2024-02-02T15:30:11.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.533+0000 7f065d472b80 1 -- 172.21.15.1:0/1508606858 shutdown_connections 2024-02-02T15:30:11.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.533+0000 7f065d472b80 1 --2- 172.21.15.1:0/1508606858 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eba3b6a7e0 0x55eba3b75900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.533+0000 7f065d472b80 1 --2- 172.21.15.1:0/1508606858 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eba3b5f010 0x55eba3b71330 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.533+0000 7f065d472b80 1 --2- 172.21.15.1:0/1508606858 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eba3b5f930 0x55eba3b69d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.533+0000 7f065d472b80 1 -- 172.21.15.1:0/1508606858 >> 172.21.15.1:0/1508606858 conn(0x55eba3abedb0 msgr2=0x55eba3b76520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:11.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.533+0000 7f065d472b80 1 -- 172.21.15.1:0/1508606858 shutdown_connections 2024-02-02T15:30:11.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.533+0000 7f065d472b80 1 -- 172.21.15.1:0/1508606858 wait complete. 2024-02-02T15:30:11.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.533+0000 7f065d472b80 1 -- start start 2024-02-02T15:30:11.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.534+0000 7f065d472b80 1 Processor -- start 2024-02-02T15:30:11.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.534+0000 7f065d472b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eba3b5f010 0x55eba3b6f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:11.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.534+0000 7f065d472b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eba3b5f930 0x55eba3b6fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:11.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.534+0000 7f065d472b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eba3b6a7e0 0x55eba3bcac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:11.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.534+0000 7f065d472b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eba3b70180 con 0x55eba3b5f010 2024-02-02T15:30:11.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.534+0000 7f065d472b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eba3bcb180 con 0x55eba3b5f930 2024-02-02T15:30:11.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.534+0000 7f065d472b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eba3bcb310 con 0x55eba3b6a7e0 2024-02-02T15:30:11.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.534+0000 7f065d472b80 10 client.0 _ll_register_callbacks cb 0x55eba3a4e290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:30:11.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.534+0000 7f0642a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eba3b5f010 0x55eba3b6f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:11.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.534+0000 7f0642a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eba3b5f010 0x55eba3b6f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54610/0 (socket says 172.21.15.1:54610) 2024-02-02T15:30:11.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.534+0000 7f0642a00700 1 -- 172.21.15.1:0/3207301406 learned_addr learned my addr 172.21.15.1:0/3207301406 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:30:11.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.534+0000 7f0643400700 1 --2- 172.21.15.1:0/3207301406 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eba3b6a7e0 0x55eba3bcac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:11.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.534+0000 7f0642000700 1 --2- 172.21.15.1:0/3207301406 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eba3b5f930 0x55eba3b6fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:11.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.535+0000 7f0643400700 1 -- 172.21.15.1:0/3207301406 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eba3b5f930 msgr2=0x55eba3b6fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:11.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.535+0000 7f0643400700 1 --2- 172.21.15.1:0/3207301406 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eba3b5f930 0x55eba3b6fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.535+0000 7f0643400700 1 -- 172.21.15.1:0/3207301406 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eba3b5f010 msgr2=0x55eba3b6f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:11.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.535+0000 7f0643400700 1 --2- 172.21.15.1:0/3207301406 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eba3b5f010 0x55eba3b6f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.535+0000 7f0643400700 1 -- 172.21.15.1:0/3207301406 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eba39323b0 con 0x55eba3b6a7e0 2024-02-02T15:30:11.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.535+0000 7f0643400700 1 --2- 172.21.15.1:0/3207301406 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eba3b6a7e0 0x55eba3bcac40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f063c098d10 tx=0x7f063c098790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c09bf34afa96c917 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:11.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.535+0000 7f0630c00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:30:11.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.535+0000 7f0630c00700 1 -- 172.21.15.1:0/3207301406 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063c098b50 con 0x55eba3b6a7e0 2024-02-02T15:30:11.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.535+0000 7f0630c00700 1 -- 172.21.15.1:0/3207301406 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f063c0b0030 con 0x55eba3b6a7e0 2024-02-02T15:30:11.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.535+0000 7f065d472b80 1 -- 172.21.15.1:0/3207301406 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f063c099250 con 0x55eba3b6a7e0 2024-02-02T15:30:11.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.535+0000 7f0630c00700 1 -- 172.21.15.1:0/3207301406 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f063c09c980 con 0x55eba3b6a7e0 2024-02-02T15:30:11.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.535+0000 7f065d472b80 10 client.26503 fetch_fsmap learned FSMap version 31 2024-02-02T15:30:11.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.535+0000 7f065d472b80 1 -- 172.21.15.1:0/3207301406 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55eba38deb60 con 0x55eba3b6a7e0 2024-02-02T15:30:11.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.536+0000 7f0630c00700 1 -- 172.21.15.1:0/3207301406 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f063c099250 con 0x55eba3b6a7e0 2024-02-02T15:30:11.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.536+0000 7f0630c00700 1 -- 172.21.15.1:0/3207301406 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 31) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f063c07b070 con 0x55eba3b6a7e0 2024-02-02T15:30:11.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.536+0000 7f065d472b80 10 client.26503 fetch_fsmap finished waiting for FSMap version 31 2024-02-02T15:30:11.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.536+0000 7f065d472b80 10 client.26503 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-02T15:30:11.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.536+0000 7f065d472b80 20 client.26503 populate_metadata read hostname 'smithi001' 2024-02-02T15:30:11.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.536+0000 7f065d472b80 1 --2- 172.21.15.1:0/3207301406 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55eba3bcc700 0x55eba3bceb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:11.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.536+0000 7f065d472b80 4 client.26503 mds_command: new command op to 24529 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:30:11.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.536+0000 7f065d472b80 1 -- 172.21.15.1:0/3207301406 --> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55eba3a6ff50 con 0x55eba3bcc700 2024-02-02T15:30:11.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.536+0000 7f0642a00700 1 --2- 172.21.15.1:0/3207301406 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55eba3bcc700 0x55eba3bceb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:11.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.537+0000 7f0642a00700 1 --2- 172.21.15.1:0/3207301406 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55eba3bcc700 0x55eba3bceb20 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f4bfb9fa5b9abf19 server_cookie=ce32453353f4bf62 in_seq=0 out_seq=0 2024-02-02T15:30:11.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.537+0000 7f0630c00700 10 client.26503 ms_handle_connect on v2:172.21.15.142:6838/321675421 2024-02-02T15:30:11.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.537+0000 7f0630c00700 1 -- 172.21.15.1:0/3207301406 <== mds.1 v2:172.21.15.142:6838/321675421 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55eba3a6ff50 con 0x55eba3bcc700 2024-02-02T15:30:11.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.537+0000 7f0630c00700 10 client.26503 handle_command_reply: tid=0 2024-02-02T15:30:11.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.537+0000 7f065d472b80 1 client.26503 shutdown 2024-02-02T15:30:11.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.537+0000 7f065d472b80 1 -- 172.21.15.1:0/3207301406 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eba3b6a7e0 msgr2=0x55eba3bcac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:11.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.537+0000 7f065d472b80 1 --2- 172.21.15.1:0/3207301406 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eba3b6a7e0 0x55eba3bcac40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f063c098d10 tx=0x7f063c098790 comp rx=0 tx=0).stop 2024-02-02T15:30:11.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.537+0000 7f065d472b80 1 -- 172.21.15.1:0/3207301406 shutdown_connections 2024-02-02T15:30:11.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.537+0000 7f065d472b80 1 --2- 172.21.15.1:0/3207301406 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55eba3bcc700 0x55eba3bceb20 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.537+0000 7f065d472b80 1 --2- 172.21.15.1:0/3207301406 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eba3b6a7e0 0x55eba3bcac40 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.537+0000 7f0630c00700 0 client.26503 ms_handle_reset on v2:172.21.15.142:6838/321675421 2024-02-02T15:30:11.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.537+0000 7f065d472b80 1 --2- 172.21.15.1:0/3207301406 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eba3b5f010 0x55eba3b6f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.537+0000 7f065d472b80 1 --2- 172.21.15.1:0/3207301406 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eba3b5f930 0x55eba3b6fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.537+0000 7f065d472b80 1 -- 172.21.15.1:0/3207301406 >> 172.21.15.1:0/3207301406 conn(0x55eba3abedb0 msgr2=0x55eba3b627a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:11.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.537+0000 7f065d472b80 1 -- 172.21.15.1:0/3207301406 shutdown_connections 2024-02-02T15:30:11.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.537+0000 7f065d472b80 1 -- 172.21.15.1:0/3207301406 wait complete. 2024-02-02T15:30:11.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.537+0000 7f065d472b80 20 client.26503 trim_cache size 0 max 16384 2024-02-02T15:30:11.543 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:30:11.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.538+0000 7f0649a00700 1 -- 172.21.15.1:0/2281459742 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f06240a0690 msgr2=0x7f06240a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:11.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.538+0000 7f0649a00700 1 --2- 172.21.15.1:0/2281459742 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f06240a0690 0x7f06240a2b50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0634009990 tx=0x7f0634004960 comp rx=0 tx=0).stop 2024-02-02T15:30:11.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.538+0000 7f0649a00700 1 -- 172.21.15.1:0/2281459742 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0644106ed0 msgr2=0x7f064419daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:11.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.538+0000 7f0649a00700 1 --2- 172.21.15.1:0/2281459742 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0644106ed0 0x7f064419daf0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f063c00c370 tx=0x7f063c00c680 comp rx=0 tx=0).stop 2024-02-02T15:30:11.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.538+0000 7f0649a00700 1 -- 172.21.15.1:0/2281459742 shutdown_connections 2024-02-02T15:30:11.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.538+0000 7f0649a00700 1 --2- 172.21.15.1:0/2281459742 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f06240a0690 0x7f06240a2b50 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.538+0000 7f0649a00700 1 --2- 172.21.15.1:0/2281459742 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0644106ed0 0x7f064419daf0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.538+0000 7f0649a00700 1 --2- 172.21.15.1:0/2281459742 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0644068010 0x7f0644199150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.538+0000 7f0649a00700 1 --2- 172.21.15.1:0/2281459742 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f06440689b0 0x7f0644199690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.538+0000 7f0649a00700 1 -- 172.21.15.1:0/2281459742 >> 172.21.15.1:0/2281459742 conn(0x7f06440757d0 msgr2=0x7f064410bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:11.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.538+0000 7f0649a00700 1 -- 172.21.15.1:0/2281459742 shutdown_connections 2024-02-02T15:30:11.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.538+0000 7f0649a00700 1 -- 172.21.15.1:0/2281459742 wait complete. 2024-02-02T15:30:11.555 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:11 smithi001 ceph-mon[35470]: pgmap v98: 129 pgs: 129 active+clean; 69 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 497 KiB/s wr, 73 op/s 2024-02-02T15:30:11.555 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:11 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/1238693356' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:30:11.556 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-02T15:30:11.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.757+0000 7f2777a00700 1 -- 172.21.15.1:0/2132050717 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2770100ff0 msgr2=0x7f27701013d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:11.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.757+0000 7f2777a00700 1 --2- 172.21.15.1:0/2132050717 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2770100ff0 0x7f27701013d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2760009990 tx=0x7f2760009ca0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.757+0000 7f2777a00700 1 -- 172.21.15.1:0/2132050717 shutdown_connections 2024-02-02T15:30:11.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.757+0000 7f2777a00700 1 --2- 172.21.15.1:0/2132050717 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2770106110 0x7f277010dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.757+0000 7f2777a00700 1 --2- 172.21.15.1:0/2132050717 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2770100ff0 0x7f27701013d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.757+0000 7f2777a00700 1 --2- 172.21.15.1:0/2132050717 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f27701019a0 0x7f27701059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.757+0000 7f2777a00700 1 -- 172.21.15.1:0/2132050717 >> 172.21.15.1:0/2132050717 conn(0x7f27700fc810 msgr2=0x7f27700fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:11.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.758+0000 7f2777a00700 1 -- 172.21.15.1:0/2132050717 shutdown_connections 2024-02-02T15:30:11.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.758+0000 7f2777a00700 1 -- 172.21.15.1:0/2132050717 wait complete. 2024-02-02T15:30:11.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.759+0000 7f2777a00700 1 Processor -- start 2024-02-02T15:30:11.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.759+0000 7f2777a00700 1 -- start start 2024-02-02T15:30:11.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.760+0000 7f2777a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2770100ff0 0x7f2770073040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:11.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.760+0000 7f2777a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2770106110 0x7f2770073580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:11.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.760+0000 7f2777a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2770073ac0 0x7f27701a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:11.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.760+0000 7f2777a00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27701a5730 con 0x7f2770106110 2024-02-02T15:30:11.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.760+0000 7f2777a00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27701a58a0 con 0x7f2770073ac0 2024-02-02T15:30:11.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.760+0000 7f2777a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27701a5a60 con 0x7f2770100ff0 2024-02-02T15:30:11.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.760+0000 7f276fe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2770106110 0x7f2770073580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:11.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.760+0000 7f2775000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2770100ff0 0x7f2770073040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:11.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.760+0000 7f2775000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2770100ff0 0x7f2770073040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:43358/0 (socket says 172.21.15.1:43358) 2024-02-02T15:30:11.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.760+0000 7f2775000700 1 -- 172.21.15.1:0/3004079198 learned_addr learned my addr 172.21.15.1:0/3004079198 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:30:11.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.760+0000 7f2775a00700 1 --2- 172.21.15.1:0/3004079198 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2770073ac0 0x7f27701a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:11.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.760+0000 7f2775000700 1 -- 172.21.15.1:0/3004079198 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2770073ac0 msgr2=0x7f27701a50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:11.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.760+0000 7f2775000700 1 --2- 172.21.15.1:0/3004079198 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2770073ac0 0x7f27701a50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.761+0000 7f2775000700 1 -- 172.21.15.1:0/3004079198 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2770106110 msgr2=0x7f2770073580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:11.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.761+0000 7f2775000700 1 --2- 172.21.15.1:0/3004079198 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2770106110 0x7f2770073580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.761+0000 7f2775000700 1 -- 172.21.15.1:0/3004079198 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27600095f0 con 0x7f2770100ff0 2024-02-02T15:30:11.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.761+0000 7f2775a00700 1 --2- 172.21.15.1:0/3004079198 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2770073ac0 0x7f27701a50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:30:11.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.761+0000 7f2775000700 1 --2- 172.21.15.1:0/3004079198 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2770100ff0 0x7f2770073040 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2760005910 tx=0x7f2760005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:11.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.761+0000 7f276d600700 1 -- 172.21.15.1:0/3004079198 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276001d070 con 0x7f2770100ff0 2024-02-02T15:30:11.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.761+0000 7f276d600700 1 -- 172.21.15.1:0/3004079198 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2760022470 con 0x7f2770100ff0 2024-02-02T15:30:11.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.761+0000 7f2777a00700 1 -- 172.21.15.1:0/3004079198 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27701a5ce0 con 0x7f2770100ff0 2024-02-02T15:30:11.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.761+0000 7f276d600700 1 -- 172.21.15.1:0/3004079198 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2760017650 con 0x7f2770100ff0 2024-02-02T15:30:11.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.761+0000 7f2777a00700 1 -- 172.21.15.1:0/3004079198 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27701a6160 con 0x7f2770100ff0 2024-02-02T15:30:11.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.764+0000 7f276d600700 1 -- 172.21.15.1:0/3004079198 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 27) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f27600225e0 con 0x7f2770100ff0 2024-02-02T15:30:11.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.765+0000 7f276d600700 1 --2- 172.21.15.1:0/3004079198 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f27500a0710 0x7f27500a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:11.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.765+0000 7f278b61cb80 1 Processor -- start 2024-02-02T15:30:11.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.765+0000 7f276d600700 1 -- 172.21.15.1:0/3004079198 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f27600cd4c0 con 0x7f2770100ff0 2024-02-02T15:30:11.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.766+0000 7f278b61cb80 1 -- start start 2024-02-02T15:30:11.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.766+0000 7f276fe00700 1 --2- 172.21.15.1:0/3004079198 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f27500a0710 0x7f27500a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:11.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.766+0000 7f278b61cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642ca724010 0x5642ca7243f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:11.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.766+0000 7f278b61cb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5642ca724930 0x5642ca72f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:11.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.766+0000 7f278b61cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5642ca72f7e0 0x5642ca731bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:11.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.766+0000 7f278b61cb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642ca6984c0 con 0x5642ca724930 2024-02-02T15:30:11.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.766+0000 7f278b61cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642ca67caa0 con 0x5642ca72f7e0 2024-02-02T15:30:11.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.766+0000 7f278b61cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642ca67a460 con 0x5642ca724010 2024-02-02T15:30:11.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.766+0000 7f2775000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642ca724010 0x5642ca7243f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:11.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.766+0000 7f276fe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5642ca724930 0x5642ca72f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:11.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.766+0000 7f2775000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642ca724010 0x5642ca7243f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:43362/0 (socket says 172.21.15.1:43362) 2024-02-02T15:30:11.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.766+0000 7f2775000700 1 -- 172.21.15.1:0/551211943 learned_addr learned my addr 172.21.15.1:0/551211943 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:30:11.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.766+0000 7f2775a00700 1 --2- 172.21.15.1:0/551211943 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5642ca72f7e0 0x5642ca731bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:11.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.767+0000 7f276fe00700 1 --2- 172.21.15.1:0/3004079198 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f27500a0710 0x7f27500a2bd0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2770074830 tx=0x7f2758009260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:11.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.767+0000 7f2775000700 1 -- 172.21.15.1:0/551211943 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5642ca72f7e0 msgr2=0x5642ca731bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:11.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.767+0000 7f2775000700 1 --2- 172.21.15.1:0/551211943 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5642ca72f7e0 0x5642ca731bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.767+0000 7f2775000700 1 -- 172.21.15.1:0/551211943 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5642ca724930 msgr2=0x5642ca72f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:11.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.767+0000 7f2775000700 1 --2- 172.21.15.1:0/551211943 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5642ca724930 0x5642ca72f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.767+0000 7f2775000700 1 -- 172.21.15.1:0/551211943 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5642ca4f73b0 con 0x5642ca724010 2024-02-02T15:30:11.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.767+0000 7f2775000700 1 --2- 172.21.15.1:0/551211943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642ca724010 0x5642ca7243f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2760029000 tx=0x7f2760009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57071730c8ec2708 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:11.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.767+0000 7f2766000700 1 -- 172.21.15.1:0/551211943 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276009a7e0 con 0x5642ca724010 2024-02-02T15:30:11.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.767+0000 7f2766000700 1 -- 172.21.15.1:0/551211943 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f276009e610 con 0x5642ca724010 2024-02-02T15:30:11.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.767+0000 7f2766000700 1 -- 172.21.15.1:0/551211943 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276009ac80 con 0x5642ca724010 2024-02-02T15:30:11.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.768+0000 7f278b61cb80 1 -- 172.21.15.1:0/551211943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642ca724010 msgr2=0x5642ca7243f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:11.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.768+0000 7f278b61cb80 1 --2- 172.21.15.1:0/551211943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642ca724010 0x5642ca7243f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2760029000 tx=0x7f2760009310 comp rx=0 tx=0).stop 2024-02-02T15:30:11.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.768+0000 7f278b61cb80 1 -- 172.21.15.1:0/551211943 shutdown_connections 2024-02-02T15:30:11.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.768+0000 7f278b61cb80 1 --2- 172.21.15.1:0/551211943 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5642ca72f7e0 0x5642ca731bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.768+0000 7f278b61cb80 1 --2- 172.21.15.1:0/551211943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642ca724010 0x5642ca7243f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.768+0000 7f278b61cb80 1 --2- 172.21.15.1:0/551211943 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5642ca724930 0x5642ca72f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.768+0000 7f278b61cb80 1 -- 172.21.15.1:0/551211943 >> 172.21.15.1:0/551211943 conn(0x5642ca683db0 msgr2=0x5642ca7203b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:11.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.768+0000 7f278b61cb80 1 -- 172.21.15.1:0/551211943 shutdown_connections 2024-02-02T15:30:11.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.768+0000 7f278b61cb80 1 -- 172.21.15.1:0/551211943 wait complete. 2024-02-02T15:30:11.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.769+0000 7f278b61cb80 1 -- start start 2024-02-02T15:30:11.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.769+0000 7f278b61cb80 1 Processor -- start 2024-02-02T15:30:11.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.769+0000 7f278b61cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642ca724010 0x5642ca72fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:11.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.769+0000 7f278b61cb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5642ca724930 0x5642ca730100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:11.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.769+0000 7f278b61cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5642ca72f7e0 0x5642ca730640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:11.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.769+0000 7f278b61cb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642ca675d50 con 0x5642ca724930 2024-02-02T15:30:11.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.769+0000 7f278b61cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642ca611d80 con 0x5642ca72f7e0 2024-02-02T15:30:11.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.769+0000 7f278b61cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642ca611a20 con 0x5642ca724010 2024-02-02T15:30:11.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.769+0000 7f278b61cb80 10 client.0 _ll_register_callbacks cb 0x5642ca613290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:30:11.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.770+0000 7f276fe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5642ca724930 0x5642ca730100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:11.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.770+0000 7f276fe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5642ca724930 0x5642ca730100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54648/0 (socket says 172.21.15.1:54648) 2024-02-02T15:30:11.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.770+0000 7f276fe00700 1 -- 172.21.15.1:0/2223009819 learned_addr learned my addr 172.21.15.1:0/2223009819 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:30:11.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.770+0000 7f2775000700 1 --2- 172.21.15.1:0/2223009819 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642ca724010 0x5642ca72fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:11.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.770+0000 7f2775a00700 1 --2- 172.21.15.1:0/2223009819 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5642ca72f7e0 0x5642ca730640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:11.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.770+0000 7f276fe00700 1 -- 172.21.15.1:0/2223009819 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642ca724010 msgr2=0x5642ca72fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:11.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.770+0000 7f276fe00700 1 --2- 172.21.15.1:0/2223009819 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642ca724010 0x5642ca72fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.770+0000 7f276fe00700 1 -- 172.21.15.1:0/2223009819 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5642ca72f7e0 msgr2=0x5642ca730640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:11.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.770+0000 7f276fe00700 1 --2- 172.21.15.1:0/2223009819 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5642ca72f7e0 0x5642ca730640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.770+0000 7f276fe00700 1 -- 172.21.15.1:0/2223009819 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27701a6160 con 0x5642ca724930 2024-02-02T15:30:11.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.770+0000 7f276fe00700 1 --2- 172.21.15.1:0/2223009819 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5642ca724930 0x5642ca730100 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f275801d8f0 tx=0x7f27580047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e7a3c747c45ca26 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:11.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.770+0000 7f275ea00700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:30:11.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.770+0000 7f275ea00700 1 -- 172.21.15.1:0/2223009819 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2758022070 con 0x5642ca724930 2024-02-02T15:30:11.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.771+0000 7f275ea00700 1 -- 172.21.15.1:0/2223009819 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f2758004830 con 0x5642ca724930 2024-02-02T15:30:11.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.771+0000 7f278b61cb80 1 -- 172.21.15.1:0/2223009819 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5642ca7180f0 con 0x5642ca724930 2024-02-02T15:30:11.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.771+0000 7f275ea00700 1 -- 172.21.15.1:0/2223009819 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2758004a30 con 0x5642ca724930 2024-02-02T15:30:11.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.771+0000 7f278b61cb80 10 client.16467 fetch_fsmap learned FSMap version 31 2024-02-02T15:30:11.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.771+0000 7f278b61cb80 1 -- 172.21.15.1:0/2223009819 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5642ca4a3b60 con 0x5642ca724930 2024-02-02T15:30:11.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.771+0000 7f275ea00700 1 -- 172.21.15.1:0/2223009819 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5642ca7180f0 con 0x5642ca724930 2024-02-02T15:30:11.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.771+0000 7f275ea00700 1 -- 172.21.15.1:0/2223009819 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 31) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f2758005040 con 0x5642ca724930 2024-02-02T15:30:11.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.771+0000 7f278b61cb80 10 client.16467 fetch_fsmap finished waiting for FSMap version 31 2024-02-02T15:30:11.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.771+0000 7f278b61cb80 10 client.16467 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:30:11.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.771+0000 7f278b61cb80 20 client.16467 populate_metadata read hostname 'smithi001' 2024-02-02T15:30:11.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.771+0000 7f278b61cb80 1 --2- 172.21.15.1:0/2223009819 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5642ca73ef70 0x5642ca741390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:11.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.771+0000 7f278b61cb80 4 client.16467 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:30:11.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.771+0000 7f278b61cb80 1 -- 172.21.15.1:0/2223009819 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5642ca66edd0 con 0x5642ca73ef70 2024-02-02T15:30:11.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.772+0000 7f2775000700 1 --2- 172.21.15.1:0/2223009819 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5642ca73ef70 0x5642ca741390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:11.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.772+0000 7f2775000700 1 --2- 172.21.15.1:0/2223009819 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5642ca73ef70 0x5642ca741390 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f98bc3f8a7b2e316 server_cookie=70f7b4a9e90ef343 in_seq=0 out_seq=0 2024-02-02T15:30:11.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.772+0000 7f275ea00700 10 client.16467 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:30:11.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.773+0000 7f275ea00700 1 -- 172.21.15.1:0/2223009819 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5642ca66edd0 con 0x5642ca73ef70 2024-02-02T15:30:11.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.773+0000 7f275ea00700 10 client.16467 handle_command_reply: tid=0 2024-02-02T15:30:11.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.773+0000 7f278b61cb80 1 client.16467 shutdown 2024-02-02T15:30:11.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.773+0000 7f278b61cb80 1 -- 172.21.15.1:0/2223009819 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5642ca724930 msgr2=0x5642ca730100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:11.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.773+0000 7f278b61cb80 1 --2- 172.21.15.1:0/2223009819 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5642ca724930 0x5642ca730100 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f275801d8f0 tx=0x7f27580047c0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.773+0000 7f278b61cb80 1 -- 172.21.15.1:0/2223009819 shutdown_connections 2024-02-02T15:30:11.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.773+0000 7f278b61cb80 1 --2- 172.21.15.1:0/2223009819 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5642ca73ef70 0x5642ca741390 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.773+0000 7f278b61cb80 1 --2- 172.21.15.1:0/2223009819 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5642ca72f7e0 0x5642ca730640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.773+0000 7f278b61cb80 1 --2- 172.21.15.1:0/2223009819 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642ca724010 0x5642ca72fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.773+0000 7f275ea00700 0 client.16467 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:30:11.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.773+0000 7f278b61cb80 1 --2- 172.21.15.1:0/2223009819 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5642ca724930 0x5642ca730100 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.773+0000 7f278b61cb80 1 -- 172.21.15.1:0/2223009819 >> 172.21.15.1:0/2223009819 conn(0x5642ca683db0 msgr2=0x5642ca71ffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:11.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.773+0000 7f278b61cb80 1 -- 172.21.15.1:0/2223009819 shutdown_connections 2024-02-02T15:30:11.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.773+0000 7f278b61cb80 1 -- 172.21.15.1:0/2223009819 wait complete. 2024-02-02T15:30:11.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.774+0000 7f278b61cb80 20 client.16467 trim_cache size 0 max 16384 2024-02-02T15:30:11.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.787+0000 7f278b61cb80 1 Processor -- start 2024-02-02T15:30:11.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.787+0000 7f278b61cb80 1 -- start start 2024-02-02T15:30:11.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.787+0000 7f278b61cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5642ca724010 0x5642ca733bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:11.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.787+0000 7f278b61cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642ca724930 0x5642ca726ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:11.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.787+0000 7f278b61cb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5642ca72f7e0 0x5642ca72bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:11.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.787+0000 7f278b61cb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642ca4f96a0 con 0x5642ca72f7e0 2024-02-02T15:30:11.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.788+0000 7f278b61cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642ca73aa80 con 0x5642ca724010 2024-02-02T15:30:11.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.788+0000 7f278b61cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642ca721a10 con 0x5642ca724930 2024-02-02T15:30:11.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.788+0000 7f2775a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5642ca72f7e0 0x5642ca72bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:11.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.788+0000 7f2775a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5642ca72f7e0 0x5642ca72bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54658/0 (socket says 172.21.15.1:54658) 2024-02-02T15:30:11.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.788+0000 7f2775000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5642ca724010 0x5642ca733bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:11.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.788+0000 7f2775a00700 1 -- 172.21.15.1:0/561355518 learned_addr learned my addr 172.21.15.1:0/561355518 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:30:11.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.788+0000 7f276fe00700 1 --2- 172.21.15.1:0/561355518 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642ca724930 0x5642ca726ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:11.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.788+0000 7f2775a00700 1 -- 172.21.15.1:0/561355518 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642ca724930 msgr2=0x5642ca726ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:11.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.788+0000 7f2775a00700 1 --2- 172.21.15.1:0/561355518 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642ca724930 0x5642ca726ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.788+0000 7f2775a00700 1 -- 172.21.15.1:0/561355518 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5642ca724010 msgr2=0x5642ca733bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:11.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.788+0000 7f2775a00700 1 --2- 172.21.15.1:0/561355518 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5642ca724010 0x5642ca733bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.788+0000 7f2775a00700 1 -- 172.21.15.1:0/561355518 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5642ca72e3e0 con 0x5642ca72f7e0 2024-02-02T15:30:11.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.788+0000 7f276fe00700 1 --2- 172.21.15.1:0/561355518 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642ca724930 0x5642ca726ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:30:11.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.788+0000 7f2775a00700 1 --2- 172.21.15.1:0/561355518 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5642ca72f7e0 0x5642ca72bae0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f2768009fd0 tx=0x7f2768007500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e419c5d31fb1e41a server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:11.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.789+0000 7f2766a00700 1 -- 172.21.15.1:0/561355518 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2768003a00 con 0x5642ca72f7e0 2024-02-02T15:30:11.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.789+0000 7f2766a00700 1 -- 172.21.15.1:0/561355518 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f276800f040 con 0x5642ca72f7e0 2024-02-02T15:30:11.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.789+0000 7f2766a00700 1 -- 172.21.15.1:0/561355518 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2768017670 con 0x5642ca72f7e0 2024-02-02T15:30:11.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.789+0000 7f278b61cb80 1 -- 172.21.15.1:0/561355518 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5642ca72f7e0 msgr2=0x5642ca72bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:11.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.789+0000 7f278b61cb80 1 --2- 172.21.15.1:0/561355518 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5642ca72f7e0 0x5642ca72bae0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f2768009fd0 tx=0x7f2768007500 comp rx=0 tx=0).stop 2024-02-02T15:30:11.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.789+0000 7f278b61cb80 1 -- 172.21.15.1:0/561355518 shutdown_connections 2024-02-02T15:30:11.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.789+0000 7f278b61cb80 1 --2- 172.21.15.1:0/561355518 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5642ca72f7e0 0x5642ca72bae0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.789+0000 7f278b61cb80 1 --2- 172.21.15.1:0/561355518 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5642ca724010 0x5642ca733bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.789+0000 7f278b61cb80 1 --2- 172.21.15.1:0/561355518 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642ca724930 0x5642ca726ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.789+0000 7f278b61cb80 1 -- 172.21.15.1:0/561355518 >> 172.21.15.1:0/561355518 conn(0x5642ca683db0 msgr2=0x5642ca73b190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:11.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.789+0000 7f278b61cb80 1 -- 172.21.15.1:0/561355518 shutdown_connections 2024-02-02T15:30:11.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.789+0000 7f278b61cb80 1 -- 172.21.15.1:0/561355518 wait complete. 2024-02-02T15:30:11.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.790+0000 7f278b61cb80 1 -- start start 2024-02-02T15:30:11.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.790+0000 7f278b61cb80 1 Processor -- start 2024-02-02T15:30:11.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.790+0000 7f278b61cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5642ca724010 0x5642ca78a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:11.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.790+0000 7f278b61cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642ca724930 0x5642ca78a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:11.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.790+0000 7f278b61cb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5642ca72f7e0 0x5642ca784a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:11.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.790+0000 7f278b61cb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642ca785000 con 0x5642ca72f7e0 2024-02-02T15:30:11.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.790+0000 7f278b61cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642ca785170 con 0x5642ca724010 2024-02-02T15:30:11.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.790+0000 7f278b61cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642ca785330 con 0x5642ca724930 2024-02-02T15:30:11.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.791+0000 7f278b61cb80 10 client.0 _ll_register_callbacks cb 0x5642ca613290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:30:11.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.791+0000 7f276fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642ca724930 0x5642ca78a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:11.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.791+0000 7f2775a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5642ca72f7e0 0x5642ca784a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:11.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.791+0000 7f276fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642ca724930 0x5642ca78a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:43386/0 (socket says 172.21.15.1:43386) 2024-02-02T15:30:11.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.791+0000 7f276fe00700 1 -- 172.21.15.1:0/717976388 learned_addr learned my addr 172.21.15.1:0/717976388 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:30:11.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.791+0000 7f2775000700 1 --2- 172.21.15.1:0/717976388 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5642ca724010 0x5642ca78a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:11.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.791+0000 7f276fe00700 1 -- 172.21.15.1:0/717976388 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5642ca724010 msgr2=0x5642ca78a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:11.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.791+0000 7f276fe00700 1 --2- 172.21.15.1:0/717976388 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5642ca724010 0x5642ca78a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.791+0000 7f276fe00700 1 -- 172.21.15.1:0/717976388 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5642ca72f7e0 msgr2=0x5642ca784a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:11.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.791+0000 7f276fe00700 1 --2- 172.21.15.1:0/717976388 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5642ca72f7e0 0x5642ca784a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.791+0000 7f276fe00700 1 -- 172.21.15.1:0/717976388 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5642ca4a3b60 con 0x5642ca724930 2024-02-02T15:30:11.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.791+0000 7f276fe00700 1 --2- 172.21.15.1:0/717976388 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642ca724930 0x5642ca78a7e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f27580054a0 tx=0x7f2758009760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57325314f511231c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:11.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.791+0000 7f275ea00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:30:11.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.792+0000 7f275ea00700 1 -- 172.21.15.1:0/717976388 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2758022070 con 0x5642ca724930 2024-02-02T15:30:11.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.792+0000 7f275ea00700 1 -- 172.21.15.1:0/717976388 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f2758004400 con 0x5642ca724930 2024-02-02T15:30:11.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.792+0000 7f278b61cb80 1 -- 172.21.15.1:0/717976388 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f276009e610 con 0x5642ca724930 2024-02-02T15:30:11.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.792+0000 7f275ea00700 1 -- 172.21.15.1:0/717976388 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f27580045e0 con 0x5642ca724930 2024-02-02T15:30:11.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.792+0000 7f278b61cb80 10 client.26521 fetch_fsmap learned FSMap version 31 2024-02-02T15:30:11.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.792+0000 7f278b61cb80 1 -- 172.21.15.1:0/717976388 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f2768009c80 con 0x5642ca724930 2024-02-02T15:30:11.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.792+0000 7f275ea00700 1 -- 172.21.15.1:0/717976388 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f276009e610 con 0x5642ca724930 2024-02-02T15:30:11.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.792+0000 7f275ea00700 1 -- 172.21.15.1:0/717976388 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 31) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f275801cac0 con 0x5642ca724930 2024-02-02T15:30:11.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.792+0000 7f278b61cb80 10 client.26521 fetch_fsmap finished waiting for FSMap version 31 2024-02-02T15:30:11.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.792+0000 7f278b61cb80 10 client.26521 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:30:11.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.792+0000 7f278b61cb80 20 client.26521 populate_metadata read hostname 'smithi001' 2024-02-02T15:30:11.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.792+0000 7f278b61cb80 1 --2- 172.21.15.1:0/717976388 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5642ca792230 0x5642ca794650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:11.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.792+0000 7f278b61cb80 4 client.26521 mds_command: new command op to 24557 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:30:11.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.792+0000 7f278b61cb80 1 -- 172.21.15.1:0/717976388 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5642ca634f50 con 0x5642ca792230 2024-02-02T15:30:11.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.793+0000 7f2775000700 1 --2- 172.21.15.1:0/717976388 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5642ca792230 0x5642ca794650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:11.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.793+0000 7f2775000700 1 --2- 172.21.15.1:0/717976388 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5642ca792230 0x5642ca794650 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=156e09a780bfd88d server_cookie=5c076e6867a6c8b4 in_seq=0 out_seq=0 2024-02-02T15:30:11.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.793+0000 7f275ea00700 10 client.26521 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:30:11.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.793+0000 7f275ea00700 1 -- 172.21.15.1:0/717976388 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5642ca634f50 con 0x5642ca792230 2024-02-02T15:30:11.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.793+0000 7f275ea00700 10 client.26521 handle_command_reply: tid=0 2024-02-02T15:30:11.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.793+0000 7f278b61cb80 1 client.26521 shutdown 2024-02-02T15:30:11.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.793+0000 7f278b61cb80 1 -- 172.21.15.1:0/717976388 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642ca724930 msgr2=0x5642ca78a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:11.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.793+0000 7f278b61cb80 1 --2- 172.21.15.1:0/717976388 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642ca724930 0x5642ca78a7e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f27580054a0 tx=0x7f2758009760 comp rx=0 tx=0).stop 2024-02-02T15:30:11.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.793+0000 7f278b61cb80 1 -- 172.21.15.1:0/717976388 shutdown_connections 2024-02-02T15:30:11.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.793+0000 7f278b61cb80 1 --2- 172.21.15.1:0/717976388 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5642ca792230 0x5642ca794650 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.793+0000 7f278b61cb80 1 --2- 172.21.15.1:0/717976388 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5642ca72f7e0 0x5642ca784a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.793+0000 7f278b61cb80 1 --2- 172.21.15.1:0/717976388 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5642ca724010 0x5642ca78a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.793+0000 7f278b61cb80 1 --2- 172.21.15.1:0/717976388 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5642ca724930 0x5642ca78a7e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.793+0000 7f275ea00700 0 client.26521 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:30:11.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.793+0000 7f278b61cb80 1 -- 172.21.15.1:0/717976388 >> 172.21.15.1:0/717976388 conn(0x5642ca683db0 msgr2=0x5642ca72d020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:11.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.794+0000 7f278b61cb80 1 -- 172.21.15.1:0/717976388 shutdown_connections 2024-02-02T15:30:11.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.794+0000 7f278b61cb80 1 -- 172.21.15.1:0/717976388 wait complete. 2024-02-02T15:30:11.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.794+0000 7f278b61cb80 20 client.26521 trim_cache size 0 max 16384 2024-02-02T15:30:11.798 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:30:11.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.794+0000 7f2777a00700 1 -- 172.21.15.1:0/3004079198 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f27500a0710 msgr2=0x7f27500a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:11.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.794+0000 7f2777a00700 1 --2- 172.21.15.1:0/3004079198 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f27500a0710 0x7f27500a2bd0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2770074830 tx=0x7f2758009260 comp rx=0 tx=0).stop 2024-02-02T15:30:11.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.794+0000 7f2777a00700 1 -- 172.21.15.1:0/3004079198 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2770100ff0 msgr2=0x7f2770073040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:11.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.794+0000 7f2777a00700 1 --2- 172.21.15.1:0/3004079198 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2770100ff0 0x7f2770073040 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2760005910 tx=0x7f2760005740 comp rx=0 tx=0).stop 2024-02-02T15:30:11.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.794+0000 7f2777a00700 1 -- 172.21.15.1:0/3004079198 shutdown_connections 2024-02-02T15:30:11.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.794+0000 7f2777a00700 1 --2- 172.21.15.1:0/3004079198 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f27500a0710 0x7f27500a2bd0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.794+0000 7f2777a00700 1 --2- 172.21.15.1:0/3004079198 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2770073ac0 0x7f27701a50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.794+0000 7f2777a00700 1 --2- 172.21.15.1:0/3004079198 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2770100ff0 0x7f2770073040 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.794+0000 7f2777a00700 1 --2- 172.21.15.1:0/3004079198 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2770106110 0x7f2770073580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:11.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.794+0000 7f2777a00700 1 -- 172.21.15.1:0/3004079198 >> 172.21.15.1:0/3004079198 conn(0x7f27700fc810 msgr2=0x7f2770104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:11.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.794+0000 7f2777a00700 1 -- 172.21.15.1:0/3004079198 shutdown_connections 2024-02-02T15:30:11.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:11.795+0000 7f2777a00700 1 -- 172.21.15.1:0/3004079198 wait complete. 2024-02-02T15:30:11.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:11 smithi071 ceph-mon[44707]: pgmap v98: 129 pgs: 129 active+clean; 69 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 497 KiB/s wr, 73 op/s 2024-02-02T15:30:11.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:11 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/1238693356' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:30:11.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:11 smithi142 ceph-mon[45728]: pgmap v98: 129 pgs: 129 active+clean; 69 MiB data, 619 MiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 497 KiB/s wr, 73 op/s 2024-02-02T15:30:11.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:11 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/1238693356' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:30:12.226 INFO:tasks.workunit.client.0.smithi001.stdout: 1 117246 6.14 MB/sec warmup 106 sec latency 877.676 ms 2024-02-02T15:30:12.813 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-02T15:30:12.813 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-02T15:30:13.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.027+0000 7f08d5200700 1 -- 172.21.15.1:0/1946952775 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f08d0100ff0 msgr2=0x7f08d01013d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:13.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.027+0000 7f08d5200700 1 --2- 172.21.15.1:0/1946952775 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f08d0100ff0 0x7f08d01013d0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f08c00098f0 tx=0x7f08c0009c00 comp rx=0 tx=0).stop 2024-02-02T15:30:13.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.027+0000 7f08d5200700 1 -- 172.21.15.1:0/1946952775 shutdown_connections 2024-02-02T15:30:13.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.027+0000 7f08d5200700 1 --2- 172.21.15.1:0/1946952775 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f08d0106110 0x7f08d010dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:13.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.027+0000 7f08d5200700 1 --2- 172.21.15.1:0/1946952775 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f08d0100ff0 0x7f08d01013d0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:13.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.027+0000 7f08d5200700 1 --2- 172.21.15.1:0/1946952775 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08d01019a0 0x7f08d01059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:13.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.027+0000 7f08d5200700 1 -- 172.21.15.1:0/1946952775 >> 172.21.15.1:0/1946952775 conn(0x7f08d00fc810 msgr2=0x7f08d00fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:13.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.027+0000 7f08d5200700 1 -- 172.21.15.1:0/1946952775 shutdown_connections 2024-02-02T15:30:13.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.027+0000 7f08d5200700 1 -- 172.21.15.1:0/1946952775 wait complete. 2024-02-02T15:30:13.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.028+0000 7f08d5200700 1 Processor -- start 2024-02-02T15:30:13.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.028+0000 7f08d5200700 1 -- start start 2024-02-02T15:30:13.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.029+0000 7f08d5200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08d0100ff0 0x7f08d0199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:13.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.029+0000 7f08d5200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f08d0106110 0x7f08d0199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:13.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.029+0000 7f08d5200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f08d019dac0 0x7f08d019df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:13.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.029+0000 7f08d5200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d019e540 con 0x7f08d0106110 2024-02-02T15:30:13.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.029+0000 7f08d5200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d019e6b0 con 0x7f08d019dac0 2024-02-02T15:30:13.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.029+0000 7f08d5200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d019e870 con 0x7f08d0100ff0 2024-02-02T15:30:13.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.029+0000 7f08cde00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08d0100ff0 0x7f08d0199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:13.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.029+0000 7f08cd400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f08d0106110 0x7f08d0199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:13.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.029+0000 7f08cde00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08d0100ff0 0x7f08d0199120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:39694/0 (socket says 172.21.15.1:39694) 2024-02-02T15:30:13.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.029+0000 7f08cde00700 1 -- 172.21.15.1:0/3308693537 learned_addr learned my addr 172.21.15.1:0/3308693537 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:30:13.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.029+0000 7f08ce800700 1 --2- 172.21.15.1:0/3308693537 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f08d019dac0 0x7f08d019df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:13.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.029+0000 7f08cde00700 1 -- 172.21.15.1:0/3308693537 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f08d019dac0 msgr2=0x7f08d019df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:13.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.029+0000 7f08cde00700 1 --2- 172.21.15.1:0/3308693537 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f08d019dac0 0x7f08d019df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:13.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.029+0000 7f08cde00700 1 -- 172.21.15.1:0/3308693537 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f08d0106110 msgr2=0x7f08d0199660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:13.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.029+0000 7f08cde00700 1 --2- 172.21.15.1:0/3308693537 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f08d0106110 0x7f08d0199660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:13.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.029+0000 7f08cde00700 1 -- 172.21.15.1:0/3308693537 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08c8009c40 con 0x7f08d0100ff0 2024-02-02T15:30:13.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.029+0000 7f08cd400700 1 --2- 172.21.15.1:0/3308693537 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f08d0106110 0x7f08d0199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:30:13.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.030+0000 7f08cde00700 1 --2- 172.21.15.1:0/3308693537 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08d0100ff0 0x7f08d0199120 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f08c0009810 tx=0x7f08c000bfc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:13.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.030+0000 7f08c6a00700 1 -- 172.21.15.1:0/3308693537 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c001c070 con 0x7f08d0100ff0 2024-02-02T15:30:13.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.030+0000 7f08d5200700 1 -- 172.21.15.1:0/3308693537 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08c0009520 con 0x7f08d0100ff0 2024-02-02T15:30:13.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.030+0000 7f08c6a00700 1 -- 172.21.15.1:0/3308693537 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f08c0005040 con 0x7f08d0100ff0 2024-02-02T15:30:13.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.030+0000 7f08c6a00700 1 -- 172.21.15.1:0/3308693537 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c00049c0 con 0x7f08d0100ff0 2024-02-02T15:30:13.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.030+0000 7f08d5200700 1 -- 172.21.15.1:0/3308693537 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08d01a6060 con 0x7f08d0100ff0 2024-02-02T15:30:13.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.033+0000 7f08c6a00700 1 -- 172.21.15.1:0/3308693537 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 27) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f08c0020070 con 0x7f08d0100ff0 2024-02-02T15:30:13.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.034+0000 7f08c6a00700 1 --2- 172.21.15.1:0/3308693537 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f08b00a0690 0x7f08b00a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:13.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.034+0000 7f08e8dfab80 1 Processor -- start 2024-02-02T15:30:13.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.034+0000 7f08c6a00700 1 -- 172.21.15.1:0/3308693537 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f08c00ccd80 con 0x7f08d0100ff0 2024-02-02T15:30:13.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.034+0000 7f08e8dfab80 1 -- start start 2024-02-02T15:30:13.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.035+0000 7f08cd400700 1 --2- 172.21.15.1:0/3308693537 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f08b00a0690 0x7f08b00a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:13.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.035+0000 7f08e8dfab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b8a88bf090 0x55b8a88bf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:13.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.035+0000 7f08e8dfab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b8a88bf9b0 0x55b8a88ca320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:13.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.035+0000 7f08e8dfab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b8a88ca860 0x55b8a88ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:13.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.035+0000 7f08e8dfab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8a8833540 con 0x55b8a88bf9b0 2024-02-02T15:30:13.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.035+0000 7f08e8dfab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8a8817b20 con 0x55b8a88bf090 2024-02-02T15:30:13.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.035+0000 7f08e8dfab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8a88154e0 con 0x55b8a88ca860 2024-02-02T15:30:13.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.035+0000 7f08cd400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b8a88bf9b0 0x55b8a88ca320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:13.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.035+0000 7f08cd400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b8a88bf9b0 0x55b8a88ca320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40680/0 (socket says 172.21.15.1:40680) 2024-02-02T15:30:13.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.035+0000 7f08cd400700 1 -- 172.21.15.1:0/1820696814 learned_addr learned my addr 172.21.15.1:0/1820696814 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:30:13.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.035+0000 7f08cde00700 1 --2- 172.21.15.1:0/1820696814 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b8a88bf090 0x55b8a88bf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:13.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.035+0000 7f08ce800700 1 --2- 172.21.15.1:0/1820696814 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b8a88ca860 0x55b8a88ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:13.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.036+0000 7f08cd400700 1 -- 172.21.15.1:0/1820696814 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b8a88ca860 msgr2=0x55b8a88ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:13.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.036+0000 7f08cd400700 1 --2- 172.21.15.1:0/1820696814 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b8a88ca860 0x55b8a88ccc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:13.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.036+0000 7f08cd400700 1 -- 172.21.15.1:0/1820696814 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b8a88bf090 msgr2=0x55b8a88bf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:13.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.036+0000 7f08cd400700 1 --2- 172.21.15.1:0/1820696814 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b8a88bf090 0x55b8a88bf470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:13.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.036+0000 7f08cd400700 1 -- 172.21.15.1:0/1820696814 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b8a8692430 con 0x55b8a88bf9b0 2024-02-02T15:30:13.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.036+0000 7f08cd400700 1 --2- 172.21.15.1:0/3308693537 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f08b00a0690 0x7f08b00a2b50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f08b8013950 tx=0x7f08b8013d10 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:13.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.036+0000 7f08cd400700 1 --2- 172.21.15.1:0/1820696814 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b8a88bf9b0 0x55b8a88ca320 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f08b800f6a0 tx=0x7f08b8020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76803a68def65f26 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:13.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.036+0000 7f08bf400700 1 -- 172.21.15.1:0/1820696814 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b800fe90 con 0x55b8a88bf9b0 2024-02-02T15:30:13.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.036+0000 7f08bf400700 1 -- 172.21.15.1:0/1820696814 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f08b80044f0 con 0x55b8a88bf9b0 2024-02-02T15:30:13.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.036+0000 7f08bf400700 1 -- 172.21.15.1:0/1820696814 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b8005020 con 0x55b8a88bf9b0 2024-02-02T15:30:13.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.037+0000 7f08e8dfab80 1 -- 172.21.15.1:0/1820696814 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b8a88bf9b0 msgr2=0x55b8a88ca320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:13.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.037+0000 7f08e8dfab80 1 --2- 172.21.15.1:0/1820696814 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b8a88bf9b0 0x55b8a88ca320 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f08b800f6a0 tx=0x7f08b8020000 comp rx=0 tx=0).stop 2024-02-02T15:30:13.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.037+0000 7f08e8dfab80 1 -- 172.21.15.1:0/1820696814 shutdown_connections 2024-02-02T15:30:13.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.037+0000 7f08e8dfab80 1 --2- 172.21.15.1:0/1820696814 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b8a88ca860 0x55b8a88ccc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:13.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.037+0000 7f08e8dfab80 1 --2- 172.21.15.1:0/1820696814 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b8a88bf090 0x55b8a88bf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:13.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.037+0000 7f08e8dfab80 1 --2- 172.21.15.1:0/1820696814 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b8a88bf9b0 0x55b8a88ca320 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:13.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.037+0000 7f08e8dfab80 1 -- 172.21.15.1:0/1820696814 >> 172.21.15.1:0/1820696814 conn(0x55b8a881ee30 msgr2=0x55b8a88bb430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:13.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.037+0000 7f08e8dfab80 1 -- 172.21.15.1:0/1820696814 shutdown_connections 2024-02-02T15:30:13.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.037+0000 7f08e8dfab80 1 -- 172.21.15.1:0/1820696814 wait complete. 2024-02-02T15:30:13.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.037+0000 7f08e8dfab80 1 -- start start 2024-02-02T15:30:13.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.038+0000 7f08e8dfab80 1 Processor -- start 2024-02-02T15:30:13.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.038+0000 7f08e8dfab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b8a88bf090 0x55b8a88ca2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:13.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.038+0000 7f08e8dfab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b8a88bf9b0 0x55b8a88c74c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:13.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.038+0000 7f08e8dfab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b8a88ca860 0x55b8a88c7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:13.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.038+0000 7f08e8dfab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8a8810dd0 con 0x55b8a88bf090 2024-02-02T15:30:13.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.038+0000 7f08e8dfab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8a87ace00 con 0x55b8a88bf9b0 2024-02-02T15:30:13.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.038+0000 7f08e8dfab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8a87acaa0 con 0x55b8a88ca860 2024-02-02T15:30:13.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.038+0000 7f08e8dfab80 10 client.0 _ll_register_callbacks cb 0x55b8a87ae310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:30:13.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.038+0000 7f08cde00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b8a88bf090 0x55b8a88ca2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:13.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.038+0000 7f08cde00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b8a88bf090 0x55b8a88ca2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40692/0 (socket says 172.21.15.1:40692) 2024-02-02T15:30:13.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.039+0000 7f08cde00700 1 -- 172.21.15.1:0/1063271805 learned_addr learned my addr 172.21.15.1:0/1063271805 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:30:13.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.039+0000 7f08cd400700 1 --2- 172.21.15.1:0/1063271805 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b8a88bf9b0 0x55b8a88c74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:13.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.039+0000 7f08ce800700 1 --2- 172.21.15.1:0/1063271805 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b8a88ca860 0x55b8a88c7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:13.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.039+0000 7f08cde00700 1 -- 172.21.15.1:0/1063271805 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b8a88ca860 msgr2=0x55b8a88c7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:13.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.039+0000 7f08cde00700 1 --2- 172.21.15.1:0/1063271805 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b8a88ca860 0x55b8a88c7c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:13.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.039+0000 7f08cde00700 1 -- 172.21.15.1:0/1063271805 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b8a88bf9b0 msgr2=0x55b8a88c74c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:13.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.039+0000 7f08cde00700 1 --2- 172.21.15.1:0/1063271805 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b8a88bf9b0 0x55b8a88c74c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:13.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.039+0000 7f08cde00700 1 -- 172.21.15.1:0/1063271805 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08b800f8b0 con 0x55b8a88bf090 2024-02-02T15:30:13.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.039+0000 7f08cde00700 1 --2- 172.21.15.1:0/1063271805 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b8a88bf090 0x55b8a88ca2b0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f08c009a030 tx=0x7f08c0099a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef5404e1accc954f server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:13.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.039+0000 7f08b7e00700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:30:13.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.039+0000 7f08b7e00700 1 -- 172.21.15.1:0/1063271805 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c00056e0 con 0x55b8a88bf090 2024-02-02T15:30:13.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.039+0000 7f08b7e00700 1 -- 172.21.15.1:0/1063271805 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f08c009ded0 con 0x55b8a88bf090 2024-02-02T15:30:13.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.039+0000 7f08e8dfab80 1 -- 172.21.15.1:0/1063271805 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b8a88b3170 con 0x55b8a88bf090 2024-02-02T15:30:13.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.040+0000 7f08b7e00700 1 -- 172.21.15.1:0/1063271805 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08c00a2080 con 0x55b8a88bf090 2024-02-02T15:30:13.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.040+0000 7f08e8dfab80 10 client.16485 fetch_fsmap learned FSMap version 31 2024-02-02T15:30:13.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.040+0000 7f08e8dfab80 1 -- 172.21.15.1:0/1063271805 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55b8a863ebe0 con 0x55b8a88bf090 2024-02-02T15:30:13.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.040+0000 7f08b7e00700 1 -- 172.21.15.1:0/1063271805 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b8a88b3170 con 0x55b8a88bf090 2024-02-02T15:30:13.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.040+0000 7f08b7e00700 1 -- 172.21.15.1:0/1063271805 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 31) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f08c00a1bf0 con 0x55b8a88bf090 2024-02-02T15:30:13.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.040+0000 7f08e8dfab80 10 client.16485 fetch_fsmap finished waiting for FSMap version 31 2024-02-02T15:30:13.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.040+0000 7f08e8dfab80 10 client.16485 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:30:13.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.040+0000 7f08e8dfab80 20 client.16485 populate_metadata read hostname 'smithi001' 2024-02-02T15:30:13.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.040+0000 7f08e8dfab80 1 --2- 172.21.15.1:0/1063271805 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55b8a88d9d60 0x55b8a88dc180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:13.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.040+0000 7f08e8dfab80 4 client.16485 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:30:13.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.040+0000 7f08e8dfab80 1 -- 172.21.15.1:0/1063271805 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b8a8809e50 con 0x55b8a88d9d60 2024-02-02T15:30:13.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.041+0000 7f08cd400700 1 --2- 172.21.15.1:0/1063271805 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55b8a88d9d60 0x55b8a88dc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:13.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.041+0000 7f08cd400700 1 --2- 172.21.15.1:0/1063271805 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55b8a88d9d60 0x55b8a88dc180 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e370124c17e1de server_cookie=642747f98d312999 in_seq=0 out_seq=0 2024-02-02T15:30:13.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.041+0000 7f08b7e00700 10 client.16485 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:30:13.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.042+0000 7f08b7e00700 1 -- 172.21.15.1:0/1063271805 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55b8a8809e50 con 0x55b8a88d9d60 2024-02-02T15:30:13.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.042+0000 7f08b7e00700 10 client.16485 handle_command_reply: tid=0 2024-02-02T15:30:13.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.042+0000 7f08e8dfab80 1 client.16485 shutdown 2024-02-02T15:30:13.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.042+0000 7f08e8dfab80 1 -- 172.21.15.1:0/1063271805 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b8a88bf090 msgr2=0x55b8a88ca2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:13.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.042+0000 7f08e8dfab80 1 --2- 172.21.15.1:0/1063271805 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b8a88bf090 0x55b8a88ca2b0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f08c009a030 tx=0x7f08c0099a80 comp rx=0 tx=0).stop 2024-02-02T15:30:13.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.042+0000 7f08e8dfab80 1 -- 172.21.15.1:0/1063271805 shutdown_connections 2024-02-02T15:30:13.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.042+0000 7f08e8dfab80 1 --2- 172.21.15.1:0/1063271805 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55b8a88d9d60 0x55b8a88dc180 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:13.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.042+0000 7f08e8dfab80 1 --2- 172.21.15.1:0/1063271805 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b8a88ca860 0x55b8a88c7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:13.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.042+0000 7f08e8dfab80 1 --2- 172.21.15.1:0/1063271805 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b8a88bf090 0x55b8a88ca2b0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:13.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.042+0000 7f08e8dfab80 1 --2- 172.21.15.1:0/1063271805 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b8a88bf9b0 0x55b8a88c74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:13.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.042+0000 7f08e8dfab80 1 -- 172.21.15.1:0/1063271805 >> 172.21.15.1:0/1063271805 conn(0x55b8a881ee30 msgr2=0x55b8a88bb020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:13.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.042+0000 7f08b7e00700 0 client.16485 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:30:13.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.042+0000 7f08e8dfab80 1 -- 172.21.15.1:0/1063271805 shutdown_connections 2024-02-02T15:30:13.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.042+0000 7f08e8dfab80 1 -- 172.21.15.1:0/1063271805 wait complete. 2024-02-02T15:30:13.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.043+0000 7f08e8dfab80 20 client.16485 trim_cache size 0 max 16384 2024-02-02T15:30:13.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.060+0000 7f08e8dfab80 1 Processor -- start 2024-02-02T15:30:13.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.060+0000 7f08e8dfab80 1 -- start start 2024-02-02T15:30:13.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.060+0000 7f08e8dfab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b8a88bf090 0x55b8a88d12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:13.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.060+0000 7f08e8dfab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b8a88bf9b0 0x55b8a88c67c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:13.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.060+0000 7f08e8dfab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b8a88ca860 0x55b8a88c9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:13.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.060+0000 7f08e8dfab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8a8694720 con 0x55b8a88bf090 2024-02-02T15:30:13.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.060+0000 7f08e8dfab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8a88bc0e0 con 0x55b8a88bf9b0 2024-02-02T15:30:13.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.060+0000 7f08e8dfab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8a88bf7f0 con 0x55b8a88ca860 2024-02-02T15:30:13.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.061+0000 7f08cde00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b8a88bf090 0x55b8a88d12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:13.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.061+0000 7f08cde00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b8a88bf090 0x55b8a88d12f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40706/0 (socket says 172.21.15.1:40706) 2024-02-02T15:30:13.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.061+0000 7f08cd400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b8a88bf9b0 0x55b8a88c67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:13.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.061+0000 7f08cde00700 1 -- 172.21.15.1:0/615702674 learned_addr learned my addr 172.21.15.1:0/615702674 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:30:13.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.061+0000 7f08ce800700 1 --2- 172.21.15.1:0/615702674 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b8a88ca860 0x55b8a88c9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:13.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.061+0000 7f08cd400700 1 -- 172.21.15.1:0/615702674 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b8a88ca860 msgr2=0x55b8a88c9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:13.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.061+0000 7f08cd400700 1 --2- 172.21.15.1:0/615702674 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b8a88ca860 0x55b8a88c9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:13.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.061+0000 7f08cd400700 1 -- 172.21.15.1:0/615702674 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b8a88bf090 msgr2=0x55b8a88d12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:13.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.061+0000 7f08cd400700 1 --2- 172.21.15.1:0/615702674 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b8a88bf090 0x55b8a88d12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:13.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.061+0000 7f08cd400700 1 -- 172.21.15.1:0/615702674 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b8a88c7080 con 0x55b8a88bf9b0 2024-02-02T15:30:13.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.061+0000 7f08cde00700 1 --2- 172.21.15.1:0/615702674 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b8a88bf090 0x55b8a88d12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:30:13.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.061+0000 7f08cd400700 1 --2- 172.21.15.1:0/615702674 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b8a88bf9b0 0x55b8a88c67c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f08b8006010 tx=0x7f08b8021000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d377e768fc1708c1 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:13.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.062+0000 7f08bfe00700 1 -- 172.21.15.1:0/615702674 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b8009290 con 0x55b8a88bf9b0 2024-02-02T15:30:13.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.062+0000 7f08bfe00700 1 -- 172.21.15.1:0/615702674 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f08b8012150 con 0x55b8a88bf9b0 2024-02-02T15:30:13.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.062+0000 7f08bfe00700 1 -- 172.21.15.1:0/615702674 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b8004ac0 con 0x55b8a88bf9b0 2024-02-02T15:30:13.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.062+0000 7f08e8dfab80 1 -- 172.21.15.1:0/615702674 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b8a88bf9b0 msgr2=0x55b8a88c67c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:13.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.062+0000 7f08e8dfab80 1 --2- 172.21.15.1:0/615702674 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b8a88bf9b0 0x55b8a88c67c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f08b8006010 tx=0x7f08b8021000 comp rx=0 tx=0).stop 2024-02-02T15:30:13.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.062+0000 7f08e8dfab80 1 -- 172.21.15.1:0/615702674 shutdown_connections 2024-02-02T15:30:13.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.062+0000 7f08e8dfab80 1 --2- 172.21.15.1:0/615702674 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b8a88ca860 0x55b8a88c9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:13.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.062+0000 7f08e8dfab80 1 --2- 172.21.15.1:0/615702674 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b8a88bf090 0x55b8a88d12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:13.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.062+0000 7f08e8dfab80 1 --2- 172.21.15.1:0/615702674 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b8a88bf9b0 0x55b8a88c67c0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:13.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.062+0000 7f08e8dfab80 1 -- 172.21.15.1:0/615702674 >> 172.21.15.1:0/615702674 conn(0x55b8a881ee30 msgr2=0x55b8a88d58c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:13.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.062+0000 7f08e8dfab80 1 -- 172.21.15.1:0/615702674 shutdown_connections 2024-02-02T15:30:13.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.062+0000 7f08e8dfab80 1 -- 172.21.15.1:0/615702674 wait complete. 2024-02-02T15:30:13.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.063+0000 7f08e8dfab80 1 -- start start 2024-02-02T15:30:13.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.063+0000 7f08e8dfab80 1 Processor -- start 2024-02-02T15:30:13.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.064+0000 7f08e8dfab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b8a88bf090 0x55b8a88d12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:13.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.064+0000 7f08e8dfab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b8a88bf9b0 0x55b8a88cf740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:13.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.064+0000 7f08e8dfab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b8a88ca860 0x55b8a88cfe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:13.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.064+0000 7f08e8dfab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8a88d0570 con 0x55b8a88bf090 2024-02-02T15:30:13.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.064+0000 7f08e8dfab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8a892b890 con 0x55b8a88ca860 2024-02-02T15:30:13.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.064+0000 7f08e8dfab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8a892ba00 con 0x55b8a88bf9b0 2024-02-02T15:30:13.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.064+0000 7f08e8dfab80 10 client.0 _ll_register_callbacks cb 0x55b8a87ae310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:30:13.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.064+0000 7f08cde00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b8a88bf090 0x55b8a88d12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:13.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.064+0000 7f08cde00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b8a88bf090 0x55b8a88d12e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40710/0 (socket says 172.21.15.1:40710) 2024-02-02T15:30:13.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.064+0000 7f08cde00700 1 -- 172.21.15.1:0/3930929250 learned_addr learned my addr 172.21.15.1:0/3930929250 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:30:13.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.064+0000 7f08cd400700 1 --2- 172.21.15.1:0/3930929250 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b8a88bf9b0 0x55b8a88cf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:13.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.064+0000 7f08ce800700 1 --2- 172.21.15.1:0/3930929250 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b8a88ca860 0x55b8a88cfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:13.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.065+0000 7f08cde00700 1 -- 172.21.15.1:0/3930929250 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b8a88bf9b0 msgr2=0x55b8a88cf740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:13.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.065+0000 7f08cde00700 1 --2- 172.21.15.1:0/3930929250 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b8a88bf9b0 0x55b8a88cf740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:13.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.065+0000 7f08cde00700 1 -- 172.21.15.1:0/3930929250 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b8a88ca860 msgr2=0x55b8a88cfe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:13.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.065+0000 7f08cde00700 1 --2- 172.21.15.1:0/3930929250 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b8a88ca860 0x55b8a88cfe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:13.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.065+0000 7f08cde00700 1 -- 172.21.15.1:0/3930929250 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b8a863ebe0 con 0x55b8a88bf090 2024-02-02T15:30:13.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.065+0000 7f08cde00700 1 --2- 172.21.15.1:0/3930929250 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b8a88bf090 0x55b8a88d12e0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f08c0024000 tx=0x7f08c0099840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb6e7ac236c4d7c4 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:13.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.065+0000 7f08b7e00700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:30:13.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.065+0000 7f08b7e00700 1 -- 172.21.15.1:0/3930929250 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c00056e0 con 0x55b8a88bf090 2024-02-02T15:30:13.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.065+0000 7f08b7e00700 1 -- 172.21.15.1:0/3930929250 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f08c009ded0 con 0x55b8a88bf090 2024-02-02T15:30:13.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.065+0000 7f08e8dfab80 1 -- 172.21.15.1:0/3930929250 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f08b80044f0 con 0x55b8a88bf090 2024-02-02T15:30:13.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.065+0000 7f08b7e00700 1 -- 172.21.15.1:0/3930929250 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08c0066080 con 0x55b8a88bf090 2024-02-02T15:30:13.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.065+0000 7f08e8dfab80 10 client.16491 fetch_fsmap learned FSMap version 31 2024-02-02T15:30:13.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.065+0000 7f08e8dfab80 1 -- 172.21.15.1:0/3930929250 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55b8a8692430 con 0x55b8a88bf090 2024-02-02T15:30:13.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.066+0000 7f08b7e00700 1 -- 172.21.15.1:0/3930929250 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f08c00a8080 con 0x55b8a88bf090 2024-02-02T15:30:13.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.066+0000 7f08b7e00700 1 -- 172.21.15.1:0/3930929250 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 31) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f08c00adc90 con 0x55b8a88bf090 2024-02-02T15:30:13.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.066+0000 7f08e8dfab80 10 client.16491 fetch_fsmap finished waiting for FSMap version 31 2024-02-02T15:30:13.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.066+0000 7f08e8dfab80 10 client.16491 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:30:13.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.066+0000 7f08e8dfab80 20 client.16491 populate_metadata read hostname 'smithi001' 2024-02-02T15:30:13.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.066+0000 7f08e8dfab80 1 --2- 172.21.15.1:0/3930929250 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55b8a892ccf0 0x55b8a892f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:13.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.066+0000 7f08e8dfab80 4 client.16491 mds_command: new command op to 24557 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-02T15:30:13.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.066+0000 7f08e8dfab80 1 -- 172.21.15.1:0/3930929250 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55b8a87cffd0 con 0x55b8a892ccf0 2024-02-02T15:30:13.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.066+0000 7f08cd400700 1 --2- 172.21.15.1:0/3930929250 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55b8a892ccf0 0x55b8a892f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:13.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.067+0000 7f08cd400700 1 --2- 172.21.15.1:0/3930929250 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55b8a892ccf0 0x55b8a892f110 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73a2079e56be0e2c server_cookie=ca7a81fab7f02217 in_seq=0 out_seq=0 2024-02-02T15:30:13.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.067+0000 7f08b7e00700 10 client.16491 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:30:13.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.068+0000 7f08b7e00700 1 -- 172.21.15.1:0/3930929250 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55b8a87cffd0 con 0x55b8a892ccf0 2024-02-02T15:30:13.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.068+0000 7f08b7e00700 10 client.16491 handle_command_reply: tid=0 2024-02-02T15:30:13.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.068+0000 7f08e8dfab80 1 client.16491 shutdown 2024-02-02T15:30:13.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.068+0000 7f08e8dfab80 1 -- 172.21.15.1:0/3930929250 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b8a88bf090 msgr2=0x55b8a88d12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:13.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.068+0000 7f08e8dfab80 1 --2- 172.21.15.1:0/3930929250 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b8a88bf090 0x55b8a88d12e0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f08c0024000 tx=0x7f08c0099840 comp rx=0 tx=0).stop 2024-02-02T15:30:13.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.068+0000 7f08e8dfab80 1 -- 172.21.15.1:0/3930929250 shutdown_connections 2024-02-02T15:30:13.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.068+0000 7f08e8dfab80 1 --2- 172.21.15.1:0/3930929250 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55b8a892ccf0 0x55b8a892f110 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:13.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.068+0000 7f08e8dfab80 1 --2- 172.21.15.1:0/3930929250 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b8a88ca860 0x55b8a88cfe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:13.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.068+0000 7f08e8dfab80 1 --2- 172.21.15.1:0/3930929250 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b8a88bf090 0x55b8a88d12e0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:13.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.068+0000 7f08e8dfab80 1 --2- 172.21.15.1:0/3930929250 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b8a88bf9b0 0x55b8a88cf740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:13.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.068+0000 7f08e8dfab80 1 -- 172.21.15.1:0/3930929250 >> 172.21.15.1:0/3930929250 conn(0x55b8a881ee30 msgr2=0x55b8a88c2860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:13.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.068+0000 7f08b7e00700 0 client.16491 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:30:13.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.068+0000 7f08e8dfab80 1 -- 172.21.15.1:0/3930929250 shutdown_connections 2024-02-02T15:30:13.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.068+0000 7f08e8dfab80 1 -- 172.21.15.1:0/3930929250 wait complete. 2024-02-02T15:30:13.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.069+0000 7f08e8dfab80 20 client.16491 trim_cache size 0 max 16384 2024-02-02T15:30:13.072 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:30:13.072 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2024-02-02T15:30:13.072 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "7f1f4e59-2431-4ca8-95f4-595644a97e35", 2024-02-02T15:30:13.072 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2024-02-02T15:30:13.072 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:30:13.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.069+0000 7f08d5200700 1 -- 172.21.15.1:0/3308693537 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f08b00a0690 msgr2=0x7f08b00a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:13.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.069+0000 7f08d5200700 1 --2- 172.21.15.1:0/3308693537 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f08b00a0690 0x7f08b00a2b50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f08b8013950 tx=0x7f08b8013d10 comp rx=0 tx=0).stop 2024-02-02T15:30:13.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.069+0000 7f08d5200700 1 -- 172.21.15.1:0/3308693537 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08d0100ff0 msgr2=0x7f08d0199120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:13.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.069+0000 7f08d5200700 1 --2- 172.21.15.1:0/3308693537 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08d0100ff0 0x7f08d0199120 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f08c0009810 tx=0x7f08c000bfc0 comp rx=0 tx=0).stop 2024-02-02T15:30:13.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.069+0000 7f08d5200700 1 -- 172.21.15.1:0/3308693537 shutdown_connections 2024-02-02T15:30:13.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.069+0000 7f08d5200700 1 --2- 172.21.15.1:0/3308693537 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f08b00a0690 0x7f08b00a2b50 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:13.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.069+0000 7f08d5200700 1 --2- 172.21.15.1:0/3308693537 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f08d019dac0 0x7f08d019df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:13.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.069+0000 7f08d5200700 1 --2- 172.21.15.1:0/3308693537 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f08d0100ff0 0x7f08d0199120 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:13.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.069+0000 7f08d5200700 1 --2- 172.21.15.1:0/3308693537 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f08d0106110 0x7f08d0199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:13.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.069+0000 7f08d5200700 1 -- 172.21.15.1:0/3308693537 >> 172.21.15.1:0/3308693537 conn(0x7f08d00fc810 msgr2=0x7f08d00fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:13.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.069+0000 7f08d5200700 1 -- 172.21.15.1:0/3308693537 shutdown_connections 2024-02-02T15:30:13.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.069+0000 7f08d5200700 1 -- 172.21.15.1:0/3308693537 wait complete. 2024-02-02T15:30:13.088 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-02T15:30:13.226 INFO:tasks.workunit.client.0.smithi001.stdout: 1 117351 6.09 MB/sec warmup 107 sec latency 1742.404 ms 2024-02-02T15:30:13.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.296+0000 7ff9ef000700 1 -- 172.21.15.1:0/1927150260 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff9e8068010 msgr2=0x7ff9e8068470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:13.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.296+0000 7ff9ef000700 1 --2- 172.21.15.1:0/1927150260 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff9e8068010 0x7ff9e8068470 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff9d0009990 tx=0x7ff9d0009ca0 comp rx=0 tx=0).stop 2024-02-02T15:30:13.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.296+0000 7ff9ef000700 1 -- 172.21.15.1:0/1927150260 shutdown_connections 2024-02-02T15:30:13.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.296+0000 7ff9ef000700 1 --2- 172.21.15.1:0/1927150260 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9e80689b0 0x7ff9e810dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:13.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.296+0000 7ff9ef000700 1 --2- 172.21.15.1:0/1927150260 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff9e8106ed0 0x7ff9e81072b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:13.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.296+0000 7ff9ef000700 1 --2- 172.21.15.1:0/1927150260 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff9e8068010 0x7ff9e8068470 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:13.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.296+0000 7ff9ef000700 1 -- 172.21.15.1:0/1927150260 >> 172.21.15.1:0/1927150260 conn(0x7ff9e80757d0 msgr2=0x7ff9e8075be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:13.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.297+0000 7ff9ef000700 1 -- 172.21.15.1:0/1927150260 shutdown_connections 2024-02-02T15:30:13.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.297+0000 7ff9ef000700 1 -- 172.21.15.1:0/1927150260 wait complete. 2024-02-02T15:30:13.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.297+0000 7ff9ef000700 1 Processor -- start 2024-02-02T15:30:13.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.298+0000 7ff9ef000700 1 -- start start 2024-02-02T15:30:13.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.298+0000 7ff9ef000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff9e8068010 0x7ff9e80fff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:13.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.298+0000 7ff9ef000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9e80689b0 0x7ff9e8100480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:13.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.298+0000 7ff9ef000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff9e8106ed0 0x7ff9e81a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:13.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.298+0000 7ff9ef000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e8100cb0 con 0x7ff9e8068010 2024-02-02T15:30:13.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.298+0000 7ff9ef000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e81a56a0 con 0x7ff9e8106ed0 2024-02-02T15:30:13.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.298+0000 7ff9ef000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e81a57e0 con 0x7ff9e80689b0 2024-02-02T15:30:13.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.298+0000 7ff9e7e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff9e8068010 0x7ff9e80fff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:13.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.298+0000 7ff9e7400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9e80689b0 0x7ff9e8100480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:13.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.298+0000 7ff9e7400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9e80689b0 0x7ff9e8100480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:39736/0 (socket says 172.21.15.1:39736) 2024-02-02T15:30:13.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.298+0000 7ff9e7400700 1 -- 172.21.15.1:0/3895285184 learned_addr learned my addr 172.21.15.1:0/3895285184 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:30:13.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.298+0000 7ff9ed000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff9e8106ed0 0x7ff9e81a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:13.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.299+0000 7ff9e7e00700 1 -- 172.21.15.1:0/3895285184 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9e80689b0 msgr2=0x7ff9e8100480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:13.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.299+0000 7ff9e7e00700 1 --2- 172.21.15.1:0/3895285184 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9e80689b0 0x7ff9e8100480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:13.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.299+0000 7ff9e7e00700 1 -- 172.21.15.1:0/3895285184 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff9e8106ed0 msgr2=0x7ff9e81a5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:13.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.299+0000 7ff9e7e00700 1 --2- 172.21.15.1:0/3895285184 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff9e8106ed0 0x7ff9e81a5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:13.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.299+0000 7ff9e7e00700 1 -- 172.21.15.1:0/3895285184 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9d00095f0 con 0x7ff9e8068010 2024-02-02T15:30:13.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.299+0000 7ff9e7e00700 1 --2- 172.21.15.1:0/3895285184 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff9e8068010 0x7ff9e80fff40 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7ff9d800e9f0 tx=0x7ff9d800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:13.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.299+0000 7ff9e4c00700 1 -- 172.21.15.1:0/3895285184 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d800c9c0 con 0x7ff9e8068010 2024-02-02T15:30:13.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.299+0000 7ff9ef000700 1 -- 172.21.15.1:0/3895285184 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9e81a5a40 con 0x7ff9e8068010 2024-02-02T15:30:13.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.299+0000 7ff9e4c00700 1 -- 172.21.15.1:0/3895285184 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff9d8010430 con 0x7ff9e8068010 2024-02-02T15:30:13.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.299+0000 7ff9ef000700 1 -- 172.21.15.1:0/3895285184 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9e81a5f70 con 0x7ff9e8068010 2024-02-02T15:30:13.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.299+0000 7ff9e4c00700 1 -- 172.21.15.1:0/3895285184 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d8018620 con 0x7ff9e8068010 2024-02-02T15:30:13.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.302+0000 7ff9e4c00700 1 -- 172.21.15.1:0/3895285184 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 27) v1 ==== 135200+0+0 (secure 0 0 0) 0x7ff9d8010860 con 0x7ff9e8068010 2024-02-02T15:30:13.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.304+0000 7ff9e4c00700 1 --2- 172.21.15.1:0/3895285184 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff9c80a0690 0x7ff9c80a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:13.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.304+0000 7ff9e4c00700 1 -- 172.21.15.1:0/3895285184 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7ff9d8014070 con 0x7ff9e8068010 2024-02-02T15:30:13.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.304+0000 7ffa02b8fb80 1 Processor -- start 2024-02-02T15:30:13.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.304+0000 7ff9e7400700 1 --2- 172.21.15.1:0/3895285184 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff9c80a0690 0x7ff9c80a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:13.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.304+0000 7ffa02b8fb80 1 -- start start 2024-02-02T15:30:13.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.305+0000 7ffa02b8fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b753cb010 0x558b753cb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:13.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.305+0000 7ffa02b8fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b753cb930 0x558b753d62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:13.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.305+0000 7ffa02b8fb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558b753d67e0 0x558b753d8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:13.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.305+0000 7ffa02b8fb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b7533f4c0 con 0x558b753d67e0 2024-02-02T15:30:13.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.305+0000 7ffa02b8fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b75323aa0 con 0x558b753cb930 2024-02-02T15:30:13.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.305+0000 7ffa02b8fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b75321460 con 0x558b753cb010 2024-02-02T15:30:13.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.305+0000 7ff9e7e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b753cb010 0x558b753cb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:13.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.305+0000 7ff9e7e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b753cb010 0x558b753cb3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:39752/0 (socket says 172.21.15.1:39752) 2024-02-02T15:30:13.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.305+0000 7ff9e7e00700 1 -- 172.21.15.1:0/1452552025 learned_addr learned my addr 172.21.15.1:0/1452552025 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:30:13.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.305+0000 7ff9ed000700 1 --2- 172.21.15.1:0/1452552025 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558b753d67e0 0x558b753d8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:13.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.305+0000 7ff9e7e00700 1 -- 172.21.15.1:0/1452552025 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b753cb930 msgr2=0x558b753d62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:13.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.305+0000 7ff9e7e00700 1 --2- 172.21.15.1:0/1452552025 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b753cb930 0x558b753d62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:13.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.305+0000 7ff9e7e00700 1 -- 172.21.15.1:0/1452552025 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558b753d67e0 msgr2=0x558b753d8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:13.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.305+0000 7ff9e7e00700 1 --2- 172.21.15.1:0/1452552025 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558b753d67e0 0x558b753d8bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:13.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.305+0000 7ff9e7e00700 1 -- 172.21.15.1:0/1452552025 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558b7519e3b0 con 0x558b753cb010 2024-02-02T15:30:13.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.305+0000 7ff9ed000700 1 --2- 172.21.15.1:0/1452552025 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558b753d67e0 0x558b753d8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:30:13.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.305+0000 7ff9e7400700 1 --2- 172.21.15.1:0/3895285184 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff9c80a0690 0x7ff9c80a2b50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8101730 tx=0x7ff9d000b260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:13.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.305+0000 7ff9e7e00700 1 --2- 172.21.15.1:0/1452552025 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b753cb010 0x558b753cb3f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff9d8099270 tx=0x7ff9d8098f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4fe7924898f78e5 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:13.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.306+0000 7ff9dd600700 1 -- 172.21.15.1:0/1452552025 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d809d110 con 0x558b753cb010 2024-02-02T15:30:13.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.306+0000 7ff9dd600700 1 -- 172.21.15.1:0/1452552025 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff9d80a1350 con 0x558b753cb010 2024-02-02T15:30:13.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.306+0000 7ff9dd600700 1 -- 172.21.15.1:0/1452552025 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d8098a10 con 0x558b753cb010 2024-02-02T15:30:13.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.306+0000 7ffa02b8fb80 1 -- 172.21.15.1:0/1452552025 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b753cb010 msgr2=0x558b753cb3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:13.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.306+0000 7ffa02b8fb80 1 --2- 172.21.15.1:0/1452552025 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b753cb010 0x558b753cb3f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff9d8099270 tx=0x7ff9d8098f20 comp rx=0 tx=0).stop 2024-02-02T15:30:13.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.307+0000 7ffa02b8fb80 1 -- 172.21.15.1:0/1452552025 shutdown_connections 2024-02-02T15:30:13.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.307+0000 7ffa02b8fb80 1 --2- 172.21.15.1:0/1452552025 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558b753d67e0 0x558b753d8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:13.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.307+0000 7ffa02b8fb80 1 --2- 172.21.15.1:0/1452552025 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b753cb010 0x558b753cb3f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:13.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.307+0000 7ffa02b8fb80 1 --2- 172.21.15.1:0/1452552025 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b753cb930 0x558b753d62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:13.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.307+0000 7ffa02b8fb80 1 -- 172.21.15.1:0/1452552025 >> 172.21.15.1:0/1452552025 conn(0x558b7532adb0 msgr2=0x558b753c73b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:13.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.307+0000 7ffa02b8fb80 1 -- 172.21.15.1:0/1452552025 shutdown_connections 2024-02-02T15:30:13.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.307+0000 7ffa02b8fb80 1 -- 172.21.15.1:0/1452552025 wait complete. 2024-02-02T15:30:13.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.307+0000 7ffa02b8fb80 1 -- start start 2024-02-02T15:30:13.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.307+0000 7ffa02b8fb80 1 Processor -- start 2024-02-02T15:30:13.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.308+0000 7ffa02b8fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b753cb010 0x558b753d61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:13.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.308+0000 7ffa02b8fb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558b753cb930 0x558b753d3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:13.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.308+0000 7ffa02b8fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b753d67e0 0x558b753d3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:13.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.308+0000 7ffa02b8fb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b7531cd50 con 0x558b753cb930 2024-02-02T15:30:13.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.308+0000 7ffa02b8fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b752b8d80 con 0x558b753cb010 2024-02-02T15:30:13.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.308+0000 7ffa02b8fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b752b8a20 con 0x558b753d67e0 2024-02-02T15:30:13.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.308+0000 7ffa02b8fb80 10 client.0 _ll_register_callbacks cb 0x558b752ba290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:30:13.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.308+0000 7ff9e7400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558b753cb930 0x558b753d3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:13.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.308+0000 7ff9e7e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b753cb010 0x558b753d61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:13.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.308+0000 7ff9ed000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b753d67e0 0x558b753d3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:13.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.308+0000 7ff9e7400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558b753cb930 0x558b753d3450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40742/0 (socket says 172.21.15.1:40742) 2024-02-02T15:30:13.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.308+0000 7ff9e7e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b753cb010 0x558b753d61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59888/0 (socket says 172.21.15.1:59888) 2024-02-02T15:30:13.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.308+0000 7ff9e7400700 1 -- 172.21.15.1:0/2357852946 learned_addr learned my addr 172.21.15.1:0/2357852946 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:30:13.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.309+0000 7ff9e7e00700 1 -- 172.21.15.1:0/2357852946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b753d67e0 msgr2=0x558b753d3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:13.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.309+0000 7ff9e7e00700 1 --2- 172.21.15.1:0/2357852946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b753d67e0 0x558b753d3a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:13.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.309+0000 7ff9e7e00700 1 -- 172.21.15.1:0/2357852946 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558b753cb930 msgr2=0x558b753d3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:13.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.309+0000 7ff9e7e00700 1 --2- 172.21.15.1:0/2357852946 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558b753cb930 0x558b753d3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:13.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.309+0000 7ff9e7e00700 1 -- 172.21.15.1:0/2357852946 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9e81a5f70 con 0x558b753cb010 2024-02-02T15:30:13.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.309+0000 7ff9e7e00700 1 --2- 172.21.15.1:0/2357852946 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b753cb010 0x558b753d61d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff9d80957e0 tx=0x7ff9d8010c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=868f806ad433f3da server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:13.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.309+0000 7ff9d6000700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:30:13.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.309+0000 7ff9d6000700 1 -- 172.21.15.1:0/2357852946 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d80b72c0 con 0x558b753cb010 2024-02-02T15:30:13.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.309+0000 7ff9d6000700 1 -- 172.21.15.1:0/2357852946 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7ff9d809d110 con 0x558b753cb010 2024-02-02T15:30:13.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.309+0000 7ffa02b8fb80 1 -- 172.21.15.1:0/2357852946 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558b753bf0f0 con 0x558b753cb010 2024-02-02T15:30:13.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.309+0000 7ff9d6000700 1 -- 172.21.15.1:0/2357852946 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff9d8097680 con 0x558b753cb010 2024-02-02T15:30:13.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.309+0000 7ffa02b8fb80 10 client.26561 fetch_fsmap learned FSMap version 31 2024-02-02T15:30:13.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.309+0000 7ffa02b8fb80 1 -- 172.21.15.1:0/2357852946 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x558b7514ab60 con 0x558b753cb010 2024-02-02T15:30:13.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.310+0000 7ff9d6000700 1 -- 172.21.15.1:0/2357852946 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff9d807d080 con 0x558b753cb010 2024-02-02T15:30:13.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.310+0000 7ff9d6000700 1 -- 172.21.15.1:0/2357852946 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 31) v1 ==== 7502+0+0 (secure 0 0 0) 0x7ff9d8082ae0 con 0x558b753cb010 2024-02-02T15:30:13.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.310+0000 7ffa02b8fb80 10 client.26561 fetch_fsmap finished waiting for FSMap version 31 2024-02-02T15:30:13.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.310+0000 7ffa02b8fb80 10 client.26561 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:30:13.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.310+0000 7ffa02b8fb80 20 client.26561 populate_metadata read hostname 'smithi001' 2024-02-02T15:30:13.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.310+0000 7ffa02b8fb80 1 --2- 172.21.15.1:0/2357852946 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x558b753e5ef0 0x558b753e8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:13.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.310+0000 7ffa02b8fb80 4 client.26561 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:30:13.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.310+0000 7ffa02b8fb80 1 -- 172.21.15.1:0/2357852946 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558b75315dd0 con 0x558b753e5ef0 2024-02-02T15:30:13.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.310+0000 7ff9e7400700 1 --2- 172.21.15.1:0/2357852946 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x558b753e5ef0 0x558b753e8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:13.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.311+0000 7ff9e7400700 1 --2- 172.21.15.1:0/2357852946 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x558b753e5ef0 0x558b753e8310 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f960f286b227d42 server_cookie=87d2c9a1a01e440c in_seq=0 out_seq=0 2024-02-02T15:30:13.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.311+0000 7ff9d6000700 10 client.26561 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:30:13.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.311+0000 7ff9d6000700 1 -- 172.21.15.1:0/2357852946 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x558b75315dd0 con 0x558b753e5ef0 2024-02-02T15:30:13.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.311+0000 7ff9d6000700 10 client.26561 handle_command_reply: tid=0 2024-02-02T15:30:13.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.311+0000 7ffa02b8fb80 1 client.26561 shutdown 2024-02-02T15:30:13.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.312+0000 7ffa02b8fb80 1 -- 172.21.15.1:0/2357852946 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b753cb010 msgr2=0x558b753d61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:13.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.312+0000 7ffa02b8fb80 1 --2- 172.21.15.1:0/2357852946 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b753cb010 0x558b753d61d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff9d80957e0 tx=0x7ff9d8010c60 comp rx=0 tx=0).stop 2024-02-02T15:30:13.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.312+0000 7ffa02b8fb80 1 -- 172.21.15.1:0/2357852946 shutdown_connections 2024-02-02T15:30:13.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.312+0000 7ffa02b8fb80 1 --2- 172.21.15.1:0/2357852946 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x558b753e5ef0 0x558b753e8310 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:13.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.312+0000 7ffa02b8fb80 1 --2- 172.21.15.1:0/2357852946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b753d67e0 0x558b753d3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:13.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.312+0000 7ff9d6000700 0 client.26561 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:30:13.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.312+0000 7ffa02b8fb80 1 --2- 172.21.15.1:0/2357852946 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b753cb010 0x558b753d61d0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:13.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.312+0000 7ffa02b8fb80 1 --2- 172.21.15.1:0/2357852946 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558b753cb930 0x558b753d3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:13.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.312+0000 7ffa02b8fb80 1 -- 172.21.15.1:0/2357852946 >> 172.21.15.1:0/2357852946 conn(0x558b7532adb0 msgr2=0x558b753c6fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:13.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.312+0000 7ffa02b8fb80 1 -- 172.21.15.1:0/2357852946 shutdown_connections 2024-02-02T15:30:13.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.312+0000 7ffa02b8fb80 1 -- 172.21.15.1:0/2357852946 wait complete. 2024-02-02T15:30:13.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.312+0000 7ffa02b8fb80 20 client.26561 trim_cache size 0 max 16384 2024-02-02T15:30:13.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.329+0000 7ffa02b8fb80 1 Processor -- start 2024-02-02T15:30:13.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.329+0000 7ffa02b8fb80 1 -- start start 2024-02-02T15:30:13.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.329+0000 7ffa02b8fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b753cb010 0x558b753dd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:13.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.330+0000 7ffa02b8fb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558b753cb930 0x558b753c5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:13.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.330+0000 7ffa02b8fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b753d67e0 0x558b753d5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:13.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.330+0000 7ffa02b8fb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b751a06a0 con 0x558b753cb930 2024-02-02T15:30:13.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.330+0000 7ffa02b8fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b753d48b0 con 0x558b753cb010 2024-02-02T15:30:13.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.330+0000 7ffa02b8fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b753c1500 con 0x558b753d67e0 2024-02-02T15:30:13.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.330+0000 7ff9e7e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b753cb010 0x558b753dd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:13.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.330+0000 7ff9e7e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b753cb010 0x558b753dd2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59896/0 (socket says 172.21.15.1:59896) 2024-02-02T15:30:13.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.330+0000 7ff9e7400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558b753cb930 0x558b753c5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:13.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.330+0000 7ff9e7e00700 1 -- 172.21.15.1:0/3616410397 learned_addr learned my addr 172.21.15.1:0/3616410397 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:30:13.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.330+0000 7ff9ed000700 1 --2- 172.21.15.1:0/3616410397 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b753d67e0 0x558b753d5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:13.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.330+0000 7ff9e7e00700 1 -- 172.21.15.1:0/3616410397 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b753d67e0 msgr2=0x558b753d5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:13.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.330+0000 7ff9e7e00700 1 --2- 172.21.15.1:0/3616410397 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b753d67e0 0x558b753d5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:13.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.330+0000 7ff9e7e00700 1 -- 172.21.15.1:0/3616410397 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558b753cb930 msgr2=0x558b753c5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:13.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.330+0000 7ff9e7e00700 1 --2- 172.21.15.1:0/3616410397 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558b753cb930 0x558b753c5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:13.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.330+0000 7ff9e7e00700 1 -- 172.21.15.1:0/3616410397 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558b753c30d0 con 0x558b753cb010 2024-02-02T15:30:13.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.330+0000 7ff9e7e00700 1 --2- 172.21.15.1:0/3616410397 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b753cb010 0x558b753dd2d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff9d800e9c0 tx=0x7ff9d80c9530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d692ede5363c7a3 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:13.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.331+0000 7ff9de000700 1 -- 172.21.15.1:0/3616410397 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d80b72c0 con 0x558b753cb010 2024-02-02T15:30:13.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.331+0000 7ff9de000700 1 -- 172.21.15.1:0/3616410397 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558b753bf0f0 con 0x558b753cb010 2024-02-02T15:30:13.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.331+0000 7ff9de000700 1 -- 172.21.15.1:0/3616410397 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d8095270 con 0x558b753cb010 2024-02-02T15:30:13.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.331+0000 7ffa02b8fb80 1 -- 172.21.15.1:0/3616410397 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b753cb010 msgr2=0x558b753dd2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:13.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.331+0000 7ffa02b8fb80 1 --2- 172.21.15.1:0/3616410397 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b753cb010 0x558b753dd2d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff9d800e9c0 tx=0x7ff9d80c9530 comp rx=0 tx=0).stop 2024-02-02T15:30:13.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.331+0000 7ffa02b8fb80 1 -- 172.21.15.1:0/3616410397 shutdown_connections 2024-02-02T15:30:13.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.331+0000 7ffa02b8fb80 1 --2- 172.21.15.1:0/3616410397 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b753d67e0 0x558b753d5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:13.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.331+0000 7ffa02b8fb80 1 --2- 172.21.15.1:0/3616410397 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b753cb010 0x558b753dd2d0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:13.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.331+0000 7ffa02b8fb80 1 --2- 172.21.15.1:0/3616410397 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558b753cb930 0x558b753c5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:13.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.331+0000 7ffa02b8fb80 1 -- 172.21.15.1:0/3616410397 >> 172.21.15.1:0/3616410397 conn(0x558b7532adb0 msgr2=0x558b753e18a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:13.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.331+0000 7ffa02b8fb80 1 -- 172.21.15.1:0/3616410397 shutdown_connections 2024-02-02T15:30:13.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.332+0000 7ffa02b8fb80 1 -- 172.21.15.1:0/3616410397 wait complete. 2024-02-02T15:30:13.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.332+0000 7ffa02b8fb80 1 -- start start 2024-02-02T15:30:13.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.332+0000 7ffa02b8fb80 1 Processor -- start 2024-02-02T15:30:13.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.333+0000 7ffa02b8fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b753cb010 0x558b753dadd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:13.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.333+0000 7ffa02b8fb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558b753cb930 0x558b753db310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:13.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.333+0000 7ffa02b8fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b753d67e0 0x558b75436a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:13.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.333+0000 7ffa02b8fb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b753dba90 con 0x558b753cb930 2024-02-02T15:30:13.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.333+0000 7ffa02b8fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b75436f80 con 0x558b753d67e0 2024-02-02T15:30:13.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.333+0000 7ffa02b8fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b75437140 con 0x558b753cb010 2024-02-02T15:30:13.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.333+0000 7ffa02b8fb80 10 client.0 _ll_register_callbacks cb 0x558b752ba290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:30:13.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.333+0000 7ff9e7e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b753cb010 0x558b753dadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:13.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.333+0000 7ff9e7e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b753cb010 0x558b753dadd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:39766/0 (socket says 172.21.15.1:39766) 2024-02-02T15:30:13.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.333+0000 7ff9e7e00700 1 -- 172.21.15.1:0/1891016544 learned_addr learned my addr 172.21.15.1:0/1891016544 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:30:13.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.333+0000 7ff9ed000700 1 --2- 172.21.15.1:0/1891016544 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b753d67e0 0x558b75436a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:13.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.333+0000 7ff9e7400700 1 --2- 172.21.15.1:0/1891016544 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558b753cb930 0x558b753db310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:13.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.334+0000 7ff9e7e00700 1 -- 172.21.15.1:0/1891016544 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b753d67e0 msgr2=0x558b75436a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:13.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.334+0000 7ff9e7e00700 1 --2- 172.21.15.1:0/1891016544 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b753d67e0 0x558b75436a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:13.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.334+0000 7ff9e7e00700 1 -- 172.21.15.1:0/1891016544 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558b753cb930 msgr2=0x558b753db310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:13.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.334+0000 7ff9e7e00700 1 --2- 172.21.15.1:0/1891016544 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558b753cb930 0x558b753db310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:13.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.334+0000 7ff9e7e00700 1 -- 172.21.15.1:0/1891016544 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558b7514ab60 con 0x558b753cb010 2024-02-02T15:30:13.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.334+0000 7ff9e7e00700 1 --2- 172.21.15.1:0/1891016544 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b753cb010 0x558b753dadd0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff9d8099270 tx=0x7ff9d80c95f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8fc119a75df47feb server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:13.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.334+0000 7ff9d6000700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:30:13.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.334+0000 7ff9d6000700 1 -- 172.21.15.1:0/1891016544 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d8097720 con 0x558b753cb010 2024-02-02T15:30:13.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.334+0000 7ff9d6000700 1 -- 172.21.15.1:0/1891016544 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7ff9d809d110 con 0x558b753cb010 2024-02-02T15:30:13.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.334+0000 7ffa02b8fb80 1 -- 172.21.15.1:0/1891016544 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff9d80a1350 con 0x558b753cb010 2024-02-02T15:30:13.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.334+0000 7ff9d6000700 1 -- 172.21.15.1:0/1891016544 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff9d80b72c0 con 0x558b753cb010 2024-02-02T15:30:13.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.334+0000 7ffa02b8fb80 10 client.26557 fetch_fsmap learned FSMap version 31 2024-02-02T15:30:13.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.334+0000 7ffa02b8fb80 1 -- 172.21.15.1:0/1891016544 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x558b752e2ea0 con 0x558b753cb010 2024-02-02T15:30:13.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.335+0000 7ff9d6000700 1 -- 172.21.15.1:0/1891016544 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff9d80a1350 con 0x558b753cb010 2024-02-02T15:30:13.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.335+0000 7ff9d6000700 1 -- 172.21.15.1:0/1891016544 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 31) v1 ==== 7502+0+0 (secure 0 0 0) 0x7ff9d8098320 con 0x558b753cb010 2024-02-02T15:30:13.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.335+0000 7ffa02b8fb80 10 client.26557 fetch_fsmap finished waiting for FSMap version 31 2024-02-02T15:30:13.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.335+0000 7ffa02b8fb80 10 client.26557 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:30:13.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.335+0000 7ffa02b8fb80 20 client.26557 populate_metadata read hostname 'smithi001' 2024-02-02T15:30:13.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.335+0000 7ffa02b8fb80 1 --2- 172.21.15.1:0/1891016544 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x558b754383b0 0x558b7543a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:13.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.335+0000 7ffa02b8fb80 4 client.26557 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2024-02-02T15:30:13.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.335+0000 7ffa02b8fb80 1 -- 172.21.15.1:0/1891016544 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558b752dbf50 con 0x558b754383b0 2024-02-02T15:30:13.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.335+0000 7ff9e7400700 1 --2- 172.21.15.1:0/1891016544 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x558b754383b0 0x558b7543a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:13.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.336+0000 7ff9e7400700 1 --2- 172.21.15.1:0/1891016544 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x558b754383b0 0x558b7543a7a0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a945688f729462cd server_cookie=6cbbc6dc050bc75f in_seq=0 out_seq=0 2024-02-02T15:30:13.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.336+0000 7ff9d6000700 10 client.26557 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:30:13.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.336+0000 7ff9d6000700 1 -- 172.21.15.1:0/1891016544 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x558b752dbf50 con 0x558b754383b0 2024-02-02T15:30:13.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.336+0000 7ff9d6000700 10 client.26557 handle_command_reply: tid=0 2024-02-02T15:30:13.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.336+0000 7ffa02b8fb80 1 client.26557 shutdown 2024-02-02T15:30:13.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.336+0000 7ffa02b8fb80 1 -- 172.21.15.1:0/1891016544 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b753cb010 msgr2=0x558b753dadd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:13.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.336+0000 7ffa02b8fb80 1 --2- 172.21.15.1:0/1891016544 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b753cb010 0x558b753dadd0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff9d8099270 tx=0x7ff9d80c95f0 comp rx=0 tx=0).stop 2024-02-02T15:30:13.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.336+0000 7ffa02b8fb80 1 -- 172.21.15.1:0/1891016544 shutdown_connections 2024-02-02T15:30:13.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.336+0000 7ffa02b8fb80 1 --2- 172.21.15.1:0/1891016544 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x558b754383b0 0x558b7543a7a0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:13.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.336+0000 7ffa02b8fb80 1 --2- 172.21.15.1:0/1891016544 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b753d67e0 0x558b75436a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:13.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.336+0000 7ff9d6000700 0 client.26557 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:30:13.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.336+0000 7ffa02b8fb80 1 --2- 172.21.15.1:0/1891016544 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b753cb010 0x558b753dadd0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:13.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.336+0000 7ffa02b8fb80 1 --2- 172.21.15.1:0/1891016544 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558b753cb930 0x558b753db310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:13.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.336+0000 7ffa02b8fb80 1 -- 172.21.15.1:0/1891016544 >> 172.21.15.1:0/1891016544 conn(0x558b7532adb0 msgr2=0x558b753d2cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:13.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.336+0000 7ffa02b8fb80 1 -- 172.21.15.1:0/1891016544 shutdown_connections 2024-02-02T15:30:13.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.336+0000 7ffa02b8fb80 1 -- 172.21.15.1:0/1891016544 wait complete. 2024-02-02T15:30:13.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.336+0000 7ffa02b8fb80 20 client.26557 trim_cache size 0 max 16384 2024-02-02T15:30:13.341 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:30:13.341 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-02T15:30:13.341 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2024-02-02T15:30:13.341 INFO:teuthology.orchestra.run.smithi001.stdout: "7f1f4e59-2431-4ca8-95f4-595644a97e35": { 2024-02-02T15:30:13.341 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2024-02-02T15:30:13.341 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "7f1f4e59-2431-4ca8-95f4-595644a97e35", 2024-02-02T15:30:13.341 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,force" 2024-02-02T15:30:13.342 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:30:13.342 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:30:13.342 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:30:13.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.337+0000 7ff9ef000700 1 -- 172.21.15.1:0/3895285184 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff9c80a0690 msgr2=0x7ff9c80a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:13.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.337+0000 7ff9ef000700 1 --2- 172.21.15.1:0/3895285184 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff9c80a0690 0x7ff9c80a2b50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8101730 tx=0x7ff9d000b260 comp rx=0 tx=0).stop 2024-02-02T15:30:13.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.337+0000 7ff9ef000700 1 -- 172.21.15.1:0/3895285184 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff9e8068010 msgr2=0x7ff9e80fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:13.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.337+0000 7ff9ef000700 1 --2- 172.21.15.1:0/3895285184 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff9e8068010 0x7ff9e80fff40 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7ff9d800e9f0 tx=0x7ff9d800ed00 comp rx=0 tx=0).stop 2024-02-02T15:30:13.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.337+0000 7ff9ef000700 1 -- 172.21.15.1:0/3895285184 shutdown_connections 2024-02-02T15:30:13.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.337+0000 7ff9ef000700 1 --2- 172.21.15.1:0/3895285184 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff9c80a0690 0x7ff9c80a2b50 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:13.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.337+0000 7ff9ef000700 1 --2- 172.21.15.1:0/3895285184 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff9e8106ed0 0x7ff9e81a5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:13.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.337+0000 7ff9ef000700 1 --2- 172.21.15.1:0/3895285184 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff9e8068010 0x7ff9e80fff40 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:13.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.337+0000 7ff9ef000700 1 --2- 172.21.15.1:0/3895285184 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff9e80689b0 0x7ff9e8100480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:13.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.337+0000 7ff9ef000700 1 -- 172.21.15.1:0/3895285184 >> 172.21.15.1:0/3895285184 conn(0x7ff9e80757d0 msgr2=0x7ff9e8104b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:13.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.337+0000 7ff9ef000700 1 -- 172.21.15.1:0/3895285184 shutdown_connections 2024-02-02T15:30:13.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:13.337+0000 7ff9ef000700 1 -- 172.21.15.1:0/3895285184 wait complete. 2024-02-02T15:30:13.356 INFO:tasks.cephfs.filesystem:scrub status for tag:7f1f4e59-2431-4ca8-95f4-595644a97e35 - {'path': '/', 'tag': '7f1f4e59-2431-4ca8-95f4-595644a97e35', 'options': 'recursive,force'} 2024-02-02T15:30:13.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:13 smithi001 ceph-mon[35470]: pgmap v99: 129 pgs: 129 active+clean; 72 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 1.1 MiB/s wr, 128 op/s 2024-02-02T15:30:13.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:13 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:30:13.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:13 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:30:13.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:13 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:30:13.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:13 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:30:13.727 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:13 smithi071 ceph-mon[44707]: pgmap v99: 129 pgs: 129 active+clean; 72 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 1.1 MiB/s wr, 128 op/s 2024-02-02T15:30:13.727 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:13 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:30:13.727 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:13 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:30:13.727 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:13 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:30:13.727 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:13 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:30:13.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:13 smithi142 ceph-mon[45728]: pgmap v99: 129 pgs: 129 active+clean; 72 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 757 KiB/s rd, 1.1 MiB/s wr, 128 op/s 2024-02-02T15:30:13.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:13 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:30:13.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:13 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:30:13.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:13 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:30:13.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:13 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:30:14.226 INFO:tasks.workunit.client.0.smithi001.stdout: 1 118758 6.09 MB/sec warmup 108 sec latency 20.326 ms 2024-02-02T15:30:14.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:14 smithi071 ceph-mon[44707]: scrub queued for path: / 2024-02-02T15:30:14.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:14 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:30:14.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:14 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:30:14.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:14 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:30:14.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:14 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:30:14.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:14 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:30:14.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:14 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:30:14.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:14 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:30:14.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:14 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:30:14.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:14 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:30:14.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:14 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:30:14.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:14 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:30:14.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:14 smithi142 ceph-mon[45728]: scrub queued for path: / 2024-02-02T15:30:14.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:14 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:30:14.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:14 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:30:14.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:14 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:30:14.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:14 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:30:14.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:14 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:30:14.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:14 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:30:14.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:14 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:30:14.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:14 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:30:14.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:14 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:30:14.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:14 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:30:14.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:14 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:30:14.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:14 smithi001 ceph-mon[35470]: scrub queued for path: / 2024-02-02T15:30:14.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:14 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:30:14.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:14 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:30:14.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:14 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:30:14.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:14 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:30:14.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:14 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:30:14.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:14 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:30:14.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:14 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:30:14.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:14 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:30:14.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:14 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:30:14.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:14 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:30:14.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:14 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:30:15.226 INFO:tasks.workunit.client.0.smithi001.stdout: 1 120901 6.15 MB/sec warmup 109 sec latency 18.363 ms 2024-02-02T15:30:15.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:15 smithi071 ceph-mon[44707]: pgmap v100: 129 pgs: 129 active+clean; 72 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 606 KiB/s wr, 66 op/s 2024-02-02T15:30:15.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:15 smithi142 ceph-mon[45728]: pgmap v100: 129 pgs: 129 active+clean; 72 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 606 KiB/s wr, 66 op/s 2024-02-02T15:30:15.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:15 smithi001 ceph-mon[35470]: pgmap v100: 129 pgs: 129 active+clean; 72 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 606 KiB/s wr, 66 op/s 2024-02-02T15:30:16.226 INFO:tasks.workunit.client.0.smithi001.stdout: 1 122328 6.15 MB/sec warmup 110 sec latency 19.788 ms 2024-02-02T15:30:17.226 INFO:tasks.workunit.client.0.smithi001.stdout: 1 124488 6.22 MB/sec warmup 111 sec latency 12.652 ms 2024-02-02T15:30:17.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:17 smithi071 ceph-mon[44707]: pgmap v101: 129 pgs: 129 active+clean; 75 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 1.2 MiB/s wr, 139 op/s 2024-02-02T15:30:17.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:17 smithi142 ceph-mon[45728]: pgmap v101: 129 pgs: 129 active+clean; 75 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 1.2 MiB/s wr, 139 op/s 2024-02-02T15:30:17.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:17 smithi001 ceph-mon[35470]: pgmap v101: 129 pgs: 129 active+clean; 75 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 1.2 MiB/s wr, 139 op/s 2024-02-02T15:30:18.226 INFO:tasks.workunit.client.0.smithi001.stdout: 1 126484 6.22 MB/sec warmup 112 sec latency 13.733 ms 2024-02-02T15:30:18.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:18 smithi071 ceph-mon[44707]: pgmap v102: 129 pgs: 129 active+clean; 75 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 902 KiB/s rd, 1.2 MiB/s wr, 128 op/s 2024-02-02T15:30:18.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:18 smithi142 ceph-mon[45728]: pgmap v102: 129 pgs: 129 active+clean; 75 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 902 KiB/s rd, 1.2 MiB/s wr, 128 op/s 2024-02-02T15:30:18.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:18 smithi001 ceph-mon[35470]: pgmap v102: 129 pgs: 129 active+clean; 75 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 902 KiB/s rd, 1.2 MiB/s wr, 128 op/s 2024-02-02T15:30:19.226 INFO:tasks.workunit.client.0.smithi001.stdout: 1 128094 6.28 MB/sec warmup 113 sec latency 40.017 ms 2024-02-02T15:30:20.226 INFO:tasks.workunit.client.0.smithi001.stdout: 1 130066 6.28 MB/sec warmup 114 sec latency 11.864 ms 2024-02-02T15:30:21.227 INFO:tasks.workunit.client.0.smithi001.stdout: 1 131672 6.33 MB/sec warmup 115 sec latency 11.667 ms 2024-02-02T15:30:21.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:21 smithi071 ceph-mon[44707]: pgmap v103: 129 pgs: 129 active+clean; 75 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 902 KiB/s rd, 1.2 MiB/s wr, 128 op/s 2024-02-02T15:30:21.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:21 smithi142 ceph-mon[45728]: pgmap v103: 129 pgs: 129 active+clean; 75 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 902 KiB/s rd, 1.2 MiB/s wr, 128 op/s 2024-02-02T15:30:21.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:21 smithi001 ceph-mon[35470]: pgmap v103: 129 pgs: 129 active+clean; 75 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 902 KiB/s rd, 1.2 MiB/s wr, 128 op/s 2024-02-02T15:30:22.226 INFO:tasks.workunit.client.0.smithi001.stdout: 1 133667 6.34 MB/sec warmup 116 sec latency 13.047 ms 2024-02-02T15:30:22.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:22 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:30:22.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:22 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:30:22.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:22 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:30:22.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:22 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:30:22.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:22 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:30:22.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:22 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:30:22.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:22 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:30:22.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:22 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:30:22.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:22 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:30:22.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:22 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:30:22.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:22 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:30:22.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:22 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:30:22.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:22 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:30:22.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:22 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:30:22.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:22 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:30:23.226 INFO:tasks.workunit.client.0.smithi001.stdout: 1 135210 6.39 MB/sec warmup 117 sec latency 12.498 ms 2024-02-02T15:30:23.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:23 smithi071 ceph-mon[44707]: pgmap v104: 129 pgs: 129 active+clean; 82 MiB data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 220 op/s 2024-02-02T15:30:23.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:23 smithi142 ceph-mon[45728]: pgmap v104: 129 pgs: 129 active+clean; 82 MiB data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 220 op/s 2024-02-02T15:30:23.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:23 smithi001 ceph-mon[35470]: pgmap v104: 129 pgs: 129 active+clean; 82 MiB data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 220 op/s 2024-02-02T15:30:24.226 INFO:tasks.workunit.client.0.smithi001.stdout: 1 137228 6.40 MB/sec warmup 118 sec latency 10.424 ms 2024-02-02T15:30:25.227 INFO:tasks.workunit.client.0.smithi001.stdout: 1 138780 6.44 MB/sec warmup 119 sec latency 13.808 ms 2024-02-02T15:30:25.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:25 smithi071 ceph-mon[44707]: pgmap v105: 129 pgs: 129 active+clean; 82 MiB data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 888 KiB/s rd, 1.3 MiB/s wr, 165 op/s 2024-02-02T15:30:25.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:25 smithi142 ceph-mon[45728]: pgmap v105: 129 pgs: 129 active+clean; 82 MiB data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 888 KiB/s rd, 1.3 MiB/s wr, 165 op/s 2024-02-02T15:30:25.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:25 smithi001 ceph-mon[35470]: pgmap v105: 129 pgs: 129 active+clean; 82 MiB data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 888 KiB/s rd, 1.3 MiB/s wr, 165 op/s 2024-02-02T15:30:27.227 INFO:tasks.workunit.client.0.smithi001.stdout: 1 142347 11.96 MB/sec execute 1 sec latency 16.011 ms 2024-02-02T15:30:27.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:27 smithi071 ceph-mon[44707]: pgmap v106: 129 pgs: 129 active+clean; 89 MiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 273 op/s 2024-02-02T15:30:27.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:27 smithi142 ceph-mon[45728]: pgmap v106: 129 pgs: 129 active+clean; 89 MiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 273 op/s 2024-02-02T15:30:27.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:27 smithi001 ceph-mon[35470]: pgmap v106: 129 pgs: 129 active+clean; 89 MiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.2 MiB/s wr, 273 op/s 2024-02-02T15:30:28.227 INFO:tasks.workunit.client.0.smithi001.stdout: 1 144365 9.56 MB/sec execute 2 sec latency 12.828 ms 2024-02-02T15:30:29.227 INFO:tasks.workunit.client.0.smithi001.stdout: 1 145904 10.36 MB/sec execute 3 sec latency 11.701 ms 2024-02-02T15:30:29.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:29 smithi071 ceph-mon[44707]: pgmap v107: 129 pgs: 129 active+clean; 89 MiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 1015 KiB/s rd, 1.6 MiB/s wr, 199 op/s 2024-02-02T15:30:29.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:29 smithi142 ceph-mon[45728]: pgmap v107: 129 pgs: 129 active+clean; 89 MiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 1015 KiB/s rd, 1.6 MiB/s wr, 199 op/s 2024-02-02T15:30:29.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:29 smithi001 ceph-mon[35470]: pgmap v107: 129 pgs: 129 active+clean; 89 MiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 1015 KiB/s rd, 1.6 MiB/s wr, 199 op/s 2024-02-02T15:30:30.227 INFO:tasks.workunit.client.0.smithi001.stdout: 1 147901 9.56 MB/sec execute 4 sec latency 12.176 ms 2024-02-02T15:30:31.227 INFO:tasks.workunit.client.0.smithi001.stdout: 1 149431 10.04 MB/sec execute 5 sec latency 11.570 ms 2024-02-02T15:30:31.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:31 smithi071 ceph-mon[44707]: pgmap v108: 129 pgs: 129 active+clean; 89 MiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 1015 KiB/s rd, 1.6 MiB/s wr, 199 op/s 2024-02-02T15:30:31.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:31 smithi142 ceph-mon[45728]: pgmap v108: 129 pgs: 129 active+clean; 89 MiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 1015 KiB/s rd, 1.6 MiB/s wr, 199 op/s 2024-02-02T15:30:31.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:31 smithi001 ceph-mon[35470]: pgmap v108: 129 pgs: 129 active+clean; 89 MiB data, 768 MiB used, 1.0 TiB / 1.0 TiB avail; 1015 KiB/s rd, 1.6 MiB/s wr, 199 op/s 2024-02-02T15:30:32.227 INFO:tasks.workunit.client.0.smithi001.stdout: 1 151468 9.64 MB/sec execute 6 sec latency 12.532 ms 2024-02-02T15:30:33.227 INFO:tasks.workunit.client.0.smithi001.stdout: 1 152990 9.90 MB/sec execute 7 sec latency 24.034 ms 2024-02-02T15:30:33.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:33 smithi071 ceph-mon[44707]: pgmap v109: 129 pgs: 129 active+clean; 94 MiB data, 804 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 295 op/s 2024-02-02T15:30:33.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:33 smithi142 ceph-mon[45728]: pgmap v109: 129 pgs: 129 active+clean; 94 MiB data, 804 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 295 op/s 2024-02-02T15:30:33.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:33 smithi001 ceph-mon[35470]: pgmap v109: 129 pgs: 129 active+clean; 94 MiB data, 804 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.2 MiB/s wr, 295 op/s 2024-02-02T15:30:34.227 INFO:tasks.workunit.client.0.smithi001.stdout: 1 155103 9.66 MB/sec execute 8 sec latency 12.584 ms 2024-02-02T15:30:35.227 INFO:tasks.workunit.client.0.smithi001.stdout: 1 156600 9.83 MB/sec execute 9 sec latency 20.966 ms 2024-02-02T15:30:35.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:35 smithi071 ceph-mon[44707]: pgmap v110: 129 pgs: 129 active+clean; 94 MiB data, 804 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 203 op/s 2024-02-02T15:30:35.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:35 smithi142 ceph-mon[45728]: pgmap v110: 129 pgs: 129 active+clean; 94 MiB data, 804 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 203 op/s 2024-02-02T15:30:35.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:35 smithi001 ceph-mon[35470]: pgmap v110: 129 pgs: 129 active+clean; 94 MiB data, 804 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 203 op/s 2024-02-02T15:30:36.227 INFO:tasks.workunit.client.0.smithi001.stdout: 1 158675 9.64 MB/sec execute 10 sec latency 22.493 ms 2024-02-02T15:30:37.227 INFO:tasks.workunit.client.0.smithi001.stdout: 1 160195 9.78 MB/sec execute 11 sec latency 10.031 ms 2024-02-02T15:30:37.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:37 smithi071 ceph-mon[44707]: pgmap v111: 129 pgs: 129 active+clean; 99 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 303 op/s 2024-02-02T15:30:37.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:37 smithi142 ceph-mon[45728]: pgmap v111: 129 pgs: 129 active+clean; 99 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 303 op/s 2024-02-02T15:30:37.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:37 smithi001 ceph-mon[35470]: pgmap v111: 129 pgs: 129 active+clean; 99 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.2 MiB/s wr, 303 op/s 2024-02-02T15:30:38.227 INFO:tasks.workunit.client.0.smithi001.stdout: 1 162465 9.89 MB/sec execute 12 sec latency 14.878 ms 2024-02-02T15:30:39.227 INFO:tasks.workunit.client.0.smithi001.stdout: 1 163762 9.75 MB/sec execute 13 sec latency 26.254 ms 2024-02-02T15:30:39.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:39 smithi071 ceph-mon[44707]: pgmap v112: 129 pgs: 129 active+clean; 99 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 195 op/s 2024-02-02T15:30:39.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:39 smithi142 ceph-mon[45728]: pgmap v112: 129 pgs: 129 active+clean; 99 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 195 op/s 2024-02-02T15:30:39.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:39 smithi001 ceph-mon[35470]: pgmap v112: 129 pgs: 129 active+clean; 99 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 195 op/s 2024-02-02T15:30:40.228 INFO:tasks.workunit.client.0.smithi001.stdout: 1 166053 9.85 MB/sec execute 14 sec latency 11.159 ms 2024-02-02T15:30:41.228 INFO:tasks.workunit.client.0.smithi001.stdout: 1 167333 9.72 MB/sec execute 15 sec latency 21.153 ms 2024-02-02T15:30:41.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:41 smithi071 ceph-mon[44707]: pgmap v113: 129 pgs: 129 active+clean; 99 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 195 op/s 2024-02-02T15:30:41.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:41 smithi142 ceph-mon[45728]: pgmap v113: 129 pgs: 129 active+clean; 99 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 195 op/s 2024-02-02T15:30:41.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:41 smithi001 ceph-mon[35470]: pgmap v113: 129 pgs: 129 active+clean; 99 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 195 op/s 2024-02-02T15:30:42.228 INFO:tasks.workunit.client.0.smithi001.stdout: 1 169625 9.85 MB/sec execute 16 sec latency 16.733 ms 2024-02-02T15:30:43.228 INFO:tasks.workunit.client.0.smithi001.stdout: 1 170868 9.70 MB/sec execute 17 sec latency 10.790 ms 2024-02-02T15:30:43.356 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-02T15:30:43.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:43 smithi071 ceph-mon[44707]: pgmap v114: 129 pgs: 129 active+clean; 105 MiB data, 912 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 288 op/s 2024-02-02T15:30:43.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.637+0000 7f40dd400700 1 -- 172.21.15.1:0/1354108124 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40d81000e0 msgr2=0x7f40d81004c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:43.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.637+0000 7f40dd400700 1 --2- 172.21.15.1:0/1354108124 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40d81000e0 0x7f40d81004c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f40c80098f0 tx=0x7f40c8009c00 comp rx=0 tx=0).stop 2024-02-02T15:30:43.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.637+0000 7f40dd400700 1 -- 172.21.15.1:0/1354108124 shutdown_connections 2024-02-02T15:30:43.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.637+0000 7f40dd400700 1 --2- 172.21.15.1:0/1354108124 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40d8105020 0x7f40d810cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:43.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.637+0000 7f40dd400700 1 --2- 172.21.15.1:0/1354108124 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40d81000e0 0x7f40d81004c0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:43.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.637+0000 7f40dd400700 1 --2- 172.21.15.1:0/1354108124 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40d8100a90 0x7f40d81048f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:43.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.637+0000 7f40dd400700 1 -- 172.21.15.1:0/1354108124 >> 172.21.15.1:0/1354108124 conn(0x7f40d80757d0 msgr2=0x7f40d8075be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:43.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.637+0000 7f40dd400700 1 -- 172.21.15.1:0/1354108124 shutdown_connections 2024-02-02T15:30:43.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.638+0000 7f40dd400700 1 -- 172.21.15.1:0/1354108124 wait complete. 2024-02-02T15:30:43.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.638+0000 7f40dd400700 1 Processor -- start 2024-02-02T15:30:43.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.639+0000 7f40dd400700 1 -- start start 2024-02-02T15:30:43.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.639+0000 7f40dd400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40d81000e0 0x7f40d810d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:43.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.639+0000 7f40dd400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40d8105020 0x7f40d810d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:43.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.639+0000 7f40dd400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40d810df30 0x7f40d81a50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:43.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.639+0000 7f40dd400700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d81a5740 con 0x7f40d810df30 2024-02-02T15:30:43.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.639+0000 7f40dd400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d81a58b0 con 0x7f40d8105020 2024-02-02T15:30:43.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.639+0000 7f40dd400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d81a5a70 con 0x7f40d81000e0 2024-02-02T15:30:43.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.639+0000 7f40d5e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40d8105020 0x7f40d810d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:43.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.639+0000 7f40d6800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40d81000e0 0x7f40d810d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:43.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.639+0000 7f40d6800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40d81000e0 0x7f40d810d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:40848/0 (socket says 172.21.15.1:40848) 2024-02-02T15:30:43.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.639+0000 7f40d5e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40d8105020 0x7f40d810d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:38490/0 (socket says 172.21.15.1:38490) 2024-02-02T15:30:43.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.639+0000 7f40d6800700 1 -- 172.21.15.1:0/3267362239 learned_addr learned my addr 172.21.15.1:0/3267362239 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:30:43.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.639+0000 7f40d7200700 1 --2- 172.21.15.1:0/3267362239 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40d810df30 0x7f40d81a50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:43.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.640+0000 7f40d6800700 1 -- 172.21.15.1:0/3267362239 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40d8105020 msgr2=0x7f40d810d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:43.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.640+0000 7f40d6800700 1 --2- 172.21.15.1:0/3267362239 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40d8105020 0x7f40d810d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:43.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.640+0000 7f40d6800700 1 -- 172.21.15.1:0/3267362239 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40d810df30 msgr2=0x7f40d81a50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:43.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.640+0000 7f40d6800700 1 --2- 172.21.15.1:0/3267362239 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40d810df30 0x7f40d81a50b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:43.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.640+0000 7f40d6800700 1 -- 172.21.15.1:0/3267362239 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40c8009520 con 0x7f40d81000e0 2024-02-02T15:30:43.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.640+0000 7f40d5e00700 1 --2- 172.21.15.1:0/3267362239 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40d8105020 0x7f40d810d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:30:43.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.640+0000 7f40d6800700 1 --2- 172.21.15.1:0/3267362239 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40d81000e0 0x7f40d810d4b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f40c8009810 tx=0x7f40c800fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:43.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.640+0000 7f40cf400700 1 -- 172.21.15.1:0/3267362239 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c801c070 con 0x7f40d81000e0 2024-02-02T15:30:43.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.640+0000 7f40dd400700 1 -- 172.21.15.1:0/3267362239 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40d81a5cf0 con 0x7f40d81000e0 2024-02-02T15:30:43.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.640+0000 7f40cf400700 1 -- 172.21.15.1:0/3267362239 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f40c8005040 con 0x7f40d81000e0 2024-02-02T15:30:43.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.640+0000 7f40cf400700 1 -- 172.21.15.1:0/3267362239 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c8004930 con 0x7f40d81000e0 2024-02-02T15:30:43.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.641+0000 7f40dd400700 1 -- 172.21.15.1:0/3267362239 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40d81a6110 con 0x7f40d81000e0 2024-02-02T15:30:43.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.643+0000 7f40cf400700 1 -- 172.21.15.1:0/3267362239 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 27) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f40c8020070 con 0x7f40d81000e0 2024-02-02T15:30:43.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.645+0000 7f40cf400700 1 --2- 172.21.15.1:0/3267362239 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f40b80a0690 0x7f40b80a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:43.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.645+0000 7f40cf400700 1 -- 172.21.15.1:0/3267362239 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f40c80cce80 con 0x7f40d81000e0 2024-02-02T15:30:43.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.645+0000 7f40f0f22b80 1 Processor -- start 2024-02-02T15:30:43.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.645+0000 7f40d5e00700 1 --2- 172.21.15.1:0/3267362239 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f40b80a0690 0x7f40b80a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:43.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.645+0000 7f40f0f22b80 1 -- start start 2024-02-02T15:30:43.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.646+0000 7f40f0f22b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561035737010 0x5610357373f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:43.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.646+0000 7f40f0f22b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561035737930 0x5610357422a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:43.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.646+0000 7f40f0f22b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5610357427e0 0x561035744bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:43.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.646+0000 7f40f0f22b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610356ab4c0 con 0x561035737010 2024-02-02T15:30:43.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.646+0000 7f40f0f22b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56103568faa0 con 0x5610357427e0 2024-02-02T15:30:43.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.646+0000 7f40f0f22b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56103568d460 con 0x561035737930 2024-02-02T15:30:43.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.646+0000 7f40d6800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561035737010 0x5610357373f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:43.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.646+0000 7f40d6800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561035737010 0x5610357373f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44938/0 (socket says 172.21.15.1:44938) 2024-02-02T15:30:43.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.646+0000 7f40d6800700 1 -- 172.21.15.1:0/2580013304 learned_addr learned my addr 172.21.15.1:0/2580013304 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:30:43.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.646+0000 7f40d7200700 1 --2- 172.21.15.1:0/2580013304 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5610357427e0 0x561035744bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:43.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.646+0000 7f40d5e00700 1 --2- 172.21.15.1:0/3267362239 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f40b80a0690 0x7f40b80a2b50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f40d810eca0 tx=0x7f40c0009310 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:43.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.646+0000 7f40d5e00700 1 --2- 172.21.15.1:0/2580013304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561035737930 0x5610357422a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:43.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.646+0000 7f40d7200700 1 -- 172.21.15.1:0/2580013304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561035737930 msgr2=0x5610357422a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:43.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.646+0000 7f40d7200700 1 --2- 172.21.15.1:0/2580013304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561035737930 0x5610357422a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:43.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.646+0000 7f40d7200700 1 -- 172.21.15.1:0/2580013304 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561035737010 msgr2=0x5610357373f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:43.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.646+0000 7f40d7200700 1 --2- 172.21.15.1:0/2580013304 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561035737010 0x5610357373f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:43.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.646+0000 7f40d7200700 1 -- 172.21.15.1:0/2580013304 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56103550a3b0 con 0x5610357427e0 2024-02-02T15:30:43.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.646+0000 7f40d6800700 1 --2- 172.21.15.1:0/2580013304 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561035737010 0x5610357373f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:30:43.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.647+0000 7f40d7200700 1 --2- 172.21.15.1:0/2580013304 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5610357427e0 0x561035744bd0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f40d00115b0 tx=0x7f40d0009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db5f86a94b89e00 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:43.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.647+0000 7f40c7400700 1 -- 172.21.15.1:0/2580013304 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d000e040 con 0x5610357427e0 2024-02-02T15:30:43.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.647+0000 7f40c7400700 1 -- 172.21.15.1:0/2580013304 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f40d000f040 con 0x5610357427e0 2024-02-02T15:30:43.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.647+0000 7f40c7400700 1 -- 172.21.15.1:0/2580013304 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d00043d0 con 0x5610357427e0 2024-02-02T15:30:43.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.648+0000 7f40f0f22b80 1 -- 172.21.15.1:0/2580013304 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5610357427e0 msgr2=0x561035744bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:43.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.648+0000 7f40f0f22b80 1 --2- 172.21.15.1:0/2580013304 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5610357427e0 0x561035744bd0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f40d00115b0 tx=0x7f40d0009920 comp rx=0 tx=0).stop 2024-02-02T15:30:43.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.648+0000 7f40f0f22b80 1 -- 172.21.15.1:0/2580013304 shutdown_connections 2024-02-02T15:30:43.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.648+0000 7f40f0f22b80 1 --2- 172.21.15.1:0/2580013304 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5610357427e0 0x561035744bd0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:43.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.648+0000 7f40f0f22b80 1 --2- 172.21.15.1:0/2580013304 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561035737010 0x5610357373f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:43.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.648+0000 7f40f0f22b80 1 --2- 172.21.15.1:0/2580013304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561035737930 0x5610357422a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:43.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.648+0000 7f40f0f22b80 1 -- 172.21.15.1:0/2580013304 >> 172.21.15.1:0/2580013304 conn(0x561035696db0 msgr2=0x5610357333b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:43.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.648+0000 7f40f0f22b80 1 -- 172.21.15.1:0/2580013304 shutdown_connections 2024-02-02T15:30:43.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.648+0000 7f40f0f22b80 1 -- 172.21.15.1:0/2580013304 wait complete. 2024-02-02T15:30:43.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.648+0000 7f40f0f22b80 1 -- start start 2024-02-02T15:30:43.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.648+0000 7f40f0f22b80 1 Processor -- start 2024-02-02T15:30:43.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.649+0000 7f40f0f22b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561035737010 0x561035742170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:43.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.649+0000 7f40f0f22b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561035737930 0x56103573f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:43.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.649+0000 7f40f0f22b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5610357427e0 0x56103573f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:43.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.649+0000 7f40f0f22b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x561035688d50 con 0x561035737010 2024-02-02T15:30:43.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.649+0000 7f40f0f22b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x561035624d80 con 0x561035737930 2024-02-02T15:30:43.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.649+0000 7f40f0f22b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561035624a20 con 0x5610357427e0 2024-02-02T15:30:43.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.649+0000 7f40f0f22b80 10 client.0 _ll_register_callbacks cb 0x561035626290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:30:43.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.649+0000 7f40d5e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561035737930 0x56103573f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:43.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.649+0000 7f40d5e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561035737930 0x56103573f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:38514/0 (socket says 172.21.15.1:38514) 2024-02-02T15:30:43.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.649+0000 7f40d6800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561035737010 0x561035742170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:43.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.649+0000 7f40d5e00700 1 -- 172.21.15.1:0/2701357689 learned_addr learned my addr 172.21.15.1:0/2701357689 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:30:43.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.649+0000 7f40d7200700 1 --2- 172.21.15.1:0/2701357689 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5610357427e0 0x56103573f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:43.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.650+0000 7f40d6800700 1 -- 172.21.15.1:0/2701357689 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5610357427e0 msgr2=0x56103573f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:43.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.650+0000 7f40d6800700 1 --2- 172.21.15.1:0/2701357689 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5610357427e0 0x56103573f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:43.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.650+0000 7f40d6800700 1 -- 172.21.15.1:0/2701357689 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561035737930 msgr2=0x56103573f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:43.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.650+0000 7f40d6800700 1 --2- 172.21.15.1:0/2701357689 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561035737930 0x56103573f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:43.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.650+0000 7f40d6800700 1 -- 172.21.15.1:0/2701357689 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40d81a6110 con 0x561035737010 2024-02-02T15:30:43.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.650+0000 7f40d6800700 1 --2- 172.21.15.1:0/2701357689 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561035737010 0x561035742170 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f40c80966a0 tx=0x7f40c80051b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b58d262035526e50 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:43.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.650+0000 7f40bfe00700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:30:43.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.650+0000 7f40bfe00700 1 -- 172.21.15.1:0/2701357689 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c809a600 con 0x561035737010 2024-02-02T15:30:43.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.650+0000 7f40bfe00700 1 -- 172.21.15.1:0/2701357689 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f40c809a8a0 con 0x561035737010 2024-02-02T15:30:43.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.650+0000 7f40f0f22b80 1 -- 172.21.15.1:0/2701357689 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56103572b0f0 con 0x561035737010 2024-02-02T15:30:43.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.651+0000 7f40bfe00700 1 -- 172.21.15.1:0/2701357689 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f40c80056a0 con 0x561035737010 2024-02-02T15:30:43.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.651+0000 7f40f0f22b80 10 client.16521 fetch_fsmap learned FSMap version 31 2024-02-02T15:30:43.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.651+0000 7f40f0f22b80 1 -- 172.21.15.1:0/2701357689 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f40d0011230 con 0x561035737010 2024-02-02T15:30:43.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.651+0000 7f40bfe00700 1 -- 172.21.15.1:0/2701357689 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f40c8064080 con 0x561035737010 2024-02-02T15:30:43.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.651+0000 7f40bfe00700 1 -- 172.21.15.1:0/2701357689 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 31) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f40c80997b0 con 0x561035737010 2024-02-02T15:30:43.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.651+0000 7f40f0f22b80 10 client.16521 fetch_fsmap finished waiting for FSMap version 31 2024-02-02T15:30:43.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.651+0000 7f40f0f22b80 10 client.16521 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:30:43.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.651+0000 7f40f0f22b80 20 client.16521 populate_metadata read hostname 'smithi001' 2024-02-02T15:30:43.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.651+0000 7f40f0f22b80 1 --2- 172.21.15.1:0/2701357689 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x561035751fa0 0x5610357543c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:43.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.651+0000 7f40f0f22b80 4 client.16521 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:30:43.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.651+0000 7f40f0f22b80 1 -- 172.21.15.1:0/2701357689 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561035681dd0 con 0x561035751fa0 2024-02-02T15:30:43.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.652+0000 7f40d5e00700 1 --2- 172.21.15.1:0/2701357689 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x561035751fa0 0x5610357543c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:43.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.652+0000 7f40bfe00700 10 client.16521 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:30:43.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.652+0000 7f40d5e00700 1 --2- 172.21.15.1:0/2701357689 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x561035751fa0 0x5610357543c0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=659d94700283e8da server_cookie=bfff6d0734a32ccd in_seq=0 out_seq=0 2024-02-02T15:30:43.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.653+0000 7f40bfe00700 1 -- 172.21.15.1:0/2701357689 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x561035681dd0 con 0x561035751fa0 2024-02-02T15:30:43.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.653+0000 7f40bfe00700 10 client.16521 handle_command_reply: tid=0 2024-02-02T15:30:43.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.653+0000 7f40f0f22b80 1 client.16521 shutdown 2024-02-02T15:30:43.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.653+0000 7f40f0f22b80 1 -- 172.21.15.1:0/2701357689 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561035737010 msgr2=0x561035742170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:43.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.653+0000 7f40f0f22b80 1 --2- 172.21.15.1:0/2701357689 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561035737010 0x561035742170 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f40c80966a0 tx=0x7f40c80051b0 comp rx=0 tx=0).stop 2024-02-02T15:30:43.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.653+0000 7f40f0f22b80 1 -- 172.21.15.1:0/2701357689 shutdown_connections 2024-02-02T15:30:43.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.653+0000 7f40f0f22b80 1 --2- 172.21.15.1:0/2701357689 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x561035751fa0 0x5610357543c0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:43.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.653+0000 7f40f0f22b80 1 --2- 172.21.15.1:0/2701357689 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5610357427e0 0x56103573f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:43.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.653+0000 7f40f0f22b80 1 --2- 172.21.15.1:0/2701357689 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561035737010 0x561035742170 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:43.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.653+0000 7f40bfe00700 0 client.16521 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:30:43.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.653+0000 7f40f0f22b80 1 --2- 172.21.15.1:0/2701357689 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561035737930 0x56103573f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:43.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.653+0000 7f40f0f22b80 1 -- 172.21.15.1:0/2701357689 >> 172.21.15.1:0/2701357689 conn(0x561035696db0 msgr2=0x561035732fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:43.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.653+0000 7f40f0f22b80 1 -- 172.21.15.1:0/2701357689 shutdown_connections 2024-02-02T15:30:43.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.653+0000 7f40f0f22b80 1 -- 172.21.15.1:0/2701357689 wait complete. 2024-02-02T15:30:43.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.653+0000 7f40f0f22b80 20 client.16521 trim_cache size 0 max 16384 2024-02-02T15:30:43.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:43 smithi142 ceph-mon[45728]: pgmap v114: 129 pgs: 129 active+clean; 105 MiB data, 912 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 288 op/s 2024-02-02T15:30:43.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.671+0000 7f40f0f22b80 1 Processor -- start 2024-02-02T15:30:43.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.671+0000 7f40f0f22b80 1 -- start start 2024-02-02T15:30:43.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.672+0000 7f40f0f22b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561035737010 0x5610357493a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:43.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.672+0000 7f40f0f22b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561035737930 0x561035741ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:43.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.672+0000 7f40f0f22b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5610357427e0 0x56103573eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:43.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.672+0000 7f40f0f22b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56103550c6a0 con 0x5610357427e0 2024-02-02T15:30:43.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.672+0000 7f40f0f22b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x561035734a30 con 0x561035737010 2024-02-02T15:30:43.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.672+0000 7f40f0f22b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56103573d8e0 con 0x561035737930 2024-02-02T15:30:43.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.672+0000 7f40d5e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561035737930 0x561035741ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:43.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.672+0000 7f40d6800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561035737010 0x5610357493a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:43.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.672+0000 7f40d5e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561035737930 0x561035741ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:40864/0 (socket says 172.21.15.1:40864) 2024-02-02T15:30:43.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.672+0000 7f40d6800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561035737010 0x5610357493a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:38522/0 (socket says 172.21.15.1:38522) 2024-02-02T15:30:43.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.672+0000 7f40d5e00700 1 -- 172.21.15.1:0/1490481217 learned_addr learned my addr 172.21.15.1:0/1490481217 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:30:43.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.672+0000 7f40d7200700 1 --2- 172.21.15.1:0/1490481217 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5610357427e0 0x56103573eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:43.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.672+0000 7f40d5e00700 1 -- 172.21.15.1:0/1490481217 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561035737010 msgr2=0x5610357493a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:43.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.672+0000 7f40d5e00700 1 --2- 172.21.15.1:0/1490481217 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561035737010 0x5610357493a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:43.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.672+0000 7f40d5e00700 1 -- 172.21.15.1:0/1490481217 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5610357427e0 msgr2=0x56103573eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:43.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.672+0000 7f40d5e00700 1 --2- 172.21.15.1:0/1490481217 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5610357427e0 0x56103573eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:43.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.672+0000 7f40d5e00700 1 -- 172.21.15.1:0/1490481217 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5610354b6b60 con 0x561035737930 2024-02-02T15:30:43.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.672+0000 7f40d6800700 1 --2- 172.21.15.1:0/1490481217 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561035737010 0x5610357493a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:30:43.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.672+0000 7f40d7200700 1 --2- 172.21.15.1:0/1490481217 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5610357427e0 0x56103573eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:30:43.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.673+0000 7f40d5e00700 1 --2- 172.21.15.1:0/1490481217 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561035737930 0x561035741ca0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x561035754c60 tx=0x7f40c0014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a7463fe54d09044 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:43.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.673+0000 7f40c7e00700 1 -- 172.21.15.1:0/1490481217 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c0024040 con 0x561035737930 2024-02-02T15:30:43.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.673+0000 7f40c7e00700 1 -- 172.21.15.1:0/1490481217 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f40c0003d40 con 0x561035737930 2024-02-02T15:30:43.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.673+0000 7f40c7e00700 1 -- 172.21.15.1:0/1490481217 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c001e8e0 con 0x561035737930 2024-02-02T15:30:43.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.674+0000 7f40f0f22b80 1 -- 172.21.15.1:0/1490481217 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561035737930 msgr2=0x561035741ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:43.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.674+0000 7f40f0f22b80 1 --2- 172.21.15.1:0/1490481217 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561035737930 0x561035741ca0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x561035754c60 tx=0x7f40c0014000 comp rx=0 tx=0).stop 2024-02-02T15:30:43.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.674+0000 7f40f0f22b80 1 -- 172.21.15.1:0/1490481217 shutdown_connections 2024-02-02T15:30:43.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.674+0000 7f40f0f22b80 1 --2- 172.21.15.1:0/1490481217 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5610357427e0 0x56103573eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:43.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.674+0000 7f40f0f22b80 1 --2- 172.21.15.1:0/1490481217 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561035737010 0x5610357493a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:43.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.674+0000 7f40f0f22b80 1 --2- 172.21.15.1:0/1490481217 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561035737930 0x561035741ca0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:43.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.674+0000 7f40f0f22b80 1 -- 172.21.15.1:0/1490481217 >> 172.21.15.1:0/1490481217 conn(0x561035696db0 msgr2=0x5610357325c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:43.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.674+0000 7f40f0f22b80 1 -- 172.21.15.1:0/1490481217 shutdown_connections 2024-02-02T15:30:43.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.674+0000 7f40f0f22b80 1 -- 172.21.15.1:0/1490481217 wait complete. 2024-02-02T15:30:43.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.674+0000 7f40f0f22b80 1 -- start start 2024-02-02T15:30:43.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.674+0000 7f40f0f22b80 1 Processor -- start 2024-02-02T15:30:43.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.675+0000 7f40f0f22b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561035737010 0x561035749180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:43.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.675+0000 7f40f0f22b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561035737930 0x561035747400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:43.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.675+0000 7f40f0f22b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5610357427e0 0x561035747940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:43.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.675+0000 7f40f0f22b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x561035747ff0 con 0x561035737010 2024-02-02T15:30:43.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.675+0000 7f40f0f22b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x561035748160 con 0x561035737930 2024-02-02T15:30:43.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.675+0000 7f40f0f22b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610357a3a90 con 0x5610357427e0 2024-02-02T15:30:43.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.675+0000 7f40f0f22b80 10 client.0 _ll_register_callbacks cb 0x561035626290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:30:43.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.675+0000 7f40d6800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561035737010 0x561035749180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:43.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.675+0000 7f40d6800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561035737010 0x561035749180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44970/0 (socket says 172.21.15.1:44970) 2024-02-02T15:30:43.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.675+0000 7f40d6800700 1 -- 172.21.15.1:0/1949255838 learned_addr learned my addr 172.21.15.1:0/1949255838 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:30:43.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.675+0000 7f40d5e00700 1 --2- 172.21.15.1:0/1949255838 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561035737930 0x561035747400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:43.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.675+0000 7f40d7200700 1 --2- 172.21.15.1:0/1949255838 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5610357427e0 0x561035747940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:43.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.676+0000 7f40d6800700 1 -- 172.21.15.1:0/1949255838 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5610357427e0 msgr2=0x561035747940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:43.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.676+0000 7f40d6800700 1 --2- 172.21.15.1:0/1949255838 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5610357427e0 0x561035747940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:43.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.676+0000 7f40d6800700 1 -- 172.21.15.1:0/1949255838 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561035737930 msgr2=0x561035747400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:43.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.676+0000 7f40d6800700 1 --2- 172.21.15.1:0/1949255838 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561035737930 0x561035747400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:43.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.676+0000 7f40d6800700 1 -- 172.21.15.1:0/1949255838 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40d0011230 con 0x561035737010 2024-02-02T15:30:43.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.676+0000 7f40d6800700 1 --2- 172.21.15.1:0/1949255838 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561035737010 0x561035749180 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f40c80984f0 tx=0x7f40c8005220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3b766f32cb4d05d server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:43.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.676+0000 7f40bfe00700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:30:43.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.676+0000 7f40bfe00700 1 -- 172.21.15.1:0/1949255838 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c809a8a0 con 0x561035737010 2024-02-02T15:30:43.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.676+0000 7f40f0f22b80 1 -- 172.21.15.1:0/1949255838 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f40d000f040 con 0x561035737010 2024-02-02T15:30:43.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.676+0000 7f40bfe00700 1 -- 172.21.15.1:0/1949255838 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f40c809a1a0 con 0x561035737010 2024-02-02T15:30:43.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.676+0000 7f40bfe00700 1 -- 172.21.15.1:0/1949255838 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f40c809dfd0 con 0x561035737010 2024-02-02T15:30:43.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.676+0000 7f40f0f22b80 10 client.16527 fetch_fsmap learned FSMap version 31 2024-02-02T15:30:43.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.676+0000 7f40f0f22b80 1 -- 172.21.15.1:0/1949255838 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x56103550a3b0 con 0x561035737010 2024-02-02T15:30:43.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.677+0000 7f40bfe00700 1 -- 172.21.15.1:0/1949255838 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f40d000f040 con 0x561035737010 2024-02-02T15:30:43.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.677+0000 7f40bfe00700 1 -- 172.21.15.1:0/1949255838 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 31) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f40c8099770 con 0x561035737010 2024-02-02T15:30:43.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.677+0000 7f40f0f22b80 10 client.16527 fetch_fsmap finished waiting for FSMap version 31 2024-02-02T15:30:43.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.677+0000 7f40f0f22b80 10 client.16527 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:30:43.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.677+0000 7f40f0f22b80 20 client.16527 populate_metadata read hostname 'smithi001' 2024-02-02T15:30:43.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.677+0000 7f40f0f22b80 1 --2- 172.21.15.1:0/1949255838 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5610357a4cf0 0x5610357a7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:43.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.677+0000 7f40f0f22b80 4 client.16527 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2024-02-02T15:30:43.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.677+0000 7f40f0f22b80 1 -- 172.21.15.1:0/1949255838 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561035647f50 con 0x5610357a4cf0 2024-02-02T15:30:43.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.677+0000 7f40d5e00700 1 --2- 172.21.15.1:0/1949255838 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5610357a4cf0 0x5610357a7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:43.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.678+0000 7f40bfe00700 10 client.16527 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:30:43.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.678+0000 7f40d5e00700 1 --2- 172.21.15.1:0/1949255838 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5610357a4cf0 0x5610357a7110 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70ec58b995ba4e32 server_cookie=7112bcf8461f8590 in_seq=0 out_seq=0 2024-02-02T15:30:43.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.678+0000 7f40bfe00700 1 -- 172.21.15.1:0/1949255838 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x561035647f50 con 0x5610357a4cf0 2024-02-02T15:30:43.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.678+0000 7f40bfe00700 10 client.16527 handle_command_reply: tid=0 2024-02-02T15:30:43.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.678+0000 7f40f0f22b80 1 client.16527 shutdown 2024-02-02T15:30:43.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.678+0000 7f40f0f22b80 1 -- 172.21.15.1:0/1949255838 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561035737010 msgr2=0x561035749180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:43.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.678+0000 7f40f0f22b80 1 --2- 172.21.15.1:0/1949255838 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561035737010 0x561035749180 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f40c80984f0 tx=0x7f40c8005220 comp rx=0 tx=0).stop 2024-02-02T15:30:43.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.678+0000 7f40f0f22b80 1 -- 172.21.15.1:0/1949255838 shutdown_connections 2024-02-02T15:30:43.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.678+0000 7f40f0f22b80 1 --2- 172.21.15.1:0/1949255838 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5610357a4cf0 0x5610357a7110 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:43.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.678+0000 7f40f0f22b80 1 --2- 172.21.15.1:0/1949255838 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5610357427e0 0x561035747940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:43.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.678+0000 7f40f0f22b80 1 --2- 172.21.15.1:0/1949255838 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561035737010 0x561035749180 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:43.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.678+0000 7f40f0f22b80 1 --2- 172.21.15.1:0/1949255838 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561035737930 0x561035747400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:43.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.678+0000 7f40f0f22b80 1 -- 172.21.15.1:0/1949255838 >> 172.21.15.1:0/1949255838 conn(0x561035696db0 msgr2=0x561035732e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:43.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.678+0000 7f40bfe00700 0 client.16527 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:30:43.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.678+0000 7f40f0f22b80 1 -- 172.21.15.1:0/1949255838 shutdown_connections 2024-02-02T15:30:43.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.678+0000 7f40f0f22b80 1 -- 172.21.15.1:0/1949255838 wait complete. 2024-02-02T15:30:43.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.678+0000 7f40f0f22b80 20 client.16527 trim_cache size 0 max 16384 2024-02-02T15:30:43.682 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:30:43.682 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2024-02-02T15:30:43.682 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2024-02-02T15:30:43.682 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:30:43.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.679+0000 7f40dd400700 1 -- 172.21.15.1:0/3267362239 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f40b80a0690 msgr2=0x7f40b80a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:43.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.679+0000 7f40dd400700 1 --2- 172.21.15.1:0/3267362239 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f40b80a0690 0x7f40b80a2b50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f40d810eca0 tx=0x7f40c0009310 comp rx=0 tx=0).stop 2024-02-02T15:30:43.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.679+0000 7f40dd400700 1 -- 172.21.15.1:0/3267362239 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40d81000e0 msgr2=0x7f40d810d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:43.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.679+0000 7f40dd400700 1 --2- 172.21.15.1:0/3267362239 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40d81000e0 0x7f40d810d4b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f40c8009810 tx=0x7f40c800fa00 comp rx=0 tx=0).stop 2024-02-02T15:30:43.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.679+0000 7f40dd400700 1 -- 172.21.15.1:0/3267362239 shutdown_connections 2024-02-02T15:30:43.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.679+0000 7f40dd400700 1 --2- 172.21.15.1:0/3267362239 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f40b80a0690 0x7f40b80a2b50 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:43.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.679+0000 7f40dd400700 1 --2- 172.21.15.1:0/3267362239 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40d810df30 0x7f40d81a50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:43.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.679+0000 7f40dd400700 1 --2- 172.21.15.1:0/3267362239 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40d81000e0 0x7f40d810d4b0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:43.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.679+0000 7f40dd400700 1 --2- 172.21.15.1:0/3267362239 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40d8105020 0x7f40d810d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:43.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.679+0000 7f40dd400700 1 -- 172.21.15.1:0/3267362239 >> 172.21.15.1:0/3267362239 conn(0x7f40d80757d0 msgr2=0x7f40d80fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:43.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.679+0000 7f40dd400700 1 -- 172.21.15.1:0/3267362239 shutdown_connections 2024-02-02T15:30:43.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.679+0000 7f40dd400700 1 -- 172.21.15.1:0/3267362239 wait complete. 2024-02-02T15:30:43.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:43 smithi001 ceph-mon[35470]: pgmap v114: 129 pgs: 129 active+clean; 105 MiB data, 912 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 288 op/s 2024-02-02T15:30:43.696 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-02T15:30:43.696 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T15:30:43.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.910+0000 7fe40bc00700 1 -- 172.21.15.1:0/2680514035 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe4041018d0 msgr2=0x7fe404105910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:43.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.910+0000 7fe40bc00700 1 --2- 172.21.15.1:0/2680514035 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe4041018d0 0x7fe404105910 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec009990 tx=0x7fe3ec009ca0 comp rx=0 tx=0).stop 2024-02-02T15:30:43.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.910+0000 7fe40bc00700 1 -- 172.21.15.1:0/2680514035 shutdown_connections 2024-02-02T15:30:43.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.910+0000 7fe40bc00700 1 --2- 172.21.15.1:0/2680514035 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe404106040 0x7fe40410dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:43.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.910+0000 7fe40bc00700 1 --2- 172.21.15.1:0/2680514035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe404100f20 0x7fe404101300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:43.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.910+0000 7fe40bc00700 1 --2- 172.21.15.1:0/2680514035 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe4041018d0 0x7fe404105910 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:43.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.910+0000 7fe40bc00700 1 -- 172.21.15.1:0/2680514035 >> 172.21.15.1:0/2680514035 conn(0x7fe4040fc7a0 msgr2=0x7fe4040febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:43.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.910+0000 7fe40bc00700 1 -- 172.21.15.1:0/2680514035 shutdown_connections 2024-02-02T15:30:43.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.910+0000 7fe40bc00700 1 -- 172.21.15.1:0/2680514035 wait complete. 2024-02-02T15:30:43.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.911+0000 7fe40bc00700 1 Processor -- start 2024-02-02T15:30:43.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.911+0000 7fe40bc00700 1 -- start start 2024-02-02T15:30:43.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.911+0000 7fe40bc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe404100f20 0x7fe404198fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:43.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.911+0000 7fe40bc00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe404106040 0x7fe404199500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:43.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.911+0000 7fe40bc00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe40419d960 0x7fe40419dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:43.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.911+0000 7fe40bc00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe40419e3e0 con 0x7fe404106040 2024-02-02T15:30:43.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.911+0000 7fe40bc00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe40419e550 con 0x7fe40419d960 2024-02-02T15:30:43.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.911+0000 7fe40bc00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe40419e710 con 0x7fe404100f20 2024-02-02T15:30:43.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.912+0000 7fe409200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe404100f20 0x7fe404198fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:43.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.912+0000 7fe409200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe404100f20 0x7fe404198fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:40894/0 (socket says 172.21.15.1:40894) 2024-02-02T15:30:43.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.912+0000 7fe409200700 1 -- 172.21.15.1:0/1568530094 learned_addr learned my addr 172.21.15.1:0/1568530094 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:30:43.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.912+0000 7fe403e00700 1 --2- 172.21.15.1:0/1568530094 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe404106040 0x7fe404199500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:43.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.912+0000 7fe409c00700 1 --2- 172.21.15.1:0/1568530094 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe40419d960 0x7fe40419dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:43.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.912+0000 7fe409200700 1 -- 172.21.15.1:0/1568530094 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe40419d960 msgr2=0x7fe40419dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:43.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.912+0000 7fe409200700 1 --2- 172.21.15.1:0/1568530094 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe40419d960 0x7fe40419dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:43.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.912+0000 7fe409200700 1 -- 172.21.15.1:0/1568530094 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe404106040 msgr2=0x7fe404199500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:43.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.912+0000 7fe409200700 1 --2- 172.21.15.1:0/1568530094 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe404106040 0x7fe404199500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:43.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.912+0000 7fe409200700 1 -- 172.21.15.1:0/1568530094 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3ec0095f0 con 0x7fe404100f20 2024-02-02T15:30:43.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.912+0000 7fe409c00700 1 --2- 172.21.15.1:0/1568530094 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe40419d960 0x7fe40419dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:30:43.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.912+0000 7fe409200700 1 --2- 172.21.15.1:0/1568530094 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe404100f20 0x7fe404198fc0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe3f400e9a0 tx=0x7fe3f400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:43.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.912+0000 7fe401600700 1 -- 172.21.15.1:0/1568530094 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f4009790 con 0x7fe404100f20 2024-02-02T15:30:43.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.913+0000 7fe40bc00700 1 -- 172.21.15.1:0/1568530094 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4041a5c70 con 0x7fe404100f20 2024-02-02T15:30:43.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.913+0000 7fe401600700 1 -- 172.21.15.1:0/1568530094 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe3f400cbd0 con 0x7fe404100f20 2024-02-02T15:30:43.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.913+0000 7fe401600700 1 -- 172.21.15.1:0/1568530094 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f4018680 con 0x7fe404100f20 2024-02-02T15:30:43.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.913+0000 7fe40bc00700 1 -- 172.21.15.1:0/1568530094 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4041a6150 con 0x7fe404100f20 2024-02-02T15:30:43.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.914+0000 7fe40bc00700 1 -- 172.21.15.1:0/1568530094 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe404067bc0 con 0x7fe404100f20 2024-02-02T15:30:43.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.915+0000 7fe401600700 1 -- 172.21.15.1:0/1568530094 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 27) v1 ==== 135200+0+0 (secure 0 0 0) 0x7fe3f400cd40 con 0x7fe404100f20 2024-02-02T15:30:43.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.917+0000 7fe401600700 1 --2- 172.21.15.1:0/1568530094 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fe3e40a0690 0x7fe3e40a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:43.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.917+0000 7fe401600700 1 -- 172.21.15.1:0/1568530094 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fe3f4014070 con 0x7fe404100f20 2024-02-02T15:30:43.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.917+0000 7fe403e00700 1 --2- 172.21.15.1:0/1568530094 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fe3e40a0690 0x7fe3e40a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:43.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.918+0000 7fe403e00700 1 --2- 172.21.15.1:0/1568530094 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fe3e40a0690 0x7fe3e40a2b50 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe40419a5d0 tx=0x7fe3ec005fb0 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:43.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:43.919+0000 7fe401600700 1 -- 172.21.15.1:0/1568530094 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7fe3f4095b40 con 0x7fe404100f20 2024-02-02T15:30:44.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.203+0000 7fe40bc00700 1 -- 172.21.15.1:0/1568530094 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe404101ad0 con 0x7fe404100f20 2024-02-02T15:30:44.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.203+0000 7fe401600700 1 -- 172.21.15.1:0/1568530094 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+10880 (secure 0 0 0) 0x7fe3f4099160 con 0x7fe404100f20 2024-02-02T15:30:44.202 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:30:44.203 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2922240448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2922240448},{"type":"v1","addr":"172.21.15.1:6835","nonce":2922240448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2729497431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2729497431},{"type":"v1","addr":"172.21.15.1:6837","nonce":2729497431}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-02T15:24:47.761435+0000","modified":"2024-02-02T15:29:20.322035+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24529,"mds_2":24518,"mds_3":24514,"mds_4":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14517":{"gid":14517,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/1218303683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":1218303683},{"type":"v1","addr":"172.21.15.1:6839","nonce":1218303683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14529":{"gid":14529,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/2059197496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2059197496},{"type":"v1","addr":"172.21.15.1:6841","nonce":2059197496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/7648986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":7648986},{"type":"v1","addr":"172.21.15.142:6833","nonce":7648986}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/3482746094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3482746094},{"type":"v1","addr":"172.21.15.71:6833","nonce":3482746094}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/388207814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":388207814},{"type":"v1","addr":"172.21.15.71:6835","nonce":388207814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6835/1052606547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1052606547},{"type":"v1","addr":"172.21.15.142:6835","nonce":1052606547}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.142:6837/2870289019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2870289019},{"type":"v1","addr":"172.21.15.142:6837","nonce":2870289019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.71:6837/628016199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":628016199},{"type":"v1","addr":"172.21.15.71:6837","nonce":628016199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.142:6839/321675421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":321675421},{"type":"v1","addr":"172.21.15.142:6839","nonce":321675421}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/1268086493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1268086493},{"type":"v1","addr":"172.21.15.71:6839","nonce":1268086493}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-02T15:30:44.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.206+0000 7fe40bc00700 1 -- 172.21.15.1:0/1568530094 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fe3e40a0690 msgr2=0x7fe3e40a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:44.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.206+0000 7fe40bc00700 1 --2- 172.21.15.1:0/1568530094 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fe3e40a0690 0x7fe3e40a2b50 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe40419a5d0 tx=0x7fe3ec005fb0 comp rx=0 tx=0).stop 2024-02-02T15:30:44.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.206+0000 7fe40bc00700 1 -- 172.21.15.1:0/1568530094 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe404100f20 msgr2=0x7fe404198fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:44.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.206+0000 7fe40bc00700 1 --2- 172.21.15.1:0/1568530094 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe404100f20 0x7fe404198fc0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe3f400e9a0 tx=0x7fe3f400ecb0 comp rx=0 tx=0).stop 2024-02-02T15:30:44.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.206+0000 7fe40bc00700 1 -- 172.21.15.1:0/1568530094 shutdown_connections 2024-02-02T15:30:44.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.206+0000 7fe40bc00700 1 --2- 172.21.15.1:0/1568530094 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fe3e40a0690 0x7fe3e40a2b50 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:44.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.206+0000 7fe40bc00700 1 --2- 172.21.15.1:0/1568530094 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe40419d960 0x7fe40419dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:44.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.206+0000 7fe40bc00700 1 --2- 172.21.15.1:0/1568530094 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe404100f20 0x7fe404198fc0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:44.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.206+0000 7fe40bc00700 1 --2- 172.21.15.1:0/1568530094 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe404106040 0x7fe404199500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:44.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.206+0000 7fe40bc00700 1 -- 172.21.15.1:0/1568530094 >> 172.21.15.1:0/1568530094 conn(0x7fe4040fc7a0 msgr2=0x7fe4040fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:44.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.206+0000 7fe40bc00700 1 -- 172.21.15.1:0/1568530094 shutdown_connections 2024-02-02T15:30:44.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.206+0000 7fe40bc00700 1 -- 172.21.15.1:0/1568530094 wait complete. 2024-02-02T15:30:44.206 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 31 2024-02-02T15:30:44.225 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-02-02T15:30:44.229 INFO:tasks.workunit.client.0.smithi001.stdout: 1 172988 9.77 MB/sec execute 18 sec latency 19.649 ms 2024-02-02T15:30:44.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.448+0000 7f27ea200700 1 -- 172.21.15.1:0/3470885176 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27e4103230 msgr2=0x7f27e4103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:44.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.448+0000 7f27ea200700 1 --2- 172.21.15.1:0/3470885176 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27e4103230 0x7f27e4103610 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f27d4009990 tx=0x7f27d4009ca0 comp rx=0 tx=0).stop 2024-02-02T15:30:44.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.448+0000 7f27ea200700 1 -- 172.21.15.1:0/3470885176 shutdown_connections 2024-02-02T15:30:44.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.448+0000 7f27ea200700 1 --2- 172.21.15.1:0/3470885176 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f27e4108350 0x7f27e410fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:44.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.448+0000 7f27ea200700 1 --2- 172.21.15.1:0/3470885176 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27e4103230 0x7f27e4103610 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:44.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.448+0000 7f27ea200700 1 --2- 172.21.15.1:0/3470885176 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27e4103be0 0x7f27e4107c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:44.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.448+0000 7f27ea200700 1 -- 172.21.15.1:0/3470885176 >> 172.21.15.1:0/3470885176 conn(0x7f27e40fea70 msgr2=0x7f27e4100e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:44.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.448+0000 7f27ea200700 1 -- 172.21.15.1:0/3470885176 shutdown_connections 2024-02-02T15:30:44.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.448+0000 7f27ea200700 1 -- 172.21.15.1:0/3470885176 wait complete. 2024-02-02T15:30:44.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.449+0000 7f27ea200700 1 Processor -- start 2024-02-02T15:30:44.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.449+0000 7f27ea200700 1 -- start start 2024-02-02T15:30:44.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.450+0000 7f27ea200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27e4103230 0x7f27e4199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:44.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.450+0000 7f27ea200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f27e4108350 0x7f27e4199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:44.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.450+0000 7f27ea200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27e419dab0 0x7f27e419df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:44.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.450+0000 7f27ea200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e419e530 con 0x7f27e4108350 2024-02-02T15:30:44.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.450+0000 7f27ea200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e419e6a0 con 0x7f27e419dab0 2024-02-02T15:30:44.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.450+0000 7f27ea200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e419e860 con 0x7f27e4103230 2024-02-02T15:30:44.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.450+0000 7f27e2a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f27e4108350 0x7f27e4199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:44.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.450+0000 7f27e2a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f27e4108350 0x7f27e4199650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45006/0 (socket says 172.21.15.1:45006) 2024-02-02T15:30:44.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.450+0000 7f27e3e00700 1 --2- 172.21.15.1:0/337995433 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27e419dab0 0x7f27e419df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:44.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.450+0000 7f27e2a00700 1 -- 172.21.15.1:0/337995433 learned_addr learned my addr 172.21.15.1:0/337995433 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:30:44.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.450+0000 7f27e3400700 1 --2- 172.21.15.1:0/337995433 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27e4103230 0x7f27e4199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:44.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.451+0000 7f27e3e00700 1 -- 172.21.15.1:0/337995433 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27e4103230 msgr2=0x7f27e4199110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:44.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.451+0000 7f27e3e00700 1 --2- 172.21.15.1:0/337995433 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27e4103230 0x7f27e4199110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:44.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.451+0000 7f27e3e00700 1 -- 172.21.15.1:0/337995433 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f27e4108350 msgr2=0x7f27e4199650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:44.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.451+0000 7f27e3e00700 1 --2- 172.21.15.1:0/337995433 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f27e4108350 0x7f27e4199650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:44.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.451+0000 7f27e3e00700 1 -- 172.21.15.1:0/337995433 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27d40095f0 con 0x7f27e419dab0 2024-02-02T15:30:44.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.451+0000 7f27e3e00700 1 --2- 172.21.15.1:0/337995433 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27e419dab0 0x7f27e419df30 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f27dc007f10 tx=0x7f27dc00d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:44.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.451+0000 7f27dbe00700 1 -- 172.21.15.1:0/337995433 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27dc004db0 con 0x7f27e419dab0 2024-02-02T15:30:44.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.451+0000 7f27ea200700 1 -- 172.21.15.1:0/337995433 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27e41a5dc0 con 0x7f27e419dab0 2024-02-02T15:30:44.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.451+0000 7f27dbe00700 1 -- 172.21.15.1:0/337995433 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27dc00f040 con 0x7f27e419dab0 2024-02-02T15:30:44.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.451+0000 7f27dbe00700 1 -- 172.21.15.1:0/337995433 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27dc005990 con 0x7f27e419dab0 2024-02-02T15:30:44.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.451+0000 7f27ea200700 1 -- 172.21.15.1:0/337995433 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27e41a62a0 con 0x7f27e419dab0 2024-02-02T15:30:44.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.455+0000 7f27dbe00700 1 -- 172.21.15.1:0/337995433 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 27) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f27dc00db20 con 0x7f27e419dab0 2024-02-02T15:30:44.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.456+0000 7f27fdd2ab80 1 Processor -- start 2024-02-02T15:30:44.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.457+0000 7f27dbe00700 1 --2- 172.21.15.1:0/337995433 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f27c40a0690 0x7f27c40a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:44.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.457+0000 7f27fdd2ab80 1 -- start start 2024-02-02T15:30:44.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.457+0000 7f27dbe00700 1 -- 172.21.15.1:0/337995433 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f27dc0cb890 con 0x7f27e419dab0 2024-02-02T15:30:44.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.457+0000 7f27e2a00700 1 --2- 172.21.15.1:0/337995433 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f27c40a0690 0x7f27c40a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:44.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.457+0000 7f27fdd2ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ee83a6010 0x557ee83a63f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:44.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.457+0000 7f27fdd2ab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557ee83a6930 0x557ee83b12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:44.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.457+0000 7f27fdd2ab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557ee83b17e0 0x557ee83b3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:44.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.457+0000 7f27fdd2ab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ee831a4c0 con 0x557ee83b17e0 2024-02-02T15:30:44.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.457+0000 7f27fdd2ab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ee82feaa0 con 0x557ee83a6930 2024-02-02T15:30:44.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.457+0000 7f27fdd2ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ee82fc460 con 0x557ee83a6010 2024-02-02T15:30:44.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.458+0000 7f27e3400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ee83a6010 0x557ee83a63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:44.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.458+0000 7f27e3400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ee83a6010 0x557ee83a63f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:40922/0 (socket says 172.21.15.1:40922) 2024-02-02T15:30:44.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.458+0000 7f27e3400700 1 -- 172.21.15.1:0/3821042958 learned_addr learned my addr 172.21.15.1:0/3821042958 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:30:44.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.458+0000 7f27e2a00700 1 --2- 172.21.15.1:0/3821042958 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557ee83a6930 0x557ee83b12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:44.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.458+0000 7f27e3e00700 1 --2- 172.21.15.1:0/3821042958 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557ee83b17e0 0x557ee83b3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:44.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.458+0000 7f27e3400700 1 -- 172.21.15.1:0/3821042958 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557ee83a6930 msgr2=0x557ee83b12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:44.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.458+0000 7f27e3400700 1 --2- 172.21.15.1:0/3821042958 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557ee83a6930 0x557ee83b12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:44.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.458+0000 7f27e3400700 1 -- 172.21.15.1:0/3821042958 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557ee83b17e0 msgr2=0x557ee83b3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:44.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.458+0000 7f27e3400700 1 --2- 172.21.15.1:0/3821042958 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557ee83b17e0 0x557ee83b3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:44.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.458+0000 7f27e2a00700 1 --2- 172.21.15.1:0/337995433 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f27c40a0690 0x7f27c40a2b50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f27e419a720 tx=0x7f27cc011720 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:44.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.458+0000 7f27e3400700 1 -- 172.21.15.1:0/3821042958 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557ee81793b0 con 0x557ee83a6010 2024-02-02T15:30:44.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.459+0000 7f27e3400700 1 --2- 172.21.15.1:0/3821042958 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ee83a6010 0x557ee83a63f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f27d400b5b0 tx=0x7f27d400b5e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89637f96a10f3d79 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:44.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.459+0000 7f27d3e00700 1 -- 172.21.15.1:0/3821042958 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d401d070 con 0x557ee83a6010 2024-02-02T15:30:44.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.459+0000 7f27d3e00700 1 -- 172.21.15.1:0/3821042958 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27d401e040 con 0x557ee83a6010 2024-02-02T15:30:44.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.459+0000 7f27d3e00700 1 -- 172.21.15.1:0/3821042958 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d40048a0 con 0x557ee83a6010 2024-02-02T15:30:44.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.460+0000 7f27fdd2ab80 1 -- 172.21.15.1:0/3821042958 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ee83a6010 msgr2=0x557ee83a63f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:44.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.460+0000 7f27fdd2ab80 1 --2- 172.21.15.1:0/3821042958 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ee83a6010 0x557ee83a63f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f27d400b5b0 tx=0x7f27d400b5e0 comp rx=0 tx=0).stop 2024-02-02T15:30:44.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.460+0000 7f27fdd2ab80 1 -- 172.21.15.1:0/3821042958 shutdown_connections 2024-02-02T15:30:44.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.460+0000 7f27fdd2ab80 1 --2- 172.21.15.1:0/3821042958 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557ee83b17e0 0x557ee83b3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:44.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.460+0000 7f27fdd2ab80 1 --2- 172.21.15.1:0/3821042958 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ee83a6010 0x557ee83a63f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:44.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.460+0000 7f27fdd2ab80 1 --2- 172.21.15.1:0/3821042958 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557ee83a6930 0x557ee83b12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:44.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.460+0000 7f27fdd2ab80 1 -- 172.21.15.1:0/3821042958 >> 172.21.15.1:0/3821042958 conn(0x557ee8305db0 msgr2=0x557ee83a23b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:44.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.460+0000 7f27fdd2ab80 1 -- 172.21.15.1:0/3821042958 shutdown_connections 2024-02-02T15:30:44.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.460+0000 7f27fdd2ab80 1 -- 172.21.15.1:0/3821042958 wait complete. 2024-02-02T15:30:44.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.460+0000 7f27fdd2ab80 1 -- start start 2024-02-02T15:30:44.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.461+0000 7f27fdd2ab80 1 Processor -- start 2024-02-02T15:30:44.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.461+0000 7f27fdd2ab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557ee83a6010 0x557ee83b1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:44.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.461+0000 7f27fdd2ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ee83a6930 0x557ee83ae3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:44.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.461+0000 7f27fdd2ab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557ee83b17e0 0x557ee83aea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:44.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.461+0000 7f27fdd2ab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ee82f7d50 con 0x557ee83b17e0 2024-02-02T15:30:44.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.461+0000 7f27fdd2ab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ee8293d80 con 0x557ee83a6010 2024-02-02T15:30:44.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.461+0000 7f27fdd2ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ee8293a20 con 0x557ee83a6930 2024-02-02T15:30:44.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.461+0000 7f27fdd2ab80 10 client.0 _ll_register_callbacks cb 0x557ee8295290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:30:44.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.461+0000 7f27e3400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557ee83a6010 0x557ee83b1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:44.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.461+0000 7f27e3400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557ee83a6010 0x557ee83b1170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:38574/0 (socket says 172.21.15.1:38574) 2024-02-02T15:30:44.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.461+0000 7f27e3400700 1 -- 172.21.15.1:0/2409085604 learned_addr learned my addr 172.21.15.1:0/2409085604 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:30:44.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.461+0000 7f27e3e00700 1 --2- 172.21.15.1:0/2409085604 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557ee83b17e0 0x557ee83aea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:44.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.462+0000 7f27e2a00700 1 --2- 172.21.15.1:0/2409085604 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ee83a6930 0x557ee83ae3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:44.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.462+0000 7f27e3400700 1 -- 172.21.15.1:0/2409085604 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ee83a6930 msgr2=0x557ee83ae3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:44.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.462+0000 7f27e3400700 1 --2- 172.21.15.1:0/2409085604 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ee83a6930 0x557ee83ae3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:44.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.462+0000 7f27e3400700 1 -- 172.21.15.1:0/2409085604 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557ee83b17e0 msgr2=0x557ee83aea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:44.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.462+0000 7f27e3400700 1 --2- 172.21.15.1:0/2409085604 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557ee83b17e0 0x557ee83aea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:44.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.462+0000 7f27e3400700 1 -- 172.21.15.1:0/2409085604 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27e4110740 con 0x557ee83a6010 2024-02-02T15:30:44.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.462+0000 7f27e3400700 1 --2- 172.21.15.1:0/2409085604 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557ee83a6010 0x557ee83b1170 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f27d400b2e0 tx=0x7f27d400ba00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5fe5605c18dbe33 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:44.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.462+0000 7f27d0c00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:30:44.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.462+0000 7f27d0c00700 1 -- 172.21.15.1:0/2409085604 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d401d030 con 0x557ee83a6010 2024-02-02T15:30:44.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.462+0000 7f27d0c00700 1 -- 172.21.15.1:0/2409085604 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f27d4004700 con 0x557ee83a6010 2024-02-02T15:30:44.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.462+0000 7f27fdd2ab80 1 -- 172.21.15.1:0/2409085604 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557ee839a0f0 con 0x557ee83a6010 2024-02-02T15:30:44.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.463+0000 7f27d0c00700 1 -- 172.21.15.1:0/2409085604 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f27d40048c0 con 0x557ee83a6010 2024-02-02T15:30:44.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.463+0000 7f27fdd2ab80 10 client.26603 fetch_fsmap learned FSMap version 31 2024-02-02T15:30:44.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.463+0000 7f27fdd2ab80 1 -- 172.21.15.1:0/2409085604 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x557ee8125b60 con 0x557ee83a6010 2024-02-02T15:30:44.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.463+0000 7f27d0c00700 1 -- 172.21.15.1:0/2409085604 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557ee839a0f0 con 0x557ee83a6010 2024-02-02T15:30:44.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.463+0000 7f27d0c00700 1 -- 172.21.15.1:0/2409085604 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 31) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f27d400fad0 con 0x557ee83a6010 2024-02-02T15:30:44.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.463+0000 7f27fdd2ab80 10 client.26603 fetch_fsmap finished waiting for FSMap version 31 2024-02-02T15:30:44.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.463+0000 7f27fdd2ab80 10 client.26603 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-02-02T15:30:44.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.463+0000 7f27fdd2ab80 20 client.26603 populate_metadata read hostname 'smithi001' 2024-02-02T15:30:44.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.463+0000 7f27fdd2ab80 1 --2- 172.21.15.1:0/2409085604 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x557ee83c0d00 0x557ee83c30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:44.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.463+0000 7f27fdd2ab80 4 client.26603 mds_command: new command op to 24502 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:30:44.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.463+0000 7f27fdd2ab80 1 -- 172.21.15.1:0/2409085604 --> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557ee82f0dd0 con 0x557ee83c0d00 2024-02-02T15:30:44.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.463+0000 7f27e2a00700 1 --2- 172.21.15.1:0/2409085604 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x557ee83c0d00 0x557ee83c30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:44.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.464+0000 7f27e2a00700 1 --2- 172.21.15.1:0/2409085604 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x557ee83c0d00 0x557ee83c30f0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8321ecabc3ec7e20 server_cookie=fcac2b4423ec649e in_seq=0 out_seq=0 2024-02-02T15:30:44.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.464+0000 7f27d0c00700 10 client.26603 ms_handle_connect on v2:172.21.15.142:6834/1052606547 2024-02-02T15:30:44.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.465+0000 7f27d0c00700 1 -- 172.21.15.1:0/2409085604 <== mds.4 v2:172.21.15.142:6834/1052606547 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x557ee82f0dd0 con 0x557ee83c0d00 2024-02-02T15:30:44.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.465+0000 7f27d0c00700 10 client.26603 handle_command_reply: tid=0 2024-02-02T15:30:44.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.465+0000 7f27fdd2ab80 1 client.26603 shutdown 2024-02-02T15:30:44.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.465+0000 7f27fdd2ab80 1 -- 172.21.15.1:0/2409085604 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557ee83a6010 msgr2=0x557ee83b1170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:44.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.465+0000 7f27fdd2ab80 1 --2- 172.21.15.1:0/2409085604 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557ee83a6010 0x557ee83b1170 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f27d400b2e0 tx=0x7f27d400ba00 comp rx=0 tx=0).stop 2024-02-02T15:30:44.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.465+0000 7f27fdd2ab80 1 -- 172.21.15.1:0/2409085604 shutdown_connections 2024-02-02T15:30:44.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.465+0000 7f27fdd2ab80 1 --2- 172.21.15.1:0/2409085604 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x557ee83c0d00 0x557ee83c30f0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:44.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.465+0000 7f27fdd2ab80 1 --2- 172.21.15.1:0/2409085604 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557ee83b17e0 0x557ee83aea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:44.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.465+0000 7f27fdd2ab80 1 --2- 172.21.15.1:0/2409085604 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557ee83a6010 0x557ee83b1170 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:44.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.465+0000 7f27fdd2ab80 1 --2- 172.21.15.1:0/2409085604 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ee83a6930 0x557ee83ae3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:44.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.465+0000 7f27fdd2ab80 1 -- 172.21.15.1:0/2409085604 >> 172.21.15.1:0/2409085604 conn(0x557ee8305db0 msgr2=0x557ee83a1f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:44.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.465+0000 7f27d0c00700 0 client.26603 ms_handle_reset on v2:172.21.15.142:6834/1052606547 2024-02-02T15:30:44.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.465+0000 7f27fdd2ab80 1 -- 172.21.15.1:0/2409085604 shutdown_connections 2024-02-02T15:30:44.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.465+0000 7f27fdd2ab80 1 -- 172.21.15.1:0/2409085604 wait complete. 2024-02-02T15:30:44.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.466+0000 7f27fdd2ab80 20 client.26603 trim_cache size 0 max 16384 2024-02-02T15:30:44.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.480+0000 7f27fdd2ab80 1 Processor -- start 2024-02-02T15:30:44.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.480+0000 7f27fdd2ab80 1 -- start start 2024-02-02T15:30:44.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.481+0000 7f27fdd2ab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557ee83a6010 0x557ee83b0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:44.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.481+0000 7f27fdd2ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ee83a6930 0x557ee83bc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:44.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.481+0000 7f27fdd2ab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557ee83b17e0 0x557ee83bcde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:44.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.481+0000 7f27fdd2ab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ee817b6a0 con 0x557ee83a6010 2024-02-02T15:30:44.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.481+0000 7f27fdd2ab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ee83a7830 con 0x557ee83b17e0 2024-02-02T15:30:44.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.481+0000 7f27fdd2ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ee83ac950 con 0x557ee83a6930 2024-02-02T15:30:44.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.481+0000 7f27e3400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557ee83a6010 0x557ee83b0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:44.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.481+0000 7f27e2a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ee83a6930 0x557ee83bc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:44.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.481+0000 7f27e3400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557ee83a6010 0x557ee83b0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45032/0 (socket says 172.21.15.1:45032) 2024-02-02T15:30:44.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.481+0000 7f27e3400700 1 -- 172.21.15.1:0/3113385311 learned_addr learned my addr 172.21.15.1:0/3113385311 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:30:44.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.481+0000 7f27e3e00700 1 --2- 172.21.15.1:0/3113385311 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557ee83b17e0 0x557ee83bcde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:44.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.481+0000 7f27e2a00700 1 -- 172.21.15.1:0/3113385311 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557ee83b17e0 msgr2=0x557ee83bcde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:44.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.481+0000 7f27e2a00700 1 --2- 172.21.15.1:0/3113385311 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557ee83b17e0 0x557ee83bcde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:44.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.482+0000 7f27e2a00700 1 -- 172.21.15.1:0/3113385311 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557ee83a6010 msgr2=0x557ee83b0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:44.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.482+0000 7f27e2a00700 1 --2- 172.21.15.1:0/3113385311 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557ee83a6010 0x557ee83b0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:44.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.482+0000 7f27e2a00700 1 -- 172.21.15.1:0/3113385311 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557ee82bdea0 con 0x557ee83a6930 2024-02-02T15:30:44.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.482+0000 7f27e2a00700 1 --2- 172.21.15.1:0/3113385311 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ee83a6930 0x557ee83bc8a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x557ee83af9a0 tx=0x7f27cc01c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c947e8a5faf2b28 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:44.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.482+0000 7f27d9200700 1 -- 172.21.15.1:0/3113385311 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27cc02c040 con 0x557ee83a6930 2024-02-02T15:30:44.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.482+0000 7f27d9200700 1 -- 172.21.15.1:0/3113385311 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27cc026d70 con 0x557ee83a6930 2024-02-02T15:30:44.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.482+0000 7f27d9200700 1 -- 172.21.15.1:0/3113385311 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27cc01dbd0 con 0x557ee83a6930 2024-02-02T15:30:44.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.483+0000 7f27fdd2ab80 1 -- 172.21.15.1:0/3113385311 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ee83a6930 msgr2=0x557ee83bc8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:44.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.483+0000 7f27fdd2ab80 1 --2- 172.21.15.1:0/3113385311 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ee83a6930 0x557ee83bc8a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x557ee83af9a0 tx=0x7f27cc01c000 comp rx=0 tx=0).stop 2024-02-02T15:30:44.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.483+0000 7f27fdd2ab80 1 -- 172.21.15.1:0/3113385311 shutdown_connections 2024-02-02T15:30:44.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.483+0000 7f27fdd2ab80 1 --2- 172.21.15.1:0/3113385311 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557ee83b17e0 0x557ee83bcde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:44.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.483+0000 7f27fdd2ab80 1 --2- 172.21.15.1:0/3113385311 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557ee83a6010 0x557ee83b0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:44.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.483+0000 7f27fdd2ab80 1 --2- 172.21.15.1:0/3113385311 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ee83a6930 0x557ee83bc8a0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:44.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.483+0000 7f27fdd2ab80 1 -- 172.21.15.1:0/3113385311 >> 172.21.15.1:0/3113385311 conn(0x557ee8305db0 msgr2=0x557ee83bd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:44.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.483+0000 7f27fdd2ab80 1 -- 172.21.15.1:0/3113385311 shutdown_connections 2024-02-02T15:30:44.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.483+0000 7f27fdd2ab80 1 -- 172.21.15.1:0/3113385311 wait complete. 2024-02-02T15:30:44.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.483+0000 7f27fdd2ab80 1 -- start start 2024-02-02T15:30:44.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.483+0000 7f27fdd2ab80 1 Processor -- start 2024-02-02T15:30:44.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.484+0000 7f27fdd2ab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557ee83a6010 0x557ee83b5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:44.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.484+0000 7f27fdd2ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ee83a6930 0x557ee83b63d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:44.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.484+0000 7f27fdd2ab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557ee83b17e0 0x557ee84117b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:44.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.484+0000 7f27fdd2ab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ee8411cf0 con 0x557ee83a6010 2024-02-02T15:30:44.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.484+0000 7f27fdd2ab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ee8411e30 con 0x557ee83b17e0 2024-02-02T15:30:44.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.484+0000 7f27fdd2ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ee8411f70 con 0x557ee83a6930 2024-02-02T15:30:44.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.484+0000 7f27fdd2ab80 10 client.0 _ll_register_callbacks cb 0x557ee8295290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:30:44.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.484+0000 7f27e3e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557ee83b17e0 0x557ee84117b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:44.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.484+0000 7f27e2a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ee83a6930 0x557ee83b63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:44.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.484+0000 7f27e3400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557ee83a6010 0x557ee83b5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:44.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.484+0000 7f27e3e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557ee83b17e0 0x557ee84117b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:38594/0 (socket says 172.21.15.1:38594) 2024-02-02T15:30:44.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.484+0000 7f27e2a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ee83a6930 0x557ee83b63d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:40950/0 (socket says 172.21.15.1:40950) 2024-02-02T15:30:44.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.484+0000 7f27e3e00700 1 -- 172.21.15.1:0/3128806839 learned_addr learned my addr 172.21.15.1:0/3128806839 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:30:44.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.485+0000 7f27e3e00700 1 -- 172.21.15.1:0/3128806839 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ee83a6930 msgr2=0x557ee83b63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:44.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.485+0000 7f27e3e00700 1 --2- 172.21.15.1:0/3128806839 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ee83a6930 0x557ee83b63d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:44.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.485+0000 7f27e3e00700 1 -- 172.21.15.1:0/3128806839 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557ee83a6010 msgr2=0x557ee83b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:44.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.485+0000 7f27e3e00700 1 --2- 172.21.15.1:0/3128806839 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557ee83a6010 0x557ee83b5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:44.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.485+0000 7f27e3e00700 1 -- 172.21.15.1:0/3128806839 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27cc026850 con 0x557ee83b17e0 2024-02-02T15:30:44.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.485+0000 7f27e3e00700 1 --2- 172.21.15.1:0/3128806839 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557ee83b17e0 0x557ee84117b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f27dc0989f0 tx=0x7f27dc0b8040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb38fc2227af70 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:44.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.485+0000 7f27d0c00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:30:44.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.485+0000 7f27d0c00700 1 -- 172.21.15.1:0/3128806839 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27dc098a60 con 0x557ee83b17e0 2024-02-02T15:30:44.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.485+0000 7f27fdd2ab80 1 -- 172.21.15.1:0/3128806839 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f27d401e040 con 0x557ee83b17e0 2024-02-02T15:30:44.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.485+0000 7f27d0c00700 1 -- 172.21.15.1:0/3128806839 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f27dc09c890 con 0x557ee83b17e0 2024-02-02T15:30:44.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.485+0000 7f27d0c00700 1 -- 172.21.15.1:0/3128806839 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f27dc0a0ad0 con 0x557ee83b17e0 2024-02-02T15:30:44.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.485+0000 7f27fdd2ab80 10 client.26609 fetch_fsmap learned FSMap version 31 2024-02-02T15:30:44.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.485+0000 7f27fdd2ab80 1 -- 172.21.15.1:0/3128806839 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x557ee817dcd0 con 0x557ee83b17e0 2024-02-02T15:30:44.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.486+0000 7f27d0c00700 1 -- 172.21.15.1:0/3128806839 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27d401e040 con 0x557ee83b17e0 2024-02-02T15:30:44.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.486+0000 7f27d0c00700 1 -- 172.21.15.1:0/3128806839 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 31) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f27dc05ed20 con 0x557ee83b17e0 2024-02-02T15:30:44.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.486+0000 7f27fdd2ab80 10 client.26609 fetch_fsmap finished waiting for FSMap version 31 2024-02-02T15:30:44.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.486+0000 7f27fdd2ab80 10 client.26609 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-02-02T15:30:44.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.486+0000 7f27fdd2ab80 20 client.26609 populate_metadata read hostname 'smithi001' 2024-02-02T15:30:44.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.486+0000 7f27fdd2ab80 1 --2- 172.21.15.1:0/3128806839 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x557ee8413640 0x557ee8415a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:44.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.486+0000 7f27fdd2ab80 4 client.26609 mds_command: new command op to 24502 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:30:44.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.486+0000 7f27fdd2ab80 1 -- 172.21.15.1:0/3128806839 --> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557ee82b6f50 con 0x557ee8413640 2024-02-02T15:30:44.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.486+0000 7f27e3400700 1 --2- 172.21.15.1:0/3128806839 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x557ee8413640 0x557ee8415a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:44.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.486+0000 7f27d0c00700 10 client.26609 ms_handle_connect on v2:172.21.15.142:6834/1052606547 2024-02-02T15:30:44.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.486+0000 7f27e3400700 1 --2- 172.21.15.1:0/3128806839 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x557ee8413640 0x557ee8415a60 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=48edb4bbb833821a server_cookie=4bd81d8a03977c49 in_seq=0 out_seq=0 2024-02-02T15:30:44.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.487+0000 7f27d0c00700 1 -- 172.21.15.1:0/3128806839 <== mds.4 v2:172.21.15.142:6834/1052606547 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557ee82b6f50 con 0x557ee8413640 2024-02-02T15:30:44.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.487+0000 7f27d0c00700 10 client.26609 handle_command_reply: tid=0 2024-02-02T15:30:44.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.487+0000 7f27fdd2ab80 1 client.26609 shutdown 2024-02-02T15:30:44.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.487+0000 7f27fdd2ab80 1 -- 172.21.15.1:0/3128806839 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557ee83b17e0 msgr2=0x557ee84117b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:44.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.487+0000 7f27fdd2ab80 1 --2- 172.21.15.1:0/3128806839 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557ee83b17e0 0x557ee84117b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f27dc0989f0 tx=0x7f27dc0b8040 comp rx=0 tx=0).stop 2024-02-02T15:30:44.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.487+0000 7f27fdd2ab80 1 -- 172.21.15.1:0/3128806839 shutdown_connections 2024-02-02T15:30:44.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.487+0000 7f27fdd2ab80 1 --2- 172.21.15.1:0/3128806839 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x557ee8413640 0x557ee8415a60 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:44.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.487+0000 7f27fdd2ab80 1 --2- 172.21.15.1:0/3128806839 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557ee83b17e0 0x557ee84117b0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:44.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.487+0000 7f27fdd2ab80 1 --2- 172.21.15.1:0/3128806839 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557ee83a6010 0x557ee83b5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:44.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.487+0000 7f27d0c00700 0 client.26609 ms_handle_reset on v2:172.21.15.142:6834/1052606547 2024-02-02T15:30:44.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.487+0000 7f27fdd2ab80 1 --2- 172.21.15.1:0/3128806839 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ee83a6930 0x557ee83b63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:44.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.487+0000 7f27fdd2ab80 1 -- 172.21.15.1:0/3128806839 >> 172.21.15.1:0/3128806839 conn(0x557ee8305db0 msgr2=0x557ee83bc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:44.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.487+0000 7f27fdd2ab80 1 -- 172.21.15.1:0/3128806839 shutdown_connections 2024-02-02T15:30:44.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.487+0000 7f27fdd2ab80 1 -- 172.21.15.1:0/3128806839 wait complete. 2024-02-02T15:30:44.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.488+0000 7f27fdd2ab80 20 client.26609 trim_cache size 0 max 16384 2024-02-02T15:30:44.492 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:30:44.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.488+0000 7f27ea200700 1 -- 172.21.15.1:0/337995433 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f27c40a0690 msgr2=0x7f27c40a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:44.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.488+0000 7f27ea200700 1 --2- 172.21.15.1:0/337995433 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f27c40a0690 0x7f27c40a2b50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f27e419a720 tx=0x7f27cc011720 comp rx=0 tx=0).stop 2024-02-02T15:30:44.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.488+0000 7f27ea200700 1 -- 172.21.15.1:0/337995433 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27e419dab0 msgr2=0x7f27e419df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:44.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.488+0000 7f27ea200700 1 --2- 172.21.15.1:0/337995433 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27e419dab0 0x7f27e419df30 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f27dc007f10 tx=0x7f27dc00d3b0 comp rx=0 tx=0).stop 2024-02-02T15:30:44.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.488+0000 7f27ea200700 1 -- 172.21.15.1:0/337995433 shutdown_connections 2024-02-02T15:30:44.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.488+0000 7f27ea200700 1 --2- 172.21.15.1:0/337995433 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f27c40a0690 0x7f27c40a2b50 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:44.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.488+0000 7f27ea200700 1 --2- 172.21.15.1:0/337995433 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27e419dab0 0x7f27e419df30 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:44.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.488+0000 7f27ea200700 1 --2- 172.21.15.1:0/337995433 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f27e4103230 0x7f27e4199110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:44.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.488+0000 7f27ea200700 1 --2- 172.21.15.1:0/337995433 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f27e4108350 0x7f27e4199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:44.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.488+0000 7f27ea200700 1 -- 172.21.15.1:0/337995433 >> 172.21.15.1:0/337995433 conn(0x7f27e40fea70 msgr2=0x7f27e4106c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:44.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.488+0000 7f27ea200700 1 -- 172.21.15.1:0/337995433 shutdown_connections 2024-02-02T15:30:44.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.488+0000 7f27ea200700 1 -- 172.21.15.1:0/337995433 wait complete. 2024-02-02T15:30:44.505 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:44 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/1568530094' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:30:44.507 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-02-02T15:30:44.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:44 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/1568530094' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:30:44.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:44 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/1568530094' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:30:44.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.717+0000 7feddce00700 1 -- 172.21.15.1:0/2278234433 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedd8100ff0 msgr2=0x7fedd81013d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:44.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.717+0000 7feddce00700 1 --2- 172.21.15.1:0/2278234433 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedd8100ff0 0x7fedd81013d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fedc8009990 tx=0x7fedc8009ca0 comp rx=0 tx=0).stop 2024-02-02T15:30:44.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.717+0000 7feddce00700 1 -- 172.21.15.1:0/2278234433 shutdown_connections 2024-02-02T15:30:44.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.717+0000 7feddce00700 1 --2- 172.21.15.1:0/2278234433 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fedd8106110 0x7fedd810dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:44.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.717+0000 7feddce00700 1 --2- 172.21.15.1:0/2278234433 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedd8100ff0 0x7fedd81013d0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:44.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.717+0000 7feddce00700 1 --2- 172.21.15.1:0/2278234433 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fedd81019a0 0x7fedd81059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:44.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.717+0000 7feddce00700 1 -- 172.21.15.1:0/2278234433 >> 172.21.15.1:0/2278234433 conn(0x7fedd80fc810 msgr2=0x7fedd80fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:44.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.717+0000 7feddce00700 1 -- 172.21.15.1:0/2278234433 shutdown_connections 2024-02-02T15:30:44.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.717+0000 7feddce00700 1 -- 172.21.15.1:0/2278234433 wait complete. 2024-02-02T15:30:44.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.718+0000 7feddce00700 1 Processor -- start 2024-02-02T15:30:44.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.718+0000 7feddce00700 1 -- start start 2024-02-02T15:30:44.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.718+0000 7feddce00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fedd8100ff0 0x7fedd8196f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:44.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.718+0000 7feddce00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedd8106110 0x7fedd8197450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:44.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.718+0000 7feddce00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fedd819b8b0 0x7fedd819bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:44.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.718+0000 7feddce00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedd819c330 con 0x7fedd8100ff0 2024-02-02T15:30:44.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.718+0000 7feddce00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedd819c4a0 con 0x7fedd819b8b0 2024-02-02T15:30:44.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.718+0000 7feddce00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedd819c660 con 0x7fedd8106110 2024-02-02T15:30:44.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.719+0000 7fedd5e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fedd8100ff0 0x7fedd8196f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:44.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.719+0000 7fedd5e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fedd8100ff0 0x7fedd8196f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45062/0 (socket says 172.21.15.1:45062) 2024-02-02T15:30:44.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.719+0000 7fedd6800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fedd819b8b0 0x7fedd819bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:44.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.719+0000 7fedd5e00700 1 -- 172.21.15.1:0/1011733882 learned_addr learned my addr 172.21.15.1:0/1011733882 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:30:44.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.719+0000 7fedd5400700 1 --2- 172.21.15.1:0/1011733882 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedd8106110 0x7fedd8197450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:44.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.719+0000 7fedd5e00700 1 -- 172.21.15.1:0/1011733882 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedd8106110 msgr2=0x7fedd8197450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:44.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.719+0000 7fedd5e00700 1 --2- 172.21.15.1:0/1011733882 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedd8106110 0x7fedd8197450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:44.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.719+0000 7fedd5e00700 1 -- 172.21.15.1:0/1011733882 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fedd819b8b0 msgr2=0x7fedd819bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:44.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.719+0000 7fedd5e00700 1 --2- 172.21.15.1:0/1011733882 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fedd819b8b0 0x7fedd819bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:44.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.719+0000 7fedd5e00700 1 -- 172.21.15.1:0/1011733882 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedc80095f0 con 0x7fedd8100ff0 2024-02-02T15:30:44.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.719+0000 7fedd6800700 1 --2- 172.21.15.1:0/1011733882 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fedd819b8b0 0x7fedd819bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:30:44.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.719+0000 7fedd5e00700 1 --2- 172.21.15.1:0/1011733882 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fedd8100ff0 0x7fedd8196f10 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fedc8005910 tx=0x7fedc8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:44.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.720+0000 7fedcea00700 1 -- 172.21.15.1:0/1011733882 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc801d070 con 0x7fedd8100ff0 2024-02-02T15:30:44.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.720+0000 7feddce00700 1 -- 172.21.15.1:0/1011733882 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedd81a3b60 con 0x7fedd8100ff0 2024-02-02T15:30:44.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.720+0000 7fedcea00700 1 -- 172.21.15.1:0/1011733882 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fedc8022470 con 0x7fedd8100ff0 2024-02-02T15:30:44.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.720+0000 7fedcea00700 1 -- 172.21.15.1:0/1011733882 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc8017650 con 0x7fedd8100ff0 2024-02-02T15:30:44.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.720+0000 7feddce00700 1 -- 172.21.15.1:0/1011733882 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedd81a3fe0 con 0x7fedd8100ff0 2024-02-02T15:30:44.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.723+0000 7fedcea00700 1 -- 172.21.15.1:0/1011733882 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 27) v1 ==== 135200+0+0 (secure 0 0 0) 0x7fedc80225e0 con 0x7fedd8100ff0 2024-02-02T15:30:44.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.724+0000 7fedf0986b80 1 Processor -- start 2024-02-02T15:30:44.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.724+0000 7fedcea00700 1 --2- 172.21.15.1:0/1011733882 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fedb80a0690 0x7fedb80a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:44.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.724+0000 7fedf0986b80 1 -- start start 2024-02-02T15:30:44.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.725+0000 7fedcea00700 1 -- 172.21.15.1:0/1011733882 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fedc80cd2c0 con 0x7fedd8100ff0 2024-02-02T15:30:44.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.725+0000 7fedd5400700 1 --2- 172.21.15.1:0/1011733882 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fedb80a0690 0x7fedb80a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:44.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.725+0000 7fedf0986b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56374bc3f420 0x56374bc3f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:44.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.725+0000 7fedf0986b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56374bc3fd40 0x56374bc4a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:44.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.725+0000 7fedf0986b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56374bc4abf0 0x56374bc4cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:44.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.725+0000 7fedf0986b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56374bbaf4c0 con 0x56374bc3f420 2024-02-02T15:30:44.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.725+0000 7fedf0986b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56374bb93aa0 con 0x56374bc3fd40 2024-02-02T15:30:44.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.725+0000 7fedf0986b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56374bb91460 con 0x56374bc4abf0 2024-02-02T15:30:44.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.725+0000 7fedd5e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56374bc3f420 0x56374bc3f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:44.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.725+0000 7fedd6800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56374bc4abf0 0x56374bc4cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:44.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.725+0000 7fedd5e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56374bc3f420 0x56374bc3f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45068/0 (socket says 172.21.15.1:45068) 2024-02-02T15:30:44.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.725+0000 7fedd5e00700 1 -- 172.21.15.1:0/1297733760 learned_addr learned my addr 172.21.15.1:0/1297733760 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:30:44.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.725+0000 7fedd5400700 1 --2- 172.21.15.1:0/1297733760 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56374bc3fd40 0x56374bc4a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:44.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.726+0000 7fedd5e00700 1 -- 172.21.15.1:0/1297733760 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56374bc4abf0 msgr2=0x56374bc4cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:44.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.726+0000 7fedd5e00700 1 --2- 172.21.15.1:0/1297733760 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56374bc4abf0 0x56374bc4cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:44.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.726+0000 7fedd5e00700 1 -- 172.21.15.1:0/1297733760 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56374bc3fd40 msgr2=0x56374bc4a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:44.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.726+0000 7fedd5e00700 1 --2- 172.21.15.1:0/1297733760 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56374bc3fd40 0x56374bc4a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:44.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.726+0000 7fedd5e00700 1 -- 172.21.15.1:0/1297733760 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56374ba0e3b0 con 0x56374bc3f420 2024-02-02T15:30:44.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.726+0000 7fedd6800700 1 --2- 172.21.15.1:0/1297733760 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56374bc4abf0 0x56374bc4cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:30:44.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.726+0000 7fedd5400700 1 --2- 172.21.15.1:0/1011733882 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fedb80a0690 0x7fedb80a2b50 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fedd8198520 tx=0x7fedc00118e0 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:44.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.726+0000 7fedd5e00700 1 --2- 172.21.15.1:0/1297733760 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56374bc3f420 0x56374bc3f800 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fedc80a1f70 tx=0x7fedc8009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d00463e6e0677ba3 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:44.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.726+0000 7fedc7400700 1 -- 172.21.15.1:0/1297733760 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc809abc0 con 0x56374bc3f420 2024-02-02T15:30:44.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.726+0000 7fedc7400700 1 -- 172.21.15.1:0/1297733760 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fedc809a4c0 con 0x56374bc3f420 2024-02-02T15:30:44.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.726+0000 7fedc7400700 1 -- 172.21.15.1:0/1297733760 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc809a720 con 0x56374bc3f420 2024-02-02T15:30:44.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.727+0000 7fedf0986b80 1 -- 172.21.15.1:0/1297733760 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56374bc3f420 msgr2=0x56374bc3f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:44.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.727+0000 7fedf0986b80 1 --2- 172.21.15.1:0/1297733760 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56374bc3f420 0x56374bc3f800 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fedc80a1f70 tx=0x7fedc8009ca0 comp rx=0 tx=0).stop 2024-02-02T15:30:44.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.727+0000 7fedf0986b80 1 -- 172.21.15.1:0/1297733760 shutdown_connections 2024-02-02T15:30:44.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.727+0000 7fedf0986b80 1 --2- 172.21.15.1:0/1297733760 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56374bc4abf0 0x56374bc4cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:44.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.727+0000 7fedf0986b80 1 --2- 172.21.15.1:0/1297733760 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56374bc3f420 0x56374bc3f800 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:44.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.727+0000 7fedf0986b80 1 --2- 172.21.15.1:0/1297733760 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56374bc3fd40 0x56374bc4a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:44.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.727+0000 7fedf0986b80 1 -- 172.21.15.1:0/1297733760 >> 172.21.15.1:0/1297733760 conn(0x56374bb9adb0 msgr2=0x56374bc373b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:44.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.727+0000 7fedf0986b80 1 -- 172.21.15.1:0/1297733760 shutdown_connections 2024-02-02T15:30:44.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.727+0000 7fedf0986b80 1 -- 172.21.15.1:0/1297733760 wait complete. 2024-02-02T15:30:44.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.728+0000 7fedf0986b80 1 -- start start 2024-02-02T15:30:44.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.728+0000 7fedf0986b80 1 Processor -- start 2024-02-02T15:30:44.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.729+0000 7fedf0986b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56374bc3f420 0x56374bc4afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:44.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.729+0000 7fedf0986b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56374bc3fd40 0x56374bc4b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:44.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.729+0000 7fedf0986b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56374bc4abf0 0x56374bc4ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:44.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.729+0000 7fedf0986b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56374bb8cd50 con 0x56374bc3f420 2024-02-02T15:30:44.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.729+0000 7fedf0986b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56374bb28d80 con 0x56374bc4abf0 2024-02-02T15:30:44.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.729+0000 7fedf0986b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56374bb28a20 con 0x56374bc3fd40 2024-02-02T15:30:44.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.729+0000 7fedf0986b80 10 client.0 _ll_register_callbacks cb 0x56374bb2a290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:30:44.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.729+0000 7fedd5400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56374bc3fd40 0x56374bc4b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:44.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.729+0000 7fedd5e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56374bc3f420 0x56374bc4afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:44.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.729+0000 7fedd5400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56374bc3fd40 0x56374bc4b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:40990/0 (socket says 172.21.15.1:40990) 2024-02-02T15:30:44.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.729+0000 7fedd5400700 1 -- 172.21.15.1:0/1248452424 learned_addr learned my addr 172.21.15.1:0/1248452424 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:30:44.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.729+0000 7fedd6800700 1 --2- 172.21.15.1:0/1248452424 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56374bc4abf0 0x56374bc4ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:44.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.729+0000 7fedd5400700 1 -- 172.21.15.1:0/1248452424 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56374bc4abf0 msgr2=0x56374bc4ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:44.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.729+0000 7fedd5400700 1 --2- 172.21.15.1:0/1248452424 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56374bc4abf0 0x56374bc4ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:44.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.729+0000 7fedd5400700 1 -- 172.21.15.1:0/1248452424 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56374bc3f420 msgr2=0x56374bc4afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:44.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.730+0000 7fedd5400700 1 --2- 172.21.15.1:0/1248452424 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56374bc3f420 0x56374bc4afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:44.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.730+0000 7fedd5400700 1 -- 172.21.15.1:0/1248452424 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedd0003a60 con 0x56374bc3fd40 2024-02-02T15:30:44.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.730+0000 7fedd5e00700 1 --2- 172.21.15.1:0/1248452424 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56374bc3f420 0x56374bc4afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:30:44.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.730+0000 7fedd5400700 1 --2- 172.21.15.1:0/1248452424 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56374bc3fd40 0x56374bc4b510 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fedc0005e90 tx=0x7fedc00047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19813fad02f24ad2 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:44.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.730+0000 7fedbfe00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:30:44.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.730+0000 7fedbfe00700 1 -- 172.21.15.1:0/1248452424 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc0003d10 con 0x56374bc3fd40 2024-02-02T15:30:44.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.730+0000 7fedbfe00700 1 -- 172.21.15.1:0/1248452424 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fedc0004540 con 0x56374bc3fd40 2024-02-02T15:30:44.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.730+0000 7fedf0986b80 1 -- 172.21.15.1:0/1248452424 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56374bc2f0f0 con 0x56374bc3fd40 2024-02-02T15:30:44.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.730+0000 7fedbfe00700 1 -- 172.21.15.1:0/1248452424 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fedc0003e70 con 0x56374bc3fd40 2024-02-02T15:30:44.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.730+0000 7fedf0986b80 10 client.26611 fetch_fsmap learned FSMap version 31 2024-02-02T15:30:44.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.730+0000 7fedf0986b80 1 -- 172.21.15.1:0/1248452424 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7fedd81a3fe0 con 0x56374bc3fd40 2024-02-02T15:30:44.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.731+0000 7fedbfe00700 1 -- 172.21.15.1:0/1248452424 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56374bc2f0f0 con 0x56374bc3fd40 2024-02-02T15:30:44.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.731+0000 7fedbfe00700 1 -- 172.21.15.1:0/1248452424 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 31) v1 ==== 7502+0+0 (secure 0 0 0) 0x7fedc0005540 con 0x56374bc3fd40 2024-02-02T15:30:44.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.731+0000 7fedf0986b80 10 client.26611 fetch_fsmap finished waiting for FSMap version 31 2024-02-02T15:30:44.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.731+0000 7fedf0986b80 10 client.26611 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-02-02T15:30:44.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.731+0000 7fedf0986b80 20 client.26611 populate_metadata read hostname 'smithi001' 2024-02-02T15:30:44.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.731+0000 7fedf0986b80 1 --2- 172.21.15.1:0/1248452424 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x56374bc5a320 0x56374bc5c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:44.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.731+0000 7fedf0986b80 4 client.26611 mds_command: new command op to 24514 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:30:44.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.731+0000 7fedf0986b80 1 -- 172.21.15.1:0/1248452424 --> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56374bb85dd0 con 0x56374bc5a320 2024-02-02T15:30:44.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.731+0000 7fedd5e00700 1 --2- 172.21.15.1:0/1248452424 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x56374bc5a320 0x56374bc5c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:44.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.732+0000 7fedd5e00700 1 --2- 172.21.15.1:0/1248452424 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x56374bc5a320 0x56374bc5c740 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=dc805f796dfa384a server_cookie=5c013f88bbe0dd93 in_seq=0 out_seq=0 2024-02-02T15:30:44.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.732+0000 7fedbfe00700 10 client.26611 ms_handle_connect on v2:172.21.15.142:6836/2870289019 2024-02-02T15:30:44.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.733+0000 7fedbfe00700 1 -- 172.21.15.1:0/1248452424 <== mds.3 v2:172.21.15.142:6836/2870289019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x56374bb85dd0 con 0x56374bc5a320 2024-02-02T15:30:44.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.733+0000 7fedbfe00700 10 client.26611 handle_command_reply: tid=0 2024-02-02T15:30:44.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.733+0000 7fedf0986b80 1 client.26611 shutdown 2024-02-02T15:30:44.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.733+0000 7fedf0986b80 1 -- 172.21.15.1:0/1248452424 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56374bc3fd40 msgr2=0x56374bc4b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:44.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.733+0000 7fedf0986b80 1 --2- 172.21.15.1:0/1248452424 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56374bc3fd40 0x56374bc4b510 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fedc0005e90 tx=0x7fedc00047c0 comp rx=0 tx=0).stop 2024-02-02T15:30:44.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.733+0000 7fedf0986b80 1 -- 172.21.15.1:0/1248452424 shutdown_connections 2024-02-02T15:30:44.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.733+0000 7fedf0986b80 1 --2- 172.21.15.1:0/1248452424 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56374bc4abf0 0x56374bc4ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:44.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.733+0000 7fedf0986b80 1 --2- 172.21.15.1:0/1248452424 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x56374bc5a320 0x56374bc5c740 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:44.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.733+0000 7fedf0986b80 1 --2- 172.21.15.1:0/1248452424 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56374bc3f420 0x56374bc4afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:44.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.733+0000 7fedf0986b80 1 --2- 172.21.15.1:0/1248452424 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56374bc3fd40 0x56374bc4b510 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:44.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.733+0000 7fedbfe00700 0 client.26611 ms_handle_reset on v2:172.21.15.142:6836/2870289019 2024-02-02T15:30:44.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.733+0000 7fedf0986b80 1 -- 172.21.15.1:0/1248452424 >> 172.21.15.1:0/1248452424 conn(0x56374bb9adb0 msgr2=0x56374bc36fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:44.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.733+0000 7fedf0986b80 1 -- 172.21.15.1:0/1248452424 shutdown_connections 2024-02-02T15:30:44.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.733+0000 7fedf0986b80 1 -- 172.21.15.1:0/1248452424 wait complete. 2024-02-02T15:30:44.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.734+0000 7fedf0986b80 20 client.26611 trim_cache size 0 max 16384 2024-02-02T15:30:44.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.747+0000 7fedf0986b80 1 Processor -- start 2024-02-02T15:30:44.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.747+0000 7fedf0986b80 1 -- start start 2024-02-02T15:30:44.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.748+0000 7fedf0986b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56374bc3f420 0x56374bc5c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:44.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.748+0000 7fedf0986b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56374bc3fd40 0x56374bc4f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:44.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.748+0000 7fedf0986b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56374bc4abf0 0x56374bc365c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:44.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.748+0000 7fedf0986b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56374ba106a0 con 0x56374bc4abf0 2024-02-02T15:30:44.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.748+0000 7fedf0986b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56374bc45aa0 con 0x56374bc3f420 2024-02-02T15:30:44.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.748+0000 7fedf0986b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56374bc3fa50 con 0x56374bc3fd40 2024-02-02T15:30:44.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.748+0000 7fedd5400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56374bc3fd40 0x56374bc4f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:44.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.748+0000 7fedd5e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56374bc3f420 0x56374bc5c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:44.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.748+0000 7fedd5400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56374bc3fd40 0x56374bc4f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:40994/0 (socket says 172.21.15.1:40994) 2024-02-02T15:30:44.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.748+0000 7fedd5400700 1 -- 172.21.15.1:0/3256153931 learned_addr learned my addr 172.21.15.1:0/3256153931 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:30:44.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.748+0000 7fedd6800700 1 --2- 172.21.15.1:0/3256153931 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56374bc4abf0 0x56374bc365c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:44.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.748+0000 7fedd5e00700 1 -- 172.21.15.1:0/3256153931 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56374bc3fd40 msgr2=0x56374bc4f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:44.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.748+0000 7fedd5e00700 1 --2- 172.21.15.1:0/3256153931 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56374bc3fd40 0x56374bc4f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:44.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.749+0000 7fedd5e00700 1 -- 172.21.15.1:0/3256153931 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56374bc4abf0 msgr2=0x56374bc365c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:44.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.749+0000 7fedd5e00700 1 --2- 172.21.15.1:0/3256153931 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56374bc4abf0 0x56374bc365c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:44.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.749+0000 7fedd5e00700 1 -- 172.21.15.1:0/3256153931 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56374b9bab60 con 0x56374bc3f420 2024-02-02T15:30:44.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.749+0000 7fedd5400700 1 --2- 172.21.15.1:0/3256153931 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56374bc3fd40 0x56374bc4f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:30:44.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.749+0000 7fedd5e00700 1 --2- 172.21.15.1:0/3256153931 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56374bc3f420 0x56374bc5c710 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fedc80a1f70 tx=0x7fedc8022cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58db379cdcd4cc20 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:44.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.749+0000 7fedc7e00700 1 -- 172.21.15.1:0/3256153931 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc80a24a0 con 0x56374bc3f420 2024-02-02T15:30:44.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.749+0000 7fedc7e00700 1 -- 172.21.15.1:0/3256153931 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fedc809e2f0 con 0x56374bc3f420 2024-02-02T15:30:44.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.749+0000 7fedc7e00700 1 -- 172.21.15.1:0/3256153931 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc809a720 con 0x56374bc3f420 2024-02-02T15:30:44.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.750+0000 7fedf0986b80 1 -- 172.21.15.1:0/3256153931 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56374bc3f420 msgr2=0x56374bc5c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:44.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.750+0000 7fedf0986b80 1 --2- 172.21.15.1:0/3256153931 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56374bc3f420 0x56374bc5c710 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fedc80a1f70 tx=0x7fedc8022cc0 comp rx=0 tx=0).stop 2024-02-02T15:30:44.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.750+0000 7fedf0986b80 1 -- 172.21.15.1:0/3256153931 shutdown_connections 2024-02-02T15:30:44.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.750+0000 7fedf0986b80 1 --2- 172.21.15.1:0/3256153931 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56374bc4abf0 0x56374bc365c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:44.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.750+0000 7fedf0986b80 1 --2- 172.21.15.1:0/3256153931 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56374bc3f420 0x56374bc5c710 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:44.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.750+0000 7fedf0986b80 1 --2- 172.21.15.1:0/3256153931 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56374bc3fd40 0x56374bc4f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:44.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.750+0000 7fedf0986b80 1 -- 172.21.15.1:0/3256153931 >> 172.21.15.1:0/3256153931 conn(0x56374bb9adb0 msgr2=0x56374bc494d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:44.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.750+0000 7fedf0986b80 1 -- 172.21.15.1:0/3256153931 shutdown_connections 2024-02-02T15:30:44.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.750+0000 7fedf0986b80 1 -- 172.21.15.1:0/3256153931 wait complete. 2024-02-02T15:30:44.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.750+0000 7fedf0986b80 1 -- start start 2024-02-02T15:30:44.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.751+0000 7fedf0986b80 1 Processor -- start 2024-02-02T15:30:44.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.751+0000 7fedf0986b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56374bc3f420 0x56374bc4ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:44.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.751+0000 7fedf0986b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56374bc3fd40 0x56374bc4e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:44.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.751+0000 7fedf0986b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56374bc4abf0 0x56374bca3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:44.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.751+0000 7fedf0986b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56374bca44a0 con 0x56374bc3fd40 2024-02-02T15:30:44.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.751+0000 7fedf0986b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56374bca4610 con 0x56374bc4abf0 2024-02-02T15:30:44.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.751+0000 7fedf0986b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56374bca47d0 con 0x56374bc3f420 2024-02-02T15:30:44.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.751+0000 7fedf0986b80 10 client.0 _ll_register_callbacks cb 0x56374bb2a290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:30:44.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.752+0000 7fedd5e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56374bc3f420 0x56374bc4ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:44.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.752+0000 7fedd5e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56374bc3f420 0x56374bc4ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:41002/0 (socket says 172.21.15.1:41002) 2024-02-02T15:30:44.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.752+0000 7fedd5e00700 1 -- 172.21.15.1:0/1800064718 learned_addr learned my addr 172.21.15.1:0/1800064718 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:30:44.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.752+0000 7fedd6800700 1 --2- 172.21.15.1:0/1800064718 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56374bc4abf0 0x56374bca3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:44.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.752+0000 7fedd5400700 1 --2- 172.21.15.1:0/1800064718 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56374bc3fd40 0x56374bc4e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:44.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.752+0000 7fedd5e00700 1 -- 172.21.15.1:0/1800064718 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56374bc4abf0 msgr2=0x56374bca3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:44.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.752+0000 7fedd5e00700 1 --2- 172.21.15.1:0/1800064718 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56374bc4abf0 0x56374bca3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:44.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.752+0000 7fedd5e00700 1 -- 172.21.15.1:0/1800064718 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56374bc3fd40 msgr2=0x56374bc4e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:44.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.752+0000 7fedd5e00700 1 --2- 172.21.15.1:0/1800064718 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56374bc3fd40 0x56374bc4e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:44.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.752+0000 7fedd5e00700 1 -- 172.21.15.1:0/1800064718 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedd81a3fe0 con 0x56374bc3f420 2024-02-02T15:30:44.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.752+0000 7fedd5400700 1 --2- 172.21.15.1:0/1800064718 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56374bc3fd40 0x56374bc4e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:30:44.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.752+0000 7fedd5e00700 1 --2- 172.21.15.1:0/1800064718 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56374bc3f420 0x56374bc4ddd0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fedc8005f50 tx=0x7fedc8005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9dcf9084d4f642dd server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:44.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.752+0000 7fedbfe00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:30:44.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.752+0000 7fedbfe00700 1 -- 172.21.15.1:0/1800064718 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc80a24a0 con 0x56374bc3f420 2024-02-02T15:30:44.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.752+0000 7fedbfe00700 1 -- 172.21.15.1:0/1800064718 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fedc80228c0 con 0x56374bc3f420 2024-02-02T15:30:44.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.752+0000 7fedf0986b80 1 -- 172.21.15.1:0/1800064718 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fedc809a4c0 con 0x56374bc3f420 2024-02-02T15:30:44.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.753+0000 7fedbfe00700 1 -- 172.21.15.1:0/1800064718 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fedc80b82c0 con 0x56374bc3f420 2024-02-02T15:30:44.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.753+0000 7fedf0986b80 10 client.26617 fetch_fsmap learned FSMap version 31 2024-02-02T15:30:44.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.753+0000 7fedf0986b80 1 -- 172.21.15.1:0/1800064718 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x56374ba0e3b0 con 0x56374bc3f420 2024-02-02T15:30:44.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.753+0000 7fedbfe00700 1 -- 172.21.15.1:0/1800064718 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fedc809a4c0 con 0x56374bc3f420 2024-02-02T15:30:44.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.753+0000 7fedbfe00700 1 -- 172.21.15.1:0/1800064718 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 31) v1 ==== 7502+0+0 (secure 0 0 0) 0x7fedc80b2bd0 con 0x56374bc3f420 2024-02-02T15:30:44.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.753+0000 7fedf0986b80 10 client.26617 fetch_fsmap finished waiting for FSMap version 31 2024-02-02T15:30:44.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.753+0000 7fedf0986b80 10 client.26617 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-02-02T15:30:44.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.753+0000 7fedf0986b80 20 client.26617 populate_metadata read hostname 'smithi001' 2024-02-02T15:30:44.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.753+0000 7fedf0986b80 1 --2- 172.21.15.1:0/1800064718 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x56374bcabaa0 0x56374bcadec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:44.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.753+0000 7fedf0986b80 4 client.26617 mds_command: new command op to 24514 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:30:44.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.753+0000 7fedf0986b80 1 -- 172.21.15.1:0/1800064718 --> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56374bb4bf50 con 0x56374bcabaa0 2024-02-02T15:30:44.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.753+0000 7fedd5400700 1 --2- 172.21.15.1:0/1800064718 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x56374bcabaa0 0x56374bcadec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:44.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.754+0000 7fedbfe00700 10 client.26617 ms_handle_connect on v2:172.21.15.142:6836/2870289019 2024-02-02T15:30:44.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.754+0000 7fedd5400700 1 --2- 172.21.15.1:0/1800064718 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x56374bcabaa0 0x56374bcadec0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fada22192284bb52 server_cookie=906ec96571178789 in_seq=0 out_seq=0 2024-02-02T15:30:44.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.754+0000 7fedbfe00700 1 -- 172.21.15.1:0/1800064718 <== mds.3 v2:172.21.15.142:6836/2870289019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56374bb4bf50 con 0x56374bcabaa0 2024-02-02T15:30:44.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.754+0000 7fedbfe00700 10 client.26617 handle_command_reply: tid=0 2024-02-02T15:30:44.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.754+0000 7fedf0986b80 1 client.26617 shutdown 2024-02-02T15:30:44.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.754+0000 7fedf0986b80 1 -- 172.21.15.1:0/1800064718 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56374bc3f420 msgr2=0x56374bc4ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:44.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.754+0000 7fedf0986b80 1 --2- 172.21.15.1:0/1800064718 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56374bc3f420 0x56374bc4ddd0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fedc8005f50 tx=0x7fedc8005fd0 comp rx=0 tx=0).stop 2024-02-02T15:30:44.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.754+0000 7fedf0986b80 1 -- 172.21.15.1:0/1800064718 shutdown_connections 2024-02-02T15:30:44.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.754+0000 7fedf0986b80 1 --2- 172.21.15.1:0/1800064718 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56374bc4abf0 0x56374bca3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:44.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.754+0000 7fedf0986b80 1 --2- 172.21.15.1:0/1800064718 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56374bc3f420 0x56374bc4ddd0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:44.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.754+0000 7fedf0986b80 1 --2- 172.21.15.1:0/1800064718 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x56374bcabaa0 0x56374bcadec0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:44.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.754+0000 7fedf0986b80 1 --2- 172.21.15.1:0/1800064718 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56374bc3fd40 0x56374bc4e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:44.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.754+0000 7fedf0986b80 1 -- 172.21.15.1:0/1800064718 >> 172.21.15.1:0/1800064718 conn(0x56374bb9adb0 msgr2=0x56374bc555e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:44.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.754+0000 7fedbfe00700 0 client.26617 ms_handle_reset on v2:172.21.15.142:6836/2870289019 2024-02-02T15:30:44.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.754+0000 7fedf0986b80 1 -- 172.21.15.1:0/1800064718 shutdown_connections 2024-02-02T15:30:44.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.754+0000 7fedf0986b80 1 -- 172.21.15.1:0/1800064718 wait complete. 2024-02-02T15:30:44.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.755+0000 7fedf0986b80 20 client.26617 trim_cache size 0 max 16384 2024-02-02T15:30:44.760 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:30:44.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.755+0000 7feddce00700 1 -- 172.21.15.1:0/1011733882 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fedb80a0690 msgr2=0x7fedb80a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:44.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.755+0000 7feddce00700 1 --2- 172.21.15.1:0/1011733882 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fedb80a0690 0x7fedb80a2b50 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fedd8198520 tx=0x7fedc00118e0 comp rx=0 tx=0).stop 2024-02-02T15:30:44.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.755+0000 7feddce00700 1 -- 172.21.15.1:0/1011733882 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fedd8100ff0 msgr2=0x7fedd8196f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:44.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.755+0000 7feddce00700 1 --2- 172.21.15.1:0/1011733882 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fedd8100ff0 0x7fedd8196f10 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fedc8005910 tx=0x7fedc8004730 comp rx=0 tx=0).stop 2024-02-02T15:30:44.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.755+0000 7feddce00700 1 -- 172.21.15.1:0/1011733882 shutdown_connections 2024-02-02T15:30:44.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.755+0000 7feddce00700 1 --2- 172.21.15.1:0/1011733882 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fedb80a0690 0x7fedb80a2b50 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:44.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.755+0000 7feddce00700 1 --2- 172.21.15.1:0/1011733882 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fedd819b8b0 0x7fedd819bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:44.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.755+0000 7feddce00700 1 --2- 172.21.15.1:0/1011733882 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fedd8100ff0 0x7fedd8196f10 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:44.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.755+0000 7feddce00700 1 --2- 172.21.15.1:0/1011733882 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fedd8106110 0x7fedd8197450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:44.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.755+0000 7feddce00700 1 -- 172.21.15.1:0/1011733882 >> 172.21.15.1:0/1011733882 conn(0x7fedd80fc810 msgr2=0x7fedd80fd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:44.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.755+0000 7feddce00700 1 -- 172.21.15.1:0/1011733882 shutdown_connections 2024-02-02T15:30:44.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:44.755+0000 7feddce00700 1 -- 172.21.15.1:0/1011733882 wait complete. 2024-02-02T15:30:44.772 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-02T15:30:45.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.006+0000 7f77afe00700 1 -- 172.21.15.1:0/3755818752 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77a8100ff0 msgr2=0x7f77a81013d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:45.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.006+0000 7f77afe00700 1 --2- 172.21.15.1:0/3755818752 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77a8100ff0 0x7f77a81013d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7798009990 tx=0x7f7798009ca0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.006+0000 7f77afe00700 1 -- 172.21.15.1:0/3755818752 shutdown_connections 2024-02-02T15:30:45.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.006+0000 7f77afe00700 1 --2- 172.21.15.1:0/3755818752 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f77a8106110 0x7f77a810dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.006+0000 7f77afe00700 1 --2- 172.21.15.1:0/3755818752 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77a8100ff0 0x7f77a81013d0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.006+0000 7f77afe00700 1 --2- 172.21.15.1:0/3755818752 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77a81019a0 0x7f77a81059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.006+0000 7f77afe00700 1 -- 172.21.15.1:0/3755818752 >> 172.21.15.1:0/3755818752 conn(0x7f77a80fc810 msgr2=0x7f77a80fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:45.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.006+0000 7f77afe00700 1 -- 172.21.15.1:0/3755818752 shutdown_connections 2024-02-02T15:30:45.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.006+0000 7f77afe00700 1 -- 172.21.15.1:0/3755818752 wait complete. 2024-02-02T15:30:45.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.007+0000 7f77afe00700 1 Processor -- start 2024-02-02T15:30:45.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.007+0000 7f77afe00700 1 -- start start 2024-02-02T15:30:45.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.008+0000 7f77afe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77a8100ff0 0x7f77a8199140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:45.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.008+0000 7f77afe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77a8106110 0x7f77a8199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:45.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.008+0000 7f77afe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f77a819dae0 0x7f77a819df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:45.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.008+0000 7f77afe00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77a819e560 con 0x7f77a819dae0 2024-02-02T15:30:45.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.008+0000 7f77afe00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77a819e6d0 con 0x7f77a8106110 2024-02-02T15:30:45.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.008+0000 7f77afe00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77a819e890 con 0x7f77a8100ff0 2024-02-02T15:30:45.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.008+0000 7f77ad400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77a8100ff0 0x7f77a8199140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:45.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.008+0000 7f77aca00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77a8106110 0x7f77a8199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:45.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.008+0000 7f77ade00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f77a819dae0 0x7f77a819df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:45.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.008+0000 7f77ad400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77a8100ff0 0x7f77a8199140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:41008/0 (socket says 172.21.15.1:41008) 2024-02-02T15:30:45.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.008+0000 7f77aca00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77a8106110 0x7f77a8199680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:38676/0 (socket says 172.21.15.1:38676) 2024-02-02T15:30:45.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.008+0000 7f77ad400700 1 -- 172.21.15.1:0/1358569212 learned_addr learned my addr 172.21.15.1:0/1358569212 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:30:45.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.009+0000 7f77ad400700 1 -- 172.21.15.1:0/1358569212 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77a8106110 msgr2=0x7f77a8199680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:45.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.009+0000 7f77ad400700 1 --2- 172.21.15.1:0/1358569212 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77a8106110 0x7f77a8199680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.009+0000 7f77ad400700 1 -- 172.21.15.1:0/1358569212 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f77a819dae0 msgr2=0x7f77a819df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:45.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.009+0000 7f77ad400700 1 --2- 172.21.15.1:0/1358569212 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f77a819dae0 0x7f77a819df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.009+0000 7f77ad400700 1 -- 172.21.15.1:0/1358569212 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77980095f0 con 0x7f77a8100ff0 2024-02-02T15:30:45.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.009+0000 7f77ad400700 1 --2- 172.21.15.1:0/1358569212 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77a8100ff0 0x7f77a8199140 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7798005910 tx=0x7f7798004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:45.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.009+0000 7f77ade00700 1 --2- 172.21.15.1:0/1358569212 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f77a819dae0 0x7f77a819df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:30:45.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.009+0000 7f77a6000700 1 -- 172.21.15.1:0/1358569212 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f779801d070 con 0x7f77a8100ff0 2024-02-02T15:30:45.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.009+0000 7f77a6000700 1 -- 172.21.15.1:0/1358569212 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7798022470 con 0x7f77a8100ff0 2024-02-02T15:30:45.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.009+0000 7f77a6000700 1 -- 172.21.15.1:0/1358569212 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7798017650 con 0x7f77a8100ff0 2024-02-02T15:30:45.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.009+0000 7f77afe00700 1 -- 172.21.15.1:0/1358569212 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77a81a5d90 con 0x7f77a8100ff0 2024-02-02T15:30:45.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.009+0000 7f77afe00700 1 -- 172.21.15.1:0/1358569212 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77a81a6160 con 0x7f77a8100ff0 2024-02-02T15:30:45.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.012+0000 7f77a6000700 1 -- 172.21.15.1:0/1358569212 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 27) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f77980225e0 con 0x7f77a8100ff0 2024-02-02T15:30:45.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.014+0000 7f77a6000700 1 --2- 172.21.15.1:0/1358569212 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f77880a0710 0x7f77880a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:45.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.014+0000 7f77a6000700 1 -- 172.21.15.1:0/1358569212 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f77980cd4e0 con 0x7f77a8100ff0 2024-02-02T15:30:45.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.014+0000 7f77c38ceb80 1 Processor -- start 2024-02-02T15:30:45.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.014+0000 7f77aca00700 1 --2- 172.21.15.1:0/1358569212 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f77880a0710 0x7f77880a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:45.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.014+0000 7f77c38ceb80 1 -- start start 2024-02-02T15:30:45.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.015+0000 7f77c38ceb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b7ba908010 0x55b7ba9083f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:45.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.015+0000 7f77c38ceb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b7ba908930 0x55b7ba9132a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:45.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.015+0000 7f77c38ceb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b7ba9137e0 0x55b7ba915bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:45.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.015+0000 7f77c38ceb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7ba87c4c0 con 0x55b7ba908930 2024-02-02T15:30:45.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.015+0000 7f77c38ceb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7ba860aa0 con 0x55b7ba908010 2024-02-02T15:30:45.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.015+0000 7f77c38ceb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7ba85e460 con 0x55b7ba9137e0 2024-02-02T15:30:45.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.015+0000 7f77ad400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b7ba908010 0x55b7ba9083f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:45.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.015+0000 7f77ad400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b7ba908010 0x55b7ba9083f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:38682/0 (socket says 172.21.15.1:38682) 2024-02-02T15:30:45.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.015+0000 7f77ad400700 1 -- 172.21.15.1:0/53438132 learned_addr learned my addr 172.21.15.1:0/53438132 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:30:45.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.016+0000 7f77aca00700 1 --2- 172.21.15.1:0/1358569212 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f77880a0710 0x7f77880a2bd0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f77a819a750 tx=0x7f7790009260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:45.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.016+0000 7f77aca00700 1 --2- 172.21.15.1:0/53438132 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b7ba908930 0x55b7ba9132a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:45.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.016+0000 7f77ade00700 1 --2- 172.21.15.1:0/53438132 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b7ba9137e0 0x55b7ba915bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:45.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.016+0000 7f77ad400700 1 -- 172.21.15.1:0/53438132 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b7ba9137e0 msgr2=0x55b7ba915bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:45.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.016+0000 7f77ad400700 1 --2- 172.21.15.1:0/53438132 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b7ba9137e0 0x55b7ba915bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.016+0000 7f77ad400700 1 -- 172.21.15.1:0/53438132 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b7ba908930 msgr2=0x55b7ba9132a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:45.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.016+0000 7f77ad400700 1 --2- 172.21.15.1:0/53438132 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b7ba908930 0x55b7ba9132a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.016+0000 7f77ad400700 1 -- 172.21.15.1:0/53438132 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b7ba6db3b0 con 0x55b7ba908010 2024-02-02T15:30:45.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.016+0000 7f77aca00700 1 --2- 172.21.15.1:0/53438132 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b7ba908930 0x55b7ba9132a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:30:45.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.016+0000 7f77ad400700 1 --2- 172.21.15.1:0/53438132 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b7ba908010 0x55b7ba9083f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7798097150 tx=0x7f7798096b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21cc42d0cb3ca7bc server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:45.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.016+0000 7f779e600700 1 -- 172.21.15.1:0/53438132 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f779809ade0 con 0x55b7ba908010 2024-02-02T15:30:45.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.016+0000 7f779e600700 1 -- 172.21.15.1:0/53438132 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f779809a6e0 con 0x55b7ba908010 2024-02-02T15:30:45.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.016+0000 7f779e600700 1 -- 172.21.15.1:0/53438132 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f779809a940 con 0x55b7ba908010 2024-02-02T15:30:45.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.018+0000 7f77c38ceb80 1 -- 172.21.15.1:0/53438132 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b7ba908010 msgr2=0x55b7ba9083f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:45.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.018+0000 7f77c38ceb80 1 --2- 172.21.15.1:0/53438132 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b7ba908010 0x55b7ba9083f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7798097150 tx=0x7f7798096b50 comp rx=0 tx=0).stop 2024-02-02T15:30:45.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.018+0000 7f77c38ceb80 1 -- 172.21.15.1:0/53438132 shutdown_connections 2024-02-02T15:30:45.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.018+0000 7f77c38ceb80 1 --2- 172.21.15.1:0/53438132 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b7ba9137e0 0x55b7ba915bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.018+0000 7f77c38ceb80 1 --2- 172.21.15.1:0/53438132 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b7ba908010 0x55b7ba9083f0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.018+0000 7f77c38ceb80 1 --2- 172.21.15.1:0/53438132 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b7ba908930 0x55b7ba9132a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.018+0000 7f77c38ceb80 1 -- 172.21.15.1:0/53438132 >> 172.21.15.1:0/53438132 conn(0x55b7ba867db0 msgr2=0x55b7ba9043b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:45.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.018+0000 7f77c38ceb80 1 -- 172.21.15.1:0/53438132 shutdown_connections 2024-02-02T15:30:45.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.018+0000 7f77c38ceb80 1 -- 172.21.15.1:0/53438132 wait complete. 2024-02-02T15:30:45.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.019+0000 7f77c38ceb80 1 -- start start 2024-02-02T15:30:45.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.019+0000 7f77c38ceb80 1 Processor -- start 2024-02-02T15:30:45.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.019+0000 7f77c38ceb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b7ba908010 0x55b7ba918940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:45.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.019+0000 7f77c38ceb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b7ba908930 0x55b7ba918e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:45.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.019+0000 7f77c38ceb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b7ba9137e0 0x55b7ba9127b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:45.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.019+0000 7f77c38ceb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7ba859d50 con 0x55b7ba908930 2024-02-02T15:30:45.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.019+0000 7f77c38ceb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7ba7f5d80 con 0x55b7ba9137e0 2024-02-02T15:30:45.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.019+0000 7f77c38ceb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7ba7f5a20 con 0x55b7ba908010 2024-02-02T15:30:45.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.020+0000 7f77c38ceb80 10 client.0 _ll_register_callbacks cb 0x55b7ba7f7290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:30:45.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.020+0000 7f77ad400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b7ba908010 0x55b7ba918940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:45.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.020+0000 7f77ad400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b7ba908010 0x55b7ba918940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:41016/0 (socket says 172.21.15.1:41016) 2024-02-02T15:30:45.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.020+0000 7f77ad400700 1 -- 172.21.15.1:0/2987810105 learned_addr learned my addr 172.21.15.1:0/2987810105 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:30:45.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.020+0000 7f77ade00700 1 --2- 172.21.15.1:0/2987810105 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b7ba9137e0 0x55b7ba9127b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:45.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.020+0000 7f77aca00700 1 --2- 172.21.15.1:0/2987810105 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b7ba908930 0x55b7ba918e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:45.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.020+0000 7f77ad400700 1 -- 172.21.15.1:0/2987810105 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b7ba9137e0 msgr2=0x55b7ba9127b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:45.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.020+0000 7f77ad400700 1 --2- 172.21.15.1:0/2987810105 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b7ba9137e0 0x55b7ba9127b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.020+0000 7f77ad400700 1 -- 172.21.15.1:0/2987810105 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b7ba908930 msgr2=0x55b7ba918e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:45.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.020+0000 7f77ad400700 1 --2- 172.21.15.1:0/2987810105 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b7ba908930 0x55b7ba918e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.020+0000 7f77ad400700 1 -- 172.21.15.1:0/2987810105 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77a81a6160 con 0x55b7ba908010 2024-02-02T15:30:45.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.021+0000 7f77ad400700 1 --2- 172.21.15.1:0/2987810105 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b7ba908010 0x55b7ba918940 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7798005f50 tx=0x7f7798009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=daca189aa905deb8 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:45.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.021+0000 7f7796a00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:30:45.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.022+0000 7f7796a00700 1 -- 172.21.15.1:0/2987810105 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77980a26c0 con 0x55b7ba908010 2024-02-02T15:30:45.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.022+0000 7f7796a00700 1 -- 172.21.15.1:0/2987810105 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f7798099de0 con 0x55b7ba908010 2024-02-02T15:30:45.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.022+0000 7f77c38ceb80 1 -- 172.21.15.1:0/2987810105 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b7ba8fc0f0 con 0x55b7ba908010 2024-02-02T15:30:45.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.022+0000 7f7796a00700 1 -- 172.21.15.1:0/2987810105 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f779809a940 con 0x55b7ba908010 2024-02-02T15:30:45.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.023+0000 7f77c38ceb80 10 client.26629 fetch_fsmap learned FSMap version 31 2024-02-02T15:30:45.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.023+0000 7f77c38ceb80 1 -- 172.21.15.1:0/2987810105 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55b7ba687b60 con 0x55b7ba908010 2024-02-02T15:30:45.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.024+0000 7f7796a00700 1 -- 172.21.15.1:0/2987810105 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b7ba8fc0f0 con 0x55b7ba908010 2024-02-02T15:30:45.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.024+0000 7f7796a00700 1 -- 172.21.15.1:0/2987810105 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 31) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f77980b8300 con 0x55b7ba908010 2024-02-02T15:30:45.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.025+0000 7f77c38ceb80 10 client.26629 fetch_fsmap finished waiting for FSMap version 31 2024-02-02T15:30:45.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.025+0000 7f77c38ceb80 10 client.26629 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-02-02T15:30:45.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.025+0000 7f77c38ceb80 20 client.26629 populate_metadata read hostname 'smithi001' 2024-02-02T15:30:45.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.025+0000 7f77c38ceb80 1 --2- 172.21.15.1:0/2987810105 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55b7ba922880 0x55b7ba924ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:45.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.025+0000 7f77c38ceb80 4 client.26629 mds_command: new command op to 24518 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:30:45.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.025+0000 7f77c38ceb80 1 -- 172.21.15.1:0/2987810105 --> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b7ba852dd0 con 0x55b7ba922880 2024-02-02T15:30:45.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.025+0000 7f77aca00700 1 --2- 172.21.15.1:0/2987810105 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55b7ba922880 0x55b7ba924ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:45.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.027+0000 7f77aca00700 1 --2- 172.21.15.1:0/2987810105 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55b7ba922880 0x55b7ba924ca0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4bcdd06cbb271937 server_cookie=9ddfdc89b092cb60 in_seq=0 out_seq=0 2024-02-02T15:30:45.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.027+0000 7f7796a00700 10 client.26629 ms_handle_connect on v2:172.21.15.71:6836/628016199 2024-02-02T15:30:45.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.029+0000 7f7796a00700 1 -- 172.21.15.1:0/2987810105 <== mds.2 v2:172.21.15.71:6836/628016199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55b7ba852dd0 con 0x55b7ba922880 2024-02-02T15:30:45.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.029+0000 7f7796a00700 10 client.26629 handle_command_reply: tid=0 2024-02-02T15:30:45.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.029+0000 7f77c38ceb80 1 client.26629 shutdown 2024-02-02T15:30:45.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.030+0000 7f77c38ceb80 1 -- 172.21.15.1:0/2987810105 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b7ba908010 msgr2=0x55b7ba918940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:45.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.030+0000 7f77c38ceb80 1 --2- 172.21.15.1:0/2987810105 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b7ba908010 0x55b7ba918940 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7798005f50 tx=0x7f7798009ca0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.030+0000 7f77c38ceb80 1 -- 172.21.15.1:0/2987810105 shutdown_connections 2024-02-02T15:30:45.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.030+0000 7f77c38ceb80 1 --2- 172.21.15.1:0/2987810105 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b7ba9137e0 0x55b7ba9127b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.030+0000 7f77c38ceb80 1 --2- 172.21.15.1:0/2987810105 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55b7ba922880 0x55b7ba924ca0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.030+0000 7f7796a00700 0 client.26629 ms_handle_reset on v2:172.21.15.71:6836/628016199 2024-02-02T15:30:45.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.030+0000 7f77c38ceb80 1 --2- 172.21.15.1:0/2987810105 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b7ba908010 0x55b7ba918940 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.030+0000 7f77c38ceb80 1 --2- 172.21.15.1:0/2987810105 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b7ba908930 0x55b7ba918e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.030+0000 7f77c38ceb80 1 -- 172.21.15.1:0/2987810105 >> 172.21.15.1:0/2987810105 conn(0x55b7ba867db0 msgr2=0x55b7ba903fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:45.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.031+0000 7f77c38ceb80 1 -- 172.21.15.1:0/2987810105 shutdown_connections 2024-02-02T15:30:45.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.031+0000 7f77c38ceb80 1 -- 172.21.15.1:0/2987810105 wait complete. 2024-02-02T15:30:45.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.031+0000 7f77c38ceb80 20 client.26629 trim_cache size 0 max 16384 2024-02-02T15:30:45.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.055+0000 7f77c38ceb80 1 Processor -- start 2024-02-02T15:30:45.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.055+0000 7f77c38ceb80 1 -- start start 2024-02-02T15:30:45.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.055+0000 7f77c38ceb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b7ba908010 0x55b7ba912050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:45.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.055+0000 7f77c38ceb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b7ba908930 0x55b7ba912590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:45.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.055+0000 7f77c38ceb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b7ba9137e0 0x55b7ba9184d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:45.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.055+0000 7f77c38ceb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7ba6dd6a0 con 0x55b7ba9137e0 2024-02-02T15:30:45.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.055+0000 7f77c38ceb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7ba9031a0 con 0x55b7ba908010 2024-02-02T15:30:45.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.055+0000 7f77c38ceb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7ba90e620 con 0x55b7ba908930 2024-02-02T15:30:45.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.055+0000 7f77aca00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b7ba908930 0x55b7ba912590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:45.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.056+0000 7f77aca00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b7ba908930 0x55b7ba912590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:41024/0 (socket says 172.21.15.1:41024) 2024-02-02T15:30:45.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.056+0000 7f77aca00700 1 -- 172.21.15.1:0/1529053946 learned_addr learned my addr 172.21.15.1:0/1529053946 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:30:45.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.056+0000 7f77ade00700 1 --2- 172.21.15.1:0/1529053946 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b7ba9137e0 0x55b7ba9184d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:45.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.056+0000 7f77ad400700 1 --2- 172.21.15.1:0/1529053946 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b7ba908010 0x55b7ba912050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:45.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.056+0000 7f77aca00700 1 -- 172.21.15.1:0/1529053946 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b7ba908010 msgr2=0x55b7ba912050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:45.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.056+0000 7f77aca00700 1 --2- 172.21.15.1:0/1529053946 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b7ba908010 0x55b7ba912050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.056+0000 7f77aca00700 1 -- 172.21.15.1:0/1529053946 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b7ba9137e0 msgr2=0x55b7ba9184d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:45.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.056+0000 7f77aca00700 1 --2- 172.21.15.1:0/1529053946 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b7ba9137e0 0x55b7ba9184d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.056+0000 7f77aca00700 1 -- 172.21.15.1:0/1529053946 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b7ba81fea0 con 0x55b7ba908930 2024-02-02T15:30:45.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.056+0000 7f77aca00700 1 --2- 172.21.15.1:0/1529053946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b7ba908930 0x55b7ba912590 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7790005950 tx=0x7f7790014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5790422821920bd server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:45.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.056+0000 7f779f000700 1 -- 172.21.15.1:0/1529053946 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7790025070 con 0x55b7ba908930 2024-02-02T15:30:45.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.056+0000 7f779f000700 1 -- 172.21.15.1:0/1529053946 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7790020110 con 0x55b7ba908930 2024-02-02T15:30:45.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.057+0000 7f779f000700 1 -- 172.21.15.1:0/1529053946 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7790004390 con 0x55b7ba908930 2024-02-02T15:30:45.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.057+0000 7f77c38ceb80 1 -- 172.21.15.1:0/1529053946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b7ba908930 msgr2=0x55b7ba912590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:45.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.057+0000 7f77c38ceb80 1 --2- 172.21.15.1:0/1529053946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b7ba908930 0x55b7ba912590 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7790005950 tx=0x7f7790014000 comp rx=0 tx=0).stop 2024-02-02T15:30:45.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.057+0000 7f77c38ceb80 1 -- 172.21.15.1:0/1529053946 shutdown_connections 2024-02-02T15:30:45.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.057+0000 7f77c38ceb80 1 --2- 172.21.15.1:0/1529053946 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b7ba9137e0 0x55b7ba9184d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.057+0000 7f77c38ceb80 1 --2- 172.21.15.1:0/1529053946 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b7ba908010 0x55b7ba912050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.057+0000 7f77c38ceb80 1 --2- 172.21.15.1:0/1529053946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b7ba908930 0x55b7ba912590 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.057+0000 7f77c38ceb80 1 -- 172.21.15.1:0/1529053946 >> 172.21.15.1:0/1529053946 conn(0x55b7ba867db0 msgr2=0x55b7ba903eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:45.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.057+0000 7f77c38ceb80 1 -- 172.21.15.1:0/1529053946 shutdown_connections 2024-02-02T15:30:45.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.058+0000 7f77c38ceb80 1 -- 172.21.15.1:0/1529053946 wait complete. 2024-02-02T15:30:45.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.058+0000 7f77c38ceb80 1 -- start start 2024-02-02T15:30:45.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.058+0000 7f77c38ceb80 1 Processor -- start 2024-02-02T15:30:45.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.059+0000 7f77c38ceb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b7ba908010 0x55b7ba968520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:45.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.059+0000 7f77c38ceb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b7ba908930 0x55b7ba968a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:45.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.059+0000 7f77c38ceb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b7ba9137e0 0x55b7ba96d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:45.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.059+0000 7f77c38ceb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7ba96d930 con 0x55b7ba9137e0 2024-02-02T15:30:45.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.059+0000 7f77c38ceb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7ba96daa0 con 0x55b7ba908930 2024-02-02T15:30:45.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.059+0000 7f77c38ceb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7ba96dc60 con 0x55b7ba908010 2024-02-02T15:30:45.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.059+0000 7f77c38ceb80 10 client.0 _ll_register_callbacks cb 0x55b7ba7f7290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:30:45.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.059+0000 7f77aca00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b7ba908930 0x55b7ba968a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:45.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.059+0000 7f77ad400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b7ba908010 0x55b7ba968520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:45.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.059+0000 7f77aca00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b7ba908930 0x55b7ba968a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:38710/0 (socket says 172.21.15.1:38710) 2024-02-02T15:30:45.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.059+0000 7f77aca00700 1 -- 172.21.15.1:0/449378988 learned_addr learned my addr 172.21.15.1:0/449378988 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:30:45.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.059+0000 7f77ade00700 1 --2- 172.21.15.1:0/449378988 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b7ba9137e0 0x55b7ba96d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:45.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.059+0000 7f77aca00700 1 -- 172.21.15.1:0/449378988 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b7ba908010 msgr2=0x55b7ba968520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:45.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.059+0000 7f77aca00700 1 --2- 172.21.15.1:0/449378988 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b7ba908010 0x55b7ba968520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.059+0000 7f77aca00700 1 -- 172.21.15.1:0/449378988 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b7ba9137e0 msgr2=0x55b7ba96d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:45.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.059+0000 7f77aca00700 1 --2- 172.21.15.1:0/449378988 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b7ba9137e0 0x55b7ba96d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.059+0000 7f77aca00700 1 -- 172.21.15.1:0/449378988 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7790017070 con 0x55b7ba908930 2024-02-02T15:30:45.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.059+0000 7f77aca00700 1 --2- 172.21.15.1:0/449378988 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b7ba908930 0x55b7ba968a60 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7790029040 tx=0x7f77900260e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a83d6f6b11c8afb1 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:45.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.059+0000 7f7796a00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:30:45.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.060+0000 7f7796a00700 1 -- 172.21.15.1:0/449378988 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7790025040 con 0x55b7ba908930 2024-02-02T15:30:45.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.060+0000 7f7796a00700 1 -- 172.21.15.1:0/449378988 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f7790004370 con 0x55b7ba908930 2024-02-02T15:30:45.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.060+0000 7f77c38ceb80 1 -- 172.21.15.1:0/449378988 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f779809a6e0 con 0x55b7ba908930 2024-02-02T15:30:45.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.060+0000 7f7796a00700 1 -- 172.21.15.1:0/449378988 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7790004530 con 0x55b7ba908930 2024-02-02T15:30:45.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.060+0000 7f77c38ceb80 10 client.26645 fetch_fsmap learned FSMap version 31 2024-02-02T15:30:45.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.060+0000 7f77c38ceb80 1 -- 172.21.15.1:0/449378988 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55b7ba6dfcd0 con 0x55b7ba908930 2024-02-02T15:30:45.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.060+0000 7f7796a00700 1 -- 172.21.15.1:0/449378988 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f779809a6e0 con 0x55b7ba908930 2024-02-02T15:30:45.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.060+0000 7f7796a00700 1 -- 172.21.15.1:0/449378988 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 31) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f779002d070 con 0x55b7ba908930 2024-02-02T15:30:45.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.060+0000 7f77c38ceb80 10 client.26645 fetch_fsmap finished waiting for FSMap version 31 2024-02-02T15:30:45.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.060+0000 7f77c38ceb80 10 client.26645 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-02-02T15:30:45.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.060+0000 7f77c38ceb80 20 client.26645 populate_metadata read hostname 'smithi001' 2024-02-02T15:30:45.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.060+0000 7f77c38ceb80 1 --2- 172.21.15.1:0/449378988 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55b7ba9753f0 0x55b7ba977810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:45.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.060+0000 7f77c38ceb80 4 client.26645 mds_command: new command op to 24518 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:30:45.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.060+0000 7f77c38ceb80 1 -- 172.21.15.1:0/449378988 --> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b7ba818f50 con 0x55b7ba9753f0 2024-02-02T15:30:45.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.061+0000 7f77ad400700 1 --2- 172.21.15.1:0/449378988 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55b7ba9753f0 0x55b7ba977810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:45.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.061+0000 7f77ad400700 1 --2- 172.21.15.1:0/449378988 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55b7ba9753f0 0x55b7ba977810 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=842f2a6b5859570f server_cookie=ab61f6952b82430e in_seq=0 out_seq=0 2024-02-02T15:30:45.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.061+0000 7f7796a00700 10 client.26645 ms_handle_connect on v2:172.21.15.71:6836/628016199 2024-02-02T15:30:45.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.061+0000 7f7796a00700 1 -- 172.21.15.1:0/449378988 <== mds.2 v2:172.21.15.71:6836/628016199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b7ba818f50 con 0x55b7ba9753f0 2024-02-02T15:30:45.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.061+0000 7f7796a00700 10 client.26645 handle_command_reply: tid=0 2024-02-02T15:30:45.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.061+0000 7f77c38ceb80 1 client.26645 shutdown 2024-02-02T15:30:45.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.062+0000 7f77c38ceb80 1 -- 172.21.15.1:0/449378988 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b7ba908930 msgr2=0x55b7ba968a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:45.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.062+0000 7f77c38ceb80 1 --2- 172.21.15.1:0/449378988 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b7ba908930 0x55b7ba968a60 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7790029040 tx=0x7f77900260e0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.062+0000 7f77c38ceb80 1 -- 172.21.15.1:0/449378988 shutdown_connections 2024-02-02T15:30:45.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.062+0000 7f77c38ceb80 1 --2- 172.21.15.1:0/449378988 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b7ba9137e0 0x55b7ba96d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.062+0000 7f77c38ceb80 1 --2- 172.21.15.1:0/449378988 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55b7ba9753f0 0x55b7ba977810 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.062+0000 7f77c38ceb80 1 --2- 172.21.15.1:0/449378988 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b7ba908010 0x55b7ba968520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.062+0000 7f77c38ceb80 1 --2- 172.21.15.1:0/449378988 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b7ba908930 0x55b7ba968a60 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.062+0000 7f7796a00700 0 client.26645 ms_handle_reset on v2:172.21.15.71:6836/628016199 2024-02-02T15:30:45.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.062+0000 7f77c38ceb80 1 -- 172.21.15.1:0/449378988 >> 172.21.15.1:0/449378988 conn(0x55b7ba867db0 msgr2=0x55b7ba912020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:45.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.062+0000 7f77c38ceb80 1 -- 172.21.15.1:0/449378988 shutdown_connections 2024-02-02T15:30:45.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.062+0000 7f77c38ceb80 1 -- 172.21.15.1:0/449378988 wait complete. 2024-02-02T15:30:45.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.062+0000 7f77c38ceb80 20 client.26645 trim_cache size 0 max 16384 2024-02-02T15:30:45.065 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:30:45.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.062+0000 7f77afe00700 1 -- 172.21.15.1:0/1358569212 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f77880a0710 msgr2=0x7f77880a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:45.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.063+0000 7f77afe00700 1 --2- 172.21.15.1:0/1358569212 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f77880a0710 0x7f77880a2bd0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f77a819a750 tx=0x7f7790009260 comp rx=0 tx=0).stop 2024-02-02T15:30:45.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.063+0000 7f77afe00700 1 -- 172.21.15.1:0/1358569212 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77a8100ff0 msgr2=0x7f77a8199140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:45.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.063+0000 7f77afe00700 1 --2- 172.21.15.1:0/1358569212 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77a8100ff0 0x7f77a8199140 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7798005910 tx=0x7f7798004730 comp rx=0 tx=0).stop 2024-02-02T15:30:45.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.063+0000 7f77afe00700 1 -- 172.21.15.1:0/1358569212 shutdown_connections 2024-02-02T15:30:45.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.063+0000 7f77afe00700 1 --2- 172.21.15.1:0/1358569212 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f77880a0710 0x7f77880a2bd0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.063+0000 7f77afe00700 1 --2- 172.21.15.1:0/1358569212 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f77a819dae0 0x7f77a819df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.063+0000 7f77afe00700 1 --2- 172.21.15.1:0/1358569212 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f77a8100ff0 0x7f77a8199140 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.063+0000 7f77afe00700 1 --2- 172.21.15.1:0/1358569212 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77a8106110 0x7f77a8199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.063+0000 7f77afe00700 1 -- 172.21.15.1:0/1358569212 >> 172.21.15.1:0/1358569212 conn(0x7f77a80fc810 msgr2=0x7f77a80fd340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:45.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.063+0000 7f77afe00700 1 -- 172.21.15.1:0/1358569212 shutdown_connections 2024-02-02T15:30:45.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.063+0000 7f77afe00700 1 -- 172.21.15.1:0/1358569212 wait complete. 2024-02-02T15:30:45.080 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-02T15:30:45.228 INFO:tasks.workunit.client.0.smithi001.stdout: 1 174281 9.69 MB/sec execute 19 sec latency 32.498 ms 2024-02-02T15:30:45.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.294+0000 7f55d6e00700 1 -- 172.21.15.1:0/3989166920 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55d0103230 msgr2=0x7f55d0103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:45.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.294+0000 7f55d6e00700 1 --2- 172.21.15.1:0/3989166920 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55d0103230 0x7f55d0103610 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f55c0009990 tx=0x7f55c0009ca0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.294+0000 7f55d6e00700 1 -- 172.21.15.1:0/3989166920 shutdown_connections 2024-02-02T15:30:45.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.294+0000 7f55d6e00700 1 --2- 172.21.15.1:0/3989166920 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f55d0108350 0x7f55d010fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.294+0000 7f55d6e00700 1 --2- 172.21.15.1:0/3989166920 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55d0103230 0x7f55d0103610 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.294+0000 7f55d6e00700 1 --2- 172.21.15.1:0/3989166920 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f55d0103be0 0x7f55d0107c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.294+0000 7f55d6e00700 1 -- 172.21.15.1:0/3989166920 >> 172.21.15.1:0/3989166920 conn(0x7f55d00fea70 msgr2=0x7f55d0100e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:45.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.295+0000 7f55d6e00700 1 -- 172.21.15.1:0/3989166920 shutdown_connections 2024-02-02T15:30:45.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.295+0000 7f55d6e00700 1 -- 172.21.15.1:0/3989166920 wait complete. 2024-02-02T15:30:45.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.296+0000 7f55d6e00700 1 Processor -- start 2024-02-02T15:30:45.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.296+0000 7f55d6e00700 1 -- start start 2024-02-02T15:30:45.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.296+0000 7f55d6e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f55d0103230 0x7f55d01990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:45.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.296+0000 7f55d6e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55d0108350 0x7f55d0199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:45.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.296+0000 7f55d6e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f55d019da60 0x7f55d019dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:45.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.296+0000 7f55d6e00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55d019e4e0 con 0x7f55d0103230 2024-02-02T15:30:45.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.296+0000 7f55d6e00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55d019e650 con 0x7f55d019da60 2024-02-02T15:30:45.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.296+0000 7f55d6e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55d019e810 con 0x7f55d0108350 2024-02-02T15:30:45.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.297+0000 7f55cfe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f55d0103230 0x7f55d01990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:45.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.297+0000 7f55cf400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55d0108350 0x7f55d0199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:45.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.297+0000 7f55cfe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f55d0103230 0x7f55d01990c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45160/0 (socket says 172.21.15.1:45160) 2024-02-02T15:30:45.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.297+0000 7f55cfe00700 1 -- 172.21.15.1:0/2472193950 learned_addr learned my addr 172.21.15.1:0/2472193950 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:30:45.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.297+0000 7f55d4e00700 1 --2- 172.21.15.1:0/2472193950 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f55d019da60 0x7f55d019dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:45.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.297+0000 7f55cfe00700 1 -- 172.21.15.1:0/2472193950 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55d0108350 msgr2=0x7f55d0199600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:45.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.297+0000 7f55cfe00700 1 --2- 172.21.15.1:0/2472193950 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55d0108350 0x7f55d0199600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.297+0000 7f55cfe00700 1 -- 172.21.15.1:0/2472193950 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f55d019da60 msgr2=0x7f55d019dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:45.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.297+0000 7f55cfe00700 1 --2- 172.21.15.1:0/2472193950 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f55d019da60 0x7f55d019dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.297+0000 7f55cfe00700 1 -- 172.21.15.1:0/2472193950 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55b8009520 con 0x7f55d0103230 2024-02-02T15:30:45.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.297+0000 7f55d4e00700 1 --2- 172.21.15.1:0/2472193950 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f55d019da60 0x7f55d019dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:30:45.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.297+0000 7f55cfe00700 1 --2- 172.21.15.1:0/2472193950 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f55d0103230 0x7f55d01990c0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f55c0005910 tx=0x7f55c0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:45.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.298+0000 7f55ccc00700 1 -- 172.21.15.1:0/2472193950 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c001d070 con 0x7f55d0103230 2024-02-02T15:30:45.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.298+0000 7f55d6e00700 1 -- 172.21.15.1:0/2472193950 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55c00095f0 con 0x7f55d0103230 2024-02-02T15:30:45.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.298+0000 7f55ccc00700 1 -- 172.21.15.1:0/2472193950 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f55c0022470 con 0x7f55d0103230 2024-02-02T15:30:45.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.298+0000 7f55ccc00700 1 -- 172.21.15.1:0/2472193950 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c0017650 con 0x7f55d0103230 2024-02-02T15:30:45.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.298+0000 7f55d6e00700 1 -- 172.21.15.1:0/2472193950 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55d01a6000 con 0x7f55d0103230 2024-02-02T15:30:45.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.301+0000 7f55ccc00700 1 -- 172.21.15.1:0/2472193950 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 27) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f55c00225e0 con 0x7f55d0103230 2024-02-02T15:30:45.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.302+0000 7f55ea971b80 1 Processor -- start 2024-02-02T15:30:45.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.302+0000 7f55ccc00700 1 --2- 172.21.15.1:0/2472193950 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f55b00a0710 0x7f55b00a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:45.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.302+0000 7f55ea971b80 1 -- start start 2024-02-02T15:30:45.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.302+0000 7f55ccc00700 1 -- 172.21.15.1:0/2472193950 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f55c00cd250 con 0x7f55d0103230 2024-02-02T15:30:45.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.302+0000 7f55cf400700 1 --2- 172.21.15.1:0/2472193950 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f55b00a0710 0x7f55b00a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:45.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.303+0000 7f55ea971b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56317d9c9010 0x56317d9c93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:45.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.303+0000 7f55ea971b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56317d9c9930 0x56317d9d42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:45.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.303+0000 7f55ea971b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56317d9d47e0 0x56317d9d6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:45.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.303+0000 7f55ea971b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56317d93d4c0 con 0x56317d9d47e0 2024-02-02T15:30:45.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.303+0000 7f55ea971b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56317d921aa0 con 0x56317d9c9930 2024-02-02T15:30:45.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.303+0000 7f55ea971b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56317d91f460 con 0x56317d9c9010 2024-02-02T15:30:45.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.303+0000 7f55cfe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56317d9c9010 0x56317d9c93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:45.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.303+0000 7f55cfe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56317d9c9010 0x56317d9c93f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:41054/0 (socket says 172.21.15.1:41054) 2024-02-02T15:30:45.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.303+0000 7f55cfe00700 1 -- 172.21.15.1:0/916347526 learned_addr learned my addr 172.21.15.1:0/916347526 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:30:45.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.303+0000 7f55d4e00700 1 --2- 172.21.15.1:0/916347526 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56317d9d47e0 0x56317d9d6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:45.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.303+0000 7f55cf400700 1 --2- 172.21.15.1:0/916347526 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56317d9c9930 0x56317d9d42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:45.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.303+0000 7f55cf400700 1 --2- 172.21.15.1:0/2472193950 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f55b00a0710 0x7f55b00a2bd0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f55d019a6d0 tx=0x7f55b8009260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:45.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.303+0000 7f55cfe00700 1 -- 172.21.15.1:0/916347526 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56317d9c9930 msgr2=0x56317d9d42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:45.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.303+0000 7f55cfe00700 1 --2- 172.21.15.1:0/916347526 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56317d9c9930 0x56317d9d42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.303+0000 7f55cfe00700 1 -- 172.21.15.1:0/916347526 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56317d9d47e0 msgr2=0x56317d9d6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:45.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.303+0000 7f55cfe00700 1 --2- 172.21.15.1:0/916347526 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56317d9d47e0 0x56317d9d6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.303+0000 7f55cfe00700 1 -- 172.21.15.1:0/916347526 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56317d79c3b0 con 0x56317d9c9010 2024-02-02T15:30:45.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.304+0000 7f55cfe00700 1 --2- 172.21.15.1:0/916347526 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56317d9c9010 0x56317d9c93f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f55c0096ec0 tx=0x7f55c009a2a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da93b852c60931b0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:45.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.304+0000 7f55c5600700 1 -- 172.21.15.1:0/916347526 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c00850e0 con 0x56317d9c9010 2024-02-02T15:30:45.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.304+0000 7f55c5600700 1 -- 172.21.15.1:0/916347526 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f55c009ab50 con 0x56317d9c9010 2024-02-02T15:30:45.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.304+0000 7f55c5600700 1 -- 172.21.15.1:0/916347526 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c009a6b0 con 0x56317d9c9010 2024-02-02T15:30:45.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.305+0000 7f55ea971b80 1 -- 172.21.15.1:0/916347526 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56317d9c9010 msgr2=0x56317d9c93f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:45.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.305+0000 7f55ea971b80 1 --2- 172.21.15.1:0/916347526 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56317d9c9010 0x56317d9c93f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f55c0096ec0 tx=0x7f55c009a2a0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.305+0000 7f55ea971b80 1 -- 172.21.15.1:0/916347526 shutdown_connections 2024-02-02T15:30:45.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.305+0000 7f55ea971b80 1 --2- 172.21.15.1:0/916347526 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56317d9d47e0 0x56317d9d6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.305+0000 7f55ea971b80 1 --2- 172.21.15.1:0/916347526 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56317d9c9010 0x56317d9c93f0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.305+0000 7f55ea971b80 1 --2- 172.21.15.1:0/916347526 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56317d9c9930 0x56317d9d42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.305+0000 7f55ea971b80 1 -- 172.21.15.1:0/916347526 >> 172.21.15.1:0/916347526 conn(0x56317d928db0 msgr2=0x56317d9c53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:45.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.305+0000 7f55ea971b80 1 -- 172.21.15.1:0/916347526 shutdown_connections 2024-02-02T15:30:45.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.305+0000 7f55ea971b80 1 -- 172.21.15.1:0/916347526 wait complete. 2024-02-02T15:30:45.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.305+0000 7f55ea971b80 1 -- start start 2024-02-02T15:30:45.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.306+0000 7f55ea971b80 1 Processor -- start 2024-02-02T15:30:45.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.306+0000 7f55ea971b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56317d9c9010 0x56317d9d4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:45.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.306+0000 7f55ea971b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56317d9c9930 0x56317d9d5100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:45.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.306+0000 7f55ea971b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56317d9d47e0 0x56317d9d5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:45.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.306+0000 7f55ea971b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56317d91ad50 con 0x56317d9d47e0 2024-02-02T15:30:45.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.306+0000 7f55ea971b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56317d8b6d80 con 0x56317d9c9010 2024-02-02T15:30:45.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.306+0000 7f55ea971b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56317d8b6a20 con 0x56317d9c9930 2024-02-02T15:30:45.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.306+0000 7f55ea971b80 10 client.0 _ll_register_callbacks cb 0x56317d8b8290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:30:45.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.306+0000 7f55cfe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56317d9c9010 0x56317d9d4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:45.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.306+0000 7f55d4e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56317d9d47e0 0x56317d9d5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:45.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.306+0000 7f55cfe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56317d9c9010 0x56317d9d4bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:38748/0 (socket says 172.21.15.1:38748) 2024-02-02T15:30:45.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.306+0000 7f55cfe00700 1 -- 172.21.15.1:0/3871875225 learned_addr learned my addr 172.21.15.1:0/3871875225 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:30:45.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.307+0000 7f55cf400700 1 --2- 172.21.15.1:0/3871875225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56317d9c9930 0x56317d9d5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:45.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.307+0000 7f55cfe00700 1 -- 172.21.15.1:0/3871875225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56317d9c9930 msgr2=0x56317d9d5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:45.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.307+0000 7f55cfe00700 1 --2- 172.21.15.1:0/3871875225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56317d9c9930 0x56317d9d5100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.307+0000 7f55cfe00700 1 -- 172.21.15.1:0/3871875225 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56317d9d47e0 msgr2=0x56317d9d5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:45.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.307+0000 7f55cfe00700 1 --2- 172.21.15.1:0/3871875225 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56317d9d47e0 0x56317d9d5640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.307+0000 7f55cfe00700 1 -- 172.21.15.1:0/3871875225 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55d01a6000 con 0x56317d9c9010 2024-02-02T15:30:45.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.307+0000 7f55cfe00700 1 --2- 172.21.15.1:0/3871875225 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56317d9c9010 0x56317d9d4bc0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f55c0005f50 tx=0x7f55c0022af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b265c817a6e6607 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:45.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.307+0000 7f55be000700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:30:45.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.307+0000 7f55be000700 1 -- 172.21.15.1:0/3871875225 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c009a450 con 0x56317d9c9010 2024-02-02T15:30:45.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.307+0000 7f55be000700 1 -- 172.21.15.1:0/3871875225 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f55c009e280 con 0x56317d9c9010 2024-02-02T15:30:45.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.307+0000 7f55ea971b80 1 -- 172.21.15.1:0/3871875225 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56317d9bd0f0 con 0x56317d9c9010 2024-02-02T15:30:45.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.308+0000 7f55be000700 1 -- 172.21.15.1:0/3871875225 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f55c009a8c0 con 0x56317d9c9010 2024-02-02T15:30:45.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.308+0000 7f55ea971b80 10 client.26657 fetch_fsmap learned FSMap version 31 2024-02-02T15:30:45.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.308+0000 7f55ea971b80 1 -- 172.21.15.1:0/3871875225 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x56317d748b60 con 0x56317d9c9010 2024-02-02T15:30:45.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.308+0000 7f55be000700 1 -- 172.21.15.1:0/3871875225 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f55c007f080 con 0x56317d9c9010 2024-02-02T15:30:45.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.308+0000 7f55be000700 1 -- 172.21.15.1:0/3871875225 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 31) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f55c00a00c0 con 0x56317d9c9010 2024-02-02T15:30:45.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.308+0000 7f55ea971b80 10 client.26657 fetch_fsmap finished waiting for FSMap version 31 2024-02-02T15:30:45.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.308+0000 7f55ea971b80 10 client.26657 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-02T15:30:45.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.308+0000 7f55ea971b80 20 client.26657 populate_metadata read hostname 'smithi001' 2024-02-02T15:30:45.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.308+0000 7f55ea971b80 1 --2- 172.21.15.1:0/3871875225 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x56317d9e3990 0x56317d9e5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:45.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.308+0000 7f55ea971b80 4 client.26657 mds_command: new command op to 24529 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:30:45.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.308+0000 7f55ea971b80 1 -- 172.21.15.1:0/3871875225 --> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56317d913dd0 con 0x56317d9e3990 2024-02-02T15:30:45.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.308+0000 7f55cf400700 1 --2- 172.21.15.1:0/3871875225 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x56317d9e3990 0x56317d9e5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:45.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.309+0000 7f55cf400700 1 --2- 172.21.15.1:0/3871875225 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x56317d9e3990 0x56317d9e5db0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c9378f3c185e7a3b server_cookie=164fb91f3ec6fa25 in_seq=0 out_seq=0 2024-02-02T15:30:45.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.309+0000 7f55be000700 10 client.26657 ms_handle_connect on v2:172.21.15.142:6838/321675421 2024-02-02T15:30:45.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.309+0000 7f55be000700 1 -- 172.21.15.1:0/3871875225 <== mds.1 v2:172.21.15.142:6838/321675421 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x56317d913dd0 con 0x56317d9e3990 2024-02-02T15:30:45.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.309+0000 7f55be000700 10 client.26657 handle_command_reply: tid=0 2024-02-02T15:30:45.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.310+0000 7f55ea971b80 1 client.26657 shutdown 2024-02-02T15:30:45.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.310+0000 7f55ea971b80 1 -- 172.21.15.1:0/3871875225 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56317d9c9010 msgr2=0x56317d9d4bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:45.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.310+0000 7f55ea971b80 1 --2- 172.21.15.1:0/3871875225 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56317d9c9010 0x56317d9d4bc0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f55c0005f50 tx=0x7f55c0022af0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.310+0000 7f55ea971b80 1 -- 172.21.15.1:0/3871875225 shutdown_connections 2024-02-02T15:30:45.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.310+0000 7f55ea971b80 1 --2- 172.21.15.1:0/3871875225 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x56317d9e3990 0x56317d9e5db0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.310+0000 7f55ea971b80 1 --2- 172.21.15.1:0/3871875225 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56317d9d47e0 0x56317d9d5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.310+0000 7f55ea971b80 1 --2- 172.21.15.1:0/3871875225 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56317d9c9010 0x56317d9d4bc0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.310+0000 7f55be000700 0 client.26657 ms_handle_reset on v2:172.21.15.142:6838/321675421 2024-02-02T15:30:45.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.310+0000 7f55ea971b80 1 --2- 172.21.15.1:0/3871875225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56317d9c9930 0x56317d9d5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.310+0000 7f55ea971b80 1 -- 172.21.15.1:0/3871875225 >> 172.21.15.1:0/3871875225 conn(0x56317d928db0 msgr2=0x56317d9c4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:45.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.310+0000 7f55ea971b80 1 -- 172.21.15.1:0/3871875225 shutdown_connections 2024-02-02T15:30:45.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.310+0000 7f55ea971b80 1 -- 172.21.15.1:0/3871875225 wait complete. 2024-02-02T15:30:45.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.310+0000 7f55ea971b80 20 client.26657 trim_cache size 0 max 16384 2024-02-02T15:30:45.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.324+0000 7f55ea971b80 1 Processor -- start 2024-02-02T15:30:45.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.325+0000 7f55ea971b80 1 -- start start 2024-02-02T15:30:45.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.325+0000 7f55ea971b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56317d9c9010 0x56317d9d8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:45.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.325+0000 7f55ea971b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56317d9c9930 0x56317d9ceb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:45.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.325+0000 7f55ea971b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56317d9d47e0 0x56317d9cf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:45.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.325+0000 7f55ea971b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56317d79e6a0 con 0x56317d9d47e0 2024-02-02T15:30:45.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.325+0000 7f55ea971b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56317d9bf500 con 0x56317d9c9930 2024-02-02T15:30:45.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.325+0000 7f55ea971b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56317d9d3eb0 con 0x56317d9c9010 2024-02-02T15:30:45.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.325+0000 7f55cfe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56317d9c9010 0x56317d9d8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:45.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.325+0000 7f55cf400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56317d9c9930 0x56317d9ceb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:45.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.325+0000 7f55cf400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56317d9c9930 0x56317d9ceb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:38760/0 (socket says 172.21.15.1:38760) 2024-02-02T15:30:45.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.325+0000 7f55cf400700 1 -- 172.21.15.1:0/4221379946 learned_addr learned my addr 172.21.15.1:0/4221379946 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:30:45.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.325+0000 7f55cfe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56317d9c9010 0x56317d9d8bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:41072/0 (socket says 172.21.15.1:41072) 2024-02-02T15:30:45.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.325+0000 7f55d4e00700 1 --2- 172.21.15.1:0/4221379946 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56317d9d47e0 0x56317d9cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:45.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.326+0000 7f55cfe00700 1 -- 172.21.15.1:0/4221379946 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56317d9c9930 msgr2=0x56317d9ceb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:45.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.326+0000 7f55cfe00700 1 --2- 172.21.15.1:0/4221379946 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56317d9c9930 0x56317d9ceb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.326+0000 7f55cfe00700 1 -- 172.21.15.1:0/4221379946 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56317d9d47e0 msgr2=0x56317d9cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:45.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.326+0000 7f55cfe00700 1 --2- 172.21.15.1:0/4221379946 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56317d9d47e0 0x56317d9cf050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.326+0000 7f55cfe00700 1 -- 172.21.15.1:0/4221379946 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56317d9c10d0 con 0x56317d9c9010 2024-02-02T15:30:45.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.326+0000 7f55cf400700 1 --2- 172.21.15.1:0/4221379946 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56317d9c9930 0x56317d9ceb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:30:45.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.326+0000 7f55cfe00700 1 --2- 172.21.15.1:0/4221379946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56317d9c9010 0x56317d9d8bf0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f55c009a150 tx=0x7f55c009acc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ca8790a7ea64c2 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:45.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.326+0000 7f55c6000700 1 -- 172.21.15.1:0/4221379946 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c0099450 con 0x56317d9c9010 2024-02-02T15:30:45.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.326+0000 7f55c6000700 1 -- 172.21.15.1:0/4221379946 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56317d9bd0f0 con 0x56317d9c9010 2024-02-02T15:30:45.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.326+0000 7f55c6000700 1 -- 172.21.15.1:0/4221379946 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c00988f0 con 0x56317d9c9010 2024-02-02T15:30:45.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.327+0000 7f55ea971b80 1 -- 172.21.15.1:0/4221379946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56317d9c9010 msgr2=0x56317d9d8bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:45.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.327+0000 7f55ea971b80 1 --2- 172.21.15.1:0/4221379946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56317d9c9010 0x56317d9d8bf0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f55c009a150 tx=0x7f55c009acc0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.327+0000 7f55ea971b80 1 -- 172.21.15.1:0/4221379946 shutdown_connections 2024-02-02T15:30:45.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.327+0000 7f55ea971b80 1 --2- 172.21.15.1:0/4221379946 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56317d9d47e0 0x56317d9cf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.327+0000 7f55ea971b80 1 --2- 172.21.15.1:0/4221379946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56317d9c9010 0x56317d9d8bf0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.327+0000 7f55ea971b80 1 --2- 172.21.15.1:0/4221379946 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56317d9c9930 0x56317d9ceb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.327+0000 7f55ea971b80 1 -- 172.21.15.1:0/4221379946 >> 172.21.15.1:0/4221379946 conn(0x56317d928db0 msgr2=0x56317d9e0180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:45.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.327+0000 7f55ea971b80 1 -- 172.21.15.1:0/4221379946 shutdown_connections 2024-02-02T15:30:45.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.327+0000 7f55ea971b80 1 -- 172.21.15.1:0/4221379946 wait complete. 2024-02-02T15:30:45.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.328+0000 7f55ea971b80 1 -- start start 2024-02-02T15:30:45.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.328+0000 7f55ea971b80 1 Processor -- start 2024-02-02T15:30:45.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.329+0000 7f55ea971b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56317d9c9010 0x56317d9d7f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:45.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.329+0000 7f55ea971b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56317d9c9930 0x56317d9d8490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:45.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.329+0000 7f55ea971b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56317d9d47e0 0x56317da2c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:45.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.329+0000 7f55ea971b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56317da2cdf0 con 0x56317d9c9930 2024-02-02T15:30:45.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.329+0000 7f55ea971b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56317da2cf60 con 0x56317d9d47e0 2024-02-02T15:30:45.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.329+0000 7f55ea971b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56317da2d120 con 0x56317d9c9010 2024-02-02T15:30:45.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.329+0000 7f55ea971b80 10 client.0 _ll_register_callbacks cb 0x56317d8b8290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:30:45.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.329+0000 7f55cfe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56317d9c9010 0x56317d9d7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:45.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.329+0000 7f55cf400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56317d9c9930 0x56317d9d8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:45.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.329+0000 7f55cfe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56317d9c9010 0x56317d9d7f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:41076/0 (socket says 172.21.15.1:41076) 2024-02-02T15:30:45.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.329+0000 7f55d4e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56317d9d47e0 0x56317da2c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:45.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.329+0000 7f55cfe00700 1 -- 172.21.15.1:0/2014696148 learned_addr learned my addr 172.21.15.1:0/2014696148 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:30:45.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.329+0000 7f55cfe00700 1 -- 172.21.15.1:0/2014696148 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56317d9d47e0 msgr2=0x56317da2c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:45.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.329+0000 7f55cfe00700 1 --2- 172.21.15.1:0/2014696148 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56317d9d47e0 0x56317da2c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.329+0000 7f55cfe00700 1 -- 172.21.15.1:0/2014696148 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56317d9c9930 msgr2=0x56317d9d8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:45.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.329+0000 7f55cfe00700 1 --2- 172.21.15.1:0/2014696148 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56317d9c9930 0x56317d9d8490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.329+0000 7f55cfe00700 1 -- 172.21.15.1:0/2014696148 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55b8021040 con 0x56317d9c9010 2024-02-02T15:30:45.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.330+0000 7f55cfe00700 1 --2- 172.21.15.1:0/2014696148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56317d9c9010 0x56317d9d7f50 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f55c00993d0 tx=0x7f55c0000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3599a0319b625d16 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:45.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.330+0000 7f55be000700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:30:45.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.330+0000 7f55be000700 1 -- 172.21.15.1:0/2014696148 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c009e280 con 0x56317d9c9010 2024-02-02T15:30:45.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.330+0000 7f55be000700 1 -- 172.21.15.1:0/2014696148 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f55c00a2430 con 0x56317d9c9010 2024-02-02T15:30:45.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.330+0000 7f55ea971b80 1 -- 172.21.15.1:0/2014696148 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f55c009ab50 con 0x56317d9c9010 2024-02-02T15:30:45.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.330+0000 7f55be000700 1 -- 172.21.15.1:0/2014696148 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f55c009a450 con 0x56317d9c9010 2024-02-02T15:30:45.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.330+0000 7f55ea971b80 10 client.26653 fetch_fsmap learned FSMap version 31 2024-02-02T15:30:45.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.330+0000 7f55ea971b80 1 -- 172.21.15.1:0/2014696148 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x56317d748b60 con 0x56317d9c9010 2024-02-02T15:30:45.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.330+0000 7f55be000700 1 -- 172.21.15.1:0/2014696148 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f55c009ab50 con 0x56317d9c9010 2024-02-02T15:30:45.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.330+0000 7f55be000700 1 -- 172.21.15.1:0/2014696148 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 31) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f55c0082030 con 0x56317d9c9010 2024-02-02T15:30:45.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.330+0000 7f55ea971b80 10 client.26653 fetch_fsmap finished waiting for FSMap version 31 2024-02-02T15:30:45.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.330+0000 7f55ea971b80 10 client.26653 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-02T15:30:45.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.330+0000 7f55ea971b80 20 client.26653 populate_metadata read hostname 'smithi001' 2024-02-02T15:30:45.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.330+0000 7f55ea971b80 1 --2- 172.21.15.1:0/2014696148 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x56317da355f0 0x56317da379e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:45.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.330+0000 7f55ea971b80 4 client.26653 mds_command: new command op to 24529 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:30:45.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.330+0000 7f55ea971b80 1 -- 172.21.15.1:0/2014696148 --> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56317d8d9f50 con 0x56317da355f0 2024-02-02T15:30:45.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.331+0000 7f55cf400700 1 --2- 172.21.15.1:0/2014696148 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x56317da355f0 0x56317da379e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:45.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.331+0000 7f55cf400700 1 --2- 172.21.15.1:0/2014696148 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x56317da355f0 0x56317da379e0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b63a0b03e06d0344 server_cookie=60b13b10a4762fb2 in_seq=0 out_seq=0 2024-02-02T15:30:45.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.331+0000 7f55be000700 10 client.26653 ms_handle_connect on v2:172.21.15.142:6838/321675421 2024-02-02T15:30:45.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.331+0000 7f55be000700 1 -- 172.21.15.1:0/2014696148 <== mds.1 v2:172.21.15.142:6838/321675421 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56317d8d9f50 con 0x56317da355f0 2024-02-02T15:30:45.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.331+0000 7f55be000700 10 client.26653 handle_command_reply: tid=0 2024-02-02T15:30:45.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.331+0000 7f55ea971b80 1 client.26653 shutdown 2024-02-02T15:30:45.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.332+0000 7f55ea971b80 1 -- 172.21.15.1:0/2014696148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56317d9c9010 msgr2=0x56317d9d7f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:45.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.332+0000 7f55ea971b80 1 --2- 172.21.15.1:0/2014696148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56317d9c9010 0x56317d9d7f50 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f55c00993d0 tx=0x7f55c0000bc0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.332+0000 7f55ea971b80 1 -- 172.21.15.1:0/2014696148 shutdown_connections 2024-02-02T15:30:45.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.332+0000 7f55ea971b80 1 --2- 172.21.15.1:0/2014696148 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x56317da355f0 0x56317da379e0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.332+0000 7f55ea971b80 1 --2- 172.21.15.1:0/2014696148 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56317d9d47e0 0x56317da2c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.332+0000 7f55ea971b80 1 --2- 172.21.15.1:0/2014696148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56317d9c9010 0x56317d9d7f50 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.332+0000 7f55ea971b80 1 --2- 172.21.15.1:0/2014696148 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56317d9c9930 0x56317d9d8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.332+0000 7f55be000700 0 client.26653 ms_handle_reset on v2:172.21.15.142:6838/321675421 2024-02-02T15:30:45.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.332+0000 7f55ea971b80 1 -- 172.21.15.1:0/2014696148 >> 172.21.15.1:0/2014696148 conn(0x56317d928db0 msgr2=0x56317d9ceb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:45.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.332+0000 7f55ea971b80 1 -- 172.21.15.1:0/2014696148 shutdown_connections 2024-02-02T15:30:45.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.332+0000 7f55ea971b80 1 -- 172.21.15.1:0/2014696148 wait complete. 2024-02-02T15:30:45.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.332+0000 7f55ea971b80 20 client.26653 trim_cache size 0 max 16384 2024-02-02T15:30:45.336 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:30:45.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.333+0000 7f55d6e00700 1 -- 172.21.15.1:0/2472193950 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f55b00a0710 msgr2=0x7f55b00a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:45.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.333+0000 7f55d6e00700 1 --2- 172.21.15.1:0/2472193950 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f55b00a0710 0x7f55b00a2bd0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f55d019a6d0 tx=0x7f55b8009260 comp rx=0 tx=0).stop 2024-02-02T15:30:45.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.333+0000 7f55d6e00700 1 -- 172.21.15.1:0/2472193950 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f55d0103230 msgr2=0x7f55d01990c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:45.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.333+0000 7f55d6e00700 1 --2- 172.21.15.1:0/2472193950 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f55d0103230 0x7f55d01990c0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f55c0005910 tx=0x7f55c0004730 comp rx=0 tx=0).stop 2024-02-02T15:30:45.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.333+0000 7f55d6e00700 1 -- 172.21.15.1:0/2472193950 shutdown_connections 2024-02-02T15:30:45.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.333+0000 7f55d6e00700 1 --2- 172.21.15.1:0/2472193950 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f55b00a0710 0x7f55b00a2bd0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.333+0000 7f55d6e00700 1 --2- 172.21.15.1:0/2472193950 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f55d019da60 0x7f55d019dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.333+0000 7f55d6e00700 1 --2- 172.21.15.1:0/2472193950 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f55d0103230 0x7f55d01990c0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.333+0000 7f55d6e00700 1 --2- 172.21.15.1:0/2472193950 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f55d0108350 0x7f55d0199600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.333+0000 7f55d6e00700 1 -- 172.21.15.1:0/2472193950 >> 172.21.15.1:0/2472193950 conn(0x7f55d00fea70 msgr2=0x7f55d0106c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:45.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.333+0000 7f55d6e00700 1 -- 172.21.15.1:0/2472193950 shutdown_connections 2024-02-02T15:30:45.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.333+0000 7f55d6e00700 1 -- 172.21.15.1:0/2472193950 wait complete. 2024-02-02T15:30:45.352 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-02T15:30:45.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.609+0000 7f5081600700 1 -- 172.21.15.1:0/545856050 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f507c103be0 msgr2=0x7f507c107c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:45.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.609+0000 7f5081600700 1 --2- 172.21.15.1:0/545856050 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f507c103be0 0x7f507c107c20 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f5064009990 tx=0x7f5064009ca0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.609+0000 7f5081600700 1 -- 172.21.15.1:0/545856050 shutdown_connections 2024-02-02T15:30:45.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.609+0000 7f5081600700 1 --2- 172.21.15.1:0/545856050 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f507c108350 0x7f507c10fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.609+0000 7f5081600700 1 --2- 172.21.15.1:0/545856050 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f507c103230 0x7f507c103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.609+0000 7f5081600700 1 --2- 172.21.15.1:0/545856050 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f507c103be0 0x7f507c107c20 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.609+0000 7f5081600700 1 -- 172.21.15.1:0/545856050 >> 172.21.15.1:0/545856050 conn(0x7f507c0fea70 msgr2=0x7f507c100e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:45.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.609+0000 7f5081600700 1 -- 172.21.15.1:0/545856050 shutdown_connections 2024-02-02T15:30:45.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.609+0000 7f5081600700 1 -- 172.21.15.1:0/545856050 wait complete. 2024-02-02T15:30:45.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.611+0000 7f5081600700 1 Processor -- start 2024-02-02T15:30:45.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.611+0000 7f5081600700 1 -- start start 2024-02-02T15:30:45.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.612+0000 7f5081600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f507c103230 0x7f507c1990b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:45.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.612+0000 7f5081600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f507c108350 0x7f507c1995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:45.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.612+0000 7f5081600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f507c19da50 0x7f507c19ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:45.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.612+0000 7f5081600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f507c19e4d0 con 0x7f507c108350 2024-02-02T15:30:45.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.612+0000 7f5081600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f507c19e640 con 0x7f507c103230 2024-02-02T15:30:45.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.612+0000 7f5081600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f507c19e800 con 0x7f507c19da50 2024-02-02T15:30:45.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.612+0000 7f507b400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f507c19da50 0x7f507c19ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:45.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.612+0000 7f507aa00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f507c103230 0x7f507c1990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:45.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.612+0000 7f507a000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f507c108350 0x7f507c1995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:45.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.612+0000 7f507b400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f507c19da50 0x7f507c19ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:41096/0 (socket says 172.21.15.1:41096) 2024-02-02T15:30:45.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.612+0000 7f507aa00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f507c103230 0x7f507c1990b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:38790/0 (socket says 172.21.15.1:38790) 2024-02-02T15:30:45.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.612+0000 7f507b400700 1 -- 172.21.15.1:0/1523615993 learned_addr learned my addr 172.21.15.1:0/1523615993 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:30:45.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.612+0000 7f507aa00700 1 -- 172.21.15.1:0/1523615993 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f507c19da50 msgr2=0x7f507c19ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:45.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.612+0000 7f507aa00700 1 --2- 172.21.15.1:0/1523615993 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f507c19da50 0x7f507c19ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.612+0000 7f507aa00700 1 -- 172.21.15.1:0/1523615993 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f507c108350 msgr2=0x7f507c1995f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:45.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.612+0000 7f507aa00700 1 --2- 172.21.15.1:0/1523615993 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f507c108350 0x7f507c1995f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.612+0000 7f507aa00700 1 -- 172.21.15.1:0/1523615993 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50640095f0 con 0x7f507c103230 2024-02-02T15:30:45.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.612+0000 7f507b400700 1 --2- 172.21.15.1:0/1523615993 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f507c19da50 0x7f507c19ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:30:45.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.612+0000 7f507a000700 1 --2- 172.21.15.1:0/1523615993 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f507c108350 0x7f507c1995f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:30:45.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.613+0000 7f507aa00700 1 --2- 172.21.15.1:0/1523615993 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f507c103230 0x7f507c1990b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f506c00e9f0 tx=0x7f506c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:45.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.613+0000 7f5073400700 1 -- 172.21.15.1:0/1523615993 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f506c00c9c0 con 0x7f507c103230 2024-02-02T15:30:45.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.613+0000 7f5073400700 1 -- 172.21.15.1:0/1523615993 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f506c010430 con 0x7f507c103230 2024-02-02T15:30:45.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.613+0000 7f5081600700 1 -- 172.21.15.1:0/1523615993 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f507c1a5d60 con 0x7f507c103230 2024-02-02T15:30:45.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.613+0000 7f5073400700 1 -- 172.21.15.1:0/1523615993 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f506c018680 con 0x7f507c103230 2024-02-02T15:30:45.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.613+0000 7f5081600700 1 -- 172.21.15.1:0/1523615993 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f507c1a6230 con 0x7f507c103230 2024-02-02T15:30:45.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:45 smithi071 ceph-mon[44707]: pgmap v115: 129 pgs: 129 active+clean; 105 MiB data, 912 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 193 op/s 2024-02-02T15:30:45.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.616+0000 7f5073400700 1 -- 172.21.15.1:0/1523615993 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 27) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f506c010860 con 0x7f507c103230 2024-02-02T15:30:45.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.617+0000 7f5073400700 1 --2- 172.21.15.1:0/1523615993 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f505c0a0710 0x7f505c0a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:45.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.617+0000 7f50951c0b80 1 Processor -- start 2024-02-02T15:30:45.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.618+0000 7f5073400700 1 -- 172.21.15.1:0/1523615993 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f506c014070 con 0x7f507c103230 2024-02-02T15:30:45.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.618+0000 7f50951c0b80 1 -- start start 2024-02-02T15:30:45.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.618+0000 7f507a000700 1 --2- 172.21.15.1:0/1523615993 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f505c0a0710 0x7f505c0a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:45.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.618+0000 7f50951c0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f52fb97010 0x55f52fb973f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:45.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.618+0000 7f50951c0b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f52fb97930 0x55f52fba22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:45.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.618+0000 7f50951c0b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f52fba27e0 0x55f52fba4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:45.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.618+0000 7f50951c0b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f52fb0b4c0 con 0x55f52fba27e0 2024-02-02T15:30:45.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.618+0000 7f50951c0b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f52faefaa0 con 0x55f52fb97930 2024-02-02T15:30:45.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.618+0000 7f50951c0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f52faed460 con 0x55f52fb97010 2024-02-02T15:30:45.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.618+0000 7f507b400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f52fba27e0 0x55f52fba4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:45.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.618+0000 7f507aa00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f52fb97010 0x55f52fb973f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:45.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.618+0000 7f507aa00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f52fb97010 0x55f52fb973f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:41108/0 (socket says 172.21.15.1:41108) 2024-02-02T15:30:45.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.619+0000 7f507b400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f52fba27e0 0x55f52fba4bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45226/0 (socket says 172.21.15.1:45226) 2024-02-02T15:30:45.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.619+0000 7f507aa00700 1 -- 172.21.15.1:0/1481891832 learned_addr learned my addr 172.21.15.1:0/1481891832 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:30:45.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.619+0000 7f507a000700 1 --2- 172.21.15.1:0/1523615993 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f505c0a0710 0x7f505c0a2bd0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f507c19a6c0 tx=0x7f5064005fb0 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:45.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.619+0000 7f507a000700 1 --2- 172.21.15.1:0/1481891832 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f52fb97930 0x55f52fba22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:45.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.619+0000 7f507aa00700 1 -- 172.21.15.1:0/1481891832 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f52fb97930 msgr2=0x55f52fba22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:45.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.619+0000 7f507aa00700 1 --2- 172.21.15.1:0/1481891832 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f52fb97930 0x55f52fba22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.619+0000 7f507aa00700 1 -- 172.21.15.1:0/1481891832 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f52fba27e0 msgr2=0x55f52fba4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:45.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.619+0000 7f507aa00700 1 --2- 172.21.15.1:0/1481891832 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f52fba27e0 0x55f52fba4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.619+0000 7f507aa00700 1 -- 172.21.15.1:0/1481891832 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f52f96a3b0 con 0x55f52fb97010 2024-02-02T15:30:45.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.619+0000 7f507aa00700 1 --2- 172.21.15.1:0/1481891832 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f52fb97010 0x55f52fb973f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f506c099020 tx=0x7f506c098d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d7377c939ae339d server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:45.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.619+0000 7f506b400700 1 -- 172.21.15.1:0/1481891832 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f506c09d150 con 0x55f52fb97010 2024-02-02T15:30:45.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.619+0000 7f506b400700 1 -- 172.21.15.1:0/1481891832 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f506c0a1390 con 0x55f52fb97010 2024-02-02T15:30:45.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.619+0000 7f506b400700 1 -- 172.21.15.1:0/1481891832 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f506c098810 con 0x55f52fb97010 2024-02-02T15:30:45.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.620+0000 7f50951c0b80 1 -- 172.21.15.1:0/1481891832 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f52fb97010 msgr2=0x55f52fb973f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:45.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.620+0000 7f50951c0b80 1 --2- 172.21.15.1:0/1481891832 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f52fb97010 0x55f52fb973f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f506c099020 tx=0x7f506c098d00 comp rx=0 tx=0).stop 2024-02-02T15:30:45.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.620+0000 7f50951c0b80 1 -- 172.21.15.1:0/1481891832 shutdown_connections 2024-02-02T15:30:45.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.620+0000 7f50951c0b80 1 --2- 172.21.15.1:0/1481891832 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f52fba27e0 0x55f52fba4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.620+0000 7f50951c0b80 1 --2- 172.21.15.1:0/1481891832 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f52fb97010 0x55f52fb973f0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.620+0000 7f50951c0b80 1 --2- 172.21.15.1:0/1481891832 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f52fb97930 0x55f52fba22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.620+0000 7f50951c0b80 1 -- 172.21.15.1:0/1481891832 >> 172.21.15.1:0/1481891832 conn(0x55f52faf6db0 msgr2=0x55f52fb933b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:45.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.620+0000 7f50951c0b80 1 -- 172.21.15.1:0/1481891832 shutdown_connections 2024-02-02T15:30:45.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.620+0000 7f50951c0b80 1 -- 172.21.15.1:0/1481891832 wait complete. 2024-02-02T15:30:45.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.621+0000 7f50951c0b80 1 -- start start 2024-02-02T15:30:45.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.621+0000 7f50951c0b80 1 Processor -- start 2024-02-02T15:30:45.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.621+0000 7f50951c0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f52fb97010 0x55f52fba21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:45.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.622+0000 7f50951c0b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f52fb97930 0x55f52fb9f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:45.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.622+0000 7f50951c0b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f52fba27e0 0x55f52fb9fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:45.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.622+0000 7f50951c0b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f52fae8d50 con 0x55f52fba27e0 2024-02-02T15:30:45.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.622+0000 7f50951c0b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f52fa84d80 con 0x55f52fb97930 2024-02-02T15:30:45.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.622+0000 7f50951c0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f52fa84a20 con 0x55f52fb97010 2024-02-02T15:30:45.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.622+0000 7f50951c0b80 10 client.0 _ll_register_callbacks cb 0x55f52fa86290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:30:45.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.622+0000 7f507aa00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f52fb97010 0x55f52fba21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:45.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.622+0000 7f507a000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f52fb97930 0x55f52fb9f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:45.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.622+0000 7f507aa00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f52fb97010 0x55f52fba21f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:41114/0 (socket says 172.21.15.1:41114) 2024-02-02T15:30:45.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.622+0000 7f507aa00700 1 -- 172.21.15.1:0/1332502502 learned_addr learned my addr 172.21.15.1:0/1332502502 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:30:45.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.622+0000 7f507b400700 1 --2- 172.21.15.1:0/1332502502 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f52fba27e0 0x55f52fb9fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:45.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.622+0000 7f507aa00700 1 -- 172.21.15.1:0/1332502502 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f52fb97930 msgr2=0x55f52fb9f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:45.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.622+0000 7f507aa00700 1 --2- 172.21.15.1:0/1332502502 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f52fb97930 0x55f52fb9f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.622+0000 7f507aa00700 1 -- 172.21.15.1:0/1332502502 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f52fba27e0 msgr2=0x55f52fb9fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:45.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.622+0000 7f507aa00700 1 --2- 172.21.15.1:0/1332502502 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f52fba27e0 0x55f52fb9fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.622+0000 7f507aa00700 1 -- 172.21.15.1:0/1332502502 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f507c1a6230 con 0x55f52fb97010 2024-02-02T15:30:45.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.622+0000 7f507b400700 1 --2- 172.21.15.1:0/1332502502 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f52fba27e0 0x55f52fb9fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:30:45.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.623+0000 7f507aa00700 1 --2- 172.21.15.1:0/1332502502 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f52fb97010 0x55f52fba21f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f506c00c5b0 tx=0x7f506c0107d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ef500aa122b3b42 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:45.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.623+0000 7f5063e00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:30:45.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.623+0000 7f5063e00700 1 -- 172.21.15.1:0/1332502502 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f506c09d150 con 0x55f52fb97010 2024-02-02T15:30:45.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.623+0000 7f5063e00700 1 -- 172.21.15.1:0/1332502502 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f506c099790 con 0x55f52fb97010 2024-02-02T15:30:45.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.623+0000 7f50951c0b80 1 -- 172.21.15.1:0/1332502502 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f52fb8b0f0 con 0x55f52fb97010 2024-02-02T15:30:45.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.623+0000 7f5063e00700 1 -- 172.21.15.1:0/1332502502 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f506c0a0f00 con 0x55f52fb97010 2024-02-02T15:30:45.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.623+0000 7f50951c0b80 10 client.26665 fetch_fsmap learned FSMap version 31 2024-02-02T15:30:45.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.623+0000 7f50951c0b80 1 -- 172.21.15.1:0/1332502502 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x55f52f916b60 con 0x55f52fb97010 2024-02-02T15:30:45.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.623+0000 7f5063e00700 1 -- 172.21.15.1:0/1332502502 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f52fb8b0f0 con 0x55f52fb97010 2024-02-02T15:30:45.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.623+0000 7f5063e00700 1 -- 172.21.15.1:0/1332502502 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 31) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f506c095110 con 0x55f52fb97010 2024-02-02T15:30:45.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.623+0000 7f50951c0b80 10 client.26665 fetch_fsmap finished waiting for FSMap version 31 2024-02-02T15:30:45.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.623+0000 7f50951c0b80 10 client.26665 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:30:45.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.623+0000 7f50951c0b80 20 client.26665 populate_metadata read hostname 'smithi001' 2024-02-02T15:30:45.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.623+0000 7f50951c0b80 1 --2- 172.21.15.1:0/1332502502 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55f52fbb1f20 0x55f52fbb4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:45.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.623+0000 7f50951c0b80 4 client.26665 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:30:45.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.623+0000 7f50951c0b80 1 -- 172.21.15.1:0/1332502502 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f52fae1dd0 con 0x55f52fbb1f20 2024-02-02T15:30:45.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.624+0000 7f507a000700 1 --2- 172.21.15.1:0/1332502502 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55f52fbb1f20 0x55f52fbb4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:45.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.624+0000 7f507a000700 1 --2- 172.21.15.1:0/1332502502 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55f52fbb1f20 0x55f52fbb4340 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=710ff9905b54d17d server_cookie=fcb823f347a3d440 in_seq=0 out_seq=0 2024-02-02T15:30:45.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.624+0000 7f5063e00700 10 client.26665 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:30:45.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.625+0000 7f5063e00700 1 -- 172.21.15.1:0/1332502502 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55f52fae1dd0 con 0x55f52fbb1f20 2024-02-02T15:30:45.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.625+0000 7f5063e00700 10 client.26665 handle_command_reply: tid=0 2024-02-02T15:30:45.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.625+0000 7f50951c0b80 1 client.26665 shutdown 2024-02-02T15:30:45.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.625+0000 7f50951c0b80 1 -- 172.21.15.1:0/1332502502 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f52fb97010 msgr2=0x55f52fba21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:45.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.625+0000 7f50951c0b80 1 --2- 172.21.15.1:0/1332502502 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f52fb97010 0x55f52fba21f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f506c00c5b0 tx=0x7f506c0107d0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.625+0000 7f50951c0b80 1 -- 172.21.15.1:0/1332502502 shutdown_connections 2024-02-02T15:30:45.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.625+0000 7f50951c0b80 1 --2- 172.21.15.1:0/1332502502 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55f52fbb1f20 0x55f52fbb4340 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.625+0000 7f50951c0b80 1 --2- 172.21.15.1:0/1332502502 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f52fba27e0 0x55f52fb9fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.625+0000 7f50951c0b80 1 --2- 172.21.15.1:0/1332502502 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f52fb97010 0x55f52fba21f0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.625+0000 7f5063e00700 0 client.26665 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:30:45.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.625+0000 7f50951c0b80 1 --2- 172.21.15.1:0/1332502502 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f52fb97930 0x55f52fb9f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.625+0000 7f50951c0b80 1 -- 172.21.15.1:0/1332502502 >> 172.21.15.1:0/1332502502 conn(0x55f52faf6db0 msgr2=0x55f52fb92fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:45.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.625+0000 7f50951c0b80 1 -- 172.21.15.1:0/1332502502 shutdown_connections 2024-02-02T15:30:45.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.625+0000 7f50951c0b80 1 -- 172.21.15.1:0/1332502502 wait complete. 2024-02-02T15:30:45.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.626+0000 7f50951c0b80 20 client.26665 trim_cache size 0 max 16384 2024-02-02T15:30:45.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.640+0000 7f50951c0b80 1 Processor -- start 2024-02-02T15:30:45.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.640+0000 7f50951c0b80 1 -- start start 2024-02-02T15:30:45.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.640+0000 7f50951c0b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f52fb97010 0x55f52fb9f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:45.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.640+0000 7f50951c0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f52fb97930 0x55f52fb8ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:45.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.640+0000 7f50951c0b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f52fba27e0 0x55f52fba1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:45.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.640+0000 7f50951c0b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f52f96c6a0 con 0x55f52fb97010 2024-02-02T15:30:45.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.640+0000 7f50951c0b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f52fb8f3e0 con 0x55f52fba27e0 2024-02-02T15:30:45.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.640+0000 7f50951c0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f52fb9b2a0 con 0x55f52fb97930 2024-02-02T15:30:45.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.641+0000 7f507aa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f52fb97010 0x55f52fb9f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:45.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.641+0000 7f507a000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f52fb97930 0x55f52fb8ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:45.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.641+0000 7f507aa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f52fb97010 0x55f52fb9f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45252/0 (socket says 172.21.15.1:45252) 2024-02-02T15:30:45.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.641+0000 7f507b400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f52fba27e0 0x55f52fba1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:45.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.641+0000 7f507aa00700 1 -- 172.21.15.1:0/1227010524 learned_addr learned my addr 172.21.15.1:0/1227010524 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:30:45.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.641+0000 7f507a000700 1 -- 172.21.15.1:0/1227010524 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f52fba27e0 msgr2=0x55f52fba1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:45.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.641+0000 7f507a000700 1 --2- 172.21.15.1:0/1227010524 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f52fba27e0 0x55f52fba1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.641+0000 7f507a000700 1 -- 172.21.15.1:0/1227010524 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f52fb97010 msgr2=0x55f52fb9f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:45.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.641+0000 7f507a000700 1 --2- 172.21.15.1:0/1227010524 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f52fb97010 0x55f52fb9f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.641+0000 7f507a000700 1 -- 172.21.15.1:0/1227010524 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f52faaeea0 con 0x55f52fb97930 2024-02-02T15:30:45.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.641+0000 7f507aa00700 1 --2- 172.21.15.1:0/1227010524 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f52fb97010 0x55f52fb9f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:30:45.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.642+0000 7f507a000700 1 --2- 172.21.15.1:0/1227010524 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f52fb97930 0x55f52fb8ee00 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5064000c00 tx=0x7f5064022000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b535b56a5d5eb75a server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:45.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.642+0000 7f506be00700 1 -- 172.21.15.1:0/1227010524 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f506400f4b0 con 0x55f52fb97930 2024-02-02T15:30:45.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.642+0000 7f506be00700 1 -- 172.21.15.1:0/1227010524 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f506402a030 con 0x55f52fb97930 2024-02-02T15:30:45.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.642+0000 7f506be00700 1 -- 172.21.15.1:0/1227010524 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5064033d30 con 0x55f52fb97930 2024-02-02T15:30:45.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.642+0000 7f50951c0b80 1 -- 172.21.15.1:0/1227010524 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f52fb97930 msgr2=0x55f52fb8ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:45.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.642+0000 7f50951c0b80 1 --2- 172.21.15.1:0/1227010524 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f52fb97930 0x55f52fb8ee00 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5064000c00 tx=0x7f5064022000 comp rx=0 tx=0).stop 2024-02-02T15:30:45.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.643+0000 7f50951c0b80 1 -- 172.21.15.1:0/1227010524 shutdown_connections 2024-02-02T15:30:45.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.643+0000 7f50951c0b80 1 --2- 172.21.15.1:0/1227010524 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f52fba27e0 0x55f52fba1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.643+0000 7f50951c0b80 1 --2- 172.21.15.1:0/1227010524 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f52fb97010 0x55f52fb9f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.643+0000 7f50951c0b80 1 --2- 172.21.15.1:0/1227010524 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f52fb97930 0x55f52fb8ee00 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.643+0000 7f50951c0b80 1 -- 172.21.15.1:0/1227010524 >> 172.21.15.1:0/1227010524 conn(0x55f52faf6db0 msgr2=0x55f52fb98830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:45.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.643+0000 7f50951c0b80 1 -- 172.21.15.1:0/1227010524 shutdown_connections 2024-02-02T15:30:45.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.643+0000 7f50951c0b80 1 -- 172.21.15.1:0/1227010524 wait complete. 2024-02-02T15:30:45.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.643+0000 7f50951c0b80 1 -- start start 2024-02-02T15:30:45.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.643+0000 7f50951c0b80 1 Processor -- start 2024-02-02T15:30:45.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.644+0000 7f50951c0b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f52fb97010 0x55f52fba6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:45.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.644+0000 7f50951c0b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f52fb97930 0x55f52fba6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:45.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.644+0000 7f50951c0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f52fba27e0 0x55f52fba7410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:45.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.644+0000 7f50951c0b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f52fc02ae0 con 0x55f52fb97010 2024-02-02T15:30:45.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.644+0000 7f50951c0b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f52fc02c50 con 0x55f52fb97930 2024-02-02T15:30:45.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.644+0000 7f50951c0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f52fc02e10 con 0x55f52fba27e0 2024-02-02T15:30:45.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.644+0000 7f50951c0b80 10 client.0 _ll_register_callbacks cb 0x55f52fa86290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:30:45.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.644+0000 7f507a000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f52fb97930 0x55f52fba6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:45.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.644+0000 7f507b400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f52fba27e0 0x55f52fba7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:45.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.644+0000 7f507aa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f52fb97010 0x55f52fba6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:45.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.644+0000 7f507a000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f52fb97930 0x55f52fba6ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:38840/0 (socket says 172.21.15.1:38840) 2024-02-02T15:30:45.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.644+0000 7f507a000700 1 -- 172.21.15.1:0/1123710879 learned_addr learned my addr 172.21.15.1:0/1123710879 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:30:45.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.644+0000 7f507b400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f52fba27e0 0x55f52fba7410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:41126/0 (socket says 172.21.15.1:41126) 2024-02-02T15:30:45.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.645+0000 7f507b400700 1 -- 172.21.15.1:0/1123710879 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f52fb97930 msgr2=0x55f52fba6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:45.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.645+0000 7f507b400700 1 --2- 172.21.15.1:0/1123710879 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f52fb97930 0x55f52fba6ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.645+0000 7f507b400700 1 -- 172.21.15.1:0/1123710879 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f52fb97010 msgr2=0x55f52fba6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:45.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.645+0000 7f507b400700 1 --2- 172.21.15.1:0/1123710879 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f52fb97010 0x55f52fba6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.645+0000 7f507b400700 1 -- 172.21.15.1:0/1123710879 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f52f916b60 con 0x55f52fba27e0 2024-02-02T15:30:45.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.645+0000 7f507b400700 1 --2- 172.21.15.1:0/1123710879 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f52fba27e0 0x55f52fba7410 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f507400e840 tx=0x7f507400eb50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be92c2c6301ab18d server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:45.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.645+0000 7f5063e00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:30:45.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.645+0000 7f5063e00700 1 -- 172.21.15.1:0/1123710879 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f507401c070 con 0x55f52fba27e0 2024-02-02T15:30:45.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.645+0000 7f5063e00700 1 -- 172.21.15.1:0/1123710879 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f5074004bb0 con 0x55f52fba27e0 2024-02-02T15:30:45.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.645+0000 7f50951c0b80 1 -- 172.21.15.1:0/1123710879 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f506c0a1390 con 0x55f52fba27e0 2024-02-02T15:30:45.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.646+0000 7f5063e00700 1 -- 172.21.15.1:0/1123710879 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5074004d70 con 0x55f52fba27e0 2024-02-02T15:30:45.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.646+0000 7f50951c0b80 10 client.26671 fetch_fsmap learned FSMap version 31 2024-02-02T15:30:45.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.646+0000 7f50951c0b80 1 -- 172.21.15.1:0/1123710879 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f507c110740 con 0x55f52fba27e0 2024-02-02T15:30:45.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.646+0000 7f5063e00700 1 -- 172.21.15.1:0/1123710879 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f506c0a1390 con 0x55f52fba27e0 2024-02-02T15:30:45.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.646+0000 7f5063e00700 1 -- 172.21.15.1:0/1123710879 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 31) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f5074004020 con 0x55f52fba27e0 2024-02-02T15:30:45.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.646+0000 7f50951c0b80 10 client.26671 fetch_fsmap finished waiting for FSMap version 31 2024-02-02T15:30:45.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.646+0000 7f50951c0b80 10 client.26671 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:30:45.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.646+0000 7f50951c0b80 20 client.26671 populate_metadata read hostname 'smithi001' 2024-02-02T15:30:45.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.646+0000 7f50951c0b80 1 --2- 172.21.15.1:0/1123710879 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55f52fc046f0 0x55f52fc06b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:45.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.646+0000 7f50951c0b80 4 client.26671 mds_command: new command op to 24557 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:30:45.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.646+0000 7f50951c0b80 1 -- 172.21.15.1:0/1123710879 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f52faa7f50 con 0x55f52fc046f0 2024-02-02T15:30:45.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.647+0000 7f507aa00700 1 --2- 172.21.15.1:0/1123710879 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55f52fc046f0 0x55f52fc06b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:45.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.647+0000 7f507aa00700 1 --2- 172.21.15.1:0/1123710879 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55f52fc046f0 0x55f52fc06b10 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3260bd9290f97ad2 server_cookie=8635899030e0c8d5 in_seq=0 out_seq=0 2024-02-02T15:30:45.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.647+0000 7f5063e00700 10 client.26671 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:30:45.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.647+0000 7f5063e00700 1 -- 172.21.15.1:0/1123710879 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f52faa7f50 con 0x55f52fc046f0 2024-02-02T15:30:45.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.647+0000 7f5063e00700 10 client.26671 handle_command_reply: tid=0 2024-02-02T15:30:45.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.647+0000 7f50951c0b80 1 client.26671 shutdown 2024-02-02T15:30:45.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.647+0000 7f50951c0b80 1 -- 172.21.15.1:0/1123710879 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f52fba27e0 msgr2=0x55f52fba7410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:45.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.647+0000 7f50951c0b80 1 --2- 172.21.15.1:0/1123710879 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f52fba27e0 0x55f52fba7410 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f507400e840 tx=0x7f507400eb50 comp rx=0 tx=0).stop 2024-02-02T15:30:45.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.648+0000 7f50951c0b80 1 -- 172.21.15.1:0/1123710879 shutdown_connections 2024-02-02T15:30:45.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.648+0000 7f50951c0b80 1 --2- 172.21.15.1:0/1123710879 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55f52fc046f0 0x55f52fc06b10 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.648+0000 7f50951c0b80 1 --2- 172.21.15.1:0/1123710879 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f52fba27e0 0x55f52fba7410 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.648+0000 7f50951c0b80 1 --2- 172.21.15.1:0/1123710879 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f52fb97010 0x55f52fba6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.648+0000 7f50951c0b80 1 --2- 172.21.15.1:0/1123710879 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f52fb97930 0x55f52fba6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.648+0000 7f50951c0b80 1 -- 172.21.15.1:0/1123710879 >> 172.21.15.1:0/1123710879 conn(0x55f52faf6db0 msgr2=0x55f52fb9a740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:45.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.648+0000 7f5063e00700 0 client.26671 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:30:45.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.648+0000 7f50951c0b80 1 -- 172.21.15.1:0/1123710879 shutdown_connections 2024-02-02T15:30:45.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.648+0000 7f50951c0b80 1 -- 172.21.15.1:0/1123710879 wait complete. 2024-02-02T15:30:45.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.648+0000 7f50951c0b80 20 client.26671 trim_cache size 0 max 16384 2024-02-02T15:30:45.651 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:30:45.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.648+0000 7f5081600700 1 -- 172.21.15.1:0/1523615993 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f505c0a0710 msgr2=0x7f505c0a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:45.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.648+0000 7f5081600700 1 --2- 172.21.15.1:0/1523615993 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f505c0a0710 0x7f505c0a2bd0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f507c19a6c0 tx=0x7f5064005fb0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.649+0000 7f5081600700 1 -- 172.21.15.1:0/1523615993 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f507c103230 msgr2=0x7f507c1990b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:45.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.649+0000 7f5081600700 1 --2- 172.21.15.1:0/1523615993 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f507c103230 0x7f507c1990b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f506c00e9f0 tx=0x7f506c00ed00 comp rx=0 tx=0).stop 2024-02-02T15:30:45.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.649+0000 7f5081600700 1 -- 172.21.15.1:0/1523615993 shutdown_connections 2024-02-02T15:30:45.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.649+0000 7f5081600700 1 --2- 172.21.15.1:0/1523615993 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f505c0a0710 0x7f505c0a2bd0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.649+0000 7f5081600700 1 --2- 172.21.15.1:0/1523615993 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f507c19da50 0x7f507c19ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.649+0000 7f5081600700 1 --2- 172.21.15.1:0/1523615993 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f507c103230 0x7f507c1990b0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.649+0000 7f5081600700 1 --2- 172.21.15.1:0/1523615993 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f507c108350 0x7f507c1995f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:45.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.649+0000 7f5081600700 1 -- 172.21.15.1:0/1523615993 >> 172.21.15.1:0/1523615993 conn(0x7f507c0fea70 msgr2=0x7f507c0ff500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:45.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.649+0000 7f5081600700 1 -- 172.21.15.1:0/1523615993 shutdown_connections 2024-02-02T15:30:45.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:45.649+0000 7f5081600700 1 -- 172.21.15.1:0/1523615993 wait complete. 2024-02-02T15:30:45.664 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:45 smithi001 ceph-mon[35470]: pgmap v115: 129 pgs: 129 active+clean; 105 MiB data, 912 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 193 op/s 2024-02-02T15:30:45.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:45 smithi142 ceph-mon[45728]: pgmap v115: 129 pgs: 129 active+clean; 105 MiB data, 912 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 193 op/s 2024-02-02T15:30:46.228 INFO:tasks.workunit.client.0.smithi001.stdout: 1 176313 9.59 MB/sec execute 20 sec latency 19.002 ms 2024-02-02T15:30:46.667 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-02T15:30:46.667 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-02T15:30:46.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.879+0000 7f8847200700 1 -- 172.21.15.1:0/243968886 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8840073a00 msgr2=0x7f8840111850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:46.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.879+0000 7f8847200700 1 --2- 172.21.15.1:0/243968886 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8840073a00 0x7f8840111850 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8828009990 tx=0x7f8828009ca0 comp rx=0 tx=0).stop 2024-02-02T15:30:46.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.879+0000 7f8847200700 1 -- 172.21.15.1:0/243968886 shutdown_connections 2024-02-02T15:30:46.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.879+0000 7f8847200700 1 --2- 172.21.15.1:0/243968886 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8840111d90 0x7f8840114180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:46.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.879+0000 7f8847200700 1 --2- 172.21.15.1:0/243968886 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f88400730e0 0x7f88400734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:46.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.879+0000 7f8847200700 1 --2- 172.21.15.1:0/243968886 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8840073a00 0x7f8840111850 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:46.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.879+0000 7f8847200700 1 -- 172.21.15.1:0/243968886 >> 172.21.15.1:0/243968886 conn(0x7f88400fc810 msgr2=0x7f88400fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:46.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.880+0000 7f8847200700 1 -- 172.21.15.1:0/243968886 shutdown_connections 2024-02-02T15:30:46.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.880+0000 7f8847200700 1 -- 172.21.15.1:0/243968886 wait complete. 2024-02-02T15:30:46.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.881+0000 7f8847200700 1 Processor -- start 2024-02-02T15:30:46.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.881+0000 7f8847200700 1 -- start start 2024-02-02T15:30:46.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.881+0000 7f8847200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88400730e0 0x7f884019d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:46.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.881+0000 7f8847200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8840073a00 0x7f884019dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:46.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.881+0000 7f8847200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8840111d90 0x7f88401a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:46.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.881+0000 7f8847200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88401a2550 con 0x7f8840073a00 2024-02-02T15:30:46.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.881+0000 7f8847200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88401a26c0 con 0x7f88400730e0 2024-02-02T15:30:46.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.881+0000 7f8847200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88401a2880 con 0x7f8840111d90 2024-02-02T15:30:46.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.881+0000 7f883f400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8840073a00 0x7f884019dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:46.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.881+0000 7f883f400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8840073a00 0x7f884019dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45270/0 (socket says 172.21.15.1:45270) 2024-02-02T15:30:46.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.881+0000 7f883f400700 1 -- 172.21.15.1:0/1881146744 learned_addr learned my addr 172.21.15.1:0/1881146744 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:30:46.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.881+0000 7f8845200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8840111d90 0x7f88401a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:46.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.881+0000 7f883fe00700 1 --2- 172.21.15.1:0/1881146744 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88400730e0 0x7f884019d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:46.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.882+0000 7f883fe00700 1 -- 172.21.15.1:0/1881146744 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8840111d90 msgr2=0x7f88401a1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:46.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.882+0000 7f883fe00700 1 --2- 172.21.15.1:0/1881146744 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8840111d90 0x7f88401a1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:46.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.882+0000 7f883fe00700 1 -- 172.21.15.1:0/1881146744 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8840073a00 msgr2=0x7f884019dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:46.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.882+0000 7f883fe00700 1 --2- 172.21.15.1:0/1881146744 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8840073a00 0x7f884019dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:46.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.882+0000 7f883fe00700 1 -- 172.21.15.1:0/1881146744 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88280095f0 con 0x7f88400730e0 2024-02-02T15:30:46.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.882+0000 7f883fe00700 1 --2- 172.21.15.1:0/1881146744 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88400730e0 0x7f884019d570 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f883000e9f0 tx=0x7f883000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:46.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.882+0000 7f883cc00700 1 -- 172.21.15.1:0/1881146744 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f883000c9c0 con 0x7f88400730e0 2024-02-02T15:30:46.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.882+0000 7f8847200700 1 -- 172.21.15.1:0/1881146744 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88401a2b60 con 0x7f88400730e0 2024-02-02T15:30:46.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.882+0000 7f883cc00700 1 -- 172.21.15.1:0/1881146744 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8830010430 con 0x7f88400730e0 2024-02-02T15:30:46.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.882+0000 7f883cc00700 1 -- 172.21.15.1:0/1881146744 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8830018680 con 0x7f88400730e0 2024-02-02T15:30:46.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.882+0000 7f8847200700 1 -- 172.21.15.1:0/1881146744 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88401aa3a0 con 0x7f88400730e0 2024-02-02T15:30:46.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.885+0000 7f883cc00700 1 -- 172.21.15.1:0/1881146744 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 27) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f8830010860 con 0x7f88400730e0 2024-02-02T15:30:46.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.887+0000 7f883cc00700 1 --2- 172.21.15.1:0/1881146744 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f88200a0690 0x7f88200a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:46.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.887+0000 7f883cc00700 1 -- 172.21.15.1:0/1881146744 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f8830014070 con 0x7f88400730e0 2024-02-02T15:30:46.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.887+0000 7f883f400700 1 --2- 172.21.15.1:0/1881146744 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f88200a0690 0x7f88200a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:46.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.887+0000 7f885ad5fb80 1 Processor -- start 2024-02-02T15:30:46.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.888+0000 7f885ad5fb80 1 -- start start 2024-02-02T15:30:46.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.888+0000 7f883f400700 1 --2- 172.21.15.1:0/1881146744 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f88200a0690 0x7f88200a2b50 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f884019eb80 tx=0x7f882800b2a0 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:46.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.888+0000 7f885ad5fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559851e03090 0x559851e03470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:46.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.888+0000 7f885ad5fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559851e039b0 0x559851e0e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:46.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.888+0000 7f885ad5fb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559851e0e860 0x559851e10c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:46.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.888+0000 7f885ad5fb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x559851d77540 con 0x559851e0e860 2024-02-02T15:30:46.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.888+0000 7f885ad5fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x559851d5bb20 con 0x559851e039b0 2024-02-02T15:30:46.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.888+0000 7f885ad5fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559851d594e0 con 0x559851e03090 2024-02-02T15:30:46.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.888+0000 7f883f400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559851e039b0 0x559851e0e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:46.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.888+0000 7f883f400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559851e039b0 0x559851e0e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:38878/0 (socket says 172.21.15.1:38878) 2024-02-02T15:30:46.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.888+0000 7f883f400700 1 -- 172.21.15.1:0/599421226 learned_addr learned my addr 172.21.15.1:0/599421226 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:30:46.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.888+0000 7f883fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559851e03090 0x559851e03470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:46.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.888+0000 7f8845200700 1 --2- 172.21.15.1:0/599421226 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559851e0e860 0x559851e10c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:46.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.889+0000 7f883fe00700 1 -- 172.21.15.1:0/599421226 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559851e039b0 msgr2=0x559851e0e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:46.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.889+0000 7f883fe00700 1 --2- 172.21.15.1:0/599421226 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559851e039b0 0x559851e0e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:46.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.889+0000 7f883fe00700 1 -- 172.21.15.1:0/599421226 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559851e0e860 msgr2=0x559851e10c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:46.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.889+0000 7f883fe00700 1 --2- 172.21.15.1:0/599421226 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559851e0e860 0x559851e10c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:46.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.889+0000 7f883fe00700 1 -- 172.21.15.1:0/599421226 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559851bd6430 con 0x559851e03090 2024-02-02T15:30:46.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.889+0000 7f883f400700 1 --2- 172.21.15.1:0/599421226 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559851e039b0 0x559851e0e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:30:46.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.889+0000 7f883fe00700 1 --2- 172.21.15.1:0/599421226 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559851e03090 0x559851e03470 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f88300c95b0 tx=0x7f8830098d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30668d06f7d00df8 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:46.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.889+0000 7f8835600700 1 -- 172.21.15.1:0/599421226 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88300993b0 con 0x559851e03090 2024-02-02T15:30:46.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.889+0000 7f8835600700 1 -- 172.21.15.1:0/599421226 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f883009d1e0 con 0x559851e03090 2024-02-02T15:30:46.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.889+0000 7f8835600700 1 -- 172.21.15.1:0/599421226 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8830099830 con 0x559851e03090 2024-02-02T15:30:46.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.890+0000 7f885ad5fb80 1 -- 172.21.15.1:0/599421226 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559851e03090 msgr2=0x559851e03470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:46.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.890+0000 7f885ad5fb80 1 --2- 172.21.15.1:0/599421226 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559851e03090 0x559851e03470 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f88300c95b0 tx=0x7f8830098d10 comp rx=0 tx=0).stop 2024-02-02T15:30:46.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.890+0000 7f885ad5fb80 1 -- 172.21.15.1:0/599421226 shutdown_connections 2024-02-02T15:30:46.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.890+0000 7f885ad5fb80 1 --2- 172.21.15.1:0/599421226 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559851e0e860 0x559851e10c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:46.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.890+0000 7f885ad5fb80 1 --2- 172.21.15.1:0/599421226 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559851e03090 0x559851e03470 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:46.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.890+0000 7f885ad5fb80 1 --2- 172.21.15.1:0/599421226 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559851e039b0 0x559851e0e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:46.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.890+0000 7f885ad5fb80 1 -- 172.21.15.1:0/599421226 >> 172.21.15.1:0/599421226 conn(0x559851d62e30 msgr2=0x559851dff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:46.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.890+0000 7f885ad5fb80 1 -- 172.21.15.1:0/599421226 shutdown_connections 2024-02-02T15:30:46.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.890+0000 7f885ad5fb80 1 -- 172.21.15.1:0/599421226 wait complete. 2024-02-02T15:30:46.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.891+0000 7f885ad5fb80 1 -- start start 2024-02-02T15:30:46.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.891+0000 7f885ad5fb80 1 Processor -- start 2024-02-02T15:30:46.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.891+0000 7f885ad5fb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559851e03090 0x559851e0e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:46.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.891+0000 7f885ad5fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559851e039b0 0x559851e0b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:46.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.891+0000 7f885ad5fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559851e0e860 0x559851e0ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:46.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.891+0000 7f885ad5fb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x559851d54dd0 con 0x559851e03090 2024-02-02T15:30:46.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.891+0000 7f885ad5fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x559851cf0e00 con 0x559851e039b0 2024-02-02T15:30:46.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.891+0000 7f885ad5fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559851cf0aa0 con 0x559851e0e860 2024-02-02T15:30:46.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.892+0000 7f885ad5fb80 10 client.0 _ll_register_callbacks cb 0x559851cf2310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:30:46.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.892+0000 7f883fe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559851e03090 0x559851e0e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:46.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.892+0000 7f883fe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559851e03090 0x559851e0e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45288/0 (socket says 172.21.15.1:45288) 2024-02-02T15:30:46.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.892+0000 7f883fe00700 1 -- 172.21.15.1:0/1818202483 learned_addr learned my addr 172.21.15.1:0/1818202483 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:30:46.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.892+0000 7f8845200700 1 --2- 172.21.15.1:0/1818202483 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559851e0e860 0x559851e0ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:46.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.892+0000 7f883f400700 1 --2- 172.21.15.1:0/1818202483 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559851e039b0 0x559851e0b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:46.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.892+0000 7f883fe00700 1 -- 172.21.15.1:0/1818202483 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559851e0e860 msgr2=0x559851e0ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:46.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.892+0000 7f883fe00700 1 --2- 172.21.15.1:0/1818202483 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559851e0e860 0x559851e0ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:46.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.892+0000 7f883fe00700 1 -- 172.21.15.1:0/1818202483 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559851e039b0 msgr2=0x559851e0b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:46.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.892+0000 7f883fe00700 1 --2- 172.21.15.1:0/1818202483 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559851e039b0 0x559851e0b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:46.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.892+0000 7f883fe00700 1 -- 172.21.15.1:0/1818202483 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8840114b30 con 0x559851e03090 2024-02-02T15:30:46.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.892+0000 7f8845200700 1 --2- 172.21.15.1:0/1818202483 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559851e0e860 0x559851e0ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:30:46.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.892+0000 7f883fe00700 1 --2- 172.21.15.1:0/1818202483 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559851e03090 0x559851e0e1e0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f883001e000 tx=0x7f88300841a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9b8b7ea7585991a server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:46.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.892+0000 7f882e000700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:30:46.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.893+0000 7f882e000700 1 -- 172.21.15.1:0/1818202483 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88300993b0 con 0x559851e03090 2024-02-02T15:30:46.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.893+0000 7f882e000700 1 -- 172.21.15.1:0/1818202483 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f88300a1420 con 0x559851e03090 2024-02-02T15:30:46.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.893+0000 7f885ad5fb80 1 -- 172.21.15.1:0/1818202483 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559851df7170 con 0x559851e03090 2024-02-02T15:30:46.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.893+0000 7f882e000700 1 -- 172.21.15.1:0/1818202483 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8830099820 con 0x559851e03090 2024-02-02T15:30:46.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.893+0000 7f885ad5fb80 10 client.16632 fetch_fsmap learned FSMap version 31 2024-02-02T15:30:46.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.893+0000 7f885ad5fb80 1 -- 172.21.15.1:0/1818202483 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f88401aa3a0 con 0x559851e03090 2024-02-02T15:30:46.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.893+0000 7f882e000700 1 -- 172.21.15.1:0/1818202483 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559851df7170 con 0x559851e03090 2024-02-02T15:30:46.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.893+0000 7f882e000700 1 -- 172.21.15.1:0/1818202483 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 31) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f88300b7080 con 0x559851e03090 2024-02-02T15:30:46.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.893+0000 7f885ad5fb80 10 client.16632 fetch_fsmap finished waiting for FSMap version 31 2024-02-02T15:30:46.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.893+0000 7f885ad5fb80 10 client.16632 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:30:46.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.893+0000 7f885ad5fb80 20 client.16632 populate_metadata read hostname 'smithi001' 2024-02-02T15:30:46.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.893+0000 7f885ad5fb80 1 --2- 172.21.15.1:0/1818202483 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x559851e1df90 0x559851e203b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:46.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.893+0000 7f885ad5fb80 4 client.16632 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:30:46.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.893+0000 7f885ad5fb80 1 -- 172.21.15.1:0/1818202483 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559851d4de50 con 0x559851e1df90 2024-02-02T15:30:46.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.894+0000 7f883f400700 1 --2- 172.21.15.1:0/1818202483 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x559851e1df90 0x559851e203b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:46.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.894+0000 7f883f400700 1 --2- 172.21.15.1:0/1818202483 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x559851e1df90 0x559851e203b0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a651cb456eac0a62 server_cookie=9a9f4fcc37e520d in_seq=0 out_seq=0 2024-02-02T15:30:46.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.894+0000 7f882e000700 10 client.16632 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:30:46.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.895+0000 7f882e000700 1 -- 172.21.15.1:0/1818202483 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x559851d4de50 con 0x559851e1df90 2024-02-02T15:30:46.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.895+0000 7f882e000700 10 client.16632 handle_command_reply: tid=0 2024-02-02T15:30:46.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.895+0000 7f885ad5fb80 1 client.16632 shutdown 2024-02-02T15:30:46.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.895+0000 7f885ad5fb80 1 -- 172.21.15.1:0/1818202483 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559851e03090 msgr2=0x559851e0e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:46.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.895+0000 7f885ad5fb80 1 --2- 172.21.15.1:0/1818202483 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559851e03090 0x559851e0e1e0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f883001e000 tx=0x7f88300841a0 comp rx=0 tx=0).stop 2024-02-02T15:30:46.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.895+0000 7f885ad5fb80 1 -- 172.21.15.1:0/1818202483 shutdown_connections 2024-02-02T15:30:46.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.895+0000 7f885ad5fb80 1 --2- 172.21.15.1:0/1818202483 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x559851e1df90 0x559851e203b0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:46.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.895+0000 7f885ad5fb80 1 --2- 172.21.15.1:0/1818202483 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559851e0e860 0x559851e0ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:46.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.895+0000 7f885ad5fb80 1 --2- 172.21.15.1:0/1818202483 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559851e03090 0x559851e0e1e0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:46.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.895+0000 7f882e000700 0 client.16632 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:30:46.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.895+0000 7f885ad5fb80 1 --2- 172.21.15.1:0/1818202483 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559851e039b0 0x559851e0b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:46.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.895+0000 7f885ad5fb80 1 -- 172.21.15.1:0/1818202483 >> 172.21.15.1:0/1818202483 conn(0x559851d62e30 msgr2=0x559851dfefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:46.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.895+0000 7f885ad5fb80 1 -- 172.21.15.1:0/1818202483 shutdown_connections 2024-02-02T15:30:46.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.895+0000 7f885ad5fb80 1 -- 172.21.15.1:0/1818202483 wait complete. 2024-02-02T15:30:46.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.895+0000 7f885ad5fb80 20 client.16632 trim_cache size 0 max 16384 2024-02-02T15:30:46.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.913+0000 7f885ad5fb80 1 Processor -- start 2024-02-02T15:30:46.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.914+0000 7f885ad5fb80 1 -- start start 2024-02-02T15:30:46.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.914+0000 7f885ad5fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559851e03090 0x559851e15350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:46.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.914+0000 7f885ad5fb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559851e039b0 0x559851e086f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:46.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.914+0000 7f885ad5fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559851e0e860 0x559851e08c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:46.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.914+0000 7f885ad5fb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x559851bd8720 con 0x559851e039b0 2024-02-02T15:30:46.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.914+0000 7f885ad5fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x559851e048b0 con 0x559851e03090 2024-02-02T15:30:46.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.914+0000 7f885ad5fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559851e0ca40 con 0x559851e0e860 2024-02-02T15:30:46.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.914+0000 7f883fe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559851e03090 0x559851e15350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:46.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.914+0000 7f883f400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559851e039b0 0x559851e086f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:46.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.914+0000 7f8845200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559851e0e860 0x559851e08c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:46.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.914+0000 7f883fe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559851e03090 0x559851e15350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:38898/0 (socket says 172.21.15.1:38898) 2024-02-02T15:30:46.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.914+0000 7f883fe00700 1 -- 172.21.15.1:0/417655324 learned_addr learned my addr 172.21.15.1:0/417655324 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:30:46.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.915+0000 7f883f400700 1 -- 172.21.15.1:0/417655324 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559851e0e860 msgr2=0x559851e08c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:46.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.915+0000 7f883f400700 1 --2- 172.21.15.1:0/417655324 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559851e0e860 0x559851e08c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:46.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.915+0000 7f883f400700 1 -- 172.21.15.1:0/417655324 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559851e03090 msgr2=0x559851e15350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:46.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.915+0000 7f883f400700 1 --2- 172.21.15.1:0/417655324 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559851e03090 0x559851e15350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:46.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.915+0000 7f883f400700 1 -- 172.21.15.1:0/417655324 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559851b82be0 con 0x559851e039b0 2024-02-02T15:30:46.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.915+0000 7f883fe00700 1 --2- 172.21.15.1:0/417655324 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559851e03090 0x559851e15350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:30:46.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.915+0000 7f883f400700 1 --2- 172.21.15.1:0/417655324 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559851e039b0 0x559851e086f0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x559851e0ca10 tx=0x7f8828020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dff5584dc91ee6c3 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:46.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.915+0000 7f8836000700 1 -- 172.21.15.1:0/417655324 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f882802d040 con 0x559851e039b0 2024-02-02T15:30:46.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.915+0000 7f8836000700 1 -- 172.21.15.1:0/417655324 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f882802e040 con 0x559851e039b0 2024-02-02T15:30:46.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.915+0000 7f8836000700 1 -- 172.21.15.1:0/417655324 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f882800f7e0 con 0x559851e039b0 2024-02-02T15:30:46.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.916+0000 7f885ad5fb80 1 -- 172.21.15.1:0/417655324 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559851e039b0 msgr2=0x559851e086f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:46.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.916+0000 7f885ad5fb80 1 --2- 172.21.15.1:0/417655324 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559851e039b0 0x559851e086f0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x559851e0ca10 tx=0x7f8828020000 comp rx=0 tx=0).stop 2024-02-02T15:30:46.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.916+0000 7f885ad5fb80 1 -- 172.21.15.1:0/417655324 shutdown_connections 2024-02-02T15:30:46.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.916+0000 7f885ad5fb80 1 --2- 172.21.15.1:0/417655324 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559851e0e860 0x559851e08c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:46.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.916+0000 7f885ad5fb80 1 --2- 172.21.15.1:0/417655324 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559851e03090 0x559851e15350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:46.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.916+0000 7f885ad5fb80 1 --2- 172.21.15.1:0/417655324 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559851e039b0 0x559851e086f0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:46.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.916+0000 7f885ad5fb80 1 -- 172.21.15.1:0/417655324 >> 172.21.15.1:0/417655324 conn(0x559851d62e30 msgr2=0x559851e0cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:46.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.916+0000 7f885ad5fb80 1 -- 172.21.15.1:0/417655324 shutdown_connections 2024-02-02T15:30:46.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.916+0000 7f885ad5fb80 1 -- 172.21.15.1:0/417655324 wait complete. 2024-02-02T15:30:46.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.916+0000 7f885ad5fb80 1 -- start start 2024-02-02T15:30:46.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.917+0000 7f885ad5fb80 1 Processor -- start 2024-02-02T15:30:46.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.917+0000 7f885ad5fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559851e03090 0x559851e14000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:46.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.917+0000 7f885ad5fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559851e039b0 0x559851e14540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:46.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.917+0000 7f885ad5fb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559851e0e860 0x559851e69a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:46.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.917+0000 7f885ad5fb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x559851e69f90 con 0x559851e0e860 2024-02-02T15:30:46.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.917+0000 7f885ad5fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x559851e6a100 con 0x559851e039b0 2024-02-02T15:30:46.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.917+0000 7f885ad5fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559851e6a2c0 con 0x559851e03090 2024-02-02T15:30:46.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.917+0000 7f885ad5fb80 10 client.0 _ll_register_callbacks cb 0x559851cf2310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:30:46.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.918+0000 7f883f400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559851e039b0 0x559851e14540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:46.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.918+0000 7f883fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559851e03090 0x559851e14000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:46.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.918+0000 7f8845200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559851e0e860 0x559851e69a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:46.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.918+0000 7f883f400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559851e039b0 0x559851e14540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:38904/0 (socket says 172.21.15.1:38904) 2024-02-02T15:30:46.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.918+0000 7f883fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559851e03090 0x559851e14000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:41184/0 (socket says 172.21.15.1:41184) 2024-02-02T15:30:46.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.918+0000 7f883f400700 1 -- 172.21.15.1:0/169686615 learned_addr learned my addr 172.21.15.1:0/169686615 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:30:46.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.918+0000 7f8845200700 1 -- 172.21.15.1:0/169686615 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559851e03090 msgr2=0x559851e14000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:46.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.918+0000 7f8845200700 1 --2- 172.21.15.1:0/169686615 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559851e03090 0x559851e14000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:46.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.918+0000 7f8845200700 1 -- 172.21.15.1:0/169686615 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559851e039b0 msgr2=0x559851e14540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:46.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.918+0000 7f8845200700 1 --2- 172.21.15.1:0/169686615 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559851e039b0 0x559851e14540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:46.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.918+0000 7f8845200700 1 -- 172.21.15.1:0/169686615 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88401aa3a0 con 0x559851e0e860 2024-02-02T15:30:46.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.918+0000 7f883f400700 1 --2- 172.21.15.1:0/169686615 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559851e039b0 0x559851e14540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:30:46.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.918+0000 7f8845200700 1 --2- 172.21.15.1:0/169686615 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559851e0e860 0x559851e69a50 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f884006a330 tx=0x7f88380077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=869ff3a143ea252c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:46.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.918+0000 7f882e000700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:30:46.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.919+0000 7f882e000700 1 -- 172.21.15.1:0/169686615 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f883801c070 con 0x559851e0e860 2024-02-02T15:30:46.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.919+0000 7f885ad5fb80 1 -- 172.21.15.1:0/169686615 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f883009d1e0 con 0x559851e0e860 2024-02-02T15:30:46.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.919+0000 7f882e000700 1 -- 172.21.15.1:0/169686615 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f8838004bb0 con 0x559851e0e860 2024-02-02T15:30:46.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.919+0000 7f882e000700 1 -- 172.21.15.1:0/169686615 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f883800e040 con 0x559851e0e860 2024-02-02T15:30:46.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.919+0000 7f885ad5fb80 10 client.16638 fetch_fsmap learned FSMap version 31 2024-02-02T15:30:46.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.919+0000 7f885ad5fb80 1 -- 172.21.15.1:0/169686615 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x7f882800beb0 con 0x559851e0e860 2024-02-02T15:30:46.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.919+0000 7f882e000700 1 -- 172.21.15.1:0/169686615 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f883009d1e0 con 0x559851e0e860 2024-02-02T15:30:46.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.919+0000 7f882e000700 1 -- 172.21.15.1:0/169686615 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 31) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f8838004630 con 0x559851e0e860 2024-02-02T15:30:46.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.919+0000 7f885ad5fb80 10 client.16638 fetch_fsmap finished waiting for FSMap version 31 2024-02-02T15:30:46.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.919+0000 7f885ad5fb80 10 client.16638 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:30:46.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.919+0000 7f885ad5fb80 20 client.16638 populate_metadata read hostname 'smithi001' 2024-02-02T15:30:46.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.919+0000 7f885ad5fb80 1 --2- 172.21.15.1:0/169686615 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x559851e71bd0 0x559851e73ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:46.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.919+0000 7f885ad5fb80 4 client.16638 mds_command: new command op to 24557 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-02T15:30:46.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.919+0000 7f885ad5fb80 1 -- 172.21.15.1:0/169686615 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x559851d13fd0 con 0x559851e71bd0 2024-02-02T15:30:46.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.920+0000 7f883fe00700 1 --2- 172.21.15.1:0/169686615 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x559851e71bd0 0x559851e73ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:46.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.920+0000 7f883fe00700 1 --2- 172.21.15.1:0/169686615 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x559851e71bd0 0x559851e73ff0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46595cd310561cc0 server_cookie=84024e86d6dbd954 in_seq=0 out_seq=0 2024-02-02T15:30:46.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.920+0000 7f882e000700 10 client.16638 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:30:46.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.921+0000 7f882e000700 1 -- 172.21.15.1:0/169686615 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x559851d13fd0 con 0x559851e71bd0 2024-02-02T15:30:46.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.921+0000 7f882e000700 10 client.16638 handle_command_reply: tid=0 2024-02-02T15:30:46.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.921+0000 7f885ad5fb80 1 client.16638 shutdown 2024-02-02T15:30:46.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.921+0000 7f885ad5fb80 1 -- 172.21.15.1:0/169686615 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559851e0e860 msgr2=0x559851e69a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:46.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.921+0000 7f885ad5fb80 1 --2- 172.21.15.1:0/169686615 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559851e0e860 0x559851e69a50 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f884006a330 tx=0x7f88380077e0 comp rx=0 tx=0).stop 2024-02-02T15:30:46.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.921+0000 7f885ad5fb80 1 -- 172.21.15.1:0/169686615 shutdown_connections 2024-02-02T15:30:46.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.921+0000 7f885ad5fb80 1 --2- 172.21.15.1:0/169686615 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x559851e71bd0 0x559851e73ff0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:46.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.921+0000 7f885ad5fb80 1 --2- 172.21.15.1:0/169686615 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559851e0e860 0x559851e69a50 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:46.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.921+0000 7f885ad5fb80 1 --2- 172.21.15.1:0/169686615 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559851e03090 0x559851e14000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:46.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.921+0000 7f885ad5fb80 1 --2- 172.21.15.1:0/169686615 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559851e039b0 0x559851e14540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:46.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.921+0000 7f885ad5fb80 1 -- 172.21.15.1:0/169686615 >> 172.21.15.1:0/169686615 conn(0x559851d62e30 msgr2=0x559851e0d600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:46.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.922+0000 7f882e000700 0 client.16638 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:30:46.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.922+0000 7f885ad5fb80 1 -- 172.21.15.1:0/169686615 shutdown_connections 2024-02-02T15:30:46.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.922+0000 7f885ad5fb80 1 -- 172.21.15.1:0/169686615 wait complete. 2024-02-02T15:30:46.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.922+0000 7f885ad5fb80 20 client.16638 trim_cache size 0 max 16384 2024-02-02T15:30:46.926 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:30:46.926 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2024-02-02T15:30:46.926 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "c998f52d-180b-4343-905b-f48ad1aab293", 2024-02-02T15:30:46.926 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2024-02-02T15:30:46.926 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:30:46.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.922+0000 7f8847200700 1 -- 172.21.15.1:0/1881146744 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f88200a0690 msgr2=0x7f88200a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:46.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.922+0000 7f8847200700 1 --2- 172.21.15.1:0/1881146744 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f88200a0690 0x7f88200a2b50 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f884019eb80 tx=0x7f882800b2a0 comp rx=0 tx=0).stop 2024-02-02T15:30:46.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.922+0000 7f8847200700 1 -- 172.21.15.1:0/1881146744 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88400730e0 msgr2=0x7f884019d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:46.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.922+0000 7f8847200700 1 --2- 172.21.15.1:0/1881146744 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88400730e0 0x7f884019d570 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f883000e9f0 tx=0x7f883000ed00 comp rx=0 tx=0).stop 2024-02-02T15:30:46.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.923+0000 7f8847200700 1 -- 172.21.15.1:0/1881146744 shutdown_connections 2024-02-02T15:30:46.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.923+0000 7f8847200700 1 --2- 172.21.15.1:0/1881146744 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f88200a0690 0x7f88200a2b50 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:46.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.923+0000 7f8847200700 1 --2- 172.21.15.1:0/1881146744 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8840111d90 0x7f88401a1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:46.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.923+0000 7f8847200700 1 --2- 172.21.15.1:0/1881146744 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f88400730e0 0x7f884019d570 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:46.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.923+0000 7f8847200700 1 --2- 172.21.15.1:0/1881146744 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8840073a00 0x7f884019dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:46.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.923+0000 7f8847200700 1 -- 172.21.15.1:0/1881146744 >> 172.21.15.1:0/1881146744 conn(0x7f88400fc810 msgr2=0x7f88400fd480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:46.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.923+0000 7f8847200700 1 -- 172.21.15.1:0/1881146744 shutdown_connections 2024-02-02T15:30:46.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:46.923+0000 7f8847200700 1 -- 172.21.15.1:0/1881146744 wait complete. 2024-02-02T15:30:46.942 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-02T15:30:47.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.154+0000 7f287fc00700 1 -- 172.21.15.1:0/3596239309 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28781019a0 msgr2=0x7f28781059e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:47.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.154+0000 7f287fc00700 1 --2- 172.21.15.1:0/3596239309 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28781019a0 0x7f28781059e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2860009990 tx=0x7f2860009ca0 comp rx=0 tx=0).stop 2024-02-02T15:30:47.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.154+0000 7f287fc00700 1 -- 172.21.15.1:0/3596239309 shutdown_connections 2024-02-02T15:30:47.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.154+0000 7f287fc00700 1 --2- 172.21.15.1:0/3596239309 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2878106110 0x7f287810dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:47.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.154+0000 7f287fc00700 1 --2- 172.21.15.1:0/3596239309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2878100ff0 0x7f28781013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:47.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.154+0000 7f287fc00700 1 --2- 172.21.15.1:0/3596239309 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28781019a0 0x7f28781059e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:47.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.154+0000 7f287fc00700 1 -- 172.21.15.1:0/3596239309 >> 172.21.15.1:0/3596239309 conn(0x7f28780fc810 msgr2=0x7f28780fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:47.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.154+0000 7f287fc00700 1 -- 172.21.15.1:0/3596239309 shutdown_connections 2024-02-02T15:30:47.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.154+0000 7f287fc00700 1 -- 172.21.15.1:0/3596239309 wait complete. 2024-02-02T15:30:47.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.155+0000 7f287fc00700 1 Processor -- start 2024-02-02T15:30:47.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.155+0000 7f287fc00700 1 -- start start 2024-02-02T15:30:47.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.156+0000 7f287fc00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2878100ff0 0x7f2878073040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:47.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.156+0000 7f287fc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2878106110 0x7f2878073580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:47.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.156+0000 7f287fc00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2878073ac0 0x7f28781a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:47.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.156+0000 7f287fc00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28781a5760 con 0x7f2878100ff0 2024-02-02T15:30:47.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.156+0000 7f287fc00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28781a58d0 con 0x7f2878073ac0 2024-02-02T15:30:47.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.156+0000 7f287fc00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28781a5a90 con 0x7f2878106110 2024-02-02T15:30:47.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.156+0000 7f287d200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2878100ff0 0x7f2878073040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:47.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.156+0000 7f2877e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2878106110 0x7f2878073580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:47.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.156+0000 7f287dc00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2878073ac0 0x7f28781a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:47.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.156+0000 7f287d200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2878100ff0 0x7f2878073040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45316/0 (socket says 172.21.15.1:45316) 2024-02-02T15:30:47.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.156+0000 7f287d200700 1 -- 172.21.15.1:0/3650990199 learned_addr learned my addr 172.21.15.1:0/3650990199 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:30:47.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.156+0000 7f2877e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2878106110 0x7f2878073580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:41198/0 (socket says 172.21.15.1:41198) 2024-02-02T15:30:47.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.156+0000 7f287d200700 1 -- 172.21.15.1:0/3650990199 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2878106110 msgr2=0x7f2878073580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:47.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.156+0000 7f287d200700 1 --2- 172.21.15.1:0/3650990199 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2878106110 0x7f2878073580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:47.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.156+0000 7f287d200700 1 -- 172.21.15.1:0/3650990199 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2878073ac0 msgr2=0x7f28781a50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:47.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.156+0000 7f287d200700 1 --2- 172.21.15.1:0/3650990199 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2878073ac0 0x7f28781a50d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:47.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.157+0000 7f287d200700 1 -- 172.21.15.1:0/3650990199 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28600095f0 con 0x7f2878100ff0 2024-02-02T15:30:47.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.157+0000 7f287d200700 1 --2- 172.21.15.1:0/3650990199 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2878100ff0 0x7f2878073040 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f286800e9a0 tx=0x7f286800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:47.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.157+0000 7f2875600700 1 -- 172.21.15.1:0/3650990199 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2868009790 con 0x7f2878100ff0 2024-02-02T15:30:47.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.157+0000 7f2875600700 1 -- 172.21.15.1:0/3650990199 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2868010430 con 0x7f2878100ff0 2024-02-02T15:30:47.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.157+0000 7f2875600700 1 -- 172.21.15.1:0/3650990199 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2868018680 con 0x7f2878100ff0 2024-02-02T15:30:47.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.157+0000 7f287fc00700 1 -- 172.21.15.1:0/3650990199 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28781a5d70 con 0x7f2878100ff0 2024-02-02T15:30:47.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.157+0000 7f287fc00700 1 -- 172.21.15.1:0/3650990199 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28781a61d0 con 0x7f2878100ff0 2024-02-02T15:30:47.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.160+0000 7f2875600700 1 -- 172.21.15.1:0/3650990199 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 27) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f28680105a0 con 0x7f2878100ff0 2024-02-02T15:30:47.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.162+0000 7f2875600700 1 --2- 172.21.15.1:0/3650990199 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f28580a0690 0x7f28580a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:47.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.162+0000 7f2875600700 1 -- 172.21.15.1:0/3650990199 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f2868014070 con 0x7f2878100ff0 2024-02-02T15:30:47.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.162+0000 7f289378cb80 1 Processor -- start 2024-02-02T15:30:47.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.162+0000 7f2877e00700 1 --2- 172.21.15.1:0/3650990199 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f28580a0690 0x7f28580a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:47.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.162+0000 7f289378cb80 1 -- start start 2024-02-02T15:30:47.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.163+0000 7f289378cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5618d2b37010 0x5618d2b373f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:47.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.163+0000 7f289378cb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5618d2b37930 0x5618d2b422a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:47.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.163+0000 7f289378cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5618d2b427e0 0x5618d2b44bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:47.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.163+0000 7f289378cb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618d2aab4c0 con 0x5618d2b37930 2024-02-02T15:30:47.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.163+0000 7f289378cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618d2a8faa0 con 0x5618d2b427e0 2024-02-02T15:30:47.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.163+0000 7f289378cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618d2a8d460 con 0x5618d2b37010 2024-02-02T15:30:47.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.163+0000 7f2877e00700 1 --2- 172.21.15.1:0/3650990199 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f28580a0690 0x7f28580a2b50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2878074830 tx=0x7f2860005fb0 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:47.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.163+0000 7f287d200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5618d2b37010 0x5618d2b373f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:47.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.163+0000 7f2877e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5618d2b37930 0x5618d2b422a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:47.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.163+0000 7f287d200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5618d2b37010 0x5618d2b373f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:41202/0 (socket says 172.21.15.1:41202) 2024-02-02T15:30:47.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.163+0000 7f287d200700 1 -- 172.21.15.1:0/2418263523 learned_addr learned my addr 172.21.15.1:0/2418263523 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:30:47.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.163+0000 7f287dc00700 1 --2- 172.21.15.1:0/2418263523 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5618d2b427e0 0x5618d2b44bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:47.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.163+0000 7f287d200700 1 -- 172.21.15.1:0/2418263523 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5618d2b427e0 msgr2=0x5618d2b44bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:47.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.163+0000 7f287d200700 1 --2- 172.21.15.1:0/2418263523 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5618d2b427e0 0x5618d2b44bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:47.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.163+0000 7f287d200700 1 -- 172.21.15.1:0/2418263523 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5618d2b37930 msgr2=0x5618d2b422a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:47.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.163+0000 7f287d200700 1 --2- 172.21.15.1:0/2418263523 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5618d2b37930 0x5618d2b422a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:47.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.163+0000 7f287d200700 1 -- 172.21.15.1:0/2418263523 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5618d290a3b0 con 0x5618d2b37010 2024-02-02T15:30:47.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.163+0000 7f2877e00700 1 --2- 172.21.15.1:0/2418263523 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5618d2b37930 0x5618d2b422a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:30:47.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.164+0000 7f287d200700 1 --2- 172.21.15.1:0/2418263523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5618d2b37010 0x5618d2b373f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f28680993e0 tx=0x7f2868098e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f46848dc0371706e server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:47.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.164+0000 7f286e000700 1 -- 172.21.15.1:0/2418263523 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2868099450 con 0x5618d2b37010 2024-02-02T15:30:47.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.164+0000 7f286e000700 1 -- 172.21.15.1:0/2418263523 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f286809d280 con 0x5618d2b37010 2024-02-02T15:30:47.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.164+0000 7f286e000700 1 -- 172.21.15.1:0/2418263523 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2868099910 con 0x5618d2b37010 2024-02-02T15:30:47.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.164+0000 7f289378cb80 1 -- 172.21.15.1:0/2418263523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5618d2b37010 msgr2=0x5618d2b373f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:47.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.164+0000 7f289378cb80 1 --2- 172.21.15.1:0/2418263523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5618d2b37010 0x5618d2b373f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f28680993e0 tx=0x7f2868098e30 comp rx=0 tx=0).stop 2024-02-02T15:30:47.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.165+0000 7f289378cb80 1 -- 172.21.15.1:0/2418263523 shutdown_connections 2024-02-02T15:30:47.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.165+0000 7f289378cb80 1 --2- 172.21.15.1:0/2418263523 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5618d2b427e0 0x5618d2b44bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:47.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.165+0000 7f289378cb80 1 --2- 172.21.15.1:0/2418263523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5618d2b37010 0x5618d2b373f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:47.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.165+0000 7f289378cb80 1 --2- 172.21.15.1:0/2418263523 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5618d2b37930 0x5618d2b422a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:47.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.165+0000 7f289378cb80 1 -- 172.21.15.1:0/2418263523 >> 172.21.15.1:0/2418263523 conn(0x5618d2a96db0 msgr2=0x5618d2b333b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:47.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.165+0000 7f289378cb80 1 -- 172.21.15.1:0/2418263523 shutdown_connections 2024-02-02T15:30:47.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.165+0000 7f289378cb80 1 -- 172.21.15.1:0/2418263523 wait complete. 2024-02-02T15:30:47.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.165+0000 7f289378cb80 1 -- start start 2024-02-02T15:30:47.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.165+0000 7f289378cb80 1 Processor -- start 2024-02-02T15:30:47.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.166+0000 7f289378cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5618d2b37010 0x5618d2b42240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:47.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.166+0000 7f289378cb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5618d2b37930 0x5618d2b3f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:47.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.166+0000 7f289378cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5618d2b427e0 0x5618d2b3fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:47.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.166+0000 7f289378cb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618d2a88d50 con 0x5618d2b37930 2024-02-02T15:30:47.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.166+0000 7f289378cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618d2a24d80 con 0x5618d2b427e0 2024-02-02T15:30:47.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.166+0000 7f289378cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618d2a24a20 con 0x5618d2b37010 2024-02-02T15:30:47.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.166+0000 7f289378cb80 10 client.0 _ll_register_callbacks cb 0x5618d2a26290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:30:47.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.166+0000 7f287d200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5618d2b37010 0x5618d2b42240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:47.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.166+0000 7f287d200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5618d2b37010 0x5618d2b42240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:41218/0 (socket says 172.21.15.1:41218) 2024-02-02T15:30:47.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.166+0000 7f287d200700 1 -- 172.21.15.1:0/1917397790 learned_addr learned my addr 172.21.15.1:0/1917397790 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:30:47.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.166+0000 7f287dc00700 1 --2- 172.21.15.1:0/1917397790 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5618d2b427e0 0x5618d2b3fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:47.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.166+0000 7f2877e00700 1 --2- 172.21.15.1:0/1917397790 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5618d2b37930 0x5618d2b3f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:47.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.167+0000 7f287d200700 1 -- 172.21.15.1:0/1917397790 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5618d2b427e0 msgr2=0x5618d2b3fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:47.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.167+0000 7f287d200700 1 --2- 172.21.15.1:0/1917397790 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5618d2b427e0 0x5618d2b3fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:47.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.167+0000 7f287d200700 1 -- 172.21.15.1:0/1917397790 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5618d2b37930 msgr2=0x5618d2b3f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:47.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.167+0000 7f287d200700 1 --2- 172.21.15.1:0/1917397790 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5618d2b37930 0x5618d2b3f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:47.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.167+0000 7f287d200700 1 -- 172.21.15.1:0/1917397790 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28781a61d0 con 0x5618d2b37010 2024-02-02T15:30:47.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.167+0000 7f287d200700 1 --2- 172.21.15.1:0/1917397790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5618d2b37010 0x5618d2b42240 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2868095900 tx=0x7f2868097a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1455ddbfd14bb003 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:47.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.167+0000 7f2866a00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:30:47.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.167+0000 7f2866a00700 1 -- 172.21.15.1:0/1917397790 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28680998b0 con 0x5618d2b37010 2024-02-02T15:30:47.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.167+0000 7f2866a00700 1 -- 172.21.15.1:0/1917397790 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f28680a14c0 con 0x5618d2b37010 2024-02-02T15:30:47.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.167+0000 7f289378cb80 1 -- 172.21.15.1:0/1917397790 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5618d2b2b0f0 con 0x5618d2b37010 2024-02-02T15:30:47.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.167+0000 7f2866a00700 1 -- 172.21.15.1:0/1917397790 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2868099450 con 0x5618d2b37010 2024-02-02T15:30:47.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.167+0000 7f289378cb80 10 client.26701 fetch_fsmap learned FSMap version 31 2024-02-02T15:30:47.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.167+0000 7f289378cb80 1 -- 172.21.15.1:0/1917397790 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5618d28b6b60 con 0x5618d2b37010 2024-02-02T15:30:47.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.168+0000 7f2866a00700 1 -- 172.21.15.1:0/1917397790 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5618d2b2b0f0 con 0x5618d2b37010 2024-02-02T15:30:47.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.168+0000 7f2866a00700 1 -- 172.21.15.1:0/1917397790 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 31) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f28680a10c0 con 0x5618d2b37010 2024-02-02T15:30:47.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.168+0000 7f289378cb80 10 client.26701 fetch_fsmap finished waiting for FSMap version 31 2024-02-02T15:30:47.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.168+0000 7f289378cb80 10 client.26701 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:30:47.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.168+0000 7f289378cb80 20 client.26701 populate_metadata read hostname 'smithi001' 2024-02-02T15:30:47.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.168+0000 7f289378cb80 1 --2- 172.21.15.1:0/1917397790 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5618d2b52060 0x5618d2b54480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:47.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.168+0000 7f289378cb80 4 client.26701 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:30:47.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.168+0000 7f289378cb80 1 -- 172.21.15.1:0/1917397790 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5618d2a81dd0 con 0x5618d2b52060 2024-02-02T15:30:47.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.168+0000 7f2877e00700 1 --2- 172.21.15.1:0/1917397790 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5618d2b52060 0x5618d2b54480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:47.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.169+0000 7f2877e00700 1 --2- 172.21.15.1:0/1917397790 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5618d2b52060 0x5618d2b54480 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a14ba92257848d9 server_cookie=bb1e736babe06e59 in_seq=0 out_seq=0 2024-02-02T15:30:47.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.169+0000 7f2866a00700 10 client.26701 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:30:47.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.169+0000 7f2866a00700 1 -- 172.21.15.1:0/1917397790 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5618d2a81dd0 con 0x5618d2b52060 2024-02-02T15:30:47.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.169+0000 7f2866a00700 10 client.26701 handle_command_reply: tid=0 2024-02-02T15:30:47.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.169+0000 7f289378cb80 1 client.26701 shutdown 2024-02-02T15:30:47.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.170+0000 7f289378cb80 1 -- 172.21.15.1:0/1917397790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5618d2b37010 msgr2=0x5618d2b42240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:47.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.170+0000 7f289378cb80 1 --2- 172.21.15.1:0/1917397790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5618d2b37010 0x5618d2b42240 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2868095900 tx=0x7f2868097a00 comp rx=0 tx=0).stop 2024-02-02T15:30:47.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.170+0000 7f289378cb80 1 -- 172.21.15.1:0/1917397790 shutdown_connections 2024-02-02T15:30:47.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.170+0000 7f289378cb80 1 --2- 172.21.15.1:0/1917397790 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5618d2b52060 0x5618d2b54480 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:47.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.170+0000 7f289378cb80 1 --2- 172.21.15.1:0/1917397790 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5618d2b427e0 0x5618d2b3fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:47.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.170+0000 7f289378cb80 1 --2- 172.21.15.1:0/1917397790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5618d2b37010 0x5618d2b42240 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:47.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.170+0000 7f289378cb80 1 --2- 172.21.15.1:0/1917397790 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5618d2b37930 0x5618d2b3f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:47.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.170+0000 7f2866a00700 0 client.26701 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:30:47.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.170+0000 7f289378cb80 1 -- 172.21.15.1:0/1917397790 >> 172.21.15.1:0/1917397790 conn(0x5618d2a96db0 msgr2=0x5618d2b32fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:47.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.170+0000 7f289378cb80 1 -- 172.21.15.1:0/1917397790 shutdown_connections 2024-02-02T15:30:47.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.170+0000 7f289378cb80 1 -- 172.21.15.1:0/1917397790 wait complete. 2024-02-02T15:30:47.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.170+0000 7f289378cb80 20 client.26701 trim_cache size 0 max 16384 2024-02-02T15:30:47.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.187+0000 7f289378cb80 1 Processor -- start 2024-02-02T15:30:47.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.188+0000 7f289378cb80 1 -- start start 2024-02-02T15:30:47.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.188+0000 7f289378cb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5618d2a96db0 0x5618d2b49330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:47.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.188+0000 7f289378cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5618d2b37010 0x5618d2b39ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:47.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.188+0000 7f289378cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5618d2b37930 0x5618d2b3a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:47.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.188+0000 7f289378cb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618d290c6a0 con 0x5618d2a96db0 2024-02-02T15:30:47.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.188+0000 7f289378cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618d2b34240 con 0x5618d2b37930 2024-02-02T15:30:47.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.188+0000 7f289378cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618d2b38830 con 0x5618d2b37010 2024-02-02T15:30:47.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.188+0000 7f287d200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5618d2a96db0 0x5618d2b49330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:47.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.188+0000 7f2877e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5618d2b37010 0x5618d2b39ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:47.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.188+0000 7f287dc00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5618d2b37930 0x5618d2b3a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:47.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.188+0000 7f2877e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5618d2b37010 0x5618d2b39ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:41220/0 (socket says 172.21.15.1:41220) 2024-02-02T15:30:47.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.188+0000 7f2877e00700 1 -- 172.21.15.1:0/2433935069 learned_addr learned my addr 172.21.15.1:0/2433935069 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:30:47.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.188+0000 7f2877e00700 1 -- 172.21.15.1:0/2433935069 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5618d2b37930 msgr2=0x5618d2b3a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:47.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.189+0000 7f2877e00700 1 --2- 172.21.15.1:0/2433935069 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5618d2b37930 0x5618d2b3a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:47.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.189+0000 7f2877e00700 1 -- 172.21.15.1:0/2433935069 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5618d2a96db0 msgr2=0x5618d2b49330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:47.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.189+0000 7f2877e00700 1 --2- 172.21.15.1:0/2433935069 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5618d2a96db0 0x5618d2b49330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:47.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.189+0000 7f2877e00700 1 -- 172.21.15.1:0/2433935069 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5618d2a4eea0 con 0x5618d2b37010 2024-02-02T15:30:47.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.189+0000 7f2877e00700 1 --2- 172.21.15.1:0/2433935069 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5618d2b37010 0x5618d2b39ac0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f287806a330 tx=0x7f2860020000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f84894bde91c09bc server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:47.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.189+0000 7f286ea00700 1 -- 172.21.15.1:0/2433935069 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f286000bd70 con 0x5618d2b37010 2024-02-02T15:30:47.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.189+0000 7f286ea00700 1 -- 172.21.15.1:0/2433935069 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f286002a030 con 0x5618d2b37010 2024-02-02T15:30:47.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.189+0000 7f286ea00700 1 -- 172.21.15.1:0/2433935069 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2860004760 con 0x5618d2b37010 2024-02-02T15:30:47.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.190+0000 7f289378cb80 1 -- 172.21.15.1:0/2433935069 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5618d2b37010 msgr2=0x5618d2b39ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:47.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.190+0000 7f289378cb80 1 --2- 172.21.15.1:0/2433935069 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5618d2b37010 0x5618d2b39ac0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f287806a330 tx=0x7f2860020000 comp rx=0 tx=0).stop 2024-02-02T15:30:47.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.190+0000 7f289378cb80 1 -- 172.21.15.1:0/2433935069 shutdown_connections 2024-02-02T15:30:47.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.190+0000 7f289378cb80 1 --2- 172.21.15.1:0/2433935069 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5618d2b37930 0x5618d2b3a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:47.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.190+0000 7f289378cb80 1 --2- 172.21.15.1:0/2433935069 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5618d2a96db0 0x5618d2b49330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:47.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.190+0000 7f289378cb80 1 --2- 172.21.15.1:0/2433935069 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5618d2b37010 0x5618d2b39ac0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:47.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.190+0000 7f289378cb80 1 -- 172.21.15.1:0/2433935069 >> 172.21.15.1:0/2433935069 conn(0x5618d2b3f080 msgr2=0x5618d2b3a840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:47.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.190+0000 7f289378cb80 1 -- 172.21.15.1:0/2433935069 shutdown_connections 2024-02-02T15:30:47.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.190+0000 7f289378cb80 1 -- 172.21.15.1:0/2433935069 wait complete. 2024-02-02T15:30:47.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.190+0000 7f289378cb80 1 -- start start 2024-02-02T15:30:47.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.191+0000 7f289378cb80 1 Processor -- start 2024-02-02T15:30:47.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.191+0000 7f289378cb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5618d2a96db0 0x5618d2b48f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:47.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.191+0000 7f289378cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5618d2b37010 0x5618d2b47310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:47.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.191+0000 7f289378cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5618d2b37930 0x5618d2b47850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:47.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.191+0000 7f289378cb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618d2b47d90 con 0x5618d2a96db0 2024-02-02T15:30:47.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.191+0000 7f289378cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618d2b47f00 con 0x5618d2b37010 2024-02-02T15:30:47.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.191+0000 7f289378cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618d2b480c0 con 0x5618d2b37930 2024-02-02T15:30:47.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.191+0000 7f289378cb80 10 client.0 _ll_register_callbacks cb 0x5618d2a26290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:30:47.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.191+0000 7f287d200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5618d2a96db0 0x5618d2b48f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:47.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.191+0000 7f287d200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5618d2a96db0 0x5618d2b48f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45358/0 (socket says 172.21.15.1:45358) 2024-02-02T15:30:47.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.191+0000 7f287d200700 1 -- 172.21.15.1:0/1603567836 learned_addr learned my addr 172.21.15.1:0/1603567836 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:30:47.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.191+0000 7f2877e00700 1 --2- 172.21.15.1:0/1603567836 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5618d2b37010 0x5618d2b47310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:47.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.191+0000 7f287dc00700 1 --2- 172.21.15.1:0/1603567836 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5618d2b37930 0x5618d2b47850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:47.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.192+0000 7f287d200700 1 -- 172.21.15.1:0/1603567836 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5618d2b37930 msgr2=0x5618d2b47850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:47.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.192+0000 7f287d200700 1 --2- 172.21.15.1:0/1603567836 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5618d2b37930 0x5618d2b47850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:47.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.192+0000 7f287d200700 1 -- 172.21.15.1:0/1603567836 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5618d2b37010 msgr2=0x5618d2b47310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:30:47.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.192+0000 7f287d200700 1 --2- 172.21.15.1:0/1603567836 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5618d2b37010 0x5618d2b47310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:47.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.192+0000 7f287d200700 1 -- 172.21.15.1:0/1603567836 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f287810e500 con 0x5618d2a96db0 2024-02-02T15:30:47.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.192+0000 7f287d200700 1 --2- 172.21.15.1:0/1603567836 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5618d2a96db0 0x5618d2b48f00 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f2868095ec0 tx=0x7f28680cc210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb9346373585df7 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:30:47.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.192+0000 7f2866a00700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:30:47.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.192+0000 7f2866a00700 1 -- 172.21.15.1:0/1603567836 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28680a1240 con 0x5618d2a96db0 2024-02-02T15:30:47.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.192+0000 7f2866a00700 1 -- 172.21.15.1:0/1603567836 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f28680a14c0 con 0x5618d2a96db0 2024-02-02T15:30:47.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.192+0000 7f289378cb80 1 -- 172.21.15.1:0/1603567836 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f286809d280 con 0x5618d2a96db0 2024-02-02T15:30:47.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.193+0000 7f2866a00700 1 -- 172.21.15.1:0/1603567836 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2868099450 con 0x5618d2a96db0 2024-02-02T15:30:47.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.193+0000 7f289378cb80 10 client.16656 fetch_fsmap learned FSMap version 31 2024-02-02T15:30:47.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.193+0000 7f289378cb80 1 -- 172.21.15.1:0/1603567836 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5618d290ecd0 con 0x5618d2a96db0 2024-02-02T15:30:47.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.193+0000 7f2866a00700 1 -- 172.21.15.1:0/1603567836 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f286809d280 con 0x5618d2a96db0 2024-02-02T15:30:47.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.193+0000 7f2866a00700 1 -- 172.21.15.1:0/1603567836 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 31) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f286809dfc0 con 0x5618d2a96db0 2024-02-02T15:30:47.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.193+0000 7f289378cb80 10 client.16656 fetch_fsmap finished waiting for FSMap version 31 2024-02-02T15:30:47.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.193+0000 7f289378cb80 10 client.16656 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:30:47.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.193+0000 7f289378cb80 20 client.16656 populate_metadata read hostname 'smithi001' 2024-02-02T15:30:47.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.193+0000 7f289378cb80 1 --2- 172.21.15.1:0/1603567836 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5618d2b427e0 0x5618d2ba6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:30:47.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.193+0000 7f289378cb80 4 client.16656 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2024-02-02T15:30:47.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.193+0000 7f289378cb80 1 -- 172.21.15.1:0/1603567836 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5618d2a47f50 con 0x5618d2b427e0 2024-02-02T15:30:47.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.193+0000 7f2877e00700 1 --2- 172.21.15.1:0/1603567836 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5618d2b427e0 0x5618d2ba6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:30:47.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.194+0000 7f2877e00700 1 --2- 172.21.15.1:0/1603567836 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5618d2b427e0 0x5618d2ba6bb0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c54e0089f2fc981 server_cookie=a4d272b7558b8427 in_seq=0 out_seq=0 2024-02-02T15:30:47.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.194+0000 7f2866a00700 10 client.16656 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:30:47.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.194+0000 7f2866a00700 1 -- 172.21.15.1:0/1603567836 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5618d2a47f50 con 0x5618d2b427e0 2024-02-02T15:30:47.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.194+0000 7f2866a00700 10 client.16656 handle_command_reply: tid=0 2024-02-02T15:30:47.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.194+0000 7f289378cb80 1 client.16656 shutdown 2024-02-02T15:30:47.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.194+0000 7f289378cb80 1 -- 172.21.15.1:0/1603567836 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5618d2a96db0 msgr2=0x5618d2b48f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:47.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.194+0000 7f289378cb80 1 --2- 172.21.15.1:0/1603567836 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5618d2a96db0 0x5618d2b48f00 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f2868095ec0 tx=0x7f28680cc210 comp rx=0 tx=0).stop 2024-02-02T15:30:47.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.194+0000 7f289378cb80 1 -- 172.21.15.1:0/1603567836 shutdown_connections 2024-02-02T15:30:47.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.194+0000 7f289378cb80 1 --2- 172.21.15.1:0/1603567836 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5618d2b427e0 0x5618d2ba6bb0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:47.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.194+0000 7f289378cb80 1 --2- 172.21.15.1:0/1603567836 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5618d2b37930 0x5618d2b47850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:47.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.194+0000 7f289378cb80 1 --2- 172.21.15.1:0/1603567836 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5618d2a96db0 0x5618d2b48f00 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:47.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.194+0000 7f2866a00700 0 client.16656 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:30:47.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.194+0000 7f289378cb80 1 --2- 172.21.15.1:0/1603567836 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5618d2b37010 0x5618d2b47310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:47.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.194+0000 7f289378cb80 1 -- 172.21.15.1:0/1603567836 >> 172.21.15.1:0/1603567836 conn(0x5618d2b3f080 msgr2=0x5618d2b4d900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:47.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.195+0000 7f289378cb80 1 -- 172.21.15.1:0/1603567836 shutdown_connections 2024-02-02T15:30:47.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.195+0000 7f289378cb80 1 -- 172.21.15.1:0/1603567836 wait complete. 2024-02-02T15:30:47.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.195+0000 7f289378cb80 20 client.16656 trim_cache size 0 max 16384 2024-02-02T15:30:47.202 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:30:47.202 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-02T15:30:47.202 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2024-02-02T15:30:47.203 INFO:teuthology.orchestra.run.smithi001.stdout: "c998f52d-180b-4343-905b-f48ad1aab293": { 2024-02-02T15:30:47.203 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2024-02-02T15:30:47.203 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "c998f52d-180b-4343-905b-f48ad1aab293", 2024-02-02T15:30:47.203 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,force" 2024-02-02T15:30:47.203 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:30:47.203 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:30:47.203 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:30:47.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.195+0000 7f287fc00700 1 -- 172.21.15.1:0/3650990199 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f28580a0690 msgr2=0x7f28580a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:47.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.195+0000 7f287fc00700 1 --2- 172.21.15.1:0/3650990199 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f28580a0690 0x7f28580a2b50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2878074830 tx=0x7f2860005fb0 comp rx=0 tx=0).stop 2024-02-02T15:30:47.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.195+0000 7f287fc00700 1 -- 172.21.15.1:0/3650990199 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2878100ff0 msgr2=0x7f2878073040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:30:47.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.195+0000 7f287fc00700 1 --2- 172.21.15.1:0/3650990199 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2878100ff0 0x7f2878073040 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f286800e9a0 tx=0x7f286800ecb0 comp rx=0 tx=0).stop 2024-02-02T15:30:47.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.195+0000 7f287fc00700 1 -- 172.21.15.1:0/3650990199 shutdown_connections 2024-02-02T15:30:47.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.195+0000 7f287fc00700 1 --2- 172.21.15.1:0/3650990199 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f28580a0690 0x7f28580a2b50 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:47.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.195+0000 7f287fc00700 1 --2- 172.21.15.1:0/3650990199 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2878073ac0 0x7f28781a50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:47.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.195+0000 7f287fc00700 1 --2- 172.21.15.1:0/3650990199 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2878100ff0 0x7f2878073040 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:47.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.195+0000 7f287fc00700 1 --2- 172.21.15.1:0/3650990199 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2878106110 0x7f2878073580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:30:47.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.195+0000 7f287fc00700 1 -- 172.21.15.1:0/3650990199 >> 172.21.15.1:0/3650990199 conn(0x7f28780fc810 msgr2=0x7f28780fd2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:30:47.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.196+0000 7f287fc00700 1 -- 172.21.15.1:0/3650990199 shutdown_connections 2024-02-02T15:30:47.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:30:47.196+0000 7f287fc00700 1 -- 172.21.15.1:0/3650990199 wait complete. 2024-02-02T15:30:47.214 INFO:tasks.cephfs.filesystem:scrub status for tag:c998f52d-180b-4343-905b-f48ad1aab293 - {'path': '/', 'tag': 'c998f52d-180b-4343-905b-f48ad1aab293', 'options': 'recursive,force'} 2024-02-02T15:30:47.228 INFO:tasks.workunit.client.0.smithi001.stdout: 1 177820 9.67 MB/sec execute 21 sec latency 11.953 ms 2024-02-02T15:30:47.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:47 smithi071 ceph-mon[44707]: pgmap v116: 129 pgs: 129 active+clean; 109 MiB data, 890 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 303 op/s 2024-02-02T15:30:47.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:47 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:30:47.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:47 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:30:47.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:47 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:30:47.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:47 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:30:47.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:47 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:30:47.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:47 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:30:47.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:47 smithi142 ceph-mon[45728]: pgmap v116: 129 pgs: 129 active+clean; 109 MiB data, 890 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 303 op/s 2024-02-02T15:30:47.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:47 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:30:47.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:47 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:30:47.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:47 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:30:47.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:47 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:30:47.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:47 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:30:47.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:47 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:30:47.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:47 smithi001 ceph-mon[35470]: pgmap v116: 129 pgs: 129 active+clean; 109 MiB data, 890 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 303 op/s 2024-02-02T15:30:47.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:47 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:30:47.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:47 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:30:47.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:47 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:30:47.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:47 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:30:47.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:47 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:30:47.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:47 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:30:48.228 INFO:tasks.workunit.client.0.smithi001.stdout: 1 179856 9.59 MB/sec execute 22 sec latency 13.757 ms 2024-02-02T15:30:48.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:48 smithi071 ceph-mon[44707]: scrub queued for path: / 2024-02-02T15:30:48.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:48 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:30:48.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:48 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:30:48.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:48 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:30:48.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:48 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:30:48.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:48 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:30:48.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:48 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:30:48.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:48 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:30:48.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:48 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:30:48.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:48 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:30:48.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:48 smithi142 ceph-mon[45728]: scrub queued for path: / 2024-02-02T15:30:48.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:48 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:30:48.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:48 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:30:48.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:48 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:30:48.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:48 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:30:48.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:48 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:30:48.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:48 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:30:48.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:48 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:30:48.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:48 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:30:48.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:48 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:30:48.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:48 smithi001 ceph-mon[35470]: scrub queued for path: / 2024-02-02T15:30:48.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:48 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:30:48.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:48 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:30:48.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:48 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:30:48.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:48 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:30:48.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:48 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:30:48.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:48 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:30:48.692 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:48 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:30:48.692 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:48 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:30:48.692 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:48 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:30:49.228 INFO:tasks.workunit.client.0.smithi001.stdout: 1 181361 9.66 MB/sec execute 23 sec latency 11.387 ms 2024-02-02T15:30:49.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:49 smithi071 ceph-mon[44707]: pgmap v117: 129 pgs: 129 active+clean; 109 MiB data, 890 MiB used, 1.0 TiB / 1.0 TiB avail; 1003 KiB/s rd, 1.7 MiB/s wr, 202 op/s 2024-02-02T15:30:49.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:49 smithi142 ceph-mon[45728]: pgmap v117: 129 pgs: 129 active+clean; 109 MiB data, 890 MiB used, 1.0 TiB / 1.0 TiB avail; 1003 KiB/s rd, 1.7 MiB/s wr, 202 op/s 2024-02-02T15:30:49.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:49 smithi001 ceph-mon[35470]: pgmap v117: 129 pgs: 129 active+clean; 109 MiB data, 890 MiB used, 1.0 TiB / 1.0 TiB avail; 1003 KiB/s rd, 1.7 MiB/s wr, 202 op/s 2024-02-02T15:30:50.228 INFO:tasks.workunit.client.0.smithi001.stdout: 1 183350 9.56 MB/sec execute 24 sec latency 44.163 ms 2024-02-02T15:30:51.228 INFO:tasks.workunit.client.0.smithi001.stdout: 1 184878 9.66 MB/sec execute 25 sec latency 15.373 ms 2024-02-02T15:30:51.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:51 smithi071 ceph-mon[44707]: pgmap v118: 129 pgs: 129 active+clean; 109 MiB data, 890 MiB used, 1.0 TiB / 1.0 TiB avail; 1003 KiB/s rd, 1.7 MiB/s wr, 202 op/s 2024-02-02T15:30:51.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:51 smithi142 ceph-mon[45728]: pgmap v118: 129 pgs: 129 active+clean; 109 MiB data, 890 MiB used, 1.0 TiB / 1.0 TiB avail; 1003 KiB/s rd, 1.7 MiB/s wr, 202 op/s 2024-02-02T15:30:51.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:51 smithi001 ceph-mon[35470]: pgmap v118: 129 pgs: 129 active+clean; 109 MiB data, 890 MiB used, 1.0 TiB / 1.0 TiB avail; 1003 KiB/s rd, 1.7 MiB/s wr, 202 op/s 2024-02-02T15:30:52.228 INFO:tasks.workunit.client.0.smithi001.stdout: 1 186858 9.55 MB/sec execute 26 sec latency 13.211 ms 2024-02-02T15:30:53.228 INFO:tasks.workunit.client.0.smithi001.stdout: 1 188427 9.65 MB/sec execute 27 sec latency 11.946 ms 2024-02-02T15:30:53.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:53 smithi071 ceph-mon[44707]: pgmap v119: 129 pgs: 129 active+clean; 115 MiB data, 903 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.7 MiB/s wr, 316 op/s 2024-02-02T15:30:53.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:53 smithi142 ceph-mon[45728]: pgmap v119: 129 pgs: 129 active+clean; 115 MiB data, 903 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.7 MiB/s wr, 316 op/s 2024-02-02T15:30:53.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:53 smithi001 ceph-mon[35470]: pgmap v119: 129 pgs: 129 active+clean; 115 MiB data, 903 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.7 MiB/s wr, 316 op/s 2024-02-02T15:30:54.229 INFO:tasks.workunit.client.0.smithi001.stdout: 1 190414 9.56 MB/sec execute 28 sec latency 18.720 ms 2024-02-02T15:30:55.229 INFO:tasks.workunit.client.0.smithi001.stdout: 1 191939 9.64 MB/sec execute 29 sec latency 43.346 ms 2024-02-02T15:30:55.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:55 smithi071 ceph-mon[44707]: pgmap v120: 129 pgs: 129 active+clean; 115 MiB data, 903 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 223 op/s 2024-02-02T15:30:55.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:55 smithi142 ceph-mon[45728]: pgmap v120: 129 pgs: 129 active+clean; 115 MiB data, 903 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 223 op/s 2024-02-02T15:30:55.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:55 smithi001 ceph-mon[35470]: pgmap v120: 129 pgs: 129 active+clean; 115 MiB data, 903 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 223 op/s 2024-02-02T15:30:56.229 INFO:tasks.workunit.client.0.smithi001.stdout: 1 193948 9.55 MB/sec execute 30 sec latency 16.517 ms 2024-02-02T15:30:57.229 INFO:tasks.workunit.client.0.smithi001.stdout: 1 195500 9.64 MB/sec execute 31 sec latency 15.401 ms 2024-02-02T15:30:57.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:57 smithi071 ceph-mon[44707]: pgmap v121: 129 pgs: 129 active+clean; 123 MiB data, 917 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.4 MiB/s wr, 343 op/s 2024-02-02T15:30:57.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:57 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:30:57.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:57 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:30:57.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:57 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:30:57.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:57 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:30:57.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:57 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:30:57.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:57 smithi142 ceph-mon[45728]: pgmap v121: 129 pgs: 129 active+clean; 123 MiB data, 917 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.4 MiB/s wr, 343 op/s 2024-02-02T15:30:57.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:57 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:30:57.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:57 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:30:57.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:57 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:30:57.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:57 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:30:57.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:57 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:30:57.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:57 smithi001 ceph-mon[35470]: pgmap v121: 129 pgs: 129 active+clean; 123 MiB data, 917 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.4 MiB/s wr, 343 op/s 2024-02-02T15:30:57.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:57 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:30:57.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:57 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:30:57.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:57 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:30:57.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:57 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:30:57.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:57 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:30:58.229 INFO:tasks.workunit.client.0.smithi001.stdout: 1 197497 9.56 MB/sec execute 32 sec latency 10.839 ms 2024-02-02T15:30:59.229 INFO:tasks.workunit.client.0.smithi001.stdout: 1 199064 9.63 MB/sec execute 33 sec latency 11.493 ms 2024-02-02T15:30:59.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:30:59 smithi071 ceph-mon[44707]: pgmap v122: 129 pgs: 129 active+clean; 123 MiB data, 917 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 233 op/s 2024-02-02T15:30:59.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:30:59 smithi142 ceph-mon[45728]: pgmap v122: 129 pgs: 129 active+clean; 123 MiB data, 917 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 233 op/s 2024-02-02T15:30:59.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:30:59 smithi001 ceph-mon[35470]: pgmap v122: 129 pgs: 129 active+clean; 123 MiB data, 917 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 233 op/s 2024-02-02T15:31:00.230 INFO:tasks.workunit.client.0.smithi001.stdout: 1 200905 9.52 MB/sec execute 34 sec latency 48.081 ms 2024-02-02T15:31:01.230 INFO:tasks.workunit.client.0.smithi001.stdout: 1 201532 9.42 MB/sec execute 35 sec latency 96.347 ms 2024-02-02T15:31:01.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:31:01 smithi071 ceph-mon[44707]: pgmap v123: 129 pgs: 129 active+clean; 123 MiB data, 917 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 233 op/s 2024-02-02T15:31:01.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:31:01 smithi142 ceph-mon[45728]: pgmap v123: 129 pgs: 129 active+clean; 123 MiB data, 917 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 233 op/s 2024-02-02T15:31:01.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:31:01 smithi001 ceph-mon[35470]: pgmap v123: 129 pgs: 129 active+clean; 123 MiB data, 917 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 233 op/s 2024-02-02T15:31:02.230 INFO:tasks.workunit.client.0.smithi001.stdout: 1 202805 9.36 MB/sec execute 36 sec latency 10.783 ms 2024-02-02T15:31:03.230 INFO:tasks.workunit.client.0.smithi001.stdout: 1 205107 9.45 MB/sec execute 37 sec latency 13.311 ms 2024-02-02T15:31:03.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:31:03 smithi071 ceph-mon[44707]: pgmap v124: 129 pgs: 129 active+clean; 128 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.5 MiB/s wr, 365 op/s 2024-02-02T15:31:03.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:31:03 smithi142 ceph-mon[45728]: pgmap v124: 129 pgs: 129 active+clean; 128 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.5 MiB/s wr, 365 op/s 2024-02-02T15:31:03.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:31:03 smithi001 ceph-mon[35470]: pgmap v124: 129 pgs: 129 active+clean; 128 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.5 MiB/s wr, 365 op/s 2024-02-02T15:31:04.230 INFO:tasks.workunit.client.0.smithi001.stdout: 1 206391 9.37 MB/sec execute 38 sec latency 13.946 ms 2024-02-02T15:31:05.230 INFO:tasks.workunit.client.0.smithi001.stdout: 1 208496 9.41 MB/sec execute 39 sec latency 13.231 ms 2024-02-02T15:31:05.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:31:05 smithi142 ceph-mon[45728]: pgmap v125: 129 pgs: 129 active+clean; 128 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 251 op/s 2024-02-02T15:31:05.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:31:05 smithi001 ceph-mon[35470]: pgmap v125: 129 pgs: 129 active+clean; 128 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 251 op/s 2024-02-02T15:31:05.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:31:05 smithi071 ceph-mon[44707]: pgmap v125: 129 pgs: 129 active+clean; 128 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 251 op/s 2024-02-02T15:31:06.230 INFO:tasks.workunit.client.0.smithi001.stdout: 1 209746 9.38 MB/sec execute 40 sec latency 63.429 ms 2024-02-02T15:31:07.230 INFO:tasks.workunit.client.0.smithi001.stdout: 1 211769 9.34 MB/sec execute 41 sec latency 24.076 ms 2024-02-02T15:31:07.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:31:07 smithi142 ceph-mon[45728]: pgmap v126: 129 pgs: 129 active+clean; 134 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.7 MiB/s wr, 364 op/s 2024-02-02T15:31:07.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:31:07 smithi001 ceph-mon[35470]: pgmap v126: 129 pgs: 129 active+clean; 134 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.7 MiB/s wr, 364 op/s 2024-02-02T15:31:07.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:31:07 smithi071 ceph-mon[44707]: pgmap v126: 129 pgs: 129 active+clean; 134 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.7 MiB/s wr, 364 op/s 2024-02-02T15:31:08.230 INFO:tasks.workunit.client.0.smithi001.stdout: 1 213264 9.39 MB/sec execute 42 sec latency 12.263 ms 2024-02-02T15:31:09.230 INFO:tasks.workunit.client.0.smithi001.stdout: 1 215365 9.35 MB/sec execute 43 sec latency 15.086 ms 2024-02-02T15:31:09.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:31:09 smithi142 ceph-mon[45728]: pgmap v127: 129 pgs: 129 active+clean; 134 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 244 op/s 2024-02-02T15:31:09.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:31:09 smithi001 ceph-mon[35470]: pgmap v127: 129 pgs: 129 active+clean; 134 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 244 op/s 2024-02-02T15:31:09.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:31:09 smithi071 ceph-mon[44707]: pgmap v127: 129 pgs: 129 active+clean; 134 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 244 op/s 2024-02-02T15:31:10.230 INFO:tasks.workunit.client.0.smithi001.stdout: 1 216900 9.40 MB/sec execute 44 sec latency 11.010 ms 2024-02-02T15:31:11.230 INFO:tasks.workunit.client.0.smithi001.stdout: 1 218791 9.34 MB/sec execute 45 sec latency 62.790 ms 2024-02-02T15:31:11.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:31:11 smithi142 ceph-mon[45728]: pgmap v128: 129 pgs: 129 active+clean; 134 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 244 op/s 2024-02-02T15:31:11.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:31:11 smithi142 ceph-mon[45728]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:31:11.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:31:11 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:31:11.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:31:11 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:31:11.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:31:11 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:31:11.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:31:11 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:31:11.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:31:11 smithi001 ceph-mon[35470]: pgmap v128: 129 pgs: 129 active+clean; 134 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 244 op/s 2024-02-02T15:31:11.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:31:11 smithi001 ceph-mon[35470]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:31:11.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:31:11 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:31:11.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:31:11 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:31:11.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:31:11 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:31:11.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:31:11 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:31:11.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:31:11 smithi071 ceph-mon[44707]: pgmap v128: 129 pgs: 129 active+clean; 134 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 244 op/s 2024-02-02T15:31:11.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:31:11 smithi071 ceph-mon[44707]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:31:11.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:31:11 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:31:11.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:31:11 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:31:11.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:31:11 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:31:11.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:31:11 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:31:12.230 INFO:tasks.workunit.client.0.smithi001.stdout: 1 220327 9.41 MB/sec execute 46 sec latency 36.834 ms 2024-02-02T15:31:13.230 INFO:tasks.workunit.client.0.smithi001.stdout: 1 222012 9.33 MB/sec execute 47 sec latency 11.023 ms 2024-02-02T15:31:13.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:31:13 smithi142 ceph-mon[45728]: pgmap v129: 129 pgs: 129 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.8 MiB/s wr, 358 op/s 2024-02-02T15:31:13.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:31:13 smithi001 ceph-mon[35470]: pgmap v129: 129 pgs: 129 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.8 MiB/s wr, 358 op/s 2024-02-02T15:31:13.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:31:13 smithi071 ceph-mon[44707]: pgmap v129: 129 pgs: 129 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.8 MiB/s wr, 358 op/s 2024-02-02T15:31:14.230 INFO:tasks.workunit.client.0.smithi001.stdout: 1 223855 9.41 MB/sec execute 48 sec latency 11.168 ms 2024-02-02T15:31:15.230 INFO:tasks.workunit.client.0.smithi001.stdout: 1 225642 9.34 MB/sec execute 49 sec latency 11.636 ms 2024-02-02T15:31:15.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:31:15 smithi142 ceph-mon[45728]: pgmap v130: 129 pgs: 129 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.7 MiB/s wr, 226 op/s 2024-02-02T15:31:15.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:31:15 smithi001 ceph-mon[35470]: pgmap v130: 129 pgs: 129 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.7 MiB/s wr, 226 op/s 2024-02-02T15:31:15.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:31:15 smithi071 ceph-mon[44707]: pgmap v130: 129 pgs: 129 active+clean; 142 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.7 MiB/s wr, 226 op/s 2024-02-02T15:31:16.230 INFO:tasks.workunit.client.0.smithi001.stdout: 1 227359 9.41 MB/sec execute 50 sec latency 38.845 ms 2024-02-02T15:31:17.215 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-02T15:31:17.231 INFO:tasks.workunit.client.0.smithi001.stdout: 1 228751 9.34 MB/sec execute 51 sec latency 25.794 ms 2024-02-02T15:31:17.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.434+0000 7fdab8800700 1 -- 172.21.15.1:0/1212474664 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdab0106120 msgr2=0x7fdab010dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:17.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.434+0000 7fdab8800700 1 --2- 172.21.15.1:0/1212474664 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdab0106120 0x7fdab010dca0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fdaa800b0a0 tx=0x7fdaa800b3b0 comp rx=0 tx=0).stop 2024-02-02T15:31:17.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.434+0000 7fdab8800700 1 -- 172.21.15.1:0/1212474664 shutdown_connections 2024-02-02T15:31:17.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.434+0000 7fdab8800700 1 --2- 172.21.15.1:0/1212474664 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdab0106120 0x7fdab010dca0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:17.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.434+0000 7fdab8800700 1 --2- 172.21.15.1:0/1212474664 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdab0101000 0x7fdab01013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:17.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.434+0000 7fdab8800700 1 --2- 172.21.15.1:0/1212474664 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdab01019b0 0x7fdab01059f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:17.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.434+0000 7fdab8800700 1 -- 172.21.15.1:0/1212474664 >> 172.21.15.1:0/1212474664 conn(0x7fdab00fc840 msgr2=0x7fdab00fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:17.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.434+0000 7fdab8800700 1 -- 172.21.15.1:0/1212474664 shutdown_connections 2024-02-02T15:31:17.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.434+0000 7fdab8800700 1 -- 172.21.15.1:0/1212474664 wait complete. 2024-02-02T15:31:17.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.435+0000 7fdab8800700 1 Processor -- start 2024-02-02T15:31:17.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.435+0000 7fdab8800700 1 -- start start 2024-02-02T15:31:17.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.436+0000 7fdab8800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdab0101000 0x7fdab0199d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:17.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.436+0000 7fdab8800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdab0106120 0x7fdab019a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:17.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.436+0000 7fdab8800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdab019ec00 0x7fdab019f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:17.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.436+0000 7fdab8800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdab019f650 con 0x7fdab0101000 2024-02-02T15:31:17.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.436+0000 7fdab8800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdab019f7c0 con 0x7fdab0106120 2024-02-02T15:31:17.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.436+0000 7fdab8800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdab019f980 con 0x7fdab019ec00 2024-02-02T15:31:17.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.436+0000 7fdab5e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdab0101000 0x7fdab0199d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:17.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.436+0000 7fdab5400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdab0106120 0x7fdab019a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:17.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.436+0000 7fdab6800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdab019ec00 0x7fdab019f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:17.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.436+0000 7fdab5400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdab0106120 0x7fdab019a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:53782/0 (socket says 172.21.15.1:53782) 2024-02-02T15:31:17.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.436+0000 7fdab5400700 1 -- 172.21.15.1:0/3959650038 learned_addr learned my addr 172.21.15.1:0/3959650038 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:31:17.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.436+0000 7fdab6800700 1 -- 172.21.15.1:0/3959650038 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdab0106120 msgr2=0x7fdab019a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:17.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.436+0000 7fdab6800700 1 --2- 172.21.15.1:0/3959650038 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdab0106120 0x7fdab019a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:17.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.436+0000 7fdab6800700 1 -- 172.21.15.1:0/3959650038 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdab0101000 msgr2=0x7fdab0199d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:17.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.436+0000 7fdab6800700 1 --2- 172.21.15.1:0/3959650038 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdab0101000 0x7fdab0199d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:17.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.436+0000 7fdab6800700 1 -- 172.21.15.1:0/3959650038 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaa8009d00 con 0x7fdab019ec00 2024-02-02T15:31:17.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.436+0000 7fdab5400700 1 --2- 172.21.15.1:0/3959650038 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdab0106120 0x7fdab019a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:31:17.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.437+0000 7fdab6800700 1 --2- 172.21.15.1:0/3959650038 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdab019ec00 0x7fdab019f080 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fdaa80062a0 tx=0x7fdaa80077f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:17.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.437+0000 7fdaaea00700 1 -- 172.21.15.1:0/3959650038 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa800e040 con 0x7fdab019ec00 2024-02-02T15:31:17.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.437+0000 7fdaaea00700 1 -- 172.21.15.1:0/3959650038 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdaa80094c0 con 0x7fdab019ec00 2024-02-02T15:31:17.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.437+0000 7fdab8800700 1 -- 172.21.15.1:0/3959650038 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdab01a5c60 con 0x7fdab019ec00 2024-02-02T15:31:17.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.437+0000 7fdaaea00700 1 -- 172.21.15.1:0/3959650038 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa8005040 con 0x7fdab019ec00 2024-02-02T15:31:17.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.437+0000 7fdab8800700 1 -- 172.21.15.1:0/3959650038 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdab01a60e0 con 0x7fdab019ec00 2024-02-02T15:31:17.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.440+0000 7fdaaea00700 1 -- 172.21.15.1:0/3959650038 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 27) v1 ==== 135200+0+0 (secure 0 0 0) 0x7fdaa801d470 con 0x7fdab019ec00 2024-02-02T15:31:17.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.443+0000 7fdaaea00700 1 --2- 172.21.15.1:0/3959650038 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fda900a0710 0x7fda900a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:17.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.443+0000 7fdaaea00700 1 -- 172.21.15.1:0/3959650038 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fdaa80cd7a0 con 0x7fdab019ec00 2024-02-02T15:31:17.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.443+0000 7fdacc40cb80 1 Processor -- start 2024-02-02T15:31:17.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.443+0000 7fdab5e00700 1 --2- 172.21.15.1:0/3959650038 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fda900a0710 0x7fda900a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:17.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.443+0000 7fdacc40cb80 1 -- start start 2024-02-02T15:31:17.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.444+0000 7fdacc40cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d88758010 0x558d887583f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:17.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.444+0000 7fdacc40cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d88758930 0x558d887632a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:17.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.444+0000 7fdacc40cb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558d887637e0 0x558d88765bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:17.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.444+0000 7fdacc40cb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d886cc4c0 con 0x558d887637e0 2024-02-02T15:31:17.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.444+0000 7fdacc40cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d886b0aa0 con 0x558d88758930 2024-02-02T15:31:17.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.444+0000 7fdacc40cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d886ae460 con 0x558d88758010 2024-02-02T15:31:17.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.444+0000 7fdab5e00700 1 --2- 172.21.15.1:0/3959650038 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fda900a0710 0x7fda900a2bd0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0009c60 tx=0x7fdaa0009260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:17.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.444+0000 7fdab5e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d88758010 0x558d887583f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:17.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.444+0000 7fdab5e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d88758010 0x558d887583f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:56038/0 (socket says 172.21.15.1:56038) 2024-02-02T15:31:17.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.444+0000 7fdab5e00700 1 -- 172.21.15.1:0/3320783130 learned_addr learned my addr 172.21.15.1:0/3320783130 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:31:17.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.444+0000 7fdab5400700 1 --2- 172.21.15.1:0/3320783130 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d88758930 0x558d887632a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:17.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.444+0000 7fdab6800700 1 --2- 172.21.15.1:0/3320783130 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558d887637e0 0x558d88765bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:17.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.445+0000 7fdab5e00700 1 -- 172.21.15.1:0/3320783130 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d88758930 msgr2=0x558d887632a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:17.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.445+0000 7fdab5e00700 1 --2- 172.21.15.1:0/3320783130 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d88758930 0x558d887632a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:17.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.445+0000 7fdab5e00700 1 -- 172.21.15.1:0/3320783130 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558d887637e0 msgr2=0x558d88765bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:17.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.445+0000 7fdab5e00700 1 --2- 172.21.15.1:0/3320783130 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558d887637e0 0x558d88765bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:17.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.445+0000 7fdab5e00700 1 -- 172.21.15.1:0/3320783130 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d8852b3b0 con 0x558d88758010 2024-02-02T15:31:17.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.445+0000 7fdab5e00700 1 --2- 172.21.15.1:0/3320783130 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d88758010 0x558d887583f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fdaa000e460 tx=0x7fdaa0014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0db87c455f24370 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:17.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.445+0000 7fdaa7000700 1 -- 172.21.15.1:0/3320783130 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa000ee00 con 0x558d88758010 2024-02-02T15:31:17.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.445+0000 7fdaa7000700 1 -- 172.21.15.1:0/3320783130 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdaa0021070 con 0x558d88758010 2024-02-02T15:31:17.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.445+0000 7fdaa7000700 1 -- 172.21.15.1:0/3320783130 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa00256e0 con 0x558d88758010 2024-02-02T15:31:17.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.446+0000 7fdacc40cb80 1 -- 172.21.15.1:0/3320783130 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d88758010 msgr2=0x558d887583f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:17.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.446+0000 7fdacc40cb80 1 --2- 172.21.15.1:0/3320783130 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d88758010 0x558d887583f0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fdaa000e460 tx=0x7fdaa0014000 comp rx=0 tx=0).stop 2024-02-02T15:31:17.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.446+0000 7fdacc40cb80 1 -- 172.21.15.1:0/3320783130 shutdown_connections 2024-02-02T15:31:17.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.446+0000 7fdacc40cb80 1 --2- 172.21.15.1:0/3320783130 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558d887637e0 0x558d88765bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:17.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.446+0000 7fdacc40cb80 1 --2- 172.21.15.1:0/3320783130 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d88758010 0x558d887583f0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:17.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.446+0000 7fdacc40cb80 1 --2- 172.21.15.1:0/3320783130 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d88758930 0x558d887632a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:17.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.446+0000 7fdacc40cb80 1 -- 172.21.15.1:0/3320783130 >> 172.21.15.1:0/3320783130 conn(0x558d886b7db0 msgr2=0x558d887543b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:17.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.446+0000 7fdacc40cb80 1 -- 172.21.15.1:0/3320783130 shutdown_connections 2024-02-02T15:31:17.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.446+0000 7fdacc40cb80 1 -- 172.21.15.1:0/3320783130 wait complete. 2024-02-02T15:31:17.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.447+0000 7fdacc40cb80 1 -- start start 2024-02-02T15:31:17.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.447+0000 7fdacc40cb80 1 Processor -- start 2024-02-02T15:31:17.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.448+0000 7fdacc40cb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558d88758010 0x558d88763250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:17.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.448+0000 7fdacc40cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d88758930 0x558d88760460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:17.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.448+0000 7fdacc40cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d887637e0 0x558d88760b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:17.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.448+0000 7fdacc40cb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d886a9d50 con 0x558d88758010 2024-02-02T15:31:17.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.448+0000 7fdacc40cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d88645d80 con 0x558d88758930 2024-02-02T15:31:17.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.448+0000 7fdacc40cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d88645a20 con 0x558d887637e0 2024-02-02T15:31:17.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.448+0000 7fdacc40cb80 10 client.0 _ll_register_callbacks cb 0x558d88647290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:31:17.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.448+0000 7fdab5e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558d88758010 0x558d88763250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:17.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.448+0000 7fdab5e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558d88758010 0x558d88763250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54716/0 (socket says 172.21.15.1:54716) 2024-02-02T15:31:17.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.448+0000 7fdab5e00700 1 -- 172.21.15.1:0/2850629265 learned_addr learned my addr 172.21.15.1:0/2850629265 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:31:17.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.448+0000 7fdab5400700 1 --2- 172.21.15.1:0/2850629265 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d88758930 0x558d88760460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:17.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.448+0000 7fdab6800700 1 --2- 172.21.15.1:0/2850629265 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d887637e0 0x558d88760b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:17.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.449+0000 7fdab5400700 1 -- 172.21.15.1:0/2850629265 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d887637e0 msgr2=0x558d88760b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:17.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.449+0000 7fdab5400700 1 --2- 172.21.15.1:0/2850629265 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d887637e0 0x558d88760b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:17.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.449+0000 7fdab5400700 1 -- 172.21.15.1:0/2850629265 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558d88758010 msgr2=0x558d88763250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:17.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.449+0000 7fdab5400700 1 --2- 172.21.15.1:0/2850629265 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558d88758010 0x558d88763250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:17.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.449+0000 7fdab5400700 1 -- 172.21.15.1:0/2850629265 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaa0017070 con 0x558d88758930 2024-02-02T15:31:17.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.449+0000 7fdab5e00700 1 --2- 172.21.15.1:0/2850629265 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558d88758010 0x558d88763250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:31:17.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.449+0000 7fdab6800700 1 --2- 172.21.15.1:0/2850629265 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d887637e0 0x558d88760b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:31:17.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.449+0000 7fdab5400700 1 --2- 172.21.15.1:0/2850629265 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d88758930 0x558d88760460 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fdab010dc30 tx=0x7fda98012f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7826b8ca8bc2cd86 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:17.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.449+0000 7fda9f400700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:31:17.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.449+0000 7fda9f400700 1 -- 172.21.15.1:0/2850629265 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda9800fb20 con 0x558d88758930 2024-02-02T15:31:17.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.449+0000 7fda9f400700 1 -- 172.21.15.1:0/2850629265 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fda9800fc80 con 0x558d88758930 2024-02-02T15:31:17.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.449+0000 7fdacc40cb80 1 -- 172.21.15.1:0/2850629265 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558d8874c0f0 con 0x558d88758930 2024-02-02T15:31:17.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.449+0000 7fda9f400700 1 -- 172.21.15.1:0/2850629265 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda9800fe40 con 0x558d88758930 2024-02-02T15:31:17.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.449+0000 7fdacc40cb80 10 client.26729 fetch_fsmap learned FSMap version 32 2024-02-02T15:31:17.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.449+0000 7fdacc40cb80 1 -- 172.21.15.1:0/2850629265 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x558d884d7b60 con 0x558d88758930 2024-02-02T15:31:17.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.450+0000 7fda9f400700 1 -- 172.21.15.1:0/2850629265 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558d8874c0f0 con 0x558d88758930 2024-02-02T15:31:17.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.450+0000 7fda9f400700 1 -- 172.21.15.1:0/2850629265 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 32) v1 ==== 7490+0+0 (secure 0 0 0) 0x7fda980105c0 con 0x558d88758930 2024-02-02T15:31:17.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.450+0000 7fdacc40cb80 10 client.26729 fetch_fsmap finished waiting for FSMap version 32 2024-02-02T15:31:17.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.450+0000 7fdacc40cb80 10 client.26729 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:31:17.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.450+0000 7fdacc40cb80 20 client.26729 populate_metadata read hostname 'smithi001' 2024-02-02T15:31:17.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.450+0000 7fdacc40cb80 1 --2- 172.21.15.1:0/2850629265 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x558d887732d0 0x558d887756f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:17.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.450+0000 7fdacc40cb80 4 client.26729 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:31:17.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.450+0000 7fdacc40cb80 1 -- 172.21.15.1:0/2850629265 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558d886a2dd0 con 0x558d887732d0 2024-02-02T15:31:17.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.450+0000 7fdab5e00700 1 --2- 172.21.15.1:0/2850629265 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x558d887732d0 0x558d887756f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:17.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.451+0000 7fdab5e00700 1 --2- 172.21.15.1:0/2850629265 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x558d887732d0 0x558d887756f0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4718bcbdb5bcd6cd server_cookie=3bb03e00f0ad2fd0 in_seq=0 out_seq=0 2024-02-02T15:31:17.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.451+0000 7fda9f400700 10 client.26729 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:31:17.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.451+0000 7fda9f400700 1 -- 172.21.15.1:0/2850629265 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x558d886a2dd0 con 0x558d887732d0 2024-02-02T15:31:17.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.451+0000 7fda9f400700 10 client.26729 handle_command_reply: tid=0 2024-02-02T15:31:17.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.451+0000 7fdacc40cb80 1 client.26729 shutdown 2024-02-02T15:31:17.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.452+0000 7fdacc40cb80 1 -- 172.21.15.1:0/2850629265 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d88758930 msgr2=0x558d88760460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:17.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.452+0000 7fdacc40cb80 1 --2- 172.21.15.1:0/2850629265 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d88758930 0x558d88760460 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fdab010dc30 tx=0x7fda98012f60 comp rx=0 tx=0).stop 2024-02-02T15:31:17.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.452+0000 7fdacc40cb80 1 -- 172.21.15.1:0/2850629265 shutdown_connections 2024-02-02T15:31:17.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.452+0000 7fdacc40cb80 1 --2- 172.21.15.1:0/2850629265 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x558d887732d0 0x558d887756f0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:17.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.452+0000 7fdacc40cb80 1 --2- 172.21.15.1:0/2850629265 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d887637e0 0x558d88760b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:17.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.452+0000 7fdacc40cb80 1 --2- 172.21.15.1:0/2850629265 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558d88758010 0x558d88763250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:17.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.452+0000 7fdacc40cb80 1 --2- 172.21.15.1:0/2850629265 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d88758930 0x558d88760460 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:17.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.452+0000 7fda9f400700 0 client.26729 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:31:17.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.452+0000 7fdacc40cb80 1 -- 172.21.15.1:0/2850629265 >> 172.21.15.1:0/2850629265 conn(0x558d886b7db0 msgr2=0x558d88753fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:17.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.452+0000 7fdacc40cb80 1 -- 172.21.15.1:0/2850629265 shutdown_connections 2024-02-02T15:31:17.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.452+0000 7fdacc40cb80 1 -- 172.21.15.1:0/2850629265 wait complete. 2024-02-02T15:31:17.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.452+0000 7fdacc40cb80 20 client.26729 trim_cache size 0 max 16384 2024-02-02T15:31:17.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.471+0000 7fdacc40cb80 1 Processor -- start 2024-02-02T15:31:17.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.471+0000 7fdacc40cb80 1 -- start start 2024-02-02T15:31:17.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.472+0000 7fdacc40cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d88758010 0x558d8875af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:17.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.472+0000 7fdacc40cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d88758930 0x558d88760410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:17.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.472+0000 7fdacc40cb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558d887637e0 0x558d88762d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:17.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.472+0000 7fdacc40cb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d8852d6a0 con 0x558d887637e0 2024-02-02T15:31:17.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.472+0000 7fdacc40cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d88761240 con 0x558d88758010 2024-02-02T15:31:17.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.472+0000 7fdacc40cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d88755000 con 0x558d88758930 2024-02-02T15:31:17.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.472+0000 7fdab5e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d88758010 0x558d8875af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:17.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.472+0000 7fdab5400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d88758930 0x558d88760410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:17.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.472+0000 7fdab5e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d88758010 0x558d8875af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:53816/0 (socket says 172.21.15.1:53816) 2024-02-02T15:31:17.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.472+0000 7fdab5e00700 1 -- 172.21.15.1:0/1991269708 learned_addr learned my addr 172.21.15.1:0/1991269708 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:31:17.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.472+0000 7fdab6800700 1 --2- 172.21.15.1:0/1991269708 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558d887637e0 0x558d88762d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:17.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.472+0000 7fdab5e00700 1 -- 172.21.15.1:0/1991269708 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d88758930 msgr2=0x558d88760410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:17.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.472+0000 7fdab5e00700 1 --2- 172.21.15.1:0/1991269708 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d88758930 0x558d88760410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:17.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.472+0000 7fdab5e00700 1 -- 172.21.15.1:0/1991269708 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558d887637e0 msgr2=0x558d88762d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:17.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.472+0000 7fdab5e00700 1 --2- 172.21.15.1:0/1991269708 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558d887637e0 0x558d88762d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:17.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.472+0000 7fdab5e00700 1 -- 172.21.15.1:0/1991269708 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558d887500d0 con 0x558d88758010 2024-02-02T15:31:17.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.472+0000 7fdab6800700 1 --2- 172.21.15.1:0/1991269708 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558d887637e0 0x558d88762d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:31:17.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.473+0000 7fdab5e00700 1 --2- 172.21.15.1:0/1991269708 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d88758010 0x558d8875af40 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fdaa000e460 tx=0x7fdaa00096d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca1cc08da2db7135 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:17.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.473+0000 7fdaa7a00700 1 -- 172.21.15.1:0/1991269708 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa0036070 con 0x558d88758010 2024-02-02T15:31:17.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.473+0000 7fdaa7a00700 1 -- 172.21.15.1:0/1991269708 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdaa000ecc0 con 0x558d88758010 2024-02-02T15:31:17.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.473+0000 7fdaa7a00700 1 -- 172.21.15.1:0/1991269708 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa0003c90 con 0x558d88758010 2024-02-02T15:31:17.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.473+0000 7fdacc40cb80 1 -- 172.21.15.1:0/1991269708 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d88758010 msgr2=0x558d8875af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:17.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.473+0000 7fdacc40cb80 1 --2- 172.21.15.1:0/1991269708 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d88758010 0x558d8875af40 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fdaa000e460 tx=0x7fdaa00096d0 comp rx=0 tx=0).stop 2024-02-02T15:31:17.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.474+0000 7fdacc40cb80 1 -- 172.21.15.1:0/1991269708 shutdown_connections 2024-02-02T15:31:17.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.474+0000 7fdacc40cb80 1 --2- 172.21.15.1:0/1991269708 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558d887637e0 0x558d88762d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:17.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.474+0000 7fdacc40cb80 1 --2- 172.21.15.1:0/1991269708 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d88758010 0x558d8875af40 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:17.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.474+0000 7fdacc40cb80 1 --2- 172.21.15.1:0/1991269708 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d88758930 0x558d88760410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:17.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.474+0000 7fdacc40cb80 1 -- 172.21.15.1:0/1991269708 >> 172.21.15.1:0/1991269708 conn(0x558d886b7db0 msgr2=0x558d8875fd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:17.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.474+0000 7fdacc40cb80 1 -- 172.21.15.1:0/1991269708 shutdown_connections 2024-02-02T15:31:17.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.474+0000 7fdacc40cb80 1 -- 172.21.15.1:0/1991269708 wait complete. 2024-02-02T15:31:17.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.474+0000 7fdacc40cb80 1 -- start start 2024-02-02T15:31:17.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.474+0000 7fdacc40cb80 1 Processor -- start 2024-02-02T15:31:17.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.475+0000 7fdacc40cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d88758010 0x558d8876a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:17.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.475+0000 7fdacc40cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d88758930 0x558d88768350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:17.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.475+0000 7fdacc40cb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558d887637e0 0x558d88768890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:17.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.475+0000 7fdacc40cb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d88768dd0 con 0x558d887637e0 2024-02-02T15:31:17.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.475+0000 7fdacc40cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d88768f10 con 0x558d88758930 2024-02-02T15:31:17.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.475+0000 7fdacc40cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558d887690d0 con 0x558d88758010 2024-02-02T15:31:17.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.475+0000 7fdacc40cb80 10 client.0 _ll_register_callbacks cb 0x558d88647290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:31:17.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.475+0000 7fdab5e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d88758010 0x558d8876a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:17.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.475+0000 7fdab5e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d88758010 0x558d8876a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:56062/0 (socket says 172.21.15.1:56062) 2024-02-02T15:31:17.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.475+0000 7fdab5e00700 1 -- 172.21.15.1:0/4249636472 learned_addr learned my addr 172.21.15.1:0/4249636472 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:31:17.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.475+0000 7fdab6800700 1 --2- 172.21.15.1:0/4249636472 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558d887637e0 0x558d88768890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:17.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.475+0000 7fdab5400700 1 --2- 172.21.15.1:0/4249636472 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d88758930 0x558d88768350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:17.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.475+0000 7fdab5e00700 1 -- 172.21.15.1:0/4249636472 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d88758930 msgr2=0x558d88768350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:17.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.475+0000 7fdab5e00700 1 --2- 172.21.15.1:0/4249636472 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d88758930 0x558d88768350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:17.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.475+0000 7fdab5e00700 1 -- 172.21.15.1:0/4249636472 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558d887637e0 msgr2=0x558d88768890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:17.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.475+0000 7fdab5e00700 1 --2- 172.21.15.1:0/4249636472 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558d887637e0 0x558d88768890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:17.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.475+0000 7fdab5e00700 1 -- 172.21.15.1:0/4249636472 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaa002a040 con 0x558d88758010 2024-02-02T15:31:17.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.475+0000 7fdab6800700 1 --2- 172.21.15.1:0/4249636472 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558d887637e0 0x558d88768890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:31:17.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.476+0000 7fdab5e00700 1 --2- 172.21.15.1:0/4249636472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d88758010 0x558d8876a020 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fdaa000efd0 tx=0x7fdaa0003730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9823505d8c577b20 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:17.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.476+0000 7fda9f400700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:31:17.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.476+0000 7fda9f400700 1 -- 172.21.15.1:0/4249636472 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa0036030 con 0x558d88758010 2024-02-02T15:31:17.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.476+0000 7fdacc40cb80 1 -- 172.21.15.1:0/4249636472 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdaa0021070 con 0x558d88758010 2024-02-02T15:31:17.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.476+0000 7fda9f400700 1 -- 172.21.15.1:0/4249636472 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fdaa0037070 con 0x558d88758010 2024-02-02T15:31:17.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.476+0000 7fda9f400700 1 -- 172.21.15.1:0/4249636472 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdaa0003a50 con 0x558d88758010 2024-02-02T15:31:17.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.476+0000 7fdacc40cb80 10 client.26725 fetch_fsmap learned FSMap version 32 2024-02-02T15:31:17.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.476+0000 7fdacc40cb80 1 -- 172.21.15.1:0/4249636472 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x558d8866fea0 con 0x558d88758010 2024-02-02T15:31:17.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.477+0000 7fda9f400700 1 -- 172.21.15.1:0/4249636472 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdaa0021070 con 0x558d88758010 2024-02-02T15:31:17.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.477+0000 7fda9f400700 1 -- 172.21.15.1:0/4249636472 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 32) v1 ==== 7490+0+0 (secure 0 0 0) 0x7fdaa001d700 con 0x558d88758010 2024-02-02T15:31:17.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.477+0000 7fdacc40cb80 10 client.26725 fetch_fsmap finished waiting for FSMap version 32 2024-02-02T15:31:17.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.477+0000 7fdacc40cb80 10 client.26725 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:31:17.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.477+0000 7fdacc40cb80 20 client.26725 populate_metadata read hostname 'smithi001' 2024-02-02T15:31:17.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.477+0000 7fdacc40cb80 1 --2- 172.21.15.1:0/4249636472 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x558d887c5f10 0x558d887c8330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:17.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.477+0000 7fdacc40cb80 4 client.26725 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2024-02-02T15:31:17.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.477+0000 7fdacc40cb80 1 -- 172.21.15.1:0/4249636472 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558d88668f50 con 0x558d887c5f10 2024-02-02T15:31:17.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.477+0000 7fdab5400700 1 --2- 172.21.15.1:0/4249636472 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x558d887c5f10 0x558d887c8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:17.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.478+0000 7fda9f400700 10 client.26725 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:31:17.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.478+0000 7fdab5400700 1 --2- 172.21.15.1:0/4249636472 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x558d887c5f10 0x558d887c8330 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1697a56221a119e server_cookie=8295bcc25ee22791 in_seq=0 out_seq=0 2024-02-02T15:31:17.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.478+0000 7fda9f400700 1 -- 172.21.15.1:0/4249636472 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x558d88668f50 con 0x558d887c5f10 2024-02-02T15:31:17.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.478+0000 7fda9f400700 10 client.26725 handle_command_reply: tid=0 2024-02-02T15:31:17.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.478+0000 7fdacc40cb80 1 client.26725 shutdown 2024-02-02T15:31:17.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.479+0000 7fdacc40cb80 1 -- 172.21.15.1:0/4249636472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d88758010 msgr2=0x558d8876a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:17.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.479+0000 7fdacc40cb80 1 --2- 172.21.15.1:0/4249636472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d88758010 0x558d8876a020 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fdaa000efd0 tx=0x7fdaa0003730 comp rx=0 tx=0).stop 2024-02-02T15:31:17.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.479+0000 7fdacc40cb80 1 -- 172.21.15.1:0/4249636472 shutdown_connections 2024-02-02T15:31:17.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.479+0000 7fdacc40cb80 1 --2- 172.21.15.1:0/4249636472 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x558d887c5f10 0x558d887c8330 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:17.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.479+0000 7fdacc40cb80 1 --2- 172.21.15.1:0/4249636472 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558d887637e0 0x558d88768890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:17.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.479+0000 7fdacc40cb80 1 --2- 172.21.15.1:0/4249636472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558d88758010 0x558d8876a020 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:17.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.479+0000 7fdacc40cb80 1 --2- 172.21.15.1:0/4249636472 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558d88758930 0x558d88768350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:17.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.479+0000 7fdacc40cb80 1 -- 172.21.15.1:0/4249636472 >> 172.21.15.1:0/4249636472 conn(0x558d886b7db0 msgr2=0x558d88753e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:17.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.479+0000 7fda9f400700 0 client.26725 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:31:17.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.479+0000 7fdacc40cb80 1 -- 172.21.15.1:0/4249636472 shutdown_connections 2024-02-02T15:31:17.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.479+0000 7fdacc40cb80 1 -- 172.21.15.1:0/4249636472 wait complete. 2024-02-02T15:31:17.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.479+0000 7fdacc40cb80 20 client.26725 trim_cache size 0 max 16384 2024-02-02T15:31:17.487 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:31:17.487 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2024-02-02T15:31:17.487 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2024-02-02T15:31:17.487 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:31:17.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.480+0000 7fdab8800700 1 -- 172.21.15.1:0/3959650038 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fda900a0710 msgr2=0x7fda900a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:17.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.480+0000 7fdab8800700 1 --2- 172.21.15.1:0/3959650038 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fda900a0710 0x7fda900a2bd0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0009c60 tx=0x7fdaa0009260 comp rx=0 tx=0).stop 2024-02-02T15:31:17.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.480+0000 7fdab8800700 1 -- 172.21.15.1:0/3959650038 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdab019ec00 msgr2=0x7fdab019f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:17.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.480+0000 7fdab8800700 1 --2- 172.21.15.1:0/3959650038 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdab019ec00 0x7fdab019f080 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fdaa80062a0 tx=0x7fdaa80077f0 comp rx=0 tx=0).stop 2024-02-02T15:31:17.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.480+0000 7fdab8800700 1 -- 172.21.15.1:0/3959650038 shutdown_connections 2024-02-02T15:31:17.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.480+0000 7fdab8800700 1 --2- 172.21.15.1:0/3959650038 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fda900a0710 0x7fda900a2bd0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:17.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.480+0000 7fdab8800700 1 --2- 172.21.15.1:0/3959650038 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdab019ec00 0x7fdab019f080 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:17.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.480+0000 7fdab8800700 1 --2- 172.21.15.1:0/3959650038 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdab0101000 0x7fdab0199d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:17.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.480+0000 7fdab8800700 1 --2- 172.21.15.1:0/3959650038 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdab0106120 0x7fdab019a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:17.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.480+0000 7fdab8800700 1 -- 172.21.15.1:0/3959650038 >> 172.21.15.1:0/3959650038 conn(0x7fdab00fc840 msgr2=0x7fdab00749c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:17.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.480+0000 7fdab8800700 1 -- 172.21.15.1:0/3959650038 shutdown_connections 2024-02-02T15:31:17.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.480+0000 7fdab8800700 1 -- 172.21.15.1:0/3959650038 wait complete. 2024-02-02T15:31:17.500 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-02T15:31:17.500 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T15:31:17.601 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:31:17 smithi001 ceph-mon[35470]: pgmap v131: 129 pgs: 129 active+clean; 144 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.8 MiB/s wr, 340 op/s 2024-02-02T15:31:17.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:31:17 smithi142 ceph-mon[45728]: pgmap v131: 129 pgs: 129 active+clean; 144 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.8 MiB/s wr, 340 op/s 2024-02-02T15:31:17.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.759+0000 7faa58800700 1 -- 172.21.15.1:0/2344686096 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa5010d4e0 msgr2=0x7faa5010f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:17.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.759+0000 7faa58800700 1 --2- 172.21.15.1:0/2344686096 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa5010d4e0 0x7faa5010f950 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7faa4800b0a0 tx=0x7faa4800b3b0 comp rx=0 tx=0).stop 2024-02-02T15:31:17.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.759+0000 7faa58800700 1 -- 172.21.15.1:0/2344686096 shutdown_connections 2024-02-02T15:31:17.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.759+0000 7faa58800700 1 --2- 172.21.15.1:0/2344686096 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa5010d4e0 0x7faa5010f950 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:17.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.759+0000 7faa58800700 1 --2- 172.21.15.1:0/2344686096 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa50073170 0x7faa50073550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:17.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.759+0000 7faa58800700 1 --2- 172.21.15.1:0/2344686096 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa50073a90 0x7faa5010cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:17.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.759+0000 7faa58800700 1 -- 172.21.15.1:0/2344686096 >> 172.21.15.1:0/2344686096 conn(0x7faa500fc790 msgr2=0x7faa500febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:17.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.759+0000 7faa58800700 1 -- 172.21.15.1:0/2344686096 shutdown_connections 2024-02-02T15:31:17.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.759+0000 7faa58800700 1 -- 172.21.15.1:0/2344686096 wait complete. 2024-02-02T15:31:17.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.761+0000 7faa58800700 1 Processor -- start 2024-02-02T15:31:17.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.761+0000 7faa58800700 1 -- start start 2024-02-02T15:31:17.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.761+0000 7faa58800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa50073170 0x7faa501990a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:17.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.761+0000 7faa58800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa50073a90 0x7faa501995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:17.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.761+0000 7faa58800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa5010d4e0 0x7faa5019da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:17.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.761+0000 7faa58800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa5019e080 con 0x7faa5010d4e0 2024-02-02T15:31:17.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.761+0000 7faa58800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa5019e1f0 con 0x7faa50073170 2024-02-02T15:31:17.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.761+0000 7faa58800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa5019e3b0 con 0x7faa50073a90 2024-02-02T15:31:17.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.762+0000 7faa55e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa50073170 0x7faa501990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:17.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.762+0000 7faa55e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa50073170 0x7faa501990a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:53836/0 (socket says 172.21.15.1:53836) 2024-02-02T15:31:17.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.762+0000 7faa55e00700 1 -- 172.21.15.1:0/3608233761 learned_addr learned my addr 172.21.15.1:0/3608233761 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:31:17.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.762+0000 7faa55400700 1 --2- 172.21.15.1:0/3608233761 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa50073a90 0x7faa501995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:17.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.762+0000 7faa56800700 1 --2- 172.21.15.1:0/3608233761 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa5010d4e0 0x7faa5019da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:17.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.762+0000 7faa55e00700 1 -- 172.21.15.1:0/3608233761 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa50073a90 msgr2=0x7faa501995e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:17.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.762+0000 7faa55e00700 1 --2- 172.21.15.1:0/3608233761 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa50073a90 0x7faa501995e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:17.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.762+0000 7faa55e00700 1 -- 172.21.15.1:0/3608233761 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa5010d4e0 msgr2=0x7faa5019da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:17.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.762+0000 7faa55e00700 1 --2- 172.21.15.1:0/3608233761 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa5010d4e0 0x7faa5019da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:17.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.762+0000 7faa55e00700 1 -- 172.21.15.1:0/3608233761 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa38009520 con 0x7faa50073170 2024-02-02T15:31:17.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.762+0000 7faa55e00700 1 --2- 172.21.15.1:0/3608233761 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa50073170 0x7faa501990a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7faa4000e9a0 tx=0x7faa4000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:17.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.763+0000 7faa4ea00700 1 -- 172.21.15.1:0/3608233761 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa40009790 con 0x7faa50073170 2024-02-02T15:31:17.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.763+0000 7faa4ea00700 1 -- 172.21.15.1:0/3608233761 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa40010430 con 0x7faa50073170 2024-02-02T15:31:17.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.763+0000 7faa4ea00700 1 -- 172.21.15.1:0/3608233761 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa40018680 con 0x7faa50073170 2024-02-02T15:31:17.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.763+0000 7faa58800700 1 -- 172.21.15.1:0/3608233761 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa48009d00 con 0x7faa50073170 2024-02-02T15:31:17.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.763+0000 7faa58800700 1 -- 172.21.15.1:0/3608233761 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa501a5cf0 con 0x7faa50073170 2024-02-02T15:31:17.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.764+0000 7faa58800700 1 -- 172.21.15.1:0/3608233761 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa50107e00 con 0x7faa50073170 2024-02-02T15:31:17.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.766+0000 7faa4ea00700 1 -- 172.21.15.1:0/3608233761 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 27) v1 ==== 135200+0+0 (secure 0 0 0) 0x7faa400105a0 con 0x7faa50073170 2024-02-02T15:31:17.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.767+0000 7faa4ea00700 1 --2- 172.21.15.1:0/3608233761 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7faa300a0690 0x7faa300a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:17.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.768+0000 7faa4ea00700 1 -- 172.21.15.1:0/3608233761 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7faa40014070 con 0x7faa50073170 2024-02-02T15:31:17.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.768+0000 7faa55400700 1 --2- 172.21.15.1:0/3608233761 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7faa300a0690 0x7faa300a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:17.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.768+0000 7faa55400700 1 --2- 172.21.15.1:0/3608233761 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7faa300a0690 0x7faa300a2b50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7faa5019a6b0 tx=0x7faa38009260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:17.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:17.773+0000 7faa4ea00700 1 -- 172.21.15.1:0/3608233761 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7faa40052110 con 0x7faa50073170 2024-02-02T15:31:17.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:31:17 smithi071 ceph-mon[44707]: pgmap v131: 129 pgs: 129 active+clean; 144 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.8 MiB/s wr, 340 op/s 2024-02-02T15:31:18.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.054+0000 7faa58800700 1 -- 172.21.15.1:0/3608233761 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faa50067bc0 con 0x7faa50073170 2024-02-02T15:31:18.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.055+0000 7faa4ea00700 1 -- 172.21.15.1:0/3608233761 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+10874 (secure 0 0 0) 0x7faa40095790 con 0x7faa50073170 2024-02-02T15:31:18.055 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:31:18.055 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2922240448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2922240448},{"type":"v1","addr":"172.21.15.1:6835","nonce":2922240448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2729497431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2729497431},{"type":"v1","addr":"172.21.15.1:6837","nonce":2729497431}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-02T15:24:47.761435+0000","modified":"2024-02-02T15:31:10.369514+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24529,"mds_2":24518,"mds_3":24514,"mds_4":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14517":{"gid":14517,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/1218303683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":1218303683},{"type":"v1","addr":"172.21.15.1:6839","nonce":1218303683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14529":{"gid":14529,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/2059197496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2059197496},{"type":"v1","addr":"172.21.15.1:6841","nonce":2059197496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/7648986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":7648986},{"type":"v1","addr":"172.21.15.142:6833","nonce":7648986}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/3482746094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3482746094},{"type":"v1","addr":"172.21.15.71:6833","nonce":3482746094}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/388207814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":388207814},{"type":"v1","addr":"172.21.15.71:6835","nonce":388207814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6835/1052606547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1052606547},{"type":"v1","addr":"172.21.15.142:6835","nonce":1052606547}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.142:6837/2870289019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2870289019},{"type":"v1","addr":"172.21.15.142:6837","nonce":2870289019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.71:6837/628016199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":628016199},{"type":"v1","addr":"172.21.15.71:6837","nonce":628016199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.142:6839/321675421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":321675421},{"type":"v1","addr":"172.21.15.142:6839","nonce":321675421}]},"join_fscid":-1,"export_targets":[4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/1268086493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1268086493},{"type":"v1","addr":"172.21.15.71:6839","nonce":1268086493}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-02T15:31:18.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.057+0000 7faa58800700 1 -- 172.21.15.1:0/3608233761 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7faa300a0690 msgr2=0x7faa300a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:18.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.057+0000 7faa58800700 1 --2- 172.21.15.1:0/3608233761 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7faa300a0690 0x7faa300a2b50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7faa5019a6b0 tx=0x7faa38009260 comp rx=0 tx=0).stop 2024-02-02T15:31:18.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.057+0000 7faa58800700 1 -- 172.21.15.1:0/3608233761 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa50073170 msgr2=0x7faa501990a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:18.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.057+0000 7faa58800700 1 --2- 172.21.15.1:0/3608233761 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa50073170 0x7faa501990a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7faa4000e9a0 tx=0x7faa4000ecb0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.057+0000 7faa58800700 1 -- 172.21.15.1:0/3608233761 shutdown_connections 2024-02-02T15:31:18.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.057+0000 7faa58800700 1 --2- 172.21.15.1:0/3608233761 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7faa300a0690 0x7faa300a2b50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.057+0000 7faa58800700 1 --2- 172.21.15.1:0/3608233761 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa5010d4e0 0x7faa5019da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.057+0000 7faa58800700 1 --2- 172.21.15.1:0/3608233761 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa50073170 0x7faa501990a0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.057+0000 7faa58800700 1 --2- 172.21.15.1:0/3608233761 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa50073a90 0x7faa501995e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.057+0000 7faa58800700 1 -- 172.21.15.1:0/3608233761 >> 172.21.15.1:0/3608233761 conn(0x7faa500fc790 msgr2=0x7faa50104f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:18.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.057+0000 7faa58800700 1 -- 172.21.15.1:0/3608233761 shutdown_connections 2024-02-02T15:31:18.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.057+0000 7faa58800700 1 -- 172.21.15.1:0/3608233761 wait complete. 2024-02-02T15:31:18.059 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 32 2024-02-02T15:31:18.076 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-02-02T15:31:18.231 INFO:tasks.workunit.client.0.smithi001.stdout: 1 230876 9.42 MB/sec execute 52 sec latency 11.009 ms 2024-02-02T15:31:18.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.292+0000 7f8bd9a00700 1 -- 172.21.15.1:0/381705872 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8bd410d5c0 msgr2=0x7f8bd410fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:18.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.292+0000 7f8bd9a00700 1 --2- 172.21.15.1:0/381705872 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8bd410d5c0 0x7f8bd410fa30 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc00b0a0 tx=0x7f8bcc00b3b0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.292+0000 7f8bd9a00700 1 -- 172.21.15.1:0/381705872 shutdown_connections 2024-02-02T15:31:18.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.292+0000 7f8bd9a00700 1 --2- 172.21.15.1:0/381705872 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8bd410d5c0 0x7f8bd410fa30 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.292+0000 7f8bd9a00700 1 --2- 172.21.15.1:0/381705872 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bd4073170 0x7f8bd4073550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.292+0000 7f8bd9a00700 1 --2- 172.21.15.1:0/381705872 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bd4073a90 0x7f8bd410d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.292+0000 7f8bd9a00700 1 -- 172.21.15.1:0/381705872 >> 172.21.15.1:0/381705872 conn(0x7f8bd40fc810 msgr2=0x7f8bd40fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:18.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.292+0000 7f8bd9a00700 1 -- 172.21.15.1:0/381705872 shutdown_connections 2024-02-02T15:31:18.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.293+0000 7f8bd9a00700 1 -- 172.21.15.1:0/381705872 wait complete. 2024-02-02T15:31:18.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.294+0000 7f8bd9a00700 1 Processor -- start 2024-02-02T15:31:18.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.294+0000 7f8bd9a00700 1 -- start start 2024-02-02T15:31:18.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.294+0000 7f8bd9a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bd4073170 0x7f8bd41991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:18.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.294+0000 7f8bd9a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8bd4073a90 0x7f8bd41996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:18.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.294+0000 7f8bd9a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bd410d5c0 0x7f8bd419db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:18.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.294+0000 7f8bd9a00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd419e180 con 0x7f8bd4073170 2024-02-02T15:31:18.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.294+0000 7f8bd9a00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd419e2f0 con 0x7f8bd4073a90 2024-02-02T15:31:18.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.294+0000 7f8bd9a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd419e4b0 con 0x7f8bd410d5c0 2024-02-02T15:31:18.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.294+0000 7f8bd2a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bd4073170 0x7f8bd41991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:18.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.294+0000 7f8bd2a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bd4073170 0x7f8bd41991a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54764/0 (socket says 172.21.15.1:54764) 2024-02-02T15:31:18.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.294+0000 7f8bd2a00700 1 -- 172.21.15.1:0/1785963844 learned_addr learned my addr 172.21.15.1:0/1785963844 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:31:18.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.294+0000 7f8bd3400700 1 --2- 172.21.15.1:0/1785963844 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bd410d5c0 0x7f8bd419db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:18.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.294+0000 7f8bcbe00700 1 --2- 172.21.15.1:0/1785963844 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8bd4073a90 0x7f8bd41996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:18.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.295+0000 7f8bd2a00700 1 -- 172.21.15.1:0/1785963844 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bd410d5c0 msgr2=0x7f8bd419db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:18.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.295+0000 7f8bd2a00700 1 --2- 172.21.15.1:0/1785963844 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bd410d5c0 0x7f8bd419db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.295+0000 7f8bd2a00700 1 -- 172.21.15.1:0/1785963844 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8bd4073a90 msgr2=0x7f8bd41996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:18.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.295+0000 7f8bd2a00700 1 --2- 172.21.15.1:0/1785963844 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8bd4073a90 0x7f8bd41996e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.295+0000 7f8bd2a00700 1 -- 172.21.15.1:0/1785963844 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bcc009d00 con 0x7f8bd4073170 2024-02-02T15:31:18.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.295+0000 7f8bd3400700 1 --2- 172.21.15.1:0/1785963844 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bd410d5c0 0x7f8bd419db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:31:18.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.295+0000 7f8bd2a00700 1 --2- 172.21.15.1:0/1785963844 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bd4073170 0x7f8bd41991a0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f8bc400e9a0 tx=0x7f8bc400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:18.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.295+0000 7f8bcb400700 1 -- 172.21.15.1:0/1785963844 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc4009790 con 0x7f8bd4073170 2024-02-02T15:31:18.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.295+0000 7f8bcb400700 1 -- 172.21.15.1:0/1785963844 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8bc4010430 con 0x7f8bd4073170 2024-02-02T15:31:18.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.295+0000 7f8bd9a00700 1 -- 172.21.15.1:0/1785963844 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bd419e790 con 0x7f8bd4073170 2024-02-02T15:31:18.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.295+0000 7f8bcb400700 1 -- 172.21.15.1:0/1785963844 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc4018680 con 0x7f8bd4073170 2024-02-02T15:31:18.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.295+0000 7f8bd9a00700 1 -- 172.21.15.1:0/1785963844 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bd41a5fd0 con 0x7f8bd4073170 2024-02-02T15:31:18.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.299+0000 7f8bcb400700 1 -- 172.21.15.1:0/1785963844 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 27) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f8bc40105a0 con 0x7f8bd4073170 2024-02-02T15:31:18.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.300+0000 7f8bcb400700 1 --2- 172.21.15.1:0/1785963844 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f8bb40a0710 0x7f8bb40a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:18.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.300+0000 7f8bcb400700 1 -- 172.21.15.1:0/1785963844 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f8bc4014070 con 0x7f8bd4073170 2024-02-02T15:31:18.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.300+0000 7f8bcbe00700 1 --2- 172.21.15.1:0/1785963844 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f8bb40a0710 0x7f8bb40a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:18.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.300+0000 7f8bed493b80 1 Processor -- start 2024-02-02T15:31:18.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.301+0000 7f8bed493b80 1 -- start start 2024-02-02T15:31:18.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.301+0000 7f8bed493b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b1c4e79010 0x55b1c4e793f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:18.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.301+0000 7f8bed493b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b1c4e79930 0x55b1c4e842a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:18.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.301+0000 7f8bed493b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b1c4e847e0 0x55b1c4e86bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:18.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.301+0000 7f8bed493b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1c4ded4c0 con 0x55b1c4e79010 2024-02-02T15:31:18.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.301+0000 7f8bed493b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1c4dd1aa0 con 0x55b1c4e847e0 2024-02-02T15:31:18.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.301+0000 7f8bed493b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1c4dcf460 con 0x55b1c4e79930 2024-02-02T15:31:18.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.301+0000 7f8bcbe00700 1 --2- 172.21.15.1:0/1785963844 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f8bb40a0710 0x7f8bb40a2bd0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8bd419a7b0 tx=0x7f8bbc009260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:18.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.301+0000 7f8bcbe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b1c4e79930 0x55b1c4e842a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:18.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.301+0000 7f8bcbe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b1c4e79930 0x55b1c4e842a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:56112/0 (socket says 172.21.15.1:56112) 2024-02-02T15:31:18.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.301+0000 7f8bcbe00700 1 -- 172.21.15.1:0/2699989098 learned_addr learned my addr 172.21.15.1:0/2699989098 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:31:18.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.301+0000 7f8bd3400700 1 --2- 172.21.15.1:0/2699989098 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b1c4e847e0 0x55b1c4e86bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:18.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.302+0000 7f8bcbe00700 1 -- 172.21.15.1:0/2699989098 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b1c4e847e0 msgr2=0x55b1c4e86bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:18.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.302+0000 7f8bcbe00700 1 --2- 172.21.15.1:0/2699989098 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b1c4e847e0 0x55b1c4e86bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.302+0000 7f8bd2a00700 1 --2- 172.21.15.1:0/2699989098 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b1c4e79010 0x55b1c4e793f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:18.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.302+0000 7f8bcbe00700 1 -- 172.21.15.1:0/2699989098 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b1c4e79010 msgr2=0x55b1c4e793f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:18.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.302+0000 7f8bcbe00700 1 --2- 172.21.15.1:0/2699989098 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b1c4e79010 0x55b1c4e793f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.302+0000 7f8bcbe00700 1 -- 172.21.15.1:0/2699989098 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b1c4c4c3b0 con 0x55b1c4e79930 2024-02-02T15:31:18.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.302+0000 7f8bd2a00700 1 --2- 172.21.15.1:0/2699989098 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b1c4e79010 0x55b1c4e793f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:31:18.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.302+0000 7f8bcbe00700 1 --2- 172.21.15.1:0/2699989098 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b1c4e79930 0x55b1c4e842a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc011d30 tx=0x7f8bbc014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cbedd623a757726e server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:18.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.302+0000 7f8bc3e00700 1 -- 172.21.15.1:0/2699989098 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc011d60 con 0x55b1c4e79930 2024-02-02T15:31:18.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.302+0000 7f8bc3e00700 1 -- 172.21.15.1:0/2699989098 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8bbc004020 con 0x55b1c4e79930 2024-02-02T15:31:18.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.302+0000 7f8bc3e00700 1 -- 172.21.15.1:0/2699989098 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc003b60 con 0x55b1c4e79930 2024-02-02T15:31:18.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.303+0000 7f8bed493b80 1 -- 172.21.15.1:0/2699989098 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b1c4e79930 msgr2=0x55b1c4e842a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:18.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.303+0000 7f8bed493b80 1 --2- 172.21.15.1:0/2699989098 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b1c4e79930 0x55b1c4e842a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc011d30 tx=0x7f8bbc014000 comp rx=0 tx=0).stop 2024-02-02T15:31:18.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.303+0000 7f8bed493b80 1 -- 172.21.15.1:0/2699989098 shutdown_connections 2024-02-02T15:31:18.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.303+0000 7f8bed493b80 1 --2- 172.21.15.1:0/2699989098 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b1c4e847e0 0x55b1c4e86bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.303+0000 7f8bed493b80 1 --2- 172.21.15.1:0/2699989098 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b1c4e79010 0x55b1c4e793f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.303+0000 7f8bed493b80 1 --2- 172.21.15.1:0/2699989098 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b1c4e79930 0x55b1c4e842a0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.303+0000 7f8bed493b80 1 -- 172.21.15.1:0/2699989098 >> 172.21.15.1:0/2699989098 conn(0x55b1c4dd8db0 msgr2=0x55b1c4e753b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:18.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.303+0000 7f8bed493b80 1 -- 172.21.15.1:0/2699989098 shutdown_connections 2024-02-02T15:31:18.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.303+0000 7f8bed493b80 1 -- 172.21.15.1:0/2699989098 wait complete. 2024-02-02T15:31:18.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.304+0000 7f8bed493b80 1 -- start start 2024-02-02T15:31:18.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.304+0000 7f8bed493b80 1 Processor -- start 2024-02-02T15:31:18.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.305+0000 7f8bed493b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b1c4e79010 0x55b1c4e841f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:18.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.305+0000 7f8bed493b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b1c4e79930 0x55b1c4e81400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:18.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.305+0000 7f8bed493b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b1c4e847e0 0x55b1c4e81a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:18.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.305+0000 7f8bed493b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1c4dcad50 con 0x55b1c4e79930 2024-02-02T15:31:18.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.305+0000 7f8bed493b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1c4d66d80 con 0x55b1c4e79010 2024-02-02T15:31:18.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.305+0000 7f8bed493b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1c4d66a20 con 0x55b1c4e847e0 2024-02-02T15:31:18.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.305+0000 7f8bed493b80 10 client.0 _ll_register_callbacks cb 0x55b1c4d68290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:31:18.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.305+0000 7f8bd2a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b1c4e79010 0x55b1c4e841f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:18.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.305+0000 7f8bd2a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b1c4e79010 0x55b1c4e841f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:53880/0 (socket says 172.21.15.1:53880) 2024-02-02T15:31:18.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.305+0000 7f8bd2a00700 1 -- 172.21.15.1:0/3878691560 learned_addr learned my addr 172.21.15.1:0/3878691560 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:31:18.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.305+0000 7f8bcbe00700 1 --2- 172.21.15.1:0/3878691560 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b1c4e79930 0x55b1c4e81400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:18.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.305+0000 7f8bd3400700 1 --2- 172.21.15.1:0/3878691560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b1c4e847e0 0x55b1c4e81a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:18.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.305+0000 7f8bd2a00700 1 -- 172.21.15.1:0/3878691560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b1c4e847e0 msgr2=0x55b1c4e81a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:18.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.305+0000 7f8bd2a00700 1 --2- 172.21.15.1:0/3878691560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b1c4e847e0 0x55b1c4e81a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.305+0000 7f8bd2a00700 1 -- 172.21.15.1:0/3878691560 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b1c4e79930 msgr2=0x55b1c4e81400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:18.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.305+0000 7f8bd2a00700 1 --2- 172.21.15.1:0/3878691560 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b1c4e79930 0x55b1c4e81400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.305+0000 7f8bd2a00700 1 -- 172.21.15.1:0/3878691560 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bbc017070 con 0x55b1c4e79010 2024-02-02T15:31:18.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.305+0000 7f8bd3400700 1 --2- 172.21.15.1:0/3878691560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b1c4e847e0 0x55b1c4e81a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:31:18.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.306+0000 7f8bd2a00700 1 --2- 172.21.15.1:0/3878691560 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b1c4e79010 0x55b1c4e841f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4095e30 tx=0x7f8bc4099bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=affc2efeeaf4d46b server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:18.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.306+0000 7f8bc0c00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:31:18.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.306+0000 7f8bc0c00700 1 -- 172.21.15.1:0/3878691560 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc4098ab0 con 0x55b1c4e79010 2024-02-02T15:31:18.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.306+0000 7f8bc0c00700 1 -- 172.21.15.1:0/3878691560 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f8bc409d1f0 con 0x55b1c4e79010 2024-02-02T15:31:18.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.306+0000 7f8bed493b80 1 -- 172.21.15.1:0/3878691560 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b1c4e6d0f0 con 0x55b1c4e79010 2024-02-02T15:31:18.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.306+0000 7f8bc0c00700 1 -- 172.21.15.1:0/3878691560 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8bc4084070 con 0x55b1c4e79010 2024-02-02T15:31:18.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.306+0000 7f8bed493b80 10 client.26753 fetch_fsmap learned FSMap version 32 2024-02-02T15:31:18.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.306+0000 7f8bed493b80 1 -- 172.21.15.1:0/3878691560 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55b1c4bf8b60 con 0x55b1c4e79010 2024-02-02T15:31:18.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.307+0000 7f8bc0c00700 1 -- 172.21.15.1:0/3878691560 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b1c4e6d0f0 con 0x55b1c4e79010 2024-02-02T15:31:18.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.307+0000 7f8bc0c00700 1 -- 172.21.15.1:0/3878691560 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 32) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f8bc4082bc0 con 0x55b1c4e79010 2024-02-02T15:31:18.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.307+0000 7f8bed493b80 10 client.26753 fetch_fsmap finished waiting for FSMap version 32 2024-02-02T15:31:18.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.307+0000 7f8bed493b80 10 client.26753 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-02-02T15:31:18.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.307+0000 7f8bed493b80 20 client.26753 populate_metadata read hostname 'smithi001' 2024-02-02T15:31:18.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.307+0000 7f8bed493b80 1 --2- 172.21.15.1:0/3878691560 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55b1c4e940d0 0x55b1c4e964f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:18.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.307+0000 7f8bed493b80 4 client.26753 mds_command: new command op to 24502 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:31:18.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.307+0000 7f8bed493b80 1 -- 172.21.15.1:0/3878691560 --> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b1c4dc3dd0 con 0x55b1c4e940d0 2024-02-02T15:31:18.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.307+0000 7f8bcbe00700 1 --2- 172.21.15.1:0/3878691560 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55b1c4e940d0 0x55b1c4e964f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:18.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.308+0000 7f8bcbe00700 1 --2- 172.21.15.1:0/3878691560 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55b1c4e940d0 0x55b1c4e964f0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ea2d1bf8c65e01ac server_cookie=966e0735c9811d75 in_seq=0 out_seq=0 2024-02-02T15:31:18.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.308+0000 7f8bc0c00700 10 client.26753 ms_handle_connect on v2:172.21.15.142:6834/1052606547 2024-02-02T15:31:18.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.308+0000 7f8bc0c00700 1 -- 172.21.15.1:0/3878691560 <== mds.4 v2:172.21.15.142:6834/1052606547 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55b1c4dc3dd0 con 0x55b1c4e940d0 2024-02-02T15:31:18.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.308+0000 7f8bc0c00700 10 client.26753 handle_command_reply: tid=0 2024-02-02T15:31:18.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.308+0000 7f8bed493b80 1 client.26753 shutdown 2024-02-02T15:31:18.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.308+0000 7f8bed493b80 1 -- 172.21.15.1:0/3878691560 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b1c4e79010 msgr2=0x55b1c4e841f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:18.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.308+0000 7f8bed493b80 1 --2- 172.21.15.1:0/3878691560 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b1c4e79010 0x55b1c4e841f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8bc4095e30 tx=0x7f8bc4099bb0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.309+0000 7f8bed493b80 1 -- 172.21.15.1:0/3878691560 shutdown_connections 2024-02-02T15:31:18.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.309+0000 7f8bed493b80 1 --2- 172.21.15.1:0/3878691560 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55b1c4e940d0 0x55b1c4e964f0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.309+0000 7f8bed493b80 1 --2- 172.21.15.1:0/3878691560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b1c4e847e0 0x55b1c4e81a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.309+0000 7f8bed493b80 1 --2- 172.21.15.1:0/3878691560 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b1c4e79010 0x55b1c4e841f0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.309+0000 7f8bc0c00700 0 client.26753 ms_handle_reset on v2:172.21.15.142:6834/1052606547 2024-02-02T15:31:18.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.309+0000 7f8bed493b80 1 --2- 172.21.15.1:0/3878691560 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b1c4e79930 0x55b1c4e81400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.309+0000 7f8bed493b80 1 -- 172.21.15.1:0/3878691560 >> 172.21.15.1:0/3878691560 conn(0x55b1c4dd8db0 msgr2=0x55b1c4e74fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:18.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.309+0000 7f8bed493b80 1 -- 172.21.15.1:0/3878691560 shutdown_connections 2024-02-02T15:31:18.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.309+0000 7f8bed493b80 1 -- 172.21.15.1:0/3878691560 wait complete. 2024-02-02T15:31:18.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.309+0000 7f8bed493b80 20 client.26753 trim_cache size 0 max 16384 2024-02-02T15:31:18.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.322+0000 7f8bed493b80 1 Processor -- start 2024-02-02T15:31:18.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.322+0000 7f8bed493b80 1 -- start start 2024-02-02T15:31:18.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.323+0000 7f8bed493b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b1c4e79010 0x55b1c4e8b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:18.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.323+0000 7f8bed493b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b1c4e79930 0x55b1c4e83d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:18.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.323+0000 7f8bed493b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b1c4e847e0 0x55b1c4e8f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:18.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.323+0000 7f8bed493b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1c4c4e6a0 con 0x55b1c4e847e0 2024-02-02T15:31:18.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.323+0000 7f8bed493b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1c4e9b170 con 0x55b1c4e79930 2024-02-02T15:31:18.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.323+0000 7f8bed493b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1c4e84260 con 0x55b1c4e79010 2024-02-02T15:31:18.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.323+0000 7f8bcbe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b1c4e79930 0x55b1c4e83d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:18.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.323+0000 7f8bcbe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b1c4e79930 0x55b1c4e83d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:53896/0 (socket says 172.21.15.1:53896) 2024-02-02T15:31:18.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.323+0000 7f8bcbe00700 1 -- 172.21.15.1:0/3339071270 learned_addr learned my addr 172.21.15.1:0/3339071270 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:31:18.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.323+0000 7f8bd2a00700 1 --2- 172.21.15.1:0/3339071270 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b1c4e79010 0x55b1c4e8b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:18.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.323+0000 7f8bd3400700 1 --2- 172.21.15.1:0/3339071270 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b1c4e847e0 0x55b1c4e8f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:18.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.323+0000 7f8bcbe00700 1 -- 172.21.15.1:0/3339071270 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b1c4e79010 msgr2=0x55b1c4e8b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:18.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.323+0000 7f8bcbe00700 1 --2- 172.21.15.1:0/3339071270 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b1c4e79010 0x55b1c4e8b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.323+0000 7f8bcbe00700 1 -- 172.21.15.1:0/3339071270 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b1c4e847e0 msgr2=0x55b1c4e8f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:18.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.323+0000 7f8bcbe00700 1 --2- 172.21.15.1:0/3339071270 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b1c4e847e0 0x55b1c4e8f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.324+0000 7f8bcbe00700 1 -- 172.21.15.1:0/3339071270 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b1c4d90ea0 con 0x55b1c4e79930 2024-02-02T15:31:18.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.324+0000 7f8bd3400700 1 --2- 172.21.15.1:0/3339071270 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b1c4e847e0 0x55b1c4e8f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:31:18.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.324+0000 7f8bcbe00700 1 --2- 172.21.15.1:0/3339071270 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b1c4e79930 0x55b1c4e83d20 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc009aa0 tx=0x7f8bbc009a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebed09c87e8a3198 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:18.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.324+0000 7f8bc8c00700 1 -- 172.21.15.1:0/3339071270 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc02c070 con 0x55b1c4e79930 2024-02-02T15:31:18.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.324+0000 7f8bc8c00700 1 -- 172.21.15.1:0/3339071270 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8bbc02d040 con 0x55b1c4e79930 2024-02-02T15:31:18.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.324+0000 7f8bc8c00700 1 -- 172.21.15.1:0/3339071270 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc033b80 con 0x55b1c4e79930 2024-02-02T15:31:18.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.325+0000 7f8bed493b80 1 -- 172.21.15.1:0/3339071270 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b1c4e79930 msgr2=0x55b1c4e83d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:18.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.325+0000 7f8bed493b80 1 --2- 172.21.15.1:0/3339071270 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b1c4e79930 0x55b1c4e83d20 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc009aa0 tx=0x7f8bbc009a30 comp rx=0 tx=0).stop 2024-02-02T15:31:18.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.325+0000 7f8bed493b80 1 -- 172.21.15.1:0/3339071270 shutdown_connections 2024-02-02T15:31:18.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.325+0000 7f8bed493b80 1 --2- 172.21.15.1:0/3339071270 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b1c4e847e0 0x55b1c4e8f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.325+0000 7f8bed493b80 1 --2- 172.21.15.1:0/3339071270 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b1c4e79010 0x55b1c4e8b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.325+0000 7f8bed493b80 1 --2- 172.21.15.1:0/3339071270 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b1c4e79930 0x55b1c4e83d20 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.325+0000 7f8bed493b80 1 -- 172.21.15.1:0/3339071270 >> 172.21.15.1:0/3339071270 conn(0x55b1c4dd8db0 msgr2=0x55b1c4e7a830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:18.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.325+0000 7f8bed493b80 1 -- 172.21.15.1:0/3339071270 shutdown_connections 2024-02-02T15:31:18.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.325+0000 7f8bed493b80 1 -- 172.21.15.1:0/3339071270 wait complete. 2024-02-02T15:31:18.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.325+0000 7f8bed493b80 1 -- start start 2024-02-02T15:31:18.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.325+0000 7f8bed493b80 1 Processor -- start 2024-02-02T15:31:18.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.326+0000 7f8bed493b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b1c4e79010 0x55b1c4e7ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:18.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.326+0000 7f8bed493b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b1c4e79930 0x55b1c4e7ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:18.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.326+0000 7f8bed493b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b1c4e847e0 0x55b1c4e7d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:18.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.326+0000 7f8bed493b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1c4e7dd50 con 0x55b1c4e79010 2024-02-02T15:31:18.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.326+0000 7f8bed493b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1c4e7dec0 con 0x55b1c4e79930 2024-02-02T15:31:18.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.326+0000 7f8bed493b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1c4e7e080 con 0x55b1c4e847e0 2024-02-02T15:31:18.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.326+0000 7f8bed493b80 10 client.0 _ll_register_callbacks cb 0x55b1c4d68290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:31:18.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.326+0000 7f8bcbe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b1c4e79930 0x55b1c4e7ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:18.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.326+0000 7f8bd2a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b1c4e79010 0x55b1c4e7ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:18.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.326+0000 7f8bd3400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b1c4e847e0 0x55b1c4e7d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:18.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.326+0000 7f8bcbe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b1c4e79930 0x55b1c4e7ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:53910/0 (socket says 172.21.15.1:53910) 2024-02-02T15:31:18.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.326+0000 7f8bcbe00700 1 -- 172.21.15.1:0/3674008735 learned_addr learned my addr 172.21.15.1:0/3674008735 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:31:18.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.327+0000 7f8bcbe00700 1 -- 172.21.15.1:0/3674008735 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b1c4e847e0 msgr2=0x55b1c4e7d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:18.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.327+0000 7f8bcbe00700 1 --2- 172.21.15.1:0/3674008735 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b1c4e847e0 0x55b1c4e7d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.327+0000 7f8bcbe00700 1 -- 172.21.15.1:0/3674008735 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b1c4e79010 msgr2=0x55b1c4e7ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:18.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.327+0000 7f8bcbe00700 1 --2- 172.21.15.1:0/3674008735 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b1c4e79010 0x55b1c4e7ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.327+0000 7f8bcbe00700 1 -- 172.21.15.1:0/3674008735 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b1c4c4c3b0 con 0x55b1c4e79930 2024-02-02T15:31:18.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.327+0000 7f8bcbe00700 1 --2- 172.21.15.1:0/3674008735 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b1c4e79930 0x55b1c4e7ef80 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc00a010 tx=0x7f8bbc0043b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83f76375bd6a335f server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:18.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.327+0000 7f8bc0c00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:31:18.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.327+0000 7f8bc0c00700 1 -- 172.21.15.1:0/3674008735 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bbc02c050 con 0x55b1c4e79930 2024-02-02T15:31:18.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.327+0000 7f8bc0c00700 1 -- 172.21.15.1:0/3674008735 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f8bbc00e970 con 0x55b1c4e79930 2024-02-02T15:31:18.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.327+0000 7f8bed493b80 1 -- 172.21.15.1:0/3674008735 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8bbc004020 con 0x55b1c4e79930 2024-02-02T15:31:18.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.327+0000 7f8bc0c00700 1 -- 172.21.15.1:0/3674008735 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8bbc00eb30 con 0x55b1c4e79930 2024-02-02T15:31:18.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.327+0000 7f8bed493b80 10 client.26759 fetch_fsmap learned FSMap version 32 2024-02-02T15:31:18.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.327+0000 7f8bed493b80 1 -- 172.21.15.1:0/3674008735 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55b1c4c50cd0 con 0x55b1c4e79930 2024-02-02T15:31:18.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.328+0000 7f8bc0c00700 1 -- 172.21.15.1:0/3674008735 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8bbc046080 con 0x55b1c4e79930 2024-02-02T15:31:18.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.328+0000 7f8bc0c00700 1 -- 172.21.15.1:0/3674008735 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 32) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f8bbc024030 con 0x55b1c4e79930 2024-02-02T15:31:18.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.328+0000 7f8bed493b80 10 client.26759 fetch_fsmap finished waiting for FSMap version 32 2024-02-02T15:31:18.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.328+0000 7f8bed493b80 10 client.26759 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-02-02T15:31:18.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.328+0000 7f8bed493b80 20 client.26759 populate_metadata read hostname 'smithi001' 2024-02-02T15:31:18.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.328+0000 7f8bed493b80 1 --2- 172.21.15.1:0/3674008735 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55b1c4ee67b0 0x55b1c4ee8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:18.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.328+0000 7f8bed493b80 4 client.26759 mds_command: new command op to 24502 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:31:18.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.328+0000 7f8bed493b80 1 -- 172.21.15.1:0/3674008735 --> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b1c4d89f50 con 0x55b1c4ee67b0 2024-02-02T15:31:18.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.328+0000 7f8bd2a00700 1 --2- 172.21.15.1:0/3674008735 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55b1c4ee67b0 0x55b1c4ee8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:18.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.329+0000 7f8bd2a00700 1 --2- 172.21.15.1:0/3674008735 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55b1c4ee67b0 0x55b1c4ee8bd0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3506a41a39e5b8da server_cookie=1be3a39a0c8576c6 in_seq=0 out_seq=0 2024-02-02T15:31:18.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.329+0000 7f8bc0c00700 10 client.26759 ms_handle_connect on v2:172.21.15.142:6834/1052606547 2024-02-02T15:31:18.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.329+0000 7f8bc0c00700 1 -- 172.21.15.1:0/3674008735 <== mds.4 v2:172.21.15.142:6834/1052606547 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b1c4d89f50 con 0x55b1c4ee67b0 2024-02-02T15:31:18.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.329+0000 7f8bc0c00700 10 client.26759 handle_command_reply: tid=0 2024-02-02T15:31:18.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.329+0000 7f8bed493b80 1 client.26759 shutdown 2024-02-02T15:31:18.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.329+0000 7f8bed493b80 1 -- 172.21.15.1:0/3674008735 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b1c4e79930 msgr2=0x55b1c4e7ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:18.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.329+0000 7f8bed493b80 1 --2- 172.21.15.1:0/3674008735 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b1c4e79930 0x55b1c4e7ef80 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8bbc00a010 tx=0x7f8bbc0043b0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.329+0000 7f8bed493b80 1 -- 172.21.15.1:0/3674008735 shutdown_connections 2024-02-02T15:31:18.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.329+0000 7f8bed493b80 1 --2- 172.21.15.1:0/3674008735 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55b1c4ee67b0 0x55b1c4ee8bd0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.329+0000 7f8bed493b80 1 --2- 172.21.15.1:0/3674008735 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b1c4e847e0 0x55b1c4e7d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.329+0000 7f8bc0c00700 0 client.26759 ms_handle_reset on v2:172.21.15.142:6834/1052606547 2024-02-02T15:31:18.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.329+0000 7f8bed493b80 1 --2- 172.21.15.1:0/3674008735 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b1c4e79010 0x55b1c4e7ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.329+0000 7f8bed493b80 1 --2- 172.21.15.1:0/3674008735 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b1c4e79930 0x55b1c4e7ef80 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.329+0000 7f8bed493b80 1 -- 172.21.15.1:0/3674008735 >> 172.21.15.1:0/3674008735 conn(0x55b1c4dd8db0 msgr2=0x55b1c4e8f900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:18.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.329+0000 7f8bed493b80 1 -- 172.21.15.1:0/3674008735 shutdown_connections 2024-02-02T15:31:18.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.329+0000 7f8bed493b80 1 -- 172.21.15.1:0/3674008735 wait complete. 2024-02-02T15:31:18.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.330+0000 7f8bed493b80 20 client.26759 trim_cache size 0 max 16384 2024-02-02T15:31:18.333 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:31:18.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.330+0000 7f8bd9a00700 1 -- 172.21.15.1:0/1785963844 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f8bb40a0710 msgr2=0x7f8bb40a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:18.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.330+0000 7f8bd9a00700 1 --2- 172.21.15.1:0/1785963844 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f8bb40a0710 0x7f8bb40a2bd0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8bd419a7b0 tx=0x7f8bbc009260 comp rx=0 tx=0).stop 2024-02-02T15:31:18.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.330+0000 7f8bd9a00700 1 -- 172.21.15.1:0/1785963844 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bd4073170 msgr2=0x7f8bd41991a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:18.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.330+0000 7f8bd9a00700 1 --2- 172.21.15.1:0/1785963844 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bd4073170 0x7f8bd41991a0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f8bc400e9a0 tx=0x7f8bc400ecb0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.330+0000 7f8bd9a00700 1 -- 172.21.15.1:0/1785963844 shutdown_connections 2024-02-02T15:31:18.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.330+0000 7f8bd9a00700 1 --2- 172.21.15.1:0/1785963844 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f8bb40a0710 0x7f8bb40a2bd0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.330+0000 7f8bd9a00700 1 --2- 172.21.15.1:0/1785963844 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bd410d5c0 0x7f8bd419db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.330+0000 7f8bd9a00700 1 --2- 172.21.15.1:0/1785963844 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bd4073170 0x7f8bd41991a0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.330+0000 7f8bd9a00700 1 --2- 172.21.15.1:0/1785963844 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8bd4073a90 0x7f8bd41996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.330+0000 7f8bd9a00700 1 -- 172.21.15.1:0/1785963844 >> 172.21.15.1:0/1785963844 conn(0x7f8bd40fc810 msgr2=0x7f8bd4105020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:18.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.330+0000 7f8bd9a00700 1 -- 172.21.15.1:0/1785963844 shutdown_connections 2024-02-02T15:31:18.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.331+0000 7f8bd9a00700 1 -- 172.21.15.1:0/1785963844 wait complete. 2024-02-02T15:31:18.350 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-02-02T15:31:18.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.578+0000 7f96a7e00700 1 -- 172.21.15.1:0/1284795459 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96a01019a0 msgr2=0x7f96a01059e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:18.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.578+0000 7f96a7e00700 1 --2- 172.21.15.1:0/1284795459 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96a01019a0 0x7f96a01059e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f9688009990 tx=0x7f9688009ca0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.578+0000 7f96a7e00700 1 -- 172.21.15.1:0/1284795459 shutdown_connections 2024-02-02T15:31:18.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.578+0000 7f96a7e00700 1 --2- 172.21.15.1:0/1284795459 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f96a0106110 0x7f96a010dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.578+0000 7f96a7e00700 1 --2- 172.21.15.1:0/1284795459 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f96a0100ff0 0x7f96a01013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.578+0000 7f96a7e00700 1 --2- 172.21.15.1:0/1284795459 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96a01019a0 0x7f96a01059e0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.578+0000 7f96a7e00700 1 -- 172.21.15.1:0/1284795459 >> 172.21.15.1:0/1284795459 conn(0x7f96a00fc810 msgr2=0x7f96a00fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:18.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.578+0000 7f96a7e00700 1 -- 172.21.15.1:0/1284795459 shutdown_connections 2024-02-02T15:31:18.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.578+0000 7f96a7e00700 1 -- 172.21.15.1:0/1284795459 wait complete. 2024-02-02T15:31:18.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.579+0000 7f96a7e00700 1 Processor -- start 2024-02-02T15:31:18.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.580+0000 7f96a7e00700 1 -- start start 2024-02-02T15:31:18.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.580+0000 7f96a7e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96a0100ff0 0x7f96a0196e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:18.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.580+0000 7f96a7e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f96a0106110 0x7f96a0197370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:18.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.580+0000 7f96a7e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f96a019b7d0 0x7f96a019bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:18.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.580+0000 7f96a7e00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96a019c250 con 0x7f96a019b7d0 2024-02-02T15:31:18.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.580+0000 7f96a7e00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96a019c3c0 con 0x7f96a0100ff0 2024-02-02T15:31:18.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.580+0000 7f96a7e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96a019c580 con 0x7f96a0106110 2024-02-02T15:31:18.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.580+0000 7f96a5400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96a0100ff0 0x7f96a0196e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:18.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.580+0000 7f96a4a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f96a0106110 0x7f96a0197370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:18.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.580+0000 7f96a5400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96a0100ff0 0x7f96a0196e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:53914/0 (socket says 172.21.15.1:53914) 2024-02-02T15:31:18.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.580+0000 7f96a5400700 1 -- 172.21.15.1:0/898358488 learned_addr learned my addr 172.21.15.1:0/898358488 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:31:18.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.580+0000 7f96a5e00700 1 --2- 172.21.15.1:0/898358488 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f96a019b7d0 0x7f96a019bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:18.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.581+0000 7f96a5400700 1 -- 172.21.15.1:0/898358488 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f96a0106110 msgr2=0x7f96a0197370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:18.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.581+0000 7f96a5400700 1 --2- 172.21.15.1:0/898358488 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f96a0106110 0x7f96a0197370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.581+0000 7f96a5400700 1 -- 172.21.15.1:0/898358488 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f96a019b7d0 msgr2=0x7f96a019bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:18.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.581+0000 7f96a5400700 1 --2- 172.21.15.1:0/898358488 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f96a019b7d0 0x7f96a019bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.581+0000 7f96a5400700 1 -- 172.21.15.1:0/898358488 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9690009520 con 0x7f96a0100ff0 2024-02-02T15:31:18.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.581+0000 7f96a5400700 1 --2- 172.21.15.1:0/898358488 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96a0100ff0 0x7f96a0196e30 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f969000eb30 tx=0x7f969000ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:18.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.581+0000 7f969e000700 1 -- 172.21.15.1:0/898358488 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9690009880 con 0x7f96a0100ff0 2024-02-02T15:31:18.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.581+0000 7f969e000700 1 -- 172.21.15.1:0/898358488 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f96900044e0 con 0x7f96a0100ff0 2024-02-02T15:31:18.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.581+0000 7f96a7e00700 1 -- 172.21.15.1:0/898358488 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96880095f0 con 0x7f96a0100ff0 2024-02-02T15:31:18.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.581+0000 7f969e000700 1 -- 172.21.15.1:0/898358488 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9690003d80 con 0x7f96a0100ff0 2024-02-02T15:31:18.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.581+0000 7f96a7e00700 1 -- 172.21.15.1:0/898358488 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96a01a3e30 con 0x7f96a0100ff0 2024-02-02T15:31:18.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.584+0000 7f969e000700 1 -- 172.21.15.1:0/898358488 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 27) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f9690003f20 con 0x7f96a0100ff0 2024-02-02T15:31:18.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.586+0000 7f96bb9a8b80 1 Processor -- start 2024-02-02T15:31:18.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.587+0000 7f96bb9a8b80 1 -- start start 2024-02-02T15:31:18.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.587+0000 7f969e000700 1 --2- 172.21.15.1:0/898358488 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f96800a0690 0x7f96800a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:18.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.587+0000 7f96bb9a8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565051970420 0x565051970800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:18.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.587+0000 7f969e000700 1 -- 172.21.15.1:0/898358488 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f9690014070 con 0x7f96a0100ff0 2024-02-02T15:31:18.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.587+0000 7f96bb9a8b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565051970d40 0x56505197b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:18.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.587+0000 7f96bb9a8b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56505197bbf0 0x56505197dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:18.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.587+0000 7f96a5e00700 1 --2- 172.21.15.1:0/898358488 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f96800a0690 0x7f96800a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:18.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.587+0000 7f96bb9a8b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650518e04c0 con 0x56505197bbf0 2024-02-02T15:31:18.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.587+0000 7f96bb9a8b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650518c4aa0 con 0x565051970d40 2024-02-02T15:31:18.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.587+0000 7f96bb9a8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650518c2460 con 0x565051970420 2024-02-02T15:31:18.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.587+0000 7f96a5400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565051970420 0x565051970800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:18.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.588+0000 7f96a4a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565051970d40 0x56505197b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:18.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.588+0000 7f96a5400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565051970420 0x565051970800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:56154/0 (socket says 172.21.15.1:56154) 2024-02-02T15:31:18.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.588+0000 7f96a5400700 1 -- 172.21.15.1:0/3968143653 learned_addr learned my addr 172.21.15.1:0/3968143653 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:31:18.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.588+0000 7f96a4a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565051970d40 0x56505197b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:53924/0 (socket says 172.21.15.1:53924) 2024-02-02T15:31:18.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.588+0000 7f96a5e00700 1 --2- 172.21.15.1:0/3968143653 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56505197bbf0 0x56505197dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:18.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.588+0000 7f96a5400700 1 -- 172.21.15.1:0/3968143653 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565051970d40 msgr2=0x56505197b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:18.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.588+0000 7f96a5400700 1 --2- 172.21.15.1:0/3968143653 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565051970d40 0x56505197b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.588+0000 7f96a5400700 1 -- 172.21.15.1:0/3968143653 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56505197bbf0 msgr2=0x56505197dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:18.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.588+0000 7f96a5400700 1 --2- 172.21.15.1:0/3968143653 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56505197bbf0 0x56505197dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.588+0000 7f96a5400700 1 -- 172.21.15.1:0/3968143653 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56505173f3b0 con 0x565051970420 2024-02-02T15:31:18.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.588+0000 7f96a5e00700 1 --2- 172.21.15.1:0/898358488 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f96800a0690 0x7f96800a2b50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f969800f600 tx=0x7f969800f9c0 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:18.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.588+0000 7f96a5400700 1 --2- 172.21.15.1:0/3968143653 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565051970420 0x565051970800 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f969000c5b0 tx=0x7f96900996f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df652e6ada03a191 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:18.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.588+0000 7f9696600700 1 -- 172.21.15.1:0/3968143653 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96900997f0 con 0x565051970420 2024-02-02T15:31:18.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.588+0000 7f9696600700 1 -- 172.21.15.1:0/3968143653 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f969009d620 con 0x565051970420 2024-02-02T15:31:18.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.588+0000 7f9696600700 1 -- 172.21.15.1:0/3968143653 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9690099a50 con 0x565051970420 2024-02-02T15:31:18.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.589+0000 7f96bb9a8b80 1 -- 172.21.15.1:0/3968143653 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565051970420 msgr2=0x565051970800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:18.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.589+0000 7f96bb9a8b80 1 --2- 172.21.15.1:0/3968143653 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565051970420 0x565051970800 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f969000c5b0 tx=0x7f96900996f0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.589+0000 7f96bb9a8b80 1 -- 172.21.15.1:0/3968143653 shutdown_connections 2024-02-02T15:31:18.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.589+0000 7f96bb9a8b80 1 --2- 172.21.15.1:0/3968143653 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56505197bbf0 0x56505197dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.589+0000 7f96bb9a8b80 1 --2- 172.21.15.1:0/3968143653 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565051970420 0x565051970800 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.589+0000 7f96bb9a8b80 1 --2- 172.21.15.1:0/3968143653 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565051970d40 0x56505197b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.589+0000 7f96bb9a8b80 1 -- 172.21.15.1:0/3968143653 >> 172.21.15.1:0/3968143653 conn(0x5650518cbdb0 msgr2=0x5650519683b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:18.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.589+0000 7f96bb9a8b80 1 -- 172.21.15.1:0/3968143653 shutdown_connections 2024-02-02T15:31:18.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.589+0000 7f96bb9a8b80 1 -- 172.21.15.1:0/3968143653 wait complete. 2024-02-02T15:31:18.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.590+0000 7f96bb9a8b80 1 -- start start 2024-02-02T15:31:18.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.590+0000 7f96bb9a8b80 1 Processor -- start 2024-02-02T15:31:18.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.591+0000 7f96bb9a8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565051970420 0x56505197b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:18.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.591+0000 7f96bb9a8b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565051970d40 0x5650519787c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:18.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.591+0000 7f96bb9a8b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56505197bbf0 0x565051978dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:18.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.591+0000 7f96bb9a8b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650518bdd50 con 0x565051970d40 2024-02-02T15:31:18.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.591+0000 7f96bb9a8b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x565051859d80 con 0x56505197bbf0 2024-02-02T15:31:18.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.591+0000 7f96bb9a8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x565051859a20 con 0x565051970420 2024-02-02T15:31:18.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.591+0000 7f96bb9a8b80 10 client.0 _ll_register_callbacks cb 0x56505185b290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:31:18.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.591+0000 7f96a5400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565051970420 0x56505197b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:18.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.591+0000 7f96a5400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565051970420 0x56505197b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:56160/0 (socket says 172.21.15.1:56160) 2024-02-02T15:31:18.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.591+0000 7f96a5400700 1 -- 172.21.15.1:0/4153389665 learned_addr learned my addr 172.21.15.1:0/4153389665 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:31:18.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.591+0000 7f96a4a00700 1 --2- 172.21.15.1:0/4153389665 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565051970d40 0x5650519787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:18.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.591+0000 7f96a5e00700 1 --2- 172.21.15.1:0/4153389665 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56505197bbf0 0x565051978dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:18.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.591+0000 7f96a5400700 1 -- 172.21.15.1:0/4153389665 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56505197bbf0 msgr2=0x565051978dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:18.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.591+0000 7f96a5400700 1 --2- 172.21.15.1:0/4153389665 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56505197bbf0 0x565051978dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.591+0000 7f96a5400700 1 -- 172.21.15.1:0/4153389665 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565051970d40 msgr2=0x5650519787c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:18.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.591+0000 7f96a5400700 1 --2- 172.21.15.1:0/4153389665 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565051970d40 0x5650519787c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.591+0000 7f96a5400700 1 -- 172.21.15.1:0/4153389665 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96a01a3e30 con 0x565051970420 2024-02-02T15:31:18.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.591+0000 7f96a5e00700 1 --2- 172.21.15.1:0/4153389665 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56505197bbf0 0x565051978dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:31:18.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.591+0000 7f96a4a00700 1 --2- 172.21.15.1:0/4153389665 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565051970d40 0x5650519787c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:31:18.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.591+0000 7f96a5400700 1 --2- 172.21.15.1:0/4153389665 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565051970420 0x56505197b5b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9690099780 tx=0x7f96900961b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b821c036ff207ef6 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:18.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.591+0000 7f968ea00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:31:18.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.592+0000 7f968ea00700 1 -- 172.21.15.1:0/4153389665 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96900cc4a0 con 0x565051970420 2024-02-02T15:31:18.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.592+0000 7f968ea00700 1 -- 172.21.15.1:0/4153389665 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f96900997f0 con 0x565051970420 2024-02-02T15:31:18.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.592+0000 7f96bb9a8b80 1 -- 172.21.15.1:0/4153389665 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5650519600f0 con 0x565051970420 2024-02-02T15:31:18.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.592+0000 7f968ea00700 1 -- 172.21.15.1:0/4153389665 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f969000cdd0 con 0x565051970420 2024-02-02T15:31:18.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.592+0000 7f96bb9a8b80 10 client.26761 fetch_fsmap learned FSMap version 32 2024-02-02T15:31:18.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.592+0000 7f96bb9a8b80 1 -- 172.21.15.1:0/4153389665 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5650516ebb60 con 0x565051970420 2024-02-02T15:31:18.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.592+0000 7f968ea00700 1 -- 172.21.15.1:0/4153389665 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f969007e070 con 0x565051970420 2024-02-02T15:31:18.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.592+0000 7f968ea00700 1 -- 172.21.15.1:0/4153389665 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 32) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f96900ae070 con 0x565051970420 2024-02-02T15:31:18.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.592+0000 7f96bb9a8b80 10 client.26761 fetch_fsmap finished waiting for FSMap version 32 2024-02-02T15:31:18.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.592+0000 7f96bb9a8b80 10 client.26761 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-02-02T15:31:18.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.592+0000 7f96bb9a8b80 20 client.26761 populate_metadata read hostname 'smithi001' 2024-02-02T15:31:18.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.592+0000 7f96bb9a8b80 1 --2- 172.21.15.1:0/4153389665 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x56505198b1a0 0x56505198d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:18.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.593+0000 7f96bb9a8b80 4 client.26761 mds_command: new command op to 24514 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:31:18.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.593+0000 7f96bb9a8b80 1 -- 172.21.15.1:0/4153389665 --> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5650518b6dd0 con 0x56505198b1a0 2024-02-02T15:31:18.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.593+0000 7f96a4a00700 1 --2- 172.21.15.1:0/4153389665 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x56505198b1a0 0x56505198d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:18.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.593+0000 7f96a4a00700 1 --2- 172.21.15.1:0/4153389665 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x56505198b1a0 0x56505198d590 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=76a1f284a9ebeb61 server_cookie=646c9e7b1c806fcf in_seq=0 out_seq=0 2024-02-02T15:31:18.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.593+0000 7f968ea00700 10 client.26761 ms_handle_connect on v2:172.21.15.142:6836/2870289019 2024-02-02T15:31:18.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.594+0000 7f968ea00700 1 -- 172.21.15.1:0/4153389665 <== mds.3 v2:172.21.15.142:6836/2870289019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5650518b6dd0 con 0x56505198b1a0 2024-02-02T15:31:18.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.594+0000 7f968ea00700 10 client.26761 handle_command_reply: tid=0 2024-02-02T15:31:18.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.594+0000 7f96bb9a8b80 1 client.26761 shutdown 2024-02-02T15:31:18.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.594+0000 7f96bb9a8b80 1 -- 172.21.15.1:0/4153389665 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565051970420 msgr2=0x56505197b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:18.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.594+0000 7f96bb9a8b80 1 --2- 172.21.15.1:0/4153389665 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565051970420 0x56505197b5b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9690099780 tx=0x7f96900961b0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.594+0000 7f96bb9a8b80 1 -- 172.21.15.1:0/4153389665 shutdown_connections 2024-02-02T15:31:18.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.594+0000 7f96bb9a8b80 1 --2- 172.21.15.1:0/4153389665 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56505197bbf0 0x565051978dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.594+0000 7f96bb9a8b80 1 --2- 172.21.15.1:0/4153389665 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565051970420 0x56505197b5b0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.594+0000 7f96bb9a8b80 1 --2- 172.21.15.1:0/4153389665 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x56505198b1a0 0x56505198d590 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.594+0000 7f96bb9a8b80 1 --2- 172.21.15.1:0/4153389665 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565051970d40 0x5650519787c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.594+0000 7f96bb9a8b80 1 -- 172.21.15.1:0/4153389665 >> 172.21.15.1:0/4153389665 conn(0x5650518cbdb0 msgr2=0x565051967f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:18.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.594+0000 7f968ea00700 0 client.26761 ms_handle_reset on v2:172.21.15.142:6836/2870289019 2024-02-02T15:31:18.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.595+0000 7f96bb9a8b80 1 -- 172.21.15.1:0/4153389665 shutdown_connections 2024-02-02T15:31:18.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.595+0000 7f96bb9a8b80 1 -- 172.21.15.1:0/4153389665 wait complete. 2024-02-02T15:31:18.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.595+0000 7f96bb9a8b80 20 client.26761 trim_cache size 0 max 16384 2024-02-02T15:31:18.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.608+0000 7f96bb9a8b80 1 Processor -- start 2024-02-02T15:31:18.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.609+0000 7f96bb9a8b80 1 -- start start 2024-02-02T15:31:18.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.609+0000 7f96bb9a8b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565051970420 0x565051978780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:18.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.609+0000 7f96bb9a8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565051970d40 0x565051978060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:18.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.609+0000 7f96bb9a8b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56505197bbf0 0x565051966b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:18.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.609+0000 7f96bb9a8b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650517416a0 con 0x56505197bbf0 2024-02-02T15:31:18.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.609+0000 7f96bb9a8b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650519785a0 con 0x565051970420 2024-02-02T15:31:18.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.609+0000 7f96bb9a8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x565051967060 con 0x565051970d40 2024-02-02T15:31:18.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.609+0000 7f96a5400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565051970420 0x565051978780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:18.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.609+0000 7f96a5400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565051970420 0x565051978780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:53932/0 (socket says 172.21.15.1:53932) 2024-02-02T15:31:18.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.609+0000 7f96a5400700 1 -- 172.21.15.1:0/1577560535 learned_addr learned my addr 172.21.15.1:0/1577560535 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:31:18.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.609+0000 7f96a4a00700 1 --2- 172.21.15.1:0/1577560535 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565051970d40 0x565051978060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:18.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.609+0000 7f96a5e00700 1 --2- 172.21.15.1:0/1577560535 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56505197bbf0 0x565051966b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:18.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.610+0000 7f96a4a00700 1 -- 172.21.15.1:0/1577560535 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565051970420 msgr2=0x565051978780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:18.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.610+0000 7f96a4a00700 1 --2- 172.21.15.1:0/1577560535 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565051970420 0x565051978780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.610+0000 7f96a4a00700 1 -- 172.21.15.1:0/1577560535 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56505197bbf0 msgr2=0x565051966b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:18.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.610+0000 7f96a4a00700 1 --2- 172.21.15.1:0/1577560535 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56505197bbf0 0x565051966b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.610+0000 7f96a4a00700 1 -- 172.21.15.1:0/1577560535 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565051883ea0 con 0x565051970d40 2024-02-02T15:31:18.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.610+0000 7f96a5400700 1 --2- 172.21.15.1:0/1577560535 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565051970420 0x565051978780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:31:18.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.610+0000 7f96a4a00700 1 --2- 172.21.15.1:0/1577560535 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565051970d40 0x565051978060 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9688000c00 tx=0x7f9688005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f233a43eeffd81c9 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:18.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.610+0000 7f9697000700 1 -- 172.21.15.1:0/1577560535 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f968801d040 con 0x565051970d40 2024-02-02T15:31:18.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.610+0000 7f9697000700 1 -- 172.21.15.1:0/1577560535 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f968801e040 con 0x565051970d40 2024-02-02T15:31:18.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.610+0000 7f9697000700 1 -- 172.21.15.1:0/1577560535 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9688005510 con 0x565051970d40 2024-02-02T15:31:18.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.611+0000 7f96bb9a8b80 1 -- 172.21.15.1:0/1577560535 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565051970d40 msgr2=0x565051978060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:18.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.611+0000 7f96bb9a8b80 1 --2- 172.21.15.1:0/1577560535 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565051970d40 0x565051978060 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9688000c00 tx=0x7f9688005fd0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.611+0000 7f96bb9a8b80 1 -- 172.21.15.1:0/1577560535 shutdown_connections 2024-02-02T15:31:18.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.611+0000 7f96bb9a8b80 1 --2- 172.21.15.1:0/1577560535 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56505197bbf0 0x565051966b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.611+0000 7f96bb9a8b80 1 --2- 172.21.15.1:0/1577560535 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x565051970420 0x565051978780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.611+0000 7f96bb9a8b80 1 --2- 172.21.15.1:0/1577560535 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565051970d40 0x565051978060 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.611+0000 7f96bb9a8b80 1 -- 172.21.15.1:0/1577560535 >> 172.21.15.1:0/1577560535 conn(0x5650518cbdb0 msgr2=0x565051986cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:18.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.611+0000 7f96bb9a8b80 1 -- 172.21.15.1:0/1577560535 shutdown_connections 2024-02-02T15:31:18.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.611+0000 7f96bb9a8b80 1 -- 172.21.15.1:0/1577560535 wait complete. 2024-02-02T15:31:18.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.611+0000 7f96bb9a8b80 1 -- start start 2024-02-02T15:31:18.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.612+0000 7f96bb9a8b80 1 Processor -- start 2024-02-02T15:31:18.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.612+0000 7f96bb9a8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565051970420 0x5650519808b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:18.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.612+0000 7f96bb9a8b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565051970d40 0x565051980df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:18.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.612+0000 7f96bb9a8b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56505197bbf0 0x5650519d5d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:18.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.612+0000 7f96bb9a8b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650519d6250 con 0x565051970d40 2024-02-02T15:31:18.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.612+0000 7f96bb9a8b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650519d63c0 con 0x56505197bbf0 2024-02-02T15:31:18.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.612+0000 7f96bb9a8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650519d6580 con 0x565051970420 2024-02-02T15:31:18.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.612+0000 7f96bb9a8b80 10 client.0 _ll_register_callbacks cb 0x56505185b290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:31:18.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.612+0000 7f96a5400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565051970420 0x5650519808b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:18.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.612+0000 7f96a4a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565051970d40 0x565051980df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:18.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.612+0000 7f96a5400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565051970420 0x5650519808b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:56176/0 (socket says 172.21.15.1:56176) 2024-02-02T15:31:18.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.612+0000 7f96a5400700 1 -- 172.21.15.1:0/41200449 learned_addr learned my addr 172.21.15.1:0/41200449 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:31:18.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.612+0000 7f96a5e00700 1 --2- 172.21.15.1:0/41200449 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56505197bbf0 0x5650519d5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:18.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.613+0000 7f96a5400700 1 -- 172.21.15.1:0/41200449 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56505197bbf0 msgr2=0x5650519d5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:18.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.613+0000 7f96a5400700 1 --2- 172.21.15.1:0/41200449 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56505197bbf0 0x5650519d5d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.613+0000 7f96a5400700 1 -- 172.21.15.1:0/41200449 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565051970d40 msgr2=0x565051980df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:18.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.613+0000 7f96a5400700 1 --2- 172.21.15.1:0/41200449 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565051970d40 0x565051980df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.613+0000 7f96a5400700 1 -- 172.21.15.1:0/41200449 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5650516ebb60 con 0x565051970420 2024-02-02T15:31:18.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.613+0000 7f96a4a00700 1 --2- 172.21.15.1:0/41200449 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565051970d40 0x565051980df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:31:18.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.613+0000 7f96a5400700 1 --2- 172.21.15.1:0/41200449 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565051970420 0x5650519808b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f96900994f0 tx=0x7f969000cd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4dc97e7a9832cb8 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:18.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.613+0000 7f968ea00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:31:18.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.613+0000 7f968ea00700 1 -- 172.21.15.1:0/41200449 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96900cc4a0 con 0x565051970420 2024-02-02T15:31:18.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.613+0000 7f968ea00700 1 -- 172.21.15.1:0/41200449 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f96900997f0 con 0x565051970420 2024-02-02T15:31:18.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.613+0000 7f96bb9a8b80 1 -- 172.21.15.1:0/41200449 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f969009d620 con 0x565051970420 2024-02-02T15:31:18.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.614+0000 7f968ea00700 1 -- 172.21.15.1:0/41200449 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9690099150 con 0x565051970420 2024-02-02T15:31:18.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.614+0000 7f96bb9a8b80 10 client.26767 fetch_fsmap learned FSMap version 32 2024-02-02T15:31:18.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.614+0000 7f96bb9a8b80 1 -- 172.21.15.1:0/41200449 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7f96a010e500 con 0x565051970420 2024-02-02T15:31:18.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.614+0000 7f968ea00700 1 -- 172.21.15.1:0/41200449 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f969009d620 con 0x565051970420 2024-02-02T15:31:18.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.614+0000 7f968ea00700 1 -- 172.21.15.1:0/41200449 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 32) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f96900c9ca0 con 0x565051970420 2024-02-02T15:31:18.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.614+0000 7f96bb9a8b80 10 client.26767 fetch_fsmap finished waiting for FSMap version 32 2024-02-02T15:31:18.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.614+0000 7f96bb9a8b80 10 client.26767 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-02-02T15:31:18.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.614+0000 7f96bb9a8b80 20 client.26767 populate_metadata read hostname 'smithi001' 2024-02-02T15:31:18.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.614+0000 7f96bb9a8b80 1 --2- 172.21.15.1:0/41200449 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x5650519dd800 0x5650519dfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:18.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.614+0000 7f96bb9a8b80 4 client.26767 mds_command: new command op to 24514 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:31:18.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.614+0000 7f96bb9a8b80 1 -- 172.21.15.1:0/41200449 --> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56505187cf50 con 0x5650519dd800 2024-02-02T15:31:18.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.614+0000 7f96a4a00700 1 --2- 172.21.15.1:0/41200449 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x5650519dd800 0x5650519dfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:18.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.615+0000 7f96a4a00700 1 --2- 172.21.15.1:0/41200449 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x5650519dd800 0x5650519dfbf0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3eddeed7bd605639 server_cookie=226bc26f17a5a7c in_seq=0 out_seq=0 2024-02-02T15:31:18.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.615+0000 7f968ea00700 10 client.26767 ms_handle_connect on v2:172.21.15.142:6836/2870289019 2024-02-02T15:31:18.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.616+0000 7f968ea00700 1 -- 172.21.15.1:0/41200449 <== mds.3 v2:172.21.15.142:6836/2870289019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56505187cf50 con 0x5650519dd800 2024-02-02T15:31:18.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.616+0000 7f968ea00700 10 client.26767 handle_command_reply: tid=0 2024-02-02T15:31:18.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.616+0000 7f96bb9a8b80 1 client.26767 shutdown 2024-02-02T15:31:18.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.616+0000 7f96bb9a8b80 1 -- 172.21.15.1:0/41200449 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565051970420 msgr2=0x5650519808b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:18.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.616+0000 7f96bb9a8b80 1 --2- 172.21.15.1:0/41200449 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565051970420 0x5650519808b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f96900994f0 tx=0x7f969000cd10 comp rx=0 tx=0).stop 2024-02-02T15:31:18.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.616+0000 7f96bb9a8b80 1 -- 172.21.15.1:0/41200449 shutdown_connections 2024-02-02T15:31:18.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.616+0000 7f96bb9a8b80 1 --2- 172.21.15.1:0/41200449 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56505197bbf0 0x5650519d5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.616+0000 7f96bb9a8b80 1 --2- 172.21.15.1:0/41200449 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x565051970420 0x5650519808b0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.616+0000 7f96bb9a8b80 1 --2- 172.21.15.1:0/41200449 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x5650519dd800 0x5650519dfbf0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.616+0000 7f96bb9a8b80 1 --2- 172.21.15.1:0/41200449 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x565051970d40 0x565051980df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.616+0000 7f96bb9a8b80 1 -- 172.21.15.1:0/41200449 >> 172.21.15.1:0/41200449 conn(0x5650518cbdb0 msgr2=0x5650519675c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:18.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.616+0000 7f968ea00700 0 client.26767 ms_handle_reset on v2:172.21.15.142:6836/2870289019 2024-02-02T15:31:18.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.617+0000 7f96bb9a8b80 1 -- 172.21.15.1:0/41200449 shutdown_connections 2024-02-02T15:31:18.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.617+0000 7f96bb9a8b80 1 -- 172.21.15.1:0/41200449 wait complete. 2024-02-02T15:31:18.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.617+0000 7f96bb9a8b80 20 client.26767 trim_cache size 0 max 16384 2024-02-02T15:31:18.621 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:31:18.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.617+0000 7f96a7e00700 1 -- 172.21.15.1:0/898358488 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f96800a0690 msgr2=0x7f96800a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:18.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.617+0000 7f96a7e00700 1 --2- 172.21.15.1:0/898358488 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f96800a0690 0x7f96800a2b50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f969800f600 tx=0x7f969800f9c0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.617+0000 7f96a7e00700 1 -- 172.21.15.1:0/898358488 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96a0100ff0 msgr2=0x7f96a0196e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:18.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.617+0000 7f96a7e00700 1 --2- 172.21.15.1:0/898358488 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96a0100ff0 0x7f96a0196e30 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f969000eb30 tx=0x7f969000ee40 comp rx=0 tx=0).stop 2024-02-02T15:31:18.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.618+0000 7f96a7e00700 1 -- 172.21.15.1:0/898358488 shutdown_connections 2024-02-02T15:31:18.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.618+0000 7f96a7e00700 1 --2- 172.21.15.1:0/898358488 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f96800a0690 0x7f96800a2b50 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.618+0000 7f96a7e00700 1 --2- 172.21.15.1:0/898358488 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f96a019b7d0 0x7f96a019bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.618+0000 7f96a7e00700 1 --2- 172.21.15.1:0/898358488 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f96a0100ff0 0x7f96a0196e30 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.618+0000 7f96a7e00700 1 --2- 172.21.15.1:0/898358488 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f96a0106110 0x7f96a0197370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.618+0000 7f96a7e00700 1 -- 172.21.15.1:0/898358488 >> 172.21.15.1:0/898358488 conn(0x7f96a00fc810 msgr2=0x7f96a01049c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:18.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.618+0000 7f96a7e00700 1 -- 172.21.15.1:0/898358488 shutdown_connections 2024-02-02T15:31:18.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.618+0000 7f96a7e00700 1 -- 172.21.15.1:0/898358488 wait complete. 2024-02-02T15:31:18.635 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:31:18 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/3608233761' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:31:18.638 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-02T15:31:18.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:31:18 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/3608233761' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:31:18.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:31:18 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/3608233761' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:31:18.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.874+0000 7fc6a3a00700 1 -- 172.21.15.1:0/4284027817 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc69c106110 msgr2=0x7fc69c10dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:18.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.874+0000 7fc6a3a00700 1 --2- 172.21.15.1:0/4284027817 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc69c106110 0x7fc69c10dc90 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fc694009f90 tx=0x7fc69400b330 comp rx=0 tx=0).stop 2024-02-02T15:31:18.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.874+0000 7fc6a3a00700 1 -- 172.21.15.1:0/4284027817 shutdown_connections 2024-02-02T15:31:18.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.874+0000 7fc6a3a00700 1 --2- 172.21.15.1:0/4284027817 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc69c106110 0x7fc69c10dc90 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.874+0000 7fc6a3a00700 1 --2- 172.21.15.1:0/4284027817 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc69c100ff0 0x7fc69c1013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.874+0000 7fc6a3a00700 1 --2- 172.21.15.1:0/4284027817 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc69c1019a0 0x7fc69c1059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.874+0000 7fc6a3a00700 1 -- 172.21.15.1:0/4284027817 >> 172.21.15.1:0/4284027817 conn(0x7fc69c0fc810 msgr2=0x7fc69c0fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:18.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.875+0000 7fc6a3a00700 1 -- 172.21.15.1:0/4284027817 shutdown_connections 2024-02-02T15:31:18.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.875+0000 7fc6a3a00700 1 -- 172.21.15.1:0/4284027817 wait complete. 2024-02-02T15:31:18.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.876+0000 7fc6a3a00700 1 Processor -- start 2024-02-02T15:31:18.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.876+0000 7fc6a3a00700 1 -- start start 2024-02-02T15:31:18.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.876+0000 7fc6a3a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc69c100ff0 0x7fc69c198fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:18.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.876+0000 7fc6a3a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc69c106110 0x7fc69c199500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:18.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.876+0000 7fc6a3a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc69c19d960 0x7fc69c19dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:18.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.876+0000 7fc6a3a00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc69c19e3e0 con 0x7fc69c19d960 2024-02-02T15:31:18.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.876+0000 7fc6a3a00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc69c19e550 con 0x7fc69c106110 2024-02-02T15:31:18.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.876+0000 7fc6a3a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc69c19e710 con 0x7fc69c100ff0 2024-02-02T15:31:18.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.876+0000 7fc6a1a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc69c19d960 0x7fc69c19dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:18.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.876+0000 7fc6a1000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc69c100ff0 0x7fc69c198fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:18.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.876+0000 7fc69be00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc69c106110 0x7fc69c199500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:18.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.876+0000 7fc6a1a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc69c19d960 0x7fc69c19dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54868/0 (socket says 172.21.15.1:54868) 2024-02-02T15:31:18.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.876+0000 7fc6a1a00700 1 -- 172.21.15.1:0/1958348899 learned_addr learned my addr 172.21.15.1:0/1958348899 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:31:18.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.877+0000 7fc69be00700 1 -- 172.21.15.1:0/1958348899 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc69c100ff0 msgr2=0x7fc69c198fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:18.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.877+0000 7fc69be00700 1 --2- 172.21.15.1:0/1958348899 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc69c100ff0 0x7fc69c198fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.877+0000 7fc69be00700 1 -- 172.21.15.1:0/1958348899 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc69c19d960 msgr2=0x7fc69c19dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:18.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.877+0000 7fc69be00700 1 --2- 172.21.15.1:0/1958348899 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc69c19d960 0x7fc69c19dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.877+0000 7fc69be00700 1 -- 172.21.15.1:0/1958348899 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc694009c40 con 0x7fc69c106110 2024-02-02T15:31:18.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.877+0000 7fc6a1000700 1 --2- 172.21.15.1:0/1958348899 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc69c100ff0 0x7fc69c198fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:31:18.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.877+0000 7fc6a1a00700 1 --2- 172.21.15.1:0/1958348899 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc69c19d960 0x7fc69c19dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:31:18.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.877+0000 7fc69be00700 1 --2- 172.21.15.1:0/1958348899 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc69c106110 0x7fc69c199500 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fc68400e9c0 tx=0x7fc68400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:18.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.877+0000 7fc699600700 1 -- 172.21.15.1:0/1958348899 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc68400c9c0 con 0x7fc69c106110 2024-02-02T15:31:18.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.877+0000 7fc699600700 1 -- 172.21.15.1:0/1958348899 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc684010430 con 0x7fc69c106110 2024-02-02T15:31:18.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.877+0000 7fc699600700 1 -- 172.21.15.1:0/1958348899 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc684018680 con 0x7fc69c106110 2024-02-02T15:31:18.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.878+0000 7fc6a3a00700 1 -- 172.21.15.1:0/1958348899 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc69c107ca0 con 0x7fc69c106110 2024-02-02T15:31:18.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.878+0000 7fc6a3a00700 1 -- 172.21.15.1:0/1958348899 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc69c108160 con 0x7fc69c106110 2024-02-02T15:31:18.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.881+0000 7fc699600700 1 -- 172.21.15.1:0/1958348899 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 27) v1 ==== 135200+0+0 (secure 0 0 0) 0x7fc684010860 con 0x7fc69c106110 2024-02-02T15:31:18.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.882+0000 7fc6b74ebb80 1 Processor -- start 2024-02-02T15:31:18.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.882+0000 7fc6b74ebb80 1 -- start start 2024-02-02T15:31:18.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.883+0000 7fc6b74ebb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563bedc8e010 0x563bedc8e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:18.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.883+0000 7fc6b74ebb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563bedc8e930 0x563bedc992a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:18.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.883+0000 7fc6b74ebb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563bedc997e0 0x563bedc9bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:18.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.883+0000 7fc6b74ebb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bedc024c0 con 0x563bedc8e930 2024-02-02T15:31:18.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.883+0000 7fc6b74ebb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bedbe6aa0 con 0x563bedc997e0 2024-02-02T15:31:18.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.883+0000 7fc6b74ebb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bedbe4460 con 0x563bedc8e010 2024-02-02T15:31:18.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.883+0000 7fc699600700 1 --2- 172.21.15.1:0/1958348899 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fc67c0a0690 0x7fc67c0a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:18.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.883+0000 7fc699600700 1 -- 172.21.15.1:0/1958348899 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fc684014070 con 0x7fc69c106110 2024-02-02T15:31:18.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.883+0000 7fc69be00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563bedc997e0 0x563bedc9bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:18.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.883+0000 7fc69be00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563bedc997e0 0x563bedc9bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:53980/0 (socket says 172.21.15.1:53980) 2024-02-02T15:31:18.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.883+0000 7fc6a1000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563bedc8e930 0x563bedc992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:18.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.883+0000 7fc69be00700 1 -- 172.21.15.1:0/2625002801 learned_addr learned my addr 172.21.15.1:0/2625002801 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:31:18.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.883+0000 7fc6a1a00700 1 --2- 172.21.15.1:0/2625002801 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563bedc8e010 0x563bedc8e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:18.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.883+0000 7fc6a1a00700 1 --2- 172.21.15.1:0/1958348899 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fc67c0a0690 0x7fc67c0a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:18.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.884+0000 7fc69be00700 1 -- 172.21.15.1:0/2625002801 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563bedc8e010 msgr2=0x563bedc8e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:18.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.884+0000 7fc69be00700 1 --2- 172.21.15.1:0/2625002801 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563bedc8e010 0x563bedc8e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.884+0000 7fc69be00700 1 -- 172.21.15.1:0/2625002801 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563bedc8e930 msgr2=0x563bedc992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:18.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.884+0000 7fc69be00700 1 --2- 172.21.15.1:0/2625002801 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563bedc8e930 0x563bedc992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.884+0000 7fc69be00700 1 -- 172.21.15.1:0/2625002801 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563beda613b0 con 0x563bedc997e0 2024-02-02T15:31:18.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.884+0000 7fc69be00700 1 --2- 172.21.15.1:0/2625002801 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563bedc997e0 0x563bedc9bbd0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fc684095850 tx=0x7fc684099230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69fb6f7e4cbd6960 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:18.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.884+0000 7fc692000700 1 -- 172.21.15.1:0/2625002801 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc684099a50 con 0x563bedc997e0 2024-02-02T15:31:18.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.884+0000 7fc692000700 1 -- 172.21.15.1:0/2625002801 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc684099350 con 0x563bedc997e0 2024-02-02T15:31:18.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.884+0000 7fc692000700 1 -- 172.21.15.1:0/2625002801 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6840c9800 con 0x563bedc997e0 2024-02-02T15:31:18.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.885+0000 7fc6b74ebb80 1 -- 172.21.15.1:0/2625002801 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563bedc997e0 msgr2=0x563bedc9bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:18.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.885+0000 7fc6a1a00700 1 --2- 172.21.15.1:0/1958348899 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fc67c0a0690 0x7fc67c0a2b50 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc694009040 tx=0x7fc69400b330 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:18.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.885+0000 7fc6b74ebb80 1 --2- 172.21.15.1:0/2625002801 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563bedc997e0 0x563bedc9bbd0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fc684095850 tx=0x7fc684099230 comp rx=0 tx=0).stop 2024-02-02T15:31:18.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.885+0000 7fc6b74ebb80 1 -- 172.21.15.1:0/2625002801 shutdown_connections 2024-02-02T15:31:18.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.885+0000 7fc6b74ebb80 1 --2- 172.21.15.1:0/2625002801 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563bedc997e0 0x563bedc9bbd0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.885+0000 7fc6b74ebb80 1 --2- 172.21.15.1:0/2625002801 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563bedc8e010 0x563bedc8e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.885+0000 7fc6b74ebb80 1 --2- 172.21.15.1:0/2625002801 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563bedc8e930 0x563bedc992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.885+0000 7fc6b74ebb80 1 -- 172.21.15.1:0/2625002801 >> 172.21.15.1:0/2625002801 conn(0x563bedbeddb0 msgr2=0x563bedc8a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:18.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.885+0000 7fc6b74ebb80 1 -- 172.21.15.1:0/2625002801 shutdown_connections 2024-02-02T15:31:18.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.885+0000 7fc6b74ebb80 1 -- 172.21.15.1:0/2625002801 wait complete. 2024-02-02T15:31:18.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.885+0000 7fc6b74ebb80 1 -- start start 2024-02-02T15:31:18.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.886+0000 7fc6b74ebb80 1 Processor -- start 2024-02-02T15:31:18.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.886+0000 7fc6b74ebb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563bedc8e010 0x563bedc965b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:18.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.886+0000 7fc6b74ebb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563bedc8e930 0x563bedc96af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:18.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.886+0000 7fc6b74ebb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563bedc997e0 0x563bedc97030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:18.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.886+0000 7fc6b74ebb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bedbdfd50 con 0x563bedc8e930 2024-02-02T15:31:18.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.886+0000 7fc6b74ebb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bedb7bd80 con 0x563bedc997e0 2024-02-02T15:31:18.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.886+0000 7fc6b74ebb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bedb7ba20 con 0x563bedc8e010 2024-02-02T15:31:18.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.886+0000 7fc6b74ebb80 10 client.0 _ll_register_callbacks cb 0x563bedb7d290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:31:18.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.887+0000 7fc6a1000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563bedc8e010 0x563bedc965b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:18.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.887+0000 7fc69be00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563bedc8e930 0x563bedc96af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:18.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.887+0000 7fc6a1000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563bedc8e010 0x563bedc965b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:56206/0 (socket says 172.21.15.1:56206) 2024-02-02T15:31:18.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.887+0000 7fc6a1000700 1 -- 172.21.15.1:0/4033811511 learned_addr learned my addr 172.21.15.1:0/4033811511 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:31:18.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.887+0000 7fc6a1a00700 1 --2- 172.21.15.1:0/4033811511 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563bedc997e0 0x563bedc97030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:18.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.887+0000 7fc6a1000700 1 -- 172.21.15.1:0/4033811511 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563bedc997e0 msgr2=0x563bedc97030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:18.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.887+0000 7fc6a1000700 1 --2- 172.21.15.1:0/4033811511 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563bedc997e0 0x563bedc97030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.887+0000 7fc6a1000700 1 -- 172.21.15.1:0/4033811511 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563bedc8e930 msgr2=0x563bedc96af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:18.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.887+0000 7fc6a1000700 1 --2- 172.21.15.1:0/4033811511 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563bedc8e930 0x563bedc96af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.887+0000 7fc6a1000700 1 -- 172.21.15.1:0/4033811511 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc69c108160 con 0x563bedc8e010 2024-02-02T15:31:18.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.887+0000 7fc6a1000700 1 --2- 172.21.15.1:0/4033811511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563bedc8e010 0x563bedc965b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc68c012d20 tx=0x7fc68c00f6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc2a055ae4e6f219 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:18.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.887+0000 7fc68aa00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:31:18.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.888+0000 7fc68aa00700 1 -- 172.21.15.1:0/4033811511 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc68c017070 con 0x563bedc8e010 2024-02-02T15:31:18.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.888+0000 7fc68aa00700 1 -- 172.21.15.1:0/4033811511 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fc68c00fb80 con 0x563bedc8e010 2024-02-02T15:31:18.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.888+0000 7fc6b74ebb80 1 -- 172.21.15.1:0/4033811511 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563bedc820f0 con 0x563bedc8e010 2024-02-02T15:31:18.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.888+0000 7fc68aa00700 1 -- 172.21.15.1:0/4033811511 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc68c00fd80 con 0x563bedc8e010 2024-02-02T15:31:18.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.888+0000 7fc6b74ebb80 10 client.26779 fetch_fsmap learned FSMap version 32 2024-02-02T15:31:18.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.888+0000 7fc6b74ebb80 1 -- 172.21.15.1:0/4033811511 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x563beda0db60 con 0x563bedc8e010 2024-02-02T15:31:18.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.889+0000 7fc68aa00700 1 -- 172.21.15.1:0/4033811511 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563bedc820f0 con 0x563bedc8e010 2024-02-02T15:31:18.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.889+0000 7fc68aa00700 1 -- 172.21.15.1:0/4033811511 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 32) v1 ==== 7490+0+0 (secure 0 0 0) 0x7fc68c003bc0 con 0x563bedc8e010 2024-02-02T15:31:18.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.889+0000 7fc6b74ebb80 10 client.26779 fetch_fsmap finished waiting for FSMap version 32 2024-02-02T15:31:18.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.889+0000 7fc6b74ebb80 10 client.26779 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-02-02T15:31:18.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.889+0000 7fc6b74ebb80 20 client.26779 populate_metadata read hostname 'smithi001' 2024-02-02T15:31:18.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.889+0000 7fc6b74ebb80 1 --2- 172.21.15.1:0/4033811511 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x563bedca92a0 0x563bedcab6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:18.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.889+0000 7fc6b74ebb80 4 client.26779 mds_command: new command op to 24518 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:31:18.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.889+0000 7fc6b74ebb80 1 -- 172.21.15.1:0/4033811511 --> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563bedbd8dd0 con 0x563bedca92a0 2024-02-02T15:31:18.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.889+0000 7fc69be00700 1 --2- 172.21.15.1:0/4033811511 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x563bedca92a0 0x563bedcab6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:18.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.890+0000 7fc69be00700 1 --2- 172.21.15.1:0/4033811511 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x563bedca92a0 0x563bedcab6c0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2b9a36b60493a2a2 server_cookie=40973a1f0e75f2d0 in_seq=0 out_seq=0 2024-02-02T15:31:18.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.890+0000 7fc68aa00700 10 client.26779 ms_handle_connect on v2:172.21.15.71:6836/628016199 2024-02-02T15:31:18.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.891+0000 7fc68aa00700 1 -- 172.21.15.1:0/4033811511 <== mds.2 v2:172.21.15.71:6836/628016199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x563bedbd8dd0 con 0x563bedca92a0 2024-02-02T15:31:18.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.891+0000 7fc68aa00700 10 client.26779 handle_command_reply: tid=0 2024-02-02T15:31:18.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.891+0000 7fc6b74ebb80 1 client.26779 shutdown 2024-02-02T15:31:18.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.891+0000 7fc6b74ebb80 1 -- 172.21.15.1:0/4033811511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563bedc8e010 msgr2=0x563bedc965b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:18.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.891+0000 7fc6b74ebb80 1 --2- 172.21.15.1:0/4033811511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563bedc8e010 0x563bedc965b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc68c012d20 tx=0x7fc68c00f6a0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.891+0000 7fc6b74ebb80 1 -- 172.21.15.1:0/4033811511 shutdown_connections 2024-02-02T15:31:18.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.891+0000 7fc6b74ebb80 1 --2- 172.21.15.1:0/4033811511 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563bedc997e0 0x563bedc97030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.891+0000 7fc6b74ebb80 1 --2- 172.21.15.1:0/4033811511 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x563bedca92a0 0x563bedcab6c0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.891+0000 7fc6b74ebb80 1 --2- 172.21.15.1:0/4033811511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563bedc8e010 0x563bedc965b0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.891+0000 7fc6b74ebb80 1 --2- 172.21.15.1:0/4033811511 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563bedc8e930 0x563bedc96af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.891+0000 7fc68aa00700 0 client.26779 ms_handle_reset on v2:172.21.15.71:6836/628016199 2024-02-02T15:31:18.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.891+0000 7fc6b74ebb80 1 -- 172.21.15.1:0/4033811511 >> 172.21.15.1:0/4033811511 conn(0x563bedbeddb0 msgr2=0x563bedc89fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:18.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.891+0000 7fc6b74ebb80 1 -- 172.21.15.1:0/4033811511 shutdown_connections 2024-02-02T15:31:18.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.891+0000 7fc6b74ebb80 1 -- 172.21.15.1:0/4033811511 wait complete. 2024-02-02T15:31:18.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.891+0000 7fc6b74ebb80 20 client.26779 trim_cache size 0 max 16384 2024-02-02T15:31:18.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.907+0000 7fc6b74ebb80 1 Processor -- start 2024-02-02T15:31:18.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.907+0000 7fc6b74ebb80 1 -- start start 2024-02-02T15:31:18.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.907+0000 7fc6b74ebb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563bedc8e010 0x563bedca0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:18.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.907+0000 7fc6b74ebb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563bedc8e930 0x563bedc85e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:18.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.907+0000 7fc6b74ebb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563bedc997e0 0x563bedc90f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:18.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.907+0000 7fc6b74ebb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x563beda636a0 con 0x563bedc8e010 2024-02-02T15:31:18.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.907+0000 7fc6b74ebb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bedc86340 con 0x563bedc997e0 2024-02-02T15:31:18.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.907+0000 7fc6b74ebb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bedc91480 con 0x563bedc8e930 2024-02-02T15:31:18.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.908+0000 7fc69be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563bedc8e930 0x563bedc85e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:18.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.908+0000 7fc69be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563bedc8e930 0x563bedc85e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:56218/0 (socket says 172.21.15.1:56218) 2024-02-02T15:31:18.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.908+0000 7fc69be00700 1 -- 172.21.15.1:0/2429942775 learned_addr learned my addr 172.21.15.1:0/2429942775 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:31:18.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.908+0000 7fc6a1000700 1 --2- 172.21.15.1:0/2429942775 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563bedc8e010 0x563bedca0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:18.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.908+0000 7fc69be00700 1 -- 172.21.15.1:0/2429942775 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563bedc997e0 msgr2=0x563bedc90f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:18.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.908+0000 7fc69be00700 1 --2- 172.21.15.1:0/2429942775 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563bedc997e0 0x563bedc90f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.908+0000 7fc69be00700 1 -- 172.21.15.1:0/2429942775 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563bedc8e010 msgr2=0x563bedca0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:18.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.908+0000 7fc69be00700 1 --2- 172.21.15.1:0/2429942775 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563bedc8e010 0x563bedca0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.908+0000 7fc69be00700 1 -- 172.21.15.1:0/2429942775 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563bedba5ea0 con 0x563bedc8e930 2024-02-02T15:31:18.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.908+0000 7fc69be00700 1 --2- 172.21.15.1:0/2429942775 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563bedc8e930 0x563bedc85e00 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc68401e000 tx=0x7fc684095040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78e8a6ae008cfc90 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:18.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.909+0000 7fc692a00700 1 -- 172.21.15.1:0/2429942775 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6840a13c0 con 0x563bedc8e930 2024-02-02T15:31:18.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.909+0000 7fc692a00700 1 -- 172.21.15.1:0/2429942775 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc6840105a0 con 0x563bedc8e930 2024-02-02T15:31:18.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.909+0000 7fc692a00700 1 -- 172.21.15.1:0/2429942775 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6840995b0 con 0x563bedc8e930 2024-02-02T15:31:18.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.909+0000 7fc6b74ebb80 1 -- 172.21.15.1:0/2429942775 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563bedc8e930 msgr2=0x563bedc85e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:18.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.909+0000 7fc6b74ebb80 1 --2- 172.21.15.1:0/2429942775 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563bedc8e930 0x563bedc85e00 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fc68401e000 tx=0x7fc684095040 comp rx=0 tx=0).stop 2024-02-02T15:31:18.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.910+0000 7fc6b74ebb80 1 -- 172.21.15.1:0/2429942775 shutdown_connections 2024-02-02T15:31:18.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.910+0000 7fc6b74ebb80 1 --2- 172.21.15.1:0/2429942775 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563bedc997e0 0x563bedc90f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.910+0000 7fc6b74ebb80 1 --2- 172.21.15.1:0/2429942775 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563bedc8e010 0x563bedca0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.910+0000 7fc6b74ebb80 1 --2- 172.21.15.1:0/2429942775 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563bedc8e930 0x563bedc85e00 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.910+0000 7fc6b74ebb80 1 -- 172.21.15.1:0/2429942775 >> 172.21.15.1:0/2429942775 conn(0x563bedbeddb0 msgr2=0x563bedca4900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:18.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.910+0000 7fc6b74ebb80 1 -- 172.21.15.1:0/2429942775 shutdown_connections 2024-02-02T15:31:18.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.910+0000 7fc6b74ebb80 1 -- 172.21.15.1:0/2429942775 wait complete. 2024-02-02T15:31:18.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.910+0000 7fc6b74ebb80 1 -- start start 2024-02-02T15:31:18.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.910+0000 7fc6b74ebb80 1 Processor -- start 2024-02-02T15:31:18.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.911+0000 7fc6b74ebb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563bedc8e010 0x563bedc9e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:18.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.911+0000 7fc6b74ebb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563bedc8e930 0x563bedc9e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:18.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.911+0000 7fc6b74ebb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563bedc997e0 0x563bedcf9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:18.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.911+0000 7fc6b74ebb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bedc9efc0 con 0x563bedc997e0 2024-02-02T15:31:18.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.911+0000 7fc6b74ebb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bedc9f130 con 0x563bedc8e010 2024-02-02T15:31:18.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.911+0000 7fc6b74ebb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bedcfa300 con 0x563bedc8e930 2024-02-02T15:31:18.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.911+0000 7fc6b74ebb80 10 client.0 _ll_register_callbacks cb 0x563bedb7d290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:31:18.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.911+0000 7fc69be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563bedc8e930 0x563bedc9e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:18.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.911+0000 7fc6a1000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563bedc8e010 0x563bedc9e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:18.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.911+0000 7fc69be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563bedc8e930 0x563bedc9e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:56230/0 (socket says 172.21.15.1:56230) 2024-02-02T15:31:18.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.912+0000 7fc69be00700 1 -- 172.21.15.1:0/1805805306 learned_addr learned my addr 172.21.15.1:0/1805805306 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:31:18.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.912+0000 7fc6a1a00700 1 --2- 172.21.15.1:0/1805805306 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563bedc997e0 0x563bedcf9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:18.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.912+0000 7fc69be00700 1 -- 172.21.15.1:0/1805805306 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563bedc8e010 msgr2=0x563bedc9e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:18.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.912+0000 7fc69be00700 1 --2- 172.21.15.1:0/1805805306 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563bedc8e010 0x563bedc9e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.912+0000 7fc69be00700 1 -- 172.21.15.1:0/1805805306 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563bedc997e0 msgr2=0x563bedcf9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:18.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.912+0000 7fc69be00700 1 --2- 172.21.15.1:0/1805805306 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563bedc997e0 0x563bedcf9dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.912+0000 7fc69be00700 1 -- 172.21.15.1:0/1805805306 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563beda613b0 con 0x563bedc8e930 2024-02-02T15:31:18.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.912+0000 7fc6a1000700 1 --2- 172.21.15.1:0/1805805306 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563bedc8e010 0x563bedc9e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:31:18.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.912+0000 7fc6a1a00700 1 --2- 172.21.15.1:0/1805805306 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563bedc997e0 0x563bedcf9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:31:18.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.912+0000 7fc69be00700 1 --2- 172.21.15.1:0/1805805306 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563bedc8e930 0x563bedc9e850 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc6840840b0 tx=0x7fc684009790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c62a1e2ed5a1e0b2 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:18.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.912+0000 7fc68aa00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:31:18.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.912+0000 7fc68aa00700 1 -- 172.21.15.1:0/1805805306 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6840c9a00 con 0x563bedc8e930 2024-02-02T15:31:18.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.912+0000 7fc68aa00700 1 -- 172.21.15.1:0/1805805306 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fc684098840 con 0x563bedc8e930 2024-02-02T15:31:18.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.912+0000 7fc6b74ebb80 1 -- 172.21.15.1:0/1805805306 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc684099350 con 0x563bedc8e930 2024-02-02T15:31:18.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.913+0000 7fc68aa00700 1 -- 172.21.15.1:0/1805805306 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc684065130 con 0x563bedc8e930 2024-02-02T15:31:18.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.913+0000 7fc6b74ebb80 10 client.26785 fetch_fsmap learned FSMap version 32 2024-02-02T15:31:18.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.913+0000 7fc6b74ebb80 1 -- 172.21.15.1:0/1805805306 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x563beda65cd0 con 0x563bedc8e930 2024-02-02T15:31:18.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.913+0000 7fc68aa00700 1 -- 172.21.15.1:0/1805805306 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc684099350 con 0x563bedc8e930 2024-02-02T15:31:18.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.913+0000 7fc68aa00700 1 -- 172.21.15.1:0/1805805306 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 32) v1 ==== 7490+0+0 (secure 0 0 0) 0x7fc6840a1030 con 0x563bedc8e930 2024-02-02T15:31:18.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.913+0000 7fc6b74ebb80 10 client.26785 fetch_fsmap finished waiting for FSMap version 32 2024-02-02T15:31:18.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.913+0000 7fc6b74ebb80 10 client.26785 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-02-02T15:31:18.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.913+0000 7fc6b74ebb80 20 client.26785 populate_metadata read hostname 'smithi001' 2024-02-02T15:31:18.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.913+0000 7fc6b74ebb80 1 --2- 172.21.15.1:0/1805805306 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x563bedcfb510 0x563bedcfd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:18.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.913+0000 7fc6b74ebb80 4 client.26785 mds_command: new command op to 24518 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:31:18.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.913+0000 7fc6b74ebb80 1 -- 172.21.15.1:0/1805805306 --> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563bedb9ef50 con 0x563bedcfb510 2024-02-02T15:31:18.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.914+0000 7fc6a1000700 1 --2- 172.21.15.1:0/1805805306 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x563bedcfb510 0x563bedcfd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:18.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.914+0000 7fc6a1000700 1 --2- 172.21.15.1:0/1805805306 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x563bedcfb510 0x563bedcfd900 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=66e0f03e57f6ddb1 server_cookie=39a38cfe5a6dbb7d in_seq=0 out_seq=0 2024-02-02T15:31:18.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.914+0000 7fc68aa00700 10 client.26785 ms_handle_connect on v2:172.21.15.71:6836/628016199 2024-02-02T15:31:18.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.915+0000 7fc68aa00700 1 -- 172.21.15.1:0/1805805306 <== mds.2 v2:172.21.15.71:6836/628016199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563bedb9ef50 con 0x563bedcfb510 2024-02-02T15:31:18.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.915+0000 7fc68aa00700 10 client.26785 handle_command_reply: tid=0 2024-02-02T15:31:18.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.915+0000 7fc6b74ebb80 1 client.26785 shutdown 2024-02-02T15:31:18.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.915+0000 7fc6b74ebb80 1 -- 172.21.15.1:0/1805805306 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563bedc8e930 msgr2=0x563bedc9e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:18.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.915+0000 7fc6b74ebb80 1 --2- 172.21.15.1:0/1805805306 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563bedc8e930 0x563bedc9e850 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc6840840b0 tx=0x7fc684009790 comp rx=0 tx=0).stop 2024-02-02T15:31:18.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.915+0000 7fc6b74ebb80 1 -- 172.21.15.1:0/1805805306 shutdown_connections 2024-02-02T15:31:18.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.915+0000 7fc6b74ebb80 1 --2- 172.21.15.1:0/1805805306 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563bedc997e0 0x563bedcf9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.915+0000 7fc6b74ebb80 1 --2- 172.21.15.1:0/1805805306 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563bedc8e010 0x563bedc9e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.915+0000 7fc6b74ebb80 1 --2- 172.21.15.1:0/1805805306 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x563bedcfb510 0x563bedcfd900 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.915+0000 7fc6b74ebb80 1 --2- 172.21.15.1:0/1805805306 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563bedc8e930 0x563bedc9e850 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.915+0000 7fc68aa00700 0 client.26785 ms_handle_reset on v2:172.21.15.71:6836/628016199 2024-02-02T15:31:18.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.915+0000 7fc6b74ebb80 1 -- 172.21.15.1:0/1805805306 >> 172.21.15.1:0/1805805306 conn(0x563bedbeddb0 msgr2=0x563bedc919a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:18.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.916+0000 7fc6b74ebb80 1 -- 172.21.15.1:0/1805805306 shutdown_connections 2024-02-02T15:31:18.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.916+0000 7fc6b74ebb80 1 -- 172.21.15.1:0/1805805306 wait complete. 2024-02-02T15:31:18.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.916+0000 7fc6b74ebb80 20 client.26785 trim_cache size 0 max 16384 2024-02-02T15:31:18.923 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:31:18.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.916+0000 7fc6a3a00700 1 -- 172.21.15.1:0/1958348899 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fc67c0a0690 msgr2=0x7fc67c0a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:18.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.916+0000 7fc6a3a00700 1 --2- 172.21.15.1:0/1958348899 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fc67c0a0690 0x7fc67c0a2b50 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc694009040 tx=0x7fc69400b330 comp rx=0 tx=0).stop 2024-02-02T15:31:18.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.916+0000 7fc6a3a00700 1 -- 172.21.15.1:0/1958348899 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc69c106110 msgr2=0x7fc69c199500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:18.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.916+0000 7fc6a3a00700 1 --2- 172.21.15.1:0/1958348899 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc69c106110 0x7fc69c199500 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fc68400e9c0 tx=0x7fc68400ecd0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.916+0000 7fc6a3a00700 1 -- 172.21.15.1:0/1958348899 shutdown_connections 2024-02-02T15:31:18.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.916+0000 7fc6a3a00700 1 --2- 172.21.15.1:0/1958348899 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fc67c0a0690 0x7fc67c0a2b50 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.916+0000 7fc6a3a00700 1 --2- 172.21.15.1:0/1958348899 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc69c19d960 0x7fc69c19dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.916+0000 7fc6a3a00700 1 --2- 172.21.15.1:0/1958348899 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc69c100ff0 0x7fc69c198fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.917+0000 7fc6a3a00700 1 --2- 172.21.15.1:0/1958348899 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc69c106110 0x7fc69c199500 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:18.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.917+0000 7fc6a3a00700 1 -- 172.21.15.1:0/1958348899 >> 172.21.15.1:0/1958348899 conn(0x7fc69c0fc810 msgr2=0x7fc69c0fd320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:18.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.917+0000 7fc6a3a00700 1 -- 172.21.15.1:0/1958348899 shutdown_connections 2024-02-02T15:31:18.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:18.917+0000 7fc6a3a00700 1 -- 172.21.15.1:0/1958348899 wait complete. 2024-02-02T15:31:18.937 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-02T15:31:19.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.153+0000 7f0700c00700 1 -- 172.21.15.1:0/3461856193 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f06fc1019a0 msgr2=0x7f06fc1059e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:19.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.153+0000 7f0700c00700 1 --2- 172.21.15.1:0/3461856193 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f06fc1019a0 0x7f06fc1059e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f06e40098f0 tx=0x7f06e4009c00 comp rx=0 tx=0).stop 2024-02-02T15:31:19.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.153+0000 7f0700c00700 1 -- 172.21.15.1:0/3461856193 shutdown_connections 2024-02-02T15:31:19.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.153+0000 7f0700c00700 1 --2- 172.21.15.1:0/3461856193 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06fc106110 0x7f06fc10dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:19.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.153+0000 7f0700c00700 1 --2- 172.21.15.1:0/3461856193 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f06fc100ff0 0x7f06fc1013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:19.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.153+0000 7f0700c00700 1 --2- 172.21.15.1:0/3461856193 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f06fc1019a0 0x7f06fc1059e0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:19.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.153+0000 7f0700c00700 1 -- 172.21.15.1:0/3461856193 >> 172.21.15.1:0/3461856193 conn(0x7f06fc0fc810 msgr2=0x7f06fc0fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:19.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.153+0000 7f0700c00700 1 -- 172.21.15.1:0/3461856193 shutdown_connections 2024-02-02T15:31:19.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.153+0000 7f0700c00700 1 -- 172.21.15.1:0/3461856193 wait complete. 2024-02-02T15:31:19.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.154+0000 7f0700c00700 1 Processor -- start 2024-02-02T15:31:19.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.154+0000 7f0700c00700 1 -- start start 2024-02-02T15:31:19.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.154+0000 7f0700c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06fc100ff0 0x7f06fc196ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:19.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.154+0000 7f0700c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f06fc106110 0x7f06fc197530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:19.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.154+0000 7f0700c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f06fc19b8f0 0x7f06fc19bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:19.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.154+0000 7f0700c00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06fc19c370 con 0x7f06fc106110 2024-02-02T15:31:19.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.154+0000 7f0700c00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06fc19c4e0 con 0x7f06fc19b8f0 2024-02-02T15:31:19.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.154+0000 7f0700c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06fc19c6a0 con 0x7f06fc100ff0 2024-02-02T15:31:19.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.155+0000 7f06f9400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f06fc106110 0x7f06fc197530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:19.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.155+0000 7f06fa800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f06fc19b8f0 0x7f06fc19bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:19.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.155+0000 7f06f9e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06fc100ff0 0x7f06fc196ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:19.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.155+0000 7f06fa800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f06fc19b8f0 0x7f06fc19bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:54012/0 (socket says 172.21.15.1:54012) 2024-02-02T15:31:19.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.155+0000 7f06f9400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f06fc106110 0x7f06fc197530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54938/0 (socket says 172.21.15.1:54938) 2024-02-02T15:31:19.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.155+0000 7f06fa800700 1 -- 172.21.15.1:0/3898539415 learned_addr learned my addr 172.21.15.1:0/3898539415 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:31:19.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.155+0000 7f06f9400700 1 -- 172.21.15.1:0/3898539415 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06fc100ff0 msgr2=0x7f06fc196ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:19.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.155+0000 7f06f9400700 1 --2- 172.21.15.1:0/3898539415 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06fc100ff0 0x7f06fc196ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:19.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.155+0000 7f06f9400700 1 -- 172.21.15.1:0/3898539415 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f06fc19b8f0 msgr2=0x7f06fc19bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:19.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.155+0000 7f06f9400700 1 --2- 172.21.15.1:0/3898539415 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f06fc19b8f0 0x7f06fc19bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:19.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.155+0000 7f06f9400700 1 -- 172.21.15.1:0/3898539415 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06ec0095f0 con 0x7f06fc106110 2024-02-02T15:31:19.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.155+0000 7f06fa800700 1 --2- 172.21.15.1:0/3898539415 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f06fc19b8f0 0x7f06fc19bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:31:19.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.155+0000 7f06f9400700 1 --2- 172.21.15.1:0/3898539415 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f06fc106110 0x7f06fc197530 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f06e4009230 tx=0x7f06e400f9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:19.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.156+0000 7f06f2a00700 1 -- 172.21.15.1:0/3898539415 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e401c070 con 0x7f06fc106110 2024-02-02T15:31:19.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.156+0000 7f0700c00700 1 -- 172.21.15.1:0/3898539415 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06e4009520 con 0x7f06fc106110 2024-02-02T15:31:19.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.156+0000 7f06f2a00700 1 -- 172.21.15.1:0/3898539415 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f06e400ba40 con 0x7f06fc106110 2024-02-02T15:31:19.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.156+0000 7f06f2a00700 1 -- 172.21.15.1:0/3898539415 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e4005720 con 0x7f06fc106110 2024-02-02T15:31:19.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.156+0000 7f0700c00700 1 -- 172.21.15.1:0/3898539415 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06fc1a3e30 con 0x7f06fc106110 2024-02-02T15:31:19.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.159+0000 7f06f2a00700 1 -- 172.21.15.1:0/3898539415 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 27) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f06e4020030 con 0x7f06fc106110 2024-02-02T15:31:19.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.160+0000 7f07146c8b80 1 Processor -- start 2024-02-02T15:31:19.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.161+0000 7f07146c8b80 1 -- start start 2024-02-02T15:31:19.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.161+0000 7f06f2a00700 1 --2- 172.21.15.1:0/3898539415 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f06dc0a0690 0x7f06dc0a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:19.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.161+0000 7f06f2a00700 1 -- 172.21.15.1:0/3898539415 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f06e40ccf30 con 0x7f06fc106110 2024-02-02T15:31:19.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.161+0000 7f07146c8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609aa59b010 0x5609aa59b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:19.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.161+0000 7f07146c8b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5609aa59b930 0x5609aa5a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:19.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.161+0000 7f07146c8b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5609aa5a67e0 0x5609aa5a8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:19.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.161+0000 7f06fa800700 1 --2- 172.21.15.1:0/3898539415 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f06dc0a0690 0x7f06dc0a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:19.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.161+0000 7f07146c8b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609aa50f4c0 con 0x5609aa59b930 2024-02-02T15:31:19.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.161+0000 7f07146c8b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609aa4f3aa0 con 0x5609aa5a67e0 2024-02-02T15:31:19.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.161+0000 7f07146c8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609aa4f1460 con 0x5609aa59b010 2024-02-02T15:31:19.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.161+0000 7f06f9e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609aa59b010 0x5609aa59b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:19.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.161+0000 7f06f9e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609aa59b010 0x5609aa59b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:56248/0 (socket says 172.21.15.1:56248) 2024-02-02T15:31:19.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.161+0000 7f06f9e00700 1 -- 172.21.15.1:0/3650112467 learned_addr learned my addr 172.21.15.1:0/3650112467 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:31:19.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.162+0000 7f06fa800700 1 --2- 172.21.15.1:0/3650112467 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5609aa5a67e0 0x5609aa5a8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:19.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.162+0000 7f06f9400700 1 --2- 172.21.15.1:0/3650112467 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5609aa59b930 0x5609aa5a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:19.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.162+0000 7f06f9e00700 1 -- 172.21.15.1:0/3650112467 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5609aa5a67e0 msgr2=0x5609aa5a8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:19.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.162+0000 7f06f9e00700 1 --2- 172.21.15.1:0/3650112467 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5609aa5a67e0 0x5609aa5a8bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:19.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.162+0000 7f06f9e00700 1 -- 172.21.15.1:0/3650112467 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5609aa59b930 msgr2=0x5609aa5a62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:19.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.162+0000 7f06f9e00700 1 --2- 172.21.15.1:0/3650112467 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5609aa59b930 0x5609aa5a62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:19.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.162+0000 7f06f9e00700 1 -- 172.21.15.1:0/3650112467 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609aa36e3b0 con 0x5609aa59b010 2024-02-02T15:31:19.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.162+0000 7f06f9e00700 1 --2- 172.21.15.1:0/3650112467 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609aa59b010 0x5609aa59b3f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f06ec011470 tx=0x7f06ec011780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8bb743eb2abd3dc9 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:19.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.162+0000 7f06fa800700 1 --2- 172.21.15.1:0/3898539415 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f06dc0a0690 0x7f06dc0a2b50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f06f4009d30 tx=0x7f06f4009920 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:19.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.162+0000 7f06eb000700 1 -- 172.21.15.1:0/3650112467 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06ec00e670 con 0x5609aa59b010 2024-02-02T15:31:19.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.162+0000 7f06eb000700 1 -- 172.21.15.1:0/3650112467 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f06ec00ec90 con 0x5609aa59b010 2024-02-02T15:31:19.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.162+0000 7f06eb000700 1 -- 172.21.15.1:0/3650112467 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06ec019cd0 con 0x5609aa59b010 2024-02-02T15:31:19.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.163+0000 7f07146c8b80 1 -- 172.21.15.1:0/3650112467 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609aa59b010 msgr2=0x5609aa59b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:19.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.163+0000 7f07146c8b80 1 --2- 172.21.15.1:0/3650112467 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609aa59b010 0x5609aa59b3f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f06ec011470 tx=0x7f06ec011780 comp rx=0 tx=0).stop 2024-02-02T15:31:19.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.163+0000 7f07146c8b80 1 -- 172.21.15.1:0/3650112467 shutdown_connections 2024-02-02T15:31:19.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.163+0000 7f07146c8b80 1 --2- 172.21.15.1:0/3650112467 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5609aa5a67e0 0x5609aa5a8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:19.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.163+0000 7f07146c8b80 1 --2- 172.21.15.1:0/3650112467 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609aa59b010 0x5609aa59b3f0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:19.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.163+0000 7f07146c8b80 1 --2- 172.21.15.1:0/3650112467 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5609aa59b930 0x5609aa5a62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:19.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.163+0000 7f07146c8b80 1 -- 172.21.15.1:0/3650112467 >> 172.21.15.1:0/3650112467 conn(0x5609aa4fadb0 msgr2=0x5609aa5973b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:19.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.163+0000 7f07146c8b80 1 -- 172.21.15.1:0/3650112467 shutdown_connections 2024-02-02T15:31:19.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.163+0000 7f07146c8b80 1 -- 172.21.15.1:0/3650112467 wait complete. 2024-02-02T15:31:19.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.164+0000 7f07146c8b80 1 -- start start 2024-02-02T15:31:19.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.164+0000 7f07146c8b80 1 Processor -- start 2024-02-02T15:31:19.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.165+0000 7f07146c8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609aa59b010 0x5609aa5a6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:19.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.165+0000 7f07146c8b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5609aa59b930 0x5609aa5a3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:19.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.165+0000 7f07146c8b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5609aa5a67e0 0x5609aa5a3b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:19.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.165+0000 7f07146c8b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609aa4ecd50 con 0x5609aa5a67e0 2024-02-02T15:31:19.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.165+0000 7f07146c8b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609aa488d80 con 0x5609aa59b930 2024-02-02T15:31:19.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.165+0000 7f07146c8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609aa488a20 con 0x5609aa59b010 2024-02-02T15:31:19.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.165+0000 7f07146c8b80 10 client.0 _ll_register_callbacks cb 0x5609aa48a290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:31:19.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.165+0000 7f06f9400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5609aa59b930 0x5609aa5a3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:19.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.165+0000 7f06f9e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609aa59b010 0x5609aa5a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:19.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.165+0000 7f06f9400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5609aa59b930 0x5609aa5a3450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:54030/0 (socket says 172.21.15.1:54030) 2024-02-02T15:31:19.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.165+0000 7f06f9400700 1 -- 172.21.15.1:0/4067550584 learned_addr learned my addr 172.21.15.1:0/4067550584 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:31:19.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.166+0000 7f06fa800700 1 --2- 172.21.15.1:0/4067550584 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5609aa5a67e0 0x5609aa5a3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:19.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.166+0000 7f06f9400700 1 -- 172.21.15.1:0/4067550584 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609aa59b010 msgr2=0x5609aa5a6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:19.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.166+0000 7f06f9400700 1 --2- 172.21.15.1:0/4067550584 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609aa59b010 0x5609aa5a6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:19.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.166+0000 7f06f9400700 1 -- 172.21.15.1:0/4067550584 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5609aa5a67e0 msgr2=0x5609aa5a3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:19.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.166+0000 7f06f9400700 1 --2- 172.21.15.1:0/4067550584 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5609aa5a67e0 0x5609aa5a3b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:19.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.166+0000 7f06f9400700 1 -- 172.21.15.1:0/4067550584 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06ec009c20 con 0x5609aa59b930 2024-02-02T15:31:19.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.166+0000 7f06f9400700 1 --2- 172.21.15.1:0/4067550584 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5609aa59b930 0x5609aa5a3450 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f06e4000c00 tx=0x7f06e40a00d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b66ec7f46c5ab27f server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:19.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.166+0000 7f06e3400700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:31:19.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.166+0000 7f06e3400700 1 -- 172.21.15.1:0/4067550584 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e409a130 con 0x5609aa59b930 2024-02-02T15:31:19.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.166+0000 7f06e3400700 1 -- 172.21.15.1:0/4067550584 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f06e409df60 con 0x5609aa59b930 2024-02-02T15:31:19.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.166+0000 7f07146c8b80 1 -- 172.21.15.1:0/4067550584 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5609aa58f0f0 con 0x5609aa59b930 2024-02-02T15:31:19.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.166+0000 7f06e3400700 1 -- 172.21.15.1:0/4067550584 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f06e40a2110 con 0x5609aa59b930 2024-02-02T15:31:19.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.166+0000 7f07146c8b80 10 client.26804 fetch_fsmap learned FSMap version 32 2024-02-02T15:31:19.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.166+0000 7f07146c8b80 1 -- 172.21.15.1:0/4067550584 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5609aa31ab60 con 0x5609aa59b930 2024-02-02T15:31:19.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.167+0000 7f06e3400700 1 -- 172.21.15.1:0/4067550584 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5609aa58f0f0 con 0x5609aa59b930 2024-02-02T15:31:19.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.167+0000 7f06e3400700 1 -- 172.21.15.1:0/4067550584 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 32) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f06e409a390 con 0x5609aa59b930 2024-02-02T15:31:19.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.167+0000 7f07146c8b80 10 client.26804 fetch_fsmap finished waiting for FSMap version 32 2024-02-02T15:31:19.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.167+0000 7f07146c8b80 10 client.26804 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-02T15:31:19.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.167+0000 7f07146c8b80 20 client.26804 populate_metadata read hostname 'smithi001' 2024-02-02T15:31:19.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.167+0000 7f07146c8b80 1 --2- 172.21.15.1:0/4067550584 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x5609aa5ba260 0x5609aa5bc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:19.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.167+0000 7f07146c8b80 4 client.26804 mds_command: new command op to 24529 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:31:19.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.167+0000 7f07146c8b80 1 -- 172.21.15.1:0/4067550584 --> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5609aa4e5dd0 con 0x5609aa5ba260 2024-02-02T15:31:19.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.167+0000 7f06f9e00700 1 --2- 172.21.15.1:0/4067550584 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x5609aa5ba260 0x5609aa5bc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:19.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.168+0000 7f06f9e00700 1 --2- 172.21.15.1:0/4067550584 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x5609aa5ba260 0x5609aa5bc680 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8334bd91ea4292af server_cookie=6d90835b1eff41ef in_seq=0 out_seq=0 2024-02-02T15:31:19.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.168+0000 7f06e3400700 10 client.26804 ms_handle_connect on v2:172.21.15.142:6838/321675421 2024-02-02T15:31:19.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.168+0000 7f06e3400700 1 -- 172.21.15.1:0/4067550584 <== mds.1 v2:172.21.15.142:6838/321675421 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5609aa4e5dd0 con 0x5609aa5ba260 2024-02-02T15:31:19.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.168+0000 7f06e3400700 10 client.26804 handle_command_reply: tid=0 2024-02-02T15:31:19.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.168+0000 7f07146c8b80 1 client.26804 shutdown 2024-02-02T15:31:19.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.169+0000 7f07146c8b80 1 -- 172.21.15.1:0/4067550584 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5609aa59b930 msgr2=0x5609aa5a3450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:19.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.169+0000 7f07146c8b80 1 --2- 172.21.15.1:0/4067550584 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5609aa59b930 0x5609aa5a3450 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f06e4000c00 tx=0x7f06e40a00d0 comp rx=0 tx=0).stop 2024-02-02T15:31:19.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.169+0000 7f07146c8b80 1 -- 172.21.15.1:0/4067550584 shutdown_connections 2024-02-02T15:31:19.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.169+0000 7f07146c8b80 1 --2- 172.21.15.1:0/4067550584 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x5609aa5ba260 0x5609aa5bc680 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:19.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.169+0000 7f07146c8b80 1 --2- 172.21.15.1:0/4067550584 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5609aa5a67e0 0x5609aa5a3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:19.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.169+0000 7f07146c8b80 1 --2- 172.21.15.1:0/4067550584 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609aa59b010 0x5609aa5a6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:19.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.169+0000 7f07146c8b80 1 --2- 172.21.15.1:0/4067550584 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5609aa59b930 0x5609aa5a3450 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:19.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.169+0000 7f06e3400700 0 client.26804 ms_handle_reset on v2:172.21.15.142:6838/321675421 2024-02-02T15:31:19.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.169+0000 7f07146c8b80 1 -- 172.21.15.1:0/4067550584 >> 172.21.15.1:0/4067550584 conn(0x5609aa4fadb0 msgr2=0x5609aa596fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:19.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.169+0000 7f07146c8b80 1 -- 172.21.15.1:0/4067550584 shutdown_connections 2024-02-02T15:31:19.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.169+0000 7f07146c8b80 1 -- 172.21.15.1:0/4067550584 wait complete. 2024-02-02T15:31:19.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.169+0000 7f07146c8b80 20 client.26804 trim_cache size 0 max 16384 2024-02-02T15:31:19.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.183+0000 7f07146c8b80 1 Processor -- start 2024-02-02T15:31:19.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.183+0000 7f07146c8b80 1 -- start start 2024-02-02T15:31:19.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.184+0000 7f07146c8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609aa59b010 0x5609aa5b1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:19.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.184+0000 7f07146c8b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5609aa59b930 0x5609aa59c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:19.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.184+0000 7f07146c8b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5609aa5a67e0 0x5609aa595b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:19.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.184+0000 7f07146c8b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609aa3706a0 con 0x5609aa59b930 2024-02-02T15:31:19.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.184+0000 7f07146c8b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609aa5a4190 con 0x5609aa5a67e0 2024-02-02T15:31:19.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.184+0000 7f07146c8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609aa59f340 con 0x5609aa59b010 2024-02-02T15:31:19.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.184+0000 7f06f9e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609aa59b010 0x5609aa5b1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:19.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.184+0000 7f06f9e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609aa59b010 0x5609aa5b1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:56274/0 (socket says 172.21.15.1:56274) 2024-02-02T15:31:19.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.184+0000 7f06f9e00700 1 -- 172.21.15.1:0/2794090114 learned_addr learned my addr 172.21.15.1:0/2794090114 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:31:19.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.184+0000 7f06fa800700 1 --2- 172.21.15.1:0/2794090114 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5609aa5a67e0 0x5609aa595b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:19.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.184+0000 7f06f9400700 1 --2- 172.21.15.1:0/2794090114 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5609aa59b930 0x5609aa59c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:19.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.184+0000 7f06f9e00700 1 -- 172.21.15.1:0/2794090114 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5609aa5a67e0 msgr2=0x5609aa595b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:19.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.184+0000 7f06f9e00700 1 --2- 172.21.15.1:0/2794090114 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5609aa5a67e0 0x5609aa595b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:19.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.184+0000 7f06f9e00700 1 -- 172.21.15.1:0/2794090114 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5609aa59b930 msgr2=0x5609aa59c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:19.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.184+0000 7f06f9e00700 1 --2- 172.21.15.1:0/2794090114 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5609aa59b930 0x5609aa59c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:19.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.184+0000 7f06f9e00700 1 -- 172.21.15.1:0/2794090114 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609aa4b2ea0 con 0x5609aa59b010 2024-02-02T15:31:19.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.185+0000 7f06f9e00700 1 --2- 172.21.15.1:0/2794090114 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609aa59b010 0x5609aa5b1780 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f06ec011470 tx=0x7f06ec019430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6598cc9a8f12ab1 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:19.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.185+0000 7f06eba00700 1 -- 172.21.15.1:0/2794090114 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06ec019ae0 con 0x5609aa59b010 2024-02-02T15:31:19.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.185+0000 7f06eba00700 1 -- 172.21.15.1:0/2794090114 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f06ec019690 con 0x5609aa59b010 2024-02-02T15:31:19.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.185+0000 7f06eba00700 1 -- 172.21.15.1:0/2794090114 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06ec019ea0 con 0x5609aa59b010 2024-02-02T15:31:19.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.186+0000 7f07146c8b80 1 -- 172.21.15.1:0/2794090114 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609aa59b010 msgr2=0x5609aa5b1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:19.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.186+0000 7f07146c8b80 1 --2- 172.21.15.1:0/2794090114 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609aa59b010 0x5609aa5b1780 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f06ec011470 tx=0x7f06ec019430 comp rx=0 tx=0).stop 2024-02-02T15:31:19.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.186+0000 7f07146c8b80 1 -- 172.21.15.1:0/2794090114 shutdown_connections 2024-02-02T15:31:19.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.186+0000 7f07146c8b80 1 --2- 172.21.15.1:0/2794090114 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5609aa5a67e0 0x5609aa595b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:19.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.186+0000 7f07146c8b80 1 --2- 172.21.15.1:0/2794090114 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609aa59b010 0x5609aa5b1780 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:19.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.186+0000 7f07146c8b80 1 --2- 172.21.15.1:0/2794090114 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5609aa59b930 0x5609aa59c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:19.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.186+0000 7f07146c8b80 1 -- 172.21.15.1:0/2794090114 >> 172.21.15.1:0/2794090114 conn(0x5609aa4fadb0 msgr2=0x5609aa5b69b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:19.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.186+0000 7f07146c8b80 1 -- 172.21.15.1:0/2794090114 shutdown_connections 2024-02-02T15:31:19.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.186+0000 7f07146c8b80 1 -- 172.21.15.1:0/2794090114 wait complete. 2024-02-02T15:31:19.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.186+0000 7f07146c8b80 1 -- start start 2024-02-02T15:31:19.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.186+0000 7f07146c8b80 1 Processor -- start 2024-02-02T15:31:19.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.187+0000 7f07146c8b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5609aa59b010 0x5609aa5b14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:19.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.187+0000 7f07146c8b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5609aa59b930 0x5609aa5af7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:19.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.187+0000 7f07146c8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609aa5a67e0 0x5609aa5afcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:19.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.187+0000 7f07146c8b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609aa5b0280 con 0x5609aa59b930 2024-02-02T15:31:19.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.187+0000 7f07146c8b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609aa5b03c0 con 0x5609aa59b010 2024-02-02T15:31:19.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.187+0000 7f07146c8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609aa5b0580 con 0x5609aa5a67e0 2024-02-02T15:31:19.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.187+0000 7f07146c8b80 10 client.0 _ll_register_callbacks cb 0x5609aa48a290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:31:19.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.187+0000 7f06f9e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5609aa59b010 0x5609aa5b14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:19.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.187+0000 7f06f9e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5609aa59b010 0x5609aa5b14c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:54038/0 (socket says 172.21.15.1:54038) 2024-02-02T15:31:19.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.187+0000 7f06f9e00700 1 -- 172.21.15.1:0/3260162049 learned_addr learned my addr 172.21.15.1:0/3260162049 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:31:19.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.187+0000 7f06f9400700 1 --2- 172.21.15.1:0/3260162049 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5609aa59b930 0x5609aa5af7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:19.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.187+0000 7f06fa800700 1 --2- 172.21.15.1:0/3260162049 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609aa5a67e0 0x5609aa5afcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:19.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.188+0000 7f06f9e00700 1 -- 172.21.15.1:0/3260162049 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609aa5a67e0 msgr2=0x5609aa5afcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:19.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.188+0000 7f06f9e00700 1 --2- 172.21.15.1:0/3260162049 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609aa5a67e0 0x5609aa5afcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:19.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.188+0000 7f06f9e00700 1 -- 172.21.15.1:0/3260162049 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5609aa59b930 msgr2=0x5609aa5af7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:19.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.188+0000 7f06f9e00700 1 --2- 172.21.15.1:0/3260162049 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5609aa59b930 0x5609aa5af7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:19.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.188+0000 7f06f9e00700 1 -- 172.21.15.1:0/3260162049 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5609aa36e3b0 con 0x5609aa59b010 2024-02-02T15:31:19.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.188+0000 7f06f9e00700 1 --2- 172.21.15.1:0/3260162049 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5609aa59b010 0x5609aa5b14c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f06ec009350 tx=0x7f06ec019800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd2c8daa1b625416 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:19.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.188+0000 7f06e3400700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:31:19.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.188+0000 7f06e3400700 1 -- 172.21.15.1:0/3260162049 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06ec019cf0 con 0x5609aa59b010 2024-02-02T15:31:19.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.188+0000 7f06e3400700 1 -- 172.21.15.1:0/3260162049 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f06ec00e470 con 0x5609aa59b010 2024-02-02T15:31:19.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.188+0000 7f07146c8b80 1 -- 172.21.15.1:0/3260162049 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f06ec00ec90 con 0x5609aa59b010 2024-02-02T15:31:19.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.189+0000 7f06e3400700 1 -- 172.21.15.1:0/3260162049 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f06ec019e50 con 0x5609aa59b010 2024-02-02T15:31:19.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.189+0000 7f07146c8b80 10 client.26810 fetch_fsmap learned FSMap version 32 2024-02-02T15:31:19.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.189+0000 7f07146c8b80 1 -- 172.21.15.1:0/3260162049 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5609aa372cd0 con 0x5609aa59b010 2024-02-02T15:31:19.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.189+0000 7f06e3400700 1 -- 172.21.15.1:0/3260162049 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f06ec00ec90 con 0x5609aa59b010 2024-02-02T15:31:19.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.189+0000 7f06e3400700 1 -- 172.21.15.1:0/3260162049 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 32) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f06ec03f070 con 0x5609aa59b010 2024-02-02T15:31:19.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.189+0000 7f07146c8b80 10 client.26810 fetch_fsmap finished waiting for FSMap version 32 2024-02-02T15:31:19.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.189+0000 7f07146c8b80 10 client.26810 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-02T15:31:19.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.189+0000 7f07146c8b80 20 client.26810 populate_metadata read hostname 'smithi001' 2024-02-02T15:31:19.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.189+0000 7f07146c8b80 1 --2- 172.21.15.1:0/3260162049 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x5609aa60ceb0 0x5609aa60f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:19.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.189+0000 7f07146c8b80 4 client.26810 mds_command: new command op to 24529 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:31:19.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.189+0000 7f07146c8b80 1 -- 172.21.15.1:0/3260162049 --> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5609aa4abf50 con 0x5609aa60ceb0 2024-02-02T15:31:19.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.189+0000 7f06f9400700 1 --2- 172.21.15.1:0/3260162049 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x5609aa60ceb0 0x5609aa60f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:19.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.190+0000 7f06f9400700 1 --2- 172.21.15.1:0/3260162049 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x5609aa60ceb0 0x5609aa60f2a0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9e53e49b323e1566 server_cookie=426247557aeaa9e8 in_seq=0 out_seq=0 2024-02-02T15:31:19.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.190+0000 7f06e3400700 10 client.26810 ms_handle_connect on v2:172.21.15.142:6838/321675421 2024-02-02T15:31:19.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.190+0000 7f06e3400700 1 -- 172.21.15.1:0/3260162049 <== mds.1 v2:172.21.15.142:6838/321675421 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5609aa4abf50 con 0x5609aa60ceb0 2024-02-02T15:31:19.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.190+0000 7f06e3400700 10 client.26810 handle_command_reply: tid=0 2024-02-02T15:31:19.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.190+0000 7f07146c8b80 1 client.26810 shutdown 2024-02-02T15:31:19.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.190+0000 7f07146c8b80 1 -- 172.21.15.1:0/3260162049 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5609aa59b010 msgr2=0x5609aa5b14c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:19.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.190+0000 7f07146c8b80 1 --2- 172.21.15.1:0/3260162049 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5609aa59b010 0x5609aa5b14c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f06ec009350 tx=0x7f06ec019800 comp rx=0 tx=0).stop 2024-02-02T15:31:19.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.190+0000 7f07146c8b80 1 -- 172.21.15.1:0/3260162049 shutdown_connections 2024-02-02T15:31:19.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.190+0000 7f07146c8b80 1 --2- 172.21.15.1:0/3260162049 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x5609aa60ceb0 0x5609aa60f2a0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:19.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.190+0000 7f07146c8b80 1 --2- 172.21.15.1:0/3260162049 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609aa5a67e0 0x5609aa5afcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:19.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.190+0000 7f07146c8b80 1 --2- 172.21.15.1:0/3260162049 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5609aa59b010 0x5609aa5b14c0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:19.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.190+0000 7f07146c8b80 1 --2- 172.21.15.1:0/3260162049 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5609aa59b930 0x5609aa5af7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:19.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.190+0000 7f06e3400700 0 client.26810 ms_handle_reset on v2:172.21.15.142:6838/321675421 2024-02-02T15:31:19.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.190+0000 7f07146c8b80 1 -- 172.21.15.1:0/3260162049 >> 172.21.15.1:0/3260162049 conn(0x5609aa4fadb0 msgr2=0x5609aa5b5d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:19.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.190+0000 7f07146c8b80 1 -- 172.21.15.1:0/3260162049 shutdown_connections 2024-02-02T15:31:19.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.191+0000 7f07146c8b80 1 -- 172.21.15.1:0/3260162049 wait complete. 2024-02-02T15:31:19.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.191+0000 7f07146c8b80 20 client.26810 trim_cache size 0 max 16384 2024-02-02T15:31:19.197 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:31:19.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.191+0000 7f0700c00700 1 -- 172.21.15.1:0/3898539415 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f06dc0a0690 msgr2=0x7f06dc0a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:19.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.191+0000 7f0700c00700 1 --2- 172.21.15.1:0/3898539415 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f06dc0a0690 0x7f06dc0a2b50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f06f4009d30 tx=0x7f06f4009920 comp rx=0 tx=0).stop 2024-02-02T15:31:19.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.191+0000 7f0700c00700 1 -- 172.21.15.1:0/3898539415 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f06fc106110 msgr2=0x7f06fc197530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:19.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.191+0000 7f0700c00700 1 --2- 172.21.15.1:0/3898539415 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f06fc106110 0x7f06fc197530 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f06e4009230 tx=0x7f06e400f9e0 comp rx=0 tx=0).stop 2024-02-02T15:31:19.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.192+0000 7f0700c00700 1 -- 172.21.15.1:0/3898539415 shutdown_connections 2024-02-02T15:31:19.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.192+0000 7f0700c00700 1 --2- 172.21.15.1:0/3898539415 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f06dc0a0690 0x7f06dc0a2b50 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:19.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.192+0000 7f0700c00700 1 --2- 172.21.15.1:0/3898539415 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f06fc19b8f0 0x7f06fc19bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:19.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.192+0000 7f0700c00700 1 --2- 172.21.15.1:0/3898539415 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f06fc100ff0 0x7f06fc196ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:19.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.192+0000 7f0700c00700 1 --2- 172.21.15.1:0/3898539415 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f06fc106110 0x7f06fc197530 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:19.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.192+0000 7f0700c00700 1 -- 172.21.15.1:0/3898539415 >> 172.21.15.1:0/3898539415 conn(0x7f06fc0fc810 msgr2=0x7f06fc0fd330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:19.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.192+0000 7f0700c00700 1 -- 172.21.15.1:0/3898539415 shutdown_connections 2024-02-02T15:31:19.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.192+0000 7f0700c00700 1 -- 172.21.15.1:0/3898539415 wait complete. 2024-02-02T15:31:19.210 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-02T15:31:19.230 INFO:tasks.workunit.client.0.smithi001.stdout: 1 232213 9.34 MB/sec execute 53 sec latency 16.694 ms 2024-02-02T15:31:19.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.428+0000 7f5bea400700 1 -- 172.21.15.1:0/1500265168 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5be4100ff0 msgr2=0x7f5be41013d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:19.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.428+0000 7f5bea400700 1 --2- 172.21.15.1:0/1500265168 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5be4100ff0 0x7f5be41013d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5bd40098f0 tx=0x7f5bd4009c00 comp rx=0 tx=0).stop 2024-02-02T15:31:19.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.429+0000 7f5bea400700 1 -- 172.21.15.1:0/1500265168 shutdown_connections 2024-02-02T15:31:19.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.429+0000 7f5bea400700 1 --2- 172.21.15.1:0/1500265168 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5be4106110 0x7f5be410dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:19.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.429+0000 7f5bea400700 1 --2- 172.21.15.1:0/1500265168 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5be4100ff0 0x7f5be41013d0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:19.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.429+0000 7f5bea400700 1 --2- 172.21.15.1:0/1500265168 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5be41019a0 0x7f5be41059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:19.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.429+0000 7f5bea400700 1 -- 172.21.15.1:0/1500265168 >> 172.21.15.1:0/1500265168 conn(0x7f5be40fc810 msgr2=0x7f5be40fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:19.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.429+0000 7f5bea400700 1 -- 172.21.15.1:0/1500265168 shutdown_connections 2024-02-02T15:31:19.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.429+0000 7f5bea400700 1 -- 172.21.15.1:0/1500265168 wait complete. 2024-02-02T15:31:19.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.430+0000 7f5bea400700 1 Processor -- start 2024-02-02T15:31:19.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.430+0000 7f5bea400700 1 -- start start 2024-02-02T15:31:19.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.430+0000 7f5bea400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5be4100ff0 0x7f5be4199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:19.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.431+0000 7f5bea400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5be4106110 0x7f5be4199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:19.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.431+0000 7f5bea400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5be419dac0 0x7f5be419df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:19.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.431+0000 7f5bea400700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5be419e540 con 0x7f5be4100ff0 2024-02-02T15:31:19.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.431+0000 7f5bea400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5be419e6b0 con 0x7f5be4106110 2024-02-02T15:31:19.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.431+0000 7f5bea400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5be419e870 con 0x7f5be419dac0 2024-02-02T15:31:19.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.431+0000 7f5be3400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5be4100ff0 0x7f5be4199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:19.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.431+0000 7f5be3e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5be419dac0 0x7f5be419df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:19.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.431+0000 7f5be2a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5be4106110 0x7f5be4199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:19.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.431+0000 7f5be3e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5be419dac0 0x7f5be419df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:56304/0 (socket says 172.21.15.1:56304) 2024-02-02T15:31:19.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.431+0000 7f5be3400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5be4100ff0 0x7f5be4199120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55002/0 (socket says 172.21.15.1:55002) 2024-02-02T15:31:19.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.431+0000 7f5be3e00700 1 -- 172.21.15.1:0/2602091504 learned_addr learned my addr 172.21.15.1:0/2602091504 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:31:19.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.431+0000 7f5be3e00700 1 -- 172.21.15.1:0/2602091504 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5be4106110 msgr2=0x7f5be4199660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:19.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.431+0000 7f5be3e00700 1 --2- 172.21.15.1:0/2602091504 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5be4106110 0x7f5be4199660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:19.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.431+0000 7f5be3e00700 1 -- 172.21.15.1:0/2602091504 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5be4100ff0 msgr2=0x7f5be4199120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:19.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.431+0000 7f5be3e00700 1 --2- 172.21.15.1:0/2602091504 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5be4100ff0 0x7f5be4199120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:19.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.431+0000 7f5be3e00700 1 -- 172.21.15.1:0/2602091504 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bd4009520 con 0x7f5be419dac0 2024-02-02T15:31:19.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.431+0000 7f5be2a00700 1 --2- 172.21.15.1:0/2602091504 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5be4106110 0x7f5be4199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:31:19.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.431+0000 7f5be3400700 1 --2- 172.21.15.1:0/2602091504 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5be4100ff0 0x7f5be4199120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:31:19.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.432+0000 7f5be3e00700 1 --2- 172.21.15.1:0/2602091504 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5be419dac0 0x7f5be419df40 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc00c110 tx=0x7f5bdc00c420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:19.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.432+0000 7f5bdbe00700 1 -- 172.21.15.1:0/2602091504 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bdc017070 con 0x7f5be419dac0 2024-02-02T15:31:19.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.432+0000 7f5bdbe00700 1 -- 172.21.15.1:0/2602091504 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5bdc00e050 con 0x7f5be419dac0 2024-02-02T15:31:19.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.432+0000 7f5bdbe00700 1 -- 172.21.15.1:0/2602091504 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bdc0078f0 con 0x7f5be419dac0 2024-02-02T15:31:19.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.432+0000 7f5bea400700 1 -- 172.21.15.1:0/2602091504 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5be41a5d70 con 0x7f5be419dac0 2024-02-02T15:31:19.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.432+0000 7f5bea400700 1 -- 172.21.15.1:0/2602091504 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5be41a6200 con 0x7f5be419dac0 2024-02-02T15:31:19.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.437+0000 7f5bdbe00700 1 -- 172.21.15.1:0/2602091504 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 27) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f5bdc007a50 con 0x7f5be419dac0 2024-02-02T15:31:19.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.438+0000 7f5bfdf56b80 1 Processor -- start 2024-02-02T15:31:19.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.438+0000 7f5bfdf56b80 1 -- start start 2024-02-02T15:31:19.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.438+0000 7f5bfdf56b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b323592010 0x55b3235923f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:19.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.438+0000 7f5bfdf56b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b323592930 0x55b32359d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:19.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.438+0000 7f5bfdf56b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b32359d7e0 0x55b32359fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:19.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.438+0000 7f5bfdf56b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3235064c0 con 0x55b323592010 2024-02-02T15:31:19.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.438+0000 7f5bfdf56b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3234eaaa0 con 0x55b323592930 2024-02-02T15:31:19.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.438+0000 7f5bfdf56b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3234e8460 con 0x55b32359d7e0 2024-02-02T15:31:19.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.439+0000 7f5be2a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b323592010 0x55b3235923f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:19.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.439+0000 7f5be3400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b323592930 0x55b32359d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:19.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.439+0000 7f5be3400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b323592930 0x55b32359d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:54066/0 (socket says 172.21.15.1:54066) 2024-02-02T15:31:19.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.439+0000 7f5be2a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b323592010 0x55b3235923f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55008/0 (socket says 172.21.15.1:55008) 2024-02-02T15:31:19.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.439+0000 7f5be3400700 1 -- 172.21.15.1:0/2868373570 learned_addr learned my addr 172.21.15.1:0/2868373570 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:31:19.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.439+0000 7f5be2a00700 1 --2- 172.21.15.1:0/2868373570 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b32359d7e0 0x55b32359fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:19.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.439+0000 7f5be3400700 1 -- 172.21.15.1:0/2868373570 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b32359d7e0 msgr2=0x55b32359fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:19.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.439+0000 7f5be3400700 1 --2- 172.21.15.1:0/2868373570 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b32359d7e0 0x55b32359fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:19.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.439+0000 7f5be3400700 1 -- 172.21.15.1:0/2868373570 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b323592010 msgr2=0x55b3235923f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:19.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.439+0000 7f5be3400700 1 --2- 172.21.15.1:0/2868373570 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b323592010 0x55b3235923f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:19.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.439+0000 7f5be3400700 1 -- 172.21.15.1:0/2868373570 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b3233653b0 con 0x55b323592930 2024-02-02T15:31:19.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.439+0000 7f5bdbe00700 1 --2- 172.21.15.1:0/2602091504 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f5bc40a0790 0x7f5bc40a2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:19.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.440+0000 7f5bdbe00700 1 -- 172.21.15.1:0/2602091504 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f5bdc0cba20 con 0x7f5be419dac0 2024-02-02T15:31:19.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.440+0000 7f5be3400700 1 --2- 172.21.15.1:0/2868373570 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b323592930 0x55b32359d2a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f5bd40098f0 tx=0x7f5bd4005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93644cdfee573b7b server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:19.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.440+0000 7f5be2a00700 1 --2- 172.21.15.1:0/2602091504 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f5bc40a0790 0x7f5bc40a2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:19.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.440+0000 7f5bd8c00700 1 -- 172.21.15.1:0/2868373570 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd401c050 con 0x55b323592930 2024-02-02T15:31:19.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.440+0000 7f5bd8c00700 1 -- 172.21.15.1:0/2868373570 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5bd401d040 con 0x55b323592930 2024-02-02T15:31:19.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.440+0000 7f5bd8c00700 1 -- 172.21.15.1:0/2868373570 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd40052a0 con 0x55b323592930 2024-02-02T15:31:19.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.441+0000 7f5bfdf56b80 1 -- 172.21.15.1:0/2868373570 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b323592930 msgr2=0x55b32359d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:19.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.441+0000 7f5be2a00700 1 --2- 172.21.15.1:0/2602091504 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f5bc40a0790 0x7f5bc40a2c50 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc005950 tx=0x7f5bcc015040 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:19.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.441+0000 7f5bfdf56b80 1 --2- 172.21.15.1:0/2868373570 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b323592930 0x55b32359d2a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f5bd40098f0 tx=0x7f5bd4005dc0 comp rx=0 tx=0).stop 2024-02-02T15:31:19.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.441+0000 7f5bfdf56b80 1 -- 172.21.15.1:0/2868373570 shutdown_connections 2024-02-02T15:31:19.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.441+0000 7f5bfdf56b80 1 --2- 172.21.15.1:0/2868373570 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b32359d7e0 0x55b32359fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:19.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.441+0000 7f5bfdf56b80 1 --2- 172.21.15.1:0/2868373570 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b323592010 0x55b3235923f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:19.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.441+0000 7f5bfdf56b80 1 --2- 172.21.15.1:0/2868373570 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b323592930 0x55b32359d2a0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:19.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.441+0000 7f5bfdf56b80 1 -- 172.21.15.1:0/2868373570 >> 172.21.15.1:0/2868373570 conn(0x55b3234f1db0 msgr2=0x55b32358e3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:19.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.441+0000 7f5bfdf56b80 1 -- 172.21.15.1:0/2868373570 shutdown_connections 2024-02-02T15:31:19.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.441+0000 7f5bfdf56b80 1 -- 172.21.15.1:0/2868373570 wait complete. 2024-02-02T15:31:19.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.442+0000 7f5bfdf56b80 1 -- start start 2024-02-02T15:31:19.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.442+0000 7f5bfdf56b80 1 Processor -- start 2024-02-02T15:31:19.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.442+0000 7f5bfdf56b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b323592010 0x55b32359d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:19.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.442+0000 7f5bfdf56b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b323592930 0x55b32359a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:19.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.442+0000 7f5bfdf56b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b32359d7e0 0x55b32359aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:19.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.442+0000 7f5bfdf56b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3234e3d50 con 0x55b32359d7e0 2024-02-02T15:31:19.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.443+0000 7f5bfdf56b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b32347fd80 con 0x55b323592930 2024-02-02T15:31:19.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.443+0000 7f5bfdf56b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b32347fa20 con 0x55b323592010 2024-02-02T15:31:19.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.443+0000 7f5bfdf56b80 10 client.0 _ll_register_callbacks cb 0x55b323481290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:31:19.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.443+0000 7f5be3400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b323592010 0x55b32359d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:19.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.443+0000 7f5be3400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b323592010 0x55b32359d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:56326/0 (socket says 172.21.15.1:56326) 2024-02-02T15:31:19.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.443+0000 7f5be3400700 1 -- 172.21.15.1:0/432691896 learned_addr learned my addr 172.21.15.1:0/432691896 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:31:19.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.443+0000 7f5be2a00700 1 --2- 172.21.15.1:0/432691896 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b323592930 0x55b32359a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:19.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.443+0000 7f5be3e00700 1 --2- 172.21.15.1:0/432691896 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b32359d7e0 0x55b32359aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:19.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.443+0000 7f5be3400700 1 -- 172.21.15.1:0/432691896 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b323592930 msgr2=0x55b32359a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:19.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.443+0000 7f5be3400700 1 --2- 172.21.15.1:0/432691896 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b323592930 0x55b32359a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:19.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.443+0000 7f5be3400700 1 -- 172.21.15.1:0/432691896 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b32359d7e0 msgr2=0x55b32359aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:19.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.443+0000 7f5be3400700 1 --2- 172.21.15.1:0/432691896 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b32359d7e0 0x55b32359aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:19.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.443+0000 7f5be3400700 1 -- 172.21.15.1:0/432691896 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5be410e500 con 0x55b323592010 2024-02-02T15:31:19.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.443+0000 7f5be2a00700 1 --2- 172.21.15.1:0/432691896 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b323592930 0x55b32359a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:31:19.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.443+0000 7f5be3e00700 1 --2- 172.21.15.1:0/432691896 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b32359d7e0 0x55b32359aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:31:19.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.443+0000 7f5be3400700 1 --2- 172.21.15.1:0/432691896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b323592010 0x55b32359d1a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5bd4005e90 tx=0x7f5bd400b730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9cc8e9d0a1bbf5c8 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:19.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.443+0000 7f5bd1600700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:31:19.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.444+0000 7f5bd1600700 1 -- 172.21.15.1:0/432691896 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd40052c0 con 0x55b323592010 2024-02-02T15:31:19.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.444+0000 7f5bd1600700 1 -- 172.21.15.1:0/432691896 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f5bd400bb30 con 0x55b323592010 2024-02-02T15:31:19.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.444+0000 7f5bfdf56b80 1 -- 172.21.15.1:0/432691896 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b3235860f0 con 0x55b323592010 2024-02-02T15:31:19.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.444+0000 7f5bd1600700 1 -- 172.21.15.1:0/432691896 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5bd400f8e0 con 0x55b323592010 2024-02-02T15:31:19.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.444+0000 7f5bfdf56b80 10 client.26815 fetch_fsmap learned FSMap version 32 2024-02-02T15:31:19.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.444+0000 7f5bfdf56b80 1 -- 172.21.15.1:0/432691896 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55b323311b60 con 0x55b323592010 2024-02-02T15:31:19.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.444+0000 7f5bd1600700 1 -- 172.21.15.1:0/432691896 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b3235860f0 con 0x55b323592010 2024-02-02T15:31:19.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.445+0000 7f5bd1600700 1 -- 172.21.15.1:0/432691896 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 32) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f5bd4015020 con 0x55b323592010 2024-02-02T15:31:19.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.445+0000 7f5bfdf56b80 10 client.26815 fetch_fsmap finished waiting for FSMap version 32 2024-02-02T15:31:19.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.445+0000 7f5bfdf56b80 10 client.26815 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:31:19.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.445+0000 7f5bfdf56b80 20 client.26815 populate_metadata read hostname 'smithi001' 2024-02-02T15:31:19.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.445+0000 7f5bfdf56b80 1 --2- 172.21.15.1:0/432691896 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55b3235acd90 0x55b3235af1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:19.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.445+0000 7f5bfdf56b80 4 client.26815 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:31:19.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.445+0000 7f5bfdf56b80 1 -- 172.21.15.1:0/432691896 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b3234dcdd0 con 0x55b3235acd90 2024-02-02T15:31:19.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.445+0000 7f5be2a00700 1 --2- 172.21.15.1:0/432691896 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55b3235acd90 0x55b3235af1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:19.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.446+0000 7f5be2a00700 1 --2- 172.21.15.1:0/432691896 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55b3235acd90 0x55b3235af1b0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7d601f946e25a72 server_cookie=6f9be5658974fa91 in_seq=0 out_seq=0 2024-02-02T15:31:19.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.446+0000 7f5bd1600700 10 client.26815 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:31:19.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.447+0000 7f5bd1600700 1 -- 172.21.15.1:0/432691896 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55b3234dcdd0 con 0x55b3235acd90 2024-02-02T15:31:19.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.447+0000 7f5bd1600700 10 client.26815 handle_command_reply: tid=0 2024-02-02T15:31:19.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.447+0000 7f5bfdf56b80 1 client.26815 shutdown 2024-02-02T15:31:19.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.447+0000 7f5bfdf56b80 1 -- 172.21.15.1:0/432691896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b323592010 msgr2=0x55b32359d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:19.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.447+0000 7f5bfdf56b80 1 --2- 172.21.15.1:0/432691896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b323592010 0x55b32359d1a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5bd4005e90 tx=0x7f5bd400b730 comp rx=0 tx=0).stop 2024-02-02T15:31:19.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.447+0000 7f5bfdf56b80 1 -- 172.21.15.1:0/432691896 shutdown_connections 2024-02-02T15:31:19.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.447+0000 7f5bfdf56b80 1 --2- 172.21.15.1:0/432691896 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55b3235acd90 0x55b3235af1b0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:19.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.447+0000 7f5bfdf56b80 1 --2- 172.21.15.1:0/432691896 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b32359d7e0 0x55b32359aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:19.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.447+0000 7f5bd1600700 0 client.26815 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:31:19.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.447+0000 7f5bfdf56b80 1 --2- 172.21.15.1:0/432691896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b323592010 0x55b32359d1a0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:19.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.447+0000 7f5bfdf56b80 1 --2- 172.21.15.1:0/432691896 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b323592930 0x55b32359a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:19.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.447+0000 7f5bfdf56b80 1 -- 172.21.15.1:0/432691896 >> 172.21.15.1:0/432691896 conn(0x55b3234f1db0 msgr2=0x55b32358dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:19.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.447+0000 7f5bfdf56b80 1 -- 172.21.15.1:0/432691896 shutdown_connections 2024-02-02T15:31:19.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.447+0000 7f5bfdf56b80 1 -- 172.21.15.1:0/432691896 wait complete. 2024-02-02T15:31:19.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.448+0000 7f5bfdf56b80 20 client.26815 trim_cache size 0 max 16384 2024-02-02T15:31:19.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.462+0000 7f5bfdf56b80 1 Processor -- start 2024-02-02T15:31:19.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.462+0000 7f5bfdf56b80 1 -- start start 2024-02-02T15:31:19.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.462+0000 7f5bfdf56b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b3234f1db0 0x55b3235a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:19.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.462+0000 7f5bfdf56b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b323592010 0x55b32359ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:19.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.462+0000 7f5bfdf56b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b323592930 0x55b3235a88a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:19.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.462+0000 7f5bfdf56b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3233676a0 con 0x55b323592010 2024-02-02T15:31:19.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.462+0000 7f5bfdf56b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b32359b0a0 con 0x55b3234f1db0 2024-02-02T15:31:19.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.462+0000 7f5bfdf56b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3235989b0 con 0x55b323592930 2024-02-02T15:31:19.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.462+0000 7f5be3400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b3234f1db0 0x55b3235a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:19.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.463+0000 7f5be3400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b3234f1db0 0x55b3235a42c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:54078/0 (socket says 172.21.15.1:54078) 2024-02-02T15:31:19.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.463+0000 7f5be3400700 1 -- 172.21.15.1:0/1477064838 learned_addr learned my addr 172.21.15.1:0/1477064838 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:31:19.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.463+0000 7f5be2a00700 1 --2- 172.21.15.1:0/1477064838 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b323592010 0x55b32359ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:19.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.463+0000 7f5be3e00700 1 --2- 172.21.15.1:0/1477064838 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b323592930 0x55b3235a88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:19.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.463+0000 7f5be3400700 1 -- 172.21.15.1:0/1477064838 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b323592930 msgr2=0x55b3235a88a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:19.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.463+0000 7f5be3400700 1 --2- 172.21.15.1:0/1477064838 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b323592930 0x55b3235a88a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:19.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.463+0000 7f5be3400700 1 -- 172.21.15.1:0/1477064838 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b323592010 msgr2=0x55b32359ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:19.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.463+0000 7f5be3400700 1 --2- 172.21.15.1:0/1477064838 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b323592010 0x55b32359ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:19.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.463+0000 7f5be3400700 1 -- 172.21.15.1:0/1477064838 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b3234a9ea0 con 0x55b3234f1db0 2024-02-02T15:31:19.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.463+0000 7f5be3400700 1 --2- 172.21.15.1:0/1477064838 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b3234f1db0 0x55b3235a42c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f5bd4009c40 tx=0x7f5bd402c570 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f64fb8f5febd9e5 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:19.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.463+0000 7f5bd9600700 1 -- 172.21.15.1:0/1477064838 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd401c070 con 0x55b3234f1db0 2024-02-02T15:31:19.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.463+0000 7f5bd9600700 1 -- 172.21.15.1:0/1477064838 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5bd400b7a0 con 0x55b3234f1db0 2024-02-02T15:31:19.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.463+0000 7f5bd9600700 1 -- 172.21.15.1:0/1477064838 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd402c900 con 0x55b3234f1db0 2024-02-02T15:31:19.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.464+0000 7f5bfdf56b80 1 -- 172.21.15.1:0/1477064838 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b3234f1db0 msgr2=0x55b3235a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:19.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.464+0000 7f5bfdf56b80 1 --2- 172.21.15.1:0/1477064838 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b3234f1db0 0x55b3235a42c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f5bd4009c40 tx=0x7f5bd402c570 comp rx=0 tx=0).stop 2024-02-02T15:31:19.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.464+0000 7f5bfdf56b80 1 -- 172.21.15.1:0/1477064838 shutdown_connections 2024-02-02T15:31:19.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.464+0000 7f5bfdf56b80 1 --2- 172.21.15.1:0/1477064838 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b323592930 0x55b3235a88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:19.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.464+0000 7f5bfdf56b80 1 --2- 172.21.15.1:0/1477064838 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b3234f1db0 0x55b3235a42c0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:19.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.464+0000 7f5bfdf56b80 1 --2- 172.21.15.1:0/1477064838 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b323592010 0x55b32359ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:19.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.464+0000 7f5bfdf56b80 1 -- 172.21.15.1:0/1477064838 >> 172.21.15.1:0/1477064838 conn(0x55b32359fd90 msgr2=0x55b32358cb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:19.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.464+0000 7f5bfdf56b80 1 -- 172.21.15.1:0/1477064838 shutdown_connections 2024-02-02T15:31:19.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.464+0000 7f5bfdf56b80 1 -- 172.21.15.1:0/1477064838 wait complete. 2024-02-02T15:31:19.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.465+0000 7f5bfdf56b80 1 -- start start 2024-02-02T15:31:19.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.465+0000 7f5bfdf56b80 1 Processor -- start 2024-02-02T15:31:19.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.466+0000 7f5bfdf56b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b3234f1db0 0x55b3235a02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:19.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.466+0000 7f5bfdf56b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b323592010 0x55b3235a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:19.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.466+0000 7f5bfdf56b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b323592930 0x55b3235a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:19.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.466+0000 7f5bfdf56b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3235a46d0 con 0x55b323592930 2024-02-02T15:31:19.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.466+0000 7f5bfdf56b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3235fd890 con 0x55b3234f1db0 2024-02-02T15:31:19.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.466+0000 7f5bfdf56b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3235fda50 con 0x55b323592010 2024-02-02T15:31:19.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.466+0000 7f5bfdf56b80 10 client.0 _ll_register_callbacks cb 0x55b323481290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:31:19.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.466+0000 7f5be3e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b323592930 0x55b3235a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:19.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.466+0000 7f5be3e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b323592930 0x55b3235a0d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55036/0 (socket says 172.21.15.1:55036) 2024-02-02T15:31:19.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.466+0000 7f5be3400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b3234f1db0 0x55b3235a02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:19.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.466+0000 7f5be3e00700 1 -- 172.21.15.1:0/771963811 learned_addr learned my addr 172.21.15.1:0/771963811 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:31:19.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.466+0000 7f5be2a00700 1 --2- 172.21.15.1:0/771963811 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b323592010 0x55b3235a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:19.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.467+0000 7f5be3e00700 1 -- 172.21.15.1:0/771963811 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b323592010 msgr2=0x55b3235a07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:19.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.467+0000 7f5be3e00700 1 --2- 172.21.15.1:0/771963811 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b323592010 0x55b3235a07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:19.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.467+0000 7f5be3e00700 1 -- 172.21.15.1:0/771963811 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b3234f1db0 msgr2=0x55b3235a02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:19.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.467+0000 7f5be3e00700 1 --2- 172.21.15.1:0/771963811 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b3234f1db0 0x55b3235a02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:19.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.467+0000 7f5be3e00700 1 -- 172.21.15.1:0/771963811 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b323311b60 con 0x55b323592930 2024-02-02T15:31:19.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.467+0000 7f5be3e00700 1 --2- 172.21.15.1:0/771963811 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b323592930 0x55b3235a0d20 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc095090 tx=0x7f5bdc0094d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6800ad1a69013a1 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:19.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.467+0000 7f5bd1600700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:31:19.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.467+0000 7f5bd1600700 1 -- 172.21.15.1:0/771963811 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bdc0c9240 con 0x55b323592930 2024-02-02T15:31:19.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.467+0000 7f5bd1600700 1 -- 172.21.15.1:0/771963811 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f5bdc09c9c0 con 0x55b323592930 2024-02-02T15:31:19.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.467+0000 7f5bfdf56b80 1 -- 172.21.15.1:0/771963811 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b3234a2f50 con 0x55b323592930 2024-02-02T15:31:19.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.467+0000 7f5bd1600700 1 -- 172.21.15.1:0/771963811 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5bdc0a0c00 con 0x55b323592930 2024-02-02T15:31:19.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.467+0000 7f5bfdf56b80 10 client.16767 fetch_fsmap learned FSMap version 32 2024-02-02T15:31:19.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.467+0000 7f5bfdf56b80 1 -- 172.21.15.1:0/771963811 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55b323369cd0 con 0x55b323592930 2024-02-02T15:31:19.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.468+0000 7f5bd1600700 1 -- 172.21.15.1:0/771963811 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b3234a2f50 con 0x55b323592930 2024-02-02T15:31:19.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.468+0000 7f5bd1600700 1 -- 172.21.15.1:0/771963811 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 32) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f5bdc098080 con 0x55b323592930 2024-02-02T15:31:19.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.468+0000 7f5bfdf56b80 10 client.16767 fetch_fsmap finished waiting for FSMap version 32 2024-02-02T15:31:19.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.468+0000 7f5bfdf56b80 10 client.16767 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:31:19.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.468+0000 7f5bfdf56b80 20 client.16767 populate_metadata read hostname 'smithi001' 2024-02-02T15:31:19.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.468+0000 7f5bfdf56b80 1 --2- 172.21.15.1:0/771963811 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55b32359d7e0 0x55b3236012a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:19.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.468+0000 7f5bfdf56b80 4 client.16767 mds_command: new command op to 24557 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:31:19.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.468+0000 7f5bfdf56b80 1 -- 172.21.15.1:0/771963811 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b323332cf0 con 0x55b32359d7e0 2024-02-02T15:31:19.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.468+0000 7f5be3400700 1 --2- 172.21.15.1:0/771963811 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55b32359d7e0 0x55b3236012a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:19.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.469+0000 7f5be3400700 1 --2- 172.21.15.1:0/771963811 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55b32359d7e0 0x55b3236012a0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82e136a514128c7d server_cookie=6286025c8df3b18b in_seq=0 out_seq=0 2024-02-02T15:31:19.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.469+0000 7f5bd1600700 10 client.16767 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:31:19.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.469+0000 7f5bd1600700 1 -- 172.21.15.1:0/771963811 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b323332cf0 con 0x55b32359d7e0 2024-02-02T15:31:19.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.469+0000 7f5bd1600700 10 client.16767 handle_command_reply: tid=0 2024-02-02T15:31:19.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.469+0000 7f5bfdf56b80 1 client.16767 shutdown 2024-02-02T15:31:19.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.469+0000 7f5bfdf56b80 1 -- 172.21.15.1:0/771963811 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b323592930 msgr2=0x55b3235a0d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:19.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.469+0000 7f5bfdf56b80 1 --2- 172.21.15.1:0/771963811 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b323592930 0x55b3235a0d20 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc095090 tx=0x7f5bdc0094d0 comp rx=0 tx=0).stop 2024-02-02T15:31:19.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.469+0000 7f5bfdf56b80 1 -- 172.21.15.1:0/771963811 shutdown_connections 2024-02-02T15:31:19.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.469+0000 7f5bfdf56b80 1 --2- 172.21.15.1:0/771963811 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55b32359d7e0 0x55b3236012a0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:19.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.469+0000 7f5bfdf56b80 1 --2- 172.21.15.1:0/771963811 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b323592930 0x55b3235a0d20 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:19.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.469+0000 7f5bfdf56b80 1 --2- 172.21.15.1:0/771963811 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b3234f1db0 0x55b3235a02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:19.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.469+0000 7f5bfdf56b80 1 --2- 172.21.15.1:0/771963811 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b323592010 0x55b3235a07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:19.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.469+0000 7f5bd1600700 0 client.16767 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:31:19.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.469+0000 7f5bfdf56b80 1 -- 172.21.15.1:0/771963811 >> 172.21.15.1:0/771963811 conn(0x55b32359fd90 msgr2=0x55b3235a88a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:19.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.469+0000 7f5bfdf56b80 1 -- 172.21.15.1:0/771963811 shutdown_connections 2024-02-02T15:31:19.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.469+0000 7f5bfdf56b80 1 -- 172.21.15.1:0/771963811 wait complete. 2024-02-02T15:31:19.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.470+0000 7f5bfdf56b80 20 client.16767 trim_cache size 0 max 16384 2024-02-02T15:31:19.473 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:31:19.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.470+0000 7f5bea400700 1 -- 172.21.15.1:0/2602091504 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f5bc40a0790 msgr2=0x7f5bc40a2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:19.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.470+0000 7f5bea400700 1 --2- 172.21.15.1:0/2602091504 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f5bc40a0790 0x7f5bc40a2c50 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc005950 tx=0x7f5bcc015040 comp rx=0 tx=0).stop 2024-02-02T15:31:19.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.470+0000 7f5bea400700 1 -- 172.21.15.1:0/2602091504 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5be419dac0 msgr2=0x7f5be419df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:19.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.470+0000 7f5bea400700 1 --2- 172.21.15.1:0/2602091504 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5be419dac0 0x7f5be419df40 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc00c110 tx=0x7f5bdc00c420 comp rx=0 tx=0).stop 2024-02-02T15:31:19.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.470+0000 7f5bea400700 1 -- 172.21.15.1:0/2602091504 shutdown_connections 2024-02-02T15:31:19.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.470+0000 7f5bea400700 1 --2- 172.21.15.1:0/2602091504 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f5bc40a0790 0x7f5bc40a2c50 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:19.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.470+0000 7f5bea400700 1 --2- 172.21.15.1:0/2602091504 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5be419dac0 0x7f5be419df40 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:19.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.470+0000 7f5bea400700 1 --2- 172.21.15.1:0/2602091504 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5be4100ff0 0x7f5be4199120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:19.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.470+0000 7f5bea400700 1 --2- 172.21.15.1:0/2602091504 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5be4106110 0x7f5be4199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:19.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.470+0000 7f5bea400700 1 -- 172.21.15.1:0/2602091504 >> 172.21.15.1:0/2602091504 conn(0x7f5be40fc810 msgr2=0x7f5be40fd690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:19.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.470+0000 7f5bea400700 1 -- 172.21.15.1:0/2602091504 shutdown_connections 2024-02-02T15:31:19.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:19.471+0000 7f5bea400700 1 -- 172.21.15.1:0/2602091504 wait complete. 2024-02-02T15:31:19.487 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:31:19 smithi001 ceph-mon[35470]: pgmap v132: 129 pgs: 129 active+clean; 144 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 227 op/s 2024-02-02T15:31:19.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:31:19 smithi142 ceph-mon[45728]: pgmap v132: 129 pgs: 129 active+clean; 144 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 227 op/s 2024-02-02T15:31:19.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:31:19 smithi071 ceph-mon[44707]: pgmap v132: 129 pgs: 129 active+clean; 144 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 227 op/s 2024-02-02T15:31:20.231 INFO:tasks.workunit.client.0.smithi001.stdout: 1 234396 9.42 MB/sec execute 54 sec latency 11.384 ms 2024-02-02T15:31:20.491 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-02T15:31:20.491 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-02T15:31:20.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.705+0000 7ff523c00700 1 -- 172.21.15.1:0/3213933264 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff51c100ff0 msgr2=0x7ff51c1013d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:20.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.705+0000 7ff523c00700 1 --2- 172.21.15.1:0/3213933264 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff51c100ff0 0x7ff51c1013d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff50c009990 tx=0x7ff50c009ca0 comp rx=0 tx=0).stop 2024-02-02T15:31:20.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.706+0000 7ff523c00700 1 -- 172.21.15.1:0/3213933264 shutdown_connections 2024-02-02T15:31:20.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.706+0000 7ff523c00700 1 --2- 172.21.15.1:0/3213933264 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff51c106110 0x7ff51c10dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:20.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.706+0000 7ff523c00700 1 --2- 172.21.15.1:0/3213933264 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff51c100ff0 0x7ff51c1013d0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:20.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.706+0000 7ff523c00700 1 --2- 172.21.15.1:0/3213933264 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff51c1019a0 0x7ff51c1059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:20.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.706+0000 7ff523c00700 1 -- 172.21.15.1:0/3213933264 >> 172.21.15.1:0/3213933264 conn(0x7ff51c0fc810 msgr2=0x7ff51c0fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:20.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.706+0000 7ff523c00700 1 -- 172.21.15.1:0/3213933264 shutdown_connections 2024-02-02T15:31:20.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.706+0000 7ff523c00700 1 -- 172.21.15.1:0/3213933264 wait complete. 2024-02-02T15:31:20.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.708+0000 7ff523c00700 1 Processor -- start 2024-02-02T15:31:20.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.710+0000 7ff523c00700 1 -- start start 2024-02-02T15:31:20.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.711+0000 7ff523c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff51c100ff0 0x7ff51c196ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:20.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.711+0000 7ff523c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff51c106110 0x7ff51c197410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:20.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.711+0000 7ff523c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff51c19b870 0x7ff51c19bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:20.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.711+0000 7ff523c00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff51c19c2f0 con 0x7ff51c19b870 2024-02-02T15:31:20.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.711+0000 7ff523c00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff51c19c460 con 0x7ff51c106110 2024-02-02T15:31:20.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.711+0000 7ff523c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff51c19c620 con 0x7ff51c100ff0 2024-02-02T15:31:20.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.711+0000 7ff521200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff51c100ff0 0x7ff51c196ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:20.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.711+0000 7ff521200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff51c100ff0 0x7ff51c196ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:56354/0 (socket says 172.21.15.1:56354) 2024-02-02T15:31:20.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.712+0000 7ff521200700 1 -- 172.21.15.1:0/4084517268 learned_addr learned my addr 172.21.15.1:0/4084517268 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:31:20.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.712+0000 7ff521c00700 1 --2- 172.21.15.1:0/4084517268 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff51c19b870 0x7ff51c19bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:20.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.712+0000 7ff51be00700 1 --2- 172.21.15.1:0/4084517268 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff51c106110 0x7ff51c197410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:20.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.712+0000 7ff521200700 1 -- 172.21.15.1:0/4084517268 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff51c106110 msgr2=0x7ff51c197410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:20.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.713+0000 7ff521200700 1 --2- 172.21.15.1:0/4084517268 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff51c106110 0x7ff51c197410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:20.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.713+0000 7ff521200700 1 -- 172.21.15.1:0/4084517268 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff51c19b870 msgr2=0x7ff51c19bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:20.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.714+0000 7ff521200700 1 --2- 172.21.15.1:0/4084517268 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff51c19b870 0x7ff51c19bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:20.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.714+0000 7ff521200700 1 -- 172.21.15.1:0/4084517268 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff50c0095f0 con 0x7ff51c100ff0 2024-02-02T15:31:20.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.714+0000 7ff521c00700 1 --2- 172.21.15.1:0/4084517268 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff51c19b870 0x7ff51c19bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:31:20.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.715+0000 7ff521200700 1 --2- 172.21.15.1:0/4084517268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff51c100ff0 0x7ff51c196ed0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff50c005910 tx=0x7ff50c005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:20.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.715+0000 7ff519600700 1 -- 172.21.15.1:0/4084517268 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff50c01d070 con 0x7ff51c100ff0 2024-02-02T15:31:20.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.715+0000 7ff519600700 1 -- 172.21.15.1:0/4084517268 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff50c022470 con 0x7ff51c100ff0 2024-02-02T15:31:20.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.715+0000 7ff519600700 1 -- 172.21.15.1:0/4084517268 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff50c017650 con 0x7ff51c100ff0 2024-02-02T15:31:20.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.716+0000 7ff523c00700 1 -- 172.21.15.1:0/4084517268 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff51c1a3b20 con 0x7ff51c100ff0 2024-02-02T15:31:20.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.717+0000 7ff523c00700 1 -- 172.21.15.1:0/4084517268 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff51c1a3ef0 con 0x7ff51c100ff0 2024-02-02T15:31:20.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.721+0000 7ff519600700 1 -- 172.21.15.1:0/4084517268 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 27) v1 ==== 135200+0+0 (secure 0 0 0) 0x7ff50c0225e0 con 0x7ff51c100ff0 2024-02-02T15:31:20.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.723+0000 7ff537687b80 1 Processor -- start 2024-02-02T15:31:20.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.724+0000 7ff519600700 1 --2- 172.21.15.1:0/4084517268 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff4fc0a0690 0x7ff4fc0a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:20.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.725+0000 7ff519600700 1 -- 172.21.15.1:0/4084517268 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7ff50c0cd3a0 con 0x7ff51c100ff0 2024-02-02T15:31:20.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.725+0000 7ff537687b80 1 -- start start 2024-02-02T15:31:20.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.726+0000 7ff537687b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563a14e36c50 0x563a14e35090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:20.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.726+0000 7ff521c00700 1 --2- 172.21.15.1:0/4084517268 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff4fc0a0690 0x7ff4fc0a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:20.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.726+0000 7ff537687b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563a14e355d0 0x563a14e35a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:20.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.726+0000 7ff537687b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a14e48f00 0x563a14e4b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:20.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.726+0000 7ff537687b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a14da9540 con 0x563a14e48f00 2024-02-02T15:31:20.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.726+0000 7ff537687b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a14d8db20 con 0x563a14e355d0 2024-02-02T15:31:20.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.726+0000 7ff537687b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a14d8b4e0 con 0x563a14e36c50 2024-02-02T15:31:20.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.726+0000 7ff521200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563a14e36c50 0x563a14e35090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:20.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.726+0000 7ff521200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563a14e36c50 0x563a14e35090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:56362/0 (socket says 172.21.15.1:56362) 2024-02-02T15:31:20.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.726+0000 7ff521200700 1 -- 172.21.15.1:0/3368195349 learned_addr learned my addr 172.21.15.1:0/3368195349 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:31:20.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.727+0000 7ff521200700 1 -- 172.21.15.1:0/3368195349 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563a14e355d0 msgr2=0x563a14e35a50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-02T15:31:20.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.727+0000 7ff521200700 1 --2- 172.21.15.1:0/3368195349 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563a14e355d0 0x563a14e35a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:20.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.727+0000 7ff521200700 1 -- 172.21.15.1:0/3368195349 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a14e48f00 msgr2=0x563a14e4b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:20.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.727+0000 7ff521200700 1 --2- 172.21.15.1:0/3368195349 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a14e48f00 0x563a14e4b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:20.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.727+0000 7ff521200700 1 -- 172.21.15.1:0/3368195349 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a14c08430 con 0x563a14e36c50 2024-02-02T15:31:20.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.728+0000 7ff521200700 1 --2- 172.21.15.1:0/3368195349 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563a14e36c50 0x563a14e35090 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff50c0a20b0 tx=0x7ff50c022c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb7833d59348ffe2 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:20.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.728+0000 7ff521c00700 1 --2- 172.21.15.1:0/4084517268 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff4fc0a0690 0x7ff4fc0a2b50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff514003e80 tx=0x7ff514009920 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:20.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.729+0000 7ff512000700 1 -- 172.21.15.1:0/3368195349 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff50c09ad00 con 0x563a14e36c50 2024-02-02T15:31:20.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.729+0000 7ff512000700 1 -- 172.21.15.1:0/3368195349 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff50c09a600 con 0x563a14e36c50 2024-02-02T15:31:20.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.729+0000 7ff512000700 1 -- 172.21.15.1:0/3368195349 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff50c09e120 con 0x563a14e36c50 2024-02-02T15:31:20.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.731+0000 7ff537687b80 1 -- 172.21.15.1:0/3368195349 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563a14e36c50 msgr2=0x563a14e35090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:20.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.731+0000 7ff537687b80 1 --2- 172.21.15.1:0/3368195349 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563a14e36c50 0x563a14e35090 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff50c0a20b0 tx=0x7ff50c022c90 comp rx=0 tx=0).stop 2024-02-02T15:31:20.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.731+0000 7ff537687b80 1 -- 172.21.15.1:0/3368195349 shutdown_connections 2024-02-02T15:31:20.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.731+0000 7ff537687b80 1 --2- 172.21.15.1:0/3368195349 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a14e48f00 0x563a14e4b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:20.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.731+0000 7ff537687b80 1 --2- 172.21.15.1:0/3368195349 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563a14e36c50 0x563a14e35090 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:20.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.731+0000 7ff537687b80 1 --2- 172.21.15.1:0/3368195349 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563a14e355d0 0x563a14e35a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:20.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.731+0000 7ff537687b80 1 -- 172.21.15.1:0/3368195349 >> 172.21.15.1:0/3368195349 conn(0x563a14d94e30 msgr2=0x563a14e31430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:20.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.732+0000 7ff537687b80 1 -- 172.21.15.1:0/3368195349 shutdown_connections 2024-02-02T15:31:20.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.733+0000 7ff537687b80 1 -- 172.21.15.1:0/3368195349 wait complete. 2024-02-02T15:31:20.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.734+0000 7ff537687b80 1 -- start start 2024-02-02T15:31:20.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.734+0000 7ff537687b80 1 Processor -- start 2024-02-02T15:31:20.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.735+0000 7ff537687b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563a14e355d0 0x563a14e3bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:20.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.735+0000 7ff537687b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a14e36c50 0x563a14e38e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:20.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.735+0000 7ff537687b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563a14e48f00 0x563a14e39360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:20.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.735+0000 7ff537687b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a14d86dd0 con 0x563a14e36c50 2024-02-02T15:31:20.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.735+0000 7ff537687b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a14d22e00 con 0x563a14e48f00 2024-02-02T15:31:20.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.735+0000 7ff537687b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a14d22aa0 con 0x563a14e355d0 2024-02-02T15:31:20.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.735+0000 7ff537687b80 10 client.0 _ll_register_callbacks cb 0x563a14d24310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:31:20.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.736+0000 7ff521c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563a14e48f00 0x563a14e39360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:20.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.736+0000 7ff521c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563a14e48f00 0x563a14e39360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:54134/0 (socket says 172.21.15.1:54134) 2024-02-02T15:31:20.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.736+0000 7ff521c00700 1 -- 172.21.15.1:0/1208770348 learned_addr learned my addr 172.21.15.1:0/1208770348 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:31:20.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.736+0000 7ff51be00700 1 --2- 172.21.15.1:0/1208770348 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a14e36c50 0x563a14e38e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:20.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.736+0000 7ff521c00700 1 -- 172.21.15.1:0/1208770348 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563a14e355d0 msgr2=0x563a14e3bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:20.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.736+0000 7ff521200700 1 --2- 172.21.15.1:0/1208770348 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563a14e355d0 0x563a14e3bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:20.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.736+0000 7ff521c00700 1 --2- 172.21.15.1:0/1208770348 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563a14e355d0 0x563a14e3bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:20.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.737+0000 7ff521c00700 1 -- 172.21.15.1:0/1208770348 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a14e36c50 msgr2=0x563a14e38e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:20.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.737+0000 7ff521c00700 1 --2- 172.21.15.1:0/1208770348 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a14e36c50 0x563a14e38e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:20.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.737+0000 7ff521c00700 1 -- 172.21.15.1:0/1208770348 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff51c1a3ef0 con 0x563a14e48f00 2024-02-02T15:31:20.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.737+0000 7ff51be00700 1 --2- 172.21.15.1:0/1208770348 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a14e36c50 0x563a14e38e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:31:20.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.737+0000 7ff521200700 1 --2- 172.21.15.1:0/1208770348 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563a14e355d0 0x563a14e3bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:31:20.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.738+0000 7ff521c00700 1 --2- 172.21.15.1:0/1208770348 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563a14e48f00 0x563a14e39360 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff51400ee40 tx=0x7ff514004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2ced5b7472a1645 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:20.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.738+0000 7ff50aa00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:31:20.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.738+0000 7ff50aa00700 1 -- 172.21.15.1:0/1208770348 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff514017ce0 con 0x563a14e48f00 2024-02-02T15:31:20.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.738+0000 7ff537687b80 1 -- 172.21.15.1:0/1208770348 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563a14e29170 con 0x563a14e48f00 2024-02-02T15:31:20.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.739+0000 7ff50aa00700 1 -- 172.21.15.1:0/1208770348 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7ff514017e40 con 0x563a14e48f00 2024-02-02T15:31:20.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.739+0000 7ff50aa00700 1 -- 172.21.15.1:0/1208770348 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff514019460 con 0x563a14e48f00 2024-02-02T15:31:20.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.739+0000 7ff537687b80 10 client.26837 fetch_fsmap learned FSMap version 32 2024-02-02T15:31:20.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.739+0000 7ff537687b80 1 -- 172.21.15.1:0/1208770348 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x563a14bb4be0 con 0x563a14e48f00 2024-02-02T15:31:20.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.740+0000 7ff50aa00700 1 -- 172.21.15.1:0/1208770348 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563a14e29170 con 0x563a14e48f00 2024-02-02T15:31:20.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.740+0000 7ff50aa00700 1 -- 172.21.15.1:0/1208770348 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 32) v1 ==== 7490+0+0 (secure 0 0 0) 0x7ff514022840 con 0x563a14e48f00 2024-02-02T15:31:20.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.740+0000 7ff537687b80 10 client.26837 fetch_fsmap finished waiting for FSMap version 32 2024-02-02T15:31:20.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.740+0000 7ff537687b80 10 client.26837 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:31:20.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.740+0000 7ff537687b80 20 client.26837 populate_metadata read hostname 'smithi001' 2024-02-02T15:31:20.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.740+0000 7ff537687b80 1 --2- 172.21.15.1:0/1208770348 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x563a14e40480 0x563a14e428a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:20.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.740+0000 7ff537687b80 4 client.26837 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:31:20.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.740+0000 7ff537687b80 1 -- 172.21.15.1:0/1208770348 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563a14d7fe50 con 0x563a14e40480 2024-02-02T15:31:20.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.741+0000 7ff521200700 1 --2- 172.21.15.1:0/1208770348 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x563a14e40480 0x563a14e428a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:20.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.743+0000 7ff521200700 1 --2- 172.21.15.1:0/1208770348 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x563a14e40480 0x563a14e428a0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f342357f32a0a29 server_cookie=698ab8edfebcf34d in_seq=0 out_seq=0 2024-02-02T15:31:20.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.743+0000 7ff50aa00700 10 client.26837 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:31:20.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.744+0000 7ff50aa00700 1 -- 172.21.15.1:0/1208770348 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x563a14d7fe50 con 0x563a14e40480 2024-02-02T15:31:20.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.744+0000 7ff50aa00700 10 client.26837 handle_command_reply: tid=0 2024-02-02T15:31:20.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.745+0000 7ff537687b80 1 client.26837 shutdown 2024-02-02T15:31:20.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.746+0000 7ff537687b80 1 -- 172.21.15.1:0/1208770348 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563a14e48f00 msgr2=0x563a14e39360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:20.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.746+0000 7ff537687b80 1 --2- 172.21.15.1:0/1208770348 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563a14e48f00 0x563a14e39360 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff51400ee40 tx=0x7ff514004b40 comp rx=0 tx=0).stop 2024-02-02T15:31:20.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.747+0000 7ff537687b80 1 -- 172.21.15.1:0/1208770348 shutdown_connections 2024-02-02T15:31:20.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.747+0000 7ff537687b80 1 --2- 172.21.15.1:0/1208770348 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x563a14e40480 0x563a14e428a0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:20.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.747+0000 7ff537687b80 1 --2- 172.21.15.1:0/1208770348 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563a14e48f00 0x563a14e39360 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:20.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.747+0000 7ff537687b80 1 --2- 172.21.15.1:0/1208770348 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563a14e355d0 0x563a14e3bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:20.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.747+0000 7ff537687b80 1 --2- 172.21.15.1:0/1208770348 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a14e36c50 0x563a14e38e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:20.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.747+0000 7ff50aa00700 0 client.26837 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:31:20.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.747+0000 7ff537687b80 1 -- 172.21.15.1:0/1208770348 >> 172.21.15.1:0/1208770348 conn(0x563a14d94e30 msgr2=0x563a14e30fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:20.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.748+0000 7ff537687b80 1 -- 172.21.15.1:0/1208770348 shutdown_connections 2024-02-02T15:31:20.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.748+0000 7ff537687b80 1 -- 172.21.15.1:0/1208770348 wait complete. 2024-02-02T15:31:20.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.749+0000 7ff537687b80 20 client.26837 trim_cache size 0 max 16384 2024-02-02T15:31:20.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.787+0000 7ff537687b80 1 Processor -- start 2024-02-02T15:31:20.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.788+0000 7ff537687b80 1 -- start start 2024-02-02T15:31:20.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.788+0000 7ff537687b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563a14e355d0 0x563a14e4f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:20.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.788+0000 7ff537687b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563a14e36c50 0x563a14e39240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:20.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.788+0000 7ff537687b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a14e48f00 0x563a14e4a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:20.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.788+0000 7ff537687b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a14c0a720 con 0x563a14e48f00 2024-02-02T15:31:20.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.788+0000 7ff537687b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a14d8b4e0 con 0x563a14e36c50 2024-02-02T15:31:20.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.788+0000 7ff537687b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a14e3c1a0 con 0x563a14e355d0 2024-02-02T15:31:20.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.788+0000 7ff51be00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563a14e36c50 0x563a14e39240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:20.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.789+0000 7ff521200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563a14e355d0 0x563a14e4f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:20.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.789+0000 7ff521c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a14e48f00 0x563a14e4a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:20.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.789+0000 7ff521200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563a14e355d0 0x563a14e4f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:56376/0 (socket says 172.21.15.1:56376) 2024-02-02T15:31:20.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.789+0000 7ff521200700 1 -- 172.21.15.1:0/3708082575 learned_addr learned my addr 172.21.15.1:0/3708082575 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:31:20.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.789+0000 7ff51be00700 1 -- 172.21.15.1:0/3708082575 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563a14e355d0 msgr2=0x563a14e4f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:20.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.789+0000 7ff51be00700 1 --2- 172.21.15.1:0/3708082575 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563a14e355d0 0x563a14e4f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:20.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.789+0000 7ff51be00700 1 -- 172.21.15.1:0/3708082575 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a14e48f00 msgr2=0x563a14e4a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:20.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.789+0000 7ff51be00700 1 --2- 172.21.15.1:0/3708082575 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a14e48f00 0x563a14e4a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:20.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.789+0000 7ff51be00700 1 -- 172.21.15.1:0/3708082575 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a14e454c0 con 0x563a14e36c50 2024-02-02T15:31:20.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.790+0000 7ff51be00700 1 --2- 172.21.15.1:0/3708082575 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563a14e36c50 0x563a14e39240 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x563a14e3a3d0 tx=0x7ff504009ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2a45bdfde89453f server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:20.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.791+0000 7ff512a00700 1 -- 172.21.15.1:0/3708082575 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff504013b60 con 0x563a14e36c50 2024-02-02T15:31:20.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.792+0000 7ff512a00700 1 -- 172.21.15.1:0/3708082575 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff5040044f0 con 0x563a14e36c50 2024-02-02T15:31:20.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.792+0000 7ff512a00700 1 -- 172.21.15.1:0/3708082575 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff504003ea0 con 0x563a14e36c50 2024-02-02T15:31:20.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.793+0000 7ff537687b80 1 -- 172.21.15.1:0/3708082575 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563a14e36c50 msgr2=0x563a14e39240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:20.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.793+0000 7ff537687b80 1 --2- 172.21.15.1:0/3708082575 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563a14e36c50 0x563a14e39240 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x563a14e3a3d0 tx=0x7ff504009ee0 comp rx=0 tx=0).stop 2024-02-02T15:31:20.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.793+0000 7ff537687b80 1 -- 172.21.15.1:0/3708082575 shutdown_connections 2024-02-02T15:31:20.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.793+0000 7ff537687b80 1 --2- 172.21.15.1:0/3708082575 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a14e48f00 0x563a14e4a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:20.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.793+0000 7ff537687b80 1 --2- 172.21.15.1:0/3708082575 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563a14e355d0 0x563a14e4f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:20.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.793+0000 7ff537687b80 1 --2- 172.21.15.1:0/3708082575 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563a14e36c50 0x563a14e39240 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:20.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.793+0000 7ff537687b80 1 -- 172.21.15.1:0/3708082575 >> 172.21.15.1:0/3708082575 conn(0x563a14d94e30 msgr2=0x563a14e53e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:20.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.793+0000 7ff537687b80 1 -- 172.21.15.1:0/3708082575 shutdown_connections 2024-02-02T15:31:20.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.794+0000 7ff537687b80 1 -- 172.21.15.1:0/3708082575 wait complete. 2024-02-02T15:31:20.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.794+0000 7ff537687b80 1 -- start start 2024-02-02T15:31:20.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.795+0000 7ff537687b80 1 Processor -- start 2024-02-02T15:31:20.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.795+0000 7ff537687b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563a14e355d0 0x563a14e46b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:20.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.795+0000 7ff537687b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a14e36c50 0x563a14e47090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:20.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.796+0000 7ff51be00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a14e36c50 0x563a14e47090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:20.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.796+0000 7ff521200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563a14e355d0 0x563a14e46b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:20.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.796+0000 7ff51be00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a14e36c50 0x563a14e47090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55108/0 (socket says 172.21.15.1:55108) 2024-02-02T15:31:20.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.796+0000 7ff521200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563a14e355d0 0x563a14e46b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:56384/0 (socket says 172.21.15.1:56384) 2024-02-02T15:31:20.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.796+0000 7ff51be00700 1 -- 172.21.15.1:0/1163608640 learned_addr learned my addr 172.21.15.1:0/1163608640 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:31:20.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.797+0000 7ff537687b80 1 --2- 172.21.15.1:0/1163608640 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563a14e48f00 0x563a14eac780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:20.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.797+0000 7ff537687b80 1 -- 172.21.15.1:0/1163608640 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a14eacd30 con 0x563a14e36c50 2024-02-02T15:31:20.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.797+0000 7ff537687b80 1 -- 172.21.15.1:0/1163608640 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a14eacea0 con 0x563a14e48f00 2024-02-02T15:31:20.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.798+0000 7ff521c00700 1 --2- 172.21.15.1:0/1163608640 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563a14e48f00 0x563a14eac780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:20.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.798+0000 7ff537687b80 1 -- 172.21.15.1:0/1163608640 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a14ead060 con 0x563a14e355d0 2024-02-02T15:31:20.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.798+0000 7ff537687b80 10 client.0 _ll_register_callbacks cb 0x563a14d24310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:31:20.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.798+0000 7ff51be00700 1 -- 172.21.15.1:0/1163608640 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563a14e355d0 msgr2=0x563a14e46b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:20.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.798+0000 7ff51be00700 1 --2- 172.21.15.1:0/1163608640 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563a14e355d0 0x563a14e46b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:20.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.798+0000 7ff51be00700 1 -- 172.21.15.1:0/1163608640 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563a14e48f00 msgr2=0x563a14eac780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:20.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.798+0000 7ff51be00700 1 --2- 172.21.15.1:0/1163608640 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563a14e48f00 0x563a14eac780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:20.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.798+0000 7ff51be00700 1 -- 172.21.15.1:0/1163608640 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5040098e0 con 0x563a14e36c50 2024-02-02T15:31:20.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.799+0000 7ff50aa00700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:31:20.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.800+0000 7ff51be00700 1 --2- 172.21.15.1:0/1163608640 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a14e36c50 0x563a14e47090 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x563a14e2db20 tx=0x7ff504013cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16a1c001d002df72 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:20.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.801+0000 7ff50aa00700 1 -- 172.21.15.1:0/1163608640 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff504020040 con 0x563a14e36c50 2024-02-02T15:31:20.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.801+0000 7ff50aa00700 1 -- 172.21.15.1:0/1163608640 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7ff504005440 con 0x563a14e36c50 2024-02-02T15:31:20.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.801+0000 7ff537687b80 1 -- 172.21.15.1:0/1163608640 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff50c09a600 con 0x563a14e36c50 2024-02-02T15:31:20.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.801+0000 7ff50aa00700 1 -- 172.21.15.1:0/1163608640 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff504005600 con 0x563a14e36c50 2024-02-02T15:31:20.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.801+0000 7ff537687b80 10 client.16782 fetch_fsmap learned FSMap version 32 2024-02-02T15:31:20.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.801+0000 7ff537687b80 1 -- 172.21.15.1:0/1163608640 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x563a14d4cf20 con 0x563a14e36c50 2024-02-02T15:31:20.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.802+0000 7ff50aa00700 1 -- 172.21.15.1:0/1163608640 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff50c09a600 con 0x563a14e36c50 2024-02-02T15:31:20.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.802+0000 7ff50aa00700 1 -- 172.21.15.1:0/1163608640 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 32) v1 ==== 7490+0+0 (secure 0 0 0) 0x7ff504023050 con 0x563a14e36c50 2024-02-02T15:31:20.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.802+0000 7ff537687b80 10 client.16782 fetch_fsmap finished waiting for FSMap version 32 2024-02-02T15:31:20.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.802+0000 7ff537687b80 10 client.16782 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:31:20.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.802+0000 7ff537687b80 20 client.16782 populate_metadata read hostname 'smithi001' 2024-02-02T15:31:20.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.803+0000 7ff537687b80 1 --2- 172.21.15.1:0/1163608640 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x563a14eb4930 0x563a14eb6d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:20.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.803+0000 7ff537687b80 4 client.16782 mds_command: new command op to 24557 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-02T15:31:20.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.803+0000 7ff537687b80 1 -- 172.21.15.1:0/1163608640 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x563a14d45fd0 con 0x563a14eb4930 2024-02-02T15:31:20.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.803+0000 7ff521200700 1 --2- 172.21.15.1:0/1163608640 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x563a14eb4930 0x563a14eb6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:20.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.804+0000 7ff521200700 1 --2- 172.21.15.1:0/1163608640 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x563a14eb4930 0x563a14eb6d50 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dccfe746377560de server_cookie=2e9064b15947751 in_seq=0 out_seq=0 2024-02-02T15:31:20.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.804+0000 7ff50aa00700 10 client.16782 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:31:20.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.806+0000 7ff50aa00700 1 -- 172.21.15.1:0/1163608640 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x563a14d45fd0 con 0x563a14eb4930 2024-02-02T15:31:20.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.807+0000 7ff50aa00700 10 client.16782 handle_command_reply: tid=0 2024-02-02T15:31:20.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.807+0000 7ff537687b80 1 client.16782 shutdown 2024-02-02T15:31:20.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.807+0000 7ff537687b80 1 -- 172.21.15.1:0/1163608640 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a14e36c50 msgr2=0x563a14e47090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:20.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.807+0000 7ff537687b80 1 --2- 172.21.15.1:0/1163608640 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a14e36c50 0x563a14e47090 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x563a14e2db20 tx=0x7ff504013cd0 comp rx=0 tx=0).stop 2024-02-02T15:31:20.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.807+0000 7ff537687b80 1 -- 172.21.15.1:0/1163608640 shutdown_connections 2024-02-02T15:31:20.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.807+0000 7ff537687b80 1 --2- 172.21.15.1:0/1163608640 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x563a14eb4930 0x563a14eb6d50 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:20.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.807+0000 7ff537687b80 1 --2- 172.21.15.1:0/1163608640 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563a14e48f00 0x563a14eac780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:20.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.807+0000 7ff537687b80 1 --2- 172.21.15.1:0/1163608640 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563a14e355d0 0x563a14e46b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:20.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.807+0000 7ff537687b80 1 --2- 172.21.15.1:0/1163608640 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563a14e36c50 0x563a14e47090 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:20.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.807+0000 7ff537687b80 1 -- 172.21.15.1:0/1163608640 >> 172.21.15.1:0/1163608640 conn(0x563a14d94e30 msgr2=0x563a14e37030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:20.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.807+0000 7ff50aa00700 0 client.16782 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:31:20.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.808+0000 7ff537687b80 1 -- 172.21.15.1:0/1163608640 shutdown_connections 2024-02-02T15:31:20.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.808+0000 7ff537687b80 1 -- 172.21.15.1:0/1163608640 wait complete. 2024-02-02T15:31:20.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.808+0000 7ff537687b80 20 client.16782 trim_cache size 0 max 16384 2024-02-02T15:31:20.809 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:31:20.809 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2024-02-02T15:31:20.809 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "e0eb4f2c-b2ed-4cf6-b4be-0b26da28ad77", 2024-02-02T15:31:20.809 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2024-02-02T15:31:20.810 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:31:20.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.808+0000 7ff523c00700 1 -- 172.21.15.1:0/4084517268 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff4fc0a0690 msgr2=0x7ff4fc0a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:20.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.809+0000 7ff523c00700 1 --2- 172.21.15.1:0/4084517268 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff4fc0a0690 0x7ff4fc0a2b50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff514003e80 tx=0x7ff514009920 comp rx=0 tx=0).stop 2024-02-02T15:31:20.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.809+0000 7ff523c00700 1 -- 172.21.15.1:0/4084517268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff51c100ff0 msgr2=0x7ff51c196ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:20.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.809+0000 7ff523c00700 1 --2- 172.21.15.1:0/4084517268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff51c100ff0 0x7ff51c196ed0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff50c005910 tx=0x7ff50c005740 comp rx=0 tx=0).stop 2024-02-02T15:31:20.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.809+0000 7ff523c00700 1 -- 172.21.15.1:0/4084517268 shutdown_connections 2024-02-02T15:31:20.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.809+0000 7ff523c00700 1 --2- 172.21.15.1:0/4084517268 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff4fc0a0690 0x7ff4fc0a2b50 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:20.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.809+0000 7ff523c00700 1 --2- 172.21.15.1:0/4084517268 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff51c19b870 0x7ff51c19bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:20.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.809+0000 7ff523c00700 1 --2- 172.21.15.1:0/4084517268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff51c100ff0 0x7ff51c196ed0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:20.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.809+0000 7ff523c00700 1 --2- 172.21.15.1:0/4084517268 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff51c106110 0x7ff51c197410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:20.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.809+0000 7ff523c00700 1 -- 172.21.15.1:0/4084517268 >> 172.21.15.1:0/4084517268 conn(0x7ff51c0fc810 msgr2=0x7ff51c1049c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:20.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.809+0000 7ff523c00700 1 -- 172.21.15.1:0/4084517268 shutdown_connections 2024-02-02T15:31:20.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:20.809+0000 7ff523c00700 1 -- 172.21.15.1:0/4084517268 wait complete. 2024-02-02T15:31:20.828 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-02T15:31:21.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.078+0000 7f93eae00700 1 -- 172.21.15.1:0/1325675990 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93e4105020 msgr2=0x7f93e410cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:21.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.078+0000 7f93eae00700 1 --2- 172.21.15.1:0/1325675990 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93e4105020 0x7f93e410cba0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f93dc009990 tx=0x7f93dc009ca0 comp rx=0 tx=0).stop 2024-02-02T15:31:21.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.078+0000 7f93eae00700 1 -- 172.21.15.1:0/1325675990 shutdown_connections 2024-02-02T15:31:21.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.078+0000 7f93eae00700 1 --2- 172.21.15.1:0/1325675990 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93e4105020 0x7f93e410cba0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:21.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.078+0000 7f93eae00700 1 --2- 172.21.15.1:0/1325675990 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93e41000e0 0x7f93e41004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:21.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.078+0000 7f93eae00700 1 --2- 172.21.15.1:0/1325675990 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f93e4100a90 0x7f93e41048f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:21.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.078+0000 7f93eae00700 1 -- 172.21.15.1:0/1325675990 >> 172.21.15.1:0/1325675990 conn(0x7f93e40757d0 msgr2=0x7f93e4075be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:21.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.079+0000 7f93eae00700 1 -- 172.21.15.1:0/1325675990 shutdown_connections 2024-02-02T15:31:21.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.079+0000 7f93eae00700 1 -- 172.21.15.1:0/1325675990 wait complete. 2024-02-02T15:31:21.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.080+0000 7f93eae00700 1 Processor -- start 2024-02-02T15:31:21.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.081+0000 7f93eae00700 1 -- start start 2024-02-02T15:31:21.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.081+0000 7f93eae00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93e41000e0 0x7f93e4197ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:21.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.081+0000 7f93eae00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93e4105020 0x7f93e41981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:21.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.081+0000 7f93eae00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f93e4198880 0x7f93e41a2d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:21.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.081+0000 7f93eae00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e4195ac0 con 0x7f93e4198880 2024-02-02T15:31:21.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.081+0000 7f93eae00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e4195c30 con 0x7f93e41000e0 2024-02-02T15:31:21.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.081+0000 7f93eae00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e4195df0 con 0x7f93e4105020 2024-02-02T15:31:21.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.082+0000 7f93db200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93e4105020 0x7f93e41981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:21.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.082+0000 7f93db200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93e4105020 0x7f93e41981e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:56402/0 (socket says 172.21.15.1:56402) 2024-02-02T15:31:21.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.082+0000 7f93db200700 1 -- 172.21.15.1:0/3086365961 learned_addr learned my addr 172.21.15.1:0/3086365961 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:31:21.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.082+0000 7f93e3e00700 1 --2- 172.21.15.1:0/3086365961 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93e41000e0 0x7f93e4197ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:21.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.082+0000 7f93db200700 1 -- 172.21.15.1:0/3086365961 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93e41000e0 msgr2=0x7f93e4197ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:21.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.082+0000 7f93db200700 1 --2- 172.21.15.1:0/3086365961 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93e41000e0 0x7f93e4197ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:21.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.082+0000 7f93db200700 1 -- 172.21.15.1:0/3086365961 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f93e4198880 msgr2=0x7f93e41a2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:21.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.082+0000 7f93db200700 1 --2- 172.21.15.1:0/3086365961 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f93e4198880 0x7f93e41a2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:21.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.082+0000 7f93db200700 1 -- 172.21.15.1:0/3086365961 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93dc0095f0 con 0x7f93e4105020 2024-02-02T15:31:21.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.083+0000 7f93db200700 1 --2- 172.21.15.1:0/3086365961 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93e4105020 0x7f93e41981e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f93cc00b500 tx=0x7f93cc00b8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:21.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.083+0000 7f93e1600700 1 -- 172.21.15.1:0/3086365961 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93cc011830 con 0x7f93e4105020 2024-02-02T15:31:21.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.083+0000 7f93e1600700 1 -- 172.21.15.1:0/3086365961 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93cc011e50 con 0x7f93e4105020 2024-02-02T15:31:21.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.084+0000 7f93eae00700 1 -- 172.21.15.1:0/3086365961 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93e41960d0 con 0x7f93e4105020 2024-02-02T15:31:21.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.084+0000 7f93eae00700 1 -- 172.21.15.1:0/3086365961 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93e4196630 con 0x7f93e4105020 2024-02-02T15:31:21.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.084+0000 7f93e1600700 1 -- 172.21.15.1:0/3086365961 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93cc00f2e0 con 0x7f93e4105020 2024-02-02T15:31:21.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.089+0000 7f93e1600700 1 -- 172.21.15.1:0/3086365961 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 27) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f93cc011990 con 0x7f93e4105020 2024-02-02T15:31:21.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.091+0000 7f93e1600700 1 --2- 172.21.15.1:0/3086365961 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f93c40a0790 0x7f93c40a2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:21.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.092+0000 7f93e1600700 1 -- 172.21.15.1:0/3086365961 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f93cc0cbab0 con 0x7f93e4105020 2024-02-02T15:31:21.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.092+0000 7f93fe8e4b80 1 Processor -- start 2024-02-02T15:31:21.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.092+0000 7f93e3e00700 1 --2- 172.21.15.1:0/3086365961 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f93c40a0790 0x7f93c40a2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:21.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.093+0000 7f93fe8e4b80 1 -- start start 2024-02-02T15:31:21.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.093+0000 7f93fe8e4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558fbc361010 0x558fbc3613f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:21.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.093+0000 7f93fe8e4b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fbc361930 0x558fbc36c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:21.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.093+0000 7f93fe8e4b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558fbc36c7e0 0x558fbc36ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:21.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.093+0000 7f93fe8e4b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fbc2d54c0 con 0x558fbc361930 2024-02-02T15:31:21.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.093+0000 7f93fe8e4b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fbc2b9aa0 con 0x558fbc36c7e0 2024-02-02T15:31:21.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.093+0000 7f93fe8e4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fbc2b7460 con 0x558fbc361010 2024-02-02T15:31:21.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.094+0000 7f93e3e00700 1 --2- 172.21.15.1:0/3086365961 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f93c40a0790 0x7f93c40a2c50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f93d4009300 tx=0x7f93d400b040 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:21.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.094+0000 7f93e3e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558fbc361010 0x558fbc3613f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:21.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.094+0000 7f93e3e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558fbc361010 0x558fbc3613f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:56412/0 (socket says 172.21.15.1:56412) 2024-02-02T15:31:21.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.094+0000 7f93e3e00700 1 -- 172.21.15.1:0/2867628930 learned_addr learned my addr 172.21.15.1:0/2867628930 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:31:21.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.094+0000 7f93db200700 1 --2- 172.21.15.1:0/2867628930 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fbc361930 0x558fbc36c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:21.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.094+0000 7f93e8e00700 1 --2- 172.21.15.1:0/2867628930 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558fbc36c7e0 0x558fbc36ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:21.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.094+0000 7f93e3e00700 1 -- 172.21.15.1:0/2867628930 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558fbc36c7e0 msgr2=0x558fbc36ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:21.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.094+0000 7f93e3e00700 1 --2- 172.21.15.1:0/2867628930 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558fbc36c7e0 0x558fbc36ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:21.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.094+0000 7f93e3e00700 1 -- 172.21.15.1:0/2867628930 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fbc361930 msgr2=0x558fbc36c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:21.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.094+0000 7f93e3e00700 1 --2- 172.21.15.1:0/2867628930 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fbc361930 0x558fbc36c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:21.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.094+0000 7f93e3e00700 1 -- 172.21.15.1:0/2867628930 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558fbc1343b0 con 0x558fbc361010 2024-02-02T15:31:21.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.094+0000 7f93e3e00700 1 --2- 172.21.15.1:0/2867628930 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558fbc361010 0x558fbc3613f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f93d4018790 tx=0x7f93d4012000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=43f00fd57b802080 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:21.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.095+0000 7f93d9400700 1 -- 172.21.15.1:0/2867628930 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d401d070 con 0x558fbc361010 2024-02-02T15:31:21.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.095+0000 7f93d9400700 1 -- 172.21.15.1:0/2867628930 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93d40180d0 con 0x558fbc361010 2024-02-02T15:31:21.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.095+0000 7f93d9400700 1 -- 172.21.15.1:0/2867628930 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d40145d0 con 0x558fbc361010 2024-02-02T15:31:21.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.095+0000 7f93fe8e4b80 1 -- 172.21.15.1:0/2867628930 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558fbc361010 msgr2=0x558fbc3613f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:21.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.095+0000 7f93fe8e4b80 1 --2- 172.21.15.1:0/2867628930 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558fbc361010 0x558fbc3613f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f93d4018790 tx=0x7f93d4012000 comp rx=0 tx=0).stop 2024-02-02T15:31:21.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.096+0000 7f93fe8e4b80 1 -- 172.21.15.1:0/2867628930 shutdown_connections 2024-02-02T15:31:21.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.096+0000 7f93fe8e4b80 1 --2- 172.21.15.1:0/2867628930 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558fbc36c7e0 0x558fbc36ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:21.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.096+0000 7f93fe8e4b80 1 --2- 172.21.15.1:0/2867628930 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558fbc361010 0x558fbc3613f0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:21.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.096+0000 7f93fe8e4b80 1 --2- 172.21.15.1:0/2867628930 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fbc361930 0x558fbc36c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:21.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.096+0000 7f93fe8e4b80 1 -- 172.21.15.1:0/2867628930 >> 172.21.15.1:0/2867628930 conn(0x558fbc2c0db0 msgr2=0x558fbc35d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:21.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.096+0000 7f93fe8e4b80 1 -- 172.21.15.1:0/2867628930 shutdown_connections 2024-02-02T15:31:21.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.096+0000 7f93fe8e4b80 1 -- 172.21.15.1:0/2867628930 wait complete. 2024-02-02T15:31:21.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.096+0000 7f93fe8e4b80 1 -- start start 2024-02-02T15:31:21.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.096+0000 7f93fe8e4b80 1 Processor -- start 2024-02-02T15:31:21.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.097+0000 7f93fe8e4b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558fbc361010 0x558fbc36c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:21.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.097+0000 7f93fe8e4b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fbc361930 0x558fbc369350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:21.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.097+0000 7f93fe8e4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558fbc36c7e0 0x558fbc3698d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:21.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.097+0000 7f93fe8e4b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fbc2b2d50 con 0x558fbc361930 2024-02-02T15:31:21.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.097+0000 7f93fe8e4b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fbc24ed80 con 0x558fbc361010 2024-02-02T15:31:21.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.097+0000 7f93fe8e4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fbc24ea20 con 0x558fbc36c7e0 2024-02-02T15:31:21.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.097+0000 7f93fe8e4b80 10 client.0 _ll_register_callbacks cb 0x558fbc250290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:31:21.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.097+0000 7f93e3e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558fbc361010 0x558fbc36c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:21.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.097+0000 7f93e8e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558fbc36c7e0 0x558fbc3698d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:21.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.097+0000 7f93e3e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558fbc361010 0x558fbc36c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:54164/0 (socket says 172.21.15.1:54164) 2024-02-02T15:31:21.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.097+0000 7f93e3e00700 1 -- 172.21.15.1:0/3266718887 learned_addr learned my addr 172.21.15.1:0/3266718887 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:31:21.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.097+0000 7f93db200700 1 --2- 172.21.15.1:0/3266718887 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fbc361930 0x558fbc369350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:21.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.098+0000 7f93e3e00700 1 -- 172.21.15.1:0/3266718887 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558fbc36c7e0 msgr2=0x558fbc3698d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:21.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.098+0000 7f93e3e00700 1 --2- 172.21.15.1:0/3266718887 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558fbc36c7e0 0x558fbc3698d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:21.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.098+0000 7f93e3e00700 1 -- 172.21.15.1:0/3266718887 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fbc361930 msgr2=0x558fbc369350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:21.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.098+0000 7f93e3e00700 1 --2- 172.21.15.1:0/3266718887 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fbc361930 0x558fbc369350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:21.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.098+0000 7f93e3e00700 1 -- 172.21.15.1:0/3266718887 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93d4018440 con 0x558fbc361010 2024-02-02T15:31:21.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.098+0000 7f93e8e00700 1 --2- 172.21.15.1:0/3266718887 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558fbc36c7e0 0x558fbc3698d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:31:21.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.098+0000 7f93db200700 1 --2- 172.21.15.1:0/3266718887 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fbc361930 0x558fbc369350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:31:21.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.098+0000 7f93e3e00700 1 --2- 172.21.15.1:0/3266718887 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558fbc361010 0x558fbc36c140 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f93d4016d90 tx=0x7f93d4018240 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f70a32b1a10cbd server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:21.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.098+0000 7f93d2000700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:31:21.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.098+0000 7f93d2000700 1 -- 172.21.15.1:0/3266718887 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d401d040 con 0x558fbc361010 2024-02-02T15:31:21.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.098+0000 7f93d2000700 1 -- 172.21.15.1:0/3266718887 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f93d4016420 con 0x558fbc361010 2024-02-02T15:31:21.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.099+0000 7f93fe8e4b80 1 -- 172.21.15.1:0/3266718887 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558fbc3550f0 con 0x558fbc361010 2024-02-02T15:31:21.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.099+0000 7f93d2000700 1 -- 172.21.15.1:0/3266718887 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f93d40165e0 con 0x558fbc361010 2024-02-02T15:31:21.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.099+0000 7f93fe8e4b80 10 client.26852 fetch_fsmap learned FSMap version 32 2024-02-02T15:31:21.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.099+0000 7f93fe8e4b80 1 -- 172.21.15.1:0/3266718887 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x558fbc0e0b60 con 0x558fbc361010 2024-02-02T15:31:21.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.099+0000 7f93d2000700 1 -- 172.21.15.1:0/3266718887 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558fbc3550f0 con 0x558fbc361010 2024-02-02T15:31:21.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.099+0000 7f93d2000700 1 -- 172.21.15.1:0/3266718887 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 32) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f93d4022420 con 0x558fbc361010 2024-02-02T15:31:21.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.099+0000 7f93fe8e4b80 10 client.26852 fetch_fsmap finished waiting for FSMap version 32 2024-02-02T15:31:21.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.099+0000 7f93fe8e4b80 10 client.26852 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:31:21.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.099+0000 7f93fe8e4b80 20 client.26852 populate_metadata read hostname 'smithi001' 2024-02-02T15:31:21.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.100+0000 7f93fe8e4b80 1 --2- 172.21.15.1:0/3266718887 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x558fbc37c1c0 0x558fbc37e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:21.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.100+0000 7f93fe8e4b80 4 client.26852 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:31:21.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.100+0000 7f93fe8e4b80 1 -- 172.21.15.1:0/3266718887 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558fbc2abdd0 con 0x558fbc37c1c0 2024-02-02T15:31:21.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.100+0000 7f93db200700 1 --2- 172.21.15.1:0/3266718887 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x558fbc37c1c0 0x558fbc37e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:21.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.100+0000 7f93db200700 1 --2- 172.21.15.1:0/3266718887 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x558fbc37c1c0 0x558fbc37e5e0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f9c361612d79545 server_cookie=19ec81cc84c986db in_seq=0 out_seq=0 2024-02-02T15:31:21.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.100+0000 7f93d2000700 10 client.26852 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:31:21.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.101+0000 7f93d2000700 1 -- 172.21.15.1:0/3266718887 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x558fbc2abdd0 con 0x558fbc37c1c0 2024-02-02T15:31:21.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.101+0000 7f93d2000700 10 client.26852 handle_command_reply: tid=0 2024-02-02T15:31:21.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.101+0000 7f93fe8e4b80 1 client.26852 shutdown 2024-02-02T15:31:21.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.101+0000 7f93fe8e4b80 1 -- 172.21.15.1:0/3266718887 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558fbc361010 msgr2=0x558fbc36c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:21.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.101+0000 7f93fe8e4b80 1 --2- 172.21.15.1:0/3266718887 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558fbc361010 0x558fbc36c140 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f93d4016d90 tx=0x7f93d4018240 comp rx=0 tx=0).stop 2024-02-02T15:31:21.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.101+0000 7f93fe8e4b80 1 -- 172.21.15.1:0/3266718887 shutdown_connections 2024-02-02T15:31:21.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.101+0000 7f93fe8e4b80 1 --2- 172.21.15.1:0/3266718887 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x558fbc37c1c0 0x558fbc37e5e0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:21.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.101+0000 7f93fe8e4b80 1 --2- 172.21.15.1:0/3266718887 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558fbc36c7e0 0x558fbc3698d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:21.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.101+0000 7f93d2000700 0 client.26852 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:31:21.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.101+0000 7f93fe8e4b80 1 --2- 172.21.15.1:0/3266718887 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558fbc361010 0x558fbc36c140 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:21.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.101+0000 7f93fe8e4b80 1 --2- 172.21.15.1:0/3266718887 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fbc361930 0x558fbc369350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:21.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.101+0000 7f93fe8e4b80 1 -- 172.21.15.1:0/3266718887 >> 172.21.15.1:0/3266718887 conn(0x558fbc2c0db0 msgr2=0x558fbc35cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:21.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.102+0000 7f93fe8e4b80 1 -- 172.21.15.1:0/3266718887 shutdown_connections 2024-02-02T15:31:21.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.102+0000 7f93fe8e4b80 1 -- 172.21.15.1:0/3266718887 wait complete. 2024-02-02T15:31:21.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.102+0000 7f93fe8e4b80 20 client.26852 trim_cache size 0 max 16384 2024-02-02T15:31:21.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.120+0000 7f93fe8e4b80 1 Processor -- start 2024-02-02T15:31:21.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.120+0000 7f93fe8e4b80 1 -- start start 2024-02-02T15:31:21.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.121+0000 7f93fe8e4b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558fbc361010 0x558fbc373370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:21.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.121+0000 7f93fe8e4b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fbc361930 0x558fbc36ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:21.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.121+0000 7f93fe8e4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558fbc36c7e0 0x558fbc36bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:21.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.121+0000 7f93fe8e4b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fbc1366a0 con 0x558fbc361930 2024-02-02T15:31:21.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.121+0000 7f93fe8e4b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fbc369ed0 con 0x558fbc361010 2024-02-02T15:31:21.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.121+0000 7f93fe8e4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fbc35e1d0 con 0x558fbc36c7e0 2024-02-02T15:31:21.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.121+0000 7f93e3e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558fbc361010 0x558fbc373370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:21.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.121+0000 7f93e3e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558fbc361010 0x558fbc373370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:54176/0 (socket says 172.21.15.1:54176) 2024-02-02T15:31:21.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.121+0000 7f93e8e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558fbc36c7e0 0x558fbc36bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:21.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.121+0000 7f93e3e00700 1 -- 172.21.15.1:0/3384825336 learned_addr learned my addr 172.21.15.1:0/3384825336 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:31:21.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.121+0000 7f93db200700 1 --2- 172.21.15.1:0/3384825336 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fbc361930 0x558fbc36ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:21.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.121+0000 7f93e3e00700 1 -- 172.21.15.1:0/3384825336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558fbc36c7e0 msgr2=0x558fbc36bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:21.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.121+0000 7f93e3e00700 1 --2- 172.21.15.1:0/3384825336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558fbc36c7e0 0x558fbc36bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:21.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.121+0000 7f93e3e00700 1 -- 172.21.15.1:0/3384825336 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fbc361930 msgr2=0x558fbc36ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:21.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.121+0000 7f93e3e00700 1 --2- 172.21.15.1:0/3384825336 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fbc361930 0x558fbc36ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:21.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.121+0000 7f93e3e00700 1 -- 172.21.15.1:0/3384825336 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558fbc278ea0 con 0x558fbc361010 2024-02-02T15:31:21.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.122+0000 7f93e3e00700 1 --2- 172.21.15.1:0/3384825336 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558fbc361010 0x558fbc373370 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f93d4017dd0 tx=0x7f93d4018e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71177d4ab5e91348 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:21.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.122+0000 7f93d9e00700 1 -- 172.21.15.1:0/3384825336 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d4022970 con 0x558fbc361010 2024-02-02T15:31:21.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.122+0000 7f93d9e00700 1 -- 172.21.15.1:0/3384825336 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93d4022ad0 con 0x558fbc361010 2024-02-02T15:31:21.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.122+0000 7f93d9e00700 1 -- 172.21.15.1:0/3384825336 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d402a890 con 0x558fbc361010 2024-02-02T15:31:21.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.123+0000 7f93fe8e4b80 1 -- 172.21.15.1:0/3384825336 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558fbc361010 msgr2=0x558fbc373370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:21.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.123+0000 7f93fe8e4b80 1 --2- 172.21.15.1:0/3384825336 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558fbc361010 0x558fbc373370 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f93d4017dd0 tx=0x7f93d4018e00 comp rx=0 tx=0).stop 2024-02-02T15:31:21.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.123+0000 7f93fe8e4b80 1 -- 172.21.15.1:0/3384825336 shutdown_connections 2024-02-02T15:31:21.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.123+0000 7f93fe8e4b80 1 --2- 172.21.15.1:0/3384825336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558fbc36c7e0 0x558fbc36bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:21.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.123+0000 7f93fe8e4b80 1 --2- 172.21.15.1:0/3384825336 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558fbc361010 0x558fbc373370 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:21.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.123+0000 7f93fe8e4b80 1 --2- 172.21.15.1:0/3384825336 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fbc361930 0x558fbc36ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:21.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.123+0000 7f93fe8e4b80 1 -- 172.21.15.1:0/3384825336 >> 172.21.15.1:0/3384825336 conn(0x558fbc2c0db0 msgr2=0x558fbc377940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:21.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.123+0000 7f93fe8e4b80 1 -- 172.21.15.1:0/3384825336 shutdown_connections 2024-02-02T15:31:21.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.123+0000 7f93fe8e4b80 1 -- 172.21.15.1:0/3384825336 wait complete. 2024-02-02T15:31:21.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.123+0000 7f93fe8e4b80 1 -- start start 2024-02-02T15:31:21.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.123+0000 7f93fe8e4b80 1 Processor -- start 2024-02-02T15:31:21.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.124+0000 7f93fe8e4b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fbc361010 0x558fbc36f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:21.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.124+0000 7f93fe8e4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558fbc361930 0x558fbc36fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:21.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.124+0000 7f93fe8e4b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558fbc36c7e0 0x558fbc3ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:21.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.124+0000 7f93fe8e4b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fbc370180 con 0x558fbc361010 2024-02-02T15:31:21.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.124+0000 7f93fe8e4b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fbc3cd1d0 con 0x558fbc36c7e0 2024-02-02T15:31:21.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.124+0000 7f93fe8e4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fbc3cd340 con 0x558fbc361930 2024-02-02T15:31:21.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.124+0000 7f93fe8e4b80 10 client.0 _ll_register_callbacks cb 0x558fbc250290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:31:21.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.124+0000 7f93db200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558fbc361930 0x558fbc36fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:21.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.124+0000 7f93e8e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558fbc36c7e0 0x558fbc3ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:21.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.124+0000 7f93db200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558fbc361930 0x558fbc36fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:56422/0 (socket says 172.21.15.1:56422) 2024-02-02T15:31:21.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.124+0000 7f93e3e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fbc361010 0x558fbc36f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:21.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.124+0000 7f93db200700 1 -- 172.21.15.1:0/2728448968 learned_addr learned my addr 172.21.15.1:0/2728448968 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:31:21.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.125+0000 7f93db200700 1 -- 172.21.15.1:0/2728448968 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558fbc36c7e0 msgr2=0x558fbc3ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:21.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.125+0000 7f93db200700 1 --2- 172.21.15.1:0/2728448968 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558fbc36c7e0 0x558fbc3ccc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:21.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.125+0000 7f93db200700 1 -- 172.21.15.1:0/2728448968 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fbc361010 msgr2=0x558fbc36f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:21.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.125+0000 7f93db200700 1 --2- 172.21.15.1:0/2728448968 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fbc361010 0x558fbc36f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:21.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.125+0000 7f93db200700 1 -- 172.21.15.1:0/2728448968 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558fbc0e0b60 con 0x558fbc361930 2024-02-02T15:31:21.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.125+0000 7f93db200700 1 --2- 172.21.15.1:0/2728448968 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558fbc361930 0x558fbc36fa70 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f93cc098a30 tx=0x7f93cc010a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6eb209e31ac58bb server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:21.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.125+0000 7f93d2000700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:31:21.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.125+0000 7f93d2000700 1 -- 172.21.15.1:0/2728448968 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93cc010380 con 0x558fbc361930 2024-02-02T15:31:21.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.125+0000 7f93d2000700 1 -- 172.21.15.1:0/2728448968 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f93cc0970d0 con 0x558fbc361930 2024-02-02T15:31:21.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.125+0000 7f93fe8e4b80 1 -- 172.21.15.1:0/2728448968 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f93d40180d0 con 0x558fbc361930 2024-02-02T15:31:21.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.125+0000 7f93d2000700 1 -- 172.21.15.1:0/2728448968 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f93cc010610 con 0x558fbc361930 2024-02-02T15:31:21.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.125+0000 7f93fe8e4b80 10 client.26854 fetch_fsmap learned FSMap version 32 2024-02-02T15:31:21.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.125+0000 7f93fe8e4b80 1 -- 172.21.15.1:0/2728448968 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x558fbc138cd0 con 0x558fbc361930 2024-02-02T15:31:21.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.126+0000 7f93d2000700 1 -- 172.21.15.1:0/2728448968 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93d40180d0 con 0x558fbc361930 2024-02-02T15:31:21.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.126+0000 7f93d2000700 1 -- 172.21.15.1:0/2728448968 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 32) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f93cc098f90 con 0x558fbc361930 2024-02-02T15:31:21.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.126+0000 7f93fe8e4b80 10 client.26854 fetch_fsmap finished waiting for FSMap version 32 2024-02-02T15:31:21.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.126+0000 7f93fe8e4b80 10 client.26854 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:31:21.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.126+0000 7f93fe8e4b80 20 client.26854 populate_metadata read hostname 'smithi001' 2024-02-02T15:31:21.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.126+0000 7f93fe8e4b80 1 --2- 172.21.15.1:0/2728448968 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x558fbc3ceae0 0x558fbc3d0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:21.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.126+0000 7f93fe8e4b80 4 client.26854 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2024-02-02T15:31:21.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.126+0000 7f93fe8e4b80 1 -- 172.21.15.1:0/2728448968 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558fbc271f50 con 0x558fbc3ceae0 2024-02-02T15:31:21.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.126+0000 7f93e3e00700 1 --2- 172.21.15.1:0/2728448968 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x558fbc3ceae0 0x558fbc3d0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:21.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.127+0000 7f93d2000700 10 client.26854 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:31:21.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.127+0000 7f93e3e00700 1 --2- 172.21.15.1:0/2728448968 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x558fbc3ceae0 0x558fbc3d0f00 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f9ea7f333d0441a server_cookie=5dd1573f3e2a1db0 in_seq=0 out_seq=0 2024-02-02T15:31:21.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.127+0000 7f93d2000700 1 -- 172.21.15.1:0/2728448968 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x558fbc271f50 con 0x558fbc3ceae0 2024-02-02T15:31:21.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.127+0000 7f93d2000700 10 client.26854 handle_command_reply: tid=0 2024-02-02T15:31:21.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.127+0000 7f93fe8e4b80 1 client.26854 shutdown 2024-02-02T15:31:21.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.127+0000 7f93fe8e4b80 1 -- 172.21.15.1:0/2728448968 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558fbc361930 msgr2=0x558fbc36fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:21.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.127+0000 7f93fe8e4b80 1 --2- 172.21.15.1:0/2728448968 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558fbc361930 0x558fbc36fa70 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f93cc098a30 tx=0x7f93cc010a10 comp rx=0 tx=0).stop 2024-02-02T15:31:21.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.127+0000 7f93fe8e4b80 1 -- 172.21.15.1:0/2728448968 shutdown_connections 2024-02-02T15:31:21.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.127+0000 7f93fe8e4b80 1 --2- 172.21.15.1:0/2728448968 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x558fbc3ceae0 0x558fbc3d0f00 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:21.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.127+0000 7f93fe8e4b80 1 --2- 172.21.15.1:0/2728448968 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558fbc36c7e0 0x558fbc3ccc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:21.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.127+0000 7f93fe8e4b80 1 --2- 172.21.15.1:0/2728448968 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fbc361010 0x558fbc36f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:21.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.127+0000 7f93fe8e4b80 1 --2- 172.21.15.1:0/2728448968 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558fbc361930 0x558fbc36fa70 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:21.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.127+0000 7f93fe8e4b80 1 -- 172.21.15.1:0/2728448968 >> 172.21.15.1:0/2728448968 conn(0x558fbc2c0db0 msgr2=0x558fbc3646a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:21.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.127+0000 7f93d2000700 0 client.26854 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:31:21.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.127+0000 7f93fe8e4b80 1 -- 172.21.15.1:0/2728448968 shutdown_connections 2024-02-02T15:31:21.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.127+0000 7f93fe8e4b80 1 -- 172.21.15.1:0/2728448968 wait complete. 2024-02-02T15:31:21.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.128+0000 7f93fe8e4b80 20 client.26854 trim_cache size 0 max 16384 2024-02-02T15:31:21.132 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:31:21.132 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-02T15:31:21.132 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2024-02-02T15:31:21.132 INFO:teuthology.orchestra.run.smithi001.stdout: "e0eb4f2c-b2ed-4cf6-b4be-0b26da28ad77": { 2024-02-02T15:31:21.132 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2024-02-02T15:31:21.132 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "e0eb4f2c-b2ed-4cf6-b4be-0b26da28ad77", 2024-02-02T15:31:21.133 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,force" 2024-02-02T15:31:21.133 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:31:21.133 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:31:21.133 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:31:21.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.128+0000 7f93eae00700 1 -- 172.21.15.1:0/3086365961 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f93c40a0790 msgr2=0x7f93c40a2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:21.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.128+0000 7f93eae00700 1 --2- 172.21.15.1:0/3086365961 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f93c40a0790 0x7f93c40a2c50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f93d4009300 tx=0x7f93d400b040 comp rx=0 tx=0).stop 2024-02-02T15:31:21.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.128+0000 7f93eae00700 1 -- 172.21.15.1:0/3086365961 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93e4105020 msgr2=0x7f93e41981e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:21.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.128+0000 7f93eae00700 1 --2- 172.21.15.1:0/3086365961 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93e4105020 0x7f93e41981e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f93cc00b500 tx=0x7f93cc00b8c0 comp rx=0 tx=0).stop 2024-02-02T15:31:21.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.129+0000 7f93eae00700 1 -- 172.21.15.1:0/3086365961 shutdown_connections 2024-02-02T15:31:21.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.129+0000 7f93eae00700 1 --2- 172.21.15.1:0/3086365961 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f93c40a0790 0x7f93c40a2c50 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:21.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.129+0000 7f93eae00700 1 --2- 172.21.15.1:0/3086365961 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f93e4198880 0x7f93e41a2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:21.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.129+0000 7f93eae00700 1 --2- 172.21.15.1:0/3086365961 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93e41000e0 0x7f93e4197ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:21.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.129+0000 7f93eae00700 1 --2- 172.21.15.1:0/3086365961 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93e4105020 0x7f93e41981e0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:21.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.129+0000 7f93eae00700 1 -- 172.21.15.1:0/3086365961 >> 172.21.15.1:0/3086365961 conn(0x7f93e40757d0 msgr2=0x7f93e40fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:21.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.129+0000 7f93eae00700 1 -- 172.21.15.1:0/3086365961 shutdown_connections 2024-02-02T15:31:21.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:21.129+0000 7f93eae00700 1 -- 172.21.15.1:0/3086365961 wait complete. 2024-02-02T15:31:21.147 INFO:tasks.cephfs.filesystem:scrub status for tag:e0eb4f2c-b2ed-4cf6-b4be-0b26da28ad77 - {'path': '/', 'tag': 'e0eb4f2c-b2ed-4cf6-b4be-0b26da28ad77', 'options': 'recursive,force'} 2024-02-02T15:31:21.231 INFO:tasks.workunit.client.0.smithi001.stdout: 1 235326 9.29 MB/sec execute 55 sec latency 51.744 ms 2024-02-02T15:31:21.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:31:21 smithi001 ceph-mon[35470]: pgmap v133: 129 pgs: 129 active+clean; 144 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 227 op/s 2024-02-02T15:31:21.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:31:21 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:31:21.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:31:21 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:31:21.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:31:21 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:31:21.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:31:21 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:31:21.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:31:21 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:31:21.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:31:21 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:31:21.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:31:21 smithi142 ceph-mon[45728]: pgmap v133: 129 pgs: 129 active+clean; 144 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 227 op/s 2024-02-02T15:31:21.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:31:21 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:31:21.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:31:21 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:31:21.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:31:21 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:31:21.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:31:21 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:31:21.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:31:21 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:31:21.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:31:21 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:31:21.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:31:21 smithi071 ceph-mon[44707]: pgmap v133: 129 pgs: 129 active+clean; 144 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 227 op/s 2024-02-02T15:31:21.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:31:21 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:31:21.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:31:21 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:31:21.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:31:21 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:31:21.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:31:21 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:31:21.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:31:21 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:31:21.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:31:21 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:31:22.231 INFO:tasks.workunit.client.0.smithi001.stdout: 1 237771 9.41 MB/sec execute 56 sec latency 18.655 ms 2024-02-02T15:31:22.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:31:22 smithi142 ceph-mon[45728]: scrub queued for path: / 2024-02-02T15:31:22.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:31:22 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:31:22.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:31:22 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:31:22.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:31:22 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:31:22.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:31:22 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:31:22.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:31:22 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:31:22.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:31:22 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:31:22.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:31:22 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:31:22.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:31:22 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:31:22.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:31:22 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:31:22.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:31:22 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:31:22.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:31:22 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:31:22.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:31:22 smithi001 ceph-mon[35470]: scrub queued for path: / 2024-02-02T15:31:22.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:31:22 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:31:22.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:31:22 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:31:22.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:31:22 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:31:22.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:31:22 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:31:22.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:31:22 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:31:22.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:31:22 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:31:22.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:31:22 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:31:22.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:31:22 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:31:22.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:31:22 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:31:22.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:31:22 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:31:22.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:31:22 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:31:22.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:31:22 smithi071 ceph-mon[44707]: scrub queued for path: / 2024-02-02T15:31:22.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:31:22 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:31:22.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:31:22 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:31:22.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:31:22 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:31:22.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:31:22 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:31:22.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:31:22 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:31:22.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:31:22 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:31:22.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:31:22 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:31:22.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:31:22 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:31:22.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:31:22 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:31:22.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:31:22 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:31:22.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:31:22 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:31:23.231 INFO:tasks.workunit.client.0.smithi001.stdout: 1 238927 9.30 MB/sec execute 57 sec latency 11.899 ms 2024-02-02T15:31:23.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:31:23 smithi142 ceph-mon[45728]: pgmap v134: 129 pgs: 129 active+clean; 151 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.8 MiB/s wr, 343 op/s 2024-02-02T15:31:23.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:31:23 smithi142 ceph-mon[45728]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:31:23.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:31:23 smithi001 ceph-mon[35470]: pgmap v134: 129 pgs: 129 active+clean; 151 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.8 MiB/s wr, 343 op/s 2024-02-02T15:31:23.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:31:23 smithi001 ceph-mon[35470]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:31:23.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:31:23 smithi071 ceph-mon[44707]: pgmap v134: 129 pgs: 129 active+clean; 151 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.8 MiB/s wr, 343 op/s 2024-02-02T15:31:23.867 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:31:23 smithi071 ceph-mon[44707]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:31:24.231 INFO:tasks.workunit.client.0.smithi001.stdout: 1 241441 9.43 MB/sec execute 58 sec latency 11.175 ms 2024-02-02T15:31:25.231 INFO:tasks.workunit.client.0.smithi001.stdout: 1 242331 9.31 MB/sec execute 59 sec latency 55.629 ms 2024-02-02T15:31:25.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:31:25 smithi142 ceph-mon[45728]: pgmap v135: 129 pgs: 129 active+clean; 151 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 229 op/s 2024-02-02T15:31:25.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:31:25 smithi001 ceph-mon[35470]: pgmap v135: 129 pgs: 129 active+clean; 151 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 229 op/s 2024-02-02T15:31:25.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:31:25 smithi071 ceph-mon[44707]: pgmap v135: 129 pgs: 129 active+clean; 151 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 229 op/s 2024-02-02T15:31:26.231 INFO:tasks.workunit.client.0.smithi001.stdout: 1 244595 9.36 MB/sec execute 60 sec latency 33.735 ms 2024-02-02T15:31:26.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:31:26 smithi142 ceph-mon[45728]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:31:26.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:31:26 smithi001 ceph-mon[35470]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:31:26.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:31:26 smithi071 ceph-mon[44707]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:31:27.231 INFO:tasks.workunit.client.0.smithi001.stdout: 1 245577 9.29 MB/sec execute 61 sec latency 30.889 ms 2024-02-02T15:31:27.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:31:27 smithi142 ceph-mon[45728]: pgmap v136: 129 pgs: 129 active+clean; 154 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.5 MiB/s wr, 339 op/s 2024-02-02T15:31:27.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:31:27 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:31:27.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:31:27 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:31:27.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:31:27 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:31:27.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:31:27 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:31:27.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:31:27 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:31:27.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:31:27 smithi001 ceph-mon[35470]: pgmap v136: 129 pgs: 129 active+clean; 154 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.5 MiB/s wr, 339 op/s 2024-02-02T15:31:27.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:31:27 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:31:27.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:31:27 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:31:27.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:31:27 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:31:27.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:31:27 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:31:27.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:31:27 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:31:27.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:31:27 smithi071 ceph-mon[44707]: pgmap v136: 129 pgs: 129 active+clean; 154 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.5 MiB/s wr, 339 op/s 2024-02-02T15:31:27.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:31:27 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:31:27.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:31:27 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:31:27.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:31:27 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:31:27.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:31:27 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:31:27.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:31:27 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:31:28.231 INFO:tasks.workunit.client.0.smithi001.stdout: 1 247798 9.34 MB/sec execute 62 sec latency 11.524 ms 2024-02-02T15:31:28.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:31:28 smithi001 ceph-mon[35470]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:31:28.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:31:28 smithi071 ceph-mon[44707]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:31:28.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:31:28 smithi142 ceph-mon[45728]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:31:29.231 INFO:tasks.workunit.client.0.smithi001.stdout: 1 249040 9.30 MB/sec execute 63 sec latency 46.354 ms 2024-02-02T15:31:29.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:31:29 smithi001 ceph-mon[35470]: pgmap v137: 129 pgs: 129 active+clean; 154 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 225 op/s 2024-02-02T15:31:29.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:31:29 smithi071 ceph-mon[44707]: pgmap v137: 129 pgs: 129 active+clean; 154 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 225 op/s 2024-02-02T15:31:29.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:31:29 smithi142 ceph-mon[45728]: pgmap v137: 129 pgs: 129 active+clean; 154 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 225 op/s 2024-02-02T15:31:30.231 INFO:tasks.workunit.client.0.smithi001.stdout: 1 251243 9.35 MB/sec execute 64 sec latency 14.955 ms 2024-02-02T15:31:31.232 INFO:tasks.workunit.client.0.smithi001.stdout: 1 252437 9.30 MB/sec execute 65 sec latency 62.422 ms 2024-02-02T15:31:31.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:31:31 smithi001 ceph-mon[35470]: pgmap v138: 129 pgs: 129 active+clean; 154 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 225 op/s 2024-02-02T15:31:31.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:31:31 smithi071 ceph-mon[44707]: pgmap v138: 129 pgs: 129 active+clean; 154 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 225 op/s 2024-02-02T15:31:31.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:31:31 smithi142 ceph-mon[45728]: pgmap v138: 129 pgs: 129 active+clean; 154 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 225 op/s 2024-02-02T15:31:32.231 INFO:tasks.workunit.client.0.smithi001.stdout: 1 254480 9.28 MB/sec execute 66 sec latency 16.042 ms 2024-02-02T15:31:33.232 INFO:tasks.workunit.client.0.smithi001.stdout: 1 255928 9.31 MB/sec execute 67 sec latency 11.979 ms 2024-02-02T15:31:33.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:31:33 smithi001 ceph-mon[35470]: pgmap v139: 129 pgs: 129 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 334 op/s 2024-02-02T15:31:33.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:31:33 smithi071 ceph-mon[44707]: pgmap v139: 129 pgs: 129 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 334 op/s 2024-02-02T15:31:33.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:31:33 smithi142 ceph-mon[45728]: pgmap v139: 129 pgs: 129 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.4 MiB/s wr, 334 op/s 2024-02-02T15:31:34.232 INFO:tasks.workunit.client.0.smithi001.stdout: 1 257924 9.29 MB/sec execute 68 sec latency 13.607 ms 2024-02-02T15:31:35.232 INFO:tasks.workunit.client.0.smithi001.stdout: 1 259432 9.32 MB/sec execute 69 sec latency 10.915 ms 2024-02-02T15:31:35.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:31:35 smithi001 ceph-mon[35470]: pgmap v140: 129 pgs: 129 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 219 op/s 2024-02-02T15:31:35.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:31:35 smithi071 ceph-mon[44707]: pgmap v140: 129 pgs: 129 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 219 op/s 2024-02-02T15:31:35.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:31:35 smithi142 ceph-mon[45728]: pgmap v140: 129 pgs: 129 active+clean; 157 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 219 op/s 2024-02-02T15:31:36.232 INFO:tasks.workunit.client.0.smithi001.stdout: 1 261364 9.29 MB/sec execute 70 sec latency 26.144 ms 2024-02-02T15:31:37.232 INFO:tasks.workunit.client.0.smithi001.stdout: 1 262878 9.32 MB/sec execute 71 sec latency 22.315 ms 2024-02-02T15:31:37.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:31:37 smithi001 ceph-mon[35470]: pgmap v141: 129 pgs: 129 active+clean; 165 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.5 MiB/s wr, 330 op/s 2024-02-02T15:31:37.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:31:37 smithi071 ceph-mon[44707]: pgmap v141: 129 pgs: 129 active+clean; 165 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.5 MiB/s wr, 330 op/s 2024-02-02T15:31:37.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:31:37 smithi142 ceph-mon[45728]: pgmap v141: 129 pgs: 129 active+clean; 165 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.5 MiB/s wr, 330 op/s 2024-02-02T15:31:38.232 INFO:tasks.workunit.client.0.smithi001.stdout: 1 264387 9.27 MB/sec execute 72 sec latency 15.690 ms 2024-02-02T15:31:39.232 INFO:tasks.workunit.client.0.smithi001.stdout: 1 266364 9.33 MB/sec execute 73 sec latency 17.488 ms 2024-02-02T15:31:39.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:31:39 smithi001 ceph-mon[35470]: pgmap v142: 129 pgs: 129 active+clean; 165 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 220 op/s 2024-02-02T15:31:39.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:31:39 smithi071 ceph-mon[44707]: pgmap v142: 129 pgs: 129 active+clean; 165 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 220 op/s 2024-02-02T15:31:39.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:31:39 smithi142 ceph-mon[45728]: pgmap v142: 129 pgs: 129 active+clean; 165 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 220 op/s 2024-02-02T15:31:40.232 INFO:tasks.workunit.client.0.smithi001.stdout: 1 268064 9.28 MB/sec execute 74 sec latency 11.107 ms 2024-02-02T15:31:41.233 INFO:tasks.workunit.client.0.smithi001.stdout: 1 269886 9.33 MB/sec execute 75 sec latency 49.727 ms 2024-02-02T15:31:41.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:31:41 smithi071 ceph-mon[44707]: pgmap v143: 129 pgs: 129 active+clean; 165 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 220 op/s 2024-02-02T15:31:41.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:31:41 smithi142 ceph-mon[45728]: pgmap v143: 129 pgs: 129 active+clean; 165 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 220 op/s 2024-02-02T15:31:41.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:31:41 smithi001 ceph-mon[35470]: pgmap v143: 129 pgs: 129 active+clean; 165 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.4 MiB/s wr, 220 op/s 2024-02-02T15:31:42.232 INFO:tasks.workunit.client.0.smithi001.stdout: 1 271002 9.28 MB/sec execute 76 sec latency 52.409 ms 2024-02-02T15:31:43.232 INFO:tasks.workunit.client.0.smithi001.stdout: 1 273346 9.34 MB/sec execute 77 sec latency 24.231 ms 2024-02-02T15:31:43.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:31:43 smithi071 ceph-mon[44707]: pgmap v144: 129 pgs: 129 active+clean; 162 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.0 MiB/s wr, 325 op/s 2024-02-02T15:31:43.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:31:43 smithi142 ceph-mon[45728]: pgmap v144: 129 pgs: 129 active+clean; 162 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.0 MiB/s wr, 325 op/s 2024-02-02T15:31:43.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:31:43 smithi001 ceph-mon[35470]: pgmap v144: 129 pgs: 129 active+clean; 162 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.0 MiB/s wr, 325 op/s 2024-02-02T15:31:44.232 INFO:tasks.workunit.client.0.smithi001.stdout: 1 274369 9.25 MB/sec execute 78 sec latency 13.265 ms 2024-02-02T15:31:45.232 INFO:tasks.workunit.client.0.smithi001.stdout: 1 276834 9.33 MB/sec execute 79 sec latency 10.534 ms 2024-02-02T15:31:45.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:31:45 smithi071 ceph-mon[44707]: pgmap v145: 129 pgs: 129 active+clean; 162 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 216 op/s 2024-02-02T15:31:45.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:31:45 smithi142 ceph-mon[45728]: pgmap v145: 129 pgs: 129 active+clean; 162 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 216 op/s 2024-02-02T15:31:45.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:31:45 smithi001 ceph-mon[35470]: pgmap v145: 129 pgs: 129 active+clean; 162 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 216 op/s 2024-02-02T15:31:46.232 INFO:tasks.workunit.client.0.smithi001.stdout: 1 277873 9.26 MB/sec execute 80 sec latency 11.485 ms 2024-02-02T15:31:47.232 INFO:tasks.workunit.client.0.smithi001.stdout: 1 280239 9.33 MB/sec execute 81 sec latency 21.295 ms 2024-02-02T15:31:47.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:31:47 smithi071 ceph-mon[44707]: pgmap v146: 129 pgs: 129 active+clean; 167 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.1 MiB/s wr, 332 op/s 2024-02-02T15:31:47.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:31:47 smithi142 ceph-mon[45728]: pgmap v146: 129 pgs: 129 active+clean; 167 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.1 MiB/s wr, 332 op/s 2024-02-02T15:31:47.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:31:47 smithi001 ceph-mon[35470]: pgmap v146: 129 pgs: 129 active+clean; 167 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.1 MiB/s wr, 332 op/s 2024-02-02T15:31:48.233 INFO:tasks.workunit.client.0.smithi001.stdout: 1 281292 9.26 MB/sec execute 82 sec latency 22.317 ms 2024-02-02T15:31:49.232 INFO:tasks.workunit.client.0.smithi001.stdout: 1 283797 9.34 MB/sec execute 83 sec latency 11.496 ms 2024-02-02T15:31:49.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:31:49 smithi071 ceph-mon[44707]: pgmap v147: 129 pgs: 129 active+clean; 167 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 220 op/s 2024-02-02T15:31:49.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:31:49 smithi142 ceph-mon[45728]: pgmap v147: 129 pgs: 129 active+clean; 167 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 220 op/s 2024-02-02T15:31:49.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:31:49 smithi001 ceph-mon[35470]: pgmap v147: 129 pgs: 129 active+clean; 167 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 220 op/s 2024-02-02T15:31:50.233 INFO:tasks.workunit.client.0.smithi001.stdout: 1 284774 9.26 MB/sec execute 84 sec latency 14.024 ms 2024-02-02T15:31:51.148 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-02T15:31:51.233 INFO:tasks.workunit.client.0.smithi001.stdout: 1 287337 9.34 MB/sec execute 85 sec latency 12.079 ms 2024-02-02T15:31:51.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.360+0000 7fe14d800700 1 -- 172.21.15.1:0/3338075255 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe1480730e0 msgr2=0x7fe1480734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:51.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.360+0000 7fe14d800700 1 --2- 172.21.15.1:0/3338075255 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe1480730e0 0x7fe1480734c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe138009990 tx=0x7fe138009ca0 comp rx=0 tx=0).stop 2024-02-02T15:31:51.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.360+0000 7fe14d800700 1 -- 172.21.15.1:0/3338075255 shutdown_connections 2024-02-02T15:31:51.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.360+0000 7fe14d800700 1 --2- 172.21.15.1:0/3338075255 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe148111da0 0x7fe148114190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:51.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.360+0000 7fe14d800700 1 --2- 172.21.15.1:0/3338075255 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe1480730e0 0x7fe1480734c0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:51.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.360+0000 7fe14d800700 1 --2- 172.21.15.1:0/3338075255 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe148073a00 0x7fe148111860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:51.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.361+0000 7fe14d800700 1 -- 172.21.15.1:0/3338075255 >> 172.21.15.1:0/3338075255 conn(0x7fe1480fc860 msgr2=0x7fe1480fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:51.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.361+0000 7fe14d800700 1 -- 172.21.15.1:0/3338075255 shutdown_connections 2024-02-02T15:31:51.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.361+0000 7fe14d800700 1 -- 172.21.15.1:0/3338075255 wait complete. 2024-02-02T15:31:51.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.362+0000 7fe14d800700 1 Processor -- start 2024-02-02T15:31:51.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.362+0000 7fe14d800700 1 -- start start 2024-02-02T15:31:51.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.362+0000 7fe14d800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe1480730e0 0x7fe14819d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:51.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.362+0000 7fe14d800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe148073a00 0x7fe14819dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:51.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.362+0000 7fe14d800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe148111da0 0x7fe1481a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:51.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.362+0000 7fe14d800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1481a2550 con 0x7fe1480730e0 2024-02-02T15:31:51.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.362+0000 7fe14d800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1481a26c0 con 0x7fe148073a00 2024-02-02T15:31:51.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.362+0000 7fe14d800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1481a2880 con 0x7fe148111da0 2024-02-02T15:31:51.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.362+0000 7fe146000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe148073a00 0x7fe14819dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:51.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.362+0000 7fe146a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe1480730e0 0x7fe14819d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:51.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.362+0000 7fe146000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe148073a00 0x7fe14819dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:58688/0 (socket says 172.21.15.1:58688) 2024-02-02T15:31:51.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.362+0000 7fe146000700 1 -- 172.21.15.1:0/3265678660 learned_addr learned my addr 172.21.15.1:0/3265678660 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:31:51.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.363+0000 7fe147400700 1 --2- 172.21.15.1:0/3265678660 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe148111da0 0x7fe1481a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:51.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.363+0000 7fe146000700 1 -- 172.21.15.1:0/3265678660 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe148111da0 msgr2=0x7fe1481a1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:51.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.363+0000 7fe146000700 1 --2- 172.21.15.1:0/3265678660 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe148111da0 0x7fe1481a1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:51.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.363+0000 7fe146000700 1 -- 172.21.15.1:0/3265678660 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe1480730e0 msgr2=0x7fe14819d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:51.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.363+0000 7fe146000700 1 --2- 172.21.15.1:0/3265678660 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe1480730e0 0x7fe14819d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:51.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.363+0000 7fe146000700 1 -- 172.21.15.1:0/3265678660 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe140009c40 con 0x7fe148073a00 2024-02-02T15:31:51.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.363+0000 7fe146000700 1 --2- 172.21.15.1:0/3265678660 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe148073a00 0x7fe14819dab0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe13000e9a0 tx=0x7fe13000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:51.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.363+0000 7fe13f400700 1 -- 172.21.15.1:0/3265678660 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe130009790 con 0x7fe148073a00 2024-02-02T15:31:51.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.363+0000 7fe13f400700 1 -- 172.21.15.1:0/3265678660 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe13000ce80 con 0x7fe148073a00 2024-02-02T15:31:51.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.363+0000 7fe14d800700 1 -- 172.21.15.1:0/3265678660 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1380095f0 con 0x7fe148073a00 2024-02-02T15:31:51.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.364+0000 7fe13f400700 1 -- 172.21.15.1:0/3265678660 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe130018680 con 0x7fe148073a00 2024-02-02T15:31:51.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.364+0000 7fe14d800700 1 -- 172.21.15.1:0/3265678660 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1481a2d50 con 0x7fe148073a00 2024-02-02T15:31:51.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.367+0000 7fe13f400700 1 -- 172.21.15.1:0/3265678660 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 27) v1 ==== 135200+0+0 (secure 0 0 0) 0x7fe130010430 con 0x7fe148073a00 2024-02-02T15:31:51.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.368+0000 7fe16141eb80 1 Processor -- start 2024-02-02T15:31:51.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.369+0000 7fe13f400700 1 --2- 172.21.15.1:0/3265678660 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fe1280a0710 0x7fe1280a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:51.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.369+0000 7fe16141eb80 1 -- start start 2024-02-02T15:31:51.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.369+0000 7fe13f400700 1 -- 172.21.15.1:0/3265678660 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fe130014070 con 0x7fe148073a00 2024-02-02T15:31:51.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.369+0000 7fe147400700 1 --2- 172.21.15.1:0/3265678660 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fe1280a0710 0x7fe1280a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:51.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.369+0000 7fe16141eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e393ba010 0x561e393ba3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:51.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.369+0000 7fe16141eb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561e393ba930 0x561e393c52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:51.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.369+0000 7fe16141eb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561e393c57e0 0x561e393c7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:51.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.369+0000 7fe16141eb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e3932e4c0 con 0x561e393ba930 2024-02-02T15:31:51.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.369+0000 7fe16141eb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e39312aa0 con 0x561e393c57e0 2024-02-02T15:31:51.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.369+0000 7fe16141eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e39310460 con 0x561e393ba010 2024-02-02T15:31:51.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.369+0000 7fe146a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e393ba010 0x561e393ba3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:51.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.369+0000 7fe146000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561e393ba930 0x561e393c52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:51.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.369+0000 7fe146a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e393ba010 0x561e393ba3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:43548/0 (socket says 172.21.15.1:43548) 2024-02-02T15:31:51.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.369+0000 7fe146a00700 1 -- 172.21.15.1:0/3227273250 learned_addr learned my addr 172.21.15.1:0/3227273250 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:31:51.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.370+0000 7fe147400700 1 --2- 172.21.15.1:0/3227273250 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561e393c57e0 0x561e393c7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:51.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.370+0000 7fe146a00700 1 -- 172.21.15.1:0/3227273250 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561e393c57e0 msgr2=0x561e393c7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:51.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.370+0000 7fe146a00700 1 --2- 172.21.15.1:0/3227273250 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561e393c57e0 0x561e393c7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:51.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.370+0000 7fe146a00700 1 -- 172.21.15.1:0/3227273250 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561e393ba930 msgr2=0x561e393c52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:51.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.370+0000 7fe146a00700 1 --2- 172.21.15.1:0/3227273250 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561e393ba930 0x561e393c52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:51.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.370+0000 7fe146a00700 1 -- 172.21.15.1:0/3227273250 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e3918d3b0 con 0x561e393ba010 2024-02-02T15:31:51.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.370+0000 7fe147400700 1 --2- 172.21.15.1:0/3227273250 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561e393c57e0 0x561e393c7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:31:51.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.370+0000 7fe147400700 1 --2- 172.21.15.1:0/3265678660 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fe1280a0710 0x7fe1280a2bd0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe140009fd0 tx=0x7fe14000b040 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:51.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.370+0000 7fe146a00700 1 --2- 172.21.15.1:0/3227273250 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e393ba010 0x561e393ba3f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe138005ed0 tx=0x7fe138005f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67bd3cc1d2cc4716 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:51.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.370+0000 7fe137e00700 1 -- 172.21.15.1:0/3227273250 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13801d030 con 0x561e393ba010 2024-02-02T15:31:51.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.370+0000 7fe137e00700 1 -- 172.21.15.1:0/3227273250 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe13801e040 con 0x561e393ba010 2024-02-02T15:31:51.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.370+0000 7fe137e00700 1 -- 172.21.15.1:0/3227273250 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1380049c0 con 0x561e393ba010 2024-02-02T15:31:51.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.371+0000 7fe16141eb80 1 -- 172.21.15.1:0/3227273250 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e393ba010 msgr2=0x561e393ba3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:51.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.371+0000 7fe16141eb80 1 --2- 172.21.15.1:0/3227273250 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e393ba010 0x561e393ba3f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe138005ed0 tx=0x7fe138005f00 comp rx=0 tx=0).stop 2024-02-02T15:31:51.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.371+0000 7fe16141eb80 1 -- 172.21.15.1:0/3227273250 shutdown_connections 2024-02-02T15:31:51.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.371+0000 7fe16141eb80 1 --2- 172.21.15.1:0/3227273250 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561e393c57e0 0x561e393c7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:51.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.371+0000 7fe16141eb80 1 --2- 172.21.15.1:0/3227273250 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e393ba010 0x561e393ba3f0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:51.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.371+0000 7fe16141eb80 1 --2- 172.21.15.1:0/3227273250 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561e393ba930 0x561e393c52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:51.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.371+0000 7fe16141eb80 1 -- 172.21.15.1:0/3227273250 >> 172.21.15.1:0/3227273250 conn(0x561e39319db0 msgr2=0x561e393b63b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:51.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.371+0000 7fe16141eb80 1 -- 172.21.15.1:0/3227273250 shutdown_connections 2024-02-02T15:31:51.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.371+0000 7fe16141eb80 1 -- 172.21.15.1:0/3227273250 wait complete. 2024-02-02T15:31:51.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.372+0000 7fe16141eb80 1 -- start start 2024-02-02T15:31:51.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.372+0000 7fe16141eb80 1 Processor -- start 2024-02-02T15:31:51.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.373+0000 7fe16141eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e393ba010 0x561e393c5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:51.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.373+0000 7fe16141eb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561e393ba930 0x561e393c24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:51.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.373+0000 7fe16141eb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561e393c57e0 0x561e393c2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:51.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.373+0000 7fe16141eb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e3930bd50 con 0x561e393ba930 2024-02-02T15:31:51.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.373+0000 7fe16141eb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e392a7d80 con 0x561e393c57e0 2024-02-02T15:31:51.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.373+0000 7fe16141eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e392a7a20 con 0x561e393ba010 2024-02-02T15:31:51.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.373+0000 7fe16141eb80 10 client.0 _ll_register_callbacks cb 0x561e392a9290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:31:51.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.373+0000 7fe146a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e393ba010 0x561e393c5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:51.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.373+0000 7fe146a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e393ba010 0x561e393c5250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:43550/0 (socket says 172.21.15.1:43550) 2024-02-02T15:31:51.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.373+0000 7fe146a00700 1 -- 172.21.15.1:0/1740316783 learned_addr learned my addr 172.21.15.1:0/1740316783 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:31:51.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.373+0000 7fe147400700 1 --2- 172.21.15.1:0/1740316783 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561e393c57e0 0x561e393c2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:51.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.373+0000 7fe146000700 1 --2- 172.21.15.1:0/1740316783 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561e393ba930 0x561e393c24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:51.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.373+0000 7fe146a00700 1 -- 172.21.15.1:0/1740316783 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561e393c57e0 msgr2=0x561e393c2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:51.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.373+0000 7fe146a00700 1 --2- 172.21.15.1:0/1740316783 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561e393c57e0 0x561e393c2bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:51.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.373+0000 7fe146a00700 1 -- 172.21.15.1:0/1740316783 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561e393ba930 msgr2=0x561e393c24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:51.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.373+0000 7fe146a00700 1 --2- 172.21.15.1:0/1740316783 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561e393ba930 0x561e393c24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:51.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.373+0000 7fe146a00700 1 -- 172.21.15.1:0/1740316783 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe148114b40 con 0x561e393ba010 2024-02-02T15:31:51.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.373+0000 7fe147400700 1 --2- 172.21.15.1:0/1740316783 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561e393c57e0 0x561e393c2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:31:51.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.373+0000 7fe146000700 1 --2- 172.21.15.1:0/1740316783 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561e393ba930 0x561e393c24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:31:51.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.374+0000 7fe146a00700 1 --2- 172.21.15.1:0/1740316783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e393ba010 0x561e393c5250 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe138005850 tx=0x7fe138005900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4e5549504956867 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:51.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.374+0000 7fe134c00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:31:51.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.374+0000 7fe134c00700 1 -- 172.21.15.1:0/1740316783 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13801d070 con 0x561e393ba010 2024-02-02T15:31:51.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.374+0000 7fe134c00700 1 -- 172.21.15.1:0/1740316783 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fe1380047a0 con 0x561e393ba010 2024-02-02T15:31:51.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.374+0000 7fe16141eb80 1 -- 172.21.15.1:0/1740316783 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561e393ae0f0 con 0x561e393ba010 2024-02-02T15:31:51.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.374+0000 7fe134c00700 1 -- 172.21.15.1:0/1740316783 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe138005bc0 con 0x561e393ba010 2024-02-02T15:31:51.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.374+0000 7fe16141eb80 10 client.26866 fetch_fsmap learned FSMap version 35 2024-02-02T15:31:51.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.374+0000 7fe16141eb80 1 -- 172.21.15.1:0/1740316783 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x561e39139b60 con 0x561e393ba010 2024-02-02T15:31:51.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.374+0000 7fe134c00700 1 -- 172.21.15.1:0/1740316783 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561e393ae0f0 con 0x561e393ba010 2024-02-02T15:31:51.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.375+0000 7fe134c00700 1 -- 172.21.15.1:0/1740316783 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 35) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe138023b90 con 0x561e393ba010 2024-02-02T15:31:51.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.375+0000 7fe16141eb80 10 client.26866 fetch_fsmap finished waiting for FSMap version 35 2024-02-02T15:31:51.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.375+0000 7fe16141eb80 10 client.26866 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:31:51.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.375+0000 7fe16141eb80 20 client.26866 populate_metadata read hostname 'smithi001' 2024-02-02T15:31:51.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.375+0000 7fe16141eb80 1 --2- 172.21.15.1:0/1740316783 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x561e393d4c60 0x561e393d7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:51.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.375+0000 7fe16141eb80 4 client.26866 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:31:51.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.375+0000 7fe16141eb80 1 -- 172.21.15.1:0/1740316783 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561e39304dd0 con 0x561e393d4c60 2024-02-02T15:31:51.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.375+0000 7fe146000700 1 --2- 172.21.15.1:0/1740316783 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x561e393d4c60 0x561e393d7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:51.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.375+0000 7fe146000700 1 --2- 172.21.15.1:0/1740316783 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x561e393d4c60 0x561e393d7050 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4a3c809c6b85b7f server_cookie=63c919ef40687aaf in_seq=0 out_seq=0 2024-02-02T15:31:51.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.375+0000 7fe134c00700 10 client.26866 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:31:51.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.376+0000 7fe134c00700 1 -- 172.21.15.1:0/1740316783 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x561e39304dd0 con 0x561e393d4c60 2024-02-02T15:31:51.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.376+0000 7fe134c00700 10 client.26866 handle_command_reply: tid=0 2024-02-02T15:31:51.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.376+0000 7fe16141eb80 1 client.26866 shutdown 2024-02-02T15:31:51.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.376+0000 7fe16141eb80 1 -- 172.21.15.1:0/1740316783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e393ba010 msgr2=0x561e393c5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:51.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.376+0000 7fe16141eb80 1 --2- 172.21.15.1:0/1740316783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e393ba010 0x561e393c5250 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe138005850 tx=0x7fe138005900 comp rx=0 tx=0).stop 2024-02-02T15:31:51.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.377+0000 7fe16141eb80 1 -- 172.21.15.1:0/1740316783 shutdown_connections 2024-02-02T15:31:51.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.377+0000 7fe16141eb80 1 --2- 172.21.15.1:0/1740316783 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x561e393d4c60 0x561e393d7050 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:51.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.377+0000 7fe16141eb80 1 --2- 172.21.15.1:0/1740316783 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561e393c57e0 0x561e393c2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:51.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.377+0000 7fe134c00700 0 client.26866 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:31:51.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.377+0000 7fe16141eb80 1 --2- 172.21.15.1:0/1740316783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e393ba010 0x561e393c5250 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:51.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.377+0000 7fe16141eb80 1 --2- 172.21.15.1:0/1740316783 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561e393ba930 0x561e393c24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:51.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.377+0000 7fe16141eb80 1 -- 172.21.15.1:0/1740316783 >> 172.21.15.1:0/1740316783 conn(0x561e39319db0 msgr2=0x561e393b5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:51.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.377+0000 7fe16141eb80 1 -- 172.21.15.1:0/1740316783 shutdown_connections 2024-02-02T15:31:51.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.377+0000 7fe16141eb80 1 -- 172.21.15.1:0/1740316783 wait complete. 2024-02-02T15:31:51.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.377+0000 7fe16141eb80 20 client.26866 trim_cache size 0 max 16384 2024-02-02T15:31:51.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.396+0000 7fe16141eb80 1 Processor -- start 2024-02-02T15:31:51.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.396+0000 7fe16141eb80 1 -- start start 2024-02-02T15:31:51.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.396+0000 7fe16141eb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561e393ba010 0x561e393bca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:51.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.396+0000 7fe16141eb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561e393ba930 0x561e393c4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:51.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.396+0000 7fe16141eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e393c57e0 0x561e393c1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:51.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.396+0000 7fe16141eb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e3918f6a0 con 0x561e393ba930 2024-02-02T15:31:51.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.396+0000 7fe16141eb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e393be320 con 0x561e393ba010 2024-02-02T15:31:51.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.396+0000 7fe16141eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e393c1a40 con 0x561e393c57e0 2024-02-02T15:31:51.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.397+0000 7fe146000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561e393ba930 0x561e393c4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:51.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.397+0000 7fe147400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e393c57e0 0x561e393c1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:51.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.397+0000 7fe146a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561e393ba010 0x561e393bca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:51.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.397+0000 7fe147400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e393c57e0 0x561e393c1d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:43566/0 (socket says 172.21.15.1:43566) 2024-02-02T15:31:51.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.397+0000 7fe146a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561e393ba010 0x561e393bca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:58708/0 (socket says 172.21.15.1:58708) 2024-02-02T15:31:51.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.397+0000 7fe147400700 1 -- 172.21.15.1:0/2686290459 learned_addr learned my addr 172.21.15.1:0/2686290459 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:31:51.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.397+0000 7fe146000700 1 -- 172.21.15.1:0/2686290459 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e393c57e0 msgr2=0x561e393c1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:51.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.397+0000 7fe146000700 1 --2- 172.21.15.1:0/2686290459 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e393c57e0 0x561e393c1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:51.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.397+0000 7fe146000700 1 -- 172.21.15.1:0/2686290459 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561e393ba010 msgr2=0x561e393bca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:51.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.397+0000 7fe146000700 1 --2- 172.21.15.1:0/2686290459 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561e393ba010 0x561e393bca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:51.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.397+0000 7fe146000700 1 -- 172.21.15.1:0/2686290459 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e392d1ea0 con 0x561e393ba930 2024-02-02T15:31:51.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.397+0000 7fe146000700 1 --2- 172.21.15.1:0/2686290459 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561e393ba930 0x561e393c4d80 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fe1300c95f0 tx=0x7fe1300cb040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b3a489b3200ed08 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:51.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.398+0000 7fe13cc00700 1 -- 172.21.15.1:0/2686290459 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1300993c0 con 0x561e393ba930 2024-02-02T15:31:51.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.398+0000 7fe13cc00700 1 -- 172.21.15.1:0/2686290459 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe1300aee90 con 0x561e393ba930 2024-02-02T15:31:51.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.398+0000 7fe13cc00700 1 -- 172.21.15.1:0/2686290459 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1300988b0 con 0x561e393ba930 2024-02-02T15:31:51.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.398+0000 7fe16141eb80 1 -- 172.21.15.1:0/2686290459 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561e393ba930 msgr2=0x561e393c4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:51.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.398+0000 7fe16141eb80 1 --2- 172.21.15.1:0/2686290459 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561e393ba930 0x561e393c4d80 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fe1300c95f0 tx=0x7fe1300cb040 comp rx=0 tx=0).stop 2024-02-02T15:31:51.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.398+0000 7fe16141eb80 1 -- 172.21.15.1:0/2686290459 shutdown_connections 2024-02-02T15:31:51.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.398+0000 7fe16141eb80 1 --2- 172.21.15.1:0/2686290459 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e393c57e0 0x561e393c1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:51.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.398+0000 7fe16141eb80 1 --2- 172.21.15.1:0/2686290459 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561e393ba010 0x561e393bca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:51.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.398+0000 7fe16141eb80 1 --2- 172.21.15.1:0/2686290459 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561e393ba930 0x561e393c4d80 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:51.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.398+0000 7fe16141eb80 1 -- 172.21.15.1:0/2686290459 >> 172.21.15.1:0/2686290459 conn(0x561e39319db0 msgr2=0x561e393bd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:51.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.398+0000 7fe16141eb80 1 -- 172.21.15.1:0/2686290459 shutdown_connections 2024-02-02T15:31:51.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.398+0000 7fe16141eb80 1 -- 172.21.15.1:0/2686290459 wait complete. 2024-02-02T15:31:51.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.399+0000 7fe16141eb80 1 -- start start 2024-02-02T15:31:51.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.399+0000 7fe16141eb80 1 Processor -- start 2024-02-02T15:31:51.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.400+0000 7fe16141eb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561e393ba010 0x561e393cbe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:51.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.400+0000 7fe16141eb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561e393ba930 0x561e393ca160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:51.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.400+0000 7fe16141eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e393c57e0 0x561e393ca6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:51.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.400+0000 7fe16141eb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e393cc740 con 0x561e393ba930 2024-02-02T15:31:51.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.400+0000 7fe16141eb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e393cabe0 con 0x561e393ba010 2024-02-02T15:31:51.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.400+0000 7fe16141eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e393cad50 con 0x561e393c57e0 2024-02-02T15:31:51.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.400+0000 7fe16141eb80 10 client.0 _ll_register_callbacks cb 0x561e392a9290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:31:51.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.400+0000 7fe146000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561e393ba930 0x561e393ca160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:51.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.400+0000 7fe146a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561e393ba010 0x561e393cbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:51.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.400+0000 7fe146000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561e393ba930 0x561e393ca160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35046/0 (socket says 172.21.15.1:35046) 2024-02-02T15:31:51.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.400+0000 7fe146000700 1 -- 172.21.15.1:0/2632998511 learned_addr learned my addr 172.21.15.1:0/2632998511 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:31:51.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.400+0000 7fe147400700 1 --2- 172.21.15.1:0/2632998511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e393c57e0 0x561e393ca6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:51.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.400+0000 7fe146a00700 1 -- 172.21.15.1:0/2632998511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e393c57e0 msgr2=0x561e393ca6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:51.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.400+0000 7fe146a00700 1 --2- 172.21.15.1:0/2632998511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e393c57e0 0x561e393ca6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:51.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.400+0000 7fe146a00700 1 -- 172.21.15.1:0/2632998511 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561e393ba930 msgr2=0x561e393ca160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:51.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.400+0000 7fe146a00700 1 --2- 172.21.15.1:0/2632998511 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561e393ba930 0x561e393ca160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:51.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.400+0000 7fe146a00700 1 -- 172.21.15.1:0/2632998511 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1481a2d50 con 0x561e393ba010 2024-02-02T15:31:51.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.400+0000 7fe147400700 1 --2- 172.21.15.1:0/2632998511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e393c57e0 0x561e393ca6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:31:51.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.401+0000 7fe146a00700 1 --2- 172.21.15.1:0/2632998511 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561e393ba010 0x561e393cbe70 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe13800b6f0 tx=0x7fe13800bef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5cb981c9bb44b96 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:51.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.401+0000 7fe134c00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:31:51.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.401+0000 7fe134c00700 1 -- 172.21.15.1:0/2632998511 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13801d070 con 0x561e393ba010 2024-02-02T15:31:51.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.401+0000 7fe134c00700 1 -- 172.21.15.1:0/2632998511 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fe138000e40 con 0x561e393ba010 2024-02-02T15:31:51.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.401+0000 7fe16141eb80 1 -- 172.21.15.1:0/2632998511 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe13801e040 con 0x561e393ba010 2024-02-02T15:31:51.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.401+0000 7fe134c00700 1 -- 172.21.15.1:0/2632998511 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe138005970 con 0x561e393ba010 2024-02-02T15:31:51.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.401+0000 7fe16141eb80 10 client.26873 fetch_fsmap learned FSMap version 35 2024-02-02T15:31:51.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.401+0000 7fe16141eb80 1 -- 172.21.15.1:0/2632998511 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x561e39191cd0 con 0x561e393ba010 2024-02-02T15:31:51.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.402+0000 7fe134c00700 1 -- 172.21.15.1:0/2632998511 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe13801e040 con 0x561e393ba010 2024-02-02T15:31:51.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.402+0000 7fe134c00700 1 -- 172.21.15.1:0/2632998511 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 35) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe13800b850 con 0x561e393ba010 2024-02-02T15:31:51.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.402+0000 7fe16141eb80 10 client.26873 fetch_fsmap finished waiting for FSMap version 35 2024-02-02T15:31:51.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.402+0000 7fe16141eb80 10 client.26873 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:31:51.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.402+0000 7fe16141eb80 20 client.26873 populate_metadata read hostname 'smithi001' 2024-02-02T15:31:51.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.402+0000 7fe16141eb80 1 --2- 172.21.15.1:0/2632998511 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x561e39427610 0x561e39429a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:51.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.402+0000 7fe16141eb80 4 client.26873 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2024-02-02T15:31:51.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.402+0000 7fe16141eb80 1 -- 172.21.15.1:0/2632998511 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561e392caf50 con 0x561e39427610 2024-02-02T15:31:51.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.403+0000 7fe146000700 1 --2- 172.21.15.1:0/2632998511 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x561e39427610 0x561e39429a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:51.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.403+0000 7fe146000700 1 --2- 172.21.15.1:0/2632998511 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x561e39427610 0x561e39429a00 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0fbc701ffa04f4f server_cookie=59edc36aa9bba0c6 in_seq=0 out_seq=0 2024-02-02T15:31:51.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.403+0000 7fe134c00700 10 client.26873 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:31:51.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.403+0000 7fe134c00700 1 -- 172.21.15.1:0/2632998511 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x561e392caf50 con 0x561e39427610 2024-02-02T15:31:51.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.403+0000 7fe134c00700 10 client.26873 handle_command_reply: tid=0 2024-02-02T15:31:51.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.403+0000 7fe16141eb80 1 client.26873 shutdown 2024-02-02T15:31:51.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.404+0000 7fe16141eb80 1 -- 172.21.15.1:0/2632998511 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561e393ba010 msgr2=0x561e393cbe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:51.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.404+0000 7fe16141eb80 1 --2- 172.21.15.1:0/2632998511 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561e393ba010 0x561e393cbe70 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe13800b6f0 tx=0x7fe13800bef0 comp rx=0 tx=0).stop 2024-02-02T15:31:51.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.404+0000 7fe16141eb80 1 -- 172.21.15.1:0/2632998511 shutdown_connections 2024-02-02T15:31:51.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.404+0000 7fe16141eb80 1 --2- 172.21.15.1:0/2632998511 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x561e39427610 0x561e39429a00 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:51.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.404+0000 7fe16141eb80 1 --2- 172.21.15.1:0/2632998511 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561e393c57e0 0x561e393ca6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:51.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.404+0000 7fe16141eb80 1 --2- 172.21.15.1:0/2632998511 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561e393ba010 0x561e393cbe70 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:51.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.404+0000 7fe134c00700 0 client.26873 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:31:51.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.404+0000 7fe16141eb80 1 --2- 172.21.15.1:0/2632998511 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561e393ba930 0x561e393ca160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:51.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.404+0000 7fe16141eb80 1 -- 172.21.15.1:0/2632998511 >> 172.21.15.1:0/2632998511 conn(0x561e39319db0 msgr2=0x561e393c1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:51.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.404+0000 7fe16141eb80 1 -- 172.21.15.1:0/2632998511 shutdown_connections 2024-02-02T15:31:51.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.404+0000 7fe16141eb80 1 -- 172.21.15.1:0/2632998511 wait complete. 2024-02-02T15:31:51.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.404+0000 7fe16141eb80 20 client.26873 trim_cache size 0 max 16384 2024-02-02T15:31:51.408 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:31:51.408 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2024-02-02T15:31:51.408 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2024-02-02T15:31:51.408 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:31:51.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.404+0000 7fe14d800700 1 -- 172.21.15.1:0/3265678660 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fe1280a0710 msgr2=0x7fe1280a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:51.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.405+0000 7fe14d800700 1 --2- 172.21.15.1:0/3265678660 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fe1280a0710 0x7fe1280a2bd0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe140009fd0 tx=0x7fe14000b040 comp rx=0 tx=0).stop 2024-02-02T15:31:51.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.405+0000 7fe14d800700 1 -- 172.21.15.1:0/3265678660 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe148073a00 msgr2=0x7fe14819dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:51.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.405+0000 7fe14d800700 1 --2- 172.21.15.1:0/3265678660 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe148073a00 0x7fe14819dab0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe13000e9a0 tx=0x7fe13000ecb0 comp rx=0 tx=0).stop 2024-02-02T15:31:51.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.405+0000 7fe14d800700 1 -- 172.21.15.1:0/3265678660 shutdown_connections 2024-02-02T15:31:51.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.405+0000 7fe14d800700 1 --2- 172.21.15.1:0/3265678660 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fe1280a0710 0x7fe1280a2bd0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:51.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.405+0000 7fe14d800700 1 --2- 172.21.15.1:0/3265678660 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe148111da0 0x7fe1481a1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:51.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.405+0000 7fe14d800700 1 --2- 172.21.15.1:0/3265678660 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe1480730e0 0x7fe14819d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:51.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.405+0000 7fe14d800700 1 --2- 172.21.15.1:0/3265678660 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe148073a00 0x7fe14819dab0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:51.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.405+0000 7fe14d800700 1 -- 172.21.15.1:0/3265678660 >> 172.21.15.1:0/3265678660 conn(0x7fe1480fc860 msgr2=0x7fe1480fd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:51.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.405+0000 7fe14d800700 1 -- 172.21.15.1:0/3265678660 shutdown_connections 2024-02-02T15:31:51.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.405+0000 7fe14d800700 1 -- 172.21.15.1:0/3265678660 wait complete. 2024-02-02T15:31:51.424 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-02T15:31:51.424 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T15:31:51.543 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:31:51 smithi001 ceph-mon[35470]: pgmap v148: 129 pgs: 129 active+clean; 167 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 220 op/s 2024-02-02T15:31:51.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.703+0000 7f8be4800700 1 -- 172.21.15.1:0/4000497718 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bdc100f10 msgr2=0x7f8bdc1012f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:51.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.703+0000 7f8be4800700 1 --2- 172.21.15.1:0/4000497718 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bdc100f10 0x7f8bdc1012f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc009990 tx=0x7f8bcc009ca0 comp rx=0 tx=0).stop 2024-02-02T15:31:51.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.703+0000 7f8be4800700 1 -- 172.21.15.1:0/4000497718 shutdown_connections 2024-02-02T15:31:51.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.703+0000 7f8be4800700 1 --2- 172.21.15.1:0/4000497718 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8bdc106030 0x7f8bdc10dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:51.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.703+0000 7f8be4800700 1 --2- 172.21.15.1:0/4000497718 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bdc100f10 0x7f8bdc1012f0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:51.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.703+0000 7f8be4800700 1 --2- 172.21.15.1:0/4000497718 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bdc1018c0 0x7f8bdc105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:51.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.703+0000 7f8be4800700 1 -- 172.21.15.1:0/4000497718 >> 172.21.15.1:0/4000497718 conn(0x7f8bdc0fc790 msgr2=0x7f8bdc0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:51.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.703+0000 7f8be4800700 1 -- 172.21.15.1:0/4000497718 shutdown_connections 2024-02-02T15:31:51.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.703+0000 7f8be4800700 1 -- 172.21.15.1:0/4000497718 wait complete. 2024-02-02T15:31:51.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.704+0000 7f8be4800700 1 Processor -- start 2024-02-02T15:31:51.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.705+0000 7f8be4800700 1 -- start start 2024-02-02T15:31:51.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.705+0000 7f8be4800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8bdc100f10 0x7f8bdc073040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:51.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.705+0000 7f8be4800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bdc106030 0x7f8bdc073580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:51.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.705+0000 7f8be4800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bdc073ac0 0x7f8bdc1a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:51.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.705+0000 7f8be4800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bdc1a56e0 con 0x7f8bdc073ac0 2024-02-02T15:31:51.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.705+0000 7f8be4800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bdc1a5850 con 0x7f8bdc100f10 2024-02-02T15:31:51.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.705+0000 7f8be4800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bdc1a5a10 con 0x7f8bdc106030 2024-02-02T15:31:51.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.705+0000 7f8be1e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8bdc100f10 0x7f8bdc073040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:51.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.705+0000 7f8be1e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8bdc100f10 0x7f8bdc073040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:58732/0 (socket says 172.21.15.1:58732) 2024-02-02T15:31:51.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.705+0000 7f8be1e00700 1 -- 172.21.15.1:0/273726625 learned_addr learned my addr 172.21.15.1:0/273726625 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:31:51.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.705+0000 7f8be1400700 1 --2- 172.21.15.1:0/273726625 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bdc106030 0x7f8bdc073580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:51.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.705+0000 7f8be2800700 1 --2- 172.21.15.1:0/273726625 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bdc073ac0 0x7f8bdc1a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:51.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.706+0000 7f8be1e00700 1 -- 172.21.15.1:0/273726625 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bdc106030 msgr2=0x7f8bdc073580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:51.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.706+0000 7f8be1e00700 1 --2- 172.21.15.1:0/273726625 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bdc106030 0x7f8bdc073580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:51.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.706+0000 7f8be1e00700 1 -- 172.21.15.1:0/273726625 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bdc073ac0 msgr2=0x7f8bdc1a5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:51.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.706+0000 7f8be1e00700 1 --2- 172.21.15.1:0/273726625 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bdc073ac0 0x7f8bdc1a5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:51.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.706+0000 7f8be1e00700 1 -- 172.21.15.1:0/273726625 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bcc0095f0 con 0x7f8bdc100f10 2024-02-02T15:31:51.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.706+0000 7f8be1e00700 1 --2- 172.21.15.1:0/273726625 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8bdc100f10 0x7f8bdc073040 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc005910 tx=0x7f8bcc004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:51.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.706+0000 7f8bdaa00700 1 -- 172.21.15.1:0/273726625 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc01d070 con 0x7f8bdc100f10 2024-02-02T15:31:51.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.706+0000 7f8bdaa00700 1 -- 172.21.15.1:0/273726625 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8bcc022470 con 0x7f8bdc100f10 2024-02-02T15:31:51.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.706+0000 7f8be4800700 1 -- 172.21.15.1:0/273726625 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bdc1a5c90 con 0x7f8bdc100f10 2024-02-02T15:31:51.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.706+0000 7f8bdaa00700 1 -- 172.21.15.1:0/273726625 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc017650 con 0x7f8bdc100f10 2024-02-02T15:31:51.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.706+0000 7f8be4800700 1 -- 172.21.15.1:0/273726625 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bdc1a60b0 con 0x7f8bdc100f10 2024-02-02T15:31:51.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.707+0000 7f8be4800700 1 -- 172.21.15.1:0/273726625 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bdc067bc0 con 0x7f8bdc100f10 2024-02-02T15:31:51.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.710+0000 7f8bdaa00700 1 -- 172.21.15.1:0/273726625 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 27) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f8bcc0225e0 con 0x7f8bdc100f10 2024-02-02T15:31:51.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.711+0000 7f8bdaa00700 1 --2- 172.21.15.1:0/273726625 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f8bbc0a0690 0x7f8bbc0a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:51.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.711+0000 7f8bdaa00700 1 -- 172.21.15.1:0/273726625 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f8bcc0cd4c0 con 0x7f8bdc100f10 2024-02-02T15:31:51.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.711+0000 7f8be1400700 1 --2- 172.21.15.1:0/273726625 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f8bbc0a0690 0x7f8bbc0a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:51.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.712+0000 7f8be1400700 1 --2- 172.21.15.1:0/273726625 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f8bbc0a0690 0x7f8bbc0a2b50 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc074830 tx=0x7f8bc4009260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:51.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.713+0000 7f8bdaa00700 1 -- 172.21.15.1:0/273726625 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f8bcc096a00 con 0x7f8bdc100f10 2024-02-02T15:31:51.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:31:51 smithi071 ceph-mon[44707]: pgmap v148: 129 pgs: 129 active+clean; 167 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 220 op/s 2024-02-02T15:31:51.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:31:51 smithi142 ceph-mon[45728]: pgmap v148: 129 pgs: 129 active+clean; 167 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 220 op/s 2024-02-02T15:31:51.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.990+0000 7f8be4800700 1 -- 172.21.15.1:0/273726625 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8bdc074570 con 0x7f8bdc100f10 2024-02-02T15:31:51.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.991+0000 7f8bdaa00700 1 -- 172.21.15.1:0/273726625 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+10866 (secure 0 0 0) 0x7f8bcc09a020 con 0x7f8bdc100f10 2024-02-02T15:31:51.991 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:31:51.991 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2922240448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2922240448},{"type":"v1","addr":"172.21.15.1:6835","nonce":2922240448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2729497431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2729497431},{"type":"v1","addr":"172.21.15.1:6837","nonce":2729497431}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-02T15:24:47.761435+0000","modified":"2024-02-02T15:31:27.413833+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24529,"mds_2":24518,"mds_3":24514,"mds_4":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14517":{"gid":14517,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/1218303683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":1218303683},{"type":"v1","addr":"172.21.15.1:6839","nonce":1218303683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14529":{"gid":14529,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/2059197496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2059197496},{"type":"v1","addr":"172.21.15.1:6841","nonce":2059197496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/7648986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":7648986},{"type":"v1","addr":"172.21.15.142:6833","nonce":7648986}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/3482746094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3482746094},{"type":"v1","addr":"172.21.15.71:6833","nonce":3482746094}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/388207814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":388207814},{"type":"v1","addr":"172.21.15.71:6835","nonce":388207814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6835/1052606547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1052606547},{"type":"v1","addr":"172.21.15.142:6835","nonce":1052606547}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.142:6837/2870289019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2870289019},{"type":"v1","addr":"172.21.15.142:6837","nonce":2870289019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.71:6837/628016199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":628016199},{"type":"v1","addr":"172.21.15.71:6837","nonce":628016199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.142:6839/321675421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":321675421},{"type":"v1","addr":"172.21.15.142:6839","nonce":321675421}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/1268086493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1268086493},{"type":"v1","addr":"172.21.15.71:6839","nonce":1268086493}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-02T15:31:51.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.993+0000 7f8be4800700 1 -- 172.21.15.1:0/273726625 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f8bbc0a0690 msgr2=0x7f8bbc0a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:51.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.993+0000 7f8be4800700 1 --2- 172.21.15.1:0/273726625 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f8bbc0a0690 0x7f8bbc0a2b50 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc074830 tx=0x7f8bc4009260 comp rx=0 tx=0).stop 2024-02-02T15:31:51.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.993+0000 7f8be4800700 1 -- 172.21.15.1:0/273726625 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8bdc100f10 msgr2=0x7f8bdc073040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:51.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.993+0000 7f8be4800700 1 --2- 172.21.15.1:0/273726625 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8bdc100f10 0x7f8bdc073040 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc005910 tx=0x7f8bcc004730 comp rx=0 tx=0).stop 2024-02-02T15:31:51.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.993+0000 7f8be4800700 1 -- 172.21.15.1:0/273726625 shutdown_connections 2024-02-02T15:31:51.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.993+0000 7f8be4800700 1 --2- 172.21.15.1:0/273726625 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f8bbc0a0690 0x7f8bbc0a2b50 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:51.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.993+0000 7f8be4800700 1 --2- 172.21.15.1:0/273726625 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8bdc073ac0 0x7f8bdc1a5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:51.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.994+0000 7f8be4800700 1 --2- 172.21.15.1:0/273726625 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8bdc100f10 0x7f8bdc073040 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:51.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.994+0000 7f8be4800700 1 --2- 172.21.15.1:0/273726625 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8bdc106030 0x7f8bdc073580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:51.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.994+0000 7f8be4800700 1 -- 172.21.15.1:0/273726625 >> 172.21.15.1:0/273726625 conn(0x7f8bdc0fc790 msgr2=0x7f8bdc104940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:51.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.994+0000 7f8be4800700 1 -- 172.21.15.1:0/273726625 shutdown_connections 2024-02-02T15:31:51.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:51.994+0000 7f8be4800700 1 -- 172.21.15.1:0/273726625 wait complete. 2024-02-02T15:31:51.995 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 35 2024-02-02T15:31:52.013 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-02-02T15:31:52.233 INFO:tasks.workunit.client.0.smithi001.stdout: 1 288206 9.26 MB/sec execute 86 sec latency 55.525 ms 2024-02-02T15:31:52.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.254+0000 7f4954000700 1 -- 172.21.15.1:0/1557931520 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f494c1019a0 msgr2=0x7f494c1059e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:52.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.254+0000 7f4954000700 1 --2- 172.21.15.1:0/1557931520 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f494c1019a0 0x7f494c1059e0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f4934009990 tx=0x7f4934009ca0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.254+0000 7f4954000700 1 -- 172.21.15.1:0/1557931520 shutdown_connections 2024-02-02T15:31:52.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.254+0000 7f4954000700 1 --2- 172.21.15.1:0/1557931520 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f494c106110 0x7f494c10dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.254+0000 7f4954000700 1 --2- 172.21.15.1:0/1557931520 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f494c100ff0 0x7f494c1013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.254+0000 7f4954000700 1 --2- 172.21.15.1:0/1557931520 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f494c1019a0 0x7f494c1059e0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.254+0000 7f4954000700 1 -- 172.21.15.1:0/1557931520 >> 172.21.15.1:0/1557931520 conn(0x7f494c0fc810 msgr2=0x7f494c0fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:52.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.254+0000 7f4954000700 1 -- 172.21.15.1:0/1557931520 shutdown_connections 2024-02-02T15:31:52.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.254+0000 7f4954000700 1 -- 172.21.15.1:0/1557931520 wait complete. 2024-02-02T15:31:52.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.255+0000 7f4954000700 1 Processor -- start 2024-02-02T15:31:52.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.255+0000 7f4954000700 1 -- start start 2024-02-02T15:31:52.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.256+0000 7f4954000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f494c100ff0 0x7f494c073040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:52.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.256+0000 7f4954000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f494c106110 0x7f494c073580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:52.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.256+0000 7f4954000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f494c073ac0 0x7f494c1a5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:52.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.256+0000 7f4954000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f494c1a56a0 con 0x7f494c100ff0 2024-02-02T15:31:52.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.256+0000 7f4954000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f494c1a5810 con 0x7f494c106110 2024-02-02T15:31:52.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.256+0000 7f4954000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f494c1a59d0 con 0x7f494c073ac0 2024-02-02T15:31:52.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.256+0000 7f4951600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f494c100ff0 0x7f494c073040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:52.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.256+0000 7f4950c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f494c106110 0x7f494c073580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:52.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.256+0000 7f4950c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f494c106110 0x7f494c073580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:58762/0 (socket says 172.21.15.1:58762) 2024-02-02T15:31:52.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.256+0000 7f4950c00700 1 -- 172.21.15.1:0/1323251958 learned_addr learned my addr 172.21.15.1:0/1323251958 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:31:52.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.256+0000 7f4952000700 1 --2- 172.21.15.1:0/1323251958 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f494c073ac0 0x7f494c1a5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:52.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.256+0000 7f4951600700 1 -- 172.21.15.1:0/1323251958 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f494c073ac0 msgr2=0x7f494c1a5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:52.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.256+0000 7f4951600700 1 --2- 172.21.15.1:0/1323251958 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f494c073ac0 0x7f494c1a5010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.256+0000 7f4951600700 1 -- 172.21.15.1:0/1323251958 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f494c106110 msgr2=0x7f494c073580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:52.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.256+0000 7f4951600700 1 --2- 172.21.15.1:0/1323251958 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f494c106110 0x7f494c073580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.256+0000 7f4951600700 1 -- 172.21.15.1:0/1323251958 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49340095f0 con 0x7f494c100ff0 2024-02-02T15:31:52.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.256+0000 7f4950c00700 1 --2- 172.21.15.1:0/1323251958 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f494c106110 0x7f494c073580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:31:52.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.257+0000 7f4951600700 1 --2- 172.21.15.1:0/1323251958 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f494c100ff0 0x7f494c073040 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f493c00e9f0 tx=0x7f493c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:52.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.257+0000 7f494a000700 1 -- 172.21.15.1:0/1323251958 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f493c00c9c0 con 0x7f494c100ff0 2024-02-02T15:31:52.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.257+0000 7f494a000700 1 -- 172.21.15.1:0/1323251958 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f493c010430 con 0x7f494c100ff0 2024-02-02T15:31:52.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.257+0000 7f4954000700 1 -- 172.21.15.1:0/1323251958 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f494c1a5cb0 con 0x7f494c100ff0 2024-02-02T15:31:52.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.257+0000 7f494a000700 1 -- 172.21.15.1:0/1323251958 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f493c018680 con 0x7f494c100ff0 2024-02-02T15:31:52.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.257+0000 7f4954000700 1 -- 172.21.15.1:0/1323251958 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f494c1a61c0 con 0x7f494c100ff0 2024-02-02T15:31:52.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.262+0000 7f494a000700 1 -- 172.21.15.1:0/1323251958 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 27) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f493c010860 con 0x7f494c100ff0 2024-02-02T15:31:52.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.263+0000 7f4967b77b80 1 Processor -- start 2024-02-02T15:31:52.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.263+0000 7f4967b77b80 1 -- start start 2024-02-02T15:31:52.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.263+0000 7f4967b77b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561da8b07010 0x561da8b073f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:52.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.263+0000 7f494a000700 1 --2- 172.21.15.1:0/1323251958 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f492c0a0690 0x7f492c0a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:52.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.263+0000 7f4967b77b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561da8b07930 0x561da8b122a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:52.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.264+0000 7f4967b77b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561da8b127e0 0x561da8b14bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:52.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.264+0000 7f4967b77b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x561da8a7b4c0 con 0x561da8b07010 2024-02-02T15:31:52.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.264+0000 7f4967b77b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x561da8a5faa0 con 0x561da8b127e0 2024-02-02T15:31:52.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.264+0000 7f4967b77b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561da8a5d460 con 0x561da8b07930 2024-02-02T15:31:52.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.264+0000 7f494a000700 1 -- 172.21.15.1:0/1323251958 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f493c014070 con 0x7f494c100ff0 2024-02-02T15:31:52.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.264+0000 7f4950c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561da8b07930 0x561da8b122a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:52.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.264+0000 7f4950c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561da8b07930 0x561da8b122a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:43618/0 (socket says 172.21.15.1:43618) 2024-02-02T15:31:52.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.264+0000 7f4950c00700 1 -- 172.21.15.1:0/608667901 learned_addr learned my addr 172.21.15.1:0/608667901 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:31:52.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.264+0000 7f4951600700 1 --2- 172.21.15.1:0/608667901 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561da8b07010 0x561da8b073f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:52.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.264+0000 7f4952000700 1 --2- 172.21.15.1:0/608667901 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561da8b127e0 0x561da8b14bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:52.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.264+0000 7f4952000700 1 --2- 172.21.15.1:0/1323251958 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f492c0a0690 0x7f492c0a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:52.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.264+0000 7f4950c00700 1 -- 172.21.15.1:0/608667901 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561da8b127e0 msgr2=0x561da8b14bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:52.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.264+0000 7f4950c00700 1 --2- 172.21.15.1:0/608667901 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561da8b127e0 0x561da8b14bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.264+0000 7f4950c00700 1 -- 172.21.15.1:0/608667901 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561da8b07010 msgr2=0x561da8b073f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:52.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.264+0000 7f4950c00700 1 --2- 172.21.15.1:0/608667901 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561da8b07010 0x561da8b073f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.264+0000 7f4950c00700 1 -- 172.21.15.1:0/608667901 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561da88da3b0 con 0x561da8b07930 2024-02-02T15:31:52.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.264+0000 7f4952000700 1 --2- 172.21.15.1:0/608667901 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561da8b127e0 0x561da8b14bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:31:52.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.265+0000 7f4950c00700 1 --2- 172.21.15.1:0/608667901 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561da8b07930 0x561da8b122a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4934005f10 tx=0x7f493400b610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b17e9256937b53c2 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:52.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.265+0000 7f4952000700 1 --2- 172.21.15.1:0/1323251958 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f492c0a0690 0x7f492c0a2b50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4944009ec0 tx=0x7f494400b040 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:52.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.265+0000 7f4942600700 1 -- 172.21.15.1:0/608667901 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f493401d070 con 0x561da8b07930 2024-02-02T15:31:52.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.265+0000 7f4942600700 1 -- 172.21.15.1:0/608667901 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f493401e040 con 0x561da8b07930 2024-02-02T15:31:52.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.265+0000 7f4942600700 1 -- 172.21.15.1:0/608667901 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49340055d0 con 0x561da8b07930 2024-02-02T15:31:52.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.266+0000 7f4967b77b80 1 -- 172.21.15.1:0/608667901 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561da8b07930 msgr2=0x561da8b122a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:52.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.266+0000 7f4967b77b80 1 --2- 172.21.15.1:0/608667901 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561da8b07930 0x561da8b122a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4934005f10 tx=0x7f493400b610 comp rx=0 tx=0).stop 2024-02-02T15:31:52.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.266+0000 7f4967b77b80 1 -- 172.21.15.1:0/608667901 shutdown_connections 2024-02-02T15:31:52.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.266+0000 7f4967b77b80 1 --2- 172.21.15.1:0/608667901 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561da8b127e0 0x561da8b14bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.266+0000 7f4967b77b80 1 --2- 172.21.15.1:0/608667901 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561da8b07010 0x561da8b073f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.266+0000 7f4967b77b80 1 --2- 172.21.15.1:0/608667901 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561da8b07930 0x561da8b122a0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.266+0000 7f4967b77b80 1 -- 172.21.15.1:0/608667901 >> 172.21.15.1:0/608667901 conn(0x561da8a66db0 msgr2=0x561da8b033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:52.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.266+0000 7f4967b77b80 1 -- 172.21.15.1:0/608667901 shutdown_connections 2024-02-02T15:31:52.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.266+0000 7f4967b77b80 1 -- 172.21.15.1:0/608667901 wait complete. 2024-02-02T15:31:52.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.267+0000 7f4967b77b80 1 -- start start 2024-02-02T15:31:52.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.267+0000 7f4967b77b80 1 Processor -- start 2024-02-02T15:31:52.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.267+0000 7f4967b77b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561da8b07010 0x561da8b12150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:52.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.267+0000 7f4967b77b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561da8b07930 0x561da8b0f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:52.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.267+0000 7f4967b77b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561da8b127e0 0x561da8b0f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:52.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.267+0000 7f4967b77b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x561da8a58d50 con 0x561da8b07010 2024-02-02T15:31:52.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.267+0000 7f4967b77b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x561da89f4d80 con 0x561da8b127e0 2024-02-02T15:31:52.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.267+0000 7f4967b77b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561da89f4a20 con 0x561da8b07930 2024-02-02T15:31:52.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.267+0000 7f4967b77b80 10 client.0 _ll_register_callbacks cb 0x561da89f6290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:31:52.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.268+0000 7f4950c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561da8b07930 0x561da8b0f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:52.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.268+0000 7f4951600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561da8b07010 0x561da8b12150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:52.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.268+0000 7f4950c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561da8b07930 0x561da8b0f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:43634/0 (socket says 172.21.15.1:43634) 2024-02-02T15:31:52.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.268+0000 7f4950c00700 1 -- 172.21.15.1:0/4152522444 learned_addr learned my addr 172.21.15.1:0/4152522444 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:31:52.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.268+0000 7f4952000700 1 --2- 172.21.15.1:0/4152522444 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561da8b127e0 0x561da8b0f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:52.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.268+0000 7f4950c00700 1 -- 172.21.15.1:0/4152522444 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561da8b127e0 msgr2=0x561da8b0f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:52.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.268+0000 7f4950c00700 1 --2- 172.21.15.1:0/4152522444 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561da8b127e0 0x561da8b0f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.268+0000 7f4950c00700 1 -- 172.21.15.1:0/4152522444 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561da8b07010 msgr2=0x561da8b12150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:52.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.268+0000 7f4950c00700 1 --2- 172.21.15.1:0/4152522444 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561da8b07010 0x561da8b12150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.268+0000 7f4950c00700 1 -- 172.21.15.1:0/4152522444 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f494c10e500 con 0x561da8b07930 2024-02-02T15:31:52.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.268+0000 7f4951600700 1 --2- 172.21.15.1:0/4152522444 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561da8b07010 0x561da8b12150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:31:52.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.268+0000 7f4950c00700 1 --2- 172.21.15.1:0/4152522444 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561da8b07930 0x561da8b0f3d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4934009ce0 tx=0x7f4934004a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e7f9906bc31425f server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:52.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.268+0000 7f493aa00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:31:52.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.269+0000 7f493aa00700 1 -- 172.21.15.1:0/4152522444 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f493401d070 con 0x561da8b07930 2024-02-02T15:31:52.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.269+0000 7f493aa00700 1 -- 172.21.15.1:0/4152522444 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f4934004c30 con 0x561da8b07930 2024-02-02T15:31:52.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.269+0000 7f4967b77b80 1 -- 172.21.15.1:0/4152522444 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561da8afb0f0 con 0x561da8b07930 2024-02-02T15:31:52.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.269+0000 7f493aa00700 1 -- 172.21.15.1:0/4152522444 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4934005940 con 0x561da8b07930 2024-02-02T15:31:52.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.269+0000 7f4967b77b80 10 client.26890 fetch_fsmap learned FSMap version 35 2024-02-02T15:31:52.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.269+0000 7f4967b77b80 1 -- 172.21.15.1:0/4152522444 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x561da8886b60 con 0x561da8b07930 2024-02-02T15:31:52.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.269+0000 7f493aa00700 1 -- 172.21.15.1:0/4152522444 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561da8afb0f0 con 0x561da8b07930 2024-02-02T15:31:52.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.269+0000 7f493aa00700 1 -- 172.21.15.1:0/4152522444 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 35) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f493402c440 con 0x561da8b07930 2024-02-02T15:31:52.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.269+0000 7f4967b77b80 10 client.26890 fetch_fsmap finished waiting for FSMap version 35 2024-02-02T15:31:52.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.269+0000 7f4967b77b80 10 client.26890 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-02-02T15:31:52.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.270+0000 7f4967b77b80 20 client.26890 populate_metadata read hostname 'smithi001' 2024-02-02T15:31:52.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.270+0000 7f4967b77b80 1 --2- 172.21.15.1:0/4152522444 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x561da8b21cb0 0x561da8b240d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:52.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.270+0000 7f4967b77b80 4 client.26890 mds_command: new command op to 24502 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:31:52.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.270+0000 7f4967b77b80 1 -- 172.21.15.1:0/4152522444 --> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561da8a51dd0 con 0x561da8b21cb0 2024-02-02T15:31:52.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.270+0000 7f4951600700 1 --2- 172.21.15.1:0/4152522444 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x561da8b21cb0 0x561da8b240d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:52.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.270+0000 7f4951600700 1 --2- 172.21.15.1:0/4152522444 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x561da8b21cb0 0x561da8b240d0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6c57493cfbb71a02 server_cookie=e65bbc7387f6cf1 in_seq=0 out_seq=0 2024-02-02T15:31:52.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.270+0000 7f493aa00700 10 client.26890 ms_handle_connect on v2:172.21.15.142:6834/1052606547 2024-02-02T15:31:52.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.271+0000 7f493aa00700 1 -- 172.21.15.1:0/4152522444 <== mds.4 v2:172.21.15.142:6834/1052606547 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x561da8a51dd0 con 0x561da8b21cb0 2024-02-02T15:31:52.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.271+0000 7f493aa00700 10 client.26890 handle_command_reply: tid=0 2024-02-02T15:31:52.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.271+0000 7f4967b77b80 1 client.26890 shutdown 2024-02-02T15:31:52.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.271+0000 7f4967b77b80 1 -- 172.21.15.1:0/4152522444 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561da8b07930 msgr2=0x561da8b0f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:52.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.271+0000 7f4967b77b80 1 --2- 172.21.15.1:0/4152522444 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561da8b07930 0x561da8b0f3d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4934009ce0 tx=0x7f4934004a10 comp rx=0 tx=0).stop 2024-02-02T15:31:52.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.271+0000 7f4967b77b80 1 -- 172.21.15.1:0/4152522444 shutdown_connections 2024-02-02T15:31:52.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.271+0000 7f4967b77b80 1 --2- 172.21.15.1:0/4152522444 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x561da8b21cb0 0x561da8b240d0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.271+0000 7f4967b77b80 1 --2- 172.21.15.1:0/4152522444 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561da8b127e0 0x561da8b0f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.271+0000 7f4967b77b80 1 --2- 172.21.15.1:0/4152522444 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561da8b07010 0x561da8b12150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.271+0000 7f493aa00700 0 client.26890 ms_handle_reset on v2:172.21.15.142:6834/1052606547 2024-02-02T15:31:52.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.271+0000 7f4967b77b80 1 --2- 172.21.15.1:0/4152522444 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561da8b07930 0x561da8b0f3d0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.271+0000 7f4967b77b80 1 -- 172.21.15.1:0/4152522444 >> 172.21.15.1:0/4152522444 conn(0x561da8a66db0 msgr2=0x561da8b02fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:52.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.271+0000 7f4967b77b80 1 -- 172.21.15.1:0/4152522444 shutdown_connections 2024-02-02T15:31:52.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.271+0000 7f4967b77b80 1 -- 172.21.15.1:0/4152522444 wait complete. 2024-02-02T15:31:52.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.272+0000 7f4967b77b80 20 client.26890 trim_cache size 0 max 16384 2024-02-02T15:31:52.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.288+0000 7f4967b77b80 1 Processor -- start 2024-02-02T15:31:52.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.288+0000 7f4967b77b80 1 -- start start 2024-02-02T15:31:52.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.288+0000 7f4967b77b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561da8a66db0 0x561da8b19300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:52.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.288+0000 7f4967b77b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561da8b07010 0x561da8b11c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:52.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.288+0000 7f4967b77b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561da8b07930 0x561da8b1d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:52.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.288+0000 7f4967b77b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x561da88dc6a0 con 0x561da8a66db0 2024-02-02T15:31:52.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.288+0000 7f4967b77b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x561da8b0ffc0 con 0x561da8b07930 2024-02-02T15:31:52.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.288+0000 7f4967b77b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561da8b0e940 con 0x561da8b07010 2024-02-02T15:31:52.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.289+0000 7f4950c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561da8b07010 0x561da8b11c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:52.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.289+0000 7f4950c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561da8b07010 0x561da8b11c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:43650/0 (socket says 172.21.15.1:43650) 2024-02-02T15:31:52.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.289+0000 7f4950c00700 1 -- 172.21.15.1:0/1952209273 learned_addr learned my addr 172.21.15.1:0/1952209273 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:31:52.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.289+0000 7f4952000700 1 --2- 172.21.15.1:0/1952209273 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561da8b07930 0x561da8b1d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:52.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.289+0000 7f4951600700 1 --2- 172.21.15.1:0/1952209273 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561da8a66db0 0x561da8b19300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:52.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.289+0000 7f4950c00700 1 -- 172.21.15.1:0/1952209273 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561da8b07930 msgr2=0x561da8b1d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:52.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.289+0000 7f4950c00700 1 --2- 172.21.15.1:0/1952209273 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561da8b07930 0x561da8b1d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.289+0000 7f4950c00700 1 -- 172.21.15.1:0/1952209273 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561da8a66db0 msgr2=0x561da8b19300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:52.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.289+0000 7f4950c00700 1 --2- 172.21.15.1:0/1952209273 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561da8a66db0 0x561da8b19300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.289+0000 7f4950c00700 1 -- 172.21.15.1:0/1952209273 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561da8a1eea0 con 0x561da8b07010 2024-02-02T15:31:52.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.289+0000 7f4950c00700 1 --2- 172.21.15.1:0/1952209273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561da8b07010 0x561da8b11c80 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4934005ac0 tx=0x7f49340050c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e74187a4615de0c0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:52.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.289+0000 7f4943000700 1 -- 172.21.15.1:0/1952209273 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f493401d070 con 0x561da8b07010 2024-02-02T15:31:52.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.290+0000 7f4943000700 1 -- 172.21.15.1:0/1952209273 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f493402c640 con 0x561da8b07010 2024-02-02T15:31:52.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.290+0000 7f4943000700 1 -- 172.21.15.1:0/1952209273 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49340053c0 con 0x561da8b07010 2024-02-02T15:31:52.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.290+0000 7f4967b77b80 1 -- 172.21.15.1:0/1952209273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561da8b07010 msgr2=0x561da8b11c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:52.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.290+0000 7f4967b77b80 1 --2- 172.21.15.1:0/1952209273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561da8b07010 0x561da8b11c80 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4934005ac0 tx=0x7f49340050c0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.290+0000 7f4967b77b80 1 -- 172.21.15.1:0/1952209273 shutdown_connections 2024-02-02T15:31:52.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.290+0000 7f4967b77b80 1 --2- 172.21.15.1:0/1952209273 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561da8b07930 0x561da8b1d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.290+0000 7f4967b77b80 1 --2- 172.21.15.1:0/1952209273 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561da8a66db0 0x561da8b19300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.290+0000 7f4967b77b80 1 --2- 172.21.15.1:0/1952209273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561da8b07010 0x561da8b11c80 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.290+0000 7f4967b77b80 1 -- 172.21.15.1:0/1952209273 >> 172.21.15.1:0/1952209273 conn(0x561da8b14d90 msgr2=0x561da8b01b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:52.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.290+0000 7f4967b77b80 1 -- 172.21.15.1:0/1952209273 shutdown_connections 2024-02-02T15:31:52.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.290+0000 7f4967b77b80 1 -- 172.21.15.1:0/1952209273 wait complete. 2024-02-02T15:31:52.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.291+0000 7f4967b77b80 1 -- start start 2024-02-02T15:31:52.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.291+0000 7f4967b77b80 1 Processor -- start 2024-02-02T15:31:52.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.292+0000 7f4967b77b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561da8a66db0 0x561da8b152e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:52.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.292+0000 7f4967b77b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561da8b07010 0x561da8b15820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:52.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.292+0000 7f4967b77b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561da8b07930 0x561da8b15d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:52.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.292+0000 7f4967b77b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x561da8b19710 con 0x561da8b07010 2024-02-02T15:31:52.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.292+0000 7f4967b77b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x561da8b72780 con 0x561da8a66db0 2024-02-02T15:31:52.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.292+0000 7f4967b77b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561da8b72910 con 0x561da8b07930 2024-02-02T15:31:52.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.292+0000 7f4967b77b80 10 client.0 _ll_register_callbacks cb 0x561da89f6290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:31:52.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.292+0000 7f4951600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561da8a66db0 0x561da8b152e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:52.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.292+0000 7f4951600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561da8a66db0 0x561da8b152e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:58788/0 (socket says 172.21.15.1:58788) 2024-02-02T15:31:52.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.292+0000 7f4951600700 1 -- 172.21.15.1:0/2445601779 learned_addr learned my addr 172.21.15.1:0/2445601779 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:31:52.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.292+0000 7f4952000700 1 --2- 172.21.15.1:0/2445601779 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561da8b07930 0x561da8b15d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:52.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.292+0000 7f4950c00700 1 --2- 172.21.15.1:0/2445601779 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561da8b07010 0x561da8b15820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:52.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.292+0000 7f4951600700 1 -- 172.21.15.1:0/2445601779 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561da8b07930 msgr2=0x561da8b15d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:52.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.292+0000 7f4951600700 1 --2- 172.21.15.1:0/2445601779 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561da8b07930 0x561da8b15d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.292+0000 7f4951600700 1 -- 172.21.15.1:0/2445601779 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561da8b07010 msgr2=0x561da8b15820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:52.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.292+0000 7f4951600700 1 --2- 172.21.15.1:0/2445601779 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561da8b07010 0x561da8b15820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.292+0000 7f4951600700 1 -- 172.21.15.1:0/2445601779 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561da8886b60 con 0x561da8a66db0 2024-02-02T15:31:52.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.292+0000 7f4950c00700 1 --2- 172.21.15.1:0/2445601779 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561da8b07010 0x561da8b15820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:31:52.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.293+0000 7f4951600700 1 --2- 172.21.15.1:0/2445601779 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561da8a66db0 0x561da8b152e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f493c0990c0 tx=0x7f493c010c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39c731e5a21ddd0d server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:52.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.293+0000 7f493aa00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:31:52.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.293+0000 7f493aa00700 1 -- 172.21.15.1:0/2445601779 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f493c0a0d90 con 0x561da8a66db0 2024-02-02T15:31:52.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.293+0000 7f4967b77b80 1 -- 172.21.15.1:0/2445601779 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f493401e040 con 0x561da8a66db0 2024-02-02T15:31:52.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.293+0000 7f493aa00700 1 -- 172.21.15.1:0/2445601779 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f493c099ac0 con 0x561da8a66db0 2024-02-02T15:31:52.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.293+0000 7f493aa00700 1 -- 172.21.15.1:0/2445601779 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f493c0a0130 con 0x561da8a66db0 2024-02-02T15:31:52.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.293+0000 7f4967b77b80 10 client.26897 fetch_fsmap learned FSMap version 35 2024-02-02T15:31:52.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.293+0000 7f4967b77b80 1 -- 172.21.15.1:0/2445601779 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x561da88decd0 con 0x561da8a66db0 2024-02-02T15:31:52.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.293+0000 7f493aa00700 1 -- 172.21.15.1:0/2445601779 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f493401e040 con 0x561da8a66db0 2024-02-02T15:31:52.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.294+0000 7f493aa00700 1 -- 172.21.15.1:0/2445601779 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 35) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f493c0988b0 con 0x561da8a66db0 2024-02-02T15:31:52.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.294+0000 7f4967b77b80 10 client.26897 fetch_fsmap finished waiting for FSMap version 35 2024-02-02T15:31:52.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.294+0000 7f4967b77b80 10 client.26897 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-02-02T15:31:52.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.294+0000 7f4967b77b80 20 client.26897 populate_metadata read hostname 'smithi001' 2024-02-02T15:31:52.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.294+0000 7f4967b77b80 1 --2- 172.21.15.1:0/2445601779 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x561da8b127e0 0x561da8b76030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:52.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.294+0000 7f4967b77b80 4 client.26897 mds_command: new command op to 24502 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:31:52.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.294+0000 7f4967b77b80 1 -- 172.21.15.1:0/2445601779 --> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561da8a17f50 con 0x561da8b127e0 2024-02-02T15:31:52.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.294+0000 7f4950c00700 1 --2- 172.21.15.1:0/2445601779 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x561da8b127e0 0x561da8b76030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:52.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.295+0000 7f4950c00700 1 --2- 172.21.15.1:0/2445601779 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x561da8b127e0 0x561da8b76030 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c6995e5fe3a3824e server_cookie=64e3abd1aabd8eca in_seq=0 out_seq=0 2024-02-02T15:31:52.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.295+0000 7f493aa00700 10 client.26897 ms_handle_connect on v2:172.21.15.142:6834/1052606547 2024-02-02T15:31:52.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.295+0000 7f493aa00700 1 -- 172.21.15.1:0/2445601779 <== mds.4 v2:172.21.15.142:6834/1052606547 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561da8a17f50 con 0x561da8b127e0 2024-02-02T15:31:52.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.295+0000 7f493aa00700 10 client.26897 handle_command_reply: tid=0 2024-02-02T15:31:52.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.295+0000 7f4967b77b80 1 client.26897 shutdown 2024-02-02T15:31:52.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.295+0000 7f4967b77b80 1 -- 172.21.15.1:0/2445601779 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561da8a66db0 msgr2=0x561da8b152e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:52.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.295+0000 7f4967b77b80 1 --2- 172.21.15.1:0/2445601779 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561da8a66db0 0x561da8b152e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f493c0990c0 tx=0x7f493c010c60 comp rx=0 tx=0).stop 2024-02-02T15:31:52.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.295+0000 7f4967b77b80 1 -- 172.21.15.1:0/2445601779 shutdown_connections 2024-02-02T15:31:52.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.295+0000 7f4967b77b80 1 --2- 172.21.15.1:0/2445601779 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x561da8b127e0 0x561da8b76030 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.295+0000 7f4967b77b80 1 --2- 172.21.15.1:0/2445601779 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561da8b07930 0x561da8b15d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.295+0000 7f4967b77b80 1 --2- 172.21.15.1:0/2445601779 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561da8a66db0 0x561da8b152e0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.295+0000 7f4967b77b80 1 --2- 172.21.15.1:0/2445601779 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561da8b07010 0x561da8b15820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.295+0000 7f493aa00700 0 client.26897 ms_handle_reset on v2:172.21.15.142:6834/1052606547 2024-02-02T15:31:52.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.295+0000 7f4967b77b80 1 -- 172.21.15.1:0/2445601779 >> 172.21.15.1:0/2445601779 conn(0x561da8b14d90 msgr2=0x561da8b1d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:52.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.295+0000 7f4967b77b80 1 -- 172.21.15.1:0/2445601779 shutdown_connections 2024-02-02T15:31:52.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.295+0000 7f4967b77b80 1 -- 172.21.15.1:0/2445601779 wait complete. 2024-02-02T15:31:52.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.295+0000 7f4967b77b80 20 client.26897 trim_cache size 0 max 16384 2024-02-02T15:31:52.299 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:31:52.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.296+0000 7f4954000700 1 -- 172.21.15.1:0/1323251958 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f492c0a0690 msgr2=0x7f492c0a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:52.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.296+0000 7f4954000700 1 --2- 172.21.15.1:0/1323251958 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f492c0a0690 0x7f492c0a2b50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4944009ec0 tx=0x7f494400b040 comp rx=0 tx=0).stop 2024-02-02T15:31:52.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.296+0000 7f4954000700 1 -- 172.21.15.1:0/1323251958 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f494c100ff0 msgr2=0x7f494c073040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:52.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.296+0000 7f4954000700 1 --2- 172.21.15.1:0/1323251958 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f494c100ff0 0x7f494c073040 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f493c00e9f0 tx=0x7f493c00ed00 comp rx=0 tx=0).stop 2024-02-02T15:31:52.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.296+0000 7f4954000700 1 -- 172.21.15.1:0/1323251958 shutdown_connections 2024-02-02T15:31:52.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.296+0000 7f4954000700 1 --2- 172.21.15.1:0/1323251958 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f492c0a0690 0x7f492c0a2b50 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.296+0000 7f4954000700 1 --2- 172.21.15.1:0/1323251958 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f494c073ac0 0x7f494c1a5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.296+0000 7f4954000700 1 --2- 172.21.15.1:0/1323251958 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f494c100ff0 0x7f494c073040 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.296+0000 7f4954000700 1 --2- 172.21.15.1:0/1323251958 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f494c106110 0x7f494c073580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.296+0000 7f4954000700 1 -- 172.21.15.1:0/1323251958 >> 172.21.15.1:0/1323251958 conn(0x7f494c0fc810 msgr2=0x7f494c0fd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:52.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.296+0000 7f4954000700 1 -- 172.21.15.1:0/1323251958 shutdown_connections 2024-02-02T15:31:52.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.296+0000 7f4954000700 1 -- 172.21.15.1:0/1323251958 wait complete. 2024-02-02T15:31:52.314 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-02-02T15:31:52.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.561+0000 7fb079e00700 1 -- 172.21.15.1:0/2421304249 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb074073170 msgr2=0x7fb074073550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:52.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.561+0000 7fb079e00700 1 --2- 172.21.15.1:0/2421304249 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb074073170 0x7fb074073550 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb064009990 tx=0x7fb064009ca0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.561+0000 7fb079e00700 1 -- 172.21.15.1:0/2421304249 shutdown_connections 2024-02-02T15:31:52.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.561+0000 7fb079e00700 1 --2- 172.21.15.1:0/2421304249 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb07410d250 0x7fb07410f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.561+0000 7fb079e00700 1 --2- 172.21.15.1:0/2421304249 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb074073170 0x7fb074073550 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.561+0000 7fb079e00700 1 --2- 172.21.15.1:0/2421304249 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb074073a90 0x7fb07410cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.561+0000 7fb079e00700 1 -- 172.21.15.1:0/2421304249 >> 172.21.15.1:0/2421304249 conn(0x7fb0740fc540 msgr2=0x7fb0740fe960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:52.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.561+0000 7fb079e00700 1 -- 172.21.15.1:0/2421304249 shutdown_connections 2024-02-02T15:31:52.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.561+0000 7fb079e00700 1 -- 172.21.15.1:0/2421304249 wait complete. 2024-02-02T15:31:52.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.562+0000 7fb079e00700 1 Processor -- start 2024-02-02T15:31:52.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.562+0000 7fb079e00700 1 -- start start 2024-02-02T15:31:52.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.564+0000 7fb079e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb074073170 0x7fb074198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:52.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.564+0000 7fb079e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb074073a90 0x7fb074199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:52.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.564+0000 7fb079e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb07410d250 0x7fb07419d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:52.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.564+0000 7fb079e00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb07419dda0 con 0x7fb074073170 2024-02-02T15:31:52.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.564+0000 7fb079e00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb07419df10 con 0x7fb074073a90 2024-02-02T15:31:52.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.564+0000 7fb079e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb07419e0d0 con 0x7fb07410d250 2024-02-02T15:31:52.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.564+0000 7fb073e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb074073a90 0x7fb074199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:52.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.564+0000 7fb078a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb074073170 0x7fb074198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:52.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.564+0000 7fb073e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb074073a90 0x7fb074199300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:58808/0 (socket says 172.21.15.1:58808) 2024-02-02T15:31:52.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.564+0000 7fb073e00700 1 -- 172.21.15.1:0/3525727358 learned_addr learned my addr 172.21.15.1:0/3525727358 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:31:52.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.564+0000 7fb079400700 1 --2- 172.21.15.1:0/3525727358 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb07410d250 0x7fb07419d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:52.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.564+0000 7fb073e00700 1 -- 172.21.15.1:0/3525727358 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb07410d250 msgr2=0x7fb07419d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:52.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.564+0000 7fb073e00700 1 --2- 172.21.15.1:0/3525727358 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb07410d250 0x7fb07419d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.564+0000 7fb073e00700 1 -- 172.21.15.1:0/3525727358 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb074073170 msgr2=0x7fb074198dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:52.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.564+0000 7fb073e00700 1 --2- 172.21.15.1:0/3525727358 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb074073170 0x7fb074198dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.564+0000 7fb073e00700 1 -- 172.21.15.1:0/3525727358 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0640095f0 con 0x7fb074073a90 2024-02-02T15:31:52.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.564+0000 7fb073e00700 1 --2- 172.21.15.1:0/3525727358 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb074073a90 0x7fb074199300 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb05c00e9a0 tx=0x7fb05c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:52.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.564+0000 7fb071600700 1 -- 172.21.15.1:0/3525727358 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb05c009790 con 0x7fb074073a90 2024-02-02T15:31:52.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.564+0000 7fb071600700 1 -- 172.21.15.1:0/3525727358 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb05c00cbd0 con 0x7fb074073a90 2024-02-02T15:31:52.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.564+0000 7fb079e00700 1 -- 172.21.15.1:0/3525727358 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb07419e3b0 con 0x7fb074073a90 2024-02-02T15:31:52.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.564+0000 7fb071600700 1 -- 172.21.15.1:0/3525727358 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb05c018680 con 0x7fb074073a90 2024-02-02T15:31:52.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.564+0000 7fb079e00700 1 -- 172.21.15.1:0/3525727358 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0741a5bf0 con 0x7fb074073a90 2024-02-02T15:31:52.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.568+0000 7fb071600700 1 -- 172.21.15.1:0/3525727358 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 27) v1 ==== 135200+0+0 (secure 0 0 0) 0x7fb05c00cd40 con 0x7fb074073a90 2024-02-02T15:31:52.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.569+0000 7fb08ef7ab80 1 Processor -- start 2024-02-02T15:31:52.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.569+0000 7fb08ef7ab80 1 -- start start 2024-02-02T15:31:52.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.569+0000 7fb071600700 1 --2- 172.21.15.1:0/3525727358 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fb0540a0690 0x7fb0540a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:52.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.569+0000 7fb071600700 1 -- 172.21.15.1:0/3525727358 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fb05c014070 con 0x7fb074073a90 2024-02-02T15:31:52.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.569+0000 7fb08ef7ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559eeb80d1f0 0x559eeb80d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:52.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.571+0000 7fb08ef7ab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559eeb80db10 0x559eeb8184b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:52.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.571+0000 7fb08ef7ab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559eeb8189f0 0x559eeb81ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:52.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.571+0000 7fb08ef7ab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eeb7814c0 con 0x559eeb80db10 2024-02-02T15:31:52.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.571+0000 7fb08ef7ab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eeb765aa0 con 0x559eeb8189f0 2024-02-02T15:31:52.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.571+0000 7fb08ef7ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eeb763460 con 0x559eeb80d1f0 2024-02-02T15:31:52.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.571+0000 7fb079400700 1 --2- 172.21.15.1:0/3525727358 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fb0540a0690 0x7fb0540a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:52.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.571+0000 7fb078a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559eeb80d1f0 0x559eeb80d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:52.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.571+0000 7fb078a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559eeb80d1f0 0x559eeb80d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:43676/0 (socket says 172.21.15.1:43676) 2024-02-02T15:31:52.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.571+0000 7fb078a00700 1 -- 172.21.15.1:0/1390752821 learned_addr learned my addr 172.21.15.1:0/1390752821 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:31:52.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.571+0000 7fb073e00700 1 --2- 172.21.15.1:0/1390752821 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559eeb80db10 0x559eeb8184b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:52.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.571+0000 7fb079400700 1 --2- 172.21.15.1:0/1390752821 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559eeb8189f0 0x559eeb81ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:52.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.571+0000 7fb078a00700 1 -- 172.21.15.1:0/1390752821 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559eeb8189f0 msgr2=0x559eeb81ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:52.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.571+0000 7fb078a00700 1 --2- 172.21.15.1:0/1390752821 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559eeb8189f0 0x559eeb81ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.571+0000 7fb078a00700 1 -- 172.21.15.1:0/1390752821 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559eeb80db10 msgr2=0x559eeb8184b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:52.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.571+0000 7fb078a00700 1 --2- 172.21.15.1:0/1390752821 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559eeb80db10 0x559eeb8184b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.571+0000 7fb078a00700 1 -- 172.21.15.1:0/1390752821 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559eeb5e03b0 con 0x559eeb80d1f0 2024-02-02T15:31:52.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.572+0000 7fb078a00700 1 --2- 172.21.15.1:0/1390752821 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559eeb80d1f0 0x559eeb80d5d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb064005ea0 tx=0x7fb064005ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce46a80ec86fa56a server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:52.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.572+0000 7fb069c00700 1 -- 172.21.15.1:0/1390752821 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb06401d070 con 0x559eeb80d1f0 2024-02-02T15:31:52.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.572+0000 7fb069c00700 1 -- 172.21.15.1:0/1390752821 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb06401e040 con 0x559eeb80d1f0 2024-02-02T15:31:52.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.572+0000 7fb069c00700 1 -- 172.21.15.1:0/1390752821 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0640049c0 con 0x559eeb80d1f0 2024-02-02T15:31:52.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.572+0000 7fb079400700 1 --2- 172.21.15.1:0/3525727358 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fb0540a0690 0x7fb0540a2b50 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb06c009510 tx=0x7fb06c009920 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:52.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.573+0000 7fb08ef7ab80 1 -- 172.21.15.1:0/1390752821 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559eeb80d1f0 msgr2=0x559eeb80d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:52.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.573+0000 7fb08ef7ab80 1 --2- 172.21.15.1:0/1390752821 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559eeb80d1f0 0x559eeb80d5d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb064005ea0 tx=0x7fb064005ed0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.573+0000 7fb08ef7ab80 1 -- 172.21.15.1:0/1390752821 shutdown_connections 2024-02-02T15:31:52.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.573+0000 7fb08ef7ab80 1 --2- 172.21.15.1:0/1390752821 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559eeb8189f0 0x559eeb81ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.573+0000 7fb08ef7ab80 1 --2- 172.21.15.1:0/1390752821 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559eeb80d1f0 0x559eeb80d5d0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.573+0000 7fb08ef7ab80 1 --2- 172.21.15.1:0/1390752821 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559eeb80db10 0x559eeb8184b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.573+0000 7fb08ef7ab80 1 -- 172.21.15.1:0/1390752821 >> 172.21.15.1:0/1390752821 conn(0x559eeb76cdb0 msgr2=0x559eeb8095c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:52.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.573+0000 7fb08ef7ab80 1 -- 172.21.15.1:0/1390752821 shutdown_connections 2024-02-02T15:31:52.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.573+0000 7fb08ef7ab80 1 -- 172.21.15.1:0/1390752821 wait complete. 2024-02-02T15:31:52.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.574+0000 7fb08ef7ab80 1 -- start start 2024-02-02T15:31:52.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.574+0000 7fb08ef7ab80 1 Processor -- start 2024-02-02T15:31:52.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.574+0000 7fb08ef7ab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559eeb80d1f0 0x559eeb8183a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:52.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.574+0000 7fb08ef7ab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559eeb80db10 0x559eeb815620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:52.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.574+0000 7fb08ef7ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559eeb8189f0 0x559eeb815c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:52.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.574+0000 7fb08ef7ab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eeb75ed50 con 0x559eeb80db10 2024-02-02T15:31:52.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.574+0000 7fb08ef7ab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eeb6fad80 con 0x559eeb80d1f0 2024-02-02T15:31:52.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.574+0000 7fb08ef7ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eeb6faa20 con 0x559eeb8189f0 2024-02-02T15:31:52.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.574+0000 7fb08ef7ab80 10 client.0 _ll_register_callbacks cb 0x559eeb6fc290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:31:52.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.574+0000 7fb079400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559eeb8189f0 0x559eeb815c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:52.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.574+0000 7fb079400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559eeb8189f0 0x559eeb815c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:43686/0 (socket says 172.21.15.1:43686) 2024-02-02T15:31:52.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.574+0000 7fb079400700 1 -- 172.21.15.1:0/4019497641 learned_addr learned my addr 172.21.15.1:0/4019497641 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:31:52.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.574+0000 7fb073e00700 1 --2- 172.21.15.1:0/4019497641 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559eeb80db10 0x559eeb815620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:52.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.574+0000 7fb078a00700 1 --2- 172.21.15.1:0/4019497641 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559eeb80d1f0 0x559eeb8183a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:52.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.574+0000 7fb079400700 1 -- 172.21.15.1:0/4019497641 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559eeb80d1f0 msgr2=0x559eeb8183a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:52.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.574+0000 7fb079400700 1 --2- 172.21.15.1:0/4019497641 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559eeb80d1f0 0x559eeb8183a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.574+0000 7fb079400700 1 -- 172.21.15.1:0/4019497641 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559eeb80db10 msgr2=0x559eeb815620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:52.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.574+0000 7fb079400700 1 --2- 172.21.15.1:0/4019497641 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559eeb80db10 0x559eeb815620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.574+0000 7fb079400700 1 -- 172.21.15.1:0/4019497641 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb074110380 con 0x559eeb8189f0 2024-02-02T15:31:52.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.574+0000 7fb073e00700 1 --2- 172.21.15.1:0/4019497641 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559eeb80db10 0x559eeb815620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:31:52.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.576+0000 7fb079400700 1 --2- 172.21.15.1:0/4019497641 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559eeb8189f0 0x559eeb815c50 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb06c01bb00 tx=0x7fb06c00b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae4f4087f0f0ff4d server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:52.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.576+0000 7fb062000700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:31:52.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.576+0000 7fb062000700 1 -- 172.21.15.1:0/4019497641 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb06c020070 con 0x559eeb8189f0 2024-02-02T15:31:52.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.576+0000 7fb08ef7ab80 1 -- 172.21.15.1:0/4019497641 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559eeb801340 con 0x559eeb8189f0 2024-02-02T15:31:52.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.576+0000 7fb062000700 1 -- 172.21.15.1:0/4019497641 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fb06c01be10 con 0x559eeb8189f0 2024-02-02T15:31:52.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.576+0000 7fb062000700 1 -- 172.21.15.1:0/4019497641 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb06c00e070 con 0x559eeb8189f0 2024-02-02T15:31:52.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.576+0000 7fb08ef7ab80 10 client.26908 fetch_fsmap learned FSMap version 35 2024-02-02T15:31:52.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.576+0000 7fb08ef7ab80 1 -- 172.21.15.1:0/4019497641 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x559eeb58cb60 con 0x559eeb8189f0 2024-02-02T15:31:52.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.577+0000 7fb062000700 1 -- 172.21.15.1:0/4019497641 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559eeb801340 con 0x559eeb8189f0 2024-02-02T15:31:52.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.577+0000 7fb062000700 1 -- 172.21.15.1:0/4019497641 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 35) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb06c024790 con 0x559eeb8189f0 2024-02-02T15:31:52.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.577+0000 7fb08ef7ab80 10 client.26908 fetch_fsmap finished waiting for FSMap version 35 2024-02-02T15:31:52.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.577+0000 7fb08ef7ab80 10 client.26908 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-02-02T15:31:52.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.577+0000 7fb08ef7ab80 20 client.26908 populate_metadata read hostname 'smithi001' 2024-02-02T15:31:52.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.577+0000 7fb08ef7ab80 1 --2- 172.21.15.1:0/4019497641 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x559eeb8282d0 0x559eeb82a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:52.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.577+0000 7fb08ef7ab80 4 client.26908 mds_command: new command op to 24514 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:31:52.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.577+0000 7fb08ef7ab80 1 -- 172.21.15.1:0/4019497641 --> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559eeb757dd0 con 0x559eeb8282d0 2024-02-02T15:31:52.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.577+0000 7fb078a00700 1 --2- 172.21.15.1:0/4019497641 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x559eeb8282d0 0x559eeb82a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:52.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.578+0000 7fb078a00700 1 --2- 172.21.15.1:0/4019497641 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x559eeb8282d0 0x559eeb82a6f0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=dff6d8400e06fa5e server_cookie=3ac0d47328754374 in_seq=0 out_seq=0 2024-02-02T15:31:52.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.578+0000 7fb062000700 10 client.26908 ms_handle_connect on v2:172.21.15.142:6836/2870289019 2024-02-02T15:31:52.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.578+0000 7fb062000700 1 -- 172.21.15.1:0/4019497641 <== mds.3 v2:172.21.15.142:6836/2870289019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x559eeb757dd0 con 0x559eeb8282d0 2024-02-02T15:31:52.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.578+0000 7fb062000700 10 client.26908 handle_command_reply: tid=0 2024-02-02T15:31:52.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.578+0000 7fb08ef7ab80 1 client.26908 shutdown 2024-02-02T15:31:52.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.578+0000 7fb08ef7ab80 1 -- 172.21.15.1:0/4019497641 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559eeb8189f0 msgr2=0x559eeb815c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:52.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.578+0000 7fb08ef7ab80 1 --2- 172.21.15.1:0/4019497641 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559eeb8189f0 0x559eeb815c50 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb06c01bb00 tx=0x7fb06c00b040 comp rx=0 tx=0).stop 2024-02-02T15:31:52.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.578+0000 7fb08ef7ab80 1 -- 172.21.15.1:0/4019497641 shutdown_connections 2024-02-02T15:31:52.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.578+0000 7fb08ef7ab80 1 --2- 172.21.15.1:0/4019497641 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559eeb8189f0 0x559eeb815c50 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.578+0000 7fb08ef7ab80 1 --2- 172.21.15.1:0/4019497641 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559eeb80d1f0 0x559eeb8183a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.578+0000 7fb08ef7ab80 1 --2- 172.21.15.1:0/4019497641 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x559eeb8282d0 0x559eeb82a6f0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.578+0000 7fb08ef7ab80 1 --2- 172.21.15.1:0/4019497641 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559eeb80db10 0x559eeb815620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.578+0000 7fb08ef7ab80 1 -- 172.21.15.1:0/4019497641 >> 172.21.15.1:0/4019497641 conn(0x559eeb76cdb0 msgr2=0x559eeb809150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:52.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.578+0000 7fb062000700 0 client.26908 ms_handle_reset on v2:172.21.15.142:6836/2870289019 2024-02-02T15:31:52.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.578+0000 7fb08ef7ab80 1 -- 172.21.15.1:0/4019497641 shutdown_connections 2024-02-02T15:31:52.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.578+0000 7fb08ef7ab80 1 -- 172.21.15.1:0/4019497641 wait complete. 2024-02-02T15:31:52.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.578+0000 7fb08ef7ab80 20 client.26908 trim_cache size 0 max 16384 2024-02-02T15:31:52.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.593+0000 7fb08ef7ab80 1 Processor -- start 2024-02-02T15:31:52.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.593+0000 7fb08ef7ab80 1 -- start start 2024-02-02T15:31:52.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.593+0000 7fb08ef7ab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559eeb80d1f0 0x559eeb823a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:52.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.593+0000 7fb08ef7ab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559eeb80db10 0x559eeb823f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:52.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.593+0000 7fb08ef7ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559eeb8189f0 0x559eeb810950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:52.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.593+0000 7fb08ef7ab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eeb5e26a0 con 0x559eeb80d1f0 2024-02-02T15:31:52.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.593+0000 7fb08ef7ab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eeb80e460 con 0x559eeb80db10 2024-02-02T15:31:52.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.593+0000 7fb08ef7ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eeb80ec80 con 0x559eeb8189f0 2024-02-02T15:31:52.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.593+0000 7fb073e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559eeb80db10 0x559eeb823f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:52.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.593+0000 7fb078a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559eeb80d1f0 0x559eeb823a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:52.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.593+0000 7fb073e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559eeb80db10 0x559eeb823f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:58830/0 (socket says 172.21.15.1:58830) 2024-02-02T15:31:52.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.593+0000 7fb079400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559eeb8189f0 0x559eeb810950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:52.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.593+0000 7fb073e00700 1 -- 172.21.15.1:0/3770769055 learned_addr learned my addr 172.21.15.1:0/3770769055 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:31:52.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.595+0000 7fb073e00700 1 -- 172.21.15.1:0/3770769055 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559eeb8189f0 msgr2=0x559eeb810950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:52.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.595+0000 7fb073e00700 1 --2- 172.21.15.1:0/3770769055 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559eeb8189f0 0x559eeb810950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.595+0000 7fb073e00700 1 -- 172.21.15.1:0/3770769055 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559eeb80d1f0 msgr2=0x559eeb823a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:52.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.595+0000 7fb073e00700 1 --2- 172.21.15.1:0/3770769055 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559eeb80d1f0 0x559eeb823a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.595+0000 7fb073e00700 1 -- 172.21.15.1:0/3770769055 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559eeb724ea0 con 0x559eeb80db10 2024-02-02T15:31:52.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.595+0000 7fb073e00700 1 --2- 172.21.15.1:0/3770769055 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559eeb80db10 0x559eeb823f90 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fb05c095fe0 tx=0x7fb05c0cc040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0840620154ee780 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:52.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.595+0000 7fb06a600700 1 -- 172.21.15.1:0/3770769055 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb05c0b0070 con 0x559eeb80db10 2024-02-02T15:31:52.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.595+0000 7fb06a600700 1 -- 172.21.15.1:0/3770769055 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb05c09d170 con 0x559eeb80db10 2024-02-02T15:31:52.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.595+0000 7fb06a600700 1 -- 172.21.15.1:0/3770769055 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb05c0997d0 con 0x559eeb80db10 2024-02-02T15:31:52.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.596+0000 7fb08ef7ab80 1 -- 172.21.15.1:0/3770769055 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559eeb80db10 msgr2=0x559eeb823f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:52.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.596+0000 7fb08ef7ab80 1 --2- 172.21.15.1:0/3770769055 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559eeb80db10 0x559eeb823f90 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fb05c095fe0 tx=0x7fb05c0cc040 comp rx=0 tx=0).stop 2024-02-02T15:31:52.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.596+0000 7fb08ef7ab80 1 -- 172.21.15.1:0/3770769055 shutdown_connections 2024-02-02T15:31:52.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.596+0000 7fb08ef7ab80 1 --2- 172.21.15.1:0/3770769055 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559eeb8189f0 0x559eeb810950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.596+0000 7fb08ef7ab80 1 --2- 172.21.15.1:0/3770769055 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559eeb80d1f0 0x559eeb823a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.596+0000 7fb08ef7ab80 1 --2- 172.21.15.1:0/3770769055 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559eeb80db10 0x559eeb823f90 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.596+0000 7fb08ef7ab80 1 -- 172.21.15.1:0/3770769055 >> 172.21.15.1:0/3770769055 conn(0x559eeb76cdb0 msgr2=0x559eeb807d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:52.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.596+0000 7fb08ef7ab80 1 -- 172.21.15.1:0/3770769055 shutdown_connections 2024-02-02T15:31:52.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.596+0000 7fb08ef7ab80 1 -- 172.21.15.1:0/3770769055 wait complete. 2024-02-02T15:31:52.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.596+0000 7fb08ef7ab80 1 -- start start 2024-02-02T15:31:52.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.597+0000 7fb08ef7ab80 1 Processor -- start 2024-02-02T15:31:52.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.597+0000 7fb08ef7ab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559eeb80d1f0 0x559eeb81f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:52.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.597+0000 7fb08ef7ab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559eeb80db10 0x559eeb81d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:52.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.597+0000 7fb08ef7ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559eeb8189f0 0x559eeb81d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:52.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.597+0000 7fb08ef7ab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eeb81f8e0 con 0x559eeb80db10 2024-02-02T15:31:52.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.597+0000 7fb08ef7ab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eeb81dee0 con 0x559eeb80d1f0 2024-02-02T15:31:52.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.597+0000 7fb08ef7ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eeb81e0a0 con 0x559eeb8189f0 2024-02-02T15:31:52.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.597+0000 7fb08ef7ab80 10 client.0 _ll_register_callbacks cb 0x559eeb6fc290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:31:52.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.597+0000 7fb078a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559eeb80d1f0 0x559eeb81f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:52.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.597+0000 7fb073e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559eeb80db10 0x559eeb81d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:52.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.597+0000 7fb079400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559eeb8189f0 0x559eeb81d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:52.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.597+0000 7fb078a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559eeb80d1f0 0x559eeb81f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:58838/0 (socket says 172.21.15.1:58838) 2024-02-02T15:31:52.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.597+0000 7fb078a00700 1 -- 172.21.15.1:0/3514680871 learned_addr learned my addr 172.21.15.1:0/3514680871 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:31:52.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.598+0000 7fb078a00700 1 -- 172.21.15.1:0/3514680871 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559eeb8189f0 msgr2=0x559eeb81d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:52.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.598+0000 7fb078a00700 1 --2- 172.21.15.1:0/3514680871 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559eeb8189f0 0x559eeb81d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.598+0000 7fb078a00700 1 -- 172.21.15.1:0/3514680871 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559eeb80db10 msgr2=0x559eeb81d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:52.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.598+0000 7fb078a00700 1 --2- 172.21.15.1:0/3514680871 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559eeb80db10 0x559eeb81d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.598+0000 7fb078a00700 1 -- 172.21.15.1:0/3514680871 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0741a5bf0 con 0x559eeb80d1f0 2024-02-02T15:31:52.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.598+0000 7fb073e00700 1 --2- 172.21.15.1:0/3514680871 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559eeb80db10 0x559eeb81d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:31:52.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.598+0000 7fb079400700 1 --2- 172.21.15.1:0/3514680871 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559eeb8189f0 0x559eeb81d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:31:52.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.598+0000 7fb078a00700 1 --2- 172.21.15.1:0/3514680871 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559eeb80d1f0 0x559eeb81f000 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fb064005f50 tx=0x7fb0640058e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a6b374fa04c5a4a server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:52.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.598+0000 7fb062000700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:31:52.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.598+0000 7fb062000700 1 -- 172.21.15.1:0/3514680871 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb064039070 con 0x559eeb80d1f0 2024-02-02T15:31:52.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.598+0000 7fb062000700 1 -- 172.21.15.1:0/3514680871 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fb064031070 con 0x559eeb80d1f0 2024-02-02T15:31:52.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.598+0000 7fb08ef7ab80 1 -- 172.21.15.1:0/3514680871 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb06401e040 con 0x559eeb80d1f0 2024-02-02T15:31:52.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.599+0000 7fb062000700 1 -- 172.21.15.1:0/3514680871 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb06401d040 con 0x559eeb80d1f0 2024-02-02T15:31:52.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.599+0000 7fb08ef7ab80 10 client.26915 fetch_fsmap learned FSMap version 35 2024-02-02T15:31:52.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.599+0000 7fb08ef7ab80 1 -- 172.21.15.1:0/3514680871 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x559eeb5e4cd0 con 0x559eeb80d1f0 2024-02-02T15:31:52.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.599+0000 7fb062000700 1 -- 172.21.15.1:0/3514680871 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb06401e040 con 0x559eeb80d1f0 2024-02-02T15:31:52.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.599+0000 7fb062000700 1 -- 172.21.15.1:0/3514680871 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 35) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb06403c030 con 0x559eeb80d1f0 2024-02-02T15:31:52.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.599+0000 7fb08ef7ab80 10 client.26915 fetch_fsmap finished waiting for FSMap version 35 2024-02-02T15:31:52.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.599+0000 7fb08ef7ab80 10 client.26915 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-02-02T15:31:52.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.599+0000 7fb08ef7ab80 20 client.26915 populate_metadata read hostname 'smithi001' 2024-02-02T15:31:52.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.599+0000 7fb08ef7ab80 1 --2- 172.21.15.1:0/3514680871 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x559eeb87ad70 0x559eeb87d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:52.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.599+0000 7fb08ef7ab80 4 client.26915 mds_command: new command op to 24514 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:31:52.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.599+0000 7fb08ef7ab80 1 -- 172.21.15.1:0/3514680871 --> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559eeb71df50 con 0x559eeb87ad70 2024-02-02T15:31:52.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.600+0000 7fb073e00700 1 --2- 172.21.15.1:0/3514680871 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x559eeb87ad70 0x559eeb87d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:52.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.600+0000 7fb062000700 10 client.26915 ms_handle_connect on v2:172.21.15.142:6836/2870289019 2024-02-02T15:31:52.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.600+0000 7fb073e00700 1 --2- 172.21.15.1:0/3514680871 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x559eeb87ad70 0x559eeb87d190 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8fddc9589426dc32 server_cookie=d7decb17e8576adb in_seq=0 out_seq=0 2024-02-02T15:31:52.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.600+0000 7fb062000700 1 -- 172.21.15.1:0/3514680871 <== mds.3 v2:172.21.15.142:6836/2870289019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559eeb71df50 con 0x559eeb87ad70 2024-02-02T15:31:52.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.600+0000 7fb062000700 10 client.26915 handle_command_reply: tid=0 2024-02-02T15:31:52.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.600+0000 7fb08ef7ab80 1 client.26915 shutdown 2024-02-02T15:31:52.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.600+0000 7fb08ef7ab80 1 -- 172.21.15.1:0/3514680871 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559eeb80d1f0 msgr2=0x559eeb81f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:52.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.600+0000 7fb08ef7ab80 1 --2- 172.21.15.1:0/3514680871 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559eeb80d1f0 0x559eeb81f000 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fb064005f50 tx=0x7fb0640058e0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.600+0000 7fb08ef7ab80 1 -- 172.21.15.1:0/3514680871 shutdown_connections 2024-02-02T15:31:52.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.600+0000 7fb08ef7ab80 1 --2- 172.21.15.1:0/3514680871 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559eeb8189f0 0x559eeb81d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.600+0000 7fb08ef7ab80 1 --2- 172.21.15.1:0/3514680871 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559eeb80d1f0 0x559eeb81f000 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.600+0000 7fb08ef7ab80 1 --2- 172.21.15.1:0/3514680871 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x559eeb87ad70 0x559eeb87d190 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.600+0000 7fb08ef7ab80 1 --2- 172.21.15.1:0/3514680871 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559eeb80db10 0x559eeb81d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.600+0000 7fb062000700 0 client.26915 ms_handle_reset on v2:172.21.15.142:6836/2870289019 2024-02-02T15:31:52.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.600+0000 7fb08ef7ab80 1 -- 172.21.15.1:0/3514680871 >> 172.21.15.1:0/3514680871 conn(0x559eeb76cdb0 msgr2=0x559eeb823a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:52.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.601+0000 7fb08ef7ab80 1 -- 172.21.15.1:0/3514680871 shutdown_connections 2024-02-02T15:31:52.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.601+0000 7fb08ef7ab80 1 -- 172.21.15.1:0/3514680871 wait complete. 2024-02-02T15:31:52.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.601+0000 7fb08ef7ab80 20 client.26915 trim_cache size 0 max 16384 2024-02-02T15:31:52.609 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:31:52.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.601+0000 7fb079e00700 1 -- 172.21.15.1:0/3525727358 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fb0540a0690 msgr2=0x7fb0540a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:52.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.601+0000 7fb079e00700 1 --2- 172.21.15.1:0/3525727358 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fb0540a0690 0x7fb0540a2b50 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb06c009510 tx=0x7fb06c009920 comp rx=0 tx=0).stop 2024-02-02T15:31:52.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.601+0000 7fb079e00700 1 -- 172.21.15.1:0/3525727358 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb074073a90 msgr2=0x7fb074199300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:52.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.601+0000 7fb079e00700 1 --2- 172.21.15.1:0/3525727358 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb074073a90 0x7fb074199300 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb05c00e9a0 tx=0x7fb05c00ecb0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.601+0000 7fb079e00700 1 -- 172.21.15.1:0/3525727358 shutdown_connections 2024-02-02T15:31:52.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.601+0000 7fb079e00700 1 --2- 172.21.15.1:0/3525727358 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fb0540a0690 0x7fb0540a2b50 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.601+0000 7fb079e00700 1 --2- 172.21.15.1:0/3525727358 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb07410d250 0x7fb07419d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.601+0000 7fb079e00700 1 --2- 172.21.15.1:0/3525727358 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb074073170 0x7fb074198dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.601+0000 7fb079e00700 1 --2- 172.21.15.1:0/3525727358 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb074073a90 0x7fb074199300 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.601+0000 7fb079e00700 1 -- 172.21.15.1:0/3525727358 >> 172.21.15.1:0/3525727358 conn(0x7fb0740fc540 msgr2=0x7fb07410f480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:52.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.601+0000 7fb079e00700 1 -- 172.21.15.1:0/3525727358 shutdown_connections 2024-02-02T15:31:52.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.601+0000 7fb079e00700 1 -- 172.21.15.1:0/3525727358 wait complete. 2024-02-02T15:31:52.617 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:31:52 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/273726625' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:31:52.620 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-02T15:31:52.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.833+0000 7effab000700 1 -- 172.21.15.1:0/1816851028 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7effa410d320 msgr2=0x7effa410f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:52.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.833+0000 7effab000700 1 --2- 172.21.15.1:0/1816851028 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7effa410d320 0x7effa410f750 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7eff9c009f90 tx=0x7eff9c00b330 comp rx=0 tx=0).stop 2024-02-02T15:31:52.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.833+0000 7effab000700 1 -- 172.21.15.1:0/1816851028 shutdown_connections 2024-02-02T15:31:52.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.833+0000 7effab000700 1 --2- 172.21.15.1:0/1816851028 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7effa410d320 0x7effa410f750 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.833+0000 7effab000700 1 --2- 172.21.15.1:0/1816851028 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7effa4073110 0x7effa40734f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.833+0000 7effab000700 1 --2- 172.21.15.1:0/1816851028 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7effa4073a30 0x7effa410cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.833+0000 7effab000700 1 -- 172.21.15.1:0/1816851028 >> 172.21.15.1:0/1816851028 conn(0x7effa40fc570 msgr2=0x7effa40fe990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:52.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.834+0000 7effab000700 1 -- 172.21.15.1:0/1816851028 shutdown_connections 2024-02-02T15:31:52.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.834+0000 7effab000700 1 -- 172.21.15.1:0/1816851028 wait complete. 2024-02-02T15:31:52.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.835+0000 7effab000700 1 Processor -- start 2024-02-02T15:31:52.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.835+0000 7effab000700 1 -- start start 2024-02-02T15:31:52.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.836+0000 7effab000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7effa4073110 0x7effa4198ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:52.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.836+0000 7effab000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7effa4073a30 0x7effa4199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:52.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.836+0000 7effab000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7effa410d320 0x7effa419d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:52.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.836+0000 7effab000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effa419dec0 con 0x7effa4073110 2024-02-02T15:31:52.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.836+0000 7effab000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effa419e030 con 0x7effa410d320 2024-02-02T15:31:52.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.836+0000 7effab000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effa419e1f0 con 0x7effa4073a30 2024-02-02T15:31:52.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.836+0000 7effa9c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7effa4073110 0x7effa4198ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:52.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.836+0000 7effa9c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7effa4073110 0x7effa4198ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:35204/0 (socket says 172.21.15.1:35204) 2024-02-02T15:31:52.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.836+0000 7effa9c00700 1 -- 172.21.15.1:0/250353856 learned_addr learned my addr 172.21.15.1:0/250353856 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:31:52.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.836+0000 7effa9200700 1 --2- 172.21.15.1:0/250353856 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7effa4073a30 0x7effa4199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:52.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.836+0000 7effaa600700 1 --2- 172.21.15.1:0/250353856 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7effa410d320 0x7effa419d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:52.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.837+0000 7effa9200700 1 -- 172.21.15.1:0/250353856 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7effa410d320 msgr2=0x7effa419d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:52.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.837+0000 7effa9200700 1 --2- 172.21.15.1:0/250353856 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7effa410d320 0x7effa419d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.837+0000 7effa9200700 1 -- 172.21.15.1:0/250353856 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7effa4073110 msgr2=0x7effa4198ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:52.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.837+0000 7effa9200700 1 --2- 172.21.15.1:0/250353856 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7effa4073110 0x7effa4198ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.837+0000 7effa9200700 1 -- 172.21.15.1:0/250353856 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff9c009c40 con 0x7effa4073a30 2024-02-02T15:31:52.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.837+0000 7effaa600700 1 --2- 172.21.15.1:0/250353856 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7effa410d320 0x7effa419d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:31:52.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.837+0000 7effa9c00700 1 --2- 172.21.15.1:0/250353856 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7effa4073110 0x7effa4198ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:31:52.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.837+0000 7effa9200700 1 --2- 172.21.15.1:0/250353856 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7effa4073a30 0x7effa4199420 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7eff8c00e9f0 tx=0x7eff8c00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:52.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.837+0000 7effa2000700 1 -- 172.21.15.1:0/250353856 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff8c00c9c0 con 0x7effa4073a30 2024-02-02T15:31:52.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.837+0000 7effa2000700 1 -- 172.21.15.1:0/250353856 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff8c010430 con 0x7effa4073a30 2024-02-02T15:31:52.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.837+0000 7effab000700 1 -- 172.21.15.1:0/250353856 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effa419e4d0 con 0x7effa4073a30 2024-02-02T15:31:52.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.837+0000 7effa2000700 1 -- 172.21.15.1:0/250353856 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff8c018680 con 0x7effa4073a30 2024-02-02T15:31:52.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.837+0000 7effab000700 1 -- 172.21.15.1:0/250353856 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effa41a5d10 con 0x7effa4073a30 2024-02-02T15:31:52.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.840+0000 7effa2000700 1 -- 172.21.15.1:0/250353856 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 27) v1 ==== 135200+0+0 (secure 0 0 0) 0x7eff8c010860 con 0x7effa4073a30 2024-02-02T15:31:52.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.842+0000 7effa2000700 1 --2- 172.21.15.1:0/250353856 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7eff840a0710 0x7eff840a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:52.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.842+0000 7effa2000700 1 -- 172.21.15.1:0/250353856 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7eff8c014070 con 0x7effa4073a30 2024-02-02T15:31:52.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.842+0000 7effc01d3b80 1 Processor -- start 2024-02-02T15:31:52.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.842+0000 7effa9c00700 1 --2- 172.21.15.1:0/250353856 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7eff840a0710 0x7eff840a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:52.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.842+0000 7effc01d3b80 1 -- start start 2024-02-02T15:31:52.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.843+0000 7effc01d3b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637b6e161f0 0x5637b6e165d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:52.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.843+0000 7effc01d3b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5637b6e16b10 0x5637b6e214b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:52.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.843+0000 7effc01d3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5637b6e219f0 0x5637b6e23de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:52.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.843+0000 7effc01d3b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637b6d8a4c0 con 0x5637b6e16b10 2024-02-02T15:31:52.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.843+0000 7effc01d3b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637b6d6eaa0 con 0x5637b6e161f0 2024-02-02T15:31:52.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.843+0000 7effc01d3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637b6d6c460 con 0x5637b6e219f0 2024-02-02T15:31:52.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.843+0000 7effa9c00700 1 --2- 172.21.15.1:0/250353856 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7eff840a0710 0x7eff840a2bd0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7eff94005950 tx=0x7eff94009310 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:52.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.843+0000 7effa9c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637b6e161f0 0x5637b6e165d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:52.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.843+0000 7effa9c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637b6e161f0 0x5637b6e165d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:58864/0 (socket says 172.21.15.1:58864) 2024-02-02T15:31:52.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.843+0000 7effa9c00700 1 -- 172.21.15.1:0/1898181505 learned_addr learned my addr 172.21.15.1:0/1898181505 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:31:52.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.843+0000 7effaa600700 1 --2- 172.21.15.1:0/1898181505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5637b6e219f0 0x5637b6e23de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:52.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.843+0000 7effa9200700 1 --2- 172.21.15.1:0/1898181505 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5637b6e16b10 0x5637b6e214b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:52.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.843+0000 7effa9c00700 1 -- 172.21.15.1:0/1898181505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5637b6e219f0 msgr2=0x5637b6e23de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:52.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.843+0000 7effa9c00700 1 --2- 172.21.15.1:0/1898181505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5637b6e219f0 0x5637b6e23de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.843+0000 7effa9c00700 1 -- 172.21.15.1:0/1898181505 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5637b6e16b10 msgr2=0x5637b6e214b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:52.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.843+0000 7effa9c00700 1 --2- 172.21.15.1:0/1898181505 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5637b6e16b10 0x5637b6e214b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.843+0000 7effa9c00700 1 -- 172.21.15.1:0/1898181505 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5637b6be93b0 con 0x5637b6e161f0 2024-02-02T15:31:52.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.844+0000 7effa9c00700 1 --2- 172.21.15.1:0/1898181505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637b6e161f0 0x5637b6e165d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7eff9400e6b0 tx=0x7eff94014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5209d0bebca96ef6 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:52.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.844+0000 7eff9aa00700 1 -- 172.21.15.1:0/1898181505 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff94009df0 con 0x5637b6e161f0 2024-02-02T15:31:52.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.844+0000 7eff9aa00700 1 -- 172.21.15.1:0/1898181505 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff94021070 con 0x5637b6e161f0 2024-02-02T15:31:52.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.844+0000 7eff9aa00700 1 -- 172.21.15.1:0/1898181505 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff940257b0 con 0x5637b6e161f0 2024-02-02T15:31:52.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.845+0000 7effc01d3b80 1 -- 172.21.15.1:0/1898181505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637b6e161f0 msgr2=0x5637b6e165d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:52.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.845+0000 7effc01d3b80 1 --2- 172.21.15.1:0/1898181505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637b6e161f0 0x5637b6e165d0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7eff9400e6b0 tx=0x7eff94014000 comp rx=0 tx=0).stop 2024-02-02T15:31:52.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.845+0000 7effc01d3b80 1 -- 172.21.15.1:0/1898181505 shutdown_connections 2024-02-02T15:31:52.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.845+0000 7effc01d3b80 1 --2- 172.21.15.1:0/1898181505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5637b6e219f0 0x5637b6e23de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.845+0000 7effc01d3b80 1 --2- 172.21.15.1:0/1898181505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637b6e161f0 0x5637b6e165d0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.845+0000 7effc01d3b80 1 --2- 172.21.15.1:0/1898181505 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5637b6e16b10 0x5637b6e214b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.845+0000 7effc01d3b80 1 -- 172.21.15.1:0/1898181505 >> 172.21.15.1:0/1898181505 conn(0x5637b6d75db0 msgr2=0x5637b6e125c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:52.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.845+0000 7effc01d3b80 1 -- 172.21.15.1:0/1898181505 shutdown_connections 2024-02-02T15:31:52.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.845+0000 7effc01d3b80 1 -- 172.21.15.1:0/1898181505 wait complete. 2024-02-02T15:31:52.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.845+0000 7effc01d3b80 1 -- start start 2024-02-02T15:31:52.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.846+0000 7effc01d3b80 1 Processor -- start 2024-02-02T15:31:52.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.846+0000 7effc01d3b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5637b6e161f0 0x5637b6e213e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:52.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.846+0000 7effc01d3b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637b6e16b10 0x5637b6e1e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:52.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.846+0000 7effc01d3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5637b6e219f0 0x5637b6e1ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:52.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.846+0000 7effc01d3b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637b6d67d50 con 0x5637b6e161f0 2024-02-02T15:31:52.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.846+0000 7effc01d3b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637b6d03d80 con 0x5637b6e16b10 2024-02-02T15:31:52.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.846+0000 7effc01d3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637b6d03a20 con 0x5637b6e219f0 2024-02-02T15:31:52.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.846+0000 7effc01d3b80 10 client.0 _ll_register_callbacks cb 0x5637b6d05290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:31:52.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.846+0000 7effa9c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5637b6e161f0 0x5637b6e213e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:52.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.846+0000 7effaa600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5637b6e219f0 0x5637b6e1ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:52.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.847+0000 7effaa600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5637b6e219f0 0x5637b6e1ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:43744/0 (socket says 172.21.15.1:43744) 2024-02-02T15:31:52.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.847+0000 7effaa600700 1 -- 172.21.15.1:0/293230611 learned_addr learned my addr 172.21.15.1:0/293230611 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:31:52.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.847+0000 7effa9200700 1 --2- 172.21.15.1:0/293230611 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637b6e16b10 0x5637b6e1e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:52.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.847+0000 7effaa600700 1 -- 172.21.15.1:0/293230611 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637b6e16b10 msgr2=0x5637b6e1e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:52.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.847+0000 7effaa600700 1 --2- 172.21.15.1:0/293230611 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637b6e16b10 0x5637b6e1e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.847+0000 7effaa600700 1 -- 172.21.15.1:0/293230611 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5637b6e161f0 msgr2=0x5637b6e213e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:52.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.847+0000 7effaa600700 1 --2- 172.21.15.1:0/293230611 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5637b6e161f0 0x5637b6e213e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.847+0000 7effaa600700 1 -- 172.21.15.1:0/293230611 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff94017070 con 0x5637b6e219f0 2024-02-02T15:31:52.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.847+0000 7effaa600700 1 --2- 172.21.15.1:0/293230611 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5637b6e219f0 0x5637b6e1ec70 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7eff9c003c30 tx=0x7eff9c003c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6c74ec172ede9d server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:52.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.847+0000 7eff93400700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:31:52.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.847+0000 7eff93400700 1 -- 172.21.15.1:0/293230611 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9c00e040 con 0x5637b6e219f0 2024-02-02T15:31:52.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.848+0000 7eff93400700 1 -- 172.21.15.1:0/293230611 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7eff9c00b9f0 con 0x5637b6e219f0 2024-02-02T15:31:52.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.848+0000 7effc01d3b80 1 -- 172.21.15.1:0/293230611 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5637b6e0a340 con 0x5637b6e219f0 2024-02-02T15:31:52.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.848+0000 7eff93400700 1 -- 172.21.15.1:0/293230611 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff9c009ac0 con 0x5637b6e219f0 2024-02-02T15:31:52.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.848+0000 7effc01d3b80 10 client.26926 fetch_fsmap learned FSMap version 35 2024-02-02T15:31:52.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.848+0000 7effc01d3b80 1 -- 172.21.15.1:0/293230611 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5637b6b95b60 con 0x5637b6e219f0 2024-02-02T15:31:52.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.848+0000 7eff93400700 1 -- 172.21.15.1:0/293230611 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5637b6e0a340 con 0x5637b6e219f0 2024-02-02T15:31:52.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.848+0000 7eff93400700 1 -- 172.21.15.1:0/293230611 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 35) v1 ==== 7470+0+0 (secure 0 0 0) 0x7eff9c019070 con 0x5637b6e219f0 2024-02-02T15:31:52.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.848+0000 7effc01d3b80 10 client.26926 fetch_fsmap finished waiting for FSMap version 35 2024-02-02T15:31:52.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.848+0000 7effc01d3b80 10 client.26926 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-02-02T15:31:52.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.848+0000 7effc01d3b80 20 client.26926 populate_metadata read hostname 'smithi001' 2024-02-02T15:31:52.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.848+0000 7effc01d3b80 1 --2- 172.21.15.1:0/293230611 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x5637b6e313a0 0x5637b6e337c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:52.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.848+0000 7effc01d3b80 4 client.26926 mds_command: new command op to 24518 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:31:52.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.848+0000 7effc01d3b80 1 -- 172.21.15.1:0/293230611 --> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5637b6d60dd0 con 0x5637b6e313a0 2024-02-02T15:31:52.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.849+0000 7effa9c00700 1 --2- 172.21.15.1:0/293230611 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x5637b6e313a0 0x5637b6e337c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:52.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.849+0000 7effa9c00700 1 --2- 172.21.15.1:0/293230611 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x5637b6e313a0 0x5637b6e337c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=45ebfb1e77b880a3 server_cookie=c4900df608cc4539 in_seq=0 out_seq=0 2024-02-02T15:31:52.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.849+0000 7eff93400700 10 client.26926 ms_handle_connect on v2:172.21.15.71:6836/628016199 2024-02-02T15:31:52.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.850+0000 7eff93400700 1 -- 172.21.15.1:0/293230611 <== mds.2 v2:172.21.15.71:6836/628016199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5637b6d60dd0 con 0x5637b6e313a0 2024-02-02T15:31:52.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.850+0000 7eff93400700 10 client.26926 handle_command_reply: tid=0 2024-02-02T15:31:52.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.850+0000 7effc01d3b80 1 client.26926 shutdown 2024-02-02T15:31:52.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.850+0000 7effc01d3b80 1 -- 172.21.15.1:0/293230611 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5637b6e219f0 msgr2=0x5637b6e1ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:52.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.850+0000 7effc01d3b80 1 --2- 172.21.15.1:0/293230611 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5637b6e219f0 0x5637b6e1ec70 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7eff9c003c30 tx=0x7eff9c003c60 comp rx=0 tx=0).stop 2024-02-02T15:31:52.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.850+0000 7effc01d3b80 1 -- 172.21.15.1:0/293230611 shutdown_connections 2024-02-02T15:31:52.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.850+0000 7effc01d3b80 1 --2- 172.21.15.1:0/293230611 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x5637b6e313a0 0x5637b6e337c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.850+0000 7effc01d3b80 1 --2- 172.21.15.1:0/293230611 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5637b6e219f0 0x5637b6e1ec70 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.850+0000 7effc01d3b80 1 --2- 172.21.15.1:0/293230611 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5637b6e161f0 0x5637b6e213e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.850+0000 7effc01d3b80 1 --2- 172.21.15.1:0/293230611 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637b6e16b10 0x5637b6e1e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.850+0000 7eff93400700 0 client.26926 ms_handle_reset on v2:172.21.15.71:6836/628016199 2024-02-02T15:31:52.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.850+0000 7effc01d3b80 1 -- 172.21.15.1:0/293230611 >> 172.21.15.1:0/293230611 conn(0x5637b6d75db0 msgr2=0x5637b6e12150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:52.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.850+0000 7effc01d3b80 1 -- 172.21.15.1:0/293230611 shutdown_connections 2024-02-02T15:31:52.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.850+0000 7effc01d3b80 1 -- 172.21.15.1:0/293230611 wait complete. 2024-02-02T15:31:52.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.851+0000 7effc01d3b80 20 client.26926 trim_cache size 0 max 16384 2024-02-02T15:31:52.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:31:52 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/273726625' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:31:52.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.865+0000 7effc01d3b80 1 Processor -- start 2024-02-02T15:31:52.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.866+0000 7effc01d3b80 1 -- start start 2024-02-02T15:31:52.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.866+0000 7effc01d3b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637b6e161f0 0x5637b6e28500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:52.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.866+0000 7effc01d3b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5637b6e16b10 0x5637b6e20f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:52.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.866+0000 7effc01d3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5637b6e219f0 0x5637b6e19950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:52.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.866+0000 7effc01d3b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637b6beb6a0 con 0x5637b6e16b10 2024-02-02T15:31:52.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.866+0000 7effc01d3b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637b6e0c780 con 0x5637b6e161f0 2024-02-02T15:31:52.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.866+0000 7effc01d3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637b6e1cbf0 con 0x5637b6e219f0 2024-02-02T15:31:52.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.866+0000 7effa9c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637b6e161f0 0x5637b6e28500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:52.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.866+0000 7effa9c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637b6e161f0 0x5637b6e28500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:58892/0 (socket says 172.21.15.1:58892) 2024-02-02T15:31:52.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.866+0000 7effa9c00700 1 -- 172.21.15.1:0/1125442166 learned_addr learned my addr 172.21.15.1:0/1125442166 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:31:52.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.866+0000 7effaa600700 1 --2- 172.21.15.1:0/1125442166 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5637b6e219f0 0x5637b6e19950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:52.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.867+0000 7effa9200700 1 --2- 172.21.15.1:0/1125442166 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5637b6e16b10 0x5637b6e20f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:52.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.867+0000 7effa9c00700 1 -- 172.21.15.1:0/1125442166 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5637b6e219f0 msgr2=0x5637b6e19950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:52.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.867+0000 7effa9c00700 1 --2- 172.21.15.1:0/1125442166 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5637b6e219f0 0x5637b6e19950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.867+0000 7effa9c00700 1 -- 172.21.15.1:0/1125442166 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5637b6e16b10 msgr2=0x5637b6e20f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:52.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.867+0000 7effa9c00700 1 --2- 172.21.15.1:0/1125442166 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5637b6e16b10 0x5637b6e20f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.867+0000 7effa9c00700 1 -- 172.21.15.1:0/1125442166 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5637b6d2dea0 con 0x5637b6e161f0 2024-02-02T15:31:52.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.867+0000 7effa9200700 1 --2- 172.21.15.1:0/1125442166 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5637b6e16b10 0x5637b6e20f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:31:52.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.867+0000 7effa9c00700 1 --2- 172.21.15.1:0/1125442166 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637b6e161f0 0x5637b6e28500 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7eff9400eca0 tx=0x7eff9401de40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14483be55388174c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:52.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.867+0000 7eff9b400700 1 -- 172.21.15.1:0/1125442166 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff940331b0 con 0x5637b6e161f0 2024-02-02T15:31:52.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.867+0000 7eff9b400700 1 -- 172.21.15.1:0/1125442166 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff94033310 con 0x5637b6e161f0 2024-02-02T15:31:52.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.867+0000 7eff9b400700 1 -- 172.21.15.1:0/1125442166 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff94033b70 con 0x5637b6e161f0 2024-02-02T15:31:52.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.868+0000 7effc01d3b80 1 -- 172.21.15.1:0/1125442166 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637b6e161f0 msgr2=0x5637b6e28500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:52.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.868+0000 7effc01d3b80 1 --2- 172.21.15.1:0/1125442166 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637b6e161f0 0x5637b6e28500 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7eff9400eca0 tx=0x7eff9401de40 comp rx=0 tx=0).stop 2024-02-02T15:31:52.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.868+0000 7effc01d3b80 1 -- 172.21.15.1:0/1125442166 shutdown_connections 2024-02-02T15:31:52.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.868+0000 7effc01d3b80 1 --2- 172.21.15.1:0/1125442166 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5637b6e219f0 0x5637b6e19950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.868+0000 7effc01d3b80 1 --2- 172.21.15.1:0/1125442166 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637b6e161f0 0x5637b6e28500 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.868+0000 7effc01d3b80 1 --2- 172.21.15.1:0/1125442166 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5637b6e16b10 0x5637b6e20f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.868+0000 7effc01d3b80 1 -- 172.21.15.1:0/1125442166 >> 172.21.15.1:0/1125442166 conn(0x5637b6d75db0 msgr2=0x5637b6e10d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:52.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.868+0000 7effc01d3b80 1 -- 172.21.15.1:0/1125442166 shutdown_connections 2024-02-02T15:31:52.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.868+0000 7effc01d3b80 1 -- 172.21.15.1:0/1125442166 wait complete. 2024-02-02T15:31:52.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.869+0000 7effc01d3b80 1 -- start start 2024-02-02T15:31:52.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.869+0000 7effc01d3b80 1 Processor -- start 2024-02-02T15:31:52.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.869+0000 7effc01d3b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637b6e161f0 0x5637b6e7c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:52.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.869+0000 7effc01d3b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5637b6e16b10 0x5637b6e7c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:52.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.869+0000 7effc01d3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5637b6e219f0 0x5637b6e244e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:52.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.869+0000 7effc01d3b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637b6e24ab0 con 0x5637b6e16b10 2024-02-02T15:31:52.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.869+0000 7effc01d3b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637b6e24c20 con 0x5637b6e161f0 2024-02-02T15:31:52.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.869+0000 7effc01d3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637b6e24de0 con 0x5637b6e219f0 2024-02-02T15:31:52.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.869+0000 7effc01d3b80 10 client.0 _ll_register_callbacks cb 0x5637b6d05290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:31:52.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.870+0000 7effa9c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637b6e161f0 0x5637b6e7c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:52.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.870+0000 7effa9c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637b6e161f0 0x5637b6e7c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:58904/0 (socket says 172.21.15.1:58904) 2024-02-02T15:31:52.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.870+0000 7effa9c00700 1 -- 172.21.15.1:0/1924688340 learned_addr learned my addr 172.21.15.1:0/1924688340 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:31:52.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.870+0000 7effaa600700 1 --2- 172.21.15.1:0/1924688340 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5637b6e219f0 0x5637b6e244e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:52.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.870+0000 7effa9200700 1 --2- 172.21.15.1:0/1924688340 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5637b6e16b10 0x5637b6e7c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:52.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.870+0000 7effa9c00700 1 -- 172.21.15.1:0/1924688340 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5637b6e219f0 msgr2=0x5637b6e244e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:52.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.870+0000 7effa9c00700 1 --2- 172.21.15.1:0/1924688340 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5637b6e219f0 0x5637b6e244e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.870+0000 7effa9c00700 1 -- 172.21.15.1:0/1924688340 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5637b6e16b10 msgr2=0x5637b6e7c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:52.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.870+0000 7effa9c00700 1 --2- 172.21.15.1:0/1924688340 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5637b6e16b10 0x5637b6e7c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.870+0000 7effa9c00700 1 -- 172.21.15.1:0/1924688340 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff9402a040 con 0x5637b6e161f0 2024-02-02T15:31:52.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.870+0000 7effa9200700 1 --2- 172.21.15.1:0/1924688340 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5637b6e16b10 0x5637b6e7c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:31:52.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.870+0000 7effa9c00700 1 --2- 172.21.15.1:0/1924688340 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637b6e161f0 0x5637b6e7c180 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7eff94025a50 tx=0x7eff9400ed50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f458ad6c026994a server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:52.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.870+0000 7eff93400700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:31:52.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.870+0000 7eff93400700 1 -- 172.21.15.1:0/1924688340 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff940253d0 con 0x5637b6e161f0 2024-02-02T15:31:52.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.870+0000 7eff93400700 1 -- 172.21.15.1:0/1924688340 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7eff94025530 con 0x5637b6e161f0 2024-02-02T15:31:52.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.870+0000 7effc01d3b80 1 -- 172.21.15.1:0/1924688340 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7eff94021070 con 0x5637b6e161f0 2024-02-02T15:31:52.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.871+0000 7eff93400700 1 -- 172.21.15.1:0/1924688340 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff94025b20 con 0x5637b6e161f0 2024-02-02T15:31:52.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.871+0000 7effc01d3b80 10 client.26933 fetch_fsmap learned FSMap version 35 2024-02-02T15:31:52.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.871+0000 7effc01d3b80 1 -- 172.21.15.1:0/1924688340 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5637b6bedcd0 con 0x5637b6e161f0 2024-02-02T15:31:52.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.871+0000 7eff93400700 1 -- 172.21.15.1:0/1924688340 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff94049020 con 0x5637b6e161f0 2024-02-02T15:31:52.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.871+0000 7eff93400700 1 -- 172.21.15.1:0/1924688340 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 35) v1 ==== 7470+0+0 (secure 0 0 0) 0x7eff94046070 con 0x5637b6e161f0 2024-02-02T15:31:52.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.871+0000 7effc01d3b80 10 client.26933 fetch_fsmap finished waiting for FSMap version 35 2024-02-02T15:31:52.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.871+0000 7effc01d3b80 10 client.26933 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-02-02T15:31:52.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.871+0000 7effc01d3b80 20 client.26933 populate_metadata read hostname 'smithi001' 2024-02-02T15:31:52.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.871+0000 7effc01d3b80 1 --2- 172.21.15.1:0/1924688340 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x5637b6e83af0 0x5637b6e85f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:52.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.871+0000 7effc01d3b80 4 client.26933 mds_command: new command op to 24518 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:31:52.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.871+0000 7effc01d3b80 1 -- 172.21.15.1:0/1924688340 --> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5637b6d26f50 con 0x5637b6e83af0 2024-02-02T15:31:52.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.872+0000 7effa9200700 1 --2- 172.21.15.1:0/1924688340 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x5637b6e83af0 0x5637b6e85f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:52.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.872+0000 7effa9200700 1 --2- 172.21.15.1:0/1924688340 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x5637b6e83af0 0x5637b6e85f10 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f73e66a8e695f799 server_cookie=da73979c4c02594e in_seq=0 out_seq=0 2024-02-02T15:31:52.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.872+0000 7eff93400700 10 client.26933 ms_handle_connect on v2:172.21.15.71:6836/628016199 2024-02-02T15:31:52.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.872+0000 7eff93400700 1 -- 172.21.15.1:0/1924688340 <== mds.2 v2:172.21.15.71:6836/628016199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5637b6d26f50 con 0x5637b6e83af0 2024-02-02T15:31:52.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.872+0000 7eff93400700 10 client.26933 handle_command_reply: tid=0 2024-02-02T15:31:52.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.872+0000 7effc01d3b80 1 client.26933 shutdown 2024-02-02T15:31:52.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.872+0000 7effc01d3b80 1 -- 172.21.15.1:0/1924688340 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637b6e161f0 msgr2=0x5637b6e7c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:52.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.872+0000 7effc01d3b80 1 --2- 172.21.15.1:0/1924688340 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637b6e161f0 0x5637b6e7c180 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7eff94025a50 tx=0x7eff9400ed50 comp rx=0 tx=0).stop 2024-02-02T15:31:52.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.872+0000 7effc01d3b80 1 -- 172.21.15.1:0/1924688340 shutdown_connections 2024-02-02T15:31:52.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.872+0000 7effc01d3b80 1 --2- 172.21.15.1:0/1924688340 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x5637b6e83af0 0x5637b6e85f10 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.872+0000 7effc01d3b80 1 --2- 172.21.15.1:0/1924688340 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5637b6e219f0 0x5637b6e244e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.872+0000 7effc01d3b80 1 --2- 172.21.15.1:0/1924688340 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5637b6e161f0 0x5637b6e7c180 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.872+0000 7eff93400700 0 client.26933 ms_handle_reset on v2:172.21.15.71:6836/628016199 2024-02-02T15:31:52.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.872+0000 7effc01d3b80 1 --2- 172.21.15.1:0/1924688340 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5637b6e16b10 0x5637b6e7c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.872+0000 7effc01d3b80 1 -- 172.21.15.1:0/1924688340 >> 172.21.15.1:0/1924688340 conn(0x5637b6d75db0 msgr2=0x5637b6e19950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:52.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.873+0000 7effc01d3b80 1 -- 172.21.15.1:0/1924688340 shutdown_connections 2024-02-02T15:31:52.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.873+0000 7effc01d3b80 1 -- 172.21.15.1:0/1924688340 wait complete. 2024-02-02T15:31:52.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.873+0000 7effc01d3b80 20 client.26933 trim_cache size 0 max 16384 2024-02-02T15:31:52.876 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:31:52.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.873+0000 7effab000700 1 -- 172.21.15.1:0/250353856 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7eff840a0710 msgr2=0x7eff840a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:52.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.873+0000 7effab000700 1 --2- 172.21.15.1:0/250353856 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7eff840a0710 0x7eff840a2bd0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7eff94005950 tx=0x7eff94009310 comp rx=0 tx=0).stop 2024-02-02T15:31:52.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.873+0000 7effab000700 1 -- 172.21.15.1:0/250353856 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7effa4073a30 msgr2=0x7effa4199420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:52.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.873+0000 7effab000700 1 --2- 172.21.15.1:0/250353856 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7effa4073a30 0x7effa4199420 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7eff8c00e9f0 tx=0x7eff8c00ed00 comp rx=0 tx=0).stop 2024-02-02T15:31:52.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.874+0000 7effab000700 1 -- 172.21.15.1:0/250353856 shutdown_connections 2024-02-02T15:31:52.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.874+0000 7effab000700 1 --2- 172.21.15.1:0/250353856 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7eff840a0710 0x7eff840a2bd0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.874+0000 7effab000700 1 --2- 172.21.15.1:0/250353856 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7effa410d320 0x7effa419d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.874+0000 7effab000700 1 --2- 172.21.15.1:0/250353856 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7effa4073110 0x7effa4198ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.874+0000 7effab000700 1 --2- 172.21.15.1:0/250353856 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7effa4073a30 0x7effa4199420 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:52.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.874+0000 7effab000700 1 -- 172.21.15.1:0/250353856 >> 172.21.15.1:0/250353856 conn(0x7effa40fc570 msgr2=0x7effa410f430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:52.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.874+0000 7effab000700 1 -- 172.21.15.1:0/250353856 shutdown_connections 2024-02-02T15:31:52.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:52.874+0000 7effab000700 1 -- 172.21.15.1:0/250353856 wait complete. 2024-02-02T15:31:52.892 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-02T15:31:52.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:31:52 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/273726625' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:31:53.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.105+0000 7f0828c00700 1 -- 172.21.15.1:0/4268647672 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0824100ff0 msgr2=0x7f08241013d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:53.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.105+0000 7f0828c00700 1 --2- 172.21.15.1:0/4268647672 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0824100ff0 0x7f08241013d0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f0814009990 tx=0x7f0814009ca0 comp rx=0 tx=0).stop 2024-02-02T15:31:53.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.105+0000 7f0828c00700 1 -- 172.21.15.1:0/4268647672 shutdown_connections 2024-02-02T15:31:53.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.105+0000 7f0828c00700 1 --2- 172.21.15.1:0/4268647672 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0824106110 0x7f082410dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:53.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.105+0000 7f0828c00700 1 --2- 172.21.15.1:0/4268647672 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0824100ff0 0x7f08241013d0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:53.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.105+0000 7f0828c00700 1 --2- 172.21.15.1:0/4268647672 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f08241019a0 0x7f08241059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:53.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.105+0000 7f0828c00700 1 -- 172.21.15.1:0/4268647672 >> 172.21.15.1:0/4268647672 conn(0x7f08240fc810 msgr2=0x7f08240fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:53.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.105+0000 7f0828c00700 1 -- 172.21.15.1:0/4268647672 shutdown_connections 2024-02-02T15:31:53.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.105+0000 7f0828c00700 1 -- 172.21.15.1:0/4268647672 wait complete. 2024-02-02T15:31:53.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.106+0000 7f0828c00700 1 Processor -- start 2024-02-02T15:31:53.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.107+0000 7f0828c00700 1 -- start start 2024-02-02T15:31:53.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.107+0000 7f0828c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0824100ff0 0x7f0824196e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:53.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.107+0000 7f0828c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0824106110 0x7f08241973d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:53.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.107+0000 7f0828c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f082419b830 0x7f082419bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:53.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.107+0000 7f0828c00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f082419c2b0 con 0x7f0824100ff0 2024-02-02T15:31:53.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.107+0000 7f0828c00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f082419c420 con 0x7f0824106110 2024-02-02T15:31:53.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.107+0000 7f0828c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f082419c5e0 con 0x7f082419b830 2024-02-02T15:31:53.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.107+0000 7f081be00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0824106110 0x7f08241973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:53.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.107+0000 7f081be00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0824106110 0x7f08241973d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59442/0 (socket says 172.21.15.1:59442) 2024-02-02T15:31:53.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.107+0000 7f081be00700 1 -- 172.21.15.1:0/964120010 learned_addr learned my addr 172.21.15.1:0/964120010 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:31:53.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.107+0000 7f0822800700 1 --2- 172.21.15.1:0/964120010 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f082419b830 0x7f082419bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:53.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.107+0000 7f0821e00700 1 --2- 172.21.15.1:0/964120010 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0824100ff0 0x7f0824196e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:53.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.108+0000 7f0822800700 1 -- 172.21.15.1:0/964120010 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0824106110 msgr2=0x7f08241973d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:53.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.108+0000 7f0822800700 1 --2- 172.21.15.1:0/964120010 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0824106110 0x7f08241973d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:53.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.108+0000 7f0822800700 1 -- 172.21.15.1:0/964120010 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0824100ff0 msgr2=0x7f0824196e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:53.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.108+0000 7f0822800700 1 --2- 172.21.15.1:0/964120010 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0824100ff0 0x7f0824196e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:53.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.108+0000 7f0822800700 1 -- 172.21.15.1:0/964120010 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08140095f0 con 0x7f082419b830 2024-02-02T15:31:53.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.108+0000 7f081be00700 1 --2- 172.21.15.1:0/964120010 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0824106110 0x7f08241973d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:31:53.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.108+0000 7f0822800700 1 --2- 172.21.15.1:0/964120010 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f082419b830 0x7f082419bcb0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f081c00efc0 tx=0x7f081c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:53.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.108+0000 7f081aa00700 1 -- 172.21.15.1:0/964120010 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f081c018070 con 0x7f082419b830 2024-02-02T15:31:53.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.108+0000 7f0828c00700 1 -- 172.21.15.1:0/964120010 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08241a3ae0 con 0x7f082419b830 2024-02-02T15:31:53.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.108+0000 7f081aa00700 1 -- 172.21.15.1:0/964120010 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f081c00f040 con 0x7f082419b830 2024-02-02T15:31:53.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.108+0000 7f081aa00700 1 -- 172.21.15.1:0/964120010 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f081c00cd40 con 0x7f082419b830 2024-02-02T15:31:53.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.108+0000 7f0828c00700 1 -- 172.21.15.1:0/964120010 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08241a4010 con 0x7f082419b830 2024-02-02T15:31:53.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.111+0000 7f081aa00700 1 -- 172.21.15.1:0/964120010 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 27) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f081c003a00 con 0x7f082419b830 2024-02-02T15:31:53.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.113+0000 7f081aa00700 1 --2- 172.21.15.1:0/964120010 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f08040a0690 0x7f08040a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:53.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.113+0000 7f081aa00700 1 -- 172.21.15.1:0/964120010 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f081c0cbc00 con 0x7f082419b830 2024-02-02T15:31:53.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.113+0000 7f083c7f4b80 1 Processor -- start 2024-02-02T15:31:53.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.113+0000 7f0821e00700 1 --2- 172.21.15.1:0/964120010 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f08040a0690 0x7f08040a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:53.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.113+0000 7f083c7f4b80 1 -- start start 2024-02-02T15:31:53.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.114+0000 7f083c7f4b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556330ae3010 0x556330ae33f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:53.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.114+0000 7f083c7f4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556330ae3930 0x556330aee2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:53.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.114+0000 7f083c7f4b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556330aee7e0 0x556330af0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:53.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.114+0000 7f083c7f4b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x556330a574c0 con 0x556330aee7e0 2024-02-02T15:31:53.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.114+0000 7f083c7f4b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x556330a3baa0 con 0x556330ae3010 2024-02-02T15:31:53.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.114+0000 7f083c7f4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556330a39460 con 0x556330ae3930 2024-02-02T15:31:53.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.114+0000 7f0821e00700 1 --2- 172.21.15.1:0/964120010 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f08040a0690 0x7f08040a2b50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f081400b2e0 tx=0x7f0814005960 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:53.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.114+0000 7f0822800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556330aee7e0 0x556330af0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:53.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.114+0000 7f0821e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556330ae3010 0x556330ae33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:53.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.114+0000 7f081be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556330ae3930 0x556330aee2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:53.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.114+0000 7f081be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556330ae3930 0x556330aee2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:47732/0 (socket says 172.21.15.1:47732) 2024-02-02T15:31:53.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.114+0000 7f0822800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556330aee7e0 0x556330af0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45566/0 (socket says 172.21.15.1:45566) 2024-02-02T15:31:53.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.114+0000 7f0821e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556330ae3010 0x556330ae33f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59454/0 (socket says 172.21.15.1:59454) 2024-02-02T15:31:53.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.114+0000 7f081be00700 1 -- 172.21.15.1:0/4009213928 learned_addr learned my addr 172.21.15.1:0/4009213928 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:31:53.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.115+0000 7f081be00700 1 -- 172.21.15.1:0/4009213928 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556330ae3010 msgr2=0x556330ae33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:53.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.115+0000 7f081be00700 1 --2- 172.21.15.1:0/4009213928 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556330ae3010 0x556330ae33f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:53.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.115+0000 7f081be00700 1 -- 172.21.15.1:0/4009213928 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556330aee7e0 msgr2=0x556330af0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:53.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.115+0000 7f081be00700 1 --2- 172.21.15.1:0/4009213928 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556330aee7e0 0x556330af0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:53.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.115+0000 7f081be00700 1 -- 172.21.15.1:0/4009213928 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5563308b63b0 con 0x556330ae3930 2024-02-02T15:31:53.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.115+0000 7f081be00700 1 --2- 172.21.15.1:0/4009213928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556330ae3930 0x556330aee2a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f080c011680 tx=0x7f080c009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad75f4338011b081 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:53.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.115+0000 7f0813000700 1 -- 172.21.15.1:0/4009213928 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f080c019410 con 0x556330ae3930 2024-02-02T15:31:53.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.115+0000 7f0813000700 1 -- 172.21.15.1:0/4009213928 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f080c019a30 con 0x556330ae3930 2024-02-02T15:31:53.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.115+0000 7f0813000700 1 -- 172.21.15.1:0/4009213928 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f080c00eaf0 con 0x556330ae3930 2024-02-02T15:31:53.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.116+0000 7f083c7f4b80 1 -- 172.21.15.1:0/4009213928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556330ae3930 msgr2=0x556330aee2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:53.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.116+0000 7f083c7f4b80 1 --2- 172.21.15.1:0/4009213928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556330ae3930 0x556330aee2a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f080c011680 tx=0x7f080c009f90 comp rx=0 tx=0).stop 2024-02-02T15:31:53.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.116+0000 7f083c7f4b80 1 -- 172.21.15.1:0/4009213928 shutdown_connections 2024-02-02T15:31:53.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.116+0000 7f083c7f4b80 1 --2- 172.21.15.1:0/4009213928 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556330aee7e0 0x556330af0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:53.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.116+0000 7f083c7f4b80 1 --2- 172.21.15.1:0/4009213928 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556330ae3010 0x556330ae33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:53.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.116+0000 7f083c7f4b80 1 --2- 172.21.15.1:0/4009213928 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556330ae3930 0x556330aee2a0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:53.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.116+0000 7f083c7f4b80 1 -- 172.21.15.1:0/4009213928 >> 172.21.15.1:0/4009213928 conn(0x556330a42db0 msgr2=0x556330adf3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:53.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.116+0000 7f083c7f4b80 1 -- 172.21.15.1:0/4009213928 shutdown_connections 2024-02-02T15:31:53.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.116+0000 7f083c7f4b80 1 -- 172.21.15.1:0/4009213928 wait complete. 2024-02-02T15:31:53.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.117+0000 7f083c7f4b80 1 -- start start 2024-02-02T15:31:53.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.117+0000 7f083c7f4b80 1 Processor -- start 2024-02-02T15:31:53.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.117+0000 7f083c7f4b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556330ae3010 0x556330aee2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:53.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.117+0000 7f083c7f4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556330ae3930 0x556330aeb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:53.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.118+0000 7f083c7f4b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556330aee7e0 0x556330aebbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:53.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.118+0000 7f083c7f4b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x556330a34d50 con 0x556330aee7e0 2024-02-02T15:31:53.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.118+0000 7f083c7f4b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563309d0d80 con 0x556330ae3010 2024-02-02T15:31:53.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.118+0000 7f083c7f4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563309d0a20 con 0x556330ae3930 2024-02-02T15:31:53.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.118+0000 7f083c7f4b80 10 client.0 _ll_register_callbacks cb 0x5563309d2290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:31:53.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.118+0000 7f081be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556330ae3930 0x556330aeb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:53.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.118+0000 7f0821e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556330ae3010 0x556330aee2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:53.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.118+0000 7f081be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556330ae3930 0x556330aeb4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:47748/0 (socket says 172.21.15.1:47748) 2024-02-02T15:31:53.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.118+0000 7f081be00700 1 -- 172.21.15.1:0/1177763313 learned_addr learned my addr 172.21.15.1:0/1177763313 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:31:53.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.118+0000 7f0822800700 1 --2- 172.21.15.1:0/1177763313 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556330aee7e0 0x556330aebbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:53.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.118+0000 7f0821e00700 1 -- 172.21.15.1:0/1177763313 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556330ae3930 msgr2=0x556330aeb4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:53.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.118+0000 7f0821e00700 1 --2- 172.21.15.1:0/1177763313 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556330ae3930 0x556330aeb4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:53.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.118+0000 7f0821e00700 1 -- 172.21.15.1:0/1177763313 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556330aee7e0 msgr2=0x556330aebbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:53.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.118+0000 7f0821e00700 1 --2- 172.21.15.1:0/1177763313 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556330aee7e0 0x556330aebbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:53.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.118+0000 7f0821e00700 1 -- 172.21.15.1:0/1177763313 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f080c009e00 con 0x556330ae3010 2024-02-02T15:31:53.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.119+0000 7f0821e00700 1 --2- 172.21.15.1:0/1177763313 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556330ae3010 0x556330aee2a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f08140059d0 tx=0x7f0814009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e39c8b186671388b server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:53.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.119+0000 7f080b400700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:31:53.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.119+0000 7f080b400700 1 -- 172.21.15.1:0/1177763313 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0814004dd0 con 0x556330ae3010 2024-02-02T15:31:53.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.119+0000 7f083c7f4b80 1 -- 172.21.15.1:0/1177763313 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556330ad70f0 con 0x556330ae3010 2024-02-02T15:31:53.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.119+0000 7f080b400700 1 -- 172.21.15.1:0/1177763313 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f081400f460 con 0x556330ae3010 2024-02-02T15:31:53.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.119+0000 7f080b400700 1 -- 172.21.15.1:0/1177763313 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f081400f620 con 0x556330ae3010 2024-02-02T15:31:53.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.119+0000 7f083c7f4b80 10 client.26945 fetch_fsmap learned FSMap version 35 2024-02-02T15:31:53.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.119+0000 7f083c7f4b80 1 -- 172.21.15.1:0/1177763313 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x556330862b60 con 0x556330ae3010 2024-02-02T15:31:53.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.119+0000 7f080b400700 1 -- 172.21.15.1:0/1177763313 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556330ad70f0 con 0x556330ae3010 2024-02-02T15:31:53.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.120+0000 7f080b400700 1 -- 172.21.15.1:0/1177763313 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 35) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0814030600 con 0x556330ae3010 2024-02-02T15:31:53.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.120+0000 7f083c7f4b80 10 client.26945 fetch_fsmap finished waiting for FSMap version 35 2024-02-02T15:31:53.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.120+0000 7f083c7f4b80 10 client.26945 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-02T15:31:53.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.120+0000 7f083c7f4b80 20 client.26945 populate_metadata read hostname 'smithi001' 2024-02-02T15:31:53.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.120+0000 7f083c7f4b80 1 --2- 172.21.15.1:0/1177763313 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x556330afe230 0x556330b00650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:53.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.120+0000 7f083c7f4b80 4 client.26945 mds_command: new command op to 24529 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:31:53.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.120+0000 7f083c7f4b80 1 -- 172.21.15.1:0/1177763313 --> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556330a2ddd0 con 0x556330afe230 2024-02-02T15:31:53.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.120+0000 7f081be00700 1 --2- 172.21.15.1:0/1177763313 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x556330afe230 0x556330b00650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:53.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.120+0000 7f081be00700 1 --2- 172.21.15.1:0/1177763313 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x556330afe230 0x556330b00650 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=910565342e5b3d54 server_cookie=cef2bacd87c48cab in_seq=0 out_seq=0 2024-02-02T15:31:53.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.120+0000 7f080b400700 10 client.26945 ms_handle_connect on v2:172.21.15.142:6838/321675421 2024-02-02T15:31:53.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.121+0000 7f080b400700 1 -- 172.21.15.1:0/1177763313 <== mds.1 v2:172.21.15.142:6838/321675421 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x556330a2ddd0 con 0x556330afe230 2024-02-02T15:31:53.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.121+0000 7f080b400700 10 client.26945 handle_command_reply: tid=0 2024-02-02T15:31:53.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.121+0000 7f083c7f4b80 1 client.26945 shutdown 2024-02-02T15:31:53.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.121+0000 7f083c7f4b80 1 -- 172.21.15.1:0/1177763313 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556330ae3010 msgr2=0x556330aee2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:53.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.121+0000 7f083c7f4b80 1 --2- 172.21.15.1:0/1177763313 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556330ae3010 0x556330aee2a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f08140059d0 tx=0x7f0814009ca0 comp rx=0 tx=0).stop 2024-02-02T15:31:53.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.122+0000 7f083c7f4b80 1 -- 172.21.15.1:0/1177763313 shutdown_connections 2024-02-02T15:31:53.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.122+0000 7f083c7f4b80 1 --2- 172.21.15.1:0/1177763313 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x556330afe230 0x556330b00650 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:53.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.122+0000 7f083c7f4b80 1 --2- 172.21.15.1:0/1177763313 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556330aee7e0 0x556330aebbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:53.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.122+0000 7f083c7f4b80 1 --2- 172.21.15.1:0/1177763313 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556330ae3010 0x556330aee2a0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:53.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.122+0000 7f083c7f4b80 1 --2- 172.21.15.1:0/1177763313 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556330ae3930 0x556330aeb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:53.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.122+0000 7f080b400700 0 client.26945 ms_handle_reset on v2:172.21.15.142:6838/321675421 2024-02-02T15:31:53.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.122+0000 7f083c7f4b80 1 -- 172.21.15.1:0/1177763313 >> 172.21.15.1:0/1177763313 conn(0x556330a42db0 msgr2=0x556330adefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:53.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.122+0000 7f083c7f4b80 1 -- 172.21.15.1:0/1177763313 shutdown_connections 2024-02-02T15:31:53.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.122+0000 7f083c7f4b80 1 -- 172.21.15.1:0/1177763313 wait complete. 2024-02-02T15:31:53.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.122+0000 7f083c7f4b80 20 client.26945 trim_cache size 0 max 16384 2024-02-02T15:31:53.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.137+0000 7f083c7f4b80 1 Processor -- start 2024-02-02T15:31:53.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.137+0000 7f083c7f4b80 1 -- start start 2024-02-02T15:31:53.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.137+0000 7f083c7f4b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556330ae3010 0x556330af5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:53.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.137+0000 7f083c7f4b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556330ae3930 0x556330adeeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:53.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.137+0000 7f083c7f4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556330aee7e0 0x556330adf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:53.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.137+0000 7f083c7f4b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563308b86a0 con 0x556330ae3930 2024-02-02T15:31:53.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.137+0000 7f083c7f4b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x556330adf930 con 0x556330ae3010 2024-02-02T15:31:53.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.137+0000 7f083c7f4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556330aec220 con 0x556330aee7e0 2024-02-02T15:31:53.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.138+0000 7f0821e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556330ae3010 0x556330af5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:53.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.138+0000 7f0821e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556330ae3010 0x556330af5330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59472/0 (socket says 172.21.15.1:59472) 2024-02-02T15:31:53.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.138+0000 7f0821e00700 1 -- 172.21.15.1:0/1789389561 learned_addr learned my addr 172.21.15.1:0/1789389561 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:31:53.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.138+0000 7f081be00700 1 --2- 172.21.15.1:0/1789389561 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556330ae3930 0x556330adeeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:53.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.138+0000 7f0822800700 1 --2- 172.21.15.1:0/1789389561 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556330aee7e0 0x556330adf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:53.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.138+0000 7f0821e00700 1 -- 172.21.15.1:0/1789389561 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556330aee7e0 msgr2=0x556330adf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:53.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.138+0000 7f0821e00700 1 --2- 172.21.15.1:0/1789389561 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556330aee7e0 0x556330adf3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:53.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.138+0000 7f0821e00700 1 -- 172.21.15.1:0/1789389561 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556330ae3930 msgr2=0x556330adeeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:53.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.138+0000 7f0821e00700 1 --2- 172.21.15.1:0/1789389561 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556330ae3930 0x556330adeeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:53.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.138+0000 7f0821e00700 1 -- 172.21.15.1:0/1789389561 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5563309faea0 con 0x556330ae3010 2024-02-02T15:31:53.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.138+0000 7f0822800700 1 --2- 172.21.15.1:0/1789389561 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556330aee7e0 0x556330adf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:31:53.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.138+0000 7f0821e00700 1 --2- 172.21.15.1:0/1789389561 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556330ae3010 0x556330af5330 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f0814006010 tx=0x7f0814005a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b1e061113c9c790 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:53.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.139+0000 7f0813a00700 1 -- 172.21.15.1:0/1789389561 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0814030c80 con 0x556330ae3010 2024-02-02T15:31:53.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.139+0000 7f0813a00700 1 -- 172.21.15.1:0/1789389561 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f081401e040 con 0x556330ae3010 2024-02-02T15:31:53.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.139+0000 7f0813a00700 1 -- 172.21.15.1:0/1789389561 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f081400f860 con 0x556330ae3010 2024-02-02T15:31:53.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.139+0000 7f083c7f4b80 1 -- 172.21.15.1:0/1789389561 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556330ae3010 msgr2=0x556330af5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:53.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.139+0000 7f083c7f4b80 1 --2- 172.21.15.1:0/1789389561 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556330ae3010 0x556330af5330 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f0814006010 tx=0x7f0814005a80 comp rx=0 tx=0).stop 2024-02-02T15:31:53.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.139+0000 7f083c7f4b80 1 -- 172.21.15.1:0/1789389561 shutdown_connections 2024-02-02T15:31:53.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.139+0000 7f083c7f4b80 1 --2- 172.21.15.1:0/1789389561 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556330aee7e0 0x556330adf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:53.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.139+0000 7f083c7f4b80 1 --2- 172.21.15.1:0/1789389561 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556330ae3010 0x556330af5330 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:53.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.139+0000 7f083c7f4b80 1 --2- 172.21.15.1:0/1789389561 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556330ae3930 0x556330adeeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:53.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.139+0000 7f083c7f4b80 1 -- 172.21.15.1:0/1789389561 >> 172.21.15.1:0/1789389561 conn(0x556330a42db0 msgr2=0x556330ae67e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:53.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.139+0000 7f083c7f4b80 1 -- 172.21.15.1:0/1789389561 shutdown_connections 2024-02-02T15:31:53.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.140+0000 7f083c7f4b80 1 -- 172.21.15.1:0/1789389561 wait complete. 2024-02-02T15:31:53.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.140+0000 7f083c7f4b80 1 -- start start 2024-02-02T15:31:53.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.140+0000 7f083c7f4b80 1 Processor -- start 2024-02-02T15:31:53.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.141+0000 7f083c7f4b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556330ae3010 0x556330af5170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:53.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.141+0000 7f083c7f4b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556330ae3930 0x556330af3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:53.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.141+0000 7f083c7f4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556330aee7e0 0x556330af3970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:53.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.141+0000 7f083c7f4b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x556330af4050 con 0x556330ae3930 2024-02-02T15:31:53.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.141+0000 7f083c7f4b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x556330af41c0 con 0x556330ae3010 2024-02-02T15:31:53.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.141+0000 7f083c7f4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556330b4fd60 con 0x556330aee7e0 2024-02-02T15:31:53.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.141+0000 7f083c7f4b80 10 client.0 _ll_register_callbacks cb 0x5563309d2290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:31:53.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.141+0000 7f0821e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556330ae3010 0x556330af5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:53.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.141+0000 7f0821e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556330ae3010 0x556330af5170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59488/0 (socket says 172.21.15.1:59488) 2024-02-02T15:31:53.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.141+0000 7f081be00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556330ae3930 0x556330af3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:53.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.141+0000 7f0821e00700 1 -- 172.21.15.1:0/3183852414 learned_addr learned my addr 172.21.15.1:0/3183852414 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:31:53.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.141+0000 7f0822800700 1 --2- 172.21.15.1:0/3183852414 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556330aee7e0 0x556330af3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:53.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.142+0000 7f0821e00700 1 -- 172.21.15.1:0/3183852414 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556330aee7e0 msgr2=0x556330af3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:53.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.142+0000 7f0821e00700 1 --2- 172.21.15.1:0/3183852414 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556330aee7e0 0x556330af3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:53.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.142+0000 7f0821e00700 1 -- 172.21.15.1:0/3183852414 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556330ae3930 msgr2=0x556330af3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:53.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.142+0000 7f0821e00700 1 --2- 172.21.15.1:0/3183852414 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556330ae3930 0x556330af3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:53.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.142+0000 7f0821e00700 1 -- 172.21.15.1:0/3183852414 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556330862b60 con 0x556330ae3010 2024-02-02T15:31:53.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.142+0000 7f0821e00700 1 --2- 172.21.15.1:0/3183852414 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556330ae3010 0x556330af5170 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f0814000c00 tx=0x7f0814004e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99fe477e2218cf92 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:53.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.142+0000 7f080b400700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:31:53.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.142+0000 7f080b400700 1 -- 172.21.15.1:0/3183852414 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0814030420 con 0x556330ae3010 2024-02-02T15:31:53.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.142+0000 7f083c7f4b80 1 -- 172.21.15.1:0/3183852414 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f080c019a30 con 0x556330ae3010 2024-02-02T15:31:53.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.142+0000 7f080b400700 1 -- 172.21.15.1:0/3183852414 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f0814047070 con 0x556330ae3010 2024-02-02T15:31:53.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.143+0000 7f080b400700 1 -- 172.21.15.1:0/3183852414 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0814030580 con 0x556330ae3010 2024-02-02T15:31:53.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.143+0000 7f083c7f4b80 10 client.26951 fetch_fsmap learned FSMap version 35 2024-02-02T15:31:53.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.143+0000 7f083c7f4b80 1 -- 172.21.15.1:0/3183852414 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5563308bacd0 con 0x556330ae3010 2024-02-02T15:31:53.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.143+0000 7f080b400700 1 -- 172.21.15.1:0/3183852414 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f080c019a30 con 0x556330ae3010 2024-02-02T15:31:53.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.143+0000 7f080b400700 1 -- 172.21.15.1:0/3183852414 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 35) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0814038070 con 0x556330ae3010 2024-02-02T15:31:53.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.143+0000 7f083c7f4b80 10 client.26951 fetch_fsmap finished waiting for FSMap version 35 2024-02-02T15:31:53.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.143+0000 7f083c7f4b80 10 client.26951 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-02T15:31:53.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.143+0000 7f083c7f4b80 20 client.26951 populate_metadata read hostname 'smithi001' 2024-02-02T15:31:53.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.143+0000 7f083c7f4b80 1 --2- 172.21.15.1:0/3183852414 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x556330b50e70 0x556330b53260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:53.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.143+0000 7f083c7f4b80 4 client.26951 mds_command: new command op to 24529 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:31:53.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.143+0000 7f083c7f4b80 1 -- 172.21.15.1:0/3183852414 --> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5563309f3f50 con 0x556330b50e70 2024-02-02T15:31:53.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.143+0000 7f081be00700 1 --2- 172.21.15.1:0/3183852414 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x556330b50e70 0x556330b53260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:53.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.144+0000 7f081be00700 1 --2- 172.21.15.1:0/3183852414 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x556330b50e70 0x556330b53260 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8d3053ce0adcffea server_cookie=1a1e4d090e2e3c0b in_seq=0 out_seq=0 2024-02-02T15:31:53.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.144+0000 7f080b400700 10 client.26951 ms_handle_connect on v2:172.21.15.142:6838/321675421 2024-02-02T15:31:53.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.144+0000 7f080b400700 1 -- 172.21.15.1:0/3183852414 <== mds.1 v2:172.21.15.142:6838/321675421 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5563309f3f50 con 0x556330b50e70 2024-02-02T15:31:53.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.144+0000 7f080b400700 10 client.26951 handle_command_reply: tid=0 2024-02-02T15:31:53.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.144+0000 7f083c7f4b80 1 client.26951 shutdown 2024-02-02T15:31:53.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.144+0000 7f083c7f4b80 1 -- 172.21.15.1:0/3183852414 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556330ae3010 msgr2=0x556330af5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:53.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.144+0000 7f083c7f4b80 1 --2- 172.21.15.1:0/3183852414 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556330ae3010 0x556330af5170 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f0814000c00 tx=0x7f0814004e40 comp rx=0 tx=0).stop 2024-02-02T15:31:53.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.144+0000 7f083c7f4b80 1 -- 172.21.15.1:0/3183852414 shutdown_connections 2024-02-02T15:31:53.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.144+0000 7f083c7f4b80 1 --2- 172.21.15.1:0/3183852414 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x556330b50e70 0x556330b53260 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:53.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.144+0000 7f083c7f4b80 1 --2- 172.21.15.1:0/3183852414 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556330aee7e0 0x556330af3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:53.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.144+0000 7f083c7f4b80 1 --2- 172.21.15.1:0/3183852414 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556330ae3010 0x556330af5170 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:53.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.144+0000 7f083c7f4b80 1 --2- 172.21.15.1:0/3183852414 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556330ae3930 0x556330af3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:53.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.144+0000 7f083c7f4b80 1 -- 172.21.15.1:0/3183852414 >> 172.21.15.1:0/3183852414 conn(0x556330a42db0 msgr2=0x556330adee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:53.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.144+0000 7f080b400700 0 client.26951 ms_handle_reset on v2:172.21.15.142:6838/321675421 2024-02-02T15:31:53.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.144+0000 7f083c7f4b80 1 -- 172.21.15.1:0/3183852414 shutdown_connections 2024-02-02T15:31:53.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.145+0000 7f083c7f4b80 1 -- 172.21.15.1:0/3183852414 wait complete. 2024-02-02T15:31:53.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.145+0000 7f083c7f4b80 20 client.26951 trim_cache size 0 max 16384 2024-02-02T15:31:53.151 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:31:53.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.145+0000 7f0828c00700 1 -- 172.21.15.1:0/964120010 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f08040a0690 msgr2=0x7f08040a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:53.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.145+0000 7f0828c00700 1 --2- 172.21.15.1:0/964120010 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f08040a0690 0x7f08040a2b50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f081400b2e0 tx=0x7f0814005960 comp rx=0 tx=0).stop 2024-02-02T15:31:53.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.145+0000 7f0828c00700 1 -- 172.21.15.1:0/964120010 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f082419b830 msgr2=0x7f082419bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:53.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.145+0000 7f0828c00700 1 --2- 172.21.15.1:0/964120010 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f082419b830 0x7f082419bcb0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f081c00efc0 tx=0x7f081c00c3a0 comp rx=0 tx=0).stop 2024-02-02T15:31:53.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.145+0000 7f0828c00700 1 -- 172.21.15.1:0/964120010 shutdown_connections 2024-02-02T15:31:53.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.145+0000 7f0828c00700 1 --2- 172.21.15.1:0/964120010 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f08040a0690 0x7f08040a2b50 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:53.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.145+0000 7f0828c00700 1 --2- 172.21.15.1:0/964120010 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f082419b830 0x7f082419bcb0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:53.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.145+0000 7f0828c00700 1 --2- 172.21.15.1:0/964120010 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0824100ff0 0x7f0824196e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:53.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.145+0000 7f0828c00700 1 --2- 172.21.15.1:0/964120010 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0824106110 0x7f08241973d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:53.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.145+0000 7f0828c00700 1 -- 172.21.15.1:0/964120010 >> 172.21.15.1:0/964120010 conn(0x7f08240fc810 msgr2=0x7f08240fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:53.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.146+0000 7f0828c00700 1 -- 172.21.15.1:0/964120010 shutdown_connections 2024-02-02T15:31:53.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.146+0000 7f0828c00700 1 -- 172.21.15.1:0/964120010 wait complete. 2024-02-02T15:31:53.164 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-02T15:31:53.233 INFO:tasks.workunit.client.0.smithi001.stdout: 1 290500 9.30 MB/sec execute 87 sec latency 20.347 ms 2024-02-02T15:31:53.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.374+0000 7f9cd2000700 1 -- 172.21.15.1:0/2401557288 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ccc111d90 msgr2=0x7f9ccc114180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:53.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.374+0000 7f9cd2000700 1 --2- 172.21.15.1:0/2401557288 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ccc111d90 0x7f9ccc114180 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f9cc4009f90 tx=0x7f9cc400b330 comp rx=0 tx=0).stop 2024-02-02T15:31:53.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.374+0000 7f9cd2000700 1 -- 172.21.15.1:0/2401557288 shutdown_connections 2024-02-02T15:31:53.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.374+0000 7f9cd2000700 1 --2- 172.21.15.1:0/2401557288 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ccc111d90 0x7f9ccc114180 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:53.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.374+0000 7f9cd2000700 1 --2- 172.21.15.1:0/2401557288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ccc0730e0 0x7f9ccc0734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:53.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.374+0000 7f9cd2000700 1 --2- 172.21.15.1:0/2401557288 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ccc073a00 0x7f9ccc111850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:53.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.374+0000 7f9cd2000700 1 -- 172.21.15.1:0/2401557288 >> 172.21.15.1:0/2401557288 conn(0x7f9ccc0fc810 msgr2=0x7f9ccc0fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:53.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.374+0000 7f9cd2000700 1 -- 172.21.15.1:0/2401557288 shutdown_connections 2024-02-02T15:31:53.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.374+0000 7f9cd2000700 1 -- 172.21.15.1:0/2401557288 wait complete. 2024-02-02T15:31:53.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.375+0000 7f9cd2000700 1 Processor -- start 2024-02-02T15:31:53.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.375+0000 7f9cd2000700 1 -- start start 2024-02-02T15:31:53.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.375+0000 7f9cd2000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ccc0730e0 0x7f9ccc19d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:53.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.376+0000 7f9cd2000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ccc073a00 0x7f9ccc19dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:53.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.376+0000 7f9cd2000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ccc111d90 0x7f9ccc1a1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:53.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.376+0000 7f9cd2000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ccc1a2580 con 0x7f9ccc073a00 2024-02-02T15:31:53.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.376+0000 7f9cd2000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ccc1a26f0 con 0x7f9ccc0730e0 2024-02-02T15:31:53.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.376+0000 7f9cd2000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ccc1a28b0 con 0x7f9ccc111d90 2024-02-02T15:31:53.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.376+0000 7f9ccbe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ccc111d90 0x7f9ccc1a1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:53.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.376+0000 7f9ccaa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ccc073a00 0x7f9ccc19dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:53.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.376+0000 7f9ccbe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ccc111d90 0x7f9ccc1a1f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:47792/0 (socket says 172.21.15.1:47792) 2024-02-02T15:31:53.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.376+0000 7f9ccb400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ccc0730e0 0x7f9ccc19d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:53.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.376+0000 7f9ccbe00700 1 -- 172.21.15.1:0/1046144250 learned_addr learned my addr 172.21.15.1:0/1046144250 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:31:53.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.376+0000 7f9ccbe00700 1 -- 172.21.15.1:0/1046144250 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ccc0730e0 msgr2=0x7f9ccc19d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:53.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.376+0000 7f9ccbe00700 1 --2- 172.21.15.1:0/1046144250 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ccc0730e0 0x7f9ccc19d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:53.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.376+0000 7f9ccbe00700 1 -- 172.21.15.1:0/1046144250 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ccc073a00 msgr2=0x7f9ccc19dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:53.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.376+0000 7f9ccbe00700 1 --2- 172.21.15.1:0/1046144250 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ccc073a00 0x7f9ccc19dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:53.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.376+0000 7f9ccbe00700 1 -- 172.21.15.1:0/1046144250 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cc4009c40 con 0x7f9ccc111d90 2024-02-02T15:31:53.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.376+0000 7f9ccb400700 1 --2- 172.21.15.1:0/1046144250 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ccc0730e0 0x7f9ccc19d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:31:53.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.377+0000 7f9ccbe00700 1 --2- 172.21.15.1:0/1046144250 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ccc111d90 0x7f9ccc1a1f40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9cc40094b0 tx=0x7f9cc40094e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:53.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.377+0000 7f9cc3e00700 1 -- 172.21.15.1:0/1046144250 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc400e040 con 0x7f9ccc111d90 2024-02-02T15:31:53.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.377+0000 7f9cc3e00700 1 -- 172.21.15.1:0/1046144250 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9cc4004e70 con 0x7f9ccc111d90 2024-02-02T15:31:53.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.377+0000 7f9cd2000700 1 -- 172.21.15.1:0/1046144250 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ccc1a2b30 con 0x7f9ccc111d90 2024-02-02T15:31:53.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.377+0000 7f9cc3e00700 1 -- 172.21.15.1:0/1046144250 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc4005040 con 0x7f9ccc111d90 2024-02-02T15:31:53.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.377+0000 7f9cd2000700 1 -- 172.21.15.1:0/1046144250 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ccc1aa3d0 con 0x7f9ccc111d90 2024-02-02T15:31:53.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.381+0000 7f9cc3e00700 1 -- 172.21.15.1:0/1046144250 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 27) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f9cc401d470 con 0x7f9ccc111d90 2024-02-02T15:31:53.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.382+0000 7f9ce5b88b80 1 Processor -- start 2024-02-02T15:31:53.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.382+0000 7f9ce5b88b80 1 -- start start 2024-02-02T15:31:53.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.383+0000 7f9ce5b88b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ce367f3010 0x55ce367f33f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:53.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.383+0000 7f9ce5b88b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ce367f3930 0x55ce367fe2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:53.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.383+0000 7f9ce5b88b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce367fe7e0 0x55ce36800bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:53.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.383+0000 7f9ce5b88b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce367674c0 con 0x55ce367f3010 2024-02-02T15:31:53.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.383+0000 7f9ce5b88b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce3674baa0 con 0x55ce367f3930 2024-02-02T15:31:53.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.383+0000 7f9ce5b88b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce36749460 con 0x55ce367fe7e0 2024-02-02T15:31:53.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.383+0000 7f9ccb400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ce367f3930 0x55ce367fe2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:53.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.383+0000 7f9ccb400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ce367f3930 0x55ce367fe2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59514/0 (socket says 172.21.15.1:59514) 2024-02-02T15:31:53.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.383+0000 7f9ccaa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ce367f3010 0x55ce367f33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:53.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.383+0000 7f9ccb400700 1 -- 172.21.15.1:0/821119521 learned_addr learned my addr 172.21.15.1:0/821119521 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:31:53.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.383+0000 7f9cc3e00700 1 --2- 172.21.15.1:0/1046144250 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f9cac0a0690 0x7f9cac0a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:53.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.383+0000 7f9ccaa00700 1 --2- 172.21.15.1:0/821119521 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce367fe7e0 0x55ce36800bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:53.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.383+0000 7f9ccb400700 1 -- 172.21.15.1:0/821119521 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce367fe7e0 msgr2=0x55ce36800bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:53.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.383+0000 7f9ccb400700 1 --2- 172.21.15.1:0/821119521 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce367fe7e0 0x55ce36800bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:53.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.383+0000 7f9ccb400700 1 -- 172.21.15.1:0/821119521 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ce367f3010 msgr2=0x55ce367f33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:53.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.383+0000 7f9ccb400700 1 --2- 172.21.15.1:0/821119521 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ce367f3010 0x55ce367f33f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:53.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.383+0000 7f9ccb400700 1 -- 172.21.15.1:0/821119521 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ce365c63b0 con 0x55ce367f3930 2024-02-02T15:31:53.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.383+0000 7f9ccaa00700 1 --2- 172.21.15.1:0/821119521 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ce367f3010 0x55ce367f33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:31:53.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.384+0000 7f9cc3e00700 1 -- 172.21.15.1:0/1046144250 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f9cc40cd4f0 con 0x7f9ccc111d90 2024-02-02T15:31:53.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.384+0000 7f9ccbe00700 1 --2- 172.21.15.1:0/1046144250 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f9cac0a0690 0x7f9cac0a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:53.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.384+0000 7f9ccb400700 1 --2- 172.21.15.1:0/821119521 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ce367f3930 0x55ce367fe2a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc0115d0 tx=0x7f9cbc0118e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31860dcb69cfd56c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:53.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.384+0000 7f9cbbe00700 1 -- 172.21.15.1:0/821119521 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cbc004230 con 0x55ce367f3930 2024-02-02T15:31:53.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.384+0000 7f9cbbe00700 1 -- 172.21.15.1:0/821119521 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9cbc004390 con 0x55ce367f3930 2024-02-02T15:31:53.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.384+0000 7f9cbbe00700 1 -- 172.21.15.1:0/821119521 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cbc00f8b0 con 0x55ce367f3930 2024-02-02T15:31:53.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.385+0000 7f9ce5b88b80 1 -- 172.21.15.1:0/821119521 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ce367f3930 msgr2=0x55ce367fe2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:53.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.385+0000 7f9ccbe00700 1 --2- 172.21.15.1:0/1046144250 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f9cac0a0690 0x7f9cac0a2b50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9cc4096c50 tx=0x7f9cc402c680 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:53.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.385+0000 7f9ce5b88b80 1 --2- 172.21.15.1:0/821119521 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ce367f3930 0x55ce367fe2a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc0115d0 tx=0x7f9cbc0118e0 comp rx=0 tx=0).stop 2024-02-02T15:31:53.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.385+0000 7f9ce5b88b80 1 -- 172.21.15.1:0/821119521 shutdown_connections 2024-02-02T15:31:53.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.385+0000 7f9ce5b88b80 1 --2- 172.21.15.1:0/821119521 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce367fe7e0 0x55ce36800bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:53.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.385+0000 7f9ce5b88b80 1 --2- 172.21.15.1:0/821119521 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ce367f3010 0x55ce367f33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:53.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.385+0000 7f9ce5b88b80 1 --2- 172.21.15.1:0/821119521 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ce367f3930 0x55ce367fe2a0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:53.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.385+0000 7f9ce5b88b80 1 -- 172.21.15.1:0/821119521 >> 172.21.15.1:0/821119521 conn(0x55ce36752db0 msgr2=0x55ce367ef3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:53.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.385+0000 7f9ce5b88b80 1 -- 172.21.15.1:0/821119521 shutdown_connections 2024-02-02T15:31:53.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.385+0000 7f9ce5b88b80 1 -- 172.21.15.1:0/821119521 wait complete. 2024-02-02T15:31:53.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.386+0000 7f9ce5b88b80 1 -- start start 2024-02-02T15:31:53.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.386+0000 7f9ce5b88b80 1 Processor -- start 2024-02-02T15:31:53.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.387+0000 7f9ce5b88b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ce367f3010 0x55ce367fe0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:53.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.387+0000 7f9ce5b88b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ce367f3930 0x55ce367fb330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:53.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.387+0000 7f9ce5b88b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce367fe7e0 0x55ce367fb890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:53.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.387+0000 7f9ce5b88b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce36744d50 con 0x55ce367f3010 2024-02-02T15:31:53.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.387+0000 7f9ce5b88b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce366e0d80 con 0x55ce367f3930 2024-02-02T15:31:53.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.387+0000 7f9ce5b88b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce366e0a20 con 0x55ce367fe7e0 2024-02-02T15:31:53.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.387+0000 7f9ce5b88b80 10 client.0 _ll_register_callbacks cb 0x55ce366e2290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:31:53.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.387+0000 7f9ccb400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ce367f3930 0x55ce367fb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:53.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.387+0000 7f9ccaa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ce367f3010 0x55ce367fe0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:53.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.387+0000 7f9ccb400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ce367f3930 0x55ce367fb330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59522/0 (socket says 172.21.15.1:59522) 2024-02-02T15:31:53.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.387+0000 7f9ccb400700 1 -- 172.21.15.1:0/3633981332 learned_addr learned my addr 172.21.15.1:0/3633981332 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:31:53.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.387+0000 7f9ccaa00700 1 --2- 172.21.15.1:0/3633981332 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce367fe7e0 0x55ce367fb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:53.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.387+0000 7f9ccb400700 1 -- 172.21.15.1:0/3633981332 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce367fe7e0 msgr2=0x55ce367fb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:53.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.387+0000 7f9ccb400700 1 --2- 172.21.15.1:0/3633981332 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce367fe7e0 0x55ce367fb890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:53.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.387+0000 7f9ccb400700 1 -- 172.21.15.1:0/3633981332 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ce367f3010 msgr2=0x55ce367fe0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:53.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.387+0000 7f9ccb400700 1 --2- 172.21.15.1:0/3633981332 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ce367f3010 0x55ce367fe0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:53.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.387+0000 7f9ccb400700 1 -- 172.21.15.1:0/3633981332 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cbc009d90 con 0x55ce367f3930 2024-02-02T15:31:53.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.388+0000 7f9ccb400700 1 --2- 172.21.15.1:0/3633981332 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ce367f3930 0x55ce367fb330 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc006010 tx=0x7f9cbc003700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=141de6b69db46a8a server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:53.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.388+0000 7f9cb8c00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:31:53.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.388+0000 7f9cb8c00700 1 -- 172.21.15.1:0/3633981332 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cbc005040 con 0x55ce367f3930 2024-02-02T15:31:53.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.388+0000 7f9ce5b88b80 1 -- 172.21.15.1:0/3633981332 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ce367e70f0 con 0x55ce367f3930 2024-02-02T15:31:53.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.388+0000 7f9cb8c00700 1 -- 172.21.15.1:0/3633981332 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f9cbc0051a0 con 0x55ce367f3930 2024-02-02T15:31:53.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.388+0000 7f9cb8c00700 1 -- 172.21.15.1:0/3633981332 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9cbc005360 con 0x55ce367f3930 2024-02-02T15:31:53.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.388+0000 7f9ce5b88b80 10 client.26963 fetch_fsmap learned FSMap version 35 2024-02-02T15:31:53.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.388+0000 7f9ce5b88b80 1 -- 172.21.15.1:0/3633981332 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55ce36572b60 con 0x55ce367f3930 2024-02-02T15:31:53.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.389+0000 7f9cb8c00700 1 -- 172.21.15.1:0/3633981332 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ce367e70f0 con 0x55ce367f3930 2024-02-02T15:31:53.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.389+0000 7f9cb8c00700 1 -- 172.21.15.1:0/3633981332 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 35) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9cbc021040 con 0x55ce367f3930 2024-02-02T15:31:53.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.389+0000 7f9ce5b88b80 10 client.26963 fetch_fsmap finished waiting for FSMap version 35 2024-02-02T15:31:53.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.389+0000 7f9ce5b88b80 10 client.26963 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:31:53.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.389+0000 7f9ce5b88b80 20 client.26963 populate_metadata read hostname 'smithi001' 2024-02-02T15:31:53.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.389+0000 7f9ce5b88b80 1 --2- 172.21.15.1:0/3633981332 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55ce3680e160 0x55ce36810580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:53.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.389+0000 7f9ce5b88b80 4 client.26963 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:31:53.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.389+0000 7f9ce5b88b80 1 -- 172.21.15.1:0/3633981332 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ce3673ddd0 con 0x55ce3680e160 2024-02-02T15:31:53.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.389+0000 7f9ccaa00700 1 --2- 172.21.15.1:0/3633981332 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55ce3680e160 0x55ce36810580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:53.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.390+0000 7f9ccaa00700 1 --2- 172.21.15.1:0/3633981332 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55ce3680e160 0x55ce36810580 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ae0f598e2b7fbf server_cookie=10b64154181d3113 in_seq=0 out_seq=0 2024-02-02T15:31:53.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.390+0000 7f9cb8c00700 10 client.26963 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:31:53.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.390+0000 7f9cb8c00700 1 -- 172.21.15.1:0/3633981332 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55ce3673ddd0 con 0x55ce3680e160 2024-02-02T15:31:53.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.390+0000 7f9cb8c00700 10 client.26963 handle_command_reply: tid=0 2024-02-02T15:31:53.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.390+0000 7f9ce5b88b80 1 client.26963 shutdown 2024-02-02T15:31:53.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.391+0000 7f9ce5b88b80 1 -- 172.21.15.1:0/3633981332 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ce367f3930 msgr2=0x55ce367fb330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:53.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.391+0000 7f9ce5b88b80 1 --2- 172.21.15.1:0/3633981332 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ce367f3930 0x55ce367fb330 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc006010 tx=0x7f9cbc003700 comp rx=0 tx=0).stop 2024-02-02T15:31:53.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.391+0000 7f9ce5b88b80 1 -- 172.21.15.1:0/3633981332 shutdown_connections 2024-02-02T15:31:53.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.391+0000 7f9ce5b88b80 1 --2- 172.21.15.1:0/3633981332 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55ce3680e160 0x55ce36810580 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:53.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.391+0000 7f9ce5b88b80 1 --2- 172.21.15.1:0/3633981332 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce367fe7e0 0x55ce367fb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:53.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.391+0000 7f9cb8c00700 0 client.26963 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:31:53.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.391+0000 7f9ce5b88b80 1 --2- 172.21.15.1:0/3633981332 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ce367f3010 0x55ce367fe0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:53.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.391+0000 7f9ce5b88b80 1 --2- 172.21.15.1:0/3633981332 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ce367f3930 0x55ce367fb330 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:53.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.391+0000 7f9ce5b88b80 1 -- 172.21.15.1:0/3633981332 >> 172.21.15.1:0/3633981332 conn(0x55ce36752db0 msgr2=0x55ce367eefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:53.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.391+0000 7f9ce5b88b80 1 -- 172.21.15.1:0/3633981332 shutdown_connections 2024-02-02T15:31:53.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.391+0000 7f9ce5b88b80 1 -- 172.21.15.1:0/3633981332 wait complete. 2024-02-02T15:31:53.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.391+0000 7f9ce5b88b80 20 client.26963 trim_cache size 0 max 16384 2024-02-02T15:31:53.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.406+0000 7f9ce5b88b80 1 Processor -- start 2024-02-02T15:31:53.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.406+0000 7f9ce5b88b80 1 -- start start 2024-02-02T15:31:53.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.406+0000 7f9ce5b88b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ce367f3010 0x55ce367fdc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:53.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.406+0000 7f9ce5b88b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce367f3930 0x55ce367faaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:53.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.406+0000 7f9ce5b88b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ce367fe7e0 0x55ce367fb030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:53.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.406+0000 7f9ce5b88b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce365c86a0 con 0x55ce367f3010 2024-02-02T15:31:53.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.406+0000 7f9ce5b88b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce367f00d0 con 0x55ce367fe7e0 2024-02-02T15:31:53.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.406+0000 7f9ce5b88b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce367e6cc0 con 0x55ce367f3930 2024-02-02T15:31:53.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.406+0000 7f9ccaa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ce367f3010 0x55ce367fdc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:53.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.406+0000 7f9ccb400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce367f3930 0x55ce367faaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:53.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.406+0000 7f9ccaa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ce367f3010 0x55ce367fdc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45652/0 (socket says 172.21.15.1:45652) 2024-02-02T15:31:53.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.406+0000 7f9ccaa00700 1 -- 172.21.15.1:0/1895189063 learned_addr learned my addr 172.21.15.1:0/1895189063 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:31:53.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.406+0000 7f9ccaa00700 1 --2- 172.21.15.1:0/1895189063 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ce367fe7e0 0x55ce367fb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:53.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.407+0000 7f9ccb400700 1 -- 172.21.15.1:0/1895189063 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ce367fe7e0 msgr2=0x55ce367fb030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:53.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.407+0000 7f9ccb400700 1 --2- 172.21.15.1:0/1895189063 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ce367fe7e0 0x55ce367fb030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:53.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.407+0000 7f9ccb400700 1 -- 172.21.15.1:0/1895189063 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ce367f3010 msgr2=0x55ce367fdc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:53.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.407+0000 7f9ccb400700 1 --2- 172.21.15.1:0/1895189063 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ce367f3010 0x55ce367fdc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:53.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.407+0000 7f9ccb400700 1 -- 172.21.15.1:0/1895189063 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ce3670aea0 con 0x55ce367f3930 2024-02-02T15:31:53.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.407+0000 7f9ccb400700 1 --2- 172.21.15.1:0/1895189063 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce367f3930 0x55ce367faaf0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc023500 tx=0x7f9cbc0094b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de85b3323d5c18d7 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:53.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.407+0000 7f9cc1200700 1 -- 172.21.15.1:0/1895189063 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cbc003770 con 0x55ce367f3930 2024-02-02T15:31:53.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.407+0000 7f9cc1200700 1 -- 172.21.15.1:0/1895189063 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9cbc023e70 con 0x55ce367f3930 2024-02-02T15:31:53.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.407+0000 7f9cc1200700 1 -- 172.21.15.1:0/1895189063 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cbc004aa0 con 0x55ce367f3930 2024-02-02T15:31:53.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.408+0000 7f9ce5b88b80 1 -- 172.21.15.1:0/1895189063 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce367f3930 msgr2=0x55ce367faaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:53.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.408+0000 7f9ce5b88b80 1 --2- 172.21.15.1:0/1895189063 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce367f3930 0x55ce367faaf0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc023500 tx=0x7f9cbc0094b0 comp rx=0 tx=0).stop 2024-02-02T15:31:53.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.408+0000 7f9ce5b88b80 1 -- 172.21.15.1:0/1895189063 shutdown_connections 2024-02-02T15:31:53.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.408+0000 7f9ce5b88b80 1 --2- 172.21.15.1:0/1895189063 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ce367fe7e0 0x55ce367fb030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:53.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.408+0000 7f9ce5b88b80 1 --2- 172.21.15.1:0/1895189063 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ce367f3010 0x55ce367fdc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:53.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.408+0000 7f9ce5b88b80 1 --2- 172.21.15.1:0/1895189063 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce367f3930 0x55ce367faaf0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:53.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.408+0000 7f9ce5b88b80 1 -- 172.21.15.1:0/1895189063 >> 172.21.15.1:0/1895189063 conn(0x55ce36752db0 msgr2=0x55ce367f6660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:53.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.408+0000 7f9ce5b88b80 1 -- 172.21.15.1:0/1895189063 shutdown_connections 2024-02-02T15:31:53.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.408+0000 7f9ce5b88b80 1 -- 172.21.15.1:0/1895189063 wait complete. 2024-02-02T15:31:53.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.408+0000 7f9ce5b88b80 1 -- start start 2024-02-02T15:31:53.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.409+0000 7f9ce5b88b80 1 Processor -- start 2024-02-02T15:31:53.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.409+0000 7f9ce5b88b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ce367f3010 0x55ce36803350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:53.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.409+0000 7f9ce5b88b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce367f3930 0x55ce36803890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:53.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.409+0000 7f9ce5b88b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ce367fe7e0 0x55ce36803dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:53.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.409+0000 7f9ce5b88b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce3685ed70 con 0x55ce367f3010 2024-02-02T15:31:53.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.409+0000 7f9ce5b88b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce3685eee0 con 0x55ce367fe7e0 2024-02-02T15:31:53.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.409+0000 7f9ce5b88b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce3685f0a0 con 0x55ce367f3930 2024-02-02T15:31:53.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.409+0000 7f9ce5b88b80 10 client.0 _ll_register_callbacks cb 0x55ce366e2290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:31:53.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.409+0000 7f9ccb400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce367f3930 0x55ce36803890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:53.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.409+0000 7f9ccaa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ce367f3010 0x55ce36803350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:53.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.410+0000 7f9ccb400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce367f3930 0x55ce36803890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:47806/0 (socket says 172.21.15.1:47806) 2024-02-02T15:31:53.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.410+0000 7f9ccb400700 1 -- 172.21.15.1:0/2795598861 learned_addr learned my addr 172.21.15.1:0/2795598861 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:31:53.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.410+0000 7f9ccaa00700 1 --2- 172.21.15.1:0/2795598861 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ce367fe7e0 0x55ce36803dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:53.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.410+0000 7f9ccb400700 1 -- 172.21.15.1:0/2795598861 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ce367fe7e0 msgr2=0x55ce36803dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:53.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.410+0000 7f9ccb400700 1 --2- 172.21.15.1:0/2795598861 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ce367fe7e0 0x55ce36803dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:53.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.410+0000 7f9ccb400700 1 -- 172.21.15.1:0/2795598861 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ce367f3010 msgr2=0x55ce36803350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:53.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.410+0000 7f9ccb400700 1 --2- 172.21.15.1:0/2795598861 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ce367f3010 0x55ce36803350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:53.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.410+0000 7f9ccb400700 1 -- 172.21.15.1:0/2795598861 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ce36572b60 con 0x55ce367f3930 2024-02-02T15:31:53.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.410+0000 7f9ccaa00700 1 --2- 172.21.15.1:0/2795598861 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ce367f3010 0x55ce36803350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:31:53.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.410+0000 7f9ccb400700 1 --2- 172.21.15.1:0/2795598861 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce367f3930 0x55ce36803890 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc000c00 tx=0x7f9cbc003df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69177028df2d1ae5 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:53.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.410+0000 7f9cb8c00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:31:53.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.411+0000 7f9cb8c00700 1 -- 172.21.15.1:0/2795598861 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cbc003700 con 0x55ce367f3930 2024-02-02T15:31:53.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.411+0000 7f9ce5b88b80 1 -- 172.21.15.1:0/2795598861 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9cbc004390 con 0x55ce367f3930 2024-02-02T15:31:53.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.411+0000 7f9cb8c00700 1 -- 172.21.15.1:0/2795598861 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f9cbc000e40 con 0x55ce367f3930 2024-02-02T15:31:53.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.411+0000 7f9cb8c00700 1 -- 172.21.15.1:0/2795598861 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9cbc023770 con 0x55ce367f3930 2024-02-02T15:31:53.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.411+0000 7f9ce5b88b80 10 client.26968 fetch_fsmap learned FSMap version 35 2024-02-02T15:31:53.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.411+0000 7f9ce5b88b80 1 -- 172.21.15.1:0/2795598861 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55ce365cacd0 con 0x55ce367f3930 2024-02-02T15:31:53.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.411+0000 7f9cb8c00700 1 -- 172.21.15.1:0/2795598861 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9cbc004390 con 0x55ce367f3930 2024-02-02T15:31:53.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.411+0000 7f9cb8c00700 1 -- 172.21.15.1:0/2795598861 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 35) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9cbc011a00 con 0x55ce367f3930 2024-02-02T15:31:53.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.411+0000 7f9ce5b88b80 10 client.26968 fetch_fsmap finished waiting for FSMap version 35 2024-02-02T15:31:53.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.411+0000 7f9ce5b88b80 10 client.26968 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:31:53.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.411+0000 7f9ce5b88b80 20 client.26968 populate_metadata read hostname 'smithi001' 2024-02-02T15:31:53.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.411+0000 7f9ce5b88b80 1 --2- 172.21.15.1:0/2795598861 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55ce368602f0 0x55ce368626e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:53.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.411+0000 7f9ce5b88b80 4 client.26968 mds_command: new command op to 24557 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:31:53.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.411+0000 7f9ce5b88b80 1 -- 172.21.15.1:0/2795598861 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ce36703f50 con 0x55ce368602f0 2024-02-02T15:31:53.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.412+0000 7f9ccaa00700 1 --2- 172.21.15.1:0/2795598861 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55ce368602f0 0x55ce368626e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:53.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.412+0000 7f9ccaa00700 1 --2- 172.21.15.1:0/2795598861 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55ce368602f0 0x55ce368626e0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f35d8fb03996a64a server_cookie=ce6a52bc13158290 in_seq=0 out_seq=0 2024-02-02T15:31:53.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.412+0000 7f9cb8c00700 10 client.26968 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:31:53.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.412+0000 7f9cb8c00700 1 -- 172.21.15.1:0/2795598861 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ce36703f50 con 0x55ce368602f0 2024-02-02T15:31:53.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.412+0000 7f9cb8c00700 10 client.26968 handle_command_reply: tid=0 2024-02-02T15:31:53.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.412+0000 7f9ce5b88b80 1 client.26968 shutdown 2024-02-02T15:31:53.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.412+0000 7f9ce5b88b80 1 -- 172.21.15.1:0/2795598861 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce367f3930 msgr2=0x55ce36803890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:53.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.412+0000 7f9ce5b88b80 1 --2- 172.21.15.1:0/2795598861 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce367f3930 0x55ce36803890 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc000c00 tx=0x7f9cbc003df0 comp rx=0 tx=0).stop 2024-02-02T15:31:53.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.413+0000 7f9ce5b88b80 1 -- 172.21.15.1:0/2795598861 shutdown_connections 2024-02-02T15:31:53.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.413+0000 7f9ce5b88b80 1 --2- 172.21.15.1:0/2795598861 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55ce368602f0 0x55ce368626e0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:53.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.413+0000 7f9ce5b88b80 1 --2- 172.21.15.1:0/2795598861 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ce367fe7e0 0x55ce36803dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:53.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.413+0000 7f9ce5b88b80 1 --2- 172.21.15.1:0/2795598861 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ce367f3010 0x55ce36803350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:53.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.413+0000 7f9ce5b88b80 1 --2- 172.21.15.1:0/2795598861 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce367f3930 0x55ce36803890 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:53.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.413+0000 7f9ce5b88b80 1 -- 172.21.15.1:0/2795598861 >> 172.21.15.1:0/2795598861 conn(0x55ce36752db0 msgr2=0x55ce367ee5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:53.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.413+0000 7f9cb8c00700 0 client.26968 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:31:53.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.413+0000 7f9ce5b88b80 1 -- 172.21.15.1:0/2795598861 shutdown_connections 2024-02-02T15:31:53.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.413+0000 7f9ce5b88b80 1 -- 172.21.15.1:0/2795598861 wait complete. 2024-02-02T15:31:53.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.413+0000 7f9ce5b88b80 20 client.26968 trim_cache size 0 max 16384 2024-02-02T15:31:53.417 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:31:53.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.413+0000 7f9cd2000700 1 -- 172.21.15.1:0/1046144250 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f9cac0a0690 msgr2=0x7f9cac0a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:53.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.413+0000 7f9cd2000700 1 --2- 172.21.15.1:0/1046144250 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f9cac0a0690 0x7f9cac0a2b50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9cc4096c50 tx=0x7f9cc402c680 comp rx=0 tx=0).stop 2024-02-02T15:31:53.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.413+0000 7f9cd2000700 1 -- 172.21.15.1:0/1046144250 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ccc111d90 msgr2=0x7f9ccc1a1f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:53.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.413+0000 7f9cd2000700 1 --2- 172.21.15.1:0/1046144250 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ccc111d90 0x7f9ccc1a1f40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9cc40094b0 tx=0x7f9cc40094e0 comp rx=0 tx=0).stop 2024-02-02T15:31:53.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.414+0000 7f9cd2000700 1 -- 172.21.15.1:0/1046144250 shutdown_connections 2024-02-02T15:31:53.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.414+0000 7f9cd2000700 1 --2- 172.21.15.1:0/1046144250 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f9cac0a0690 0x7f9cac0a2b50 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:53.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.414+0000 7f9cd2000700 1 --2- 172.21.15.1:0/1046144250 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9ccc111d90 0x7f9ccc1a1f40 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:53.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.414+0000 7f9cd2000700 1 --2- 172.21.15.1:0/1046144250 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9ccc0730e0 0x7f9ccc19d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:53.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.414+0000 7f9cd2000700 1 --2- 172.21.15.1:0/1046144250 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9ccc073a00 0x7f9ccc19dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:53.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.414+0000 7f9cd2000700 1 -- 172.21.15.1:0/1046144250 >> 172.21.15.1:0/1046144250 conn(0x7f9ccc0fc810 msgr2=0x7f9ccc0fd480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:53.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.414+0000 7f9cd2000700 1 -- 172.21.15.1:0/1046144250 shutdown_connections 2024-02-02T15:31:53.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:53.414+0000 7f9cd2000700 1 -- 172.21.15.1:0/1046144250 wait complete. 2024-02-02T15:31:53.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:31:53 smithi001 ceph-mon[35470]: pgmap v149: 129 pgs: 129 active+clean; 172 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.1 MiB/s wr, 336 op/s 2024-02-02T15:31:53.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:31:53 smithi071 ceph-mon[44707]: pgmap v149: 129 pgs: 129 active+clean; 172 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.1 MiB/s wr, 336 op/s 2024-02-02T15:31:53.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:31:53 smithi142 ceph-mon[45728]: pgmap v149: 129 pgs: 129 active+clean; 172 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.1 MiB/s wr, 336 op/s 2024-02-02T15:31:54.233 INFO:tasks.workunit.client.0.smithi001.stdout: 1 291608 9.27 MB/sec execute 88 sec latency 11.601 ms 2024-02-02T15:31:54.432 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-02T15:31:54.432 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-02T15:31:54.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.663+0000 7f8c1e200700 1 -- 172.21.15.1:0/568571404 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8c18106110 msgr2=0x7f8c1810dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:54.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.663+0000 7f8c1e200700 1 --2- 172.21.15.1:0/568571404 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8c18106110 0x7f8c1810dc90 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f8c1000b0a0 tx=0x7f8c1000b3b0 comp rx=0 tx=0).stop 2024-02-02T15:31:54.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.664+0000 7f8c1e200700 1 -- 172.21.15.1:0/568571404 shutdown_connections 2024-02-02T15:31:54.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.664+0000 7f8c1e200700 1 --2- 172.21.15.1:0/568571404 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8c18106110 0x7f8c1810dc90 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:54.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.664+0000 7f8c1e200700 1 --2- 172.21.15.1:0/568571404 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8c18100ff0 0x7f8c181013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:54.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.664+0000 7f8c1e200700 1 --2- 172.21.15.1:0/568571404 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c181019a0 0x7f8c181059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:54.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.664+0000 7f8c1e200700 1 -- 172.21.15.1:0/568571404 >> 172.21.15.1:0/568571404 conn(0x7f8c180fc810 msgr2=0x7f8c180fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:54.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.664+0000 7f8c1e200700 1 -- 172.21.15.1:0/568571404 shutdown_connections 2024-02-02T15:31:54.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.664+0000 7f8c1e200700 1 -- 172.21.15.1:0/568571404 wait complete. 2024-02-02T15:31:54.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.665+0000 7f8c1e200700 1 Processor -- start 2024-02-02T15:31:54.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.665+0000 7f8c1e200700 1 -- start start 2024-02-02T15:31:54.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.665+0000 7f8c1e200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8c18100ff0 0x7f8c18198fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:54.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.665+0000 7f8c1e200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c18106110 0x7f8c181994f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:54.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.665+0000 7f8c1e200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8c1819d950 0x7f8c1819ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:54.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.665+0000 7f8c1e200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1819e3d0 con 0x7f8c18100ff0 2024-02-02T15:31:54.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.665+0000 7f8c1e200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1819e540 con 0x7f8c1819d950 2024-02-02T15:31:54.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.665+0000 7f8c1e200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1819e700 con 0x7f8c18106110 2024-02-02T15:31:54.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.666+0000 7f8c17400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8c18100ff0 0x7f8c18198fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:54.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.666+0000 7f8c17e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8c1819d950 0x7f8c1819ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:54.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.666+0000 7f8c17e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8c1819d950 0x7f8c1819ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59554/0 (socket says 172.21.15.1:59554) 2024-02-02T15:31:54.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.666+0000 7f8c17e00700 1 -- 172.21.15.1:0/2860078631 learned_addr learned my addr 172.21.15.1:0/2860078631 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:31:54.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.666+0000 7f8c17400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8c18100ff0 0x7f8c18198fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45690/0 (socket says 172.21.15.1:45690) 2024-02-02T15:31:54.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.666+0000 7f8c16a00700 1 --2- 172.21.15.1:0/2860078631 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c18106110 0x7f8c181994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:54.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.666+0000 7f8c17e00700 1 -- 172.21.15.1:0/2860078631 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c18106110 msgr2=0x7f8c181994f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:54.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.666+0000 7f8c17e00700 1 --2- 172.21.15.1:0/2860078631 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c18106110 0x7f8c181994f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:54.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.666+0000 7f8c17e00700 1 -- 172.21.15.1:0/2860078631 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8c18100ff0 msgr2=0x7f8c18198fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:54.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.666+0000 7f8c17e00700 1 --2- 172.21.15.1:0/2860078631 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8c18100ff0 0x7f8c18198fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:54.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.666+0000 7f8c17e00700 1 -- 172.21.15.1:0/2860078631 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c08009520 con 0x7f8c1819d950 2024-02-02T15:31:54.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.666+0000 7f8c17400700 1 --2- 172.21.15.1:0/2860078631 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8c18100ff0 0x7f8c18198fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:31:54.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.666+0000 7f8c17e00700 1 --2- 172.21.15.1:0/2860078631 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8c1819d950 0x7f8c1819ddd0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f8c10009850 tx=0x7f8c100077f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:54.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.666+0000 7f8c0fe00700 1 -- 172.21.15.1:0/2860078631 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c1000e040 con 0x7f8c1819d950 2024-02-02T15:31:54.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.667+0000 7f8c0fe00700 1 -- 172.21.15.1:0/2860078631 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c100079e0 con 0x7f8c1819d950 2024-02-02T15:31:54.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.667+0000 7f8c1e200700 1 -- 172.21.15.1:0/2860078631 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c10009d00 con 0x7f8c1819d950 2024-02-02T15:31:54.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.667+0000 7f8c0fe00700 1 -- 172.21.15.1:0/2860078631 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c100052a0 con 0x7f8c1819d950 2024-02-02T15:31:54.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.667+0000 7f8c1e200700 1 -- 172.21.15.1:0/2860078631 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c181a5ea0 con 0x7f8c1819d950 2024-02-02T15:31:54.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.670+0000 7f8c0fe00700 1 -- 172.21.15.1:0/2860078631 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 27) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f8c1001d470 con 0x7f8c1819d950 2024-02-02T15:31:54.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.671+0000 7f8c0fe00700 1 --2- 172.21.15.1:0/2860078631 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f8bf80a0690 0x7f8bf80a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:54.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.671+0000 7f8c31e0bb80 1 Processor -- start 2024-02-02T15:31:54.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.671+0000 7f8c0fe00700 1 -- 172.21.15.1:0/2860078631 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f8c100cd450 con 0x7f8c1819d950 2024-02-02T15:31:54.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.671+0000 7f8c31e0bb80 1 -- start start 2024-02-02T15:31:54.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.671+0000 7f8c17400700 1 --2- 172.21.15.1:0/2860078631 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f8bf80a0690 0x7f8bf80a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:54.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.672+0000 7f8c31e0bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578cb21f090 0x5578cb21f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:54.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.672+0000 7f8c31e0bb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578cb21f9b0 0x5578cb22a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:54.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.672+0000 7f8c31e0bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5578cb22a860 0x5578cb22cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:54.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.672+0000 7f8c31e0bb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578cb193540 con 0x5578cb21f9b0 2024-02-02T15:31:54.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.672+0000 7f8c31e0bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578cb177b20 con 0x5578cb22a860 2024-02-02T15:31:54.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.672+0000 7f8c31e0bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578cb1754e0 con 0x5578cb21f090 2024-02-02T15:31:54.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.672+0000 7f8c17400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578cb21f090 0x5578cb21f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:54.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.672+0000 7f8c17e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5578cb22a860 0x5578cb22cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:54.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.672+0000 7f8c17e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5578cb22a860 0x5578cb22cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59562/0 (socket says 172.21.15.1:59562) 2024-02-02T15:31:54.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.672+0000 7f8c17e00700 1 -- 172.21.15.1:0/3911277390 learned_addr learned my addr 172.21.15.1:0/3911277390 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:31:54.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.672+0000 7f8c16a00700 1 --2- 172.21.15.1:0/3911277390 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578cb21f9b0 0x5578cb22a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:54.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.672+0000 7f8c17400700 1 --2- 172.21.15.1:0/2860078631 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f8bf80a0690 0x7f8bf80a2b50 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8c08009fd0 tx=0x7f8c08009260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:54.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.672+0000 7f8c17e00700 1 -- 172.21.15.1:0/3911277390 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578cb21f090 msgr2=0x5578cb21f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:54.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.672+0000 7f8c17e00700 1 --2- 172.21.15.1:0/3911277390 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578cb21f090 0x5578cb21f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:54.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.672+0000 7f8c17e00700 1 -- 172.21.15.1:0/3911277390 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578cb21f9b0 msgr2=0x5578cb22a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:54.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.672+0000 7f8c17e00700 1 --2- 172.21.15.1:0/3911277390 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578cb21f9b0 0x5578cb22a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:54.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.672+0000 7f8c17e00700 1 -- 172.21.15.1:0/3911277390 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5578caff2430 con 0x5578cb22a860 2024-02-02T15:31:54.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.673+0000 7f8c17e00700 1 --2- 172.21.15.1:0/3911277390 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5578cb22a860 0x5578cb22cc50 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f8c10096b50 tx=0x7f8c1009a420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7438cfcbaac2f2c8 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:54.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.673+0000 7f8c07e00700 1 -- 172.21.15.1:0/3911277390 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c1000a040 con 0x5578cb22a860 2024-02-02T15:31:54.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.673+0000 7f8c07e00700 1 -- 172.21.15.1:0/3911277390 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c10004e60 con 0x5578cb22a860 2024-02-02T15:31:54.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.673+0000 7f8c07e00700 1 -- 172.21.15.1:0/3911277390 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c10098b10 con 0x5578cb22a860 2024-02-02T15:31:54.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.673+0000 7f8c31e0bb80 1 -- 172.21.15.1:0/3911277390 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5578cb22a860 msgr2=0x5578cb22cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:54.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.673+0000 7f8c31e0bb80 1 --2- 172.21.15.1:0/3911277390 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5578cb22a860 0x5578cb22cc50 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f8c10096b50 tx=0x7f8c1009a420 comp rx=0 tx=0).stop 2024-02-02T15:31:54.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.673+0000 7f8c31e0bb80 1 -- 172.21.15.1:0/3911277390 shutdown_connections 2024-02-02T15:31:54.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.673+0000 7f8c31e0bb80 1 --2- 172.21.15.1:0/3911277390 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5578cb22a860 0x5578cb22cc50 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:54.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.673+0000 7f8c31e0bb80 1 --2- 172.21.15.1:0/3911277390 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578cb21f090 0x5578cb21f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:54.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.673+0000 7f8c31e0bb80 1 --2- 172.21.15.1:0/3911277390 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578cb21f9b0 0x5578cb22a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:54.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.673+0000 7f8c31e0bb80 1 -- 172.21.15.1:0/3911277390 >> 172.21.15.1:0/3911277390 conn(0x5578cb17ee30 msgr2=0x5578cb21b430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:54.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.674+0000 7f8c31e0bb80 1 -- 172.21.15.1:0/3911277390 shutdown_connections 2024-02-02T15:31:54.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.674+0000 7f8c31e0bb80 1 -- 172.21.15.1:0/3911277390 wait complete. 2024-02-02T15:31:54.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.674+0000 7f8c31e0bb80 1 -- start start 2024-02-02T15:31:54.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.675+0000 7f8c31e0bb80 1 Processor -- start 2024-02-02T15:31:54.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.675+0000 7f8c31e0bb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578cb21f090 0x5578cb22a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:54.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.675+0000 7f8c31e0bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578cb21f9b0 0x5578cb227450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:54.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.675+0000 7f8c31e0bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5578cb22a860 0x5578cb2279d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:54.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.675+0000 7f8c31e0bb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578cb170dd0 con 0x5578cb21f090 2024-02-02T15:31:54.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.675+0000 7f8c31e0bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578cb10ce00 con 0x5578cb22a860 2024-02-02T15:31:54.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.675+0000 7f8c31e0bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578cb10caa0 con 0x5578cb21f9b0 2024-02-02T15:31:54.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.675+0000 7f8c31e0bb80 10 client.0 _ll_register_callbacks cb 0x5578cb10e310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:31:54.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.676+0000 7f8c17e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5578cb22a860 0x5578cb2279d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:54.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.676+0000 7f8c16a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578cb21f9b0 0x5578cb227450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:54.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.676+0000 7f8c17400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578cb21f090 0x5578cb22a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:54.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.676+0000 7f8c17e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5578cb22a860 0x5578cb2279d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59564/0 (socket says 172.21.15.1:59564) 2024-02-02T15:31:54.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.676+0000 7f8c17e00700 1 -- 172.21.15.1:0/3098126010 learned_addr learned my addr 172.21.15.1:0/3098126010 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:31:54.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.676+0000 7f8c16a00700 1 -- 172.21.15.1:0/3098126010 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5578cb22a860 msgr2=0x5578cb2279d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:54.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.676+0000 7f8c16a00700 1 --2- 172.21.15.1:0/3098126010 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5578cb22a860 0x5578cb2279d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:54.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.676+0000 7f8c16a00700 1 -- 172.21.15.1:0/3098126010 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578cb21f090 msgr2=0x5578cb22a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:54.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.676+0000 7f8c16a00700 1 --2- 172.21.15.1:0/3098126010 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578cb21f090 0x5578cb22a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:54.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.676+0000 7f8c16a00700 1 -- 172.21.15.1:0/3098126010 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c181a5ea0 con 0x5578cb21f9b0 2024-02-02T15:31:54.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.676+0000 7f8c16a00700 1 --2- 172.21.15.1:0/3098126010 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578cb21f9b0 0x5578cb227450 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8c1806a330 tx=0x7f8c000119a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=494c7649d0a31f87 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:54.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.676+0000 7f8c04c00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:31:54.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.677+0000 7f8c04c00700 1 -- 172.21.15.1:0/3098126010 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c00016070 con 0x5578cb21f9b0 2024-02-02T15:31:54.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.677+0000 7f8c04c00700 1 -- 172.21.15.1:0/3098126010 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f8c00004260 con 0x5578cb21f9b0 2024-02-02T15:31:54.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.677+0000 7f8c31e0bb80 1 -- 172.21.15.1:0/3098126010 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5578cb213170 con 0x5578cb21f9b0 2024-02-02T15:31:54.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.677+0000 7f8c04c00700 1 -- 172.21.15.1:0/3098126010 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c00004420 con 0x5578cb21f9b0 2024-02-02T15:31:54.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.677+0000 7f8c31e0bb80 10 client.26980 fetch_fsmap learned FSMap version 35 2024-02-02T15:31:54.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.677+0000 7f8c31e0bb80 1 -- 172.21.15.1:0/3098126010 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5578caf9ebe0 con 0x5578cb21f9b0 2024-02-02T15:31:54.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.677+0000 7f8c04c00700 1 -- 172.21.15.1:0/3098126010 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5578cb213170 con 0x5578cb21f9b0 2024-02-02T15:31:54.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.677+0000 7f8c04c00700 1 -- 172.21.15.1:0/3098126010 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 35) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8c0000f6a0 con 0x5578cb21f9b0 2024-02-02T15:31:54.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.677+0000 7f8c31e0bb80 10 client.26980 fetch_fsmap finished waiting for FSMap version 35 2024-02-02T15:31:54.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.677+0000 7f8c31e0bb80 10 client.26980 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:31:54.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.677+0000 7f8c31e0bb80 20 client.26980 populate_metadata read hostname 'smithi001' 2024-02-02T15:31:54.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.678+0000 7f8c31e0bb80 1 --2- 172.21.15.1:0/3098126010 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5578cb23a090 0x5578cb23c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:54.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.678+0000 7f8c31e0bb80 4 client.26980 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:31:54.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.678+0000 7f8c31e0bb80 1 -- 172.21.15.1:0/3098126010 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5578cb169e50 con 0x5578cb23a090 2024-02-02T15:31:54.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.678+0000 7f8c17400700 1 --2- 172.21.15.1:0/3098126010 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5578cb23a090 0x5578cb23c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:54.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.678+0000 7f8c17400700 1 --2- 172.21.15.1:0/3098126010 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5578cb23a090 0x5578cb23c4b0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2fea2112a8d262 server_cookie=ba8c269ba447c9b in_seq=0 out_seq=0 2024-02-02T15:31:54.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.678+0000 7f8c04c00700 10 client.26980 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:31:54.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.679+0000 7f8c04c00700 1 -- 172.21.15.1:0/3098126010 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5578cb169e50 con 0x5578cb23a090 2024-02-02T15:31:54.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.679+0000 7f8c04c00700 10 client.26980 handle_command_reply: tid=0 2024-02-02T15:31:54.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.679+0000 7f8c31e0bb80 1 client.26980 shutdown 2024-02-02T15:31:54.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.680+0000 7f8c31e0bb80 1 -- 172.21.15.1:0/3098126010 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578cb21f9b0 msgr2=0x5578cb227450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:54.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.680+0000 7f8c31e0bb80 1 --2- 172.21.15.1:0/3098126010 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578cb21f9b0 0x5578cb227450 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8c1806a330 tx=0x7f8c000119a0 comp rx=0 tx=0).stop 2024-02-02T15:31:54.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.680+0000 7f8c31e0bb80 1 -- 172.21.15.1:0/3098126010 shutdown_connections 2024-02-02T15:31:54.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.680+0000 7f8c31e0bb80 1 --2- 172.21.15.1:0/3098126010 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5578cb23a090 0x5578cb23c4b0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:54.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.680+0000 7f8c31e0bb80 1 --2- 172.21.15.1:0/3098126010 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5578cb22a860 0x5578cb2279d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:54.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.680+0000 7f8c31e0bb80 1 --2- 172.21.15.1:0/3098126010 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578cb21f090 0x5578cb22a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:54.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.680+0000 7f8c31e0bb80 1 --2- 172.21.15.1:0/3098126010 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578cb21f9b0 0x5578cb227450 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:54.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.680+0000 7f8c31e0bb80 1 -- 172.21.15.1:0/3098126010 >> 172.21.15.1:0/3098126010 conn(0x5578cb17ee30 msgr2=0x5578cb21b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:54.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.680+0000 7f8c04c00700 0 client.26980 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:31:54.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.680+0000 7f8c31e0bb80 1 -- 172.21.15.1:0/3098126010 shutdown_connections 2024-02-02T15:31:54.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.680+0000 7f8c31e0bb80 1 -- 172.21.15.1:0/3098126010 wait complete. 2024-02-02T15:31:54.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.680+0000 7f8c31e0bb80 20 client.26980 trim_cache size 0 max 16384 2024-02-02T15:31:54.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.702+0000 7f8c31e0bb80 1 Processor -- start 2024-02-02T15:31:54.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.702+0000 7f8c31e0bb80 1 -- start start 2024-02-02T15:31:54.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.702+0000 7f8c31e0bb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578cb21f090 0x5578cb231250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:54.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.703+0000 7f8c31e0bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578cb21f9b0 0x5578cb229d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:54.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.703+0000 7f8c31e0bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5578cb22a860 0x5578cb2261b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:54.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.703+0000 7f8c31e0bb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578caff4720 con 0x5578cb21f090 2024-02-02T15:31:54.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.703+0000 7f8c31e0bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578cb237710 con 0x5578cb22a860 2024-02-02T15:31:54.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.703+0000 7f8c31e0bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578cb21ca20 con 0x5578cb21f9b0 2024-02-02T15:31:54.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.703+0000 7f8c16a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578cb21f9b0 0x5578cb229d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:54.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.703+0000 7f8c17e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5578cb22a860 0x5578cb2261b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:54.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.703+0000 7f8c16a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578cb21f9b0 0x5578cb229d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:47862/0 (socket says 172.21.15.1:47862) 2024-02-02T15:31:54.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.703+0000 7f8c16a00700 1 -- 172.21.15.1:0/4170456540 learned_addr learned my addr 172.21.15.1:0/4170456540 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:31:54.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.703+0000 7f8c17400700 1 --2- 172.21.15.1:0/4170456540 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578cb21f090 0x5578cb231250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:54.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.703+0000 7f8c16a00700 1 -- 172.21.15.1:0/4170456540 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5578cb22a860 msgr2=0x5578cb2261b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:54.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.703+0000 7f8c16a00700 1 --2- 172.21.15.1:0/4170456540 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5578cb22a860 0x5578cb2261b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:54.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.703+0000 7f8c16a00700 1 -- 172.21.15.1:0/4170456540 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578cb21f090 msgr2=0x5578cb231250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:54.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.703+0000 7f8c16a00700 1 --2- 172.21.15.1:0/4170456540 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578cb21f090 0x5578cb231250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:54.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.703+0000 7f8c16a00700 1 -- 172.21.15.1:0/4170456540 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5578cb136f20 con 0x5578cb21f9b0 2024-02-02T15:31:54.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.703+0000 7f8c17e00700 1 --2- 172.21.15.1:0/4170456540 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5578cb22a860 0x5578cb2261b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:31:54.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.704+0000 7f8c16a00700 1 --2- 172.21.15.1:0/4170456540 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578cb21f9b0 0x5578cb229d00 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f8c00009fe0 tx=0x7f8c0000fbd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=abc18aaaafea1227 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:54.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.704+0000 7f8c0d200700 1 -- 172.21.15.1:0/4170456540 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c00016070 con 0x5578cb21f9b0 2024-02-02T15:31:54.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.704+0000 7f8c0d200700 1 -- 172.21.15.1:0/4170456540 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c000092d0 con 0x5578cb21f9b0 2024-02-02T15:31:54.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.704+0000 7f8c0d200700 1 -- 172.21.15.1:0/4170456540 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c000038e0 con 0x5578cb21f9b0 2024-02-02T15:31:54.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.704+0000 7f8c31e0bb80 1 -- 172.21.15.1:0/4170456540 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578cb21f9b0 msgr2=0x5578cb229d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:54.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.704+0000 7f8c31e0bb80 1 --2- 172.21.15.1:0/4170456540 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578cb21f9b0 0x5578cb229d00 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f8c00009fe0 tx=0x7f8c0000fbd0 comp rx=0 tx=0).stop 2024-02-02T15:31:54.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.705+0000 7f8c31e0bb80 1 -- 172.21.15.1:0/4170456540 shutdown_connections 2024-02-02T15:31:54.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.705+0000 7f8c31e0bb80 1 --2- 172.21.15.1:0/4170456540 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5578cb22a860 0x5578cb2261b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:54.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.705+0000 7f8c31e0bb80 1 --2- 172.21.15.1:0/4170456540 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578cb21f090 0x5578cb231250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:54.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.705+0000 7f8c31e0bb80 1 --2- 172.21.15.1:0/4170456540 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578cb21f9b0 0x5578cb229d00 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:54.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.705+0000 7f8c31e0bb80 1 -- 172.21.15.1:0/4170456540 >> 172.21.15.1:0/4170456540 conn(0x5578cb17ee30 msgr2=0x5578cb225aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:54.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.705+0000 7f8c31e0bb80 1 -- 172.21.15.1:0/4170456540 shutdown_connections 2024-02-02T15:31:54.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.705+0000 7f8c31e0bb80 1 -- 172.21.15.1:0/4170456540 wait complete. 2024-02-02T15:31:54.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.705+0000 7f8c31e0bb80 1 -- start start 2024-02-02T15:31:54.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.705+0000 7f8c31e0bb80 1 Processor -- start 2024-02-02T15:31:54.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.706+0000 7f8c31e0bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578cb21f090 0x5578cb2854c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:54.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.706+0000 7f8c31e0bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5578cb21f9b0 0x5578cb285a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:54.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.706+0000 7f8c31e0bb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578cb22a860 0x5578cb2305c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:54.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.706+0000 7f8c31e0bb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578cb230b90 con 0x5578cb22a860 2024-02-02T15:31:54.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.706+0000 7f8c31e0bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578cb230d00 con 0x5578cb21f9b0 2024-02-02T15:31:54.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.706+0000 7f8c31e0bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578cb230ec0 con 0x5578cb21f090 2024-02-02T15:31:54.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.706+0000 7f8c31e0bb80 10 client.0 _ll_register_callbacks cb 0x5578cb10e310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:31:54.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.706+0000 7f8c17400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578cb21f090 0x5578cb2854c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:54.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.706+0000 7f8c16a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5578cb21f9b0 0x5578cb285a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:54.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.706+0000 7f8c17400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578cb21f090 0x5578cb2854c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:47872/0 (socket says 172.21.15.1:47872) 2024-02-02T15:31:54.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.706+0000 7f8c17400700 1 -- 172.21.15.1:0/1159253802 learned_addr learned my addr 172.21.15.1:0/1159253802 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:31:54.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.706+0000 7f8c17e00700 1 --2- 172.21.15.1:0/1159253802 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578cb22a860 0x5578cb2305c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:54.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.707+0000 7f8c17400700 1 -- 172.21.15.1:0/1159253802 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5578cb21f9b0 msgr2=0x5578cb285a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:54.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.707+0000 7f8c17400700 1 --2- 172.21.15.1:0/1159253802 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5578cb21f9b0 0x5578cb285a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:54.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.707+0000 7f8c17400700 1 -- 172.21.15.1:0/1159253802 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578cb22a860 msgr2=0x5578cb2305c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:54.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.707+0000 7f8c17400700 1 --2- 172.21.15.1:0/1159253802 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578cb22a860 0x5578cb2305c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:54.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.707+0000 7f8c17400700 1 -- 172.21.15.1:0/1159253802 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5578caff2430 con 0x5578cb21f090 2024-02-02T15:31:54.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.707+0000 7f8c17400700 1 --2- 172.21.15.1:0/1159253802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578cb21f090 0x5578cb2854c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8c08011da0 tx=0x7f8c08013040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=577379985e9b8fe2 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:54.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.707+0000 7f8c04c00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:31:54.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.707+0000 7f8c04c00700 1 -- 172.21.15.1:0/1159253802 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c080288f0 con 0x5578cb21f090 2024-02-02T15:31:54.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.707+0000 7f8c31e0bb80 1 -- 172.21.15.1:0/1159253802 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8c10004e60 con 0x5578cb21f090 2024-02-02T15:31:54.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.707+0000 7f8c04c00700 1 -- 172.21.15.1:0/1159253802 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f8c08028a50 con 0x5578cb21f090 2024-02-02T15:31:54.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.707+0000 7f8c04c00700 1 -- 172.21.15.1:0/1159253802 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c08028c10 con 0x5578cb21f090 2024-02-02T15:31:54.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.708+0000 7f8c31e0bb80 10 client.26986 fetch_fsmap learned FSMap version 35 2024-02-02T15:31:54.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.708+0000 7f8c31e0bb80 1 -- 172.21.15.1:0/1159253802 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5578caf9ebe0 con 0x5578cb21f090 2024-02-02T15:31:54.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.708+0000 7f8c04c00700 1 -- 172.21.15.1:0/1159253802 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c10004e60 con 0x5578cb21f090 2024-02-02T15:31:54.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.708+0000 7f8c04c00700 1 -- 172.21.15.1:0/1159253802 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 35) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8c080363f0 con 0x5578cb21f090 2024-02-02T15:31:54.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.708+0000 7f8c31e0bb80 10 client.26986 fetch_fsmap finished waiting for FSMap version 35 2024-02-02T15:31:54.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.708+0000 7f8c31e0bb80 10 client.26986 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:31:54.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.708+0000 7f8c31e0bb80 20 client.26986 populate_metadata read hostname 'smithi001' 2024-02-02T15:31:54.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.708+0000 7f8c31e0bb80 1 --2- 172.21.15.1:0/1159253802 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5578cb28e4e0 0x5578cb290900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:54.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.708+0000 7f8c31e0bb80 4 client.26986 mds_command: new command op to 24557 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-02T15:31:54.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.708+0000 7f8c31e0bb80 1 -- 172.21.15.1:0/1159253802 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5578cb12ffd0 con 0x5578cb28e4e0 2024-02-02T15:31:54.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.709+0000 7f8c16a00700 1 --2- 172.21.15.1:0/1159253802 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5578cb28e4e0 0x5578cb290900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:54.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.709+0000 7f8c16a00700 1 --2- 172.21.15.1:0/1159253802 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5578cb28e4e0 0x5578cb290900 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99999a770073f47f server_cookie=874150a9852a63c7 in_seq=0 out_seq=0 2024-02-02T15:31:54.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.709+0000 7f8c04c00700 10 client.26986 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:31:54.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.710+0000 7f8c04c00700 1 -- 172.21.15.1:0/1159253802 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5578cb12ffd0 con 0x5578cb28e4e0 2024-02-02T15:31:54.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.710+0000 7f8c04c00700 10 client.26986 handle_command_reply: tid=0 2024-02-02T15:31:54.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.710+0000 7f8c31e0bb80 1 client.26986 shutdown 2024-02-02T15:31:54.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.710+0000 7f8c31e0bb80 1 -- 172.21.15.1:0/1159253802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578cb21f090 msgr2=0x5578cb2854c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:54.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.710+0000 7f8c31e0bb80 1 --2- 172.21.15.1:0/1159253802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578cb21f090 0x5578cb2854c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8c08011da0 tx=0x7f8c08013040 comp rx=0 tx=0).stop 2024-02-02T15:31:54.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.710+0000 7f8c31e0bb80 1 -- 172.21.15.1:0/1159253802 shutdown_connections 2024-02-02T15:31:54.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.710+0000 7f8c31e0bb80 1 --2- 172.21.15.1:0/1159253802 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5578cb28e4e0 0x5578cb290900 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:54.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.710+0000 7f8c31e0bb80 1 --2- 172.21.15.1:0/1159253802 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5578cb22a860 0x5578cb2305c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:54.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.710+0000 7f8c31e0bb80 1 --2- 172.21.15.1:0/1159253802 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5578cb21f090 0x5578cb2854c0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:54.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.710+0000 7f8c31e0bb80 1 --2- 172.21.15.1:0/1159253802 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5578cb21f9b0 0x5578cb285a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:54.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.711+0000 7f8c31e0bb80 1 -- 172.21.15.1:0/1159253802 >> 172.21.15.1:0/1159253802 conn(0x5578cb17ee30 msgr2=0x5578cb226330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:54.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.710+0000 7f8c04c00700 0 client.26986 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:31:54.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.711+0000 7f8c31e0bb80 1 -- 172.21.15.1:0/1159253802 shutdown_connections 2024-02-02T15:31:54.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.711+0000 7f8c31e0bb80 1 -- 172.21.15.1:0/1159253802 wait complete. 2024-02-02T15:31:54.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.711+0000 7f8c31e0bb80 20 client.26986 trim_cache size 0 max 16384 2024-02-02T15:31:54.713 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:31:54.714 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2024-02-02T15:31:54.714 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "b3c6d97b-8664-4a16-9c80-5ab1eb773225", 2024-02-02T15:31:54.714 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2024-02-02T15:31:54.714 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:31:54.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.711+0000 7f8c1e200700 1 -- 172.21.15.1:0/2860078631 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f8bf80a0690 msgr2=0x7f8bf80a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:54.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.711+0000 7f8c1e200700 1 --2- 172.21.15.1:0/2860078631 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f8bf80a0690 0x7f8bf80a2b50 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8c08009fd0 tx=0x7f8c08009260 comp rx=0 tx=0).stop 2024-02-02T15:31:54.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.711+0000 7f8c1e200700 1 -- 172.21.15.1:0/2860078631 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8c1819d950 msgr2=0x7f8c1819ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:54.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.711+0000 7f8c1e200700 1 --2- 172.21.15.1:0/2860078631 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8c1819d950 0x7f8c1819ddd0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f8c10009850 tx=0x7f8c100077f0 comp rx=0 tx=0).stop 2024-02-02T15:31:54.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.712+0000 7f8c1e200700 1 -- 172.21.15.1:0/2860078631 shutdown_connections 2024-02-02T15:31:54.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.712+0000 7f8c1e200700 1 --2- 172.21.15.1:0/2860078631 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f8bf80a0690 0x7f8bf80a2b50 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:54.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.712+0000 7f8c1e200700 1 --2- 172.21.15.1:0/2860078631 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8c1819d950 0x7f8c1819ddd0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:54.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.712+0000 7f8c1e200700 1 --2- 172.21.15.1:0/2860078631 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8c18100ff0 0x7f8c18198fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:54.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.712+0000 7f8c1e200700 1 --2- 172.21.15.1:0/2860078631 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c18106110 0x7f8c181994f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:54.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.712+0000 7f8c1e200700 1 -- 172.21.15.1:0/2860078631 >> 172.21.15.1:0/2860078631 conn(0x7f8c180fc810 msgr2=0x7f8c180fd240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:54.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.712+0000 7f8c1e200700 1 -- 172.21.15.1:0/2860078631 shutdown_connections 2024-02-02T15:31:54.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.712+0000 7f8c1e200700 1 -- 172.21.15.1:0/2860078631 wait complete. 2024-02-02T15:31:54.728 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:31:54 smithi001 ceph-mon[35470]: pgmap v150: 129 pgs: 129 active+clean; 172 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 231 op/s 2024-02-02T15:31:54.730 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-02T15:31:54.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:31:54 smithi071 ceph-mon[44707]: pgmap v150: 129 pgs: 129 active+clean; 172 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 231 op/s 2024-02-02T15:31:54.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:31:54 smithi142 ceph-mon[45728]: pgmap v150: 129 pgs: 129 active+clean; 172 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 231 op/s 2024-02-02T15:31:54.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.944+0000 7f2a47000700 1 -- 172.21.15.1:0/3989844094 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a401019b0 msgr2=0x7f2a401059f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:54.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.944+0000 7f2a47000700 1 --2- 172.21.15.1:0/3989844094 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a401019b0 0x7f2a401059f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2a28009990 tx=0x7f2a28009ca0 comp rx=0 tx=0).stop 2024-02-02T15:31:54.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.944+0000 7f2a47000700 1 -- 172.21.15.1:0/3989844094 shutdown_connections 2024-02-02T15:31:54.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.944+0000 7f2a47000700 1 --2- 172.21.15.1:0/3989844094 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a40106120 0x7f2a4010dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:54.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.944+0000 7f2a47000700 1 --2- 172.21.15.1:0/3989844094 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a40101000 0x7f2a401013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:54.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.944+0000 7f2a47000700 1 --2- 172.21.15.1:0/3989844094 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a401019b0 0x7f2a401059f0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:54.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.944+0000 7f2a47000700 1 -- 172.21.15.1:0/3989844094 >> 172.21.15.1:0/3989844094 conn(0x7f2a400fc860 msgr2=0x7f2a400fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:54.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.944+0000 7f2a47000700 1 -- 172.21.15.1:0/3989844094 shutdown_connections 2024-02-02T15:31:54.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.944+0000 7f2a47000700 1 -- 172.21.15.1:0/3989844094 wait complete. 2024-02-02T15:31:54.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.945+0000 7f2a47000700 1 Processor -- start 2024-02-02T15:31:54.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.945+0000 7f2a47000700 1 -- start start 2024-02-02T15:31:54.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.946+0000 7f2a47000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a40101000 0x7f2a40196fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:54.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.946+0000 7f2a47000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a40106120 0x7f2a401974f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:54.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.946+0000 7f2a47000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a4019b950 0x7f2a4019bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:54.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.946+0000 7f2a47000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4019c3d0 con 0x7f2a40101000 2024-02-02T15:31:54.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.946+0000 7f2a47000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4019c540 con 0x7f2a4019b950 2024-02-02T15:31:54.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.946+0000 7f2a47000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4019c700 con 0x7f2a40106120 2024-02-02T15:31:54.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.946+0000 7f2a3fe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a40101000 0x7f2a40196fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:54.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.946+0000 7f2a3f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a40106120 0x7f2a401974f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:54.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.946+0000 7f2a45000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a4019b950 0x7f2a4019bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:54.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.946+0000 7f2a45000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a4019b950 0x7f2a4019bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59594/0 (socket says 172.21.15.1:59594) 2024-02-02T15:31:54.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.946+0000 7f2a45000700 1 -- 172.21.15.1:0/3958160036 learned_addr learned my addr 172.21.15.1:0/3958160036 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:31:54.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.946+0000 7f2a45000700 1 -- 172.21.15.1:0/3958160036 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a40106120 msgr2=0x7f2a401974f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:54.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.946+0000 7f2a45000700 1 --2- 172.21.15.1:0/3958160036 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a40106120 0x7f2a401974f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:54.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.946+0000 7f2a45000700 1 -- 172.21.15.1:0/3958160036 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a40101000 msgr2=0x7f2a40196fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:54.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.946+0000 7f2a45000700 1 --2- 172.21.15.1:0/3958160036 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a40101000 0x7f2a40196fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:54.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.946+0000 7f2a45000700 1 -- 172.21.15.1:0/3958160036 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a280095f0 con 0x7f2a4019b950 2024-02-02T15:31:54.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.947+0000 7f2a45000700 1 --2- 172.21.15.1:0/3958160036 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a4019b950 0x7f2a4019bdd0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2a3800ef30 tx=0x7f2a3800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:54.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.947+0000 7f2a3cc00700 1 -- 172.21.15.1:0/3958160036 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a38018070 con 0x7f2a4019b950 2024-02-02T15:31:54.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.947+0000 7f2a47000700 1 -- 172.21.15.1:0/3958160036 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a401a3c00 con 0x7f2a4019b950 2024-02-02T15:31:54.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.947+0000 7f2a3cc00700 1 -- 172.21.15.1:0/3958160036 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a3800f040 con 0x7f2a4019b950 2024-02-02T15:31:54.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.947+0000 7f2a3cc00700 1 -- 172.21.15.1:0/3958160036 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3800cd00 con 0x7f2a4019b950 2024-02-02T15:31:54.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.947+0000 7f2a47000700 1 -- 172.21.15.1:0/3958160036 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a401a4130 con 0x7f2a4019b950 2024-02-02T15:31:54.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.951+0000 7f2a3cc00700 1 -- 172.21.15.1:0/3958160036 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 27) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f2a38003a00 con 0x7f2a4019b950 2024-02-02T15:31:54.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.953+0000 7f2a5ab59b80 1 Processor -- start 2024-02-02T15:31:54.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.953+0000 7f2a5ab59b80 1 -- start start 2024-02-02T15:31:54.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.953+0000 7f2a3cc00700 1 --2- 172.21.15.1:0/3958160036 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f2a200a0690 0x7f2a200a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:54.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.953+0000 7f2a5ab59b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55df2cb0d010 0x55df2cb0d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:54.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.953+0000 7f2a3cc00700 1 -- 172.21.15.1:0/3958160036 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f2a380cbb40 con 0x7f2a4019b950 2024-02-02T15:31:54.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.953+0000 7f2a5ab59b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55df2cb0d930 0x55df2cb182a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:54.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.953+0000 7f2a5ab59b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df2cb187e0 0x55df2cb1abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:54.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.953+0000 7f2a5ab59b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df2ca814c0 con 0x55df2cb0d930 2024-02-02T15:31:54.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.953+0000 7f2a5ab59b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df2ca65aa0 con 0x55df2cb0d010 2024-02-02T15:31:54.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.953+0000 7f2a5ab59b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df2ca63460 con 0x55df2cb187e0 2024-02-02T15:31:54.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.953+0000 7f2a3fe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55df2cb0d010 0x55df2cb0d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:54.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.954+0000 7f2a3f400700 1 --2- 172.21.15.1:0/3958160036 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f2a200a0690 0x7f2a200a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:54.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.954+0000 7f2a3fe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55df2cb0d010 0x55df2cb0d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59608/0 (socket says 172.21.15.1:59608) 2024-02-02T15:31:54.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.954+0000 7f2a3fe00700 1 -- 172.21.15.1:0/2091483632 learned_addr learned my addr 172.21.15.1:0/2091483632 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:31:54.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.954+0000 7f2a45000700 1 --2- 172.21.15.1:0/2091483632 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df2cb187e0 0x55df2cb1abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:54.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.954+0000 7f2a3f400700 1 --2- 172.21.15.1:0/2091483632 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55df2cb0d930 0x55df2cb182a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:54.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.954+0000 7f2a3fe00700 1 -- 172.21.15.1:0/2091483632 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df2cb187e0 msgr2=0x55df2cb1abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:54.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.954+0000 7f2a3fe00700 1 --2- 172.21.15.1:0/2091483632 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df2cb187e0 0x55df2cb1abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:54.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.954+0000 7f2a3fe00700 1 -- 172.21.15.1:0/2091483632 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55df2cb0d930 msgr2=0x55df2cb182a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:54.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.954+0000 7f2a3fe00700 1 --2- 172.21.15.1:0/2091483632 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55df2cb0d930 0x55df2cb182a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:54.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.954+0000 7f2a3fe00700 1 -- 172.21.15.1:0/2091483632 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df2c8e03b0 con 0x55df2cb0d010 2024-02-02T15:31:54.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.954+0000 7f2a3fe00700 1 --2- 172.21.15.1:0/2091483632 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55df2cb0d010 0x55df2cb0d3f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f2a30009db0 tx=0x7f2a30011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64a5f55d0489335e server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:54.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.954+0000 7f2a3f400700 1 --2- 172.21.15.1:0/3958160036 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f2a200a0690 0x7f2a200a2b50 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2a401985c0 tx=0x7f2a2801a040 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:54.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.955+0000 7f2a35600700 1 -- 172.21.15.1:0/2091483632 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a30004230 con 0x55df2cb0d010 2024-02-02T15:31:54.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.955+0000 7f2a35600700 1 -- 172.21.15.1:0/2091483632 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a30004390 con 0x55df2cb0d010 2024-02-02T15:31:54.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.955+0000 7f2a35600700 1 -- 172.21.15.1:0/2091483632 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a300198c0 con 0x55df2cb0d010 2024-02-02T15:31:54.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.955+0000 7f2a5ab59b80 1 -- 172.21.15.1:0/2091483632 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55df2cb0d010 msgr2=0x55df2cb0d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:54.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.955+0000 7f2a5ab59b80 1 --2- 172.21.15.1:0/2091483632 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55df2cb0d010 0x55df2cb0d3f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f2a30009db0 tx=0x7f2a30011720 comp rx=0 tx=0).stop 2024-02-02T15:31:54.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.955+0000 7f2a5ab59b80 1 -- 172.21.15.1:0/2091483632 shutdown_connections 2024-02-02T15:31:54.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.955+0000 7f2a5ab59b80 1 --2- 172.21.15.1:0/2091483632 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df2cb187e0 0x55df2cb1abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:54.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.955+0000 7f2a5ab59b80 1 --2- 172.21.15.1:0/2091483632 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55df2cb0d010 0x55df2cb0d3f0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:54.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.955+0000 7f2a5ab59b80 1 --2- 172.21.15.1:0/2091483632 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55df2cb0d930 0x55df2cb182a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:54.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.955+0000 7f2a5ab59b80 1 -- 172.21.15.1:0/2091483632 >> 172.21.15.1:0/2091483632 conn(0x55df2ca6cdb0 msgr2=0x55df2cb093b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:54.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.956+0000 7f2a5ab59b80 1 -- 172.21.15.1:0/2091483632 shutdown_connections 2024-02-02T15:31:54.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.956+0000 7f2a5ab59b80 1 -- 172.21.15.1:0/2091483632 wait complete. 2024-02-02T15:31:54.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.956+0000 7f2a5ab59b80 1 -- start start 2024-02-02T15:31:54.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.956+0000 7f2a5ab59b80 1 Processor -- start 2024-02-02T15:31:54.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.957+0000 7f2a5ab59b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df2cb0d010 0x55df2cb18210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:54.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.957+0000 7f2a5ab59b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55df2cb0d930 0x55df2cb15420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:54.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.957+0000 7f2a5ab59b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55df2cb187e0 0x55df2cb15b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:54.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.957+0000 7f2a5ab59b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df2ca5ed50 con 0x55df2cb0d930 2024-02-02T15:31:54.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.957+0000 7f2a5ab59b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df2c9fad80 con 0x55df2cb187e0 2024-02-02T15:31:54.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.957+0000 7f2a5ab59b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df2c9faa20 con 0x55df2cb0d010 2024-02-02T15:31:54.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.957+0000 7f2a5ab59b80 10 client.0 _ll_register_callbacks cb 0x55df2c9fc290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:31:54.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.957+0000 7f2a3fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df2cb0d010 0x55df2cb18210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:54.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.957+0000 7f2a3fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df2cb0d010 0x55df2cb18210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:47900/0 (socket says 172.21.15.1:47900) 2024-02-02T15:31:54.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.957+0000 7f2a3f400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55df2cb0d930 0x55df2cb15420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:54.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.957+0000 7f2a3fe00700 1 -- 172.21.15.1:0/764895760 learned_addr learned my addr 172.21.15.1:0/764895760 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:31:54.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.957+0000 7f2a45000700 1 --2- 172.21.15.1:0/764895760 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55df2cb187e0 0x55df2cb15b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:54.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.957+0000 7f2a3fe00700 1 -- 172.21.15.1:0/764895760 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55df2cb187e0 msgr2=0x55df2cb15b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:54.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.957+0000 7f2a3fe00700 1 --2- 172.21.15.1:0/764895760 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55df2cb187e0 0x55df2cb15b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:54.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.957+0000 7f2a3fe00700 1 -- 172.21.15.1:0/764895760 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55df2cb0d930 msgr2=0x55df2cb15420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:54.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.957+0000 7f2a3fe00700 1 --2- 172.21.15.1:0/764895760 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55df2cb0d930 0x55df2cb15420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:54.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.957+0000 7f2a3fe00700 1 -- 172.21.15.1:0/764895760 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a300099e0 con 0x55df2cb0d010 2024-02-02T15:31:54.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.957+0000 7f2a45000700 1 --2- 172.21.15.1:0/764895760 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55df2cb187e0 0x55df2cb15b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:31:54.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.958+0000 7f2a3fe00700 1 --2- 172.21.15.1:0/764895760 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df2cb0d010 0x55df2cb18210 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f2a30000c00 tx=0x7f2a300038b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0ae9e17cd64ba3 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:54.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.958+0000 7f2a2e000700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:31:54.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.958+0000 7f2a2e000700 1 -- 172.21.15.1:0/764895760 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a30005450 con 0x55df2cb0d010 2024-02-02T15:31:54.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.958+0000 7f2a5ab59b80 1 -- 172.21.15.1:0/764895760 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55df2cb010f0 con 0x55df2cb0d010 2024-02-02T15:31:54.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.958+0000 7f2a2e000700 1 -- 172.21.15.1:0/764895760 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f2a300055b0 con 0x55df2cb0d010 2024-02-02T15:31:54.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.958+0000 7f2a2e000700 1 -- 172.21.15.1:0/764895760 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a30004500 con 0x55df2cb0d010 2024-02-02T15:31:54.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.958+0000 7f2a5ab59b80 10 client.26998 fetch_fsmap learned FSMap version 35 2024-02-02T15:31:54.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.958+0000 7f2a5ab59b80 1 -- 172.21.15.1:0/764895760 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55df2c88cb60 con 0x55df2cb0d010 2024-02-02T15:31:54.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.959+0000 7f2a2e000700 1 -- 172.21.15.1:0/764895760 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55df2cb010f0 con 0x55df2cb0d010 2024-02-02T15:31:54.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.959+0000 7f2a2e000700 1 -- 172.21.15.1:0/764895760 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 35) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2a30021040 con 0x55df2cb0d010 2024-02-02T15:31:54.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.959+0000 7f2a5ab59b80 10 client.26998 fetch_fsmap finished waiting for FSMap version 35 2024-02-02T15:31:54.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.959+0000 7f2a5ab59b80 10 client.26998 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:31:54.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.959+0000 7f2a5ab59b80 20 client.26998 populate_metadata read hostname 'smithi001' 2024-02-02T15:31:54.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.959+0000 7f2a5ab59b80 1 --2- 172.21.15.1:0/764895760 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55df2cb27fd0 0x55df2cb2a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:54.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.959+0000 7f2a5ab59b80 4 client.26998 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:31:54.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.959+0000 7f2a5ab59b80 1 -- 172.21.15.1:0/764895760 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55df2ca57dd0 con 0x55df2cb27fd0 2024-02-02T15:31:54.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.959+0000 7f2a3f400700 1 --2- 172.21.15.1:0/764895760 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55df2cb27fd0 0x55df2cb2a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:54.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.960+0000 7f2a3f400700 1 --2- 172.21.15.1:0/764895760 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55df2cb27fd0 0x55df2cb2a3f0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b0523abf0bfbe46 server_cookie=fa6fb1268bf3bba6 in_seq=0 out_seq=0 2024-02-02T15:31:54.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.960+0000 7f2a2e000700 10 client.26998 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:31:54.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.960+0000 7f2a2e000700 1 -- 172.21.15.1:0/764895760 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55df2ca57dd0 con 0x55df2cb27fd0 2024-02-02T15:31:54.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.960+0000 7f2a2e000700 10 client.26998 handle_command_reply: tid=0 2024-02-02T15:31:54.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.960+0000 7f2a5ab59b80 1 client.26998 shutdown 2024-02-02T15:31:54.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.961+0000 7f2a5ab59b80 1 -- 172.21.15.1:0/764895760 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df2cb0d010 msgr2=0x55df2cb18210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:54.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.961+0000 7f2a5ab59b80 1 --2- 172.21.15.1:0/764895760 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df2cb0d010 0x55df2cb18210 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f2a30000c00 tx=0x7f2a300038b0 comp rx=0 tx=0).stop 2024-02-02T15:31:54.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.961+0000 7f2a5ab59b80 1 -- 172.21.15.1:0/764895760 shutdown_connections 2024-02-02T15:31:54.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.961+0000 7f2a5ab59b80 1 --2- 172.21.15.1:0/764895760 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55df2cb27fd0 0x55df2cb2a3f0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:54.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.961+0000 7f2a5ab59b80 1 --2- 172.21.15.1:0/764895760 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55df2cb187e0 0x55df2cb15b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:54.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.961+0000 7f2a5ab59b80 1 --2- 172.21.15.1:0/764895760 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df2cb0d010 0x55df2cb18210 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:54.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.961+0000 7f2a5ab59b80 1 --2- 172.21.15.1:0/764895760 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55df2cb0d930 0x55df2cb15420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:54.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.961+0000 7f2a2e000700 0 client.26998 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:31:54.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.961+0000 7f2a5ab59b80 1 -- 172.21.15.1:0/764895760 >> 172.21.15.1:0/764895760 conn(0x55df2ca6cdb0 msgr2=0x55df2cb08fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:54.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.961+0000 7f2a5ab59b80 1 -- 172.21.15.1:0/764895760 shutdown_connections 2024-02-02T15:31:54.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.961+0000 7f2a5ab59b80 1 -- 172.21.15.1:0/764895760 wait complete. 2024-02-02T15:31:54.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.961+0000 7f2a5ab59b80 20 client.26998 trim_cache size 0 max 16384 2024-02-02T15:31:54.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.982+0000 7f2a5ab59b80 1 Processor -- start 2024-02-02T15:31:54.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.982+0000 7f2a5ab59b80 1 -- start start 2024-02-02T15:31:54.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.982+0000 7f2a5ab59b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55df2cb0d010 0x55df2cb0f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:54.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.982+0000 7f2a5ab59b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55df2cb0d930 0x55df2cb17d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:54.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.982+0000 7f2a5ab59b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df2cb187e0 0x55df2cb14cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:54.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.982+0000 7f2a5ab59b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df2c8e26a0 con 0x55df2cb0d930 2024-02-02T15:31:54.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.982+0000 7f2a5ab59b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df2cb16190 con 0x55df2cb0d010 2024-02-02T15:31:54.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.982+0000 7f2a5ab59b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df2cb10ff0 con 0x55df2cb187e0 2024-02-02T15:31:54.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.982+0000 7f2a3fe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55df2cb0d010 0x55df2cb0f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:54.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.983+0000 7f2a3fe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55df2cb0d010 0x55df2cb0f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59634/0 (socket says 172.21.15.1:59634) 2024-02-02T15:31:54.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.983+0000 7f2a3fe00700 1 -- 172.21.15.1:0/1340806308 learned_addr learned my addr 172.21.15.1:0/1340806308 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:31:54.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.983+0000 7f2a3f400700 1 --2- 172.21.15.1:0/1340806308 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55df2cb0d930 0x55df2cb17d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:54.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.983+0000 7f2a45000700 1 --2- 172.21.15.1:0/1340806308 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df2cb187e0 0x55df2cb14cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:54.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.983+0000 7f2a3fe00700 1 -- 172.21.15.1:0/1340806308 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df2cb187e0 msgr2=0x55df2cb14cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:54.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.983+0000 7f2a3fe00700 1 --2- 172.21.15.1:0/1340806308 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df2cb187e0 0x55df2cb14cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:54.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.983+0000 7f2a3fe00700 1 -- 172.21.15.1:0/1340806308 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55df2cb0d930 msgr2=0x55df2cb17d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:54.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.983+0000 7f2a3fe00700 1 --2- 172.21.15.1:0/1340806308 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55df2cb0d930 0x55df2cb17d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:54.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.983+0000 7f2a3fe00700 1 -- 172.21.15.1:0/1340806308 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df2ca24ea0 con 0x55df2cb0d010 2024-02-02T15:31:54.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.983+0000 7f2a3fe00700 1 --2- 172.21.15.1:0/1340806308 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55df2cb0d010 0x55df2cb0f840 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f2a30009700 tx=0x7f2a30005a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83084daaa02704bc server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:54.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.983+0000 7f2a36000700 1 -- 172.21.15.1:0/1340806308 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3000e3b0 con 0x55df2cb0d010 2024-02-02T15:31:54.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.983+0000 7f2a36000700 1 -- 172.21.15.1:0/1340806308 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a30023e80 con 0x55df2cb0d010 2024-02-02T15:31:54.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.983+0000 7f2a36000700 1 -- 172.21.15.1:0/1340806308 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a30005710 con 0x55df2cb0d010 2024-02-02T15:31:54.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.984+0000 7f2a5ab59b80 1 -- 172.21.15.1:0/1340806308 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55df2cb0d010 msgr2=0x55df2cb0f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:54.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.984+0000 7f2a5ab59b80 1 --2- 172.21.15.1:0/1340806308 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55df2cb0d010 0x55df2cb0f840 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f2a30009700 tx=0x7f2a30005a00 comp rx=0 tx=0).stop 2024-02-02T15:31:54.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.984+0000 7f2a5ab59b80 1 -- 172.21.15.1:0/1340806308 shutdown_connections 2024-02-02T15:31:54.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.984+0000 7f2a5ab59b80 1 --2- 172.21.15.1:0/1340806308 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df2cb187e0 0x55df2cb14cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:54.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.984+0000 7f2a5ab59b80 1 --2- 172.21.15.1:0/1340806308 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55df2cb0d010 0x55df2cb0f840 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:54.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.984+0000 7f2a5ab59b80 1 --2- 172.21.15.1:0/1340806308 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55df2cb0d930 0x55df2cb17d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:54.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.984+0000 7f2a5ab59b80 1 -- 172.21.15.1:0/1340806308 >> 172.21.15.1:0/1340806308 conn(0x55df2ca6cdb0 msgr2=0x55df2cb24520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:54.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.984+0000 7f2a5ab59b80 1 -- 172.21.15.1:0/1340806308 shutdown_connections 2024-02-02T15:31:54.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.984+0000 7f2a5ab59b80 1 -- 172.21.15.1:0/1340806308 wait complete. 2024-02-02T15:31:54.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.985+0000 7f2a5ab59b80 1 -- start start 2024-02-02T15:31:54.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.985+0000 7f2a5ab59b80 1 Processor -- start 2024-02-02T15:31:54.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.985+0000 7f2a5ab59b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55df2cb0d010 0x55df2cb1c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:54.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.985+0000 7f2a5ab59b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55df2cb0d930 0x55df2cb1cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:54.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.985+0000 7f2a5ab59b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df2cb187e0 0x55df2cb1d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:54.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.985+0000 7f2a5ab59b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df2cb1f7a0 con 0x55df2cb0d010 2024-02-02T15:31:54.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.985+0000 7f2a5ab59b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df2cb78b10 con 0x55df2cb0d930 2024-02-02T15:31:54.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.985+0000 7f2a5ab59b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df2cb78ca0 con 0x55df2cb187e0 2024-02-02T15:31:54.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.985+0000 7f2a5ab59b80 10 client.0 _ll_register_callbacks cb 0x55df2c9fc290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:31:54.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.986+0000 7f2a3f400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55df2cb0d930 0x55df2cb1cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:54.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.986+0000 7f2a3fe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55df2cb0d010 0x55df2cb1c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:54.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.986+0000 7f2a45000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df2cb187e0 0x55df2cb1d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:54.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.986+0000 7f2a3f400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55df2cb0d930 0x55df2cb1cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59648/0 (socket says 172.21.15.1:59648) 2024-02-02T15:31:54.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.986+0000 7f2a3f400700 1 -- 172.21.15.1:0/3167961779 learned_addr learned my addr 172.21.15.1:0/3167961779 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:31:54.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.986+0000 7f2a45000700 1 -- 172.21.15.1:0/3167961779 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55df2cb0d930 msgr2=0x55df2cb1cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:54.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.986+0000 7f2a45000700 1 --2- 172.21.15.1:0/3167961779 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55df2cb0d930 0x55df2cb1cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:54.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.986+0000 7f2a45000700 1 -- 172.21.15.1:0/3167961779 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55df2cb0d010 msgr2=0x55df2cb1c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:31:54.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.986+0000 7f2a45000700 1 --2- 172.21.15.1:0/3167961779 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55df2cb0d010 0x55df2cb1c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:54.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.986+0000 7f2a45000700 1 -- 172.21.15.1:0/3167961779 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55df2c88cb60 con 0x55df2cb187e0 2024-02-02T15:31:54.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.986+0000 7f2a3f400700 1 --2- 172.21.15.1:0/3167961779 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55df2cb0d930 0x55df2cb1cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:31:54.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.986+0000 7f2a45000700 1 --2- 172.21.15.1:0/3167961779 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df2cb187e0 0x55df2cb1d1c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2a380c8f40 tx=0x7f2a380c94a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e37a894375b1913 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:31:54.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.986+0000 7f2a2e000700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:31:54.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.986+0000 7f2a2e000700 1 -- 172.21.15.1:0/3167961779 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3801b320 con 0x55df2cb187e0 2024-02-02T15:31:54.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.986+0000 7f2a2e000700 1 -- 172.21.15.1:0/3167961779 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f2a380994a0 con 0x55df2cb187e0 2024-02-02T15:31:54.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.986+0000 7f2a5ab59b80 1 -- 172.21.15.1:0/3167961779 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55df2ca1df50 con 0x55df2cb187e0 2024-02-02T15:31:54.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.987+0000 7f2a2e000700 1 -- 172.21.15.1:0/3167961779 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a3800ce60 con 0x55df2cb187e0 2024-02-02T15:31:54.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.987+0000 7f2a5ab59b80 10 client.27004 fetch_fsmap learned FSMap version 35 2024-02-02T15:31:54.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.987+0000 7f2a5ab59b80 1 -- 172.21.15.1:0/3167961779 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55df2c8e4cd0 con 0x55df2cb187e0 2024-02-02T15:31:54.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.987+0000 7f2a2e000700 1 -- 172.21.15.1:0/3167961779 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55df2ca1df50 con 0x55df2cb187e0 2024-02-02T15:31:54.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.987+0000 7f2a2e000700 1 -- 172.21.15.1:0/3167961779 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 35) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2a380a08f0 con 0x55df2cb187e0 2024-02-02T15:31:54.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.987+0000 7f2a5ab59b80 10 client.27004 fetch_fsmap finished waiting for FSMap version 35 2024-02-02T15:31:54.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.987+0000 7f2a5ab59b80 10 client.27004 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:31:54.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.987+0000 7f2a5ab59b80 20 client.27004 populate_metadata read hostname 'smithi001' 2024-02-02T15:31:54.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.987+0000 7f2a5ab59b80 1 --2- 172.21.15.1:0/3167961779 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55df2cb7a500 0x55df2cb7c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:31:54.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.987+0000 7f2a5ab59b80 4 client.27004 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2024-02-02T15:31:54.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.987+0000 7f2a5ab59b80 1 -- 172.21.15.1:0/3167961779 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55df2c8adcf0 con 0x55df2cb7a500 2024-02-02T15:31:54.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.987+0000 7f2a3fe00700 1 --2- 172.21.15.1:0/3167961779 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55df2cb7a500 0x55df2cb7c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:31:54.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.988+0000 7f2a3fe00700 1 --2- 172.21.15.1:0/3167961779 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55df2cb7a500 0x55df2cb7c8f0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6767b14573b180e7 server_cookie=7765e108a59c6a52 in_seq=0 out_seq=0 2024-02-02T15:31:54.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.988+0000 7f2a2e000700 10 client.27004 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:31:54.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.988+0000 7f2a2e000700 1 -- 172.21.15.1:0/3167961779 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55df2c8adcf0 con 0x55df2cb7a500 2024-02-02T15:31:54.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.988+0000 7f2a2e000700 10 client.27004 handle_command_reply: tid=0 2024-02-02T15:31:54.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.988+0000 7f2a5ab59b80 1 client.27004 shutdown 2024-02-02T15:31:54.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.988+0000 7f2a5ab59b80 1 -- 172.21.15.1:0/3167961779 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df2cb187e0 msgr2=0x55df2cb1d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:54.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.988+0000 7f2a5ab59b80 1 --2- 172.21.15.1:0/3167961779 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df2cb187e0 0x55df2cb1d1c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2a380c8f40 tx=0x7f2a380c94a0 comp rx=0 tx=0).stop 2024-02-02T15:31:54.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.988+0000 7f2a5ab59b80 1 -- 172.21.15.1:0/3167961779 shutdown_connections 2024-02-02T15:31:54.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.988+0000 7f2a5ab59b80 1 --2- 172.21.15.1:0/3167961779 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55df2cb7a500 0x55df2cb7c8f0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:54.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.988+0000 7f2a5ab59b80 1 --2- 172.21.15.1:0/3167961779 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df2cb187e0 0x55df2cb1d1c0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:54.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.988+0000 7f2a5ab59b80 1 --2- 172.21.15.1:0/3167961779 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55df2cb0d010 0x55df2cb1c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:54.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.988+0000 7f2a2e000700 0 client.27004 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:31:54.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.988+0000 7f2a5ab59b80 1 --2- 172.21.15.1:0/3167961779 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55df2cb0d930 0x55df2cb1cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:54.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.988+0000 7f2a5ab59b80 1 -- 172.21.15.1:0/3167961779 >> 172.21.15.1:0/3167961779 conn(0x55df2ca6cdb0 msgr2=0x55df2cb085c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:54.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.989+0000 7f2a5ab59b80 1 -- 172.21.15.1:0/3167961779 shutdown_connections 2024-02-02T15:31:54.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.989+0000 7f2a5ab59b80 1 -- 172.21.15.1:0/3167961779 wait complete. 2024-02-02T15:31:54.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.989+0000 7f2a5ab59b80 20 client.27004 trim_cache size 0 max 16384 2024-02-02T15:31:54.995 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:31:54.995 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-02T15:31:54.995 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2024-02-02T15:31:54.995 INFO:teuthology.orchestra.run.smithi001.stdout: "b3c6d97b-8664-4a16-9c80-5ab1eb773225": { 2024-02-02T15:31:54.995 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2024-02-02T15:31:54.995 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "b3c6d97b-8664-4a16-9c80-5ab1eb773225", 2024-02-02T15:31:54.995 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,force" 2024-02-02T15:31:54.996 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:31:54.996 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:31:54.996 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:31:54.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.989+0000 7f2a47000700 1 -- 172.21.15.1:0/3958160036 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f2a200a0690 msgr2=0x7f2a200a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:54.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.989+0000 7f2a47000700 1 --2- 172.21.15.1:0/3958160036 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f2a200a0690 0x7f2a200a2b50 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2a401985c0 tx=0x7f2a2801a040 comp rx=0 tx=0).stop 2024-02-02T15:31:54.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.989+0000 7f2a47000700 1 -- 172.21.15.1:0/3958160036 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a4019b950 msgr2=0x7f2a4019bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:31:54.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.989+0000 7f2a47000700 1 --2- 172.21.15.1:0/3958160036 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a4019b950 0x7f2a4019bdd0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2a3800ef30 tx=0x7f2a3800c3a0 comp rx=0 tx=0).stop 2024-02-02T15:31:54.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.989+0000 7f2a47000700 1 -- 172.21.15.1:0/3958160036 shutdown_connections 2024-02-02T15:31:54.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.989+0000 7f2a47000700 1 --2- 172.21.15.1:0/3958160036 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f2a200a0690 0x7f2a200a2b50 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:54.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.989+0000 7f2a47000700 1 --2- 172.21.15.1:0/3958160036 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a4019b950 0x7f2a4019bdd0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:54.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.989+0000 7f2a47000700 1 --2- 172.21.15.1:0/3958160036 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a40101000 0x7f2a40196fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:54.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.989+0000 7f2a47000700 1 --2- 172.21.15.1:0/3958160036 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a40106120 0x7f2a401974f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:31:54.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.989+0000 7f2a47000700 1 -- 172.21.15.1:0/3958160036 >> 172.21.15.1:0/3958160036 conn(0x7f2a400fc860 msgr2=0x7f2a400fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:31:54.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.990+0000 7f2a47000700 1 -- 172.21.15.1:0/3958160036 shutdown_connections 2024-02-02T15:31:54.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:31:54.990+0000 7f2a47000700 1 -- 172.21.15.1:0/3958160036 wait complete. 2024-02-02T15:31:55.007 INFO:tasks.cephfs.filesystem:scrub status for tag:b3c6d97b-8664-4a16-9c80-5ab1eb773225 - {'path': '/', 'tag': 'b3c6d97b-8664-4a16-9c80-5ab1eb773225', 'options': 'recursive,force'} 2024-02-02T15:31:55.233 INFO:tasks.workunit.client.0.smithi001.stdout: 1 293804 9.30 MB/sec execute 89 sec latency 13.479 ms 2024-02-02T15:31:55.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:31:55 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:31:55.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:31:55 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:31:55.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:31:55 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:31:55.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:31:55 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:31:55.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:31:55 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:31:55.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:31:55 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:31:55.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:31:55 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:31:55.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:31:55 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:31:55.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:31:55 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:31:55.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:31:55 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:31:55.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:31:55 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:31:55.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:31:55 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:31:55.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:31:55 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:31:55.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:31:55 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:31:55.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:31:55 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:31:55.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:31:55 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:31:55.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:31:55 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:31:55.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:31:55 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:31:56.233 INFO:tasks.workunit.client.0.smithi001.stdout: 1 295100 9.27 MB/sec execute 90 sec latency 13.879 ms 2024-02-02T15:31:56.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:31:56 smithi071 ceph-mon[44707]: scrub queued for path: / 2024-02-02T15:31:56.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:31:56 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:31:56.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:31:56 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:31:56.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:31:56 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:31:56.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:31:56 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:31:56.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:31:56 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:31:56.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:31:56 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:31:56.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:31:56 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:31:56.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:31:56 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:31:56.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:31:56 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:31:56.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:31:56 smithi071 ceph-mon[44707]: pgmap v151: 129 pgs: 129 active+clean; 176 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.6 MiB/s wr, 342 op/s 2024-02-02T15:31:56.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:31:56 smithi142 ceph-mon[45728]: scrub queued for path: / 2024-02-02T15:31:56.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:31:56 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:31:56.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:31:56 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:31:56.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:31:56 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:31:56.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:31:56 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:31:56.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:31:56 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:31:56.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:31:56 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:31:56.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:31:56 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:31:56.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:31:56 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:31:56.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:31:56 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:31:56.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:31:56 smithi142 ceph-mon[45728]: pgmap v151: 129 pgs: 129 active+clean; 176 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.6 MiB/s wr, 342 op/s 2024-02-02T15:31:56.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:31:56 smithi001 ceph-mon[35470]: scrub queued for path: / 2024-02-02T15:31:56.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:31:56 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:31:56.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:31:56 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:31:56.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:31:56 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:31:56.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:31:56 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:31:56.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:31:56 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:31:56.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:31:56 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:31:56.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:31:56 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:31:56.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:31:56 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:31:56.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:31:56 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:31:56.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:31:56 smithi001 ceph-mon[35470]: pgmap v151: 129 pgs: 129 active+clean; 176 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.6 MiB/s wr, 342 op/s 2024-02-02T15:31:57.233 INFO:tasks.workunit.client.0.smithi001.stdout: 1 297191 9.29 MB/sec execute 91 sec latency 26.282 ms 2024-02-02T15:31:58.233 INFO:tasks.workunit.client.0.smithi001.stdout: 1 298443 9.27 MB/sec execute 92 sec latency 23.060 ms 2024-02-02T15:31:59.233 INFO:tasks.workunit.client.0.smithi001.stdout: 1 300410 9.25 MB/sec execute 93 sec latency 13.482 ms 2024-02-02T15:31:59.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:31:59 smithi071 ceph-mon[44707]: pgmap v152: 129 pgs: 129 active+clean; 176 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 227 op/s 2024-02-02T15:31:59.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:31:59 smithi142 ceph-mon[45728]: pgmap v152: 129 pgs: 129 active+clean; 176 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 227 op/s 2024-02-02T15:31:59.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:31:59 smithi001 ceph-mon[35470]: pgmap v152: 129 pgs: 129 active+clean; 176 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 227 op/s 2024-02-02T15:32:00.233 INFO:tasks.workunit.client.0.smithi001.stdout: 1 301915 9.28 MB/sec execute 94 sec latency 13.084 ms 2024-02-02T15:32:01.233 INFO:tasks.workunit.client.0.smithi001.stdout: 1 302552 9.20 MB/sec execute 95 sec latency 59.080 ms 2024-02-02T15:32:01.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:32:01 smithi071 ceph-mon[44707]: pgmap v153: 129 pgs: 129 active+clean; 176 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 227 op/s 2024-02-02T15:32:01.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:32:01 smithi142 ceph-mon[45728]: pgmap v153: 129 pgs: 129 active+clean; 176 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 227 op/s 2024-02-02T15:32:01.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:32:01 smithi001 ceph-mon[35470]: pgmap v153: 129 pgs: 129 active+clean; 176 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.4 MiB/s wr, 227 op/s 2024-02-02T15:32:02.233 INFO:tasks.workunit.client.0.smithi001.stdout: 1 304240 9.20 MB/sec execute 96 sec latency 56.527 ms 2024-02-02T15:32:02.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:32:02 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:32:02.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:32:02 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:32:02.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:32:02 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:32:02.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:32:02 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:32:02.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:32:02 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:32:02.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:32:02 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:32:02.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:32:02 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:32:02.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:32:02 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:32:02.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:32:02 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:32:02.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:32:02 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:32:02.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:32:02 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:32:02.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:32:02 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:32:02.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:32:02 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:32:02.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:32:02 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:32:02.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:32:02 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:32:03.233 INFO:tasks.workunit.client.0.smithi001.stdout: 1 305574 9.19 MB/sec execute 97 sec latency 19.387 ms 2024-02-02T15:32:03.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:32:03 smithi071 ceph-mon[44707]: pgmap v154: 129 pgs: 129 active+clean; 181 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.7 MiB/s wr, 358 op/s 2024-02-02T15:32:03.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:32:03 smithi142 ceph-mon[45728]: pgmap v154: 129 pgs: 129 active+clean; 181 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.7 MiB/s wr, 358 op/s 2024-02-02T15:32:03.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:32:03 smithi001 ceph-mon[35470]: pgmap v154: 129 pgs: 129 active+clean; 181 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.7 MiB/s wr, 358 op/s 2024-02-02T15:32:04.233 INFO:tasks.workunit.client.0.smithi001.stdout: 1 307673 9.18 MB/sec execute 98 sec latency 11.744 ms 2024-02-02T15:32:05.234 INFO:tasks.workunit.client.0.smithi001.stdout: 1 309120 9.20 MB/sec execute 99 sec latency 11.017 ms 2024-02-02T15:32:05.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:32:05 smithi071 ceph-mon[44707]: pgmap v155: 129 pgs: 129 active+clean; 181 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 242 op/s 2024-02-02T15:32:05.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:32:05 smithi142 ceph-mon[45728]: pgmap v155: 129 pgs: 129 active+clean; 181 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 242 op/s 2024-02-02T15:32:05.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:32:05 smithi001 ceph-mon[35470]: pgmap v155: 129 pgs: 129 active+clean; 181 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.4 MiB/s wr, 242 op/s 2024-02-02T15:32:06.234 INFO:tasks.workunit.client.0.smithi001.stdout: 1 311184 9.19 MB/sec execute 100 sec latency 13.894 ms 2024-02-02T15:32:07.234 INFO:tasks.workunit.client.0.smithi001.stdout: 1 312583 9.21 MB/sec execute 101 sec latency 69.902 ms 2024-02-02T15:32:07.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:32:07 smithi071 ceph-mon[44707]: pgmap v156: 129 pgs: 129 active+clean; 187 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.6 MiB/s wr, 356 op/s 2024-02-02T15:32:07.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:32:07 smithi142 ceph-mon[45728]: pgmap v156: 129 pgs: 129 active+clean; 187 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.6 MiB/s wr, 356 op/s 2024-02-02T15:32:07.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:32:07 smithi001 ceph-mon[35470]: pgmap v156: 129 pgs: 129 active+clean; 187 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 3.6 MiB/s wr, 356 op/s 2024-02-02T15:32:08.234 INFO:tasks.workunit.client.0.smithi001.stdout: 1 314556 9.18 MB/sec execute 102 sec latency 19.761 ms 2024-02-02T15:32:09.234 INFO:tasks.workunit.client.0.smithi001.stdout: 1 316102 9.21 MB/sec execute 103 sec latency 31.487 ms 2024-02-02T15:32:09.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:32:09 smithi071 ceph-mon[44707]: pgmap v157: 129 pgs: 129 active+clean; 187 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 244 op/s 2024-02-02T15:32:09.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:32:09 smithi142 ceph-mon[45728]: pgmap v157: 129 pgs: 129 active+clean; 187 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 244 op/s 2024-02-02T15:32:09.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:32:09 smithi001 ceph-mon[35470]: pgmap v157: 129 pgs: 129 active+clean; 187 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 244 op/s 2024-02-02T15:32:10.236 INFO:tasks.workunit.client.0.smithi001.stdout: 1 318040 9.19 MB/sec execute 104 sec latency 49.846 ms 2024-02-02T15:32:11.234 INFO:tasks.workunit.client.0.smithi001.stdout: 1 319574 9.22 MB/sec execute 105 sec latency 12.298 ms 2024-02-02T15:32:11.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:32:11 smithi071 ceph-mon[44707]: pgmap v158: 129 pgs: 129 active+clean; 187 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 244 op/s 2024-02-02T15:32:11.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:32:11 smithi142 ceph-mon[45728]: pgmap v158: 129 pgs: 129 active+clean; 187 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 244 op/s 2024-02-02T15:32:11.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:32:11 smithi001 ceph-mon[35470]: pgmap v158: 129 pgs: 129 active+clean; 187 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.5 MiB/s wr, 244 op/s 2024-02-02T15:32:12.234 INFO:tasks.workunit.client.0.smithi001.stdout: 1 320433 9.15 MB/sec execute 106 sec latency 55.495 ms 2024-02-02T15:32:12.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:32:12 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:32:12.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:32:12 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:32:12.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:32:12 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:32:13.234 INFO:tasks.workunit.client.0.smithi001.stdout: 1 322875 9.21 MB/sec execute 107 sec latency 16.374 ms 2024-02-02T15:32:13.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:32:13 smithi071 ceph-mon[44707]: pgmap v159: 129 pgs: 129 active+clean; 193 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.9 MiB/s wr, 356 op/s 2024-02-02T15:32:13.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:32:13 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:32:13.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:32:13 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:32:13.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:32:13 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:32:13.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:32:13 smithi142 ceph-mon[45728]: pgmap v159: 129 pgs: 129 active+clean; 193 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.9 MiB/s wr, 356 op/s 2024-02-02T15:32:13.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:32:13 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:32:13.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:32:13 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:32:13.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:32:13 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:32:13.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:32:13 smithi001 ceph-mon[35470]: pgmap v159: 129 pgs: 129 active+clean; 193 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 3.9 MiB/s wr, 356 op/s 2024-02-02T15:32:13.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:32:13 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:32:13.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:32:13 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:32:13.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:32:13 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:32:14.234 INFO:tasks.workunit.client.0.smithi001.stdout: 1 323950 9.16 MB/sec execute 108 sec latency 12.047 ms 2024-02-02T15:32:15.234 INFO:tasks.workunit.client.0.smithi001.stdout: 1 326477 9.22 MB/sec execute 109 sec latency 10.366 ms 2024-02-02T15:32:15.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:32:15 smithi071 ceph-mon[44707]: pgmap v160: 129 pgs: 129 active+clean; 193 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 225 op/s 2024-02-02T15:32:15.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:32:15 smithi142 ceph-mon[45728]: pgmap v160: 129 pgs: 129 active+clean; 193 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 225 op/s 2024-02-02T15:32:15.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:32:15 smithi001 ceph-mon[35470]: pgmap v160: 129 pgs: 129 active+clean; 193 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.5 MiB/s wr, 225 op/s 2024-02-02T15:32:16.234 INFO:tasks.workunit.client.0.smithi001.stdout: 1 327516 9.17 MB/sec execute 110 sec latency 20.074 ms 2024-02-02T15:32:17.234 INFO:tasks.workunit.client.0.smithi001.stdout: 1 329847 9.22 MB/sec execute 111 sec latency 46.063 ms 2024-02-02T15:32:17.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:32:17 smithi071 ceph-mon[44707]: pgmap v161: 129 pgs: 129 active+clean; 200 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.9 MiB/s wr, 338 op/s 2024-02-02T15:32:17.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:32:17 smithi142 ceph-mon[45728]: pgmap v161: 129 pgs: 129 active+clean; 200 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.9 MiB/s wr, 338 op/s 2024-02-02T15:32:17.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:32:17 smithi001 ceph-mon[35470]: pgmap v161: 129 pgs: 129 active+clean; 200 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.9 MiB/s wr, 338 op/s 2024-02-02T15:32:18.234 INFO:tasks.workunit.client.0.smithi001.stdout: 1 330809 9.16 MB/sec execute 112 sec latency 23.618 ms 2024-02-02T15:32:19.234 INFO:tasks.workunit.client.0.smithi001.stdout: 1 333179 9.20 MB/sec execute 113 sec latency 12.938 ms 2024-02-02T15:32:19.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:32:19 smithi071 ceph-mon[44707]: pgmap v162: 129 pgs: 129 active+clean; 200 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 224 op/s 2024-02-02T15:32:19.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:32:19 smithi142 ceph-mon[45728]: pgmap v162: 129 pgs: 129 active+clean; 200 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 224 op/s 2024-02-02T15:32:19.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:32:19 smithi001 ceph-mon[35470]: pgmap v162: 129 pgs: 129 active+clean; 200 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 224 op/s 2024-02-02T15:32:20.234 INFO:tasks.workunit.client.0.smithi001.stdout: 1 334241 9.17 MB/sec execute 114 sec latency 10.884 ms 2024-02-02T15:32:21.234 INFO:tasks.workunit.client.0.smithi001.stdout: 1 336634 9.20 MB/sec execute 115 sec latency 12.055 ms 2024-02-02T15:32:21.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:32:21 smithi071 ceph-mon[44707]: pgmap v163: 129 pgs: 129 active+clean; 200 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 224 op/s 2024-02-02T15:32:21.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:32:21 smithi142 ceph-mon[45728]: pgmap v163: 129 pgs: 129 active+clean; 200 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 224 op/s 2024-02-02T15:32:21.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:32:21 smithi001 ceph-mon[35470]: pgmap v163: 129 pgs: 129 active+clean; 200 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.7 MiB/s wr, 224 op/s 2024-02-02T15:32:22.234 INFO:tasks.workunit.client.0.smithi001.stdout: 1 337658 9.17 MB/sec execute 116 sec latency 42.417 ms 2024-02-02T15:32:23.234 INFO:tasks.workunit.client.0.smithi001.stdout: 1 339367 9.15 MB/sec execute 117 sec latency 25.382 ms 2024-02-02T15:32:23.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:32:23 smithi071 ceph-mon[44707]: pgmap v164: 129 pgs: 129 active+clean; 206 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.1 MiB/s wr, 339 op/s 2024-02-02T15:32:23.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:32:23 smithi142 ceph-mon[45728]: pgmap v164: 129 pgs: 129 active+clean; 206 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.1 MiB/s wr, 339 op/s 2024-02-02T15:32:23.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:32:23 smithi001 ceph-mon[35470]: pgmap v164: 129 pgs: 129 active+clean; 206 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.1 MiB/s wr, 339 op/s 2024-02-02T15:32:24.235 INFO:tasks.workunit.client.0.smithi001.stdout: 1 340906 9.18 MB/sec execute 118 sec latency 15.655 ms 2024-02-02T15:32:25.009 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-02T15:32:25.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.220+0000 7f2103400700 1 -- 172.21.15.1:0/4046230287 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f20fc106ed0 msgr2=0x7f20fc1072b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:25.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.220+0000 7f2103400700 1 --2- 172.21.15.1:0/4046230287 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f20fc106ed0 0x7f20fc1072b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f20ec0099c0 tx=0x7f20ec009cd0 comp rx=0 tx=0).stop 2024-02-02T15:32:25.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.220+0000 7f2103400700 1 -- 172.21.15.1:0/4046230287 shutdown_connections 2024-02-02T15:32:25.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.220+0000 7f2103400700 1 --2- 172.21.15.1:0/4046230287 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f20fc0689b0 0x7f20fc10dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:25.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.220+0000 7f2103400700 1 --2- 172.21.15.1:0/4046230287 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f20fc106ed0 0x7f20fc1072b0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:25.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.220+0000 7f2103400700 1 --2- 172.21.15.1:0/4046230287 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20fc068010 0x7f20fc068470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:25.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.220+0000 7f2103400700 1 -- 172.21.15.1:0/4046230287 >> 172.21.15.1:0/4046230287 conn(0x7f20fc0757d0 msgr2=0x7f20fc075be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:32:25.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.220+0000 7f2103400700 1 -- 172.21.15.1:0/4046230287 shutdown_connections 2024-02-02T15:32:25.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.220+0000 7f2103400700 1 -- 172.21.15.1:0/4046230287 wait complete. 2024-02-02T15:32:25.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.221+0000 7f2103400700 1 Processor -- start 2024-02-02T15:32:25.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.221+0000 7f2103400700 1 -- start start 2024-02-02T15:32:25.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.222+0000 7f2103400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f20fc068010 0x7f20fc1991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:25.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.222+0000 7f2103400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f20fc0689b0 0x7f20fc199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:25.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.222+0000 7f2103400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20fc106ed0 0x7f20fc19db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:25.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.222+0000 7f2103400700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20fc19e1b0 con 0x7f20fc0689b0 2024-02-02T15:32:25.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.222+0000 7f2103400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20fc19e320 con 0x7f20fc106ed0 2024-02-02T15:32:25.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.222+0000 7f2103400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20fc19e4e0 con 0x7f20fc068010 2024-02-02T15:32:25.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.222+0000 7f20fbe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f20fc0689b0 0x7f20fc199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:25.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.222+0000 7f2101400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20fc106ed0 0x7f20fc19db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:25.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.222+0000 7f2101400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20fc106ed0 0x7f20fc19db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:47702/0 (socket says 172.21.15.1:47702) 2024-02-02T15:32:25.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.222+0000 7f2100a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f20fc068010 0x7f20fc1991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:25.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.222+0000 7f2101400700 1 -- 172.21.15.1:0/2373962120 learned_addr learned my addr 172.21.15.1:0/2373962120 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:32:25.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.222+0000 7f2101400700 1 -- 172.21.15.1:0/2373962120 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f20fc068010 msgr2=0x7f20fc1991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:25.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.222+0000 7f2101400700 1 --2- 172.21.15.1:0/2373962120 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f20fc068010 0x7f20fc1991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:25.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.223+0000 7f2101400700 1 -- 172.21.15.1:0/2373962120 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f20fc0689b0 msgr2=0x7f20fc199710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:25.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.223+0000 7f2101400700 1 --2- 172.21.15.1:0/2373962120 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f20fc0689b0 0x7f20fc199710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:25.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.223+0000 7f2101400700 1 -- 172.21.15.1:0/2373962120 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20f4009c40 con 0x7f20fc106ed0 2024-02-02T15:32:25.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.223+0000 7f20fbe00700 1 --2- 172.21.15.1:0/2373962120 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f20fc0689b0 0x7f20fc199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:32:25.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.223+0000 7f2101400700 1 --2- 172.21.15.1:0/2373962120 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20fc106ed0 0x7f20fc19db70 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f20f400c370 tx=0x7f20f400c680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:32:25.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.223+0000 7f20f9600700 1 -- 172.21.15.1:0/2373962120 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f4017070 con 0x7f20fc106ed0 2024-02-02T15:32:25.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.223+0000 7f20f9600700 1 -- 172.21.15.1:0/2373962120 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f20f400f040 con 0x7f20fc106ed0 2024-02-02T15:32:25.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.223+0000 7f2103400700 1 -- 172.21.15.1:0/2373962120 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20ec0095f0 con 0x7f20fc106ed0 2024-02-02T15:32:25.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.223+0000 7f20f9600700 1 -- 172.21.15.1:0/2373962120 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f400e050 con 0x7f20fc106ed0 2024-02-02T15:32:25.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.223+0000 7f2103400700 1 -- 172.21.15.1:0/2373962120 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20fc1a5e20 con 0x7f20fc106ed0 2024-02-02T15:32:25.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.226+0000 7f20f9600700 1 -- 172.21.15.1:0/2373962120 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 27) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f20f400cb60 con 0x7f20fc106ed0 2024-02-02T15:32:25.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.228+0000 7f2116fc2b80 1 Processor -- start 2024-02-02T15:32:25.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.228+0000 7f2116fc2b80 1 -- start start 2024-02-02T15:32:25.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.228+0000 7f2116fc2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba43954010 0x55ba439543f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:25.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.228+0000 7f2116fc2b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba43954930 0x55ba4395f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:25.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.228+0000 7f2116fc2b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba4395f7e0 0x55ba43961bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:25.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.229+0000 7f2116fc2b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba438c84c0 con 0x55ba43954930 2024-02-02T15:32:25.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.229+0000 7f2116fc2b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba438acaa0 con 0x55ba4395f7e0 2024-02-02T15:32:25.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.229+0000 7f2116fc2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba438aa460 con 0x55ba43954010 2024-02-02T15:32:25.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.229+0000 7f20f9600700 1 --2- 172.21.15.1:0/2373962120 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f20dc0a0690 0x7f20dc0a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:25.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.229+0000 7f20fbe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba43954010 0x55ba439543f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:25.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.229+0000 7f20fbe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba43954010 0x55ba439543f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:37074/0 (socket says 172.21.15.1:37074) 2024-02-02T15:32:25.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.229+0000 7f20fbe00700 1 -- 172.21.15.1:0/3355620293 learned_addr learned my addr 172.21.15.1:0/3355620293 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:32:25.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.229+0000 7f20fbe00700 1 --2- 172.21.15.1:0/3355620293 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba4395f7e0 0x55ba43961bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:25.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.229+0000 7f20f9600700 1 -- 172.21.15.1:0/2373962120 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f20f40cb9f0 con 0x7f20fc106ed0 2024-02-02T15:32:25.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.229+0000 7f2100a00700 1 --2- 172.21.15.1:0/3355620293 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba43954930 0x55ba4395f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:25.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.229+0000 7f2101400700 1 --2- 172.21.15.1:0/2373962120 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f20dc0a0690 0x7f20dc0a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:25.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.229+0000 7f20fbe00700 1 -- 172.21.15.1:0/3355620293 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba4395f7e0 msgr2=0x55ba43961bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:32:25.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.229+0000 7f20fbe00700 1 --2- 172.21.15.1:0/3355620293 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba4395f7e0 0x55ba43961bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:25.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.229+0000 7f20fbe00700 1 -- 172.21.15.1:0/3355620293 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba43954930 msgr2=0x55ba4395f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:32:25.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.229+0000 7f20fbe00700 1 --2- 172.21.15.1:0/3355620293 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba43954930 0x55ba4395f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:25.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.229+0000 7f20fbe00700 1 -- 172.21.15.1:0/3355620293 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba437273b0 con 0x55ba43954010 2024-02-02T15:32:25.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.229+0000 7f20fbe00700 1 --2- 172.21.15.1:0/3355620293 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba43954010 0x55ba439543f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f20e4011ca0 tx=0x7f20e4009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0db1bf03ea5ee37 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:32:25.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.230+0000 7f2101400700 1 --2- 172.21.15.1:0/2373962120 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f20dc0a0690 0x7f20dc0a2b50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f20f40c8df0 tx=0x7f20f4009630 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:32:25.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.230+0000 7f20f1c00700 1 -- 172.21.15.1:0/3355620293 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e400f6d0 con 0x55ba43954010 2024-02-02T15:32:25.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.230+0000 7f20f1c00700 1 -- 172.21.15.1:0/3355620293 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f20e400fcf0 con 0x55ba43954010 2024-02-02T15:32:25.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.230+0000 7f20f1c00700 1 -- 172.21.15.1:0/3355620293 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e401fc00 con 0x55ba43954010 2024-02-02T15:32:25.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.230+0000 7f2116fc2b80 1 -- 172.21.15.1:0/3355620293 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba43954010 msgr2=0x55ba439543f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:25.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.230+0000 7f2116fc2b80 1 --2- 172.21.15.1:0/3355620293 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba43954010 0x55ba439543f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f20e4011ca0 tx=0x7f20e4009c30 comp rx=0 tx=0).stop 2024-02-02T15:32:25.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.230+0000 7f2116fc2b80 1 -- 172.21.15.1:0/3355620293 shutdown_connections 2024-02-02T15:32:25.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.230+0000 7f2116fc2b80 1 --2- 172.21.15.1:0/3355620293 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba4395f7e0 0x55ba43961bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:25.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.230+0000 7f2116fc2b80 1 --2- 172.21.15.1:0/3355620293 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba43954010 0x55ba439543f0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:25.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.230+0000 7f2116fc2b80 1 --2- 172.21.15.1:0/3355620293 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba43954930 0x55ba4395f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:25.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.230+0000 7f2116fc2b80 1 -- 172.21.15.1:0/3355620293 >> 172.21.15.1:0/3355620293 conn(0x55ba438b3db0 msgr2=0x55ba439503b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:32:25.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.231+0000 7f2116fc2b80 1 -- 172.21.15.1:0/3355620293 shutdown_connections 2024-02-02T15:32:25.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.231+0000 7f2116fc2b80 1 -- 172.21.15.1:0/3355620293 wait complete. 2024-02-02T15:32:25.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.231+0000 7f2116fc2b80 1 -- start start 2024-02-02T15:32:25.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.231+0000 7f2116fc2b80 1 Processor -- start 2024-02-02T15:32:25.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.232+0000 7f2116fc2b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba43954010 0x55ba4395f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:25.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.232+0000 7f2116fc2b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba43954930 0x55ba4395c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:25.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.232+0000 7f2116fc2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba4395f7e0 0x55ba4395cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:25.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.232+0000 7f2116fc2b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba438a5d50 con 0x55ba43954010 2024-02-02T15:32:25.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.232+0000 7f2116fc2b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba43841d80 con 0x55ba43954930 2024-02-02T15:32:25.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.232+0000 7f2116fc2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba43841a20 con 0x55ba4395f7e0 2024-02-02T15:32:25.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.232+0000 7f2116fc2b80 10 client.0 _ll_register_callbacks cb 0x55ba43843290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:32:25.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.232+0000 7f2100a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba43954930 0x55ba4395c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:25.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.232+0000 7f2100a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba43954930 0x55ba4395c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:47710/0 (socket says 172.21.15.1:47710) 2024-02-02T15:32:25.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.232+0000 7f2100a00700 1 -- 172.21.15.1:0/167465699 learned_addr learned my addr 172.21.15.1:0/167465699 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:32:25.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.232+0000 7f20fbe00700 1 --2- 172.21.15.1:0/167465699 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba43954010 0x55ba4395f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:25.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.232+0000 7f20fbe00700 1 --2- 172.21.15.1:0/167465699 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba4395f7e0 0x55ba4395cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:25.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.232+0000 7f2100a00700 1 -- 172.21.15.1:0/167465699 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba4395f7e0 msgr2=0x55ba4395cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:32:25.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.232+0000 7f2100a00700 1 --2- 172.21.15.1:0/167465699 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba4395f7e0 0x55ba4395cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:25.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.232+0000 7f2100a00700 1 -- 172.21.15.1:0/167465699 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba43954010 msgr2=0x55ba4395f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:32:25.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.232+0000 7f2100a00700 1 --2- 172.21.15.1:0/167465699 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba43954010 0x55ba4395f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:25.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.232+0000 7f2100a00700 1 -- 172.21.15.1:0/167465699 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20e4019040 con 0x55ba43954930 2024-02-02T15:32:25.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.232+0000 7f20fbe00700 1 --2- 172.21.15.1:0/167465699 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba43954010 0x55ba4395f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:32:25.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.233+0000 7f2100a00700 1 --2- 172.21.15.1:0/167465699 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba43954930 0x55ba4395c500 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f20ec0058e0 tx=0x7f20ec003f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24d962e5e47374f server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:32:25.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.233+0000 7f20ea000700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:32:25.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.233+0000 7f20ea000700 1 -- 172.21.15.1:0/167465699 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ec0056c0 con 0x55ba43954930 2024-02-02T15:32:25.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.233+0000 7f20ea000700 1 -- 172.21.15.1:0/167465699 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f20ec00b790 con 0x55ba43954930 2024-02-02T15:32:25.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.233+0000 7f2116fc2b80 1 -- 172.21.15.1:0/167465699 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ba439480f0 con 0x55ba43954930 2024-02-02T15:32:25.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.233+0000 7f20ea000700 1 -- 172.21.15.1:0/167465699 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f20ec00b950 con 0x55ba43954930 2024-02-02T15:32:25.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.233+0000 7f2116fc2b80 10 client.27017 fetch_fsmap learned FSMap version 35 2024-02-02T15:32:25.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.233+0000 7f2116fc2b80 1 -- 172.21.15.1:0/167465699 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55ba436d3b60 con 0x55ba43954930 2024-02-02T15:32:25.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.234+0000 7f20ea000700 1 -- 172.21.15.1:0/167465699 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ba439480f0 con 0x55ba43954930 2024-02-02T15:32:25.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.234+0000 7f20ea000700 1 -- 172.21.15.1:0/167465699 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 35) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f20ec00fb30 con 0x55ba43954930 2024-02-02T15:32:25.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.234+0000 7f2116fc2b80 10 client.27017 fetch_fsmap finished waiting for FSMap version 35 2024-02-02T15:32:25.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.234+0000 7f2116fc2b80 10 client.27017 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:32:25.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.234+0000 7f2116fc2b80 20 client.27017 populate_metadata read hostname 'smithi001' 2024-02-02T15:32:25.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.234+0000 7f2116fc2b80 1 --2- 172.21.15.1:0/167465699 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55ba4396f250 0x55ba43971670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:25.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.234+0000 7f2116fc2b80 4 client.27017 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:32:25.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.234+0000 7f2116fc2b80 1 -- 172.21.15.1:0/167465699 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ba4389edd0 con 0x55ba4396f250 2024-02-02T15:32:25.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.234+0000 7f20fbe00700 1 --2- 172.21.15.1:0/167465699 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55ba4396f250 0x55ba43971670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:25.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.235+0000 7f20fbe00700 1 --2- 172.21.15.1:0/167465699 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55ba4396f250 0x55ba43971670 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=135bef0023e06b82 server_cookie=b1ffba66505118e7 in_seq=0 out_seq=0 2024-02-02T15:32:25.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.235+0000 7f20ea000700 10 client.27017 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:32:25.239 INFO:tasks.workunit.client.0.smithi001.stdout: 1 342024 9.14 MB/sec execute 119 sec latency 28.524 ms 2024-02-02T15:32:25.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.236+0000 7f20ea000700 1 -- 172.21.15.1:0/167465699 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55ba4389edd0 con 0x55ba4396f250 2024-02-02T15:32:25.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.236+0000 7f20ea000700 10 client.27017 handle_command_reply: tid=0 2024-02-02T15:32:25.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.236+0000 7f2116fc2b80 1 client.27017 shutdown 2024-02-02T15:32:25.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.236+0000 7f2116fc2b80 1 -- 172.21.15.1:0/167465699 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba43954930 msgr2=0x55ba4395c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:25.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.236+0000 7f2116fc2b80 1 --2- 172.21.15.1:0/167465699 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba43954930 0x55ba4395c500 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f20ec0058e0 tx=0x7f20ec003f90 comp rx=0 tx=0).stop 2024-02-02T15:32:25.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.236+0000 7f2116fc2b80 1 -- 172.21.15.1:0/167465699 shutdown_connections 2024-02-02T15:32:25.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.236+0000 7f2116fc2b80 1 --2- 172.21.15.1:0/167465699 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55ba4396f250 0x55ba43971670 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:25.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.236+0000 7f2116fc2b80 1 --2- 172.21.15.1:0/167465699 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba4395f7e0 0x55ba4395cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:25.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.236+0000 7f20ea000700 0 client.27017 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:32:25.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.236+0000 7f2116fc2b80 1 --2- 172.21.15.1:0/167465699 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba43954010 0x55ba4395f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:25.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.236+0000 7f2116fc2b80 1 --2- 172.21.15.1:0/167465699 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba43954930 0x55ba4395c500 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:25.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.236+0000 7f2116fc2b80 1 -- 172.21.15.1:0/167465699 >> 172.21.15.1:0/167465699 conn(0x55ba438b3db0 msgr2=0x55ba4394ffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:32:25.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.236+0000 7f2116fc2b80 1 -- 172.21.15.1:0/167465699 shutdown_connections 2024-02-02T15:32:25.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.236+0000 7f2116fc2b80 1 -- 172.21.15.1:0/167465699 wait complete. 2024-02-02T15:32:25.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.237+0000 7f2116fc2b80 20 client.27017 trim_cache size 0 max 16384 2024-02-02T15:32:25.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.254+0000 7f2116fc2b80 1 Processor -- start 2024-02-02T15:32:25.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.254+0000 7f2116fc2b80 1 -- start start 2024-02-02T15:32:25.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.255+0000 7f2116fc2b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba43954010 0x55ba43955830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:25.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.255+0000 7f2116fc2b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba43954930 0x55ba4395edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:25.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.255+0000 7f2116fc2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba4395f7e0 0x55ba439577e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:25.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.255+0000 7f2116fc2b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba437296a0 con 0x55ba43954010 2024-02-02T15:32:25.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.255+0000 7f2116fc2b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba43951930 con 0x55ba43954930 2024-02-02T15:32:25.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.255+0000 7f2116fc2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba43955d70 con 0x55ba4395f7e0 2024-02-02T15:32:25.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.255+0000 7f20fbe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba43954010 0x55ba43955830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:25.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.255+0000 7f2100a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba43954930 0x55ba4395edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:25.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.255+0000 7f20fbe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba43954010 0x55ba43955830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39108/0 (socket says 172.21.15.1:39108) 2024-02-02T15:32:25.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.255+0000 7f20fbe00700 1 -- 172.21.15.1:0/1966836296 learned_addr learned my addr 172.21.15.1:0/1966836296 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:32:25.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.255+0000 7f20fbe00700 1 --2- 172.21.15.1:0/1966836296 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba4395f7e0 0x55ba439577e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:25.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.255+0000 7f2100a00700 1 -- 172.21.15.1:0/1966836296 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba4395f7e0 msgr2=0x55ba439577e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:32:25.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.255+0000 7f2100a00700 1 --2- 172.21.15.1:0/1966836296 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba4395f7e0 0x55ba439577e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:25.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.255+0000 7f2100a00700 1 -- 172.21.15.1:0/1966836296 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba43954010 msgr2=0x55ba43955830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:32:25.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.255+0000 7f2100a00700 1 --2- 172.21.15.1:0/1966836296 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba43954010 0x55ba43955830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:25.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.255+0000 7f2100a00700 1 -- 172.21.15.1:0/1966836296 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba4386bea0 con 0x55ba43954930 2024-02-02T15:32:25.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.256+0000 7f2100a00700 1 --2- 172.21.15.1:0/1966836296 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba43954930 0x55ba4395edb0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f20ec000c00 tx=0x7f20ec005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b6b6485217c1f57 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:32:25.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.256+0000 7f20f2600700 1 -- 172.21.15.1:0/1966836296 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ec009290 con 0x55ba43954930 2024-02-02T15:32:25.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.256+0000 7f20f2600700 1 -- 172.21.15.1:0/1966836296 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f20ec038070 con 0x55ba43954930 2024-02-02T15:32:25.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.256+0000 7f20f2600700 1 -- 172.21.15.1:0/1966836296 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ec032670 con 0x55ba43954930 2024-02-02T15:32:25.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.257+0000 7f2116fc2b80 1 -- 172.21.15.1:0/1966836296 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba43954930 msgr2=0x55ba4395edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:25.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.257+0000 7f2116fc2b80 1 --2- 172.21.15.1:0/1966836296 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba43954930 0x55ba4395edb0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f20ec000c00 tx=0x7f20ec005e80 comp rx=0 tx=0).stop 2024-02-02T15:32:25.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.257+0000 7f2116fc2b80 1 -- 172.21.15.1:0/1966836296 shutdown_connections 2024-02-02T15:32:25.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.257+0000 7f2116fc2b80 1 --2- 172.21.15.1:0/1966836296 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba4395f7e0 0x55ba439577e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:25.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.257+0000 7f2116fc2b80 1 --2- 172.21.15.1:0/1966836296 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba43954010 0x55ba43955830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:25.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.257+0000 7f2116fc2b80 1 --2- 172.21.15.1:0/1966836296 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba43954930 0x55ba4395edb0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:25.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.257+0000 7f2116fc2b80 1 -- 172.21.15.1:0/1966836296 >> 172.21.15.1:0/1966836296 conn(0x55ba438b3db0 msgr2=0x55ba4396b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:32:25.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.257+0000 7f2116fc2b80 1 -- 172.21.15.1:0/1966836296 shutdown_connections 2024-02-02T15:32:25.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.257+0000 7f2116fc2b80 1 -- 172.21.15.1:0/1966836296 wait complete. 2024-02-02T15:32:25.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.257+0000 7f2116fc2b80 1 -- start start 2024-02-02T15:32:25.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.257+0000 7f2116fc2b80 1 Processor -- start 2024-02-02T15:32:25.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.258+0000 7f2116fc2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba43954010 0x55ba43963700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:25.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.258+0000 7f2116fc2b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba43954930 0x55ba43963c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:25.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.258+0000 7f2116fc2b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba4395f7e0 0x55ba43964180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:25.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.258+0000 7f2116fc2b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba439bfd50 con 0x55ba43954930 2024-02-02T15:32:25.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.258+0000 7f2116fc2b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba439bfe90 con 0x55ba4395f7e0 2024-02-02T15:32:25.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.258+0000 7f2116fc2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba439bffd0 con 0x55ba43954010 2024-02-02T15:32:25.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.258+0000 7f2116fc2b80 10 client.0 _ll_register_callbacks cb 0x55ba43843290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:32:25.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.258+0000 7f2100a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba43954930 0x55ba43963c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:25.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.258+0000 7f20fbe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba43954010 0x55ba43963700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:25.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.258+0000 7f20fbe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba43954010 0x55ba43963700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:37092/0 (socket says 172.21.15.1:37092) 2024-02-02T15:32:25.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.258+0000 7f20fbe00700 1 -- 172.21.15.1:0/1883664813 learned_addr learned my addr 172.21.15.1:0/1883664813 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:32:25.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.258+0000 7f20fbe00700 1 --2- 172.21.15.1:0/1883664813 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba4395f7e0 0x55ba43964180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:25.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.259+0000 7f20fbe00700 1 -- 172.21.15.1:0/1883664813 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba4395f7e0 msgr2=0x55ba43964180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:32:25.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.259+0000 7f20fbe00700 1 --2- 172.21.15.1:0/1883664813 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba4395f7e0 0x55ba43964180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:25.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.259+0000 7f20fbe00700 1 -- 172.21.15.1:0/1883664813 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba43954930 msgr2=0x55ba43963c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:32:25.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.259+0000 7f20fbe00700 1 --2- 172.21.15.1:0/1883664813 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba43954930 0x55ba43963c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:25.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.259+0000 7f20fbe00700 1 -- 172.21.15.1:0/1883664813 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ba436d3b60 con 0x55ba43954010 2024-02-02T15:32:25.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.259+0000 7f2100a00700 1 --2- 172.21.15.1:0/1883664813 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba43954930 0x55ba43963c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:32:25.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.259+0000 7f20fbe00700 1 --2- 172.21.15.1:0/1883664813 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba43954010 0x55ba43963700 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f20e400a010 tx=0x7f20e4011440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c56bc17f352d46b4 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:32:25.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.259+0000 7f20ea000700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:32:25.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.259+0000 7f20ea000700 1 -- 172.21.15.1:0/1883664813 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e402c330 con 0x55ba43954010 2024-02-02T15:32:25.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.259+0000 7f20ea000700 1 -- 172.21.15.1:0/1883664813 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f20e4035060 con 0x55ba43954010 2024-02-02T15:32:25.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.259+0000 7f2116fc2b80 1 -- 172.21.15.1:0/1883664813 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f20e400fcf0 con 0x55ba43954010 2024-02-02T15:32:25.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.259+0000 7f20ea000700 1 -- 172.21.15.1:0/1883664813 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f20e40050d0 con 0x55ba43954010 2024-02-02T15:32:25.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.259+0000 7f2116fc2b80 10 client.27022 fetch_fsmap learned FSMap version 35 2024-02-02T15:32:25.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.259+0000 7f2116fc2b80 1 -- 172.21.15.1:0/1883664813 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55ba4372bcd0 con 0x55ba43954010 2024-02-02T15:32:25.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.260+0000 7f20ea000700 1 -- 172.21.15.1:0/1883664813 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f20e400fcf0 con 0x55ba43954010 2024-02-02T15:32:25.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.260+0000 7f20ea000700 1 -- 172.21.15.1:0/1883664813 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 35) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f20e401f7f0 con 0x55ba43954010 2024-02-02T15:32:25.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.260+0000 7f2116fc2b80 10 client.27022 fetch_fsmap finished waiting for FSMap version 35 2024-02-02T15:32:25.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.260+0000 7f2116fc2b80 10 client.27022 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:32:25.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.260+0000 7f2116fc2b80 20 client.27022 populate_metadata read hostname 'smithi001' 2024-02-02T15:32:25.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.260+0000 7f2116fc2b80 1 --2- 172.21.15.1:0/1883664813 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55ba4396f250 0x55ba439c3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:25.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.260+0000 7f2116fc2b80 4 client.27022 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2024-02-02T15:32:25.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.260+0000 7f2116fc2b80 1 -- 172.21.15.1:0/1883664813 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ba43864f50 con 0x55ba4396f250 2024-02-02T15:32:25.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.260+0000 7f2100a00700 1 --2- 172.21.15.1:0/1883664813 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55ba4396f250 0x55ba439c3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:25.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.261+0000 7f2100a00700 1 --2- 172.21.15.1:0/1883664813 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55ba4396f250 0x55ba439c3670 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ebd723f143b4f1b server_cookie=6ba54a0e32ae48b5 in_seq=0 out_seq=0 2024-02-02T15:32:25.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.261+0000 7f20ea000700 10 client.27022 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:32:25.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.262+0000 7f20ea000700 1 -- 172.21.15.1:0/1883664813 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55ba43864f50 con 0x55ba4396f250 2024-02-02T15:32:25.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.262+0000 7f20ea000700 10 client.27022 handle_command_reply: tid=0 2024-02-02T15:32:25.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.262+0000 7f2116fc2b80 1 client.27022 shutdown 2024-02-02T15:32:25.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.262+0000 7f2116fc2b80 1 -- 172.21.15.1:0/1883664813 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba43954010 msgr2=0x55ba43963700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:25.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.262+0000 7f2116fc2b80 1 --2- 172.21.15.1:0/1883664813 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba43954010 0x55ba43963700 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f20e400a010 tx=0x7f20e4011440 comp rx=0 tx=0).stop 2024-02-02T15:32:25.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.262+0000 7f2116fc2b80 1 -- 172.21.15.1:0/1883664813 shutdown_connections 2024-02-02T15:32:25.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.262+0000 7f2116fc2b80 1 --2- 172.21.15.1:0/1883664813 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55ba4396f250 0x55ba439c3670 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:25.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.262+0000 7f2116fc2b80 1 --2- 172.21.15.1:0/1883664813 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba4395f7e0 0x55ba43964180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:25.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.262+0000 7f2116fc2b80 1 --2- 172.21.15.1:0/1883664813 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba43954010 0x55ba43963700 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:25.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.262+0000 7f2116fc2b80 1 --2- 172.21.15.1:0/1883664813 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba43954930 0x55ba43963c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:25.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.262+0000 7f20ea000700 0 client.27022 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:32:25.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.262+0000 7f2116fc2b80 1 -- 172.21.15.1:0/1883664813 >> 172.21.15.1:0/1883664813 conn(0x55ba438b3db0 msgr2=0x55ba439577e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:32:25.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.262+0000 7f2116fc2b80 1 -- 172.21.15.1:0/1883664813 shutdown_connections 2024-02-02T15:32:25.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.262+0000 7f2116fc2b80 1 -- 172.21.15.1:0/1883664813 wait complete. 2024-02-02T15:32:25.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.263+0000 7f2116fc2b80 20 client.27022 trim_cache size 0 max 16384 2024-02-02T15:32:25.268 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:32:25.268 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2024-02-02T15:32:25.268 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2024-02-02T15:32:25.268 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:32:25.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.263+0000 7f2103400700 1 -- 172.21.15.1:0/2373962120 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f20dc0a0690 msgr2=0x7f20dc0a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:25.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.263+0000 7f2103400700 1 --2- 172.21.15.1:0/2373962120 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f20dc0a0690 0x7f20dc0a2b50 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f20f40c8df0 tx=0x7f20f4009630 comp rx=0 tx=0).stop 2024-02-02T15:32:25.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.263+0000 7f2103400700 1 -- 172.21.15.1:0/2373962120 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20fc106ed0 msgr2=0x7f20fc19db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:25.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.263+0000 7f2103400700 1 --2- 172.21.15.1:0/2373962120 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20fc106ed0 0x7f20fc19db70 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f20f400c370 tx=0x7f20f400c680 comp rx=0 tx=0).stop 2024-02-02T15:32:25.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.263+0000 7f2103400700 1 -- 172.21.15.1:0/2373962120 shutdown_connections 2024-02-02T15:32:25.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.263+0000 7f2103400700 1 --2- 172.21.15.1:0/2373962120 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f20dc0a0690 0x7f20dc0a2b50 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:25.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.263+0000 7f2103400700 1 --2- 172.21.15.1:0/2373962120 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20fc106ed0 0x7f20fc19db70 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:25.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.263+0000 7f2103400700 1 --2- 172.21.15.1:0/2373962120 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f20fc068010 0x7f20fc1991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:25.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.263+0000 7f2103400700 1 --2- 172.21.15.1:0/2373962120 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f20fc0689b0 0x7f20fc199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:25.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.263+0000 7f2103400700 1 -- 172.21.15.1:0/2373962120 >> 172.21.15.1:0/2373962120 conn(0x7f20fc0757d0 msgr2=0x7f20fc10bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:32:25.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.263+0000 7f2103400700 1 -- 172.21.15.1:0/2373962120 shutdown_connections 2024-02-02T15:32:25.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.264+0000 7f2103400700 1 -- 172.21.15.1:0/2373962120 wait complete. 2024-02-02T15:32:25.281 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-02T15:32:25.282 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T15:32:25.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.489+0000 7f8fd8a00700 1 -- 172.21.15.1:0/2199239999 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8fd0073170 msgr2=0x7f8fd0073550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:25.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.489+0000 7f8fd8a00700 1 --2- 172.21.15.1:0/2199239999 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8fd0073170 0x7f8fd0073550 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0009990 tx=0x7f8fc0009ca0 comp rx=0 tx=0).stop 2024-02-02T15:32:25.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.489+0000 7f8fd8a00700 1 -- 172.21.15.1:0/2199239999 shutdown_connections 2024-02-02T15:32:25.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.489+0000 7f8fd8a00700 1 --2- 172.21.15.1:0/2199239999 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8fd010d4e0 0x7f8fd010f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:25.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.489+0000 7f8fd8a00700 1 --2- 172.21.15.1:0/2199239999 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8fd0073170 0x7f8fd0073550 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:25.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.489+0000 7f8fd8a00700 1 --2- 172.21.15.1:0/2199239999 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8fd0073a90 0x7f8fd010cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:25.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.489+0000 7f8fd8a00700 1 -- 172.21.15.1:0/2199239999 >> 172.21.15.1:0/2199239999 conn(0x7f8fd00fc790 msgr2=0x7f8fd00febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:32:25.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.489+0000 7f8fd8a00700 1 -- 172.21.15.1:0/2199239999 shutdown_connections 2024-02-02T15:32:25.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.489+0000 7f8fd8a00700 1 -- 172.21.15.1:0/2199239999 wait complete. 2024-02-02T15:32:25.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.490+0000 7f8fd8a00700 1 Processor -- start 2024-02-02T15:32:25.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.490+0000 7f8fd8a00700 1 -- start start 2024-02-02T15:32:25.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.490+0000 7f8fd8a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8fd0073170 0x7f8fd0199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:25.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.490+0000 7f8fd8a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8fd0073a90 0x7f8fd0199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:25.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.490+0000 7f8fd8a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8fd010d4e0 0x7f8fd019dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:25.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.490+0000 7f8fd8a00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd019e1e0 con 0x7f8fd0073a90 2024-02-02T15:32:25.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.490+0000 7f8fd8a00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd019e350 con 0x7f8fd0073170 2024-02-02T15:32:25.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.490+0000 7f8fd8a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd019e510 con 0x7f8fd010d4e0 2024-02-02T15:32:25.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.491+0000 7f8fd6000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8fd0073170 0x7f8fd0199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:25.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.491+0000 7f8fd5600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8fd0073a90 0x7f8fd0199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:25.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.491+0000 7f8fd6000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8fd0073170 0x7f8fd0199200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:47728/0 (socket says 172.21.15.1:47728) 2024-02-02T15:32:25.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.491+0000 7f8fd6a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8fd010d4e0 0x7f8fd019dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:25.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.491+0000 7f8fd6000700 1 -- 172.21.15.1:0/3638953390 learned_addr learned my addr 172.21.15.1:0/3638953390 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:32:25.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.491+0000 7f8fd6000700 1 -- 172.21.15.1:0/3638953390 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8fd010d4e0 msgr2=0x7f8fd019dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:25.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.491+0000 7f8fd6000700 1 --2- 172.21.15.1:0/3638953390 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8fd010d4e0 0x7f8fd019dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:25.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.491+0000 7f8fd6000700 1 -- 172.21.15.1:0/3638953390 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8fd0073a90 msgr2=0x7f8fd0199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:25.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.491+0000 7f8fd6000700 1 --2- 172.21.15.1:0/3638953390 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8fd0073a90 0x7f8fd0199740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:25.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.491+0000 7f8fd6000700 1 -- 172.21.15.1:0/3638953390 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fc00095f0 con 0x7f8fd0073170 2024-02-02T15:32:25.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.491+0000 7f8fd6000700 1 --2- 172.21.15.1:0/3638953390 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8fd0073170 0x7f8fd0199200 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0005910 tx=0x7f8fc0005720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:32:25.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.492+0000 7f8fcea00700 1 -- 172.21.15.1:0/3638953390 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc001d070 con 0x7f8fd0073170 2024-02-02T15:32:25.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.492+0000 7f8fcea00700 1 -- 172.21.15.1:0/3638953390 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8fc0022470 con 0x7f8fd0073170 2024-02-02T15:32:25.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.492+0000 7f8fcea00700 1 -- 172.21.15.1:0/3638953390 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc0017650 con 0x7f8fd0073170 2024-02-02T15:32:25.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.492+0000 7f8fd8a00700 1 -- 172.21.15.1:0/3638953390 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fd019e790 con 0x7f8fd0073170 2024-02-02T15:32:25.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.492+0000 7f8fd8a00700 1 -- 172.21.15.1:0/3638953390 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fd01a6030 con 0x7f8fd0073170 2024-02-02T15:32:25.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.493+0000 7f8fd8a00700 1 -- 172.21.15.1:0/3638953390 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fd01086f0 con 0x7f8fd0073170 2024-02-02T15:32:25.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.495+0000 7f8fcea00700 1 -- 172.21.15.1:0/3638953390 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 27) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f8fc0053020 con 0x7f8fd0073170 2024-02-02T15:32:25.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.496+0000 7f8fcea00700 1 --2- 172.21.15.1:0/3638953390 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f8fb00a0710 0x7f8fb00a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:25.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.496+0000 7f8fcea00700 1 -- 172.21.15.1:0/3638953390 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f8fc00cd7f0 con 0x7f8fd0073170 2024-02-02T15:32:25.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.496+0000 7f8fd5600700 1 --2- 172.21.15.1:0/3638953390 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f8fb00a0710 0x7f8fb00a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:25.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.497+0000 7f8fd5600700 1 --2- 172.21.15.1:0/3638953390 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f8fb00a0710 0x7f8fb00a2bd0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8fd019a810 tx=0x7f8fb8009260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:32:25.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.499+0000 7f8fcea00700 1 -- 172.21.15.1:0/3638953390 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f8fc0096d30 con 0x7f8fd0073170 2024-02-02T15:32:25.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:32:25 smithi071 ceph-mon[44707]: pgmap v165: 129 pgs: 129 active+clean; 206 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.7 MiB/s wr, 227 op/s 2024-02-02T15:32:25.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:32:25 smithi142 ceph-mon[45728]: pgmap v165: 129 pgs: 129 active+clean; 206 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.7 MiB/s wr, 227 op/s 2024-02-02T15:32:25.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:32:25 smithi001 ceph-mon[35470]: pgmap v165: 129 pgs: 129 active+clean; 206 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.7 MiB/s wr, 227 op/s 2024-02-02T15:32:25.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.766+0000 7f8fd8a00700 1 -- 172.21.15.1:0/3638953390 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8fd00fe9b0 con 0x7f8fd0073170 2024-02-02T15:32:25.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.767+0000 7f8fcea00700 1 -- 172.21.15.1:0/3638953390 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+10866 (secure 0 0 0) 0x7f8fc00225e0 con 0x7f8fd0073170 2024-02-02T15:32:25.767 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:32:25.767 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2922240448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2922240448},{"type":"v1","addr":"172.21.15.1:6835","nonce":2922240448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2729497431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2729497431},{"type":"v1","addr":"172.21.15.1:6837","nonce":2729497431}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-02T15:24:47.761435+0000","modified":"2024-02-02T15:31:27.413833+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24529,"mds_2":24518,"mds_3":24514,"mds_4":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14517":{"gid":14517,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/1218303683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":1218303683},{"type":"v1","addr":"172.21.15.1:6839","nonce":1218303683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14529":{"gid":14529,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/2059197496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2059197496},{"type":"v1","addr":"172.21.15.1:6841","nonce":2059197496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/7648986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":7648986},{"type":"v1","addr":"172.21.15.142:6833","nonce":7648986}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/3482746094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3482746094},{"type":"v1","addr":"172.21.15.71:6833","nonce":3482746094}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/388207814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":388207814},{"type":"v1","addr":"172.21.15.71:6835","nonce":388207814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6835/1052606547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1052606547},{"type":"v1","addr":"172.21.15.142:6835","nonce":1052606547}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.142:6837/2870289019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2870289019},{"type":"v1","addr":"172.21.15.142:6837","nonce":2870289019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.71:6837/628016199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":628016199},{"type":"v1","addr":"172.21.15.71:6837","nonce":628016199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.142:6839/321675421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":321675421},{"type":"v1","addr":"172.21.15.142:6839","nonce":321675421}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/1268086493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1268086493},{"type":"v1","addr":"172.21.15.71:6839","nonce":1268086493}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-02T15:32:25.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.769+0000 7f8fd8a00700 1 -- 172.21.15.1:0/3638953390 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f8fb00a0710 msgr2=0x7f8fb00a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:25.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.769+0000 7f8fd8a00700 1 --2- 172.21.15.1:0/3638953390 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f8fb00a0710 0x7f8fb00a2bd0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8fd019a810 tx=0x7f8fb8009260 comp rx=0 tx=0).stop 2024-02-02T15:32:25.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.769+0000 7f8fd8a00700 1 -- 172.21.15.1:0/3638953390 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8fd0073170 msgr2=0x7f8fd0199200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:25.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.769+0000 7f8fd8a00700 1 --2- 172.21.15.1:0/3638953390 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8fd0073170 0x7f8fd0199200 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0005910 tx=0x7f8fc0005720 comp rx=0 tx=0).stop 2024-02-02T15:32:25.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.770+0000 7f8fd8a00700 1 -- 172.21.15.1:0/3638953390 shutdown_connections 2024-02-02T15:32:25.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.770+0000 7f8fd8a00700 1 --2- 172.21.15.1:0/3638953390 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f8fb00a0710 0x7f8fb00a2bd0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:25.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.770+0000 7f8fd8a00700 1 --2- 172.21.15.1:0/3638953390 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8fd010d4e0 0x7f8fd019dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:25.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.770+0000 7f8fd8a00700 1 --2- 172.21.15.1:0/3638953390 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8fd0073170 0x7f8fd0199200 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:25.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.770+0000 7f8fd8a00700 1 --2- 172.21.15.1:0/3638953390 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8fd0073a90 0x7f8fd0199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:25.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.770+0000 7f8fd8a00700 1 -- 172.21.15.1:0/3638953390 >> 172.21.15.1:0/3638953390 conn(0x7f8fd00fc790 msgr2=0x7f8fd010f720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:32:25.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.770+0000 7f8fd8a00700 1 -- 172.21.15.1:0/3638953390 shutdown_connections 2024-02-02T15:32:25.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.770+0000 7f8fd8a00700 1 -- 172.21.15.1:0/3638953390 wait complete. 2024-02-02T15:32:25.771 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 35 2024-02-02T15:32:25.789 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-02-02T15:32:25.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.994+0000 7fa510200700 1 -- 172.21.15.1:0/4135283890 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5081019a0 msgr2=0x7fa5081059e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:25.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.994+0000 7fa510200700 1 --2- 172.21.15.1:0/4135283890 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5081019a0 0x7fa5081059e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa4f00098f0 tx=0x7fa4f0009c00 comp rx=0 tx=0).stop 2024-02-02T15:32:25.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.994+0000 7fa510200700 1 -- 172.21.15.1:0/4135283890 shutdown_connections 2024-02-02T15:32:25.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.994+0000 7fa510200700 1 --2- 172.21.15.1:0/4135283890 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa508106110 0x7fa50810dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:25.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.994+0000 7fa510200700 1 --2- 172.21.15.1:0/4135283890 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa508100ff0 0x7fa5081013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:25.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.994+0000 7fa510200700 1 --2- 172.21.15.1:0/4135283890 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5081019a0 0x7fa5081059e0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:25.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.994+0000 7fa510200700 1 -- 172.21.15.1:0/4135283890 >> 172.21.15.1:0/4135283890 conn(0x7fa5080fc810 msgr2=0x7fa5080fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:32:25.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.994+0000 7fa510200700 1 -- 172.21.15.1:0/4135283890 shutdown_connections 2024-02-02T15:32:25.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.994+0000 7fa510200700 1 -- 172.21.15.1:0/4135283890 wait complete. 2024-02-02T15:32:25.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.996+0000 7fa510200700 1 Processor -- start 2024-02-02T15:32:25.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.996+0000 7fa510200700 1 -- start start 2024-02-02T15:32:25.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.996+0000 7fa510200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa508100ff0 0x7fa508196ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:25.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.996+0000 7fa510200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa508106110 0x7fa508197400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:25.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.997+0000 7fa510200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa50819b860 0x7fa50819bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:25.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.997+0000 7fa510200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa50819c2e0 con 0x7fa508100ff0 2024-02-02T15:32:25.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.997+0000 7fa510200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa50819c450 con 0x7fa50819b860 2024-02-02T15:32:25.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.997+0000 7fa510200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa50819c610 con 0x7fa508106110 2024-02-02T15:32:25.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.997+0000 7fa50d800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa508100ff0 0x7fa508196ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:25.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.997+0000 7fa50e200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa50819b860 0x7fa50819bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:25.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.997+0000 7fa50ce00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa508106110 0x7fa508197400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:25.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.997+0000 7fa50e200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa50819b860 0x7fa50819bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:47748/0 (socket says 172.21.15.1:47748) 2024-02-02T15:32:25.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.997+0000 7fa50d800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa508100ff0 0x7fa508196ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39154/0 (socket says 172.21.15.1:39154) 2024-02-02T15:32:25.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.997+0000 7fa50e200700 1 -- 172.21.15.1:0/56989430 learned_addr learned my addr 172.21.15.1:0/56989430 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:32:25.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.997+0000 7fa50e200700 1 -- 172.21.15.1:0/56989430 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa508106110 msgr2=0x7fa508197400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:25.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.997+0000 7fa50e200700 1 --2- 172.21.15.1:0/56989430 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa508106110 0x7fa508197400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:25.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.997+0000 7fa50e200700 1 -- 172.21.15.1:0/56989430 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa508100ff0 msgr2=0x7fa508196ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:26.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.997+0000 7fa50e200700 1 --2- 172.21.15.1:0/56989430 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa508100ff0 0x7fa508196ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.997+0000 7fa50e200700 1 -- 172.21.15.1:0/56989430 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4f8009520 con 0x7fa50819b860 2024-02-02T15:32:26.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.997+0000 7fa50d800700 1 --2- 172.21.15.1:0/56989430 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa508100ff0 0x7fa508196ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:32:26.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.998+0000 7fa50e200700 1 --2- 172.21.15.1:0/56989430 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa50819b860 0x7fa50819bce0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fa50000c110 tx=0x7fa50000c420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:32:26.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.998+0000 7fa506000700 1 -- 172.21.15.1:0/56989430 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa500017070 con 0x7fa50819b860 2024-02-02T15:32:26.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.998+0000 7fa506000700 1 -- 172.21.15.1:0/56989430 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa50000e050 con 0x7fa50819b860 2024-02-02T15:32:26.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.998+0000 7fa506000700 1 -- 172.21.15.1:0/56989430 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5000078f0 con 0x7fa50819b860 2024-02-02T15:32:26.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.998+0000 7fa510200700 1 -- 172.21.15.1:0/56989430 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4f0009520 con 0x7fa50819b860 2024-02-02T15:32:26.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:25.998+0000 7fa510200700 1 -- 172.21.15.1:0/56989430 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5081a3e60 con 0x7fa50819b860 2024-02-02T15:32:26.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.001+0000 7fa506000700 1 -- 172.21.15.1:0/56989430 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 27) v1 ==== 135200+0+0 (secure 0 0 0) 0x7fa500007a50 con 0x7fa50819b860 2024-02-02T15:32:26.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.002+0000 7fa523ccfb80 1 Processor -- start 2024-02-02T15:32:26.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.002+0000 7fa523ccfb80 1 -- start start 2024-02-02T15:32:26.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.002+0000 7fa506000700 1 --2- 172.21.15.1:0/56989430 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fa4e80a0710 0x7fa4e80a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:26.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.002+0000 7fa523ccfb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560dc20c9010 0x560dc20c93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:26.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.002+0000 7fa523ccfb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560dc20c9930 0x560dc20d42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:26.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.002+0000 7fa523ccfb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560dc20d47e0 0x560dc20d6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:26.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.003+0000 7fa523ccfb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dc203d4c0 con 0x560dc20c9010 2024-02-02T15:32:26.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.003+0000 7fa523ccfb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dc2021aa0 con 0x560dc20d47e0 2024-02-02T15:32:26.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.003+0000 7fa523ccfb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dc201f460 con 0x560dc20c9930 2024-02-02T15:32:26.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.003+0000 7fa506000700 1 -- 172.21.15.1:0/56989430 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fa5000cba40 con 0x7fa50819b860 2024-02-02T15:32:26.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.003+0000 7fa50d800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560dc20c9010 0x560dc20c93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:26.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.003+0000 7fa50ce00700 1 --2- 172.21.15.1:0/56989430 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fa4e80a0710 0x7fa4e80a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:26.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.003+0000 7fa50d800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560dc20c9010 0x560dc20c93f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39164/0 (socket says 172.21.15.1:39164) 2024-02-02T15:32:26.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.003+0000 7fa50d800700 1 -- 172.21.15.1:0/3592949305 learned_addr learned my addr 172.21.15.1:0/3592949305 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:32:26.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.003+0000 7fa50e200700 1 --2- 172.21.15.1:0/3592949305 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560dc20d47e0 0x560dc20d6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:26.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.003+0000 7fa50ce00700 1 --2- 172.21.15.1:0/3592949305 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560dc20c9930 0x560dc20d42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:26.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.003+0000 7fa50d800700 1 -- 172.21.15.1:0/3592949305 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560dc20c9930 msgr2=0x560dc20d42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:32:26.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.003+0000 7fa50d800700 1 --2- 172.21.15.1:0/3592949305 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560dc20c9930 0x560dc20d42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.003+0000 7fa50d800700 1 -- 172.21.15.1:0/3592949305 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560dc20d47e0 msgr2=0x560dc20d6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:32:26.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.003+0000 7fa50d800700 1 --2- 172.21.15.1:0/3592949305 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560dc20d47e0 0x560dc20d6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.003+0000 7fa50d800700 1 -- 172.21.15.1:0/3592949305 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560dc1e9c3b0 con 0x560dc20c9010 2024-02-02T15:32:26.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.003+0000 7fa50e200700 1 --2- 172.21.15.1:0/3592949305 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560dc20d47e0 0x560dc20d6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:32:26.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.004+0000 7fa50d800700 1 --2- 172.21.15.1:0/3592949305 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560dc20c9010 0x560dc20c93f0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fa4f8011760 tx=0x7fa4f8011a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc34a8aa598acf07 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:32:26.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.004+0000 7fa50ce00700 1 --2- 172.21.15.1:0/56989430 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fa4e80a0710 0x7fa4e80a2bd0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa4f0009fd0 tx=0x7fa4f0019040 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:32:26.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.004+0000 7fa4fea00700 1 -- 172.21.15.1:0/3592949305 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f800f4e0 con 0x560dc20c9010 2024-02-02T15:32:26.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.004+0000 7fa4fea00700 1 -- 172.21.15.1:0/3592949305 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa4f800fb00 con 0x560dc20c9010 2024-02-02T15:32:26.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.004+0000 7fa4fea00700 1 -- 172.21.15.1:0/3592949305 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f8019b80 con 0x560dc20c9010 2024-02-02T15:32:26.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.005+0000 7fa523ccfb80 1 -- 172.21.15.1:0/3592949305 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560dc20c9010 msgr2=0x560dc20c93f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:26.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.005+0000 7fa523ccfb80 1 --2- 172.21.15.1:0/3592949305 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560dc20c9010 0x560dc20c93f0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fa4f8011760 tx=0x7fa4f8011a70 comp rx=0 tx=0).stop 2024-02-02T15:32:26.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.005+0000 7fa523ccfb80 1 -- 172.21.15.1:0/3592949305 shutdown_connections 2024-02-02T15:32:26.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.005+0000 7fa523ccfb80 1 --2- 172.21.15.1:0/3592949305 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560dc20d47e0 0x560dc20d6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.005+0000 7fa523ccfb80 1 --2- 172.21.15.1:0/3592949305 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560dc20c9010 0x560dc20c93f0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.005+0000 7fa523ccfb80 1 --2- 172.21.15.1:0/3592949305 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560dc20c9930 0x560dc20d42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.005+0000 7fa523ccfb80 1 -- 172.21.15.1:0/3592949305 >> 172.21.15.1:0/3592949305 conn(0x560dc2028db0 msgr2=0x560dc20c53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:32:26.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.005+0000 7fa523ccfb80 1 -- 172.21.15.1:0/3592949305 shutdown_connections 2024-02-02T15:32:26.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.005+0000 7fa523ccfb80 1 -- 172.21.15.1:0/3592949305 wait complete. 2024-02-02T15:32:26.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.005+0000 7fa523ccfb80 1 -- start start 2024-02-02T15:32:26.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.006+0000 7fa523ccfb80 1 Processor -- start 2024-02-02T15:32:26.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.006+0000 7fa523ccfb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560dc20c9010 0x560dc20d41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:26.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.006+0000 7fa523ccfb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560dc20c9930 0x560dc20d1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:26.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.006+0000 7fa523ccfb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560dc20d47e0 0x560dc20d1a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:26.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.006+0000 7fa523ccfb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dc201ad50 con 0x560dc20c9930 2024-02-02T15:32:26.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.006+0000 7fa523ccfb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dc1fb6d80 con 0x560dc20c9010 2024-02-02T15:32:26.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.006+0000 7fa523ccfb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dc1fb6a20 con 0x560dc20d47e0 2024-02-02T15:32:26.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.006+0000 7fa50d800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560dc20c9010 0x560dc20d41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:26.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.006+0000 7fa523ccfb80 10 client.0 _ll_register_callbacks cb 0x560dc1fb8290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:32:26.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.006+0000 7fa50d800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560dc20c9010 0x560dc20d41f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:47770/0 (socket says 172.21.15.1:47770) 2024-02-02T15:32:26.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.006+0000 7fa50d800700 1 -- 172.21.15.1:0/4082078386 learned_addr learned my addr 172.21.15.1:0/4082078386 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:32:26.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.006+0000 7fa50ce00700 1 --2- 172.21.15.1:0/4082078386 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560dc20c9930 0x560dc20d1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:26.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.007+0000 7fa50e200700 1 --2- 172.21.15.1:0/4082078386 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560dc20d47e0 0x560dc20d1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:26.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.007+0000 7fa50d800700 1 -- 172.21.15.1:0/4082078386 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560dc20d47e0 msgr2=0x560dc20d1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:32:26.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.007+0000 7fa50d800700 1 --2- 172.21.15.1:0/4082078386 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560dc20d47e0 0x560dc20d1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.007+0000 7fa50d800700 1 -- 172.21.15.1:0/4082078386 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560dc20c9930 msgr2=0x560dc20d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:32:26.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.007+0000 7fa50d800700 1 --2- 172.21.15.1:0/4082078386 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560dc20c9930 0x560dc20d1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.007+0000 7fa50d800700 1 -- 172.21.15.1:0/4082078386 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5081a3e60 con 0x560dc20c9010 2024-02-02T15:32:26.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.007+0000 7fa50d800700 1 --2- 172.21.15.1:0/4082078386 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560dc20c9010 0x560dc20d41f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fa4f8000c00 tx=0x7fa4f800f710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d90a25d96e50ac7d server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:32:26.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.007+0000 7fa4f7400700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:32:26.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.007+0000 7fa4f7400700 1 -- 172.21.15.1:0/4082078386 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f800f900 con 0x560dc20c9010 2024-02-02T15:32:26.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.007+0000 7fa4f7400700 1 -- 172.21.15.1:0/4082078386 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fa4f8004250 con 0x560dc20c9010 2024-02-02T15:32:26.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.007+0000 7fa523ccfb80 1 -- 172.21.15.1:0/4082078386 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560dc20bd0f0 con 0x560dc20c9010 2024-02-02T15:32:26.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.008+0000 7fa4f7400700 1 -- 172.21.15.1:0/4082078386 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa4f8004410 con 0x560dc20c9010 2024-02-02T15:32:26.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.008+0000 7fa523ccfb80 10 client.27041 fetch_fsmap learned FSMap version 35 2024-02-02T15:32:26.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.008+0000 7fa523ccfb80 1 -- 172.21.15.1:0/4082078386 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x7fa4f8011440 con 0x560dc20c9010 2024-02-02T15:32:26.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.008+0000 7fa4f7400700 1 -- 172.21.15.1:0/4082078386 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560dc20bd0f0 con 0x560dc20c9010 2024-02-02T15:32:26.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.008+0000 7fa4f7400700 1 -- 172.21.15.1:0/4082078386 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 35) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa4f8003920 con 0x560dc20c9010 2024-02-02T15:32:26.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.008+0000 7fa523ccfb80 10 client.27041 fetch_fsmap finished waiting for FSMap version 35 2024-02-02T15:32:26.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.008+0000 7fa523ccfb80 10 client.27041 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-02-02T15:32:26.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.008+0000 7fa523ccfb80 20 client.27041 populate_metadata read hostname 'smithi001' 2024-02-02T15:32:26.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.008+0000 7fa523ccfb80 1 --2- 172.21.15.1:0/4082078386 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x560dc20e40b0 0x560dc20e64d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:26.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.008+0000 7fa523ccfb80 4 client.27041 mds_command: new command op to 24502 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:32:26.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.008+0000 7fa523ccfb80 1 -- 172.21.15.1:0/4082078386 --> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560dc2013dd0 con 0x560dc20e40b0 2024-02-02T15:32:26.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.009+0000 7fa50ce00700 1 --2- 172.21.15.1:0/4082078386 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x560dc20e40b0 0x560dc20e64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:26.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.009+0000 7fa50ce00700 1 --2- 172.21.15.1:0/4082078386 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x560dc20e40b0 0x560dc20e64d0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b78bc0e5921dfc4d server_cookie=e4c5f0b5eb6831c3 in_seq=0 out_seq=0 2024-02-02T15:32:26.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.009+0000 7fa4f7400700 10 client.27041 ms_handle_connect on v2:172.21.15.142:6834/1052606547 2024-02-02T15:32:26.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.010+0000 7fa4f7400700 1 -- 172.21.15.1:0/4082078386 <== mds.4 v2:172.21.15.142:6834/1052606547 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x560dc2013dd0 con 0x560dc20e40b0 2024-02-02T15:32:26.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.010+0000 7fa4f7400700 10 client.27041 handle_command_reply: tid=0 2024-02-02T15:32:26.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.010+0000 7fa523ccfb80 1 client.27041 shutdown 2024-02-02T15:32:26.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.010+0000 7fa523ccfb80 1 -- 172.21.15.1:0/4082078386 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560dc20c9010 msgr2=0x560dc20d41f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:26.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.010+0000 7fa523ccfb80 1 --2- 172.21.15.1:0/4082078386 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560dc20c9010 0x560dc20d41f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fa4f8000c00 tx=0x7fa4f800f710 comp rx=0 tx=0).stop 2024-02-02T15:32:26.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.010+0000 7fa523ccfb80 1 -- 172.21.15.1:0/4082078386 shutdown_connections 2024-02-02T15:32:26.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.010+0000 7fa523ccfb80 1 --2- 172.21.15.1:0/4082078386 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x560dc20e40b0 0x560dc20e64d0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.010+0000 7fa523ccfb80 1 --2- 172.21.15.1:0/4082078386 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560dc20d47e0 0x560dc20d1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.010+0000 7fa523ccfb80 1 --2- 172.21.15.1:0/4082078386 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560dc20c9010 0x560dc20d41f0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.010+0000 7fa4f7400700 0 client.27041 ms_handle_reset on v2:172.21.15.142:6834/1052606547 2024-02-02T15:32:26.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.010+0000 7fa523ccfb80 1 --2- 172.21.15.1:0/4082078386 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560dc20c9930 0x560dc20d1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.010+0000 7fa523ccfb80 1 -- 172.21.15.1:0/4082078386 >> 172.21.15.1:0/4082078386 conn(0x560dc2028db0 msgr2=0x560dc20c4fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:32:26.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.010+0000 7fa523ccfb80 1 -- 172.21.15.1:0/4082078386 shutdown_connections 2024-02-02T15:32:26.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.010+0000 7fa523ccfb80 1 -- 172.21.15.1:0/4082078386 wait complete. 2024-02-02T15:32:26.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.011+0000 7fa523ccfb80 20 client.27041 trim_cache size 0 max 16384 2024-02-02T15:32:26.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.024+0000 7fa523ccfb80 1 Processor -- start 2024-02-02T15:32:26.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.024+0000 7fa523ccfb80 1 -- start start 2024-02-02T15:32:26.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.025+0000 7fa523ccfb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560dc20c9010 0x560dc20d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:26.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.025+0000 7fa523ccfb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560dc20c9930 0x560dc20d3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:26.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.025+0000 7fa523ccfb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560dc20d47e0 0x560dc20cc700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:26.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.025+0000 7fa523ccfb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dc1e9e6a0 con 0x560dc20d47e0 2024-02-02T15:32:26.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.025+0000 7fa523ccfb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dc20d6d60 con 0x560dc20c9010 2024-02-02T15:32:26.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.025+0000 7fa523ccfb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dc20e0010 con 0x560dc20c9930 2024-02-02T15:32:26.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.025+0000 7fa50e200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560dc20d47e0 0x560dc20cc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:26.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.025+0000 7fa50d800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560dc20c9010 0x560dc20d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:26.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.025+0000 7fa50e200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560dc20d47e0 0x560dc20cc700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39190/0 (socket says 172.21.15.1:39190) 2024-02-02T15:32:26.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.025+0000 7fa50ce00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560dc20c9930 0x560dc20d3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:26.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.025+0000 7fa50e200700 1 -- 172.21.15.1:0/2268794839 learned_addr learned my addr 172.21.15.1:0/2268794839 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:32:26.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.025+0000 7fa50e200700 1 -- 172.21.15.1:0/2268794839 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560dc20c9930 msgr2=0x560dc20d3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:32:26.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.025+0000 7fa50e200700 1 --2- 172.21.15.1:0/2268794839 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560dc20c9930 0x560dc20d3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.025+0000 7fa50e200700 1 -- 172.21.15.1:0/2268794839 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560dc20c9010 msgr2=0x560dc20d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:32:26.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.025+0000 7fa50e200700 1 --2- 172.21.15.1:0/2268794839 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560dc20c9010 0x560dc20d19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.025+0000 7fa50e200700 1 -- 172.21.15.1:0/2268794839 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560dc1e48b60 con 0x560dc20d47e0 2024-02-02T15:32:26.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.025+0000 7fa50ce00700 1 --2- 172.21.15.1:0/2268794839 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560dc20c9930 0x560dc20d3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:32:26.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.025+0000 7fa50d800700 1 --2- 172.21.15.1:0/2268794839 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560dc20c9010 0x560dc20d19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:32:26.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.026+0000 7fa50e200700 1 --2- 172.21.15.1:0/2268794839 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560dc20d47e0 0x560dc20cc700 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fa50000c770 tx=0x7fa500094f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dce510271197b83 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:32:26.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.026+0000 7fa4ff400700 1 -- 172.21.15.1:0/2268794839 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa500098bb0 con 0x560dc20d47e0 2024-02-02T15:32:26.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.026+0000 7fa4ff400700 1 -- 172.21.15.1:0/2268794839 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa50000f040 con 0x560dc20d47e0 2024-02-02T15:32:26.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.026+0000 7fa4ff400700 1 -- 172.21.15.1:0/2268794839 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5000c9060 con 0x560dc20d47e0 2024-02-02T15:32:26.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.026+0000 7fa523ccfb80 1 -- 172.21.15.1:0/2268794839 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560dc20d47e0 msgr2=0x560dc20cc700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:26.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.026+0000 7fa523ccfb80 1 --2- 172.21.15.1:0/2268794839 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560dc20d47e0 0x560dc20cc700 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fa50000c770 tx=0x7fa500094f70 comp rx=0 tx=0).stop 2024-02-02T15:32:26.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.026+0000 7fa523ccfb80 1 -- 172.21.15.1:0/2268794839 shutdown_connections 2024-02-02T15:32:26.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.026+0000 7fa523ccfb80 1 --2- 172.21.15.1:0/2268794839 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560dc20d47e0 0x560dc20cc700 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.026+0000 7fa523ccfb80 1 --2- 172.21.15.1:0/2268794839 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560dc20c9010 0x560dc20d19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.026+0000 7fa523ccfb80 1 --2- 172.21.15.1:0/2268794839 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560dc20c9930 0x560dc20d3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.026+0000 7fa523ccfb80 1 -- 172.21.15.1:0/2268794839 >> 172.21.15.1:0/2268794839 conn(0x560dc2028db0 msgr2=0x560dc20df900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:32:26.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.027+0000 7fa523ccfb80 1 -- 172.21.15.1:0/2268794839 shutdown_connections 2024-02-02T15:32:26.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.027+0000 7fa523ccfb80 1 -- 172.21.15.1:0/2268794839 wait complete. 2024-02-02T15:32:26.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.027+0000 7fa523ccfb80 1 -- start start 2024-02-02T15:32:26.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.027+0000 7fa523ccfb80 1 Processor -- start 2024-02-02T15:32:26.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.028+0000 7fa523ccfb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560dc20c9010 0x560dc20d9310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:26.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.028+0000 7fa523ccfb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560dc20c9930 0x560dc20d9850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:26.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.028+0000 7fa523ccfb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560dc20d47e0 0x560dc20d9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:26.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.028+0000 7fa523ccfb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dc2134cb0 con 0x560dc20d47e0 2024-02-02T15:32:26.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.028+0000 7fa523ccfb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dc2134e20 con 0x560dc20c9010 2024-02-02T15:32:26.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.028+0000 7fa523ccfb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560dc2134fe0 con 0x560dc20c9930 2024-02-02T15:32:26.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.028+0000 7fa523ccfb80 10 client.0 _ll_register_callbacks cb 0x560dc1fb8290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:32:26.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.028+0000 7fa50d800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560dc20c9010 0x560dc20d9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:26.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.028+0000 7fa50ce00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560dc20c9930 0x560dc20d9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:26.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.028+0000 7fa50d800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560dc20c9010 0x560dc20d9310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:47792/0 (socket says 172.21.15.1:47792) 2024-02-02T15:32:26.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.028+0000 7fa50e200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560dc20d47e0 0x560dc20d9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:26.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.028+0000 7fa50d800700 1 -- 172.21.15.1:0/107819942 learned_addr learned my addr 172.21.15.1:0/107819942 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:32:26.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.028+0000 7fa50d800700 1 -- 172.21.15.1:0/107819942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560dc20c9930 msgr2=0x560dc20d9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:32:26.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.028+0000 7fa50d800700 1 --2- 172.21.15.1:0/107819942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560dc20c9930 0x560dc20d9850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.028+0000 7fa50d800700 1 -- 172.21.15.1:0/107819942 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560dc20d47e0 msgr2=0x560dc20d9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:32:26.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.028+0000 7fa50d800700 1 --2- 172.21.15.1:0/107819942 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560dc20d47e0 0x560dc20d9d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.028+0000 7fa50d800700 1 -- 172.21.15.1:0/107819942 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4f8011440 con 0x560dc20c9010 2024-02-02T15:32:26.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.028+0000 7fa50ce00700 1 --2- 172.21.15.1:0/107819942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560dc20c9930 0x560dc20d9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:32:26.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.029+0000 7fa50d800700 1 --2- 172.21.15.1:0/107819942 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560dc20c9010 0x560dc20d9310 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fa4f801c040 tx=0x7fa4f800f9b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af3e6d901a1f1c92 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:32:26.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.029+0000 7fa4f7400700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:32:26.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.029+0000 7fa4f7400700 1 -- 172.21.15.1:0/107819942 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f8003680 con 0x560dc20c9010 2024-02-02T15:32:26.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.029+0000 7fa4f7400700 1 -- 172.21.15.1:0/107819942 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fa4f80037e0 con 0x560dc20c9010 2024-02-02T15:32:26.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.029+0000 7fa523ccfb80 1 -- 172.21.15.1:0/107819942 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa4f800fb00 con 0x560dc20c9010 2024-02-02T15:32:26.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.029+0000 7fa4f7400700 1 -- 172.21.15.1:0/107819942 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa4f80039a0 con 0x560dc20c9010 2024-02-02T15:32:26.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.029+0000 7fa523ccfb80 10 client.27047 fetch_fsmap learned FSMap version 35 2024-02-02T15:32:26.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.029+0000 7fa523ccfb80 1 -- 172.21.15.1:0/107819942 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x7fa4f001f040 con 0x560dc20c9010 2024-02-02T15:32:26.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.029+0000 7fa4f7400700 1 -- 172.21.15.1:0/107819942 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa4f800fb00 con 0x560dc20c9010 2024-02-02T15:32:26.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.030+0000 7fa4f7400700 1 -- 172.21.15.1:0/107819942 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 35) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa4f80194d0 con 0x560dc20c9010 2024-02-02T15:32:26.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.030+0000 7fa523ccfb80 10 client.27047 fetch_fsmap finished waiting for FSMap version 35 2024-02-02T15:32:26.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.030+0000 7fa523ccfb80 10 client.27047 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-02-02T15:32:26.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.030+0000 7fa523ccfb80 20 client.27047 populate_metadata read hostname 'smithi001' 2024-02-02T15:32:26.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.030+0000 7fa523ccfb80 1 --2- 172.21.15.1:0/107819942 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x560dc21363f0 0x560dc21387e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:26.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.030+0000 7fa523ccfb80 4 client.27047 mds_command: new command op to 24502 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:32:26.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.030+0000 7fa523ccfb80 1 -- 172.21.15.1:0/107819942 --> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560dc1fd9f50 con 0x560dc21363f0 2024-02-02T15:32:26.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.030+0000 7fa50ce00700 1 --2- 172.21.15.1:0/107819942 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x560dc21363f0 0x560dc21387e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:26.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.030+0000 7fa50ce00700 1 --2- 172.21.15.1:0/107819942 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x560dc21363f0 0x560dc21387e0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3d51784e6a1db463 server_cookie=c106efbe58b56df in_seq=0 out_seq=0 2024-02-02T15:32:26.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.030+0000 7fa4f7400700 10 client.27047 ms_handle_connect on v2:172.21.15.142:6834/1052606547 2024-02-02T15:32:26.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.031+0000 7fa4f7400700 1 -- 172.21.15.1:0/107819942 <== mds.4 v2:172.21.15.142:6834/1052606547 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560dc1fd9f50 con 0x560dc21363f0 2024-02-02T15:32:26.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.031+0000 7fa4f7400700 10 client.27047 handle_command_reply: tid=0 2024-02-02T15:32:26.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.031+0000 7fa523ccfb80 1 client.27047 shutdown 2024-02-02T15:32:26.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.031+0000 7fa523ccfb80 1 -- 172.21.15.1:0/107819942 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560dc20c9010 msgr2=0x560dc20d9310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:26.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.031+0000 7fa523ccfb80 1 --2- 172.21.15.1:0/107819942 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560dc20c9010 0x560dc20d9310 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fa4f801c040 tx=0x7fa4f800f9b0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.031+0000 7fa523ccfb80 1 -- 172.21.15.1:0/107819942 shutdown_connections 2024-02-02T15:32:26.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.031+0000 7fa523ccfb80 1 --2- 172.21.15.1:0/107819942 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x560dc21363f0 0x560dc21387e0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.031+0000 7fa523ccfb80 1 --2- 172.21.15.1:0/107819942 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560dc20d47e0 0x560dc20d9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.031+0000 7fa4f7400700 0 client.27047 ms_handle_reset on v2:172.21.15.142:6834/1052606547 2024-02-02T15:32:26.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.031+0000 7fa523ccfb80 1 --2- 172.21.15.1:0/107819942 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560dc20c9010 0x560dc20d9310 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.031+0000 7fa523ccfb80 1 --2- 172.21.15.1:0/107819942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560dc20c9930 0x560dc20d9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.031+0000 7fa523ccfb80 1 -- 172.21.15.1:0/107819942 >> 172.21.15.1:0/107819942 conn(0x560dc2028db0 msgr2=0x560dc20cc700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:32:26.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.031+0000 7fa523ccfb80 1 -- 172.21.15.1:0/107819942 shutdown_connections 2024-02-02T15:32:26.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.031+0000 7fa523ccfb80 1 -- 172.21.15.1:0/107819942 wait complete. 2024-02-02T15:32:26.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.031+0000 7fa523ccfb80 20 client.27047 trim_cache size 0 max 16384 2024-02-02T15:32:26.038 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:32:26.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.032+0000 7fa510200700 1 -- 172.21.15.1:0/56989430 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fa4e80a0710 msgr2=0x7fa4e80a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:26.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.032+0000 7fa510200700 1 --2- 172.21.15.1:0/56989430 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fa4e80a0710 0x7fa4e80a2bd0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa4f0009fd0 tx=0x7fa4f0019040 comp rx=0 tx=0).stop 2024-02-02T15:32:26.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.032+0000 7fa510200700 1 -- 172.21.15.1:0/56989430 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa50819b860 msgr2=0x7fa50819bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:26.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.032+0000 7fa510200700 1 --2- 172.21.15.1:0/56989430 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa50819b860 0x7fa50819bce0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fa50000c110 tx=0x7fa50000c420 comp rx=0 tx=0).stop 2024-02-02T15:32:26.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.032+0000 7fa510200700 1 -- 172.21.15.1:0/56989430 shutdown_connections 2024-02-02T15:32:26.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.032+0000 7fa510200700 1 --2- 172.21.15.1:0/56989430 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fa4e80a0710 0x7fa4e80a2bd0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.032+0000 7fa510200700 1 --2- 172.21.15.1:0/56989430 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa50819b860 0x7fa50819bce0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.032+0000 7fa510200700 1 --2- 172.21.15.1:0/56989430 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa508100ff0 0x7fa508196ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.032+0000 7fa510200700 1 --2- 172.21.15.1:0/56989430 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa508106110 0x7fa508197400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.032+0000 7fa510200700 1 -- 172.21.15.1:0/56989430 >> 172.21.15.1:0/56989430 conn(0x7fa5080fc810 msgr2=0x7fa5080fd360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:32:26.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.032+0000 7fa510200700 1 -- 172.21.15.1:0/56989430 shutdown_connections 2024-02-02T15:32:26.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.032+0000 7fa510200700 1 -- 172.21.15.1:0/56989430 wait complete. 2024-02-02T15:32:26.050 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-02-02T15:32:26.235 INFO:tasks.workunit.client.0.smithi001.stdout: 1 344137 9.17 MB/sec execute 120 sec latency 16.727 ms 2024-02-02T15:32:26.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.260+0000 7f0428800700 1 -- 172.21.15.1:0/2931784112 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f04201019a0 msgr2=0x7f04201059e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:26.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.260+0000 7f0428800700 1 --2- 172.21.15.1:0/2931784112 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f04201019a0 0x7f04201059e0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f04080098f0 tx=0x7f0408009c00 comp rx=0 tx=0).stop 2024-02-02T15:32:26.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.260+0000 7f0428800700 1 -- 172.21.15.1:0/2931784112 shutdown_connections 2024-02-02T15:32:26.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.260+0000 7f0428800700 1 --2- 172.21.15.1:0/2931784112 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0420106110 0x7f042010dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.260+0000 7f0428800700 1 --2- 172.21.15.1:0/2931784112 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0420100ff0 0x7f04201013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.260+0000 7f0428800700 1 --2- 172.21.15.1:0/2931784112 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f04201019a0 0x7f04201059e0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.260+0000 7f0428800700 1 -- 172.21.15.1:0/2931784112 >> 172.21.15.1:0/2931784112 conn(0x7f04200fc810 msgr2=0x7f04200fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:32:26.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.260+0000 7f0428800700 1 -- 172.21.15.1:0/2931784112 shutdown_connections 2024-02-02T15:32:26.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.260+0000 7f0428800700 1 -- 172.21.15.1:0/2931784112 wait complete. 2024-02-02T15:32:26.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.261+0000 7f0428800700 1 Processor -- start 2024-02-02T15:32:26.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.262+0000 7f0428800700 1 -- start start 2024-02-02T15:32:26.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.262+0000 7f0428800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0420100ff0 0x7f0420199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:26.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.262+0000 7f0428800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0420106110 0x7f0420199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:26.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.262+0000 7f0428800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f042019dad0 0x7f042019df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:26.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.262+0000 7f0428800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f042019e550 con 0x7f0420106110 2024-02-02T15:32:26.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.262+0000 7f0428800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f042019e6c0 con 0x7f0420100ff0 2024-02-02T15:32:26.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.262+0000 7f0428800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f042019e880 con 0x7f042019dad0 2024-02-02T15:32:26.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.262+0000 7f0425e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0420100ff0 0x7f0420199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:26.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.262+0000 7f0425e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0420100ff0 0x7f0420199130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:47808/0 (socket says 172.21.15.1:47808) 2024-02-02T15:32:26.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.262+0000 7f0425400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0420106110 0x7f0420199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:26.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.262+0000 7f0425e00700 1 -- 172.21.15.1:0/1141641170 learned_addr learned my addr 172.21.15.1:0/1141641170 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:32:26.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.262+0000 7f0426800700 1 --2- 172.21.15.1:0/1141641170 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f042019dad0 0x7f042019df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:26.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.262+0000 7f0425e00700 1 -- 172.21.15.1:0/1141641170 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f042019dad0 msgr2=0x7f042019df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:26.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.262+0000 7f0425e00700 1 --2- 172.21.15.1:0/1141641170 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f042019dad0 0x7f042019df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.262+0000 7f0425e00700 1 -- 172.21.15.1:0/1141641170 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0420106110 msgr2=0x7f0420199670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:26.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.262+0000 7f0425e00700 1 --2- 172.21.15.1:0/1141641170 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0420106110 0x7f0420199670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.263+0000 7f0425e00700 1 -- 172.21.15.1:0/1141641170 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0408009520 con 0x7f0420100ff0 2024-02-02T15:32:26.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.263+0000 7f0425e00700 1 --2- 172.21.15.1:0/1141641170 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0420100ff0 0x7f0420199130 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f041000e9c0 tx=0x7f041000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:32:26.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.263+0000 7f041ea00700 1 -- 172.21.15.1:0/1141641170 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0410004a30 con 0x7f0420100ff0 2024-02-02T15:32:26.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.263+0000 7f0428800700 1 -- 172.21.15.1:0/1141641170 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04201a5de0 con 0x7f0420100ff0 2024-02-02T15:32:26.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.263+0000 7f041ea00700 1 -- 172.21.15.1:0/1141641170 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f041000cc80 con 0x7f0420100ff0 2024-02-02T15:32:26.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.263+0000 7f041ea00700 1 -- 172.21.15.1:0/1141641170 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04100054e0 con 0x7f0420100ff0 2024-02-02T15:32:26.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.263+0000 7f0428800700 1 -- 172.21.15.1:0/1141641170 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04201a6260 con 0x7f0420100ff0 2024-02-02T15:32:26.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.266+0000 7f041ea00700 1 -- 172.21.15.1:0/1141641170 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 27) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f0410004b90 con 0x7f0420100ff0 2024-02-02T15:32:26.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.268+0000 7f043c36fb80 1 Processor -- start 2024-02-02T15:32:26.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.268+0000 7f043c36fb80 1 -- start start 2024-02-02T15:32:26.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.269+0000 7f041ea00700 1 --2- 172.21.15.1:0/1141641170 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f04000a0690 0x7f04000a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:26.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.269+0000 7f043c36fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587988b9010 0x5587988b93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:26.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.269+0000 7f043c36fb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5587988b9930 0x5587988c42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:26.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.269+0000 7f043c36fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5587988c47e0 0x5587988c6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:26.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.269+0000 7f043c36fb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55879882d4c0 con 0x5587988b9930 2024-02-02T15:32:26.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.269+0000 7f043c36fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x558798811aa0 con 0x5587988c47e0 2024-02-02T15:32:26.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.269+0000 7f043c36fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55879880f460 con 0x5587988b9010 2024-02-02T15:32:26.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.269+0000 7f041ea00700 1 -- 172.21.15.1:0/1141641170 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f0410014070 con 0x7f0420100ff0 2024-02-02T15:32:26.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.269+0000 7f0425e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587988b9010 0x5587988b93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:26.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.269+0000 7f0426800700 1 --2- 172.21.15.1:0/1141641170 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f04000a0690 0x7f04000a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:26.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.269+0000 7f0425e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587988b9010 0x5587988b93f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:37196/0 (socket says 172.21.15.1:37196) 2024-02-02T15:32:26.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.269+0000 7f0425e00700 1 -- 172.21.15.1:0/838927530 learned_addr learned my addr 172.21.15.1:0/838927530 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:32:26.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.269+0000 7f0425400700 1 --2- 172.21.15.1:0/838927530 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5587988b9930 0x5587988c42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:26.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.269+0000 7f0426800700 1 --2- 172.21.15.1:0/838927530 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5587988c47e0 0x5587988c6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:26.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.270+0000 7f0425e00700 1 -- 172.21.15.1:0/838927530 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5587988c47e0 msgr2=0x5587988c6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:32:26.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.270+0000 7f0425e00700 1 --2- 172.21.15.1:0/838927530 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5587988c47e0 0x5587988c6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.270+0000 7f0425e00700 1 -- 172.21.15.1:0/838927530 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5587988b9930 msgr2=0x5587988c42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:32:26.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.270+0000 7f0425e00700 1 --2- 172.21.15.1:0/838927530 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5587988b9930 0x5587988c42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.270+0000 7f0425e00700 1 -- 172.21.15.1:0/838927530 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55879868c3b0 con 0x5587988b9010 2024-02-02T15:32:26.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.270+0000 7f0426800700 1 --2- 172.21.15.1:0/1141641170 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f04000a0690 0x7f04000a2b50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0418009ec0 tx=0x7f0418009920 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:32:26.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.270+0000 7f0425e00700 1 --2- 172.21.15.1:0/838927530 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587988b9010 0x5587988b93f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f041000cfd0 tx=0x7f0410098cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7890b7d3e7e86332 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:32:26.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.270+0000 7f0417000700 1 -- 172.21.15.1:0/838927530 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0410099a50 con 0x5587988b9010 2024-02-02T15:32:26.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.270+0000 7f0417000700 1 -- 172.21.15.1:0/838927530 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0410099350 con 0x5587988b9010 2024-02-02T15:32:26.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.270+0000 7f0417000700 1 -- 172.21.15.1:0/838927530 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0410097800 con 0x5587988b9010 2024-02-02T15:32:26.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.271+0000 7f043c36fb80 1 -- 172.21.15.1:0/838927530 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587988b9010 msgr2=0x5587988b93f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:26.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.271+0000 7f043c36fb80 1 --2- 172.21.15.1:0/838927530 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587988b9010 0x5587988b93f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f041000cfd0 tx=0x7f0410098cb0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.271+0000 7f043c36fb80 1 -- 172.21.15.1:0/838927530 shutdown_connections 2024-02-02T15:32:26.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.271+0000 7f043c36fb80 1 --2- 172.21.15.1:0/838927530 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5587988c47e0 0x5587988c6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.271+0000 7f043c36fb80 1 --2- 172.21.15.1:0/838927530 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587988b9010 0x5587988b93f0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.271+0000 7f043c36fb80 1 --2- 172.21.15.1:0/838927530 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5587988b9930 0x5587988c42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.271+0000 7f043c36fb80 1 -- 172.21.15.1:0/838927530 >> 172.21.15.1:0/838927530 conn(0x558798818db0 msgr2=0x5587988b53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:32:26.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.271+0000 7f043c36fb80 1 -- 172.21.15.1:0/838927530 shutdown_connections 2024-02-02T15:32:26.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.271+0000 7f043c36fb80 1 -- 172.21.15.1:0/838927530 wait complete. 2024-02-02T15:32:26.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.272+0000 7f043c36fb80 1 -- start start 2024-02-02T15:32:26.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.272+0000 7f043c36fb80 1 Processor -- start 2024-02-02T15:32:26.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.272+0000 7f043c36fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587988b9010 0x5587988c41b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:26.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.273+0000 7f043c36fb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5587988b9930 0x5587988c13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:26.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.273+0000 7f043c36fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5587988c47e0 0x5587988c19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:26.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.273+0000 7f043c36fb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55879880ad50 con 0x5587988b9930 2024-02-02T15:32:26.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.273+0000 7f043c36fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587987a6d80 con 0x5587988c47e0 2024-02-02T15:32:26.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.273+0000 7f043c36fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587987a6a20 con 0x5587988b9010 2024-02-02T15:32:26.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.273+0000 7f043c36fb80 10 client.0 _ll_register_callbacks cb 0x5587987a8290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:32:26.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.273+0000 7f0425e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587988b9010 0x5587988c41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:26.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.273+0000 7f0425e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587988b9010 0x5587988c41b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:37202/0 (socket says 172.21.15.1:37202) 2024-02-02T15:32:26.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.273+0000 7f0425e00700 1 -- 172.21.15.1:0/2618732494 learned_addr learned my addr 172.21.15.1:0/2618732494 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:32:26.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.273+0000 7f0425400700 1 --2- 172.21.15.1:0/2618732494 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5587988b9930 0x5587988c13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:26.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.273+0000 7f0426800700 1 --2- 172.21.15.1:0/2618732494 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5587988c47e0 0x5587988c19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:26.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.273+0000 7f0425e00700 1 -- 172.21.15.1:0/2618732494 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5587988c47e0 msgr2=0x5587988c19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:32:26.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.273+0000 7f0425e00700 1 --2- 172.21.15.1:0/2618732494 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5587988c47e0 0x5587988c19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.273+0000 7f0425e00700 1 -- 172.21.15.1:0/2618732494 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5587988b9930 msgr2=0x5587988c13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:32:26.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.273+0000 7f0425e00700 1 --2- 172.21.15.1:0/2618732494 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5587988b9930 0x5587988c13c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.273+0000 7f0425e00700 1 -- 172.21.15.1:0/2618732494 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04201a6260 con 0x5587988b9010 2024-02-02T15:32:26.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.273+0000 7f0426800700 1 --2- 172.21.15.1:0/2618732494 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5587988c47e0 0x5587988c19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:32:26.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.273+0000 7f0425e00700 1 --2- 172.21.15.1:0/2618732494 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587988b9010 0x5587988c41b0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f0410095dc0 tx=0x7f0410004fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d0965a413bba306 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:32:26.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.274+0000 7f040f400700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:32:26.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.274+0000 7f040f400700 1 -- 172.21.15.1:0/2618732494 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04100c9a00 con 0x5587988b9010 2024-02-02T15:32:26.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.274+0000 7f040f400700 1 -- 172.21.15.1:0/2618732494 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f041009d180 con 0x5587988b9010 2024-02-02T15:32:26.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.274+0000 7f043c36fb80 1 -- 172.21.15.1:0/2618732494 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5587988ad0f0 con 0x5587988b9010 2024-02-02T15:32:26.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.274+0000 7f040f400700 1 -- 172.21.15.1:0/2618732494 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f041007d120 con 0x5587988b9010 2024-02-02T15:32:26.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.274+0000 7f043c36fb80 10 client.27058 fetch_fsmap learned FSMap version 35 2024-02-02T15:32:26.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.274+0000 7f043c36fb80 1 -- 172.21.15.1:0/2618732494 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x558798638b60 con 0x5587988b9010 2024-02-02T15:32:26.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.274+0000 7f040f400700 1 -- 172.21.15.1:0/2618732494 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0410081020 con 0x5587988b9010 2024-02-02T15:32:26.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.274+0000 7f040f400700 1 -- 172.21.15.1:0/2618732494 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 35) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f04100a0f30 con 0x5587988b9010 2024-02-02T15:32:26.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.274+0000 7f043c36fb80 10 client.27058 fetch_fsmap finished waiting for FSMap version 35 2024-02-02T15:32:26.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.274+0000 7f043c36fb80 10 client.27058 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-02-02T15:32:26.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.274+0000 7f043c36fb80 20 client.27058 populate_metadata read hostname 'smithi001' 2024-02-02T15:32:26.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.274+0000 7f043c36fb80 1 --2- 172.21.15.1:0/2618732494 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x5587988d3eb0 0x5587988d62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:26.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.274+0000 7f043c36fb80 4 client.27058 mds_command: new command op to 24514 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:32:26.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.274+0000 7f043c36fb80 1 -- 172.21.15.1:0/2618732494 --> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558798803dd0 con 0x5587988d3eb0 2024-02-02T15:32:26.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.275+0000 7f0425400700 1 --2- 172.21.15.1:0/2618732494 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x5587988d3eb0 0x5587988d62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:26.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.275+0000 7f0425400700 1 --2- 172.21.15.1:0/2618732494 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x5587988d3eb0 0x5587988d62a0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=23f2a3908dd6e171 server_cookie=e12983ecd485e0f5 in_seq=0 out_seq=0 2024-02-02T15:32:26.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.275+0000 7f040f400700 10 client.27058 ms_handle_connect on v2:172.21.15.142:6836/2870289019 2024-02-02T15:32:26.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.276+0000 7f040f400700 1 -- 172.21.15.1:0/2618732494 <== mds.3 v2:172.21.15.142:6836/2870289019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x558798803dd0 con 0x5587988d3eb0 2024-02-02T15:32:26.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.276+0000 7f040f400700 10 client.27058 handle_command_reply: tid=0 2024-02-02T15:32:26.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.276+0000 7f043c36fb80 1 client.27058 shutdown 2024-02-02T15:32:26.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.276+0000 7f043c36fb80 1 -- 172.21.15.1:0/2618732494 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587988b9010 msgr2=0x5587988c41b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:26.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.276+0000 7f043c36fb80 1 --2- 172.21.15.1:0/2618732494 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587988b9010 0x5587988c41b0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f0410095dc0 tx=0x7f0410004fb0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.276+0000 7f043c36fb80 1 -- 172.21.15.1:0/2618732494 shutdown_connections 2024-02-02T15:32:26.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.276+0000 7f043c36fb80 1 --2- 172.21.15.1:0/2618732494 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5587988c47e0 0x5587988c19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.276+0000 7f043c36fb80 1 --2- 172.21.15.1:0/2618732494 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587988b9010 0x5587988c41b0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.276+0000 7f043c36fb80 1 --2- 172.21.15.1:0/2618732494 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x5587988d3eb0 0x5587988d62a0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.276+0000 7f043c36fb80 1 --2- 172.21.15.1:0/2618732494 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5587988b9930 0x5587988c13c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.276+0000 7f043c36fb80 1 -- 172.21.15.1:0/2618732494 >> 172.21.15.1:0/2618732494 conn(0x558798818db0 msgr2=0x5587988b4fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:32:26.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.276+0000 7f040f400700 0 client.27058 ms_handle_reset on v2:172.21.15.142:6836/2870289019 2024-02-02T15:32:26.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.276+0000 7f043c36fb80 1 -- 172.21.15.1:0/2618732494 shutdown_connections 2024-02-02T15:32:26.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.276+0000 7f043c36fb80 1 -- 172.21.15.1:0/2618732494 wait complete. 2024-02-02T15:32:26.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.277+0000 7f043c36fb80 20 client.27058 trim_cache size 0 max 16384 2024-02-02T15:32:26.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.292+0000 7f043c36fb80 1 Processor -- start 2024-02-02T15:32:26.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.292+0000 7f043c36fb80 1 -- start start 2024-02-02T15:32:26.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.292+0000 7f043c36fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5587988b9010 0x5587988cb330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:26.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.292+0000 7f043c36fb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5587988b9930 0x5587988b3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:26.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.292+0000 7f043c36fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587988c47e0 0x5587988c3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:26.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.292+0000 7f043c36fb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55879868e6a0 con 0x5587988b9930 2024-02-02T15:32:26.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.292+0000 7f043c36fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587988c6bd0 con 0x5587988b9010 2024-02-02T15:32:26.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.292+0000 7f043c36fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587988bd210 con 0x5587988c47e0 2024-02-02T15:32:26.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.292+0000 7f0425e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5587988b9010 0x5587988cb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:26.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.292+0000 7f0425400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5587988b9930 0x5587988b3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:26.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.292+0000 7f0425e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5587988b9010 0x5587988cb330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:47834/0 (socket says 172.21.15.1:47834) 2024-02-02T15:32:26.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.292+0000 7f0425e00700 1 -- 172.21.15.1:0/725508003 learned_addr learned my addr 172.21.15.1:0/725508003 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:32:26.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.292+0000 7f0426800700 1 --2- 172.21.15.1:0/725508003 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587988c47e0 0x5587988c3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:26.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.293+0000 7f0425e00700 1 -- 172.21.15.1:0/725508003 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587988c47e0 msgr2=0x5587988c3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:32:26.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.293+0000 7f0425e00700 1 --2- 172.21.15.1:0/725508003 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587988c47e0 0x5587988c3ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.293+0000 7f0425e00700 1 -- 172.21.15.1:0/725508003 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5587988b9930 msgr2=0x5587988b3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:32:26.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.293+0000 7f0425e00700 1 --2- 172.21.15.1:0/725508003 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5587988b9930 0x5587988b3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.293+0000 7f0425e00700 1 -- 172.21.15.1:0/725508003 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5587987d0ea0 con 0x5587988b9010 2024-02-02T15:32:26.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.293+0000 7f0425400700 1 --2- 172.21.15.1:0/725508003 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5587988b9930 0x5587988b3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:32:26.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.293+0000 7f0425e00700 1 --2- 172.21.15.1:0/725508003 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5587988b9010 0x5587988cb330 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f04100976a0 tx=0x7f0410095090 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa8f40909b77230 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:32:26.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.293+0000 7f0417a00700 1 -- 172.21.15.1:0/725508003 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0410098120 con 0x5587988b9010 2024-02-02T15:32:26.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.293+0000 7f0417a00700 1 -- 172.21.15.1:0/725508003 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0410081030 con 0x5587988b9010 2024-02-02T15:32:26.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.293+0000 7f0417a00700 1 -- 172.21.15.1:0/725508003 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0410098840 con 0x5587988b9010 2024-02-02T15:32:26.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.294+0000 7f043c36fb80 1 -- 172.21.15.1:0/725508003 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5587988b9010 msgr2=0x5587988cb330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:26.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.294+0000 7f043c36fb80 1 --2- 172.21.15.1:0/725508003 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5587988b9010 0x5587988cb330 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f04100976a0 tx=0x7f0410095090 comp rx=0 tx=0).stop 2024-02-02T15:32:26.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.294+0000 7f043c36fb80 1 -- 172.21.15.1:0/725508003 shutdown_connections 2024-02-02T15:32:26.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.294+0000 7f043c36fb80 1 --2- 172.21.15.1:0/725508003 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587988c47e0 0x5587988c3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.294+0000 7f043c36fb80 1 --2- 172.21.15.1:0/725508003 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5587988b9010 0x5587988cb330 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.294+0000 7f043c36fb80 1 --2- 172.21.15.1:0/725508003 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5587988b9930 0x5587988b3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.294+0000 7f043c36fb80 1 -- 172.21.15.1:0/725508003 >> 172.21.15.1:0/725508003 conn(0x558798818db0 msgr2=0x5587988d0540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:32:26.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.294+0000 7f043c36fb80 1 -- 172.21.15.1:0/725508003 shutdown_connections 2024-02-02T15:32:26.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.294+0000 7f043c36fb80 1 -- 172.21.15.1:0/725508003 wait complete. 2024-02-02T15:32:26.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.294+0000 7f043c36fb80 1 -- start start 2024-02-02T15:32:26.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.295+0000 7f043c36fb80 1 Processor -- start 2024-02-02T15:32:26.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.295+0000 7f043c36fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5587988b9010 0x5587988cab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:26.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.295+0000 7f043c36fb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5587988b9930 0x5587988cb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:26.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.295+0000 7f043c36fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587988c47e0 0x5587988c7300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:26.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.295+0000 7f043c36fb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587988c78d0 con 0x5587988b9930 2024-02-02T15:32:26.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.295+0000 7f043c36fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587988c7a40 con 0x5587988b9010 2024-02-02T15:32:26.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.295+0000 7f043c36fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587988c7c00 con 0x5587988c47e0 2024-02-02T15:32:26.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.295+0000 7f043c36fb80 10 client.0 _ll_register_callbacks cb 0x5587987a8290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:32:26.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.295+0000 7f0425e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5587988b9010 0x5587988cab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:26.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.295+0000 7f0426800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587988c47e0 0x5587988c7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:26.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.295+0000 7f0425400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5587988b9930 0x5587988cb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:26.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.295+0000 7f0425e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5587988b9010 0x5587988cab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:47844/0 (socket says 172.21.15.1:47844) 2024-02-02T15:32:26.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.295+0000 7f0426800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587988c47e0 0x5587988c7300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:37230/0 (socket says 172.21.15.1:37230) 2024-02-02T15:32:26.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.295+0000 7f0425e00700 1 -- 172.21.15.1:0/1682372248 learned_addr learned my addr 172.21.15.1:0/1682372248 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:32:26.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.296+0000 7f0425e00700 1 -- 172.21.15.1:0/1682372248 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587988c47e0 msgr2=0x5587988c7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:32:26.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.296+0000 7f0425e00700 1 --2- 172.21.15.1:0/1682372248 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587988c47e0 0x5587988c7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.296+0000 7f0425e00700 1 -- 172.21.15.1:0/1682372248 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5587988b9930 msgr2=0x5587988cb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:32:26.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.296+0000 7f0425e00700 1 --2- 172.21.15.1:0/1682372248 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5587988b9930 0x5587988cb090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.296+0000 7f0425e00700 1 -- 172.21.15.1:0/1682372248 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f042010e500 con 0x5587988b9010 2024-02-02T15:32:26.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.296+0000 7f0425e00700 1 --2- 172.21.15.1:0/1682372248 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5587988b9010 0x5587988cab50 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f0410005950 tx=0x7f0410004fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f85d45ea85c1062e server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:32:26.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.296+0000 7f040f400700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:32:26.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.296+0000 7f040f400700 1 -- 172.21.15.1:0/1682372248 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0410098a40 con 0x5587988b9010 2024-02-02T15:32:26.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.296+0000 7f040f400700 1 -- 172.21.15.1:0/1682372248 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f0410082080 con 0x5587988b9010 2024-02-02T15:32:26.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.296+0000 7f043c36fb80 1 -- 172.21.15.1:0/1682372248 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0410099350 con 0x5587988b9010 2024-02-02T15:32:26.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.296+0000 7f040f400700 1 -- 172.21.15.1:0/1682372248 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f04100a1140 con 0x5587988b9010 2024-02-02T15:32:26.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.296+0000 7f043c36fb80 10 client.27065 fetch_fsmap learned FSMap version 35 2024-02-02T15:32:26.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.296+0000 7f043c36fb80 1 -- 172.21.15.1:0/1682372248 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x558798638b60 con 0x5587988b9010 2024-02-02T15:32:26.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.297+0000 7f040f400700 1 -- 172.21.15.1:0/1682372248 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0410099350 con 0x5587988b9010 2024-02-02T15:32:26.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.297+0000 7f040f400700 1 -- 172.21.15.1:0/1682372248 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 35) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0410095140 con 0x5587988b9010 2024-02-02T15:32:26.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.297+0000 7f043c36fb80 10 client.27065 fetch_fsmap finished waiting for FSMap version 35 2024-02-02T15:32:26.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.297+0000 7f043c36fb80 10 client.27065 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-02-02T15:32:26.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.297+0000 7f043c36fb80 20 client.27065 populate_metadata read hostname 'smithi001' 2024-02-02T15:32:26.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.297+0000 7f043c36fb80 1 --2- 172.21.15.1:0/1682372248 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x558798926700 0x558798928af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:26.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.297+0000 7f043c36fb80 4 client.27065 mds_command: new command op to 24514 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:32:26.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.297+0000 7f043c36fb80 1 -- 172.21.15.1:0/1682372248 --> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5587987c9f50 con 0x558798926700 2024-02-02T15:32:26.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.297+0000 7f0425400700 1 --2- 172.21.15.1:0/1682372248 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x558798926700 0x558798928af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:26.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.298+0000 7f0425400700 1 --2- 172.21.15.1:0/1682372248 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x558798926700 0x558798928af0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=def9bd89c031dd19 server_cookie=fcf7ac795cdddd3d in_seq=0 out_seq=0 2024-02-02T15:32:26.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.298+0000 7f040f400700 10 client.27065 ms_handle_connect on v2:172.21.15.142:6836/2870289019 2024-02-02T15:32:26.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.298+0000 7f040f400700 1 -- 172.21.15.1:0/1682372248 <== mds.3 v2:172.21.15.142:6836/2870289019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5587987c9f50 con 0x558798926700 2024-02-02T15:32:26.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.298+0000 7f040f400700 10 client.27065 handle_command_reply: tid=0 2024-02-02T15:32:26.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.298+0000 7f043c36fb80 1 client.27065 shutdown 2024-02-02T15:32:26.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.298+0000 7f043c36fb80 1 -- 172.21.15.1:0/1682372248 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5587988b9010 msgr2=0x5587988cab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:26.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.298+0000 7f043c36fb80 1 --2- 172.21.15.1:0/1682372248 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5587988b9010 0x5587988cab50 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f0410005950 tx=0x7f0410004fb0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.298+0000 7f043c36fb80 1 -- 172.21.15.1:0/1682372248 shutdown_connections 2024-02-02T15:32:26.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.298+0000 7f043c36fb80 1 --2- 172.21.15.1:0/1682372248 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5587988c47e0 0x5587988c7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.298+0000 7f043c36fb80 1 --2- 172.21.15.1:0/1682372248 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5587988b9010 0x5587988cab50 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.298+0000 7f043c36fb80 1 --2- 172.21.15.1:0/1682372248 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x558798926700 0x558798928af0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.298+0000 7f043c36fb80 1 --2- 172.21.15.1:0/1682372248 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5587988b9930 0x5587988cb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.298+0000 7f043c36fb80 1 -- 172.21.15.1:0/1682372248 >> 172.21.15.1:0/1682372248 conn(0x558798818db0 msgr2=0x5587988bc700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:32:26.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.298+0000 7f040f400700 0 client.27065 ms_handle_reset on v2:172.21.15.142:6836/2870289019 2024-02-02T15:32:26.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.298+0000 7f043c36fb80 1 -- 172.21.15.1:0/1682372248 shutdown_connections 2024-02-02T15:32:26.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.298+0000 7f043c36fb80 1 -- 172.21.15.1:0/1682372248 wait complete. 2024-02-02T15:32:26.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.299+0000 7f043c36fb80 20 client.27065 trim_cache size 0 max 16384 2024-02-02T15:32:26.307 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:32:26.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.299+0000 7f0428800700 1 -- 172.21.15.1:0/1141641170 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f04000a0690 msgr2=0x7f04000a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:26.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.299+0000 7f0428800700 1 --2- 172.21.15.1:0/1141641170 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f04000a0690 0x7f04000a2b50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0418009ec0 tx=0x7f0418009920 comp rx=0 tx=0).stop 2024-02-02T15:32:26.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.299+0000 7f0428800700 1 -- 172.21.15.1:0/1141641170 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0420100ff0 msgr2=0x7f0420199130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:26.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.299+0000 7f0428800700 1 --2- 172.21.15.1:0/1141641170 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0420100ff0 0x7f0420199130 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f041000e9c0 tx=0x7f041000ecd0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.299+0000 7f0428800700 1 -- 172.21.15.1:0/1141641170 shutdown_connections 2024-02-02T15:32:26.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.299+0000 7f0428800700 1 --2- 172.21.15.1:0/1141641170 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f04000a0690 0x7f04000a2b50 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.299+0000 7f0428800700 1 --2- 172.21.15.1:0/1141641170 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f042019dad0 0x7f042019df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.299+0000 7f0428800700 1 --2- 172.21.15.1:0/1141641170 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0420100ff0 0x7f0420199130 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.299+0000 7f0428800700 1 --2- 172.21.15.1:0/1141641170 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0420106110 0x7f0420199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.299+0000 7f0428800700 1 -- 172.21.15.1:0/1141641170 >> 172.21.15.1:0/1141641170 conn(0x7f04200fc810 msgr2=0x7f04200fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:32:26.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.299+0000 7f0428800700 1 -- 172.21.15.1:0/1141641170 shutdown_connections 2024-02-02T15:32:26.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.299+0000 7f0428800700 1 -- 172.21.15.1:0/1141641170 wait complete. 2024-02-02T15:32:26.318 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-02T15:32:26.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.524+0000 7fba39200700 1 -- 172.21.15.1:0/2745908667 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fba34100ff0 msgr2=0x7fba341013d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:26.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.524+0000 7fba39200700 1 --2- 172.21.15.1:0/2745908667 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fba34100ff0 0x7fba341013d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fba24009990 tx=0x7fba24009ca0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.524+0000 7fba39200700 1 -- 172.21.15.1:0/2745908667 shutdown_connections 2024-02-02T15:32:26.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.524+0000 7fba39200700 1 --2- 172.21.15.1:0/2745908667 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fba34106110 0x7fba3410dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.524+0000 7fba39200700 1 --2- 172.21.15.1:0/2745908667 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fba34100ff0 0x7fba341013d0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.524+0000 7fba39200700 1 --2- 172.21.15.1:0/2745908667 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba341019a0 0x7fba341059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.524+0000 7fba39200700 1 -- 172.21.15.1:0/2745908667 >> 172.21.15.1:0/2745908667 conn(0x7fba340fc810 msgr2=0x7fba340fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:32:26.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.524+0000 7fba39200700 1 -- 172.21.15.1:0/2745908667 shutdown_connections 2024-02-02T15:32:26.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.524+0000 7fba39200700 1 -- 172.21.15.1:0/2745908667 wait complete. 2024-02-02T15:32:26.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.525+0000 7fba39200700 1 Processor -- start 2024-02-02T15:32:26.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.525+0000 7fba39200700 1 -- start start 2024-02-02T15:32:26.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.526+0000 7fba39200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fba34100ff0 0x7fba34196ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:26.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.526+0000 7fba39200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba34106110 0x7fba341973e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:26.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.526+0000 7fba39200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fba3419b840 0x7fba3419bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:26.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.526+0000 7fba39200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba3419c2c0 con 0x7fba3419b840 2024-02-02T15:32:26.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.526+0000 7fba39200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba3419c430 con 0x7fba34100ff0 2024-02-02T15:32:26.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.526+0000 7fba39200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba3419c5f0 con 0x7fba34106110 2024-02-02T15:32:26.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.526+0000 7fba31e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fba34100ff0 0x7fba34196ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:26.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.526+0000 7fba31e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fba34100ff0 0x7fba34196ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:47864/0 (socket says 172.21.15.1:47864) 2024-02-02T15:32:26.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.526+0000 7fba31e00700 1 -- 172.21.15.1:0/1005440455 learned_addr learned my addr 172.21.15.1:0/1005440455 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:32:26.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.526+0000 7fba31400700 1 --2- 172.21.15.1:0/1005440455 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba34106110 0x7fba341973e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:26.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.526+0000 7fba32800700 1 --2- 172.21.15.1:0/1005440455 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fba3419b840 0x7fba3419bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:26.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.526+0000 7fba31e00700 1 -- 172.21.15.1:0/1005440455 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba34106110 msgr2=0x7fba341973e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:26.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.526+0000 7fba31e00700 1 --2- 172.21.15.1:0/1005440455 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba34106110 0x7fba341973e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.526+0000 7fba31e00700 1 -- 172.21.15.1:0/1005440455 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fba3419b840 msgr2=0x7fba3419bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:26.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.526+0000 7fba31e00700 1 --2- 172.21.15.1:0/1005440455 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fba3419b840 0x7fba3419bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.527+0000 7fba31e00700 1 -- 172.21.15.1:0/1005440455 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba240095f0 con 0x7fba34100ff0 2024-02-02T15:32:26.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.527+0000 7fba31400700 1 --2- 172.21.15.1:0/1005440455 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba34106110 0x7fba341973e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:32:26.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.527+0000 7fba32800700 1 --2- 172.21.15.1:0/1005440455 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fba3419b840 0x7fba3419bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:32:26.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.527+0000 7fba31e00700 1 --2- 172.21.15.1:0/1005440455 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fba34100ff0 0x7fba34196ea0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fba24005910 tx=0x7fba24004710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:32:26.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.527+0000 7fba2aa00700 1 -- 172.21.15.1:0/1005440455 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba2401d070 con 0x7fba34100ff0 2024-02-02T15:32:26.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.527+0000 7fba2aa00700 1 -- 172.21.15.1:0/1005440455 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba24022470 con 0x7fba34100ff0 2024-02-02T15:32:26.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.527+0000 7fba2aa00700 1 -- 172.21.15.1:0/1005440455 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba24017650 con 0x7fba34100ff0 2024-02-02T15:32:26.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.527+0000 7fba39200700 1 -- 172.21.15.1:0/1005440455 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba341a3af0 con 0x7fba34100ff0 2024-02-02T15:32:26.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.527+0000 7fba39200700 1 -- 172.21.15.1:0/1005440455 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba341a3f20 con 0x7fba34100ff0 2024-02-02T15:32:26.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.532+0000 7fba2aa00700 1 -- 172.21.15.1:0/1005440455 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 27) v1 ==== 135200+0+0 (secure 0 0 0) 0x7fba240225e0 con 0x7fba34100ff0 2024-02-02T15:32:26.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.533+0000 7fba4cd74b80 1 Processor -- start 2024-02-02T15:32:26.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.533+0000 7fba4cd74b80 1 -- start start 2024-02-02T15:32:26.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.533+0000 7fba4cd74b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5650a060a010 0x5650a060a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:26.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.533+0000 7fba4cd74b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650a060a930 0x5650a06196b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:26.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.533+0000 7fba4cd74b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5650a0619bf0 0x5650a061bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:26.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.533+0000 7fba4cd74b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650a057e4c0 con 0x5650a0619bf0 2024-02-02T15:32:26.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.533+0000 7fba4cd74b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650a0562aa0 con 0x5650a060a930 2024-02-02T15:32:26.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.533+0000 7fba4cd74b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650a0560460 con 0x5650a060a010 2024-02-02T15:32:26.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.533+0000 7fba2aa00700 1 --2- 172.21.15.1:0/1005440455 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fba140a0710 0x7fba140a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:26.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.534+0000 7fba31e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650a060a930 0x5650a06196b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:26.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.534+0000 7fba31e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650a060a930 0x5650a06196b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:47878/0 (socket says 172.21.15.1:47878) 2024-02-02T15:32:26.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.534+0000 7fba31e00700 1 -- 172.21.15.1:0/1419977138 learned_addr learned my addr 172.21.15.1:0/1419977138 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:32:26.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.534+0000 7fba31400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5650a0619bf0 0x5650a061bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:26.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.534+0000 7fba2aa00700 1 -- 172.21.15.1:0/1005440455 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fba240cd250 con 0x7fba34100ff0 2024-02-02T15:32:26.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.534+0000 7fba32800700 1 --2- 172.21.15.1:0/1005440455 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fba140a0710 0x7fba140a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:26.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.534+0000 7fba32800700 1 --2- 172.21.15.1:0/1419977138 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5650a060a010 0x5650a060a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:26.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.534+0000 7fba31e00700 1 -- 172.21.15.1:0/1419977138 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5650a060a010 msgr2=0x5650a060a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:32:26.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.534+0000 7fba31e00700 1 --2- 172.21.15.1:0/1419977138 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5650a060a010 0x5650a060a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.534+0000 7fba31e00700 1 -- 172.21.15.1:0/1419977138 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5650a0619bf0 msgr2=0x5650a061bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:32:26.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.534+0000 7fba31e00700 1 --2- 172.21.15.1:0/1419977138 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5650a0619bf0 0x5650a061bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.534+0000 7fba31e00700 1 -- 172.21.15.1:0/1419977138 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5650a03dd3b0 con 0x5650a060a930 2024-02-02T15:32:26.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.534+0000 7fba32800700 1 --2- 172.21.15.1:0/1419977138 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5650a060a010 0x5650a060a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:32:26.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.534+0000 7fba31e00700 1 --2- 172.21.15.1:0/1419977138 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650a060a930 0x5650a06196b0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fba240ca560 tx=0x7fba24022cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40a3045d9a5c646d server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:32:26.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.535+0000 7fba23400700 1 -- 172.21.15.1:0/1419977138 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba240850b0 con 0x5650a060a930 2024-02-02T15:32:26.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.535+0000 7fba23400700 1 -- 172.21.15.1:0/1419977138 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba24099f90 con 0x5650a060a930 2024-02-02T15:32:26.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.535+0000 7fba23400700 1 -- 172.21.15.1:0/1419977138 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba240a2430 con 0x5650a060a930 2024-02-02T15:32:26.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.535+0000 7fba32800700 1 --2- 172.21.15.1:0/1005440455 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fba140a0710 0x7fba140a2bd0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fba2c009fd0 tx=0x7fba2c009920 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:32:26.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.535+0000 7fba4cd74b80 1 -- 172.21.15.1:0/1419977138 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650a060a930 msgr2=0x5650a06196b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:26.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.535+0000 7fba4cd74b80 1 --2- 172.21.15.1:0/1419977138 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650a060a930 0x5650a06196b0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fba240ca560 tx=0x7fba24022cd0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.535+0000 7fba4cd74b80 1 -- 172.21.15.1:0/1419977138 shutdown_connections 2024-02-02T15:32:26.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.535+0000 7fba4cd74b80 1 --2- 172.21.15.1:0/1419977138 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5650a0619bf0 0x5650a061bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.535+0000 7fba4cd74b80 1 --2- 172.21.15.1:0/1419977138 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5650a060a010 0x5650a060a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.535+0000 7fba4cd74b80 1 --2- 172.21.15.1:0/1419977138 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650a060a930 0x5650a06196b0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.535+0000 7fba4cd74b80 1 -- 172.21.15.1:0/1419977138 >> 172.21.15.1:0/1419977138 conn(0x5650a0569db0 msgr2=0x5650a06063b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:32:26.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.535+0000 7fba4cd74b80 1 -- 172.21.15.1:0/1419977138 shutdown_connections 2024-02-02T15:32:26.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.535+0000 7fba4cd74b80 1 -- 172.21.15.1:0/1419977138 wait complete. 2024-02-02T15:32:26.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.536+0000 7fba4cd74b80 1 -- start start 2024-02-02T15:32:26.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.536+0000 7fba4cd74b80 1 Processor -- start 2024-02-02T15:32:26.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.537+0000 7fba4cd74b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5650a060a010 0x5650a0619fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:26.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.537+0000 7fba4cd74b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5650a060a930 0x5650a061a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:26.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.537+0000 7fba4cd74b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650a0619bf0 0x5650a061aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:26.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.537+0000 7fba4cd74b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650a055bd50 con 0x5650a060a010 2024-02-02T15:32:26.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.537+0000 7fba4cd74b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650a04f7d80 con 0x5650a0619bf0 2024-02-02T15:32:26.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.537+0000 7fba4cd74b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650a04f7a20 con 0x5650a060a930 2024-02-02T15:32:26.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.537+0000 7fba4cd74b80 10 client.0 _ll_register_callbacks cb 0x5650a04f9290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:32:26.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.537+0000 7fba31e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5650a060a010 0x5650a0619fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:26.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.537+0000 7fba31400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5650a060a930 0x5650a061a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:26.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.537+0000 7fba31400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5650a060a930 0x5650a061a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:37280/0 (socket says 172.21.15.1:37280) 2024-02-02T15:32:26.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.537+0000 7fba31e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5650a060a010 0x5650a0619fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39288/0 (socket says 172.21.15.1:39288) 2024-02-02T15:32:26.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.537+0000 7fba31400700 1 -- 172.21.15.1:0/2913835774 learned_addr learned my addr 172.21.15.1:0/2913835774 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:32:26.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.537+0000 7fba32800700 1 --2- 172.21.15.1:0/2913835774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650a0619bf0 0x5650a061aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:26.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.538+0000 7fba31e00700 1 -- 172.21.15.1:0/2913835774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5650a060a930 msgr2=0x5650a061a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:32:26.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.538+0000 7fba31e00700 1 --2- 172.21.15.1:0/2913835774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5650a060a930 0x5650a061a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.538+0000 7fba31e00700 1 -- 172.21.15.1:0/2913835774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650a0619bf0 msgr2=0x5650a061aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:32:26.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.538+0000 7fba31e00700 1 --2- 172.21.15.1:0/2913835774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650a0619bf0 0x5650a061aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.538+0000 7fba31e00700 1 -- 172.21.15.1:0/2913835774 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba341a3f20 con 0x5650a060a010 2024-02-02T15:32:26.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.538+0000 7fba31400700 1 --2- 172.21.15.1:0/2913835774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5650a060a930 0x5650a061a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:32:26.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.538+0000 7fba31e00700 1 --2- 172.21.15.1:0/2913835774 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5650a060a010 0x5650a0619fd0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fba24096950 tx=0x7fba240ca590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb78dca869ddc5e1 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:32:26.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.538+0000 7fba1be00700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:32:26.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.538+0000 7fba1be00700 1 -- 172.21.15.1:0/2913835774 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba240850b0 con 0x5650a060a010 2024-02-02T15:32:26.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.538+0000 7fba1be00700 1 -- 172.21.15.1:0/2913835774 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fba24083e40 con 0x5650a060a010 2024-02-02T15:32:26.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.538+0000 7fba4cd74b80 1 -- 172.21.15.1:0/2913835774 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5650a05fe0f0 con 0x5650a060a010 2024-02-02T15:32:26.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.538+0000 7fba1be00700 1 -- 172.21.15.1:0/2913835774 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fba2409ab50 con 0x5650a060a010 2024-02-02T15:32:26.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.538+0000 7fba4cd74b80 10 client.17016 fetch_fsmap learned FSMap version 35 2024-02-02T15:32:26.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.538+0000 7fba4cd74b80 1 -- 172.21.15.1:0/2913835774 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5650a0389b60 con 0x5650a060a010 2024-02-02T15:32:26.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.539+0000 7fba1be00700 1 -- 172.21.15.1:0/2913835774 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5650a05fe0f0 con 0x5650a060a010 2024-02-02T15:32:26.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.539+0000 7fba1be00700 1 -- 172.21.15.1:0/2913835774 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 35) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fba24099940 con 0x5650a060a010 2024-02-02T15:32:26.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.539+0000 7fba4cd74b80 10 client.17016 fetch_fsmap finished waiting for FSMap version 35 2024-02-02T15:32:26.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.539+0000 7fba4cd74b80 10 client.17016 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-02-02T15:32:26.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.539+0000 7fba4cd74b80 20 client.17016 populate_metadata read hostname 'smithi001' 2024-02-02T15:32:26.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.539+0000 7fba4cd74b80 1 --2- 172.21.15.1:0/2913835774 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x5650a0628d00 0x5650a062b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:26.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.539+0000 7fba4cd74b80 4 client.17016 mds_command: new command op to 24518 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:32:26.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.539+0000 7fba4cd74b80 1 -- 172.21.15.1:0/2913835774 --> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5650a0554dd0 con 0x5650a0628d00 2024-02-02T15:32:26.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.539+0000 7fba31400700 1 --2- 172.21.15.1:0/2913835774 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x5650a0628d00 0x5650a062b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:26.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.540+0000 7fba31400700 1 --2- 172.21.15.1:0/2913835774 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x5650a0628d00 0x5650a062b0f0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d5f9250fd7ff5299 server_cookie=6f5eb3dd276bcf5d in_seq=0 out_seq=0 2024-02-02T15:32:26.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.540+0000 7fba1be00700 10 client.17016 ms_handle_connect on v2:172.21.15.71:6836/628016199 2024-02-02T15:32:26.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.540+0000 7fba1be00700 1 -- 172.21.15.1:0/2913835774 <== mds.2 v2:172.21.15.71:6836/628016199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5650a0554dd0 con 0x5650a0628d00 2024-02-02T15:32:26.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.540+0000 7fba1be00700 10 client.17016 handle_command_reply: tid=0 2024-02-02T15:32:26.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.540+0000 7fba4cd74b80 1 client.17016 shutdown 2024-02-02T15:32:26.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.540+0000 7fba4cd74b80 1 -- 172.21.15.1:0/2913835774 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5650a060a010 msgr2=0x5650a0619fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:26.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.540+0000 7fba4cd74b80 1 --2- 172.21.15.1:0/2913835774 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5650a060a010 0x5650a0619fd0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fba24096950 tx=0x7fba240ca590 comp rx=0 tx=0).stop 2024-02-02T15:32:26.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.541+0000 7fba4cd74b80 1 -- 172.21.15.1:0/2913835774 shutdown_connections 2024-02-02T15:32:26.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.541+0000 7fba4cd74b80 1 --2- 172.21.15.1:0/2913835774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650a0619bf0 0x5650a061aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.541+0000 7fba4cd74b80 1 --2- 172.21.15.1:0/2913835774 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5650a060a010 0x5650a0619fd0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.541+0000 7fba4cd74b80 1 --2- 172.21.15.1:0/2913835774 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x5650a0628d00 0x5650a062b0f0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.541+0000 7fba4cd74b80 1 --2- 172.21.15.1:0/2913835774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5650a060a930 0x5650a061a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.541+0000 7fba1be00700 0 client.17016 ms_handle_reset on v2:172.21.15.71:6836/628016199 2024-02-02T15:32:26.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.541+0000 7fba4cd74b80 1 -- 172.21.15.1:0/2913835774 >> 172.21.15.1:0/2913835774 conn(0x5650a0569db0 msgr2=0x5650a0605fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:32:26.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.541+0000 7fba4cd74b80 1 -- 172.21.15.1:0/2913835774 shutdown_connections 2024-02-02T15:32:26.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.541+0000 7fba4cd74b80 1 -- 172.21.15.1:0/2913835774 wait complete. 2024-02-02T15:32:26.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.541+0000 7fba4cd74b80 20 client.17016 trim_cache size 0 max 16384 2024-02-02T15:32:26.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.567+0000 7fba4cd74b80 1 Processor -- start 2024-02-02T15:32:26.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.567+0000 7fba4cd74b80 1 -- start start 2024-02-02T15:32:26.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.567+0000 7fba4cd74b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5650a060a010 0x5650a061e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:26.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.567+0000 7fba4cd74b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650a060a930 0x5650a060bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:26.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.567+0000 7fba4cd74b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5650a0619bf0 0x5650a06245e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:26.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.567+0000 7fba4cd74b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650a03df6a0 con 0x5650a060a010 2024-02-02T15:32:26.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.567+0000 7fba4cd74b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650a060ad50 con 0x5650a060a930 2024-02-02T15:32:26.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.567+0000 7fba4cd74b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650a06192c0 con 0x5650a0619bf0 2024-02-02T15:32:26.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.568+0000 7fba31400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650a060a930 0x5650a060bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:26.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.568+0000 7fba32800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5650a0619bf0 0x5650a06245e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:26.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.568+0000 7fba32800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5650a0619bf0 0x5650a06245e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:37292/0 (socket says 172.21.15.1:37292) 2024-02-02T15:32:26.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.568+0000 7fba32800700 1 -- 172.21.15.1:0/650799419 learned_addr learned my addr 172.21.15.1:0/650799419 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:32:26.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.568+0000 7fba31e00700 1 --2- 172.21.15.1:0/650799419 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5650a060a010 0x5650a061e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:26.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.569+0000 7fba32800700 1 -- 172.21.15.1:0/650799419 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650a060a930 msgr2=0x5650a060bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:32:26.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.569+0000 7fba32800700 1 --2- 172.21.15.1:0/650799419 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650a060a930 0x5650a060bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.569+0000 7fba32800700 1 -- 172.21.15.1:0/650799419 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5650a060a010 msgr2=0x5650a061e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:32:26.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.569+0000 7fba32800700 1 --2- 172.21.15.1:0/650799419 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5650a060a010 0x5650a061e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.569+0000 7fba32800700 1 -- 172.21.15.1:0/650799419 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5650a0521ea0 con 0x5650a0619bf0 2024-02-02T15:32:26.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.569+0000 7fba32800700 1 --2- 172.21.15.1:0/650799419 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5650a0619bf0 0x5650a06245e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fba2c00e570 tx=0x7fba2c016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1f5e664e5fd5dfe server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:32:26.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.570+0000 7fba23e00700 1 -- 172.21.15.1:0/650799419 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba2c019480 con 0x5650a0619bf0 2024-02-02T15:32:26.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.570+0000 7fba23e00700 1 -- 172.21.15.1:0/650799419 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba2c00f040 con 0x5650a0619bf0 2024-02-02T15:32:26.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.570+0000 7fba23e00700 1 -- 172.21.15.1:0/650799419 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba2c017a20 con 0x5650a0619bf0 2024-02-02T15:32:26.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.571+0000 7fba4cd74b80 1 -- 172.21.15.1:0/650799419 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5650a0619bf0 msgr2=0x5650a06245e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:26.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.571+0000 7fba4cd74b80 1 --2- 172.21.15.1:0/650799419 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5650a0619bf0 0x5650a06245e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fba2c00e570 tx=0x7fba2c016000 comp rx=0 tx=0).stop 2024-02-02T15:32:26.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.571+0000 7fba4cd74b80 1 -- 172.21.15.1:0/650799419 shutdown_connections 2024-02-02T15:32:26.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.571+0000 7fba4cd74b80 1 --2- 172.21.15.1:0/650799419 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5650a0619bf0 0x5650a06245e0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.571+0000 7fba4cd74b80 1 --2- 172.21.15.1:0/650799419 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5650a060a010 0x5650a061e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.571+0000 7fba4cd74b80 1 --2- 172.21.15.1:0/650799419 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650a060a930 0x5650a060bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.571+0000 7fba4cd74b80 1 -- 172.21.15.1:0/650799419 >> 172.21.15.1:0/650799419 conn(0x5650a0569db0 msgr2=0x5650a0625540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:32:26.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.571+0000 7fba4cd74b80 1 -- 172.21.15.1:0/650799419 shutdown_connections 2024-02-02T15:32:26.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.571+0000 7fba4cd74b80 1 -- 172.21.15.1:0/650799419 wait complete. 2024-02-02T15:32:26.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.572+0000 7fba4cd74b80 1 -- start start 2024-02-02T15:32:26.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.572+0000 7fba4cd74b80 1 Processor -- start 2024-02-02T15:32:26.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.573+0000 7fba4cd74b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5650a060a010 0x5650a0619fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:26.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.573+0000 7fba4cd74b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5650a060a930 0x5650a061a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:26.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.573+0000 7fba4cd74b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650a0619bf0 0x5650a061aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:26.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.573+0000 7fba4cd74b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650a067dfd0 con 0x5650a060a010 2024-02-02T15:32:26.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.573+0000 7fba4cd74b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650a067e140 con 0x5650a0619bf0 2024-02-02T15:32:26.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.573+0000 7fba4cd74b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650a067e300 con 0x5650a060a930 2024-02-02T15:32:26.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.573+0000 7fba4cd74b80 10 client.0 _ll_register_callbacks cb 0x5650a04f9290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:32:26.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.573+0000 7fba31e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5650a060a010 0x5650a0619fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:26.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.574+0000 7fba31e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5650a060a010 0x5650a0619fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39292/0 (socket says 172.21.15.1:39292) 2024-02-02T15:32:26.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.574+0000 7fba31e00700 1 -- 172.21.15.1:0/3530679383 learned_addr learned my addr 172.21.15.1:0/3530679383 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:32:26.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.574+0000 7fba31400700 1 --2- 172.21.15.1:0/3530679383 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5650a060a930 0x5650a061a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:26.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.574+0000 7fba31e00700 1 -- 172.21.15.1:0/3530679383 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5650a060a930 msgr2=0x5650a061a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:32:26.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.574+0000 7fba31e00700 1 --2- 172.21.15.1:0/3530679383 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5650a060a930 0x5650a061a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.574+0000 7fba31e00700 1 -- 172.21.15.1:0/3530679383 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650a0619bf0 msgr2=0x5650a061aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:32:26.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.574+0000 7fba32800700 1 --2- 172.21.15.1:0/3530679383 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650a0619bf0 0x5650a061aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:26.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.574+0000 7fba31e00700 1 --2- 172.21.15.1:0/3530679383 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650a0619bf0 0x5650a061aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.574+0000 7fba31e00700 1 -- 172.21.15.1:0/3530679383 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba2c008f70 con 0x5650a060a010 2024-02-02T15:32:26.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.574+0000 7fba32800700 1 --2- 172.21.15.1:0/3530679383 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650a0619bf0 0x5650a061aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:32:26.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.575+0000 7fba31e00700 1 --2- 172.21.15.1:0/3530679383 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5650a060a010 0x5650a0619fd0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fba240c6040 tx=0x7fba240a21b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9fba05425cab3f4 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:32:26.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.575+0000 7fba1be00700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:32:26.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.575+0000 7fba1be00700 1 -- 172.21.15.1:0/3530679383 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba2409a8c0 con 0x5650a060a010 2024-02-02T15:32:26.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.575+0000 7fba1be00700 1 -- 172.21.15.1:0/3530679383 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fba2409ab50 con 0x5650a060a010 2024-02-02T15:32:26.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.575+0000 7fba4cd74b80 1 -- 172.21.15.1:0/3530679383 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fba24099f90 con 0x5650a060a010 2024-02-02T15:32:26.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.575+0000 7fba1be00700 1 -- 172.21.15.1:0/3530679383 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fba240a2430 con 0x5650a060a010 2024-02-02T15:32:26.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.575+0000 7fba4cd74b80 10 client.17022 fetch_fsmap learned FSMap version 35 2024-02-02T15:32:26.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.576+0000 7fba4cd74b80 1 -- 172.21.15.1:0/3530679383 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x5650a03e1cd0 con 0x5650a060a010 2024-02-02T15:32:26.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.576+0000 7fba1be00700 1 -- 172.21.15.1:0/3530679383 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba24099f90 con 0x5650a060a010 2024-02-02T15:32:26.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.576+0000 7fba1be00700 1 -- 172.21.15.1:0/3530679383 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 35) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fba2400fd20 con 0x5650a060a010 2024-02-02T15:32:26.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.576+0000 7fba4cd74b80 10 client.17022 fetch_fsmap finished waiting for FSMap version 35 2024-02-02T15:32:26.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.576+0000 7fba4cd74b80 10 client.17022 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-02-02T15:32:26.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.576+0000 7fba4cd74b80 20 client.17022 populate_metadata read hostname 'smithi001' 2024-02-02T15:32:26.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.576+0000 7fba4cd74b80 1 --2- 172.21.15.1:0/3530679383 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x5650a067f530 0x5650a0681920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:26.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.576+0000 7fba4cd74b80 4 client.17022 mds_command: new command op to 24518 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:32:26.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.576+0000 7fba4cd74b80 1 -- 172.21.15.1:0/3530679383 --> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5650a051af50 con 0x5650a067f530 2024-02-02T15:32:26.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.577+0000 7fba31400700 1 --2- 172.21.15.1:0/3530679383 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x5650a067f530 0x5650a0681920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:26.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.578+0000 7fba31400700 1 --2- 172.21.15.1:0/3530679383 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x5650a067f530 0x5650a0681920 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ad90cfe846429049 server_cookie=c10bf91eedf3f4d2 in_seq=0 out_seq=0 2024-02-02T15:32:26.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.578+0000 7fba1be00700 10 client.17022 ms_handle_connect on v2:172.21.15.71:6836/628016199 2024-02-02T15:32:26.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.579+0000 7fba1be00700 1 -- 172.21.15.1:0/3530679383 <== mds.2 v2:172.21.15.71:6836/628016199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5650a051af50 con 0x5650a067f530 2024-02-02T15:32:26.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.579+0000 7fba1be00700 10 client.17022 handle_command_reply: tid=0 2024-02-02T15:32:26.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.579+0000 7fba4cd74b80 1 client.17022 shutdown 2024-02-02T15:32:26.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.579+0000 7fba4cd74b80 1 -- 172.21.15.1:0/3530679383 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5650a060a010 msgr2=0x5650a0619fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:26.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.579+0000 7fba4cd74b80 1 --2- 172.21.15.1:0/3530679383 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5650a060a010 0x5650a0619fd0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fba240c6040 tx=0x7fba240a21b0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.579+0000 7fba4cd74b80 1 -- 172.21.15.1:0/3530679383 shutdown_connections 2024-02-02T15:32:26.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.579+0000 7fba4cd74b80 1 --2- 172.21.15.1:0/3530679383 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5650a0619bf0 0x5650a061aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.579+0000 7fba4cd74b80 1 --2- 172.21.15.1:0/3530679383 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5650a060a010 0x5650a0619fd0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.579+0000 7fba4cd74b80 1 --2- 172.21.15.1:0/3530679383 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x5650a067f530 0x5650a0681920 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.579+0000 7fba4cd74b80 1 --2- 172.21.15.1:0/3530679383 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5650a060a930 0x5650a061a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.580+0000 7fba4cd74b80 1 -- 172.21.15.1:0/3530679383 >> 172.21.15.1:0/3530679383 conn(0x5650a0569db0 msgr2=0x5650a06245e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:32:26.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.580+0000 7fba1be00700 0 client.17022 ms_handle_reset on v2:172.21.15.71:6836/628016199 2024-02-02T15:32:26.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.580+0000 7fba4cd74b80 1 -- 172.21.15.1:0/3530679383 shutdown_connections 2024-02-02T15:32:26.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.580+0000 7fba4cd74b80 1 -- 172.21.15.1:0/3530679383 wait complete. 2024-02-02T15:32:26.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.581+0000 7fba4cd74b80 20 client.17022 trim_cache size 0 max 16384 2024-02-02T15:32:26.581 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:32:26.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.582+0000 7fba39200700 1 -- 172.21.15.1:0/1005440455 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fba140a0710 msgr2=0x7fba140a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:26.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.582+0000 7fba39200700 1 --2- 172.21.15.1:0/1005440455 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fba140a0710 0x7fba140a2bd0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fba2c009fd0 tx=0x7fba2c009920 comp rx=0 tx=0).stop 2024-02-02T15:32:26.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.582+0000 7fba39200700 1 -- 172.21.15.1:0/1005440455 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fba34100ff0 msgr2=0x7fba34196ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:26.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.582+0000 7fba39200700 1 --2- 172.21.15.1:0/1005440455 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fba34100ff0 0x7fba34196ea0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fba24005910 tx=0x7fba24004710 comp rx=0 tx=0).stop 2024-02-02T15:32:26.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.582+0000 7fba39200700 1 -- 172.21.15.1:0/1005440455 shutdown_connections 2024-02-02T15:32:26.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.582+0000 7fba39200700 1 --2- 172.21.15.1:0/1005440455 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fba140a0710 0x7fba140a2bd0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.582+0000 7fba39200700 1 --2- 172.21.15.1:0/1005440455 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fba3419b840 0x7fba3419bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.582+0000 7fba39200700 1 --2- 172.21.15.1:0/1005440455 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fba34100ff0 0x7fba34196ea0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.582+0000 7fba39200700 1 --2- 172.21.15.1:0/1005440455 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba34106110 0x7fba341973e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.582+0000 7fba39200700 1 -- 172.21.15.1:0/1005440455 >> 172.21.15.1:0/1005440455 conn(0x7fba340fc810 msgr2=0x7fba34104a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:32:26.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.582+0000 7fba39200700 1 -- 172.21.15.1:0/1005440455 shutdown_connections 2024-02-02T15:32:26.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.583+0000 7fba39200700 1 -- 172.21.15.1:0/1005440455 wait complete. 2024-02-02T15:32:26.599 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:32:26 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/3638953390' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:32:26.601 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-02T15:32:26.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:32:26 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/3638953390' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:32:26.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:32:26 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/3638953390' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:32:26.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.808+0000 7fce83c00700 1 -- 172.21.15.1:0/1475061743 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce7c101000 msgr2=0x7fce7c1013e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:26.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.808+0000 7fce83c00700 1 --2- 172.21.15.1:0/1475061743 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce7c101000 0x7fce7c1013e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fce6c009990 tx=0x7fce6c009ca0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.808+0000 7fce83c00700 1 -- 172.21.15.1:0/1475061743 shutdown_connections 2024-02-02T15:32:26.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.808+0000 7fce83c00700 1 --2- 172.21.15.1:0/1475061743 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fce7c106120 0x7fce7c10dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.808+0000 7fce83c00700 1 --2- 172.21.15.1:0/1475061743 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce7c101000 0x7fce7c1013e0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.808+0000 7fce83c00700 1 --2- 172.21.15.1:0/1475061743 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce7c1019b0 0x7fce7c1059f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.808+0000 7fce83c00700 1 -- 172.21.15.1:0/1475061743 >> 172.21.15.1:0/1475061743 conn(0x7fce7c0fc860 msgr2=0x7fce7c0fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:32:26.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.808+0000 7fce83c00700 1 -- 172.21.15.1:0/1475061743 shutdown_connections 2024-02-02T15:32:26.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.808+0000 7fce83c00700 1 -- 172.21.15.1:0/1475061743 wait complete. 2024-02-02T15:32:26.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.809+0000 7fce83c00700 1 Processor -- start 2024-02-02T15:32:26.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.810+0000 7fce83c00700 1 -- start start 2024-02-02T15:32:26.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.810+0000 7fce83c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce7c101000 0x7fce7c196f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:26.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.810+0000 7fce83c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce7c106120 0x7fce7c197460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:26.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.810+0000 7fce83c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fce7c19b8c0 0x7fce7c19bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:26.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.810+0000 7fce83c00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce7c19c340 con 0x7fce7c19b8c0 2024-02-02T15:32:26.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.810+0000 7fce83c00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce7c19c4b0 con 0x7fce7c101000 2024-02-02T15:32:26.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.810+0000 7fce83c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce7c19c670 con 0x7fce7c106120 2024-02-02T15:32:26.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.810+0000 7fce81200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce7c101000 0x7fce7c196f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:26.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.810+0000 7fce81200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce7c101000 0x7fce7c196f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:47940/0 (socket says 172.21.15.1:47940) 2024-02-02T15:32:26.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.810+0000 7fce7be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce7c106120 0x7fce7c197460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:26.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.810+0000 7fce81200700 1 -- 172.21.15.1:0/93308962 learned_addr learned my addr 172.21.15.1:0/93308962 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:32:26.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.810+0000 7fce81c00700 1 --2- 172.21.15.1:0/93308962 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fce7c19b8c0 0x7fce7c19bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:26.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.811+0000 7fce81200700 1 -- 172.21.15.1:0/93308962 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce7c106120 msgr2=0x7fce7c197460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:26.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.811+0000 7fce81200700 1 --2- 172.21.15.1:0/93308962 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce7c106120 0x7fce7c197460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.811+0000 7fce81200700 1 -- 172.21.15.1:0/93308962 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fce7c19b8c0 msgr2=0x7fce7c19bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:26.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.811+0000 7fce81200700 1 --2- 172.21.15.1:0/93308962 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fce7c19b8c0 0x7fce7c19bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.811+0000 7fce81200700 1 -- 172.21.15.1:0/93308962 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce6c0095f0 con 0x7fce7c101000 2024-02-02T15:32:26.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.811+0000 7fce81200700 1 --2- 172.21.15.1:0/93308962 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce7c101000 0x7fce7c196f20 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fce6c005910 tx=0x7fce6c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:32:26.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.811+0000 7fce79600700 1 -- 172.21.15.1:0/93308962 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6c01d070 con 0x7fce7c101000 2024-02-02T15:32:26.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.811+0000 7fce79600700 1 -- 172.21.15.1:0/93308962 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fce6c022470 con 0x7fce7c101000 2024-02-02T15:32:26.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.811+0000 7fce79600700 1 -- 172.21.15.1:0/93308962 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6c017650 con 0x7fce7c101000 2024-02-02T15:32:26.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.811+0000 7fce83c00700 1 -- 172.21.15.1:0/93308962 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce7c1a3b70 con 0x7fce7c101000 2024-02-02T15:32:26.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.812+0000 7fce83c00700 1 -- 172.21.15.1:0/93308962 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce7c1a3f40 con 0x7fce7c101000 2024-02-02T15:32:26.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.815+0000 7fce79600700 1 -- 172.21.15.1:0/93308962 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 27) v1 ==== 135200+0+0 (secure 0 0 0) 0x7fce6c0225e0 con 0x7fce7c101000 2024-02-02T15:32:26.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.816+0000 7fce97733b80 1 Processor -- start 2024-02-02T15:32:26.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.816+0000 7fce97733b80 1 -- start start 2024-02-02T15:32:26.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.817+0000 7fce97733b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563b3772d420 0x563b3772d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:26.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.817+0000 7fce97733b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b3772dd40 0x563b377386b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:26.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.817+0000 7fce97733b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563b37738bf0 0x563b3773afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:26.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.817+0000 7fce97733b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b3769d4c0 con 0x563b37738bf0 2024-02-02T15:32:26.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.817+0000 7fce97733b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b37681aa0 con 0x563b3772d420 2024-02-02T15:32:26.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.817+0000 7fce97733b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b3767f460 con 0x563b3772dd40 2024-02-02T15:32:26.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.817+0000 7fce79600700 1 --2- 172.21.15.1:0/93308962 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fce5c0a0710 0x7fce5c0a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:26.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.817+0000 7fce81200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b3772dd40 0x563b377386b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:26.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.817+0000 7fce81200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b3772dd40 0x563b377386b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:37344/0 (socket says 172.21.15.1:37344) 2024-02-02T15:32:26.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.817+0000 7fce81200700 1 -- 172.21.15.1:0/2707423629 learned_addr learned my addr 172.21.15.1:0/2707423629 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:32:26.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.817+0000 7fce81c00700 1 --2- 172.21.15.1:0/2707423629 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563b3772d420 0x563b3772d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:26.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.817+0000 7fce7be00700 1 --2- 172.21.15.1:0/2707423629 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563b37738bf0 0x563b3773afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:26.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.817+0000 7fce81c00700 1 --2- 172.21.15.1:0/93308962 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fce5c0a0710 0x7fce5c0a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:26.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.818+0000 7fce79600700 1 -- 172.21.15.1:0/93308962 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fce6c0cd290 con 0x7fce7c101000 2024-02-02T15:32:26.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.818+0000 7fce81200700 1 -- 172.21.15.1:0/2707423629 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563b3772d420 msgr2=0x563b3772d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:32:26.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.818+0000 7fce81200700 1 --2- 172.21.15.1:0/2707423629 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563b3772d420 0x563b3772d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.818+0000 7fce81200700 1 -- 172.21.15.1:0/2707423629 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563b37738bf0 msgr2=0x563b3773afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:32:26.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.818+0000 7fce81200700 1 --2- 172.21.15.1:0/2707423629 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563b37738bf0 0x563b3773afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.818+0000 7fce81200700 1 -- 172.21.15.1:0/2707423629 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b374fc3b0 con 0x563b3772dd40 2024-02-02T15:32:26.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.818+0000 7fce81200700 1 --2- 172.21.15.1:0/2707423629 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b3772dd40 0x563b377386b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fce6c0ca5a0 tx=0x7fce6c009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae75d1e138778e2e server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:32:26.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.818+0000 7fce72000700 1 -- 172.21.15.1:0/2707423629 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6c09a260 con 0x563b3772dd40 2024-02-02T15:32:26.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.818+0000 7fce81c00700 1 --2- 172.21.15.1:0/93308962 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fce5c0a0710 0x7fce5c0a2bd0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fce74009ec0 tx=0x7fce7400b040 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:32:26.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.818+0000 7fce72000700 1 -- 172.21.15.1:0/2707423629 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fce6c099fd0 con 0x563b3772dd40 2024-02-02T15:32:26.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.818+0000 7fce72000700 1 -- 172.21.15.1:0/2707423629 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6c0228c0 con 0x563b3772dd40 2024-02-02T15:32:26.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.819+0000 7fce97733b80 1 -- 172.21.15.1:0/2707423629 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b3772dd40 msgr2=0x563b377386b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:26.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.819+0000 7fce97733b80 1 --2- 172.21.15.1:0/2707423629 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b3772dd40 0x563b377386b0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fce6c0ca5a0 tx=0x7fce6c009ca0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.819+0000 7fce97733b80 1 -- 172.21.15.1:0/2707423629 shutdown_connections 2024-02-02T15:32:26.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.819+0000 7fce97733b80 1 --2- 172.21.15.1:0/2707423629 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563b37738bf0 0x563b3773afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.819+0000 7fce97733b80 1 --2- 172.21.15.1:0/2707423629 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563b3772d420 0x563b3772d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.819+0000 7fce97733b80 1 --2- 172.21.15.1:0/2707423629 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b3772dd40 0x563b377386b0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.819+0000 7fce97733b80 1 -- 172.21.15.1:0/2707423629 >> 172.21.15.1:0/2707423629 conn(0x563b37688db0 msgr2=0x563b377253b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:32:26.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.819+0000 7fce97733b80 1 -- 172.21.15.1:0/2707423629 shutdown_connections 2024-02-02T15:32:26.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.819+0000 7fce97733b80 1 -- 172.21.15.1:0/2707423629 wait complete. 2024-02-02T15:32:26.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.820+0000 7fce97733b80 1 -- start start 2024-02-02T15:32:26.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.820+0000 7fce97733b80 1 Processor -- start 2024-02-02T15:32:26.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.820+0000 7fce97733b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563b3772d420 0x563b3773dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:26.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.820+0000 7fce97733b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b3772dd40 0x563b3773e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:26.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.820+0000 7fce97733b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563b37738bf0 0x563b37737bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:26.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.820+0000 7fce97733b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b3767ad50 con 0x563b37738bf0 2024-02-02T15:32:26.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.820+0000 7fce97733b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b37616d80 con 0x563b3772d420 2024-02-02T15:32:26.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.820+0000 7fce97733b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b37616a20 con 0x563b3772dd40 2024-02-02T15:32:26.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.821+0000 7fce97733b80 10 client.0 _ll_register_callbacks cb 0x563b37618290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:32:26.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.821+0000 7fce7be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b3772dd40 0x563b3773e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:26.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.821+0000 7fce7be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b3772dd40 0x563b3773e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:37348/0 (socket says 172.21.15.1:37348) 2024-02-02T15:32:26.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.821+0000 7fce7be00700 1 -- 172.21.15.1:0/520067165 learned_addr learned my addr 172.21.15.1:0/520067165 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:32:26.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.821+0000 7fce81200700 1 --2- 172.21.15.1:0/520067165 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563b3772d420 0x563b3773dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:26.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.821+0000 7fce81c00700 1 --2- 172.21.15.1:0/520067165 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563b37738bf0 0x563b37737bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:26.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.821+0000 7fce81200700 1 -- 172.21.15.1:0/520067165 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b3772dd40 msgr2=0x563b3773e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:32:26.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.821+0000 7fce81200700 1 --2- 172.21.15.1:0/520067165 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b3772dd40 0x563b3773e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.821+0000 7fce81200700 1 -- 172.21.15.1:0/520067165 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563b37738bf0 msgr2=0x563b37737bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:32:26.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.821+0000 7fce81200700 1 --2- 172.21.15.1:0/520067165 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563b37738bf0 0x563b37737bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.821+0000 7fce81200700 1 -- 172.21.15.1:0/520067165 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce7c1a3f40 con 0x563b3772d420 2024-02-02T15:32:26.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.821+0000 7fce7be00700 1 --2- 172.21.15.1:0/520067165 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b3772dd40 0x563b3773e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:32:26.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.821+0000 7fce81200700 1 --2- 172.21.15.1:0/520067165 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563b3772d420 0x563b3773dcc0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fce6c096f00 tx=0x7fce6c0ca610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe5a1a81a1a0069e server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:32:26.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.821+0000 7fce6aa00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:32:26.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.821+0000 7fce6aa00700 1 -- 172.21.15.1:0/520067165 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6c09e2c0 con 0x563b3772d420 2024-02-02T15:32:26.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.822+0000 7fce6aa00700 1 -- 172.21.15.1:0/520067165 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fce6c09a490 con 0x563b3772d420 2024-02-02T15:32:26.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.822+0000 7fce97733b80 1 -- 172.21.15.1:0/520067165 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563b3771d0f0 con 0x563b3772d420 2024-02-02T15:32:26.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.822+0000 7fce6aa00700 1 -- 172.21.15.1:0/520067165 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce6c09a260 con 0x563b3772d420 2024-02-02T15:32:26.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.822+0000 7fce97733b80 10 client.27092 fetch_fsmap learned FSMap version 35 2024-02-02T15:32:26.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.822+0000 7fce97733b80 1 -- 172.21.15.1:0/520067165 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x563b374a8b60 con 0x563b3772d420 2024-02-02T15:32:26.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.822+0000 7fce6aa00700 1 -- 172.21.15.1:0/520067165 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563b3771d0f0 con 0x563b3772d420 2024-02-02T15:32:26.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.822+0000 7fce6aa00700 1 -- 172.21.15.1:0/520067165 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 35) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fce6c0b8150 con 0x563b3772d420 2024-02-02T15:32:26.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.822+0000 7fce97733b80 10 client.27092 fetch_fsmap finished waiting for FSMap version 35 2024-02-02T15:32:26.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.822+0000 7fce97733b80 10 client.27092 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-02T15:32:26.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.822+0000 7fce97733b80 20 client.27092 populate_metadata read hostname 'smithi001' 2024-02-02T15:32:26.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.822+0000 7fce97733b80 1 --2- 172.21.15.1:0/520067165 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x563b37747cb0 0x563b3774a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:26.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.822+0000 7fce97733b80 4 client.27092 mds_command: new command op to 24529 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:32:26.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.822+0000 7fce97733b80 1 -- 172.21.15.1:0/520067165 --> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563b37673dd0 con 0x563b37747cb0 2024-02-02T15:32:26.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.823+0000 7fce7be00700 1 --2- 172.21.15.1:0/520067165 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x563b37747cb0 0x563b3774a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:26.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.823+0000 7fce7be00700 1 --2- 172.21.15.1:0/520067165 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x563b37747cb0 0x563b3774a0d0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=48b4a16972d0f62c server_cookie=bf30a745a9abfd8a in_seq=0 out_seq=0 2024-02-02T15:32:26.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.823+0000 7fce6aa00700 10 client.27092 ms_handle_connect on v2:172.21.15.142:6838/321675421 2024-02-02T15:32:26.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.824+0000 7fce6aa00700 1 -- 172.21.15.1:0/520067165 <== mds.1 v2:172.21.15.142:6838/321675421 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x563b37673dd0 con 0x563b37747cb0 2024-02-02T15:32:26.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.824+0000 7fce6aa00700 10 client.27092 handle_command_reply: tid=0 2024-02-02T15:32:26.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.824+0000 7fce97733b80 1 client.27092 shutdown 2024-02-02T15:32:26.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.824+0000 7fce97733b80 1 -- 172.21.15.1:0/520067165 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563b3772d420 msgr2=0x563b3773dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:26.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.824+0000 7fce97733b80 1 --2- 172.21.15.1:0/520067165 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563b3772d420 0x563b3773dcc0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fce6c096f00 tx=0x7fce6c0ca610 comp rx=0 tx=0).stop 2024-02-02T15:32:26.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.824+0000 7fce97733b80 1 -- 172.21.15.1:0/520067165 shutdown_connections 2024-02-02T15:32:26.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.824+0000 7fce97733b80 1 --2- 172.21.15.1:0/520067165 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x563b37747cb0 0x563b3774a0d0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.824+0000 7fce97733b80 1 --2- 172.21.15.1:0/520067165 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563b37738bf0 0x563b37737bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.824+0000 7fce97733b80 1 --2- 172.21.15.1:0/520067165 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563b3772d420 0x563b3773dcc0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.824+0000 7fce6aa00700 0 client.27092 ms_handle_reset on v2:172.21.15.142:6838/321675421 2024-02-02T15:32:26.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.824+0000 7fce97733b80 1 --2- 172.21.15.1:0/520067165 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b3772dd40 0x563b3773e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.824+0000 7fce97733b80 1 -- 172.21.15.1:0/520067165 >> 172.21.15.1:0/520067165 conn(0x563b37688db0 msgr2=0x563b37724fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:32:26.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.824+0000 7fce97733b80 1 -- 172.21.15.1:0/520067165 shutdown_connections 2024-02-02T15:32:26.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.824+0000 7fce97733b80 1 -- 172.21.15.1:0/520067165 wait complete. 2024-02-02T15:32:26.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.824+0000 7fce97733b80 20 client.27092 trim_cache size 0 max 16384 2024-02-02T15:32:26.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.838+0000 7fce97733b80 1 Processor -- start 2024-02-02T15:32:26.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.838+0000 7fce97733b80 1 -- start start 2024-02-02T15:32:26.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.838+0000 7fce97733b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b3772d420 0x563b3773d870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:26.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.838+0000 7fce97733b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563b3772dd40 0x563b37737b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:26.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.838+0000 7fce97733b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563b37738bf0 0x563b37734e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:26.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.838+0000 7fce97733b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b374fe6a0 con 0x563b37738bf0 2024-02-02T15:32:26.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.838+0000 7fce97733b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b3774a730 con 0x563b3772dd40 2024-02-02T15:32:26.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.838+0000 7fce97733b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b37736110 con 0x563b3772d420 2024-02-02T15:32:26.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.838+0000 7fce81200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b3772d420 0x563b3773d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:26.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.838+0000 7fce7be00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563b3772dd40 0x563b37737b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:26.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.838+0000 7fce81200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b3772d420 0x563b3773d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:37354/0 (socket says 172.21.15.1:37354) 2024-02-02T15:32:26.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.838+0000 7fce81200700 1 -- 172.21.15.1:0/417488004 learned_addr learned my addr 172.21.15.1:0/417488004 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:32:26.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.838+0000 7fce81c00700 1 --2- 172.21.15.1:0/417488004 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563b37738bf0 0x563b37734e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:26.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.839+0000 7fce7be00700 1 -- 172.21.15.1:0/417488004 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b3772d420 msgr2=0x563b3773d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:32:26.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.839+0000 7fce7be00700 1 --2- 172.21.15.1:0/417488004 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b3772d420 0x563b3773d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.839+0000 7fce7be00700 1 -- 172.21.15.1:0/417488004 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563b37738bf0 msgr2=0x563b37734e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:32:26.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.839+0000 7fce7be00700 1 --2- 172.21.15.1:0/417488004 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563b37738bf0 0x563b37734e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.839+0000 7fce7be00700 1 -- 172.21.15.1:0/417488004 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b377210d0 con 0x563b3772dd40 2024-02-02T15:32:26.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.839+0000 7fce81c00700 1 --2- 172.21.15.1:0/417488004 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563b37738bf0 0x563b37734e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:32:26.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.839+0000 7fce81200700 1 --2- 172.21.15.1:0/417488004 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b3772d420 0x563b3773d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:32:26.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.839+0000 7fce7be00700 1 --2- 172.21.15.1:0/417488004 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563b3772dd40 0x563b37737b60 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x563b37748090 tx=0x7fce64009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9040f030eddaad6 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:32:26.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.839+0000 7fce72a00700 1 -- 172.21.15.1:0/417488004 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce64016030 con 0x563b3772dd40 2024-02-02T15:32:26.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.839+0000 7fce72a00700 1 -- 172.21.15.1:0/417488004 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fce64017040 con 0x563b3772dd40 2024-02-02T15:32:26.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.839+0000 7fce72a00700 1 -- 172.21.15.1:0/417488004 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6401da40 con 0x563b3772dd40 2024-02-02T15:32:26.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.840+0000 7fce97733b80 1 -- 172.21.15.1:0/417488004 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563b3772dd40 msgr2=0x563b37737b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:26.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.840+0000 7fce97733b80 1 --2- 172.21.15.1:0/417488004 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563b3772dd40 0x563b37737b60 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x563b37748090 tx=0x7fce64009260 comp rx=0 tx=0).stop 2024-02-02T15:32:26.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.840+0000 7fce97733b80 1 -- 172.21.15.1:0/417488004 shutdown_connections 2024-02-02T15:32:26.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.840+0000 7fce97733b80 1 --2- 172.21.15.1:0/417488004 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563b37738bf0 0x563b37734e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.840+0000 7fce97733b80 1 --2- 172.21.15.1:0/417488004 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b3772d420 0x563b3773d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.840+0000 7fce97733b80 1 --2- 172.21.15.1:0/417488004 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563b3772dd40 0x563b37737b60 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.840+0000 7fce97733b80 1 -- 172.21.15.1:0/417488004 >> 172.21.15.1:0/417488004 conn(0x563b37688db0 msgr2=0x563b37737450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:32:26.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.840+0000 7fce97733b80 1 -- 172.21.15.1:0/417488004 shutdown_connections 2024-02-02T15:32:26.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.840+0000 7fce97733b80 1 -- 172.21.15.1:0/417488004 wait complete. 2024-02-02T15:32:26.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.840+0000 7fce97733b80 1 -- start start 2024-02-02T15:32:26.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.841+0000 7fce97733b80 1 Processor -- start 2024-02-02T15:32:26.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.841+0000 7fce97733b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563b3772d420 0x563b3778d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:26.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.841+0000 7fce97733b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563b3772dd40 0x563b3778dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:26.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.841+0000 7fce97733b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b37738bf0 0x563b377987a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:26.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.841+0000 7fce97733b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b3778e460 con 0x563b3772dd40 2024-02-02T15:32:26.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.841+0000 7fce97733b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b3778e5d0 con 0x563b3772d420 2024-02-02T15:32:26.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.841+0000 7fce97733b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b37798d60 con 0x563b37738bf0 2024-02-02T15:32:26.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.841+0000 7fce97733b80 10 client.0 _ll_register_callbacks cb 0x563b37618290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:32:26.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.841+0000 7fce81200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563b3772d420 0x563b3778d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:26.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.841+0000 7fce81200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563b3772d420 0x563b3778d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:47990/0 (socket says 172.21.15.1:47990) 2024-02-02T15:32:26.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.841+0000 7fce81200700 1 -- 172.21.15.1:0/2617628162 learned_addr learned my addr 172.21.15.1:0/2617628162 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:32:26.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.841+0000 7fce81c00700 1 --2- 172.21.15.1:0/2617628162 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b37738bf0 0x563b377987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:26.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.841+0000 7fce7be00700 1 --2- 172.21.15.1:0/2617628162 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563b3772dd40 0x563b3778dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:26.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.842+0000 7fce81200700 1 -- 172.21.15.1:0/2617628162 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b37738bf0 msgr2=0x563b377987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:32:26.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.842+0000 7fce81200700 1 --2- 172.21.15.1:0/2617628162 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b37738bf0 0x563b377987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.842+0000 7fce81200700 1 -- 172.21.15.1:0/2617628162 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563b3772dd40 msgr2=0x563b3778dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:32:26.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.842+0000 7fce81200700 1 --2- 172.21.15.1:0/2617628162 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563b3772dd40 0x563b3778dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.842+0000 7fce81200700 1 -- 172.21.15.1:0/2617628162 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563b374a8b60 con 0x563b3772d420 2024-02-02T15:32:26.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.842+0000 7fce81200700 1 --2- 172.21.15.1:0/2617628162 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563b3772d420 0x563b3778d720 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fce6c09a140 tx=0x7fce6c022af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5df0994f46016670 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:32:26.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.842+0000 7fce6aa00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:32:26.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.842+0000 7fce6aa00700 1 -- 172.21.15.1:0/2617628162 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6c0a21e0 con 0x563b3772d420 2024-02-02T15:32:26.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.842+0000 7fce6aa00700 1 -- 172.21.15.1:0/2617628162 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fce6c09ab90 con 0x563b3772d420 2024-02-02T15:32:26.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.842+0000 7fce97733b80 1 -- 172.21.15.1:0/2617628162 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fce6c099fd0 con 0x563b3772d420 2024-02-02T15:32:26.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.842+0000 7fce6aa00700 1 -- 172.21.15.1:0/2617628162 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce6c09a260 con 0x563b3772d420 2024-02-02T15:32:26.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.843+0000 7fce97733b80 10 client.27098 fetch_fsmap learned FSMap version 35 2024-02-02T15:32:26.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.843+0000 7fce97733b80 1 -- 172.21.15.1:0/2617628162 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x7fce64004040 con 0x563b3772d420 2024-02-02T15:32:26.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.843+0000 7fce6aa00700 1 -- 172.21.15.1:0/2617628162 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fce6c099fd0 con 0x563b3772d420 2024-02-02T15:32:26.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.843+0000 7fce6aa00700 1 -- 172.21.15.1:0/2617628162 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 35) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fce6c0b2ba0 con 0x563b3772d420 2024-02-02T15:32:26.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.843+0000 7fce97733b80 10 client.27098 fetch_fsmap finished waiting for FSMap version 35 2024-02-02T15:32:26.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.843+0000 7fce97733b80 10 client.27098 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-02T15:32:26.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.843+0000 7fce97733b80 20 client.27098 populate_metadata read hostname 'smithi001' 2024-02-02T15:32:26.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.843+0000 7fce97733b80 1 --2- 172.21.15.1:0/2617628162 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x563b37799fe0 0x563b3779c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:26.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.843+0000 7fce97733b80 4 client.27098 mds_command: new command op to 24529 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:32:26.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.843+0000 7fce97733b80 1 -- 172.21.15.1:0/2617628162 --> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563b37639f50 con 0x563b37799fe0 2024-02-02T15:32:26.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.843+0000 7fce7be00700 1 --2- 172.21.15.1:0/2617628162 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x563b37799fe0 0x563b3779c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:26.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.844+0000 7fce7be00700 1 --2- 172.21.15.1:0/2617628162 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x563b37799fe0 0x563b3779c400 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bf6af1ba0e9f20f3 server_cookie=67005cebf5a9521e in_seq=0 out_seq=0 2024-02-02T15:32:26.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.844+0000 7fce6aa00700 10 client.27098 ms_handle_connect on v2:172.21.15.142:6838/321675421 2024-02-02T15:32:26.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.844+0000 7fce6aa00700 1 -- 172.21.15.1:0/2617628162 <== mds.1 v2:172.21.15.142:6838/321675421 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563b37639f50 con 0x563b37799fe0 2024-02-02T15:32:26.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.844+0000 7fce6aa00700 10 client.27098 handle_command_reply: tid=0 2024-02-02T15:32:26.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.844+0000 7fce97733b80 1 client.27098 shutdown 2024-02-02T15:32:26.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.844+0000 7fce97733b80 1 -- 172.21.15.1:0/2617628162 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563b3772d420 msgr2=0x563b3778d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:26.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.844+0000 7fce97733b80 1 --2- 172.21.15.1:0/2617628162 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563b3772d420 0x563b3778d720 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fce6c09a140 tx=0x7fce6c022af0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.844+0000 7fce97733b80 1 -- 172.21.15.1:0/2617628162 shutdown_connections 2024-02-02T15:32:26.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.844+0000 7fce97733b80 1 --2- 172.21.15.1:0/2617628162 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x563b37799fe0 0x563b3779c400 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.844+0000 7fce97733b80 1 --2- 172.21.15.1:0/2617628162 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563b37738bf0 0x563b377987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.844+0000 7fce97733b80 1 --2- 172.21.15.1:0/2617628162 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563b3772d420 0x563b3778d720 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.844+0000 7fce6aa00700 0 client.27098 ms_handle_reset on v2:172.21.15.142:6838/321675421 2024-02-02T15:32:26.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.844+0000 7fce97733b80 1 --2- 172.21.15.1:0/2617628162 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563b3772dd40 0x563b3778dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.844+0000 7fce97733b80 1 -- 172.21.15.1:0/2617628162 >> 172.21.15.1:0/2617628162 conn(0x563b37688db0 msgr2=0x563b37736360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:32:26.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.844+0000 7fce97733b80 1 -- 172.21.15.1:0/2617628162 shutdown_connections 2024-02-02T15:32:26.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.844+0000 7fce97733b80 1 -- 172.21.15.1:0/2617628162 wait complete. 2024-02-02T15:32:26.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.844+0000 7fce97733b80 20 client.27098 trim_cache size 0 max 16384 2024-02-02T15:32:26.849 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:32:26.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.845+0000 7fce83c00700 1 -- 172.21.15.1:0/93308962 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fce5c0a0710 msgr2=0x7fce5c0a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:26.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.845+0000 7fce83c00700 1 --2- 172.21.15.1:0/93308962 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fce5c0a0710 0x7fce5c0a2bd0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fce74009ec0 tx=0x7fce7400b040 comp rx=0 tx=0).stop 2024-02-02T15:32:26.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.845+0000 7fce83c00700 1 -- 172.21.15.1:0/93308962 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce7c101000 msgr2=0x7fce7c196f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:26.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.845+0000 7fce83c00700 1 --2- 172.21.15.1:0/93308962 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce7c101000 0x7fce7c196f20 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fce6c005910 tx=0x7fce6c004730 comp rx=0 tx=0).stop 2024-02-02T15:32:26.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.845+0000 7fce83c00700 1 -- 172.21.15.1:0/93308962 shutdown_connections 2024-02-02T15:32:26.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.845+0000 7fce83c00700 1 --2- 172.21.15.1:0/93308962 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fce5c0a0710 0x7fce5c0a2bd0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.845+0000 7fce83c00700 1 --2- 172.21.15.1:0/93308962 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fce7c19b8c0 0x7fce7c19bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.845+0000 7fce83c00700 1 --2- 172.21.15.1:0/93308962 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fce7c101000 0x7fce7c196f20 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.845+0000 7fce83c00700 1 --2- 172.21.15.1:0/93308962 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fce7c106120 0x7fce7c197460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:26.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.845+0000 7fce83c00700 1 -- 172.21.15.1:0/93308962 >> 172.21.15.1:0/93308962 conn(0x7fce7c0fc860 msgr2=0x7fce7c0fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:32:26.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.845+0000 7fce83c00700 1 -- 172.21.15.1:0/93308962 shutdown_connections 2024-02-02T15:32:26.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:26.845+0000 7fce83c00700 1 -- 172.21.15.1:0/93308962 wait complete. 2024-02-02T15:32:26.863 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-02T15:32:27.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.070+0000 7f665fc00700 1 -- 172.21.15.1:0/1090786627 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f66581019a0 msgr2=0x7f66581059e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:27.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.070+0000 7f665fc00700 1 --2- 172.21.15.1:0/1090786627 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f66581019a0 0x7f66581059e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f6640009990 tx=0x7f6640009ca0 comp rx=0 tx=0).stop 2024-02-02T15:32:27.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.070+0000 7f665fc00700 1 -- 172.21.15.1:0/1090786627 shutdown_connections 2024-02-02T15:32:27.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.070+0000 7f665fc00700 1 --2- 172.21.15.1:0/1090786627 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6658106110 0x7f665810dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:27.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.070+0000 7f665fc00700 1 --2- 172.21.15.1:0/1090786627 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6658100ff0 0x7f66581013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:27.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.070+0000 7f665fc00700 1 --2- 172.21.15.1:0/1090786627 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f66581019a0 0x7f66581059e0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:27.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.070+0000 7f665fc00700 1 -- 172.21.15.1:0/1090786627 >> 172.21.15.1:0/1090786627 conn(0x7f66580fc810 msgr2=0x7f66580fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:32:27.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.070+0000 7f665fc00700 1 -- 172.21.15.1:0/1090786627 shutdown_connections 2024-02-02T15:32:27.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.070+0000 7f665fc00700 1 -- 172.21.15.1:0/1090786627 wait complete. 2024-02-02T15:32:27.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.071+0000 7f665fc00700 1 Processor -- start 2024-02-02T15:32:27.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.071+0000 7f665fc00700 1 -- start start 2024-02-02T15:32:27.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.072+0000 7f665fc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6658100ff0 0x7f6658199070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:27.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.072+0000 7f665fc00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6658106110 0x7f66581995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:27.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.072+0000 7f665fc00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f665819da10 0x7f665819de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:27.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.072+0000 7f665fc00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f665819e490 con 0x7f6658106110 2024-02-02T15:32:27.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.072+0000 7f665fc00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f665819e600 con 0x7f665819da10 2024-02-02T15:32:27.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.072+0000 7f665fc00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f665819e7c0 con 0x7f6658100ff0 2024-02-02T15:32:27.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.072+0000 7f665d200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6658100ff0 0x7f6658199070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:27.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.072+0000 7f6657e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6658106110 0x7f66581995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:27.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.072+0000 7f665dc00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f665819da10 0x7f665819de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:27.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.072+0000 7f6657e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6658106110 0x7f66581995b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39394/0 (socket says 172.21.15.1:39394) 2024-02-02T15:32:27.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.072+0000 7f665d200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6658100ff0 0x7f6658199070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:37380/0 (socket says 172.21.15.1:37380) 2024-02-02T15:32:27.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.072+0000 7f6657e00700 1 -- 172.21.15.1:0/920647698 learned_addr learned my addr 172.21.15.1:0/920647698 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:32:27.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.072+0000 7f6657e00700 1 -- 172.21.15.1:0/920647698 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6658100ff0 msgr2=0x7f6658199070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:27.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.072+0000 7f6657e00700 1 --2- 172.21.15.1:0/920647698 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6658100ff0 0x7f6658199070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:27.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.072+0000 7f6657e00700 1 -- 172.21.15.1:0/920647698 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f665819da10 msgr2=0x7f665819de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:27.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.072+0000 7f6657e00700 1 --2- 172.21.15.1:0/920647698 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f665819da10 0x7f665819de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:27.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.072+0000 7f6657e00700 1 -- 172.21.15.1:0/920647698 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6648009520 con 0x7f6658106110 2024-02-02T15:32:27.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.072+0000 7f665dc00700 1 --2- 172.21.15.1:0/920647698 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f665819da10 0x7f665819de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:32:27.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.072+0000 7f665d200700 1 --2- 172.21.15.1:0/920647698 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6658100ff0 0x7f6658199070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:32:27.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.073+0000 7f6657e00700 1 --2- 172.21.15.1:0/920647698 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6658106110 0x7f66581995b0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f6640000c00 tx=0x7f6640004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:32:27.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.073+0000 7f6655600700 1 -- 172.21.15.1:0/920647698 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f664001d070 con 0x7f6658106110 2024-02-02T15:32:27.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.073+0000 7f665fc00700 1 -- 172.21.15.1:0/920647698 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66400095f0 con 0x7f6658106110 2024-02-02T15:32:27.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.073+0000 7f6655600700 1 -- 172.21.15.1:0/920647698 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6640021410 con 0x7f6658106110 2024-02-02T15:32:27.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.073+0000 7f6655600700 1 -- 172.21.15.1:0/920647698 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6640017650 con 0x7f6658106110 2024-02-02T15:32:27.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.073+0000 7f665fc00700 1 -- 172.21.15.1:0/920647698 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66581a6010 con 0x7f6658106110 2024-02-02T15:32:27.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.078+0000 7f66737abb80 1 Processor -- start 2024-02-02T15:32:27.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.078+0000 7f66737abb80 1 -- start start 2024-02-02T15:32:27.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.078+0000 7f6655600700 1 -- 172.21.15.1:0/920647698 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 27) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f6640021580 con 0x7f6658106110 2024-02-02T15:32:27.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.078+0000 7f66737abb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6bf46d010 0x55f6bf46d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:27.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.078+0000 7f66737abb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6bf46d930 0x55f6bf4782a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:27.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.078+0000 7f66737abb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6bf4787e0 0x55f6bf47abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:27.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.078+0000 7f66737abb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6bf3e14c0 con 0x55f6bf4787e0 2024-02-02T15:32:27.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.078+0000 7f66737abb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6bf3c5aa0 con 0x55f6bf46d930 2024-02-02T15:32:27.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.078+0000 7f66737abb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6bf3c3460 con 0x55f6bf46d010 2024-02-02T15:32:27.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.078+0000 7f665d200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6bf46d930 0x55f6bf4782a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:27.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.078+0000 7f665dc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6bf46d010 0x55f6bf46d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:27.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.078+0000 7f665d200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6bf46d930 0x55f6bf4782a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:48000/0 (socket says 172.21.15.1:48000) 2024-02-02T15:32:27.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.078+0000 7f665d200700 1 -- 172.21.15.1:0/49290158 learned_addr learned my addr 172.21.15.1:0/49290158 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:32:27.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.079+0000 7f6657e00700 1 --2- 172.21.15.1:0/49290158 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6bf4787e0 0x55f6bf47abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:27.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.079+0000 7f665d200700 1 -- 172.21.15.1:0/49290158 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6bf46d010 msgr2=0x55f6bf46d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:32:27.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.079+0000 7f665d200700 1 --2- 172.21.15.1:0/49290158 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6bf46d010 0x55f6bf46d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:27.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.079+0000 7f665d200700 1 -- 172.21.15.1:0/49290158 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6bf4787e0 msgr2=0x55f6bf47abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:32:27.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.079+0000 7f665d200700 1 --2- 172.21.15.1:0/49290158 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6bf4787e0 0x55f6bf47abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:27.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.079+0000 7f665d200700 1 -- 172.21.15.1:0/49290158 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f6bf2403b0 con 0x55f6bf46d930 2024-02-02T15:32:27.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.079+0000 7f6657e00700 1 --2- 172.21.15.1:0/49290158 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6bf4787e0 0x55f6bf47abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:32:27.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.079+0000 7f665d200700 1 --2- 172.21.15.1:0/49290158 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6bf46d930 0x55f6bf4782a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f6648011790 tx=0x7f6648011aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c85c1c038cf94185 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:32:27.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.079+0000 7f664e000700 1 -- 172.21.15.1:0/49290158 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f664800f4e0 con 0x55f6bf46d930 2024-02-02T15:32:27.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.079+0000 7f664e000700 1 -- 172.21.15.1:0/49290158 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f664800fb00 con 0x55f6bf46d930 2024-02-02T15:32:27.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.079+0000 7f664e000700 1 -- 172.21.15.1:0/49290158 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6648019b70 con 0x55f6bf46d930 2024-02-02T15:32:27.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.080+0000 7f6655600700 1 --2- 172.21.15.1:0/920647698 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f66380a0690 0x7f66380a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:27.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.080+0000 7f6655600700 1 -- 172.21.15.1:0/920647698 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f66400cd310 con 0x7f6658106110 2024-02-02T15:32:27.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.080+0000 7f665dc00700 1 --2- 172.21.15.1:0/920647698 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f66380a0690 0x7f66380a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:27.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.080+0000 7f66737abb80 1 -- 172.21.15.1:0/49290158 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6bf46d930 msgr2=0x55f6bf4782a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:27.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.080+0000 7f66737abb80 1 --2- 172.21.15.1:0/49290158 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6bf46d930 0x55f6bf4782a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f6648011790 tx=0x7f6648011aa0 comp rx=0 tx=0).stop 2024-02-02T15:32:27.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.080+0000 7f66737abb80 1 -- 172.21.15.1:0/49290158 shutdown_connections 2024-02-02T15:32:27.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.080+0000 7f66737abb80 1 --2- 172.21.15.1:0/49290158 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6bf4787e0 0x55f6bf47abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:27.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.080+0000 7f66737abb80 1 --2- 172.21.15.1:0/49290158 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6bf46d010 0x55f6bf46d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:27.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.080+0000 7f66737abb80 1 --2- 172.21.15.1:0/49290158 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6bf46d930 0x55f6bf4782a0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:27.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.080+0000 7f665dc00700 1 --2- 172.21.15.1:0/920647698 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f66380a0690 0x7f66380a2b50 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6650004090 tx=0x7f665000b040 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:32:27.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.080+0000 7f66737abb80 1 -- 172.21.15.1:0/49290158 >> 172.21.15.1:0/49290158 conn(0x55f6bf3ccdb0 msgr2=0x55f6bf4693b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:32:27.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.081+0000 7f66737abb80 1 -- 172.21.15.1:0/49290158 shutdown_connections 2024-02-02T15:32:27.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.081+0000 7f66737abb80 1 -- 172.21.15.1:0/49290158 wait complete. 2024-02-02T15:32:27.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.081+0000 7f66737abb80 1 -- start start 2024-02-02T15:32:27.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.081+0000 7f66737abb80 1 Processor -- start 2024-02-02T15:32:27.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.082+0000 7f66737abb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6bf46d010 0x55f6bf478120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:27.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.082+0000 7f66737abb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6bf46d930 0x55f6bf475330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:27.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.082+0000 7f66737abb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6bf4787e0 0x55f6bf475890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:27.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.082+0000 7f66737abb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6bf3bed50 con 0x55f6bf4787e0 2024-02-02T15:32:27.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.082+0000 7f66737abb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6bf35ad80 con 0x55f6bf46d010 2024-02-02T15:32:27.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.082+0000 7f66737abb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6bf35aa20 con 0x55f6bf46d930 2024-02-02T15:32:27.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.082+0000 7f66737abb80 10 client.0 _ll_register_callbacks cb 0x55f6bf35c290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:32:27.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.082+0000 7f665d200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6bf46d010 0x55f6bf478120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:27.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.082+0000 7f665dc00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6bf4787e0 0x55f6bf475890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:27.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.082+0000 7f665d200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6bf46d010 0x55f6bf478120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:48008/0 (socket says 172.21.15.1:48008) 2024-02-02T15:32:27.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.082+0000 7f665d200700 1 -- 172.21.15.1:0/207748216 learned_addr learned my addr 172.21.15.1:0/207748216 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:32:27.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.082+0000 7f6657e00700 1 --2- 172.21.15.1:0/207748216 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6bf46d930 0x55f6bf475330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:27.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.082+0000 7f665d200700 1 -- 172.21.15.1:0/207748216 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6bf46d930 msgr2=0x55f6bf475330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:32:27.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.082+0000 7f665d200700 1 --2- 172.21.15.1:0/207748216 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6bf46d930 0x55f6bf475330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:27.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.082+0000 7f665d200700 1 -- 172.21.15.1:0/207748216 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6bf4787e0 msgr2=0x55f6bf475890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:32:27.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.082+0000 7f665d200700 1 --2- 172.21.15.1:0/207748216 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6bf4787e0 0x55f6bf475890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:27.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.082+0000 7f665d200700 1 -- 172.21.15.1:0/207748216 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6648011440 con 0x55f6bf46d010 2024-02-02T15:32:27.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.083+0000 7f665d200700 1 --2- 172.21.15.1:0/207748216 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6bf46d010 0x55f6bf478120 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6648009f20 tx=0x7f664800f560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9e70487f41bb331 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:32:27.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.083+0000 7f6646a00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:32:27.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.083+0000 7f6646a00700 1 -- 172.21.15.1:0/207748216 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6648005040 con 0x55f6bf46d010 2024-02-02T15:32:27.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.083+0000 7f66737abb80 1 -- 172.21.15.1:0/207748216 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f6bf4610f0 con 0x55f6bf46d010 2024-02-02T15:32:27.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.083+0000 7f6646a00700 1 -- 172.21.15.1:0/207748216 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f66480051a0 con 0x55f6bf46d010 2024-02-02T15:32:27.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.083+0000 7f6646a00700 1 -- 172.21.15.1:0/207748216 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6648005360 con 0x55f6bf46d010 2024-02-02T15:32:27.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.083+0000 7f66737abb80 10 client.27110 fetch_fsmap learned FSMap version 35 2024-02-02T15:32:27.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.083+0000 7f66737abb80 1 -- 172.21.15.1:0/207748216 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55f6bf1ecb60 con 0x55f6bf46d010 2024-02-02T15:32:27.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.084+0000 7f6646a00700 1 -- 172.21.15.1:0/207748216 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f6bf4610f0 con 0x55f6bf46d010 2024-02-02T15:32:27.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.084+0000 7f6646a00700 1 -- 172.21.15.1:0/207748216 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 35) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f66480047e0 con 0x55f6bf46d010 2024-02-02T15:32:27.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.084+0000 7f66737abb80 10 client.27110 fetch_fsmap finished waiting for FSMap version 35 2024-02-02T15:32:27.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.084+0000 7f66737abb80 10 client.27110 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:32:27.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.084+0000 7f66737abb80 20 client.27110 populate_metadata read hostname 'smithi001' 2024-02-02T15:32:27.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.084+0000 7f66737abb80 1 --2- 172.21.15.1:0/207748216 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55f6bf487fe0 0x55f6bf48a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:27.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.084+0000 7f66737abb80 4 client.27110 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:32:27.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.084+0000 7f66737abb80 1 -- 172.21.15.1:0/207748216 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f6bf3b7dd0 con 0x55f6bf487fe0 2024-02-02T15:32:27.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.084+0000 7f6657e00700 1 --2- 172.21.15.1:0/207748216 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55f6bf487fe0 0x55f6bf48a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:27.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.085+0000 7f6657e00700 1 --2- 172.21.15.1:0/207748216 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55f6bf487fe0 0x55f6bf48a400 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d667e1392262499 server_cookie=cd35e93df39e in_seq=0 out_seq=0 2024-02-02T15:32:27.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.085+0000 7f6646a00700 10 client.27110 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:32:27.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.086+0000 7f6646a00700 1 -- 172.21.15.1:0/207748216 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55f6bf3b7dd0 con 0x55f6bf487fe0 2024-02-02T15:32:27.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.086+0000 7f6646a00700 10 client.27110 handle_command_reply: tid=0 2024-02-02T15:32:27.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.086+0000 7f66737abb80 1 client.27110 shutdown 2024-02-02T15:32:27.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.086+0000 7f66737abb80 1 -- 172.21.15.1:0/207748216 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6bf46d010 msgr2=0x55f6bf478120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:27.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.086+0000 7f66737abb80 1 --2- 172.21.15.1:0/207748216 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6bf46d010 0x55f6bf478120 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6648009f20 tx=0x7f664800f560 comp rx=0 tx=0).stop 2024-02-02T15:32:27.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.086+0000 7f66737abb80 1 -- 172.21.15.1:0/207748216 shutdown_connections 2024-02-02T15:32:27.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.086+0000 7f66737abb80 1 --2- 172.21.15.1:0/207748216 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55f6bf487fe0 0x55f6bf48a400 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:27.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.086+0000 7f66737abb80 1 --2- 172.21.15.1:0/207748216 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6bf4787e0 0x55f6bf475890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:27.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.086+0000 7f6646a00700 0 client.27110 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:32:27.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.086+0000 7f66737abb80 1 --2- 172.21.15.1:0/207748216 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6bf46d010 0x55f6bf478120 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:27.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.086+0000 7f66737abb80 1 --2- 172.21.15.1:0/207748216 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6bf46d930 0x55f6bf475330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:27.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.086+0000 7f66737abb80 1 -- 172.21.15.1:0/207748216 >> 172.21.15.1:0/207748216 conn(0x55f6bf3ccdb0 msgr2=0x55f6bf468fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:32:27.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.086+0000 7f66737abb80 1 -- 172.21.15.1:0/207748216 shutdown_connections 2024-02-02T15:32:27.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.086+0000 7f66737abb80 1 -- 172.21.15.1:0/207748216 wait complete. 2024-02-02T15:32:27.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.086+0000 7f66737abb80 20 client.27110 trim_cache size 0 max 16384 2024-02-02T15:32:27.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.101+0000 7f66737abb80 1 Processor -- start 2024-02-02T15:32:27.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.101+0000 7f66737abb80 1 -- start start 2024-02-02T15:32:27.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.101+0000 7f66737abb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6bf46d010 0x55f6bf477c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:27.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.101+0000 7f66737abb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6bf46d930 0x55f6bf46f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:27.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.101+0000 7f66737abb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6bf4787e0 0x55f6bf46fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:27.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.101+0000 7f66737abb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6bf2426a0 con 0x55f6bf4787e0 2024-02-02T15:32:27.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.101+0000 7f66737abb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6bf460cc0 con 0x55f6bf46d010 2024-02-02T15:32:27.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.101+0000 7f66737abb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6bf473890 con 0x55f6bf46d930 2024-02-02T15:32:27.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.101+0000 7f6657e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6bf46d930 0x55f6bf46f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:27.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.101+0000 7f665dc00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6bf4787e0 0x55f6bf46fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:27.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.101+0000 7f665d200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6bf46d010 0x55f6bf477c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:27.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.101+0000 7f665d200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6bf46d010 0x55f6bf477c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:48014/0 (socket says 172.21.15.1:48014) 2024-02-02T15:32:27.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.101+0000 7f665d200700 1 -- 172.21.15.1:0/3399164638 learned_addr learned my addr 172.21.15.1:0/3399164638 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:32:27.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.102+0000 7f665d200700 1 -- 172.21.15.1:0/3399164638 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6bf46d930 msgr2=0x55f6bf46f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:32:27.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.102+0000 7f665d200700 1 --2- 172.21.15.1:0/3399164638 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6bf46d930 0x55f6bf46f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:27.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.102+0000 7f665d200700 1 -- 172.21.15.1:0/3399164638 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6bf4787e0 msgr2=0x55f6bf46fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:32:27.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.102+0000 7f665d200700 1 --2- 172.21.15.1:0/3399164638 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6bf4787e0 0x55f6bf46fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:27.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.102+0000 7f665d200700 1 -- 172.21.15.1:0/3399164638 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f6bf384ea0 con 0x55f6bf46d010 2024-02-02T15:32:27.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.102+0000 7f665d200700 1 --2- 172.21.15.1:0/3399164638 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6bf46d010 0x55f6bf477c50 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f664800f010 tx=0x7f6648011aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9cf4bc129b36ac1 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:32:27.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.102+0000 7f664ea00700 1 -- 172.21.15.1:0/3399164638 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6648005240 con 0x55f6bf46d010 2024-02-02T15:32:27.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.102+0000 7f664ea00700 1 -- 172.21.15.1:0/3399164638 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f664800f680 con 0x55f6bf46d010 2024-02-02T15:32:27.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.102+0000 7f664ea00700 1 -- 172.21.15.1:0/3399164638 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6648003680 con 0x55f6bf46d010 2024-02-02T15:32:27.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.103+0000 7f66737abb80 1 -- 172.21.15.1:0/3399164638 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6bf46d010 msgr2=0x55f6bf477c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:27.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.103+0000 7f66737abb80 1 --2- 172.21.15.1:0/3399164638 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6bf46d010 0x55f6bf477c50 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f664800f010 tx=0x7f6648011aa0 comp rx=0 tx=0).stop 2024-02-02T15:32:27.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.103+0000 7f66737abb80 1 -- 172.21.15.1:0/3399164638 shutdown_connections 2024-02-02T15:32:27.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.103+0000 7f66737abb80 1 --2- 172.21.15.1:0/3399164638 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6bf4787e0 0x55f6bf46fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:27.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.103+0000 7f66737abb80 1 --2- 172.21.15.1:0/3399164638 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6bf46d010 0x55f6bf477c50 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:27.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.103+0000 7f66737abb80 1 --2- 172.21.15.1:0/3399164638 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6bf46d930 0x55f6bf46f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:27.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.103+0000 7f66737abb80 1 -- 172.21.15.1:0/3399164638 >> 172.21.15.1:0/3399164638 conn(0x55f6bf3ccdb0 msgr2=0x55f6bf483900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:32:27.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.103+0000 7f66737abb80 1 -- 172.21.15.1:0/3399164638 shutdown_connections 2024-02-02T15:32:27.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.103+0000 7f66737abb80 1 -- 172.21.15.1:0/3399164638 wait complete. 2024-02-02T15:32:27.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.104+0000 7f66737abb80 1 -- start start 2024-02-02T15:32:27.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.104+0000 7f66737abb80 1 Processor -- start 2024-02-02T15:32:27.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.104+0000 7f66737abb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6bf46d010 0x55f6bf47cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:27.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.104+0000 7f66737abb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6bf46d930 0x55f6bf47d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:27.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.104+0000 7f66737abb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6bf4787e0 0x55f6bf4d8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:27.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.104+0000 7f66737abb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6bf47d890 con 0x55f6bf4787e0 2024-02-02T15:32:27.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.104+0000 7f66737abb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6bf47da00 con 0x55f6bf46d010 2024-02-02T15:32:27.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.104+0000 7f66737abb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6bf4d90c0 con 0x55f6bf46d930 2024-02-02T15:32:27.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.104+0000 7f66737abb80 10 client.0 _ll_register_callbacks cb 0x55f6bf35c290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:32:27.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.105+0000 7f665d200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6bf46d010 0x55f6bf47cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:27.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.105+0000 7f6657e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6bf46d930 0x55f6bf47d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:27.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.105+0000 7f6657e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6bf46d930 0x55f6bf47d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:37426/0 (socket says 172.21.15.1:37426) 2024-02-02T15:32:27.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.105+0000 7f665d200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6bf46d010 0x55f6bf47cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:48020/0 (socket says 172.21.15.1:48020) 2024-02-02T15:32:27.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.105+0000 7f6657e00700 1 -- 172.21.15.1:0/2482219777 learned_addr learned my addr 172.21.15.1:0/2482219777 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:32:27.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.105+0000 7f665dc00700 1 --2- 172.21.15.1:0/2482219777 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6bf4787e0 0x55f6bf4d8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:27.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.105+0000 7f6657e00700 1 -- 172.21.15.1:0/2482219777 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6bf46d010 msgr2=0x55f6bf47cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:32:27.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.105+0000 7f6657e00700 1 --2- 172.21.15.1:0/2482219777 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6bf46d010 0x55f6bf47cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:27.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.105+0000 7f6657e00700 1 -- 172.21.15.1:0/2482219777 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6bf4787e0 msgr2=0x55f6bf4d8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:32:27.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.105+0000 7f6657e00700 1 --2- 172.21.15.1:0/2482219777 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6bf4787e0 0x55f6bf4d8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:27.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.105+0000 7f6657e00700 1 -- 172.21.15.1:0/2482219777 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f6bf1ecb60 con 0x55f6bf46d930 2024-02-02T15:32:27.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.105+0000 7f665dc00700 1 --2- 172.21.15.1:0/2482219777 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6bf4787e0 0x55f6bf4d8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:32:27.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.105+0000 7f6657e00700 1 --2- 172.21.15.1:0/2482219777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6bf46d930 0x55f6bf47d090 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f6640004800 tx=0x7f6640009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=902bbf6d0f9c7bef server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:32:27.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.105+0000 7f6646a00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:32:27.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.105+0000 7f6646a00700 1 -- 172.21.15.1:0/2482219777 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f664009a0b0 con 0x55f6bf46d930 2024-02-02T15:32:27.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.105+0000 7f6646a00700 1 -- 172.21.15.1:0/2482219777 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f66400cd150 con 0x55f6bf46d930 2024-02-02T15:32:27.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.105+0000 7f66737abb80 1 -- 172.21.15.1:0/2482219777 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f6bf37df50 con 0x55f6bf46d930 2024-02-02T15:32:27.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.106+0000 7f6646a00700 1 -- 172.21.15.1:0/2482219777 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6640099ed0 con 0x55f6bf46d930 2024-02-02T15:32:27.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.106+0000 7f66737abb80 10 client.27115 fetch_fsmap learned FSMap version 35 2024-02-02T15:32:27.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.106+0000 7f66737abb80 1 -- 172.21.15.1:0/2482219777 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55f6bf244cd0 con 0x55f6bf46d930 2024-02-02T15:32:27.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.106+0000 7f6646a00700 1 -- 172.21.15.1:0/2482219777 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f6bf37df50 con 0x55f6bf46d930 2024-02-02T15:32:27.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.106+0000 7f6646a00700 1 -- 172.21.15.1:0/2482219777 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 35) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f66400afd90 con 0x55f6bf46d930 2024-02-02T15:32:27.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.106+0000 7f66737abb80 10 client.27115 fetch_fsmap finished waiting for FSMap version 35 2024-02-02T15:32:27.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.106+0000 7f66737abb80 10 client.27115 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:32:27.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.106+0000 7f66737abb80 20 client.27115 populate_metadata read hostname 'smithi001' 2024-02-02T15:32:27.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.106+0000 7f66737abb80 1 --2- 172.21.15.1:0/2482219777 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55f6bf4da490 0x55f6bf4dc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:27.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.106+0000 7f66737abb80 4 client.27115 mds_command: new command op to 24557 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:32:27.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.106+0000 7f66737abb80 1 -- 172.21.15.1:0/2482219777 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f6bf20dcf0 con 0x55f6bf4da490 2024-02-02T15:32:27.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.106+0000 7f665d200700 1 --2- 172.21.15.1:0/2482219777 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55f6bf4da490 0x55f6bf4dc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:27.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.107+0000 7f665d200700 1 --2- 172.21.15.1:0/2482219777 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55f6bf4da490 0x55f6bf4dc8b0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a05319fcde546a21 server_cookie=bc28d5cf4939caad in_seq=0 out_seq=0 2024-02-02T15:32:27.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.107+0000 7f6646a00700 10 client.27115 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:32:27.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.107+0000 7f6646a00700 1 -- 172.21.15.1:0/2482219777 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f6bf20dcf0 con 0x55f6bf4da490 2024-02-02T15:32:27.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.107+0000 7f6646a00700 10 client.27115 handle_command_reply: tid=0 2024-02-02T15:32:27.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.107+0000 7f66737abb80 1 client.27115 shutdown 2024-02-02T15:32:27.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.107+0000 7f66737abb80 1 -- 172.21.15.1:0/2482219777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6bf46d930 msgr2=0x55f6bf47d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:27.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.107+0000 7f66737abb80 1 --2- 172.21.15.1:0/2482219777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6bf46d930 0x55f6bf47d090 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f6640004800 tx=0x7f6640009ca0 comp rx=0 tx=0).stop 2024-02-02T15:32:27.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.107+0000 7f66737abb80 1 -- 172.21.15.1:0/2482219777 shutdown_connections 2024-02-02T15:32:27.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.107+0000 7f66737abb80 1 --2- 172.21.15.1:0/2482219777 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55f6bf4da490 0x55f6bf4dc8b0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:27.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.107+0000 7f66737abb80 1 --2- 172.21.15.1:0/2482219777 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6bf4787e0 0x55f6bf4d8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:27.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.107+0000 7f66737abb80 1 --2- 172.21.15.1:0/2482219777 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6bf46d010 0x55f6bf47cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:27.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.107+0000 7f66737abb80 1 --2- 172.21.15.1:0/2482219777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6bf46d930 0x55f6bf47d090 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:27.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.107+0000 7f66737abb80 1 -- 172.21.15.1:0/2482219777 >> 172.21.15.1:0/2482219777 conn(0x55f6bf3ccdb0 msgr2=0x55f6bf474ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:32:27.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.107+0000 7f6646a00700 0 client.27115 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:32:27.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.107+0000 7f66737abb80 1 -- 172.21.15.1:0/2482219777 shutdown_connections 2024-02-02T15:32:27.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.107+0000 7f66737abb80 1 -- 172.21.15.1:0/2482219777 wait complete. 2024-02-02T15:32:27.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.108+0000 7f66737abb80 20 client.27115 trim_cache size 0 max 16384 2024-02-02T15:32:27.113 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:32:27.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.108+0000 7f665fc00700 1 -- 172.21.15.1:0/920647698 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f66380a0690 msgr2=0x7f66380a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:27.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.108+0000 7f665fc00700 1 --2- 172.21.15.1:0/920647698 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f66380a0690 0x7f66380a2b50 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6650004090 tx=0x7f665000b040 comp rx=0 tx=0).stop 2024-02-02T15:32:27.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.108+0000 7f665fc00700 1 -- 172.21.15.1:0/920647698 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6658106110 msgr2=0x7f66581995b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:27.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.108+0000 7f665fc00700 1 --2- 172.21.15.1:0/920647698 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6658106110 0x7f66581995b0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f6640000c00 tx=0x7f6640004790 comp rx=0 tx=0).stop 2024-02-02T15:32:27.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.108+0000 7f665fc00700 1 -- 172.21.15.1:0/920647698 shutdown_connections 2024-02-02T15:32:27.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.108+0000 7f665fc00700 1 --2- 172.21.15.1:0/920647698 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f66380a0690 0x7f66380a2b50 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:27.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.108+0000 7f665fc00700 1 --2- 172.21.15.1:0/920647698 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f665819da10 0x7f665819de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:27.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.108+0000 7f665fc00700 1 --2- 172.21.15.1:0/920647698 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6658100ff0 0x7f6658199070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:27.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.108+0000 7f665fc00700 1 --2- 172.21.15.1:0/920647698 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6658106110 0x7f66581995b0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:27.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.108+0000 7f665fc00700 1 -- 172.21.15.1:0/920647698 >> 172.21.15.1:0/920647698 conn(0x7f66580fc810 msgr2=0x7f66580fd320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:32:27.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.108+0000 7f665fc00700 1 -- 172.21.15.1:0/920647698 shutdown_connections 2024-02-02T15:32:27.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:27.108+0000 7f665fc00700 1 -- 172.21.15.1:0/920647698 wait complete. 2024-02-02T15:32:27.235 INFO:tasks.workunit.client.0.smithi001.stdout: 1 345105 9.12 MB/sec execute 121 sec latency 15.853 ms 2024-02-02T15:32:27.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:32:27 smithi001 ceph-mon[35470]: pgmap v166: 129 pgs: 129 active+clean; 211 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.9 MiB/s wr, 331 op/s 2024-02-02T15:32:27.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:32:27 smithi071 ceph-mon[44707]: pgmap v166: 129 pgs: 129 active+clean; 211 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.9 MiB/s wr, 331 op/s 2024-02-02T15:32:27.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:32:27 smithi142 ceph-mon[45728]: pgmap v166: 129 pgs: 129 active+clean; 211 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.9 MiB/s wr, 331 op/s 2024-02-02T15:32:28.127 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-02T15:32:28.127 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-02T15:32:28.235 INFO:tasks.workunit.client.0.smithi001.stdout: 1 346861 9.12 MB/sec execute 122 sec latency 25.134 ms 2024-02-02T15:32:28.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.342+0000 7fb06ea00700 1 -- 172.21.15.1:0/3791499736 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb068101ca0 msgr2=0x7fb068102080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:28.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.342+0000 7fb06ea00700 1 --2- 172.21.15.1:0/3791499736 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb068101ca0 0x7fb068102080 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fb058009990 tx=0x7fb058009ca0 comp rx=0 tx=0).stop 2024-02-02T15:32:28.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.342+0000 7fb06ea00700 1 -- 172.21.15.1:0/3791499736 shutdown_connections 2024-02-02T15:32:28.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.342+0000 7fb06ea00700 1 --2- 172.21.15.1:0/3791499736 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb06810b0c0 0x7fb06810d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:28.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.342+0000 7fb06ea00700 1 --2- 172.21.15.1:0/3791499736 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb068101ca0 0x7fb068102080 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:28.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.342+0000 7fb06ea00700 1 --2- 172.21.15.1:0/3791499736 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb0681025c0 0x7fb06810aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:28.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.342+0000 7fb06ea00700 1 -- 172.21.15.1:0/3791499736 >> 172.21.15.1:0/3791499736 conn(0x7fb0680fb480 msgr2=0x7fb0680fd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:32:28.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.342+0000 7fb06ea00700 1 -- 172.21.15.1:0/3791499736 shutdown_connections 2024-02-02T15:32:28.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.342+0000 7fb06ea00700 1 -- 172.21.15.1:0/3791499736 wait complete. 2024-02-02T15:32:28.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.344+0000 7fb06ea00700 1 Processor -- start 2024-02-02T15:32:28.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.344+0000 7fb06ea00700 1 -- start start 2024-02-02T15:32:28.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.344+0000 7fb06ea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb068101ca0 0x7fb06819ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:28.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.344+0000 7fb06ea00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0681025c0 0x7fb06819cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:28.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.344+0000 7fb06ea00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb06810b0c0 0x7fb068196ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:28.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.344+0000 7fb06ea00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb068197120 con 0x7fb06810b0c0 2024-02-02T15:32:28.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.344+0000 7fb06ea00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb068197290 con 0x7fb0681025c0 2024-02-02T15:32:28.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.344+0000 7fb06ea00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb068197450 con 0x7fb068101ca0 2024-02-02T15:32:28.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.345+0000 7fb06d600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb068101ca0 0x7fb06819ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:28.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.345+0000 7fb06d600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb068101ca0 0x7fb06819ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:37444/0 (socket says 172.21.15.1:37444) 2024-02-02T15:32:28.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.345+0000 7fb06d600700 1 -- 172.21.15.1:0/191193671 learned_addr learned my addr 172.21.15.1:0/191193671 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:32:28.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.345+0000 7fb06cc00700 1 --2- 172.21.15.1:0/191193671 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0681025c0 0x7fb06819cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:28.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.345+0000 7fb06e000700 1 --2- 172.21.15.1:0/191193671 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb06810b0c0 0x7fb068196ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:28.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.345+0000 7fb06d600700 1 -- 172.21.15.1:0/191193671 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0681025c0 msgr2=0x7fb06819cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:28.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.345+0000 7fb06d600700 1 --2- 172.21.15.1:0/191193671 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0681025c0 0x7fb06819cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:28.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.345+0000 7fb06d600700 1 -- 172.21.15.1:0/191193671 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb06810b0c0 msgr2=0x7fb068196ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:28.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.345+0000 7fb06d600700 1 --2- 172.21.15.1:0/191193671 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb06810b0c0 0x7fb068196ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:28.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.345+0000 7fb06d600700 1 -- 172.21.15.1:0/191193671 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0580095f0 con 0x7fb068101ca0 2024-02-02T15:32:28.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.345+0000 7fb06e000700 1 --2- 172.21.15.1:0/191193671 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb06810b0c0 0x7fb068196ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:32:28.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.345+0000 7fb06d600700 1 --2- 172.21.15.1:0/191193671 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb068101ca0 0x7fb06819ca40 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fb058005910 tx=0x7fb058005720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:32:28.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.346+0000 7fb066000700 1 -- 172.21.15.1:0/191193671 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb05801d070 con 0x7fb068101ca0 2024-02-02T15:32:28.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.346+0000 7fb066000700 1 -- 172.21.15.1:0/191193671 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb058022470 con 0x7fb068101ca0 2024-02-02T15:32:28.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.346+0000 7fb066000700 1 -- 172.21.15.1:0/191193671 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb058017650 con 0x7fb068101ca0 2024-02-02T15:32:28.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.346+0000 7fb06ea00700 1 -- 172.21.15.1:0/191193671 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0681976d0 con 0x7fb068101ca0 2024-02-02T15:32:28.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.346+0000 7fb06ea00700 1 -- 172.21.15.1:0/191193671 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0681a3a30 con 0x7fb068101ca0 2024-02-02T15:32:28.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.349+0000 7fb066000700 1 -- 172.21.15.1:0/191193671 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 27) v1 ==== 135200+0+0 (secure 0 0 0) 0x7fb0580225e0 con 0x7fb068101ca0 2024-02-02T15:32:28.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.351+0000 7fb083b83b80 1 Processor -- start 2024-02-02T15:32:28.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.351+0000 7fb066000700 1 --2- 172.21.15.1:0/191193671 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fb0480a0710 0x7fb0480a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:28.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.351+0000 7fb083b83b80 1 -- start start 2024-02-02T15:32:28.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.352+0000 7fb066000700 1 -- 172.21.15.1:0/191193671 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fb0580cd470 con 0x7fb068101ca0 2024-02-02T15:32:28.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.352+0000 7fb06e000700 1 --2- 172.21.15.1:0/191193671 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fb0480a0710 0x7fb0480a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:28.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.352+0000 7fb083b83b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6b9a83270 0x55e6b9a83650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:28.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.352+0000 7fb083b83b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e6b9a83b90 0x55e6b9a8e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:28.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.352+0000 7fb083b83b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e6b9a8ea70 0x55e6b9a90e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:28.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.352+0000 7fb083b83b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6b99f7540 con 0x55e6b9a8ea70 2024-02-02T15:32:28.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.352+0000 7fb083b83b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6b99dbb20 con 0x55e6b9a83b90 2024-02-02T15:32:28.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.352+0000 7fb083b83b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6b99d94e0 con 0x55e6b9a83270 2024-02-02T15:32:28.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.352+0000 7fb06d600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6b9a83270 0x55e6b9a83650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:28.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.352+0000 7fb06cc00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e6b9a83b90 0x55e6b9a8e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:28.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.352+0000 7fb06d600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6b9a83270 0x55e6b9a83650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:37458/0 (socket says 172.21.15.1:37458) 2024-02-02T15:32:28.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.352+0000 7fb06d600700 1 -- 172.21.15.1:0/2073413101 learned_addr learned my addr 172.21.15.1:0/2073413101 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:32:28.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.352+0000 7fb06e000700 1 --2- 172.21.15.1:0/2073413101 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e6b9a8ea70 0x55e6b9a90e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:28.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.353+0000 7fb06d600700 1 -- 172.21.15.1:0/2073413101 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e6b9a83b90 msgr2=0x55e6b9a8e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:32:28.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.353+0000 7fb06d600700 1 --2- 172.21.15.1:0/2073413101 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e6b9a83b90 0x55e6b9a8e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:28.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.353+0000 7fb06d600700 1 -- 172.21.15.1:0/2073413101 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e6b9a8ea70 msgr2=0x55e6b9a90e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:32:28.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.353+0000 7fb06d600700 1 --2- 172.21.15.1:0/2073413101 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e6b9a8ea70 0x55e6b9a90e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:28.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.353+0000 7fb06d600700 1 -- 172.21.15.1:0/2073413101 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e6b9856430 con 0x55e6b9a83270 2024-02-02T15:32:28.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.353+0000 7fb06cc00700 1 --2- 172.21.15.1:0/2073413101 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e6b9a83b90 0x55e6b9a8e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:32:28.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.353+0000 7fb06d600700 1 --2- 172.21.15.1:0/2073413101 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6b9a83270 0x55e6b9a83650 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fb058005f50 tx=0x7fb058009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e20b01d17920750f server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:32:28.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.353+0000 7fb06e000700 1 --2- 172.21.15.1:0/191193671 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fb0480a0710 0x7fb0480a2bd0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb060009fd0 tx=0x7fb060007b00 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:32:28.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.353+0000 7fb05e600700 1 -- 172.21.15.1:0/2073413101 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb058099fd0 con 0x55e6b9a83270 2024-02-02T15:32:28.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.353+0000 7fb05e600700 1 -- 172.21.15.1:0/2073413101 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb0580b0020 con 0x55e6b9a83270 2024-02-02T15:32:28.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.353+0000 7fb05e600700 1 -- 172.21.15.1:0/2073413101 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0580228c0 con 0x55e6b9a83270 2024-02-02T15:32:28.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.354+0000 7fb083b83b80 1 -- 172.21.15.1:0/2073413101 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6b9a83270 msgr2=0x55e6b9a83650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:28.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.354+0000 7fb083b83b80 1 --2- 172.21.15.1:0/2073413101 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6b9a83270 0x55e6b9a83650 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fb058005f50 tx=0x7fb058009ca0 comp rx=0 tx=0).stop 2024-02-02T15:32:28.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.354+0000 7fb083b83b80 1 -- 172.21.15.1:0/2073413101 shutdown_connections 2024-02-02T15:32:28.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.354+0000 7fb083b83b80 1 --2- 172.21.15.1:0/2073413101 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e6b9a8ea70 0x55e6b9a90e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:28.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.354+0000 7fb083b83b80 1 --2- 172.21.15.1:0/2073413101 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6b9a83270 0x55e6b9a83650 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:28.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.354+0000 7fb083b83b80 1 --2- 172.21.15.1:0/2073413101 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e6b9a83b90 0x55e6b9a8e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:28.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.354+0000 7fb083b83b80 1 -- 172.21.15.1:0/2073413101 >> 172.21.15.1:0/2073413101 conn(0x55e6b99e2e30 msgr2=0x55e6b9a7f640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:32:28.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.354+0000 7fb083b83b80 1 -- 172.21.15.1:0/2073413101 shutdown_connections 2024-02-02T15:32:28.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.354+0000 7fb083b83b80 1 -- 172.21.15.1:0/2073413101 wait complete. 2024-02-02T15:32:28.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.355+0000 7fb083b83b80 1 -- start start 2024-02-02T15:32:28.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.355+0000 7fb083b83b80 1 Processor -- start 2024-02-02T15:32:28.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.356+0000 7fb083b83b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e6b9a83270 0x55e6b9a8e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:28.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.356+0000 7fb083b83b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e6b9a83b90 0x55e6b9a8b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:28.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.356+0000 7fb083b83b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6b9a8ea70 0x55e6b9a8bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:28.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.356+0000 7fb083b83b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6b99d4dd0 con 0x55e6b9a83b90 2024-02-02T15:32:28.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.356+0000 7fb083b83b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6b9970e00 con 0x55e6b9a83270 2024-02-02T15:32:28.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.356+0000 7fb083b83b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6b9970aa0 con 0x55e6b9a8ea70 2024-02-02T15:32:28.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.356+0000 7fb083b83b80 10 client.0 _ll_register_callbacks cb 0x55e6b9972310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:32:28.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.356+0000 7fb06d600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e6b9a83270 0x55e6b9a8e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:28.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.356+0000 7fb06d600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e6b9a83270 0x55e6b9a8e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:48078/0 (socket says 172.21.15.1:48078) 2024-02-02T15:32:28.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.356+0000 7fb06d600700 1 -- 172.21.15.1:0/1809170026 learned_addr learned my addr 172.21.15.1:0/1809170026 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:32:28.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.356+0000 7fb06e000700 1 --2- 172.21.15.1:0/1809170026 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6b9a8ea70 0x55e6b9a8bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:28.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.356+0000 7fb06cc00700 1 --2- 172.21.15.1:0/1809170026 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e6b9a83b90 0x55e6b9a8b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:28.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.356+0000 7fb06d600700 1 -- 172.21.15.1:0/1809170026 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6b9a8ea70 msgr2=0x55e6b9a8bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:32:28.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.356+0000 7fb06d600700 1 --2- 172.21.15.1:0/1809170026 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6b9a8ea70 0x55e6b9a8bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:28.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.356+0000 7fb06d600700 1 -- 172.21.15.1:0/1809170026 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e6b9a83b90 msgr2=0x55e6b9a8b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:32:28.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.356+0000 7fb06d600700 1 --2- 172.21.15.1:0/1809170026 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e6b9a83b90 0x55e6b9a8b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:28.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.356+0000 7fb06d600700 1 -- 172.21.15.1:0/1809170026 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0500098d0 con 0x55e6b9a83270 2024-02-02T15:32:28.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.357+0000 7fb06d600700 1 --2- 172.21.15.1:0/1809170026 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e6b9a83270 0x55e6b9a8e360 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fb05800f740 tx=0x7fb058022c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78b3093c017c2365 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:32:28.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.357+0000 7fb056a00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:32:28.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.357+0000 7fb056a00700 1 -- 172.21.15.1:0/1809170026 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb058099fd0 con 0x55e6b9a83270 2024-02-02T15:32:28.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.357+0000 7fb056a00700 1 -- 172.21.15.1:0/1809170026 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fb0580b1070 con 0x55e6b9a83270 2024-02-02T15:32:28.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.357+0000 7fb083b83b80 1 -- 172.21.15.1:0/1809170026 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e6b9a773c0 con 0x55e6b9a83270 2024-02-02T15:32:28.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.357+0000 7fb056a00700 1 -- 172.21.15.1:0/1809170026 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb05809a670 con 0x55e6b9a83270 2024-02-02T15:32:28.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.357+0000 7fb083b83b80 10 client.27128 fetch_fsmap learned FSMap version 35 2024-02-02T15:32:28.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.357+0000 7fb083b83b80 1 -- 172.21.15.1:0/1809170026 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x7fb0681a3a30 con 0x55e6b9a83270 2024-02-02T15:32:28.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.357+0000 7fb056a00700 1 -- 172.21.15.1:0/1809170026 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e6b9a773c0 con 0x55e6b9a83270 2024-02-02T15:32:28.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.357+0000 7fb056a00700 1 -- 172.21.15.1:0/1809170026 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 35) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb05809a960 con 0x55e6b9a83270 2024-02-02T15:32:28.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.358+0000 7fb083b83b80 10 client.27128 fetch_fsmap finished waiting for FSMap version 35 2024-02-02T15:32:28.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.358+0000 7fb083b83b80 10 client.27128 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:32:28.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.358+0000 7fb083b83b80 20 client.27128 populate_metadata read hostname 'smithi001' 2024-02-02T15:32:28.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.358+0000 7fb083b83b80 1 --2- 172.21.15.1:0/1809170026 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55e6b9aa2110 0x55e6b9aa4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:28.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.358+0000 7fb083b83b80 4 client.27128 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:32:28.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.358+0000 7fb083b83b80 1 -- 172.21.15.1:0/1809170026 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e6b99cde50 con 0x55e6b9aa2110 2024-02-02T15:32:28.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.358+0000 7fb06cc00700 1 --2- 172.21.15.1:0/1809170026 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55e6b9aa2110 0x55e6b9aa4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:28.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.358+0000 7fb06cc00700 1 --2- 172.21.15.1:0/1809170026 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55e6b9aa2110 0x55e6b9aa4530 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbbe8a40c5d70e27 server_cookie=e6a47b17d68c5a02 in_seq=0 out_seq=0 2024-02-02T15:32:28.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.358+0000 7fb056a00700 10 client.27128 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:32:28.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.359+0000 7fb056a00700 1 -- 172.21.15.1:0/1809170026 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55e6b99cde50 con 0x55e6b9aa2110 2024-02-02T15:32:28.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.359+0000 7fb056a00700 10 client.27128 handle_command_reply: tid=0 2024-02-02T15:32:28.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.359+0000 7fb083b83b80 1 client.27128 shutdown 2024-02-02T15:32:28.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.359+0000 7fb083b83b80 1 -- 172.21.15.1:0/1809170026 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e6b9a83270 msgr2=0x55e6b9a8e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:28.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.359+0000 7fb083b83b80 1 --2- 172.21.15.1:0/1809170026 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e6b9a83270 0x55e6b9a8e360 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fb05800f740 tx=0x7fb058022c90 comp rx=0 tx=0).stop 2024-02-02T15:32:28.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.359+0000 7fb083b83b80 1 -- 172.21.15.1:0/1809170026 shutdown_connections 2024-02-02T15:32:28.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.359+0000 7fb083b83b80 1 --2- 172.21.15.1:0/1809170026 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55e6b9aa2110 0x55e6b9aa4530 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:28.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.359+0000 7fb083b83b80 1 --2- 172.21.15.1:0/1809170026 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6b9a8ea70 0x55e6b9a8bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:28.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.360+0000 7fb056a00700 0 client.27128 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:32:28.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.359+0000 7fb083b83b80 1 --2- 172.21.15.1:0/1809170026 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e6b9a83270 0x55e6b9a8e360 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:28.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.360+0000 7fb083b83b80 1 --2- 172.21.15.1:0/1809170026 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e6b9a83b90 0x55e6b9a8b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:28.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.360+0000 7fb083b83b80 1 -- 172.21.15.1:0/1809170026 >> 172.21.15.1:0/1809170026 conn(0x55e6b99e2e30 msgr2=0x55e6b9a7f170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:32:28.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.360+0000 7fb083b83b80 1 -- 172.21.15.1:0/1809170026 shutdown_connections 2024-02-02T15:32:28.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.360+0000 7fb083b83b80 1 -- 172.21.15.1:0/1809170026 wait complete. 2024-02-02T15:32:28.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.360+0000 7fb083b83b80 20 client.27128 trim_cache size 0 max 16384 2024-02-02T15:32:28.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.378+0000 7fb083b83b80 1 Processor -- start 2024-02-02T15:32:28.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.378+0000 7fb083b83b80 1 -- start start 2024-02-02T15:32:28.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.378+0000 7fb083b83b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6b9a83270 0x55e6b9a85a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:28.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.378+0000 7fb083b83b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e6b9a83b90 0x55e6b9a8abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:28.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.378+0000 7fb083b83b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e6b9a8ea70 0x55e6b9a8fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:28.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.378+0000 7fb083b83b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6b9858720 con 0x55e6b9a83b90 2024-02-02T15:32:28.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.378+0000 7fb083b83b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6b9a84d00 con 0x55e6b9a8ea70 2024-02-02T15:32:28.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.378+0000 7fb083b83b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6b9a844e0 con 0x55e6b9a83270 2024-02-02T15:32:28.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.378+0000 7fb06cc00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e6b9a83b90 0x55e6b9a8abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:28.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.378+0000 7fb06cc00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e6b9a83b90 0x55e6b9a8abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39480/0 (socket says 172.21.15.1:39480) 2024-02-02T15:32:28.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.378+0000 7fb06cc00700 1 -- 172.21.15.1:0/1793037774 learned_addr learned my addr 172.21.15.1:0/1793037774 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:32:28.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.378+0000 7fb06e000700 1 --2- 172.21.15.1:0/1793037774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e6b9a8ea70 0x55e6b9a8fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:28.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.378+0000 7fb06d600700 1 --2- 172.21.15.1:0/1793037774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6b9a83270 0x55e6b9a85a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:28.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.379+0000 7fb06cc00700 1 -- 172.21.15.1:0/1793037774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6b9a83270 msgr2=0x55e6b9a85a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:32:28.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.379+0000 7fb06cc00700 1 --2- 172.21.15.1:0/1793037774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6b9a83270 0x55e6b9a85a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:28.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.379+0000 7fb06cc00700 1 -- 172.21.15.1:0/1793037774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e6b9a8ea70 msgr2=0x55e6b9a8fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:32:28.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.379+0000 7fb06cc00700 1 --2- 172.21.15.1:0/1793037774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e6b9a8ea70 0x55e6b9a8fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:28.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.379+0000 7fb06cc00700 1 -- 172.21.15.1:0/1793037774 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e6b9802be0 con 0x55e6b9a83b90 2024-02-02T15:32:28.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.379+0000 7fb06cc00700 1 --2- 172.21.15.1:0/1793037774 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e6b9a83b90 0x55e6b9a8abc0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fb06810c5f0 tx=0x7fb050009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1962398b99d76083 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:32:28.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.379+0000 7fb05f000700 1 -- 172.21.15.1:0/1793037774 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb050017040 con 0x55e6b9a83b90 2024-02-02T15:32:28.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.379+0000 7fb05f000700 1 -- 172.21.15.1:0/1793037774 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb050018040 con 0x55e6b9a83b90 2024-02-02T15:32:28.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.379+0000 7fb05f000700 1 -- 172.21.15.1:0/1793037774 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb050010690 con 0x55e6b9a83b90 2024-02-02T15:32:28.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.380+0000 7fb083b83b80 1 -- 172.21.15.1:0/1793037774 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e6b9a83b90 msgr2=0x55e6b9a8abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:28.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.380+0000 7fb083b83b80 1 --2- 172.21.15.1:0/1793037774 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e6b9a83b90 0x55e6b9a8abc0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fb06810c5f0 tx=0x7fb050009260 comp rx=0 tx=0).stop 2024-02-02T15:32:28.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.380+0000 7fb083b83b80 1 -- 172.21.15.1:0/1793037774 shutdown_connections 2024-02-02T15:32:28.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.380+0000 7fb083b83b80 1 --2- 172.21.15.1:0/1793037774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e6b9a8ea70 0x55e6b9a8fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:28.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.380+0000 7fb083b83b80 1 --2- 172.21.15.1:0/1793037774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6b9a83270 0x55e6b9a85a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:28.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.380+0000 7fb083b83b80 1 --2- 172.21.15.1:0/1793037774 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e6b9a83b90 0x55e6b9a8abc0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:28.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.380+0000 7fb083b83b80 1 -- 172.21.15.1:0/1793037774 >> 172.21.15.1:0/1793037774 conn(0x55e6b99e2e30 msgr2=0x55e6b9a85380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:32:28.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.380+0000 7fb083b83b80 1 -- 172.21.15.1:0/1793037774 shutdown_connections 2024-02-02T15:32:28.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.380+0000 7fb083b83b80 1 -- 172.21.15.1:0/1793037774 wait complete. 2024-02-02T15:32:28.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.381+0000 7fb083b83b80 1 -- start start 2024-02-02T15:32:28.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.381+0000 7fb083b83b80 1 Processor -- start 2024-02-02T15:32:28.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.382+0000 7fb083b83b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6b9a83270 0x55e6b9aedfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:28.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.382+0000 7fb083b83b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e6b9a83b90 0x55e6b9aee4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:28.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.382+0000 7fb083b83b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e6b9a8ea70 0x55e6b9a964b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:28.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.382+0000 7fb083b83b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6b9a96a80 con 0x55e6b9a83b90 2024-02-02T15:32:28.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.382+0000 7fb083b83b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6b9a96bf0 con 0x55e6b9a8ea70 2024-02-02T15:32:28.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.382+0000 7fb083b83b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6b9a96db0 con 0x55e6b9a83270 2024-02-02T15:32:28.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.382+0000 7fb083b83b80 10 client.0 _ll_register_callbacks cb 0x55e6b9972310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:32:28.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.382+0000 7fb06d600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6b9a83270 0x55e6b9aedfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:28.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.382+0000 7fb06cc00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e6b9a83b90 0x55e6b9aee4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:28.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.382+0000 7fb06d600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6b9a83270 0x55e6b9aedfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:37492/0 (socket says 172.21.15.1:37492) 2024-02-02T15:32:28.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.382+0000 7fb06d600700 1 -- 172.21.15.1:0/1503828606 learned_addr learned my addr 172.21.15.1:0/1503828606 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:32:28.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.382+0000 7fb06e000700 1 --2- 172.21.15.1:0/1503828606 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e6b9a8ea70 0x55e6b9a964b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:28.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.382+0000 7fb06d600700 1 -- 172.21.15.1:0/1503828606 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e6b9a8ea70 msgr2=0x55e6b9a964b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:32:28.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.382+0000 7fb06d600700 1 --2- 172.21.15.1:0/1503828606 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e6b9a8ea70 0x55e6b9a964b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:28.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.382+0000 7fb06d600700 1 -- 172.21.15.1:0/1503828606 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e6b9a83b90 msgr2=0x55e6b9aee4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:32:28.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.382+0000 7fb06d600700 1 --2- 172.21.15.1:0/1503828606 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e6b9a83b90 0x55e6b9aee4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:28.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.382+0000 7fb06d600700 1 -- 172.21.15.1:0/1503828606 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb05001f870 con 0x55e6b9a83270 2024-02-02T15:32:28.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.383+0000 7fb06d600700 1 --2- 172.21.15.1:0/1503828606 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6b9a83270 0x55e6b9aedfb0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb058096b70 tx=0x7fb058022af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8cdeff81d537bd9f server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:32:28.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.383+0000 7fb056a00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:32:28.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.383+0000 7fb056a00700 1 -- 172.21.15.1:0/1503828606 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb05800fd00 con 0x55e6b9a83270 2024-02-02T15:32:28.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.383+0000 7fb056a00700 1 -- 172.21.15.1:0/1503828606 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fb0580b1070 con 0x55e6b9a83270 2024-02-02T15:32:28.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.383+0000 7fb083b83b80 1 -- 172.21.15.1:0/1503828606 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb0580b0020 con 0x55e6b9a83270 2024-02-02T15:32:28.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.383+0000 7fb056a00700 1 -- 172.21.15.1:0/1503828606 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb05800fe60 con 0x55e6b9a83270 2024-02-02T15:32:28.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.383+0000 7fb083b83b80 10 client.27133 fetch_fsmap learned FSMap version 35 2024-02-02T15:32:28.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.383+0000 7fb083b83b80 1 -- 172.21.15.1:0/1503828606 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55e6b999af20 con 0x55e6b9a83270 2024-02-02T15:32:28.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.383+0000 7fb056a00700 1 -- 172.21.15.1:0/1503828606 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb0580b0020 con 0x55e6b9a83270 2024-02-02T15:32:28.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.383+0000 7fb056a00700 1 -- 172.21.15.1:0/1503828606 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 35) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb0580cd0a0 con 0x55e6b9a83270 2024-02-02T15:32:28.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.383+0000 7fb083b83b80 10 client.27133 fetch_fsmap finished waiting for FSMap version 35 2024-02-02T15:32:28.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.383+0000 7fb083b83b80 10 client.27133 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:32:28.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.383+0000 7fb083b83b80 20 client.27133 populate_metadata read hostname 'smithi001' 2024-02-02T15:32:28.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.383+0000 7fb083b83b80 1 --2- 172.21.15.1:0/1503828606 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55e6b9aa2110 0x55e6b9af7920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:28.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.383+0000 7fb083b83b80 4 client.27133 mds_command: new command op to 24557 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-02T15:32:28.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.383+0000 7fb083b83b80 1 -- 172.21.15.1:0/1503828606 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55e6b9993fd0 con 0x55e6b9aa2110 2024-02-02T15:32:28.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.384+0000 7fb06cc00700 1 --2- 172.21.15.1:0/1503828606 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55e6b9aa2110 0x55e6b9af7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:28.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.384+0000 7fb06cc00700 1 --2- 172.21.15.1:0/1503828606 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55e6b9aa2110 0x55e6b9af7920 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a89505eefd1fbe33 server_cookie=72f636bf39319006 in_seq=0 out_seq=0 2024-02-02T15:32:28.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.384+0000 7fb056a00700 10 client.27133 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:32:28.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.385+0000 7fb056a00700 1 -- 172.21.15.1:0/1503828606 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55e6b9993fd0 con 0x55e6b9aa2110 2024-02-02T15:32:28.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.385+0000 7fb056a00700 10 client.27133 handle_command_reply: tid=0 2024-02-02T15:32:28.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.385+0000 7fb083b83b80 1 client.27133 shutdown 2024-02-02T15:32:28.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.385+0000 7fb083b83b80 1 -- 172.21.15.1:0/1503828606 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6b9a83270 msgr2=0x55e6b9aedfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:28.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.386+0000 7fb083b83b80 1 --2- 172.21.15.1:0/1503828606 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6b9a83270 0x55e6b9aedfb0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fb058096b70 tx=0x7fb058022af0 comp rx=0 tx=0).stop 2024-02-02T15:32:28.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.386+0000 7fb083b83b80 1 -- 172.21.15.1:0/1503828606 shutdown_connections 2024-02-02T15:32:28.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.386+0000 7fb083b83b80 1 --2- 172.21.15.1:0/1503828606 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55e6b9aa2110 0x55e6b9af7920 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:28.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.386+0000 7fb083b83b80 1 --2- 172.21.15.1:0/1503828606 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e6b9a8ea70 0x55e6b9a964b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:28.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.386+0000 7fb083b83b80 1 --2- 172.21.15.1:0/1503828606 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6b9a83270 0x55e6b9aedfb0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:28.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.386+0000 7fb056a00700 0 client.27133 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:32:28.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.386+0000 7fb083b83b80 1 --2- 172.21.15.1:0/1503828606 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e6b9a83b90 0x55e6b9aee4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:28.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.386+0000 7fb083b83b80 1 -- 172.21.15.1:0/1503828606 >> 172.21.15.1:0/1503828606 conn(0x55e6b99e2e30 msgr2=0x55e6b9a85c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:32:28.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.386+0000 7fb083b83b80 1 -- 172.21.15.1:0/1503828606 shutdown_connections 2024-02-02T15:32:28.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.386+0000 7fb083b83b80 1 -- 172.21.15.1:0/1503828606 wait complete. 2024-02-02T15:32:28.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.386+0000 7fb083b83b80 20 client.27133 trim_cache size 0 max 16384 2024-02-02T15:32:28.390 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:32:28.390 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2024-02-02T15:32:28.390 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "fc847698-fb77-4bda-ada9-387389e573de", 2024-02-02T15:32:28.390 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2024-02-02T15:32:28.390 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:32:28.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.387+0000 7fb06ea00700 1 -- 172.21.15.1:0/191193671 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fb0480a0710 msgr2=0x7fb0480a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:28.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.387+0000 7fb06ea00700 1 --2- 172.21.15.1:0/191193671 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fb0480a0710 0x7fb0480a2bd0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb060009fd0 tx=0x7fb060007b00 comp rx=0 tx=0).stop 2024-02-02T15:32:28.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.387+0000 7fb06ea00700 1 -- 172.21.15.1:0/191193671 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb068101ca0 msgr2=0x7fb06819ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:28.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.387+0000 7fb06ea00700 1 --2- 172.21.15.1:0/191193671 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb068101ca0 0x7fb06819ca40 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fb058005910 tx=0x7fb058005720 comp rx=0 tx=0).stop 2024-02-02T15:32:28.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.387+0000 7fb06ea00700 1 -- 172.21.15.1:0/191193671 shutdown_connections 2024-02-02T15:32:28.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.387+0000 7fb06ea00700 1 --2- 172.21.15.1:0/191193671 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fb0480a0710 0x7fb0480a2bd0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:28.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.387+0000 7fb06ea00700 1 --2- 172.21.15.1:0/191193671 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb06810b0c0 0x7fb068196ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:28.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.387+0000 7fb06ea00700 1 --2- 172.21.15.1:0/191193671 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb068101ca0 0x7fb06819ca40 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:28.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.387+0000 7fb06ea00700 1 --2- 172.21.15.1:0/191193671 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb0681025c0 0x7fb06819cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:28.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.387+0000 7fb06ea00700 1 -- 172.21.15.1:0/191193671 >> 172.21.15.1:0/191193671 conn(0x7fb0680fb480 msgr2=0x7fb06810d280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:32:28.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.387+0000 7fb06ea00700 1 -- 172.21.15.1:0/191193671 shutdown_connections 2024-02-02T15:32:28.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.387+0000 7fb06ea00700 1 -- 172.21.15.1:0/191193671 wait complete. 2024-02-02T15:32:28.406 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-02T15:32:28.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.634+0000 7fca87000700 1 -- 172.21.15.1:0/988804350 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca80100ff0 msgr2=0x7fca801013d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:28.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.634+0000 7fca87000700 1 --2- 172.21.15.1:0/988804350 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca80100ff0 0x7fca801013d0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fca70009990 tx=0x7fca70009ca0 comp rx=0 tx=0).stop 2024-02-02T15:32:28.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.634+0000 7fca87000700 1 -- 172.21.15.1:0/988804350 shutdown_connections 2024-02-02T15:32:28.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.634+0000 7fca87000700 1 --2- 172.21.15.1:0/988804350 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca80106110 0x7fca8010dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:28.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.634+0000 7fca87000700 1 --2- 172.21.15.1:0/988804350 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca80100ff0 0x7fca801013d0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:28.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.634+0000 7fca87000700 1 --2- 172.21.15.1:0/988804350 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca801019a0 0x7fca801059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:28.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.634+0000 7fca87000700 1 -- 172.21.15.1:0/988804350 >> 172.21.15.1:0/988804350 conn(0x7fca800fc810 msgr2=0x7fca800fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:32:28.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.634+0000 7fca87000700 1 -- 172.21.15.1:0/988804350 shutdown_connections 2024-02-02T15:32:28.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.634+0000 7fca87000700 1 -- 172.21.15.1:0/988804350 wait complete. 2024-02-02T15:32:28.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.635+0000 7fca87000700 1 Processor -- start 2024-02-02T15:32:28.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.635+0000 7fca87000700 1 -- start start 2024-02-02T15:32:28.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.636+0000 7fca87000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca80100ff0 0x7fca80199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:28.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.636+0000 7fca87000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca80106110 0x7fca80199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:28.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.636+0000 7fca87000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca8019dab0 0x7fca8019df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:28.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.636+0000 7fca87000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca8019e530 con 0x7fca8019dab0 2024-02-02T15:32:28.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.636+0000 7fca87000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca8019e6a0 con 0x7fca80100ff0 2024-02-02T15:32:28.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.636+0000 7fca87000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca8019e860 con 0x7fca80106110 2024-02-02T15:32:28.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.636+0000 7fca7fe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca80100ff0 0x7fca80199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:28.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.636+0000 7fca85000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca8019dab0 0x7fca8019df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:28.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.636+0000 7fca7fe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca80100ff0 0x7fca80199110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:48112/0 (socket says 172.21.15.1:48112) 2024-02-02T15:32:28.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.636+0000 7fca7fe00700 1 -- 172.21.15.1:0/4061161932 learned_addr learned my addr 172.21.15.1:0/4061161932 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:32:28.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.636+0000 7fca7f400700 1 --2- 172.21.15.1:0/4061161932 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca80106110 0x7fca80199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:28.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.636+0000 7fca7fe00700 1 -- 172.21.15.1:0/4061161932 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca80106110 msgr2=0x7fca80199650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:28.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.636+0000 7fca7fe00700 1 --2- 172.21.15.1:0/4061161932 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca80106110 0x7fca80199650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:28.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.636+0000 7fca7fe00700 1 -- 172.21.15.1:0/4061161932 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca8019dab0 msgr2=0x7fca8019df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:28.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.636+0000 7fca7fe00700 1 --2- 172.21.15.1:0/4061161932 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca8019dab0 0x7fca8019df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:28.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.636+0000 7fca7fe00700 1 -- 172.21.15.1:0/4061161932 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca700095f0 con 0x7fca80100ff0 2024-02-02T15:32:28.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.636+0000 7fca7f400700 1 --2- 172.21.15.1:0/4061161932 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca80106110 0x7fca80199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:32:28.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.636+0000 7fca85000700 1 --2- 172.21.15.1:0/4061161932 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca8019dab0 0x7fca8019df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:32:28.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.637+0000 7fca7fe00700 1 --2- 172.21.15.1:0/4061161932 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca80100ff0 0x7fca80199110 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fca70005770 tx=0x7fca700057a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:32:28.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.637+0000 7fca7cc00700 1 -- 172.21.15.1:0/4061161932 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca7001d070 con 0x7fca80100ff0 2024-02-02T15:32:28.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.637+0000 7fca7cc00700 1 -- 172.21.15.1:0/4061161932 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca70022470 con 0x7fca80100ff0 2024-02-02T15:32:28.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.637+0000 7fca87000700 1 -- 172.21.15.1:0/4061161932 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca801a5d60 con 0x7fca80100ff0 2024-02-02T15:32:28.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.637+0000 7fca7cc00700 1 -- 172.21.15.1:0/4061161932 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca70017650 con 0x7fca80100ff0 2024-02-02T15:32:28.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.637+0000 7fca87000700 1 -- 172.21.15.1:0/4061161932 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca801a6180 con 0x7fca80100ff0 2024-02-02T15:32:28.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.640+0000 7fca7cc00700 1 -- 172.21.15.1:0/4061161932 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 27) v1 ==== 135200+0+0 (secure 0 0 0) 0x7fca7000fb40 con 0x7fca80100ff0 2024-02-02T15:32:28.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.641+0000 7fca7cc00700 1 --2- 172.21.15.1:0/4061161932 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fca600a0690 0x7fca600a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:28.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.642+0000 7fca9aaddb80 1 Processor -- start 2024-02-02T15:32:28.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.642+0000 7fca7cc00700 1 -- 172.21.15.1:0/4061161932 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fca700cd510 con 0x7fca80100ff0 2024-02-02T15:32:28.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.642+0000 7fca7f400700 1 --2- 172.21.15.1:0/4061161932 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fca600a0690 0x7fca600a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:28.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.642+0000 7fca9aaddb80 1 -- start start 2024-02-02T15:32:28.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.642+0000 7fca9aaddb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b30d1f5010 0x55b30d1f53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:28.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.642+0000 7fca9aaddb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b30d1f5930 0x55b30d2002a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:28.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.642+0000 7fca9aaddb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b30d2007e0 0x55b30d202bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:28.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.642+0000 7fca9aaddb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b30d1694c0 con 0x55b30d1f5010 2024-02-02T15:32:28.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.642+0000 7fca9aaddb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b30d14daa0 con 0x55b30d2007e0 2024-02-02T15:32:28.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.642+0000 7fca9aaddb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b30d14b460 con 0x55b30d1f5930 2024-02-02T15:32:28.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.643+0000 7fca7fe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b30d1f5010 0x55b30d1f53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:28.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.643+0000 7fca85000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b30d2007e0 0x55b30d202bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:28.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.643+0000 7fca7fe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b30d1f5010 0x55b30d1f53f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39504/0 (socket says 172.21.15.1:39504) 2024-02-02T15:32:28.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.643+0000 7fca7fe00700 1 -- 172.21.15.1:0/859673678 learned_addr learned my addr 172.21.15.1:0/859673678 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:32:28.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.643+0000 7fca7f400700 1 --2- 172.21.15.1:0/4061161932 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fca600a0690 0x7fca600a2b50 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fca8019a720 tx=0x7fca68009260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:32:28.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.643+0000 7fca7f400700 1 --2- 172.21.15.1:0/859673678 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b30d1f5930 0x55b30d2002a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:28.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.643+0000 7fca85000700 1 -- 172.21.15.1:0/859673678 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b30d1f5930 msgr2=0x55b30d2002a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:32:28.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.643+0000 7fca85000700 1 --2- 172.21.15.1:0/859673678 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b30d1f5930 0x55b30d2002a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:28.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.643+0000 7fca85000700 1 -- 172.21.15.1:0/859673678 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b30d1f5010 msgr2=0x55b30d1f53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:32:28.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.643+0000 7fca85000700 1 --2- 172.21.15.1:0/859673678 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b30d1f5010 0x55b30d1f53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:28.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.643+0000 7fca85000700 1 -- 172.21.15.1:0/859673678 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b30cfc83b0 con 0x55b30d2007e0 2024-02-02T15:32:28.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.643+0000 7fca85000700 1 --2- 172.21.15.1:0/859673678 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b30d2007e0 0x55b30d202bd0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fca78003de0 tx=0x7fca78009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2ee7a8021a2a3e9 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:32:28.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.643+0000 7fca75600700 1 -- 172.21.15.1:0/859673678 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca78004bb0 con 0x55b30d2007e0 2024-02-02T15:32:28.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.644+0000 7fca75600700 1 -- 172.21.15.1:0/859673678 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca7800f040 con 0x55b30d2007e0 2024-02-02T15:32:28.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.644+0000 7fca75600700 1 -- 172.21.15.1:0/859673678 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca78007cb0 con 0x55b30d2007e0 2024-02-02T15:32:28.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.644+0000 7fca9aaddb80 1 -- 172.21.15.1:0/859673678 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b30d2007e0 msgr2=0x55b30d202bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:28.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.644+0000 7fca9aaddb80 1 --2- 172.21.15.1:0/859673678 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b30d2007e0 0x55b30d202bd0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fca78003de0 tx=0x7fca78009920 comp rx=0 tx=0).stop 2024-02-02T15:32:28.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.644+0000 7fca9aaddb80 1 -- 172.21.15.1:0/859673678 shutdown_connections 2024-02-02T15:32:28.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.644+0000 7fca9aaddb80 1 --2- 172.21.15.1:0/859673678 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b30d2007e0 0x55b30d202bd0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:28.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.644+0000 7fca9aaddb80 1 --2- 172.21.15.1:0/859673678 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b30d1f5010 0x55b30d1f53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:28.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.644+0000 7fca9aaddb80 1 --2- 172.21.15.1:0/859673678 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b30d1f5930 0x55b30d2002a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:28.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.644+0000 7fca9aaddb80 1 -- 172.21.15.1:0/859673678 >> 172.21.15.1:0/859673678 conn(0x55b30d154db0 msgr2=0x55b30d1f13b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:32:28.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.644+0000 7fca9aaddb80 1 -- 172.21.15.1:0/859673678 shutdown_connections 2024-02-02T15:32:28.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.644+0000 7fca9aaddb80 1 -- 172.21.15.1:0/859673678 wait complete. 2024-02-02T15:32:28.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.645+0000 7fca9aaddb80 1 -- start start 2024-02-02T15:32:28.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.645+0000 7fca9aaddb80 1 Processor -- start 2024-02-02T15:32:28.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.646+0000 7fca9aaddb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b30d1f5010 0x55b30d200220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:28.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.646+0000 7fca9aaddb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b30d1f5930 0x55b30d1fd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:28.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.646+0000 7fca9aaddb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b30d2007e0 0x55b30d1fdb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:28.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.646+0000 7fca9aaddb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b30d146d50 con 0x55b30d1f5930 2024-02-02T15:32:28.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.646+0000 7fca9aaddb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b30d0e2d80 con 0x55b30d1f5010 2024-02-02T15:32:28.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.646+0000 7fca9aaddb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b30d0e2a20 con 0x55b30d2007e0 2024-02-02T15:32:28.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.646+0000 7fca9aaddb80 10 client.0 _ll_register_callbacks cb 0x55b30d0e4290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:32:28.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.646+0000 7fca7fe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b30d1f5010 0x55b30d200220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:28.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.646+0000 7fca85000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b30d2007e0 0x55b30d1fdb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:28.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.646+0000 7fca7fe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b30d1f5010 0x55b30d200220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:48128/0 (socket says 172.21.15.1:48128) 2024-02-02T15:32:28.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.646+0000 7fca7fe00700 1 -- 172.21.15.1:0/378700899 learned_addr learned my addr 172.21.15.1:0/378700899 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:32:28.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.646+0000 7fca7f400700 1 --2- 172.21.15.1:0/378700899 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b30d1f5930 0x55b30d1fd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:28.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.646+0000 7fca7fe00700 1 -- 172.21.15.1:0/378700899 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b30d2007e0 msgr2=0x55b30d1fdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:32:28.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.646+0000 7fca7fe00700 1 --2- 172.21.15.1:0/378700899 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b30d2007e0 0x55b30d1fdb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:28.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.646+0000 7fca7fe00700 1 -- 172.21.15.1:0/378700899 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b30d1f5930 msgr2=0x55b30d1fd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:32:28.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.646+0000 7fca7fe00700 1 --2- 172.21.15.1:0/378700899 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b30d1f5930 0x55b30d1fd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:28.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.646+0000 7fca7fe00700 1 -- 172.21.15.1:0/378700899 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca78003a60 con 0x55b30d1f5010 2024-02-02T15:32:28.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.646+0000 7fca7f400700 1 --2- 172.21.15.1:0/378700899 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b30d1f5930 0x55b30d1fd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:32:28.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.646+0000 7fca85000700 1 --2- 172.21.15.1:0/378700899 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b30d2007e0 0x55b30d1fdb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:32:28.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.647+0000 7fca7fe00700 1 --2- 172.21.15.1:0/378700899 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b30d1f5010 0x55b30d200220 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fca700a21c0 tx=0x7fca70005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e4fc87096a22469 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:32:28.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.647+0000 7fca6e000700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:32:28.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.647+0000 7fca6e000700 1 -- 172.21.15.1:0/378700899 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca7009ae10 con 0x55b30d1f5010 2024-02-02T15:32:28.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.647+0000 7fca6e000700 1 -- 172.21.15.1:0/378700899 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fca7009a710 con 0x55b30d1f5010 2024-02-02T15:32:28.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.647+0000 7fca9aaddb80 1 -- 172.21.15.1:0/378700899 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b30d1e90f0 con 0x55b30d1f5010 2024-02-02T15:32:28.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.647+0000 7fca6e000700 1 -- 172.21.15.1:0/378700899 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca7009e540 con 0x55b30d1f5010 2024-02-02T15:32:28.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.647+0000 7fca9aaddb80 10 client.27146 fetch_fsmap learned FSMap version 35 2024-02-02T15:32:28.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.647+0000 7fca9aaddb80 1 -- 172.21.15.1:0/378700899 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55b30cf74b60 con 0x55b30d1f5010 2024-02-02T15:32:28.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.647+0000 7fca6e000700 1 -- 172.21.15.1:0/378700899 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b30d1e90f0 con 0x55b30d1f5010 2024-02-02T15:32:28.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.648+0000 7fca6e000700 1 -- 172.21.15.1:0/378700899 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 35) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fca70099d00 con 0x55b30d1f5010 2024-02-02T15:32:28.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.648+0000 7fca9aaddb80 10 client.27146 fetch_fsmap finished waiting for FSMap version 35 2024-02-02T15:32:28.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.648+0000 7fca9aaddb80 10 client.27146 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:32:28.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.648+0000 7fca9aaddb80 20 client.27146 populate_metadata read hostname 'smithi001' 2024-02-02T15:32:28.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.648+0000 7fca9aaddb80 1 --2- 172.21.15.1:0/378700899 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55b30d20fe40 0x55b30d212260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:28.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.648+0000 7fca9aaddb80 4 client.27146 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:32:28.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.648+0000 7fca9aaddb80 1 -- 172.21.15.1:0/378700899 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b30d13fdd0 con 0x55b30d20fe40 2024-02-02T15:32:28.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.648+0000 7fca7f400700 1 --2- 172.21.15.1:0/378700899 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55b30d20fe40 0x55b30d212260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:28.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.648+0000 7fca7f400700 1 --2- 172.21.15.1:0/378700899 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55b30d20fe40 0x55b30d212260 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8840954aef12dffc server_cookie=228df0ac1f303c76 in_seq=0 out_seq=0 2024-02-02T15:32:28.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.648+0000 7fca6e000700 10 client.27146 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:32:28.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.649+0000 7fca6e000700 1 -- 172.21.15.1:0/378700899 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55b30d13fdd0 con 0x55b30d20fe40 2024-02-02T15:32:28.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.649+0000 7fca6e000700 10 client.27146 handle_command_reply: tid=0 2024-02-02T15:32:28.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.649+0000 7fca9aaddb80 1 client.27146 shutdown 2024-02-02T15:32:28.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.649+0000 7fca9aaddb80 1 -- 172.21.15.1:0/378700899 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b30d1f5010 msgr2=0x55b30d200220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:28.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.649+0000 7fca9aaddb80 1 --2- 172.21.15.1:0/378700899 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b30d1f5010 0x55b30d200220 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fca700a21c0 tx=0x7fca70005fd0 comp rx=0 tx=0).stop 2024-02-02T15:32:28.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.649+0000 7fca9aaddb80 1 -- 172.21.15.1:0/378700899 shutdown_connections 2024-02-02T15:32:28.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.649+0000 7fca9aaddb80 1 --2- 172.21.15.1:0/378700899 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55b30d20fe40 0x55b30d212260 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:28.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.649+0000 7fca9aaddb80 1 --2- 172.21.15.1:0/378700899 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b30d2007e0 0x55b30d1fdb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:28.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.649+0000 7fca9aaddb80 1 --2- 172.21.15.1:0/378700899 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b30d1f5010 0x55b30d200220 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:28.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.649+0000 7fca9aaddb80 1 --2- 172.21.15.1:0/378700899 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b30d1f5930 0x55b30d1fd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:28.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.649+0000 7fca6e000700 0 client.27146 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:32:28.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.649+0000 7fca9aaddb80 1 -- 172.21.15.1:0/378700899 >> 172.21.15.1:0/378700899 conn(0x55b30d154db0 msgr2=0x55b30d1f0fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:32:28.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.650+0000 7fca9aaddb80 1 -- 172.21.15.1:0/378700899 shutdown_connections 2024-02-02T15:32:28.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.650+0000 7fca9aaddb80 1 -- 172.21.15.1:0/378700899 wait complete. 2024-02-02T15:32:28.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.650+0000 7fca9aaddb80 20 client.27146 trim_cache size 0 max 16384 2024-02-02T15:32:28.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.668+0000 7fca9aaddb80 1 Processor -- start 2024-02-02T15:32:28.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.668+0000 7fca9aaddb80 1 -- start start 2024-02-02T15:32:28.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.668+0000 7fca9aaddb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b30d1f5010 0x55b30d207370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:28.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.668+0000 7fca9aaddb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b30d1f5930 0x55b30d1efb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:28.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.668+0000 7fca9aaddb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b30d2007e0 0x55b30d1ffd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:28.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.668+0000 7fca9aaddb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b30cfca6a0 con 0x55b30d1f5930 2024-02-02T15:32:28.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.668+0000 7fca9aaddb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b30d1ed9b0 con 0x55b30d2007e0 2024-02-02T15:32:28.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.668+0000 7fca9aaddb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b30d1fe1f0 con 0x55b30d1f5010 2024-02-02T15:32:28.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.668+0000 7fca7f400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b30d1f5930 0x55b30d1efb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:28.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.669+0000 7fca85000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b30d2007e0 0x55b30d1ffd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:28.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.668+0000 7fca7fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b30d1f5010 0x55b30d207370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:28.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.669+0000 7fca85000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b30d2007e0 0x55b30d1ffd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:48140/0 (socket says 172.21.15.1:48140) 2024-02-02T15:32:28.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.669+0000 7fca85000700 1 -- 172.21.15.1:0/485265227 learned_addr learned my addr 172.21.15.1:0/485265227 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:32:28.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.669+0000 7fca7f400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b30d1f5930 0x55b30d1efb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39514/0 (socket says 172.21.15.1:39514) 2024-02-02T15:32:28.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.669+0000 7fca85000700 1 -- 172.21.15.1:0/485265227 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b30d1f5010 msgr2=0x55b30d207370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:32:28.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.669+0000 7fca85000700 1 --2- 172.21.15.1:0/485265227 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b30d1f5010 0x55b30d207370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:28.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.669+0000 7fca85000700 1 -- 172.21.15.1:0/485265227 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b30d1f5930 msgr2=0x55b30d1efb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:32:28.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.669+0000 7fca85000700 1 --2- 172.21.15.1:0/485265227 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b30d1f5930 0x55b30d1efb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:28.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.669+0000 7fca85000700 1 -- 172.21.15.1:0/485265227 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b30d10cea0 con 0x55b30d2007e0 2024-02-02T15:32:28.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.669+0000 7fca85000700 1 --2- 172.21.15.1:0/485265227 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b30d2007e0 0x55b30d1ffd50 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fca78009ec0 tx=0x7fca78003d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6cc87247046d155a server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:32:28.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.669+0000 7fca76000700 1 -- 172.21.15.1:0/485265227 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca780049a0 con 0x55b30d2007e0 2024-02-02T15:32:28.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.669+0000 7fca76000700 1 -- 172.21.15.1:0/485265227 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca78007500 con 0x55b30d2007e0 2024-02-02T15:32:28.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.670+0000 7fca76000700 1 -- 172.21.15.1:0/485265227 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca7800e750 con 0x55b30d2007e0 2024-02-02T15:32:28.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.670+0000 7fca9aaddb80 1 -- 172.21.15.1:0/485265227 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b30d2007e0 msgr2=0x55b30d1ffd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:28.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.670+0000 7fca9aaddb80 1 --2- 172.21.15.1:0/485265227 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b30d2007e0 0x55b30d1ffd50 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fca78009ec0 tx=0x7fca78003d70 comp rx=0 tx=0).stop 2024-02-02T15:32:28.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.670+0000 7fca9aaddb80 1 -- 172.21.15.1:0/485265227 shutdown_connections 2024-02-02T15:32:28.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.670+0000 7fca9aaddb80 1 --2- 172.21.15.1:0/485265227 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b30d2007e0 0x55b30d1ffd50 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:28.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.670+0000 7fca9aaddb80 1 --2- 172.21.15.1:0/485265227 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b30d1f5010 0x55b30d207370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:28.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.670+0000 7fca9aaddb80 1 --2- 172.21.15.1:0/485265227 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b30d1f5930 0x55b30d1efb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:28.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.670+0000 7fca9aaddb80 1 -- 172.21.15.1:0/485265227 >> 172.21.15.1:0/485265227 conn(0x55b30d154db0 msgr2=0x55b30d20c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:32:28.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.670+0000 7fca9aaddb80 1 -- 172.21.15.1:0/485265227 shutdown_connections 2024-02-02T15:32:28.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.670+0000 7fca9aaddb80 1 -- 172.21.15.1:0/485265227 wait complete. 2024-02-02T15:32:28.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.671+0000 7fca9aaddb80 1 -- start start 2024-02-02T15:32:28.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.671+0000 7fca9aaddb80 1 Processor -- start 2024-02-02T15:32:28.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.671+0000 7fca9aaddb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b30d1f5010 0x55b30d207100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:28.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.671+0000 7fca9aaddb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b30d1f5930 0x55b30d205430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:28.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.671+0000 7fca9aaddb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b30d2007e0 0x55b30d205970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:28.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.671+0000 7fca9aaddb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b30d205f40 con 0x55b30d1f5930 2024-02-02T15:32:28.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.671+0000 7fca9aaddb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b30d2060b0 con 0x55b30d1f5010 2024-02-02T15:32:28.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.671+0000 7fca9aaddb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b30d206270 con 0x55b30d2007e0 2024-02-02T15:32:28.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.671+0000 7fca9aaddb80 10 client.0 _ll_register_callbacks cb 0x55b30d0e4290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:32:28.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.671+0000 7fca7fe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b30d1f5010 0x55b30d207100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:28.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.671+0000 7fca7f400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b30d1f5930 0x55b30d205430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:28.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.672+0000 7fca85000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b30d2007e0 0x55b30d205970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:28.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.672+0000 7fca7fe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b30d1f5010 0x55b30d207100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:48154/0 (socket says 172.21.15.1:48154) 2024-02-02T15:32:28.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.672+0000 7fca7fe00700 1 -- 172.21.15.1:0/1377240796 learned_addr learned my addr 172.21.15.1:0/1377240796 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:32:28.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.672+0000 7fca7fe00700 1 -- 172.21.15.1:0/1377240796 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b30d2007e0 msgr2=0x55b30d205970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:32:28.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.672+0000 7fca7fe00700 1 --2- 172.21.15.1:0/1377240796 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b30d2007e0 0x55b30d205970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:28.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.672+0000 7fca7fe00700 1 -- 172.21.15.1:0/1377240796 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b30d1f5930 msgr2=0x55b30d205430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:32:28.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.672+0000 7fca7fe00700 1 --2- 172.21.15.1:0/1377240796 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b30d1f5930 0x55b30d205430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:28.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.672+0000 7fca7fe00700 1 -- 172.21.15.1:0/1377240796 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b30cfc83b0 con 0x55b30d1f5010 2024-02-02T15:32:28.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.672+0000 7fca7f400700 1 --2- 172.21.15.1:0/1377240796 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b30d1f5930 0x55b30d205430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:32:28.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.672+0000 7fca85000700 1 --2- 172.21.15.1:0/1377240796 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b30d2007e0 0x55b30d205970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:32:28.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.672+0000 7fca7fe00700 1 --2- 172.21.15.1:0/1377240796 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b30d1f5010 0x55b30d207100 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fca70098a60 tx=0x7fca7009a590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e01fd841150315d server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:32:28.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.672+0000 7fca6e000700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:32:28.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.672+0000 7fca6e000700 1 -- 172.21.15.1:0/1377240796 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca7009ae10 con 0x55b30d1f5010 2024-02-02T15:32:28.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.672+0000 7fca6e000700 1 -- 172.21.15.1:0/1377240796 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fca7009a710 con 0x55b30d1f5010 2024-02-02T15:32:28.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.672+0000 7fca9aaddb80 1 -- 172.21.15.1:0/1377240796 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fca7800f040 con 0x55b30d1f5010 2024-02-02T15:32:28.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.673+0000 7fca6e000700 1 -- 172.21.15.1:0/1377240796 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca7009e540 con 0x55b30d1f5010 2024-02-02T15:32:28.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.673+0000 7fca9aaddb80 10 client.27152 fetch_fsmap learned FSMap version 35 2024-02-02T15:32:28.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.673+0000 7fca9aaddb80 1 -- 172.21.15.1:0/1377240796 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55b30cfcccd0 con 0x55b30d1f5010 2024-02-02T15:32:28.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.673+0000 7fca6e000700 1 -- 172.21.15.1:0/1377240796 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca7800f040 con 0x55b30d1f5010 2024-02-02T15:32:28.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.673+0000 7fca6e000700 1 -- 172.21.15.1:0/1377240796 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 35) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fca700a22f0 con 0x55b30d1f5010 2024-02-02T15:32:28.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.673+0000 7fca9aaddb80 10 client.27152 fetch_fsmap finished waiting for FSMap version 35 2024-02-02T15:32:28.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.673+0000 7fca9aaddb80 10 client.27152 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:32:28.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.673+0000 7fca9aaddb80 20 client.27152 populate_metadata read hostname 'smithi001' 2024-02-02T15:32:28.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.673+0000 7fca9aaddb80 1 --2- 172.21.15.1:0/1377240796 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55b30d2629f0 0x55b30d264de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:28.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.673+0000 7fca9aaddb80 4 client.27152 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2024-02-02T15:32:28.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.673+0000 7fca9aaddb80 1 -- 172.21.15.1:0/1377240796 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b30d105f50 con 0x55b30d2629f0 2024-02-02T15:32:28.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.673+0000 7fca7f400700 1 --2- 172.21.15.1:0/1377240796 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55b30d2629f0 0x55b30d264de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:28.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.674+0000 7fca7f400700 1 --2- 172.21.15.1:0/1377240796 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55b30d2629f0 0x55b30d264de0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39623e45dc449065 server_cookie=405c4341d1198cf in_seq=0 out_seq=0 2024-02-02T15:32:28.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.674+0000 7fca6e000700 10 client.27152 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:32:28.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.674+0000 7fca6e000700 1 -- 172.21.15.1:0/1377240796 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55b30d105f50 con 0x55b30d2629f0 2024-02-02T15:32:28.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.674+0000 7fca6e000700 10 client.27152 handle_command_reply: tid=0 2024-02-02T15:32:28.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.674+0000 7fca9aaddb80 1 client.27152 shutdown 2024-02-02T15:32:28.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.674+0000 7fca9aaddb80 1 -- 172.21.15.1:0/1377240796 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b30d1f5010 msgr2=0x55b30d207100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:28.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.674+0000 7fca9aaddb80 1 --2- 172.21.15.1:0/1377240796 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b30d1f5010 0x55b30d207100 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fca70098a60 tx=0x7fca7009a590 comp rx=0 tx=0).stop 2024-02-02T15:32:28.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.674+0000 7fca9aaddb80 1 -- 172.21.15.1:0/1377240796 shutdown_connections 2024-02-02T15:32:28.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.674+0000 7fca9aaddb80 1 --2- 172.21.15.1:0/1377240796 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55b30d2629f0 0x55b30d264de0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:28.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.674+0000 7fca9aaddb80 1 --2- 172.21.15.1:0/1377240796 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b30d2007e0 0x55b30d205970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:28.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.674+0000 7fca9aaddb80 1 --2- 172.21.15.1:0/1377240796 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b30d1f5010 0x55b30d207100 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:28.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.674+0000 7fca6e000700 0 client.27152 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:32:28.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.674+0000 7fca9aaddb80 1 --2- 172.21.15.1:0/1377240796 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b30d1f5930 0x55b30d205430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:28.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.674+0000 7fca9aaddb80 1 -- 172.21.15.1:0/1377240796 >> 172.21.15.1:0/1377240796 conn(0x55b30d154db0 msgr2=0x55b30d1f05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:32:28.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.674+0000 7fca9aaddb80 1 -- 172.21.15.1:0/1377240796 shutdown_connections 2024-02-02T15:32:28.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.675+0000 7fca9aaddb80 1 -- 172.21.15.1:0/1377240796 wait complete. 2024-02-02T15:32:28.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.675+0000 7fca9aaddb80 20 client.27152 trim_cache size 0 max 16384 2024-02-02T15:32:28.680 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:32:28.680 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-02T15:32:28.680 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2024-02-02T15:32:28.680 INFO:teuthology.orchestra.run.smithi001.stdout: "fc847698-fb77-4bda-ada9-387389e573de": { 2024-02-02T15:32:28.680 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2024-02-02T15:32:28.680 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "fc847698-fb77-4bda-ada9-387389e573de", 2024-02-02T15:32:28.680 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,force" 2024-02-02T15:32:28.680 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:32:28.680 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:32:28.681 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:32:28.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.675+0000 7fca87000700 1 -- 172.21.15.1:0/4061161932 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fca600a0690 msgr2=0x7fca600a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:28.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.675+0000 7fca87000700 1 --2- 172.21.15.1:0/4061161932 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fca600a0690 0x7fca600a2b50 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fca8019a720 tx=0x7fca68009260 comp rx=0 tx=0).stop 2024-02-02T15:32:28.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.675+0000 7fca87000700 1 -- 172.21.15.1:0/4061161932 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca80100ff0 msgr2=0x7fca80199110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:28.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.675+0000 7fca87000700 1 --2- 172.21.15.1:0/4061161932 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca80100ff0 0x7fca80199110 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fca70005770 tx=0x7fca700057a0 comp rx=0 tx=0).stop 2024-02-02T15:32:28.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.675+0000 7fca87000700 1 -- 172.21.15.1:0/4061161932 shutdown_connections 2024-02-02T15:32:28.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.675+0000 7fca87000700 1 --2- 172.21.15.1:0/4061161932 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fca600a0690 0x7fca600a2b50 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:28.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.675+0000 7fca87000700 1 --2- 172.21.15.1:0/4061161932 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca8019dab0 0x7fca8019df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:28.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.675+0000 7fca87000700 1 --2- 172.21.15.1:0/4061161932 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca80100ff0 0x7fca80199110 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:28.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.675+0000 7fca87000700 1 --2- 172.21.15.1:0/4061161932 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca80106110 0x7fca80199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:28.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.675+0000 7fca87000700 1 -- 172.21.15.1:0/4061161932 >> 172.21.15.1:0/4061161932 conn(0x7fca800fc810 msgr2=0x7fca800fd690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:32:28.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.675+0000 7fca87000700 1 -- 172.21.15.1:0/4061161932 shutdown_connections 2024-02-02T15:32:28.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:28.675+0000 7fca87000700 1 -- 172.21.15.1:0/4061161932 wait complete. 2024-02-02T15:32:28.693 INFO:tasks.cephfs.filesystem:scrub status for tag:fc847698-fb77-4bda-ada9-387389e573de - {'path': '/', 'tag': 'fc847698-fb77-4bda-ada9-387389e573de', 'options': 'recursive,force'} 2024-02-02T15:32:29.235 INFO:tasks.workunit.client.0.smithi001.stdout: 1 348165 9.11 MB/sec execute 123 sec latency 25.119 ms 2024-02-02T15:32:29.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:32:29 smithi142 ceph-mon[45728]: pgmap v167: 129 pgs: 129 active+clean; 211 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 218 op/s 2024-02-02T15:32:29.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:32:29 smithi142 ceph-mon[45728]: scrub queued for path: / 2024-02-02T15:32:29.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:32:29 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:32:29.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:32:29 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:32:29.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:32:29 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:32:29.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:32:29 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:32:29.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:32:29 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:32:29.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:32:29 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:32:29.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:32:29 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:32:29.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:32:29 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:32:29.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:32:29 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:32:29.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:32:29 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:32:29.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:32:29 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:32:29.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:32:29 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:32:29.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:32:29 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:32:29.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:32:29 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:32:29.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:32:29 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:32:29.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:32:29 smithi001 ceph-mon[35470]: pgmap v167: 129 pgs: 129 active+clean; 211 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 218 op/s 2024-02-02T15:32:29.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:32:29 smithi001 ceph-mon[35470]: scrub queued for path: / 2024-02-02T15:32:29.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:32:29 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:32:29.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:32:29 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:32:29.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:32:29 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:32:29.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:32:29 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:32:29.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:32:29 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:32:29.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:32:29 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:32:29.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:32:29 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:32:29.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:32:29 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:32:29.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:32:29 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:32:29.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:32:29 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:32:29.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:32:29 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:32:29.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:32:29 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:32:29.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:32:29 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:32:29.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:32:29 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:32:29.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:32:29 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:32:29.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:32:29 smithi071 ceph-mon[44707]: pgmap v167: 129 pgs: 129 active+clean; 211 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 218 op/s 2024-02-02T15:32:29.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:32:29 smithi071 ceph-mon[44707]: scrub queued for path: / 2024-02-02T15:32:29.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:32:29 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:32:29.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:32:29 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:32:29.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:32:29 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:32:29.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:32:29 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:32:29.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:32:29 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:32:29.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:32:29 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:32:29.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:32:29 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:32:29.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:32:29 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:32:29.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:32:29 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:32:29.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:32:29 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:32:29.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:32:29 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:32:29.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:32:29 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:32:29.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:32:29 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:32:29.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:32:29 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:32:29.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:32:29 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:32:30.235 INFO:tasks.workunit.client.0.smithi001.stdout: 1 350012 9.10 MB/sec execute 124 sec latency 14.777 ms 2024-02-02T15:32:31.235 INFO:tasks.workunit.client.0.smithi001.stdout: 1 351530 9.12 MB/sec execute 125 sec latency 12.381 ms 2024-02-02T15:32:31.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:32:31 smithi142 ceph-mon[45728]: pgmap v168: 129 pgs: 129 active+clean; 211 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 218 op/s 2024-02-02T15:32:31.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:32:31 smithi001 ceph-mon[35470]: pgmap v168: 129 pgs: 129 active+clean; 211 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 218 op/s 2024-02-02T15:32:31.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:32:31 smithi071 ceph-mon[44707]: pgmap v168: 129 pgs: 129 active+clean; 211 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.5 MiB/s wr, 218 op/s 2024-02-02T15:32:32.235 INFO:tasks.workunit.client.0.smithi001.stdout: 1 352717 9.09 MB/sec execute 126 sec latency 19.139 ms 2024-02-02T15:32:33.235 INFO:tasks.workunit.client.0.smithi001.stdout: 1 354707 9.11 MB/sec execute 127 sec latency 28.876 ms 2024-02-02T15:32:33.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:32:33 smithi142 ceph-mon[45728]: pgmap v169: 129 pgs: 129 active+clean; 216 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.7 MiB/s wr, 327 op/s 2024-02-02T15:32:33.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:32:33 smithi142 ceph-mon[45728]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:32:33.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:32:33 smithi001 ceph-mon[35470]: pgmap v169: 129 pgs: 129 active+clean; 216 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.7 MiB/s wr, 327 op/s 2024-02-02T15:32:33.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:32:33 smithi001 ceph-mon[35470]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:32:33.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:32:33 smithi071 ceph-mon[44707]: pgmap v169: 129 pgs: 129 active+clean; 216 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.7 MiB/s wr, 327 op/s 2024-02-02T15:32:33.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:32:33 smithi071 ceph-mon[44707]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:32:34.235 INFO:tasks.workunit.client.0.smithi001.stdout: 1 355739 9.07 MB/sec execute 128 sec latency 14.650 ms 2024-02-02T15:32:35.235 INFO:tasks.workunit.client.0.smithi001.stdout: 1 357262 9.05 MB/sec execute 129 sec latency 15.439 ms 2024-02-02T15:32:35.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:32:35 smithi142 ceph-mon[45728]: pgmap v170: 129 pgs: 129 active+clean; 216 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 213 op/s 2024-02-02T15:32:35.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:32:35 smithi001 ceph-mon[35470]: pgmap v170: 129 pgs: 129 active+clean; 216 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 213 op/s 2024-02-02T15:32:35.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:32:35 smithi071 ceph-mon[44707]: pgmap v170: 129 pgs: 129 active+clean; 216 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 213 op/s 2024-02-02T15:32:36.235 INFO:tasks.workunit.client.0.smithi001.stdout: 1 358703 9.06 MB/sec execute 130 sec latency 12.304 ms 2024-02-02T15:32:37.235 INFO:tasks.workunit.client.0.smithi001.stdout: 1 360341 9.04 MB/sec execute 131 sec latency 16.556 ms 2024-02-02T15:32:37.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:32:37 smithi142 ceph-mon[45728]: pgmap v171: 129 pgs: 129 active+clean; 221 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.3 MiB/s wr, 313 op/s 2024-02-02T15:32:37.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:32:37 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:32:37.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:32:37 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:32:37.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:32:37 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:32:37.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:32:37 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:32:37.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:32:37 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:32:37.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:32:37 smithi001 ceph-mon[35470]: pgmap v171: 129 pgs: 129 active+clean; 221 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.3 MiB/s wr, 313 op/s 2024-02-02T15:32:37.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:32:37 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:32:37.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:32:37 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:32:37.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:32:37 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:32:37.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:32:37 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:32:37.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:32:37 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:32:37.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:32:37 smithi071 ceph-mon[44707]: pgmap v171: 129 pgs: 129 active+clean; 221 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.3 MiB/s wr, 313 op/s 2024-02-02T15:32:37.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:32:37 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:32:37.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:32:37 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:32:37.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:32:37 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:32:37.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:32:37 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:32:37.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:32:37 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:32:38.235 INFO:tasks.workunit.client.0.smithi001.stdout: 1 362040 9.07 MB/sec execute 132 sec latency 36.126 ms 2024-02-02T15:32:38.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:32:38 smithi142 ceph-mon[45728]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:32:38.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:32:38 smithi001 ceph-mon[35470]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:32:38.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:32:38 smithi071 ceph-mon[44707]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:32:39.235 INFO:tasks.workunit.client.0.smithi001.stdout: 1 363201 9.04 MB/sec execute 133 sec latency 15.532 ms 2024-02-02T15:32:39.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:32:39 smithi142 ceph-mon[45728]: pgmap v172: 129 pgs: 129 active+clean; 221 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 208 op/s 2024-02-02T15:32:39.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:32:39 smithi001 ceph-mon[35470]: pgmap v172: 129 pgs: 129 active+clean; 221 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 208 op/s 2024-02-02T15:32:39.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:32:39 smithi071 ceph-mon[44707]: pgmap v172: 129 pgs: 129 active+clean; 221 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 208 op/s 2024-02-02T15:32:40.236 INFO:tasks.workunit.client.0.smithi001.stdout: 1 365155 9.04 MB/sec execute 134 sec latency 19.613 ms 2024-02-02T15:32:41.236 INFO:tasks.workunit.client.0.smithi001.stdout: 1 366226 9.02 MB/sec execute 135 sec latency 14.833 ms 2024-02-02T15:32:41.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:32:41 smithi142 ceph-mon[45728]: pgmap v173: 129 pgs: 129 active+clean; 221 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 208 op/s 2024-02-02T15:32:41.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:32:41 smithi001 ceph-mon[35470]: pgmap v173: 129 pgs: 129 active+clean; 221 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 208 op/s 2024-02-02T15:32:41.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:32:41 smithi071 ceph-mon[44707]: pgmap v173: 129 pgs: 129 active+clean; 221 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 208 op/s 2024-02-02T15:32:42.236 INFO:tasks.workunit.client.0.smithi001.stdout: 1 367788 9.00 MB/sec execute 136 sec latency 19.988 ms 2024-02-02T15:32:43.236 INFO:tasks.workunit.client.0.smithi001.stdout: 1 369270 9.02 MB/sec execute 137 sec latency 23.996 ms 2024-02-02T15:32:43.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:32:43 smithi142 ceph-mon[45728]: pgmap v174: 129 pgs: 129 active+clean; 226 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.3 MiB/s wr, 317 op/s 2024-02-02T15:32:43.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:32:43 smithi142 ceph-mon[45728]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:32:43.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:32:43 smithi001 ceph-mon[35470]: pgmap v174: 129 pgs: 129 active+clean; 226 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.3 MiB/s wr, 317 op/s 2024-02-02T15:32:43.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:32:43 smithi001 ceph-mon[35470]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:32:43.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:32:43 smithi071 ceph-mon[44707]: pgmap v174: 129 pgs: 129 active+clean; 226 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.3 MiB/s wr, 317 op/s 2024-02-02T15:32:43.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:32:43 smithi071 ceph-mon[44707]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:32:44.236 INFO:tasks.workunit.client.0.smithi001.stdout: 1 370532 8.99 MB/sec execute 138 sec latency 18.304 ms 2024-02-02T15:32:45.236 INFO:tasks.workunit.client.0.smithi001.stdout: 1 372587 9.02 MB/sec execute 139 sec latency 16.648 ms 2024-02-02T15:32:45.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:32:45 smithi142 ceph-mon[45728]: pgmap v175: 129 pgs: 129 active+clean; 226 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.1 MiB/s wr, 208 op/s 2024-02-02T15:32:45.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:32:45 smithi001 ceph-mon[35470]: pgmap v175: 129 pgs: 129 active+clean; 226 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.1 MiB/s wr, 208 op/s 2024-02-02T15:32:45.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:32:45 smithi071 ceph-mon[44707]: pgmap v175: 129 pgs: 129 active+clean; 226 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.1 MiB/s wr, 208 op/s 2024-02-02T15:32:46.236 INFO:tasks.workunit.client.0.smithi001.stdout: 1 373682 8.98 MB/sec execute 140 sec latency 13.160 ms 2024-02-02T15:32:47.236 INFO:tasks.workunit.client.0.smithi001.stdout: 1 374833 8.95 MB/sec execute 141 sec latency 17.620 ms 2024-02-02T15:32:47.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:32:47 smithi142 ceph-mon[45728]: pgmap v176: 129 pgs: 129 active+clean; 231 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 318 op/s 2024-02-02T15:32:47.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:32:47 smithi001 ceph-mon[35470]: pgmap v176: 129 pgs: 129 active+clean; 231 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 318 op/s 2024-02-02T15:32:47.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:32:47 smithi071 ceph-mon[44707]: pgmap v176: 129 pgs: 129 active+clean; 231 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 318 op/s 2024-02-02T15:32:48.236 INFO:tasks.workunit.client.0.smithi001.stdout: 1 376341 8.97 MB/sec execute 142 sec latency 39.401 ms 2024-02-02T15:32:49.236 INFO:tasks.workunit.client.0.smithi001.stdout: 1 377533 8.94 MB/sec execute 143 sec latency 22.208 ms 2024-02-02T15:32:49.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:32:49 smithi142 ceph-mon[45728]: pgmap v177: 129 pgs: 129 active+clean; 231 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 218 op/s 2024-02-02T15:32:49.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:32:49 smithi001 ceph-mon[35470]: pgmap v177: 129 pgs: 129 active+clean; 231 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 218 op/s 2024-02-02T15:32:49.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:32:49 smithi071 ceph-mon[44707]: pgmap v177: 129 pgs: 129 active+clean; 231 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 218 op/s 2024-02-02T15:32:50.236 INFO:tasks.workunit.client.0.smithi001.stdout: 1 379674 8.97 MB/sec execute 144 sec latency 16.665 ms 2024-02-02T15:32:51.236 INFO:tasks.workunit.client.0.smithi001.stdout: 1 380699 8.93 MB/sec execute 145 sec latency 17.914 ms 2024-02-02T15:32:51.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:32:51 smithi142 ceph-mon[45728]: pgmap v178: 129 pgs: 129 active+clean; 231 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 218 op/s 2024-02-02T15:32:51.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:32:51 smithi001 ceph-mon[35470]: pgmap v178: 129 pgs: 129 active+clean; 231 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 218 op/s 2024-02-02T15:32:51.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:32:51 smithi071 ceph-mon[44707]: pgmap v178: 129 pgs: 129 active+clean; 231 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 218 op/s 2024-02-02T15:32:52.236 INFO:tasks.workunit.client.0.smithi001.stdout: 1 382736 8.95 MB/sec execute 146 sec latency 16.114 ms 2024-02-02T15:32:53.236 INFO:tasks.workunit.client.0.smithi001.stdout: 1 383827 8.93 MB/sec execute 147 sec latency 40.442 ms 2024-02-02T15:32:53.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:32:53 smithi142 ceph-mon[45728]: pgmap v179: 129 pgs: 129 active+clean; 236 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.6 MiB/s wr, 321 op/s 2024-02-02T15:32:53.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:32:53 smithi001 ceph-mon[35470]: pgmap v179: 129 pgs: 129 active+clean; 236 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.6 MiB/s wr, 321 op/s 2024-02-02T15:32:53.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:32:53 smithi071 ceph-mon[44707]: pgmap v179: 129 pgs: 129 active+clean; 236 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.6 MiB/s wr, 321 op/s 2024-02-02T15:32:54.236 INFO:tasks.workunit.client.0.smithi001.stdout: 1 385329 8.91 MB/sec execute 148 sec latency 21.741 ms 2024-02-02T15:32:55.236 INFO:tasks.workunit.client.0.smithi001.stdout: 1 387025 8.94 MB/sec execute 149 sec latency 14.126 ms 2024-02-02T15:32:55.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:32:55 smithi142 ceph-mon[45728]: pgmap v180: 129 pgs: 129 active+clean; 236 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.5 MiB/s wr, 213 op/s 2024-02-02T15:32:55.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:32:55 smithi001 ceph-mon[35470]: pgmap v180: 129 pgs: 129 active+clean; 236 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.5 MiB/s wr, 213 op/s 2024-02-02T15:32:55.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:32:55 smithi071 ceph-mon[44707]: pgmap v180: 129 pgs: 129 active+clean; 236 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.5 MiB/s wr, 213 op/s 2024-02-02T15:32:56.236 INFO:tasks.workunit.client.0.smithi001.stdout: 1 388314 8.91 MB/sec execute 150 sec latency 16.648 ms 2024-02-02T15:32:57.237 INFO:tasks.workunit.client.0.smithi001.stdout: 1 390335 8.93 MB/sec execute 151 sec latency 16.459 ms 2024-02-02T15:32:57.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:32:57 smithi142 ceph-mon[45728]: pgmap v181: 129 pgs: 129 active+clean; 241 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.5 MiB/s wr, 318 op/s 2024-02-02T15:32:57.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:32:57 smithi001 ceph-mon[35470]: pgmap v181: 129 pgs: 129 active+clean; 241 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.5 MiB/s wr, 318 op/s 2024-02-02T15:32:57.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:32:57 smithi071 ceph-mon[44707]: pgmap v181: 129 pgs: 129 active+clean; 241 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.5 MiB/s wr, 318 op/s 2024-02-02T15:32:58.237 INFO:tasks.workunit.client.0.smithi001.stdout: 1 391199 8.89 MB/sec execute 152 sec latency 44.837 ms 2024-02-02T15:32:58.694 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-02T15:32:58.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.907+0000 7ff016200700 1 -- 172.21.15.1:0/2005436716 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff010068f30 msgr2=0x7ff010069310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:58.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.907+0000 7ff016200700 1 --2- 172.21.15.1:0/2005436716 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff010068f30 0x7ff010069310 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff000009990 tx=0x7ff000009ca0 comp rx=0 tx=0).stop 2024-02-02T15:32:58.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.907+0000 7ff016200700 1 -- 172.21.15.1:0/2005436716 shutdown_connections 2024-02-02T15:32:58.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.907+0000 7ff016200700 1 --2- 172.21.15.1:0/2005436716 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff010106230 0x7ff01010dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:58.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.907+0000 7ff016200700 1 --2- 172.21.15.1:0/2005436716 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff010068f30 0x7ff010069310 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:58.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.907+0000 7ff016200700 1 --2- 172.21.15.1:0/2005436716 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff0100698e0 0x7ff010105b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:58.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.907+0000 7ff016200700 1 -- 172.21.15.1:0/2005436716 >> 172.21.15.1:0/2005436716 conn(0x7ff010077bf0 msgr2=0x7ff010078000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:32:58.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.907+0000 7ff016200700 1 -- 172.21.15.1:0/2005436716 shutdown_connections 2024-02-02T15:32:58.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.907+0000 7ff016200700 1 -- 172.21.15.1:0/2005436716 wait complete. 2024-02-02T15:32:58.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.908+0000 7ff016200700 1 Processor -- start 2024-02-02T15:32:58.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.908+0000 7ff016200700 1 -- start start 2024-02-02T15:32:58.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.908+0000 7ff016200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff010068f30 0x7ff0101991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:58.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.908+0000 7ff016200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff010106230 0x7ff010199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:58.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.908+0000 7ff016200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff01019db70 0x7ff01019dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:58.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.908+0000 7ff016200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff01019e5f0 con 0x7ff010068f30 2024-02-02T15:32:58.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.908+0000 7ff016200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff01019e760 con 0x7ff01019db70 2024-02-02T15:32:58.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.908+0000 7ff016200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff01019e920 con 0x7ff010106230 2024-02-02T15:32:58.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.909+0000 7ff00f400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff010068f30 0x7ff0101991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:58.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.909+0000 7ff00f400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff010068f30 0x7ff0101991d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44870/0 (socket says 172.21.15.1:44870) 2024-02-02T15:32:58.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.909+0000 7ff00ea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff010106230 0x7ff010199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:58.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.909+0000 7ff00f400700 1 -- 172.21.15.1:0/385095411 learned_addr learned my addr 172.21.15.1:0/385095411 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:32:58.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.909+0000 7ff00fe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff01019db70 0x7ff01019dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:58.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.909+0000 7ff00fe00700 1 -- 172.21.15.1:0/385095411 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff010106230 msgr2=0x7ff010199710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:58.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.909+0000 7ff00fe00700 1 --2- 172.21.15.1:0/385095411 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff010106230 0x7ff010199710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:58.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.909+0000 7ff00fe00700 1 -- 172.21.15.1:0/385095411 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff010068f30 msgr2=0x7ff0101991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:58.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.909+0000 7ff00fe00700 1 --2- 172.21.15.1:0/385095411 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff010068f30 0x7ff0101991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:58.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.909+0000 7ff00fe00700 1 -- 172.21.15.1:0/385095411 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0000095f0 con 0x7ff01019db70 2024-02-02T15:32:58.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.909+0000 7ff00ea00700 1 --2- 172.21.15.1:0/385095411 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff010106230 0x7ff010199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:32:58.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.910+0000 7ff00fe00700 1 --2- 172.21.15.1:0/385095411 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff01019db70 0x7ff01019dff0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ff00800efc0 tx=0x7ff00800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:32:58.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.910+0000 7ff007e00700 1 -- 172.21.15.1:0/385095411 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff008018070 con 0x7ff01019db70 2024-02-02T15:32:58.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.910+0000 7ff007e00700 1 -- 172.21.15.1:0/385095411 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff00800f040 con 0x7ff01019db70 2024-02-02T15:32:58.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.910+0000 7ff007e00700 1 -- 172.21.15.1:0/385095411 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff00800cc80 con 0x7ff01019db70 2024-02-02T15:32:58.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.910+0000 7ff016200700 1 -- 172.21.15.1:0/385095411 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0101a5e20 con 0x7ff01019db70 2024-02-02T15:32:58.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.910+0000 7ff016200700 1 -- 172.21.15.1:0/385095411 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0101a62b0 con 0x7ff01019db70 2024-02-02T15:32:58.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.913+0000 7ff007e00700 1 -- 172.21.15.1:0/385095411 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 27) v1 ==== 135200+0+0 (secure 0 0 0) 0x7ff008003a00 con 0x7ff01019db70 2024-02-02T15:32:58.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.914+0000 7ff007e00700 1 --2- 172.21.15.1:0/385095411 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7feff00a0710 0x7feff00a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:58.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.915+0000 7ff007e00700 1 -- 172.21.15.1:0/385095411 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7ff0080cbbb0 con 0x7ff01019db70 2024-02-02T15:32:58.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.915+0000 7ff00f400700 1 --2- 172.21.15.1:0/385095411 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7feff00a0710 0x7feff00a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:58.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.915+0000 7ff029c4db80 1 Processor -- start 2024-02-02T15:32:58.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.915+0000 7ff029c4db80 1 -- start start 2024-02-02T15:32:58.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.915+0000 7ff00f400700 1 --2- 172.21.15.1:0/385095411 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7feff00a0710 0x7feff00a2bd0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff000006010 tx=0x7ff00000b260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:32:58.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.916+0000 7ff029c4db80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cacdf99010 0x55cacdf993f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:58.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.916+0000 7ff029c4db80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cacdf99930 0x55cacdfa42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:58.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.916+0000 7ff029c4db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cacdfa47e0 0x55cacdfa6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:58.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.916+0000 7ff029c4db80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cacdf0d4c0 con 0x55cacdf99930 2024-02-02T15:32:58.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.916+0000 7ff029c4db80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cacdef1aa0 con 0x55cacdf99010 2024-02-02T15:32:58.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.916+0000 7ff029c4db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cacdeef460 con 0x55cacdfa47e0 2024-02-02T15:32:58.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.916+0000 7ff00f400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cacdf99010 0x55cacdf993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:58.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.916+0000 7ff00f400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cacdf99010 0x55cacdf993f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:33612/0 (socket says 172.21.15.1:33612) 2024-02-02T15:32:58.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.916+0000 7ff00f400700 1 -- 172.21.15.1:0/3327159089 learned_addr learned my addr 172.21.15.1:0/3327159089 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:32:58.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.916+0000 7ff00ea00700 1 --2- 172.21.15.1:0/3327159089 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cacdf99930 0x55cacdfa42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:58.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.916+0000 7ff00fe00700 1 --2- 172.21.15.1:0/3327159089 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cacdfa47e0 0x55cacdfa6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:58.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.916+0000 7ff00f400700 1 -- 172.21.15.1:0/3327159089 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cacdfa47e0 msgr2=0x55cacdfa6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:32:58.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.916+0000 7ff00f400700 1 --2- 172.21.15.1:0/3327159089 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cacdfa47e0 0x55cacdfa6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:58.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.916+0000 7ff00f400700 1 -- 172.21.15.1:0/3327159089 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cacdf99930 msgr2=0x55cacdfa42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:32:58.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.916+0000 7ff00f400700 1 --2- 172.21.15.1:0/3327159089 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cacdf99930 0x55cacdfa42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:58.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.916+0000 7ff00f400700 1 -- 172.21.15.1:0/3327159089 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cacdd6c3b0 con 0x55cacdf99010 2024-02-02T15:32:58.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.917+0000 7ff00f400700 1 --2- 172.21.15.1:0/3327159089 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cacdf99010 0x55cacdf993f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ff00000bea0 tx=0x7ff000020000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93b9730a997c039b server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:32:58.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.917+0000 7fefffe00700 1 -- 172.21.15.1:0/3327159089 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff00002b070 con 0x55cacdf99010 2024-02-02T15:32:58.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.917+0000 7fefffe00700 1 -- 172.21.15.1:0/3327159089 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff00001e040 con 0x55cacdf99010 2024-02-02T15:32:58.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.917+0000 7fefffe00700 1 -- 172.21.15.1:0/3327159089 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff00000f6a0 con 0x55cacdf99010 2024-02-02T15:32:58.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.917+0000 7ff029c4db80 1 -- 172.21.15.1:0/3327159089 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cacdf99010 msgr2=0x55cacdf993f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:58.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.917+0000 7ff029c4db80 1 --2- 172.21.15.1:0/3327159089 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cacdf99010 0x55cacdf993f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ff00000bea0 tx=0x7ff000020000 comp rx=0 tx=0).stop 2024-02-02T15:32:58.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.918+0000 7ff029c4db80 1 -- 172.21.15.1:0/3327159089 shutdown_connections 2024-02-02T15:32:58.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.918+0000 7ff029c4db80 1 --2- 172.21.15.1:0/3327159089 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cacdfa47e0 0x55cacdfa6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:58.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.918+0000 7ff029c4db80 1 --2- 172.21.15.1:0/3327159089 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cacdf99010 0x55cacdf993f0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:58.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.918+0000 7ff029c4db80 1 --2- 172.21.15.1:0/3327159089 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cacdf99930 0x55cacdfa42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:58.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.918+0000 7ff029c4db80 1 -- 172.21.15.1:0/3327159089 >> 172.21.15.1:0/3327159089 conn(0x55cacdef8db0 msgr2=0x55cacdf953b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:32:58.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.918+0000 7ff029c4db80 1 -- 172.21.15.1:0/3327159089 shutdown_connections 2024-02-02T15:32:58.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.918+0000 7ff029c4db80 1 -- 172.21.15.1:0/3327159089 wait complete. 2024-02-02T15:32:58.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.918+0000 7ff029c4db80 1 -- start start 2024-02-02T15:32:58.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.918+0000 7ff029c4db80 1 Processor -- start 2024-02-02T15:32:58.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.919+0000 7ff029c4db80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cacdf99010 0x55cacdfa4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:58.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.919+0000 7ff029c4db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cacdf99930 0x55cacdfa1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:58.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.919+0000 7ff029c4db80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cacdfa47e0 0x55cacdfa1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:58.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.919+0000 7ff029c4db80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cacdeead50 con 0x55cacdf99010 2024-02-02T15:32:58.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.919+0000 7ff029c4db80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cacde86d80 con 0x55cacdfa47e0 2024-02-02T15:32:58.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.919+0000 7ff029c4db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cacde86a20 con 0x55cacdf99930 2024-02-02T15:32:58.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.919+0000 7ff029c4db80 10 client.0 _ll_register_callbacks cb 0x55cacde88290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:32:58.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.919+0000 7ff00fe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cacdfa47e0 0x55cacdfa1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:58.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.919+0000 7ff00f400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cacdf99010 0x55cacdfa4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:58.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.919+0000 7ff00fe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cacdfa47e0 0x55cacdfa1aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:33614/0 (socket says 172.21.15.1:33614) 2024-02-02T15:32:58.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.919+0000 7ff00fe00700 1 -- 172.21.15.1:0/892401754 learned_addr learned my addr 172.21.15.1:0/892401754 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:32:58.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.919+0000 7ff00ea00700 1 --2- 172.21.15.1:0/892401754 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cacdf99930 0x55cacdfa1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:58.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.920+0000 7ff00fe00700 1 -- 172.21.15.1:0/892401754 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cacdf99930 msgr2=0x55cacdfa1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:32:58.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.920+0000 7ff00fe00700 1 --2- 172.21.15.1:0/892401754 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cacdf99930 0x55cacdfa1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:58.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.920+0000 7ff00fe00700 1 -- 172.21.15.1:0/892401754 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cacdf99010 msgr2=0x55cacdfa4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:32:58.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.920+0000 7ff00fe00700 1 --2- 172.21.15.1:0/892401754 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cacdf99010 0x55cacdfa4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:58.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.920+0000 7ff00fe00700 1 -- 172.21.15.1:0/892401754 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff01010e640 con 0x55cacdfa47e0 2024-02-02T15:32:58.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.920+0000 7ff00ea00700 1 --2- 172.21.15.1:0/892401754 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cacdf99930 0x55cacdfa1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:32:58.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.920+0000 7ff00fe00700 1 --2- 172.21.15.1:0/892401754 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cacdfa47e0 0x55cacdfa1aa0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ff008098a10 tx=0x7ff008004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48d489134f95a9cb server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:32:58.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.920+0000 7feffcc00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:32:58.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.920+0000 7feffcc00700 1 -- 172.21.15.1:0/892401754 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0080b82b0 con 0x55cacdfa47e0 2024-02-02T15:32:58.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.920+0000 7feffcc00700 1 -- 172.21.15.1:0/892401754 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7ff0080b1200 con 0x55cacdfa47e0 2024-02-02T15:32:58.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.920+0000 7ff029c4db80 1 -- 172.21.15.1:0/892401754 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cacdf8d0f0 con 0x55cacdfa47e0 2024-02-02T15:32:58.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.920+0000 7feffcc00700 1 -- 172.21.15.1:0/892401754 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff008098d10 con 0x55cacdfa47e0 2024-02-02T15:32:58.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.920+0000 7ff029c4db80 10 client.27164 fetch_fsmap learned FSMap version 38 2024-02-02T15:32:58.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.920+0000 7ff029c4db80 1 -- 172.21.15.1:0/892401754 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55cacdd18b60 con 0x55cacdfa47e0 2024-02-02T15:32:58.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.920+0000 7feffcc00700 1 -- 172.21.15.1:0/892401754 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff00807b020 con 0x55cacdfa47e0 2024-02-02T15:32:58.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.921+0000 7feffcc00700 1 -- 172.21.15.1:0/892401754 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 38) v1 ==== 7486+0+0 (secure 0 0 0) 0x7ff008099d20 con 0x55cacdfa47e0 2024-02-02T15:32:58.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.921+0000 7ff029c4db80 10 client.27164 fetch_fsmap finished waiting for FSMap version 38 2024-02-02T15:32:58.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.921+0000 7ff029c4db80 10 client.27164 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:32:58.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.921+0000 7ff029c4db80 20 client.27164 populate_metadata read hostname 'smithi001' 2024-02-02T15:32:58.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.921+0000 7ff029c4db80 1 --2- 172.21.15.1:0/892401754 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55cacdfb4090 0x55cacdfb64b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:58.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.921+0000 7ff029c4db80 4 client.27164 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:32:58.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.921+0000 7ff029c4db80 1 -- 172.21.15.1:0/892401754 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cacdee3dd0 con 0x55cacdfb4090 2024-02-02T15:32:58.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.921+0000 7ff00f400700 1 --2- 172.21.15.1:0/892401754 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55cacdfb4090 0x55cacdfb64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:58.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.921+0000 7feffcc00700 10 client.27164 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:32:58.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.921+0000 7ff00f400700 1 --2- 172.21.15.1:0/892401754 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55cacdfb4090 0x55cacdfb64b0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=159f86bf621149b0 server_cookie=bd482f4f27162ad4 in_seq=0 out_seq=0 2024-02-02T15:32:58.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.922+0000 7feffcc00700 1 -- 172.21.15.1:0/892401754 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55cacdee3dd0 con 0x55cacdfb4090 2024-02-02T15:32:58.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.922+0000 7feffcc00700 10 client.27164 handle_command_reply: tid=0 2024-02-02T15:32:58.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.922+0000 7ff029c4db80 1 client.27164 shutdown 2024-02-02T15:32:58.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.922+0000 7ff029c4db80 1 -- 172.21.15.1:0/892401754 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cacdfa47e0 msgr2=0x55cacdfa1aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:58.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.922+0000 7ff029c4db80 1 --2- 172.21.15.1:0/892401754 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cacdfa47e0 0x55cacdfa1aa0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ff008098a10 tx=0x7ff008004ad0 comp rx=0 tx=0).stop 2024-02-02T15:32:58.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.923+0000 7ff029c4db80 1 -- 172.21.15.1:0/892401754 shutdown_connections 2024-02-02T15:32:58.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.923+0000 7ff029c4db80 1 --2- 172.21.15.1:0/892401754 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55cacdfb4090 0x55cacdfb64b0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:58.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.923+0000 7ff029c4db80 1 --2- 172.21.15.1:0/892401754 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cacdfa47e0 0x55cacdfa1aa0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:58.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.923+0000 7ff029c4db80 1 --2- 172.21.15.1:0/892401754 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cacdf99010 0x55cacdfa4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:58.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.923+0000 7ff029c4db80 1 --2- 172.21.15.1:0/892401754 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cacdf99930 0x55cacdfa1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:58.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.923+0000 7ff029c4db80 1 -- 172.21.15.1:0/892401754 >> 172.21.15.1:0/892401754 conn(0x55cacdef8db0 msgr2=0x55cacdf94fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:32:58.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.923+0000 7feffcc00700 0 client.27164 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:32:58.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.923+0000 7ff029c4db80 1 -- 172.21.15.1:0/892401754 shutdown_connections 2024-02-02T15:32:58.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.923+0000 7ff029c4db80 1 -- 172.21.15.1:0/892401754 wait complete. 2024-02-02T15:32:58.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.923+0000 7ff029c4db80 20 client.27164 trim_cache size 0 max 16384 2024-02-02T15:32:58.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.943+0000 7ff029c4db80 1 Processor -- start 2024-02-02T15:32:58.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.943+0000 7ff029c4db80 1 -- start start 2024-02-02T15:32:58.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.943+0000 7ff029c4db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cacdf99010 0x55cacdfab2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:58.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.943+0000 7ff029c4db80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cacdf99930 0x55cacdfa0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:58.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.943+0000 7ff029c4db80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cacdfa47e0 0x55cacdf9bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:58.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.943+0000 7ff029c4db80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cacdd6e6a0 con 0x55cacdf99930 2024-02-02T15:32:58.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.943+0000 7ff029c4db80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cacdf9faa0 con 0x55cacdfa47e0 2024-02-02T15:32:58.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.943+0000 7ff029c4db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cacdf94220 con 0x55cacdf99010 2024-02-02T15:32:58.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.944+0000 7ff00fe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cacdfa47e0 0x55cacdf9bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:58.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.944+0000 7ff00ea00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cacdf99930 0x55cacdfa0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:58.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.944+0000 7ff00fe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cacdfa47e0 0x55cacdf9bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:33626/0 (socket says 172.21.15.1:33626) 2024-02-02T15:32:58.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.944+0000 7ff00fe00700 1 -- 172.21.15.1:0/460377869 learned_addr learned my addr 172.21.15.1:0/460377869 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:32:58.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.944+0000 7ff00f400700 1 --2- 172.21.15.1:0/460377869 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cacdf99010 0x55cacdfab2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:58.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.944+0000 7ff00fe00700 1 -- 172.21.15.1:0/460377869 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cacdf99010 msgr2=0x55cacdfab2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:32:58.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.944+0000 7ff00fe00700 1 --2- 172.21.15.1:0/460377869 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cacdf99010 0x55cacdfab2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:58.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.944+0000 7ff00fe00700 1 -- 172.21.15.1:0/460377869 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cacdf99930 msgr2=0x55cacdfa0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:32:58.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.944+0000 7ff00fe00700 1 --2- 172.21.15.1:0/460377869 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cacdf99930 0x55cacdfa0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:58.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.944+0000 7ff00fe00700 1 -- 172.21.15.1:0/460377869 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cacdeb0ea0 con 0x55cacdfa47e0 2024-02-02T15:32:58.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.944+0000 7ff00fe00700 1 --2- 172.21.15.1:0/460377869 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cacdfa47e0 0x55cacdf9bab0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ff0080a07c0 tx=0x7ff0080972c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5375ed137896422 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:32:58.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.945+0000 7ff005200700 1 -- 172.21.15.1:0/460377869 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0080984a0 con 0x55cacdfa47e0 2024-02-02T15:32:58.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.945+0000 7ff005200700 1 -- 172.21.15.1:0/460377869 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff00807b070 con 0x55cacdfa47e0 2024-02-02T15:32:58.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.945+0000 7ff005200700 1 -- 172.21.15.1:0/460377869 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0080b8310 con 0x55cacdfa47e0 2024-02-02T15:32:58.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.945+0000 7ff029c4db80 1 -- 172.21.15.1:0/460377869 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cacdfa47e0 msgr2=0x55cacdf9bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:58.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.945+0000 7ff029c4db80 1 --2- 172.21.15.1:0/460377869 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cacdfa47e0 0x55cacdf9bab0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ff0080a07c0 tx=0x7ff0080972c0 comp rx=0 tx=0).stop 2024-02-02T15:32:58.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.945+0000 7ff029c4db80 1 -- 172.21.15.1:0/460377869 shutdown_connections 2024-02-02T15:32:58.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.945+0000 7ff029c4db80 1 --2- 172.21.15.1:0/460377869 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cacdfa47e0 0x55cacdf9bab0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:58.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.945+0000 7ff029c4db80 1 --2- 172.21.15.1:0/460377869 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cacdf99010 0x55cacdfab2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:58.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.945+0000 7ff029c4db80 1 --2- 172.21.15.1:0/460377869 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cacdf99930 0x55cacdfa0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:58.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.945+0000 7ff029c4db80 1 -- 172.21.15.1:0/460377869 >> 172.21.15.1:0/460377869 conn(0x55cacdef8db0 msgr2=0x55cacdf9c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:32:58.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.945+0000 7ff029c4db80 1 -- 172.21.15.1:0/460377869 shutdown_connections 2024-02-02T15:32:58.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.946+0000 7ff029c4db80 1 -- 172.21.15.1:0/460377869 wait complete. 2024-02-02T15:32:58.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.946+0000 7ff029c4db80 1 -- start start 2024-02-02T15:32:58.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.946+0000 7ff029c4db80 1 Processor -- start 2024-02-02T15:32:58.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.946+0000 7ff029c4db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cacdf99010 0x55cacdfa9000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:58.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.947+0000 7ff029c4db80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cacdf99930 0x55cacdfa9540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:58.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.947+0000 7ff029c4db80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cacdfa47e0 0x55cacdffebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:58.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.947+0000 7ff029c4db80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cacdfff0f0 con 0x55cacdf99930 2024-02-02T15:32:58.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.947+0000 7ff029c4db80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cacdfff260 con 0x55cacdfa47e0 2024-02-02T15:32:58.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.947+0000 7ff029c4db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cacdfff3d0 con 0x55cacdf99010 2024-02-02T15:32:58.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.947+0000 7ff029c4db80 10 client.0 _ll_register_callbacks cb 0x55cacde88290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:32:58.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.947+0000 7ff00f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cacdf99010 0x55cacdfa9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:58.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.947+0000 7ff00f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cacdf99010 0x55cacdfa9000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:39486/0 (socket says 172.21.15.1:39486) 2024-02-02T15:32:58.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.947+0000 7ff00f400700 1 -- 172.21.15.1:0/2655921676 learned_addr learned my addr 172.21.15.1:0/2655921676 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:32:58.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.947+0000 7ff00fe00700 1 --2- 172.21.15.1:0/2655921676 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cacdfa47e0 0x55cacdffebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:58.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.947+0000 7ff00ea00700 1 --2- 172.21.15.1:0/2655921676 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cacdf99930 0x55cacdfa9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:58.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.947+0000 7ff00f400700 1 -- 172.21.15.1:0/2655921676 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cacdfa47e0 msgr2=0x55cacdffebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:32:58.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.947+0000 7ff00f400700 1 --2- 172.21.15.1:0/2655921676 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cacdfa47e0 0x55cacdffebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:58.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.947+0000 7ff00f400700 1 -- 172.21.15.1:0/2655921676 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cacdf99930 msgr2=0x55cacdfa9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:32:58.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.947+0000 7ff00f400700 1 --2- 172.21.15.1:0/2655921676 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cacdf99930 0x55cacdfa9540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:58.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.947+0000 7ff00f400700 1 -- 172.21.15.1:0/2655921676 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cacdd18b60 con 0x55cacdf99010 2024-02-02T15:32:58.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.947+0000 7ff00fe00700 1 --2- 172.21.15.1:0/2655921676 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cacdfa47e0 0x55cacdffebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:32:58.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.948+0000 7ff00f400700 1 --2- 172.21.15.1:0/2655921676 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cacdf99010 0x55cacdfa9000 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff000031040 tx=0x7ff000009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e96dc797a6e1f9de server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:32:58.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.948+0000 7feffcc00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:32:58.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.948+0000 7feffcc00700 1 -- 172.21.15.1:0/2655921676 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff00002b040 con 0x55cacdf99010 2024-02-02T15:32:58.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.948+0000 7feffcc00700 1 -- 172.21.15.1:0/2655921676 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7ff00003c960 con 0x55cacdf99010 2024-02-02T15:32:58.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.948+0000 7ff029c4db80 1 -- 172.21.15.1:0/2655921676 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff00001e040 con 0x55cacdf99010 2024-02-02T15:32:58.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.948+0000 7feffcc00700 1 -- 172.21.15.1:0/2655921676 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff00003cb20 con 0x55cacdf99010 2024-02-02T15:32:58.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.948+0000 7ff029c4db80 10 client.27169 fetch_fsmap learned FSMap version 38 2024-02-02T15:32:58.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.948+0000 7ff029c4db80 1 -- 172.21.15.1:0/2655921676 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55cacdd70cd0 con 0x55cacdf99010 2024-02-02T15:32:58.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.948+0000 7feffcc00700 1 -- 172.21.15.1:0/2655921676 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff00001e040 con 0x55cacdf99010 2024-02-02T15:32:58.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.948+0000 7feffcc00700 1 -- 172.21.15.1:0/2655921676 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 38) v1 ==== 7486+0+0 (secure 0 0 0) 0x7ff000045590 con 0x55cacdf99010 2024-02-02T15:32:58.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.949+0000 7ff029c4db80 10 client.27169 fetch_fsmap finished waiting for FSMap version 38 2024-02-02T15:32:58.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.949+0000 7ff029c4db80 10 client.27169 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:32:58.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.949+0000 7ff029c4db80 20 client.27169 populate_metadata read hostname 'smithi001' 2024-02-02T15:32:58.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.949+0000 7ff029c4db80 1 --2- 172.21.15.1:0/2655921676 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55cace006780 0x55cace008ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:58.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.949+0000 7ff029c4db80 4 client.27169 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2024-02-02T15:32:58.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.949+0000 7ff029c4db80 1 -- 172.21.15.1:0/2655921676 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55cacdea9f50 con 0x55cace006780 2024-02-02T15:32:58.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.949+0000 7ff00ea00700 1 --2- 172.21.15.1:0/2655921676 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55cace006780 0x55cace008ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:58.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.949+0000 7ff00ea00700 1 --2- 172.21.15.1:0/2655921676 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55cace006780 0x55cace008ba0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93f78b77f1ef1c02 server_cookie=e5b2780db9095a26 in_seq=0 out_seq=0 2024-02-02T15:32:58.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.949+0000 7feffcc00700 10 client.27169 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:32:58.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.949+0000 7feffcc00700 1 -- 172.21.15.1:0/2655921676 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55cacdea9f50 con 0x55cace006780 2024-02-02T15:32:58.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.949+0000 7feffcc00700 10 client.27169 handle_command_reply: tid=0 2024-02-02T15:32:58.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.949+0000 7ff029c4db80 1 client.27169 shutdown 2024-02-02T15:32:58.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.950+0000 7ff029c4db80 1 -- 172.21.15.1:0/2655921676 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cacdf99010 msgr2=0x55cacdfa9000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:58.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.950+0000 7ff029c4db80 1 --2- 172.21.15.1:0/2655921676 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cacdf99010 0x55cacdfa9000 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff000031040 tx=0x7ff000009ca0 comp rx=0 tx=0).stop 2024-02-02T15:32:58.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.950+0000 7ff029c4db80 1 -- 172.21.15.1:0/2655921676 shutdown_connections 2024-02-02T15:32:58.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.950+0000 7ff029c4db80 1 --2- 172.21.15.1:0/2655921676 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55cace006780 0x55cace008ba0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:58.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.950+0000 7ff029c4db80 1 --2- 172.21.15.1:0/2655921676 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cacdfa47e0 0x55cacdffebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:58.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.950+0000 7ff029c4db80 1 --2- 172.21.15.1:0/2655921676 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cacdf99010 0x55cacdfa9000 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:58.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.950+0000 7feffcc00700 0 client.27169 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:32:58.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.950+0000 7ff029c4db80 1 --2- 172.21.15.1:0/2655921676 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cacdf99930 0x55cacdfa9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:58.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.950+0000 7ff029c4db80 1 -- 172.21.15.1:0/2655921676 >> 172.21.15.1:0/2655921676 conn(0x55cacdef8db0 msgr2=0x55cacdf94e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:32:58.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.950+0000 7ff029c4db80 1 -- 172.21.15.1:0/2655921676 shutdown_connections 2024-02-02T15:32:58.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.950+0000 7ff029c4db80 1 -- 172.21.15.1:0/2655921676 wait complete. 2024-02-02T15:32:58.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.950+0000 7ff029c4db80 20 client.27169 trim_cache size 0 max 16384 2024-02-02T15:32:58.955 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:32:58.955 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2024-02-02T15:32:58.955 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2024-02-02T15:32:58.955 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:32:58.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.951+0000 7ff016200700 1 -- 172.21.15.1:0/385095411 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7feff00a0710 msgr2=0x7feff00a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:58.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.951+0000 7ff016200700 1 --2- 172.21.15.1:0/385095411 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7feff00a0710 0x7feff00a2bd0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff000006010 tx=0x7ff00000b260 comp rx=0 tx=0).stop 2024-02-02T15:32:58.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.951+0000 7ff016200700 1 -- 172.21.15.1:0/385095411 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff01019db70 msgr2=0x7ff01019dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:58.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.951+0000 7ff016200700 1 --2- 172.21.15.1:0/385095411 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff01019db70 0x7ff01019dff0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ff00800efc0 tx=0x7ff00800c3a0 comp rx=0 tx=0).stop 2024-02-02T15:32:58.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.951+0000 7ff016200700 1 -- 172.21.15.1:0/385095411 shutdown_connections 2024-02-02T15:32:58.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.951+0000 7ff016200700 1 --2- 172.21.15.1:0/385095411 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7feff00a0710 0x7feff00a2bd0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:58.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.951+0000 7ff016200700 1 --2- 172.21.15.1:0/385095411 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff01019db70 0x7ff01019dff0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:58.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.951+0000 7ff016200700 1 --2- 172.21.15.1:0/385095411 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff010068f30 0x7ff0101991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:58.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.951+0000 7ff016200700 1 --2- 172.21.15.1:0/385095411 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff010106230 0x7ff010199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:58.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.951+0000 7ff016200700 1 -- 172.21.15.1:0/385095411 >> 172.21.15.1:0/385095411 conn(0x7ff010077bf0 msgr2=0x7ff0100fe950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:32:58.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.951+0000 7ff016200700 1 -- 172.21.15.1:0/385095411 shutdown_connections 2024-02-02T15:32:58.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:58.951+0000 7ff016200700 1 -- 172.21.15.1:0/385095411 wait complete. 2024-02-02T15:32:58.970 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-02T15:32:58.971 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T15:32:59.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.178+0000 7f3fd0600700 1 -- 172.21.15.1:0/1248429150 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fc81018d0 msgr2=0x7f3fc8105910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:59.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.178+0000 7f3fd0600700 1 --2- 172.21.15.1:0/1248429150 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fc81018d0 0x7f3fc8105910 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f3fb0009990 tx=0x7f3fb0009ca0 comp rx=0 tx=0).stop 2024-02-02T15:32:59.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.178+0000 7f3fd0600700 1 -- 172.21.15.1:0/1248429150 shutdown_connections 2024-02-02T15:32:59.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.178+0000 7f3fd0600700 1 --2- 172.21.15.1:0/1248429150 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3fc8106040 0x7f3fc810dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:59.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.178+0000 7f3fd0600700 1 --2- 172.21.15.1:0/1248429150 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3fc8100f20 0x7f3fc8101300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:59.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.178+0000 7f3fd0600700 1 --2- 172.21.15.1:0/1248429150 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fc81018d0 0x7f3fc8105910 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:59.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.178+0000 7f3fd0600700 1 -- 172.21.15.1:0/1248429150 >> 172.21.15.1:0/1248429150 conn(0x7f3fc80fc7a0 msgr2=0x7f3fc80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:32:59.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.178+0000 7f3fd0600700 1 -- 172.21.15.1:0/1248429150 shutdown_connections 2024-02-02T15:32:59.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.178+0000 7f3fd0600700 1 -- 172.21.15.1:0/1248429150 wait complete. 2024-02-02T15:32:59.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.179+0000 7f3fd0600700 1 Processor -- start 2024-02-02T15:32:59.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.180+0000 7f3fd0600700 1 -- start start 2024-02-02T15:32:59.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.180+0000 7f3fd0600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3fc8100f20 0x7f3fc8199090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:59.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.180+0000 7f3fd0600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3fc8106040 0x7f3fc81995d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:59.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.180+0000 7f3fd0600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fc819da30 0x7f3fc819deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:59.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.180+0000 7f3fd0600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc819e4b0 con 0x7f3fc8106040 2024-02-02T15:32:59.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.180+0000 7f3fd0600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc819e620 con 0x7f3fc8100f20 2024-02-02T15:32:59.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.180+0000 7f3fd0600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc819e7e0 con 0x7f3fc819da30 2024-02-02T15:32:59.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.180+0000 7f3fcdc00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3fc8100f20 0x7f3fc8199090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:59.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.180+0000 7f3fcd200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3fc8106040 0x7f3fc81995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:59.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.180+0000 7f3fcdc00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3fc8100f20 0x7f3fc8199090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:33656/0 (socket says 172.21.15.1:33656) 2024-02-02T15:32:59.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.180+0000 7f3fcdc00700 1 -- 172.21.15.1:0/1170264682 learned_addr learned my addr 172.21.15.1:0/1170264682 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:32:59.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.180+0000 7f3fce600700 1 --2- 172.21.15.1:0/1170264682 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fc819da30 0x7f3fc819deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:59.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.181+0000 7f3fcdc00700 1 -- 172.21.15.1:0/1170264682 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fc819da30 msgr2=0x7f3fc819deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:59.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.181+0000 7f3fcdc00700 1 --2- 172.21.15.1:0/1170264682 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fc819da30 0x7f3fc819deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:59.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.181+0000 7f3fcdc00700 1 -- 172.21.15.1:0/1170264682 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3fc8106040 msgr2=0x7f3fc81995d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:59.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.181+0000 7f3fcdc00700 1 --2- 172.21.15.1:0/1170264682 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3fc8106040 0x7f3fc81995d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:59.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.181+0000 7f3fcdc00700 1 -- 172.21.15.1:0/1170264682 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fb00095f0 con 0x7f3fc8100f20 2024-02-02T15:32:59.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.181+0000 7f3fcdc00700 1 --2- 172.21.15.1:0/1170264682 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3fc8100f20 0x7f3fc8199090 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f3fb800e9a0 tx=0x7f3fb800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:32:59.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.181+0000 7f3fc6000700 1 -- 172.21.15.1:0/1170264682 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb8009790 con 0x7f3fc8100f20 2024-02-02T15:32:59.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.181+0000 7f3fd0600700 1 -- 172.21.15.1:0/1170264682 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fc81a5d40 con 0x7f3fc8100f20 2024-02-02T15:32:59.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.181+0000 7f3fc6000700 1 -- 172.21.15.1:0/1170264682 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3fb8010430 con 0x7f3fc8100f20 2024-02-02T15:32:59.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.181+0000 7f3fc6000700 1 -- 172.21.15.1:0/1170264682 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb8018680 con 0x7f3fc8100f20 2024-02-02T15:32:59.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.181+0000 7f3fd0600700 1 -- 172.21.15.1:0/1170264682 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fc81a6210 con 0x7f3fc8100f20 2024-02-02T15:32:59.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.182+0000 7f3fd0600700 1 -- 172.21.15.1:0/1170264682 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fc8067bc0 con 0x7f3fc8100f20 2024-02-02T15:32:59.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.184+0000 7f3fc6000700 1 -- 172.21.15.1:0/1170264682 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 27) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f3fb80105a0 con 0x7f3fc8100f20 2024-02-02T15:32:59.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.186+0000 7f3fc6000700 1 --2- 172.21.15.1:0/1170264682 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f3fa80a0710 0x7f3fa80a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:59.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.186+0000 7f3fc6000700 1 -- 172.21.15.1:0/1170264682 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f3fb8014070 con 0x7f3fc8100f20 2024-02-02T15:32:59.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.186+0000 7f3fcd200700 1 --2- 172.21.15.1:0/1170264682 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f3fa80a0710 0x7f3fa80a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:59.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.186+0000 7f3fcd200700 1 --2- 172.21.15.1:0/1170264682 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f3fa80a0710 0x7f3fa80a2bd0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3fc819a6a0 tx=0x7f3fb0005fb0 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:32:59.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.188+0000 7f3fc6000700 1 -- 172.21.15.1:0/1170264682 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f3fb8095970 con 0x7f3fc8100f20 2024-02-02T15:32:59.237 INFO:tasks.workunit.client.0.smithi001.stdout: 1 393112 8.90 MB/sec execute 153 sec latency 24.133 ms 2024-02-02T15:32:59.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.473+0000 7f3fd0600700 1 -- 172.21.15.1:0/1170264682 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3fc819a3e0 con 0x7f3fc8100f20 2024-02-02T15:32:59.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.474+0000 7f3fc6000700 1 -- 172.21.15.1:0/1170264682 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+10873 (secure 0 0 0) 0x7f3fb8098f90 con 0x7f3fc8100f20 2024-02-02T15:32:59.474 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:32:59.474 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2922240448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2922240448},{"type":"v1","addr":"172.21.15.1:6835","nonce":2922240448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2729497431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2729497431},{"type":"v1","addr":"172.21.15.1:6837","nonce":2729497431}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":38,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-02T15:24:47.761435+0000","modified":"2024-02-02T15:32:42.392323+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24529,"mds_2":24518,"mds_3":24514,"mds_4":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14517":{"gid":14517,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/1218303683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":1218303683},{"type":"v1","addr":"172.21.15.1:6839","nonce":1218303683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14529":{"gid":14529,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/2059197496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2059197496},{"type":"v1","addr":"172.21.15.1:6841","nonce":2059197496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/7648986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":7648986},{"type":"v1","addr":"172.21.15.142:6833","nonce":7648986}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/3482746094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3482746094},{"type":"v1","addr":"172.21.15.71:6833","nonce":3482746094}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/388207814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":388207814},{"type":"v1","addr":"172.21.15.71:6835","nonce":388207814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6835/1052606547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1052606547},{"type":"v1","addr":"172.21.15.142:6835","nonce":1052606547}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.142:6837/2870289019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2870289019},{"type":"v1","addr":"172.21.15.142:6837","nonce":2870289019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.71:6837/628016199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":628016199},{"type":"v1","addr":"172.21.15.71:6837","nonce":628016199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.142:6839/321675421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":321675421},{"type":"v1","addr":"172.21.15.142:6839","nonce":321675421}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/1268086493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1268086493},{"type":"v1","addr":"172.21.15.71:6839","nonce":1268086493}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-02T15:32:59.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.477+0000 7f3fd0600700 1 -- 172.21.15.1:0/1170264682 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f3fa80a0710 msgr2=0x7f3fa80a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:59.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.477+0000 7f3fd0600700 1 --2- 172.21.15.1:0/1170264682 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f3fa80a0710 0x7f3fa80a2bd0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f3fc819a6a0 tx=0x7f3fb0005fb0 comp rx=0 tx=0).stop 2024-02-02T15:32:59.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.477+0000 7f3fd0600700 1 -- 172.21.15.1:0/1170264682 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3fc8100f20 msgr2=0x7f3fc8199090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:59.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.477+0000 7f3fd0600700 1 --2- 172.21.15.1:0/1170264682 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3fc8100f20 0x7f3fc8199090 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f3fb800e9a0 tx=0x7f3fb800ecb0 comp rx=0 tx=0).stop 2024-02-02T15:32:59.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.477+0000 7f3fd0600700 1 -- 172.21.15.1:0/1170264682 shutdown_connections 2024-02-02T15:32:59.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.477+0000 7f3fd0600700 1 --2- 172.21.15.1:0/1170264682 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f3fa80a0710 0x7f3fa80a2bd0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:59.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.477+0000 7f3fd0600700 1 --2- 172.21.15.1:0/1170264682 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fc819da30 0x7f3fc819deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:59.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.477+0000 7f3fd0600700 1 --2- 172.21.15.1:0/1170264682 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3fc8100f20 0x7f3fc8199090 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:59.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.477+0000 7f3fd0600700 1 --2- 172.21.15.1:0/1170264682 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3fc8106040 0x7f3fc81995d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:59.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.477+0000 7f3fd0600700 1 -- 172.21.15.1:0/1170264682 >> 172.21.15.1:0/1170264682 conn(0x7f3fc80fc7a0 msgr2=0x7f3fc80fd410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:32:59.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.477+0000 7f3fd0600700 1 -- 172.21.15.1:0/1170264682 shutdown_connections 2024-02-02T15:32:59.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.477+0000 7f3fd0600700 1 -- 172.21.15.1:0/1170264682 wait complete. 2024-02-02T15:32:59.478 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 38 2024-02-02T15:32:59.496 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-02-02T15:32:59.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:32:59 smithi142 ceph-mon[45728]: pgmap v182: 129 pgs: 129 active+clean; 241 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 207 op/s 2024-02-02T15:32:59.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:32:59 smithi001 ceph-mon[35470]: pgmap v182: 129 pgs: 129 active+clean; 241 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 207 op/s 2024-02-02T15:32:59.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.712+0000 7ff4a7800700 1 -- 172.21.15.1:0/1989424244 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4a0101000 msgr2=0x7ff4a01013e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:59.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.712+0000 7ff4a7800700 1 --2- 172.21.15.1:0/1989424244 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4a0101000 0x7ff4a01013e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff490009990 tx=0x7ff490009ca0 comp rx=0 tx=0).stop 2024-02-02T15:32:59.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.713+0000 7ff4a7800700 1 -- 172.21.15.1:0/1989424244 shutdown_connections 2024-02-02T15:32:59.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.713+0000 7ff4a7800700 1 --2- 172.21.15.1:0/1989424244 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4a0106120 0x7ff4a010dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:59.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.713+0000 7ff4a7800700 1 --2- 172.21.15.1:0/1989424244 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4a0101000 0x7ff4a01013e0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:59.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.713+0000 7ff4a7800700 1 --2- 172.21.15.1:0/1989424244 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff4a01019b0 0x7ff4a01059f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:59.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.713+0000 7ff4a7800700 1 -- 172.21.15.1:0/1989424244 >> 172.21.15.1:0/1989424244 conn(0x7ff4a00fc860 msgr2=0x7ff4a00fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:32:59.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.713+0000 7ff4a7800700 1 -- 172.21.15.1:0/1989424244 shutdown_connections 2024-02-02T15:32:59.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.713+0000 7ff4a7800700 1 -- 172.21.15.1:0/1989424244 wait complete. 2024-02-02T15:32:59.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.714+0000 7ff4a7800700 1 Processor -- start 2024-02-02T15:32:59.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.714+0000 7ff4a7800700 1 -- start start 2024-02-02T15:32:59.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.715+0000 7ff4a7800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4a0101000 0x7ff4a01991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:59.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.715+0000 7ff4a7800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4a0106120 0x7ff4a0199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:59.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.715+0000 7ff4a7800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff4a019db70 0x7ff4a019dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:59.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.715+0000 7ff4a7800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a019e5f0 con 0x7ff4a019db70 2024-02-02T15:32:59.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.715+0000 7ff4a7800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a019e760 con 0x7ff4a0101000 2024-02-02T15:32:59.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.715+0000 7ff4a7800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a019e920 con 0x7ff4a0106120 2024-02-02T15:32:59.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.715+0000 7ff4a4e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4a0101000 0x7ff4a01991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:59.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.715+0000 7ff4a4e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4a0101000 0x7ff4a01991d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:33664/0 (socket says 172.21.15.1:33664) 2024-02-02T15:32:59.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.715+0000 7ff4a4e00700 1 -- 172.21.15.1:0/770989056 learned_addr learned my addr 172.21.15.1:0/770989056 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:32:59.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.715+0000 7ff49fe00700 1 --2- 172.21.15.1:0/770989056 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4a0106120 0x7ff4a0199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:59.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.715+0000 7ff4a5800700 1 --2- 172.21.15.1:0/770989056 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff4a019db70 0x7ff4a019dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:59.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.716+0000 7ff4a4e00700 1 -- 172.21.15.1:0/770989056 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4a0106120 msgr2=0x7ff4a0199710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:59.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.716+0000 7ff4a4e00700 1 --2- 172.21.15.1:0/770989056 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4a0106120 0x7ff4a0199710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:59.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.716+0000 7ff4a4e00700 1 -- 172.21.15.1:0/770989056 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff4a019db70 msgr2=0x7ff4a019dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:59.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.716+0000 7ff4a4e00700 1 --2- 172.21.15.1:0/770989056 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff4a019db70 0x7ff4a019dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:59.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.716+0000 7ff4a4e00700 1 -- 172.21.15.1:0/770989056 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4900095f0 con 0x7ff4a0101000 2024-02-02T15:32:59.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.716+0000 7ff49fe00700 1 --2- 172.21.15.1:0/770989056 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4a0106120 0x7ff4a0199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:32:59.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.716+0000 7ff4a4e00700 1 --2- 172.21.15.1:0/770989056 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4a0101000 0x7ff4a01991d0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff490005910 tx=0x7ff490005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:32:59.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.716+0000 7ff49d600700 1 -- 172.21.15.1:0/770989056 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49001d070 con 0x7ff4a0101000 2024-02-02T15:32:59.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.716+0000 7ff4a7800700 1 -- 172.21.15.1:0/770989056 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4a01a5e20 con 0x7ff4a0101000 2024-02-02T15:32:59.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.716+0000 7ff49d600700 1 -- 172.21.15.1:0/770989056 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff490022470 con 0x7ff4a0101000 2024-02-02T15:32:59.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.716+0000 7ff49d600700 1 -- 172.21.15.1:0/770989056 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff490017650 con 0x7ff4a0101000 2024-02-02T15:32:59.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.716+0000 7ff4a7800700 1 -- 172.21.15.1:0/770989056 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4a01a61f0 con 0x7ff4a0101000 2024-02-02T15:32:59.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.719+0000 7ff49d600700 1 -- 172.21.15.1:0/770989056 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 27) v1 ==== 135200+0+0 (secure 0 0 0) 0x7ff4900225e0 con 0x7ff4a0101000 2024-02-02T15:32:59.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.721+0000 7ff49d600700 1 --2- 172.21.15.1:0/770989056 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff4800a0690 0x7ff4800a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:59.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.721+0000 7ff49d600700 1 -- 172.21.15.1:0/770989056 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7ff4900cd510 con 0x7ff4a0101000 2024-02-02T15:32:59.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.721+0000 7ff4bb413b80 1 Processor -- start 2024-02-02T15:32:59.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.721+0000 7ff49fe00700 1 --2- 172.21.15.1:0/770989056 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff4800a0690 0x7ff4800a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:59.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.721+0000 7ff4bb413b80 1 -- start start 2024-02-02T15:32:59.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.721+0000 7ff4bb413b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ce1480a010 0x55ce1480a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:59.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.721+0000 7ff4bb413b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ce1480a930 0x55ce148152a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:59.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.721+0000 7ff4bb413b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce148157e0 0x55ce14817bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:59.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.721+0000 7ff4bb413b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce1477e4c0 con 0x55ce1480a930 2024-02-02T15:32:59.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.721+0000 7ff4bb413b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce14762aa0 con 0x55ce1480a010 2024-02-02T15:32:59.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.721+0000 7ff4bb413b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce14760460 con 0x55ce148157e0 2024-02-02T15:32:59.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.722+0000 7ff4a4e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ce1480a010 0x55ce1480a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:59.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.722+0000 7ff4a5800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce148157e0 0x55ce14817bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:59.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.722+0000 7ff49fe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ce1480a930 0x55ce148152a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:59.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.722+0000 7ff4a4e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ce1480a010 0x55ce1480a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:33672/0 (socket says 172.21.15.1:33672) 2024-02-02T15:32:59.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.722+0000 7ff4a4e00700 1 -- 172.21.15.1:0/2860617578 learned_addr learned my addr 172.21.15.1:0/2860617578 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:32:59.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.722+0000 7ff4a4e00700 1 -- 172.21.15.1:0/2860617578 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce148157e0 msgr2=0x55ce14817bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:32:59.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.722+0000 7ff4a4e00700 1 --2- 172.21.15.1:0/2860617578 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce148157e0 0x55ce14817bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:59.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.722+0000 7ff4a4e00700 1 -- 172.21.15.1:0/2860617578 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ce1480a930 msgr2=0x55ce148152a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:32:59.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.722+0000 7ff4a4e00700 1 --2- 172.21.15.1:0/2860617578 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ce1480a930 0x55ce148152a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:59.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.722+0000 7ff4a4e00700 1 -- 172.21.15.1:0/2860617578 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ce145dd3b0 con 0x55ce1480a010 2024-02-02T15:32:59.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.722+0000 7ff49fe00700 1 --2- 172.21.15.1:0/770989056 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff4800a0690 0x7ff4800a2b50 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff4a019a7e0 tx=0x7ff488011cd0 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:32:59.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.722+0000 7ff4a4e00700 1 --2- 172.21.15.1:0/2860617578 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ce1480a010 0x55ce1480a3f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff490006010 tx=0x7ff49009a2b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77c1d7c9cf76817a server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:32:59.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.723+0000 7ff496000700 1 -- 172.21.15.1:0/2860617578 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49009a770 con 0x55ce1480a010 2024-02-02T15:32:59.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.723+0000 7ff496000700 1 -- 172.21.15.1:0/2860617578 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff49009e5a0 con 0x55ce1480a010 2024-02-02T15:32:59.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.723+0000 7ff496000700 1 -- 172.21.15.1:0/2860617578 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49009e170 con 0x55ce1480a010 2024-02-02T15:32:59.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.723+0000 7ff4bb413b80 1 -- 172.21.15.1:0/2860617578 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ce1480a010 msgr2=0x55ce1480a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:59.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.723+0000 7ff4bb413b80 1 --2- 172.21.15.1:0/2860617578 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ce1480a010 0x55ce1480a3f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff490006010 tx=0x7ff49009a2b0 comp rx=0 tx=0).stop 2024-02-02T15:32:59.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.724+0000 7ff4bb413b80 1 -- 172.21.15.1:0/2860617578 shutdown_connections 2024-02-02T15:32:59.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.724+0000 7ff4bb413b80 1 --2- 172.21.15.1:0/2860617578 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce148157e0 0x55ce14817bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:59.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.724+0000 7ff4bb413b80 1 --2- 172.21.15.1:0/2860617578 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ce1480a010 0x55ce1480a3f0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:59.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.724+0000 7ff4bb413b80 1 --2- 172.21.15.1:0/2860617578 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ce1480a930 0x55ce148152a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:59.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.724+0000 7ff4bb413b80 1 -- 172.21.15.1:0/2860617578 >> 172.21.15.1:0/2860617578 conn(0x55ce14769db0 msgr2=0x55ce148063b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:32:59.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.724+0000 7ff4bb413b80 1 -- 172.21.15.1:0/2860617578 shutdown_connections 2024-02-02T15:32:59.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.724+0000 7ff4bb413b80 1 -- 172.21.15.1:0/2860617578 wait complete. 2024-02-02T15:32:59.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.724+0000 7ff4bb413b80 1 -- start start 2024-02-02T15:32:59.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.725+0000 7ff4bb413b80 1 Processor -- start 2024-02-02T15:32:59.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.725+0000 7ff4bb413b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ce1480a010 0x55ce14815bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:59.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.725+0000 7ff4bb413b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce1480a930 0x55ce14816100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:59.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.725+0000 7ff4bb413b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ce148157e0 0x55ce14816640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:59.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.725+0000 7ff4bb413b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce1475bd50 con 0x55ce148157e0 2024-02-02T15:32:59.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.725+0000 7ff4bb413b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce146f7d80 con 0x55ce1480a010 2024-02-02T15:32:59.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.725+0000 7ff4bb413b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce146f7a20 con 0x55ce1480a930 2024-02-02T15:32:59.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.725+0000 7ff4bb413b80 10 client.0 _ll_register_callbacks cb 0x55ce146f9290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:32:59.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.725+0000 7ff4a4e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ce1480a010 0x55ce14815bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:59.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.725+0000 7ff49fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce1480a930 0x55ce14816100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:59.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.725+0000 7ff4a4e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ce1480a010 0x55ce14815bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:33684/0 (socket says 172.21.15.1:33684) 2024-02-02T15:32:59.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.725+0000 7ff4a4e00700 1 -- 172.21.15.1:0/885253630 learned_addr learned my addr 172.21.15.1:0/885253630 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:32:59.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.726+0000 7ff4a5800700 1 --2- 172.21.15.1:0/885253630 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ce148157e0 0x55ce14816640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:59.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.726+0000 7ff4a4e00700 1 -- 172.21.15.1:0/885253630 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce1480a930 msgr2=0x55ce14816100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:32:59.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.726+0000 7ff4a4e00700 1 --2- 172.21.15.1:0/885253630 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce1480a930 0x55ce14816100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:59.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.726+0000 7ff4a4e00700 1 -- 172.21.15.1:0/885253630 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ce148157e0 msgr2=0x55ce14816640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:32:59.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.726+0000 7ff4a4e00700 1 --2- 172.21.15.1:0/885253630 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ce148157e0 0x55ce14816640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:59.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.726+0000 7ff4a4e00700 1 -- 172.21.15.1:0/885253630 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4a01a61f0 con 0x55ce1480a010 2024-02-02T15:32:59.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.726+0000 7ff49fe00700 1 --2- 172.21.15.1:0/885253630 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce1480a930 0x55ce14816100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:32:59.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.726+0000 7ff4a4e00700 1 --2- 172.21.15.1:0/885253630 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ce1480a010 0x55ce14815bc0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ff490005f50 tx=0x7ff490096b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28388dbf10e262c4 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:32:59.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.726+0000 7ff48ea00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:32:59.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.726+0000 7ff48ea00700 1 -- 172.21.15.1:0/885253630 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4900cd370 con 0x55ce1480a010 2024-02-02T15:32:59.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.726+0000 7ff48ea00700 1 -- 172.21.15.1:0/885253630 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7ff49009a770 con 0x55ce1480a010 2024-02-02T15:32:59.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.726+0000 7ff4bb413b80 1 -- 172.21.15.1:0/885253630 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ce147fe0f0 con 0x55ce1480a010 2024-02-02T15:32:59.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.726+0000 7ff48ea00700 1 -- 172.21.15.1:0/885253630 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff49009e170 con 0x55ce1480a010 2024-02-02T15:32:59.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.726+0000 7ff4bb413b80 10 client.27188 fetch_fsmap learned FSMap version 38 2024-02-02T15:32:59.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.726+0000 7ff4bb413b80 1 -- 172.21.15.1:0/885253630 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55ce14589b60 con 0x55ce1480a010 2024-02-02T15:32:59.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.727+0000 7ff48ea00700 1 -- 172.21.15.1:0/885253630 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ce147fe0f0 con 0x55ce1480a010 2024-02-02T15:32:59.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.727+0000 7ff48ea00700 1 -- 172.21.15.1:0/885253630 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 38) v1 ==== 7486+0+0 (secure 0 0 0) 0x7ff4900b1030 con 0x55ce1480a010 2024-02-02T15:32:59.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.727+0000 7ff4bb413b80 10 client.27188 fetch_fsmap finished waiting for FSMap version 38 2024-02-02T15:32:59.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.727+0000 7ff4bb413b80 10 client.27188 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-02-02T15:32:59.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.727+0000 7ff4bb413b80 20 client.27188 populate_metadata read hostname 'smithi001' 2024-02-02T15:32:59.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.727+0000 7ff4bb413b80 1 --2- 172.21.15.1:0/885253630 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55ce14824a10 0x55ce14826e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:59.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.727+0000 7ff4bb413b80 4 client.27188 mds_command: new command op to 24502 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:32:59.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.727+0000 7ff4bb413b80 1 -- 172.21.15.1:0/885253630 --> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ce14754dd0 con 0x55ce14824a10 2024-02-02T15:32:59.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.727+0000 7ff49fe00700 1 --2- 172.21.15.1:0/885253630 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55ce14824a10 0x55ce14826e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:59.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.728+0000 7ff49fe00700 1 --2- 172.21.15.1:0/885253630 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55ce14824a10 0x55ce14826e00 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5aa63365c0597111 server_cookie=941cdd0a9576a801 in_seq=0 out_seq=0 2024-02-02T15:32:59.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.728+0000 7ff48ea00700 10 client.27188 ms_handle_connect on v2:172.21.15.142:6834/1052606547 2024-02-02T15:32:59.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.728+0000 7ff48ea00700 1 -- 172.21.15.1:0/885253630 <== mds.4 v2:172.21.15.142:6834/1052606547 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55ce14754dd0 con 0x55ce14824a10 2024-02-02T15:32:59.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.728+0000 7ff48ea00700 10 client.27188 handle_command_reply: tid=0 2024-02-02T15:32:59.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.728+0000 7ff4bb413b80 1 client.27188 shutdown 2024-02-02T15:32:59.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.729+0000 7ff4bb413b80 1 -- 172.21.15.1:0/885253630 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ce1480a010 msgr2=0x55ce14815bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:59.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.729+0000 7ff4bb413b80 1 --2- 172.21.15.1:0/885253630 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ce1480a010 0x55ce14815bc0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ff490005f50 tx=0x7ff490096b40 comp rx=0 tx=0).stop 2024-02-02T15:32:59.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.729+0000 7ff4bb413b80 1 -- 172.21.15.1:0/885253630 shutdown_connections 2024-02-02T15:32:59.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.729+0000 7ff4bb413b80 1 --2- 172.21.15.1:0/885253630 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55ce14824a10 0x55ce14826e00 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:59.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.729+0000 7ff4bb413b80 1 --2- 172.21.15.1:0/885253630 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ce148157e0 0x55ce14816640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:59.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.729+0000 7ff4bb413b80 1 --2- 172.21.15.1:0/885253630 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ce1480a010 0x55ce14815bc0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:59.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.729+0000 7ff48ea00700 0 client.27188 ms_handle_reset on v2:172.21.15.142:6834/1052606547 2024-02-02T15:32:59.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.729+0000 7ff4bb413b80 1 --2- 172.21.15.1:0/885253630 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce1480a930 0x55ce14816100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:59.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.729+0000 7ff4bb413b80 1 -- 172.21.15.1:0/885253630 >> 172.21.15.1:0/885253630 conn(0x55ce14769db0 msgr2=0x55ce14805fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:32:59.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.729+0000 7ff4bb413b80 1 -- 172.21.15.1:0/885253630 shutdown_connections 2024-02-02T15:32:59.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.729+0000 7ff4bb413b80 1 -- 172.21.15.1:0/885253630 wait complete. 2024-02-02T15:32:59.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.729+0000 7ff4bb413b80 20 client.27188 trim_cache size 0 max 16384 2024-02-02T15:32:59.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.744+0000 7ff4bb413b80 1 Processor -- start 2024-02-02T15:32:59.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.744+0000 7ff4bb413b80 1 -- start start 2024-02-02T15:32:59.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.745+0000 7ff4bb413b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ce1480a010 0x55ce14819bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:59.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.745+0000 7ff4bb413b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce1480a930 0x55ce1480fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:59.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.745+0000 7ff4bb413b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ce148157e0 0x55ce148100f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:59.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.745+0000 7ff4bb413b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce145df6a0 con 0x55ce148157e0 2024-02-02T15:32:59.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.745+0000 7ff4bb413b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce14814e70 con 0x55ce1480a010 2024-02-02T15:32:59.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.745+0000 7ff4bb413b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce14820a90 con 0x55ce1480a930 2024-02-02T15:32:59.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.745+0000 7ff4a4e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ce1480a010 0x55ce14819bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:59.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.745+0000 7ff49fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce1480a930 0x55ce1480fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:59.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.745+0000 7ff4a4e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ce1480a010 0x55ce14819bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:33696/0 (socket says 172.21.15.1:33696) 2024-02-02T15:32:59.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.745+0000 7ff4a4e00700 1 -- 172.21.15.1:0/3740912708 learned_addr learned my addr 172.21.15.1:0/3740912708 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:32:59.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.745+0000 7ff4a5800700 1 --2- 172.21.15.1:0/3740912708 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ce148157e0 0x55ce148100f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:59.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.745+0000 7ff4a4e00700 1 -- 172.21.15.1:0/3740912708 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce1480a930 msgr2=0x55ce1480fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:32:59.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.745+0000 7ff4a4e00700 1 --2- 172.21.15.1:0/3740912708 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce1480a930 0x55ce1480fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:59.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.745+0000 7ff4a4e00700 1 -- 172.21.15.1:0/3740912708 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ce148157e0 msgr2=0x55ce148100f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:32:59.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.745+0000 7ff4a4e00700 1 --2- 172.21.15.1:0/3740912708 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ce148157e0 0x55ce148100f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:59.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.745+0000 7ff4a4e00700 1 -- 172.21.15.1:0/3740912708 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ce14721ea0 con 0x55ce1480a010 2024-02-02T15:32:59.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.746+0000 7ff4a4e00700 1 --2- 172.21.15.1:0/3740912708 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ce1480a010 0x55ce14819bf0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7ff4900a2910 tx=0x7ff490009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7032c464692dede2 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:32:59.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.746+0000 7ff496a00700 1 -- 172.21.15.1:0/3740912708 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4900cd370 con 0x55ce1480a010 2024-02-02T15:32:59.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.746+0000 7ff496a00700 1 -- 172.21.15.1:0/3740912708 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff49009a2b0 con 0x55ce1480a010 2024-02-02T15:32:59.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.746+0000 7ff496a00700 1 -- 172.21.15.1:0/3740912708 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49009e170 con 0x55ce1480a010 2024-02-02T15:32:59.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.746+0000 7ff4bb413b80 1 -- 172.21.15.1:0/3740912708 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ce1480a010 msgr2=0x55ce14819bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:59.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.746+0000 7ff4bb413b80 1 --2- 172.21.15.1:0/3740912708 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ce1480a010 0x55ce14819bf0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7ff4900a2910 tx=0x7ff490009310 comp rx=0 tx=0).stop 2024-02-02T15:32:59.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.747+0000 7ff4bb413b80 1 -- 172.21.15.1:0/3740912708 shutdown_connections 2024-02-02T15:32:59.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.747+0000 7ff4bb413b80 1 --2- 172.21.15.1:0/3740912708 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ce148157e0 0x55ce148100f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:59.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.747+0000 7ff4bb413b80 1 --2- 172.21.15.1:0/3740912708 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ce1480a010 0x55ce14819bf0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:59.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.747+0000 7ff4bb413b80 1 --2- 172.21.15.1:0/3740912708 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce1480a930 0x55ce1480fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:59.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.747+0000 7ff4bb413b80 1 -- 172.21.15.1:0/3740912708 >> 172.21.15.1:0/3740912708 conn(0x55ce14769db0 msgr2=0x55ce14804b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:32:59.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.747+0000 7ff4bb413b80 1 -- 172.21.15.1:0/3740912708 shutdown_connections 2024-02-02T15:32:59.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.747+0000 7ff4bb413b80 1 -- 172.21.15.1:0/3740912708 wait complete. 2024-02-02T15:32:59.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.747+0000 7ff4bb413b80 1 -- start start 2024-02-02T15:32:59.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.747+0000 7ff4bb413b80 1 Processor -- start 2024-02-02T15:32:59.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.748+0000 7ff4bb413b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce1480a010 0x55ce1486dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:59.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.748+0000 7ff4bb413b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ce1480a930 0x55ce1486e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:59.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.748+0000 7ff4bb413b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ce148157e0 0x55ce14818e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:59.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.748+0000 7ff4bb413b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce14819590 con 0x55ce148157e0 2024-02-02T15:32:59.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.748+0000 7ff4bb413b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce14819700 con 0x55ce1480a930 2024-02-02T15:32:59.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.748+0000 7ff4bb413b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce148198c0 con 0x55ce1480a010 2024-02-02T15:32:59.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.748+0000 7ff4bb413b80 10 client.0 _ll_register_callbacks cb 0x55ce146f9290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:32:59.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.748+0000 7ff49fe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ce1480a930 0x55ce1486e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:59.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.748+0000 7ff49fe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ce1480a930 0x55ce1486e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:33702/0 (socket says 172.21.15.1:33702) 2024-02-02T15:32:59.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.748+0000 7ff49fe00700 1 -- 172.21.15.1:0/3121066288 learned_addr learned my addr 172.21.15.1:0/3121066288 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:32:59.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.748+0000 7ff4a5800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ce148157e0 0x55ce14818e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:59.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.748+0000 7ff4a4e00700 1 --2- 172.21.15.1:0/3121066288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce1480a010 0x55ce1486dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:59.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.748+0000 7ff49fe00700 1 -- 172.21.15.1:0/3121066288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce1480a010 msgr2=0x55ce1486dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:32:59.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.748+0000 7ff49fe00700 1 --2- 172.21.15.1:0/3121066288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce1480a010 0x55ce1486dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:59.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.748+0000 7ff49fe00700 1 -- 172.21.15.1:0/3121066288 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ce148157e0 msgr2=0x55ce14818e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:32:59.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.748+0000 7ff49fe00700 1 --2- 172.21.15.1:0/3121066288 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ce148157e0 0x55ce14818e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:59.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.748+0000 7ff49fe00700 1 -- 172.21.15.1:0/3121066288 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ce14589b60 con 0x55ce1480a930 2024-02-02T15:32:59.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.749+0000 7ff49fe00700 1 --2- 172.21.15.1:0/3121066288 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ce1480a930 0x55ce1486e260 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ff488011720 tx=0x7ff48801b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca9cf94fe4f76172 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:32:59.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.749+0000 7ff48ea00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:32:59.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.749+0000 7ff48ea00700 1 -- 172.21.15.1:0/3121066288 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff488028040 con 0x55ce1480a930 2024-02-02T15:32:59.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.749+0000 7ff4bb413b80 1 -- 172.21.15.1:0/3121066288 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff49009e5a0 con 0x55ce1480a930 2024-02-02T15:32:59.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.749+0000 7ff48ea00700 1 -- 172.21.15.1:0/3121066288 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7ff488028260 con 0x55ce1480a930 2024-02-02T15:32:59.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.749+0000 7ff48ea00700 1 -- 172.21.15.1:0/3121066288 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff488028420 con 0x55ce1480a930 2024-02-02T15:32:59.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.749+0000 7ff4bb413b80 10 client.27194 fetch_fsmap learned FSMap version 38 2024-02-02T15:32:59.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.749+0000 7ff4bb413b80 1 -- 172.21.15.1:0/3121066288 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55ce145e1cd0 con 0x55ce1480a930 2024-02-02T15:32:59.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.749+0000 7ff48ea00700 1 -- 172.21.15.1:0/3121066288 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff49009e5a0 con 0x55ce1480a930 2024-02-02T15:32:59.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.750+0000 7ff48ea00700 1 -- 172.21.15.1:0/3121066288 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 38) v1 ==== 7486+0+0 (secure 0 0 0) 0x7ff488003810 con 0x55ce1480a930 2024-02-02T15:32:59.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.750+0000 7ff4bb413b80 10 client.27194 fetch_fsmap finished waiting for FSMap version 38 2024-02-02T15:32:59.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.750+0000 7ff4bb413b80 10 client.27194 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-02-02T15:32:59.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.750+0000 7ff4bb413b80 20 client.27194 populate_metadata read hostname 'smithi001' 2024-02-02T15:32:59.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.750+0000 7ff4bb413b80 1 --2- 172.21.15.1:0/3121066288 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55ce14876cd0 0x55ce148790f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:59.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.750+0000 7ff4bb413b80 4 client.27194 mds_command: new command op to 24502 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:32:59.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.750+0000 7ff4bb413b80 1 -- 172.21.15.1:0/3121066288 --> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ce1471af50 con 0x55ce14876cd0 2024-02-02T15:32:59.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.750+0000 7ff4a4e00700 1 --2- 172.21.15.1:0/3121066288 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55ce14876cd0 0x55ce148790f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:59.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.750+0000 7ff4a4e00700 1 --2- 172.21.15.1:0/3121066288 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55ce14876cd0 0x55ce148790f0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=71945cb9c057b884 server_cookie=af441feb06093be6 in_seq=0 out_seq=0 2024-02-02T15:32:59.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.750+0000 7ff48ea00700 10 client.27194 ms_handle_connect on v2:172.21.15.142:6834/1052606547 2024-02-02T15:32:59.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.751+0000 7ff48ea00700 1 -- 172.21.15.1:0/3121066288 <== mds.4 v2:172.21.15.142:6834/1052606547 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ce1471af50 con 0x55ce14876cd0 2024-02-02T15:32:59.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.751+0000 7ff48ea00700 10 client.27194 handle_command_reply: tid=0 2024-02-02T15:32:59.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.751+0000 7ff4bb413b80 1 client.27194 shutdown 2024-02-02T15:32:59.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.751+0000 7ff4bb413b80 1 -- 172.21.15.1:0/3121066288 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ce1480a930 msgr2=0x55ce1486e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:59.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.751+0000 7ff4bb413b80 1 --2- 172.21.15.1:0/3121066288 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ce1480a930 0x55ce1486e260 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ff488011720 tx=0x7ff48801b000 comp rx=0 tx=0).stop 2024-02-02T15:32:59.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.751+0000 7ff4bb413b80 1 -- 172.21.15.1:0/3121066288 shutdown_connections 2024-02-02T15:32:59.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.751+0000 7ff4bb413b80 1 --2- 172.21.15.1:0/3121066288 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55ce14876cd0 0x55ce148790f0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:59.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.751+0000 7ff4bb413b80 1 --2- 172.21.15.1:0/3121066288 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ce148157e0 0x55ce14818e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:59.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.751+0000 7ff4bb413b80 1 --2- 172.21.15.1:0/3121066288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ce1480a010 0x55ce1486dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:59.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.751+0000 7ff4bb413b80 1 --2- 172.21.15.1:0/3121066288 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ce1480a930 0x55ce1486e260 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:59.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.751+0000 7ff48ea00700 0 client.27194 ms_handle_reset on v2:172.21.15.142:6834/1052606547 2024-02-02T15:32:59.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.751+0000 7ff4bb413b80 1 -- 172.21.15.1:0/3121066288 >> 172.21.15.1:0/3121066288 conn(0x55ce14769db0 msgr2=0x55ce1480fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:32:59.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.751+0000 7ff4bb413b80 1 -- 172.21.15.1:0/3121066288 shutdown_connections 2024-02-02T15:32:59.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.751+0000 7ff4bb413b80 1 -- 172.21.15.1:0/3121066288 wait complete. 2024-02-02T15:32:59.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.751+0000 7ff4bb413b80 20 client.27194 trim_cache size 0 max 16384 2024-02-02T15:32:59.756 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:32:59.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.752+0000 7ff4a7800700 1 -- 172.21.15.1:0/770989056 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff4800a0690 msgr2=0x7ff4800a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:59.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.752+0000 7ff4a7800700 1 --2- 172.21.15.1:0/770989056 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff4800a0690 0x7ff4800a2b50 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ff4a019a7e0 tx=0x7ff488011cd0 comp rx=0 tx=0).stop 2024-02-02T15:32:59.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.752+0000 7ff4a7800700 1 -- 172.21.15.1:0/770989056 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4a0101000 msgr2=0x7ff4a01991d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:59.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.752+0000 7ff4a7800700 1 --2- 172.21.15.1:0/770989056 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4a0101000 0x7ff4a01991d0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff490005910 tx=0x7ff490005740 comp rx=0 tx=0).stop 2024-02-02T15:32:59.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.752+0000 7ff4a7800700 1 -- 172.21.15.1:0/770989056 shutdown_connections 2024-02-02T15:32:59.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.752+0000 7ff4a7800700 1 --2- 172.21.15.1:0/770989056 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff4800a0690 0x7ff4800a2b50 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:59.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.752+0000 7ff4a7800700 1 --2- 172.21.15.1:0/770989056 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff4a019db70 0x7ff4a019dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:59.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.752+0000 7ff4a7800700 1 --2- 172.21.15.1:0/770989056 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4a0101000 0x7ff4a01991d0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:59.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.752+0000 7ff4a7800700 1 --2- 172.21.15.1:0/770989056 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4a0106120 0x7ff4a0199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:59.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.752+0000 7ff4a7800700 1 -- 172.21.15.1:0/770989056 >> 172.21.15.1:0/770989056 conn(0x7ff4a00fc860 msgr2=0x7ff4a01029c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:32:59.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.752+0000 7ff4a7800700 1 -- 172.21.15.1:0/770989056 shutdown_connections 2024-02-02T15:32:59.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.752+0000 7ff4a7800700 1 -- 172.21.15.1:0/770989056 wait complete. 2024-02-02T15:32:59.771 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-02-02T15:32:59.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:32:59 smithi071 ceph-mon[44707]: pgmap v182: 129 pgs: 129 active+clean; 241 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 207 op/s 2024-02-02T15:32:59.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.985+0000 7f0499000700 1 -- 172.21.15.1:0/4018767364 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0494068010 msgr2=0x7f0494068490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:59.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.985+0000 7f0499000700 1 --2- 172.21.15.1:0/4018767364 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0494068010 0x7f0494068490 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f047c009990 tx=0x7f047c009ca0 comp rx=0 tx=0).stop 2024-02-02T15:32:59.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.985+0000 7f0499000700 1 -- 172.21.15.1:0/4018767364 shutdown_connections 2024-02-02T15:32:59.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.985+0000 7f0499000700 1 --2- 172.21.15.1:0/4018767364 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f04940689d0 0x7f049410d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:59.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.985+0000 7f0499000700 1 --2- 172.21.15.1:0/4018767364 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0494106ab0 0x7f0494106e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:59.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.985+0000 7f0499000700 1 --2- 172.21.15.1:0/4018767364 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0494068010 0x7f0494068490 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:59.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.985+0000 7f0499000700 1 -- 172.21.15.1:0/4018767364 >> 172.21.15.1:0/4018767364 conn(0x7f0494076770 msgr2=0x7f0494076b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:32:59.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.985+0000 7f0499000700 1 -- 172.21.15.1:0/4018767364 shutdown_connections 2024-02-02T15:32:59.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.985+0000 7f0499000700 1 -- 172.21.15.1:0/4018767364 wait complete. 2024-02-02T15:32:59.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.986+0000 7f0499000700 1 Processor -- start 2024-02-02T15:32:59.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.986+0000 7f0499000700 1 -- start start 2024-02-02T15:32:59.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.987+0000 7f0499000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0494068010 0x7f04940ffc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:59.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.987+0000 7f0499000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f04940689d0 0x7f04941001a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:59.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.987+0000 7f0499000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0494106ab0 0x7f04941006e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:59.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.987+0000 7f0499000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04941a4f90 con 0x7f0494106ab0 2024-02-02T15:32:59.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.987+0000 7f0499000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04941a5100 con 0x7f04940689d0 2024-02-02T15:32:59.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.987+0000 7f0499000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04941a52c0 con 0x7f0494068010 2024-02-02T15:32:59.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.987+0000 7f0492a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f04940689d0 0x7f04941001a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:59.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.987+0000 7f0492a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f04940689d0 0x7f04941001a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:33724/0 (socket says 172.21.15.1:33724) 2024-02-02T15:32:59.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.987+0000 7f0492a00700 1 -- 172.21.15.1:0/1364271482 learned_addr learned my addr 172.21.15.1:0/1364271482 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:32:59.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.987+0000 7f0493400700 1 --2- 172.21.15.1:0/1364271482 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0494068010 0x7f04940ffc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:59.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.987+0000 7f0493e00700 1 --2- 172.21.15.1:0/1364271482 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0494106ab0 0x7f04941006e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:59.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.987+0000 7f0492a00700 1 -- 172.21.15.1:0/1364271482 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0494068010 msgr2=0x7f04940ffc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:59.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.987+0000 7f0492a00700 1 --2- 172.21.15.1:0/1364271482 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0494068010 0x7f04940ffc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:59.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.987+0000 7f0492a00700 1 -- 172.21.15.1:0/1364271482 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0494106ab0 msgr2=0x7f04941006e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:32:59.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.987+0000 7f0492a00700 1 --2- 172.21.15.1:0/1364271482 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0494106ab0 0x7f04941006e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:59.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.987+0000 7f0492a00700 1 -- 172.21.15.1:0/1364271482 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f048c009c40 con 0x7f04940689d0 2024-02-02T15:32:59.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.987+0000 7f0493400700 1 --2- 172.21.15.1:0/1364271482 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0494068010 0x7f04940ffc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:32:59.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.988+0000 7f0492a00700 1 --2- 172.21.15.1:0/1364271482 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f04940689d0 0x7f04941001a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f047c000c00 tx=0x7f047c005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:32:59.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.988+0000 7f048be00700 1 -- 172.21.15.1:0/1364271482 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f047c01d070 con 0x7f04940689d0 2024-02-02T15:32:59.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.988+0000 7f048be00700 1 -- 172.21.15.1:0/1364271482 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f047c022470 con 0x7f04940689d0 2024-02-02T15:32:59.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.988+0000 7f0499000700 1 -- 172.21.15.1:0/1364271482 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f047c0095f0 con 0x7f04940689d0 2024-02-02T15:32:59.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.988+0000 7f048be00700 1 -- 172.21.15.1:0/1364271482 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f047c017650 con 0x7f04940689d0 2024-02-02T15:32:59.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.988+0000 7f0499000700 1 -- 172.21.15.1:0/1364271482 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04941a58e0 con 0x7f04940689d0 2024-02-02T15:32:59.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.992+0000 7f048be00700 1 -- 172.21.15.1:0/1364271482 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 27) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f047c0225e0 con 0x7f04940689d0 2024-02-02T15:32:59.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.993+0000 7f04ae165b80 1 Processor -- start 2024-02-02T15:32:59.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.993+0000 7f04ae165b80 1 -- start start 2024-02-02T15:32:59.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.993+0000 7f048be00700 1 --2- 172.21.15.1:0/1364271482 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f04740a0710 0x7f04740a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:59.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.993+0000 7f04ae165b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f30ef071f0 0x55f30ef075d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:59.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.994+0000 7f04ae165b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f30ef07b10 0x55f30ef124b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:59.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.994+0000 7f048be00700 1 -- 172.21.15.1:0/1364271482 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f047c0cd410 con 0x7f04940689d0 2024-02-02T15:32:59.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.994+0000 7f04ae165b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f30ef129f0 0x55f30ef14de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:32:59.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.994+0000 7f04ae165b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f30ee7b4c0 con 0x55f30ef129f0 2024-02-02T15:32:59.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.994+0000 7f04ae165b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f30ee5faa0 con 0x55f30ef07b10 2024-02-02T15:32:59.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.994+0000 7f04ae165b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f30ee5d460 con 0x55f30ef071f0 2024-02-02T15:32:59.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.994+0000 7f0493400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f30ef071f0 0x55f30ef075d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:59.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.994+0000 7f0493400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f30ef071f0 0x55f30ef075d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:39598/0 (socket says 172.21.15.1:39598) 2024-02-02T15:32:59.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.994+0000 7f0493400700 1 -- 172.21.15.1:0/2140541051 learned_addr learned my addr 172.21.15.1:0/2140541051 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:32:59.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.994+0000 7f0492a00700 1 --2- 172.21.15.1:0/2140541051 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f30ef07b10 0x55f30ef124b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:59.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.994+0000 7f0493e00700 1 --2- 172.21.15.1:0/1364271482 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f04740a0710 0x7f04740a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:59.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.994+0000 7f0493e00700 1 --2- 172.21.15.1:0/2140541051 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f30ef129f0 0x55f30ef14de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:32:59.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.994+0000 7f0492a00700 1 -- 172.21.15.1:0/2140541051 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f30ef071f0 msgr2=0x55f30ef075d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:32:59.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.994+0000 7f0492a00700 1 --2- 172.21.15.1:0/2140541051 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f30ef071f0 0x55f30ef075d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:59.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.994+0000 7f0492a00700 1 -- 172.21.15.1:0/2140541051 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f30ef129f0 msgr2=0x55f30ef14de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:32:59.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.994+0000 7f0492a00700 1 --2- 172.21.15.1:0/2140541051 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f30ef129f0 0x55f30ef14de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:32:59.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.994+0000 7f0492a00700 1 -- 172.21.15.1:0/2140541051 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f30ecda3b0 con 0x55f30ef07b10 2024-02-02T15:32:59.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.994+0000 7f0493400700 1 --2- 172.21.15.1:0/2140541051 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f30ef071f0 0x55f30ef075d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:32:59.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.995+0000 7f0492a00700 1 --2- 172.21.15.1:0/2140541051 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f30ef07b10 0x55f30ef124b0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f047c0ca810 tx=0x7f047c09a510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f298088eec577150 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:32:59.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.995+0000 7f0493e00700 1 --2- 172.21.15.1:0/1364271482 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f04740a0710 0x7f04740a2bd0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f048c009fd0 tx=0x7f048c009920 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:32:59.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.995+0000 7f0488c00700 1 -- 172.21.15.1:0/2140541051 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f047c0228e0 con 0x55f30ef07b10 2024-02-02T15:32:59.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.995+0000 7f0488c00700 1 -- 172.21.15.1:0/2140541051 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f047c09ad70 con 0x55f30ef07b10 2024-02-02T15:32:59.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.995+0000 7f0488c00700 1 -- 172.21.15.1:0/2140541051 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f047c0cd020 con 0x55f30ef07b10 2024-02-02T15:33:00.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.995+0000 7f04ae165b80 1 -- 172.21.15.1:0/2140541051 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f30ef07b10 msgr2=0x55f30ef124b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:00.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.995+0000 7f04ae165b80 1 --2- 172.21.15.1:0/2140541051 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f30ef07b10 0x55f30ef124b0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f047c0ca810 tx=0x7f047c09a510 comp rx=0 tx=0).stop 2024-02-02T15:33:00.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.996+0000 7f04ae165b80 1 -- 172.21.15.1:0/2140541051 shutdown_connections 2024-02-02T15:33:00.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.996+0000 7f04ae165b80 1 --2- 172.21.15.1:0/2140541051 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f30ef129f0 0x55f30ef14de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.996+0000 7f04ae165b80 1 --2- 172.21.15.1:0/2140541051 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f30ef071f0 0x55f30ef075d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.996+0000 7f04ae165b80 1 --2- 172.21.15.1:0/2140541051 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f30ef07b10 0x55f30ef124b0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.996+0000 7f04ae165b80 1 -- 172.21.15.1:0/2140541051 >> 172.21.15.1:0/2140541051 conn(0x55f30ee66db0 msgr2=0x55f30ef035c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:00.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.996+0000 7f04ae165b80 1 -- 172.21.15.1:0/2140541051 shutdown_connections 2024-02-02T15:33:00.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.996+0000 7f04ae165b80 1 -- 172.21.15.1:0/2140541051 wait complete. 2024-02-02T15:33:00.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.996+0000 7f04ae165b80 1 -- start start 2024-02-02T15:33:00.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.997+0000 7f04ae165b80 1 Processor -- start 2024-02-02T15:33:00.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.997+0000 7f04ae165b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f30ef071f0 0x55f30ef12dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:00.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.997+0000 7f04ae165b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f30ef07b10 0x55f30ef13310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:00.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.997+0000 7f04ae165b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f30ef129f0 0x55f30ef13850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:00.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.997+0000 7f04ae165b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f30ee58d50 con 0x55f30ef07b10 2024-02-02T15:33:00.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.997+0000 7f04ae165b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f30edf4d80 con 0x55f30ef071f0 2024-02-02T15:33:00.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.997+0000 7f04ae165b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f30edf4a20 con 0x55f30ef129f0 2024-02-02T15:33:00.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.998+0000 7f04ae165b80 10 client.0 _ll_register_callbacks cb 0x55f30edf6290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:33:00.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.998+0000 7f0492a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f30ef07b10 0x55f30ef13310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:00.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.998+0000 7f0493400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f30ef071f0 0x55f30ef12dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:00.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.998+0000 7f0493e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f30ef129f0 0x55f30ef13850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:00.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.998+0000 7f0493400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f30ef071f0 0x55f30ef12dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:33744/0 (socket says 172.21.15.1:33744) 2024-02-02T15:33:00.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.998+0000 7f0493400700 1 -- 172.21.15.1:0/2675193582 learned_addr learned my addr 172.21.15.1:0/2675193582 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:33:00.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.998+0000 7f0493400700 1 -- 172.21.15.1:0/2675193582 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f30ef129f0 msgr2=0x55f30ef13850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:00.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.998+0000 7f0493400700 1 --2- 172.21.15.1:0/2675193582 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f30ef129f0 0x55f30ef13850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.998+0000 7f0493400700 1 -- 172.21.15.1:0/2675193582 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f30ef07b10 msgr2=0x55f30ef13310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:00.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.998+0000 7f0493400700 1 --2- 172.21.15.1:0/2675193582 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f30ef07b10 0x55f30ef13310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.998+0000 7f0493400700 1 -- 172.21.15.1:0/2675193582 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0484009d50 con 0x55f30ef071f0 2024-02-02T15:33:00.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.998+0000 7f0493e00700 1 --2- 172.21.15.1:0/2675193582 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f30ef129f0 0x55f30ef13850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:33:00.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.998+0000 7f0493400700 1 --2- 172.21.15.1:0/2675193582 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f30ef071f0 0x55f30ef12dd0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f0484011e10 tx=0x7f048400f6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2316350a81da79 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:33:00.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.998+0000 7f0481600700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:33:00.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.999+0000 7f0481600700 1 -- 172.21.15.1:0/2675193582 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0484016070 con 0x55f30ef071f0 2024-02-02T15:33:00.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.999+0000 7f0481600700 1 -- 172.21.15.1:0/2675193582 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f048400fe50 con 0x55f30ef071f0 2024-02-02T15:33:00.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.999+0000 7f04ae165b80 1 -- 172.21.15.1:0/2675193582 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f30eefb340 con 0x55f30ef071f0 2024-02-02T15:33:00.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.999+0000 7f0481600700 1 -- 172.21.15.1:0/2675193582 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0484011050 con 0x55f30ef071f0 2024-02-02T15:33:00.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.999+0000 7f04ae165b80 10 client.27206 fetch_fsmap learned FSMap version 38 2024-02-02T15:33:00.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.999+0000 7f04ae165b80 1 -- 172.21.15.1:0/2675193582 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x7f04941a58e0 con 0x55f30ef071f0 2024-02-02T15:33:00.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.999+0000 7f0481600700 1 -- 172.21.15.1:0/2675193582 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f30eefb340 con 0x55f30ef071f0 2024-02-02T15:33:00.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.999+0000 7f0481600700 1 -- 172.21.15.1:0/2675193582 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 38) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f04840039f0 con 0x55f30ef071f0 2024-02-02T15:33:00.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.999+0000 7f04ae165b80 10 client.27206 fetch_fsmap finished waiting for FSMap version 38 2024-02-02T15:33:00.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.999+0000 7f04ae165b80 10 client.27206 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-02-02T15:33:00.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.999+0000 7f04ae165b80 20 client.27206 populate_metadata read hostname 'smithi001' 2024-02-02T15:33:00.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.999+0000 7f04ae165b80 1 --2- 172.21.15.1:0/2675193582 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55f30ef22080 0x55f30ef244a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:00.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.999+0000 7f04ae165b80 4 client.27206 mds_command: new command op to 24514 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:33:00.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:32:59.999+0000 7f04ae165b80 1 -- 172.21.15.1:0/2675193582 --> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f30ee51dd0 con 0x55f30ef22080 2024-02-02T15:33:00.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.000+0000 7f0492a00700 1 --2- 172.21.15.1:0/2675193582 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55f30ef22080 0x55f30ef244a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:00.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.000+0000 7f0492a00700 1 --2- 172.21.15.1:0/2675193582 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55f30ef22080 0x55f30ef244a0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=413f1995a6562f3d server_cookie=3c44cb98a015fa3 in_seq=0 out_seq=0 2024-02-02T15:33:00.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.000+0000 7f0481600700 10 client.27206 ms_handle_connect on v2:172.21.15.142:6836/2870289019 2024-02-02T15:33:00.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.001+0000 7f0481600700 1 -- 172.21.15.1:0/2675193582 <== mds.3 v2:172.21.15.142:6836/2870289019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55f30ee51dd0 con 0x55f30ef22080 2024-02-02T15:33:00.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.001+0000 7f0481600700 10 client.27206 handle_command_reply: tid=0 2024-02-02T15:33:00.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.001+0000 7f04ae165b80 1 client.27206 shutdown 2024-02-02T15:33:00.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.001+0000 7f04ae165b80 1 -- 172.21.15.1:0/2675193582 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f30ef071f0 msgr2=0x55f30ef12dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:00.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.001+0000 7f04ae165b80 1 --2- 172.21.15.1:0/2675193582 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f30ef071f0 0x55f30ef12dd0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f0484011e10 tx=0x7f048400f6a0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.001+0000 7f04ae165b80 1 -- 172.21.15.1:0/2675193582 shutdown_connections 2024-02-02T15:33:00.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.001+0000 7f04ae165b80 1 --2- 172.21.15.1:0/2675193582 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f30ef129f0 0x55f30ef13850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.001+0000 7f04ae165b80 1 --2- 172.21.15.1:0/2675193582 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f30ef071f0 0x55f30ef12dd0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.001+0000 7f04ae165b80 1 --2- 172.21.15.1:0/2675193582 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55f30ef22080 0x55f30ef244a0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.001+0000 7f04ae165b80 1 --2- 172.21.15.1:0/2675193582 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f30ef07b10 0x55f30ef13310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.001+0000 7f0481600700 0 client.27206 ms_handle_reset on v2:172.21.15.142:6836/2870289019 2024-02-02T15:33:00.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.001+0000 7f04ae165b80 1 -- 172.21.15.1:0/2675193582 >> 172.21.15.1:0/2675193582 conn(0x55f30ee66db0 msgr2=0x55f30ef03150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:00.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.001+0000 7f04ae165b80 1 -- 172.21.15.1:0/2675193582 shutdown_connections 2024-02-02T15:33:00.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.001+0000 7f04ae165b80 1 -- 172.21.15.1:0/2675193582 wait complete. 2024-02-02T15:33:00.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.001+0000 7f04ae165b80 20 client.27206 trim_cache size 0 max 16384 2024-02-02T15:33:00.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.015+0000 7f04ae165b80 1 Processor -- start 2024-02-02T15:33:00.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.015+0000 7f04ae165b80 1 -- start start 2024-02-02T15:33:00.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.015+0000 7f04ae165b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f30ef071f0 0x55f30ef16e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:00.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.015+0000 7f04ae165b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f30ef07b10 0x55f30ef0c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:00.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.016+0000 7f04ae165b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f30ef129f0 0x55f30ef0cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:00.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.016+0000 7f04ae165b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f30ecdc6a0 con 0x55f30ef129f0 2024-02-02T15:33:00.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.016+0000 7f04ae165b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f30ef1ea90 con 0x55f30ef071f0 2024-02-02T15:33:00.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.016+0000 7f04ae165b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f30eefd750 con 0x55f30ef07b10 2024-02-02T15:33:00.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.016+0000 7f0493400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f30ef071f0 0x55f30ef16e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:00.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.016+0000 7f0492a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f30ef07b10 0x55f30ef0c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:00.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.016+0000 7f0493400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f30ef071f0 0x55f30ef16e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:33746/0 (socket says 172.21.15.1:33746) 2024-02-02T15:33:00.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.016+0000 7f0493400700 1 -- 172.21.15.1:0/369276539 learned_addr learned my addr 172.21.15.1:0/369276539 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:33:00.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.016+0000 7f0493e00700 1 --2- 172.21.15.1:0/369276539 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f30ef129f0 0x55f30ef0cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:00.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.016+0000 7f0493400700 1 -- 172.21.15.1:0/369276539 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f30ef07b10 msgr2=0x55f30ef0c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:00.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.016+0000 7f0493400700 1 --2- 172.21.15.1:0/369276539 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f30ef07b10 0x55f30ef0c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.016+0000 7f0493400700 1 -- 172.21.15.1:0/369276539 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f30ef129f0 msgr2=0x55f30ef0cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:00.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.016+0000 7f0493400700 1 --2- 172.21.15.1:0/369276539 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f30ef129f0 0x55f30ef0cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.016+0000 7f0493400700 1 -- 172.21.15.1:0/369276539 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f30ec86b60 con 0x55f30ef071f0 2024-02-02T15:33:00.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.016+0000 7f0493400700 1 --2- 172.21.15.1:0/369276539 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f30ef071f0 0x55f30ef16e00 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0484009f60 tx=0x7f0484009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eac5775ee92fad0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:33:00.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.017+0000 7f0489600700 1 -- 172.21.15.1:0/369276539 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0484016040 con 0x55f30ef071f0 2024-02-02T15:33:00.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.017+0000 7f0489600700 1 -- 172.21.15.1:0/369276539 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0484003b40 con 0x55f30ef071f0 2024-02-02T15:33:00.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.017+0000 7f0489600700 1 -- 172.21.15.1:0/369276539 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f048401a400 con 0x55f30ef071f0 2024-02-02T15:33:00.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.017+0000 7f04ae165b80 1 -- 172.21.15.1:0/369276539 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f30ef071f0 msgr2=0x55f30ef16e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:00.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.017+0000 7f04ae165b80 1 --2- 172.21.15.1:0/369276539 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f30ef071f0 0x55f30ef16e00 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0484009f60 tx=0x7f0484009f90 comp rx=0 tx=0).stop 2024-02-02T15:33:00.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.017+0000 7f04ae165b80 1 -- 172.21.15.1:0/369276539 shutdown_connections 2024-02-02T15:33:00.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.018+0000 7f04ae165b80 1 --2- 172.21.15.1:0/369276539 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f30ef129f0 0x55f30ef0cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.018+0000 7f04ae165b80 1 --2- 172.21.15.1:0/369276539 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f30ef071f0 0x55f30ef16e00 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.018+0000 7f04ae165b80 1 --2- 172.21.15.1:0/369276539 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f30ef07b10 0x55f30ef0c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.018+0000 7f04ae165b80 1 -- 172.21.15.1:0/369276539 >> 172.21.15.1:0/369276539 conn(0x55f30ee66db0 msgr2=0x55f30ef1e380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:00.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.018+0000 7f04ae165b80 1 -- 172.21.15.1:0/369276539 shutdown_connections 2024-02-02T15:33:00.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.018+0000 7f04ae165b80 1 -- 172.21.15.1:0/369276539 wait complete. 2024-02-02T15:33:00.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.018+0000 7f04ae165b80 1 -- start start 2024-02-02T15:33:00.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.018+0000 7f04ae165b80 1 Processor -- start 2024-02-02T15:33:00.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.019+0000 7f04ae165b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f30ef071f0 0x55f30ef67b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:00.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.019+0000 7f04ae165b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f30ef07b10 0x55f30ef68070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:00.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.019+0000 7f04ae165b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f30ef129f0 0x55f30ef685b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:00.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.019+0000 7f04ae165b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f30ef72cf0 con 0x55f30ef07b10 2024-02-02T15:33:00.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.019+0000 7f04ae165b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f30ef72e60 con 0x55f30ef129f0 2024-02-02T15:33:00.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.019+0000 7f04ae165b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f30ef73020 con 0x55f30ef071f0 2024-02-02T15:33:00.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.019+0000 7f04ae165b80 10 client.0 _ll_register_callbacks cb 0x55f30edf6290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:33:00.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.019+0000 7f0493e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f30ef129f0 0x55f30ef685b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:00.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.019+0000 7f0493400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f30ef071f0 0x55f30ef67b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:00.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.019+0000 7f0493e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f30ef129f0 0x55f30ef685b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:33756/0 (socket says 172.21.15.1:33756) 2024-02-02T15:33:00.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.019+0000 7f0493e00700 1 -- 172.21.15.1:0/3118436475 learned_addr learned my addr 172.21.15.1:0/3118436475 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:33:00.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.019+0000 7f0492a00700 1 --2- 172.21.15.1:0/3118436475 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f30ef07b10 0x55f30ef68070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:00.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.020+0000 7f0493400700 1 -- 172.21.15.1:0/3118436475 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f30ef129f0 msgr2=0x55f30ef685b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:00.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.020+0000 7f0493400700 1 --2- 172.21.15.1:0/3118436475 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f30ef129f0 0x55f30ef685b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.020+0000 7f0493400700 1 -- 172.21.15.1:0/3118436475 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f30ef07b10 msgr2=0x55f30ef68070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:00.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.020+0000 7f0493400700 1 --2- 172.21.15.1:0/3118436475 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f30ef07b10 0x55f30ef68070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.020+0000 7f0493400700 1 -- 172.21.15.1:0/3118436475 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04941a58e0 con 0x55f30ef071f0 2024-02-02T15:33:00.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.020+0000 7f0493400700 1 --2- 172.21.15.1:0/3118436475 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f30ef071f0 0x55f30ef67b30 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f0484003d50 tx=0x7f048400ffd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf1cb9d8f16a94f3 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:33:00.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.020+0000 7f0481600700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:33:00.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.020+0000 7f0481600700 1 -- 172.21.15.1:0/3118436475 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0484016040 con 0x55f30ef071f0 2024-02-02T15:33:00.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.020+0000 7f0481600700 1 -- 172.21.15.1:0/3118436475 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f0484000e40 con 0x55f30ef071f0 2024-02-02T15:33:00.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.020+0000 7f04ae165b80 1 -- 172.21.15.1:0/3118436475 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f047c09ad70 con 0x55f30ef071f0 2024-02-02T15:33:00.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.020+0000 7f0481600700 1 -- 172.21.15.1:0/3118436475 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0484011220 con 0x55f30ef071f0 2024-02-02T15:33:00.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.021+0000 7f04ae165b80 10 client.27211 fetch_fsmap learned FSMap version 38 2024-02-02T15:33:00.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.021+0000 7f04ae165b80 1 -- 172.21.15.1:0/3118436475 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55f30ee1eea0 con 0x55f30ef071f0 2024-02-02T15:33:00.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.021+0000 7f0481600700 1 -- 172.21.15.1:0/3118436475 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f047c09ad70 con 0x55f30ef071f0 2024-02-02T15:33:00.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.021+0000 7f0481600700 1 -- 172.21.15.1:0/3118436475 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 38) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f0484030040 con 0x55f30ef071f0 2024-02-02T15:33:00.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.021+0000 7f04ae165b80 10 client.27211 fetch_fsmap finished waiting for FSMap version 38 2024-02-02T15:33:00.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.021+0000 7f04ae165b80 10 client.27211 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-02-02T15:33:00.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.021+0000 7f04ae165b80 20 client.27211 populate_metadata read hostname 'smithi001' 2024-02-02T15:33:00.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.021+0000 7f04ae165b80 1 --2- 172.21.15.1:0/3118436475 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55f30ef743e0 0x55f30ef767d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:00.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.021+0000 7f04ae165b80 4 client.27211 mds_command: new command op to 24514 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:33:00.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.021+0000 7f04ae165b80 1 -- 172.21.15.1:0/3118436475 --> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f30ee17f50 con 0x55f30ef743e0 2024-02-02T15:33:00.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.021+0000 7f0492a00700 1 --2- 172.21.15.1:0/3118436475 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55f30ef743e0 0x55f30ef767d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:00.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.022+0000 7f0492a00700 1 --2- 172.21.15.1:0/3118436475 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55f30ef743e0 0x55f30ef767d0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c5575e34407931f1 server_cookie=30c0a2f02b109eae in_seq=0 out_seq=0 2024-02-02T15:33:00.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.022+0000 7f0481600700 10 client.27211 ms_handle_connect on v2:172.21.15.142:6836/2870289019 2024-02-02T15:33:00.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.022+0000 7f0481600700 1 -- 172.21.15.1:0/3118436475 <== mds.3 v2:172.21.15.142:6836/2870289019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f30ee17f50 con 0x55f30ef743e0 2024-02-02T15:33:00.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.022+0000 7f0481600700 10 client.27211 handle_command_reply: tid=0 2024-02-02T15:33:00.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.022+0000 7f04ae165b80 1 client.27211 shutdown 2024-02-02T15:33:00.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.022+0000 7f04ae165b80 1 -- 172.21.15.1:0/3118436475 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f30ef071f0 msgr2=0x55f30ef67b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:00.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.022+0000 7f04ae165b80 1 --2- 172.21.15.1:0/3118436475 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f30ef071f0 0x55f30ef67b30 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f0484003d50 tx=0x7f048400ffd0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.022+0000 7f04ae165b80 1 -- 172.21.15.1:0/3118436475 shutdown_connections 2024-02-02T15:33:00.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.022+0000 7f04ae165b80 1 --2- 172.21.15.1:0/3118436475 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f30ef129f0 0x55f30ef685b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.022+0000 7f04ae165b80 1 --2- 172.21.15.1:0/3118436475 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f30ef071f0 0x55f30ef67b30 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.022+0000 7f04ae165b80 1 --2- 172.21.15.1:0/3118436475 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55f30ef743e0 0x55f30ef767d0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.022+0000 7f04ae165b80 1 --2- 172.21.15.1:0/3118436475 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f30ef07b10 0x55f30ef68070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.022+0000 7f04ae165b80 1 -- 172.21.15.1:0/3118436475 >> 172.21.15.1:0/3118436475 conn(0x55f30ee66db0 msgr2=0x55f30ef0cdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:00.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.022+0000 7f0481600700 0 client.27211 ms_handle_reset on v2:172.21.15.142:6836/2870289019 2024-02-02T15:33:00.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.022+0000 7f04ae165b80 1 -- 172.21.15.1:0/3118436475 shutdown_connections 2024-02-02T15:33:00.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.022+0000 7f04ae165b80 1 -- 172.21.15.1:0/3118436475 wait complete. 2024-02-02T15:33:00.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.023+0000 7f04ae165b80 20 client.27211 trim_cache size 0 max 16384 2024-02-02T15:33:00.027 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:33:00.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.023+0000 7f0499000700 1 -- 172.21.15.1:0/1364271482 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f04740a0710 msgr2=0x7f04740a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:00.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.023+0000 7f0499000700 1 --2- 172.21.15.1:0/1364271482 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f04740a0710 0x7f04740a2bd0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f048c009fd0 tx=0x7f048c009920 comp rx=0 tx=0).stop 2024-02-02T15:33:00.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.023+0000 7f0499000700 1 -- 172.21.15.1:0/1364271482 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f04940689d0 msgr2=0x7f04941001a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:00.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.023+0000 7f0499000700 1 --2- 172.21.15.1:0/1364271482 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f04940689d0 0x7f04941001a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f047c000c00 tx=0x7f047c005740 comp rx=0 tx=0).stop 2024-02-02T15:33:00.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.023+0000 7f0499000700 1 -- 172.21.15.1:0/1364271482 shutdown_connections 2024-02-02T15:33:00.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.023+0000 7f0499000700 1 --2- 172.21.15.1:0/1364271482 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f04740a0710 0x7f04740a2bd0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.023+0000 7f0499000700 1 --2- 172.21.15.1:0/1364271482 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0494106ab0 0x7f04941006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.023+0000 7f0499000700 1 --2- 172.21.15.1:0/1364271482 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0494068010 0x7f04940ffc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.023+0000 7f0499000700 1 --2- 172.21.15.1:0/1364271482 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f04940689d0 0x7f04941001a0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.023+0000 7f0499000700 1 -- 172.21.15.1:0/1364271482 >> 172.21.15.1:0/1364271482 conn(0x7f0494076770 msgr2=0x7f04940fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:00.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.023+0000 7f0499000700 1 -- 172.21.15.1:0/1364271482 shutdown_connections 2024-02-02T15:33:00.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.023+0000 7f0499000700 1 -- 172.21.15.1:0/1364271482 wait complete. 2024-02-02T15:33:00.042 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-02T15:33:00.237 INFO:tasks.workunit.client.0.smithi001.stdout: 1 394337 8.89 MB/sec execute 154 sec latency 12.601 ms 2024-02-02T15:33:00.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.266+0000 7fbbbe400700 1 -- 172.21.15.1:0/1964326200 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbbb8100ff0 msgr2=0x7fbbb81013d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:00.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.266+0000 7fbbbe400700 1 --2- 172.21.15.1:0/1964326200 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbbb8100ff0 0x7fbbb81013d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fbba8009990 tx=0x7fbba8009ca0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.267+0000 7fbbbe400700 1 -- 172.21.15.1:0/1964326200 shutdown_connections 2024-02-02T15:33:00.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.267+0000 7fbbbe400700 1 --2- 172.21.15.1:0/1964326200 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbbb8106110 0x7fbbb810dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.267+0000 7fbbbe400700 1 --2- 172.21.15.1:0/1964326200 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbbb8100ff0 0x7fbbb81013d0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.267+0000 7fbbbe400700 1 --2- 172.21.15.1:0/1964326200 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbbb81019a0 0x7fbbb81059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.267+0000 7fbbbe400700 1 -- 172.21.15.1:0/1964326200 >> 172.21.15.1:0/1964326200 conn(0x7fbbb80fc810 msgr2=0x7fbbb80fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:00.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.267+0000 7fbbbe400700 1 -- 172.21.15.1:0/1964326200 shutdown_connections 2024-02-02T15:33:00.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.267+0000 7fbbbe400700 1 -- 172.21.15.1:0/1964326200 wait complete. 2024-02-02T15:33:00.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.268+0000 7fbbbe400700 1 Processor -- start 2024-02-02T15:33:00.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.268+0000 7fbbbe400700 1 -- start start 2024-02-02T15:33:00.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.268+0000 7fbbbe400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbbb8100ff0 0x7fbbb8199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:00.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.268+0000 7fbbbe400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbbb8106110 0x7fbbb8199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:00.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.268+0000 7fbbbe400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbbb819dad0 0x7fbbb819df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:00.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.268+0000 7fbbbe400700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb819e550 con 0x7fbbb8106110 2024-02-02T15:33:00.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.268+0000 7fbbbe400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb819e6c0 con 0x7fbbb8100ff0 2024-02-02T15:33:00.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.268+0000 7fbbbe400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb819e880 con 0x7fbbb819dad0 2024-02-02T15:33:00.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.268+0000 7fbbb6a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbbb8106110 0x7fbbb8199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:00.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.268+0000 7fbbb7400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbbb8100ff0 0x7fbbb8199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:00.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.269+0000 7fbbb7400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbbb8100ff0 0x7fbbb8199130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:33774/0 (socket says 172.21.15.1:33774) 2024-02-02T15:33:00.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.269+0000 7fbbb7400700 1 -- 172.21.15.1:0/1484108898 learned_addr learned my addr 172.21.15.1:0/1484108898 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:33:00.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.269+0000 7fbbb7e00700 1 --2- 172.21.15.1:0/1484108898 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbbb819dad0 0x7fbbb819df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:00.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.269+0000 7fbbb7400700 1 -- 172.21.15.1:0/1484108898 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbbb819dad0 msgr2=0x7fbbb819df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:00.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.269+0000 7fbbb7400700 1 --2- 172.21.15.1:0/1484108898 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbbb819dad0 0x7fbbb819df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.269+0000 7fbbb7400700 1 -- 172.21.15.1:0/1484108898 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbbb8106110 msgr2=0x7fbbb8199670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:00.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.269+0000 7fbbb7400700 1 --2- 172.21.15.1:0/1484108898 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbbb8106110 0x7fbbb8199670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.269+0000 7fbbb7400700 1 -- 172.21.15.1:0/1484108898 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbba80095f0 con 0x7fbbb8100ff0 2024-02-02T15:33:00.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.269+0000 7fbbb7400700 1 --2- 172.21.15.1:0/1484108898 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbbb8100ff0 0x7fbbb8199130 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fbba8005910 tx=0x7fbba8005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:33:00.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.269+0000 7fbbafe00700 1 -- 172.21.15.1:0/1484108898 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba801d070 con 0x7fbbb8100ff0 2024-02-02T15:33:00.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.270+0000 7fbbbe400700 1 -- 172.21.15.1:0/1484108898 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbb81a5d80 con 0x7fbbb8100ff0 2024-02-02T15:33:00.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.270+0000 7fbbafe00700 1 -- 172.21.15.1:0/1484108898 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbba8022470 con 0x7fbbb8100ff0 2024-02-02T15:33:00.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.270+0000 7fbbafe00700 1 -- 172.21.15.1:0/1484108898 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba8017650 con 0x7fbbb8100ff0 2024-02-02T15:33:00.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.270+0000 7fbbbe400700 1 -- 172.21.15.1:0/1484108898 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbb81a61b0 con 0x7fbbb8100ff0 2024-02-02T15:33:00.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.273+0000 7fbbafe00700 1 -- 172.21.15.1:0/1484108898 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 27) v1 ==== 135200+0+0 (secure 0 0 0) 0x7fbba80225e0 con 0x7fbbb8100ff0 2024-02-02T15:33:00.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.274+0000 7fbbafe00700 1 --2- 172.21.15.1:0/1484108898 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fbb980a0710 0x7fbb980a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:00.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.274+0000 7fbbd1ec2b80 1 Processor -- start 2024-02-02T15:33:00.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.274+0000 7fbbafe00700 1 -- 172.21.15.1:0/1484108898 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fbba80cd3c0 con 0x7fbbb8100ff0 2024-02-02T15:33:00.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.274+0000 7fbbb6a00700 1 --2- 172.21.15.1:0/1484108898 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fbb980a0710 0x7fbb980a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:00.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.274+0000 7fbbd1ec2b80 1 -- start start 2024-02-02T15:33:00.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.275+0000 7fbbd1ec2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be49b42010 0x55be49b423f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:00.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.275+0000 7fbbd1ec2b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55be49b42930 0x55be49b4d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:00.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.275+0000 7fbbd1ec2b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be49b4d7e0 0x55be49b4fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:00.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.275+0000 7fbbd1ec2b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be49ab64c0 con 0x55be49b42930 2024-02-02T15:33:00.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.275+0000 7fbbd1ec2b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be49a9aaa0 con 0x55be49b4d7e0 2024-02-02T15:33:00.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.275+0000 7fbbd1ec2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be49a98460 con 0x55be49b42010 2024-02-02T15:33:00.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.275+0000 7fbbb7400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be49b42010 0x55be49b423f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:00.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.275+0000 7fbbb7400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be49b42010 0x55be49b423f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:39680/0 (socket says 172.21.15.1:39680) 2024-02-02T15:33:00.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.275+0000 7fbbb7400700 1 -- 172.21.15.1:0/1283579145 learned_addr learned my addr 172.21.15.1:0/1283579145 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:33:00.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.275+0000 7fbbb7e00700 1 --2- 172.21.15.1:0/1283579145 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be49b4d7e0 0x55be49b4fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:00.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.275+0000 7fbbb6a00700 1 --2- 172.21.15.1:0/1283579145 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55be49b42930 0x55be49b4d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:00.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.275+0000 7fbbb6a00700 1 --2- 172.21.15.1:0/1484108898 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fbb980a0710 0x7fbb980a2bd0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fbbb819a740 tx=0x7fbba0009260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:33:00.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.275+0000 7fbbb7400700 1 -- 172.21.15.1:0/1283579145 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be49b4d7e0 msgr2=0x55be49b4fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:00.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.275+0000 7fbbb7400700 1 --2- 172.21.15.1:0/1283579145 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be49b4d7e0 0x55be49b4fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.275+0000 7fbbb7400700 1 -- 172.21.15.1:0/1283579145 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55be49b42930 msgr2=0x55be49b4d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:00.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.275+0000 7fbbb7400700 1 --2- 172.21.15.1:0/1283579145 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55be49b42930 0x55be49b4d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.275+0000 7fbbb7400700 1 -- 172.21.15.1:0/1283579145 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be499153b0 con 0x55be49b42010 2024-02-02T15:33:00.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.275+0000 7fbbb6a00700 1 --2- 172.21.15.1:0/1283579145 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55be49b42930 0x55be49b4d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:33:00.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.276+0000 7fbbb7400700 1 --2- 172.21.15.1:0/1283579145 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be49b42010 0x55be49b423f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fbba8000c00 tx=0x7fbba8009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cbbc2f75b0b239ef server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:33:00.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.276+0000 7fbbacc00700 1 -- 172.21.15.1:0/1283579145 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba80a0050 con 0x55be49b42010 2024-02-02T15:33:00.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.276+0000 7fbbacc00700 1 -- 172.21.15.1:0/1283579145 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbba8099f20 con 0x55be49b42010 2024-02-02T15:33:00.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.276+0000 7fbbacc00700 1 -- 172.21.15.1:0/1283579145 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba80228c0 con 0x55be49b42010 2024-02-02T15:33:00.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.277+0000 7fbbd1ec2b80 1 -- 172.21.15.1:0/1283579145 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be49b42010 msgr2=0x55be49b423f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:00.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.277+0000 7fbbd1ec2b80 1 --2- 172.21.15.1:0/1283579145 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be49b42010 0x55be49b423f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fbba8000c00 tx=0x7fbba8009310 comp rx=0 tx=0).stop 2024-02-02T15:33:00.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.277+0000 7fbbd1ec2b80 1 -- 172.21.15.1:0/1283579145 shutdown_connections 2024-02-02T15:33:00.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.277+0000 7fbbd1ec2b80 1 --2- 172.21.15.1:0/1283579145 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be49b4d7e0 0x55be49b4fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.277+0000 7fbbd1ec2b80 1 --2- 172.21.15.1:0/1283579145 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be49b42010 0x55be49b423f0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.277+0000 7fbbd1ec2b80 1 --2- 172.21.15.1:0/1283579145 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55be49b42930 0x55be49b4d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.277+0000 7fbbd1ec2b80 1 -- 172.21.15.1:0/1283579145 >> 172.21.15.1:0/1283579145 conn(0x55be49aa1db0 msgr2=0x55be49b3e3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:00.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.277+0000 7fbbd1ec2b80 1 -- 172.21.15.1:0/1283579145 shutdown_connections 2024-02-02T15:33:00.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.277+0000 7fbbd1ec2b80 1 -- 172.21.15.1:0/1283579145 wait complete. 2024-02-02T15:33:00.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.277+0000 7fbbd1ec2b80 1 -- start start 2024-02-02T15:33:00.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.278+0000 7fbbd1ec2b80 1 Processor -- start 2024-02-02T15:33:00.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.278+0000 7fbbd1ec2b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be49b42010 0x55be49b4dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:00.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.279+0000 7fbbd1ec2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be49b42930 0x55be49b4e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:00.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.279+0000 7fbbd1ec2b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55be49b4d7e0 0x55be49b4e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:00.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.279+0000 7fbbd1ec2b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be49a93d50 con 0x55be49b4d7e0 2024-02-02T15:33:00.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.279+0000 7fbbd1ec2b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be49a2fd80 con 0x55be49b42010 2024-02-02T15:33:00.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.279+0000 7fbbd1ec2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be49a2fa20 con 0x55be49b42930 2024-02-02T15:33:00.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.279+0000 7fbbd1ec2b80 10 client.0 _ll_register_callbacks cb 0x55be49a31290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:33:00.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.279+0000 7fbbb7400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be49b42010 0x55be49b4dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:00.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.279+0000 7fbbb6a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be49b42930 0x55be49b4e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:00.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.279+0000 7fbbb7400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be49b42010 0x55be49b4dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:33788/0 (socket says 172.21.15.1:33788) 2024-02-02T15:33:00.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.279+0000 7fbbb6a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be49b42930 0x55be49b4e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:39692/0 (socket says 172.21.15.1:39692) 2024-02-02T15:33:00.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.279+0000 7fbbb7400700 1 -- 172.21.15.1:0/4052126286 learned_addr learned my addr 172.21.15.1:0/4052126286 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:33:00.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.279+0000 7fbbb7e00700 1 --2- 172.21.15.1:0/4052126286 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55be49b4d7e0 0x55be49b4e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:00.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.279+0000 7fbbb7400700 1 -- 172.21.15.1:0/4052126286 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be49b42930 msgr2=0x55be49b4e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:00.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.279+0000 7fbbb7400700 1 --2- 172.21.15.1:0/4052126286 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be49b42930 0x55be49b4e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.279+0000 7fbbb7400700 1 -- 172.21.15.1:0/4052126286 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55be49b4d7e0 msgr2=0x55be49b4e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:00.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.279+0000 7fbbb7400700 1 --2- 172.21.15.1:0/4052126286 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55be49b4d7e0 0x55be49b4e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.279+0000 7fbbb7400700 1 -- 172.21.15.1:0/4052126286 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbb81a61b0 con 0x55be49b42010 2024-02-02T15:33:00.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.279+0000 7fbbb7e00700 1 --2- 172.21.15.1:0/4052126286 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55be49b4d7e0 0x55be49b4e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:33:00.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.279+0000 7fbbb7400700 1 --2- 172.21.15.1:0/4052126286 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be49b42010 0x55be49b4dbc0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fbba80a2070 tx=0x7fbba80ca780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1aee7c2352766a0a server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:33:00.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.279+0000 7fbba5600700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:33:00.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.280+0000 7fbba5600700 1 -- 172.21.15.1:0/4052126286 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba80a25a0 con 0x55be49b42010 2024-02-02T15:33:00.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.280+0000 7fbba5600700 1 -- 172.21.15.1:0/4052126286 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fbba809acc0 con 0x55be49b42010 2024-02-02T15:33:00.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.280+0000 7fbbd1ec2b80 1 -- 172.21.15.1:0/4052126286 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55be49b360f0 con 0x55be49b42010 2024-02-02T15:33:00.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.280+0000 7fbba5600700 1 -- 172.21.15.1:0/4052126286 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbba80228c0 con 0x55be49b42010 2024-02-02T15:33:00.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.280+0000 7fbbd1ec2b80 10 client.27224 fetch_fsmap learned FSMap version 38 2024-02-02T15:33:00.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.280+0000 7fbbd1ec2b80 1 -- 172.21.15.1:0/4052126286 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55be498c1b60 con 0x55be49b42010 2024-02-02T15:33:00.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.280+0000 7fbba5600700 1 -- 172.21.15.1:0/4052126286 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55be49b360f0 con 0x55be49b42010 2024-02-02T15:33:00.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.280+0000 7fbba5600700 1 -- 172.21.15.1:0/4052126286 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 38) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fbba809a820 con 0x55be49b42010 2024-02-02T15:33:00.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.280+0000 7fbbd1ec2b80 10 client.27224 fetch_fsmap finished waiting for FSMap version 38 2024-02-02T15:33:00.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.280+0000 7fbbd1ec2b80 10 client.27224 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-02-02T15:33:00.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.280+0000 7fbbd1ec2b80 20 client.27224 populate_metadata read hostname 'smithi001' 2024-02-02T15:33:00.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.280+0000 7fbbd1ec2b80 1 --2- 172.21.15.1:0/4052126286 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55be49b5ca80 0x55be49b5eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:00.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.280+0000 7fbbd1ec2b80 4 client.27224 mds_command: new command op to 24518 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:33:00.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.280+0000 7fbbd1ec2b80 1 -- 172.21.15.1:0/4052126286 --> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55be49a8cdd0 con 0x55be49b5ca80 2024-02-02T15:33:00.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.281+0000 7fbbb6a00700 1 --2- 172.21.15.1:0/4052126286 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55be49b5ca80 0x55be49b5eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:00.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.281+0000 7fbbb6a00700 1 --2- 172.21.15.1:0/4052126286 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55be49b5ca80 0x55be49b5eea0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ca48becb32ac67a9 server_cookie=28c887bf676ee8fd in_seq=0 out_seq=0 2024-02-02T15:33:00.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.281+0000 7fbba5600700 10 client.27224 ms_handle_connect on v2:172.21.15.71:6836/628016199 2024-02-02T15:33:00.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.282+0000 7fbba5600700 1 -- 172.21.15.1:0/4052126286 <== mds.2 v2:172.21.15.71:6836/628016199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55be49a8cdd0 con 0x55be49b5ca80 2024-02-02T15:33:00.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.282+0000 7fbba5600700 10 client.27224 handle_command_reply: tid=0 2024-02-02T15:33:00.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.282+0000 7fbbd1ec2b80 1 client.27224 shutdown 2024-02-02T15:33:00.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.282+0000 7fbbd1ec2b80 1 -- 172.21.15.1:0/4052126286 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be49b42010 msgr2=0x55be49b4dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:00.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.282+0000 7fbbd1ec2b80 1 --2- 172.21.15.1:0/4052126286 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be49b42010 0x55be49b4dbc0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fbba80a2070 tx=0x7fbba80ca780 comp rx=0 tx=0).stop 2024-02-02T15:33:00.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.282+0000 7fbbd1ec2b80 1 -- 172.21.15.1:0/4052126286 shutdown_connections 2024-02-02T15:33:00.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.282+0000 7fbbd1ec2b80 1 --2- 172.21.15.1:0/4052126286 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55be49b4d7e0 0x55be49b4e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.282+0000 7fbbd1ec2b80 1 --2- 172.21.15.1:0/4052126286 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be49b42010 0x55be49b4dbc0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.282+0000 7fbbd1ec2b80 1 --2- 172.21.15.1:0/4052126286 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55be49b5ca80 0x55be49b5eea0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.282+0000 7fbbd1ec2b80 1 --2- 172.21.15.1:0/4052126286 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be49b42930 0x55be49b4e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.282+0000 7fbbd1ec2b80 1 -- 172.21.15.1:0/4052126286 >> 172.21.15.1:0/4052126286 conn(0x55be49aa1db0 msgr2=0x55be49b3dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:00.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.282+0000 7fbba5600700 0 client.27224 ms_handle_reset on v2:172.21.15.71:6836/628016199 2024-02-02T15:33:00.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.283+0000 7fbbd1ec2b80 1 -- 172.21.15.1:0/4052126286 shutdown_connections 2024-02-02T15:33:00.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.283+0000 7fbbd1ec2b80 1 -- 172.21.15.1:0/4052126286 wait complete. 2024-02-02T15:33:00.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.283+0000 7fbbd1ec2b80 20 client.27224 trim_cache size 0 max 16384 2024-02-02T15:33:00.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.298+0000 7fbbd1ec2b80 1 Processor -- start 2024-02-02T15:33:00.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.298+0000 7fbbd1ec2b80 1 -- start start 2024-02-02T15:33:00.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.298+0000 7fbbd1ec2b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55be49b42010 0x55be49b4e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:00.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.298+0000 7fbbd1ec2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be49b42930 0x55be49b581b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:00.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.298+0000 7fbbd1ec2b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be49b4d7e0 0x55be49b586f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:00.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.298+0000 7fbbd1ec2b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be499176a0 con 0x55be49b42010 2024-02-02T15:33:00.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.298+0000 7fbbd1ec2b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be49b38560 con 0x55be49b4d7e0 2024-02-02T15:33:00.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.298+0000 7fbbd1ec2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be49b4ceb0 con 0x55be49b42930 2024-02-02T15:33:00.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.298+0000 7fbbb7400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55be49b42010 0x55be49b4e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:00.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.298+0000 7fbbb7400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55be49b42010 0x55be49b4e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45112/0 (socket says 172.21.15.1:45112) 2024-02-02T15:33:00.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.298+0000 7fbbb7400700 1 -- 172.21.15.1:0/1853602196 learned_addr learned my addr 172.21.15.1:0/1853602196 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:33:00.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.298+0000 7fbbb7e00700 1 --2- 172.21.15.1:0/1853602196 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be49b4d7e0 0x55be49b586f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:00.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.298+0000 7fbbb6a00700 1 --2- 172.21.15.1:0/1853602196 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be49b42930 0x55be49b581b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:00.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.299+0000 7fbbb7400700 1 -- 172.21.15.1:0/1853602196 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be49b42930 msgr2=0x55be49b581b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:00.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.299+0000 7fbbb7400700 1 --2- 172.21.15.1:0/1853602196 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be49b42930 0x55be49b581b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.299+0000 7fbbb7400700 1 -- 172.21.15.1:0/1853602196 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be49b4d7e0 msgr2=0x55be49b586f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:00.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.299+0000 7fbbb7400700 1 --2- 172.21.15.1:0/1853602196 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be49b4d7e0 0x55be49b586f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.299+0000 7fbbb7400700 1 -- 172.21.15.1:0/1853602196 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be49a59ea0 con 0x55be49b42010 2024-02-02T15:33:00.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.299+0000 7fbbb7400700 1 --2- 172.21.15.1:0/1853602196 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55be49b42010 0x55be49b4e2d0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fbba800f740 tx=0x7fbba8009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ec5407c15051d2d server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:33:00.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.299+0000 7fbbad600700 1 -- 172.21.15.1:0/1853602196 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba80a0040 con 0x55be49b42010 2024-02-02T15:33:00.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.299+0000 7fbbad600700 1 -- 172.21.15.1:0/1853602196 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbba80b2060 con 0x55be49b42010 2024-02-02T15:33:00.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.299+0000 7fbbad600700 1 -- 172.21.15.1:0/1853602196 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba80228c0 con 0x55be49b42010 2024-02-02T15:33:00.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.300+0000 7fbbd1ec2b80 1 -- 172.21.15.1:0/1853602196 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55be49b42010 msgr2=0x55be49b4e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:00.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.300+0000 7fbbd1ec2b80 1 --2- 172.21.15.1:0/1853602196 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55be49b42010 0x55be49b4e2d0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fbba800f740 tx=0x7fbba8009ca0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.300+0000 7fbbd1ec2b80 1 -- 172.21.15.1:0/1853602196 shutdown_connections 2024-02-02T15:33:00.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.300+0000 7fbbd1ec2b80 1 --2- 172.21.15.1:0/1853602196 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be49b4d7e0 0x55be49b586f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.300+0000 7fbbd1ec2b80 1 --2- 172.21.15.1:0/1853602196 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55be49b42010 0x55be49b4e2d0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.300+0000 7fbbd1ec2b80 1 --2- 172.21.15.1:0/1853602196 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be49b42930 0x55be49b581b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.300+0000 7fbbd1ec2b80 1 -- 172.21.15.1:0/1853602196 >> 172.21.15.1:0/1853602196 conn(0x55be49aa1db0 msgr2=0x55be49b4dbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:00.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.300+0000 7fbbd1ec2b80 1 -- 172.21.15.1:0/1853602196 shutdown_connections 2024-02-02T15:33:00.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.300+0000 7fbbd1ec2b80 1 -- 172.21.15.1:0/1853602196 wait complete. 2024-02-02T15:33:00.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.301+0000 7fbbd1ec2b80 1 -- start start 2024-02-02T15:33:00.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.301+0000 7fbbd1ec2b80 1 Processor -- start 2024-02-02T15:33:00.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.302+0000 7fbbd1ec2b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55be49b42010 0x55be49b52b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:00.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.302+0000 7fbbd1ec2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be49b42930 0x55be49b50000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:00.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.302+0000 7fbbd1ec2b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be49b4d7e0 0x55be49b50630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:00.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.302+0000 7fbbd1ec2b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be49b50d70 con 0x55be49b42010 2024-02-02T15:33:00.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.302+0000 7fbbd1ec2b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be49bad560 con 0x55be49b4d7e0 2024-02-02T15:33:00.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.302+0000 7fbbd1ec2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be49bad6a0 con 0x55be49b42930 2024-02-02T15:33:00.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.302+0000 7fbbd1ec2b80 10 client.0 _ll_register_callbacks cb 0x55be49a31290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:33:00.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.302+0000 7fbbb6a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be49b42930 0x55be49b50000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:00.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.302+0000 7fbbb6a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be49b42930 0x55be49b50000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:39716/0 (socket says 172.21.15.1:39716) 2024-02-02T15:33:00.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.302+0000 7fbbb6a00700 1 -- 172.21.15.1:0/1566703646 learned_addr learned my addr 172.21.15.1:0/1566703646 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:33:00.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.302+0000 7fbbb7400700 1 --2- 172.21.15.1:0/1566703646 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55be49b42010 0x55be49b52b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:00.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.302+0000 7fbbb7e00700 1 --2- 172.21.15.1:0/1566703646 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be49b4d7e0 0x55be49b50630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:00.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.302+0000 7fbbb6a00700 1 -- 172.21.15.1:0/1566703646 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be49b4d7e0 msgr2=0x55be49b50630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:00.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.302+0000 7fbbb6a00700 1 --2- 172.21.15.1:0/1566703646 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be49b4d7e0 0x55be49b50630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.302+0000 7fbbb6a00700 1 -- 172.21.15.1:0/1566703646 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55be49b42010 msgr2=0x55be49b52b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:00.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.302+0000 7fbbb6a00700 1 --2- 172.21.15.1:0/1566703646 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55be49b42010 0x55be49b52b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.302+0000 7fbbb6a00700 1 -- 172.21.15.1:0/1566703646 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55be498c1b60 con 0x55be49b42930 2024-02-02T15:33:00.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.303+0000 7fbbb6a00700 1 --2- 172.21.15.1:0/1566703646 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be49b42930 0x55be49b50000 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x55be49b4ce30 tx=0x7fbba00047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94eb516af6b1ae3a server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:33:00.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.303+0000 7fbba5600700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:33:00.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.303+0000 7fbba5600700 1 -- 172.21.15.1:0/1566703646 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba0023040 con 0x55be49b42930 2024-02-02T15:33:00.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.303+0000 7fbba5600700 1 -- 172.21.15.1:0/1566703646 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fbba00273c0 con 0x55be49b42930 2024-02-02T15:33:00.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.303+0000 7fbbd1ec2b80 1 -- 172.21.15.1:0/1566703646 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbba8099f20 con 0x55be49b42930 2024-02-02T15:33:00.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.303+0000 7fbba5600700 1 -- 172.21.15.1:0/1566703646 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbba00275b0 con 0x55be49b42930 2024-02-02T15:33:00.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.303+0000 7fbbd1ec2b80 10 client.27229 fetch_fsmap learned FSMap version 38 2024-02-02T15:33:00.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.303+0000 7fbbd1ec2b80 1 -- 172.21.15.1:0/1566703646 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55be49919cd0 con 0x55be49b42930 2024-02-02T15:33:00.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.304+0000 7fbba5600700 1 -- 172.21.15.1:0/1566703646 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbba8099f20 con 0x55be49b42930 2024-02-02T15:33:00.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.304+0000 7fbba5600700 1 -- 172.21.15.1:0/1566703646 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 38) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fbba001d410 con 0x55be49b42930 2024-02-02T15:33:00.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.304+0000 7fbbd1ec2b80 10 client.27229 fetch_fsmap finished waiting for FSMap version 38 2024-02-02T15:33:00.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.304+0000 7fbbd1ec2b80 10 client.27229 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-02-02T15:33:00.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.304+0000 7fbbd1ec2b80 20 client.27229 populate_metadata read hostname 'smithi001' 2024-02-02T15:33:00.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.304+0000 7fbbd1ec2b80 1 --2- 172.21.15.1:0/1566703646 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55be49baed60 0x55be49bb1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:00.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.304+0000 7fbbd1ec2b80 4 client.27229 mds_command: new command op to 24518 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:33:00.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.304+0000 7fbbd1ec2b80 1 -- 172.21.15.1:0/1566703646 --> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55be49a52f50 con 0x55be49baed60 2024-02-02T15:33:00.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.304+0000 7fbbb7400700 1 --2- 172.21.15.1:0/1566703646 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55be49baed60 0x55be49bb1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:00.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.305+0000 7fbba5600700 10 client.27229 ms_handle_connect on v2:172.21.15.71:6836/628016199 2024-02-02T15:33:00.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.305+0000 7fbbb7400700 1 --2- 172.21.15.1:0/1566703646 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55be49baed60 0x55be49bb1180 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cdb4c3a9f663f0a2 server_cookie=a69cd3bc69b16215 in_seq=0 out_seq=0 2024-02-02T15:33:00.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.305+0000 7fbba5600700 1 -- 172.21.15.1:0/1566703646 <== mds.2 v2:172.21.15.71:6836/628016199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55be49a52f50 con 0x55be49baed60 2024-02-02T15:33:00.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.305+0000 7fbba5600700 10 client.27229 handle_command_reply: tid=0 2024-02-02T15:33:00.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.305+0000 7fbbd1ec2b80 1 client.27229 shutdown 2024-02-02T15:33:00.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.305+0000 7fbbd1ec2b80 1 -- 172.21.15.1:0/1566703646 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be49b42930 msgr2=0x55be49b50000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:00.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.305+0000 7fbbd1ec2b80 1 --2- 172.21.15.1:0/1566703646 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be49b42930 0x55be49b50000 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x55be49b4ce30 tx=0x7fbba00047c0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.305+0000 7fbbd1ec2b80 1 -- 172.21.15.1:0/1566703646 shutdown_connections 2024-02-02T15:33:00.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.305+0000 7fbbd1ec2b80 1 --2- 172.21.15.1:0/1566703646 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be49b4d7e0 0x55be49b50630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.305+0000 7fbbd1ec2b80 1 --2- 172.21.15.1:0/1566703646 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55be49b42010 0x55be49b52b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.305+0000 7fbbd1ec2b80 1 --2- 172.21.15.1:0/1566703646 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55be49baed60 0x55be49bb1180 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.305+0000 7fbbd1ec2b80 1 --2- 172.21.15.1:0/1566703646 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be49b42930 0x55be49b50000 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.305+0000 7fbba5600700 0 client.27229 ms_handle_reset on v2:172.21.15.71:6836/628016199 2024-02-02T15:33:00.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.305+0000 7fbbd1ec2b80 1 -- 172.21.15.1:0/1566703646 >> 172.21.15.1:0/1566703646 conn(0x55be49aa1db0 msgr2=0x55be49b4b5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:00.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.306+0000 7fbbd1ec2b80 1 -- 172.21.15.1:0/1566703646 shutdown_connections 2024-02-02T15:33:00.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.306+0000 7fbbd1ec2b80 1 -- 172.21.15.1:0/1566703646 wait complete. 2024-02-02T15:33:00.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.306+0000 7fbbd1ec2b80 20 client.27229 trim_cache size 0 max 16384 2024-02-02T15:33:00.310 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:33:00.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.307+0000 7fbbbe400700 1 -- 172.21.15.1:0/1484108898 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fbb980a0710 msgr2=0x7fbb980a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:00.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.307+0000 7fbbbe400700 1 --2- 172.21.15.1:0/1484108898 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fbb980a0710 0x7fbb980a2bd0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fbbb819a740 tx=0x7fbba0009260 comp rx=0 tx=0).stop 2024-02-02T15:33:00.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.307+0000 7fbbbe400700 1 -- 172.21.15.1:0/1484108898 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbbb8100ff0 msgr2=0x7fbbb8199130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:00.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.307+0000 7fbbbe400700 1 --2- 172.21.15.1:0/1484108898 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbbb8100ff0 0x7fbbb8199130 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fbba8005910 tx=0x7fbba8005740 comp rx=0 tx=0).stop 2024-02-02T15:33:00.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.307+0000 7fbbbe400700 1 -- 172.21.15.1:0/1484108898 shutdown_connections 2024-02-02T15:33:00.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.307+0000 7fbbbe400700 1 --2- 172.21.15.1:0/1484108898 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fbb980a0710 0x7fbb980a2bd0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.307+0000 7fbbbe400700 1 --2- 172.21.15.1:0/1484108898 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbbb819dad0 0x7fbbb819df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.307+0000 7fbbbe400700 1 --2- 172.21.15.1:0/1484108898 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbbb8100ff0 0x7fbbb8199130 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.307+0000 7fbbbe400700 1 --2- 172.21.15.1:0/1484108898 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbbb8106110 0x7fbbb8199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.307+0000 7fbbbe400700 1 -- 172.21.15.1:0/1484108898 >> 172.21.15.1:0/1484108898 conn(0x7fbbb80fc810 msgr2=0x7fbbb80fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:00.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.307+0000 7fbbbe400700 1 -- 172.21.15.1:0/1484108898 shutdown_connections 2024-02-02T15:33:00.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.307+0000 7fbbbe400700 1 -- 172.21.15.1:0/1484108898 wait complete. 2024-02-02T15:33:00.327 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-02T15:33:00.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.547+0000 7f93f8200700 1 -- 172.21.15.1:0/1106054751 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f93f0073170 msgr2=0x7f93f0073550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:00.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.547+0000 7f93f8200700 1 --2- 172.21.15.1:0/1106054751 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f93f0073170 0x7f93f0073550 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f93e00098f0 tx=0x7f93e0009c00 comp rx=0 tx=0).stop 2024-02-02T15:33:00.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.547+0000 7f93f8200700 1 -- 172.21.15.1:0/1106054751 shutdown_connections 2024-02-02T15:33:00.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.547+0000 7f93f8200700 1 --2- 172.21.15.1:0/1106054751 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93f010d5c0 0x7f93f010fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.547+0000 7f93f8200700 1 --2- 172.21.15.1:0/1106054751 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f93f0073170 0x7f93f0073550 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.547+0000 7f93f8200700 1 --2- 172.21.15.1:0/1106054751 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93f0073a90 0x7f93f010d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.547+0000 7f93f8200700 1 -- 172.21.15.1:0/1106054751 >> 172.21.15.1:0/1106054751 conn(0x7f93f00fc810 msgr2=0x7f93f00fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:00.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.547+0000 7f93f8200700 1 -- 172.21.15.1:0/1106054751 shutdown_connections 2024-02-02T15:33:00.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.547+0000 7f93f8200700 1 -- 172.21.15.1:0/1106054751 wait complete. 2024-02-02T15:33:00.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.548+0000 7f93f8200700 1 Processor -- start 2024-02-02T15:33:00.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.548+0000 7f93f8200700 1 -- start start 2024-02-02T15:33:00.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.549+0000 7f93f8200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f93f0073170 0x7f93f01992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:00.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.549+0000 7f93f8200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93f0073a90 0x7f93f0199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:00.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.549+0000 7f93f8200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93f010d5c0 0x7f93f019dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:00.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.549+0000 7f93f8200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f019e250 con 0x7f93f0073170 2024-02-02T15:33:00.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.549+0000 7f93f8200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f019e3c0 con 0x7f93f010d5c0 2024-02-02T15:33:00.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.549+0000 7f93f8200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f019e580 con 0x7f93f0073a90 2024-02-02T15:33:00.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.549+0000 7f93f6200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93f010d5c0 0x7f93f019dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:00.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.549+0000 7f93f6200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93f010d5c0 0x7f93f019dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:33822/0 (socket says 172.21.15.1:33822) 2024-02-02T15:33:00.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.549+0000 7f93f6200700 1 -- 172.21.15.1:0/4163217087 learned_addr learned my addr 172.21.15.1:0/4163217087 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:33:00.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.549+0000 7f93f4e00700 1 --2- 172.21.15.1:0/4163217087 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93f0073a90 0x7f93f0199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:00.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.549+0000 7f93f5800700 1 --2- 172.21.15.1:0/4163217087 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f93f0073170 0x7f93f01992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:00.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.549+0000 7f93f6200700 1 -- 172.21.15.1:0/4163217087 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93f0073a90 msgr2=0x7f93f0199820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:00.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.549+0000 7f93f6200700 1 --2- 172.21.15.1:0/4163217087 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93f0073a90 0x7f93f0199820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.549+0000 7f93f6200700 1 -- 172.21.15.1:0/4163217087 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f93f0073170 msgr2=0x7f93f01992e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:00.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.549+0000 7f93f6200700 1 --2- 172.21.15.1:0/4163217087 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f93f0073170 0x7f93f01992e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.549+0000 7f93f6200700 1 -- 172.21.15.1:0/4163217087 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93d80095f0 con 0x7f93f010d5c0 2024-02-02T15:33:00.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.550+0000 7f93f6200700 1 --2- 172.21.15.1:0/4163217087 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93f010d5c0 0x7f93f019dc10 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f93e800ef30 tx=0x7f93e800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:33:00.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.550+0000 7f93ee000700 1 -- 172.21.15.1:0/4163217087 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e8018070 con 0x7f93f010d5c0 2024-02-02T15:33:00.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.550+0000 7f93f8200700 1 -- 172.21.15.1:0/4163217087 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93e0009520 con 0x7f93f010d5c0 2024-02-02T15:33:00.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.550+0000 7f93ee000700 1 -- 172.21.15.1:0/4163217087 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93e800f040 con 0x7f93f010d5c0 2024-02-02T15:33:00.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.550+0000 7f93ee000700 1 -- 172.21.15.1:0/4163217087 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e800cd00 con 0x7f93f010d5c0 2024-02-02T15:33:00.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.550+0000 7f93f8200700 1 -- 172.21.15.1:0/4163217087 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93f01a5ec0 con 0x7f93f010d5c0 2024-02-02T15:33:00.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.555+0000 7f93ee000700 1 -- 172.21.15.1:0/4163217087 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 27) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f93e8003a00 con 0x7f93f010d5c0 2024-02-02T15:33:00.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.556+0000 7f940be00b80 1 Processor -- start 2024-02-02T15:33:00.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.556+0000 7f940be00b80 1 -- start start 2024-02-02T15:33:00.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.557+0000 7f940be00b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56257f1ca010 0x56257f1ca3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:00.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.557+0000 7f940be00b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56257f1ca930 0x56257f1d52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:00.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.557+0000 7f940be00b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56257f1d57e0 0x56257f1d7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:00.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.557+0000 7f940be00b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56257f13e4c0 con 0x56257f1ca010 2024-02-02T15:33:00.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.557+0000 7f940be00b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56257f122aa0 con 0x56257f1d57e0 2024-02-02T15:33:00.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.557+0000 7f940be00b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56257f120460 con 0x56257f1ca930 2024-02-02T15:33:00.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.557+0000 7f93f5800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56257f1ca930 0x56257f1d52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:00.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.557+0000 7f93ee000700 1 --2- 172.21.15.1:0/4163217087 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f93d00a0690 0x7f93d00a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:00.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.557+0000 7f93f5800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56257f1ca930 0x56257f1d52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:39734/0 (socket says 172.21.15.1:39734) 2024-02-02T15:33:00.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.557+0000 7f93f5800700 1 -- 172.21.15.1:0/1006701573 learned_addr learned my addr 172.21.15.1:0/1006701573 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:33:00.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.557+0000 7f93f4e00700 1 --2- 172.21.15.1:0/1006701573 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56257f1ca010 0x56257f1ca3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:00.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.557+0000 7f93f4e00700 1 --2- 172.21.15.1:0/1006701573 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56257f1d57e0 0x56257f1d7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:00.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.557+0000 7f93ee000700 1 -- 172.21.15.1:0/4163217087 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f93e80cbba0 con 0x7f93f010d5c0 2024-02-02T15:33:00.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.557+0000 7f93f5800700 1 -- 172.21.15.1:0/1006701573 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56257f1d57e0 msgr2=0x56257f1d7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:00.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.557+0000 7f93f5800700 1 --2- 172.21.15.1:0/1006701573 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56257f1d57e0 0x56257f1d7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.557+0000 7f93f5800700 1 -- 172.21.15.1:0/1006701573 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56257f1ca010 msgr2=0x56257f1ca3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:00.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.557+0000 7f93f5800700 1 --2- 172.21.15.1:0/1006701573 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56257f1ca010 0x56257f1ca3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.557+0000 7f93f6200700 1 --2- 172.21.15.1:0/4163217087 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f93d00a0690 0x7f93d00a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:00.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.557+0000 7f93f5800700 1 -- 172.21.15.1:0/1006701573 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56257ef9d3b0 con 0x56257f1ca930 2024-02-02T15:33:00.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.558+0000 7f93f5800700 1 --2- 172.21.15.1:0/1006701573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56257f1ca930 0x56257f1d52a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f93e00098f0 tx=0x7f93e0005dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6132a5c103aae42 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:33:00.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.558+0000 7f93f6200700 1 --2- 172.21.15.1:0/4163217087 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f93d00a0690 0x7f93d00a2b50 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f93e800ef00 tx=0x7f93e80092e0 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:33:00.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.558+0000 7f93e6a00700 1 -- 172.21.15.1:0/1006701573 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e000f720 con 0x56257f1ca930 2024-02-02T15:33:00.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.558+0000 7f93e6a00700 1 -- 172.21.15.1:0/1006701573 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93e001d040 con 0x56257f1ca930 2024-02-02T15:33:00.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.558+0000 7f93e6a00700 1 -- 172.21.15.1:0/1006701573 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e00055e0 con 0x56257f1ca930 2024-02-02T15:33:00.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.559+0000 7f940be00b80 1 -- 172.21.15.1:0/1006701573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56257f1ca930 msgr2=0x56257f1d52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:00.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.559+0000 7f940be00b80 1 --2- 172.21.15.1:0/1006701573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56257f1ca930 0x56257f1d52a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f93e00098f0 tx=0x7f93e0005dc0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.559+0000 7f940be00b80 1 -- 172.21.15.1:0/1006701573 shutdown_connections 2024-02-02T15:33:00.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.559+0000 7f940be00b80 1 --2- 172.21.15.1:0/1006701573 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56257f1d57e0 0x56257f1d7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.559+0000 7f940be00b80 1 --2- 172.21.15.1:0/1006701573 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56257f1ca010 0x56257f1ca3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.559+0000 7f940be00b80 1 --2- 172.21.15.1:0/1006701573 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56257f1ca930 0x56257f1d52a0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.559+0000 7f940be00b80 1 -- 172.21.15.1:0/1006701573 >> 172.21.15.1:0/1006701573 conn(0x56257f129db0 msgr2=0x56257f1c63b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:00.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.559+0000 7f940be00b80 1 -- 172.21.15.1:0/1006701573 shutdown_connections 2024-02-02T15:33:00.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.559+0000 7f940be00b80 1 -- 172.21.15.1:0/1006701573 wait complete. 2024-02-02T15:33:00.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.559+0000 7f940be00b80 1 -- start start 2024-02-02T15:33:00.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.560+0000 7f940be00b80 1 Processor -- start 2024-02-02T15:33:00.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.560+0000 7f940be00b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56257f1ca010 0x56257f1d50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:00.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.560+0000 7f940be00b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56257f1ca930 0x56257f1d23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:00.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.560+0000 7f940be00b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56257f1d57e0 0x56257f1d2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:00.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.560+0000 7f940be00b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56257f11bd50 con 0x56257f1ca010 2024-02-02T15:33:00.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.560+0000 7f940be00b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56257f0b7d80 con 0x56257f1ca930 2024-02-02T15:33:00.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.560+0000 7f940be00b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56257f0b7a20 con 0x56257f1d57e0 2024-02-02T15:33:00.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.560+0000 7f940be00b80 10 client.0 _ll_register_callbacks cb 0x56257f0b9290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:33:00.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.560+0000 7f93f5800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56257f1ca930 0x56257f1d23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:00.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.560+0000 7f93f4e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56257f1ca010 0x56257f1d50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:00.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.560+0000 7f93f5800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56257f1ca930 0x56257f1d23d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:33842/0 (socket says 172.21.15.1:33842) 2024-02-02T15:33:00.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.560+0000 7f93f5800700 1 -- 172.21.15.1:0/1380585360 learned_addr learned my addr 172.21.15.1:0/1380585360 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:33:00.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.561+0000 7f93f4e00700 1 --2- 172.21.15.1:0/1380585360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56257f1d57e0 0x56257f1d2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:00.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.561+0000 7f93f5800700 1 -- 172.21.15.1:0/1380585360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56257f1d57e0 msgr2=0x56257f1d2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:00.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.561+0000 7f93f5800700 1 --2- 172.21.15.1:0/1380585360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56257f1d57e0 0x56257f1d2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.561+0000 7f93f5800700 1 -- 172.21.15.1:0/1380585360 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56257f1ca010 msgr2=0x56257f1d50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:00.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.561+0000 7f93f5800700 1 --2- 172.21.15.1:0/1380585360 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56257f1ca010 0x56257f1d50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.561+0000 7f93f5800700 1 -- 172.21.15.1:0/1380585360 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93f01106f0 con 0x56257f1ca930 2024-02-02T15:33:00.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.561+0000 7f93f5800700 1 --2- 172.21.15.1:0/1380585360 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56257f1ca930 0x56257f1d23d0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f93e0009fd0 tx=0x7f93e0009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56b4827eeef6153f server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:33:00.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.561+0000 7f93df400700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:33:00.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.561+0000 7f93df400700 1 -- 172.21.15.1:0/1380585360 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e0005420 con 0x56257f1ca930 2024-02-02T15:33:00.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.561+0000 7f93df400700 1 -- 172.21.15.1:0/1380585360 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f93e0005580 con 0x56257f1ca930 2024-02-02T15:33:00.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.561+0000 7f940be00b80 1 -- 172.21.15.1:0/1380585360 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56257f1be0f0 con 0x56257f1ca930 2024-02-02T15:33:00.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.562+0000 7f93df400700 1 -- 172.21.15.1:0/1380585360 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f93e000b990 con 0x56257f1ca930 2024-02-02T15:33:00.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.562+0000 7f940be00b80 10 client.27242 fetch_fsmap learned FSMap version 38 2024-02-02T15:33:00.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.562+0000 7f940be00b80 1 -- 172.21.15.1:0/1380585360 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x56257ef49b60 con 0x56257f1ca930 2024-02-02T15:33:00.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.562+0000 7f93df400700 1 -- 172.21.15.1:0/1380585360 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56257f1be0f0 con 0x56257f1ca930 2024-02-02T15:33:00.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.562+0000 7f93df400700 1 -- 172.21.15.1:0/1380585360 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 38) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f93e0015020 con 0x56257f1ca930 2024-02-02T15:33:00.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.562+0000 7f940be00b80 10 client.27242 fetch_fsmap finished waiting for FSMap version 38 2024-02-02T15:33:00.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.562+0000 7f940be00b80 10 client.27242 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-02T15:33:00.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.562+0000 7f940be00b80 20 client.27242 populate_metadata read hostname 'smithi001' 2024-02-02T15:33:00.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.562+0000 7f940be00b80 1 --2- 172.21.15.1:0/1380585360 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x56257f1e4c70 0x56257f1e7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:00.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.562+0000 7f940be00b80 4 client.27242 mds_command: new command op to 24529 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:33:00.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.562+0000 7f940be00b80 1 -- 172.21.15.1:0/1380585360 --> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56257f114dd0 con 0x56257f1e4c70 2024-02-02T15:33:00.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.563+0000 7f93f4e00700 1 --2- 172.21.15.1:0/1380585360 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x56257f1e4c70 0x56257f1e7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:00.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.563+0000 7f93f4e00700 1 --2- 172.21.15.1:0/1380585360 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x56257f1e4c70 0x56257f1e7090 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=17228bc3dd820c0a server_cookie=fa109772464f313d in_seq=0 out_seq=0 2024-02-02T15:33:00.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.563+0000 7f93df400700 10 client.27242 ms_handle_connect on v2:172.21.15.142:6838/321675421 2024-02-02T15:33:00.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.564+0000 7f93df400700 1 -- 172.21.15.1:0/1380585360 <== mds.1 v2:172.21.15.142:6838/321675421 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x56257f114dd0 con 0x56257f1e4c70 2024-02-02T15:33:00.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.564+0000 7f93df400700 10 client.27242 handle_command_reply: tid=0 2024-02-02T15:33:00.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.564+0000 7f940be00b80 1 client.27242 shutdown 2024-02-02T15:33:00.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.564+0000 7f940be00b80 1 -- 172.21.15.1:0/1380585360 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56257f1ca930 msgr2=0x56257f1d23d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:00.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.564+0000 7f940be00b80 1 --2- 172.21.15.1:0/1380585360 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56257f1ca930 0x56257f1d23d0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f93e0009fd0 tx=0x7f93e0009c00 comp rx=0 tx=0).stop 2024-02-02T15:33:00.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.564+0000 7f940be00b80 1 -- 172.21.15.1:0/1380585360 shutdown_connections 2024-02-02T15:33:00.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.564+0000 7f940be00b80 1 --2- 172.21.15.1:0/1380585360 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x56257f1e4c70 0x56257f1e7090 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.564+0000 7f940be00b80 1 --2- 172.21.15.1:0/1380585360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56257f1d57e0 0x56257f1d2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.564+0000 7f940be00b80 1 --2- 172.21.15.1:0/1380585360 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56257f1ca010 0x56257f1d50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.564+0000 7f940be00b80 1 --2- 172.21.15.1:0/1380585360 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56257f1ca930 0x56257f1d23d0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.564+0000 7f93df400700 0 client.27242 ms_handle_reset on v2:172.21.15.142:6838/321675421 2024-02-02T15:33:00.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.564+0000 7f940be00b80 1 -- 172.21.15.1:0/1380585360 >> 172.21.15.1:0/1380585360 conn(0x56257f129db0 msgr2=0x56257f1c5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:00.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.564+0000 7f940be00b80 1 -- 172.21.15.1:0/1380585360 shutdown_connections 2024-02-02T15:33:00.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.565+0000 7f940be00b80 1 -- 172.21.15.1:0/1380585360 wait complete. 2024-02-02T15:33:00.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.565+0000 7f940be00b80 20 client.27242 trim_cache size 0 max 16384 2024-02-02T15:33:00.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.579+0000 7f940be00b80 1 Processor -- start 2024-02-02T15:33:00.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.580+0000 7f940be00b80 1 -- start start 2024-02-02T15:33:00.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.580+0000 7f940be00b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56257f1ca010 0x56257f1dc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:00.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.580+0000 7f940be00b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56257f1ca930 0x56257f1d1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:00.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.580+0000 7f940be00b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56257f1d57e0 0x56257f1e08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:00.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.580+0000 7f940be00b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56257ef9f6a0 con 0x56257f1d57e0 2024-02-02T15:33:00.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.580+0000 7f940be00b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56257f1c0530 con 0x56257f1ca010 2024-02-02T15:33:00.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.580+0000 7f940be00b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56257f1ce270 con 0x56257f1ca930 2024-02-02T15:33:00.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.580+0000 7f93f5800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56257f1ca930 0x56257f1d1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:00.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.580+0000 7f93f5800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56257f1ca930 0x56257f1d1b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:39766/0 (socket says 172.21.15.1:39766) 2024-02-02T15:33:00.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.580+0000 7f93f5800700 1 -- 172.21.15.1:0/439144778 learned_addr learned my addr 172.21.15.1:0/439144778 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:33:00.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.580+0000 7f93f4e00700 1 --2- 172.21.15.1:0/439144778 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56257f1ca010 0x56257f1dc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:00.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.580+0000 7f93f4e00700 1 --2- 172.21.15.1:0/439144778 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56257f1d57e0 0x56257f1e08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:00.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.580+0000 7f93f5800700 1 -- 172.21.15.1:0/439144778 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56257f1ca010 msgr2=0x56257f1dc2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:00.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.580+0000 7f93f5800700 1 --2- 172.21.15.1:0/439144778 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56257f1ca010 0x56257f1dc2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.581+0000 7f93f5800700 1 -- 172.21.15.1:0/439144778 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56257f1d57e0 msgr2=0x56257f1e08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:00.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.581+0000 7f93f5800700 1 --2- 172.21.15.1:0/439144778 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56257f1d57e0 0x56257f1e08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.581+0000 7f93f5800700 1 -- 172.21.15.1:0/439144778 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56257f0e1ea0 con 0x56257f1ca930 2024-02-02T15:33:00.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.581+0000 7f93f5800700 1 --2- 172.21.15.1:0/439144778 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56257f1ca930 0x56257f1d1b90 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f93e0005150 tx=0x7f93e000b340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=adf1725dd553e8c9 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:33:00.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.581+0000 7f93e7400700 1 -- 172.21.15.1:0/439144778 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e002adb0 con 0x56257f1ca930 2024-02-02T15:33:00.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.581+0000 7f93e7400700 1 -- 172.21.15.1:0/439144778 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93e0005920 con 0x56257f1ca930 2024-02-02T15:33:00.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.581+0000 7f93e7400700 1 -- 172.21.15.1:0/439144778 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e00056c0 con 0x56257f1ca930 2024-02-02T15:33:00.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.582+0000 7f940be00b80 1 -- 172.21.15.1:0/439144778 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56257f1ca930 msgr2=0x56257f1d1b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:00.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.582+0000 7f940be00b80 1 --2- 172.21.15.1:0/439144778 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56257f1ca930 0x56257f1d1b90 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f93e0005150 tx=0x7f93e000b340 comp rx=0 tx=0).stop 2024-02-02T15:33:00.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.582+0000 7f940be00b80 1 -- 172.21.15.1:0/439144778 shutdown_connections 2024-02-02T15:33:00.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.582+0000 7f940be00b80 1 --2- 172.21.15.1:0/439144778 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56257f1d57e0 0x56257f1e08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.582+0000 7f940be00b80 1 --2- 172.21.15.1:0/439144778 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56257f1ca010 0x56257f1dc2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.582+0000 7f940be00b80 1 --2- 172.21.15.1:0/439144778 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56257f1ca930 0x56257f1d1b90 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.582+0000 7f940be00b80 1 -- 172.21.15.1:0/439144778 >> 172.21.15.1:0/439144778 conn(0x56257f129db0 msgr2=0x56257f1cd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:00.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.582+0000 7f940be00b80 1 -- 172.21.15.1:0/439144778 shutdown_connections 2024-02-02T15:33:00.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.582+0000 7f940be00b80 1 -- 172.21.15.1:0/439144778 wait complete. 2024-02-02T15:33:00.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.583+0000 7f940be00b80 1 -- start start 2024-02-02T15:33:00.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.583+0000 7f940be00b80 1 Processor -- start 2024-02-02T15:33:00.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.583+0000 7f940be00b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56257f1ca010 0x56257f1d82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:00.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.583+0000 7f940be00b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56257f1ca930 0x56257f1d87e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:00.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.583+0000 7f940be00b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56257f1d57e0 0x56257f235780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:00.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.583+0000 7f940be00b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56257f1d90c0 con 0x56257f1ca010 2024-02-02T15:33:00.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.583+0000 7f940be00b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56257f235cc0 con 0x56257f1d57e0 2024-02-02T15:33:00.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.583+0000 7f940be00b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56257f235e00 con 0x56257f1ca930 2024-02-02T15:33:00.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.584+0000 7f940be00b80 10 client.0 _ll_register_callbacks cb 0x56257f0b9290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:33:00.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.584+0000 7f93f5800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56257f1ca930 0x56257f1d87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:00.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.584+0000 7f93f5800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56257f1ca930 0x56257f1d87e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:39776/0 (socket says 172.21.15.1:39776) 2024-02-02T15:33:00.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.584+0000 7f93f5800700 1 -- 172.21.15.1:0/1916590146 learned_addr learned my addr 172.21.15.1:0/1916590146 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:33:00.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.584+0000 7f93f4e00700 1 --2- 172.21.15.1:0/1916590146 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56257f1ca010 0x56257f1d82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:00.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.584+0000 7f93f4e00700 1 --2- 172.21.15.1:0/1916590146 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56257f1d57e0 0x56257f235780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:00.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.584+0000 7f93f5800700 1 -- 172.21.15.1:0/1916590146 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56257f1d57e0 msgr2=0x56257f235780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:00.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.584+0000 7f93f5800700 1 --2- 172.21.15.1:0/1916590146 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56257f1d57e0 0x56257f235780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.584+0000 7f93f5800700 1 -- 172.21.15.1:0/1916590146 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56257f1ca010 msgr2=0x56257f1d82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:00.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.584+0000 7f93f5800700 1 --2- 172.21.15.1:0/1916590146 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56257f1ca010 0x56257f1d82a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.584+0000 7f93f5800700 1 -- 172.21.15.1:0/1916590146 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56257ef49b60 con 0x56257f1ca930 2024-02-02T15:33:00.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.584+0000 7f93f4e00700 1 --2- 172.21.15.1:0/1916590146 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56257f1ca010 0x56257f1d82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:33:00.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.584+0000 7f93f5800700 1 --2- 172.21.15.1:0/1916590146 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56257f1ca930 0x56257f1d87e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f93e002aab0 tx=0x7f93e00050e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65467a1056d9faf server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:33:00.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.584+0000 7f93df400700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:33:00.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.585+0000 7f93df400700 1 -- 172.21.15.1:0/1916590146 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e002ae30 con 0x56257f1ca930 2024-02-02T15:33:00.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.585+0000 7f93df400700 1 -- 172.21.15.1:0/1916590146 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f93e000b770 con 0x56257f1ca930 2024-02-02T15:33:00.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.585+0000 7f940be00b80 1 -- 172.21.15.1:0/1916590146 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f93e001d040 con 0x56257f1ca930 2024-02-02T15:33:00.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.585+0000 7f93df400700 1 -- 172.21.15.1:0/1916590146 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f93e000b930 con 0x56257f1ca930 2024-02-02T15:33:00.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.585+0000 7f940be00b80 10 client.27247 fetch_fsmap learned FSMap version 38 2024-02-02T15:33:00.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.585+0000 7f940be00b80 1 -- 172.21.15.1:0/1916590146 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x56257efa1cd0 con 0x56257f1ca930 2024-02-02T15:33:00.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.585+0000 7f93df400700 1 -- 172.21.15.1:0/1916590146 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f93e001d040 con 0x56257f1ca930 2024-02-02T15:33:00.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.585+0000 7f93df400700 1 -- 172.21.15.1:0/1916590146 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 38) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f93e0015070 con 0x56257f1ca930 2024-02-02T15:33:00.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.585+0000 7f940be00b80 10 client.27247 fetch_fsmap finished waiting for FSMap version 38 2024-02-02T15:33:00.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.585+0000 7f940be00b80 10 client.27247 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-02T15:33:00.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.585+0000 7f940be00b80 20 client.27247 populate_metadata read hostname 'smithi001' 2024-02-02T15:33:00.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.585+0000 7f940be00b80 1 --2- 172.21.15.1:0/1916590146 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x56257f2370a0 0x56257f239490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:00.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.585+0000 7f940be00b80 4 client.27247 mds_command: new command op to 24529 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:33:00.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.585+0000 7f940be00b80 1 -- 172.21.15.1:0/1916590146 --> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56257f0daf50 con 0x56257f2370a0 2024-02-02T15:33:00.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.586+0000 7f93f4e00700 1 --2- 172.21.15.1:0/1916590146 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x56257f2370a0 0x56257f239490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:00.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.586+0000 7f93f4e00700 1 --2- 172.21.15.1:0/1916590146 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x56257f2370a0 0x56257f239490 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=306447ec4dcb84cd server_cookie=e1904bb30a763440 in_seq=0 out_seq=0 2024-02-02T15:33:00.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.586+0000 7f93df400700 10 client.27247 ms_handle_connect on v2:172.21.15.142:6838/321675421 2024-02-02T15:33:00.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.586+0000 7f93df400700 1 -- 172.21.15.1:0/1916590146 <== mds.1 v2:172.21.15.142:6838/321675421 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56257f0daf50 con 0x56257f2370a0 2024-02-02T15:33:00.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.586+0000 7f93df400700 10 client.27247 handle_command_reply: tid=0 2024-02-02T15:33:00.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.586+0000 7f940be00b80 1 client.27247 shutdown 2024-02-02T15:33:00.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.586+0000 7f940be00b80 1 -- 172.21.15.1:0/1916590146 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56257f1ca930 msgr2=0x56257f1d87e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:00.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.586+0000 7f940be00b80 1 --2- 172.21.15.1:0/1916590146 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56257f1ca930 0x56257f1d87e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f93e002aab0 tx=0x7f93e00050e0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.587+0000 7f940be00b80 1 -- 172.21.15.1:0/1916590146 shutdown_connections 2024-02-02T15:33:00.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.587+0000 7f940be00b80 1 --2- 172.21.15.1:0/1916590146 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x56257f2370a0 0x56257f239490 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.587+0000 7f940be00b80 1 --2- 172.21.15.1:0/1916590146 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56257f1d57e0 0x56257f235780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.587+0000 7f940be00b80 1 --2- 172.21.15.1:0/1916590146 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56257f1ca010 0x56257f1d82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.587+0000 7f940be00b80 1 --2- 172.21.15.1:0/1916590146 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56257f1ca930 0x56257f1d87e0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.587+0000 7f93df400700 0 client.27247 ms_handle_reset on v2:172.21.15.142:6838/321675421 2024-02-02T15:33:00.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.587+0000 7f940be00b80 1 -- 172.21.15.1:0/1916590146 >> 172.21.15.1:0/1916590146 conn(0x56257f129db0 msgr2=0x56257f1e08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:00.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.587+0000 7f940be00b80 1 -- 172.21.15.1:0/1916590146 shutdown_connections 2024-02-02T15:33:00.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.587+0000 7f940be00b80 1 -- 172.21.15.1:0/1916590146 wait complete. 2024-02-02T15:33:00.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.587+0000 7f940be00b80 20 client.27247 trim_cache size 0 max 16384 2024-02-02T15:33:00.592 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:33:00.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.587+0000 7f93f8200700 1 -- 172.21.15.1:0/4163217087 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f93d00a0690 msgr2=0x7f93d00a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:00.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.587+0000 7f93f8200700 1 --2- 172.21.15.1:0/4163217087 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f93d00a0690 0x7f93d00a2b50 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f93e800ef00 tx=0x7f93e80092e0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.587+0000 7f93f8200700 1 -- 172.21.15.1:0/4163217087 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93f010d5c0 msgr2=0x7f93f019dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:00.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.587+0000 7f93f8200700 1 --2- 172.21.15.1:0/4163217087 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93f010d5c0 0x7f93f019dc10 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f93e800ef30 tx=0x7f93e800c3a0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.588+0000 7f93f8200700 1 -- 172.21.15.1:0/4163217087 shutdown_connections 2024-02-02T15:33:00.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.588+0000 7f93f8200700 1 --2- 172.21.15.1:0/4163217087 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f93d00a0690 0x7f93d00a2b50 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.588+0000 7f93f8200700 1 --2- 172.21.15.1:0/4163217087 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f93f010d5c0 0x7f93f019dc10 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.588+0000 7f93f8200700 1 --2- 172.21.15.1:0/4163217087 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f93f0073170 0x7f93f01992e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.588+0000 7f93f8200700 1 --2- 172.21.15.1:0/4163217087 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f93f0073a90 0x7f93f0199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.588+0000 7f93f8200700 1 -- 172.21.15.1:0/4163217087 >> 172.21.15.1:0/4163217087 conn(0x7f93f00fc810 msgr2=0x7f93f010f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:00.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.588+0000 7f93f8200700 1 -- 172.21.15.1:0/4163217087 shutdown_connections 2024-02-02T15:33:00.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.588+0000 7f93f8200700 1 -- 172.21.15.1:0/4163217087 wait complete. 2024-02-02T15:33:00.604 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:00 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/1170264682' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:33:00.607 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-02T15:33:00.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.815+0000 7f6615000700 1 -- 172.21.15.1:0/2606159427 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f66101019a0 msgr2=0x7f66101059e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:00.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.815+0000 7f6615000700 1 --2- 172.21.15.1:0/2606159427 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f66101019a0 0x7f66101059e0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f65f8009990 tx=0x7f65f8009ca0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.815+0000 7f6615000700 1 -- 172.21.15.1:0/2606159427 shutdown_connections 2024-02-02T15:33:00.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.815+0000 7f6615000700 1 --2- 172.21.15.1:0/2606159427 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6610106110 0x7f661010dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.815+0000 7f6615000700 1 --2- 172.21.15.1:0/2606159427 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6610100ff0 0x7f66101013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.815+0000 7f6615000700 1 --2- 172.21.15.1:0/2606159427 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f66101019a0 0x7f66101059e0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.815+0000 7f6615000700 1 -- 172.21.15.1:0/2606159427 >> 172.21.15.1:0/2606159427 conn(0x7f66100fc810 msgr2=0x7f66100fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:00.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.815+0000 7f6615000700 1 -- 172.21.15.1:0/2606159427 shutdown_connections 2024-02-02T15:33:00.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.815+0000 7f6615000700 1 -- 172.21.15.1:0/2606159427 wait complete. 2024-02-02T15:33:00.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.816+0000 7f6615000700 1 Processor -- start 2024-02-02T15:33:00.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.816+0000 7f6615000700 1 -- start start 2024-02-02T15:33:00.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.816+0000 7f6615000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6610100ff0 0x7f6610199030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:00.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.816+0000 7f6615000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6610106110 0x7f6610199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:00.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.816+0000 7f6615000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f661019d9d0 0x7f661019de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:00.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.816+0000 7f6615000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f661019e450 con 0x7f661019d9d0 2024-02-02T15:33:00.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.816+0000 7f6615000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f661019e5c0 con 0x7f6610106110 2024-02-02T15:33:00.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.816+0000 7f6615000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f661019e780 con 0x7f6610100ff0 2024-02-02T15:33:00.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.817+0000 7f660de00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6610100ff0 0x7f6610199030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:00.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.817+0000 7f660d400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6610106110 0x7f6610199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:00.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.817+0000 7f660de00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6610100ff0 0x7f6610199030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:39808/0 (socket says 172.21.15.1:39808) 2024-02-02T15:33:00.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.817+0000 7f660de00700 1 -- 172.21.15.1:0/4238928624 learned_addr learned my addr 172.21.15.1:0/4238928624 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:33:00.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.817+0000 7f660e800700 1 --2- 172.21.15.1:0/4238928624 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f661019d9d0 0x7f661019de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:00.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.817+0000 7f660de00700 1 -- 172.21.15.1:0/4238928624 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6610106110 msgr2=0x7f6610199570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:00.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.817+0000 7f660de00700 1 --2- 172.21.15.1:0/4238928624 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6610106110 0x7f6610199570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.817+0000 7f660de00700 1 -- 172.21.15.1:0/4238928624 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f661019d9d0 msgr2=0x7f661019de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:00.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.817+0000 7f660de00700 1 --2- 172.21.15.1:0/4238928624 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f661019d9d0 0x7f661019de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.817+0000 7f660de00700 1 -- 172.21.15.1:0/4238928624 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65f80095f0 con 0x7f6610100ff0 2024-02-02T15:33:00.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.817+0000 7f660d400700 1 --2- 172.21.15.1:0/4238928624 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6610106110 0x7f6610199570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:33:00.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.817+0000 7f660de00700 1 --2- 172.21.15.1:0/4238928624 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6610100ff0 0x7f6610199030 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f660000e9a0 tx=0x7f660000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:33:00.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.818+0000 7f6606a00700 1 -- 172.21.15.1:0/4238928624 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6600009790 con 0x7f6610100ff0 2024-02-02T15:33:00.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.818+0000 7f6606a00700 1 -- 172.21.15.1:0/4238928624 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6600010430 con 0x7f6610100ff0 2024-02-02T15:33:00.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.818+0000 7f6615000700 1 -- 172.21.15.1:0/4238928624 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66101a5ce0 con 0x7f6610100ff0 2024-02-02T15:33:00.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.818+0000 7f6606a00700 1 -- 172.21.15.1:0/4238928624 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6600018680 con 0x7f6610100ff0 2024-02-02T15:33:00.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.818+0000 7f6615000700 1 -- 172.21.15.1:0/4238928624 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66101a61c0 con 0x7f6610100ff0 2024-02-02T15:33:00.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.821+0000 7f6606a00700 1 -- 172.21.15.1:0/4238928624 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 27) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f66000105a0 con 0x7f6610100ff0 2024-02-02T15:33:00.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.823+0000 7f6628a7db80 1 Processor -- start 2024-02-02T15:33:00.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.823+0000 7f6606a00700 1 --2- 172.21.15.1:0/4238928624 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f65f00a0690 0x7f65f00a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:00.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.823+0000 7f6606a00700 1 -- 172.21.15.1:0/4238928624 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f6600014070 con 0x7f6610100ff0 2024-02-02T15:33:00.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.823+0000 7f6628a7db80 1 -- start start 2024-02-02T15:33:00.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.823+0000 7f660d400700 1 --2- 172.21.15.1:0/4238928624 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f65f00a0690 0x7f65f00a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:00.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.824+0000 7f6628a7db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56042c631010 0x56042c6313f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:00.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.824+0000 7f6628a7db80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56042c631930 0x56042c63c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:00.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.824+0000 7f6628a7db80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56042c63c7e0 0x56042c63ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:00.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.824+0000 7f6628a7db80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56042c5a54c0 con 0x56042c631930 2024-02-02T15:33:00.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.824+0000 7f6628a7db80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56042c589aa0 con 0x56042c63c7e0 2024-02-02T15:33:00.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.824+0000 7f6628a7db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56042c587460 con 0x56042c631010 2024-02-02T15:33:00.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.824+0000 7f660de00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56042c631010 0x56042c6313f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:00.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.824+0000 7f660de00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56042c631010 0x56042c6313f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:39818/0 (socket says 172.21.15.1:39818) 2024-02-02T15:33:00.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.824+0000 7f660de00700 1 -- 172.21.15.1:0/4294894561 learned_addr learned my addr 172.21.15.1:0/4294894561 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:33:00.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.824+0000 7f660e800700 1 --2- 172.21.15.1:0/4294894561 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56042c63c7e0 0x56042c63ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:00.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.824+0000 7f660d400700 1 --2- 172.21.15.1:0/4294894561 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56042c631930 0x56042c63c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:00.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.825+0000 7f660de00700 1 -- 172.21.15.1:0/4294894561 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56042c63c7e0 msgr2=0x56042c63ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:00.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.825+0000 7f660de00700 1 --2- 172.21.15.1:0/4294894561 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56042c63c7e0 0x56042c63ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.825+0000 7f660de00700 1 -- 172.21.15.1:0/4294894561 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56042c631930 msgr2=0x56042c63c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:00.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.825+0000 7f660de00700 1 --2- 172.21.15.1:0/4294894561 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56042c631930 0x56042c63c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.825+0000 7f660de00700 1 -- 172.21.15.1:0/4294894561 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56042c4043b0 con 0x56042c631010 2024-02-02T15:33:00.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.825+0000 7f660d400700 1 --2- 172.21.15.1:0/4238928624 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f65f00a0690 0x7f65f00a2b50 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f65f800b2e0 tx=0x7f65f8004580 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:33:00.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.825+0000 7f660de00700 1 --2- 172.21.15.1:0/4294894561 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56042c631010 0x56042c6313f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f66000993e0 tx=0x7f6600098e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c2b154170556a73 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:33:00.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.825+0000 7f65ff400700 1 -- 172.21.15.1:0/4294894561 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6600099450 con 0x56042c631010 2024-02-02T15:33:00.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.825+0000 7f65ff400700 1 -- 172.21.15.1:0/4294894561 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f660009d280 con 0x56042c631010 2024-02-02T15:33:00.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.825+0000 7f65ff400700 1 -- 172.21.15.1:0/4294894561 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66000998f0 con 0x56042c631010 2024-02-02T15:33:00.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.826+0000 7f6628a7db80 1 -- 172.21.15.1:0/4294894561 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56042c631010 msgr2=0x56042c6313f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:00.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.826+0000 7f6628a7db80 1 --2- 172.21.15.1:0/4294894561 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56042c631010 0x56042c6313f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f66000993e0 tx=0x7f6600098e30 comp rx=0 tx=0).stop 2024-02-02T15:33:00.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.826+0000 7f6628a7db80 1 -- 172.21.15.1:0/4294894561 shutdown_connections 2024-02-02T15:33:00.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.826+0000 7f6628a7db80 1 --2- 172.21.15.1:0/4294894561 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56042c63c7e0 0x56042c63ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.826+0000 7f6628a7db80 1 --2- 172.21.15.1:0/4294894561 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56042c631010 0x56042c6313f0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.826+0000 7f6628a7db80 1 --2- 172.21.15.1:0/4294894561 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56042c631930 0x56042c63c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.826+0000 7f6628a7db80 1 -- 172.21.15.1:0/4294894561 >> 172.21.15.1:0/4294894561 conn(0x56042c590db0 msgr2=0x56042c62d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:00.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.826+0000 7f6628a7db80 1 -- 172.21.15.1:0/4294894561 shutdown_connections 2024-02-02T15:33:00.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.826+0000 7f6628a7db80 1 -- 172.21.15.1:0/4294894561 wait complete. 2024-02-02T15:33:00.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.826+0000 7f6628a7db80 1 -- start start 2024-02-02T15:33:00.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.827+0000 7f6628a7db80 1 Processor -- start 2024-02-02T15:33:00.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.827+0000 7f6628a7db80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56042c631010 0x56042c63c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:00.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.827+0000 7f6628a7db80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56042c631930 0x56042c6394c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:00.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.827+0000 7f6628a7db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56042c63c7e0 0x56042c639c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:00.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.827+0000 7f6628a7db80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56042c582d50 con 0x56042c631010 2024-02-02T15:33:00.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.827+0000 7f6628a7db80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56042c51ed80 con 0x56042c631930 2024-02-02T15:33:00.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.827+0000 7f6628a7db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56042c51ea20 con 0x56042c63c7e0 2024-02-02T15:33:00.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.827+0000 7f6628a7db80 10 client.0 _ll_register_callbacks cb 0x56042c520290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:33:00.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.827+0000 7f660e800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56042c63c7e0 0x56042c639c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:00.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.827+0000 7f660d400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56042c631930 0x56042c6394c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:00.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.827+0000 7f660e800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56042c63c7e0 0x56042c639c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:39824/0 (socket says 172.21.15.1:39824) 2024-02-02T15:33:00.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.827+0000 7f660e800700 1 -- 172.21.15.1:0/1054064266 learned_addr learned my addr 172.21.15.1:0/1054064266 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:33:00.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.827+0000 7f660de00700 1 --2- 172.21.15.1:0/1054064266 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56042c631010 0x56042c63c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:00.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.828+0000 7f660d400700 1 -- 172.21.15.1:0/1054064266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56042c63c7e0 msgr2=0x56042c639c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:00.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.828+0000 7f660d400700 1 --2- 172.21.15.1:0/1054064266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56042c63c7e0 0x56042c639c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.828+0000 7f660d400700 1 -- 172.21.15.1:0/1054064266 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56042c631010 msgr2=0x56042c63c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:00.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.828+0000 7f660d400700 1 --2- 172.21.15.1:0/1054064266 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56042c631010 0x56042c63c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.828+0000 7f660d400700 1 -- 172.21.15.1:0/1054064266 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66101a61c0 con 0x56042c631930 2024-02-02T15:33:00.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.828+0000 7f660d400700 1 --2- 172.21.15.1:0/1054064266 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56042c631930 0x56042c6394c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f65f80059d0 tx=0x7f65f8009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5bd1ee3677962409 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:33:00.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.828+0000 7f65f7e00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:33:00.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.828+0000 7f65f7e00700 1 -- 172.21.15.1:0/1054064266 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f8027610 con 0x56042c631930 2024-02-02T15:33:00.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.828+0000 7f6628a7db80 1 -- 172.21.15.1:0/1054064266 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56042c6250f0 con 0x56042c631930 2024-02-02T15:33:00.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.828+0000 7f65f7e00700 1 -- 172.21.15.1:0/1054064266 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f65f802c040 con 0x56042c631930 2024-02-02T15:33:00.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.829+0000 7f65f7e00700 1 -- 172.21.15.1:0/1054064266 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f65f8027770 con 0x56042c631930 2024-02-02T15:33:00.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.829+0000 7f6628a7db80 10 client.27260 fetch_fsmap learned FSMap version 38 2024-02-02T15:33:00.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.829+0000 7f6628a7db80 1 -- 172.21.15.1:0/1054064266 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x56042c3b0b60 con 0x56042c631930 2024-02-02T15:33:00.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.829+0000 7f65f7e00700 1 -- 172.21.15.1:0/1054064266 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56042c6250f0 con 0x56042c631930 2024-02-02T15:33:00.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.829+0000 7f65f7e00700 1 -- 172.21.15.1:0/1054064266 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 38) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f65f8030940 con 0x56042c631930 2024-02-02T15:33:00.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.829+0000 7f6628a7db80 10 client.27260 fetch_fsmap finished waiting for FSMap version 38 2024-02-02T15:33:00.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.829+0000 7f6628a7db80 10 client.27260 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:33:00.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.829+0000 7f6628a7db80 20 client.27260 populate_metadata read hostname 'smithi001' 2024-02-02T15:33:00.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.829+0000 7f6628a7db80 1 --2- 172.21.15.1:0/1054064266 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56042c64c190 0x56042c64e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:00.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.829+0000 7f6628a7db80 4 client.27260 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:33:00.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.829+0000 7f6628a7db80 1 -- 172.21.15.1:0/1054064266 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56042c57bdd0 con 0x56042c64c190 2024-02-02T15:33:00.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.829+0000 7f660de00700 1 --2- 172.21.15.1:0/1054064266 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56042c64c190 0x56042c64e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:00.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.830+0000 7f660de00700 1 --2- 172.21.15.1:0/1054064266 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56042c64c190 0x56042c64e5b0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bad38a84c10ae646 server_cookie=8019017b52fead7e in_seq=0 out_seq=0 2024-02-02T15:33:00.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.830+0000 7f65f7e00700 10 client.27260 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:33:00.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.831+0000 7f65f7e00700 1 -- 172.21.15.1:0/1054064266 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x56042c57bdd0 con 0x56042c64c190 2024-02-02T15:33:00.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.831+0000 7f65f7e00700 10 client.27260 handle_command_reply: tid=0 2024-02-02T15:33:00.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.831+0000 7f6628a7db80 1 client.27260 shutdown 2024-02-02T15:33:00.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.831+0000 7f6628a7db80 1 -- 172.21.15.1:0/1054064266 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56042c631930 msgr2=0x56042c6394c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:00.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.831+0000 7f6628a7db80 1 --2- 172.21.15.1:0/1054064266 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56042c631930 0x56042c6394c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f65f80059d0 tx=0x7f65f8009ca0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.831+0000 7f6628a7db80 1 -- 172.21.15.1:0/1054064266 shutdown_connections 2024-02-02T15:33:00.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.831+0000 7f6628a7db80 1 --2- 172.21.15.1:0/1054064266 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56042c64c190 0x56042c64e5b0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.831+0000 7f6628a7db80 1 --2- 172.21.15.1:0/1054064266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56042c63c7e0 0x56042c639c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.831+0000 7f6628a7db80 1 --2- 172.21.15.1:0/1054064266 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56042c631010 0x56042c63c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.831+0000 7f65f7e00700 0 client.27260 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:33:00.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.831+0000 7f6628a7db80 1 --2- 172.21.15.1:0/1054064266 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56042c631930 0x56042c6394c0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.831+0000 7f6628a7db80 1 -- 172.21.15.1:0/1054064266 >> 172.21.15.1:0/1054064266 conn(0x56042c590db0 msgr2=0x56042c62cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:00.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.831+0000 7f6628a7db80 1 -- 172.21.15.1:0/1054064266 shutdown_connections 2024-02-02T15:33:00.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.831+0000 7f6628a7db80 1 -- 172.21.15.1:0/1054064266 wait complete. 2024-02-02T15:33:00.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.831+0000 7f6628a7db80 20 client.27260 trim_cache size 0 max 16384 2024-02-02T15:33:00.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.845+0000 7f6628a7db80 1 Processor -- start 2024-02-02T15:33:00.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.845+0000 7f6628a7db80 1 -- start start 2024-02-02T15:33:00.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.846+0000 7f6628a7db80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56042c631010 0x56042c643330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:00.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.846+0000 7f6628a7db80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56042c631930 0x56042c633ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:00.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.846+0000 7f6628a7db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56042c63c7e0 0x56042c634000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:00.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.846+0000 7f6628a7db80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56042c4066a0 con 0x56042c631010 2024-02-02T15:33:00.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.846+0000 7f6628a7db80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56042c632830 con 0x56042c631930 2024-02-02T15:33:00.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.846+0000 7f6628a7db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56042c624cf0 con 0x56042c63c7e0 2024-02-02T15:33:00.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.846+0000 7f660e800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56042c63c7e0 0x56042c634000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:00.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.846+0000 7f660d400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56042c631930 0x56042c633ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:00.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.846+0000 7f660de00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56042c631010 0x56042c643330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:00.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.846+0000 7f660e800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56042c63c7e0 0x56042c634000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:39836/0 (socket says 172.21.15.1:39836) 2024-02-02T15:33:00.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.846+0000 7f660e800700 1 -- 172.21.15.1:0/1748031436 learned_addr learned my addr 172.21.15.1:0/1748031436 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:33:00.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.846+0000 7f660d400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56042c631930 0x56042c633ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:33904/0 (socket says 172.21.15.1:33904) 2024-02-02T15:33:00.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.846+0000 7f660e800700 1 -- 172.21.15.1:0/1748031436 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56042c631930 msgr2=0x56042c633ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:00.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.846+0000 7f660e800700 1 --2- 172.21.15.1:0/1748031436 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56042c631930 0x56042c633ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.846+0000 7f660e800700 1 -- 172.21.15.1:0/1748031436 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56042c631010 msgr2=0x56042c643330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:00.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.846+0000 7f660e800700 1 --2- 172.21.15.1:0/1748031436 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56042c631010 0x56042c643330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.846+0000 7f660e800700 1 -- 172.21.15.1:0/1748031436 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56042c548ea0 con 0x56042c63c7e0 2024-02-02T15:33:00.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.846+0000 7f660d400700 1 --2- 172.21.15.1:0/1748031436 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56042c631930 0x56042c633ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:33:00.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.846+0000 7f660de00700 1 --2- 172.21.15.1:0/1748031436 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56042c631010 0x56042c643330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:33:00.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.847+0000 7f660e800700 1 --2- 172.21.15.1:0/1748031436 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56042c63c7e0 0x56042c634000 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f66080119e0 tx=0x7f6608009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60ceae873f0f3391 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:33:00.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.847+0000 7f65ffe00700 1 -- 172.21.15.1:0/1748031436 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f660800e040 con 0x56042c63c7e0 2024-02-02T15:33:00.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.847+0000 7f65ffe00700 1 -- 172.21.15.1:0/1748031436 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f660800f040 con 0x56042c63c7e0 2024-02-02T15:33:00.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.847+0000 7f65ffe00700 1 -- 172.21.15.1:0/1748031436 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6608017ca0 con 0x56042c63c7e0 2024-02-02T15:33:00.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.847+0000 7f6628a7db80 1 -- 172.21.15.1:0/1748031436 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56042c63c7e0 msgr2=0x56042c634000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:00.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.847+0000 7f6628a7db80 1 --2- 172.21.15.1:0/1748031436 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56042c63c7e0 0x56042c634000 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f66080119e0 tx=0x7f6608009920 comp rx=0 tx=0).stop 2024-02-02T15:33:00.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.847+0000 7f6628a7db80 1 -- 172.21.15.1:0/1748031436 shutdown_connections 2024-02-02T15:33:00.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.847+0000 7f6628a7db80 1 --2- 172.21.15.1:0/1748031436 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56042c63c7e0 0x56042c634000 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.847+0000 7f6628a7db80 1 --2- 172.21.15.1:0/1748031436 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56042c631010 0x56042c643330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.847+0000 7f6628a7db80 1 --2- 172.21.15.1:0/1748031436 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56042c631930 0x56042c633ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.847+0000 7f6628a7db80 1 -- 172.21.15.1:0/1748031436 >> 172.21.15.1:0/1748031436 conn(0x56042c590db0 msgr2=0x56042c648640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:00.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.848+0000 7f6628a7db80 1 -- 172.21.15.1:0/1748031436 shutdown_connections 2024-02-02T15:33:00.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.848+0000 7f6628a7db80 1 -- 172.21.15.1:0/1748031436 wait complete. 2024-02-02T15:33:00.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.848+0000 7f6628a7db80 1 -- start start 2024-02-02T15:33:00.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.848+0000 7f6628a7db80 1 Processor -- start 2024-02-02T15:33:00.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.849+0000 7f6628a7db80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56042c631010 0x56042c642e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:00.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.849+0000 7f6628a7db80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56042c631930 0x56042c641310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:00.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.849+0000 7f6628a7db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56042c63c7e0 0x56042c641850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:00.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.849+0000 7f6628a7db80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56042c643780 con 0x56042c631010 2024-02-02T15:33:00.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.849+0000 7f6628a7db80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56042c641d90 con 0x56042c631930 2024-02-02T15:33:00.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.849+0000 7f6628a7db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56042c641f00 con 0x56042c63c7e0 2024-02-02T15:33:00.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.849+0000 7f6628a7db80 10 client.0 _ll_register_callbacks cb 0x56042c520290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:33:00.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.849+0000 7f660d400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56042c631930 0x56042c641310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:00.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.849+0000 7f660e800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56042c63c7e0 0x56042c641850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:00.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.849+0000 7f660d400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56042c631930 0x56042c641310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:33912/0 (socket says 172.21.15.1:33912) 2024-02-02T15:33:00.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.849+0000 7f660d400700 1 -- 172.21.15.1:0/3399375179 learned_addr learned my addr 172.21.15.1:0/3399375179 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:33:00.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.849+0000 7f660de00700 1 --2- 172.21.15.1:0/3399375179 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56042c631010 0x56042c642e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:00.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.849+0000 7f660d400700 1 -- 172.21.15.1:0/3399375179 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56042c63c7e0 msgr2=0x56042c641850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:00.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.849+0000 7f660d400700 1 --2- 172.21.15.1:0/3399375179 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56042c63c7e0 0x56042c641850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.849+0000 7f660d400700 1 -- 172.21.15.1:0/3399375179 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56042c631010 msgr2=0x56042c642e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:00.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.849+0000 7f660d400700 1 --2- 172.21.15.1:0/3399375179 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56042c631010 0x56042c642e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.849+0000 7f660d400700 1 -- 172.21.15.1:0/3399375179 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56042c4043b0 con 0x56042c631930 2024-02-02T15:33:00.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.850+0000 7f660d400700 1 --2- 172.21.15.1:0/3399375179 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56042c631930 0x56042c641310 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f65f80059a0 tx=0x7f65f80165d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96a890444e8f2762 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:33:00.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.850+0000 7f65f7e00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:33:00.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.850+0000 7f65f7e00700 1 -- 172.21.15.1:0/3399375179 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f8030850 con 0x56042c631930 2024-02-02T15:33:00.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.850+0000 7f65f7e00700 1 -- 172.21.15.1:0/3399375179 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f65f802c070 con 0x56042c631930 2024-02-02T15:33:00.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.850+0000 7f6628a7db80 1 -- 172.21.15.1:0/3399375179 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f660009d280 con 0x56042c631930 2024-02-02T15:33:00.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.850+0000 7f65f7e00700 1 -- 172.21.15.1:0/3399375179 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f65f800b310 con 0x56042c631930 2024-02-02T15:33:00.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.850+0000 7f6628a7db80 10 client.27266 fetch_fsmap learned FSMap version 38 2024-02-02T15:33:00.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.850+0000 7f6628a7db80 1 -- 172.21.15.1:0/3399375179 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x56042c3b0b60 con 0x56042c631930 2024-02-02T15:33:00.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.850+0000 7f65f7e00700 1 -- 172.21.15.1:0/3399375179 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f660009d280 con 0x56042c631930 2024-02-02T15:33:00.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.851+0000 7f65f7e00700 1 -- 172.21.15.1:0/3399375179 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 38) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f65f800f7b0 con 0x56042c631930 2024-02-02T15:33:00.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.851+0000 7f6628a7db80 10 client.27266 fetch_fsmap finished waiting for FSMap version 38 2024-02-02T15:33:00.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.851+0000 7f6628a7db80 10 client.27266 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:33:00.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.851+0000 7f6628a7db80 20 client.27266 populate_metadata read hostname 'smithi001' 2024-02-02T15:33:00.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.851+0000 7f6628a7db80 1 --2- 172.21.15.1:0/3399375179 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56042c69ec00 0x56042c6a1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:00.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.851+0000 7f6628a7db80 4 client.27266 mds_command: new command op to 24557 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:33:00.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.851+0000 7f6628a7db80 1 -- 172.21.15.1:0/3399375179 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56042c541f50 con 0x56042c69ec00 2024-02-02T15:33:00.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.851+0000 7f660de00700 1 --2- 172.21.15.1:0/3399375179 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56042c69ec00 0x56042c6a1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:00.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.852+0000 7f660de00700 1 --2- 172.21.15.1:0/3399375179 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56042c69ec00 0x56042c6a1020 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0478f709318ca07 server_cookie=9439b9bc62bd8993 in_seq=0 out_seq=0 2024-02-02T15:33:00.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.852+0000 7f65f7e00700 10 client.27266 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:33:00.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.852+0000 7f65f7e00700 1 -- 172.21.15.1:0/3399375179 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56042c541f50 con 0x56042c69ec00 2024-02-02T15:33:00.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.852+0000 7f65f7e00700 10 client.27266 handle_command_reply: tid=0 2024-02-02T15:33:00.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.852+0000 7f6628a7db80 1 client.27266 shutdown 2024-02-02T15:33:00.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.852+0000 7f6628a7db80 1 -- 172.21.15.1:0/3399375179 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56042c631930 msgr2=0x56042c641310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:00.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.852+0000 7f6628a7db80 1 --2- 172.21.15.1:0/3399375179 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56042c631930 0x56042c641310 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f65f80059a0 tx=0x7f65f80165d0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.852+0000 7f6628a7db80 1 -- 172.21.15.1:0/3399375179 shutdown_connections 2024-02-02T15:33:00.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.852+0000 7f6628a7db80 1 --2- 172.21.15.1:0/3399375179 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56042c69ec00 0x56042c6a1020 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.852+0000 7f6628a7db80 1 --2- 172.21.15.1:0/3399375179 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56042c63c7e0 0x56042c641850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.852+0000 7f6628a7db80 1 --2- 172.21.15.1:0/3399375179 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56042c631010 0x56042c642e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.852+0000 7f6628a7db80 1 --2- 172.21.15.1:0/3399375179 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56042c631930 0x56042c641310 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.852+0000 7f65f7e00700 0 client.27266 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:33:00.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.852+0000 7f6628a7db80 1 -- 172.21.15.1:0/3399375179 >> 172.21.15.1:0/3399375179 conn(0x56042c590db0 msgr2=0x56042c647900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:00.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.852+0000 7f6628a7db80 1 -- 172.21.15.1:0/3399375179 shutdown_connections 2024-02-02T15:33:00.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.852+0000 7f6628a7db80 1 -- 172.21.15.1:0/3399375179 wait complete. 2024-02-02T15:33:00.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.852+0000 7f6628a7db80 20 client.27266 trim_cache size 0 max 16384 2024-02-02T15:33:00.857 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:33:00.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.853+0000 7f6615000700 1 -- 172.21.15.1:0/4238928624 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f65f00a0690 msgr2=0x7f65f00a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:00.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.853+0000 7f6615000700 1 --2- 172.21.15.1:0/4238928624 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f65f00a0690 0x7f65f00a2b50 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f65f800b2e0 tx=0x7f65f8004580 comp rx=0 tx=0).stop 2024-02-02T15:33:00.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.853+0000 7f6615000700 1 -- 172.21.15.1:0/4238928624 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6610100ff0 msgr2=0x7f6610199030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:00.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.853+0000 7f6615000700 1 --2- 172.21.15.1:0/4238928624 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6610100ff0 0x7f6610199030 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f660000e9a0 tx=0x7f660000ecb0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.853+0000 7f6615000700 1 -- 172.21.15.1:0/4238928624 shutdown_connections 2024-02-02T15:33:00.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.853+0000 7f6615000700 1 --2- 172.21.15.1:0/4238928624 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f65f00a0690 0x7f65f00a2b50 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.853+0000 7f6615000700 1 --2- 172.21.15.1:0/4238928624 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f661019d9d0 0x7f661019de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.853+0000 7f6615000700 1 --2- 172.21.15.1:0/4238928624 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6610100ff0 0x7f6610199030 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.853+0000 7f6615000700 1 --2- 172.21.15.1:0/4238928624 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6610106110 0x7f6610199570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:00.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.853+0000 7f6615000700 1 -- 172.21.15.1:0/4238928624 >> 172.21.15.1:0/4238928624 conn(0x7f66100fc810 msgr2=0x7f66100fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:00.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.853+0000 7f6615000700 1 -- 172.21.15.1:0/4238928624 shutdown_connections 2024-02-02T15:33:00.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:00.853+0000 7f6615000700 1 -- 172.21.15.1:0/4238928624 wait complete. 2024-02-02T15:33:00.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:00 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/1170264682' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:33:00.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:00 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/1170264682' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:33:01.237 INFO:tasks.workunit.client.0.smithi001.stdout: 1 394801 8.85 MB/sec execute 155 sec latency 64.532 ms 2024-02-02T15:33:01.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:01 smithi001 ceph-mon[35470]: pgmap v183: 129 pgs: 129 active+clean; 241 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 207 op/s 2024-02-02T15:33:01.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:01 smithi071 ceph-mon[44707]: pgmap v183: 129 pgs: 129 active+clean; 241 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 207 op/s 2024-02-02T15:33:01.872 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-02T15:33:01.873 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-02T15:33:01.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:01 smithi142 ceph-mon[45728]: pgmap v183: 129 pgs: 129 active+clean; 241 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 207 op/s 2024-02-02T15:33:02.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.086+0000 7fb104c00700 1 -- 172.21.15.1:0/2676965952 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb100073a00 msgr2=0x7fb100111850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:02.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.086+0000 7fb104c00700 1 --2- 172.21.15.1:0/2676965952 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb100073a00 0x7fb100111850 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb0e8009990 tx=0x7fb0e8009ca0 comp rx=0 tx=0).stop 2024-02-02T15:33:02.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.086+0000 7fb104c00700 1 -- 172.21.15.1:0/2676965952 shutdown_connections 2024-02-02T15:33:02.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.086+0000 7fb104c00700 1 --2- 172.21.15.1:0/2676965952 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb100111d90 0x7fb100114180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:02.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.086+0000 7fb104c00700 1 --2- 172.21.15.1:0/2676965952 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb1000730e0 0x7fb1000734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:02.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.086+0000 7fb104c00700 1 --2- 172.21.15.1:0/2676965952 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb100073a00 0x7fb100111850 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:02.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.086+0000 7fb104c00700 1 -- 172.21.15.1:0/2676965952 >> 172.21.15.1:0/2676965952 conn(0x7fb1000fc810 msgr2=0x7fb1000fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:02.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.086+0000 7fb104c00700 1 -- 172.21.15.1:0/2676965952 shutdown_connections 2024-02-02T15:33:02.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.086+0000 7fb104c00700 1 -- 172.21.15.1:0/2676965952 wait complete. 2024-02-02T15:33:02.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.087+0000 7fb104c00700 1 Processor -- start 2024-02-02T15:33:02.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.087+0000 7fb104c00700 1 -- start start 2024-02-02T15:33:02.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.088+0000 7fb104c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb1000730e0 0x7fb10019d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:02.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.088+0000 7fb104c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb100073a00 0x7fb10019dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:02.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.088+0000 7fb104c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb100111d90 0x7fb1001a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:02.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.088+0000 7fb104c00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1001a2550 con 0x7fb1000730e0 2024-02-02T15:33:02.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.088+0000 7fb104c00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1001a26c0 con 0x7fb100111d90 2024-02-02T15:33:02.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.088+0000 7fb104c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1001a2880 con 0x7fb100073a00 2024-02-02T15:33:02.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.088+0000 7fb0fde00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb1000730e0 0x7fb10019d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:02.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.088+0000 7fb0fd400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb100073a00 0x7fb10019dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:02.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.088+0000 7fb0fd400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb100073a00 0x7fb10019dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:39852/0 (socket says 172.21.15.1:39852) 2024-02-02T15:33:02.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.088+0000 7fb0fd400700 1 -- 172.21.15.1:0/857442732 learned_addr learned my addr 172.21.15.1:0/857442732 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:33:02.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.088+0000 7fb0fe800700 1 --2- 172.21.15.1:0/857442732 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb100111d90 0x7fb1001a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:02.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.089+0000 7fb0fd400700 1 -- 172.21.15.1:0/857442732 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb100111d90 msgr2=0x7fb1001a1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:02.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.089+0000 7fb0fd400700 1 --2- 172.21.15.1:0/857442732 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb100111d90 0x7fb1001a1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:02.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.089+0000 7fb0fd400700 1 -- 172.21.15.1:0/857442732 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb1000730e0 msgr2=0x7fb10019d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:02.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.089+0000 7fb0fd400700 1 --2- 172.21.15.1:0/857442732 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb1000730e0 0x7fb10019d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:02.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.089+0000 7fb0fd400700 1 -- 172.21.15.1:0/857442732 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0e80095f0 con 0x7fb100073a00 2024-02-02T15:33:02.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.089+0000 7fb0fde00700 1 --2- 172.21.15.1:0/857442732 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb1000730e0 0x7fb10019d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:33:02.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.089+0000 7fb0fd400700 1 --2- 172.21.15.1:0/857442732 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb100073a00 0x7fb10019dab0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb0e8005910 tx=0x7fb0e8005720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:33:02.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.089+0000 7fb0f6a00700 1 -- 172.21.15.1:0/857442732 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e801d070 con 0x7fb100073a00 2024-02-02T15:33:02.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.089+0000 7fb0f6a00700 1 -- 172.21.15.1:0/857442732 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb0e8022470 con 0x7fb100073a00 2024-02-02T15:33:02.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.089+0000 7fb104c00700 1 -- 172.21.15.1:0/857442732 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1001a2b00 con 0x7fb100073a00 2024-02-02T15:33:02.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.089+0000 7fb0f6a00700 1 -- 172.21.15.1:0/857442732 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e8017650 con 0x7fb100073a00 2024-02-02T15:33:02.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.089+0000 7fb104c00700 1 -- 172.21.15.1:0/857442732 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1001aa3a0 con 0x7fb100073a00 2024-02-02T15:33:02.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.092+0000 7fb0f6a00700 1 -- 172.21.15.1:0/857442732 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 27) v1 ==== 135200+0+0 (secure 0 0 0) 0x7fb0e80225e0 con 0x7fb100073a00 2024-02-02T15:33:02.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.094+0000 7fb0f6a00700 1 --2- 172.21.15.1:0/857442732 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fb0e00a0710 0x7fb0e00a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:02.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.094+0000 7fb1187cbb80 1 Processor -- start 2024-02-02T15:33:02.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.094+0000 7fb0f6a00700 1 -- 172.21.15.1:0/857442732 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fb0e80cd3e0 con 0x7fb100073a00 2024-02-02T15:33:02.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.094+0000 7fb1187cbb80 1 -- start start 2024-02-02T15:33:02.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.094+0000 7fb0fde00700 1 --2- 172.21.15.1:0/857442732 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fb0e00a0710 0x7fb0e00a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:02.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.094+0000 7fb1187cbb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55793b402090 0x55793b402470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:02.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.094+0000 7fb1187cbb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55793b4029b0 0x55793b40d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:02.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.094+0000 7fb1187cbb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55793b40d860 0x55793b40fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:02.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.094+0000 7fb1187cbb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55793b376540 con 0x55793b4029b0 2024-02-02T15:33:02.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.094+0000 7fb1187cbb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55793b35ab20 con 0x55793b402090 2024-02-02T15:33:02.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.094+0000 7fb1187cbb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55793b3584e0 con 0x55793b40d860 2024-02-02T15:33:02.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.095+0000 7fb0fde00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55793b402090 0x55793b402470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:02.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.095+0000 7fb0fd400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55793b4029b0 0x55793b40d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:02.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.095+0000 7fb0fe800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55793b40d860 0x55793b40fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:02.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.095+0000 7fb0fd400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55793b4029b0 0x55793b40d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45252/0 (socket says 172.21.15.1:45252) 2024-02-02T15:33:02.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.095+0000 7fb0fd400700 1 -- 172.21.15.1:0/3718276078 learned_addr learned my addr 172.21.15.1:0/3718276078 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:33:02.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.095+0000 7fb0fe800700 1 --2- 172.21.15.1:0/3718276078 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55793b40d860 0x55793b40fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:39866/0 (socket says 172.21.15.1:39866) 2024-02-02T15:33:02.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.095+0000 7fb0fde00700 1 --2- 172.21.15.1:0/857442732 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fb0e00a0710 0x7fb0e00a2bd0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0011550 tx=0x7fb0f0009260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:33:02.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.095+0000 7fb0fd400700 1 -- 172.21.15.1:0/3718276078 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55793b40d860 msgr2=0x55793b40fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:02.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.095+0000 7fb0fd400700 1 --2- 172.21.15.1:0/3718276078 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55793b40d860 0x55793b40fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:02.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.095+0000 7fb0fd400700 1 -- 172.21.15.1:0/3718276078 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55793b402090 msgr2=0x55793b402470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:02.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.095+0000 7fb0fd400700 1 --2- 172.21.15.1:0/3718276078 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55793b402090 0x55793b402470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:02.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.095+0000 7fb0fd400700 1 -- 172.21.15.1:0/3718276078 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55793b1d5430 con 0x55793b4029b0 2024-02-02T15:33:02.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.095+0000 7fb0fe800700 1 --2- 172.21.15.1:0/3718276078 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55793b40d860 0x55793b40fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:33:02.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.095+0000 7fb0fd400700 1 --2- 172.21.15.1:0/3718276078 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55793b4029b0 0x55793b40d320 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fb0e8096c00 tx=0x7fb0e8022c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf90699745c9ddfb server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:33:02.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.096+0000 7fb0ef000700 1 -- 172.21.15.1:0/3718276078 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e80af030 con 0x55793b4029b0 2024-02-02T15:33:02.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.096+0000 7fb0ef000700 1 -- 172.21.15.1:0/3718276078 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb0e809a060 con 0x55793b4029b0 2024-02-02T15:33:02.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.096+0000 7fb0ef000700 1 -- 172.21.15.1:0/3718276078 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e80228c0 con 0x55793b4029b0 2024-02-02T15:33:02.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.096+0000 7fb1187cbb80 1 -- 172.21.15.1:0/3718276078 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55793b4029b0 msgr2=0x55793b40d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:02.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.096+0000 7fb1187cbb80 1 --2- 172.21.15.1:0/3718276078 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55793b4029b0 0x55793b40d320 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fb0e8096c00 tx=0x7fb0e8022c90 comp rx=0 tx=0).stop 2024-02-02T15:33:02.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.096+0000 7fb1187cbb80 1 -- 172.21.15.1:0/3718276078 shutdown_connections 2024-02-02T15:33:02.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.096+0000 7fb1187cbb80 1 --2- 172.21.15.1:0/3718276078 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55793b40d860 0x55793b40fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:02.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.097+0000 7fb1187cbb80 1 --2- 172.21.15.1:0/3718276078 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55793b402090 0x55793b402470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:02.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.097+0000 7fb1187cbb80 1 --2- 172.21.15.1:0/3718276078 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55793b4029b0 0x55793b40d320 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:02.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.097+0000 7fb1187cbb80 1 -- 172.21.15.1:0/3718276078 >> 172.21.15.1:0/3718276078 conn(0x55793b361e30 msgr2=0x55793b3fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:02.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.097+0000 7fb1187cbb80 1 -- 172.21.15.1:0/3718276078 shutdown_connections 2024-02-02T15:33:02.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.097+0000 7fb1187cbb80 1 -- 172.21.15.1:0/3718276078 wait complete. 2024-02-02T15:33:02.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.097+0000 7fb1187cbb80 1 -- start start 2024-02-02T15:33:02.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.097+0000 7fb1187cbb80 1 Processor -- start 2024-02-02T15:33:02.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.098+0000 7fb1187cbb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55793b402090 0x55793b40dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:02.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.098+0000 7fb1187cbb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55793b4029b0 0x55793b40e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:02.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.098+0000 7fb1187cbb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55793b40d860 0x55793b40e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:02.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.098+0000 7fb1187cbb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55793b353dd0 con 0x55793b402090 2024-02-02T15:33:02.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.098+0000 7fb1187cbb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55793b2efe00 con 0x55793b4029b0 2024-02-02T15:33:02.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.098+0000 7fb1187cbb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55793b2efaa0 con 0x55793b40d860 2024-02-02T15:33:02.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.098+0000 7fb1187cbb80 10 client.0 _ll_register_callbacks cb 0x55793b2f1310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:33:02.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.098+0000 7fb0fde00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55793b402090 0x55793b40dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:02.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.098+0000 7fb0fde00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55793b402090 0x55793b40dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45256/0 (socket says 172.21.15.1:45256) 2024-02-02T15:33:02.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.098+0000 7fb0fde00700 1 -- 172.21.15.1:0/2472560772 learned_addr learned my addr 172.21.15.1:0/2472560772 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:33:02.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.098+0000 7fb0fd400700 1 --2- 172.21.15.1:0/2472560772 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55793b4029b0 0x55793b40e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:02.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.098+0000 7fb0fe800700 1 --2- 172.21.15.1:0/2472560772 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55793b40d860 0x55793b40e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:02.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.099+0000 7fb0fe800700 1 -- 172.21.15.1:0/2472560772 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55793b4029b0 msgr2=0x55793b40e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:02.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.099+0000 7fb0fe800700 1 --2- 172.21.15.1:0/2472560772 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55793b4029b0 0x55793b40e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:02.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.099+0000 7fb0fe800700 1 -- 172.21.15.1:0/2472560772 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55793b402090 msgr2=0x55793b40dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:02.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.099+0000 7fb0fe800700 1 --2- 172.21.15.1:0/2472560772 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55793b402090 0x55793b40dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:02.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.099+0000 7fb0fe800700 1 -- 172.21.15.1:0/2472560772 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0f8011230 con 0x55793b40d860 2024-02-02T15:33:02.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.099+0000 7fb0fd400700 1 --2- 172.21.15.1:0/2472560772 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55793b4029b0 0x55793b40e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:33:02.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.099+0000 7fb0fe800700 1 --2- 172.21.15.1:0/2472560772 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55793b40d860 0x55793b40e6c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fb0f8007c80 tx=0x7fb0f8007f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b11e4a52b2cc7637 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:33:02.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.099+0000 7fb0e7400700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:33:02.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.100+0000 7fb0e7400700 1 -- 172.21.15.1:0/2472560772 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f800e040 con 0x55793b40d860 2024-02-02T15:33:02.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.100+0000 7fb0e7400700 1 -- 172.21.15.1:0/2472560772 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fb0f8004db0 con 0x55793b40d860 2024-02-02T15:33:02.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.100+0000 7fb1187cbb80 1 -- 172.21.15.1:0/2472560772 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55793b3f6170 con 0x55793b40d860 2024-02-02T15:33:02.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.100+0000 7fb0e7400700 1 -- 172.21.15.1:0/2472560772 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0f8011b60 con 0x55793b40d860 2024-02-02T15:33:02.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.100+0000 7fb1187cbb80 10 client.27277 fetch_fsmap learned FSMap version 38 2024-02-02T15:33:02.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.100+0000 7fb1187cbb80 1 -- 172.21.15.1:0/2472560772 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x7fb1001aa3a0 con 0x55793b40d860 2024-02-02T15:33:02.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.101+0000 7fb0e7400700 1 -- 172.21.15.1:0/2472560772 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55793b3f6170 con 0x55793b40d860 2024-02-02T15:33:02.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.101+0000 7fb0e7400700 1 -- 172.21.15.1:0/2472560772 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 38) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fb0f8016400 con 0x55793b40d860 2024-02-02T15:33:02.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.101+0000 7fb1187cbb80 10 client.27277 fetch_fsmap finished waiting for FSMap version 38 2024-02-02T15:33:02.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.101+0000 7fb1187cbb80 10 client.27277 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:33:02.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.101+0000 7fb1187cbb80 20 client.27277 populate_metadata read hostname 'smithi001' 2024-02-02T15:33:02.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.101+0000 7fb1187cbb80 1 --2- 172.21.15.1:0/2472560772 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55793b41cf40 0x55793b41f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:02.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.101+0000 7fb1187cbb80 4 client.27277 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:33:02.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.101+0000 7fb1187cbb80 1 -- 172.21.15.1:0/2472560772 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55793b34ce50 con 0x55793b41cf40 2024-02-02T15:33:02.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.101+0000 7fb0fde00700 1 --2- 172.21.15.1:0/2472560772 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55793b41cf40 0x55793b41f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:02.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.102+0000 7fb0fde00700 1 --2- 172.21.15.1:0/2472560772 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55793b41cf40 0x55793b41f360 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3d6b6e21ba9ed1c server_cookie=a5d0d0c3c9331a49 in_seq=0 out_seq=0 2024-02-02T15:33:02.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.102+0000 7fb0e7400700 10 client.27277 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:33:02.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.102+0000 7fb0e7400700 1 -- 172.21.15.1:0/2472560772 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55793b34ce50 con 0x55793b41cf40 2024-02-02T15:33:02.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.102+0000 7fb0e7400700 10 client.27277 handle_command_reply: tid=0 2024-02-02T15:33:02.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.102+0000 7fb1187cbb80 1 client.27277 shutdown 2024-02-02T15:33:02.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.103+0000 7fb1187cbb80 1 -- 172.21.15.1:0/2472560772 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55793b40d860 msgr2=0x55793b40e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:02.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.103+0000 7fb1187cbb80 1 --2- 172.21.15.1:0/2472560772 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55793b40d860 0x55793b40e6c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fb0f8007c80 tx=0x7fb0f8007f90 comp rx=0 tx=0).stop 2024-02-02T15:33:02.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.103+0000 7fb1187cbb80 1 -- 172.21.15.1:0/2472560772 shutdown_connections 2024-02-02T15:33:02.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.103+0000 7fb1187cbb80 1 --2- 172.21.15.1:0/2472560772 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55793b41cf40 0x55793b41f360 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:02.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.103+0000 7fb1187cbb80 1 --2- 172.21.15.1:0/2472560772 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55793b40d860 0x55793b40e6c0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:02.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.103+0000 7fb1187cbb80 1 --2- 172.21.15.1:0/2472560772 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55793b402090 0x55793b40dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:02.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.103+0000 7fb1187cbb80 1 --2- 172.21.15.1:0/2472560772 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55793b4029b0 0x55793b40e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:02.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.103+0000 7fb0e7400700 0 client.27277 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:33:02.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.103+0000 7fb1187cbb80 1 -- 172.21.15.1:0/2472560772 >> 172.21.15.1:0/2472560772 conn(0x55793b361e30 msgr2=0x55793b3fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:02.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.103+0000 7fb1187cbb80 1 -- 172.21.15.1:0/2472560772 shutdown_connections 2024-02-02T15:33:02.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.103+0000 7fb1187cbb80 1 -- 172.21.15.1:0/2472560772 wait complete. 2024-02-02T15:33:02.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.103+0000 7fb1187cbb80 20 client.27277 trim_cache size 0 max 16384 2024-02-02T15:33:02.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.121+0000 7fb1187cbb80 1 Processor -- start 2024-02-02T15:33:02.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.121+0000 7fb1187cbb80 1 -- start start 2024-02-02T15:33:02.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.121+0000 7fb1187cbb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55793b402090 0x55793b411c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:02.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.121+0000 7fb1187cbb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55793b4029b0 0x55793b40c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:02.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.121+0000 7fb1187cbb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55793b40d860 0x55793b409ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:02.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.121+0000 7fb1187cbb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55793b1d7720 con 0x55793b40d860 2024-02-02T15:33:02.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.121+0000 7fb1187cbb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55793b41fa80 con 0x55793b402090 2024-02-02T15:33:02.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.121+0000 7fb1187cbb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55793b40ae30 con 0x55793b4029b0 2024-02-02T15:33:02.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.122+0000 7fb0fd400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55793b4029b0 0x55793b40c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:02.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.122+0000 7fb0fe800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55793b40d860 0x55793b409ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:02.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.122+0000 7fb0fde00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55793b402090 0x55793b411c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:02.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.122+0000 7fb0fd400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55793b4029b0 0x55793b40c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:39880/0 (socket says 172.21.15.1:39880) 2024-02-02T15:33:02.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.122+0000 7fb0fd400700 1 -- 172.21.15.1:0/992570278 learned_addr learned my addr 172.21.15.1:0/992570278 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:33:02.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.122+0000 7fb0fd400700 1 -- 172.21.15.1:0/992570278 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55793b402090 msgr2=0x55793b411c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:02.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.122+0000 7fb0fd400700 1 --2- 172.21.15.1:0/992570278 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55793b402090 0x55793b411c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:02.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.122+0000 7fb0fd400700 1 -- 172.21.15.1:0/992570278 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55793b40d860 msgr2=0x55793b409ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:02.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.122+0000 7fb0fd400700 1 --2- 172.21.15.1:0/992570278 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55793b40d860 0x55793b409ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:02.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.122+0000 7fb0fd400700 1 -- 172.21.15.1:0/992570278 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55793b403b80 con 0x55793b4029b0 2024-02-02T15:33:02.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.122+0000 7fb0fde00700 1 --2- 172.21.15.1:0/992570278 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55793b402090 0x55793b411c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:33:02.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.122+0000 7fb0fe800700 1 --2- 172.21.15.1:0/992570278 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55793b40d860 0x55793b409ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:33:02.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.122+0000 7fb0fd400700 1 --2- 172.21.15.1:0/992570278 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55793b4029b0 0x55793b40c010 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fb0e800f740 tx=0x7fb0e80cc040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3eaaa0676d92eb5 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:33:02.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.123+0000 7fb0efa00700 1 -- 172.21.15.1:0/992570278 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e80af040 con 0x55793b4029b0 2024-02-02T15:33:02.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.123+0000 7fb0efa00700 1 -- 172.21.15.1:0/992570278 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb0e80a26e0 con 0x55793b4029b0 2024-02-02T15:33:02.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.123+0000 7fb0efa00700 1 -- 172.21.15.1:0/992570278 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e809e530 con 0x55793b4029b0 2024-02-02T15:33:02.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.123+0000 7fb1187cbb80 1 -- 172.21.15.1:0/992570278 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55793b4029b0 msgr2=0x55793b40c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:02.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.123+0000 7fb1187cbb80 1 --2- 172.21.15.1:0/992570278 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55793b4029b0 0x55793b40c010 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fb0e800f740 tx=0x7fb0e80cc040 comp rx=0 tx=0).stop 2024-02-02T15:33:02.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.123+0000 7fb1187cbb80 1 -- 172.21.15.1:0/992570278 shutdown_connections 2024-02-02T15:33:02.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.123+0000 7fb1187cbb80 1 --2- 172.21.15.1:0/992570278 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55793b40d860 0x55793b409ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:02.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.123+0000 7fb1187cbb80 1 --2- 172.21.15.1:0/992570278 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55793b402090 0x55793b411c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:02.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.123+0000 7fb1187cbb80 1 --2- 172.21.15.1:0/992570278 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55793b4029b0 0x55793b40c010 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:02.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.123+0000 7fb1187cbb80 1 -- 172.21.15.1:0/992570278 >> 172.21.15.1:0/992570278 conn(0x55793b361e30 msgr2=0x55793b407b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:02.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.123+0000 7fb1187cbb80 1 -- 172.21.15.1:0/992570278 shutdown_connections 2024-02-02T15:33:02.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.123+0000 7fb1187cbb80 1 -- 172.21.15.1:0/992570278 wait complete. 2024-02-02T15:33:02.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.124+0000 7fb1187cbb80 1 -- start start 2024-02-02T15:33:02.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.124+0000 7fb1187cbb80 1 Processor -- start 2024-02-02T15:33:02.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.124+0000 7fb1187cbb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55793b402090 0x55793b466f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:02.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.124+0000 7fb1187cbb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55793b4029b0 0x55793b4674d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:02.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.125+0000 7fb1187cbb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55793b40d860 0x55793b4629b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:02.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.125+0000 7fb1187cbb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55793b4630c0 con 0x55793b402090 2024-02-02T15:33:02.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.125+0000 7fb1187cbb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55793b463230 con 0x55793b4029b0 2024-02-02T15:33:02.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.125+0000 7fb1187cbb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55793b4633f0 con 0x55793b40d860 2024-02-02T15:33:02.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.125+0000 7fb1187cbb80 10 client.0 _ll_register_callbacks cb 0x55793b2f1310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:33:02.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.125+0000 7fb0fde00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55793b402090 0x55793b466f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:02.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.125+0000 7fb0fe800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55793b40d860 0x55793b4629b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:02.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.125+0000 7fb0fde00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55793b402090 0x55793b466f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45284/0 (socket says 172.21.15.1:45284) 2024-02-02T15:33:02.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.125+0000 7fb0fd400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55793b4029b0 0x55793b4674d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:02.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.125+0000 7fb0fde00700 1 -- 172.21.15.1:0/2953485527 learned_addr learned my addr 172.21.15.1:0/2953485527 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:33:02.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.125+0000 7fb0fe800700 1 -- 172.21.15.1:0/2953485527 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55793b4029b0 msgr2=0x55793b4674d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:02.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.125+0000 7fb0fe800700 1 --2- 172.21.15.1:0/2953485527 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55793b4029b0 0x55793b4674d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:02.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.125+0000 7fb0fe800700 1 -- 172.21.15.1:0/2953485527 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55793b402090 msgr2=0x55793b466f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:02.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.125+0000 7fb0fe800700 1 --2- 172.21.15.1:0/2953485527 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55793b402090 0x55793b466f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:02.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.125+0000 7fb0fe800700 1 -- 172.21.15.1:0/2953485527 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1001aa3a0 con 0x55793b40d860 2024-02-02T15:33:02.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.125+0000 7fb0fd400700 1 --2- 172.21.15.1:0/2953485527 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55793b4029b0 0x55793b4674d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:33:02.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.126+0000 7fb0fe800700 1 --2- 172.21.15.1:0/2953485527 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55793b40d860 0x55793b4629b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb0f8002230 tx=0x7fb0f8016630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63651b877851f9de server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:33:02.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.126+0000 7fb0e7400700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:33:02.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.126+0000 7fb0e7400700 1 -- 172.21.15.1:0/2953485527 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f800e070 con 0x55793b40d860 2024-02-02T15:33:02.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.126+0000 7fb0e7400700 1 -- 172.21.15.1:0/2953485527 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7fb0f8016a80 con 0x55793b40d860 2024-02-02T15:33:02.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.126+0000 7fb1187cbb80 1 -- 172.21.15.1:0/2953485527 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb0e809a060 con 0x55793b40d860 2024-02-02T15:33:02.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.126+0000 7fb0e7400700 1 -- 172.21.15.1:0/2953485527 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0f8011d60 con 0x55793b40d860 2024-02-02T15:33:02.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.126+0000 7fb1187cbb80 10 client.27283 fetch_fsmap learned FSMap version 38 2024-02-02T15:33:02.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.126+0000 7fb1187cbb80 1 -- 172.21.15.1:0/2953485527 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55793b1d5430 con 0x55793b40d860 2024-02-02T15:33:02.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.126+0000 7fb0e7400700 1 -- 172.21.15.1:0/2953485527 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb0e809a060 con 0x55793b40d860 2024-02-02T15:33:02.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.126+0000 7fb0e7400700 1 -- 172.21.15.1:0/2953485527 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 38) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fb0f8032020 con 0x55793b40d860 2024-02-02T15:33:02.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.127+0000 7fb1187cbb80 10 client.27283 fetch_fsmap finished waiting for FSMap version 38 2024-02-02T15:33:02.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.127+0000 7fb1187cbb80 10 client.27283 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:33:02.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.127+0000 7fb1187cbb80 20 client.27283 populate_metadata read hostname 'smithi001' 2024-02-02T15:33:02.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.127+0000 7fb1187cbb80 1 --2- 172.21.15.1:0/2953485527 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55793b46ff30 0x55793b472350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:02.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.127+0000 7fb1187cbb80 4 client.27283 mds_command: new command op to 24557 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-02T15:33:02.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.127+0000 7fb1187cbb80 1 -- 172.21.15.1:0/2953485527 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55793b312fd0 con 0x55793b46ff30 2024-02-02T15:33:02.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.127+0000 7fb0fde00700 1 --2- 172.21.15.1:0/2953485527 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55793b46ff30 0x55793b472350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:02.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.127+0000 7fb0fde00700 1 --2- 172.21.15.1:0/2953485527 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55793b46ff30 0x55793b472350 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23f2022f3b1356c1 server_cookie=85cf785e589339f8 in_seq=0 out_seq=0 2024-02-02T15:33:02.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.127+0000 7fb0e7400700 10 client.27283 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:33:02.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.128+0000 7fb0e7400700 1 -- 172.21.15.1:0/2953485527 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55793b312fd0 con 0x55793b46ff30 2024-02-02T15:33:02.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.128+0000 7fb0e7400700 10 client.27283 handle_command_reply: tid=0 2024-02-02T15:33:02.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.128+0000 7fb1187cbb80 1 client.27283 shutdown 2024-02-02T15:33:02.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.128+0000 7fb1187cbb80 1 -- 172.21.15.1:0/2953485527 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55793b40d860 msgr2=0x55793b4629b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:02.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.128+0000 7fb1187cbb80 1 --2- 172.21.15.1:0/2953485527 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55793b40d860 0x55793b4629b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb0f8002230 tx=0x7fb0f8016630 comp rx=0 tx=0).stop 2024-02-02T15:33:02.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.128+0000 7fb1187cbb80 1 -- 172.21.15.1:0/2953485527 shutdown_connections 2024-02-02T15:33:02.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.128+0000 7fb1187cbb80 1 --2- 172.21.15.1:0/2953485527 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55793b46ff30 0x55793b472350 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:02.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.129+0000 7fb1187cbb80 1 --2- 172.21.15.1:0/2953485527 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55793b40d860 0x55793b4629b0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:02.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.129+0000 7fb1187cbb80 1 --2- 172.21.15.1:0/2953485527 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55793b402090 0x55793b466f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:02.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.129+0000 7fb1187cbb80 1 --2- 172.21.15.1:0/2953485527 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55793b4029b0 0x55793b4674d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:02.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.129+0000 7fb1187cbb80 1 -- 172.21.15.1:0/2953485527 >> 172.21.15.1:0/2953485527 conn(0x55793b361e30 msgr2=0x55793b403f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:02.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.129+0000 7fb0e7400700 0 client.27283 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:33:02.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.129+0000 7fb1187cbb80 1 -- 172.21.15.1:0/2953485527 shutdown_connections 2024-02-02T15:33:02.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.129+0000 7fb1187cbb80 1 -- 172.21.15.1:0/2953485527 wait complete. 2024-02-02T15:33:02.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.129+0000 7fb1187cbb80 20 client.27283 trim_cache size 0 max 16384 2024-02-02T15:33:02.133 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:33:02.133 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2024-02-02T15:33:02.133 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "27022107-d9ae-4852-a23b-370fce62e779", 2024-02-02T15:33:02.133 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2024-02-02T15:33:02.133 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:33:02.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.129+0000 7fb104c00700 1 -- 172.21.15.1:0/857442732 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fb0e00a0710 msgr2=0x7fb0e00a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:02.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.129+0000 7fb104c00700 1 --2- 172.21.15.1:0/857442732 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fb0e00a0710 0x7fb0e00a2bd0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0011550 tx=0x7fb0f0009260 comp rx=0 tx=0).stop 2024-02-02T15:33:02.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.129+0000 7fb104c00700 1 -- 172.21.15.1:0/857442732 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb100073a00 msgr2=0x7fb10019dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:02.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.129+0000 7fb104c00700 1 --2- 172.21.15.1:0/857442732 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb100073a00 0x7fb10019dab0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb0e8005910 tx=0x7fb0e8005720 comp rx=0 tx=0).stop 2024-02-02T15:33:02.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.130+0000 7fb104c00700 1 -- 172.21.15.1:0/857442732 shutdown_connections 2024-02-02T15:33:02.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.130+0000 7fb104c00700 1 --2- 172.21.15.1:0/857442732 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fb0e00a0710 0x7fb0e00a2bd0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:02.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.130+0000 7fb104c00700 1 --2- 172.21.15.1:0/857442732 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb100111d90 0x7fb1001a1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:02.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.130+0000 7fb104c00700 1 --2- 172.21.15.1:0/857442732 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb1000730e0 0x7fb10019d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:02.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.130+0000 7fb104c00700 1 --2- 172.21.15.1:0/857442732 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb100073a00 0x7fb10019dab0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:02.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.130+0000 7fb104c00700 1 -- 172.21.15.1:0/857442732 >> 172.21.15.1:0/857442732 conn(0x7fb1000fc810 msgr2=0x7fb1000fd480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:02.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.130+0000 7fb104c00700 1 -- 172.21.15.1:0/857442732 shutdown_connections 2024-02-02T15:33:02.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.130+0000 7fb104c00700 1 -- 172.21.15.1:0/857442732 wait complete. 2024-02-02T15:33:02.149 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-02T15:33:02.237 INFO:tasks.workunit.client.0.smithi001.stdout: 1 396326 8.83 MB/sec execute 156 sec latency 60.147 ms 2024-02-02T15:33:02.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.362+0000 7f0a12200700 1 -- 172.21.15.1:0/2764724625 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a0c0730e0 msgr2=0x7f0a0c0734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:02.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.362+0000 7f0a12200700 1 --2- 172.21.15.1:0/2764724625 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a0c0730e0 0x7f0a0c0734c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f09fc009990 tx=0x7f09fc009ca0 comp rx=0 tx=0).stop 2024-02-02T15:33:02.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.362+0000 7f0a12200700 1 -- 172.21.15.1:0/2764724625 shutdown_connections 2024-02-02T15:33:02.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.362+0000 7f0a12200700 1 --2- 172.21.15.1:0/2764724625 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a0c111aa0 0x7f0a0c113e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:02.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.362+0000 7f0a12200700 1 --2- 172.21.15.1:0/2764724625 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a0c0730e0 0x7f0a0c0734c0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:02.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.362+0000 7f0a12200700 1 --2- 172.21.15.1:0/2764724625 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a0c073a00 0x7f0a0c111560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:02.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.362+0000 7f0a12200700 1 -- 172.21.15.1:0/2764724625 >> 172.21.15.1:0/2764724625 conn(0x7f0a0c0fc580 msgr2=0x7f0a0c0fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:02.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.362+0000 7f0a12200700 1 -- 172.21.15.1:0/2764724625 shutdown_connections 2024-02-02T15:33:02.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.362+0000 7f0a12200700 1 -- 172.21.15.1:0/2764724625 wait complete. 2024-02-02T15:33:02.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.364+0000 7f0a12200700 1 Processor -- start 2024-02-02T15:33:02.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.364+0000 7f0a12200700 1 -- start start 2024-02-02T15:33:02.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.364+0000 7f0a12200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a0c0730e0 0x7f0a0c19d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:02.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.364+0000 7f0a12200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a0c073a00 0x7f0a0c19d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:02.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.364+0000 7f0a12200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a0c111aa0 0x7f0a0c1a1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:02.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.364+0000 7f0a12200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0c1a2200 con 0x7f0a0c111aa0 2024-02-02T15:33:02.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.364+0000 7f0a12200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0c1a2370 con 0x7f0a0c073a00 2024-02-02T15:33:02.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.364+0000 7f0a12200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0c1a2530 con 0x7f0a0c0730e0 2024-02-02T15:33:02.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.365+0000 7f0a0be00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a0c073a00 0x7f0a0c19d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:02.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.365+0000 7f0a10e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a0c0730e0 0x7f0a0c19d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:02.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.365+0000 7f0a0be00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a0c073a00 0x7f0a0c19d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:33986/0 (socket says 172.21.15.1:33986) 2024-02-02T15:33:02.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.365+0000 7f0a0be00700 1 -- 172.21.15.1:0/3837234279 learned_addr learned my addr 172.21.15.1:0/3837234279 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:33:02.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.365+0000 7f0a11800700 1 --2- 172.21.15.1:0/3837234279 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a0c111aa0 0x7f0a0c1a1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:02.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.365+0000 7f0a0be00700 1 -- 172.21.15.1:0/3837234279 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a0c0730e0 msgr2=0x7f0a0c19d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:02.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.365+0000 7f0a0be00700 1 --2- 172.21.15.1:0/3837234279 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a0c0730e0 0x7f0a0c19d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:02.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.365+0000 7f0a0be00700 1 -- 172.21.15.1:0/3837234279 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a0c111aa0 msgr2=0x7f0a0c1a1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:02.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.365+0000 7f0a0be00700 1 --2- 172.21.15.1:0/3837234279 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a0c111aa0 0x7f0a0c1a1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:02.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.365+0000 7f0a0be00700 1 -- 172.21.15.1:0/3837234279 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a04009c40 con 0x7f0a0c073a00 2024-02-02T15:33:02.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.365+0000 7f0a10e00700 1 --2- 172.21.15.1:0/3837234279 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a0c0730e0 0x7f0a0c19d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:33:02.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.365+0000 7f0a11800700 1 --2- 172.21.15.1:0/3837234279 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a0c111aa0 0x7f0a0c1a1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:33:02.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.365+0000 7f0a0be00700 1 --2- 172.21.15.1:0/3837234279 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a0c073a00 0x7f0a0c19d760 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f09f400e9a0 tx=0x7f09f400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:33:02.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.366+0000 7f0a09600700 1 -- 172.21.15.1:0/3837234279 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f4009790 con 0x7f0a0c073a00 2024-02-02T15:33:02.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.366+0000 7f0a12200700 1 -- 172.21.15.1:0/3837234279 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09fc0095f0 con 0x7f0a0c073a00 2024-02-02T15:33:02.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.366+0000 7f0a09600700 1 -- 172.21.15.1:0/3837234279 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f09f400ce80 con 0x7f0a0c073a00 2024-02-02T15:33:02.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.366+0000 7f0a09600700 1 -- 172.21.15.1:0/3837234279 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f4018680 con 0x7f0a0c073a00 2024-02-02T15:33:02.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.366+0000 7f0a12200700 1 -- 172.21.15.1:0/3837234279 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a0c1a9e70 con 0x7f0a0c073a00 2024-02-02T15:33:02.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.370+0000 7f0a09600700 1 -- 172.21.15.1:0/3837234279 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 27) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f09f4010430 con 0x7f0a0c073a00 2024-02-02T15:33:02.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.370+0000 7f0a273b4b80 1 Processor -- start 2024-02-02T15:33:02.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.371+0000 7f0a273b4b80 1 -- start start 2024-02-02T15:33:02.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.371+0000 7f0a273b4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fdb08f21f0 0x55fdb08f25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:02.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.371+0000 7f0a273b4b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fdb08f2b10 0x55fdb08fd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:02.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.371+0000 7f0a273b4b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fdb08fd9f0 0x55fdb08ffde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:02.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.371+0000 7f0a273b4b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdb08664c0 con 0x55fdb08fd9f0 2024-02-02T15:33:02.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.371+0000 7f0a273b4b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdb084aaa0 con 0x55fdb08f2b10 2024-02-02T15:33:02.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.371+0000 7f0a273b4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdb0848460 con 0x55fdb08f21f0 2024-02-02T15:33:02.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.371+0000 7f0a10e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fdb08f2b10 0x55fdb08fd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:02.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.371+0000 7f0a11800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fdb08f21f0 0x55fdb08f25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:02.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.371+0000 7f0a11800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fdb08f21f0 0x55fdb08f25d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:39910/0 (socket says 172.21.15.1:39910) 2024-02-02T15:33:02.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.371+0000 7f0a11800700 1 -- 172.21.15.1:0/3929738201 learned_addr learned my addr 172.21.15.1:0/3929738201 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:33:02.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.371+0000 7f0a10e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fdb08f2b10 0x55fdb08fd4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:33990/0 (socket says 172.21.15.1:33990) 2024-02-02T15:33:02.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.371+0000 7f0a0be00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fdb08fd9f0 0x55fdb08ffde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:02.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.372+0000 7f0a10e00700 1 -- 172.21.15.1:0/3929738201 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fdb08f21f0 msgr2=0x55fdb08f25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:02.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.372+0000 7f0a10e00700 1 --2- 172.21.15.1:0/3929738201 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fdb08f21f0 0x55fdb08f25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:02.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.372+0000 7f0a10e00700 1 -- 172.21.15.1:0/3929738201 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fdb08fd9f0 msgr2=0x55fdb08ffde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:02.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.372+0000 7f0a10e00700 1 --2- 172.21.15.1:0/3929738201 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fdb08fd9f0 0x55fdb08ffde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:02.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.372+0000 7f0a10e00700 1 -- 172.21.15.1:0/3929738201 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fdb06c53b0 con 0x55fdb08f2b10 2024-02-02T15:33:02.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.372+0000 7f0a11800700 1 --2- 172.21.15.1:0/3929738201 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fdb08f21f0 0x55fdb08f25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:33:02.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.372+0000 7f0a10e00700 1 --2- 172.21.15.1:0/3929738201 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fdb08f2b10 0x55fdb08fd4b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f09fc000c00 tx=0x7f09fc005ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7be0d7445d280e9d server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:33:02.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.372+0000 7f0a09600700 1 --2- 172.21.15.1:0/3837234279 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f09ec0a0690 0x7f09ec0a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:02.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.372+0000 7f0a02000700 1 -- 172.21.15.1:0/3929738201 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09fc01d050 con 0x55fdb08f2b10 2024-02-02T15:33:02.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.372+0000 7f0a02000700 1 -- 172.21.15.1:0/3929738201 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f09fc01e040 con 0x55fdb08f2b10 2024-02-02T15:33:02.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.372+0000 7f0a09600700 1 -- 172.21.15.1:0/3837234279 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f09f4014070 con 0x7f0a0c073a00 2024-02-02T15:33:02.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.372+0000 7f0a02000700 1 -- 172.21.15.1:0/3929738201 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09fc005430 con 0x55fdb08f2b10 2024-02-02T15:33:02.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.372+0000 7f0a11800700 1 --2- 172.21.15.1:0/3837234279 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f09ec0a0690 0x7f09ec0a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:02.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.373+0000 7f0a273b4b80 1 -- 172.21.15.1:0/3929738201 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fdb08f2b10 msgr2=0x55fdb08fd4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:02.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.373+0000 7f0a273b4b80 1 --2- 172.21.15.1:0/3929738201 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fdb08f2b10 0x55fdb08fd4b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f09fc000c00 tx=0x7f09fc005ea0 comp rx=0 tx=0).stop 2024-02-02T15:33:02.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.373+0000 7f0a11800700 1 --2- 172.21.15.1:0/3837234279 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f09ec0a0690 0x7f09ec0a2b50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0a040079f0 tx=0x7f0a0400b040 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:33:02.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.373+0000 7f0a273b4b80 1 -- 172.21.15.1:0/3929738201 shutdown_connections 2024-02-02T15:33:02.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.373+0000 7f0a273b4b80 1 --2- 172.21.15.1:0/3929738201 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fdb08fd9f0 0x55fdb08ffde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:02.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.373+0000 7f0a273b4b80 1 --2- 172.21.15.1:0/3929738201 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fdb08f21f0 0x55fdb08f25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:02.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.373+0000 7f0a273b4b80 1 --2- 172.21.15.1:0/3929738201 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fdb08f2b10 0x55fdb08fd4b0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:02.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.373+0000 7f0a273b4b80 1 -- 172.21.15.1:0/3929738201 >> 172.21.15.1:0/3929738201 conn(0x55fdb0851db0 msgr2=0x55fdb08ee5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:02.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.373+0000 7f0a273b4b80 1 -- 172.21.15.1:0/3929738201 shutdown_connections 2024-02-02T15:33:02.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.373+0000 7f0a273b4b80 1 -- 172.21.15.1:0/3929738201 wait complete. 2024-02-02T15:33:02.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.374+0000 7f0a273b4b80 1 -- start start 2024-02-02T15:33:02.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.374+0000 7f0a273b4b80 1 Processor -- start 2024-02-02T15:33:02.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.375+0000 7f0a273b4b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fdb08f21f0 0x55fdb08fd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:02.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.375+0000 7f0a273b4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fdb08f2b10 0x55fdb08fa600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:02.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.375+0000 7f0a273b4b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fdb08fd9f0 0x55fdb08fac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:02.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.375+0000 7f0a273b4b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdb0843d50 con 0x55fdb08fd9f0 2024-02-02T15:33:02.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.375+0000 7f0a273b4b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdb07dfd80 con 0x55fdb08f21f0 2024-02-02T15:33:02.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.375+0000 7f0a273b4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdb07dfa20 con 0x55fdb08f2b10 2024-02-02T15:33:02.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.375+0000 7f0a273b4b80 10 client.0 _ll_register_callbacks cb 0x55fdb07e1290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:33:02.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.375+0000 7f0a10e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fdb08f21f0 0x55fdb08fd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:02.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.375+0000 7f0a10e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fdb08f21f0 0x55fdb08fd3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:34000/0 (socket says 172.21.15.1:34000) 2024-02-02T15:33:02.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.375+0000 7f0a10e00700 1 -- 172.21.15.1:0/2522553303 learned_addr learned my addr 172.21.15.1:0/2522553303 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:33:02.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.375+0000 7f0a11800700 1 --2- 172.21.15.1:0/2522553303 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fdb08fd9f0 0x55fdb08fac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:02.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.375+0000 7f0a0be00700 1 --2- 172.21.15.1:0/2522553303 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fdb08f2b10 0x55fdb08fa600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:02.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.375+0000 7f0a10e00700 1 -- 172.21.15.1:0/2522553303 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fdb08f2b10 msgr2=0x55fdb08fa600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:02.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.375+0000 7f0a10e00700 1 --2- 172.21.15.1:0/2522553303 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fdb08f2b10 0x55fdb08fa600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:02.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.375+0000 7f0a10e00700 1 -- 172.21.15.1:0/2522553303 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fdb08fd9f0 msgr2=0x55fdb08fac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:02.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.375+0000 7f0a10e00700 1 --2- 172.21.15.1:0/2522553303 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fdb08fd9f0 0x55fdb08fac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:02.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.375+0000 7f0a10e00700 1 -- 172.21.15.1:0/2522553303 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a04009220 con 0x55fdb08f21f0 2024-02-02T15:33:02.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.376+0000 7f0a10e00700 1 --2- 172.21.15.1:0/2522553303 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fdb08f21f0 0x55fdb08fd3f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f09fc026040 tx=0x7f09fc004a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9142d1feaec84f9c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:33:02.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.376+0000 7f09faa00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:33:02.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.376+0000 7f09faa00700 1 -- 172.21.15.1:0/2522553303 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09fc0052d0 con 0x55fdb08f21f0 2024-02-02T15:33:02.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.376+0000 7f09faa00700 1 -- 172.21.15.1:0/2522553303 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f09fc005430 con 0x55fdb08f21f0 2024-02-02T15:33:02.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.376+0000 7f0a273b4b80 1 -- 172.21.15.1:0/2522553303 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fdb08e6340 con 0x55fdb08f21f0 2024-02-02T15:33:02.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.376+0000 7f09faa00700 1 -- 172.21.15.1:0/2522553303 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f09fc01d020 con 0x55fdb08f21f0 2024-02-02T15:33:02.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.376+0000 7f0a273b4b80 10 client.27296 fetch_fsmap learned FSMap version 38 2024-02-02T15:33:02.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.376+0000 7f0a273b4b80 1 -- 172.21.15.1:0/2522553303 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x7f0a0c114840 con 0x55fdb08f21f0 2024-02-02T15:33:02.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.376+0000 7f09faa00700 1 -- 172.21.15.1:0/2522553303 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fdb08e6340 con 0x55fdb08f21f0 2024-02-02T15:33:02.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.377+0000 7f09faa00700 1 -- 172.21.15.1:0/2522553303 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 38) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f09fc023460 con 0x55fdb08f21f0 2024-02-02T15:33:02.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.377+0000 7f0a273b4b80 10 client.27296 fetch_fsmap finished waiting for FSMap version 38 2024-02-02T15:33:02.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.377+0000 7f0a273b4b80 10 client.27296 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:33:02.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.377+0000 7f0a273b4b80 20 client.27296 populate_metadata read hostname 'smithi001' 2024-02-02T15:33:02.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.377+0000 7f0a273b4b80 1 --2- 172.21.15.1:0/2522553303 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55fdb090ce60 0x55fdb090f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:02.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.377+0000 7f0a273b4b80 4 client.27296 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:33:02.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.377+0000 7f0a273b4b80 1 -- 172.21.15.1:0/2522553303 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fdb083cdd0 con 0x55fdb090ce60 2024-02-02T15:33:02.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.377+0000 7f0a0be00700 1 --2- 172.21.15.1:0/2522553303 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55fdb090ce60 0x55fdb090f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:02.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.377+0000 7f0a0be00700 1 --2- 172.21.15.1:0/2522553303 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55fdb090ce60 0x55fdb090f280 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75f60006fb24adc server_cookie=148caa3e6bc7884c in_seq=0 out_seq=0 2024-02-02T15:33:02.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.377+0000 7f09faa00700 10 client.27296 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:33:02.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.378+0000 7f09faa00700 1 -- 172.21.15.1:0/2522553303 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55fdb083cdd0 con 0x55fdb090ce60 2024-02-02T15:33:02.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.378+0000 7f09faa00700 10 client.27296 handle_command_reply: tid=0 2024-02-02T15:33:02.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.378+0000 7f0a273b4b80 1 client.27296 shutdown 2024-02-02T15:33:02.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.378+0000 7f0a273b4b80 1 -- 172.21.15.1:0/2522553303 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fdb08f21f0 msgr2=0x55fdb08fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:02.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.378+0000 7f0a273b4b80 1 --2- 172.21.15.1:0/2522553303 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fdb08f21f0 0x55fdb08fd3f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f09fc026040 tx=0x7f09fc004a90 comp rx=0 tx=0).stop 2024-02-02T15:33:02.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.378+0000 7f0a273b4b80 1 -- 172.21.15.1:0/2522553303 shutdown_connections 2024-02-02T15:33:02.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.378+0000 7f0a273b4b80 1 --2- 172.21.15.1:0/2522553303 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55fdb090ce60 0x55fdb090f280 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:02.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.378+0000 7f0a273b4b80 1 --2- 172.21.15.1:0/2522553303 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fdb08fd9f0 0x55fdb08fac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:02.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.378+0000 7f0a273b4b80 1 --2- 172.21.15.1:0/2522553303 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fdb08f21f0 0x55fdb08fd3f0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:02.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.378+0000 7f09faa00700 0 client.27296 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:33:02.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.378+0000 7f0a273b4b80 1 --2- 172.21.15.1:0/2522553303 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fdb08f2b10 0x55fdb08fa600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:02.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.378+0000 7f0a273b4b80 1 -- 172.21.15.1:0/2522553303 >> 172.21.15.1:0/2522553303 conn(0x55fdb0851db0 msgr2=0x55fdb08ee1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:02.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.379+0000 7f0a273b4b80 1 -- 172.21.15.1:0/2522553303 shutdown_connections 2024-02-02T15:33:02.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.379+0000 7f0a273b4b80 1 -- 172.21.15.1:0/2522553303 wait complete. 2024-02-02T15:33:02.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.379+0000 7f0a273b4b80 20 client.27296 trim_cache size 0 max 16384 2024-02-02T15:33:02.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.396+0000 7f0a273b4b80 1 Processor -- start 2024-02-02T15:33:02.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.396+0000 7f0a273b4b80 1 -- start start 2024-02-02T15:33:02.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.396+0000 7f0a273b4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fdb08f21f0 0x55fdb0904450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:02.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.396+0000 7f0a273b4b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fdb08f2b10 0x55fdb08f4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:02.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.396+0000 7f0a273b4b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fdb08fd9f0 0x55fdb08f5950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:02.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.396+0000 7f0a273b4b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdb06c76a0 con 0x55fdb08f2b10 2024-02-02T15:33:02.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.396+0000 7f0a273b4b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdb08f5450 con 0x55fdb08fd9f0 2024-02-02T15:33:02.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.396+0000 7f0a273b4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdb08f8bb0 con 0x55fdb08f21f0 2024-02-02T15:33:02.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.396+0000 7f0a0be00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fdb08f2b10 0x55fdb08f4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:02.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.397+0000 7f0a10e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fdb08f21f0 0x55fdb0904450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:02.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.397+0000 7f0a0be00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fdb08f2b10 0x55fdb08f4a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45338/0 (socket says 172.21.15.1:45338) 2024-02-02T15:33:02.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.397+0000 7f0a0be00700 1 -- 172.21.15.1:0/1479585681 learned_addr learned my addr 172.21.15.1:0/1479585681 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:33:02.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.397+0000 7f0a11800700 1 --2- 172.21.15.1:0/1479585681 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fdb08fd9f0 0x55fdb08f5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:02.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.397+0000 7f0a10e00700 1 -- 172.21.15.1:0/1479585681 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fdb08fd9f0 msgr2=0x55fdb08f5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:02.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.397+0000 7f0a10e00700 1 --2- 172.21.15.1:0/1479585681 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fdb08fd9f0 0x55fdb08f5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:02.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.397+0000 7f0a10e00700 1 -- 172.21.15.1:0/1479585681 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fdb08f2b10 msgr2=0x55fdb08f4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:02.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.397+0000 7f0a10e00700 1 --2- 172.21.15.1:0/1479585681 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fdb08f2b10 0x55fdb08f4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:02.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.397+0000 7f0a10e00700 1 -- 172.21.15.1:0/1479585681 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fdb0671b60 con 0x55fdb08f21f0 2024-02-02T15:33:02.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.397+0000 7f0a10e00700 1 --2- 172.21.15.1:0/1479585681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fdb08f21f0 0x55fdb0904450 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f09fc009ce0 tx=0x7f09fc005fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66e42ae7ffead0d9 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:33:02.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.397+0000 7f0a02a00700 1 -- 172.21.15.1:0/1479585681 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09fc01d040 con 0x55fdb08f21f0 2024-02-02T15:33:02.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.397+0000 7f0a02a00700 1 -- 172.21.15.1:0/1479585681 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f09fc004b00 con 0x55fdb08f21f0 2024-02-02T15:33:02.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.397+0000 7f0a02a00700 1 -- 172.21.15.1:0/1479585681 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09fc00f460 con 0x55fdb08f21f0 2024-02-02T15:33:02.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.398+0000 7f0a273b4b80 1 -- 172.21.15.1:0/1479585681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fdb08f21f0 msgr2=0x55fdb0904450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:02.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.398+0000 7f0a273b4b80 1 --2- 172.21.15.1:0/1479585681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fdb08f21f0 0x55fdb0904450 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f09fc009ce0 tx=0x7f09fc005fb0 comp rx=0 tx=0).stop 2024-02-02T15:33:02.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.398+0000 7f0a273b4b80 1 -- 172.21.15.1:0/1479585681 shutdown_connections 2024-02-02T15:33:02.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.398+0000 7f0a273b4b80 1 --2- 172.21.15.1:0/1479585681 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fdb08fd9f0 0x55fdb08f5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:02.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.398+0000 7f0a273b4b80 1 --2- 172.21.15.1:0/1479585681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fdb08f21f0 0x55fdb0904450 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:02.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.398+0000 7f0a273b4b80 1 --2- 172.21.15.1:0/1479585681 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fdb08f2b10 0x55fdb08f4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:02.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.398+0000 7f0a273b4b80 1 -- 172.21.15.1:0/1479585681 >> 172.21.15.1:0/1479585681 conn(0x55fdb0851db0 msgr2=0x55fdb0908a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:02.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.398+0000 7f0a273b4b80 1 -- 172.21.15.1:0/1479585681 shutdown_connections 2024-02-02T15:33:02.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.398+0000 7f0a273b4b80 1 -- 172.21.15.1:0/1479585681 wait complete. 2024-02-02T15:33:02.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.398+0000 7f0a273b4b80 1 -- start start 2024-02-02T15:33:02.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.399+0000 7f0a273b4b80 1 Processor -- start 2024-02-02T15:33:02.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.399+0000 7f0a273b4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fdb08f21f0 0x55fdb0903a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:02.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.399+0000 7f0a273b4b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fdb08f2b10 0x55fdb0903fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:02.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.399+0000 7f0a273b4b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fdb08fd9f0 0x55fdb0957950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:02.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.399+0000 7f0a273b4b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdb0957f00 con 0x55fdb08f2b10 2024-02-02T15:33:02.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.399+0000 7f0a273b4b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdb0958070 con 0x55fdb08fd9f0 2024-02-02T15:33:02.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.399+0000 7f0a273b4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdb0958230 con 0x55fdb08f21f0 2024-02-02T15:33:02.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.399+0000 7f0a273b4b80 10 client.0 _ll_register_callbacks cb 0x55fdb07e1290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:33:02.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.399+0000 7f0a11800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fdb08fd9f0 0x55fdb0957950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:02.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.399+0000 7f0a10e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fdb08f21f0 0x55fdb0903a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:02.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.399+0000 7f0a11800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fdb08fd9f0 0x55fdb0957950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:34024/0 (socket says 172.21.15.1:34024) 2024-02-02T15:33:02.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.399+0000 7f0a11800700 1 -- 172.21.15.1:0/3497581160 learned_addr learned my addr 172.21.15.1:0/3497581160 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:33:02.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.399+0000 7f0a0be00700 1 --2- 172.21.15.1:0/3497581160 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fdb08f2b10 0x55fdb0903fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:02.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.400+0000 7f0a11800700 1 -- 172.21.15.1:0/3497581160 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fdb08f21f0 msgr2=0x55fdb0903a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:02.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.400+0000 7f0a11800700 1 --2- 172.21.15.1:0/3497581160 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fdb08f21f0 0x55fdb0903a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:02.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.400+0000 7f0a11800700 1 -- 172.21.15.1:0/3497581160 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fdb08f2b10 msgr2=0x55fdb0903fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:02.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.400+0000 7f0a11800700 1 --2- 172.21.15.1:0/3497581160 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fdb08f2b10 0x55fdb0903fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:02.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.400+0000 7f0a11800700 1 -- 172.21.15.1:0/3497581160 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a0c114840 con 0x55fdb08fd9f0 2024-02-02T15:33:02.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.400+0000 7f0a11800700 1 --2- 172.21.15.1:0/3497581160 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fdb08fd9f0 0x55fdb0957950 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f0a040236b0 tx=0x7f0a04004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a802b883f55d8fb server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:33:02.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.400+0000 7f09faa00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:33:02.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.400+0000 7f09faa00700 1 -- 172.21.15.1:0/3497581160 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a04028070 con 0x55fdb08fd9f0 2024-02-02T15:33:02.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.400+0000 7f0a273b4b80 1 -- 172.21.15.1:0/3497581160 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f09fc01e040 con 0x55fdb08fd9f0 2024-02-02T15:33:02.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.400+0000 7f09faa00700 1 -- 172.21.15.1:0/3497581160 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f0a04023d00 con 0x55fdb08fd9f0 2024-02-02T15:33:02.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.401+0000 7f09faa00700 1 -- 172.21.15.1:0/3497581160 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a04016400 con 0x55fdb08fd9f0 2024-02-02T15:33:02.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.401+0000 7f0a273b4b80 10 client.27302 fetch_fsmap learned FSMap version 38 2024-02-02T15:33:02.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.401+0000 7f0a273b4b80 1 -- 172.21.15.1:0/3497581160 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55fdb0809ea0 con 0x55fdb08fd9f0 2024-02-02T15:33:02.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.401+0000 7f09faa00700 1 -- 172.21.15.1:0/3497581160 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f09fc01e040 con 0x55fdb08fd9f0 2024-02-02T15:33:02.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.401+0000 7f09faa00700 1 -- 172.21.15.1:0/3497581160 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 38) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f0a0402c5b0 con 0x55fdb08fd9f0 2024-02-02T15:33:02.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.401+0000 7f0a273b4b80 10 client.27302 fetch_fsmap finished waiting for FSMap version 38 2024-02-02T15:33:02.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.401+0000 7f0a273b4b80 10 client.27302 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:33:02.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.401+0000 7f0a273b4b80 20 client.27302 populate_metadata read hostname 'smithi001' 2024-02-02T15:33:02.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.401+0000 7f0a273b4b80 1 --2- 172.21.15.1:0/3497581160 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55fdb095fbc0 0x55fdb0961fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:02.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.401+0000 7f0a273b4b80 4 client.27302 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2024-02-02T15:33:02.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.401+0000 7f0a273b4b80 1 -- 172.21.15.1:0/3497581160 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55fdb0802f50 con 0x55fdb095fbc0 2024-02-02T15:33:02.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.401+0000 7f0a10e00700 1 --2- 172.21.15.1:0/3497581160 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55fdb095fbc0 0x55fdb0961fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:02.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.402+0000 7f0a10e00700 1 --2- 172.21.15.1:0/3497581160 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55fdb095fbc0 0x55fdb0961fe0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6cd3a423c22d54a2 server_cookie=9a2332ba1124416a in_seq=0 out_seq=0 2024-02-02T15:33:02.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.402+0000 7f09faa00700 10 client.27302 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:33:02.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.402+0000 7f09faa00700 1 -- 172.21.15.1:0/3497581160 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55fdb0802f50 con 0x55fdb095fbc0 2024-02-02T15:33:02.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.402+0000 7f09faa00700 10 client.27302 handle_command_reply: tid=0 2024-02-02T15:33:02.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.402+0000 7f0a273b4b80 1 client.27302 shutdown 2024-02-02T15:33:02.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.402+0000 7f0a273b4b80 1 -- 172.21.15.1:0/3497581160 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fdb08fd9f0 msgr2=0x55fdb0957950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:02.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.402+0000 7f0a273b4b80 1 --2- 172.21.15.1:0/3497581160 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fdb08fd9f0 0x55fdb0957950 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f0a040236b0 tx=0x7f0a04004b40 comp rx=0 tx=0).stop 2024-02-02T15:33:02.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.402+0000 7f0a273b4b80 1 -- 172.21.15.1:0/3497581160 shutdown_connections 2024-02-02T15:33:02.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.402+0000 7f0a273b4b80 1 --2- 172.21.15.1:0/3497581160 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55fdb095fbc0 0x55fdb0961fe0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:02.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.402+0000 7f0a273b4b80 1 --2- 172.21.15.1:0/3497581160 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fdb08fd9f0 0x55fdb0957950 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:02.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.402+0000 7f0a273b4b80 1 --2- 172.21.15.1:0/3497581160 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fdb08f21f0 0x55fdb0903a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:02.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.402+0000 7f0a273b4b80 1 --2- 172.21.15.1:0/3497581160 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fdb08f2b10 0x55fdb0903fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:02.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.402+0000 7f0a273b4b80 1 -- 172.21.15.1:0/3497581160 >> 172.21.15.1:0/3497581160 conn(0x55fdb0851db0 msgr2=0x55fdb08f5950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:02.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.402+0000 7f09faa00700 0 client.27302 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:33:02.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.403+0000 7f0a273b4b80 1 -- 172.21.15.1:0/3497581160 shutdown_connections 2024-02-02T15:33:02.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.403+0000 7f0a273b4b80 1 -- 172.21.15.1:0/3497581160 wait complete. 2024-02-02T15:33:02.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.403+0000 7f0a273b4b80 20 client.27302 trim_cache size 0 max 16384 2024-02-02T15:33:02.407 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:33:02.407 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-02T15:33:02.407 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2024-02-02T15:33:02.407 INFO:teuthology.orchestra.run.smithi001.stdout: "27022107-d9ae-4852-a23b-370fce62e779": { 2024-02-02T15:33:02.408 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2024-02-02T15:33:02.408 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "27022107-d9ae-4852-a23b-370fce62e779", 2024-02-02T15:33:02.408 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,force" 2024-02-02T15:33:02.408 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:33:02.408 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:33:02.408 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:33:02.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.403+0000 7f0a12200700 1 -- 172.21.15.1:0/3837234279 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f09ec0a0690 msgr2=0x7f09ec0a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:02.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.403+0000 7f0a12200700 1 --2- 172.21.15.1:0/3837234279 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f09ec0a0690 0x7f09ec0a2b50 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0a040079f0 tx=0x7f0a0400b040 comp rx=0 tx=0).stop 2024-02-02T15:33:02.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.403+0000 7f0a12200700 1 -- 172.21.15.1:0/3837234279 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a0c073a00 msgr2=0x7f0a0c19d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:02.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.403+0000 7f0a12200700 1 --2- 172.21.15.1:0/3837234279 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a0c073a00 0x7f0a0c19d760 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f09f400e9a0 tx=0x7f09f400ecb0 comp rx=0 tx=0).stop 2024-02-02T15:33:02.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.403+0000 7f0a12200700 1 -- 172.21.15.1:0/3837234279 shutdown_connections 2024-02-02T15:33:02.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.403+0000 7f0a12200700 1 --2- 172.21.15.1:0/3837234279 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f09ec0a0690 0x7f09ec0a2b50 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:02.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.403+0000 7f0a12200700 1 --2- 172.21.15.1:0/3837234279 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0a0c111aa0 0x7f0a0c1a1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:02.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.403+0000 7f0a12200700 1 --2- 172.21.15.1:0/3837234279 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0a0c0730e0 0x7f0a0c19d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:02.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.403+0000 7f0a12200700 1 --2- 172.21.15.1:0/3837234279 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0a0c073a00 0x7f0a0c19d760 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:02.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.403+0000 7f0a12200700 1 -- 172.21.15.1:0/3837234279 >> 172.21.15.1:0/3837234279 conn(0x7f0a0c0fc580 msgr2=0x7f0a0c0fd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:02.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.403+0000 7f0a12200700 1 -- 172.21.15.1:0/3837234279 shutdown_connections 2024-02-02T15:33:02.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:02.403+0000 7f0a12200700 1 -- 172.21.15.1:0/3837234279 wait complete. 2024-02-02T15:33:02.421 INFO:tasks.cephfs.filesystem:scrub status for tag:27022107-d9ae-4852-a23b-370fce62e779 - {'path': '/', 'tag': '27022107-d9ae-4852-a23b-370fce62e779', 'options': 'recursive,force'} 2024-02-02T15:33:03.237 INFO:tasks.workunit.client.0.smithi001.stdout: 1 397759 8.85 MB/sec execute 157 sec latency 34.817 ms 2024-02-02T15:33:03.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:03 smithi001 ceph-mon[35470]: scrub queued for path: / 2024-02-02T15:33:03.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:03 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:33:03.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:03 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:33:03.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:03 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:33:03.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:03 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:33:03.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:03 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:33:03.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:03 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:33:03.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:03 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:33:03.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:03 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:33:03.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:03 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:33:03.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:03 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:33:03.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:03 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:33:03.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:03 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:33:03.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:03 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:33:03.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:03 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:33:03.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:03 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:33:03.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:03 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:33:03.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:03 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:33:03.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:03 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:33:03.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:03 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:33:03.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:03 smithi001 ceph-mon[35470]: pgmap v184: 129 pgs: 129 active+clean; 245 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.5 MiB/s wr, 335 op/s 2024-02-02T15:33:03.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:03 smithi071 ceph-mon[44707]: scrub queued for path: / 2024-02-02T15:33:03.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:03 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:33:03.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:03 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:33:03.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:03 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:33:03.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:03 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:33:03.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:03 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:33:03.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:03 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:33:03.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:03 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:33:03.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:03 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:33:03.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:03 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:33:03.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:03 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:33:03.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:03 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:33:03.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:03 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:33:03.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:03 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:33:03.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:03 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:33:03.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:03 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:33:03.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:03 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:33:03.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:03 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:33:03.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:03 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:33:03.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:03 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:33:03.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:03 smithi071 ceph-mon[44707]: pgmap v184: 129 pgs: 129 active+clean; 245 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.5 MiB/s wr, 335 op/s 2024-02-02T15:33:03.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:03 smithi142 ceph-mon[45728]: scrub queued for path: / 2024-02-02T15:33:03.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:03 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:33:03.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:03 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:33:03.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:03 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:33:03.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:03 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:33:03.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:03 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:33:03.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:03 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:33:03.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:03 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:33:03.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:03 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:33:03.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:03 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:33:03.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:03 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:33:03.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:03 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:33:03.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:03 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:33:03.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:03 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:33:03.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:03 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:33:03.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:03 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:33:03.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:03 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:33:03.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:03 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:33:03.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:03 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:33:03.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:03 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:33:03.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:03 smithi142 ceph-mon[45728]: pgmap v184: 129 pgs: 129 active+clean; 245 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.5 MiB/s wr, 335 op/s 2024-02-02T15:33:04.237 INFO:tasks.workunit.client.0.smithi001.stdout: 1 399085 8.82 MB/sec execute 158 sec latency 27.824 ms 2024-02-02T15:33:05.237 INFO:tasks.workunit.client.0.smithi001.stdout: 1 401095 8.85 MB/sec execute 159 sec latency 15.160 ms 2024-02-02T15:33:05.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:05 smithi001 ceph-mon[35470]: pgmap v185: 129 pgs: 129 active+clean; 245 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 232 op/s 2024-02-02T15:33:05.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:05 smithi071 ceph-mon[44707]: pgmap v185: 129 pgs: 129 active+clean; 245 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 232 op/s 2024-02-02T15:33:05.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:05 smithi142 ceph-mon[45728]: pgmap v185: 129 pgs: 129 active+clean; 245 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 232 op/s 2024-02-02T15:33:06.237 INFO:tasks.workunit.client.0.smithi001.stdout: 1 402268 8.83 MB/sec execute 160 sec latency 17.605 ms 2024-02-02T15:33:07.237 INFO:tasks.workunit.client.0.smithi001.stdout: 1 402518 8.78 MB/sec execute 161 sec latency 793.219 ms 2024-02-02T15:33:07.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:07 smithi001 ceph-mon[35470]: pgmap v186: 129 pgs: 129 active+clean; 251 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.5 MiB/s wr, 342 op/s 2024-02-02T15:33:07.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:07 smithi071 ceph-mon[44707]: pgmap v186: 129 pgs: 129 active+clean; 251 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.5 MiB/s wr, 342 op/s 2024-02-02T15:33:07.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:07 smithi142 ceph-mon[45728]: pgmap v186: 129 pgs: 129 active+clean; 251 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.5 MiB/s wr, 342 op/s 2024-02-02T15:33:08.239 INFO:tasks.workunit.client.0.smithi001.stdout: 1 402518 8.72 MB/sec execute 162 sec latency 1793.505 ms 2024-02-02T15:33:09.238 INFO:tasks.workunit.client.0.smithi001.stdout: 1 402518 8.67 MB/sec execute 163 sec latency 2793.779 ms 2024-02-02T15:33:09.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:09 smithi001 ceph-mon[35470]: pgmap v187: 129 pgs: 129 active+clean; 251 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 237 op/s 2024-02-02T15:33:09.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:09 smithi071 ceph-mon[44707]: pgmap v187: 129 pgs: 129 active+clean; 251 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 237 op/s 2024-02-02T15:33:09.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:09 smithi142 ceph-mon[45728]: pgmap v187: 129 pgs: 129 active+clean; 251 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 237 op/s 2024-02-02T15:33:10.238 INFO:tasks.workunit.client.0.smithi001.stdout: 1 402518 8.62 MB/sec execute 164 sec latency 3793.872 ms 2024-02-02T15:33:11.238 INFO:tasks.workunit.client.0.smithi001.stdout: 1 402518 8.56 MB/sec execute 165 sec latency 4794.020 ms 2024-02-02T15:33:11.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:11 smithi001 ceph-mon[35470]: pgmap v188: 129 pgs: 129 active+clean; 251 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 237 op/s 2024-02-02T15:33:11.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:11 smithi071 ceph-mon[44707]: pgmap v188: 129 pgs: 129 active+clean; 251 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 237 op/s 2024-02-02T15:33:11.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:11 smithi142 ceph-mon[45728]: pgmap v188: 129 pgs: 129 active+clean; 251 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.4 MiB/s wr, 237 op/s 2024-02-02T15:33:12.238 INFO:tasks.workunit.client.0.smithi001.stdout: 1 402518 8.51 MB/sec execute 166 sec latency 5794.161 ms 2024-02-02T15:33:12.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:12 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:33:12.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:12 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:33:12.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:12 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:33:12.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:12 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:33:12.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:12 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:33:12.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:12 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:33:12.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:12 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:33:12.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:12 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:33:12.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:12 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:33:12.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:12 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:33:12.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:12 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:33:12.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:12 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:33:12.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:12 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:33:12.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:12 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:33:12.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:12 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:33:13.238 INFO:tasks.workunit.client.0.smithi001.stdout: 1 403757 8.50 MB/sec execute 167 sec latency 6197.907 ms 2024-02-02T15:33:13.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:13 smithi071 ceph-mon[44707]: pgmap v189: 129 pgs: 129 active+clean; 256 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.1 MiB/s wr, 269 op/s 2024-02-02T15:33:13.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:13 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:33:13.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:13 smithi142 ceph-mon[45728]: pgmap v189: 129 pgs: 129 active+clean; 256 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.1 MiB/s wr, 269 op/s 2024-02-02T15:33:13.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:13 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:33:13.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:13 smithi001 ceph-mon[35470]: pgmap v189: 129 pgs: 129 active+clean; 256 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.1 MiB/s wr, 269 op/s 2024-02-02T15:33:13.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:13 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:33:14.238 INFO:tasks.workunit.client.0.smithi001.stdout: 1 404867 8.49 MB/sec execute 168 sec latency 32.799 ms 2024-02-02T15:33:14.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:14 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:33:14.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:14 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:33:14.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:14 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:33:14.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:14 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:33:14.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:14 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:33:14.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:14 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:33:14.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:14 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:33:14.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:14 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:33:14.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:14 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:33:15.238 INFO:tasks.workunit.client.0.smithi001.stdout: 1 406292 8.48 MB/sec execute 169 sec latency 21.448 ms 2024-02-02T15:33:15.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:15 smithi071 ceph-mon[44707]: pgmap v190: 129 pgs: 129 active+clean; 256 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 141 op/s 2024-02-02T15:33:15.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:15 smithi142 ceph-mon[45728]: pgmap v190: 129 pgs: 129 active+clean; 256 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 141 op/s 2024-02-02T15:33:15.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:15 smithi001 ceph-mon[35470]: pgmap v190: 129 pgs: 129 active+clean; 256 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 141 op/s 2024-02-02T15:33:16.238 INFO:tasks.workunit.client.0.smithi001.stdout: 1 408201 8.50 MB/sec execute 170 sec latency 15.044 ms 2024-02-02T15:33:17.239 INFO:tasks.workunit.client.0.smithi001.stdout: 1 409181 8.47 MB/sec execute 171 sec latency 60.520 ms 2024-02-02T15:33:17.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:17 smithi071 ceph-mon[44707]: pgmap v191: 129 pgs: 129 active+clean; 257 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.1 MiB/s wr, 228 op/s 2024-02-02T15:33:17.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:17 smithi142 ceph-mon[45728]: pgmap v191: 129 pgs: 129 active+clean; 257 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.1 MiB/s wr, 228 op/s 2024-02-02T15:33:17.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:17 smithi001 ceph-mon[35470]: pgmap v191: 129 pgs: 129 active+clean; 257 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 3.1 MiB/s wr, 228 op/s 2024-02-02T15:33:18.239 INFO:tasks.workunit.client.0.smithi001.stdout: 1 410427 8.45 MB/sec execute 172 sec latency 17.780 ms 2024-02-02T15:33:19.239 INFO:tasks.workunit.client.0.smithi001.stdout: 1 411857 8.47 MB/sec execute 173 sec latency 44.633 ms 2024-02-02T15:33:19.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:19 smithi071 ceph-mon[44707]: pgmap v192: 129 pgs: 129 active+clean; 257 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 118 op/s 2024-02-02T15:33:19.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:19 smithi142 ceph-mon[45728]: pgmap v192: 129 pgs: 129 active+clean; 257 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 118 op/s 2024-02-02T15:33:19.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:19 smithi001 ceph-mon[35470]: pgmap v192: 129 pgs: 129 active+clean; 257 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 118 op/s 2024-02-02T15:33:20.239 INFO:tasks.workunit.client.0.smithi001.stdout: 1 413021 8.45 MB/sec execute 174 sec latency 43.003 ms 2024-02-02T15:33:21.239 INFO:tasks.workunit.client.0.smithi001.stdout: 1 415126 8.47 MB/sec execute 175 sec latency 16.125 ms 2024-02-02T15:33:21.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:21 smithi071 ceph-mon[44707]: pgmap v193: 129 pgs: 129 active+clean; 257 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 118 op/s 2024-02-02T15:33:21.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:21 smithi142 ceph-mon[45728]: pgmap v193: 129 pgs: 129 active+clean; 257 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 118 op/s 2024-02-02T15:33:21.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:21 smithi001 ceph-mon[35470]: pgmap v193: 129 pgs: 129 active+clean; 257 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 118 op/s 2024-02-02T15:33:22.239 INFO:tasks.workunit.client.0.smithi001.stdout: 1 416030 8.44 MB/sec execute 176 sec latency 17.498 ms 2024-02-02T15:33:23.239 INFO:tasks.workunit.client.0.smithi001.stdout: 1 416732 8.42 MB/sec execute 177 sec latency 316.411 ms 2024-02-02T15:33:23.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:23 smithi071 ceph-mon[44707]: pgmap v194: 129 pgs: 129 active+clean; 263 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 215 op/s 2024-02-02T15:33:23.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:23 smithi142 ceph-mon[45728]: pgmap v194: 129 pgs: 129 active+clean; 263 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 215 op/s 2024-02-02T15:33:23.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:23 smithi001 ceph-mon[35470]: pgmap v194: 129 pgs: 129 active+clean; 263 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 215 op/s 2024-02-02T15:33:24.239 INFO:tasks.workunit.client.0.smithi001.stdout: 1 416732 8.37 MB/sec execute 178 sec latency 1316.540 ms 2024-02-02T15:33:25.239 INFO:tasks.workunit.client.0.smithi001.stdout: 1 418062 8.36 MB/sec execute 179 sec latency 1630.974 ms 2024-02-02T15:33:25.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:25 smithi071 ceph-mon[44707]: pgmap v195: 129 pgs: 129 active+clean; 263 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.3 MiB/s wr, 183 op/s 2024-02-02T15:33:25.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:25 smithi142 ceph-mon[45728]: pgmap v195: 129 pgs: 129 active+clean; 263 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.3 MiB/s wr, 183 op/s 2024-02-02T15:33:25.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:25 smithi001 ceph-mon[35470]: pgmap v195: 129 pgs: 129 active+clean; 263 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.3 MiB/s wr, 183 op/s 2024-02-02T15:33:26.239 INFO:tasks.workunit.client.0.smithi001.stdout: 1 419294 8.35 MB/sec execute 180 sec latency 17.091 ms 2024-02-02T15:33:27.239 INFO:tasks.workunit.client.0.smithi001.stdout: 1 420275 8.34 MB/sec execute 181 sec latency 79.711 ms 2024-02-02T15:33:27.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:27 smithi071 ceph-mon[44707]: pgmap v196: 129 pgs: 129 active+clean; 266 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.2 MiB/s wr, 261 op/s 2024-02-02T15:33:27.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:27 smithi142 ceph-mon[45728]: pgmap v196: 129 pgs: 129 active+clean; 266 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.2 MiB/s wr, 261 op/s 2024-02-02T15:33:27.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:27 smithi001 ceph-mon[35470]: pgmap v196: 129 pgs: 129 active+clean; 266 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.2 MiB/s wr, 261 op/s 2024-02-02T15:33:28.240 INFO:tasks.workunit.client.0.smithi001.stdout: 1 420275 8.29 MB/sec execute 182 sec latency 1079.794 ms 2024-02-02T15:33:29.239 INFO:tasks.workunit.client.0.smithi001.stdout: 1 420275 8.24 MB/sec execute 183 sec latency 2079.874 ms 2024-02-02T15:33:29.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:29 smithi071 ceph-mon[44707]: pgmap v197: 129 pgs: 129 active+clean; 266 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 174 op/s 2024-02-02T15:33:29.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:29 smithi142 ceph-mon[45728]: pgmap v197: 129 pgs: 129 active+clean; 266 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 174 op/s 2024-02-02T15:33:29.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:29 smithi001 ceph-mon[35470]: pgmap v197: 129 pgs: 129 active+clean; 266 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 174 op/s 2024-02-02T15:33:30.240 INFO:tasks.workunit.client.0.smithi001.stdout: 1 421749 8.24 MB/sec execute 184 sec latency 2388.958 ms 2024-02-02T15:33:31.240 INFO:tasks.workunit.client.0.smithi001.stdout: 1 422842 8.23 MB/sec execute 185 sec latency 20.237 ms 2024-02-02T15:33:31.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:31 smithi071 ceph-mon[44707]: pgmap v198: 129 pgs: 129 active+clean; 266 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 174 op/s 2024-02-02T15:33:31.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:31 smithi142 ceph-mon[45728]: pgmap v198: 129 pgs: 129 active+clean; 266 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 174 op/s 2024-02-02T15:33:31.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:31 smithi001 ceph-mon[35470]: pgmap v198: 129 pgs: 129 active+clean; 266 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 174 op/s 2024-02-02T15:33:32.240 INFO:tasks.workunit.client.0.smithi001.stdout: 1 423824 8.21 MB/sec execute 186 sec latency 65.314 ms 2024-02-02T15:33:32.422 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-02T15:33:32.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.645+0000 7f0ffa600700 1 -- 172.21.15.1:0/1304354129 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ff4100ff0 msgr2=0x7f0ff41013d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:32.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.645+0000 7f0ffa600700 1 --2- 172.21.15.1:0/1304354129 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ff4100ff0 0x7f0ff41013d0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f0fe4009990 tx=0x7f0fe4009ca0 comp rx=0 tx=0).stop 2024-02-02T15:33:32.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.645+0000 7f0ffa600700 1 -- 172.21.15.1:0/1304354129 shutdown_connections 2024-02-02T15:33:32.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.645+0000 7f0ffa600700 1 --2- 172.21.15.1:0/1304354129 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ff4106110 0x7f0ff410dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:32.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.645+0000 7f0ffa600700 1 --2- 172.21.15.1:0/1304354129 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ff4100ff0 0x7f0ff41013d0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:32.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.645+0000 7f0ffa600700 1 --2- 172.21.15.1:0/1304354129 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ff41019a0 0x7f0ff41059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:32.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.645+0000 7f0ffa600700 1 -- 172.21.15.1:0/1304354129 >> 172.21.15.1:0/1304354129 conn(0x7f0ff40fc810 msgr2=0x7f0ff40fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:32.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.645+0000 7f0ffa600700 1 -- 172.21.15.1:0/1304354129 shutdown_connections 2024-02-02T15:33:32.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.645+0000 7f0ffa600700 1 -- 172.21.15.1:0/1304354129 wait complete. 2024-02-02T15:33:32.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.646+0000 7f0ffa600700 1 Processor -- start 2024-02-02T15:33:32.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.646+0000 7f0ffa600700 1 -- start start 2024-02-02T15:33:32.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.647+0000 7f0ffa600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ff4100ff0 0x7f0ff4199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:32.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.647+0000 7f0ffa600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ff4106110 0x7f0ff4199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:32.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.647+0000 7f0ffa600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ff419daa0 0x7f0ff419df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:32.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.647+0000 7f0ffa600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff419e520 con 0x7f0ff4106110 2024-02-02T15:33:32.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.647+0000 7f0ffa600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff419e690 con 0x7f0ff419daa0 2024-02-02T15:33:32.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.647+0000 7f0ffa600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff419e850 con 0x7f0ff4100ff0 2024-02-02T15:33:32.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.647+0000 7f0ff2a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ff4106110 0x7f0ff4199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:32.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.647+0000 7f0ff2a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ff4106110 0x7f0ff4199640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37210/0 (socket says 172.21.15.1:37210) 2024-02-02T15:33:32.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.647+0000 7f0ff2a00700 1 -- 172.21.15.1:0/689325987 learned_addr learned my addr 172.21.15.1:0/689325987 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:33:32.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.647+0000 7f0ff3e00700 1 --2- 172.21.15.1:0/689325987 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ff419daa0 0x7f0ff419df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:32.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.647+0000 7f0ff3400700 1 --2- 172.21.15.1:0/689325987 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ff4100ff0 0x7f0ff4199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:32.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.647+0000 7f0ff2a00700 1 -- 172.21.15.1:0/689325987 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ff4100ff0 msgr2=0x7f0ff4199100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:32.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.647+0000 7f0ff2a00700 1 --2- 172.21.15.1:0/689325987 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ff4100ff0 0x7f0ff4199100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:32.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.647+0000 7f0ff2a00700 1 -- 172.21.15.1:0/689325987 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ff419daa0 msgr2=0x7f0ff419df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:32.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.647+0000 7f0ff2a00700 1 --2- 172.21.15.1:0/689325987 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ff419daa0 0x7f0ff419df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:32.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.647+0000 7f0ff2a00700 1 -- 172.21.15.1:0/689325987 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fe40095f0 con 0x7f0ff4106110 2024-02-02T15:33:32.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.647+0000 7f0ff3e00700 1 --2- 172.21.15.1:0/689325987 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ff419daa0 0x7f0ff419df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:33:32.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.648+0000 7f0ff2a00700 1 --2- 172.21.15.1:0/689325987 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ff4106110 0x7f0ff4199640 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc00e970 tx=0x7f0fdc00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:33:32.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.648+0000 7f0febe00700 1 -- 172.21.15.1:0/689325987 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fdc009790 con 0x7f0ff4106110 2024-02-02T15:33:32.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.648+0000 7f0febe00700 1 -- 172.21.15.1:0/689325987 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0fdc00c7e0 con 0x7f0ff4106110 2024-02-02T15:33:32.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.648+0000 7f0ffa600700 1 -- 172.21.15.1:0/689325987 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ff41a5db0 con 0x7f0ff4106110 2024-02-02T15:33:32.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.648+0000 7f0febe00700 1 -- 172.21.15.1:0/689325987 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fdc0054e0 con 0x7f0ff4106110 2024-02-02T15:33:32.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.648+0000 7f0ffa600700 1 -- 172.21.15.1:0/689325987 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ff41a61e0 con 0x7f0ff4106110 2024-02-02T15:33:32.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.651+0000 7f0febe00700 1 -- 172.21.15.1:0/689325987 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 27) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f0fdc00c950 con 0x7f0ff4106110 2024-02-02T15:33:32.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.652+0000 7f0febe00700 1 --2- 172.21.15.1:0/689325987 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f0fd40a0690 0x7f0fd40a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:32.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.652+0000 7f100e206b80 1 Processor -- start 2024-02-02T15:33:32.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.652+0000 7f0febe00700 1 -- 172.21.15.1:0/689325987 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f0fdc014070 con 0x7f0ff4106110 2024-02-02T15:33:32.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.653+0000 7f100e206b80 1 -- start start 2024-02-02T15:33:32.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.653+0000 7f0ff3400700 1 --2- 172.21.15.1:0/689325987 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f0fd40a0690 0x7f0fd40a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:32.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.653+0000 7f100e206b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5606f0fe9010 0x5606f0fe93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:32.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.653+0000 7f100e206b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5606f0fe9930 0x5606f0ff42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:32.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.653+0000 7f100e206b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606f0ff47e0 0x5606f0ff6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:32.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.653+0000 7f100e206b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606f0f5d4c0 con 0x5606f0fe9930 2024-02-02T15:33:32.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.653+0000 7f100e206b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606f0f41aa0 con 0x5606f0ff47e0 2024-02-02T15:33:32.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.653+0000 7f100e206b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606f0f3f460 con 0x5606f0fe9010 2024-02-02T15:33:32.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.653+0000 7f0ff2a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5606f0fe9930 0x5606f0ff42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:32.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.653+0000 7f0ff2a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5606f0fe9930 0x5606f0ff42a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37216/0 (socket says 172.21.15.1:37216) 2024-02-02T15:33:32.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.653+0000 7f0ff2a00700 1 -- 172.21.15.1:0/1980096752 learned_addr learned my addr 172.21.15.1:0/1980096752 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:33:32.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.653+0000 7f0ff3400700 1 --2- 172.21.15.1:0/689325987 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f0fd40a0690 0x7f0fd40a2b50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0fe400b2e0 tx=0x7f0fe4005fd0 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:33:32.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.653+0000 7f0ff3e00700 1 --2- 172.21.15.1:0/1980096752 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606f0ff47e0 0x5606f0ff6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:32.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.654+0000 7f0ff3400700 1 --2- 172.21.15.1:0/1980096752 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5606f0fe9010 0x5606f0fe93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:32.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.654+0000 7f0ff3e00700 1 -- 172.21.15.1:0/1980096752 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5606f0fe9010 msgr2=0x5606f0fe93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:32.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.654+0000 7f0ff3e00700 1 --2- 172.21.15.1:0/1980096752 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5606f0fe9010 0x5606f0fe93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:32.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.654+0000 7f0ff3e00700 1 -- 172.21.15.1:0/1980096752 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5606f0fe9930 msgr2=0x5606f0ff42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:32.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.654+0000 7f0ff3e00700 1 --2- 172.21.15.1:0/1980096752 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5606f0fe9930 0x5606f0ff42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:32.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.654+0000 7f0ff3e00700 1 -- 172.21.15.1:0/1980096752 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5606f0dbc3b0 con 0x5606f0ff47e0 2024-02-02T15:33:32.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.654+0000 7f0ff2a00700 1 --2- 172.21.15.1:0/1980096752 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5606f0fe9930 0x5606f0ff42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:33:32.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.654+0000 7f0ff3e00700 1 --2- 172.21.15.1:0/1980096752 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606f0ff47e0 0x5606f0ff6bd0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f0fec003de0 tx=0x7f0fec009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce1faacac990a39a server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:33:32.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.654+0000 7f0fe8c00700 1 -- 172.21.15.1:0/1980096752 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fec004bb0 con 0x5606f0ff47e0 2024-02-02T15:33:32.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.654+0000 7f0fe8c00700 1 -- 172.21.15.1:0/1980096752 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0fec00f040 con 0x5606f0ff47e0 2024-02-02T15:33:32.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.654+0000 7f0fe8c00700 1 -- 172.21.15.1:0/1980096752 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fec007c20 con 0x5606f0ff47e0 2024-02-02T15:33:32.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.655+0000 7f100e206b80 1 -- 172.21.15.1:0/1980096752 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606f0ff47e0 msgr2=0x5606f0ff6bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:32.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.655+0000 7f100e206b80 1 --2- 172.21.15.1:0/1980096752 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606f0ff47e0 0x5606f0ff6bd0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f0fec003de0 tx=0x7f0fec009920 comp rx=0 tx=0).stop 2024-02-02T15:33:32.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.655+0000 7f100e206b80 1 -- 172.21.15.1:0/1980096752 shutdown_connections 2024-02-02T15:33:32.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.655+0000 7f100e206b80 1 --2- 172.21.15.1:0/1980096752 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606f0ff47e0 0x5606f0ff6bd0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:32.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.655+0000 7f100e206b80 1 --2- 172.21.15.1:0/1980096752 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5606f0fe9010 0x5606f0fe93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:32.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.655+0000 7f100e206b80 1 --2- 172.21.15.1:0/1980096752 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5606f0fe9930 0x5606f0ff42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:32.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.655+0000 7f100e206b80 1 -- 172.21.15.1:0/1980096752 >> 172.21.15.1:0/1980096752 conn(0x5606f0f48db0 msgr2=0x5606f0fe53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:32.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.655+0000 7f100e206b80 1 -- 172.21.15.1:0/1980096752 shutdown_connections 2024-02-02T15:33:32.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.655+0000 7f100e206b80 1 -- 172.21.15.1:0/1980096752 wait complete. 2024-02-02T15:33:32.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.656+0000 7f100e206b80 1 -- start start 2024-02-02T15:33:32.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.656+0000 7f100e206b80 1 Processor -- start 2024-02-02T15:33:32.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.656+0000 7f100e206b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5606f0fe9010 0x5606f0ff41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:32.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.656+0000 7f100e206b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606f0fe9930 0x5606f0ff1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:32.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.656+0000 7f100e206b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5606f0ff47e0 0x5606f0ff1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:32.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.656+0000 7f100e206b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606f0f3ad50 con 0x5606f0fe9010 2024-02-02T15:33:32.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.656+0000 7f100e206b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606f0ed6d80 con 0x5606f0fe9930 2024-02-02T15:33:32.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.656+0000 7f100e206b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606f0ed6a20 con 0x5606f0ff47e0 2024-02-02T15:33:32.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.656+0000 7f100e206b80 10 client.0 _ll_register_callbacks cb 0x5606f0ed8290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:33:32.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.657+0000 7f0ff3400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5606f0fe9010 0x5606f0ff41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:32.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.657+0000 7f0ff3e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5606f0ff47e0 0x5606f0ff1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:32.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.657+0000 7f0ff3400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5606f0fe9010 0x5606f0ff41f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37218/0 (socket says 172.21.15.1:37218) 2024-02-02T15:33:32.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.657+0000 7f0ff3400700 1 -- 172.21.15.1:0/3685609333 learned_addr learned my addr 172.21.15.1:0/3685609333 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:33:32.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.657+0000 7f0ff2a00700 1 --2- 172.21.15.1:0/3685609333 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606f0fe9930 0x5606f0ff1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:32.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.657+0000 7f0ff3400700 1 -- 172.21.15.1:0/3685609333 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5606f0ff47e0 msgr2=0x5606f0ff1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:32.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.657+0000 7f0ff3400700 1 --2- 172.21.15.1:0/3685609333 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5606f0ff47e0 0x5606f0ff1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:32.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.657+0000 7f0ff3400700 1 -- 172.21.15.1:0/3685609333 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606f0fe9930 msgr2=0x5606f0ff1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:32.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.657+0000 7f0ff3400700 1 --2- 172.21.15.1:0/3685609333 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606f0fe9930 0x5606f0ff1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:32.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.657+0000 7f0ff3400700 1 -- 172.21.15.1:0/3685609333 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ff41a61e0 con 0x5606f0fe9010 2024-02-02T15:33:32.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.657+0000 7f0ff3e00700 1 --2- 172.21.15.1:0/3685609333 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5606f0ff47e0 0x5606f0ff1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:33:32.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.657+0000 7f0ff3400700 1 --2- 172.21.15.1:0/3685609333 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5606f0fe9010 0x5606f0ff41f0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f0fe400bf20 tx=0x7f0fe401a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49216a323d019a10 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:33:32.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.657+0000 7f0fe1600700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:33:32.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.658+0000 7f0fe1600700 1 -- 172.21.15.1:0/3685609333 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe40056b0 con 0x5606f0fe9010 2024-02-02T15:33:32.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.658+0000 7f0fe1600700 1 -- 172.21.15.1:0/3685609333 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f0fe400f460 con 0x5606f0fe9010 2024-02-02T15:33:32.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.658+0000 7f100e206b80 1 -- 172.21.15.1:0/3685609333 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5606f0fdd0f0 con 0x5606f0fe9010 2024-02-02T15:33:32.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.658+0000 7f0fe1600700 1 -- 172.21.15.1:0/3685609333 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0fe400f620 con 0x5606f0fe9010 2024-02-02T15:33:32.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.658+0000 7f100e206b80 10 client.17253 fetch_fsmap learned FSMap version 38 2024-02-02T15:33:32.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.658+0000 7f100e206b80 1 -- 172.21.15.1:0/3685609333 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x7f0fec003a60 con 0x5606f0fe9010 2024-02-02T15:33:32.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.658+0000 7f0fe1600700 1 -- 172.21.15.1:0/3685609333 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5606f0fdd0f0 con 0x5606f0fe9010 2024-02-02T15:33:32.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.658+0000 7f0fe1600700 1 -- 172.21.15.1:0/3685609333 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 38) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f0fe402f850 con 0x5606f0fe9010 2024-02-02T15:33:32.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.658+0000 7f100e206b80 10 client.17253 fetch_fsmap finished waiting for FSMap version 38 2024-02-02T15:33:32.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.658+0000 7f100e206b80 10 client.17253 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:33:32.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.658+0000 7f100e206b80 20 client.17253 populate_metadata read hostname 'smithi001' 2024-02-02T15:33:32.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.658+0000 7f100e206b80 1 --2- 172.21.15.1:0/3685609333 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5606f10041e0 0x5606f1006600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:32.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.658+0000 7f100e206b80 4 client.17253 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:33:32.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.658+0000 7f100e206b80 1 -- 172.21.15.1:0/3685609333 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5606f0f33dd0 con 0x5606f10041e0 2024-02-02T15:33:32.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.659+0000 7f0ff2a00700 1 --2- 172.21.15.1:0/3685609333 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5606f10041e0 0x5606f1006600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:32.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.659+0000 7f0ff2a00700 1 --2- 172.21.15.1:0/3685609333 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5606f10041e0 0x5606f1006600 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f126628bd989e9f7 server_cookie=d7db6920ac3bb472 in_seq=0 out_seq=0 2024-02-02T15:33:32.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.659+0000 7f0fe1600700 10 client.17253 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:33:32.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.660+0000 7f0fe1600700 1 -- 172.21.15.1:0/3685609333 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5606f0f33dd0 con 0x5606f10041e0 2024-02-02T15:33:32.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.660+0000 7f0fe1600700 10 client.17253 handle_command_reply: tid=0 2024-02-02T15:33:32.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.660+0000 7f100e206b80 1 client.17253 shutdown 2024-02-02T15:33:32.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.660+0000 7f100e206b80 1 -- 172.21.15.1:0/3685609333 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5606f0fe9010 msgr2=0x5606f0ff41f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:32.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.660+0000 7f100e206b80 1 --2- 172.21.15.1:0/3685609333 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5606f0fe9010 0x5606f0ff41f0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f0fe400bf20 tx=0x7f0fe401a050 comp rx=0 tx=0).stop 2024-02-02T15:33:32.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.660+0000 7f100e206b80 1 -- 172.21.15.1:0/3685609333 shutdown_connections 2024-02-02T15:33:32.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.660+0000 7f100e206b80 1 --2- 172.21.15.1:0/3685609333 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5606f10041e0 0x5606f1006600 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:32.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.660+0000 7f100e206b80 1 --2- 172.21.15.1:0/3685609333 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5606f0ff47e0 0x5606f0ff1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:32.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.660+0000 7f100e206b80 1 --2- 172.21.15.1:0/3685609333 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5606f0fe9010 0x5606f0ff41f0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:32.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.660+0000 7f0fe1600700 0 client.17253 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:33:32.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.660+0000 7f100e206b80 1 --2- 172.21.15.1:0/3685609333 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606f0fe9930 0x5606f0ff1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:32.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.660+0000 7f100e206b80 1 -- 172.21.15.1:0/3685609333 >> 172.21.15.1:0/3685609333 conn(0x5606f0f48db0 msgr2=0x5606f0fe4fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:32.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.660+0000 7f100e206b80 1 -- 172.21.15.1:0/3685609333 shutdown_connections 2024-02-02T15:33:32.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.660+0000 7f100e206b80 1 -- 172.21.15.1:0/3685609333 wait complete. 2024-02-02T15:33:32.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.660+0000 7f100e206b80 20 client.17253 trim_cache size 0 max 16384 2024-02-02T15:33:32.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.677+0000 7f100e206b80 1 Processor -- start 2024-02-02T15:33:32.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.677+0000 7f100e206b80 1 -- start start 2024-02-02T15:33:32.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.678+0000 7f100e206b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606f0fe9010 0x5606f0fe0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:32.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.678+0000 7f100e206b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5606f0fe9930 0x5606f0ff3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:32.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.678+0000 7f100e206b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5606f0ff47e0 0x5606f0fec7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:32.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.678+0000 7f100e206b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606f0dbe6a0 con 0x5606f0ff47e0 2024-02-02T15:33:32.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.678+0000 7f100e206b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606f10001f0 con 0x5606f0fe9010 2024-02-02T15:33:32.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.678+0000 7f100e206b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606f0ff20a0 con 0x5606f0fe9930 2024-02-02T15:33:32.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.678+0000 7f0ff3e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5606f0ff47e0 0x5606f0fec7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:32.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.678+0000 7f0ff2a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5606f0fe9930 0x5606f0ff3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:32.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.678+0000 7f0ff2a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5606f0fe9930 0x5606f0ff3d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:35346/0 (socket says 172.21.15.1:35346) 2024-02-02T15:33:32.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.678+0000 7f0ff2a00700 1 -- 172.21.15.1:0/1029122798 learned_addr learned my addr 172.21.15.1:0/1029122798 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:33:32.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.678+0000 7f0ff3400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606f0fe9010 0x5606f0fe0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:32.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.678+0000 7f0ff2a00700 1 -- 172.21.15.1:0/1029122798 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606f0fe9010 msgr2=0x5606f0fe0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:32.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.678+0000 7f0ff2a00700 1 --2- 172.21.15.1:0/1029122798 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606f0fe9010 0x5606f0fe0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:32.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.678+0000 7f0ff2a00700 1 -- 172.21.15.1:0/1029122798 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5606f0ff47e0 msgr2=0x5606f0fec7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:32.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.678+0000 7f0ff2a00700 1 --2- 172.21.15.1:0/1029122798 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5606f0ff47e0 0x5606f0fec7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:32.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.678+0000 7f0ff2a00700 1 -- 172.21.15.1:0/1029122798 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5606f0d68b60 con 0x5606f0fe9930 2024-02-02T15:33:32.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.679+0000 7f0ff2a00700 1 --2- 172.21.15.1:0/1029122798 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5606f0fe9930 0x5606f0ff3d20 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc004800 tx=0x7f0fdc0cb040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4233830b7f26932 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:33:32.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.679+0000 7f0fe9600700 1 -- 172.21.15.1:0/1029122798 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fdc0cc120 con 0x5606f0fe9930 2024-02-02T15:33:32.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.679+0000 7f0fe9600700 1 -- 172.21.15.1:0/1029122798 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0fdc0994c0 con 0x5606f0fe9930 2024-02-02T15:33:32.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.679+0000 7f0fe9600700 1 -- 172.21.15.1:0/1029122798 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fdc0978f0 con 0x5606f0fe9930 2024-02-02T15:33:32.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.680+0000 7f100e206b80 1 -- 172.21.15.1:0/1029122798 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5606f0fe9930 msgr2=0x5606f0ff3d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:32.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.680+0000 7f100e206b80 1 --2- 172.21.15.1:0/1029122798 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5606f0fe9930 0x5606f0ff3d20 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc004800 tx=0x7f0fdc0cb040 comp rx=0 tx=0).stop 2024-02-02T15:33:32.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.680+0000 7f100e206b80 1 -- 172.21.15.1:0/1029122798 shutdown_connections 2024-02-02T15:33:32.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.680+0000 7f100e206b80 1 --2- 172.21.15.1:0/1029122798 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5606f0ff47e0 0x5606f0fec7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:32.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.680+0000 7f100e206b80 1 --2- 172.21.15.1:0/1029122798 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606f0fe9010 0x5606f0fe0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:32.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.680+0000 7f100e206b80 1 --2- 172.21.15.1:0/1029122798 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5606f0fe9930 0x5606f0ff3d20 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:32.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.680+0000 7f100e206b80 1 -- 172.21.15.1:0/1029122798 >> 172.21.15.1:0/1029122798 conn(0x5606f0f48db0 msgr2=0x5606f0fe3b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:32.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.680+0000 7f100e206b80 1 -- 172.21.15.1:0/1029122798 shutdown_connections 2024-02-02T15:33:32.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.680+0000 7f100e206b80 1 -- 172.21.15.1:0/1029122798 wait complete. 2024-02-02T15:33:32.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.680+0000 7f100e206b80 1 -- start start 2024-02-02T15:33:32.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.680+0000 7f100e206b80 1 Processor -- start 2024-02-02T15:33:32.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.681+0000 7f100e206b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5606f0fe9010 0x5606f0ff9560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:32.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.681+0000 7f100e206b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606f0fe9930 0x5606f0ff9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:32.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.681+0000 7f100e206b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5606f0ff47e0 0x5606f104ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:32.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.681+0000 7f100e206b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606f104f210 con 0x5606f0fe9010 2024-02-02T15:33:32.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.681+0000 7f100e206b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606f104f380 con 0x5606f0fe9930 2024-02-02T15:33:32.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.681+0000 7f100e206b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606f104f540 con 0x5606f0ff47e0 2024-02-02T15:33:32.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.681+0000 7f100e206b80 10 client.0 _ll_register_callbacks cb 0x5606f0ed8290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:33:32.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.681+0000 7f0ff3400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5606f0fe9010 0x5606f0ff9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:32.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.681+0000 7f0ff3400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5606f0fe9010 0x5606f0ff9560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37240/0 (socket says 172.21.15.1:37240) 2024-02-02T15:33:32.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.681+0000 7f0ff3400700 1 -- 172.21.15.1:0/2253538328 learned_addr learned my addr 172.21.15.1:0/2253538328 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:33:32.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.681+0000 7f0ff2a00700 1 --2- 172.21.15.1:0/2253538328 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606f0fe9930 0x5606f0ff9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:32.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.681+0000 7f0ff3e00700 1 --2- 172.21.15.1:0/2253538328 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5606f0ff47e0 0x5606f104ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:32.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.682+0000 7f0ff3400700 1 -- 172.21.15.1:0/2253538328 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5606f0ff47e0 msgr2=0x5606f104ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:32.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.682+0000 7f0ff3400700 1 --2- 172.21.15.1:0/2253538328 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5606f0ff47e0 0x5606f104ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:32.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.682+0000 7f0ff3400700 1 -- 172.21.15.1:0/2253538328 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606f0fe9930 msgr2=0x5606f0ff9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:32.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.682+0000 7f0ff3400700 1 --2- 172.21.15.1:0/2253538328 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606f0fe9930 0x5606f0ff9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:32.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.682+0000 7f0ff3400700 1 -- 172.21.15.1:0/2253538328 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ff41a5db0 con 0x5606f0fe9010 2024-02-02T15:33:32.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.682+0000 7f0ff2a00700 1 --2- 172.21.15.1:0/2253538328 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606f0fe9930 0x5606f0ff9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:33:32.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.682+0000 7f0ff3400700 1 --2- 172.21.15.1:0/2253538328 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5606f0fe9010 0x5606f0ff9560 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f0fe4000c00 tx=0x7f0fe40057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b166953dc608a115 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:33:32.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.682+0000 7f0fe1600700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:33:32.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.682+0000 7f0fe1600700 1 -- 172.21.15.1:0/2253538328 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe402fea0 con 0x5606f0fe9010 2024-02-02T15:33:32.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.682+0000 7f0fe1600700 1 -- 172.21.15.1:0/2253538328 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f0fe402f620 con 0x5606f0fe9010 2024-02-02T15:33:32.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.682+0000 7f100e206b80 1 -- 172.21.15.1:0/2253538328 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0fec00f040 con 0x5606f0fe9010 2024-02-02T15:33:32.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.683+0000 7f0fe1600700 1 -- 172.21.15.1:0/2253538328 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0fe4004ac0 con 0x5606f0fe9010 2024-02-02T15:33:32.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.683+0000 7f100e206b80 10 client.17259 fetch_fsmap learned FSMap version 38 2024-02-02T15:33:32.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.683+0000 7f100e206b80 1 -- 172.21.15.1:0/2253538328 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x5606f0f00ea0 con 0x5606f0fe9010 2024-02-02T15:33:32.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.683+0000 7f0fe1600700 1 -- 172.21.15.1:0/2253538328 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0fec00f040 con 0x5606f0fe9010 2024-02-02T15:33:32.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.683+0000 7f0fe1600700 1 -- 172.21.15.1:0/2253538328 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 38) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f0fe4037070 con 0x5606f0fe9010 2024-02-02T15:33:32.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.683+0000 7f100e206b80 10 client.17259 fetch_fsmap finished waiting for FSMap version 38 2024-02-02T15:33:32.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.683+0000 7f100e206b80 10 client.17259 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:33:32.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.683+0000 7f100e206b80 20 client.17259 populate_metadata read hostname 'smithi001' 2024-02-02T15:33:32.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.683+0000 7f100e206b80 1 --2- 172.21.15.1:0/2253538328 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5606f1056910 0x5606f1058d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:32.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.683+0000 7f100e206b80 4 client.17259 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2024-02-02T15:33:32.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.683+0000 7f100e206b80 1 -- 172.21.15.1:0/2253538328 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5606f0ef9f50 con 0x5606f1056910 2024-02-02T15:33:32.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.683+0000 7f0ff2a00700 1 --2- 172.21.15.1:0/2253538328 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5606f1056910 0x5606f1058d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:32.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.684+0000 7f0ff2a00700 1 --2- 172.21.15.1:0/2253538328 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5606f1056910 0x5606f1058d30 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf94adf69602d22b server_cookie=70e5f73d4fffac54 in_seq=0 out_seq=0 2024-02-02T15:33:32.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.684+0000 7f0fe1600700 10 client.17259 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:33:32.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.684+0000 7f0fe1600700 1 -- 172.21.15.1:0/2253538328 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5606f0ef9f50 con 0x5606f1056910 2024-02-02T15:33:32.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.684+0000 7f0fe1600700 10 client.17259 handle_command_reply: tid=0 2024-02-02T15:33:32.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.684+0000 7f100e206b80 1 client.17259 shutdown 2024-02-02T15:33:32.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.684+0000 7f100e206b80 1 -- 172.21.15.1:0/2253538328 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5606f0fe9010 msgr2=0x5606f0ff9560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:32.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.684+0000 7f100e206b80 1 --2- 172.21.15.1:0/2253538328 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5606f0fe9010 0x5606f0ff9560 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f0fe4000c00 tx=0x7f0fe40057c0 comp rx=0 tx=0).stop 2024-02-02T15:33:32.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.684+0000 7f100e206b80 1 -- 172.21.15.1:0/2253538328 shutdown_connections 2024-02-02T15:33:32.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.684+0000 7f100e206b80 1 --2- 172.21.15.1:0/2253538328 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5606f1056910 0x5606f1058d30 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:32.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.684+0000 7f100e206b80 1 --2- 172.21.15.1:0/2253538328 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5606f0ff47e0 0x5606f104ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:32.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.684+0000 7f100e206b80 1 --2- 172.21.15.1:0/2253538328 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5606f0fe9010 0x5606f0ff9560 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:32.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.684+0000 7f0fe1600700 0 client.17259 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:33:32.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.684+0000 7f100e206b80 1 --2- 172.21.15.1:0/2253538328 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5606f0fe9930 0x5606f0ff9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:32.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.684+0000 7f100e206b80 1 -- 172.21.15.1:0/2253538328 >> 172.21.15.1:0/2253538328 conn(0x5606f0f48db0 msgr2=0x5606f0fec7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:32.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.684+0000 7f100e206b80 1 -- 172.21.15.1:0/2253538328 shutdown_connections 2024-02-02T15:33:32.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.684+0000 7f100e206b80 1 -- 172.21.15.1:0/2253538328 wait complete. 2024-02-02T15:33:32.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.685+0000 7f100e206b80 20 client.17259 trim_cache size 0 max 16384 2024-02-02T15:33:32.690 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:33:32.690 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2024-02-02T15:33:32.690 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2024-02-02T15:33:32.690 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:33:32.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.685+0000 7f0ffa600700 1 -- 172.21.15.1:0/689325987 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f0fd40a0690 msgr2=0x7f0fd40a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:32.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.685+0000 7f0ffa600700 1 --2- 172.21.15.1:0/689325987 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f0fd40a0690 0x7f0fd40a2b50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0fe400b2e0 tx=0x7f0fe4005fd0 comp rx=0 tx=0).stop 2024-02-02T15:33:32.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.685+0000 7f0ffa600700 1 -- 172.21.15.1:0/689325987 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ff4106110 msgr2=0x7f0ff4199640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:32.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.685+0000 7f0ffa600700 1 --2- 172.21.15.1:0/689325987 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ff4106110 0x7f0ff4199640 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc00e970 tx=0x7f0fdc00ec80 comp rx=0 tx=0).stop 2024-02-02T15:33:32.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.685+0000 7f0ffa600700 1 -- 172.21.15.1:0/689325987 shutdown_connections 2024-02-02T15:33:32.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.685+0000 7f0ffa600700 1 --2- 172.21.15.1:0/689325987 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f0fd40a0690 0x7f0fd40a2b50 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:32.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.685+0000 7f0ffa600700 1 --2- 172.21.15.1:0/689325987 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0ff419daa0 0x7f0ff419df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:32.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.685+0000 7f0ffa600700 1 --2- 172.21.15.1:0/689325987 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0ff4100ff0 0x7f0ff4199100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:32.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.685+0000 7f0ffa600700 1 --2- 172.21.15.1:0/689325987 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0ff4106110 0x7f0ff4199640 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:32.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.685+0000 7f0ffa600700 1 -- 172.21.15.1:0/689325987 >> 172.21.15.1:0/689325987 conn(0x7f0ff40fc810 msgr2=0x7f0ff410cf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:32.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.685+0000 7f0ffa600700 1 -- 172.21.15.1:0/689325987 shutdown_connections 2024-02-02T15:33:32.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.685+0000 7f0ffa600700 1 -- 172.21.15.1:0/689325987 wait complete. 2024-02-02T15:33:32.703 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-02T15:33:32.703 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T15:33:32.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.900+0000 7f1f1e800700 1 -- 172.21.15.1:0/2575284027 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f18106030 msgr2=0x7f1f1810dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:32.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.900+0000 7f1f1e800700 1 --2- 172.21.15.1:0/2575284027 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f18106030 0x7f1f1810dbb0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f1f1000b0a0 tx=0x7f1f1000b3b0 comp rx=0 tx=0).stop 2024-02-02T15:33:32.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.900+0000 7f1f1e800700 1 -- 172.21.15.1:0/2575284027 shutdown_connections 2024-02-02T15:33:32.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.900+0000 7f1f1e800700 1 --2- 172.21.15.1:0/2575284027 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f18106030 0x7f1f1810dbb0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:32.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.900+0000 7f1f1e800700 1 --2- 172.21.15.1:0/2575284027 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f18100f10 0x7f1f181012f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:32.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.900+0000 7f1f1e800700 1 --2- 172.21.15.1:0/2575284027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f181018c0 0x7f1f18105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:32.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.900+0000 7f1f1e800700 1 -- 172.21.15.1:0/2575284027 >> 172.21.15.1:0/2575284027 conn(0x7f1f180fc790 msgr2=0x7f1f180febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:32.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.900+0000 7f1f1e800700 1 -- 172.21.15.1:0/2575284027 shutdown_connections 2024-02-02T15:33:32.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.900+0000 7f1f1e800700 1 -- 172.21.15.1:0/2575284027 wait complete. 2024-02-02T15:33:32.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.901+0000 7f1f1e800700 1 Processor -- start 2024-02-02T15:33:32.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.901+0000 7f1f1e800700 1 -- start start 2024-02-02T15:33:32.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.901+0000 7f1f1e800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f18100f10 0x7f1f18198ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:32.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.901+0000 7f1f1e800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f18106030 0x7f1f181993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:32.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.901+0000 7f1f1e800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f1819d840 0x7f1f1819dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:32.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.901+0000 7f1f1e800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1819e2c0 con 0x7f1f18106030 2024-02-02T15:33:32.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.901+0000 7f1f1e800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1819e430 con 0x7f1f1819d840 2024-02-02T15:33:32.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.901+0000 7f1f1e800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1819e5f0 con 0x7f1f18100f10 2024-02-02T15:33:32.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.902+0000 7f1f17e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f1819d840 0x7f1f1819dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:32.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.902+0000 7f1f17e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f1819d840 0x7f1f1819dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:50100/0 (socket says 172.21.15.1:50100) 2024-02-02T15:33:32.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.902+0000 7f1f17e00700 1 -- 172.21.15.1:0/3502498195 learned_addr learned my addr 172.21.15.1:0/3502498195 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:33:32.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.902+0000 7f1f16a00700 1 --2- 172.21.15.1:0/3502498195 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f18106030 0x7f1f181993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:32.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.902+0000 7f1f17400700 1 --2- 172.21.15.1:0/3502498195 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f18100f10 0x7f1f18198ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:32.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.902+0000 7f1f17e00700 1 -- 172.21.15.1:0/3502498195 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f18100f10 msgr2=0x7f1f18198ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:32.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.902+0000 7f1f17e00700 1 --2- 172.21.15.1:0/3502498195 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f18100f10 0x7f1f18198ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:32.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.902+0000 7f1f17e00700 1 -- 172.21.15.1:0/3502498195 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f18106030 msgr2=0x7f1f181993e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:32.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.902+0000 7f1f17e00700 1 --2- 172.21.15.1:0/3502498195 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f18106030 0x7f1f181993e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:32.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.902+0000 7f1f17e00700 1 -- 172.21.15.1:0/3502498195 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f10009d00 con 0x7f1f1819d840 2024-02-02T15:33:32.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.902+0000 7f1f17e00700 1 --2- 172.21.15.1:0/3502498195 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f1819d840 0x7f1f1819dcc0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f1f10009850 tx=0x7f1f100077f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:33:32.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.903+0000 7f1f0fe00700 1 -- 172.21.15.1:0/3502498195 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f1000e040 con 0x7f1f1819d840 2024-02-02T15:33:32.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.903+0000 7f1f0fe00700 1 -- 172.21.15.1:0/3502498195 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f100079e0 con 0x7f1f1819d840 2024-02-02T15:33:32.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.903+0000 7f1f1e800700 1 -- 172.21.15.1:0/3502498195 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f181a5af0 con 0x7f1f1819d840 2024-02-02T15:33:32.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.903+0000 7f1f0fe00700 1 -- 172.21.15.1:0/3502498195 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f100052a0 con 0x7f1f1819d840 2024-02-02T15:33:32.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.903+0000 7f1f1e800700 1 -- 172.21.15.1:0/3502498195 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f181a5f70 con 0x7f1f1819d840 2024-02-02T15:33:32.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.904+0000 7f1f1e800700 1 -- 172.21.15.1:0/3502498195 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f18067bc0 con 0x7f1f1819d840 2024-02-02T15:33:32.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.906+0000 7f1f0fe00700 1 -- 172.21.15.1:0/3502498195 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 27) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f1f10003a00 con 0x7f1f1819d840 2024-02-02T15:33:32.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.907+0000 7f1f0fe00700 1 --2- 172.21.15.1:0/3502498195 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f1ef80a0710 0x7f1ef80a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:32.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.907+0000 7f1f0fe00700 1 -- 172.21.15.1:0/3502498195 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f1f100cd790 con 0x7f1f1819d840 2024-02-02T15:33:32.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.907+0000 7f1f17400700 1 --2- 172.21.15.1:0/3502498195 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f1ef80a0710 0x7f1ef80a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:32.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.908+0000 7f1f17400700 1 --2- 172.21.15.1:0/3502498195 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f1ef80a0710 0x7f1ef80a2bd0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1f08009c60 tx=0x7f1f08009260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:33:32.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:32.909+0000 7f1f0fe00700 1 -- 172.21.15.1:0/3502498195 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f1f10096d60 con 0x7f1f1819d840 2024-02-02T15:33:33.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.169+0000 7f1f1e800700 1 -- 172.21.15.1:0/3502498195 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1f1819a1f0 con 0x7f1f1819d840 2024-02-02T15:33:33.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.170+0000 7f1f0fe00700 1 -- 172.21.15.1:0/3502498195 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+10873 (secure 0 0 0) 0x7f1f10004e60 con 0x7f1f1819d840 2024-02-02T15:33:33.170 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:33:33.170 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2922240448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2922240448},{"type":"v1","addr":"172.21.15.1:6835","nonce":2922240448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2729497431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2729497431},{"type":"v1","addr":"172.21.15.1:6837","nonce":2729497431}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":38,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-02T15:24:47.761435+0000","modified":"2024-02-02T15:32:42.392323+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24529,"mds_2":24518,"mds_3":24514,"mds_4":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14517":{"gid":14517,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/1218303683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":1218303683},{"type":"v1","addr":"172.21.15.1:6839","nonce":1218303683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14529":{"gid":14529,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/2059197496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2059197496},{"type":"v1","addr":"172.21.15.1:6841","nonce":2059197496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/7648986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":7648986},{"type":"v1","addr":"172.21.15.142:6833","nonce":7648986}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/3482746094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3482746094},{"type":"v1","addr":"172.21.15.71:6833","nonce":3482746094}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/388207814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":388207814},{"type":"v1","addr":"172.21.15.71:6835","nonce":388207814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6835/1052606547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1052606547},{"type":"v1","addr":"172.21.15.142:6835","nonce":1052606547}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.142:6837/2870289019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2870289019},{"type":"v1","addr":"172.21.15.142:6837","nonce":2870289019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.71:6837/628016199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":628016199},{"type":"v1","addr":"172.21.15.71:6837","nonce":628016199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.142:6839/321675421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":321675421},{"type":"v1","addr":"172.21.15.142:6839","nonce":321675421}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/1268086493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1268086493},{"type":"v1","addr":"172.21.15.71:6839","nonce":1268086493}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-02T15:33:33.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.172+0000 7f1f1e800700 1 -- 172.21.15.1:0/3502498195 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f1ef80a0710 msgr2=0x7f1ef80a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:33.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.172+0000 7f1f1e800700 1 --2- 172.21.15.1:0/3502498195 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f1ef80a0710 0x7f1ef80a2bd0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1f08009c60 tx=0x7f1f08009260 comp rx=0 tx=0).stop 2024-02-02T15:33:33.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.172+0000 7f1f1e800700 1 -- 172.21.15.1:0/3502498195 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f1819d840 msgr2=0x7f1f1819dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:33.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.172+0000 7f1f1e800700 1 --2- 172.21.15.1:0/3502498195 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f1819d840 0x7f1f1819dcc0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f1f10009850 tx=0x7f1f100077f0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.173+0000 7f1f1e800700 1 -- 172.21.15.1:0/3502498195 shutdown_connections 2024-02-02T15:33:33.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.173+0000 7f1f1e800700 1 --2- 172.21.15.1:0/3502498195 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f1ef80a0710 0x7f1ef80a2bd0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.173+0000 7f1f1e800700 1 --2- 172.21.15.1:0/3502498195 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f1819d840 0x7f1f1819dcc0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.173+0000 7f1f1e800700 1 --2- 172.21.15.1:0/3502498195 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f18100f10 0x7f1f18198ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.173+0000 7f1f1e800700 1 --2- 172.21.15.1:0/3502498195 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f18106030 0x7f1f181993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.173+0000 7f1f1e800700 1 -- 172.21.15.1:0/3502498195 >> 172.21.15.1:0/3502498195 conn(0x7f1f180fc790 msgr2=0x7f1f180fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:33.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.173+0000 7f1f1e800700 1 -- 172.21.15.1:0/3502498195 shutdown_connections 2024-02-02T15:33:33.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.173+0000 7f1f1e800700 1 -- 172.21.15.1:0/3502498195 wait complete. 2024-02-02T15:33:33.174 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 38 2024-02-02T15:33:33.191 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-02-02T15:33:33.240 INFO:tasks.workunit.client.0.smithi001.stdout: 1 423824 8.17 MB/sec execute 187 sec latency 1065.385 ms 2024-02-02T15:33:33.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.391+0000 7f7144600700 1 -- 172.21.15.1:0/1691225487 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f713c100ff0 msgr2=0x7f713c1013d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:33.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.391+0000 7f7144600700 1 --2- 172.21.15.1:0/1691225487 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f713c100ff0 0x7f713c1013d0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f712c009990 tx=0x7f712c009ca0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.392+0000 7f7144600700 1 -- 172.21.15.1:0/1691225487 shutdown_connections 2024-02-02T15:33:33.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.392+0000 7f7144600700 1 --2- 172.21.15.1:0/1691225487 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f713c106110 0x7f713c10dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.392+0000 7f7144600700 1 --2- 172.21.15.1:0/1691225487 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f713c100ff0 0x7f713c1013d0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.392+0000 7f7144600700 1 --2- 172.21.15.1:0/1691225487 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f713c1019a0 0x7f713c1059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.392+0000 7f7144600700 1 -- 172.21.15.1:0/1691225487 >> 172.21.15.1:0/1691225487 conn(0x7f713c0fc810 msgr2=0x7f713c0fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:33.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.392+0000 7f7144600700 1 -- 172.21.15.1:0/1691225487 shutdown_connections 2024-02-02T15:33:33.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.392+0000 7f7144600700 1 -- 172.21.15.1:0/1691225487 wait complete. 2024-02-02T15:33:33.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.393+0000 7f7144600700 1 Processor -- start 2024-02-02T15:33:33.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.393+0000 7f7144600700 1 -- start start 2024-02-02T15:33:33.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.393+0000 7f7144600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f713c100ff0 0x7f713c073040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:33.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.393+0000 7f7144600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f713c106110 0x7f713c073580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:33.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.393+0000 7f7144600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f713c073ac0 0x7f713c1a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:33.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.393+0000 7f7144600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f713c1a5770 con 0x7f713c073ac0 2024-02-02T15:33:33.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.393+0000 7f7144600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f713c1a58e0 con 0x7f713c106110 2024-02-02T15:33:33.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.393+0000 7f7144600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f713c1a5aa0 con 0x7f713c100ff0 2024-02-02T15:33:33.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.394+0000 7f7141c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f713c100ff0 0x7f713c073040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:33.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.394+0000 7f7142600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f713c073ac0 0x7f713c1a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:33.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.394+0000 7f7141c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f713c100ff0 0x7f713c073040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:55884/0 (socket says 172.21.15.1:55884) 2024-02-02T15:33:33.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.394+0000 7f7141c00700 1 -- 172.21.15.1:0/972882594 learned_addr learned my addr 172.21.15.1:0/972882594 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:33:33.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.394+0000 7f7141200700 1 --2- 172.21.15.1:0/972882594 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f713c106110 0x7f713c073580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:33.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.394+0000 7f7141c00700 1 -- 172.21.15.1:0/972882594 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f713c106110 msgr2=0x7f713c073580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:33.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.394+0000 7f7141c00700 1 --2- 172.21.15.1:0/972882594 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f713c106110 0x7f713c073580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.394+0000 7f7141c00700 1 -- 172.21.15.1:0/972882594 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f713c073ac0 msgr2=0x7f713c1a50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:33.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.394+0000 7f7141c00700 1 --2- 172.21.15.1:0/972882594 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f713c073ac0 0x7f713c1a50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.394+0000 7f7141c00700 1 -- 172.21.15.1:0/972882594 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f712c0095f0 con 0x7f713c100ff0 2024-02-02T15:33:33.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.394+0000 7f7141c00700 1 --2- 172.21.15.1:0/972882594 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f713c100ff0 0x7f713c073040 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f712c005910 tx=0x7f712c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:33:33.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.394+0000 7f713a000700 1 -- 172.21.15.1:0/972882594 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712c01d070 con 0x7f713c100ff0 2024-02-02T15:33:33.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.395+0000 7f713a000700 1 -- 172.21.15.1:0/972882594 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f712c022470 con 0x7f713c100ff0 2024-02-02T15:33:33.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.395+0000 7f713a000700 1 -- 172.21.15.1:0/972882594 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712c017650 con 0x7f713c100ff0 2024-02-02T15:33:33.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.395+0000 7f7144600700 1 -- 172.21.15.1:0/972882594 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f713c1a5d20 con 0x7f713c100ff0 2024-02-02T15:33:33.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.395+0000 7f7144600700 1 -- 172.21.15.1:0/972882594 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f713c1a6140 con 0x7f713c100ff0 2024-02-02T15:33:33.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.398+0000 7f713a000700 1 -- 172.21.15.1:0/972882594 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 27) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f712c0225e0 con 0x7f713c100ff0 2024-02-02T15:33:33.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.399+0000 7f71581b8b80 1 Processor -- start 2024-02-02T15:33:33.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.400+0000 7f71581b8b80 1 -- start start 2024-02-02T15:33:33.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.400+0000 7f71581b8b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582e2744010 0x5582e27443f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:33.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.400+0000 7f71581b8b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5582e2744930 0x5582e274f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:33.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.400+0000 7f71581b8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582e274f7e0 0x5582e2751bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:33.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.400+0000 7f71581b8b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582e26b84c0 con 0x5582e2744930 2024-02-02T15:33:33.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.400+0000 7f71581b8b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582e269caa0 con 0x5582e2744010 2024-02-02T15:33:33.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.400+0000 7f71581b8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582e269a460 con 0x5582e274f7e0 2024-02-02T15:33:33.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.400+0000 7f713a000700 1 --2- 172.21.15.1:0/972882594 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f711c0a0710 0x7f711c0a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:33.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.400+0000 7f7141c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5582e2744930 0x5582e274f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:33.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.400+0000 7f7141200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582e274f7e0 0x5582e2751bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:33.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.400+0000 7f7142600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582e2744010 0x5582e27443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:33.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.400+0000 7f7141c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5582e2744930 0x5582e274f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57802/0 (socket says 172.21.15.1:57802) 2024-02-02T15:33:33.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.401+0000 7f7141c00700 1 -- 172.21.15.1:0/2227761879 learned_addr learned my addr 172.21.15.1:0/2227761879 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:33:33.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.401+0000 7f713a000700 1 -- 172.21.15.1:0/972882594 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f712c0cd290 con 0x7f713c100ff0 2024-02-02T15:33:33.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.401+0000 7f7142600700 1 --2- 172.21.15.1:0/972882594 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f711c0a0710 0x7f711c0a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:33.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.401+0000 7f7141c00700 1 -- 172.21.15.1:0/2227761879 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582e274f7e0 msgr2=0x5582e2751bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:33.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.401+0000 7f7141c00700 1 --2- 172.21.15.1:0/2227761879 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582e274f7e0 0x5582e2751bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.401+0000 7f7141c00700 1 -- 172.21.15.1:0/2227761879 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582e2744010 msgr2=0x5582e27443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:33.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.401+0000 7f7141c00700 1 --2- 172.21.15.1:0/2227761879 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582e2744010 0x5582e27443f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.401+0000 7f7141c00700 1 -- 172.21.15.1:0/2227761879 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5582e25173b0 con 0x5582e2744930 2024-02-02T15:33:33.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.401+0000 7f7141200700 1 --2- 172.21.15.1:0/2227761879 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582e274f7e0 0x5582e2751bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:33:33.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.401+0000 7f7141c00700 1 --2- 172.21.15.1:0/2227761879 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5582e2744930 0x5582e274f2a0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f712c006010 tx=0x7f712c0c4770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54940b3f348e1c36 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:33:33.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.401+0000 7f7132a00700 1 -- 172.21.15.1:0/2227761879 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712c09ab90 con 0x5582e2744930 2024-02-02T15:33:33.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.401+0000 7f7142600700 1 --2- 172.21.15.1:0/972882594 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f711c0a0710 0x7f711c0a2bd0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7134009ec0 tx=0x7f713400b040 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:33:33.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.401+0000 7f7132a00700 1 -- 172.21.15.1:0/2227761879 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f712c09a490 con 0x5582e2744930 2024-02-02T15:33:33.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.401+0000 7f7132a00700 1 -- 172.21.15.1:0/2227761879 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712c065e50 con 0x5582e2744930 2024-02-02T15:33:33.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.402+0000 7f71581b8b80 1 -- 172.21.15.1:0/2227761879 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5582e2744930 msgr2=0x5582e274f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:33.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.402+0000 7f71581b8b80 1 --2- 172.21.15.1:0/2227761879 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5582e2744930 0x5582e274f2a0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f712c006010 tx=0x7f712c0c4770 comp rx=0 tx=0).stop 2024-02-02T15:33:33.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.402+0000 7f71581b8b80 1 -- 172.21.15.1:0/2227761879 shutdown_connections 2024-02-02T15:33:33.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.402+0000 7f71581b8b80 1 --2- 172.21.15.1:0/2227761879 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582e274f7e0 0x5582e2751bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.402+0000 7f71581b8b80 1 --2- 172.21.15.1:0/2227761879 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582e2744010 0x5582e27443f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.402+0000 7f71581b8b80 1 --2- 172.21.15.1:0/2227761879 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5582e2744930 0x5582e274f2a0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.402+0000 7f71581b8b80 1 -- 172.21.15.1:0/2227761879 >> 172.21.15.1:0/2227761879 conn(0x5582e26a3db0 msgr2=0x5582e27403b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:33.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.402+0000 7f71581b8b80 1 -- 172.21.15.1:0/2227761879 shutdown_connections 2024-02-02T15:33:33.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.402+0000 7f71581b8b80 1 -- 172.21.15.1:0/2227761879 wait complete. 2024-02-02T15:33:33.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.403+0000 7f71581b8b80 1 -- start start 2024-02-02T15:33:33.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.403+0000 7f71581b8b80 1 Processor -- start 2024-02-02T15:33:33.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.404+0000 7f71581b8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582e2744010 0x5582e2754810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:33.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.404+0000 7f71581b8b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582e2744930 0x5582e2754d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:33.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.404+0000 7f71581b8b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5582e274f7e0 0x5582e274e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:33.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.404+0000 7f71581b8b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582e2695d50 con 0x5582e274f7e0 2024-02-02T15:33:33.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.404+0000 7f71581b8b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582e2631d80 con 0x5582e2744930 2024-02-02T15:33:33.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.404+0000 7f71581b8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582e2631a20 con 0x5582e2744010 2024-02-02T15:33:33.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.404+0000 7f71581b8b80 10 client.0 _ll_register_callbacks cb 0x5582e2633290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:33:33.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.404+0000 7f7142600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5582e274f7e0 0x5582e274e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:33.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.404+0000 7f7141200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582e2744930 0x5582e2754d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:33.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.404+0000 7f7141200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582e2744930 0x5582e2754d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:50134/0 (socket says 172.21.15.1:50134) 2024-02-02T15:33:33.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.404+0000 7f7141200700 1 -- 172.21.15.1:0/3816364308 learned_addr learned my addr 172.21.15.1:0/3816364308 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:33:33.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.404+0000 7f7141c00700 1 --2- 172.21.15.1:0/3816364308 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582e2744010 0x5582e2754810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:33.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.405+0000 7f7141200700 1 -- 172.21.15.1:0/3816364308 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582e2744010 msgr2=0x5582e2754810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:33.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.405+0000 7f7141200700 1 --2- 172.21.15.1:0/3816364308 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582e2744010 0x5582e2754810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.405+0000 7f7141200700 1 -- 172.21.15.1:0/3816364308 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5582e274f7e0 msgr2=0x5582e274e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:33.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.405+0000 7f7141200700 1 --2- 172.21.15.1:0/3816364308 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5582e274f7e0 0x5582e274e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.405+0000 7f7141200700 1 -- 172.21.15.1:0/3816364308 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71240098e0 con 0x5582e2744930 2024-02-02T15:33:33.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.405+0000 7f7141200700 1 --2- 172.21.15.1:0/3816364308 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582e2744930 0x5582e2754d50 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f7124011970 tx=0x7f7124011c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6fd8e87c53aca476 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:33:33.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.405+0000 7f712b400700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:33:33.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.405+0000 7f712b400700 1 -- 172.21.15.1:0/3816364308 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7124016070 con 0x5582e2744930 2024-02-02T15:33:33.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.405+0000 7f712b400700 1 -- 172.21.15.1:0/3816364308 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f7124004300 con 0x5582e2744930 2024-02-02T15:33:33.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.405+0000 7f71581b8b80 1 -- 172.21.15.1:0/3816364308 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5582e27380f0 con 0x5582e2744930 2024-02-02T15:33:33.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.406+0000 7f712b400700 1 -- 172.21.15.1:0/3816364308 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71240044c0 con 0x5582e2744930 2024-02-02T15:33:33.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.406+0000 7f71581b8b80 10 client.27338 fetch_fsmap learned FSMap version 38 2024-02-02T15:33:33.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.406+0000 7f71581b8b80 1 -- 172.21.15.1:0/3816364308 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x7f713c1a6140 con 0x5582e2744930 2024-02-02T15:33:33.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.406+0000 7f712b400700 1 -- 172.21.15.1:0/3816364308 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5582e27380f0 con 0x5582e2744930 2024-02-02T15:33:33.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.406+0000 7f712b400700 1 -- 172.21.15.1:0/3816364308 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 38) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f712400fab0 con 0x5582e2744930 2024-02-02T15:33:33.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.406+0000 7f71581b8b80 10 client.27338 fetch_fsmap finished waiting for FSMap version 38 2024-02-02T15:33:33.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.406+0000 7f71581b8b80 10 client.27338 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-02-02T15:33:33.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.406+0000 7f71581b8b80 20 client.27338 populate_metadata read hostname 'smithi001' 2024-02-02T15:33:33.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.406+0000 7f71581b8b80 1 --2- 172.21.15.1:0/3816364308 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x5582e275ee60 0x5582e2761280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:33.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.406+0000 7f71581b8b80 4 client.27338 mds_command: new command op to 24502 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:33:33.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.406+0000 7f71581b8b80 1 -- 172.21.15.1:0/3816364308 --> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5582e268edd0 con 0x5582e275ee60 2024-02-02T15:33:33.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.406+0000 7f7141c00700 1 --2- 172.21.15.1:0/3816364308 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x5582e275ee60 0x5582e2761280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:33.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.407+0000 7f7141c00700 1 --2- 172.21.15.1:0/3816364308 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x5582e275ee60 0x5582e2761280 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2b4ac782c7b622ed server_cookie=ce7b3d97402776ef in_seq=0 out_seq=0 2024-02-02T15:33:33.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.407+0000 7f712b400700 10 client.27338 ms_handle_connect on v2:172.21.15.142:6834/1052606547 2024-02-02T15:33:33.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.408+0000 7f712b400700 1 -- 172.21.15.1:0/3816364308 <== mds.4 v2:172.21.15.142:6834/1052606547 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5582e268edd0 con 0x5582e275ee60 2024-02-02T15:33:33.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.408+0000 7f712b400700 10 client.27338 handle_command_reply: tid=0 2024-02-02T15:33:33.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.408+0000 7f71581b8b80 1 client.27338 shutdown 2024-02-02T15:33:33.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.408+0000 7f71581b8b80 1 -- 172.21.15.1:0/3816364308 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582e2744930 msgr2=0x5582e2754d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:33.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.408+0000 7f71581b8b80 1 --2- 172.21.15.1:0/3816364308 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582e2744930 0x5582e2754d50 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f7124011970 tx=0x7f7124011c80 comp rx=0 tx=0).stop 2024-02-02T15:33:33.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.408+0000 7f71581b8b80 1 -- 172.21.15.1:0/3816364308 shutdown_connections 2024-02-02T15:33:33.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.408+0000 7f71581b8b80 1 --2- 172.21.15.1:0/3816364308 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x5582e275ee60 0x5582e2761280 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.408+0000 7f71581b8b80 1 --2- 172.21.15.1:0/3816364308 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5582e274f7e0 0x5582e274e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.408+0000 7f71581b8b80 1 --2- 172.21.15.1:0/3816364308 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582e2744010 0x5582e2754810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.408+0000 7f71581b8b80 1 --2- 172.21.15.1:0/3816364308 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582e2744930 0x5582e2754d50 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.408+0000 7f712b400700 0 client.27338 ms_handle_reset on v2:172.21.15.142:6834/1052606547 2024-02-02T15:33:33.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.408+0000 7f71581b8b80 1 -- 172.21.15.1:0/3816364308 >> 172.21.15.1:0/3816364308 conn(0x5582e26a3db0 msgr2=0x5582e273ffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:33.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.408+0000 7f71581b8b80 1 -- 172.21.15.1:0/3816364308 shutdown_connections 2024-02-02T15:33:33.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.408+0000 7f71581b8b80 1 -- 172.21.15.1:0/3816364308 wait complete. 2024-02-02T15:33:33.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.408+0000 7f71581b8b80 20 client.27338 trim_cache size 0 max 16384 2024-02-02T15:33:33.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.421+0000 7f71581b8b80 1 Processor -- start 2024-02-02T15:33:33.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.421+0000 7f71581b8b80 1 -- start start 2024-02-02T15:33:33.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.422+0000 7f71581b8b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5582e2744010 0x5582e27543e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:33.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.422+0000 7f71581b8b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582e2744930 0x5582e274e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:33.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.422+0000 7f71581b8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582e274f7e0 0x5582e274b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:33.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.422+0000 7f71581b8b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582e25196a0 con 0x5582e2744010 2024-02-02T15:33:33.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.422+0000 7f71581b8b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582e274a350 con 0x5582e2744930 2024-02-02T15:33:33.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.422+0000 7f71581b8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582e274ebf0 con 0x5582e274f7e0 2024-02-02T15:33:33.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.422+0000 7f7141c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5582e2744010 0x5582e27543e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:33.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.422+0000 7f7141c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5582e2744010 0x5582e27543e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57816/0 (socket says 172.21.15.1:57816) 2024-02-02T15:33:33.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.422+0000 7f7141c00700 1 -- 172.21.15.1:0/2777016467 learned_addr learned my addr 172.21.15.1:0/2777016467 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:33:33.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.422+0000 7f7142600700 1 --2- 172.21.15.1:0/2777016467 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582e274f7e0 0x5582e274b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:33.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.422+0000 7f7141200700 1 --2- 172.21.15.1:0/2777016467 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582e2744930 0x5582e274e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:33.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.423+0000 7f7141c00700 1 -- 172.21.15.1:0/2777016467 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582e274f7e0 msgr2=0x5582e274b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:33.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.423+0000 7f7141c00700 1 --2- 172.21.15.1:0/2777016467 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582e274f7e0 0x5582e274b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.423+0000 7f7141c00700 1 -- 172.21.15.1:0/2777016467 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582e2744930 msgr2=0x5582e274e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:33.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.423+0000 7f7141c00700 1 --2- 172.21.15.1:0/2777016467 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582e2744930 0x5582e274e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.423+0000 7f7141c00700 1 -- 172.21.15.1:0/2777016467 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5582e24c3b60 con 0x5582e2744010 2024-02-02T15:33:33.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.423+0000 7f7141c00700 1 --2- 172.21.15.1:0/2777016467 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5582e2744010 0x5582e27543e0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f712c096810 tx=0x7f712c0cc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2baac84883dbae8c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:33:33.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.423+0000 7f7133400700 1 -- 172.21.15.1:0/2777016467 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712c0a2470 con 0x5582e2744010 2024-02-02T15:33:33.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.423+0000 7f7133400700 1 -- 172.21.15.1:0/2777016467 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f712c09e2c0 con 0x5582e2744010 2024-02-02T15:33:33.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.423+0000 7f7133400700 1 -- 172.21.15.1:0/2777016467 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712c0228c0 con 0x5582e2744010 2024-02-02T15:33:33.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.424+0000 7f71581b8b80 1 -- 172.21.15.1:0/2777016467 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5582e2744010 msgr2=0x5582e27543e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:33.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.424+0000 7f71581b8b80 1 --2- 172.21.15.1:0/2777016467 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5582e2744010 0x5582e27543e0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f712c096810 tx=0x7f712c0cc040 comp rx=0 tx=0).stop 2024-02-02T15:33:33.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.424+0000 7f71581b8b80 1 -- 172.21.15.1:0/2777016467 shutdown_connections 2024-02-02T15:33:33.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.424+0000 7f71581b8b80 1 --2- 172.21.15.1:0/2777016467 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582e274f7e0 0x5582e274b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.424+0000 7f71581b8b80 1 --2- 172.21.15.1:0/2777016467 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5582e2744010 0x5582e27543e0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.424+0000 7f71581b8b80 1 --2- 172.21.15.1:0/2777016467 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582e2744930 0x5582e274e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.424+0000 7f71581b8b80 1 -- 172.21.15.1:0/2777016467 >> 172.21.15.1:0/2777016467 conn(0x5582e26a3db0 msgr2=0x5582e274dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:33.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.424+0000 7f71581b8b80 1 -- 172.21.15.1:0/2777016467 shutdown_connections 2024-02-02T15:33:33.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.424+0000 7f71581b8b80 1 -- 172.21.15.1:0/2777016467 wait complete. 2024-02-02T15:33:33.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.424+0000 7f71581b8b80 1 -- start start 2024-02-02T15:33:33.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.425+0000 7f71581b8b80 1 Processor -- start 2024-02-02T15:33:33.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.425+0000 7f71581b8b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5582e2744010 0x5582e27a4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:33.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.425+0000 7f71581b8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582e2744930 0x5582e27a5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:33.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.425+0000 7f71581b8b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582e274f7e0 0x5582e27af990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:33.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.425+0000 7f71581b8b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582e27afed0 con 0x5582e2744010 2024-02-02T15:33:33.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.425+0000 7f71581b8b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582e27b0010 con 0x5582e274f7e0 2024-02-02T15:33:33.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.425+0000 7f71581b8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582e27b0150 con 0x5582e2744930 2024-02-02T15:33:33.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.425+0000 7f71581b8b80 10 client.0 _ll_register_callbacks cb 0x5582e2633290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:33:33.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.425+0000 7f7141c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5582e2744010 0x5582e27a4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:33.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.426+0000 7f7141c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5582e2744010 0x5582e27a4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57832/0 (socket says 172.21.15.1:57832) 2024-02-02T15:33:33.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.426+0000 7f7141c00700 1 -- 172.21.15.1:0/2049489732 learned_addr learned my addr 172.21.15.1:0/2049489732 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:33:33.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.426+0000 7f7142600700 1 --2- 172.21.15.1:0/2049489732 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582e274f7e0 0x5582e27af990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:33.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.426+0000 7f7141200700 1 --2- 172.21.15.1:0/2049489732 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582e2744930 0x5582e27a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:33.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.426+0000 7f7141c00700 1 -- 172.21.15.1:0/2049489732 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582e2744930 msgr2=0x5582e27a5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:33.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.426+0000 7f7141c00700 1 --2- 172.21.15.1:0/2049489732 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582e2744930 0x5582e27a5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.426+0000 7f7141c00700 1 -- 172.21.15.1:0/2049489732 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582e274f7e0 msgr2=0x5582e27af990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:33.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.426+0000 7f7141c00700 1 --2- 172.21.15.1:0/2049489732 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582e274f7e0 0x5582e27af990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.426+0000 7f7141c00700 1 -- 172.21.15.1:0/2049489732 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5582e25173b0 con 0x5582e2744010 2024-02-02T15:33:33.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.426+0000 7f7142600700 1 --2- 172.21.15.1:0/2049489732 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582e274f7e0 0x5582e27af990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:33:33.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.426+0000 7f7141c00700 1 --2- 172.21.15.1:0/2049489732 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5582e2744010 0x5582e27a4b00 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f712c096990 tx=0x7f712c099ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3ee2bae2336142c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:33:33.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.426+0000 7f712b400700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:33:33.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.426+0000 7f712b400700 1 -- 172.21.15.1:0/2049489732 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712c0a2470 con 0x5582e2744010 2024-02-02T15:33:33.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.426+0000 7f712b400700 1 -- 172.21.15.1:0/2049489732 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f712c0228c0 con 0x5582e2744010 2024-02-02T15:33:33.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.426+0000 7f71581b8b80 1 -- 172.21.15.1:0/2049489732 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f712c09a490 con 0x5582e2744010 2024-02-02T15:33:33.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.427+0000 7f712b400700 1 -- 172.21.15.1:0/2049489732 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f712c0b8270 con 0x5582e2744010 2024-02-02T15:33:33.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.427+0000 7f71581b8b80 10 client.17283 fetch_fsmap learned FSMap version 38 2024-02-02T15:33:33.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.427+0000 7f71581b8b80 1 -- 172.21.15.1:0/2049489732 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x5582e265bea0 con 0x5582e2744010 2024-02-02T15:33:33.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.427+0000 7f712b400700 1 -- 172.21.15.1:0/2049489732 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f712c0a8020 con 0x5582e2744010 2024-02-02T15:33:33.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.427+0000 7f712b400700 1 -- 172.21.15.1:0/2049489732 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 38) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f712c09a760 con 0x5582e2744010 2024-02-02T15:33:33.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.427+0000 7f71581b8b80 10 client.17283 fetch_fsmap finished waiting for FSMap version 38 2024-02-02T15:33:33.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.427+0000 7f71581b8b80 10 client.17283 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-02-02T15:33:33.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.427+0000 7f71581b8b80 20 client.17283 populate_metadata read hostname 'smithi001' 2024-02-02T15:33:33.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.427+0000 7f71581b8b80 1 --2- 172.21.15.1:0/2049489732 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x5582e27b13f0 0x5582e27b3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:33.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.427+0000 7f71581b8b80 4 client.17283 mds_command: new command op to 24502 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:33:33.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.427+0000 7f71581b8b80 1 -- 172.21.15.1:0/2049489732 --> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5582e2654f50 con 0x5582e27b13f0 2024-02-02T15:33:33.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.428+0000 7f7141200700 1 --2- 172.21.15.1:0/2049489732 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x5582e27b13f0 0x5582e27b3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:33.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.428+0000 7f7141200700 1 --2- 172.21.15.1:0/2049489732 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x5582e27b13f0 0x5582e27b3810 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3efb6586d9a031ed server_cookie=fa64c27070a398ff in_seq=0 out_seq=0 2024-02-02T15:33:33.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.428+0000 7f712b400700 10 client.17283 ms_handle_connect on v2:172.21.15.142:6834/1052606547 2024-02-02T15:33:33.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.428+0000 7f712b400700 1 -- 172.21.15.1:0/2049489732 <== mds.4 v2:172.21.15.142:6834/1052606547 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5582e2654f50 con 0x5582e27b13f0 2024-02-02T15:33:33.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.428+0000 7f712b400700 10 client.17283 handle_command_reply: tid=0 2024-02-02T15:33:33.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.428+0000 7f71581b8b80 1 client.17283 shutdown 2024-02-02T15:33:33.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.429+0000 7f71581b8b80 1 -- 172.21.15.1:0/2049489732 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5582e2744010 msgr2=0x5582e27a4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:33.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.429+0000 7f71581b8b80 1 --2- 172.21.15.1:0/2049489732 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5582e2744010 0x5582e27a4b00 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f712c096990 tx=0x7f712c099ef0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.429+0000 7f71581b8b80 1 -- 172.21.15.1:0/2049489732 shutdown_connections 2024-02-02T15:33:33.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.429+0000 7f71581b8b80 1 --2- 172.21.15.1:0/2049489732 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x5582e27b13f0 0x5582e27b3810 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.429+0000 7f71581b8b80 1 --2- 172.21.15.1:0/2049489732 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5582e274f7e0 0x5582e27af990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.429+0000 7f71581b8b80 1 --2- 172.21.15.1:0/2049489732 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5582e2744010 0x5582e27a4b00 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.429+0000 7f71581b8b80 1 --2- 172.21.15.1:0/2049489732 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5582e2744930 0x5582e27a5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.429+0000 7f712b400700 0 client.17283 ms_handle_reset on v2:172.21.15.142:6834/1052606547 2024-02-02T15:33:33.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.429+0000 7f71581b8b80 1 -- 172.21.15.1:0/2049489732 >> 172.21.15.1:0/2049489732 conn(0x5582e26a3db0 msgr2=0x5582e274ceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:33.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.429+0000 7f71581b8b80 1 -- 172.21.15.1:0/2049489732 shutdown_connections 2024-02-02T15:33:33.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.429+0000 7f71581b8b80 1 -- 172.21.15.1:0/2049489732 wait complete. 2024-02-02T15:33:33.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.429+0000 7f71581b8b80 20 client.17283 trim_cache size 0 max 16384 2024-02-02T15:33:33.433 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:33:33.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.430+0000 7f7144600700 1 -- 172.21.15.1:0/972882594 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f711c0a0710 msgr2=0x7f711c0a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:33.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.430+0000 7f7144600700 1 --2- 172.21.15.1:0/972882594 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f711c0a0710 0x7f711c0a2bd0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7134009ec0 tx=0x7f713400b040 comp rx=0 tx=0).stop 2024-02-02T15:33:33.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.430+0000 7f7144600700 1 -- 172.21.15.1:0/972882594 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f713c100ff0 msgr2=0x7f713c073040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:33.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.430+0000 7f7144600700 1 --2- 172.21.15.1:0/972882594 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f713c100ff0 0x7f713c073040 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f712c005910 tx=0x7f712c004730 comp rx=0 tx=0).stop 2024-02-02T15:33:33.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.430+0000 7f7144600700 1 -- 172.21.15.1:0/972882594 shutdown_connections 2024-02-02T15:33:33.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.430+0000 7f7144600700 1 --2- 172.21.15.1:0/972882594 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f711c0a0710 0x7f711c0a2bd0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.430+0000 7f7144600700 1 --2- 172.21.15.1:0/972882594 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f713c073ac0 0x7f713c1a50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.430+0000 7f7144600700 1 --2- 172.21.15.1:0/972882594 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f713c100ff0 0x7f713c073040 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.430+0000 7f7144600700 1 --2- 172.21.15.1:0/972882594 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f713c106110 0x7f713c073580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.430+0000 7f7144600700 1 -- 172.21.15.1:0/972882594 >> 172.21.15.1:0/972882594 conn(0x7f713c0fc810 msgr2=0x7f713c104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:33.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.430+0000 7f7144600700 1 -- 172.21.15.1:0/972882594 shutdown_connections 2024-02-02T15:33:33.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.430+0000 7f7144600700 1 -- 172.21.15.1:0/972882594 wait complete. 2024-02-02T15:33:33.448 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-02-02T15:33:33.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.649+0000 7f58b3800700 1 -- 172.21.15.1:0/3916632896 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f58ac068010 msgr2=0x7f58ac068470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:33.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.649+0000 7f58b3800700 1 --2- 172.21.15.1:0/3916632896 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f58ac068010 0x7f58ac068470 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f5894009990 tx=0x7f5894009ca0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.650+0000 7f58b3800700 1 -- 172.21.15.1:0/3916632896 shutdown_connections 2024-02-02T15:33:33.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.650+0000 7f58b3800700 1 --2- 172.21.15.1:0/3916632896 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f58ac0689b0 0x7f58ac10dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.650+0000 7f58b3800700 1 --2- 172.21.15.1:0/3916632896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f58ac106ed0 0x7f58ac1072b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.650+0000 7f58b3800700 1 --2- 172.21.15.1:0/3916632896 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f58ac068010 0x7f58ac068470 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.650+0000 7f58b3800700 1 -- 172.21.15.1:0/3916632896 >> 172.21.15.1:0/3916632896 conn(0x7f58ac0757d0 msgr2=0x7f58ac075be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:33.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.650+0000 7f58b3800700 1 -- 172.21.15.1:0/3916632896 shutdown_connections 2024-02-02T15:33:33.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.650+0000 7f58b3800700 1 -- 172.21.15.1:0/3916632896 wait complete. 2024-02-02T15:33:33.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.651+0000 7f58b3800700 1 Processor -- start 2024-02-02T15:33:33.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.651+0000 7f58b3800700 1 -- start start 2024-02-02T15:33:33.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.651+0000 7f58b3800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f58ac068010 0x7f58ac199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:33.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.651+0000 7f58b3800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f58ac0689b0 0x7f58ac199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:33.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.651+0000 7f58b3800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f58ac106ed0 0x7f58ac19da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:33.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.651+0000 7f58b3800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58ac19e070 con 0x7f58ac068010 2024-02-02T15:33:33.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.651+0000 7f58b3800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58ac19e1e0 con 0x7f58ac106ed0 2024-02-02T15:33:33.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.651+0000 7f58b3800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58ac19e3a0 con 0x7f58ac0689b0 2024-02-02T15:33:33.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.652+0000 7f58b0e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f58ac068010 0x7f58ac199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:33.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.652+0000 7f58b0e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f58ac068010 0x7f58ac199100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57850/0 (socket says 172.21.15.1:57850) 2024-02-02T15:33:33.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.652+0000 7f58b0e00700 1 -- 172.21.15.1:0/2222495477 learned_addr learned my addr 172.21.15.1:0/2222495477 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:33:33.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.652+0000 7f58abe00700 1 --2- 172.21.15.1:0/2222495477 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f58ac0689b0 0x7f58ac199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:33.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.652+0000 7f58b1800700 1 --2- 172.21.15.1:0/2222495477 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f58ac106ed0 0x7f58ac19da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:33.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.652+0000 7f58b0e00700 1 -- 172.21.15.1:0/2222495477 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f58ac0689b0 msgr2=0x7f58ac199640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:33.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.652+0000 7f58b0e00700 1 --2- 172.21.15.1:0/2222495477 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f58ac0689b0 0x7f58ac199640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.652+0000 7f58b0e00700 1 -- 172.21.15.1:0/2222495477 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f58ac106ed0 msgr2=0x7f58ac19da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:33.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.652+0000 7f58b0e00700 1 --2- 172.21.15.1:0/2222495477 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f58ac106ed0 0x7f58ac19da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.652+0000 7f58b0e00700 1 -- 172.21.15.1:0/2222495477 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58940095f0 con 0x7f58ac068010 2024-02-02T15:33:33.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.652+0000 7f58b1800700 1 --2- 172.21.15.1:0/2222495477 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f58ac106ed0 0x7f58ac19da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:33:33.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.652+0000 7f58b0e00700 1 --2- 172.21.15.1:0/2222495477 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f58ac068010 0x7f58ac199100 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f589c00e9a0 tx=0x7f589c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:33:33.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.652+0000 7f58a9600700 1 -- 172.21.15.1:0/2222495477 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589c009790 con 0x7f58ac068010 2024-02-02T15:33:33.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.653+0000 7f58b3800700 1 -- 172.21.15.1:0/2222495477 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58ac19e680 con 0x7f58ac068010 2024-02-02T15:33:33.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.653+0000 7f58a9600700 1 -- 172.21.15.1:0/2222495477 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f589c00cbd0 con 0x7f58ac068010 2024-02-02T15:33:33.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.653+0000 7f58a9600700 1 -- 172.21.15.1:0/2222495477 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589c018680 con 0x7f58ac068010 2024-02-02T15:33:33.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.653+0000 7f58b3800700 1 -- 172.21.15.1:0/2222495477 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58ac1a5ec0 con 0x7f58ac068010 2024-02-02T15:33:33.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.656+0000 7f58a9600700 1 -- 172.21.15.1:0/2222495477 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 27) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f589c00cd40 con 0x7f58ac068010 2024-02-02T15:33:33.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.657+0000 7f58c73c3b80 1 Processor -- start 2024-02-02T15:33:33.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.657+0000 7f58c73c3b80 1 -- start start 2024-02-02T15:33:33.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.657+0000 7f58a9600700 1 --2- 172.21.15.1:0/2222495477 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f588c0a0690 0x7f588c0a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:33.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.657+0000 7f58a9600700 1 -- 172.21.15.1:0/2222495477 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f589c014070 con 0x7f58ac068010 2024-02-02T15:33:33.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.657+0000 7f58c73c3b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bdd6852010 0x55bdd68523f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:33.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.657+0000 7f58c73c3b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bdd6852930 0x55bdd685d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:33.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.657+0000 7f58c73c3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bdd685d7e0 0x55bdd685fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:33.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.657+0000 7f58c73c3b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdd67c64c0 con 0x55bdd6852930 2024-02-02T15:33:33.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.657+0000 7f58c73c3b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdd67aaaa0 con 0x55bdd6852010 2024-02-02T15:33:33.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.657+0000 7f58c73c3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdd67a8460 con 0x55bdd685d7e0 2024-02-02T15:33:33.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.657+0000 7f58b1800700 1 --2- 172.21.15.1:0/2222495477 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f588c0a0690 0x7f588c0a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:33.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.658+0000 7f58abe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bdd6852930 0x55bdd685d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:33.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.658+0000 7f58abe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bdd6852930 0x55bdd685d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57860/0 (socket says 172.21.15.1:57860) 2024-02-02T15:33:33.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.658+0000 7f58abe00700 1 -- 172.21.15.1:0/2898486369 learned_addr learned my addr 172.21.15.1:0/2898486369 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:33:33.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.658+0000 7f58b0e00700 1 --2- 172.21.15.1:0/2898486369 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bdd6852010 0x55bdd68523f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:33.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.658+0000 7f58b1800700 1 --2- 172.21.15.1:0/2898486369 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bdd685d7e0 0x55bdd685fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:33.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.658+0000 7f58b0e00700 1 -- 172.21.15.1:0/2898486369 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bdd685d7e0 msgr2=0x55bdd685fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:33.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.658+0000 7f58b0e00700 1 --2- 172.21.15.1:0/2898486369 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bdd685d7e0 0x55bdd685fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.658+0000 7f58b0e00700 1 -- 172.21.15.1:0/2898486369 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bdd6852930 msgr2=0x55bdd685d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:33.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.658+0000 7f58b0e00700 1 --2- 172.21.15.1:0/2898486369 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bdd6852930 0x55bdd685d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.658+0000 7f58b0e00700 1 -- 172.21.15.1:0/2898486369 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bdd66253b0 con 0x55bdd6852010 2024-02-02T15:33:33.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.658+0000 7f58b0e00700 1 --2- 172.21.15.1:0/2898486369 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bdd6852010 0x55bdd68523f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f589c0c9680 tx=0x7f589c099360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca33b1b2e6e356a4 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:33:33.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.658+0000 7f58b1800700 1 --2- 172.21.15.1:0/2222495477 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f588c0a0690 0x7f588c0a2b50 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f58a4007770 tx=0x7f58a4007b30 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:33:33.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.658+0000 7f58a2000700 1 -- 172.21.15.1:0/2898486369 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589c099480 con 0x55bdd6852010 2024-02-02T15:33:33.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.659+0000 7f58a2000700 1 -- 172.21.15.1:0/2898486369 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f589c09d2b0 con 0x55bdd6852010 2024-02-02T15:33:33.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.659+0000 7f58a2000700 1 -- 172.21.15.1:0/2898486369 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589c099940 con 0x55bdd6852010 2024-02-02T15:33:33.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.659+0000 7f58c73c3b80 1 -- 172.21.15.1:0/2898486369 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bdd6852010 msgr2=0x55bdd68523f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:33.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.659+0000 7f58c73c3b80 1 --2- 172.21.15.1:0/2898486369 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bdd6852010 0x55bdd68523f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f589c0c9680 tx=0x7f589c099360 comp rx=0 tx=0).stop 2024-02-02T15:33:33.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.659+0000 7f58c73c3b80 1 -- 172.21.15.1:0/2898486369 shutdown_connections 2024-02-02T15:33:33.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.659+0000 7f58c73c3b80 1 --2- 172.21.15.1:0/2898486369 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bdd685d7e0 0x55bdd685fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.659+0000 7f58c73c3b80 1 --2- 172.21.15.1:0/2898486369 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bdd6852010 0x55bdd68523f0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.659+0000 7f58c73c3b80 1 --2- 172.21.15.1:0/2898486369 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bdd6852930 0x55bdd685d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.659+0000 7f58c73c3b80 1 -- 172.21.15.1:0/2898486369 >> 172.21.15.1:0/2898486369 conn(0x55bdd67b1db0 msgr2=0x55bdd684e3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:33.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.659+0000 7f58c73c3b80 1 -- 172.21.15.1:0/2898486369 shutdown_connections 2024-02-02T15:33:33.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.659+0000 7f58c73c3b80 1 -- 172.21.15.1:0/2898486369 wait complete. 2024-02-02T15:33:33.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.660+0000 7f58c73c3b80 1 -- start start 2024-02-02T15:33:33.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.660+0000 7f58c73c3b80 1 Processor -- start 2024-02-02T15:33:33.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.661+0000 7f58c73c3b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bdd6852010 0x55bdd685d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:33.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.661+0000 7f58c73c3b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bdd6852930 0x55bdd685a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:33.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.661+0000 7f58c73c3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bdd685d7e0 0x55bdd685abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:33.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.661+0000 7f58c73c3b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdd67a3d50 con 0x55bdd6852930 2024-02-02T15:33:33.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.661+0000 7f58c73c3b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdd673fd80 con 0x55bdd6852010 2024-02-02T15:33:33.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.661+0000 7f58c73c3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdd673fa20 con 0x55bdd685d7e0 2024-02-02T15:33:33.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.661+0000 7f58c73c3b80 10 client.0 _ll_register_callbacks cb 0x55bdd6741290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:33:33.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.661+0000 7f58b0e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bdd6852010 0x55bdd685d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:33.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.661+0000 7f58abe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bdd6852930 0x55bdd685a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:33.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.661+0000 7f58b0e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bdd6852010 0x55bdd685d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:50188/0 (socket says 172.21.15.1:50188) 2024-02-02T15:33:33.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.661+0000 7f58b1800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bdd685d7e0 0x55bdd685abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:33.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.661+0000 7f58abe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bdd6852930 0x55bdd685a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57864/0 (socket says 172.21.15.1:57864) 2024-02-02T15:33:33.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.661+0000 7f58abe00700 1 -- 172.21.15.1:0/317145664 learned_addr learned my addr 172.21.15.1:0/317145664 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:33:33.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.661+0000 7f58b0e00700 1 -- 172.21.15.1:0/317145664 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bdd685d7e0 msgr2=0x55bdd685abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:33.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.661+0000 7f58b0e00700 1 --2- 172.21.15.1:0/317145664 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bdd685d7e0 0x55bdd685abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.661+0000 7f58b0e00700 1 -- 172.21.15.1:0/317145664 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bdd6852930 msgr2=0x55bdd685a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:33.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.661+0000 7f58b0e00700 1 --2- 172.21.15.1:0/317145664 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bdd6852930 0x55bdd685a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.661+0000 7f58b0e00700 1 -- 172.21.15.1:0/317145664 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58ac1a5ec0 con 0x55bdd6852010 2024-02-02T15:33:33.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.662+0000 7f58b0e00700 1 --2- 172.21.15.1:0/317145664 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bdd6852010 0x55bdd685d280 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f589c0c96b0 tx=0x7f589c010aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=836658395abcfdcd server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:33:33.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.662+0000 7f589aa00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:33:33.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.662+0000 7f589aa00700 1 -- 172.21.15.1:0/317145664 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589c099480 con 0x55bdd6852010 2024-02-02T15:33:33.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.662+0000 7f589aa00700 1 -- 172.21.15.1:0/317145664 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f589c0a14f0 con 0x55bdd6852010 2024-02-02T15:33:33.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.662+0000 7f58c73c3b80 1 -- 172.21.15.1:0/317145664 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bdd68460f0 con 0x55bdd6852010 2024-02-02T15:33:33.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.662+0000 7f589aa00700 1 -- 172.21.15.1:0/317145664 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f589c0996e0 con 0x55bdd6852010 2024-02-02T15:33:33.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.662+0000 7f58c73c3b80 10 client.27356 fetch_fsmap learned FSMap version 38 2024-02-02T15:33:33.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.662+0000 7f58c73c3b80 1 -- 172.21.15.1:0/317145664 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55bdd65d1b60 con 0x55bdd6852010 2024-02-02T15:33:33.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.662+0000 7f589aa00700 1 -- 172.21.15.1:0/317145664 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bdd68460f0 con 0x55bdd6852010 2024-02-02T15:33:33.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.662+0000 7f589aa00700 1 -- 172.21.15.1:0/317145664 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 38) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f589c0b71b0 con 0x55bdd6852010 2024-02-02T15:33:33.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.662+0000 7f58c73c3b80 10 client.27356 fetch_fsmap finished waiting for FSMap version 38 2024-02-02T15:33:33.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.662+0000 7f58c73c3b80 10 client.27356 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-02-02T15:33:33.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.662+0000 7f58c73c3b80 20 client.27356 populate_metadata read hostname 'smithi001' 2024-02-02T15:33:33.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.662+0000 7f58c73c3b80 1 --2- 172.21.15.1:0/317145664 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55bdd6875580 0x55bdd6877970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:33.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.662+0000 7f58c73c3b80 4 client.27356 mds_command: new command op to 24514 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:33:33.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.662+0000 7f58c73c3b80 1 -- 172.21.15.1:0/317145664 --> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bdd679cdd0 con 0x55bdd6875580 2024-02-02T15:33:33.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.663+0000 7f58abe00700 1 --2- 172.21.15.1:0/317145664 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55bdd6875580 0x55bdd6877970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:33.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.663+0000 7f58abe00700 1 --2- 172.21.15.1:0/317145664 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55bdd6875580 0x55bdd6877970 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bf1eb7cf76f321b6 server_cookie=7fe179b8d878a5e5 in_seq=0 out_seq=0 2024-02-02T15:33:33.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.663+0000 7f589aa00700 10 client.27356 ms_handle_connect on v2:172.21.15.142:6836/2870289019 2024-02-02T15:33:33.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.664+0000 7f589aa00700 1 -- 172.21.15.1:0/317145664 <== mds.3 v2:172.21.15.142:6836/2870289019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55bdd679cdd0 con 0x55bdd6875580 2024-02-02T15:33:33.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.664+0000 7f589aa00700 10 client.27356 handle_command_reply: tid=0 2024-02-02T15:33:33.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.664+0000 7f58c73c3b80 1 client.27356 shutdown 2024-02-02T15:33:33.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.664+0000 7f58c73c3b80 1 -- 172.21.15.1:0/317145664 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bdd6852010 msgr2=0x55bdd685d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:33.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.664+0000 7f58c73c3b80 1 --2- 172.21.15.1:0/317145664 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bdd6852010 0x55bdd685d280 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f589c0c96b0 tx=0x7f589c010aa0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.664+0000 7f58c73c3b80 1 -- 172.21.15.1:0/317145664 shutdown_connections 2024-02-02T15:33:33.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.664+0000 7f58c73c3b80 1 --2- 172.21.15.1:0/317145664 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bdd685d7e0 0x55bdd685abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.664+0000 7f58c73c3b80 1 --2- 172.21.15.1:0/317145664 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bdd6852010 0x55bdd685d280 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.664+0000 7f58c73c3b80 1 --2- 172.21.15.1:0/317145664 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55bdd6875580 0x55bdd6877970 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.664+0000 7f58c73c3b80 1 --2- 172.21.15.1:0/317145664 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bdd6852930 0x55bdd685a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.664+0000 7f589aa00700 0 client.27356 ms_handle_reset on v2:172.21.15.142:6836/2870289019 2024-02-02T15:33:33.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.664+0000 7f58c73c3b80 1 -- 172.21.15.1:0/317145664 >> 172.21.15.1:0/317145664 conn(0x55bdd67b1db0 msgr2=0x55bdd684dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:33.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.665+0000 7f58c73c3b80 1 -- 172.21.15.1:0/317145664 shutdown_connections 2024-02-02T15:33:33.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.665+0000 7f58c73c3b80 1 -- 172.21.15.1:0/317145664 wait complete. 2024-02-02T15:33:33.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.665+0000 7f58c73c3b80 20 client.27356 trim_cache size 0 max 16384 2024-02-02T15:33:33.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.678+0000 7f58c73c3b80 1 Processor -- start 2024-02-02T15:33:33.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.678+0000 7f58c73c3b80 1 -- start start 2024-02-02T15:33:33.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.678+0000 7f58c73c3b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bdd6852010 0x55bdd685cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:33.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.678+0000 7f58c73c3b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bdd6852930 0x55bdd6859d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:33.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.678+0000 7f58c73c3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bdd685d7e0 0x55bdd685a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:33.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.678+0000 7f58c73c3b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdd66276a0 con 0x55bdd6852010 2024-02-02T15:33:33.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.678+0000 7f58c73c3b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdd684f250 con 0x55bdd6852930 2024-02-02T15:33:33.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.678+0000 7f58c73c3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdd6852d10 con 0x55bdd685d7e0 2024-02-02T15:33:33.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.678+0000 7f58b0e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bdd6852010 0x55bdd685cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:33.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.679+0000 7f58b0e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bdd6852010 0x55bdd685cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57876/0 (socket says 172.21.15.1:57876) 2024-02-02T15:33:33.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.679+0000 7f58b0e00700 1 -- 172.21.15.1:0/4024110411 learned_addr learned my addr 172.21.15.1:0/4024110411 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:33:33.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.679+0000 7f58abe00700 1 --2- 172.21.15.1:0/4024110411 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bdd6852930 0x55bdd6859d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:33.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.679+0000 7f58b1800700 1 --2- 172.21.15.1:0/4024110411 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bdd685d7e0 0x55bdd685a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:33.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.679+0000 7f58b0e00700 1 -- 172.21.15.1:0/4024110411 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bdd685d7e0 msgr2=0x55bdd685a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:33.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.679+0000 7f58b0e00700 1 --2- 172.21.15.1:0/4024110411 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bdd685d7e0 0x55bdd685a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.679+0000 7f58b0e00700 1 -- 172.21.15.1:0/4024110411 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bdd6852930 msgr2=0x55bdd6859d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:33.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.679+0000 7f58b0e00700 1 --2- 172.21.15.1:0/4024110411 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bdd6852930 0x55bdd6859d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.679+0000 7f58b0e00700 1 -- 172.21.15.1:0/4024110411 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bdd6769ea0 con 0x55bdd6852010 2024-02-02T15:33:33.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.679+0000 7f58abe00700 1 --2- 172.21.15.1:0/4024110411 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bdd6852930 0x55bdd6859d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:33:33.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.679+0000 7f58b0e00700 1 --2- 172.21.15.1:0/4024110411 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bdd6852010 0x55bdd685cdb0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f589c099180 tx=0x7f589c097a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52f80f296b77fa33 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:33:33.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.679+0000 7f58a2a00700 1 -- 172.21.15.1:0/4024110411 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589c0a14f0 con 0x55bdd6852010 2024-02-02T15:33:33.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.679+0000 7f58a2a00700 1 -- 172.21.15.1:0/4024110411 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f589c099480 con 0x55bdd6852010 2024-02-02T15:33:33.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.679+0000 7f58a2a00700 1 -- 172.21.15.1:0/4024110411 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589c0a1060 con 0x55bdd6852010 2024-02-02T15:33:33.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.680+0000 7f58c73c3b80 1 -- 172.21.15.1:0/4024110411 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bdd6852010 msgr2=0x55bdd685cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:33.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.680+0000 7f58c73c3b80 1 --2- 172.21.15.1:0/4024110411 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bdd6852010 0x55bdd685cdb0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f589c099180 tx=0x7f589c097a30 comp rx=0 tx=0).stop 2024-02-02T15:33:33.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.680+0000 7f58c73c3b80 1 -- 172.21.15.1:0/4024110411 shutdown_connections 2024-02-02T15:33:33.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.680+0000 7f58c73c3b80 1 --2- 172.21.15.1:0/4024110411 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bdd685d7e0 0x55bdd685a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.680+0000 7f58c73c3b80 1 --2- 172.21.15.1:0/4024110411 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bdd6852010 0x55bdd685cdb0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.680+0000 7f58c73c3b80 1 --2- 172.21.15.1:0/4024110411 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bdd6852930 0x55bdd6859d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.680+0000 7f58c73c3b80 1 -- 172.21.15.1:0/4024110411 >> 172.21.15.1:0/4024110411 conn(0x55bdd67b1db0 msgr2=0x55bdd6871e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:33.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.680+0000 7f58c73c3b80 1 -- 172.21.15.1:0/4024110411 shutdown_connections 2024-02-02T15:33:33.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.680+0000 7f58c73c3b80 1 -- 172.21.15.1:0/4024110411 wait complete. 2024-02-02T15:33:33.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.681+0000 7f58c73c3b80 1 -- start start 2024-02-02T15:33:33.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.681+0000 7f58c73c3b80 1 Processor -- start 2024-02-02T15:33:33.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.681+0000 7f58c73c3b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bdd6852010 0x55bdd686ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:33.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.681+0000 7f58c73c3b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bdd6852930 0x55bdd686b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:33.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.681+0000 7f58c73c3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bdd685d7e0 0x55bdd68c5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:33.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.681+0000 7f58c73c3b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdd68c64d0 con 0x55bdd6852010 2024-02-02T15:33:33.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.681+0000 7f58c73c3b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdd68c6610 con 0x55bdd6852930 2024-02-02T15:33:33.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.681+0000 7f58c73c3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdd68c6780 con 0x55bdd685d7e0 2024-02-02T15:33:33.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.681+0000 7f58c73c3b80 10 client.0 _ll_register_callbacks cb 0x55bdd6741290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:33:33.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.682+0000 7f58b0e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bdd6852010 0x55bdd686ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:33.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.682+0000 7f58b0e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bdd6852010 0x55bdd686ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57882/0 (socket says 172.21.15.1:57882) 2024-02-02T15:33:33.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.682+0000 7f58b0e00700 1 -- 172.21.15.1:0/2830303462 learned_addr learned my addr 172.21.15.1:0/2830303462 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:33:33.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.682+0000 7f58abe00700 1 --2- 172.21.15.1:0/2830303462 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bdd6852930 0x55bdd686b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:33.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.682+0000 7f58b1800700 1 --2- 172.21.15.1:0/2830303462 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bdd685d7e0 0x55bdd68c5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:33.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.682+0000 7f58b0e00700 1 -- 172.21.15.1:0/2830303462 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bdd685d7e0 msgr2=0x55bdd68c5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:33.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.682+0000 7f58b0e00700 1 --2- 172.21.15.1:0/2830303462 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bdd685d7e0 0x55bdd68c5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.682+0000 7f58b0e00700 1 -- 172.21.15.1:0/2830303462 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bdd6852930 msgr2=0x55bdd686b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:33.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.682+0000 7f58b0e00700 1 --2- 172.21.15.1:0/2830303462 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bdd6852930 0x55bdd686b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.682+0000 7f58b0e00700 1 -- 172.21.15.1:0/2830303462 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bdd65d1b60 con 0x55bdd6852010 2024-02-02T15:33:33.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.682+0000 7f58b0e00700 1 --2- 172.21.15.1:0/2830303462 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bdd6852010 0x55bdd686ad10 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f589c095980 tx=0x7f589c0a1270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7de699bae554ea71 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:33:33.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.682+0000 7f589aa00700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:33:33.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.683+0000 7f589aa00700 1 -- 172.21.15.1:0/2830303462 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589c0b3df0 con 0x55bdd6852010 2024-02-02T15:33:33.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.683+0000 7f589aa00700 1 -- 172.21.15.1:0/2830303462 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f589c0a14f0 con 0x55bdd6852010 2024-02-02T15:33:33.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.683+0000 7f58c73c3b80 1 -- 172.21.15.1:0/2830303462 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f589c09d2b0 con 0x55bdd6852010 2024-02-02T15:33:33.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.683+0000 7f589aa00700 1 -- 172.21.15.1:0/2830303462 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f589c0998f0 con 0x55bdd6852010 2024-02-02T15:33:33.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.683+0000 7f58c73c3b80 10 client.17301 fetch_fsmap learned FSMap version 38 2024-02-02T15:33:33.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.683+0000 7f58c73c3b80 1 -- 172.21.15.1:0/2830303462 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55bdd6629cd0 con 0x55bdd6852010 2024-02-02T15:33:33.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.683+0000 7f589aa00700 1 -- 172.21.15.1:0/2830303462 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f589c09d2b0 con 0x55bdd6852010 2024-02-02T15:33:33.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.683+0000 7f589aa00700 1 -- 172.21.15.1:0/2830303462 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 38) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f589c09dff0 con 0x55bdd6852010 2024-02-02T15:33:33.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.683+0000 7f58c73c3b80 10 client.17301 fetch_fsmap finished waiting for FSMap version 38 2024-02-02T15:33:33.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.683+0000 7f58c73c3b80 10 client.17301 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-02-02T15:33:33.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.683+0000 7f58c73c3b80 20 client.17301 populate_metadata read hostname 'smithi001' 2024-02-02T15:33:33.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.683+0000 7f58c73c3b80 1 --2- 172.21.15.1:0/2830303462 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55bdd68c7a90 0x55bdd68c9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:33.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.683+0000 7f58c73c3b80 4 client.17301 mds_command: new command op to 24514 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:33:33.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.683+0000 7f58c73c3b80 1 -- 172.21.15.1:0/2830303462 --> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55bdd6762f50 con 0x55bdd68c7a90 2024-02-02T15:33:33.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.684+0000 7f58abe00700 1 --2- 172.21.15.1:0/2830303462 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55bdd68c7a90 0x55bdd68c9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:33.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.684+0000 7f58abe00700 1 --2- 172.21.15.1:0/2830303462 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55bdd68c7a90 0x55bdd68c9eb0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1357ac8d74a49ee5 server_cookie=19934647c18984b3 in_seq=0 out_seq=0 2024-02-02T15:33:33.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.684+0000 7f589aa00700 10 client.17301 ms_handle_connect on v2:172.21.15.142:6836/2870289019 2024-02-02T15:33:33.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.685+0000 7f589aa00700 1 -- 172.21.15.1:0/2830303462 <== mds.3 v2:172.21.15.142:6836/2870289019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55bdd6762f50 con 0x55bdd68c7a90 2024-02-02T15:33:33.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.685+0000 7f589aa00700 10 client.17301 handle_command_reply: tid=0 2024-02-02T15:33:33.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.685+0000 7f58c73c3b80 1 client.17301 shutdown 2024-02-02T15:33:33.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.685+0000 7f58c73c3b80 1 -- 172.21.15.1:0/2830303462 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bdd6852010 msgr2=0x55bdd686ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:33.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.685+0000 7f58c73c3b80 1 --2- 172.21.15.1:0/2830303462 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bdd6852010 0x55bdd686ad10 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f589c095980 tx=0x7f589c0a1270 comp rx=0 tx=0).stop 2024-02-02T15:33:33.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.685+0000 7f58c73c3b80 1 -- 172.21.15.1:0/2830303462 shutdown_connections 2024-02-02T15:33:33.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.685+0000 7f58c73c3b80 1 --2- 172.21.15.1:0/2830303462 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bdd685d7e0 0x55bdd68c5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.685+0000 7f58c73c3b80 1 --2- 172.21.15.1:0/2830303462 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55bdd68c7a90 0x55bdd68c9eb0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.685+0000 7f58c73c3b80 1 --2- 172.21.15.1:0/2830303462 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bdd6852010 0x55bdd686ad10 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.685+0000 7f58c73c3b80 1 --2- 172.21.15.1:0/2830303462 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bdd6852930 0x55bdd686b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.685+0000 7f589aa00700 0 client.17301 ms_handle_reset on v2:172.21.15.142:6836/2870289019 2024-02-02T15:33:33.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.685+0000 7f58c73c3b80 1 -- 172.21.15.1:0/2830303462 >> 172.21.15.1:0/2830303462 conn(0x55bdd67b1db0 msgr2=0x55bdd6859d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:33.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.685+0000 7f58c73c3b80 1 -- 172.21.15.1:0/2830303462 shutdown_connections 2024-02-02T15:33:33.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.685+0000 7f58c73c3b80 1 -- 172.21.15.1:0/2830303462 wait complete. 2024-02-02T15:33:33.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.685+0000 7f58c73c3b80 20 client.17301 trim_cache size 0 max 16384 2024-02-02T15:33:33.689 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:33:33.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.686+0000 7f58b3800700 1 -- 172.21.15.1:0/2222495477 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f588c0a0690 msgr2=0x7f588c0a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:33.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.686+0000 7f58b3800700 1 --2- 172.21.15.1:0/2222495477 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f588c0a0690 0x7f588c0a2b50 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f58a4007770 tx=0x7f58a4007b30 comp rx=0 tx=0).stop 2024-02-02T15:33:33.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.686+0000 7f58b3800700 1 -- 172.21.15.1:0/2222495477 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f58ac068010 msgr2=0x7f58ac199100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:33.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.686+0000 7f58b3800700 1 --2- 172.21.15.1:0/2222495477 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f58ac068010 0x7f58ac199100 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f589c00e9a0 tx=0x7f589c00ecb0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.686+0000 7f58b3800700 1 -- 172.21.15.1:0/2222495477 shutdown_connections 2024-02-02T15:33:33.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.686+0000 7f58b3800700 1 --2- 172.21.15.1:0/2222495477 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f588c0a0690 0x7f588c0a2b50 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.686+0000 7f58b3800700 1 --2- 172.21.15.1:0/2222495477 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f58ac106ed0 0x7f58ac19da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.686+0000 7f58b3800700 1 --2- 172.21.15.1:0/2222495477 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f58ac068010 0x7f58ac199100 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.686+0000 7f58b3800700 1 --2- 172.21.15.1:0/2222495477 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f58ac0689b0 0x7f58ac199640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.686+0000 7f58b3800700 1 -- 172.21.15.1:0/2222495477 >> 172.21.15.1:0/2222495477 conn(0x7f58ac0757d0 msgr2=0x7f58ac0fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:33.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.686+0000 7f58b3800700 1 -- 172.21.15.1:0/2222495477 shutdown_connections 2024-02-02T15:33:33.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.686+0000 7f58b3800700 1 -- 172.21.15.1:0/2222495477 wait complete. 2024-02-02T15:33:33.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:33 smithi001 ceph-mon[35470]: pgmap v199: 129 pgs: 129 active+clean; 269 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.9 MiB/s wr, 242 op/s 2024-02-02T15:33:33.701 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:33 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/3502498195' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:33:33.704 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-02T15:33:33.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:33 smithi071 ceph-mon[44707]: pgmap v199: 129 pgs: 129 active+clean; 269 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.9 MiB/s wr, 242 op/s 2024-02-02T15:33:33.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:33 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/3502498195' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:33:33.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:33 smithi142 ceph-mon[45728]: pgmap v199: 129 pgs: 129 active+clean; 269 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.9 MiB/s wr, 242 op/s 2024-02-02T15:33:33.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:33 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/3502498195' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:33:33.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.929+0000 7f0c73600700 1 -- 172.21.15.1:0/2586416381 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c6c100ff0 msgr2=0x7f0c6c1013d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:33.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.929+0000 7f0c73600700 1 --2- 172.21.15.1:0/2586416381 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c6c100ff0 0x7f0c6c1013d0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c009990 tx=0x7f0c5c009ca0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.929+0000 7f0c73600700 1 -- 172.21.15.1:0/2586416381 shutdown_connections 2024-02-02T15:33:33.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.929+0000 7f0c73600700 1 --2- 172.21.15.1:0/2586416381 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c6c106110 0x7f0c6c10dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.929+0000 7f0c73600700 1 --2- 172.21.15.1:0/2586416381 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c6c100ff0 0x7f0c6c1013d0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.929+0000 7f0c73600700 1 --2- 172.21.15.1:0/2586416381 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c6c1019a0 0x7f0c6c1059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.929+0000 7f0c73600700 1 -- 172.21.15.1:0/2586416381 >> 172.21.15.1:0/2586416381 conn(0x7f0c6c0fc810 msgr2=0x7f0c6c0fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:33.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.929+0000 7f0c73600700 1 -- 172.21.15.1:0/2586416381 shutdown_connections 2024-02-02T15:33:33.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.929+0000 7f0c73600700 1 -- 172.21.15.1:0/2586416381 wait complete. 2024-02-02T15:33:33.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.930+0000 7f0c73600700 1 Processor -- start 2024-02-02T15:33:33.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.930+0000 7f0c73600700 1 -- start start 2024-02-02T15:33:33.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.931+0000 7f0c73600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c6c100ff0 0x7f0c6c073040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:33.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.931+0000 7f0c73600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c6c106110 0x7f0c6c073580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:33.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.931+0000 7f0c73600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c6c073ac0 0x7f0c6c1a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:33.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.931+0000 7f0c73600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c6c1a56d0 con 0x7f0c6c073ac0 2024-02-02T15:33:33.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.931+0000 7f0c73600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c6c1a5840 con 0x7f0c6c100ff0 2024-02-02T15:33:33.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.931+0000 7f0c73600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c6c1a5a00 con 0x7f0c6c106110 2024-02-02T15:33:33.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.931+0000 7f0c70c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c6c100ff0 0x7f0c6c073040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:33.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.931+0000 7f0c70c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c6c100ff0 0x7f0c6c073040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:50218/0 (socket says 172.21.15.1:50218) 2024-02-02T15:33:33.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.931+0000 7f0c70c00700 1 -- 172.21.15.1:0/1946365574 learned_addr learned my addr 172.21.15.1:0/1946365574 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:33:33.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.931+0000 7f0c71600700 1 --2- 172.21.15.1:0/1946365574 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c6c073ac0 0x7f0c6c1a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:33.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.931+0000 7f0c6be00700 1 --2- 172.21.15.1:0/1946365574 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c6c106110 0x7f0c6c073580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:33.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.931+0000 7f0c70c00700 1 -- 172.21.15.1:0/1946365574 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c6c106110 msgr2=0x7f0c6c073580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:33.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.931+0000 7f0c70c00700 1 --2- 172.21.15.1:0/1946365574 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c6c106110 0x7f0c6c073580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.931+0000 7f0c70c00700 1 -- 172.21.15.1:0/1946365574 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c6c073ac0 msgr2=0x7f0c6c1a5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:33.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.931+0000 7f0c70c00700 1 --2- 172.21.15.1:0/1946365574 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c6c073ac0 0x7f0c6c1a5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.931+0000 7f0c70c00700 1 -- 172.21.15.1:0/1946365574 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c5c0095f0 con 0x7f0c6c100ff0 2024-02-02T15:33:33.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.932+0000 7f0c70c00700 1 --2- 172.21.15.1:0/1946365574 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c6c100ff0 0x7f0c6c073040 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c005910 tx=0x7f0c5c005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:33:33.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.932+0000 7f0c69600700 1 -- 172.21.15.1:0/1946365574 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c5c01d070 con 0x7f0c6c100ff0 2024-02-02T15:33:33.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.932+0000 7f0c69600700 1 -- 172.21.15.1:0/1946365574 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0c5c022470 con 0x7f0c6c100ff0 2024-02-02T15:33:33.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.932+0000 7f0c69600700 1 -- 172.21.15.1:0/1946365574 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c5c017650 con 0x7f0c6c100ff0 2024-02-02T15:33:33.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.932+0000 7f0c73600700 1 -- 172.21.15.1:0/1946365574 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c6c1a5c80 con 0x7f0c6c100ff0 2024-02-02T15:33:33.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.932+0000 7f0c73600700 1 -- 172.21.15.1:0/1946365574 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c6c1a60b0 con 0x7f0c6c100ff0 2024-02-02T15:33:33.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.936+0000 7f0c69600700 1 -- 172.21.15.1:0/1946365574 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 27) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f0c5c00fad0 con 0x7f0c6c100ff0 2024-02-02T15:33:33.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.937+0000 7f0c870cab80 1 Processor -- start 2024-02-02T15:33:33.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.937+0000 7f0c870cab80 1 -- start start 2024-02-02T15:33:33.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.938+0000 7f0c69600700 1 --2- 172.21.15.1:0/1946365574 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f0c4c0a0710 0x7f0c4c0a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:33.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.938+0000 7f0c69600700 1 -- 172.21.15.1:0/1946365574 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f0c5c0cd450 con 0x7f0c6c100ff0 2024-02-02T15:33:33.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.938+0000 7f0c870cab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556182b41010 0x556182b413f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:33.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.938+0000 7f0c870cab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556182b41930 0x556182b4c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:33.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.938+0000 7f0c870cab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556182b4c7e0 0x556182b4ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:33.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.938+0000 7f0c870cab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x556182ab54c0 con 0x556182b41010 2024-02-02T15:33:33.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.938+0000 7f0c870cab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x556182a99aa0 con 0x556182b41930 2024-02-02T15:33:33.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.938+0000 7f0c870cab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556182a97460 con 0x556182b4c7e0 2024-02-02T15:33:33.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.938+0000 7f0c71600700 1 --2- 172.21.15.1:0/1946365574 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f0c4c0a0710 0x7f0c4c0a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:33.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.938+0000 7f0c70c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556182b41010 0x556182b413f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:33.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.938+0000 7f0c70c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556182b41010 0x556182b413f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57906/0 (socket says 172.21.15.1:57906) 2024-02-02T15:33:33.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.938+0000 7f0c70c00700 1 -- 172.21.15.1:0/3144080254 learned_addr learned my addr 172.21.15.1:0/3144080254 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:33:33.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.938+0000 7f0c71600700 1 --2- 172.21.15.1:0/3144080254 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556182b4c7e0 0x556182b4ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:33.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.938+0000 7f0c6be00700 1 --2- 172.21.15.1:0/3144080254 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556182b41930 0x556182b4c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:33.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.938+0000 7f0c70c00700 1 -- 172.21.15.1:0/3144080254 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556182b4c7e0 msgr2=0x556182b4ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:33.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.939+0000 7f0c70c00700 1 --2- 172.21.15.1:0/3144080254 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556182b4c7e0 0x556182b4ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.939+0000 7f0c70c00700 1 -- 172.21.15.1:0/3144080254 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556182b41930 msgr2=0x556182b4c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:33.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.939+0000 7f0c70c00700 1 --2- 172.21.15.1:0/3144080254 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556182b41930 0x556182b4c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.939+0000 7f0c70c00700 1 -- 172.21.15.1:0/3144080254 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5561829143b0 con 0x556182b41010 2024-02-02T15:33:33.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.939+0000 7f0c6be00700 1 --2- 172.21.15.1:0/3144080254 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556182b41930 0x556182b4c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:33:33.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.939+0000 7f0c71600700 1 --2- 172.21.15.1:0/1946365574 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f0c4c0a0710 0x7f0c4c0a2bd0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0c64003fd0 tx=0x7f0c64003dc0 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:33:33.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.939+0000 7f0c70c00700 1 --2- 172.21.15.1:0/3144080254 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556182b41010 0x556182b413f0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c09a640 tx=0x7f0c5c09a500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf5a516e07c9e6ea server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:33:33.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.939+0000 7f0c61c00700 1 -- 172.21.15.1:0/3144080254 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c5c09adb0 con 0x556182b41010 2024-02-02T15:33:33.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.939+0000 7f0c61c00700 1 -- 172.21.15.1:0/3144080254 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0c5c0b0070 con 0x556182b41010 2024-02-02T15:33:33.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.939+0000 7f0c61c00700 1 -- 172.21.15.1:0/3144080254 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c5c022850 con 0x556182b41010 2024-02-02T15:33:33.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.940+0000 7f0c870cab80 1 -- 172.21.15.1:0/3144080254 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556182b41010 msgr2=0x556182b413f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:33.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.940+0000 7f0c870cab80 1 --2- 172.21.15.1:0/3144080254 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556182b41010 0x556182b413f0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c09a640 tx=0x7f0c5c09a500 comp rx=0 tx=0).stop 2024-02-02T15:33:33.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.940+0000 7f0c870cab80 1 -- 172.21.15.1:0/3144080254 shutdown_connections 2024-02-02T15:33:33.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.940+0000 7f0c870cab80 1 --2- 172.21.15.1:0/3144080254 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556182b4c7e0 0x556182b4ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.940+0000 7f0c870cab80 1 --2- 172.21.15.1:0/3144080254 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556182b41010 0x556182b413f0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.940+0000 7f0c870cab80 1 --2- 172.21.15.1:0/3144080254 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556182b41930 0x556182b4c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.940+0000 7f0c870cab80 1 -- 172.21.15.1:0/3144080254 >> 172.21.15.1:0/3144080254 conn(0x556182aa0db0 msgr2=0x556182b3d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:33.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.940+0000 7f0c870cab80 1 -- 172.21.15.1:0/3144080254 shutdown_connections 2024-02-02T15:33:33.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.940+0000 7f0c870cab80 1 -- 172.21.15.1:0/3144080254 wait complete. 2024-02-02T15:33:33.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.940+0000 7f0c870cab80 1 -- start start 2024-02-02T15:33:33.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.941+0000 7f0c870cab80 1 Processor -- start 2024-02-02T15:33:33.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.941+0000 7f0c870cab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556182b41010 0x556182b4cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:33.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.941+0000 7f0c870cab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556182b41930 0x556182b4d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:33.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.941+0000 7f0c870cab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556182b4c7e0 0x556182b4d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:33.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.941+0000 7f0c870cab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x556182a92d50 con 0x556182b41930 2024-02-02T15:33:33.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.941+0000 7f0c870cab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x556182a2ed80 con 0x556182b41010 2024-02-02T15:33:33.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.941+0000 7f0c870cab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556182a2ea20 con 0x556182b4c7e0 2024-02-02T15:33:33.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.941+0000 7f0c870cab80 10 client.0 _ll_register_callbacks cb 0x556182a30290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:33:33.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.942+0000 7f0c70c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556182b41010 0x556182b4cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:33.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.942+0000 7f0c70c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556182b41010 0x556182b4cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:50234/0 (socket says 172.21.15.1:50234) 2024-02-02T15:33:33.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.942+0000 7f0c70c00700 1 -- 172.21.15.1:0/1615417521 learned_addr learned my addr 172.21.15.1:0/1615417521 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:33:33.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.942+0000 7f0c71600700 1 --2- 172.21.15.1:0/1615417521 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556182b4c7e0 0x556182b4d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:33.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.942+0000 7f0c6be00700 1 --2- 172.21.15.1:0/1615417521 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556182b41930 0x556182b4d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:33.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.942+0000 7f0c70c00700 1 -- 172.21.15.1:0/1615417521 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556182b4c7e0 msgr2=0x556182b4d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:33.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.942+0000 7f0c70c00700 1 --2- 172.21.15.1:0/1615417521 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556182b4c7e0 0x556182b4d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.942+0000 7f0c70c00700 1 -- 172.21.15.1:0/1615417521 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556182b41930 msgr2=0x556182b4d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:33.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.942+0000 7f0c70c00700 1 --2- 172.21.15.1:0/1615417521 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556182b41930 0x556182b4d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.942+0000 7f0c70c00700 1 -- 172.21.15.1:0/1615417521 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c540098e0 con 0x556182b41010 2024-02-02T15:33:33.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.942+0000 7f0c70c00700 1 --2- 172.21.15.1:0/1615417521 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556182b41010 0x556182b4cbc0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c097120 tx=0x7f0c5c0ca7c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8080238db73dfec1 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:33:33.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.942+0000 7f0c5a000700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:33:33.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.943+0000 7f0c5a000700 1 -- 172.21.15.1:0/1615417521 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c5c09a6b0 con 0x556182b41010 2024-02-02T15:33:33.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.943+0000 7f0c870cab80 1 -- 172.21.15.1:0/1615417521 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556182b350f0 con 0x556182b41010 2024-02-02T15:33:33.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.943+0000 7f0c5a000700 1 -- 172.21.15.1:0/1615417521 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f0c5c09e4e0 con 0x556182b41010 2024-02-02T15:33:33.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.943+0000 7f0c5a000700 1 -- 172.21.15.1:0/1615417521 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c5c098a50 con 0x556182b41010 2024-02-02T15:33:33.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.943+0000 7f0c870cab80 10 client.27374 fetch_fsmap learned FSMap version 38 2024-02-02T15:33:33.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.943+0000 7f0c870cab80 1 -- 172.21.15.1:0/1615417521 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x7f0c6c1a60b0 con 0x556182b41010 2024-02-02T15:33:33.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.943+0000 7f0c5a000700 1 -- 172.21.15.1:0/1615417521 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556182b350f0 con 0x556182b41010 2024-02-02T15:33:33.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.943+0000 7f0c5a000700 1 -- 172.21.15.1:0/1615417521 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 38) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f0c5c09a910 con 0x556182b41010 2024-02-02T15:33:33.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.943+0000 7f0c870cab80 10 client.27374 fetch_fsmap finished waiting for FSMap version 38 2024-02-02T15:33:33.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.943+0000 7f0c870cab80 10 client.27374 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-02-02T15:33:33.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.944+0000 7f0c870cab80 20 client.27374 populate_metadata read hostname 'smithi001' 2024-02-02T15:33:33.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.944+0000 7f0c870cab80 1 --2- 172.21.15.1:0/1615417521 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x556182b5ba20 0x556182b5de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:33.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.944+0000 7f0c870cab80 4 client.27374 mds_command: new command op to 24518 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:33:33.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.944+0000 7f0c870cab80 1 -- 172.21.15.1:0/1615417521 --> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556182a8bdd0 con 0x556182b5ba20 2024-02-02T15:33:33.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.944+0000 7f0c6be00700 1 --2- 172.21.15.1:0/1615417521 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x556182b5ba20 0x556182b5de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:33.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.944+0000 7f0c6be00700 1 --2- 172.21.15.1:0/1615417521 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x556182b5ba20 0x556182b5de40 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3d619d9d31dfea4a server_cookie=25b508308985d97f in_seq=0 out_seq=0 2024-02-02T15:33:33.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.944+0000 7f0c5a000700 10 client.27374 ms_handle_connect on v2:172.21.15.71:6836/628016199 2024-02-02T15:33:33.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.945+0000 7f0c5a000700 1 -- 172.21.15.1:0/1615417521 <== mds.2 v2:172.21.15.71:6836/628016199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x556182a8bdd0 con 0x556182b5ba20 2024-02-02T15:33:33.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.945+0000 7f0c5a000700 10 client.27374 handle_command_reply: tid=0 2024-02-02T15:33:33.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.945+0000 7f0c870cab80 1 client.27374 shutdown 2024-02-02T15:33:33.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.945+0000 7f0c870cab80 1 -- 172.21.15.1:0/1615417521 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556182b41010 msgr2=0x556182b4cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:33.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.945+0000 7f0c870cab80 1 --2- 172.21.15.1:0/1615417521 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556182b41010 0x556182b4cbc0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c097120 tx=0x7f0c5c0ca7c0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.945+0000 7f0c870cab80 1 -- 172.21.15.1:0/1615417521 shutdown_connections 2024-02-02T15:33:33.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.945+0000 7f0c870cab80 1 --2- 172.21.15.1:0/1615417521 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x556182b5ba20 0x556182b5de40 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.945+0000 7f0c870cab80 1 --2- 172.21.15.1:0/1615417521 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556182b4c7e0 0x556182b4d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.945+0000 7f0c870cab80 1 --2- 172.21.15.1:0/1615417521 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556182b41010 0x556182b4cbc0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.945+0000 7f0c870cab80 1 --2- 172.21.15.1:0/1615417521 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556182b41930 0x556182b4d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.945+0000 7f0c5a000700 0 client.27374 ms_handle_reset on v2:172.21.15.71:6836/628016199 2024-02-02T15:33:33.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.945+0000 7f0c870cab80 1 -- 172.21.15.1:0/1615417521 >> 172.21.15.1:0/1615417521 conn(0x556182aa0db0 msgr2=0x556182b3cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:33.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.946+0000 7f0c870cab80 1 -- 172.21.15.1:0/1615417521 shutdown_connections 2024-02-02T15:33:33.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.946+0000 7f0c870cab80 1 -- 172.21.15.1:0/1615417521 wait complete. 2024-02-02T15:33:33.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.946+0000 7f0c870cab80 20 client.27374 trim_cache size 0 max 16384 2024-02-02T15:33:33.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.959+0000 7f0c870cab80 1 Processor -- start 2024-02-02T15:33:33.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.959+0000 7f0c870cab80 1 -- start start 2024-02-02T15:33:33.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.959+0000 7f0c870cab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556182b41010 0x556182b4d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:33.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.959+0000 7f0c870cab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556182b41930 0x556182b571b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:33.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.959+0000 7f0c870cab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556182b4c7e0 0x556182b576f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:33.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.959+0000 7f0c870cab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561829166a0 con 0x556182b4c7e0 2024-02-02T15:33:33.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.959+0000 7f0c870cab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x556182b3e7e0 con 0x556182b41010 2024-02-02T15:33:33.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.959+0000 7f0c870cab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556182b49db0 con 0x556182b41930 2024-02-02T15:33:33.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.960+0000 7f0c71600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556182b4c7e0 0x556182b576f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:33.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.960+0000 7f0c71600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556182b4c7e0 0x556182b576f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57914/0 (socket says 172.21.15.1:57914) 2024-02-02T15:33:33.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.960+0000 7f0c71600700 1 -- 172.21.15.1:0/2363578533 learned_addr learned my addr 172.21.15.1:0/2363578533 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:33:33.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.960+0000 7f0c70c00700 1 --2- 172.21.15.1:0/2363578533 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556182b41010 0x556182b4d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:33.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.960+0000 7f0c6be00700 1 --2- 172.21.15.1:0/2363578533 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556182b41930 0x556182b571b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:33.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.960+0000 7f0c71600700 1 -- 172.21.15.1:0/2363578533 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556182b41930 msgr2=0x556182b571b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:33.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.960+0000 7f0c71600700 1 --2- 172.21.15.1:0/2363578533 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556182b41930 0x556182b571b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.960+0000 7f0c71600700 1 -- 172.21.15.1:0/2363578533 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556182b41010 msgr2=0x556182b4d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:33.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.960+0000 7f0c71600700 1 --2- 172.21.15.1:0/2363578533 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556182b41010 0x556182b4d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.960+0000 7f0c71600700 1 -- 172.21.15.1:0/2363578533 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5561828c0b60 con 0x556182b4c7e0 2024-02-02T15:33:33.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.960+0000 7f0c70c00700 1 --2- 172.21.15.1:0/2363578533 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556182b41010 0x556182b4d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:33:33.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.960+0000 7f0c71600700 1 --2- 172.21.15.1:0/2363578533 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556182b4c7e0 0x556182b576f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f0c64007630 tx=0x7f0c6401c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96eba45b353abb4d server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:33:33.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.961+0000 7f0c62600700 1 -- 172.21.15.1:0/2363578533 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c6401b450 con 0x556182b4c7e0 2024-02-02T15:33:33.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.961+0000 7f0c62600700 1 -- 172.21.15.1:0/2363578533 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0c6400f040 con 0x556182b4c7e0 2024-02-02T15:33:33.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.961+0000 7f0c62600700 1 -- 172.21.15.1:0/2363578533 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c64025710 con 0x556182b4c7e0 2024-02-02T15:33:33.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.961+0000 7f0c870cab80 1 -- 172.21.15.1:0/2363578533 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556182b4c7e0 msgr2=0x556182b576f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:33.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.961+0000 7f0c870cab80 1 --2- 172.21.15.1:0/2363578533 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556182b4c7e0 0x556182b576f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f0c64007630 tx=0x7f0c6401c000 comp rx=0 tx=0).stop 2024-02-02T15:33:33.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.961+0000 7f0c870cab80 1 -- 172.21.15.1:0/2363578533 shutdown_connections 2024-02-02T15:33:33.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.961+0000 7f0c870cab80 1 --2- 172.21.15.1:0/2363578533 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556182b4c7e0 0x556182b576f0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.961+0000 7f0c870cab80 1 --2- 172.21.15.1:0/2363578533 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556182b41010 0x556182b4d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.961+0000 7f0c870cab80 1 --2- 172.21.15.1:0/2363578533 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556182b41930 0x556182b571b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.961+0000 7f0c870cab80 1 -- 172.21.15.1:0/2363578533 >> 172.21.15.1:0/2363578533 conn(0x556182aa0db0 msgr2=0x556182b4cbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:33.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.961+0000 7f0c870cab80 1 -- 172.21.15.1:0/2363578533 shutdown_connections 2024-02-02T15:33:33.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.961+0000 7f0c870cab80 1 -- 172.21.15.1:0/2363578533 wait complete. 2024-02-02T15:33:33.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.962+0000 7f0c870cab80 1 -- start start 2024-02-02T15:33:33.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.962+0000 7f0c870cab80 1 Processor -- start 2024-02-02T15:33:33.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.962+0000 7f0c870cab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556182b41010 0x556182b4f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:33.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.962+0000 7f0c870cab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556182b41930 0x556182b4fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:33.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.963+0000 7f0c870cab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556182b4c7e0 0x556182ba64d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:33.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.963+0000 7f0c870cab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x556182ba6a10 con 0x556182b41930 2024-02-02T15:33:33.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.963+0000 7f0c870cab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x556182ba6b50 con 0x556182b41010 2024-02-02T15:33:33.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.963+0000 7f0c870cab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556182ba6d10 con 0x556182b4c7e0 2024-02-02T15:33:33.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.963+0000 7f0c870cab80 10 client.0 _ll_register_callbacks cb 0x556182a30290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:33:33.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.963+0000 7f0c6be00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556182b41930 0x556182b4fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:33.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.963+0000 7f0c70c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556182b41010 0x556182b4f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:33.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.963+0000 7f0c6be00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556182b41930 0x556182b4fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57930/0 (socket says 172.21.15.1:57930) 2024-02-02T15:33:33.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.963+0000 7f0c6be00700 1 -- 172.21.15.1:0/1490166840 learned_addr learned my addr 172.21.15.1:0/1490166840 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:33:33.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.963+0000 7f0c71600700 1 --2- 172.21.15.1:0/1490166840 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556182b4c7e0 0x556182ba64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:33.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.963+0000 7f0c6be00700 1 -- 172.21.15.1:0/1490166840 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556182b4c7e0 msgr2=0x556182ba64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:33.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.963+0000 7f0c6be00700 1 --2- 172.21.15.1:0/1490166840 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556182b4c7e0 0x556182ba64d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.963+0000 7f0c6be00700 1 -- 172.21.15.1:0/1490166840 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556182b41010 msgr2=0x556182b4f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:33.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.963+0000 7f0c6be00700 1 --2- 172.21.15.1:0/1490166840 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556182b41010 0x556182b4f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.963+0000 7f0c6be00700 1 -- 172.21.15.1:0/1490166840 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c6c1a60b0 con 0x556182b41930 2024-02-02T15:33:33.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.963+0000 7f0c70c00700 1 --2- 172.21.15.1:0/1490166840 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556182b41010 0x556182b4f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:33:33.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.963+0000 7f0c6be00700 1 --2- 172.21.15.1:0/1490166840 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556182b41930 0x556182b4fdd0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x556182b4bde0 tx=0x7f0c54004320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a134e661ab312f3b server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:33:33.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.963+0000 7f0c5a000700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:33:33.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.964+0000 7f0c5a000700 1 -- 172.21.15.1:0/1490166840 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c54017040 con 0x556182b41930 2024-02-02T15:33:33.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.964+0000 7f0c5a000700 1 -- 172.21.15.1:0/1490166840 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f0c54003c60 con 0x556182b41930 2024-02-02T15:33:33.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.964+0000 7f0c870cab80 1 -- 172.21.15.1:0/1490166840 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0c5c0b0070 con 0x556182b41930 2024-02-02T15:33:33.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.964+0000 7f0c5a000700 1 -- 172.21.15.1:0/1490166840 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c5401b5b0 con 0x556182b41930 2024-02-02T15:33:33.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.964+0000 7f0c870cab80 10 client.17316 fetch_fsmap learned FSMap version 38 2024-02-02T15:33:33.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.964+0000 7f0c870cab80 1 -- 172.21.15.1:0/1490166840 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x7f0c64009a60 con 0x556182b41930 2024-02-02T15:33:33.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.964+0000 7f0c5a000700 1 -- 172.21.15.1:0/1490166840 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0c5c0b0070 con 0x556182b41930 2024-02-02T15:33:33.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.964+0000 7f0c5a000700 1 -- 172.21.15.1:0/1490166840 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 38) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f0c540246b0 con 0x556182b41930 2024-02-02T15:33:33.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.964+0000 7f0c870cab80 10 client.17316 fetch_fsmap finished waiting for FSMap version 38 2024-02-02T15:33:33.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.964+0000 7f0c870cab80 10 client.17316 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-02-02T15:33:33.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.964+0000 7f0c870cab80 20 client.17316 populate_metadata read hostname 'smithi001' 2024-02-02T15:33:33.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.964+0000 7f0c870cab80 1 --2- 172.21.15.1:0/1490166840 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x556182bae560 0x556182bb0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:33.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.965+0000 7f0c870cab80 4 client.17316 mds_command: new command op to 24518 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:33:33.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.965+0000 7f0c870cab80 1 -- 172.21.15.1:0/1490166840 --> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556182a51f50 con 0x556182bae560 2024-02-02T15:33:33.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.965+0000 7f0c70c00700 1 --2- 172.21.15.1:0/1490166840 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x556182bae560 0x556182bb0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:33.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.965+0000 7f0c70c00700 1 --2- 172.21.15.1:0/1490166840 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x556182bae560 0x556182bb0980 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b7fe28893e38de2b server_cookie=f07fe4c6f70f8af in_seq=0 out_seq=0 2024-02-02T15:33:33.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.965+0000 7f0c5a000700 10 client.17316 ms_handle_connect on v2:172.21.15.71:6836/628016199 2024-02-02T15:33:33.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.965+0000 7f0c5a000700 1 -- 172.21.15.1:0/1490166840 <== mds.2 v2:172.21.15.71:6836/628016199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556182a51f50 con 0x556182bae560 2024-02-02T15:33:33.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.965+0000 7f0c5a000700 10 client.17316 handle_command_reply: tid=0 2024-02-02T15:33:33.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.965+0000 7f0c870cab80 1 client.17316 shutdown 2024-02-02T15:33:33.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.966+0000 7f0c870cab80 1 -- 172.21.15.1:0/1490166840 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556182b41930 msgr2=0x556182b4fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:33.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.966+0000 7f0c870cab80 1 --2- 172.21.15.1:0/1490166840 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556182b41930 0x556182b4fdd0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x556182b4bde0 tx=0x7f0c54004320 comp rx=0 tx=0).stop 2024-02-02T15:33:33.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.966+0000 7f0c870cab80 1 -- 172.21.15.1:0/1490166840 shutdown_connections 2024-02-02T15:33:33.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.966+0000 7f0c870cab80 1 --2- 172.21.15.1:0/1490166840 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x556182bae560 0x556182bb0980 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.966+0000 7f0c870cab80 1 --2- 172.21.15.1:0/1490166840 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556182b4c7e0 0x556182ba64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.966+0000 7f0c870cab80 1 --2- 172.21.15.1:0/1490166840 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556182b41010 0x556182b4f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.966+0000 7f0c5a000700 0 client.17316 ms_handle_reset on v2:172.21.15.71:6836/628016199 2024-02-02T15:33:33.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.966+0000 7f0c870cab80 1 --2- 172.21.15.1:0/1490166840 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556182b41930 0x556182b4fdd0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.966+0000 7f0c870cab80 1 -- 172.21.15.1:0/1490166840 >> 172.21.15.1:0/1490166840 conn(0x556182aa0db0 msgr2=0x556182b4a890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:33.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.966+0000 7f0c870cab80 1 -- 172.21.15.1:0/1490166840 shutdown_connections 2024-02-02T15:33:33.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.966+0000 7f0c870cab80 1 -- 172.21.15.1:0/1490166840 wait complete. 2024-02-02T15:33:33.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.966+0000 7f0c870cab80 20 client.17316 trim_cache size 0 max 16384 2024-02-02T15:33:33.971 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:33:33.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.966+0000 7f0c73600700 1 -- 172.21.15.1:0/1946365574 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f0c4c0a0710 msgr2=0x7f0c4c0a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:33.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.966+0000 7f0c73600700 1 --2- 172.21.15.1:0/1946365574 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f0c4c0a0710 0x7f0c4c0a2bd0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0c64003fd0 tx=0x7f0c64003dc0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.966+0000 7f0c73600700 1 -- 172.21.15.1:0/1946365574 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c6c100ff0 msgr2=0x7f0c6c073040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:33.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.967+0000 7f0c73600700 1 --2- 172.21.15.1:0/1946365574 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c6c100ff0 0x7f0c6c073040 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f0c5c005910 tx=0x7f0c5c005740 comp rx=0 tx=0).stop 2024-02-02T15:33:33.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.967+0000 7f0c73600700 1 -- 172.21.15.1:0/1946365574 shutdown_connections 2024-02-02T15:33:33.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.967+0000 7f0c73600700 1 --2- 172.21.15.1:0/1946365574 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f0c4c0a0710 0x7f0c4c0a2bd0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.967+0000 7f0c73600700 1 --2- 172.21.15.1:0/1946365574 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c6c073ac0 0x7f0c6c1a5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.967+0000 7f0c73600700 1 --2- 172.21.15.1:0/1946365574 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c6c100ff0 0x7f0c6c073040 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.967+0000 7f0c73600700 1 --2- 172.21.15.1:0/1946365574 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c6c106110 0x7f0c6c073580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:33.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.967+0000 7f0c73600700 1 -- 172.21.15.1:0/1946365574 >> 172.21.15.1:0/1946365574 conn(0x7f0c6c0fc810 msgr2=0x7f0c6c1049c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:33.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.967+0000 7f0c73600700 1 -- 172.21.15.1:0/1946365574 shutdown_connections 2024-02-02T15:33:33.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:33.967+0000 7f0c73600700 1 -- 172.21.15.1:0/1946365574 wait complete. 2024-02-02T15:33:33.985 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-02T15:33:34.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.202+0000 7f9089600700 1 -- 172.21.15.1:0/2083675549 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f908410b400 msgr2=0x7f908410d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:34.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.202+0000 7f9089600700 1 --2- 172.21.15.1:0/2083675549 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f908410b400 0x7f908410d850 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f907c009f90 tx=0x7f907c00b330 comp rx=0 tx=0).stop 2024-02-02T15:33:34.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.203+0000 7f9089600700 1 -- 172.21.15.1:0/2083675549 shutdown_connections 2024-02-02T15:33:34.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.203+0000 7f9089600700 1 --2- 172.21.15.1:0/2083675549 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f908410b400 0x7f908410d850 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:34.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.203+0000 7f9089600700 1 --2- 172.21.15.1:0/2083675549 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9084102030 0x7f9084102410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:34.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.203+0000 7f9089600700 1 --2- 172.21.15.1:0/2083675549 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9084102950 0x7f908410ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:34.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.203+0000 7f9089600700 1 -- 172.21.15.1:0/2083675549 >> 172.21.15.1:0/2083675549 conn(0x7f9084076d70 msgr2=0x7f9084077180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:34.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.203+0000 7f9089600700 1 -- 172.21.15.1:0/2083675549 shutdown_connections 2024-02-02T15:33:34.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.203+0000 7f9089600700 1 -- 172.21.15.1:0/2083675549 wait complete. 2024-02-02T15:33:34.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.204+0000 7f9089600700 1 Processor -- start 2024-02-02T15:33:34.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.204+0000 7f9089600700 1 -- start start 2024-02-02T15:33:34.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.205+0000 7f9089600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9084102030 0x7f90840ffc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:34.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.205+0000 7f9089600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9084102950 0x7f90841001c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:34.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.205+0000 7f9089600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f908410b400 0x7f90841a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:34.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.205+0000 7f9089600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90841009f0 con 0x7f9084102950 2024-02-02T15:33:34.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.205+0000 7f9089600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90841a57a0 con 0x7f908410b400 2024-02-02T15:33:34.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.205+0000 7f9089600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90841a58e0 con 0x7f9084102030 2024-02-02T15:33:34.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.205+0000 7f9082000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9084102950 0x7f90841001c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:34.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.205+0000 7f9082a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9084102030 0x7f90840ffc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:34.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.205+0000 7f9082000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9084102950 0x7f90841001c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57948/0 (socket says 172.21.15.1:57948) 2024-02-02T15:33:34.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.205+0000 7f9082000700 1 -- 172.21.15.1:0/2615587662 learned_addr learned my addr 172.21.15.1:0/2615587662 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:33:34.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.205+0000 7f9083400700 1 --2- 172.21.15.1:0/2615587662 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f908410b400 0x7f90841a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:34.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.205+0000 7f9082a00700 1 -- 172.21.15.1:0/2615587662 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f908410b400 msgr2=0x7f90841a5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:34.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.205+0000 7f9082a00700 1 --2- 172.21.15.1:0/2615587662 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f908410b400 0x7f90841a5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:34.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.205+0000 7f9082a00700 1 -- 172.21.15.1:0/2615587662 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9084102950 msgr2=0x7f90841001c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:34.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.205+0000 7f9082a00700 1 --2- 172.21.15.1:0/2615587662 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9084102950 0x7f90841001c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:34.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.205+0000 7f9082a00700 1 -- 172.21.15.1:0/2615587662 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f907c009c40 con 0x7f9084102030 2024-02-02T15:33:34.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.205+0000 7f9082000700 1 --2- 172.21.15.1:0/2615587662 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9084102950 0x7f90841001c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:33:34.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.206+0000 7f9082a00700 1 --2- 172.21.15.1:0/2615587662 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9084102030 0x7f90840ffc80 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f907400e9c0 tx=0x7f907400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:33:34.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.206+0000 7f907b400700 1 -- 172.21.15.1:0/2615587662 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9074004220 con 0x7f9084102030 2024-02-02T15:33:34.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.206+0000 7f9089600700 1 -- 172.21.15.1:0/2615587662 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90841a5b40 con 0x7f9084102030 2024-02-02T15:33:34.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.206+0000 7f907b400700 1 -- 172.21.15.1:0/2615587662 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9074004cf0 con 0x7f9084102030 2024-02-02T15:33:34.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.206+0000 7f907b400700 1 -- 172.21.15.1:0/2615587662 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f907400cd30 con 0x7f9084102030 2024-02-02T15:33:34.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.206+0000 7f9089600700 1 -- 172.21.15.1:0/2615587662 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90841a5fd0 con 0x7f9084102030 2024-02-02T15:33:34.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.209+0000 7f907b400700 1 -- 172.21.15.1:0/2615587662 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 27) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f9074004750 con 0x7f9084102030 2024-02-02T15:33:34.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.210+0000 7f907b400700 1 --2- 172.21.15.1:0/2615587662 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f90640a0690 0x7f90640a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:34.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.211+0000 7f907b400700 1 -- 172.21.15.1:0/2615587662 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f9074014070 con 0x7f9084102030 2024-02-02T15:33:34.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.211+0000 7f9082000700 1 --2- 172.21.15.1:0/2615587662 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f90640a0690 0x7f90640a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:34.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.211+0000 7f909d0afb80 1 Processor -- start 2024-02-02T15:33:34.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.211+0000 7f909d0afb80 1 -- start start 2024-02-02T15:33:34.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.211+0000 7f9082000700 1 --2- 172.21.15.1:0/2615587662 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f90640a0690 0x7f90640a2b50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9084101470 tx=0x7f906c009260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:33:34.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.211+0000 7f909d0afb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5596eea46010 0x5596eea463f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:34.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.211+0000 7f909d0afb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5596eea46930 0x5596eea512a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:34.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.212+0000 7f909d0afb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5596eea517e0 0x5596eea53bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:34.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.212+0000 7f909d0afb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596ee9ba4c0 con 0x5596eea46010 2024-02-02T15:33:34.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.212+0000 7f909d0afb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596ee99eaa0 con 0x5596eea517e0 2024-02-02T15:33:34.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.212+0000 7f909d0afb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596ee99c460 con 0x5596eea46930 2024-02-02T15:33:34.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.212+0000 7f9082000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5596eea46930 0x5596eea512a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:34.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.212+0000 7f9082000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5596eea46930 0x5596eea512a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:56048/0 (socket says 172.21.15.1:56048) 2024-02-02T15:33:34.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.212+0000 7f9082000700 1 -- 172.21.15.1:0/2110620540 learned_addr learned my addr 172.21.15.1:0/2110620540 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:33:34.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.212+0000 7f9083400700 1 --2- 172.21.15.1:0/2110620540 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5596eea517e0 0x5596eea53bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:34.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.212+0000 7f9082a00700 1 --2- 172.21.15.1:0/2110620540 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5596eea46010 0x5596eea463f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:34.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.212+0000 7f9082000700 1 -- 172.21.15.1:0/2110620540 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5596eea517e0 msgr2=0x5596eea53bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:34.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.212+0000 7f9082000700 1 --2- 172.21.15.1:0/2110620540 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5596eea517e0 0x5596eea53bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:34.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.212+0000 7f9082000700 1 -- 172.21.15.1:0/2110620540 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5596eea46010 msgr2=0x5596eea463f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:34.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.212+0000 7f9082000700 1 --2- 172.21.15.1:0/2110620540 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5596eea46010 0x5596eea463f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:34.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.212+0000 7f9082000700 1 -- 172.21.15.1:0/2110620540 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5596ee8193b0 con 0x5596eea46930 2024-02-02T15:33:34.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.212+0000 7f9082a00700 1 --2- 172.21.15.1:0/2110620540 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5596eea46010 0x5596eea463f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:33:34.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.212+0000 7f9082000700 1 --2- 172.21.15.1:0/2110620540 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5596eea46930 0x5596eea512a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f906c00e760 tx=0x7f906c014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26aa23553d436087 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:33:34.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.213+0000 7f9073400700 1 -- 172.21.15.1:0/2110620540 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f906c004830 con 0x5596eea46930 2024-02-02T15:33:34.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.213+0000 7f9073400700 1 -- 172.21.15.1:0/2110620540 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f906c004e50 con 0x5596eea46930 2024-02-02T15:33:34.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.213+0000 7f9073400700 1 -- 172.21.15.1:0/2110620540 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f906c01e680 con 0x5596eea46930 2024-02-02T15:33:34.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.213+0000 7f909d0afb80 1 -- 172.21.15.1:0/2110620540 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5596eea46930 msgr2=0x5596eea512a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:34.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.213+0000 7f909d0afb80 1 --2- 172.21.15.1:0/2110620540 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5596eea46930 0x5596eea512a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f906c00e760 tx=0x7f906c014000 comp rx=0 tx=0).stop 2024-02-02T15:33:34.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.213+0000 7f909d0afb80 1 -- 172.21.15.1:0/2110620540 shutdown_connections 2024-02-02T15:33:34.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.213+0000 7f909d0afb80 1 --2- 172.21.15.1:0/2110620540 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5596eea517e0 0x5596eea53bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:34.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.214+0000 7f909d0afb80 1 --2- 172.21.15.1:0/2110620540 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5596eea46010 0x5596eea463f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:34.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.214+0000 7f909d0afb80 1 --2- 172.21.15.1:0/2110620540 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5596eea46930 0x5596eea512a0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:34.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.214+0000 7f909d0afb80 1 -- 172.21.15.1:0/2110620540 >> 172.21.15.1:0/2110620540 conn(0x5596ee9a5db0 msgr2=0x5596eea423b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:34.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.214+0000 7f909d0afb80 1 -- 172.21.15.1:0/2110620540 shutdown_connections 2024-02-02T15:33:34.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.214+0000 7f909d0afb80 1 -- 172.21.15.1:0/2110620540 wait complete. 2024-02-02T15:33:34.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.214+0000 7f909d0afb80 1 -- start start 2024-02-02T15:33:34.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.215+0000 7f909d0afb80 1 Processor -- start 2024-02-02T15:33:34.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.215+0000 7f909d0afb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5596eea46010 0x5596eea51220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:34.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.215+0000 7f909d0afb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5596eea46930 0x5596eea4e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:34.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.215+0000 7f909d0afb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5596eea517e0 0x5596eea4eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:34.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.215+0000 7f909d0afb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596ee997d50 con 0x5596eea517e0 2024-02-02T15:33:34.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.215+0000 7f909d0afb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596ee933d80 con 0x5596eea46010 2024-02-02T15:33:34.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.215+0000 7f909d0afb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596ee933a20 con 0x5596eea46930 2024-02-02T15:33:34.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.215+0000 7f909d0afb80 10 client.0 _ll_register_callbacks cb 0x5596ee935290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:33:34.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.215+0000 7f9082a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5596eea46010 0x5596eea51220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:34.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.215+0000 7f9082a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5596eea46010 0x5596eea51220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:50284/0 (socket says 172.21.15.1:50284) 2024-02-02T15:33:34.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.215+0000 7f9082a00700 1 -- 172.21.15.1:0/3965320979 learned_addr learned my addr 172.21.15.1:0/3965320979 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:33:34.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.215+0000 7f9083400700 1 --2- 172.21.15.1:0/3965320979 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5596eea517e0 0x5596eea4eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:34.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.216+0000 7f9082000700 1 --2- 172.21.15.1:0/3965320979 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5596eea46930 0x5596eea4e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:34.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.216+0000 7f9082a00700 1 -- 172.21.15.1:0/3965320979 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5596eea46930 msgr2=0x5596eea4e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:34.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.216+0000 7f9082a00700 1 --2- 172.21.15.1:0/3965320979 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5596eea46930 0x5596eea4e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:34.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.216+0000 7f9082a00700 1 -- 172.21.15.1:0/3965320979 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5596eea517e0 msgr2=0x5596eea4eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:34.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.216+0000 7f9082a00700 1 --2- 172.21.15.1:0/3965320979 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5596eea517e0 0x5596eea4eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:34.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.216+0000 7f9082a00700 1 -- 172.21.15.1:0/3965320979 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f906c017070 con 0x5596eea46010 2024-02-02T15:33:34.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.216+0000 7f9083400700 1 --2- 172.21.15.1:0/3965320979 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5596eea517e0 0x5596eea4eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:33:34.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.216+0000 7f9082a00700 1 --2- 172.21.15.1:0/3965320979 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5596eea46010 0x5596eea51220 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f9074005fd0 tx=0x7f9074098a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36eb77abde4e575a server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:33:34.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.216+0000 7f906be00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:33:34.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.216+0000 7f906be00700 1 -- 172.21.15.1:0/3965320979 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9074099060 con 0x5596eea46010 2024-02-02T15:33:34.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.216+0000 7f906be00700 1 -- 172.21.15.1:0/3965320979 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f907409ce90 con 0x5596eea46010 2024-02-02T15:33:34.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.216+0000 7f909d0afb80 1 -- 172.21.15.1:0/3965320979 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5596eea3a0f0 con 0x5596eea46010 2024-02-02T15:33:34.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.216+0000 7f906be00700 1 -- 172.21.15.1:0/3965320979 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f90740a10d0 con 0x5596eea46010 2024-02-02T15:33:34.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.216+0000 7f909d0afb80 10 client.27392 fetch_fsmap learned FSMap version 38 2024-02-02T15:33:34.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.216+0000 7f909d0afb80 1 -- 172.21.15.1:0/3965320979 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x5596ee7c5b60 con 0x5596eea46010 2024-02-02T15:33:34.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.217+0000 7f906be00700 1 -- 172.21.15.1:0/3965320979 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5596eea3a0f0 con 0x5596eea46010 2024-02-02T15:33:34.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.217+0000 7f906be00700 1 -- 172.21.15.1:0/3965320979 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 38) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f90740a0d60 con 0x5596eea46010 2024-02-02T15:33:34.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.217+0000 7f909d0afb80 10 client.27392 fetch_fsmap finished waiting for FSMap version 38 2024-02-02T15:33:34.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.217+0000 7f909d0afb80 10 client.27392 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-02T15:33:34.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.217+0000 7f909d0afb80 20 client.27392 populate_metadata read hostname 'smithi001' 2024-02-02T15:33:34.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.217+0000 7f909d0afb80 1 --2- 172.21.15.1:0/3965320979 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x5596eea60f50 0x5596eea63340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:34.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.217+0000 7f909d0afb80 4 client.27392 mds_command: new command op to 24529 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:33:34.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.217+0000 7f909d0afb80 1 -- 172.21.15.1:0/3965320979 --> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5596ee990dd0 con 0x5596eea60f50 2024-02-02T15:33:34.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.217+0000 7f9082000700 1 --2- 172.21.15.1:0/3965320979 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x5596eea60f50 0x5596eea63340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:34.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.218+0000 7f906be00700 10 client.27392 ms_handle_connect on v2:172.21.15.142:6838/321675421 2024-02-02T15:33:34.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.218+0000 7f9082000700 1 --2- 172.21.15.1:0/3965320979 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x5596eea60f50 0x5596eea63340 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=db7e20f3fd55d42a server_cookie=29e41c09564963de in_seq=0 out_seq=0 2024-02-02T15:33:34.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.219+0000 7f906be00700 1 -- 172.21.15.1:0/3965320979 <== mds.1 v2:172.21.15.142:6838/321675421 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5596ee990dd0 con 0x5596eea60f50 2024-02-02T15:33:34.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.219+0000 7f906be00700 10 client.27392 handle_command_reply: tid=0 2024-02-02T15:33:34.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.219+0000 7f909d0afb80 1 client.27392 shutdown 2024-02-02T15:33:34.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.219+0000 7f909d0afb80 1 -- 172.21.15.1:0/3965320979 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5596eea46010 msgr2=0x5596eea51220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:34.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.219+0000 7f909d0afb80 1 --2- 172.21.15.1:0/3965320979 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5596eea46010 0x5596eea51220 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f9074005fd0 tx=0x7f9074098a40 comp rx=0 tx=0).stop 2024-02-02T15:33:34.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.219+0000 7f909d0afb80 1 -- 172.21.15.1:0/3965320979 shutdown_connections 2024-02-02T15:33:34.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.219+0000 7f909d0afb80 1 --2- 172.21.15.1:0/3965320979 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x5596eea60f50 0x5596eea63340 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:34.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.219+0000 7f909d0afb80 1 --2- 172.21.15.1:0/3965320979 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5596eea517e0 0x5596eea4eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:34.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.219+0000 7f909d0afb80 1 --2- 172.21.15.1:0/3965320979 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5596eea46010 0x5596eea51220 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:34.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.219+0000 7f909d0afb80 1 --2- 172.21.15.1:0/3965320979 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5596eea46930 0x5596eea4e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:34.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.219+0000 7f906be00700 0 client.27392 ms_handle_reset on v2:172.21.15.142:6838/321675421 2024-02-02T15:33:34.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.219+0000 7f909d0afb80 1 -- 172.21.15.1:0/3965320979 >> 172.21.15.1:0/3965320979 conn(0x5596ee9a5db0 msgr2=0x5596eea41fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:34.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.219+0000 7f909d0afb80 1 -- 172.21.15.1:0/3965320979 shutdown_connections 2024-02-02T15:33:34.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.219+0000 7f909d0afb80 1 -- 172.21.15.1:0/3965320979 wait complete. 2024-02-02T15:33:34.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.220+0000 7f909d0afb80 20 client.27392 trim_cache size 0 max 16384 2024-02-02T15:33:34.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.233+0000 7f909d0afb80 1 Processor -- start 2024-02-02T15:33:34.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.233+0000 7f909d0afb80 1 -- start start 2024-02-02T15:33:34.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.234+0000 7f909d0afb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5596eea46010 0x5596eea50d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:34.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.234+0000 7f909d0afb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5596eea46930 0x5596eea49740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:34.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.234+0000 7f909d0afb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5596eea517e0 0x5596eea49c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:34.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.234+0000 7f909d0afb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596ee81b6a0 con 0x5596eea46010 2024-02-02T15:33:34.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.234+0000 7f909d0afb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596eea53e20 con 0x5596eea46930 2024-02-02T15:33:34.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.234+0000 7f909d0afb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596eea53fe0 con 0x5596eea517e0 2024-02-02T15:33:34.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.234+0000 7f9082000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5596eea46930 0x5596eea49740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:34.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.234+0000 7f9082000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5596eea46930 0x5596eea49740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:50290/0 (socket says 172.21.15.1:50290) 2024-02-02T15:33:34.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.234+0000 7f9082000700 1 -- 172.21.15.1:0/3185086665 learned_addr learned my addr 172.21.15.1:0/3185086665 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:33:34.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.234+0000 7f9082a00700 1 --2- 172.21.15.1:0/3185086665 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5596eea46010 0x5596eea50d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:34.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.234+0000 7f9083400700 1 --2- 172.21.15.1:0/3185086665 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5596eea517e0 0x5596eea49c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:34.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.234+0000 7f9082000700 1 -- 172.21.15.1:0/3185086665 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5596eea517e0 msgr2=0x5596eea49c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:34.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.234+0000 7f9082000700 1 --2- 172.21.15.1:0/3185086665 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5596eea517e0 0x5596eea49c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:34.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.234+0000 7f9082000700 1 -- 172.21.15.1:0/3185086665 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5596eea46010 msgr2=0x5596eea50d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:34.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.234+0000 7f9082000700 1 --2- 172.21.15.1:0/3185086665 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5596eea46010 0x5596eea50d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:34.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.234+0000 7f9082000700 1 -- 172.21.15.1:0/3185086665 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5596ee95dea0 con 0x5596eea46930 2024-02-02T15:33:34.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.235+0000 7f9082000700 1 --2- 172.21.15.1:0/3185086665 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5596eea46930 0x5596eea49740 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f906c00e700 tx=0x7f906c013040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=148324d189c02fc0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:33:34.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.235+0000 7f9073e00700 1 -- 172.21.15.1:0/3185086665 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f906c036070 con 0x5596eea46930 2024-02-02T15:33:34.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.235+0000 7f9073e00700 1 -- 172.21.15.1:0/3185086665 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f906c02d040 con 0x5596eea46930 2024-02-02T15:33:34.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.235+0000 7f9073e00700 1 -- 172.21.15.1:0/3185086665 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f906c032790 con 0x5596eea46930 2024-02-02T15:33:34.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.236+0000 7f909d0afb80 1 -- 172.21.15.1:0/3185086665 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5596eea46930 msgr2=0x5596eea49740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:34.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.236+0000 7f909d0afb80 1 --2- 172.21.15.1:0/3185086665 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5596eea46930 0x5596eea49740 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f906c00e700 tx=0x7f906c013040 comp rx=0 tx=0).stop 2024-02-02T15:33:34.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.236+0000 7f909d0afb80 1 -- 172.21.15.1:0/3185086665 shutdown_connections 2024-02-02T15:33:34.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.236+0000 7f909d0afb80 1 --2- 172.21.15.1:0/3185086665 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5596eea517e0 0x5596eea49c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:34.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.236+0000 7f909d0afb80 1 --2- 172.21.15.1:0/3185086665 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5596eea46010 0x5596eea50d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:34.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.236+0000 7f909d0afb80 1 --2- 172.21.15.1:0/3185086665 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5596eea46930 0x5596eea49740 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:34.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.236+0000 7f909d0afb80 1 -- 172.21.15.1:0/3185086665 >> 172.21.15.1:0/3185086665 conn(0x5596ee9a5db0 msgr2=0x5596eea5c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:34.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.236+0000 7f909d0afb80 1 -- 172.21.15.1:0/3185086665 shutdown_connections 2024-02-02T15:33:34.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.236+0000 7f909d0afb80 1 -- 172.21.15.1:0/3185086665 wait complete. 2024-02-02T15:33:34.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.236+0000 7f909d0afb80 1 -- start start 2024-02-02T15:33:34.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.237+0000 7f909d0afb80 1 Processor -- start 2024-02-02T15:33:34.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.237+0000 7f909d0afb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5596eea46010 0x5596eea57e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:34.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.237+0000 7f909d0afb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5596eea46930 0x5596eea583a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:34.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.237+0000 7f909d0afb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5596eea517e0 0x5596eea56390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:34.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.237+0000 7f909d0afb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596eea56a60 con 0x5596eea517e0 2024-02-02T15:33:34.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.237+0000 7f909d0afb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596eea56bd0 con 0x5596eea46010 2024-02-02T15:33:34.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.237+0000 7f909d0afb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596eea56d90 con 0x5596eea46930 2024-02-02T15:33:34.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.237+0000 7f909d0afb80 10 client.0 _ll_register_callbacks cb 0x5596ee935290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:33:34.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.237+0000 7f9082a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5596eea46010 0x5596eea57e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:34.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.238+0000 7f9082a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5596eea46010 0x5596eea57e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:50294/0 (socket says 172.21.15.1:50294) 2024-02-02T15:33:34.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.238+0000 7f9082a00700 1 -- 172.21.15.1:0/2990046779 learned_addr learned my addr 172.21.15.1:0/2990046779 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:33:34.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.238+0000 7f9082000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5596eea46930 0x5596eea583a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:34.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.238+0000 7f9083400700 1 --2- 172.21.15.1:0/2990046779 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5596eea517e0 0x5596eea56390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:34.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.238+0000 7f9082a00700 1 -- 172.21.15.1:0/2990046779 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5596eea46930 msgr2=0x5596eea583a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:34.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.238+0000 7f9082a00700 1 --2- 172.21.15.1:0/2990046779 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5596eea46930 0x5596eea583a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:34.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.238+0000 7f9082a00700 1 -- 172.21.15.1:0/2990046779 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5596eea517e0 msgr2=0x5596eea56390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:34.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.238+0000 7f9082a00700 1 --2- 172.21.15.1:0/2990046779 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5596eea517e0 0x5596eea56390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:34.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.238+0000 7f9082a00700 1 -- 172.21.15.1:0/2990046779 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5596ee8193b0 con 0x5596eea46010 2024-02-02T15:33:34.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.238+0000 7f9082000700 1 --2- 172.21.15.1:0/2990046779 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5596eea46930 0x5596eea583a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:33:34.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.238+0000 7f9083400700 1 --2- 172.21.15.1:0/2990046779 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5596eea517e0 0x5596eea56390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:33:34.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.238+0000 7f9082a00700 1 --2- 172.21.15.1:0/2990046779 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5596eea46010 0x5596eea57e60 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f90740a0ba0 tx=0x7f9074004610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=771ea0f89b3b73c3 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:33:34.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.238+0000 7f906be00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:33:34.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.238+0000 7f906be00700 1 -- 172.21.15.1:0/2990046779 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9074004e60 con 0x5596eea46010 2024-02-02T15:33:34.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.239+0000 7f906be00700 1 -- 172.21.15.1:0/2990046779 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f90740b15e0 con 0x5596eea46010 2024-02-02T15:33:34.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.239+0000 7f909d0afb80 1 -- 172.21.15.1:0/2990046779 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f906c004e50 con 0x5596eea46010 2024-02-02T15:33:34.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.239+0000 7f906be00700 1 -- 172.21.15.1:0/2990046779 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f907409ce90 con 0x5596eea46010 2024-02-02T15:33:34.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.239+0000 7f909d0afb80 10 client.27398 fetch_fsmap learned FSMap version 38 2024-02-02T15:33:34.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.239+0000 7f909d0afb80 1 -- 172.21.15.1:0/2990046779 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x5596ee81dcd0 con 0x5596eea46010 2024-02-02T15:33:34.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.239+0000 7f906be00700 1 -- 172.21.15.1:0/2990046779 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f906c004e50 con 0x5596eea46010 2024-02-02T15:33:34.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.239+0000 7f906be00700 1 -- 172.21.15.1:0/2990046779 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 38) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f9074097fc0 con 0x5596eea46010 2024-02-02T15:33:34.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.239+0000 7f909d0afb80 10 client.27398 fetch_fsmap finished waiting for FSMap version 38 2024-02-02T15:33:34.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.239+0000 7f909d0afb80 10 client.27398 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-02T15:33:34.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.239+0000 7f909d0afb80 20 client.27398 populate_metadata read hostname 'smithi001' 2024-02-02T15:33:34.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.239+0000 7f909d0afb80 1 --2- 172.21.15.1:0/2990046779 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x5596eeab3970 0x5596eeab5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:34.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.239+0000 7f909d0afb80 4 client.27398 mds_command: new command op to 24529 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:33:34.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.239+0000 7f909d0afb80 1 -- 172.21.15.1:0/2990046779 --> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5596ee956f50 con 0x5596eeab3970 2024-02-02T15:33:34.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.239+0000 7f9082000700 1 --2- 172.21.15.1:0/2990046779 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x5596eeab3970 0x5596eeab5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:34.243 INFO:tasks.workunit.client.0.smithi001.stdout: 1 424196 8.13 MB/sec execute 188 sec latency 1910.407 ms 2024-02-02T15:33:34.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.240+0000 7f906be00700 10 client.27398 ms_handle_connect on v2:172.21.15.142:6838/321675421 2024-02-02T15:33:34.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.240+0000 7f9082000700 1 --2- 172.21.15.1:0/2990046779 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x5596eeab3970 0x5596eeab5d90 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d396152af63a890 server_cookie=2a2fb7681ca5eaa2 in_seq=0 out_seq=0 2024-02-02T15:33:34.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.240+0000 7f906be00700 1 -- 172.21.15.1:0/2990046779 <== mds.1 v2:172.21.15.142:6838/321675421 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5596ee956f50 con 0x5596eeab3970 2024-02-02T15:33:34.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.240+0000 7f906be00700 10 client.27398 handle_command_reply: tid=0 2024-02-02T15:33:34.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.240+0000 7f909d0afb80 1 client.27398 shutdown 2024-02-02T15:33:34.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.240+0000 7f909d0afb80 1 -- 172.21.15.1:0/2990046779 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5596eea46010 msgr2=0x5596eea57e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:34.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.240+0000 7f909d0afb80 1 --2- 172.21.15.1:0/2990046779 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5596eea46010 0x5596eea57e60 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f90740a0ba0 tx=0x7f9074004610 comp rx=0 tx=0).stop 2024-02-02T15:33:34.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.240+0000 7f909d0afb80 1 -- 172.21.15.1:0/2990046779 shutdown_connections 2024-02-02T15:33:34.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.240+0000 7f909d0afb80 1 --2- 172.21.15.1:0/2990046779 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x5596eeab3970 0x5596eeab5d90 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:34.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.240+0000 7f909d0afb80 1 --2- 172.21.15.1:0/2990046779 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5596eea517e0 0x5596eea56390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:34.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.240+0000 7f909d0afb80 1 --2- 172.21.15.1:0/2990046779 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5596eea46010 0x5596eea57e60 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:34.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.240+0000 7f906be00700 0 client.27398 ms_handle_reset on v2:172.21.15.142:6838/321675421 2024-02-02T15:33:34.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.240+0000 7f909d0afb80 1 --2- 172.21.15.1:0/2990046779 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5596eea46930 0x5596eea583a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:34.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.241+0000 7f909d0afb80 1 -- 172.21.15.1:0/2990046779 >> 172.21.15.1:0/2990046779 conn(0x5596ee9a5db0 msgr2=0x5596eea49740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:34.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.241+0000 7f909d0afb80 1 -- 172.21.15.1:0/2990046779 shutdown_connections 2024-02-02T15:33:34.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.241+0000 7f909d0afb80 1 -- 172.21.15.1:0/2990046779 wait complete. 2024-02-02T15:33:34.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.241+0000 7f909d0afb80 20 client.27398 trim_cache size 0 max 16384 2024-02-02T15:33:34.246 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:33:34.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.241+0000 7f9089600700 1 -- 172.21.15.1:0/2615587662 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f90640a0690 msgr2=0x7f90640a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:34.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.241+0000 7f9089600700 1 --2- 172.21.15.1:0/2615587662 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f90640a0690 0x7f90640a2b50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9084101470 tx=0x7f906c009260 comp rx=0 tx=0).stop 2024-02-02T15:33:34.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.241+0000 7f9089600700 1 -- 172.21.15.1:0/2615587662 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9084102030 msgr2=0x7f90840ffc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:34.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.241+0000 7f9089600700 1 --2- 172.21.15.1:0/2615587662 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9084102030 0x7f90840ffc80 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f907400e9c0 tx=0x7f907400ecd0 comp rx=0 tx=0).stop 2024-02-02T15:33:34.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.241+0000 7f9089600700 1 -- 172.21.15.1:0/2615587662 shutdown_connections 2024-02-02T15:33:34.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.241+0000 7f9089600700 1 --2- 172.21.15.1:0/2615587662 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f90640a0690 0x7f90640a2b50 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:34.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.241+0000 7f9089600700 1 --2- 172.21.15.1:0/2615587662 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f908410b400 0x7f90841a5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:34.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.241+0000 7f9089600700 1 --2- 172.21.15.1:0/2615587662 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9084102030 0x7f90840ffc80 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:34.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.241+0000 7f9089600700 1 --2- 172.21.15.1:0/2615587662 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9084102950 0x7f90841001c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:34.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.241+0000 7f9089600700 1 -- 172.21.15.1:0/2615587662 >> 172.21.15.1:0/2615587662 conn(0x7f9084076d70 msgr2=0x7f90840fcb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:34.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.242+0000 7f9089600700 1 -- 172.21.15.1:0/2615587662 shutdown_connections 2024-02-02T15:33:34.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.242+0000 7f9089600700 1 -- 172.21.15.1:0/2615587662 wait complete. 2024-02-02T15:33:34.260 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-02T15:33:34.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.469+0000 7f5a35e00700 1 -- 172.21.15.1:0/2018385658 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a30100fa0 msgr2=0x7f5a30101380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:34.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.469+0000 7f5a35e00700 1 --2- 172.21.15.1:0/2018385658 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a30100fa0 0x7f5a30101380 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f5a20009990 tx=0x7f5a20009ca0 comp rx=0 tx=0).stop 2024-02-02T15:33:34.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.469+0000 7f5a35e00700 1 -- 172.21.15.1:0/2018385658 shutdown_connections 2024-02-02T15:33:34.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.469+0000 7f5a35e00700 1 --2- 172.21.15.1:0/2018385658 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5a301060c0 0x7f5a3010dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:34.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.469+0000 7f5a35e00700 1 --2- 172.21.15.1:0/2018385658 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a30100fa0 0x7f5a30101380 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:34.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.469+0000 7f5a35e00700 1 --2- 172.21.15.1:0/2018385658 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5a30101950 0x7f5a30105990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:34.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.469+0000 7f5a35e00700 1 -- 172.21.15.1:0/2018385658 >> 172.21.15.1:0/2018385658 conn(0x7f5a300fc800 msgr2=0x7f5a300fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:34.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.469+0000 7f5a35e00700 1 -- 172.21.15.1:0/2018385658 shutdown_connections 2024-02-02T15:33:34.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.470+0000 7f5a35e00700 1 -- 172.21.15.1:0/2018385658 wait complete. 2024-02-02T15:33:34.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.470+0000 7f5a35e00700 1 Processor -- start 2024-02-02T15:33:34.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.471+0000 7f5a35e00700 1 -- start start 2024-02-02T15:33:34.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.471+0000 7f5a35e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5a30100fa0 0x7f5a30073040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:34.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.471+0000 7f5a35e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a301060c0 0x7f5a30073580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:34.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.471+0000 7f5a35e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5a30073ac0 0x7f5a301a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:34.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.471+0000 7f5a35e00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a301a5760 con 0x7f5a30073ac0 2024-02-02T15:33:34.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.471+0000 7f5a35e00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a301a58d0 con 0x7f5a30100fa0 2024-02-02T15:33:34.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.471+0000 7f5a35e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a301a5a90 con 0x7f5a301060c0 2024-02-02T15:33:34.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.471+0000 7f5a2e000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a301060c0 0x7f5a30073580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:34.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.471+0000 7f5a2e000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a301060c0 0x7f5a30073580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:56086/0 (socket says 172.21.15.1:56086) 2024-02-02T15:33:34.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.471+0000 7f5a2f400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5a30073ac0 0x7f5a301a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:34.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.471+0000 7f5a2e000700 1 -- 172.21.15.1:0/4065814074 learned_addr learned my addr 172.21.15.1:0/4065814074 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:33:34.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.471+0000 7f5a2ea00700 1 --2- 172.21.15.1:0/4065814074 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5a30100fa0 0x7f5a30073040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:34.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.472+0000 7f5a2e000700 1 -- 172.21.15.1:0/4065814074 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5a30100fa0 msgr2=0x7f5a30073040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:34.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.472+0000 7f5a2e000700 1 --2- 172.21.15.1:0/4065814074 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5a30100fa0 0x7f5a30073040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:34.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.472+0000 7f5a2e000700 1 -- 172.21.15.1:0/4065814074 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5a30073ac0 msgr2=0x7f5a301a50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:34.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.472+0000 7f5a2e000700 1 --2- 172.21.15.1:0/4065814074 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5a30073ac0 0x7f5a301a50d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:34.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.472+0000 7f5a2e000700 1 -- 172.21.15.1:0/4065814074 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a200095f0 con 0x7f5a301060c0 2024-02-02T15:33:34.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.472+0000 7f5a2e000700 1 --2- 172.21.15.1:0/4065814074 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a301060c0 0x7f5a30073580 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f5a1800e970 tx=0x7f5a1800ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:33:34.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.472+0000 7f5a34a00700 1 -- 172.21.15.1:0/4065814074 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a18009790 con 0x7f5a301060c0 2024-02-02T15:33:34.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.472+0000 7f5a35e00700 1 -- 172.21.15.1:0/4065814074 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a301a5d70 con 0x7f5a301060c0 2024-02-02T15:33:34.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.472+0000 7f5a34a00700 1 -- 172.21.15.1:0/4065814074 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a1800c7e0 con 0x7f5a301060c0 2024-02-02T15:33:34.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.472+0000 7f5a34a00700 1 -- 172.21.15.1:0/4065814074 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a180054e0 con 0x7f5a301060c0 2024-02-02T15:33:34.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.472+0000 7f5a35e00700 1 -- 172.21.15.1:0/4065814074 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a301a6220 con 0x7f5a301060c0 2024-02-02T15:33:34.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.477+0000 7f5a34a00700 1 -- 172.21.15.1:0/4065814074 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 27) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f5a1800c950 con 0x7f5a301060c0 2024-02-02T15:33:34.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.477+0000 7f5a4985fb80 1 Processor -- start 2024-02-02T15:33:34.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.478+0000 7f5a4985fb80 1 -- start start 2024-02-02T15:33:34.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.478+0000 7f5a4985fb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3de79d010 0x55c3de79d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:34.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.478+0000 7f5a4985fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3de79d930 0x55c3de7a82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:34.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.478+0000 7f5a4985fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c3de7a87e0 0x55c3de7aabd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:34.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.478+0000 7f5a4985fb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3de7114c0 con 0x55c3de79d010 2024-02-02T15:33:34.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.478+0000 7f5a4985fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3de6f5aa0 con 0x55c3de7a87e0 2024-02-02T15:33:34.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.478+0000 7f5a4985fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3de6f3460 con 0x55c3de79d930 2024-02-02T15:33:34.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.478+0000 7f5a2e000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c3de7a87e0 0x55c3de7aabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:34.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.478+0000 7f5a2f400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3de79d010 0x55c3de79d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:34.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.478+0000 7f5a2ea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3de79d930 0x55c3de7a82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:34.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.478+0000 7f5a2f400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3de79d010 0x55c3de79d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58014/0 (socket says 172.21.15.1:58014) 2024-02-02T15:33:34.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.478+0000 7f5a2f400700 1 -- 172.21.15.1:0/4119597533 learned_addr learned my addr 172.21.15.1:0/4119597533 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:33:34.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.479+0000 7f5a2e000700 1 -- 172.21.15.1:0/4119597533 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3de79d930 msgr2=0x55c3de7a82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:34.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.479+0000 7f5a2e000700 1 --2- 172.21.15.1:0/4119597533 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3de79d930 0x55c3de7a82a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:34.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.479+0000 7f5a2e000700 1 -- 172.21.15.1:0/4119597533 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3de79d010 msgr2=0x55c3de79d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:34.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.479+0000 7f5a2e000700 1 --2- 172.21.15.1:0/4119597533 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3de79d010 0x55c3de79d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:34.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.479+0000 7f5a2e000700 1 -- 172.21.15.1:0/4119597533 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3de5703b0 con 0x55c3de7a87e0 2024-02-02T15:33:34.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.479+0000 7f5a2f400700 1 --2- 172.21.15.1:0/4119597533 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3de79d010 0x55c3de79d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:33:34.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.479+0000 7f5a34a00700 1 --2- 172.21.15.1:0/4065814074 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f5a100a0690 0x7f5a100a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:34.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.479+0000 7f5a2e000700 1 --2- 172.21.15.1:0/4119597533 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c3de7a87e0 0x55c3de7aabd0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f5a180c96c0 tx=0x7f5a180d41b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e07b249fe76be07 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:33:34.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.479+0000 7f5a34a00700 1 -- 172.21.15.1:0/4065814074 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f5a18014070 con 0x7f5a301060c0 2024-02-02T15:33:34.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.479+0000 7f5a2f400700 1 --2- 172.21.15.1:0/4065814074 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f5a100a0690 0x7f5a100a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:34.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.479+0000 7f5a1fe00700 1 -- 172.21.15.1:0/4119597533 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a18095800 con 0x55c3de7a87e0 2024-02-02T15:33:34.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.480+0000 7f5a1fe00700 1 -- 172.21.15.1:0/4119597533 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a18099290 con 0x55c3de7a87e0 2024-02-02T15:33:34.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.480+0000 7f5a1fe00700 1 -- 172.21.15.1:0/4119597533 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a18099bc0 con 0x55c3de7a87e0 2024-02-02T15:33:34.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.480+0000 7f5a4985fb80 1 -- 172.21.15.1:0/4119597533 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c3de7a87e0 msgr2=0x55c3de7aabd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:34.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.480+0000 7f5a2f400700 1 --2- 172.21.15.1:0/4065814074 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f5a100a0690 0x7f5a100a2b50 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5a28007610 tx=0x7f5a2800b040 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:33:34.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.480+0000 7f5a4985fb80 1 --2- 172.21.15.1:0/4119597533 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c3de7a87e0 0x55c3de7aabd0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f5a180c96c0 tx=0x7f5a180d41b0 comp rx=0 tx=0).stop 2024-02-02T15:33:34.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.481+0000 7f5a4985fb80 1 -- 172.21.15.1:0/4119597533 shutdown_connections 2024-02-02T15:33:34.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.481+0000 7f5a4985fb80 1 --2- 172.21.15.1:0/4119597533 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c3de7a87e0 0x55c3de7aabd0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:34.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.481+0000 7f5a4985fb80 1 --2- 172.21.15.1:0/4119597533 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3de79d010 0x55c3de79d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:34.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.481+0000 7f5a4985fb80 1 --2- 172.21.15.1:0/4119597533 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3de79d930 0x55c3de7a82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:34.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.481+0000 7f5a4985fb80 1 -- 172.21.15.1:0/4119597533 >> 172.21.15.1:0/4119597533 conn(0x55c3de6fcdb0 msgr2=0x55c3de7993b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:34.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.481+0000 7f5a4985fb80 1 -- 172.21.15.1:0/4119597533 shutdown_connections 2024-02-02T15:33:34.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.481+0000 7f5a4985fb80 1 -- 172.21.15.1:0/4119597533 wait complete. 2024-02-02T15:33:34.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.481+0000 7f5a4985fb80 1 -- start start 2024-02-02T15:33:34.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.481+0000 7f5a4985fb80 1 Processor -- start 2024-02-02T15:33:34.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.482+0000 7f5a4985fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3de79d010 0x55c3de7a8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:34.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.482+0000 7f5a4985fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c3de79d930 0x55c3de7a53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:34.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.482+0000 7f5a4985fb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3de7a87e0 0x55c3de7a5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:34.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.482+0000 7f5a4985fb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3de6eed50 con 0x55c3de7a87e0 2024-02-02T15:33:34.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.482+0000 7f5a4985fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3de68ad80 con 0x55c3de79d930 2024-02-02T15:33:34.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.482+0000 7f5a4985fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3de68aa20 con 0x55c3de79d010 2024-02-02T15:33:34.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.482+0000 7f5a4985fb80 10 client.0 _ll_register_callbacks cb 0x55c3de68c290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:33:34.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.482+0000 7f5a2ea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3de79d010 0x55c3de7a8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:34.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.482+0000 7f5a2e000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c3de79d930 0x55c3de7a53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:34.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.482+0000 7f5a2f400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3de7a87e0 0x55c3de7a5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:34.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.482+0000 7f5a2ea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3de79d010 0x55c3de7a8170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:56112/0 (socket says 172.21.15.1:56112) 2024-02-02T15:33:34.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.482+0000 7f5a2e000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c3de79d930 0x55c3de7a53f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:50314/0 (socket says 172.21.15.1:50314) 2024-02-02T15:33:34.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.482+0000 7f5a2ea00700 1 -- 172.21.15.1:0/363064683 learned_addr learned my addr 172.21.15.1:0/363064683 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:33:34.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.483+0000 7f5a2e000700 1 -- 172.21.15.1:0/363064683 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3de79d010 msgr2=0x55c3de7a8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:34.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.483+0000 7f5a2e000700 1 --2- 172.21.15.1:0/363064683 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3de79d010 0x55c3de7a8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:34.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.483+0000 7f5a2e000700 1 -- 172.21.15.1:0/363064683 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3de7a87e0 msgr2=0x55c3de7a5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:34.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.483+0000 7f5a2e000700 1 --2- 172.21.15.1:0/363064683 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3de7a87e0 0x55c3de7a5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:34.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.483+0000 7f5a2e000700 1 -- 172.21.15.1:0/363064683 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a28011330 con 0x55c3de79d930 2024-02-02T15:33:34.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.483+0000 7f5a2e000700 1 --2- 172.21.15.1:0/363064683 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c3de79d930 0x55c3de7a53f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f5a1800efd0 tx=0x7f5a1800ce40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92b5e0177a412f53 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:33:34.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.483+0000 7f5a1cc00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:33:34.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.483+0000 7f5a1cc00700 1 -- 172.21.15.1:0/363064683 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a18095800 con 0x55c3de79d930 2024-02-02T15:33:34.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.483+0000 7f5a4985fb80 1 -- 172.21.15.1:0/363064683 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c3de7910f0 con 0x55c3de79d930 2024-02-02T15:33:34.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.483+0000 7f5a1cc00700 1 -- 172.21.15.1:0/363064683 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f5a180c9e40 con 0x55c3de79d930 2024-02-02T15:33:34.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.483+0000 7f5a1cc00700 1 -- 172.21.15.1:0/363064683 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a18099000 con 0x55c3de79d930 2024-02-02T15:33:34.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.483+0000 7f5a4985fb80 10 client.27410 fetch_fsmap learned FSMap version 38 2024-02-02T15:33:34.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.483+0000 7f5a4985fb80 1 -- 172.21.15.1:0/363064683 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x7f5a301a6220 con 0x55c3de79d930 2024-02-02T15:33:34.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.484+0000 7f5a1cc00700 1 -- 172.21.15.1:0/363064683 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c3de7910f0 con 0x55c3de79d930 2024-02-02T15:33:34.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.484+0000 7f5a1cc00700 1 -- 172.21.15.1:0/363064683 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 38) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f5a180c9970 con 0x55c3de79d930 2024-02-02T15:33:34.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.484+0000 7f5a4985fb80 10 client.27410 fetch_fsmap finished waiting for FSMap version 38 2024-02-02T15:33:34.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.484+0000 7f5a4985fb80 10 client.27410 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:33:34.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.484+0000 7f5a4985fb80 20 client.27410 populate_metadata read hostname 'smithi001' 2024-02-02T15:33:34.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.484+0000 7f5a4985fb80 1 --2- 172.21.15.1:0/363064683 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55c3de7b7d10 0x55c3de7ba100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:34.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.484+0000 7f5a4985fb80 4 client.27410 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:33:34.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.484+0000 7f5a4985fb80 1 -- 172.21.15.1:0/363064683 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c3de6e7dd0 con 0x55c3de7b7d10 2024-02-02T15:33:34.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.484+0000 7f5a2ea00700 1 --2- 172.21.15.1:0/363064683 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55c3de7b7d10 0x55c3de7ba100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:34.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.485+0000 7f5a2ea00700 1 --2- 172.21.15.1:0/363064683 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55c3de7b7d10 0x55c3de7ba100 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c21a5e959a0d081 server_cookie=891c0fe7fe42a93f in_seq=0 out_seq=0 2024-02-02T15:33:34.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.485+0000 7f5a1cc00700 10 client.27410 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:33:34.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.485+0000 7f5a1cc00700 1 -- 172.21.15.1:0/363064683 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55c3de6e7dd0 con 0x55c3de7b7d10 2024-02-02T15:33:34.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.485+0000 7f5a1cc00700 10 client.27410 handle_command_reply: tid=0 2024-02-02T15:33:34.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.486+0000 7f5a4985fb80 1 client.27410 shutdown 2024-02-02T15:33:34.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.486+0000 7f5a4985fb80 1 -- 172.21.15.1:0/363064683 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c3de79d930 msgr2=0x55c3de7a53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:34.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.486+0000 7f5a4985fb80 1 --2- 172.21.15.1:0/363064683 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c3de79d930 0x55c3de7a53f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f5a1800efd0 tx=0x7f5a1800ce40 comp rx=0 tx=0).stop 2024-02-02T15:33:34.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.486+0000 7f5a4985fb80 1 -- 172.21.15.1:0/363064683 shutdown_connections 2024-02-02T15:33:34.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.486+0000 7f5a4985fb80 1 --2- 172.21.15.1:0/363064683 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55c3de7b7d10 0x55c3de7ba100 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:34.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.486+0000 7f5a4985fb80 1 --2- 172.21.15.1:0/363064683 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3de7a87e0 0x55c3de7a5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:34.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.486+0000 7f5a4985fb80 1 --2- 172.21.15.1:0/363064683 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3de79d010 0x55c3de7a8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:34.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.486+0000 7f5a4985fb80 1 --2- 172.21.15.1:0/363064683 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c3de79d930 0x55c3de7a53f0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:34.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.486+0000 7f5a4985fb80 1 -- 172.21.15.1:0/363064683 >> 172.21.15.1:0/363064683 conn(0x55c3de6fcdb0 msgr2=0x55c3de798fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:34.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.486+0000 7f5a1cc00700 0 client.27410 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:33:34.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.486+0000 7f5a4985fb80 1 -- 172.21.15.1:0/363064683 shutdown_connections 2024-02-02T15:33:34.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.486+0000 7f5a4985fb80 1 -- 172.21.15.1:0/363064683 wait complete. 2024-02-02T15:33:34.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.486+0000 7f5a4985fb80 20 client.27410 trim_cache size 0 max 16384 2024-02-02T15:33:34.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.502+0000 7f5a4985fb80 1 Processor -- start 2024-02-02T15:33:34.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.502+0000 7f5a4985fb80 1 -- start start 2024-02-02T15:33:34.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.502+0000 7f5a4985fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3de79d010 0x55c3de7a7d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:34.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.502+0000 7f5a4985fb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3de79d930 0x55c3de79f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:34.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.502+0000 7f5a4985fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c3de7a87e0 0x55c3de79fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:34.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.502+0000 7f5a4985fb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3de5726a0 con 0x55c3de79d930 2024-02-02T15:33:34.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.502+0000 7f5a4985fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3de7a3950 con 0x55c3de7a87e0 2024-02-02T15:33:34.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.502+0000 7f5a4985fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3de7a1240 con 0x55c3de79d010 2024-02-02T15:33:34.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.502+0000 7f5a2ea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3de79d010 0x55c3de7a7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:34.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.503+0000 7f5a2e000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3de79d930 0x55c3de79f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:34.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.503+0000 7f5a2f400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c3de7a87e0 0x55c3de79fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:34.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.503+0000 7f5a2ea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3de79d010 0x55c3de7a7d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:56118/0 (socket says 172.21.15.1:56118) 2024-02-02T15:33:34.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.503+0000 7f5a2ea00700 1 -- 172.21.15.1:0/423866227 learned_addr learned my addr 172.21.15.1:0/423866227 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:33:34.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.503+0000 7f5a2ea00700 1 -- 172.21.15.1:0/423866227 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c3de7a87e0 msgr2=0x55c3de79fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:34.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.503+0000 7f5a2ea00700 1 --2- 172.21.15.1:0/423866227 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c3de7a87e0 0x55c3de79fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:34.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.503+0000 7f5a2ea00700 1 -- 172.21.15.1:0/423866227 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3de79d930 msgr2=0x55c3de79f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:34.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.503+0000 7f5a2ea00700 1 --2- 172.21.15.1:0/423866227 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3de79d930 0x55c3de79f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:34.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.503+0000 7f5a2ea00700 1 -- 172.21.15.1:0/423866227 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3de51cb60 con 0x55c3de79d010 2024-02-02T15:33:34.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.503+0000 7f5a2e000700 1 --2- 172.21.15.1:0/423866227 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3de79d930 0x55c3de79f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:33:34.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.503+0000 7f5a2ea00700 1 --2- 172.21.15.1:0/423866227 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3de79d010 0x55c3de7a7d10 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f5a200098e0 tx=0x7f5a20005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6337c0c135c228ce server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:33:34.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.503+0000 7f5a25200700 1 -- 172.21.15.1:0/423866227 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a2002a070 con 0x55c3de79d010 2024-02-02T15:33:34.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.503+0000 7f5a25200700 1 -- 172.21.15.1:0/423866227 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a2001e040 con 0x55c3de79d010 2024-02-02T15:33:34.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.503+0000 7f5a25200700 1 -- 172.21.15.1:0/423866227 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a2001d040 con 0x55c3de79d010 2024-02-02T15:33:34.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.504+0000 7f5a4985fb80 1 -- 172.21.15.1:0/423866227 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3de79d010 msgr2=0x55c3de7a7d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:34.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.504+0000 7f5a4985fb80 1 --2- 172.21.15.1:0/423866227 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3de79d010 0x55c3de7a7d10 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f5a200098e0 tx=0x7f5a20005fd0 comp rx=0 tx=0).stop 2024-02-02T15:33:34.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.504+0000 7f5a4985fb80 1 -- 172.21.15.1:0/423866227 shutdown_connections 2024-02-02T15:33:34.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.504+0000 7f5a4985fb80 1 --2- 172.21.15.1:0/423866227 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c3de7a87e0 0x55c3de79fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:34.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.504+0000 7f5a4985fb80 1 --2- 172.21.15.1:0/423866227 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3de79d010 0x55c3de7a7d10 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:34.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.504+0000 7f5a4985fb80 1 --2- 172.21.15.1:0/423866227 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3de79d930 0x55c3de79f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:34.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.504+0000 7f5a4985fb80 1 -- 172.21.15.1:0/423866227 >> 172.21.15.1:0/423866227 conn(0x55c3de6fcdb0 msgr2=0x55c3de7a0740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:34.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.504+0000 7f5a4985fb80 1 -- 172.21.15.1:0/423866227 shutdown_connections 2024-02-02T15:33:34.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.504+0000 7f5a4985fb80 1 -- 172.21.15.1:0/423866227 wait complete. 2024-02-02T15:33:34.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.505+0000 7f5a4985fb80 1 -- start start 2024-02-02T15:33:34.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.505+0000 7f5a4985fb80 1 Processor -- start 2024-02-02T15:33:34.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.505+0000 7f5a4985fb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3de79d010 0x55c3de7af390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:34.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.505+0000 7f5a4985fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c3de79d930 0x55c3de7ad6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:34.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.505+0000 7f5a4985fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3de7a87e0 0x55c3de7ade00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:34.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.505+0000 7f5a4985fb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3de7ae510 con 0x55c3de79d010 2024-02-02T15:33:34.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.505+0000 7f5a4985fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3de8097e0 con 0x55c3de79d930 2024-02-02T15:33:34.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.505+0000 7f5a4985fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3de809920 con 0x55c3de7a87e0 2024-02-02T15:33:34.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.505+0000 7f5a4985fb80 10 client.0 _ll_register_callbacks cb 0x55c3de68c290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:33:34.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.506+0000 7f5a2ea00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3de79d010 0x55c3de7af390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:34.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.506+0000 7f5a2e000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c3de79d930 0x55c3de7ad6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:34.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.506+0000 7f5a2e000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c3de79d930 0x55c3de7ad6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:50322/0 (socket says 172.21.15.1:50322) 2024-02-02T15:33:34.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.506+0000 7f5a2e000700 1 -- 172.21.15.1:0/3705643970 learned_addr learned my addr 172.21.15.1:0/3705643970 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:33:34.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.506+0000 7f5a2f400700 1 --2- 172.21.15.1:0/3705643970 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3de7a87e0 0x55c3de7ade00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:34.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.506+0000 7f5a2ea00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3de79d010 0x55c3de7af390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:58044/0 (socket says 172.21.15.1:58044) 2024-02-02T15:33:34.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.506+0000 7f5a2e000700 1 -- 172.21.15.1:0/3705643970 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3de7a87e0 msgr2=0x55c3de7ade00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:34.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.506+0000 7f5a2e000700 1 --2- 172.21.15.1:0/3705643970 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3de7a87e0 0x55c3de7ade00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:34.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.506+0000 7f5a2e000700 1 -- 172.21.15.1:0/3705643970 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3de79d010 msgr2=0x55c3de7af390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:34.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.506+0000 7f5a2e000700 1 --2- 172.21.15.1:0/3705643970 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3de79d010 0x55c3de7af390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:34.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.506+0000 7f5a2e000700 1 -- 172.21.15.1:0/3705643970 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a301a6220 con 0x55c3de79d930 2024-02-02T15:33:34.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.506+0000 7f5a2e000700 1 --2- 172.21.15.1:0/3705643970 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c3de79d930 0x55c3de7ad6c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f5a18095f30 tx=0x7f5a18097a70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9088a115b21a88c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:33:34.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.506+0000 7f5a1cc00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:33:34.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.507+0000 7f5a1cc00700 1 -- 172.21.15.1:0/3705643970 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a18099000 con 0x55c3de79d930 2024-02-02T15:33:34.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.507+0000 7f5a1cc00700 1 -- 172.21.15.1:0/3705643970 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f5a180c9e40 con 0x55c3de79d930 2024-02-02T15:33:34.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.507+0000 7f5a4985fb80 1 -- 172.21.15.1:0/3705643970 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5a18099290 con 0x55c3de79d930 2024-02-02T15:33:34.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.507+0000 7f5a1cc00700 1 -- 172.21.15.1:0/3705643970 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a18095800 con 0x55c3de79d930 2024-02-02T15:33:34.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.507+0000 7f5a4985fb80 10 client.27416 fetch_fsmap learned FSMap version 38 2024-02-02T15:33:34.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.507+0000 7f5a4985fb80 1 -- 172.21.15.1:0/3705643970 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x7f5a3010e4b0 con 0x55c3de79d930 2024-02-02T15:33:34.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.507+0000 7f5a1cc00700 1 -- 172.21.15.1:0/3705643970 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5a18099290 con 0x55c3de79d930 2024-02-02T15:33:34.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.507+0000 7f5a1cc00700 1 -- 172.21.15.1:0/3705643970 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 38) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f5a18099720 con 0x55c3de79d930 2024-02-02T15:33:34.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.507+0000 7f5a4985fb80 10 client.27416 fetch_fsmap finished waiting for FSMap version 38 2024-02-02T15:33:34.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.507+0000 7f5a4985fb80 10 client.27416 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:33:34.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.507+0000 7f5a4985fb80 20 client.27416 populate_metadata read hostname 'smithi001' 2024-02-02T15:33:34.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.507+0000 7f5a4985fb80 1 --2- 172.21.15.1:0/3705643970 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55c3de80abc0 0x55c3de80cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:34.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.507+0000 7f5a4985fb80 4 client.27416 mds_command: new command op to 24557 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:33:34.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.507+0000 7f5a4985fb80 1 -- 172.21.15.1:0/3705643970 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c3de6adf50 con 0x55c3de80abc0 2024-02-02T15:33:34.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.508+0000 7f5a2ea00700 1 --2- 172.21.15.1:0/3705643970 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55c3de80abc0 0x55c3de80cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:34.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.508+0000 7f5a2ea00700 1 --2- 172.21.15.1:0/3705643970 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55c3de80abc0 0x55c3de80cfe0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a0396181ad8ab64 server_cookie=2c6e03d4a3c389cc in_seq=0 out_seq=0 2024-02-02T15:33:34.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.508+0000 7f5a1cc00700 10 client.27416 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:33:34.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.508+0000 7f5a1cc00700 1 -- 172.21.15.1:0/3705643970 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c3de6adf50 con 0x55c3de80abc0 2024-02-02T15:33:34.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.508+0000 7f5a1cc00700 10 client.27416 handle_command_reply: tid=0 2024-02-02T15:33:34.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.508+0000 7f5a4985fb80 1 client.27416 shutdown 2024-02-02T15:33:34.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.509+0000 7f5a4985fb80 1 -- 172.21.15.1:0/3705643970 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c3de79d930 msgr2=0x55c3de7ad6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:34.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.509+0000 7f5a4985fb80 1 --2- 172.21.15.1:0/3705643970 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c3de79d930 0x55c3de7ad6c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f5a18095f30 tx=0x7f5a18097a70 comp rx=0 tx=0).stop 2024-02-02T15:33:34.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.509+0000 7f5a4985fb80 1 -- 172.21.15.1:0/3705643970 shutdown_connections 2024-02-02T15:33:34.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.509+0000 7f5a4985fb80 1 --2- 172.21.15.1:0/3705643970 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55c3de80abc0 0x55c3de80cfe0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:34.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.509+0000 7f5a4985fb80 1 --2- 172.21.15.1:0/3705643970 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3de7a87e0 0x55c3de7ade00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:34.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.509+0000 7f5a1cc00700 0 client.27416 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:33:34.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.509+0000 7f5a4985fb80 1 --2- 172.21.15.1:0/3705643970 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3de79d010 0x55c3de7af390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:34.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.509+0000 7f5a4985fb80 1 --2- 172.21.15.1:0/3705643970 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c3de79d930 0x55c3de7ad6c0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:34.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.509+0000 7f5a4985fb80 1 -- 172.21.15.1:0/3705643970 >> 172.21.15.1:0/3705643970 conn(0x55c3de6fcdb0 msgr2=0x55c3de7985c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:34.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.509+0000 7f5a4985fb80 1 -- 172.21.15.1:0/3705643970 shutdown_connections 2024-02-02T15:33:34.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.509+0000 7f5a4985fb80 1 -- 172.21.15.1:0/3705643970 wait complete. 2024-02-02T15:33:34.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.509+0000 7f5a4985fb80 20 client.27416 trim_cache size 0 max 16384 2024-02-02T15:33:34.513 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:33:34.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.510+0000 7f5a35e00700 1 -- 172.21.15.1:0/4065814074 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f5a100a0690 msgr2=0x7f5a100a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:34.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.510+0000 7f5a35e00700 1 --2- 172.21.15.1:0/4065814074 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f5a100a0690 0x7f5a100a2b50 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5a28007610 tx=0x7f5a2800b040 comp rx=0 tx=0).stop 2024-02-02T15:33:34.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.510+0000 7f5a35e00700 1 -- 172.21.15.1:0/4065814074 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a301060c0 msgr2=0x7f5a30073580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:34.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.510+0000 7f5a35e00700 1 --2- 172.21.15.1:0/4065814074 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a301060c0 0x7f5a30073580 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f5a1800e970 tx=0x7f5a1800ec80 comp rx=0 tx=0).stop 2024-02-02T15:33:34.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.510+0000 7f5a35e00700 1 -- 172.21.15.1:0/4065814074 shutdown_connections 2024-02-02T15:33:34.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.510+0000 7f5a35e00700 1 --2- 172.21.15.1:0/4065814074 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f5a100a0690 0x7f5a100a2b50 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:34.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.510+0000 7f5a35e00700 1 --2- 172.21.15.1:0/4065814074 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5a30073ac0 0x7f5a301a50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:34.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.510+0000 7f5a35e00700 1 --2- 172.21.15.1:0/4065814074 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5a30100fa0 0x7f5a30073040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:34.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.510+0000 7f5a35e00700 1 --2- 172.21.15.1:0/4065814074 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5a301060c0 0x7f5a30073580 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:34.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.510+0000 7f5a35e00700 1 -- 172.21.15.1:0/4065814074 >> 172.21.15.1:0/4065814074 conn(0x7f5a300fc800 msgr2=0x7f5a301049d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:34.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.510+0000 7f5a35e00700 1 -- 172.21.15.1:0/4065814074 shutdown_connections 2024-02-02T15:33:34.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:34.510+0000 7f5a35e00700 1 -- 172.21.15.1:0/4065814074 wait complete. 2024-02-02T15:33:34.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:34 smithi071 ceph-mon[44707]: pgmap v200: 129 pgs: 129 active+clean; 269 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 145 op/s 2024-02-02T15:33:34.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:34 smithi142 ceph-mon[45728]: pgmap v200: 129 pgs: 129 active+clean; 269 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 145 op/s 2024-02-02T15:33:34.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:34 smithi001 ceph-mon[35470]: pgmap v200: 129 pgs: 129 active+clean; 269 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.7 MiB/s wr, 145 op/s 2024-02-02T15:33:35.240 INFO:tasks.workunit.client.0.smithi001.stdout: 1 425898 8.16 MB/sec execute 189 sec latency 23.407 ms 2024-02-02T15:33:35.529 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-02T15:33:35.530 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-02T15:33:35.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.744+0000 7f23c6400700 1 -- 172.21.15.1:0/3955683071 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f23c01019a0 msgr2=0x7f23c01059e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:35.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.744+0000 7f23c6400700 1 --2- 172.21.15.1:0/3955683071 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f23c01019a0 0x7f23c01059e0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f23a8009990 tx=0x7f23a8009ca0 comp rx=0 tx=0).stop 2024-02-02T15:33:35.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.744+0000 7f23c6400700 1 -- 172.21.15.1:0/3955683071 shutdown_connections 2024-02-02T15:33:35.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.744+0000 7f23c6400700 1 --2- 172.21.15.1:0/3955683071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f23c0106110 0x7f23c010dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:35.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.744+0000 7f23c6400700 1 --2- 172.21.15.1:0/3955683071 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f23c0100ff0 0x7f23c01013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:35.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.744+0000 7f23c6400700 1 --2- 172.21.15.1:0/3955683071 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f23c01019a0 0x7f23c01059e0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:35.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.744+0000 7f23c6400700 1 -- 172.21.15.1:0/3955683071 >> 172.21.15.1:0/3955683071 conn(0x7f23c00fc810 msgr2=0x7f23c00fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:35.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.744+0000 7f23c6400700 1 -- 172.21.15.1:0/3955683071 shutdown_connections 2024-02-02T15:33:35.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.744+0000 7f23c6400700 1 -- 172.21.15.1:0/3955683071 wait complete. 2024-02-02T15:33:35.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.745+0000 7f23c6400700 1 Processor -- start 2024-02-02T15:33:35.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.745+0000 7f23c6400700 1 -- start start 2024-02-02T15:33:35.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.746+0000 7f23c6400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f23c0100ff0 0x7f23c0199060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:35.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.746+0000 7f23c6400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f23c0106110 0x7f23c01995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:35.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.746+0000 7f23c6400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f23c019da00 0x7f23c019de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:35.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.746+0000 7f23c6400700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23c019e480 con 0x7f23c0106110 2024-02-02T15:33:35.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.746+0000 7f23c6400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23c019e5f0 con 0x7f23c019da00 2024-02-02T15:33:35.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.746+0000 7f23c6400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23c019e7b0 con 0x7f23c0100ff0 2024-02-02T15:33:35.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.746+0000 7f23bf400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f23c0100ff0 0x7f23c0199060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:35.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.746+0000 7f23bf400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f23c0100ff0 0x7f23c0199060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:56130/0 (socket says 172.21.15.1:56130) 2024-02-02T15:33:35.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.746+0000 7f23bf400700 1 -- 172.21.15.1:0/2785564680 learned_addr learned my addr 172.21.15.1:0/2785564680 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:33:35.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.746+0000 7f23bea00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f23c0106110 0x7f23c01995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:35.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.746+0000 7f23bfe00700 1 --2- 172.21.15.1:0/2785564680 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f23c019da00 0x7f23c019de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:35.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.746+0000 7f23bf400700 1 -- 172.21.15.1:0/2785564680 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f23c019da00 msgr2=0x7f23c019de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:35.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.746+0000 7f23bf400700 1 --2- 172.21.15.1:0/2785564680 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f23c019da00 0x7f23c019de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:35.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.746+0000 7f23bf400700 1 -- 172.21.15.1:0/2785564680 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f23c0106110 msgr2=0x7f23c01995a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:35.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.746+0000 7f23bf400700 1 --2- 172.21.15.1:0/2785564680 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f23c0106110 0x7f23c01995a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:35.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.746+0000 7f23bf400700 1 -- 172.21.15.1:0/2785564680 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23a80095f0 con 0x7f23c0100ff0 2024-02-02T15:33:35.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.746+0000 7f23bfe00700 1 --2- 172.21.15.1:0/2785564680 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f23c019da00 0x7f23c019de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:33:35.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.746+0000 7f23bea00700 1 --2- 172.21.15.1:0/2785564680 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f23c0106110 0x7f23c01995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:33:35.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.747+0000 7f23bf400700 1 --2- 172.21.15.1:0/2785564680 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f23c0100ff0 0x7f23c0199060 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f23b000e9a0 tx=0x7f23b000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:33:35.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.747+0000 7f23b7e00700 1 -- 172.21.15.1:0/2785564680 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b0009790 con 0x7f23c0100ff0 2024-02-02T15:33:35.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.747+0000 7f23b7e00700 1 -- 172.21.15.1:0/2785564680 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23b0010430 con 0x7f23c0100ff0 2024-02-02T15:33:35.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.747+0000 7f23c6400700 1 -- 172.21.15.1:0/2785564680 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23c01a5d10 con 0x7f23c0100ff0 2024-02-02T15:33:35.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.747+0000 7f23b7e00700 1 -- 172.21.15.1:0/2785564680 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b0018680 con 0x7f23c0100ff0 2024-02-02T15:33:35.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.747+0000 7f23c6400700 1 -- 172.21.15.1:0/2785564680 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23c01a61a0 con 0x7f23c0100ff0 2024-02-02T15:33:35.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.750+0000 7f23b7e00700 1 -- 172.21.15.1:0/2785564680 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 27) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f23b00105a0 con 0x7f23c0100ff0 2024-02-02T15:33:35.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.752+0000 7f23d9f4bb80 1 Processor -- start 2024-02-02T15:33:35.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.752+0000 7f23b7e00700 1 --2- 172.21.15.1:0/2785564680 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f23a00a0690 0x7f23a00a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:35.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.752+0000 7f23d9f4bb80 1 -- start start 2024-02-02T15:33:35.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.752+0000 7f23b7e00700 1 -- 172.21.15.1:0/2785564680 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f23b0014070 con 0x7f23c0100ff0 2024-02-02T15:33:35.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.752+0000 7f23bfe00700 1 --2- 172.21.15.1:0/2785564680 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f23a00a0690 0x7f23a00a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:35.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.752+0000 7f23d9f4bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563aac30e090 0x563aac30e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:35.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.752+0000 7f23d9f4bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563aac30e9b0 0x563aac319320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:35.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.752+0000 7f23d9f4bb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563aac319860 0x563aac31bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:35.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.752+0000 7f23d9f4bb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x563aac282540 con 0x563aac319860 2024-02-02T15:33:35.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.752+0000 7f23d9f4bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x563aac266b20 con 0x563aac30e9b0 2024-02-02T15:33:35.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.752+0000 7f23d9f4bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563aac2644e0 con 0x563aac30e090 2024-02-02T15:33:35.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.752+0000 7f23bf400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563aac30e090 0x563aac30e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:35.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.753+0000 7f23bf400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563aac30e090 0x563aac30e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:56144/0 (socket says 172.21.15.1:56144) 2024-02-02T15:33:35.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.753+0000 7f23bf400700 1 -- 172.21.15.1:0/352874229 learned_addr learned my addr 172.21.15.1:0/352874229 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:33:35.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.753+0000 7f23bea00700 1 --2- 172.21.15.1:0/352874229 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563aac30e9b0 0x563aac319320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:35.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.753+0000 7f23bf400700 1 -- 172.21.15.1:0/352874229 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563aac30e9b0 msgr2=0x563aac319320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:35.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.753+0000 7f23bf400700 1 --2- 172.21.15.1:0/352874229 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563aac30e9b0 0x563aac319320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:35.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.753+0000 7f23bf400700 1 -- 172.21.15.1:0/352874229 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563aac319860 msgr2=0x563aac31bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:35.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.753+0000 7f23bfe00700 1 --2- 172.21.15.1:0/352874229 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563aac319860 0x563aac31bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:35.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.753+0000 7f23bf400700 1 --2- 172.21.15.1:0/352874229 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563aac319860 0x563aac31bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:35.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.753+0000 7f23bf400700 1 -- 172.21.15.1:0/352874229 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563aac0e1430 con 0x563aac30e090 2024-02-02T15:33:35.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.753+0000 7f23bfe00700 1 --2- 172.21.15.1:0/352874229 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563aac319860 0x563aac31bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:33:35.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.753+0000 7f23bfe00700 1 --2- 172.21.15.1:0/2785564680 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f23a00a0690 0x7f23a00a2b50 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f23b8003e80 tx=0x7f23b8009920 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:33:35.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.753+0000 7f23bf400700 1 --2- 172.21.15.1:0/352874229 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563aac30e090 0x563aac30e470 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f23b0099390 tx=0x7f23b0098de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1255024b4cbfa0c8 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:33:35.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.753+0000 7f23b4c00700 1 -- 172.21.15.1:0/352874229 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b0099400 con 0x563aac30e090 2024-02-02T15:33:35.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.754+0000 7f23b4c00700 1 -- 172.21.15.1:0/352874229 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23b009d230 con 0x563aac30e090 2024-02-02T15:33:35.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.754+0000 7f23b4c00700 1 -- 172.21.15.1:0/352874229 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b00998c0 con 0x563aac30e090 2024-02-02T15:33:35.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.754+0000 7f23d9f4bb80 1 -- 172.21.15.1:0/352874229 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563aac30e090 msgr2=0x563aac30e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:35.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.754+0000 7f23d9f4bb80 1 --2- 172.21.15.1:0/352874229 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563aac30e090 0x563aac30e470 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f23b0099390 tx=0x7f23b0098de0 comp rx=0 tx=0).stop 2024-02-02T15:33:35.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.754+0000 7f23d9f4bb80 1 -- 172.21.15.1:0/352874229 shutdown_connections 2024-02-02T15:33:35.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.754+0000 7f23d9f4bb80 1 --2- 172.21.15.1:0/352874229 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563aac319860 0x563aac31bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:35.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.754+0000 7f23d9f4bb80 1 --2- 172.21.15.1:0/352874229 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563aac30e090 0x563aac30e470 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:35.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.754+0000 7f23d9f4bb80 1 --2- 172.21.15.1:0/352874229 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563aac30e9b0 0x563aac319320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:35.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.754+0000 7f23d9f4bb80 1 -- 172.21.15.1:0/352874229 >> 172.21.15.1:0/352874229 conn(0x563aac26de30 msgr2=0x563aac30a430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:35.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.754+0000 7f23d9f4bb80 1 -- 172.21.15.1:0/352874229 shutdown_connections 2024-02-02T15:33:35.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.754+0000 7f23d9f4bb80 1 -- 172.21.15.1:0/352874229 wait complete. 2024-02-02T15:33:35.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.755+0000 7f23d9f4bb80 1 -- start start 2024-02-02T15:33:35.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.755+0000 7f23d9f4bb80 1 Processor -- start 2024-02-02T15:33:35.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.756+0000 7f23d9f4bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563aac30e090 0x563aac319310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:35.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.756+0000 7f23d9f4bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563aac30e9b0 0x563aac316520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:35.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.756+0000 7f23d9f4bb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563aac319860 0x563aac316c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:35.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.756+0000 7f23d9f4bb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x563aac25fdd0 con 0x563aac319860 2024-02-02T15:33:35.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.756+0000 7f23d9f4bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x563aac1fbe00 con 0x563aac30e9b0 2024-02-02T15:33:35.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.756+0000 7f23d9f4bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563aac1fbaa0 con 0x563aac30e090 2024-02-02T15:33:35.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.756+0000 7f23d9f4bb80 10 client.0 _ll_register_callbacks cb 0x563aac1fd310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:33:35.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.756+0000 7f23bea00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563aac30e9b0 0x563aac316520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:35.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.756+0000 7f23bf400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563aac30e090 0x563aac319310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:35.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.756+0000 7f23bea00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563aac30e9b0 0x563aac316520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:50356/0 (socket says 172.21.15.1:50356) 2024-02-02T15:33:35.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.756+0000 7f23bea00700 1 -- 172.21.15.1:0/1449550925 learned_addr learned my addr 172.21.15.1:0/1449550925 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:33:35.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.756+0000 7f23bfe00700 1 --2- 172.21.15.1:0/1449550925 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563aac319860 0x563aac316c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:35.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.756+0000 7f23bea00700 1 -- 172.21.15.1:0/1449550925 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563aac30e090 msgr2=0x563aac319310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:35.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.756+0000 7f23bea00700 1 --2- 172.21.15.1:0/1449550925 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563aac30e090 0x563aac319310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:35.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.756+0000 7f23bea00700 1 -- 172.21.15.1:0/1449550925 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563aac319860 msgr2=0x563aac316c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:35.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.756+0000 7f23bea00700 1 --2- 172.21.15.1:0/1449550925 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563aac319860 0x563aac316c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:35.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.756+0000 7f23bea00700 1 -- 172.21.15.1:0/1449550925 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23c01a61a0 con 0x563aac30e9b0 2024-02-02T15:33:35.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.756+0000 7f23bfe00700 1 --2- 172.21.15.1:0/1449550925 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563aac319860 0x563aac316c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:33:35.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.757+0000 7f23bea00700 1 --2- 172.21.15.1:0/1449550925 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563aac30e9b0 0x563aac316520 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f23a8009990 tx=0x7f23a8005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e70b5275947bb598 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:33:35.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.757+0000 7f23ad600700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:33:35.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.757+0000 7f23ad600700 1 -- 172.21.15.1:0/1449550925 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a801d070 con 0x563aac30e9b0 2024-02-02T15:33:35.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.757+0000 7f23d9f4bb80 1 -- 172.21.15.1:0/1449550925 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563aac302170 con 0x563aac30e9b0 2024-02-02T15:33:35.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.757+0000 7f23ad600700 1 -- 172.21.15.1:0/1449550925 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f23a8004910 con 0x563aac30e9b0 2024-02-02T15:33:35.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.757+0000 7f23ad600700 1 -- 172.21.15.1:0/1449550925 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23a8004ad0 con 0x563aac30e9b0 2024-02-02T15:33:35.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.757+0000 7f23d9f4bb80 10 client.27428 fetch_fsmap learned FSMap version 38 2024-02-02T15:33:35.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.757+0000 7f23d9f4bb80 1 -- 172.21.15.1:0/1449550925 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x563aac08dbe0 con 0x563aac30e9b0 2024-02-02T15:33:35.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.757+0000 7f23ad600700 1 -- 172.21.15.1:0/1449550925 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563aac302170 con 0x563aac30e9b0 2024-02-02T15:33:35.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.758+0000 7f23ad600700 1 -- 172.21.15.1:0/1449550925 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 38) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f23a8024450 con 0x563aac30e9b0 2024-02-02T15:33:35.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.758+0000 7f23d9f4bb80 10 client.27428 fetch_fsmap finished waiting for FSMap version 38 2024-02-02T15:33:35.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.758+0000 7f23d9f4bb80 10 client.27428 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:33:35.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.758+0000 7f23d9f4bb80 20 client.27428 populate_metadata read hostname 'smithi001' 2024-02-02T15:33:35.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.758+0000 7f23d9f4bb80 1 --2- 172.21.15.1:0/1449550925 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x563aac3293e0 0x563aac32b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:35.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.758+0000 7f23d9f4bb80 4 client.27428 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:33:35.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.758+0000 7f23d9f4bb80 1 -- 172.21.15.1:0/1449550925 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563aac258e50 con 0x563aac3293e0 2024-02-02T15:33:35.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.758+0000 7f23bf400700 1 --2- 172.21.15.1:0/1449550925 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x563aac3293e0 0x563aac32b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:35.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.758+0000 7f23bf400700 1 --2- 172.21.15.1:0/1449550925 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x563aac3293e0 0x563aac32b800 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f56d667fed16eee2 server_cookie=430576202b07d098 in_seq=0 out_seq=0 2024-02-02T15:33:35.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.758+0000 7f23ad600700 10 client.27428 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:33:35.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.759+0000 7f23ad600700 1 -- 172.21.15.1:0/1449550925 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x563aac258e50 con 0x563aac3293e0 2024-02-02T15:33:35.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.759+0000 7f23ad600700 10 client.27428 handle_command_reply: tid=0 2024-02-02T15:33:35.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.759+0000 7f23d9f4bb80 1 client.27428 shutdown 2024-02-02T15:33:35.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.759+0000 7f23d9f4bb80 1 -- 172.21.15.1:0/1449550925 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563aac30e9b0 msgr2=0x563aac316520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:35.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.759+0000 7f23d9f4bb80 1 --2- 172.21.15.1:0/1449550925 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563aac30e9b0 0x563aac316520 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f23a8009990 tx=0x7f23a8005270 comp rx=0 tx=0).stop 2024-02-02T15:33:35.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.759+0000 7f23d9f4bb80 1 -- 172.21.15.1:0/1449550925 shutdown_connections 2024-02-02T15:33:35.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.759+0000 7f23d9f4bb80 1 --2- 172.21.15.1:0/1449550925 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x563aac3293e0 0x563aac32b800 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:35.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.759+0000 7f23d9f4bb80 1 --2- 172.21.15.1:0/1449550925 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563aac319860 0x563aac316c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:35.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.759+0000 7f23d9f4bb80 1 --2- 172.21.15.1:0/1449550925 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563aac30e090 0x563aac319310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:35.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.759+0000 7f23d9f4bb80 1 --2- 172.21.15.1:0/1449550925 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563aac30e9b0 0x563aac316520 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:35.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.759+0000 7f23ad600700 0 client.27428 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:33:35.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.759+0000 7f23d9f4bb80 1 -- 172.21.15.1:0/1449550925 >> 172.21.15.1:0/1449550925 conn(0x563aac26de30 msgr2=0x563aac30a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:35.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.760+0000 7f23d9f4bb80 1 -- 172.21.15.1:0/1449550925 shutdown_connections 2024-02-02T15:33:35.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.760+0000 7f23d9f4bb80 1 -- 172.21.15.1:0/1449550925 wait complete. 2024-02-02T15:33:35.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.760+0000 7f23d9f4bb80 20 client.27428 trim_cache size 0 max 16384 2024-02-02T15:33:35.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.779+0000 7f23d9f4bb80 1 Processor -- start 2024-02-02T15:33:35.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.779+0000 7f23d9f4bb80 1 -- start start 2024-02-02T15:33:35.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.780+0000 7f23d9f4bb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563aac30e090 0x563aac3203f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:35.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.780+0000 7f23d9f4bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563aac30e9b0 0x563aac311820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:35.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.780+0000 7f23d9f4bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563aac319860 0x563aac311d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:35.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.780+0000 7f23d9f4bb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x563aac0e3720 con 0x563aac30e090 2024-02-02T15:33:35.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.780+0000 7f23d9f4bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x563aac325d40 con 0x563aac319860 2024-02-02T15:33:35.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.780+0000 7f23d9f4bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563aac317260 con 0x563aac30e9b0 2024-02-02T15:33:35.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.780+0000 7f23bf400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563aac30e090 0x563aac3203f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:35.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.780+0000 7f23bfe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563aac319860 0x563aac311d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:35.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.780+0000 7f23bfe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563aac319860 0x563aac311d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:50364/0 (socket says 172.21.15.1:50364) 2024-02-02T15:33:35.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.780+0000 7f23bfe00700 1 -- 172.21.15.1:0/454241756 learned_addr learned my addr 172.21.15.1:0/454241756 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:33:35.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.780+0000 7f23bea00700 1 --2- 172.21.15.1:0/454241756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563aac30e9b0 0x563aac311820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:35.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.780+0000 7f23bfe00700 1 -- 172.21.15.1:0/454241756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563aac30e9b0 msgr2=0x563aac311820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:35.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.780+0000 7f23bfe00700 1 --2- 172.21.15.1:0/454241756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563aac30e9b0 0x563aac311820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:35.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.780+0000 7f23bfe00700 1 -- 172.21.15.1:0/454241756 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563aac30e090 msgr2=0x563aac3203f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:35.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.780+0000 7f23bfe00700 1 --2- 172.21.15.1:0/454241756 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563aac30e090 0x563aac3203f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:35.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.780+0000 7f23bfe00700 1 -- 172.21.15.1:0/454241756 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563aac30fb80 con 0x563aac319860 2024-02-02T15:33:35.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.781+0000 7f23bfe00700 1 --2- 172.21.15.1:0/454241756 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563aac319860 0x563aac311d60 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f23b800e890 tx=0x7f23b800b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32f92b02bd09af74 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:33:35.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.781+0000 7f23b5600700 1 -- 172.21.15.1:0/454241756 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b8004930 con 0x563aac319860 2024-02-02T15:33:35.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.781+0000 7f23b5600700 1 -- 172.21.15.1:0/454241756 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23b800f040 con 0x563aac319860 2024-02-02T15:33:35.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.781+0000 7f23b5600700 1 -- 172.21.15.1:0/454241756 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b8017d00 con 0x563aac319860 2024-02-02T15:33:35.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.782+0000 7f23d9f4bb80 1 -- 172.21.15.1:0/454241756 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563aac319860 msgr2=0x563aac311d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:35.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.782+0000 7f23d9f4bb80 1 --2- 172.21.15.1:0/454241756 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563aac319860 0x563aac311d60 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f23b800e890 tx=0x7f23b800b040 comp rx=0 tx=0).stop 2024-02-02T15:33:35.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.782+0000 7f23d9f4bb80 1 -- 172.21.15.1:0/454241756 shutdown_connections 2024-02-02T15:33:35.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.782+0000 7f23d9f4bb80 1 --2- 172.21.15.1:0/454241756 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563aac319860 0x563aac311d60 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:35.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.782+0000 7f23d9f4bb80 1 --2- 172.21.15.1:0/454241756 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563aac30e090 0x563aac3203f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:35.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.782+0000 7f23d9f4bb80 1 --2- 172.21.15.1:0/454241756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563aac30e9b0 0x563aac311820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:35.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.782+0000 7f23d9f4bb80 1 -- 172.21.15.1:0/454241756 >> 172.21.15.1:0/454241756 conn(0x563aac26de30 msgr2=0x563aac3249c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:35.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.782+0000 7f23d9f4bb80 1 -- 172.21.15.1:0/454241756 shutdown_connections 2024-02-02T15:33:35.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.782+0000 7f23d9f4bb80 1 -- 172.21.15.1:0/454241756 wait complete. 2024-02-02T15:33:35.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.782+0000 7f23d9f4bb80 1 -- start start 2024-02-02T15:33:35.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.783+0000 7f23d9f4bb80 1 Processor -- start 2024-02-02T15:33:35.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.783+0000 7f23d9f4bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563aac30e090 0x563aac31ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:35.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.783+0000 7f23d9f4bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563aac30e9b0 0x563aac31f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:35.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.783+0000 7f23d9f4bb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563aac319860 0x563aac37af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:35.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.783+0000 7f23d9f4bb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x563aac31f9e0 con 0x563aac319860 2024-02-02T15:33:35.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.783+0000 7f23d9f4bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x563aac37b450 con 0x563aac30e9b0 2024-02-02T15:33:35.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.783+0000 7f23d9f4bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563aac37b590 con 0x563aac30e090 2024-02-02T15:33:35.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.783+0000 7f23d9f4bb80 10 client.0 _ll_register_callbacks cb 0x563aac1fd310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:33:35.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.783+0000 7f23bf400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563aac30e090 0x563aac31ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:35.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.784+0000 7f23bea00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563aac30e9b0 0x563aac31f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:35.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.784+0000 7f23bf400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563aac30e090 0x563aac31ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:56166/0 (socket says 172.21.15.1:56166) 2024-02-02T15:33:35.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.784+0000 7f23bf400700 1 -- 172.21.15.1:0/1376993040 learned_addr learned my addr 172.21.15.1:0/1376993040 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:33:35.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.784+0000 7f23bfe00700 1 --2- 172.21.15.1:0/1376993040 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563aac319860 0x563aac37af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:35.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.784+0000 7f23bea00700 1 -- 172.21.15.1:0/1376993040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563aac30e090 msgr2=0x563aac31ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:35.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.784+0000 7f23bea00700 1 --2- 172.21.15.1:0/1376993040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563aac30e090 0x563aac31ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:35.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.784+0000 7f23bea00700 1 -- 172.21.15.1:0/1376993040 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563aac319860 msgr2=0x563aac37af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:35.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.784+0000 7f23bea00700 1 --2- 172.21.15.1:0/1376993040 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563aac319860 0x563aac37af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:35.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.784+0000 7f23bea00700 1 -- 172.21.15.1:0/1376993040 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23b800e4f0 con 0x563aac30e9b0 2024-02-02T15:33:35.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.784+0000 7f23bf400700 1 --2- 172.21.15.1:0/1376993040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563aac30e090 0x563aac31ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:33:35.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.784+0000 7f23bea00700 1 --2- 172.21.15.1:0/1376993040 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563aac30e9b0 0x563aac31f100 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f23a8024780 tx=0x7f23a8024680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85cebb65f7a9b248 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:33:35.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.784+0000 7f23ad600700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:33:35.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.784+0000 7f23ad600700 1 -- 172.21.15.1:0/1376993040 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a801d070 con 0x563aac30e9b0 2024-02-02T15:33:35.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.784+0000 7f23d9f4bb80 1 -- 172.21.15.1:0/1376993040 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f23b009d230 con 0x563aac30e9b0 2024-02-02T15:33:35.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.784+0000 7f23ad600700 1 -- 172.21.15.1:0/1376993040 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f23a8000e40 con 0x563aac30e9b0 2024-02-02T15:33:35.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.785+0000 7f23ad600700 1 -- 172.21.15.1:0/1376993040 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23a8004710 con 0x563aac30e9b0 2024-02-02T15:33:35.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.785+0000 7f23d9f4bb80 10 client.27434 fetch_fsmap learned FSMap version 38 2024-02-02T15:33:35.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.785+0000 7f23d9f4bb80 1 -- 172.21.15.1:0/1376993040 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x563aac225f20 con 0x563aac30e9b0 2024-02-02T15:33:35.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.785+0000 7f23ad600700 1 -- 172.21.15.1:0/1376993040 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23b009d230 con 0x563aac30e9b0 2024-02-02T15:33:35.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.785+0000 7f23ad600700 1 -- 172.21.15.1:0/1376993040 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 38) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f23a800f730 con 0x563aac30e9b0 2024-02-02T15:33:35.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.785+0000 7f23d9f4bb80 10 client.27434 fetch_fsmap finished waiting for FSMap version 38 2024-02-02T15:33:35.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.785+0000 7f23d9f4bb80 10 client.27434 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:33:35.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.785+0000 7f23d9f4bb80 20 client.27434 populate_metadata read hostname 'smithi001' 2024-02-02T15:33:35.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.785+0000 7f23d9f4bb80 1 --2- 172.21.15.1:0/1376993040 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x563aac37c940 0x563aac37ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:35.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.785+0000 7f23d9f4bb80 4 client.27434 mds_command: new command op to 24557 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-02T15:33:35.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.785+0000 7f23d9f4bb80 1 -- 172.21.15.1:0/1376993040 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x563aac21efd0 con 0x563aac37c940 2024-02-02T15:33:35.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.785+0000 7f23bf400700 1 --2- 172.21.15.1:0/1376993040 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x563aac37c940 0x563aac37ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:35.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.786+0000 7f23bf400700 1 --2- 172.21.15.1:0/1376993040 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x563aac37c940 0x563aac37ed60 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61e4a09fca49b943 server_cookie=786f3a2fbaef25a8 in_seq=0 out_seq=0 2024-02-02T15:33:35.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.786+0000 7f23ad600700 10 client.27434 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:33:35.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.787+0000 7f23ad600700 1 -- 172.21.15.1:0/1376993040 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x563aac21efd0 con 0x563aac37c940 2024-02-02T15:33:35.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.787+0000 7f23ad600700 10 client.27434 handle_command_reply: tid=0 2024-02-02T15:33:35.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.787+0000 7f23d9f4bb80 1 client.27434 shutdown 2024-02-02T15:33:35.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.787+0000 7f23d9f4bb80 1 -- 172.21.15.1:0/1376993040 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563aac30e9b0 msgr2=0x563aac31f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:35.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.787+0000 7f23d9f4bb80 1 --2- 172.21.15.1:0/1376993040 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563aac30e9b0 0x563aac31f100 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f23a8024780 tx=0x7f23a8024680 comp rx=0 tx=0).stop 2024-02-02T15:33:35.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.787+0000 7f23d9f4bb80 1 -- 172.21.15.1:0/1376993040 shutdown_connections 2024-02-02T15:33:35.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.787+0000 7f23d9f4bb80 1 --2- 172.21.15.1:0/1376993040 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x563aac37c940 0x563aac37ed60 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:35.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.788+0000 7f23d9f4bb80 1 --2- 172.21.15.1:0/1376993040 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563aac319860 0x563aac37af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:35.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.788+0000 7f23d9f4bb80 1 --2- 172.21.15.1:0/1376993040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563aac30e090 0x563aac31ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:35.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.788+0000 7f23ad600700 0 client.27434 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:33:35.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.788+0000 7f23d9f4bb80 1 --2- 172.21.15.1:0/1376993040 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563aac30e9b0 0x563aac31f100 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:35.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.788+0000 7f23d9f4bb80 1 -- 172.21.15.1:0/1376993040 >> 172.21.15.1:0/1376993040 conn(0x563aac26de30 msgr2=0x563aac311820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:35.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.788+0000 7f23d9f4bb80 1 -- 172.21.15.1:0/1376993040 shutdown_connections 2024-02-02T15:33:35.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.788+0000 7f23d9f4bb80 1 -- 172.21.15.1:0/1376993040 wait complete. 2024-02-02T15:33:35.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.788+0000 7f23d9f4bb80 20 client.27434 trim_cache size 0 max 16384 2024-02-02T15:33:35.791 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:33:35.791 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2024-02-02T15:33:35.791 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "892f5b50-0196-4e99-b0a0-22d72a9f3048", 2024-02-02T15:33:35.791 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2024-02-02T15:33:35.791 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:33:35.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.788+0000 7f23c6400700 1 -- 172.21.15.1:0/2785564680 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f23a00a0690 msgr2=0x7f23a00a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:35.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.788+0000 7f23c6400700 1 --2- 172.21.15.1:0/2785564680 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f23a00a0690 0x7f23a00a2b50 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f23b8003e80 tx=0x7f23b8009920 comp rx=0 tx=0).stop 2024-02-02T15:33:35.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.788+0000 7f23c6400700 1 -- 172.21.15.1:0/2785564680 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f23c0100ff0 msgr2=0x7f23c0199060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:35.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.788+0000 7f23c6400700 1 --2- 172.21.15.1:0/2785564680 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f23c0100ff0 0x7f23c0199060 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f23b000e9a0 tx=0x7f23b000ecb0 comp rx=0 tx=0).stop 2024-02-02T15:33:35.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.789+0000 7f23c6400700 1 -- 172.21.15.1:0/2785564680 shutdown_connections 2024-02-02T15:33:35.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.789+0000 7f23c6400700 1 --2- 172.21.15.1:0/2785564680 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f23a00a0690 0x7f23a00a2b50 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:35.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.789+0000 7f23c6400700 1 --2- 172.21.15.1:0/2785564680 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f23c019da00 0x7f23c019de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:35.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.789+0000 7f23c6400700 1 --2- 172.21.15.1:0/2785564680 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f23c0100ff0 0x7f23c0199060 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:35.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.789+0000 7f23c6400700 1 --2- 172.21.15.1:0/2785564680 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f23c0106110 0x7f23c01995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:35.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.789+0000 7f23c6400700 1 -- 172.21.15.1:0/2785564680 >> 172.21.15.1:0/2785564680 conn(0x7f23c00fc810 msgr2=0x7f23c00fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:35.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.789+0000 7f23c6400700 1 -- 172.21.15.1:0/2785564680 shutdown_connections 2024-02-02T15:33:35.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:35.789+0000 7f23c6400700 1 -- 172.21.15.1:0/2785564680 wait complete. 2024-02-02T15:33:35.811 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-02T15:33:36.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.029+0000 7f739cc00700 1 -- 172.21.15.1:0/66871564 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7398103230 msgr2=0x7f7398103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:36.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.029+0000 7f739cc00700 1 --2- 172.21.15.1:0/66871564 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7398103230 0x7f7398103610 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f7388009990 tx=0x7f7388009ca0 comp rx=0 tx=0).stop 2024-02-02T15:33:36.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.030+0000 7f739cc00700 1 -- 172.21.15.1:0/66871564 shutdown_connections 2024-02-02T15:33:36.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.030+0000 7f739cc00700 1 --2- 172.21.15.1:0/66871564 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7398108350 0x7f739810fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:36.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.030+0000 7f739cc00700 1 --2- 172.21.15.1:0/66871564 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7398103230 0x7f7398103610 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:36.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.030+0000 7f739cc00700 1 --2- 172.21.15.1:0/66871564 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7398103be0 0x7f7398107c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:36.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.030+0000 7f739cc00700 1 -- 172.21.15.1:0/66871564 >> 172.21.15.1:0/66871564 conn(0x7f73980fea70 msgr2=0x7f7398100e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:36.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.030+0000 7f739cc00700 1 -- 172.21.15.1:0/66871564 shutdown_connections 2024-02-02T15:33:36.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.030+0000 7f739cc00700 1 -- 172.21.15.1:0/66871564 wait complete. 2024-02-02T15:33:36.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.031+0000 7f739cc00700 1 Processor -- start 2024-02-02T15:33:36.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.031+0000 7f739cc00700 1 -- start start 2024-02-02T15:33:36.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.031+0000 7f739cc00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7398103230 0x7f7398199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:36.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.032+0000 7f739cc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7398108350 0x7f7398199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:36.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.032+0000 7f739cc00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f739819dab0 0x7f739819df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:36.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.032+0000 7f739cc00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f739819e530 con 0x7f739819dab0 2024-02-02T15:33:36.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.032+0000 7f739cc00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f739819e6a0 con 0x7f7398103230 2024-02-02T15:33:36.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.032+0000 7f739cc00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f739819e860 con 0x7f7398108350 2024-02-02T15:33:36.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.032+0000 7f7395e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7398103230 0x7f7398199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:36.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.032+0000 7f7395e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7398103230 0x7f7398199110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:50402/0 (socket says 172.21.15.1:50402) 2024-02-02T15:33:36.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.032+0000 7f7395e00700 1 -- 172.21.15.1:0/3757827708 learned_addr learned my addr 172.21.15.1:0/3757827708 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:33:36.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.032+0000 7f7396800700 1 --2- 172.21.15.1:0/3757827708 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f739819dab0 0x7f739819df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:36.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.032+0000 7f7395400700 1 --2- 172.21.15.1:0/3757827708 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7398108350 0x7f7398199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:36.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.032+0000 7f7395e00700 1 -- 172.21.15.1:0/3757827708 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7398108350 msgr2=0x7f7398199650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:36.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.032+0000 7f7395e00700 1 --2- 172.21.15.1:0/3757827708 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7398108350 0x7f7398199650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:36.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.032+0000 7f7395e00700 1 -- 172.21.15.1:0/3757827708 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f739819dab0 msgr2=0x7f739819df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:36.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.032+0000 7f7395e00700 1 --2- 172.21.15.1:0/3757827708 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f739819dab0 0x7f739819df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:36.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.032+0000 7f7395e00700 1 -- 172.21.15.1:0/3757827708 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7380009520 con 0x7f7398103230 2024-02-02T15:33:36.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.032+0000 7f7396800700 1 --2- 172.21.15.1:0/3757827708 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f739819dab0 0x7f739819df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:33:36.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.033+0000 7f7395e00700 1 --2- 172.21.15.1:0/3757827708 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7398103230 0x7f7398199110 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f7388005910 tx=0x7f7388005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:33:36.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.033+0000 7f738ea00700 1 -- 172.21.15.1:0/3757827708 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f738801d070 con 0x7f7398103230 2024-02-02T15:33:36.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.033+0000 7f738ea00700 1 -- 172.21.15.1:0/3757827708 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7388022470 con 0x7f7398103230 2024-02-02T15:33:36.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.033+0000 7f739cc00700 1 -- 172.21.15.1:0/3757827708 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73880095f0 con 0x7f7398103230 2024-02-02T15:33:36.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.033+0000 7f738ea00700 1 -- 172.21.15.1:0/3757827708 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7388017650 con 0x7f7398103230 2024-02-02T15:33:36.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.034+0000 7f739cc00700 1 -- 172.21.15.1:0/3757827708 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73981a5ff0 con 0x7f7398103230 2024-02-02T15:33:36.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.036+0000 7f738ea00700 1 -- 172.21.15.1:0/3757827708 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 27) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f73880225e0 con 0x7f7398103230 2024-02-02T15:33:36.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.037+0000 7f738ea00700 1 --2- 172.21.15.1:0/3757827708 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f73780a0710 0x7f73780a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:36.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.037+0000 7f738ea00700 1 -- 172.21.15.1:0/3757827708 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f73880cd490 con 0x7f7398103230 2024-02-02T15:33:36.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.038+0000 7f7395400700 1 --2- 172.21.15.1:0/3757827708 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f73780a0710 0x7f73780a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:36.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.038+0000 7f73b078bb80 1 Processor -- start 2024-02-02T15:33:36.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.039+0000 7f7395400700 1 --2- 172.21.15.1:0/3757827708 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f73780a0710 0x7f73780a2bd0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f739819a720 tx=0x7f7380009260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:33:36.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.039+0000 7f73b078bb80 1 -- start start 2024-02-02T15:33:36.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.039+0000 7f73b078bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557def1ed010 0x557def1ed3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:36.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.039+0000 7f73b078bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557def1ed930 0x557def200ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:36.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.039+0000 7f73b078bb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557def200ff0 0x557def2033e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:36.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.039+0000 7f73b078bb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x557def1614c0 con 0x557def200ff0 2024-02-02T15:33:36.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.039+0000 7f73b078bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x557def145aa0 con 0x557def1ed930 2024-02-02T15:33:36.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.039+0000 7f73b078bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557def143460 con 0x557def1ed010 2024-02-02T15:33:36.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.039+0000 7f7395400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557def1ed930 0x557def200ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:36.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.040+0000 7f7395e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557def1ed010 0x557def1ed3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:36.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.040+0000 7f7395400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557def1ed930 0x557def200ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:50410/0 (socket says 172.21.15.1:50410) 2024-02-02T15:33:36.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.040+0000 7f7395400700 1 -- 172.21.15.1:0/636416705 learned_addr learned my addr 172.21.15.1:0/636416705 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:33:36.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.040+0000 7f7396800700 1 --2- 172.21.15.1:0/636416705 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557def200ff0 0x557def2033e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:36.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.040+0000 7f7395400700 1 -- 172.21.15.1:0/636416705 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557def1ed010 msgr2=0x557def1ed3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:36.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.040+0000 7f7395400700 1 --2- 172.21.15.1:0/636416705 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557def1ed010 0x557def1ed3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:36.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.040+0000 7f7395400700 1 -- 172.21.15.1:0/636416705 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557def200ff0 msgr2=0x557def2033e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:36.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.040+0000 7f7395400700 1 --2- 172.21.15.1:0/636416705 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557def200ff0 0x557def2033e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:36.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.040+0000 7f7395400700 1 -- 172.21.15.1:0/636416705 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557deefc03b0 con 0x557def1ed930 2024-02-02T15:33:36.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.040+0000 7f7395400700 1 --2- 172.21.15.1:0/636416705 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557def1ed930 0x557def200ab0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f738000e640 tx=0x7f7380014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6af01c8ec806e2a0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:33:36.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.041+0000 7f7387000700 1 -- 172.21.15.1:0/636416705 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f738000eea0 con 0x557def1ed930 2024-02-02T15:33:36.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.041+0000 7f7387000700 1 -- 172.21.15.1:0/636416705 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7380021070 con 0x557def1ed930 2024-02-02T15:33:36.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.041+0000 7f7387000700 1 -- 172.21.15.1:0/636416705 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f738001d670 con 0x557def1ed930 2024-02-02T15:33:36.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.041+0000 7f73b078bb80 1 -- 172.21.15.1:0/636416705 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557def1ed930 msgr2=0x557def200ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:36.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.041+0000 7f73b078bb80 1 --2- 172.21.15.1:0/636416705 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557def1ed930 0x557def200ab0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f738000e640 tx=0x7f7380014000 comp rx=0 tx=0).stop 2024-02-02T15:33:36.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.042+0000 7f73b078bb80 1 -- 172.21.15.1:0/636416705 shutdown_connections 2024-02-02T15:33:36.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.042+0000 7f73b078bb80 1 --2- 172.21.15.1:0/636416705 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557def200ff0 0x557def2033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:36.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.042+0000 7f73b078bb80 1 --2- 172.21.15.1:0/636416705 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557def1ed010 0x557def1ed3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:36.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.042+0000 7f73b078bb80 1 --2- 172.21.15.1:0/636416705 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557def1ed930 0x557def200ab0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:36.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.042+0000 7f73b078bb80 1 -- 172.21.15.1:0/636416705 >> 172.21.15.1:0/636416705 conn(0x557def14cdb0 msgr2=0x557def1e93b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:36.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.042+0000 7f73b078bb80 1 -- 172.21.15.1:0/636416705 shutdown_connections 2024-02-02T15:33:36.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.042+0000 7f73b078bb80 1 -- 172.21.15.1:0/636416705 wait complete. 2024-02-02T15:33:36.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.042+0000 7f73b078bb80 1 -- start start 2024-02-02T15:33:36.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.043+0000 7f73b078bb80 1 Processor -- start 2024-02-02T15:33:36.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.043+0000 7f73b078bb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557def1ed010 0x557def2009f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:36.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.043+0000 7f73b078bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557def1ed930 0x557def1fdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:36.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.043+0000 7f73b078bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557def200ff0 0x557def1fe210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:36.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.043+0000 7f73b078bb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x557def13ed50 con 0x557def1ed010 2024-02-02T15:33:36.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.043+0000 7f73b078bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x557def0dad80 con 0x557def200ff0 2024-02-02T15:33:36.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.043+0000 7f73b078bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557def0daa20 con 0x557def1ed930 2024-02-02T15:33:36.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.043+0000 7f73b078bb80 10 client.0 _ll_register_callbacks cb 0x557def0dc290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:33:36.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.043+0000 7f7396800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557def200ff0 0x557def1fe210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:36.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.043+0000 7f7395400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557def1ed930 0x557def1fdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:36.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.043+0000 7f7395e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557def1ed010 0x557def2009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:36.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.043+0000 7f7396800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557def200ff0 0x557def1fe210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:50420/0 (socket says 172.21.15.1:50420) 2024-02-02T15:33:36.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.043+0000 7f7396800700 1 -- 172.21.15.1:0/1503820505 learned_addr learned my addr 172.21.15.1:0/1503820505 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:33:36.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.044+0000 7f7395400700 1 -- 172.21.15.1:0/1503820505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557def200ff0 msgr2=0x557def1fe210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:36.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.044+0000 7f7395400700 1 --2- 172.21.15.1:0/1503820505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557def200ff0 0x557def1fe210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:36.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.044+0000 7f7395400700 1 -- 172.21.15.1:0/1503820505 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557def1ed010 msgr2=0x557def2009f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:36.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.044+0000 7f7395400700 1 --2- 172.21.15.1:0/1503820505 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557def1ed010 0x557def2009f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:36.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.044+0000 7f7395400700 1 -- 172.21.15.1:0/1503820505 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7380017070 con 0x557def1ed930 2024-02-02T15:33:36.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.044+0000 7f7395400700 1 --2- 172.21.15.1:0/1503820505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557def1ed930 0x557def1fdc00 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f738000e3f0 tx=0x7f738000ef30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2ba0b8a3db2ae97 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:33:36.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.044+0000 7f737f400700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:33:36.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.044+0000 7f737f400700 1 -- 172.21.15.1:0/1503820505 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7380003880 con 0x557def1ed930 2024-02-02T15:33:36.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.044+0000 7f73b078bb80 1 -- 172.21.15.1:0/1503820505 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557def1e10f0 con 0x557def1ed930 2024-02-02T15:33:36.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.044+0000 7f737f400700 1 -- 172.21.15.1:0/1503820505 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f73800039e0 con 0x557def1ed930 2024-02-02T15:33:36.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.045+0000 7f737f400700 1 -- 172.21.15.1:0/1503820505 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7380003ba0 con 0x557def1ed930 2024-02-02T15:33:36.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.045+0000 7f73b078bb80 10 client.27442 fetch_fsmap learned FSMap version 38 2024-02-02T15:33:36.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.045+0000 7f73b078bb80 1 -- 172.21.15.1:0/1503820505 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x557deef6cb60 con 0x557def1ed930 2024-02-02T15:33:36.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.045+0000 7f737f400700 1 -- 172.21.15.1:0/1503820505 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557def1e10f0 con 0x557def1ed930 2024-02-02T15:33:36.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.045+0000 7f737f400700 1 -- 172.21.15.1:0/1503820505 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 38) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f7380026480 con 0x557def1ed930 2024-02-02T15:33:36.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.045+0000 7f73b078bb80 10 client.27442 fetch_fsmap finished waiting for FSMap version 38 2024-02-02T15:33:36.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.045+0000 7f73b078bb80 10 client.27442 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:33:36.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.045+0000 7f73b078bb80 20 client.27442 populate_metadata read hostname 'smithi001' 2024-02-02T15:33:36.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.045+0000 7f73b078bb80 1 --2- 172.21.15.1:0/1503820505 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x557def210980 0x557def212da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:36.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.045+0000 7f73b078bb80 4 client.27442 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:33:36.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.045+0000 7f73b078bb80 1 -- 172.21.15.1:0/1503820505 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557def137dd0 con 0x557def210980 2024-02-02T15:33:36.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.046+0000 7f7395e00700 1 --2- 172.21.15.1:0/1503820505 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x557def210980 0x557def212da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:36.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.046+0000 7f7395e00700 1 --2- 172.21.15.1:0/1503820505 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x557def210980 0x557def212da0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0d8114b2f512344 server_cookie=f67c001433e65265 in_seq=0 out_seq=0 2024-02-02T15:33:36.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.046+0000 7f737f400700 10 client.27442 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:33:36.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.047+0000 7f737f400700 1 -- 172.21.15.1:0/1503820505 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x557def137dd0 con 0x557def210980 2024-02-02T15:33:36.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.047+0000 7f737f400700 10 client.27442 handle_command_reply: tid=0 2024-02-02T15:33:36.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.047+0000 7f73b078bb80 1 client.27442 shutdown 2024-02-02T15:33:36.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.047+0000 7f73b078bb80 1 -- 172.21.15.1:0/1503820505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557def1ed930 msgr2=0x557def1fdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:36.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.047+0000 7f73b078bb80 1 --2- 172.21.15.1:0/1503820505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557def1ed930 0x557def1fdc00 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f738000e3f0 tx=0x7f738000ef30 comp rx=0 tx=0).stop 2024-02-02T15:33:36.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.048+0000 7f73b078bb80 1 -- 172.21.15.1:0/1503820505 shutdown_connections 2024-02-02T15:33:36.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.048+0000 7f73b078bb80 1 --2- 172.21.15.1:0/1503820505 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x557def210980 0x557def212da0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:36.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.048+0000 7f73b078bb80 1 --2- 172.21.15.1:0/1503820505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557def200ff0 0x557def1fe210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:36.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.048+0000 7f73b078bb80 1 --2- 172.21.15.1:0/1503820505 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557def1ed010 0x557def2009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:36.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.048+0000 7f73b078bb80 1 --2- 172.21.15.1:0/1503820505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557def1ed930 0x557def1fdc00 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:36.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.048+0000 7f737f400700 0 client.27442 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:33:36.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.048+0000 7f73b078bb80 1 -- 172.21.15.1:0/1503820505 >> 172.21.15.1:0/1503820505 conn(0x557def14cdb0 msgr2=0x557def1e8fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:36.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.048+0000 7f73b078bb80 1 -- 172.21.15.1:0/1503820505 shutdown_connections 2024-02-02T15:33:36.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.048+0000 7f73b078bb80 1 -- 172.21.15.1:0/1503820505 wait complete. 2024-02-02T15:33:36.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.048+0000 7f73b078bb80 20 client.27442 trim_cache size 0 max 16384 2024-02-02T15:33:36.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.067+0000 7f73b078bb80 1 Processor -- start 2024-02-02T15:33:36.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.067+0000 7f73b078bb80 1 -- start start 2024-02-02T15:33:36.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.067+0000 7f73b078bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557def1ed010 0x557def1fdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:36.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.067+0000 7f73b078bb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557def1ed930 0x557def1ef840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:36.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.067+0000 7f73b078bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557def200ff0 0x557def200520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:36.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.067+0000 7f73b078bb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x557deefc26a0 con 0x557def1ed930 2024-02-02T15:33:36.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.067+0000 7f73b078bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x557def1e0fc0 con 0x557def200ff0 2024-02-02T15:33:36.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.067+0000 7f73b078bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557def1f8a50 con 0x557def1ed010 2024-02-02T15:33:36.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.068+0000 7f7395e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557def1ed010 0x557def1fdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:36.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.068+0000 7f7395e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557def1ed010 0x557def1fdbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:56220/0 (socket says 172.21.15.1:56220) 2024-02-02T15:33:36.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.068+0000 7f7395e00700 1 -- 172.21.15.1:0/1916337169 learned_addr learned my addr 172.21.15.1:0/1916337169 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:33:36.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.068+0000 7f7396800700 1 --2- 172.21.15.1:0/1916337169 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557def200ff0 0x557def200520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:36.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.068+0000 7f7395400700 1 --2- 172.21.15.1:0/1916337169 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557def1ed930 0x557def1ef840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:36.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.068+0000 7f7395e00700 1 -- 172.21.15.1:0/1916337169 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557def200ff0 msgr2=0x557def200520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:36.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.068+0000 7f7395e00700 1 --2- 172.21.15.1:0/1916337169 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557def200ff0 0x557def200520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:36.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.068+0000 7f7395e00700 1 -- 172.21.15.1:0/1916337169 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557def1ed930 msgr2=0x557def1ef840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:36.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.068+0000 7f7395e00700 1 --2- 172.21.15.1:0/1916337169 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557def1ed930 0x557def1ef840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:36.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.068+0000 7f7395e00700 1 -- 172.21.15.1:0/1916337169 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557def104ea0 con 0x557def1ed010 2024-02-02T15:33:36.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.068+0000 7f7395e00700 1 --2- 172.21.15.1:0/1916337169 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557def1ed010 0x557def1fdbb0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7388006010 tx=0x7f7388022c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da1d87dfa91456f9 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:33:36.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.069+0000 7f7387a00700 1 -- 172.21.15.1:0/1916337169 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73880b6070 con 0x557def1ed010 2024-02-02T15:33:36.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.069+0000 7f7387a00700 1 -- 172.21.15.1:0/1916337169 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f738809adf0 con 0x557def1ed010 2024-02-02T15:33:36.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.069+0000 7f7387a00700 1 -- 172.21.15.1:0/1916337169 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7388098a90 con 0x557def1ed010 2024-02-02T15:33:36.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.069+0000 7f73b078bb80 1 -- 172.21.15.1:0/1916337169 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557def1ed010 msgr2=0x557def1fdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:36.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.069+0000 7f73b078bb80 1 --2- 172.21.15.1:0/1916337169 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557def1ed010 0x557def1fdbb0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7388006010 tx=0x7f7388022c90 comp rx=0 tx=0).stop 2024-02-02T15:33:36.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.069+0000 7f73b078bb80 1 -- 172.21.15.1:0/1916337169 shutdown_connections 2024-02-02T15:33:36.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.069+0000 7f73b078bb80 1 --2- 172.21.15.1:0/1916337169 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557def200ff0 0x557def200520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:36.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.069+0000 7f73b078bb80 1 --2- 172.21.15.1:0/1916337169 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557def1ed010 0x557def1fdbb0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:36.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.069+0000 7f73b078bb80 1 --2- 172.21.15.1:0/1916337169 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557def1ed930 0x557def1ef840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:36.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.069+0000 7f73b078bb80 1 -- 172.21.15.1:0/1916337169 >> 172.21.15.1:0/1916337169 conn(0x557def14cdb0 msgr2=0x557def1fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:36.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.069+0000 7f73b078bb80 1 -- 172.21.15.1:0/1916337169 shutdown_connections 2024-02-02T15:33:36.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.069+0000 7f73b078bb80 1 -- 172.21.15.1:0/1916337169 wait complete. 2024-02-02T15:33:36.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.070+0000 7f73b078bb80 1 -- start start 2024-02-02T15:33:36.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.070+0000 7f73b078bb80 1 Processor -- start 2024-02-02T15:33:36.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.071+0000 7f73b078bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557def1ed010 0x557def2057e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:36.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.071+0000 7f73b078bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557def1ed930 0x557def205d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:36.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.071+0000 7f73b078bb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557def200ff0 0x557def25b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:36.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.071+0000 7f73b078bb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x557def25b970 con 0x557def200ff0 2024-02-02T15:33:36.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.071+0000 7f73b078bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x557def25bae0 con 0x557def1ed930 2024-02-02T15:33:36.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.071+0000 7f73b078bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557def25bca0 con 0x557def1ed010 2024-02-02T15:33:36.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.071+0000 7f73b078bb80 10 client.0 _ll_register_callbacks cb 0x557def0dc290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:33:36.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.071+0000 7f7395e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557def1ed010 0x557def2057e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:36.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.071+0000 7f7395400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557def1ed930 0x557def205d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:36.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.071+0000 7f7395e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557def1ed010 0x557def2057e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:56226/0 (socket says 172.21.15.1:56226) 2024-02-02T15:33:36.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.071+0000 7f7395e00700 1 -- 172.21.15.1:0/3757552809 learned_addr learned my addr 172.21.15.1:0/3757552809 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:33:36.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.071+0000 7f7396800700 1 --2- 172.21.15.1:0/3757552809 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557def200ff0 0x557def25b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:36.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.071+0000 7f7395400700 1 -- 172.21.15.1:0/3757552809 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557def1ed010 msgr2=0x557def2057e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:36.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.071+0000 7f7395400700 1 --2- 172.21.15.1:0/3757552809 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557def1ed010 0x557def2057e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:36.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.071+0000 7f7395400700 1 -- 172.21.15.1:0/3757552809 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557def200ff0 msgr2=0x557def25b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:33:36.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.071+0000 7f7395400700 1 --2- 172.21.15.1:0/3757552809 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557def200ff0 0x557def25b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:36.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.071+0000 7f7395400700 1 -- 172.21.15.1:0/3757552809 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73981a5ff0 con 0x557def1ed930 2024-02-02T15:33:36.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.071+0000 7f7395e00700 1 --2- 172.21.15.1:0/3757552809 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557def1ed010 0x557def2057e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:33:36.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.072+0000 7f7395400700 1 --2- 172.21.15.1:0/3757552809 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557def1ed930 0x557def205d20 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f7380026890 tx=0x7f73800266b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=870a9fb878cfeae5 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:33:36.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.072+0000 7f737f400700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:33:36.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.072+0000 7f737f400700 1 -- 172.21.15.1:0/3757552809 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7380003a90 con 0x557def1ed930 2024-02-02T15:33:36.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.072+0000 7f737f400700 1 -- 172.21.15.1:0/3757552809 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 9365+0+0 (secure 0 0 0) 0x7f7380026b00 con 0x557def1ed930 2024-02-02T15:33:36.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.072+0000 7f73b078bb80 1 -- 172.21.15.1:0/3757552809 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7380021070 con 0x557def1ed930 2024-02-02T15:33:36.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.072+0000 7f737f400700 1 -- 172.21.15.1:0/3757552809 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f738000ec30 con 0x557def1ed930 2024-02-02T15:33:36.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.072+0000 7f73b078bb80 10 client.27452 fetch_fsmap learned FSMap version 38 2024-02-02T15:33:36.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.072+0000 7f73b078bb80 1 -- 172.21.15.1:0/3757552809 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x557deefc4cd0 con 0x557def1ed930 2024-02-02T15:33:36.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.073+0000 7f737f400700 1 -- 172.21.15.1:0/3757552809 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7380021070 con 0x557def1ed930 2024-02-02T15:33:36.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.073+0000 7f737f400700 1 -- 172.21.15.1:0/3757552809 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 38) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f738001db60 con 0x557def1ed930 2024-02-02T15:33:36.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.073+0000 7f73b078bb80 10 client.27452 fetch_fsmap finished waiting for FSMap version 38 2024-02-02T15:33:36.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.073+0000 7f73b078bb80 10 client.27452 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:33:36.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.073+0000 7f73b078bb80 20 client.27452 populate_metadata read hostname 'smithi001' 2024-02-02T15:33:36.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.073+0000 7f73b078bb80 1 --2- 172.21.15.1:0/3757552809 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x557def2630f0 0x557def265510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:33:36.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.073+0000 7f73b078bb80 4 client.27452 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2024-02-02T15:33:36.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.073+0000 7f73b078bb80 1 -- 172.21.15.1:0/3757552809 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557def0fdf50 con 0x557def2630f0 2024-02-02T15:33:36.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.073+0000 7f7395e00700 1 --2- 172.21.15.1:0/3757552809 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x557def2630f0 0x557def265510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:33:36.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.073+0000 7f7395e00700 1 --2- 172.21.15.1:0/3757552809 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x557def2630f0 0x557def265510 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7eb50cc9461d10d server_cookie=ff564012f1bc168a in_seq=0 out_seq=0 2024-02-02T15:33:36.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.073+0000 7f737f400700 10 client.27452 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:33:36.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.074+0000 7f737f400700 1 -- 172.21.15.1:0/3757552809 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x557def0fdf50 con 0x557def2630f0 2024-02-02T15:33:36.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.074+0000 7f737f400700 10 client.27452 handle_command_reply: tid=0 2024-02-02T15:33:36.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.074+0000 7f73b078bb80 1 client.27452 shutdown 2024-02-02T15:33:36.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.074+0000 7f73b078bb80 1 -- 172.21.15.1:0/3757552809 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557def1ed930 msgr2=0x557def205d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:36.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.074+0000 7f73b078bb80 1 --2- 172.21.15.1:0/3757552809 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557def1ed930 0x557def205d20 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f7380026890 tx=0x7f73800266b0 comp rx=0 tx=0).stop 2024-02-02T15:33:36.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.074+0000 7f73b078bb80 1 -- 172.21.15.1:0/3757552809 shutdown_connections 2024-02-02T15:33:36.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.074+0000 7f73b078bb80 1 --2- 172.21.15.1:0/3757552809 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x557def2630f0 0x557def265510 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:36.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.074+0000 7f73b078bb80 1 --2- 172.21.15.1:0/3757552809 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557def200ff0 0x557def25b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:36.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.074+0000 7f737f400700 0 client.27452 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:33:36.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.074+0000 7f73b078bb80 1 --2- 172.21.15.1:0/3757552809 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557def1ed010 0x557def2057e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:36.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.074+0000 7f73b078bb80 1 --2- 172.21.15.1:0/3757552809 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557def1ed930 0x557def205d20 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:36.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.074+0000 7f73b078bb80 1 -- 172.21.15.1:0/3757552809 >> 172.21.15.1:0/3757552809 conn(0x557def14cdb0 msgr2=0x557def1fc160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:36.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.074+0000 7f73b078bb80 1 -- 172.21.15.1:0/3757552809 shutdown_connections 2024-02-02T15:33:36.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.074+0000 7f73b078bb80 1 -- 172.21.15.1:0/3757552809 wait complete. 2024-02-02T15:33:36.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.075+0000 7f73b078bb80 20 client.27452 trim_cache size 0 max 16384 2024-02-02T15:33:36.078 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:33:36.078 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-02T15:33:36.078 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2024-02-02T15:33:36.078 INFO:teuthology.orchestra.run.smithi001.stdout: "892f5b50-0196-4e99-b0a0-22d72a9f3048": { 2024-02-02T15:33:36.079 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2024-02-02T15:33:36.079 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "892f5b50-0196-4e99-b0a0-22d72a9f3048", 2024-02-02T15:33:36.079 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,force" 2024-02-02T15:33:36.079 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:33:36.079 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:33:36.079 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:33:36.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.075+0000 7f739cc00700 1 -- 172.21.15.1:0/3757827708 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f73780a0710 msgr2=0x7f73780a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:36.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.075+0000 7f739cc00700 1 --2- 172.21.15.1:0/3757827708 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f73780a0710 0x7f73780a2bd0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f739819a720 tx=0x7f7380009260 comp rx=0 tx=0).stop 2024-02-02T15:33:36.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.075+0000 7f739cc00700 1 -- 172.21.15.1:0/3757827708 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7398103230 msgr2=0x7f7398199110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:33:36.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.075+0000 7f739cc00700 1 --2- 172.21.15.1:0/3757827708 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7398103230 0x7f7398199110 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f7388005910 tx=0x7f7388005740 comp rx=0 tx=0).stop 2024-02-02T15:33:36.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.075+0000 7f739cc00700 1 -- 172.21.15.1:0/3757827708 shutdown_connections 2024-02-02T15:33:36.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.075+0000 7f739cc00700 1 --2- 172.21.15.1:0/3757827708 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f73780a0710 0x7f73780a2bd0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:36.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.075+0000 7f739cc00700 1 --2- 172.21.15.1:0/3757827708 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f739819dab0 0x7f739819df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:36.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.075+0000 7f739cc00700 1 --2- 172.21.15.1:0/3757827708 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7398103230 0x7f7398199110 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:36.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.075+0000 7f739cc00700 1 --2- 172.21.15.1:0/3757827708 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7398108350 0x7f7398199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:33:36.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.075+0000 7f739cc00700 1 -- 172.21.15.1:0/3757827708 >> 172.21.15.1:0/3757827708 conn(0x7f73980fea70 msgr2=0x7f7398106c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:33:36.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.075+0000 7f739cc00700 1 -- 172.21.15.1:0/3757827708 shutdown_connections 2024-02-02T15:33:36.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:33:36.076+0000 7f739cc00700 1 -- 172.21.15.1:0/3757827708 wait complete. 2024-02-02T15:33:36.094 INFO:tasks.cephfs.filesystem:scrub status for tag:892f5b50-0196-4e99-b0a0-22d72a9f3048 - {'path': '/', 'tag': '892f5b50-0196-4e99-b0a0-22d72a9f3048', 'options': 'recursive,force'} 2024-02-02T15:33:36.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:36 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:33:36.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:36 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:33:36.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:36 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:33:36.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:36 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:33:36.240 INFO:tasks.workunit.client.0.smithi001.stdout: 1 426841 8.12 MB/sec execute 190 sec latency 27.373 ms 2024-02-02T15:33:36.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:36 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:33:36.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:36 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:33:36.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:36 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:33:36.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:36 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:33:36.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:36 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:33:36.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:36 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:33:36.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:36 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:33:36.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:36 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:33:37.240 INFO:tasks.workunit.client.0.smithi001.stdout: 1 427365 8.10 MB/sec execute 191 sec latency 397.237 ms 2024-02-02T15:33:37.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:37 smithi071 ceph-mon[44707]: scrub queued for path: / 2024-02-02T15:33:37.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:37 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:33:37.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:37 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:33:37.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:37 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:33:37.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:37 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:33:37.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:37 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:33:37.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:37 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:33:37.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:37 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:33:37.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:37 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:33:37.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:37 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:33:37.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:37 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:33:37.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:37 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:33:37.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:37 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:33:37.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:37 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:33:37.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:37 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:33:37.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:37 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:33:37.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:37 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:33:37.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:37 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:33:37.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:37 smithi071 ceph-mon[44707]: pgmap v201: 129 pgs: 129 active+clean; 272 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 215 op/s 2024-02-02T15:33:37.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:37 smithi142 ceph-mon[45728]: scrub queued for path: / 2024-02-02T15:33:37.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:37 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:33:37.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:37 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:33:37.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:37 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:33:37.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:37 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:33:37.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:37 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:33:37.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:37 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:33:37.421 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:37 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:33:37.421 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:37 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:33:37.421 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:37 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:33:37.421 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:37 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:33:37.421 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:37 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:33:37.421 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:37 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:33:37.421 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:37 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:33:37.421 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:37 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:33:37.421 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:37 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:33:37.421 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:37 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:33:37.421 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:37 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:33:37.421 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:37 smithi142 ceph-mon[45728]: pgmap v201: 129 pgs: 129 active+clean; 272 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 215 op/s 2024-02-02T15:33:37.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:37 smithi001 ceph-mon[35470]: scrub queued for path: / 2024-02-02T15:33:37.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:37 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:33:37.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:37 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:33:37.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:37 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:33:37.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:37 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:33:37.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:37 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:33:37.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:37 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:33:37.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:37 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:33:37.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:37 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:33:37.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:37 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:33:37.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:37 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:33:37.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:37 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:33:37.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:37 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:33:37.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:37 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:33:37.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:37 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:33:37.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:37 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:33:37.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:37 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:33:37.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:37 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:33:37.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:37 smithi001 ceph-mon[35470]: pgmap v201: 129 pgs: 129 active+clean; 272 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.4 MiB/s wr, 215 op/s 2024-02-02T15:33:38.240 INFO:tasks.workunit.client.0.smithi001.stdout: 1 427365 8.06 MB/sec execute 192 sec latency 1397.359 ms 2024-02-02T15:33:39.240 INFO:tasks.workunit.client.0.smithi001.stdout: 1 427365 8.02 MB/sec execute 193 sec latency 2397.456 ms 2024-02-02T15:33:39.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:39 smithi071 ceph-mon[44707]: pgmap v202: 129 pgs: 129 active+clean; 272 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 137 op/s 2024-02-02T15:33:39.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:39 smithi142 ceph-mon[45728]: pgmap v202: 129 pgs: 129 active+clean; 272 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 137 op/s 2024-02-02T15:33:39.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:39 smithi001 ceph-mon[35470]: pgmap v202: 129 pgs: 129 active+clean; 272 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 137 op/s 2024-02-02T15:33:40.240 INFO:tasks.workunit.client.0.smithi001.stdout: 1 428684 8.01 MB/sec execute 194 sec latency 2707.828 ms 2024-02-02T15:33:41.240 INFO:tasks.workunit.client.0.smithi001.stdout: 1 429825 8.00 MB/sec execute 195 sec latency 16.134 ms 2024-02-02T15:33:41.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:41 smithi071 ceph-mon[44707]: pgmap v203: 129 pgs: 129 active+clean; 272 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 137 op/s 2024-02-02T15:33:41.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:41 smithi142 ceph-mon[45728]: pgmap v203: 129 pgs: 129 active+clean; 272 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 137 op/s 2024-02-02T15:33:41.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:41 smithi001 ceph-mon[35470]: pgmap v203: 129 pgs: 129 active+clean; 272 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 137 op/s 2024-02-02T15:33:42.240 INFO:tasks.workunit.client.0.smithi001.stdout: 1 430914 7.99 MB/sec execute 196 sec latency 267.373 ms 2024-02-02T15:33:42.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:42 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:33:42.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:42 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:33:42.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:42 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:33:42.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:42 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:33:42.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:42 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:33:42.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:42 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:33:42.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:42 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:33:42.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:42 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:33:42.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:42 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:33:42.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:42 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:33:42.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:42 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:33:42.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:42 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:33:42.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:42 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:33:42.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:42 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:33:42.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:42 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:33:43.241 INFO:tasks.workunit.client.0.smithi001.stdout: 1 430914 7.95 MB/sec execute 197 sec latency 1267.509 ms 2024-02-02T15:33:43.615 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:43 smithi071 ceph-mon[44707]: pgmap v204: 129 pgs: 129 active+clean; 274 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 189 op/s 2024-02-02T15:33:43.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:43 smithi142 ceph-mon[45728]: pgmap v204: 129 pgs: 129 active+clean; 274 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 189 op/s 2024-02-02T15:33:43.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:43 smithi001 ceph-mon[35470]: pgmap v204: 129 pgs: 129 active+clean; 274 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 189 op/s 2024-02-02T15:33:44.241 INFO:tasks.workunit.client.0.smithi001.stdout: 1 431257 7.91 MB/sec execute 198 sec latency 2126.033 ms 2024-02-02T15:33:45.241 INFO:tasks.workunit.client.0.smithi001.stdout: 1 433051 7.94 MB/sec execute 199 sec latency 16.217 ms 2024-02-02T15:33:45.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:45 smithi071 ceph-mon[44707]: pgmap v205: 129 pgs: 129 active+clean; 274 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 121 op/s 2024-02-02T15:33:45.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:45 smithi142 ceph-mon[45728]: pgmap v205: 129 pgs: 129 active+clean; 274 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 121 op/s 2024-02-02T15:33:45.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:45 smithi001 ceph-mon[35470]: pgmap v205: 129 pgs: 129 active+clean; 274 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 121 op/s 2024-02-02T15:33:46.241 INFO:tasks.workunit.client.0.smithi001.stdout: 1 434200 7.92 MB/sec execute 200 sec latency 48.100 ms 2024-02-02T15:33:47.241 INFO:tasks.workunit.client.0.smithi001.stdout: 1 434458 7.89 MB/sec execute 201 sec latency 797.174 ms 2024-02-02T15:33:47.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:47 smithi071 ceph-mon[44707]: pgmap v206: 129 pgs: 129 active+clean; 278 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 187 op/s 2024-02-02T15:33:47.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:47 smithi142 ceph-mon[45728]: pgmap v206: 129 pgs: 129 active+clean; 278 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 187 op/s 2024-02-02T15:33:47.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:47 smithi001 ceph-mon[35470]: pgmap v206: 129 pgs: 129 active+clean; 278 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 187 op/s 2024-02-02T15:33:48.241 INFO:tasks.workunit.client.0.smithi001.stdout: 1 434458 7.85 MB/sec execute 202 sec latency 1797.331 ms 2024-02-02T15:33:49.242 INFO:tasks.workunit.client.0.smithi001.stdout: 1 434458 7.81 MB/sec execute 203 sec latency 2797.393 ms 2024-02-02T15:33:49.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:49 smithi071 ceph-mon[44707]: pgmap v207: 129 pgs: 129 active+clean; 278 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 117 op/s 2024-02-02T15:33:49.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:49 smithi142 ceph-mon[45728]: pgmap v207: 129 pgs: 129 active+clean; 278 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 117 op/s 2024-02-02T15:33:49.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:49 smithi001 ceph-mon[35470]: pgmap v207: 129 pgs: 129 active+clean; 278 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 117 op/s 2024-02-02T15:33:50.242 INFO:tasks.workunit.client.0.smithi001.stdout: 1 435931 7.81 MB/sec execute 204 sec latency 3096.862 ms 2024-02-02T15:33:51.241 INFO:tasks.workunit.client.0.smithi001.stdout: 1 437023 7.80 MB/sec execute 205 sec latency 24.846 ms 2024-02-02T15:33:51.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:51 smithi142 ceph-mon[45728]: pgmap v208: 129 pgs: 129 active+clean; 278 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 117 op/s 2024-02-02T15:33:51.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:51 smithi142 ceph-mon[45728]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:33:51.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:51 smithi001 ceph-mon[35470]: pgmap v208: 129 pgs: 129 active+clean; 278 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 117 op/s 2024-02-02T15:33:51.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:51 smithi001 ceph-mon[35470]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:33:51.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:51 smithi071 ceph-mon[44707]: pgmap v208: 129 pgs: 129 active+clean; 278 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 117 op/s 2024-02-02T15:33:51.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:51 smithi071 ceph-mon[44707]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:33:52.242 INFO:tasks.workunit.client.0.smithi001.stdout: 1 438005 7.79 MB/sec execute 206 sec latency 306.979 ms 2024-02-02T15:33:53.242 INFO:tasks.workunit.client.0.smithi001.stdout: 1 438005 7.75 MB/sec execute 207 sec latency 1307.067 ms 2024-02-02T15:33:53.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:53 smithi142 ceph-mon[45728]: pgmap v209: 129 pgs: 129 active+clean; 279 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 173 op/s 2024-02-02T15:33:53.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:53 smithi001 ceph-mon[35470]: pgmap v209: 129 pgs: 129 active+clean; 279 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 173 op/s 2024-02-02T15:33:53.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:53 smithi071 ceph-mon[44707]: pgmap v209: 129 pgs: 129 active+clean; 279 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 173 op/s 2024-02-02T15:33:54.242 INFO:tasks.workunit.client.0.smithi001.stdout: 1 438005 7.71 MB/sec execute 208 sec latency 2307.205 ms 2024-02-02T15:33:55.242 INFO:tasks.workunit.client.0.smithi001.stdout: 1 439500 7.71 MB/sec execute 209 sec latency 2610.455 ms 2024-02-02T15:33:55.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:55 smithi142 ceph-mon[45728]: pgmap v210: 129 pgs: 129 active+clean; 279 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 121 op/s 2024-02-02T15:33:55.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:55 smithi001 ceph-mon[35470]: pgmap v210: 129 pgs: 129 active+clean; 279 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 121 op/s 2024-02-02T15:33:55.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:55 smithi071 ceph-mon[44707]: pgmap v210: 129 pgs: 129 active+clean; 279 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 121 op/s 2024-02-02T15:33:56.242 INFO:tasks.workunit.client.0.smithi001.stdout: 1 440593 7.71 MB/sec execute 210 sec latency 18.676 ms 2024-02-02T15:33:56.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:56 smithi142 ceph-mon[45728]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:33:56.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:56 smithi001 ceph-mon[35470]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:33:56.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:56 smithi071 ceph-mon[44707]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:33:57.242 INFO:tasks.workunit.client.0.smithi001.stdout: 1 441547 7.69 MB/sec execute 211 sec latency 369.771 ms 2024-02-02T15:33:57.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:57 smithi142 ceph-mon[45728]: pgmap v211: 129 pgs: 129 active+clean; 282 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 182 op/s 2024-02-02T15:33:57.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:57 smithi001 ceph-mon[35470]: pgmap v211: 129 pgs: 129 active+clean; 282 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 182 op/s 2024-02-02T15:33:57.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:57 smithi071 ceph-mon[44707]: pgmap v211: 129 pgs: 129 active+clean; 282 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 182 op/s 2024-02-02T15:33:58.242 INFO:tasks.workunit.client.0.smithi001.stdout: 1 441547 7.66 MB/sec execute 212 sec latency 1369.854 ms 2024-02-02T15:33:59.242 INFO:tasks.workunit.client.0.smithi001.stdout: 1 441547 7.62 MB/sec execute 213 sec latency 2369.966 ms 2024-02-02T15:33:59.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:33:59 smithi142 ceph-mon[45728]: pgmap v212: 129 pgs: 129 active+clean; 282 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 116 op/s 2024-02-02T15:33:59.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:33:59 smithi001 ceph-mon[35470]: pgmap v212: 129 pgs: 129 active+clean; 282 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 116 op/s 2024-02-02T15:33:59.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:33:59 smithi071 ceph-mon[44707]: pgmap v212: 129 pgs: 129 active+clean; 282 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 116 op/s 2024-02-02T15:34:00.242 INFO:tasks.workunit.client.0.smithi001.stdout: 1 443118 7.63 MB/sec execute 214 sec latency 2666.615 ms 2024-02-02T15:34:01.190 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:34:00 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:34:00.880+0000 7f1978400700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-02T15:34:01.191 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:34:00 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:34:00.880+0000 7f1978400700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-02T15:34:01.191 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:34:00 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:34:00.880+0000 7f1978400700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-02T15:34:01.191 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:34:00 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:34:00.880+0000 7f1978400700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-02T15:34:01.191 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:34:00 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:34:00.880+0000 7f1978400700 -1 client.0 error registering admin socket command: (17) File exists 2024-02-02T15:34:01.243 INFO:tasks.workunit.client.0.smithi001.stdout: 1 444207 7.62 MB/sec execute 215 sec latency 17.007 ms 2024-02-02T15:34:01.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:01 smithi142 ceph-mon[45728]: pgmap v213: 129 pgs: 129 active+clean; 282 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 116 op/s 2024-02-02T15:34:01.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:01 smithi001 ceph-mon[35470]: pgmap v213: 129 pgs: 129 active+clean; 282 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 116 op/s 2024-02-02T15:34:01.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:01 smithi071 ceph-mon[44707]: pgmap v213: 129 pgs: 129 active+clean; 282 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 116 op/s 2024-02-02T15:34:02.243 INFO:tasks.workunit.client.0.smithi001.stdout: 1 444789 7.60 MB/sec execute 216 sec latency 60.456 ms 2024-02-02T15:34:03.243 INFO:tasks.workunit.client.0.smithi001.stdout: 1 445096 7.57 MB/sec execute 217 sec latency 696.516 ms 2024-02-02T15:34:03.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:03 smithi142 ceph-mon[45728]: pgmap v214: 129 pgs: 129 active+clean; 285 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 191 op/s 2024-02-02T15:34:03.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:03 smithi142 ceph-mon[45728]: mgrmap e28: x(active, since 7m), standbys: y, z 2024-02-02T15:34:03.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:03 smithi142 ceph-mon[45728]: osdmap e90: 12 total, 12 up, 12 in 2024-02-02T15:34:03.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:03 smithi001 ceph-mon[35470]: pgmap v214: 129 pgs: 129 active+clean; 285 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 191 op/s 2024-02-02T15:34:03.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:03 smithi001 ceph-mon[35470]: mgrmap e28: x(active, since 7m), standbys: y, z 2024-02-02T15:34:03.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:03 smithi001 ceph-mon[35470]: osdmap e90: 12 total, 12 up, 12 in 2024-02-02T15:34:03.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:03 smithi071 ceph-mon[44707]: pgmap v214: 129 pgs: 129 active+clean; 285 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 191 op/s 2024-02-02T15:34:03.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:03 smithi071 ceph-mon[44707]: mgrmap e28: x(active, since 7m), standbys: y, z 2024-02-02T15:34:03.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:03 smithi071 ceph-mon[44707]: osdmap e90: 12 total, 12 up, 12 in 2024-02-02T15:34:04.243 INFO:tasks.workunit.client.0.smithi001.stdout: 1 445449 7.54 MB/sec execute 218 sec latency 1536.841 ms 2024-02-02T15:34:05.243 INFO:tasks.workunit.client.0.smithi001.stdout: 1 447187 7.56 MB/sec execute 219 sec latency 23.701 ms 2024-02-02T15:34:05.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:05 smithi142 ceph-mon[45728]: pgmap v216: 129 pgs: 129 active+clean; 285 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 162 op/s 2024-02-02T15:34:05.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:05 smithi001 ceph-mon[35470]: pgmap v216: 129 pgs: 129 active+clean; 285 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 162 op/s 2024-02-02T15:34:05.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:05 smithi071 ceph-mon[44707]: pgmap v216: 129 pgs: 129 active+clean; 285 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 162 op/s 2024-02-02T15:34:06.095 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-02T15:34:06.243 INFO:tasks.workunit.client.0.smithi001.stdout: 1 448178 7.54 MB/sec execute 220 sec latency 18.239 ms 2024-02-02T15:34:06.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.313+0000 7feea0c00700 1 -- 172.21.15.1:0/2710404463 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fee9c1019a0 msgr2=0x7fee9c1059e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:06.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.313+0000 7feea0c00700 1 --2- 172.21.15.1:0/2710404463 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fee9c1019a0 0x7fee9c1059e0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fee84009990 tx=0x7fee84009ca0 comp rx=0 tx=0).stop 2024-02-02T15:34:06.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.313+0000 7feea0c00700 1 -- 172.21.15.1:0/2710404463 shutdown_connections 2024-02-02T15:34:06.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.313+0000 7feea0c00700 1 --2- 172.21.15.1:0/2710404463 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee9c106110 0x7fee9c10dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:06.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.313+0000 7feea0c00700 1 --2- 172.21.15.1:0/2710404463 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee9c100ff0 0x7fee9c1013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:06.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.313+0000 7feea0c00700 1 --2- 172.21.15.1:0/2710404463 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fee9c1019a0 0x7fee9c1059e0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:06.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.313+0000 7feea0c00700 1 -- 172.21.15.1:0/2710404463 >> 172.21.15.1:0/2710404463 conn(0x7fee9c0fc810 msgr2=0x7fee9c0fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:06.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.313+0000 7feea0c00700 1 -- 172.21.15.1:0/2710404463 shutdown_connections 2024-02-02T15:34:06.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.313+0000 7feea0c00700 1 -- 172.21.15.1:0/2710404463 wait complete. 2024-02-02T15:34:06.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.314+0000 7feea0c00700 1 Processor -- start 2024-02-02T15:34:06.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.314+0000 7feea0c00700 1 -- start start 2024-02-02T15:34:06.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.315+0000 7feea0c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fee9c100ff0 0x7fee9c1990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:06.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.315+0000 7feea0c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee9c106110 0x7fee9c199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:06.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.315+0000 7feea0c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee9c19da80 0x7fee9c19df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:06.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.315+0000 7feea0c00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee9c19e500 con 0x7fee9c100ff0 2024-02-02T15:34:06.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.315+0000 7feea0c00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee9c19e670 con 0x7fee9c106110 2024-02-02T15:34:06.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.315+0000 7feea0c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee9c19e830 con 0x7fee9c19da80 2024-02-02T15:34:06.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.315+0000 7fee99400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee9c106110 0x7fee9c199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:06.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.315+0000 7fee99e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fee9c100ff0 0x7fee9c1990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:06.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.315+0000 7fee99400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee9c106110 0x7fee9c199620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:33680/0 (socket says 172.21.15.1:33680) 2024-02-02T15:34:06.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.315+0000 7fee99400700 1 -- 172.21.15.1:0/2073387965 learned_addr learned my addr 172.21.15.1:0/2073387965 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:34:06.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.315+0000 7fee9a800700 1 --2- 172.21.15.1:0/2073387965 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee9c19da80 0x7fee9c19df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:06.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.315+0000 7fee99400700 1 -- 172.21.15.1:0/2073387965 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee9c19da80 msgr2=0x7fee9c19df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:06.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.315+0000 7fee99400700 1 --2- 172.21.15.1:0/2073387965 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee9c19da80 0x7fee9c19df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:06.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.315+0000 7fee99400700 1 -- 172.21.15.1:0/2073387965 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fee9c100ff0 msgr2=0x7fee9c1990e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:06.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.315+0000 7fee99400700 1 --2- 172.21.15.1:0/2073387965 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fee9c100ff0 0x7fee9c1990e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:06.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.315+0000 7fee99400700 1 -- 172.21.15.1:0/2073387965 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee840095f0 con 0x7fee9c106110 2024-02-02T15:34:06.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.315+0000 7fee9a800700 1 --2- 172.21.15.1:0/2073387965 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee9c19da80 0x7fee9c19df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:34:06.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.316+0000 7fee99400700 1 --2- 172.21.15.1:0/2073387965 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee9c106110 0x7fee9c199620 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fee84004760 tx=0x7fee84004790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:06.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.316+0000 7fee92a00700 1 -- 172.21.15.1:0/2073387965 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee8401d070 con 0x7fee9c106110 2024-02-02T15:34:06.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.316+0000 7fee92a00700 1 -- 172.21.15.1:0/2073387965 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fee84004830 con 0x7fee9c106110 2024-02-02T15:34:06.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.316+0000 7feea0c00700 1 -- 172.21.15.1:0/2073387965 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee9c1a5d30 con 0x7fee9c106110 2024-02-02T15:34:06.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.316+0000 7fee92a00700 1 -- 172.21.15.1:0/2073387965 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee84005310 con 0x7fee9c106110 2024-02-02T15:34:06.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.316+0000 7feea0c00700 1 -- 172.21.15.1:0/2073387965 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee9c1a60f0 con 0x7fee9c106110 2024-02-02T15:34:06.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.320+0000 7fee92a00700 1 -- 172.21.15.1:0/2073387965 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7fee840054b0 con 0x7fee9c106110 2024-02-02T15:34:06.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.322+0000 7feeb4636b80 1 Processor -- start 2024-02-02T15:34:06.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.322+0000 7fee92a00700 1 --2- 172.21.15.1:0/2073387965 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fee7c0a0690 0x7fee7c0a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:06.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.322+0000 7feeb4636b80 1 -- start start 2024-02-02T15:34:06.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.322+0000 7fee92a00700 1 -- 172.21.15.1:0/2073387965 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9541+0+0 (secure 0 0 0) 0x7fee840cd240 con 0x7fee9c106110 2024-02-02T15:34:06.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.322+0000 7feeb4636b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559449cdf010 0x559449cdf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:06.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.322+0000 7fee9a800700 1 --2- 172.21.15.1:0/2073387965 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fee7c0a0690 0x7fee7c0a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:06.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.322+0000 7feeb4636b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559449cdf930 0x559449cea2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:06.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.322+0000 7feeb4636b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559449cea7e0 0x559449cecbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:06.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.322+0000 7feeb4636b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x559449c534c0 con 0x559449cdf010 2024-02-02T15:34:06.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.322+0000 7feeb4636b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x559449c37aa0 con 0x559449cea7e0 2024-02-02T15:34:06.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.322+0000 7feeb4636b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559449c35460 con 0x559449cdf930 2024-02-02T15:34:06.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.323+0000 7fee9a800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559449cea7e0 0x559449cecbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:06.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.323+0000 7fee99e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559449cdf010 0x559449cdf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:06.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.323+0000 7fee99400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559449cdf930 0x559449cea2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:06.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.323+0000 7fee9a800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559449cea7e0 0x559449cecbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:33690/0 (socket says 172.21.15.1:33690) 2024-02-02T15:34:06.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.323+0000 7fee9a800700 1 -- 172.21.15.1:0/2935557799 learned_addr learned my addr 172.21.15.1:0/2935557799 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:34:06.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.323+0000 7fee99400700 1 -- 172.21.15.1:0/2935557799 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559449cea7e0 msgr2=0x559449cecbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:06.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.323+0000 7fee99400700 1 --2- 172.21.15.1:0/2935557799 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559449cea7e0 0x559449cecbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:06.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.323+0000 7fee99400700 1 -- 172.21.15.1:0/2935557799 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559449cdf010 msgr2=0x559449cdf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:06.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.323+0000 7fee99400700 1 --2- 172.21.15.1:0/2935557799 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559449cdf010 0x559449cdf3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:06.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.323+0000 7fee99400700 1 -- 172.21.15.1:0/2935557799 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559449ab23b0 con 0x559449cdf930 2024-02-02T15:34:06.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.323+0000 7fee9a800700 1 --2- 172.21.15.1:0/2073387965 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fee7c0a0690 0x7fee7c0a2b50 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fee94007810 tx=0x7fee94007bd0 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:06.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.323+0000 7fee99400700 1 --2- 172.21.15.1:0/2935557799 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559449cdf930 0x559449cea2a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fee840ca500 tx=0x7fee84099f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58529423c9ae9f9a server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:06.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.323+0000 7fee8b000700 1 -- 172.21.15.1:0/2935557799 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee840cd020 con 0x559449cdf930 2024-02-02T15:34:06.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.323+0000 7fee8b000700 1 -- 172.21.15.1:0/2935557799 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fee8409aa90 con 0x559449cdf930 2024-02-02T15:34:06.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.323+0000 7fee8b000700 1 -- 172.21.15.1:0/2935557799 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee840a0070 con 0x559449cdf930 2024-02-02T15:34:06.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.324+0000 7feeb4636b80 1 -- 172.21.15.1:0/2935557799 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559449cdf930 msgr2=0x559449cea2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:06.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.324+0000 7feeb4636b80 1 --2- 172.21.15.1:0/2935557799 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559449cdf930 0x559449cea2a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fee840ca500 tx=0x7fee84099f80 comp rx=0 tx=0).stop 2024-02-02T15:34:06.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.324+0000 7feeb4636b80 1 -- 172.21.15.1:0/2935557799 shutdown_connections 2024-02-02T15:34:06.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.324+0000 7feeb4636b80 1 --2- 172.21.15.1:0/2935557799 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559449cea7e0 0x559449cecbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:06.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.324+0000 7feeb4636b80 1 --2- 172.21.15.1:0/2935557799 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559449cdf010 0x559449cdf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:06.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.324+0000 7feeb4636b80 1 --2- 172.21.15.1:0/2935557799 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559449cdf930 0x559449cea2a0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:06.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.324+0000 7feeb4636b80 1 -- 172.21.15.1:0/2935557799 >> 172.21.15.1:0/2935557799 conn(0x559449c3edb0 msgr2=0x559449cdb3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:06.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.324+0000 7feeb4636b80 1 -- 172.21.15.1:0/2935557799 shutdown_connections 2024-02-02T15:34:06.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.324+0000 7feeb4636b80 1 -- 172.21.15.1:0/2935557799 wait complete. 2024-02-02T15:34:06.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.325+0000 7feeb4636b80 1 -- start start 2024-02-02T15:34:06.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.325+0000 7feeb4636b80 1 Processor -- start 2024-02-02T15:34:06.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.325+0000 7feeb4636b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559449cdf010 0x559449ceabc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:06.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.326+0000 7feeb4636b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559449cdf930 0x559449ceb100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:06.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.326+0000 7feeb4636b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559449cea7e0 0x559449ceb640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:06.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.326+0000 7feeb4636b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x559449c30d50 con 0x559449cdf930 2024-02-02T15:34:06.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.326+0000 7feeb4636b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x559449bccd80 con 0x559449cea7e0 2024-02-02T15:34:06.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.326+0000 7feeb4636b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559449bcca20 con 0x559449cdf010 2024-02-02T15:34:06.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.326+0000 7feeb4636b80 10 client.0 _ll_register_callbacks cb 0x559449bce290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:34:06.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.326+0000 7fee9a800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559449cea7e0 0x559449ceb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:06.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.326+0000 7fee99e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559449cdf010 0x559449ceabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:06.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.326+0000 7fee9a800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559449cea7e0 0x559449ceb640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:33696/0 (socket says 172.21.15.1:33696) 2024-02-02T15:34:06.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.326+0000 7fee9a800700 1 -- 172.21.15.1:0/480776597 learned_addr learned my addr 172.21.15.1:0/480776597 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:34:06.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.326+0000 7fee99400700 1 --2- 172.21.15.1:0/480776597 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559449cdf930 0x559449ceb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:06.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.326+0000 7fee9a800700 1 -- 172.21.15.1:0/480776597 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559449cdf010 msgr2=0x559449ceabc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:06.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.326+0000 7fee9a800700 1 --2- 172.21.15.1:0/480776597 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559449cdf010 0x559449ceabc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:06.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.326+0000 7fee9a800700 1 -- 172.21.15.1:0/480776597 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559449cdf930 msgr2=0x559449ceb100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:06.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.326+0000 7fee9a800700 1 --2- 172.21.15.1:0/480776597 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559449cdf930 0x559449ceb100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:06.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.326+0000 7fee9a800700 1 -- 172.21.15.1:0/480776597 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee9c1a60f0 con 0x559449cea7e0 2024-02-02T15:34:06.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.326+0000 7fee9a800700 1 --2- 172.21.15.1:0/480776597 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559449cea7e0 0x559449ceb640 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fee94017ec0 tx=0x7fee9401a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d60d0b258c7cd564 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:06.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.327+0000 7fee83400700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:34:06.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.327+0000 7fee83400700 1 -- 172.21.15.1:0/480776597 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee9400e050 con 0x559449cea7e0 2024-02-02T15:34:06.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.327+0000 7fee83400700 1 -- 172.21.15.1:0/480776597 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9541+0+0 (secure 0 0 0) 0x7fee940235f0 con 0x559449cea7e0 2024-02-02T15:34:06.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.327+0000 7feeb4636b80 1 -- 172.21.15.1:0/480776597 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559449cd30f0 con 0x559449cea7e0 2024-02-02T15:34:06.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.327+0000 7fee83400700 1 -- 172.21.15.1:0/480776597 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fee9401aea0 con 0x559449cea7e0 2024-02-02T15:34:06.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.327+0000 7feeb4636b80 10 client.27470 fetch_fsmap learned FSMap version 40 2024-02-02T15:34:06.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.327+0000 7feeb4636b80 1 -- 172.21.15.1:0/480776597 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x559449a5eb60 con 0x559449cea7e0 2024-02-02T15:34:06.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.327+0000 7fee83400700 1 -- 172.21.15.1:0/480776597 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559449cd30f0 con 0x559449cea7e0 2024-02-02T15:34:06.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.328+0000 7fee83400700 1 -- 172.21.15.1:0/480776597 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 40) v1 ==== 7502+0+0 (secure 0 0 0) 0x7fee94022390 con 0x559449cea7e0 2024-02-02T15:34:06.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.328+0000 7feeb4636b80 10 client.27470 fetch_fsmap finished waiting for FSMap version 40 2024-02-02T15:34:06.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.328+0000 7feeb4636b80 10 client.27470 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:34:06.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.328+0000 7feeb4636b80 20 client.27470 populate_metadata read hostname 'smithi001' 2024-02-02T15:34:06.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.328+0000 7feeb4636b80 1 --2- 172.21.15.1:0/480776597 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x559449cf9ed0 0x559449cfc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:06.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.328+0000 7feeb4636b80 4 client.27470 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:34:06.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.328+0000 7feeb4636b80 1 -- 172.21.15.1:0/480776597 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559449c29dd0 con 0x559449cf9ed0 2024-02-02T15:34:06.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.328+0000 7fee99e00700 1 --2- 172.21.15.1:0/480776597 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x559449cf9ed0 0x559449cfc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:06.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.328+0000 7fee99e00700 1 --2- 172.21.15.1:0/480776597 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x559449cf9ed0 0x559449cfc2f0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74cece7c1772e3e7 server_cookie=e59f60ed6eaf4ad3 in_seq=0 out_seq=0 2024-02-02T15:34:06.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.328+0000 7fee83400700 10 client.27470 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:34:06.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.329+0000 7fee83400700 1 -- 172.21.15.1:0/480776597 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x559449c29dd0 con 0x559449cf9ed0 2024-02-02T15:34:06.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.329+0000 7fee83400700 10 client.27470 handle_command_reply: tid=0 2024-02-02T15:34:06.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.329+0000 7feeb4636b80 1 client.27470 shutdown 2024-02-02T15:34:06.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.330+0000 7feeb4636b80 1 -- 172.21.15.1:0/480776597 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559449cea7e0 msgr2=0x559449ceb640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:06.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.330+0000 7feeb4636b80 1 --2- 172.21.15.1:0/480776597 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559449cea7e0 0x559449ceb640 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fee94017ec0 tx=0x7fee9401a000 comp rx=0 tx=0).stop 2024-02-02T15:34:06.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.330+0000 7feeb4636b80 1 -- 172.21.15.1:0/480776597 shutdown_connections 2024-02-02T15:34:06.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.330+0000 7feeb4636b80 1 --2- 172.21.15.1:0/480776597 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x559449cf9ed0 0x559449cfc2f0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:06.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.330+0000 7feeb4636b80 1 --2- 172.21.15.1:0/480776597 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559449cea7e0 0x559449ceb640 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:06.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.330+0000 7feeb4636b80 1 --2- 172.21.15.1:0/480776597 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559449cdf010 0x559449ceabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:06.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.330+0000 7feeb4636b80 1 --2- 172.21.15.1:0/480776597 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559449cdf930 0x559449ceb100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:06.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.330+0000 7feeb4636b80 1 -- 172.21.15.1:0/480776597 >> 172.21.15.1:0/480776597 conn(0x559449c3edb0 msgr2=0x559449cdafa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:06.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.330+0000 7fee83400700 0 client.27470 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:34:06.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.330+0000 7feeb4636b80 1 -- 172.21.15.1:0/480776597 shutdown_connections 2024-02-02T15:34:06.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.330+0000 7feeb4636b80 1 -- 172.21.15.1:0/480776597 wait complete. 2024-02-02T15:34:06.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.330+0000 7feeb4636b80 20 client.27470 trim_cache size 0 max 16384 2024-02-02T15:34:06.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.348+0000 7feeb4636b80 1 Processor -- start 2024-02-02T15:34:06.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.348+0000 7feeb4636b80 1 -- start start 2024-02-02T15:34:06.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.348+0000 7feeb4636b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559449cdf010 0x559449ce91d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:06.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.348+0000 7feeb4636b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559449cdf930 0x559449cf61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:06.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.348+0000 7feeb4636b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559449cea7e0 0x559449ce4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:06.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.348+0000 7feeb4636b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x559449ab46a0 con 0x559449cdf930 2024-02-02T15:34:06.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.348+0000 7feeb4636b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x559449cdc980 con 0x559449cea7e0 2024-02-02T15:34:06.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.348+0000 7feeb4636b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559449ce9ef0 con 0x559449cdf010 2024-02-02T15:34:06.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.349+0000 7fee99e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559449cdf010 0x559449ce91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:06.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.349+0000 7fee99e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559449cdf010 0x559449ce91d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:39008/0 (socket says 172.21.15.1:39008) 2024-02-02T15:34:06.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.349+0000 7fee99e00700 1 -- 172.21.15.1:0/3606846425 learned_addr learned my addr 172.21.15.1:0/3606846425 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:34:06.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.349+0000 7fee9a800700 1 --2- 172.21.15.1:0/3606846425 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559449cea7e0 0x559449ce4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:06.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.349+0000 7fee99400700 1 --2- 172.21.15.1:0/3606846425 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559449cdf930 0x559449cf61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:06.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.349+0000 7fee99e00700 1 -- 172.21.15.1:0/3606846425 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559449cea7e0 msgr2=0x559449ce4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:06.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.349+0000 7fee99e00700 1 --2- 172.21.15.1:0/3606846425 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559449cea7e0 0x559449ce4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:06.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.349+0000 7fee99e00700 1 -- 172.21.15.1:0/3606846425 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559449cdf930 msgr2=0x559449cf61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:06.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.349+0000 7fee99e00700 1 --2- 172.21.15.1:0/3606846425 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559449cdf930 0x559449cf61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:06.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.349+0000 7fee99e00700 1 -- 172.21.15.1:0/3606846425 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559449bf6ea0 con 0x559449cdf010 2024-02-02T15:34:06.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.349+0000 7fee99400700 1 --2- 172.21.15.1:0/3606846425 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559449cdf930 0x559449cf61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:34:06.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.350+0000 7fee99e00700 1 --2- 172.21.15.1:0/3606846425 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559449cdf010 0x559449ce91d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x559449cfcbd0 tx=0x7fee8c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23f38a5704312a53 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:06.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.352+0000 7fee8ba00700 1 -- 172.21.15.1:0/3606846425 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee8c016070 con 0x559449cdf010 2024-02-02T15:34:06.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.352+0000 7fee8ba00700 1 -- 172.21.15.1:0/3606846425 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fee8c017040 con 0x559449cdf010 2024-02-02T15:34:06.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.352+0000 7fee8ba00700 1 -- 172.21.15.1:0/3606846425 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee8c01c440 con 0x559449cdf010 2024-02-02T15:34:06.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.353+0000 7feeb4636b80 1 -- 172.21.15.1:0/3606846425 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559449cdf010 msgr2=0x559449ce91d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:06.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.353+0000 7feeb4636b80 1 --2- 172.21.15.1:0/3606846425 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559449cdf010 0x559449ce91d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x559449cfcbd0 tx=0x7fee8c004750 comp rx=0 tx=0).stop 2024-02-02T15:34:06.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.353+0000 7feeb4636b80 1 -- 172.21.15.1:0/3606846425 shutdown_connections 2024-02-02T15:34:06.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.353+0000 7feeb4636b80 1 --2- 172.21.15.1:0/3606846425 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559449cea7e0 0x559449ce4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:06.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.353+0000 7feeb4636b80 1 --2- 172.21.15.1:0/3606846425 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559449cdf010 0x559449ce91d0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:06.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.353+0000 7feeb4636b80 1 --2- 172.21.15.1:0/3606846425 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559449cdf930 0x559449cf61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:06.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.353+0000 7feeb4636b80 1 -- 172.21.15.1:0/3606846425 >> 172.21.15.1:0/3606846425 conn(0x559449c3edb0 msgr2=0x559449ce41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:06.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.353+0000 7feeb4636b80 1 -- 172.21.15.1:0/3606846425 shutdown_connections 2024-02-02T15:34:06.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.353+0000 7feeb4636b80 1 -- 172.21.15.1:0/3606846425 wait complete. 2024-02-02T15:34:06.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.354+0000 7feeb4636b80 1 -- start start 2024-02-02T15:34:06.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.354+0000 7feeb4636b80 1 Processor -- start 2024-02-02T15:34:06.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.354+0000 7feeb4636b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559449cdf010 0x559449ceeac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:06.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.354+0000 7feeb4636b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559449cdf930 0x559449cecdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:06.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.354+0000 7feeb4636b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559449cea7e0 0x559449ced370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:06.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.354+0000 7feeb4636b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x559449ceda80 con 0x559449cdf010 2024-02-02T15:34:06.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.354+0000 7feeb4636b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x559449cedbf0 con 0x559449cea7e0 2024-02-02T15:34:06.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.354+0000 7feeb4636b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559449d4ba00 con 0x559449cdf930 2024-02-02T15:34:06.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.354+0000 7feeb4636b80 10 client.0 _ll_register_callbacks cb 0x559449bce290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:34:06.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.355+0000 7fee99400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559449cdf930 0x559449cecdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:06.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.355+0000 7fee99e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559449cdf010 0x559449ceeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:06.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.355+0000 7fee99400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559449cdf930 0x559449cecdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:39016/0 (socket says 172.21.15.1:39016) 2024-02-02T15:34:06.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.355+0000 7fee99400700 1 -- 172.21.15.1:0/3251452437 learned_addr learned my addr 172.21.15.1:0/3251452437 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:34:06.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.355+0000 7fee9a800700 1 --2- 172.21.15.1:0/3251452437 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559449cea7e0 0x559449ced370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:06.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.355+0000 7fee99400700 1 -- 172.21.15.1:0/3251452437 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559449cea7e0 msgr2=0x559449ced370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:06.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.355+0000 7fee99400700 1 --2- 172.21.15.1:0/3251452437 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559449cea7e0 0x559449ced370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:06.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.355+0000 7fee99400700 1 -- 172.21.15.1:0/3251452437 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559449cdf010 msgr2=0x559449ceeac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:06.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.355+0000 7fee99400700 1 --2- 172.21.15.1:0/3251452437 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559449cdf010 0x559449ceeac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:06.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.355+0000 7fee99400700 1 -- 172.21.15.1:0/3251452437 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559449ab23b0 con 0x559449cdf930 2024-02-02T15:34:06.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.355+0000 7fee99400700 1 --2- 172.21.15.1:0/3251452437 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559449cdf930 0x559449cecdd0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fee8409a090 tx=0x7fee84009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66e108da6496106f server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:06.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.355+0000 7fee83400700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:34:06.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.355+0000 7fee83400700 1 -- 172.21.15.1:0/3251452437 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee8409a7f0 con 0x559449cdf930 2024-02-02T15:34:06.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.356+0000 7fee83400700 1 -- 172.21.15.1:0/3251452437 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9541+0+0 (secure 0 0 0) 0x7fee8409a390 con 0x559449cdf930 2024-02-02T15:34:06.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.356+0000 7feeb4636b80 1 -- 172.21.15.1:0/3251452437 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fee8409aa90 con 0x559449cdf930 2024-02-02T15:34:06.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.356+0000 7fee83400700 1 -- 172.21.15.1:0/3251452437 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fee840cd040 con 0x559449cdf930 2024-02-02T15:34:06.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.356+0000 7feeb4636b80 10 client.27472 fetch_fsmap learned FSMap version 40 2024-02-02T15:34:06.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.356+0000 7feeb4636b80 1 -- 172.21.15.1:0/3251452437 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x559449ab6cd0 con 0x559449cdf930 2024-02-02T15:34:06.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.356+0000 7fee83400700 1 -- 172.21.15.1:0/3251452437 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fee8409aa90 con 0x559449cdf930 2024-02-02T15:34:06.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.356+0000 7fee83400700 1 -- 172.21.15.1:0/3251452437 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 40) v1 ==== 7502+0+0 (secure 0 0 0) 0x7fee840a1ee0 con 0x559449cdf930 2024-02-02T15:34:06.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.356+0000 7feeb4636b80 10 client.27472 fetch_fsmap finished waiting for FSMap version 40 2024-02-02T15:34:06.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.356+0000 7feeb4636b80 10 client.27472 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:34:06.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.356+0000 7feeb4636b80 20 client.27472 populate_metadata read hostname 'smithi001' 2024-02-02T15:34:06.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.356+0000 7feeb4636b80 1 --2- 172.21.15.1:0/3251452437 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x559449d4cd00 0x559449d4f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:06.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.356+0000 7feeb4636b80 4 client.27472 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2024-02-02T15:34:06.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.356+0000 7feeb4636b80 1 -- 172.21.15.1:0/3251452437 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559449beff50 con 0x559449d4cd00 2024-02-02T15:34:06.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.356+0000 7fee99e00700 1 --2- 172.21.15.1:0/3251452437 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x559449d4cd00 0x559449d4f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:06.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.357+0000 7fee99e00700 1 --2- 172.21.15.1:0/3251452437 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x559449d4cd00 0x559449d4f120 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=200237dfe33c293c server_cookie=94727a9dfdd8af7c in_seq=0 out_seq=0 2024-02-02T15:34:06.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.357+0000 7fee83400700 10 client.27472 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:34:06.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.357+0000 7fee83400700 1 -- 172.21.15.1:0/3251452437 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x559449beff50 con 0x559449d4cd00 2024-02-02T15:34:06.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.357+0000 7fee83400700 10 client.27472 handle_command_reply: tid=0 2024-02-02T15:34:06.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.357+0000 7feeb4636b80 1 client.27472 shutdown 2024-02-02T15:34:06.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.357+0000 7feeb4636b80 1 -- 172.21.15.1:0/3251452437 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559449cdf930 msgr2=0x559449cecdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:06.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.357+0000 7feeb4636b80 1 --2- 172.21.15.1:0/3251452437 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559449cdf930 0x559449cecdd0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fee8409a090 tx=0x7fee84009310 comp rx=0 tx=0).stop 2024-02-02T15:34:06.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.357+0000 7feeb4636b80 1 -- 172.21.15.1:0/3251452437 shutdown_connections 2024-02-02T15:34:06.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.357+0000 7feeb4636b80 1 --2- 172.21.15.1:0/3251452437 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x559449d4cd00 0x559449d4f120 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:06.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.357+0000 7feeb4636b80 1 --2- 172.21.15.1:0/3251452437 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559449cea7e0 0x559449ced370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:06.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.357+0000 7fee83400700 0 client.27472 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:34:06.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.357+0000 7feeb4636b80 1 --2- 172.21.15.1:0/3251452437 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559449cdf010 0x559449ceeac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:06.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.357+0000 7feeb4636b80 1 --2- 172.21.15.1:0/3251452437 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559449cdf930 0x559449cecdd0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:06.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.357+0000 7feeb4636b80 1 -- 172.21.15.1:0/3251452437 >> 172.21.15.1:0/3251452437 conn(0x559449c3edb0 msgr2=0x559449cf61d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:06.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.357+0000 7feeb4636b80 1 -- 172.21.15.1:0/3251452437 shutdown_connections 2024-02-02T15:34:06.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.358+0000 7feeb4636b80 1 -- 172.21.15.1:0/3251452437 wait complete. 2024-02-02T15:34:06.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.358+0000 7feeb4636b80 20 client.27472 trim_cache size 0 max 16384 2024-02-02T15:34:06.363 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:34:06.363 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2024-02-02T15:34:06.363 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2024-02-02T15:34:06.363 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:34:06.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.358+0000 7feea0c00700 1 -- 172.21.15.1:0/2073387965 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fee7c0a0690 msgr2=0x7fee7c0a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:06.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.358+0000 7feea0c00700 1 --2- 172.21.15.1:0/2073387965 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fee7c0a0690 0x7fee7c0a2b50 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fee94007810 tx=0x7fee94007bd0 comp rx=0 tx=0).stop 2024-02-02T15:34:06.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.358+0000 7feea0c00700 1 -- 172.21.15.1:0/2073387965 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee9c106110 msgr2=0x7fee9c199620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:06.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.358+0000 7feea0c00700 1 --2- 172.21.15.1:0/2073387965 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee9c106110 0x7fee9c199620 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fee84004760 tx=0x7fee84004790 comp rx=0 tx=0).stop 2024-02-02T15:34:06.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.359+0000 7feea0c00700 1 -- 172.21.15.1:0/2073387965 shutdown_connections 2024-02-02T15:34:06.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.359+0000 7feea0c00700 1 --2- 172.21.15.1:0/2073387965 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fee7c0a0690 0x7fee7c0a2b50 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:06.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.359+0000 7feea0c00700 1 --2- 172.21.15.1:0/2073387965 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fee9c19da80 0x7fee9c19df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:06.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.359+0000 7feea0c00700 1 --2- 172.21.15.1:0/2073387965 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fee9c100ff0 0x7fee9c1990e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:06.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.359+0000 7feea0c00700 1 --2- 172.21.15.1:0/2073387965 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fee9c106110 0x7fee9c199620 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:06.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.359+0000 7feea0c00700 1 -- 172.21.15.1:0/2073387965 >> 172.21.15.1:0/2073387965 conn(0x7fee9c0fc810 msgr2=0x7fee9c0fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:06.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.359+0000 7feea0c00700 1 -- 172.21.15.1:0/2073387965 shutdown_connections 2024-02-02T15:34:06.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.359+0000 7feea0c00700 1 -- 172.21.15.1:0/2073387965 wait complete. 2024-02-02T15:34:06.377 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-02T15:34:06.377 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T15:34:06.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.585+0000 7f8678c00700 1 -- 172.21.15.1:0/2961126751 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8674106ac0 msgr2=0x7f8674106ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:06.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.585+0000 7f8678c00700 1 --2- 172.21.15.1:0/2961126751 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8674106ac0 0x7f8674106ea0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f8664009990 tx=0x7f8664009ca0 comp rx=0 tx=0).stop 2024-02-02T15:34:06.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.585+0000 7f8678c00700 1 -- 172.21.15.1:0/2961126751 shutdown_connections 2024-02-02T15:34:06.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.585+0000 7f8678c00700 1 --2- 172.21.15.1:0/2961126751 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8674101470 0x7f867410d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:06.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.585+0000 7f8678c00700 1 --2- 172.21.15.1:0/2961126751 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8674106ac0 0x7f8674106ea0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:06.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.585+0000 7f8678c00700 1 --2- 172.21.15.1:0/2961126751 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8674100ab0 0x7f8674100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:06.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.585+0000 7f8678c00700 1 -- 172.21.15.1:0/2961126751 >> 172.21.15.1:0/2961126751 conn(0x7f86740fc560 msgr2=0x7f86740fe980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:06.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.585+0000 7f8678c00700 1 -- 172.21.15.1:0/2961126751 shutdown_connections 2024-02-02T15:34:06.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.585+0000 7f8678c00700 1 -- 172.21.15.1:0/2961126751 wait complete. 2024-02-02T15:34:06.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.586+0000 7f8678c00700 1 Processor -- start 2024-02-02T15:34:06.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.586+0000 7f8678c00700 1 -- start start 2024-02-02T15:34:06.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.587+0000 7f8678c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8674100ab0 0x7f8674198d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:06.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.587+0000 7f8678c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8674101470 0x7f86741992b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:06.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.587+0000 7f8678c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8674106ac0 0x7f867419d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:06.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.587+0000 7f8678c00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f867419dd50 con 0x7f8674100ab0 2024-02-02T15:34:06.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.587+0000 7f8678c00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f867419dec0 con 0x7f8674101470 2024-02-02T15:34:06.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.587+0000 7f8678c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f867419e080 con 0x7f8674106ac0 2024-02-02T15:34:06.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.587+0000 7f8673e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8674106ac0 0x7f867419d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:06.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.587+0000 7f8672a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8674101470 0x7f86741992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:06.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.587+0000 7f8673e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8674106ac0 0x7f867419d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:39034/0 (socket says 172.21.15.1:39034) 2024-02-02T15:34:06.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.587+0000 7f8673e00700 1 -- 172.21.15.1:0/3709274727 learned_addr learned my addr 172.21.15.1:0/3709274727 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:34:06.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.587+0000 7f8673400700 1 --2- 172.21.15.1:0/3709274727 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8674100ab0 0x7f8674198d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:06.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.588+0000 7f8673e00700 1 -- 172.21.15.1:0/3709274727 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8674101470 msgr2=0x7f86741992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:06.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.588+0000 7f8673e00700 1 --2- 172.21.15.1:0/3709274727 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8674101470 0x7f86741992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:06.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.588+0000 7f8673e00700 1 -- 172.21.15.1:0/3709274727 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8674100ab0 msgr2=0x7f8674198d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:06.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.588+0000 7f8673e00700 1 --2- 172.21.15.1:0/3709274727 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8674100ab0 0x7f8674198d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:06.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.588+0000 7f8673e00700 1 -- 172.21.15.1:0/3709274727 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86640095f0 con 0x7f8674106ac0 2024-02-02T15:34:06.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.588+0000 7f8672a00700 1 --2- 172.21.15.1:0/3709274727 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8674101470 0x7f86741992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:34:06.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.588+0000 7f8673e00700 1 --2- 172.21.15.1:0/3709274727 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8674106ac0 0x7f867419d710 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f866c00ef30 tx=0x7f866c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:06.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.588+0000 7f866be00700 1 -- 172.21.15.1:0/3709274727 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f866c018070 con 0x7f8674106ac0 2024-02-02T15:34:06.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.588+0000 7f866be00700 1 -- 172.21.15.1:0/3709274727 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f866c00f040 con 0x7f8674106ac0 2024-02-02T15:34:06.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.588+0000 7f8678c00700 1 -- 172.21.15.1:0/3709274727 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f867419e300 con 0x7f8674106ac0 2024-02-02T15:34:06.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.588+0000 7f866be00700 1 -- 172.21.15.1:0/3709274727 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f866c00cd00 con 0x7f8674106ac0 2024-02-02T15:34:06.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.588+0000 7f8678c00700 1 -- 172.21.15.1:0/3709274727 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86741a5ba0 con 0x7f8674106ac0 2024-02-02T15:34:06.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.589+0000 7f8678c00700 1 -- 172.21.15.1:0/3709274727 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f867404ea40 con 0x7f8674106ac0 2024-02-02T15:34:06.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.591+0000 7f866be00700 1 -- 172.21.15.1:0/3709274727 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f866c003a00 con 0x7f8674106ac0 2024-02-02T15:34:06.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.593+0000 7f866be00700 1 --2- 172.21.15.1:0/3709274727 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f86540a0690 0x7f86540a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:06.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.593+0000 7f866be00700 1 -- 172.21.15.1:0/3709274727 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f866c0cc6c0 con 0x7f8674106ac0 2024-02-02T15:34:06.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.593+0000 7f8673400700 1 --2- 172.21.15.1:0/3709274727 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f86540a0690 0x7f86540a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:06.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.594+0000 7f8673400700 1 --2- 172.21.15.1:0/3709274727 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f86540a0690 0x7f86540a2b50 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f866400b2e0 tx=0x7f8664005fd0 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:06.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.595+0000 7f866be00700 1 -- 172.21.15.1:0/3709274727 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f866c095b50 con 0x7f8674106ac0 2024-02-02T15:34:06.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.869+0000 7f8678c00700 1 -- 172.21.15.1:0/3709274727 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f867419a0c0 con 0x7f8674106ac0 2024-02-02T15:34:06.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.870+0000 7f866be00700 1 -- 172.21.15.1:0/3709274727 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+10880 (secure 0 0 0) 0x7f866c099170 con 0x7f8674106ac0 2024-02-02T15:34:06.870 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:34:06.870 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2922240448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2922240448},{"type":"v1","addr":"172.21.15.1:6835","nonce":2922240448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2729497431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2729497431},{"type":"v1","addr":"172.21.15.1:6837","nonce":2729497431}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":40,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-02T15:24:47.761435+0000","modified":"2024-02-02T15:33:55.371616+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24529,"mds_2":24518,"mds_3":24514,"mds_4":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14517":{"gid":14517,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/1218303683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":1218303683},{"type":"v1","addr":"172.21.15.1:6839","nonce":1218303683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14529":{"gid":14529,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/2059197496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2059197496},{"type":"v1","addr":"172.21.15.1:6841","nonce":2059197496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/7648986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":7648986},{"type":"v1","addr":"172.21.15.142:6833","nonce":7648986}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/3482746094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3482746094},{"type":"v1","addr":"172.21.15.71:6833","nonce":3482746094}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/388207814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":388207814},{"type":"v1","addr":"172.21.15.71:6835","nonce":388207814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6835/1052606547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1052606547},{"type":"v1","addr":"172.21.15.142:6835","nonce":1052606547}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.142:6837/2870289019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2870289019},{"type":"v1","addr":"172.21.15.142:6837","nonce":2870289019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.71:6837/628016199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":628016199},{"type":"v1","addr":"172.21.15.71:6837","nonce":628016199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.142:6839/321675421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":321675421},{"type":"v1","addr":"172.21.15.142:6839","nonce":321675421}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/1268086493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1268086493},{"type":"v1","addr":"172.21.15.71:6839","nonce":1268086493}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-02T15:34:06.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.872+0000 7f8678c00700 1 -- 172.21.15.1:0/3709274727 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f86540a0690 msgr2=0x7f86540a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:06.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.872+0000 7f8678c00700 1 --2- 172.21.15.1:0/3709274727 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f86540a0690 0x7f86540a2b50 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f866400b2e0 tx=0x7f8664005fd0 comp rx=0 tx=0).stop 2024-02-02T15:34:06.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.872+0000 7f8678c00700 1 -- 172.21.15.1:0/3709274727 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8674106ac0 msgr2=0x7f867419d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:06.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.872+0000 7f8678c00700 1 --2- 172.21.15.1:0/3709274727 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8674106ac0 0x7f867419d710 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f866c00ef30 tx=0x7f866c00c3a0 comp rx=0 tx=0).stop 2024-02-02T15:34:06.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.873+0000 7f8678c00700 1 -- 172.21.15.1:0/3709274727 shutdown_connections 2024-02-02T15:34:06.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.873+0000 7f8678c00700 1 --2- 172.21.15.1:0/3709274727 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f86540a0690 0x7f86540a2b50 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:06.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.873+0000 7f8678c00700 1 --2- 172.21.15.1:0/3709274727 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8674106ac0 0x7f867419d710 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:06.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.873+0000 7f8678c00700 1 --2- 172.21.15.1:0/3709274727 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8674100ab0 0x7f8674198d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:06.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.873+0000 7f8678c00700 1 --2- 172.21.15.1:0/3709274727 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8674101470 0x7f86741992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:06.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.873+0000 7f8678c00700 1 -- 172.21.15.1:0/3709274727 >> 172.21.15.1:0/3709274727 conn(0x7f86740fc560 msgr2=0x7f867410be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:06.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.873+0000 7f8678c00700 1 -- 172.21.15.1:0/3709274727 shutdown_connections 2024-02-02T15:34:06.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:06.873+0000 7f8678c00700 1 -- 172.21.15.1:0/3709274727 wait complete. 2024-02-02T15:34:06.874 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 40 2024-02-02T15:34:06.891 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-02-02T15:34:07.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.090+0000 7f4cd1600700 1 -- 172.21.15.1:0/1653761386 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ccc1019a0 msgr2=0x7f4ccc1059e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:07.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.090+0000 7f4cd1600700 1 --2- 172.21.15.1:0/1653761386 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ccc1019a0 0x7f4ccc1059e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4009990 tx=0x7f4cb4009ca0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.090+0000 7f4cd1600700 1 -- 172.21.15.1:0/1653761386 shutdown_connections 2024-02-02T15:34:07.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.090+0000 7f4cd1600700 1 --2- 172.21.15.1:0/1653761386 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ccc106110 0x7f4ccc10dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.090+0000 7f4cd1600700 1 --2- 172.21.15.1:0/1653761386 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ccc100ff0 0x7f4ccc1013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.090+0000 7f4cd1600700 1 --2- 172.21.15.1:0/1653761386 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ccc1019a0 0x7f4ccc1059e0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.090+0000 7f4cd1600700 1 -- 172.21.15.1:0/1653761386 >> 172.21.15.1:0/1653761386 conn(0x7f4ccc0fc810 msgr2=0x7f4ccc0fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:07.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.091+0000 7f4cd1600700 1 -- 172.21.15.1:0/1653761386 shutdown_connections 2024-02-02T15:34:07.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.091+0000 7f4cd1600700 1 -- 172.21.15.1:0/1653761386 wait complete. 2024-02-02T15:34:07.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.092+0000 7f4cd1600700 1 Processor -- start 2024-02-02T15:34:07.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.092+0000 7f4cd1600700 1 -- start start 2024-02-02T15:34:07.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.092+0000 7f4cd1600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ccc100ff0 0x7f4ccc199140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:07.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.092+0000 7f4cd1600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ccc106110 0x7f4ccc199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:07.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.092+0000 7f4cd1600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ccc19dae0 0x7f4ccc19df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:07.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.092+0000 7f4cd1600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ccc19e560 con 0x7f4ccc106110 2024-02-02T15:34:07.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.092+0000 7f4cd1600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ccc19e6d0 con 0x7f4ccc100ff0 2024-02-02T15:34:07.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.092+0000 7f4cd1600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ccc19e890 con 0x7f4ccc19dae0 2024-02-02T15:34:07.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.092+0000 7f4ccaa00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ccc100ff0 0x7f4ccc199140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:07.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.092+0000 7f4cca000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ccc106110 0x7f4ccc199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:07.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.092+0000 7f4ccaa00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ccc100ff0 0x7f4ccc199140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:33750/0 (socket says 172.21.15.1:33750) 2024-02-02T15:34:07.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.092+0000 7f4ccaa00700 1 -- 172.21.15.1:0/2890902752 learned_addr learned my addr 172.21.15.1:0/2890902752 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:34:07.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.093+0000 7f4ccb400700 1 --2- 172.21.15.1:0/2890902752 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ccc19dae0 0x7f4ccc19df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:07.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.093+0000 7f4ccaa00700 1 -- 172.21.15.1:0/2890902752 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ccc19dae0 msgr2=0x7f4ccc19df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:07.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.093+0000 7f4ccaa00700 1 --2- 172.21.15.1:0/2890902752 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ccc19dae0 0x7f4ccc19df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.093+0000 7f4ccaa00700 1 -- 172.21.15.1:0/2890902752 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ccc106110 msgr2=0x7f4ccc199680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:07.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.093+0000 7f4ccaa00700 1 --2- 172.21.15.1:0/2890902752 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ccc106110 0x7f4ccc199680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.093+0000 7f4ccaa00700 1 -- 172.21.15.1:0/2890902752 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cb40095f0 con 0x7f4ccc100ff0 2024-02-02T15:34:07.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.093+0000 7f4ccaa00700 1 --2- 172.21.15.1:0/2890902752 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ccc100ff0 0x7f4ccc199140 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc00e9a0 tx=0x7f4cbc00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:07.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.093+0000 7f4cc3400700 1 -- 172.21.15.1:0/2890902752 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cbc009790 con 0x7f4ccc100ff0 2024-02-02T15:34:07.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.093+0000 7f4cc3400700 1 -- 172.21.15.1:0/2890902752 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4cbc00cbd0 con 0x7f4ccc100ff0 2024-02-02T15:34:07.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.093+0000 7f4cd1600700 1 -- 172.21.15.1:0/2890902752 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ccc1a5df0 con 0x7f4ccc100ff0 2024-02-02T15:34:07.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.093+0000 7f4cc3400700 1 -- 172.21.15.1:0/2890902752 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cbc018680 con 0x7f4ccc100ff0 2024-02-02T15:34:07.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.094+0000 7f4cd1600700 1 -- 172.21.15.1:0/2890902752 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ccc1a6270 con 0x7f4ccc100ff0 2024-02-02T15:34:07.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.097+0000 7f4cc3400700 1 -- 172.21.15.1:0/2890902752 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f4cbc00cd40 con 0x7f4ccc100ff0 2024-02-02T15:34:07.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.098+0000 7f4cc3400700 1 --2- 172.21.15.1:0/2890902752 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f4cac0a0690 0x7f4cac0a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:07.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.098+0000 7f4ce504bb80 1 Processor -- start 2024-02-02T15:34:07.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.098+0000 7f4cc3400700 1 -- 172.21.15.1:0/2890902752 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f4cbc014070 con 0x7f4ccc100ff0 2024-02-02T15:34:07.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.098+0000 7f4cca000700 1 --2- 172.21.15.1:0/2890902752 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f4cac0a0690 0x7f4cac0a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:07.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.098+0000 7f4ce504bb80 1 -- start start 2024-02-02T15:34:07.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.099+0000 7f4ce504bb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558b380fe010 0x558b380fe3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:07.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.099+0000 7f4ce504bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b380fe930 0x558b381092a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:07.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.099+0000 7f4ce504bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b381097e0 0x558b3810bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:07.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.099+0000 7f4ce504bb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b380724c0 con 0x558b380fe010 2024-02-02T15:34:07.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.099+0000 7f4ce504bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b38056aa0 con 0x558b381097e0 2024-02-02T15:34:07.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.099+0000 7f4ce504bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b38054460 con 0x558b380fe930 2024-02-02T15:34:07.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.099+0000 7f4ccaa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558b380fe010 0x558b380fe3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:07.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.099+0000 7f4ccb400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b381097e0 0x558b3810bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:07.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.099+0000 7f4ccaa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558b380fe010 0x558b380fe3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48636/0 (socket says 172.21.15.1:48636) 2024-02-02T15:34:07.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.099+0000 7f4ccaa00700 1 -- 172.21.15.1:0/2130058008 learned_addr learned my addr 172.21.15.1:0/2130058008 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:34:07.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.099+0000 7f4cca000700 1 --2- 172.21.15.1:0/2890902752 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f4cac0a0690 0x7f4cac0a2b50 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc19a750 tx=0x7f4cb4005fb0 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:07.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.099+0000 7f4cca000700 1 --2- 172.21.15.1:0/2130058008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b380fe930 0x558b381092a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:07.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.099+0000 7f4ccb400700 1 -- 172.21.15.1:0/2130058008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b380fe930 msgr2=0x558b381092a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:07.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.099+0000 7f4ccb400700 1 --2- 172.21.15.1:0/2130058008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b380fe930 0x558b381092a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.099+0000 7f4ccb400700 1 -- 172.21.15.1:0/2130058008 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558b380fe010 msgr2=0x558b380fe3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:07.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.099+0000 7f4ccb400700 1 --2- 172.21.15.1:0/2130058008 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558b380fe010 0x558b380fe3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.099+0000 7f4ccb400700 1 -- 172.21.15.1:0/2130058008 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558b37ed13b0 con 0x558b381097e0 2024-02-02T15:34:07.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.100+0000 7f4ccb400700 1 --2- 172.21.15.1:0/2130058008 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b381097e0 0x558b3810bbd0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f4cc4009340 tx=0x7f4cc4009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8cc2f66874750fa3 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:07.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.100+0000 7f4cbb400700 1 -- 172.21.15.1:0/2130058008 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc400e030 con 0x558b381097e0 2024-02-02T15:34:07.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.100+0000 7f4cbb400700 1 -- 172.21.15.1:0/2130058008 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4cc400f040 con 0x558b381097e0 2024-02-02T15:34:07.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.100+0000 7f4cbb400700 1 -- 172.21.15.1:0/2130058008 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc4015c70 con 0x558b381097e0 2024-02-02T15:34:07.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.101+0000 7f4ce504bb80 1 -- 172.21.15.1:0/2130058008 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b381097e0 msgr2=0x558b3810bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:07.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.101+0000 7f4ce504bb80 1 --2- 172.21.15.1:0/2130058008 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b381097e0 0x558b3810bbd0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f4cc4009340 tx=0x7f4cc4009920 comp rx=0 tx=0).stop 2024-02-02T15:34:07.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.101+0000 7f4ce504bb80 1 -- 172.21.15.1:0/2130058008 shutdown_connections 2024-02-02T15:34:07.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.101+0000 7f4ce504bb80 1 --2- 172.21.15.1:0/2130058008 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b381097e0 0x558b3810bbd0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.101+0000 7f4ce504bb80 1 --2- 172.21.15.1:0/2130058008 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558b380fe010 0x558b380fe3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.101+0000 7f4ce504bb80 1 --2- 172.21.15.1:0/2130058008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b380fe930 0x558b381092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.101+0000 7f4ce504bb80 1 -- 172.21.15.1:0/2130058008 >> 172.21.15.1:0/2130058008 conn(0x558b3805ddb0 msgr2=0x558b380fa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:07.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.101+0000 7f4ce504bb80 1 -- 172.21.15.1:0/2130058008 shutdown_connections 2024-02-02T15:34:07.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.101+0000 7f4ce504bb80 1 -- 172.21.15.1:0/2130058008 wait complete. 2024-02-02T15:34:07.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.101+0000 7f4ce504bb80 1 -- start start 2024-02-02T15:34:07.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.101+0000 7f4ce504bb80 1 Processor -- start 2024-02-02T15:34:07.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.102+0000 7f4ce504bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b380fe010 0x558b381091e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:07.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.102+0000 7f4ce504bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b380fe930 0x558b381063f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:07.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.102+0000 7f4ce504bb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558b381097e0 0x558b38106a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:07.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.102+0000 7f4ce504bb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b3804fd50 con 0x558b381097e0 2024-02-02T15:34:07.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.102+0000 7f4ce504bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b37febd80 con 0x558b380fe010 2024-02-02T15:34:07.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.102+0000 7f4ce504bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b37feba20 con 0x558b380fe930 2024-02-02T15:34:07.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.102+0000 7f4ce504bb80 10 client.0 _ll_register_callbacks cb 0x558b37fed290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:34:07.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.102+0000 7f4ccaa00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b380fe010 0x558b381091e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:07.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.102+0000 7f4ccaa00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b380fe010 0x558b381091e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:33770/0 (socket says 172.21.15.1:33770) 2024-02-02T15:34:07.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.102+0000 7f4ccaa00700 1 -- 172.21.15.1:0/1826383644 learned_addr learned my addr 172.21.15.1:0/1826383644 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:34:07.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.102+0000 7f4ccb400700 1 --2- 172.21.15.1:0/1826383644 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558b381097e0 0x558b38106a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:07.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.102+0000 7f4cca000700 1 --2- 172.21.15.1:0/1826383644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b380fe930 0x558b381063f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:07.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.102+0000 7f4ccaa00700 1 -- 172.21.15.1:0/1826383644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b380fe930 msgr2=0x558b381063f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:07.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.102+0000 7f4ccaa00700 1 --2- 172.21.15.1:0/1826383644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b380fe930 0x558b381063f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.102+0000 7f4ccaa00700 1 -- 172.21.15.1:0/1826383644 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558b381097e0 msgr2=0x558b38106a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:07.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.102+0000 7f4ccaa00700 1 --2- 172.21.15.1:0/1826383644 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558b381097e0 0x558b38106a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.102+0000 7f4ccaa00700 1 -- 172.21.15.1:0/1826383644 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cc4009050 con 0x558b380fe010 2024-02-02T15:34:07.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.102+0000 7f4ccb400700 1 --2- 172.21.15.1:0/1826383644 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558b381097e0 0x558b38106a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:34:07.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.103+0000 7f4ccaa00700 1 --2- 172.21.15.1:0/1826383644 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b380fe010 0x558b381091e0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc00e970 tx=0x7f4cbc099c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7c7e4a389cb1f2d server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:07.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.103+0000 7f4cb3e00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:34:07.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.103+0000 7f4cb3e00700 1 -- 172.21.15.1:0/1826383644 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cbc09d120 con 0x558b380fe010 2024-02-02T15:34:07.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.103+0000 7f4cb3e00700 1 -- 172.21.15.1:0/1826383644 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f4cbc09d310 con 0x558b380fe010 2024-02-02T15:34:07.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.103+0000 7f4ce504bb80 1 -- 172.21.15.1:0/1826383644 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558b380f20f0 con 0x558b380fe010 2024-02-02T15:34:07.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.103+0000 7f4cb3e00700 1 -- 172.21.15.1:0/1826383644 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4cbc0a1550 con 0x558b380fe010 2024-02-02T15:34:07.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.103+0000 7f4ce504bb80 10 client.27494 fetch_fsmap learned FSMap version 40 2024-02-02T15:34:07.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.103+0000 7f4ce504bb80 1 -- 172.21.15.1:0/1826383644 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x558b37e7db60 con 0x558b380fe010 2024-02-02T15:34:07.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.103+0000 7f4cb3e00700 1 -- 172.21.15.1:0/1826383644 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558b380f20f0 con 0x558b380fe010 2024-02-02T15:34:07.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.103+0000 7f4cb3e00700 1 -- 172.21.15.1:0/1826383644 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 40) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f4cbc0b7030 con 0x558b380fe010 2024-02-02T15:34:07.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.104+0000 7f4ce504bb80 10 client.27494 fetch_fsmap finished waiting for FSMap version 40 2024-02-02T15:34:07.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.104+0000 7f4ce504bb80 10 client.27494 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-02-02T15:34:07.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.104+0000 7f4ce504bb80 20 client.27494 populate_metadata read hostname 'smithi001' 2024-02-02T15:34:07.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.104+0000 7f4ce504bb80 1 --2- 172.21.15.1:0/1826383644 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x558b38119280 0x558b3811b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:07.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.104+0000 7f4ce504bb80 4 client.27494 mds_command: new command op to 24502 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:34:07.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.104+0000 7f4ce504bb80 1 -- 172.21.15.1:0/1826383644 --> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558b38048dd0 con 0x558b38119280 2024-02-02T15:34:07.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.104+0000 7f4cca000700 1 --2- 172.21.15.1:0/1826383644 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x558b38119280 0x558b3811b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:07.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.104+0000 7f4cca000700 1 --2- 172.21.15.1:0/1826383644 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x558b38119280 0x558b3811b6a0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5ae3dfce6166c105 server_cookie=ee8766953ca5ace0 in_seq=0 out_seq=0 2024-02-02T15:34:07.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.104+0000 7f4cb3e00700 10 client.27494 ms_handle_connect on v2:172.21.15.142:6834/1052606547 2024-02-02T15:34:07.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.105+0000 7f4cb3e00700 1 -- 172.21.15.1:0/1826383644 <== mds.4 v2:172.21.15.142:6834/1052606547 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x558b38048dd0 con 0x558b38119280 2024-02-02T15:34:07.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.105+0000 7f4cb3e00700 10 client.27494 handle_command_reply: tid=0 2024-02-02T15:34:07.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.105+0000 7f4ce504bb80 1 client.27494 shutdown 2024-02-02T15:34:07.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.105+0000 7f4ce504bb80 1 -- 172.21.15.1:0/1826383644 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b380fe010 msgr2=0x558b381091e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:07.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.105+0000 7f4ce504bb80 1 --2- 172.21.15.1:0/1826383644 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b380fe010 0x558b381091e0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc00e970 tx=0x7f4cbc099c90 comp rx=0 tx=0).stop 2024-02-02T15:34:07.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.105+0000 7f4ce504bb80 1 -- 172.21.15.1:0/1826383644 shutdown_connections 2024-02-02T15:34:07.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.105+0000 7f4ce504bb80 1 --2- 172.21.15.1:0/1826383644 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x558b38119280 0x558b3811b6a0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.105+0000 7f4ce504bb80 1 --2- 172.21.15.1:0/1826383644 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558b381097e0 0x558b38106a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.105+0000 7f4ce504bb80 1 --2- 172.21.15.1:0/1826383644 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b380fe010 0x558b381091e0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.105+0000 7f4ce504bb80 1 --2- 172.21.15.1:0/1826383644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b380fe930 0x558b381063f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.105+0000 7f4cb3e00700 0 client.27494 ms_handle_reset on v2:172.21.15.142:6834/1052606547 2024-02-02T15:34:07.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.105+0000 7f4ce504bb80 1 -- 172.21.15.1:0/1826383644 >> 172.21.15.1:0/1826383644 conn(0x558b3805ddb0 msgr2=0x558b380f9fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:07.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.106+0000 7f4ce504bb80 1 -- 172.21.15.1:0/1826383644 shutdown_connections 2024-02-02T15:34:07.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.106+0000 7f4ce504bb80 1 -- 172.21.15.1:0/1826383644 wait complete. 2024-02-02T15:34:07.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.106+0000 7f4ce504bb80 20 client.27494 trim_cache size 0 max 16384 2024-02-02T15:34:07.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.119+0000 7f4ce504bb80 1 Processor -- start 2024-02-02T15:34:07.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.119+0000 7f4ce504bb80 1 -- start start 2024-02-02T15:34:07.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.119+0000 7f4ce504bb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558b380fe010 0x558b38110360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:07.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.119+0000 7f4ce504bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b380fe930 0x558b38108d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:07.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.119+0000 7f4ce504bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b381097e0 0x558b38101700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:07.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.119+0000 7f4ce504bb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b37ed36a0 con 0x558b380fe010 2024-02-02T15:34:07.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.119+0000 7f4ce504bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b380f4500 con 0x558b380fe930 2024-02-02T15:34:07.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.119+0000 7f4ce504bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b38102240 con 0x558b381097e0 2024-02-02T15:34:07.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.120+0000 7f4ccb400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b381097e0 0x558b38101700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:07.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.120+0000 7f4cca000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b380fe930 0x558b38108d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:07.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.120+0000 7f4ccaa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558b380fe010 0x558b38110360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:07.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.120+0000 7f4cca000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b380fe930 0x558b38108d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:33774/0 (socket says 172.21.15.1:33774) 2024-02-02T15:34:07.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.120+0000 7f4ccaa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558b380fe010 0x558b38110360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48646/0 (socket says 172.21.15.1:48646) 2024-02-02T15:34:07.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.120+0000 7f4cca000700 1 -- 172.21.15.1:0/2572437071 learned_addr learned my addr 172.21.15.1:0/2572437071 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:34:07.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.120+0000 7f4ccb400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b381097e0 0x558b38101700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:39088/0 (socket says 172.21.15.1:39088) 2024-02-02T15:34:07.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.120+0000 7f4cca000700 1 -- 172.21.15.1:0/2572437071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b381097e0 msgr2=0x558b38101700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:07.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.120+0000 7f4cca000700 1 --2- 172.21.15.1:0/2572437071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b381097e0 0x558b38101700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.120+0000 7f4cca000700 1 -- 172.21.15.1:0/2572437071 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558b380fe010 msgr2=0x558b38110360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:07.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.120+0000 7f4cca000700 1 --2- 172.21.15.1:0/2572437071 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558b380fe010 0x558b38110360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.120+0000 7f4cca000700 1 -- 172.21.15.1:0/2572437071 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558b38015ea0 con 0x558b380fe930 2024-02-02T15:34:07.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.120+0000 7f4cca000700 1 --2- 172.21.15.1:0/2572437071 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b380fe930 0x558b38108d10 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4005950 tx=0x7f4cb4023000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af2ca280fa11bd2a server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:07.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.121+0000 7f4cbbe00700 1 -- 172.21.15.1:0/2572437071 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb4032820 con 0x558b380fe930 2024-02-02T15:34:07.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.121+0000 7f4cbbe00700 1 -- 172.21.15.1:0/2572437071 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4cb401e040 con 0x558b380fe930 2024-02-02T15:34:07.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.121+0000 7f4cbbe00700 1 -- 172.21.15.1:0/2572437071 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb4004be0 con 0x558b380fe930 2024-02-02T15:34:07.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.121+0000 7f4ce504bb80 1 -- 172.21.15.1:0/2572437071 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b380fe930 msgr2=0x558b38108d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:07.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.121+0000 7f4ce504bb80 1 --2- 172.21.15.1:0/2572437071 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b380fe930 0x558b38108d10 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4005950 tx=0x7f4cb4023000 comp rx=0 tx=0).stop 2024-02-02T15:34:07.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.121+0000 7f4ce504bb80 1 -- 172.21.15.1:0/2572437071 shutdown_connections 2024-02-02T15:34:07.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.121+0000 7f4ce504bb80 1 --2- 172.21.15.1:0/2572437071 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b381097e0 0x558b38101700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.121+0000 7f4ce504bb80 1 --2- 172.21.15.1:0/2572437071 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558b380fe010 0x558b38110360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.121+0000 7f4ce504bb80 1 --2- 172.21.15.1:0/2572437071 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b380fe930 0x558b38108d10 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.121+0000 7f4ce504bb80 1 -- 172.21.15.1:0/2572437071 >> 172.21.15.1:0/2572437071 conn(0x558b3805ddb0 msgr2=0x558b38114940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:07.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.121+0000 7f4ce504bb80 1 -- 172.21.15.1:0/2572437071 shutdown_connections 2024-02-02T15:34:07.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.121+0000 7f4ce504bb80 1 -- 172.21.15.1:0/2572437071 wait complete. 2024-02-02T15:34:07.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.122+0000 7f4ce504bb80 1 -- start start 2024-02-02T15:34:07.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.122+0000 7f4ce504bb80 1 Processor -- start 2024-02-02T15:34:07.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.122+0000 7f4ce504bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b380fe010 0x558b381100a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:07.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.122+0000 7f4ce504bb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558b380fe930 0x558b3810c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:07.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.122+0000 7f4ce504bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b381097e0 0x558b3810c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:07.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.122+0000 7f4ce504bb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b3810cec0 con 0x558b380fe930 2024-02-02T15:34:07.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.122+0000 7f4ce504bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b3810d030 con 0x558b380fe010 2024-02-02T15:34:07.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.122+0000 7f4ce504bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b3810d1f0 con 0x558b381097e0 2024-02-02T15:34:07.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.122+0000 7f4ce504bb80 10 client.0 _ll_register_callbacks cb 0x558b37fed290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:34:07.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.123+0000 7f4cca000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558b380fe930 0x558b3810c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:07.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.123+0000 7f4cca000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558b380fe930 0x558b3810c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48652/0 (socket says 172.21.15.1:48652) 2024-02-02T15:34:07.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.123+0000 7f4cca000700 1 -- 172.21.15.1:0/1176271343 learned_addr learned my addr 172.21.15.1:0/1176271343 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:34:07.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.123+0000 7f4ccaa00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b380fe010 0x558b381100a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:07.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.123+0000 7f4ccb400700 1 --2- 172.21.15.1:0/1176271343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b381097e0 0x558b3810c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:07.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.123+0000 7f4cca000700 1 -- 172.21.15.1:0/1176271343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b381097e0 msgr2=0x558b3810c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:07.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.123+0000 7f4cca000700 1 --2- 172.21.15.1:0/1176271343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b381097e0 0x558b3810c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.123+0000 7f4cca000700 1 -- 172.21.15.1:0/1176271343 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b380fe010 msgr2=0x558b381100a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:07.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.123+0000 7f4cca000700 1 --2- 172.21.15.1:0/1176271343 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b380fe010 0x558b381100a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.123+0000 7f4cca000700 1 -- 172.21.15.1:0/1176271343 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cb402c040 con 0x558b380fe930 2024-02-02T15:34:07.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.123+0000 7f4ccaa00700 1 --2- 172.21.15.1:0/1176271343 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b380fe010 0x558b381100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:34:07.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.123+0000 7f4cca000700 1 --2- 172.21.15.1:0/1176271343 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558b380fe930 0x558b3810c400 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4009350 tx=0x7f4cb400b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=debd5eefca55b657 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:07.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.123+0000 7f4cb3e00700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:34:07.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.124+0000 7f4cb3e00700 1 -- 172.21.15.1:0/1176271343 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cb4041070 con 0x558b380fe930 2024-02-02T15:34:07.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.124+0000 7f4ce504bb80 1 -- 172.21.15.1:0/1176271343 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4cc400f040 con 0x558b380fe930 2024-02-02T15:34:07.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.124+0000 7f4cb3e00700 1 -- 172.21.15.1:0/1176271343 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f4cb4032b70 con 0x558b380fe930 2024-02-02T15:34:07.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.124+0000 7f4cb3e00700 1 -- 172.21.15.1:0/1176271343 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4cb4032d30 con 0x558b380fe930 2024-02-02T15:34:07.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.124+0000 7f4ce504bb80 10 client.17430 fetch_fsmap learned FSMap version 40 2024-02-02T15:34:07.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.124+0000 7f4ce504bb80 1 -- 172.21.15.1:0/1176271343 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x558b37ed5cd0 con 0x558b380fe930 2024-02-02T15:34:07.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.124+0000 7f4cb3e00700 1 -- 172.21.15.1:0/1176271343 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4cc400f040 con 0x558b380fe930 2024-02-02T15:34:07.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.124+0000 7f4cb3e00700 1 -- 172.21.15.1:0/1176271343 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 40) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f4cb4044030 con 0x558b380fe930 2024-02-02T15:34:07.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.124+0000 7f4ce504bb80 10 client.17430 fetch_fsmap finished waiting for FSMap version 40 2024-02-02T15:34:07.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.124+0000 7f4ce504bb80 10 client.17430 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-02-02T15:34:07.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.124+0000 7f4ce504bb80 20 client.17430 populate_metadata read hostname 'smithi001' 2024-02-02T15:34:07.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.124+0000 7f4ce504bb80 1 --2- 172.21.15.1:0/1176271343 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x558b3816bdb0 0x558b3816e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:07.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.124+0000 7f4ce504bb80 4 client.17430 mds_command: new command op to 24502 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:34:07.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.124+0000 7f4ce504bb80 1 -- 172.21.15.1:0/1176271343 --> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558b3800ef50 con 0x558b3816bdb0 2024-02-02T15:34:07.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.125+0000 7f4ccaa00700 1 --2- 172.21.15.1:0/1176271343 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x558b3816bdb0 0x558b3816e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:07.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.125+0000 7f4ccaa00700 1 --2- 172.21.15.1:0/1176271343 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x558b3816bdb0 0x558b3816e1a0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=eaf3247455f3ea50 server_cookie=26631b99cfc6318b in_seq=0 out_seq=0 2024-02-02T15:34:07.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.125+0000 7f4cb3e00700 10 client.17430 ms_handle_connect on v2:172.21.15.142:6834/1052606547 2024-02-02T15:34:07.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.125+0000 7f4cb3e00700 1 -- 172.21.15.1:0/1176271343 <== mds.4 v2:172.21.15.142:6834/1052606547 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558b3800ef50 con 0x558b3816bdb0 2024-02-02T15:34:07.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.125+0000 7f4cb3e00700 10 client.17430 handle_command_reply: tid=0 2024-02-02T15:34:07.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.125+0000 7f4ce504bb80 1 client.17430 shutdown 2024-02-02T15:34:07.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.125+0000 7f4ce504bb80 1 -- 172.21.15.1:0/1176271343 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558b380fe930 msgr2=0x558b3810c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:07.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.125+0000 7f4ce504bb80 1 --2- 172.21.15.1:0/1176271343 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558b380fe930 0x558b3810c400 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4009350 tx=0x7f4cb400b390 comp rx=0 tx=0).stop 2024-02-02T15:34:07.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.126+0000 7f4ce504bb80 1 -- 172.21.15.1:0/1176271343 shutdown_connections 2024-02-02T15:34:07.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.126+0000 7f4ce504bb80 1 --2- 172.21.15.1:0/1176271343 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x558b3816bdb0 0x558b3816e1a0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.126+0000 7f4ce504bb80 1 --2- 172.21.15.1:0/1176271343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558b381097e0 0x558b3810c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.126+0000 7f4ce504bb80 1 --2- 172.21.15.1:0/1176271343 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558b380fe010 0x558b381100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.126+0000 7f4ce504bb80 1 --2- 172.21.15.1:0/1176271343 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558b380fe930 0x558b3810c400 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.126+0000 7f4cb3e00700 0 client.17430 ms_handle_reset on v2:172.21.15.142:6834/1052606547 2024-02-02T15:34:07.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.126+0000 7f4ce504bb80 1 -- 172.21.15.1:0/1176271343 >> 172.21.15.1:0/1176271343 conn(0x558b3805ddb0 msgr2=0x558b38101700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:07.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.126+0000 7f4ce504bb80 1 -- 172.21.15.1:0/1176271343 shutdown_connections 2024-02-02T15:34:07.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.126+0000 7f4ce504bb80 1 -- 172.21.15.1:0/1176271343 wait complete. 2024-02-02T15:34:07.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.126+0000 7f4ce504bb80 20 client.17430 trim_cache size 0 max 16384 2024-02-02T15:34:07.131 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:34:07.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.126+0000 7f4cd1600700 1 -- 172.21.15.1:0/2890902752 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f4cac0a0690 msgr2=0x7f4cac0a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:07.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.126+0000 7f4cd1600700 1 --2- 172.21.15.1:0/2890902752 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f4cac0a0690 0x7f4cac0a2b50 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc19a750 tx=0x7f4cb4005fb0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.126+0000 7f4cd1600700 1 -- 172.21.15.1:0/2890902752 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ccc100ff0 msgr2=0x7f4ccc199140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:07.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.126+0000 7f4cd1600700 1 --2- 172.21.15.1:0/2890902752 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ccc100ff0 0x7f4ccc199140 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc00e9a0 tx=0x7f4cbc00ecb0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.127+0000 7f4cd1600700 1 -- 172.21.15.1:0/2890902752 shutdown_connections 2024-02-02T15:34:07.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.127+0000 7f4cd1600700 1 --2- 172.21.15.1:0/2890902752 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f4cac0a0690 0x7f4cac0a2b50 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.127+0000 7f4cd1600700 1 --2- 172.21.15.1:0/2890902752 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ccc19dae0 0x7f4ccc19df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.127+0000 7f4cd1600700 1 --2- 172.21.15.1:0/2890902752 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ccc100ff0 0x7f4ccc199140 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.127+0000 7f4cd1600700 1 --2- 172.21.15.1:0/2890902752 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ccc106110 0x7f4ccc199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.127+0000 7f4cd1600700 1 -- 172.21.15.1:0/2890902752 >> 172.21.15.1:0/2890902752 conn(0x7f4ccc0fc810 msgr2=0x7f4ccc0fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:07.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.127+0000 7f4cd1600700 1 -- 172.21.15.1:0/2890902752 shutdown_connections 2024-02-02T15:34:07.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.127+0000 7f4cd1600700 1 -- 172.21.15.1:0/2890902752 wait complete. 2024-02-02T15:34:07.144 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-02-02T15:34:07.243 INFO:tasks.workunit.client.0.smithi001.stdout: 1 448638 7.52 MB/sec execute 221 sec latency 552.122 ms 2024-02-02T15:34:07.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.346+0000 7f492f600700 1 -- 172.21.15.1:0/2441935519 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49280730e0 msgr2=0x7f49280734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:07.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.346+0000 7f492f600700 1 --2- 172.21.15.1:0/2441935519 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49280730e0 0x7f49280734c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f4918009990 tx=0x7f4918009ca0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.347+0000 7f492f600700 1 -- 172.21.15.1:0/2441935519 shutdown_connections 2024-02-02T15:34:07.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.347+0000 7f492f600700 1 --2- 172.21.15.1:0/2441935519 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4928111d90 0x7f4928114180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.347+0000 7f492f600700 1 --2- 172.21.15.1:0/2441935519 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49280730e0 0x7f49280734c0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.347+0000 7f492f600700 1 --2- 172.21.15.1:0/2441935519 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4928073a00 0x7f4928111850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.347+0000 7f492f600700 1 -- 172.21.15.1:0/2441935519 >> 172.21.15.1:0/2441935519 conn(0x7f49280fc810 msgr2=0x7f49280fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:07.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.347+0000 7f492f600700 1 -- 172.21.15.1:0/2441935519 shutdown_connections 2024-02-02T15:34:07.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.347+0000 7f492f600700 1 -- 172.21.15.1:0/2441935519 wait complete. 2024-02-02T15:34:07.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.348+0000 7f492f600700 1 Processor -- start 2024-02-02T15:34:07.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.348+0000 7f492f600700 1 -- start start 2024-02-02T15:34:07.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.348+0000 7f492f600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49280730e0 0x7f492819d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:07.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.348+0000 7f492f600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4928073a00 0x7f492819da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:07.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.348+0000 7f492f600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4928111d90 0x7f49281a1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:07.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.348+0000 7f492f600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49281a24e0 con 0x7f4928073a00 2024-02-02T15:34:07.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.348+0000 7f492f600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49281a2650 con 0x7f4928111d90 2024-02-02T15:34:07.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.348+0000 7f492f600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49281a2810 con 0x7f49280730e0 2024-02-02T15:34:07.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.349+0000 7f492cc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49280730e0 0x7f492819d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:07.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.349+0000 7f4927e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4928073a00 0x7f492819da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:07.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.349+0000 7f492cc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49280730e0 0x7f492819d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:39120/0 (socket says 172.21.15.1:39120) 2024-02-02T15:34:07.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.349+0000 7f492cc00700 1 -- 172.21.15.1:0/696328939 learned_addr learned my addr 172.21.15.1:0/696328939 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:34:07.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.349+0000 7f492d600700 1 --2- 172.21.15.1:0/696328939 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4928111d90 0x7f49281a1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:07.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.349+0000 7f492cc00700 1 -- 172.21.15.1:0/696328939 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4928111d90 msgr2=0x7f49281a1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:07.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.349+0000 7f492cc00700 1 --2- 172.21.15.1:0/696328939 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4928111d90 0x7f49281a1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.349+0000 7f492cc00700 1 -- 172.21.15.1:0/696328939 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4928073a00 msgr2=0x7f492819da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:07.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.349+0000 7f492cc00700 1 --2- 172.21.15.1:0/696328939 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4928073a00 0x7f492819da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.349+0000 7f492cc00700 1 -- 172.21.15.1:0/696328939 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4920009c40 con 0x7f49280730e0 2024-02-02T15:34:07.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.349+0000 7f4927e00700 1 --2- 172.21.15.1:0/696328939 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4928073a00 0x7f492819da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:34:07.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.349+0000 7f492cc00700 1 --2- 172.21.15.1:0/696328939 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49280730e0 0x7f492819d500 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f4918005910 tx=0x7f4918005720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:07.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.349+0000 7f4925600700 1 -- 172.21.15.1:0/696328939 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f491801d070 con 0x7f49280730e0 2024-02-02T15:34:07.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.349+0000 7f4925600700 1 -- 172.21.15.1:0/696328939 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4918022470 con 0x7f49280730e0 2024-02-02T15:34:07.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.349+0000 7f4925600700 1 -- 172.21.15.1:0/696328939 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4918017650 con 0x7f49280730e0 2024-02-02T15:34:07.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.350+0000 7f492f600700 1 -- 172.21.15.1:0/696328939 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49180095f0 con 0x7f49280730e0 2024-02-02T15:34:07.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.350+0000 7f492f600700 1 -- 172.21.15.1:0/696328939 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49281aa150 con 0x7f49280730e0 2024-02-02T15:34:07.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.353+0000 7f4925600700 1 -- 172.21.15.1:0/696328939 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f49180225e0 con 0x7f49280730e0 2024-02-02T15:34:07.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.354+0000 7f4925600700 1 --2- 172.21.15.1:0/696328939 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f49080a0690 0x7f49080a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:07.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.354+0000 7f4943218b80 1 Processor -- start 2024-02-02T15:34:07.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.354+0000 7f4925600700 1 -- 172.21.15.1:0/696328939 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f49180cd4e0 con 0x7f49280730e0 2024-02-02T15:34:07.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.354+0000 7f4927e00700 1 --2- 172.21.15.1:0/696328939 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f49080a0690 0x7f49080a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:07.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.354+0000 7f4943218b80 1 -- start start 2024-02-02T15:34:07.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.355+0000 7f4943218b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555854678010 0x5558546783f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:07.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.355+0000 7f4943218b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555854678930 0x5558546832a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:07.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.355+0000 7f4943218b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558546837e0 0x555854685bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:07.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.355+0000 7f4943218b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558545ec4c0 con 0x5558546837e0 2024-02-02T15:34:07.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.355+0000 7f4943218b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558545d0aa0 con 0x555854678930 2024-02-02T15:34:07.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.355+0000 7f4943218b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558545ce460 con 0x555854678010 2024-02-02T15:34:07.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.355+0000 7f492cc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555854678010 0x5558546783f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:07.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.355+0000 7f492d600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558546837e0 0x555854685bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:07.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.355+0000 7f492cc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555854678010 0x5558546783f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:39128/0 (socket says 172.21.15.1:39128) 2024-02-02T15:34:07.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.355+0000 7f492cc00700 1 -- 172.21.15.1:0/2171915898 learned_addr learned my addr 172.21.15.1:0/2171915898 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:34:07.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.355+0000 7f4927e00700 1 --2- 172.21.15.1:0/696328939 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f49080a0690 0x7f49080a2b50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f492819eb10 tx=0x7f4910009260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:07.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.355+0000 7f4927e00700 1 --2- 172.21.15.1:0/2171915898 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555854678930 0x5558546832a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:07.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.355+0000 7f492cc00700 1 -- 172.21.15.1:0/2171915898 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555854678930 msgr2=0x5558546832a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:07.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.355+0000 7f492cc00700 1 --2- 172.21.15.1:0/2171915898 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555854678930 0x5558546832a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.355+0000 7f492cc00700 1 -- 172.21.15.1:0/2171915898 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558546837e0 msgr2=0x555854685bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:07.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.355+0000 7f492cc00700 1 --2- 172.21.15.1:0/2171915898 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558546837e0 0x555854685bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.355+0000 7f492cc00700 1 -- 172.21.15.1:0/2171915898 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55585444b3b0 con 0x555854678010 2024-02-02T15:34:07.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.356+0000 7f492cc00700 1 --2- 172.21.15.1:0/2171915898 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555854678010 0x5558546783f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f4918006010 tx=0x7f4918009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b1b55cbd2d09b9d server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:07.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.356+0000 7f491dc00700 1 -- 172.21.15.1:0/2171915898 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f491809ad30 con 0x555854678010 2024-02-02T15:34:07.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.356+0000 7f491dc00700 1 -- 172.21.15.1:0/2171915898 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f491809a630 con 0x555854678010 2024-02-02T15:34:07.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.356+0000 7f491dc00700 1 -- 172.21.15.1:0/2171915898 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49180cd070 con 0x555854678010 2024-02-02T15:34:07.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.356+0000 7f4943218b80 1 -- 172.21.15.1:0/2171915898 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555854678010 msgr2=0x5558546783f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:07.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.356+0000 7f4943218b80 1 --2- 172.21.15.1:0/2171915898 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555854678010 0x5558546783f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f4918006010 tx=0x7f4918009ca0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.357+0000 7f4943218b80 1 -- 172.21.15.1:0/2171915898 shutdown_connections 2024-02-02T15:34:07.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.357+0000 7f4943218b80 1 --2- 172.21.15.1:0/2171915898 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558546837e0 0x555854685bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.357+0000 7f4943218b80 1 --2- 172.21.15.1:0/2171915898 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555854678010 0x5558546783f0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.357+0000 7f4943218b80 1 --2- 172.21.15.1:0/2171915898 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555854678930 0x5558546832a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.357+0000 7f4943218b80 1 -- 172.21.15.1:0/2171915898 >> 172.21.15.1:0/2171915898 conn(0x5558545d7db0 msgr2=0x5558546743b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:07.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.357+0000 7f4943218b80 1 -- 172.21.15.1:0/2171915898 shutdown_connections 2024-02-02T15:34:07.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.357+0000 7f4943218b80 1 -- 172.21.15.1:0/2171915898 wait complete. 2024-02-02T15:34:07.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.357+0000 7f4943218b80 1 -- start start 2024-02-02T15:34:07.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.357+0000 7f4943218b80 1 Processor -- start 2024-02-02T15:34:07.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.358+0000 7f4943218b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555854678010 0x555854683bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:07.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.358+0000 7f4943218b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555854678930 0x555854684100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:07.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.358+0000 7f4943218b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558546837e0 0x555854684640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:07.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.358+0000 7f4943218b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558545c9d50 con 0x5558546837e0 2024-02-02T15:34:07.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.358+0000 7f4943218b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x555854565d80 con 0x555854678930 2024-02-02T15:34:07.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.358+0000 7f4943218b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x555854565a20 con 0x555854678010 2024-02-02T15:34:07.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.358+0000 7f4943218b80 10 client.0 _ll_register_callbacks cb 0x555854567290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:34:07.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.358+0000 7f492cc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555854678010 0x555854683bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:07.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.358+0000 7f492cc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555854678010 0x555854683bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:39138/0 (socket says 172.21.15.1:39138) 2024-02-02T15:34:07.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.358+0000 7f492cc00700 1 -- 172.21.15.1:0/2958361199 learned_addr learned my addr 172.21.15.1:0/2958361199 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:34:07.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.358+0000 7f4927e00700 1 --2- 172.21.15.1:0/2958361199 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555854678930 0x555854684100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:07.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.358+0000 7f492d600700 1 --2- 172.21.15.1:0/2958361199 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558546837e0 0x555854684640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:07.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.358+0000 7f492cc00700 1 -- 172.21.15.1:0/2958361199 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555854678930 msgr2=0x555854684100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:07.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.358+0000 7f492cc00700 1 --2- 172.21.15.1:0/2958361199 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555854678930 0x555854684100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.358+0000 7f492cc00700 1 -- 172.21.15.1:0/2958361199 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558546837e0 msgr2=0x555854684640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:07.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.358+0000 7f492cc00700 1 --2- 172.21.15.1:0/2958361199 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558546837e0 0x555854684640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.358+0000 7f492cc00700 1 -- 172.21.15.1:0/2958361199 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49281aa150 con 0x555854678010 2024-02-02T15:34:07.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.358+0000 7f492d600700 1 --2- 172.21.15.1:0/2958361199 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558546837e0 0x555854684640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:34:07.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.359+0000 7f492cc00700 1 --2- 172.21.15.1:0/2958361199 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555854678010 0x555854683bc0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f49180cad50 tx=0x7f491809a050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc0cd806ee44d51b server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:07.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.359+0000 7f4916000700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:34:07.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.359+0000 7f4916000700 1 -- 172.21.15.1:0/2958361199 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f491809ad30 con 0x555854678010 2024-02-02T15:34:07.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.359+0000 7f4916000700 1 -- 172.21.15.1:0/2958361199 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f491809e460 con 0x555854678010 2024-02-02T15:34:07.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.359+0000 7f4943218b80 1 -- 172.21.15.1:0/2958361199 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55585466c0f0 con 0x555854678010 2024-02-02T15:34:07.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.359+0000 7f4916000700 1 -- 172.21.15.1:0/2958361199 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f49180a2610 con 0x555854678010 2024-02-02T15:34:07.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.359+0000 7f4943218b80 10 client.27508 fetch_fsmap learned FSMap version 40 2024-02-02T15:34:07.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.359+0000 7f4943218b80 1 -- 172.21.15.1:0/2958361199 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x5558543f7b60 con 0x555854678010 2024-02-02T15:34:07.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.359+0000 7f4916000700 1 -- 172.21.15.1:0/2958361199 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55585466c0f0 con 0x555854678010 2024-02-02T15:34:07.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.359+0000 7f4916000700 1 -- 172.21.15.1:0/2958361199 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 40) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f49180cd040 con 0x555854678010 2024-02-02T15:34:07.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.360+0000 7f4943218b80 10 client.27508 fetch_fsmap finished waiting for FSMap version 40 2024-02-02T15:34:07.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.360+0000 7f4943218b80 10 client.27508 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-02-02T15:34:07.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.360+0000 7f4943218b80 20 client.27508 populate_metadata read hostname 'smithi001' 2024-02-02T15:34:07.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.360+0000 7f4943218b80 1 --2- 172.21.15.1:0/2958361199 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x555854692910 0x555854694d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:07.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.360+0000 7f4943218b80 4 client.27508 mds_command: new command op to 24514 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:34:07.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.360+0000 7f4943218b80 1 -- 172.21.15.1:0/2958361199 --> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5558545c2dd0 con 0x555854692910 2024-02-02T15:34:07.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.360+0000 7f4927e00700 1 --2- 172.21.15.1:0/2958361199 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x555854692910 0x555854694d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:07.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.360+0000 7f4916000700 10 client.27508 ms_handle_connect on v2:172.21.15.142:6836/2870289019 2024-02-02T15:34:07.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.360+0000 7f4927e00700 1 --2- 172.21.15.1:0/2958361199 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x555854692910 0x555854694d00 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7ba286f4680be73d server_cookie=18b5ff27462d52eb in_seq=0 out_seq=0 2024-02-02T15:34:07.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.361+0000 7f4916000700 1 -- 172.21.15.1:0/2958361199 <== mds.3 v2:172.21.15.142:6836/2870289019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5558545c2dd0 con 0x555854692910 2024-02-02T15:34:07.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.361+0000 7f4916000700 10 client.27508 handle_command_reply: tid=0 2024-02-02T15:34:07.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.361+0000 7f4943218b80 1 client.27508 shutdown 2024-02-02T15:34:07.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.361+0000 7f4943218b80 1 -- 172.21.15.1:0/2958361199 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555854678010 msgr2=0x555854683bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:07.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.361+0000 7f4943218b80 1 --2- 172.21.15.1:0/2958361199 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555854678010 0x555854683bc0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f49180cad50 tx=0x7f491809a050 comp rx=0 tx=0).stop 2024-02-02T15:34:07.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.361+0000 7f4943218b80 1 -- 172.21.15.1:0/2958361199 shutdown_connections 2024-02-02T15:34:07.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.361+0000 7f4943218b80 1 --2- 172.21.15.1:0/2958361199 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x555854692910 0x555854694d00 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.361+0000 7f4943218b80 1 --2- 172.21.15.1:0/2958361199 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558546837e0 0x555854684640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.361+0000 7f4943218b80 1 --2- 172.21.15.1:0/2958361199 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555854678010 0x555854683bc0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.361+0000 7f4943218b80 1 --2- 172.21.15.1:0/2958361199 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555854678930 0x555854684100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.361+0000 7f4916000700 0 client.27508 ms_handle_reset on v2:172.21.15.142:6836/2870289019 2024-02-02T15:34:07.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.361+0000 7f4943218b80 1 -- 172.21.15.1:0/2958361199 >> 172.21.15.1:0/2958361199 conn(0x5558545d7db0 msgr2=0x555854673fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:07.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.362+0000 7f4943218b80 1 -- 172.21.15.1:0/2958361199 shutdown_connections 2024-02-02T15:34:07.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.362+0000 7f4943218b80 1 -- 172.21.15.1:0/2958361199 wait complete. 2024-02-02T15:34:07.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.362+0000 7f4943218b80 20 client.27508 trim_cache size 0 max 16384 2024-02-02T15:34:07.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.375+0000 7f4943218b80 1 Processor -- start 2024-02-02T15:34:07.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.375+0000 7f4943218b80 1 -- start start 2024-02-02T15:34:07.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.375+0000 7f4943218b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555854678010 0x555854687bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:07.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.375+0000 7f4943218b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555854678930 0x55585467dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:07.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.375+0000 7f4943218b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5558546837e0 0x55585468e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:07.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.375+0000 7f4943218b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55585444d6a0 con 0x555854678930 2024-02-02T15:34:07.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.375+0000 7f4943218b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x555854679830 con 0x5558546837e0 2024-02-02T15:34:07.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.375+0000 7f4943218b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x555854680d70 con 0x555854678010 2024-02-02T15:34:07.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.375+0000 7f4927e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555854678930 0x55585467dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:07.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.376+0000 7f4927e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555854678930 0x55585467dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48694/0 (socket says 172.21.15.1:48694) 2024-02-02T15:34:07.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.376+0000 7f4927e00700 1 -- 172.21.15.1:0/775682476 learned_addr learned my addr 172.21.15.1:0/775682476 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:34:07.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.376+0000 7f492cc00700 1 --2- 172.21.15.1:0/775682476 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555854678010 0x555854687bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:07.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.376+0000 7f492d600700 1 --2- 172.21.15.1:0/775682476 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5558546837e0 0x55585468e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:07.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.376+0000 7f4927e00700 1 -- 172.21.15.1:0/775682476 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555854678010 msgr2=0x555854687bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:07.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.376+0000 7f4927e00700 1 --2- 172.21.15.1:0/775682476 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555854678010 0x555854687bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.376+0000 7f4927e00700 1 -- 172.21.15.1:0/775682476 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5558546837e0 msgr2=0x55585468e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:07.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.376+0000 7f4927e00700 1 --2- 172.21.15.1:0/775682476 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5558546837e0 0x55585468e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.376+0000 7f4927e00700 1 -- 172.21.15.1:0/775682476 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55585458fea0 con 0x555854678930 2024-02-02T15:34:07.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.376+0000 7f4927e00700 1 --2- 172.21.15.1:0/775682476 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555854678930 0x55585467dbb0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x555854682e60 tx=0x7f4910014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6c3f3683a35f06f server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:07.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.376+0000 7f491e600700 1 -- 172.21.15.1:0/775682476 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4910024040 con 0x555854678930 2024-02-02T15:34:07.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.376+0000 7f491e600700 1 -- 172.21.15.1:0/775682476 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f491002a380 con 0x555854678930 2024-02-02T15:34:07.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.376+0000 7f491e600700 1 -- 172.21.15.1:0/775682476 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f491001f6d0 con 0x555854678930 2024-02-02T15:34:07.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.377+0000 7f4943218b80 1 -- 172.21.15.1:0/775682476 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555854678930 msgr2=0x55585467dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:07.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.377+0000 7f4943218b80 1 --2- 172.21.15.1:0/775682476 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555854678930 0x55585467dbb0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x555854682e60 tx=0x7f4910014000 comp rx=0 tx=0).stop 2024-02-02T15:34:07.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.377+0000 7f4943218b80 1 -- 172.21.15.1:0/775682476 shutdown_connections 2024-02-02T15:34:07.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.377+0000 7f4943218b80 1 --2- 172.21.15.1:0/775682476 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5558546837e0 0x55585468e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.377+0000 7f4943218b80 1 --2- 172.21.15.1:0/775682476 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555854678010 0x555854687bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.377+0000 7f4943218b80 1 --2- 172.21.15.1:0/775682476 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555854678930 0x55585467dbb0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.377+0000 7f4943218b80 1 -- 172.21.15.1:0/775682476 >> 172.21.15.1:0/775682476 conn(0x5558545d7db0 msgr2=0x55585468f130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:07.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.377+0000 7f4943218b80 1 -- 172.21.15.1:0/775682476 shutdown_connections 2024-02-02T15:34:07.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.377+0000 7f4943218b80 1 -- 172.21.15.1:0/775682476 wait complete. 2024-02-02T15:34:07.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.378+0000 7f4943218b80 1 -- start start 2024-02-02T15:34:07.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.378+0000 7f4943218b80 1 Processor -- start 2024-02-02T15:34:07.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.378+0000 7f4943218b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555854678010 0x5558546dd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:07.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.378+0000 7f4943218b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555854678930 0x5558546dde20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:07.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.378+0000 7f4943218b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5558546837e0 0x5558546d8390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:07.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.378+0000 7f4943218b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558546d8920 con 0x555854678010 2024-02-02T15:34:07.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.378+0000 7f4943218b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558546d8a90 con 0x5558546837e0 2024-02-02T15:34:07.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.378+0000 7f4943218b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558546d8c50 con 0x555854678930 2024-02-02T15:34:07.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.378+0000 7f4943218b80 10 client.0 _ll_register_callbacks cb 0x555854567290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:34:07.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.378+0000 7f492cc00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555854678010 0x5558546dd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:07.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.379+0000 7f4927e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555854678930 0x5558546dde20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:07.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.379+0000 7f492cc00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555854678010 0x5558546dd8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48696/0 (socket says 172.21.15.1:48696) 2024-02-02T15:34:07.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.379+0000 7f492cc00700 1 -- 172.21.15.1:0/535259319 learned_addr learned my addr 172.21.15.1:0/535259319 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:34:07.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.379+0000 7f492d600700 1 --2- 172.21.15.1:0/535259319 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5558546837e0 0x5558546d8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:07.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.379+0000 7f492cc00700 1 -- 172.21.15.1:0/535259319 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555854678930 msgr2=0x5558546dde20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:07.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.379+0000 7f492cc00700 1 --2- 172.21.15.1:0/535259319 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555854678930 0x5558546dde20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.379+0000 7f492cc00700 1 -- 172.21.15.1:0/535259319 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5558546837e0 msgr2=0x5558546d8390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:07.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.379+0000 7f492cc00700 1 --2- 172.21.15.1:0/535259319 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5558546837e0 0x5558546d8390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.379+0000 7f492cc00700 1 -- 172.21.15.1:0/535259319 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4910017070 con 0x555854678010 2024-02-02T15:34:07.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.379+0000 7f492d600700 1 --2- 172.21.15.1:0/535259319 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5558546837e0 0x5558546d8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:34:07.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.379+0000 7f492cc00700 1 --2- 172.21.15.1:0/535259319 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555854678010 0x5558546dd8e0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f4918005f50 tx=0x7f491809a0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64794b7f9cb3c0a9 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:07.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.379+0000 7f4916000700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:34:07.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.379+0000 7f4916000700 1 -- 172.21.15.1:0/535259319 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4918066570 con 0x555854678010 2024-02-02T15:34:07.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.379+0000 7f4916000700 1 -- 172.21.15.1:0/535259319 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f491809ad30 con 0x555854678010 2024-02-02T15:34:07.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.379+0000 7f4943218b80 1 -- 172.21.15.1:0/535259319 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f491809a630 con 0x555854678010 2024-02-02T15:34:07.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.380+0000 7f4916000700 1 -- 172.21.15.1:0/535259319 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f491809a400 con 0x555854678010 2024-02-02T15:34:07.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.380+0000 7f4943218b80 10 client.17448 fetch_fsmap learned FSMap version 40 2024-02-02T15:34:07.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.380+0000 7f4943218b80 1 -- 172.21.15.1:0/535259319 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55585444fcd0 con 0x555854678010 2024-02-02T15:34:07.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.380+0000 7f4916000700 1 -- 172.21.15.1:0/535259319 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f491809a630 con 0x555854678010 2024-02-02T15:34:07.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.380+0000 7f4916000700 1 -- 172.21.15.1:0/535259319 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 40) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f49180a0210 con 0x555854678010 2024-02-02T15:34:07.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.380+0000 7f4943218b80 10 client.17448 fetch_fsmap finished waiting for FSMap version 40 2024-02-02T15:34:07.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.380+0000 7f4943218b80 10 client.17448 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-02-02T15:34:07.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.380+0000 7f4943218b80 20 client.17448 populate_metadata read hostname 'smithi001' 2024-02-02T15:34:07.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.380+0000 7f4943218b80 1 --2- 172.21.15.1:0/535259319 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x5558546e5290 0x5558546e7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:07.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.380+0000 7f4943218b80 4 client.17448 mds_command: new command op to 24514 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:34:07.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.380+0000 7f4943218b80 1 -- 172.21.15.1:0/535259319 --> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555854588f50 con 0x5558546e5290 2024-02-02T15:34:07.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.380+0000 7f4927e00700 1 --2- 172.21.15.1:0/535259319 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x5558546e5290 0x5558546e7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:07.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.381+0000 7f4927e00700 1 --2- 172.21.15.1:0/535259319 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x5558546e5290 0x5558546e7680 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d62754330777dbe6 server_cookie=71f81dd3e8fdd86e in_seq=0 out_seq=0 2024-02-02T15:34:07.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.381+0000 7f4916000700 10 client.17448 ms_handle_connect on v2:172.21.15.142:6836/2870289019 2024-02-02T15:34:07.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.381+0000 7f4916000700 1 -- 172.21.15.1:0/535259319 <== mds.3 v2:172.21.15.142:6836/2870289019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555854588f50 con 0x5558546e5290 2024-02-02T15:34:07.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.381+0000 7f4916000700 10 client.17448 handle_command_reply: tid=0 2024-02-02T15:34:07.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.381+0000 7f4943218b80 1 client.17448 shutdown 2024-02-02T15:34:07.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.381+0000 7f4943218b80 1 -- 172.21.15.1:0/535259319 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555854678010 msgr2=0x5558546dd8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:07.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.381+0000 7f4943218b80 1 --2- 172.21.15.1:0/535259319 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555854678010 0x5558546dd8e0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f4918005f50 tx=0x7f491809a0c0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.381+0000 7f4943218b80 1 -- 172.21.15.1:0/535259319 shutdown_connections 2024-02-02T15:34:07.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.381+0000 7f4943218b80 1 --2- 172.21.15.1:0/535259319 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5558546837e0 0x5558546d8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.381+0000 7f4943218b80 1 --2- 172.21.15.1:0/535259319 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x5558546e5290 0x5558546e7680 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.381+0000 7f4943218b80 1 --2- 172.21.15.1:0/535259319 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555854678010 0x5558546dd8e0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.381+0000 7f4943218b80 1 --2- 172.21.15.1:0/535259319 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555854678930 0x5558546dde20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.381+0000 7f4916000700 0 client.17448 ms_handle_reset on v2:172.21.15.142:6836/2870289019 2024-02-02T15:34:07.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.381+0000 7f4943218b80 1 -- 172.21.15.1:0/535259319 >> 172.21.15.1:0/535259319 conn(0x5558545d7db0 msgr2=0x55585467dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:07.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.382+0000 7f4943218b80 1 -- 172.21.15.1:0/535259319 shutdown_connections 2024-02-02T15:34:07.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.382+0000 7f4943218b80 1 -- 172.21.15.1:0/535259319 wait complete. 2024-02-02T15:34:07.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.382+0000 7f4943218b80 20 client.17448 trim_cache size 0 max 16384 2024-02-02T15:34:07.387 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:34:07.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.382+0000 7f492f600700 1 -- 172.21.15.1:0/696328939 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f49080a0690 msgr2=0x7f49080a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:07.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.382+0000 7f492f600700 1 --2- 172.21.15.1:0/696328939 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f49080a0690 0x7f49080a2b50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f492819eb10 tx=0x7f4910009260 comp rx=0 tx=0).stop 2024-02-02T15:34:07.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.382+0000 7f492f600700 1 -- 172.21.15.1:0/696328939 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49280730e0 msgr2=0x7f492819d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:07.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.382+0000 7f492f600700 1 --2- 172.21.15.1:0/696328939 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49280730e0 0x7f492819d500 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f4918005910 tx=0x7f4918005720 comp rx=0 tx=0).stop 2024-02-02T15:34:07.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.382+0000 7f492f600700 1 -- 172.21.15.1:0/696328939 shutdown_connections 2024-02-02T15:34:07.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.382+0000 7f492f600700 1 --2- 172.21.15.1:0/696328939 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f49080a0690 0x7f49080a2b50 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.382+0000 7f492f600700 1 --2- 172.21.15.1:0/696328939 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4928111d90 0x7f49281a1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.382+0000 7f492f600700 1 --2- 172.21.15.1:0/696328939 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f49280730e0 0x7f492819d500 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.382+0000 7f492f600700 1 --2- 172.21.15.1:0/696328939 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4928073a00 0x7f492819da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.382+0000 7f492f600700 1 -- 172.21.15.1:0/696328939 >> 172.21.15.1:0/696328939 conn(0x7f49280fc810 msgr2=0x7f49280fd480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:07.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.382+0000 7f492f600700 1 -- 172.21.15.1:0/696328939 shutdown_connections 2024-02-02T15:34:07.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.383+0000 7f492f600700 1 -- 172.21.15.1:0/696328939 wait complete. 2024-02-02T15:34:07.400 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-02T15:34:07.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.606+0000 7efd5fe00700 1 -- 172.21.15.1:0/3823115972 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd58101000 msgr2=0x7efd581013e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:07.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.606+0000 7efd5fe00700 1 --2- 172.21.15.1:0/3823115972 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd58101000 0x7efd581013e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7efd48009990 tx=0x7efd48009ca0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.607+0000 7efd5fe00700 1 -- 172.21.15.1:0/3823115972 shutdown_connections 2024-02-02T15:34:07.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.607+0000 7efd5fe00700 1 --2- 172.21.15.1:0/3823115972 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd58106120 0x7efd5810dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.607+0000 7efd5fe00700 1 --2- 172.21.15.1:0/3823115972 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd58101000 0x7efd581013e0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.607+0000 7efd5fe00700 1 --2- 172.21.15.1:0/3823115972 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efd581019b0 0x7efd581059f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.607+0000 7efd5fe00700 1 -- 172.21.15.1:0/3823115972 >> 172.21.15.1:0/3823115972 conn(0x7efd580fc860 msgr2=0x7efd580fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:07.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.608+0000 7efd5fe00700 1 -- 172.21.15.1:0/3823115972 shutdown_connections 2024-02-02T15:34:07.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.608+0000 7efd5fe00700 1 -- 172.21.15.1:0/3823115972 wait complete. 2024-02-02T15:34:07.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.610+0000 7efd5fe00700 1 Processor -- start 2024-02-02T15:34:07.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.610+0000 7efd5fe00700 1 -- start start 2024-02-02T15:34:07.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.611+0000 7efd5fe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd58101000 0x7efd5810a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:07.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.611+0000 7efd5fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd58106120 0x7efd5810abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:07.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.611+0000 7efd5fe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efd581a31c0 0x7efd581a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:07.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.611+0000 7efd5fe00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd581a3c70 con 0x7efd581a31c0 2024-02-02T15:34:07.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.611+0000 7efd5fe00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd581a3de0 con 0x7efd58101000 2024-02-02T15:34:07.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.611+0000 7efd5fe00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd581a3fa0 con 0x7efd58106120 2024-02-02T15:34:07.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.611+0000 7efd5d400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd58101000 0x7efd5810a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:07.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.611+0000 7efd5ca00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd58106120 0x7efd5810abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:07.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.611+0000 7efd5d400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd58101000 0x7efd5810a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:33826/0 (socket says 172.21.15.1:33826) 2024-02-02T15:34:07.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.611+0000 7efd5d400700 1 -- 172.21.15.1:0/3222039559 learned_addr learned my addr 172.21.15.1:0/3222039559 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:34:07.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.612+0000 7efd5d400700 1 -- 172.21.15.1:0/3222039559 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd58106120 msgr2=0x7efd5810abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:07.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.612+0000 7efd5d400700 1 --2- 172.21.15.1:0/3222039559 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd58106120 0x7efd5810abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.612+0000 7efd5d400700 1 -- 172.21.15.1:0/3222039559 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efd581a31c0 msgr2=0x7efd581a3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:07.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.612+0000 7efd5d400700 1 --2- 172.21.15.1:0/3222039559 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efd581a31c0 0x7efd581a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.612+0000 7efd5d400700 1 -- 172.21.15.1:0/3222039559 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd480095f0 con 0x7efd58101000 2024-02-02T15:34:07.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.613+0000 7efd5d400700 1 --2- 172.21.15.1:0/3222039559 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd58101000 0x7efd5810a640 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7efd48005910 tx=0x7efd48005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:07.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.613+0000 7efd56000700 1 -- 172.21.15.1:0/3222039559 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd4801d070 con 0x7efd58101000 2024-02-02T15:34:07.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.613+0000 7efd5fe00700 1 -- 172.21.15.1:0/3222039559 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd581aa2d0 con 0x7efd58101000 2024-02-02T15:34:07.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.614+0000 7efd5fe00700 1 -- 172.21.15.1:0/3222039559 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd581aa7a0 con 0x7efd58101000 2024-02-02T15:34:07.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.615+0000 7efd56000700 1 -- 172.21.15.1:0/3222039559 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd48022470 con 0x7efd58101000 2024-02-02T15:34:07.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.615+0000 7efd56000700 1 -- 172.21.15.1:0/3222039559 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd48017650 con 0x7efd58101000 2024-02-02T15:34:07.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.622+0000 7efd56000700 1 -- 172.21.15.1:0/3222039559 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7efd48017870 con 0x7efd58101000 2024-02-02T15:34:07.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.623+0000 7efd73931b80 1 Processor -- start 2024-02-02T15:34:07.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.623+0000 7efd73931b80 1 -- start start 2024-02-02T15:34:07.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.624+0000 7efd73931b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56128ee18010 0x56128ee183f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:07.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.624+0000 7efd73931b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56128ee18930 0x56128ee232a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:07.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.624+0000 7efd73931b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56128ee237e0 0x56128ee25bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:07.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.624+0000 7efd73931b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56128ed8c4c0 con 0x56128ee237e0 2024-02-02T15:34:07.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.624+0000 7efd73931b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56128ed70aa0 con 0x56128ee18010 2024-02-02T15:34:07.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.624+0000 7efd73931b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56128ed6e460 con 0x56128ee18930 2024-02-02T15:34:07.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.624+0000 7efd5d400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56128ee18930 0x56128ee232a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:07.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.624+0000 7efd5ca00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56128ee237e0 0x56128ee25bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:07.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.624+0000 7efd5de00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56128ee18010 0x56128ee183f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:07.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.624+0000 7efd5de00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56128ee18010 0x56128ee183f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:33838/0 (socket says 172.21.15.1:33838) 2024-02-02T15:34:07.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.624+0000 7efd5de00700 1 -- 172.21.15.1:0/2370495783 learned_addr learned my addr 172.21.15.1:0/2370495783 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:34:07.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.624+0000 7efd56000700 1 --2- 172.21.15.1:0/3222039559 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7efd380a0710 0x7efd380a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:07.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.625+0000 7efd5d400700 1 -- 172.21.15.1:0/2370495783 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56128ee18010 msgr2=0x56128ee183f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:07.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.625+0000 7efd5d400700 1 --2- 172.21.15.1:0/2370495783 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56128ee18010 0x56128ee183f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.625+0000 7efd5d400700 1 -- 172.21.15.1:0/2370495783 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56128ee237e0 msgr2=0x56128ee25bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:07.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.625+0000 7efd5d400700 1 --2- 172.21.15.1:0/2370495783 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56128ee237e0 0x56128ee25bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.625+0000 7efd5d400700 1 -- 172.21.15.1:0/2370495783 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56128ebeb3b0 con 0x56128ee18930 2024-02-02T15:34:07.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.625+0000 7efd5ca00700 1 --2- 172.21.15.1:0/2370495783 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56128ee237e0 0x56128ee25bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:34:07.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.625+0000 7efd5de00700 1 --2- 172.21.15.1:0/3222039559 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7efd380a0710 0x7efd380a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:07.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.625+0000 7efd56000700 1 -- 172.21.15.1:0/3222039559 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9541+0+0 (secure 0 0 0) 0x7efd480cd0a0 con 0x7efd58101000 2024-02-02T15:34:07.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.625+0000 7efd5d400700 1 --2- 172.21.15.1:0/2370495783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56128ee18930 0x56128ee232a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7efd480ca3c0 tx=0x7efd4802f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c9052e91c27e023 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:07.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.625+0000 7efd4e600700 1 -- 172.21.15.1:0/2370495783 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd4809a500 con 0x56128ee18930 2024-02-02T15:34:07.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.625+0000 7efd4e600700 1 -- 172.21.15.1:0/2370495783 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd4809a270 con 0x56128ee18930 2024-02-02T15:34:07.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.625+0000 7efd4e600700 1 -- 172.21.15.1:0/2370495783 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd4809a090 con 0x56128ee18930 2024-02-02T15:34:07.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.626+0000 7efd73931b80 1 -- 172.21.15.1:0/2370495783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56128ee18930 msgr2=0x56128ee232a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:07.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.626+0000 7efd73931b80 1 --2- 172.21.15.1:0/2370495783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56128ee18930 0x56128ee232a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7efd480ca3c0 tx=0x7efd4802f000 comp rx=0 tx=0).stop 2024-02-02T15:34:07.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.626+0000 7efd5de00700 1 --2- 172.21.15.1:0/3222039559 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7efd380a0710 0x7efd380a2bd0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7efd50009c10 tx=0x7efd50009920 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:07.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.627+0000 7efd73931b80 1 -- 172.21.15.1:0/2370495783 shutdown_connections 2024-02-02T15:34:07.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.627+0000 7efd73931b80 1 --2- 172.21.15.1:0/2370495783 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56128ee237e0 0x56128ee25bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.627+0000 7efd73931b80 1 --2- 172.21.15.1:0/2370495783 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56128ee18010 0x56128ee183f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.627+0000 7efd73931b80 1 --2- 172.21.15.1:0/2370495783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56128ee18930 0x56128ee232a0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.627+0000 7efd73931b80 1 -- 172.21.15.1:0/2370495783 >> 172.21.15.1:0/2370495783 conn(0x56128ed77db0 msgr2=0x56128ee143b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:07.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.627+0000 7efd73931b80 1 -- 172.21.15.1:0/2370495783 shutdown_connections 2024-02-02T15:34:07.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.627+0000 7efd73931b80 1 -- 172.21.15.1:0/2370495783 wait complete. 2024-02-02T15:34:07.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.628+0000 7efd73931b80 1 -- start start 2024-02-02T15:34:07.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.628+0000 7efd73931b80 1 Processor -- start 2024-02-02T15:34:07.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.629+0000 7efd73931b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56128ee18010 0x56128ee28840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:07.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.629+0000 7efd73931b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56128ee18930 0x56128ee28d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:07.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.629+0000 7efd73931b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56128ee237e0 0x56128ee226f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:07.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.629+0000 7efd73931b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56128ed69d50 con 0x56128ee18930 2024-02-02T15:34:07.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.629+0000 7efd73931b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56128ed05d80 con 0x56128ee237e0 2024-02-02T15:34:07.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.629+0000 7efd73931b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56128ed05a20 con 0x56128ee18010 2024-02-02T15:34:07.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.629+0000 7efd73931b80 10 client.0 _ll_register_callbacks cb 0x56128ed07290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:34:07.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.629+0000 7efd5de00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56128ee237e0 0x56128ee226f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:07.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.629+0000 7efd5de00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56128ee237e0 0x56128ee226f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:33852/0 (socket says 172.21.15.1:33852) 2024-02-02T15:34:07.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.629+0000 7efd5de00700 1 -- 172.21.15.1:0/3455014724 learned_addr learned my addr 172.21.15.1:0/3455014724 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:34:07.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.630+0000 7efd5ca00700 1 --2- 172.21.15.1:0/3455014724 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56128ee18930 0x56128ee28d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:07.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.630+0000 7efd5d400700 1 --2- 172.21.15.1:0/3455014724 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56128ee18010 0x56128ee28840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:07.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.630+0000 7efd5ca00700 1 -- 172.21.15.1:0/3455014724 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56128ee18010 msgr2=0x56128ee28840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:07.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.630+0000 7efd5ca00700 1 --2- 172.21.15.1:0/3455014724 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56128ee18010 0x56128ee28840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.630+0000 7efd5ca00700 1 -- 172.21.15.1:0/3455014724 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56128ee237e0 msgr2=0x56128ee226f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:07.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.630+0000 7efd5ca00700 1 --2- 172.21.15.1:0/3455014724 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56128ee237e0 0x56128ee226f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.630+0000 7efd5ca00700 1 -- 172.21.15.1:0/3455014724 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd400098e0 con 0x56128ee18930 2024-02-02T15:34:07.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.630+0000 7efd5d400700 1 --2- 172.21.15.1:0/3455014724 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56128ee18010 0x56128ee28840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:34:07.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.630+0000 7efd5ca00700 1 --2- 172.21.15.1:0/3455014724 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56128ee18930 0x56128ee28d80 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7efd5806a340 tx=0x7efd40012c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=554a750e416d9fc7 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:07.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.630+0000 7efd46a00700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:34:07.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.631+0000 7efd46a00700 1 -- 172.21.15.1:0/3455014724 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd40017070 con 0x56128ee18930 2024-02-02T15:34:07.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.631+0000 7efd46a00700 1 -- 172.21.15.1:0/3455014724 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9541+0+0 (secure 0 0 0) 0x7efd40004030 con 0x56128ee18930 2024-02-02T15:34:07.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.631+0000 7efd73931b80 1 -- 172.21.15.1:0/3455014724 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56128ee0c0f0 con 0x56128ee18930 2024-02-02T15:34:07.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.631+0000 7efd46a00700 1 -- 172.21.15.1:0/3455014724 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7efd400041f0 con 0x56128ee18930 2024-02-02T15:34:07.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.631+0000 7efd73931b80 10 client.17457 fetch_fsmap learned FSMap version 40 2024-02-02T15:34:07.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.631+0000 7efd73931b80 1 -- 172.21.15.1:0/3455014724 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x7efd581aa7a0 con 0x56128ee18930 2024-02-02T15:34:07.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.632+0000 7efd46a00700 1 -- 172.21.15.1:0/3455014724 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56128ee0c0f0 con 0x56128ee18930 2024-02-02T15:34:07.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.632+0000 7efd46a00700 1 -- 172.21.15.1:0/3455014724 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 40) v1 ==== 7502+0+0 (secure 0 0 0) 0x7efd4000f690 con 0x56128ee18930 2024-02-02T15:34:07.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.632+0000 7efd73931b80 10 client.17457 fetch_fsmap finished waiting for FSMap version 40 2024-02-02T15:34:07.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.632+0000 7efd73931b80 10 client.17457 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-02-02T15:34:07.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.632+0000 7efd73931b80 20 client.17457 populate_metadata read hostname 'smithi001' 2024-02-02T15:34:07.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.632+0000 7efd73931b80 1 --2- 172.21.15.1:0/3455014724 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x56128ee32d00 0x56128ee35120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:07.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.632+0000 7efd73931b80 4 client.17457 mds_command: new command op to 24518 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:34:07.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.632+0000 7efd73931b80 1 -- 172.21.15.1:0/3455014724 --> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56128ed62dd0 con 0x56128ee32d00 2024-02-02T15:34:07.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.632+0000 7efd5d400700 1 --2- 172.21.15.1:0/3455014724 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x56128ee32d00 0x56128ee35120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:07.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.633+0000 7efd5d400700 1 --2- 172.21.15.1:0/3455014724 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x56128ee32d00 0x56128ee35120 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=77348fa6dab5bdcb server_cookie=9108dbb3f4fefe58 in_seq=0 out_seq=0 2024-02-02T15:34:07.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.633+0000 7efd46a00700 10 client.17457 ms_handle_connect on v2:172.21.15.71:6836/628016199 2024-02-02T15:34:07.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.634+0000 7efd46a00700 1 -- 172.21.15.1:0/3455014724 <== mds.2 v2:172.21.15.71:6836/628016199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x56128ed62dd0 con 0x56128ee32d00 2024-02-02T15:34:07.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.634+0000 7efd46a00700 10 client.17457 handle_command_reply: tid=0 2024-02-02T15:34:07.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.634+0000 7efd73931b80 1 client.17457 shutdown 2024-02-02T15:34:07.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.634+0000 7efd73931b80 1 -- 172.21.15.1:0/3455014724 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56128ee18930 msgr2=0x56128ee28d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:07.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.634+0000 7efd73931b80 1 --2- 172.21.15.1:0/3455014724 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56128ee18930 0x56128ee28d80 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7efd5806a340 tx=0x7efd40012c30 comp rx=0 tx=0).stop 2024-02-02T15:34:07.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.634+0000 7efd73931b80 1 -- 172.21.15.1:0/3455014724 shutdown_connections 2024-02-02T15:34:07.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.634+0000 7efd73931b80 1 --2- 172.21.15.1:0/3455014724 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56128ee237e0 0x56128ee226f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.634+0000 7efd73931b80 1 --2- 172.21.15.1:0/3455014724 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x56128ee32d00 0x56128ee35120 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.634+0000 7efd73931b80 1 --2- 172.21.15.1:0/3455014724 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56128ee18010 0x56128ee28840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.634+0000 7efd73931b80 1 --2- 172.21.15.1:0/3455014724 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56128ee18930 0x56128ee28d80 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.634+0000 7efd46a00700 0 client.17457 ms_handle_reset on v2:172.21.15.71:6836/628016199 2024-02-02T15:34:07.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.634+0000 7efd73931b80 1 -- 172.21.15.1:0/3455014724 >> 172.21.15.1:0/3455014724 conn(0x56128ed77db0 msgr2=0x56128ee13fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:07.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.635+0000 7efd73931b80 1 -- 172.21.15.1:0/3455014724 shutdown_connections 2024-02-02T15:34:07.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.635+0000 7efd73931b80 1 -- 172.21.15.1:0/3455014724 wait complete. 2024-02-02T15:34:07.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.635+0000 7efd73931b80 20 client.17457 trim_cache size 0 max 16384 2024-02-02T15:34:07.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.659+0000 7efd73931b80 1 Processor -- start 2024-02-02T15:34:07.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.659+0000 7efd73931b80 1 -- start start 2024-02-02T15:34:07.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.660+0000 7efd73931b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56128ee18010 0x56128ee28410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:07.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.660+0000 7efd73931b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56128ee18930 0x56128ee13cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:07.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.660+0000 7efd73931b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56128ee237e0 0x56128ee14210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:07.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.660+0000 7efd73931b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56128ebed6a0 con 0x56128ee18930 2024-02-02T15:34:07.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.660+0000 7efd73931b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56128ee23ec0 con 0x56128ee18010 2024-02-02T15:34:07.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.660+0000 7efd73931b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56128ee0e500 con 0x56128ee237e0 2024-02-02T15:34:07.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.660+0000 7efd5de00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56128ee237e0 0x56128ee14210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:07.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.660+0000 7efd5de00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56128ee237e0 0x56128ee14210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:39194/0 (socket says 172.21.15.1:39194) 2024-02-02T15:34:07.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.660+0000 7efd5de00700 1 -- 172.21.15.1:0/2540412740 learned_addr learned my addr 172.21.15.1:0/2540412740 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:34:07.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.661+0000 7efd5ca00700 1 --2- 172.21.15.1:0/2540412740 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56128ee18930 0x56128ee13cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:07.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.661+0000 7efd5d400700 1 --2- 172.21.15.1:0/2540412740 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56128ee18010 0x56128ee28410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:07.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.661+0000 7efd5ca00700 1 -- 172.21.15.1:0/2540412740 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56128ee237e0 msgr2=0x56128ee14210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:07.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.661+0000 7efd5ca00700 1 --2- 172.21.15.1:0/2540412740 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56128ee237e0 0x56128ee14210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.661+0000 7efd5ca00700 1 -- 172.21.15.1:0/2540412740 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56128ee18010 msgr2=0x56128ee28410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:07.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.661+0000 7efd5ca00700 1 --2- 172.21.15.1:0/2540412740 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56128ee18010 0x56128ee28410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.661+0000 7efd5ca00700 1 -- 172.21.15.1:0/2540412740 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56128eb97b60 con 0x56128ee18930 2024-02-02T15:34:07.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.662+0000 7efd5ca00700 1 --2- 172.21.15.1:0/2540412740 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56128ee18930 0x56128ee13cd0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7efd40000c00 tx=0x7efd4000fb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28eb5cc44481c9ed server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:07.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.662+0000 7efd4f000700 1 -- 172.21.15.1:0/2540412740 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd40017040 con 0x56128ee18930 2024-02-02T15:34:07.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.662+0000 7efd4f000700 1 -- 172.21.15.1:0/2540412740 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd4000fcc0 con 0x56128ee18930 2024-02-02T15:34:07.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.663+0000 7efd4f000700 1 -- 172.21.15.1:0/2540412740 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd40017040 con 0x56128ee18930 2024-02-02T15:34:07.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.664+0000 7efd73931b80 1 -- 172.21.15.1:0/2540412740 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56128ee18930 msgr2=0x56128ee13cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:07.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.664+0000 7efd73931b80 1 --2- 172.21.15.1:0/2540412740 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56128ee18930 0x56128ee13cd0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7efd40000c00 tx=0x7efd4000fb70 comp rx=0 tx=0).stop 2024-02-02T15:34:07.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.664+0000 7efd73931b80 1 -- 172.21.15.1:0/2540412740 shutdown_connections 2024-02-02T15:34:07.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.664+0000 7efd73931b80 1 --2- 172.21.15.1:0/2540412740 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56128ee237e0 0x56128ee14210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.664+0000 7efd73931b80 1 --2- 172.21.15.1:0/2540412740 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56128ee18010 0x56128ee28410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.664+0000 7efd73931b80 1 --2- 172.21.15.1:0/2540412740 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56128ee18930 0x56128ee13cd0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.664+0000 7efd73931b80 1 -- 172.21.15.1:0/2540412740 >> 172.21.15.1:0/2540412740 conn(0x56128ed77db0 msgr2=0x56128ee135c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:07.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.664+0000 7efd73931b80 1 -- 172.21.15.1:0/2540412740 shutdown_connections 2024-02-02T15:34:07.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.665+0000 7efd73931b80 1 -- 172.21.15.1:0/2540412740 wait complete. 2024-02-02T15:34:07.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.665+0000 7efd73931b80 1 -- start start 2024-02-02T15:34:07.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.665+0000 7efd73931b80 1 Processor -- start 2024-02-02T15:34:07.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.666+0000 7efd73931b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56128ee18010 0x56128ee789a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:07.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.666+0000 7efd73931b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56128ee18930 0x56128ee78ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:07.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.666+0000 7efd73931b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56128ee237e0 0x56128ee83830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:07.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.666+0000 7efd73931b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56128ee795b0 con 0x56128ee18930 2024-02-02T15:34:07.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.666+0000 7efd73931b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56128ee83d70 con 0x56128ee18010 2024-02-02T15:34:07.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.666+0000 7efd73931b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56128ee83eb0 con 0x56128ee237e0 2024-02-02T15:34:07.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.666+0000 7efd73931b80 10 client.0 _ll_register_callbacks cb 0x56128ed07290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:34:07.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.666+0000 7efd5d400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56128ee18010 0x56128ee789a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:07.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.666+0000 7efd5d400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56128ee18010 0x56128ee789a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:33872/0 (socket says 172.21.15.1:33872) 2024-02-02T15:34:07.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.666+0000 7efd5d400700 1 -- 172.21.15.1:0/869422781 learned_addr learned my addr 172.21.15.1:0/869422781 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:34:07.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.666+0000 7efd5de00700 1 --2- 172.21.15.1:0/869422781 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56128ee237e0 0x56128ee83830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:07.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.667+0000 7efd5ca00700 1 --2- 172.21.15.1:0/869422781 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56128ee18930 0x56128ee78ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:07.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.667+0000 7efd5d400700 1 -- 172.21.15.1:0/869422781 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56128ee237e0 msgr2=0x56128ee83830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:07.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.667+0000 7efd5d400700 1 --2- 172.21.15.1:0/869422781 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56128ee237e0 0x56128ee83830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.667+0000 7efd5d400700 1 -- 172.21.15.1:0/869422781 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56128ee18930 msgr2=0x56128ee78ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:07.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.667+0000 7efd5d400700 1 --2- 172.21.15.1:0/869422781 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56128ee18930 0x56128ee78ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.667+0000 7efd5d400700 1 -- 172.21.15.1:0/869422781 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd581aa7a0 con 0x56128ee18010 2024-02-02T15:34:07.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.667+0000 7efd5de00700 1 --2- 172.21.15.1:0/869422781 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56128ee237e0 0x56128ee83830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:34:07.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.667+0000 7efd5ca00700 1 --2- 172.21.15.1:0/869422781 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56128ee18930 0x56128ee78ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:34:07.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.667+0000 7efd5d400700 1 --2- 172.21.15.1:0/869422781 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56128ee18010 0x56128ee789a0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7efd480ca9f0 tx=0x7efd48009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9000229fb1ee918 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:07.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.667+0000 7efd46a00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:34:07.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.667+0000 7efd46a00700 1 -- 172.21.15.1:0/869422781 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd4809a090 con 0x56128ee18010 2024-02-02T15:34:07.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.668+0000 7efd46a00700 1 -- 172.21.15.1:0/869422781 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9541+0+0 (secure 0 0 0) 0x7efd4800ce70 con 0x56128ee18010 2024-02-02T15:34:07.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.668+0000 7efd73931b80 1 -- 172.21.15.1:0/869422781 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efd4809a270 con 0x56128ee18010 2024-02-02T15:34:07.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.668+0000 7efd46a00700 1 -- 172.21.15.1:0/869422781 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7efd4809a500 con 0x56128ee18010 2024-02-02T15:34:07.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.668+0000 7efd73931b80 10 client.27536 fetch_fsmap learned FSMap version 40 2024-02-02T15:34:07.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.668+0000 7efd73931b80 1 -- 172.21.15.1:0/869422781 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x56128ed2fea0 con 0x56128ee18010 2024-02-02T15:34:07.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.668+0000 7efd46a00700 1 -- 172.21.15.1:0/869422781 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efd4809a270 con 0x56128ee18010 2024-02-02T15:34:07.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.668+0000 7efd46a00700 1 -- 172.21.15.1:0/869422781 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 40) v1 ==== 7502+0+0 (secure 0 0 0) 0x7efd4809e150 con 0x56128ee18010 2024-02-02T15:34:07.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.668+0000 7efd73931b80 10 client.27536 fetch_fsmap finished waiting for FSMap version 40 2024-02-02T15:34:07.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.668+0000 7efd73931b80 10 client.27536 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-02-02T15:34:07.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.668+0000 7efd73931b80 20 client.27536 populate_metadata read hostname 'smithi001' 2024-02-02T15:34:07.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.668+0000 7efd73931b80 1 --2- 172.21.15.1:0/869422781 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x56128ee85130 0x56128ee87550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:07.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.668+0000 7efd73931b80 4 client.27536 mds_command: new command op to 24518 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:34:07.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.668+0000 7efd73931b80 1 -- 172.21.15.1:0/869422781 --> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56128ed28f50 con 0x56128ee85130 2024-02-02T15:34:07.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.668+0000 7efd5ca00700 1 --2- 172.21.15.1:0/869422781 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x56128ee85130 0x56128ee87550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:07.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:07 smithi142 ceph-mon[45728]: pgmap v217: 129 pgs: 129 active+clean; 289 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 193 op/s 2024-02-02T15:34:07.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:07 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/3709274727' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:34:07.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.669+0000 7efd5ca00700 1 --2- 172.21.15.1:0/869422781 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x56128ee85130 0x56128ee87550 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3257dd1503885e02 server_cookie=31e4cbd880736c8f in_seq=0 out_seq=0 2024-02-02T15:34:07.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.669+0000 7efd46a00700 10 client.27536 ms_handle_connect on v2:172.21.15.71:6836/628016199 2024-02-02T15:34:07.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.669+0000 7efd46a00700 1 -- 172.21.15.1:0/869422781 <== mds.2 v2:172.21.15.71:6836/628016199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56128ed28f50 con 0x56128ee85130 2024-02-02T15:34:07.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.669+0000 7efd46a00700 10 client.27536 handle_command_reply: tid=0 2024-02-02T15:34:07.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.669+0000 7efd73931b80 1 client.27536 shutdown 2024-02-02T15:34:07.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.669+0000 7efd73931b80 1 -- 172.21.15.1:0/869422781 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56128ee18010 msgr2=0x56128ee789a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:07.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.669+0000 7efd73931b80 1 --2- 172.21.15.1:0/869422781 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56128ee18010 0x56128ee789a0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7efd480ca9f0 tx=0x7efd48009310 comp rx=0 tx=0).stop 2024-02-02T15:34:07.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.669+0000 7efd73931b80 1 -- 172.21.15.1:0/869422781 shutdown_connections 2024-02-02T15:34:07.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.669+0000 7efd73931b80 1 --2- 172.21.15.1:0/869422781 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x56128ee85130 0x56128ee87550 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.669+0000 7efd73931b80 1 --2- 172.21.15.1:0/869422781 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56128ee237e0 0x56128ee83830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.669+0000 7efd73931b80 1 --2- 172.21.15.1:0/869422781 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56128ee18010 0x56128ee789a0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.669+0000 7efd46a00700 0 client.27536 ms_handle_reset on v2:172.21.15.71:6836/628016199 2024-02-02T15:34:07.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.669+0000 7efd73931b80 1 --2- 172.21.15.1:0/869422781 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56128ee18930 0x56128ee78ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.669+0000 7efd73931b80 1 -- 172.21.15.1:0/869422781 >> 172.21.15.1:0/869422781 conn(0x56128ed77db0 msgr2=0x56128ee13e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:07.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.669+0000 7efd73931b80 1 -- 172.21.15.1:0/869422781 shutdown_connections 2024-02-02T15:34:07.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.669+0000 7efd73931b80 1 -- 172.21.15.1:0/869422781 wait complete. 2024-02-02T15:34:07.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.670+0000 7efd73931b80 20 client.27536 trim_cache size 0 max 16384 2024-02-02T15:34:07.674 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:34:07.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.670+0000 7efd5fe00700 1 -- 172.21.15.1:0/3222039559 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7efd380a0710 msgr2=0x7efd380a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:07.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.670+0000 7efd5fe00700 1 --2- 172.21.15.1:0/3222039559 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7efd380a0710 0x7efd380a2bd0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7efd50009c10 tx=0x7efd50009920 comp rx=0 tx=0).stop 2024-02-02T15:34:07.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.670+0000 7efd5fe00700 1 -- 172.21.15.1:0/3222039559 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd58101000 msgr2=0x7efd5810a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:07.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.670+0000 7efd5fe00700 1 --2- 172.21.15.1:0/3222039559 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd58101000 0x7efd5810a640 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7efd48005910 tx=0x7efd48005740 comp rx=0 tx=0).stop 2024-02-02T15:34:07.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.670+0000 7efd5fe00700 1 -- 172.21.15.1:0/3222039559 shutdown_connections 2024-02-02T15:34:07.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.670+0000 7efd5fe00700 1 --2- 172.21.15.1:0/3222039559 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7efd380a0710 0x7efd380a2bd0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.670+0000 7efd5fe00700 1 --2- 172.21.15.1:0/3222039559 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efd581a31c0 0x7efd581a3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.670+0000 7efd5fe00700 1 --2- 172.21.15.1:0/3222039559 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd58101000 0x7efd5810a640 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.670+0000 7efd5fe00700 1 --2- 172.21.15.1:0/3222039559 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efd58106120 0x7efd5810abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.670+0000 7efd5fe00700 1 -- 172.21.15.1:0/3222039559 >> 172.21.15.1:0/3222039559 conn(0x7efd580fc860 msgr2=0x7efd581118d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:07.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.670+0000 7efd5fe00700 1 -- 172.21.15.1:0/3222039559 shutdown_connections 2024-02-02T15:34:07.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.671+0000 7efd5fe00700 1 -- 172.21.15.1:0/3222039559 wait complete. 2024-02-02T15:34:07.686 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:07 smithi001 ceph-mon[35470]: pgmap v217: 129 pgs: 129 active+clean; 289 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 193 op/s 2024-02-02T15:34:07.686 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:07 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/3709274727' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:34:07.688 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-02T15:34:07.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:07 smithi071 ceph-mon[44707]: pgmap v217: 129 pgs: 129 active+clean; 289 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 193 op/s 2024-02-02T15:34:07.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:07 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/3709274727' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:34:07.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.894+0000 7fbfe7000700 1 -- 172.21.15.1:0/3399043057 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbfe0106110 msgr2=0x7fbfe010dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:07.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.894+0000 7fbfe7000700 1 --2- 172.21.15.1:0/3399043057 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbfe0106110 0x7fbfe010dc90 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fbfd800b0a0 tx=0x7fbfd800b3b0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.894+0000 7fbfe7000700 1 -- 172.21.15.1:0/3399043057 shutdown_connections 2024-02-02T15:34:07.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.894+0000 7fbfe7000700 1 --2- 172.21.15.1:0/3399043057 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbfe0106110 0x7fbfe010dc90 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.894+0000 7fbfe7000700 1 --2- 172.21.15.1:0/3399043057 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbfe0100ff0 0x7fbfe01013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.894+0000 7fbfe7000700 1 --2- 172.21.15.1:0/3399043057 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbfe01019a0 0x7fbfe01059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.894+0000 7fbfe7000700 1 -- 172.21.15.1:0/3399043057 >> 172.21.15.1:0/3399043057 conn(0x7fbfe00fc810 msgr2=0x7fbfe00fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:07.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.894+0000 7fbfe7000700 1 -- 172.21.15.1:0/3399043057 shutdown_connections 2024-02-02T15:34:07.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.894+0000 7fbfe7000700 1 -- 172.21.15.1:0/3399043057 wait complete. 2024-02-02T15:34:07.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.895+0000 7fbfe7000700 1 Processor -- start 2024-02-02T15:34:07.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.895+0000 7fbfe7000700 1 -- start start 2024-02-02T15:34:07.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.895+0000 7fbfe7000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbfe0100ff0 0x7fbfe0073040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:07.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.895+0000 7fbfe7000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbfe0106110 0x7fbfe0073580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:07.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.895+0000 7fbfe7000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbfe0073ac0 0x7fbfe01a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:07.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.895+0000 7fbfe7000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe01a5840 con 0x7fbfe0106110 2024-02-02T15:34:07.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.895+0000 7fbfe7000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe01a59b0 con 0x7fbfe0100ff0 2024-02-02T15:34:07.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.895+0000 7fbfe7000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe01a5b70 con 0x7fbfe0073ac0 2024-02-02T15:34:07.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.896+0000 7fbfd7e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbfe0106110 0x7fbfe0073580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:07.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.896+0000 7fbfdfe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbfe0100ff0 0x7fbfe0073040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:07.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.896+0000 7fbfd7e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbfe0106110 0x7fbfe0073580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48778/0 (socket says 172.21.15.1:48778) 2024-02-02T15:34:07.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.896+0000 7fbfd7e00700 1 -- 172.21.15.1:0/2277978806 learned_addr learned my addr 172.21.15.1:0/2277978806 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:34:07.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.896+0000 7fbfe5000700 1 --2- 172.21.15.1:0/2277978806 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbfe0073ac0 0x7fbfe01a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:07.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.896+0000 7fbfdfe00700 1 -- 172.21.15.1:0/2277978806 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbfe0073ac0 msgr2=0x7fbfe01a51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:07.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.896+0000 7fbfdfe00700 1 --2- 172.21.15.1:0/2277978806 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbfe0073ac0 0x7fbfe01a51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.896+0000 7fbfdfe00700 1 -- 172.21.15.1:0/2277978806 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbfe0106110 msgr2=0x7fbfe0073580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:07.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.896+0000 7fbfdfe00700 1 --2- 172.21.15.1:0/2277978806 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbfe0106110 0x7fbfe0073580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.896+0000 7fbfdfe00700 1 -- 172.21.15.1:0/2277978806 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfd8009d00 con 0x7fbfe0100ff0 2024-02-02T15:34:07.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.896+0000 7fbfdfe00700 1 --2- 172.21.15.1:0/2277978806 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbfe0100ff0 0x7fbfe0073040 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fbfd000e9f0 tx=0x7fbfd000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:07.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.897+0000 7fbfdd600700 1 -- 172.21.15.1:0/2277978806 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd000c9c0 con 0x7fbfe0100ff0 2024-02-02T15:34:07.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.897+0000 7fbfdd600700 1 -- 172.21.15.1:0/2277978806 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbfd0010430 con 0x7fbfe0100ff0 2024-02-02T15:34:07.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.897+0000 7fbfe7000700 1 -- 172.21.15.1:0/2277978806 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfe01a5e50 con 0x7fbfe0100ff0 2024-02-02T15:34:07.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.897+0000 7fbfdd600700 1 -- 172.21.15.1:0/2277978806 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd0018680 con 0x7fbfe0100ff0 2024-02-02T15:34:07.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.897+0000 7fbfe7000700 1 -- 172.21.15.1:0/2277978806 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfe01a6320 con 0x7fbfe0100ff0 2024-02-02T15:34:07.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.899+0000 7fbfdd600700 1 -- 172.21.15.1:0/2277978806 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7fbfd0010860 con 0x7fbfe0100ff0 2024-02-02T15:34:07.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.901+0000 7fbfdd600700 1 --2- 172.21.15.1:0/2277978806 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fbfc00a0710 0x7fbfc00a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:07.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.901+0000 7fbffaaf4b80 1 Processor -- start 2024-02-02T15:34:07.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.901+0000 7fbfdd600700 1 -- 172.21.15.1:0/2277978806 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9541+0+0 (secure 0 0 0) 0x7fbfd0014070 con 0x7fbfe0100ff0 2024-02-02T15:34:07.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.901+0000 7fbfd7e00700 1 --2- 172.21.15.1:0/2277978806 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fbfc00a0710 0x7fbfc00a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:07.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.901+0000 7fbffaaf4b80 1 -- start start 2024-02-02T15:34:07.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.901+0000 7fbffaaf4b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eeeeabb010 0x55eeeeabb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:07.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.902+0000 7fbffaaf4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eeeeabb930 0x55eeeeac62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:07.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.902+0000 7fbffaaf4b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eeeeac67e0 0x55eeeeac8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:07.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.902+0000 7fbffaaf4b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eeeea2f4c0 con 0x55eeeeac67e0 2024-02-02T15:34:07.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.902+0000 7fbffaaf4b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eeeea13aa0 con 0x55eeeeabb010 2024-02-02T15:34:07.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.902+0000 7fbffaaf4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eeeea11460 con 0x55eeeeabb930 2024-02-02T15:34:07.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.902+0000 7fbfdfe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eeeeabb010 0x55eeeeabb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:07.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.902+0000 7fbfdfe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eeeeabb010 0x55eeeeabb3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:33892/0 (socket says 172.21.15.1:33892) 2024-02-02T15:34:07.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.902+0000 7fbfdfe00700 1 -- 172.21.15.1:0/2901454762 learned_addr learned my addr 172.21.15.1:0/2901454762 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:34:07.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.902+0000 7fbfd7e00700 1 --2- 172.21.15.1:0/2277978806 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fbfc00a0710 0x7fbfc00a2bd0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbfe0074830 tx=0x7fbfc8009260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:07.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.902+0000 7fbfe5000700 1 --2- 172.21.15.1:0/2901454762 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eeeeac67e0 0x55eeeeac8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:07.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.902+0000 7fbfd7e00700 1 --2- 172.21.15.1:0/2901454762 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eeeeabb930 0x55eeeeac62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:07.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.902+0000 7fbfdfe00700 1 -- 172.21.15.1:0/2901454762 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eeeeabb930 msgr2=0x55eeeeac62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:07.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.902+0000 7fbfdfe00700 1 --2- 172.21.15.1:0/2901454762 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eeeeabb930 0x55eeeeac62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.902+0000 7fbfdfe00700 1 -- 172.21.15.1:0/2901454762 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eeeeac67e0 msgr2=0x55eeeeac8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:07.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.902+0000 7fbfdfe00700 1 --2- 172.21.15.1:0/2901454762 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eeeeac67e0 0x55eeeeac8bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.902+0000 7fbfdfe00700 1 -- 172.21.15.1:0/2901454762 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eeee88e3b0 con 0x55eeeeabb010 2024-02-02T15:34:07.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.902+0000 7fbfe5000700 1 --2- 172.21.15.1:0/2901454762 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eeeeac67e0 0x55eeeeac8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:34:07.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.902+0000 7fbfdfe00700 1 --2- 172.21.15.1:0/2901454762 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eeeeabb010 0x55eeeeabb3f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fbfd000e9c0 tx=0x7fbfd0098d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3945ac9b3578357f server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:07.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.903+0000 7fbfd5600700 1 -- 172.21.15.1:0/2901454762 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd009d150 con 0x55eeeeabb010 2024-02-02T15:34:07.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.903+0000 7fbfd5600700 1 -- 172.21.15.1:0/2901454762 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbfd00a1390 con 0x55eeeeabb010 2024-02-02T15:34:07.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.903+0000 7fbfd5600700 1 -- 172.21.15.1:0/2901454762 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd0098810 con 0x55eeeeabb010 2024-02-02T15:34:07.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.903+0000 7fbffaaf4b80 1 -- 172.21.15.1:0/2901454762 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eeeeabb010 msgr2=0x55eeeeabb3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:07.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.903+0000 7fbffaaf4b80 1 --2- 172.21.15.1:0/2901454762 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eeeeabb010 0x55eeeeabb3f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fbfd000e9c0 tx=0x7fbfd0098d00 comp rx=0 tx=0).stop 2024-02-02T15:34:07.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.903+0000 7fbffaaf4b80 1 -- 172.21.15.1:0/2901454762 shutdown_connections 2024-02-02T15:34:07.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.903+0000 7fbffaaf4b80 1 --2- 172.21.15.1:0/2901454762 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eeeeac67e0 0x55eeeeac8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.903+0000 7fbffaaf4b80 1 --2- 172.21.15.1:0/2901454762 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eeeeabb010 0x55eeeeabb3f0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.903+0000 7fbffaaf4b80 1 --2- 172.21.15.1:0/2901454762 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eeeeabb930 0x55eeeeac62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.903+0000 7fbffaaf4b80 1 -- 172.21.15.1:0/2901454762 >> 172.21.15.1:0/2901454762 conn(0x55eeeea1adb0 msgr2=0x55eeeeab73b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:07.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.903+0000 7fbffaaf4b80 1 -- 172.21.15.1:0/2901454762 shutdown_connections 2024-02-02T15:34:07.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.904+0000 7fbffaaf4b80 1 -- 172.21.15.1:0/2901454762 wait complete. 2024-02-02T15:34:07.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.904+0000 7fbffaaf4b80 1 -- start start 2024-02-02T15:34:07.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.904+0000 7fbffaaf4b80 1 Processor -- start 2024-02-02T15:34:07.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.905+0000 7fbffaaf4b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eeeeabb010 0x55eeeeac6230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:07.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.905+0000 7fbffaaf4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eeeeabb930 0x55eeeeac3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:07.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.905+0000 7fbffaaf4b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eeeeac67e0 0x55eeeeac3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:07.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.905+0000 7fbffaaf4b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eeeea0cd50 con 0x55eeeeac67e0 2024-02-02T15:34:07.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.905+0000 7fbffaaf4b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eeee9a8d80 con 0x55eeeeabb010 2024-02-02T15:34:07.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.905+0000 7fbffaaf4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eeee9a8a20 con 0x55eeeeabb930 2024-02-02T15:34:07.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.905+0000 7fbffaaf4b80 10 client.0 _ll_register_callbacks cb 0x55eeee9aa290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:34:07.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.906+0000 7fbfdfe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eeeeabb010 0x55eeeeac6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:07.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.906+0000 7fbfdfe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eeeeabb010 0x55eeeeac6230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:33896/0 (socket says 172.21.15.1:33896) 2024-02-02T15:34:07.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.906+0000 7fbfdfe00700 1 -- 172.21.15.1:0/1544621067 learned_addr learned my addr 172.21.15.1:0/1544621067 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:34:07.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.906+0000 7fbfd7e00700 1 --2- 172.21.15.1:0/1544621067 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eeeeabb930 0x55eeeeac3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:07.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.906+0000 7fbfe5000700 1 --2- 172.21.15.1:0/1544621067 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eeeeac67e0 0x55eeeeac3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:07.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.906+0000 7fbfdfe00700 1 -- 172.21.15.1:0/1544621067 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eeeeabb930 msgr2=0x55eeeeac3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:07.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.906+0000 7fbfdfe00700 1 --2- 172.21.15.1:0/1544621067 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eeeeabb930 0x55eeeeac3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.906+0000 7fbfdfe00700 1 -- 172.21.15.1:0/1544621067 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eeeeac67e0 msgr2=0x55eeeeac3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:07.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.906+0000 7fbfdfe00700 1 --2- 172.21.15.1:0/1544621067 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eeeeac67e0 0x55eeeeac3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.906+0000 7fbfdfe00700 1 -- 172.21.15.1:0/1544621067 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfe01a6320 con 0x55eeeeabb010 2024-02-02T15:34:07.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.906+0000 7fbfdfe00700 1 --2- 172.21.15.1:0/1544621067 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eeeeabb010 0x55eeeeac6230 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fbfd0095820 tx=0x7fbfd00107d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89a16e29e40baedc server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:07.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.906+0000 7fbfce000700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:34:07.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.906+0000 7fbfce000700 1 -- 172.21.15.1:0/1544621067 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd009d1d0 con 0x55eeeeabb010 2024-02-02T15:34:07.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.906+0000 7fbfce000700 1 -- 172.21.15.1:0/1544621067 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9541+0+0 (secure 0 0 0) 0x7fbfd00976c0 con 0x55eeeeabb010 2024-02-02T15:34:07.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.906+0000 7fbffaaf4b80 1 -- 172.21.15.1:0/1544621067 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55eeeeaaf0f0 con 0x55eeeeabb010 2024-02-02T15:34:07.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.907+0000 7fbfce000700 1 -- 172.21.15.1:0/1544621067 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbfd0098a20 con 0x55eeeeabb010 2024-02-02T15:34:07.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.907+0000 7fbffaaf4b80 10 client.27548 fetch_fsmap learned FSMap version 40 2024-02-02T15:34:07.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.907+0000 7fbffaaf4b80 1 -- 172.21.15.1:0/1544621067 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55eeee83ab60 con 0x55eeeeabb010 2024-02-02T15:34:07.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.907+0000 7fbfce000700 1 -- 172.21.15.1:0/1544621067 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55eeeeaaf0f0 con 0x55eeeeabb010 2024-02-02T15:34:07.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.907+0000 7fbfce000700 1 -- 172.21.15.1:0/1544621067 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 40) v1 ==== 7502+0+0 (secure 0 0 0) 0x7fbfd000ccb0 con 0x55eeeeabb010 2024-02-02T15:34:07.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.907+0000 7fbffaaf4b80 10 client.27548 fetch_fsmap finished waiting for FSMap version 40 2024-02-02T15:34:07.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.907+0000 7fbffaaf4b80 10 client.27548 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-02T15:34:07.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.907+0000 7fbffaaf4b80 20 client.27548 populate_metadata read hostname 'smithi001' 2024-02-02T15:34:07.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.907+0000 7fbffaaf4b80 1 --2- 172.21.15.1:0/1544621067 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55eeeead5e10 0x55eeeead8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:07.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.907+0000 7fbffaaf4b80 4 client.27548 mds_command: new command op to 24529 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:34:07.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.907+0000 7fbffaaf4b80 1 -- 172.21.15.1:0/1544621067 --> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eeeea05dd0 con 0x55eeeead5e10 2024-02-02T15:34:07.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.907+0000 7fbfd7e00700 1 --2- 172.21.15.1:0/1544621067 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55eeeead5e10 0x55eeeead8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:07.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.908+0000 7fbfd7e00700 1 --2- 172.21.15.1:0/1544621067 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55eeeead5e10 0x55eeeead8230 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6c6cb742f935f9d4 server_cookie=ed9c56b8af9540b2 in_seq=0 out_seq=0 2024-02-02T15:34:07.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.908+0000 7fbfce000700 10 client.27548 ms_handle_connect on v2:172.21.15.142:6838/321675421 2024-02-02T15:34:07.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.909+0000 7fbfce000700 1 -- 172.21.15.1:0/1544621067 <== mds.1 v2:172.21.15.142:6838/321675421 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55eeeea05dd0 con 0x55eeeead5e10 2024-02-02T15:34:07.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.909+0000 7fbfce000700 10 client.27548 handle_command_reply: tid=0 2024-02-02T15:34:07.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.909+0000 7fbffaaf4b80 1 client.27548 shutdown 2024-02-02T15:34:07.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.909+0000 7fbffaaf4b80 1 -- 172.21.15.1:0/1544621067 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eeeeabb010 msgr2=0x55eeeeac6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:07.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.909+0000 7fbffaaf4b80 1 --2- 172.21.15.1:0/1544621067 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eeeeabb010 0x55eeeeac6230 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fbfd0095820 tx=0x7fbfd00107d0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.909+0000 7fbffaaf4b80 1 -- 172.21.15.1:0/1544621067 shutdown_connections 2024-02-02T15:34:07.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.909+0000 7fbffaaf4b80 1 --2- 172.21.15.1:0/1544621067 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55eeeead5e10 0x55eeeead8230 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.909+0000 7fbffaaf4b80 1 --2- 172.21.15.1:0/1544621067 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eeeeac67e0 0x55eeeeac3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.909+0000 7fbffaaf4b80 1 --2- 172.21.15.1:0/1544621067 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eeeeabb010 0x55eeeeac6230 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.909+0000 7fbfce000700 0 client.27548 ms_handle_reset on v2:172.21.15.142:6838/321675421 2024-02-02T15:34:07.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.909+0000 7fbffaaf4b80 1 --2- 172.21.15.1:0/1544621067 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eeeeabb930 0x55eeeeac3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.909+0000 7fbffaaf4b80 1 -- 172.21.15.1:0/1544621067 >> 172.21.15.1:0/1544621067 conn(0x55eeeea1adb0 msgr2=0x55eeeeab6fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:07.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.909+0000 7fbffaaf4b80 1 -- 172.21.15.1:0/1544621067 shutdown_connections 2024-02-02T15:34:07.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.909+0000 7fbffaaf4b80 1 -- 172.21.15.1:0/1544621067 wait complete. 2024-02-02T15:34:07.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.910+0000 7fbffaaf4b80 20 client.27548 trim_cache size 0 max 16384 2024-02-02T15:34:07.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.923+0000 7fbffaaf4b80 1 Processor -- start 2024-02-02T15:34:07.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.923+0000 7fbffaaf4b80 1 -- start start 2024-02-02T15:34:07.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.923+0000 7fbffaaf4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eeeeabb010 0x55eeeeacd330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:07.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.923+0000 7fbffaaf4b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eeeeabb930 0x55eeeeabe7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:07.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.923+0000 7fbffaaf4b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eeeeac67e0 0x55eeeeabed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:07.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.923+0000 7fbffaaf4b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eeee8906a0 con 0x55eeeeac67e0 2024-02-02T15:34:07.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.923+0000 7fbffaaf4b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eeeeab8050 con 0x55eeeeabb930 2024-02-02T15:34:07.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.923+0000 7fbffaaf4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eeeeabb770 con 0x55eeeeabb010 2024-02-02T15:34:07.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.923+0000 7fbfe5000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eeeeac67e0 0x55eeeeabed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:07.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.923+0000 7fbfd7e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eeeeabb930 0x55eeeeabe7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:07.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.923+0000 7fbfd7e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eeeeabb930 0x55eeeeabe7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:33900/0 (socket says 172.21.15.1:33900) 2024-02-02T15:34:07.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.923+0000 7fbfd7e00700 1 -- 172.21.15.1:0/3577044558 learned_addr learned my addr 172.21.15.1:0/3577044558 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:34:07.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.923+0000 7fbfe5000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eeeeac67e0 0x55eeeeabed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48804/0 (socket says 172.21.15.1:48804) 2024-02-02T15:34:07.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.923+0000 7fbfdfe00700 1 --2- 172.21.15.1:0/3577044558 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eeeeabb010 0x55eeeeacd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:07.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.924+0000 7fbfd7e00700 1 -- 172.21.15.1:0/3577044558 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eeeeabb010 msgr2=0x55eeeeacd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:07.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.924+0000 7fbfd7e00700 1 --2- 172.21.15.1:0/3577044558 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eeeeabb010 0x55eeeeacd330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.924+0000 7fbfd7e00700 1 -- 172.21.15.1:0/3577044558 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eeeeac67e0 msgr2=0x55eeeeabed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:07.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.924+0000 7fbfd7e00700 1 --2- 172.21.15.1:0/3577044558 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eeeeac67e0 0x55eeeeabed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.924+0000 7fbfd7e00700 1 -- 172.21.15.1:0/3577044558 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eeee9d2ea0 con 0x55eeeeabb930 2024-02-02T15:34:07.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.924+0000 7fbfd7e00700 1 --2- 172.21.15.1:0/3577044558 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eeeeabb930 0x55eeeeabe7e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fbfc8009880 tx=0x7fbfc8014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65ef3cdee352bbc server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:07.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.924+0000 7fbfd6000700 1 -- 172.21.15.1:0/3577044558 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc8024040 con 0x55eeeeabb930 2024-02-02T15:34:07.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.924+0000 7fbfd6000700 1 -- 172.21.15.1:0/3577044558 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbfc802a410 con 0x55eeeeabb930 2024-02-02T15:34:07.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.924+0000 7fbfd6000700 1 -- 172.21.15.1:0/3577044558 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfc801f660 con 0x55eeeeabb930 2024-02-02T15:34:07.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.925+0000 7fbffaaf4b80 1 -- 172.21.15.1:0/3577044558 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eeeeabb930 msgr2=0x55eeeeabe7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:07.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.925+0000 7fbffaaf4b80 1 --2- 172.21.15.1:0/3577044558 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eeeeabb930 0x55eeeeabe7e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fbfc8009880 tx=0x7fbfc8014000 comp rx=0 tx=0).stop 2024-02-02T15:34:07.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.925+0000 7fbffaaf4b80 1 -- 172.21.15.1:0/3577044558 shutdown_connections 2024-02-02T15:34:07.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.925+0000 7fbffaaf4b80 1 --2- 172.21.15.1:0/3577044558 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eeeeac67e0 0x55eeeeabed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.925+0000 7fbffaaf4b80 1 --2- 172.21.15.1:0/3577044558 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eeeeabb010 0x55eeeeacd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.925+0000 7fbffaaf4b80 1 --2- 172.21.15.1:0/3577044558 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eeeeabb930 0x55eeeeabe7e0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.925+0000 7fbffaaf4b80 1 -- 172.21.15.1:0/3577044558 >> 172.21.15.1:0/3577044558 conn(0x55eeeea1adb0 msgr2=0x55eeeeabcfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:07.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.925+0000 7fbffaaf4b80 1 -- 172.21.15.1:0/3577044558 shutdown_connections 2024-02-02T15:34:07.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.925+0000 7fbffaaf4b80 1 -- 172.21.15.1:0/3577044558 wait complete. 2024-02-02T15:34:07.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.925+0000 7fbffaaf4b80 1 -- start start 2024-02-02T15:34:07.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.925+0000 7fbffaaf4b80 1 Processor -- start 2024-02-02T15:34:07.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.926+0000 7fbffaaf4b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eeeeabb010 0x55eeeeb1ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:07.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.926+0000 7fbffaaf4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eeeeabb930 0x55eeeeb1bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:07.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.926+0000 7fbffaaf4b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eeeeac67e0 0x55eeeeb20940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:07.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.926+0000 7fbffaaf4b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eeeeb20e80 con 0x55eeeeabb010 2024-02-02T15:34:07.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.926+0000 7fbffaaf4b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eeeeb20fc0 con 0x55eeeeac67e0 2024-02-02T15:34:07.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.926+0000 7fbffaaf4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eeeeb21180 con 0x55eeeeabb930 2024-02-02T15:34:07.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.926+0000 7fbffaaf4b80 10 client.0 _ll_register_callbacks cb 0x55eeee9aa290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:34:07.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.926+0000 7fbfdfe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eeeeabb010 0x55eeeeb1ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:07.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.926+0000 7fbfd7e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eeeeabb930 0x55eeeeb1bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:07.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.926+0000 7fbfdfe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eeeeabb010 0x55eeeeb1ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48818/0 (socket says 172.21.15.1:48818) 2024-02-02T15:34:07.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.926+0000 7fbfdfe00700 1 -- 172.21.15.1:0/3717635502 learned_addr learned my addr 172.21.15.1:0/3717635502 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:34:07.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.926+0000 7fbfe5000700 1 --2- 172.21.15.1:0/3717635502 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eeeeac67e0 0x55eeeeb20940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:07.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.927+0000 7fbfdfe00700 1 -- 172.21.15.1:0/3717635502 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eeeeabb930 msgr2=0x55eeeeb1bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:07.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.927+0000 7fbfdfe00700 1 --2- 172.21.15.1:0/3717635502 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eeeeabb930 0x55eeeeb1bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.927+0000 7fbfdfe00700 1 -- 172.21.15.1:0/3717635502 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eeeeac67e0 msgr2=0x55eeeeb20940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:07.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.927+0000 7fbfdfe00700 1 --2- 172.21.15.1:0/3717635502 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eeeeac67e0 0x55eeeeb20940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.927+0000 7fbfdfe00700 1 -- 172.21.15.1:0/3717635502 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfc8017070 con 0x55eeeeabb010 2024-02-02T15:34:07.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.927+0000 7fbfdfe00700 1 --2- 172.21.15.1:0/3717635502 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eeeeabb010 0x55eeeeb1ba70 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fbfd00c9ad0 tx=0x7fbfd0095710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f37586fc8410065a server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:07.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.927+0000 7fbfce000700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:34:07.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.927+0000 7fbfce000700 1 -- 172.21.15.1:0/3717635502 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfd0099a20 con 0x55eeeeabb010 2024-02-02T15:34:07.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.927+0000 7fbfce000700 1 -- 172.21.15.1:0/3717635502 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9541+0+0 (secure 0 0 0) 0x7fbfd0095320 con 0x55eeeeabb010 2024-02-02T15:34:07.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.927+0000 7fbffaaf4b80 1 -- 172.21.15.1:0/3717635502 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbfd00a1390 con 0x55eeeeabb010 2024-02-02T15:34:07.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.927+0000 7fbfce000700 1 -- 172.21.15.1:0/3717635502 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbfd009d150 con 0x55eeeeabb010 2024-02-02T15:34:07.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.927+0000 7fbffaaf4b80 10 client.17478 fetch_fsmap learned FSMap version 40 2024-02-02T15:34:07.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.927+0000 7fbffaaf4b80 1 -- 172.21.15.1:0/3717635502 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55eeee892cd0 con 0x55eeeeabb010 2024-02-02T15:34:07.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.927+0000 7fbfce000700 1 -- 172.21.15.1:0/3717635502 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbfd00a1390 con 0x55eeeeabb010 2024-02-02T15:34:07.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.928+0000 7fbfce000700 1 -- 172.21.15.1:0/3717635502 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 40) v1 ==== 7502+0+0 (secure 0 0 0) 0x7fbfd00b1ac0 con 0x55eeeeabb010 2024-02-02T15:34:07.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.928+0000 7fbffaaf4b80 10 client.17478 fetch_fsmap finished waiting for FSMap version 40 2024-02-02T15:34:07.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.928+0000 7fbffaaf4b80 10 client.17478 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-02T15:34:07.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.928+0000 7fbffaaf4b80 20 client.17478 populate_metadata read hostname 'smithi001' 2024-02-02T15:34:07.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.928+0000 7fbffaaf4b80 1 --2- 172.21.15.1:0/3717635502 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55eeeeb28510 0x55eeeeb2a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:07.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.928+0000 7fbffaaf4b80 4 client.17478 mds_command: new command op to 24529 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:34:07.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.928+0000 7fbffaaf4b80 1 -- 172.21.15.1:0/3717635502 --> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55eeee9cbf50 con 0x55eeeeb28510 2024-02-02T15:34:07.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.928+0000 7fbfd7e00700 1 --2- 172.21.15.1:0/3717635502 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55eeeeb28510 0x55eeeeb2a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:07.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.928+0000 7fbfd7e00700 1 --2- 172.21.15.1:0/3717635502 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55eeeeb28510 0x55eeeeb2a930 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a3d3db8c4498cfb4 server_cookie=a9296679956a8654 in_seq=0 out_seq=0 2024-02-02T15:34:07.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.928+0000 7fbfce000700 10 client.17478 ms_handle_connect on v2:172.21.15.142:6838/321675421 2024-02-02T15:34:07.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.929+0000 7fbfce000700 1 -- 172.21.15.1:0/3717635502 <== mds.1 v2:172.21.15.142:6838/321675421 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55eeee9cbf50 con 0x55eeeeb28510 2024-02-02T15:34:07.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.929+0000 7fbfce000700 10 client.17478 handle_command_reply: tid=0 2024-02-02T15:34:07.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.929+0000 7fbffaaf4b80 1 client.17478 shutdown 2024-02-02T15:34:07.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.929+0000 7fbffaaf4b80 1 -- 172.21.15.1:0/3717635502 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eeeeabb010 msgr2=0x55eeeeb1ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:07.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.929+0000 7fbffaaf4b80 1 --2- 172.21.15.1:0/3717635502 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eeeeabb010 0x55eeeeb1ba70 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fbfd00c9ad0 tx=0x7fbfd0095710 comp rx=0 tx=0).stop 2024-02-02T15:34:07.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.929+0000 7fbffaaf4b80 1 -- 172.21.15.1:0/3717635502 shutdown_connections 2024-02-02T15:34:07.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.929+0000 7fbffaaf4b80 1 --2- 172.21.15.1:0/3717635502 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55eeeeb28510 0x55eeeeb2a930 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.929+0000 7fbffaaf4b80 1 --2- 172.21.15.1:0/3717635502 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eeeeac67e0 0x55eeeeb20940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.929+0000 7fbffaaf4b80 1 --2- 172.21.15.1:0/3717635502 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eeeeabb010 0x55eeeeb1ba70 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.929+0000 7fbffaaf4b80 1 --2- 172.21.15.1:0/3717635502 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eeeeabb930 0x55eeeeb1bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.929+0000 7fbfce000700 0 client.17478 ms_handle_reset on v2:172.21.15.142:6838/321675421 2024-02-02T15:34:07.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.929+0000 7fbffaaf4b80 1 -- 172.21.15.1:0/3717635502 >> 172.21.15.1:0/3717635502 conn(0x55eeeea1adb0 msgr2=0x55eeeeabe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:07.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.929+0000 7fbffaaf4b80 1 -- 172.21.15.1:0/3717635502 shutdown_connections 2024-02-02T15:34:07.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.929+0000 7fbffaaf4b80 1 -- 172.21.15.1:0/3717635502 wait complete. 2024-02-02T15:34:07.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.929+0000 7fbffaaf4b80 20 client.17478 trim_cache size 0 max 16384 2024-02-02T15:34:07.934 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:34:07.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.930+0000 7fbfe7000700 1 -- 172.21.15.1:0/2277978806 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fbfc00a0710 msgr2=0x7fbfc00a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:07.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.930+0000 7fbfe7000700 1 --2- 172.21.15.1:0/2277978806 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fbfc00a0710 0x7fbfc00a2bd0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbfe0074830 tx=0x7fbfc8009260 comp rx=0 tx=0).stop 2024-02-02T15:34:07.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.930+0000 7fbfe7000700 1 -- 172.21.15.1:0/2277978806 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbfe0100ff0 msgr2=0x7fbfe0073040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:07.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.930+0000 7fbfe7000700 1 --2- 172.21.15.1:0/2277978806 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbfe0100ff0 0x7fbfe0073040 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fbfd000e9f0 tx=0x7fbfd000ed00 comp rx=0 tx=0).stop 2024-02-02T15:34:07.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.930+0000 7fbfe7000700 1 -- 172.21.15.1:0/2277978806 shutdown_connections 2024-02-02T15:34:07.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.930+0000 7fbfe7000700 1 --2- 172.21.15.1:0/2277978806 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fbfc00a0710 0x7fbfc00a2bd0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.930+0000 7fbfe7000700 1 --2- 172.21.15.1:0/2277978806 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbfe0073ac0 0x7fbfe01a51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.930+0000 7fbfe7000700 1 --2- 172.21.15.1:0/2277978806 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbfe0100ff0 0x7fbfe0073040 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.930+0000 7fbfe7000700 1 --2- 172.21.15.1:0/2277978806 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbfe0106110 0x7fbfe0073580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:07.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.930+0000 7fbfe7000700 1 -- 172.21.15.1:0/2277978806 >> 172.21.15.1:0/2277978806 conn(0x7fbfe00fc810 msgr2=0x7fbfe00fd260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:07.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.930+0000 7fbfe7000700 1 -- 172.21.15.1:0/2277978806 shutdown_connections 2024-02-02T15:34:07.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:07.930+0000 7fbfe7000700 1 -- 172.21.15.1:0/2277978806 wait complete. 2024-02-02T15:34:07.948 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-02T15:34:08.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.148+0000 7f71db800700 1 -- 172.21.15.1:0/2280598217 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f71d41019a0 msgr2=0x7f71d41059e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:08.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.148+0000 7f71db800700 1 --2- 172.21.15.1:0/2280598217 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f71d41019a0 0x7f71d41059e0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f71bc009990 tx=0x7f71bc009ca0 comp rx=0 tx=0).stop 2024-02-02T15:34:08.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.148+0000 7f71db800700 1 -- 172.21.15.1:0/2280598217 shutdown_connections 2024-02-02T15:34:08.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.148+0000 7f71db800700 1 --2- 172.21.15.1:0/2280598217 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71d4106110 0x7f71d410dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:08.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.148+0000 7f71db800700 1 --2- 172.21.15.1:0/2280598217 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f71d4100ff0 0x7f71d41013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:08.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.148+0000 7f71db800700 1 --2- 172.21.15.1:0/2280598217 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f71d41019a0 0x7f71d41059e0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:08.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.148+0000 7f71db800700 1 -- 172.21.15.1:0/2280598217 >> 172.21.15.1:0/2280598217 conn(0x7f71d40fc810 msgr2=0x7f71d40fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:08.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.148+0000 7f71db800700 1 -- 172.21.15.1:0/2280598217 shutdown_connections 2024-02-02T15:34:08.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.149+0000 7f71db800700 1 -- 172.21.15.1:0/2280598217 wait complete. 2024-02-02T15:34:08.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.149+0000 7f71db800700 1 Processor -- start 2024-02-02T15:34:08.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.150+0000 7f71db800700 1 -- start start 2024-02-02T15:34:08.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.150+0000 7f71db800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f71d4100ff0 0x7f71d4199060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:08.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.150+0000 7f71db800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71d4106110 0x7f71d41995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:08.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.150+0000 7f71db800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f71d419da00 0x7f71d419de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:08.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.150+0000 7f71db800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71d419e480 con 0x7f71d419da00 2024-02-02T15:34:08.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.150+0000 7f71db800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71d419e5f0 con 0x7f71d4100ff0 2024-02-02T15:34:08.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.150+0000 7f71db800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71d419e7b0 con 0x7f71d4106110 2024-02-02T15:34:08.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.150+0000 7f71d8e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f71d4100ff0 0x7f71d4199060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:08.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.150+0000 7f71d9800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f71d419da00 0x7f71d419de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:08.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.150+0000 7f71d8e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f71d4100ff0 0x7f71d4199060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:33940/0 (socket says 172.21.15.1:33940) 2024-02-02T15:34:08.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.150+0000 7f71d8e00700 1 -- 172.21.15.1:0/567936517 learned_addr learned my addr 172.21.15.1:0/567936517 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:34:08.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.150+0000 7f71d3e00700 1 --2- 172.21.15.1:0/567936517 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71d4106110 0x7f71d41995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:08.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.151+0000 7f71d8e00700 1 -- 172.21.15.1:0/567936517 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71d4106110 msgr2=0x7f71d41995a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:08.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.151+0000 7f71d8e00700 1 --2- 172.21.15.1:0/567936517 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71d4106110 0x7f71d41995a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:08.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.151+0000 7f71d8e00700 1 -- 172.21.15.1:0/567936517 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f71d419da00 msgr2=0x7f71d419de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:08.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.151+0000 7f71d8e00700 1 --2- 172.21.15.1:0/567936517 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f71d419da00 0x7f71d419de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:08.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.151+0000 7f71d8e00700 1 -- 172.21.15.1:0/567936517 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71bc0095f0 con 0x7f71d4100ff0 2024-02-02T15:34:08.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.151+0000 7f71d9800700 1 --2- 172.21.15.1:0/567936517 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f71d419da00 0x7f71d419de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:34:08.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.151+0000 7f71d8e00700 1 --2- 172.21.15.1:0/567936517 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f71d4100ff0 0x7f71d4199060 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f71c400e9a0 tx=0x7f71c400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:08.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.151+0000 7f71d1600700 1 -- 172.21.15.1:0/567936517 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c4009790 con 0x7f71d4100ff0 2024-02-02T15:34:08.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.151+0000 7f71d1600700 1 -- 172.21.15.1:0/567936517 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f71c4010430 con 0x7f71d4100ff0 2024-02-02T15:34:08.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.151+0000 7f71db800700 1 -- 172.21.15.1:0/567936517 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71d41a5d10 con 0x7f71d4100ff0 2024-02-02T15:34:08.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.151+0000 7f71d1600700 1 -- 172.21.15.1:0/567936517 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c4018680 con 0x7f71d4100ff0 2024-02-02T15:34:08.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.151+0000 7f71db800700 1 -- 172.21.15.1:0/567936517 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71d41a61e0 con 0x7f71d4100ff0 2024-02-02T15:34:08.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.154+0000 7f71d1600700 1 -- 172.21.15.1:0/567936517 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f71c40105a0 con 0x7f71d4100ff0 2024-02-02T15:34:08.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.155+0000 7f71d1600700 1 --2- 172.21.15.1:0/567936517 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f71b40a0710 0x7f71b40a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:08.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.156+0000 7f71ef2d0b80 1 Processor -- start 2024-02-02T15:34:08.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.156+0000 7f71d1600700 1 -- 172.21.15.1:0/567936517 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f71c4014070 con 0x7f71d4100ff0 2024-02-02T15:34:08.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.156+0000 7f71ef2d0b80 1 -- start start 2024-02-02T15:34:08.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.156+0000 7f71d3e00700 1 --2- 172.21.15.1:0/567936517 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f71b40a0710 0x7f71b40a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:08.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.156+0000 7f71ef2d0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7a2b5f010 0x55f7a2b5f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:08.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.156+0000 7f71ef2d0b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f7a2b5f930 0x55f7a2b6a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:08.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.156+0000 7f71ef2d0b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f7a2b6a7e0 0x55f7a2b6cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:08.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.156+0000 7f71ef2d0b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7a2ad34c0 con 0x55f7a2b6a7e0 2024-02-02T15:34:08.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.156+0000 7f71ef2d0b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7a2ab7aa0 con 0x55f7a2b5f930 2024-02-02T15:34:08.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.156+0000 7f71ef2d0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7a2ab5460 con 0x55f7a2b5f010 2024-02-02T15:34:08.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.157+0000 7f71d9800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f7a2b6a7e0 0x55f7a2b6cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:08.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.157+0000 7f71d8e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7a2b5f010 0x55f7a2b5f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:08.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.157+0000 7f71d9800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f7a2b6a7e0 0x55f7a2b6cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48846/0 (socket says 172.21.15.1:48846) 2024-02-02T15:34:08.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.157+0000 7f71d9800700 1 -- 172.21.15.1:0/1318118392 learned_addr learned my addr 172.21.15.1:0/1318118392 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:34:08.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.157+0000 7f71d3e00700 1 --2- 172.21.15.1:0/567936517 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f71b40a0710 0x7f71b40a2bd0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f71d419a670 tx=0x7f71bc005fd0 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:08.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.157+0000 7f71d3e00700 1 --2- 172.21.15.1:0/1318118392 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f7a2b5f930 0x55f7a2b6a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:08.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.157+0000 7f71d8e00700 1 -- 172.21.15.1:0/1318118392 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f7a2b5f930 msgr2=0x55f7a2b6a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:08.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.157+0000 7f71d8e00700 1 --2- 172.21.15.1:0/1318118392 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f7a2b5f930 0x55f7a2b6a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:08.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.157+0000 7f71d8e00700 1 -- 172.21.15.1:0/1318118392 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f7a2b6a7e0 msgr2=0x55f7a2b6cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:08.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.157+0000 7f71d8e00700 1 --2- 172.21.15.1:0/1318118392 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f7a2b6a7e0 0x55f7a2b6cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:08.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.157+0000 7f71d8e00700 1 -- 172.21.15.1:0/1318118392 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f7a29323b0 con 0x55f7a2b5f010 2024-02-02T15:34:08.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.157+0000 7f71d9800700 1 --2- 172.21.15.1:0/1318118392 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f7a2b6a7e0 0x55f7a2b6cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:34:08.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.157+0000 7f71d8e00700 1 --2- 172.21.15.1:0/1318118392 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7a2b5f010 0x55f7a2b5f3f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f71c40c95e0 tx=0x7f71c4098dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef99093c91fa68e4 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:08.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.157+0000 7f71ca000700 1 -- 172.21.15.1:0/1318118392 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c40acea0 con 0x55f7a2b5f010 2024-02-02T15:34:08.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.157+0000 7f71ca000700 1 -- 172.21.15.1:0/1318118392 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f71c40993e0 con 0x55f7a2b5f010 2024-02-02T15:34:08.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.157+0000 7f71ca000700 1 -- 172.21.15.1:0/1318118392 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c4082be0 con 0x55f7a2b5f010 2024-02-02T15:34:08.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.158+0000 7f71ef2d0b80 1 -- 172.21.15.1:0/1318118392 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7a2b5f010 msgr2=0x55f7a2b5f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:08.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.158+0000 7f71ef2d0b80 1 --2- 172.21.15.1:0/1318118392 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7a2b5f010 0x55f7a2b5f3f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f71c40c95e0 tx=0x7f71c4098dc0 comp rx=0 tx=0).stop 2024-02-02T15:34:08.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.158+0000 7f71ef2d0b80 1 -- 172.21.15.1:0/1318118392 shutdown_connections 2024-02-02T15:34:08.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.158+0000 7f71ef2d0b80 1 --2- 172.21.15.1:0/1318118392 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f7a2b6a7e0 0x55f7a2b6cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:08.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.158+0000 7f71ef2d0b80 1 --2- 172.21.15.1:0/1318118392 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7a2b5f010 0x55f7a2b5f3f0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:08.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.158+0000 7f71ef2d0b80 1 --2- 172.21.15.1:0/1318118392 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f7a2b5f930 0x55f7a2b6a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:08.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.158+0000 7f71ef2d0b80 1 -- 172.21.15.1:0/1318118392 >> 172.21.15.1:0/1318118392 conn(0x55f7a2abedb0 msgr2=0x55f7a2b5b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:08.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.158+0000 7f71ef2d0b80 1 -- 172.21.15.1:0/1318118392 shutdown_connections 2024-02-02T15:34:08.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.158+0000 7f71ef2d0b80 1 -- 172.21.15.1:0/1318118392 wait complete. 2024-02-02T15:34:08.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.159+0000 7f71ef2d0b80 1 -- start start 2024-02-02T15:34:08.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.159+0000 7f71ef2d0b80 1 Processor -- start 2024-02-02T15:34:08.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.159+0000 7f71ef2d0b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f7a2b5f010 0x55f7a2b6a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:08.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.159+0000 7f71ef2d0b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f7a2b5f930 0x55f7a2b673b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:08.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.159+0000 7f71ef2d0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7a2b6a7e0 0x55f7a2b679c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:08.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.159+0000 7f71ef2d0b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7a2ab0d50 con 0x55f7a2b5f010 2024-02-02T15:34:08.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.159+0000 7f71ef2d0b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7a2a4cd80 con 0x55f7a2b5f930 2024-02-02T15:34:08.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.159+0000 7f71ef2d0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7a2a4ca20 con 0x55f7a2b6a7e0 2024-02-02T15:34:08.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.160+0000 7f71ef2d0b80 10 client.0 _ll_register_callbacks cb 0x55f7a2a4e290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:34:08.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.160+0000 7f71d8e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f7a2b5f010 0x55f7a2b6a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:08.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.160+0000 7f71d3e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f7a2b5f930 0x55f7a2b673b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:08.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.160+0000 7f71d9800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7a2b6a7e0 0x55f7a2b679c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:08.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.160+0000 7f71d8e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f7a2b5f010 0x55f7a2b6a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48860/0 (socket says 172.21.15.1:48860) 2024-02-02T15:34:08.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.160+0000 7f71d8e00700 1 -- 172.21.15.1:0/1026306514 learned_addr learned my addr 172.21.15.1:0/1026306514 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:34:08.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.160+0000 7f71d8e00700 1 -- 172.21.15.1:0/1026306514 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7a2b6a7e0 msgr2=0x55f7a2b679c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:08.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.160+0000 7f71d8e00700 1 --2- 172.21.15.1:0/1026306514 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7a2b6a7e0 0x55f7a2b679c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:08.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.160+0000 7f71d8e00700 1 -- 172.21.15.1:0/1026306514 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f7a2b5f930 msgr2=0x55f7a2b673b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:08.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.160+0000 7f71d8e00700 1 --2- 172.21.15.1:0/1026306514 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f7a2b5f930 0x55f7a2b673b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:08.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.160+0000 7f71d8e00700 1 -- 172.21.15.1:0/1026306514 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71cc009e00 con 0x55f7a2b5f010 2024-02-02T15:34:08.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.160+0000 7f71d8e00700 1 --2- 172.21.15.1:0/1026306514 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f7a2b5f010 0x55f7a2b6a1a0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f71c400e970 tx=0x7f71c4097990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71dc0e3104e0ea33 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:08.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.160+0000 7f71c2a00700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:34:08.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.161+0000 7f71c2a00700 1 -- 172.21.15.1:0/1026306514 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c40acea0 con 0x55f7a2b5f010 2024-02-02T15:34:08.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.161+0000 7f71c2a00700 1 -- 172.21.15.1:0/1026306514 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f71c409d210 con 0x55f7a2b5f010 2024-02-02T15:34:08.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.161+0000 7f71ef2d0b80 1 -- 172.21.15.1:0/1026306514 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f7a2b530f0 con 0x55f7a2b5f010 2024-02-02T15:34:08.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.161+0000 7f71c2a00700 1 -- 172.21.15.1:0/1026306514 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71c4082de0 con 0x55f7a2b5f010 2024-02-02T15:34:08.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.161+0000 7f71ef2d0b80 10 client.17490 fetch_fsmap learned FSMap version 40 2024-02-02T15:34:08.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.161+0000 7f71ef2d0b80 1 -- 172.21.15.1:0/1026306514 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x7f71d41a61e0 con 0x55f7a2b5f010 2024-02-02T15:34:08.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.161+0000 7f71c2a00700 1 -- 172.21.15.1:0/1026306514 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f7a2b530f0 con 0x55f7a2b5f010 2024-02-02T15:34:08.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.161+0000 7f71c2a00700 1 -- 172.21.15.1:0/1026306514 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 40) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f71c40a0fc0 con 0x55f7a2b5f010 2024-02-02T15:34:08.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.161+0000 7f71ef2d0b80 10 client.17490 fetch_fsmap finished waiting for FSMap version 40 2024-02-02T15:34:08.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.161+0000 7f71ef2d0b80 10 client.17490 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:34:08.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.161+0000 7f71ef2d0b80 20 client.17490 populate_metadata read hostname 'smithi001' 2024-02-02T15:34:08.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.161+0000 7f71ef2d0b80 1 --2- 172.21.15.1:0/1026306514 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55f7a2b79f00 0x55f7a2b7c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:08.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.161+0000 7f71ef2d0b80 4 client.17490 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:34:08.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.161+0000 7f71ef2d0b80 1 -- 172.21.15.1:0/1026306514 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f7a2aa9dd0 con 0x55f7a2b79f00 2024-02-02T15:34:08.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.162+0000 7f71d3e00700 1 --2- 172.21.15.1:0/1026306514 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55f7a2b79f00 0x55f7a2b7c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:08.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.162+0000 7f71d3e00700 1 --2- 172.21.15.1:0/1026306514 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55f7a2b79f00 0x55f7a2b7c320 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5499025fc0732d20 server_cookie=a90bd4faeb6abc25 in_seq=0 out_seq=0 2024-02-02T15:34:08.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.162+0000 7f71c2a00700 10 client.17490 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:34:08.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.163+0000 7f71c2a00700 1 -- 172.21.15.1:0/1026306514 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55f7a2aa9dd0 con 0x55f7a2b79f00 2024-02-02T15:34:08.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.163+0000 7f71c2a00700 10 client.17490 handle_command_reply: tid=0 2024-02-02T15:34:08.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.163+0000 7f71ef2d0b80 1 client.17490 shutdown 2024-02-02T15:34:08.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.163+0000 7f71ef2d0b80 1 -- 172.21.15.1:0/1026306514 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f7a2b5f010 msgr2=0x55f7a2b6a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:08.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.163+0000 7f71ef2d0b80 1 --2- 172.21.15.1:0/1026306514 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f7a2b5f010 0x55f7a2b6a1a0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f71c400e970 tx=0x7f71c4097990 comp rx=0 tx=0).stop 2024-02-02T15:34:08.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.163+0000 7f71ef2d0b80 1 -- 172.21.15.1:0/1026306514 shutdown_connections 2024-02-02T15:34:08.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.163+0000 7f71ef2d0b80 1 --2- 172.21.15.1:0/1026306514 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55f7a2b79f00 0x55f7a2b7c320 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:08.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.163+0000 7f71ef2d0b80 1 --2- 172.21.15.1:0/1026306514 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7a2b6a7e0 0x55f7a2b679c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:08.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.163+0000 7f71ef2d0b80 1 --2- 172.21.15.1:0/1026306514 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f7a2b5f010 0x55f7a2b6a1a0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:08.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.163+0000 7f71ef2d0b80 1 --2- 172.21.15.1:0/1026306514 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f7a2b5f930 0x55f7a2b673b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:08.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.163+0000 7f71c2a00700 0 client.17490 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:34:08.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.163+0000 7f71ef2d0b80 1 -- 172.21.15.1:0/1026306514 >> 172.21.15.1:0/1026306514 conn(0x55f7a2abedb0 msgr2=0x55f7a2b5afa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:08.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.164+0000 7f71ef2d0b80 1 -- 172.21.15.1:0/1026306514 shutdown_connections 2024-02-02T15:34:08.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.164+0000 7f71ef2d0b80 1 -- 172.21.15.1:0/1026306514 wait complete. 2024-02-02T15:34:08.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.164+0000 7f71ef2d0b80 20 client.17490 trim_cache size 0 max 16384 2024-02-02T15:34:08.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.177+0000 7f71ef2d0b80 1 Processor -- start 2024-02-02T15:34:08.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.177+0000 7f71ef2d0b80 1 -- start start 2024-02-02T15:34:08.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.177+0000 7f71ef2d0b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f7a2b5f010 0x55f7a2b61cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:08.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.177+0000 7f71ef2d0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7a2b5f930 0x55f7a2b69cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:08.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.177+0000 7f71ef2d0b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f7a2b6a7e0 0x55f7a2b758a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:08.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.177+0000 7f71ef2d0b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7a29346a0 con 0x55f7a2b5f010 2024-02-02T15:34:08.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.177+0000 7f71ef2d0b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7a2b5ff40 con 0x55f7a2b6a7e0 2024-02-02T15:34:08.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.177+0000 7f71ef2d0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7a2b60ad0 con 0x55f7a2b5f930 2024-02-02T15:34:08.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.177+0000 7f71d8e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f7a2b5f010 0x55f7a2b61cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:08.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.177+0000 7f71d8e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f7a2b5f010 0x55f7a2b61cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48874/0 (socket says 172.21.15.1:48874) 2024-02-02T15:34:08.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.177+0000 7f71d8e00700 1 -- 172.21.15.1:0/1343848363 learned_addr learned my addr 172.21.15.1:0/1343848363 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:34:08.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.178+0000 7f71d3e00700 1 --2- 172.21.15.1:0/1343848363 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7a2b5f930 0x55f7a2b69cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:08.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.178+0000 7f71d9800700 1 --2- 172.21.15.1:0/1343848363 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f7a2b6a7e0 0x55f7a2b758a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:08.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.178+0000 7f71d3e00700 1 -- 172.21.15.1:0/1343848363 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f7a2b6a7e0 msgr2=0x55f7a2b758a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:08.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.178+0000 7f71d3e00700 1 --2- 172.21.15.1:0/1343848363 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f7a2b6a7e0 0x55f7a2b758a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:08.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.178+0000 7f71d3e00700 1 -- 172.21.15.1:0/1343848363 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f7a2b5f010 msgr2=0x55f7a2b61cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:08.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.178+0000 7f71d3e00700 1 --2- 172.21.15.1:0/1343848363 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f7a2b5f010 0x55f7a2b61cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:08.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.178+0000 7f71d3e00700 1 -- 172.21.15.1:0/1343848363 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f7a28deb60 con 0x55f7a2b5f930 2024-02-02T15:34:08.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.178+0000 7f71d8e00700 1 --2- 172.21.15.1:0/1343848363 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f7a2b5f010 0x55f7a2b61cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:34:08.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.178+0000 7f71d3e00700 1 --2- 172.21.15.1:0/1343848363 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7a2b5f930 0x55f7a2b69cd0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f71bc000c00 tx=0x7f71bc023000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97fdfd2a67fa553c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:08.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.178+0000 7f71caa00700 1 -- 172.21.15.1:0/1343848363 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71bc033050 con 0x55f7a2b5f930 2024-02-02T15:34:08.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.178+0000 7f71caa00700 1 -- 172.21.15.1:0/1343848363 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f71bc01e040 con 0x55f7a2b5f930 2024-02-02T15:34:08.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.178+0000 7f71caa00700 1 -- 172.21.15.1:0/1343848363 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71bc00f7e0 con 0x55f7a2b5f930 2024-02-02T15:34:08.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.179+0000 7f71ef2d0b80 1 -- 172.21.15.1:0/1343848363 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7a2b5f930 msgr2=0x55f7a2b69cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:08.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.179+0000 7f71ef2d0b80 1 --2- 172.21.15.1:0/1343848363 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7a2b5f930 0x55f7a2b69cd0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f71bc000c00 tx=0x7f71bc023000 comp rx=0 tx=0).stop 2024-02-02T15:34:08.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.179+0000 7f71ef2d0b80 1 -- 172.21.15.1:0/1343848363 shutdown_connections 2024-02-02T15:34:08.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.179+0000 7f71ef2d0b80 1 --2- 172.21.15.1:0/1343848363 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f7a2b6a7e0 0x55f7a2b758a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:08.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.179+0000 7f71ef2d0b80 1 --2- 172.21.15.1:0/1343848363 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f7a2b5f010 0x55f7a2b61cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:08.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.179+0000 7f71ef2d0b80 1 --2- 172.21.15.1:0/1343848363 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7a2b5f930 0x55f7a2b69cd0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:08.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.179+0000 7f71ef2d0b80 1 -- 172.21.15.1:0/1343848363 >> 172.21.15.1:0/1343848363 conn(0x55f7a2abedb0 msgr2=0x55f7a2b764e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:08.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.179+0000 7f71ef2d0b80 1 -- 172.21.15.1:0/1343848363 shutdown_connections 2024-02-02T15:34:08.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.179+0000 7f71ef2d0b80 1 -- 172.21.15.1:0/1343848363 wait complete. 2024-02-02T15:34:08.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.179+0000 7f71ef2d0b80 1 -- start start 2024-02-02T15:34:08.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.180+0000 7f71ef2d0b80 1 Processor -- start 2024-02-02T15:34:08.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.180+0000 7f71ef2d0b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f7a2b5f010 0x55f7a2b6eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:08.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.180+0000 7f71ef2d0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7a2b5f930 0x55f7a2b6f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:08.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.180+0000 7f71ef2d0b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f7a2b6a7e0 0x55f7a2bca9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:08.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.180+0000 7f71ef2d0b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7a2bcaf20 con 0x55f7a2b6a7e0 2024-02-02T15:34:08.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.180+0000 7f71ef2d0b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7a2bcb060 con 0x55f7a2b5f010 2024-02-02T15:34:08.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.180+0000 7f71ef2d0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7a2bcb1f0 con 0x55f7a2b5f930 2024-02-02T15:34:08.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.180+0000 7f71ef2d0b80 10 client.0 _ll_register_callbacks cb 0x55f7a2b5cb40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:34:08.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.180+0000 7f71d9800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f7a2b6a7e0 0x55f7a2bca9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:08.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.180+0000 7f71d8e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f7a2b5f010 0x55f7a2b6eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:08.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.180+0000 7f71d8e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f7a2b5f010 0x55f7a2b6eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:33970/0 (socket says 172.21.15.1:33970) 2024-02-02T15:34:08.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.180+0000 7f71d8e00700 1 -- 172.21.15.1:0/554210247 learned_addr learned my addr 172.21.15.1:0/554210247 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:34:08.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.180+0000 7f71d9800700 1 --2- 172.21.15.1:0/554210247 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f7a2b6a7e0 0x55f7a2bca9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48888/0 (socket says 172.21.15.1:48888) 2024-02-02T15:34:08.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.181+0000 7f71d3e00700 1 --2- 172.21.15.1:0/554210247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7a2b5f930 0x55f7a2b6f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:08.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.181+0000 7f71d8e00700 1 -- 172.21.15.1:0/554210247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7a2b5f930 msgr2=0x55f7a2b6f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:08.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.181+0000 7f71d8e00700 1 --2- 172.21.15.1:0/554210247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7a2b5f930 0x55f7a2b6f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:08.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.181+0000 7f71d8e00700 1 -- 172.21.15.1:0/554210247 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f7a2b6a7e0 msgr2=0x55f7a2bca9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:08.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.181+0000 7f71d8e00700 1 --2- 172.21.15.1:0/554210247 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f7a2b6a7e0 0x55f7a2bca9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:08.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.181+0000 7f71d8e00700 1 -- 172.21.15.1:0/554210247 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71d41a61e0 con 0x55f7a2b5f010 2024-02-02T15:34:08.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.181+0000 7f71d3e00700 1 --2- 172.21.15.1:0/554210247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7a2b5f930 0x55f7a2b6f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:34:08.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.181+0000 7f71d8e00700 1 --2- 172.21.15.1:0/554210247 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f7a2b5f010 0x55f7a2b6eec0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f71c4098eb0 tx=0x7f71c40b74d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86dd752a4efbfa5 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:08.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.181+0000 7f71c2a00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:34:08.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.181+0000 7f71c2a00700 1 -- 172.21.15.1:0/554210247 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c40acea0 con 0x55f7a2b5f010 2024-02-02T15:34:08.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.181+0000 7f71ef2d0b80 1 -- 172.21.15.1:0/554210247 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f71c40993e0 con 0x55f7a2b5f010 2024-02-02T15:34:08.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.181+0000 7f71c2a00700 1 -- 172.21.15.1:0/554210247 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f71c409d210 con 0x55f7a2b5f010 2024-02-02T15:34:08.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.181+0000 7f71c2a00700 1 -- 172.21.15.1:0/554210247 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71c4098720 con 0x55f7a2b5f010 2024-02-02T15:34:08.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.181+0000 7f71ef2d0b80 10 client.27572 fetch_fsmap learned FSMap version 40 2024-02-02T15:34:08.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.181+0000 7f71ef2d0b80 1 -- 172.21.15.1:0/554210247 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x7f71bc02c040 con 0x55f7a2b5f010 2024-02-02T15:34:08.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.182+0000 7f71c2a00700 1 -- 172.21.15.1:0/554210247 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f71c40993e0 con 0x55f7a2b5f010 2024-02-02T15:34:08.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.182+0000 7f71c2a00700 1 -- 172.21.15.1:0/554210247 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 40) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f71c40b1b80 con 0x55f7a2b5f010 2024-02-02T15:34:08.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.182+0000 7f71ef2d0b80 10 client.27572 fetch_fsmap finished waiting for FSMap version 40 2024-02-02T15:34:08.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.182+0000 7f71ef2d0b80 10 client.27572 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:34:08.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.182+0000 7f71ef2d0b80 20 client.27572 populate_metadata read hostname 'smithi001' 2024-02-02T15:34:08.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.182+0000 7f71ef2d0b80 1 --2- 172.21.15.1:0/554210247 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55f7a2bcc590 0x55f7a2bce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:08.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.182+0000 7f71ef2d0b80 4 client.27572 mds_command: new command op to 24557 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:34:08.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.182+0000 7f71ef2d0b80 1 -- 172.21.15.1:0/554210247 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f7a2a6ff50 con 0x55f7a2bcc590 2024-02-02T15:34:08.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.182+0000 7f71d3e00700 1 --2- 172.21.15.1:0/554210247 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55f7a2bcc590 0x55f7a2bce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:08.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.183+0000 7f71d3e00700 1 --2- 172.21.15.1:0/554210247 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55f7a2bcc590 0x55f7a2bce9b0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3dbe6afdc9cbaa0e server_cookie=a02753136ae36df5 in_seq=0 out_seq=0 2024-02-02T15:34:08.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.183+0000 7f71c2a00700 10 client.27572 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:34:08.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.183+0000 7f71c2a00700 1 -- 172.21.15.1:0/554210247 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f7a2a6ff50 con 0x55f7a2bcc590 2024-02-02T15:34:08.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.183+0000 7f71c2a00700 10 client.27572 handle_command_reply: tid=0 2024-02-02T15:34:08.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.183+0000 7f71ef2d0b80 1 client.27572 shutdown 2024-02-02T15:34:08.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.183+0000 7f71ef2d0b80 1 -- 172.21.15.1:0/554210247 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f7a2b5f010 msgr2=0x55f7a2b6eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:08.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.183+0000 7f71ef2d0b80 1 --2- 172.21.15.1:0/554210247 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f7a2b5f010 0x55f7a2b6eec0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f71c4098eb0 tx=0x7f71c40b74d0 comp rx=0 tx=0).stop 2024-02-02T15:34:08.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.183+0000 7f71ef2d0b80 1 -- 172.21.15.1:0/554210247 shutdown_connections 2024-02-02T15:34:08.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.183+0000 7f71ef2d0b80 1 --2- 172.21.15.1:0/554210247 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55f7a2bcc590 0x55f7a2bce9b0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:08.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.183+0000 7f71ef2d0b80 1 --2- 172.21.15.1:0/554210247 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f7a2b6a7e0 0x55f7a2bca9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:08.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.183+0000 7f71ef2d0b80 1 --2- 172.21.15.1:0/554210247 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f7a2b5f010 0x55f7a2b6eec0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:08.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.183+0000 7f71ef2d0b80 1 --2- 172.21.15.1:0/554210247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f7a2b5f930 0x55f7a2b6f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:08.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.183+0000 7f71c2a00700 0 client.27572 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:34:08.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.183+0000 7f71ef2d0b80 1 -- 172.21.15.1:0/554210247 >> 172.21.15.1:0/554210247 conn(0x55f7a2abedb0 msgr2=0x55f7a2b758a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:08.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.183+0000 7f71ef2d0b80 1 -- 172.21.15.1:0/554210247 shutdown_connections 2024-02-02T15:34:08.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.183+0000 7f71ef2d0b80 1 -- 172.21.15.1:0/554210247 wait complete. 2024-02-02T15:34:08.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.183+0000 7f71ef2d0b80 20 client.27572 trim_cache size 0 max 16384 2024-02-02T15:34:08.192 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:34:08.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.184+0000 7f71db800700 1 -- 172.21.15.1:0/567936517 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f71b40a0710 msgr2=0x7f71b40a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:08.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.184+0000 7f71db800700 1 --2- 172.21.15.1:0/567936517 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f71b40a0710 0x7f71b40a2bd0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f71d419a670 tx=0x7f71bc005fd0 comp rx=0 tx=0).stop 2024-02-02T15:34:08.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.184+0000 7f71db800700 1 -- 172.21.15.1:0/567936517 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f71d4100ff0 msgr2=0x7f71d4199060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:08.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.184+0000 7f71db800700 1 --2- 172.21.15.1:0/567936517 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f71d4100ff0 0x7f71d4199060 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f71c400e9a0 tx=0x7f71c400ecb0 comp rx=0 tx=0).stop 2024-02-02T15:34:08.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.184+0000 7f71db800700 1 -- 172.21.15.1:0/567936517 shutdown_connections 2024-02-02T15:34:08.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.184+0000 7f71db800700 1 --2- 172.21.15.1:0/567936517 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f71b40a0710 0x7f71b40a2bd0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:08.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.184+0000 7f71db800700 1 --2- 172.21.15.1:0/567936517 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f71d419da00 0x7f71d419de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:08.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.184+0000 7f71db800700 1 --2- 172.21.15.1:0/567936517 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f71d4100ff0 0x7f71d4199060 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:08.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.184+0000 7f71db800700 1 --2- 172.21.15.1:0/567936517 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f71d4106110 0x7f71d41995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:08.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.184+0000 7f71db800700 1 -- 172.21.15.1:0/567936517 >> 172.21.15.1:0/567936517 conn(0x7f71d40fc810 msgr2=0x7f71d40fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:08.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.184+0000 7f71db800700 1 -- 172.21.15.1:0/567936517 shutdown_connections 2024-02-02T15:34:08.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:08.184+0000 7f71db800700 1 -- 172.21.15.1:0/567936517 wait complete. 2024-02-02T15:34:08.243 INFO:tasks.workunit.client.0.smithi001.stdout: 1 448638 7.48 MB/sec execute 222 sec latency 1552.204 ms 2024-02-02T15:34:09.203 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-02T15:34:09.203 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-02T15:34:09.243 INFO:tasks.workunit.client.0.smithi001.stdout: 1 448638 7.45 MB/sec execute 223 sec latency 2552.260 ms 2024-02-02T15:34:09.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.418+0000 7f3bb6c00700 1 -- 172.21.15.1:0/890676331 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3bb010b400 msgr2=0x7f3bb010d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:09.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.418+0000 7f3bb6c00700 1 --2- 172.21.15.1:0/890676331 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3bb010b400 0x7f3bb010d850 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f3ba800b0a0 tx=0x7f3ba800b3b0 comp rx=0 tx=0).stop 2024-02-02T15:34:09.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.418+0000 7f3bb6c00700 1 -- 172.21.15.1:0/890676331 shutdown_connections 2024-02-02T15:34:09.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.418+0000 7f3bb6c00700 1 --2- 172.21.15.1:0/890676331 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3bb010b400 0x7f3bb010d850 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:09.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.418+0000 7f3bb6c00700 1 --2- 172.21.15.1:0/890676331 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3bb0102030 0x7f3bb0102410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:09.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.418+0000 7f3bb6c00700 1 --2- 172.21.15.1:0/890676331 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3bb0102950 0x7f3bb010ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:09.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.418+0000 7f3bb6c00700 1 -- 172.21.15.1:0/890676331 >> 172.21.15.1:0/890676331 conn(0x7f3bb0076d70 msgr2=0x7f3bb0077180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:09.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.418+0000 7f3bb6c00700 1 -- 172.21.15.1:0/890676331 shutdown_connections 2024-02-02T15:34:09.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.418+0000 7f3bb6c00700 1 -- 172.21.15.1:0/890676331 wait complete. 2024-02-02T15:34:09.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.419+0000 7f3bb6c00700 1 Processor -- start 2024-02-02T15:34:09.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.419+0000 7f3bb6c00700 1 -- start start 2024-02-02T15:34:09.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.420+0000 7f3bb6c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3bb0102030 0x7f3bb0199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:09.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.420+0000 7f3bb6c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3bb0102950 0x7f3bb01996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:09.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.420+0000 7f3bb6c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3bb010b400 0x7f3bb019db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:09.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.420+0000 7f3bb6c00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb019e170 con 0x7f3bb010b400 2024-02-02T15:34:09.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.420+0000 7f3bb6c00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb019e2e0 con 0x7f3bb0102950 2024-02-02T15:34:09.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.420+0000 7f3bb6c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb019e4a0 con 0x7f3bb0102030 2024-02-02T15:34:09.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.420+0000 7f3bafe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3bb0102030 0x7f3bb0199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:09.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.420+0000 7f3bb4c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3bb010b400 0x7f3bb019db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:09.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.420+0000 7f3bafe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3bb0102030 0x7f3bb0199190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:39306/0 (socket says 172.21.15.1:39306) 2024-02-02T15:34:09.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.420+0000 7f3bafe00700 1 -- 172.21.15.1:0/3699395411 learned_addr learned my addr 172.21.15.1:0/3699395411 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:34:09.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.420+0000 7f3baf400700 1 --2- 172.21.15.1:0/3699395411 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3bb0102950 0x7f3bb01996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:09.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.420+0000 7f3bb4c00700 1 -- 172.21.15.1:0/3699395411 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3bb0102030 msgr2=0x7f3bb0199190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:09.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.420+0000 7f3bb4c00700 1 --2- 172.21.15.1:0/3699395411 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3bb0102030 0x7f3bb0199190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:09.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.420+0000 7f3bb4c00700 1 -- 172.21.15.1:0/3699395411 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3bb0102950 msgr2=0x7f3bb01996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:09.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.420+0000 7f3bb4c00700 1 --2- 172.21.15.1:0/3699395411 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3bb0102950 0x7f3bb01996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:09.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.420+0000 7f3bb4c00700 1 -- 172.21.15.1:0/3699395411 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ba8009d00 con 0x7f3bb010b400 2024-02-02T15:34:09.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.420+0000 7f3baf400700 1 --2- 172.21.15.1:0/3699395411 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3bb0102950 0x7f3bb01996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:34:09.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.421+0000 7f3bb4c00700 1 --2- 172.21.15.1:0/3699395411 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3bb010b400 0x7f3bb019db30 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f3ba8009fd0 tx=0x7f3ba8007880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:09.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.421+0000 7f3bacc00700 1 -- 172.21.15.1:0/3699395411 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba800e040 con 0x7f3bb010b400 2024-02-02T15:34:09.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.421+0000 7f3bacc00700 1 -- 172.21.15.1:0/3699395411 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ba801d470 con 0x7f3bb010b400 2024-02-02T15:34:09.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.421+0000 7f3bb6c00700 1 -- 172.21.15.1:0/3699395411 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bb019e720 con 0x7f3bb010b400 2024-02-02T15:34:09.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.421+0000 7f3bacc00700 1 -- 172.21.15.1:0/3699395411 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba80052a0 con 0x7f3bb010b400 2024-02-02T15:34:09.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.421+0000 7f3bb6c00700 1 -- 172.21.15.1:0/3699395411 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bb01a5fc0 con 0x7f3bb010b400 2024-02-02T15:34:09.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.424+0000 7f3bacc00700 1 -- 172.21.15.1:0/3699395411 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f3ba801d5e0 con 0x7f3bb010b400 2024-02-02T15:34:09.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.425+0000 7f3bacc00700 1 --2- 172.21.15.1:0/3699395411 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f3b900a0690 0x7f3b900a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:09.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.425+0000 7f3bacc00700 1 -- 172.21.15.1:0/3699395411 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f3ba80cd570 con 0x7f3bb010b400 2024-02-02T15:34:09.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.425+0000 7f3bca6c8b80 1 Processor -- start 2024-02-02T15:34:09.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.426+0000 7f3bafe00700 1 --2- 172.21.15.1:0/3699395411 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f3b900a0690 0x7f3b900a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:09.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.426+0000 7f3bca6c8b80 1 -- start start 2024-02-02T15:34:09.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.426+0000 7f3bca6c8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dce4399090 0x55dce4399470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:09.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.426+0000 7f3bafe00700 1 --2- 172.21.15.1:0/3699395411 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f3b900a0690 0x7f3b900a2b50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0009d20 tx=0x7f3ba0009260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:09.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.426+0000 7f3bca6c8b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dce43999b0 0x55dce43a4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:09.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.426+0000 7f3bca6c8b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dce43a4860 0x55dce43a6c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:09.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.426+0000 7f3bca6c8b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dce430d540 con 0x55dce43a4860 2024-02-02T15:34:09.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.426+0000 7f3bca6c8b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dce42f1b20 con 0x55dce43999b0 2024-02-02T15:34:09.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.426+0000 7f3bca6c8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dce42ef4e0 con 0x55dce4399090 2024-02-02T15:34:09.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.427+0000 7f3bafe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dce4399090 0x55dce4399470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:09.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.427+0000 7f3baf400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dce43999b0 0x55dce43a4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:09.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.427+0000 7f3bb4c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dce43a4860 0x55dce43a6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:09.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.427+0000 7f3baf400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dce43999b0 0x55dce43a4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:34014/0 (socket says 172.21.15.1:34014) 2024-02-02T15:34:09.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.427+0000 7f3baf400700 1 -- 172.21.15.1:0/4130389618 learned_addr learned my addr 172.21.15.1:0/4130389618 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:34:09.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.427+0000 7f3bafe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dce4399090 0x55dce4399470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:39320/0 (socket says 172.21.15.1:39320) 2024-02-02T15:34:09.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.427+0000 7f3bafe00700 1 -- 172.21.15.1:0/4130389618 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dce43999b0 msgr2=0x55dce43a4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:09.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.427+0000 7f3bafe00700 1 --2- 172.21.15.1:0/4130389618 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dce43999b0 0x55dce43a4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:09.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.427+0000 7f3bafe00700 1 -- 172.21.15.1:0/4130389618 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dce43a4860 msgr2=0x55dce43a6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:09.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.427+0000 7f3bafe00700 1 --2- 172.21.15.1:0/4130389618 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dce43a4860 0x55dce43a6c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:09.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.427+0000 7f3bafe00700 1 -- 172.21.15.1:0/4130389618 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dce416c430 con 0x55dce4399090 2024-02-02T15:34:09.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.427+0000 7f3bafe00700 1 --2- 172.21.15.1:0/4130389618 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dce4399090 0x55dce4399470 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0009be0 tx=0x7f3ba0014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a9079ac9c081681 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:09.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.428+0000 7f3ba5200700 1 -- 172.21.15.1:0/4130389618 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba000ee20 con 0x55dce4399090 2024-02-02T15:34:09.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.428+0000 7f3ba5200700 1 -- 172.21.15.1:0/4130389618 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ba0021070 con 0x55dce4399090 2024-02-02T15:34:09.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.428+0000 7f3ba5200700 1 -- 172.21.15.1:0/4130389618 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba00256c0 con 0x55dce4399090 2024-02-02T15:34:09.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.428+0000 7f3bca6c8b80 1 -- 172.21.15.1:0/4130389618 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dce4399090 msgr2=0x55dce4399470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:09.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.428+0000 7f3bca6c8b80 1 --2- 172.21.15.1:0/4130389618 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dce4399090 0x55dce4399470 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0009be0 tx=0x7f3ba0014000 comp rx=0 tx=0).stop 2024-02-02T15:34:09.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.428+0000 7f3bca6c8b80 1 -- 172.21.15.1:0/4130389618 shutdown_connections 2024-02-02T15:34:09.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.429+0000 7f3bca6c8b80 1 --2- 172.21.15.1:0/4130389618 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dce43a4860 0x55dce43a6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:09.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.429+0000 7f3bca6c8b80 1 --2- 172.21.15.1:0/4130389618 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dce4399090 0x55dce4399470 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:09.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.429+0000 7f3bca6c8b80 1 --2- 172.21.15.1:0/4130389618 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dce43999b0 0x55dce43a4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:09.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.429+0000 7f3bca6c8b80 1 -- 172.21.15.1:0/4130389618 >> 172.21.15.1:0/4130389618 conn(0x55dce42f8e30 msgr2=0x55dce4395430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:09.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.429+0000 7f3bca6c8b80 1 -- 172.21.15.1:0/4130389618 shutdown_connections 2024-02-02T15:34:09.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.429+0000 7f3bca6c8b80 1 -- 172.21.15.1:0/4130389618 wait complete. 2024-02-02T15:34:09.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.429+0000 7f3bca6c8b80 1 -- start start 2024-02-02T15:34:09.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.429+0000 7f3bca6c8b80 1 Processor -- start 2024-02-02T15:34:09.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.430+0000 7f3bca6c8b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dce4399090 0x55dce43a4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:09.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.430+0000 7f3bca6c8b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dce43999b0 0x55dce43a1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:09.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.430+0000 7f3bca6c8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dce43a4860 0x55dce43a19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:09.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.430+0000 7f3bca6c8b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dce42eadd0 con 0x55dce43999b0 2024-02-02T15:34:09.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.430+0000 7f3bca6c8b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dce4286e00 con 0x55dce4399090 2024-02-02T15:34:09.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.430+0000 7f3bca6c8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dce4286aa0 con 0x55dce43a4860 2024-02-02T15:34:09.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.430+0000 7f3bca6c8b80 10 client.0 _ll_register_callbacks cb 0x55dce4288310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:34:09.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.430+0000 7f3bb4c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dce43a4860 0x55dce43a19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:09.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.430+0000 7f3baf400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dce43999b0 0x55dce43a1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:09.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.430+0000 7f3bb4c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dce43a4860 0x55dce43a19a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:39336/0 (socket says 172.21.15.1:39336) 2024-02-02T15:34:09.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.430+0000 7f3bb4c00700 1 -- 172.21.15.1:0/3936539920 learned_addr learned my addr 172.21.15.1:0/3936539920 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:34:09.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.430+0000 7f3bafe00700 1 --2- 172.21.15.1:0/3936539920 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dce4399090 0x55dce43a4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:09.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.431+0000 7f3bb4c00700 1 -- 172.21.15.1:0/3936539920 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dce4399090 msgr2=0x55dce43a4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:09.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.431+0000 7f3bb4c00700 1 --2- 172.21.15.1:0/3936539920 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dce4399090 0x55dce43a4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:09.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.431+0000 7f3bb4c00700 1 -- 172.21.15.1:0/3936539920 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dce43999b0 msgr2=0x55dce43a1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:09.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.431+0000 7f3bb4c00700 1 --2- 172.21.15.1:0/3936539920 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dce43999b0 0x55dce43a1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:09.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.431+0000 7f3bb4c00700 1 -- 172.21.15.1:0/3936539920 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ba0017070 con 0x55dce43a4860 2024-02-02T15:34:09.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.431+0000 7f3baf400700 1 --2- 172.21.15.1:0/3936539920 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dce43999b0 0x55dce43a1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:34:09.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.431+0000 7f3bb4c00700 1 --2- 172.21.15.1:0/3936539920 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dce43a4860 0x55dce43a19a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f3ba809a420 tx=0x7f3ba809a260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3c3738b7aa28e68 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:09.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.431+0000 7f3b9d600700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:34:09.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.431+0000 7f3b9d600700 1 -- 172.21.15.1:0/3936539920 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba801daf0 con 0x55dce43a4860 2024-02-02T15:34:09.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.431+0000 7f3b9d600700 1 -- 172.21.15.1:0/3936539920 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f3ba809a720 con 0x55dce43a4860 2024-02-02T15:34:09.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.431+0000 7f3bca6c8b80 1 -- 172.21.15.1:0/3936539920 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dce438d170 con 0x55dce43a4860 2024-02-02T15:34:09.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.431+0000 7f3b9d600700 1 -- 172.21.15.1:0/3936539920 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ba809e370 con 0x55dce43a4860 2024-02-02T15:34:09.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.431+0000 7f3bca6c8b80 10 client.27580 fetch_fsmap learned FSMap version 40 2024-02-02T15:34:09.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.431+0000 7f3bca6c8b80 1 -- 172.21.15.1:0/3936539920 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55dce4118be0 con 0x55dce43a4860 2024-02-02T15:34:09.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.432+0000 7f3b9d600700 1 -- 172.21.15.1:0/3936539920 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55dce438d170 con 0x55dce43a4860 2024-02-02T15:34:09.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.432+0000 7f3b9d600700 1 -- 172.21.15.1:0/3936539920 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 40) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f3ba8099c10 con 0x55dce43a4860 2024-02-02T15:34:09.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.432+0000 7f3bca6c8b80 10 client.27580 fetch_fsmap finished waiting for FSMap version 40 2024-02-02T15:34:09.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.432+0000 7f3bca6c8b80 10 client.27580 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:34:09.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.432+0000 7f3bca6c8b80 20 client.27580 populate_metadata read hostname 'smithi001' 2024-02-02T15:34:09.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.432+0000 7f3bca6c8b80 1 --2- 172.21.15.1:0/3936539920 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55dce43b40d0 0x55dce43b64f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:09.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.432+0000 7f3bca6c8b80 4 client.27580 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:34:09.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.432+0000 7f3bca6c8b80 1 -- 172.21.15.1:0/3936539920 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dce42e3e50 con 0x55dce43b40d0 2024-02-02T15:34:09.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.432+0000 7f3bafe00700 1 --2- 172.21.15.1:0/3936539920 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55dce43b40d0 0x55dce43b64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:09.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.433+0000 7f3bafe00700 1 --2- 172.21.15.1:0/3936539920 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55dce43b40d0 0x55dce43b64f0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10a601eec2fa2ed1 server_cookie=eb9d31470bab5757 in_seq=0 out_seq=0 2024-02-02T15:34:09.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.433+0000 7f3b9d600700 10 client.27580 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:34:09.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.434+0000 7f3b9d600700 1 -- 172.21.15.1:0/3936539920 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55dce42e3e50 con 0x55dce43b40d0 2024-02-02T15:34:09.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.434+0000 7f3b9d600700 10 client.27580 handle_command_reply: tid=0 2024-02-02T15:34:09.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.434+0000 7f3bca6c8b80 1 client.27580 shutdown 2024-02-02T15:34:09.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.434+0000 7f3bca6c8b80 1 -- 172.21.15.1:0/3936539920 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dce43a4860 msgr2=0x55dce43a19a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:09.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.434+0000 7f3bca6c8b80 1 --2- 172.21.15.1:0/3936539920 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dce43a4860 0x55dce43a19a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f3ba809a420 tx=0x7f3ba809a260 comp rx=0 tx=0).stop 2024-02-02T15:34:09.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.434+0000 7f3bca6c8b80 1 -- 172.21.15.1:0/3936539920 shutdown_connections 2024-02-02T15:34:09.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.434+0000 7f3bca6c8b80 1 --2- 172.21.15.1:0/3936539920 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55dce43b40d0 0x55dce43b64f0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:09.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.434+0000 7f3bca6c8b80 1 --2- 172.21.15.1:0/3936539920 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dce43a4860 0x55dce43a19a0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:09.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.434+0000 7f3bca6c8b80 1 --2- 172.21.15.1:0/3936539920 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dce4399090 0x55dce43a4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:09.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.434+0000 7f3b9d600700 0 client.27580 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:34:09.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.434+0000 7f3bca6c8b80 1 --2- 172.21.15.1:0/3936539920 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dce43999b0 0x55dce43a1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:09.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.434+0000 7f3bca6c8b80 1 -- 172.21.15.1:0/3936539920 >> 172.21.15.1:0/3936539920 conn(0x55dce42f8e30 msgr2=0x55dce4395020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:09.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.434+0000 7f3bca6c8b80 1 -- 172.21.15.1:0/3936539920 shutdown_connections 2024-02-02T15:34:09.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.434+0000 7f3bca6c8b80 1 -- 172.21.15.1:0/3936539920 wait complete. 2024-02-02T15:34:09.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.435+0000 7f3bca6c8b80 20 client.27580 trim_cache size 0 max 16384 2024-02-02T15:34:09.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.452+0000 7f3bca6c8b80 1 Processor -- start 2024-02-02T15:34:09.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.452+0000 7f3bca6c8b80 1 -- start start 2024-02-02T15:34:09.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.452+0000 7f3bca6c8b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dce4399090 0x55dce43ab3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:09.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.452+0000 7f3bca6c8b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dce43999b0 0x55dce43a3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:09.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.452+0000 7f3bca6c8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dce43a4860 0x55dce439e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:09.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.452+0000 7f3bca6c8b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dce416e720 con 0x55dce43999b0 2024-02-02T15:34:09.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.452+0000 7f3bca6c8b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dce439a8b0 con 0x55dce4399090 2024-02-02T15:34:09.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.452+0000 7f3bca6c8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dce4396ab0 con 0x55dce43a4860 2024-02-02T15:34:09.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.452+0000 7f3baf400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dce43999b0 0x55dce43a3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:09.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.452+0000 7f3baf400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dce43999b0 0x55dce43a3420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48942/0 (socket says 172.21.15.1:48942) 2024-02-02T15:34:09.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.452+0000 7f3baf400700 1 -- 172.21.15.1:0/2831230338 learned_addr learned my addr 172.21.15.1:0/2831230338 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:34:09.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.452+0000 7f3bafe00700 1 --2- 172.21.15.1:0/2831230338 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dce4399090 0x55dce43ab3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:09.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.452+0000 7f3bb4c00700 1 --2- 172.21.15.1:0/2831230338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dce43a4860 0x55dce439e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:09.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.453+0000 7f3baf400700 1 -- 172.21.15.1:0/2831230338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dce43a4860 msgr2=0x55dce439e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:09.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.453+0000 7f3baf400700 1 --2- 172.21.15.1:0/2831230338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dce43a4860 0x55dce439e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:09.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.453+0000 7f3baf400700 1 -- 172.21.15.1:0/2831230338 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dce4399090 msgr2=0x55dce43ab3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:09.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.453+0000 7f3baf400700 1 --2- 172.21.15.1:0/2831230338 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dce4399090 0x55dce43ab3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:09.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.453+0000 7f3baf400700 1 -- 172.21.15.1:0/2831230338 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dce42b0f20 con 0x55dce43999b0 2024-02-02T15:34:09.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.453+0000 7f3baf400700 1 --2- 172.21.15.1:0/2831230338 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dce43999b0 0x55dce43a3420 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f3b98009c20 tx=0x7f3b98009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f7f52ab221dc52a server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:09.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.453+0000 7f3ba5c00700 1 -- 172.21.15.1:0/2831230338 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b98016070 con 0x55dce43999b0 2024-02-02T15:34:09.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.453+0000 7f3ba5c00700 1 -- 172.21.15.1:0/2831230338 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b98017040 con 0x55dce43999b0 2024-02-02T15:34:09.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.453+0000 7f3ba5c00700 1 -- 172.21.15.1:0/2831230338 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b9801b660 con 0x55dce43999b0 2024-02-02T15:34:09.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.454+0000 7f3bca6c8b80 1 -- 172.21.15.1:0/2831230338 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dce43999b0 msgr2=0x55dce43a3420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:09.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.454+0000 7f3bca6c8b80 1 --2- 172.21.15.1:0/2831230338 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dce43999b0 0x55dce43a3420 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f3b98009c20 tx=0x7f3b98009f30 comp rx=0 tx=0).stop 2024-02-02T15:34:09.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.454+0000 7f3bca6c8b80 1 -- 172.21.15.1:0/2831230338 shutdown_connections 2024-02-02T15:34:09.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.454+0000 7f3bca6c8b80 1 --2- 172.21.15.1:0/2831230338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dce43a4860 0x55dce439e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:09.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.454+0000 7f3bca6c8b80 1 --2- 172.21.15.1:0/2831230338 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dce4399090 0x55dce43ab3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:09.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.454+0000 7f3bca6c8b80 1 --2- 172.21.15.1:0/2831230338 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dce43999b0 0x55dce43a3420 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:09.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.454+0000 7f3bca6c8b80 1 -- 172.21.15.1:0/2831230338 >> 172.21.15.1:0/2831230338 conn(0x55dce42f8e30 msgr2=0x55dce43a2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:09.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.454+0000 7f3bca6c8b80 1 -- 172.21.15.1:0/2831230338 shutdown_connections 2024-02-02T15:34:09.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.454+0000 7f3bca6c8b80 1 -- 172.21.15.1:0/2831230338 wait complete. 2024-02-02T15:34:09.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.454+0000 7f3bca6c8b80 1 -- start start 2024-02-02T15:34:09.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.455+0000 7f3bca6c8b80 1 Processor -- start 2024-02-02T15:34:09.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.455+0000 7f3bca6c8b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dce4399090 0x55dce43fe440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:09.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.455+0000 7f3bca6c8b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dce43999b0 0x55dce43fe980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:09.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.455+0000 7f3bca6c8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dce43a4860 0x55dce43f9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:09.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.455+0000 7f3bca6c8b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dce43fa2c0 con 0x55dce4399090 2024-02-02T15:34:09.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.455+0000 7f3bca6c8b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dce43fa430 con 0x55dce43999b0 2024-02-02T15:34:09.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.455+0000 7f3bca6c8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dce43fa5f0 con 0x55dce43a4860 2024-02-02T15:34:09.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.455+0000 7f3bca6c8b80 10 client.0 _ll_register_callbacks cb 0x55dce43b0d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:34:09.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.455+0000 7f3bafe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dce4399090 0x55dce43fe440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:09.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.455+0000 7f3bafe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dce4399090 0x55dce43fe440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48958/0 (socket says 172.21.15.1:48958) 2024-02-02T15:34:09.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.455+0000 7f3bafe00700 1 -- 172.21.15.1:0/374453110 learned_addr learned my addr 172.21.15.1:0/374453110 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:34:09.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.455+0000 7f3baf400700 1 --2- 172.21.15.1:0/374453110 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dce43999b0 0x55dce43fe980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:09.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.455+0000 7f3bb4c00700 1 --2- 172.21.15.1:0/374453110 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dce43a4860 0x55dce43f9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:09.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.456+0000 7f3bafe00700 1 -- 172.21.15.1:0/374453110 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dce43a4860 msgr2=0x55dce43f9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:09.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.456+0000 7f3bafe00700 1 --2- 172.21.15.1:0/374453110 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dce43a4860 0x55dce43f9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:09.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.456+0000 7f3bafe00700 1 -- 172.21.15.1:0/374453110 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dce43999b0 msgr2=0x55dce43fe980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:09.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.456+0000 7f3bafe00700 1 --2- 172.21.15.1:0/374453110 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dce43999b0 0x55dce43fe980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:09.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.456+0000 7f3bafe00700 1 -- 172.21.15.1:0/374453110 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b980098d0 con 0x55dce4399090 2024-02-02T15:34:09.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.456+0000 7f3bb4c00700 1 --2- 172.21.15.1:0/374453110 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dce43a4860 0x55dce43f9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:34:09.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.456+0000 7f3bafe00700 1 --2- 172.21.15.1:0/374453110 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dce4399090 0x55dce43fe440 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0009340 tx=0x7f3ba000eb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c7324c5c51b6913 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:09.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.456+0000 7f3b9d600700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:34:09.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.456+0000 7f3b9d600700 1 -- 172.21.15.1:0/374453110 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba001dea0 con 0x55dce4399090 2024-02-02T15:34:09.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.456+0000 7f3bca6c8b80 1 -- 172.21.15.1:0/374453110 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3ba0021070 con 0x55dce4399090 2024-02-02T15:34:09.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.456+0000 7f3b9d600700 1 -- 172.21.15.1:0/374453110 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f3ba0033220 con 0x55dce4399090 2024-02-02T15:34:09.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.457+0000 7f3b9d600700 1 -- 172.21.15.1:0/374453110 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ba00333e0 con 0x55dce4399090 2024-02-02T15:34:09.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.457+0000 7f3bca6c8b80 10 client.17514 fetch_fsmap learned FSMap version 40 2024-02-02T15:34:09.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.457+0000 7f3bca6c8b80 1 -- 172.21.15.1:0/374453110 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55dce4170d50 con 0x55dce4399090 2024-02-02T15:34:09.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.457+0000 7f3b9d600700 1 -- 172.21.15.1:0/374453110 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ba0021070 con 0x55dce4399090 2024-02-02T15:34:09.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.457+0000 7f3b9d600700 1 -- 172.21.15.1:0/374453110 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 40) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f3ba00054d0 con 0x55dce4399090 2024-02-02T15:34:09.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.457+0000 7f3bca6c8b80 10 client.17514 fetch_fsmap finished waiting for FSMap version 40 2024-02-02T15:34:09.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.457+0000 7f3bca6c8b80 10 client.17514 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:34:09.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.457+0000 7f3bca6c8b80 20 client.17514 populate_metadata read hostname 'smithi001' 2024-02-02T15:34:09.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.457+0000 7f3bca6c8b80 1 --2- 172.21.15.1:0/374453110 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55dce44073c0 0x55dce44097e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:09.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.457+0000 7f3bca6c8b80 4 client.17514 mds_command: new command op to 24557 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-02T15:34:09.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.457+0000 7f3bca6c8b80 1 -- 172.21.15.1:0/374453110 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55dce42a9fd0 con 0x55dce44073c0 2024-02-02T15:34:09.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.457+0000 7f3baf400700 1 --2- 172.21.15.1:0/374453110 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55dce44073c0 0x55dce44097e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:09.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.458+0000 7f3baf400700 1 --2- 172.21.15.1:0/374453110 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55dce44073c0 0x55dce44097e0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad63baa2275797df server_cookie=3515adb8f02a9c3a in_seq=0 out_seq=0 2024-02-02T15:34:09.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.458+0000 7f3b9d600700 10 client.17514 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:34:09.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.459+0000 7f3b9d600700 1 -- 172.21.15.1:0/374453110 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55dce42a9fd0 con 0x55dce44073c0 2024-02-02T15:34:09.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.459+0000 7f3b9d600700 10 client.17514 handle_command_reply: tid=0 2024-02-02T15:34:09.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.459+0000 7f3bca6c8b80 1 client.17514 shutdown 2024-02-02T15:34:09.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.460+0000 7f3bca6c8b80 1 -- 172.21.15.1:0/374453110 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dce4399090 msgr2=0x55dce43fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:09.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.460+0000 7f3bca6c8b80 1 --2- 172.21.15.1:0/374453110 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dce4399090 0x55dce43fe440 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0009340 tx=0x7f3ba000eb30 comp rx=0 tx=0).stop 2024-02-02T15:34:09.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.460+0000 7f3bca6c8b80 1 -- 172.21.15.1:0/374453110 shutdown_connections 2024-02-02T15:34:09.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.460+0000 7f3bca6c8b80 1 --2- 172.21.15.1:0/374453110 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55dce44073c0 0x55dce44097e0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:09.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.460+0000 7f3bca6c8b80 1 --2- 172.21.15.1:0/374453110 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dce43a4860 0x55dce43f9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:09.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.460+0000 7f3bca6c8b80 1 --2- 172.21.15.1:0/374453110 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dce4399090 0x55dce43fe440 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:09.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.460+0000 7f3bca6c8b80 1 --2- 172.21.15.1:0/374453110 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dce43999b0 0x55dce43fe980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:09.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.460+0000 7f3b9d600700 0 client.17514 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:34:09.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.460+0000 7f3bca6c8b80 1 -- 172.21.15.1:0/374453110 >> 172.21.15.1:0/374453110 conn(0x55dce42f8e30 msgr2=0x55dce43a35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:09.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.460+0000 7f3bca6c8b80 1 -- 172.21.15.1:0/374453110 shutdown_connections 2024-02-02T15:34:09.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.460+0000 7f3bca6c8b80 1 -- 172.21.15.1:0/374453110 wait complete. 2024-02-02T15:34:09.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.460+0000 7f3bca6c8b80 20 client.17514 trim_cache size 0 max 16384 2024-02-02T15:34:09.463 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:34:09.463 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2024-02-02T15:34:09.463 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "287ce58a-8981-4868-8e0c-0936ba078166", 2024-02-02T15:34:09.463 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2024-02-02T15:34:09.464 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:34:09.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.461+0000 7f3bb6c00700 1 -- 172.21.15.1:0/3699395411 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f3b900a0690 msgr2=0x7f3b900a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:09.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.461+0000 7f3bb6c00700 1 --2- 172.21.15.1:0/3699395411 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f3b900a0690 0x7f3b900a2b50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0009d20 tx=0x7f3ba0009260 comp rx=0 tx=0).stop 2024-02-02T15:34:09.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.461+0000 7f3bb6c00700 1 -- 172.21.15.1:0/3699395411 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3bb010b400 msgr2=0x7f3bb019db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:09.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.461+0000 7f3bb6c00700 1 --2- 172.21.15.1:0/3699395411 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3bb010b400 0x7f3bb019db30 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f3ba8009fd0 tx=0x7f3ba8007880 comp rx=0 tx=0).stop 2024-02-02T15:34:09.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.461+0000 7f3bb6c00700 1 -- 172.21.15.1:0/3699395411 shutdown_connections 2024-02-02T15:34:09.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.461+0000 7f3bb6c00700 1 --2- 172.21.15.1:0/3699395411 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f3b900a0690 0x7f3b900a2b50 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:09.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.461+0000 7f3bb6c00700 1 --2- 172.21.15.1:0/3699395411 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3bb010b400 0x7f3bb019db30 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:09.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.461+0000 7f3bb6c00700 1 --2- 172.21.15.1:0/3699395411 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3bb0102030 0x7f3bb0199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:09.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.461+0000 7f3bb6c00700 1 --2- 172.21.15.1:0/3699395411 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3bb0102950 0x7f3bb01996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:09.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.461+0000 7f3bb6c00700 1 -- 172.21.15.1:0/3699395411 >> 172.21.15.1:0/3699395411 conn(0x7f3bb0076d70 msgr2=0x7f3bb00ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:09.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.461+0000 7f3bb6c00700 1 -- 172.21.15.1:0/3699395411 shutdown_connections 2024-02-02T15:34:09.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.461+0000 7f3bb6c00700 1 -- 172.21.15.1:0/3699395411 wait complete. 2024-02-02T15:34:09.479 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-02T15:34:09.581 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:09 smithi001 ceph-mon[35470]: pgmap v218: 129 pgs: 129 active+clean; 289 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 193 op/s 2024-02-02T15:34:09.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:09 smithi142 ceph-mon[45728]: pgmap v218: 129 pgs: 129 active+clean; 289 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 193 op/s 2024-02-02T15:34:09.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.746+0000 7f0e8b600700 1 -- 172.21.15.1:0/4021165736 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e8410da10 msgr2=0x7f0e8410fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:09.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.746+0000 7f0e8b600700 1 --2- 172.21.15.1:0/4021165736 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e8410da10 0x7f0e8410fe00 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c00b0a0 tx=0x7f0e7c00b3b0 comp rx=0 tx=0).stop 2024-02-02T15:34:09.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.746+0000 7f0e8b600700 1 -- 172.21.15.1:0/4021165736 shutdown_connections 2024-02-02T15:34:09.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.746+0000 7f0e8b600700 1 --2- 172.21.15.1:0/4021165736 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e8410da10 0x7f0e8410fe00 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:09.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.746+0000 7f0e8b600700 1 --2- 172.21.15.1:0/4021165736 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e84068cc0 0x7f0e840690a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:09.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.746+0000 7f0e8b600700 1 --2- 172.21.15.1:0/4021165736 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e840695e0 0x7f0e8410d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:09.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.746+0000 7f0e8b600700 1 -- 172.21.15.1:0/4021165736 >> 172.21.15.1:0/4021165736 conn(0x7f0e84077bf0 msgr2=0x7f0e84078000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:09.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.746+0000 7f0e8b600700 1 -- 172.21.15.1:0/4021165736 shutdown_connections 2024-02-02T15:34:09.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.746+0000 7f0e8b600700 1 -- 172.21.15.1:0/4021165736 wait complete. 2024-02-02T15:34:09.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.747+0000 7f0e8b600700 1 Processor -- start 2024-02-02T15:34:09.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.747+0000 7f0e8b600700 1 -- start start 2024-02-02T15:34:09.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.748+0000 7f0e8b600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e84068cc0 0x7f0e84199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:09.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.748+0000 7f0e8b600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e840695e0 0x7f0e841997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:09.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.748+0000 7f0e8b600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e8410da10 0x7f0e8419dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:09.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.748+0000 7f0e8b600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8419e270 con 0x7f0e84068cc0 2024-02-02T15:34:09.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.748+0000 7f0e8b600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8419e3e0 con 0x7f0e8410da10 2024-02-02T15:34:09.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.748+0000 7f0e8b600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8419e5a0 con 0x7f0e840695e0 2024-02-02T15:34:09.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.748+0000 7f0e88c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e84068cc0 0x7f0e84199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:09.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.748+0000 7f0e89600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e8410da10 0x7f0e8419dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:09.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.748+0000 7f0e83e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e840695e0 0x7f0e841997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:09.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.748+0000 7f0e88c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e84068cc0 0x7f0e84199290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48974/0 (socket says 172.21.15.1:48974) 2024-02-02T15:34:09.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.748+0000 7f0e88c00700 1 -- 172.21.15.1:0/607935360 learned_addr learned my addr 172.21.15.1:0/607935360 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:34:09.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.748+0000 7f0e89600700 1 -- 172.21.15.1:0/607935360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e840695e0 msgr2=0x7f0e841997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:09.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.748+0000 7f0e89600700 1 --2- 172.21.15.1:0/607935360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e840695e0 0x7f0e841997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:09.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.748+0000 7f0e89600700 1 -- 172.21.15.1:0/607935360 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e84068cc0 msgr2=0x7f0e84199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:09.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.748+0000 7f0e89600700 1 --2- 172.21.15.1:0/607935360 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e84068cc0 0x7f0e84199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:09.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.748+0000 7f0e89600700 1 -- 172.21.15.1:0/607935360 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e7c009d00 con 0x7f0e8410da10 2024-02-02T15:34:09.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.749+0000 7f0e89600700 1 --2- 172.21.15.1:0/607935360 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e8410da10 0x7f0e8419dc30 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c009fd0 tx=0x7f0e7c007930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:09.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.749+0000 7f0e81600700 1 -- 172.21.15.1:0/607935360 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c00e040 con 0x7f0e8410da10 2024-02-02T15:34:09.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.749+0000 7f0e81600700 1 -- 172.21.15.1:0/607935360 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e7c00be80 con 0x7f0e8410da10 2024-02-02T15:34:09.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.749+0000 7f0e8b600700 1 -- 172.21.15.1:0/607935360 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e8419e820 con 0x7f0e8410da10 2024-02-02T15:34:09.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.749+0000 7f0e81600700 1 -- 172.21.15.1:0/607935360 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c005040 con 0x7f0e8410da10 2024-02-02T15:34:09.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.749+0000 7f0e8b600700 1 -- 172.21.15.1:0/607935360 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e841a60c0 con 0x7f0e8410da10 2024-02-02T15:34:09.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.752+0000 7f0e81600700 1 -- 172.21.15.1:0/607935360 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f0e7c01d470 con 0x7f0e8410da10 2024-02-02T15:34:09.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.753+0000 7f0e9f07bb80 1 Processor -- start 2024-02-02T15:34:09.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.754+0000 7f0e9f07bb80 1 -- start start 2024-02-02T15:34:09.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.754+0000 7f0e81600700 1 --2- 172.21.15.1:0/607935360 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f0e640a0710 0x7f0e640a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:09.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.754+0000 7f0e9f07bb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d61e139010 0x55d61e1393f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:09.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.754+0000 7f0e9f07bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d61e139930 0x55d61e1442a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:09.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.754+0000 7f0e9f07bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d61e1447e0 0x55d61e146bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:09.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.754+0000 7f0e9f07bb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d61e0ad4c0 con 0x55d61e139010 2024-02-02T15:34:09.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.754+0000 7f0e9f07bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d61e091aa0 con 0x55d61e1447e0 2024-02-02T15:34:09.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.754+0000 7f0e9f07bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d61e08f460 con 0x55d61e139930 2024-02-02T15:34:09.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.754+0000 7f0e81600700 1 -- 172.21.15.1:0/607935360 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f0e7c0cd6c0 con 0x7f0e8410da10 2024-02-02T15:34:09.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.754+0000 7f0e83e00700 1 --2- 172.21.15.1:0/607935360 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f0e640a0710 0x7f0e640a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:09.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.754+0000 7f0e89600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d61e1447e0 0x55d61e146bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:09.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.754+0000 7f0e88c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d61e139010 0x55d61e1393f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:09.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.754+0000 7f0e89600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d61e1447e0 0x55d61e146bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:34080/0 (socket says 172.21.15.1:34080) 2024-02-02T15:34:09.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.754+0000 7f0e89600700 1 -- 172.21.15.1:0/4022218279 learned_addr learned my addr 172.21.15.1:0/4022218279 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:34:09.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.754+0000 7f0e83e00700 1 --2- 172.21.15.1:0/4022218279 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d61e139930 0x55d61e1442a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:09.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.755+0000 7f0e89600700 1 -- 172.21.15.1:0/4022218279 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d61e139930 msgr2=0x55d61e1442a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:09.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.755+0000 7f0e89600700 1 --2- 172.21.15.1:0/4022218279 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d61e139930 0x55d61e1442a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:09.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.755+0000 7f0e89600700 1 -- 172.21.15.1:0/4022218279 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d61e139010 msgr2=0x55d61e1393f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:09.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.755+0000 7f0e89600700 1 --2- 172.21.15.1:0/4022218279 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d61e139010 0x55d61e1393f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:09.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.755+0000 7f0e89600700 1 -- 172.21.15.1:0/4022218279 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d61df0c3b0 con 0x55d61e1447e0 2024-02-02T15:34:09.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.755+0000 7f0e89600700 1 --2- 172.21.15.1:0/4022218279 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d61e1447e0 0x55d61e146bd0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c09a510 tx=0x7f0e7c09a3d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=442e33d2d9e0fee1 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:09.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.755+0000 7f0e79c00700 1 -- 172.21.15.1:0/4022218279 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c09af10 con 0x55d61e1447e0 2024-02-02T15:34:09.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.755+0000 7f0e83e00700 1 --2- 172.21.15.1:0/607935360 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f0e640a0710 0x7f0e640a2bd0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0e8419a8a0 tx=0x7f0e6c007710 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:09.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.755+0000 7f0e79c00700 1 -- 172.21.15.1:0/4022218279 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e7c09a810 con 0x55d61e1447e0 2024-02-02T15:34:09.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.755+0000 7f0e79c00700 1 -- 172.21.15.1:0/4022218279 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c01d720 con 0x55d61e1447e0 2024-02-02T15:34:09.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.756+0000 7f0e9f07bb80 1 -- 172.21.15.1:0/4022218279 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d61e1447e0 msgr2=0x55d61e146bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:09.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.756+0000 7f0e9f07bb80 1 --2- 172.21.15.1:0/4022218279 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d61e1447e0 0x55d61e146bd0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c09a510 tx=0x7f0e7c09a3d0 comp rx=0 tx=0).stop 2024-02-02T15:34:09.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.756+0000 7f0e9f07bb80 1 -- 172.21.15.1:0/4022218279 shutdown_connections 2024-02-02T15:34:09.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.756+0000 7f0e9f07bb80 1 --2- 172.21.15.1:0/4022218279 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d61e1447e0 0x55d61e146bd0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:09.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.756+0000 7f0e9f07bb80 1 --2- 172.21.15.1:0/4022218279 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d61e139010 0x55d61e1393f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:09.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.756+0000 7f0e9f07bb80 1 --2- 172.21.15.1:0/4022218279 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d61e139930 0x55d61e1442a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:09.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.756+0000 7f0e9f07bb80 1 -- 172.21.15.1:0/4022218279 >> 172.21.15.1:0/4022218279 conn(0x55d61e098db0 msgr2=0x55d61e1353b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:09.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.756+0000 7f0e9f07bb80 1 -- 172.21.15.1:0/4022218279 shutdown_connections 2024-02-02T15:34:09.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.756+0000 7f0e9f07bb80 1 -- 172.21.15.1:0/4022218279 wait complete. 2024-02-02T15:34:09.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.757+0000 7f0e9f07bb80 1 -- start start 2024-02-02T15:34:09.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.757+0000 7f0e9f07bb80 1 Processor -- start 2024-02-02T15:34:09.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.757+0000 7f0e9f07bb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d61e139010 0x55d61e144170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:09.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.757+0000 7f0e9f07bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d61e139930 0x55d61e1413f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:09.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.757+0000 7f0e9f07bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d61e1447e0 0x55d61e141990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:09.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.757+0000 7f0e9f07bb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d61e08ad50 con 0x55d61e139010 2024-02-02T15:34:09.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.757+0000 7f0e9f07bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d61e026d80 con 0x55d61e1447e0 2024-02-02T15:34:09.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.757+0000 7f0e9f07bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d61e026a20 con 0x55d61e139930 2024-02-02T15:34:09.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.758+0000 7f0e9f07bb80 10 client.0 _ll_register_callbacks cb 0x55d61e028290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:34:09.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.758+0000 7f0e89600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d61e1447e0 0x55d61e141990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:09.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.758+0000 7f0e83e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d61e139930 0x55d61e1413f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:09.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.758+0000 7f0e89600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d61e1447e0 0x55d61e141990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:34092/0 (socket says 172.21.15.1:34092) 2024-02-02T15:34:09.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.758+0000 7f0e89600700 1 -- 172.21.15.1:0/1204258660 learned_addr learned my addr 172.21.15.1:0/1204258660 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:34:09.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.758+0000 7f0e88c00700 1 --2- 172.21.15.1:0/1204258660 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d61e139010 0x55d61e144170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:09.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.758+0000 7f0e89600700 1 -- 172.21.15.1:0/1204258660 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d61e139930 msgr2=0x55d61e1413f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:09.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.758+0000 7f0e89600700 1 --2- 172.21.15.1:0/1204258660 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d61e139930 0x55d61e1413f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:09.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.758+0000 7f0e89600700 1 -- 172.21.15.1:0/1204258660 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d61e139010 msgr2=0x55d61e144170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:09.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.758+0000 7f0e89600700 1 --2- 172.21.15.1:0/1204258660 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d61e139010 0x55d61e144170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:09.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.758+0000 7f0e89600700 1 -- 172.21.15.1:0/1204258660 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e841a60c0 con 0x55d61e1447e0 2024-02-02T15:34:09.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.758+0000 7f0e89600700 1 --2- 172.21.15.1:0/1204258660 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d61e1447e0 0x55d61e141990 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c01db50 tx=0x7f0e7c009a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0243507519da112 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:09.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.758+0000 7f0e72000700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:34:09.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.758+0000 7f0e72000700 1 -- 172.21.15.1:0/1204258660 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c01d930 con 0x55d61e1447e0 2024-02-02T15:34:09.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.759+0000 7f0e72000700 1 -- 172.21.15.1:0/1204258660 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f0e7c0a27f0 con 0x55d61e1447e0 2024-02-02T15:34:09.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.759+0000 7f0e9f07bb80 1 -- 172.21.15.1:0/1204258660 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d61e12d0f0 con 0x55d61e1447e0 2024-02-02T15:34:09.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.759+0000 7f0e72000700 1 -- 172.21.15.1:0/1204258660 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0e7c011e30 con 0x55d61e1447e0 2024-02-02T15:34:09.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.759+0000 7f0e9f07bb80 10 client.27602 fetch_fsmap learned FSMap version 40 2024-02-02T15:34:09.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.759+0000 7f0e9f07bb80 1 -- 172.21.15.1:0/1204258660 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55d61deb8b60 con 0x55d61e1447e0 2024-02-02T15:34:09.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.759+0000 7f0e72000700 1 -- 172.21.15.1:0/1204258660 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d61e12d0f0 con 0x55d61e1447e0 2024-02-02T15:34:09.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.759+0000 7f0e72000700 1 -- 172.21.15.1:0/1204258660 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 40) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f0e7c0a2360 con 0x55d61e1447e0 2024-02-02T15:34:09.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.759+0000 7f0e9f07bb80 10 client.27602 fetch_fsmap finished waiting for FSMap version 40 2024-02-02T15:34:09.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.759+0000 7f0e9f07bb80 10 client.27602 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:34:09.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.759+0000 7f0e9f07bb80 20 client.27602 populate_metadata read hostname 'smithi001' 2024-02-02T15:34:09.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.759+0000 7f0e9f07bb80 1 --2- 172.21.15.1:0/1204258660 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55d61e153be0 0x55d61e156000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:09.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.759+0000 7f0e9f07bb80 4 client.27602 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:34:09.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.759+0000 7f0e9f07bb80 1 -- 172.21.15.1:0/1204258660 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d61e083dd0 con 0x55d61e153be0 2024-02-02T15:34:09.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.760+0000 7f0e88c00700 1 --2- 172.21.15.1:0/1204258660 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55d61e153be0 0x55d61e156000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:09.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.760+0000 7f0e88c00700 1 --2- 172.21.15.1:0/1204258660 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55d61e153be0 0x55d61e156000 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f43c58257a907800 server_cookie=282ba9a1f46d9b55 in_seq=0 out_seq=0 2024-02-02T15:34:09.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.760+0000 7f0e72000700 10 client.27602 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:34:09.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.761+0000 7f0e72000700 1 -- 172.21.15.1:0/1204258660 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55d61e083dd0 con 0x55d61e153be0 2024-02-02T15:34:09.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.761+0000 7f0e72000700 10 client.27602 handle_command_reply: tid=0 2024-02-02T15:34:09.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.761+0000 7f0e9f07bb80 1 client.27602 shutdown 2024-02-02T15:34:09.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.761+0000 7f0e9f07bb80 1 -- 172.21.15.1:0/1204258660 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d61e1447e0 msgr2=0x55d61e141990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:09.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.761+0000 7f0e9f07bb80 1 --2- 172.21.15.1:0/1204258660 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d61e1447e0 0x55d61e141990 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c01db50 tx=0x7f0e7c009a60 comp rx=0 tx=0).stop 2024-02-02T15:34:09.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.761+0000 7f0e9f07bb80 1 -- 172.21.15.1:0/1204258660 shutdown_connections 2024-02-02T15:34:09.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.761+0000 7f0e9f07bb80 1 --2- 172.21.15.1:0/1204258660 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55d61e153be0 0x55d61e156000 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:09.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.761+0000 7f0e9f07bb80 1 --2- 172.21.15.1:0/1204258660 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d61e1447e0 0x55d61e141990 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:09.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.761+0000 7f0e9f07bb80 1 --2- 172.21.15.1:0/1204258660 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d61e139010 0x55d61e144170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:09.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.761+0000 7f0e9f07bb80 1 --2- 172.21.15.1:0/1204258660 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d61e139930 0x55d61e1413f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:09.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.761+0000 7f0e9f07bb80 1 -- 172.21.15.1:0/1204258660 >> 172.21.15.1:0/1204258660 conn(0x55d61e098db0 msgr2=0x55d61e134fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:09.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.761+0000 7f0e72000700 0 client.27602 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:34:09.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.761+0000 7f0e9f07bb80 1 -- 172.21.15.1:0/1204258660 shutdown_connections 2024-02-02T15:34:09.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.761+0000 7f0e9f07bb80 1 -- 172.21.15.1:0/1204258660 wait complete. 2024-02-02T15:34:09.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.762+0000 7f0e9f07bb80 20 client.27602 trim_cache size 0 max 16384 2024-02-02T15:34:09.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.779+0000 7f0e9f07bb80 1 Processor -- start 2024-02-02T15:34:09.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.779+0000 7f0e9f07bb80 1 -- start start 2024-02-02T15:34:09.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.780+0000 7f0e9f07bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d61e139010 0x55d61e143ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:09.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.780+0000 7f0e9f07bb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d61e139930 0x55d61e14f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:09.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.780+0000 7f0e9f07bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d61e1447e0 0x55d61e13c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:09.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.780+0000 7f0e9f07bb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d61df0e6a0 con 0x55d61e139930 2024-02-02T15:34:09.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.780+0000 7f0e9f07bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d61e141f90 con 0x55d61e1447e0 2024-02-02T15:34:09.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.780+0000 7f0e9f07bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d61e13d2e0 con 0x55d61e139010 2024-02-02T15:34:09.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.780+0000 7f0e88c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d61e139010 0x55d61e143ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:09.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.780+0000 7f0e89600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d61e1447e0 0x55d61e13c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:09.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.780+0000 7f0e88c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d61e139010 0x55d61e143ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:39402/0 (socket says 172.21.15.1:39402) 2024-02-02T15:34:09.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.780+0000 7f0e88c00700 1 -- 172.21.15.1:0/2757902299 learned_addr learned my addr 172.21.15.1:0/2757902299 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:34:09.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.780+0000 7f0e83e00700 1 --2- 172.21.15.1:0/2757902299 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d61e139930 0x55d61e14f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:09.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.780+0000 7f0e88c00700 1 -- 172.21.15.1:0/2757902299 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d61e1447e0 msgr2=0x55d61e13c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:09.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.780+0000 7f0e88c00700 1 --2- 172.21.15.1:0/2757902299 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d61e1447e0 0x55d61e13c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:09.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.780+0000 7f0e88c00700 1 -- 172.21.15.1:0/2757902299 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d61e139930 msgr2=0x55d61e14f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:09.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.781+0000 7f0e88c00700 1 --2- 172.21.15.1:0/2757902299 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d61e139930 0x55d61e14f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:09.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.781+0000 7f0e88c00700 1 -- 172.21.15.1:0/2757902299 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d61e050ea0 con 0x55d61e139010 2024-02-02T15:34:09.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.781+0000 7f0e83e00700 1 --2- 172.21.15.1:0/2757902299 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d61e139930 0x55d61e14f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:34:09.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.781+0000 7f0e88c00700 1 --2- 172.21.15.1:0/2757902299 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d61e139010 0x55d61e143ca0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x55d61e1568d0 tx=0x7f0e74009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=693d8feb6c72d0c5 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:09.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.781+0000 7f0e7a600700 1 -- 172.21.15.1:0/2757902299 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7401f130 con 0x55d61e139010 2024-02-02T15:34:09.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.781+0000 7f0e7a600700 1 -- 172.21.15.1:0/2757902299 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e74018040 con 0x55d61e139010 2024-02-02T15:34:09.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.781+0000 7f0e7a600700 1 -- 172.21.15.1:0/2757902299 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e74003cb0 con 0x55d61e139010 2024-02-02T15:34:09.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.782+0000 7f0e9f07bb80 1 -- 172.21.15.1:0/2757902299 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d61e139010 msgr2=0x55d61e143ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:09.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.782+0000 7f0e9f07bb80 1 --2- 172.21.15.1:0/2757902299 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d61e139010 0x55d61e143ca0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x55d61e1568d0 tx=0x7f0e74009260 comp rx=0 tx=0).stop 2024-02-02T15:34:09.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.782+0000 7f0e9f07bb80 1 -- 172.21.15.1:0/2757902299 shutdown_connections 2024-02-02T15:34:09.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.782+0000 7f0e9f07bb80 1 --2- 172.21.15.1:0/2757902299 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d61e1447e0 0x55d61e13c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:09.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.782+0000 7f0e9f07bb80 1 --2- 172.21.15.1:0/2757902299 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d61e139010 0x55d61e143ca0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:09.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.782+0000 7f0e9f07bb80 1 --2- 172.21.15.1:0/2757902299 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d61e139930 0x55d61e14f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:09.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.782+0000 7f0e9f07bb80 1 -- 172.21.15.1:0/2757902299 >> 172.21.15.1:0/2757902299 conn(0x55d61e098db0 msgr2=0x55d61e155bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:09.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.782+0000 7f0e9f07bb80 1 -- 172.21.15.1:0/2757902299 shutdown_connections 2024-02-02T15:34:09.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.782+0000 7f0e9f07bb80 1 -- 172.21.15.1:0/2757902299 wait complete. 2024-02-02T15:34:09.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.782+0000 7f0e9f07bb80 1 -- start start 2024-02-02T15:34:09.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.783+0000 7f0e9f07bb80 1 Processor -- start 2024-02-02T15:34:09.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.783+0000 7f0e9f07bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d61e139010 0x55d61e14ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:09.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.783+0000 7f0e9f07bb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d61e139930 0x55d61e149120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:09.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.783+0000 7f0e9f07bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d61e1447e0 0x55d61e149660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:09.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.783+0000 7f0e9f07bb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d61e149ba0 con 0x55d61e139930 2024-02-02T15:34:09.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.783+0000 7f0e9f07bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d61e149d10 con 0x55d61e139010 2024-02-02T15:34:09.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.783+0000 7f0e9f07bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d61e149ed0 con 0x55d61e1447e0 2024-02-02T15:34:09.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.783+0000 7f0e9f07bb80 10 client.0 _ll_register_callbacks cb 0x55d61e028290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:34:09.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.783+0000 7f0e88c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d61e139010 0x55d61e14ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:09.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.783+0000 7f0e88c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d61e139010 0x55d61e14ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:34104/0 (socket says 172.21.15.1:34104) 2024-02-02T15:34:09.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.783+0000 7f0e88c00700 1 -- 172.21.15.1:0/4046370362 learned_addr learned my addr 172.21.15.1:0/4046370362 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:34:09.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.783+0000 7f0e83e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d61e139930 0x55d61e149120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:09.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.784+0000 7f0e89600700 1 --2- 172.21.15.1:0/4046370362 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d61e1447e0 0x55d61e149660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:09.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.784+0000 7f0e88c00700 1 -- 172.21.15.1:0/4046370362 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d61e1447e0 msgr2=0x55d61e149660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:09.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.784+0000 7f0e88c00700 1 --2- 172.21.15.1:0/4046370362 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d61e1447e0 0x55d61e149660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:09.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.784+0000 7f0e88c00700 1 -- 172.21.15.1:0/4046370362 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d61e139930 msgr2=0x55d61e149120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:09.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.784+0000 7f0e88c00700 1 --2- 172.21.15.1:0/4046370362 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d61e139930 0x55d61e149120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:09.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.784+0000 7f0e88c00700 1 -- 172.21.15.1:0/4046370362 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e74009c50 con 0x55d61e139010 2024-02-02T15:34:09.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.784+0000 7f0e88c00700 1 --2- 172.21.15.1:0/4046370362 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d61e139010 0x55d61e14ae30 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f0e740096b0 tx=0x7f0e7401f290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e348382927a0f21f server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:09.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.784+0000 7f0e72000700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:34:09.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.784+0000 7f0e72000700 1 -- 172.21.15.1:0/4046370362 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e74004460 con 0x55d61e139010 2024-02-02T15:34:09.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.784+0000 7f0e72000700 1 -- 172.21.15.1:0/4046370362 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f0e74003890 con 0x55d61e139010 2024-02-02T15:34:09.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.784+0000 7f0e9f07bb80 1 -- 172.21.15.1:0/4046370362 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0e7c09a810 con 0x55d61e139010 2024-02-02T15:34:09.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.785+0000 7f0e72000700 1 -- 172.21.15.1:0/4046370362 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0e74003a50 con 0x55d61e139010 2024-02-02T15:34:09.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.785+0000 7f0e9f07bb80 10 client.27608 fetch_fsmap learned FSMap version 40 2024-02-02T15:34:09.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.785+0000 7f0e9f07bb80 1 -- 172.21.15.1:0/4046370362 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55d61df10cd0 con 0x55d61e139010 2024-02-02T15:34:09.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.785+0000 7f0e72000700 1 -- 172.21.15.1:0/4046370362 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e7c09a810 con 0x55d61e139010 2024-02-02T15:34:09.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.785+0000 7f0e72000700 1 -- 172.21.15.1:0/4046370362 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 40) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f0e74005040 con 0x55d61e139010 2024-02-02T15:34:09.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.785+0000 7f0e9f07bb80 10 client.27608 fetch_fsmap finished waiting for FSMap version 40 2024-02-02T15:34:09.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.785+0000 7f0e9f07bb80 10 client.27608 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:34:09.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.785+0000 7f0e9f07bb80 20 client.27608 populate_metadata read hostname 'smithi001' 2024-02-02T15:34:09.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.785+0000 7f0e9f07bb80 1 --2- 172.21.15.1:0/4046370362 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55d61e1a6a60 0x55d61e1a8e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:09.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.785+0000 7f0e9f07bb80 4 client.27608 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2024-02-02T15:34:09.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.785+0000 7f0e9f07bb80 1 -- 172.21.15.1:0/4046370362 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d61e049f50 con 0x55d61e1a6a60 2024-02-02T15:34:09.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.785+0000 7f0e83e00700 1 --2- 172.21.15.1:0/4046370362 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55d61e1a6a60 0x55d61e1a8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:09.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.786+0000 7f0e72000700 10 client.27608 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:34:09.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.786+0000 7f0e83e00700 1 --2- 172.21.15.1:0/4046370362 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55d61e1a6a60 0x55d61e1a8e80 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed0bed461fe12d85 server_cookie=7609762f7638398d in_seq=0 out_seq=0 2024-02-02T15:34:09.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.788+0000 7f0e72000700 1 -- 172.21.15.1:0/4046370362 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55d61e049f50 con 0x55d61e1a6a60 2024-02-02T15:34:09.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.788+0000 7f0e72000700 10 client.27608 handle_command_reply: tid=0 2024-02-02T15:34:09.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.788+0000 7f0e9f07bb80 1 client.27608 shutdown 2024-02-02T15:34:09.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.788+0000 7f0e9f07bb80 1 -- 172.21.15.1:0/4046370362 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d61e139010 msgr2=0x55d61e14ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:09.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.788+0000 7f0e9f07bb80 1 --2- 172.21.15.1:0/4046370362 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d61e139010 0x55d61e14ae30 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f0e740096b0 tx=0x7f0e7401f290 comp rx=0 tx=0).stop 2024-02-02T15:34:09.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.788+0000 7f0e9f07bb80 1 -- 172.21.15.1:0/4046370362 shutdown_connections 2024-02-02T15:34:09.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.788+0000 7f0e9f07bb80 1 --2- 172.21.15.1:0/4046370362 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55d61e1a6a60 0x55d61e1a8e80 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:09.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.788+0000 7f0e9f07bb80 1 --2- 172.21.15.1:0/4046370362 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d61e1447e0 0x55d61e149660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:09.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.788+0000 7f0e9f07bb80 1 --2- 172.21.15.1:0/4046370362 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d61e139010 0x55d61e14ae30 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:09.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.788+0000 7f0e9f07bb80 1 --2- 172.21.15.1:0/4046370362 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d61e139930 0x55d61e149120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:09.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.788+0000 7f0e72000700 0 client.27608 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:34:09.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.788+0000 7f0e9f07bb80 1 -- 172.21.15.1:0/4046370362 >> 172.21.15.1:0/4046370362 conn(0x55d61e098db0 msgr2=0x55d61e13c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:09.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.788+0000 7f0e9f07bb80 1 -- 172.21.15.1:0/4046370362 shutdown_connections 2024-02-02T15:34:09.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.788+0000 7f0e9f07bb80 1 -- 172.21.15.1:0/4046370362 wait complete. 2024-02-02T15:34:09.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.788+0000 7f0e9f07bb80 20 client.27608 trim_cache size 0 max 16384 2024-02-02T15:34:09.791 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:34:09.791 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-02T15:34:09.791 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2024-02-02T15:34:09.791 INFO:teuthology.orchestra.run.smithi001.stdout: "287ce58a-8981-4868-8e0c-0936ba078166": { 2024-02-02T15:34:09.791 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2024-02-02T15:34:09.791 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "287ce58a-8981-4868-8e0c-0936ba078166", 2024-02-02T15:34:09.791 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,force" 2024-02-02T15:34:09.791 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:34:09.791 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:34:09.791 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:34:09.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.789+0000 7f0e8b600700 1 -- 172.21.15.1:0/607935360 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f0e640a0710 msgr2=0x7f0e640a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:09.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.789+0000 7f0e8b600700 1 --2- 172.21.15.1:0/607935360 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f0e640a0710 0x7f0e640a2bd0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0e8419a8a0 tx=0x7f0e6c007710 comp rx=0 tx=0).stop 2024-02-02T15:34:09.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.789+0000 7f0e8b600700 1 -- 172.21.15.1:0/607935360 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e8410da10 msgr2=0x7f0e8419dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:09.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.789+0000 7f0e8b600700 1 --2- 172.21.15.1:0/607935360 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e8410da10 0x7f0e8419dc30 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c009fd0 tx=0x7f0e7c007930 comp rx=0 tx=0).stop 2024-02-02T15:34:09.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.789+0000 7f0e8b600700 1 -- 172.21.15.1:0/607935360 shutdown_connections 2024-02-02T15:34:09.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.789+0000 7f0e8b600700 1 --2- 172.21.15.1:0/607935360 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f0e640a0710 0x7f0e640a2bd0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:09.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.789+0000 7f0e8b600700 1 --2- 172.21.15.1:0/607935360 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e8410da10 0x7f0e8419dc30 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:09.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.789+0000 7f0e8b600700 1 --2- 172.21.15.1:0/607935360 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e84068cc0 0x7f0e84199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:09.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.789+0000 7f0e8b600700 1 --2- 172.21.15.1:0/607935360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e840695e0 0x7f0e841997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:09.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.789+0000 7f0e8b600700 1 -- 172.21.15.1:0/607935360 >> 172.21.15.1:0/607935360 conn(0x7f0e84077bf0 msgr2=0x7f0e8410ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:09.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.789+0000 7f0e8b600700 1 -- 172.21.15.1:0/607935360 shutdown_connections 2024-02-02T15:34:09.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:09.789+0000 7f0e8b600700 1 -- 172.21.15.1:0/607935360 wait complete. 2024-02-02T15:34:09.808 INFO:tasks.cephfs.filesystem:scrub status for tag:287ce58a-8981-4868-8e0c-0936ba078166 - {'path': '/', 'tag': '287ce58a-8981-4868-8e0c-0936ba078166', 'options': 'recursive,force'} 2024-02-02T15:34:09.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:09 smithi071 ceph-mon[44707]: pgmap v218: 129 pgs: 129 active+clean; 289 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 193 op/s 2024-02-02T15:34:10.243 INFO:tasks.workunit.client.0.smithi001.stdout: 1 450113 7.45 MB/sec execute 224 sec latency 2804.461 ms 2024-02-02T15:34:10.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:10 smithi142 ceph-mon[45728]: scrub queued for path: / 2024-02-02T15:34:10.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:10 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:34:10.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:10 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:34:10.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:10 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:34:10.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:10 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:34:10.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:10 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:34:10.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:10 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:34:10.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:10 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:34:10.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:10 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:34:10.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:10 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:34:10.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:10 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:34:10.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:10 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:34:10.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:10 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:34:10.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:10 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:34:10.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:10 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:34:10.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:10 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:34:10.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:10 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:34:10.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:10 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:34:10.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:10 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:34:10.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:10 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:34:10.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:10 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:34:10.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:10 smithi001 ceph-mon[35470]: scrub queued for path: / 2024-02-02T15:34:10.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:10 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:34:10.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:10 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:34:10.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:10 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:34:10.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:10 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:34:10.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:10 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:34:10.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:10 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:34:10.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:10 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:34:10.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:10 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:34:10.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:10 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:34:10.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:10 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:34:10.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:10 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:34:10.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:10 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:34:10.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:10 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:34:10.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:10 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:34:10.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:10 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:34:10.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:10 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:34:10.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:10 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:34:10.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:10 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:34:10.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:10 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:34:10.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:10 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:34:10.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:10 smithi071 ceph-mon[44707]: scrub queued for path: / 2024-02-02T15:34:10.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:10 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:34:10.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:10 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:34:10.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:10 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:34:10.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:10 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:34:10.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:10 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:34:10.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:10 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:34:10.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:10 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:34:10.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:10 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:34:10.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:10 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:34:10.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:10 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:34:10.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:10 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:34:10.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:10 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:34:10.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:10 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:34:10.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:10 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:34:10.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:10 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:34:10.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:10 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:34:10.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:10 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:34:10.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:10 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:34:10.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:10 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:34:10.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:10 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:34:11.243 INFO:tasks.workunit.client.0.smithi001.stdout: 1 451183 7.45 MB/sec execute 225 sec latency 14.798 ms 2024-02-02T15:34:11.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:11 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:34:11.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:11 smithi142 ceph-mon[45728]: pgmap v219: 129 pgs: 129 active+clean; 289 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 193 op/s 2024-02-02T15:34:11.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:11 smithi142 ceph-mon[45728]: osdmap e91: 12 total, 12 up, 12 in 2024-02-02T15:34:11.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:11 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:34:11.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:11 smithi001 ceph-mon[35470]: pgmap v219: 129 pgs: 129 active+clean; 289 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 193 op/s 2024-02-02T15:34:11.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:11 smithi001 ceph-mon[35470]: osdmap e91: 12 total, 12 up, 12 in 2024-02-02T15:34:11.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:11 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:34:11.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:11 smithi071 ceph-mon[44707]: pgmap v219: 129 pgs: 129 active+clean; 289 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 193 op/s 2024-02-02T15:34:11.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:11 smithi071 ceph-mon[44707]: osdmap e91: 12 total, 12 up, 12 in 2024-02-02T15:34:12.243 INFO:tasks.workunit.client.0.smithi001.stdout: 1 451898 7.44 MB/sec execute 226 sec latency 482.745 ms 2024-02-02T15:34:13.243 INFO:tasks.workunit.client.0.smithi001.stdout: 1 451898 7.40 MB/sec execute 227 sec latency 1482.871 ms 2024-02-02T15:34:13.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:13 smithi142 ceph-mon[45728]: pgmap v221: 129 pgs: 129 active+clean; 292 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 164 op/s 2024-02-02T15:34:13.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:13 smithi001 ceph-mon[35470]: pgmap v221: 129 pgs: 129 active+clean; 292 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 164 op/s 2024-02-02T15:34:13.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:13 smithi071 ceph-mon[44707]: pgmap v221: 129 pgs: 129 active+clean; 292 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 164 op/s 2024-02-02T15:34:14.243 INFO:tasks.workunit.client.0.smithi001.stdout: 1 451898 7.37 MB/sec execute 228 sec latency 2482.957 ms 2024-02-02T15:34:14.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:14 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:34:14.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:14 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:34:14.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:14 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:34:15.243 INFO:tasks.workunit.client.0.smithi001.stdout: 1 451898 7.34 MB/sec execute 229 sec latency 3483.053 ms 2024-02-02T15:34:15.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:15 smithi001 ceph-mon[35470]: pgmap v222: 129 pgs: 129 active+clean; 292 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 161 op/s 2024-02-02T15:34:15.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:15 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:34:15.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:15 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:34:15.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:15 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:34:15.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:15 smithi071 ceph-mon[44707]: pgmap v222: 129 pgs: 129 active+clean; 292 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 161 op/s 2024-02-02T15:34:15.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:15 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:34:15.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:15 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:34:15.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:15 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:34:15.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:15 smithi142 ceph-mon[45728]: pgmap v222: 129 pgs: 129 active+clean; 292 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 161 op/s 2024-02-02T15:34:15.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:15 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:34:15.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:15 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:34:15.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:15 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:34:16.244 INFO:tasks.workunit.client.0.smithi001.stdout: 1 451898 7.31 MB/sec execute 230 sec latency 4483.174 ms 2024-02-02T15:34:17.244 INFO:tasks.workunit.client.0.smithi001.stdout: 1 451898 7.28 MB/sec execute 231 sec latency 5483.342 ms 2024-02-02T15:34:17.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:17 smithi001 ceph-mon[35470]: pgmap v223: 129 pgs: 129 active+clean; 301 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 850 KiB/s rd, 1.5 MiB/s wr, 95 op/s 2024-02-02T15:34:17.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:17 smithi071 ceph-mon[44707]: pgmap v223: 129 pgs: 129 active+clean; 301 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 850 KiB/s rd, 1.5 MiB/s wr, 95 op/s 2024-02-02T15:34:17.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:17 smithi142 ceph-mon[45728]: pgmap v223: 129 pgs: 129 active+clean; 301 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 850 KiB/s rd, 1.5 MiB/s wr, 95 op/s 2024-02-02T15:34:18.244 INFO:tasks.workunit.client.0.smithi001.stdout: 1 451898 7.24 MB/sec execute 232 sec latency 6483.442 ms 2024-02-02T15:34:19.244 INFO:tasks.workunit.client.0.smithi001.stdout: 1 451898 7.21 MB/sec execute 233 sec latency 7483.551 ms 2024-02-02T15:34:19.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:19 smithi001 ceph-mon[35470]: pgmap v224: 129 pgs: 129 active+clean; 301 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 850 KiB/s rd, 1.5 MiB/s wr, 95 op/s 2024-02-02T15:34:19.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:19 smithi071 ceph-mon[44707]: pgmap v224: 129 pgs: 129 active+clean; 301 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 850 KiB/s rd, 1.5 MiB/s wr, 95 op/s 2024-02-02T15:34:19.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:19 smithi142 ceph-mon[45728]: pgmap v224: 129 pgs: 129 active+clean; 301 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 850 KiB/s rd, 1.5 MiB/s wr, 95 op/s 2024-02-02T15:34:20.244 INFO:tasks.workunit.client.0.smithi001.stdout: 1 451898 7.18 MB/sec execute 234 sec latency 8483.623 ms 2024-02-02T15:34:21.244 INFO:tasks.workunit.client.0.smithi001.stdout: 1 451898 7.15 MB/sec execute 235 sec latency 9483.742 ms 2024-02-02T15:34:21.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:21 smithi001 ceph-mon[35470]: pgmap v225: 129 pgs: 129 active+clean; 301 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 850 KiB/s rd, 1.5 MiB/s wr, 95 op/s 2024-02-02T15:34:21.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:21 smithi071 ceph-mon[44707]: pgmap v225: 129 pgs: 129 active+clean; 301 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 850 KiB/s rd, 1.5 MiB/s wr, 95 op/s 2024-02-02T15:34:21.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:21 smithi142 ceph-mon[45728]: pgmap v225: 129 pgs: 129 active+clean; 301 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 850 KiB/s rd, 1.5 MiB/s wr, 95 op/s 2024-02-02T15:34:22.244 INFO:tasks.workunit.client.0.smithi001.stdout: 1 451898 7.12 MB/sec execute 236 sec latency 10483.821 ms 2024-02-02T15:34:22.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:22 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:34:22.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:22 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:34:22.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:22 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:34:22.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:22 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:34:22.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:22 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:34:22.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:22 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:34:22.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:22 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:34:22.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:22 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:34:22.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:22 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:34:22.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:22 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:34:22.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:22 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:34:22.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:22 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:34:22.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:22 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:34:22.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:22 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:34:22.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:22 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:34:23.244 INFO:tasks.workunit.client.0.smithi001.stdout: 1 451898 7.09 MB/sec execute 237 sec latency 11483.945 ms 2024-02-02T15:34:23.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:23 smithi071 ceph-mon[44707]: pgmap v226: 129 pgs: 129 active+clean; 297 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 1.1 MiB/s wr, 58 op/s 2024-02-02T15:34:23.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:23 smithi142 ceph-mon[45728]: pgmap v226: 129 pgs: 129 active+clean; 297 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 1.1 MiB/s wr, 58 op/s 2024-02-02T15:34:23.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:23 smithi001 ceph-mon[35470]: pgmap v226: 129 pgs: 129 active+clean; 297 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 1.1 MiB/s wr, 58 op/s 2024-02-02T15:34:24.245 INFO:tasks.workunit.client.0.smithi001.stdout: 1 451898 7.06 MB/sec execute 238 sec latency 12484.085 ms 2024-02-02T15:34:25.245 INFO:tasks.workunit.client.0.smithi001.stdout: 1 452187 7.03 MB/sec execute 239 sec latency 12788.368 ms 2024-02-02T15:34:25.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:25 smithi071 ceph-mon[44707]: pgmap v227: 129 pgs: 129 active+clean; 297 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 981 KiB/s wr, 52 op/s 2024-02-02T15:34:25.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:25 smithi142 ceph-mon[45728]: pgmap v227: 129 pgs: 129 active+clean; 297 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 981 KiB/s wr, 52 op/s 2024-02-02T15:34:25.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:25 smithi001 ceph-mon[35470]: pgmap v227: 129 pgs: 129 active+clean; 297 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 981 KiB/s wr, 52 op/s 2024-02-02T15:34:26.245 INFO:tasks.workunit.client.0.smithi001.stdout: 1 452187 7.00 MB/sec execute 240 sec latency 1423.000 ms 2024-02-02T15:34:27.245 INFO:tasks.workunit.client.0.smithi001.stdout: 1 452187 6.97 MB/sec execute 241 sec latency 2423.057 ms 2024-02-02T15:34:27.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:27 smithi071 ceph-mon[44707]: pgmap v228: 129 pgs: 129 active+clean; 300 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 1.3 MiB/s wr, 74 op/s 2024-02-02T15:34:27.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:27 smithi142 ceph-mon[45728]: pgmap v228: 129 pgs: 129 active+clean; 300 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 1.3 MiB/s wr, 74 op/s 2024-02-02T15:34:27.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:27 smithi001 ceph-mon[35470]: pgmap v228: 129 pgs: 129 active+clean; 300 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 495 KiB/s rd, 1.3 MiB/s wr, 74 op/s 2024-02-02T15:34:28.245 INFO:tasks.workunit.client.0.smithi001.stdout: 1 452187 6.95 MB/sec execute 242 sec latency 3423.184 ms 2024-02-02T15:34:29.245 INFO:tasks.workunit.client.0.smithi001.stdout: 1 452187 6.92 MB/sec execute 243 sec latency 4423.284 ms 2024-02-02T15:34:29.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:29 smithi071 ceph-mon[44707]: pgmap v229: 129 pgs: 129 active+clean; 300 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 548 KiB/s wr, 42 op/s 2024-02-02T15:34:29.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:29 smithi142 ceph-mon[45728]: pgmap v229: 129 pgs: 129 active+clean; 300 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 548 KiB/s wr, 42 op/s 2024-02-02T15:34:29.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:29 smithi001 ceph-mon[35470]: pgmap v229: 129 pgs: 129 active+clean; 300 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 548 KiB/s wr, 42 op/s 2024-02-02T15:34:30.245 INFO:tasks.workunit.client.0.smithi001.stdout: 1 453504 6.92 MB/sec execute 244 sec latency 4719.299 ms 2024-02-02T15:34:31.245 INFO:tasks.workunit.client.0.smithi001.stdout: 1 454716 6.92 MB/sec execute 245 sec latency 14.880 ms 2024-02-02T15:34:31.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:31 smithi071 ceph-mon[44707]: pgmap v230: 129 pgs: 129 active+clean; 300 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 548 KiB/s wr, 42 op/s 2024-02-02T15:34:31.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:31 smithi142 ceph-mon[45728]: pgmap v230: 129 pgs: 129 active+clean; 300 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 548 KiB/s wr, 42 op/s 2024-02-02T15:34:31.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:31 smithi001 ceph-mon[35470]: pgmap v230: 129 pgs: 129 active+clean; 300 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 350 KiB/s rd, 548 KiB/s wr, 42 op/s 2024-02-02T15:34:32.245 INFO:tasks.workunit.client.0.smithi001.stdout: 1 455730 6.91 MB/sec execute 246 sec latency 274.459 ms 2024-02-02T15:34:33.245 INFO:tasks.workunit.client.0.smithi001.stdout: 1 455730 6.88 MB/sec execute 247 sec latency 1274.586 ms 2024-02-02T15:34:33.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:33 smithi071 ceph-mon[44707]: pgmap v231: 129 pgs: 129 active+clean; 299 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 970 KiB/s wr, 86 op/s 2024-02-02T15:34:33.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:33 smithi142 ceph-mon[45728]: pgmap v231: 129 pgs: 129 active+clean; 299 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 970 KiB/s wr, 86 op/s 2024-02-02T15:34:33.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:33 smithi001 ceph-mon[35470]: pgmap v231: 129 pgs: 129 active+clean; 299 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 537 KiB/s rd, 970 KiB/s wr, 86 op/s 2024-02-02T15:34:34.245 INFO:tasks.workunit.client.0.smithi001.stdout: 1 455730 6.85 MB/sec execute 248 sec latency 2274.709 ms 2024-02-02T15:34:35.245 INFO:tasks.workunit.client.0.smithi001.stdout: 1 457035 6.86 MB/sec execute 249 sec latency 2572.559 ms 2024-02-02T15:34:35.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:35 smithi071 ceph-mon[44707]: pgmap v232: 129 pgs: 129 active+clean; 299 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 767 KiB/s wr, 65 op/s 2024-02-02T15:34:35.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:35 smithi142 ceph-mon[45728]: pgmap v232: 129 pgs: 129 active+clean; 299 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 767 KiB/s wr, 65 op/s 2024-02-02T15:34:35.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:35 smithi001 ceph-mon[35470]: pgmap v232: 129 pgs: 129 active+clean; 299 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 767 KiB/s wr, 65 op/s 2024-02-02T15:34:36.246 INFO:tasks.workunit.client.0.smithi001.stdout: 1 458143 6.85 MB/sec execute 250 sec latency 18.561 ms 2024-02-02T15:34:37.246 INFO:tasks.workunit.client.0.smithi001.stdout: 1 458821 6.90 MB/sec execute 251 sec latency 62.635 ms 2024-02-02T15:34:37.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:37 smithi071 ceph-mon[44707]: pgmap v233: 129 pgs: 129 active+clean; 303 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 122 op/s 2024-02-02T15:34:37.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:37 smithi142 ceph-mon[45728]: pgmap v233: 129 pgs: 129 active+clean; 303 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 122 op/s 2024-02-02T15:34:37.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:37 smithi001 ceph-mon[35470]: pgmap v233: 129 pgs: 129 active+clean; 303 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 122 op/s 2024-02-02T15:34:38.246 INFO:tasks.workunit.client.0.smithi001.stdout: 1 460030 6.91 MB/sec execute 252 sec latency 219.021 ms 2024-02-02T15:34:39.246 INFO:tasks.workunit.client.0.smithi001.stdout: 1 460432 6.88 MB/sec execute 253 sec latency 1056.007 ms 2024-02-02T15:34:39.809 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-02T15:34:39.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:39 smithi071 ceph-mon[44707]: pgmap v234: 129 pgs: 129 active+clean; 303 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 1.0 MiB/s wr, 100 op/s 2024-02-02T15:34:39.874 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:39 smithi001 ceph-mon[35470]: pgmap v234: 129 pgs: 129 active+clean; 303 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 1.0 MiB/s wr, 100 op/s 2024-02-02T15:34:39.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:39 smithi142 ceph-mon[45728]: pgmap v234: 129 pgs: 129 active+clean; 303 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 1.0 MiB/s wr, 100 op/s 2024-02-02T15:34:40.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.024+0000 7fed72c00700 1 -- 172.21.15.1:0/3063167792 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed6c101c70 msgr2=0x7fed6c068010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:40.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.024+0000 7fed72c00700 1 --2- 172.21.15.1:0/3063167792 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed6c101c70 0x7fed6c068010 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fed5c009990 tx=0x7fed5c009ca0 comp rx=0 tx=0).stop 2024-02-02T15:34:40.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.024+0000 7fed72c00700 1 -- 172.21.15.1:0/3063167792 shutdown_connections 2024-02-02T15:34:40.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.024+0000 7fed72c00700 1 --2- 172.21.15.1:0/3063167792 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fed6c10d610 0x7fed6c10fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:40.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.024+0000 7fed72c00700 1 --2- 172.21.15.1:0/3063167792 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed6c101c70 0x7fed6c068010 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:40.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.024+0000 7fed72c00700 1 --2- 172.21.15.1:0/3063167792 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed6c0685e0 0x7fed6c068a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:40.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.024+0000 7fed72c00700 1 -- 172.21.15.1:0/3063167792 >> 172.21.15.1:0/3063167792 conn(0x7fed6c0757d0 msgr2=0x7fed6c075be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:40.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.024+0000 7fed72c00700 1 -- 172.21.15.1:0/3063167792 shutdown_connections 2024-02-02T15:34:40.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.024+0000 7fed72c00700 1 -- 172.21.15.1:0/3063167792 wait complete. 2024-02-02T15:34:40.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.025+0000 7fed72c00700 1 Processor -- start 2024-02-02T15:34:40.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.026+0000 7fed72c00700 1 -- start start 2024-02-02T15:34:40.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.026+0000 7fed72c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed6c0685e0 0x7fed6c10ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:40.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.026+0000 7fed72c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fed6c101c70 0x7fed6c10cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:40.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.026+0000 7fed72c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed6c10d610 0x7fed6c109c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:40.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.026+0000 7fed72c00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed6c10a430 con 0x7fed6c10d610 2024-02-02T15:34:40.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.026+0000 7fed72c00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed6c10a5a0 con 0x7fed6c101c70 2024-02-02T15:34:40.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.026+0000 7fed72c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed6c10a760 con 0x7fed6c0685e0 2024-02-02T15:34:40.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.026+0000 7fed6be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed6c0685e0 0x7fed6c10ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:40.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.026+0000 7fed6be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed6c0685e0 0x7fed6c10ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:60642/0 (socket says 172.21.15.1:60642) 2024-02-02T15:34:40.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.026+0000 7fed6be00700 1 -- 172.21.15.1:0/4032924024 learned_addr learned my addr 172.21.15.1:0/4032924024 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:34:40.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.026+0000 7fed70c00700 1 --2- 172.21.15.1:0/4032924024 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed6c10d610 0x7fed6c109c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:40.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.026+0000 7fed6b400700 1 --2- 172.21.15.1:0/4032924024 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fed6c101c70 0x7fed6c10cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:40.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.027+0000 7fed6be00700 1 -- 172.21.15.1:0/4032924024 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fed6c101c70 msgr2=0x7fed6c10cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:40.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.027+0000 7fed6be00700 1 --2- 172.21.15.1:0/4032924024 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fed6c101c70 0x7fed6c10cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:40.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.027+0000 7fed6be00700 1 -- 172.21.15.1:0/4032924024 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed6c10d610 msgr2=0x7fed6c109c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:40.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.027+0000 7fed6be00700 1 --2- 172.21.15.1:0/4032924024 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed6c10d610 0x7fed6c109c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:40.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.027+0000 7fed6be00700 1 -- 172.21.15.1:0/4032924024 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed5c0095f0 con 0x7fed6c0685e0 2024-02-02T15:34:40.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.027+0000 7fed6be00700 1 --2- 172.21.15.1:0/4032924024 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed6c0685e0 0x7fed6c10ca40 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fed5c005910 tx=0x7fed5c005720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:40.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.027+0000 7fed68c00700 1 -- 172.21.15.1:0/4032924024 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed5c01d070 con 0x7fed6c0685e0 2024-02-02T15:34:40.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.027+0000 7fed72c00700 1 -- 172.21.15.1:0/4032924024 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed6c1a9df0 con 0x7fed6c0685e0 2024-02-02T15:34:40.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.027+0000 7fed68c00700 1 -- 172.21.15.1:0/4032924024 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed5c022470 con 0x7fed6c0685e0 2024-02-02T15:34:40.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.027+0000 7fed68c00700 1 -- 172.21.15.1:0/4032924024 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed5c017650 con 0x7fed6c0685e0 2024-02-02T15:34:40.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.027+0000 7fed72c00700 1 -- 172.21.15.1:0/4032924024 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed6c1aa1b0 con 0x7fed6c0685e0 2024-02-02T15:34:40.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.030+0000 7fed68c00700 1 -- 172.21.15.1:0/4032924024 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7fed5c0225e0 con 0x7fed6c0685e0 2024-02-02T15:34:40.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.032+0000 7fed68c00700 1 --2- 172.21.15.1:0/4032924024 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fed4c0a0690 0x7fed4c0a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:40.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.032+0000 7fed86662b80 1 Processor -- start 2024-02-02T15:34:40.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.032+0000 7fed68c00700 1 -- 172.21.15.1:0/4032924024 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fed5c0cd4c0 con 0x7fed6c0685e0 2024-02-02T15:34:40.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.032+0000 7fed6b400700 1 --2- 172.21.15.1:0/4032924024 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fed4c0a0690 0x7fed4c0a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:40.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.032+0000 7fed86662b80 1 -- start start 2024-02-02T15:34:40.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.032+0000 7fed86662b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d68aa46010 0x55d68aa463f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:40.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.032+0000 7fed86662b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d68aa46930 0x55d68aa512a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:40.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.033+0000 7fed86662b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d68aa517e0 0x55d68aa53bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:40.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.033+0000 7fed86662b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d68a9ba4c0 con 0x55d68aa46930 2024-02-02T15:34:40.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.033+0000 7fed86662b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d68a99eaa0 con 0x55d68aa46010 2024-02-02T15:34:40.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.033+0000 7fed86662b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d68a99c460 con 0x55d68aa517e0 2024-02-02T15:34:40.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.033+0000 7fed6be00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d68aa46010 0x55d68aa463f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:40.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.033+0000 7fed6be00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d68aa46010 0x55d68aa463f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:58718/0 (socket says 172.21.15.1:58718) 2024-02-02T15:34:40.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.033+0000 7fed6be00700 1 -- 172.21.15.1:0/665638887 learned_addr learned my addr 172.21.15.1:0/665638887 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:34:40.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.033+0000 7fed70c00700 1 --2- 172.21.15.1:0/665638887 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d68aa517e0 0x55d68aa53bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:40.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.033+0000 7fed6b400700 1 --2- 172.21.15.1:0/665638887 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d68aa46930 0x55d68aa512a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:40.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.033+0000 7fed6b400700 1 --2- 172.21.15.1:0/4032924024 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fed4c0a0690 0x7fed4c0a2b50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fed6c10b240 tx=0x7fed54009260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:40.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.033+0000 7fed6be00700 1 -- 172.21.15.1:0/665638887 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d68aa517e0 msgr2=0x55d68aa53bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:40.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.033+0000 7fed6be00700 1 --2- 172.21.15.1:0/665638887 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d68aa517e0 0x55d68aa53bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:40.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.033+0000 7fed6be00700 1 -- 172.21.15.1:0/665638887 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d68aa46930 msgr2=0x55d68aa512a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:40.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.033+0000 7fed6be00700 1 --2- 172.21.15.1:0/665638887 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d68aa46930 0x55d68aa512a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:40.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.033+0000 7fed6be00700 1 -- 172.21.15.1:0/665638887 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d68a8193b0 con 0x55d68aa46010 2024-02-02T15:34:40.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.033+0000 7fed6be00700 1 --2- 172.21.15.1:0/665638887 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d68aa46010 0x55d68aa463f0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fed5c006010 tx=0x7fed5c022c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8cce7c1d5d355501 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:40.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.034+0000 7fed61200700 1 -- 172.21.15.1:0/665638887 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed5c09a0f0 con 0x55d68aa46010 2024-02-02T15:34:40.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.034+0000 7fed61200700 1 -- 172.21.15.1:0/665638887 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed5c09ae90 con 0x55d68aa46010 2024-02-02T15:34:40.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.034+0000 7fed61200700 1 -- 172.21.15.1:0/665638887 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed5c09a9f0 con 0x55d68aa46010 2024-02-02T15:34:40.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.034+0000 7fed86662b80 1 -- 172.21.15.1:0/665638887 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d68aa46010 msgr2=0x55d68aa463f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:40.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.034+0000 7fed86662b80 1 --2- 172.21.15.1:0/665638887 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d68aa46010 0x55d68aa463f0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fed5c006010 tx=0x7fed5c022c90 comp rx=0 tx=0).stop 2024-02-02T15:34:40.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.035+0000 7fed86662b80 1 -- 172.21.15.1:0/665638887 shutdown_connections 2024-02-02T15:34:40.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.035+0000 7fed86662b80 1 --2- 172.21.15.1:0/665638887 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d68aa517e0 0x55d68aa53bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:40.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.035+0000 7fed86662b80 1 --2- 172.21.15.1:0/665638887 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d68aa46010 0x55d68aa463f0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:40.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.035+0000 7fed86662b80 1 --2- 172.21.15.1:0/665638887 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d68aa46930 0x55d68aa512a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:40.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.035+0000 7fed86662b80 1 -- 172.21.15.1:0/665638887 >> 172.21.15.1:0/665638887 conn(0x55d68a9a5db0 msgr2=0x55d68aa423b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:40.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.035+0000 7fed86662b80 1 -- 172.21.15.1:0/665638887 shutdown_connections 2024-02-02T15:34:40.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.035+0000 7fed86662b80 1 -- 172.21.15.1:0/665638887 wait complete. 2024-02-02T15:34:40.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.035+0000 7fed86662b80 1 -- start start 2024-02-02T15:34:40.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.036+0000 7fed86662b80 1 Processor -- start 2024-02-02T15:34:40.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.036+0000 7fed86662b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d68aa46010 0x55d68aa51bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:40.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.036+0000 7fed86662b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d68aa46930 0x55d68aa52100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:40.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.036+0000 7fed86662b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d68aa517e0 0x55d68aa52660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:40.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.036+0000 7fed86662b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d68a997d50 con 0x55d68aa46010 2024-02-02T15:34:40.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.036+0000 7fed86662b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d68a933d80 con 0x55d68aa517e0 2024-02-02T15:34:40.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.036+0000 7fed86662b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d68a933a20 con 0x55d68aa46930 2024-02-02T15:34:40.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.037+0000 7fed86662b80 10 client.0 _ll_register_callbacks cb 0x55d68a935290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:34:40.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.037+0000 7fed70c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d68aa517e0 0x55d68aa52660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:40.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.037+0000 7fed6b400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d68aa46930 0x55d68aa52100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:40.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.037+0000 7fed6be00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d68aa46010 0x55d68aa51bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:40.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.037+0000 7fed70c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d68aa517e0 0x55d68aa52660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:58720/0 (socket says 172.21.15.1:58720) 2024-02-02T15:34:40.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.037+0000 7fed6b400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d68aa46930 0x55d68aa52100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:60658/0 (socket says 172.21.15.1:60658) 2024-02-02T15:34:40.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.037+0000 7fed70c00700 1 -- 172.21.15.1:0/4078397230 learned_addr learned my addr 172.21.15.1:0/4078397230 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:34:40.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.037+0000 7fed70c00700 1 -- 172.21.15.1:0/4078397230 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d68aa46930 msgr2=0x55d68aa52100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:40.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.037+0000 7fed70c00700 1 --2- 172.21.15.1:0/4078397230 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d68aa46930 0x55d68aa52100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:40.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.037+0000 7fed70c00700 1 -- 172.21.15.1:0/4078397230 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d68aa46010 msgr2=0x55d68aa51bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:40.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.037+0000 7fed70c00700 1 --2- 172.21.15.1:0/4078397230 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d68aa46010 0x55d68aa51bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:40.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.037+0000 7fed70c00700 1 -- 172.21.15.1:0/4078397230 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed6c1aa1b0 con 0x55d68aa517e0 2024-02-02T15:34:40.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.037+0000 7fed70c00700 1 --2- 172.21.15.1:0/4078397230 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d68aa517e0 0x55d68aa52660 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fed64007b60 tx=0x7fed64007b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3dab418f760e2489 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:40.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.037+0000 7fed59600700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:34:40.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.038+0000 7fed59600700 1 -- 172.21.15.1:0/4078397230 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed6400e040 con 0x55d68aa517e0 2024-02-02T15:34:40.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.038+0000 7fed59600700 1 -- 172.21.15.1:0/4078397230 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fed64015720 con 0x55d68aa517e0 2024-02-02T15:34:40.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.038+0000 7fed86662b80 1 -- 172.21.15.1:0/4078397230 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d68aa3a0f0 con 0x55d68aa517e0 2024-02-02T15:34:40.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.038+0000 7fed59600700 1 -- 172.21.15.1:0/4078397230 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed640158e0 con 0x55d68aa517e0 2024-02-02T15:34:40.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.038+0000 7fed86662b80 10 client.27620 fetch_fsmap learned FSMap version 41 2024-02-02T15:34:40.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.038+0000 7fed86662b80 1 -- 172.21.15.1:0/4078397230 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55d68a7c5b60 con 0x55d68aa517e0 2024-02-02T15:34:40.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.038+0000 7fed59600700 1 -- 172.21.15.1:0/4078397230 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d68aa3a0f0 con 0x55d68aa517e0 2024-02-02T15:34:40.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.038+0000 7fed59600700 1 -- 172.21.15.1:0/4078397230 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 41) v1 ==== 7506+0+0 (secure 0 0 0) 0x7fed64016c20 con 0x55d68aa517e0 2024-02-02T15:34:40.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.038+0000 7fed86662b80 10 client.27620 fetch_fsmap finished waiting for FSMap version 41 2024-02-02T15:34:40.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.038+0000 7fed86662b80 10 client.27620 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:34:40.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.038+0000 7fed86662b80 20 client.27620 populate_metadata read hostname 'smithi001' 2024-02-02T15:34:40.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.039+0000 7fed86662b80 1 --2- 172.21.15.1:0/4078397230 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55d68aa60fe0 0x55d68aa63400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:40.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.039+0000 7fed86662b80 4 client.27620 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:34:40.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.039+0000 7fed86662b80 1 -- 172.21.15.1:0/4078397230 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d68a990dd0 con 0x55d68aa60fe0 2024-02-02T15:34:40.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.039+0000 7fed6be00700 1 --2- 172.21.15.1:0/4078397230 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55d68aa60fe0 0x55d68aa63400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:40.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.039+0000 7fed6be00700 1 --2- 172.21.15.1:0/4078397230 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55d68aa60fe0 0x55d68aa63400 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=765801893e3b8fe2 server_cookie=5d41f3d3b4ece9 in_seq=0 out_seq=0 2024-02-02T15:34:40.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.039+0000 7fed59600700 10 client.27620 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:34:40.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.040+0000 7fed59600700 1 -- 172.21.15.1:0/4078397230 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55d68a990dd0 con 0x55d68aa60fe0 2024-02-02T15:34:40.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.040+0000 7fed59600700 10 client.27620 handle_command_reply: tid=0 2024-02-02T15:34:40.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.040+0000 7fed86662b80 1 client.27620 shutdown 2024-02-02T15:34:40.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.040+0000 7fed86662b80 1 -- 172.21.15.1:0/4078397230 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d68aa517e0 msgr2=0x55d68aa52660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:40.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.040+0000 7fed86662b80 1 --2- 172.21.15.1:0/4078397230 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d68aa517e0 0x55d68aa52660 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fed64007b60 tx=0x7fed64007b90 comp rx=0 tx=0).stop 2024-02-02T15:34:40.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.040+0000 7fed86662b80 1 -- 172.21.15.1:0/4078397230 shutdown_connections 2024-02-02T15:34:40.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.040+0000 7fed86662b80 1 --2- 172.21.15.1:0/4078397230 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55d68aa60fe0 0x55d68aa63400 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:40.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.040+0000 7fed86662b80 1 --2- 172.21.15.1:0/4078397230 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d68aa517e0 0x55d68aa52660 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:40.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.040+0000 7fed86662b80 1 --2- 172.21.15.1:0/4078397230 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d68aa46010 0x55d68aa51bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:40.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.040+0000 7fed86662b80 1 --2- 172.21.15.1:0/4078397230 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d68aa46930 0x55d68aa52100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:40.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.040+0000 7fed86662b80 1 -- 172.21.15.1:0/4078397230 >> 172.21.15.1:0/4078397230 conn(0x55d68a9a5db0 msgr2=0x55d68aa41fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:40.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.040+0000 7fed59600700 0 client.27620 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:34:40.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.040+0000 7fed86662b80 1 -- 172.21.15.1:0/4078397230 shutdown_connections 2024-02-02T15:34:40.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.040+0000 7fed86662b80 1 -- 172.21.15.1:0/4078397230 wait complete. 2024-02-02T15:34:40.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.040+0000 7fed86662b80 20 client.27620 trim_cache size 0 max 16384 2024-02-02T15:34:40.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.058+0000 7fed86662b80 1 Processor -- start 2024-02-02T15:34:40.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.058+0000 7fed86662b80 1 -- start start 2024-02-02T15:34:40.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.058+0000 7fed86662b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d68aa46010 0x55d68aa522d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:40.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.058+0000 7fed86662b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d68aa46930 0x55d68aa5c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:40.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.058+0000 7fed86662b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d68aa517e0 0x55d68aa4aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:40.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.058+0000 7fed86662b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d68a81b6a0 con 0x55d68aa46930 2024-02-02T15:34:40.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.058+0000 7fed86662b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d68aa50ee0 con 0x55d68aa517e0 2024-02-02T15:34:40.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.058+0000 7fed86662b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d68aa4edc0 con 0x55d68aa46010 2024-02-02T15:34:40.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.059+0000 7fed6be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d68aa46010 0x55d68aa522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:40.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.059+0000 7fed70c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d68aa517e0 0x55d68aa4aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:40.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.059+0000 7fed6be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d68aa46010 0x55d68aa522d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:60668/0 (socket says 172.21.15.1:60668) 2024-02-02T15:34:40.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.059+0000 7fed70c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d68aa517e0 0x55d68aa4aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:58722/0 (socket says 172.21.15.1:58722) 2024-02-02T15:34:40.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.059+0000 7fed6be00700 1 -- 172.21.15.1:0/4275768249 learned_addr learned my addr 172.21.15.1:0/4275768249 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:34:40.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.059+0000 7fed6b400700 1 --2- 172.21.15.1:0/4275768249 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d68aa46930 0x55d68aa5c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:40.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.059+0000 7fed6be00700 1 -- 172.21.15.1:0/4275768249 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d68aa517e0 msgr2=0x55d68aa4aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:40.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.059+0000 7fed6be00700 1 --2- 172.21.15.1:0/4275768249 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d68aa517e0 0x55d68aa4aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:40.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.059+0000 7fed6be00700 1 -- 172.21.15.1:0/4275768249 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d68aa46930 msgr2=0x55d68aa5c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:40.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.059+0000 7fed6be00700 1 --2- 172.21.15.1:0/4275768249 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d68aa46930 0x55d68aa5c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:40.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.059+0000 7fed6be00700 1 -- 172.21.15.1:0/4275768249 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d68aa3e0d0 con 0x55d68aa46010 2024-02-02T15:34:40.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.059+0000 7fed70c00700 1 --2- 172.21.15.1:0/4275768249 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d68aa517e0 0x55d68aa4aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:34:40.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.059+0000 7fed6be00700 1 --2- 172.21.15.1:0/4275768249 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d68aa46010 0x55d68aa522d0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fed5c000c00 tx=0x7fed5c0b4040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3df258a0eb68c461 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:40.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.060+0000 7fed61c00700 1 -- 172.21.15.1:0/4275768249 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed5c07d070 con 0x55d68aa46010 2024-02-02T15:34:40.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.060+0000 7fed61c00700 1 -- 172.21.15.1:0/4275768249 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed5c09a0f0 con 0x55d68aa46010 2024-02-02T15:34:40.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.060+0000 7fed61c00700 1 -- 172.21.15.1:0/4275768249 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed5c0b2ea0 con 0x55d68aa46010 2024-02-02T15:34:40.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.060+0000 7fed86662b80 1 -- 172.21.15.1:0/4275768249 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d68aa46010 msgr2=0x55d68aa522d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:40.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.060+0000 7fed86662b80 1 --2- 172.21.15.1:0/4275768249 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d68aa46010 0x55d68aa522d0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fed5c000c00 tx=0x7fed5c0b4040 comp rx=0 tx=0).stop 2024-02-02T15:34:40.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.060+0000 7fed86662b80 1 -- 172.21.15.1:0/4275768249 shutdown_connections 2024-02-02T15:34:40.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.060+0000 7fed86662b80 1 --2- 172.21.15.1:0/4275768249 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d68aa517e0 0x55d68aa4aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:40.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.060+0000 7fed86662b80 1 --2- 172.21.15.1:0/4275768249 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d68aa46010 0x55d68aa522d0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:40.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.060+0000 7fed86662b80 1 --2- 172.21.15.1:0/4275768249 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d68aa46930 0x55d68aa5c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:40.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.060+0000 7fed86662b80 1 -- 172.21.15.1:0/4275768249 >> 172.21.15.1:0/4275768249 conn(0x55d68a9a5db0 msgr2=0x55d68aa51bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:40.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.061+0000 7fed86662b80 1 -- 172.21.15.1:0/4275768249 shutdown_connections 2024-02-02T15:34:40.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.061+0000 7fed86662b80 1 -- 172.21.15.1:0/4275768249 wait complete. 2024-02-02T15:34:40.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.061+0000 7fed86662b80 1 -- start start 2024-02-02T15:34:40.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.061+0000 7fed86662b80 1 Processor -- start 2024-02-02T15:34:40.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.062+0000 7fed86662b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d68aa46010 0x55d68aa541c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:40.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.062+0000 7fed86662b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d68aa46930 0x55d68aa54700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:40.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.062+0000 7fed86662b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d68aa517e0 0x55d68aa54c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:40.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.062+0000 7fed86662b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d68aab1b00 con 0x55d68aa517e0 2024-02-02T15:34:40.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.062+0000 7fed86662b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d68aab1c70 con 0x55d68aa46930 2024-02-02T15:34:40.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.062+0000 7fed86662b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d68aab1e30 con 0x55d68aa46010 2024-02-02T15:34:40.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.062+0000 7fed86662b80 10 client.0 _ll_register_callbacks cb 0x55d68a935290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:34:40.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.062+0000 7fed6b400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d68aa46930 0x55d68aa54700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:40.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.062+0000 7fed6be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d68aa46010 0x55d68aa541c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:40.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.062+0000 7fed6b400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d68aa46930 0x55d68aa54700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:58730/0 (socket says 172.21.15.1:58730) 2024-02-02T15:34:40.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.062+0000 7fed6b400700 1 -- 172.21.15.1:0/274235400 learned_addr learned my addr 172.21.15.1:0/274235400 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:34:40.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.062+0000 7fed6be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d68aa46010 0x55d68aa541c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:60682/0 (socket says 172.21.15.1:60682) 2024-02-02T15:34:40.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.062+0000 7fed70c00700 1 --2- 172.21.15.1:0/274235400 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d68aa517e0 0x55d68aa54c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:40.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.062+0000 7fed6b400700 1 -- 172.21.15.1:0/274235400 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d68aa46010 msgr2=0x55d68aa541c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:40.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.062+0000 7fed6b400700 1 --2- 172.21.15.1:0/274235400 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d68aa46010 0x55d68aa541c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:40.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.062+0000 7fed6b400700 1 -- 172.21.15.1:0/274235400 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d68aa517e0 msgr2=0x55d68aa54c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:40.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.062+0000 7fed6b400700 1 --2- 172.21.15.1:0/274235400 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d68aa517e0 0x55d68aa54c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:40.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.062+0000 7fed6b400700 1 -- 172.21.15.1:0/274235400 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d68a7c5b60 con 0x55d68aa46930 2024-02-02T15:34:40.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.062+0000 7fed70c00700 1 --2- 172.21.15.1:0/274235400 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d68aa517e0 0x55d68aa54c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:34:40.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.062+0000 7fed6be00700 1 --2- 172.21.15.1:0/274235400 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d68aa46010 0x55d68aa541c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:34:40.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.063+0000 7fed6b400700 1 --2- 172.21.15.1:0/274235400 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d68aa46930 0x55d68aa54700 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fed6c101170 tx=0x7fed540047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2bc91db2927fed5 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:40.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.063+0000 7fed59600700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:34:40.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.063+0000 7fed59600700 1 -- 172.21.15.1:0/274235400 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed54004020 con 0x55d68aa46930 2024-02-02T15:34:40.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.063+0000 7fed59600700 1 -- 172.21.15.1:0/274235400 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fed54004180 con 0x55d68aa46930 2024-02-02T15:34:40.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.063+0000 7fed86662b80 1 -- 172.21.15.1:0/274235400 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fed5c09ae90 con 0x55d68aa46930 2024-02-02T15:34:40.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.063+0000 7fed59600700 1 -- 172.21.15.1:0/274235400 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed54004340 con 0x55d68aa46930 2024-02-02T15:34:40.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.063+0000 7fed86662b80 10 client.27626 fetch_fsmap learned FSMap version 41 2024-02-02T15:34:40.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.063+0000 7fed86662b80 1 -- 172.21.15.1:0/274235400 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55d68a8193b0 con 0x55d68aa46930 2024-02-02T15:34:40.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.064+0000 7fed59600700 1 -- 172.21.15.1:0/274235400 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed5c09ae90 con 0x55d68aa46930 2024-02-02T15:34:40.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.064+0000 7fed59600700 1 -- 172.21.15.1:0/274235400 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 41) v1 ==== 7506+0+0 (secure 0 0 0) 0x7fed540038f0 con 0x55d68aa46930 2024-02-02T15:34:40.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.064+0000 7fed86662b80 10 client.27626 fetch_fsmap finished waiting for FSMap version 41 2024-02-02T15:34:40.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.064+0000 7fed86662b80 10 client.27626 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:34:40.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.064+0000 7fed86662b80 20 client.27626 populate_metadata read hostname 'smithi001' 2024-02-02T15:34:40.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.064+0000 7fed86662b80 1 --2- 172.21.15.1:0/274235400 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55d68aab3750 0x55d68aab5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:40.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.064+0000 7fed86662b80 4 client.27626 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2024-02-02T15:34:40.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.064+0000 7fed86662b80 1 -- 172.21.15.1:0/274235400 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d68a956f50 con 0x55d68aab3750 2024-02-02T15:34:40.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.064+0000 7fed6be00700 1 --2- 172.21.15.1:0/274235400 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55d68aab3750 0x55d68aab5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:40.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.064+0000 7fed6be00700 1 --2- 172.21.15.1:0/274235400 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55d68aab3750 0x55d68aab5b70 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=faf5dfafdc8d8c6f server_cookie=9d0f4f183be8d33b in_seq=0 out_seq=0 2024-02-02T15:34:40.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.064+0000 7fed59600700 10 client.27626 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:34:40.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.065+0000 7fed59600700 1 -- 172.21.15.1:0/274235400 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55d68a956f50 con 0x55d68aab3750 2024-02-02T15:34:40.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.065+0000 7fed59600700 10 client.27626 handle_command_reply: tid=0 2024-02-02T15:34:40.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.065+0000 7fed86662b80 1 client.27626 shutdown 2024-02-02T15:34:40.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.065+0000 7fed86662b80 1 -- 172.21.15.1:0/274235400 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d68aa46930 msgr2=0x55d68aa54700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:40.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.065+0000 7fed86662b80 1 --2- 172.21.15.1:0/274235400 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d68aa46930 0x55d68aa54700 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fed6c101170 tx=0x7fed540047c0 comp rx=0 tx=0).stop 2024-02-02T15:34:40.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.065+0000 7fed86662b80 1 -- 172.21.15.1:0/274235400 shutdown_connections 2024-02-02T15:34:40.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.065+0000 7fed86662b80 1 --2- 172.21.15.1:0/274235400 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55d68aab3750 0x55d68aab5b70 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:40.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.065+0000 7fed86662b80 1 --2- 172.21.15.1:0/274235400 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d68aa517e0 0x55d68aa54c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:40.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.065+0000 7fed86662b80 1 --2- 172.21.15.1:0/274235400 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d68aa46010 0x55d68aa541c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:40.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.065+0000 7fed59600700 0 client.27626 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:34:40.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.065+0000 7fed86662b80 1 --2- 172.21.15.1:0/274235400 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d68aa46930 0x55d68aa54700 secure :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fed6c101170 tx=0x7fed540047c0 comp rx=0 tx=0).stop 2024-02-02T15:34:40.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.065+0000 7fed86662b80 1 -- 172.21.15.1:0/274235400 >> 172.21.15.1:0/274235400 conn(0x55d68a9a5db0 msgr2=0x55d68aa52450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:40.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.065+0000 7fed86662b80 1 -- 172.21.15.1:0/274235400 shutdown_connections 2024-02-02T15:34:40.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.065+0000 7fed86662b80 1 -- 172.21.15.1:0/274235400 wait complete. 2024-02-02T15:34:40.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.065+0000 7fed86662b80 20 client.27626 trim_cache size 0 max 16384 2024-02-02T15:34:40.071 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:34:40.071 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2024-02-02T15:34:40.071 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2024-02-02T15:34:40.071 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:34:40.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.066+0000 7fed72c00700 1 -- 172.21.15.1:0/4032924024 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fed4c0a0690 msgr2=0x7fed4c0a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:40.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.066+0000 7fed72c00700 1 --2- 172.21.15.1:0/4032924024 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fed4c0a0690 0x7fed4c0a2b50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fed6c10b240 tx=0x7fed54009260 comp rx=0 tx=0).stop 2024-02-02T15:34:40.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.066+0000 7fed72c00700 1 -- 172.21.15.1:0/4032924024 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed6c0685e0 msgr2=0x7fed6c10ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:40.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.066+0000 7fed72c00700 1 --2- 172.21.15.1:0/4032924024 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed6c0685e0 0x7fed6c10ca40 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fed5c005910 tx=0x7fed5c005720 comp rx=0 tx=0).stop 2024-02-02T15:34:40.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.066+0000 7fed72c00700 1 -- 172.21.15.1:0/4032924024 shutdown_connections 2024-02-02T15:34:40.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.066+0000 7fed72c00700 1 --2- 172.21.15.1:0/4032924024 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fed4c0a0690 0x7fed4c0a2b50 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:40.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.066+0000 7fed72c00700 1 --2- 172.21.15.1:0/4032924024 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed6c10d610 0x7fed6c109c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:40.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.066+0000 7fed72c00700 1 --2- 172.21.15.1:0/4032924024 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed6c0685e0 0x7fed6c10ca40 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:40.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.066+0000 7fed72c00700 1 --2- 172.21.15.1:0/4032924024 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fed6c101c70 0x7fed6c10cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:40.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.066+0000 7fed72c00700 1 -- 172.21.15.1:0/4032924024 >> 172.21.15.1:0/4032924024 conn(0x7fed6c0757d0 msgr2=0x7fed6c0fee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:40.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.066+0000 7fed72c00700 1 -- 172.21.15.1:0/4032924024 shutdown_connections 2024-02-02T15:34:40.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.066+0000 7fed72c00700 1 -- 172.21.15.1:0/4032924024 wait complete. 2024-02-02T15:34:40.085 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-02T15:34:40.085 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T15:34:40.246 INFO:tasks.workunit.client.0.smithi001.stdout: 1 462188 6.90 MB/sec execute 254 sec latency 14.952 ms 2024-02-02T15:34:40.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.293+0000 7ff825c00700 1 -- 172.21.15.1:0/1891467606 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8201018c0 msgr2=0x7ff820105900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:40.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.293+0000 7ff825c00700 1 --2- 172.21.15.1:0/1891467606 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8201018c0 0x7ff820105900 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff808009990 tx=0x7ff808009ca0 comp rx=0 tx=0).stop 2024-02-02T15:34:40.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.293+0000 7ff825c00700 1 -- 172.21.15.1:0/1891467606 shutdown_connections 2024-02-02T15:34:40.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.293+0000 7ff825c00700 1 --2- 172.21.15.1:0/1891467606 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff820106030 0x7ff82010dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:40.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.293+0000 7ff825c00700 1 --2- 172.21.15.1:0/1891467606 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff820100f10 0x7ff8201012f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:40.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.293+0000 7ff825c00700 1 --2- 172.21.15.1:0/1891467606 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8201018c0 0x7ff820105900 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:40.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.293+0000 7ff825c00700 1 -- 172.21.15.1:0/1891467606 >> 172.21.15.1:0/1891467606 conn(0x7ff8200fc790 msgr2=0x7ff8200febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:40.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.293+0000 7ff825c00700 1 -- 172.21.15.1:0/1891467606 shutdown_connections 2024-02-02T15:34:40.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.293+0000 7ff825c00700 1 -- 172.21.15.1:0/1891467606 wait complete. 2024-02-02T15:34:40.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.294+0000 7ff825c00700 1 Processor -- start 2024-02-02T15:34:40.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.294+0000 7ff825c00700 1 -- start start 2024-02-02T15:34:40.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.295+0000 7ff825c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff820100f10 0x7ff820196e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:40.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.295+0000 7ff825c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff820106030 0x7ff8201973c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:40.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.295+0000 7ff825c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff82019b820 0x7ff82019bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:40.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.295+0000 7ff825c00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff82019c2a0 con 0x7ff82019b820 2024-02-02T15:34:40.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.295+0000 7ff825c00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff82019c410 con 0x7ff820100f10 2024-02-02T15:34:40.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.295+0000 7ff825c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff82019c5d0 con 0x7ff820106030 2024-02-02T15:34:40.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.295+0000 7ff81ea00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff820100f10 0x7ff820196e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:40.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.295+0000 7ff81e000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff820106030 0x7ff8201973c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:40.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.295+0000 7ff81f400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff82019b820 0x7ff82019bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:40.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.295+0000 7ff81e000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff820106030 0x7ff8201973c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:60698/0 (socket says 172.21.15.1:60698) 2024-02-02T15:34:40.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.295+0000 7ff81e000700 1 -- 172.21.15.1:0/2785382803 learned_addr learned my addr 172.21.15.1:0/2785382803 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:34:40.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.295+0000 7ff81ea00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff820100f10 0x7ff820196e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:58744/0 (socket says 172.21.15.1:58744) 2024-02-02T15:34:40.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.295+0000 7ff81e000700 1 -- 172.21.15.1:0/2785382803 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff820100f10 msgr2=0x7ff820196e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:40.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.295+0000 7ff81e000700 1 --2- 172.21.15.1:0/2785382803 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff820100f10 0x7ff820196e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:40.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.295+0000 7ff81e000700 1 -- 172.21.15.1:0/2785382803 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff82019b820 msgr2=0x7ff82019bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:40.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.295+0000 7ff81e000700 1 --2- 172.21.15.1:0/2785382803 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff82019b820 0x7ff82019bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:40.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.295+0000 7ff81e000700 1 -- 172.21.15.1:0/2785382803 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8080095f0 con 0x7ff820106030 2024-02-02T15:34:40.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.295+0000 7ff81f400700 1 --2- 172.21.15.1:0/2785382803 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff82019b820 0x7ff82019bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:34:40.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.296+0000 7ff81e000700 1 --2- 172.21.15.1:0/2785382803 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff820106030 0x7ff8201973c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ff808004760 tx=0x7ff808004790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:40.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.296+0000 7ff817400700 1 -- 172.21.15.1:0/2785382803 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff80801d070 con 0x7ff820106030 2024-02-02T15:34:40.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.296+0000 7ff825c00700 1 -- 172.21.15.1:0/2785382803 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8201a3ad0 con 0x7ff820106030 2024-02-02T15:34:40.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.296+0000 7ff817400700 1 -- 172.21.15.1:0/2785382803 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff808004830 con 0x7ff820106030 2024-02-02T15:34:40.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.296+0000 7ff817400700 1 -- 172.21.15.1:0/2785382803 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff808005280 con 0x7ff820106030 2024-02-02T15:34:40.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.296+0000 7ff825c00700 1 -- 172.21.15.1:0/2785382803 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8201a3ef0 con 0x7ff820106030 2024-02-02T15:34:40.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.297+0000 7ff825c00700 1 -- 172.21.15.1:0/2785382803 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff820067bc0 con 0x7ff820106030 2024-02-02T15:34:40.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.299+0000 7ff817400700 1 -- 172.21.15.1:0/2785382803 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7ff8080053e0 con 0x7ff820106030 2024-02-02T15:34:40.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.300+0000 7ff817400700 1 --2- 172.21.15.1:0/2785382803 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff8000a0710 0x7ff8000a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:40.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.300+0000 7ff817400700 1 -- 172.21.15.1:0/2785382803 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9453+0+0 (secure 0 0 0) 0x7ff8080cd500 con 0x7ff820106030 2024-02-02T15:34:40.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.301+0000 7ff81ea00700 1 --2- 172.21.15.1:0/2785382803 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff8000a0710 0x7ff8000a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:40.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.301+0000 7ff81ea00700 1 --2- 172.21.15.1:0/2785382803 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff8000a0710 0x7ff8000a2bd0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff810009c60 tx=0x7ff810009260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:40.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.303+0000 7ff817400700 1 -- 172.21.15.1:0/2785382803 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7ff808096a50 con 0x7ff820106030 2024-02-02T15:34:40.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.616+0000 7ff825c00700 1 -- 172.21.15.1:0/2785382803 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff8201981d0 con 0x7ff820106030 2024-02-02T15:34:40.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.618+0000 7ff817400700 1 -- 172.21.15.1:0/2785382803 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+10881 (secure 0 0 0) 0x7ff80802f090 con 0x7ff820106030 2024-02-02T15:34:40.618 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:34:40.618 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2922240448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2922240448},{"type":"v1","addr":"172.21.15.1:6835","nonce":2922240448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2729497431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2729497431},{"type":"v1","addr":"172.21.15.1:6837","nonce":2729497431}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":41,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-02T15:24:47.761435+0000","modified":"2024-02-02T15:34:39.471895+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24529,"mds_2":24518,"mds_3":24514,"mds_4":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14517":{"gid":14517,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/1218303683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":1218303683},{"type":"v1","addr":"172.21.15.1:6839","nonce":1218303683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14529":{"gid":14529,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/2059197496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2059197496},{"type":"v1","addr":"172.21.15.1:6841","nonce":2059197496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/7648986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":7648986},{"type":"v1","addr":"172.21.15.142:6833","nonce":7648986}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/3482746094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3482746094},{"type":"v1","addr":"172.21.15.71:6833","nonce":3482746094}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/388207814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":388207814},{"type":"v1","addr":"172.21.15.71:6835","nonce":388207814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6835/1052606547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1052606547},{"type":"v1","addr":"172.21.15.142:6835","nonce":1052606547}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.142:6837/2870289019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2870289019},{"type":"v1","addr":"172.21.15.142:6837","nonce":2870289019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.71:6837/628016199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":628016199},{"type":"v1","addr":"172.21.15.71:6837","nonce":628016199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.142:6839/321675421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":321675421},{"type":"v1","addr":"172.21.15.142:6839","nonce":321675421}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/1268086493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1268086493},{"type":"v1","addr":"172.21.15.71:6839","nonce":1268086493}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-02T15:34:40.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.624+0000 7ff825c00700 1 -- 172.21.15.1:0/2785382803 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff8000a0710 msgr2=0x7ff8000a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:40.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.624+0000 7ff825c00700 1 --2- 172.21.15.1:0/2785382803 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff8000a0710 0x7ff8000a2bd0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff810009c60 tx=0x7ff810009260 comp rx=0 tx=0).stop 2024-02-02T15:34:40.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.624+0000 7ff825c00700 1 -- 172.21.15.1:0/2785382803 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff820106030 msgr2=0x7ff8201973c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:40.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.624+0000 7ff825c00700 1 --2- 172.21.15.1:0/2785382803 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff820106030 0x7ff8201973c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ff808004760 tx=0x7ff808004790 comp rx=0 tx=0).stop 2024-02-02T15:34:40.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.624+0000 7ff825c00700 1 -- 172.21.15.1:0/2785382803 shutdown_connections 2024-02-02T15:34:40.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.624+0000 7ff825c00700 1 --2- 172.21.15.1:0/2785382803 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff8000a0710 0x7ff8000a2bd0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:40.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.624+0000 7ff825c00700 1 --2- 172.21.15.1:0/2785382803 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff82019b820 0x7ff82019bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:40.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.624+0000 7ff825c00700 1 --2- 172.21.15.1:0/2785382803 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff820100f10 0x7ff820196e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:40.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.624+0000 7ff825c00700 1 --2- 172.21.15.1:0/2785382803 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff820106030 0x7ff8201973c0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:40.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.624+0000 7ff825c00700 1 -- 172.21.15.1:0/2785382803 >> 172.21.15.1:0/2785382803 conn(0x7ff8200fc790 msgr2=0x7ff8200fd270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:40.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.624+0000 7ff825c00700 1 -- 172.21.15.1:0/2785382803 shutdown_connections 2024-02-02T15:34:40.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.624+0000 7ff825c00700 1 -- 172.21.15.1:0/2785382803 wait complete. 2024-02-02T15:34:40.626 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 41 2024-02-02T15:34:40.654 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-02-02T15:34:40.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:40 smithi071 ceph-mon[44707]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:34:40.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.895+0000 7f44f0200700 1 -- 172.21.15.1:0/916917932 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f44e8075270 msgr2=0x7f44e8075650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:40.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.895+0000 7f44f0200700 1 --2- 172.21.15.1:0/916917932 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f44e8075270 0x7f44e8075650 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f44d0009990 tx=0x7f44d0009ca0 comp rx=0 tx=0).stop 2024-02-02T15:34:40.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.895+0000 7f44f0200700 1 -- 172.21.15.1:0/916917932 shutdown_connections 2024-02-02T15:34:40.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.896+0000 7f44f0200700 1 --2- 172.21.15.1:0/916917932 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44e8111cc0 0x7f44e81140b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:40.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.896+0000 7f44f0200700 1 --2- 172.21.15.1:0/916917932 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f44e8075270 0x7f44e8075650 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:40.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.896+0000 7f44f0200700 1 --2- 172.21.15.1:0/916917932 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44e8075b90 0x7f44e8111780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:40.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.896+0000 7f44f0200700 1 -- 172.21.15.1:0/916917932 >> 172.21.15.1:0/916917932 conn(0x7f44e80feb30 msgr2=0x7f44e8100f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:40.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.896+0000 7f44f0200700 1 -- 172.21.15.1:0/916917932 shutdown_connections 2024-02-02T15:34:40.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.896+0000 7f44f0200700 1 -- 172.21.15.1:0/916917932 wait complete. 2024-02-02T15:34:40.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.897+0000 7f44f0200700 1 Processor -- start 2024-02-02T15:34:40.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.898+0000 7f44f0200700 1 -- start start 2024-02-02T15:34:40.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.898+0000 7f44f0200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44e8075270 0x7f44e819d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:40.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.898+0000 7f44f0200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44e8075b90 0x7f44e819dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:40.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.898+0000 7f44f0200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f44e8111cc0 0x7f44e81a1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:40.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.898+0000 7f44f0200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e81a2560 con 0x7f44e8111cc0 2024-02-02T15:34:40.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.898+0000 7f44f0200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e81a26d0 con 0x7f44e8075b90 2024-02-02T15:34:40.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.898+0000 7f44f0200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e81a2890 con 0x7f44e8075270 2024-02-02T15:34:40.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.898+0000 7f44ed800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44e8075270 0x7f44e819d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:40.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.898+0000 7f44ed800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44e8075270 0x7f44e819d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:60726/0 (socket says 172.21.15.1:60726) 2024-02-02T15:34:40.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.898+0000 7f44ed800700 1 -- 172.21.15.1:0/313589118 learned_addr learned my addr 172.21.15.1:0/313589118 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:34:40.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.899+0000 7f44ece00700 1 --2- 172.21.15.1:0/313589118 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44e8075b90 0x7f44e819dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:40.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.899+0000 7f44ee200700 1 --2- 172.21.15.1:0/313589118 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f44e8111cc0 0x7f44e81a1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:40.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.899+0000 7f44ece00700 1 -- 172.21.15.1:0/313589118 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44e8075270 msgr2=0x7f44e819d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:40.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.899+0000 7f44ece00700 1 --2- 172.21.15.1:0/313589118 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44e8075270 0x7f44e819d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:40.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.899+0000 7f44ece00700 1 -- 172.21.15.1:0/313589118 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f44e8111cc0 msgr2=0x7f44e81a1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:40.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.899+0000 7f44ece00700 1 --2- 172.21.15.1:0/313589118 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f44e8111cc0 0x7f44e81a1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:40.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.899+0000 7f44ece00700 1 -- 172.21.15.1:0/313589118 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44d00095f0 con 0x7f44e8075b90 2024-02-02T15:34:40.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.899+0000 7f44ed800700 1 --2- 172.21.15.1:0/313589118 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44e8075270 0x7f44e819d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:34:40.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.900+0000 7f44ece00700 1 --2- 172.21.15.1:0/313589118 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44e8075b90 0x7f44e819dac0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f44d800e9f0 tx=0x7f44d800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:40.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.900+0000 7f44e6000700 1 -- 172.21.15.1:0/313589118 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d800c9c0 con 0x7f44e8075b90 2024-02-02T15:34:40.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.900+0000 7f44e6000700 1 -- 172.21.15.1:0/313589118 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f44d8010430 con 0x7f44e8075b90 2024-02-02T15:34:40.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.900+0000 7f44f0200700 1 -- 172.21.15.1:0/313589118 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44e81a2b70 con 0x7f44e8075b90 2024-02-02T15:34:40.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.900+0000 7f44e6000700 1 -- 172.21.15.1:0/313589118 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d8018680 con 0x7f44e8075b90 2024-02-02T15:34:40.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.900+0000 7f44f0200700 1 -- 172.21.15.1:0/313589118 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44e81aa3b0 con 0x7f44e8075b90 2024-02-02T15:34:40.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.903+0000 7f44e6000700 1 -- 172.21.15.1:0/313589118 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f44d8010860 con 0x7f44e8075b90 2024-02-02T15:34:40.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.905+0000 7f44e6000700 1 --2- 172.21.15.1:0/313589118 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f44c80a0710 0x7f44c80a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:40.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.906+0000 7f44e6000700 1 -- 172.21.15.1:0/313589118 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f44d8014070 con 0x7f44e8075b90 2024-02-02T15:34:40.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.906+0000 7f44ed800700 1 --2- 172.21.15.1:0/313589118 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f44c80a0710 0x7f44c80a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:40.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.906+0000 7f4503c99b80 1 Processor -- start 2024-02-02T15:34:40.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.907+0000 7f4503c99b80 1 -- start start 2024-02-02T15:34:40.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.907+0000 7f4503c99b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e431a17010 0x55e431a173f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:40.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.907+0000 7f4503c99b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e431a17930 0x55e431a222a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:40.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.907+0000 7f44ed800700 1 --2- 172.21.15.1:0/313589118 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f44c80a0710 0x7f44c80a2bd0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f44d0006010 tx=0x7f44d000b260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:40.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.907+0000 7f4503c99b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e431a227e0 0x55e431a24bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:40.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.907+0000 7f4503c99b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e43198b4c0 con 0x55e431a17930 2024-02-02T15:34:40.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.907+0000 7f4503c99b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e43196faa0 con 0x55e431a227e0 2024-02-02T15:34:40.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.907+0000 7f4503c99b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e43196d460 con 0x55e431a17010 2024-02-02T15:34:40.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.907+0000 7f44ed800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e431a17010 0x55e431a173f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:40.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.908+0000 7f44ed800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e431a17010 0x55e431a173f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:60742/0 (socket says 172.21.15.1:60742) 2024-02-02T15:34:40.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.908+0000 7f44ed800700 1 -- 172.21.15.1:0/396393082 learned_addr learned my addr 172.21.15.1:0/396393082 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:34:40.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.908+0000 7f44ece00700 1 --2- 172.21.15.1:0/396393082 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e431a17930 0x55e431a222a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:40.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.908+0000 7f44ee200700 1 --2- 172.21.15.1:0/396393082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e431a227e0 0x55e431a24bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:40.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.908+0000 7f44ed800700 1 -- 172.21.15.1:0/396393082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e431a227e0 msgr2=0x55e431a24bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:40.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.908+0000 7f44ed800700 1 --2- 172.21.15.1:0/396393082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e431a227e0 0x55e431a24bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:40.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.908+0000 7f44ed800700 1 -- 172.21.15.1:0/396393082 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e431a17930 msgr2=0x55e431a222a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:40.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.908+0000 7f44ed800700 1 --2- 172.21.15.1:0/396393082 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e431a17930 0x55e431a222a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:40.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.908+0000 7f44ed800700 1 -- 172.21.15.1:0/396393082 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e4317ea3b0 con 0x55e431a17010 2024-02-02T15:34:40.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.908+0000 7f44ed800700 1 --2- 172.21.15.1:0/396393082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e431a17010 0x55e431a173f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f44d00057a0 tx=0x7f44d0020000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f2ba4a697e0d7f7 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:40.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.909+0000 7f44dea00700 1 -- 172.21.15.1:0/396393082 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d002b070 con 0x55e431a17010 2024-02-02T15:34:40.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.909+0000 7f44dea00700 1 -- 172.21.15.1:0/396393082 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f44d001e040 con 0x55e431a17010 2024-02-02T15:34:40.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.909+0000 7f44dea00700 1 -- 172.21.15.1:0/396393082 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d0004bd0 con 0x55e431a17010 2024-02-02T15:34:40.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.909+0000 7f4503c99b80 1 -- 172.21.15.1:0/396393082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e431a17010 msgr2=0x55e431a173f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:40.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.909+0000 7f4503c99b80 1 --2- 172.21.15.1:0/396393082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e431a17010 0x55e431a173f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f44d00057a0 tx=0x7f44d0020000 comp rx=0 tx=0).stop 2024-02-02T15:34:40.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.910+0000 7f4503c99b80 1 -- 172.21.15.1:0/396393082 shutdown_connections 2024-02-02T15:34:40.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.910+0000 7f4503c99b80 1 --2- 172.21.15.1:0/396393082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e431a227e0 0x55e431a24bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:40.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.910+0000 7f4503c99b80 1 --2- 172.21.15.1:0/396393082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e431a17010 0x55e431a173f0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:40.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.910+0000 7f4503c99b80 1 --2- 172.21.15.1:0/396393082 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e431a17930 0x55e431a222a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:40.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.910+0000 7f4503c99b80 1 -- 172.21.15.1:0/396393082 >> 172.21.15.1:0/396393082 conn(0x55e431976db0 msgr2=0x55e431a133b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:40.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.910+0000 7f4503c99b80 1 -- 172.21.15.1:0/396393082 shutdown_connections 2024-02-02T15:34:40.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.910+0000 7f4503c99b80 1 -- 172.21.15.1:0/396393082 wait complete. 2024-02-02T15:34:40.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.910+0000 7f4503c99b80 1 -- start start 2024-02-02T15:34:40.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.910+0000 7f4503c99b80 1 Processor -- start 2024-02-02T15:34:40.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.911+0000 7f4503c99b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e431a17010 0x55e431a22110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:40.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.911+0000 7f4503c99b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e431a17930 0x55e431a1f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:40.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.911+0000 7f4503c99b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e431a227e0 0x55e431a1f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:40.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.911+0000 7f4503c99b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e431968d50 con 0x55e431a17010 2024-02-02T15:34:40.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.911+0000 7f4503c99b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e431904d80 con 0x55e431a17930 2024-02-02T15:34:40.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.911+0000 7f4503c99b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e431904a20 con 0x55e431a227e0 2024-02-02T15:34:40.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.911+0000 7f4503c99b80 10 client.0 _ll_register_callbacks cb 0x55e431906290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:34:40.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.911+0000 7f44ece00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e431a17930 0x55e431a1f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:40.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.911+0000 7f44ece00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e431a17930 0x55e431a1f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:58782/0 (socket says 172.21.15.1:58782) 2024-02-02T15:34:40.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.911+0000 7f44ece00700 1 -- 172.21.15.1:0/228518883 learned_addr learned my addr 172.21.15.1:0/228518883 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:34:40.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.911+0000 7f44ed800700 1 --2- 172.21.15.1:0/228518883 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e431a17010 0x55e431a22110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:40.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.911+0000 7f44ee200700 1 --2- 172.21.15.1:0/228518883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e431a227e0 0x55e431a1f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:40.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.912+0000 7f44ece00700 1 -- 172.21.15.1:0/228518883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e431a227e0 msgr2=0x55e431a1f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:40.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.912+0000 7f44ece00700 1 --2- 172.21.15.1:0/228518883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e431a227e0 0x55e431a1f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:40.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.912+0000 7f44ece00700 1 -- 172.21.15.1:0/228518883 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e431a17010 msgr2=0x55e431a22110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:40.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.912+0000 7f44ece00700 1 --2- 172.21.15.1:0/228518883 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e431a17010 0x55e431a22110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:40.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.912+0000 7f44ece00700 1 -- 172.21.15.1:0/228518883 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44e8114ba0 con 0x55e431a17930 2024-02-02T15:34:40.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.912+0000 7f44ee200700 1 --2- 172.21.15.1:0/228518883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e431a227e0 0x55e431a1f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:34:40.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.912+0000 7f44ece00700 1 --2- 172.21.15.1:0/228518883 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e431a17930 0x55e431a1f390 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f44d8095d80 tx=0x7f44d8099200 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc91cc3f1dac5779 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:40.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.912+0000 7f44d7400700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:34:40.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.912+0000 7f44d7400700 1 -- 172.21.15.1:0/228518883 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d8098a00 con 0x55e431a17930 2024-02-02T15:34:40.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.912+0000 7f44d7400700 1 -- 172.21.15.1:0/228518883 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f44d809d140 con 0x55e431a17930 2024-02-02T15:34:40.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.912+0000 7f4503c99b80 1 -- 172.21.15.1:0/228518883 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e431a0b0f0 con 0x55e431a17930 2024-02-02T15:34:40.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.912+0000 7f44d7400700 1 -- 172.21.15.1:0/228518883 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f44d80a1380 con 0x55e431a17930 2024-02-02T15:34:40.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.912+0000 7f4503c99b80 10 client.27644 fetch_fsmap learned FSMap version 41 2024-02-02T15:34:40.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.912+0000 7f4503c99b80 1 -- 172.21.15.1:0/228518883 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55e431796b60 con 0x55e431a17930 2024-02-02T15:34:40.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.913+0000 7f44d7400700 1 -- 172.21.15.1:0/228518883 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f44d807c080 con 0x55e431a17930 2024-02-02T15:34:40.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.913+0000 7f44d7400700 1 -- 172.21.15.1:0/228518883 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 41) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f44d8082b10 con 0x55e431a17930 2024-02-02T15:34:40.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.913+0000 7f4503c99b80 10 client.27644 fetch_fsmap finished waiting for FSMap version 41 2024-02-02T15:34:40.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.913+0000 7f4503c99b80 10 client.27644 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-02-02T15:34:40.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.913+0000 7f4503c99b80 20 client.27644 populate_metadata read hostname 'smithi001' 2024-02-02T15:34:40.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.913+0000 7f4503c99b80 1 --2- 172.21.15.1:0/228518883 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55e431a32220 0x55e431a34640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:40.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.913+0000 7f4503c99b80 4 client.27644 mds_command: new command op to 24502 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:34:40.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.913+0000 7f4503c99b80 1 -- 172.21.15.1:0/228518883 --> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e431961dd0 con 0x55e431a32220 2024-02-02T15:34:40.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.913+0000 7f44ed800700 1 --2- 172.21.15.1:0/228518883 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55e431a32220 0x55e431a34640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:40.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.914+0000 7f44ed800700 1 --2- 172.21.15.1:0/228518883 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55e431a32220 0x55e431a34640 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=462256d7d85eaf54 server_cookie=69011e2eecc67a8a in_seq=0 out_seq=0 2024-02-02T15:34:40.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.914+0000 7f44d7400700 10 client.27644 ms_handle_connect on v2:172.21.15.142:6834/1052606547 2024-02-02T15:34:40.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.914+0000 7f44d7400700 1 -- 172.21.15.1:0/228518883 <== mds.4 v2:172.21.15.142:6834/1052606547 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55e431961dd0 con 0x55e431a32220 2024-02-02T15:34:40.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.914+0000 7f44d7400700 10 client.27644 handle_command_reply: tid=0 2024-02-02T15:34:40.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.914+0000 7f4503c99b80 1 client.27644 shutdown 2024-02-02T15:34:40.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.915+0000 7f4503c99b80 1 -- 172.21.15.1:0/228518883 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e431a17930 msgr2=0x55e431a1f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:40.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.915+0000 7f4503c99b80 1 --2- 172.21.15.1:0/228518883 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e431a17930 0x55e431a1f390 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f44d8095d80 tx=0x7f44d8099200 comp rx=0 tx=0).stop 2024-02-02T15:34:40.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.915+0000 7f4503c99b80 1 -- 172.21.15.1:0/228518883 shutdown_connections 2024-02-02T15:34:40.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.915+0000 7f4503c99b80 1 --2- 172.21.15.1:0/228518883 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55e431a32220 0x55e431a34640 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:40.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.915+0000 7f4503c99b80 1 --2- 172.21.15.1:0/228518883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e431a227e0 0x55e431a1f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:40.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.915+0000 7f4503c99b80 1 --2- 172.21.15.1:0/228518883 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e431a17010 0x55e431a22110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:40.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.915+0000 7f44d7400700 0 client.27644 ms_handle_reset on v2:172.21.15.142:6834/1052606547 2024-02-02T15:34:40.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.915+0000 7f4503c99b80 1 --2- 172.21.15.1:0/228518883 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e431a17930 0x55e431a1f390 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:40.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.915+0000 7f4503c99b80 1 -- 172.21.15.1:0/228518883 >> 172.21.15.1:0/228518883 conn(0x55e431976db0 msgr2=0x55e431a12fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:40.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.915+0000 7f4503c99b80 1 -- 172.21.15.1:0/228518883 shutdown_connections 2024-02-02T15:34:40.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.915+0000 7f4503c99b80 1 -- 172.21.15.1:0/228518883 wait complete. 2024-02-02T15:34:40.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.915+0000 7f4503c99b80 20 client.27644 trim_cache size 0 max 16384 2024-02-02T15:34:40.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:40 smithi142 ceph-mon[45728]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:34:40.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.929+0000 7f4503c99b80 1 Processor -- start 2024-02-02T15:34:40.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.929+0000 7f4503c99b80 1 -- start start 2024-02-02T15:34:40.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.929+0000 7f4503c99b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e431a17010 0x55e431a2d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:40.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.929+0000 7f4503c99b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e431a17930 0x55e431a2dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:40.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.929+0000 7f4503c99b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e431a227e0 0x55e431a23bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:40.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.929+0000 7f4503c99b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4317ec6a0 con 0x55e431a17010 2024-02-02T15:34:40.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.929+0000 7f4503c99b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e431961dd0 con 0x55e431a227e0 2024-02-02T15:34:40.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.929+0000 7f4503c99b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e431a0f9b0 con 0x55e431a17930 2024-02-02T15:34:40.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.930+0000 7f44ece00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e431a17930 0x55e431a2dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:40.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.930+0000 7f44ece00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e431a17930 0x55e431a2dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:60768/0 (socket says 172.21.15.1:60768) 2024-02-02T15:34:40.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.930+0000 7f44ece00700 1 -- 172.21.15.1:0/3041730633 learned_addr learned my addr 172.21.15.1:0/3041730633 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:34:40.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.930+0000 7f44ed800700 1 --2- 172.21.15.1:0/3041730633 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e431a17010 0x55e431a2d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:40.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.930+0000 7f44ee200700 1 --2- 172.21.15.1:0/3041730633 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e431a227e0 0x55e431a23bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:40.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.930+0000 7f44ece00700 1 -- 172.21.15.1:0/3041730633 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e431a227e0 msgr2=0x55e431a23bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:40.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.930+0000 7f44ece00700 1 --2- 172.21.15.1:0/3041730633 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e431a227e0 0x55e431a23bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:40.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.930+0000 7f44ece00700 1 -- 172.21.15.1:0/3041730633 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e431a17010 msgr2=0x55e431a2d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:40.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.930+0000 7f44ece00700 1 --2- 172.21.15.1:0/3041730633 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e431a17010 0x55e431a2d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:40.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.930+0000 7f44ece00700 1 -- 172.21.15.1:0/3041730633 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e43192eea0 con 0x55e431a17930 2024-02-02T15:34:40.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.931+0000 7f44ece00700 1 --2- 172.21.15.1:0/3041730633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e431a17930 0x55e431a2dde0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f44d8099010 tx=0x7f44d8082f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ceb80d88f0246922 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:40.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.931+0000 7f44df400700 1 -- 172.21.15.1:0/3041730633 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d809d140 con 0x55e431a17930 2024-02-02T15:34:40.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.931+0000 7f44df400700 1 -- 172.21.15.1:0/3041730633 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e431a0b0f0 con 0x55e431a17930 2024-02-02T15:34:40.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.931+0000 7f44df400700 1 -- 172.21.15.1:0/3041730633 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d80a0ef0 con 0x55e431a17930 2024-02-02T15:34:40.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.932+0000 7f4503c99b80 1 -- 172.21.15.1:0/3041730633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e431a17930 msgr2=0x55e431a2dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:40.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.932+0000 7f4503c99b80 1 --2- 172.21.15.1:0/3041730633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e431a17930 0x55e431a2dde0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f44d8099010 tx=0x7f44d8082f50 comp rx=0 tx=0).stop 2024-02-02T15:34:40.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.932+0000 7f4503c99b80 1 -- 172.21.15.1:0/3041730633 shutdown_connections 2024-02-02T15:34:40.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.932+0000 7f4503c99b80 1 --2- 172.21.15.1:0/3041730633 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e431a227e0 0x55e431a23bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:40.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.932+0000 7f4503c99b80 1 --2- 172.21.15.1:0/3041730633 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e431a17010 0x55e431a2d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:40.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.932+0000 7f4503c99b80 1 --2- 172.21.15.1:0/3041730633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e431a17930 0x55e431a2dde0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:40.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.932+0000 7f4503c99b80 1 -- 172.21.15.1:0/3041730633 >> 172.21.15.1:0/3041730633 conn(0x55e431976db0 msgr2=0x55e431a1a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:40.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.932+0000 7f4503c99b80 1 -- 172.21.15.1:0/3041730633 shutdown_connections 2024-02-02T15:34:40.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.932+0000 7f4503c99b80 1 -- 172.21.15.1:0/3041730633 wait complete. 2024-02-02T15:34:40.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.933+0000 7f4503c99b80 1 -- start start 2024-02-02T15:34:40.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.933+0000 7f4503c99b80 1 Processor -- start 2024-02-02T15:34:40.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.934+0000 7f4503c99b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e431a17010 0x55e431a28ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:40.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.934+0000 7f4503c99b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e431a17930 0x55e431a29000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:40.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.934+0000 7f4503c99b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e431a227e0 0x55e431a26f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:40.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.934+0000 7f4503c99b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e431a275f0 con 0x55e431a17930 2024-02-02T15:34:40.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.934+0000 7f4503c99b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e431a27760 con 0x55e431a17010 2024-02-02T15:34:40.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.934+0000 7f4503c99b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e431a27920 con 0x55e431a227e0 2024-02-02T15:34:40.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.934+0000 7f4503c99b80 10 client.0 _ll_register_callbacks cb 0x55e431906290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:34:40.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.934+0000 7f44ed800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e431a17010 0x55e431a28ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:40.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.934+0000 7f44ed800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e431a17010 0x55e431a28ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:58800/0 (socket says 172.21.15.1:58800) 2024-02-02T15:34:40.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.934+0000 7f44ed800700 1 -- 172.21.15.1:0/1528259756 learned_addr learned my addr 172.21.15.1:0/1528259756 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:34:40.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.934+0000 7f44ee200700 1 --2- 172.21.15.1:0/1528259756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e431a227e0 0x55e431a26f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:40.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.934+0000 7f44ece00700 1 --2- 172.21.15.1:0/1528259756 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e431a17930 0x55e431a29000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:40.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.934+0000 7f44ed800700 1 -- 172.21.15.1:0/1528259756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e431a227e0 msgr2=0x55e431a26f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:40.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.934+0000 7f44ed800700 1 --2- 172.21.15.1:0/1528259756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e431a227e0 0x55e431a26f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:40.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.934+0000 7f44ed800700 1 -- 172.21.15.1:0/1528259756 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e431a17930 msgr2=0x55e431a29000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:40.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.934+0000 7f44ed800700 1 --2- 172.21.15.1:0/1528259756 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e431a17930 0x55e431a29000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:40.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.934+0000 7f44ed800700 1 -- 172.21.15.1:0/1528259756 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e431796b60 con 0x55e431a17010 2024-02-02T15:34:40.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.934+0000 7f44ece00700 1 --2- 172.21.15.1:0/1528259756 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e431a17930 0x55e431a29000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:34:40.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.935+0000 7f44ed800700 1 --2- 172.21.15.1:0/1528259756 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e431a17010 0x55e431a28ac0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f44d0009350 tx=0x7f44d000bfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb0b3be0d0e63f17 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:40.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.935+0000 7f44d7400700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:34:40.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.935+0000 7f44d7400700 1 -- 172.21.15.1:0/1528259756 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d002b040 con 0x55e431a17010 2024-02-02T15:34:40.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.935+0000 7f4503c99b80 1 -- 172.21.15.1:0/1528259756 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f44d001e040 con 0x55e431a17010 2024-02-02T15:34:40.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.935+0000 7f44d7400700 1 -- 172.21.15.1:0/1528259756 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f44d0039050 con 0x55e431a17010 2024-02-02T15:34:40.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.935+0000 7f44d7400700 1 -- 172.21.15.1:0/1528259756 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f44d0039210 con 0x55e431a17010 2024-02-02T15:34:40.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.935+0000 7f4503c99b80 10 client.27650 fetch_fsmap learned FSMap version 41 2024-02-02T15:34:40.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.935+0000 7f4503c99b80 1 -- 172.21.15.1:0/1528259756 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x55e4317eecd0 con 0x55e431a17010 2024-02-02T15:34:40.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.936+0000 7f44d7400700 1 -- 172.21.15.1:0/1528259756 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f44d001e040 con 0x55e431a17010 2024-02-02T15:34:40.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.936+0000 7f44d7400700 1 -- 172.21.15.1:0/1528259756 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 41) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f44d000f460 con 0x55e431a17010 2024-02-02T15:34:40.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.936+0000 7f4503c99b80 10 client.27650 fetch_fsmap finished waiting for FSMap version 41 2024-02-02T15:34:40.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.936+0000 7f4503c99b80 10 client.27650 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-02-02T15:34:40.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.936+0000 7f4503c99b80 20 client.27650 populate_metadata read hostname 'smithi001' 2024-02-02T15:34:40.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.936+0000 7f4503c99b80 1 --2- 172.21.15.1:0/1528259756 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55e431a84990 0x55e431a86db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:40.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.936+0000 7f4503c99b80 4 client.27650 mds_command: new command op to 24502 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:34:40.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.936+0000 7f4503c99b80 1 -- 172.21.15.1:0/1528259756 --> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e431927f50 con 0x55e431a84990 2024-02-02T15:34:40.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.936+0000 7f44ece00700 1 --2- 172.21.15.1:0/1528259756 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55e431a84990 0x55e431a86db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:40.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.937+0000 7f44ece00700 1 --2- 172.21.15.1:0/1528259756 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55e431a84990 0x55e431a86db0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=afd391cd96c70f72 server_cookie=49198a55332d508 in_seq=0 out_seq=0 2024-02-02T15:34:40.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.937+0000 7f44d7400700 10 client.27650 ms_handle_connect on v2:172.21.15.142:6834/1052606547 2024-02-02T15:34:40.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.937+0000 7f44d7400700 1 -- 172.21.15.1:0/1528259756 <== mds.4 v2:172.21.15.142:6834/1052606547 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e431927f50 con 0x55e431a84990 2024-02-02T15:34:40.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.937+0000 7f44d7400700 10 client.27650 handle_command_reply: tid=0 2024-02-02T15:34:40.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.937+0000 7f4503c99b80 1 client.27650 shutdown 2024-02-02T15:34:40.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.937+0000 7f4503c99b80 1 -- 172.21.15.1:0/1528259756 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e431a17010 msgr2=0x55e431a28ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:40.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.937+0000 7f4503c99b80 1 --2- 172.21.15.1:0/1528259756 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e431a17010 0x55e431a28ac0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f44d0009350 tx=0x7f44d000bfd0 comp rx=0 tx=0).stop 2024-02-02T15:34:40.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.937+0000 7f4503c99b80 1 -- 172.21.15.1:0/1528259756 shutdown_connections 2024-02-02T15:34:40.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.937+0000 7f4503c99b80 1 --2- 172.21.15.1:0/1528259756 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55e431a84990 0x55e431a86db0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:40.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.937+0000 7f4503c99b80 1 --2- 172.21.15.1:0/1528259756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e431a227e0 0x55e431a26f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:40.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.937+0000 7f4503c99b80 1 --2- 172.21.15.1:0/1528259756 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e431a17010 0x55e431a28ac0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:40.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.937+0000 7f44d7400700 0 client.27650 ms_handle_reset on v2:172.21.15.142:6834/1052606547 2024-02-02T15:34:40.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.937+0000 7f4503c99b80 1 --2- 172.21.15.1:0/1528259756 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e431a17930 0x55e431a29000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:40.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.937+0000 7f4503c99b80 1 -- 172.21.15.1:0/1528259756 >> 172.21.15.1:0/1528259756 conn(0x55e431976db0 msgr2=0x55e431a125c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:40.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.937+0000 7f4503c99b80 1 -- 172.21.15.1:0/1528259756 shutdown_connections 2024-02-02T15:34:40.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.937+0000 7f4503c99b80 1 -- 172.21.15.1:0/1528259756 wait complete. 2024-02-02T15:34:40.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.938+0000 7f4503c99b80 20 client.27650 trim_cache size 0 max 16384 2024-02-02T15:34:40.941 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:34:40.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.938+0000 7f44f0200700 1 -- 172.21.15.1:0/313589118 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f44c80a0710 msgr2=0x7f44c80a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:40.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.938+0000 7f44f0200700 1 --2- 172.21.15.1:0/313589118 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f44c80a0710 0x7f44c80a2bd0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f44d0006010 tx=0x7f44d000b260 comp rx=0 tx=0).stop 2024-02-02T15:34:40.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.938+0000 7f44f0200700 1 -- 172.21.15.1:0/313589118 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44e8075b90 msgr2=0x7f44e819dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:40.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.938+0000 7f44f0200700 1 --2- 172.21.15.1:0/313589118 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44e8075b90 0x7f44e819dac0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f44d800e9f0 tx=0x7f44d800ed00 comp rx=0 tx=0).stop 2024-02-02T15:34:40.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.938+0000 7f44f0200700 1 -- 172.21.15.1:0/313589118 shutdown_connections 2024-02-02T15:34:40.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.938+0000 7f44f0200700 1 --2- 172.21.15.1:0/313589118 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f44c80a0710 0x7f44c80a2bd0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:40.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.938+0000 7f44f0200700 1 --2- 172.21.15.1:0/313589118 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f44e8111cc0 0x7f44e81a1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:40.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.938+0000 7f44f0200700 1 --2- 172.21.15.1:0/313589118 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44e8075270 0x7f44e819d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:40.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.938+0000 7f44f0200700 1 --2- 172.21.15.1:0/313589118 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44e8075b90 0x7f44e819dac0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:40.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.938+0000 7f44f0200700 1 -- 172.21.15.1:0/313589118 >> 172.21.15.1:0/313589118 conn(0x7f44e80feb30 msgr2=0x7f44e8100f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:40.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.938+0000 7f44f0200700 1 -- 172.21.15.1:0/313589118 shutdown_connections 2024-02-02T15:34:40.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:40.939+0000 7f44f0200700 1 -- 172.21.15.1:0/313589118 wait complete. 2024-02-02T15:34:40.942 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:40 smithi001 ceph-mon[35470]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:34:40.959 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-02-02T15:34:41.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.179+0000 7f63c6000700 1 -- 172.21.15.1:0/2069734910 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f63c0108340 msgr2=0x7f63c010fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:41.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.179+0000 7f63c6000700 1 --2- 172.21.15.1:0/2069734910 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f63c0108340 0x7f63c010fec0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f63b800b0a0 tx=0x7f63b800b3b0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.179+0000 7f63c6000700 1 -- 172.21.15.1:0/2069734910 shutdown_connections 2024-02-02T15:34:41.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.179+0000 7f63c6000700 1 --2- 172.21.15.1:0/2069734910 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f63c0108340 0x7f63c010fec0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.179+0000 7f63c6000700 1 --2- 172.21.15.1:0/2069734910 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63c0103220 0x7f63c0103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.179+0000 7f63c6000700 1 --2- 172.21.15.1:0/2069734910 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f63c0103bd0 0x7f63c0107c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.179+0000 7f63c6000700 1 -- 172.21.15.1:0/2069734910 >> 172.21.15.1:0/2069734910 conn(0x7f63c00fea60 msgr2=0x7f63c0100e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:41.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.179+0000 7f63c6000700 1 -- 172.21.15.1:0/2069734910 shutdown_connections 2024-02-02T15:34:41.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.179+0000 7f63c6000700 1 -- 172.21.15.1:0/2069734910 wait complete. 2024-02-02T15:34:41.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.180+0000 7f63c6000700 1 Processor -- start 2024-02-02T15:34:41.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.180+0000 7f63c6000700 1 -- start start 2024-02-02T15:34:41.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.180+0000 7f63c6000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63c0103220 0x7f63c0199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:41.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.181+0000 7f63c6000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f63c0108340 0x7f63c0199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:41.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.181+0000 7f63c6000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f63c019dbc0 0x7f63c019e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:41.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.181+0000 7f63c6000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63c019e640 con 0x7f63c019dbc0 2024-02-02T15:34:41.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.181+0000 7f63c6000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63c019e7b0 con 0x7f63c0108340 2024-02-02T15:34:41.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.181+0000 7f63c6000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63c019e970 con 0x7f63c0103220 2024-02-02T15:34:41.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.181+0000 7f63bf400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63c0103220 0x7f63c0199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:41.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.181+0000 7f63bea00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f63c0108340 0x7f63c0199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:41.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.181+0000 7f63bf400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63c0103220 0x7f63c0199220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:60810/0 (socket says 172.21.15.1:60810) 2024-02-02T15:34:41.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.181+0000 7f63bf400700 1 -- 172.21.15.1:0/2185381273 learned_addr learned my addr 172.21.15.1:0/2185381273 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:34:41.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.181+0000 7f63bfe00700 1 --2- 172.21.15.1:0/2185381273 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f63c019dbc0 0x7f63c019e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:41.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.181+0000 7f63bea00700 1 -- 172.21.15.1:0/2185381273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63c0103220 msgr2=0x7f63c0199220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:41.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.181+0000 7f63bea00700 1 --2- 172.21.15.1:0/2185381273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63c0103220 0x7f63c0199220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.181+0000 7f63bea00700 1 -- 172.21.15.1:0/2185381273 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f63c019dbc0 msgr2=0x7f63c019e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:41.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.181+0000 7f63bea00700 1 --2- 172.21.15.1:0/2185381273 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f63c019dbc0 0x7f63c019e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.181+0000 7f63bea00700 1 -- 172.21.15.1:0/2185381273 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63b8009d00 con 0x7f63c0108340 2024-02-02T15:34:41.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.181+0000 7f63bfe00700 1 --2- 172.21.15.1:0/2185381273 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f63c019dbc0 0x7f63c019e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:34:41.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.181+0000 7f63bf400700 1 --2- 172.21.15.1:0/2185381273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63c0103220 0x7f63c0199220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:34:41.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.182+0000 7f63bea00700 1 --2- 172.21.15.1:0/2185381273 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f63c0108340 0x7f63c0199760 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f63c0068d60 tx=0x7f63a800dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:41.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.182+0000 7f63b7e00700 1 -- 172.21.15.1:0/2185381273 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a800f890 con 0x7f63c0108340 2024-02-02T15:34:41.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.182+0000 7f63b7e00700 1 -- 172.21.15.1:0/2185381273 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63a8010460 con 0x7f63c0108340 2024-02-02T15:34:41.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.182+0000 7f63b7e00700 1 -- 172.21.15.1:0/2185381273 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a800e610 con 0x7f63c0108340 2024-02-02T15:34:41.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.182+0000 7f63c6000700 1 -- 172.21.15.1:0/2185381273 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63c01a5ed0 con 0x7f63c0108340 2024-02-02T15:34:41.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.182+0000 7f63c6000700 1 -- 172.21.15.1:0/2185381273 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63c01a62a0 con 0x7f63c0108340 2024-02-02T15:34:41.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.185+0000 7f63b7e00700 1 -- 172.21.15.1:0/2185381273 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f63a8010890 con 0x7f63c0108340 2024-02-02T15:34:41.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.186+0000 7f63d9b18b80 1 Processor -- start 2024-02-02T15:34:41.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.187+0000 7f63d9b18b80 1 -- start start 2024-02-02T15:34:41.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.187+0000 7f63b7e00700 1 --2- 172.21.15.1:0/2185381273 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f63a00a0690 0x7f63a00a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:41.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.187+0000 7f63d9b18b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561b86484010 0x561b864843f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:41.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.187+0000 7f63b7e00700 1 -- 172.21.15.1:0/2185381273 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f63a80cbd40 con 0x7f63c0108340 2024-02-02T15:34:41.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.187+0000 7f63d9b18b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561b86484930 0x561b8648f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:41.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.187+0000 7f63bfe00700 1 --2- 172.21.15.1:0/2185381273 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f63a00a0690 0x7f63a00a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:41.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.187+0000 7f63d9b18b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561b8648f7e0 0x561b86491bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:41.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.187+0000 7f63d9b18b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b863f84c0 con 0x561b8648f7e0 2024-02-02T15:34:41.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.187+0000 7f63d9b18b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b863dcaa0 con 0x561b86484930 2024-02-02T15:34:41.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.187+0000 7f63d9b18b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b863da460 con 0x561b86484010 2024-02-02T15:34:41.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.187+0000 7f63bfe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561b8648f7e0 0x561b86491bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:41.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.188+0000 7f63bfe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561b8648f7e0 0x561b86491bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45354/0 (socket says 172.21.15.1:45354) 2024-02-02T15:34:41.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.188+0000 7f63bfe00700 1 -- 172.21.15.1:0/2124036049 learned_addr learned my addr 172.21.15.1:0/2124036049 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:34:41.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.187+0000 7f63bf400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561b86484010 0x561b864843f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:41.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.188+0000 7f63bea00700 1 --2- 172.21.15.1:0/2124036049 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561b86484930 0x561b8648f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:41.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.188+0000 7f63bf400700 1 -- 172.21.15.1:0/2124036049 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561b86484930 msgr2=0x561b8648f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:41.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.188+0000 7f63bf400700 1 --2- 172.21.15.1:0/2124036049 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561b86484930 0x561b8648f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.188+0000 7f63bf400700 1 -- 172.21.15.1:0/2124036049 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561b8648f7e0 msgr2=0x561b86491bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:41.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.188+0000 7f63bf400700 1 --2- 172.21.15.1:0/2124036049 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561b8648f7e0 0x561b86491bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.188+0000 7f63bf400700 1 -- 172.21.15.1:0/2124036049 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561b862573b0 con 0x561b86484010 2024-02-02T15:34:41.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.189+0000 7f63bfe00700 1 --2- 172.21.15.1:0/2185381273 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f63a00a0690 0x7f63a00a2b50 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f63b800d010 tx=0x7f63b8009420 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:41.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.189+0000 7f63bf400700 1 --2- 172.21.15.1:0/2124036049 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561b86484010 0x561b864843f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f63b00115a0 tx=0x7f63b00118b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41317603a7b130fb server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:41.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.189+0000 7f63afe00700 1 -- 172.21.15.1:0/2124036049 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b0004230 con 0x561b86484010 2024-02-02T15:34:41.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.189+0000 7f63afe00700 1 -- 172.21.15.1:0/2124036049 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63b0004390 con 0x561b86484010 2024-02-02T15:34:41.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.189+0000 7f63afe00700 1 -- 172.21.15.1:0/2124036049 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b0019920 con 0x561b86484010 2024-02-02T15:34:41.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.190+0000 7f63d9b18b80 1 -- 172.21.15.1:0/2124036049 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561b86484010 msgr2=0x561b864843f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:41.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.190+0000 7f63d9b18b80 1 --2- 172.21.15.1:0/2124036049 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561b86484010 0x561b864843f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f63b00115a0 tx=0x7f63b00118b0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.190+0000 7f63d9b18b80 1 -- 172.21.15.1:0/2124036049 shutdown_connections 2024-02-02T15:34:41.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.190+0000 7f63d9b18b80 1 --2- 172.21.15.1:0/2124036049 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561b8648f7e0 0x561b86491bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.190+0000 7f63d9b18b80 1 --2- 172.21.15.1:0/2124036049 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561b86484010 0x561b864843f0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.190+0000 7f63d9b18b80 1 --2- 172.21.15.1:0/2124036049 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561b86484930 0x561b8648f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.190+0000 7f63d9b18b80 1 -- 172.21.15.1:0/2124036049 >> 172.21.15.1:0/2124036049 conn(0x561b863e3db0 msgr2=0x561b864803b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:41.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.190+0000 7f63d9b18b80 1 -- 172.21.15.1:0/2124036049 shutdown_connections 2024-02-02T15:34:41.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.190+0000 7f63d9b18b80 1 -- 172.21.15.1:0/2124036049 wait complete. 2024-02-02T15:34:41.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.191+0000 7f63d9b18b80 1 -- start start 2024-02-02T15:34:41.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.191+0000 7f63d9b18b80 1 Processor -- start 2024-02-02T15:34:41.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.192+0000 7f63d9b18b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561b86484010 0x561b8648f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:41.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.192+0000 7f63d9b18b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561b86484930 0x561b8648c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:41.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.192+0000 7f63d9b18b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561b8648f7e0 0x561b8648c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:41.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.192+0000 7f63d9b18b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b863d5d50 con 0x561b86484010 2024-02-02T15:34:41.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.192+0000 7f63d9b18b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b86371d80 con 0x561b8648f7e0 2024-02-02T15:34:41.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.192+0000 7f63d9b18b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b86371a20 con 0x561b86484930 2024-02-02T15:34:41.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.192+0000 7f63bfe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561b8648f7e0 0x561b8648c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:41.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.192+0000 7f63bfe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561b8648f7e0 0x561b8648c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:58846/0 (socket says 172.21.15.1:58846) 2024-02-02T15:34:41.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.192+0000 7f63bfe00700 1 -- 172.21.15.1:0/3489399915 learned_addr learned my addr 172.21.15.1:0/3489399915 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:34:41.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.192+0000 7f63bf400700 1 --2- 172.21.15.1:0/3489399915 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561b86484010 0x561b8648f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:41.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.193+0000 7f63d9b18b80 10 client.0 _ll_register_callbacks cb 0x561b86373290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:34:41.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.193+0000 7f63bea00700 1 --2- 172.21.15.1:0/3489399915 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561b86484930 0x561b8648c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:41.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.193+0000 7f63bea00700 1 -- 172.21.15.1:0/3489399915 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561b8648f7e0 msgr2=0x561b8648c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:41.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.193+0000 7f63bea00700 1 --2- 172.21.15.1:0/3489399915 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561b8648f7e0 0x561b8648c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.193+0000 7f63bea00700 1 -- 172.21.15.1:0/3489399915 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561b86484010 msgr2=0x561b8648f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:41.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.193+0000 7f63bea00700 1 --2- 172.21.15.1:0/3489399915 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561b86484010 0x561b8648f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.193+0000 7f63bea00700 1 -- 172.21.15.1:0/3489399915 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63b0009d60 con 0x561b86484930 2024-02-02T15:34:41.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.193+0000 7f63bf400700 1 --2- 172.21.15.1:0/3489399915 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561b86484010 0x561b8648f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:34:41.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.194+0000 7f63bea00700 1 --2- 172.21.15.1:0/3489399915 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561b86484930 0x561b8648c390 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f63a80c9090 tx=0x7f63a80995e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38731f45ba489970 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:41.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.194+0000 7f63acc00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:34:41.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.194+0000 7f63acc00700 1 -- 172.21.15.1:0/3489399915 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a80992c0 con 0x561b86484930 2024-02-02T15:34:41.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.194+0000 7f63acc00700 1 -- 172.21.15.1:0/3489399915 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f63a80a0ed0 con 0x561b86484930 2024-02-02T15:34:41.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.194+0000 7f63d9b18b80 1 -- 172.21.15.1:0/3489399915 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561b864780f0 con 0x561b86484930 2024-02-02T15:34:41.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.194+0000 7f63acc00700 1 -- 172.21.15.1:0/3489399915 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63a80b1200 con 0x561b86484930 2024-02-02T15:34:41.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.194+0000 7f63d9b18b80 10 client.27658 fetch_fsmap learned FSMap version 41 2024-02-02T15:34:41.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.194+0000 7f63d9b18b80 1 -- 172.21.15.1:0/3489399915 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x561b86203b60 con 0x561b86484930 2024-02-02T15:34:41.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.195+0000 7f63acc00700 1 -- 172.21.15.1:0/3489399915 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561b864780f0 con 0x561b86484930 2024-02-02T15:34:41.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.196+0000 7f63acc00700 1 -- 172.21.15.1:0/3489399915 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 41) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f63a80a0b40 con 0x561b86484930 2024-02-02T15:34:41.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.197+0000 7f63d9b18b80 10 client.27658 fetch_fsmap finished waiting for FSMap version 41 2024-02-02T15:34:41.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.197+0000 7f63d9b18b80 10 client.27658 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-02-02T15:34:41.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.197+0000 7f63d9b18b80 20 client.27658 populate_metadata read hostname 'smithi001' 2024-02-02T15:34:41.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.197+0000 7f63d9b18b80 1 --2- 172.21.15.1:0/3489399915 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x561b86495fb0 0x561b864983d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:41.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.197+0000 7f63d9b18b80 4 client.27658 mds_command: new command op to 24514 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:34:41.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.197+0000 7f63d9b18b80 1 -- 172.21.15.1:0/3489399915 --> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561b863cedd0 con 0x561b86495fb0 2024-02-02T15:34:41.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.197+0000 7f63bf400700 1 --2- 172.21.15.1:0/3489399915 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x561b86495fb0 0x561b864983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:41.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.200+0000 7f63bf400700 1 --2- 172.21.15.1:0/3489399915 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x561b86495fb0 0x561b864983d0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9d69e1e8af37f55d server_cookie=e9d089ce4cd00184 in_seq=0 out_seq=0 2024-02-02T15:34:41.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.200+0000 7f63acc00700 10 client.27658 ms_handle_connect on v2:172.21.15.142:6836/2870289019 2024-02-02T15:34:41.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.201+0000 7f63acc00700 1 -- 172.21.15.1:0/3489399915 <== mds.3 v2:172.21.15.142:6836/2870289019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x561b863cedd0 con 0x561b86495fb0 2024-02-02T15:34:41.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.201+0000 7f63acc00700 10 client.27658 handle_command_reply: tid=0 2024-02-02T15:34:41.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.201+0000 7f63d9b18b80 1 client.27658 shutdown 2024-02-02T15:34:41.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.202+0000 7f63d9b18b80 1 -- 172.21.15.1:0/3489399915 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561b86484930 msgr2=0x561b8648c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:41.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.202+0000 7f63d9b18b80 1 --2- 172.21.15.1:0/3489399915 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561b86484930 0x561b8648c390 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f63a80c9090 tx=0x7f63a80995e0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.203+0000 7f63d9b18b80 1 -- 172.21.15.1:0/3489399915 shutdown_connections 2024-02-02T15:34:41.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.203+0000 7f63d9b18b80 1 --2- 172.21.15.1:0/3489399915 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561b8648f7e0 0x561b8648c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.203+0000 7f63d9b18b80 1 --2- 172.21.15.1:0/3489399915 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x561b86495fb0 0x561b864983d0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.203+0000 7f63d9b18b80 1 --2- 172.21.15.1:0/3489399915 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561b86484010 0x561b8648f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.203+0000 7f63d9b18b80 1 --2- 172.21.15.1:0/3489399915 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561b86484930 0x561b8648c390 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.203+0000 7f63d9b18b80 1 -- 172.21.15.1:0/3489399915 >> 172.21.15.1:0/3489399915 conn(0x561b863e3db0 msgr2=0x561b8647ffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:41.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.203+0000 7f63acc00700 0 client.27658 ms_handle_reset on v2:172.21.15.142:6836/2870289019 2024-02-02T15:34:41.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.203+0000 7f63d9b18b80 1 -- 172.21.15.1:0/3489399915 shutdown_connections 2024-02-02T15:34:41.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.204+0000 7f63d9b18b80 1 -- 172.21.15.1:0/3489399915 wait complete. 2024-02-02T15:34:41.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.204+0000 7f63d9b18b80 20 client.27658 trim_cache size 0 max 16384 2024-02-02T15:34:41.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.227+0000 7f63d9b18b80 1 Processor -- start 2024-02-02T15:34:41.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.227+0000 7f63d9b18b80 1 -- start start 2024-02-02T15:34:41.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.228+0000 7f63d9b18b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561b86484010 0x561b8648ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:41.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.228+0000 7f63d9b18b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561b86484930 0x561b864876a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:41.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.228+0000 7f63d9b18b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561b8648f7e0 0x561b86487be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:41.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.228+0000 7f63d9b18b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b862596a0 con 0x561b86484010 2024-02-02T15:34:41.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.228+0000 7f63d9b18b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b86481060 con 0x561b8648f7e0 2024-02-02T15:34:41.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.228+0000 7f63d9b18b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b8647f1a0 con 0x561b86484930 2024-02-02T15:34:41.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.228+0000 7f63bfe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561b8648f7e0 0x561b86487be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:41.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.228+0000 7f63bf400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561b86484010 0x561b8648ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:41.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.228+0000 7f63bfe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561b8648f7e0 0x561b86487be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:58856/0 (socket says 172.21.15.1:58856) 2024-02-02T15:34:41.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.228+0000 7f63bfe00700 1 -- 172.21.15.1:0/4186088887 learned_addr learned my addr 172.21.15.1:0/4186088887 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:34:41.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.228+0000 7f63bea00700 1 --2- 172.21.15.1:0/4186088887 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561b86484930 0x561b864876a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:41.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.228+0000 7f63bfe00700 1 -- 172.21.15.1:0/4186088887 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561b86484930 msgr2=0x561b864876a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:41.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.228+0000 7f63bfe00700 1 --2- 172.21.15.1:0/4186088887 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561b86484930 0x561b864876a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.228+0000 7f63bfe00700 1 -- 172.21.15.1:0/4186088887 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561b86484010 msgr2=0x561b8648ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:41.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.228+0000 7f63bfe00700 1 --2- 172.21.15.1:0/4186088887 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561b86484010 0x561b8648ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.228+0000 7f63bfe00700 1 -- 172.21.15.1:0/4186088887 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561b8639bea0 con 0x561b8648f7e0 2024-02-02T15:34:41.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.229+0000 7f63bfe00700 1 --2- 172.21.15.1:0/4186088887 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561b8648f7e0 0x561b86487be0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f63b800b0a0 tx=0x7f63b8028000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d2356f681b9b37b server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:41.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.229+0000 7f63b5200700 1 -- 172.21.15.1:0/4186088887 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b8025070 con 0x561b8648f7e0 2024-02-02T15:34:41.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.229+0000 7f63b5200700 1 -- 172.21.15.1:0/4186088887 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63b8020450 con 0x561b8648f7e0 2024-02-02T15:34:41.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.229+0000 7f63b5200700 1 -- 172.21.15.1:0/4186088887 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b802b680 con 0x561b8648f7e0 2024-02-02T15:34:41.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.230+0000 7f63d9b18b80 1 -- 172.21.15.1:0/4186088887 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561b8648f7e0 msgr2=0x561b86487be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:41.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.230+0000 7f63d9b18b80 1 --2- 172.21.15.1:0/4186088887 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561b8648f7e0 0x561b86487be0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f63b800b0a0 tx=0x7f63b8028000 comp rx=0 tx=0).stop 2024-02-02T15:34:41.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.230+0000 7f63d9b18b80 1 -- 172.21.15.1:0/4186088887 shutdown_connections 2024-02-02T15:34:41.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.230+0000 7f63d9b18b80 1 --2- 172.21.15.1:0/4186088887 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561b8648f7e0 0x561b86487be0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.230+0000 7f63d9b18b80 1 --2- 172.21.15.1:0/4186088887 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561b86484010 0x561b8648ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.230+0000 7f63d9b18b80 1 --2- 172.21.15.1:0/4186088887 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561b86484930 0x561b864876a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.230+0000 7f63d9b18b80 1 -- 172.21.15.1:0/4186088887 >> 172.21.15.1:0/4186088887 conn(0x561b863e3db0 msgr2=0x561b86497fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:41.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.230+0000 7f63d9b18b80 1 -- 172.21.15.1:0/4186088887 shutdown_connections 2024-02-02T15:34:41.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.230+0000 7f63d9b18b80 1 -- 172.21.15.1:0/4186088887 wait complete. 2024-02-02T15:34:41.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.230+0000 7f63d9b18b80 1 -- start start 2024-02-02T15:34:41.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.231+0000 7f63d9b18b80 1 Processor -- start 2024-02-02T15:34:41.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.231+0000 7f63d9b18b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561b86484010 0x561b8649a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:41.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.231+0000 7f63d9b18b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561b86484930 0x561b8649aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:41.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.231+0000 7f63d9b18b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561b8648f7e0 0x561b865009f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:41.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.231+0000 7f63d9b18b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b8649b210 con 0x561b8648f7e0 2024-02-02T15:34:41.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.231+0000 7f63d9b18b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b8649b380 con 0x561b86484930 2024-02-02T15:34:41.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.231+0000 7f63d9b18b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561b86500f60 con 0x561b86484010 2024-02-02T15:34:41.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.231+0000 7f63d9b18b80 10 client.0 _ll_register_callbacks cb 0x561b86373290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:34:41.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.231+0000 7f63bf400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561b86484010 0x561b8649a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:41.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.231+0000 7f63bf400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561b86484010 0x561b8649a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:60858/0 (socket says 172.21.15.1:60858) 2024-02-02T15:34:41.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.231+0000 7f63bf400700 1 -- 172.21.15.1:0/2477853532 learned_addr learned my addr 172.21.15.1:0/2477853532 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:34:41.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.231+0000 7f63bea00700 1 --2- 172.21.15.1:0/2477853532 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561b86484930 0x561b8649aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:41.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.232+0000 7f63bf400700 1 -- 172.21.15.1:0/2477853532 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561b86484930 msgr2=0x561b8649aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:41.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.232+0000 7f63bf400700 1 --2- 172.21.15.1:0/2477853532 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561b86484930 0x561b8649aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.232+0000 7f63bf400700 1 -- 172.21.15.1:0/2477853532 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561b8648f7e0 msgr2=0x561b865009f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:41.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.232+0000 7f63bfe00700 1 --2- 172.21.15.1:0/2477853532 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561b8648f7e0 0x561b865009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:41.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.232+0000 7f63bf400700 1 --2- 172.21.15.1:0/2477853532 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561b8648f7e0 0x561b865009f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.232+0000 7f63bf400700 1 -- 172.21.15.1:0/2477853532 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63c0110730 con 0x561b86484010 2024-02-02T15:34:41.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.232+0000 7f63bfe00700 1 --2- 172.21.15.1:0/2477853532 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561b8648f7e0 0x561b865009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:34:41.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.232+0000 7f63bf400700 1 --2- 172.21.15.1:0/2477853532 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561b86484010 0x561b8649a520 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f63b0011c00 tx=0x7f63b00042c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b49c414e2a9890ff server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:41.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.232+0000 7f63acc00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:34:41.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.232+0000 7f63acc00700 1 -- 172.21.15.1:0/2477853532 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b0005610 con 0x561b86484010 2024-02-02T15:34:41.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.232+0000 7f63acc00700 1 -- 172.21.15.1:0/2477853532 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f63b0029070 con 0x561b86484010 2024-02-02T15:34:41.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.232+0000 7f63d9b18b80 1 -- 172.21.15.1:0/2477853532 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f63b0004390 con 0x561b86484010 2024-02-02T15:34:41.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.233+0000 7f63acc00700 1 -- 172.21.15.1:0/2477853532 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63b00046f0 con 0x561b86484010 2024-02-02T15:34:41.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.233+0000 7f63d9b18b80 10 client.27664 fetch_fsmap learned FSMap version 41 2024-02-02T15:34:41.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.233+0000 7f63d9b18b80 1 -- 172.21.15.1:0/2477853532 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) v3 -- 0x561b8625bcd0 con 0x561b86484010 2024-02-02T15:34:41.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.233+0000 7f63acc00700 1 -- 172.21.15.1:0/2477853532 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f63b0004390 con 0x561b86484010 2024-02-02T15:34:41.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.233+0000 7f63acc00700 1 -- 172.21.15.1:0/2477853532 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 41) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f63b00115d0 con 0x561b86484010 2024-02-02T15:34:41.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.233+0000 7f63d9b18b80 10 client.27664 fetch_fsmap finished waiting for FSMap version 41 2024-02-02T15:34:41.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.233+0000 7f63d9b18b80 10 client.27664 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-02-02T15:34:41.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.233+0000 7f63d9b18b80 20 client.27664 populate_metadata read hostname 'smithi001' 2024-02-02T15:34:41.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.233+0000 7f63d9b18b80 1 --2- 172.21.15.1:0/2477853532 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x561b865025a0 0x561b865049c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:41.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.233+0000 7f63d9b18b80 4 client.27664 mds_command: new command op to 24514 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:34:41.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.233+0000 7f63d9b18b80 1 -- 172.21.15.1:0/2477853532 --> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561b86394f50 con 0x561b865025a0 2024-02-02T15:34:41.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.234+0000 7f63bea00700 1 --2- 172.21.15.1:0/2477853532 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x561b865025a0 0x561b865049c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:41.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.234+0000 7f63bea00700 1 --2- 172.21.15.1:0/2477853532 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x561b865025a0 0x561b865049c0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=199cd0fd26467db4 server_cookie=d53b598281f9126f in_seq=0 out_seq=0 2024-02-02T15:34:41.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.234+0000 7f63acc00700 10 client.27664 ms_handle_connect on v2:172.21.15.142:6836/2870289019 2024-02-02T15:34:41.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.234+0000 7f63acc00700 1 -- 172.21.15.1:0/2477853532 <== mds.3 v2:172.21.15.142:6836/2870289019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561b86394f50 con 0x561b865025a0 2024-02-02T15:34:41.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.234+0000 7f63acc00700 10 client.27664 handle_command_reply: tid=0 2024-02-02T15:34:41.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.234+0000 7f63d9b18b80 1 client.27664 shutdown 2024-02-02T15:34:41.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.234+0000 7f63d9b18b80 1 -- 172.21.15.1:0/2477853532 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561b86484010 msgr2=0x561b8649a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:41.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.234+0000 7f63d9b18b80 1 --2- 172.21.15.1:0/2477853532 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561b86484010 0x561b8649a520 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f63b0011c00 tx=0x7f63b00042c0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.235+0000 7f63d9b18b80 1 -- 172.21.15.1:0/2477853532 shutdown_connections 2024-02-02T15:34:41.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.235+0000 7f63d9b18b80 1 --2- 172.21.15.1:0/2477853532 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x561b865025a0 0x561b865049c0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.235+0000 7f63d9b18b80 1 --2- 172.21.15.1:0/2477853532 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561b8648f7e0 0x561b865009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.235+0000 7f63d9b18b80 1 --2- 172.21.15.1:0/2477853532 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561b86484010 0x561b8649a520 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.235+0000 7f63d9b18b80 1 --2- 172.21.15.1:0/2477853532 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561b86484930 0x561b8649aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.235+0000 7f63acc00700 0 client.27664 ms_handle_reset on v2:172.21.15.142:6836/2870289019 2024-02-02T15:34:41.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.235+0000 7f63d9b18b80 1 -- 172.21.15.1:0/2477853532 >> 172.21.15.1:0/2477853532 conn(0x561b863e3db0 msgr2=0x561b864876a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:41.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.235+0000 7f63d9b18b80 1 -- 172.21.15.1:0/2477853532 shutdown_connections 2024-02-02T15:34:41.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.235+0000 7f63d9b18b80 1 -- 172.21.15.1:0/2477853532 wait complete. 2024-02-02T15:34:41.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.235+0000 7f63d9b18b80 20 client.27664 trim_cache size 0 max 16384 2024-02-02T15:34:41.239 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:34:41.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.235+0000 7f63c6000700 1 -- 172.21.15.1:0/2185381273 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f63a00a0690 msgr2=0x7f63a00a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:41.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.235+0000 7f63c6000700 1 --2- 172.21.15.1:0/2185381273 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f63a00a0690 0x7f63a00a2b50 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f63b800d010 tx=0x7f63b8009420 comp rx=0 tx=0).stop 2024-02-02T15:34:41.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.235+0000 7f63c6000700 1 -- 172.21.15.1:0/2185381273 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f63c0108340 msgr2=0x7f63c0199760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:41.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.235+0000 7f63c6000700 1 --2- 172.21.15.1:0/2185381273 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f63c0108340 0x7f63c0199760 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f63c0068d60 tx=0x7f63a800dc30 comp rx=0 tx=0).stop 2024-02-02T15:34:41.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.236+0000 7f63c6000700 1 -- 172.21.15.1:0/2185381273 shutdown_connections 2024-02-02T15:34:41.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.236+0000 7f63c6000700 1 --2- 172.21.15.1:0/2185381273 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f63a00a0690 0x7f63a00a2b50 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.236+0000 7f63c6000700 1 --2- 172.21.15.1:0/2185381273 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f63c019dbc0 0x7f63c019e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.236+0000 7f63c6000700 1 --2- 172.21.15.1:0/2185381273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f63c0103220 0x7f63c0199220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.236+0000 7f63c6000700 1 --2- 172.21.15.1:0/2185381273 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f63c0108340 0x7f63c0199760 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.236+0000 7f63c6000700 1 -- 172.21.15.1:0/2185381273 >> 172.21.15.1:0/2185381273 conn(0x7f63c00fea60 msgr2=0x7f63c00ff500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:41.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.236+0000 7f63c6000700 1 -- 172.21.15.1:0/2185381273 shutdown_connections 2024-02-02T15:34:41.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.236+0000 7f63c6000700 1 -- 172.21.15.1:0/2185381273 wait complete. 2024-02-02T15:34:41.246 INFO:tasks.workunit.client.0.smithi001.stdout: 1 463338 6.90 MB/sec execute 255 sec latency 32.820 ms 2024-02-02T15:34:41.254 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-02T15:34:41.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.467+0000 7f56bf000700 1 -- 172.21.15.1:0/2436393733 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56b8101010 msgr2=0x7f56b81013f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:41.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.467+0000 7f56bf000700 1 --2- 172.21.15.1:0/2436393733 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56b8101010 0x7f56b81013f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f56a8009990 tx=0x7f56a8009ca0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.467+0000 7f56bf000700 1 -- 172.21.15.1:0/2436393733 shutdown_connections 2024-02-02T15:34:41.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.467+0000 7f56bf000700 1 --2- 172.21.15.1:0/2436393733 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f56b8106130 0x7f56b810dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.467+0000 7f56bf000700 1 --2- 172.21.15.1:0/2436393733 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56b8101010 0x7f56b81013f0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.467+0000 7f56bf000700 1 --2- 172.21.15.1:0/2436393733 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f56b81019c0 0x7f56b8105a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.467+0000 7f56bf000700 1 -- 172.21.15.1:0/2436393733 >> 172.21.15.1:0/2436393733 conn(0x7f56b80fc8b0 msgr2=0x7f56b80fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:41.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.467+0000 7f56bf000700 1 -- 172.21.15.1:0/2436393733 shutdown_connections 2024-02-02T15:34:41.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.468+0000 7f56bf000700 1 -- 172.21.15.1:0/2436393733 wait complete. 2024-02-02T15:34:41.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.468+0000 7f56bf000700 1 Processor -- start 2024-02-02T15:34:41.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.469+0000 7f56bf000700 1 -- start start 2024-02-02T15:34:41.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.469+0000 7f56bf000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f56b8101010 0x7f56b8073040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:41.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.469+0000 7f56bf000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56b8106130 0x7f56b8073580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:41.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.469+0000 7f56bf000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f56b8073ac0 0x7f56b81a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:41.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.469+0000 7f56bf000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56b81a5750 con 0x7f56b8101010 2024-02-02T15:34:41.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.469+0000 7f56bf000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56b81a58c0 con 0x7f56b8106130 2024-02-02T15:34:41.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.469+0000 7f56bf000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56b81a5a80 con 0x7f56b8073ac0 2024-02-02T15:34:41.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.469+0000 7f56b7e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f56b8101010 0x7f56b8073040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:41.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.469+0000 7f56bd000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f56b8073ac0 0x7f56b81a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:41.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.469+0000 7f56afe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56b8106130 0x7f56b8073580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:41.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.469+0000 7f56b7e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f56b8101010 0x7f56b8073040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45382/0 (socket says 172.21.15.1:45382) 2024-02-02T15:34:41.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.469+0000 7f56b7e00700 1 -- 172.21.15.1:0/752926653 learned_addr learned my addr 172.21.15.1:0/752926653 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:34:41.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.470+0000 7f56bd000700 1 -- 172.21.15.1:0/752926653 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56b8106130 msgr2=0x7f56b8073580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:41.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.470+0000 7f56bd000700 1 --2- 172.21.15.1:0/752926653 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56b8106130 0x7f56b8073580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.470+0000 7f56bd000700 1 -- 172.21.15.1:0/752926653 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f56b8101010 msgr2=0x7f56b8073040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:41.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.470+0000 7f56bd000700 1 --2- 172.21.15.1:0/752926653 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f56b8101010 0x7f56b8073040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.470+0000 7f56bd000700 1 -- 172.21.15.1:0/752926653 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56b0009c40 con 0x7f56b8073ac0 2024-02-02T15:34:41.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.470+0000 7f56b7e00700 1 --2- 172.21.15.1:0/752926653 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f56b8101010 0x7f56b8073040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:34:41.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.470+0000 7f56bd000700 1 --2- 172.21.15.1:0/752926653 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f56b8073ac0 0x7f56b81a50c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f56b000efb0 tx=0x7f56b000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:41.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.470+0000 7f56b5600700 1 -- 172.21.15.1:0/752926653 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b0003a00 con 0x7f56b8073ac0 2024-02-02T15:34:41.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.470+0000 7f56b5600700 1 -- 172.21.15.1:0/752926653 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f56b000f040 con 0x7f56b8073ac0 2024-02-02T15:34:41.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.470+0000 7f56bf000700 1 -- 172.21.15.1:0/752926653 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56a80095f0 con 0x7f56b8073ac0 2024-02-02T15:34:41.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.470+0000 7f56b5600700 1 -- 172.21.15.1:0/752926653 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b0004540 con 0x7f56b8073ac0 2024-02-02T15:34:41.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.471+0000 7f56bf000700 1 -- 172.21.15.1:0/752926653 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56b81a6130 con 0x7f56b8073ac0 2024-02-02T15:34:41.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.473+0000 7f56b5600700 1 -- 172.21.15.1:0/752926653 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f56b0003bf0 con 0x7f56b8073ac0 2024-02-02T15:34:41.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.475+0000 7f56b5600700 1 --2- 172.21.15.1:0/752926653 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f56980a0710 0x7f56980a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:41.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.475+0000 7f56b5600700 1 -- 172.21.15.1:0/752926653 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f56b00cbc80 con 0x7f56b8073ac0 2024-02-02T15:34:41.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.475+0000 7f56b7e00700 1 --2- 172.21.15.1:0/752926653 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f56980a0710 0x7f56980a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:41.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.475+0000 7f56d2bd4b80 1 Processor -- start 2024-02-02T15:34:41.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.475+0000 7f56d2bd4b80 1 -- start start 2024-02-02T15:34:41.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.476+0000 7f56d2bd4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e24c68c820 0x55e24c68cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:41.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.476+0000 7f56d2bd4b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e24c68d140 0x55e24c697ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:41.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.476+0000 7f56d2bd4b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e24c697ff0 0x55e24c69a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:41.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.476+0000 7f56d2bd4b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e24c5f84c0 con 0x55e24c697ff0 2024-02-02T15:34:41.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.476+0000 7f56d2bd4b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e24c5dcaa0 con 0x55e24c68d140 2024-02-02T15:34:41.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.476+0000 7f56d2bd4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e24c5da460 con 0x55e24c68c820 2024-02-02T15:34:41.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.476+0000 7f56b7e00700 1 --2- 172.21.15.1:0/752926653 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f56980a0710 0x7f56980a2bd0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f56a8006010 tx=0x7f56a800b260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:41.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.476+0000 7f56b7e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e24c68c820 0x55e24c68cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:41.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.476+0000 7f56afe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e24c68d140 0x55e24c697ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:41.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.476+0000 7f56afe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e24c68d140 0x55e24c697ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:58904/0 (socket says 172.21.15.1:58904) 2024-02-02T15:34:41.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.476+0000 7f56afe00700 1 -- 172.21.15.1:0/3792339308 learned_addr learned my addr 172.21.15.1:0/3792339308 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:34:41.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.476+0000 7f56bd000700 1 --2- 172.21.15.1:0/3792339308 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e24c697ff0 0x55e24c69a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:41.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.477+0000 7f56afe00700 1 -- 172.21.15.1:0/3792339308 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e24c68c820 msgr2=0x55e24c68cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:41.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.477+0000 7f56afe00700 1 --2- 172.21.15.1:0/3792339308 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e24c68c820 0x55e24c68cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.477+0000 7f56afe00700 1 -- 172.21.15.1:0/3792339308 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e24c697ff0 msgr2=0x55e24c69a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:41.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.477+0000 7f56afe00700 1 --2- 172.21.15.1:0/3792339308 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e24c697ff0 0x55e24c69a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.477+0000 7f56afe00700 1 -- 172.21.15.1:0/3792339308 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e24c4573b0 con 0x55e24c68d140 2024-02-02T15:34:41.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.477+0000 7f56b7e00700 1 --2- 172.21.15.1:0/3792339308 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e24c68c820 0x55e24c68cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:34:41.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.477+0000 7f56bd000700 1 --2- 172.21.15.1:0/3792339308 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e24c697ff0 0x55e24c69a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:34:41.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.477+0000 7f56afe00700 1 --2- 172.21.15.1:0/3792339308 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e24c68d140 0x55e24c697ab0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f56a0009d20 tx=0x7f56a0011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8b1cec6e0396bc2 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:41.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.477+0000 7f56ad600700 1 -- 172.21.15.1:0/3792339308 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a0004250 con 0x55e24c68d140 2024-02-02T15:34:41.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.477+0000 7f56ad600700 1 -- 172.21.15.1:0/3792339308 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f56a0004cf0 con 0x55e24c68d140 2024-02-02T15:34:41.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.477+0000 7f56ad600700 1 -- 172.21.15.1:0/3792339308 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a00055f0 con 0x55e24c68d140 2024-02-02T15:34:41.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.478+0000 7f56d2bd4b80 1 -- 172.21.15.1:0/3792339308 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e24c68d140 msgr2=0x55e24c697ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:41.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.478+0000 7f56d2bd4b80 1 --2- 172.21.15.1:0/3792339308 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e24c68d140 0x55e24c697ab0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f56a0009d20 tx=0x7f56a0011720 comp rx=0 tx=0).stop 2024-02-02T15:34:41.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.478+0000 7f56d2bd4b80 1 -- 172.21.15.1:0/3792339308 shutdown_connections 2024-02-02T15:34:41.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.478+0000 7f56d2bd4b80 1 --2- 172.21.15.1:0/3792339308 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e24c697ff0 0x55e24c69a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.478+0000 7f56d2bd4b80 1 --2- 172.21.15.1:0/3792339308 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e24c68c820 0x55e24c68cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.478+0000 7f56d2bd4b80 1 --2- 172.21.15.1:0/3792339308 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e24c68d140 0x55e24c697ab0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.478+0000 7f56d2bd4b80 1 -- 172.21.15.1:0/3792339308 >> 172.21.15.1:0/3792339308 conn(0x55e24c5e3db0 msgr2=0x55e24c6803b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:41.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.478+0000 7f56d2bd4b80 1 -- 172.21.15.1:0/3792339308 shutdown_connections 2024-02-02T15:34:41.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.478+0000 7f56d2bd4b80 1 -- 172.21.15.1:0/3792339308 wait complete. 2024-02-02T15:34:41.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.479+0000 7f56d2bd4b80 1 -- start start 2024-02-02T15:34:41.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.479+0000 7f56d2bd4b80 1 Processor -- start 2024-02-02T15:34:41.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.480+0000 7f56d2bd4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e24c68c820 0x55e24c6979e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:41.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.480+0000 7f56d2bd4b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e24c68d140 0x55e24c694bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:41.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.480+0000 7f56d2bd4b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e24c697ff0 0x55e24c695200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:41.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.480+0000 7f56d2bd4b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e24c5d5d50 con 0x55e24c68d140 2024-02-02T15:34:41.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.480+0000 7f56d2bd4b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e24c571d80 con 0x55e24c697ff0 2024-02-02T15:34:41.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.480+0000 7f56d2bd4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e24c571a20 con 0x55e24c68c820 2024-02-02T15:34:41.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.480+0000 7f56d2bd4b80 10 client.0 _ll_register_callbacks cb 0x55e24c573290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:34:41.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.480+0000 7f56b7e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e24c68c820 0x55e24c6979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:41.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.480+0000 7f56afe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e24c68d140 0x55e24c694bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:41.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.480+0000 7f56b7e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e24c68c820 0x55e24c6979e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:60908/0 (socket says 172.21.15.1:60908) 2024-02-02T15:34:41.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.480+0000 7f56b7e00700 1 -- 172.21.15.1:0/869713905 learned_addr learned my addr 172.21.15.1:0/869713905 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:34:41.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.480+0000 7f56bd000700 1 --2- 172.21.15.1:0/869713905 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e24c697ff0 0x55e24c695200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:41.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.481+0000 7f56b7e00700 1 -- 172.21.15.1:0/869713905 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e24c697ff0 msgr2=0x55e24c695200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:41.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.481+0000 7f56b7e00700 1 --2- 172.21.15.1:0/869713905 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e24c697ff0 0x55e24c695200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.481+0000 7f56b7e00700 1 -- 172.21.15.1:0/869713905 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e24c68d140 msgr2=0x55e24c694bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:41.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.481+0000 7f56b7e00700 1 --2- 172.21.15.1:0/869713905 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e24c68d140 0x55e24c694bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.481+0000 7f56b7e00700 1 -- 172.21.15.1:0/869713905 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56b81a6130 con 0x55e24c68c820 2024-02-02T15:34:41.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.481+0000 7f56b7e00700 1 --2- 172.21.15.1:0/869713905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e24c68c820 0x55e24c6979e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f56a800fbe0 tx=0x7f56a801a050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31949d186159a8bd server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:41.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.481+0000 7f56a6000700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:34:41.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.484+0000 7f56a6000700 1 -- 172.21.15.1:0/869713905 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a8028030 con 0x55e24c68c820 2024-02-02T15:34:41.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.485+0000 7f56d2bd4b80 1 -- 172.21.15.1:0/869713905 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e24c6780f0 con 0x55e24c68c820 2024-02-02T15:34:41.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.485+0000 7f56a6000700 1 -- 172.21.15.1:0/869713905 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f56a8019aa0 con 0x55e24c68c820 2024-02-02T15:34:41.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.488+0000 7f56a6000700 1 -- 172.21.15.1:0/869713905 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f56a8019ca0 con 0x55e24c68c820 2024-02-02T15:34:41.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.488+0000 7f56d2bd4b80 10 client.27676 fetch_fsmap learned FSMap version 42 2024-02-02T15:34:41.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.488+0000 7f56d2bd4b80 1 -- 172.21.15.1:0/869713905 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7f56a0009920 con 0x55e24c68c820 2024-02-02T15:34:41.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.488+0000 7f56a6000700 1 -- 172.21.15.1:0/869713905 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e24c6780f0 con 0x55e24c68c820 2024-02-02T15:34:41.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.488+0000 7f56a6000700 1 -- 172.21.15.1:0/869713905 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 42) v1 ==== 7510+0+0 (secure 0 0 0) 0x7f56a802fc60 con 0x55e24c68c820 2024-02-02T15:34:41.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.489+0000 7f56d2bd4b80 10 client.27676 fetch_fsmap finished waiting for FSMap version 42 2024-02-02T15:34:41.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.489+0000 7f56d2bd4b80 10 client.27676 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-02-02T15:34:41.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.489+0000 7f56d2bd4b80 20 client.27676 populate_metadata read hostname 'smithi001' 2024-02-02T15:34:41.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.489+0000 7f56d2bd4b80 1 --2- 172.21.15.1:0/869713905 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55e24c6a79b0 0x55e24c6a9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:41.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.489+0000 7f56d2bd4b80 4 client.27676 mds_command: new command op to 24518 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:34:41.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.489+0000 7f56d2bd4b80 1 -- 172.21.15.1:0/869713905 --> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e24c5cedd0 con 0x55e24c6a79b0 2024-02-02T15:34:41.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.489+0000 7f56afe00700 1 --2- 172.21.15.1:0/869713905 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55e24c6a79b0 0x55e24c6a9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:41.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.490+0000 7f56afe00700 1 --2- 172.21.15.1:0/869713905 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55e24c6a79b0 0x55e24c6a9dd0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6242eb6725cf6dcf server_cookie=72149eb9073049bd in_seq=0 out_seq=0 2024-02-02T15:34:41.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.490+0000 7f56a6000700 10 client.27676 ms_handle_connect on v2:172.21.15.71:6836/628016199 2024-02-02T15:34:41.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.490+0000 7f56a6000700 1 -- 172.21.15.1:0/869713905 <== mds.2 v2:172.21.15.71:6836/628016199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55e24c5cedd0 con 0x55e24c6a79b0 2024-02-02T15:34:41.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.490+0000 7f56a6000700 10 client.27676 handle_command_reply: tid=0 2024-02-02T15:34:41.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.491+0000 7f56d2bd4b80 1 client.27676 shutdown 2024-02-02T15:34:41.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.491+0000 7f56d2bd4b80 1 -- 172.21.15.1:0/869713905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e24c68c820 msgr2=0x55e24c6979e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:41.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.491+0000 7f56d2bd4b80 1 --2- 172.21.15.1:0/869713905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e24c68c820 0x55e24c6979e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f56a800fbe0 tx=0x7f56a801a050 comp rx=0 tx=0).stop 2024-02-02T15:34:41.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.491+0000 7f56d2bd4b80 1 -- 172.21.15.1:0/869713905 shutdown_connections 2024-02-02T15:34:41.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.491+0000 7f56d2bd4b80 1 --2- 172.21.15.1:0/869713905 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e24c697ff0 0x55e24c695200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.491+0000 7f56d2bd4b80 1 --2- 172.21.15.1:0/869713905 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55e24c6a79b0 0x55e24c6a9dd0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.491+0000 7f56d2bd4b80 1 --2- 172.21.15.1:0/869713905 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e24c68c820 0x55e24c6979e0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.491+0000 7f56d2bd4b80 1 --2- 172.21.15.1:0/869713905 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e24c68d140 0x55e24c694bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.491+0000 7f56a6000700 0 client.27676 ms_handle_reset on v2:172.21.15.71:6836/628016199 2024-02-02T15:34:41.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.491+0000 7f56d2bd4b80 1 -- 172.21.15.1:0/869713905 >> 172.21.15.1:0/869713905 conn(0x55e24c5e3db0 msgr2=0x55e24c67ffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:41.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.491+0000 7f56d2bd4b80 1 -- 172.21.15.1:0/869713905 shutdown_connections 2024-02-02T15:34:41.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.491+0000 7f56d2bd4b80 1 -- 172.21.15.1:0/869713905 wait complete. 2024-02-02T15:34:41.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.491+0000 7f56d2bd4b80 20 client.27676 trim_cache size 0 max 16384 2024-02-02T15:34:41.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.507+0000 7f56d2bd4b80 1 Processor -- start 2024-02-02T15:34:41.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.507+0000 7f56d2bd4b80 1 -- start start 2024-02-02T15:34:41.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.508+0000 7f56d2bd4b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e24c68c820 0x55e24c697510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:41.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.508+0000 7f56d2bd4b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e24c68d140 0x55e24c694490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:41.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.508+0000 7f56d2bd4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e24c697ff0 0x55e24c6949d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:41.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.508+0000 7f56d2bd4b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e24c4596a0 con 0x55e24c68c820 2024-02-02T15:34:41.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.508+0000 7f56d2bd4b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e24c690800 con 0x55e24c68d140 2024-02-02T15:34:41.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.508+0000 7f56d2bd4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e24c6a39e0 con 0x55e24c697ff0 2024-02-02T15:34:41.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.508+0000 7f56afe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e24c68d140 0x55e24c694490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:41.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.508+0000 7f56bd000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e24c697ff0 0x55e24c6949d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:41.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.508+0000 7f56afe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e24c68d140 0x55e24c694490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:58920/0 (socket says 172.21.15.1:58920) 2024-02-02T15:34:41.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.508+0000 7f56afe00700 1 -- 172.21.15.1:0/1346469904 learned_addr learned my addr 172.21.15.1:0/1346469904 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:34:41.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.508+0000 7f56b7e00700 1 --2- 172.21.15.1:0/1346469904 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e24c68c820 0x55e24c697510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:41.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.509+0000 7f56afe00700 1 -- 172.21.15.1:0/1346469904 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e24c697ff0 msgr2=0x55e24c6949d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:41.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.509+0000 7f56afe00700 1 --2- 172.21.15.1:0/1346469904 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e24c697ff0 0x55e24c6949d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.509+0000 7f56afe00700 1 -- 172.21.15.1:0/1346469904 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e24c68c820 msgr2=0x55e24c697510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:41.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.509+0000 7f56afe00700 1 --2- 172.21.15.1:0/1346469904 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e24c68c820 0x55e24c697510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.509+0000 7f56afe00700 1 -- 172.21.15.1:0/1346469904 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e24c403b60 con 0x55e24c68d140 2024-02-02T15:34:41.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.509+0000 7f56b7e00700 1 --2- 172.21.15.1:0/1346469904 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e24c68c820 0x55e24c697510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:34:41.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.509+0000 7f56afe00700 1 --2- 172.21.15.1:0/1346469904 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e24c68d140 0x55e24c694490 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f56a0009d20 tx=0x7f56a0005de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24bca800a8817aa0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:41.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.509+0000 7f56ae000700 1 -- 172.21.15.1:0/1346469904 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a0004f80 con 0x55e24c68d140 2024-02-02T15:34:41.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.509+0000 7f56ae000700 1 -- 172.21.15.1:0/1346469904 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f56a0004510 con 0x55e24c68d140 2024-02-02T15:34:41.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.509+0000 7f56ae000700 1 -- 172.21.15.1:0/1346469904 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a0005610 con 0x55e24c68d140 2024-02-02T15:34:41.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.510+0000 7f56d2bd4b80 1 -- 172.21.15.1:0/1346469904 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e24c68d140 msgr2=0x55e24c694490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:41.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.510+0000 7f56d2bd4b80 1 --2- 172.21.15.1:0/1346469904 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e24c68d140 0x55e24c694490 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f56a0009d20 tx=0x7f56a0005de0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.510+0000 7f56d2bd4b80 1 -- 172.21.15.1:0/1346469904 shutdown_connections 2024-02-02T15:34:41.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.510+0000 7f56d2bd4b80 1 --2- 172.21.15.1:0/1346469904 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e24c697ff0 0x55e24c6949d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.510+0000 7f56d2bd4b80 1 --2- 172.21.15.1:0/1346469904 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e24c68c820 0x55e24c697510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.510+0000 7f56d2bd4b80 1 --2- 172.21.15.1:0/1346469904 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e24c68d140 0x55e24c694490 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.510+0000 7f56d2bd4b80 1 -- 172.21.15.1:0/1346469904 >> 172.21.15.1:0/1346469904 conn(0x55e24c5e3db0 msgr2=0x55e24c6a3d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:41.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.511+0000 7f56d2bd4b80 1 -- 172.21.15.1:0/1346469904 shutdown_connections 2024-02-02T15:34:41.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.511+0000 7f56d2bd4b80 1 -- 172.21.15.1:0/1346469904 wait complete. 2024-02-02T15:34:41.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.511+0000 7f56d2bd4b80 1 -- start start 2024-02-02T15:34:41.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.512+0000 7f56d2bd4b80 1 Processor -- start 2024-02-02T15:34:41.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.513+0000 7f56d2bd4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e24c68c820 0x55e24c69e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:41.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.513+0000 7f56d2bd4b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e24c68d140 0x55e24c69cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:41.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.513+0000 7f56d2bd4b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e24c697ff0 0x55e24c69d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:41.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.513+0000 7f56d2bd4b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e24c69d8f0 con 0x55e24c697ff0 2024-02-02T15:34:41.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.513+0000 7f56d2bd4b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e24c69da60 con 0x55e24c68d140 2024-02-02T15:34:41.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.513+0000 7f56d2bd4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e24c6f94e0 con 0x55e24c68c820 2024-02-02T15:34:41.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.513+0000 7f56d2bd4b80 10 client.0 _ll_register_callbacks cb 0x55e24c573290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:34:41.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.514+0000 7f56afe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e24c68d140 0x55e24c69cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:41.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.514+0000 7f56b7e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e24c68c820 0x55e24c69e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:41.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.514+0000 7f56afe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e24c68d140 0x55e24c69cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:58932/0 (socket says 172.21.15.1:58932) 2024-02-02T15:34:41.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.514+0000 7f56afe00700 1 -- 172.21.15.1:0/4042058450 learned_addr learned my addr 172.21.15.1:0/4042058450 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:34:41.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.514+0000 7f56bd000700 1 --2- 172.21.15.1:0/4042058450 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e24c697ff0 0x55e24c69d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:41.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.514+0000 7f56afe00700 1 -- 172.21.15.1:0/4042058450 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e24c68c820 msgr2=0x55e24c69e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:41.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.514+0000 7f56afe00700 1 --2- 172.21.15.1:0/4042058450 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e24c68c820 0x55e24c69e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.514+0000 7f56afe00700 1 -- 172.21.15.1:0/4042058450 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e24c697ff0 msgr2=0x55e24c69d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:41.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.514+0000 7f56afe00700 1 --2- 172.21.15.1:0/4042058450 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e24c697ff0 0x55e24c69d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.514+0000 7f56afe00700 1 -- 172.21.15.1:0/4042058450 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56a001d040 con 0x55e24c68d140 2024-02-02T15:34:41.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.515+0000 7f56b7e00700 1 --2- 172.21.15.1:0/4042058450 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e24c68c820 0x55e24c69e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:34:41.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.515+0000 7f56bd000700 1 --2- 172.21.15.1:0/4042058450 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e24c697ff0 0x55e24c69d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:34:41.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.515+0000 7f56a6000700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:34:41.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.515+0000 7f56afe00700 1 --2- 172.21.15.1:0/4042058450 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e24c68d140 0x55e24c69cca0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f56a0009d20 tx=0x7f56a0011b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc88c4897c1dfad3 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:41.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.515+0000 7f56a6000700 1 -- 172.21.15.1:0/4042058450 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a0004a90 con 0x55e24c68d140 2024-02-02T15:34:41.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.516+0000 7f56a6000700 1 -- 172.21.15.1:0/4042058450 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f56a0004050 con 0x55e24c68d140 2024-02-02T15:34:41.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.516+0000 7f56d2bd4b80 1 -- 172.21.15.1:0/4042058450 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f56a0004cf0 con 0x55e24c68d140 2024-02-02T15:34:41.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.518+0000 7f56a6000700 1 -- 172.21.15.1:0/4042058450 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f56a0004210 con 0x55e24c68d140 2024-02-02T15:34:41.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.518+0000 7f56d2bd4b80 10 client.27686 fetch_fsmap learned FSMap version 42 2024-02-02T15:34:41.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.518+0000 7f56d2bd4b80 1 -- 172.21.15.1:0/4042058450 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55e24c59bea0 con 0x55e24c68d140 2024-02-02T15:34:41.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.519+0000 7f56a6000700 1 -- 172.21.15.1:0/4042058450 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f56a003f030 con 0x55e24c68d140 2024-02-02T15:34:41.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.519+0000 7f56a6000700 1 -- 172.21.15.1:0/4042058450 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 42) v1 ==== 7510+0+0 (secure 0 0 0) 0x7f56a0031070 con 0x55e24c68d140 2024-02-02T15:34:41.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.519+0000 7f56d2bd4b80 10 client.27686 fetch_fsmap finished waiting for FSMap version 42 2024-02-02T15:34:41.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.519+0000 7f56d2bd4b80 10 client.27686 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-02-02T15:34:41.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.519+0000 7f56d2bd4b80 20 client.27686 populate_metadata read hostname 'smithi001' 2024-02-02T15:34:41.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.519+0000 7f56d2bd4b80 1 --2- 172.21.15.1:0/4042058450 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55e24c6fa780 0x55e24c6fcba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:41.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.519+0000 7f56d2bd4b80 4 client.27686 mds_command: new command op to 24518 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:34:41.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.519+0000 7f56d2bd4b80 1 -- 172.21.15.1:0/4042058450 --> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e24c594f50 con 0x55e24c6fa780 2024-02-02T15:34:41.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.520+0000 7f56b7e00700 1 --2- 172.21.15.1:0/4042058450 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55e24c6fa780 0x55e24c6fcba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:41.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.521+0000 7f56b7e00700 1 --2- 172.21.15.1:0/4042058450 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55e24c6fa780 0x55e24c6fcba0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8d569cfb2d602e13 server_cookie=2c5c31837ffd4b50 in_seq=0 out_seq=0 2024-02-02T15:34:41.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.521+0000 7f56a6000700 10 client.27686 ms_handle_connect on v2:172.21.15.71:6836/628016199 2024-02-02T15:34:41.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.521+0000 7f56a6000700 1 -- 172.21.15.1:0/4042058450 <== mds.2 v2:172.21.15.71:6836/628016199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e24c594f50 con 0x55e24c6fa780 2024-02-02T15:34:41.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.521+0000 7f56a6000700 10 client.27686 handle_command_reply: tid=0 2024-02-02T15:34:41.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.521+0000 7f56d2bd4b80 1 client.27686 shutdown 2024-02-02T15:34:41.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.522+0000 7f56d2bd4b80 1 -- 172.21.15.1:0/4042058450 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e24c68d140 msgr2=0x55e24c69cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:41.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.522+0000 7f56d2bd4b80 1 --2- 172.21.15.1:0/4042058450 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e24c68d140 0x55e24c69cca0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f56a0009d20 tx=0x7f56a0011b60 comp rx=0 tx=0).stop 2024-02-02T15:34:41.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.523+0000 7f56d2bd4b80 1 -- 172.21.15.1:0/4042058450 shutdown_connections 2024-02-02T15:34:41.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.523+0000 7f56d2bd4b80 1 --2- 172.21.15.1:0/4042058450 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e24c697ff0 0x55e24c69d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.523+0000 7f56d2bd4b80 1 --2- 172.21.15.1:0/4042058450 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55e24c6fa780 0x55e24c6fcba0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.523+0000 7f56d2bd4b80 1 --2- 172.21.15.1:0/4042058450 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e24c68c820 0x55e24c69e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.523+0000 7f56d2bd4b80 1 --2- 172.21.15.1:0/4042058450 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e24c68d140 0x55e24c69cca0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.523+0000 7f56d2bd4b80 1 -- 172.21.15.1:0/4042058450 >> 172.21.15.1:0/4042058450 conn(0x55e24c5e3db0 msgr2=0x55e24c694460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:41.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.523+0000 7f56a6000700 0 client.27686 ms_handle_reset on v2:172.21.15.71:6836/628016199 2024-02-02T15:34:41.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.524+0000 7f56d2bd4b80 1 -- 172.21.15.1:0/4042058450 shutdown_connections 2024-02-02T15:34:41.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.524+0000 7f56d2bd4b80 1 -- 172.21.15.1:0/4042058450 wait complete. 2024-02-02T15:34:41.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.525+0000 7f56d2bd4b80 20 client.27686 trim_cache size 0 max 16384 2024-02-02T15:34:41.526 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:34:41.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.526+0000 7f56bf000700 1 -- 172.21.15.1:0/752926653 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f56980a0710 msgr2=0x7f56980a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:41.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.526+0000 7f56bf000700 1 --2- 172.21.15.1:0/752926653 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f56980a0710 0x7f56980a2bd0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f56a8006010 tx=0x7f56a800b260 comp rx=0 tx=0).stop 2024-02-02T15:34:41.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.526+0000 7f56bf000700 1 -- 172.21.15.1:0/752926653 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f56b8073ac0 msgr2=0x7f56b81a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:41.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.526+0000 7f56bf000700 1 --2- 172.21.15.1:0/752926653 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f56b8073ac0 0x7f56b81a50c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f56b000efb0 tx=0x7f56b000c3a0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.527+0000 7f56bf000700 1 -- 172.21.15.1:0/752926653 shutdown_connections 2024-02-02T15:34:41.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.527+0000 7f56bf000700 1 --2- 172.21.15.1:0/752926653 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f56980a0710 0x7f56980a2bd0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.527+0000 7f56bf000700 1 --2- 172.21.15.1:0/752926653 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f56b8073ac0 0x7f56b81a50c0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.527+0000 7f56bf000700 1 --2- 172.21.15.1:0/752926653 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f56b8101010 0x7f56b8073040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.527+0000 7f56bf000700 1 --2- 172.21.15.1:0/752926653 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f56b8106130 0x7f56b8073580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.527+0000 7f56bf000700 1 -- 172.21.15.1:0/752926653 >> 172.21.15.1:0/752926653 conn(0x7f56b80fc8b0 msgr2=0x7f56b80fd390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:41.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.527+0000 7f56bf000700 1 -- 172.21.15.1:0/752926653 shutdown_connections 2024-02-02T15:34:41.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.527+0000 7f56bf000700 1 -- 172.21.15.1:0/752926653 wait complete. 2024-02-02T15:34:41.543 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:41 smithi001 ceph-mon[35470]: pgmap v235: 129 pgs: 129 active+clean; 303 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 1.0 MiB/s wr, 100 op/s 2024-02-02T15:34:41.543 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:41 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/2785382803' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:34:41.545 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-02T15:34:41.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.780+0000 7ff8fca00700 1 -- 172.21.15.1:0/96787781 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8f40730e0 msgr2=0x7ff8f40734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:41.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.780+0000 7ff8fca00700 1 --2- 172.21.15.1:0/96787781 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8f40730e0 0x7ff8f40734c0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4009990 tx=0x7ff8e4009ca0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.781+0000 7ff8fca00700 1 -- 172.21.15.1:0/96787781 shutdown_connections 2024-02-02T15:34:41.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.781+0000 7ff8fca00700 1 --2- 172.21.15.1:0/96787781 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8f4111d90 0x7ff8f4114180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.781+0000 7ff8fca00700 1 --2- 172.21.15.1:0/96787781 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8f40730e0 0x7ff8f40734c0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.781+0000 7ff8fca00700 1 --2- 172.21.15.1:0/96787781 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff8f4073a00 0x7ff8f4111850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.781+0000 7ff8fca00700 1 -- 172.21.15.1:0/96787781 >> 172.21.15.1:0/96787781 conn(0x7ff8f40fc810 msgr2=0x7ff8f40fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:41.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.781+0000 7ff8fca00700 1 -- 172.21.15.1:0/96787781 shutdown_connections 2024-02-02T15:34:41.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.781+0000 7ff8fca00700 1 -- 172.21.15.1:0/96787781 wait complete. 2024-02-02T15:34:41.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.782+0000 7ff8fca00700 1 Processor -- start 2024-02-02T15:34:41.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.782+0000 7ff8fca00700 1 -- start start 2024-02-02T15:34:41.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.782+0000 7ff8fca00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8f40730e0 0x7ff8f419d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:41.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.782+0000 7ff8fca00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff8f4073a00 0x7ff8f419daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:41.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.782+0000 7ff8fca00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8f4111d90 0x7ff8f41a1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:41.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.782+0000 7ff8fca00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f41a2540 con 0x7ff8f4073a00 2024-02-02T15:34:41.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.782+0000 7ff8fca00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f41a26b0 con 0x7ff8f40730e0 2024-02-02T15:34:41.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.782+0000 7ff8fca00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f41a2870 con 0x7ff8f4111d90 2024-02-02T15:34:41.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.783+0000 7ff8f9600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff8f4073a00 0x7ff8f419daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:41.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.783+0000 7ff8f9600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff8f4073a00 0x7ff8f419daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45438/0 (socket says 172.21.15.1:45438) 2024-02-02T15:34:41.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.783+0000 7ff8f9600700 1 -- 172.21.15.1:0/2662627875 learned_addr learned my addr 172.21.15.1:0/2662627875 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:34:41.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.783+0000 7ff8fa000700 1 --2- 172.21.15.1:0/2662627875 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8f40730e0 0x7ff8f419d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:41.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.783+0000 7ff8faa00700 1 --2- 172.21.15.1:0/2662627875 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8f4111d90 0x7ff8f41a1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:41.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.783+0000 7ff8fa000700 1 -- 172.21.15.1:0/2662627875 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8f4111d90 msgr2=0x7ff8f41a1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:41.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.783+0000 7ff8fa000700 1 --2- 172.21.15.1:0/2662627875 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8f4111d90 0x7ff8f41a1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.783+0000 7ff8fa000700 1 -- 172.21.15.1:0/2662627875 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff8f4073a00 msgr2=0x7ff8f419daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:41.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.783+0000 7ff8fa000700 1 --2- 172.21.15.1:0/2662627875 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff8f4073a00 0x7ff8f419daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.783+0000 7ff8fa000700 1 -- 172.21.15.1:0/2662627875 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8e40095f0 con 0x7ff8f40730e0 2024-02-02T15:34:41.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.783+0000 7ff8fa000700 1 --2- 172.21.15.1:0/2662627875 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8f40730e0 0x7ff8f419d560 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4000c00 tx=0x7ff8e4005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:41.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.783+0000 7ff8f2a00700 1 -- 172.21.15.1:0/2662627875 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e401d070 con 0x7ff8f40730e0 2024-02-02T15:34:41.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.784+0000 7ff8f2a00700 1 -- 172.21.15.1:0/2662627875 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff8e4021410 con 0x7ff8f40730e0 2024-02-02T15:34:41.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.784+0000 7ff8fca00700 1 -- 172.21.15.1:0/2662627875 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8f41a2af0 con 0x7ff8f40730e0 2024-02-02T15:34:41.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.784+0000 7ff8f2a00700 1 -- 172.21.15.1:0/2662627875 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e4017650 con 0x7ff8f40730e0 2024-02-02T15:34:41.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.784+0000 7ff8fca00700 1 -- 172.21.15.1:0/2662627875 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8f41aa390 con 0x7ff8f40730e0 2024-02-02T15:34:41.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.787+0000 7ff8f2a00700 1 -- 172.21.15.1:0/2662627875 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7ff8e4021580 con 0x7ff8f40730e0 2024-02-02T15:34:41.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.788+0000 7ff910435b80 1 Processor -- start 2024-02-02T15:34:41.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.788+0000 7ff8f2a00700 1 --2- 172.21.15.1:0/2662627875 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff8d40a0710 0x7ff8d40a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:41.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.788+0000 7ff910435b80 1 -- start start 2024-02-02T15:34:41.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.788+0000 7ff8f2a00700 1 -- 172.21.15.1:0/2662627875 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9453+0+0 (secure 0 0 0) 0x7ff8e40cd470 con 0x7ff8f40730e0 2024-02-02T15:34:41.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.788+0000 7ff910435b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1f4222010 0x55a1f42223f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:41.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.788+0000 7ff910435b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a1f4222930 0x55a1f422d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:41.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.788+0000 7ff8faa00700 1 --2- 172.21.15.1:0/2662627875 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff8d40a0710 0x7ff8d40a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:41.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.788+0000 7ff910435b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1f422d7e0 0x55a1f422fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:41.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.788+0000 7ff910435b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1f41964c0 con 0x55a1f4222010 2024-02-02T15:34:41.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.788+0000 7ff910435b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1f417aaa0 con 0x55a1f422d7e0 2024-02-02T15:34:41.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.788+0000 7ff910435b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1f4178460 con 0x55a1f4222930 2024-02-02T15:34:41.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.789+0000 7ff8fa000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1f4222010 0x55a1f42223f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:41.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.789+0000 7ff8f9600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a1f4222930 0x55a1f422d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:41.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.789+0000 7ff8fa000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1f4222010 0x55a1f42223f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45454/0 (socket says 172.21.15.1:45454) 2024-02-02T15:34:41.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.789+0000 7ff8fa000700 1 -- 172.21.15.1:0/3327109457 learned_addr learned my addr 172.21.15.1:0/3327109457 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:34:41.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.789+0000 7ff8faa00700 1 --2- 172.21.15.1:0/3327109457 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1f422d7e0 0x55a1f422fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:41.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.789+0000 7ff8f9600700 1 -- 172.21.15.1:0/3327109457 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1f422d7e0 msgr2=0x55a1f422fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:41.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.789+0000 7ff8f9600700 1 --2- 172.21.15.1:0/3327109457 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1f422d7e0 0x55a1f422fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.789+0000 7ff8f9600700 1 -- 172.21.15.1:0/3327109457 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1f4222010 msgr2=0x55a1f42223f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:41.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.789+0000 7ff8f9600700 1 --2- 172.21.15.1:0/3327109457 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1f4222010 0x55a1f42223f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.789+0000 7ff8f9600700 1 -- 172.21.15.1:0/3327109457 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a1f3ff53b0 con 0x55a1f4222930 2024-02-02T15:34:41.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.789+0000 7ff8fa000700 1 --2- 172.21.15.1:0/3327109457 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1f4222010 0x55a1f42223f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:34:41.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.789+0000 7ff8faa00700 1 --2- 172.21.15.1:0/2662627875 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff8d40a0710 0x7ff8d40a2bd0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff8f406a330 tx=0x7ff8ec004040 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:41.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.789+0000 7ff8f9600700 1 --2- 172.21.15.1:0/3327109457 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a1f4222930 0x55a1f422d2a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc009d10 tx=0x7ff8dc011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4b789f408a872ae server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:41.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.790+0000 7ff8eb000700 1 -- 172.21.15.1:0/3327109457 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc004250 con 0x55a1f4222930 2024-02-02T15:34:41.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.790+0000 7ff8eb000700 1 -- 172.21.15.1:0/3327109457 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff8dc004cf0 con 0x55a1f4222930 2024-02-02T15:34:41.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.790+0000 7ff8eb000700 1 -- 172.21.15.1:0/3327109457 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc005540 con 0x55a1f4222930 2024-02-02T15:34:41.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.790+0000 7ff910435b80 1 -- 172.21.15.1:0/3327109457 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a1f4222930 msgr2=0x55a1f422d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:41.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.790+0000 7ff910435b80 1 --2- 172.21.15.1:0/3327109457 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a1f4222930 0x55a1f422d2a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc009d10 tx=0x7ff8dc011720 comp rx=0 tx=0).stop 2024-02-02T15:34:41.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.790+0000 7ff910435b80 1 -- 172.21.15.1:0/3327109457 shutdown_connections 2024-02-02T15:34:41.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.790+0000 7ff910435b80 1 --2- 172.21.15.1:0/3327109457 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1f422d7e0 0x55a1f422fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.790+0000 7ff910435b80 1 --2- 172.21.15.1:0/3327109457 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1f4222010 0x55a1f42223f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.790+0000 7ff910435b80 1 --2- 172.21.15.1:0/3327109457 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a1f4222930 0x55a1f422d2a0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.790+0000 7ff910435b80 1 -- 172.21.15.1:0/3327109457 >> 172.21.15.1:0/3327109457 conn(0x55a1f4181db0 msgr2=0x55a1f421e3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:41.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.790+0000 7ff910435b80 1 -- 172.21.15.1:0/3327109457 shutdown_connections 2024-02-02T15:34:41.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.791+0000 7ff910435b80 1 -- 172.21.15.1:0/3327109457 wait complete. 2024-02-02T15:34:41.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.791+0000 7ff910435b80 1 -- start start 2024-02-02T15:34:41.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.791+0000 7ff910435b80 1 Processor -- start 2024-02-02T15:34:41.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.792+0000 7ff910435b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1f4222010 0x55a1f422d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:41.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.792+0000 7ff910435b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a1f4222930 0x55a1f422a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:41.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.792+0000 7ff910435b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1f422d7e0 0x55a1f422a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:41.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.792+0000 7ff910435b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1f4173d50 con 0x55a1f422d7e0 2024-02-02T15:34:41.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.792+0000 7ff910435b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1f410fd80 con 0x55a1f4222010 2024-02-02T15:34:41.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.792+0000 7ff910435b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1f410fa20 con 0x55a1f4222930 2024-02-02T15:34:41.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.792+0000 7ff910435b80 10 client.0 _ll_register_callbacks cb 0x55a1f4111290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:34:41.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.792+0000 7ff8fa000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1f4222010 0x55a1f422d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:41.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.792+0000 7ff8fa000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1f4222010 0x55a1f422d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:58970/0 (socket says 172.21.15.1:58970) 2024-02-02T15:34:41.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.792+0000 7ff8fa000700 1 -- 172.21.15.1:0/407559053 learned_addr learned my addr 172.21.15.1:0/407559053 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:34:41.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.792+0000 7ff8f9600700 1 --2- 172.21.15.1:0/407559053 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a1f4222930 0x55a1f422a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:41.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.792+0000 7ff8faa00700 1 --2- 172.21.15.1:0/407559053 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1f422d7e0 0x55a1f422a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:41.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.792+0000 7ff8fa000700 1 -- 172.21.15.1:0/407559053 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a1f4222930 msgr2=0x55a1f422a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:41.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.792+0000 7ff8fa000700 1 --2- 172.21.15.1:0/407559053 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a1f4222930 0x55a1f422a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.792+0000 7ff8fa000700 1 -- 172.21.15.1:0/407559053 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1f422d7e0 msgr2=0x55a1f422a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:41.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.792+0000 7ff8fa000700 1 --2- 172.21.15.1:0/407559053 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1f422d7e0 0x55a1f422a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.792+0000 7ff8fa000700 1 -- 172.21.15.1:0/407559053 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8f41aa390 con 0x55a1f4222010 2024-02-02T15:34:41.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.792+0000 7ff8faa00700 1 --2- 172.21.15.1:0/407559053 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1f422d7e0 0x55a1f422a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:34:41.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.793+0000 7ff8fa000700 1 --2- 172.21.15.1:0/407559053 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1f4222010 0x55a1f422d1b0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ff8e400b2e0 tx=0x7ff8e4098bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6036ff6f749d4e05 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:41.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.793+0000 7ff8e3400700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:34:41.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.793+0000 7ff8e3400700 1 -- 172.21.15.1:0/407559053 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e409a620 con 0x55a1f4222010 2024-02-02T15:34:41.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.793+0000 7ff8e3400700 1 -- 172.21.15.1:0/407559053 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9453+0+0 (secure 0 0 0) 0x7ff8e40a0040 con 0x55a1f4222010 2024-02-02T15:34:41.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.793+0000 7ff910435b80 1 -- 172.21.15.1:0/407559053 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a1f42160f0 con 0x55a1f4222010 2024-02-02T15:34:41.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.793+0000 7ff8e3400700 1 -- 172.21.15.1:0/407559053 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff8e409e450 con 0x55a1f4222010 2024-02-02T15:34:41.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.793+0000 7ff910435b80 10 client.27698 fetch_fsmap learned FSMap version 42 2024-02-02T15:34:41.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.793+0000 7ff910435b80 1 -- 172.21.15.1:0/407559053 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7ff8dc009970 con 0x55a1f4222010 2024-02-02T15:34:41.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.793+0000 7ff8e3400700 1 -- 172.21.15.1:0/407559053 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a1f42160f0 con 0x55a1f4222010 2024-02-02T15:34:41.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.793+0000 7ff8e3400700 1 -- 172.21.15.1:0/407559053 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 42) v1 ==== 7510+0+0 (secure 0 0 0) 0x7ff8e409a880 con 0x55a1f4222010 2024-02-02T15:34:41.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.793+0000 7ff910435b80 10 client.27698 fetch_fsmap finished waiting for FSMap version 42 2024-02-02T15:34:41.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.793+0000 7ff910435b80 10 client.27698 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-02T15:34:41.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.793+0000 7ff910435b80 20 client.27698 populate_metadata read hostname 'smithi001' 2024-02-02T15:34:41.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.794+0000 7ff910435b80 1 --2- 172.21.15.1:0/407559053 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55a1f423cf20 0x55a1f423f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:41.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.794+0000 7ff910435b80 4 client.27698 mds_command: new command op to 24529 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:34:41.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.794+0000 7ff910435b80 1 -- 172.21.15.1:0/407559053 --> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a1f416cdd0 con 0x55a1f423cf20 2024-02-02T15:34:41.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.794+0000 7ff8f9600700 1 --2- 172.21.15.1:0/407559053 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55a1f423cf20 0x55a1f423f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:41.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.794+0000 7ff8f9600700 1 --2- 172.21.15.1:0/407559053 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55a1f423cf20 0x55a1f423f340 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4c9ea9f956d923d0 server_cookie=8d6ef508297848ee in_seq=0 out_seq=0 2024-02-02T15:34:41.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.794+0000 7ff8e3400700 10 client.27698 ms_handle_connect on v2:172.21.15.142:6838/321675421 2024-02-02T15:34:41.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.795+0000 7ff8e3400700 1 -- 172.21.15.1:0/407559053 <== mds.1 v2:172.21.15.142:6838/321675421 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55a1f416cdd0 con 0x55a1f423cf20 2024-02-02T15:34:41.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.795+0000 7ff8e3400700 10 client.27698 handle_command_reply: tid=0 2024-02-02T15:34:41.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.795+0000 7ff910435b80 1 client.27698 shutdown 2024-02-02T15:34:41.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.795+0000 7ff910435b80 1 -- 172.21.15.1:0/407559053 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1f4222010 msgr2=0x55a1f422d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:41.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.795+0000 7ff910435b80 1 --2- 172.21.15.1:0/407559053 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1f4222010 0x55a1f422d1b0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ff8e400b2e0 tx=0x7ff8e4098bd0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.795+0000 7ff910435b80 1 -- 172.21.15.1:0/407559053 shutdown_connections 2024-02-02T15:34:41.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.795+0000 7ff910435b80 1 --2- 172.21.15.1:0/407559053 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55a1f423cf20 0x55a1f423f340 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.795+0000 7ff910435b80 1 --2- 172.21.15.1:0/407559053 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1f422d7e0 0x55a1f422a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.795+0000 7ff910435b80 1 --2- 172.21.15.1:0/407559053 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1f4222010 0x55a1f422d1b0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.795+0000 7ff8e3400700 0 client.27698 ms_handle_reset on v2:172.21.15.142:6838/321675421 2024-02-02T15:34:41.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.795+0000 7ff910435b80 1 --2- 172.21.15.1:0/407559053 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a1f4222930 0x55a1f422a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.796+0000 7ff910435b80 1 -- 172.21.15.1:0/407559053 >> 172.21.15.1:0/407559053 conn(0x55a1f4181db0 msgr2=0x55a1f421dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:41.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.796+0000 7ff910435b80 1 -- 172.21.15.1:0/407559053 shutdown_connections 2024-02-02T15:34:41.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.796+0000 7ff910435b80 1 -- 172.21.15.1:0/407559053 wait complete. 2024-02-02T15:34:41.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.796+0000 7ff910435b80 20 client.27698 trim_cache size 0 max 16384 2024-02-02T15:34:41.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.809+0000 7ff910435b80 1 Processor -- start 2024-02-02T15:34:41.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.809+0000 7ff910435b80 1 -- start start 2024-02-02T15:34:41.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.810+0000 7ff910435b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1f4222010 0x55a1f4234330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:41.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.810+0000 7ff910435b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1f4222930 0x55a1f422fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:41.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.810+0000 7ff910435b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a1f422d7e0 0x55a1f422a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:41.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.810+0000 7ff910435b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1f3ff76a0 con 0x55a1f4222010 2024-02-02T15:34:41.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.810+0000 7ff910435b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1f4226210 con 0x55a1f4222930 2024-02-02T15:34:41.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.810+0000 7ff910435b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1f4225000 con 0x55a1f422d7e0 2024-02-02T15:34:41.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.810+0000 7ff8fa000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1f4222010 0x55a1f4234330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:41.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.810+0000 7ff8faa00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a1f422d7e0 0x55a1f422a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:41.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.810+0000 7ff8fa000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1f4222010 0x55a1f4234330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45462/0 (socket says 172.21.15.1:45462) 2024-02-02T15:34:41.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.810+0000 7ff8fa000700 1 -- 172.21.15.1:0/1669756615 learned_addr learned my addr 172.21.15.1:0/1669756615 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:34:41.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.810+0000 7ff8f9600700 1 --2- 172.21.15.1:0/1669756615 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1f4222930 0x55a1f422fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:41.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.810+0000 7ff8fa000700 1 -- 172.21.15.1:0/1669756615 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a1f422d7e0 msgr2=0x55a1f422a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:41.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.810+0000 7ff8fa000700 1 --2- 172.21.15.1:0/1669756615 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a1f422d7e0 0x55a1f422a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.810+0000 7ff8fa000700 1 -- 172.21.15.1:0/1669756615 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1f4222930 msgr2=0x55a1f422fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:41.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.810+0000 7ff8fa000700 1 --2- 172.21.15.1:0/1669756615 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1f4222930 0x55a1f422fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.810+0000 7ff8fa000700 1 -- 172.21.15.1:0/1669756615 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a1f3fa1b60 con 0x55a1f4222010 2024-02-02T15:34:41.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.811+0000 7ff8fa000700 1 --2- 172.21.15.1:0/1669756615 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1f4222010 0x55a1f4234330 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7ff8e40a20d0 tx=0x7ff8e409a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88650da12331aafd server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:41.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.811+0000 7ff8eba00700 1 -- 172.21.15.1:0/1669756615 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e4099c00 con 0x55a1f4222010 2024-02-02T15:34:41.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.811+0000 7ff8eba00700 1 -- 172.21.15.1:0/1669756615 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff8e4099f80 con 0x55a1f4222010 2024-02-02T15:34:41.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.811+0000 7ff8eba00700 1 -- 172.21.15.1:0/1669756615 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e409e450 con 0x55a1f4222010 2024-02-02T15:34:41.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.811+0000 7ff910435b80 1 -- 172.21.15.1:0/1669756615 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1f4222010 msgr2=0x55a1f4234330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:41.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.811+0000 7ff910435b80 1 --2- 172.21.15.1:0/1669756615 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1f4222010 0x55a1f4234330 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7ff8e40a20d0 tx=0x7ff8e409a290 comp rx=0 tx=0).stop 2024-02-02T15:34:41.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.811+0000 7ff910435b80 1 -- 172.21.15.1:0/1669756615 shutdown_connections 2024-02-02T15:34:41.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.811+0000 7ff910435b80 1 --2- 172.21.15.1:0/1669756615 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a1f422d7e0 0x55a1f422a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.811+0000 7ff910435b80 1 --2- 172.21.15.1:0/1669756615 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1f4222010 0x55a1f4234330 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.811+0000 7ff910435b80 1 --2- 172.21.15.1:0/1669756615 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1f4222930 0x55a1f422fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.811+0000 7ff910435b80 1 -- 172.21.15.1:0/1669756615 >> 172.21.15.1:0/1669756615 conn(0x55a1f4181db0 msgr2=0x55a1f4219e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:41.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.812+0000 7ff910435b80 1 -- 172.21.15.1:0/1669756615 shutdown_connections 2024-02-02T15:34:41.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.812+0000 7ff910435b80 1 -- 172.21.15.1:0/1669756615 wait complete. 2024-02-02T15:34:41.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.812+0000 7ff910435b80 1 -- start start 2024-02-02T15:34:41.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.812+0000 7ff910435b80 1 Processor -- start 2024-02-02T15:34:41.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.813+0000 7ff910435b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1f4222010 0x55a1f422fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:41.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.813+0000 7ff910435b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1f4222930 0x55a1f4230300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:41.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.813+0000 7ff910435b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a1f422d7e0 0x55a1f42308a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:41.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.813+0000 7ff910435b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1f4230fb0 con 0x55a1f4222010 2024-02-02T15:34:41.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.813+0000 7ff910435b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1f4231120 con 0x55a1f4222930 2024-02-02T15:34:41.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.813+0000 7ff910435b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1f428da30 con 0x55a1f422d7e0 2024-02-02T15:34:41.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.813+0000 7ff910435b80 10 client.0 _ll_register_callbacks cb 0x55a1f4111290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:34:41.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.813+0000 7ff8f9600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1f4222930 0x55a1f4230300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:41.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.813+0000 7ff8fa000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1f4222010 0x55a1f422fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:41.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.813+0000 7ff8f9600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1f4222930 0x55a1f4230300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:58986/0 (socket says 172.21.15.1:58986) 2024-02-02T15:34:41.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.813+0000 7ff8f9600700 1 -- 172.21.15.1:0/3287220052 learned_addr learned my addr 172.21.15.1:0/3287220052 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:34:41.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.813+0000 7ff8faa00700 1 --2- 172.21.15.1:0/3287220052 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a1f422d7e0 0x55a1f42308a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:41.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.813+0000 7ff8f9600700 1 -- 172.21.15.1:0/3287220052 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a1f422d7e0 msgr2=0x55a1f42308a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:41.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.813+0000 7ff8f9600700 1 --2- 172.21.15.1:0/3287220052 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a1f422d7e0 0x55a1f42308a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.813+0000 7ff8f9600700 1 -- 172.21.15.1:0/3287220052 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1f4222010 msgr2=0x55a1f422fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:41.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.813+0000 7ff8f9600700 1 --2- 172.21.15.1:0/3287220052 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1f4222010 0x55a1f422fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.813+0000 7ff8f9600700 1 -- 172.21.15.1:0/3287220052 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8dc009970 con 0x55a1f4222930 2024-02-02T15:34:41.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.813+0000 7ff8faa00700 1 --2- 172.21.15.1:0/3287220052 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a1f422d7e0 0x55a1f42308a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:34:41.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.813+0000 7ff8fa000700 1 --2- 172.21.15.1:0/3287220052 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1f4222010 0x55a1f422fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:34:41.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.814+0000 7ff8f9600700 1 --2- 172.21.15.1:0/3287220052 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1f4222930 0x55a1f4230300 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc009d10 tx=0x7ff8dc013040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79825505513028f9 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:41.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.814+0000 7ff8e3400700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:34:41.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.814+0000 7ff8e3400700 1 -- 172.21.15.1:0/3287220052 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8dc0042b0 con 0x55a1f4222930 2024-02-02T15:34:41.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.814+0000 7ff8e3400700 1 -- 172.21.15.1:0/3287220052 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9453+0+0 (secure 0 0 0) 0x7ff8dc004690 con 0x55a1f4222930 2024-02-02T15:34:41.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.814+0000 7ff910435b80 1 -- 172.21.15.1:0/3287220052 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff8dc004cf0 con 0x55a1f4222930 2024-02-02T15:34:41.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.814+0000 7ff8e3400700 1 -- 172.21.15.1:0/3287220052 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff8dc004410 con 0x55a1f4222930 2024-02-02T15:34:41.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.814+0000 7ff910435b80 10 client.27704 fetch_fsmap learned FSMap version 42 2024-02-02T15:34:41.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.814+0000 7ff910435b80 1 -- 172.21.15.1:0/3287220052 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55a1f4139ea0 con 0x55a1f4222930 2024-02-02T15:34:41.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.814+0000 7ff8e3400700 1 -- 172.21.15.1:0/3287220052 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff8dc004cf0 con 0x55a1f4222930 2024-02-02T15:34:41.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.814+0000 7ff8e3400700 1 -- 172.21.15.1:0/3287220052 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 42) v1 ==== 7510+0+0 (secure 0 0 0) 0x7ff8dc004e60 con 0x55a1f4222930 2024-02-02T15:34:41.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.814+0000 7ff910435b80 10 client.27704 fetch_fsmap finished waiting for FSMap version 42 2024-02-02T15:34:41.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.814+0000 7ff910435b80 10 client.27704 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-02T15:34:41.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.814+0000 7ff910435b80 20 client.27704 populate_metadata read hostname 'smithi001' 2024-02-02T15:34:41.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.814+0000 7ff910435b80 1 --2- 172.21.15.1:0/3287220052 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55a1f428f1c0 0x55a1f42915e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:41.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.814+0000 7ff910435b80 4 client.27704 mds_command: new command op to 24529 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:34:41.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.814+0000 7ff910435b80 1 -- 172.21.15.1:0/3287220052 --> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a1f4132f50 con 0x55a1f428f1c0 2024-02-02T15:34:41.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.815+0000 7ff8fa000700 1 --2- 172.21.15.1:0/3287220052 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55a1f428f1c0 0x55a1f42915e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:41.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.815+0000 7ff8fa000700 1 --2- 172.21.15.1:0/3287220052 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55a1f428f1c0 0x55a1f42915e0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4a49265b67c73bbf server_cookie=d5c81c213240b1f6 in_seq=0 out_seq=0 2024-02-02T15:34:41.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.815+0000 7ff8e3400700 10 client.27704 ms_handle_connect on v2:172.21.15.142:6838/321675421 2024-02-02T15:34:41.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.815+0000 7ff8e3400700 1 -- 172.21.15.1:0/3287220052 <== mds.1 v2:172.21.15.142:6838/321675421 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a1f4132f50 con 0x55a1f428f1c0 2024-02-02T15:34:41.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.815+0000 7ff8e3400700 10 client.27704 handle_command_reply: tid=0 2024-02-02T15:34:41.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.815+0000 7ff910435b80 1 client.27704 shutdown 2024-02-02T15:34:41.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.816+0000 7ff910435b80 1 -- 172.21.15.1:0/3287220052 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1f4222930 msgr2=0x55a1f4230300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:41.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.816+0000 7ff910435b80 1 --2- 172.21.15.1:0/3287220052 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1f4222930 0x55a1f4230300 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc009d10 tx=0x7ff8dc013040 comp rx=0 tx=0).stop 2024-02-02T15:34:41.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.816+0000 7ff910435b80 1 -- 172.21.15.1:0/3287220052 shutdown_connections 2024-02-02T15:34:41.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.816+0000 7ff910435b80 1 --2- 172.21.15.1:0/3287220052 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55a1f428f1c0 0x55a1f42915e0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.816+0000 7ff910435b80 1 --2- 172.21.15.1:0/3287220052 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a1f422d7e0 0x55a1f42308a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.816+0000 7ff910435b80 1 --2- 172.21.15.1:0/3287220052 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1f4222010 0x55a1f422fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.816+0000 7ff910435b80 1 --2- 172.21.15.1:0/3287220052 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1f4222930 0x55a1f4230300 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.816+0000 7ff8e3400700 0 client.27704 ms_handle_reset on v2:172.21.15.142:6838/321675421 2024-02-02T15:34:41.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.816+0000 7ff910435b80 1 -- 172.21.15.1:0/3287220052 >> 172.21.15.1:0/3287220052 conn(0x55a1f4181db0 msgr2=0x55a1f42394e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:41.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.816+0000 7ff910435b80 1 -- 172.21.15.1:0/3287220052 shutdown_connections 2024-02-02T15:34:41.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.816+0000 7ff910435b80 1 -- 172.21.15.1:0/3287220052 wait complete. 2024-02-02T15:34:41.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.816+0000 7ff910435b80 20 client.27704 trim_cache size 0 max 16384 2024-02-02T15:34:41.821 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:34:41.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.817+0000 7ff8fca00700 1 -- 172.21.15.1:0/2662627875 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff8d40a0710 msgr2=0x7ff8d40a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:41.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.817+0000 7ff8fca00700 1 --2- 172.21.15.1:0/2662627875 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff8d40a0710 0x7ff8d40a2bd0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff8f406a330 tx=0x7ff8ec004040 comp rx=0 tx=0).stop 2024-02-02T15:34:41.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.817+0000 7ff8fca00700 1 -- 172.21.15.1:0/2662627875 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8f40730e0 msgr2=0x7ff8f419d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:41.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.817+0000 7ff8fca00700 1 --2- 172.21.15.1:0/2662627875 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8f40730e0 0x7ff8f419d560 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4000c00 tx=0x7ff8e4005740 comp rx=0 tx=0).stop 2024-02-02T15:34:41.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.817+0000 7ff8fca00700 1 -- 172.21.15.1:0/2662627875 shutdown_connections 2024-02-02T15:34:41.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.817+0000 7ff8fca00700 1 --2- 172.21.15.1:0/2662627875 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff8d40a0710 0x7ff8d40a2bd0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.817+0000 7ff8fca00700 1 --2- 172.21.15.1:0/2662627875 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8f4111d90 0x7ff8f41a1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.817+0000 7ff8fca00700 1 --2- 172.21.15.1:0/2662627875 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff8f40730e0 0x7ff8f419d560 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.817+0000 7ff8fca00700 1 --2- 172.21.15.1:0/2662627875 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff8f4073a00 0x7ff8f419daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:41.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.817+0000 7ff8fca00700 1 -- 172.21.15.1:0/2662627875 >> 172.21.15.1:0/2662627875 conn(0x7ff8f40fc810 msgr2=0x7ff8f40fd480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:41.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.817+0000 7ff8fca00700 1 -- 172.21.15.1:0/2662627875 shutdown_connections 2024-02-02T15:34:41.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:41.817+0000 7ff8fca00700 1 -- 172.21.15.1:0/2662627875 wait complete. 2024-02-02T15:34:41.834 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-02T15:34:41.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:41 smithi071 ceph-mon[44707]: pgmap v235: 129 pgs: 129 active+clean; 303 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 1.0 MiB/s wr, 100 op/s 2024-02-02T15:34:41.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:41 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/2785382803' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:34:41.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:41 smithi142 ceph-mon[45728]: pgmap v235: 129 pgs: 129 active+clean; 303 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 776 KiB/s rd, 1.0 MiB/s wr, 100 op/s 2024-02-02T15:34:41.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:41 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/2785382803' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:34:42.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.036+0000 7f779e200700 1 -- 172.21.15.1:0/3674801699 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7798111d90 msgr2=0x7f7798114180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:42.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.036+0000 7f779e200700 1 --2- 172.21.15.1:0/3674801699 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7798111d90 0x7f7798114180 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f779000b0a0 tx=0x7f779000b3b0 comp rx=0 tx=0).stop 2024-02-02T15:34:42.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.037+0000 7f779e200700 1 -- 172.21.15.1:0/3674801699 shutdown_connections 2024-02-02T15:34:42.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.037+0000 7f779e200700 1 --2- 172.21.15.1:0/3674801699 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7798111d90 0x7f7798114180 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:42.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.037+0000 7f779e200700 1 --2- 172.21.15.1:0/3674801699 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77980730e0 0x7f77980734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:42.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.037+0000 7f779e200700 1 --2- 172.21.15.1:0/3674801699 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7798073a00 0x7f7798111850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:42.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.037+0000 7f779e200700 1 -- 172.21.15.1:0/3674801699 >> 172.21.15.1:0/3674801699 conn(0x7f77980fc810 msgr2=0x7f77980fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:42.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.037+0000 7f779e200700 1 -- 172.21.15.1:0/3674801699 shutdown_connections 2024-02-02T15:34:42.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.037+0000 7f779e200700 1 -- 172.21.15.1:0/3674801699 wait complete. 2024-02-02T15:34:42.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.038+0000 7f779e200700 1 Processor -- start 2024-02-02T15:34:42.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.038+0000 7f779e200700 1 -- start start 2024-02-02T15:34:42.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.038+0000 7f779e200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77980730e0 0x7f779819d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:42.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.038+0000 7f779e200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7798073a00 0x7f779819db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:42.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.038+0000 7f779e200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7798111d90 0x7f77981a1f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:42.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.038+0000 7f779e200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77981a25b0 con 0x7f7798111d90 2024-02-02T15:34:42.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.038+0000 7f779e200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77981a2720 con 0x7f77980730e0 2024-02-02T15:34:42.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.038+0000 7f779e200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77981a28e0 con 0x7f7798073a00 2024-02-02T15:34:42.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.038+0000 7f7797e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7798111d90 0x7f77981a1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:42.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.039+0000 7f7797e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7798111d90 0x7f77981a1f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45498/0 (socket says 172.21.15.1:45498) 2024-02-02T15:34:42.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.039+0000 7f7797e00700 1 -- 172.21.15.1:0/3620797794 learned_addr learned my addr 172.21.15.1:0/3620797794 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:34:42.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.039+0000 7f7796a00700 1 --2- 172.21.15.1:0/3620797794 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7798073a00 0x7f779819db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:42.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.039+0000 7f7797400700 1 --2- 172.21.15.1:0/3620797794 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77980730e0 0x7f779819d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:42.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.039+0000 7f7797e00700 1 -- 172.21.15.1:0/3620797794 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7798073a00 msgr2=0x7f779819db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:42.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.039+0000 7f7797e00700 1 --2- 172.21.15.1:0/3620797794 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7798073a00 0x7f779819db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:42.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.039+0000 7f7797e00700 1 -- 172.21.15.1:0/3620797794 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77980730e0 msgr2=0x7f779819d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:42.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.039+0000 7f7797e00700 1 --2- 172.21.15.1:0/3620797794 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77980730e0 0x7f779819d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:42.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.039+0000 7f7797e00700 1 -- 172.21.15.1:0/3620797794 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7788009520 con 0x7f7798111d90 2024-02-02T15:34:42.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.039+0000 7f7796a00700 1 --2- 172.21.15.1:0/3620797794 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7798073a00 0x7f779819db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:34:42.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.039+0000 7f7797400700 1 --2- 172.21.15.1:0/3620797794 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77980730e0 0x7f779819d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:34:42.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.039+0000 7f7797e00700 1 --2- 172.21.15.1:0/3620797794 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7798111d90 0x7f77981a1f70 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f7790009fd0 tx=0x7f779000bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:42.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.040+0000 7f778fe00700 1 -- 172.21.15.1:0/3620797794 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f779000e040 con 0x7f7798111d90 2024-02-02T15:34:42.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.040+0000 7f778fe00700 1 -- 172.21.15.1:0/3620797794 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f77900094a0 con 0x7f7798111d90 2024-02-02T15:34:42.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.040+0000 7f779e200700 1 -- 172.21.15.1:0/3620797794 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7780009520 con 0x7f7798111d90 2024-02-02T15:34:42.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.040+0000 7f778fe00700 1 -- 172.21.15.1:0/3620797794 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f779001b460 con 0x7f7798111d90 2024-02-02T15:34:42.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.040+0000 7f779e200700 1 -- 172.21.15.1:0/3620797794 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7790009d00 con 0x7f7798111d90 2024-02-02T15:34:42.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.044+0000 7f778fe00700 1 -- 172.21.15.1:0/3620797794 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f779001d470 con 0x7f7798111d90 2024-02-02T15:34:42.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.045+0000 7f77b1c6db80 1 Processor -- start 2024-02-02T15:34:42.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.045+0000 7f77b1c6db80 1 -- start start 2024-02-02T15:34:42.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.046+0000 7f77b1c6db80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c430cc5010 0x55c430cc53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:42.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.046+0000 7f77b1c6db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c430cc5930 0x55c430cd02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:42.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.046+0000 7f77b1c6db80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c430cd07e0 0x55c430cd2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:42.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.046+0000 7f77b1c6db80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c430c394c0 con 0x55c430cd07e0 2024-02-02T15:34:42.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.046+0000 7f77b1c6db80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c430c1daa0 con 0x55c430cc5010 2024-02-02T15:34:42.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.046+0000 7f77b1c6db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c430c1b460 con 0x55c430cc5930 2024-02-02T15:34:42.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.046+0000 7f7796a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c430cd07e0 0x55c430cd2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:42.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.046+0000 7f7796a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c430cc5010 0x55c430cc53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:42.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.046+0000 7f7797400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c430cc5930 0x55c430cd02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:42.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.046+0000 7f7797400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c430cc5930 0x55c430cd02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:60994/0 (socket says 172.21.15.1:60994) 2024-02-02T15:34:42.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.046+0000 7f7797400700 1 -- 172.21.15.1:0/3735698883 learned_addr learned my addr 172.21.15.1:0/3735698883 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:34:42.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.046+0000 7f778fe00700 1 --2- 172.21.15.1:0/3620797794 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f77780a0690 0x7f77780a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:42.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.046+0000 7f778fe00700 1 -- 172.21.15.1:0/3620797794 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f77900cd250 con 0x7f7798111d90 2024-02-02T15:34:42.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.046+0000 7f7797e00700 1 --2- 172.21.15.1:0/3620797794 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f77780a0690 0x7f77780a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:42.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.046+0000 7f7796a00700 1 -- 172.21.15.1:0/3735698883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c430cc5930 msgr2=0x55c430cd02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:42.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.046+0000 7f7796a00700 1 --2- 172.21.15.1:0/3735698883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c430cc5930 0x55c430cd02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:42.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.046+0000 7f7796a00700 1 -- 172.21.15.1:0/3735698883 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c430cd07e0 msgr2=0x55c430cd2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:42.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.046+0000 7f7796a00700 1 --2- 172.21.15.1:0/3735698883 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c430cd07e0 0x55c430cd2bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:42.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.047+0000 7f7796a00700 1 -- 172.21.15.1:0/3735698883 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c430a983b0 con 0x55c430cc5010 2024-02-02T15:34:42.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.047+0000 7f7796a00700 1 --2- 172.21.15.1:0/3735698883 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c430cc5010 0x55c430cc53f0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f7780013d40 tx=0x7f7780009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0b92e1b78227afa server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:42.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.047+0000 7f7787e00700 1 -- 172.21.15.1:0/3735698883 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7780013110 con 0x55c430cc5010 2024-02-02T15:34:42.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.047+0000 7f7787e00700 1 -- 172.21.15.1:0/3735698883 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7780010e80 con 0x55c430cc5010 2024-02-02T15:34:42.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.047+0000 7f7797e00700 1 --2- 172.21.15.1:0/3620797794 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f77780a0690 0x7f77780a2b50 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f77900a1eb0 tx=0x7f77900092e0 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:42.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.047+0000 7f7787e00700 1 -- 172.21.15.1:0/3735698883 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7780004830 con 0x55c430cc5010 2024-02-02T15:34:42.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.048+0000 7f77b1c6db80 1 -- 172.21.15.1:0/3735698883 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c430cc5010 msgr2=0x55c430cc53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:42.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.048+0000 7f77b1c6db80 1 --2- 172.21.15.1:0/3735698883 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c430cc5010 0x55c430cc53f0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f7780013d40 tx=0x7f7780009260 comp rx=0 tx=0).stop 2024-02-02T15:34:42.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.048+0000 7f77b1c6db80 1 -- 172.21.15.1:0/3735698883 shutdown_connections 2024-02-02T15:34:42.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.048+0000 7f77b1c6db80 1 --2- 172.21.15.1:0/3735698883 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c430cd07e0 0x55c430cd2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:42.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.048+0000 7f77b1c6db80 1 --2- 172.21.15.1:0/3735698883 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c430cc5010 0x55c430cc53f0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:42.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.048+0000 7f77b1c6db80 1 --2- 172.21.15.1:0/3735698883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c430cc5930 0x55c430cd02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:42.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.048+0000 7f77b1c6db80 1 -- 172.21.15.1:0/3735698883 >> 172.21.15.1:0/3735698883 conn(0x55c430c24db0 msgr2=0x55c430cc13b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:42.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.048+0000 7f77b1c6db80 1 -- 172.21.15.1:0/3735698883 shutdown_connections 2024-02-02T15:34:42.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.048+0000 7f77b1c6db80 1 -- 172.21.15.1:0/3735698883 wait complete. 2024-02-02T15:34:42.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.048+0000 7f77b1c6db80 1 -- start start 2024-02-02T15:34:42.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.049+0000 7f77b1c6db80 1 Processor -- start 2024-02-02T15:34:42.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.049+0000 7f77b1c6db80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c430cc5010 0x55c430cd0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:42.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.049+0000 7f77b1c6db80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c430cc5930 0x55c430ccd490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:42.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.049+0000 7f77b1c6db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c430cd07e0 0x55c430ccdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:42.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.049+0000 7f77b1c6db80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c430c16d50 con 0x55c430cc5930 2024-02-02T15:34:42.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.049+0000 7f77b1c6db80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c430bb2d80 con 0x55c430cc5010 2024-02-02T15:34:42.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.049+0000 7f77b1c6db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c430bb2a20 con 0x55c430cd07e0 2024-02-02T15:34:42.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.049+0000 7f77b1c6db80 10 client.0 _ll_register_callbacks cb 0x55c430bb4290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:34:42.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.049+0000 7f7797400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c430cc5930 0x55c430ccd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:42.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.049+0000 7f7797400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c430cc5930 0x55c430ccd490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45514/0 (socket says 172.21.15.1:45514) 2024-02-02T15:34:42.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.049+0000 7f7797400700 1 -- 172.21.15.1:0/1720525667 learned_addr learned my addr 172.21.15.1:0/1720525667 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:34:42.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.049+0000 7f7796a00700 1 --2- 172.21.15.1:0/1720525667 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c430cc5010 0x55c430cd0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:42.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.050+0000 7f7796a00700 1 --2- 172.21.15.1:0/1720525667 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c430cd07e0 0x55c430ccdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:42.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.050+0000 7f7797400700 1 -- 172.21.15.1:0/1720525667 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c430cd07e0 msgr2=0x55c430ccdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:42.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.050+0000 7f7797400700 1 --2- 172.21.15.1:0/1720525667 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c430cd07e0 0x55c430ccdbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:42.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.050+0000 7f7797400700 1 -- 172.21.15.1:0/1720525667 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c430cc5010 msgr2=0x55c430cd0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:42.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.050+0000 7f7797400700 1 --2- 172.21.15.1:0/1720525667 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c430cc5010 0x55c430cd0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:42.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.050+0000 7f7797400700 1 -- 172.21.15.1:0/1720525667 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f778001b040 con 0x55c430cc5930 2024-02-02T15:34:42.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.050+0000 7f7797400700 1 --2- 172.21.15.1:0/1720525667 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c430cc5930 0x55c430ccd490 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f7788012df0 tx=0x7f778800f6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed04b8e86e0031f7 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:42.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.050+0000 7f7784c00700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:34:42.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.050+0000 7f7784c00700 1 -- 172.21.15.1:0/1720525667 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7788017070 con 0x55c430cc5930 2024-02-02T15:34:42.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.050+0000 7f77b1c6db80 1 -- 172.21.15.1:0/1720525667 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c430cb90f0 con 0x55c430cc5930 2024-02-02T15:34:42.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.050+0000 7f7784c00700 1 -- 172.21.15.1:0/1720525667 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f778800fc30 con 0x55c430cc5930 2024-02-02T15:34:42.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.051+0000 7f7784c00700 1 -- 172.21.15.1:0/1720525667 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f77880122e0 con 0x55c430cc5930 2024-02-02T15:34:42.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.051+0000 7f77b1c6db80 10 client.17634 fetch_fsmap learned FSMap version 42 2024-02-02T15:34:42.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.051+0000 7f77b1c6db80 1 -- 172.21.15.1:0/1720525667 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55c430a44b60 con 0x55c430cc5930 2024-02-02T15:34:42.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.051+0000 7f7784c00700 1 -- 172.21.15.1:0/1720525667 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c430cb90f0 con 0x55c430cc5930 2024-02-02T15:34:42.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.051+0000 7f7784c00700 1 -- 172.21.15.1:0/1720525667 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 42) v1 ==== 7510+0+0 (secure 0 0 0) 0x7f7788003ac0 con 0x55c430cc5930 2024-02-02T15:34:42.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.051+0000 7f77b1c6db80 10 client.17634 fetch_fsmap finished waiting for FSMap version 42 2024-02-02T15:34:42.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.051+0000 7f77b1c6db80 10 client.17634 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:34:42.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.051+0000 7f77b1c6db80 20 client.17634 populate_metadata read hostname 'smithi001' 2024-02-02T15:34:42.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.051+0000 7f77b1c6db80 1 --2- 172.21.15.1:0/1720525667 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55c430ce0260 0x55c430ce2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:42.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.051+0000 7f77b1c6db80 4 client.17634 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:34:42.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.051+0000 7f77b1c6db80 1 -- 172.21.15.1:0/1720525667 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c430c0fdd0 con 0x55c430ce0260 2024-02-02T15:34:42.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.052+0000 7f7796a00700 1 --2- 172.21.15.1:0/1720525667 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55c430ce0260 0x55c430ce2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:42.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.052+0000 7f7796a00700 1 --2- 172.21.15.1:0/1720525667 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55c430ce0260 0x55c430ce2680 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fd02914f73263d3 server_cookie=2ed1ba0a5ad3d9dc in_seq=0 out_seq=0 2024-02-02T15:34:42.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.052+0000 7f7784c00700 10 client.17634 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:34:42.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.053+0000 7f7784c00700 1 -- 172.21.15.1:0/1720525667 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55c430c0fdd0 con 0x55c430ce0260 2024-02-02T15:34:42.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.053+0000 7f7784c00700 10 client.17634 handle_command_reply: tid=0 2024-02-02T15:34:42.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.053+0000 7f77b1c6db80 1 client.17634 shutdown 2024-02-02T15:34:42.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.053+0000 7f77b1c6db80 1 -- 172.21.15.1:0/1720525667 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c430cc5930 msgr2=0x55c430ccd490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:42.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.053+0000 7f77b1c6db80 1 --2- 172.21.15.1:0/1720525667 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c430cc5930 0x55c430ccd490 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f7788012df0 tx=0x7f778800f6a0 comp rx=0 tx=0).stop 2024-02-02T15:34:42.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.053+0000 7f77b1c6db80 1 -- 172.21.15.1:0/1720525667 shutdown_connections 2024-02-02T15:34:42.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.053+0000 7f77b1c6db80 1 --2- 172.21.15.1:0/1720525667 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55c430ce0260 0x55c430ce2680 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:42.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.053+0000 7f77b1c6db80 1 --2- 172.21.15.1:0/1720525667 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c430cd07e0 0x55c430ccdbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:42.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.053+0000 7f77b1c6db80 1 --2- 172.21.15.1:0/1720525667 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c430cc5010 0x55c430cd0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:42.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.053+0000 7f77b1c6db80 1 --2- 172.21.15.1:0/1720525667 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c430cc5930 0x55c430ccd490 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:42.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.053+0000 7f77b1c6db80 1 -- 172.21.15.1:0/1720525667 >> 172.21.15.1:0/1720525667 conn(0x55c430c24db0 msgr2=0x55c430cc0fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:42.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.053+0000 7f7784c00700 0 client.17634 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:34:42.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.053+0000 7f77b1c6db80 1 -- 172.21.15.1:0/1720525667 shutdown_connections 2024-02-02T15:34:42.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.053+0000 7f77b1c6db80 1 -- 172.21.15.1:0/1720525667 wait complete. 2024-02-02T15:34:42.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.053+0000 7f77b1c6db80 20 client.17634 trim_cache size 0 max 16384 2024-02-02T15:34:42.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.066+0000 7f77b1c6db80 1 Processor -- start 2024-02-02T15:34:42.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.066+0000 7f77b1c6db80 1 -- start start 2024-02-02T15:34:42.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.067+0000 7f77b1c6db80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c430cc5010 0x55c430ccfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:42.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.067+0000 7f77b1c6db80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c430cc5930 0x55c430cc7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:42.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.067+0000 7f77b1c6db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c430cd07e0 0x55c430cc7d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:42.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.067+0000 7f77b1c6db80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c430a9a6a0 con 0x55c430cc5930 2024-02-02T15:34:42.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.067+0000 7f77b1c6db80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c430cc82c0 con 0x55c430cc5010 2024-02-02T15:34:42.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.067+0000 7f77b1c6db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c430cdc130 con 0x55c430cd07e0 2024-02-02T15:34:42.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.067+0000 7f7796a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c430cc5010 0x55c430ccfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:42.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.067+0000 7f7796a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c430cc5010 0x55c430ccfdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59028/0 (socket says 172.21.15.1:59028) 2024-02-02T15:34:42.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.067+0000 7f7796a00700 1 -- 172.21.15.1:0/2963280348 learned_addr learned my addr 172.21.15.1:0/2963280348 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:34:42.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.067+0000 7f7797400700 1 --2- 172.21.15.1:0/2963280348 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c430cc5930 0x55c430cc7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:42.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.067+0000 7f7796a00700 1 --2- 172.21.15.1:0/2963280348 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c430cd07e0 0x55c430cc7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:42.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.067+0000 7f7796a00700 1 -- 172.21.15.1:0/2963280348 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c430cd07e0 msgr2=0x55c430cc7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:42.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.067+0000 7f7796a00700 1 --2- 172.21.15.1:0/2963280348 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c430cd07e0 0x55c430cc7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:42.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.068+0000 7f7796a00700 1 -- 172.21.15.1:0/2963280348 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c430cc5930 msgr2=0x55c430cc7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:42.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.068+0000 7f7796a00700 1 --2- 172.21.15.1:0/2963280348 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c430cc5930 0x55c430cc7840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:42.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.068+0000 7f7796a00700 1 -- 172.21.15.1:0/2963280348 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c430cbd0d0 con 0x55c430cc5010 2024-02-02T15:34:42.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.068+0000 7f7796a00700 1 --2- 172.21.15.1:0/2963280348 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c430cc5010 0x55c430ccfdb0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f7780005950 tx=0x7f7780018040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1fa94ceac9772f1 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:42.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.068+0000 7f778d200700 1 -- 172.21.15.1:0/2963280348 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f778002d400 con 0x55c430cc5010 2024-02-02T15:34:42.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.068+0000 7f778d200700 1 -- 172.21.15.1:0/2963280348 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f77800130c0 con 0x55c430cc5010 2024-02-02T15:34:42.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.068+0000 7f778d200700 1 -- 172.21.15.1:0/2963280348 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7780005470 con 0x55c430cc5010 2024-02-02T15:34:42.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.069+0000 7f77b1c6db80 1 -- 172.21.15.1:0/2963280348 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c430cc5010 msgr2=0x55c430ccfdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:42.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.069+0000 7f77b1c6db80 1 --2- 172.21.15.1:0/2963280348 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c430cc5010 0x55c430ccfdb0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f7780005950 tx=0x7f7780018040 comp rx=0 tx=0).stop 2024-02-02T15:34:42.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.069+0000 7f77b1c6db80 1 -- 172.21.15.1:0/2963280348 shutdown_connections 2024-02-02T15:34:42.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.069+0000 7f77b1c6db80 1 --2- 172.21.15.1:0/2963280348 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c430cd07e0 0x55c430cc7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:42.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.069+0000 7f77b1c6db80 1 --2- 172.21.15.1:0/2963280348 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c430cc5010 0x55c430ccfdb0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:42.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.069+0000 7f77b1c6db80 1 --2- 172.21.15.1:0/2963280348 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c430cc5930 0x55c430cc7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:42.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.069+0000 7f77b1c6db80 1 -- 172.21.15.1:0/2963280348 >> 172.21.15.1:0/2963280348 conn(0x55c430c24db0 msgr2=0x55c430cc87a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:42.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.069+0000 7f77b1c6db80 1 -- 172.21.15.1:0/2963280348 shutdown_connections 2024-02-02T15:34:42.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.069+0000 7f77b1c6db80 1 -- 172.21.15.1:0/2963280348 wait complete. 2024-02-02T15:34:42.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.069+0000 7f77b1c6db80 1 -- start start 2024-02-02T15:34:42.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.069+0000 7f77b1c6db80 1 Processor -- start 2024-02-02T15:34:42.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.070+0000 7f77b1c6db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c430cc5010 0x55c430cd52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:42.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.070+0000 7f77b1c6db80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c430cc5930 0x55c430cd57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:42.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.070+0000 7f77b1c6db80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c430cd07e0 0x55c430cd5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:42.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.070+0000 7f77b1c6db80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c430d30e90 con 0x55c430cc5930 2024-02-02T15:34:42.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.070+0000 7f77b1c6db80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c430d31000 con 0x55c430cd07e0 2024-02-02T15:34:42.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.070+0000 7f77b1c6db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c430d311c0 con 0x55c430cc5010 2024-02-02T15:34:42.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.070+0000 7f77b1c6db80 10 client.0 _ll_register_callbacks cb 0x55c430bb4290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:34:42.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.070+0000 7f7797400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c430cc5930 0x55c430cd57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:42.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.070+0000 7f7797400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c430cc5930 0x55c430cd57f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45544/0 (socket says 172.21.15.1:45544) 2024-02-02T15:34:42.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.070+0000 7f7797400700 1 -- 172.21.15.1:0/619601815 learned_addr learned my addr 172.21.15.1:0/619601815 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:34:42.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.070+0000 7f7796a00700 1 --2- 172.21.15.1:0/619601815 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c430cc5010 0x55c430cd52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:42.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.070+0000 7f7796a00700 1 --2- 172.21.15.1:0/619601815 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c430cd07e0 0x55c430cd5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:42.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.071+0000 7f7797400700 1 -- 172.21.15.1:0/619601815 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c430cc5010 msgr2=0x55c430cd52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:42.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.071+0000 7f7797400700 1 --2- 172.21.15.1:0/619601815 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c430cc5010 0x55c430cd52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:42.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.071+0000 7f7797400700 1 -- 172.21.15.1:0/619601815 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c430cd07e0 msgr2=0x55c430cd5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:42.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.071+0000 7f7797400700 1 --2- 172.21.15.1:0/619601815 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c430cd07e0 0x55c430cd5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:42.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.071+0000 7f7797400700 1 -- 172.21.15.1:0/619601815 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c430a983b0 con 0x55c430cc5930 2024-02-02T15:34:42.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.071+0000 7f7797400700 1 --2- 172.21.15.1:0/619601815 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c430cc5930 0x55c430cd57f0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f7788005a80 tx=0x7f77880122e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=451b9303963496ff server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:42.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.071+0000 7f7784c00700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:34:42.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.071+0000 7f7784c00700 1 -- 172.21.15.1:0/619601815 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7788017040 con 0x55c430cc5930 2024-02-02T15:34:42.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.071+0000 7f7784c00700 1 -- 172.21.15.1:0/619601815 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f778800fe40 con 0x55c430cc5930 2024-02-02T15:34:42.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.071+0000 7f77b1c6db80 1 -- 172.21.15.1:0/619601815 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7780010e80 con 0x55c430cc5930 2024-02-02T15:34:42.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.071+0000 7f7784c00700 1 -- 172.21.15.1:0/619601815 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7788003880 con 0x55c430cc5930 2024-02-02T15:34:42.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.071+0000 7f77b1c6db80 10 client.17640 fetch_fsmap learned FSMap version 42 2024-02-02T15:34:42.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.072+0000 7f77b1c6db80 1 -- 172.21.15.1:0/619601815 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55c430bdcea0 con 0x55c430cc5930 2024-02-02T15:34:42.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.072+0000 7f7784c00700 1 -- 172.21.15.1:0/619601815 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7780010e80 con 0x55c430cc5930 2024-02-02T15:34:42.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.072+0000 7f7784c00700 1 -- 172.21.15.1:0/619601815 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 42) v1 ==== 7510+0+0 (secure 0 0 0) 0x7f778801c770 con 0x55c430cc5930 2024-02-02T15:34:42.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.072+0000 7f77b1c6db80 10 client.17640 fetch_fsmap finished waiting for FSMap version 42 2024-02-02T15:34:42.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.072+0000 7f77b1c6db80 10 client.17640 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:34:42.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.072+0000 7f77b1c6db80 20 client.17640 populate_metadata read hostname 'smithi001' 2024-02-02T15:34:42.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.072+0000 7f77b1c6db80 1 --2- 172.21.15.1:0/619601815 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55c430d32900 0x55c430d34d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:42.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.072+0000 7f77b1c6db80 4 client.17640 mds_command: new command op to 24557 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:34:42.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.072+0000 7f77b1c6db80 1 -- 172.21.15.1:0/619601815 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c430bd5f50 con 0x55c430d32900 2024-02-02T15:34:42.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.072+0000 7f7796a00700 1 --2- 172.21.15.1:0/619601815 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55c430d32900 0x55c430d34d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:42.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.073+0000 7f7796a00700 1 --2- 172.21.15.1:0/619601815 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55c430d32900 0x55c430d34d20 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfdb132dd02a1bf server_cookie=8a5f685551fa010c in_seq=0 out_seq=0 2024-02-02T15:34:42.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.073+0000 7f7784c00700 10 client.17640 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:34:42.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.073+0000 7f7784c00700 1 -- 172.21.15.1:0/619601815 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c430bd5f50 con 0x55c430d32900 2024-02-02T15:34:42.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.073+0000 7f7784c00700 10 client.17640 handle_command_reply: tid=0 2024-02-02T15:34:42.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.073+0000 7f77b1c6db80 1 client.17640 shutdown 2024-02-02T15:34:42.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.073+0000 7f77b1c6db80 1 -- 172.21.15.1:0/619601815 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c430cc5930 msgr2=0x55c430cd57f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:42.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.073+0000 7f77b1c6db80 1 --2- 172.21.15.1:0/619601815 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c430cc5930 0x55c430cd57f0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f7788005a80 tx=0x7f77880122e0 comp rx=0 tx=0).stop 2024-02-02T15:34:42.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.073+0000 7f77b1c6db80 1 -- 172.21.15.1:0/619601815 shutdown_connections 2024-02-02T15:34:42.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.073+0000 7f77b1c6db80 1 --2- 172.21.15.1:0/619601815 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55c430d32900 0x55c430d34d20 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:42.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.073+0000 7f77b1c6db80 1 --2- 172.21.15.1:0/619601815 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c430cd07e0 0x55c430cd5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:42.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.073+0000 7f77b1c6db80 1 --2- 172.21.15.1:0/619601815 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c430cc5010 0x55c430cd52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:42.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.073+0000 7f7784c00700 0 client.17640 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:34:42.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.073+0000 7f77b1c6db80 1 --2- 172.21.15.1:0/619601815 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c430cc5930 0x55c430cd57f0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:42.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.073+0000 7f77b1c6db80 1 -- 172.21.15.1:0/619601815 >> 172.21.15.1:0/619601815 conn(0x55c430c24db0 msgr2=0x55c430cccd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:42.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.073+0000 7f77b1c6db80 1 -- 172.21.15.1:0/619601815 shutdown_connections 2024-02-02T15:34:42.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.073+0000 7f77b1c6db80 1 -- 172.21.15.1:0/619601815 wait complete. 2024-02-02T15:34:42.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.074+0000 7f77b1c6db80 20 client.17640 trim_cache size 0 max 16384 2024-02-02T15:34:42.081 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:34:42.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.074+0000 7f779e200700 1 -- 172.21.15.1:0/3620797794 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f77780a0690 msgr2=0x7f77780a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:42.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.074+0000 7f779e200700 1 --2- 172.21.15.1:0/3620797794 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f77780a0690 0x7f77780a2b50 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f77900a1eb0 tx=0x7f77900092e0 comp rx=0 tx=0).stop 2024-02-02T15:34:42.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.074+0000 7f779e200700 1 -- 172.21.15.1:0/3620797794 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7798111d90 msgr2=0x7f77981a1f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:42.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.074+0000 7f779e200700 1 --2- 172.21.15.1:0/3620797794 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7798111d90 0x7f77981a1f70 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f7790009fd0 tx=0x7f779000bce0 comp rx=0 tx=0).stop 2024-02-02T15:34:42.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.074+0000 7f779e200700 1 -- 172.21.15.1:0/3620797794 shutdown_connections 2024-02-02T15:34:42.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.074+0000 7f779e200700 1 --2- 172.21.15.1:0/3620797794 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f77780a0690 0x7f77780a2b50 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:42.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.074+0000 7f779e200700 1 --2- 172.21.15.1:0/3620797794 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7798111d90 0x7f77981a1f70 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:42.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.074+0000 7f779e200700 1 --2- 172.21.15.1:0/3620797794 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f77980730e0 0x7f779819d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:42.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.074+0000 7f779e200700 1 --2- 172.21.15.1:0/3620797794 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7798073a00 0x7f779819db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:42.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.074+0000 7f779e200700 1 -- 172.21.15.1:0/3620797794 >> 172.21.15.1:0/3620797794 conn(0x7f77980fc810 msgr2=0x7f77980fd480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:42.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.074+0000 7f779e200700 1 -- 172.21.15.1:0/3620797794 shutdown_connections 2024-02-02T15:34:42.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:42.075+0000 7f779e200700 1 -- 172.21.15.1:0/3620797794 wait complete. 2024-02-02T15:34:42.246 INFO:tasks.workunit.client.0.smithi001.stdout: 1 464304 6.88 MB/sec execute 256 sec latency 470.771 ms 2024-02-02T15:34:42.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:42 smithi071 ceph-mon[44707]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:34:42.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:42 smithi142 ceph-mon[45728]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:34:42.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:42 smithi001 ceph-mon[35470]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:34:43.093 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-02T15:34:43.093 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-02T15:34:43.246 INFO:tasks.workunit.client.0.smithi001.stdout: 1 464344 6.85 MB/sec execute 257 sec latency 859.477 ms 2024-02-02T15:34:43.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.306+0000 7f0d06c00700 1 -- 172.21.15.1:0/4115751114 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d001019a0 msgr2=0x7f0d001059e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:43.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.306+0000 7f0d06c00700 1 --2- 172.21.15.1:0/4115751114 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d001019a0 0x7f0d001059e0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8009990 tx=0x7f0ce8009ca0 comp rx=0 tx=0).stop 2024-02-02T15:34:43.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.307+0000 7f0d06c00700 1 -- 172.21.15.1:0/4115751114 shutdown_connections 2024-02-02T15:34:43.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.307+0000 7f0d06c00700 1 --2- 172.21.15.1:0/4115751114 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d00106110 0x7f0d0010dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:43.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.307+0000 7f0d06c00700 1 --2- 172.21.15.1:0/4115751114 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d00100ff0 0x7f0d001013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:43.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.307+0000 7f0d06c00700 1 --2- 172.21.15.1:0/4115751114 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d001019a0 0x7f0d001059e0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:43.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.307+0000 7f0d06c00700 1 -- 172.21.15.1:0/4115751114 >> 172.21.15.1:0/4115751114 conn(0x7f0d000fc810 msgr2=0x7f0d000fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:43.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.307+0000 7f0d06c00700 1 -- 172.21.15.1:0/4115751114 shutdown_connections 2024-02-02T15:34:43.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.307+0000 7f0d06c00700 1 -- 172.21.15.1:0/4115751114 wait complete. 2024-02-02T15:34:43.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.308+0000 7f0d06c00700 1 Processor -- start 2024-02-02T15:34:43.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.308+0000 7f0d06c00700 1 -- start start 2024-02-02T15:34:43.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.308+0000 7f0d06c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d00100ff0 0x7f0d00073040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:43.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.308+0000 7f0d06c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d00106110 0x7f0d00073580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:43.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.308+0000 7f0d06c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d00073ac0 0x7f0d001a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:43.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.308+0000 7f0d06c00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d001a5780 con 0x7f0d00073ac0 2024-02-02T15:34:43.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.308+0000 7f0d06c00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d001a58f0 con 0x7f0d00106110 2024-02-02T15:34:43.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.308+0000 7f0d06c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d001a5ab0 con 0x7f0d00100ff0 2024-02-02T15:34:43.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.308+0000 7f0d04c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d00073ac0 0x7f0d001a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:43.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.309+0000 7f0d04c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d00073ac0 0x7f0d001a50f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:34212/0 (socket says 172.21.15.1:34212) 2024-02-02T15:34:43.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.309+0000 7f0d04c00700 1 -- 172.21.15.1:0/3320258062 learned_addr learned my addr 172.21.15.1:0/3320258062 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:34:43.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.309+0000 7f0cffe00700 1 --2- 172.21.15.1:0/3320258062 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d00100ff0 0x7f0d00073040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:43.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.309+0000 7f0cff400700 1 --2- 172.21.15.1:0/3320258062 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d00106110 0x7f0d00073580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:43.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.309+0000 7f0d04c00700 1 -- 172.21.15.1:0/3320258062 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d00100ff0 msgr2=0x7f0d00073040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:43.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.309+0000 7f0d04c00700 1 --2- 172.21.15.1:0/3320258062 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d00100ff0 0x7f0d00073040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:43.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.309+0000 7f0d04c00700 1 -- 172.21.15.1:0/3320258062 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d00106110 msgr2=0x7f0d00073580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:43.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.309+0000 7f0d04c00700 1 --2- 172.21.15.1:0/3320258062 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d00106110 0x7f0d00073580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:43.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.309+0000 7f0d04c00700 1 -- 172.21.15.1:0/3320258062 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ce80095f0 con 0x7f0d00073ac0 2024-02-02T15:34:43.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.309+0000 7f0d04c00700 1 --2- 172.21.15.1:0/3320258062 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d00073ac0 0x7f0d001a50f0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8007ee0 tx=0x7f0cf800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:43.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.309+0000 7f0cfcc00700 1 -- 172.21.15.1:0/3320258062 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf8017070 con 0x7f0d00073ac0 2024-02-02T15:34:43.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.310+0000 7f0cfcc00700 1 -- 172.21.15.1:0/3320258062 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0cf800f040 con 0x7f0d00073ac0 2024-02-02T15:34:43.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.310+0000 7f0d06c00700 1 -- 172.21.15.1:0/3320258062 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d001a5d90 con 0x7f0d00073ac0 2024-02-02T15:34:43.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.310+0000 7f0cfcc00700 1 -- 172.21.15.1:0/3320258062 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf8005580 con 0x7f0d00073ac0 2024-02-02T15:34:43.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.310+0000 7f0d06c00700 1 -- 172.21.15.1:0/3320258062 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d001a62a0 con 0x7f0d00073ac0 2024-02-02T15:34:43.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.313+0000 7f0cfcc00700 1 -- 172.21.15.1:0/3320258062 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f0cf800db20 con 0x7f0d00073ac0 2024-02-02T15:34:43.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.315+0000 7f0cfcc00700 1 --2- 172.21.15.1:0/3320258062 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f0ce00a0690 0x7f0ce00a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:43.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.315+0000 7f0d1a6e4b80 1 Processor -- start 2024-02-02T15:34:43.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.315+0000 7f0cfcc00700 1 -- 172.21.15.1:0/3320258062 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f0cf80cb820 con 0x7f0d00073ac0 2024-02-02T15:34:43.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.315+0000 7f0d1a6e4b80 1 -- start start 2024-02-02T15:34:43.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.315+0000 7f0cffe00700 1 --2- 172.21.15.1:0/3320258062 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f0ce00a0690 0x7f0ce00a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:43.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.315+0000 7f0d1a6e4b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5588a7209090 0x5588a7209470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:43.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.315+0000 7f0d1a6e4b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588a72099b0 0x5588a7214320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:43.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.315+0000 7f0d1a6e4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588a7214860 0x5588a7216c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:43.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.315+0000 7f0d1a6e4b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588a717d540 con 0x5588a7209090 2024-02-02T15:34:43.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.315+0000 7f0d1a6e4b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588a7161b20 con 0x5588a72099b0 2024-02-02T15:34:43.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.315+0000 7f0d1a6e4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588a715f4e0 con 0x5588a7214860 2024-02-02T15:34:43.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.315+0000 7f0cffe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5588a7209090 0x5588a7209470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:43.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.316+0000 7f0cffe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5588a7209090 0x5588a7209470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:34222/0 (socket says 172.21.15.1:34222) 2024-02-02T15:34:43.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.316+0000 7f0cff400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588a72099b0 0x5588a7214320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:43.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.316+0000 7f0cffe00700 1 -- 172.21.15.1:0/3910131081 learned_addr learned my addr 172.21.15.1:0/3910131081 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:34:43.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.316+0000 7f0d04c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588a7214860 0x5588a7216c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:43.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.316+0000 7f0cff400700 1 -- 172.21.15.1:0/3910131081 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588a7214860 msgr2=0x5588a7216c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:43.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.316+0000 7f0cff400700 1 --2- 172.21.15.1:0/3910131081 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588a7214860 0x5588a7216c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:43.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.316+0000 7f0cff400700 1 -- 172.21.15.1:0/3910131081 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5588a7209090 msgr2=0x5588a7209470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:43.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.316+0000 7f0cff400700 1 --2- 172.21.15.1:0/3910131081 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5588a7209090 0x5588a7209470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:43.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.316+0000 7f0cff400700 1 -- 172.21.15.1:0/3910131081 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5588a6fdc430 con 0x5588a72099b0 2024-02-02T15:34:43.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.316+0000 7f0cffe00700 1 --2- 172.21.15.1:0/3320258062 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f0ce00a0690 0x7f0ce00a2b50 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0cf0013650 tx=0x7f0cf0013a10 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:43.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.316+0000 7f0d04c00700 1 --2- 172.21.15.1:0/3910131081 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588a7214860 0x5588a7216c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:34:43.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.316+0000 7f0cff400700 1 --2- 172.21.15.1:0/3910131081 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588a72099b0 0x5588a7214320 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8006010 tx=0x7f0ce8005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd148348e71fc15d server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:43.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.317+0000 7f0cf5200700 1 -- 172.21.15.1:0/3910131081 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce801d070 con 0x5588a72099b0 2024-02-02T15:34:43.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.317+0000 7f0cf5200700 1 -- 172.21.15.1:0/3910131081 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ce801e040 con 0x5588a72099b0 2024-02-02T15:34:43.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.317+0000 7f0cf5200700 1 -- 172.21.15.1:0/3910131081 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce8004740 con 0x5588a72099b0 2024-02-02T15:34:43.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.317+0000 7f0d1a6e4b80 1 -- 172.21.15.1:0/3910131081 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588a72099b0 msgr2=0x5588a7214320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:43.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.317+0000 7f0d1a6e4b80 1 --2- 172.21.15.1:0/3910131081 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588a72099b0 0x5588a7214320 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8006010 tx=0x7f0ce8005c00 comp rx=0 tx=0).stop 2024-02-02T15:34:43.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.318+0000 7f0d1a6e4b80 1 -- 172.21.15.1:0/3910131081 shutdown_connections 2024-02-02T15:34:43.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.318+0000 7f0d1a6e4b80 1 --2- 172.21.15.1:0/3910131081 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588a7214860 0x5588a7216c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:43.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.318+0000 7f0d1a6e4b80 1 --2- 172.21.15.1:0/3910131081 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5588a7209090 0x5588a7209470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:43.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.318+0000 7f0d1a6e4b80 1 --2- 172.21.15.1:0/3910131081 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588a72099b0 0x5588a7214320 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:43.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.318+0000 7f0d1a6e4b80 1 -- 172.21.15.1:0/3910131081 >> 172.21.15.1:0/3910131081 conn(0x5588a7168e30 msgr2=0x5588a7205430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:43.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.318+0000 7f0d1a6e4b80 1 -- 172.21.15.1:0/3910131081 shutdown_connections 2024-02-02T15:34:43.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.318+0000 7f0d1a6e4b80 1 -- 172.21.15.1:0/3910131081 wait complete. 2024-02-02T15:34:43.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.318+0000 7f0d1a6e4b80 1 -- start start 2024-02-02T15:34:43.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.318+0000 7f0d1a6e4b80 1 Processor -- start 2024-02-02T15:34:43.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.319+0000 7f0d1a6e4b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588a7209090 0x5588a72141d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:43.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.319+0000 7f0d1a6e4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588a72099b0 0x5588a72113e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:43.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.319+0000 7f0d1a6e4b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5588a7214860 0x5588a7211960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:43.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.319+0000 7f0d1a6e4b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588a715add0 con 0x5588a7214860 2024-02-02T15:34:43.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.319+0000 7f0d1a6e4b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588a70f6e00 con 0x5588a7209090 2024-02-02T15:34:43.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.319+0000 7f0d1a6e4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588a70f6aa0 con 0x5588a72099b0 2024-02-02T15:34:43.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.319+0000 7f0d1a6e4b80 10 client.0 _ll_register_callbacks cb 0x5588a70f8310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:34:43.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.319+0000 7f0cffe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588a7209090 0x5588a72141d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:43.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.319+0000 7f0d04c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5588a7214860 0x5588a7211960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:43.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.319+0000 7f0cffe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588a7209090 0x5588a72141d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:43784/0 (socket says 172.21.15.1:43784) 2024-02-02T15:34:43.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.319+0000 7f0cffe00700 1 -- 172.21.15.1:0/2708830026 learned_addr learned my addr 172.21.15.1:0/2708830026 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:34:43.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.319+0000 7f0d04c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5588a7214860 0x5588a7211960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:34236/0 (socket says 172.21.15.1:34236) 2024-02-02T15:34:43.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.319+0000 7f0cff400700 1 --2- 172.21.15.1:0/2708830026 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588a72099b0 0x5588a72113e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:43.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.320+0000 7f0d04c00700 1 -- 172.21.15.1:0/2708830026 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588a72099b0 msgr2=0x5588a72113e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:43.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.320+0000 7f0d04c00700 1 --2- 172.21.15.1:0/2708830026 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588a72099b0 0x5588a72113e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:43.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.320+0000 7f0d04c00700 1 -- 172.21.15.1:0/2708830026 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588a7209090 msgr2=0x5588a72141d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:43.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.320+0000 7f0d04c00700 1 --2- 172.21.15.1:0/2708830026 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588a7209090 0x5588a72141d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:43.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.320+0000 7f0d04c00700 1 -- 172.21.15.1:0/2708830026 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d001a62a0 con 0x5588a7214860 2024-02-02T15:34:43.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.320+0000 7f0cffe00700 1 --2- 172.21.15.1:0/2708830026 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588a7209090 0x5588a72141d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:34:43.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.320+0000 7f0d04c00700 1 --2- 172.21.15.1:0/2708830026 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5588a7214860 0x5588a7211960 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f0cf80c8b70 tx=0x7f0cf8005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c780925e761d84f server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:43.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.320+0000 7f0ced600700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:34:43.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.320+0000 7f0ced600700 1 -- 172.21.15.1:0/2708830026 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf8098940 con 0x5588a7214860 2024-02-02T15:34:43.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.320+0000 7f0ced600700 1 -- 172.21.15.1:0/2708830026 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f0cf809c770 con 0x5588a7214860 2024-02-02T15:34:43.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.320+0000 7f0d1a6e4b80 1 -- 172.21.15.1:0/2708830026 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5588a71fd170 con 0x5588a7214860 2024-02-02T15:34:43.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.320+0000 7f0ced600700 1 -- 172.21.15.1:0/2708830026 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0cf80a09b0 con 0x5588a7214860 2024-02-02T15:34:43.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.320+0000 7f0d1a6e4b80 10 client.17652 fetch_fsmap learned FSMap version 42 2024-02-02T15:34:43.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.320+0000 7f0d1a6e4b80 1 -- 172.21.15.1:0/2708830026 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7f0d0010e500 con 0x5588a7214860 2024-02-02T15:34:43.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.321+0000 7f0ced600700 1 -- 172.21.15.1:0/2708830026 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5588a71fd170 con 0x5588a7214860 2024-02-02T15:34:43.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.321+0000 7f0ced600700 1 -- 172.21.15.1:0/2708830026 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 42) v1 ==== 7510+0+0 (secure 0 0 0) 0x7f0cf8098ba0 con 0x5588a7214860 2024-02-02T15:34:43.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.321+0000 7f0d1a6e4b80 10 client.17652 fetch_fsmap finished waiting for FSMap version 42 2024-02-02T15:34:43.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.321+0000 7f0d1a6e4b80 10 client.17652 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:34:43.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.321+0000 7f0d1a6e4b80 20 client.17652 populate_metadata read hostname 'smithi001' 2024-02-02T15:34:43.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.321+0000 7f0d1a6e4b80 1 --2- 172.21.15.1:0/2708830026 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5588a72241f0 0x5588a7226610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:43.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.321+0000 7f0d1a6e4b80 4 client.17652 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:34:43.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.321+0000 7f0d1a6e4b80 1 -- 172.21.15.1:0/2708830026 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5588a7153e50 con 0x5588a72241f0 2024-02-02T15:34:43.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.321+0000 7f0cffe00700 1 --2- 172.21.15.1:0/2708830026 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5588a72241f0 0x5588a7226610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:43.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.322+0000 7f0cffe00700 1 --2- 172.21.15.1:0/2708830026 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5588a72241f0 0x5588a7226610 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8af9b3e3f37accb server_cookie=4f9e1cb2ec653d02 in_seq=0 out_seq=0 2024-02-02T15:34:43.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.322+0000 7f0ced600700 10 client.17652 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:34:43.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.322+0000 7f0ced600700 1 -- 172.21.15.1:0/2708830026 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5588a7153e50 con 0x5588a72241f0 2024-02-02T15:34:43.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.323+0000 7f0ced600700 10 client.17652 handle_command_reply: tid=0 2024-02-02T15:34:43.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.323+0000 7f0d1a6e4b80 1 client.17652 shutdown 2024-02-02T15:34:43.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.323+0000 7f0d1a6e4b80 1 -- 172.21.15.1:0/2708830026 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5588a7214860 msgr2=0x5588a7211960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:43.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.323+0000 7f0d1a6e4b80 1 --2- 172.21.15.1:0/2708830026 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5588a7214860 0x5588a7211960 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f0cf80c8b70 tx=0x7f0cf8005020 comp rx=0 tx=0).stop 2024-02-02T15:34:43.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.323+0000 7f0d1a6e4b80 1 -- 172.21.15.1:0/2708830026 shutdown_connections 2024-02-02T15:34:43.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.323+0000 7f0d1a6e4b80 1 --2- 172.21.15.1:0/2708830026 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5588a72241f0 0x5588a7226610 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:43.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.323+0000 7f0d1a6e4b80 1 --2- 172.21.15.1:0/2708830026 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5588a7214860 0x5588a7211960 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:43.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.323+0000 7f0d1a6e4b80 1 --2- 172.21.15.1:0/2708830026 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588a7209090 0x5588a72141d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:43.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.323+0000 7f0d1a6e4b80 1 --2- 172.21.15.1:0/2708830026 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588a72099b0 0x5588a72113e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:43.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.323+0000 7f0d1a6e4b80 1 -- 172.21.15.1:0/2708830026 >> 172.21.15.1:0/2708830026 conn(0x5588a7168e30 msgr2=0x5588a7205020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:43.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.323+0000 7f0ced600700 0 client.17652 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:34:43.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.323+0000 7f0d1a6e4b80 1 -- 172.21.15.1:0/2708830026 shutdown_connections 2024-02-02T15:34:43.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.323+0000 7f0d1a6e4b80 1 -- 172.21.15.1:0/2708830026 wait complete. 2024-02-02T15:34:43.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.323+0000 7f0d1a6e4b80 20 client.17652 trim_cache size 0 max 16384 2024-02-02T15:34:43.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.341+0000 7f0d1a6e4b80 1 Processor -- start 2024-02-02T15:34:43.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.341+0000 7f0d1a6e4b80 1 -- start start 2024-02-02T15:34:43.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.341+0000 7f0d1a6e4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588a7209090 0x5588a72055b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:43.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.341+0000 7f0d1a6e4b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5588a72099b0 0x5588a721b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:43.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.341+0000 7f0d1a6e4b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588a7214860 0x5588a720b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:43.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.341+0000 7f0d1a6e4b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588a6fde720 con 0x5588a72099b0 2024-02-02T15:34:43.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.341+0000 7f0d1a6e4b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588a71fd040 con 0x5588a7214860 2024-02-02T15:34:43.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.341+0000 7f0d1a6e4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588a7211f60 con 0x5588a7209090 2024-02-02T15:34:43.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.341+0000 7f0cff400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5588a72099b0 0x5588a721b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:43.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.341+0000 7f0cff400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5588a72099b0 0x5588a721b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:34244/0 (socket says 172.21.15.1:34244) 2024-02-02T15:34:43.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.341+0000 7f0cff400700 1 -- 172.21.15.1:0/2267956604 learned_addr learned my addr 172.21.15.1:0/2267956604 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:34:43.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.342+0000 7f0cffe00700 1 --2- 172.21.15.1:0/2267956604 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588a7209090 0x5588a72055b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:43.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.342+0000 7f0d04c00700 1 --2- 172.21.15.1:0/2267956604 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588a7214860 0x5588a720b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:43.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.342+0000 7f0cff400700 1 -- 172.21.15.1:0/2267956604 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588a7209090 msgr2=0x5588a72055b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:43.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.342+0000 7f0cff400700 1 --2- 172.21.15.1:0/2267956604 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588a7209090 0x5588a72055b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:43.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.342+0000 7f0cff400700 1 -- 172.21.15.1:0/2267956604 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588a7214860 msgr2=0x5588a720b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:43.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.342+0000 7f0cff400700 1 --2- 172.21.15.1:0/2267956604 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588a7214860 0x5588a720b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:43.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.342+0000 7f0cff400700 1 -- 172.21.15.1:0/2267956604 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5588a6f88be0 con 0x5588a72099b0 2024-02-02T15:34:43.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.342+0000 7f0cff400700 1 --2- 172.21.15.1:0/2267956604 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5588a72099b0 0x5588a721b350 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8006010 tx=0x7f0ce800b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f7f08517bfc7acb server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:43.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.342+0000 7f0cf5c00700 1 -- 172.21.15.1:0/2267956604 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce801d040 con 0x5588a72099b0 2024-02-02T15:34:43.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.342+0000 7f0cf5c00700 1 -- 172.21.15.1:0/2267956604 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ce80056a0 con 0x5588a72099b0 2024-02-02T15:34:43.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.342+0000 7f0cf5c00700 1 -- 172.21.15.1:0/2267956604 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce8004be0 con 0x5588a72099b0 2024-02-02T15:34:43.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.343+0000 7f0d1a6e4b80 1 -- 172.21.15.1:0/2267956604 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5588a72099b0 msgr2=0x5588a721b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:43.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.343+0000 7f0d1a6e4b80 1 --2- 172.21.15.1:0/2267956604 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5588a72099b0 0x5588a721b350 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8006010 tx=0x7f0ce800b5b0 comp rx=0 tx=0).stop 2024-02-02T15:34:43.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.343+0000 7f0d1a6e4b80 1 -- 172.21.15.1:0/2267956604 shutdown_connections 2024-02-02T15:34:43.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.343+0000 7f0d1a6e4b80 1 --2- 172.21.15.1:0/2267956604 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588a7214860 0x5588a720b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:43.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.343+0000 7f0d1a6e4b80 1 --2- 172.21.15.1:0/2267956604 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588a7209090 0x5588a72055b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:43.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.343+0000 7f0d1a6e4b80 1 --2- 172.21.15.1:0/2267956604 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5588a72099b0 0x5588a721b350 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:43.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.343+0000 7f0d1a6e4b80 1 -- 172.21.15.1:0/2267956604 >> 172.21.15.1:0/2267956604 conn(0x5588a7168e30 msgr2=0x5588a72201b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:43.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.343+0000 7f0d1a6e4b80 1 -- 172.21.15.1:0/2267956604 shutdown_connections 2024-02-02T15:34:43.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.343+0000 7f0d1a6e4b80 1 -- 172.21.15.1:0/2267956604 wait complete. 2024-02-02T15:34:43.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.344+0000 7f0d1a6e4b80 1 -- start start 2024-02-02T15:34:43.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.344+0000 7f0d1a6e4b80 1 Processor -- start 2024-02-02T15:34:43.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.344+0000 7f0d1a6e4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588a7209090 0x5588a7269e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:43.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.344+0000 7f0d1a6e4b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5588a72099b0 0x5588a726a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:43.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.344+0000 7f0d1a6e4b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588a7214860 0x5588a726df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:43.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.344+0000 7f0d1a6e4b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588a726e520 con 0x5588a72099b0 2024-02-02T15:34:43.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.344+0000 7f0d1a6e4b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588a726e690 con 0x5588a7214860 2024-02-02T15:34:43.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.344+0000 7f0d1a6e4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588a726e850 con 0x5588a7209090 2024-02-02T15:34:43.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.344+0000 7f0d1a6e4b80 10 client.0 _ll_register_callbacks cb 0x5588a70f8310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:34:43.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.345+0000 7f0cff400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5588a72099b0 0x5588a726a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:43.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.345+0000 7f0cff400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5588a72099b0 0x5588a726a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:34252/0 (socket says 172.21.15.1:34252) 2024-02-02T15:34:43.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.345+0000 7f0cff400700 1 -- 172.21.15.1:0/4200781783 learned_addr learned my addr 172.21.15.1:0/4200781783 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:34:43.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.345+0000 7f0cffe00700 1 --2- 172.21.15.1:0/4200781783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588a7209090 0x5588a7269e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:43.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.345+0000 7f0d04c00700 1 --2- 172.21.15.1:0/4200781783 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588a7214860 0x5588a726df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:43.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.345+0000 7f0cff400700 1 -- 172.21.15.1:0/4200781783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588a7209090 msgr2=0x5588a7269e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:43.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.345+0000 7f0cff400700 1 --2- 172.21.15.1:0/4200781783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588a7209090 0x5588a7269e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:43.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.345+0000 7f0cff400700 1 -- 172.21.15.1:0/4200781783 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588a7214860 msgr2=0x5588a726df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:43.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.345+0000 7f0cff400700 1 --2- 172.21.15.1:0/4200781783 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588a7214860 0x5588a726df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:43.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.345+0000 7f0cff400700 1 -- 172.21.15.1:0/4200781783 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5588a6fdc430 con 0x5588a72099b0 2024-02-02T15:34:43.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.345+0000 7f0cff400700 1 --2- 172.21.15.1:0/4200781783 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5588a72099b0 0x5588a726a3d0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8009990 tx=0x7f0ce800bf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68d677350d00481a server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:43.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.345+0000 7f0ced600700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:34:43.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.345+0000 7f0ced600700 1 -- 172.21.15.1:0/4200781783 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ce801d070 con 0x5588a72099b0 2024-02-02T15:34:43.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.345+0000 7f0ced600700 1 -- 172.21.15.1:0/4200781783 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f0ce8004950 con 0x5588a72099b0 2024-02-02T15:34:43.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.345+0000 7f0d1a6e4b80 1 -- 172.21.15.1:0/4200781783 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0ce801e040 con 0x5588a72099b0 2024-02-02T15:34:43.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.346+0000 7f0ced600700 1 -- 172.21.15.1:0/4200781783 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ce8004b10 con 0x5588a72099b0 2024-02-02T15:34:43.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.346+0000 7f0d1a6e4b80 10 client.17658 fetch_fsmap learned FSMap version 42 2024-02-02T15:34:43.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.346+0000 7f0d1a6e4b80 1 -- 172.21.15.1:0/4200781783 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x5588a7120f20 con 0x5588a72099b0 2024-02-02T15:34:43.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.346+0000 7f0ced600700 1 -- 172.21.15.1:0/4200781783 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ce801e040 con 0x5588a72099b0 2024-02-02T15:34:43.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.346+0000 7f0ced600700 1 -- 172.21.15.1:0/4200781783 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 42) v1 ==== 7510+0+0 (secure 0 0 0) 0x7f0ce8035cb0 con 0x5588a72099b0 2024-02-02T15:34:43.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.346+0000 7f0d1a6e4b80 10 client.17658 fetch_fsmap finished waiting for FSMap version 42 2024-02-02T15:34:43.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.346+0000 7f0d1a6e4b80 10 client.17658 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:34:43.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.346+0000 7f0d1a6e4b80 20 client.17658 populate_metadata read hostname 'smithi001' 2024-02-02T15:34:43.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.346+0000 7f0d1a6e4b80 1 --2- 172.21.15.1:0/4200781783 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5588a7276db0 0x5588a72791d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:43.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.346+0000 7f0d1a6e4b80 4 client.17658 mds_command: new command op to 24557 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-02T15:34:43.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.346+0000 7f0d1a6e4b80 1 -- 172.21.15.1:0/4200781783 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5588a7119fd0 con 0x5588a7276db0 2024-02-02T15:34:43.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.347+0000 7f0cffe00700 1 --2- 172.21.15.1:0/4200781783 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5588a7276db0 0x5588a72791d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:43.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.347+0000 7f0cffe00700 1 --2- 172.21.15.1:0/4200781783 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5588a7276db0 0x5588a72791d0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75f1944eb6bb0d10 server_cookie=99dbc680f01dcda7 in_seq=0 out_seq=0 2024-02-02T15:34:43.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.347+0000 7f0ced600700 10 client.17658 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:34:43.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.348+0000 7f0ced600700 1 -- 172.21.15.1:0/4200781783 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5588a7119fd0 con 0x5588a7276db0 2024-02-02T15:34:43.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.348+0000 7f0ced600700 10 client.17658 handle_command_reply: tid=0 2024-02-02T15:34:43.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.348+0000 7f0d1a6e4b80 1 client.17658 shutdown 2024-02-02T15:34:43.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.348+0000 7f0d1a6e4b80 1 -- 172.21.15.1:0/4200781783 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5588a72099b0 msgr2=0x5588a726a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:43.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.348+0000 7f0d1a6e4b80 1 --2- 172.21.15.1:0/4200781783 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5588a72099b0 0x5588a726a3d0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f0ce8009990 tx=0x7f0ce800bf00 comp rx=0 tx=0).stop 2024-02-02T15:34:43.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.349+0000 7f0d1a6e4b80 1 -- 172.21.15.1:0/4200781783 shutdown_connections 2024-02-02T15:34:43.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.349+0000 7f0d1a6e4b80 1 --2- 172.21.15.1:0/4200781783 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5588a7276db0 0x5588a72791d0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:43.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.349+0000 7f0d1a6e4b80 1 --2- 172.21.15.1:0/4200781783 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588a7214860 0x5588a726df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:43.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.349+0000 7f0d1a6e4b80 1 --2- 172.21.15.1:0/4200781783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588a7209090 0x5588a7269e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:43.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.349+0000 7f0d1a6e4b80 1 --2- 172.21.15.1:0/4200781783 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5588a72099b0 0x5588a726a3d0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:43.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.349+0000 7f0d1a6e4b80 1 -- 172.21.15.1:0/4200781783 >> 172.21.15.1:0/4200781783 conn(0x5588a7168e30 msgr2=0x5588a7212cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:43.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.349+0000 7f0ced600700 0 client.17658 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:34:43.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.349+0000 7f0d1a6e4b80 1 -- 172.21.15.1:0/4200781783 shutdown_connections 2024-02-02T15:34:43.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.349+0000 7f0d1a6e4b80 1 -- 172.21.15.1:0/4200781783 wait complete. 2024-02-02T15:34:43.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.349+0000 7f0d1a6e4b80 20 client.17658 trim_cache size 0 max 16384 2024-02-02T15:34:43.352 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:34:43.352 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2024-02-02T15:34:43.352 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "9080a4be-fa36-4747-ae91-3ab38512cacd", 2024-02-02T15:34:43.352 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2024-02-02T15:34:43.352 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:34:43.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.350+0000 7f0d06c00700 1 -- 172.21.15.1:0/3320258062 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f0ce00a0690 msgr2=0x7f0ce00a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:43.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.350+0000 7f0d06c00700 1 --2- 172.21.15.1:0/3320258062 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f0ce00a0690 0x7f0ce00a2b50 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0cf0013650 tx=0x7f0cf0013a10 comp rx=0 tx=0).stop 2024-02-02T15:34:43.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.350+0000 7f0d06c00700 1 -- 172.21.15.1:0/3320258062 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d00073ac0 msgr2=0x7f0d001a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:43.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.350+0000 7f0d06c00700 1 --2- 172.21.15.1:0/3320258062 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d00073ac0 0x7f0d001a50f0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8007ee0 tx=0x7f0cf800d3b0 comp rx=0 tx=0).stop 2024-02-02T15:34:43.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.350+0000 7f0d06c00700 1 -- 172.21.15.1:0/3320258062 shutdown_connections 2024-02-02T15:34:43.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.350+0000 7f0d06c00700 1 --2- 172.21.15.1:0/3320258062 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f0ce00a0690 0x7f0ce00a2b50 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:43.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.350+0000 7f0d06c00700 1 --2- 172.21.15.1:0/3320258062 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0d00073ac0 0x7f0d001a50f0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:43.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.350+0000 7f0d06c00700 1 --2- 172.21.15.1:0/3320258062 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0d00100ff0 0x7f0d00073040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:43.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.350+0000 7f0d06c00700 1 --2- 172.21.15.1:0/3320258062 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0d00106110 0x7f0d00073580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:43.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.350+0000 7f0d06c00700 1 -- 172.21.15.1:0/3320258062 >> 172.21.15.1:0/3320258062 conn(0x7f0d000fc810 msgr2=0x7f0d000fd2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:43.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.350+0000 7f0d06c00700 1 -- 172.21.15.1:0/3320258062 shutdown_connections 2024-02-02T15:34:43.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.350+0000 7f0d06c00700 1 -- 172.21.15.1:0/3320258062 wait complete. 2024-02-02T15:34:43.369 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-02T15:34:43.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.577+0000 7fcb30200700 1 -- 172.21.15.1:0/3444115885 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb2810b400 msgr2=0x7fcb2810d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:43.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.577+0000 7fcb30200700 1 --2- 172.21.15.1:0/3444115885 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb2810b400 0x7fcb2810d850 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fcb2000b0a0 tx=0x7fcb2000b3b0 comp rx=0 tx=0).stop 2024-02-02T15:34:43.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.577+0000 7fcb30200700 1 -- 172.21.15.1:0/3444115885 shutdown_connections 2024-02-02T15:34:43.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.577+0000 7fcb30200700 1 --2- 172.21.15.1:0/3444115885 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb2810b400 0x7fcb2810d850 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:43.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.577+0000 7fcb30200700 1 --2- 172.21.15.1:0/3444115885 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb28102030 0x7fcb28102410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:43.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.577+0000 7fcb30200700 1 --2- 172.21.15.1:0/3444115885 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb28102950 0x7fcb2810ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:43.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.577+0000 7fcb30200700 1 -- 172.21.15.1:0/3444115885 >> 172.21.15.1:0/3444115885 conn(0x7fcb28076d70 msgr2=0x7fcb28077180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:43.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.577+0000 7fcb30200700 1 -- 172.21.15.1:0/3444115885 shutdown_connections 2024-02-02T15:34:43.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.577+0000 7fcb30200700 1 -- 172.21.15.1:0/3444115885 wait complete. 2024-02-02T15:34:43.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.578+0000 7fcb30200700 1 Processor -- start 2024-02-02T15:34:43.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.578+0000 7fcb30200700 1 -- start start 2024-02-02T15:34:43.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.579+0000 7fcb30200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb28102030 0x7fcb280ffc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:43.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.579+0000 7fcb30200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb28102950 0x7fcb281001c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:43.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.579+0000 7fcb30200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb2810b400 0x7fcb281a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:43.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.579+0000 7fcb30200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb281009f0 con 0x7fcb28102030 2024-02-02T15:34:43.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.579+0000 7fcb30200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb281a56c0 con 0x7fcb28102950 2024-02-02T15:34:43.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.579+0000 7fcb30200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb281a5800 con 0x7fcb2810b400 2024-02-02T15:34:43.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.579+0000 7fcb2d800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb28102030 0x7fcb280ffc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:43.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.579+0000 7fcb2d800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb28102030 0x7fcb280ffc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:34274/0 (socket says 172.21.15.1:34274) 2024-02-02T15:34:43.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.579+0000 7fcb2d800700 1 -- 172.21.15.1:0/2018711332 learned_addr learned my addr 172.21.15.1:0/2018711332 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:34:43.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.579+0000 7fcb2ce00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb28102950 0x7fcb281001c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:43.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.579+0000 7fcb2e200700 1 --2- 172.21.15.1:0/2018711332 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb2810b400 0x7fcb281a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:43.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.579+0000 7fcb2ce00700 1 -- 172.21.15.1:0/2018711332 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb2810b400 msgr2=0x7fcb281a5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:43.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.579+0000 7fcb2ce00700 1 --2- 172.21.15.1:0/2018711332 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb2810b400 0x7fcb281a5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:43.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.579+0000 7fcb2ce00700 1 -- 172.21.15.1:0/2018711332 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb28102030 msgr2=0x7fcb280ffc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:43.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.579+0000 7fcb2ce00700 1 --2- 172.21.15.1:0/2018711332 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb28102030 0x7fcb280ffc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:43.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.579+0000 7fcb2ce00700 1 -- 172.21.15.1:0/2018711332 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb20009d00 con 0x7fcb28102950 2024-02-02T15:34:43.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.580+0000 7fcb2ce00700 1 --2- 172.21.15.1:0/2018711332 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb28102950 0x7fcb281001c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fcb1000e9f0 tx=0x7fcb1000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:43.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.580+0000 7fcb26000700 1 -- 172.21.15.1:0/2018711332 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb1000c9c0 con 0x7fcb28102950 2024-02-02T15:34:43.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.580+0000 7fcb26000700 1 -- 172.21.15.1:0/2018711332 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb10010430 con 0x7fcb28102950 2024-02-02T15:34:43.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.580+0000 7fcb30200700 1 -- 172.21.15.1:0/2018711332 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb281a5a60 con 0x7fcb28102950 2024-02-02T15:34:43.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.580+0000 7fcb26000700 1 -- 172.21.15.1:0/2018711332 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb10018680 con 0x7fcb28102950 2024-02-02T15:34:43.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.580+0000 7fcb30200700 1 -- 172.21.15.1:0/2018711332 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb281a5f90 con 0x7fcb28102950 2024-02-02T15:34:43.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.585+0000 7fcb26000700 1 -- 172.21.15.1:0/2018711332 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7fcb10010860 con 0x7fcb28102950 2024-02-02T15:34:43.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.586+0000 7fcb43ddab80 1 Processor -- start 2024-02-02T15:34:43.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.586+0000 7fcb43ddab80 1 -- start start 2024-02-02T15:34:43.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.586+0000 7fcb26000700 1 --2- 172.21.15.1:0/2018711332 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fcb080a0710 0x7fcb080a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:43.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.586+0000 7fcb26000700 1 -- 172.21.15.1:0/2018711332 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fcb10014070 con 0x7fcb28102950 2024-02-02T15:34:43.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.586+0000 7fcb43ddab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc84af5010 0x55bc84af53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:43.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.586+0000 7fcb2e200700 1 --2- 172.21.15.1:0/2018711332 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fcb080a0710 0x7fcb080a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:43.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.586+0000 7fcb43ddab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc84af5930 0x55bc84b002a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:43.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.586+0000 7fcb43ddab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc84b007e0 0x55bc84b02bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:43.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.586+0000 7fcb43ddab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc84a694c0 con 0x55bc84b007e0 2024-02-02T15:34:43.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.586+0000 7fcb43ddab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc84a4daa0 con 0x55bc84af5930 2024-02-02T15:34:43.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.586+0000 7fcb43ddab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc84a4b460 con 0x55bc84af5010 2024-02-02T15:34:43.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.587+0000 7fcb2e200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc84b007e0 0x55bc84b02bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:43.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.587+0000 7fcb2ce00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc84af5930 0x55bc84b002a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:43.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.587+0000 7fcb2ce00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc84af5930 0x55bc84b002a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:43830/0 (socket says 172.21.15.1:43830) 2024-02-02T15:34:43.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.587+0000 7fcb2ce00700 1 -- 172.21.15.1:0/2082669337 learned_addr learned my addr 172.21.15.1:0/2082669337 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:34:43.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.587+0000 7fcb2e200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc84b007e0 0x55bc84b02bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:34288/0 (socket says 172.21.15.1:34288) 2024-02-02T15:34:43.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.587+0000 7fcb2d800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc84af5010 0x55bc84af53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:43.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.587+0000 7fcb2ce00700 1 -- 172.21.15.1:0/2082669337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc84af5010 msgr2=0x55bc84af53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:43.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.587+0000 7fcb2ce00700 1 --2- 172.21.15.1:0/2082669337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc84af5010 0x55bc84af53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:43.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.587+0000 7fcb2ce00700 1 -- 172.21.15.1:0/2082669337 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc84b007e0 msgr2=0x55bc84b02bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:43.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.587+0000 7fcb2ce00700 1 --2- 172.21.15.1:0/2082669337 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc84b007e0 0x55bc84b02bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:43.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.587+0000 7fcb2ce00700 1 -- 172.21.15.1:0/2082669337 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc848c83b0 con 0x55bc84af5930 2024-02-02T15:34:43.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.587+0000 7fcb2e200700 1 --2- 172.21.15.1:0/2082669337 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc84b007e0 0x55bc84b02bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:34:43.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.587+0000 7fcb2e200700 1 --2- 172.21.15.1:0/2018711332 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fcb080a0710 0x7fcb080a2bd0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fcb28101d90 tx=0x7fcb2000bcf0 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:43.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.587+0000 7fcb2ce00700 1 --2- 172.21.15.1:0/2082669337 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc84af5930 0x55bc84b002a0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fcb1000e9c0 tx=0x7fcb10098cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bad9c80df455cae6 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:43.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.587+0000 7fcb1ea00700 1 -- 172.21.15.1:0/2082669337 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb1009d0b0 con 0x55bc84af5930 2024-02-02T15:34:43.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.587+0000 7fcb1ea00700 1 -- 172.21.15.1:0/2082669337 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb10093e90 con 0x55bc84af5930 2024-02-02T15:34:43.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.588+0000 7fcb1ea00700 1 -- 172.21.15.1:0/2082669337 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb10098770 con 0x55bc84af5930 2024-02-02T15:34:43.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.588+0000 7fcb43ddab80 1 -- 172.21.15.1:0/2082669337 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc84af5930 msgr2=0x55bc84b002a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:43.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.588+0000 7fcb43ddab80 1 --2- 172.21.15.1:0/2082669337 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc84af5930 0x55bc84b002a0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fcb1000e9c0 tx=0x7fcb10098cd0 comp rx=0 tx=0).stop 2024-02-02T15:34:43.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.588+0000 7fcb43ddab80 1 -- 172.21.15.1:0/2082669337 shutdown_connections 2024-02-02T15:34:43.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.588+0000 7fcb43ddab80 1 --2- 172.21.15.1:0/2082669337 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc84b007e0 0x55bc84b02bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:43.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.588+0000 7fcb43ddab80 1 --2- 172.21.15.1:0/2082669337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc84af5010 0x55bc84af53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:43.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.588+0000 7fcb43ddab80 1 --2- 172.21.15.1:0/2082669337 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc84af5930 0x55bc84b002a0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:43.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.588+0000 7fcb43ddab80 1 -- 172.21.15.1:0/2082669337 >> 172.21.15.1:0/2082669337 conn(0x55bc84a54db0 msgr2=0x55bc84af13b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:43.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.588+0000 7fcb43ddab80 1 -- 172.21.15.1:0/2082669337 shutdown_connections 2024-02-02T15:34:43.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.588+0000 7fcb43ddab80 1 -- 172.21.15.1:0/2082669337 wait complete. 2024-02-02T15:34:43.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.589+0000 7fcb43ddab80 1 -- start start 2024-02-02T15:34:43.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.589+0000 7fcb43ddab80 1 Processor -- start 2024-02-02T15:34:43.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.589+0000 7fcb43ddab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc84af5010 0x55bc84b00200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:43.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.589+0000 7fcb43ddab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc84af5930 0x55bc84afd410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:43.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.589+0000 7fcb43ddab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc84b007e0 0x55bc84afda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:43.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.589+0000 7fcb43ddab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc84a46d50 con 0x55bc84af5930 2024-02-02T15:34:43.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.589+0000 7fcb43ddab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc849e2d80 con 0x55bc84af5010 2024-02-02T15:34:43.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.589+0000 7fcb43ddab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc849e2a20 con 0x55bc84b007e0 2024-02-02T15:34:43.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.590+0000 7fcb43ddab80 10 client.0 _ll_register_callbacks cb 0x55bc849e4290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:34:43.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.590+0000 7fcb2ce00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc84af5930 0x55bc84afd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:43.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.590+0000 7fcb2ce00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc84af5930 0x55bc84afd410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:34298/0 (socket says 172.21.15.1:34298) 2024-02-02T15:34:43.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.590+0000 7fcb2ce00700 1 -- 172.21.15.1:0/1929279105 learned_addr learned my addr 172.21.15.1:0/1929279105 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:34:43.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.590+0000 7fcb2e200700 1 --2- 172.21.15.1:0/1929279105 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc84b007e0 0x55bc84afda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:43.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.590+0000 7fcb2d800700 1 --2- 172.21.15.1:0/1929279105 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc84af5010 0x55bc84b00200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:43.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.590+0000 7fcb2ce00700 1 -- 172.21.15.1:0/1929279105 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc84b007e0 msgr2=0x55bc84afda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:43.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.590+0000 7fcb2ce00700 1 --2- 172.21.15.1:0/1929279105 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc84b007e0 0x55bc84afda40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:43.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.590+0000 7fcb2ce00700 1 -- 172.21.15.1:0/1929279105 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc84af5010 msgr2=0x55bc84b00200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:43.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.590+0000 7fcb2ce00700 1 --2- 172.21.15.1:0/1929279105 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc84af5010 0x55bc84b00200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:43.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.590+0000 7fcb2ce00700 1 -- 172.21.15.1:0/1929279105 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb281a5f90 con 0x55bc84af5930 2024-02-02T15:34:43.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.591+0000 7fcb2ce00700 1 --2- 172.21.15.1:0/1929279105 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc84af5930 0x55bc84afd410 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fcb10099210 tx=0x7fcb100107d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2f94c29ee0c98fa server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:43.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.591+0000 7fcb17400700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:34:43.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.591+0000 7fcb17400700 1 -- 172.21.15.1:0/1929279105 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb1009d0b0 con 0x55bc84af5930 2024-02-02T15:34:43.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.591+0000 7fcb17400700 1 -- 172.21.15.1:0/1929279105 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fcb100996f0 con 0x55bc84af5930 2024-02-02T15:34:43.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.591+0000 7fcb43ddab80 1 -- 172.21.15.1:0/1929279105 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bc84ae90f0 con 0x55bc84af5930 2024-02-02T15:34:43.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.591+0000 7fcb17400700 1 -- 172.21.15.1:0/1929279105 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb100a0e60 con 0x55bc84af5930 2024-02-02T15:34:43.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.591+0000 7fcb43ddab80 10 client.17670 fetch_fsmap learned FSMap version 42 2024-02-02T15:34:43.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.591+0000 7fcb43ddab80 1 -- 172.21.15.1:0/1929279105 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55bc84874b60 con 0x55bc84af5930 2024-02-02T15:34:43.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.591+0000 7fcb17400700 1 -- 172.21.15.1:0/1929279105 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bc84ae90f0 con 0x55bc84af5930 2024-02-02T15:34:43.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.591+0000 7fcb17400700 1 -- 172.21.15.1:0/1929279105 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 42) v1 ==== 7510+0+0 (secure 0 0 0) 0x7fcb100981e0 con 0x55bc84af5930 2024-02-02T15:34:43.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.592+0000 7fcb43ddab80 10 client.17670 fetch_fsmap finished waiting for FSMap version 42 2024-02-02T15:34:43.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.592+0000 7fcb43ddab80 10 client.17670 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:34:43.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.592+0000 7fcb43ddab80 20 client.17670 populate_metadata read hostname 'smithi001' 2024-02-02T15:34:43.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.592+0000 7fcb43ddab80 1 --2- 172.21.15.1:0/1929279105 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55bc84b0fd60 0x55bc84b12180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:43.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.592+0000 7fcb43ddab80 4 client.17670 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:34:43.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.592+0000 7fcb43ddab80 1 -- 172.21.15.1:0/1929279105 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bc84a3fdd0 con 0x55bc84b0fd60 2024-02-02T15:34:43.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.592+0000 7fcb2d800700 1 --2- 172.21.15.1:0/1929279105 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55bc84b0fd60 0x55bc84b12180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:43.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.592+0000 7fcb17400700 10 client.17670 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:34:43.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.592+0000 7fcb2d800700 1 --2- 172.21.15.1:0/1929279105 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55bc84b0fd60 0x55bc84b12180 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e951db1fd4ce3711 server_cookie=e1a56927dea35582 in_seq=0 out_seq=0 2024-02-02T15:34:43.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.593+0000 7fcb17400700 1 -- 172.21.15.1:0/1929279105 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55bc84a3fdd0 con 0x55bc84b0fd60 2024-02-02T15:34:43.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.593+0000 7fcb17400700 10 client.17670 handle_command_reply: tid=0 2024-02-02T15:34:43.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.593+0000 7fcb43ddab80 1 client.17670 shutdown 2024-02-02T15:34:43.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.593+0000 7fcb43ddab80 1 -- 172.21.15.1:0/1929279105 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc84af5930 msgr2=0x55bc84afd410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:43.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.593+0000 7fcb43ddab80 1 --2- 172.21.15.1:0/1929279105 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc84af5930 0x55bc84afd410 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fcb10099210 tx=0x7fcb100107d0 comp rx=0 tx=0).stop 2024-02-02T15:34:43.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.593+0000 7fcb43ddab80 1 -- 172.21.15.1:0/1929279105 shutdown_connections 2024-02-02T15:34:43.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.593+0000 7fcb43ddab80 1 --2- 172.21.15.1:0/1929279105 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55bc84b0fd60 0x55bc84b12180 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:43.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.593+0000 7fcb43ddab80 1 --2- 172.21.15.1:0/1929279105 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc84b007e0 0x55bc84afda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:43.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.594+0000 7fcb43ddab80 1 --2- 172.21.15.1:0/1929279105 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc84af5010 0x55bc84b00200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:43.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.594+0000 7fcb43ddab80 1 --2- 172.21.15.1:0/1929279105 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc84af5930 0x55bc84afd410 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:43.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.594+0000 7fcb17400700 0 client.17670 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:34:43.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.594+0000 7fcb43ddab80 1 -- 172.21.15.1:0/1929279105 >> 172.21.15.1:0/1929279105 conn(0x55bc84a54db0 msgr2=0x55bc84af0fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:43.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.594+0000 7fcb43ddab80 1 -- 172.21.15.1:0/1929279105 shutdown_connections 2024-02-02T15:34:43.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.594+0000 7fcb43ddab80 1 -- 172.21.15.1:0/1929279105 wait complete. 2024-02-02T15:34:43.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.594+0000 7fcb43ddab80 20 client.17670 trim_cache size 0 max 16384 2024-02-02T15:34:43.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.614+0000 7fcb43ddab80 1 Processor -- start 2024-02-02T15:34:43.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.614+0000 7fcb43ddab80 1 -- start start 2024-02-02T15:34:43.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.615+0000 7fcb43ddab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc84af5010 0x55bc84affd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:43.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.615+0000 7fcb43ddab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc84af5930 0x55bc84afcd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:43.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.615+0000 7fcb43ddab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc84b007e0 0x55bc84afd240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:43.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.615+0000 7fcb43ddab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc848ca6a0 con 0x55bc84af5010 2024-02-02T15:34:43.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.615+0000 7fcb43ddab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc84af9090 con 0x55bc84af5930 2024-02-02T15:34:43.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.615+0000 7fcb43ddab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc84afb9c0 con 0x55bc84b007e0 2024-02-02T15:34:43.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.615+0000 7fcb2ce00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc84af5930 0x55bc84afcd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:43.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.615+0000 7fcb2d800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc84af5010 0x55bc84affd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:43.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.615+0000 7fcb2ce00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc84af5930 0x55bc84afcd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:43838/0 (socket says 172.21.15.1:43838) 2024-02-02T15:34:43.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.615+0000 7fcb2ce00700 1 -- 172.21.15.1:0/2108968568 learned_addr learned my addr 172.21.15.1:0/2108968568 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:34:43.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.615+0000 7fcb2e200700 1 --2- 172.21.15.1:0/2108968568 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc84b007e0 0x55bc84afd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:43.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.615+0000 7fcb2ce00700 1 -- 172.21.15.1:0/2108968568 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc84b007e0 msgr2=0x55bc84afd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:43.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.615+0000 7fcb2ce00700 1 --2- 172.21.15.1:0/2108968568 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc84b007e0 0x55bc84afd240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:43.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.615+0000 7fcb2ce00700 1 -- 172.21.15.1:0/2108968568 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc84af5010 msgr2=0x55bc84affd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:43.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.615+0000 7fcb2ce00700 1 --2- 172.21.15.1:0/2108968568 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc84af5010 0x55bc84affd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:43.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.615+0000 7fcb2ce00700 1 -- 172.21.15.1:0/2108968568 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc84a0cea0 con 0x55bc84af5930 2024-02-02T15:34:43.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.616+0000 7fcb2ce00700 1 --2- 172.21.15.1:0/2108968568 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc84af5930 0x55bc84afcd00 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fcb1000c5b0 tx=0x7fcb100956f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fffc19067fb16817 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:43.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.616+0000 7fcb1f400700 1 -- 172.21.15.1:0/2108968568 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb100cbe50 con 0x55bc84af5930 2024-02-02T15:34:43.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.616+0000 7fcb1f400700 1 -- 172.21.15.1:0/2108968568 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb1009d0b0 con 0x55bc84af5930 2024-02-02T15:34:43.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.616+0000 7fcb1f400700 1 -- 172.21.15.1:0/2108968568 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb100a10c0 con 0x55bc84af5930 2024-02-02T15:34:43.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.616+0000 7fcb43ddab80 1 -- 172.21.15.1:0/2108968568 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc84af5930 msgr2=0x55bc84afcd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:43.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.617+0000 7fcb43ddab80 1 --2- 172.21.15.1:0/2108968568 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc84af5930 0x55bc84afcd00 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fcb1000c5b0 tx=0x7fcb100956f0 comp rx=0 tx=0).stop 2024-02-02T15:34:43.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.617+0000 7fcb43ddab80 1 -- 172.21.15.1:0/2108968568 shutdown_connections 2024-02-02T15:34:43.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.617+0000 7fcb43ddab80 1 --2- 172.21.15.1:0/2108968568 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc84b007e0 0x55bc84afd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:43.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.617+0000 7fcb43ddab80 1 --2- 172.21.15.1:0/2108968568 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc84af5010 0x55bc84affd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:43.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.617+0000 7fcb43ddab80 1 --2- 172.21.15.1:0/2108968568 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc84af5930 0x55bc84afcd00 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:43.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.617+0000 7fcb43ddab80 1 -- 172.21.15.1:0/2108968568 >> 172.21.15.1:0/2108968568 conn(0x55bc84a54db0 msgr2=0x55bc84b0b840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:43.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.617+0000 7fcb43ddab80 1 -- 172.21.15.1:0/2108968568 shutdown_connections 2024-02-02T15:34:43.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.617+0000 7fcb43ddab80 1 -- 172.21.15.1:0/2108968568 wait complete. 2024-02-02T15:34:43.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.617+0000 7fcb43ddab80 1 -- start start 2024-02-02T15:34:43.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.617+0000 7fcb43ddab80 1 Processor -- start 2024-02-02T15:34:43.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.618+0000 7fcb43ddab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc84af5010 0x55bc84b06d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:43.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.618+0000 7fcb43ddab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc84af5930 0x55bc84b072b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:43.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.618+0000 7fcb43ddab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc84b007e0 0x55bc84b052a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:43.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.618+0000 7fcb43ddab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc84b05980 con 0x55bc84af5010 2024-02-02T15:34:43.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.618+0000 7fcb43ddab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc84b05af0 con 0x55bc84b007e0 2024-02-02T15:34:43.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.618+0000 7fcb43ddab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc84b05cb0 con 0x55bc84af5930 2024-02-02T15:34:43.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.618+0000 7fcb43ddab80 10 client.0 _ll_register_callbacks cb 0x55bc849e4290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:34:43.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.618+0000 7fcb2d800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc84af5010 0x55bc84b06d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:43.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.618+0000 7fcb2d800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc84af5010 0x55bc84b06d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:34318/0 (socket says 172.21.15.1:34318) 2024-02-02T15:34:43.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.618+0000 7fcb2e200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc84b007e0 0x55bc84b052a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:43.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.618+0000 7fcb2d800700 1 -- 172.21.15.1:0/3890439785 learned_addr learned my addr 172.21.15.1:0/3890439785 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:34:43.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.619+0000 7fcb2ce00700 1 --2- 172.21.15.1:0/3890439785 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc84af5930 0x55bc84b072b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:43.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.619+0000 7fcb2d800700 1 -- 172.21.15.1:0/3890439785 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc84af5930 msgr2=0x55bc84b072b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:43.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.619+0000 7fcb2d800700 1 --2- 172.21.15.1:0/3890439785 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc84af5930 0x55bc84b072b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:43.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.619+0000 7fcb2d800700 1 -- 172.21.15.1:0/3890439785 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc84b007e0 msgr2=0x55bc84b052a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:34:43.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.619+0000 7fcb2d800700 1 --2- 172.21.15.1:0/3890439785 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc84b007e0 0x55bc84b052a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:43.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.619+0000 7fcb2d800700 1 -- 172.21.15.1:0/3890439785 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bc84874b60 con 0x55bc84af5010 2024-02-02T15:34:43.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.619+0000 7fcb2ce00700 1 --2- 172.21.15.1:0/3890439785 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc84af5930 0x55bc84b072b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:34:43.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.619+0000 7fcb2e200700 1 --2- 172.21.15.1:0/3890439785 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc84b007e0 0x55bc84b052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:34:43.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.619+0000 7fcb2d800700 1 --2- 172.21.15.1:0/3890439785 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc84af5010 0x55bc84b06d70 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x55bc84b129f0 tx=0x7fcb180137a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6609c0f87cbc2d5 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:34:43.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.619+0000 7fcb17400700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:34:43.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.619+0000 7fcb17400700 1 -- 172.21.15.1:0/3890439785 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb18017070 con 0x55bc84af5010 2024-02-02T15:34:43.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.619+0000 7fcb17400700 1 -- 172.21.15.1:0/3890439785 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fcb18021070 con 0x55bc84af5010 2024-02-02T15:34:43.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.619+0000 7fcb43ddab80 1 -- 172.21.15.1:0/3890439785 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcb10093e90 con 0x55bc84af5010 2024-02-02T15:34:43.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.620+0000 7fcb17400700 1 -- 172.21.15.1:0/3890439785 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb18021230 con 0x55bc84af5010 2024-02-02T15:34:43.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.620+0000 7fcb43ddab80 10 client.17676 fetch_fsmap learned FSMap version 42 2024-02-02T15:34:43.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.620+0000 7fcb43ddab80 1 -- 172.21.15.1:0/3890439785 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55bc848cccd0 con 0x55bc84af5010 2024-02-02T15:34:43.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.620+0000 7fcb17400700 1 -- 172.21.15.1:0/3890439785 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcb10093e90 con 0x55bc84af5010 2024-02-02T15:34:43.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.620+0000 7fcb17400700 1 -- 172.21.15.1:0/3890439785 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 42) v1 ==== 7510+0+0 (secure 0 0 0) 0x7fcb18010810 con 0x55bc84af5010 2024-02-02T15:34:43.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.620+0000 7fcb43ddab80 10 client.17676 fetch_fsmap finished waiting for FSMap version 42 2024-02-02T15:34:43.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.620+0000 7fcb43ddab80 10 client.17676 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:34:43.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.620+0000 7fcb43ddab80 20 client.17676 populate_metadata read hostname 'smithi001' 2024-02-02T15:34:43.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.620+0000 7fcb43ddab80 1 --2- 172.21.15.1:0/3890439785 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55bc84b62c00 0x55bc84b65020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:34:43.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.620+0000 7fcb43ddab80 4 client.17676 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2024-02-02T15:34:43.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.620+0000 7fcb43ddab80 1 -- 172.21.15.1:0/3890439785 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55bc84a05f50 con 0x55bc84b62c00 2024-02-02T15:34:43.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.621+0000 7fcb2ce00700 1 --2- 172.21.15.1:0/3890439785 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55bc84b62c00 0x55bc84b65020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:34:43.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.621+0000 7fcb2ce00700 1 --2- 172.21.15.1:0/3890439785 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55bc84b62c00 0x55bc84b65020 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c685b208ccc32c14 server_cookie=6e1052672ae43fea in_seq=0 out_seq=0 2024-02-02T15:34:43.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.621+0000 7fcb17400700 10 client.17676 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:34:43.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.621+0000 7fcb17400700 1 -- 172.21.15.1:0/3890439785 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55bc84a05f50 con 0x55bc84b62c00 2024-02-02T15:34:43.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.621+0000 7fcb17400700 10 client.17676 handle_command_reply: tid=0 2024-02-02T15:34:43.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.621+0000 7fcb43ddab80 1 client.17676 shutdown 2024-02-02T15:34:43.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.621+0000 7fcb43ddab80 1 -- 172.21.15.1:0/3890439785 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc84af5010 msgr2=0x55bc84b06d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:43.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.621+0000 7fcb43ddab80 1 --2- 172.21.15.1:0/3890439785 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc84af5010 0x55bc84b06d70 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x55bc84b129f0 tx=0x7fcb180137a0 comp rx=0 tx=0).stop 2024-02-02T15:34:43.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.621+0000 7fcb43ddab80 1 -- 172.21.15.1:0/3890439785 shutdown_connections 2024-02-02T15:34:43.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.621+0000 7fcb43ddab80 1 --2- 172.21.15.1:0/3890439785 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55bc84b62c00 0x55bc84b65020 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:43.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.621+0000 7fcb43ddab80 1 --2- 172.21.15.1:0/3890439785 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bc84b007e0 0x55bc84b052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:43.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.621+0000 7fcb43ddab80 1 --2- 172.21.15.1:0/3890439785 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bc84af5010 0x55bc84b06d70 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:43.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.621+0000 7fcb43ddab80 1 --2- 172.21.15.1:0/3890439785 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bc84af5930 0x55bc84b072b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:43.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.621+0000 7fcb17400700 0 client.17676 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:34:43.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.621+0000 7fcb43ddab80 1 -- 172.21.15.1:0/3890439785 >> 172.21.15.1:0/3890439785 conn(0x55bc84a54db0 msgr2=0x55bc84afcd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:43.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.622+0000 7fcb43ddab80 1 -- 172.21.15.1:0/3890439785 shutdown_connections 2024-02-02T15:34:43.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.622+0000 7fcb43ddab80 1 -- 172.21.15.1:0/3890439785 wait complete. 2024-02-02T15:34:43.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.622+0000 7fcb43ddab80 20 client.17676 trim_cache size 0 max 16384 2024-02-02T15:34:43.626 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:34:43.626 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-02T15:34:43.626 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2024-02-02T15:34:43.626 INFO:teuthology.orchestra.run.smithi001.stdout: "9080a4be-fa36-4747-ae91-3ab38512cacd": { 2024-02-02T15:34:43.627 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2024-02-02T15:34:43.627 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "9080a4be-fa36-4747-ae91-3ab38512cacd", 2024-02-02T15:34:43.627 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,force" 2024-02-02T15:34:43.627 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:34:43.627 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:34:43.627 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:34:43.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.622+0000 7fcb30200700 1 -- 172.21.15.1:0/2018711332 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fcb080a0710 msgr2=0x7fcb080a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:43.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.622+0000 7fcb30200700 1 --2- 172.21.15.1:0/2018711332 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fcb080a0710 0x7fcb080a2bd0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fcb28101d90 tx=0x7fcb2000bcf0 comp rx=0 tx=0).stop 2024-02-02T15:34:43.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.622+0000 7fcb30200700 1 -- 172.21.15.1:0/2018711332 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb28102950 msgr2=0x7fcb281001c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:34:43.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.622+0000 7fcb30200700 1 --2- 172.21.15.1:0/2018711332 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb28102950 0x7fcb281001c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fcb1000e9f0 tx=0x7fcb1000ed00 comp rx=0 tx=0).stop 2024-02-02T15:34:43.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.622+0000 7fcb30200700 1 -- 172.21.15.1:0/2018711332 shutdown_connections 2024-02-02T15:34:43.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.622+0000 7fcb30200700 1 --2- 172.21.15.1:0/2018711332 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fcb080a0710 0x7fcb080a2bd0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:43.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.622+0000 7fcb30200700 1 --2- 172.21.15.1:0/2018711332 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcb2810b400 0x7fcb281a5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:43.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.622+0000 7fcb30200700 1 --2- 172.21.15.1:0/2018711332 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcb28102030 0x7fcb280ffc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:43.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.622+0000 7fcb30200700 1 --2- 172.21.15.1:0/2018711332 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb28102950 0x7fcb281001c0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:34:43.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.622+0000 7fcb30200700 1 -- 172.21.15.1:0/2018711332 >> 172.21.15.1:0/2018711332 conn(0x7fcb28076d70 msgr2=0x7fcb280fcb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:34:43.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.623+0000 7fcb30200700 1 -- 172.21.15.1:0/2018711332 shutdown_connections 2024-02-02T15:34:43.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:34:43.623+0000 7fcb30200700 1 -- 172.21.15.1:0/2018711332 wait complete. 2024-02-02T15:34:43.641 INFO:tasks.cephfs.filesystem:scrub status for tag:9080a4be-fa36-4747-ae91-3ab38512cacd - {'path': '/', 'tag': '9080a4be-fa36-4747-ae91-3ab38512cacd', 'options': 'recursive,force'} 2024-02-02T15:34:43.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:43 smithi001 ceph-mon[35470]: pgmap v236: 129 pgs: 129 active+clean; 310 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 178 op/s 2024-02-02T15:34:43.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:43 smithi071 ceph-mon[44707]: pgmap v236: 129 pgs: 129 active+clean; 310 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 178 op/s 2024-02-02T15:34:43.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:43 smithi142 ceph-mon[45728]: pgmap v236: 129 pgs: 129 active+clean; 310 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.9 MiB/s wr, 178 op/s 2024-02-02T15:34:44.246 INFO:tasks.workunit.client.0.smithi001.stdout: 1 464461 6.83 MB/sec execute 258 sec latency 781.699 ms 2024-02-02T15:34:44.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:44 smithi071 ceph-mon[44707]: scrub queued for path: / 2024-02-02T15:34:44.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:44 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:34:44.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:44 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:34:44.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:44 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:34:44.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:44 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:34:44.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:44 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:34:44.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:44 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:34:44.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:44 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:34:44.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:44 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:34:44.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:44 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:34:44.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:44 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:34:44.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:44 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:34:44.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:44 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:34:44.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:44 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:34:44.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:44 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:34:44.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:44 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:34:44.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:44 smithi071 ceph-mon[44707]: pgmap v237: 129 pgs: 129 active+clean; 310 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 135 op/s 2024-02-02T15:34:44.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:44 smithi142 ceph-mon[45728]: scrub queued for path: / 2024-02-02T15:34:44.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:44 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:34:44.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:44 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:34:44.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:44 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:34:44.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:44 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:34:44.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:44 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:34:44.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:44 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:34:44.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:44 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:34:44.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:44 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:34:44.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:44 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:34:44.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:44 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:34:44.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:44 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:34:44.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:44 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:34:44.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:44 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:34:44.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:44 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:34:44.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:44 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:34:44.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:44 smithi142 ceph-mon[45728]: pgmap v237: 129 pgs: 129 active+clean; 310 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 135 op/s 2024-02-02T15:34:44.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:44 smithi001 ceph-mon[35470]: scrub queued for path: / 2024-02-02T15:34:44.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:44 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:34:44.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:44 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:34:44.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:44 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:34:44.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:44 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:34:44.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:44 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:34:44.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:44 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:34:44.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:44 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:34:44.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:44 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:34:44.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:44 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:34:44.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:44 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:34:44.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:44 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:34:44.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:44 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:34:44.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:44 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:34:44.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:44 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:34:44.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:44 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:34:44.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:44 smithi001 ceph-mon[35470]: pgmap v237: 129 pgs: 129 active+clean; 310 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.5 MiB/s wr, 135 op/s 2024-02-02T15:34:45.246 INFO:tasks.workunit.client.0.smithi001.stdout: 1 464461 6.80 MB/sec execute 259 sec latency 1781.809 ms 2024-02-02T15:34:46.246 INFO:tasks.workunit.client.0.smithi001.stdout: 1 464461 6.78 MB/sec execute 260 sec latency 2781.946 ms 2024-02-02T15:34:47.247 INFO:tasks.workunit.client.0.smithi001.stdout: 1 465790 6.79 MB/sec execute 261 sec latency 2998.330 ms 2024-02-02T15:34:47.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:47 smithi142 ceph-mon[45728]: pgmap v238: 129 pgs: 129 active+clean; 309 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 164 op/s 2024-02-02T15:34:47.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:47 smithi001 ceph-mon[35470]: pgmap v238: 129 pgs: 129 active+clean; 309 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 164 op/s 2024-02-02T15:34:47.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:47 smithi071 ceph-mon[44707]: pgmap v238: 129 pgs: 129 active+clean; 309 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 164 op/s 2024-02-02T15:34:48.247 INFO:tasks.workunit.client.0.smithi001.stdout: 1 467133 6.79 MB/sec execute 262 sec latency 16.573 ms 2024-02-02T15:34:49.247 INFO:tasks.workunit.client.0.smithi001.stdout: 1 467138 6.76 MB/sec execute 263 sec latency 999.978 ms 2024-02-02T15:34:49.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:49 smithi142 ceph-mon[45728]: pgmap v239: 129 pgs: 129 active+clean; 309 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 1013 KiB/s wr, 107 op/s 2024-02-02T15:34:49.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:49 smithi001 ceph-mon[35470]: pgmap v239: 129 pgs: 129 active+clean; 309 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 1013 KiB/s wr, 107 op/s 2024-02-02T15:34:49.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:49 smithi071 ceph-mon[44707]: pgmap v239: 129 pgs: 129 active+clean; 309 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 1013 KiB/s wr, 107 op/s 2024-02-02T15:34:50.247 INFO:tasks.workunit.client.0.smithi001.stdout: 1 468764 6.77 MB/sec execute 264 sec latency 1297.921 ms 2024-02-02T15:34:51.247 INFO:tasks.workunit.client.0.smithi001.stdout: 1 469905 6.77 MB/sec execute 265 sec latency 12.371 ms 2024-02-02T15:34:51.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:51 smithi142 ceph-mon[45728]: pgmap v240: 129 pgs: 129 active+clean; 309 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 1013 KiB/s wr, 107 op/s 2024-02-02T15:34:51.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:51 smithi001 ceph-mon[35470]: pgmap v240: 129 pgs: 129 active+clean; 309 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 1013 KiB/s wr, 107 op/s 2024-02-02T15:34:51.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:51 smithi071 ceph-mon[44707]: pgmap v240: 129 pgs: 129 active+clean; 309 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 1013 KiB/s wr, 107 op/s 2024-02-02T15:34:52.247 INFO:tasks.workunit.client.0.smithi001.stdout: 1 471332 6.76 MB/sec execute 266 sec latency 242.878 ms 2024-02-02T15:34:52.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:52 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:34:52.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:52 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:34:52.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:52 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:34:52.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:52 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:34:52.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:52 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:34:52.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:52 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:34:52.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:52 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:34:52.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:52 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:34:52.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:52 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:34:52.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:52 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:34:52.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:52 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:34:52.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:52 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:34:52.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:52 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:34:52.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:52 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:34:52.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:52 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:34:53.247 INFO:tasks.workunit.client.0.smithi001.stdout: 1 471332 6.74 MB/sec execute 267 sec latency 1242.998 ms 2024-02-02T15:34:53.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:53 smithi142 ceph-mon[45728]: pgmap v241: 129 pgs: 129 active+clean; 313 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 954 KiB/s rd, 1.5 MiB/s wr, 170 op/s 2024-02-02T15:34:53.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:53 smithi001 ceph-mon[35470]: pgmap v241: 129 pgs: 129 active+clean; 313 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 954 KiB/s rd, 1.5 MiB/s wr, 170 op/s 2024-02-02T15:34:53.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:53 smithi071 ceph-mon[44707]: pgmap v241: 129 pgs: 129 active+clean; 313 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 954 KiB/s rd, 1.5 MiB/s wr, 170 op/s 2024-02-02T15:34:54.247 INFO:tasks.workunit.client.0.smithi001.stdout: 1 471332 6.71 MB/sec execute 268 sec latency 2243.147 ms 2024-02-02T15:34:55.248 INFO:tasks.workunit.client.0.smithi001.stdout: 1 471332 6.69 MB/sec execute 269 sec latency 3243.249 ms 2024-02-02T15:34:55.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:55 smithi142 ceph-mon[45728]: pgmap v242: 129 pgs: 129 active+clean; 313 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 683 KiB/s wr, 92 op/s 2024-02-02T15:34:55.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:55 smithi001 ceph-mon[35470]: pgmap v242: 129 pgs: 129 active+clean; 313 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 683 KiB/s wr, 92 op/s 2024-02-02T15:34:55.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:55 smithi071 ceph-mon[44707]: pgmap v242: 129 pgs: 129 active+clean; 313 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 683 KiB/s wr, 92 op/s 2024-02-02T15:34:56.248 INFO:tasks.workunit.client.0.smithi001.stdout: 1 471386 6.66 MB/sec execute 270 sec latency 3914.426 ms 2024-02-02T15:34:57.248 INFO:tasks.workunit.client.0.smithi001.stdout: 1 471386 6.64 MB/sec execute 271 sec latency 1287.093 ms 2024-02-02T15:34:57.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:57 smithi142 ceph-mon[45728]: pgmap v243: 129 pgs: 129 active+clean; 315 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 839 KiB/s wr, 116 op/s 2024-02-02T15:34:57.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:57 smithi001 ceph-mon[35470]: pgmap v243: 129 pgs: 129 active+clean; 315 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 839 KiB/s wr, 116 op/s 2024-02-02T15:34:57.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:57 smithi071 ceph-mon[44707]: pgmap v243: 129 pgs: 129 active+clean; 315 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 839 KiB/s wr, 116 op/s 2024-02-02T15:34:58.248 INFO:tasks.workunit.client.0.smithi001.stdout: 1 471386 6.61 MB/sec execute 272 sec latency 2287.179 ms 2024-02-02T15:34:59.248 INFO:tasks.workunit.client.0.smithi001.stdout: 1 471386 6.59 MB/sec execute 273 sec latency 3287.257 ms 2024-02-02T15:34:59.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:34:59 smithi142 ceph-mon[45728]: pgmap v244: 129 pgs: 129 active+clean; 315 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 676 KiB/s wr, 87 op/s 2024-02-02T15:34:59.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:34:59 smithi001 ceph-mon[35470]: pgmap v244: 129 pgs: 129 active+clean; 315 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 676 KiB/s wr, 87 op/s 2024-02-02T15:34:59.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:34:59 smithi071 ceph-mon[44707]: pgmap v244: 129 pgs: 129 active+clean; 315 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 676 KiB/s wr, 87 op/s 2024-02-02T15:35:00.248 INFO:tasks.workunit.client.0.smithi001.stdout: 1 471386 6.57 MB/sec execute 274 sec latency 4287.395 ms 2024-02-02T15:35:01.248 INFO:tasks.workunit.client.0.smithi001.stdout: 1 471451 6.54 MB/sec execute 275 sec latency 5040.432 ms 2024-02-02T15:35:01.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:01 smithi142 ceph-mon[45728]: pgmap v245: 129 pgs: 129 active+clean; 315 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 676 KiB/s wr, 87 op/s 2024-02-02T15:35:01.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:01 smithi001 ceph-mon[35470]: pgmap v245: 129 pgs: 129 active+clean; 315 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 676 KiB/s wr, 87 op/s 2024-02-02T15:35:01.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:01 smithi071 ceph-mon[44707]: pgmap v245: 129 pgs: 129 active+clean; 315 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 676 KiB/s wr, 87 op/s 2024-02-02T15:35:02.248 INFO:tasks.workunit.client.0.smithi001.stdout: 1 471451 6.52 MB/sec execute 276 sec latency 1129.160 ms 2024-02-02T15:35:03.249 INFO:tasks.workunit.client.0.smithi001.stdout: 1 471451 6.50 MB/sec execute 277 sec latency 2129.314 ms 2024-02-02T15:35:03.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:03 smithi142 ceph-mon[45728]: pgmap v246: 129 pgs: 129 active+clean; 341 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 2.9 MiB/s wr, 128 op/s 2024-02-02T15:35:03.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:03 smithi142 ceph-mon[45728]: osdmap e92: 12 total, 12 up, 12 in 2024-02-02T15:35:03.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:03 smithi001 ceph-mon[35470]: pgmap v246: 129 pgs: 129 active+clean; 341 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 2.9 MiB/s wr, 128 op/s 2024-02-02T15:35:03.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:03 smithi001 ceph-mon[35470]: osdmap e92: 12 total, 12 up, 12 in 2024-02-02T15:35:03.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:03 smithi071 ceph-mon[44707]: pgmap v246: 129 pgs: 129 active+clean; 341 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 535 KiB/s rd, 2.9 MiB/s wr, 128 op/s 2024-02-02T15:35:03.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:03 smithi071 ceph-mon[44707]: osdmap e92: 12 total, 12 up, 12 in 2024-02-02T15:35:04.249 INFO:tasks.workunit.client.0.smithi001.stdout: 1 471451 6.47 MB/sec execute 278 sec latency 3129.487 ms 2024-02-02T15:35:05.249 INFO:tasks.workunit.client.0.smithi001.stdout: 1 471451 6.45 MB/sec execute 279 sec latency 4129.588 ms 2024-02-02T15:35:05.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:05 smithi142 ceph-mon[45728]: pgmap v248: 129 pgs: 129 active+clean; 341 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 2.9 MiB/s wr, 77 op/s 2024-02-02T15:35:05.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:05 smithi001 ceph-mon[35470]: pgmap v248: 129 pgs: 129 active+clean; 341 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 2.9 MiB/s wr, 77 op/s 2024-02-02T15:35:05.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:05 smithi071 ceph-mon[44707]: pgmap v248: 129 pgs: 129 active+clean; 341 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 2.9 MiB/s wr, 77 op/s 2024-02-02T15:35:06.249 INFO:tasks.workunit.client.0.smithi001.stdout: 1 471890 6.44 MB/sec execute 280 sec latency 4799.652 ms 2024-02-02T15:35:07.249 INFO:tasks.workunit.client.0.smithi001.stdout: 1 472345 6.43 MB/sec execute 281 sec latency 801.485 ms 2024-02-02T15:35:07.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:07 smithi142 ceph-mon[45728]: pgmap v249: 129 pgs: 129 active+clean; 342 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 3.7 MiB/s wr, 90 op/s 2024-02-02T15:35:07.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:07 smithi001 ceph-mon[35470]: pgmap v249: 129 pgs: 129 active+clean; 342 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 3.7 MiB/s wr, 90 op/s 2024-02-02T15:35:07.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:07 smithi071 ceph-mon[44707]: pgmap v249: 129 pgs: 129 active+clean; 342 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 3.7 MiB/s wr, 90 op/s 2024-02-02T15:35:08.249 INFO:tasks.workunit.client.0.smithi001.stdout: 1 472345 6.41 MB/sec execute 282 sec latency 1801.623 ms 2024-02-02T15:35:09.249 INFO:tasks.workunit.client.0.smithi001.stdout: 1 472345 6.38 MB/sec execute 283 sec latency 2801.727 ms 2024-02-02T15:35:09.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:09 smithi142 ceph-mon[45728]: pgmap v250: 129 pgs: 129 active+clean; 342 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 3.7 MiB/s wr, 90 op/s 2024-02-02T15:35:09.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:09 smithi001 ceph-mon[35470]: pgmap v250: 129 pgs: 129 active+clean; 342 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 3.7 MiB/s wr, 90 op/s 2024-02-02T15:35:09.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:09 smithi071 ceph-mon[44707]: pgmap v250: 129 pgs: 129 active+clean; 342 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 3.7 MiB/s wr, 90 op/s 2024-02-02T15:35:10.250 INFO:tasks.workunit.client.0.smithi001.stdout: 1 473011 6.38 MB/sec execute 284 sec latency 3094.749 ms 2024-02-02T15:35:11.249 INFO:tasks.workunit.client.0.smithi001.stdout: 1 474630 6.38 MB/sec execute 285 sec latency 19.527 ms 2024-02-02T15:35:11.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:11 smithi142 ceph-mon[45728]: pgmap v251: 129 pgs: 129 active+clean; 342 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 3.7 MiB/s wr, 90 op/s 2024-02-02T15:35:11.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:11 smithi142 ceph-mon[45728]: osdmap e93: 12 total, 12 up, 12 in 2024-02-02T15:35:11.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:11 smithi001 ceph-mon[35470]: pgmap v251: 129 pgs: 129 active+clean; 342 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 3.7 MiB/s wr, 90 op/s 2024-02-02T15:35:11.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:11 smithi001 ceph-mon[35470]: osdmap e93: 12 total, 12 up, 12 in 2024-02-02T15:35:11.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:11 smithi071 ceph-mon[44707]: pgmap v251: 129 pgs: 129 active+clean; 342 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 3.7 MiB/s wr, 90 op/s 2024-02-02T15:35:11.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:11 smithi071 ceph-mon[44707]: osdmap e93: 12 total, 12 up, 12 in 2024-02-02T15:35:12.249 INFO:tasks.workunit.client.0.smithi001.stdout: 1 476371 6.40 MB/sec execute 286 sec latency 13.597 ms 2024-02-02T15:35:13.250 INFO:tasks.workunit.client.0.smithi001.stdout: 1 477541 6.40 MB/sec execute 287 sec latency 16.753 ms 2024-02-02T15:35:13.642 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-02T15:35:13.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:13 smithi142 ceph-mon[45728]: pgmap v253: 129 pgs: 129 active+clean; 335 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 1.4 MiB/s wr, 111 op/s 2024-02-02T15:35:13.692 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:13 smithi001 ceph-mon[35470]: pgmap v253: 129 pgs: 129 active+clean; 335 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 1.4 MiB/s wr, 111 op/s 2024-02-02T15:35:13.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:13 smithi071 ceph-mon[44707]: pgmap v253: 129 pgs: 129 active+clean; 335 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 1.4 MiB/s wr, 111 op/s 2024-02-02T15:35:13.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.902+0000 7fb09ea00700 1 -- 172.21.15.1:0/4142165787 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb098106aa0 msgr2=0x7fb098106e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:13.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.902+0000 7fb09ea00700 1 --2- 172.21.15.1:0/4142165787 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb098106aa0 0x7fb098106e80 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fb088009990 tx=0x7fb088009ca0 comp rx=0 tx=0).stop 2024-02-02T15:35:13.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.903+0000 7fb09ea00700 1 -- 172.21.15.1:0/4142165787 shutdown_connections 2024-02-02T15:35:13.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.903+0000 7fb09ea00700 1 --2- 172.21.15.1:0/4142165787 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb098101450 0x7fb09810d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:13.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.903+0000 7fb09ea00700 1 --2- 172.21.15.1:0/4142165787 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb098106aa0 0x7fb098106e80 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:13.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.903+0000 7fb09ea00700 1 --2- 172.21.15.1:0/4142165787 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb098100a90 0x7fb098100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:13.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.903+0000 7fb09ea00700 1 -- 172.21.15.1:0/4142165787 >> 172.21.15.1:0/4142165787 conn(0x7fb0980fc560 msgr2=0x7fb0980fe980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:13.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.903+0000 7fb09ea00700 1 -- 172.21.15.1:0/4142165787 shutdown_connections 2024-02-02T15:35:13.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.903+0000 7fb09ea00700 1 -- 172.21.15.1:0/4142165787 wait complete. 2024-02-02T15:35:13.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.904+0000 7fb09ea00700 1 Processor -- start 2024-02-02T15:35:13.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.904+0000 7fb09ea00700 1 -- start start 2024-02-02T15:35:13.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.904+0000 7fb09ea00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb098100a90 0x7fb09819c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:13.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.904+0000 7fb09ea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb098101450 0x7fb09819ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:13.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.904+0000 7fb09ea00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb098106aa0 0x7fb0981969b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:13.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.904+0000 7fb09ea00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb098197020 con 0x7fb098100a90 2024-02-02T15:35:13.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.904+0000 7fb09ea00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb098197190 con 0x7fb098106aa0 2024-02-02T15:35:13.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.904+0000 7fb09ea00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb098197350 con 0x7fb098101450 2024-02-02T15:35:13.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.905+0000 7fb09e000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb098106aa0 0x7fb0981969b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:13.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.905+0000 7fb09d600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb098100a90 0x7fb09819c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:13.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.905+0000 7fb09cc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb098101450 0x7fb09819ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:13.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.905+0000 7fb09e000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb098106aa0 0x7fb0981969b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:54726/0 (socket says 172.21.15.1:54726) 2024-02-02T15:35:13.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.905+0000 7fb09e000700 1 -- 172.21.15.1:0/2108170147 learned_addr learned my addr 172.21.15.1:0/2108170147 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:35:13.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.905+0000 7fb09e000700 1 -- 172.21.15.1:0/2108170147 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb098101450 msgr2=0x7fb09819ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:13.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.905+0000 7fb09e000700 1 --2- 172.21.15.1:0/2108170147 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb098101450 0x7fb09819ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:13.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.905+0000 7fb09e000700 1 -- 172.21.15.1:0/2108170147 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb098100a90 msgr2=0x7fb09819c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:13.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.905+0000 7fb09e000700 1 --2- 172.21.15.1:0/2108170147 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb098100a90 0x7fb09819c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:13.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.905+0000 7fb09e000700 1 -- 172.21.15.1:0/2108170147 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb080009520 con 0x7fb098106aa0 2024-02-02T15:35:13.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.905+0000 7fb09cc00700 1 --2- 172.21.15.1:0/2108170147 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb098101450 0x7fb09819ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:35:13.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.905+0000 7fb09d600700 1 --2- 172.21.15.1:0/2108170147 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb098100a90 0x7fb09819c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:35:13.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.905+0000 7fb09e000700 1 --2- 172.21.15.1:0/2108170147 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb098106aa0 0x7fb0981969b0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fb09000ef30 tx=0x7fb09000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:13.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.906+0000 7fb096000700 1 -- 172.21.15.1:0/2108170147 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb090018070 con 0x7fb098106aa0 2024-02-02T15:35:13.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.906+0000 7fb096000700 1 -- 172.21.15.1:0/2108170147 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb09000f040 con 0x7fb098106aa0 2024-02-02T15:35:13.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.906+0000 7fb09ea00700 1 -- 172.21.15.1:0/2108170147 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0880095f0 con 0x7fb098106aa0 2024-02-02T15:35:13.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.906+0000 7fb096000700 1 -- 172.21.15.1:0/2108170147 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb09000cd00 con 0x7fb098106aa0 2024-02-02T15:35:13.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.906+0000 7fb09ea00700 1 -- 172.21.15.1:0/2108170147 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0981a3750 con 0x7fb098106aa0 2024-02-02T15:35:13.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.909+0000 7fb096000700 1 -- 172.21.15.1:0/2108170147 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7fb090003a00 con 0x7fb098106aa0 2024-02-02T15:35:13.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.910+0000 7fb096000700 1 --2- 172.21.15.1:0/2108170147 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fb0780a0690 0x7fb0780a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:13.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.911+0000 7fb096000700 1 -- 172.21.15.1:0/2108170147 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fb0900cbc10 con 0x7fb098106aa0 2024-02-02T15:35:13.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.911+0000 7fb0b3bfbb80 1 Processor -- start 2024-02-02T15:35:13.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.911+0000 7fb09d600700 1 --2- 172.21.15.1:0/2108170147 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fb0780a0690 0x7fb0780a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:13.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.911+0000 7fb0b3bfbb80 1 -- start start 2024-02-02T15:35:13.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.911+0000 7fb0b3bfbb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ed8bd81f0 0x559ed8bd85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:13.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.911+0000 7fb0b3bfbb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ed8bd8b10 0x559ed8be34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:13.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.911+0000 7fb0b3bfbb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559ed8be39f0 0x559ed8be5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:13.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.911+0000 7fb0b3bfbb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ed8b4c4c0 con 0x559ed8bd81f0 2024-02-02T15:35:13.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.911+0000 7fb0b3bfbb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ed8b30aa0 con 0x559ed8be39f0 2024-02-02T15:35:13.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.912+0000 7fb0b3bfbb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ed8b2e460 con 0x559ed8bd8b10 2024-02-02T15:35:13.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.912+0000 7fb09d600700 1 --2- 172.21.15.1:0/2108170147 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fb0780a0690 0x7fb0780a2b50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb08800b2e0 tx=0x7fb088005fb0 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:13.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.912+0000 7fb09d600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ed8bd81f0 0x559ed8bd85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:13.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.912+0000 7fb09cc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ed8bd8b10 0x559ed8be34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:13.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.912+0000 7fb09e000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559ed8be39f0 0x559ed8be5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:13.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.912+0000 7fb09d600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ed8bd81f0 0x559ed8bd85d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46228/0 (socket says 172.21.15.1:46228) 2024-02-02T15:35:13.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.912+0000 7fb09cc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ed8bd8b10 0x559ed8be34b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:55942/0 (socket says 172.21.15.1:55942) 2024-02-02T15:35:13.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.912+0000 7fb09d600700 1 -- 172.21.15.1:0/317329574 learned_addr learned my addr 172.21.15.1:0/317329574 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:35:13.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.912+0000 7fb09d600700 1 -- 172.21.15.1:0/317329574 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ed8bd8b10 msgr2=0x559ed8be34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:13.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.912+0000 7fb09d600700 1 --2- 172.21.15.1:0/317329574 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ed8bd8b10 0x559ed8be34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:13.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.912+0000 7fb09d600700 1 -- 172.21.15.1:0/317329574 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559ed8be39f0 msgr2=0x559ed8be5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:13.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.912+0000 7fb09d600700 1 --2- 172.21.15.1:0/317329574 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559ed8be39f0 0x559ed8be5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:13.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.912+0000 7fb09d600700 1 -- 172.21.15.1:0/317329574 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ed89ab3b0 con 0x559ed8bd81f0 2024-02-02T15:35:13.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.912+0000 7fb09cc00700 1 --2- 172.21.15.1:0/317329574 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ed8bd8b10 0x559ed8be34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:35:13.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.912+0000 7fb09e000700 1 --2- 172.21.15.1:0/317329574 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559ed8be39f0 0x559ed8be5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:35:13.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.913+0000 7fb09d600700 1 --2- 172.21.15.1:0/317329574 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ed8bd81f0 0x559ed8bd85d0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fb08800bec0 tx=0x7fb088020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed403267ee5d8f8f server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:13.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.913+0000 7fb08e600700 1 -- 172.21.15.1:0/317329574 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb08800fdd0 con 0x559ed8bd81f0 2024-02-02T15:35:13.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.913+0000 7fb08e600700 1 -- 172.21.15.1:0/317329574 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb08801e040 con 0x559ed8bd81f0 2024-02-02T15:35:13.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.913+0000 7fb08e600700 1 -- 172.21.15.1:0/317329574 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb08802f5f0 con 0x559ed8bd81f0 2024-02-02T15:35:13.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.914+0000 7fb0b3bfbb80 1 -- 172.21.15.1:0/317329574 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ed8bd81f0 msgr2=0x559ed8bd85d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:13.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.914+0000 7fb0b3bfbb80 1 --2- 172.21.15.1:0/317329574 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ed8bd81f0 0x559ed8bd85d0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fb08800bec0 tx=0x7fb088020000 comp rx=0 tx=0).stop 2024-02-02T15:35:13.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.914+0000 7fb0b3bfbb80 1 -- 172.21.15.1:0/317329574 shutdown_connections 2024-02-02T15:35:13.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.914+0000 7fb0b3bfbb80 1 --2- 172.21.15.1:0/317329574 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559ed8be39f0 0x559ed8be5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:13.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.914+0000 7fb0b3bfbb80 1 --2- 172.21.15.1:0/317329574 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ed8bd81f0 0x559ed8bd85d0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:13.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.914+0000 7fb0b3bfbb80 1 --2- 172.21.15.1:0/317329574 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ed8bd8b10 0x559ed8be34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:13.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.914+0000 7fb0b3bfbb80 1 -- 172.21.15.1:0/317329574 >> 172.21.15.1:0/317329574 conn(0x559ed8b37db0 msgr2=0x559ed8bd45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:13.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.914+0000 7fb0b3bfbb80 1 -- 172.21.15.1:0/317329574 shutdown_connections 2024-02-02T15:35:13.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.914+0000 7fb0b3bfbb80 1 -- 172.21.15.1:0/317329574 wait complete. 2024-02-02T15:35:13.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.914+0000 7fb0b3bfbb80 1 -- start start 2024-02-02T15:35:13.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.915+0000 7fb0b3bfbb80 1 Processor -- start 2024-02-02T15:35:13.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.915+0000 7fb0b3bfbb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ed8bd81f0 0x559ed8be33b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:13.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.915+0000 7fb0b3bfbb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ed8bd8b10 0x559ed8be05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:13.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.915+0000 7fb0b3bfbb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559ed8be39f0 0x559ed8be0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:13.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.915+0000 7fb0b3bfbb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ed8b29d50 con 0x559ed8bd8b10 2024-02-02T15:35:13.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.915+0000 7fb0b3bfbb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ed8ac5d80 con 0x559ed8be39f0 2024-02-02T15:35:13.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.915+0000 7fb0b3bfbb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ed8ac5a20 con 0x559ed8bd81f0 2024-02-02T15:35:13.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.915+0000 7fb0b3bfbb80 10 client.0 _ll_register_callbacks cb 0x559ed8ac7290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:35:13.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.916+0000 7fb09d600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ed8bd81f0 0x559ed8be33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:13.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.916+0000 7fb09cc00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ed8bd8b10 0x559ed8be05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:13.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.916+0000 7fb09d600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ed8bd81f0 0x559ed8be33b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:55958/0 (socket says 172.21.15.1:55958) 2024-02-02T15:35:13.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.916+0000 7fb09d600700 1 -- 172.21.15.1:0/3061899650 learned_addr learned my addr 172.21.15.1:0/3061899650 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:35:13.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.916+0000 7fb09e000700 1 --2- 172.21.15.1:0/3061899650 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559ed8be39f0 0x559ed8be0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:13.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.916+0000 7fb09d600700 1 -- 172.21.15.1:0/3061899650 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559ed8be39f0 msgr2=0x559ed8be0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:13.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.916+0000 7fb09d600700 1 --2- 172.21.15.1:0/3061899650 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559ed8be39f0 0x559ed8be0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:13.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.916+0000 7fb09d600700 1 -- 172.21.15.1:0/3061899650 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ed8bd8b10 msgr2=0x559ed8be05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:13.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.916+0000 7fb09d600700 1 --2- 172.21.15.1:0/3061899650 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ed8bd8b10 0x559ed8be05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:13.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.916+0000 7fb09d600700 1 -- 172.21.15.1:0/3061899650 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0981a3750 con 0x559ed8bd81f0 2024-02-02T15:35:13.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.916+0000 7fb09cc00700 1 --2- 172.21.15.1:0/3061899650 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ed8bd8b10 0x559ed8be05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:35:13.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.916+0000 7fb09d600700 1 --2- 172.21.15.1:0/3061899650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ed8bd81f0 0x559ed8be33b0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fb088009350 tx=0x7fb088009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a19c6415be6ea1ad server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:13.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.916+0000 7fb086a00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:35:13.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.917+0000 7fb086a00700 1 -- 172.21.15.1:0/3061899650 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb08802f830 con 0x559ed8bd81f0 2024-02-02T15:35:13.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.917+0000 7fb086a00700 1 -- 172.21.15.1:0/3061899650 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fb08802f990 con 0x559ed8bd81f0 2024-02-02T15:35:13.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.917+0000 7fb0b3bfbb80 1 -- 172.21.15.1:0/3061899650 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559ed8bcc340 con 0x559ed8bd81f0 2024-02-02T15:35:13.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.917+0000 7fb086a00700 1 -- 172.21.15.1:0/3061899650 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb08802fb50 con 0x559ed8bd81f0 2024-02-02T15:35:13.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.917+0000 7fb0b3bfbb80 10 client.27766 fetch_fsmap learned FSMap version 42 2024-02-02T15:35:13.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.917+0000 7fb0b3bfbb80 1 -- 172.21.15.1:0/3061899650 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7fb09810e260 con 0x559ed8bd81f0 2024-02-02T15:35:13.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.917+0000 7fb086a00700 1 -- 172.21.15.1:0/3061899650 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559ed8bcc340 con 0x559ed8bd81f0 2024-02-02T15:35:13.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.917+0000 7fb086a00700 1 -- 172.21.15.1:0/3061899650 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 42) v1 ==== 7510+0+0 (secure 0 0 0) 0x7fb088042080 con 0x559ed8bd81f0 2024-02-02T15:35:13.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.917+0000 7fb0b3bfbb80 10 client.27766 fetch_fsmap finished waiting for FSMap version 42 2024-02-02T15:35:13.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.917+0000 7fb0b3bfbb80 10 client.27766 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:35:13.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.918+0000 7fb0b3bfbb80 20 client.27766 populate_metadata read hostname 'smithi001' 2024-02-02T15:35:13.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.918+0000 7fb0b3bfbb80 1 --2- 172.21.15.1:0/3061899650 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x559ed8bf2f00 0x559ed8bf5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:13.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.918+0000 7fb0b3bfbb80 4 client.27766 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:35:13.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.918+0000 7fb0b3bfbb80 1 -- 172.21.15.1:0/3061899650 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559ed8b22dd0 con 0x559ed8bf2f00 2024-02-02T15:35:13.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.918+0000 7fb09cc00700 1 --2- 172.21.15.1:0/3061899650 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x559ed8bf2f00 0x559ed8bf5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:13.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.918+0000 7fb09cc00700 1 --2- 172.21.15.1:0/3061899650 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x559ed8bf2f00 0x559ed8bf5320 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=440a18465558229f server_cookie=8fc96b2f2214679e in_seq=0 out_seq=0 2024-02-02T15:35:13.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.918+0000 7fb086a00700 10 client.27766 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:35:13.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.919+0000 7fb086a00700 1 -- 172.21.15.1:0/3061899650 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x559ed8b22dd0 con 0x559ed8bf2f00 2024-02-02T15:35:13.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.919+0000 7fb086a00700 10 client.27766 handle_command_reply: tid=0 2024-02-02T15:35:13.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.919+0000 7fb0b3bfbb80 1 client.27766 shutdown 2024-02-02T15:35:13.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.920+0000 7fb0b3bfbb80 1 -- 172.21.15.1:0/3061899650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ed8bd81f0 msgr2=0x559ed8be33b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:13.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.920+0000 7fb0b3bfbb80 1 --2- 172.21.15.1:0/3061899650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ed8bd81f0 0x559ed8be33b0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fb088009350 tx=0x7fb088009580 comp rx=0 tx=0).stop 2024-02-02T15:35:13.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.920+0000 7fb0b3bfbb80 1 -- 172.21.15.1:0/3061899650 shutdown_connections 2024-02-02T15:35:13.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.920+0000 7fb0b3bfbb80 1 --2- 172.21.15.1:0/3061899650 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x559ed8bf2f00 0x559ed8bf5320 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:13.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.920+0000 7fb0b3bfbb80 1 --2- 172.21.15.1:0/3061899650 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559ed8be39f0 0x559ed8be0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:13.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.920+0000 7fb0b3bfbb80 1 --2- 172.21.15.1:0/3061899650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ed8bd81f0 0x559ed8be33b0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:13.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.920+0000 7fb086a00700 0 client.27766 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:35:13.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.920+0000 7fb0b3bfbb80 1 --2- 172.21.15.1:0/3061899650 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ed8bd8b10 0x559ed8be05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:13.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.920+0000 7fb0b3bfbb80 1 -- 172.21.15.1:0/3061899650 >> 172.21.15.1:0/3061899650 conn(0x559ed8b37db0 msgr2=0x559ed8bd41b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:13.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.920+0000 7fb0b3bfbb80 1 -- 172.21.15.1:0/3061899650 shutdown_connections 2024-02-02T15:35:13.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.920+0000 7fb0b3bfbb80 1 -- 172.21.15.1:0/3061899650 wait complete. 2024-02-02T15:35:13.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.920+0000 7fb0b3bfbb80 20 client.27766 trim_cache size 0 max 16384 2024-02-02T15:35:13.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.940+0000 7fb0b3bfbb80 1 Processor -- start 2024-02-02T15:35:13.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.940+0000 7fb0b3bfbb80 1 -- start start 2024-02-02T15:35:13.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.940+0000 7fb0b3bfbb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ed8bd81f0 0x559ed8bea470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:13.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.940+0000 7fb0b3bfbb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559ed8bd8b10 0x559ed8beea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:13.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.940+0000 7fb0b3bfbb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ed8be39f0 0x559ed8beef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:13.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.940+0000 7fb0b3bfbb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ed89ad6a0 con 0x559ed8be39f0 2024-02-02T15:35:13.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.940+0000 7fb0b3bfbb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ed8bd9250 con 0x559ed8bd8b10 2024-02-02T15:35:13.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.940+0000 7fb0b3bfbb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ed8be11d0 con 0x559ed8bd81f0 2024-02-02T15:35:13.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.941+0000 7fb09cc00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559ed8bd8b10 0x559ed8beea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:13.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.941+0000 7fb09d600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ed8bd81f0 0x559ed8bea470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:13.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.941+0000 7fb09cc00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559ed8bd8b10 0x559ed8beea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:54744/0 (socket says 172.21.15.1:54744) 2024-02-02T15:35:13.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.941+0000 7fb09cc00700 1 -- 172.21.15.1:0/2696060340 learned_addr learned my addr 172.21.15.1:0/2696060340 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:35:13.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.941+0000 7fb09e000700 1 --2- 172.21.15.1:0/2696060340 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ed8be39f0 0x559ed8beef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:13.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.941+0000 7fb09cc00700 1 -- 172.21.15.1:0/2696060340 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ed8bd81f0 msgr2=0x559ed8bea470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:13.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.941+0000 7fb09cc00700 1 --2- 172.21.15.1:0/2696060340 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ed8bd81f0 0x559ed8bea470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:13.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.941+0000 7fb09cc00700 1 -- 172.21.15.1:0/2696060340 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ed8be39f0 msgr2=0x559ed8beef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:13.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.941+0000 7fb09cc00700 1 --2- 172.21.15.1:0/2696060340 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ed8be39f0 0x559ed8beef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:13.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.941+0000 7fb09cc00700 1 -- 172.21.15.1:0/2696060340 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ed8957b60 con 0x559ed8bd8b10 2024-02-02T15:35:13.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.941+0000 7fb09d600700 1 --2- 172.21.15.1:0/2696060340 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ed8bd81f0 0x559ed8bea470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:35:13.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.941+0000 7fb09cc00700 1 --2- 172.21.15.1:0/2696060340 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559ed8bd8b10 0x559ed8beea50 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fb098106120 tx=0x7fb080009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c126224e58972b28 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:13.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.942+0000 7fb08f000700 1 -- 172.21.15.1:0/2696060340 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb080017040 con 0x559ed8bd8b10 2024-02-02T15:35:13.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.942+0000 7fb08f000700 1 -- 172.21.15.1:0/2696060340 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb080018040 con 0x559ed8bd8b10 2024-02-02T15:35:13.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.942+0000 7fb08f000700 1 -- 172.21.15.1:0/2696060340 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb080005280 con 0x559ed8bd8b10 2024-02-02T15:35:13.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.943+0000 7fb0b3bfbb80 1 -- 172.21.15.1:0/2696060340 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559ed8bd8b10 msgr2=0x559ed8beea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:13.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.943+0000 7fb0b3bfbb80 1 --2- 172.21.15.1:0/2696060340 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559ed8bd8b10 0x559ed8beea50 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fb098106120 tx=0x7fb080009260 comp rx=0 tx=0).stop 2024-02-02T15:35:13.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.943+0000 7fb0b3bfbb80 1 -- 172.21.15.1:0/2696060340 shutdown_connections 2024-02-02T15:35:13.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.943+0000 7fb0b3bfbb80 1 --2- 172.21.15.1:0/2696060340 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ed8be39f0 0x559ed8beef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:13.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.943+0000 7fb0b3bfbb80 1 --2- 172.21.15.1:0/2696060340 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ed8bd81f0 0x559ed8bea470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:13.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.943+0000 7fb0b3bfbb80 1 --2- 172.21.15.1:0/2696060340 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559ed8bd8b10 0x559ed8beea50 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:13.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.943+0000 7fb0b3bfbb80 1 -- 172.21.15.1:0/2696060340 >> 172.21.15.1:0/2696060340 conn(0x559ed8b37db0 msgr2=0x559ed8bd2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:13.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.943+0000 7fb0b3bfbb80 1 -- 172.21.15.1:0/2696060340 shutdown_connections 2024-02-02T15:35:13.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.943+0000 7fb0b3bfbb80 1 -- 172.21.15.1:0/2696060340 wait complete. 2024-02-02T15:35:13.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.943+0000 7fb0b3bfbb80 1 -- start start 2024-02-02T15:35:13.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.944+0000 7fb0b3bfbb80 1 Processor -- start 2024-02-02T15:35:13.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.944+0000 7fb0b3bfbb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559ed8bd81f0 0x559ed8c40ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:13.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.944+0000 7fb0b3bfbb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ed8bd8b10 0x559ed8c41220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:13.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.944+0000 7fb0b3bfbb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ed8be39f0 0x559ed8c41a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:13.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.944+0000 7fb0b3bfbb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ed8c42140 con 0x559ed8bd8b10 2024-02-02T15:35:13.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.944+0000 7fb0b3bfbb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ed8c422b0 con 0x559ed8bd81f0 2024-02-02T15:35:13.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.944+0000 7fb0b3bfbb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ed8c42470 con 0x559ed8be39f0 2024-02-02T15:35:13.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.944+0000 7fb0b3bfbb80 10 client.0 _ll_register_callbacks cb 0x559ed8ac7290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:35:13.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.944+0000 7fb09d600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559ed8bd81f0 0x559ed8c40ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:13.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.945+0000 7fb09d600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559ed8bd81f0 0x559ed8c40ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:54760/0 (socket says 172.21.15.1:54760) 2024-02-02T15:35:13.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.945+0000 7fb09d600700 1 -- 172.21.15.1:0/1232040530 learned_addr learned my addr 172.21.15.1:0/1232040530 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:35:13.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.945+0000 7fb09e000700 1 --2- 172.21.15.1:0/1232040530 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ed8be39f0 0x559ed8c41a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:13.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.945+0000 7fb09cc00700 1 --2- 172.21.15.1:0/1232040530 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ed8bd8b10 0x559ed8c41220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:13.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.945+0000 7fb09d600700 1 -- 172.21.15.1:0/1232040530 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ed8be39f0 msgr2=0x559ed8c41a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:13.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.945+0000 7fb09d600700 1 --2- 172.21.15.1:0/1232040530 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ed8be39f0 0x559ed8c41a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:13.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.945+0000 7fb09d600700 1 -- 172.21.15.1:0/1232040530 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ed8bd8b10 msgr2=0x559ed8c41220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:13.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.945+0000 7fb09d600700 1 --2- 172.21.15.1:0/1232040530 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ed8bd8b10 0x559ed8c41220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:13.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.945+0000 7fb09d600700 1 -- 172.21.15.1:0/1232040530 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb09810e260 con 0x559ed8bd81f0 2024-02-02T15:35:13.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.945+0000 7fb09d600700 1 --2- 172.21.15.1:0/1232040530 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559ed8bd81f0 0x559ed8c40ce0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fb08800b6f0 tx=0x7fb08800bee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=854b7efc29cffd1d server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:13.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.945+0000 7fb086a00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:35:13.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.945+0000 7fb086a00700 1 -- 172.21.15.1:0/1232040530 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb088039db0 con 0x559ed8bd81f0 2024-02-02T15:35:13.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.946+0000 7fb086a00700 1 -- 172.21.15.1:0/1232040530 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fb088039760 con 0x559ed8bd81f0 2024-02-02T15:35:13.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.946+0000 7fb0b3bfbb80 1 -- 172.21.15.1:0/1232040530 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb08801e040 con 0x559ed8bd81f0 2024-02-02T15:35:13.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.946+0000 7fb086a00700 1 -- 172.21.15.1:0/1232040530 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb088004830 con 0x559ed8bd81f0 2024-02-02T15:35:13.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.946+0000 7fb0b3bfbb80 10 client.27776 fetch_fsmap learned FSMap version 42 2024-02-02T15:35:13.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.946+0000 7fb0b3bfbb80 1 -- 172.21.15.1:0/1232040530 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7fb080012cc0 con 0x559ed8bd81f0 2024-02-02T15:35:13.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.946+0000 7fb086a00700 1 -- 172.21.15.1:0/1232040530 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb08801e040 con 0x559ed8bd81f0 2024-02-02T15:35:13.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.946+0000 7fb086a00700 1 -- 172.21.15.1:0/1232040530 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 42) v1 ==== 7510+0+0 (secure 0 0 0) 0x7fb088037070 con 0x559ed8bd81f0 2024-02-02T15:35:13.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.946+0000 7fb0b3bfbb80 10 client.27776 fetch_fsmap finished waiting for FSMap version 42 2024-02-02T15:35:13.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.946+0000 7fb0b3bfbb80 10 client.27776 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:35:13.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.946+0000 7fb0b3bfbb80 20 client.27776 populate_metadata read hostname 'smithi001' 2024-02-02T15:35:13.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.946+0000 7fb0b3bfbb80 1 --2- 172.21.15.1:0/1232040530 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x559ed8c497c0 0x559ed8c4bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:13.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.946+0000 7fb0b3bfbb80 4 client.27776 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2024-02-02T15:35:13.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.946+0000 7fb0b3bfbb80 1 -- 172.21.15.1:0/1232040530 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559ed8ae8f50 con 0x559ed8c497c0 2024-02-02T15:35:13.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.947+0000 7fb09cc00700 1 --2- 172.21.15.1:0/1232040530 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x559ed8c497c0 0x559ed8c4bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:13.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.947+0000 7fb086a00700 10 client.27776 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:35:13.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.947+0000 7fb09cc00700 1 --2- 172.21.15.1:0/1232040530 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x559ed8c497c0 0x559ed8c4bbe0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=146935b73ce6c422 server_cookie=7380b0533d6f715c in_seq=0 out_seq=0 2024-02-02T15:35:13.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.947+0000 7fb086a00700 1 -- 172.21.15.1:0/1232040530 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x559ed8ae8f50 con 0x559ed8c497c0 2024-02-02T15:35:13.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.947+0000 7fb086a00700 10 client.27776 handle_command_reply: tid=0 2024-02-02T15:35:13.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.947+0000 7fb0b3bfbb80 1 client.27776 shutdown 2024-02-02T15:35:13.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.947+0000 7fb0b3bfbb80 1 -- 172.21.15.1:0/1232040530 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559ed8bd81f0 msgr2=0x559ed8c40ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:13.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.947+0000 7fb0b3bfbb80 1 --2- 172.21.15.1:0/1232040530 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559ed8bd81f0 0x559ed8c40ce0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fb08800b6f0 tx=0x7fb08800bee0 comp rx=0 tx=0).stop 2024-02-02T15:35:13.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.947+0000 7fb0b3bfbb80 1 -- 172.21.15.1:0/1232040530 shutdown_connections 2024-02-02T15:35:13.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.947+0000 7fb0b3bfbb80 1 --2- 172.21.15.1:0/1232040530 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x559ed8c497c0 0x559ed8c4bbe0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:13.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.948+0000 7fb0b3bfbb80 1 --2- 172.21.15.1:0/1232040530 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559ed8be39f0 0x559ed8c41a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:13.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.948+0000 7fb0b3bfbb80 1 --2- 172.21.15.1:0/1232040530 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559ed8bd81f0 0x559ed8c40ce0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:13.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.948+0000 7fb0b3bfbb80 1 --2- 172.21.15.1:0/1232040530 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559ed8bd8b10 0x559ed8c41220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:13.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.948+0000 7fb0b3bfbb80 1 -- 172.21.15.1:0/1232040530 >> 172.21.15.1:0/1232040530 conn(0x559ed8b37db0 msgr2=0x559ed8bdb8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:13.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.948+0000 7fb086a00700 0 client.27776 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:35:13.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.948+0000 7fb0b3bfbb80 1 -- 172.21.15.1:0/1232040530 shutdown_connections 2024-02-02T15:35:13.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.948+0000 7fb0b3bfbb80 1 -- 172.21.15.1:0/1232040530 wait complete. 2024-02-02T15:35:13.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.948+0000 7fb0b3bfbb80 20 client.27776 trim_cache size 0 max 16384 2024-02-02T15:35:13.952 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:35:13.952 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2024-02-02T15:35:13.952 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2024-02-02T15:35:13.952 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:35:13.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.949+0000 7fb09ea00700 1 -- 172.21.15.1:0/2108170147 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fb0780a0690 msgr2=0x7fb0780a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:13.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.949+0000 7fb09ea00700 1 --2- 172.21.15.1:0/2108170147 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fb0780a0690 0x7fb0780a2b50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb08800b2e0 tx=0x7fb088005fb0 comp rx=0 tx=0).stop 2024-02-02T15:35:13.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.949+0000 7fb09ea00700 1 -- 172.21.15.1:0/2108170147 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb098106aa0 msgr2=0x7fb0981969b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:13.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.949+0000 7fb09ea00700 1 --2- 172.21.15.1:0/2108170147 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb098106aa0 0x7fb0981969b0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fb09000ef30 tx=0x7fb09000c3a0 comp rx=0 tx=0).stop 2024-02-02T15:35:13.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.949+0000 7fb09ea00700 1 -- 172.21.15.1:0/2108170147 shutdown_connections 2024-02-02T15:35:13.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.949+0000 7fb09ea00700 1 --2- 172.21.15.1:0/2108170147 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fb0780a0690 0x7fb0780a2b50 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:13.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.949+0000 7fb09ea00700 1 --2- 172.21.15.1:0/2108170147 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb098106aa0 0x7fb0981969b0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:13.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.949+0000 7fb09ea00700 1 --2- 172.21.15.1:0/2108170147 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb098100a90 0x7fb09819c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:13.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.949+0000 7fb09ea00700 1 --2- 172.21.15.1:0/2108170147 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb098101450 0x7fb09819ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:13.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.949+0000 7fb09ea00700 1 -- 172.21.15.1:0/2108170147 >> 172.21.15.1:0/2108170147 conn(0x7fb0980fc560 msgr2=0x7fb0980fcb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:13.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.949+0000 7fb09ea00700 1 -- 172.21.15.1:0/2108170147 shutdown_connections 2024-02-02T15:35:13.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:13.949+0000 7fb09ea00700 1 -- 172.21.15.1:0/2108170147 wait complete. 2024-02-02T15:35:13.968 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-02T15:35:13.968 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T15:35:14.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.184+0000 7feac1c00700 1 -- 172.21.15.1:0/2897628417 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feabc106040 msgr2=0x7feabc10dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:14.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.184+0000 7feac1c00700 1 --2- 172.21.15.1:0/2897628417 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feabc106040 0x7feabc10dbc0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7feab400b0a0 tx=0x7feab400b3b0 comp rx=0 tx=0).stop 2024-02-02T15:35:14.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.184+0000 7feac1c00700 1 -- 172.21.15.1:0/2897628417 shutdown_connections 2024-02-02T15:35:14.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.184+0000 7feac1c00700 1 --2- 172.21.15.1:0/2897628417 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feabc106040 0x7feabc10dbc0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:14.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.184+0000 7feac1c00700 1 --2- 172.21.15.1:0/2897628417 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feabc100f20 0x7feabc101300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:14.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.184+0000 7feac1c00700 1 --2- 172.21.15.1:0/2897628417 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feabc1018d0 0x7feabc105910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:14.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.184+0000 7feac1c00700 1 -- 172.21.15.1:0/2897628417 >> 172.21.15.1:0/2897628417 conn(0x7feabc0fc7a0 msgr2=0x7feabc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:14.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.184+0000 7feac1c00700 1 -- 172.21.15.1:0/2897628417 shutdown_connections 2024-02-02T15:35:14.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.184+0000 7feac1c00700 1 -- 172.21.15.1:0/2897628417 wait complete. 2024-02-02T15:35:14.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.185+0000 7feac1c00700 1 Processor -- start 2024-02-02T15:35:14.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.185+0000 7feac1c00700 1 -- start start 2024-02-02T15:35:14.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.186+0000 7feac1c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feabc100f20 0x7feabc198f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:14.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.186+0000 7feac1c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feabc106040 0x7feabc199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:14.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.186+0000 7feac1c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feabc19d8c0 0x7feabc19dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:14.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.186+0000 7feac1c00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feabc19e340 con 0x7feabc100f20 2024-02-02T15:35:14.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.186+0000 7feac1c00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feabc19e4b0 con 0x7feabc19d8c0 2024-02-02T15:35:14.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.186+0000 7feac1c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feabc19e670 con 0x7feabc106040 2024-02-02T15:35:14.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.186+0000 7feaba000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feabc106040 0x7feabc199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:14.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.186+0000 7feabaa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feabc100f20 0x7feabc198f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:14.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.186+0000 7feabb400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feabc19d8c0 0x7feabc19dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:14.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.186+0000 7feaba000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feabc106040 0x7feabc199460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:55992/0 (socket says 172.21.15.1:55992) 2024-02-02T15:35:14.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.186+0000 7feabaa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feabc100f20 0x7feabc198f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46256/0 (socket says 172.21.15.1:46256) 2024-02-02T15:35:14.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.186+0000 7feaba000700 1 -- 172.21.15.1:0/3305157595 learned_addr learned my addr 172.21.15.1:0/3305157595 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:35:14.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.186+0000 7feabaa00700 1 -- 172.21.15.1:0/3305157595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feabc106040 msgr2=0x7feabc199460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:14.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.186+0000 7feabaa00700 1 --2- 172.21.15.1:0/3305157595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feabc106040 0x7feabc199460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:14.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.186+0000 7feabaa00700 1 -- 172.21.15.1:0/3305157595 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feabc19d8c0 msgr2=0x7feabc19dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:14.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.186+0000 7feabaa00700 1 --2- 172.21.15.1:0/3305157595 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feabc19d8c0 0x7feabc19dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:14.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.186+0000 7feabaa00700 1 -- 172.21.15.1:0/3305157595 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feab4009d00 con 0x7feabc100f20 2024-02-02T15:35:14.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.186+0000 7feabb400700 1 --2- 172.21.15.1:0/3305157595 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feabc19d8c0 0x7feabc19dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:35:14.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.187+0000 7feabaa00700 1 --2- 172.21.15.1:0/3305157595 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feabc100f20 0x7feabc198f20 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7feaac00e9f0 tx=0x7feaac00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:14.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.187+0000 7feab3400700 1 -- 172.21.15.1:0/3305157595 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaac00c9c0 con 0x7feabc100f20 2024-02-02T15:35:14.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.187+0000 7feab3400700 1 -- 172.21.15.1:0/3305157595 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feaac010430 con 0x7feabc100f20 2024-02-02T15:35:14.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.187+0000 7feab3400700 1 -- 172.21.15.1:0/3305157595 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaac018680 con 0x7feabc100f20 2024-02-02T15:35:14.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.187+0000 7feac1c00700 1 -- 172.21.15.1:0/3305157595 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feabc1a5bd0 con 0x7feabc100f20 2024-02-02T15:35:14.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.187+0000 7feac1c00700 1 -- 172.21.15.1:0/3305157595 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feabc1a6060 con 0x7feabc100f20 2024-02-02T15:35:14.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.188+0000 7feac1c00700 1 -- 172.21.15.1:0/3305157595 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feabc067bc0 con 0x7feabc100f20 2024-02-02T15:35:14.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.191+0000 7feab3400700 1 -- 172.21.15.1:0/3305157595 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7feaac010860 con 0x7feabc100f20 2024-02-02T15:35:14.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.192+0000 7feab3400700 1 --2- 172.21.15.1:0/3305157595 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fea9c0a0710 0x7fea9c0a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:14.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.192+0000 7feab3400700 1 -- 172.21.15.1:0/3305157595 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9453+0+0 (secure 0 0 0) 0x7feaac014070 con 0x7feabc100f20 2024-02-02T15:35:14.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.193+0000 7feaba000700 1 --2- 172.21.15.1:0/3305157595 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fea9c0a0710 0x7fea9c0a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:14.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.193+0000 7feaba000700 1 --2- 172.21.15.1:0/3305157595 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fea9c0a0710 0x7fea9c0a2bd0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7feabc19a530 tx=0x7feaa4009260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:14.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.195+0000 7feab3400700 1 -- 172.21.15.1:0/3305157595 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7feaac0957e0 con 0x7feabc100f20 2024-02-02T15:35:14.250 INFO:tasks.workunit.client.0.smithi001.stdout: 1 479336 6.40 MB/sec execute 288 sec latency 24.081 ms 2024-02-02T15:35:14.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.491+0000 7feac1c00700 1 -- 172.21.15.1:0/3305157595 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feabc101b00 con 0x7feabc100f20 2024-02-02T15:35:14.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.492+0000 7feab3400700 1 -- 172.21.15.1:0/3305157595 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+10882 (secure 0 0 0) 0x7feaac026090 con 0x7feabc100f20 2024-02-02T15:35:14.492 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:35:14.493 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":42,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2922240448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2922240448},{"type":"v1","addr":"172.21.15.1:6835","nonce":2922240448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2729497431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2729497431},{"type":"v1","addr":"172.21.15.1:6837","nonce":2729497431}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":42,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-02T15:24:47.761435+0000","modified":"2024-02-02T15:34:41.481636+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24529,"mds_2":24518,"mds_3":24514,"mds_4":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14517":{"gid":14517,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/1218303683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":1218303683},{"type":"v1","addr":"172.21.15.1:6839","nonce":1218303683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14529":{"gid":14529,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/2059197496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2059197496},{"type":"v1","addr":"172.21.15.1:6841","nonce":2059197496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/7648986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":7648986},{"type":"v1","addr":"172.21.15.142:6833","nonce":7648986}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/3482746094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3482746094},{"type":"v1","addr":"172.21.15.71:6833","nonce":3482746094}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/388207814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":388207814},{"type":"v1","addr":"172.21.15.71:6835","nonce":388207814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6835/1052606547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1052606547},{"type":"v1","addr":"172.21.15.142:6835","nonce":1052606547}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.142:6837/2870289019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2870289019},{"type":"v1","addr":"172.21.15.142:6837","nonce":2870289019}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.71:6837/628016199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":628016199},{"type":"v1","addr":"172.21.15.71:6837","nonce":628016199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.142:6839/321675421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":321675421},{"type":"v1","addr":"172.21.15.142:6839","nonce":321675421}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/1268086493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1268086493},{"type":"v1","addr":"172.21.15.71:6839","nonce":1268086493}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-02T15:35:14.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.495+0000 7feac1c00700 1 -- 172.21.15.1:0/3305157595 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fea9c0a0710 msgr2=0x7fea9c0a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:14.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.495+0000 7feac1c00700 1 --2- 172.21.15.1:0/3305157595 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fea9c0a0710 0x7fea9c0a2bd0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7feabc19a530 tx=0x7feaa4009260 comp rx=0 tx=0).stop 2024-02-02T15:35:14.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.495+0000 7feac1c00700 1 -- 172.21.15.1:0/3305157595 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feabc100f20 msgr2=0x7feabc198f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:14.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.495+0000 7feac1c00700 1 --2- 172.21.15.1:0/3305157595 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feabc100f20 0x7feabc198f20 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7feaac00e9f0 tx=0x7feaac00ed00 comp rx=0 tx=0).stop 2024-02-02T15:35:14.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.495+0000 7feac1c00700 1 -- 172.21.15.1:0/3305157595 shutdown_connections 2024-02-02T15:35:14.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.495+0000 7feac1c00700 1 --2- 172.21.15.1:0/3305157595 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fea9c0a0710 0x7fea9c0a2bd0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:14.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.495+0000 7feac1c00700 1 --2- 172.21.15.1:0/3305157595 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7feabc19d8c0 0x7feabc19dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:14.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.495+0000 7feac1c00700 1 --2- 172.21.15.1:0/3305157595 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7feabc100f20 0x7feabc198f20 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:14.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.495+0000 7feac1c00700 1 --2- 172.21.15.1:0/3305157595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7feabc106040 0x7feabc199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:14.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.495+0000 7feac1c00700 1 -- 172.21.15.1:0/3305157595 >> 172.21.15.1:0/3305157595 conn(0x7feabc0fc7a0 msgr2=0x7feabc10c380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:14.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.495+0000 7feac1c00700 1 -- 172.21.15.1:0/3305157595 shutdown_connections 2024-02-02T15:35:14.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.495+0000 7feac1c00700 1 -- 172.21.15.1:0/3305157595 wait complete. 2024-02-02T15:35:14.539 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 42 2024-02-02T15:35:14.540 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-02-02T15:35:14.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.759+0000 7fbdb9400700 1 -- 172.21.15.1:0/322822424 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbdb4103bb0 msgr2=0x7fbdb4107bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:14.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.759+0000 7fbdb9400700 1 --2- 172.21.15.1:0/322822424 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbdb4103bb0 0x7fbdb4107bf0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c009990 tx=0x7fbd9c009ca0 comp rx=0 tx=0).stop 2024-02-02T15:35:14.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.759+0000 7fbdb9400700 1 -- 172.21.15.1:0/322822424 shutdown_connections 2024-02-02T15:35:14.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.759+0000 7fbdb9400700 1 --2- 172.21.15.1:0/322822424 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbdb4108320 0x7fbdb410fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:14.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.759+0000 7fbdb9400700 1 --2- 172.21.15.1:0/322822424 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbdb4103200 0x7fbdb41035e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:14.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.759+0000 7fbdb9400700 1 --2- 172.21.15.1:0/322822424 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbdb4103bb0 0x7fbdb4107bf0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:14.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.759+0000 7fbdb9400700 1 -- 172.21.15.1:0/322822424 >> 172.21.15.1:0/322822424 conn(0x7fbdb40feaa0 msgr2=0x7fbdb4100ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:14.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.759+0000 7fbdb9400700 1 -- 172.21.15.1:0/322822424 shutdown_connections 2024-02-02T15:35:14.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.759+0000 7fbdb9400700 1 -- 172.21.15.1:0/322822424 wait complete. 2024-02-02T15:35:14.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.760+0000 7fbdb9400700 1 Processor -- start 2024-02-02T15:35:14.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.761+0000 7fbdb9400700 1 -- start start 2024-02-02T15:35:14.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.761+0000 7fbdb9400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbdb4103200 0x7fbdb4199070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:14.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.761+0000 7fbdb9400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbdb4108320 0x7fbdb41995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:14.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.761+0000 7fbdb9400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbdb419da10 0x7fbdb419de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:14.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.761+0000 7fbdb9400700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb419e490 con 0x7fbdb4103200 2024-02-02T15:35:14.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.761+0000 7fbdb9400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb419e600 con 0x7fbdb419da10 2024-02-02T15:35:14.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.761+0000 7fbdb9400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdb419e7c0 con 0x7fbdb4108320 2024-02-02T15:35:14.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.761+0000 7fbdb1e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbdb4108320 0x7fbdb41995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:14.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.761+0000 7fbdb1e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbdb4108320 0x7fbdb41995b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:56008/0 (socket says 172.21.15.1:56008) 2024-02-02T15:35:14.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.761+0000 7fbdb1e00700 1 -- 172.21.15.1:0/4171908550 learned_addr learned my addr 172.21.15.1:0/4171908550 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:35:14.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.762+0000 7fbdb2800700 1 --2- 172.21.15.1:0/4171908550 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbdb4103200 0x7fbdb4199070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:14.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.762+0000 7fbdb3200700 1 --2- 172.21.15.1:0/4171908550 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbdb419da10 0x7fbdb419de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:14.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.762+0000 7fbdb1e00700 1 -- 172.21.15.1:0/4171908550 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbdb419da10 msgr2=0x7fbdb419de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:14.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.762+0000 7fbdb1e00700 1 --2- 172.21.15.1:0/4171908550 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbdb419da10 0x7fbdb419de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:14.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.762+0000 7fbdb1e00700 1 -- 172.21.15.1:0/4171908550 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbdb4103200 msgr2=0x7fbdb4199070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:14.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.762+0000 7fbdb1e00700 1 --2- 172.21.15.1:0/4171908550 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbdb4103200 0x7fbdb4199070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:14.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.762+0000 7fbdb1e00700 1 -- 172.21.15.1:0/4171908550 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbda4009520 con 0x7fbdb4108320 2024-02-02T15:35:14.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.763+0000 7fbdb1e00700 1 --2- 172.21.15.1:0/4171908550 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbdb4108320 0x7fbdb41995b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c000c00 tx=0x7fbd9c005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:14.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.763+0000 7fbdab400700 1 -- 172.21.15.1:0/4171908550 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c01d070 con 0x7fbdb4108320 2024-02-02T15:35:14.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.763+0000 7fbdab400700 1 -- 172.21.15.1:0/4171908550 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbd9c021410 con 0x7fbdb4108320 2024-02-02T15:35:14.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.763+0000 7fbdb9400700 1 -- 172.21.15.1:0/4171908550 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd9c0095f0 con 0x7fbdb4108320 2024-02-02T15:35:14.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.763+0000 7fbdab400700 1 -- 172.21.15.1:0/4171908550 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c017650 con 0x7fbdb4108320 2024-02-02T15:35:14.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.763+0000 7fbdb9400700 1 -- 172.21.15.1:0/4171908550 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdb41a5fb0 con 0x7fbdb4108320 2024-02-02T15:35:14.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.766+0000 7fbdab400700 1 -- 172.21.15.1:0/4171908550 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7fbd9c021580 con 0x7fbdb4108320 2024-02-02T15:35:14.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.768+0000 7fbdccfa4b80 1 Processor -- start 2024-02-02T15:35:14.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.769+0000 7fbdab400700 1 --2- 172.21.15.1:0/4171908550 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fbd940a0690 0x7fbd940a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:14.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.769+0000 7fbdccfa4b80 1 -- start start 2024-02-02T15:35:14.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.769+0000 7fbdab400700 1 -- 172.21.15.1:0/4171908550 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fbd9c0cd430 con 0x7fbdb4108320 2024-02-02T15:35:14.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.769+0000 7fbdb3200700 1 --2- 172.21.15.1:0/4171908550 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fbd940a0690 0x7fbd940a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:14.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.769+0000 7fbdccfa4b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e93e9e3010 0x55e93e9e33f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:14.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.769+0000 7fbdccfa4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e93e9e3930 0x55e93e9ee2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:14.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.769+0000 7fbdccfa4b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e93e9ee7e0 0x55e93e9f0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:14.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.769+0000 7fbdccfa4b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e93e9574c0 con 0x55e93e9ee7e0 2024-02-02T15:35:14.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.769+0000 7fbdccfa4b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e93e93baa0 con 0x55e93e9e3010 2024-02-02T15:35:14.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.769+0000 7fbdccfa4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e93e939460 con 0x55e93e9e3930 2024-02-02T15:35:14.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.769+0000 7fbdb2800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e93e9e3010 0x55e93e9e33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:14.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.769+0000 7fbdb2800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e93e9e3010 0x55e93e9e33f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:54818/0 (socket says 172.21.15.1:54818) 2024-02-02T15:35:14.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.769+0000 7fbdb2800700 1 -- 172.21.15.1:0/990913437 learned_addr learned my addr 172.21.15.1:0/990913437 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:35:14.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.770+0000 7fbdb1e00700 1 --2- 172.21.15.1:0/990913437 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e93e9e3930 0x55e93e9ee2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:14.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.770+0000 7fbdb3200700 1 --2- 172.21.15.1:0/990913437 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e93e9ee7e0 0x55e93e9f0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:14.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.770+0000 7fbdb2800700 1 -- 172.21.15.1:0/990913437 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e93e9e3930 msgr2=0x55e93e9ee2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:14.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.770+0000 7fbdb2800700 1 --2- 172.21.15.1:0/990913437 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e93e9e3930 0x55e93e9ee2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:14.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.770+0000 7fbdb2800700 1 -- 172.21.15.1:0/990913437 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e93e9ee7e0 msgr2=0x55e93e9f0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:14.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.770+0000 7fbdb2800700 1 --2- 172.21.15.1:0/990913437 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e93e9ee7e0 0x55e93e9f0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:14.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.770+0000 7fbdb2800700 1 -- 172.21.15.1:0/990913437 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e93e7b63b0 con 0x55e93e9e3010 2024-02-02T15:35:14.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.770+0000 7fbdb3200700 1 --2- 172.21.15.1:0/990913437 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e93e9ee7e0 0x55e93e9f0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:35:14.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.770+0000 7fbdb2800700 1 --2- 172.21.15.1:0/990913437 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e93e9e3010 0x55e93e9e33f0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fbda4009f40 tx=0x7fbda4011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74a71279b6c1852b server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:14.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.770+0000 7fbdb3200700 1 --2- 172.21.15.1:0/4171908550 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fbd940a0690 0x7fbd940a2b50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fbdac009a60 tx=0x7fbdac003a50 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:14.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.771+0000 7fbda3400700 1 -- 172.21.15.1:0/990913437 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbda4019630 con 0x55e93e9e3010 2024-02-02T15:35:14.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.771+0000 7fbda3400700 1 -- 172.21.15.1:0/990913437 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbda4019c50 con 0x55e93e9e3010 2024-02-02T15:35:14.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.771+0000 7fbda3400700 1 -- 172.21.15.1:0/990913437 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbda400ecb0 con 0x55e93e9e3010 2024-02-02T15:35:14.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.772+0000 7fbdccfa4b80 1 -- 172.21.15.1:0/990913437 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e93e9e3010 msgr2=0x55e93e9e33f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:14.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.772+0000 7fbdccfa4b80 1 --2- 172.21.15.1:0/990913437 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e93e9e3010 0x55e93e9e33f0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fbda4009f40 tx=0x7fbda4011720 comp rx=0 tx=0).stop 2024-02-02T15:35:14.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.772+0000 7fbdccfa4b80 1 -- 172.21.15.1:0/990913437 shutdown_connections 2024-02-02T15:35:14.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.772+0000 7fbdccfa4b80 1 --2- 172.21.15.1:0/990913437 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e93e9ee7e0 0x55e93e9f0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:14.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.772+0000 7fbdccfa4b80 1 --2- 172.21.15.1:0/990913437 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e93e9e3010 0x55e93e9e33f0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:14.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.772+0000 7fbdccfa4b80 1 --2- 172.21.15.1:0/990913437 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e93e9e3930 0x55e93e9ee2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:14.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.772+0000 7fbdccfa4b80 1 -- 172.21.15.1:0/990913437 >> 172.21.15.1:0/990913437 conn(0x55e93e942db0 msgr2=0x55e93e9df3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:14.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.772+0000 7fbdccfa4b80 1 -- 172.21.15.1:0/990913437 shutdown_connections 2024-02-02T15:35:14.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.772+0000 7fbdccfa4b80 1 -- 172.21.15.1:0/990913437 wait complete. 2024-02-02T15:35:14.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.773+0000 7fbdccfa4b80 1 -- start start 2024-02-02T15:35:14.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.773+0000 7fbdccfa4b80 1 Processor -- start 2024-02-02T15:35:14.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.773+0000 7fbdccfa4b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e93e9e3010 0x55e93e9ee240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:14.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.773+0000 7fbdccfa4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e93e9e3930 0x55e93e9eb450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:14.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.773+0000 7fbdccfa4b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e93e9ee7e0 0x55e93e9ebb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:14.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.773+0000 7fbdccfa4b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e93e934d50 con 0x55e93e9e3010 2024-02-02T15:35:14.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.773+0000 7fbdccfa4b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e93e8d0d80 con 0x55e93e9ee7e0 2024-02-02T15:35:14.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.773+0000 7fbdccfa4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e93e8d0a20 con 0x55e93e9e3930 2024-02-02T15:35:14.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.773+0000 7fbdccfa4b80 10 client.0 _ll_register_callbacks cb 0x55e93e8d2290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:35:14.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.774+0000 7fbdb1e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e93e9e3930 0x55e93e9eb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:14.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.774+0000 7fbdb1e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e93e9e3930 0x55e93e9eb450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:56028/0 (socket says 172.21.15.1:56028) 2024-02-02T15:35:14.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.774+0000 7fbdb1e00700 1 -- 172.21.15.1:0/2739586848 learned_addr learned my addr 172.21.15.1:0/2739586848 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:35:14.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.774+0000 7fbdb3200700 1 --2- 172.21.15.1:0/2739586848 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e93e9ee7e0 0x55e93e9ebb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:14.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.774+0000 7fbdb2800700 1 --2- 172.21.15.1:0/2739586848 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e93e9e3010 0x55e93e9ee240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:14.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.774+0000 7fbdb1e00700 1 -- 172.21.15.1:0/2739586848 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e93e9ee7e0 msgr2=0x55e93e9ebb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:14.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.774+0000 7fbdb1e00700 1 --2- 172.21.15.1:0/2739586848 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e93e9ee7e0 0x55e93e9ebb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:14.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.774+0000 7fbdb1e00700 1 -- 172.21.15.1:0/2739586848 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e93e9e3010 msgr2=0x55e93e9ee240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:14.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.774+0000 7fbdb1e00700 1 --2- 172.21.15.1:0/2739586848 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e93e9e3010 0x55e93e9ee240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:14.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.774+0000 7fbdb1e00700 1 -- 172.21.15.1:0/2739586848 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbda4009b70 con 0x55e93e9e3930 2024-02-02T15:35:14.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.774+0000 7fbdb3200700 1 --2- 172.21.15.1:0/2739586848 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e93e9ee7e0 0x55e93e9ebb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:35:14.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.775+0000 7fbdb1e00700 1 --2- 172.21.15.1:0/2739586848 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e93e9e3930 0x55e93e9eb450 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c096ae0 tx=0x7fbd9c09a030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5844aea2a966eab server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:14.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.775+0000 7fbd9be00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:35:14.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.775+0000 7fbd9be00700 1 -- 172.21.15.1:0/2739586848 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c09a5e0 con 0x55e93e9e3930 2024-02-02T15:35:14.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.775+0000 7fbd9be00700 1 -- 172.21.15.1:0/2739586848 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fbd9c09e410 con 0x55e93e9e3930 2024-02-02T15:35:14.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.775+0000 7fbdccfa4b80 1 -- 172.21.15.1:0/2739586848 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e93e9d70f0 con 0x55e93e9e3930 2024-02-02T15:35:14.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.775+0000 7fbd9be00700 1 -- 172.21.15.1:0/2739586848 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbd9c0a25c0 con 0x55e93e9e3930 2024-02-02T15:35:14.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.775+0000 7fbdccfa4b80 10 client.27790 fetch_fsmap learned FSMap version 42 2024-02-02T15:35:14.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.776+0000 7fbdccfa4b80 1 -- 172.21.15.1:0/2739586848 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55e93e762b60 con 0x55e93e9e3930 2024-02-02T15:35:14.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.776+0000 7fbd9be00700 1 -- 172.21.15.1:0/2739586848 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e93e9d70f0 con 0x55e93e9e3930 2024-02-02T15:35:14.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.776+0000 7fbd9be00700 1 -- 172.21.15.1:0/2739586848 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 42) v1 ==== 7510+0+0 (secure 0 0 0) 0x7fbd9c099ad0 con 0x55e93e9e3930 2024-02-02T15:35:14.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.776+0000 7fbdccfa4b80 10 client.27790 fetch_fsmap finished waiting for FSMap version 42 2024-02-02T15:35:14.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.776+0000 7fbdccfa4b80 10 client.27790 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-02-02T15:35:14.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.776+0000 7fbdccfa4b80 20 client.27790 populate_metadata read hostname 'smithi001' 2024-02-02T15:35:14.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.776+0000 7fbdccfa4b80 1 --2- 172.21.15.1:0/2739586848 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55e93e9fde10 0x55e93ea00230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:14.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.776+0000 7fbdccfa4b80 4 client.27790 mds_command: new command op to 24502 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:35:14.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.776+0000 7fbdccfa4b80 1 -- 172.21.15.1:0/2739586848 --> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e93e92ddd0 con 0x55e93e9fde10 2024-02-02T15:35:14.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.776+0000 7fbdb2800700 1 --2- 172.21.15.1:0/2739586848 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55e93e9fde10 0x55e93ea00230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:14.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.777+0000 7fbdb2800700 1 --2- 172.21.15.1:0/2739586848 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55e93e9fde10 0x55e93ea00230 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=96d54e9940e3ab22 server_cookie=162a71eba090c8f6 in_seq=0 out_seq=0 2024-02-02T15:35:14.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.777+0000 7fbd9be00700 10 client.27790 ms_handle_connect on v2:172.21.15.142:6834/1052606547 2024-02-02T15:35:14.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.778+0000 7fbd9be00700 1 -- 172.21.15.1:0/2739586848 <== mds.4 v2:172.21.15.142:6834/1052606547 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55e93e92ddd0 con 0x55e93e9fde10 2024-02-02T15:35:14.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.778+0000 7fbd9be00700 10 client.27790 handle_command_reply: tid=0 2024-02-02T15:35:14.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.778+0000 7fbdccfa4b80 1 client.27790 shutdown 2024-02-02T15:35:14.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.778+0000 7fbdccfa4b80 1 -- 172.21.15.1:0/2739586848 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e93e9e3930 msgr2=0x55e93e9eb450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:14.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.778+0000 7fbdccfa4b80 1 --2- 172.21.15.1:0/2739586848 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e93e9e3930 0x55e93e9eb450 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c096ae0 tx=0x7fbd9c09a030 comp rx=0 tx=0).stop 2024-02-02T15:35:14.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.778+0000 7fbdccfa4b80 1 -- 172.21.15.1:0/2739586848 shutdown_connections 2024-02-02T15:35:14.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.778+0000 7fbdccfa4b80 1 --2- 172.21.15.1:0/2739586848 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55e93e9fde10 0x55e93ea00230 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:14.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.778+0000 7fbdccfa4b80 1 --2- 172.21.15.1:0/2739586848 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e93e9ee7e0 0x55e93e9ebb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:14.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.778+0000 7fbdccfa4b80 1 --2- 172.21.15.1:0/2739586848 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e93e9e3010 0x55e93e9ee240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:14.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.778+0000 7fbdccfa4b80 1 --2- 172.21.15.1:0/2739586848 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e93e9e3930 0x55e93e9eb450 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:14.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.778+0000 7fbdccfa4b80 1 -- 172.21.15.1:0/2739586848 >> 172.21.15.1:0/2739586848 conn(0x55e93e942db0 msgr2=0x55e93e9defa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:14.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.778+0000 7fbd9be00700 0 client.27790 ms_handle_reset on v2:172.21.15.142:6834/1052606547 2024-02-02T15:35:14.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.778+0000 7fbdccfa4b80 1 -- 172.21.15.1:0/2739586848 shutdown_connections 2024-02-02T15:35:14.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.778+0000 7fbdccfa4b80 1 -- 172.21.15.1:0/2739586848 wait complete. 2024-02-02T15:35:14.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.779+0000 7fbdccfa4b80 20 client.27790 trim_cache size 0 max 16384 2024-02-02T15:35:14.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.794+0000 7fbdccfa4b80 1 Processor -- start 2024-02-02T15:35:14.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.794+0000 7fbdccfa4b80 1 -- start start 2024-02-02T15:35:14.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.795+0000 7fbdccfa4b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e93e9e3010 0x55e93e9f5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:14.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.795+0000 7fbdccfa4b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e93e9ee7e0 0x55e93e9edd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:14.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.795+0000 7fbdccfa4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e93e9e6e50 0x55e93e9f9900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:14.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.795+0000 7fbdccfa4b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e93e7b86a0 con 0x55e93e9e3010 2024-02-02T15:35:14.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.795+0000 7fbdccfa4b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e93e92de00 con 0x55e93e9ee7e0 2024-02-02T15:35:14.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.795+0000 7fbdccfa4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e93e9e6240 con 0x55e93e9e6e50 2024-02-02T15:35:14.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.795+0000 7fbdb2800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e93e9e3010 0x55e93e9f5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:14.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.795+0000 7fbdb2800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e93e9e3010 0x55e93e9f5330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46288/0 (socket says 172.21.15.1:46288) 2024-02-02T15:35:14.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.795+0000 7fbdb2800700 1 -- 172.21.15.1:0/3645352539 learned_addr learned my addr 172.21.15.1:0/3645352539 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:35:14.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.795+0000 7fbdb3200700 1 --2- 172.21.15.1:0/3645352539 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e93e9e6e50 0x55e93e9f9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:14.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.796+0000 7fbdb1e00700 1 --2- 172.21.15.1:0/3645352539 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e93e9ee7e0 0x55e93e9edd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:14.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.796+0000 7fbdb3200700 1 -- 172.21.15.1:0/3645352539 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e93e9ee7e0 msgr2=0x55e93e9edd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:14.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.796+0000 7fbdb3200700 1 --2- 172.21.15.1:0/3645352539 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e93e9ee7e0 0x55e93e9edd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:14.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.796+0000 7fbdb3200700 1 -- 172.21.15.1:0/3645352539 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e93e9e3010 msgr2=0x55e93e9f5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:14.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.796+0000 7fbdb3200700 1 --2- 172.21.15.1:0/3645352539 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e93e9e3010 0x55e93e9f5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:14.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.796+0000 7fbdb3200700 1 -- 172.21.15.1:0/3645352539 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e93e9fa910 con 0x55e93e9e6e50 2024-02-02T15:35:14.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.796+0000 7fbdb3200700 1 --2- 172.21.15.1:0/3645352539 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e93e9e6e50 0x55e93e9f9900 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fbdac004320 tx=0x7fbdac010000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aefa557eae1f32bd server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:14.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.797+0000 7fbda3e00700 1 -- 172.21.15.1:0/3645352539 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdac0089a0 con 0x55e93e9e6e50 2024-02-02T15:35:14.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.797+0000 7fbda3e00700 1 -- 172.21.15.1:0/3645352539 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbdac00f040 con 0x55e93e9e6e50 2024-02-02T15:35:14.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.797+0000 7fbda3e00700 1 -- 172.21.15.1:0/3645352539 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdac019d50 con 0x55e93e9e6e50 2024-02-02T15:35:14.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.797+0000 7fbdccfa4b80 1 -- 172.21.15.1:0/3645352539 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e93e9e6e50 msgr2=0x55e93e9f9900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:14.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.797+0000 7fbdccfa4b80 1 --2- 172.21.15.1:0/3645352539 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e93e9e6e50 0x55e93e9f9900 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fbdac004320 tx=0x7fbdac010000 comp rx=0 tx=0).stop 2024-02-02T15:35:14.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.798+0000 7fbdccfa4b80 1 -- 172.21.15.1:0/3645352539 shutdown_connections 2024-02-02T15:35:14.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.798+0000 7fbdccfa4b80 1 --2- 172.21.15.1:0/3645352539 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e93e9e6e50 0x55e93e9f9900 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:14.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.798+0000 7fbdccfa4b80 1 --2- 172.21.15.1:0/3645352539 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e93e9e3010 0x55e93e9f5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:14.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.798+0000 7fbdccfa4b80 1 --2- 172.21.15.1:0/3645352539 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e93e9ee7e0 0x55e93e9edd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:14.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.798+0000 7fbdccfa4b80 1 -- 172.21.15.1:0/3645352539 >> 172.21.15.1:0/3645352539 conn(0x55e93e942db0 msgr2=0x55e93e9e6740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:14.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.798+0000 7fbdccfa4b80 1 -- 172.21.15.1:0/3645352539 shutdown_connections 2024-02-02T15:35:14.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.798+0000 7fbdccfa4b80 1 -- 172.21.15.1:0/3645352539 wait complete. 2024-02-02T15:35:14.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.798+0000 7fbdccfa4b80 1 -- start start 2024-02-02T15:35:14.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.798+0000 7fbdccfa4b80 1 Processor -- start 2024-02-02T15:35:14.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.799+0000 7fbdccfa4b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e93e9e3010 0x55e93e9f4c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:14.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.799+0000 7fbdccfa4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e93e9e6e50 0x55e93e9f5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:14.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.799+0000 7fbdccfa4b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e93e9ee7e0 0x55e93e9f2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:14.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.799+0000 7fbdccfa4b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e93e9f36d0 con 0x55e93e9ee7e0 2024-02-02T15:35:14.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.799+0000 7fbdccfa4b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e93e9f3840 con 0x55e93e9e3010 2024-02-02T15:35:14.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.799+0000 7fbdccfa4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e93e9f3a00 con 0x55e93e9e6e50 2024-02-02T15:35:14.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.799+0000 7fbdccfa4b80 10 client.0 _ll_register_callbacks cb 0x55e93e8d2290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:35:14.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.799+0000 7fbdb1e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e93e9e6e50 0x55e93e9f5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:14.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.799+0000 7fbdb2800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e93e9e3010 0x55e93e9f4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:14.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.799+0000 7fbdb1e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e93e9e6e50 0x55e93e9f5140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:56048/0 (socket says 172.21.15.1:56048) 2024-02-02T15:35:14.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.799+0000 7fbdb1e00700 1 -- 172.21.15.1:0/1862074911 learned_addr learned my addr 172.21.15.1:0/1862074911 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:35:14.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.799+0000 7fbdb3200700 1 --2- 172.21.15.1:0/1862074911 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e93e9ee7e0 0x55e93e9f2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:14.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.800+0000 7fbdb1e00700 1 -- 172.21.15.1:0/1862074911 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e93e9e3010 msgr2=0x55e93e9f4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:14.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.800+0000 7fbdb1e00700 1 --2- 172.21.15.1:0/1862074911 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e93e9e3010 0x55e93e9f4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:14.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.800+0000 7fbdb1e00700 1 -- 172.21.15.1:0/1862074911 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e93e9ee7e0 msgr2=0x55e93e9f2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:14.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.800+0000 7fbdb1e00700 1 --2- 172.21.15.1:0/1862074911 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e93e9ee7e0 0x55e93e9f2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:14.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.800+0000 7fbdb1e00700 1 -- 172.21.15.1:0/1862074911 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdac004450 con 0x55e93e9e6e50 2024-02-02T15:35:14.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.800+0000 7fbdb3200700 1 --2- 172.21.15.1:0/1862074911 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e93e9ee7e0 0x55e93e9f2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:35:14.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.800+0000 7fbdb1e00700 1 --2- 172.21.15.1:0/1862074911 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e93e9e6e50 0x55e93e9f5140 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c0a0a80 tx=0x7fbd9c0a0930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=efa5c9661bff3170 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:14.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.800+0000 7fbd9be00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:35:14.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.800+0000 7fbd9be00700 1 -- 172.21.15.1:0/1862074911 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9c0b8540 con 0x55e93e9e6e50 2024-02-02T15:35:14.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.800+0000 7fbd9be00700 1 -- 172.21.15.1:0/1862074911 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fbd9c0850b0 con 0x55e93e9e6e50 2024-02-02T15:35:14.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.800+0000 7fbdccfa4b80 1 -- 172.21.15.1:0/1862074911 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbda4019c50 con 0x55e93e9e6e50 2024-02-02T15:35:14.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.800+0000 7fbd9be00700 1 -- 172.21.15.1:0/1862074911 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbd9c0852c0 con 0x55e93e9e6e50 2024-02-02T15:35:14.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.800+0000 7fbdccfa4b80 10 client.27796 fetch_fsmap learned FSMap version 42 2024-02-02T15:35:14.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.800+0000 7fbdccfa4b80 1 -- 172.21.15.1:0/1862074911 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55e93e8faea0 con 0x55e93e9e6e50 2024-02-02T15:35:14.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.801+0000 7fbd9be00700 1 -- 172.21.15.1:0/1862074911 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbd9c07f020 con 0x55e93e9e6e50 2024-02-02T15:35:14.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.801+0000 7fbd9be00700 1 -- 172.21.15.1:0/1862074911 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 42) v1 ==== 7510+0+0 (secure 0 0 0) 0x7fbd9c09e040 con 0x55e93e9e6e50 2024-02-02T15:35:14.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.801+0000 7fbdccfa4b80 10 client.27796 fetch_fsmap finished waiting for FSMap version 42 2024-02-02T15:35:14.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.801+0000 7fbdccfa4b80 10 client.27796 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-02-02T15:35:14.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.801+0000 7fbdccfa4b80 20 client.27796 populate_metadata read hostname 'smithi001' 2024-02-02T15:35:14.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.801+0000 7fbdccfa4b80 1 --2- 172.21.15.1:0/1862074911 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55e93ea505a0 0x55e93ea529c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:14.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.801+0000 7fbdccfa4b80 4 client.27796 mds_command: new command op to 24502 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:35:14.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.801+0000 7fbdccfa4b80 1 -- 172.21.15.1:0/1862074911 --> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e93e8f3f50 con 0x55e93ea505a0 2024-02-02T15:35:14.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.801+0000 7fbdb2800700 1 --2- 172.21.15.1:0/1862074911 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55e93ea505a0 0x55e93ea529c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:14.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.802+0000 7fbdb2800700 1 --2- 172.21.15.1:0/1862074911 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55e93ea505a0 0x55e93ea529c0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a6bf478894e83f30 server_cookie=e2f8b65d613a08d5 in_seq=0 out_seq=0 2024-02-02T15:35:14.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.802+0000 7fbd9be00700 10 client.27796 ms_handle_connect on v2:172.21.15.142:6834/1052606547 2024-02-02T15:35:14.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.802+0000 7fbd9be00700 1 -- 172.21.15.1:0/1862074911 <== mds.4 v2:172.21.15.142:6834/1052606547 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e93e8f3f50 con 0x55e93ea505a0 2024-02-02T15:35:14.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.802+0000 7fbd9be00700 10 client.27796 handle_command_reply: tid=0 2024-02-02T15:35:14.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.802+0000 7fbdccfa4b80 1 client.27796 shutdown 2024-02-02T15:35:14.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.802+0000 7fbdccfa4b80 1 -- 172.21.15.1:0/1862074911 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e93e9e6e50 msgr2=0x55e93e9f5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:14.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.802+0000 7fbdccfa4b80 1 --2- 172.21.15.1:0/1862074911 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e93e9e6e50 0x55e93e9f5140 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c0a0a80 tx=0x7fbd9c0a0930 comp rx=0 tx=0).stop 2024-02-02T15:35:14.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.803+0000 7fbdccfa4b80 1 -- 172.21.15.1:0/1862074911 shutdown_connections 2024-02-02T15:35:14.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.803+0000 7fbdccfa4b80 1 --2- 172.21.15.1:0/1862074911 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55e93ea505a0 0x55e93ea529c0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:14.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.803+0000 7fbdccfa4b80 1 --2- 172.21.15.1:0/1862074911 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e93e9ee7e0 0x55e93e9f2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:14.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.803+0000 7fbdccfa4b80 1 --2- 172.21.15.1:0/1862074911 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e93e9e3010 0x55e93e9f4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:14.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.803+0000 7fbdccfa4b80 1 --2- 172.21.15.1:0/1862074911 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e93e9e6e50 0x55e93e9f5140 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:14.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.803+0000 7fbdccfa4b80 1 -- 172.21.15.1:0/1862074911 >> 172.21.15.1:0/1862074911 conn(0x55e93e942db0 msgr2=0x55e93e9f9900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:14.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.803+0000 7fbd9be00700 0 client.27796 ms_handle_reset on v2:172.21.15.142:6834/1052606547 2024-02-02T15:35:14.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.803+0000 7fbdccfa4b80 1 -- 172.21.15.1:0/1862074911 shutdown_connections 2024-02-02T15:35:14.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.803+0000 7fbdccfa4b80 1 -- 172.21.15.1:0/1862074911 wait complete. 2024-02-02T15:35:14.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.803+0000 7fbdccfa4b80 20 client.27796 trim_cache size 0 max 16384 2024-02-02T15:35:14.806 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:35:14.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.804+0000 7fbdb9400700 1 -- 172.21.15.1:0/4171908550 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fbd940a0690 msgr2=0x7fbd940a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:14.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.804+0000 7fbdb9400700 1 --2- 172.21.15.1:0/4171908550 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fbd940a0690 0x7fbd940a2b50 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fbdac009a60 tx=0x7fbdac003a50 comp rx=0 tx=0).stop 2024-02-02T15:35:14.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.804+0000 7fbdb9400700 1 -- 172.21.15.1:0/4171908550 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbdb4108320 msgr2=0x7fbdb41995b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:14.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.804+0000 7fbdb9400700 1 --2- 172.21.15.1:0/4171908550 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbdb4108320 0x7fbdb41995b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c000c00 tx=0x7fbd9c005740 comp rx=0 tx=0).stop 2024-02-02T15:35:14.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.804+0000 7fbdb9400700 1 -- 172.21.15.1:0/4171908550 shutdown_connections 2024-02-02T15:35:14.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.804+0000 7fbdb9400700 1 --2- 172.21.15.1:0/4171908550 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fbd940a0690 0x7fbd940a2b50 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:14.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.804+0000 7fbdb9400700 1 --2- 172.21.15.1:0/4171908550 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbdb419da10 0x7fbdb419de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:14.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.804+0000 7fbdb9400700 1 --2- 172.21.15.1:0/4171908550 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbdb4103200 0x7fbdb4199070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:14.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.804+0000 7fbdb9400700 1 --2- 172.21.15.1:0/4171908550 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbdb4108320 0x7fbdb41995b0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:14.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.804+0000 7fbdb9400700 1 -- 172.21.15.1:0/4171908550 >> 172.21.15.1:0/4171908550 conn(0x7fbdb40feaa0 msgr2=0x7fbdb40ff5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:14.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.804+0000 7fbdb9400700 1 -- 172.21.15.1:0/4171908550 shutdown_connections 2024-02-02T15:35:14.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:14.804+0000 7fbdb9400700 1 -- 172.21.15.1:0/4171908550 wait complete. 2024-02-02T15:35:14.825 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-02-02T15:35:15.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.049+0000 7f52ae000700 1 -- 172.21.15.1:0/1345816298 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f52a80730e0 msgr2=0x7f52a80734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:15.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.049+0000 7f52ae000700 1 --2- 172.21.15.1:0/1345816298 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f52a80730e0 0x7f52a80734c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f5298009990 tx=0x7f5298009ca0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.049+0000 7f52ae000700 1 -- 172.21.15.1:0/1345816298 shutdown_connections 2024-02-02T15:35:15.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.049+0000 7f52ae000700 1 --2- 172.21.15.1:0/1345816298 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52a8111d90 0x7f52a8114180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.049+0000 7f52ae000700 1 --2- 172.21.15.1:0/1345816298 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f52a80730e0 0x7f52a80734c0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.049+0000 7f52ae000700 1 --2- 172.21.15.1:0/1345816298 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f52a8073a00 0x7f52a8111850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.049+0000 7f52ae000700 1 -- 172.21.15.1:0/1345816298 >> 172.21.15.1:0/1345816298 conn(0x7f52a80fc810 msgr2=0x7f52a80fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:15.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.050+0000 7f52ae000700 1 -- 172.21.15.1:0/1345816298 shutdown_connections 2024-02-02T15:35:15.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.050+0000 7f52ae000700 1 -- 172.21.15.1:0/1345816298 wait complete. 2024-02-02T15:35:15.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.051+0000 7f52ae000700 1 Processor -- start 2024-02-02T15:35:15.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.051+0000 7f52ae000700 1 -- start start 2024-02-02T15:35:15.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.051+0000 7f52ae000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f52a80730e0 0x7f52a819d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:15.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.051+0000 7f52ae000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f52a8073a00 0x7f52a819da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:15.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.051+0000 7f52ae000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52a8111d90 0x7f52a81a1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:15.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.051+0000 7f52ae000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52a81a24d0 con 0x7f52a8073a00 2024-02-02T15:35:15.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.051+0000 7f52ae000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52a81a2640 con 0x7f52a8111d90 2024-02-02T15:35:15.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.051+0000 7f52ae000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52a81a2800 con 0x7f52a80730e0 2024-02-02T15:35:15.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.052+0000 7f52a7e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52a8111d90 0x7f52a81a1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:15.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.052+0000 7f52a7400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f52a80730e0 0x7f52a819d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:15.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.052+0000 7f52a7400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f52a80730e0 0x7f52a819d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:56064/0 (socket says 172.21.15.1:56064) 2024-02-02T15:35:15.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.052+0000 7f52a7e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52a8111d90 0x7f52a81a1e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:54860/0 (socket says 172.21.15.1:54860) 2024-02-02T15:35:15.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.052+0000 7f52a7400700 1 -- 172.21.15.1:0/4100756680 learned_addr learned my addr 172.21.15.1:0/4100756680 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:35:15.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.052+0000 7f52a6a00700 1 --2- 172.21.15.1:0/4100756680 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f52a8073a00 0x7f52a819da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:15.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.052+0000 7f52a7e00700 1 -- 172.21.15.1:0/4100756680 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f52a80730e0 msgr2=0x7f52a819d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:15.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.052+0000 7f52a7e00700 1 --2- 172.21.15.1:0/4100756680 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f52a80730e0 0x7f52a819d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.052+0000 7f52a7e00700 1 -- 172.21.15.1:0/4100756680 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f52a8073a00 msgr2=0x7f52a819da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:15.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.052+0000 7f52a7e00700 1 --2- 172.21.15.1:0/4100756680 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f52a8073a00 0x7f52a819da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.052+0000 7f52a7e00700 1 -- 172.21.15.1:0/4100756680 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5290009520 con 0x7f52a8111d90 2024-02-02T15:35:15.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.052+0000 7f52a7e00700 1 --2- 172.21.15.1:0/4100756680 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52a8111d90 0x7f52a81a1e90 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f52a000ef90 tx=0x7f52a000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:15.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.052+0000 7f529fe00700 1 -- 172.21.15.1:0/4100756680 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a0018070 con 0x7f52a8111d90 2024-02-02T15:35:15.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.053+0000 7f529fe00700 1 -- 172.21.15.1:0/4100756680 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f52a000f040 con 0x7f52a8111d90 2024-02-02T15:35:15.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.053+0000 7f529fe00700 1 -- 172.21.15.1:0/4100756680 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a000cc30 con 0x7f52a8111d90 2024-02-02T15:35:15.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.053+0000 7f52ae000700 1 -- 172.21.15.1:0/4100756680 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52980095f0 con 0x7f52a8111d90 2024-02-02T15:35:15.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.053+0000 7f52ae000700 1 -- 172.21.15.1:0/4100756680 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52a81aa140 con 0x7f52a8111d90 2024-02-02T15:35:15.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.056+0000 7f529fe00700 1 -- 172.21.15.1:0/4100756680 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f52a0003a00 con 0x7f52a8111d90 2024-02-02T15:35:15.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.057+0000 7f52c1bfbb80 1 Processor -- start 2024-02-02T15:35:15.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.057+0000 7f529fe00700 1 --2- 172.21.15.1:0/4100756680 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f52880a0710 0x7f52880a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:15.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.057+0000 7f529fe00700 1 -- 172.21.15.1:0/4100756680 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f52a00cbbb0 con 0x7f52a8111d90 2024-02-02T15:35:15.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.057+0000 7f52c1bfbb80 1 -- start start 2024-02-02T15:35:15.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.058+0000 7f52a6a00700 1 --2- 172.21.15.1:0/4100756680 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f52880a0710 0x7f52880a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:15.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.058+0000 7f52c1bfbb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559f115e4010 0x559f115e43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:15.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.058+0000 7f52c1bfbb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559f115e4930 0x559f115ef2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:15.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.058+0000 7f52c1bfbb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559f115ef7e0 0x559f115f1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:15.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.058+0000 7f52c1bfbb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f115584c0 con 0x559f115e4930 2024-02-02T15:35:15.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.058+0000 7f52c1bfbb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f1153caa0 con 0x559f115ef7e0 2024-02-02T15:35:15.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.058+0000 7f52c1bfbb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f1153a460 con 0x559f115e4010 2024-02-02T15:35:15.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.058+0000 7f52a7e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559f115ef7e0 0x559f115f1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:15.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.058+0000 7f52a7e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559f115ef7e0 0x559f115f1bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:54868/0 (socket says 172.21.15.1:54868) 2024-02-02T15:35:15.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.058+0000 7f52a7400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559f115e4010 0x559f115e43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:15.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.058+0000 7f52a7e00700 1 -- 172.21.15.1:0/703445096 learned_addr learned my addr 172.21.15.1:0/703445096 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:35:15.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.058+0000 7f52a7e00700 1 -- 172.21.15.1:0/703445096 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559f115e4010 msgr2=0x559f115e43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:15.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.059+0000 7f52a7e00700 1 --2- 172.21.15.1:0/703445096 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559f115e4010 0x559f115e43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.059+0000 7f52a7e00700 1 -- 172.21.15.1:0/703445096 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559f115e4930 msgr2=0x559f115ef2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:15.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.059+0000 7f52a6a00700 1 --2- 172.21.15.1:0/703445096 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559f115e4930 0x559f115ef2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:15.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.059+0000 7f52a7e00700 1 --2- 172.21.15.1:0/703445096 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559f115e4930 0x559f115ef2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.059+0000 7f52a7e00700 1 -- 172.21.15.1:0/703445096 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559f113b73b0 con 0x559f115ef7e0 2024-02-02T15:35:15.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.059+0000 7f52a7400700 1 --2- 172.21.15.1:0/703445096 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559f115e4010 0x559f115e43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:35:15.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.059+0000 7f52a6a00700 1 --2- 172.21.15.1:0/703445096 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559f115e4930 0x559f115ef2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:35:15.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.059+0000 7f52a6a00700 1 --2- 172.21.15.1:0/4100756680 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f52880a0710 0x7f52880a2bd0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f52a819eb00 tx=0x7f5290011720 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:15.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.059+0000 7f52a7e00700 1 --2- 172.21.15.1:0/703445096 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559f115ef7e0 0x559f115f1bd0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f52a00c8ea0 tx=0x7f52a0098920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=701ad7a8a41f4b71 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:15.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.059+0000 7f5297e00700 1 -- 172.21.15.1:0/703445096 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a0099460 con 0x559f115ef7e0 2024-02-02T15:35:15.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.059+0000 7f5297e00700 1 -- 172.21.15.1:0/703445096 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f52a0098d60 con 0x559f115ef7e0 2024-02-02T15:35:15.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.059+0000 7f5297e00700 1 -- 172.21.15.1:0/703445096 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a0095470 con 0x559f115ef7e0 2024-02-02T15:35:15.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.060+0000 7f52c1bfbb80 1 -- 172.21.15.1:0/703445096 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559f115ef7e0 msgr2=0x559f115f1bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:15.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.060+0000 7f52c1bfbb80 1 --2- 172.21.15.1:0/703445096 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559f115ef7e0 0x559f115f1bd0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f52a00c8ea0 tx=0x7f52a0098920 comp rx=0 tx=0).stop 2024-02-02T15:35:15.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.060+0000 7f52c1bfbb80 1 -- 172.21.15.1:0/703445096 shutdown_connections 2024-02-02T15:35:15.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.060+0000 7f52c1bfbb80 1 --2- 172.21.15.1:0/703445096 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559f115ef7e0 0x559f115f1bd0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.060+0000 7f52c1bfbb80 1 --2- 172.21.15.1:0/703445096 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559f115e4010 0x559f115e43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.060+0000 7f52c1bfbb80 1 --2- 172.21.15.1:0/703445096 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559f115e4930 0x559f115ef2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.060+0000 7f52c1bfbb80 1 -- 172.21.15.1:0/703445096 >> 172.21.15.1:0/703445096 conn(0x559f11543db0 msgr2=0x559f115e03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:15.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.060+0000 7f52c1bfbb80 1 -- 172.21.15.1:0/703445096 shutdown_connections 2024-02-02T15:35:15.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.060+0000 7f52c1bfbb80 1 -- 172.21.15.1:0/703445096 wait complete. 2024-02-02T15:35:15.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.061+0000 7f52c1bfbb80 1 -- start start 2024-02-02T15:35:15.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.061+0000 7f52c1bfbb80 1 Processor -- start 2024-02-02T15:35:15.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.062+0000 7f52c1bfbb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559f115e4010 0x559f115ef280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:15.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.062+0000 7f52c1bfbb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559f115e4930 0x559f115ec490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:15.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.062+0000 7f52c1bfbb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559f115ef7e0 0x559f115ecbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:15.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.062+0000 7f52c1bfbb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f11535d50 con 0x559f115e4930 2024-02-02T15:35:15.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.062+0000 7f52c1bfbb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f114d1d80 con 0x559f115e4010 2024-02-02T15:35:15.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.062+0000 7f52c1bfbb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f114d1a20 con 0x559f115ef7e0 2024-02-02T15:35:15.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.062+0000 7f52c1bfbb80 10 client.0 _ll_register_callbacks cb 0x559f114d3290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:35:15.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.062+0000 7f52a7400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559f115e4010 0x559f115ef280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:15.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.062+0000 7f52a7400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559f115e4010 0x559f115ef280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:54876/0 (socket says 172.21.15.1:54876) 2024-02-02T15:35:15.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.062+0000 7f52a7400700 1 -- 172.21.15.1:0/3872389733 learned_addr learned my addr 172.21.15.1:0/3872389733 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:35:15.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.062+0000 7f52a6a00700 1 --2- 172.21.15.1:0/3872389733 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559f115e4930 0x559f115ec490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:15.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.062+0000 7f52a7e00700 1 --2- 172.21.15.1:0/3872389733 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559f115ef7e0 0x559f115ecbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:15.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.063+0000 7f52a7400700 1 -- 172.21.15.1:0/3872389733 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559f115ef7e0 msgr2=0x559f115ecbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:15.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.063+0000 7f52a7400700 1 --2- 172.21.15.1:0/3872389733 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559f115ef7e0 0x559f115ecbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.063+0000 7f52a7400700 1 -- 172.21.15.1:0/3872389733 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559f115e4930 msgr2=0x559f115ec490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:15.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.063+0000 7f52a7400700 1 --2- 172.21.15.1:0/3872389733 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559f115e4930 0x559f115ec490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.063+0000 7f52a7400700 1 -- 172.21.15.1:0/3872389733 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52a8114b30 con 0x559f115e4010 2024-02-02T15:35:15.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.063+0000 7f52a7e00700 1 --2- 172.21.15.1:0/3872389733 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559f115ef7e0 0x559f115ecbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:35:15.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.063+0000 7f52a7400700 1 --2- 172.21.15.1:0/3872389733 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559f115e4010 0x559f115ef280 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f529800b650 tx=0x7f529800b680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77cc14fc0fbcc7f8 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:15.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.063+0000 7f5294c00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:35:15.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.063+0000 7f5294c00700 1 -- 172.21.15.1:0/3872389733 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f529801d070 con 0x559f115e4010 2024-02-02T15:35:15.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.063+0000 7f5294c00700 1 -- 172.21.15.1:0/3872389733 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f5298004700 con 0x559f115e4010 2024-02-02T15:35:15.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.063+0000 7f52c1bfbb80 1 -- 172.21.15.1:0/3872389733 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559f115d80f0 con 0x559f115e4010 2024-02-02T15:35:15.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.064+0000 7f5294c00700 1 -- 172.21.15.1:0/3872389733 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f52980056f0 con 0x559f115e4010 2024-02-02T15:35:15.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.064+0000 7f52c1bfbb80 10 client.27812 fetch_fsmap learned FSMap version 42 2024-02-02T15:35:15.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.064+0000 7f52c1bfbb80 1 -- 172.21.15.1:0/3872389733 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7f52a81aa140 con 0x559f115e4010 2024-02-02T15:35:15.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.064+0000 7f5294c00700 1 -- 172.21.15.1:0/3872389733 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559f115d80f0 con 0x559f115e4010 2024-02-02T15:35:15.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.064+0000 7f5294c00700 1 -- 172.21.15.1:0/3872389733 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 42) v1 ==== 7510+0+0 (secure 0 0 0) 0x7f529800f960 con 0x559f115e4010 2024-02-02T15:35:15.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.064+0000 7f52c1bfbb80 10 client.27812 fetch_fsmap finished waiting for FSMap version 42 2024-02-02T15:35:15.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.064+0000 7f52c1bfbb80 10 client.27812 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-02-02T15:35:15.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.064+0000 7f52c1bfbb80 20 client.27812 populate_metadata read hostname 'smithi001' 2024-02-02T15:35:15.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.064+0000 7f52c1bfbb80 1 --2- 172.21.15.1:0/3872389733 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x559f115ff310 0x559f11601730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:15.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.064+0000 7f52c1bfbb80 4 client.27812 mds_command: new command op to 24514 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:35:15.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.064+0000 7f52c1bfbb80 1 -- 172.21.15.1:0/3872389733 --> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559f1152edd0 con 0x559f115ff310 2024-02-02T15:35:15.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.064+0000 7f52a6a00700 1 --2- 172.21.15.1:0/3872389733 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x559f115ff310 0x559f11601730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:15.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.065+0000 7f52a6a00700 1 --2- 172.21.15.1:0/3872389733 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x559f115ff310 0x559f11601730 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d88c93c388432fa0 server_cookie=9070ebc9094e62ce in_seq=0 out_seq=0 2024-02-02T15:35:15.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.065+0000 7f5294c00700 10 client.27812 ms_handle_connect on v2:172.21.15.142:6836/2870289019 2024-02-02T15:35:15.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.066+0000 7f5294c00700 1 -- 172.21.15.1:0/3872389733 <== mds.3 v2:172.21.15.142:6836/2870289019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x559f1152edd0 con 0x559f115ff310 2024-02-02T15:35:15.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.066+0000 7f5294c00700 10 client.27812 handle_command_reply: tid=0 2024-02-02T15:35:15.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.066+0000 7f52c1bfbb80 1 client.27812 shutdown 2024-02-02T15:35:15.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.066+0000 7f52c1bfbb80 1 -- 172.21.15.1:0/3872389733 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559f115e4010 msgr2=0x559f115ef280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:15.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.066+0000 7f52c1bfbb80 1 --2- 172.21.15.1:0/3872389733 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559f115e4010 0x559f115ef280 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f529800b650 tx=0x7f529800b680 comp rx=0 tx=0).stop 2024-02-02T15:35:15.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.066+0000 7f52c1bfbb80 1 -- 172.21.15.1:0/3872389733 shutdown_connections 2024-02-02T15:35:15.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.066+0000 7f52c1bfbb80 1 --2- 172.21.15.1:0/3872389733 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559f115ef7e0 0x559f115ecbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.066+0000 7f52c1bfbb80 1 --2- 172.21.15.1:0/3872389733 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559f115e4010 0x559f115ef280 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.066+0000 7f52c1bfbb80 1 --2- 172.21.15.1:0/3872389733 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x559f115ff310 0x559f11601730 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.066+0000 7f52c1bfbb80 1 --2- 172.21.15.1:0/3872389733 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559f115e4930 0x559f115ec490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.066+0000 7f52c1bfbb80 1 -- 172.21.15.1:0/3872389733 >> 172.21.15.1:0/3872389733 conn(0x559f11543db0 msgr2=0x559f115dffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:15.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.066+0000 7f5294c00700 0 client.27812 ms_handle_reset on v2:172.21.15.142:6836/2870289019 2024-02-02T15:35:15.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.066+0000 7f52c1bfbb80 1 -- 172.21.15.1:0/3872389733 shutdown_connections 2024-02-02T15:35:15.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.067+0000 7f52c1bfbb80 1 -- 172.21.15.1:0/3872389733 wait complete. 2024-02-02T15:35:15.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.067+0000 7f52c1bfbb80 20 client.27812 trim_cache size 0 max 16384 2024-02-02T15:35:15.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.092+0000 7f52c1bfbb80 1 Processor -- start 2024-02-02T15:35:15.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.092+0000 7f52c1bfbb80 1 -- start start 2024-02-02T15:35:15.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.093+0000 7f52c1bfbb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559f115e4010 0x559f115f6330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:15.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.093+0000 7f52c1bfbb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559f115e4930 0x559f115e5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:15.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.093+0000 7f52c1bfbb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559f115ef7e0 0x559f115eedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:15.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.093+0000 7f52c1bfbb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f113b96a0 con 0x559f115e4010 2024-02-02T15:35:15.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.093+0000 7f52c1bfbb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f115e1060 con 0x559f115ef7e0 2024-02-02T15:35:15.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.093+0000 7f52c1bfbb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f115d7cc0 con 0x559f115e4930 2024-02-02T15:35:15.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.093+0000 7f52a6a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559f115e4930 0x559f115e5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:15.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.093+0000 7f52a7400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559f115e4010 0x559f115f6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:15.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.093+0000 7f52a7e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559f115ef7e0 0x559f115eedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:15.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.093+0000 7f52a6a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559f115e4930 0x559f115e5f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:56086/0 (socket says 172.21.15.1:56086) 2024-02-02T15:35:15.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.093+0000 7f52a6a00700 1 -- 172.21.15.1:0/2667494706 learned_addr learned my addr 172.21.15.1:0/2667494706 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:35:15.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.093+0000 7f52a7e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559f115ef7e0 0x559f115eedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:54890/0 (socket says 172.21.15.1:54890) 2024-02-02T15:35:15.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.094+0000 7f52a7e00700 1 -- 172.21.15.1:0/2667494706 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559f115e4930 msgr2=0x559f115e5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:15.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.094+0000 7f52a7e00700 1 --2- 172.21.15.1:0/2667494706 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559f115e4930 0x559f115e5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.094+0000 7f52a7e00700 1 -- 172.21.15.1:0/2667494706 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559f115e4010 msgr2=0x559f115f6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:15.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.094+0000 7f52a7e00700 1 --2- 172.21.15.1:0/2667494706 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559f115e4010 0x559f115f6330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.094+0000 7f52a7e00700 1 -- 172.21.15.1:0/2667494706 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559f11363b60 con 0x559f115ef7e0 2024-02-02T15:35:15.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.094+0000 7f52a7e00700 1 --2- 172.21.15.1:0/2667494706 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559f115ef7e0 0x559f115eedb0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f52a0095260 tx=0x7f52a0000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b2dfba2430ff68a server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:15.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.094+0000 7f529d200700 1 -- 172.21.15.1:0/2667494706 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a0099460 con 0x559f115ef7e0 2024-02-02T15:35:15.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.094+0000 7f529d200700 1 -- 172.21.15.1:0/2667494706 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f52a00a0d40 con 0x559f115ef7e0 2024-02-02T15:35:15.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.094+0000 7f529d200700 1 -- 172.21.15.1:0/2667494706 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a0099220 con 0x559f115ef7e0 2024-02-02T15:35:15.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.095+0000 7f52c1bfbb80 1 -- 172.21.15.1:0/2667494706 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559f115ef7e0 msgr2=0x559f115eedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:15.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.095+0000 7f52c1bfbb80 1 --2- 172.21.15.1:0/2667494706 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559f115ef7e0 0x559f115eedb0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f52a0095260 tx=0x7f52a0000f40 comp rx=0 tx=0).stop 2024-02-02T15:35:15.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.095+0000 7f52c1bfbb80 1 -- 172.21.15.1:0/2667494706 shutdown_connections 2024-02-02T15:35:15.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.095+0000 7f52c1bfbb80 1 --2- 172.21.15.1:0/2667494706 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559f115ef7e0 0x559f115eedb0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.095+0000 7f52c1bfbb80 1 --2- 172.21.15.1:0/2667494706 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559f115e4010 0x559f115f6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.095+0000 7f52c1bfbb80 1 --2- 172.21.15.1:0/2667494706 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559f115e4930 0x559f115e5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.095+0000 7f52c1bfbb80 1 -- 172.21.15.1:0/2667494706 >> 172.21.15.1:0/2667494706 conn(0x559f11543db0 msgr2=0x559f115fb5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:15.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.095+0000 7f52c1bfbb80 1 -- 172.21.15.1:0/2667494706 shutdown_connections 2024-02-02T15:35:15.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.095+0000 7f52c1bfbb80 1 -- 172.21.15.1:0/2667494706 wait complete. 2024-02-02T15:35:15.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.096+0000 7f52c1bfbb80 1 -- start start 2024-02-02T15:35:15.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.096+0000 7f52c1bfbb80 1 Processor -- start 2024-02-02T15:35:15.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.097+0000 7f52c1bfbb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559f115e4010 0x559f115f4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:15.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.097+0000 7f52c1bfbb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559f115e4930 0x559f115f4850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:15.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.097+0000 7f52c1bfbb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559f115ef7e0 0x559f115f4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:15.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.097+0000 7f52c1bfbb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f1164ff70 con 0x559f115ef7e0 2024-02-02T15:35:15.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.097+0000 7f52c1bfbb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f116500e0 con 0x559f115e4010 2024-02-02T15:35:15.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.097+0000 7f52c1bfbb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f116502a0 con 0x559f115e4930 2024-02-02T15:35:15.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.097+0000 7f52c1bfbb80 10 client.0 _ll_register_callbacks cb 0x559f114d3290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:35:15.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.097+0000 7f52a7400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559f115e4010 0x559f115f4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:15.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.097+0000 7f52a7400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559f115e4010 0x559f115f4310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:54902/0 (socket says 172.21.15.1:54902) 2024-02-02T15:35:15.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.097+0000 7f52a7400700 1 -- 172.21.15.1:0/1565311196 learned_addr learned my addr 172.21.15.1:0/1565311196 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:35:15.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.097+0000 7f52a7e00700 1 --2- 172.21.15.1:0/1565311196 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559f115ef7e0 0x559f115f4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:15.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.097+0000 7f52a6a00700 1 --2- 172.21.15.1:0/1565311196 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559f115e4930 0x559f115f4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:15.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.097+0000 7f52a7400700 1 -- 172.21.15.1:0/1565311196 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559f115e4930 msgr2=0x559f115f4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:15.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.097+0000 7f52a7400700 1 --2- 172.21.15.1:0/1565311196 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559f115e4930 0x559f115f4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.097+0000 7f52a7400700 1 -- 172.21.15.1:0/1565311196 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559f115ef7e0 msgr2=0x559f115f4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:15.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.097+0000 7f52a7400700 1 --2- 172.21.15.1:0/1565311196 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559f115ef7e0 0x559f115f4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.097+0000 7f52a7400700 1 -- 172.21.15.1:0/1565311196 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559f113b73b0 con 0x559f115e4010 2024-02-02T15:35:15.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.098+0000 7f52a7400700 1 --2- 172.21.15.1:0/1565311196 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559f115e4010 0x559f115f4310 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f529800b6f0 tx=0x7f5298005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90386fde11914fa6 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:15.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.098+0000 7f5294c00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:35:15.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.098+0000 7f5294c00700 1 -- 172.21.15.1:0/1565311196 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f529801d040 con 0x559f115e4010 2024-02-02T15:35:15.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.098+0000 7f5294c00700 1 -- 172.21.15.1:0/1565311196 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f5298024450 con 0x559f115e4010 2024-02-02T15:35:15.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.098+0000 7f52c1bfbb80 1 -- 172.21.15.1:0/1565311196 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f52a0098d60 con 0x559f115e4010 2024-02-02T15:35:15.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.098+0000 7f5294c00700 1 -- 172.21.15.1:0/1565311196 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f52980056f0 con 0x559f115e4010 2024-02-02T15:35:15.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.098+0000 7f52c1bfbb80 10 client.27818 fetch_fsmap learned FSMap version 42 2024-02-02T15:35:15.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.098+0000 7f52c1bfbb80 1 -- 172.21.15.1:0/1565311196 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x559f114fbea0 con 0x559f115e4010 2024-02-02T15:35:15.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.099+0000 7f5294c00700 1 -- 172.21.15.1:0/1565311196 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f52a0098d60 con 0x559f115e4010 2024-02-02T15:35:15.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.099+0000 7f5294c00700 1 -- 172.21.15.1:0/1565311196 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 42) v1 ==== 7510+0+0 (secure 0 0 0) 0x7f5298016a80 con 0x559f115e4010 2024-02-02T15:35:15.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.099+0000 7f52c1bfbb80 10 client.27818 fetch_fsmap finished waiting for FSMap version 42 2024-02-02T15:35:15.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.099+0000 7f52c1bfbb80 10 client.27818 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-02-02T15:35:15.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.099+0000 7f52c1bfbb80 20 client.27818 populate_metadata read hostname 'smithi001' 2024-02-02T15:35:15.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.099+0000 7f52c1bfbb80 1 --2- 172.21.15.1:0/1565311196 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x559f11651530 0x559f11653950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:15.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.099+0000 7f52c1bfbb80 4 client.27818 mds_command: new command op to 24514 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:35:15.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.099+0000 7f52c1bfbb80 1 -- 172.21.15.1:0/1565311196 --> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559f114f4f50 con 0x559f11651530 2024-02-02T15:35:15.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.100+0000 7f52a6a00700 1 --2- 172.21.15.1:0/1565311196 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x559f11651530 0x559f11653950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:15.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.100+0000 7f52a6a00700 1 --2- 172.21.15.1:0/1565311196 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x559f11651530 0x559f11653950 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=780d4a8a00fded47 server_cookie=3a78bbaeaef18cdd in_seq=0 out_seq=0 2024-02-02T15:35:15.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.100+0000 7f5294c00700 10 client.27818 ms_handle_connect on v2:172.21.15.142:6836/2870289019 2024-02-02T15:35:15.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.100+0000 7f5294c00700 1 -- 172.21.15.1:0/1565311196 <== mds.3 v2:172.21.15.142:6836/2870289019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559f114f4f50 con 0x559f11651530 2024-02-02T15:35:15.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.100+0000 7f5294c00700 10 client.27818 handle_command_reply: tid=0 2024-02-02T15:35:15.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.100+0000 7f52c1bfbb80 1 client.27818 shutdown 2024-02-02T15:35:15.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.101+0000 7f52c1bfbb80 1 -- 172.21.15.1:0/1565311196 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559f115e4010 msgr2=0x559f115f4310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:15.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.101+0000 7f52c1bfbb80 1 --2- 172.21.15.1:0/1565311196 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559f115e4010 0x559f115f4310 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f529800b6f0 tx=0x7f5298005e80 comp rx=0 tx=0).stop 2024-02-02T15:35:15.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.101+0000 7f52c1bfbb80 1 -- 172.21.15.1:0/1565311196 shutdown_connections 2024-02-02T15:35:15.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.101+0000 7f52c1bfbb80 1 --2- 172.21.15.1:0/1565311196 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x559f11651530 0x559f11653950 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.101+0000 7f52c1bfbb80 1 --2- 172.21.15.1:0/1565311196 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559f115ef7e0 0x559f115f4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.101+0000 7f52c1bfbb80 1 --2- 172.21.15.1:0/1565311196 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559f115e4010 0x559f115f4310 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.101+0000 7f5294c00700 0 client.27818 ms_handle_reset on v2:172.21.15.142:6836/2870289019 2024-02-02T15:35:15.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.101+0000 7f52c1bfbb80 1 --2- 172.21.15.1:0/1565311196 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559f115e4930 0x559f115f4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.101+0000 7f52c1bfbb80 1 -- 172.21.15.1:0/1565311196 >> 172.21.15.1:0/1565311196 conn(0x559f11543db0 msgr2=0x559f115e5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:15.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.101+0000 7f52c1bfbb80 1 -- 172.21.15.1:0/1565311196 shutdown_connections 2024-02-02T15:35:15.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.101+0000 7f52c1bfbb80 1 -- 172.21.15.1:0/1565311196 wait complete. 2024-02-02T15:35:15.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.101+0000 7f52c1bfbb80 20 client.27818 trim_cache size 0 max 16384 2024-02-02T15:35:15.104 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:35:15.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.102+0000 7f52ae000700 1 -- 172.21.15.1:0/4100756680 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f52880a0710 msgr2=0x7f52880a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:15.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.102+0000 7f52ae000700 1 --2- 172.21.15.1:0/4100756680 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f52880a0710 0x7f52880a2bd0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f52a819eb00 tx=0x7f5290011720 comp rx=0 tx=0).stop 2024-02-02T15:35:15.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.102+0000 7f52ae000700 1 -- 172.21.15.1:0/4100756680 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52a8111d90 msgr2=0x7f52a81a1e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:15.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.102+0000 7f52ae000700 1 --2- 172.21.15.1:0/4100756680 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52a8111d90 0x7f52a81a1e90 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f52a000ef90 tx=0x7f52a000c3a0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.102+0000 7f52ae000700 1 -- 172.21.15.1:0/4100756680 shutdown_connections 2024-02-02T15:35:15.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.102+0000 7f52ae000700 1 --2- 172.21.15.1:0/4100756680 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f52880a0710 0x7f52880a2bd0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.102+0000 7f52ae000700 1 --2- 172.21.15.1:0/4100756680 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f52a8111d90 0x7f52a81a1e90 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.102+0000 7f52ae000700 1 --2- 172.21.15.1:0/4100756680 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f52a80730e0 0x7f52a819d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.102+0000 7f52ae000700 1 --2- 172.21.15.1:0/4100756680 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f52a8073a00 0x7f52a819da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.102+0000 7f52ae000700 1 -- 172.21.15.1:0/4100756680 >> 172.21.15.1:0/4100756680 conn(0x7f52a80fc810 msgr2=0x7f52a80fd480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:15.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.102+0000 7f52ae000700 1 -- 172.21.15.1:0/4100756680 shutdown_connections 2024-02-02T15:35:15.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.102+0000 7f52ae000700 1 -- 172.21.15.1:0/4100756680 wait complete. 2024-02-02T15:35:15.128 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-02T15:35:15.250 INFO:tasks.workunit.client.0.smithi001.stdout: 1 480288 6.40 MB/sec execute 289 sec latency 28.140 ms 2024-02-02T15:35:15.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.351+0000 7f6d91a00700 1 -- 172.21.15.1:0/2641593355 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d8c100740 msgr2=0x7f6d8c104c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:15.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.351+0000 7f6d91a00700 1 --2- 172.21.15.1:0/2641593355 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d8c100740 0x7f6d8c104c00 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f6d740098f0 tx=0x7f6d74009c00 comp rx=0 tx=0).stop 2024-02-02T15:35:15.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.351+0000 7f6d91a00700 1 -- 172.21.15.1:0/2641593355 shutdown_connections 2024-02-02T15:35:15.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.351+0000 7f6d91a00700 1 --2- 172.21.15.1:0/2641593355 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d8c105220 0x7f6d8c1056d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.351+0000 7f6d91a00700 1 --2- 172.21.15.1:0/2641593355 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6d8c0ffe20 0x7f6d8c100200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.351+0000 7f6d91a00700 1 --2- 172.21.15.1:0/2641593355 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d8c100740 0x7f6d8c104c00 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.351+0000 7f6d91a00700 1 -- 172.21.15.1:0/2641593355 >> 172.21.15.1:0/2641593355 conn(0x7f6d8c076d70 msgr2=0x7f6d8c077180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:15.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.351+0000 7f6d91a00700 1 -- 172.21.15.1:0/2641593355 shutdown_connections 2024-02-02T15:35:15.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.351+0000 7f6d91a00700 1 -- 172.21.15.1:0/2641593355 wait complete. 2024-02-02T15:35:15.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.352+0000 7f6d91a00700 1 Processor -- start 2024-02-02T15:35:15.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.353+0000 7f6d91a00700 1 -- start start 2024-02-02T15:35:15.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.353+0000 7f6d91a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d8c0ffe20 0x7f6d8c19d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:15.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.353+0000 7f6d91a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6d8c100740 0x7f6d8c19da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:15.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.353+0000 7f6d91a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d8c105220 0x7f6d8c1a1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:15.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.353+0000 7f6d91a00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8c1a2530 con 0x7f6d8c0ffe20 2024-02-02T15:35:15.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.353+0000 7f6d91a00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8c1a26a0 con 0x7f6d8c100740 2024-02-02T15:35:15.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.353+0000 7f6d91a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8c1a2860 con 0x7f6d8c105220 2024-02-02T15:35:15.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.353+0000 7f6d8aa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d8c0ffe20 0x7f6d8c19d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:15.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.353+0000 7f6d8a000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6d8c100740 0x7f6d8c19da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:15.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.353+0000 7f6d8a000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6d8c100740 0x7f6d8c19da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:54914/0 (socket says 172.21.15.1:54914) 2024-02-02T15:35:15.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.353+0000 7f6d8a000700 1 -- 172.21.15.1:0/98290162 learned_addr learned my addr 172.21.15.1:0/98290162 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:35:15.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.353+0000 7f6d8aa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d8c0ffe20 0x7f6d8c19d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46382/0 (socket says 172.21.15.1:46382) 2024-02-02T15:35:15.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.353+0000 7f6d8b400700 1 --2- 172.21.15.1:0/98290162 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d8c105220 0x7f6d8c1a1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:15.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.354+0000 7f6d8a000700 1 -- 172.21.15.1:0/98290162 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d8c105220 msgr2=0x7f6d8c1a1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:15.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.354+0000 7f6d8a000700 1 --2- 172.21.15.1:0/98290162 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d8c105220 0x7f6d8c1a1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.354+0000 7f6d8a000700 1 -- 172.21.15.1:0/98290162 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d8c0ffe20 msgr2=0x7f6d8c19d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:15.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.354+0000 7f6d8a000700 1 --2- 172.21.15.1:0/98290162 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d8c0ffe20 0x7f6d8c19d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.354+0000 7f6d8a000700 1 -- 172.21.15.1:0/98290162 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d7c0095f0 con 0x7f6d8c100740 2024-02-02T15:35:15.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.354+0000 7f6d8a000700 1 --2- 172.21.15.1:0/98290162 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6d8c100740 0x7f6d8c19da90 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f6d74009810 tx=0x7f6d7400fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:15.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.354+0000 7f6d83400700 1 -- 172.21.15.1:0/98290162 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d7401c070 con 0x7f6d8c100740 2024-02-02T15:35:15.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.354+0000 7f6d83400700 1 -- 172.21.15.1:0/98290162 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d7400bac0 con 0x7f6d8c100740 2024-02-02T15:35:15.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.354+0000 7f6d83400700 1 -- 172.21.15.1:0/98290162 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d74004930 con 0x7f6d8c100740 2024-02-02T15:35:15.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.354+0000 7f6d91a00700 1 -- 172.21.15.1:0/98290162 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d74009520 con 0x7f6d8c100740 2024-02-02T15:35:15.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.354+0000 7f6d91a00700 1 -- 172.21.15.1:0/98290162 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d8c1aa1a0 con 0x7f6d8c100740 2024-02-02T15:35:15.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.358+0000 7f6d83400700 1 -- 172.21.15.1:0/98290162 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f6d74020070 con 0x7f6d8c100740 2024-02-02T15:35:15.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.359+0000 7f6d83400700 1 --2- 172.21.15.1:0/98290162 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f6d6c0a0690 0x7f6d6c0a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:15.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.359+0000 7f6d83400700 1 -- 172.21.15.1:0/98290162 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f6d740ccf50 con 0x7f6d8c100740 2024-02-02T15:35:15.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.359+0000 7f6d8aa00700 1 --2- 172.21.15.1:0/98290162 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f6d6c0a0690 0x7f6d6c0a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:15.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.360+0000 7f6d8aa00700 1 --2- 172.21.15.1:0/98290162 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f6d6c0a0690 0x7f6d6c0a2b50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c005fd0 tx=0x7f6d7c009310 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:15.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.361+0000 7f6da5617b80 1 Processor -- start 2024-02-02T15:35:15.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.361+0000 7f6da5617b80 1 -- start start 2024-02-02T15:35:15.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.361+0000 7f6da5617b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f187341010 0x55f1873413f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:15.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.361+0000 7f6da5617b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f187341930 0x55f18734c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:15.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.362+0000 7f6da5617b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f18734c7e0 0x55f18734ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:15.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.362+0000 7f6da5617b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1872b54c0 con 0x55f187341930 2024-02-02T15:35:15.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.362+0000 7f6da5617b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f187299aa0 con 0x55f18734c7e0 2024-02-02T15:35:15.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.362+0000 7f6da5617b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f187297460 con 0x55f187341010 2024-02-02T15:35:15.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.362+0000 7f6d8aa00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f187341010 0x55f1873413f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:15.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.362+0000 7f6d8b400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f18734c7e0 0x55f18734ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:15.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.362+0000 7f6d8a000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f187341930 0x55f18734c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:15.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.362+0000 7f6d8aa00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f187341010 0x55f1873413f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:56134/0 (socket says 172.21.15.1:56134) 2024-02-02T15:35:15.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.362+0000 7f6d8aa00700 1 -- 172.21.15.1:0/3630870412 learned_addr learned my addr 172.21.15.1:0/3630870412 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:35:15.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.362+0000 7f6d8b400700 1 -- 172.21.15.1:0/3630870412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f187341010 msgr2=0x55f1873413f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:15.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.362+0000 7f6d8b400700 1 --2- 172.21.15.1:0/3630870412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f187341010 0x55f1873413f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.362+0000 7f6d8b400700 1 -- 172.21.15.1:0/3630870412 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f187341930 msgr2=0x55f18734c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:15.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.362+0000 7f6d8b400700 1 --2- 172.21.15.1:0/3630870412 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f187341930 0x55f18734c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.362+0000 7f6d8b400700 1 -- 172.21.15.1:0/3630870412 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f1871143b0 con 0x55f18734c7e0 2024-02-02T15:35:15.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.363+0000 7f6d8b400700 1 --2- 172.21.15.1:0/3630870412 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f18734c7e0 0x55f18734ebd0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f6d84003ec0 tx=0x7f6d84009ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c84d1fe4dda0837f server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:15.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.363+0000 7f6d7be00700 1 -- 172.21.15.1:0/3630870412 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d84019070 con 0x55f18734c7e0 2024-02-02T15:35:15.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.363+0000 7f6d7be00700 1 -- 172.21.15.1:0/3630870412 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d8400f040 con 0x55f18734c7e0 2024-02-02T15:35:15.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.363+0000 7f6d7be00700 1 -- 172.21.15.1:0/3630870412 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d84007960 con 0x55f18734c7e0 2024-02-02T15:35:15.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.363+0000 7f6da5617b80 1 -- 172.21.15.1:0/3630870412 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f18734c7e0 msgr2=0x55f18734ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:15.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.363+0000 7f6da5617b80 1 --2- 172.21.15.1:0/3630870412 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f18734c7e0 0x55f18734ebd0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f6d84003ec0 tx=0x7f6d84009ba0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.364+0000 7f6da5617b80 1 -- 172.21.15.1:0/3630870412 shutdown_connections 2024-02-02T15:35:15.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.364+0000 7f6da5617b80 1 --2- 172.21.15.1:0/3630870412 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f18734c7e0 0x55f18734ebd0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.364+0000 7f6da5617b80 1 --2- 172.21.15.1:0/3630870412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f187341010 0x55f1873413f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.364+0000 7f6da5617b80 1 --2- 172.21.15.1:0/3630870412 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f187341930 0x55f18734c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.364+0000 7f6da5617b80 1 -- 172.21.15.1:0/3630870412 >> 172.21.15.1:0/3630870412 conn(0x55f1872a0db0 msgr2=0x55f18733d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:15.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.364+0000 7f6da5617b80 1 -- 172.21.15.1:0/3630870412 shutdown_connections 2024-02-02T15:35:15.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.364+0000 7f6da5617b80 1 -- 172.21.15.1:0/3630870412 wait complete. 2024-02-02T15:35:15.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.364+0000 7f6da5617b80 1 -- start start 2024-02-02T15:35:15.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.364+0000 7f6da5617b80 1 Processor -- start 2024-02-02T15:35:15.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.365+0000 7f6da5617b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f187341010 0x55f18734c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:15.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.365+0000 7f6da5617b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f187341930 0x55f1873493c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:15.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.365+0000 7f6da5617b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f18734c7e0 0x55f1873499d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:15.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.365+0000 7f6da5617b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f187292d50 con 0x55f187341930 2024-02-02T15:35:15.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.365+0000 7f6da5617b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f18722ed80 con 0x55f18734c7e0 2024-02-02T15:35:15.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.365+0000 7f6da5617b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f18722ea20 con 0x55f187341010 2024-02-02T15:35:15.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.365+0000 7f6da5617b80 10 client.0 _ll_register_callbacks cb 0x55f187230290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:35:15.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.365+0000 7f6d8aa00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f187341010 0x55f18734c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:15.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.365+0000 7f6d8aa00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f187341010 0x55f18734c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:56138/0 (socket says 172.21.15.1:56138) 2024-02-02T15:35:15.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.365+0000 7f6d8aa00700 1 -- 172.21.15.1:0/420937922 learned_addr learned my addr 172.21.15.1:0/420937922 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:35:15.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.365+0000 7f6d8a000700 1 --2- 172.21.15.1:0/420937922 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f187341930 0x55f1873493c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:15.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.365+0000 7f6d8b400700 1 --2- 172.21.15.1:0/420937922 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f18734c7e0 0x55f1873499d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:15.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.365+0000 7f6d8aa00700 1 -- 172.21.15.1:0/420937922 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f18734c7e0 msgr2=0x55f1873499d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:15.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.366+0000 7f6d8aa00700 1 --2- 172.21.15.1:0/420937922 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f18734c7e0 0x55f1873499d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.366+0000 7f6d8aa00700 1 -- 172.21.15.1:0/420937922 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f187341930 msgr2=0x55f1873493c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:15.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.366+0000 7f6d8aa00700 1 --2- 172.21.15.1:0/420937922 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f187341930 0x55f1873493c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.366+0000 7f6d8aa00700 1 -- 172.21.15.1:0/420937922 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d84003ac0 con 0x55f187341010 2024-02-02T15:35:15.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.366+0000 7f6d8a000700 1 --2- 172.21.15.1:0/420937922 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f187341930 0x55f1873493c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:35:15.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.366+0000 7f6d8aa00700 1 --2- 172.21.15.1:0/420937922 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f187341010 0x55f18734c1b0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c00edb0 tx=0x7f6d7c0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f76b53e37117c9f server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:15.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.366+0000 7f6d78c00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:35:15.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.366+0000 7f6d78c00700 1 -- 172.21.15.1:0/420937922 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d7c022070 con 0x55f187341010 2024-02-02T15:35:15.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.366+0000 7f6da5617b80 1 -- 172.21.15.1:0/420937922 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f1873350f0 con 0x55f187341010 2024-02-02T15:35:15.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.366+0000 7f6d78c00700 1 -- 172.21.15.1:0/420937922 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f6d7c01da30 con 0x55f187341010 2024-02-02T15:35:15.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.366+0000 7f6d78c00700 1 -- 172.21.15.1:0/420937922 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d7c00ee90 con 0x55f187341010 2024-02-02T15:35:15.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.366+0000 7f6da5617b80 10 client.27826 fetch_fsmap learned FSMap version 42 2024-02-02T15:35:15.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.366+0000 7f6da5617b80 1 -- 172.21.15.1:0/420937922 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55f1870c0b60 con 0x55f187341010 2024-02-02T15:35:15.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.367+0000 7f6d78c00700 1 -- 172.21.15.1:0/420937922 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f1873350f0 con 0x55f187341010 2024-02-02T15:35:15.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.367+0000 7f6d78c00700 1 -- 172.21.15.1:0/420937922 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 42) v1 ==== 7510+0+0 (secure 0 0 0) 0x7f6d7c01c400 con 0x55f187341010 2024-02-02T15:35:15.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.367+0000 7f6da5617b80 10 client.27826 fetch_fsmap finished waiting for FSMap version 42 2024-02-02T15:35:15.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.367+0000 7f6da5617b80 10 client.27826 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-02-02T15:35:15.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.367+0000 7f6da5617b80 20 client.27826 populate_metadata read hostname 'smithi001' 2024-02-02T15:35:15.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.367+0000 7f6da5617b80 1 --2- 172.21.15.1:0/420937922 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55f18735c130 0x55f18735e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:15.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.367+0000 7f6da5617b80 4 client.27826 mds_command: new command op to 24518 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:35:15.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.367+0000 7f6da5617b80 1 -- 172.21.15.1:0/420937922 --> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f18728bdd0 con 0x55f18735c130 2024-02-02T15:35:15.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.367+0000 7f6d8a000700 1 --2- 172.21.15.1:0/420937922 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55f18735c130 0x55f18735e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:15.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.368+0000 7f6d8a000700 1 --2- 172.21.15.1:0/420937922 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55f18735c130 0x55f18735e550 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7ca485cbb807d911 server_cookie=42a74c27e05584e2 in_seq=0 out_seq=0 2024-02-02T15:35:15.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.368+0000 7f6d78c00700 10 client.27826 ms_handle_connect on v2:172.21.15.71:6836/628016199 2024-02-02T15:35:15.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.368+0000 7f6d78c00700 1 -- 172.21.15.1:0/420937922 <== mds.2 v2:172.21.15.71:6836/628016199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55f18728bdd0 con 0x55f18735c130 2024-02-02T15:35:15.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.369+0000 7f6d78c00700 10 client.27826 handle_command_reply: tid=0 2024-02-02T15:35:15.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.369+0000 7f6da5617b80 1 client.27826 shutdown 2024-02-02T15:35:15.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.369+0000 7f6da5617b80 1 -- 172.21.15.1:0/420937922 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f187341010 msgr2=0x55f18734c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:15.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.369+0000 7f6da5617b80 1 --2- 172.21.15.1:0/420937922 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f187341010 0x55f18734c1b0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c00edb0 tx=0x7f6d7c0047c0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.369+0000 7f6da5617b80 1 -- 172.21.15.1:0/420937922 shutdown_connections 2024-02-02T15:35:15.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.369+0000 7f6da5617b80 1 --2- 172.21.15.1:0/420937922 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f18734c7e0 0x55f1873499d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.369+0000 7f6da5617b80 1 --2- 172.21.15.1:0/420937922 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55f18735c130 0x55f18735e550 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.369+0000 7f6da5617b80 1 --2- 172.21.15.1:0/420937922 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f187341010 0x55f18734c1b0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.369+0000 7f6d78c00700 0 client.27826 ms_handle_reset on v2:172.21.15.71:6836/628016199 2024-02-02T15:35:15.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.369+0000 7f6da5617b80 1 --2- 172.21.15.1:0/420937922 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f187341930 0x55f1873493c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.369+0000 7f6da5617b80 1 -- 172.21.15.1:0/420937922 >> 172.21.15.1:0/420937922 conn(0x55f1872a0db0 msgr2=0x55f18733cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:15.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.369+0000 7f6da5617b80 1 -- 172.21.15.1:0/420937922 shutdown_connections 2024-02-02T15:35:15.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.369+0000 7f6da5617b80 1 -- 172.21.15.1:0/420937922 wait complete. 2024-02-02T15:35:15.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.369+0000 7f6da5617b80 20 client.27826 trim_cache size 0 max 16384 2024-02-02T15:35:15.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.384+0000 7f6da5617b80 1 Processor -- start 2024-02-02T15:35:15.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.384+0000 7f6da5617b80 1 -- start start 2024-02-02T15:35:15.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.385+0000 7f6da5617b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f187341010 0x55f187353320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:15.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.385+0000 7f6da5617b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f187341930 0x55f1873584c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:15.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.385+0000 7f6da5617b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f18734c7e0 0x55f18734bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:15.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.385+0000 7f6da5617b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1871166a0 con 0x55f187341930 2024-02-02T15:35:15.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.385+0000 7f6da5617b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f187359900 con 0x55f18734c7e0 2024-02-02T15:35:15.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.385+0000 7f6da5617b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1873479e0 con 0x55f187341010 2024-02-02T15:35:15.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.385+0000 7f6d8b400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f18734c7e0 0x55f18734bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:15.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.385+0000 7f6d8b400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f18734c7e0 0x55f18734bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:54938/0 (socket says 172.21.15.1:54938) 2024-02-02T15:35:15.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.385+0000 7f6d8aa00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f187341010 0x55f187353320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:15.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.385+0000 7f6d8b400700 1 -- 172.21.15.1:0/3577078810 learned_addr learned my addr 172.21.15.1:0/3577078810 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:35:15.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.385+0000 7f6d8a000700 1 --2- 172.21.15.1:0/3577078810 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f187341930 0x55f1873584c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:15.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.386+0000 7f6d8aa00700 1 -- 172.21.15.1:0/3577078810 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f18734c7e0 msgr2=0x55f18734bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:15.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.386+0000 7f6d8aa00700 1 --2- 172.21.15.1:0/3577078810 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f18734c7e0 0x55f18734bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.386+0000 7f6d8aa00700 1 -- 172.21.15.1:0/3577078810 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f187341930 msgr2=0x55f1873584c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:15.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.386+0000 7f6d8aa00700 1 --2- 172.21.15.1:0/3577078810 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f187341930 0x55f1873584c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.386+0000 7f6d8aa00700 1 -- 172.21.15.1:0/3577078810 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f187258ea0 con 0x55f187341010 2024-02-02T15:35:15.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.386+0000 7f6d8aa00700 1 --2- 172.21.15.1:0/3577078810 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f187341010 0x55f187353320 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c01c7e0 tx=0x7f6d7c009380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b40a08942d4cfca5 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:15.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.387+0000 7f6d80c00700 1 -- 172.21.15.1:0/3577078810 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d7c03a070 con 0x55f187341010 2024-02-02T15:35:15.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.387+0000 7f6d80c00700 1 -- 172.21.15.1:0/3577078810 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d7c00ee90 con 0x55f187341010 2024-02-02T15:35:15.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.387+0000 7f6d80c00700 1 -- 172.21.15.1:0/3577078810 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d7c022050 con 0x55f187341010 2024-02-02T15:35:15.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.387+0000 7f6da5617b80 1 -- 172.21.15.1:0/3577078810 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f187341010 msgr2=0x55f187353320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:15.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.387+0000 7f6da5617b80 1 --2- 172.21.15.1:0/3577078810 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f187341010 0x55f187353320 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c01c7e0 tx=0x7f6d7c009380 comp rx=0 tx=0).stop 2024-02-02T15:35:15.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.387+0000 7f6da5617b80 1 -- 172.21.15.1:0/3577078810 shutdown_connections 2024-02-02T15:35:15.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.387+0000 7f6da5617b80 1 --2- 172.21.15.1:0/3577078810 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f18734c7e0 0x55f18734bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.387+0000 7f6da5617b80 1 --2- 172.21.15.1:0/3577078810 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f187341010 0x55f187353320 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.387+0000 7f6da5617b80 1 --2- 172.21.15.1:0/3577078810 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f187341930 0x55f1873584c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.387+0000 7f6da5617b80 1 -- 172.21.15.1:0/3577078810 >> 172.21.15.1:0/3577078810 conn(0x55f1872a0db0 msgr2=0x55f18733bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:15.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.388+0000 7f6da5617b80 1 -- 172.21.15.1:0/3577078810 shutdown_connections 2024-02-02T15:35:15.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.388+0000 7f6da5617b80 1 -- 172.21.15.1:0/3577078810 wait complete. 2024-02-02T15:35:15.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.388+0000 7f6da5617b80 1 -- start start 2024-02-02T15:35:15.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.388+0000 7f6da5617b80 1 Processor -- start 2024-02-02T15:35:15.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.389+0000 7f6da5617b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f187341010 0x55f187346b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:15.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.389+0000 7f6da5617b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f187341930 0x55f1873470c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:15.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.389+0000 7f6da5617b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f18734c7e0 0x55f187345880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:15.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.389+0000 7f6da5617b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f187345f60 con 0x55f187341930 2024-02-02T15:35:15.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.389+0000 7f6da5617b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1873460d0 con 0x55f187341010 2024-02-02T15:35:15.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.389+0000 7f6da5617b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f187346290 con 0x55f18734c7e0 2024-02-02T15:35:15.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.389+0000 7f6da5617b80 10 client.0 _ll_register_callbacks cb 0x55f187230290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:35:15.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.389+0000 7f6d8aa00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f187341010 0x55f187346b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:15.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.389+0000 7f6d8aa00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f187341010 0x55f187346b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:54942/0 (socket says 172.21.15.1:54942) 2024-02-02T15:35:15.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.389+0000 7f6d8aa00700 1 -- 172.21.15.1:0/3130684970 learned_addr learned my addr 172.21.15.1:0/3130684970 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:35:15.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.390+0000 7f6d8a000700 1 --2- 172.21.15.1:0/3130684970 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f187341930 0x55f1873470c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:15.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.390+0000 7f6d8b400700 1 --2- 172.21.15.1:0/3130684970 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f18734c7e0 0x55f187345880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:15.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.390+0000 7f6d8aa00700 1 -- 172.21.15.1:0/3130684970 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f18734c7e0 msgr2=0x55f187345880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:15.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.390+0000 7f6d8aa00700 1 --2- 172.21.15.1:0/3130684970 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f18734c7e0 0x55f187345880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.390+0000 7f6d8aa00700 1 -- 172.21.15.1:0/3130684970 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f187341930 msgr2=0x55f1873470c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:15.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.390+0000 7f6d8aa00700 1 --2- 172.21.15.1:0/3130684970 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f187341930 0x55f1873470c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.390+0000 7f6d8aa00700 1 -- 172.21.15.1:0/3130684970 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f1870c0b60 con 0x55f187341010 2024-02-02T15:35:15.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.390+0000 7f6d8a000700 1 --2- 172.21.15.1:0/3130684970 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f187341930 0x55f1873470c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:35:15.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.390+0000 7f6d8aa00700 1 --2- 172.21.15.1:0/3130684970 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f187341010 0x55f187346b80 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c009a60 tx=0x7f6d7c01cf40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0cefc97671a0161 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:15.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.390+0000 7f6d78c00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:35:15.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.390+0000 7f6d78c00700 1 -- 172.21.15.1:0/3130684970 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d7c03a070 con 0x55f187341010 2024-02-02T15:35:15.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.390+0000 7f6d78c00700 1 -- 172.21.15.1:0/3130684970 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f6d7c01da30 con 0x55f187341010 2024-02-02T15:35:15.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.391+0000 7f6da5617b80 1 -- 172.21.15.1:0/3130684970 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6d8400f040 con 0x55f187341010 2024-02-02T15:35:15.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.391+0000 7f6d78c00700 1 -- 172.21.15.1:0/3130684970 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d7c022040 con 0x55f187341010 2024-02-02T15:35:15.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.391+0000 7f6da5617b80 10 client.27836 fetch_fsmap learned FSMap version 42 2024-02-02T15:35:15.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.391+0000 7f6da5617b80 1 -- 172.21.15.1:0/3130684970 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55f187118cd0 con 0x55f187341010 2024-02-02T15:35:15.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.391+0000 7f6d78c00700 1 -- 172.21.15.1:0/3130684970 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6d8400f040 con 0x55f187341010 2024-02-02T15:35:15.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.391+0000 7f6d78c00700 1 -- 172.21.15.1:0/3130684970 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 42) v1 ==== 7510+0+0 (secure 0 0 0) 0x7f6d7c02e030 con 0x55f187341010 2024-02-02T15:35:15.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.392+0000 7f6da5617b80 10 client.27836 fetch_fsmap finished waiting for FSMap version 42 2024-02-02T15:35:15.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.392+0000 7f6da5617b80 10 client.27836 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-02-02T15:35:15.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.392+0000 7f6da5617b80 20 client.27836 populate_metadata read hostname 'smithi001' 2024-02-02T15:35:15.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.392+0000 7f6da5617b80 1 --2- 172.21.15.1:0/3130684970 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55f1873aeaa0 0x55f1873b0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:15.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.392+0000 7f6da5617b80 4 client.27836 mds_command: new command op to 24518 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:35:15.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.392+0000 7f6da5617b80 1 -- 172.21.15.1:0/3130684970 --> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f187251f50 con 0x55f1873aeaa0 2024-02-02T15:35:15.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.392+0000 7f6d8a000700 1 --2- 172.21.15.1:0/3130684970 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55f1873aeaa0 0x55f1873b0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:15.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.392+0000 7f6d8a000700 1 --2- 172.21.15.1:0/3130684970 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55f1873aeaa0 0x55f1873b0e90 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8e9c0c56136df0af server_cookie=db030f76dfcfce8a in_seq=0 out_seq=0 2024-02-02T15:35:15.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.392+0000 7f6d78c00700 10 client.27836 ms_handle_connect on v2:172.21.15.71:6836/628016199 2024-02-02T15:35:15.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.393+0000 7f6d78c00700 1 -- 172.21.15.1:0/3130684970 <== mds.2 v2:172.21.15.71:6836/628016199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f187251f50 con 0x55f1873aeaa0 2024-02-02T15:35:15.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.393+0000 7f6d78c00700 10 client.27836 handle_command_reply: tid=0 2024-02-02T15:35:15.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.393+0000 7f6da5617b80 1 client.27836 shutdown 2024-02-02T15:35:15.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.393+0000 7f6da5617b80 1 -- 172.21.15.1:0/3130684970 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f187341010 msgr2=0x55f187346b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:15.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.393+0000 7f6da5617b80 1 --2- 172.21.15.1:0/3130684970 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f187341010 0x55f187346b80 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c009a60 tx=0x7f6d7c01cf40 comp rx=0 tx=0).stop 2024-02-02T15:35:15.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.393+0000 7f6da5617b80 1 -- 172.21.15.1:0/3130684970 shutdown_connections 2024-02-02T15:35:15.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.393+0000 7f6da5617b80 1 --2- 172.21.15.1:0/3130684970 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55f1873aeaa0 0x55f1873b0e90 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.393+0000 7f6da5617b80 1 --2- 172.21.15.1:0/3130684970 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f18734c7e0 0x55f187345880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.393+0000 7f6da5617b80 1 --2- 172.21.15.1:0/3130684970 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f187341010 0x55f187346b80 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.393+0000 7f6da5617b80 1 --2- 172.21.15.1:0/3130684970 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f187341930 0x55f1873470c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.393+0000 7f6da5617b80 1 -- 172.21.15.1:0/3130684970 >> 172.21.15.1:0/3130684970 conn(0x55f1872a0db0 msgr2=0x55f187344700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:15.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.393+0000 7f6d78c00700 0 client.27836 ms_handle_reset on v2:172.21.15.71:6836/628016199 2024-02-02T15:35:15.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.393+0000 7f6da5617b80 1 -- 172.21.15.1:0/3130684970 shutdown_connections 2024-02-02T15:35:15.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.393+0000 7f6da5617b80 1 -- 172.21.15.1:0/3130684970 wait complete. 2024-02-02T15:35:15.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.394+0000 7f6da5617b80 20 client.27836 trim_cache size 0 max 16384 2024-02-02T15:35:15.397 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:35:15.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.394+0000 7f6d91a00700 1 -- 172.21.15.1:0/98290162 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f6d6c0a0690 msgr2=0x7f6d6c0a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:15.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.394+0000 7f6d91a00700 1 --2- 172.21.15.1:0/98290162 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f6d6c0a0690 0x7f6d6c0a2b50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c005fd0 tx=0x7f6d7c009310 comp rx=0 tx=0).stop 2024-02-02T15:35:15.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.394+0000 7f6d91a00700 1 -- 172.21.15.1:0/98290162 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6d8c100740 msgr2=0x7f6d8c19da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:15.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.394+0000 7f6d91a00700 1 --2- 172.21.15.1:0/98290162 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6d8c100740 0x7f6d8c19da90 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f6d74009810 tx=0x7f6d7400fa00 comp rx=0 tx=0).stop 2024-02-02T15:35:15.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.395+0000 7f6d91a00700 1 -- 172.21.15.1:0/98290162 shutdown_connections 2024-02-02T15:35:15.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.395+0000 7f6d91a00700 1 --2- 172.21.15.1:0/98290162 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f6d6c0a0690 0x7f6d6c0a2b50 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.395+0000 7f6d91a00700 1 --2- 172.21.15.1:0/98290162 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6d8c105220 0x7f6d8c1a1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.395+0000 7f6d91a00700 1 --2- 172.21.15.1:0/98290162 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6d8c0ffe20 0x7f6d8c19d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.395+0000 7f6d91a00700 1 --2- 172.21.15.1:0/98290162 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6d8c100740 0x7f6d8c19da90 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.395+0000 7f6d91a00700 1 -- 172.21.15.1:0/98290162 >> 172.21.15.1:0/98290162 conn(0x7f6d8c076d70 msgr2=0x7f6d8c102cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:15.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.395+0000 7f6d91a00700 1 -- 172.21.15.1:0/98290162 shutdown_connections 2024-02-02T15:35:15.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.395+0000 7f6d91a00700 1 -- 172.21.15.1:0/98290162 wait complete. 2024-02-02T15:35:15.416 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-02T15:35:15.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.631+0000 7f37db000700 1 -- 172.21.15.1:0/2685120138 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37d4100ff0 msgr2=0x7f37d41013d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:15.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.631+0000 7f37db000700 1 --2- 172.21.15.1:0/2685120138 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37d4100ff0 0x7f37d41013d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f37c4009990 tx=0x7f37c4009ca0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.631+0000 7f37db000700 1 -- 172.21.15.1:0/2685120138 shutdown_connections 2024-02-02T15:35:15.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.631+0000 7f37db000700 1 --2- 172.21.15.1:0/2685120138 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37d4106110 0x7f37d410dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.631+0000 7f37db000700 1 --2- 172.21.15.1:0/2685120138 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37d4100ff0 0x7f37d41013d0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.631+0000 7f37db000700 1 --2- 172.21.15.1:0/2685120138 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f37d41019a0 0x7f37d41059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.631+0000 7f37db000700 1 -- 172.21.15.1:0/2685120138 >> 172.21.15.1:0/2685120138 conn(0x7f37d40fc810 msgr2=0x7f37d40fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:15.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.631+0000 7f37db000700 1 -- 172.21.15.1:0/2685120138 shutdown_connections 2024-02-02T15:35:15.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.631+0000 7f37db000700 1 -- 172.21.15.1:0/2685120138 wait complete. 2024-02-02T15:35:15.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.632+0000 7f37db000700 1 Processor -- start 2024-02-02T15:35:15.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.632+0000 7f37db000700 1 -- start start 2024-02-02T15:35:15.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.632+0000 7f37db000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f37d4100ff0 0x7f37d41990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:15.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.632+0000 7f37db000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37d4106110 0x7f37d4199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:15.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.632+0000 7f37db000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37d419da90 0x7f37d419df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:15.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.633+0000 7f37db000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37d419e510 con 0x7f37d4100ff0 2024-02-02T15:35:15.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.633+0000 7f37db000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37d419e680 con 0x7f37d419da90 2024-02-02T15:35:15.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.633+0000 7f37db000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37d419e840 con 0x7f37d4106110 2024-02-02T15:35:15.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.633+0000 7f37d3400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37d4106110 0x7f37d4199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:15.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.633+0000 7f37d3400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37d4106110 0x7f37d4199630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:56180/0 (socket says 172.21.15.1:56180) 2024-02-02T15:35:15.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.633+0000 7f37d3400700 1 -- 172.21.15.1:0/3065291479 learned_addr learned my addr 172.21.15.1:0/3065291479 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:35:15.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.633+0000 7f37d3e00700 1 --2- 172.21.15.1:0/3065291479 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f37d4100ff0 0x7f37d41990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:15.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.633+0000 7f37d9000700 1 --2- 172.21.15.1:0/3065291479 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37d419da90 0x7f37d419df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:15.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.633+0000 7f37d3400700 1 -- 172.21.15.1:0/3065291479 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37d419da90 msgr2=0x7f37d419df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:15.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.633+0000 7f37d3400700 1 --2- 172.21.15.1:0/3065291479 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37d419da90 0x7f37d419df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.633+0000 7f37d3400700 1 -- 172.21.15.1:0/3065291479 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f37d4100ff0 msgr2=0x7f37d41990f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:15.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.633+0000 7f37d3400700 1 --2- 172.21.15.1:0/3065291479 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f37d4100ff0 0x7f37d41990f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.633+0000 7f37d3400700 1 -- 172.21.15.1:0/3065291479 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37c40095f0 con 0x7f37d4106110 2024-02-02T15:35:15.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.634+0000 7f37d3400700 1 --2- 172.21.15.1:0/3065291479 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37d4106110 0x7f37d4199630 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f37bc00e970 tx=0x7f37bc00ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:15.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.634+0000 7f37d0c00700 1 -- 172.21.15.1:0/3065291479 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37bc009790 con 0x7f37d4106110 2024-02-02T15:35:15.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.634+0000 7f37d0c00700 1 -- 172.21.15.1:0/3065291479 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f37bc00c7e0 con 0x7f37d4106110 2024-02-02T15:35:15.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.634+0000 7f37d0c00700 1 -- 172.21.15.1:0/3065291479 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37bc0054e0 con 0x7f37d4106110 2024-02-02T15:35:15.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.634+0000 7f37db000700 1 -- 172.21.15.1:0/3065291479 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37d41a5da0 con 0x7f37d4106110 2024-02-02T15:35:15.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.634+0000 7f37db000700 1 -- 172.21.15.1:0/3065291479 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37d41a61d0 con 0x7f37d4106110 2024-02-02T15:35:15.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.637+0000 7f37d0c00700 1 -- 172.21.15.1:0/3065291479 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f37bc00c950 con 0x7f37d4106110 2024-02-02T15:35:15.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.639+0000 7f37d0c00700 1 --2- 172.21.15.1:0/3065291479 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f37b40a0710 0x7f37b40a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:15.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.639+0000 7f37d0c00700 1 -- 172.21.15.1:0/3065291479 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f37bc014070 con 0x7f37d4106110 2024-02-02T15:35:15.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.639+0000 7f37d3e00700 1 --2- 172.21.15.1:0/3065291479 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f37b40a0710 0x7f37b40a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:15.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.640+0000 7f37eeb8ab80 1 Processor -- start 2024-02-02T15:35:15.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.640+0000 7f37eeb8ab80 1 -- start start 2024-02-02T15:35:15.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.640+0000 7f37eeb8ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558998d4d010 0x558998d4d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:15.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.640+0000 7f37eeb8ab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558998d4d930 0x558998d582a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:15.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.640+0000 7f37eeb8ab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558998d587e0 0x558998d5abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:15.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.640+0000 7f37eeb8ab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x558998cc14c0 con 0x558998d587e0 2024-02-02T15:35:15.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.640+0000 7f37eeb8ab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x558998ca5aa0 con 0x558998d4d930 2024-02-02T15:35:15.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.640+0000 7f37eeb8ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558998ca3460 con 0x558998d4d010 2024-02-02T15:35:15.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.641+0000 7f37d3400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558998d4d930 0x558998d582a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:15.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.641+0000 7f37d3400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558998d4d930 0x558998d582a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:54962/0 (socket says 172.21.15.1:54962) 2024-02-02T15:35:15.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.641+0000 7f37d3400700 1 -- 172.21.15.1:0/112862028 learned_addr learned my addr 172.21.15.1:0/112862028 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:35:15.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.641+0000 7f37d3e00700 1 --2- 172.21.15.1:0/112862028 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558998d4d010 0x558998d4d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:15.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.641+0000 7f37d9000700 1 --2- 172.21.15.1:0/112862028 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558998d587e0 0x558998d5abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:15.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.641+0000 7f37d3e00700 1 --2- 172.21.15.1:0/3065291479 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f37b40a0710 0x7f37b40a2bd0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f37c400b2e0 tx=0x7f37c4005fd0 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:15.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.641+0000 7f37d3e00700 1 -- 172.21.15.1:0/112862028 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558998d4d930 msgr2=0x558998d582a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:15.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.641+0000 7f37d3e00700 1 --2- 172.21.15.1:0/112862028 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558998d4d930 0x558998d582a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.641+0000 7f37d3e00700 1 -- 172.21.15.1:0/112862028 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558998d587e0 msgr2=0x558998d5abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:15.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.641+0000 7f37d3e00700 1 --2- 172.21.15.1:0/112862028 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558998d587e0 0x558998d5abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.641+0000 7f37d3e00700 1 -- 172.21.15.1:0/112862028 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558998b203b0 con 0x558998d4d010 2024-02-02T15:35:15.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.641+0000 7f37d3400700 1 --2- 172.21.15.1:0/112862028 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558998d4d930 0x558998d582a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:35:15.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.641+0000 7f37d9000700 1 --2- 172.21.15.1:0/112862028 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558998d587e0 0x558998d5abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:35:15.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.642+0000 7f37d3e00700 1 --2- 172.21.15.1:0/112862028 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558998d4d010 0x558998d4d3f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f37c4000c00 tx=0x7f37c4020000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e2607beacf767fa server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:15.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.642+0000 7f37c9600700 1 -- 172.21.15.1:0/112862028 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37c4004500 con 0x558998d4d010 2024-02-02T15:35:15.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.642+0000 7f37c9600700 1 -- 172.21.15.1:0/112862028 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f37c401e040 con 0x558998d4d010 2024-02-02T15:35:15.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.642+0000 7f37c9600700 1 -- 172.21.15.1:0/112862028 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37c400f8b0 con 0x558998d4d010 2024-02-02T15:35:15.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.643+0000 7f37eeb8ab80 1 -- 172.21.15.1:0/112862028 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558998d4d010 msgr2=0x558998d4d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:15.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.643+0000 7f37eeb8ab80 1 --2- 172.21.15.1:0/112862028 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558998d4d010 0x558998d4d3f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f37c4000c00 tx=0x7f37c4020000 comp rx=0 tx=0).stop 2024-02-02T15:35:15.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.643+0000 7f37eeb8ab80 1 -- 172.21.15.1:0/112862028 shutdown_connections 2024-02-02T15:35:15.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.643+0000 7f37eeb8ab80 1 --2- 172.21.15.1:0/112862028 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558998d587e0 0x558998d5abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.643+0000 7f37eeb8ab80 1 --2- 172.21.15.1:0/112862028 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558998d4d010 0x558998d4d3f0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.643+0000 7f37eeb8ab80 1 --2- 172.21.15.1:0/112862028 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558998d4d930 0x558998d582a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.643+0000 7f37eeb8ab80 1 -- 172.21.15.1:0/112862028 >> 172.21.15.1:0/112862028 conn(0x558998cacdb0 msgr2=0x558998d493b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:15.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.643+0000 7f37eeb8ab80 1 -- 172.21.15.1:0/112862028 shutdown_connections 2024-02-02T15:35:15.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.643+0000 7f37eeb8ab80 1 -- 172.21.15.1:0/112862028 wait complete. 2024-02-02T15:35:15.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.643+0000 7f37eeb8ab80 1 -- start start 2024-02-02T15:35:15.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.644+0000 7f37eeb8ab80 1 Processor -- start 2024-02-02T15:35:15.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.644+0000 7f37eeb8ab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558998d4d010 0x558998d58180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:15.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.644+0000 7f37eeb8ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558998d4d930 0x558998d55390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:15.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.644+0000 7f37eeb8ab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558998d587e0 0x558998d55950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:15.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.644+0000 7f37eeb8ab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x558998c9ed50 con 0x558998d4d010 2024-02-02T15:35:15.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.644+0000 7f37eeb8ab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x558998c3ad80 con 0x558998d587e0 2024-02-02T15:35:15.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.644+0000 7f37eeb8ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558998c3aa20 con 0x558998d4d930 2024-02-02T15:35:15.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.644+0000 7f37eeb8ab80 10 client.0 _ll_register_callbacks cb 0x558998c3c290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:35:15.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.644+0000 7f37d3e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558998d4d010 0x558998d58180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:15.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.644+0000 7f37d9000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558998d587e0 0x558998d55950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:15.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.644+0000 7f37d3400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558998d4d930 0x558998d55390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:15.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.644+0000 7f37d9000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558998d587e0 0x558998d55950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:54970/0 (socket says 172.21.15.1:54970) 2024-02-02T15:35:15.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.644+0000 7f37d9000700 1 -- 172.21.15.1:0/970770739 learned_addr learned my addr 172.21.15.1:0/970770739 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:35:15.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.644+0000 7f37d3400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558998d4d930 0x558998d55390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:56204/0 (socket says 172.21.15.1:56204) 2024-02-02T15:35:15.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.645+0000 7f37d3400700 1 -- 172.21.15.1:0/970770739 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558998d587e0 msgr2=0x558998d55950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:15.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.645+0000 7f37d3400700 1 --2- 172.21.15.1:0/970770739 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558998d587e0 0x558998d55950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.645+0000 7f37d3400700 1 -- 172.21.15.1:0/970770739 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558998d4d010 msgr2=0x558998d58180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:15.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.645+0000 7f37d3400700 1 --2- 172.21.15.1:0/970770739 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558998d4d010 0x558998d58180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.645+0000 7f37d3400700 1 -- 172.21.15.1:0/970770739 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37cc011230 con 0x558998d4d930 2024-02-02T15:35:15.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.645+0000 7f37d3400700 1 --2- 172.21.15.1:0/970770739 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558998d4d930 0x558998d55390 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f37bc095ec0 tx=0x7f37bc098db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75121b2b1b249bc2 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:15.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.645+0000 7f37c2000700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:35:15.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.645+0000 7f37c2000700 1 -- 172.21.15.1:0/970770739 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37bc099b50 con 0x558998d4d930 2024-02-02T15:35:15.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.645+0000 7f37c2000700 1 -- 172.21.15.1:0/970770739 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f37bc099450 con 0x558998d4d930 2024-02-02T15:35:15.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.645+0000 7f37eeb8ab80 1 -- 172.21.15.1:0/970770739 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558998d410f0 con 0x558998d4d930 2024-02-02T15:35:15.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.646+0000 7f37c2000700 1 -- 172.21.15.1:0/970770739 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f37bc09d280 con 0x558998d4d930 2024-02-02T15:35:15.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.646+0000 7f37eeb8ab80 10 client.27844 fetch_fsmap learned FSMap version 42 2024-02-02T15:35:15.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.646+0000 7f37eeb8ab80 1 -- 172.21.15.1:0/970770739 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7f37d410e500 con 0x558998d4d930 2024-02-02T15:35:15.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.646+0000 7f37c2000700 1 -- 172.21.15.1:0/970770739 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558998d410f0 con 0x558998d4d930 2024-02-02T15:35:15.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.646+0000 7f37c2000700 1 -- 172.21.15.1:0/970770739 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 42) v1 ==== 7510+0+0 (secure 0 0 0) 0x7f37bc0996b0 con 0x558998d4d930 2024-02-02T15:35:15.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.646+0000 7f37eeb8ab80 10 client.27844 fetch_fsmap finished waiting for FSMap version 42 2024-02-02T15:35:15.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.646+0000 7f37eeb8ab80 10 client.27844 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-02T15:35:15.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.646+0000 7f37eeb8ab80 20 client.27844 populate_metadata read hostname 'smithi001' 2024-02-02T15:35:15.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.646+0000 7f37eeb8ab80 1 --2- 172.21.15.1:0/970770739 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x558998d68200 0x558998d6a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:15.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.646+0000 7f37eeb8ab80 4 client.27844 mds_command: new command op to 24529 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:35:15.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.646+0000 7f37eeb8ab80 1 -- 172.21.15.1:0/970770739 --> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558998c97dd0 con 0x558998d68200 2024-02-02T15:35:15.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.647+0000 7f37d3e00700 1 --2- 172.21.15.1:0/970770739 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x558998d68200 0x558998d6a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:15.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.647+0000 7f37d3e00700 1 --2- 172.21.15.1:0/970770739 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x558998d68200 0x558998d6a620 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=335a9313b172361c server_cookie=9a9814838277a2a1 in_seq=0 out_seq=0 2024-02-02T15:35:15.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.647+0000 7f37c2000700 10 client.27844 ms_handle_connect on v2:172.21.15.142:6838/321675421 2024-02-02T15:35:15.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.648+0000 7f37c2000700 1 -- 172.21.15.1:0/970770739 <== mds.1 v2:172.21.15.142:6838/321675421 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x558998c97dd0 con 0x558998d68200 2024-02-02T15:35:15.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.648+0000 7f37c2000700 10 client.27844 handle_command_reply: tid=0 2024-02-02T15:35:15.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.648+0000 7f37eeb8ab80 1 client.27844 shutdown 2024-02-02T15:35:15.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.648+0000 7f37eeb8ab80 1 -- 172.21.15.1:0/970770739 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558998d4d930 msgr2=0x558998d55390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:15.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.648+0000 7f37eeb8ab80 1 --2- 172.21.15.1:0/970770739 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558998d4d930 0x558998d55390 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f37bc095ec0 tx=0x7f37bc098db0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.648+0000 7f37eeb8ab80 1 -- 172.21.15.1:0/970770739 shutdown_connections 2024-02-02T15:35:15.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.648+0000 7f37eeb8ab80 1 --2- 172.21.15.1:0/970770739 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x558998d68200 0x558998d6a620 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.648+0000 7f37eeb8ab80 1 --2- 172.21.15.1:0/970770739 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558998d587e0 0x558998d55950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.648+0000 7f37eeb8ab80 1 --2- 172.21.15.1:0/970770739 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558998d4d010 0x558998d58180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.648+0000 7f37c2000700 0 client.27844 ms_handle_reset on v2:172.21.15.142:6838/321675421 2024-02-02T15:35:15.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.648+0000 7f37eeb8ab80 1 --2- 172.21.15.1:0/970770739 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558998d4d930 0x558998d55390 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.648+0000 7f37eeb8ab80 1 -- 172.21.15.1:0/970770739 >> 172.21.15.1:0/970770739 conn(0x558998cacdb0 msgr2=0x558998d48fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:15.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.648+0000 7f37eeb8ab80 1 -- 172.21.15.1:0/970770739 shutdown_connections 2024-02-02T15:35:15.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.648+0000 7f37eeb8ab80 1 -- 172.21.15.1:0/970770739 wait complete. 2024-02-02T15:35:15.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.649+0000 7f37eeb8ab80 20 client.27844 trim_cache size 0 max 16384 2024-02-02T15:35:15.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.663+0000 7f37eeb8ab80 1 Processor -- start 2024-02-02T15:35:15.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.663+0000 7f37eeb8ab80 1 -- start start 2024-02-02T15:35:15.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.663+0000 7f37eeb8ab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558998d4d010 0x558998d5f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:15.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.663+0000 7f37eeb8ab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558998d4d930 0x558998d644b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:15.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.663+0000 7f37eeb8ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558998d587e0 0x558998d57cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:15.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.663+0000 7f37eeb8ab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x558998b226a0 con 0x558998d4d010 2024-02-02T15:35:15.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.663+0000 7f37eeb8ab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x558998d56350 con 0x558998d4d930 2024-02-02T15:35:15.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.663+0000 7f37eeb8ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558998d4a960 con 0x558998d587e0 2024-02-02T15:35:15.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.664+0000 7f37d3400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558998d4d930 0x558998d644b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:15.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.664+0000 7f37d9000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558998d587e0 0x558998d57cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:15.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.664+0000 7f37d3400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558998d4d930 0x558998d644b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:54974/0 (socket says 172.21.15.1:54974) 2024-02-02T15:35:15.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.664+0000 7f37d3400700 1 -- 172.21.15.1:0/1616603240 learned_addr learned my addr 172.21.15.1:0/1616603240 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:35:15.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.664+0000 7f37d3e00700 1 --2- 172.21.15.1:0/1616603240 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558998d4d010 0x558998d5f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:15.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.664+0000 7f37d9000700 1 -- 172.21.15.1:0/1616603240 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558998d4d930 msgr2=0x558998d644b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:15.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.664+0000 7f37d9000700 1 --2- 172.21.15.1:0/1616603240 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558998d4d930 0x558998d644b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.664+0000 7f37d9000700 1 -- 172.21.15.1:0/1616603240 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558998d4d010 msgr2=0x558998d5f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:15.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.664+0000 7f37d9000700 1 --2- 172.21.15.1:0/1616603240 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558998d4d010 0x558998d5f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.664+0000 7f37d9000700 1 -- 172.21.15.1:0/1616603240 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558998accb60 con 0x558998d587e0 2024-02-02T15:35:15.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.664+0000 7f37d9000700 1 --2- 172.21.15.1:0/1616603240 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558998d587e0 0x558998d57cb0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f37cc004a60 tx=0x7f37cc009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a04d4a0be34985a server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:15.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.665+0000 7f37ca000700 1 -- 172.21.15.1:0/1616603240 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37cc00e040 con 0x558998d587e0 2024-02-02T15:35:15.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.665+0000 7f37ca000700 1 -- 172.21.15.1:0/1616603240 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f37cc00f040 con 0x558998d587e0 2024-02-02T15:35:15.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.665+0000 7f37ca000700 1 -- 172.21.15.1:0/1616603240 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37cc01f800 con 0x558998d587e0 2024-02-02T15:35:15.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.666+0000 7f37eeb8ab80 1 -- 172.21.15.1:0/1616603240 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558998d587e0 msgr2=0x558998d57cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:15.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.666+0000 7f37eeb8ab80 1 --2- 172.21.15.1:0/1616603240 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558998d587e0 0x558998d57cb0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f37cc004a60 tx=0x7f37cc009920 comp rx=0 tx=0).stop 2024-02-02T15:35:15.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.666+0000 7f37eeb8ab80 1 -- 172.21.15.1:0/1616603240 shutdown_connections 2024-02-02T15:35:15.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.666+0000 7f37eeb8ab80 1 --2- 172.21.15.1:0/1616603240 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558998d587e0 0x558998d57cb0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.666+0000 7f37eeb8ab80 1 --2- 172.21.15.1:0/1616603240 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558998d4d010 0x558998d5f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.666+0000 7f37eeb8ab80 1 --2- 172.21.15.1:0/1616603240 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558998d4d930 0x558998d644b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.666+0000 7f37eeb8ab80 1 -- 172.21.15.1:0/1616603240 >> 172.21.15.1:0/1616603240 conn(0x558998cacdb0 msgr2=0x558998d44e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:15.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.666+0000 7f37eeb8ab80 1 -- 172.21.15.1:0/1616603240 shutdown_connections 2024-02-02T15:35:15.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.666+0000 7f37eeb8ab80 1 -- 172.21.15.1:0/1616603240 wait complete. 2024-02-02T15:35:15.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.666+0000 7f37eeb8ab80 1 -- start start 2024-02-02T15:35:15.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.667+0000 7f37eeb8ab80 1 Processor -- start 2024-02-02T15:35:15.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.667+0000 7f37eeb8ab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558998d4d010 0x558998d5efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:15.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.667+0000 7f37eeb8ab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558998d4d930 0x558998d5d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:15.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.667+0000 7f37eeb8ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558998d587e0 0x558998d5d830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:15.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.667+0000 7f37eeb8ab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x558998d5dd70 con 0x558998d4d930 2024-02-02T15:35:15.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.667+0000 7f37eeb8ab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x558998d5dee0 con 0x558998d4d010 2024-02-02T15:35:15.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.667+0000 7f37eeb8ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558998d5e0a0 con 0x558998d587e0 2024-02-02T15:35:15.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.668+0000 7f37eeb8ab80 10 client.0 _ll_register_callbacks cb 0x558998c3c290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:35:15.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.668+0000 7f37d3e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558998d4d010 0x558998d5efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:15.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.668+0000 7f37d9000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558998d587e0 0x558998d5d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:15.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.668+0000 7f37d3e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558998d4d010 0x558998d5efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:54982/0 (socket says 172.21.15.1:54982) 2024-02-02T15:35:15.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.668+0000 7f37d3e00700 1 -- 172.21.15.1:0/3147526852 learned_addr learned my addr 172.21.15.1:0/3147526852 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:35:15.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.668+0000 7f37d3400700 1 --2- 172.21.15.1:0/3147526852 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558998d4d930 0x558998d5d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:15.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.668+0000 7f37d9000700 1 -- 172.21.15.1:0/3147526852 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558998d4d010 msgr2=0x558998d5efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:15.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.668+0000 7f37d9000700 1 --2- 172.21.15.1:0/3147526852 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558998d4d010 0x558998d5efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.668+0000 7f37d9000700 1 -- 172.21.15.1:0/3147526852 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558998d4d930 msgr2=0x558998d5d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:15.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.668+0000 7f37d9000700 1 --2- 172.21.15.1:0/3147526852 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558998d4d930 0x558998d5d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.668+0000 7f37d9000700 1 -- 172.21.15.1:0/3147526852 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37cc009240 con 0x558998d587e0 2024-02-02T15:35:15.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.668+0000 7f37d3e00700 1 --2- 172.21.15.1:0/3147526852 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558998d4d010 0x558998d5efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:35:15.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.668+0000 7f37d3400700 1 --2- 172.21.15.1:0/3147526852 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558998d4d930 0x558998d5d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:35:15.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.668+0000 7f37d9000700 1 --2- 172.21.15.1:0/3147526852 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558998d587e0 0x558998d5d830 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f37cc009130 tx=0x7f37cc0168c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31583c76ea731550 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:15.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.668+0000 7f37c2000700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:35:15.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.669+0000 7f37c2000700 1 -- 172.21.15.1:0/3147526852 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37cc00e040 con 0x558998d587e0 2024-02-02T15:35:15.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.669+0000 7f37c2000700 1 -- 172.21.15.1:0/3147526852 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f37cc003a00 con 0x558998d587e0 2024-02-02T15:35:15.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.669+0000 7f37eeb8ab80 1 -- 172.21.15.1:0/3147526852 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f37c401e040 con 0x558998d587e0 2024-02-02T15:35:15.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.669+0000 7f37c2000700 1 -- 172.21.15.1:0/3147526852 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f37cc016db0 con 0x558998d587e0 2024-02-02T15:35:15.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.669+0000 7f37eeb8ab80 10 client.27850 fetch_fsmap learned FSMap version 42 2024-02-02T15:35:15.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.669+0000 7f37eeb8ab80 1 -- 172.21.15.1:0/3147526852 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x558998c64ea0 con 0x558998d587e0 2024-02-02T15:35:15.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.669+0000 7f37c2000700 1 -- 172.21.15.1:0/3147526852 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f37c401e040 con 0x558998d587e0 2024-02-02T15:35:15.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.669+0000 7f37c2000700 1 -- 172.21.15.1:0/3147526852 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 42) v1 ==== 7510+0+0 (secure 0 0 0) 0x7f37cc026070 con 0x558998d587e0 2024-02-02T15:35:15.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.669+0000 7f37eeb8ab80 10 client.27850 fetch_fsmap finished waiting for FSMap version 42 2024-02-02T15:35:15.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.669+0000 7f37eeb8ab80 10 client.27850 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-02T15:35:15.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.669+0000 7f37eeb8ab80 20 client.27850 populate_metadata read hostname 'smithi001' 2024-02-02T15:35:15.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.669+0000 7f37eeb8ab80 1 --2- 172.21.15.1:0/3147526852 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x558998dbb200 0x558998dbd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:15.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.669+0000 7f37eeb8ab80 4 client.27850 mds_command: new command op to 24529 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:35:15.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.669+0000 7f37eeb8ab80 1 -- 172.21.15.1:0/3147526852 --> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558998c5df50 con 0x558998dbb200 2024-02-02T15:35:15.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.670+0000 7f37d3e00700 1 --2- 172.21.15.1:0/3147526852 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x558998dbb200 0x558998dbd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:15.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.670+0000 7f37c2000700 10 client.27850 ms_handle_connect on v2:172.21.15.142:6838/321675421 2024-02-02T15:35:15.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.670+0000 7f37d3e00700 1 --2- 172.21.15.1:0/3147526852 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x558998dbb200 0x558998dbd620 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=69103d40ae0b6a28 server_cookie=fa04fffd54748a5f in_seq=0 out_seq=0 2024-02-02T15:35:15.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.670+0000 7f37c2000700 1 -- 172.21.15.1:0/3147526852 <== mds.1 v2:172.21.15.142:6838/321675421 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558998c5df50 con 0x558998dbb200 2024-02-02T15:35:15.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.670+0000 7f37c2000700 10 client.27850 handle_command_reply: tid=0 2024-02-02T15:35:15.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.670+0000 7f37eeb8ab80 1 client.27850 shutdown 2024-02-02T15:35:15.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.670+0000 7f37eeb8ab80 1 -- 172.21.15.1:0/3147526852 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558998d587e0 msgr2=0x558998d5d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:15.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.670+0000 7f37eeb8ab80 1 --2- 172.21.15.1:0/3147526852 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558998d587e0 0x558998d5d830 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f37cc009130 tx=0x7f37cc0168c0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.671+0000 7f37eeb8ab80 1 -- 172.21.15.1:0/3147526852 shutdown_connections 2024-02-02T15:35:15.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.671+0000 7f37eeb8ab80 1 --2- 172.21.15.1:0/3147526852 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x558998dbb200 0x558998dbd620 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.671+0000 7f37eeb8ab80 1 --2- 172.21.15.1:0/3147526852 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558998d587e0 0x558998d5d830 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.671+0000 7f37eeb8ab80 1 --2- 172.21.15.1:0/3147526852 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558998d4d010 0x558998d5efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.671+0000 7f37eeb8ab80 1 --2- 172.21.15.1:0/3147526852 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558998d4d930 0x558998d5d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.671+0000 7f37c2000700 0 client.27850 ms_handle_reset on v2:172.21.15.142:6838/321675421 2024-02-02T15:35:15.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.671+0000 7f37eeb8ab80 1 -- 172.21.15.1:0/3147526852 >> 172.21.15.1:0/3147526852 conn(0x558998cacdb0 msgr2=0x558998d63910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:15.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.671+0000 7f37eeb8ab80 1 -- 172.21.15.1:0/3147526852 shutdown_connections 2024-02-02T15:35:15.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.671+0000 7f37eeb8ab80 1 -- 172.21.15.1:0/3147526852 wait complete. 2024-02-02T15:35:15.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.671+0000 7f37eeb8ab80 20 client.27850 trim_cache size 0 max 16384 2024-02-02T15:35:15.675 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:35:15.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.672+0000 7f37db000700 1 -- 172.21.15.1:0/3065291479 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f37b40a0710 msgr2=0x7f37b40a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:15.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.672+0000 7f37db000700 1 --2- 172.21.15.1:0/3065291479 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f37b40a0710 0x7f37b40a2bd0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f37c400b2e0 tx=0x7f37c4005fd0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.672+0000 7f37db000700 1 -- 172.21.15.1:0/3065291479 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37d4106110 msgr2=0x7f37d4199630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:15.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.672+0000 7f37db000700 1 --2- 172.21.15.1:0/3065291479 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37d4106110 0x7f37d4199630 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f37bc00e970 tx=0x7f37bc00ec80 comp rx=0 tx=0).stop 2024-02-02T15:35:15.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.672+0000 7f37db000700 1 -- 172.21.15.1:0/3065291479 shutdown_connections 2024-02-02T15:35:15.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.672+0000 7f37db000700 1 --2- 172.21.15.1:0/3065291479 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f37b40a0710 0x7f37b40a2bd0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.672+0000 7f37db000700 1 --2- 172.21.15.1:0/3065291479 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37d419da90 0x7f37d419df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.672+0000 7f37db000700 1 --2- 172.21.15.1:0/3065291479 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f37d4100ff0 0x7f37d41990f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.672+0000 7f37db000700 1 --2- 172.21.15.1:0/3065291479 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37d4106110 0x7f37d4199630 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.672+0000 7f37db000700 1 -- 172.21.15.1:0/3065291479 >> 172.21.15.1:0/3065291479 conn(0x7f37d40fc810 msgr2=0x7f37d410cf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:15.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.672+0000 7f37db000700 1 -- 172.21.15.1:0/3065291479 shutdown_connections 2024-02-02T15:35:15.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.672+0000 7f37db000700 1 -- 172.21.15.1:0/3065291479 wait complete. 2024-02-02T15:35:15.678 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:15 smithi142 ceph-mon[45728]: pgmap v254: 129 pgs: 129 active+clean; 335 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 1.4 MiB/s wr, 109 op/s 2024-02-02T15:35:15.678 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:15 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/3305157595' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:35:15.678 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:15 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:35:15.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:15 smithi001 ceph-mon[35470]: pgmap v254: 129 pgs: 129 active+clean; 335 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 1.4 MiB/s wr, 109 op/s 2024-02-02T15:35:15.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:15 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/3305157595' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:35:15.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:15 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:35:15.692 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-02T15:35:15.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:15 smithi071 ceph-mon[44707]: pgmap v254: 129 pgs: 129 active+clean; 335 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 1.4 MiB/s wr, 109 op/s 2024-02-02T15:35:15.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:15 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/3305157595' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:35:15.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:15 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:35:15.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.909+0000 7fa9d1000700 1 -- 172.21.15.1:0/3004991139 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9cc0730e0 msgr2=0x7fa9cc0734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:15.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.909+0000 7fa9d1000700 1 --2- 172.21.15.1:0/3004991139 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9cc0730e0 0x7fa9cc0734c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc009990 tx=0x7fa9bc009ca0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.909+0000 7fa9d1000700 1 -- 172.21.15.1:0/3004991139 shutdown_connections 2024-02-02T15:35:15.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.909+0000 7fa9d1000700 1 --2- 172.21.15.1:0/3004991139 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa9cc111d90 0x7fa9cc114180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.909+0000 7fa9d1000700 1 --2- 172.21.15.1:0/3004991139 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9cc0730e0 0x7fa9cc0734c0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.909+0000 7fa9d1000700 1 --2- 172.21.15.1:0/3004991139 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa9cc073a00 0x7fa9cc111850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.909+0000 7fa9d1000700 1 -- 172.21.15.1:0/3004991139 >> 172.21.15.1:0/3004991139 conn(0x7fa9cc0fc810 msgr2=0x7fa9cc0fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:15.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.909+0000 7fa9d1000700 1 -- 172.21.15.1:0/3004991139 shutdown_connections 2024-02-02T15:35:15.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.910+0000 7fa9d1000700 1 -- 172.21.15.1:0/3004991139 wait complete. 2024-02-02T15:35:15.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.910+0000 7fa9d1000700 1 Processor -- start 2024-02-02T15:35:15.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.911+0000 7fa9d1000700 1 -- start start 2024-02-02T15:35:15.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.911+0000 7fa9d1000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9cc0730e0 0x7fa9cc19d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:15.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.911+0000 7fa9d1000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa9cc073a00 0x7fa9cc19da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:15.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.911+0000 7fa9d1000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa9cc111d90 0x7fa9cc1a1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:15.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.911+0000 7fa9d1000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9cc1a24d0 con 0x7fa9cc073a00 2024-02-02T15:35:15.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.911+0000 7fa9d1000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9cc1a2640 con 0x7fa9cc111d90 2024-02-02T15:35:15.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.911+0000 7fa9d1000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9cc1a2800 con 0x7fa9cc0730e0 2024-02-02T15:35:15.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.911+0000 7fa9c9400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa9cc073a00 0x7fa9cc19da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:15.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.911+0000 7fa9c9e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9cc0730e0 0x7fa9cc19d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:15.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.911+0000 7fa9c9e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9cc0730e0 0x7fa9cc19d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:56222/0 (socket says 172.21.15.1:56222) 2024-02-02T15:35:15.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.911+0000 7fa9c9e00700 1 -- 172.21.15.1:0/999989363 learned_addr learned my addr 172.21.15.1:0/999989363 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:35:15.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.911+0000 7fa9c9400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa9cc073a00 0x7fa9cc19da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46488/0 (socket says 172.21.15.1:46488) 2024-02-02T15:35:15.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.911+0000 7fa9ca800700 1 --2- 172.21.15.1:0/999989363 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa9cc111d90 0x7fa9cc1a1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:15.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.912+0000 7fa9c9e00700 1 -- 172.21.15.1:0/999989363 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa9cc111d90 msgr2=0x7fa9cc1a1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:15.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.912+0000 7fa9c9e00700 1 --2- 172.21.15.1:0/999989363 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa9cc111d90 0x7fa9cc1a1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.912+0000 7fa9c9e00700 1 -- 172.21.15.1:0/999989363 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa9cc073a00 msgr2=0x7fa9cc19da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:15.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.912+0000 7fa9c9e00700 1 --2- 172.21.15.1:0/999989363 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa9cc073a00 0x7fa9cc19da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.912+0000 7fa9c9e00700 1 -- 172.21.15.1:0/999989363 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9bc0095f0 con 0x7fa9cc0730e0 2024-02-02T15:35:15.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.912+0000 7fa9c9400700 1 --2- 172.21.15.1:0/999989363 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa9cc073a00 0x7fa9cc19da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:35:15.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.912+0000 7fa9c9e00700 1 --2- 172.21.15.1:0/999989363 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9cc0730e0 0x7fa9cc19d4f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc005910 tx=0x7fa9bc005720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:15.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.912+0000 7fa9c2a00700 1 -- 172.21.15.1:0/999989363 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9bc01d070 con 0x7fa9cc0730e0 2024-02-02T15:35:15.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.912+0000 7fa9c2a00700 1 -- 172.21.15.1:0/999989363 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa9bc022470 con 0x7fa9cc0730e0 2024-02-02T15:35:15.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.912+0000 7fa9c2a00700 1 -- 172.21.15.1:0/999989363 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9bc017650 con 0x7fa9cc0730e0 2024-02-02T15:35:15.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.912+0000 7fa9d1000700 1 -- 172.21.15.1:0/999989363 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9cc1a2a80 con 0x7fa9cc0730e0 2024-02-02T15:35:15.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.912+0000 7fa9d1000700 1 -- 172.21.15.1:0/999989363 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9cc1a2d00 con 0x7fa9cc0730e0 2024-02-02T15:35:15.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.916+0000 7fa9c2a00700 1 -- 172.21.15.1:0/999989363 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7fa9bc0225e0 con 0x7fa9cc0730e0 2024-02-02T15:35:15.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.917+0000 7fa9c2a00700 1 --2- 172.21.15.1:0/999989363 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fa9ac0a0710 0x7fa9ac0a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:15.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.917+0000 7fa9e4adbb80 1 Processor -- start 2024-02-02T15:35:15.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.917+0000 7fa9c2a00700 1 -- 172.21.15.1:0/999989363 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fa9bc0cd2b0 con 0x7fa9cc0730e0 2024-02-02T15:35:15.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.917+0000 7fa9e4adbb80 1 -- start start 2024-02-02T15:35:15.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.918+0000 7fa9c9400700 1 --2- 172.21.15.1:0/999989363 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fa9ac0a0710 0x7fa9ac0a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:15.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.918+0000 7fa9e4adbb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1afe04010 0x55f1afe043f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:15.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.918+0000 7fa9e4adbb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f1afe04930 0x55f1afe0f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:15.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.918+0000 7fa9e4adbb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f1afe0f7e0 0x55f1afe11bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:15.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.918+0000 7fa9e4adbb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1afd784c0 con 0x55f1afe04010 2024-02-02T15:35:15.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.918+0000 7fa9e4adbb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1afd5caa0 con 0x55f1afe04930 2024-02-02T15:35:15.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.918+0000 7fa9e4adbb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1afd5a460 con 0x55f1afe0f7e0 2024-02-02T15:35:15.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.918+0000 7fa9c9e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1afe04010 0x55f1afe043f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:15.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.918+0000 7fa9c9e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1afe04010 0x55f1afe043f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46500/0 (socket says 172.21.15.1:46500) 2024-02-02T15:35:15.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.918+0000 7fa9c9e00700 1 -- 172.21.15.1:0/1048470841 learned_addr learned my addr 172.21.15.1:0/1048470841 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:35:15.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.918+0000 7fa9c9400700 1 --2- 172.21.15.1:0/1048470841 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f1afe04930 0x55f1afe0f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:15.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.918+0000 7fa9ca800700 1 --2- 172.21.15.1:0/1048470841 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f1afe0f7e0 0x55f1afe11bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:15.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.918+0000 7fa9c9400700 1 --2- 172.21.15.1:0/999989363 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fa9ac0a0710 0x7fa9ac0a2bd0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc19eb00 tx=0x7fa9b4009260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:15.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.919+0000 7fa9c9e00700 1 -- 172.21.15.1:0/1048470841 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f1afe0f7e0 msgr2=0x55f1afe11bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:15.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.919+0000 7fa9c9e00700 1 --2- 172.21.15.1:0/1048470841 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f1afe0f7e0 0x55f1afe11bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.919+0000 7fa9c9e00700 1 -- 172.21.15.1:0/1048470841 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f1afe04930 msgr2=0x55f1afe0f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:15.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.919+0000 7fa9c9e00700 1 --2- 172.21.15.1:0/1048470841 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f1afe04930 0x55f1afe0f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.919+0000 7fa9c9e00700 1 -- 172.21.15.1:0/1048470841 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f1afbd73b0 con 0x55f1afe04010 2024-02-02T15:35:15.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.919+0000 7fa9c9400700 1 --2- 172.21.15.1:0/1048470841 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f1afe04930 0x55f1afe0f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:35:15.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.919+0000 7fa9c9e00700 1 --2- 172.21.15.1:0/1048470841 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1afe04010 0x55f1afe043f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc09efd0 tx=0x7fa9bc009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=139373651cac5fed server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:15.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.919+0000 7fa9bb400700 1 -- 172.21.15.1:0/1048470841 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9bc099dc0 con 0x55f1afe04010 2024-02-02T15:35:15.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.919+0000 7fa9bb400700 1 -- 172.21.15.1:0/1048470841 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa9bc0ade90 con 0x55f1afe04010 2024-02-02T15:35:15.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.919+0000 7fa9bb400700 1 -- 172.21.15.1:0/1048470841 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9bc0a2440 con 0x55f1afe04010 2024-02-02T15:35:15.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.920+0000 7fa9e4adbb80 1 -- 172.21.15.1:0/1048470841 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1afe04010 msgr2=0x55f1afe043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:15.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.920+0000 7fa9e4adbb80 1 --2- 172.21.15.1:0/1048470841 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1afe04010 0x55f1afe043f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc09efd0 tx=0x7fa9bc009ca0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.920+0000 7fa9e4adbb80 1 -- 172.21.15.1:0/1048470841 shutdown_connections 2024-02-02T15:35:15.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.920+0000 7fa9e4adbb80 1 --2- 172.21.15.1:0/1048470841 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f1afe0f7e0 0x55f1afe11bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.920+0000 7fa9e4adbb80 1 --2- 172.21.15.1:0/1048470841 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1afe04010 0x55f1afe043f0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.920+0000 7fa9e4adbb80 1 --2- 172.21.15.1:0/1048470841 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f1afe04930 0x55f1afe0f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.920+0000 7fa9e4adbb80 1 -- 172.21.15.1:0/1048470841 >> 172.21.15.1:0/1048470841 conn(0x55f1afd63db0 msgr2=0x55f1afe003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:15.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.920+0000 7fa9e4adbb80 1 -- 172.21.15.1:0/1048470841 shutdown_connections 2024-02-02T15:35:15.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.920+0000 7fa9e4adbb80 1 -- 172.21.15.1:0/1048470841 wait complete. 2024-02-02T15:35:15.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.921+0000 7fa9e4adbb80 1 -- start start 2024-02-02T15:35:15.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.921+0000 7fa9e4adbb80 1 Processor -- start 2024-02-02T15:35:15.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.922+0000 7fa9e4adbb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1afe04010 0x55f1afe0fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:15.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.922+0000 7fa9e4adbb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f1afe04930 0x55f1afe10100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:15.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.922+0000 7fa9e4adbb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f1afe0f7e0 0x55f1afe10640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:15.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.922+0000 7fa9e4adbb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1afd55d50 con 0x55f1afe04010 2024-02-02T15:35:15.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.922+0000 7fa9e4adbb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1afcf1d80 con 0x55f1afe04930 2024-02-02T15:35:15.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.922+0000 7fa9e4adbb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1afcf1a20 con 0x55f1afe0f7e0 2024-02-02T15:35:15.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.922+0000 7fa9e4adbb80 10 client.0 _ll_register_callbacks cb 0x55f1afcf3290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:35:15.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.922+0000 7fa9c9e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1afe04010 0x55f1afe0fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:15.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.922+0000 7fa9ca800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f1afe0f7e0 0x55f1afe10640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:15.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.922+0000 7fa9c9e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1afe04010 0x55f1afe0fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46510/0 (socket says 172.21.15.1:46510) 2024-02-02T15:35:15.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.922+0000 7fa9c9e00700 1 -- 172.21.15.1:0/4153268799 learned_addr learned my addr 172.21.15.1:0/4153268799 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:35:15.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.922+0000 7fa9c9400700 1 --2- 172.21.15.1:0/4153268799 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f1afe04930 0x55f1afe10100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:15.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.923+0000 7fa9ca800700 1 -- 172.21.15.1:0/4153268799 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f1afe04930 msgr2=0x55f1afe10100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:15.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.923+0000 7fa9ca800700 1 --2- 172.21.15.1:0/4153268799 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f1afe04930 0x55f1afe10100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.923+0000 7fa9ca800700 1 -- 172.21.15.1:0/4153268799 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1afe04010 msgr2=0x55f1afe0fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:15.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.923+0000 7fa9ca800700 1 --2- 172.21.15.1:0/4153268799 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1afe04010 0x55f1afe0fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.923+0000 7fa9ca800700 1 -- 172.21.15.1:0/4153268799 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9b4017050 con 0x55f1afe0f7e0 2024-02-02T15:35:15.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.923+0000 7fa9c9400700 1 --2- 172.21.15.1:0/4153268799 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f1afe04930 0x55f1afe10100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:35:15.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.923+0000 7fa9c9e00700 1 --2- 172.21.15.1:0/4153268799 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1afe04010 0x55f1afe0fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:35:15.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.923+0000 7fa9ca800700 1 --2- 172.21.15.1:0/4153268799 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f1afe0f7e0 0x55f1afe10640 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fa9c4011e60 tx=0x7fa9c4014600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=599f4e65ed2c3e03 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:15.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.923+0000 7fa9b3e00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:35:15.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.923+0000 7fa9b3e00700 1 -- 172.21.15.1:0/4153268799 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c400e040 con 0x55f1afe0f7e0 2024-02-02T15:35:15.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.923+0000 7fa9b3e00700 1 -- 172.21.15.1:0/4153268799 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fa9c40093a0 con 0x55f1afe0f7e0 2024-02-02T15:35:15.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.923+0000 7fa9e4adbb80 1 -- 172.21.15.1:0/4153268799 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f1afdf80f0 con 0x55f1afe0f7e0 2024-02-02T15:35:15.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.924+0000 7fa9b3e00700 1 -- 172.21.15.1:0/4153268799 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa9c4004db0 con 0x55f1afe0f7e0 2024-02-02T15:35:15.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.924+0000 7fa9e4adbb80 10 client.27862 fetch_fsmap learned FSMap version 42 2024-02-02T15:35:15.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.924+0000 7fa9e4adbb80 1 -- 172.21.15.1:0/4153268799 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7fa9cc1a2a80 con 0x55f1afe0f7e0 2024-02-02T15:35:15.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.924+0000 7fa9b3e00700 1 -- 172.21.15.1:0/4153268799 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f1afdf80f0 con 0x55f1afe0f7e0 2024-02-02T15:35:15.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.924+0000 7fa9b3e00700 1 -- 172.21.15.1:0/4153268799 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 42) v1 ==== 7510+0+0 (secure 0 0 0) 0x7fa9c401c030 con 0x55f1afe0f7e0 2024-02-02T15:35:15.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.924+0000 7fa9e4adbb80 10 client.27862 fetch_fsmap finished waiting for FSMap version 42 2024-02-02T15:35:15.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.924+0000 7fa9e4adbb80 10 client.27862 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:35:15.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.924+0000 7fa9e4adbb80 20 client.27862 populate_metadata read hostname 'smithi001' 2024-02-02T15:35:15.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.924+0000 7fa9e4adbb80 1 --2- 172.21.15.1:0/4153268799 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55f1afe1eef0 0x55f1afe21310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:15.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.924+0000 7fa9e4adbb80 4 client.27862 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:35:15.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.924+0000 7fa9e4adbb80 1 -- 172.21.15.1:0/4153268799 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f1afd4edd0 con 0x55f1afe1eef0 2024-02-02T15:35:15.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.924+0000 7fa9c9e00700 1 --2- 172.21.15.1:0/4153268799 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55f1afe1eef0 0x55f1afe21310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:15.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.925+0000 7fa9c9e00700 1 --2- 172.21.15.1:0/4153268799 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55f1afe1eef0 0x55f1afe21310 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee20e3e22a224bb9 server_cookie=7b6640317e69a9a1 in_seq=0 out_seq=0 2024-02-02T15:35:15.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.925+0000 7fa9b3e00700 10 client.27862 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:35:15.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.925+0000 7fa9b3e00700 1 -- 172.21.15.1:0/4153268799 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55f1afd4edd0 con 0x55f1afe1eef0 2024-02-02T15:35:15.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.925+0000 7fa9b3e00700 10 client.27862 handle_command_reply: tid=0 2024-02-02T15:35:15.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.926+0000 7fa9e4adbb80 1 client.27862 shutdown 2024-02-02T15:35:15.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.926+0000 7fa9e4adbb80 1 -- 172.21.15.1:0/4153268799 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f1afe0f7e0 msgr2=0x55f1afe10640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:15.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.926+0000 7fa9e4adbb80 1 --2- 172.21.15.1:0/4153268799 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f1afe0f7e0 0x55f1afe10640 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fa9c4011e60 tx=0x7fa9c4014600 comp rx=0 tx=0).stop 2024-02-02T15:35:15.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.926+0000 7fa9e4adbb80 1 -- 172.21.15.1:0/4153268799 shutdown_connections 2024-02-02T15:35:15.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.926+0000 7fa9e4adbb80 1 --2- 172.21.15.1:0/4153268799 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55f1afe1eef0 0x55f1afe21310 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.926+0000 7fa9e4adbb80 1 --2- 172.21.15.1:0/4153268799 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f1afe0f7e0 0x55f1afe10640 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.926+0000 7fa9e4adbb80 1 --2- 172.21.15.1:0/4153268799 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1afe04010 0x55f1afe0fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.926+0000 7fa9e4adbb80 1 --2- 172.21.15.1:0/4153268799 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f1afe04930 0x55f1afe10100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.926+0000 7fa9b3e00700 0 client.27862 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:35:15.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.926+0000 7fa9e4adbb80 1 -- 172.21.15.1:0/4153268799 >> 172.21.15.1:0/4153268799 conn(0x55f1afd63db0 msgr2=0x55f1afdfffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:15.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.926+0000 7fa9e4adbb80 1 -- 172.21.15.1:0/4153268799 shutdown_connections 2024-02-02T15:35:15.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.926+0000 7fa9e4adbb80 1 -- 172.21.15.1:0/4153268799 wait complete. 2024-02-02T15:35:15.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.926+0000 7fa9e4adbb80 20 client.27862 trim_cache size 0 max 16384 2024-02-02T15:35:15.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.941+0000 7fa9e4adbb80 1 Processor -- start 2024-02-02T15:35:15.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.942+0000 7fa9e4adbb80 1 -- start start 2024-02-02T15:35:15.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.942+0000 7fa9e4adbb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f1afe04010 0x55f1afe13bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:15.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.942+0000 7fa9e4adbb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1afe04930 0x55f1afe06840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:15.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.942+0000 7fa9e4adbb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f1afe0f7e0 0x55f1afe1b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:15.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.942+0000 7fa9e4adbb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1afbd96a0 con 0x55f1afe04930 2024-02-02T15:35:15.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.942+0000 7fa9e4adbb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1afe0a470 con 0x55f1afe0f7e0 2024-02-02T15:35:15.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.942+0000 7fa9e4adbb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1afe0cda0 con 0x55f1afe04010 2024-02-02T15:35:15.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.942+0000 7fa9c9e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f1afe04010 0x55f1afe13bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:15.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.942+0000 7fa9c9e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f1afe04010 0x55f1afe13bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:56238/0 (socket says 172.21.15.1:56238) 2024-02-02T15:35:15.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.942+0000 7fa9c9e00700 1 -- 172.21.15.1:0/4168885405 learned_addr learned my addr 172.21.15.1:0/4168885405 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:35:15.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.942+0000 7fa9c9400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1afe04930 0x55f1afe06840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:15.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.942+0000 7fa9ca800700 1 --2- 172.21.15.1:0/4168885405 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f1afe0f7e0 0x55f1afe1b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:15.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.943+0000 7fa9c9e00700 1 -- 172.21.15.1:0/4168885405 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f1afe0f7e0 msgr2=0x55f1afe1b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:15.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.943+0000 7fa9c9e00700 1 --2- 172.21.15.1:0/4168885405 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f1afe0f7e0 0x55f1afe1b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.943+0000 7fa9c9e00700 1 -- 172.21.15.1:0/4168885405 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1afe04930 msgr2=0x55f1afe06840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:15.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.943+0000 7fa9c9e00700 1 --2- 172.21.15.1:0/4168885405 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1afe04930 0x55f1afe06840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.943+0000 7fa9c9e00700 1 -- 172.21.15.1:0/4168885405 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f1afb83b60 con 0x55f1afe04010 2024-02-02T15:35:15.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.943+0000 7fa9c9e00700 1 --2- 172.21.15.1:0/4168885405 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f1afe04010 0x55f1afe13bf0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc0961e0 tx=0x7fa9bc022c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7162f5bcea5c7c2f server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:15.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.943+0000 7fa9bbe00700 1 -- 172.21.15.1:0/4168885405 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9bc099dc0 con 0x55f1afe04010 2024-02-02T15:35:15.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.943+0000 7fa9bbe00700 1 -- 172.21.15.1:0/4168885405 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa9bc09ab60 con 0x55f1afe04010 2024-02-02T15:35:15.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.943+0000 7fa9bbe00700 1 -- 172.21.15.1:0/4168885405 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9bc09a4f0 con 0x55f1afe04010 2024-02-02T15:35:15.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.944+0000 7fa9e4adbb80 1 -- 172.21.15.1:0/4168885405 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f1afe04010 msgr2=0x55f1afe13bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:15.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.944+0000 7fa9e4adbb80 1 --2- 172.21.15.1:0/4168885405 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f1afe04010 0x55f1afe13bf0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc0961e0 tx=0x7fa9bc022c90 comp rx=0 tx=0).stop 2024-02-02T15:35:15.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.944+0000 7fa9e4adbb80 1 -- 172.21.15.1:0/4168885405 shutdown_connections 2024-02-02T15:35:15.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.944+0000 7fa9e4adbb80 1 --2- 172.21.15.1:0/4168885405 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f1afe0f7e0 0x55f1afe1b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.944+0000 7fa9e4adbb80 1 --2- 172.21.15.1:0/4168885405 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f1afe04010 0x55f1afe13bf0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.944+0000 7fa9e4adbb80 1 --2- 172.21.15.1:0/4168885405 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1afe04930 0x55f1afe06840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.944+0000 7fa9e4adbb80 1 -- 172.21.15.1:0/4168885405 >> 172.21.15.1:0/4168885405 conn(0x55f1afd63db0 msgr2=0x55f1afe090b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:15.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.944+0000 7fa9e4adbb80 1 -- 172.21.15.1:0/4168885405 shutdown_connections 2024-02-02T15:35:15.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.944+0000 7fa9e4adbb80 1 -- 172.21.15.1:0/4168885405 wait complete. 2024-02-02T15:35:15.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.944+0000 7fa9e4adbb80 1 -- start start 2024-02-02T15:35:15.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.945+0000 7fa9e4adbb80 1 Processor -- start 2024-02-02T15:35:15.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.945+0000 7fa9e4adbb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f1afe04010 0x55f1afe111d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:15.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.945+0000 7fa9e4adbb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1afe04930 0x55f1afe11710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:15.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.945+0000 7fa9e4adbb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f1afe0f7e0 0x55f1afe11c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:15.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.945+0000 7fa9e4adbb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1afe6fa40 con 0x55f1afe04930 2024-02-02T15:35:15.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.945+0000 7fa9e4adbb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1afe6fbb0 con 0x55f1afe0f7e0 2024-02-02T15:35:15.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.945+0000 7fa9e4adbb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1afe6fd70 con 0x55f1afe04010 2024-02-02T15:35:15.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.945+0000 7fa9e4adbb80 10 client.0 _ll_register_callbacks cb 0x55f1afcf3290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:35:15.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.945+0000 7fa9c9e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f1afe04010 0x55f1afe111d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:15.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.945+0000 7fa9c9e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f1afe04010 0x55f1afe111d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:56254/0 (socket says 172.21.15.1:56254) 2024-02-02T15:35:15.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.945+0000 7fa9c9e00700 1 -- 172.21.15.1:0/3112396641 learned_addr learned my addr 172.21.15.1:0/3112396641 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:35:15.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.945+0000 7fa9ca800700 1 --2- 172.21.15.1:0/3112396641 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f1afe0f7e0 0x55f1afe11c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:15.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.945+0000 7fa9c9400700 1 --2- 172.21.15.1:0/3112396641 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1afe04930 0x55f1afe11710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:15.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.946+0000 7fa9c9e00700 1 -- 172.21.15.1:0/3112396641 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f1afe0f7e0 msgr2=0x55f1afe11c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:15.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.946+0000 7fa9c9e00700 1 --2- 172.21.15.1:0/3112396641 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f1afe0f7e0 0x55f1afe11c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.946+0000 7fa9c9e00700 1 -- 172.21.15.1:0/3112396641 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1afe04930 msgr2=0x55f1afe11710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:15.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.946+0000 7fa9c9e00700 1 --2- 172.21.15.1:0/3112396641 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1afe04930 0x55f1afe11710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.946+0000 7fa9c9e00700 1 -- 172.21.15.1:0/3112396641 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f1afbd73b0 con 0x55f1afe04010 2024-02-02T15:35:15.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.946+0000 7fa9c9400700 1 --2- 172.21.15.1:0/3112396641 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1afe04930 0x55f1afe11710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:35:15.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.946+0000 7fa9c9e00700 1 --2- 172.21.15.1:0/3112396641 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f1afe04010 0x55f1afe111d0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc00f740 tx=0x7fa9bc09e120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b11a707eb93802eb server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:15.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.946+0000 7fa9b3e00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:35:15.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.946+0000 7fa9b3e00700 1 -- 172.21.15.1:0/3112396641 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9bc099dc0 con 0x55f1afe04010 2024-02-02T15:35:15.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.946+0000 7fa9b3e00700 1 -- 172.21.15.1:0/3112396641 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fa9bc09e290 con 0x55f1afe04010 2024-02-02T15:35:15.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.946+0000 7fa9e4adbb80 1 -- 172.21.15.1:0/3112396641 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa9bc0ade90 con 0x55f1afe04010 2024-02-02T15:35:15.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.946+0000 7fa9b3e00700 1 -- 172.21.15.1:0/3112396641 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa9bc0a2440 con 0x55f1afe04010 2024-02-02T15:35:15.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.946+0000 7fa9e4adbb80 10 client.27868 fetch_fsmap learned FSMap version 42 2024-02-02T15:35:15.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.946+0000 7fa9e4adbb80 1 -- 172.21.15.1:0/3112396641 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55f1afd1bea0 con 0x55f1afe04010 2024-02-02T15:35:15.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.947+0000 7fa9b3e00700 1 -- 172.21.15.1:0/3112396641 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa9bc0ade90 con 0x55f1afe04010 2024-02-02T15:35:15.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.947+0000 7fa9b3e00700 1 -- 172.21.15.1:0/3112396641 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 42) v1 ==== 7510+0+0 (secure 0 0 0) 0x7fa9bc081050 con 0x55f1afe04010 2024-02-02T15:35:15.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.947+0000 7fa9e4adbb80 10 client.27868 fetch_fsmap finished waiting for FSMap version 42 2024-02-02T15:35:15.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.947+0000 7fa9e4adbb80 10 client.27868 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:35:15.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.947+0000 7fa9e4adbb80 20 client.27868 populate_metadata read hostname 'smithi001' 2024-02-02T15:35:15.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.947+0000 7fa9e4adbb80 1 --2- 172.21.15.1:0/3112396641 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55f1afe710c0 0x55f1afe734e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:15.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.947+0000 7fa9e4adbb80 4 client.27868 mds_command: new command op to 24557 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:35:15.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.947+0000 7fa9e4adbb80 1 -- 172.21.15.1:0/3112396641 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f1afd14f50 con 0x55f1afe710c0 2024-02-02T15:35:15.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.947+0000 7fa9c9400700 1 --2- 172.21.15.1:0/3112396641 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55f1afe710c0 0x55f1afe734e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:15.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.948+0000 7fa9c9400700 1 --2- 172.21.15.1:0/3112396641 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55f1afe710c0 0x55f1afe734e0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e36c06c684b5444b server_cookie=4f391f0ba16733af in_seq=0 out_seq=0 2024-02-02T15:35:15.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.948+0000 7fa9b3e00700 10 client.27868 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:35:15.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.948+0000 7fa9b3e00700 1 -- 172.21.15.1:0/3112396641 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f1afd14f50 con 0x55f1afe710c0 2024-02-02T15:35:15.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.948+0000 7fa9b3e00700 10 client.27868 handle_command_reply: tid=0 2024-02-02T15:35:15.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.948+0000 7fa9e4adbb80 1 client.27868 shutdown 2024-02-02T15:35:15.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.948+0000 7fa9e4adbb80 1 -- 172.21.15.1:0/3112396641 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f1afe04010 msgr2=0x55f1afe111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:15.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.948+0000 7fa9e4adbb80 1 --2- 172.21.15.1:0/3112396641 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f1afe04010 0x55f1afe111d0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc00f740 tx=0x7fa9bc09e120 comp rx=0 tx=0).stop 2024-02-02T15:35:15.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.948+0000 7fa9e4adbb80 1 -- 172.21.15.1:0/3112396641 shutdown_connections 2024-02-02T15:35:15.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.948+0000 7fa9e4adbb80 1 --2- 172.21.15.1:0/3112396641 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55f1afe710c0 0x55f1afe734e0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.948+0000 7fa9e4adbb80 1 --2- 172.21.15.1:0/3112396641 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f1afe0f7e0 0x55f1afe11c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.948+0000 7fa9e4adbb80 1 --2- 172.21.15.1:0/3112396641 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f1afe04010 0x55f1afe111d0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.948+0000 7fa9b3e00700 0 client.27868 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:35:15.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.948+0000 7fa9e4adbb80 1 --2- 172.21.15.1:0/3112396641 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1afe04930 0x55f1afe11710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.948+0000 7fa9e4adbb80 1 -- 172.21.15.1:0/3112396641 >> 172.21.15.1:0/3112396641 conn(0x55f1afd63db0 msgr2=0x55f1afe0cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:15.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.948+0000 7fa9e4adbb80 1 -- 172.21.15.1:0/3112396641 shutdown_connections 2024-02-02T15:35:15.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.948+0000 7fa9e4adbb80 1 -- 172.21.15.1:0/3112396641 wait complete. 2024-02-02T15:35:15.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.948+0000 7fa9e4adbb80 20 client.27868 trim_cache size 0 max 16384 2024-02-02T15:35:15.954 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:35:15.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.949+0000 7fa9d1000700 1 -- 172.21.15.1:0/999989363 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fa9ac0a0710 msgr2=0x7fa9ac0a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:15.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.949+0000 7fa9d1000700 1 --2- 172.21.15.1:0/999989363 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fa9ac0a0710 0x7fa9ac0a2bd0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc19eb00 tx=0x7fa9b4009260 comp rx=0 tx=0).stop 2024-02-02T15:35:15.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.949+0000 7fa9d1000700 1 -- 172.21.15.1:0/999989363 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9cc0730e0 msgr2=0x7fa9cc19d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:15.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.949+0000 7fa9d1000700 1 --2- 172.21.15.1:0/999989363 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9cc0730e0 0x7fa9cc19d4f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc005910 tx=0x7fa9bc005720 comp rx=0 tx=0).stop 2024-02-02T15:35:15.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.949+0000 7fa9d1000700 1 -- 172.21.15.1:0/999989363 shutdown_connections 2024-02-02T15:35:15.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.949+0000 7fa9d1000700 1 --2- 172.21.15.1:0/999989363 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fa9ac0a0710 0x7fa9ac0a2bd0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.949+0000 7fa9d1000700 1 --2- 172.21.15.1:0/999989363 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa9cc111d90 0x7fa9cc1a1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.949+0000 7fa9d1000700 1 --2- 172.21.15.1:0/999989363 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa9cc0730e0 0x7fa9cc19d4f0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.949+0000 7fa9d1000700 1 --2- 172.21.15.1:0/999989363 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa9cc073a00 0x7fa9cc19da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:15.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.949+0000 7fa9d1000700 1 -- 172.21.15.1:0/999989363 >> 172.21.15.1:0/999989363 conn(0x7fa9cc0fc810 msgr2=0x7fa9cc0fd480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:15.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.949+0000 7fa9d1000700 1 -- 172.21.15.1:0/999989363 shutdown_connections 2024-02-02T15:35:15.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:15.949+0000 7fa9d1000700 1 -- 172.21.15.1:0/999989363 wait complete. 2024-02-02T15:35:16.250 INFO:tasks.workunit.client.0.smithi001.stdout: 1 481372 6.40 MB/sec execute 290 sec latency 33.223 ms 2024-02-02T15:35:16.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:16 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:35:16.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:16 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:35:16.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:16 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:35:16.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:16 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:35:16.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:16 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:35:16.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:16 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:35:16.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:16 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:35:16.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:16 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:35:16.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:16 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:35:16.969 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-02T15:35:16.970 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-02T15:35:17.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.183+0000 7f1ac5c00700 1 -- 172.21.15.1:0/2608881420 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ac01019b0 msgr2=0x7f1ac01059f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:17.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.183+0000 7f1ac5c00700 1 --2- 172.21.15.1:0/2608881420 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ac01019b0 0x7f1ac01059f0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f1aa8009990 tx=0x7f1aa8009ca0 comp rx=0 tx=0).stop 2024-02-02T15:35:17.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.184+0000 7f1ac5c00700 1 -- 172.21.15.1:0/2608881420 shutdown_connections 2024-02-02T15:35:17.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.184+0000 7f1ac5c00700 1 --2- 172.21.15.1:0/2608881420 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ac0106120 0x7f1ac010dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:17.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.184+0000 7f1ac5c00700 1 --2- 172.21.15.1:0/2608881420 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ac0101000 0x7f1ac01013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:17.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.184+0000 7f1ac5c00700 1 --2- 172.21.15.1:0/2608881420 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ac01019b0 0x7f1ac01059f0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:17.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.184+0000 7f1ac5c00700 1 -- 172.21.15.1:0/2608881420 >> 172.21.15.1:0/2608881420 conn(0x7f1ac00fc860 msgr2=0x7f1ac00fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:17.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.184+0000 7f1ac5c00700 1 -- 172.21.15.1:0/2608881420 shutdown_connections 2024-02-02T15:35:17.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.184+0000 7f1ac5c00700 1 -- 172.21.15.1:0/2608881420 wait complete. 2024-02-02T15:35:17.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.185+0000 7f1ac5c00700 1 Processor -- start 2024-02-02T15:35:17.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.185+0000 7f1ac5c00700 1 -- start start 2024-02-02T15:35:17.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.185+0000 7f1ac5c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ac0101000 0x7f1ac0199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:17.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.185+0000 7f1ac5c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ac0106120 0x7f1ac0199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:17.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.185+0000 7f1ac5c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ac019daa0 0x7f1ac019df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:17.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.185+0000 7f1ac5c00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac019e520 con 0x7f1ac019daa0 2024-02-02T15:35:17.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.185+0000 7f1ac5c00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac019e690 con 0x7f1ac0106120 2024-02-02T15:35:17.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.185+0000 7f1ac5c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac019e850 con 0x7f1ac0101000 2024-02-02T15:35:17.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.185+0000 7f1abe000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ac0106120 0x7f1ac0199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:17.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.185+0000 7f1abea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ac0101000 0x7f1ac0199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:17.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.185+0000 7f1abe000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ac0106120 0x7f1ac0199640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:55052/0 (socket says 172.21.15.1:55052) 2024-02-02T15:35:17.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.185+0000 7f1abe000700 1 -- 172.21.15.1:0/2166591001 learned_addr learned my addr 172.21.15.1:0/2166591001 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:35:17.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.185+0000 7f1abea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ac0101000 0x7f1ac0199100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:56270/0 (socket says 172.21.15.1:56270) 2024-02-02T15:35:17.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.185+0000 7f1abf400700 1 --2- 172.21.15.1:0/2166591001 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ac019daa0 0x7f1ac019df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:17.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.186+0000 7f1abe000700 1 -- 172.21.15.1:0/2166591001 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ac0101000 msgr2=0x7f1ac0199100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:17.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.186+0000 7f1abe000700 1 --2- 172.21.15.1:0/2166591001 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ac0101000 0x7f1ac0199100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:17.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.186+0000 7f1abe000700 1 -- 172.21.15.1:0/2166591001 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ac019daa0 msgr2=0x7f1ac019df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:17.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.186+0000 7f1abe000700 1 --2- 172.21.15.1:0/2166591001 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ac019daa0 0x7f1ac019df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:17.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.186+0000 7f1abe000700 1 -- 172.21.15.1:0/2166591001 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aa80095f0 con 0x7f1ac0106120 2024-02-02T15:35:17.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.186+0000 7f1abf400700 1 --2- 172.21.15.1:0/2166591001 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ac019daa0 0x7f1ac019df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:35:17.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.186+0000 7f1abe000700 1 --2- 172.21.15.1:0/2166591001 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ac0106120 0x7f1ac0199640 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f1aa8004740 tx=0x7f1aa8009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:17.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.186+0000 7f1ab7400700 1 -- 172.21.15.1:0/2166591001 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa801d070 con 0x7f1ac0106120 2024-02-02T15:35:17.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.186+0000 7f1ab7400700 1 -- 172.21.15.1:0/2166591001 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1aa8004830 con 0x7f1ac0106120 2024-02-02T15:35:17.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.186+0000 7f1ac5c00700 1 -- 172.21.15.1:0/2166591001 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ac01a5d50 con 0x7f1ac0106120 2024-02-02T15:35:17.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.186+0000 7f1ab7400700 1 -- 172.21.15.1:0/2166591001 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa8005280 con 0x7f1ac0106120 2024-02-02T15:35:17.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.187+0000 7f1ac5c00700 1 -- 172.21.15.1:0/2166591001 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ac01a61d0 con 0x7f1ac0106120 2024-02-02T15:35:17.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.189+0000 7f1ab7400700 1 -- 172.21.15.1:0/2166591001 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f1aa800fb00 con 0x7f1ac0106120 2024-02-02T15:35:17.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.191+0000 7f1ad96b7b80 1 Processor -- start 2024-02-02T15:35:17.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.191+0000 7f1ab7400700 1 --2- 172.21.15.1:0/2166591001 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f1aa00a0710 0x7f1aa00a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:17.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.191+0000 7f1ab7400700 1 -- 172.21.15.1:0/2166591001 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f1aa80cd100 con 0x7f1ac0106120 2024-02-02T15:35:17.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.191+0000 7f1ad96b7b80 1 -- start start 2024-02-02T15:35:17.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.191+0000 7f1abf400700 1 --2- 172.21.15.1:0/2166591001 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f1aa00a0710 0x7f1aa00a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:17.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.191+0000 7f1ad96b7b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5556ae44f090 0x5556ae44f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:17.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.191+0000 7f1ad96b7b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556ae44f9b0 0x5556ae45a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:17.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.191+0000 7f1ad96b7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556ae45a860 0x5556ae45cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:17.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.191+0000 7f1ad96b7b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556ae3c3540 con 0x5556ae44f090 2024-02-02T15:35:17.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.191+0000 7f1ad96b7b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556ae3a7b20 con 0x5556ae44f9b0 2024-02-02T15:35:17.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.191+0000 7f1ad96b7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556ae3a54e0 con 0x5556ae45a860 2024-02-02T15:35:17.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.192+0000 7f1abea00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5556ae44f090 0x5556ae44f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:17.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.192+0000 7f1abea00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5556ae44f090 0x5556ae44f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46538/0 (socket says 172.21.15.1:46538) 2024-02-02T15:35:17.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.192+0000 7f1abea00700 1 -- 172.21.15.1:0/2227041885 learned_addr learned my addr 172.21.15.1:0/2227041885 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:35:17.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.192+0000 7f1abe000700 1 --2- 172.21.15.1:0/2227041885 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556ae44f9b0 0x5556ae45a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:17.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.192+0000 7f1abf400700 1 --2- 172.21.15.1:0/2227041885 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556ae45a860 0x5556ae45cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:17.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.192+0000 7f1abe000700 1 -- 172.21.15.1:0/2227041885 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556ae45a860 msgr2=0x5556ae45cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:17.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.192+0000 7f1abe000700 1 --2- 172.21.15.1:0/2227041885 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556ae45a860 0x5556ae45cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:17.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.192+0000 7f1abe000700 1 -- 172.21.15.1:0/2227041885 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5556ae44f090 msgr2=0x5556ae44f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:17.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.192+0000 7f1abe000700 1 --2- 172.21.15.1:0/2227041885 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5556ae44f090 0x5556ae44f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:17.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.192+0000 7f1abe000700 1 -- 172.21.15.1:0/2227041885 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5556ae222430 con 0x5556ae44f9b0 2024-02-02T15:35:17.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.192+0000 7f1abea00700 1 --2- 172.21.15.1:0/2227041885 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5556ae44f090 0x5556ae44f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:35:17.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.192+0000 7f1abf400700 1 --2- 172.21.15.1:0/2166591001 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f1aa00a0710 0x7f1aa00a2bd0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8003e80 tx=0x7f1ab8009920 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:17.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.192+0000 7f1abe000700 1 --2- 172.21.15.1:0/2227041885 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556ae44f9b0 0x5556ae45a320 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f1aa80ca420 tx=0x7f1aa8099f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dea8ffd3d7c6e42a server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:17.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.193+0000 7f1aafe00700 1 -- 172.21.15.1:0/2227041885 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa809e0e0 con 0x5556ae44f9b0 2024-02-02T15:35:17.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.193+0000 7f1aafe00700 1 -- 172.21.15.1:0/2227041885 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1aa80a2290 con 0x5556ae44f9b0 2024-02-02T15:35:17.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.193+0000 7f1aafe00700 1 -- 172.21.15.1:0/2227041885 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa80997a0 con 0x5556ae44f9b0 2024-02-02T15:35:17.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.193+0000 7f1ad96b7b80 1 -- 172.21.15.1:0/2227041885 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556ae44f9b0 msgr2=0x5556ae45a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:17.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.193+0000 7f1ad96b7b80 1 --2- 172.21.15.1:0/2227041885 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556ae44f9b0 0x5556ae45a320 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f1aa80ca420 tx=0x7f1aa8099f00 comp rx=0 tx=0).stop 2024-02-02T15:35:17.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.193+0000 7f1ad96b7b80 1 -- 172.21.15.1:0/2227041885 shutdown_connections 2024-02-02T15:35:17.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.193+0000 7f1ad96b7b80 1 --2- 172.21.15.1:0/2227041885 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556ae45a860 0x5556ae45cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:17.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.193+0000 7f1ad96b7b80 1 --2- 172.21.15.1:0/2227041885 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5556ae44f090 0x5556ae44f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:17.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.193+0000 7f1ad96b7b80 1 --2- 172.21.15.1:0/2227041885 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556ae44f9b0 0x5556ae45a320 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:17.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.193+0000 7f1ad96b7b80 1 -- 172.21.15.1:0/2227041885 >> 172.21.15.1:0/2227041885 conn(0x5556ae3aee30 msgr2=0x5556ae44b430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:17.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.194+0000 7f1ad96b7b80 1 -- 172.21.15.1:0/2227041885 shutdown_connections 2024-02-02T15:35:17.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.194+0000 7f1ad96b7b80 1 -- 172.21.15.1:0/2227041885 wait complete. 2024-02-02T15:35:17.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.194+0000 7f1ad96b7b80 1 -- start start 2024-02-02T15:35:17.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.194+0000 7f1ad96b7b80 1 Processor -- start 2024-02-02T15:35:17.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.195+0000 7f1ad96b7b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556ae44f090 0x5556ae45ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:17.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.195+0000 7f1ad96b7b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5556ae44f9b0 0x5556ae45b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:17.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.195+0000 7f1ad96b7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556ae45a860 0x5556ae45b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:17.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.195+0000 7f1ad96b7b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556ae3a0dd0 con 0x5556ae44f9b0 2024-02-02T15:35:17.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.195+0000 7f1ad96b7b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556ae33ce00 con 0x5556ae44f090 2024-02-02T15:35:17.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.195+0000 7f1ad96b7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556ae33caa0 con 0x5556ae45a860 2024-02-02T15:35:17.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.195+0000 7f1ad96b7b80 10 client.0 _ll_register_callbacks cb 0x5556ae33e310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:35:17.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.195+0000 7f1abea00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556ae44f090 0x5556ae45ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:17.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.195+0000 7f1abea00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556ae44f090 0x5556ae45ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:55072/0 (socket says 172.21.15.1:55072) 2024-02-02T15:35:17.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.195+0000 7f1abea00700 1 -- 172.21.15.1:0/1697463787 learned_addr learned my addr 172.21.15.1:0/1697463787 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:35:17.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.195+0000 7f1abf400700 1 --2- 172.21.15.1:0/1697463787 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556ae45a860 0x5556ae45b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:17.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.195+0000 7f1abe000700 1 --2- 172.21.15.1:0/1697463787 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5556ae44f9b0 0x5556ae45b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:17.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.195+0000 7f1abe000700 1 -- 172.21.15.1:0/1697463787 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556ae45a860 msgr2=0x5556ae45b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:17.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.195+0000 7f1abe000700 1 --2- 172.21.15.1:0/1697463787 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556ae45a860 0x5556ae45b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:17.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.196+0000 7f1abe000700 1 -- 172.21.15.1:0/1697463787 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556ae44f090 msgr2=0x5556ae45ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:17.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.196+0000 7f1abe000700 1 --2- 172.21.15.1:0/1697463787 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556ae44f090 0x5556ae45ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:17.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.196+0000 7f1abe000700 1 -- 172.21.15.1:0/1697463787 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ab00099e0 con 0x5556ae44f9b0 2024-02-02T15:35:17.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.196+0000 7f1abf400700 1 --2- 172.21.15.1:0/1697463787 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556ae45a860 0x5556ae45b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:35:17.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.196+0000 7f1abea00700 1 --2- 172.21.15.1:0/1697463787 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556ae44f090 0x5556ae45ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:35:17.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.196+0000 7f1abe000700 1 --2- 172.21.15.1:0/1697463787 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5556ae44f9b0 0x5556ae45b180 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f1aa809a240 tx=0x7f1aa8094ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2f6c62002ba6a03 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:17.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.196+0000 7f1aacc00700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:35:17.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.196+0000 7f1aacc00700 1 -- 172.21.15.1:0/1697463787 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa809a510 con 0x5556ae44f9b0 2024-02-02T15:35:17.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.196+0000 7f1aacc00700 1 -- 172.21.15.1:0/1697463787 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f1aa809a670 con 0x5556ae44f9b0 2024-02-02T15:35:17.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.196+0000 7f1ad96b7b80 1 -- 172.21.15.1:0/1697463787 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5556ae443170 con 0x5556ae44f9b0 2024-02-02T15:35:17.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.196+0000 7f1aacc00700 1 -- 172.21.15.1:0/1697463787 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1aa80b80d0 con 0x5556ae44f9b0 2024-02-02T15:35:17.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.196+0000 7f1ad96b7b80 10 client.17796 fetch_fsmap learned FSMap version 42 2024-02-02T15:35:17.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.196+0000 7f1ad96b7b80 1 -- 172.21.15.1:0/1697463787 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7f1ac01a61d0 con 0x5556ae44f9b0 2024-02-02T15:35:17.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.196+0000 7f1aacc00700 1 -- 172.21.15.1:0/1697463787 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5556ae443170 con 0x5556ae44f9b0 2024-02-02T15:35:17.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.197+0000 7f1aacc00700 1 -- 172.21.15.1:0/1697463787 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 42) v1 ==== 7510+0+0 (secure 0 0 0) 0x7f1aa80cacc0 con 0x5556ae44f9b0 2024-02-02T15:35:17.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.197+0000 7f1ad96b7b80 10 client.17796 fetch_fsmap finished waiting for FSMap version 42 2024-02-02T15:35:17.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.197+0000 7f1ad96b7b80 10 client.17796 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:35:17.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.197+0000 7f1ad96b7b80 20 client.17796 populate_metadata read hostname 'smithi001' 2024-02-02T15:35:17.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.197+0000 7f1ad96b7b80 1 --2- 172.21.15.1:0/1697463787 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5556ae469a90 0x5556ae46be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:17.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.197+0000 7f1ad96b7b80 4 client.17796 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:35:17.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.197+0000 7f1ad96b7b80 1 -- 172.21.15.1:0/1697463787 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5556ae399e50 con 0x5556ae469a90 2024-02-02T15:35:17.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.197+0000 7f1abea00700 1 --2- 172.21.15.1:0/1697463787 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5556ae469a90 0x5556ae46be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:17.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.198+0000 7f1abea00700 1 --2- 172.21.15.1:0/1697463787 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5556ae469a90 0x5556ae46be80 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70bc8a91116c4bb server_cookie=86cb6a5d0e3cabb1 in_seq=0 out_seq=0 2024-02-02T15:35:17.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.198+0000 7f1aacc00700 10 client.17796 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:35:17.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.198+0000 7f1aacc00700 1 -- 172.21.15.1:0/1697463787 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5556ae399e50 con 0x5556ae469a90 2024-02-02T15:35:17.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.198+0000 7f1aacc00700 10 client.17796 handle_command_reply: tid=0 2024-02-02T15:35:17.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.198+0000 7f1ad96b7b80 1 client.17796 shutdown 2024-02-02T15:35:17.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.199+0000 7f1ad96b7b80 1 -- 172.21.15.1:0/1697463787 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5556ae44f9b0 msgr2=0x5556ae45b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:17.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.199+0000 7f1ad96b7b80 1 --2- 172.21.15.1:0/1697463787 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5556ae44f9b0 0x5556ae45b180 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f1aa809a240 tx=0x7f1aa8094ec0 comp rx=0 tx=0).stop 2024-02-02T15:35:17.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.199+0000 7f1ad96b7b80 1 -- 172.21.15.1:0/1697463787 shutdown_connections 2024-02-02T15:35:17.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.199+0000 7f1ad96b7b80 1 --2- 172.21.15.1:0/1697463787 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5556ae469a90 0x5556ae46be80 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:17.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.199+0000 7f1ad96b7b80 1 --2- 172.21.15.1:0/1697463787 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556ae45a860 0x5556ae45b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:17.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.199+0000 7f1ad96b7b80 1 --2- 172.21.15.1:0/1697463787 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556ae44f090 0x5556ae45ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:17.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.199+0000 7f1aacc00700 0 client.17796 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:35:17.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.199+0000 7f1ad96b7b80 1 --2- 172.21.15.1:0/1697463787 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5556ae44f9b0 0x5556ae45b180 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:17.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.199+0000 7f1ad96b7b80 1 -- 172.21.15.1:0/1697463787 >> 172.21.15.1:0/1697463787 conn(0x5556ae3aee30 msgr2=0x5556ae44b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:17.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.199+0000 7f1ad96b7b80 1 -- 172.21.15.1:0/1697463787 shutdown_connections 2024-02-02T15:35:17.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.199+0000 7f1ad96b7b80 1 -- 172.21.15.1:0/1697463787 wait complete. 2024-02-02T15:35:17.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.199+0000 7f1ad96b7b80 20 client.17796 trim_cache size 0 max 16384 2024-02-02T15:35:17.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.217+0000 7f1ad96b7b80 1 Processor -- start 2024-02-02T15:35:17.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.217+0000 7f1ad96b7b80 1 -- start start 2024-02-02T15:35:17.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.217+0000 7f1ad96b7b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5556ae44f090 0x5556ae465260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:17.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.217+0000 7f1ad96b7b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556ae44f9b0 0x5556ae44af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:17.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.217+0000 7f1ad96b7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556ae45a860 0x5556ae44b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:17.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.217+0000 7f1ad96b7b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556ae224720 con 0x5556ae44f090 2024-02-02T15:35:17.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.217+0000 7f1ad96b7b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556ae450340 con 0x5556ae44f9b0 2024-02-02T15:35:17.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.217+0000 7f1ad96b7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556ae465d60 con 0x5556ae45a860 2024-02-02T15:35:17.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.218+0000 7f1abe000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556ae44f9b0 0x5556ae44af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:17.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.218+0000 7f1abf400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556ae45a860 0x5556ae44b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:17.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.218+0000 7f1abea00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5556ae44f090 0x5556ae465260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:17.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.218+0000 7f1abe000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556ae44f9b0 0x5556ae44af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:55078/0 (socket says 172.21.15.1:55078) 2024-02-02T15:35:17.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.218+0000 7f1abf400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556ae45a860 0x5556ae44b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:56294/0 (socket says 172.21.15.1:56294) 2024-02-02T15:35:17.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.218+0000 7f1abe000700 1 -- 172.21.15.1:0/861783039 learned_addr learned my addr 172.21.15.1:0/861783039 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:35:17.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.218+0000 7f1abe000700 1 -- 172.21.15.1:0/861783039 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556ae45a860 msgr2=0x5556ae44b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:17.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.218+0000 7f1abe000700 1 --2- 172.21.15.1:0/861783039 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556ae45a860 0x5556ae44b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:17.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.218+0000 7f1abe000700 1 -- 172.21.15.1:0/861783039 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5556ae44f090 msgr2=0x5556ae465260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:17.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.218+0000 7f1abe000700 1 --2- 172.21.15.1:0/861783039 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5556ae44f090 0x5556ae465260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:17.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.218+0000 7f1abe000700 1 -- 172.21.15.1:0/861783039 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5556ae1cebe0 con 0x5556ae44f9b0 2024-02-02T15:35:17.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.218+0000 7f1abe000700 1 --2- 172.21.15.1:0/861783039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556ae44f9b0 0x5556ae44af10 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f1aa800b2e0 tx=0x7f1aa80ca530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b18d04e7fc48e660 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:17.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.218+0000 7f1ab4c00700 1 -- 172.21.15.1:0/861783039 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa80b80d0 con 0x5556ae44f9b0 2024-02-02T15:35:17.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.218+0000 7f1ab4c00700 1 -- 172.21.15.1:0/861783039 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1aa80979a0 con 0x5556ae44f9b0 2024-02-02T15:35:17.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.218+0000 7f1ab4c00700 1 -- 172.21.15.1:0/861783039 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa80a2060 con 0x5556ae44f9b0 2024-02-02T15:35:17.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.219+0000 7f1ad96b7b80 1 -- 172.21.15.1:0/861783039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556ae44f9b0 msgr2=0x5556ae44af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:17.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.219+0000 7f1ad96b7b80 1 --2- 172.21.15.1:0/861783039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556ae44f9b0 0x5556ae44af10 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f1aa800b2e0 tx=0x7f1aa80ca530 comp rx=0 tx=0).stop 2024-02-02T15:35:17.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.219+0000 7f1ad96b7b80 1 -- 172.21.15.1:0/861783039 shutdown_connections 2024-02-02T15:35:17.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.219+0000 7f1ad96b7b80 1 --2- 172.21.15.1:0/861783039 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556ae45a860 0x5556ae44b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:17.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.219+0000 7f1ad96b7b80 1 --2- 172.21.15.1:0/861783039 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5556ae44f090 0x5556ae465260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:17.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.219+0000 7f1ad96b7b80 1 --2- 172.21.15.1:0/861783039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556ae44f9b0 0x5556ae44af10 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:17.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.219+0000 7f1ad96b7b80 1 -- 172.21.15.1:0/861783039 >> 172.21.15.1:0/861783039 conn(0x5556ae3aee30 msgr2=0x5556ae466280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:17.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.219+0000 7f1ad96b7b80 1 -- 172.21.15.1:0/861783039 shutdown_connections 2024-02-02T15:35:17.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.219+0000 7f1ad96b7b80 1 -- 172.21.15.1:0/861783039 wait complete. 2024-02-02T15:35:17.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.220+0000 7f1ad96b7b80 1 -- start start 2024-02-02T15:35:17.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.220+0000 7f1ad96b7b80 1 Processor -- start 2024-02-02T15:35:17.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.220+0000 7f1ad96b7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556ae44f090 0x5556ae45fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:17.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.220+0000 7f1ad96b7b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5556ae44f9b0 0x5556ae45c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:17.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.220+0000 7f1ad96b7b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556ae45a860 0x5556ae45c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:17.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.220+0000 7f1ad96b7b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556ae45cd00 con 0x5556ae44f9b0 2024-02-02T15:35:17.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.220+0000 7f1ad96b7b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556ae45ce70 con 0x5556ae45a860 2024-02-02T15:35:17.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.220+0000 7f1ad96b7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556ae4bb510 con 0x5556ae44f090 2024-02-02T15:35:17.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.220+0000 7f1ad96b7b80 10 client.0 _ll_register_callbacks cb 0x5556ae33e310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:35:17.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.220+0000 7f1abf400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556ae45a860 0x5556ae45c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:17.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.220+0000 7f1abe000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5556ae44f9b0 0x5556ae45c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:17.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.221+0000 7f1abf400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556ae45a860 0x5556ae45c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:55082/0 (socket says 172.21.15.1:55082) 2024-02-02T15:35:17.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.221+0000 7f1abf400700 1 -- 172.21.15.1:0/696550440 learned_addr learned my addr 172.21.15.1:0/696550440 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:35:17.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.221+0000 7f1abea00700 1 --2- 172.21.15.1:0/696550440 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556ae44f090 0x5556ae45fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:17.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.221+0000 7f1abe000700 1 -- 172.21.15.1:0/696550440 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556ae44f090 msgr2=0x5556ae45fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:17.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.221+0000 7f1abe000700 1 --2- 172.21.15.1:0/696550440 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556ae44f090 0x5556ae45fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:17.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.221+0000 7f1abe000700 1 -- 172.21.15.1:0/696550440 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556ae45a860 msgr2=0x5556ae45c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:17.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.221+0000 7f1abe000700 1 --2- 172.21.15.1:0/696550440 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556ae45a860 0x5556ae45c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:17.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.221+0000 7f1abe000700 1 -- 172.21.15.1:0/696550440 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ac01a61d0 con 0x5556ae44f9b0 2024-02-02T15:35:17.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.221+0000 7f1abf400700 1 --2- 172.21.15.1:0/696550440 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556ae45a860 0x5556ae45c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:35:17.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.221+0000 7f1abe000700 1 --2- 172.21.15.1:0/696550440 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5556ae44f9b0 0x5556ae45c060 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x5556ae447a50 tx=0x7f1aa809aab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6bba197f5f0da47 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:17.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.221+0000 7f1aacc00700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:35:17.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.221+0000 7f1aacc00700 1 -- 172.21.15.1:0/696550440 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa809a710 con 0x5556ae44f9b0 2024-02-02T15:35:17.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.221+0000 7f1aacc00700 1 -- 172.21.15.1:0/696550440 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f1aa80caba0 con 0x5556ae44f9b0 2024-02-02T15:35:17.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.221+0000 7f1ad96b7b80 1 -- 172.21.15.1:0/696550440 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1aa80a2290 con 0x5556ae44f9b0 2024-02-02T15:35:17.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.222+0000 7f1aacc00700 1 -- 172.21.15.1:0/696550440 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1aa8097b10 con 0x5556ae44f9b0 2024-02-02T15:35:17.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.222+0000 7f1ad96b7b80 10 client.17802 fetch_fsmap learned FSMap version 42 2024-02-02T15:35:17.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.222+0000 7f1ad96b7b80 1 -- 172.21.15.1:0/696550440 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x5556ae366f20 con 0x5556ae44f9b0 2024-02-02T15:35:17.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.222+0000 7f1aacc00700 1 -- 172.21.15.1:0/696550440 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1aa80a2290 con 0x5556ae44f9b0 2024-02-02T15:35:17.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.222+0000 7f1aacc00700 1 -- 172.21.15.1:0/696550440 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 42) v1 ==== 7510+0+0 (secure 0 0 0) 0x7f1aa8099210 con 0x5556ae44f9b0 2024-02-02T15:35:17.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.222+0000 7f1ad96b7b80 10 client.17802 fetch_fsmap finished waiting for FSMap version 42 2024-02-02T15:35:17.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.222+0000 7f1ad96b7b80 10 client.17802 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:35:17.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.222+0000 7f1ad96b7b80 20 client.17802 populate_metadata read hostname 'smithi001' 2024-02-02T15:35:17.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.222+0000 7f1ad96b7b80 1 --2- 172.21.15.1:0/696550440 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5556ae4bc750 0x5556ae4beb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:17.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.222+0000 7f1ad96b7b80 4 client.17802 mds_command: new command op to 24557 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-02T15:35:17.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.222+0000 7f1ad96b7b80 1 -- 172.21.15.1:0/696550440 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5556ae35ffd0 con 0x5556ae4bc750 2024-02-02T15:35:17.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.222+0000 7f1abea00700 1 --2- 172.21.15.1:0/696550440 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5556ae4bc750 0x5556ae4beb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:17.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.223+0000 7f1abea00700 1 --2- 172.21.15.1:0/696550440 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5556ae4bc750 0x5556ae4beb40 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b09a70a2144a372b server_cookie=ebb7e6d33e4a345 in_seq=0 out_seq=0 2024-02-02T15:35:17.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.223+0000 7f1aacc00700 10 client.17802 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:35:17.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.224+0000 7f1aacc00700 1 -- 172.21.15.1:0/696550440 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5556ae35ffd0 con 0x5556ae4bc750 2024-02-02T15:35:17.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.224+0000 7f1aacc00700 10 client.17802 handle_command_reply: tid=0 2024-02-02T15:35:17.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.224+0000 7f1ad96b7b80 1 client.17802 shutdown 2024-02-02T15:35:17.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.224+0000 7f1ad96b7b80 1 -- 172.21.15.1:0/696550440 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5556ae44f9b0 msgr2=0x5556ae45c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:17.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.224+0000 7f1ad96b7b80 1 --2- 172.21.15.1:0/696550440 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5556ae44f9b0 0x5556ae45c060 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x5556ae447a50 tx=0x7f1aa809aab0 comp rx=0 tx=0).stop 2024-02-02T15:35:17.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.225+0000 7f1ad96b7b80 1 -- 172.21.15.1:0/696550440 shutdown_connections 2024-02-02T15:35:17.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.225+0000 7f1ad96b7b80 1 --2- 172.21.15.1:0/696550440 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5556ae4bc750 0x5556ae4beb40 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:17.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.225+0000 7f1ad96b7b80 1 --2- 172.21.15.1:0/696550440 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556ae45a860 0x5556ae45c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:17.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.225+0000 7f1ad96b7b80 1 --2- 172.21.15.1:0/696550440 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556ae44f090 0x5556ae45fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:17.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.225+0000 7f1ad96b7b80 1 --2- 172.21.15.1:0/696550440 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5556ae44f9b0 0x5556ae45c060 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:17.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.225+0000 7f1ad96b7b80 1 -- 172.21.15.1:0/696550440 >> 172.21.15.1:0/696550440 conn(0x5556ae3aee30 msgr2=0x5556ae44af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:17.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.225+0000 7f1aacc00700 0 client.17802 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:35:17.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.225+0000 7f1ad96b7b80 1 -- 172.21.15.1:0/696550440 shutdown_connections 2024-02-02T15:35:17.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.225+0000 7f1ad96b7b80 1 -- 172.21.15.1:0/696550440 wait complete. 2024-02-02T15:35:17.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.225+0000 7f1ad96b7b80 20 client.17802 trim_cache size 0 max 16384 2024-02-02T15:35:17.229 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:35:17.229 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2024-02-02T15:35:17.229 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "1bf855c0-56d6-427b-ae57-04b778bf6c55", 2024-02-02T15:35:17.229 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2024-02-02T15:35:17.229 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:35:17.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.225+0000 7f1ac5c00700 1 -- 172.21.15.1:0/2166591001 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f1aa00a0710 msgr2=0x7f1aa00a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:17.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.225+0000 7f1ac5c00700 1 --2- 172.21.15.1:0/2166591001 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f1aa00a0710 0x7f1aa00a2bd0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8003e80 tx=0x7f1ab8009920 comp rx=0 tx=0).stop 2024-02-02T15:35:17.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.225+0000 7f1ac5c00700 1 -- 172.21.15.1:0/2166591001 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ac0106120 msgr2=0x7f1ac0199640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:17.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.225+0000 7f1ac5c00700 1 --2- 172.21.15.1:0/2166591001 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ac0106120 0x7f1ac0199640 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f1aa8004740 tx=0x7f1aa8009580 comp rx=0 tx=0).stop 2024-02-02T15:35:17.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.226+0000 7f1ac5c00700 1 -- 172.21.15.1:0/2166591001 shutdown_connections 2024-02-02T15:35:17.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.226+0000 7f1ac5c00700 1 --2- 172.21.15.1:0/2166591001 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f1aa00a0710 0x7f1aa00a2bd0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:17.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.226+0000 7f1ac5c00700 1 --2- 172.21.15.1:0/2166591001 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ac019daa0 0x7f1ac019df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:17.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.226+0000 7f1ac5c00700 1 --2- 172.21.15.1:0/2166591001 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ac0101000 0x7f1ac0199100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:17.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.226+0000 7f1ac5c00700 1 --2- 172.21.15.1:0/2166591001 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ac0106120 0x7f1ac0199640 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:17.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.226+0000 7f1ac5c00700 1 -- 172.21.15.1:0/2166591001 >> 172.21.15.1:0/2166591001 conn(0x7f1ac00fc860 msgr2=0x7f1ac00fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:17.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.226+0000 7f1ac5c00700 1 -- 172.21.15.1:0/2166591001 shutdown_connections 2024-02-02T15:35:17.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.226+0000 7f1ac5c00700 1 -- 172.21.15.1:0/2166591001 wait complete. 2024-02-02T15:35:17.245 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-02T15:35:17.250 INFO:tasks.workunit.client.0.smithi001.stdout: 1 482640 6.40 MB/sec execute 291 sec latency 226.562 ms 2024-02-02T15:35:17.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.459+0000 7f1f3e600700 1 -- 172.21.15.1:0/2742913981 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f381019b0 msgr2=0x7f1f381059f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:17.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.459+0000 7f1f3e600700 1 --2- 172.21.15.1:0/2742913981 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f381019b0 0x7f1f381059f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f1f200098f0 tx=0x7f1f20009c00 comp rx=0 tx=0).stop 2024-02-02T15:35:17.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.459+0000 7f1f3e600700 1 -- 172.21.15.1:0/2742913981 shutdown_connections 2024-02-02T15:35:17.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.459+0000 7f1f3e600700 1 --2- 172.21.15.1:0/2742913981 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f38106120 0x7f1f3810dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:17.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.459+0000 7f1f3e600700 1 --2- 172.21.15.1:0/2742913981 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f38101000 0x7f1f381013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:17.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.459+0000 7f1f3e600700 1 --2- 172.21.15.1:0/2742913981 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f381019b0 0x7f1f381059f0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:17.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.459+0000 7f1f3e600700 1 -- 172.21.15.1:0/2742913981 >> 172.21.15.1:0/2742913981 conn(0x7f1f380fc860 msgr2=0x7f1f380fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:17.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.459+0000 7f1f3e600700 1 -- 172.21.15.1:0/2742913981 shutdown_connections 2024-02-02T15:35:17.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.459+0000 7f1f3e600700 1 -- 172.21.15.1:0/2742913981 wait complete. 2024-02-02T15:35:17.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.460+0000 7f1f3e600700 1 Processor -- start 2024-02-02T15:35:17.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.460+0000 7f1f3e600700 1 -- start start 2024-02-02T15:35:17.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.460+0000 7f1f3e600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f38101000 0x7f1f38199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:17.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.460+0000 7f1f3e600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f38106120 0x7f1f38199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:17.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.460+0000 7f1f3e600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f3819daa0 0x7f1f3819df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:17.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.460+0000 7f1f3e600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3819e520 con 0x7f1f38101000 2024-02-02T15:35:17.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.460+0000 7f1f3e600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3819e690 con 0x7f1f38106120 2024-02-02T15:35:17.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.460+0000 7f1f3e600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f3819e850 con 0x7f1f3819daa0 2024-02-02T15:35:17.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.461+0000 7f1f37400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f38101000 0x7f1f38199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:17.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.461+0000 7f1f36a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f38106120 0x7f1f38199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:17.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.461+0000 7f1f37e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f3819daa0 0x7f1f3819df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:17.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.461+0000 7f1f37400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f38101000 0x7f1f38199100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46594/0 (socket says 172.21.15.1:46594) 2024-02-02T15:35:17.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.461+0000 7f1f37400700 1 -- 172.21.15.1:0/739475173 learned_addr learned my addr 172.21.15.1:0/739475173 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:35:17.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.461+0000 7f1f37e00700 1 -- 172.21.15.1:0/739475173 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f38106120 msgr2=0x7f1f38199640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:17.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.461+0000 7f1f37e00700 1 --2- 172.21.15.1:0/739475173 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f38106120 0x7f1f38199640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:17.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.461+0000 7f1f37e00700 1 -- 172.21.15.1:0/739475173 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f38101000 msgr2=0x7f1f38199100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:17.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.461+0000 7f1f37e00700 1 --2- 172.21.15.1:0/739475173 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f38101000 0x7f1f38199100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:17.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.461+0000 7f1f37e00700 1 -- 172.21.15.1:0/739475173 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f30009d00 con 0x7f1f3819daa0 2024-02-02T15:35:17.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.461+0000 7f1f37400700 1 --2- 172.21.15.1:0/739475173 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f38101000 0x7f1f38199100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:35:17.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.461+0000 7f1f36a00700 1 --2- 172.21.15.1:0/739475173 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f38106120 0x7f1f38199640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:35:17.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.461+0000 7f1f37e00700 1 --2- 172.21.15.1:0/739475173 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f3819daa0 0x7f1f3819df20 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f1f30009cd0 tx=0x7f1f3000c620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:17.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.462+0000 7f1f2fe00700 1 -- 172.21.15.1:0/739475173 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f30017070 con 0x7f1f3819daa0 2024-02-02T15:35:17.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.462+0000 7f1f2fe00700 1 -- 172.21.15.1:0/739475173 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f3000f040 con 0x7f1f3819daa0 2024-02-02T15:35:17.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.462+0000 7f1f2fe00700 1 -- 172.21.15.1:0/739475173 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3000e050 con 0x7f1f3819daa0 2024-02-02T15:35:17.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.462+0000 7f1f3e600700 1 -- 172.21.15.1:0/739475173 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f20009520 con 0x7f1f3819daa0 2024-02-02T15:35:17.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.462+0000 7f1f3e600700 1 -- 172.21.15.1:0/739475173 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f381a6050 con 0x7f1f3819daa0 2024-02-02T15:35:17.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.465+0000 7f1f2fe00700 1 -- 172.21.15.1:0/739475173 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f1f3000cd10 con 0x7f1f3819daa0 2024-02-02T15:35:17.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.467+0000 7f1f2fe00700 1 --2- 172.21.15.1:0/739475173 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f1f180a0710 0x7f1f180a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:17.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.467+0000 7f1f2fe00700 1 -- 172.21.15.1:0/739475173 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f1f300cb970 con 0x7f1f3819daa0 2024-02-02T15:35:17.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.467+0000 7f1f521e4b80 1 Processor -- start 2024-02-02T15:35:17.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.467+0000 7f1f37400700 1 --2- 172.21.15.1:0/739475173 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f1f180a0710 0x7f1f180a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:17.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.467+0000 7f1f521e4b80 1 -- start start 2024-02-02T15:35:17.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.468+0000 7f1f37400700 1 --2- 172.21.15.1:0/739475173 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f1f180a0710 0x7f1f180a2bd0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1f280104e0 tx=0x7f1f28007480 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:17.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.468+0000 7f1f521e4b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5591aa382010 0x5591aa3823f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:17.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.468+0000 7f1f521e4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5591aa382930 0x5591aa38d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:17.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.468+0000 7f1f521e4b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5591aa38d7e0 0x5591aa38fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:17.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.468+0000 7f1f521e4b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591aa2f64c0 con 0x5591aa38d7e0 2024-02-02T15:35:17.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.468+0000 7f1f521e4b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591aa2daaa0 con 0x5591aa382010 2024-02-02T15:35:17.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.468+0000 7f1f521e4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591aa2d8460 con 0x5591aa382930 2024-02-02T15:35:17.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.468+0000 7f1f37400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5591aa382010 0x5591aa3823f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:17.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.468+0000 7f1f37400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5591aa382010 0x5591aa3823f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:55112/0 (socket says 172.21.15.1:55112) 2024-02-02T15:35:17.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.468+0000 7f1f37400700 1 -- 172.21.15.1:0/3836132358 learned_addr learned my addr 172.21.15.1:0/3836132358 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:35:17.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.468+0000 7f1f36a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5591aa382930 0x5591aa38d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:17.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.468+0000 7f1f37e00700 1 --2- 172.21.15.1:0/3836132358 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5591aa38d7e0 0x5591aa38fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:17.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.468+0000 7f1f36a00700 1 -- 172.21.15.1:0/3836132358 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5591aa382010 msgr2=0x5591aa3823f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:17.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.468+0000 7f1f36a00700 1 --2- 172.21.15.1:0/3836132358 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5591aa382010 0x5591aa3823f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:17.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.469+0000 7f1f36a00700 1 -- 172.21.15.1:0/3836132358 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5591aa38d7e0 msgr2=0x5591aa38fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:17.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.469+0000 7f1f36a00700 1 --2- 172.21.15.1:0/3836132358 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5591aa38d7e0 0x5591aa38fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:17.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.469+0000 7f1f36a00700 1 -- 172.21.15.1:0/3836132358 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5591aa1553b0 con 0x5591aa382930 2024-02-02T15:35:17.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.469+0000 7f1f37400700 1 --2- 172.21.15.1:0/3836132358 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5591aa382010 0x5591aa3823f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:35:17.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.469+0000 7f1f36a00700 1 --2- 172.21.15.1:0/3836132358 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5591aa382930 0x5591aa38d2a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f1f20000c00 tx=0x7f1f20005b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4646865d6725fe36 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:17.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.469+0000 7f1f2cc00700 1 -- 172.21.15.1:0/3836132358 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f2001c050 con 0x5591aa382930 2024-02-02T15:35:17.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.469+0000 7f1f2cc00700 1 -- 172.21.15.1:0/3836132358 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f2001d040 con 0x5591aa382930 2024-02-02T15:35:17.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.469+0000 7f1f2cc00700 1 -- 172.21.15.1:0/3836132358 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f200052a0 con 0x5591aa382930 2024-02-02T15:35:17.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.470+0000 7f1f521e4b80 1 -- 172.21.15.1:0/3836132358 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5591aa382930 msgr2=0x5591aa38d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:17.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.470+0000 7f1f521e4b80 1 --2- 172.21.15.1:0/3836132358 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5591aa382930 0x5591aa38d2a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f1f20000c00 tx=0x7f1f20005b40 comp rx=0 tx=0).stop 2024-02-02T15:35:17.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.470+0000 7f1f521e4b80 1 -- 172.21.15.1:0/3836132358 shutdown_connections 2024-02-02T15:35:17.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.470+0000 7f1f521e4b80 1 --2- 172.21.15.1:0/3836132358 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5591aa38d7e0 0x5591aa38fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:17.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.470+0000 7f1f521e4b80 1 --2- 172.21.15.1:0/3836132358 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5591aa382010 0x5591aa3823f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:17.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.470+0000 7f1f521e4b80 1 --2- 172.21.15.1:0/3836132358 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5591aa382930 0x5591aa38d2a0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:17.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.470+0000 7f1f521e4b80 1 -- 172.21.15.1:0/3836132358 >> 172.21.15.1:0/3836132358 conn(0x5591aa2e1db0 msgr2=0x5591aa37e3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:17.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.470+0000 7f1f521e4b80 1 -- 172.21.15.1:0/3836132358 shutdown_connections 2024-02-02T15:35:17.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.470+0000 7f1f521e4b80 1 -- 172.21.15.1:0/3836132358 wait complete. 2024-02-02T15:35:17.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.471+0000 7f1f521e4b80 1 -- start start 2024-02-02T15:35:17.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.471+0000 7f1f521e4b80 1 Processor -- start 2024-02-02T15:35:17.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.472+0000 7f1f521e4b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5591aa382010 0x5591aa38d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:17.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.472+0000 7f1f521e4b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5591aa382930 0x5591aa38a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:17.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.472+0000 7f1f521e4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5591aa38d7e0 0x5591aa38aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:17.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.472+0000 7f1f521e4b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591aa2d3d50 con 0x5591aa382930 2024-02-02T15:35:17.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.472+0000 7f1f521e4b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591aa26fd80 con 0x5591aa382010 2024-02-02T15:35:17.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.472+0000 7f1f521e4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591aa26fa20 con 0x5591aa38d7e0 2024-02-02T15:35:17.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.472+0000 7f1f521e4b80 10 client.0 _ll_register_callbacks cb 0x5591aa271290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:35:17.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.472+0000 7f1f36a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5591aa382930 0x5591aa38a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:17.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.472+0000 7f1f36a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5591aa382930 0x5591aa38a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46622/0 (socket says 172.21.15.1:46622) 2024-02-02T15:35:17.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.472+0000 7f1f36a00700 1 -- 172.21.15.1:0/1843938816 learned_addr learned my addr 172.21.15.1:0/1843938816 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:35:17.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.472+0000 7f1f37400700 1 --2- 172.21.15.1:0/1843938816 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5591aa382010 0x5591aa38d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:17.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.472+0000 7f1f37e00700 1 --2- 172.21.15.1:0/1843938816 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5591aa38d7e0 0x5591aa38aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:17.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.472+0000 7f1f36a00700 1 -- 172.21.15.1:0/1843938816 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5591aa38d7e0 msgr2=0x5591aa38aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:17.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.472+0000 7f1f36a00700 1 --2- 172.21.15.1:0/1843938816 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5591aa38d7e0 0x5591aa38aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:17.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.472+0000 7f1f36a00700 1 -- 172.21.15.1:0/1843938816 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5591aa382010 msgr2=0x5591aa38d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:17.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.472+0000 7f1f36a00700 1 --2- 172.21.15.1:0/1843938816 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5591aa382010 0x5591aa38d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:17.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.473+0000 7f1f36a00700 1 -- 172.21.15.1:0/1843938816 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f28011440 con 0x5591aa382930 2024-02-02T15:35:17.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.473+0000 7f1f36a00700 1 --2- 172.21.15.1:0/1843938816 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5591aa382930 0x5591aa38a420 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f1f200092a0 tx=0x7f1f2000b580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cd550311db2f273 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:17.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.473+0000 7f1f25600700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:35:17.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.473+0000 7f1f25600700 1 -- 172.21.15.1:0/1843938816 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f2000ba40 con 0x5591aa382930 2024-02-02T15:35:17.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.473+0000 7f1f25600700 1 -- 172.21.15.1:0/1843938816 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f1f200050c0 con 0x5591aa382930 2024-02-02T15:35:17.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.473+0000 7f1f521e4b80 1 -- 172.21.15.1:0/1843938816 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5591aa3760f0 con 0x5591aa382930 2024-02-02T15:35:17.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.473+0000 7f1f25600700 1 -- 172.21.15.1:0/1843938816 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f2000bba0 con 0x5591aa382930 2024-02-02T15:35:17.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.473+0000 7f1f521e4b80 10 client.17814 fetch_fsmap learned FSMap version 42 2024-02-02T15:35:17.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.473+0000 7f1f521e4b80 1 -- 172.21.15.1:0/1843938816 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7f1f3810e510 con 0x5591aa382930 2024-02-02T15:35:17.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.474+0000 7f1f25600700 1 -- 172.21.15.1:0/1843938816 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5591aa3760f0 con 0x5591aa382930 2024-02-02T15:35:17.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.474+0000 7f1f25600700 1 -- 172.21.15.1:0/1843938816 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 42) v1 ==== 7510+0+0 (secure 0 0 0) 0x7f1f20009c70 con 0x5591aa382930 2024-02-02T15:35:17.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.474+0000 7f1f521e4b80 10 client.17814 fetch_fsmap finished waiting for FSMap version 42 2024-02-02T15:35:17.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.474+0000 7f1f521e4b80 10 client.17814 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:35:17.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.474+0000 7f1f521e4b80 20 client.17814 populate_metadata read hostname 'smithi001' 2024-02-02T15:35:17.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.474+0000 7f1f521e4b80 1 --2- 172.21.15.1:0/1843938816 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5591aa39cd00 0x5591aa39f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:17.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.474+0000 7f1f521e4b80 4 client.17814 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:35:17.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.474+0000 7f1f521e4b80 1 -- 172.21.15.1:0/1843938816 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5591aa2ccdd0 con 0x5591aa39cd00 2024-02-02T15:35:17.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.474+0000 7f1f37400700 1 --2- 172.21.15.1:0/1843938816 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5591aa39cd00 0x5591aa39f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:17.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.475+0000 7f1f37400700 1 --2- 172.21.15.1:0/1843938816 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5591aa39cd00 0x5591aa39f120 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc57a5e74b4088b server_cookie=b147ff92169a593d in_seq=0 out_seq=0 2024-02-02T15:35:17.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.475+0000 7f1f25600700 10 client.17814 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:35:17.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.475+0000 7f1f25600700 1 -- 172.21.15.1:0/1843938816 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5591aa2ccdd0 con 0x5591aa39cd00 2024-02-02T15:35:17.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.476+0000 7f1f25600700 10 client.17814 handle_command_reply: tid=0 2024-02-02T15:35:17.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.476+0000 7f1f521e4b80 1 client.17814 shutdown 2024-02-02T15:35:17.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.476+0000 7f1f521e4b80 1 -- 172.21.15.1:0/1843938816 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5591aa382930 msgr2=0x5591aa38a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:17.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.476+0000 7f1f521e4b80 1 --2- 172.21.15.1:0/1843938816 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5591aa382930 0x5591aa38a420 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f1f200092a0 tx=0x7f1f2000b580 comp rx=0 tx=0).stop 2024-02-02T15:35:17.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.476+0000 7f1f521e4b80 1 -- 172.21.15.1:0/1843938816 shutdown_connections 2024-02-02T15:35:17.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.476+0000 7f1f521e4b80 1 --2- 172.21.15.1:0/1843938816 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5591aa39cd00 0x5591aa39f120 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:17.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.476+0000 7f1f521e4b80 1 --2- 172.21.15.1:0/1843938816 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5591aa38d7e0 0x5591aa38aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:17.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.476+0000 7f1f521e4b80 1 --2- 172.21.15.1:0/1843938816 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5591aa382010 0x5591aa38d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:17.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.476+0000 7f1f25600700 0 client.17814 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:35:17.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.476+0000 7f1f521e4b80 1 --2- 172.21.15.1:0/1843938816 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5591aa382930 0x5591aa38a420 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:17.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.476+0000 7f1f521e4b80 1 -- 172.21.15.1:0/1843938816 >> 172.21.15.1:0/1843938816 conn(0x5591aa2e1db0 msgr2=0x5591aa37dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:17.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.476+0000 7f1f521e4b80 1 -- 172.21.15.1:0/1843938816 shutdown_connections 2024-02-02T15:35:17.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.476+0000 7f1f521e4b80 1 -- 172.21.15.1:0/1843938816 wait complete. 2024-02-02T15:35:17.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.476+0000 7f1f521e4b80 20 client.17814 trim_cache size 0 max 16384 2024-02-02T15:35:17.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.494+0000 7f1f521e4b80 1 Processor -- start 2024-02-02T15:35:17.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.494+0000 7f1f521e4b80 1 -- start start 2024-02-02T15:35:17.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.495+0000 7f1f521e4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5591aa382010 0x5591aa3942d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:17.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.495+0000 7f1f521e4b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5591aa382930 0x5591aa38ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:17.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.495+0000 7f1f521e4b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5591aa38d7e0 0x5591aa37dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:17.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.495+0000 7f1f521e4b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591aa1576a0 con 0x5591aa38d7e0 2024-02-02T15:35:17.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.495+0000 7f1f521e4b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591aa386050 con 0x5591aa382930 2024-02-02T15:35:17.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.495+0000 7f1f521e4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591aa39f780 con 0x5591aa382010 2024-02-02T15:35:17.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.495+0000 7f1f36a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5591aa382930 0x5591aa38ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:17.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.495+0000 7f1f37e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5591aa38d7e0 0x5591aa37dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:17.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.495+0000 7f1f36a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5591aa382930 0x5591aa38ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:55136/0 (socket says 172.21.15.1:55136) 2024-02-02T15:35:17.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.495+0000 7f1f36a00700 1 -- 172.21.15.1:0/3469377675 learned_addr learned my addr 172.21.15.1:0/3469377675 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:35:17.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.495+0000 7f1f37400700 1 --2- 172.21.15.1:0/3469377675 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5591aa382010 0x5591aa3942d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:17.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.495+0000 7f1f36a00700 1 -- 172.21.15.1:0/3469377675 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5591aa382010 msgr2=0x5591aa3942d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:17.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.495+0000 7f1f36a00700 1 --2- 172.21.15.1:0/3469377675 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5591aa382010 0x5591aa3942d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:17.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.495+0000 7f1f36a00700 1 -- 172.21.15.1:0/3469377675 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5591aa38d7e0 msgr2=0x5591aa37dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:17.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.495+0000 7f1f36a00700 1 --2- 172.21.15.1:0/3469377675 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5591aa38d7e0 0x5591aa37dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:17.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.495+0000 7f1f36a00700 1 -- 172.21.15.1:0/3469377675 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5591aa101b60 con 0x5591aa382930 2024-02-02T15:35:17.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.495+0000 7f1f37e00700 1 --2- 172.21.15.1:0/3469377675 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5591aa38d7e0 0x5591aa37dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:35:17.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.496+0000 7f1f36a00700 1 --2- 172.21.15.1:0/3469377675 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5591aa382930 0x5591aa38ccd0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f1f20003ef0 tx=0x7f1f20009230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d959aa62361f1769 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:17.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.496+0000 7f1f2d600700 1 -- 172.21.15.1:0/3469377675 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f2001c070 con 0x5591aa382930 2024-02-02T15:35:17.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.496+0000 7f1f2d600700 1 -- 172.21.15.1:0/3469377675 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f2002c5d0 con 0x5591aa382930 2024-02-02T15:35:17.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.496+0000 7f1f2d600700 1 -- 172.21.15.1:0/3469377675 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f20009c20 con 0x5591aa382930 2024-02-02T15:35:17.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.496+0000 7f1f521e4b80 1 -- 172.21.15.1:0/3469377675 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5591aa382930 msgr2=0x5591aa38ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:17.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.497+0000 7f1f521e4b80 1 --2- 172.21.15.1:0/3469377675 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5591aa382930 0x5591aa38ccd0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f1f20003ef0 tx=0x7f1f20009230 comp rx=0 tx=0).stop 2024-02-02T15:35:17.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.497+0000 7f1f521e4b80 1 -- 172.21.15.1:0/3469377675 shutdown_connections 2024-02-02T15:35:17.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.497+0000 7f1f521e4b80 1 --2- 172.21.15.1:0/3469377675 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5591aa38d7e0 0x5591aa37dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:17.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.497+0000 7f1f521e4b80 1 --2- 172.21.15.1:0/3469377675 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5591aa382010 0x5591aa3942d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:17.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.497+0000 7f1f521e4b80 1 --2- 172.21.15.1:0/3469377675 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5591aa382930 0x5591aa38ccd0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:17.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.497+0000 7f1f521e4b80 1 -- 172.21.15.1:0/3469377675 >> 172.21.15.1:0/3469377675 conn(0x5591aa2e1db0 msgr2=0x5591aa37d5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:17.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.497+0000 7f1f521e4b80 1 -- 172.21.15.1:0/3469377675 shutdown_connections 2024-02-02T15:35:17.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.497+0000 7f1f521e4b80 1 -- 172.21.15.1:0/3469377675 wait complete. 2024-02-02T15:35:17.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.497+0000 7f1f521e4b80 1 -- start start 2024-02-02T15:35:17.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.497+0000 7f1f521e4b80 1 Processor -- start 2024-02-02T15:35:17.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.498+0000 7f1f521e4b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5591aa382010 0x5591aa393ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:17.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.498+0000 7f1f521e4b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5591aa382930 0x5591aa394000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:17.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.498+0000 7f1f521e4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5591aa38d7e0 0x5591aa391ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:17.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.498+0000 7f1f521e4b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591aa3925f0 con 0x5591aa382930 2024-02-02T15:35:17.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.498+0000 7f1f521e4b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591aa392760 con 0x5591aa382010 2024-02-02T15:35:17.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.498+0000 7f1f521e4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591aa392920 con 0x5591aa38d7e0 2024-02-02T15:35:17.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.498+0000 7f1f521e4b80 10 client.0 _ll_register_callbacks cb 0x5591aa399c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:35:17.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.498+0000 7f1f37400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5591aa382010 0x5591aa393ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:17.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.498+0000 7f1f37400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5591aa382010 0x5591aa393ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:55144/0 (socket says 172.21.15.1:55144) 2024-02-02T15:35:17.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.498+0000 7f1f37400700 1 -- 172.21.15.1:0/3437155902 learned_addr learned my addr 172.21.15.1:0/3437155902 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:35:17.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.498+0000 7f1f37e00700 1 --2- 172.21.15.1:0/3437155902 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5591aa38d7e0 0x5591aa391ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:17.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.498+0000 7f1f36a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5591aa382930 0x5591aa394000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:17.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.498+0000 7f1f37400700 1 -- 172.21.15.1:0/3437155902 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5591aa38d7e0 msgr2=0x5591aa391ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:17.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.498+0000 7f1f37400700 1 --2- 172.21.15.1:0/3437155902 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5591aa38d7e0 0x5591aa391ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:17.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.498+0000 7f1f37400700 1 -- 172.21.15.1:0/3437155902 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5591aa382930 msgr2=0x5591aa394000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:17.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.498+0000 7f1f37400700 1 --2- 172.21.15.1:0/3437155902 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5591aa382930 0x5591aa394000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:17.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.498+0000 7f1f37400700 1 -- 172.21.15.1:0/3437155902 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f381a6050 con 0x5591aa382010 2024-02-02T15:35:17.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.499+0000 7f1f37400700 1 --2- 172.21.15.1:0/3437155902 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5591aa382010 0x5591aa393ac0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x5591aa39f9f0 tx=0x7f1f280047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30528352b10de236 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:17.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.499+0000 7f1f25600700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:35:17.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.499+0000 7f1f25600700 1 -- 172.21.15.1:0/3437155902 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f28024040 con 0x5591aa382010 2024-02-02T15:35:17.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.499+0000 7f1f25600700 1 -- 172.21.15.1:0/3437155902 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f1f28017070 con 0x5591aa382010 2024-02-02T15:35:17.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.499+0000 7f1f521e4b80 1 -- 172.21.15.1:0/3437155902 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1f2001d040 con 0x5591aa382010 2024-02-02T15:35:17.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.499+0000 7f1f25600700 1 -- 172.21.15.1:0/3437155902 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f28029b90 con 0x5591aa382010 2024-02-02T15:35:17.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.499+0000 7f1f521e4b80 10 client.27908 fetch_fsmap learned FSMap version 42 2024-02-02T15:35:17.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.499+0000 7f1f521e4b80 1 -- 172.21.15.1:0/3437155902 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7f1f3810e510 con 0x5591aa382010 2024-02-02T15:35:17.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.500+0000 7f1f25600700 1 -- 172.21.15.1:0/3437155902 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f2001d040 con 0x5591aa382010 2024-02-02T15:35:17.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.500+0000 7f1f25600700 1 -- 172.21.15.1:0/3437155902 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 42) v1 ==== 7510+0+0 (secure 0 0 0) 0x7f1f2801fc60 con 0x5591aa382010 2024-02-02T15:35:17.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.500+0000 7f1f521e4b80 10 client.27908 fetch_fsmap finished waiting for FSMap version 42 2024-02-02T15:35:17.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.500+0000 7f1f521e4b80 10 client.27908 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:35:17.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.500+0000 7f1f521e4b80 20 client.27908 populate_metadata read hostname 'smithi001' 2024-02-02T15:35:17.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.500+0000 7f1f521e4b80 1 --2- 172.21.15.1:0/3437155902 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5591aa3efa80 0x5591aa3f1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:17.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.500+0000 7f1f521e4b80 4 client.27908 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2024-02-02T15:35:17.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.500+0000 7f1f521e4b80 1 -- 172.21.15.1:0/3437155902 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5591aa292f50 con 0x5591aa3efa80 2024-02-02T15:35:17.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.500+0000 7f1f36a00700 1 --2- 172.21.15.1:0/3437155902 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5591aa3efa80 0x5591aa3f1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:17.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.501+0000 7f1f36a00700 1 --2- 172.21.15.1:0/3437155902 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5591aa3efa80 0x5591aa3f1ea0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d517600b8eeb10e4 server_cookie=5feed2297095657f in_seq=0 out_seq=0 2024-02-02T15:35:17.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.501+0000 7f1f25600700 10 client.27908 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:35:17.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.501+0000 7f1f25600700 1 -- 172.21.15.1:0/3437155902 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5591aa292f50 con 0x5591aa3efa80 2024-02-02T15:35:17.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.501+0000 7f1f25600700 10 client.27908 handle_command_reply: tid=0 2024-02-02T15:35:17.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.501+0000 7f1f521e4b80 1 client.27908 shutdown 2024-02-02T15:35:17.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.501+0000 7f1f521e4b80 1 -- 172.21.15.1:0/3437155902 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5591aa382010 msgr2=0x5591aa393ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:17.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.501+0000 7f1f521e4b80 1 --2- 172.21.15.1:0/3437155902 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5591aa382010 0x5591aa393ac0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x5591aa39f9f0 tx=0x7f1f280047c0 comp rx=0 tx=0).stop 2024-02-02T15:35:17.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.501+0000 7f1f521e4b80 1 -- 172.21.15.1:0/3437155902 shutdown_connections 2024-02-02T15:35:17.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.501+0000 7f1f521e4b80 1 --2- 172.21.15.1:0/3437155902 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5591aa3efa80 0x5591aa3f1ea0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:17.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.501+0000 7f1f521e4b80 1 --2- 172.21.15.1:0/3437155902 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5591aa38d7e0 0x5591aa391ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:17.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.501+0000 7f1f521e4b80 1 --2- 172.21.15.1:0/3437155902 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5591aa382010 0x5591aa393ac0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:17.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.501+0000 7f1f521e4b80 1 --2- 172.21.15.1:0/3437155902 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5591aa382930 0x5591aa394000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:17.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.501+0000 7f1f25600700 0 client.27908 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:35:17.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.501+0000 7f1f521e4b80 1 -- 172.21.15.1:0/3437155902 >> 172.21.15.1:0/3437155902 conn(0x5591aa2e1db0 msgr2=0x5591aa37de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:17.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.501+0000 7f1f521e4b80 1 -- 172.21.15.1:0/3437155902 shutdown_connections 2024-02-02T15:35:17.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.501+0000 7f1f521e4b80 1 -- 172.21.15.1:0/3437155902 wait complete. 2024-02-02T15:35:17.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.501+0000 7f1f521e4b80 20 client.27908 trim_cache size 0 max 16384 2024-02-02T15:35:17.506 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:35:17.506 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-02T15:35:17.506 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2024-02-02T15:35:17.506 INFO:teuthology.orchestra.run.smithi001.stdout: "1bf855c0-56d6-427b-ae57-04b778bf6c55": { 2024-02-02T15:35:17.506 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2024-02-02T15:35:17.506 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "1bf855c0-56d6-427b-ae57-04b778bf6c55", 2024-02-02T15:35:17.506 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,force" 2024-02-02T15:35:17.506 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:35:17.506 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:35:17.506 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:35:17.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.502+0000 7f1f3e600700 1 -- 172.21.15.1:0/739475173 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f1f180a0710 msgr2=0x7f1f180a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:17.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.502+0000 7f1f3e600700 1 --2- 172.21.15.1:0/739475173 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f1f180a0710 0x7f1f180a2bd0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1f280104e0 tx=0x7f1f28007480 comp rx=0 tx=0).stop 2024-02-02T15:35:17.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.502+0000 7f1f3e600700 1 -- 172.21.15.1:0/739475173 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f3819daa0 msgr2=0x7f1f3819df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:17.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.502+0000 7f1f3e600700 1 --2- 172.21.15.1:0/739475173 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f3819daa0 0x7f1f3819df20 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f1f30009cd0 tx=0x7f1f3000c620 comp rx=0 tx=0).stop 2024-02-02T15:35:17.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.502+0000 7f1f3e600700 1 -- 172.21.15.1:0/739475173 shutdown_connections 2024-02-02T15:35:17.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.502+0000 7f1f3e600700 1 --2- 172.21.15.1:0/739475173 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f1f180a0710 0x7f1f180a2bd0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:17.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.502+0000 7f1f3e600700 1 --2- 172.21.15.1:0/739475173 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f3819daa0 0x7f1f3819df20 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:17.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.502+0000 7f1f3e600700 1 --2- 172.21.15.1:0/739475173 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f38101000 0x7f1f38199100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:17.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.502+0000 7f1f3e600700 1 --2- 172.21.15.1:0/739475173 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f38106120 0x7f1f38199640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:17.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.502+0000 7f1f3e600700 1 -- 172.21.15.1:0/739475173 >> 172.21.15.1:0/739475173 conn(0x7f1f380fc860 msgr2=0x7f1f380fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:17.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.502+0000 7f1f3e600700 1 -- 172.21.15.1:0/739475173 shutdown_connections 2024-02-02T15:35:17.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:17.502+0000 7f1f3e600700 1 -- 172.21.15.1:0/739475173 wait complete. 2024-02-02T15:35:17.521 INFO:tasks.cephfs.filesystem:scrub status for tag:1bf855c0-56d6-427b-ae57-04b778bf6c55 - {'path': '/', 'tag': '1bf855c0-56d6-427b-ae57-04b778bf6c55', 'options': 'recursive,force'} 2024-02-02T15:35:17.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:17 smithi142 ceph-mon[45728]: pgmap v255: 129 pgs: 129 active+clean; 343 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.2 MiB/s wr, 166 op/s 2024-02-02T15:35:17.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:17 smithi001 ceph-mon[35470]: pgmap v255: 129 pgs: 129 active+clean; 343 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.2 MiB/s wr, 166 op/s 2024-02-02T15:35:17.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:17 smithi071 ceph-mon[44707]: pgmap v255: 129 pgs: 129 active+clean; 343 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.2 MiB/s wr, 166 op/s 2024-02-02T15:35:18.250 INFO:tasks.workunit.client.0.smithi001.stdout: 1 483821 6.40 MB/sec execute 292 sec latency 13.276 ms 2024-02-02T15:35:18.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:18 smithi001 ceph-mon[35470]: scrub queued for path: / 2024-02-02T15:35:18.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:18 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:35:18.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:18 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:35:18.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:18 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:35:18.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:18 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:35:18.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:18 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:35:18.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:18 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:35:18.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:18 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:35:18.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:18 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:35:18.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:18 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:35:18.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:18 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:35:18.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:18 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:35:18.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:18 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:35:18.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:18 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:35:18.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:18 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:35:18.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:18 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:35:18.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:18 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:35:18.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:18 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:35:18.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:18 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:35:18.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:18 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:35:18.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:18 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:35:18.692 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:18 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:35:18.692 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:18 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:35:18.692 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:18 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:35:18.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:18 smithi071 ceph-mon[44707]: scrub queued for path: / 2024-02-02T15:35:18.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:18 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:35:18.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:18 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:35:18.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:18 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:35:18.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:18 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:35:18.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:18 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:35:18.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:18 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:35:18.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:18 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:35:18.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:18 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:35:18.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:18 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:35:18.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:18 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:35:18.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:18 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:35:18.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:18 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:35:18.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:18 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:35:18.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:18 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:35:18.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:18 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:35:18.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:18 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:35:18.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:18 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:35:18.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:18 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:35:18.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:18 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:35:18.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:18 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:35:18.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:18 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:35:18.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:18 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:35:18.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:18 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:35:18.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:18 smithi142 ceph-mon[45728]: scrub queued for path: / 2024-02-02T15:35:18.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:18 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:35:18.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:18 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:35:18.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:18 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:35:18.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:18 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:35:18.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:18 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:35:18.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:18 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:35:18.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:18 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:35:18.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:18 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:35:18.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:18 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:35:18.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:18 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:35:18.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:18 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:35:18.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:18 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:35:18.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:18 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:35:18.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:18 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:35:18.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:18 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:35:18.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:18 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:35:18.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:18 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:35:18.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:18 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:35:18.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:18 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:35:18.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:18 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:35:18.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:18 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:35:18.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:18 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:35:18.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:18 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:35:19.250 INFO:tasks.workunit.client.0.smithi001.stdout: 1 484995 6.40 MB/sec execute 293 sec latency 19.816 ms 2024-02-02T15:35:19.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:19 smithi001 ceph-mon[35470]: pgmap v256: 129 pgs: 129 active+clean; 343 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.2 MiB/s wr, 166 op/s 2024-02-02T15:35:19.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:19 smithi071 ceph-mon[44707]: pgmap v256: 129 pgs: 129 active+clean; 343 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.2 MiB/s wr, 166 op/s 2024-02-02T15:35:19.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:19 smithi142 ceph-mon[45728]: pgmap v256: 129 pgs: 129 active+clean; 343 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.2 MiB/s wr, 166 op/s 2024-02-02T15:35:20.250 INFO:tasks.workunit.client.0.smithi001.stdout: 1 486906 6.42 MB/sec execute 294 sec latency 18.344 ms 2024-02-02T15:35:21.250 INFO:tasks.workunit.client.0.smithi001.stdout: 1 487930 6.41 MB/sec execute 295 sec latency 19.208 ms 2024-02-02T15:35:21.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:21 smithi001 ceph-mon[35470]: pgmap v257: 129 pgs: 129 active+clean; 343 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.2 MiB/s wr, 166 op/s 2024-02-02T15:35:21.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:21 smithi071 ceph-mon[44707]: pgmap v257: 129 pgs: 129 active+clean; 343 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.2 MiB/s wr, 166 op/s 2024-02-02T15:35:21.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:21 smithi142 ceph-mon[45728]: pgmap v257: 129 pgs: 129 active+clean; 343 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.2 MiB/s wr, 166 op/s 2024-02-02T15:35:22.250 INFO:tasks.workunit.client.0.smithi001.stdout: 1 488939 6.40 MB/sec execute 296 sec latency 18.781 ms 2024-02-02T15:35:23.250 INFO:tasks.workunit.client.0.smithi001.stdout: 1 490628 6.42 MB/sec execute 297 sec latency 13.823 ms 2024-02-02T15:35:23.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:23 smithi001 ceph-mon[35470]: pgmap v258: 129 pgs: 129 active+clean; 350 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 1.5 MiB/s wr, 198 op/s 2024-02-02T15:35:23.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:23 smithi071 ceph-mon[44707]: pgmap v258: 129 pgs: 129 active+clean; 350 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 1.5 MiB/s wr, 198 op/s 2024-02-02T15:35:23.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:23 smithi142 ceph-mon[45728]: pgmap v258: 129 pgs: 129 active+clean; 350 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 1.5 MiB/s wr, 198 op/s 2024-02-02T15:35:24.250 INFO:tasks.workunit.client.0.smithi001.stdout: 1 491761 6.42 MB/sec execute 298 sec latency 18.221 ms 2024-02-02T15:35:25.250 INFO:tasks.workunit.client.0.smithi001.stdout: 1 493495 6.42 MB/sec execute 299 sec latency 19.358 ms 2024-02-02T15:35:25.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:25 smithi001 ceph-mon[35470]: pgmap v259: 129 pgs: 129 active+clean; 350 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 1.4 MiB/s wr, 180 op/s 2024-02-02T15:35:25.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:25 smithi071 ceph-mon[44707]: pgmap v259: 129 pgs: 129 active+clean; 350 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 1.4 MiB/s wr, 180 op/s 2024-02-02T15:35:25.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:25 smithi142 ceph-mon[45728]: pgmap v259: 129 pgs: 129 active+clean; 350 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 1.4 MiB/s wr, 180 op/s 2024-02-02T15:35:26.251 INFO:tasks.workunit.client.0.smithi001.stdout: 1 494573 6.42 MB/sec execute 300 sec latency 14.533 ms 2024-02-02T15:35:27.251 INFO:tasks.workunit.client.0.smithi001.stdout: 1 496253 6.42 MB/sec execute 301 sec latency 22.508 ms 2024-02-02T15:35:27.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:27 smithi001 ceph-mon[35470]: pgmap v260: 129 pgs: 129 active+clean; 357 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 2.0 MiB/s wr, 257 op/s 2024-02-02T15:35:27.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:27 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:35:27.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:27 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:35:27.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:27 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:35:27.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:27 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:35:27.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:27 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:35:27.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:27 smithi071 ceph-mon[44707]: pgmap v260: 129 pgs: 129 active+clean; 357 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 2.0 MiB/s wr, 257 op/s 2024-02-02T15:35:27.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:27 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:35:27.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:27 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:35:27.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:27 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:35:27.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:27 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:35:27.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:27 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:35:27.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:27 smithi142 ceph-mon[45728]: pgmap v260: 129 pgs: 129 active+clean; 357 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 2.0 MiB/s wr, 257 op/s 2024-02-02T15:35:27.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:27 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:35:27.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:27 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:35:27.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:27 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:35:27.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:27 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:35:27.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:27 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:35:28.251 INFO:tasks.workunit.client.0.smithi001.stdout: 1 497213 6.43 MB/sec execute 302 sec latency 515.771 ms 2024-02-02T15:35:29.251 INFO:tasks.workunit.client.0.smithi001.stdout: 1 497213 6.40 MB/sec execute 303 sec latency 1515.887 ms 2024-02-02T15:35:29.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:29 smithi001 ceph-mon[35470]: pgmap v261: 129 pgs: 129 active+clean; 357 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 1.3 MiB/s wr, 175 op/s 2024-02-02T15:35:29.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:29 smithi071 ceph-mon[44707]: pgmap v261: 129 pgs: 129 active+clean; 357 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 1.3 MiB/s wr, 175 op/s 2024-02-02T15:35:29.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:29 smithi142 ceph-mon[45728]: pgmap v261: 129 pgs: 129 active+clean; 357 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 1.3 MiB/s wr, 175 op/s 2024-02-02T15:35:30.251 INFO:tasks.workunit.client.0.smithi001.stdout: 1 497226 6.38 MB/sec execute 304 sec latency 1807.704 ms 2024-02-02T15:35:31.251 INFO:tasks.workunit.client.0.smithi001.stdout: 1 497226 6.36 MB/sec execute 305 sec latency 1689.294 ms 2024-02-02T15:35:31.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:31 smithi071 ceph-mon[44707]: pgmap v262: 129 pgs: 129 active+clean; 357 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 1.3 MiB/s wr, 175 op/s 2024-02-02T15:35:31.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:31 smithi142 ceph-mon[45728]: pgmap v262: 129 pgs: 129 active+clean; 357 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 1.3 MiB/s wr, 175 op/s 2024-02-02T15:35:31.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:31 smithi001 ceph-mon[35470]: pgmap v262: 129 pgs: 129 active+clean; 357 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 1.3 MiB/s wr, 175 op/s 2024-02-02T15:35:32.251 INFO:tasks.workunit.client.0.smithi001.stdout: 1 497957 6.36 MB/sec execute 306 sec latency 1907.007 ms 2024-02-02T15:35:33.251 INFO:tasks.workunit.client.0.smithi001.stdout: 1 499543 6.36 MB/sec execute 307 sec latency 18.575 ms 2024-02-02T15:35:33.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:33 smithi071 ceph-mon[44707]: pgmap v263: 129 pgs: 129 active+clean; 360 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 1.6 MiB/s wr, 213 op/s 2024-02-02T15:35:33.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:33 smithi142 ceph-mon[45728]: pgmap v263: 129 pgs: 129 active+clean; 360 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 1.6 MiB/s wr, 213 op/s 2024-02-02T15:35:33.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:33 smithi001 ceph-mon[35470]: pgmap v263: 129 pgs: 129 active+clean; 360 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 1.6 MiB/s wr, 213 op/s 2024-02-02T15:35:34.251 INFO:tasks.workunit.client.0.smithi001.stdout: 1 501246 6.38 MB/sec execute 308 sec latency 14.463 ms 2024-02-02T15:35:35.251 INFO:tasks.workunit.client.0.smithi001.stdout: 1 502394 6.37 MB/sec execute 309 sec latency 21.177 ms 2024-02-02T15:35:35.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:35 smithi071 ceph-mon[44707]: pgmap v264: 129 pgs: 129 active+clean; 360 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 934 KiB/s wr, 115 op/s 2024-02-02T15:35:35.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:35 smithi142 ceph-mon[45728]: pgmap v264: 129 pgs: 129 active+clean; 360 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 934 KiB/s wr, 115 op/s 2024-02-02T15:35:35.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:35 smithi001 ceph-mon[35470]: pgmap v264: 129 pgs: 129 active+clean; 360 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 934 KiB/s wr, 115 op/s 2024-02-02T15:35:36.252 INFO:tasks.workunit.client.0.smithi001.stdout: 1 504319 6.38 MB/sec execute 310 sec latency 21.799 ms 2024-02-02T15:35:37.251 INFO:tasks.workunit.client.0.smithi001.stdout: 1 505481 6.38 MB/sec execute 311 sec latency 15.536 ms 2024-02-02T15:35:37.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:37 smithi071 ceph-mon[44707]: pgmap v265: 129 pgs: 129 active+clean; 367 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.6 MiB/s wr, 199 op/s 2024-02-02T15:35:37.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:37 smithi142 ceph-mon[45728]: pgmap v265: 129 pgs: 129 active+clean; 367 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.6 MiB/s wr, 199 op/s 2024-02-02T15:35:37.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:37 smithi001 ceph-mon[35470]: pgmap v265: 129 pgs: 129 active+clean; 367 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.6 MiB/s wr, 199 op/s 2024-02-02T15:35:38.252 INFO:tasks.workunit.client.0.smithi001.stdout: 1 506945 6.38 MB/sec execute 312 sec latency 19.001 ms 2024-02-02T15:35:39.252 INFO:tasks.workunit.client.0.smithi001.stdout: 1 508470 6.40 MB/sec execute 313 sec latency 10.204 ms 2024-02-02T15:35:39.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:39 smithi071 ceph-mon[44707]: pgmap v266: 129 pgs: 129 active+clean; 367 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1011 KiB/s wr, 121 op/s 2024-02-02T15:35:39.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:39 smithi142 ceph-mon[45728]: pgmap v266: 129 pgs: 129 active+clean; 367 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1011 KiB/s wr, 121 op/s 2024-02-02T15:35:39.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:39 smithi001 ceph-mon[35470]: pgmap v266: 129 pgs: 129 active+clean; 367 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1011 KiB/s wr, 121 op/s 2024-02-02T15:35:40.252 INFO:tasks.workunit.client.0.smithi001.stdout: 1 509566 6.39 MB/sec execute 314 sec latency 18.589 ms 2024-02-02T15:35:41.252 INFO:tasks.workunit.client.0.smithi001.stdout: 1 510948 6.40 MB/sec execute 315 sec latency 17.653 ms 2024-02-02T15:35:41.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:41 smithi071 ceph-mon[44707]: pgmap v267: 129 pgs: 129 active+clean; 367 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1011 KiB/s wr, 121 op/s 2024-02-02T15:35:41.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:41 smithi142 ceph-mon[45728]: pgmap v267: 129 pgs: 129 active+clean; 367 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1011 KiB/s wr, 121 op/s 2024-02-02T15:35:41.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:41 smithi001 ceph-mon[35470]: pgmap v267: 129 pgs: 129 active+clean; 367 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1011 KiB/s wr, 121 op/s 2024-02-02T15:35:42.252 INFO:tasks.workunit.client.0.smithi001.stdout: 1 512063 6.40 MB/sec execute 316 sec latency 29.479 ms 2024-02-02T15:35:43.252 INFO:tasks.workunit.client.0.smithi001.stdout: 1 513170 6.39 MB/sec execute 317 sec latency 19.680 ms 2024-02-02T15:35:43.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:43 smithi071 ceph-mon[44707]: pgmap v268: 129 pgs: 129 active+clean; 376 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.9 MiB/s wr, 201 op/s 2024-02-02T15:35:43.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:43 smithi142 ceph-mon[45728]: pgmap v268: 129 pgs: 129 active+clean; 376 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.9 MiB/s wr, 201 op/s 2024-02-02T15:35:43.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:43 smithi001 ceph-mon[35470]: pgmap v268: 129 pgs: 129 active+clean; 376 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.9 MiB/s wr, 201 op/s 2024-02-02T15:35:44.252 INFO:tasks.workunit.client.0.smithi001.stdout: 1 515188 6.41 MB/sec execute 318 sec latency 18.492 ms 2024-02-02T15:35:45.252 INFO:tasks.workunit.client.0.smithi001.stdout: 1 515992 6.40 MB/sec execute 319 sec latency 52.309 ms 2024-02-02T15:35:45.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:45 smithi071 ceph-mon[44707]: pgmap v269: 129 pgs: 129 active+clean; 376 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.6 MiB/s wr, 163 op/s 2024-02-02T15:35:45.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:45 smithi142 ceph-mon[45728]: pgmap v269: 129 pgs: 129 active+clean; 376 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.6 MiB/s wr, 163 op/s 2024-02-02T15:35:45.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:45 smithi001 ceph-mon[35470]: pgmap v269: 129 pgs: 129 active+clean; 376 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.6 MiB/s wr, 163 op/s 2024-02-02T15:35:46.252 INFO:tasks.workunit.client.0.smithi001.stdout: 1 516969 6.39 MB/sec execute 320 sec latency 19.732 ms 2024-02-02T15:35:46.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:46 smithi071 ceph-mon[44707]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:35:46.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:46 smithi142 ceph-mon[45728]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:35:46.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:46 smithi001 ceph-mon[35470]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:35:47.252 INFO:tasks.workunit.client.0.smithi001.stdout: 1 518780 6.41 MB/sec execute 321 sec latency 37.209 ms 2024-02-02T15:35:47.522 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-02T15:35:47.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.795+0000 7fac58c00700 1 -- 172.21.15.1:0/1363991018 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac54106110 msgr2=0x7fac5410dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:47.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.795+0000 7fac58c00700 1 --2- 172.21.15.1:0/1363991018 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac54106110 0x7fac5410dc90 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fac4c00b0a0 tx=0x7fac4c00b3b0 comp rx=0 tx=0).stop 2024-02-02T15:35:47.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.795+0000 7fac58c00700 1 -- 172.21.15.1:0/1363991018 shutdown_connections 2024-02-02T15:35:47.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.795+0000 7fac58c00700 1 --2- 172.21.15.1:0/1363991018 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac54106110 0x7fac5410dc90 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:47.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.795+0000 7fac58c00700 1 --2- 172.21.15.1:0/1363991018 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fac54100ff0 0x7fac541013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:47.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.795+0000 7fac58c00700 1 --2- 172.21.15.1:0/1363991018 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fac541019a0 0x7fac541059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:47.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.795+0000 7fac58c00700 1 -- 172.21.15.1:0/1363991018 >> 172.21.15.1:0/1363991018 conn(0x7fac540fc810 msgr2=0x7fac540fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:47.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.795+0000 7fac58c00700 1 -- 172.21.15.1:0/1363991018 shutdown_connections 2024-02-02T15:35:47.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.795+0000 7fac58c00700 1 -- 172.21.15.1:0/1363991018 wait complete. 2024-02-02T15:35:47.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.796+0000 7fac58c00700 1 Processor -- start 2024-02-02T15:35:47.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.797+0000 7fac58c00700 1 -- start start 2024-02-02T15:35:47.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.797+0000 7fac58c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac54100ff0 0x7fac54196db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:47.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.797+0000 7fac58c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fac54106110 0x7fac541972f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:47.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.797+0000 7fac58c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fac5419b750 0x7fac5419bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:47.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.797+0000 7fac58c00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac5419c1d0 con 0x7fac5419b750 2024-02-02T15:35:47.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.797+0000 7fac58c00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac5419c340 con 0x7fac54100ff0 2024-02-02T15:35:47.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.797+0000 7fac58c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac5419c500 con 0x7fac54106110 2024-02-02T15:35:47.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.797+0000 7fac51400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fac54106110 0x7fac541972f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:47.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.797+0000 7fac51400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fac54106110 0x7fac541972f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:44002/0 (socket says 172.21.15.1:44002) 2024-02-02T15:35:47.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.797+0000 7fac51400700 1 -- 172.21.15.1:0/152402931 learned_addr learned my addr 172.21.15.1:0/152402931 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:35:47.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.797+0000 7fac51e00700 1 --2- 172.21.15.1:0/152402931 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac54100ff0 0x7fac54196db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:47.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.797+0000 7fac52800700 1 --2- 172.21.15.1:0/152402931 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fac5419b750 0x7fac5419bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:47.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.798+0000 7fac51400700 1 -- 172.21.15.1:0/152402931 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac54100ff0 msgr2=0x7fac54196db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:47.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.798+0000 7fac51400700 1 --2- 172.21.15.1:0/152402931 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac54100ff0 0x7fac54196db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:47.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.798+0000 7fac51400700 1 -- 172.21.15.1:0/152402931 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fac5419b750 msgr2=0x7fac5419bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:47.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.798+0000 7fac51400700 1 --2- 172.21.15.1:0/152402931 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fac5419b750 0x7fac5419bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:47.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.798+0000 7fac51400700 1 -- 172.21.15.1:0/152402931 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac4c009d00 con 0x7fac54106110 2024-02-02T15:35:47.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.798+0000 7fac51e00700 1 --2- 172.21.15.1:0/152402931 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac54100ff0 0x7fac54196db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:35:47.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.798+0000 7fac51400700 1 --2- 172.21.15.1:0/152402931 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fac54106110 0x7fac541972f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fac3c00e970 tx=0x7fac3c00ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:47.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.798+0000 7fac4aa00700 1 -- 172.21.15.1:0/152402931 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac3c009790 con 0x7fac54106110 2024-02-02T15:35:47.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.798+0000 7fac4aa00700 1 -- 172.21.15.1:0/152402931 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fac3c00c7e0 con 0x7fac54106110 2024-02-02T15:35:47.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.798+0000 7fac4aa00700 1 -- 172.21.15.1:0/152402931 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac3c0054e0 con 0x7fac54106110 2024-02-02T15:35:47.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.799+0000 7fac58c00700 1 -- 172.21.15.1:0/152402931 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac541a3a60 con 0x7fac54106110 2024-02-02T15:35:47.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.799+0000 7fac58c00700 1 -- 172.21.15.1:0/152402931 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac541a3ef0 con 0x7fac54106110 2024-02-02T15:35:47.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.802+0000 7fac4aa00700 1 -- 172.21.15.1:0/152402931 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7fac3c00c950 con 0x7fac54106110 2024-02-02T15:35:47.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.804+0000 7fac6c64db80 1 Processor -- start 2024-02-02T15:35:47.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.804+0000 7fac4aa00700 1 --2- 172.21.15.1:0/152402931 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fac340a0690 0x7fac340a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:47.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.804+0000 7fac6c64db80 1 -- start start 2024-02-02T15:35:47.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.805+0000 7fac4aa00700 1 -- 172.21.15.1:0/152402931 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fac3c014070 con 0x7fac54106110 2024-02-02T15:35:47.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.805+0000 7fac6c64db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56464cba2010 0x56464cba23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:47.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.805+0000 7fac6c64db80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56464cba2930 0x56464cbad2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:47.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.805+0000 7fac6c64db80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56464cbad7e0 0x56464cbafbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:47.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.805+0000 7fac6c64db80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56464cb164c0 con 0x56464cbad7e0 2024-02-02T15:35:47.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.805+0000 7fac6c64db80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56464cafaaa0 con 0x56464cba2930 2024-02-02T15:35:47.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.805+0000 7fac6c64db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56464caf8460 con 0x56464cba2010 2024-02-02T15:35:47.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.805+0000 7fac51400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56464cba2930 0x56464cbad2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:47.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.805+0000 7fac51400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56464cba2930 0x56464cbad2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:41850/0 (socket says 172.21.15.1:41850) 2024-02-02T15:35:47.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.805+0000 7fac51400700 1 -- 172.21.15.1:0/1320540163 learned_addr learned my addr 172.21.15.1:0/1320540163 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:35:47.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.805+0000 7fac51e00700 1 --2- 172.21.15.1:0/1320540163 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56464cba2010 0x56464cba23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:47.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.806+0000 7fac51400700 1 -- 172.21.15.1:0/1320540163 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56464cba2010 msgr2=0x56464cba23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:47.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.806+0000 7fac52800700 1 --2- 172.21.15.1:0/152402931 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fac340a0690 0x7fac340a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:47.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.806+0000 7fac51400700 1 --2- 172.21.15.1:0/1320540163 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56464cba2010 0x56464cba23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:47.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.806+0000 7fac51400700 1 -- 172.21.15.1:0/1320540163 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56464cbad7e0 msgr2=0x56464cbafbd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-02T15:35:47.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.806+0000 7fac51400700 1 --2- 172.21.15.1:0/1320540163 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56464cbad7e0 0x56464cbafbd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:47.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.806+0000 7fac51400700 1 -- 172.21.15.1:0/1320540163 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56464c9753b0 con 0x56464cba2930 2024-02-02T15:35:47.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.806+0000 7fac51400700 1 --2- 172.21.15.1:0/1320540163 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56464cba2930 0x56464cbad2a0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fac3c00cc00 tx=0x7fac3c098e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=136e8158aa2202aa server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:47.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.806+0000 7fac43000700 1 -- 172.21.15.1:0/1320540163 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac3c099430 con 0x56464cba2930 2024-02-02T15:35:47.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.806+0000 7fac43000700 1 -- 172.21.15.1:0/1320540163 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fac3c09d260 con 0x56464cba2930 2024-02-02T15:35:47.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.806+0000 7fac43000700 1 -- 172.21.15.1:0/1320540163 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac3c095b40 con 0x56464cba2930 2024-02-02T15:35:47.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.807+0000 7fac6c64db80 1 -- 172.21.15.1:0/1320540163 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56464cba2930 msgr2=0x56464cbad2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:47.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.807+0000 7fac6c64db80 1 --2- 172.21.15.1:0/1320540163 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56464cba2930 0x56464cbad2a0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fac3c00cc00 tx=0x7fac3c098e10 comp rx=0 tx=0).stop 2024-02-02T15:35:47.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.807+0000 7fac52800700 1 --2- 172.21.15.1:0/152402931 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fac340a0690 0x7fac340a2b50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fac4c009610 tx=0x7fac4c006230 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:47.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.807+0000 7fac6c64db80 1 -- 172.21.15.1:0/1320540163 shutdown_connections 2024-02-02T15:35:47.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.807+0000 7fac6c64db80 1 --2- 172.21.15.1:0/1320540163 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56464cbad7e0 0x56464cbafbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:47.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.807+0000 7fac6c64db80 1 --2- 172.21.15.1:0/1320540163 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56464cba2010 0x56464cba23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:47.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.807+0000 7fac6c64db80 1 --2- 172.21.15.1:0/1320540163 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56464cba2930 0x56464cbad2a0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:47.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.807+0000 7fac6c64db80 1 -- 172.21.15.1:0/1320540163 >> 172.21.15.1:0/1320540163 conn(0x56464cb01db0 msgr2=0x56464cb9e3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:47.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.807+0000 7fac6c64db80 1 -- 172.21.15.1:0/1320540163 shutdown_connections 2024-02-02T15:35:47.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.807+0000 7fac6c64db80 1 -- 172.21.15.1:0/1320540163 wait complete. 2024-02-02T15:35:47.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.808+0000 7fac6c64db80 1 -- start start 2024-02-02T15:35:47.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.808+0000 7fac6c64db80 1 Processor -- start 2024-02-02T15:35:47.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.808+0000 7fac6c64db80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56464cba2010 0x56464cbad210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:47.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.808+0000 7fac6c64db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56464cba2930 0x56464cbaa500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:47.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.808+0000 7fac6c64db80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56464cbad7e0 0x56464cbaac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:47.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.808+0000 7fac6c64db80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56464caf3d50 con 0x56464cbad7e0 2024-02-02T15:35:47.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.808+0000 7fac6c64db80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56464ca8fd80 con 0x56464cba2010 2024-02-02T15:35:47.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.808+0000 7fac6c64db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56464ca8fa20 con 0x56464cba2930 2024-02-02T15:35:47.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.808+0000 7fac6c64db80 10 client.0 _ll_register_callbacks cb 0x56464ca91290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:35:47.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.808+0000 7fac51e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56464cba2010 0x56464cbad210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:47.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.809+0000 7fac51e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56464cba2010 0x56464cbad210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:41860/0 (socket says 172.21.15.1:41860) 2024-02-02T15:35:47.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.809+0000 7fac51e00700 1 -- 172.21.15.1:0/1624833779 learned_addr learned my addr 172.21.15.1:0/1624833779 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:35:47.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.809+0000 7fac51400700 1 --2- 172.21.15.1:0/1624833779 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56464cba2930 0x56464cbaa500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:47.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.809+0000 7fac52800700 1 --2- 172.21.15.1:0/1624833779 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56464cbad7e0 0x56464cbaac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:47.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.809+0000 7fac51e00700 1 -- 172.21.15.1:0/1624833779 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56464cba2930 msgr2=0x56464cbaa500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:47.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.809+0000 7fac51e00700 1 --2- 172.21.15.1:0/1624833779 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56464cba2930 0x56464cbaa500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:47.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.809+0000 7fac51e00700 1 -- 172.21.15.1:0/1624833779 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56464cbad7e0 msgr2=0x56464cbaac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:47.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.809+0000 7fac51e00700 1 --2- 172.21.15.1:0/1624833779 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56464cbad7e0 0x56464cbaac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:47.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.809+0000 7fac51e00700 1 -- 172.21.15.1:0/1624833779 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac541a3ef0 con 0x56464cba2010 2024-02-02T15:35:47.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.809+0000 7fac51e00700 1 --2- 172.21.15.1:0/1624833779 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56464cba2010 0x56464cbad210 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fac44012d20 tx=0x7fac4400f6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9196c0043a0e8d3 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:47.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.809+0000 7fac3b400700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:35:47.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.809+0000 7fac3b400700 1 -- 172.21.15.1:0/1624833779 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac44017070 con 0x56464cba2010 2024-02-02T15:35:47.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.810+0000 7fac3b400700 1 -- 172.21.15.1:0/1624833779 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fac4400fc30 con 0x56464cba2010 2024-02-02T15:35:47.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.810+0000 7fac6c64db80 1 -- 172.21.15.1:0/1624833779 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56464cb960f0 con 0x56464cba2010 2024-02-02T15:35:47.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.810+0000 7fac3b400700 1 -- 172.21.15.1:0/1624833779 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac440122e0 con 0x56464cba2010 2024-02-02T15:35:47.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.810+0000 7fac6c64db80 10 client.27920 fetch_fsmap learned FSMap version 43 2024-02-02T15:35:47.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.810+0000 7fac6c64db80 1 -- 172.21.15.1:0/1624833779 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x56464c921b60 con 0x56464cba2010 2024-02-02T15:35:47.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.810+0000 7fac3b400700 1 -- 172.21.15.1:0/1624833779 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56464cb960f0 con 0x56464cba2010 2024-02-02T15:35:47.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.810+0000 7fac3b400700 1 -- 172.21.15.1:0/1624833779 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 43) v1 ==== 7506+0+0 (secure 0 0 0) 0x7fac4401c420 con 0x56464cba2010 2024-02-02T15:35:47.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.810+0000 7fac6c64db80 10 client.27920 fetch_fsmap finished waiting for FSMap version 43 2024-02-02T15:35:47.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.810+0000 7fac6c64db80 10 client.27920 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:35:47.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.810+0000 7fac6c64db80 20 client.27920 populate_metadata read hostname 'smithi001' 2024-02-02T15:35:47.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.810+0000 7fac6c64db80 1 --2- 172.21.15.1:0/1624833779 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56464cbbd350 0x56464cbbf770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:47.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.810+0000 7fac6c64db80 4 client.27920 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:35:47.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.810+0000 7fac6c64db80 1 -- 172.21.15.1:0/1624833779 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56464caecdd0 con 0x56464cbbd350 2024-02-02T15:35:47.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.811+0000 7fac51400700 1 --2- 172.21.15.1:0/1624833779 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56464cbbd350 0x56464cbbf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:47.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.811+0000 7fac3b400700 10 client.27920 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:35:47.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.811+0000 7fac51400700 1 --2- 172.21.15.1:0/1624833779 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56464cbbd350 0x56464cbbf770 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50d20e3559fed577 server_cookie=9c3aa777aca93c7c in_seq=0 out_seq=0 2024-02-02T15:35:47.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.812+0000 7fac3b400700 1 -- 172.21.15.1:0/1624833779 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x56464caecdd0 con 0x56464cbbd350 2024-02-02T15:35:47.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.812+0000 7fac3b400700 10 client.27920 handle_command_reply: tid=0 2024-02-02T15:35:47.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.812+0000 7fac6c64db80 1 client.27920 shutdown 2024-02-02T15:35:47.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.812+0000 7fac6c64db80 1 -- 172.21.15.1:0/1624833779 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56464cba2010 msgr2=0x56464cbad210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:47.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.812+0000 7fac6c64db80 1 --2- 172.21.15.1:0/1624833779 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56464cba2010 0x56464cbad210 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fac44012d20 tx=0x7fac4400f6a0 comp rx=0 tx=0).stop 2024-02-02T15:35:47.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.812+0000 7fac6c64db80 1 -- 172.21.15.1:0/1624833779 shutdown_connections 2024-02-02T15:35:47.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.812+0000 7fac6c64db80 1 --2- 172.21.15.1:0/1624833779 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56464cbbd350 0x56464cbbf770 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:47.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.812+0000 7fac6c64db80 1 --2- 172.21.15.1:0/1624833779 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56464cbad7e0 0x56464cbaac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:47.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.812+0000 7fac3b400700 0 client.27920 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:35:47.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.812+0000 7fac6c64db80 1 --2- 172.21.15.1:0/1624833779 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56464cba2010 0x56464cbad210 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:47.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.812+0000 7fac6c64db80 1 --2- 172.21.15.1:0/1624833779 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56464cba2930 0x56464cbaa500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:47.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.812+0000 7fac6c64db80 1 -- 172.21.15.1:0/1624833779 >> 172.21.15.1:0/1624833779 conn(0x56464cb01db0 msgr2=0x56464cb9df40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:47.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.812+0000 7fac6c64db80 1 -- 172.21.15.1:0/1624833779 shutdown_connections 2024-02-02T15:35:47.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.812+0000 7fac6c64db80 1 -- 172.21.15.1:0/1624833779 wait complete. 2024-02-02T15:35:47.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.812+0000 7fac6c64db80 20 client.27920 trim_cache size 0 max 16384 2024-02-02T15:35:47.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.832+0000 7fac6c64db80 1 Processor -- start 2024-02-02T15:35:47.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.832+0000 7fac6c64db80 1 -- start start 2024-02-02T15:35:47.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.832+0000 7fac6c64db80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56464cba2010 0x56464cbacdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:47.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.832+0000 7fac6c64db80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56464cba2930 0x56464cba9a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:47.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.832+0000 7fac6c64db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56464cbad7e0 0x56464cba7770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:47.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.832+0000 7fac6c64db80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56464c9776a0 con 0x56464cba2930 2024-02-02T15:35:47.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.832+0000 7fac6c64db80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56464cb98500 con 0x56464cba2010 2024-02-02T15:35:47.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.832+0000 7fac6c64db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56464cbba910 con 0x56464cbad7e0 2024-02-02T15:35:47.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.833+0000 7fac51e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56464cba2010 0x56464cbacdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:47.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.833+0000 7fac51400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56464cba2930 0x56464cba9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:47.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.833+0000 7fac51e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56464cba2010 0x56464cbacdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:41876/0 (socket says 172.21.15.1:41876) 2024-02-02T15:35:47.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.833+0000 7fac51e00700 1 -- 172.21.15.1:0/781969069 learned_addr learned my addr 172.21.15.1:0/781969069 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:35:47.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.833+0000 7fac51e00700 1 -- 172.21.15.1:0/781969069 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56464cbad7e0 msgr2=0x56464cba7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:47.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.833+0000 7fac51e00700 1 --2- 172.21.15.1:0/781969069 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56464cbad7e0 0x56464cba7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:47.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.833+0000 7fac51e00700 1 -- 172.21.15.1:0/781969069 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56464cba2930 msgr2=0x56464cba9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:47.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.833+0000 7fac51e00700 1 --2- 172.21.15.1:0/781969069 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56464cba2930 0x56464cba9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:47.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.833+0000 7fac51e00700 1 -- 172.21.15.1:0/781969069 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56464cab9ea0 con 0x56464cba2010 2024-02-02T15:35:47.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.833+0000 7fac51e00700 1 --2- 172.21.15.1:0/781969069 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56464cba2010 0x56464cbacdb0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fac44000c00 tx=0x7fac4400faa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=138343304f539330 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:47.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.834+0000 7fac43a00700 1 -- 172.21.15.1:0/781969069 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac44017070 con 0x56464cba2010 2024-02-02T15:35:47.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.834+0000 7fac43a00700 1 -- 172.21.15.1:0/781969069 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fac44018040 con 0x56464cba2010 2024-02-02T15:35:47.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.834+0000 7fac43a00700 1 -- 172.21.15.1:0/781969069 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac440055d0 con 0x56464cba2010 2024-02-02T15:35:47.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.835+0000 7fac6c64db80 1 -- 172.21.15.1:0/781969069 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56464cba2010 msgr2=0x56464cbacdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:47.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.835+0000 7fac6c64db80 1 --2- 172.21.15.1:0/781969069 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56464cba2010 0x56464cbacdb0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fac44000c00 tx=0x7fac4400faa0 comp rx=0 tx=0).stop 2024-02-02T15:35:47.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.835+0000 7fac6c64db80 1 -- 172.21.15.1:0/781969069 shutdown_connections 2024-02-02T15:35:47.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.835+0000 7fac6c64db80 1 --2- 172.21.15.1:0/781969069 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56464cbad7e0 0x56464cba7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:47.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.835+0000 7fac6c64db80 1 --2- 172.21.15.1:0/781969069 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56464cba2010 0x56464cbacdb0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:47.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.835+0000 7fac6c64db80 1 --2- 172.21.15.1:0/781969069 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56464cba2930 0x56464cba9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:47.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.835+0000 7fac6c64db80 1 -- 172.21.15.1:0/781969069 >> 172.21.15.1:0/781969069 conn(0x56464cb01db0 msgr2=0x56464cbb9560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:47.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.835+0000 7fac6c64db80 1 -- 172.21.15.1:0/781969069 shutdown_connections 2024-02-02T15:35:47.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.835+0000 7fac6c64db80 1 -- 172.21.15.1:0/781969069 wait complete. 2024-02-02T15:35:47.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.835+0000 7fac6c64db80 1 -- start start 2024-02-02T15:35:47.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.835+0000 7fac6c64db80 1 Processor -- start 2024-02-02T15:35:47.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.836+0000 7fac6c64db80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56464cba2010 0x56464cbb3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:47.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.836+0000 7fac6c64db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56464cba2930 0x56464cbb43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:47.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.836+0000 7fac6c64db80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56464cbad7e0 0x56464cbb22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:47.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.836+0000 7fac6c64db80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56464cbb29b0 con 0x56464cbad7e0 2024-02-02T15:35:47.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.836+0000 7fac6c64db80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56464cbb2b20 con 0x56464cba2010 2024-02-02T15:35:47.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.836+0000 7fac6c64db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56464cbb2ce0 con 0x56464cba2930 2024-02-02T15:35:47.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.836+0000 7fac6c64db80 10 client.0 _ll_register_callbacks cb 0x56464ca91290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:35:47.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.836+0000 7fac51400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56464cba2930 0x56464cbb43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:47.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.836+0000 7fac52800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56464cbad7e0 0x56464cbb22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:47.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.836+0000 7fac51400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56464cba2930 0x56464cbb43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:44036/0 (socket says 172.21.15.1:44036) 2024-02-02T15:35:47.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.836+0000 7fac51400700 1 -- 172.21.15.1:0/1181692731 learned_addr learned my addr 172.21.15.1:0/1181692731 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:35:47.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.836+0000 7fac51e00700 1 --2- 172.21.15.1:0/1181692731 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56464cba2010 0x56464cbb3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:47.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.837+0000 7fac51400700 1 -- 172.21.15.1:0/1181692731 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56464cba2010 msgr2=0x56464cbb3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:47.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.837+0000 7fac51400700 1 --2- 172.21.15.1:0/1181692731 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56464cba2010 0x56464cbb3e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:47.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.837+0000 7fac51400700 1 -- 172.21.15.1:0/1181692731 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56464cbad7e0 msgr2=0x56464cbb22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:47.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.837+0000 7fac51400700 1 --2- 172.21.15.1:0/1181692731 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56464cbad7e0 0x56464cbb22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:47.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.837+0000 7fac51400700 1 -- 172.21.15.1:0/1181692731 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56464c921b60 con 0x56464cba2930 2024-02-02T15:35:47.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.837+0000 7fac51400700 1 --2- 172.21.15.1:0/1181692731 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56464cba2930 0x56464cbb43c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fac3c095930 tx=0x7fac3c00cf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d22b172bcd539eca server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:47.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.837+0000 7fac3b400700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:35:47.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.837+0000 7fac3b400700 1 -- 172.21.15.1:0/1181692731 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac3c0a1520 con 0x56464cba2930 2024-02-02T15:35:47.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.837+0000 7fac3b400700 1 -- 172.21.15.1:0/1181692731 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fac3c0b6040 con 0x56464cba2930 2024-02-02T15:35:47.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.837+0000 7fac6c64db80 1 -- 172.21.15.1:0/1181692731 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fac3c09d260 con 0x56464cba2930 2024-02-02T15:35:47.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.837+0000 7fac3b400700 1 -- 172.21.15.1:0/1181692731 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac3c065200 con 0x56464cba2930 2024-02-02T15:35:47.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.837+0000 7fac6c64db80 10 client.27919 fetch_fsmap learned FSMap version 43 2024-02-02T15:35:47.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.837+0000 7fac6c64db80 1 -- 172.21.15.1:0/1181692731 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x56464c979cd0 con 0x56464cba2930 2024-02-02T15:35:47.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.838+0000 7fac3b400700 1 -- 172.21.15.1:0/1181692731 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fac3c09d260 con 0x56464cba2930 2024-02-02T15:35:47.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.838+0000 7fac3b400700 1 -- 172.21.15.1:0/1181692731 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 43) v1 ==== 7506+0+0 (secure 0 0 0) 0x7fac3c099690 con 0x56464cba2930 2024-02-02T15:35:47.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.838+0000 7fac6c64db80 10 client.27919 fetch_fsmap finished waiting for FSMap version 43 2024-02-02T15:35:47.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.838+0000 7fac6c64db80 10 client.27919 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:35:47.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.838+0000 7fac6c64db80 20 client.27919 populate_metadata read hostname 'smithi001' 2024-02-02T15:35:47.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.838+0000 7fac6c64db80 1 --2- 172.21.15.1:0/1181692731 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56464cc0fe50 0x56464cc12270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:47.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.838+0000 7fac6c64db80 4 client.27919 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2024-02-02T15:35:47.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.838+0000 7fac6c64db80 1 -- 172.21.15.1:0/1181692731 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56464cab2f50 con 0x56464cc0fe50 2024-02-02T15:35:47.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.838+0000 7fac51e00700 1 --2- 172.21.15.1:0/1181692731 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56464cc0fe50 0x56464cc12270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:47.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.840+0000 7fac51e00700 1 --2- 172.21.15.1:0/1181692731 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56464cc0fe50 0x56464cc12270 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f92ffa74950bd4ea server_cookie=d5939e1946962b70 in_seq=0 out_seq=0 2024-02-02T15:35:47.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.840+0000 7fac3b400700 10 client.27919 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:35:47.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.840+0000 7fac3b400700 1 -- 172.21.15.1:0/1181692731 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56464cab2f50 con 0x56464cc0fe50 2024-02-02T15:35:47.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.840+0000 7fac3b400700 10 client.27919 handle_command_reply: tid=0 2024-02-02T15:35:47.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.840+0000 7fac6c64db80 1 client.27919 shutdown 2024-02-02T15:35:47.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.840+0000 7fac6c64db80 1 -- 172.21.15.1:0/1181692731 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56464cba2930 msgr2=0x56464cbb43c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:47.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.840+0000 7fac6c64db80 1 --2- 172.21.15.1:0/1181692731 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56464cba2930 0x56464cbb43c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fac3c095930 tx=0x7fac3c00cf50 comp rx=0 tx=0).stop 2024-02-02T15:35:47.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.840+0000 7fac6c64db80 1 -- 172.21.15.1:0/1181692731 shutdown_connections 2024-02-02T15:35:47.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.840+0000 7fac6c64db80 1 --2- 172.21.15.1:0/1181692731 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56464cc0fe50 0x56464cc12270 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:47.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.840+0000 7fac6c64db80 1 --2- 172.21.15.1:0/1181692731 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56464cbad7e0 0x56464cbb22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:47.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.840+0000 7fac6c64db80 1 --2- 172.21.15.1:0/1181692731 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56464cba2010 0x56464cbb3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:47.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.840+0000 7fac6c64db80 1 --2- 172.21.15.1:0/1181692731 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56464cba2930 0x56464cbb43c0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:47.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.840+0000 7fac6c64db80 1 -- 172.21.15.1:0/1181692731 >> 172.21.15.1:0/1181692731 conn(0x56464cb01db0 msgr2=0x56464cb9d5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:47.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.840+0000 7fac3b400700 0 client.27919 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:35:47.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.840+0000 7fac6c64db80 1 -- 172.21.15.1:0/1181692731 shutdown_connections 2024-02-02T15:35:47.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.840+0000 7fac6c64db80 1 -- 172.21.15.1:0/1181692731 wait complete. 2024-02-02T15:35:47.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.840+0000 7fac6c64db80 20 client.27919 trim_cache size 0 max 16384 2024-02-02T15:35:47.842 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:35:47.842 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2024-02-02T15:35:47.842 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2024-02-02T15:35:47.842 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:35:47.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.840+0000 7fac58c00700 1 -- 172.21.15.1:0/152402931 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fac340a0690 msgr2=0x7fac340a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:47.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.840+0000 7fac58c00700 1 --2- 172.21.15.1:0/152402931 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fac340a0690 0x7fac340a2b50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fac4c009610 tx=0x7fac4c006230 comp rx=0 tx=0).stop 2024-02-02T15:35:47.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.840+0000 7fac58c00700 1 -- 172.21.15.1:0/152402931 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fac54106110 msgr2=0x7fac541972f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:47.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.840+0000 7fac58c00700 1 --2- 172.21.15.1:0/152402931 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fac54106110 0x7fac541972f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fac3c00e970 tx=0x7fac3c00ec80 comp rx=0 tx=0).stop 2024-02-02T15:35:47.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.840+0000 7fac58c00700 1 -- 172.21.15.1:0/152402931 shutdown_connections 2024-02-02T15:35:47.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.840+0000 7fac58c00700 1 --2- 172.21.15.1:0/152402931 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fac340a0690 0x7fac340a2b50 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:47.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.840+0000 7fac58c00700 1 --2- 172.21.15.1:0/152402931 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fac5419b750 0x7fac5419bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:47.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.840+0000 7fac58c00700 1 --2- 172.21.15.1:0/152402931 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac54100ff0 0x7fac54196db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:47.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.840+0000 7fac58c00700 1 --2- 172.21.15.1:0/152402931 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fac54106110 0x7fac541972f0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:47.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.840+0000 7fac58c00700 1 -- 172.21.15.1:0/152402931 >> 172.21.15.1:0/152402931 conn(0x7fac540fc810 msgr2=0x7fac540fd240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:47.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.840+0000 7fac58c00700 1 -- 172.21.15.1:0/152402931 shutdown_connections 2024-02-02T15:35:47.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:47.840+0000 7fac58c00700 1 -- 172.21.15.1:0/152402931 wait complete. 2024-02-02T15:35:47.857 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:47 smithi001 ceph-mon[35470]: pgmap v270: 129 pgs: 129 active+clean; 382 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.6 MiB/s wr, 254 op/s 2024-02-02T15:35:47.860 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-02T15:35:47.860 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T15:35:47.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:47 smithi071 ceph-mon[44707]: pgmap v270: 129 pgs: 129 active+clean; 382 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.6 MiB/s wr, 254 op/s 2024-02-02T15:35:47.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:47 smithi142 ceph-mon[45728]: pgmap v270: 129 pgs: 129 active+clean; 382 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.6 MiB/s wr, 254 op/s 2024-02-02T15:35:48.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.072+0000 7faf15c00700 1 -- 172.21.15.1:0/2057820677 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf10068a30 msgr2=0x7faf1010dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:48.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.072+0000 7faf15c00700 1 --2- 172.21.15.1:0/2057820677 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf10068a30 0x7faf1010dac0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7faf0800b0a0 tx=0x7faf0800b3b0 comp rx=0 tx=0).stop 2024-02-02T15:35:48.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.072+0000 7faf15c00700 1 -- 172.21.15.1:0/2057820677 shutdown_connections 2024-02-02T15:35:48.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.072+0000 7faf15c00700 1 --2- 172.21.15.1:0/2057820677 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf10068a30 0x7faf1010dac0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:48.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.072+0000 7faf15c00700 1 --2- 172.21.15.1:0/2057820677 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf10106e20 0x7faf10107200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:48.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.072+0000 7faf15c00700 1 --2- 172.21.15.1:0/2057820677 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf10068070 0x7faf100684f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:48.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.072+0000 7faf15c00700 1 -- 172.21.15.1:0/2057820677 >> 172.21.15.1:0/2057820677 conn(0x7faf100769c0 msgr2=0x7faf10076dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:48.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.072+0000 7faf15c00700 1 -- 172.21.15.1:0/2057820677 shutdown_connections 2024-02-02T15:35:48.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.072+0000 7faf15c00700 1 -- 172.21.15.1:0/2057820677 wait complete. 2024-02-02T15:35:48.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.073+0000 7faf15c00700 1 Processor -- start 2024-02-02T15:35:48.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.073+0000 7faf15c00700 1 -- start start 2024-02-02T15:35:48.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.074+0000 7faf15c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf10068070 0x7faf100ffc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:48.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.074+0000 7faf15c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf10068a30 0x7faf10100170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:48.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.074+0000 7faf15c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf10106e20 0x7faf101a4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:48.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.074+0000 7faf15c00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf101009a0 con 0x7faf10068a30 2024-02-02T15:35:48.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.074+0000 7faf15c00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf101a5500 con 0x7faf10106e20 2024-02-02T15:35:48.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.074+0000 7faf15c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf101a5640 con 0x7faf10068070 2024-02-02T15:35:48.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.074+0000 7faf0ea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf10068070 0x7faf100ffc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:48.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.074+0000 7faf0ea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf10068070 0x7faf100ffc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:44050/0 (socket says 172.21.15.1:44050) 2024-02-02T15:35:48.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.074+0000 7faf0ea00700 1 -- 172.21.15.1:0/805197174 learned_addr learned my addr 172.21.15.1:0/805197174 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:35:48.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.074+0000 7faf0f400700 1 --2- 172.21.15.1:0/805197174 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf10106e20 0x7faf101a4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:48.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.074+0000 7faf0e000700 1 --2- 172.21.15.1:0/805197174 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf10068a30 0x7faf10100170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:48.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.075+0000 7faf0f400700 1 -- 172.21.15.1:0/805197174 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf10068070 msgr2=0x7faf100ffc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:48.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.075+0000 7faf0f400700 1 --2- 172.21.15.1:0/805197174 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf10068070 0x7faf100ffc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:48.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.075+0000 7faf0f400700 1 -- 172.21.15.1:0/805197174 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf10068a30 msgr2=0x7faf10100170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:48.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.075+0000 7faf0f400700 1 --2- 172.21.15.1:0/805197174 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf10068a30 0x7faf10100170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:48.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.075+0000 7faf0f400700 1 -- 172.21.15.1:0/805197174 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faef8009520 con 0x7faf10106e20 2024-02-02T15:35:48.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.075+0000 7faf0ea00700 1 --2- 172.21.15.1:0/805197174 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf10068070 0x7faf100ffc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:35:48.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.075+0000 7faf0f400700 1 --2- 172.21.15.1:0/805197174 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf10106e20 0x7faf101a4fc0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7faf08009fd0 tx=0x7faf08007930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:48.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.075+0000 7faf07400700 1 -- 172.21.15.1:0/805197174 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf0800e040 con 0x7faf10106e20 2024-02-02T15:35:48.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.075+0000 7faf07400700 1 -- 172.21.15.1:0/805197174 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faf0801d470 con 0x7faf10106e20 2024-02-02T15:35:48.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.075+0000 7faf07400700 1 -- 172.21.15.1:0/805197174 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf080052a0 con 0x7faf10106e20 2024-02-02T15:35:48.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.075+0000 7faf15c00700 1 -- 172.21.15.1:0/805197174 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf08009d00 con 0x7faf10106e20 2024-02-02T15:35:48.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.075+0000 7faf15c00700 1 -- 172.21.15.1:0/805197174 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf101a5ae0 con 0x7faf10106e20 2024-02-02T15:35:48.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.076+0000 7faf15c00700 1 -- 172.21.15.1:0/805197174 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf10067bc0 con 0x7faf10106e20 2024-02-02T15:35:48.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.078+0000 7faf07400700 1 -- 172.21.15.1:0/805197174 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7faf0801d5e0 con 0x7faf10106e20 2024-02-02T15:35:48.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.079+0000 7faf07400700 1 --2- 172.21.15.1:0/805197174 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7faef00a0710 0x7faef00a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:48.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.079+0000 7faf07400700 1 -- 172.21.15.1:0/805197174 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9453+0+0 (secure 0 0 0) 0x7faf080cd490 con 0x7faf10106e20 2024-02-02T15:35:48.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.080+0000 7faf0ea00700 1 --2- 172.21.15.1:0/805197174 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7faef00a0710 0x7faef00a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:48.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.080+0000 7faf0ea00700 1 --2- 172.21.15.1:0/805197174 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7faef00a0710 0x7faef00a2bd0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7faf00011550 tx=0x7faf00009260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:48.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.086+0000 7faf07400700 1 -- 172.21.15.1:0/805197174 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7faf08096980 con 0x7faf10106e20 2024-02-02T15:35:48.252 INFO:tasks.workunit.client.0.smithi001.stdout: 1 519586 6.40 MB/sec execute 322 sec latency 23.088 ms 2024-02-02T15:35:48.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.372+0000 7faf15c00700 1 -- 172.21.15.1:0/805197174 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faf10101160 con 0x7faf10106e20 2024-02-02T15:35:48.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.373+0000 7faf07400700 1 -- 172.21.15.1:0/805197174 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+10880 (secure 0 0 0) 0x7faf08099fa0 con 0x7faf10106e20 2024-02-02T15:35:48.373 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:35:48.373 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2922240448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2922240448},{"type":"v1","addr":"172.21.15.1:6835","nonce":2922240448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2729497431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2729497431},{"type":"v1","addr":"172.21.15.1:6837","nonce":2729497431}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":43,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-02T15:24:47.761435+0000","modified":"2024-02-02T15:35:45.458105+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24529,"mds_2":24518,"mds_3":24514,"mds_4":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14517":{"gid":14517,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/1218303683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":1218303683},{"type":"v1","addr":"172.21.15.1:6839","nonce":1218303683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14529":{"gid":14529,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/2059197496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2059197496},{"type":"v1","addr":"172.21.15.1:6841","nonce":2059197496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/7648986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":7648986},{"type":"v1","addr":"172.21.15.142:6833","nonce":7648986}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/3482746094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3482746094},{"type":"v1","addr":"172.21.15.71:6833","nonce":3482746094}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/388207814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":388207814},{"type":"v1","addr":"172.21.15.71:6835","nonce":388207814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6835/1052606547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1052606547},{"type":"v1","addr":"172.21.15.142:6835","nonce":1052606547}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.142:6837/2870289019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2870289019},{"type":"v1","addr":"172.21.15.142:6837","nonce":2870289019}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.71:6837/628016199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":628016199},{"type":"v1","addr":"172.21.15.71:6837","nonce":628016199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.142:6839/321675421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":321675421},{"type":"v1","addr":"172.21.15.142:6839","nonce":321675421}]},"join_fscid":-1,"export_targets":[0,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/1268086493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1268086493},{"type":"v1","addr":"172.21.15.71:6839","nonce":1268086493}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-02T15:35:48.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.375+0000 7faf15c00700 1 -- 172.21.15.1:0/805197174 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7faef00a0710 msgr2=0x7faef00a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:48.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.375+0000 7faf15c00700 1 --2- 172.21.15.1:0/805197174 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7faef00a0710 0x7faef00a2bd0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7faf00011550 tx=0x7faf00009260 comp rx=0 tx=0).stop 2024-02-02T15:35:48.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.375+0000 7faf15c00700 1 -- 172.21.15.1:0/805197174 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf10106e20 msgr2=0x7faf101a4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:48.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.375+0000 7faf15c00700 1 --2- 172.21.15.1:0/805197174 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf10106e20 0x7faf101a4fc0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7faf08009fd0 tx=0x7faf08007930 comp rx=0 tx=0).stop 2024-02-02T15:35:48.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.375+0000 7faf15c00700 1 -- 172.21.15.1:0/805197174 shutdown_connections 2024-02-02T15:35:48.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.375+0000 7faf15c00700 1 --2- 172.21.15.1:0/805197174 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7faef00a0710 0x7faef00a2bd0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:48.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.375+0000 7faf15c00700 1 --2- 172.21.15.1:0/805197174 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf10106e20 0x7faf101a4fc0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:48.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.375+0000 7faf15c00700 1 --2- 172.21.15.1:0/805197174 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf10068070 0x7faf100ffc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:48.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.375+0000 7faf15c00700 1 --2- 172.21.15.1:0/805197174 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf10068a30 0x7faf10100170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:48.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.375+0000 7faf15c00700 1 -- 172.21.15.1:0/805197174 >> 172.21.15.1:0/805197174 conn(0x7faf100769c0 msgr2=0x7faf100fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:48.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.376+0000 7faf15c00700 1 -- 172.21.15.1:0/805197174 shutdown_connections 2024-02-02T15:35:48.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.376+0000 7faf15c00700 1 -- 172.21.15.1:0/805197174 wait complete. 2024-02-02T15:35:48.377 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 43 2024-02-02T15:35:48.395 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-02-02T15:35:48.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.615+0000 7f2757000700 1 -- 172.21.15.1:0/3380883842 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2750106ed0 msgr2=0x7f27501072b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:48.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.615+0000 7f2757000700 1 --2- 172.21.15.1:0/3380883842 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2750106ed0 0x7f27501072b0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f27400099c0 tx=0x7f2740009cd0 comp rx=0 tx=0).stop 2024-02-02T15:35:48.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.615+0000 7f2757000700 1 -- 172.21.15.1:0/3380883842 shutdown_connections 2024-02-02T15:35:48.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.615+0000 7f2757000700 1 --2- 172.21.15.1:0/3380883842 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27500689b0 0x7f275010dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:48.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.615+0000 7f2757000700 1 --2- 172.21.15.1:0/3380883842 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2750106ed0 0x7f27501072b0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:48.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.615+0000 7f2757000700 1 --2- 172.21.15.1:0/3380883842 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2750068010 0x7f2750068470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:48.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.615+0000 7f2757000700 1 -- 172.21.15.1:0/3380883842 >> 172.21.15.1:0/3380883842 conn(0x7f27500757d0 msgr2=0x7f2750075be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:48.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.615+0000 7f2757000700 1 -- 172.21.15.1:0/3380883842 shutdown_connections 2024-02-02T15:35:48.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.615+0000 7f2757000700 1 -- 172.21.15.1:0/3380883842 wait complete. 2024-02-02T15:35:48.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.616+0000 7f2757000700 1 Processor -- start 2024-02-02T15:35:48.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.616+0000 7f2757000700 1 -- start start 2024-02-02T15:35:48.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.616+0000 7f2757000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2750068010 0x7f2750199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:48.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.616+0000 7f2757000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27500689b0 0x7f27501997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:48.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.616+0000 7f2757000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2750106ed0 0x7f275019dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:48.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.617+0000 7f2757000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f275019e240 con 0x7f2750068010 2024-02-02T15:35:48.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.617+0000 7f2757000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f275019e3b0 con 0x7f27500689b0 2024-02-02T15:35:48.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.617+0000 7f2757000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f275019e570 con 0x7f2750106ed0 2024-02-02T15:35:48.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.617+0000 7f274fe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2750068010 0x7f2750199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:48.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.617+0000 7f274f400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27500689b0 0x7f27501997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:48.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.617+0000 7f274f400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27500689b0 0x7f27501997a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:41932/0 (socket says 172.21.15.1:41932) 2024-02-02T15:35:48.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.617+0000 7f274f400700 1 -- 172.21.15.1:0/3112035720 learned_addr learned my addr 172.21.15.1:0/3112035720 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:35:48.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.617+0000 7f274fe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2750068010 0x7f2750199260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49134/0 (socket says 172.21.15.1:49134) 2024-02-02T15:35:48.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.617+0000 7f2755000700 1 --2- 172.21.15.1:0/3112035720 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2750106ed0 0x7f275019dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:48.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.617+0000 7f274f400700 1 -- 172.21.15.1:0/3112035720 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2750106ed0 msgr2=0x7f275019dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:48.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.617+0000 7f274f400700 1 --2- 172.21.15.1:0/3112035720 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2750106ed0 0x7f275019dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:48.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.617+0000 7f274f400700 1 -- 172.21.15.1:0/3112035720 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2750068010 msgr2=0x7f2750199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:48.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.617+0000 7f274f400700 1 --2- 172.21.15.1:0/3112035720 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2750068010 0x7f2750199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:48.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.617+0000 7f274f400700 1 -- 172.21.15.1:0/3112035720 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27400095f0 con 0x7f27500689b0 2024-02-02T15:35:48.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.617+0000 7f274fe00700 1 --2- 172.21.15.1:0/3112035720 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2750068010 0x7f2750199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:35:48.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.617+0000 7f274f400700 1 --2- 172.21.15.1:0/3112035720 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27500689b0 0x7f27501997a0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f273800e9a0 tx=0x7f273800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:48.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.618+0000 7f274cc00700 1 -- 172.21.15.1:0/3112035720 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2738009790 con 0x7f27500689b0 2024-02-02T15:35:48.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.618+0000 7f2757000700 1 -- 172.21.15.1:0/3112035720 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f275019e850 con 0x7f27500689b0 2024-02-02T15:35:48.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.618+0000 7f274cc00700 1 -- 172.21.15.1:0/3112035720 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2738010430 con 0x7f27500689b0 2024-02-02T15:35:48.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.618+0000 7f274cc00700 1 -- 172.21.15.1:0/3112035720 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2738018680 con 0x7f27500689b0 2024-02-02T15:35:48.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.618+0000 7f2757000700 1 -- 172.21.15.1:0/3112035720 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27501a6090 con 0x7f27500689b0 2024-02-02T15:35:48.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.621+0000 7f274cc00700 1 -- 172.21.15.1:0/3112035720 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f27380105a0 con 0x7f27500689b0 2024-02-02T15:35:48.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.623+0000 7f274cc00700 1 --2- 172.21.15.1:0/3112035720 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f27300a0710 0x7f27300a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:48.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.623+0000 7f276ac1fb80 1 Processor -- start 2024-02-02T15:35:48.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.623+0000 7f274cc00700 1 -- 172.21.15.1:0/3112035720 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f2738014070 con 0x7f27500689b0 2024-02-02T15:35:48.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.623+0000 7f276ac1fb80 1 -- start start 2024-02-02T15:35:48.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.623+0000 7f274fe00700 1 --2- 172.21.15.1:0/3112035720 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f27300a0710 0x7f27300a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:48.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.623+0000 7f276ac1fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564be90fc010 0x564be90fc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:48.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.623+0000 7f276ac1fb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564be90fc930 0x564be91072a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:48.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.623+0000 7f276ac1fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564be91077e0 0x564be9109bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:48.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.623+0000 7f276ac1fb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x564be90704c0 con 0x564be90fc930 2024-02-02T15:35:48.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.623+0000 7f276ac1fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x564be9054aa0 con 0x564be91077e0 2024-02-02T15:35:48.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.623+0000 7f276ac1fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564be9052460 con 0x564be90fc010 2024-02-02T15:35:48.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.623+0000 7f274f400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564be90fc930 0x564be91072a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:48.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.623+0000 7f274fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564be90fc010 0x564be90fc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:48.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.623+0000 7f2755000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564be91077e0 0x564be9109bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:48.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.623+0000 7f274f400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564be90fc930 0x564be91072a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49140/0 (socket says 172.21.15.1:49140) 2024-02-02T15:35:48.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.623+0000 7f274f400700 1 -- 172.21.15.1:0/2075694074 learned_addr learned my addr 172.21.15.1:0/2075694074 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:35:48.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.624+0000 7f2755000700 1 -- 172.21.15.1:0/2075694074 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564be90fc010 msgr2=0x564be90fc3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:48.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.624+0000 7f2755000700 1 --2- 172.21.15.1:0/2075694074 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564be90fc010 0x564be90fc3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:48.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.624+0000 7f2755000700 1 -- 172.21.15.1:0/2075694074 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564be90fc930 msgr2=0x564be91072a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:48.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.624+0000 7f2755000700 1 --2- 172.21.15.1:0/2075694074 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564be90fc930 0x564be91072a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:48.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.624+0000 7f274fe00700 1 --2- 172.21.15.1:0/3112035720 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f27300a0710 0x7f27300a2bd0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2740009990 tx=0x7f274001a040 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:48.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.624+0000 7f2755000700 1 -- 172.21.15.1:0/2075694074 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564be8ecf3b0 con 0x564be91077e0 2024-02-02T15:35:48.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.624+0000 7f274f400700 1 --2- 172.21.15.1:0/2075694074 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564be90fc930 0x564be91072a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:35:48.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.624+0000 7f2755000700 1 --2- 172.21.15.1:0/2075694074 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564be91077e0 0x564be9109bd0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f27480115b0 tx=0x7f2748009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0f371661969b25d server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:48.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.624+0000 7f2745600700 1 -- 172.21.15.1:0/2075694074 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f274800e040 con 0x564be91077e0 2024-02-02T15:35:48.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.624+0000 7f2745600700 1 -- 172.21.15.1:0/2075694074 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f274800f040 con 0x564be91077e0 2024-02-02T15:35:48.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.624+0000 7f2745600700 1 -- 172.21.15.1:0/2075694074 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27480043d0 con 0x564be91077e0 2024-02-02T15:35:48.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.625+0000 7f276ac1fb80 1 -- 172.21.15.1:0/2075694074 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564be91077e0 msgr2=0x564be9109bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:48.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.625+0000 7f276ac1fb80 1 --2- 172.21.15.1:0/2075694074 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564be91077e0 0x564be9109bd0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f27480115b0 tx=0x7f2748009920 comp rx=0 tx=0).stop 2024-02-02T15:35:48.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.625+0000 7f276ac1fb80 1 -- 172.21.15.1:0/2075694074 shutdown_connections 2024-02-02T15:35:48.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.625+0000 7f276ac1fb80 1 --2- 172.21.15.1:0/2075694074 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564be91077e0 0x564be9109bd0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:48.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.625+0000 7f276ac1fb80 1 --2- 172.21.15.1:0/2075694074 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564be90fc010 0x564be90fc3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:48.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.625+0000 7f276ac1fb80 1 --2- 172.21.15.1:0/2075694074 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564be90fc930 0x564be91072a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:48.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.625+0000 7f276ac1fb80 1 -- 172.21.15.1:0/2075694074 >> 172.21.15.1:0/2075694074 conn(0x564be905bdb0 msgr2=0x564be90f83b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:48.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.625+0000 7f276ac1fb80 1 -- 172.21.15.1:0/2075694074 shutdown_connections 2024-02-02T15:35:48.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.625+0000 7f276ac1fb80 1 -- 172.21.15.1:0/2075694074 wait complete. 2024-02-02T15:35:48.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.626+0000 7f276ac1fb80 1 -- start start 2024-02-02T15:35:48.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.626+0000 7f276ac1fb80 1 Processor -- start 2024-02-02T15:35:48.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.626+0000 7f276ac1fb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564be90fc010 0x564be9107190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:48.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.626+0000 7f276ac1fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564be90fc930 0x564be9104410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:48.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.626+0000 7f276ac1fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564be91077e0 0x564be91049d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:48.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.626+0000 7f276ac1fb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x564be904dd50 con 0x564be90fc010 2024-02-02T15:35:48.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.626+0000 7f276ac1fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x564be8fe9d80 con 0x564be91077e0 2024-02-02T15:35:48.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.626+0000 7f276ac1fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564be8fe9a20 con 0x564be90fc930 2024-02-02T15:35:48.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.627+0000 7f276ac1fb80 10 client.0 _ll_register_callbacks cb 0x564be8feb290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:35:48.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.627+0000 7f274fe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564be90fc010 0x564be9107190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:48.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.627+0000 7f274f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564be90fc930 0x564be9104410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:48.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.627+0000 7f2755000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564be91077e0 0x564be91049d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:48.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.627+0000 7f274fe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564be90fc010 0x564be9107190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49148/0 (socket says 172.21.15.1:49148) 2024-02-02T15:35:48.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.627+0000 7f274fe00700 1 -- 172.21.15.1:0/3785247541 learned_addr learned my addr 172.21.15.1:0/3785247541 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:35:48.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.627+0000 7f274f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564be90fc930 0x564be9104410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:44086/0 (socket says 172.21.15.1:44086) 2024-02-02T15:35:48.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.627+0000 7f274fe00700 1 -- 172.21.15.1:0/3785247541 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564be90fc930 msgr2=0x564be9104410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:48.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.627+0000 7f274fe00700 1 --2- 172.21.15.1:0/3785247541 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564be90fc930 0x564be9104410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:48.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.627+0000 7f274fe00700 1 -- 172.21.15.1:0/3785247541 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564be91077e0 msgr2=0x564be91049d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:48.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.627+0000 7f274fe00700 1 --2- 172.21.15.1:0/3785247541 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564be91077e0 0x564be91049d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:48.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.627+0000 7f274fe00700 1 -- 172.21.15.1:0/3785247541 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27501a6090 con 0x564be90fc010 2024-02-02T15:35:48.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.627+0000 7f274f400700 1 --2- 172.21.15.1:0/3785247541 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564be90fc930 0x564be9104410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:35:48.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.628+0000 7f274fe00700 1 --2- 172.21.15.1:0/3785247541 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564be90fc010 0x564be9107190 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f2740004800 tx=0x7f2740009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47ce0aecb1277e1d server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:48.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.628+0000 7f273e000700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:35:48.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.628+0000 7f273e000700 1 -- 172.21.15.1:0/3785247541 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2740031070 con 0x564be90fc010 2024-02-02T15:35:48.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.628+0000 7f273e000700 1 -- 172.21.15.1:0/3785247541 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f2740004830 con 0x564be90fc010 2024-02-02T15:35:48.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.628+0000 7f276ac1fb80 1 -- 172.21.15.1:0/3785247541 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564be90f00f0 con 0x564be90fc010 2024-02-02T15:35:48.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.628+0000 7f273e000700 1 -- 172.21.15.1:0/3785247541 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f27400049f0 con 0x564be90fc010 2024-02-02T15:35:48.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.628+0000 7f276ac1fb80 10 client.17853 fetch_fsmap learned FSMap version 43 2024-02-02T15:35:48.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.628+0000 7f276ac1fb80 1 -- 172.21.15.1:0/3785247541 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x7f2748011230 con 0x564be90fc010 2024-02-02T15:35:48.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.628+0000 7f273e000700 1 -- 172.21.15.1:0/3785247541 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564be90f00f0 con 0x564be90fc010 2024-02-02T15:35:48.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.629+0000 7f273e000700 1 -- 172.21.15.1:0/3785247541 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 43) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f2740034020 con 0x564be90fc010 2024-02-02T15:35:48.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.629+0000 7f276ac1fb80 10 client.17853 fetch_fsmap finished waiting for FSMap version 43 2024-02-02T15:35:48.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.629+0000 7f276ac1fb80 10 client.17853 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-02-02T15:35:48.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.629+0000 7f276ac1fb80 20 client.17853 populate_metadata read hostname 'smithi001' 2024-02-02T15:35:48.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.629+0000 7f276ac1fb80 1 --2- 172.21.15.1:0/3785247541 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x564be9117200 0x564be9119620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:48.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.629+0000 7f276ac1fb80 4 client.17853 mds_command: new command op to 24502 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:35:48.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.629+0000 7f276ac1fb80 1 -- 172.21.15.1:0/3785247541 --> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564be9046dd0 con 0x564be9117200 2024-02-02T15:35:48.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.629+0000 7f274f400700 1 --2- 172.21.15.1:0/3785247541 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x564be9117200 0x564be9119620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:48.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.629+0000 7f274f400700 1 --2- 172.21.15.1:0/3785247541 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x564be9117200 0x564be9119620 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a57e8c691871084c server_cookie=3b46825bb71782b2 in_seq=0 out_seq=0 2024-02-02T15:35:48.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.629+0000 7f273e000700 10 client.17853 ms_handle_connect on v2:172.21.15.142:6834/1052606547 2024-02-02T15:35:48.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.630+0000 7f273e000700 1 -- 172.21.15.1:0/3785247541 <== mds.4 v2:172.21.15.142:6834/1052606547 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x564be9046dd0 con 0x564be9117200 2024-02-02T15:35:48.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.630+0000 7f273e000700 10 client.17853 handle_command_reply: tid=0 2024-02-02T15:35:48.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.630+0000 7f276ac1fb80 1 client.17853 shutdown 2024-02-02T15:35:48.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.630+0000 7f276ac1fb80 1 -- 172.21.15.1:0/3785247541 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564be90fc010 msgr2=0x564be9107190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:48.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.630+0000 7f276ac1fb80 1 --2- 172.21.15.1:0/3785247541 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564be90fc010 0x564be9107190 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f2740004800 tx=0x7f2740009310 comp rx=0 tx=0).stop 2024-02-02T15:35:48.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.630+0000 7f276ac1fb80 1 -- 172.21.15.1:0/3785247541 shutdown_connections 2024-02-02T15:35:48.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.630+0000 7f276ac1fb80 1 --2- 172.21.15.1:0/3785247541 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x564be9117200 0x564be9119620 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:48.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.630+0000 7f276ac1fb80 1 --2- 172.21.15.1:0/3785247541 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564be91077e0 0x564be91049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:48.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.630+0000 7f276ac1fb80 1 --2- 172.21.15.1:0/3785247541 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564be90fc010 0x564be9107190 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:48.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.630+0000 7f276ac1fb80 1 --2- 172.21.15.1:0/3785247541 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564be90fc930 0x564be9104410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:48.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.630+0000 7f276ac1fb80 1 -- 172.21.15.1:0/3785247541 >> 172.21.15.1:0/3785247541 conn(0x564be905bdb0 msgr2=0x564be90f7fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:48.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.630+0000 7f273e000700 0 client.17853 ms_handle_reset on v2:172.21.15.142:6834/1052606547 2024-02-02T15:35:48.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.631+0000 7f276ac1fb80 1 -- 172.21.15.1:0/3785247541 shutdown_connections 2024-02-02T15:35:48.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.631+0000 7f276ac1fb80 1 -- 172.21.15.1:0/3785247541 wait complete. 2024-02-02T15:35:48.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.631+0000 7f276ac1fb80 20 client.17853 trim_cache size 0 max 16384 2024-02-02T15:35:48.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.645+0000 7f276ac1fb80 1 Processor -- start 2024-02-02T15:35:48.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.645+0000 7f276ac1fb80 1 -- start start 2024-02-02T15:35:48.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.645+0000 7f276ac1fb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564be90fc010 0x564be91134e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:48.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.645+0000 7f276ac1fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564be90fc930 0x564be9106cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:48.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.645+0000 7f276ac1fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564be91077e0 0x564be9103d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:48.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.645+0000 7f276ac1fb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x564be8ed16a0 con 0x564be90fc010 2024-02-02T15:35:48.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.645+0000 7f276ac1fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x564be91002b0 con 0x564be90fc930 2024-02-02T15:35:48.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.645+0000 7f276ac1fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564be9109bd0 con 0x564be91077e0 2024-02-02T15:35:48.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.645+0000 7f274f400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564be90fc930 0x564be9106cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:48.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.645+0000 7f274f400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564be90fc930 0x564be9106cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:41964/0 (socket says 172.21.15.1:41964) 2024-02-02T15:35:48.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.645+0000 7f2755000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564be91077e0 0x564be9103d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:48.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.645+0000 7f274f400700 1 -- 172.21.15.1:0/1592305520 learned_addr learned my addr 172.21.15.1:0/1592305520 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:35:48.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.645+0000 7f274fe00700 1 --2- 172.21.15.1:0/1592305520 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564be90fc010 0x564be91134e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:48.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.646+0000 7f274f400700 1 -- 172.21.15.1:0/1592305520 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564be91077e0 msgr2=0x564be9103d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:48.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.646+0000 7f274f400700 1 --2- 172.21.15.1:0/1592305520 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564be91077e0 0x564be9103d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:48.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.646+0000 7f274f400700 1 -- 172.21.15.1:0/1592305520 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564be90fc010 msgr2=0x564be91134e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:48.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.646+0000 7f274f400700 1 --2- 172.21.15.1:0/1592305520 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564be90fc010 0x564be91134e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:48.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.646+0000 7f274f400700 1 -- 172.21.15.1:0/1592305520 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564be8e7bb60 con 0x564be90fc930 2024-02-02T15:35:48.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.646+0000 7f2755000700 1 --2- 172.21.15.1:0/1592305520 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564be91077e0 0x564be9103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:35:48.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.646+0000 7f274fe00700 1 --2- 172.21.15.1:0/1592305520 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564be90fc010 0x564be91134e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:35:48.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.646+0000 7f274f400700 1 --2- 172.21.15.1:0/1592305520 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564be90fc930 0x564be9106cc0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f27380958c0 tx=0x7f2738010c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e93d12a5ac35d5b8 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:48.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.646+0000 7f2746000700 1 -- 172.21.15.1:0/1592305520 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27380840b0 con 0x564be90fc930 2024-02-02T15:35:48.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.646+0000 7f2746000700 1 -- 172.21.15.1:0/1592305520 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2738099ac0 con 0x564be90fc930 2024-02-02T15:35:48.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.646+0000 7f2746000700 1 -- 172.21.15.1:0/1592305520 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2738010880 con 0x564be90fc930 2024-02-02T15:35:48.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.647+0000 7f276ac1fb80 1 -- 172.21.15.1:0/1592305520 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564be90fc930 msgr2=0x564be9106cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:48.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.647+0000 7f276ac1fb80 1 --2- 172.21.15.1:0/1592305520 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564be90fc930 0x564be9106cc0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f27380958c0 tx=0x7f2738010c80 comp rx=0 tx=0).stop 2024-02-02T15:35:48.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.647+0000 7f276ac1fb80 1 -- 172.21.15.1:0/1592305520 shutdown_connections 2024-02-02T15:35:48.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.647+0000 7f276ac1fb80 1 --2- 172.21.15.1:0/1592305520 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564be91077e0 0x564be9103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:48.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.647+0000 7f276ac1fb80 1 --2- 172.21.15.1:0/1592305520 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564be90fc010 0x564be91134e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:48.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.647+0000 7f276ac1fb80 1 --2- 172.21.15.1:0/1592305520 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564be90fc930 0x564be9106cc0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:48.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.647+0000 7f276ac1fb80 1 -- 172.21.15.1:0/1592305520 >> 172.21.15.1:0/1592305520 conn(0x564be905bdb0 msgr2=0x564be90f3e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:48.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.647+0000 7f276ac1fb80 1 -- 172.21.15.1:0/1592305520 shutdown_connections 2024-02-02T15:35:48.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.647+0000 7f276ac1fb80 1 -- 172.21.15.1:0/1592305520 wait complete. 2024-02-02T15:35:48.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.647+0000 7f276ac1fb80 1 -- start start 2024-02-02T15:35:48.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.648+0000 7f276ac1fb80 1 Processor -- start 2024-02-02T15:35:48.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.648+0000 7f276ac1fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564be90fc010 0x564be910a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:48.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.648+0000 7f276ac1fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564be90fc930 0x564be910a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:48.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.648+0000 7f276ac1fb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564be91077e0 0x564be9167d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:48.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.648+0000 7f276ac1fb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x564be910b150 con 0x564be91077e0 2024-02-02T15:35:48.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.648+0000 7f276ac1fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x564be9168250 con 0x564be90fc930 2024-02-02T15:35:48.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.648+0000 7f276ac1fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564be9168390 con 0x564be90fc010 2024-02-02T15:35:48.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.648+0000 7f276ac1fb80 10 client.0 _ll_register_callbacks cb 0x564be8feb290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:35:48.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.648+0000 7f274f400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564be90fc930 0x564be910a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:48.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.649+0000 7f274fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564be90fc010 0x564be910a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:48.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.649+0000 7f274f400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564be90fc930 0x564be910a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:41972/0 (socket says 172.21.15.1:41972) 2024-02-02T15:35:48.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.649+0000 7f274f400700 1 -- 172.21.15.1:0/2492952308 learned_addr learned my addr 172.21.15.1:0/2492952308 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:35:48.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.649+0000 7f2755000700 1 --2- 172.21.15.1:0/2492952308 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564be91077e0 0x564be9167d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:48.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.649+0000 7f274f400700 1 -- 172.21.15.1:0/2492952308 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564be90fc010 msgr2=0x564be910a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:48.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.649+0000 7f274f400700 1 --2- 172.21.15.1:0/2492952308 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564be90fc010 0x564be910a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:48.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.649+0000 7f274f400700 1 -- 172.21.15.1:0/2492952308 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564be91077e0 msgr2=0x564be9167d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:48.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.649+0000 7f274f400700 1 --2- 172.21.15.1:0/2492952308 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564be91077e0 0x564be9167d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:48.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.649+0000 7f274f400700 1 -- 172.21.15.1:0/2492952308 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564be8ecf3b0 con 0x564be90fc930 2024-02-02T15:35:48.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.649+0000 7f274fe00700 1 --2- 172.21.15.1:0/2492952308 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564be90fc010 0x564be910a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:35:48.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.649+0000 7f274f400700 1 --2- 172.21.15.1:0/2492952308 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564be90fc930 0x564be910a840 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f27380a0f00 tx=0x7f2738095820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50b00d3069a4fae2 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:48.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.649+0000 7f273e000700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:35:48.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.649+0000 7f273e000700 1 -- 172.21.15.1:0/2492952308 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2738097800 con 0x564be90fc930 2024-02-02T15:35:48.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.649+0000 7f273e000700 1 -- 172.21.15.1:0/2492952308 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f27380a1430 con 0x564be90fc930 2024-02-02T15:35:48.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.649+0000 7f276ac1fb80 1 -- 172.21.15.1:0/2492952308 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f274800f040 con 0x564be90fc930 2024-02-02T15:35:48.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.650+0000 7f273e000700 1 -- 172.21.15.1:0/2492952308 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f27380993c0 con 0x564be90fc930 2024-02-02T15:35:48.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.650+0000 7f276ac1fb80 10 client.27950 fetch_fsmap learned FSMap version 43 2024-02-02T15:35:48.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.650+0000 7f276ac1fb80 1 -- 172.21.15.1:0/2492952308 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x7f275019e850 con 0x564be90fc930 2024-02-02T15:35:48.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.650+0000 7f273e000700 1 -- 172.21.15.1:0/2492952308 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2738082020 con 0x564be90fc930 2024-02-02T15:35:48.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.650+0000 7f273e000700 1 -- 172.21.15.1:0/2492952308 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 43) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f2738082bc0 con 0x564be90fc930 2024-02-02T15:35:48.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.650+0000 7f276ac1fb80 10 client.27950 fetch_fsmap finished waiting for FSMap version 43 2024-02-02T15:35:48.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.650+0000 7f276ac1fb80 10 client.27950 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-02-02T15:35:48.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.650+0000 7f276ac1fb80 20 client.27950 populate_metadata read hostname 'smithi001' 2024-02-02T15:35:48.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.650+0000 7f276ac1fb80 1 --2- 172.21.15.1:0/2492952308 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x564be9169740 0x564be916bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:48.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.650+0000 7f276ac1fb80 4 client.27950 mds_command: new command op to 24502 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:35:48.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.650+0000 7f276ac1fb80 1 -- 172.21.15.1:0/2492952308 --> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564be900cf50 con 0x564be9169740 2024-02-02T15:35:48.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.650+0000 7f274fe00700 1 --2- 172.21.15.1:0/2492952308 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x564be9169740 0x564be916bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:48.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.651+0000 7f273e000700 10 client.27950 ms_handle_connect on v2:172.21.15.142:6834/1052606547 2024-02-02T15:35:48.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.651+0000 7f274fe00700 1 --2- 172.21.15.1:0/2492952308 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x564be9169740 0x564be916bb30 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=75978e188b875d72 server_cookie=5b9fed71d2bd5d89 in_seq=0 out_seq=0 2024-02-02T15:35:48.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.651+0000 7f273e000700 1 -- 172.21.15.1:0/2492952308 <== mds.4 v2:172.21.15.142:6834/1052606547 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564be900cf50 con 0x564be9169740 2024-02-02T15:35:48.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.651+0000 7f273e000700 10 client.27950 handle_command_reply: tid=0 2024-02-02T15:35:48.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.651+0000 7f276ac1fb80 1 client.27950 shutdown 2024-02-02T15:35:48.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.651+0000 7f276ac1fb80 1 -- 172.21.15.1:0/2492952308 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564be90fc930 msgr2=0x564be910a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:48.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.651+0000 7f276ac1fb80 1 --2- 172.21.15.1:0/2492952308 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564be90fc930 0x564be910a840 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f27380a0f00 tx=0x7f2738095820 comp rx=0 tx=0).stop 2024-02-02T15:35:48.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.652+0000 7f276ac1fb80 1 -- 172.21.15.1:0/2492952308 shutdown_connections 2024-02-02T15:35:48.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.652+0000 7f276ac1fb80 1 --2- 172.21.15.1:0/2492952308 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x564be9169740 0x564be916bb30 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:48.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.652+0000 7f276ac1fb80 1 --2- 172.21.15.1:0/2492952308 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564be91077e0 0x564be9167d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:48.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.652+0000 7f276ac1fb80 1 --2- 172.21.15.1:0/2492952308 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564be90fc010 0x564be910a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:48.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.652+0000 7f276ac1fb80 1 --2- 172.21.15.1:0/2492952308 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564be90fc930 0x564be910a840 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:48.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.652+0000 7f276ac1fb80 1 -- 172.21.15.1:0/2492952308 >> 172.21.15.1:0/2492952308 conn(0x564be905bdb0 msgr2=0x564be91134e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:48.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.652+0000 7f273e000700 0 client.27950 ms_handle_reset on v2:172.21.15.142:6834/1052606547 2024-02-02T15:35:48.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.652+0000 7f276ac1fb80 1 -- 172.21.15.1:0/2492952308 shutdown_connections 2024-02-02T15:35:48.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.652+0000 7f276ac1fb80 1 -- 172.21.15.1:0/2492952308 wait complete. 2024-02-02T15:35:48.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.653+0000 7f276ac1fb80 20 client.27950 trim_cache size 0 max 16384 2024-02-02T15:35:48.658 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:35:48.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.654+0000 7f2757000700 1 -- 172.21.15.1:0/3112035720 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f27300a0710 msgr2=0x7f27300a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:48.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.654+0000 7f2757000700 1 --2- 172.21.15.1:0/3112035720 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f27300a0710 0x7f27300a2bd0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2740009990 tx=0x7f274001a040 comp rx=0 tx=0).stop 2024-02-02T15:35:48.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.654+0000 7f2757000700 1 -- 172.21.15.1:0/3112035720 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27500689b0 msgr2=0x7f27501997a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:48.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.654+0000 7f2757000700 1 --2- 172.21.15.1:0/3112035720 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27500689b0 0x7f27501997a0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f273800e9a0 tx=0x7f273800ecb0 comp rx=0 tx=0).stop 2024-02-02T15:35:48.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.655+0000 7f2757000700 1 -- 172.21.15.1:0/3112035720 shutdown_connections 2024-02-02T15:35:48.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.655+0000 7f2757000700 1 --2- 172.21.15.1:0/3112035720 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f27300a0710 0x7f27300a2bd0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:48.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.655+0000 7f2757000700 1 --2- 172.21.15.1:0/3112035720 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2750106ed0 0x7f275019dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:48.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.655+0000 7f2757000700 1 --2- 172.21.15.1:0/3112035720 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2750068010 0x7f2750199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:48.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.655+0000 7f2757000700 1 --2- 172.21.15.1:0/3112035720 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f27500689b0 0x7f27501997a0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:48.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.655+0000 7f2757000700 1 -- 172.21.15.1:0/3112035720 >> 172.21.15.1:0/3112035720 conn(0x7f27500757d0 msgr2=0x7f275010bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:48.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.655+0000 7f2757000700 1 -- 172.21.15.1:0/3112035720 shutdown_connections 2024-02-02T15:35:48.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.656+0000 7f2757000700 1 -- 172.21.15.1:0/3112035720 wait complete. 2024-02-02T15:35:48.683 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:48 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/805197174' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:35:48.686 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-02-02T15:35:48.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:48 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/805197174' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:35:48.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.911+0000 7f19f8800700 1 -- 172.21.15.1:0/396673227 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f19f0102950 msgr2=0x7f19f010ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:48.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.911+0000 7f19f8800700 1 --2- 172.21.15.1:0/396673227 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f19f0102950 0x7f19f010ae30 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f19d8009990 tx=0x7f19d8009ca0 comp rx=0 tx=0).stop 2024-02-02T15:35:48.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.911+0000 7f19f8800700 1 -- 172.21.15.1:0/396673227 shutdown_connections 2024-02-02T15:35:48.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.911+0000 7f19f8800700 1 --2- 172.21.15.1:0/396673227 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f19f010b400 0x7f19f010d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:48.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.911+0000 7f19f8800700 1 --2- 172.21.15.1:0/396673227 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19f0102030 0x7f19f0102410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:48.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.911+0000 7f19f8800700 1 --2- 172.21.15.1:0/396673227 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f19f0102950 0x7f19f010ae30 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:48.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.911+0000 7f19f8800700 1 -- 172.21.15.1:0/396673227 >> 172.21.15.1:0/396673227 conn(0x7f19f0076d70 msgr2=0x7f19f0077180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:48.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.911+0000 7f19f8800700 1 -- 172.21.15.1:0/396673227 shutdown_connections 2024-02-02T15:35:48.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.911+0000 7f19f8800700 1 -- 172.21.15.1:0/396673227 wait complete. 2024-02-02T15:35:48.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.912+0000 7f19f8800700 1 Processor -- start 2024-02-02T15:35:48.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.912+0000 7f19f8800700 1 -- start start 2024-02-02T15:35:48.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.913+0000 7f19f8800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f19f0102030 0x7f19f01990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:48.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.913+0000 7f19f8800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f19f0102950 0x7f19f0199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:48.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.913+0000 7f19f8800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19f010b400 0x7f19f019da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:48.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.913+0000 7f19f8800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f019e0a0 con 0x7f19f0102950 2024-02-02T15:35:48.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.913+0000 7f19f8800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f019e210 con 0x7f19f0102030 2024-02-02T15:35:48.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.913+0000 7f19f8800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f019e3d0 con 0x7f19f010b400 2024-02-02T15:35:48.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.913+0000 7f19f5e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f19f0102030 0x7f19f01990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:48.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.913+0000 7f19f5e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f19f0102030 0x7f19f01990c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:41984/0 (socket says 172.21.15.1:41984) 2024-02-02T15:35:48.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.913+0000 7f19f5400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f19f0102950 0x7f19f0199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:48.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.913+0000 7f19f5e00700 1 -- 172.21.15.1:0/512445045 learned_addr learned my addr 172.21.15.1:0/512445045 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:35:48.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.913+0000 7f19f6800700 1 --2- 172.21.15.1:0/512445045 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19f010b400 0x7f19f019da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:48.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.913+0000 7f19f5e00700 1 -- 172.21.15.1:0/512445045 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19f010b400 msgr2=0x7f19f019da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:48.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.913+0000 7f19f5e00700 1 --2- 172.21.15.1:0/512445045 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19f010b400 0x7f19f019da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:48.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.913+0000 7f19f5e00700 1 -- 172.21.15.1:0/512445045 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f19f0102950 msgr2=0x7f19f0199600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:48.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.913+0000 7f19f5e00700 1 --2- 172.21.15.1:0/512445045 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f19f0102950 0x7f19f0199600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:48.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.913+0000 7f19f5e00700 1 -- 172.21.15.1:0/512445045 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19d80095f0 con 0x7f19f0102030 2024-02-02T15:35:48.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.913+0000 7f19f5400700 1 --2- 172.21.15.1:0/512445045 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f19f0102950 0x7f19f0199600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:35:48.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.914+0000 7f19f5e00700 1 --2- 172.21.15.1:0/512445045 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f19f0102030 0x7f19f01990c0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f19e000d980 tx=0x7f19e000dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:48.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.914+0000 7f19eea00700 1 -- 172.21.15.1:0/512445045 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e0004220 con 0x7f19f0102030 2024-02-02T15:35:48.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.914+0000 7f19eea00700 1 -- 172.21.15.1:0/512445045 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f19e0007640 con 0x7f19f0102030 2024-02-02T15:35:48.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.914+0000 7f19f8800700 1 -- 172.21.15.1:0/512445045 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19f019e6b0 con 0x7f19f0102030 2024-02-02T15:35:48.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.914+0000 7f19eea00700 1 -- 172.21.15.1:0/512445045 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e0005690 con 0x7f19f0102030 2024-02-02T15:35:48.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.914+0000 7f19f8800700 1 -- 172.21.15.1:0/512445045 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19f01a5ef0 con 0x7f19f0102030 2024-02-02T15:35:48.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.917+0000 7f19eea00700 1 -- 172.21.15.1:0/512445045 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f19e0004750 con 0x7f19f0102030 2024-02-02T15:35:48.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.918+0000 7f1a0c3f7b80 1 Processor -- start 2024-02-02T15:35:48.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.919+0000 7f1a0c3f7b80 1 -- start start 2024-02-02T15:35:48.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.919+0000 7f1a0c3f7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc967ca010 0x55cc967ca3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:48.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.919+0000 7f1a0c3f7b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cc967ca930 0x55cc967d52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:48.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.919+0000 7f1a0c3f7b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc967d57e0 0x55cc967d7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:48.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.919+0000 7f1a0c3f7b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc9673e4c0 con 0x55cc967ca930 2024-02-02T15:35:48.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.919+0000 7f1a0c3f7b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc96722aa0 con 0x55cc967d57e0 2024-02-02T15:35:48.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.919+0000 7f1a0c3f7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc96720460 con 0x55cc967ca010 2024-02-02T15:35:48.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.919+0000 7f19f6800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc967ca010 0x55cc967ca3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:48.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.919+0000 7f19f5400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc967d57e0 0x55cc967d7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:48.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.919+0000 7f19f6800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc967ca010 0x55cc967ca3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:44134/0 (socket says 172.21.15.1:44134) 2024-02-02T15:35:48.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.919+0000 7f19f5e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cc967ca930 0x55cc967d52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:48.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.919+0000 7f19f6800700 1 -- 172.21.15.1:0/607413233 learned_addr learned my addr 172.21.15.1:0/607413233 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:35:48.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:48 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/805197174' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:35:48.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.921+0000 7f19eea00700 1 --2- 172.21.15.1:0/512445045 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f19d00a0710 0x7f19d00a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:48.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.921+0000 7f19f5400700 1 -- 172.21.15.1:0/607413233 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc967ca010 msgr2=0x55cc967ca3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:48.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.921+0000 7f19f5400700 1 --2- 172.21.15.1:0/607413233 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc967ca010 0x55cc967ca3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:48.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.921+0000 7f19f5400700 1 -- 172.21.15.1:0/607413233 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cc967ca930 msgr2=0x55cc967d52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:48.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.921+0000 7f19f5400700 1 --2- 172.21.15.1:0/607413233 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cc967ca930 0x55cc967d52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:48.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.921+0000 7f19f5400700 1 -- 172.21.15.1:0/607413233 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc9659d3b0 con 0x55cc967d57e0 2024-02-02T15:35:48.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.921+0000 7f19eea00700 1 -- 172.21.15.1:0/512445045 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f19e00cbec0 con 0x7f19f0102030 2024-02-02T15:35:48.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.921+0000 7f19f6800700 1 --2- 172.21.15.1:0/512445045 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f19d00a0710 0x7f19d00a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:48.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.921+0000 7f19f5400700 1 --2- 172.21.15.1:0/607413233 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc967d57e0 0x55cc967d7bd0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f19d80098e0 tx=0x7f19d8005bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8ea4f893a71d9a6 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:48.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.921+0000 7f19e7000700 1 -- 172.21.15.1:0/607413233 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d801d030 con 0x55cc967d57e0 2024-02-02T15:35:48.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.921+0000 7f19e7000700 1 -- 172.21.15.1:0/607413233 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f19d801e040 con 0x55cc967d57e0 2024-02-02T15:35:48.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.921+0000 7f19e7000700 1 -- 172.21.15.1:0/607413233 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d8023460 con 0x55cc967d57e0 2024-02-02T15:35:48.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.921+0000 7f1a0c3f7b80 1 -- 172.21.15.1:0/607413233 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc967d57e0 msgr2=0x55cc967d7bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:48.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.921+0000 7f1a0c3f7b80 1 --2- 172.21.15.1:0/607413233 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc967d57e0 0x55cc967d7bd0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f19d80098e0 tx=0x7f19d8005bc0 comp rx=0 tx=0).stop 2024-02-02T15:35:48.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.921+0000 7f19f6800700 1 --2- 172.21.15.1:0/512445045 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f19d00a0710 0x7f19d00a2bd0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f19e8009fd0 tx=0x7f19e8009040 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:48.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.921+0000 7f1a0c3f7b80 1 -- 172.21.15.1:0/607413233 shutdown_connections 2024-02-02T15:35:48.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.921+0000 7f1a0c3f7b80 1 --2- 172.21.15.1:0/607413233 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc967d57e0 0x55cc967d7bd0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:48.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.921+0000 7f1a0c3f7b80 1 --2- 172.21.15.1:0/607413233 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc967ca010 0x55cc967ca3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:48.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.921+0000 7f1a0c3f7b80 1 --2- 172.21.15.1:0/607413233 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cc967ca930 0x55cc967d52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:48.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.921+0000 7f1a0c3f7b80 1 -- 172.21.15.1:0/607413233 >> 172.21.15.1:0/607413233 conn(0x55cc96729db0 msgr2=0x55cc967c63b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:48.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.921+0000 7f1a0c3f7b80 1 -- 172.21.15.1:0/607413233 shutdown_connections 2024-02-02T15:35:48.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.921+0000 7f1a0c3f7b80 1 -- 172.21.15.1:0/607413233 wait complete. 2024-02-02T15:35:48.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.922+0000 7f1a0c3f7b80 1 -- start start 2024-02-02T15:35:48.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.922+0000 7f1a0c3f7b80 1 Processor -- start 2024-02-02T15:35:48.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.923+0000 7f1a0c3f7b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc967ca010 0x55cc967d2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:48.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.923+0000 7f1a0c3f7b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cc967ca930 0x55cc967d2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:48.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.923+0000 7f1a0c3f7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc967d57e0 0x55cc967d2f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:48.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.923+0000 7f1a0c3f7b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc9671bd50 con 0x55cc967ca930 2024-02-02T15:35:48.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.923+0000 7f1a0c3f7b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc966b7d80 con 0x55cc967ca010 2024-02-02T15:35:48.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.923+0000 7f1a0c3f7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc966b7a20 con 0x55cc967d57e0 2024-02-02T15:35:48.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.923+0000 7f1a0c3f7b80 10 client.0 _ll_register_callbacks cb 0x55cc966b9290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:35:48.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.923+0000 7f19f5e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc967ca010 0x55cc967d2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:48.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.923+0000 7f19f5400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cc967ca930 0x55cc967d2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:48.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.923+0000 7f19f5e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc967ca010 0x55cc967d2510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:41996/0 (socket says 172.21.15.1:41996) 2024-02-02T15:35:48.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.924+0000 7f19f5e00700 1 -- 172.21.15.1:0/1822065110 learned_addr learned my addr 172.21.15.1:0/1822065110 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:35:48.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.924+0000 7f19f6800700 1 --2- 172.21.15.1:0/1822065110 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc967d57e0 0x55cc967d2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:48.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.924+0000 7f19f5e00700 1 -- 172.21.15.1:0/1822065110 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc967d57e0 msgr2=0x55cc967d2f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:48.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.924+0000 7f19f5e00700 1 --2- 172.21.15.1:0/1822065110 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc967d57e0 0x55cc967d2f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:48.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.924+0000 7f19f5e00700 1 -- 172.21.15.1:0/1822065110 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cc967ca930 msgr2=0x55cc967d2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:48.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.924+0000 7f19f5e00700 1 --2- 172.21.15.1:0/1822065110 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cc967ca930 0x55cc967d2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:48.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.924+0000 7f19f5e00700 1 -- 172.21.15.1:0/1822065110 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19f010e560 con 0x55cc967ca010 2024-02-02T15:35:48.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.924+0000 7f19f5400700 1 --2- 172.21.15.1:0/1822065110 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cc967ca930 0x55cc967d2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:35:48.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.924+0000 7f19f6800700 1 --2- 172.21.15.1:0/1822065110 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc967d57e0 0x55cc967d2f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:35:48.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.924+0000 7f19f5e00700 1 --2- 172.21.15.1:0/1822065110 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc967ca010 0x55cc967d2510 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f19e0007fd0 tx=0x7f19e0098c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82147dce16526e0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:48.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.924+0000 7f19df400700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:35:48.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.924+0000 7f19df400700 1 -- 172.21.15.1:0/1822065110 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e0099440 con 0x55cc967ca010 2024-02-02T15:35:48.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.924+0000 7f19df400700 1 -- 172.21.15.1:0/1822065110 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f19e0098fe0 con 0x55cc967ca010 2024-02-02T15:35:48.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.924+0000 7f1a0c3f7b80 1 -- 172.21.15.1:0/1822065110 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cc967be0f0 con 0x55cc967ca010 2024-02-02T15:35:48.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.925+0000 7f19df400700 1 -- 172.21.15.1:0/1822065110 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f19e009ce10 con 0x55cc967ca010 2024-02-02T15:35:48.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.925+0000 7f1a0c3f7b80 10 client.27962 fetch_fsmap learned FSMap version 43 2024-02-02T15:35:48.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.925+0000 7f1a0c3f7b80 1 -- 172.21.15.1:0/1822065110 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55cc96549b60 con 0x55cc967ca010 2024-02-02T15:35:48.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.925+0000 7f19df400700 1 -- 172.21.15.1:0/1822065110 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cc967be0f0 con 0x55cc967ca010 2024-02-02T15:35:48.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.925+0000 7f19df400700 1 -- 172.21.15.1:0/1822065110 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 43) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f19e00a0bc0 con 0x55cc967ca010 2024-02-02T15:35:48.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.925+0000 7f1a0c3f7b80 10 client.27962 fetch_fsmap finished waiting for FSMap version 43 2024-02-02T15:35:48.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.925+0000 7f1a0c3f7b80 10 client.27962 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-02-02T15:35:48.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.925+0000 7f1a0c3f7b80 20 client.27962 populate_metadata read hostname 'smithi001' 2024-02-02T15:35:48.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.925+0000 7f1a0c3f7b80 1 --2- 172.21.15.1:0/1822065110 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55cc967e51d0 0x55cc967e75f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:48.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.925+0000 7f1a0c3f7b80 4 client.27962 mds_command: new command op to 24514 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:35:48.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.925+0000 7f1a0c3f7b80 1 -- 172.21.15.1:0/1822065110 --> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cc96714dd0 con 0x55cc967e51d0 2024-02-02T15:35:48.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.925+0000 7f19f5400700 1 --2- 172.21.15.1:0/1822065110 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55cc967e51d0 0x55cc967e75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:48.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.926+0000 7f19f5400700 1 --2- 172.21.15.1:0/1822065110 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55cc967e51d0 0x55cc967e75f0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9aeca2f55fcd25ec server_cookie=456fcc83c7f9b655 in_seq=0 out_seq=0 2024-02-02T15:35:48.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.926+0000 7f19df400700 10 client.27962 ms_handle_connect on v2:172.21.15.142:6836/2870289019 2024-02-02T15:35:48.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.927+0000 7f19df400700 1 -- 172.21.15.1:0/1822065110 <== mds.3 v2:172.21.15.142:6836/2870289019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55cc96714dd0 con 0x55cc967e51d0 2024-02-02T15:35:48.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.927+0000 7f19df400700 10 client.27962 handle_command_reply: tid=0 2024-02-02T15:35:48.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.927+0000 7f1a0c3f7b80 1 client.27962 shutdown 2024-02-02T15:35:48.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.927+0000 7f1a0c3f7b80 1 -- 172.21.15.1:0/1822065110 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc967ca010 msgr2=0x55cc967d2510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:48.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.927+0000 7f1a0c3f7b80 1 --2- 172.21.15.1:0/1822065110 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc967ca010 0x55cc967d2510 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f19e0007fd0 tx=0x7f19e0098c50 comp rx=0 tx=0).stop 2024-02-02T15:35:48.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.927+0000 7f1a0c3f7b80 1 -- 172.21.15.1:0/1822065110 shutdown_connections 2024-02-02T15:35:48.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.927+0000 7f1a0c3f7b80 1 --2- 172.21.15.1:0/1822065110 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc967d57e0 0x55cc967d2f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:48.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.927+0000 7f1a0c3f7b80 1 --2- 172.21.15.1:0/1822065110 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc967ca010 0x55cc967d2510 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:48.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.927+0000 7f1a0c3f7b80 1 --2- 172.21.15.1:0/1822065110 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55cc967e51d0 0x55cc967e75f0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:48.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.927+0000 7f1a0c3f7b80 1 --2- 172.21.15.1:0/1822065110 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cc967ca930 0x55cc967d2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:48.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.927+0000 7f19df400700 0 client.27962 ms_handle_reset on v2:172.21.15.142:6836/2870289019 2024-02-02T15:35:48.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.927+0000 7f1a0c3f7b80 1 -- 172.21.15.1:0/1822065110 >> 172.21.15.1:0/1822065110 conn(0x55cc96729db0 msgr2=0x55cc967c5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:48.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.927+0000 7f1a0c3f7b80 1 -- 172.21.15.1:0/1822065110 shutdown_connections 2024-02-02T15:35:48.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.927+0000 7f1a0c3f7b80 1 -- 172.21.15.1:0/1822065110 wait complete. 2024-02-02T15:35:48.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.927+0000 7f1a0c3f7b80 20 client.27962 trim_cache size 0 max 16384 2024-02-02T15:35:48.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.941+0000 7f1a0c3f7b80 1 Processor -- start 2024-02-02T15:35:48.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.942+0000 7f1a0c3f7b80 1 -- start start 2024-02-02T15:35:48.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.942+0000 7f1a0c3f7b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc967ca010 0x55cc967dc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:48.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.942+0000 7f1a0c3f7b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cc967ca930 0x55cc967d2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:48.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.942+0000 7f1a0c3f7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc967d57e0 0x55cc967cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:48.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.942+0000 7f1a0c3f7b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc9659f6a0 con 0x55cc967ca930 2024-02-02T15:35:48.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.942+0000 7f1a0c3f7b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc967cb260 con 0x55cc967ca010 2024-02-02T15:35:48.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.942+0000 7f1a0c3f7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc967e1160 con 0x55cc967d57e0 2024-02-02T15:35:48.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.942+0000 7f19f5e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc967ca010 0x55cc967dc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:48.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.942+0000 7f19f5e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc967ca010 0x55cc967dc2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42000/0 (socket says 172.21.15.1:42000) 2024-02-02T15:35:48.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.942+0000 7f19f6800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc967d57e0 0x55cc967cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:48.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.942+0000 7f19f5e00700 1 -- 172.21.15.1:0/2251189210 learned_addr learned my addr 172.21.15.1:0/2251189210 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:35:48.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.942+0000 7f19f5400700 1 --2- 172.21.15.1:0/2251189210 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cc967ca930 0x55cc967d2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:48.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.943+0000 7f19f5e00700 1 -- 172.21.15.1:0/2251189210 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc967d57e0 msgr2=0x55cc967cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:48.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.943+0000 7f19f5e00700 1 --2- 172.21.15.1:0/2251189210 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc967d57e0 0x55cc967cf840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:48.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.943+0000 7f19f5e00700 1 -- 172.21.15.1:0/2251189210 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cc967ca930 msgr2=0x55cc967d2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:48.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.943+0000 7f19f5e00700 1 --2- 172.21.15.1:0/2251189210 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cc967ca930 0x55cc967d2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:48.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.943+0000 7f19f5e00700 1 -- 172.21.15.1:0/2251189210 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc966e1ea0 con 0x55cc967ca010 2024-02-02T15:35:48.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.943+0000 7f19f5e00700 1 --2- 172.21.15.1:0/2251189210 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc967ca010 0x55cc967dc2c0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f19e0007e80 tx=0x7f19e0000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c2b7b38649fe661 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:48.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.943+0000 7f19e7a00700 1 -- 172.21.15.1:0/2251189210 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e0098fe0 con 0x55cc967ca010 2024-02-02T15:35:48.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.943+0000 7f19e7a00700 1 -- 172.21.15.1:0/2251189210 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f19e009ce10 con 0x55cc967ca010 2024-02-02T15:35:48.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.943+0000 7f19e7a00700 1 -- 172.21.15.1:0/2251189210 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e0099240 con 0x55cc967ca010 2024-02-02T15:35:48.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.945+0000 7f1a0c3f7b80 1 -- 172.21.15.1:0/2251189210 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc967ca010 msgr2=0x55cc967dc2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:48.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.945+0000 7f1a0c3f7b80 1 --2- 172.21.15.1:0/2251189210 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc967ca010 0x55cc967dc2c0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f19e0007e80 tx=0x7f19e0000bc0 comp rx=0 tx=0).stop 2024-02-02T15:35:48.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.945+0000 7f1a0c3f7b80 1 -- 172.21.15.1:0/2251189210 shutdown_connections 2024-02-02T15:35:48.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.945+0000 7f1a0c3f7b80 1 --2- 172.21.15.1:0/2251189210 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc967d57e0 0x55cc967cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:48.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.945+0000 7f1a0c3f7b80 1 --2- 172.21.15.1:0/2251189210 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc967ca010 0x55cc967dc2c0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:48.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.945+0000 7f1a0c3f7b80 1 --2- 172.21.15.1:0/2251189210 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cc967ca930 0x55cc967d2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:48.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.945+0000 7f1a0c3f7b80 1 -- 172.21.15.1:0/2251189210 >> 172.21.15.1:0/2251189210 conn(0x55cc96729db0 msgr2=0x55cc967cd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:48.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.945+0000 7f1a0c3f7b80 1 -- 172.21.15.1:0/2251189210 shutdown_connections 2024-02-02T15:35:48.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.945+0000 7f1a0c3f7b80 1 -- 172.21.15.1:0/2251189210 wait complete. 2024-02-02T15:35:48.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.945+0000 7f1a0c3f7b80 1 -- start start 2024-02-02T15:35:48.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.945+0000 7f1a0c3f7b80 1 Processor -- start 2024-02-02T15:35:48.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.946+0000 7f1a0c3f7b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc967ca010 0x55cc967da4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:48.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.946+0000 7f1a0c3f7b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cc967ca930 0x55cc967da9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:48.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.946+0000 7f1a0c3f7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc967d57e0 0x55cc96835d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:48.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.946+0000 7f1a0c3f7b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc96836250 con 0x55cc967ca930 2024-02-02T15:35:48.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.946+0000 7f1a0c3f7b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc968363c0 con 0x55cc967ca010 2024-02-02T15:35:48.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.946+0000 7f1a0c3f7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc96836580 con 0x55cc967d57e0 2024-02-02T15:35:48.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.946+0000 7f1a0c3f7b80 10 client.0 _ll_register_callbacks cb 0x55cc966b9290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:35:48.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.946+0000 7f19f5e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc967ca010 0x55cc967da4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:48.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.946+0000 7f19f5e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc967ca010 0x55cc967da4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42010/0 (socket says 172.21.15.1:42010) 2024-02-02T15:35:48.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.946+0000 7f19f5e00700 1 -- 172.21.15.1:0/1788470932 learned_addr learned my addr 172.21.15.1:0/1788470932 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:35:48.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.946+0000 7f19f5400700 1 --2- 172.21.15.1:0/1788470932 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cc967ca930 0x55cc967da9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:48.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.946+0000 7f19f6800700 1 --2- 172.21.15.1:0/1788470932 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc967d57e0 0x55cc96835d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:48.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.947+0000 7f19f5e00700 1 -- 172.21.15.1:0/1788470932 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc967d57e0 msgr2=0x55cc96835d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:48.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.947+0000 7f19f5e00700 1 --2- 172.21.15.1:0/1788470932 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc967d57e0 0x55cc96835d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:48.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.947+0000 7f19f5e00700 1 -- 172.21.15.1:0/1788470932 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cc967ca930 msgr2=0x55cc967da9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:48.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.947+0000 7f19f5e00700 1 --2- 172.21.15.1:0/1788470932 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cc967ca930 0x55cc967da9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:48.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.947+0000 7f19f5e00700 1 -- 172.21.15.1:0/1788470932 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cc96549b60 con 0x55cc967ca010 2024-02-02T15:35:48.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.947+0000 7f19f5e00700 1 --2- 172.21.15.1:0/1788470932 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc967ca010 0x55cc967da4a0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f19e00c9810 tx=0x7f19e00a0df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30d9795ae7f75ab6 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:48.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.947+0000 7f19df400700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:35:48.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.947+0000 7f19df400700 1 -- 172.21.15.1:0/1788470932 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e00b1420 con 0x55cc967ca010 2024-02-02T15:35:48.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.947+0000 7f19df400700 1 -- 172.21.15.1:0/1788470932 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f19e0098fe0 con 0x55cc967ca010 2024-02-02T15:35:48.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.947+0000 7f1a0c3f7b80 1 -- 172.21.15.1:0/1788470932 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f19d801e040 con 0x55cc967ca010 2024-02-02T15:35:48.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.947+0000 7f19df400700 1 -- 172.21.15.1:0/1788470932 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f19e00b1580 con 0x55cc967ca010 2024-02-02T15:35:48.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.947+0000 7f1a0c3f7b80 10 client.27968 fetch_fsmap learned FSMap version 43 2024-02-02T15:35:48.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.947+0000 7f1a0c3f7b80 1 -- 172.21.15.1:0/1788470932 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55cc965a1cd0 con 0x55cc967ca010 2024-02-02T15:35:48.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.949+0000 7f19df400700 1 -- 172.21.15.1:0/1788470932 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f19e00a8080 con 0x55cc967ca010 2024-02-02T15:35:48.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.949+0000 7f19df400700 1 -- 172.21.15.1:0/1788470932 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 43) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f19e0098140 con 0x55cc967ca010 2024-02-02T15:35:48.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.949+0000 7f1a0c3f7b80 10 client.27968 fetch_fsmap finished waiting for FSMap version 43 2024-02-02T15:35:48.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.949+0000 7f1a0c3f7b80 10 client.27968 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-02-02T15:35:48.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.949+0000 7f1a0c3f7b80 20 client.27968 populate_metadata read hostname 'smithi001' 2024-02-02T15:35:48.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.949+0000 7f1a0c3f7b80 1 --2- 172.21.15.1:0/1788470932 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55cc968377b0 0x55cc96839ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:48.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.949+0000 7f1a0c3f7b80 4 client.27968 mds_command: new command op to 24514 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:35:48.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.949+0000 7f1a0c3f7b80 1 -- 172.21.15.1:0/1788470932 --> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55cc966daf50 con 0x55cc968377b0 2024-02-02T15:35:48.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.949+0000 7f19f5400700 1 --2- 172.21.15.1:0/1788470932 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55cc968377b0 0x55cc96839ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:48.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.949+0000 7f19df400700 10 client.27968 ms_handle_connect on v2:172.21.15.142:6836/2870289019 2024-02-02T15:35:48.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.949+0000 7f19f5400700 1 --2- 172.21.15.1:0/1788470932 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55cc968377b0 0x55cc96839ba0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=23148b97a75f8793 server_cookie=31140c26f2042dea in_seq=0 out_seq=0 2024-02-02T15:35:48.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.949+0000 7f19df400700 1 -- 172.21.15.1:0/1788470932 <== mds.3 v2:172.21.15.142:6836/2870289019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55cc966daf50 con 0x55cc968377b0 2024-02-02T15:35:48.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.949+0000 7f19df400700 10 client.27968 handle_command_reply: tid=0 2024-02-02T15:35:48.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.949+0000 7f1a0c3f7b80 1 client.27968 shutdown 2024-02-02T15:35:48.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.949+0000 7f1a0c3f7b80 1 -- 172.21.15.1:0/1788470932 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc967ca010 msgr2=0x55cc967da4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:48.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.949+0000 7f1a0c3f7b80 1 --2- 172.21.15.1:0/1788470932 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc967ca010 0x55cc967da4a0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f19e00c9810 tx=0x7f19e00a0df0 comp rx=0 tx=0).stop 2024-02-02T15:35:48.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.949+0000 7f1a0c3f7b80 1 -- 172.21.15.1:0/1788470932 shutdown_connections 2024-02-02T15:35:48.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.949+0000 7f1a0c3f7b80 1 --2- 172.21.15.1:0/1788470932 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cc967d57e0 0x55cc96835d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:48.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.949+0000 7f1a0c3f7b80 1 --2- 172.21.15.1:0/1788470932 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cc967ca010 0x55cc967da4a0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:48.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.949+0000 7f1a0c3f7b80 1 --2- 172.21.15.1:0/1788470932 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55cc968377b0 0x55cc96839ba0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:48.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.949+0000 7f1a0c3f7b80 1 --2- 172.21.15.1:0/1788470932 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cc967ca930 0x55cc967da9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:48.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.949+0000 7f1a0c3f7b80 1 -- 172.21.15.1:0/1788470932 >> 172.21.15.1:0/1788470932 conn(0x55cc96729db0 msgr2=0x55cc967d0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:48.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.949+0000 7f19df400700 0 client.27968 ms_handle_reset on v2:172.21.15.142:6836/2870289019 2024-02-02T15:35:48.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.949+0000 7f1a0c3f7b80 1 -- 172.21.15.1:0/1788470932 shutdown_connections 2024-02-02T15:35:48.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.949+0000 7f1a0c3f7b80 1 -- 172.21.15.1:0/1788470932 wait complete. 2024-02-02T15:35:48.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.951+0000 7f1a0c3f7b80 20 client.27968 trim_cache size 0 max 16384 2024-02-02T15:35:48.954 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:35:48.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.951+0000 7f19f8800700 1 -- 172.21.15.1:0/512445045 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f19d00a0710 msgr2=0x7f19d00a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:48.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.951+0000 7f19f8800700 1 --2- 172.21.15.1:0/512445045 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f19d00a0710 0x7f19d00a2bd0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f19e8009fd0 tx=0x7f19e8009040 comp rx=0 tx=0).stop 2024-02-02T15:35:48.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.951+0000 7f19f8800700 1 -- 172.21.15.1:0/512445045 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f19f0102030 msgr2=0x7f19f01990c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:48.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.951+0000 7f19f8800700 1 --2- 172.21.15.1:0/512445045 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f19f0102030 0x7f19f01990c0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f19e000d980 tx=0x7f19e000dc90 comp rx=0 tx=0).stop 2024-02-02T15:35:48.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.951+0000 7f19f8800700 1 -- 172.21.15.1:0/512445045 shutdown_connections 2024-02-02T15:35:48.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.951+0000 7f19f8800700 1 --2- 172.21.15.1:0/512445045 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f19d00a0710 0x7f19d00a2bd0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:48.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.951+0000 7f19f8800700 1 --2- 172.21.15.1:0/512445045 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f19f010b400 0x7f19f019da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:48.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.951+0000 7f19f8800700 1 --2- 172.21.15.1:0/512445045 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f19f0102030 0x7f19f01990c0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:48.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.951+0000 7f19f8800700 1 --2- 172.21.15.1:0/512445045 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f19f0102950 0x7f19f0199600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:48.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.951+0000 7f19f8800700 1 -- 172.21.15.1:0/512445045 >> 172.21.15.1:0/512445045 conn(0x7f19f0076d70 msgr2=0x7f19f00fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:48.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.951+0000 7f19f8800700 1 -- 172.21.15.1:0/512445045 shutdown_connections 2024-02-02T15:35:48.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:48.951+0000 7f19f8800700 1 -- 172.21.15.1:0/512445045 wait complete. 2024-02-02T15:35:48.968 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-02T15:35:49.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.190+0000 7fde52400700 1 -- 172.21.15.1:0/564926510 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde4c103230 msgr2=0x7fde4c103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:49.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.190+0000 7fde52400700 1 --2- 172.21.15.1:0/564926510 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde4c103230 0x7fde4c103610 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fde3c00b0a0 tx=0x7fde3c00b3b0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.191+0000 7fde52400700 1 -- 172.21.15.1:0/564926510 shutdown_connections 2024-02-02T15:35:49.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.191+0000 7fde52400700 1 --2- 172.21.15.1:0/564926510 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde4c108350 0x7fde4c10fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.191+0000 7fde52400700 1 --2- 172.21.15.1:0/564926510 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde4c103230 0x7fde4c103610 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.191+0000 7fde52400700 1 --2- 172.21.15.1:0/564926510 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde4c103be0 0x7fde4c107c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.191+0000 7fde52400700 1 -- 172.21.15.1:0/564926510 >> 172.21.15.1:0/564926510 conn(0x7fde4c0fea70 msgr2=0x7fde4c100e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:49.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.191+0000 7fde52400700 1 -- 172.21.15.1:0/564926510 shutdown_connections 2024-02-02T15:35:49.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.191+0000 7fde52400700 1 -- 172.21.15.1:0/564926510 wait complete. 2024-02-02T15:35:49.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.192+0000 7fde52400700 1 Processor -- start 2024-02-02T15:35:49.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.192+0000 7fde52400700 1 -- start start 2024-02-02T15:35:49.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.192+0000 7fde52400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde4c103230 0x7fde4c199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:49.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.192+0000 7fde52400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde4c108350 0x7fde4c199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:49.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.192+0000 7fde52400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde4c19dad0 0x7fde4c19df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:49.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.192+0000 7fde52400700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde4c19e550 con 0x7fde4c103230 2024-02-02T15:35:49.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.192+0000 7fde52400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde4c19e6c0 con 0x7fde4c19dad0 2024-02-02T15:35:49.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.192+0000 7fde52400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde4c19e880 con 0x7fde4c108350 2024-02-02T15:35:49.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.193+0000 7fde4b400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde4c103230 0x7fde4c199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:49.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.193+0000 7fde4be00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde4c19dad0 0x7fde4c19df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:49.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.193+0000 7fde42800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde4c108350 0x7fde4c199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:49.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.193+0000 7fde4b400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde4c103230 0x7fde4c199130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49246/0 (socket says 172.21.15.1:49246) 2024-02-02T15:35:49.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.193+0000 7fde4be00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde4c19dad0 0x7fde4c19df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42028/0 (socket says 172.21.15.1:42028) 2024-02-02T15:35:49.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.193+0000 7fde4b400700 1 -- 172.21.15.1:0/598488782 learned_addr learned my addr 172.21.15.1:0/598488782 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:35:49.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.193+0000 7fde4be00700 1 -- 172.21.15.1:0/598488782 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde4c108350 msgr2=0x7fde4c199670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:49.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.193+0000 7fde4be00700 1 --2- 172.21.15.1:0/598488782 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde4c108350 0x7fde4c199670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.193+0000 7fde4be00700 1 -- 172.21.15.1:0/598488782 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde4c103230 msgr2=0x7fde4c199130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:49.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.193+0000 7fde4be00700 1 --2- 172.21.15.1:0/598488782 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde4c103230 0x7fde4c199130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.193+0000 7fde4be00700 1 -- 172.21.15.1:0/598488782 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde44009520 con 0x7fde4c19dad0 2024-02-02T15:35:49.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.193+0000 7fde42800700 1 --2- 172.21.15.1:0/598488782 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde4c108350 0x7fde4c199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:35:49.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.193+0000 7fde4be00700 1 --2- 172.21.15.1:0/598488782 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde4c19dad0 0x7fde4c19df50 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fde4400e900 tx=0x7fde4400ec10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:49.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.194+0000 7fde48c00700 1 -- 172.21.15.1:0/598488782 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde4400cbd0 con 0x7fde4c19dad0 2024-02-02T15:35:49.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.194+0000 7fde48c00700 1 -- 172.21.15.1:0/598488782 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde4400ee50 con 0x7fde4c19dad0 2024-02-02T15:35:49.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.194+0000 7fde48c00700 1 -- 172.21.15.1:0/598488782 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde44003d30 con 0x7fde4c19dad0 2024-02-02T15:35:49.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.194+0000 7fde52400700 1 -- 172.21.15.1:0/598488782 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde3c009d00 con 0x7fde4c19dad0 2024-02-02T15:35:49.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.194+0000 7fde52400700 1 -- 172.21.15.1:0/598488782 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde4c1a60e0 con 0x7fde4c19dad0 2024-02-02T15:35:49.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.197+0000 7fde48c00700 1 -- 172.21.15.1:0/598488782 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7fde4400cd30 con 0x7fde4c19dad0 2024-02-02T15:35:49.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.198+0000 7fde48c00700 1 --2- 172.21.15.1:0/598488782 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fde2c0a0710 0x7fde2c0a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:49.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.198+0000 7fde48c00700 1 -- 172.21.15.1:0/598488782 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fde44014070 con 0x7fde4c19dad0 2024-02-02T15:35:49.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.199+0000 7fde65f2eb80 1 Processor -- start 2024-02-02T15:35:49.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.199+0000 7fde4b400700 1 --2- 172.21.15.1:0/598488782 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fde2c0a0710 0x7fde2c0a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:49.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.199+0000 7fde65f2eb80 1 -- start start 2024-02-02T15:35:49.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.199+0000 7fde65f2eb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55645c785010 0x55645c7853f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:49.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.199+0000 7fde65f2eb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55645c785930 0x55645c7902a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:49.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.199+0000 7fde65f2eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55645c7907e0 0x55645c792bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:49.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.199+0000 7fde65f2eb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55645c6f94c0 con 0x55645c785010 2024-02-02T15:35:49.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.199+0000 7fde65f2eb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55645c6ddaa0 con 0x55645c785930 2024-02-02T15:35:49.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.199+0000 7fde65f2eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55645c6db460 con 0x55645c7907e0 2024-02-02T15:35:49.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.200+0000 7fde42800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55645c785930 0x55645c7902a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:49.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.200+0000 7fde42800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55645c785930 0x55645c7902a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42038/0 (socket says 172.21.15.1:42038) 2024-02-02T15:35:49.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.200+0000 7fde42800700 1 -- 172.21.15.1:0/4246952686 learned_addr learned my addr 172.21.15.1:0/4246952686 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:35:49.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.200+0000 7fde4b400700 1 --2- 172.21.15.1:0/598488782 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fde2c0a0710 0x7fde2c0a2bd0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fde3c00b780 tx=0x7fde3c009570 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:49.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.200+0000 7fde4be00700 1 --2- 172.21.15.1:0/4246952686 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55645c7907e0 0x55645c792bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:49.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.200+0000 7fde4b400700 1 --2- 172.21.15.1:0/4246952686 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55645c785010 0x55645c7853f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:49.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.200+0000 7fde42800700 1 -- 172.21.15.1:0/4246952686 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55645c7907e0 msgr2=0x55645c792bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:49.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.200+0000 7fde42800700 1 --2- 172.21.15.1:0/4246952686 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55645c7907e0 0x55645c792bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.200+0000 7fde42800700 1 -- 172.21.15.1:0/4246952686 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55645c785010 msgr2=0x55645c7853f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:49.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.200+0000 7fde42800700 1 --2- 172.21.15.1:0/4246952686 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55645c785010 0x55645c7853f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.200+0000 7fde42800700 1 -- 172.21.15.1:0/4246952686 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55645c5583b0 con 0x55645c785930 2024-02-02T15:35:49.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.200+0000 7fde4be00700 1 --2- 172.21.15.1:0/4246952686 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55645c7907e0 0x55645c792bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:35:49.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.200+0000 7fde42800700 1 --2- 172.21.15.1:0/4246952686 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55645c785930 0x55645c7902a0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fde34009d70 tx=0x7fde340116c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a3a53198d62c07c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:49.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.201+0000 7fde40a00700 1 -- 172.21.15.1:0/4246952686 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde34011e20 con 0x55645c785930 2024-02-02T15:35:49.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.201+0000 7fde40a00700 1 -- 172.21.15.1:0/4246952686 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde340044e0 con 0x55645c785930 2024-02-02T15:35:49.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.201+0000 7fde40a00700 1 -- 172.21.15.1:0/4246952686 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde34012450 con 0x55645c785930 2024-02-02T15:35:49.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.201+0000 7fde65f2eb80 1 -- 172.21.15.1:0/4246952686 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55645c785930 msgr2=0x55645c7902a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:49.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.201+0000 7fde65f2eb80 1 --2- 172.21.15.1:0/4246952686 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55645c785930 0x55645c7902a0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fde34009d70 tx=0x7fde340116c0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.201+0000 7fde65f2eb80 1 -- 172.21.15.1:0/4246952686 shutdown_connections 2024-02-02T15:35:49.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.201+0000 7fde65f2eb80 1 --2- 172.21.15.1:0/4246952686 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55645c7907e0 0x55645c792bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.202+0000 7fde65f2eb80 1 --2- 172.21.15.1:0/4246952686 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55645c785010 0x55645c7853f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.202+0000 7fde65f2eb80 1 --2- 172.21.15.1:0/4246952686 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55645c785930 0x55645c7902a0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.202+0000 7fde65f2eb80 1 -- 172.21.15.1:0/4246952686 >> 172.21.15.1:0/4246952686 conn(0x55645c6e4db0 msgr2=0x55645c7813b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:49.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.202+0000 7fde65f2eb80 1 -- 172.21.15.1:0/4246952686 shutdown_connections 2024-02-02T15:35:49.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.202+0000 7fde65f2eb80 1 -- 172.21.15.1:0/4246952686 wait complete. 2024-02-02T15:35:49.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.202+0000 7fde65f2eb80 1 -- start start 2024-02-02T15:35:49.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.202+0000 7fde65f2eb80 1 Processor -- start 2024-02-02T15:35:49.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.203+0000 7fde65f2eb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55645c785010 0x55645c7901c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:49.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.203+0000 7fde65f2eb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55645c785930 0x55645c78d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:49.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.203+0000 7fde65f2eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55645c7907e0 0x55645c78d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:49.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.203+0000 7fde65f2eb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55645c6d6d50 con 0x55645c785010 2024-02-02T15:35:49.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.203+0000 7fde65f2eb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55645c672d80 con 0x55645c785930 2024-02-02T15:35:49.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.203+0000 7fde65f2eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55645c672a20 con 0x55645c7907e0 2024-02-02T15:35:49.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.203+0000 7fde65f2eb80 10 client.0 _ll_register_callbacks cb 0x55645c674290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:35:49.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.203+0000 7fde42800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55645c785930 0x55645c78d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:49.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.203+0000 7fde4b400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55645c785010 0x55645c7901c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:49.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.203+0000 7fde42800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55645c785930 0x55645c78d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42052/0 (socket says 172.21.15.1:42052) 2024-02-02T15:35:49.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.203+0000 7fde4be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55645c7907e0 0x55645c78d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:49.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.203+0000 7fde42800700 1 -- 172.21.15.1:0/731074816 learned_addr learned my addr 172.21.15.1:0/731074816 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:35:49.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.203+0000 7fde42800700 1 -- 172.21.15.1:0/731074816 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55645c7907e0 msgr2=0x55645c78d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:49.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.203+0000 7fde42800700 1 --2- 172.21.15.1:0/731074816 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55645c7907e0 0x55645c78d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.203+0000 7fde42800700 1 -- 172.21.15.1:0/731074816 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55645c785010 msgr2=0x55645c7901c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:49.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.204+0000 7fde42800700 1 --2- 172.21.15.1:0/731074816 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55645c785010 0x55645c7901c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.204+0000 7fde42800700 1 -- 172.21.15.1:0/731074816 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde4c1a60e0 con 0x55645c785930 2024-02-02T15:35:49.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.204+0000 7fde42800700 1 --2- 172.21.15.1:0/731074816 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55645c785930 0x55645c78d3d0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fde34003950 tx=0x7fde34003980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca3382c4ced1ba32 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:49.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.204+0000 7fde39600700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:35:49.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.204+0000 7fde39600700 1 -- 172.21.15.1:0/731074816 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde34003b90 con 0x55645c785930 2024-02-02T15:35:49.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.204+0000 7fde65f2eb80 1 -- 172.21.15.1:0/731074816 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55645c7790f0 con 0x55645c785930 2024-02-02T15:35:49.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.204+0000 7fde39600700 1 -- 172.21.15.1:0/731074816 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fde34003cf0 con 0x55645c785930 2024-02-02T15:35:49.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.204+0000 7fde39600700 1 -- 172.21.15.1:0/731074816 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fde34003eb0 con 0x55645c785930 2024-02-02T15:35:49.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.204+0000 7fde65f2eb80 10 client.27980 fetch_fsmap learned FSMap version 43 2024-02-02T15:35:49.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.204+0000 7fde65f2eb80 1 -- 172.21.15.1:0/731074816 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x7fde34009970 con 0x55645c785930 2024-02-02T15:35:49.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.205+0000 7fde39600700 1 -- 172.21.15.1:0/731074816 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55645c7790f0 con 0x55645c785930 2024-02-02T15:35:49.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.205+0000 7fde39600700 1 -- 172.21.15.1:0/731074816 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 43) v1 ==== 7506+0+0 (secure 0 0 0) 0x7fde3401a400 con 0x55645c785930 2024-02-02T15:35:49.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.205+0000 7fde65f2eb80 10 client.27980 fetch_fsmap finished waiting for FSMap version 43 2024-02-02T15:35:49.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.205+0000 7fde65f2eb80 10 client.27980 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-02-02T15:35:49.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.205+0000 7fde65f2eb80 20 client.27980 populate_metadata read hostname 'smithi001' 2024-02-02T15:35:49.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.205+0000 7fde65f2eb80 1 --2- 172.21.15.1:0/731074816 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55645c7a00c0 0x55645c7a24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:49.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.205+0000 7fde65f2eb80 4 client.27980 mds_command: new command op to 24518 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:35:49.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.205+0000 7fde65f2eb80 1 -- 172.21.15.1:0/731074816 --> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55645c6cfdd0 con 0x55645c7a00c0 2024-02-02T15:35:49.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.205+0000 7fde4b400700 1 --2- 172.21.15.1:0/731074816 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55645c7a00c0 0x55645c7a24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:49.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.206+0000 7fde4b400700 1 --2- 172.21.15.1:0/731074816 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55645c7a00c0 0x55645c7a24e0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=10afd730cdf299ef server_cookie=eb2001d2583301aa in_seq=0 out_seq=0 2024-02-02T15:35:49.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.206+0000 7fde39600700 10 client.27980 ms_handle_connect on v2:172.21.15.71:6836/628016199 2024-02-02T15:35:49.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.206+0000 7fde39600700 1 -- 172.21.15.1:0/731074816 <== mds.2 v2:172.21.15.71:6836/628016199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55645c6cfdd0 con 0x55645c7a00c0 2024-02-02T15:35:49.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.207+0000 7fde39600700 10 client.27980 handle_command_reply: tid=0 2024-02-02T15:35:49.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.207+0000 7fde65f2eb80 1 client.27980 shutdown 2024-02-02T15:35:49.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.207+0000 7fde65f2eb80 1 -- 172.21.15.1:0/731074816 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55645c785930 msgr2=0x55645c78d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:49.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.207+0000 7fde65f2eb80 1 --2- 172.21.15.1:0/731074816 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55645c785930 0x55645c78d3d0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fde34003950 tx=0x7fde34003980 comp rx=0 tx=0).stop 2024-02-02T15:35:49.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.207+0000 7fde65f2eb80 1 -- 172.21.15.1:0/731074816 shutdown_connections 2024-02-02T15:35:49.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.207+0000 7fde65f2eb80 1 --2- 172.21.15.1:0/731074816 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55645c7a00c0 0x55645c7a24e0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.207+0000 7fde65f2eb80 1 --2- 172.21.15.1:0/731074816 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55645c7907e0 0x55645c78d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.207+0000 7fde65f2eb80 1 --2- 172.21.15.1:0/731074816 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55645c785010 0x55645c7901c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.207+0000 7fde65f2eb80 1 --2- 172.21.15.1:0/731074816 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55645c785930 0x55645c78d3d0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.207+0000 7fde65f2eb80 1 -- 172.21.15.1:0/731074816 >> 172.21.15.1:0/731074816 conn(0x55645c6e4db0 msgr2=0x55645c780fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:49.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.207+0000 7fde39600700 0 client.27980 ms_handle_reset on v2:172.21.15.71:6836/628016199 2024-02-02T15:35:49.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.207+0000 7fde65f2eb80 1 -- 172.21.15.1:0/731074816 shutdown_connections 2024-02-02T15:35:49.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.207+0000 7fde65f2eb80 1 -- 172.21.15.1:0/731074816 wait complete. 2024-02-02T15:35:49.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.207+0000 7fde65f2eb80 20 client.27980 trim_cache size 0 max 16384 2024-02-02T15:35:49.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.225+0000 7fde65f2eb80 1 Processor -- start 2024-02-02T15:35:49.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.225+0000 7fde65f2eb80 1 -- start start 2024-02-02T15:35:49.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.226+0000 7fde65f2eb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55645c785010 0x55645c792bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:49.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.226+0000 7fde65f2eb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55645c785930 0x55645c78fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:49.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.226+0000 7fde65f2eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55645c7907e0 0x55645c79b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:49.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.226+0000 7fde65f2eb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55645c55a6a0 con 0x55645c785010 2024-02-02T15:35:49.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.226+0000 7fde65f2eb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55645c7892f0 con 0x55645c785930 2024-02-02T15:35:49.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.226+0000 7fde65f2eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55645c781fd0 con 0x55645c7907e0 2024-02-02T15:35:49.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.226+0000 7fde4b400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55645c785010 0x55645c792bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:49.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.226+0000 7fde42800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55645c785930 0x55645c78fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:49.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.226+0000 7fde4be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55645c7907e0 0x55645c79b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:49.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.226+0000 7fde42800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55645c785930 0x55645c78fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42060/0 (socket says 172.21.15.1:42060) 2024-02-02T15:35:49.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.226+0000 7fde42800700 1 -- 172.21.15.1:0/573431380 learned_addr learned my addr 172.21.15.1:0/573431380 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:35:49.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.226+0000 7fde4be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55645c7907e0 0x55645c79b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:44202/0 (socket says 172.21.15.1:44202) 2024-02-02T15:35:49.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.227+0000 7fde42800700 1 -- 172.21.15.1:0/573431380 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55645c7907e0 msgr2=0x55645c79b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:49.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.227+0000 7fde42800700 1 --2- 172.21.15.1:0/573431380 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55645c7907e0 0x55645c79b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.227+0000 7fde42800700 1 -- 172.21.15.1:0/573431380 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55645c785010 msgr2=0x55645c792bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:49.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.227+0000 7fde42800700 1 --2- 172.21.15.1:0/573431380 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55645c785010 0x55645c792bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.227+0000 7fde42800700 1 -- 172.21.15.1:0/573431380 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55645c504b60 con 0x55645c785930 2024-02-02T15:35:49.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.227+0000 7fde4b400700 1 --2- 172.21.15.1:0/573431380 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55645c785010 0x55645c792bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:35:49.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.227+0000 7fde4be00700 1 --2- 172.21.15.1:0/573431380 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55645c7907e0 0x55645c79b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:35:49.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.227+0000 7fde42800700 1 --2- 172.21.15.1:0/573431380 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55645c785930 0x55645c78fcf0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fde34009c60 tx=0x7fde34005b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=101e78fe40b98741 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:49.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.227+0000 7fde41400700 1 -- 172.21.15.1:0/573431380 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde34005940 con 0x55645c785930 2024-02-02T15:35:49.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.227+0000 7fde41400700 1 -- 172.21.15.1:0/573431380 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde34009260 con 0x55645c785930 2024-02-02T15:35:49.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.227+0000 7fde41400700 1 -- 172.21.15.1:0/573431380 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde340052a0 con 0x55645c785930 2024-02-02T15:35:49.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.228+0000 7fde65f2eb80 1 -- 172.21.15.1:0/573431380 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55645c785930 msgr2=0x55645c78fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:49.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.228+0000 7fde65f2eb80 1 --2- 172.21.15.1:0/573431380 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55645c785930 0x55645c78fcf0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fde34009c60 tx=0x7fde34005b80 comp rx=0 tx=0).stop 2024-02-02T15:35:49.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.228+0000 7fde65f2eb80 1 -- 172.21.15.1:0/573431380 shutdown_connections 2024-02-02T15:35:49.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.228+0000 7fde65f2eb80 1 --2- 172.21.15.1:0/573431380 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55645c7907e0 0x55645c79b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.228+0000 7fde65f2eb80 1 --2- 172.21.15.1:0/573431380 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55645c785010 0x55645c792bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.228+0000 7fde65f2eb80 1 --2- 172.21.15.1:0/573431380 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55645c785930 0x55645c78fcf0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.228+0000 7fde65f2eb80 1 -- 172.21.15.1:0/573431380 >> 172.21.15.1:0/573431380 conn(0x55645c6e4db0 msgr2=0x55645c79c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:49.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.228+0000 7fde65f2eb80 1 -- 172.21.15.1:0/573431380 shutdown_connections 2024-02-02T15:35:49.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.228+0000 7fde65f2eb80 1 -- 172.21.15.1:0/573431380 wait complete. 2024-02-02T15:35:49.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.229+0000 7fde65f2eb80 1 -- start start 2024-02-02T15:35:49.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.229+0000 7fde65f2eb80 1 Processor -- start 2024-02-02T15:35:49.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.229+0000 7fde65f2eb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55645c785010 0x55645c7934f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:49.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.229+0000 7fde65f2eb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55645c785930 0x55645c793a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:49.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.229+0000 7fde65f2eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55645c7907e0 0x55645c7f0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:49.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.229+0000 7fde65f2eb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55645c7940f0 con 0x55645c785010 2024-02-02T15:35:49.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.229+0000 7fde65f2eb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55645c7f1110 con 0x55645c785930 2024-02-02T15:35:49.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.229+0000 7fde65f2eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55645c7f1250 con 0x55645c7907e0 2024-02-02T15:35:49.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.229+0000 7fde65f2eb80 10 client.0 _ll_register_callbacks cb 0x55645c674290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:35:49.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.230+0000 7fde42800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55645c785930 0x55645c793a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:49.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.230+0000 7fde42800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55645c785930 0x55645c793a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42072/0 (socket says 172.21.15.1:42072) 2024-02-02T15:35:49.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.230+0000 7fde42800700 1 -- 172.21.15.1:0/1854601072 learned_addr learned my addr 172.21.15.1:0/1854601072 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:35:49.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.230+0000 7fde4b400700 1 --2- 172.21.15.1:0/1854601072 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55645c785010 0x55645c7934f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:49.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.230+0000 7fde4be00700 1 --2- 172.21.15.1:0/1854601072 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55645c7907e0 0x55645c7f0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:49.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.230+0000 7fde42800700 1 -- 172.21.15.1:0/1854601072 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55645c7907e0 msgr2=0x55645c7f0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:49.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.230+0000 7fde42800700 1 --2- 172.21.15.1:0/1854601072 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55645c7907e0 0x55645c7f0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.230+0000 7fde42800700 1 -- 172.21.15.1:0/1854601072 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55645c785010 msgr2=0x55645c7934f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:49.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.230+0000 7fde42800700 1 --2- 172.21.15.1:0/1854601072 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55645c785010 0x55645c7934f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.230+0000 7fde42800700 1 -- 172.21.15.1:0/1854601072 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde4c110740 con 0x55645c785930 2024-02-02T15:35:49.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.230+0000 7fde42800700 1 --2- 172.21.15.1:0/1854601072 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55645c785930 0x55645c793a30 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fde34005c10 tx=0x7fde34009f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=885934a60fbc8163 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:49.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.230+0000 7fde39600700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:35:49.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.230+0000 7fde39600700 1 -- 172.21.15.1:0/1854601072 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde34004310 con 0x55645c785930 2024-02-02T15:35:49.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.230+0000 7fde39600700 1 -- 172.21.15.1:0/1854601072 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fde34003aa0 con 0x55645c785930 2024-02-02T15:35:49.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.230+0000 7fde65f2eb80 1 -- 172.21.15.1:0/1854601072 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fde340044e0 con 0x55645c785930 2024-02-02T15:35:49.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.231+0000 7fde39600700 1 -- 172.21.15.1:0/1854601072 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fde34003cc0 con 0x55645c785930 2024-02-02T15:35:49.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.231+0000 7fde65f2eb80 10 client.27986 fetch_fsmap learned FSMap version 43 2024-02-02T15:35:49.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.231+0000 7fde65f2eb80 1 -- 172.21.15.1:0/1854601072 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x7fde3c009d00 con 0x55645c785930 2024-02-02T15:35:49.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.231+0000 7fde39600700 1 -- 172.21.15.1:0/1854601072 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fde340044e0 con 0x55645c785930 2024-02-02T15:35:49.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.231+0000 7fde39600700 1 -- 172.21.15.1:0/1854601072 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 43) v1 ==== 7506+0+0 (secure 0 0 0) 0x7fde3403f6d0 con 0x55645c785930 2024-02-02T15:35:49.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.231+0000 7fde65f2eb80 10 client.27986 fetch_fsmap finished waiting for FSMap version 43 2024-02-02T15:35:49.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.231+0000 7fde65f2eb80 10 client.27986 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-02-02T15:35:49.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.231+0000 7fde65f2eb80 20 client.27986 populate_metadata read hostname 'smithi001' 2024-02-02T15:35:49.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.231+0000 7fde65f2eb80 1 --2- 172.21.15.1:0/1854601072 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55645c7f2450 0x55645c7f4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:49.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.231+0000 7fde65f2eb80 4 client.27986 mds_command: new command op to 24518 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:35:49.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.231+0000 7fde65f2eb80 1 -- 172.21.15.1:0/1854601072 --> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55645c695f50 con 0x55645c7f2450 2024-02-02T15:35:49.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.231+0000 7fde4b400700 1 --2- 172.21.15.1:0/1854601072 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55645c7f2450 0x55645c7f4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:49.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.232+0000 7fde4b400700 1 --2- 172.21.15.1:0/1854601072 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55645c7f2450 0x55645c7f4870 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d6d9c0ff08b7b395 server_cookie=127016bf300aa4c6 in_seq=0 out_seq=0 2024-02-02T15:35:49.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.232+0000 7fde39600700 10 client.27986 ms_handle_connect on v2:172.21.15.71:6836/628016199 2024-02-02T15:35:49.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.232+0000 7fde39600700 1 -- 172.21.15.1:0/1854601072 <== mds.2 v2:172.21.15.71:6836/628016199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55645c695f50 con 0x55645c7f2450 2024-02-02T15:35:49.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.232+0000 7fde39600700 10 client.27986 handle_command_reply: tid=0 2024-02-02T15:35:49.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.232+0000 7fde65f2eb80 1 client.27986 shutdown 2024-02-02T15:35:49.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.232+0000 7fde65f2eb80 1 -- 172.21.15.1:0/1854601072 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55645c785930 msgr2=0x55645c793a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:49.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.232+0000 7fde65f2eb80 1 --2- 172.21.15.1:0/1854601072 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55645c785930 0x55645c793a30 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fde34005c10 tx=0x7fde34009f40 comp rx=0 tx=0).stop 2024-02-02T15:35:49.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.232+0000 7fde65f2eb80 1 -- 172.21.15.1:0/1854601072 shutdown_connections 2024-02-02T15:35:49.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.232+0000 7fde65f2eb80 1 --2- 172.21.15.1:0/1854601072 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55645c7f2450 0x55645c7f4870 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.232+0000 7fde65f2eb80 1 --2- 172.21.15.1:0/1854601072 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55645c7907e0 0x55645c7f0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.232+0000 7fde39600700 0 client.27986 ms_handle_reset on v2:172.21.15.71:6836/628016199 2024-02-02T15:35:49.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.232+0000 7fde65f2eb80 1 --2- 172.21.15.1:0/1854601072 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55645c785010 0x55645c7934f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.232+0000 7fde65f2eb80 1 --2- 172.21.15.1:0/1854601072 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55645c785930 0x55645c793a30 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.232+0000 7fde65f2eb80 1 -- 172.21.15.1:0/1854601072 >> 172.21.15.1:0/1854601072 conn(0x55645c6e4db0 msgr2=0x55645c79b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:49.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.233+0000 7fde65f2eb80 1 -- 172.21.15.1:0/1854601072 shutdown_connections 2024-02-02T15:35:49.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.233+0000 7fde65f2eb80 1 -- 172.21.15.1:0/1854601072 wait complete. 2024-02-02T15:35:49.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.233+0000 7fde65f2eb80 20 client.27986 trim_cache size 0 max 16384 2024-02-02T15:35:49.237 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:35:49.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.233+0000 7fde52400700 1 -- 172.21.15.1:0/598488782 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fde2c0a0710 msgr2=0x7fde2c0a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:49.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.233+0000 7fde52400700 1 --2- 172.21.15.1:0/598488782 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fde2c0a0710 0x7fde2c0a2bd0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fde3c00b780 tx=0x7fde3c009570 comp rx=0 tx=0).stop 2024-02-02T15:35:49.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.233+0000 7fde52400700 1 -- 172.21.15.1:0/598488782 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde4c19dad0 msgr2=0x7fde4c19df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:49.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.233+0000 7fde52400700 1 --2- 172.21.15.1:0/598488782 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde4c19dad0 0x7fde4c19df50 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fde4400e900 tx=0x7fde4400ec10 comp rx=0 tx=0).stop 2024-02-02T15:35:49.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.233+0000 7fde52400700 1 -- 172.21.15.1:0/598488782 shutdown_connections 2024-02-02T15:35:49.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.233+0000 7fde52400700 1 --2- 172.21.15.1:0/598488782 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fde2c0a0710 0x7fde2c0a2bd0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.233+0000 7fde52400700 1 --2- 172.21.15.1:0/598488782 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde4c19dad0 0x7fde4c19df50 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.233+0000 7fde52400700 1 --2- 172.21.15.1:0/598488782 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fde4c103230 0x7fde4c199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.233+0000 7fde52400700 1 --2- 172.21.15.1:0/598488782 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fde4c108350 0x7fde4c199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.233+0000 7fde52400700 1 -- 172.21.15.1:0/598488782 >> 172.21.15.1:0/598488782 conn(0x7fde4c0fea70 msgr2=0x7fde4c106c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:49.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.234+0000 7fde52400700 1 -- 172.21.15.1:0/598488782 shutdown_connections 2024-02-02T15:35:49.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.234+0000 7fde52400700 1 -- 172.21.15.1:0/598488782 wait complete. 2024-02-02T15:35:49.252 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-02T15:35:49.253 INFO:tasks.workunit.client.0.smithi001.stdout: 1 521142 6.40 MB/sec execute 323 sec latency 23.632 ms 2024-02-02T15:35:49.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.466+0000 7f80d1c00700 1 -- 172.21.15.1:0/3485230485 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f80cc069410 msgr2=0x7f80cc0697f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:49.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.466+0000 7f80d1c00700 1 --2- 172.21.15.1:0/3485230485 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f80cc069410 0x7f80cc0697f0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f80bc009990 tx=0x7f80bc009ca0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.466+0000 7f80d1c00700 1 -- 172.21.15.1:0/3485230485 shutdown_connections 2024-02-02T15:35:49.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.466+0000 7f80d1c00700 1 --2- 172.21.15.1:0/3485230485 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f80cc106150 0x7f80cc10dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.466+0000 7f80d1c00700 1 --2- 172.21.15.1:0/3485230485 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f80cc069410 0x7f80cc0697f0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.466+0000 7f80d1c00700 1 --2- 172.21.15.1:0/3485230485 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f80cc069dc0 0x7f80cc105a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.466+0000 7f80d1c00700 1 -- 172.21.15.1:0/3485230485 >> 172.21.15.1:0/3485230485 conn(0x7f80cc076d60 msgr2=0x7f80cc077170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:49.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.466+0000 7f80d1c00700 1 -- 172.21.15.1:0/3485230485 shutdown_connections 2024-02-02T15:35:49.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.466+0000 7f80d1c00700 1 -- 172.21.15.1:0/3485230485 wait complete. 2024-02-02T15:35:49.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.468+0000 7f80d1c00700 1 Processor -- start 2024-02-02T15:35:49.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.468+0000 7f80d1c00700 1 -- start start 2024-02-02T15:35:49.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.469+0000 7f80d1c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f80cc069410 0x7f80cc0ffc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:49.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.469+0000 7f80d1c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f80cc106150 0x7f80cc1001c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:49.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.469+0000 7f80d1c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f80cc100700 0x7f80cc1a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:49.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.469+0000 7f80d1c00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80cc1a5760 con 0x7f80cc100700 2024-02-02T15:35:49.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.469+0000 7f80d1c00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80cc1a58d0 con 0x7f80cc069410 2024-02-02T15:35:49.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.469+0000 7f80d1c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80cc1a5a90 con 0x7f80cc106150 2024-02-02T15:35:49.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.469+0000 7f80caa00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f80cc069410 0x7f80cc0ffc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:49.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.469+0000 7f80caa00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f80cc069410 0x7f80cc0ffc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42094/0 (socket says 172.21.15.1:42094) 2024-02-02T15:35:49.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.469+0000 7f80caa00700 1 -- 172.21.15.1:0/4265597661 learned_addr learned my addr 172.21.15.1:0/4265597661 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:35:49.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.469+0000 7f80ca000700 1 --2- 172.21.15.1:0/4265597661 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f80cc106150 0x7f80cc1001c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:49.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.469+0000 7f80cb400700 1 --2- 172.21.15.1:0/4265597661 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f80cc100700 0x7f80cc1a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:49.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.469+0000 7f80caa00700 1 -- 172.21.15.1:0/4265597661 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f80cc106150 msgr2=0x7f80cc1001c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:49.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.469+0000 7f80caa00700 1 --2- 172.21.15.1:0/4265597661 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f80cc106150 0x7f80cc1001c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.469+0000 7f80caa00700 1 -- 172.21.15.1:0/4265597661 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f80cc100700 msgr2=0x7f80cc1a50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:49.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.469+0000 7f80caa00700 1 --2- 172.21.15.1:0/4265597661 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f80cc100700 0x7f80cc1a50d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.469+0000 7f80caa00700 1 -- 172.21.15.1:0/4265597661 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80bc0095f0 con 0x7f80cc069410 2024-02-02T15:35:49.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.470+0000 7f80caa00700 1 --2- 172.21.15.1:0/4265597661 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f80cc069410 0x7f80cc0ffc80 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f80bc005910 tx=0x7f80bc004710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:49.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.470+0000 7f80c3400700 1 -- 172.21.15.1:0/4265597661 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80bc01d070 con 0x7f80cc069410 2024-02-02T15:35:49.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.470+0000 7f80c3400700 1 -- 172.21.15.1:0/4265597661 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f80bc022470 con 0x7f80cc069410 2024-02-02T15:35:49.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.470+0000 7f80c3400700 1 -- 172.21.15.1:0/4265597661 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80bc017650 con 0x7f80cc069410 2024-02-02T15:35:49.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.470+0000 7f80d1c00700 1 -- 172.21.15.1:0/4265597661 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80cc1a5d10 con 0x7f80cc069410 2024-02-02T15:35:49.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.470+0000 7f80d1c00700 1 -- 172.21.15.1:0/4265597661 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80cc1a60e0 con 0x7f80cc069410 2024-02-02T15:35:49.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.473+0000 7f80c3400700 1 -- 172.21.15.1:0/4265597661 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f80bc0225e0 con 0x7f80cc069410 2024-02-02T15:35:49.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.475+0000 7f80c3400700 1 --2- 172.21.15.1:0/4265597661 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f80ac0a0710 0x7f80ac0a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:49.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.475+0000 7f80e5730b80 1 Processor -- start 2024-02-02T15:35:49.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.475+0000 7f80c3400700 1 -- 172.21.15.1:0/4265597661 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f80bc0cd340 con 0x7f80cc069410 2024-02-02T15:35:49.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.475+0000 7f80e5730b80 1 -- start start 2024-02-02T15:35:49.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.475+0000 7f80ca000700 1 --2- 172.21.15.1:0/4265597661 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f80ac0a0710 0x7f80ac0a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:49.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.476+0000 7f80e5730b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bcdbf93010 0x55bcdbf933f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:49.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.476+0000 7f80e5730b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcdbf93930 0x55bcdbfa6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:49.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.476+0000 7f80e5730b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bcdbfa6ff0 0x55bcdbfa93e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:49.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.476+0000 7f80e5730b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcdbf074c0 con 0x55bcdbfa6ff0 2024-02-02T15:35:49.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.476+0000 7f80e5730b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcdbeebaa0 con 0x55bcdbf93010 2024-02-02T15:35:49.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.476+0000 7f80e5730b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcdbee9460 con 0x55bcdbf93930 2024-02-02T15:35:49.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.476+0000 7f80ca000700 1 --2- 172.21.15.1:0/4265597661 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f80ac0a0710 0x7f80ac0a2bd0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f80cc101470 tx=0x7f80b4009260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:49.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.476+0000 7f80ca000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcdbf93930 0x55bcdbfa6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:49.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.476+0000 7f80ca000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcdbf93930 0x55bcdbfa6ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:44240/0 (socket says 172.21.15.1:44240) 2024-02-02T15:35:49.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.476+0000 7f80ca000700 1 -- 172.21.15.1:0/3558078586 learned_addr learned my addr 172.21.15.1:0/3558078586 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:35:49.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.476+0000 7f80caa00700 1 --2- 172.21.15.1:0/3558078586 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bcdbf93010 0x55bcdbf933f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:49.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.476+0000 7f80cb400700 1 --2- 172.21.15.1:0/3558078586 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bcdbfa6ff0 0x55bcdbfa93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:49.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.476+0000 7f80ca000700 1 -- 172.21.15.1:0/3558078586 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bcdbf93010 msgr2=0x55bcdbf933f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:49.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.476+0000 7f80ca000700 1 --2- 172.21.15.1:0/3558078586 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bcdbf93010 0x55bcdbf933f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.476+0000 7f80ca000700 1 -- 172.21.15.1:0/3558078586 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bcdbfa6ff0 msgr2=0x55bcdbfa93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:49.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.476+0000 7f80ca000700 1 --2- 172.21.15.1:0/3558078586 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bcdbfa6ff0 0x55bcdbfa93e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.476+0000 7f80ca000700 1 -- 172.21.15.1:0/3558078586 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bcdbd663b0 con 0x55bcdbf93930 2024-02-02T15:35:49.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.476+0000 7f80caa00700 1 --2- 172.21.15.1:0/3558078586 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bcdbf93010 0x55bcdbf933f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:35:49.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.477+0000 7f80ca000700 1 --2- 172.21.15.1:0/3558078586 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcdbf93930 0x55bcdbfa6ab0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f80b4011d40 tx=0x7f80b4014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ce2eaf577d3eabf server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:49.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.477+0000 7f80bbe00700 1 -- 172.21.15.1:0/3558078586 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b40099f0 con 0x55bcdbf93930 2024-02-02T15:35:49.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.477+0000 7f80bbe00700 1 -- 172.21.15.1:0/3558078586 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f80b4021070 con 0x55bcdbf93930 2024-02-02T15:35:49.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.477+0000 7f80bbe00700 1 -- 172.21.15.1:0/3558078586 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b4005480 con 0x55bcdbf93930 2024-02-02T15:35:49.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.478+0000 7f80e5730b80 1 -- 172.21.15.1:0/3558078586 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcdbf93930 msgr2=0x55bcdbfa6ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:49.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.478+0000 7f80e5730b80 1 --2- 172.21.15.1:0/3558078586 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcdbf93930 0x55bcdbfa6ab0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f80b4011d40 tx=0x7f80b4014000 comp rx=0 tx=0).stop 2024-02-02T15:35:49.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.478+0000 7f80e5730b80 1 -- 172.21.15.1:0/3558078586 shutdown_connections 2024-02-02T15:35:49.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.478+0000 7f80e5730b80 1 --2- 172.21.15.1:0/3558078586 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bcdbfa6ff0 0x55bcdbfa93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.478+0000 7f80e5730b80 1 --2- 172.21.15.1:0/3558078586 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bcdbf93010 0x55bcdbf933f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.478+0000 7f80e5730b80 1 --2- 172.21.15.1:0/3558078586 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcdbf93930 0x55bcdbfa6ab0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.478+0000 7f80e5730b80 1 -- 172.21.15.1:0/3558078586 >> 172.21.15.1:0/3558078586 conn(0x55bcdbef2db0 msgr2=0x55bcdbf8f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:49.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.478+0000 7f80e5730b80 1 -- 172.21.15.1:0/3558078586 shutdown_connections 2024-02-02T15:35:49.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.478+0000 7f80e5730b80 1 -- 172.21.15.1:0/3558078586 wait complete. 2024-02-02T15:35:49.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.479+0000 7f80e5730b80 1 -- start start 2024-02-02T15:35:49.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.479+0000 7f80e5730b80 1 Processor -- start 2024-02-02T15:35:49.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.480+0000 7f80e5730b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bcdbf93010 0x55bcdbfa3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:49.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.480+0000 7f80e5730b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bcdbf93930 0x55bcdbfa4250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:49.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.480+0000 7f80e5730b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcdbfa6ff0 0x55bcdbfa4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:49.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.480+0000 7f80e5730b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcdbee4d50 con 0x55bcdbf93010 2024-02-02T15:35:49.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.480+0000 7f80e5730b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcdbe80d80 con 0x55bcdbf93930 2024-02-02T15:35:49.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.480+0000 7f80e5730b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcdbe80a20 con 0x55bcdbfa6ff0 2024-02-02T15:35:49.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.480+0000 7f80e5730b80 10 client.0 _ll_register_callbacks cb 0x55bcdbe82290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:35:49.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.480+0000 7f80caa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bcdbf93010 0x55bcdbfa3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:49.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.480+0000 7f80ca000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bcdbf93930 0x55bcdbfa4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:49.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.480+0000 7f80cb400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcdbfa6ff0 0x55bcdbfa4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:49.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.480+0000 7f80ca000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bcdbf93930 0x55bcdbfa4250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42126/0 (socket says 172.21.15.1:42126) 2024-02-02T15:35:49.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.480+0000 7f80ca000700 1 -- 172.21.15.1:0/3894944831 learned_addr learned my addr 172.21.15.1:0/3894944831 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:35:49.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.480+0000 7f80caa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bcdbf93010 0x55bcdbfa3d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49322/0 (socket says 172.21.15.1:49322) 2024-02-02T15:35:49.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.481+0000 7f80cb400700 1 -- 172.21.15.1:0/3894944831 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bcdbf93930 msgr2=0x55bcdbfa4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:49.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.481+0000 7f80cb400700 1 --2- 172.21.15.1:0/3894944831 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bcdbf93930 0x55bcdbfa4250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.481+0000 7f80cb400700 1 -- 172.21.15.1:0/3894944831 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bcdbf93010 msgr2=0x55bcdbfa3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:49.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.481+0000 7f80cb400700 1 --2- 172.21.15.1:0/3894944831 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bcdbf93010 0x55bcdbfa3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.481+0000 7f80cb400700 1 -- 172.21.15.1:0/3894944831 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80bc096040 con 0x55bcdbfa6ff0 2024-02-02T15:35:49.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.481+0000 7f80ca000700 1 --2- 172.21.15.1:0/3894944831 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bcdbf93930 0x55bcdbfa4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:35:49.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.481+0000 7f80caa00700 1 --2- 172.21.15.1:0/3894944831 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bcdbf93010 0x55bcdbfa3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:35:49.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.481+0000 7f80cb400700 1 --2- 172.21.15.1:0/3894944831 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcdbfa6ff0 0x55bcdbfa4790 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f80c4007b50 tx=0x7f80c4007b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=340891bf6d0fd887 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:49.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.481+0000 7f80b8c00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:35:49.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.481+0000 7f80b8c00700 1 -- 172.21.15.1:0/3894944831 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c400e070 con 0x55bcdbfa6ff0 2024-02-02T15:35:49.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.481+0000 7f80e5730b80 1 -- 172.21.15.1:0/3894944831 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bcdbf870f0 con 0x55bcdbfa6ff0 2024-02-02T15:35:49.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.481+0000 7f80b8c00700 1 -- 172.21.15.1:0/3894944831 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f80c4003a00 con 0x55bcdbfa6ff0 2024-02-02T15:35:49.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.482+0000 7f80b8c00700 1 -- 172.21.15.1:0/3894944831 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f80c4003bc0 con 0x55bcdbfa6ff0 2024-02-02T15:35:49.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.482+0000 7f80e5730b80 10 client.27988 fetch_fsmap learned FSMap version 43 2024-02-02T15:35:49.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.482+0000 7f80e5730b80 1 -- 172.21.15.1:0/3894944831 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x7f80b4017070 con 0x55bcdbfa6ff0 2024-02-02T15:35:49.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.482+0000 7f80b8c00700 1 -- 172.21.15.1:0/3894944831 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bcdbf870f0 con 0x55bcdbfa6ff0 2024-02-02T15:35:49.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.482+0000 7f80b8c00700 1 -- 172.21.15.1:0/3894944831 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 43) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f80c40041b0 con 0x55bcdbfa6ff0 2024-02-02T15:35:49.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.482+0000 7f80e5730b80 10 client.27988 fetch_fsmap finished waiting for FSMap version 43 2024-02-02T15:35:49.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.482+0000 7f80e5730b80 10 client.27988 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-02T15:35:49.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.482+0000 7f80e5730b80 20 client.27988 populate_metadata read hostname 'smithi001' 2024-02-02T15:35:49.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.482+0000 7f80e5730b80 1 --2- 172.21.15.1:0/3894944831 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55bcdbfb6be0 0x55bcdbfb9000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:49.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.482+0000 7f80e5730b80 4 client.27988 mds_command: new command op to 24529 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:35:49.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.482+0000 7f80e5730b80 1 -- 172.21.15.1:0/3894944831 --> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bcdbedddd0 con 0x55bcdbfb6be0 2024-02-02T15:35:49.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.482+0000 7f80caa00700 1 --2- 172.21.15.1:0/3894944831 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55bcdbfb6be0 0x55bcdbfb9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:49.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.483+0000 7f80caa00700 1 --2- 172.21.15.1:0/3894944831 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55bcdbfb6be0 0x55bcdbfb9000 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=52bdb891804b2a07 server_cookie=694e8b5021ffe963 in_seq=0 out_seq=0 2024-02-02T15:35:49.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.483+0000 7f80b8c00700 10 client.27988 ms_handle_connect on v2:172.21.15.142:6838/321675421 2024-02-02T15:35:49.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.483+0000 7f80b8c00700 1 -- 172.21.15.1:0/3894944831 <== mds.1 v2:172.21.15.142:6838/321675421 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55bcdbedddd0 con 0x55bcdbfb6be0 2024-02-02T15:35:49.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.483+0000 7f80b8c00700 10 client.27988 handle_command_reply: tid=0 2024-02-02T15:35:49.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.483+0000 7f80e5730b80 1 client.27988 shutdown 2024-02-02T15:35:49.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.484+0000 7f80e5730b80 1 -- 172.21.15.1:0/3894944831 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcdbfa6ff0 msgr2=0x55bcdbfa4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:49.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.484+0000 7f80e5730b80 1 --2- 172.21.15.1:0/3894944831 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcdbfa6ff0 0x55bcdbfa4790 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f80c4007b50 tx=0x7f80c4007b80 comp rx=0 tx=0).stop 2024-02-02T15:35:49.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.484+0000 7f80e5730b80 1 -- 172.21.15.1:0/3894944831 shutdown_connections 2024-02-02T15:35:49.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.484+0000 7f80e5730b80 1 --2- 172.21.15.1:0/3894944831 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55bcdbfb6be0 0x55bcdbfb9000 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.484+0000 7f80e5730b80 1 --2- 172.21.15.1:0/3894944831 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcdbfa6ff0 0x55bcdbfa4790 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.484+0000 7f80e5730b80 1 --2- 172.21.15.1:0/3894944831 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bcdbf93010 0x55bcdbfa3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.484+0000 7f80b8c00700 0 client.27988 ms_handle_reset on v2:172.21.15.142:6838/321675421 2024-02-02T15:35:49.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.484+0000 7f80e5730b80 1 --2- 172.21.15.1:0/3894944831 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bcdbf93930 0x55bcdbfa4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.484+0000 7f80e5730b80 1 -- 172.21.15.1:0/3894944831 >> 172.21.15.1:0/3894944831 conn(0x55bcdbef2db0 msgr2=0x55bcdbf8efa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:49.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.484+0000 7f80e5730b80 1 -- 172.21.15.1:0/3894944831 shutdown_connections 2024-02-02T15:35:49.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.484+0000 7f80e5730b80 1 -- 172.21.15.1:0/3894944831 wait complete. 2024-02-02T15:35:49.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.484+0000 7f80e5730b80 20 client.27988 trim_cache size 0 max 16384 2024-02-02T15:35:49.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.499+0000 7f80e5730b80 1 Processor -- start 2024-02-02T15:35:49.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.499+0000 7f80e5730b80 1 -- start start 2024-02-02T15:35:49.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.499+0000 7f80e5730b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcdbf93010 0x55bcdbfa45c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:49.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.499+0000 7f80e5730b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bcdbf93930 0x55bcdbf95840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:49.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.499+0000 7f80e5730b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bcdbfa6ff0 0x55bcdbf9f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:49.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.499+0000 7f80e5730b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcdbd686a0 con 0x55bcdbfa6ff0 2024-02-02T15:35:49.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.499+0000 7f80e5730b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcdbf8ff80 con 0x55bcdbf93930 2024-02-02T15:35:49.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.499+0000 7f80e5730b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcdbf8ae00 con 0x55bcdbf93010 2024-02-02T15:35:49.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.500+0000 7f80ca000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bcdbf93930 0x55bcdbf95840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:49.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.500+0000 7f80caa00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcdbf93010 0x55bcdbfa45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:49.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.500+0000 7f80ca000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bcdbf93930 0x55bcdbf95840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42130/0 (socket says 172.21.15.1:42130) 2024-02-02T15:35:49.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.500+0000 7f80ca000700 1 -- 172.21.15.1:0/1916350792 learned_addr learned my addr 172.21.15.1:0/1916350792 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:35:49.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.500+0000 7f80cb400700 1 --2- 172.21.15.1:0/1916350792 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bcdbfa6ff0 0x55bcdbf9f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:49.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.500+0000 7f80caa00700 1 -- 172.21.15.1:0/1916350792 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bcdbf93930 msgr2=0x55bcdbf95840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:49.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.500+0000 7f80caa00700 1 --2- 172.21.15.1:0/1916350792 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bcdbf93930 0x55bcdbf95840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.500+0000 7f80caa00700 1 -- 172.21.15.1:0/1916350792 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bcdbfa6ff0 msgr2=0x55bcdbf9f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:49.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.500+0000 7f80caa00700 1 --2- 172.21.15.1:0/1916350792 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bcdbfa6ff0 0x55bcdbf9f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.500+0000 7f80caa00700 1 -- 172.21.15.1:0/1916350792 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bcdbd12b60 con 0x55bcdbf93010 2024-02-02T15:35:49.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.500+0000 7f80caa00700 1 --2- 172.21.15.1:0/1916350792 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcdbf93010 0x55bcdbfa45c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f80bc0ca6f0 tx=0x7f80bc01a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7af8bce9d4942006 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:49.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.500+0000 7f80c0c00700 1 -- 172.21.15.1:0/1916350792 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80bc09ac50 con 0x55bcdbf93010 2024-02-02T15:35:49.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.501+0000 7f80c0c00700 1 -- 172.21.15.1:0/1916350792 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f80bc09a550 con 0x55bcdbf93010 2024-02-02T15:35:49.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.501+0000 7f80c0c00700 1 -- 172.21.15.1:0/1916350792 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80bc022900 con 0x55bcdbf93010 2024-02-02T15:35:49.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.501+0000 7f80e5730b80 1 -- 172.21.15.1:0/1916350792 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcdbf93010 msgr2=0x55bcdbfa45c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:49.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.501+0000 7f80e5730b80 1 --2- 172.21.15.1:0/1916350792 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcdbf93010 0x55bcdbfa45c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f80bc0ca6f0 tx=0x7f80bc01a040 comp rx=0 tx=0).stop 2024-02-02T15:35:49.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.501+0000 7f80e5730b80 1 -- 172.21.15.1:0/1916350792 shutdown_connections 2024-02-02T15:35:49.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.501+0000 7f80e5730b80 1 --2- 172.21.15.1:0/1916350792 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bcdbfa6ff0 0x55bcdbf9f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.501+0000 7f80e5730b80 1 --2- 172.21.15.1:0/1916350792 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcdbf93010 0x55bcdbfa45c0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.501+0000 7f80e5730b80 1 --2- 172.21.15.1:0/1916350792 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bcdbf93930 0x55bcdbf95840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.501+0000 7f80e5730b80 1 -- 172.21.15.1:0/1916350792 >> 172.21.15.1:0/1916350792 conn(0x55bcdbef2db0 msgr2=0x55bcdbfb2150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:49.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.501+0000 7f80e5730b80 1 -- 172.21.15.1:0/1916350792 shutdown_connections 2024-02-02T15:35:49.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.502+0000 7f80e5730b80 1 -- 172.21.15.1:0/1916350792 wait complete. 2024-02-02T15:35:49.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.502+0000 7f80e5730b80 1 -- start start 2024-02-02T15:35:49.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.502+0000 7f80e5730b80 1 Processor -- start 2024-02-02T15:35:49.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.503+0000 7f80e5730b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcdbf93010 0x55bcdbfaafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:49.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.503+0000 7f80e5730b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bcdbf93930 0x55bcdbfab4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:49.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.503+0000 7f80e5730b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bcdbfa6ff0 0x55bcdbfaba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:49.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.503+0000 7f80e5730b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcdc0076c0 con 0x55bcdbf93930 2024-02-02T15:35:49.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.503+0000 7f80e5730b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcdc007800 con 0x55bcdbfa6ff0 2024-02-02T15:35:49.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.503+0000 7f80e5730b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bcdc007940 con 0x55bcdbf93010 2024-02-02T15:35:49.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.503+0000 7f80e5730b80 10 client.0 _ll_register_callbacks cb 0x55bcdbe82290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:35:49.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.503+0000 7f80caa00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcdbf93010 0x55bcdbfaafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:49.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.503+0000 7f80cb400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bcdbfa6ff0 0x55bcdbfaba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:49.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.503+0000 7f80ca000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bcdbf93930 0x55bcdbfab4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:49.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.503+0000 7f80caa00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcdbf93010 0x55bcdbfaafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:44272/0 (socket says 172.21.15.1:44272) 2024-02-02T15:35:49.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.503+0000 7f80caa00700 1 -- 172.21.15.1:0/775351426 learned_addr learned my addr 172.21.15.1:0/775351426 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:35:49.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.503+0000 7f80cb400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bcdbfa6ff0 0x55bcdbfaba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42144/0 (socket says 172.21.15.1:42144) 2024-02-02T15:35:49.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.504+0000 7f80caa00700 1 -- 172.21.15.1:0/775351426 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bcdbfa6ff0 msgr2=0x55bcdbfaba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:49.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.504+0000 7f80caa00700 1 --2- 172.21.15.1:0/775351426 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bcdbfa6ff0 0x55bcdbfaba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.504+0000 7f80caa00700 1 -- 172.21.15.1:0/775351426 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bcdbf93930 msgr2=0x55bcdbfab4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:49.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.504+0000 7f80caa00700 1 --2- 172.21.15.1:0/775351426 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bcdbf93930 0x55bcdbfab4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.504+0000 7f80caa00700 1 -- 172.21.15.1:0/775351426 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80cc1a60e0 con 0x55bcdbf93010 2024-02-02T15:35:49.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.504+0000 7f80cb400700 1 --2- 172.21.15.1:0/775351426 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bcdbfa6ff0 0x55bcdbfaba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:35:49.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.504+0000 7f80caa00700 1 --2- 172.21.15.1:0/775351426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcdbf93010 0x55bcdbfaafa0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f80bc09a4e0 tx=0x7f80bc09a6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c4bc4d973a49f3c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:49.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.504+0000 7f80b8c00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:35:49.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.504+0000 7f80b8c00700 1 -- 172.21.15.1:0/775351426 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80bc0a2530 con 0x55bcdbf93010 2024-02-02T15:35:49.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.504+0000 7f80b8c00700 1 -- 172.21.15.1:0/775351426 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f80bc0cae40 con 0x55bcdbf93010 2024-02-02T15:35:49.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.504+0000 7f80e5730b80 1 -- 172.21.15.1:0/775351426 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f80b4021070 con 0x55bcdbf93010 2024-02-02T15:35:49.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.504+0000 7f80b8c00700 1 -- 172.21.15.1:0/775351426 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f80bc098690 con 0x55bcdbf93010 2024-02-02T15:35:49.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.504+0000 7f80e5730b80 10 client.27994 fetch_fsmap learned FSMap version 43 2024-02-02T15:35:49.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.504+0000 7f80e5730b80 1 -- 172.21.15.1:0/775351426 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55bcdbeaaea0 con 0x55bcdbf93010 2024-02-02T15:35:49.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.505+0000 7f80b8c00700 1 -- 172.21.15.1:0/775351426 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f80b4021070 con 0x55bcdbf93010 2024-02-02T15:35:49.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.505+0000 7f80b8c00700 1 -- 172.21.15.1:0/775351426 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 43) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f80bc099b40 con 0x55bcdbf93010 2024-02-02T15:35:49.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.505+0000 7f80e5730b80 10 client.27994 fetch_fsmap finished waiting for FSMap version 43 2024-02-02T15:35:49.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.505+0000 7f80e5730b80 10 client.27994 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-02T15:35:49.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.505+0000 7f80e5730b80 20 client.27994 populate_metadata read hostname 'smithi001' 2024-02-02T15:35:49.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.505+0000 7f80e5730b80 1 --2- 172.21.15.1:0/775351426 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55bcdc008ab0 0x55bcdc00aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:49.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.505+0000 7f80e5730b80 4 client.27994 mds_command: new command op to 24529 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:35:49.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.505+0000 7f80e5730b80 1 -- 172.21.15.1:0/775351426 --> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55bcdbea3f50 con 0x55bcdc008ab0 2024-02-02T15:35:49.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.505+0000 7f80ca000700 1 --2- 172.21.15.1:0/775351426 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55bcdc008ab0 0x55bcdc00aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:49.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.506+0000 7f80ca000700 1 --2- 172.21.15.1:0/775351426 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55bcdc008ab0 0x55bcdc00aea0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=581b87968e5895fa server_cookie=a0bb0be72d2b57d5 in_seq=0 out_seq=0 2024-02-02T15:35:49.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.506+0000 7f80b8c00700 10 client.27994 ms_handle_connect on v2:172.21.15.142:6838/321675421 2024-02-02T15:35:49.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.506+0000 7f80b8c00700 1 -- 172.21.15.1:0/775351426 <== mds.1 v2:172.21.15.142:6838/321675421 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55bcdbea3f50 con 0x55bcdc008ab0 2024-02-02T15:35:49.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.506+0000 7f80b8c00700 10 client.27994 handle_command_reply: tid=0 2024-02-02T15:35:49.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.506+0000 7f80e5730b80 1 client.27994 shutdown 2024-02-02T15:35:49.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.506+0000 7f80e5730b80 1 -- 172.21.15.1:0/775351426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcdbf93010 msgr2=0x55bcdbfaafa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:49.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.506+0000 7f80e5730b80 1 --2- 172.21.15.1:0/775351426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcdbf93010 0x55bcdbfaafa0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f80bc09a4e0 tx=0x7f80bc09a6c0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.506+0000 7f80e5730b80 1 -- 172.21.15.1:0/775351426 shutdown_connections 2024-02-02T15:35:49.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.506+0000 7f80e5730b80 1 --2- 172.21.15.1:0/775351426 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55bcdc008ab0 0x55bcdc00aea0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.506+0000 7f80e5730b80 1 --2- 172.21.15.1:0/775351426 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bcdbfa6ff0 0x55bcdbfaba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.506+0000 7f80e5730b80 1 --2- 172.21.15.1:0/775351426 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bcdbf93010 0x55bcdbfaafa0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.506+0000 7f80e5730b80 1 --2- 172.21.15.1:0/775351426 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bcdbf93930 0x55bcdbfab4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.506+0000 7f80b8c00700 0 client.27994 ms_handle_reset on v2:172.21.15.142:6838/321675421 2024-02-02T15:35:49.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.506+0000 7f80e5730b80 1 -- 172.21.15.1:0/775351426 >> 172.21.15.1:0/775351426 conn(0x55bcdbef2db0 msgr2=0x55bcdbf9f050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:49.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.507+0000 7f80e5730b80 1 -- 172.21.15.1:0/775351426 shutdown_connections 2024-02-02T15:35:49.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.507+0000 7f80e5730b80 1 -- 172.21.15.1:0/775351426 wait complete. 2024-02-02T15:35:49.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.507+0000 7f80e5730b80 20 client.27994 trim_cache size 0 max 16384 2024-02-02T15:35:49.512 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:35:49.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.507+0000 7f80d1c00700 1 -- 172.21.15.1:0/4265597661 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f80ac0a0710 msgr2=0x7f80ac0a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:49.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.507+0000 7f80d1c00700 1 --2- 172.21.15.1:0/4265597661 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f80ac0a0710 0x7f80ac0a2bd0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f80cc101470 tx=0x7f80b4009260 comp rx=0 tx=0).stop 2024-02-02T15:35:49.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.507+0000 7f80d1c00700 1 -- 172.21.15.1:0/4265597661 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f80cc069410 msgr2=0x7f80cc0ffc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:49.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.507+0000 7f80d1c00700 1 --2- 172.21.15.1:0/4265597661 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f80cc069410 0x7f80cc0ffc80 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f80bc005910 tx=0x7f80bc004710 comp rx=0 tx=0).stop 2024-02-02T15:35:49.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.508+0000 7f80d1c00700 1 -- 172.21.15.1:0/4265597661 shutdown_connections 2024-02-02T15:35:49.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.508+0000 7f80d1c00700 1 --2- 172.21.15.1:0/4265597661 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f80ac0a0710 0x7f80ac0a2bd0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.508+0000 7f80d1c00700 1 --2- 172.21.15.1:0/4265597661 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f80cc100700 0x7f80cc1a50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.508+0000 7f80d1c00700 1 --2- 172.21.15.1:0/4265597661 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f80cc069410 0x7f80cc0ffc80 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.508+0000 7f80d1c00700 1 --2- 172.21.15.1:0/4265597661 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f80cc106150 0x7f80cc1001c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.508+0000 7f80d1c00700 1 -- 172.21.15.1:0/4265597661 >> 172.21.15.1:0/4265597661 conn(0x7f80cc076d60 msgr2=0x7f80cc0fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:49.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.508+0000 7f80d1c00700 1 -- 172.21.15.1:0/4265597661 shutdown_connections 2024-02-02T15:35:49.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.508+0000 7f80d1c00700 1 -- 172.21.15.1:0/4265597661 wait complete. 2024-02-02T15:35:49.523 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:49 smithi001 ceph-mon[35470]: pgmap v271: 129 pgs: 129 active+clean; 382 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.8 MiB/s wr, 170 op/s 2024-02-02T15:35:49.525 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-02T15:35:49.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.742+0000 7fecb2000700 1 -- 172.21.15.1:0/3365169753 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fecac106120 msgr2=0x7fecac10dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:49.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.742+0000 7fecb2000700 1 --2- 172.21.15.1:0/3365169753 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fecac106120 0x7fecac10dca0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7feca400b0a0 tx=0x7feca400b3b0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.743+0000 7fecb2000700 1 -- 172.21.15.1:0/3365169753 shutdown_connections 2024-02-02T15:35:49.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.743+0000 7fecb2000700 1 --2- 172.21.15.1:0/3365169753 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fecac106120 0x7fecac10dca0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.743+0000 7fecb2000700 1 --2- 172.21.15.1:0/3365169753 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fecac101000 0x7fecac1013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.743+0000 7fecb2000700 1 --2- 172.21.15.1:0/3365169753 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fecac1019b0 0x7fecac1059f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.743+0000 7fecb2000700 1 -- 172.21.15.1:0/3365169753 >> 172.21.15.1:0/3365169753 conn(0x7fecac0fc840 msgr2=0x7fecac0fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:49.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.743+0000 7fecb2000700 1 -- 172.21.15.1:0/3365169753 shutdown_connections 2024-02-02T15:35:49.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.743+0000 7fecb2000700 1 -- 172.21.15.1:0/3365169753 wait complete. 2024-02-02T15:35:49.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.744+0000 7fecb2000700 1 Processor -- start 2024-02-02T15:35:49.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.744+0000 7fecb2000700 1 -- start start 2024-02-02T15:35:49.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.744+0000 7fecb2000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fecac101000 0x7fecac199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:49.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.744+0000 7fecb2000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fecac106120 0x7fecac1996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:49.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.744+0000 7fecb2000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fecac19db10 0x7fecac19df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:49.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.744+0000 7fecb2000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecac19e590 con 0x7fecac19db10 2024-02-02T15:35:49.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.745+0000 7fecb2000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecac19e700 con 0x7fecac106120 2024-02-02T15:35:49.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.745+0000 7fecb2000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecac19e8c0 con 0x7fecac101000 2024-02-02T15:35:49.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.745+0000 7fecab400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fecac101000 0x7fecac199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:49.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.745+0000 7fecaaa00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fecac106120 0x7fecac1996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:49.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.745+0000 7fecab400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fecac101000 0x7fecac199170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:44290/0 (socket says 172.21.15.1:44290) 2024-02-02T15:35:49.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.745+0000 7fecab400700 1 -- 172.21.15.1:0/884125051 learned_addr learned my addr 172.21.15.1:0/884125051 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:35:49.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.745+0000 7fecabe00700 1 --2- 172.21.15.1:0/884125051 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fecac19db10 0x7fecac19df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:49.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.745+0000 7fecaaa00700 1 -- 172.21.15.1:0/884125051 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fecac101000 msgr2=0x7fecac199170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:49.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.745+0000 7fecaaa00700 1 --2- 172.21.15.1:0/884125051 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fecac101000 0x7fecac199170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.745+0000 7fecaaa00700 1 -- 172.21.15.1:0/884125051 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fecac19db10 msgr2=0x7fecac19df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:49.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.745+0000 7fecaaa00700 1 --2- 172.21.15.1:0/884125051 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fecac19db10 0x7fecac19df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.745+0000 7fecaaa00700 1 -- 172.21.15.1:0/884125051 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec9c009520 con 0x7fecac106120 2024-02-02T15:35:49.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.745+0000 7fecab400700 1 --2- 172.21.15.1:0/884125051 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fecac101000 0x7fecac199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:35:49.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.745+0000 7fecaaa00700 1 --2- 172.21.15.1:0/884125051 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fecac106120 0x7fecac1996b0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fec9400e9c0 tx=0x7fec9400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:49.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.746+0000 7feca3e00700 1 -- 172.21.15.1:0/884125051 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec9400c9c0 con 0x7fecac106120 2024-02-02T15:35:49.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.746+0000 7feca3e00700 1 -- 172.21.15.1:0/884125051 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fec94010430 con 0x7fecac106120 2024-02-02T15:35:49.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.746+0000 7fecb2000700 1 -- 172.21.15.1:0/884125051 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feca4009d00 con 0x7fecac106120 2024-02-02T15:35:49.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.746+0000 7feca3e00700 1 -- 172.21.15.1:0/884125051 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec94018680 con 0x7fecac106120 2024-02-02T15:35:49.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.746+0000 7fecb2000700 1 -- 172.21.15.1:0/884125051 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecac1a6170 con 0x7fecac106120 2024-02-02T15:35:49.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.750+0000 7feca3e00700 1 -- 172.21.15.1:0/884125051 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7fec9400cb20 con 0x7fecac106120 2024-02-02T15:35:49.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.752+0000 7fecc5addb80 1 Processor -- start 2024-02-02T15:35:49.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.752+0000 7fecc5addb80 1 -- start start 2024-02-02T15:35:49.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.752+0000 7fecc5addb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55abe2e6e010 0x55abe2e6e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:49.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.752+0000 7fecc5addb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55abe2e6e930 0x55abe2e792a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:49.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.752+0000 7fecc5addb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55abe2e797e0 0x55abe2e7bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:49.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.752+0000 7fecc5addb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abe2de24c0 con 0x55abe2e6e930 2024-02-02T15:35:49.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.752+0000 7fecc5addb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abe2dc6aa0 con 0x55abe2e6e010 2024-02-02T15:35:49.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.752+0000 7fecc5addb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abe2dc4460 con 0x55abe2e797e0 2024-02-02T15:35:49.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.752+0000 7feca3e00700 1 --2- 172.21.15.1:0/884125051 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fec8c0a0690 0x7fec8c0a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:49.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.752+0000 7fecab400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55abe2e6e930 0x55abe2e792a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:49.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.752+0000 7fecaaa00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55abe2e797e0 0x55abe2e7bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:49.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.752+0000 7fecabe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55abe2e6e010 0x55abe2e6e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:49.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.752+0000 7fecab400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55abe2e6e930 0x55abe2e792a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49394/0 (socket says 172.21.15.1:49394) 2024-02-02T15:35:49.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.753+0000 7fecab400700 1 -- 172.21.15.1:0/3984695785 learned_addr learned my addr 172.21.15.1:0/3984695785 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:35:49.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.752+0000 7fecabe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55abe2e6e010 0x55abe2e6e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42180/0 (socket says 172.21.15.1:42180) 2024-02-02T15:35:49.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.753+0000 7fecabe00700 1 --2- 172.21.15.1:0/884125051 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fec8c0a0690 0x7fec8c0a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:49.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.753+0000 7feca3e00700 1 -- 172.21.15.1:0/884125051 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fec94014070 con 0x7fecac106120 2024-02-02T15:35:49.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.753+0000 7fecaaa00700 1 -- 172.21.15.1:0/3984695785 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55abe2e6e010 msgr2=0x55abe2e6e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:49.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.753+0000 7fecaaa00700 1 --2- 172.21.15.1:0/3984695785 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55abe2e6e010 0x55abe2e6e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.753+0000 7fecaaa00700 1 -- 172.21.15.1:0/3984695785 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55abe2e6e930 msgr2=0x55abe2e792a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:49.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.753+0000 7fecaaa00700 1 --2- 172.21.15.1:0/3984695785 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55abe2e6e930 0x55abe2e792a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.753+0000 7fecaaa00700 1 -- 172.21.15.1:0/3984695785 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55abe2c413b0 con 0x55abe2e797e0 2024-02-02T15:35:49.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.753+0000 7fecabe00700 1 --2- 172.21.15.1:0/3984695785 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55abe2e6e010 0x55abe2e6e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:35:49.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.753+0000 7fecab400700 1 --2- 172.21.15.1:0/3984695785 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55abe2e6e930 0x55abe2e792a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:35:49.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.753+0000 7fecaaa00700 1 --2- 172.21.15.1:0/3984695785 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55abe2e797e0 0x55abe2e7bbd0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fec940c9530 tx=0x7fec9408ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9cd81e47edfc542 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:49.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.753+0000 7fecabe00700 1 --2- 172.21.15.1:0/884125051 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fec8c0a0690 0x7fec8c0a2b50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7feca40062a0 tx=0x7feca4006210 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:49.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.753+0000 7fec9be00700 1 -- 172.21.15.1:0/3984695785 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec94095670 con 0x55abe2e797e0 2024-02-02T15:35:49.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.754+0000 7fec9be00700 1 -- 172.21.15.1:0/3984695785 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fec94099100 con 0x55abe2e797e0 2024-02-02T15:35:49.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.754+0000 7fec9be00700 1 -- 172.21.15.1:0/3984695785 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec9409d160 con 0x55abe2e797e0 2024-02-02T15:35:49.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.754+0000 7fecc5addb80 1 -- 172.21.15.1:0/3984695785 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55abe2e797e0 msgr2=0x55abe2e7bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:49.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.754+0000 7fecc5addb80 1 --2- 172.21.15.1:0/3984695785 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55abe2e797e0 0x55abe2e7bbd0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fec940c9530 tx=0x7fec9408ecc0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.754+0000 7fecc5addb80 1 -- 172.21.15.1:0/3984695785 shutdown_connections 2024-02-02T15:35:49.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.754+0000 7fecc5addb80 1 --2- 172.21.15.1:0/3984695785 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55abe2e797e0 0x55abe2e7bbd0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.754+0000 7fecc5addb80 1 --2- 172.21.15.1:0/3984695785 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55abe2e6e010 0x55abe2e6e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.754+0000 7fecc5addb80 1 --2- 172.21.15.1:0/3984695785 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55abe2e6e930 0x55abe2e792a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.754+0000 7fecc5addb80 1 -- 172.21.15.1:0/3984695785 >> 172.21.15.1:0/3984695785 conn(0x55abe2dcddb0 msgr2=0x55abe2e6a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:49.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.755+0000 7fecc5addb80 1 -- 172.21.15.1:0/3984695785 shutdown_connections 2024-02-02T15:35:49.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.755+0000 7fecc5addb80 1 -- 172.21.15.1:0/3984695785 wait complete. 2024-02-02T15:35:49.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.755+0000 7fecc5addb80 1 -- start start 2024-02-02T15:35:49.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.755+0000 7fecc5addb80 1 Processor -- start 2024-02-02T15:35:49.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.756+0000 7fecc5addb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55abe2e6e010 0x55abe2e79260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:49.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.756+0000 7fecc5addb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55abe2e6e930 0x55abe2e764e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:49.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.756+0000 7fecc5addb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55abe2e797e0 0x55abe2e76bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:49.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.756+0000 7fecc5addb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abe2dbfd50 con 0x55abe2e6e010 2024-02-02T15:35:49.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.756+0000 7fecc5addb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abe2d5bd80 con 0x55abe2e797e0 2024-02-02T15:35:49.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.756+0000 7fecc5addb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abe2d5ba20 con 0x55abe2e6e930 2024-02-02T15:35:49.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.756+0000 7fecc5addb80 10 client.0 _ll_register_callbacks cb 0x55abe2d5d290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:35:49.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.756+0000 7fecaaa00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55abe2e6e930 0x55abe2e764e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:49.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.756+0000 7fecab400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55abe2e6e010 0x55abe2e79260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:49.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.756+0000 7fecab400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55abe2e6e010 0x55abe2e79260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49396/0 (socket says 172.21.15.1:49396) 2024-02-02T15:35:49.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.756+0000 7fecab400700 1 -- 172.21.15.1:0/4099885926 learned_addr learned my addr 172.21.15.1:0/4099885926 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:35:49.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.756+0000 7fecaaa00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55abe2e6e930 0x55abe2e764e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:44304/0 (socket says 172.21.15.1:44304) 2024-02-02T15:35:49.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.756+0000 7fecabe00700 1 --2- 172.21.15.1:0/4099885926 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55abe2e797e0 0x55abe2e76bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:49.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.757+0000 7fecaaa00700 1 -- 172.21.15.1:0/4099885926 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55abe2e797e0 msgr2=0x55abe2e76bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:49.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.757+0000 7fecaaa00700 1 --2- 172.21.15.1:0/4099885926 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55abe2e797e0 0x55abe2e76bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.757+0000 7fecaaa00700 1 -- 172.21.15.1:0/4099885926 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55abe2e6e010 msgr2=0x55abe2e79260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:49.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.757+0000 7fecaaa00700 1 --2- 172.21.15.1:0/4099885926 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55abe2e6e010 0x55abe2e79260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.757+0000 7fecaaa00700 1 -- 172.21.15.1:0/4099885926 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec9c0114a0 con 0x55abe2e6e930 2024-02-02T15:35:49.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.757+0000 7fecaaa00700 1 --2- 172.21.15.1:0/4099885926 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55abe2e6e930 0x55abe2e764e0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x55abe2e6ee80 tx=0x7fec940107b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42adf7b34c0b7dbc server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:49.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.757+0000 7fec98c00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:35:49.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.757+0000 7fec98c00700 1 -- 172.21.15.1:0/4099885926 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec94098c90 con 0x55abe2e6e930 2024-02-02T15:35:49.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.757+0000 7fecc5addb80 1 -- 172.21.15.1:0/4099885926 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55abe2e620f0 con 0x55abe2e6e930 2024-02-02T15:35:49.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.757+0000 7fec98c00700 1 -- 172.21.15.1:0/4099885926 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fec94099a30 con 0x55abe2e6e930 2024-02-02T15:35:49.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.757+0000 7fec98c00700 1 -- 172.21.15.1:0/4099885926 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec940a13a0 con 0x55abe2e6e930 2024-02-02T15:35:49.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.757+0000 7fecc5addb80 10 client.28006 fetch_fsmap learned FSMap version 43 2024-02-02T15:35:49.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.757+0000 7fecc5addb80 1 -- 172.21.15.1:0/4099885926 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x7fecac1a6170 con 0x55abe2e6e930 2024-02-02T15:35:49.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.758+0000 7fec98c00700 1 -- 172.21.15.1:0/4099885926 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fec94092020 con 0x55abe2e6e930 2024-02-02T15:35:49.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.758+0000 7fec98c00700 1 -- 172.21.15.1:0/4099885926 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 43) v1 ==== 7506+0+0 (secure 0 0 0) 0x7fec94099590 con 0x55abe2e6e930 2024-02-02T15:35:49.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.758+0000 7fecc5addb80 10 client.28006 fetch_fsmap finished waiting for FSMap version 43 2024-02-02T15:35:49.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.758+0000 7fecc5addb80 10 client.28006 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:35:49.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.758+0000 7fecc5addb80 20 client.28006 populate_metadata read hostname 'smithi001' 2024-02-02T15:35:49.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.758+0000 7fecc5addb80 1 --2- 172.21.15.1:0/4099885926 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55abe2e88d90 0x55abe2e8b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:49.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.758+0000 7fecc5addb80 4 client.28006 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:35:49.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.758+0000 7fecc5addb80 1 -- 172.21.15.1:0/4099885926 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55abe2db8dd0 con 0x55abe2e88d90 2024-02-02T15:35:49.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.758+0000 7fecab400700 1 --2- 172.21.15.1:0/4099885926 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55abe2e88d90 0x55abe2e8b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:49.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.759+0000 7fecab400700 1 --2- 172.21.15.1:0/4099885926 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55abe2e88d90 0x55abe2e8b1b0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c2ffc3413eed7c7 server_cookie=a9df9b1e8708fff in_seq=0 out_seq=0 2024-02-02T15:35:49.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.759+0000 7fec98c00700 10 client.28006 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:35:49.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.759+0000 7fec98c00700 1 -- 172.21.15.1:0/4099885926 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55abe2db8dd0 con 0x55abe2e88d90 2024-02-02T15:35:49.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.759+0000 7fec98c00700 10 client.28006 handle_command_reply: tid=0 2024-02-02T15:35:49.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.760+0000 7fecc5addb80 1 client.28006 shutdown 2024-02-02T15:35:49.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.760+0000 7fecc5addb80 1 -- 172.21.15.1:0/4099885926 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55abe2e6e930 msgr2=0x55abe2e764e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:49.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.760+0000 7fecc5addb80 1 --2- 172.21.15.1:0/4099885926 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55abe2e6e930 0x55abe2e764e0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x55abe2e6ee80 tx=0x7fec940107b0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.760+0000 7fecc5addb80 1 -- 172.21.15.1:0/4099885926 shutdown_connections 2024-02-02T15:35:49.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.760+0000 7fecc5addb80 1 --2- 172.21.15.1:0/4099885926 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55abe2e88d90 0x55abe2e8b1b0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.760+0000 7fecc5addb80 1 --2- 172.21.15.1:0/4099885926 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55abe2e797e0 0x55abe2e76bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.760+0000 7fecc5addb80 1 --2- 172.21.15.1:0/4099885926 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55abe2e6e010 0x55abe2e79260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.760+0000 7fec98c00700 0 client.28006 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:35:49.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.760+0000 7fecc5addb80 1 --2- 172.21.15.1:0/4099885926 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55abe2e6e930 0x55abe2e764e0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.760+0000 7fecc5addb80 1 -- 172.21.15.1:0/4099885926 >> 172.21.15.1:0/4099885926 conn(0x55abe2dcddb0 msgr2=0x55abe2e69fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:49.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.760+0000 7fecc5addb80 1 -- 172.21.15.1:0/4099885926 shutdown_connections 2024-02-02T15:35:49.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.760+0000 7fecc5addb80 1 -- 172.21.15.1:0/4099885926 wait complete. 2024-02-02T15:35:49.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.760+0000 7fecc5addb80 20 client.28006 trim_cache size 0 max 16384 2024-02-02T15:35:49.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.776+0000 7fecc5addb80 1 Processor -- start 2024-02-02T15:35:49.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.776+0000 7fecc5addb80 1 -- start start 2024-02-02T15:35:49.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.776+0000 7fecc5addb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55abe2e6e010 0x55abe2e70a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:49.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.776+0000 7fecc5addb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55abe2e6e930 0x55abe2e78d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:49.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.776+0000 7fecc5addb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55abe2e797e0 0x55abe2e75d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:49.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.776+0000 7fecc5addb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abe2c436a0 con 0x55abe2e6e930 2024-02-02T15:35:49.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.776+0000 7fecc5addb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abe2e61cc0 con 0x55abe2e6e010 2024-02-02T15:35:49.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.776+0000 7fecc5addb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abe2e6b1e0 con 0x55abe2e797e0 2024-02-02T15:35:49.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.776+0000 7fecaaa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55abe2e6e930 0x55abe2e78d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:49.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.776+0000 7fecabe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55abe2e797e0 0x55abe2e75d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:49.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.776+0000 7fecaaa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55abe2e6e930 0x55abe2e78d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49402/0 (socket says 172.21.15.1:49402) 2024-02-02T15:35:49.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.776+0000 7fecaaa00700 1 -- 172.21.15.1:0/1597797252 learned_addr learned my addr 172.21.15.1:0/1597797252 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:35:49.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.776+0000 7fecab400700 1 --2- 172.21.15.1:0/1597797252 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55abe2e6e010 0x55abe2e70a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:49.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.777+0000 7fecabe00700 1 -- 172.21.15.1:0/1597797252 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55abe2e6e010 msgr2=0x55abe2e70a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:49.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.777+0000 7fecabe00700 1 --2- 172.21.15.1:0/1597797252 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55abe2e6e010 0x55abe2e70a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.777+0000 7fecabe00700 1 -- 172.21.15.1:0/1597797252 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55abe2e6e930 msgr2=0x55abe2e78d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:49.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.777+0000 7fecabe00700 1 --2- 172.21.15.1:0/1597797252 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55abe2e6e930 0x55abe2e78d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.777+0000 7fecabe00700 1 -- 172.21.15.1:0/1597797252 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55abe2bedb60 con 0x55abe2e797e0 2024-02-02T15:35:49.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.777+0000 7fecab400700 1 --2- 172.21.15.1:0/1597797252 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55abe2e6e010 0x55abe2e70a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:35:49.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.777+0000 7fecabe00700 1 --2- 172.21.15.1:0/1597797252 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55abe2e797e0 0x55abe2e75d80 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7feca4009fd0 tx=0x7feca4009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a5199781d1fc574 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:49.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.777+0000 7feca1200700 1 -- 172.21.15.1:0/1597797252 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feca4030070 con 0x55abe2e797e0 2024-02-02T15:35:49.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.777+0000 7feca1200700 1 -- 172.21.15.1:0/1597797252 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7feca4026070 con 0x55abe2e797e0 2024-02-02T15:35:49.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.777+0000 7feca1200700 1 -- 172.21.15.1:0/1597797252 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feca400e050 con 0x55abe2e797e0 2024-02-02T15:35:49.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.778+0000 7fecc5addb80 1 -- 172.21.15.1:0/1597797252 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55abe2e797e0 msgr2=0x55abe2e75d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:49.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.778+0000 7fecc5addb80 1 --2- 172.21.15.1:0/1597797252 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55abe2e797e0 0x55abe2e75d80 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7feca4009fd0 tx=0x7feca4009b80 comp rx=0 tx=0).stop 2024-02-02T15:35:49.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.778+0000 7fecc5addb80 1 -- 172.21.15.1:0/1597797252 shutdown_connections 2024-02-02T15:35:49.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.778+0000 7fecc5addb80 1 --2- 172.21.15.1:0/1597797252 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55abe2e797e0 0x55abe2e75d80 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.778+0000 7fecc5addb80 1 --2- 172.21.15.1:0/1597797252 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55abe2e6e010 0x55abe2e70a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.778+0000 7fecc5addb80 1 --2- 172.21.15.1:0/1597797252 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55abe2e6e930 0x55abe2e78d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.778+0000 7fecc5addb80 1 -- 172.21.15.1:0/1597797252 >> 172.21.15.1:0/1597797252 conn(0x55abe2dcddb0 msgr2=0x55abe2e84900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:49.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.778+0000 7fecc5addb80 1 -- 172.21.15.1:0/1597797252 shutdown_connections 2024-02-02T15:35:49.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.778+0000 7fecc5addb80 1 -- 172.21.15.1:0/1597797252 wait complete. 2024-02-02T15:35:49.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.779+0000 7fecc5addb80 1 -- start start 2024-02-02T15:35:49.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.779+0000 7fecc5addb80 1 Processor -- start 2024-02-02T15:35:49.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.779+0000 7fecc5addb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55abe2e6e010 0x55abe2e7dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:49.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.779+0000 7fecc5addb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55abe2e6e930 0x55abe2e7e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:49.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.779+0000 7fecc5addb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55abe2e797e0 0x55abe2ed3840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:49.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.779+0000 7fecc5addb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abe2ed3d80 con 0x55abe2e6e930 2024-02-02T15:35:49.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.779+0000 7fecc5addb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abe2ed3ef0 con 0x55abe2e797e0 2024-02-02T15:35:49.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.779+0000 7fecc5addb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55abe2ed40b0 con 0x55abe2e6e010 2024-02-02T15:35:49.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.779+0000 7fecc5addb80 10 client.0 _ll_register_callbacks cb 0x55abe2d5d290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:35:49.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.780+0000 7fecaaa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55abe2e6e930 0x55abe2e7e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:49.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.780+0000 7fecabe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55abe2e797e0 0x55abe2ed3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:49.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.780+0000 7fecaaa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55abe2e6e930 0x55abe2e7e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49410/0 (socket says 172.21.15.1:49410) 2024-02-02T15:35:49.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.780+0000 7fecaaa00700 1 -- 172.21.15.1:0/350952329 learned_addr learned my addr 172.21.15.1:0/350952329 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:35:49.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.780+0000 7fecab400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55abe2e6e010 0x55abe2e7dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:49.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.780+0000 7fecaaa00700 1 -- 172.21.15.1:0/350952329 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55abe2e6e010 msgr2=0x55abe2e7dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:49.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.780+0000 7fecaaa00700 1 --2- 172.21.15.1:0/350952329 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55abe2e6e010 0x55abe2e7dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.780+0000 7fecaaa00700 1 -- 172.21.15.1:0/350952329 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55abe2e797e0 msgr2=0x55abe2ed3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:49.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.780+0000 7fecaaa00700 1 --2- 172.21.15.1:0/350952329 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55abe2e797e0 0x55abe2ed3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.780+0000 7fecaaa00700 1 -- 172.21.15.1:0/350952329 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec9c020880 con 0x55abe2e6e930 2024-02-02T15:35:49.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.780+0000 7fecaaa00700 1 --2- 172.21.15.1:0/350952329 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55abe2e6e930 0x55abe2e7e4e0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fec94010be0 tx=0x7fec940840b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=401ca0adae808303 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:49.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.780+0000 7fec98c00700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:35:49.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.780+0000 7fec98c00700 1 -- 172.21.15.1:0/350952329 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec94098c90 con 0x55abe2e6e930 2024-02-02T15:35:49.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.780+0000 7fec98c00700 1 -- 172.21.15.1:0/350952329 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fec94093030 con 0x55abe2e6e930 2024-02-02T15:35:49.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.780+0000 7fecc5addb80 1 -- 172.21.15.1:0/350952329 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fec94099100 con 0x55abe2e6e930 2024-02-02T15:35:49.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.781+0000 7fec98c00700 1 -- 172.21.15.1:0/350952329 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec94098e70 con 0x55abe2e6e930 2024-02-02T15:35:49.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.781+0000 7fecc5addb80 10 client.17931 fetch_fsmap learned FSMap version 43 2024-02-02T15:35:49.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.781+0000 7fecc5addb80 1 -- 172.21.15.1:0/350952329 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x7feca400ce00 con 0x55abe2e6e930 2024-02-02T15:35:49.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.781+0000 7fec98c00700 1 -- 172.21.15.1:0/350952329 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fec94099100 con 0x55abe2e6e930 2024-02-02T15:35:49.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.781+0000 7fec98c00700 1 -- 172.21.15.1:0/350952329 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 43) v1 ==== 7506+0+0 (secure 0 0 0) 0x7fec94095120 con 0x55abe2e6e930 2024-02-02T15:35:49.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.781+0000 7fecc5addb80 10 client.17931 fetch_fsmap finished waiting for FSMap version 43 2024-02-02T15:35:49.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.781+0000 7fecc5addb80 10 client.17931 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:35:49.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.781+0000 7fecc5addb80 20 client.17931 populate_metadata read hostname 'smithi001' 2024-02-02T15:35:49.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.781+0000 7fecc5addb80 1 --2- 172.21.15.1:0/350952329 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55abe2edb3d0 0x55abe2edd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:49.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.781+0000 7fecc5addb80 4 client.17931 mds_command: new command op to 24557 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:35:49.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.781+0000 7fecc5addb80 1 -- 172.21.15.1:0/350952329 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55abe2d7ef50 con 0x55abe2edb3d0 2024-02-02T15:35:49.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.781+0000 7fecab400700 1 --2- 172.21.15.1:0/350952329 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55abe2edb3d0 0x55abe2edd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:49.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.782+0000 7fecab400700 1 --2- 172.21.15.1:0/350952329 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55abe2edb3d0 0x55abe2edd7f0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a19a1aaa925a60b server_cookie=da4bcdbee435a69e in_seq=0 out_seq=0 2024-02-02T15:35:49.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.782+0000 7fec98c00700 10 client.17931 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:35:49.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.782+0000 7fec98c00700 1 -- 172.21.15.1:0/350952329 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55abe2d7ef50 con 0x55abe2edb3d0 2024-02-02T15:35:49.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.782+0000 7fec98c00700 10 client.17931 handle_command_reply: tid=0 2024-02-02T15:35:49.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.783+0000 7fecc5addb80 1 client.17931 shutdown 2024-02-02T15:35:49.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.783+0000 7fecc5addb80 1 -- 172.21.15.1:0/350952329 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55abe2e6e930 msgr2=0x55abe2e7e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:49.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.783+0000 7fecc5addb80 1 --2- 172.21.15.1:0/350952329 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55abe2e6e930 0x55abe2e7e4e0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fec94010be0 tx=0x7fec940840b0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.783+0000 7fecc5addb80 1 -- 172.21.15.1:0/350952329 shutdown_connections 2024-02-02T15:35:49.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.783+0000 7fecc5addb80 1 --2- 172.21.15.1:0/350952329 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55abe2edb3d0 0x55abe2edd7f0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.783+0000 7fecc5addb80 1 --2- 172.21.15.1:0/350952329 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55abe2e797e0 0x55abe2ed3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.783+0000 7fecc5addb80 1 --2- 172.21.15.1:0/350952329 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55abe2e6e010 0x55abe2e7dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.783+0000 7fecc5addb80 1 --2- 172.21.15.1:0/350952329 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55abe2e6e930 0x55abe2e7e4e0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.783+0000 7fec98c00700 0 client.17931 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:35:49.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.783+0000 7fecc5addb80 1 -- 172.21.15.1:0/350952329 >> 172.21.15.1:0/350952329 conn(0x55abe2dcddb0 msgr2=0x55abe2e695c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:49.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.783+0000 7fecc5addb80 1 -- 172.21.15.1:0/350952329 shutdown_connections 2024-02-02T15:35:49.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.783+0000 7fecc5addb80 1 -- 172.21.15.1:0/350952329 wait complete. 2024-02-02T15:35:49.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.783+0000 7fecc5addb80 20 client.17931 trim_cache size 0 max 16384 2024-02-02T15:35:49.786 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:35:49.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.784+0000 7fecb2000700 1 -- 172.21.15.1:0/884125051 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fec8c0a0690 msgr2=0x7fec8c0a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:49.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.784+0000 7fecb2000700 1 --2- 172.21.15.1:0/884125051 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fec8c0a0690 0x7fec8c0a2b50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7feca40062a0 tx=0x7feca4006210 comp rx=0 tx=0).stop 2024-02-02T15:35:49.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.784+0000 7fecb2000700 1 -- 172.21.15.1:0/884125051 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fecac106120 msgr2=0x7fecac1996b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:49.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.784+0000 7fecb2000700 1 --2- 172.21.15.1:0/884125051 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fecac106120 0x7fecac1996b0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fec9400e9c0 tx=0x7fec9400ecd0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.784+0000 7fecb2000700 1 -- 172.21.15.1:0/884125051 shutdown_connections 2024-02-02T15:35:49.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.784+0000 7fecb2000700 1 --2- 172.21.15.1:0/884125051 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fec8c0a0690 0x7fec8c0a2b50 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.784+0000 7fecb2000700 1 --2- 172.21.15.1:0/884125051 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fecac19db10 0x7fecac19df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.784+0000 7fecb2000700 1 --2- 172.21.15.1:0/884125051 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fecac101000 0x7fecac199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.784+0000 7fecb2000700 1 --2- 172.21.15.1:0/884125051 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fecac106120 0x7fecac1996b0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:49.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.784+0000 7fecb2000700 1 -- 172.21.15.1:0/884125051 >> 172.21.15.1:0/884125051 conn(0x7fecac0fc840 msgr2=0x7fecac10c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:49.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.784+0000 7fecb2000700 1 -- 172.21.15.1:0/884125051 shutdown_connections 2024-02-02T15:35:49.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:49.784+0000 7fecb2000700 1 -- 172.21.15.1:0/884125051 wait complete. 2024-02-02T15:35:49.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:49 smithi071 ceph-mon[44707]: pgmap v271: 129 pgs: 129 active+clean; 382 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.8 MiB/s wr, 170 op/s 2024-02-02T15:35:49.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:49 smithi142 ceph-mon[45728]: pgmap v271: 129 pgs: 129 active+clean; 382 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.8 MiB/s wr, 170 op/s 2024-02-02T15:35:50.252 INFO:tasks.workunit.client.0.smithi001.stdout: 1 522649 6.42 MB/sec execute 324 sec latency 13.299 ms 2024-02-02T15:35:50.802 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-02T15:35:50.802 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-02T15:35:51.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.025+0000 7f0b40400700 1 -- 172.21.15.1:0/3825535246 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0b381019a0 msgr2=0x7f0b381059e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:51.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.025+0000 7f0b40400700 1 --2- 172.21.15.1:0/3825535246 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0b381019a0 0x7f0b381059e0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f0b200098f0 tx=0x7f0b20009c00 comp rx=0 tx=0).stop 2024-02-02T15:35:51.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.025+0000 7f0b40400700 1 -- 172.21.15.1:0/3825535246 shutdown_connections 2024-02-02T15:35:51.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.025+0000 7f0b40400700 1 --2- 172.21.15.1:0/3825535246 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b38106110 0x7f0b3810dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:51.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.026+0000 7f0b40400700 1 --2- 172.21.15.1:0/3825535246 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b38100ff0 0x7f0b381013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:51.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.026+0000 7f0b40400700 1 --2- 172.21.15.1:0/3825535246 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0b381019a0 0x7f0b381059e0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:51.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.026+0000 7f0b40400700 1 -- 172.21.15.1:0/3825535246 >> 172.21.15.1:0/3825535246 conn(0x7f0b380fc810 msgr2=0x7f0b380fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:51.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.026+0000 7f0b40400700 1 -- 172.21.15.1:0/3825535246 shutdown_connections 2024-02-02T15:35:51.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.026+0000 7f0b40400700 1 -- 172.21.15.1:0/3825535246 wait complete. 2024-02-02T15:35:51.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.027+0000 7f0b40400700 1 Processor -- start 2024-02-02T15:35:51.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.027+0000 7f0b40400700 1 -- start start 2024-02-02T15:35:51.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.027+0000 7f0b40400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b38100ff0 0x7f0b38073040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:51.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.027+0000 7f0b40400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0b38106110 0x7f0b38073580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:51.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.027+0000 7f0b40400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b38073ac0 0x7f0b381a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:51.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.027+0000 7f0b40400700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b381a5780 con 0x7f0b38106110 2024-02-02T15:35:51.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.027+0000 7f0b40400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b381a58f0 con 0x7f0b38100ff0 2024-02-02T15:35:51.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.027+0000 7f0b40400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b381a5ab0 con 0x7f0b38073ac0 2024-02-02T15:35:51.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.028+0000 7f0b3e400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b38073ac0 0x7f0b381a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:51.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.028+0000 7f0b3d000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0b38106110 0x7f0b38073580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:51.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.028+0000 7f0b3e400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b38073ac0 0x7f0b381a50f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:44348/0 (socket says 172.21.15.1:44348) 2024-02-02T15:35:51.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.028+0000 7f0b3e400700 1 -- 172.21.15.1:0/792358199 learned_addr learned my addr 172.21.15.1:0/792358199 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:35:51.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.028+0000 7f0b3da00700 1 --2- 172.21.15.1:0/792358199 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b38100ff0 0x7f0b38073040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:51.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.028+0000 7f0b3da00700 1 -- 172.21.15.1:0/792358199 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b38073ac0 msgr2=0x7f0b381a50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:51.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.028+0000 7f0b3da00700 1 --2- 172.21.15.1:0/792358199 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b38073ac0 0x7f0b381a50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:51.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.028+0000 7f0b3da00700 1 -- 172.21.15.1:0/792358199 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0b38106110 msgr2=0x7f0b38073580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:51.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.028+0000 7f0b3da00700 1 --2- 172.21.15.1:0/792358199 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0b38106110 0x7f0b38073580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:51.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.028+0000 7f0b3da00700 1 -- 172.21.15.1:0/792358199 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b20009520 con 0x7f0b38100ff0 2024-02-02T15:35:51.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.028+0000 7f0b3e400700 1 --2- 172.21.15.1:0/792358199 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b38073ac0 0x7f0b381a50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:35:51.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.028+0000 7f0b3d000700 1 --2- 172.21.15.1:0/792358199 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0b38106110 0x7f0b38073580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:35:51.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.028+0000 7f0b3da00700 1 --2- 172.21.15.1:0/792358199 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b38100ff0 0x7f0b38073040 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f0b2800e9c0 tx=0x7f0b2800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:51.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.029+0000 7f0b36000700 1 -- 172.21.15.1:0/792358199 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b2800c9c0 con 0x7f0b38100ff0 2024-02-02T15:35:51.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.029+0000 7f0b36000700 1 -- 172.21.15.1:0/792358199 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b28010430 con 0x7f0b38100ff0 2024-02-02T15:35:51.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.029+0000 7f0b36000700 1 -- 172.21.15.1:0/792358199 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b28018680 con 0x7f0b38100ff0 2024-02-02T15:35:51.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.029+0000 7f0b40400700 1 -- 172.21.15.1:0/792358199 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b381a5d90 con 0x7f0b38100ff0 2024-02-02T15:35:51.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.029+0000 7f0b40400700 1 -- 172.21.15.1:0/792358199 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b381a6210 con 0x7f0b38100ff0 2024-02-02T15:35:51.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.032+0000 7f0b36000700 1 -- 172.21.15.1:0/792358199 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f0b28010860 con 0x7f0b38100ff0 2024-02-02T15:35:51.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.034+0000 7f0b53f9eb80 1 Processor -- start 2024-02-02T15:35:51.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.034+0000 7f0b36000700 1 --2- 172.21.15.1:0/792358199 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f0b180a0710 0x7f0b180a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:51.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.034+0000 7f0b53f9eb80 1 -- start start 2024-02-02T15:35:51.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.034+0000 7f0b36000700 1 -- 172.21.15.1:0/792358199 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f0b28014070 con 0x7f0b38100ff0 2024-02-02T15:35:51.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.034+0000 7f0b3e400700 1 --2- 172.21.15.1:0/792358199 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f0b180a0710 0x7f0b180a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:51.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.035+0000 7f0b53f9eb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fa260d5090 0x55fa260d5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:51.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.035+0000 7f0b53f9eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fa260d59b0 0x55fa260e0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:51.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.035+0000 7f0b53f9eb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fa260e0860 0x55fa260e2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:51.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.035+0000 7f0b53f9eb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa26049540 con 0x55fa260d5090 2024-02-02T15:35:51.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.035+0000 7f0b53f9eb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa2602db20 con 0x55fa260e0860 2024-02-02T15:35:51.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.035+0000 7f0b53f9eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa2602b4e0 con 0x55fa260d59b0 2024-02-02T15:35:51.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.035+0000 7f0b3d000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fa260d59b0 0x55fa260e0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:51.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.035+0000 7f0b3d000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fa260d59b0 0x55fa260e0320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:44360/0 (socket says 172.21.15.1:44360) 2024-02-02T15:35:51.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.035+0000 7f0b3da00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fa260d5090 0x55fa260d5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:51.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.035+0000 7f0b3d000700 1 -- 172.21.15.1:0/3818785654 learned_addr learned my addr 172.21.15.1:0/3818785654 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:35:51.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.035+0000 7f0b3e400700 1 --2- 172.21.15.1:0/3818785654 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fa260e0860 0x55fa260e2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:51.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.035+0000 7f0b3d000700 1 -- 172.21.15.1:0/3818785654 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fa260e0860 msgr2=0x55fa260e2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:51.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.035+0000 7f0b3d000700 1 --2- 172.21.15.1:0/3818785654 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fa260e0860 0x55fa260e2c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:51.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.035+0000 7f0b3d000700 1 -- 172.21.15.1:0/3818785654 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fa260d5090 msgr2=0x55fa260d5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:51.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.035+0000 7f0b3d000700 1 --2- 172.21.15.1:0/3818785654 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fa260d5090 0x55fa260d5470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:51.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.035+0000 7f0b3d000700 1 -- 172.21.15.1:0/3818785654 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fa25ea8430 con 0x55fa260d59b0 2024-02-02T15:35:51.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.036+0000 7f0b3e400700 1 --2- 172.21.15.1:0/792358199 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f0b180a0710 0x7f0b180a2bd0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0b30009c10 tx=0x7f0b30009920 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:51.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.036+0000 7f0b3d000700 1 --2- 172.21.15.1:0/3818785654 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fa260d59b0 0x55fa260e0320 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f0b20016000 tx=0x7f0b20010fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4cc48c92c4e9b55 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:51.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.036+0000 7f0b2ea00700 1 -- 172.21.15.1:0/3818785654 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b2001c050 con 0x55fa260d59b0 2024-02-02T15:35:51.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.036+0000 7f0b2ea00700 1 -- 172.21.15.1:0/3818785654 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b2001d040 con 0x55fa260d59b0 2024-02-02T15:35:51.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.036+0000 7f0b2ea00700 1 -- 172.21.15.1:0/3818785654 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b200052a0 con 0x55fa260d59b0 2024-02-02T15:35:51.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.037+0000 7f0b53f9eb80 1 -- 172.21.15.1:0/3818785654 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fa260d59b0 msgr2=0x55fa260e0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:51.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.037+0000 7f0b53f9eb80 1 --2- 172.21.15.1:0/3818785654 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fa260d59b0 0x55fa260e0320 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f0b20016000 tx=0x7f0b20010fb0 comp rx=0 tx=0).stop 2024-02-02T15:35:51.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.037+0000 7f0b53f9eb80 1 -- 172.21.15.1:0/3818785654 shutdown_connections 2024-02-02T15:35:51.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.037+0000 7f0b53f9eb80 1 --2- 172.21.15.1:0/3818785654 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fa260e0860 0x55fa260e2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:51.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.037+0000 7f0b53f9eb80 1 --2- 172.21.15.1:0/3818785654 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fa260d5090 0x55fa260d5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:51.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.037+0000 7f0b53f9eb80 1 --2- 172.21.15.1:0/3818785654 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fa260d59b0 0x55fa260e0320 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:51.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.037+0000 7f0b53f9eb80 1 -- 172.21.15.1:0/3818785654 >> 172.21.15.1:0/3818785654 conn(0x55fa26034e30 msgr2=0x55fa260d1430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:51.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.037+0000 7f0b53f9eb80 1 -- 172.21.15.1:0/3818785654 shutdown_connections 2024-02-02T15:35:51.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.037+0000 7f0b53f9eb80 1 -- 172.21.15.1:0/3818785654 wait complete. 2024-02-02T15:35:51.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.037+0000 7f0b53f9eb80 1 -- start start 2024-02-02T15:35:51.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.038+0000 7f0b53f9eb80 1 Processor -- start 2024-02-02T15:35:51.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.038+0000 7f0b53f9eb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fa260d5090 0x55fa260e0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:51.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.038+0000 7f0b53f9eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fa260d59b0 0x55fa260dd480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:51.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.038+0000 7f0b53f9eb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fa260e0860 0x55fa260ddab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:51.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.038+0000 7f0b53f9eb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa26026dd0 con 0x55fa260e0860 2024-02-02T15:35:51.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.038+0000 7f0b53f9eb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa25fc2e00 con 0x55fa260d5090 2024-02-02T15:35:51.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.038+0000 7f0b53f9eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa25fc2aa0 con 0x55fa260d59b0 2024-02-02T15:35:51.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.038+0000 7f0b53f9eb80 10 client.0 _ll_register_callbacks cb 0x55fa25fc4310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:35:51.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.038+0000 7f0b3da00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fa260d5090 0x55fa260e0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:51.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.038+0000 7f0b3d000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fa260d59b0 0x55fa260dd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:51.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.038+0000 7f0b3da00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fa260d5090 0x55fa260e0270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42234/0 (socket says 172.21.15.1:42234) 2024-02-02T15:35:51.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.039+0000 7f0b3da00700 1 -- 172.21.15.1:0/992179086 learned_addr learned my addr 172.21.15.1:0/992179086 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:35:51.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.039+0000 7f0b3e400700 1 --2- 172.21.15.1:0/992179086 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fa260e0860 0x55fa260ddab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:51.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.039+0000 7f0b3d000700 1 -- 172.21.15.1:0/992179086 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fa260d5090 msgr2=0x55fa260e0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:51.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.039+0000 7f0b3d000700 1 --2- 172.21.15.1:0/992179086 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fa260d5090 0x55fa260e0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:51.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.039+0000 7f0b3d000700 1 -- 172.21.15.1:0/992179086 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fa260e0860 msgr2=0x55fa260ddab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:51.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.039+0000 7f0b3d000700 1 --2- 172.21.15.1:0/992179086 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fa260e0860 0x55fa260ddab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:51.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.039+0000 7f0b3d000700 1 -- 172.21.15.1:0/992179086 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b3810e500 con 0x55fa260d59b0 2024-02-02T15:35:51.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.039+0000 7f0b3e400700 1 --2- 172.21.15.1:0/992179086 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fa260e0860 0x55fa260ddab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:35:51.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.039+0000 7f0b3da00700 1 --2- 172.21.15.1:0/992179086 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fa260d5090 0x55fa260e0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:35:51.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.039+0000 7f0b3d000700 1 --2- 172.21.15.1:0/992179086 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fa260d59b0 0x55fa260dd480 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f0b20000c00 tx=0x7f0b2000bc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa7af54e5315e742 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:51.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.039+0000 7f0b27400700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:35:51.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.039+0000 7f0b27400700 1 -- 172.21.15.1:0/992179086 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b200052c0 con 0x55fa260d59b0 2024-02-02T15:35:51.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.039+0000 7f0b27400700 1 -- 172.21.15.1:0/992179086 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f0b20005420 con 0x55fa260d59b0 2024-02-02T15:35:51.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.039+0000 7f0b53f9eb80 1 -- 172.21.15.1:0/992179086 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fa260c9170 con 0x55fa260d59b0 2024-02-02T15:35:51.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.040+0000 7f0b27400700 1 -- 172.21.15.1:0/992179086 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b2000f8d0 con 0x55fa260d59b0 2024-02-02T15:35:51.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.040+0000 7f0b53f9eb80 10 client.28024 fetch_fsmap learned FSMap version 43 2024-02-02T15:35:51.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.040+0000 7f0b53f9eb80 1 -- 172.21.15.1:0/992179086 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55fa25e54be0 con 0x55fa260d59b0 2024-02-02T15:35:51.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.040+0000 7f0b27400700 1 -- 172.21.15.1:0/992179086 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fa260c9170 con 0x55fa260d59b0 2024-02-02T15:35:51.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.040+0000 7f0b27400700 1 -- 172.21.15.1:0/992179086 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 43) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f0b20009c70 con 0x55fa260d59b0 2024-02-02T15:35:51.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.040+0000 7f0b53f9eb80 10 client.28024 fetch_fsmap finished waiting for FSMap version 43 2024-02-02T15:35:51.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.040+0000 7f0b53f9eb80 10 client.28024 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:35:51.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.040+0000 7f0b53f9eb80 20 client.28024 populate_metadata read hostname 'smithi001' 2024-02-02T15:35:51.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.040+0000 7f0b53f9eb80 1 --2- 172.21.15.1:0/992179086 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55fa260efe10 0x55fa260f2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:51.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.040+0000 7f0b53f9eb80 4 client.28024 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:35:51.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.040+0000 7f0b53f9eb80 1 -- 172.21.15.1:0/992179086 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fa2601fe50 con 0x55fa260efe10 2024-02-02T15:35:51.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.041+0000 7f0b3da00700 1 --2- 172.21.15.1:0/992179086 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55fa260efe10 0x55fa260f2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:51.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.041+0000 7f0b3da00700 1 --2- 172.21.15.1:0/992179086 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55fa260efe10 0x55fa260f2230 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b39e923b90ad7d22 server_cookie=348f349ef9534c2c in_seq=0 out_seq=0 2024-02-02T15:35:51.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.041+0000 7f0b27400700 10 client.28024 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:35:51.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.042+0000 7f0b27400700 1 -- 172.21.15.1:0/992179086 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55fa2601fe50 con 0x55fa260efe10 2024-02-02T15:35:51.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.042+0000 7f0b27400700 10 client.28024 handle_command_reply: tid=0 2024-02-02T15:35:51.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.042+0000 7f0b53f9eb80 1 client.28024 shutdown 2024-02-02T15:35:51.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.042+0000 7f0b53f9eb80 1 -- 172.21.15.1:0/992179086 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fa260d59b0 msgr2=0x55fa260dd480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:51.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.042+0000 7f0b53f9eb80 1 --2- 172.21.15.1:0/992179086 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fa260d59b0 0x55fa260dd480 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f0b20000c00 tx=0x7f0b2000bc40 comp rx=0 tx=0).stop 2024-02-02T15:35:51.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.042+0000 7f0b53f9eb80 1 -- 172.21.15.1:0/992179086 shutdown_connections 2024-02-02T15:35:51.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.042+0000 7f0b53f9eb80 1 --2- 172.21.15.1:0/992179086 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55fa260efe10 0x55fa260f2230 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:51.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.042+0000 7f0b53f9eb80 1 --2- 172.21.15.1:0/992179086 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fa260e0860 0x55fa260ddab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:51.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.042+0000 7f0b53f9eb80 1 --2- 172.21.15.1:0/992179086 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fa260d5090 0x55fa260e0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:51.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.042+0000 7f0b27400700 0 client.28024 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:35:51.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.042+0000 7f0b53f9eb80 1 --2- 172.21.15.1:0/992179086 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fa260d59b0 0x55fa260dd480 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:51.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.042+0000 7f0b53f9eb80 1 -- 172.21.15.1:0/992179086 >> 172.21.15.1:0/992179086 conn(0x55fa26034e30 msgr2=0x55fa260d1020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:51.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.042+0000 7f0b53f9eb80 1 -- 172.21.15.1:0/992179086 shutdown_connections 2024-02-02T15:35:51.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.042+0000 7f0b53f9eb80 1 -- 172.21.15.1:0/992179086 wait complete. 2024-02-02T15:35:51.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.043+0000 7f0b53f9eb80 20 client.28024 trim_cache size 0 max 16384 2024-02-02T15:35:51.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.061+0000 7f0b53f9eb80 1 Processor -- start 2024-02-02T15:35:51.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.061+0000 7f0b53f9eb80 1 -- start start 2024-02-02T15:35:51.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.061+0000 7f0b53f9eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fa260d5090 0x55fa260dfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:51.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.061+0000 7f0b53f9eb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fa260d59b0 0x55fa260dcd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:51.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.062+0000 7f0b53f9eb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fa260e0860 0x55fa260dd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:51.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.062+0000 7f0b53f9eb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa25eaa720 con 0x55fa260d59b0 2024-02-02T15:35:51.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.062+0000 7f0b53f9eb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa260d0220 con 0x55fa260e0860 2024-02-02T15:35:51.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.062+0000 7f0b53f9eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa260d92d0 con 0x55fa260d5090 2024-02-02T15:35:51.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.062+0000 7f0b3da00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fa260d5090 0x55fa260dfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:51.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.062+0000 7f0b3da00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fa260d5090 0x55fa260dfda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:44378/0 (socket says 172.21.15.1:44378) 2024-02-02T15:35:51.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.062+0000 7f0b3da00700 1 -- 172.21.15.1:0/146134868 learned_addr learned my addr 172.21.15.1:0/146134868 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:35:51.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.062+0000 7f0b3e400700 1 --2- 172.21.15.1:0/146134868 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fa260e0860 0x55fa260dd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:51.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.062+0000 7f0b3d000700 1 --2- 172.21.15.1:0/146134868 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fa260d59b0 0x55fa260dcd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:51.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.062+0000 7f0b3da00700 1 -- 172.21.15.1:0/146134868 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fa260e0860 msgr2=0x55fa260dd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:51.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.062+0000 7f0b3da00700 1 --2- 172.21.15.1:0/146134868 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fa260e0860 0x55fa260dd260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:51.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.062+0000 7f0b3da00700 1 -- 172.21.15.1:0/146134868 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fa260d59b0 msgr2=0x55fa260dcd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:51.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.062+0000 7f0b3da00700 1 --2- 172.21.15.1:0/146134868 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fa260d59b0 0x55fa260dcd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:51.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.062+0000 7f0b3da00700 1 -- 172.21.15.1:0/146134868 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fa260d6b80 con 0x55fa260d5090 2024-02-02T15:35:51.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.062+0000 7f0b3da00700 1 --2- 172.21.15.1:0/146134868 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fa260d5090 0x55fa260dfda0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0b28005fd0 tx=0x7f0b28023040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=633da308ba4027ff server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:51.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.063+0000 7f0b2f400700 1 -- 172.21.15.1:0/146134868 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b280b7210 con 0x55fa260d5090 2024-02-02T15:35:51.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.063+0000 7f0b2f400700 1 -- 172.21.15.1:0/146134868 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b28099270 con 0x55fa260d5090 2024-02-02T15:35:51.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.063+0000 7f0b2f400700 1 -- 172.21.15.1:0/146134868 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b280989c0 con 0x55fa260d5090 2024-02-02T15:35:51.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.063+0000 7f0b53f9eb80 1 -- 172.21.15.1:0/146134868 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fa260d5090 msgr2=0x55fa260dfda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:51.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.063+0000 7f0b53f9eb80 1 --2- 172.21.15.1:0/146134868 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fa260d5090 0x55fa260dfda0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0b28005fd0 tx=0x7f0b28023040 comp rx=0 tx=0).stop 2024-02-02T15:35:51.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.063+0000 7f0b53f9eb80 1 -- 172.21.15.1:0/146134868 shutdown_connections 2024-02-02T15:35:51.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.063+0000 7f0b53f9eb80 1 --2- 172.21.15.1:0/146134868 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fa260e0860 0x55fa260dd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:51.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.063+0000 7f0b53f9eb80 1 --2- 172.21.15.1:0/146134868 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fa260d5090 0x55fa260dfda0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:51.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.063+0000 7f0b53f9eb80 1 --2- 172.21.15.1:0/146134868 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fa260d59b0 0x55fa260dcd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:51.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.063+0000 7f0b53f9eb80 1 -- 172.21.15.1:0/146134868 >> 172.21.15.1:0/146134868 conn(0x55fa26034e30 msgr2=0x55fa260d8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:51.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.063+0000 7f0b53f9eb80 1 -- 172.21.15.1:0/146134868 shutdown_connections 2024-02-02T15:35:51.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.063+0000 7f0b53f9eb80 1 -- 172.21.15.1:0/146134868 wait complete. 2024-02-02T15:35:51.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.064+0000 7f0b53f9eb80 1 -- start start 2024-02-02T15:35:51.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.064+0000 7f0b53f9eb80 1 Processor -- start 2024-02-02T15:35:51.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.064+0000 7f0b53f9eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fa260d5090 0x55fa260e71f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:51.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.064+0000 7f0b53f9eb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fa260d59b0 0x55fa260e5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:51.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.065+0000 7f0b53f9eb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fa260e0860 0x55fa260e5a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:51.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.065+0000 7f0b53f9eb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa260e6130 con 0x55fa260d59b0 2024-02-02T15:35:51.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.065+0000 7f0b53f9eb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa260e62a0 con 0x55fa260e0860 2024-02-02T15:35:51.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.065+0000 7f0b53f9eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fa26141940 con 0x55fa260d5090 2024-02-02T15:35:51.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.065+0000 7f0b53f9eb80 10 client.0 _ll_register_callbacks cb 0x55fa260d2bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:35:51.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.065+0000 7f0b3da00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fa260d5090 0x55fa260e71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:51.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.065+0000 7f0b3d000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fa260d59b0 0x55fa260e5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:51.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.065+0000 7f0b3da00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fa260d5090 0x55fa260e71f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:44382/0 (socket says 172.21.15.1:44382) 2024-02-02T15:35:51.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.065+0000 7f0b3da00700 1 -- 172.21.15.1:0/565783196 learned_addr learned my addr 172.21.15.1:0/565783196 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:35:51.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.065+0000 7f0b3e400700 1 --2- 172.21.15.1:0/565783196 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fa260e0860 0x55fa260e5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:51.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.065+0000 7f0b3da00700 1 -- 172.21.15.1:0/565783196 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fa260e0860 msgr2=0x55fa260e5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:51.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.065+0000 7f0b3da00700 1 --2- 172.21.15.1:0/565783196 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fa260e0860 0x55fa260e5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:51.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.065+0000 7f0b3da00700 1 -- 172.21.15.1:0/565783196 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fa260d59b0 msgr2=0x55fa260e5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:51.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.065+0000 7f0b3da00700 1 --2- 172.21.15.1:0/565783196 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fa260d59b0 0x55fa260e5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:51.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.065+0000 7f0b3da00700 1 -- 172.21.15.1:0/565783196 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b381a6210 con 0x55fa260d5090 2024-02-02T15:35:51.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.065+0000 7f0b3da00700 1 --2- 172.21.15.1:0/565783196 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fa260d5090 0x55fa260e71f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0b28095ce0 tx=0x7f0b28099870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20fb8ac540b99113 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:51.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.065+0000 7f0b27400700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:35:51.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.066+0000 7f0b27400700 1 -- 172.21.15.1:0/565783196 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b2809d0a0 con 0x55fa260d5090 2024-02-02T15:35:51.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.066+0000 7f0b27400700 1 -- 172.21.15.1:0/565783196 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f0b280a12e0 con 0x55fa260d5090 2024-02-02T15:35:51.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.066+0000 7f0b53f9eb80 1 -- 172.21.15.1:0/565783196 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0b2001d040 con 0x55fa260d5090 2024-02-02T15:35:51.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.066+0000 7f0b27400700 1 -- 172.21.15.1:0/565783196 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0b28098760 con 0x55fa260d5090 2024-02-02T15:35:51.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.066+0000 7f0b53f9eb80 10 client.28030 fetch_fsmap learned FSMap version 43 2024-02-02T15:35:51.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.066+0000 7f0b53f9eb80 1 -- 172.21.15.1:0/565783196 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55fa25fecf20 con 0x55fa260d5090 2024-02-02T15:35:51.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.066+0000 7f0b27400700 1 -- 172.21.15.1:0/565783196 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b2001d040 con 0x55fa260d5090 2024-02-02T15:35:51.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.066+0000 7f0b27400700 1 -- 172.21.15.1:0/565783196 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 43) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f0b28095060 con 0x55fa260d5090 2024-02-02T15:35:51.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.066+0000 7f0b53f9eb80 10 client.28030 fetch_fsmap finished waiting for FSMap version 43 2024-02-02T15:35:51.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.066+0000 7f0b53f9eb80 10 client.28030 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:35:51.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.066+0000 7f0b53f9eb80 20 client.28030 populate_metadata read hostname 'smithi001' 2024-02-02T15:35:51.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.066+0000 7f0b53f9eb80 1 --2- 172.21.15.1:0/565783196 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55fa26142cf0 0x55fa26145110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:51.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.066+0000 7f0b53f9eb80 4 client.28030 mds_command: new command op to 24557 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-02T15:35:51.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.066+0000 7f0b53f9eb80 1 -- 172.21.15.1:0/565783196 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55fa25fe5fd0 con 0x55fa26142cf0 2024-02-02T15:35:51.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.067+0000 7f0b3d000700 1 --2- 172.21.15.1:0/565783196 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55fa26142cf0 0x55fa26145110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:51.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.067+0000 7f0b3d000700 1 --2- 172.21.15.1:0/565783196 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55fa26142cf0 0x55fa26145110 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e21b1fb4bd1441bd server_cookie=cad5eec89b0b0001 in_seq=0 out_seq=0 2024-02-02T15:35:51.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.067+0000 7f0b27400700 10 client.28030 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:35:51.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.068+0000 7f0b27400700 1 -- 172.21.15.1:0/565783196 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55fa25fe5fd0 con 0x55fa26142cf0 2024-02-02T15:35:51.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.068+0000 7f0b27400700 10 client.28030 handle_command_reply: tid=0 2024-02-02T15:35:51.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.068+0000 7f0b53f9eb80 1 client.28030 shutdown 2024-02-02T15:35:51.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.068+0000 7f0b53f9eb80 1 -- 172.21.15.1:0/565783196 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fa260d5090 msgr2=0x55fa260e71f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:51.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.068+0000 7f0b53f9eb80 1 --2- 172.21.15.1:0/565783196 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fa260d5090 0x55fa260e71f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0b28095ce0 tx=0x7f0b28099870 comp rx=0 tx=0).stop 2024-02-02T15:35:51.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.068+0000 7f0b53f9eb80 1 -- 172.21.15.1:0/565783196 shutdown_connections 2024-02-02T15:35:51.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.068+0000 7f0b53f9eb80 1 --2- 172.21.15.1:0/565783196 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55fa26142cf0 0x55fa26145110 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:51.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.068+0000 7f0b53f9eb80 1 --2- 172.21.15.1:0/565783196 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fa260e0860 0x55fa260e5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:51.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.068+0000 7f0b27400700 0 client.28030 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:35:51.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.068+0000 7f0b53f9eb80 1 --2- 172.21.15.1:0/565783196 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fa260d5090 0x55fa260e71f0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:51.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.068+0000 7f0b53f9eb80 1 --2- 172.21.15.1:0/565783196 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fa260d59b0 0x55fa260e5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:51.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.068+0000 7f0b53f9eb80 1 -- 172.21.15.1:0/565783196 >> 172.21.15.1:0/565783196 conn(0x55fa26034e30 msgr2=0x55fa260d8740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:51.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.068+0000 7f0b53f9eb80 1 -- 172.21.15.1:0/565783196 shutdown_connections 2024-02-02T15:35:51.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.068+0000 7f0b53f9eb80 1 -- 172.21.15.1:0/565783196 wait complete. 2024-02-02T15:35:51.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.069+0000 7f0b53f9eb80 20 client.28030 trim_cache size 0 max 16384 2024-02-02T15:35:51.072 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:35:51.072 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2024-02-02T15:35:51.073 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "a940f11a-50d4-4565-a1d2-b2551f200a13", 2024-02-02T15:35:51.073 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2024-02-02T15:35:51.073 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:35:51.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.069+0000 7f0b40400700 1 -- 172.21.15.1:0/792358199 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f0b180a0710 msgr2=0x7f0b180a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:51.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.069+0000 7f0b40400700 1 --2- 172.21.15.1:0/792358199 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f0b180a0710 0x7f0b180a2bd0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0b30009c10 tx=0x7f0b30009920 comp rx=0 tx=0).stop 2024-02-02T15:35:51.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.069+0000 7f0b40400700 1 -- 172.21.15.1:0/792358199 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b38100ff0 msgr2=0x7f0b38073040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:51.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.069+0000 7f0b40400700 1 --2- 172.21.15.1:0/792358199 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b38100ff0 0x7f0b38073040 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f0b2800e9c0 tx=0x7f0b2800ecd0 comp rx=0 tx=0).stop 2024-02-02T15:35:51.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.069+0000 7f0b40400700 1 -- 172.21.15.1:0/792358199 shutdown_connections 2024-02-02T15:35:51.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.069+0000 7f0b40400700 1 --2- 172.21.15.1:0/792358199 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f0b180a0710 0x7f0b180a2bd0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:51.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.069+0000 7f0b40400700 1 --2- 172.21.15.1:0/792358199 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b38073ac0 0x7f0b381a50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:51.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.069+0000 7f0b40400700 1 --2- 172.21.15.1:0/792358199 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b38100ff0 0x7f0b38073040 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:51.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.069+0000 7f0b40400700 1 --2- 172.21.15.1:0/792358199 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0b38106110 0x7f0b38073580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:51.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.069+0000 7f0b40400700 1 -- 172.21.15.1:0/792358199 >> 172.21.15.1:0/792358199 conn(0x7f0b380fc810 msgr2=0x7f0b380fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:51.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.069+0000 7f0b40400700 1 -- 172.21.15.1:0/792358199 shutdown_connections 2024-02-02T15:35:51.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.070+0000 7f0b40400700 1 -- 172.21.15.1:0/792358199 wait complete. 2024-02-02T15:35:51.088 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-02T15:35:51.252 INFO:tasks.workunit.client.0.smithi001.stdout: 1 523732 6.41 MB/sec execute 325 sec latency 28.925 ms 2024-02-02T15:35:51.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.303+0000 7f703a800700 1 -- 172.21.15.1:0/168462134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7034106ed0 msgr2=0x7f70341072b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:51.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.303+0000 7f703a800700 1 --2- 172.21.15.1:0/168462134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7034106ed0 0x7f70341072b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f70240099c0 tx=0x7f7024009cd0 comp rx=0 tx=0).stop 2024-02-02T15:35:51.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.303+0000 7f703a800700 1 -- 172.21.15.1:0/168462134 shutdown_connections 2024-02-02T15:35:51.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.303+0000 7f703a800700 1 --2- 172.21.15.1:0/168462134 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70340689b0 0x7f703410dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:51.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.303+0000 7f703a800700 1 --2- 172.21.15.1:0/168462134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7034106ed0 0x7f70341072b0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:51.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.303+0000 7f703a800700 1 --2- 172.21.15.1:0/168462134 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7034068010 0x7f7034068470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:51.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.303+0000 7f703a800700 1 -- 172.21.15.1:0/168462134 >> 172.21.15.1:0/168462134 conn(0x7f70340757d0 msgr2=0x7f7034075be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:51.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.303+0000 7f703a800700 1 -- 172.21.15.1:0/168462134 shutdown_connections 2024-02-02T15:35:51.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.303+0000 7f703a800700 1 -- 172.21.15.1:0/168462134 wait complete. 2024-02-02T15:35:51.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.304+0000 7f703a800700 1 Processor -- start 2024-02-02T15:35:51.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.304+0000 7f703a800700 1 -- start start 2024-02-02T15:35:51.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.305+0000 7f703a800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7034068010 0x7f70340fff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:51.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.305+0000 7f703a800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f70340689b0 0x7f7034100480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:51.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.305+0000 7f703a800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7034106ed0 0x7f70341a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:51.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.305+0000 7f703a800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7034100cb0 con 0x7f70340689b0 2024-02-02T15:35:51.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.305+0000 7f703a800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70341a56e0 con 0x7f7034106ed0 2024-02-02T15:35:51.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.305+0000 7f703a800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70341a5820 con 0x7f7034068010 2024-02-02T15:35:51.305 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.305+0000 7f7033e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7034106ed0 0x7f70341a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:51.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.305+0000 7f7033400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7034068010 0x7f70340fff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:51.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.305+0000 7f7033400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7034068010 0x7f70340fff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:44392/0 (socket says 172.21.15.1:44392) 2024-02-02T15:35:51.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.305+0000 7f7033e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7034106ed0 0x7f70341a51a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42272/0 (socket says 172.21.15.1:42272) 2024-02-02T15:35:51.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.305+0000 7f7033400700 1 -- 172.21.15.1:0/463563721 learned_addr learned my addr 172.21.15.1:0/463563721 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:35:51.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.305+0000 7f7032a00700 1 --2- 172.21.15.1:0/463563721 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f70340689b0 0x7f7034100480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:51.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.305+0000 7f7033e00700 1 -- 172.21.15.1:0/463563721 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7034068010 msgr2=0x7f70340fff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:51.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.305+0000 7f7033e00700 1 --2- 172.21.15.1:0/463563721 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7034068010 0x7f70340fff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:51.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.305+0000 7f7033e00700 1 -- 172.21.15.1:0/463563721 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f70340689b0 msgr2=0x7f7034100480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:51.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.305+0000 7f7033e00700 1 --2- 172.21.15.1:0/463563721 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f70340689b0 0x7f7034100480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:51.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.305+0000 7f7033e00700 1 -- 172.21.15.1:0/463563721 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f701c009520 con 0x7f7034106ed0 2024-02-02T15:35:51.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.306+0000 7f7033e00700 1 --2- 172.21.15.1:0/463563721 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7034106ed0 0x7f70341a51a0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f702c00efc0 tx=0x7f702c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:51.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.306+0000 7f702be00700 1 -- 172.21.15.1:0/463563721 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f702c018070 con 0x7f7034106ed0 2024-02-02T15:35:51.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.306+0000 7f702be00700 1 -- 172.21.15.1:0/463563721 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f702c00f040 con 0x7f7034106ed0 2024-02-02T15:35:51.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.306+0000 7f703a800700 1 -- 172.21.15.1:0/463563721 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70240095f0 con 0x7f7034106ed0 2024-02-02T15:35:51.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.306+0000 7f702be00700 1 -- 172.21.15.1:0/463563721 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f702c00cd40 con 0x7f7034106ed0 2024-02-02T15:35:51.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.306+0000 7f703a800700 1 -- 172.21.15.1:0/463563721 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70341a5d70 con 0x7f7034106ed0 2024-02-02T15:35:51.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.309+0000 7f702be00700 1 -- 172.21.15.1:0/463563721 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f702c003a00 con 0x7f7034106ed0 2024-02-02T15:35:51.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.311+0000 7f704e2a4b80 1 Processor -- start 2024-02-02T15:35:51.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.311+0000 7f704e2a4b80 1 -- start start 2024-02-02T15:35:51.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.311+0000 7f702be00700 1 --2- 172.21.15.1:0/463563721 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f70140a0690 0x7f70140a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:51.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.312+0000 7f702be00700 1 -- 172.21.15.1:0/463563721 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f702c0cbc50 con 0x7f7034106ed0 2024-02-02T15:35:51.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.312+0000 7f704e2a4b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55990b680010 0x55990b6803f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:51.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.312+0000 7f704e2a4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55990b680930 0x55990b68b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:51.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.312+0000 7f704e2a4b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55990b68b7e0 0x55990b68dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:51.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.312+0000 7f704e2a4b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55990b5f44c0 con 0x55990b680010 2024-02-02T15:35:51.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.312+0000 7f704e2a4b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55990b5d8aa0 con 0x55990b68b7e0 2024-02-02T15:35:51.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.312+0000 7f704e2a4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55990b5d6460 con 0x55990b680930 2024-02-02T15:35:51.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.312+0000 7f7032a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55990b680930 0x55990b68b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:51.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.312+0000 7f7032a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55990b680930 0x55990b68b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:44394/0 (socket says 172.21.15.1:44394) 2024-02-02T15:35:51.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.312+0000 7f7032a00700 1 -- 172.21.15.1:0/888518481 learned_addr learned my addr 172.21.15.1:0/888518481 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:35:51.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.312+0000 7f7033e00700 1 --2- 172.21.15.1:0/888518481 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55990b68b7e0 0x55990b68dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:51.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.312+0000 7f7033400700 1 --2- 172.21.15.1:0/888518481 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55990b680010 0x55990b6803f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:51.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.312+0000 7f7032a00700 1 --2- 172.21.15.1:0/463563721 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f70140a0690 0x7f70140a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:51.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.312+0000 7f7033e00700 1 -- 172.21.15.1:0/888518481 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55990b680930 msgr2=0x55990b68b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:51.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.312+0000 7f7033e00700 1 --2- 172.21.15.1:0/888518481 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55990b680930 0x55990b68b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:51.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.312+0000 7f7033e00700 1 -- 172.21.15.1:0/888518481 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55990b680010 msgr2=0x55990b6803f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:51.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.312+0000 7f7033e00700 1 --2- 172.21.15.1:0/888518481 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55990b680010 0x55990b6803f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:51.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.312+0000 7f7033e00700 1 -- 172.21.15.1:0/888518481 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55990b4533b0 con 0x55990b68b7e0 2024-02-02T15:35:51.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.313+0000 7f7033e00700 1 --2- 172.21.15.1:0/888518481 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55990b68b7e0 0x55990b68dbd0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f702c0c8f70 tx=0x7f702c0951c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a2fa1f951a88fab server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:51.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.313+0000 7f7032a00700 1 --2- 172.21.15.1:0/463563721 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f70140a0690 0x7f70140a2b50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f701c009fd0 tx=0x7f701c009260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:51.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.313+0000 7f7028c00700 1 -- 172.21.15.1:0/888518481 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f702c01b320 con 0x55990b68b7e0 2024-02-02T15:35:51.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.313+0000 7f7028c00700 1 -- 172.21.15.1:0/888518481 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f702c004ad0 con 0x55990b68b7e0 2024-02-02T15:35:51.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.313+0000 7f7028c00700 1 -- 172.21.15.1:0/888518481 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f702c0a0e70 con 0x55990b68b7e0 2024-02-02T15:35:51.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.314+0000 7f704e2a4b80 1 -- 172.21.15.1:0/888518481 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55990b68b7e0 msgr2=0x55990b68dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:51.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.314+0000 7f704e2a4b80 1 --2- 172.21.15.1:0/888518481 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55990b68b7e0 0x55990b68dbd0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f702c0c8f70 tx=0x7f702c0951c0 comp rx=0 tx=0).stop 2024-02-02T15:35:51.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.314+0000 7f704e2a4b80 1 -- 172.21.15.1:0/888518481 shutdown_connections 2024-02-02T15:35:51.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.314+0000 7f704e2a4b80 1 --2- 172.21.15.1:0/888518481 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55990b68b7e0 0x55990b68dbd0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:51.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.314+0000 7f704e2a4b80 1 --2- 172.21.15.1:0/888518481 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55990b680010 0x55990b6803f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:51.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.314+0000 7f704e2a4b80 1 --2- 172.21.15.1:0/888518481 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55990b680930 0x55990b68b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:51.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.314+0000 7f704e2a4b80 1 -- 172.21.15.1:0/888518481 >> 172.21.15.1:0/888518481 conn(0x55990b5dfdb0 msgr2=0x55990b67c3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:51.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.314+0000 7f704e2a4b80 1 -- 172.21.15.1:0/888518481 shutdown_connections 2024-02-02T15:35:51.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.314+0000 7f704e2a4b80 1 -- 172.21.15.1:0/888518481 wait complete. 2024-02-02T15:35:51.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.314+0000 7f704e2a4b80 1 -- start start 2024-02-02T15:35:51.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.315+0000 7f704e2a4b80 1 Processor -- start 2024-02-02T15:35:51.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.315+0000 7f704e2a4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55990b680010 0x55990b68b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:51.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.315+0000 7f704e2a4b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55990b680930 0x55990b6884c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:51.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.315+0000 7f704e2a4b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55990b68b7e0 0x55990b688c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:51.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.315+0000 7f704e2a4b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55990b5d1d50 con 0x55990b680930 2024-02-02T15:35:51.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.315+0000 7f704e2a4b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55990b56dd80 con 0x55990b68b7e0 2024-02-02T15:35:51.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.315+0000 7f704e2a4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55990b56da20 con 0x55990b680010 2024-02-02T15:35:51.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.315+0000 7f704e2a4b80 10 client.0 _ll_register_callbacks cb 0x55990b56f290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:35:51.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.315+0000 7f7033400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55990b680010 0x55990b68b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:51.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.315+0000 7f7033e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55990b68b7e0 0x55990b688c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:51.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.315+0000 7f7033400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55990b680010 0x55990b68b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:44406/0 (socket says 172.21.15.1:44406) 2024-02-02T15:35:51.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.315+0000 7f7033400700 1 -- 172.21.15.1:0/3866733220 learned_addr learned my addr 172.21.15.1:0/3866733220 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:35:51.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.316+0000 7f7032a00700 1 --2- 172.21.15.1:0/3866733220 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55990b680930 0x55990b6884c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:51.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.316+0000 7f7033400700 1 -- 172.21.15.1:0/3866733220 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55990b68b7e0 msgr2=0x55990b688c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:51.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.316+0000 7f7033400700 1 --2- 172.21.15.1:0/3866733220 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55990b68b7e0 0x55990b688c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:51.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.316+0000 7f7033400700 1 -- 172.21.15.1:0/3866733220 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55990b680930 msgr2=0x55990b6884c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:51.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.316+0000 7f7033400700 1 --2- 172.21.15.1:0/3866733220 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55990b680930 0x55990b6884c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:51.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.316+0000 7f7033400700 1 -- 172.21.15.1:0/3866733220 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70341a5d70 con 0x55990b680010 2024-02-02T15:35:51.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.316+0000 7f7033400700 1 --2- 172.21.15.1:0/3866733220 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55990b680010 0x55990b68b240 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f7024003f90 tx=0x7f7024005bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd873755fa69d075 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:51.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.316+0000 7f7021600700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:35:51.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.316+0000 7f7021600700 1 -- 172.21.15.1:0/3866733220 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7024005580 con 0x55990b680010 2024-02-02T15:35:51.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.317+0000 7f7021600700 1 -- 172.21.15.1:0/3866733220 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f702400b790 con 0x55990b680010 2024-02-02T15:35:51.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.317+0000 7f704e2a4b80 1 -- 172.21.15.1:0/3866733220 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55990b6740f0 con 0x55990b680010 2024-02-02T15:35:51.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.317+0000 7f7021600700 1 -- 172.21.15.1:0/3866733220 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f70240056e0 con 0x55990b680010 2024-02-02T15:35:51.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.317+0000 7f704e2a4b80 10 client.28042 fetch_fsmap learned FSMap version 43 2024-02-02T15:35:51.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.317+0000 7f704e2a4b80 1 -- 172.21.15.1:0/3866733220 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55990b3ffb60 con 0x55990b680010 2024-02-02T15:35:51.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.317+0000 7f7021600700 1 -- 172.21.15.1:0/3866733220 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55990b6740f0 con 0x55990b680010 2024-02-02T15:35:51.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.318+0000 7f7021600700 1 -- 172.21.15.1:0/3866733220 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 43) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f702400fb00 con 0x55990b680010 2024-02-02T15:35:51.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.318+0000 7f704e2a4b80 10 client.28042 fetch_fsmap finished waiting for FSMap version 43 2024-02-02T15:35:51.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.318+0000 7f704e2a4b80 10 client.28042 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:35:51.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.318+0000 7f704e2a4b80 20 client.28042 populate_metadata read hostname 'smithi001' 2024-02-02T15:35:51.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.318+0000 7f704e2a4b80 1 --2- 172.21.15.1:0/3866733220 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55990b69b250 0x55990b69d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:51.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.318+0000 7f704e2a4b80 4 client.28042 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:35:51.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.318+0000 7f704e2a4b80 1 -- 172.21.15.1:0/3866733220 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55990b5cadd0 con 0x55990b69b250 2024-02-02T15:35:51.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.318+0000 7f7032a00700 1 --2- 172.21.15.1:0/3866733220 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55990b69b250 0x55990b69d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:51.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.319+0000 7f7032a00700 1 --2- 172.21.15.1:0/3866733220 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55990b69b250 0x55990b69d670 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=440d54067ddae9c8 server_cookie=65be1a6730780761 in_seq=0 out_seq=0 2024-02-02T15:35:51.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.319+0000 7f7021600700 10 client.28042 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:35:51.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.319+0000 7f7021600700 1 -- 172.21.15.1:0/3866733220 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55990b5cadd0 con 0x55990b69b250 2024-02-02T15:35:51.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.319+0000 7f7021600700 10 client.28042 handle_command_reply: tid=0 2024-02-02T15:35:51.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.319+0000 7f704e2a4b80 1 client.28042 shutdown 2024-02-02T15:35:51.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.320+0000 7f704e2a4b80 1 -- 172.21.15.1:0/3866733220 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55990b680010 msgr2=0x55990b68b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:51.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.320+0000 7f704e2a4b80 1 --2- 172.21.15.1:0/3866733220 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55990b680010 0x55990b68b240 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f7024003f90 tx=0x7f7024005bc0 comp rx=0 tx=0).stop 2024-02-02T15:35:51.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.320+0000 7f704e2a4b80 1 -- 172.21.15.1:0/3866733220 shutdown_connections 2024-02-02T15:35:51.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.320+0000 7f704e2a4b80 1 --2- 172.21.15.1:0/3866733220 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55990b69b250 0x55990b69d670 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:51.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.320+0000 7f704e2a4b80 1 --2- 172.21.15.1:0/3866733220 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55990b68b7e0 0x55990b688c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:51.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.320+0000 7f704e2a4b80 1 --2- 172.21.15.1:0/3866733220 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55990b680010 0x55990b68b240 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:51.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.320+0000 7f7021600700 0 client.28042 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:35:51.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.320+0000 7f704e2a4b80 1 --2- 172.21.15.1:0/3866733220 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55990b680930 0x55990b6884c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:51.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.320+0000 7f704e2a4b80 1 -- 172.21.15.1:0/3866733220 >> 172.21.15.1:0/3866733220 conn(0x55990b5dfdb0 msgr2=0x55990b67bfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:51.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.320+0000 7f704e2a4b80 1 -- 172.21.15.1:0/3866733220 shutdown_connections 2024-02-02T15:35:51.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.320+0000 7f704e2a4b80 1 -- 172.21.15.1:0/3866733220 wait complete. 2024-02-02T15:35:51.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.320+0000 7f704e2a4b80 20 client.28042 trim_cache size 0 max 16384 2024-02-02T15:35:51.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.341+0000 7f704e2a4b80 1 Processor -- start 2024-02-02T15:35:51.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.341+0000 7f704e2a4b80 1 -- start start 2024-02-02T15:35:51.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.341+0000 7f704e2a4b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55990b680010 0x55990b6884c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:51.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.341+0000 7f704e2a4b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55990b680930 0x55990b682f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:51.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.341+0000 7f704e2a4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55990b68b7e0 0x55990b68ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:51.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.341+0000 7f704e2a4b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55990b4556a0 con 0x55990b680930 2024-02-02T15:35:51.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.341+0000 7f704e2a4b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55990b673fc0 con 0x55990b680010 2024-02-02T15:35:51.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.341+0000 7f704e2a4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55990b676500 con 0x55990b68b7e0 2024-02-02T15:35:51.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.342+0000 7f7032a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55990b680930 0x55990b682f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:51.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.342+0000 7f7033400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55990b680010 0x55990b6884c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:51.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.342+0000 7f7033e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55990b68b7e0 0x55990b68ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:51.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.342+0000 7f7033400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55990b680010 0x55990b6884c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42296/0 (socket says 172.21.15.1:42296) 2024-02-02T15:35:51.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.342+0000 7f7032a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55990b680930 0x55990b682f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49504/0 (socket says 172.21.15.1:49504) 2024-02-02T15:35:51.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.342+0000 7f7033400700 1 -- 172.21.15.1:0/491086035 learned_addr learned my addr 172.21.15.1:0/491086035 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:35:51.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.342+0000 7f7033400700 1 -- 172.21.15.1:0/491086035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55990b68b7e0 msgr2=0x55990b68ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:51.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.342+0000 7f7033400700 1 --2- 172.21.15.1:0/491086035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55990b68b7e0 0x55990b68ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:51.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.342+0000 7f7033400700 1 -- 172.21.15.1:0/491086035 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55990b680930 msgr2=0x55990b682f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:51.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.342+0000 7f7033400700 1 --2- 172.21.15.1:0/491086035 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55990b680930 0x55990b682f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:51.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.342+0000 7f7033400700 1 -- 172.21.15.1:0/491086035 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55990b697890 con 0x55990b680010 2024-02-02T15:35:51.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.342+0000 7f7033400700 1 --2- 172.21.15.1:0/491086035 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55990b680010 0x55990b6884c0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f7024009350 tx=0x7f7024005a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4619cb4b767efd2 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:51.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.343+0000 7f7029600700 1 -- 172.21.15.1:0/491086035 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70240324e0 con 0x55990b680010 2024-02-02T15:35:51.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.343+0000 7f7029600700 1 -- 172.21.15.1:0/491086035 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f702401e040 con 0x55990b680010 2024-02-02T15:35:51.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.343+0000 7f7029600700 1 -- 172.21.15.1:0/491086035 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7024017700 con 0x55990b680010 2024-02-02T15:35:51.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.343+0000 7f704e2a4b80 1 -- 172.21.15.1:0/491086035 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55990b680010 msgr2=0x55990b6884c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:51.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.343+0000 7f704e2a4b80 1 --2- 172.21.15.1:0/491086035 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55990b680010 0x55990b6884c0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f7024009350 tx=0x7f7024005a90 comp rx=0 tx=0).stop 2024-02-02T15:35:51.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.343+0000 7f704e2a4b80 1 -- 172.21.15.1:0/491086035 shutdown_connections 2024-02-02T15:35:51.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.343+0000 7f704e2a4b80 1 --2- 172.21.15.1:0/491086035 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55990b68b7e0 0x55990b68ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:51.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.343+0000 7f704e2a4b80 1 --2- 172.21.15.1:0/491086035 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55990b680010 0x55990b6884c0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:51.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.343+0000 7f704e2a4b80 1 --2- 172.21.15.1:0/491086035 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55990b680930 0x55990b682f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:51.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.343+0000 7f704e2a4b80 1 -- 172.21.15.1:0/491086035 >> 172.21.15.1:0/491086035 conn(0x55990b5dfdb0 msgr2=0x55990b687db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:51.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.344+0000 7f704e2a4b80 1 -- 172.21.15.1:0/491086035 shutdown_connections 2024-02-02T15:35:51.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.344+0000 7f704e2a4b80 1 -- 172.21.15.1:0/491086035 wait complete. 2024-02-02T15:35:51.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.344+0000 7f704e2a4b80 1 -- start start 2024-02-02T15:35:51.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.344+0000 7f704e2a4b80 1 Processor -- start 2024-02-02T15:35:51.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.345+0000 7f704e2a4b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55990b680010 0x55990b692250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:51.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.345+0000 7f704e2a4b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55990b680930 0x55990b6904d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:51.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.345+0000 7f704e2a4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55990b68b7e0 0x55990b690b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:51.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.345+0000 7f704e2a4b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55990b6911d0 con 0x55990b680010 2024-02-02T15:35:51.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.345+0000 7f704e2a4b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55990b691340 con 0x55990b680930 2024-02-02T15:35:51.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.345+0000 7f704e2a4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55990b6ecd80 con 0x55990b68b7e0 2024-02-02T15:35:51.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.345+0000 7f704e2a4b80 10 client.0 _ll_register_callbacks cb 0x55990b56f290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:35:51.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.345+0000 7f7032a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55990b680930 0x55990b6904d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:51.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.345+0000 7f7033400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55990b680010 0x55990b692250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:51.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.345+0000 7f7032a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55990b680930 0x55990b6904d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42300/0 (socket says 172.21.15.1:42300) 2024-02-02T15:35:51.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.345+0000 7f7032a00700 1 -- 172.21.15.1:0/408979076 learned_addr learned my addr 172.21.15.1:0/408979076 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:35:51.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.345+0000 7f7033e00700 1 --2- 172.21.15.1:0/408979076 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55990b68b7e0 0x55990b690b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:51.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.345+0000 7f7032a00700 1 -- 172.21.15.1:0/408979076 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55990b68b7e0 msgr2=0x55990b690b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:51.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.345+0000 7f7032a00700 1 --2- 172.21.15.1:0/408979076 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55990b68b7e0 0x55990b690b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:51.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.345+0000 7f7032a00700 1 -- 172.21.15.1:0/408979076 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55990b680010 msgr2=0x55990b692250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:35:51.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.345+0000 7f7032a00700 1 --2- 172.21.15.1:0/408979076 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55990b680010 0x55990b692250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:51.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.345+0000 7f7032a00700 1 -- 172.21.15.1:0/408979076 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55990b3ffb60 con 0x55990b680930 2024-02-02T15:35:51.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.346+0000 7f7032a00700 1 --2- 172.21.15.1:0/408979076 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55990b680930 0x55990b6904d0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f701c011a30 tx=0x7f701c0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf364c972ee6cdaf server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:35:51.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.346+0000 7f7021600700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:35:51.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.346+0000 7f7021600700 1 -- 172.21.15.1:0/408979076 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f701c026040 con 0x55990b680930 2024-02-02T15:35:51.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.346+0000 7f7021600700 1 -- 172.21.15.1:0/408979076 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f701c00fe10 con 0x55990b680930 2024-02-02T15:35:51.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.346+0000 7f704e2a4b80 1 -- 172.21.15.1:0/408979076 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f702c004ad0 con 0x55990b680930 2024-02-02T15:35:51.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.346+0000 7f7021600700 1 -- 172.21.15.1:0/408979076 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f701c004310 con 0x55990b680930 2024-02-02T15:35:51.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.346+0000 7f704e2a4b80 10 client.28058 fetch_fsmap learned FSMap version 43 2024-02-02T15:35:51.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.346+0000 7f704e2a4b80 1 -- 172.21.15.1:0/408979076 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55990b597ea0 con 0x55990b680930 2024-02-02T15:35:51.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.346+0000 7f7021600700 1 -- 172.21.15.1:0/408979076 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f702c004ad0 con 0x55990b680930 2024-02-02T15:35:51.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.347+0000 7f7021600700 1 -- 172.21.15.1:0/408979076 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 43) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f701c0051e0 con 0x55990b680930 2024-02-02T15:35:51.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.347+0000 7f704e2a4b80 10 client.28058 fetch_fsmap finished waiting for FSMap version 43 2024-02-02T15:35:51.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.347+0000 7f704e2a4b80 10 client.28058 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:35:51.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.347+0000 7f704e2a4b80 20 client.28058 populate_metadata read hostname 'smithi001' 2024-02-02T15:35:51.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.347+0000 7f704e2a4b80 1 --2- 172.21.15.1:0/408979076 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55990b6ee5d0 0x55990b6f09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:35:51.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.347+0000 7f704e2a4b80 4 client.28058 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2024-02-02T15:35:51.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.347+0000 7f704e2a4b80 1 -- 172.21.15.1:0/408979076 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55990b590f50 con 0x55990b6ee5d0 2024-02-02T15:35:51.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.347+0000 7f7033400700 1 --2- 172.21.15.1:0/408979076 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55990b6ee5d0 0x55990b6f09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:35:51.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.347+0000 7f7033400700 1 --2- 172.21.15.1:0/408979076 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55990b6ee5d0 0x55990b6f09f0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44e825bdb66c2e8e server_cookie=762e586760a445e3 in_seq=0 out_seq=0 2024-02-02T15:35:51.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.347+0000 7f7021600700 10 client.28058 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:35:51.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.348+0000 7f7021600700 1 -- 172.21.15.1:0/408979076 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55990b590f50 con 0x55990b6ee5d0 2024-02-02T15:35:51.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.348+0000 7f7021600700 10 client.28058 handle_command_reply: tid=0 2024-02-02T15:35:51.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.348+0000 7f704e2a4b80 1 client.28058 shutdown 2024-02-02T15:35:51.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.349+0000 7f704e2a4b80 1 -- 172.21.15.1:0/408979076 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55990b680930 msgr2=0x55990b6904d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:51.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.349+0000 7f704e2a4b80 1 --2- 172.21.15.1:0/408979076 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55990b680930 0x55990b6904d0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f701c011a30 tx=0x7f701c0047c0 comp rx=0 tx=0).stop 2024-02-02T15:35:51.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.349+0000 7f704e2a4b80 1 -- 172.21.15.1:0/408979076 shutdown_connections 2024-02-02T15:35:51.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.349+0000 7f704e2a4b80 1 --2- 172.21.15.1:0/408979076 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55990b6ee5d0 0x55990b6f09f0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:51.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.349+0000 7f704e2a4b80 1 --2- 172.21.15.1:0/408979076 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55990b68b7e0 0x55990b690b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:51.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.349+0000 7f704e2a4b80 1 --2- 172.21.15.1:0/408979076 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55990b680010 0x55990b692250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:51.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.349+0000 7f7021600700 0 client.28058 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:35:51.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.349+0000 7f704e2a4b80 1 --2- 172.21.15.1:0/408979076 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55990b680930 0x55990b6904d0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:51.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.349+0000 7f704e2a4b80 1 -- 172.21.15.1:0/408979076 >> 172.21.15.1:0/408979076 conn(0x55990b5dfdb0 msgr2=0x55990b686a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:51.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.349+0000 7f704e2a4b80 1 -- 172.21.15.1:0/408979076 shutdown_connections 2024-02-02T15:35:51.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.349+0000 7f704e2a4b80 1 -- 172.21.15.1:0/408979076 wait complete. 2024-02-02T15:35:51.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.349+0000 7f704e2a4b80 20 client.28058 trim_cache size 0 max 16384 2024-02-02T15:35:51.351 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:35:51.351 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-02T15:35:51.351 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2024-02-02T15:35:51.352 INFO:teuthology.orchestra.run.smithi001.stdout: "a940f11a-50d4-4565-a1d2-b2551f200a13": { 2024-02-02T15:35:51.352 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2024-02-02T15:35:51.352 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "a940f11a-50d4-4565-a1d2-b2551f200a13", 2024-02-02T15:35:51.352 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,force" 2024-02-02T15:35:51.352 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:35:51.352 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:35:51.352 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:35:51.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.350+0000 7f703a800700 1 -- 172.21.15.1:0/463563721 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f70140a0690 msgr2=0x7f70140a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:51.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.350+0000 7f703a800700 1 --2- 172.21.15.1:0/463563721 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f70140a0690 0x7f70140a2b50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f701c009fd0 tx=0x7f701c009260 comp rx=0 tx=0).stop 2024-02-02T15:35:51.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.350+0000 7f703a800700 1 -- 172.21.15.1:0/463563721 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7034106ed0 msgr2=0x7f70341a51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:35:51.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.350+0000 7f703a800700 1 --2- 172.21.15.1:0/463563721 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7034106ed0 0x7f70341a51a0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f702c00efc0 tx=0x7f702c00c3a0 comp rx=0 tx=0).stop 2024-02-02T15:35:51.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.350+0000 7f703a800700 1 -- 172.21.15.1:0/463563721 shutdown_connections 2024-02-02T15:35:51.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.350+0000 7f703a800700 1 --2- 172.21.15.1:0/463563721 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f70140a0690 0x7f70140a2b50 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:51.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.350+0000 7f703a800700 1 --2- 172.21.15.1:0/463563721 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7034106ed0 0x7f70341a51a0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:51.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.350+0000 7f703a800700 1 --2- 172.21.15.1:0/463563721 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7034068010 0x7f70340fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:51.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.350+0000 7f703a800700 1 --2- 172.21.15.1:0/463563721 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f70340689b0 0x7f7034100480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:35:51.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.350+0000 7f703a800700 1 -- 172.21.15.1:0/463563721 >> 172.21.15.1:0/463563721 conn(0x7f70340757d0 msgr2=0x7f70340fea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:35:51.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.350+0000 7f703a800700 1 -- 172.21.15.1:0/463563721 shutdown_connections 2024-02-02T15:35:51.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:35:51.350+0000 7f703a800700 1 -- 172.21.15.1:0/463563721 wait complete. 2024-02-02T15:35:51.368 INFO:tasks.cephfs.filesystem:scrub status for tag:a940f11a-50d4-4565-a1d2-b2551f200a13 - {'path': '/', 'tag': 'a940f11a-50d4-4565-a1d2-b2551f200a13', 'options': 'recursive,force'} 2024-02-02T15:35:51.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:51 smithi071 ceph-mon[44707]: pgmap v272: 129 pgs: 129 active+clean; 382 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.8 MiB/s wr, 170 op/s 2024-02-02T15:35:51.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:51 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:35:51.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:51 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:35:51.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:51 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:35:51.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:51 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:35:51.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:51 smithi142 ceph-mon[45728]: pgmap v272: 129 pgs: 129 active+clean; 382 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.8 MiB/s wr, 170 op/s 2024-02-02T15:35:51.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:51 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:35:51.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:51 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:35:51.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:51 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:35:51.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:51 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:35:51.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:51 smithi001 ceph-mon[35470]: pgmap v272: 129 pgs: 129 active+clean; 382 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.8 MiB/s wr, 170 op/s 2024-02-02T15:35:51.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:51 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:35:51.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:51 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:35:51.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:51 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:35:51.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:51 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:35:52.252 INFO:tasks.workunit.client.0.smithi001.stdout: 1 524822 6.40 MB/sec execute 326 sec latency 26.808 ms 2024-02-02T15:35:52.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:52 smithi071 ceph-mon[44707]: scrub queued for path: / 2024-02-02T15:35:52.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:52 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:35:52.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:52 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:35:52.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:52 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:35:52.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:52 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:35:52.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:52 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:35:52.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:52 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:35:52.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:52 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:35:52.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:52 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:35:52.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:52 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:35:52.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:52 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:35:52.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:52 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:35:52.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:52 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:35:52.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:52 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:35:52.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:52 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:35:52.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:52 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:35:52.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:52 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:35:52.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:52 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:35:52.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:52 smithi142 ceph-mon[45728]: scrub queued for path: / 2024-02-02T15:35:52.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:52 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:35:52.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:52 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:35:52.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:52 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:35:52.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:52 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:35:52.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:52 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:35:52.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:52 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:35:52.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:52 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:35:52.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:52 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:35:52.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:52 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:35:52.922 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:52 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:35:52.922 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:52 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:35:52.922 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:52 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:35:52.922 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:52 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:35:52.922 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:52 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:35:52.922 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:52 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:35:52.923 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:52 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:35:52.923 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:52 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:35:52.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:52 smithi001 ceph-mon[35470]: scrub queued for path: / 2024-02-02T15:35:52.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:52 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:35:52.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:52 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:35:52.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:52 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:35:52.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:52 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:35:52.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:52 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:35:52.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:52 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:35:52.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:52 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:35:52.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:52 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:35:52.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:52 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:35:52.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:52 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:35:52.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:52 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:35:52.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:52 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:35:52.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:52 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:35:52.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:52 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:35:52.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:52 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:35:52.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:52 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:35:52.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:52 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:35:53.252 INFO:tasks.workunit.client.0.smithi001.stdout: 1 526207 6.42 MB/sec execute 327 sec latency 34.333 ms 2024-02-02T15:35:53.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:53 smithi071 ceph-mon[44707]: pgmap v273: 129 pgs: 129 active+clean; 389 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.9 MiB/s wr, 266 op/s 2024-02-02T15:35:53.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:53 smithi142 ceph-mon[45728]: pgmap v273: 129 pgs: 129 active+clean; 389 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.9 MiB/s wr, 266 op/s 2024-02-02T15:35:53.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:53 smithi001 ceph-mon[35470]: pgmap v273: 129 pgs: 129 active+clean; 389 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 2.9 MiB/s wr, 266 op/s 2024-02-02T15:35:54.252 INFO:tasks.workunit.client.0.smithi001.stdout: 1 527540 6.41 MB/sec execute 328 sec latency 19.279 ms 2024-02-02T15:35:55.253 INFO:tasks.workunit.client.0.smithi001.stdout: 1 529517 6.43 MB/sec execute 329 sec latency 14.880 ms 2024-02-02T15:35:55.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:55 smithi071 ceph-mon[44707]: pgmap v274: 129 pgs: 129 active+clean; 389 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.0 MiB/s wr, 186 op/s 2024-02-02T15:35:55.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:55 smithi142 ceph-mon[45728]: pgmap v274: 129 pgs: 129 active+clean; 389 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.0 MiB/s wr, 186 op/s 2024-02-02T15:35:55.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:55 smithi001 ceph-mon[35470]: pgmap v274: 129 pgs: 129 active+clean; 389 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.0 MiB/s wr, 186 op/s 2024-02-02T15:35:56.253 INFO:tasks.workunit.client.0.smithi001.stdout: 1 530470 6.42 MB/sec execute 330 sec latency 17.719 ms 2024-02-02T15:35:56.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:56 smithi071 ceph-mon[44707]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:35:56.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:56 smithi142 ceph-mon[45728]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:35:56.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:56 smithi001 ceph-mon[35470]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:35:57.252 INFO:tasks.workunit.client.0.smithi001.stdout: 1 530999 6.41 MB/sec execute 331 sec latency 363.513 ms 2024-02-02T15:35:57.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:57 smithi071 ceph-mon[44707]: pgmap v275: 129 pgs: 129 active+clean; 395 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 282 op/s 2024-02-02T15:35:57.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:57 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:35:57.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:57 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:35:57.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:57 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:35:57.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:57 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:35:57.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:57 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:35:57.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:57 smithi142 ceph-mon[45728]: pgmap v275: 129 pgs: 129 active+clean; 395 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 282 op/s 2024-02-02T15:35:57.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:57 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:35:57.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:57 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:35:57.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:57 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:35:57.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:57 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:35:57.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:57 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:35:57.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:57 smithi001 ceph-mon[35470]: pgmap v275: 129 pgs: 129 active+clean; 395 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 282 op/s 2024-02-02T15:35:57.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:57 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:35:57.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:57 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:35:57.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:57 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:35:57.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:57 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:35:57.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:57 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:35:58.253 INFO:tasks.workunit.client.0.smithi001.stdout: 1 532217 6.41 MB/sec execute 332 sec latency 753.720 ms 2024-02-02T15:35:58.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:35:58 smithi071 ceph-mon[44707]: pgmap v276: 129 pgs: 129 active+clean; 395 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 192 op/s 2024-02-02T15:35:58.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:35:58 smithi142 ceph-mon[45728]: pgmap v276: 129 pgs: 129 active+clean; 395 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 192 op/s 2024-02-02T15:35:58.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:35:58 smithi001 ceph-mon[35470]: pgmap v276: 129 pgs: 129 active+clean; 395 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 192 op/s 2024-02-02T15:35:59.253 INFO:tasks.workunit.client.0.smithi001.stdout: 1 533367 6.42 MB/sec execute 333 sec latency 15.911 ms 2024-02-02T15:36:00.253 INFO:tasks.workunit.client.0.smithi001.stdout: 1 534445 6.41 MB/sec execute 334 sec latency 20.175 ms 2024-02-02T15:36:01.253 INFO:tasks.workunit.client.0.smithi001.stdout: 1 534579 6.39 MB/sec execute 335 sec latency 801.184 ms 2024-02-02T15:36:01.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:01 smithi071 ceph-mon[44707]: pgmap v277: 129 pgs: 129 active+clean; 395 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 192 op/s 2024-02-02T15:36:01.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:01 smithi142 ceph-mon[45728]: pgmap v277: 129 pgs: 129 active+clean; 395 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 192 op/s 2024-02-02T15:36:01.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:01 smithi001 ceph-mon[35470]: pgmap v277: 129 pgs: 129 active+clean; 395 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 192 op/s 2024-02-02T15:36:02.253 INFO:tasks.workunit.client.0.smithi001.stdout: 1 534579 6.37 MB/sec execute 336 sec latency 1801.320 ms 2024-02-02T15:36:03.253 INFO:tasks.workunit.client.0.smithi001.stdout: 1 535173 6.36 MB/sec execute 337 sec latency 2186.078 ms 2024-02-02T15:36:03.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:03 smithi142 ceph-mon[45728]: pgmap v278: 129 pgs: 129 active+clean; 399 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 280 op/s 2024-02-02T15:36:03.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:03 smithi142 ceph-mon[45728]: osdmap e94: 12 total, 12 up, 12 in 2024-02-02T15:36:03.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:03 smithi001 ceph-mon[35470]: pgmap v278: 129 pgs: 129 active+clean; 399 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 280 op/s 2024-02-02T15:36:03.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:03 smithi001 ceph-mon[35470]: osdmap e94: 12 total, 12 up, 12 in 2024-02-02T15:36:03.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:03 smithi071 ceph-mon[44707]: pgmap v278: 129 pgs: 129 active+clean; 399 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.2 MiB/s wr, 280 op/s 2024-02-02T15:36:03.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:03 smithi071 ceph-mon[44707]: osdmap e94: 12 total, 12 up, 12 in 2024-02-02T15:36:04.253 INFO:tasks.workunit.client.0.smithi001.stdout: 1 536539 6.37 MB/sec execute 338 sec latency 54.550 ms 2024-02-02T15:36:05.253 INFO:tasks.workunit.client.0.smithi001.stdout: 1 537489 6.36 MB/sec execute 339 sec latency 13.103 ms 2024-02-02T15:36:05.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:05 smithi142 ceph-mon[45728]: pgmap v280: 129 pgs: 129 active+clean; 399 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 220 op/s 2024-02-02T15:36:05.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:05 smithi001 ceph-mon[35470]: pgmap v280: 129 pgs: 129 active+clean; 399 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 220 op/s 2024-02-02T15:36:05.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:05 smithi071 ceph-mon[44707]: pgmap v280: 129 pgs: 129 active+clean; 399 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.6 MiB/s wr, 220 op/s 2024-02-02T15:36:06.254 INFO:tasks.workunit.client.0.smithi001.stdout: 1 538122 6.36 MB/sec execute 340 sec latency 567.384 ms 2024-02-02T15:36:07.254 INFO:tasks.workunit.client.0.smithi001.stdout: 1 538122 6.34 MB/sec execute 341 sec latency 1567.476 ms 2024-02-02T15:36:07.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:07 smithi142 ceph-mon[45728]: pgmap v281: 129 pgs: 129 active+clean; 400 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 192 op/s 2024-02-02T15:36:07.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:07 smithi001 ceph-mon[35470]: pgmap v281: 129 pgs: 129 active+clean; 400 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 192 op/s 2024-02-02T15:36:07.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:07 smithi071 ceph-mon[44707]: pgmap v281: 129 pgs: 129 active+clean; 400 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 192 op/s 2024-02-02T15:36:08.254 INFO:tasks.workunit.client.0.smithi001.stdout: 1 539358 6.34 MB/sec execute 342 sec latency 1959.747 ms 2024-02-02T15:36:09.254 INFO:tasks.workunit.client.0.smithi001.stdout: 1 540599 6.34 MB/sec execute 343 sec latency 22.503 ms 2024-02-02T15:36:09.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:09 smithi142 ceph-mon[45728]: pgmap v282: 129 pgs: 129 active+clean; 400 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 192 op/s 2024-02-02T15:36:09.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:09 smithi001 ceph-mon[35470]: pgmap v282: 129 pgs: 129 active+clean; 400 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 192 op/s 2024-02-02T15:36:09.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:09 smithi071 ceph-mon[44707]: pgmap v282: 129 pgs: 129 active+clean; 400 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 192 op/s 2024-02-02T15:36:10.254 INFO:tasks.workunit.client.0.smithi001.stdout: 1 541671 6.34 MB/sec execute 344 sec latency 248.861 ms 2024-02-02T15:36:11.254 INFO:tasks.workunit.client.0.smithi001.stdout: 1 542404 6.32 MB/sec execute 345 sec latency 915.065 ms 2024-02-02T15:36:11.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:11 smithi142 ceph-mon[45728]: pgmap v283: 129 pgs: 129 active+clean; 400 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 192 op/s 2024-02-02T15:36:11.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:11 smithi142 ceph-mon[45728]: osdmap e95: 12 total, 12 up, 12 in 2024-02-02T15:36:11.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:11 smithi001 ceph-mon[35470]: pgmap v283: 129 pgs: 129 active+clean; 400 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 192 op/s 2024-02-02T15:36:11.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:11 smithi001 ceph-mon[35470]: osdmap e95: 12 total, 12 up, 12 in 2024-02-02T15:36:11.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:11 smithi071 ceph-mon[44707]: pgmap v283: 129 pgs: 129 active+clean; 400 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.0 MiB/s wr, 192 op/s 2024-02-02T15:36:11.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:11 smithi071 ceph-mon[44707]: osdmap e95: 12 total, 12 up, 12 in 2024-02-02T15:36:12.254 INFO:tasks.workunit.client.0.smithi001.stdout: 1 543913 6.34 MB/sec execute 346 sec latency 25.333 ms 2024-02-02T15:36:13.254 INFO:tasks.workunit.client.0.smithi001.stdout: 1 545039 6.34 MB/sec execute 347 sec latency 18.920 ms 2024-02-02T15:36:13.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:13 smithi142 ceph-mon[45728]: pgmap v285: 129 pgs: 129 active+clean; 405 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 174 op/s 2024-02-02T15:36:13.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:13 smithi001 ceph-mon[35470]: pgmap v285: 129 pgs: 129 active+clean; 405 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 174 op/s 2024-02-02T15:36:13.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:13 smithi071 ceph-mon[44707]: pgmap v285: 129 pgs: 129 active+clean; 405 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.1 MiB/s wr, 174 op/s 2024-02-02T15:36:14.255 INFO:tasks.workunit.client.0.smithi001.stdout: 1 545212 6.32 MB/sec execute 348 sec latency 865.177 ms 2024-02-02T15:36:15.255 INFO:tasks.workunit.client.0.smithi001.stdout: 1 545212 6.30 MB/sec execute 349 sec latency 1865.310 ms 2024-02-02T15:36:15.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:15 smithi142 ceph-mon[45728]: pgmap v286: 129 pgs: 129 active+clean; 405 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.1 MiB/s wr, 172 op/s 2024-02-02T15:36:15.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:15 smithi001 ceph-mon[35470]: pgmap v286: 129 pgs: 129 active+clean; 405 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.1 MiB/s wr, 172 op/s 2024-02-02T15:36:15.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:15 smithi071 ceph-mon[44707]: pgmap v286: 129 pgs: 129 active+clean; 405 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.1 MiB/s wr, 172 op/s 2024-02-02T15:36:16.255 INFO:tasks.workunit.client.0.smithi001.stdout: 1 545212 6.28 MB/sec execute 350 sec latency 2865.399 ms 2024-02-02T15:36:16.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:16 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:36:16.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:16 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:36:16.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:16 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:36:17.255 INFO:tasks.workunit.client.0.smithi001.stdout: 1 545212 6.27 MB/sec execute 351 sec latency 3865.508 ms 2024-02-02T15:36:17.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:17 smithi142 ceph-mon[45728]: pgmap v287: 129 pgs: 129 active+clean; 408 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 160 op/s 2024-02-02T15:36:17.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:17 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:36:17.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:17 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:36:17.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:17 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:36:17.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:17 smithi001 ceph-mon[35470]: pgmap v287: 129 pgs: 129 active+clean; 408 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 160 op/s 2024-02-02T15:36:17.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:17 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:36:17.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:17 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:36:17.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:17 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:36:17.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:17 smithi071 ceph-mon[44707]: pgmap v287: 129 pgs: 129 active+clean; 408 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 160 op/s 2024-02-02T15:36:17.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:17 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:36:17.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:17 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:36:17.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:17 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:36:18.255 INFO:tasks.workunit.client.0.smithi001.stdout: 1 546409 6.27 MB/sec execute 352 sec latency 4254.707 ms 2024-02-02T15:36:19.255 INFO:tasks.workunit.client.0.smithi001.stdout: 1 547572 6.27 MB/sec execute 353 sec latency 18.529 ms 2024-02-02T15:36:19.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:19 smithi142 ceph-mon[45728]: pgmap v288: 129 pgs: 129 active+clean; 408 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 160 op/s 2024-02-02T15:36:19.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:19 smithi001 ceph-mon[35470]: pgmap v288: 129 pgs: 129 active+clean; 408 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 160 op/s 2024-02-02T15:36:19.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:19 smithi071 ceph-mon[44707]: pgmap v288: 129 pgs: 129 active+clean; 408 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 160 op/s 2024-02-02T15:36:20.256 INFO:tasks.workunit.client.0.smithi001.stdout: 1 548762 6.27 MB/sec execute 354 sec latency 120.659 ms 2024-02-02T15:36:21.256 INFO:tasks.workunit.client.0.smithi001.stdout: 1 548762 6.25 MB/sec execute 355 sec latency 1120.774 ms 2024-02-02T15:36:21.369 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-02T15:36:21.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.583+0000 7fc92b200700 1 -- 172.21.15.1:0/1822668620 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc924100ff0 msgr2=0x7fc9241013d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:21.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.583+0000 7fc92b200700 1 --2- 172.21.15.1:0/1822668620 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc924100ff0 0x7fc9241013d0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fc914009990 tx=0x7fc914009ca0 comp rx=0 tx=0).stop 2024-02-02T15:36:21.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.583+0000 7fc92b200700 1 -- 172.21.15.1:0/1822668620 shutdown_connections 2024-02-02T15:36:21.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.583+0000 7fc92b200700 1 --2- 172.21.15.1:0/1822668620 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc924106110 0x7fc92410dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:21.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.583+0000 7fc92b200700 1 --2- 172.21.15.1:0/1822668620 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc924100ff0 0x7fc9241013d0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:21.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.583+0000 7fc92b200700 1 --2- 172.21.15.1:0/1822668620 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc9241019a0 0x7fc9241059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:21.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.583+0000 7fc92b200700 1 -- 172.21.15.1:0/1822668620 >> 172.21.15.1:0/1822668620 conn(0x7fc9240fc810 msgr2=0x7fc9240fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:21.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.583+0000 7fc92b200700 1 -- 172.21.15.1:0/1822668620 shutdown_connections 2024-02-02T15:36:21.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.583+0000 7fc92b200700 1 -- 172.21.15.1:0/1822668620 wait complete. 2024-02-02T15:36:21.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.584+0000 7fc92b200700 1 Processor -- start 2024-02-02T15:36:21.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.584+0000 7fc92b200700 1 -- start start 2024-02-02T15:36:21.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.584+0000 7fc92b200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc924100ff0 0x7fc924196e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:21.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.584+0000 7fc92b200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc924106110 0x7fc924197350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:21.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.584+0000 7fc92b200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc92419b7b0 0x7fc92419bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:21.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.584+0000 7fc92b200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc92419c230 con 0x7fc924106110 2024-02-02T15:36:21.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.584+0000 7fc92b200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc92419c3a0 con 0x7fc92419b7b0 2024-02-02T15:36:21.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.584+0000 7fc92b200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc92419c560 con 0x7fc924100ff0 2024-02-02T15:36:21.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.585+0000 7fc923400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc924106110 0x7fc924197350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:21.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.585+0000 7fc923e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc924100ff0 0x7fc924196e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:21.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.585+0000 7fc923e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc924100ff0 0x7fc924196e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:45358/0 (socket says 172.21.15.1:45358) 2024-02-02T15:36:21.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.585+0000 7fc923400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc924106110 0x7fc924197350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48872/0 (socket says 172.21.15.1:48872) 2024-02-02T15:36:21.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.585+0000 7fc923e00700 1 -- 172.21.15.1:0/2917712896 learned_addr learned my addr 172.21.15.1:0/2917712896 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:36:21.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.585+0000 7fc929200700 1 --2- 172.21.15.1:0/2917712896 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc92419b7b0 0x7fc92419bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:21.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.585+0000 7fc923e00700 1 -- 172.21.15.1:0/2917712896 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc92419b7b0 msgr2=0x7fc92419bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:21.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.585+0000 7fc923e00700 1 --2- 172.21.15.1:0/2917712896 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc92419b7b0 0x7fc92419bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:21.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.585+0000 7fc923e00700 1 -- 172.21.15.1:0/2917712896 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc924106110 msgr2=0x7fc924197350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:21.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.585+0000 7fc923e00700 1 --2- 172.21.15.1:0/2917712896 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc924106110 0x7fc924197350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:21.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.585+0000 7fc923e00700 1 -- 172.21.15.1:0/2917712896 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9140095f0 con 0x7fc924100ff0 2024-02-02T15:36:21.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.585+0000 7fc923e00700 1 --2- 172.21.15.1:0/2917712896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc924100ff0 0x7fc924196e10 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fc914005750 tx=0x7fc914005780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:21.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.586+0000 7fc920c00700 1 -- 172.21.15.1:0/2917712896 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc91401d070 con 0x7fc924100ff0 2024-02-02T15:36:21.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.586+0000 7fc92b200700 1 -- 172.21.15.1:0/2917712896 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9241a3a60 con 0x7fc924100ff0 2024-02-02T15:36:21.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.586+0000 7fc920c00700 1 -- 172.21.15.1:0/2917712896 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc914022470 con 0x7fc924100ff0 2024-02-02T15:36:21.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.586+0000 7fc920c00700 1 -- 172.21.15.1:0/2917712896 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc914017650 con 0x7fc924100ff0 2024-02-02T15:36:21.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.586+0000 7fc92b200700 1 -- 172.21.15.1:0/2917712896 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9241a3f30 con 0x7fc924100ff0 2024-02-02T15:36:21.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.590+0000 7fc920c00700 1 -- 172.21.15.1:0/2917712896 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7fc9140225e0 con 0x7fc924100ff0 2024-02-02T15:36:21.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.591+0000 7fc93edb3b80 1 Processor -- start 2024-02-02T15:36:21.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.591+0000 7fc93edb3b80 1 -- start start 2024-02-02T15:36:21.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.592+0000 7fc93edb3b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf593c3010 0x55bf593c33f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:21.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.592+0000 7fc93edb3b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bf593c3930 0x55bf593ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:21.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.592+0000 7fc93edb3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf593ce7e0 0x55bf593d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:21.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.592+0000 7fc93edb3b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf593374c0 con 0x55bf593c3010 2024-02-02T15:36:21.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.592+0000 7fc93edb3b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf5931baa0 con 0x55bf593c3930 2024-02-02T15:36:21.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.592+0000 7fc93edb3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf59319460 con 0x55bf593ce7e0 2024-02-02T15:36:21.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.592+0000 7fc923e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bf593c3930 0x55bf593ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:21.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.592+0000 7fc929200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf593c3010 0x55bf593c33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:21.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.592+0000 7fc920c00700 1 --2- 172.21.15.1:0/2917712896 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fc9040a0690 0x7fc9040a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:21.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.592+0000 7fc923e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bf593c3930 0x55bf593ce2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:57768/0 (socket says 172.21.15.1:57768) 2024-02-02T15:36:21.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.592+0000 7fc929200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf593c3010 0x55bf593c33f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48874/0 (socket says 172.21.15.1:48874) 2024-02-02T15:36:21.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.592+0000 7fc923e00700 1 -- 172.21.15.1:0/1584128839 learned_addr learned my addr 172.21.15.1:0/1584128839 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:36:21.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.592+0000 7fc923400700 1 --2- 172.21.15.1:0/1584128839 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf593ce7e0 0x55bf593d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:21.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.592+0000 7fc920c00700 1 -- 172.21.15.1:0/2917712896 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fc9140cd380 con 0x7fc924100ff0 2024-02-02T15:36:21.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.592+0000 7fc923e00700 1 -- 172.21.15.1:0/1584128839 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf593ce7e0 msgr2=0x55bf593d0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:21.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.592+0000 7fc929200700 1 --2- 172.21.15.1:0/2917712896 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fc9040a0690 0x7fc9040a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:21.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.592+0000 7fc923e00700 1 --2- 172.21.15.1:0/1584128839 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf593ce7e0 0x55bf593d0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:21.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.593+0000 7fc923e00700 1 -- 172.21.15.1:0/1584128839 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf593c3010 msgr2=0x55bf593c33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:21.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.593+0000 7fc923e00700 1 --2- 172.21.15.1:0/1584128839 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf593c3010 0x55bf593c33f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:21.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.593+0000 7fc923e00700 1 -- 172.21.15.1:0/1584128839 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf591963b0 con 0x55bf593c3930 2024-02-02T15:36:21.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.593+0000 7fc923400700 1 --2- 172.21.15.1:0/1584128839 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf593ce7e0 0x55bf593d0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:36:21.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.593+0000 7fc923e00700 1 --2- 172.21.15.1:0/1584128839 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bf593c3930 0x55bf593ce2a0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fc91409a6f0 tx=0x7fc914005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36500b52a8ef7a01 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:21.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.593+0000 7fc919600700 1 -- 172.21.15.1:0/1584128839 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc91409a070 con 0x55bf593c3930 2024-02-02T15:36:21.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.593+0000 7fc919600700 1 -- 172.21.15.1:0/1584128839 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc914099e90 con 0x55bf593c3930 2024-02-02T15:36:21.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.593+0000 7fc919600700 1 -- 172.21.15.1:0/1584128839 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9140850f0 con 0x55bf593c3930 2024-02-02T15:36:21.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.594+0000 7fc93edb3b80 1 -- 172.21.15.1:0/1584128839 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bf593c3930 msgr2=0x55bf593ce2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:21.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.594+0000 7fc929200700 1 --2- 172.21.15.1:0/2917712896 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fc9040a0690 0x7fc9040a2b50 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc91c009ec0 tx=0x7fc91c004b40 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:21.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.594+0000 7fc93edb3b80 1 --2- 172.21.15.1:0/1584128839 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bf593c3930 0x55bf593ce2a0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fc91409a6f0 tx=0x7fc914005fd0 comp rx=0 tx=0).stop 2024-02-02T15:36:21.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.594+0000 7fc93edb3b80 1 -- 172.21.15.1:0/1584128839 shutdown_connections 2024-02-02T15:36:21.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.594+0000 7fc93edb3b80 1 --2- 172.21.15.1:0/1584128839 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf593ce7e0 0x55bf593d0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:21.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.594+0000 7fc93edb3b80 1 --2- 172.21.15.1:0/1584128839 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf593c3010 0x55bf593c33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:21.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.594+0000 7fc93edb3b80 1 --2- 172.21.15.1:0/1584128839 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bf593c3930 0x55bf593ce2a0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:21.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.594+0000 7fc93edb3b80 1 -- 172.21.15.1:0/1584128839 >> 172.21.15.1:0/1584128839 conn(0x55bf59322db0 msgr2=0x55bf593bf3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:21.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.594+0000 7fc93edb3b80 1 -- 172.21.15.1:0/1584128839 shutdown_connections 2024-02-02T15:36:21.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.594+0000 7fc93edb3b80 1 -- 172.21.15.1:0/1584128839 wait complete. 2024-02-02T15:36:21.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.594+0000 7fc93edb3b80 1 -- start start 2024-02-02T15:36:21.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.595+0000 7fc93edb3b80 1 Processor -- start 2024-02-02T15:36:21.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.595+0000 7fc93edb3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf593c3010 0x55bf593cebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:21.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.595+0000 7fc93edb3b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf593c3930 0x55bf593cf100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:21.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.595+0000 7fc93edb3b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bf593ce7e0 0x55bf593cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:21.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.595+0000 7fc93edb3b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf59314d50 con 0x55bf593c3930 2024-02-02T15:36:21.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.595+0000 7fc93edb3b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf592b0d80 con 0x55bf593ce7e0 2024-02-02T15:36:21.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.595+0000 7fc93edb3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf592b0a20 con 0x55bf593c3010 2024-02-02T15:36:21.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.595+0000 7fc93edb3b80 10 client.0 _ll_register_callbacks cb 0x55bf592b2290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:36:21.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.595+0000 7fc923400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf593c3930 0x55bf593cf100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:21.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.595+0000 7fc923400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf593c3930 0x55bf593cf100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48880/0 (socket says 172.21.15.1:48880) 2024-02-02T15:36:21.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.595+0000 7fc923400700 1 -- 172.21.15.1:0/2603962249 learned_addr learned my addr 172.21.15.1:0/2603962249 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:36:21.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.595+0000 7fc923e00700 1 --2- 172.21.15.1:0/2603962249 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf593c3010 0x55bf593cebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:21.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.595+0000 7fc929200700 1 --2- 172.21.15.1:0/2603962249 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bf593ce7e0 0x55bf593cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:21.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.596+0000 7fc923400700 1 -- 172.21.15.1:0/2603962249 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf593c3010 msgr2=0x55bf593cebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:21.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.596+0000 7fc923400700 1 --2- 172.21.15.1:0/2603962249 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf593c3010 0x55bf593cebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:21.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.596+0000 7fc923400700 1 -- 172.21.15.1:0/2603962249 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bf593ce7e0 msgr2=0x55bf593cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:21.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.596+0000 7fc923400700 1 --2- 172.21.15.1:0/2603962249 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bf593ce7e0 0x55bf593cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:21.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.596+0000 7fc923400700 1 -- 172.21.15.1:0/2603962249 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc90c0098e0 con 0x55bf593c3930 2024-02-02T15:36:21.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.596+0000 7fc923400700 1 --2- 172.21.15.1:0/2603962249 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf593c3930 0x55bf593cf100 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fc90c011970 tx=0x7fc90c011c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f65daffba826a1b1 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:21.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.596+0000 7fc912000700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:36:21.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.596+0000 7fc912000700 1 -- 172.21.15.1:0/2603962249 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc90c016070 con 0x55bf593c3930 2024-02-02T15:36:21.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.596+0000 7fc912000700 1 -- 172.21.15.1:0/2603962249 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fc90c004300 con 0x55bf593c3930 2024-02-02T15:36:21.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.596+0000 7fc93edb3b80 1 -- 172.21.15.1:0/2603962249 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bf593b70f0 con 0x55bf593c3930 2024-02-02T15:36:21.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.596+0000 7fc912000700 1 -- 172.21.15.1:0/2603962249 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc90c0044c0 con 0x55bf593c3930 2024-02-02T15:36:21.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.597+0000 7fc93edb3b80 10 client.17979 fetch_fsmap learned FSMap version 44 2024-02-02T15:36:21.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.597+0000 7fc93edb3b80 1 -- 172.21.15.1:0/2603962249 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x7fc9241a3f30 con 0x55bf593c3930 2024-02-02T15:36:21.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.597+0000 7fc912000700 1 -- 172.21.15.1:0/2603962249 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bf593b70f0 con 0x55bf593c3930 2024-02-02T15:36:21.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.597+0000 7fc912000700 1 -- 172.21.15.1:0/2603962249 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 44) v1 ==== 7510+0+0 (secure 0 0 0) 0x7fc90c005020 con 0x55bf593c3930 2024-02-02T15:36:21.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.597+0000 7fc93edb3b80 10 client.17979 fetch_fsmap finished waiting for FSMap version 44 2024-02-02T15:36:21.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.597+0000 7fc93edb3b80 10 client.17979 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:36:21.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.597+0000 7fc93edb3b80 20 client.17979 populate_metadata read hostname 'smithi001' 2024-02-02T15:36:21.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.597+0000 7fc93edb3b80 1 --2- 172.21.15.1:0/2603962249 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55bf593ddf40 0x55bf593e0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:21.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.597+0000 7fc93edb3b80 4 client.17979 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:36:21.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.597+0000 7fc93edb3b80 1 -- 172.21.15.1:0/2603962249 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bf5930ddd0 con 0x55bf593ddf40 2024-02-02T15:36:21.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.597+0000 7fc923e00700 1 --2- 172.21.15.1:0/2603962249 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55bf593ddf40 0x55bf593e0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:21.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.598+0000 7fc923e00700 1 --2- 172.21.15.1:0/2603962249 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55bf593ddf40 0x55bf593e0360 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=980accab79dd5393 server_cookie=9dc281fe22c13c6f in_seq=0 out_seq=0 2024-02-02T15:36:21.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.598+0000 7fc912000700 10 client.17979 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:36:21.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.598+0000 7fc912000700 1 -- 172.21.15.1:0/2603962249 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55bf5930ddd0 con 0x55bf593ddf40 2024-02-02T15:36:21.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.598+0000 7fc912000700 10 client.17979 handle_command_reply: tid=0 2024-02-02T15:36:21.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.599+0000 7fc93edb3b80 1 client.17979 shutdown 2024-02-02T15:36:21.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.599+0000 7fc93edb3b80 1 -- 172.21.15.1:0/2603962249 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf593c3930 msgr2=0x55bf593cf100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:21.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.599+0000 7fc93edb3b80 1 --2- 172.21.15.1:0/2603962249 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf593c3930 0x55bf593cf100 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fc90c011970 tx=0x7fc90c011c80 comp rx=0 tx=0).stop 2024-02-02T15:36:21.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.599+0000 7fc93edb3b80 1 -- 172.21.15.1:0/2603962249 shutdown_connections 2024-02-02T15:36:21.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.599+0000 7fc93edb3b80 1 --2- 172.21.15.1:0/2603962249 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55bf593ddf40 0x55bf593e0360 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:21.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.599+0000 7fc93edb3b80 1 --2- 172.21.15.1:0/2603962249 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bf593ce7e0 0x55bf593cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:21.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.599+0000 7fc93edb3b80 1 --2- 172.21.15.1:0/2603962249 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf593c3010 0x55bf593cebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:21.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.599+0000 7fc912000700 0 client.17979 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:36:21.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.599+0000 7fc93edb3b80 1 --2- 172.21.15.1:0/2603962249 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf593c3930 0x55bf593cf100 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:21.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.599+0000 7fc93edb3b80 1 -- 172.21.15.1:0/2603962249 >> 172.21.15.1:0/2603962249 conn(0x55bf59322db0 msgr2=0x55bf593befa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:21.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.599+0000 7fc93edb3b80 1 -- 172.21.15.1:0/2603962249 shutdown_connections 2024-02-02T15:36:21.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.599+0000 7fc93edb3b80 1 -- 172.21.15.1:0/2603962249 wait complete. 2024-02-02T15:36:21.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.599+0000 7fc93edb3b80 20 client.17979 trim_cache size 0 max 16384 2024-02-02T15:36:21.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.616+0000 7fc93edb3b80 1 Processor -- start 2024-02-02T15:36:21.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.616+0000 7fc93edb3b80 1 -- start start 2024-02-02T15:36:21.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.617+0000 7fc93edb3b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf59322db0 0x55bf593c8130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:21.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.617+0000 7fc93edb3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf593c3010 0x55bf593d91b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:21.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.617+0000 7fc93edb3b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bf593c3930 0x55bf593d96f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:21.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.617+0000 7fc93edb3b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf591986a0 con 0x55bf59322db0 2024-02-02T15:36:21.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.617+0000 7fc93edb3b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf593c4260 con 0x55bf593c3930 2024-02-02T15:36:21.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.617+0000 7fc93edb3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf593c0910 con 0x55bf593c3010 2024-02-02T15:36:21.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.617+0000 7fc923e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf59322db0 0x55bf593c8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:21.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.617+0000 7fc923e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf59322db0 0x55bf593c8130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48890/0 (socket says 172.21.15.1:48890) 2024-02-02T15:36:21.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.617+0000 7fc923e00700 1 -- 172.21.15.1:0/1204467380 learned_addr learned my addr 172.21.15.1:0/1204467380 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:36:21.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.617+0000 7fc923400700 1 --2- 172.21.15.1:0/1204467380 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf593c3010 0x55bf593d91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:21.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.617+0000 7fc929200700 1 --2- 172.21.15.1:0/1204467380 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bf593c3930 0x55bf593d96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:21.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.617+0000 7fc923e00700 1 -- 172.21.15.1:0/1204467380 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf593c3010 msgr2=0x55bf593d91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:21.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.618+0000 7fc923e00700 1 --2- 172.21.15.1:0/1204467380 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf593c3010 0x55bf593d91b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:21.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.618+0000 7fc923e00700 1 -- 172.21.15.1:0/1204467380 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bf593c3930 msgr2=0x55bf593d96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:21.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.618+0000 7fc923e00700 1 --2- 172.21.15.1:0/1204467380 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bf593c3930 0x55bf593d96f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:21.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.618+0000 7fc923e00700 1 -- 172.21.15.1:0/1204467380 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf59142b60 con 0x55bf59322db0 2024-02-02T15:36:21.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.618+0000 7fc923e00700 1 --2- 172.21.15.1:0/1204467380 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf59322db0 0x55bf593c8130 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fc914096a30 tx=0x7fc914085040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc216fd99dea033f server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:21.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.618+0000 7fc91a000700 1 -- 172.21.15.1:0/1204467380 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc91409ac30 con 0x55bf59322db0 2024-02-02T15:36:21.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.618+0000 7fc91a000700 1 -- 172.21.15.1:0/1204467380 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc91409e120 con 0x55bf59322db0 2024-02-02T15:36:21.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.618+0000 7fc91a000700 1 -- 172.21.15.1:0/1204467380 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9140228e0 con 0x55bf59322db0 2024-02-02T15:36:21.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.619+0000 7fc93edb3b80 1 -- 172.21.15.1:0/1204467380 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf59322db0 msgr2=0x55bf593c8130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:21.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.619+0000 7fc93edb3b80 1 --2- 172.21.15.1:0/1204467380 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf59322db0 0x55bf593c8130 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fc914096a30 tx=0x7fc914085040 comp rx=0 tx=0).stop 2024-02-02T15:36:21.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.619+0000 7fc93edb3b80 1 -- 172.21.15.1:0/1204467380 shutdown_connections 2024-02-02T15:36:21.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.619+0000 7fc93edb3b80 1 --2- 172.21.15.1:0/1204467380 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bf593c3930 0x55bf593d96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:21.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.619+0000 7fc93edb3b80 1 --2- 172.21.15.1:0/1204467380 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf59322db0 0x55bf593c8130 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:21.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.619+0000 7fc93edb3b80 1 --2- 172.21.15.1:0/1204467380 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf593c3010 0x55bf593d91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:21.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.619+0000 7fc93edb3b80 1 -- 172.21.15.1:0/1204467380 >> 172.21.15.1:0/1204467380 conn(0x55bf593cd480 msgr2=0x55bf593cf450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:21.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.619+0000 7fc93edb3b80 1 -- 172.21.15.1:0/1204467380 shutdown_connections 2024-02-02T15:36:21.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.619+0000 7fc93edb3b80 1 -- 172.21.15.1:0/1204467380 wait complete. 2024-02-02T15:36:21.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.619+0000 7fc93edb3b80 1 -- start start 2024-02-02T15:36:21.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.619+0000 7fc93edb3b80 1 Processor -- start 2024-02-02T15:36:21.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.620+0000 7fc93edb3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf59322db0 0x55bf593d3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:21.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.620+0000 7fc93edb3b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf593c3010 0x55bf593cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:21.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.620+0000 7fc93edb3b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bf593c3930 0x55bf593d0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:21.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.620+0000 7fc93edb3b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf593d05a0 con 0x55bf593c3010 2024-02-02T15:36:21.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.620+0000 7fc93edb3b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf593d06e0 con 0x55bf593c3930 2024-02-02T15:36:21.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.620+0000 7fc93edb3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf593d08a0 con 0x55bf59322db0 2024-02-02T15:36:21.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.620+0000 7fc93edb3b80 10 client.0 _ll_register_callbacks cb 0x55bf592b2290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:36:21.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.620+0000 7fc923e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf59322db0 0x55bf593d3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:21.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.620+0000 7fc923400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf593c3010 0x55bf593cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:21.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.620+0000 7fc923e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf59322db0 0x55bf593d3810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:45400/0 (socket says 172.21.15.1:45400) 2024-02-02T15:36:21.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.620+0000 7fc929200700 1 --2- 172.21.15.1:0/856996344 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bf593c3930 0x55bf593d0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:21.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.620+0000 7fc923e00700 1 -- 172.21.15.1:0/856996344 learned_addr learned my addr 172.21.15.1:0/856996344 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:36:21.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.620+0000 7fc923400700 1 -- 172.21.15.1:0/856996344 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf59322db0 msgr2=0x55bf593d3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:21.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.620+0000 7fc923400700 1 --2- 172.21.15.1:0/856996344 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf59322db0 0x55bf593d3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:21.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.620+0000 7fc923400700 1 -- 172.21.15.1:0/856996344 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bf593c3930 msgr2=0x55bf593d0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:21.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.620+0000 7fc923400700 1 --2- 172.21.15.1:0/856996344 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bf593c3930 0x55bf593d0060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:21.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.620+0000 7fc923400700 1 -- 172.21.15.1:0/856996344 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bf591963b0 con 0x55bf593c3010 2024-02-02T15:36:21.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.621+0000 7fc923400700 1 --2- 172.21.15.1:0/856996344 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf593c3010 0x55bf593cfb20 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fc90c005380 tx=0x7fc90c0057d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=679cd9f736e218ec server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:21.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.621+0000 7fc912000700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:36:21.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.621+0000 7fc912000700 1 -- 172.21.15.1:0/856996344 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc90c016040 con 0x55bf593c3010 2024-02-02T15:36:21.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.621+0000 7fc912000700 1 -- 172.21.15.1:0/856996344 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fc90c000e40 con 0x55bf593c3010 2024-02-02T15:36:21.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.621+0000 7fc93edb3b80 1 -- 172.21.15.1:0/856996344 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc914099e90 con 0x55bf593c3010 2024-02-02T15:36:21.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.621+0000 7fc912000700 1 -- 172.21.15.1:0/856996344 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc90c00fb00 con 0x55bf593c3010 2024-02-02T15:36:21.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.621+0000 7fc93edb3b80 10 client.17985 fetch_fsmap learned FSMap version 44 2024-02-02T15:36:21.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.621+0000 7fc93edb3b80 1 -- 172.21.15.1:0/856996344 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x55bf592daea0 con 0x55bf593c3010 2024-02-02T15:36:21.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.622+0000 7fc912000700 1 -- 172.21.15.1:0/856996344 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc914099e90 con 0x55bf593c3010 2024-02-02T15:36:21.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.622+0000 7fc912000700 1 -- 172.21.15.1:0/856996344 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 44) v1 ==== 7510+0+0 (secure 0 0 0) 0x7fc90c032470 con 0x55bf593c3010 2024-02-02T15:36:21.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.622+0000 7fc93edb3b80 10 client.17985 fetch_fsmap finished waiting for FSMap version 44 2024-02-02T15:36:21.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.622+0000 7fc93edb3b80 10 client.17985 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:36:21.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.622+0000 7fc93edb3b80 20 client.17985 populate_metadata read hostname 'smithi001' 2024-02-02T15:36:21.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.622+0000 7fc93edb3b80 1 --2- 172.21.15.1:0/856996344 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55bf593ce7e0 0x55bf59432fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:21.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.622+0000 7fc93edb3b80 4 client.17985 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2024-02-02T15:36:21.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.622+0000 7fc93edb3b80 1 -- 172.21.15.1:0/856996344 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55bf592d3f50 con 0x55bf593ce7e0 2024-02-02T15:36:21.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.622+0000 7fc923e00700 1 --2- 172.21.15.1:0/856996344 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55bf593ce7e0 0x55bf59432fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:21.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.623+0000 7fc923e00700 1 --2- 172.21.15.1:0/856996344 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55bf593ce7e0 0x55bf59432fe0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=154c1dc59b3d0316 server_cookie=e6240eb079f44eb9 in_seq=0 out_seq=0 2024-02-02T15:36:21.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.623+0000 7fc912000700 10 client.17985 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:36:21.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.623+0000 7fc912000700 1 -- 172.21.15.1:0/856996344 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55bf592d3f50 con 0x55bf593ce7e0 2024-02-02T15:36:21.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.623+0000 7fc912000700 10 client.17985 handle_command_reply: tid=0 2024-02-02T15:36:21.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.623+0000 7fc93edb3b80 1 client.17985 shutdown 2024-02-02T15:36:21.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.623+0000 7fc93edb3b80 1 -- 172.21.15.1:0/856996344 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf593c3010 msgr2=0x55bf593cfb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:21.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.623+0000 7fc93edb3b80 1 --2- 172.21.15.1:0/856996344 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf593c3010 0x55bf593cfb20 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fc90c005380 tx=0x7fc90c0057d0 comp rx=0 tx=0).stop 2024-02-02T15:36:21.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.623+0000 7fc93edb3b80 1 -- 172.21.15.1:0/856996344 shutdown_connections 2024-02-02T15:36:21.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.623+0000 7fc93edb3b80 1 --2- 172.21.15.1:0/856996344 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55bf593ce7e0 0x55bf59432fe0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:21.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.623+0000 7fc93edb3b80 1 --2- 172.21.15.1:0/856996344 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bf593c3930 0x55bf593d0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:21.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.623+0000 7fc93edb3b80 1 --2- 172.21.15.1:0/856996344 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bf59322db0 0x55bf593d3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:21.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.623+0000 7fc912000700 0 client.17985 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:36:21.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.623+0000 7fc93edb3b80 1 --2- 172.21.15.1:0/856996344 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bf593c3010 0x55bf593cfb20 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:21.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.623+0000 7fc93edb3b80 1 -- 172.21.15.1:0/856996344 >> 172.21.15.1:0/856996344 conn(0x55bf593cd480 msgr2=0x55bf593c8130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:21.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.623+0000 7fc93edb3b80 1 -- 172.21.15.1:0/856996344 shutdown_connections 2024-02-02T15:36:21.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.623+0000 7fc93edb3b80 1 -- 172.21.15.1:0/856996344 wait complete. 2024-02-02T15:36:21.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.623+0000 7fc93edb3b80 20 client.17985 trim_cache size 0 max 16384 2024-02-02T15:36:21.628 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:36:21.628 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2024-02-02T15:36:21.628 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2024-02-02T15:36:21.628 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:36:21.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.624+0000 7fc92b200700 1 -- 172.21.15.1:0/2917712896 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fc9040a0690 msgr2=0x7fc9040a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:21.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.624+0000 7fc92b200700 1 --2- 172.21.15.1:0/2917712896 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fc9040a0690 0x7fc9040a2b50 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc91c009ec0 tx=0x7fc91c004b40 comp rx=0 tx=0).stop 2024-02-02T15:36:21.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.624+0000 7fc92b200700 1 -- 172.21.15.1:0/2917712896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc924100ff0 msgr2=0x7fc924196e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:21.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.624+0000 7fc92b200700 1 --2- 172.21.15.1:0/2917712896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc924100ff0 0x7fc924196e10 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fc914005750 tx=0x7fc914005780 comp rx=0 tx=0).stop 2024-02-02T15:36:21.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.624+0000 7fc92b200700 1 -- 172.21.15.1:0/2917712896 shutdown_connections 2024-02-02T15:36:21.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.624+0000 7fc92b200700 1 --2- 172.21.15.1:0/2917712896 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fc9040a0690 0x7fc9040a2b50 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:21.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.624+0000 7fc92b200700 1 --2- 172.21.15.1:0/2917712896 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc92419b7b0 0x7fc92419bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:21.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.624+0000 7fc92b200700 1 --2- 172.21.15.1:0/2917712896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc924100ff0 0x7fc924196e10 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:21.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.624+0000 7fc92b200700 1 --2- 172.21.15.1:0/2917712896 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc924106110 0x7fc924197350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:21.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.624+0000 7fc92b200700 1 -- 172.21.15.1:0/2917712896 >> 172.21.15.1:0/2917712896 conn(0x7fc9240fc810 msgr2=0x7fc9240fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:21.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.624+0000 7fc92b200700 1 -- 172.21.15.1:0/2917712896 shutdown_connections 2024-02-02T15:36:21.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.624+0000 7fc92b200700 1 -- 172.21.15.1:0/2917712896 wait complete. 2024-02-02T15:36:21.642 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-02T15:36:21.642 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T15:36:21.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:21 smithi001 ceph-mon[35470]: pgmap v289: 129 pgs: 129 active+clean; 408 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 160 op/s 2024-02-02T15:36:21.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.841+0000 7f9633c00700 1 -- 172.21.15.1:0/4209329643 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f962c100f10 msgr2=0x7f962c1012f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:21.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.841+0000 7f9633c00700 1 --2- 172.21.15.1:0/4209329643 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f962c100f10 0x7f962c1012f0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f961c009990 tx=0x7f961c009ca0 comp rx=0 tx=0).stop 2024-02-02T15:36:21.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.841+0000 7f9633c00700 1 -- 172.21.15.1:0/4209329643 shutdown_connections 2024-02-02T15:36:21.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.841+0000 7f9633c00700 1 --2- 172.21.15.1:0/4209329643 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f962c106030 0x7f962c10dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:21.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.841+0000 7f9633c00700 1 --2- 172.21.15.1:0/4209329643 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f962c100f10 0x7f962c1012f0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:21.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.841+0000 7f9633c00700 1 --2- 172.21.15.1:0/4209329643 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f962c1018c0 0x7f962c105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:21.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.841+0000 7f9633c00700 1 -- 172.21.15.1:0/4209329643 >> 172.21.15.1:0/4209329643 conn(0x7f962c0fc790 msgr2=0x7f962c0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:21.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.841+0000 7f9633c00700 1 -- 172.21.15.1:0/4209329643 shutdown_connections 2024-02-02T15:36:21.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.841+0000 7f9633c00700 1 -- 172.21.15.1:0/4209329643 wait complete. 2024-02-02T15:36:21.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.842+0000 7f9633c00700 1 Processor -- start 2024-02-02T15:36:21.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.843+0000 7f9633c00700 1 -- start start 2024-02-02T15:36:21.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.843+0000 7f9633c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f962c100f10 0x7f962c199030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:21.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.843+0000 7f9633c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f962c106030 0x7f962c199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:21.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.843+0000 7f9633c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f962c19d9d0 0x7f962c19de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:21.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.843+0000 7f9633c00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f962c19e450 con 0x7f962c100f10 2024-02-02T15:36:21.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.843+0000 7f9633c00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f962c19e5c0 con 0x7f962c106030 2024-02-02T15:36:21.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.843+0000 7f9633c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f962c19e780 con 0x7f962c19d9d0 2024-02-02T15:36:21.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.843+0000 7f9631200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f962c100f10 0x7f962c199030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:21.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.843+0000 7f9631200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f962c100f10 0x7f962c199030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48932/0 (socket says 172.21.15.1:48932) 2024-02-02T15:36:21.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.843+0000 7f9631200700 1 -- 172.21.15.1:0/2365878231 learned_addr learned my addr 172.21.15.1:0/2365878231 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:36:21.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.843+0000 7f962be00700 1 --2- 172.21.15.1:0/2365878231 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f962c106030 0x7f962c199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:21.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.843+0000 7f9631c00700 1 --2- 172.21.15.1:0/2365878231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f962c19d9d0 0x7f962c19de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:21.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.844+0000 7f9631200700 1 -- 172.21.15.1:0/2365878231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f962c19d9d0 msgr2=0x7f962c19de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:21.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.844+0000 7f9631200700 1 --2- 172.21.15.1:0/2365878231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f962c19d9d0 0x7f962c19de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:21.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.844+0000 7f9631200700 1 -- 172.21.15.1:0/2365878231 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f962c106030 msgr2=0x7f962c199570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:21.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.844+0000 7f9631200700 1 --2- 172.21.15.1:0/2365878231 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f962c106030 0x7f962c199570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:21.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.844+0000 7f9631200700 1 -- 172.21.15.1:0/2365878231 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f961c0095f0 con 0x7f962c100f10 2024-02-02T15:36:21.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.844+0000 7f9631200700 1 --2- 172.21.15.1:0/2365878231 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f962c100f10 0x7f962c199030 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f961c005910 tx=0x7f961c005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:21.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.844+0000 7f9629600700 1 -- 172.21.15.1:0/2365878231 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f961c01d070 con 0x7f962c100f10 2024-02-02T15:36:21.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.844+0000 7f9633c00700 1 -- 172.21.15.1:0/2365878231 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f962c1a5c80 con 0x7f962c100f10 2024-02-02T15:36:21.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.844+0000 7f9629600700 1 -- 172.21.15.1:0/2365878231 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f961c022470 con 0x7f962c100f10 2024-02-02T15:36:21.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.844+0000 7f9629600700 1 -- 172.21.15.1:0/2365878231 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f961c017650 con 0x7f962c100f10 2024-02-02T15:36:21.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.844+0000 7f9633c00700 1 -- 172.21.15.1:0/2365878231 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f962c1a6100 con 0x7f962c100f10 2024-02-02T15:36:21.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.845+0000 7f9633c00700 1 -- 172.21.15.1:0/2365878231 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f962c067bc0 con 0x7f962c100f10 2024-02-02T15:36:21.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.847+0000 7f9629600700 1 -- 172.21.15.1:0/2365878231 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f961c0225e0 con 0x7f962c100f10 2024-02-02T15:36:21.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.849+0000 7f9629600700 1 --2- 172.21.15.1:0/2365878231 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f960c0a0690 0x7f960c0a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:21.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.849+0000 7f9629600700 1 -- 172.21.15.1:0/2365878231 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f961c0cd4b0 con 0x7f962c100f10 2024-02-02T15:36:21.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.850+0000 7f962be00700 1 --2- 172.21.15.1:0/2365878231 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f960c0a0690 0x7f960c0a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:21.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.850+0000 7f962be00700 1 --2- 172.21.15.1:0/2365878231 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f960c0a0690 0x7f960c0a2b50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f962c19a640 tx=0x7f9614009260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:21.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:21.854+0000 7f9629600700 1 -- 172.21.15.1:0/2365878231 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f961c0969a0 con 0x7f962c100f10 2024-02-02T15:36:21.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:21 smithi071 ceph-mon[44707]: pgmap v289: 129 pgs: 129 active+clean; 408 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 160 op/s 2024-02-02T15:36:21.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:21 smithi142 ceph-mon[45728]: pgmap v289: 129 pgs: 129 active+clean; 408 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.3 MiB/s wr, 160 op/s 2024-02-02T15:36:22.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.109+0000 7f9633c00700 1 -- 172.21.15.1:0/2365878231 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f962c19a380 con 0x7f962c100f10 2024-02-02T15:36:22.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.109+0000 7f9629600700 1 -- 172.21.15.1:0/2365878231 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+10882 (secure 0 0 0) 0x7f962c19a380 con 0x7f962c100f10 2024-02-02T15:36:22.110 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:36:22.110 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2922240448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2922240448},{"type":"v1","addr":"172.21.15.1:6835","nonce":2922240448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2729497431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2729497431},{"type":"v1","addr":"172.21.15.1:6837","nonce":2729497431}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":44,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-02T15:24:47.761435+0000","modified":"2024-02-02T15:35:55.485122+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24529,"mds_2":24518,"mds_3":24514,"mds_4":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14517":{"gid":14517,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/1218303683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":1218303683},{"type":"v1","addr":"172.21.15.1:6839","nonce":1218303683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14529":{"gid":14529,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/2059197496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2059197496},{"type":"v1","addr":"172.21.15.1:6841","nonce":2059197496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/7648986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":7648986},{"type":"v1","addr":"172.21.15.142:6833","nonce":7648986}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/3482746094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3482746094},{"type":"v1","addr":"172.21.15.71:6833","nonce":3482746094}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/388207814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":388207814},{"type":"v1","addr":"172.21.15.71:6835","nonce":388207814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6835/1052606547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1052606547},{"type":"v1","addr":"172.21.15.142:6835","nonce":1052606547}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.142:6837/2870289019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2870289019},{"type":"v1","addr":"172.21.15.142:6837","nonce":2870289019}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.71:6837/628016199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":628016199},{"type":"v1","addr":"172.21.15.71:6837","nonce":628016199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.142:6839/321675421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":321675421},{"type":"v1","addr":"172.21.15.142:6839","nonce":321675421}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/1268086493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1268086493},{"type":"v1","addr":"172.21.15.71:6839","nonce":1268086493}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-02T15:36:22.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.112+0000 7f9633c00700 1 -- 172.21.15.1:0/2365878231 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f960c0a0690 msgr2=0x7f960c0a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:22.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.112+0000 7f9633c00700 1 --2- 172.21.15.1:0/2365878231 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f960c0a0690 0x7f960c0a2b50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f962c19a640 tx=0x7f9614009260 comp rx=0 tx=0).stop 2024-02-02T15:36:22.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.112+0000 7f9633c00700 1 -- 172.21.15.1:0/2365878231 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f962c100f10 msgr2=0x7f962c199030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:22.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.112+0000 7f9633c00700 1 --2- 172.21.15.1:0/2365878231 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f962c100f10 0x7f962c199030 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f961c005910 tx=0x7f961c005740 comp rx=0 tx=0).stop 2024-02-02T15:36:22.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.112+0000 7f9633c00700 1 -- 172.21.15.1:0/2365878231 shutdown_connections 2024-02-02T15:36:22.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.112+0000 7f9633c00700 1 --2- 172.21.15.1:0/2365878231 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f960c0a0690 0x7f960c0a2b50 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.112+0000 7f9633c00700 1 --2- 172.21.15.1:0/2365878231 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f962c19d9d0 0x7f962c19de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.112+0000 7f9633c00700 1 --2- 172.21.15.1:0/2365878231 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f962c100f10 0x7f962c199030 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.112+0000 7f9633c00700 1 --2- 172.21.15.1:0/2365878231 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f962c106030 0x7f962c199570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.112+0000 7f9633c00700 1 -- 172.21.15.1:0/2365878231 >> 172.21.15.1:0/2365878231 conn(0x7f962c0fc790 msgr2=0x7f962c0fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:22.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.112+0000 7f9633c00700 1 -- 172.21.15.1:0/2365878231 shutdown_connections 2024-02-02T15:36:22.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.112+0000 7f9633c00700 1 -- 172.21.15.1:0/2365878231 wait complete. 2024-02-02T15:36:22.113 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 44 2024-02-02T15:36:22.131 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-02-02T15:36:22.256 INFO:tasks.workunit.client.0.smithi001.stdout: 1 548762 6.23 MB/sec execute 356 sec latency 2120.844 ms 2024-02-02T15:36:22.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.331+0000 7f4b0f200700 1 -- 172.21.15.1:0/1731530116 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b08106110 msgr2=0x7f4b0810dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:22.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.331+0000 7f4b0f200700 1 --2- 172.21.15.1:0/1731530116 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b08106110 0x7f4b0810dc90 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f4b00009f90 tx=0x7f4b0000b330 comp rx=0 tx=0).stop 2024-02-02T15:36:22.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.331+0000 7f4b0f200700 1 -- 172.21.15.1:0/1731530116 shutdown_connections 2024-02-02T15:36:22.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.331+0000 7f4b0f200700 1 --2- 172.21.15.1:0/1731530116 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b08106110 0x7f4b0810dc90 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.331+0000 7f4b0f200700 1 --2- 172.21.15.1:0/1731530116 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b08100ff0 0x7f4b081013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.331+0000 7f4b0f200700 1 --2- 172.21.15.1:0/1731530116 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b081019a0 0x7f4b081059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.331+0000 7f4b0f200700 1 -- 172.21.15.1:0/1731530116 >> 172.21.15.1:0/1731530116 conn(0x7f4b080fc810 msgr2=0x7f4b080fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:22.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.331+0000 7f4b0f200700 1 -- 172.21.15.1:0/1731530116 shutdown_connections 2024-02-02T15:36:22.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.331+0000 7f4b0f200700 1 -- 172.21.15.1:0/1731530116 wait complete. 2024-02-02T15:36:22.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.332+0000 7f4b0f200700 1 Processor -- start 2024-02-02T15:36:22.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.332+0000 7f4b0f200700 1 -- start start 2024-02-02T15:36:22.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.332+0000 7f4b0f200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b08100ff0 0x7f4b08073040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:22.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.332+0000 7f4b0f200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b08106110 0x7f4b08073580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:22.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.332+0000 7f4b0f200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b08073ac0 0x7f4b081a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:22.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.332+0000 7f4b0f200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b081a58b0 con 0x7f4b08073ac0 2024-02-02T15:36:22.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.332+0000 7f4b0f200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b081a5a20 con 0x7f4b08106110 2024-02-02T15:36:22.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.332+0000 7f4b0f200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b081a5be0 con 0x7f4b08100ff0 2024-02-02T15:36:22.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.333+0000 7f4b0d200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b08073ac0 0x7f4b081a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:22.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.333+0000 7f4b0d200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b08073ac0 0x7f4b081a51a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48936/0 (socket says 172.21.15.1:48936) 2024-02-02T15:36:22.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.333+0000 7f4b0d200700 1 -- 172.21.15.1:0/3510126535 learned_addr learned my addr 172.21.15.1:0/3510126535 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:36:22.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.333+0000 7f4b07e00700 1 --2- 172.21.15.1:0/3510126535 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b08100ff0 0x7f4b08073040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:22.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.333+0000 7f4b07400700 1 --2- 172.21.15.1:0/3510126535 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b08106110 0x7f4b08073580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:22.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.333+0000 7f4b0d200700 1 -- 172.21.15.1:0/3510126535 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b08100ff0 msgr2=0x7f4b08073040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:22.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.333+0000 7f4b0d200700 1 --2- 172.21.15.1:0/3510126535 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b08100ff0 0x7f4b08073040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.333+0000 7f4b0d200700 1 -- 172.21.15.1:0/3510126535 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b08106110 msgr2=0x7f4b08073580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:22.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.333+0000 7f4b0d200700 1 --2- 172.21.15.1:0/3510126535 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b08106110 0x7f4b08073580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.333+0000 7f4b0d200700 1 -- 172.21.15.1:0/3510126535 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4af00095f0 con 0x7f4b08073ac0 2024-02-02T15:36:22.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.333+0000 7f4b07e00700 1 --2- 172.21.15.1:0/3510126535 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b08100ff0 0x7f4b08073040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:36:22.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.333+0000 7f4b0d200700 1 --2- 172.21.15.1:0/3510126535 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b08073ac0 0x7f4b081a51a0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f4b00009f60 tx=0x7f4b00004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:22.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.334+0000 7f4b04c00700 1 -- 172.21.15.1:0/3510126535 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b0000e040 con 0x7f4b08073ac0 2024-02-02T15:36:22.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.334+0000 7f4b04c00700 1 -- 172.21.15.1:0/3510126535 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4b00004e20 con 0x7f4b08073ac0 2024-02-02T15:36:22.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.334+0000 7f4b04c00700 1 -- 172.21.15.1:0/3510126535 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b000052a0 con 0x7f4b08073ac0 2024-02-02T15:36:22.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.334+0000 7f4b0f200700 1 -- 172.21.15.1:0/3510126535 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b00009c40 con 0x7f4b08073ac0 2024-02-02T15:36:22.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.334+0000 7f4b0f200700 1 -- 172.21.15.1:0/3510126535 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b081a6100 con 0x7f4b08073ac0 2024-02-02T15:36:22.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.337+0000 7f4b04c00700 1 -- 172.21.15.1:0/3510126535 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f4b0001d470 con 0x7f4b08073ac0 2024-02-02T15:36:22.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.338+0000 7f4b22d37b80 1 Processor -- start 2024-02-02T15:36:22.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.338+0000 7f4b04c00700 1 --2- 172.21.15.1:0/3510126535 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f4ae80a0690 0x7f4ae80a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:22.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.338+0000 7f4b22d37b80 1 -- start start 2024-02-02T15:36:22.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.338+0000 7f4b04c00700 1 -- 172.21.15.1:0/3510126535 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f4b000cd720 con 0x7f4b08073ac0 2024-02-02T15:36:22.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.339+0000 7f4b22d37b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c50040b010 0x55c50040b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:22.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.339+0000 7f4b07400700 1 --2- 172.21.15.1:0/3510126535 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f4ae80a0690 0x7f4ae80a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:22.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.339+0000 7f4b22d37b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c50040b930 0x55c5004162a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:22.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.339+0000 7f4b22d37b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c5004167e0 0x55c500418bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:22.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.339+0000 7f4b22d37b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c50037f4c0 con 0x55c50040b930 2024-02-02T15:36:22.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.339+0000 7f4b22d37b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c500363aa0 con 0x55c50040b010 2024-02-02T15:36:22.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.339+0000 7f4b22d37b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c500361460 con 0x55c5004167e0 2024-02-02T15:36:22.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.339+0000 7f4b07400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c50040b930 0x55c5004162a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:22.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.339+0000 7f4b07e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c50040b010 0x55c50040b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:22.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.339+0000 7f4b07e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c50040b010 0x55c50040b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:57844/0 (socket says 172.21.15.1:57844) 2024-02-02T15:36:22.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.339+0000 7f4b07e00700 1 -- 172.21.15.1:0/1637687227 learned_addr learned my addr 172.21.15.1:0/1637687227 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:36:22.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.339+0000 7f4b0d200700 1 --2- 172.21.15.1:0/1637687227 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c5004167e0 0x55c500418bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:22.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.339+0000 7f4b07e00700 1 -- 172.21.15.1:0/1637687227 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c5004167e0 msgr2=0x55c500418bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:22.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.339+0000 7f4b07e00700 1 --2- 172.21.15.1:0/1637687227 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c5004167e0 0x55c500418bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.339+0000 7f4b07e00700 1 -- 172.21.15.1:0/1637687227 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c50040b930 msgr2=0x55c5004162a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:22.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.339+0000 7f4b07e00700 1 --2- 172.21.15.1:0/1637687227 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c50040b930 0x55c5004162a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.340+0000 7f4b07e00700 1 -- 172.21.15.1:0/1637687227 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5001de3b0 con 0x55c50040b010 2024-02-02T15:36:22.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.340+0000 7f4b07400700 1 --2- 172.21.15.1:0/3510126535 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f4ae80a0690 0x7f4ae80a2b50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4b08074830 tx=0x7f4af0009c40 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:22.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.340+0000 7f4b07e00700 1 --2- 172.21.15.1:0/1637687227 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c50040b010 0x55c50040b3f0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f4af8011620 tx=0x7f4af8011930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a1078def69d69d9 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:22.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.340+0000 7f4afd600700 1 -- 172.21.15.1:0/1637687227 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af8019630 con 0x55c50040b010 2024-02-02T15:36:22.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.340+0000 7f4afd600700 1 -- 172.21.15.1:0/1637687227 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4af8019c50 con 0x55c50040b010 2024-02-02T15:36:22.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.340+0000 7f4afd600700 1 -- 172.21.15.1:0/1637687227 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af800ebc0 con 0x55c50040b010 2024-02-02T15:36:22.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.341+0000 7f4b22d37b80 1 -- 172.21.15.1:0/1637687227 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c50040b010 msgr2=0x55c50040b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:22.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.341+0000 7f4b22d37b80 1 --2- 172.21.15.1:0/1637687227 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c50040b010 0x55c50040b3f0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f4af8011620 tx=0x7f4af8011930 comp rx=0 tx=0).stop 2024-02-02T15:36:22.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.341+0000 7f4b22d37b80 1 -- 172.21.15.1:0/1637687227 shutdown_connections 2024-02-02T15:36:22.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.341+0000 7f4b22d37b80 1 --2- 172.21.15.1:0/1637687227 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c5004167e0 0x55c500418bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.341+0000 7f4b22d37b80 1 --2- 172.21.15.1:0/1637687227 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c50040b010 0x55c50040b3f0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.341+0000 7f4b22d37b80 1 --2- 172.21.15.1:0/1637687227 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c50040b930 0x55c5004162a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.341+0000 7f4b22d37b80 1 -- 172.21.15.1:0/1637687227 >> 172.21.15.1:0/1637687227 conn(0x55c50036adb0 msgr2=0x55c5004073b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:22.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.341+0000 7f4b22d37b80 1 -- 172.21.15.1:0/1637687227 shutdown_connections 2024-02-02T15:36:22.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.341+0000 7f4b22d37b80 1 -- 172.21.15.1:0/1637687227 wait complete. 2024-02-02T15:36:22.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.341+0000 7f4b22d37b80 1 -- start start 2024-02-02T15:36:22.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.342+0000 7f4b22d37b80 1 Processor -- start 2024-02-02T15:36:22.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.342+0000 7f4b22d37b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c50040b010 0x55c500416150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:22.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.342+0000 7f4b22d37b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c50040b930 0x55c5004133d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:22.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.342+0000 7f4b22d37b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5004167e0 0x55c500413a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:22.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.342+0000 7f4b22d37b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c50035cd50 con 0x55c50040b010 2024-02-02T15:36:22.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.342+0000 7f4b22d37b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5002f8d80 con 0x55c5004167e0 2024-02-02T15:36:22.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.342+0000 7f4b22d37b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5002f8a20 con 0x55c50040b930 2024-02-02T15:36:22.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.342+0000 7f4b22d37b80 10 client.0 _ll_register_callbacks cb 0x55c5002fa290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:36:22.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.342+0000 7f4b07e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c50040b010 0x55c500416150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:22.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.342+0000 7f4b07e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c50040b010 0x55c500416150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48950/0 (socket says 172.21.15.1:48950) 2024-02-02T15:36:22.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.343+0000 7f4b07e00700 1 -- 172.21.15.1:0/1170828608 learned_addr learned my addr 172.21.15.1:0/1170828608 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:36:22.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.343+0000 7f4b07400700 1 --2- 172.21.15.1:0/1170828608 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c50040b930 0x55c5004133d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:22.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.343+0000 7f4b0d200700 1 --2- 172.21.15.1:0/1170828608 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5004167e0 0x55c500413a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:22.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.343+0000 7f4b07e00700 1 -- 172.21.15.1:0/1170828608 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c50040b930 msgr2=0x55c5004133d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:22.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.343+0000 7f4b07e00700 1 --2- 172.21.15.1:0/1170828608 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c50040b930 0x55c5004133d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.343+0000 7f4b07e00700 1 -- 172.21.15.1:0/1170828608 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5004167e0 msgr2=0x55c500413a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:22.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.343+0000 7f4b07e00700 1 --2- 172.21.15.1:0/1170828608 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5004167e0 0x55c500413a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.343+0000 7f4b07e00700 1 -- 172.21.15.1:0/1170828608 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4af8009dd0 con 0x55c50040b010 2024-02-02T15:36:22.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.343+0000 7f4b0d200700 1 --2- 172.21.15.1:0/1170828608 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5004167e0 0x55c500413a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:36:22.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.343+0000 7f4b07400700 1 --2- 172.21.15.1:0/1170828608 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c50040b930 0x55c5004133d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:36:22.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.343+0000 7f4b07e00700 1 --2- 172.21.15.1:0/1170828608 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c50040b010 0x55c500416150 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f4af80095f0 tx=0x7f4af800e5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70bec15c3417b47c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:22.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.343+0000 7f4af6000700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:36:22.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.343+0000 7f4af6000700 1 -- 172.21.15.1:0/1170828608 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af800ec70 con 0x55c50040b010 2024-02-02T15:36:22.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.344+0000 7f4b22d37b80 1 -- 172.21.15.1:0/1170828608 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c5003ff0f0 con 0x55c50040b010 2024-02-02T15:36:22.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.344+0000 7f4af6000700 1 -- 172.21.15.1:0/1170828608 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f4af800e3b0 con 0x55c50040b010 2024-02-02T15:36:22.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.344+0000 7f4af6000700 1 -- 172.21.15.1:0/1170828608 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4af800edd0 con 0x55c50040b010 2024-02-02T15:36:22.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.344+0000 7f4b22d37b80 10 client.18003 fetch_fsmap learned FSMap version 44 2024-02-02T15:36:22.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.344+0000 7f4b22d37b80 1 -- 172.21.15.1:0/1170828608 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x55c50018ab60 con 0x55c50040b010 2024-02-02T15:36:22.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.344+0000 7f4af6000700 1 -- 172.21.15.1:0/1170828608 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c5003ff0f0 con 0x55c50040b010 2024-02-02T15:36:22.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.344+0000 7f4af6000700 1 -- 172.21.15.1:0/1170828608 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 44) v1 ==== 7510+0+0 (secure 0 0 0) 0x7f4af80198b0 con 0x55c50040b010 2024-02-02T15:36:22.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.344+0000 7f4b22d37b80 10 client.18003 fetch_fsmap finished waiting for FSMap version 44 2024-02-02T15:36:22.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.344+0000 7f4b22d37b80 10 client.18003 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-02-02T15:36:22.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.344+0000 7f4b22d37b80 20 client.18003 populate_metadata read hostname 'smithi001' 2024-02-02T15:36:22.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.344+0000 7f4b22d37b80 1 --2- 172.21.15.1:0/1170828608 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55c500426140 0x55c500428560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:22.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.344+0000 7f4b22d37b80 4 client.18003 mds_command: new command op to 24502 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:36:22.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.344+0000 7f4b22d37b80 1 -- 172.21.15.1:0/1170828608 --> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c500355dd0 con 0x55c500426140 2024-02-02T15:36:22.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.345+0000 7f4b07400700 1 --2- 172.21.15.1:0/1170828608 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55c500426140 0x55c500428560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:22.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.345+0000 7f4b07400700 1 --2- 172.21.15.1:0/1170828608 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55c500426140 0x55c500428560 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4a399e04f0858f2e server_cookie=2141a3ca33ca5bcd in_seq=0 out_seq=0 2024-02-02T15:36:22.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.345+0000 7f4af6000700 10 client.18003 ms_handle_connect on v2:172.21.15.142:6834/1052606547 2024-02-02T15:36:22.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.346+0000 7f4af6000700 1 -- 172.21.15.1:0/1170828608 <== mds.4 v2:172.21.15.142:6834/1052606547 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55c500355dd0 con 0x55c500426140 2024-02-02T15:36:22.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.346+0000 7f4af6000700 10 client.18003 handle_command_reply: tid=0 2024-02-02T15:36:22.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.346+0000 7f4b22d37b80 1 client.18003 shutdown 2024-02-02T15:36:22.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.346+0000 7f4b22d37b80 1 -- 172.21.15.1:0/1170828608 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c50040b010 msgr2=0x55c500416150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:22.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.346+0000 7f4b22d37b80 1 --2- 172.21.15.1:0/1170828608 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c50040b010 0x55c500416150 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f4af80095f0 tx=0x7f4af800e5a0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.346+0000 7f4b22d37b80 1 -- 172.21.15.1:0/1170828608 shutdown_connections 2024-02-02T15:36:22.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.346+0000 7f4b22d37b80 1 --2- 172.21.15.1:0/1170828608 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55c500426140 0x55c500428560 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.346+0000 7f4b22d37b80 1 --2- 172.21.15.1:0/1170828608 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5004167e0 0x55c500413a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.346+0000 7f4b22d37b80 1 --2- 172.21.15.1:0/1170828608 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c50040b010 0x55c500416150 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.346+0000 7f4b22d37b80 1 --2- 172.21.15.1:0/1170828608 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c50040b930 0x55c5004133d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.346+0000 7f4af6000700 0 client.18003 ms_handle_reset on v2:172.21.15.142:6834/1052606547 2024-02-02T15:36:22.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.346+0000 7f4b22d37b80 1 -- 172.21.15.1:0/1170828608 >> 172.21.15.1:0/1170828608 conn(0x55c50036adb0 msgr2=0x55c500406f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:22.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.347+0000 7f4b22d37b80 1 -- 172.21.15.1:0/1170828608 shutdown_connections 2024-02-02T15:36:22.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.347+0000 7f4b22d37b80 1 -- 172.21.15.1:0/1170828608 wait complete. 2024-02-02T15:36:22.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.347+0000 7f4b22d37b80 20 client.18003 trim_cache size 0 max 16384 2024-02-02T15:36:22.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.360+0000 7f4b22d37b80 1 Processor -- start 2024-02-02T15:36:22.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.360+0000 7f4b22d37b80 1 -- start start 2024-02-02T15:36:22.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.361+0000 7f4b22d37b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c50040b010 0x55c50040d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:22.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.361+0000 7f4b22d37b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c50040b930 0x55c50040dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:22.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.361+0000 7f4b22d37b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c5004167e0 0x55c500412b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:22.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.361+0000 7f4b22d37b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5001e06a0 con 0x55c50040b010 2024-02-02T15:36:22.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.361+0000 7f4b22d37b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c500407ff0 con 0x55c50040b930 2024-02-02T15:36:22.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.361+0000 7f4b22d37b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c500422bc0 con 0x55c5004167e0 2024-02-02T15:36:22.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.361+0000 7f4b07e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c50040b010 0x55c50040d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:22.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.361+0000 7f4b07e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c50040b010 0x55c50040d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48956/0 (socket says 172.21.15.1:48956) 2024-02-02T15:36:22.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.361+0000 7f4b07e00700 1 -- 172.21.15.1:0/57872624 learned_addr learned my addr 172.21.15.1:0/57872624 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:36:22.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.361+0000 7f4b07400700 1 --2- 172.21.15.1:0/57872624 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c50040b930 0x55c50040dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:22.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.361+0000 7f4b0d200700 1 --2- 172.21.15.1:0/57872624 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c5004167e0 0x55c500412b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:22.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.361+0000 7f4b07e00700 1 -- 172.21.15.1:0/57872624 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c5004167e0 msgr2=0x55c500412b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:22.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.361+0000 7f4b07e00700 1 --2- 172.21.15.1:0/57872624 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c5004167e0 0x55c500412b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.361+0000 7f4b07e00700 1 -- 172.21.15.1:0/57872624 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c50040b930 msgr2=0x55c50040dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:22.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.361+0000 7f4b07e00700 1 --2- 172.21.15.1:0/57872624 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c50040b930 0x55c50040dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.361+0000 7f4b07e00700 1 -- 172.21.15.1:0/57872624 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c500322ea0 con 0x55c50040b010 2024-02-02T15:36:22.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.361+0000 7f4b07400700 1 --2- 172.21.15.1:0/57872624 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c50040b930 0x55c50040dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:36:22.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.362+0000 7f4b07e00700 1 --2- 172.21.15.1:0/57872624 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c50040b010 0x55c50040d840 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f4af8004870 tx=0x7f4af80048a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bd1735fb897e520 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:22.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.362+0000 7f4afe000700 1 -- 172.21.15.1:0/57872624 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af80114f0 con 0x55c50040b010 2024-02-02T15:36:22.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.362+0000 7f4afe000700 1 -- 172.21.15.1:0/57872624 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4af800e3b0 con 0x55c50040b010 2024-02-02T15:36:22.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.362+0000 7f4afe000700 1 -- 172.21.15.1:0/57872624 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af8019990 con 0x55c50040b010 2024-02-02T15:36:22.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.362+0000 7f4b22d37b80 1 -- 172.21.15.1:0/57872624 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c50040b010 msgr2=0x55c50040d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:22.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.362+0000 7f4b22d37b80 1 --2- 172.21.15.1:0/57872624 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c50040b010 0x55c50040d840 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f4af8004870 tx=0x7f4af80048a0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.362+0000 7f4b22d37b80 1 -- 172.21.15.1:0/57872624 shutdown_connections 2024-02-02T15:36:22.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.362+0000 7f4b22d37b80 1 --2- 172.21.15.1:0/57872624 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c5004167e0 0x55c500412b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.362+0000 7f4b22d37b80 1 --2- 172.21.15.1:0/57872624 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c50040b010 0x55c50040d840 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.362+0000 7f4b22d37b80 1 --2- 172.21.15.1:0/57872624 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c50040b930 0x55c50040dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.362+0000 7f4b22d37b80 1 -- 172.21.15.1:0/57872624 >> 172.21.15.1:0/57872624 conn(0x55c50036adb0 msgr2=0x55c500405b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:22.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.362+0000 7f4b22d37b80 1 -- 172.21.15.1:0/57872624 shutdown_connections 2024-02-02T15:36:22.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.362+0000 7f4b22d37b80 1 -- 172.21.15.1:0/57872624 wait complete. 2024-02-02T15:36:22.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.363+0000 7f4b22d37b80 1 -- start start 2024-02-02T15:36:22.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.363+0000 7f4b22d37b80 1 Processor -- start 2024-02-02T15:36:22.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.363+0000 7f4b22d37b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c50040b010 0x55c50041b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:22.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.363+0000 7f4b22d37b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c50040b930 0x55c50041b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:22.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.364+0000 7f4b22d37b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5004167e0 0x55c50041bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:22.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.364+0000 7f4b22d37b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c500476cb0 con 0x55c50040b010 2024-02-02T15:36:22.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.364+0000 7f4b22d37b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c500476df0 con 0x55c5004167e0 2024-02-02T15:36:22.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.364+0000 7f4b22d37b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c500476fb0 con 0x55c50040b930 2024-02-02T15:36:22.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.364+0000 7f4b22d37b80 10 client.0 _ll_register_callbacks cb 0x55c5002fa290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:36:22.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.364+0000 7f4b07e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c50040b010 0x55c50041b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:22.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.364+0000 7f4b07e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c50040b010 0x55c50041b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48968/0 (socket says 172.21.15.1:48968) 2024-02-02T15:36:22.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.364+0000 7f4b07e00700 1 -- 172.21.15.1:0/4112343356 learned_addr learned my addr 172.21.15.1:0/4112343356 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:36:22.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.364+0000 7f4b07400700 1 --2- 172.21.15.1:0/4112343356 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c50040b930 0x55c50041b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:22.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.364+0000 7f4b0d200700 1 --2- 172.21.15.1:0/4112343356 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5004167e0 0x55c50041bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:22.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.364+0000 7f4b07e00700 1 -- 172.21.15.1:0/4112343356 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c50040b930 msgr2=0x55c50041b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:22.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.364+0000 7f4b07e00700 1 --2- 172.21.15.1:0/4112343356 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c50040b930 0x55c50041b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.364+0000 7f4b07e00700 1 -- 172.21.15.1:0/4112343356 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5004167e0 msgr2=0x55c50041bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:22.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.364+0000 7f4b07e00700 1 --2- 172.21.15.1:0/4112343356 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5004167e0 0x55c50041bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.364+0000 7f4b07e00700 1 -- 172.21.15.1:0/4112343356 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c50018ab60 con 0x55c50040b010 2024-02-02T15:36:22.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.364+0000 7f4b0d200700 1 --2- 172.21.15.1:0/4112343356 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5004167e0 0x55c50041bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:36:22.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.364+0000 7f4b07e00700 1 --2- 172.21.15.1:0/4112343356 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c50040b010 0x55c50041b310 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f4af8014040 tx=0x7f4af80114f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0580a18c6a11c6d server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:22.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.364+0000 7f4af6000700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:36:22.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.365+0000 7f4af6000700 1 -- 172.21.15.1:0/4112343356 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af800ee80 con 0x55c50040b010 2024-02-02T15:36:22.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.365+0000 7f4af6000700 1 -- 172.21.15.1:0/4112343356 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f4af8000e40 con 0x55c50040b010 2024-02-02T15:36:22.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.365+0000 7f4b22d37b80 1 -- 172.21.15.1:0/4112343356 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4af8019c50 con 0x55c50040b010 2024-02-02T15:36:22.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.365+0000 7f4af6000700 1 -- 172.21.15.1:0/4112343356 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4af8019430 con 0x55c50040b010 2024-02-02T15:36:22.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.365+0000 7f4b22d37b80 10 client.18009 fetch_fsmap learned FSMap version 44 2024-02-02T15:36:22.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.365+0000 7f4b22d37b80 1 -- 172.21.15.1:0/4112343356 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x55c5001e2cd0 con 0x55c50040b010 2024-02-02T15:36:22.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.365+0000 7f4af6000700 1 -- 172.21.15.1:0/4112343356 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4af8019c50 con 0x55c50040b010 2024-02-02T15:36:22.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.365+0000 7f4af6000700 1 -- 172.21.15.1:0/4112343356 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 44) v1 ==== 7510+0+0 (secure 0 0 0) 0x7f4af8003bb0 con 0x55c50040b010 2024-02-02T15:36:22.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.365+0000 7f4b22d37b80 10 client.18009 fetch_fsmap finished waiting for FSMap version 44 2024-02-02T15:36:22.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.365+0000 7f4b22d37b80 10 client.18009 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-02-02T15:36:22.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.365+0000 7f4b22d37b80 20 client.18009 populate_metadata read hostname 'smithi001' 2024-02-02T15:36:22.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.365+0000 7f4b22d37b80 1 --2- 172.21.15.1:0/4112343356 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55c500478280 0x55c50047a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:22.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.365+0000 7f4b22d37b80 4 client.18009 mds_command: new command op to 24502 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:36:22.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.365+0000 7f4b22d37b80 1 -- 172.21.15.1:0/4112343356 --> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c50031bf50 con 0x55c500478280 2024-02-02T15:36:22.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.366+0000 7f4b07400700 1 --2- 172.21.15.1:0/4112343356 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55c500478280 0x55c50047a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:22.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.366+0000 7f4b07400700 1 --2- 172.21.15.1:0/4112343356 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55c500478280 0x55c50047a670 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=95ffff965bcc4a69 server_cookie=59ee2ea960bb9c57 in_seq=0 out_seq=0 2024-02-02T15:36:22.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.366+0000 7f4af6000700 10 client.18009 ms_handle_connect on v2:172.21.15.142:6834/1052606547 2024-02-02T15:36:22.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.367+0000 7f4af6000700 1 -- 172.21.15.1:0/4112343356 <== mds.4 v2:172.21.15.142:6834/1052606547 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c50031bf50 con 0x55c500478280 2024-02-02T15:36:22.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.367+0000 7f4af6000700 10 client.18009 handle_command_reply: tid=0 2024-02-02T15:36:22.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.367+0000 7f4b22d37b80 1 client.18009 shutdown 2024-02-02T15:36:22.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.367+0000 7f4b22d37b80 1 -- 172.21.15.1:0/4112343356 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c50040b010 msgr2=0x55c50041b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:22.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.367+0000 7f4b22d37b80 1 --2- 172.21.15.1:0/4112343356 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c50040b010 0x55c50041b310 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f4af8014040 tx=0x7f4af80114f0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.367+0000 7f4b22d37b80 1 -- 172.21.15.1:0/4112343356 shutdown_connections 2024-02-02T15:36:22.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.367+0000 7f4b22d37b80 1 --2- 172.21.15.1:0/4112343356 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55c500478280 0x55c50047a670 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.367+0000 7f4b22d37b80 1 --2- 172.21.15.1:0/4112343356 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5004167e0 0x55c50041bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.367+0000 7f4b22d37b80 1 --2- 172.21.15.1:0/4112343356 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c50040b010 0x55c50041b310 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.367+0000 7f4b22d37b80 1 --2- 172.21.15.1:0/4112343356 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c50040b930 0x55c50041b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.367+0000 7f4af6000700 0 client.18009 ms_handle_reset on v2:172.21.15.142:6834/1052606547 2024-02-02T15:36:22.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.367+0000 7f4b22d37b80 1 -- 172.21.15.1:0/4112343356 >> 172.21.15.1:0/4112343356 conn(0x55c50036adb0 msgr2=0x55c5004065c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:22.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.367+0000 7f4b22d37b80 1 -- 172.21.15.1:0/4112343356 shutdown_connections 2024-02-02T15:36:22.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.367+0000 7f4b22d37b80 1 -- 172.21.15.1:0/4112343356 wait complete. 2024-02-02T15:36:22.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.367+0000 7f4b22d37b80 20 client.18009 trim_cache size 0 max 16384 2024-02-02T15:36:22.372 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:36:22.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.368+0000 7f4b0f200700 1 -- 172.21.15.1:0/3510126535 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f4ae80a0690 msgr2=0x7f4ae80a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:22.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.368+0000 7f4b0f200700 1 --2- 172.21.15.1:0/3510126535 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f4ae80a0690 0x7f4ae80a2b50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4b08074830 tx=0x7f4af0009c40 comp rx=0 tx=0).stop 2024-02-02T15:36:22.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.368+0000 7f4b0f200700 1 -- 172.21.15.1:0/3510126535 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b08073ac0 msgr2=0x7f4b081a51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:22.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.368+0000 7f4b0f200700 1 --2- 172.21.15.1:0/3510126535 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b08073ac0 0x7f4b081a51a0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f4b00009f60 tx=0x7f4b00004c80 comp rx=0 tx=0).stop 2024-02-02T15:36:22.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.368+0000 7f4b0f200700 1 -- 172.21.15.1:0/3510126535 shutdown_connections 2024-02-02T15:36:22.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.368+0000 7f4b0f200700 1 --2- 172.21.15.1:0/3510126535 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f4ae80a0690 0x7f4ae80a2b50 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.368+0000 7f4b0f200700 1 --2- 172.21.15.1:0/3510126535 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4b08073ac0 0x7f4b081a51a0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.368+0000 7f4b0f200700 1 --2- 172.21.15.1:0/3510126535 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4b08100ff0 0x7f4b08073040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.368+0000 7f4b0f200700 1 --2- 172.21.15.1:0/3510126535 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4b08106110 0x7f4b08073580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.368+0000 7f4b0f200700 1 -- 172.21.15.1:0/3510126535 >> 172.21.15.1:0/3510126535 conn(0x7f4b080fc810 msgr2=0x7f4b080fd220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:22.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.368+0000 7f4b0f200700 1 -- 172.21.15.1:0/3510126535 shutdown_connections 2024-02-02T15:36:22.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.368+0000 7f4b0f200700 1 -- 172.21.15.1:0/3510126535 wait complete. 2024-02-02T15:36:22.386 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-02-02T15:36:22.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.587+0000 7f798ca00700 1 -- 172.21.15.1:0/899383723 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79840685e0 msgr2=0x7f7984068a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:22.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.587+0000 7f798ca00700 1 --2- 172.21.15.1:0/899383723 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79840685e0 0x7f7984068a60 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f796c009990 tx=0x7f796c009ca0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.587+0000 7f798ca00700 1 -- 172.21.15.1:0/899383723 shutdown_connections 2024-02-02T15:36:22.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.587+0000 7f798ca00700 1 --2- 172.21.15.1:0/899383723 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f798410d610 0x7f798410fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.587+0000 7f798ca00700 1 --2- 172.21.15.1:0/899383723 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7984101c70 0x7f7984068010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.587+0000 7f798ca00700 1 --2- 172.21.15.1:0/899383723 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f79840685e0 0x7f7984068a60 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.587+0000 7f798ca00700 1 -- 172.21.15.1:0/899383723 >> 172.21.15.1:0/899383723 conn(0x7f79840757d0 msgr2=0x7f7984075be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:22.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.587+0000 7f798ca00700 1 -- 172.21.15.1:0/899383723 shutdown_connections 2024-02-02T15:36:22.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.587+0000 7f798ca00700 1 -- 172.21.15.1:0/899383723 wait complete. 2024-02-02T15:36:22.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.588+0000 7f798ca00700 1 Processor -- start 2024-02-02T15:36:22.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.588+0000 7f798ca00700 1 -- start start 2024-02-02T15:36:22.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.589+0000 7f798ca00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79840685e0 0x7f7984199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:22.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.589+0000 7f798ca00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7984101c70 0x7f7984199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:22.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.589+0000 7f798ca00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f798410d610 0x7f798419dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:22.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.589+0000 7f798ca00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f798419e110 con 0x7f798410d610 2024-02-02T15:36:22.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.589+0000 7f798ca00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f798419e280 con 0x7f79840685e0 2024-02-02T15:36:22.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.589+0000 7f798ca00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f798419e440 con 0x7f7984101c70 2024-02-02T15:36:22.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.589+0000 7f798aa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f798410d610 0x7f798419dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:22.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.589+0000 7f798a000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79840685e0 0x7f7984199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:22.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.589+0000 7f798aa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f798410d610 0x7f798419dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48994/0 (socket says 172.21.15.1:48994) 2024-02-02T15:36:22.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.589+0000 7f798aa00700 1 -- 172.21.15.1:0/608109113 learned_addr learned my addr 172.21.15.1:0/608109113 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:36:22.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.589+0000 7f7989600700 1 --2- 172.21.15.1:0/608109113 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7984101c70 0x7f7984199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:22.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.590+0000 7f798a000700 1 -- 172.21.15.1:0/608109113 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7984101c70 msgr2=0x7f7984199670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:22.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.590+0000 7f798a000700 1 --2- 172.21.15.1:0/608109113 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7984101c70 0x7f7984199670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.590+0000 7f798a000700 1 -- 172.21.15.1:0/608109113 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f798410d610 msgr2=0x7f798419dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:22.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.590+0000 7f798a000700 1 --2- 172.21.15.1:0/608109113 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f798410d610 0x7f798419dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.590+0000 7f798a000700 1 -- 172.21.15.1:0/608109113 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f796c0095f0 con 0x7f79840685e0 2024-02-02T15:36:22.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.590+0000 7f798aa00700 1 --2- 172.21.15.1:0/608109113 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f798410d610 0x7f798419dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:36:22.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.590+0000 7f798a000700 1 --2- 172.21.15.1:0/608109113 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79840685e0 0x7f7984199130 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f797400e9f0 tx=0x7f797400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:22.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.590+0000 7f7982a00700 1 -- 172.21.15.1:0/608109113 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f797400c9c0 con 0x7f79840685e0 2024-02-02T15:36:22.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.590+0000 7f798ca00700 1 -- 172.21.15.1:0/608109113 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f798419e720 con 0x7f79840685e0 2024-02-02T15:36:22.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.590+0000 7f7982a00700 1 -- 172.21.15.1:0/608109113 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7974010430 con 0x7f79840685e0 2024-02-02T15:36:22.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.590+0000 7f7982a00700 1 -- 172.21.15.1:0/608109113 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7974018680 con 0x7f79840685e0 2024-02-02T15:36:22.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.590+0000 7f798ca00700 1 -- 172.21.15.1:0/608109113 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79841a5f60 con 0x7f79840685e0 2024-02-02T15:36:22.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.593+0000 7f7982a00700 1 -- 172.21.15.1:0/608109113 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f7974010860 con 0x7f79840685e0 2024-02-02T15:36:22.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.594+0000 7f79a054db80 1 Processor -- start 2024-02-02T15:36:22.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.595+0000 7f79a054db80 1 -- start start 2024-02-02T15:36:22.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.595+0000 7f79a054db80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5604294c9010 0x5604294c93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:22.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.595+0000 7f79a054db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5604294c9930 0x5604294d42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:22.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.595+0000 7f79a054db80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604294d47e0 0x5604294d6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:22.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.595+0000 7f79a054db80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56042943d4c0 con 0x5604294c9010 2024-02-02T15:36:22.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.595+0000 7f79a054db80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x560429421aa0 con 0x5604294d47e0 2024-02-02T15:36:22.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.595+0000 7f79a054db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56042941f460 con 0x5604294c9930 2024-02-02T15:36:22.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.595+0000 7f7982a00700 1 --2- 172.21.15.1:0/608109113 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f79640a0690 0x7f79640a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:22.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.595+0000 7f798aa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5604294c9010 0x5604294c93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:22.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.595+0000 7f7982a00700 1 -- 172.21.15.1:0/608109113 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f7974014070 con 0x7f79840685e0 2024-02-02T15:36:22.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.595+0000 7f798aa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5604294c9010 0x5604294c93f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49008/0 (socket says 172.21.15.1:49008) 2024-02-02T15:36:22.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.595+0000 7f798aa00700 1 -- 172.21.15.1:0/1479525788 learned_addr learned my addr 172.21.15.1:0/1479525788 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:36:22.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.596+0000 7f798a000700 1 --2- 172.21.15.1:0/1479525788 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5604294c9930 0x5604294d42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:22.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.596+0000 7f7989600700 1 --2- 172.21.15.1:0/1479525788 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604294d47e0 0x5604294d6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:22.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.596+0000 7f798aa00700 1 --2- 172.21.15.1:0/608109113 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f79640a0690 0x7f79640a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:22.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.596+0000 7f798a000700 1 -- 172.21.15.1:0/1479525788 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604294d47e0 msgr2=0x5604294d6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:22.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.596+0000 7f798a000700 1 --2- 172.21.15.1:0/1479525788 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604294d47e0 0x5604294d6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.596+0000 7f798a000700 1 -- 172.21.15.1:0/1479525788 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5604294c9010 msgr2=0x5604294c93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:22.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.596+0000 7f798a000700 1 --2- 172.21.15.1:0/1479525788 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5604294c9010 0x5604294c93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.596+0000 7f798a000700 1 -- 172.21.15.1:0/1479525788 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56042929c3b0 con 0x5604294c9930 2024-02-02T15:36:22.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.596+0000 7f798a000700 1 --2- 172.21.15.1:0/1479525788 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5604294c9930 0x5604294d42a0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f79740c95c0 tx=0x7f797406b4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2af45171f04d9050 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:22.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.596+0000 7f798aa00700 1 --2- 172.21.15.1:0/608109113 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f79640a0690 0x7f79640a2b50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f797c009550 tx=0x7f797c00b040 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:22.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.596+0000 7f797b000700 1 -- 172.21.15.1:0/1479525788 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7974099ac0 con 0x5604294c9930 2024-02-02T15:36:22.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.596+0000 7f797b000700 1 -- 172.21.15.1:0/1479525788 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f79740993c0 con 0x5604294c9930 2024-02-02T15:36:22.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.596+0000 7f797b000700 1 -- 172.21.15.1:0/1479525788 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79740106c0 con 0x5604294c9930 2024-02-02T15:36:22.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.597+0000 7f79a054db80 1 -- 172.21.15.1:0/1479525788 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5604294c9930 msgr2=0x5604294d42a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:22.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.597+0000 7f79a054db80 1 --2- 172.21.15.1:0/1479525788 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5604294c9930 0x5604294d42a0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f79740c95c0 tx=0x7f797406b4a0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.597+0000 7f79a054db80 1 -- 172.21.15.1:0/1479525788 shutdown_connections 2024-02-02T15:36:22.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.597+0000 7f79a054db80 1 --2- 172.21.15.1:0/1479525788 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604294d47e0 0x5604294d6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.597+0000 7f79a054db80 1 --2- 172.21.15.1:0/1479525788 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5604294c9010 0x5604294c93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.597+0000 7f79a054db80 1 --2- 172.21.15.1:0/1479525788 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5604294c9930 0x5604294d42a0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.597+0000 7f79a054db80 1 -- 172.21.15.1:0/1479525788 >> 172.21.15.1:0/1479525788 conn(0x560429428db0 msgr2=0x5604294c53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:22.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.597+0000 7f79a054db80 1 -- 172.21.15.1:0/1479525788 shutdown_connections 2024-02-02T15:36:22.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.597+0000 7f79a054db80 1 -- 172.21.15.1:0/1479525788 wait complete. 2024-02-02T15:36:22.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.598+0000 7f79a054db80 1 -- start start 2024-02-02T15:36:22.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.598+0000 7f79a054db80 1 Processor -- start 2024-02-02T15:36:22.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.599+0000 7f79a054db80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604294c9010 0x5604294d1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:22.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.599+0000 7f79a054db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5604294c9930 0x5604294d1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:22.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.599+0000 7f79a054db80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5604294d47e0 0x5604294d2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:22.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.599+0000 7f79a054db80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56042941ad50 con 0x5604294d47e0 2024-02-02T15:36:22.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.599+0000 7f79a054db80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604293b6d80 con 0x5604294c9010 2024-02-02T15:36:22.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.599+0000 7f79a054db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604293b6a20 con 0x5604294c9930 2024-02-02T15:36:22.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.599+0000 7f79a054db80 10 client.0 _ll_register_callbacks cb 0x5604293b8290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:36:22.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.599+0000 7f798aa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5604294d47e0 0x5604294d2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:22.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.599+0000 7f7989600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5604294c9930 0x5604294d1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:22.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.599+0000 7f7989600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5604294c9930 0x5604294d1ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:45530/0 (socket says 172.21.15.1:45530) 2024-02-02T15:36:22.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.599+0000 7f798aa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5604294d47e0 0x5604294d2000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49012/0 (socket says 172.21.15.1:49012) 2024-02-02T15:36:22.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.599+0000 7f7989600700 1 -- 172.21.15.1:0/1964370373 learned_addr learned my addr 172.21.15.1:0/1964370373 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:36:22.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.599+0000 7f798a000700 1 --2- 172.21.15.1:0/1964370373 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604294c9010 0x5604294d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:22.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.599+0000 7f798aa00700 1 -- 172.21.15.1:0/1964370373 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5604294c9930 msgr2=0x5604294d1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:22.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.599+0000 7f798aa00700 1 --2- 172.21.15.1:0/1964370373 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5604294c9930 0x5604294d1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.599+0000 7f798aa00700 1 -- 172.21.15.1:0/1964370373 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604294c9010 msgr2=0x5604294d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:22.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.599+0000 7f798aa00700 1 --2- 172.21.15.1:0/1964370373 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604294c9010 0x5604294d1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.599+0000 7f798aa00700 1 -- 172.21.15.1:0/1964370373 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79841a5f60 con 0x5604294d47e0 2024-02-02T15:36:22.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.599+0000 7f7989600700 1 --2- 172.21.15.1:0/1964370373 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5604294c9930 0x5604294d1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:36:22.603 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.600+0000 7f798aa00700 1 --2- 172.21.15.1:0/1964370373 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5604294d47e0 0x5604294d2000 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f797c0090f0 tx=0x7f797c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62ede107d2eb9063 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:22.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.600+0000 7f7973400700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:36:22.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.600+0000 7f7973400700 1 -- 172.21.15.1:0/1964370373 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f797c02a070 con 0x5604294d47e0 2024-02-02T15:36:22.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.600+0000 7f79a054db80 1 -- 172.21.15.1:0/1964370373 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5604294bd0f0 con 0x5604294d47e0 2024-02-02T15:36:22.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.600+0000 7f7973400700 1 -- 172.21.15.1:0/1964370373 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f797c020040 con 0x5604294d47e0 2024-02-02T15:36:22.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.600+0000 7f7973400700 1 -- 172.21.15.1:0/1964370373 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f797c01be40 con 0x5604294d47e0 2024-02-02T15:36:22.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.600+0000 7f79a054db80 10 client.18021 fetch_fsmap learned FSMap version 44 2024-02-02T15:36:22.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.600+0000 7f79a054db80 1 -- 172.21.15.1:0/1964370373 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x560429248b60 con 0x5604294d47e0 2024-02-02T15:36:22.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.601+0000 7f7973400700 1 -- 172.21.15.1:0/1964370373 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5604294bd0f0 con 0x5604294d47e0 2024-02-02T15:36:22.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.601+0000 7f7973400700 1 -- 172.21.15.1:0/1964370373 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 44) v1 ==== 7510+0+0 (secure 0 0 0) 0x7f797c02d030 con 0x5604294d47e0 2024-02-02T15:36:22.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.601+0000 7f79a054db80 10 client.18021 fetch_fsmap finished waiting for FSMap version 44 2024-02-02T15:36:22.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.601+0000 7f79a054db80 10 client.18021 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-02-02T15:36:22.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.601+0000 7f79a054db80 20 client.18021 populate_metadata read hostname 'smithi001' 2024-02-02T15:36:22.604 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.601+0000 7f79a054db80 1 --2- 172.21.15.1:0/1964370373 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x5604294e4360 0x5604294e6780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:22.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.601+0000 7f79a054db80 4 client.18021 mds_command: new command op to 24514 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:36:22.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.601+0000 7f79a054db80 1 -- 172.21.15.1:0/1964370373 --> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560429413dd0 con 0x5604294e4360 2024-02-02T15:36:22.605 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.601+0000 7f798a000700 1 --2- 172.21.15.1:0/1964370373 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x5604294e4360 0x5604294e6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:22.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.602+0000 7f798a000700 1 --2- 172.21.15.1:0/1964370373 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x5604294e4360 0x5604294e6780 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e2554e8113d86e96 server_cookie=e7d6a3eed9f22e62 in_seq=0 out_seq=0 2024-02-02T15:36:22.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.602+0000 7f7973400700 10 client.18021 ms_handle_connect on v2:172.21.15.142:6836/2870289019 2024-02-02T15:36:22.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.603+0000 7f7973400700 1 -- 172.21.15.1:0/1964370373 <== mds.3 v2:172.21.15.142:6836/2870289019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x560429413dd0 con 0x5604294e4360 2024-02-02T15:36:22.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.603+0000 7f7973400700 10 client.18021 handle_command_reply: tid=0 2024-02-02T15:36:22.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.603+0000 7f79a054db80 1 client.18021 shutdown 2024-02-02T15:36:22.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.603+0000 7f79a054db80 1 -- 172.21.15.1:0/1964370373 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5604294d47e0 msgr2=0x5604294d2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:22.606 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.603+0000 7f79a054db80 1 --2- 172.21.15.1:0/1964370373 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5604294d47e0 0x5604294d2000 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f797c0090f0 tx=0x7f797c004ad0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.603+0000 7f79a054db80 1 -- 172.21.15.1:0/1964370373 shutdown_connections 2024-02-02T15:36:22.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.603+0000 7f79a054db80 1 --2- 172.21.15.1:0/1964370373 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x5604294e4360 0x5604294e6780 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.603+0000 7f79a054db80 1 --2- 172.21.15.1:0/1964370373 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5604294d47e0 0x5604294d2000 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.603+0000 7f79a054db80 1 --2- 172.21.15.1:0/1964370373 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604294c9010 0x5604294d1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.603+0000 7f79a054db80 1 --2- 172.21.15.1:0/1964370373 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5604294c9930 0x5604294d1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.603+0000 7f79a054db80 1 -- 172.21.15.1:0/1964370373 >> 172.21.15.1:0/1964370373 conn(0x560429428db0 msgr2=0x5604294c4fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:22.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.603+0000 7f7973400700 0 client.18021 ms_handle_reset on v2:172.21.15.142:6836/2870289019 2024-02-02T15:36:22.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.603+0000 7f79a054db80 1 -- 172.21.15.1:0/1964370373 shutdown_connections 2024-02-02T15:36:22.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.603+0000 7f79a054db80 1 -- 172.21.15.1:0/1964370373 wait complete. 2024-02-02T15:36:22.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.604+0000 7f79a054db80 20 client.18021 trim_cache size 0 max 16384 2024-02-02T15:36:22.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.617+0000 7f79a054db80 1 Processor -- start 2024-02-02T15:36:22.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.617+0000 7f79a054db80 1 -- start start 2024-02-02T15:36:22.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.617+0000 7f79a054db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5604294c9010 0x5604294d1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:22.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.617+0000 7f79a054db80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604294c9930 0x5604294c4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:22.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.617+0000 7f79a054db80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5604294d47e0 0x5604294c53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:22.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.617+0000 7f79a054db80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56042929e6a0 con 0x5604294d47e0 2024-02-02T15:36:22.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.617+0000 7f79a054db80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604294c5930 con 0x5604294c9930 2024-02-02T15:36:22.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.618+0000 7f79a054db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604294e0190 con 0x5604294c9010 2024-02-02T15:36:22.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.618+0000 7f798aa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5604294d47e0 0x5604294c53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:22.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.618+0000 7f798aa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5604294d47e0 0x5604294c53f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49024/0 (socket says 172.21.15.1:49024) 2024-02-02T15:36:22.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.618+0000 7f798aa00700 1 -- 172.21.15.1:0/2887839750 learned_addr learned my addr 172.21.15.1:0/2887839750 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:36:22.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.618+0000 7f798a000700 1 --2- 172.21.15.1:0/2887839750 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5604294c9010 0x5604294d1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:22.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.618+0000 7f7989600700 1 --2- 172.21.15.1:0/2887839750 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604294c9930 0x5604294c4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:22.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.618+0000 7f798aa00700 1 -- 172.21.15.1:0/2887839750 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5604294c9010 msgr2=0x5604294d1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:22.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.618+0000 7f798aa00700 1 --2- 172.21.15.1:0/2887839750 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5604294c9010 0x5604294d1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.618+0000 7f798aa00700 1 -- 172.21.15.1:0/2887839750 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604294c9930 msgr2=0x5604294c4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:22.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.618+0000 7f798aa00700 1 --2- 172.21.15.1:0/2887839750 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604294c9930 0x5604294c4eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.618+0000 7f798aa00700 1 -- 172.21.15.1:0/2887839750 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5604294e6b30 con 0x5604294d47e0 2024-02-02T15:36:22.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.618+0000 7f7989600700 1 --2- 172.21.15.1:0/2887839750 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604294c9930 0x5604294c4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:36:22.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.618+0000 7f798aa00700 1 --2- 172.21.15.1:0/2887839750 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5604294d47e0 0x5604294c53f0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f797c009230 tx=0x7f797c024cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72097a6950631c37 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:22.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.619+0000 7f797ba00700 1 -- 172.21.15.1:0/2887839750 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f797c02a070 con 0x5604294d47e0 2024-02-02T15:36:22.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.619+0000 7f797ba00700 1 -- 172.21.15.1:0/2887839750 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f797c00f040 con 0x5604294d47e0 2024-02-02T15:36:22.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.619+0000 7f797ba00700 1 -- 172.21.15.1:0/2887839750 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f797c0041f0 con 0x5604294d47e0 2024-02-02T15:36:22.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.619+0000 7f79a054db80 1 -- 172.21.15.1:0/2887839750 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5604294d47e0 msgr2=0x5604294c53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:22.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.619+0000 7f79a054db80 1 --2- 172.21.15.1:0/2887839750 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5604294d47e0 0x5604294c53f0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f797c009230 tx=0x7f797c024cb0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.619+0000 7f79a054db80 1 -- 172.21.15.1:0/2887839750 shutdown_connections 2024-02-02T15:36:22.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.619+0000 7f79a054db80 1 --2- 172.21.15.1:0/2887839750 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5604294d47e0 0x5604294c53f0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.619+0000 7f79a054db80 1 --2- 172.21.15.1:0/2887839750 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5604294c9010 0x5604294d1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.619+0000 7f79a054db80 1 --2- 172.21.15.1:0/2887839750 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604294c9930 0x5604294c4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.619+0000 7f79a054db80 1 -- 172.21.15.1:0/2887839750 >> 172.21.15.1:0/2887839750 conn(0x560429428db0 msgr2=0x5604294cc8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:22.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.619+0000 7f79a054db80 1 -- 172.21.15.1:0/2887839750 shutdown_connections 2024-02-02T15:36:22.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.619+0000 7f79a054db80 1 -- 172.21.15.1:0/2887839750 wait complete. 2024-02-02T15:36:22.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.620+0000 7f79a054db80 1 -- start start 2024-02-02T15:36:22.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.620+0000 7f79a054db80 1 Processor -- start 2024-02-02T15:36:22.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.620+0000 7f79a054db80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604294c9010 0x5604294db330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:22.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.620+0000 7f79a054db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5604294c9930 0x5604294d9310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:22.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.620+0000 7f79a054db80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5604294d47e0 0x5604294d98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:22.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.620+0000 7f79a054db80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604294d9fc0 con 0x5604294d47e0 2024-02-02T15:36:22.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.620+0000 7f79a054db80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604294da130 con 0x5604294c9010 2024-02-02T15:36:22.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.620+0000 7f79a054db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560429534e40 con 0x5604294c9930 2024-02-02T15:36:22.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.620+0000 7f79a054db80 10 client.0 _ll_register_callbacks cb 0x5604293b8290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:36:22.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.621+0000 7f798aa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5604294d47e0 0x5604294d98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:22.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.621+0000 7f7989600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5604294c9930 0x5604294d9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:22.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.621+0000 7f798aa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5604294d47e0 0x5604294d98b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49032/0 (socket says 172.21.15.1:49032) 2024-02-02T15:36:22.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.621+0000 7f798aa00700 1 -- 172.21.15.1:0/3032879086 learned_addr learned my addr 172.21.15.1:0/3032879086 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:36:22.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.621+0000 7f798a000700 1 --2- 172.21.15.1:0/3032879086 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604294c9010 0x5604294db330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:22.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.621+0000 7f798aa00700 1 -- 172.21.15.1:0/3032879086 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5604294c9930 msgr2=0x5604294d9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:22.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.621+0000 7f798aa00700 1 --2- 172.21.15.1:0/3032879086 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5604294c9930 0x5604294d9310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.621+0000 7f798aa00700 1 -- 172.21.15.1:0/3032879086 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604294c9010 msgr2=0x5604294db330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:22.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.621+0000 7f798aa00700 1 --2- 172.21.15.1:0/3032879086 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604294c9010 0x5604294db330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.621+0000 7f798aa00700 1 -- 172.21.15.1:0/3032879086 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560429248b60 con 0x5604294d47e0 2024-02-02T15:36:22.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.621+0000 7f798a000700 1 --2- 172.21.15.1:0/3032879086 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604294c9010 0x5604294db330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:36:22.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.621+0000 7f798aa00700 1 --2- 172.21.15.1:0/3032879086 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5604294d47e0 0x5604294d98b0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f797c003cb0 tx=0x7f797c009a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3127ca82922206f1 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:22.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.621+0000 7f7973400700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:36:22.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.622+0000 7f7973400700 1 -- 172.21.15.1:0/3032879086 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f797c02a070 con 0x5604294d47e0 2024-02-02T15:36:22.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.622+0000 7f79a054db80 1 -- 172.21.15.1:0/3032879086 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f79740993c0 con 0x5604294d47e0 2024-02-02T15:36:22.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.622+0000 7f7973400700 1 -- 172.21.15.1:0/3032879086 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f797c024830 con 0x5604294d47e0 2024-02-02T15:36:22.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.622+0000 7f7973400700 1 -- 172.21.15.1:0/3032879086 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f797c003d60 con 0x5604294d47e0 2024-02-02T15:36:22.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.622+0000 7f79a054db80 10 client.18027 fetch_fsmap learned FSMap version 44 2024-02-02T15:36:22.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.622+0000 7f79a054db80 1 -- 172.21.15.1:0/3032879086 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x5604294c3df0 con 0x5604294d47e0 2024-02-02T15:36:22.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.622+0000 7f7973400700 1 -- 172.21.15.1:0/3032879086 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f79740993c0 con 0x5604294d47e0 2024-02-02T15:36:22.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.622+0000 7f7973400700 1 -- 172.21.15.1:0/3032879086 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 44) v1 ==== 7510+0+0 (secure 0 0 0) 0x7f797c03e020 con 0x5604294d47e0 2024-02-02T15:36:22.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.622+0000 7f79a054db80 10 client.18027 fetch_fsmap finished waiting for FSMap version 44 2024-02-02T15:36:22.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.622+0000 7f79a054db80 10 client.18027 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-02-02T15:36:22.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.622+0000 7f79a054db80 20 client.18027 populate_metadata read hostname 'smithi001' 2024-02-02T15:36:22.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.623+0000 7f79a054db80 1 --2- 172.21.15.1:0/3032879086 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x560429536060 0x560429538450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:22.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.623+0000 7f79a054db80 4 client.18027 mds_command: new command op to 24514 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:36:22.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.623+0000 7f79a054db80 1 -- 172.21.15.1:0/3032879086 --> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5604293d9f50 con 0x560429536060 2024-02-02T15:36:22.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.623+0000 7f798a000700 1 --2- 172.21.15.1:0/3032879086 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x560429536060 0x560429538450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:22.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.623+0000 7f798a000700 1 --2- 172.21.15.1:0/3032879086 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x560429536060 0x560429538450 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=782910f83f8fd4d5 server_cookie=5db292b976b4bcd4 in_seq=0 out_seq=0 2024-02-02T15:36:22.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.623+0000 7f7973400700 10 client.18027 ms_handle_connect on v2:172.21.15.142:6836/2870289019 2024-02-02T15:36:22.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.623+0000 7f7973400700 1 -- 172.21.15.1:0/3032879086 <== mds.3 v2:172.21.15.142:6836/2870289019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5604293d9f50 con 0x560429536060 2024-02-02T15:36:22.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.623+0000 7f7973400700 10 client.18027 handle_command_reply: tid=0 2024-02-02T15:36:22.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.623+0000 7f79a054db80 1 client.18027 shutdown 2024-02-02T15:36:22.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.624+0000 7f79a054db80 1 -- 172.21.15.1:0/3032879086 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5604294d47e0 msgr2=0x5604294d98b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:22.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.624+0000 7f79a054db80 1 --2- 172.21.15.1:0/3032879086 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5604294d47e0 0x5604294d98b0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f797c003cb0 tx=0x7f797c009a10 comp rx=0 tx=0).stop 2024-02-02T15:36:22.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.624+0000 7f79a054db80 1 -- 172.21.15.1:0/3032879086 shutdown_connections 2024-02-02T15:36:22.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.624+0000 7f79a054db80 1 --2- 172.21.15.1:0/3032879086 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x560429536060 0x560429538450 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.624+0000 7f79a054db80 1 --2- 172.21.15.1:0/3032879086 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5604294d47e0 0x5604294d98b0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.624+0000 7f79a054db80 1 --2- 172.21.15.1:0/3032879086 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5604294c9010 0x5604294db330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.624+0000 7f79a054db80 1 --2- 172.21.15.1:0/3032879086 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5604294c9930 0x5604294d9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.624+0000 7f79a054db80 1 -- 172.21.15.1:0/3032879086 >> 172.21.15.1:0/3032879086 conn(0x560429428db0 msgr2=0x5604294cc8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:22.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.624+0000 7f7973400700 0 client.18027 ms_handle_reset on v2:172.21.15.142:6836/2870289019 2024-02-02T15:36:22.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.624+0000 7f79a054db80 1 -- 172.21.15.1:0/3032879086 shutdown_connections 2024-02-02T15:36:22.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.624+0000 7f79a054db80 1 -- 172.21.15.1:0/3032879086 wait complete. 2024-02-02T15:36:22.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.624+0000 7f79a054db80 20 client.18027 trim_cache size 0 max 16384 2024-02-02T15:36:22.629 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:36:22.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.625+0000 7f798ca00700 1 -- 172.21.15.1:0/608109113 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f79640a0690 msgr2=0x7f79640a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:22.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.625+0000 7f798ca00700 1 --2- 172.21.15.1:0/608109113 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f79640a0690 0x7f79640a2b50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f797c009550 tx=0x7f797c00b040 comp rx=0 tx=0).stop 2024-02-02T15:36:22.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.625+0000 7f798ca00700 1 -- 172.21.15.1:0/608109113 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79840685e0 msgr2=0x7f7984199130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:22.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.625+0000 7f798ca00700 1 --2- 172.21.15.1:0/608109113 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79840685e0 0x7f7984199130 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f797400e9f0 tx=0x7f797400ed00 comp rx=0 tx=0).stop 2024-02-02T15:36:22.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.625+0000 7f798ca00700 1 -- 172.21.15.1:0/608109113 shutdown_connections 2024-02-02T15:36:22.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.625+0000 7f798ca00700 1 --2- 172.21.15.1:0/608109113 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f79640a0690 0x7f79640a2b50 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.625+0000 7f798ca00700 1 --2- 172.21.15.1:0/608109113 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f798410d610 0x7f798419dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.625+0000 7f798ca00700 1 --2- 172.21.15.1:0/608109113 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f79840685e0 0x7f7984199130 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.625+0000 7f798ca00700 1 --2- 172.21.15.1:0/608109113 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7984101c70 0x7f7984199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.625+0000 7f798ca00700 1 -- 172.21.15.1:0/608109113 >> 172.21.15.1:0/608109113 conn(0x7f79840757d0 msgr2=0x7f79840fee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:22.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.625+0000 7f798ca00700 1 -- 172.21.15.1:0/608109113 shutdown_connections 2024-02-02T15:36:22.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.625+0000 7f798ca00700 1 -- 172.21.15.1:0/608109113 wait complete. 2024-02-02T15:36:22.643 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:22 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/2365878231' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:36:22.646 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-02T15:36:22.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:22 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/2365878231' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:36:22.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.890+0000 7fe38ee00700 1 -- 172.21.15.1:0/1204620466 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe388100ff0 msgr2=0x7fe3881013d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:22.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.890+0000 7fe38ee00700 1 --2- 172.21.15.1:0/1204620466 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe388100ff0 0x7fe3881013d0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fe378009990 tx=0x7fe378009ca0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.891+0000 7fe38ee00700 1 -- 172.21.15.1:0/1204620466 shutdown_connections 2024-02-02T15:36:22.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.891+0000 7fe38ee00700 1 --2- 172.21.15.1:0/1204620466 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe388106110 0x7fe38810dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.891+0000 7fe38ee00700 1 --2- 172.21.15.1:0/1204620466 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe388100ff0 0x7fe3881013d0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.891+0000 7fe38ee00700 1 --2- 172.21.15.1:0/1204620466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3881019a0 0x7fe3881059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.891+0000 7fe38ee00700 1 -- 172.21.15.1:0/1204620466 >> 172.21.15.1:0/1204620466 conn(0x7fe3880fc810 msgr2=0x7fe3880fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:22.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.891+0000 7fe38ee00700 1 -- 172.21.15.1:0/1204620466 shutdown_connections 2024-02-02T15:36:22.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.891+0000 7fe38ee00700 1 -- 172.21.15.1:0/1204620466 wait complete. 2024-02-02T15:36:22.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.892+0000 7fe38ee00700 1 Processor -- start 2024-02-02T15:36:22.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.892+0000 7fe38ee00700 1 -- start start 2024-02-02T15:36:22.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.892+0000 7fe38ee00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe388100ff0 0x7fe3881990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:22.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.892+0000 7fe38ee00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe388106110 0x7fe388199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:22.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.892+0000 7fe38ee00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe38819da60 0x7fe38819dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:22.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.892+0000 7fe38ee00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe38819e4e0 con 0x7fe388100ff0 2024-02-02T15:36:22.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.892+0000 7fe38ee00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe38819e650 con 0x7fe388106110 2024-02-02T15:36:22.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.892+0000 7fe38ee00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe38819e810 con 0x7fe38819da60 2024-02-02T15:36:22.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.893+0000 7fe387400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe388106110 0x7fe388199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:22.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.893+0000 7fe387e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe388100ff0 0x7fe3881990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:22.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.893+0000 7fe387400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe388106110 0x7fe388199600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:39768/0 (socket says 172.21.15.1:39768) 2024-02-02T15:36:22.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.893+0000 7fe387400700 1 -- 172.21.15.1:0/3915829971 learned_addr learned my addr 172.21.15.1:0/3915829971 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:36:22.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.893+0000 7fe38ce00700 1 --2- 172.21.15.1:0/3915829971 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe38819da60 0x7fe38819dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:22.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.893+0000 7fe387400700 1 -- 172.21.15.1:0/3915829971 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe38819da60 msgr2=0x7fe38819dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:22.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.893+0000 7fe387400700 1 --2- 172.21.15.1:0/3915829971 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe38819da60 0x7fe38819dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.893+0000 7fe387400700 1 -- 172.21.15.1:0/3915829971 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe388100ff0 msgr2=0x7fe3881990c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:22.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.893+0000 7fe387400700 1 --2- 172.21.15.1:0/3915829971 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe388100ff0 0x7fe3881990c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.893+0000 7fe387400700 1 -- 172.21.15.1:0/3915829971 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3780095f0 con 0x7fe388106110 2024-02-02T15:36:22.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.893+0000 7fe387400700 1 --2- 172.21.15.1:0/3915829971 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe388106110 0x7fe388199600 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fe37000e970 tx=0x7fe37000ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:22.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.894+0000 7fe384c00700 1 -- 172.21.15.1:0/3915829971 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe370009790 con 0x7fe388106110 2024-02-02T15:36:22.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.894+0000 7fe384c00700 1 -- 172.21.15.1:0/3915829971 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe37000c7e0 con 0x7fe388106110 2024-02-02T15:36:22.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.894+0000 7fe38ee00700 1 -- 172.21.15.1:0/3915829971 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3881a5d70 con 0x7fe388106110 2024-02-02T15:36:22.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.894+0000 7fe384c00700 1 -- 172.21.15.1:0/3915829971 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3700054e0 con 0x7fe388106110 2024-02-02T15:36:22.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.894+0000 7fe38ee00700 1 -- 172.21.15.1:0/3915829971 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3881a61f0 con 0x7fe388106110 2024-02-02T15:36:22.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.897+0000 7fe384c00700 1 -- 172.21.15.1:0/3915829971 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7fe37000c950 con 0x7fe388106110 2024-02-02T15:36:22.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.898+0000 7fe384c00700 1 --2- 172.21.15.1:0/3915829971 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fe3680a0690 0x7fe3680a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:22.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.898+0000 7fe384c00700 1 -- 172.21.15.1:0/3915829971 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fe370014070 con 0x7fe388106110 2024-02-02T15:36:22.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.899+0000 7fe387e00700 1 --2- 172.21.15.1:0/3915829971 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fe3680a0690 0x7fe3680a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:22.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.899+0000 7fe3a299db80 1 Processor -- start 2024-02-02T15:36:22.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.899+0000 7fe3a299db80 1 -- start start 2024-02-02T15:36:22.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.899+0000 7fe3a299db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf40f8e010 0x55cf40f8e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:22.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.899+0000 7fe3a299db80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cf40f8e930 0x55cf40f992a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:22.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.899+0000 7fe3a299db80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cf40f997e0 0x55cf40f9bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:22.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.899+0000 7fe3a299db80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf40f024c0 con 0x55cf40f8e930 2024-02-02T15:36:22.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.899+0000 7fe3a299db80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf40ee6aa0 con 0x55cf40f997e0 2024-02-02T15:36:22.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.899+0000 7fe3a299db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf40ee4460 con 0x55cf40f8e010 2024-02-02T15:36:22.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.899+0000 7fe387e00700 1 --2- 172.21.15.1:0/3915829971 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fe3680a0690 0x7fe3680a2b50 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe37800b2e0 tx=0x7fe378005fb0 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:22.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.900+0000 7fe387e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf40f8e010 0x55cf40f8e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:22.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.900+0000 7fe38ce00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cf40f997e0 0x55cf40f9bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:22.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.900+0000 7fe387e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf40f8e010 0x55cf40f8e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:60630/0 (socket says 172.21.15.1:60630) 2024-02-02T15:36:22.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.900+0000 7fe38ce00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cf40f997e0 0x55cf40f9bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:39780/0 (socket says 172.21.15.1:39780) 2024-02-02T15:36:22.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.900+0000 7fe387e00700 1 -- 172.21.15.1:0/2829915783 learned_addr learned my addr 172.21.15.1:0/2829915783 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:36:22.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.900+0000 7fe387400700 1 --2- 172.21.15.1:0/2829915783 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cf40f8e930 0x55cf40f992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:22.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.900+0000 7fe387e00700 1 -- 172.21.15.1:0/2829915783 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cf40f997e0 msgr2=0x55cf40f9bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:22.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.900+0000 7fe387e00700 1 --2- 172.21.15.1:0/2829915783 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cf40f997e0 0x55cf40f9bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.900+0000 7fe387e00700 1 -- 172.21.15.1:0/2829915783 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cf40f8e930 msgr2=0x55cf40f992a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:22.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.900+0000 7fe387e00700 1 --2- 172.21.15.1:0/2829915783 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cf40f8e930 0x55cf40f992a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.900+0000 7fe387e00700 1 -- 172.21.15.1:0/2829915783 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cf40d613b0 con 0x55cf40f8e010 2024-02-02T15:36:22.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.900+0000 7fe387400700 1 --2- 172.21.15.1:0/2829915783 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cf40f8e930 0x55cf40f992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:36:22.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.900+0000 7fe387e00700 1 --2- 172.21.15.1:0/2829915783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf40f8e010 0x55cf40f8e3f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fe378015000 tx=0x7fe378020000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69f41ff28cec0886 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:22.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.901+0000 7fe37d600700 1 -- 172.21.15.1:0/2829915783 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe378004700 con 0x55cf40f8e010 2024-02-02T15:36:22.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.901+0000 7fe37d600700 1 -- 172.21.15.1:0/2829915783 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe37801e040 con 0x55cf40f8e010 2024-02-02T15:36:22.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.901+0000 7fe37d600700 1 -- 172.21.15.1:0/2829915783 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37800fb00 con 0x55cf40f8e010 2024-02-02T15:36:22.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.901+0000 7fe3a299db80 1 -- 172.21.15.1:0/2829915783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf40f8e010 msgr2=0x55cf40f8e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:22.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.901+0000 7fe3a299db80 1 --2- 172.21.15.1:0/2829915783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf40f8e010 0x55cf40f8e3f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fe378015000 tx=0x7fe378020000 comp rx=0 tx=0).stop 2024-02-02T15:36:22.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.902+0000 7fe3a299db80 1 -- 172.21.15.1:0/2829915783 shutdown_connections 2024-02-02T15:36:22.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.902+0000 7fe3a299db80 1 --2- 172.21.15.1:0/2829915783 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cf40f997e0 0x55cf40f9bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.902+0000 7fe3a299db80 1 --2- 172.21.15.1:0/2829915783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf40f8e010 0x55cf40f8e3f0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.902+0000 7fe3a299db80 1 --2- 172.21.15.1:0/2829915783 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cf40f8e930 0x55cf40f992a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.902+0000 7fe3a299db80 1 -- 172.21.15.1:0/2829915783 >> 172.21.15.1:0/2829915783 conn(0x55cf40eeddb0 msgr2=0x55cf40f8a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:22.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.902+0000 7fe3a299db80 1 -- 172.21.15.1:0/2829915783 shutdown_connections 2024-02-02T15:36:22.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.902+0000 7fe3a299db80 1 -- 172.21.15.1:0/2829915783 wait complete. 2024-02-02T15:36:22.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.902+0000 7fe3a299db80 1 -- start start 2024-02-02T15:36:22.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.902+0000 7fe3a299db80 1 Processor -- start 2024-02-02T15:36:22.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.903+0000 7fe3a299db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf40f8e010 0x55cf40f99240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:22.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.903+0000 7fe3a299db80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cf40f8e930 0x55cf40f964c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:22.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.903+0000 7fe3a299db80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cf40f997e0 0x55cf40f96c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:22.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.903+0000 7fe3a299db80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf40edfd50 con 0x55cf40f8e930 2024-02-02T15:36:22.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.903+0000 7fe3a299db80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf40e7bd80 con 0x55cf40f997e0 2024-02-02T15:36:22.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.903+0000 7fe3a299db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf40e7ba20 con 0x55cf40f8e010 2024-02-02T15:36:22.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.903+0000 7fe3a299db80 10 client.0 _ll_register_callbacks cb 0x55cf40e7d290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:36:22.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.904+0000 7fe387400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cf40f8e930 0x55cf40f964c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:22.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.904+0000 7fe387e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf40f8e010 0x55cf40f99240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:22.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.904+0000 7fe387e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf40f8e010 0x55cf40f99240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:60638/0 (socket says 172.21.15.1:60638) 2024-02-02T15:36:22.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.904+0000 7fe387e00700 1 -- 172.21.15.1:0/3574368220 learned_addr learned my addr 172.21.15.1:0/3574368220 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:36:22.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.904+0000 7fe38ce00700 1 --2- 172.21.15.1:0/3574368220 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cf40f997e0 0x55cf40f96c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:22.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.904+0000 7fe387e00700 1 -- 172.21.15.1:0/3574368220 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cf40f997e0 msgr2=0x55cf40f96c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:22.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.904+0000 7fe387e00700 1 --2- 172.21.15.1:0/3574368220 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cf40f997e0 0x55cf40f96c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.904+0000 7fe387e00700 1 -- 172.21.15.1:0/3574368220 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cf40f8e930 msgr2=0x55cf40f964c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:22.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.904+0000 7fe387e00700 1 --2- 172.21.15.1:0/3574368220 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cf40f8e930 0x55cf40f964c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.904+0000 7fe387e00700 1 -- 172.21.15.1:0/3574368220 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3881a61f0 con 0x55cf40f8e010 2024-02-02T15:36:22.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.904+0000 7fe38ce00700 1 --2- 172.21.15.1:0/3574368220 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cf40f997e0 0x55cf40f96c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:36:22.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.904+0000 7fe387e00700 1 --2- 172.21.15.1:0/3574368220 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf40f8e010 0x55cf40f99240 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fe3780095c0 tx=0x7fe37800fe00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e16dee8eeadd9a0f server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:22.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.904+0000 7fe376000700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:36:22.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.905+0000 7fe376000700 1 -- 172.21.15.1:0/3574368220 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe378004910 con 0x55cf40f8e010 2024-02-02T15:36:22.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.905+0000 7fe376000700 1 -- 172.21.15.1:0/3574368220 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fe37800bca0 con 0x55cf40f8e010 2024-02-02T15:36:22.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.905+0000 7fe3a299db80 1 -- 172.21.15.1:0/3574368220 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cf40f820f0 con 0x55cf40f8e010 2024-02-02T15:36:22.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.905+0000 7fe376000700 1 -- 172.21.15.1:0/3574368220 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe378004a70 con 0x55cf40f8e010 2024-02-02T15:36:22.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.905+0000 7fe3a299db80 10 client.28120 fetch_fsmap learned FSMap version 44 2024-02-02T15:36:22.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.905+0000 7fe3a299db80 1 -- 172.21.15.1:0/3574368220 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x7fe38810e500 con 0x55cf40f8e010 2024-02-02T15:36:22.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.905+0000 7fe376000700 1 -- 172.21.15.1:0/3574368220 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55cf40f820f0 con 0x55cf40f8e010 2024-02-02T15:36:22.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.905+0000 7fe376000700 1 -- 172.21.15.1:0/3574368220 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 44) v1 ==== 7510+0+0 (secure 0 0 0) 0x7fe378024bd0 con 0x55cf40f8e010 2024-02-02T15:36:22.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.905+0000 7fe3a299db80 10 client.28120 fetch_fsmap finished waiting for FSMap version 44 2024-02-02T15:36:22.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.905+0000 7fe3a299db80 10 client.28120 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-02-02T15:36:22.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.905+0000 7fe3a299db80 20 client.28120 populate_metadata read hostname 'smithi001' 2024-02-02T15:36:22.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.905+0000 7fe3a299db80 1 --2- 172.21.15.1:0/3574368220 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55cf40fa8d70 0x55cf40fab160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:22.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.905+0000 7fe3a299db80 4 client.28120 mds_command: new command op to 24518 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:36:22.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.905+0000 7fe3a299db80 1 -- 172.21.15.1:0/3574368220 --> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cf40ed8dd0 con 0x55cf40fa8d70 2024-02-02T15:36:22.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.906+0000 7fe387400700 1 --2- 172.21.15.1:0/3574368220 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55cf40fa8d70 0x55cf40fab160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:22.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.906+0000 7fe387400700 1 --2- 172.21.15.1:0/3574368220 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55cf40fa8d70 0x55cf40fab160 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=be7c08990843d974 server_cookie=3fa8a72ab44c1807 in_seq=0 out_seq=0 2024-02-02T15:36:22.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.906+0000 7fe376000700 10 client.28120 ms_handle_connect on v2:172.21.15.71:6836/628016199 2024-02-02T15:36:22.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.907+0000 7fe376000700 1 -- 172.21.15.1:0/3574368220 <== mds.2 v2:172.21.15.71:6836/628016199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55cf40ed8dd0 con 0x55cf40fa8d70 2024-02-02T15:36:22.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.907+0000 7fe376000700 10 client.28120 handle_command_reply: tid=0 2024-02-02T15:36:22.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.907+0000 7fe3a299db80 1 client.28120 shutdown 2024-02-02T15:36:22.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.907+0000 7fe3a299db80 1 -- 172.21.15.1:0/3574368220 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf40f8e010 msgr2=0x55cf40f99240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:22.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.907+0000 7fe3a299db80 1 --2- 172.21.15.1:0/3574368220 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf40f8e010 0x55cf40f99240 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fe3780095c0 tx=0x7fe37800fe00 comp rx=0 tx=0).stop 2024-02-02T15:36:22.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.907+0000 7fe3a299db80 1 -- 172.21.15.1:0/3574368220 shutdown_connections 2024-02-02T15:36:22.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.907+0000 7fe3a299db80 1 --2- 172.21.15.1:0/3574368220 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cf40f997e0 0x55cf40f96c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.907+0000 7fe3a299db80 1 --2- 172.21.15.1:0/3574368220 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55cf40fa8d70 0x55cf40fab160 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.907+0000 7fe3a299db80 1 --2- 172.21.15.1:0/3574368220 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf40f8e010 0x55cf40f99240 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.907+0000 7fe3a299db80 1 --2- 172.21.15.1:0/3574368220 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cf40f8e930 0x55cf40f964c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.907+0000 7fe376000700 0 client.28120 ms_handle_reset on v2:172.21.15.71:6836/628016199 2024-02-02T15:36:22.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.907+0000 7fe3a299db80 1 -- 172.21.15.1:0/3574368220 >> 172.21.15.1:0/3574368220 conn(0x55cf40eeddb0 msgr2=0x55cf40f89fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:22.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.907+0000 7fe3a299db80 1 -- 172.21.15.1:0/3574368220 shutdown_connections 2024-02-02T15:36:22.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.907+0000 7fe3a299db80 1 -- 172.21.15.1:0/3574368220 wait complete. 2024-02-02T15:36:22.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.907+0000 7fe3a299db80 20 client.28120 trim_cache size 0 max 16384 2024-02-02T15:36:22.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:22 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/2365878231' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:36:22.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.922+0000 7fe3a299db80 1 Processor -- start 2024-02-02T15:36:22.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.922+0000 7fe3a299db80 1 -- start start 2024-02-02T15:36:22.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.922+0000 7fe3a299db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf40f8e010 0x55cf40f90ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:22.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.922+0000 7fe3a299db80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cf40f8e930 0x55cf40f98d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:22.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.922+0000 7fe3a299db80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cf40f997e0 0x55cf40f917e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:22.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.922+0000 7fe3a299db80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf40d636a0 con 0x55cf40f8e930 2024-02-02T15:36:22.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.922+0000 7fe3a299db80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf40fa5180 con 0x55cf40f997e0 2024-02-02T15:36:22.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.922+0000 7fe3a299db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf40f97200 con 0x55cf40f8e010 2024-02-02T15:36:22.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.922+0000 7fe387e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf40f8e010 0x55cf40f90ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:22.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.922+0000 7fe38ce00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cf40f997e0 0x55cf40f917e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:22.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.922+0000 7fe387e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf40f8e010 0x55cf40f90ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:60644/0 (socket says 172.21.15.1:60644) 2024-02-02T15:36:22.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.922+0000 7fe387e00700 1 -- 172.21.15.1:0/3445661952 learned_addr learned my addr 172.21.15.1:0/3445661952 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:36:22.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.922+0000 7fe387400700 1 --2- 172.21.15.1:0/3445661952 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cf40f8e930 0x55cf40f98d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:22.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.923+0000 7fe387e00700 1 -- 172.21.15.1:0/3445661952 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cf40f997e0 msgr2=0x55cf40f917e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:22.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.923+0000 7fe387e00700 1 --2- 172.21.15.1:0/3445661952 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cf40f997e0 0x55cf40f917e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.923+0000 7fe387e00700 1 -- 172.21.15.1:0/3445661952 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cf40f8e930 msgr2=0x55cf40f98d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:22.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.923+0000 7fe387e00700 1 --2- 172.21.15.1:0/3445661952 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cf40f8e930 0x55cf40f98d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.923+0000 7fe387e00700 1 -- 172.21.15.1:0/3445661952 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cf40d0db60 con 0x55cf40f8e010 2024-02-02T15:36:22.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.923+0000 7fe38ce00700 1 --2- 172.21.15.1:0/3445661952 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cf40f997e0 0x55cf40f917e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:36:22.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.923+0000 7fe387e00700 1 --2- 172.21.15.1:0/3445661952 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf40f8e010 0x55cf40f90ac0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fe37800b6f0 tx=0x7fe37800f550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33e08a45e5aa82bf server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:22.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.923+0000 7fe37e000700 1 -- 172.21.15.1:0/3445661952 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe378005600 con 0x55cf40f8e010 2024-02-02T15:36:22.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.923+0000 7fe37e000700 1 -- 172.21.15.1:0/3445661952 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe378004bd0 con 0x55cf40f8e010 2024-02-02T15:36:22.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.923+0000 7fe37e000700 1 -- 172.21.15.1:0/3445661952 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37802fd50 con 0x55cf40f8e010 2024-02-02T15:36:22.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.924+0000 7fe3a299db80 1 -- 172.21.15.1:0/3445661952 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf40f8e010 msgr2=0x55cf40f90ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:22.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.924+0000 7fe3a299db80 1 --2- 172.21.15.1:0/3445661952 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf40f8e010 0x55cf40f90ac0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fe37800b6f0 tx=0x7fe37800f550 comp rx=0 tx=0).stop 2024-02-02T15:36:22.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.924+0000 7fe3a299db80 1 -- 172.21.15.1:0/3445661952 shutdown_connections 2024-02-02T15:36:22.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.924+0000 7fe3a299db80 1 --2- 172.21.15.1:0/3445661952 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cf40f997e0 0x55cf40f917e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.924+0000 7fe3a299db80 1 --2- 172.21.15.1:0/3445661952 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf40f8e010 0x55cf40f90ac0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.924+0000 7fe3a299db80 1 --2- 172.21.15.1:0/3445661952 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cf40f8e930 0x55cf40f98d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.924+0000 7fe3a299db80 1 -- 172.21.15.1:0/3445661952 >> 172.21.15.1:0/3445661952 conn(0x55cf40eeddb0 msgr2=0x55cf40f88b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:22.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.924+0000 7fe3a299db80 1 -- 172.21.15.1:0/3445661952 shutdown_connections 2024-02-02T15:36:22.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.924+0000 7fe3a299db80 1 -- 172.21.15.1:0/3445661952 wait complete. 2024-02-02T15:36:22.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.925+0000 7fe3a299db80 1 -- start start 2024-02-02T15:36:22.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.925+0000 7fe3a299db80 1 Processor -- start 2024-02-02T15:36:22.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.925+0000 7fe3a299db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf40f8e010 0x55cf40f93ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:22.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.925+0000 7fe3a299db80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cf40f8e930 0x55cf40f9e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:22.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.925+0000 7fe3a299db80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cf40f997e0 0x55cf40f9e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:22.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.925+0000 7fe3a299db80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf40f9ed80 con 0x55cf40f997e0 2024-02-02T15:36:22.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.925+0000 7fe3a299db80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf40f9eef0 con 0x55cf40f8e930 2024-02-02T15:36:22.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.925+0000 7fe3a299db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf40f9f0b0 con 0x55cf40f8e010 2024-02-02T15:36:22.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.926+0000 7fe3a299db80 10 client.0 _ll_register_callbacks cb 0x55cf40e7d290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:36:22.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.926+0000 7fe387400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cf40f8e930 0x55cf40f9e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:22.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.926+0000 7fe387e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf40f8e010 0x55cf40f93ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:22.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.926+0000 7fe38ce00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cf40f997e0 0x55cf40f9e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:22.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.926+0000 7fe387400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cf40f8e930 0x55cf40f9e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:39810/0 (socket says 172.21.15.1:39810) 2024-02-02T15:36:22.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.926+0000 7fe387400700 1 -- 172.21.15.1:0/320643250 learned_addr learned my addr 172.21.15.1:0/320643250 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:36:22.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.926+0000 7fe387e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf40f8e010 0x55cf40f93ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:60654/0 (socket says 172.21.15.1:60654) 2024-02-02T15:36:22.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.926+0000 7fe387e00700 1 -- 172.21.15.1:0/320643250 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cf40f8e930 msgr2=0x55cf40f9e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:22.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.926+0000 7fe387e00700 1 --2- 172.21.15.1:0/320643250 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cf40f8e930 0x55cf40f9e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.926+0000 7fe387e00700 1 -- 172.21.15.1:0/320643250 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cf40f997e0 msgr2=0x55cf40f9e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:22.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.926+0000 7fe387e00700 1 --2- 172.21.15.1:0/320643250 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cf40f997e0 0x55cf40f9e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.926+0000 7fe387e00700 1 -- 172.21.15.1:0/320643250 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe380009a50 con 0x55cf40f8e010 2024-02-02T15:36:22.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.926+0000 7fe387400700 1 --2- 172.21.15.1:0/320643250 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cf40f8e930 0x55cf40f9e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:36:22.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.926+0000 7fe387e00700 1 --2- 172.21.15.1:0/320643250 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf40f8e010 0x55cf40f93ff0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fe37800bf50 tx=0x7fe37800fd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3262e93b2e1e76 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:22.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.926+0000 7fe376000700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:36:22.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.927+0000 7fe376000700 1 -- 172.21.15.1:0/320643250 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe378024610 con 0x55cf40f8e010 2024-02-02T15:36:22.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.927+0000 7fe3a299db80 1 -- 172.21.15.1:0/320643250 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe37801e040 con 0x55cf40f8e010 2024-02-02T15:36:22.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.927+0000 7fe376000700 1 -- 172.21.15.1:0/320643250 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fe378024770 con 0x55cf40f8e010 2024-02-02T15:36:22.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.927+0000 7fe376000700 1 -- 172.21.15.1:0/320643250 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe378024930 con 0x55cf40f8e010 2024-02-02T15:36:22.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.927+0000 7fe3a299db80 10 client.28126 fetch_fsmap learned FSMap version 44 2024-02-02T15:36:22.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.927+0000 7fe3a299db80 1 -- 172.21.15.1:0/320643250 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x7fe38810e500 con 0x55cf40f8e010 2024-02-02T15:36:22.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.927+0000 7fe376000700 1 -- 172.21.15.1:0/320643250 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe37801e040 con 0x55cf40f8e010 2024-02-02T15:36:22.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.927+0000 7fe376000700 1 -- 172.21.15.1:0/320643250 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 44) v1 ==== 7510+0+0 (secure 0 0 0) 0x7fe37804e080 con 0x55cf40f8e010 2024-02-02T15:36:22.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.927+0000 7fe3a299db80 10 client.28126 fetch_fsmap finished waiting for FSMap version 44 2024-02-02T15:36:22.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.927+0000 7fe3a299db80 10 client.28126 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-02-02T15:36:22.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.927+0000 7fe3a299db80 20 client.28126 populate_metadata read hostname 'smithi001' 2024-02-02T15:36:22.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.927+0000 7fe3a299db80 1 --2- 172.21.15.1:0/320643250 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55cf40ffa7c0 0x55cf40ffcbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:22.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.927+0000 7fe3a299db80 4 client.28126 mds_command: new command op to 24518 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:36:22.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.927+0000 7fe3a299db80 1 -- 172.21.15.1:0/320643250 --> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55cf40e9ef50 con 0x55cf40ffa7c0 2024-02-02T15:36:22.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.928+0000 7fe387400700 1 --2- 172.21.15.1:0/320643250 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55cf40ffa7c0 0x55cf40ffcbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:22.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.928+0000 7fe376000700 10 client.28126 ms_handle_connect on v2:172.21.15.71:6836/628016199 2024-02-02T15:36:22.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.928+0000 7fe387400700 1 --2- 172.21.15.1:0/320643250 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55cf40ffa7c0 0x55cf40ffcbe0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ead8bd30cb7a448c server_cookie=f6d51abb664b7c06 in_seq=0 out_seq=0 2024-02-02T15:36:22.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.928+0000 7fe376000700 1 -- 172.21.15.1:0/320643250 <== mds.2 v2:172.21.15.71:6836/628016199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55cf40e9ef50 con 0x55cf40ffa7c0 2024-02-02T15:36:22.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.928+0000 7fe376000700 10 client.28126 handle_command_reply: tid=0 2024-02-02T15:36:22.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.928+0000 7fe3a299db80 1 client.28126 shutdown 2024-02-02T15:36:22.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.928+0000 7fe3a299db80 1 -- 172.21.15.1:0/320643250 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf40f8e010 msgr2=0x55cf40f93ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:22.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.928+0000 7fe3a299db80 1 --2- 172.21.15.1:0/320643250 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf40f8e010 0x55cf40f93ff0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fe37800bf50 tx=0x7fe37800fd80 comp rx=0 tx=0).stop 2024-02-02T15:36:22.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.928+0000 7fe3a299db80 1 -- 172.21.15.1:0/320643250 shutdown_connections 2024-02-02T15:36:22.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.928+0000 7fe3a299db80 1 --2- 172.21.15.1:0/320643250 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55cf40f997e0 0x55cf40f9e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.928+0000 7fe3a299db80 1 --2- 172.21.15.1:0/320643250 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55cf40ffa7c0 0x55cf40ffcbe0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.928+0000 7fe3a299db80 1 --2- 172.21.15.1:0/320643250 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55cf40f8e010 0x55cf40f93ff0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.928+0000 7fe3a299db80 1 --2- 172.21.15.1:0/320643250 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55cf40f8e930 0x55cf40f9e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.928+0000 7fe376000700 0 client.28126 ms_handle_reset on v2:172.21.15.71:6836/628016199 2024-02-02T15:36:22.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.928+0000 7fe3a299db80 1 -- 172.21.15.1:0/320643250 >> 172.21.15.1:0/320643250 conn(0x55cf40eeddb0 msgr2=0x55cf40f917e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:22.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.929+0000 7fe3a299db80 1 -- 172.21.15.1:0/320643250 shutdown_connections 2024-02-02T15:36:22.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.929+0000 7fe3a299db80 1 -- 172.21.15.1:0/320643250 wait complete. 2024-02-02T15:36:22.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.929+0000 7fe3a299db80 20 client.28126 trim_cache size 0 max 16384 2024-02-02T15:36:22.934 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:36:22.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.929+0000 7fe38ee00700 1 -- 172.21.15.1:0/3915829971 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fe3680a0690 msgr2=0x7fe3680a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:22.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.929+0000 7fe38ee00700 1 --2- 172.21.15.1:0/3915829971 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fe3680a0690 0x7fe3680a2b50 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe37800b2e0 tx=0x7fe378005fb0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.929+0000 7fe38ee00700 1 -- 172.21.15.1:0/3915829971 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe388106110 msgr2=0x7fe388199600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:22.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.929+0000 7fe38ee00700 1 --2- 172.21.15.1:0/3915829971 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe388106110 0x7fe388199600 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fe37000e970 tx=0x7fe37000ec80 comp rx=0 tx=0).stop 2024-02-02T15:36:22.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.929+0000 7fe38ee00700 1 -- 172.21.15.1:0/3915829971 shutdown_connections 2024-02-02T15:36:22.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.930+0000 7fe38ee00700 1 --2- 172.21.15.1:0/3915829971 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fe3680a0690 0x7fe3680a2b50 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.930+0000 7fe38ee00700 1 --2- 172.21.15.1:0/3915829971 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe38819da60 0x7fe38819dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.930+0000 7fe38ee00700 1 --2- 172.21.15.1:0/3915829971 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe388100ff0 0x7fe3881990c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.930+0000 7fe38ee00700 1 --2- 172.21.15.1:0/3915829971 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe388106110 0x7fe388199600 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:22.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.930+0000 7fe38ee00700 1 -- 172.21.15.1:0/3915829971 >> 172.21.15.1:0/3915829971 conn(0x7fe3880fc810 msgr2=0x7fe3881029b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:22.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.930+0000 7fe38ee00700 1 -- 172.21.15.1:0/3915829971 shutdown_connections 2024-02-02T15:36:22.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:22.930+0000 7fe38ee00700 1 -- 172.21.15.1:0/3915829971 wait complete. 2024-02-02T15:36:22.948 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-02T15:36:23.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.162+0000 7f668c800700 1 -- 172.21.15.1:0/1194249505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6684100ff0 msgr2=0x7f66841013d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:23.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.162+0000 7f668c800700 1 --2- 172.21.15.1:0/1194249505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6684100ff0 0x7f66841013d0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f6674009990 tx=0x7f6674009ca0 comp rx=0 tx=0).stop 2024-02-02T15:36:23.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.163+0000 7f668c800700 1 -- 172.21.15.1:0/1194249505 shutdown_connections 2024-02-02T15:36:23.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.163+0000 7f668c800700 1 --2- 172.21.15.1:0/1194249505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6684106110 0x7f668410dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:23.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.163+0000 7f668c800700 1 --2- 172.21.15.1:0/1194249505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6684100ff0 0x7f66841013d0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:23.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.163+0000 7f668c800700 1 --2- 172.21.15.1:0/1194249505 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f66841019a0 0x7f66841059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:23.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.163+0000 7f668c800700 1 -- 172.21.15.1:0/1194249505 >> 172.21.15.1:0/1194249505 conn(0x7f66840fc810 msgr2=0x7f66840fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:23.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.163+0000 7f668c800700 1 -- 172.21.15.1:0/1194249505 shutdown_connections 2024-02-02T15:36:23.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.163+0000 7f668c800700 1 -- 172.21.15.1:0/1194249505 wait complete. 2024-02-02T15:36:23.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.164+0000 7f668c800700 1 Processor -- start 2024-02-02T15:36:23.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.164+0000 7f668c800700 1 -- start start 2024-02-02T15:36:23.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.164+0000 7f668c800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6684100ff0 0x7f6684196e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:23.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.164+0000 7f668c800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6684106110 0x7f6684197390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:23.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.164+0000 7f668c800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f668419b7f0 0x7f668419bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:23.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.164+0000 7f668c800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f668419c270 con 0x7f6684100ff0 2024-02-02T15:36:23.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.164+0000 7f668c800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f668419c3e0 con 0x7f668419b7f0 2024-02-02T15:36:23.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.164+0000 7f668c800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f668419c5a0 con 0x7f6684106110 2024-02-02T15:36:23.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.164+0000 7f6689e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6684100ff0 0x7f6684196e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:23.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.165+0000 7f668a800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f668419b7f0 0x7f668419bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:23.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.165+0000 7f6689400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6684106110 0x7f6684197390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:23.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.165+0000 7f668a800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f668419b7f0 0x7f668419bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:39838/0 (socket says 172.21.15.1:39838) 2024-02-02T15:36:23.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.165+0000 7f668a800700 1 -- 172.21.15.1:0/779745194 learned_addr learned my addr 172.21.15.1:0/779745194 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:36:23.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.165+0000 7f668a800700 1 -- 172.21.15.1:0/779745194 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6684106110 msgr2=0x7f6684197390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:23.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.165+0000 7f668a800700 1 --2- 172.21.15.1:0/779745194 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6684106110 0x7f6684197390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:23.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.165+0000 7f668a800700 1 -- 172.21.15.1:0/779745194 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6684100ff0 msgr2=0x7f6684196e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:23.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.165+0000 7f668a800700 1 --2- 172.21.15.1:0/779745194 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6684100ff0 0x7f6684196e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:23.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.165+0000 7f668a800700 1 -- 172.21.15.1:0/779745194 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66740095f0 con 0x7f668419b7f0 2024-02-02T15:36:23.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.165+0000 7f668a800700 1 --2- 172.21.15.1:0/779745194 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f668419b7f0 0x7f668419bc70 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f667c00ef30 tx=0x7f667c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:23.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.165+0000 7f6682a00700 1 -- 172.21.15.1:0/779745194 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f667c018070 con 0x7f668419b7f0 2024-02-02T15:36:23.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.165+0000 7f6682a00700 1 -- 172.21.15.1:0/779745194 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f667c00f040 con 0x7f668419b7f0 2024-02-02T15:36:23.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.165+0000 7f668c800700 1 -- 172.21.15.1:0/779745194 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66841a3aa0 con 0x7f668419b7f0 2024-02-02T15:36:23.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.165+0000 7f6682a00700 1 -- 172.21.15.1:0/779745194 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f667c00cd00 con 0x7f668419b7f0 2024-02-02T15:36:23.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.166+0000 7f668c800700 1 -- 172.21.15.1:0/779745194 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66841a3fd0 con 0x7f668419b7f0 2024-02-02T15:36:23.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.169+0000 7f6682a00700 1 -- 172.21.15.1:0/779745194 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f667c003a00 con 0x7f668419b7f0 2024-02-02T15:36:23.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.170+0000 7f66a02c6b80 1 Processor -- start 2024-02-02T15:36:23.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.170+0000 7f6682a00700 1 --2- 172.21.15.1:0/779745194 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f66640a0710 0x7f66640a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:23.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.170+0000 7f6682a00700 1 -- 172.21.15.1:0/779745194 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f667c0cbb60 con 0x7f668419b7f0 2024-02-02T15:36:23.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.170+0000 7f66a02c6b80 1 -- start start 2024-02-02T15:36:23.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.170+0000 7f6689400700 1 --2- 172.21.15.1:0/779745194 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f66640a0710 0x7f66640a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:23.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.171+0000 7f66a02c6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c894255010 0x55c8942553f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:23.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.171+0000 7f66a02c6b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c894255930 0x55c8942602a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:23.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.171+0000 7f66a02c6b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c8942607e0 0x55c894262bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:23.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.171+0000 7f66a02c6b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8941c94c0 con 0x55c8942607e0 2024-02-02T15:36:23.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.171+0000 7f66a02c6b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8941adaa0 con 0x55c894255930 2024-02-02T15:36:23.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.171+0000 7f66a02c6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8941ab460 con 0x55c894255010 2024-02-02T15:36:23.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.171+0000 7f6689e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c894255010 0x55c8942553f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:23.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.171+0000 7f6689e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c894255010 0x55c8942553f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:60674/0 (socket says 172.21.15.1:60674) 2024-02-02T15:36:23.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.171+0000 7f6689e00700 1 -- 172.21.15.1:0/2024048856 learned_addr learned my addr 172.21.15.1:0/2024048856 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:36:23.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.171+0000 7f668a800700 1 --2- 172.21.15.1:0/2024048856 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c8942607e0 0x55c894262bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:23.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.172+0000 7f6689400700 1 --2- 172.21.15.1:0/2024048856 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c894255930 0x55c8942602a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:23.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.172+0000 7f6689e00700 1 -- 172.21.15.1:0/2024048856 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c894255930 msgr2=0x55c8942602a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:23.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.172+0000 7f6689e00700 1 --2- 172.21.15.1:0/2024048856 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c894255930 0x55c8942602a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:23.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.172+0000 7f6689e00700 1 -- 172.21.15.1:0/2024048856 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c8942607e0 msgr2=0x55c894262bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:23.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.172+0000 7f6689e00700 1 --2- 172.21.15.1:0/2024048856 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c8942607e0 0x55c894262bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:23.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.172+0000 7f6689e00700 1 -- 172.21.15.1:0/2024048856 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c8940283b0 con 0x55c894255010 2024-02-02T15:36:23.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.172+0000 7f6689400700 1 --2- 172.21.15.1:0/2024048856 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c894255930 0x55c8942602a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:36:23.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.172+0000 7f668a800700 1 --2- 172.21.15.1:0/2024048856 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c8942607e0 0x55c894262bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:36:23.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.172+0000 7f6689400700 1 --2- 172.21.15.1:0/779745194 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f66640a0710 0x7f66640a2bd0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6684198460 tx=0x7f666c009260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:23.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.172+0000 7f6689e00700 1 --2- 172.21.15.1:0/2024048856 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c894255010 0x55c8942553f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f6674005f20 tx=0x7f667400b610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b7f11111878586c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:23.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.172+0000 7f667b000700 1 -- 172.21.15.1:0/2024048856 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f667401d030 con 0x55c894255010 2024-02-02T15:36:23.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.172+0000 7f667b000700 1 -- 172.21.15.1:0/2024048856 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f667401e040 con 0x55c894255010 2024-02-02T15:36:23.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.172+0000 7f667b000700 1 -- 172.21.15.1:0/2024048856 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6674004900 con 0x55c894255010 2024-02-02T15:36:23.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.173+0000 7f66a02c6b80 1 -- 172.21.15.1:0/2024048856 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c894255010 msgr2=0x55c8942553f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:23.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.173+0000 7f66a02c6b80 1 --2- 172.21.15.1:0/2024048856 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c894255010 0x55c8942553f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f6674005f20 tx=0x7f667400b610 comp rx=0 tx=0).stop 2024-02-02T15:36:23.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.173+0000 7f66a02c6b80 1 -- 172.21.15.1:0/2024048856 shutdown_connections 2024-02-02T15:36:23.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.173+0000 7f66a02c6b80 1 --2- 172.21.15.1:0/2024048856 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c8942607e0 0x55c894262bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:23.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.173+0000 7f66a02c6b80 1 --2- 172.21.15.1:0/2024048856 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c894255010 0x55c8942553f0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:23.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.173+0000 7f66a02c6b80 1 --2- 172.21.15.1:0/2024048856 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c894255930 0x55c8942602a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:23.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.173+0000 7f66a02c6b80 1 -- 172.21.15.1:0/2024048856 >> 172.21.15.1:0/2024048856 conn(0x55c8941b4db0 msgr2=0x55c8942513b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:23.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.173+0000 7f66a02c6b80 1 -- 172.21.15.1:0/2024048856 shutdown_connections 2024-02-02T15:36:23.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.173+0000 7f66a02c6b80 1 -- 172.21.15.1:0/2024048856 wait complete. 2024-02-02T15:36:23.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.174+0000 7f66a02c6b80 1 -- start start 2024-02-02T15:36:23.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.174+0000 7f66a02c6b80 1 Processor -- start 2024-02-02T15:36:23.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.174+0000 7f66a02c6b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c894255010 0x55c894260230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:23.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.174+0000 7f66a02c6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c894255930 0x55c89425d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:23.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.174+0000 7f66a02c6b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c8942607e0 0x55c89425db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:23.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.174+0000 7f66a02c6b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8941a6d50 con 0x55c8942607e0 2024-02-02T15:36:23.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.175+0000 7f66a02c6b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c894142d80 con 0x55c894255010 2024-02-02T15:36:23.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.175+0000 7f66a02c6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c894142a20 con 0x55c894255930 2024-02-02T15:36:23.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.175+0000 7f66a02c6b80 10 client.0 _ll_register_callbacks cb 0x55c894144290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:36:23.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.175+0000 7f6689e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c894255010 0x55c894260230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:23.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.175+0000 7f6689e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c894255010 0x55c894260230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:39856/0 (socket says 172.21.15.1:39856) 2024-02-02T15:36:23.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.175+0000 7f6689e00700 1 -- 172.21.15.1:0/3526787875 learned_addr learned my addr 172.21.15.1:0/3526787875 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:36:23.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.175+0000 7f6689400700 1 --2- 172.21.15.1:0/3526787875 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c894255930 0x55c89425d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:23.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.175+0000 7f668a800700 1 --2- 172.21.15.1:0/3526787875 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c8942607e0 0x55c89425db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:23.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.175+0000 7f6689e00700 1 -- 172.21.15.1:0/3526787875 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c894255930 msgr2=0x55c89425d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:23.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.175+0000 7f6689e00700 1 --2- 172.21.15.1:0/3526787875 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c894255930 0x55c89425d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:23.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.175+0000 7f6689e00700 1 -- 172.21.15.1:0/3526787875 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c8942607e0 msgr2=0x55c89425db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:23.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.175+0000 7f6689e00700 1 --2- 172.21.15.1:0/3526787875 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c8942607e0 0x55c89425db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:23.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.175+0000 7f6689e00700 1 -- 172.21.15.1:0/3526787875 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66841a3fd0 con 0x55c894255010 2024-02-02T15:36:23.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.175+0000 7f668a800700 1 --2- 172.21.15.1:0/3526787875 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c8942607e0 0x55c89425db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:36:23.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.175+0000 7f6689e00700 1 --2- 172.21.15.1:0/3526787875 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c894255010 0x55c894260230 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f6674006010 tx=0x7f6674005280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4fce7fa73d04b3bb server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:23.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.175+0000 7f6673400700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:36:23.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.176+0000 7f6673400700 1 -- 172.21.15.1:0/3526787875 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f667401d070 con 0x55c894255010 2024-02-02T15:36:23.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.176+0000 7f6673400700 1 -- 172.21.15.1:0/3526787875 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f6674005560 con 0x55c894255010 2024-02-02T15:36:23.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.176+0000 7f66a02c6b80 1 -- 172.21.15.1:0/3526787875 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c8942490f0 con 0x55c894255010 2024-02-02T15:36:23.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.176+0000 7f6673400700 1 -- 172.21.15.1:0/3526787875 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6674004910 con 0x55c894255010 2024-02-02T15:36:23.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.176+0000 7f66a02c6b80 10 client.28145 fetch_fsmap learned FSMap version 44 2024-02-02T15:36:23.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.176+0000 7f66a02c6b80 1 -- 172.21.15.1:0/3526787875 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x7f668410e500 con 0x55c894255010 2024-02-02T15:36:23.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.176+0000 7f6673400700 1 -- 172.21.15.1:0/3526787875 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c8942490f0 con 0x55c894255010 2024-02-02T15:36:23.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.176+0000 7f6673400700 1 -- 172.21.15.1:0/3526787875 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 44) v1 ==== 7510+0+0 (secure 0 0 0) 0x7f6674023b70 con 0x55c894255010 2024-02-02T15:36:23.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.176+0000 7f66a02c6b80 10 client.28145 fetch_fsmap finished waiting for FSMap version 44 2024-02-02T15:36:23.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.176+0000 7f66a02c6b80 10 client.28145 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-02T15:36:23.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.177+0000 7f66a02c6b80 20 client.28145 populate_metadata read hostname 'smithi001' 2024-02-02T15:36:23.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.177+0000 7f66a02c6b80 1 --2- 172.21.15.1:0/3526787875 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55c8942740f0 0x55c894276510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:23.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.177+0000 7f66a02c6b80 4 client.28145 mds_command: new command op to 24529 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:36:23.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.177+0000 7f66a02c6b80 1 -- 172.21.15.1:0/3526787875 --> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c89419fdd0 con 0x55c8942740f0 2024-02-02T15:36:23.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.177+0000 7f6689400700 1 --2- 172.21.15.1:0/3526787875 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55c8942740f0 0x55c894276510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:23.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.177+0000 7f6689400700 1 --2- 172.21.15.1:0/3526787875 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55c8942740f0 0x55c894276510 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7955446cb4ab5336 server_cookie=c8feeb948c407516 in_seq=0 out_seq=0 2024-02-02T15:36:23.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.177+0000 7f6673400700 10 client.28145 ms_handle_connect on v2:172.21.15.142:6838/321675421 2024-02-02T15:36:23.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.178+0000 7f6673400700 1 -- 172.21.15.1:0/3526787875 <== mds.1 v2:172.21.15.142:6838/321675421 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55c89419fdd0 con 0x55c8942740f0 2024-02-02T15:36:23.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.178+0000 7f6673400700 10 client.28145 handle_command_reply: tid=0 2024-02-02T15:36:23.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.178+0000 7f66a02c6b80 1 client.28145 shutdown 2024-02-02T15:36:23.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.179+0000 7f66a02c6b80 1 -- 172.21.15.1:0/3526787875 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c894255010 msgr2=0x55c894260230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:23.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.179+0000 7f66a02c6b80 1 --2- 172.21.15.1:0/3526787875 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c894255010 0x55c894260230 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f6674006010 tx=0x7f6674005280 comp rx=0 tx=0).stop 2024-02-02T15:36:23.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.179+0000 7f66a02c6b80 1 -- 172.21.15.1:0/3526787875 shutdown_connections 2024-02-02T15:36:23.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.179+0000 7f66a02c6b80 1 --2- 172.21.15.1:0/3526787875 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55c8942740f0 0x55c894276510 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:23.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.179+0000 7f66a02c6b80 1 --2- 172.21.15.1:0/3526787875 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c8942607e0 0x55c89425db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:23.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.179+0000 7f66a02c6b80 1 --2- 172.21.15.1:0/3526787875 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c894255010 0x55c894260230 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:23.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.179+0000 7f66a02c6b80 1 --2- 172.21.15.1:0/3526787875 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c894255930 0x55c89425d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:23.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.179+0000 7f6673400700 0 client.28145 ms_handle_reset on v2:172.21.15.142:6838/321675421 2024-02-02T15:36:23.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.179+0000 7f66a02c6b80 1 -- 172.21.15.1:0/3526787875 >> 172.21.15.1:0/3526787875 conn(0x55c8941b4db0 msgr2=0x55c894250fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:23.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.179+0000 7f66a02c6b80 1 -- 172.21.15.1:0/3526787875 shutdown_connections 2024-02-02T15:36:23.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.179+0000 7f66a02c6b80 1 -- 172.21.15.1:0/3526787875 wait complete. 2024-02-02T15:36:23.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.179+0000 7f66a02c6b80 20 client.28145 trim_cache size 0 max 16384 2024-02-02T15:36:23.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.193+0000 7f66a02c6b80 1 Processor -- start 2024-02-02T15:36:23.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.193+0000 7f66a02c6b80 1 -- start start 2024-02-02T15:36:23.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.193+0000 7f66a02c6b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c894255010 0x55c89425d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:23.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.193+0000 7f66a02c6b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c894255930 0x55c894257840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:23.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.193+0000 7f66a02c6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c8942607e0 0x55c89425fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:23.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.193+0000 7f66a02c6b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c89402a6a0 con 0x55c894255930 2024-02-02T15:36:23.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.193+0000 7f66a02c6b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c894259360 con 0x55c894255010 2024-02-02T15:36:23.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.193+0000 7f66a02c6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8942590a0 con 0x55c8942607e0 2024-02-02T15:36:23.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.194+0000 7f6689400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c894255930 0x55c894257840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:23.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.194+0000 7f6689e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c894255010 0x55c89425d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:23.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.194+0000 7f668a800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c8942607e0 0x55c89425fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:23.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.194+0000 7f6689400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c894255930 0x55c894257840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54140/0 (socket says 172.21.15.1:54140) 2024-02-02T15:36:23.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.194+0000 7f6689400700 1 -- 172.21.15.1:0/2821094126 learned_addr learned my addr 172.21.15.1:0/2821094126 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:36:23.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.194+0000 7f6689400700 1 -- 172.21.15.1:0/2821094126 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c8942607e0 msgr2=0x55c89425fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:23.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.194+0000 7f6689400700 1 --2- 172.21.15.1:0/2821094126 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c8942607e0 0x55c89425fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:23.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.194+0000 7f6689400700 1 -- 172.21.15.1:0/2821094126 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c894255010 msgr2=0x55c89425d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:23.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.194+0000 7f6689400700 1 --2- 172.21.15.1:0/2821094126 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c894255010 0x55c89425d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:23.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.194+0000 7f6689400700 1 -- 172.21.15.1:0/2821094126 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c893fd4b60 con 0x55c894255930 2024-02-02T15:36:23.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.194+0000 7f668a800700 1 --2- 172.21.15.1:0/2821094126 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c8942607e0 0x55c89425fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:36:23.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.194+0000 7f6689e00700 1 --2- 172.21.15.1:0/2821094126 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c894255010 0x55c89425d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:36:23.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.194+0000 7f6689400700 1 --2- 172.21.15.1:0/2821094126 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c894255930 0x55c894257840 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x55c89425eaf0 tx=0x7f666c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e129076114a7432 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:23.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.195+0000 7f667ba00700 1 -- 172.21.15.1:0/2821094126 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f666c022040 con 0x55c894255930 2024-02-02T15:36:23.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.195+0000 7f667ba00700 1 -- 172.21.15.1:0/2821094126 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f666c027420 con 0x55c894255930 2024-02-02T15:36:23.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.195+0000 7f667ba00700 1 -- 172.21.15.1:0/2821094126 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f666c01c400 con 0x55c894255930 2024-02-02T15:36:23.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.196+0000 7f66a02c6b80 1 -- 172.21.15.1:0/2821094126 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c894255930 msgr2=0x55c894257840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:23.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.196+0000 7f66a02c6b80 1 --2- 172.21.15.1:0/2821094126 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c894255930 0x55c894257840 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x55c89425eaf0 tx=0x7f666c014000 comp rx=0 tx=0).stop 2024-02-02T15:36:23.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.196+0000 7f66a02c6b80 1 -- 172.21.15.1:0/2821094126 shutdown_connections 2024-02-02T15:36:23.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.196+0000 7f66a02c6b80 1 --2- 172.21.15.1:0/2821094126 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c8942607e0 0x55c89425fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:23.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.196+0000 7f66a02c6b80 1 --2- 172.21.15.1:0/2821094126 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c894255010 0x55c89425d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:23.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.196+0000 7f66a02c6b80 1 --2- 172.21.15.1:0/2821094126 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c894255930 0x55c894257840 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:23.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.196+0000 7f66a02c6b80 1 -- 172.21.15.1:0/2821094126 >> 172.21.15.1:0/2821094126 conn(0x55c8941b4db0 msgr2=0x55c89425cce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:23.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.196+0000 7f66a02c6b80 1 -- 172.21.15.1:0/2821094126 shutdown_connections 2024-02-02T15:36:23.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.196+0000 7f66a02c6b80 1 -- 172.21.15.1:0/2821094126 wait complete. 2024-02-02T15:36:23.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.196+0000 7f66a02c6b80 1 -- start start 2024-02-02T15:36:23.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.197+0000 7f66a02c6b80 1 Processor -- start 2024-02-02T15:36:23.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.197+0000 7f66a02c6b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c894255010 0x55c89426b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:23.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.197+0000 7f66a02c6b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c894255930 0x55c894269730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:23.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.197+0000 7f66a02c6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c8942607e0 0x55c894269c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:23.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.197+0000 7f66a02c6b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c89426a1f0 con 0x55c894255930 2024-02-02T15:36:23.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.197+0000 7f66a02c6b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c89426a330 con 0x55c894255010 2024-02-02T15:36:23.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.197+0000 7f66a02c6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c89426a4f0 con 0x55c8942607e0 2024-02-02T15:36:23.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.197+0000 7f66a02c6b80 10 client.0 _ll_register_callbacks cb 0x55c894144290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:36:23.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.197+0000 7f6689400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c894255930 0x55c894269730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:23.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.197+0000 7f6689e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c894255010 0x55c89426b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:23.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.197+0000 7f6689400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c894255930 0x55c894269730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54154/0 (socket says 172.21.15.1:54154) 2024-02-02T15:36:23.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.197+0000 7f668a800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c8942607e0 0x55c894269c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:23.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.197+0000 7f6689400700 1 -- 172.21.15.1:0/1995142259 learned_addr learned my addr 172.21.15.1:0/1995142259 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:36:23.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.198+0000 7f6689400700 1 -- 172.21.15.1:0/1995142259 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c8942607e0 msgr2=0x55c894269c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:23.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.198+0000 7f6689400700 1 --2- 172.21.15.1:0/1995142259 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c8942607e0 0x55c894269c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:23.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.198+0000 7f6689400700 1 -- 172.21.15.1:0/1995142259 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c894255010 msgr2=0x55c89426b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:23.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.198+0000 7f6689400700 1 --2- 172.21.15.1:0/1995142259 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c894255010 0x55c89426b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:23.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.198+0000 7f6689400700 1 -- 172.21.15.1:0/1995142259 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f668410e500 con 0x55c894255930 2024-02-02T15:36:23.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.198+0000 7f668a800700 1 --2- 172.21.15.1:0/1995142259 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c8942607e0 0x55c894269c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:36:23.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.198+0000 7f6689e00700 1 --2- 172.21.15.1:0/1995142259 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c894255010 0x55c89426b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:36:23.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.198+0000 7f6689400700 1 --2- 172.21.15.1:0/1995142259 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c894255930 0x55c894269730 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f666c02b000 tx=0x7f666c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69619616e604dbfd server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:23.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.198+0000 7f6673400700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:36:23.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.198+0000 7f6673400700 1 -- 172.21.15.1:0/1995142259 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f666c022050 con 0x55c894255930 2024-02-02T15:36:23.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.199+0000 7f66a02c6b80 1 -- 172.21.15.1:0/1995142259 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f667401e040 con 0x55c894255930 2024-02-02T15:36:23.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.199+0000 7f6673400700 1 -- 172.21.15.1:0/1995142259 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f666c027780 con 0x55c894255930 2024-02-02T15:36:23.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.199+0000 7f6673400700 1 -- 172.21.15.1:0/1995142259 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f666c027940 con 0x55c894255930 2024-02-02T15:36:23.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.199+0000 7f66a02c6b80 10 client.18063 fetch_fsmap learned FSMap version 44 2024-02-02T15:36:23.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.199+0000 7f66a02c6b80 1 -- 172.21.15.1:0/1995142259 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x7f66841a3aa0 con 0x55c894255930 2024-02-02T15:36:23.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.199+0000 7f6673400700 1 -- 172.21.15.1:0/1995142259 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f667401e040 con 0x55c894255930 2024-02-02T15:36:23.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.199+0000 7f6673400700 1 -- 172.21.15.1:0/1995142259 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 44) v1 ==== 7510+0+0 (secure 0 0 0) 0x7f666c039bd0 con 0x55c894255930 2024-02-02T15:36:23.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.199+0000 7f66a02c6b80 10 client.18063 fetch_fsmap finished waiting for FSMap version 44 2024-02-02T15:36:23.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.199+0000 7f66a02c6b80 10 client.18063 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-02T15:36:23.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.199+0000 7f66a02c6b80 20 client.18063 populate_metadata read hostname 'smithi001' 2024-02-02T15:36:23.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.199+0000 7f66a02c6b80 1 --2- 172.21.15.1:0/1995142259 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55c8942c7180 0x55c8942c95a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:23.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.199+0000 7f66a02c6b80 4 client.18063 mds_command: new command op to 24529 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:36:23.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.199+0000 7f66a02c6b80 1 -- 172.21.15.1:0/1995142259 --> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c894165f50 con 0x55c8942c7180 2024-02-02T15:36:23.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.200+0000 7f6689e00700 1 --2- 172.21.15.1:0/1995142259 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55c8942c7180 0x55c8942c95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:23.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.200+0000 7f6689e00700 1 --2- 172.21.15.1:0/1995142259 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55c8942c7180 0x55c8942c95a0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=804e769abd61a750 server_cookie=b8faf3ac633e804c in_seq=0 out_seq=0 2024-02-02T15:36:23.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.200+0000 7f6673400700 10 client.18063 ms_handle_connect on v2:172.21.15.142:6838/321675421 2024-02-02T15:36:23.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.200+0000 7f6673400700 1 -- 172.21.15.1:0/1995142259 <== mds.1 v2:172.21.15.142:6838/321675421 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c894165f50 con 0x55c8942c7180 2024-02-02T15:36:23.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.200+0000 7f6673400700 10 client.18063 handle_command_reply: tid=0 2024-02-02T15:36:23.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.200+0000 7f66a02c6b80 1 client.18063 shutdown 2024-02-02T15:36:23.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.200+0000 7f66a02c6b80 1 -- 172.21.15.1:0/1995142259 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c894255930 msgr2=0x55c894269730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:23.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.200+0000 7f66a02c6b80 1 --2- 172.21.15.1:0/1995142259 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c894255930 0x55c894269730 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f666c02b000 tx=0x7f666c0057f0 comp rx=0 tx=0).stop 2024-02-02T15:36:23.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.201+0000 7f66a02c6b80 1 -- 172.21.15.1:0/1995142259 shutdown_connections 2024-02-02T15:36:23.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.201+0000 7f66a02c6b80 1 --2- 172.21.15.1:0/1995142259 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55c8942c7180 0x55c8942c95a0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:23.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.201+0000 7f66a02c6b80 1 --2- 172.21.15.1:0/1995142259 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c8942607e0 0x55c894269c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:23.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.201+0000 7f66a02c6b80 1 --2- 172.21.15.1:0/1995142259 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c894255010 0x55c89426b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:23.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.201+0000 7f6673400700 0 client.18063 ms_handle_reset on v2:172.21.15.142:6838/321675421 2024-02-02T15:36:23.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.201+0000 7f66a02c6b80 1 --2- 172.21.15.1:0/1995142259 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c894255930 0x55c894269730 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:23.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.201+0000 7f66a02c6b80 1 -- 172.21.15.1:0/1995142259 >> 172.21.15.1:0/1995142259 conn(0x55c8941b4db0 msgr2=0x55c894250e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:23.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.201+0000 7f66a02c6b80 1 -- 172.21.15.1:0/1995142259 shutdown_connections 2024-02-02T15:36:23.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.201+0000 7f66a02c6b80 1 -- 172.21.15.1:0/1995142259 wait complete. 2024-02-02T15:36:23.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.201+0000 7f66a02c6b80 20 client.18063 trim_cache size 0 max 16384 2024-02-02T15:36:23.207 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:36:23.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.201+0000 7f668c800700 1 -- 172.21.15.1:0/779745194 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f66640a0710 msgr2=0x7f66640a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:23.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.201+0000 7f668c800700 1 --2- 172.21.15.1:0/779745194 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f66640a0710 0x7f66640a2bd0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6684198460 tx=0x7f666c009260 comp rx=0 tx=0).stop 2024-02-02T15:36:23.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.202+0000 7f668c800700 1 -- 172.21.15.1:0/779745194 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f668419b7f0 msgr2=0x7f668419bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:23.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.202+0000 7f668c800700 1 --2- 172.21.15.1:0/779745194 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f668419b7f0 0x7f668419bc70 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f667c00ef30 tx=0x7f667c00c3a0 comp rx=0 tx=0).stop 2024-02-02T15:36:23.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.202+0000 7f668c800700 1 -- 172.21.15.1:0/779745194 shutdown_connections 2024-02-02T15:36:23.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.202+0000 7f668c800700 1 --2- 172.21.15.1:0/779745194 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f66640a0710 0x7f66640a2bd0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:23.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.202+0000 7f668c800700 1 --2- 172.21.15.1:0/779745194 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f668419b7f0 0x7f668419bc70 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:23.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.202+0000 7f668c800700 1 --2- 172.21.15.1:0/779745194 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6684100ff0 0x7f6684196e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:23.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.202+0000 7f668c800700 1 --2- 172.21.15.1:0/779745194 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6684106110 0x7f6684197390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:23.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.202+0000 7f668c800700 1 -- 172.21.15.1:0/779745194 >> 172.21.15.1:0/779745194 conn(0x7f66840fc810 msgr2=0x7f6684104a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:23.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.202+0000 7f668c800700 1 -- 172.21.15.1:0/779745194 shutdown_connections 2024-02-02T15:36:23.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.202+0000 7f668c800700 1 -- 172.21.15.1:0/779745194 wait complete. 2024-02-02T15:36:23.220 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-02T15:36:23.256 INFO:tasks.workunit.client.0.smithi001.stdout: 1 549863 6.23 MB/sec execute 357 sec latency 2507.011 ms 2024-02-02T15:36:23.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.449+0000 7f0633200700 1 -- 172.21.15.1:0/3067852706 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f062c100a90 msgr2=0x7f062c100f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:23.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.449+0000 7f0633200700 1 --2- 172.21.15.1:0/3067852706 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f062c100a90 0x7f062c100f10 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f0614009990 tx=0x7f0614009ca0 comp rx=0 tx=0).stop 2024-02-02T15:36:23.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.449+0000 7f0633200700 1 -- 172.21.15.1:0/3067852706 shutdown_connections 2024-02-02T15:36:23.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.449+0000 7f0633200700 1 --2- 172.21.15.1:0/3067852706 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f062c101450 0x7f062c10d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:23.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.449+0000 7f0633200700 1 --2- 172.21.15.1:0/3067852706 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f062c106aa0 0x7f062c106e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:23.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.449+0000 7f0633200700 1 --2- 172.21.15.1:0/3067852706 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f062c100a90 0x7f062c100f10 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:23.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.449+0000 7f0633200700 1 -- 172.21.15.1:0/3067852706 >> 172.21.15.1:0/3067852706 conn(0x7f062c0fc580 msgr2=0x7f062c0fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:23.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.449+0000 7f0633200700 1 -- 172.21.15.1:0/3067852706 shutdown_connections 2024-02-02T15:36:23.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.449+0000 7f0633200700 1 -- 172.21.15.1:0/3067852706 wait complete. 2024-02-02T15:36:23.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.450+0000 7f0633200700 1 Processor -- start 2024-02-02T15:36:23.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.450+0000 7f0633200700 1 -- start start 2024-02-02T15:36:23.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.450+0000 7f0633200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f062c100a90 0x7f062c19a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:23.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.450+0000 7f0633200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f062c101450 0x7f062c19acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:23.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.450+0000 7f0633200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f062c106aa0 0x7f062c194820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:23.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.451+0000 7f0633200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f062c194e90 con 0x7f062c106aa0 2024-02-02T15:36:23.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.451+0000 7f0633200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f062c195000 con 0x7f062c100a90 2024-02-02T15:36:23.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.451+0000 7f0633200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f062c1951c0 con 0x7f062c101450 2024-02-02T15:36:23.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.451+0000 7f0631e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f062c100a90 0x7f062c19a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:23.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.451+0000 7f0631e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f062c100a90 0x7f062c19a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:39882/0 (socket says 172.21.15.1:39882) 2024-02-02T15:36:23.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.451+0000 7f0631e00700 1 -- 172.21.15.1:0/112476608 learned_addr learned my addr 172.21.15.1:0/112476608 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:36:23.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.451+0000 7f0631400700 1 --2- 172.21.15.1:0/112476608 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f062c101450 0x7f062c19acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:23.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.451+0000 7f0632800700 1 --2- 172.21.15.1:0/112476608 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f062c106aa0 0x7f062c194820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:23.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.451+0000 7f0631e00700 1 -- 172.21.15.1:0/112476608 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f062c101450 msgr2=0x7f062c19acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:23.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.451+0000 7f0631e00700 1 --2- 172.21.15.1:0/112476608 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f062c101450 0x7f062c19acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:23.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.451+0000 7f0631e00700 1 -- 172.21.15.1:0/112476608 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f062c106aa0 msgr2=0x7f062c194820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:23.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.451+0000 7f0631e00700 1 --2- 172.21.15.1:0/112476608 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f062c106aa0 0x7f062c194820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:23.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.451+0000 7f0631e00700 1 -- 172.21.15.1:0/112476608 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06140095f0 con 0x7f062c100a90 2024-02-02T15:36:23.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.451+0000 7f0631e00700 1 --2- 172.21.15.1:0/112476608 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f062c100a90 0x7f062c19a7b0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f061c012fc0 tx=0x7f061c0100c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:23.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.452+0000 7f062aa00700 1 -- 172.21.15.1:0/112476608 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061c00dba0 con 0x7f062c100a90 2024-02-02T15:36:23.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.452+0000 7f062aa00700 1 -- 172.21.15.1:0/112476608 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f061c013040 con 0x7f062c100a90 2024-02-02T15:36:23.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.452+0000 7f062aa00700 1 -- 172.21.15.1:0/112476608 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061c00bd40 con 0x7f062c100a90 2024-02-02T15:36:23.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.452+0000 7f0633200700 1 -- 172.21.15.1:0/112476608 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f062c195440 con 0x7f062c100a90 2024-02-02T15:36:23.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.452+0000 7f0633200700 1 -- 172.21.15.1:0/112476608 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f062c1a17a0 con 0x7f062c100a90 2024-02-02T15:36:23.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.455+0000 7f062aa00700 1 -- 172.21.15.1:0/112476608 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f061c010740 con 0x7f062c100a90 2024-02-02T15:36:23.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.456+0000 7f064836cb80 1 Processor -- start 2024-02-02T15:36:23.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.457+0000 7f064836cb80 1 -- start start 2024-02-02T15:36:23.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.457+0000 7f062aa00700 1 --2- 172.21.15.1:0/112476608 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f060c0a0710 0x7f060c0a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:23.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.457+0000 7f062aa00700 1 -- 172.21.15.1:0/112476608 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f061c0d0070 con 0x7f062c100a90 2024-02-02T15:36:23.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.457+0000 7f064836cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c75ba61f0 0x562c75ba65d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:23.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.457+0000 7f0632800700 1 --2- 172.21.15.1:0/112476608 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f060c0a0710 0x7f060c0a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:23.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.457+0000 7f064836cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562c75ba6b10 0x562c75bb14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:23.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.457+0000 7f064836cb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c75bb19f0 0x562c75bb3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:23.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.457+0000 7f064836cb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c75b1a4c0 con 0x562c75bb19f0 2024-02-02T15:36:23.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.457+0000 7f064836cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c75afeaa0 con 0x562c75ba61f0 2024-02-02T15:36:23.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.457+0000 7f064836cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c75afc460 con 0x562c75ba6b10 2024-02-02T15:36:23.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.457+0000 7f0631e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c75ba61f0 0x562c75ba65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:23.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.458+0000 7f0631e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c75ba61f0 0x562c75ba65d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:39890/0 (socket says 172.21.15.1:39890) 2024-02-02T15:36:23.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.458+0000 7f0631e00700 1 -- 172.21.15.1:0/2798478885 learned_addr learned my addr 172.21.15.1:0/2798478885 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:36:23.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.458+0000 7f0631400700 1 --2- 172.21.15.1:0/2798478885 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562c75ba6b10 0x562c75bb14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:23.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.458+0000 7f0632800700 1 --2- 172.21.15.1:0/2798478885 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c75bb19f0 0x562c75bb3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:23.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.458+0000 7f0631e00700 1 -- 172.21.15.1:0/2798478885 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562c75ba6b10 msgr2=0x562c75bb14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:23.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.458+0000 7f0631e00700 1 --2- 172.21.15.1:0/2798478885 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562c75ba6b10 0x562c75bb14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:23.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.458+0000 7f0631e00700 1 -- 172.21.15.1:0/2798478885 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c75bb19f0 msgr2=0x562c75bb3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:23.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.458+0000 7f0631e00700 1 --2- 172.21.15.1:0/2798478885 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c75bb19f0 0x562c75bb3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:23.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.458+0000 7f0631e00700 1 -- 172.21.15.1:0/2798478885 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562c759793b0 con 0x562c75ba61f0 2024-02-02T15:36:23.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.458+0000 7f0632800700 1 --2- 172.21.15.1:0/2798478885 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c75bb19f0 0x562c75bb3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:36:23.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.458+0000 7f0631400700 1 --2- 172.21.15.1:0/2798478885 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562c75ba6b10 0x562c75bb14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:36:23.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.458+0000 7f0632800700 1 --2- 172.21.15.1:0/112476608 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f060c0a0710 0x7f060c0a2bd0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f062c19b4b0 tx=0x7f062400fc00 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:23.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.458+0000 7f0631e00700 1 --2- 172.21.15.1:0/2798478885 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c75ba61f0 0x562c75ba65d0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f061c0a4cd0 tx=0x7f061c09d080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=509acd0683aaf0a8 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:23.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.459+0000 7f0623000700 1 -- 172.21.15.1:0/2798478885 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061c09d890 con 0x562c75ba61f0 2024-02-02T15:36:23.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.459+0000 7f0623000700 1 -- 172.21.15.1:0/2798478885 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f061c09d190 con 0x562c75ba61f0 2024-02-02T15:36:23.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.459+0000 7f0623000700 1 -- 172.21.15.1:0/2798478885 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061c09d650 con 0x562c75ba61f0 2024-02-02T15:36:23.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.460+0000 7f064836cb80 1 -- 172.21.15.1:0/2798478885 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c75ba61f0 msgr2=0x562c75ba65d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:23.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.460+0000 7f064836cb80 1 --2- 172.21.15.1:0/2798478885 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c75ba61f0 0x562c75ba65d0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f061c0a4cd0 tx=0x7f061c09d080 comp rx=0 tx=0).stop 2024-02-02T15:36:23.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.460+0000 7f064836cb80 1 -- 172.21.15.1:0/2798478885 shutdown_connections 2024-02-02T15:36:23.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.460+0000 7f064836cb80 1 --2- 172.21.15.1:0/2798478885 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c75bb19f0 0x562c75bb3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:23.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.460+0000 7f064836cb80 1 --2- 172.21.15.1:0/2798478885 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c75ba61f0 0x562c75ba65d0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:23.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.460+0000 7f064836cb80 1 --2- 172.21.15.1:0/2798478885 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562c75ba6b10 0x562c75bb14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:23.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.460+0000 7f064836cb80 1 -- 172.21.15.1:0/2798478885 >> 172.21.15.1:0/2798478885 conn(0x562c75b05db0 msgr2=0x562c75ba25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:23.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.460+0000 7f064836cb80 1 -- 172.21.15.1:0/2798478885 shutdown_connections 2024-02-02T15:36:23.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.460+0000 7f064836cb80 1 -- 172.21.15.1:0/2798478885 wait complete. 2024-02-02T15:36:23.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.461+0000 7f064836cb80 1 -- start start 2024-02-02T15:36:23.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.461+0000 7f064836cb80 1 Processor -- start 2024-02-02T15:36:23.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.461+0000 7f064836cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c75ba61f0 0x562c75bb1440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:23.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.461+0000 7f064836cb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c75ba6b10 0x562c75bae6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:23.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.461+0000 7f064836cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562c75bb19f0 0x562c75baee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:23.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.461+0000 7f064836cb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c75af7d50 con 0x562c75ba6b10 2024-02-02T15:36:23.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.461+0000 7f064836cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c75a93d80 con 0x562c75ba61f0 2024-02-02T15:36:23.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.461+0000 7f064836cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c75a93a20 con 0x562c75bb19f0 2024-02-02T15:36:23.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.461+0000 7f064836cb80 10 client.0 _ll_register_callbacks cb 0x562c75a95290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:36:23.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.462+0000 7f0631e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c75ba61f0 0x562c75bb1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:23.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.462+0000 7f0631e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c75ba61f0 0x562c75bb1440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:39894/0 (socket says 172.21.15.1:39894) 2024-02-02T15:36:23.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.462+0000 7f0631e00700 1 -- 172.21.15.1:0/2673109902 learned_addr learned my addr 172.21.15.1:0/2673109902 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:36:23.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.462+0000 7f0631400700 1 --2- 172.21.15.1:0/2673109902 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c75ba6b10 0x562c75bae6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:23.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.462+0000 7f0632800700 1 --2- 172.21.15.1:0/2673109902 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562c75bb19f0 0x562c75baee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:23.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.462+0000 7f0631e00700 1 -- 172.21.15.1:0/2673109902 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562c75bb19f0 msgr2=0x562c75baee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:23.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.462+0000 7f0631e00700 1 --2- 172.21.15.1:0/2673109902 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562c75bb19f0 0x562c75baee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:23.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.462+0000 7f0631e00700 1 -- 172.21.15.1:0/2673109902 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c75ba6b10 msgr2=0x562c75bae6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:23.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.462+0000 7f0631e00700 1 --2- 172.21.15.1:0/2673109902 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c75ba6b10 0x562c75bae6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:23.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.462+0000 7f0631e00700 1 -- 172.21.15.1:0/2673109902 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f062c10e260 con 0x562c75ba61f0 2024-02-02T15:36:23.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.462+0000 7f0631e00700 1 --2- 172.21.15.1:0/2673109902 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c75ba61f0 0x562c75bb1440 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f061c099c00 tx=0x7f061c09d960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=196299844b0f897e server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:23.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.462+0000 7f061b400700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:36:23.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.462+0000 7f061b400700 1 -- 172.21.15.1:0/2673109902 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061c09d3f0 con 0x562c75ba61f0 2024-02-02T15:36:23.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.462+0000 7f061b400700 1 -- 172.21.15.1:0/2673109902 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f061c09d550 con 0x562c75ba61f0 2024-02-02T15:36:23.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.462+0000 7f064836cb80 1 -- 172.21.15.1:0/2673109902 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562c75b9a340 con 0x562c75ba61f0 2024-02-02T15:36:23.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.463+0000 7f061b400700 1 -- 172.21.15.1:0/2673109902 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f061c0a4d70 con 0x562c75ba61f0 2024-02-02T15:36:23.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.463+0000 7f064836cb80 10 client.28163 fetch_fsmap learned FSMap version 44 2024-02-02T15:36:23.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.463+0000 7f064836cb80 1 -- 172.21.15.1:0/2673109902 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x7f062c1a17a0 con 0x562c75ba61f0 2024-02-02T15:36:23.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.463+0000 7f061b400700 1 -- 172.21.15.1:0/2673109902 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562c75b9a340 con 0x562c75ba61f0 2024-02-02T15:36:23.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.463+0000 7f061b400700 1 -- 172.21.15.1:0/2673109902 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 44) v1 ==== 7510+0+0 (secure 0 0 0) 0x7f061c098f80 con 0x562c75ba61f0 2024-02-02T15:36:23.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.463+0000 7f064836cb80 10 client.28163 fetch_fsmap finished waiting for FSMap version 44 2024-02-02T15:36:23.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.463+0000 7f064836cb80 10 client.28163 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:36:23.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.463+0000 7f064836cb80 20 client.28163 populate_metadata read hostname 'smithi001' 2024-02-02T15:36:23.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.463+0000 7f064836cb80 1 --2- 172.21.15.1:0/2673109902 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x562c75bc10c0 0x562c75bc34e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:23.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.463+0000 7f064836cb80 4 client.28163 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:36:23.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.463+0000 7f064836cb80 1 -- 172.21.15.1:0/2673109902 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562c75af0dd0 con 0x562c75bc10c0 2024-02-02T15:36:23.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.463+0000 7f0631400700 1 --2- 172.21.15.1:0/2673109902 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x562c75bc10c0 0x562c75bc34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:23.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.464+0000 7f0631400700 1 --2- 172.21.15.1:0/2673109902 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x562c75bc10c0 0x562c75bc34e0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=770ebbfe7477556f server_cookie=8072099f7be57441 in_seq=0 out_seq=0 2024-02-02T15:36:23.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.464+0000 7f061b400700 10 client.28163 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:36:23.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.465+0000 7f061b400700 1 -- 172.21.15.1:0/2673109902 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x562c75af0dd0 con 0x562c75bc10c0 2024-02-02T15:36:23.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.465+0000 7f061b400700 10 client.28163 handle_command_reply: tid=0 2024-02-02T15:36:23.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.465+0000 7f064836cb80 1 client.28163 shutdown 2024-02-02T15:36:23.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.465+0000 7f064836cb80 1 -- 172.21.15.1:0/2673109902 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c75ba61f0 msgr2=0x562c75bb1440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:23.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.465+0000 7f064836cb80 1 --2- 172.21.15.1:0/2673109902 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c75ba61f0 0x562c75bb1440 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f061c099c00 tx=0x7f061c09d960 comp rx=0 tx=0).stop 2024-02-02T15:36:23.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.465+0000 7f064836cb80 1 -- 172.21.15.1:0/2673109902 shutdown_connections 2024-02-02T15:36:23.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.465+0000 7f064836cb80 1 --2- 172.21.15.1:0/2673109902 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x562c75bc10c0 0x562c75bc34e0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:23.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.465+0000 7f064836cb80 1 --2- 172.21.15.1:0/2673109902 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562c75bb19f0 0x562c75baee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:23.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.465+0000 7f064836cb80 1 --2- 172.21.15.1:0/2673109902 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c75ba61f0 0x562c75bb1440 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:23.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.465+0000 7f064836cb80 1 --2- 172.21.15.1:0/2673109902 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c75ba6b10 0x562c75bae6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:23.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.465+0000 7f061b400700 0 client.28163 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:36:23.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.465+0000 7f064836cb80 1 -- 172.21.15.1:0/2673109902 >> 172.21.15.1:0/2673109902 conn(0x562c75b05db0 msgr2=0x562c75ba21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:23.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.465+0000 7f064836cb80 1 -- 172.21.15.1:0/2673109902 shutdown_connections 2024-02-02T15:36:23.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.466+0000 7f064836cb80 1 -- 172.21.15.1:0/2673109902 wait complete. 2024-02-02T15:36:23.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.466+0000 7f064836cb80 20 client.28163 trim_cache size 0 max 16384 2024-02-02T15:36:23.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.480+0000 7f064836cb80 1 Processor -- start 2024-02-02T15:36:23.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.480+0000 7f064836cb80 1 -- start start 2024-02-02T15:36:23.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.480+0000 7f064836cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c75ba61f0 0x562c75bb8480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:23.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.480+0000 7f064836cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562c75ba6b10 0x562c75bbd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:23.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.480+0000 7f064836cb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c75bb19f0 0x562c75ba0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:23.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.480+0000 7f064836cb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c7597b6a0 con 0x562c75bb19f0 2024-02-02T15:36:23.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.480+0000 7f064836cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c75baa510 con 0x562c75ba61f0 2024-02-02T15:36:23.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.480+0000 7f064836cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c75bbdcf0 con 0x562c75ba6b10 2024-02-02T15:36:23.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.480+0000 7f0631e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c75ba61f0 0x562c75bb8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:23.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.480+0000 7f0631400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562c75ba6b10 0x562c75bbd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:23.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.480+0000 7f0631e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c75ba61f0 0x562c75bb8480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:39908/0 (socket says 172.21.15.1:39908) 2024-02-02T15:36:23.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.480+0000 7f0631e00700 1 -- 172.21.15.1:0/445367654 learned_addr learned my addr 172.21.15.1:0/445367654 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:36:23.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.480+0000 7f0632800700 1 --2- 172.21.15.1:0/445367654 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c75bb19f0 0x562c75ba0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:23.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.481+0000 7f0631400700 1 -- 172.21.15.1:0/445367654 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c75ba61f0 msgr2=0x562c75bb8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:23.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.481+0000 7f0631400700 1 --2- 172.21.15.1:0/445367654 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c75ba61f0 0x562c75bb8480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:23.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.481+0000 7f0631400700 1 -- 172.21.15.1:0/445367654 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c75bb19f0 msgr2=0x562c75ba0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:23.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.481+0000 7f0631400700 1 --2- 172.21.15.1:0/445367654 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c75bb19f0 0x562c75ba0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:23.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.481+0000 7f0631400700 1 -- 172.21.15.1:0/445367654 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562c75925b60 con 0x562c75ba6b10 2024-02-02T15:36:23.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.481+0000 7f0631e00700 1 --2- 172.21.15.1:0/445367654 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c75ba61f0 0x562c75bb8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:36:23.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.481+0000 7f0632800700 1 --2- 172.21.15.1:0/445367654 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c75bb19f0 0x562c75ba0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:36:23.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.481+0000 7f0631400700 1 --2- 172.21.15.1:0/445367654 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562c75ba6b10 0x562c75bbd7b0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f061400b2e0 tx=0x7f0614005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78f281f38e9aef31 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:23.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.481+0000 7f0623a00700 1 -- 172.21.15.1:0/445367654 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061401d040 con 0x562c75ba6b10 2024-02-02T15:36:23.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.481+0000 7f0623a00700 1 -- 172.21.15.1:0/445367654 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f061401e040 con 0x562c75ba6b10 2024-02-02T15:36:23.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.481+0000 7f0623a00700 1 -- 172.21.15.1:0/445367654 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061400f460 con 0x562c75ba6b10 2024-02-02T15:36:23.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.482+0000 7f064836cb80 1 -- 172.21.15.1:0/445367654 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562c75ba6b10 msgr2=0x562c75bbd7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:23.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.482+0000 7f064836cb80 1 --2- 172.21.15.1:0/445367654 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562c75ba6b10 0x562c75bbd7b0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f061400b2e0 tx=0x7f0614005fd0 comp rx=0 tx=0).stop 2024-02-02T15:36:23.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.482+0000 7f064836cb80 1 -- 172.21.15.1:0/445367654 shutdown_connections 2024-02-02T15:36:23.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.482+0000 7f064836cb80 1 --2- 172.21.15.1:0/445367654 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c75bb19f0 0x562c75ba0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:23.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.482+0000 7f064836cb80 1 --2- 172.21.15.1:0/445367654 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c75ba61f0 0x562c75bb8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:23.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.482+0000 7f064836cb80 1 --2- 172.21.15.1:0/445367654 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562c75ba6b10 0x562c75bbd7b0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:23.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.482+0000 7f064836cb80 1 -- 172.21.15.1:0/445367654 >> 172.21.15.1:0/445367654 conn(0x562c75b05db0 msgr2=0x562c75ba7a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:23.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.482+0000 7f064836cb80 1 -- 172.21.15.1:0/445367654 shutdown_connections 2024-02-02T15:36:23.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.482+0000 7f064836cb80 1 -- 172.21.15.1:0/445367654 wait complete. 2024-02-02T15:36:23.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.483+0000 7f064836cb80 1 -- start start 2024-02-02T15:36:23.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.483+0000 7f064836cb80 1 Processor -- start 2024-02-02T15:36:23.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.483+0000 7f064836cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562c75ba61f0 0x562c75bb60f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:23.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.483+0000 7f064836cb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c75ba6b10 0x562c75bb6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:23.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.483+0000 7f064836cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c75bb19f0 0x562c75c0bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:23.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.483+0000 7f064836cb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c75c0c0f0 con 0x562c75ba6b10 2024-02-02T15:36:23.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.483+0000 7f064836cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c75c0c260 con 0x562c75bb19f0 2024-02-02T15:36:23.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.483+0000 7f064836cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c75c0c420 con 0x562c75ba61f0 2024-02-02T15:36:23.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.483+0000 7f064836cb80 10 client.0 _ll_register_callbacks cb 0x562c75a95290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:36:23.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.483+0000 7f0631e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562c75ba61f0 0x562c75bb60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:23.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.483+0000 7f0632800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c75bb19f0 0x562c75c0bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:23.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.483+0000 7f0631e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562c75ba61f0 0x562c75bb60f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:60770/0 (socket says 172.21.15.1:60770) 2024-02-02T15:36:23.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.484+0000 7f0631e00700 1 -- 172.21.15.1:0/4118636276 learned_addr learned my addr 172.21.15.1:0/4118636276 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:36:23.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.483+0000 7f0632800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c75bb19f0 0x562c75c0bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:39924/0 (socket says 172.21.15.1:39924) 2024-02-02T15:36:23.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.484+0000 7f0631400700 1 --2- 172.21.15.1:0/4118636276 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c75ba6b10 0x562c75bb6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:23.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.484+0000 7f0631e00700 1 -- 172.21.15.1:0/4118636276 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c75bb19f0 msgr2=0x562c75c0bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:23.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.484+0000 7f0631e00700 1 --2- 172.21.15.1:0/4118636276 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c75bb19f0 0x562c75c0bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:23.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.484+0000 7f0631e00700 1 -- 172.21.15.1:0/4118636276 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c75ba6b10 msgr2=0x562c75bb6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:23.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.484+0000 7f0631e00700 1 --2- 172.21.15.1:0/4118636276 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c75ba6b10 0x562c75bb6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:23.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.484+0000 7f0631e00700 1 -- 172.21.15.1:0/4118636276 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f062c1a17a0 con 0x562c75ba61f0 2024-02-02T15:36:23.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.484+0000 7f0631e00700 1 --2- 172.21.15.1:0/4118636276 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562c75ba61f0 0x562c75bb60f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f061c010d90 tx=0x7f061c00b7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cddc58ec4335dbde server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:23.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.484+0000 7f061b400700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:36:23.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.484+0000 7f061b400700 1 -- 172.21.15.1:0/4118636276 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061c0a4f70 con 0x562c75ba61f0 2024-02-02T15:36:23.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.485+0000 7f061b400700 1 -- 172.21.15.1:0/4118636276 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f061c09d3f0 con 0x562c75ba61f0 2024-02-02T15:36:23.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.485+0000 7f064836cb80 1 -- 172.21.15.1:0/4118636276 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f061c09d190 con 0x562c75ba61f0 2024-02-02T15:36:23.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.485+0000 7f061b400700 1 -- 172.21.15.1:0/4118636276 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f061c09d5b0 con 0x562c75ba61f0 2024-02-02T15:36:23.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.485+0000 7f064836cb80 10 client.28162 fetch_fsmap learned FSMap version 44 2024-02-02T15:36:23.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.485+0000 7f064836cb80 1 -- 172.21.15.1:0/4118636276 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x7f0614005510 con 0x562c75ba61f0 2024-02-02T15:36:23.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.485+0000 7f061b400700 1 -- 172.21.15.1:0/4118636276 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f061c09d190 con 0x562c75ba61f0 2024-02-02T15:36:23.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.485+0000 7f061b400700 1 -- 172.21.15.1:0/4118636276 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 44) v1 ==== 7510+0+0 (secure 0 0 0) 0x7f061c0cdb10 con 0x562c75ba61f0 2024-02-02T15:36:23.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.485+0000 7f064836cb80 10 client.28162 fetch_fsmap finished waiting for FSMap version 44 2024-02-02T15:36:23.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.485+0000 7f064836cb80 10 client.28162 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:36:23.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.485+0000 7f064836cb80 20 client.28162 populate_metadata read hostname 'smithi001' 2024-02-02T15:36:23.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.485+0000 7f064836cb80 1 --2- 172.21.15.1:0/4118636276 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x562c75c13810 0x562c75c15c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:23.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.485+0000 7f064836cb80 4 client.28162 mds_command: new command op to 24557 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:36:23.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.485+0000 7f064836cb80 1 -- 172.21.15.1:0/4118636276 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562c75ab6f50 con 0x562c75c13810 2024-02-02T15:36:23.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.486+0000 7f0631400700 1 --2- 172.21.15.1:0/4118636276 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x562c75c13810 0x562c75c15c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:23.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.486+0000 7f061b400700 10 client.28162 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:36:23.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.486+0000 7f0631400700 1 --2- 172.21.15.1:0/4118636276 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x562c75c13810 0x562c75c15c30 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b71ee1023c141b40 server_cookie=335ab07440835727 in_seq=0 out_seq=0 2024-02-02T15:36:23.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.487+0000 7f061b400700 1 -- 172.21.15.1:0/4118636276 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562c75ab6f50 con 0x562c75c13810 2024-02-02T15:36:23.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.487+0000 7f061b400700 10 client.28162 handle_command_reply: tid=0 2024-02-02T15:36:23.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.487+0000 7f064836cb80 1 client.28162 shutdown 2024-02-02T15:36:23.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.487+0000 7f064836cb80 1 -- 172.21.15.1:0/4118636276 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562c75ba61f0 msgr2=0x562c75bb60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:23.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.487+0000 7f064836cb80 1 --2- 172.21.15.1:0/4118636276 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562c75ba61f0 0x562c75bb60f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f061c010d90 tx=0x7f061c00b7e0 comp rx=0 tx=0).stop 2024-02-02T15:36:23.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.487+0000 7f064836cb80 1 -- 172.21.15.1:0/4118636276 shutdown_connections 2024-02-02T15:36:23.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.487+0000 7f064836cb80 1 --2- 172.21.15.1:0/4118636276 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x562c75c13810 0x562c75c15c30 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:23.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.487+0000 7f064836cb80 1 --2- 172.21.15.1:0/4118636276 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562c75bb19f0 0x562c75c0bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:23.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.487+0000 7f064836cb80 1 --2- 172.21.15.1:0/4118636276 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562c75ba61f0 0x562c75bb60f0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:23.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.487+0000 7f064836cb80 1 --2- 172.21.15.1:0/4118636276 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562c75ba6b10 0x562c75bb6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:23.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.487+0000 7f061b400700 0 client.28162 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:36:23.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.487+0000 7f064836cb80 1 -- 172.21.15.1:0/4118636276 >> 172.21.15.1:0/4118636276 conn(0x562c75b05db0 msgr2=0x562c75ba7a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:23.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.487+0000 7f064836cb80 1 -- 172.21.15.1:0/4118636276 shutdown_connections 2024-02-02T15:36:23.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.487+0000 7f064836cb80 1 -- 172.21.15.1:0/4118636276 wait complete. 2024-02-02T15:36:23.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.488+0000 7f064836cb80 20 client.28162 trim_cache size 0 max 16384 2024-02-02T15:36:23.492 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:36:23.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.488+0000 7f0633200700 1 -- 172.21.15.1:0/112476608 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f060c0a0710 msgr2=0x7f060c0a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:23.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.488+0000 7f0633200700 1 --2- 172.21.15.1:0/112476608 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f060c0a0710 0x7f060c0a2bd0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f062c19b4b0 tx=0x7f062400fc00 comp rx=0 tx=0).stop 2024-02-02T15:36:23.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.488+0000 7f0633200700 1 -- 172.21.15.1:0/112476608 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f062c100a90 msgr2=0x7f062c19a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:23.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.488+0000 7f0633200700 1 --2- 172.21.15.1:0/112476608 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f062c100a90 0x7f062c19a7b0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f061c012fc0 tx=0x7f061c0100c0 comp rx=0 tx=0).stop 2024-02-02T15:36:23.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.488+0000 7f0633200700 1 -- 172.21.15.1:0/112476608 shutdown_connections 2024-02-02T15:36:23.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.488+0000 7f0633200700 1 --2- 172.21.15.1:0/112476608 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f060c0a0710 0x7f060c0a2bd0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:23.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.488+0000 7f0633200700 1 --2- 172.21.15.1:0/112476608 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f062c106aa0 0x7f062c194820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:23.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.488+0000 7f0633200700 1 --2- 172.21.15.1:0/112476608 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f062c100a90 0x7f062c19a7b0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:23.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.489+0000 7f0633200700 1 --2- 172.21.15.1:0/112476608 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f062c101450 0x7f062c19acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:23.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.489+0000 7f0633200700 1 -- 172.21.15.1:0/112476608 >> 172.21.15.1:0/112476608 conn(0x7f062c0fc580 msgr2=0x7f062c0fcaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:23.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.489+0000 7f0633200700 1 -- 172.21.15.1:0/112476608 shutdown_connections 2024-02-02T15:36:23.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:23.489+0000 7f0633200700 1 -- 172.21.15.1:0/112476608 wait complete. 2024-02-02T15:36:23.507 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:23 smithi001 ceph-mon[35470]: pgmap v290: 129 pgs: 129 active+clean; 412 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 143 op/s 2024-02-02T15:36:23.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:23 smithi071 ceph-mon[44707]: pgmap v290: 129 pgs: 129 active+clean; 412 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 143 op/s 2024-02-02T15:36:23.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:23 smithi142 ceph-mon[45728]: pgmap v290: 129 pgs: 129 active+clean; 412 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 143 op/s 2024-02-02T15:36:24.256 INFO:tasks.workunit.client.0.smithi001.stdout: 1 551070 6.23 MB/sec execute 358 sec latency 16.729 ms 2024-02-02T15:36:24.510 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-02T15:36:24.510 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-02T15:36:24.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.767+0000 7f6927e00700 1 -- 172.21.15.1:0/1949998046 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f692010d660 msgr2=0x7f692010fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:24.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.767+0000 7f6927e00700 1 --2- 172.21.15.1:0/1949998046 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f692010d660 0x7f692010fa50 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f691800b0a0 tx=0x7f691800b3b0 comp rx=0 tx=0).stop 2024-02-02T15:36:24.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.768+0000 7f6927e00700 1 -- 172.21.15.1:0/1949998046 shutdown_connections 2024-02-02T15:36:24.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.768+0000 7f6927e00700 1 --2- 172.21.15.1:0/1949998046 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f692010d660 0x7f692010fa50 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:24.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.768+0000 7f6927e00700 1 --2- 172.21.15.1:0/1949998046 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69200730f0 0x7f69200734d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:24.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.768+0000 7f6927e00700 1 --2- 172.21.15.1:0/1949998046 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6920073a10 0x7f692010d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:24.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.768+0000 7f6927e00700 1 -- 172.21.15.1:0/1949998046 >> 172.21.15.1:0/1949998046 conn(0x7f69200fc8b0 msgr2=0x7f69200fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:24.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.768+0000 7f6927e00700 1 -- 172.21.15.1:0/1949998046 shutdown_connections 2024-02-02T15:36:24.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.768+0000 7f6927e00700 1 -- 172.21.15.1:0/1949998046 wait complete. 2024-02-02T15:36:24.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.769+0000 7f6927e00700 1 Processor -- start 2024-02-02T15:36:24.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.769+0000 7f6927e00700 1 -- start start 2024-02-02T15:36:24.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.770+0000 7f6927e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69200730f0 0x7f6920199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:24.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.770+0000 7f6927e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6920073a10 0x7f6920199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:24.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.770+0000 7f6927e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f692010d660 0x7f692019dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:24.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.770+0000 7f6927e00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f692019e1f0 con 0x7f69200730f0 2024-02-02T15:36:24.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.770+0000 7f6927e00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f692019e360 con 0x7f692010d660 2024-02-02T15:36:24.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.770+0000 7f6927e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f692019e520 con 0x7f6920073a10 2024-02-02T15:36:24.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.770+0000 7f6925400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69200730f0 0x7f6920199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:24.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.770+0000 7f6924a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6920073a10 0x7f6920199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:24.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.770+0000 7f6925e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f692010d660 0x7f692019dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:24.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.770+0000 7f6925e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f692010d660 0x7f692019dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:39946/0 (socket says 172.21.15.1:39946) 2024-02-02T15:36:24.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.770+0000 7f6925e00700 1 -- 172.21.15.1:0/3715027859 learned_addr learned my addr 172.21.15.1:0/3715027859 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:36:24.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.770+0000 7f6925400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69200730f0 0x7f6920199210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54206/0 (socket says 172.21.15.1:54206) 2024-02-02T15:36:24.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.771+0000 7f6925e00700 1 -- 172.21.15.1:0/3715027859 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6920073a10 msgr2=0x7f6920199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:24.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.771+0000 7f6925e00700 1 --2- 172.21.15.1:0/3715027859 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6920073a10 0x7f6920199750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:24.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.771+0000 7f6925e00700 1 -- 172.21.15.1:0/3715027859 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69200730f0 msgr2=0x7f6920199210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:24.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.771+0000 7f6925e00700 1 --2- 172.21.15.1:0/3715027859 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69200730f0 0x7f6920199210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:24.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.771+0000 7f6925e00700 1 -- 172.21.15.1:0/3715027859 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6910009520 con 0x7f692010d660 2024-02-02T15:36:24.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.771+0000 7f6925e00700 1 --2- 172.21.15.1:0/3715027859 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f692010d660 0x7f692019dbb0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f6918009fd0 tx=0x7f691800bce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:24.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.771+0000 7f691e000700 1 -- 172.21.15.1:0/3715027859 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f691800e040 con 0x7f692010d660 2024-02-02T15:36:24.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.771+0000 7f691e000700 1 -- 172.21.15.1:0/3715027859 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6918024070 con 0x7f692010d660 2024-02-02T15:36:24.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.771+0000 7f6927e00700 1 -- 172.21.15.1:0/3715027859 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6908009520 con 0x7f692010d660 2024-02-02T15:36:24.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.771+0000 7f691e000700 1 -- 172.21.15.1:0/3715027859 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f691802bb60 con 0x7f692010d660 2024-02-02T15:36:24.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.772+0000 7f6927e00700 1 -- 172.21.15.1:0/3715027859 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6918009d00 con 0x7f692010d660 2024-02-02T15:36:24.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.775+0000 7f691e000700 1 -- 172.21.15.1:0/3715027859 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f6918005040 con 0x7f692010d660 2024-02-02T15:36:24.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.776+0000 7f691e000700 1 --2- 172.21.15.1:0/3715027859 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f69000a0710 0x7f69000a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:24.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.776+0000 7f691e000700 1 -- 172.21.15.1:0/3715027859 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f6918026070 con 0x7f692010d660 2024-02-02T15:36:24.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.777+0000 7f693b997b80 1 Processor -- start 2024-02-02T15:36:24.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.777+0000 7f6925400700 1 --2- 172.21.15.1:0/3715027859 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f69000a0710 0x7f69000a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:24.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.777+0000 7f693b997b80 1 -- start start 2024-02-02T15:36:24.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.777+0000 7f693b997b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f1d14fb090 0x55f1d14fb470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:24.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.777+0000 7f693b997b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1d14fb9b0 0x55f1d1506320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:24.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.777+0000 7f693b997b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f1d1506860 0x55f1d1508c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:24.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.777+0000 7f693b997b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1d146f540 con 0x55f1d14fb9b0 2024-02-02T15:36:24.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.777+0000 7f693b997b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1d1453b20 con 0x55f1d14fb090 2024-02-02T15:36:24.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.777+0000 7f693b997b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1d14514e0 con 0x55f1d1506860 2024-02-02T15:36:24.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.777+0000 7f6925400700 1 --2- 172.21.15.1:0/3715027859 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f69000a0710 0x7f69000a2bd0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6910009df0 tx=0x7f6910009260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:24.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.777+0000 7f6924a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1d14fb9b0 0x55f1d1506320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:24.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.778+0000 7f6925e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f1d1506860 0x55f1d1508c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:24.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.778+0000 7f6925400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f1d14fb090 0x55f1d14fb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:24.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.778+0000 7f6925e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f1d1506860 0x55f1d1508c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:60802/0 (socket says 172.21.15.1:60802) 2024-02-02T15:36:24.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.778+0000 7f6925e00700 1 -- 172.21.15.1:0/28545593 learned_addr learned my addr 172.21.15.1:0/28545593 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:36:24.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.778+0000 7f6925e00700 1 -- 172.21.15.1:0/28545593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f1d14fb090 msgr2=0x55f1d14fb470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:24.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.778+0000 7f6925e00700 1 --2- 172.21.15.1:0/28545593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f1d14fb090 0x55f1d14fb470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:24.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.778+0000 7f6925e00700 1 -- 172.21.15.1:0/28545593 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1d14fb9b0 msgr2=0x55f1d1506320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:24.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.778+0000 7f6925e00700 1 --2- 172.21.15.1:0/28545593 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1d14fb9b0 0x55f1d1506320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:24.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.778+0000 7f6925e00700 1 -- 172.21.15.1:0/28545593 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f1d12ce430 con 0x55f1d1506860 2024-02-02T15:36:24.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.778+0000 7f6925e00700 1 --2- 172.21.15.1:0/28545593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f1d1506860 0x55f1d1508c50 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f691809ef00 tx=0x7f691800bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b18a63f8a32bd2a2 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:24.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.778+0000 7f6916600700 1 -- 172.21.15.1:0/28545593 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69180a2360 con 0x55f1d1506860 2024-02-02T15:36:24.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.778+0000 7f6916600700 1 -- 172.21.15.1:0/28545593 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f69180a3100 con 0x55f1d1506860 2024-02-02T15:36:24.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.778+0000 7f6916600700 1 -- 172.21.15.1:0/28545593 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69180d5320 con 0x55f1d1506860 2024-02-02T15:36:24.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.779+0000 7f693b997b80 1 -- 172.21.15.1:0/28545593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f1d1506860 msgr2=0x55f1d1508c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:24.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.779+0000 7f693b997b80 1 --2- 172.21.15.1:0/28545593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f1d1506860 0x55f1d1508c50 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f691809ef00 tx=0x7f691800bf10 comp rx=0 tx=0).stop 2024-02-02T15:36:24.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.779+0000 7f693b997b80 1 -- 172.21.15.1:0/28545593 shutdown_connections 2024-02-02T15:36:24.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.779+0000 7f693b997b80 1 --2- 172.21.15.1:0/28545593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f1d1506860 0x55f1d1508c50 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:24.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.779+0000 7f693b997b80 1 --2- 172.21.15.1:0/28545593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f1d14fb090 0x55f1d14fb470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:24.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.779+0000 7f693b997b80 1 --2- 172.21.15.1:0/28545593 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1d14fb9b0 0x55f1d1506320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:24.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.779+0000 7f693b997b80 1 -- 172.21.15.1:0/28545593 >> 172.21.15.1:0/28545593 conn(0x55f1d145ae30 msgr2=0x55f1d14f7430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:24.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.779+0000 7f693b997b80 1 -- 172.21.15.1:0/28545593 shutdown_connections 2024-02-02T15:36:24.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.779+0000 7f693b997b80 1 -- 172.21.15.1:0/28545593 wait complete. 2024-02-02T15:36:24.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.780+0000 7f693b997b80 1 -- start start 2024-02-02T15:36:24.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.780+0000 7f693b997b80 1 Processor -- start 2024-02-02T15:36:24.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.781+0000 7f693b997b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f1d14fb090 0x55f1d1506220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:24.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.781+0000 7f693b997b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f1d14fb9b0 0x55f1d15034a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:24.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.781+0000 7f693b997b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1d1506860 0x55f1d1503ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:24.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.781+0000 7f693b997b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1d144cdd0 con 0x55f1d1506860 2024-02-02T15:36:24.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.781+0000 7f693b997b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1d13e8e00 con 0x55f1d14fb090 2024-02-02T15:36:24.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.781+0000 7f693b997b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1d13e8aa0 con 0x55f1d14fb9b0 2024-02-02T15:36:24.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.781+0000 7f693b997b80 10 client.0 _ll_register_callbacks cb 0x55f1d13ea310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:36:24.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.781+0000 7f6925400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f1d14fb090 0x55f1d1506220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:24.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.781+0000 7f6924a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f1d14fb9b0 0x55f1d15034a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:24.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.781+0000 7f6925400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f1d14fb090 0x55f1d1506220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:39954/0 (socket says 172.21.15.1:39954) 2024-02-02T15:36:24.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.781+0000 7f6925400700 1 -- 172.21.15.1:0/468122825 learned_addr learned my addr 172.21.15.1:0/468122825 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:36:24.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.781+0000 7f6925e00700 1 --2- 172.21.15.1:0/468122825 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1d1506860 0x55f1d1503ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:24.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.781+0000 7f6924a00700 1 -- 172.21.15.1:0/468122825 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f1d14fb090 msgr2=0x55f1d1506220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:24.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.781+0000 7f6924a00700 1 --2- 172.21.15.1:0/468122825 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f1d14fb090 0x55f1d1506220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:24.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.781+0000 7f6924a00700 1 -- 172.21.15.1:0/468122825 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1d1506860 msgr2=0x55f1d1503ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:24.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.781+0000 7f6924a00700 1 --2- 172.21.15.1:0/468122825 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1d1506860 0x55f1d1503ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:24.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.781+0000 7f6924a00700 1 -- 172.21.15.1:0/468122825 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6918009d00 con 0x55f1d14fb9b0 2024-02-02T15:36:24.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.782+0000 7f6925e00700 1 --2- 172.21.15.1:0/468122825 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1d1506860 0x55f1d1503ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:36:24.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.782+0000 7f6924a00700 1 --2- 172.21.15.1:0/468122825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f1d14fb9b0 0x55f1d15034a0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f6908012810 tx=0x7f6908012bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1dd91f853be4cd5c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:24.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.782+0000 7f690ea00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:36:24.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.782+0000 7f690ea00700 1 -- 172.21.15.1:0/468122825 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6908017070 con 0x55f1d14fb9b0 2024-02-02T15:36:24.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.782+0000 7f693b997b80 1 -- 172.21.15.1:0/468122825 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f1d14ef170 con 0x55f1d14fb9b0 2024-02-02T15:36:24.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.782+0000 7f690ea00700 1 -- 172.21.15.1:0/468122825 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f6908004830 con 0x55f1d14fb9b0 2024-02-02T15:36:24.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.782+0000 7f690ea00700 1 -- 172.21.15.1:0/468122825 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f69080049f0 con 0x55f1d14fb9b0 2024-02-02T15:36:24.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.782+0000 7f693b997b80 10 client.28174 fetch_fsmap learned FSMap version 44 2024-02-02T15:36:24.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.782+0000 7f693b997b80 1 -- 172.21.15.1:0/468122825 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x55f1d127abe0 con 0x55f1d14fb9b0 2024-02-02T15:36:24.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.783+0000 7f690ea00700 1 -- 172.21.15.1:0/468122825 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f1d14ef170 con 0x55f1d14fb9b0 2024-02-02T15:36:24.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.783+0000 7f690ea00700 1 -- 172.21.15.1:0/468122825 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 44) v1 ==== 7510+0+0 (secure 0 0 0) 0x7f690801c840 con 0x55f1d14fb9b0 2024-02-02T15:36:24.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.783+0000 7f693b997b80 10 client.28174 fetch_fsmap finished waiting for FSMap version 44 2024-02-02T15:36:24.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.783+0000 7f693b997b80 10 client.28174 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:36:24.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.783+0000 7f693b997b80 20 client.28174 populate_metadata read hostname 'smithi001' 2024-02-02T15:36:24.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.783+0000 7f693b997b80 1 --2- 172.21.15.1:0/468122825 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55f1d1516130 0x55f1d1518550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:24.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.783+0000 7f693b997b80 4 client.28174 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:36:24.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.783+0000 7f693b997b80 1 -- 172.21.15.1:0/468122825 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f1d1445e50 con 0x55f1d1516130 2024-02-02T15:36:24.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.783+0000 7f6925400700 1 --2- 172.21.15.1:0/468122825 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55f1d1516130 0x55f1d1518550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:24.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.784+0000 7f6925400700 1 --2- 172.21.15.1:0/468122825 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55f1d1516130 0x55f1d1518550 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c561bba11146fa7e server_cookie=113abaf288a474ec in_seq=0 out_seq=0 2024-02-02T15:36:24.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.784+0000 7f690ea00700 10 client.28174 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:36:24.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.784+0000 7f690ea00700 1 -- 172.21.15.1:0/468122825 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55f1d1445e50 con 0x55f1d1516130 2024-02-02T15:36:24.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.784+0000 7f690ea00700 10 client.28174 handle_command_reply: tid=0 2024-02-02T15:36:24.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.785+0000 7f693b997b80 1 client.28174 shutdown 2024-02-02T15:36:24.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.785+0000 7f693b997b80 1 -- 172.21.15.1:0/468122825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f1d14fb9b0 msgr2=0x55f1d15034a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:24.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.785+0000 7f693b997b80 1 --2- 172.21.15.1:0/468122825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f1d14fb9b0 0x55f1d15034a0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f6908012810 tx=0x7f6908012bd0 comp rx=0 tx=0).stop 2024-02-02T15:36:24.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.785+0000 7f693b997b80 1 -- 172.21.15.1:0/468122825 shutdown_connections 2024-02-02T15:36:24.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.785+0000 7f693b997b80 1 --2- 172.21.15.1:0/468122825 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55f1d1516130 0x55f1d1518550 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:24.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.785+0000 7f690ea00700 0 client.28174 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:36:24.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.785+0000 7f693b997b80 1 --2- 172.21.15.1:0/468122825 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1d1506860 0x55f1d1503ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:24.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.785+0000 7f693b997b80 1 --2- 172.21.15.1:0/468122825 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f1d14fb090 0x55f1d1506220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:24.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.785+0000 7f693b997b80 1 --2- 172.21.15.1:0/468122825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f1d14fb9b0 0x55f1d15034a0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:24.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.785+0000 7f693b997b80 1 -- 172.21.15.1:0/468122825 >> 172.21.15.1:0/468122825 conn(0x55f1d145ae30 msgr2=0x55f1d14f7020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:24.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.785+0000 7f693b997b80 1 -- 172.21.15.1:0/468122825 shutdown_connections 2024-02-02T15:36:24.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.785+0000 7f693b997b80 1 -- 172.21.15.1:0/468122825 wait complete. 2024-02-02T15:36:24.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.785+0000 7f693b997b80 20 client.28174 trim_cache size 0 max 16384 2024-02-02T15:36:24.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.804+0000 7f693b997b80 1 Processor -- start 2024-02-02T15:36:24.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.805+0000 7f693b997b80 1 -- start start 2024-02-02T15:36:24.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.805+0000 7f693b997b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1d14fb090 0x55f1d1502d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:24.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.805+0000 7f693b997b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f1d14fb9b0 0x55f1d1505d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:24.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.805+0000 7f693b997b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f1d1506860 0x55f1d15007b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:24.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.805+0000 7f693b997b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1d12d0720 con 0x55f1d14fb090 2024-02-02T15:36:24.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.805+0000 7f693b997b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1d14f3a60 con 0x55f1d1506860 2024-02-02T15:36:24.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.805+0000 7f693b997b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1d15120d0 con 0x55f1d14fb9b0 2024-02-02T15:36:24.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.805+0000 7f6924a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f1d14fb9b0 0x55f1d1505d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:24.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.805+0000 7f6924a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f1d14fb9b0 0x55f1d1505d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:60820/0 (socket says 172.21.15.1:60820) 2024-02-02T15:36:24.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.805+0000 7f6924a00700 1 -- 172.21.15.1:0/1106941242 learned_addr learned my addr 172.21.15.1:0/1106941242 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:36:24.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.805+0000 7f6925400700 1 --2- 172.21.15.1:0/1106941242 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1d14fb090 0x55f1d1502d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:24.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.806+0000 7f6924a00700 1 -- 172.21.15.1:0/1106941242 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f1d1506860 msgr2=0x55f1d15007b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-02T15:36:24.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.806+0000 7f6924a00700 1 --2- 172.21.15.1:0/1106941242 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f1d1506860 0x55f1d15007b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:24.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.806+0000 7f6924a00700 1 -- 172.21.15.1:0/1106941242 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1d14fb090 msgr2=0x55f1d1502d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:24.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.806+0000 7f6924a00700 1 --2- 172.21.15.1:0/1106941242 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1d14fb090 0x55f1d1502d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:24.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.806+0000 7f6924a00700 1 -- 172.21.15.1:0/1106941242 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f1d14f3150 con 0x55f1d14fb9b0 2024-02-02T15:36:24.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.806+0000 7f6924a00700 1 --2- 172.21.15.1:0/1106941242 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f1d14fb9b0 0x55f1d1505d50 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f6908005a80 tx=0x7f690800f3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6225d4c43ad17993 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:24.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.806+0000 7f6917000700 1 -- 172.21.15.1:0/1106941242 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6908017070 con 0x55f1d14fb9b0 2024-02-02T15:36:24.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.806+0000 7f6917000700 1 -- 172.21.15.1:0/1106941242 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6908018040 con 0x55f1d14fb9b0 2024-02-02T15:36:24.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.806+0000 7f6917000700 1 -- 172.21.15.1:0/1106941242 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6908010db0 con 0x55f1d14fb9b0 2024-02-02T15:36:24.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.807+0000 7f693b997b80 1 -- 172.21.15.1:0/1106941242 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f1d14fb9b0 msgr2=0x55f1d1505d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:24.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.807+0000 7f693b997b80 1 --2- 172.21.15.1:0/1106941242 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f1d14fb9b0 0x55f1d1505d50 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f6908005a80 tx=0x7f690800f3c0 comp rx=0 tx=0).stop 2024-02-02T15:36:24.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.807+0000 7f693b997b80 1 -- 172.21.15.1:0/1106941242 shutdown_connections 2024-02-02T15:36:24.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.807+0000 7f693b997b80 1 --2- 172.21.15.1:0/1106941242 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f1d1506860 0x55f1d15007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:24.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.807+0000 7f693b997b80 1 --2- 172.21.15.1:0/1106941242 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1d14fb090 0x55f1d1502d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:24.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.807+0000 7f693b997b80 1 --2- 172.21.15.1:0/1106941242 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f1d14fb9b0 0x55f1d1505d50 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:24.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.807+0000 7f693b997b80 1 -- 172.21.15.1:0/1106941242 >> 172.21.15.1:0/1106941242 conn(0x55f1d145ae30 msgr2=0x55f1d14fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:24.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.807+0000 7f693b997b80 1 -- 172.21.15.1:0/1106941242 shutdown_connections 2024-02-02T15:36:24.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.807+0000 7f693b997b80 1 -- 172.21.15.1:0/1106941242 wait complete. 2024-02-02T15:36:24.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.808+0000 7f693b997b80 1 -- start start 2024-02-02T15:36:24.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.808+0000 7f693b997b80 1 Processor -- start 2024-02-02T15:36:24.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.808+0000 7f693b997b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f1d14fb090 0x55f1d150ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:24.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.808+0000 7f693b997b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1d14fb9b0 0x55f1d150bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:24.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.808+0000 7f693b997b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f1d1506860 0x55f1d1567c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:24.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.808+0000 7f693b997b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1d150c7d0 con 0x55f1d14fb9b0 2024-02-02T15:36:24.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.808+0000 7f693b997b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1d150c940 con 0x55f1d14fb090 2024-02-02T15:36:24.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.808+0000 7f693b997b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1d1568220 con 0x55f1d1506860 2024-02-02T15:36:24.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.808+0000 7f693b997b80 10 client.0 _ll_register_callbacks cb 0x55f1d13ea310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:36:24.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.809+0000 7f6925e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f1d1506860 0x55f1d1567c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:24.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.809+0000 7f6925e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f1d1506860 0x55f1d1567c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:60828/0 (socket says 172.21.15.1:60828) 2024-02-02T15:36:24.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.809+0000 7f6925e00700 1 -- 172.21.15.1:0/646756798 learned_addr learned my addr 172.21.15.1:0/646756798 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:36:24.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.809+0000 7f6925400700 1 --2- 172.21.15.1:0/646756798 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f1d14fb090 0x55f1d150ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:24.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.809+0000 7f6924a00700 1 --2- 172.21.15.1:0/646756798 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1d14fb9b0 0x55f1d150bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:24.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.809+0000 7f6925e00700 1 -- 172.21.15.1:0/646756798 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f1d14fb090 msgr2=0x55f1d150ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:24.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.809+0000 7f6925e00700 1 --2- 172.21.15.1:0/646756798 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f1d14fb090 0x55f1d150ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:24.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.809+0000 7f6925e00700 1 -- 172.21.15.1:0/646756798 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1d14fb9b0 msgr2=0x55f1d150bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:24.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.809+0000 7f6925e00700 1 --2- 172.21.15.1:0/646756798 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1d14fb9b0 0x55f1d150bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:24.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.809+0000 7f6925e00700 1 -- 172.21.15.1:0/646756798 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f1d127abe0 con 0x55f1d1506860 2024-02-02T15:36:24.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.809+0000 7f6925400700 1 --2- 172.21.15.1:0/646756798 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f1d14fb090 0x55f1d150ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:36:24.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.809+0000 7f6925e00700 1 --2- 172.21.15.1:0/646756798 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f1d1506860 0x55f1d1567c60 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f691800b070 tx=0x7f69180099e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea0dea0e46ce9443 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:24.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.809+0000 7f690ea00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:36:24.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.810+0000 7f690ea00700 1 -- 172.21.15.1:0/646756798 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69180bf070 con 0x55f1d1506860 2024-02-02T15:36:24.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.810+0000 7f690ea00700 1 -- 172.21.15.1:0/646756798 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f69180aa9e0 con 0x55f1d1506860 2024-02-02T15:36:24.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.810+0000 7f693b997b80 1 -- 172.21.15.1:0/646756798 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f69180a3100 con 0x55f1d1506860 2024-02-02T15:36:24.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.810+0000 7f690ea00700 1 -- 172.21.15.1:0/646756798 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f69180ae020 con 0x55f1d1506860 2024-02-02T15:36:24.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.810+0000 7f693b997b80 10 client.28180 fetch_fsmap learned FSMap version 44 2024-02-02T15:36:24.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.810+0000 7f693b997b80 1 -- 172.21.15.1:0/646756798 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x55f1d1412f20 con 0x55f1d1506860 2024-02-02T15:36:24.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.810+0000 7f690ea00700 1 -- 172.21.15.1:0/646756798 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f69180a3100 con 0x55f1d1506860 2024-02-02T15:36:24.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.810+0000 7f690ea00700 1 -- 172.21.15.1:0/646756798 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 44) v1 ==== 7510+0+0 (secure 0 0 0) 0x7f69180a2c60 con 0x55f1d1506860 2024-02-02T15:36:24.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.810+0000 7f693b997b80 10 client.28180 fetch_fsmap finished waiting for FSMap version 44 2024-02-02T15:36:24.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.810+0000 7f693b997b80 10 client.28180 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:36:24.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.810+0000 7f693b997b80 20 client.28180 populate_metadata read hostname 'smithi001' 2024-02-02T15:36:24.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.810+0000 7f693b997b80 1 --2- 172.21.15.1:0/646756798 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55f1d15695e0 0x55f1d156ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:24.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.810+0000 7f693b997b80 4 client.28180 mds_command: new command op to 24557 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-02T15:36:24.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.810+0000 7f693b997b80 1 -- 172.21.15.1:0/646756798 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55f1d140bfd0 con 0x55f1d15695e0 2024-02-02T15:36:24.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.810+0000 7f6925400700 1 --2- 172.21.15.1:0/646756798 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55f1d15695e0 0x55f1d156ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:24.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.811+0000 7f6925400700 1 --2- 172.21.15.1:0/646756798 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55f1d15695e0 0x55f1d156ba00 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2ae40ebf19a9ec1 server_cookie=211d816babc210d7 in_seq=0 out_seq=0 2024-02-02T15:36:24.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.811+0000 7f690ea00700 10 client.28180 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:36:24.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.812+0000 7f690ea00700 1 -- 172.21.15.1:0/646756798 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55f1d140bfd0 con 0x55f1d15695e0 2024-02-02T15:36:24.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.812+0000 7f690ea00700 10 client.28180 handle_command_reply: tid=0 2024-02-02T15:36:24.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.812+0000 7f693b997b80 1 client.28180 shutdown 2024-02-02T15:36:24.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.812+0000 7f693b997b80 1 -- 172.21.15.1:0/646756798 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f1d1506860 msgr2=0x55f1d1567c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:24.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.812+0000 7f693b997b80 1 --2- 172.21.15.1:0/646756798 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f1d1506860 0x55f1d1567c60 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f691800b070 tx=0x7f69180099e0 comp rx=0 tx=0).stop 2024-02-02T15:36:24.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.812+0000 7f693b997b80 1 -- 172.21.15.1:0/646756798 shutdown_connections 2024-02-02T15:36:24.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.812+0000 7f693b997b80 1 --2- 172.21.15.1:0/646756798 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55f1d15695e0 0x55f1d156ba00 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:24.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.812+0000 7f693b997b80 1 --2- 172.21.15.1:0/646756798 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f1d1506860 0x55f1d1567c60 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:24.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.812+0000 7f693b997b80 1 --2- 172.21.15.1:0/646756798 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f1d14fb090 0x55f1d150ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:24.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.812+0000 7f693b997b80 1 --2- 172.21.15.1:0/646756798 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f1d14fb9b0 0x55f1d150bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:24.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.812+0000 7f693b997b80 1 -- 172.21.15.1:0/646756798 >> 172.21.15.1:0/646756798 conn(0x55f1d145ae30 msgr2=0x55f1d14f6640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:24.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.812+0000 7f690ea00700 0 client.28180 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:36:24.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.813+0000 7f693b997b80 1 -- 172.21.15.1:0/646756798 shutdown_connections 2024-02-02T15:36:24.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.813+0000 7f693b997b80 1 -- 172.21.15.1:0/646756798 wait complete. 2024-02-02T15:36:24.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.813+0000 7f693b997b80 20 client.28180 trim_cache size 0 max 16384 2024-02-02T15:36:24.816 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:36:24.817 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2024-02-02T15:36:24.817 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "5eb42348-95f0-467f-9871-02e9588e3a8a", 2024-02-02T15:36:24.817 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2024-02-02T15:36:24.817 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:36:24.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.813+0000 7f6927e00700 1 -- 172.21.15.1:0/3715027859 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f69000a0710 msgr2=0x7f69000a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:24.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.813+0000 7f6927e00700 1 --2- 172.21.15.1:0/3715027859 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f69000a0710 0x7f69000a2bd0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6910009df0 tx=0x7f6910009260 comp rx=0 tx=0).stop 2024-02-02T15:36:24.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.813+0000 7f6927e00700 1 -- 172.21.15.1:0/3715027859 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f692010d660 msgr2=0x7f692019dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:24.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.813+0000 7f6927e00700 1 --2- 172.21.15.1:0/3715027859 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f692010d660 0x7f692019dbb0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f6918009fd0 tx=0x7f691800bce0 comp rx=0 tx=0).stop 2024-02-02T15:36:24.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.814+0000 7f6927e00700 1 -- 172.21.15.1:0/3715027859 shutdown_connections 2024-02-02T15:36:24.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.814+0000 7f6927e00700 1 --2- 172.21.15.1:0/3715027859 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f69000a0710 0x7f69000a2bd0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:24.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.814+0000 7f6927e00700 1 --2- 172.21.15.1:0/3715027859 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f692010d660 0x7f692019dbb0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:24.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.814+0000 7f6927e00700 1 --2- 172.21.15.1:0/3715027859 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69200730f0 0x7f6920199210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:24.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.814+0000 7f6927e00700 1 --2- 172.21.15.1:0/3715027859 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6920073a10 0x7f6920199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:24.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.814+0000 7f6927e00700 1 -- 172.21.15.1:0/3715027859 >> 172.21.15.1:0/3715027859 conn(0x7f69200fc8b0 msgr2=0x7f69201050c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:24.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.814+0000 7f6927e00700 1 -- 172.21.15.1:0/3715027859 shutdown_connections 2024-02-02T15:36:24.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:24.814+0000 7f6927e00700 1 -- 172.21.15.1:0/3715027859 wait complete. 2024-02-02T15:36:24.837 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-02T15:36:25.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.054+0000 7f69f4800700 1 -- 172.21.15.1:0/1190820783 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69ec102030 msgr2=0x7f69ec102410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:25.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.054+0000 7f69f4800700 1 --2- 172.21.15.1:0/1190820783 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69ec102030 0x7f69ec102410 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f69dc0098f0 tx=0x7f69dc009c00 comp rx=0 tx=0).stop 2024-02-02T15:36:25.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.054+0000 7f69f4800700 1 -- 172.21.15.1:0/1190820783 shutdown_connections 2024-02-02T15:36:25.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.054+0000 7f69f4800700 1 --2- 172.21.15.1:0/1190820783 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f69ec10b400 0x7f69ec10d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:25.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.054+0000 7f69f4800700 1 --2- 172.21.15.1:0/1190820783 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69ec102030 0x7f69ec102410 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:25.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.054+0000 7f69f4800700 1 --2- 172.21.15.1:0/1190820783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69ec102950 0x7f69ec10ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:25.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.054+0000 7f69f4800700 1 -- 172.21.15.1:0/1190820783 >> 172.21.15.1:0/1190820783 conn(0x7f69ec076d70 msgr2=0x7f69ec077180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:25.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.054+0000 7f69f4800700 1 -- 172.21.15.1:0/1190820783 shutdown_connections 2024-02-02T15:36:25.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.054+0000 7f69f4800700 1 -- 172.21.15.1:0/1190820783 wait complete. 2024-02-02T15:36:25.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.055+0000 7f69f4800700 1 Processor -- start 2024-02-02T15:36:25.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.055+0000 7f69f4800700 1 -- start start 2024-02-02T15:36:25.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.056+0000 7f69f4800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69ec102030 0x7f69ec0ffc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:25.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.056+0000 7f69f4800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f69ec102950 0x7f69ec1001c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:25.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.056+0000 7f69f4800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69ec10b400 0x7f69ec1a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:25.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.056+0000 7f69f4800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69ec1009f0 con 0x7f69ec102030 2024-02-02T15:36:25.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.056+0000 7f69f4800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69ec1a5610 con 0x7f69ec102950 2024-02-02T15:36:25.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.056+0000 7f69f4800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69ec1a5750 con 0x7f69ec10b400 2024-02-02T15:36:25.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.056+0000 7f69f1400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f69ec102950 0x7f69ec1001c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:25.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.056+0000 7f69f1e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69ec102030 0x7f69ec0ffc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:25.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.056+0000 7f69f1400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f69ec102950 0x7f69ec1001c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:39984/0 (socket says 172.21.15.1:39984) 2024-02-02T15:36:25.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.056+0000 7f69f1400700 1 -- 172.21.15.1:0/3992609181 learned_addr learned my addr 172.21.15.1:0/3992609181 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:36:25.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.056+0000 7f69f2800700 1 --2- 172.21.15.1:0/3992609181 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69ec10b400 0x7f69ec1a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:25.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.056+0000 7f69f1400700 1 -- 172.21.15.1:0/3992609181 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69ec10b400 msgr2=0x7f69ec1a50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:25.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.056+0000 7f69f1400700 1 --2- 172.21.15.1:0/3992609181 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69ec10b400 0x7f69ec1a50d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:25.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.056+0000 7f69f1400700 1 -- 172.21.15.1:0/3992609181 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69ec102030 msgr2=0x7f69ec0ffc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:25.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.056+0000 7f69f1400700 1 --2- 172.21.15.1:0/3992609181 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69ec102030 0x7f69ec0ffc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:25.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.056+0000 7f69f1400700 1 -- 172.21.15.1:0/3992609181 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69dc009520 con 0x7f69ec102950 2024-02-02T15:36:25.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.056+0000 7f69f1e00700 1 --2- 172.21.15.1:0/3992609181 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69ec102030 0x7f69ec0ffc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:36:25.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.057+0000 7f69f1400700 1 --2- 172.21.15.1:0/3992609181 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f69ec102950 0x7f69ec1001c0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f69d400e9c0 tx=0x7f69d400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:25.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.057+0000 7f69eaa00700 1 -- 172.21.15.1:0/3992609181 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d4004a30 con 0x7f69ec102950 2024-02-02T15:36:25.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.057+0000 7f69f4800700 1 -- 172.21.15.1:0/3992609181 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69ec1a59b0 con 0x7f69ec102950 2024-02-02T15:36:25.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.057+0000 7f69eaa00700 1 -- 172.21.15.1:0/3992609181 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f69d400cc80 con 0x7f69ec102950 2024-02-02T15:36:25.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.057+0000 7f69eaa00700 1 -- 172.21.15.1:0/3992609181 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d40054e0 con 0x7f69ec102950 2024-02-02T15:36:25.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.057+0000 7f69f4800700 1 -- 172.21.15.1:0/3992609181 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69ec1a5e90 con 0x7f69ec102950 2024-02-02T15:36:25.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.061+0000 7f69eaa00700 1 -- 172.21.15.1:0/3992609181 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f69d4004b90 con 0x7f69ec102950 2024-02-02T15:36:25.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.061+0000 7f6a083ecb80 1 Processor -- start 2024-02-02T15:36:25.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.062+0000 7f6a083ecb80 1 -- start start 2024-02-02T15:36:25.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.062+0000 7f6a083ecb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ca86905010 0x55ca869053f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:25.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.062+0000 7f6a083ecb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca86905930 0x55ca869102a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:25.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.062+0000 7f6a083ecb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca869107e0 0x55ca86912bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:25.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.062+0000 7f6a083ecb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca868794c0 con 0x55ca86905930 2024-02-02T15:36:25.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.062+0000 7f6a083ecb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca8685daa0 con 0x55ca86905010 2024-02-02T15:36:25.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.062+0000 7f6a083ecb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca8685b460 con 0x55ca869107e0 2024-02-02T15:36:25.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.062+0000 7f69f2800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ca86905010 0x55ca869053f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:25.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.062+0000 7f69f2800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ca86905010 0x55ca869053f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:39994/0 (socket says 172.21.15.1:39994) 2024-02-02T15:36:25.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.062+0000 7f69f2800700 1 -- 172.21.15.1:0/463575549 learned_addr learned my addr 172.21.15.1:0/463575549 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:36:25.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.062+0000 7f69f1e00700 1 --2- 172.21.15.1:0/463575549 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca86905930 0x55ca869102a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:25.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.062+0000 7f69f1400700 1 --2- 172.21.15.1:0/463575549 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca869107e0 0x55ca86912bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:25.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.063+0000 7f69f2800700 1 -- 172.21.15.1:0/463575549 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca869107e0 msgr2=0x55ca86912bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:25.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.063+0000 7f69f2800700 1 --2- 172.21.15.1:0/463575549 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca869107e0 0x55ca86912bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:25.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.063+0000 7f69f2800700 1 -- 172.21.15.1:0/463575549 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca86905930 msgr2=0x55ca869102a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:25.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.063+0000 7f69f2800700 1 --2- 172.21.15.1:0/463575549 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca86905930 0x55ca869102a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:25.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.063+0000 7f69f2800700 1 -- 172.21.15.1:0/463575549 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca866d83b0 con 0x55ca86905010 2024-02-02T15:36:25.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.063+0000 7f69f1e00700 1 --2- 172.21.15.1:0/463575549 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca86905930 0x55ca869102a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:36:25.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.063+0000 7f69f2800700 1 --2- 172.21.15.1:0/463575549 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ca86905010 0x55ca869053f0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f69e4011620 tx=0x7f69e4009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c89149a852b3a0c8 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:25.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.063+0000 7f69e3000700 1 -- 172.21.15.1:0/463575549 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e400e040 con 0x55ca86905010 2024-02-02T15:36:25.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.063+0000 7f69e3000700 1 -- 172.21.15.1:0/463575549 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f69e400f040 con 0x55ca86905010 2024-02-02T15:36:25.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.063+0000 7f69e3000700 1 -- 172.21.15.1:0/463575549 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e40047d0 con 0x55ca86905010 2024-02-02T15:36:25.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.064+0000 7f69eaa00700 1 --2- 172.21.15.1:0/3992609181 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f69cc0a0690 0x7f69cc0a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:25.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.064+0000 7f69eaa00700 1 -- 172.21.15.1:0/3992609181 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f69d4014070 con 0x7f69ec102950 2024-02-02T15:36:25.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.064+0000 7f6a083ecb80 1 -- 172.21.15.1:0/463575549 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ca86905010 msgr2=0x55ca869053f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:25.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.064+0000 7f6a083ecb80 1 --2- 172.21.15.1:0/463575549 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ca86905010 0x55ca869053f0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f69e4011620 tx=0x7f69e4009920 comp rx=0 tx=0).stop 2024-02-02T15:36:25.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.064+0000 7f69f1e00700 1 --2- 172.21.15.1:0/3992609181 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f69cc0a0690 0x7f69cc0a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:25.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.064+0000 7f6a083ecb80 1 -- 172.21.15.1:0/463575549 shutdown_connections 2024-02-02T15:36:25.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.064+0000 7f6a083ecb80 1 --2- 172.21.15.1:0/463575549 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca869107e0 0x55ca86912bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:25.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.064+0000 7f6a083ecb80 1 --2- 172.21.15.1:0/463575549 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ca86905010 0x55ca869053f0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:25.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.064+0000 7f6a083ecb80 1 --2- 172.21.15.1:0/463575549 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca86905930 0x55ca869102a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:25.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.065+0000 7f6a083ecb80 1 -- 172.21.15.1:0/463575549 >> 172.21.15.1:0/463575549 conn(0x55ca86864db0 msgr2=0x55ca869013b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:25.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.065+0000 7f6a083ecb80 1 -- 172.21.15.1:0/463575549 shutdown_connections 2024-02-02T15:36:25.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.065+0000 7f6a083ecb80 1 -- 172.21.15.1:0/463575549 wait complete. 2024-02-02T15:36:25.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.065+0000 7f69f1e00700 1 --2- 172.21.15.1:0/3992609181 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f69cc0a0690 0x7f69cc0a2b50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f69dc005e90 tx=0x7f69dc019040 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:25.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.066+0000 7f6a083ecb80 1 -- start start 2024-02-02T15:36:25.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.066+0000 7f6a083ecb80 1 Processor -- start 2024-02-02T15:36:25.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.066+0000 7f6a083ecb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ca86905010 0x55ca869101c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:25.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.066+0000 7f6a083ecb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca86905930 0x55ca8690d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:25.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.066+0000 7f6a083ecb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca869107e0 0x55ca8690d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:25.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.066+0000 7f6a083ecb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca86856d50 con 0x55ca869107e0 2024-02-02T15:36:25.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.067+0000 7f6a083ecb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca867f2d80 con 0x55ca86905010 2024-02-02T15:36:25.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.067+0000 7f6a083ecb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca867f2a20 con 0x55ca86905930 2024-02-02T15:36:25.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.067+0000 7f6a083ecb80 10 client.0 _ll_register_callbacks cb 0x55ca867f4290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:36:25.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.067+0000 7f69f1400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca86905930 0x55ca8690d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:25.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.067+0000 7f69f1e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ca86905010 0x55ca869101c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:25.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.067+0000 7f69f1400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca86905930 0x55ca8690d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:60868/0 (socket says 172.21.15.1:60868) 2024-02-02T15:36:25.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.067+0000 7f69f1400700 1 -- 172.21.15.1:0/662024309 learned_addr learned my addr 172.21.15.1:0/662024309 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:36:25.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.067+0000 7f69f2800700 1 --2- 172.21.15.1:0/662024309 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca869107e0 0x55ca8690d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:25.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.067+0000 7f69f1400700 1 -- 172.21.15.1:0/662024309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ca86905010 msgr2=0x55ca869101c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:25.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.067+0000 7f69f1400700 1 --2- 172.21.15.1:0/662024309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ca86905010 0x55ca869101c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:25.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.067+0000 7f69f1400700 1 -- 172.21.15.1:0/662024309 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca869107e0 msgr2=0x55ca8690d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:25.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.067+0000 7f69f1400700 1 --2- 172.21.15.1:0/662024309 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca869107e0 0x55ca8690d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:25.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.067+0000 7f69f1400700 1 -- 172.21.15.1:0/662024309 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69ec10e560 con 0x55ca86905930 2024-02-02T15:36:25.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.067+0000 7f69f1e00700 1 --2- 172.21.15.1:0/662024309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ca86905010 0x55ca869101c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:36:25.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.067+0000 7f69f1400700 1 --2- 172.21.15.1:0/662024309 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca86905930 0x55ca8690d3d0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f69d4020000 tx=0x7f69d40cf710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7dd067968341be4d server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:25.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.068+0000 7f69db400700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:36:25.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.068+0000 7f69db400700 1 -- 172.21.15.1:0/662024309 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d4099120 con 0x55ca86905930 2024-02-02T15:36:25.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.068+0000 7f69db400700 1 -- 172.21.15.1:0/662024309 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f69d400cdf0 con 0x55ca86905930 2024-02-02T15:36:25.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.068+0000 7f6a083ecb80 1 -- 172.21.15.1:0/662024309 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ca868f90f0 con 0x55ca86905930 2024-02-02T15:36:25.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.068+0000 7f69db400700 1 -- 172.21.15.1:0/662024309 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f69d4098e90 con 0x55ca86905930 2024-02-02T15:36:25.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.068+0000 7f6a083ecb80 10 client.28192 fetch_fsmap learned FSMap version 44 2024-02-02T15:36:25.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.068+0000 7f6a083ecb80 1 -- 172.21.15.1:0/662024309 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x7f69e40112d0 con 0x55ca86905930 2024-02-02T15:36:25.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.068+0000 7f69db400700 1 -- 172.21.15.1:0/662024309 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ca868f90f0 con 0x55ca86905930 2024-02-02T15:36:25.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.068+0000 7f69db400700 1 -- 172.21.15.1:0/662024309 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 44) v1 ==== 7510+0+0 (secure 0 0 0) 0x7f69d40ce0d0 con 0x55ca86905930 2024-02-02T15:36:25.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.068+0000 7f6a083ecb80 10 client.28192 fetch_fsmap finished waiting for FSMap version 44 2024-02-02T15:36:25.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.068+0000 7f6a083ecb80 10 client.28192 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:36:25.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.068+0000 7f6a083ecb80 20 client.28192 populate_metadata read hostname 'smithi001' 2024-02-02T15:36:25.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.069+0000 7f6a083ecb80 1 --2- 172.21.15.1:0/662024309 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55ca86920140 0x55ca86922560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:25.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.069+0000 7f6a083ecb80 4 client.28192 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:36:25.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.069+0000 7f6a083ecb80 1 -- 172.21.15.1:0/662024309 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ca8684fdd0 con 0x55ca86920140 2024-02-02T15:36:25.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.069+0000 7f69f1e00700 1 --2- 172.21.15.1:0/662024309 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55ca86920140 0x55ca86922560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:25.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.069+0000 7f69f1e00700 1 --2- 172.21.15.1:0/662024309 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55ca86920140 0x55ca86922560 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29613da0b8a59fa3 server_cookie=728cc0eda36b0ae5 in_seq=0 out_seq=0 2024-02-02T15:36:25.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.069+0000 7f69db400700 10 client.28192 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:36:25.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.070+0000 7f69db400700 1 -- 172.21.15.1:0/662024309 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55ca8684fdd0 con 0x55ca86920140 2024-02-02T15:36:25.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.070+0000 7f69db400700 10 client.28192 handle_command_reply: tid=0 2024-02-02T15:36:25.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.070+0000 7f6a083ecb80 1 client.28192 shutdown 2024-02-02T15:36:25.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.070+0000 7f6a083ecb80 1 -- 172.21.15.1:0/662024309 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca86905930 msgr2=0x55ca8690d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:25.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.070+0000 7f6a083ecb80 1 --2- 172.21.15.1:0/662024309 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca86905930 0x55ca8690d3d0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f69d4020000 tx=0x7f69d40cf710 comp rx=0 tx=0).stop 2024-02-02T15:36:25.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.070+0000 7f6a083ecb80 1 -- 172.21.15.1:0/662024309 shutdown_connections 2024-02-02T15:36:25.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.070+0000 7f6a083ecb80 1 --2- 172.21.15.1:0/662024309 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55ca86920140 0x55ca86922560 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:25.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.070+0000 7f6a083ecb80 1 --2- 172.21.15.1:0/662024309 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca869107e0 0x55ca8690d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:25.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.070+0000 7f6a083ecb80 1 --2- 172.21.15.1:0/662024309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ca86905010 0x55ca869101c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:25.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.070+0000 7f6a083ecb80 1 --2- 172.21.15.1:0/662024309 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca86905930 0x55ca8690d3d0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:25.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.070+0000 7f6a083ecb80 1 -- 172.21.15.1:0/662024309 >> 172.21.15.1:0/662024309 conn(0x55ca86864db0 msgr2=0x55ca86900fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:25.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.070+0000 7f69db400700 0 client.28192 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:36:25.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.070+0000 7f6a083ecb80 1 -- 172.21.15.1:0/662024309 shutdown_connections 2024-02-02T15:36:25.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.070+0000 7f6a083ecb80 1 -- 172.21.15.1:0/662024309 wait complete. 2024-02-02T15:36:25.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.071+0000 7f6a083ecb80 20 client.28192 trim_cache size 0 max 16384 2024-02-02T15:36:25.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.087+0000 7f6a083ecb80 1 Processor -- start 2024-02-02T15:36:25.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.088+0000 7f6a083ecb80 1 -- start start 2024-02-02T15:36:25.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.088+0000 7f6a083ecb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca86905010 0x55ca8690d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:25.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.088+0000 7f6a083ecb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca86905930 0x55ca86907840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:25.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.088+0000 7f6a083ecb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ca869107e0 0x55ca8690fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:25.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.088+0000 7f6a083ecb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca866da6a0 con 0x55ca86905930 2024-02-02T15:36:25.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.088+0000 7f6a083ecb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca8691d820 con 0x55ca869107e0 2024-02-02T15:36:25.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.088+0000 7f6a083ecb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca8690b970 con 0x55ca86905010 2024-02-02T15:36:25.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.088+0000 7f69f1e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca86905010 0x55ca8690d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:25.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.088+0000 7f69f1400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca86905930 0x55ca86907840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:25.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.088+0000 7f69f1e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca86905010 0x55ca8690d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:60872/0 (socket says 172.21.15.1:60872) 2024-02-02T15:36:25.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.088+0000 7f69f1e00700 1 -- 172.21.15.1:0/1700116830 learned_addr learned my addr 172.21.15.1:0/1700116830 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:36:25.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.088+0000 7f69f2800700 1 --2- 172.21.15.1:0/1700116830 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ca869107e0 0x55ca8690fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:25.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.088+0000 7f69f1e00700 1 -- 172.21.15.1:0/1700116830 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ca869107e0 msgr2=0x55ca8690fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:25.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.088+0000 7f69f1e00700 1 --2- 172.21.15.1:0/1700116830 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ca869107e0 0x55ca8690fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:25.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.088+0000 7f69f1e00700 1 -- 172.21.15.1:0/1700116830 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca86905930 msgr2=0x55ca86907840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:25.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.088+0000 7f69f1e00700 1 --2- 172.21.15.1:0/1700116830 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca86905930 0x55ca86907840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:25.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.088+0000 7f69f1e00700 1 -- 172.21.15.1:0/1700116830 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca86684b60 con 0x55ca86905010 2024-02-02T15:36:25.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.088+0000 7f69f1400700 1 --2- 172.21.15.1:0/1700116830 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca86905930 0x55ca86907840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:36:25.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.089+0000 7f69f1e00700 1 --2- 172.21.15.1:0/1700116830 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca86905010 0x55ca8690d380 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f69dc0098c0 tx=0x7f69dc025000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=474c29ee157a3887 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:25.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.089+0000 7f69e3a00700 1 -- 172.21.15.1:0/1700116830 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69dc01c040 con 0x55ca86905010 2024-02-02T15:36:25.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.089+0000 7f69e3a00700 1 -- 172.21.15.1:0/1700116830 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f69dc01d040 con 0x55ca86905010 2024-02-02T15:36:25.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.089+0000 7f69e3a00700 1 -- 172.21.15.1:0/1700116830 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69dc003680 con 0x55ca86905010 2024-02-02T15:36:25.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.090+0000 7f6a083ecb80 1 -- 172.21.15.1:0/1700116830 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca86905010 msgr2=0x55ca8690d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:25.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.090+0000 7f6a083ecb80 1 --2- 172.21.15.1:0/1700116830 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca86905010 0x55ca8690d380 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f69dc0098c0 tx=0x7f69dc025000 comp rx=0 tx=0).stop 2024-02-02T15:36:25.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.090+0000 7f6a083ecb80 1 -- 172.21.15.1:0/1700116830 shutdown_connections 2024-02-02T15:36:25.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.090+0000 7f6a083ecb80 1 --2- 172.21.15.1:0/1700116830 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ca869107e0 0x55ca8690fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:25.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.090+0000 7f6a083ecb80 1 --2- 172.21.15.1:0/1700116830 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca86905010 0x55ca8690d380 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:25.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.090+0000 7f6a083ecb80 1 --2- 172.21.15.1:0/1700116830 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca86905930 0x55ca86907840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:25.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.090+0000 7f6a083ecb80 1 -- 172.21.15.1:0/1700116830 >> 172.21.15.1:0/1700116830 conn(0x55ca86864db0 msgr2=0x55ca8690cc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:25.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.090+0000 7f6a083ecb80 1 -- 172.21.15.1:0/1700116830 shutdown_connections 2024-02-02T15:36:25.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.090+0000 7f6a083ecb80 1 -- 172.21.15.1:0/1700116830 wait complete. 2024-02-02T15:36:25.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.090+0000 7f6a083ecb80 1 -- start start 2024-02-02T15:36:25.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.090+0000 7f6a083ecb80 1 Processor -- start 2024-02-02T15:36:25.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.091+0000 7f6a083ecb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca86905930 0x55ca86915760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:25.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.091+0000 7f6a083ecb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ca869107e0 0x55ca86915ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:25.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.091+0000 7f6a083ecb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca869161e0 0x55ca86971c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:25.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.091+0000 7f6a083ecb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca86917570 con 0x55ca86905930 2024-02-02T15:36:25.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.091+0000 7f6a083ecb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca869176b0 con 0x55ca869107e0 2024-02-02T15:36:25.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.091+0000 7f6a083ecb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca869721b0 con 0x55ca869161e0 2024-02-02T15:36:25.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.091+0000 7f6a083ecb80 10 client.0 _ll_register_callbacks cb 0x55ca867f4290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:36:25.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.091+0000 7f69f1e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca86905930 0x55ca86915760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:25.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.091+0000 7f69f2800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca869161e0 0x55ca86971c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:25.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.091+0000 7f69f1e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca86905930 0x55ca86915760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:54316/0 (socket says 172.21.15.1:54316) 2024-02-02T15:36:25.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.091+0000 7f69f2800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca869161e0 0x55ca86971c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:60886/0 (socket says 172.21.15.1:60886) 2024-02-02T15:36:25.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.091+0000 7f69f1e00700 1 -- 172.21.15.1:0/3414795120 learned_addr learned my addr 172.21.15.1:0/3414795120 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:36:25.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.091+0000 7f69f1400700 1 --2- 172.21.15.1:0/3414795120 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ca869107e0 0x55ca86915ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:25.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.092+0000 7f69f1e00700 1 -- 172.21.15.1:0/3414795120 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca869161e0 msgr2=0x55ca86971c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:25.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.092+0000 7f69f1e00700 1 --2- 172.21.15.1:0/3414795120 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca869161e0 0x55ca86971c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:25.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.092+0000 7f69f1e00700 1 -- 172.21.15.1:0/3414795120 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ca869107e0 msgr2=0x55ca86915ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:25.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.092+0000 7f69f1e00700 1 --2- 172.21.15.1:0/3414795120 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ca869107e0 0x55ca86915ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:25.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.092+0000 7f69f1e00700 1 -- 172.21.15.1:0/3414795120 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69e40112d0 con 0x55ca86905930 2024-02-02T15:36:25.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.092+0000 7f69f2800700 1 --2- 172.21.15.1:0/3414795120 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca869161e0 0x55ca86971c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:36:25.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.092+0000 7f69f1400700 1 --2- 172.21.15.1:0/3414795120 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ca869107e0 0x55ca86915ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:36:25.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.092+0000 7f69f1e00700 1 --2- 172.21.15.1:0/3414795120 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca86905930 0x55ca86915760 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f69dc00c010 tx=0x7f69dc00b560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf21cc1b8b3a2557 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:25.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.092+0000 7f69db400700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:36:25.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.092+0000 7f69db400700 1 -- 172.21.15.1:0/3414795120 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69dc01c070 con 0x55ca86905930 2024-02-02T15:36:25.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.092+0000 7f6a083ecb80 1 -- 172.21.15.1:0/3414795120 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f69e400f040 con 0x55ca86905930 2024-02-02T15:36:25.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.092+0000 7f69db400700 1 -- 172.21.15.1:0/3414795120 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f69dc033050 con 0x55ca86905930 2024-02-02T15:36:25.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.092+0000 7f69db400700 1 -- 172.21.15.1:0/3414795120 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f69dc033210 con 0x55ca86905930 2024-02-02T15:36:25.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.092+0000 7f6a083ecb80 10 client.18114 fetch_fsmap learned FSMap version 44 2024-02-02T15:36:25.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.092+0000 7f6a083ecb80 1 -- 172.21.15.1:0/3414795120 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x7f69dc00b8c0 con 0x55ca86905930 2024-02-02T15:36:25.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.093+0000 7f69db400700 1 -- 172.21.15.1:0/3414795120 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f69e400f040 con 0x55ca86905930 2024-02-02T15:36:25.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.093+0000 7f69db400700 1 -- 172.21.15.1:0/3414795120 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 44) v1 ==== 7510+0+0 (secure 0 0 0) 0x7f69dc047070 con 0x55ca86905930 2024-02-02T15:36:25.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.093+0000 7f6a083ecb80 10 client.18114 fetch_fsmap finished waiting for FSMap version 44 2024-02-02T15:36:25.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.093+0000 7f6a083ecb80 10 client.18114 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:36:25.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.093+0000 7f6a083ecb80 20 client.18114 populate_metadata read hostname 'smithi001' 2024-02-02T15:36:25.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.093+0000 7f6a083ecb80 1 --2- 172.21.15.1:0/3414795120 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55ca869735c0 0x55ca869759e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:25.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.093+0000 7f6a083ecb80 4 client.18114 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2024-02-02T15:36:25.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.093+0000 7f6a083ecb80 1 -- 172.21.15.1:0/3414795120 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ca86815f50 con 0x55ca869735c0 2024-02-02T15:36:25.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.093+0000 7f69f1400700 1 --2- 172.21.15.1:0/3414795120 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55ca869735c0 0x55ca869759e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:25.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.094+0000 7f69f1400700 1 --2- 172.21.15.1:0/3414795120 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55ca869735c0 0x55ca869759e0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=424241bd0ecc0232 server_cookie=6d0ebc0c907e7939 in_seq=0 out_seq=0 2024-02-02T15:36:25.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.094+0000 7f69db400700 10 client.18114 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:36:25.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.094+0000 7f69db400700 1 -- 172.21.15.1:0/3414795120 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55ca86815f50 con 0x55ca869735c0 2024-02-02T15:36:25.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.094+0000 7f69db400700 10 client.18114 handle_command_reply: tid=0 2024-02-02T15:36:25.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.094+0000 7f6a083ecb80 1 client.18114 shutdown 2024-02-02T15:36:25.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.094+0000 7f6a083ecb80 1 -- 172.21.15.1:0/3414795120 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca86905930 msgr2=0x55ca86915760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:25.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.094+0000 7f6a083ecb80 1 --2- 172.21.15.1:0/3414795120 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca86905930 0x55ca86915760 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f69dc00c010 tx=0x7f69dc00b560 comp rx=0 tx=0).stop 2024-02-02T15:36:25.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.094+0000 7f6a083ecb80 1 -- 172.21.15.1:0/3414795120 shutdown_connections 2024-02-02T15:36:25.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.094+0000 7f6a083ecb80 1 --2- 172.21.15.1:0/3414795120 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55ca869735c0 0x55ca869759e0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:25.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.094+0000 7f6a083ecb80 1 --2- 172.21.15.1:0/3414795120 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ca869161e0 0x55ca86971c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:25.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.094+0000 7f6a083ecb80 1 --2- 172.21.15.1:0/3414795120 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ca86905930 0x55ca86915760 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:25.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.094+0000 7f69db400700 0 client.18114 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:36:25.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.094+0000 7f6a083ecb80 1 --2- 172.21.15.1:0/3414795120 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ca869107e0 0x55ca86915ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:25.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.094+0000 7f6a083ecb80 1 -- 172.21.15.1:0/3414795120 >> 172.21.15.1:0/3414795120 conn(0x55ca86864db0 msgr2=0x55ca8690bbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:25.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.094+0000 7f6a083ecb80 1 -- 172.21.15.1:0/3414795120 shutdown_connections 2024-02-02T15:36:25.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.094+0000 7f6a083ecb80 1 -- 172.21.15.1:0/3414795120 wait complete. 2024-02-02T15:36:25.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.095+0000 7f6a083ecb80 20 client.18114 trim_cache size 0 max 16384 2024-02-02T15:36:25.099 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:36:25.099 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-02T15:36:25.099 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2024-02-02T15:36:25.099 INFO:teuthology.orchestra.run.smithi001.stdout: "5eb42348-95f0-467f-9871-02e9588e3a8a": { 2024-02-02T15:36:25.099 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2024-02-02T15:36:25.099 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "5eb42348-95f0-467f-9871-02e9588e3a8a", 2024-02-02T15:36:25.099 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,force" 2024-02-02T15:36:25.100 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:36:25.100 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:36:25.100 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:36:25.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.095+0000 7f69f4800700 1 -- 172.21.15.1:0/3992609181 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f69cc0a0690 msgr2=0x7f69cc0a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:25.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.095+0000 7f69f4800700 1 --2- 172.21.15.1:0/3992609181 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f69cc0a0690 0x7f69cc0a2b50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f69dc005e90 tx=0x7f69dc019040 comp rx=0 tx=0).stop 2024-02-02T15:36:25.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.095+0000 7f69f4800700 1 -- 172.21.15.1:0/3992609181 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f69ec102950 msgr2=0x7f69ec1001c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:25.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.095+0000 7f69f4800700 1 --2- 172.21.15.1:0/3992609181 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f69ec102950 0x7f69ec1001c0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f69d400e9c0 tx=0x7f69d400ecd0 comp rx=0 tx=0).stop 2024-02-02T15:36:25.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.095+0000 7f69f4800700 1 -- 172.21.15.1:0/3992609181 shutdown_connections 2024-02-02T15:36:25.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.095+0000 7f69f4800700 1 --2- 172.21.15.1:0/3992609181 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f69cc0a0690 0x7f69cc0a2b50 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:25.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.095+0000 7f69f4800700 1 --2- 172.21.15.1:0/3992609181 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f69ec10b400 0x7f69ec1a50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:25.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.095+0000 7f69f4800700 1 --2- 172.21.15.1:0/3992609181 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f69ec102030 0x7f69ec0ffc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:25.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.095+0000 7f69f4800700 1 --2- 172.21.15.1:0/3992609181 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f69ec102950 0x7f69ec1001c0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:25.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.095+0000 7f69f4800700 1 -- 172.21.15.1:0/3992609181 >> 172.21.15.1:0/3992609181 conn(0x7f69ec076d70 msgr2=0x7f69ec0fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:25.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.095+0000 7f69f4800700 1 -- 172.21.15.1:0/3992609181 shutdown_connections 2024-02-02T15:36:25.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:25.095+0000 7f69f4800700 1 -- 172.21.15.1:0/3992609181 wait complete. 2024-02-02T15:36:25.115 INFO:tasks.cephfs.filesystem:scrub status for tag:5eb42348-95f0-467f-9871-02e9588e3a8a - {'path': '/', 'tag': '5eb42348-95f0-467f-9871-02e9588e3a8a', 'options': 'recursive,force'} 2024-02-02T15:36:25.256 INFO:tasks.workunit.client.0.smithi001.stdout: 1 552171 6.23 MB/sec execute 359 sec latency 20.393 ms 2024-02-02T15:36:25.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:25 smithi001 ceph-mon[35470]: pgmap v291: 129 pgs: 129 active+clean; 412 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 130 op/s 2024-02-02T15:36:25.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:25 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:36:25.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:25 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:36:25.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:25 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:36:25.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:25 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:36:25.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:25 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:36:25.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:25 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:36:25.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:25 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:36:25.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:25 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:36:25.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:25 smithi071 ceph-mon[44707]: pgmap v291: 129 pgs: 129 active+clean; 412 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 130 op/s 2024-02-02T15:36:25.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:25 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:36:25.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:25 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:36:25.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:25 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:36:25.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:25 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:36:25.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:25 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:36:25.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:25 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:36:25.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:25 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:36:25.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:25 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:36:25.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:25 smithi142 ceph-mon[45728]: pgmap v291: 129 pgs: 129 active+clean; 412 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 130 op/s 2024-02-02T15:36:25.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:25 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:36:25.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:25 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:36:25.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:25 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:36:25.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:25 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:36:25.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:25 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:36:25.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:25 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:36:25.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:25 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:36:25.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:25 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:36:26.256 INFO:tasks.workunit.client.0.smithi001.stdout: 1 552305 6.21 MB/sec execute 360 sec latency 840.503 ms 2024-02-02T15:36:26.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:26 smithi001 ceph-mon[35470]: scrub queued for path: / 2024-02-02T15:36:26.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:26 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:36:26.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:26 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:36:26.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:26 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:36:26.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:26 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:36:26.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:26 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:36:26.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:26 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:36:26.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:26 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:36:26.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:26 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:36:26.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:26 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:36:26.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:26 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:36:26.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:26 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:36:26.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:26 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:36:26.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:26 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:36:26.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:26 smithi071 ceph-mon[44707]: scrub queued for path: / 2024-02-02T15:36:26.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:26 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:36:26.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:26 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:36:26.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:26 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:36:26.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:26 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:36:26.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:26 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:36:26.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:26 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:36:26.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:26 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:36:26.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:26 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:36:26.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:26 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:36:26.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:26 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:36:26.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:26 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:36:26.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:26 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:36:26.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:26 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:36:26.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:26 smithi142 ceph-mon[45728]: scrub queued for path: / 2024-02-02T15:36:26.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:26 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:36:26.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:26 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:36:26.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:26 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:36:26.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:26 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:36:26.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:26 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:36:26.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:26 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:36:26.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:26 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:36:26.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:26 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:36:26.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:26 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:36:26.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:26 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:36:26.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:26 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:36:26.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:26 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:36:26.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:26 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:36:27.256 INFO:tasks.workunit.client.0.smithi001.stdout: 1 552305 6.20 MB/sec execute 361 sec latency 1840.622 ms 2024-02-02T15:36:27.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:27 smithi001 ceph-mon[35470]: pgmap v292: 129 pgs: 129 active+clean; 416 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.7 MiB/s wr, 197 op/s 2024-02-02T15:36:27.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:27 smithi071 ceph-mon[44707]: pgmap v292: 129 pgs: 129 active+clean; 416 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.7 MiB/s wr, 197 op/s 2024-02-02T15:36:27.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:27 smithi142 ceph-mon[45728]: pgmap v292: 129 pgs: 129 active+clean; 416 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.7 MiB/s wr, 197 op/s 2024-02-02T15:36:28.256 INFO:tasks.workunit.client.0.smithi001.stdout: 1 553491 6.20 MB/sec execute 362 sec latency 2227.284 ms 2024-02-02T15:36:29.256 INFO:tasks.workunit.client.0.smithi001.stdout: 1 554636 6.20 MB/sec execute 363 sec latency 17.312 ms 2024-02-02T15:36:29.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:29 smithi001 ceph-mon[35470]: pgmap v293: 129 pgs: 129 active+clean; 416 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 135 op/s 2024-02-02T15:36:29.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:29 smithi071 ceph-mon[44707]: pgmap v293: 129 pgs: 129 active+clean; 416 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 135 op/s 2024-02-02T15:36:29.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:29 smithi142 ceph-mon[45728]: pgmap v293: 129 pgs: 129 active+clean; 416 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 135 op/s 2024-02-02T15:36:30.257 INFO:tasks.workunit.client.0.smithi001.stdout: 1 555852 6.20 MB/sec execute 364 sec latency 91.767 ms 2024-02-02T15:36:31.256 INFO:tasks.workunit.client.0.smithi001.stdout: 1 556590 6.19 MB/sec execute 365 sec latency 753.949 ms 2024-02-02T15:36:31.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:31 smithi071 ceph-mon[44707]: pgmap v294: 129 pgs: 129 active+clean; 416 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 135 op/s 2024-02-02T15:36:31.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:31 smithi142 ceph-mon[45728]: pgmap v294: 129 pgs: 129 active+clean; 416 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 135 op/s 2024-02-02T15:36:31.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:31 smithi001 ceph-mon[35470]: pgmap v294: 129 pgs: 129 active+clean; 416 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 135 op/s 2024-02-02T15:36:32.256 INFO:tasks.workunit.client.0.smithi001.stdout: 1 558081 6.20 MB/sec execute 366 sec latency 14.115 ms 2024-02-02T15:36:32.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:32 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:36:32.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:32 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:36:32.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:32 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:36:32.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:32 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:36:32.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:32 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:36:32.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:32 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:36:32.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:32 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:36:32.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:32 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:36:32.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:32 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:36:32.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:32 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:36:32.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:32 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:36:32.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:32 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:36:32.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:32 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:36:32.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:32 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:36:32.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:32 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:36:33.257 INFO:tasks.workunit.client.0.smithi001.stdout: 1 559198 6.20 MB/sec execute 367 sec latency 21.780 ms 2024-02-02T15:36:33.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:33 smithi071 ceph-mon[44707]: pgmap v295: 129 pgs: 129 active+clean; 420 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 199 op/s 2024-02-02T15:36:33.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:33 smithi142 ceph-mon[45728]: pgmap v295: 129 pgs: 129 active+clean; 420 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 199 op/s 2024-02-02T15:36:33.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:33 smithi001 ceph-mon[35470]: pgmap v295: 129 pgs: 129 active+clean; 420 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 199 op/s 2024-02-02T15:36:34.257 INFO:tasks.workunit.client.0.smithi001.stdout: 1 559397 6.18 MB/sec execute 368 sec latency 817.974 ms 2024-02-02T15:36:35.257 INFO:tasks.workunit.client.0.smithi001.stdout: 1 559397 6.17 MB/sec execute 369 sec latency 1818.047 ms 2024-02-02T15:36:35.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:35 smithi071 ceph-mon[44707]: pgmap v296: 129 pgs: 129 active+clean; 420 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 131 op/s 2024-02-02T15:36:35.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:35 smithi142 ceph-mon[45728]: pgmap v296: 129 pgs: 129 active+clean; 420 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 131 op/s 2024-02-02T15:36:35.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:35 smithi001 ceph-mon[35470]: pgmap v296: 129 pgs: 129 active+clean; 420 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 131 op/s 2024-02-02T15:36:36.257 INFO:tasks.workunit.client.0.smithi001.stdout: 1 559397 6.15 MB/sec execute 370 sec latency 2818.155 ms 2024-02-02T15:36:37.257 INFO:tasks.workunit.client.0.smithi001.stdout: 1 559397 6.13 MB/sec execute 371 sec latency 3818.244 ms 2024-02-02T15:36:37.867 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:37 smithi071 ceph-mon[44707]: pgmap v297: 129 pgs: 129 active+clean; 423 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 185 op/s 2024-02-02T15:36:37.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:37 smithi142 ceph-mon[45728]: pgmap v297: 129 pgs: 129 active+clean; 423 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 185 op/s 2024-02-02T15:36:37.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:37 smithi001 ceph-mon[35470]: pgmap v297: 129 pgs: 129 active+clean; 423 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 185 op/s 2024-02-02T15:36:38.257 INFO:tasks.workunit.client.0.smithi001.stdout: 1 560477 6.13 MB/sec execute 372 sec latency 4204.224 ms 2024-02-02T15:36:39.257 INFO:tasks.workunit.client.0.smithi001.stdout: 1 561691 6.14 MB/sec execute 373 sec latency 17.563 ms 2024-02-02T15:36:39.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:39 smithi071 ceph-mon[44707]: pgmap v298: 129 pgs: 129 active+clean; 423 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 118 op/s 2024-02-02T15:36:39.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:39 smithi142 ceph-mon[45728]: pgmap v298: 129 pgs: 129 active+clean; 423 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 118 op/s 2024-02-02T15:36:39.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:39 smithi001 ceph-mon[35470]: pgmap v298: 129 pgs: 129 active+clean; 423 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 118 op/s 2024-02-02T15:36:40.257 INFO:tasks.workunit.client.0.smithi001.stdout: 1 562946 6.14 MB/sec execute 374 sec latency 53.360 ms 2024-02-02T15:36:41.257 INFO:tasks.workunit.client.0.smithi001.stdout: 1 562946 6.12 MB/sec execute 375 sec latency 1053.430 ms 2024-02-02T15:36:41.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:41 smithi071 ceph-mon[44707]: pgmap v299: 129 pgs: 129 active+clean; 423 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 118 op/s 2024-02-02T15:36:41.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:41 smithi142 ceph-mon[45728]: pgmap v299: 129 pgs: 129 active+clean; 423 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 118 op/s 2024-02-02T15:36:41.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:41 smithi001 ceph-mon[35470]: pgmap v299: 129 pgs: 129 active+clean; 423 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 118 op/s 2024-02-02T15:36:42.257 INFO:tasks.workunit.client.0.smithi001.stdout: 1 562946 6.10 MB/sec execute 376 sec latency 2053.544 ms 2024-02-02T15:36:43.257 INFO:tasks.workunit.client.0.smithi001.stdout: 1 564153 6.10 MB/sec execute 377 sec latency 2440.183 ms 2024-02-02T15:36:43.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:43 smithi071 ceph-mon[44707]: pgmap v300: 129 pgs: 129 active+clean; 427 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 175 op/s 2024-02-02T15:36:43.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:43 smithi142 ceph-mon[45728]: pgmap v300: 129 pgs: 129 active+clean; 427 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 175 op/s 2024-02-02T15:36:43.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:43 smithi001 ceph-mon[35470]: pgmap v300: 129 pgs: 129 active+clean; 427 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.1 MiB/s wr, 175 op/s 2024-02-02T15:36:44.257 INFO:tasks.workunit.client.0.smithi001.stdout: 1 565261 6.11 MB/sec execute 378 sec latency 44.092 ms 2024-02-02T15:36:45.258 INFO:tasks.workunit.client.0.smithi001.stdout: 1 566486 6.11 MB/sec execute 379 sec latency 83.784 ms 2024-02-02T15:36:45.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:45 smithi071 ceph-mon[44707]: pgmap v301: 129 pgs: 129 active+clean; 427 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 111 op/s 2024-02-02T15:36:45.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:45 smithi142 ceph-mon[45728]: pgmap v301: 129 pgs: 129 active+clean; 427 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 111 op/s 2024-02-02T15:36:45.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:45 smithi001 ceph-mon[35470]: pgmap v301: 129 pgs: 129 active+clean; 427 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 111 op/s 2024-02-02T15:36:46.258 INFO:tasks.workunit.client.0.smithi001.stdout: 1 566486 6.09 MB/sec execute 380 sec latency 1083.875 ms 2024-02-02T15:36:47.258 INFO:tasks.workunit.client.0.smithi001.stdout: 1 566486 6.07 MB/sec execute 381 sec latency 2083.969 ms 2024-02-02T15:36:47.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:47 smithi071 ceph-mon[44707]: pgmap v302: 129 pgs: 129 active+clean; 431 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 173 op/s 2024-02-02T15:36:47.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:47 smithi142 ceph-mon[45728]: pgmap v302: 129 pgs: 129 active+clean; 431 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 173 op/s 2024-02-02T15:36:47.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:47 smithi001 ceph-mon[35470]: pgmap v302: 129 pgs: 129 active+clean; 431 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 173 op/s 2024-02-02T15:36:48.258 INFO:tasks.workunit.client.0.smithi001.stdout: 1 567235 6.06 MB/sec execute 382 sec latency 2467.949 ms 2024-02-02T15:36:49.258 INFO:tasks.workunit.client.0.smithi001.stdout: 1 568736 6.08 MB/sec execute 383 sec latency 16.367 ms 2024-02-02T15:36:49.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:49 smithi071 ceph-mon[44707]: pgmap v303: 129 pgs: 129 active+clean; 431 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 119 op/s 2024-02-02T15:36:49.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:49 smithi142 ceph-mon[45728]: pgmap v303: 129 pgs: 129 active+clean; 431 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 119 op/s 2024-02-02T15:36:49.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:49 smithi001 ceph-mon[35470]: pgmap v303: 129 pgs: 129 active+clean; 431 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 119 op/s 2024-02-02T15:36:50.258 INFO:tasks.workunit.client.0.smithi001.stdout: 1 569852 6.08 MB/sec execute 384 sec latency 17.072 ms 2024-02-02T15:36:50.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:50 smithi071 ceph-mon[44707]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:36:50.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:50 smithi142 ceph-mon[45728]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:36:50.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:50 smithi001 ceph-mon[35470]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:36:51.258 INFO:tasks.workunit.client.0.smithi001.stdout: 1 570036 6.06 MB/sec execute 385 sec latency 800.198 ms 2024-02-02T15:36:51.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:51 smithi071 ceph-mon[44707]: pgmap v304: 129 pgs: 129 active+clean; 431 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 119 op/s 2024-02-02T15:36:51.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:51 smithi142 ceph-mon[45728]: pgmap v304: 129 pgs: 129 active+clean; 431 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 119 op/s 2024-02-02T15:36:51.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:51 smithi001 ceph-mon[35470]: pgmap v304: 129 pgs: 129 active+clean; 431 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 119 op/s 2024-02-02T15:36:52.258 INFO:tasks.workunit.client.0.smithi001.stdout: 1 570036 6.04 MB/sec execute 386 sec latency 1800.346 ms 2024-02-02T15:36:53.258 INFO:tasks.workunit.client.0.smithi001.stdout: 1 571274 6.05 MB/sec execute 387 sec latency 2181.611 ms 2024-02-02T15:36:53.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:53 smithi071 ceph-mon[44707]: pgmap v305: 129 pgs: 129 active+clean; 435 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 174 op/s 2024-02-02T15:36:53.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:53 smithi142 ceph-mon[45728]: pgmap v305: 129 pgs: 129 active+clean; 435 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 174 op/s 2024-02-02T15:36:53.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:53 smithi001 ceph-mon[35470]: pgmap v305: 129 pgs: 129 active+clean; 435 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 174 op/s 2024-02-02T15:36:54.258 INFO:tasks.workunit.client.0.smithi001.stdout: 1 572479 6.05 MB/sec execute 388 sec latency 29.098 ms 2024-02-02T15:36:55.116 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-02T15:36:55.258 INFO:tasks.workunit.client.0.smithi001.stdout: 1 573579 6.05 MB/sec execute 389 sec latency 172.960 ms 2024-02-02T15:36:55.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.319+0000 7f7d95200700 1 -- 172.21.15.1:0/801999029 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7d900730e0 msgr2=0x7f7d900734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:55.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.319+0000 7f7d95200700 1 --2- 172.21.15.1:0/801999029 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7d900730e0 0x7f7d900734c0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f7d800098f0 tx=0x7f7d80009c00 comp rx=0 tx=0).stop 2024-02-02T15:36:55.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.320+0000 7f7d95200700 1 -- 172.21.15.1:0/801999029 shutdown_connections 2024-02-02T15:36:55.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.320+0000 7f7d95200700 1 --2- 172.21.15.1:0/801999029 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d90111d90 0x7f7d90114180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:55.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.320+0000 7f7d95200700 1 --2- 172.21.15.1:0/801999029 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7d900730e0 0x7f7d900734c0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:55.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.320+0000 7f7d95200700 1 --2- 172.21.15.1:0/801999029 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d90073a00 0x7f7d90111850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:55.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.320+0000 7f7d95200700 1 -- 172.21.15.1:0/801999029 >> 172.21.15.1:0/801999029 conn(0x7f7d900fc810 msgr2=0x7f7d900fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:55.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.320+0000 7f7d95200700 1 -- 172.21.15.1:0/801999029 shutdown_connections 2024-02-02T15:36:55.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.320+0000 7f7d95200700 1 -- 172.21.15.1:0/801999029 wait complete. 2024-02-02T15:36:55.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.321+0000 7f7d95200700 1 Processor -- start 2024-02-02T15:36:55.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.321+0000 7f7d95200700 1 -- start start 2024-02-02T15:36:55.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.321+0000 7f7d95200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d900730e0 0x7f7d9019d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:55.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.321+0000 7f7d95200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d90073a00 0x7f7d9019da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:55.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.321+0000 7f7d95200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7d90111d90 0x7f7d901a1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:55.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.321+0000 7f7d95200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d901a2510 con 0x7f7d90111d90 2024-02-02T15:36:55.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.321+0000 7f7d95200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d901a2680 con 0x7f7d900730e0 2024-02-02T15:36:55.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.321+0000 7f7d95200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d901a2840 con 0x7f7d90073a00 2024-02-02T15:36:55.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.321+0000 7f7d8e800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7d90111d90 0x7f7d901a1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:55.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.321+0000 7f7d8d400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d90073a00 0x7f7d9019da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:55.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.321+0000 7f7d8e800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7d90111d90 0x7f7d901a1ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48614/0 (socket says 172.21.15.1:48614) 2024-02-02T15:36:55.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.322+0000 7f7d8e800700 1 -- 172.21.15.1:0/2949716896 learned_addr learned my addr 172.21.15.1:0/2949716896 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:36:55.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.322+0000 7f7d8de00700 1 --2- 172.21.15.1:0/2949716896 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d900730e0 0x7f7d9019d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:55.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.322+0000 7f7d8d400700 1 -- 172.21.15.1:0/2949716896 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d900730e0 msgr2=0x7f7d9019d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:55.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.322+0000 7f7d8d400700 1 --2- 172.21.15.1:0/2949716896 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d900730e0 0x7f7d9019d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:55.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.322+0000 7f7d8d400700 1 -- 172.21.15.1:0/2949716896 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7d90111d90 msgr2=0x7f7d901a1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:55.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.322+0000 7f7d8d400700 1 --2- 172.21.15.1:0/2949716896 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7d90111d90 0x7f7d901a1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:55.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.322+0000 7f7d8d400700 1 -- 172.21.15.1:0/2949716896 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d80009520 con 0x7f7d90073a00 2024-02-02T15:36:55.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.322+0000 7f7d8d400700 1 --2- 172.21.15.1:0/2949716896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d90073a00 0x7f7d9019da70 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f7d7800e9f0 tx=0x7f7d7800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:55.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.322+0000 7f7d86a00700 1 -- 172.21.15.1:0/2949716896 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d7800c9c0 con 0x7f7d90073a00 2024-02-02T15:36:55.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.322+0000 7f7d86a00700 1 -- 172.21.15.1:0/2949716896 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d78010430 con 0x7f7d90073a00 2024-02-02T15:36:55.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.322+0000 7f7d86a00700 1 -- 172.21.15.1:0/2949716896 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d78018680 con 0x7f7d90073a00 2024-02-02T15:36:55.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.323+0000 7f7d95200700 1 -- 172.21.15.1:0/2949716896 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d901a2b20 con 0x7f7d90073a00 2024-02-02T15:36:55.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.323+0000 7f7d95200700 1 -- 172.21.15.1:0/2949716896 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d901aa360 con 0x7f7d90073a00 2024-02-02T15:36:55.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.326+0000 7f7d86a00700 1 -- 172.21.15.1:0/2949716896 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f7d78010860 con 0x7f7d90073a00 2024-02-02T15:36:55.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.327+0000 7f7d86a00700 1 --2- 172.21.15.1:0/2949716896 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f7d700a0690 0x7f7d700a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:55.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.327+0000 7f7da8c55b80 1 Processor -- start 2024-02-02T15:36:55.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.327+0000 7f7d86a00700 1 -- 172.21.15.1:0/2949716896 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f7d78014070 con 0x7f7d90073a00 2024-02-02T15:36:55.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.327+0000 7f7da8c55b80 1 -- start start 2024-02-02T15:36:55.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.327+0000 7f7d8de00700 1 --2- 172.21.15.1:0/2949716896 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f7d700a0690 0x7f7d700a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:55.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.327+0000 7f7da8c55b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7aaa51010 0x55d7aaa513f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:55.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.327+0000 7f7da8c55b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7aaa51930 0x55d7aaa5c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:55.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.327+0000 7f7da8c55b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7aaa5c7e0 0x55d7aaa5ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:55.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.327+0000 7f7da8c55b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7aa9c54c0 con 0x55d7aaa51010 2024-02-02T15:36:55.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.328+0000 7f7da8c55b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7aa9a9aa0 con 0x55d7aaa5c7e0 2024-02-02T15:36:55.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.328+0000 7f7da8c55b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7aa9a7460 con 0x55d7aaa51930 2024-02-02T15:36:55.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.328+0000 7f7d8d400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7aaa51930 0x55d7aaa5c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:55.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.328+0000 7f7d8e800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7aaa5c7e0 0x55d7aaa5ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:55.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.328+0000 7f7d8d400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7aaa51930 0x55d7aaa5c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:33968/0 (socket says 172.21.15.1:33968) 2024-02-02T15:36:55.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.328+0000 7f7d8d400700 1 -- 172.21.15.1:0/2946624423 learned_addr learned my addr 172.21.15.1:0/2946624423 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:36:55.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.328+0000 7f7d8de00700 1 --2- 172.21.15.1:0/2946624423 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7aaa51010 0x55d7aaa513f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:55.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.328+0000 7f7d8d400700 1 -- 172.21.15.1:0/2946624423 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7aaa5c7e0 msgr2=0x55d7aaa5ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:55.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.328+0000 7f7d8d400700 1 --2- 172.21.15.1:0/2946624423 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7aaa5c7e0 0x55d7aaa5ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:55.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.328+0000 7f7d8d400700 1 -- 172.21.15.1:0/2946624423 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7aaa51010 msgr2=0x55d7aaa513f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:55.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.328+0000 7f7d8d400700 1 --2- 172.21.15.1:0/2946624423 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7aaa51010 0x55d7aaa513f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:55.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.328+0000 7f7d8de00700 1 --2- 172.21.15.1:0/2949716896 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f7d700a0690 0x7f7d700a2b50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7d80009fd0 tx=0x7f7d8000f620 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:55.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.328+0000 7f7d8d400700 1 -- 172.21.15.1:0/2946624423 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d7aa8243b0 con 0x55d7aaa51930 2024-02-02T15:36:55.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.328+0000 7f7d8e800700 1 --2- 172.21.15.1:0/2946624423 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7aaa5c7e0 0x55d7aaa5ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:36:55.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.328+0000 7f7d8d400700 1 --2- 172.21.15.1:0/2946624423 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7aaa51930 0x55d7aaa5c2a0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f7d780990b0 tx=0x7f7d78098d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=952398dfd3873546 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:55.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.329+0000 7f7d7f400700 1 -- 172.21.15.1:0/2946624423 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d780993b0 con 0x55d7aaa51930 2024-02-02T15:36:55.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.329+0000 7f7d7f400700 1 -- 172.21.15.1:0/2946624423 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d7809d1e0 con 0x55d7aaa51930 2024-02-02T15:36:55.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.329+0000 7f7d7f400700 1 -- 172.21.15.1:0/2946624423 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d78099850 con 0x55d7aaa51930 2024-02-02T15:36:55.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.329+0000 7f7da8c55b80 1 -- 172.21.15.1:0/2946624423 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7aaa51930 msgr2=0x55d7aaa5c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:55.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.329+0000 7f7da8c55b80 1 --2- 172.21.15.1:0/2946624423 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7aaa51930 0x55d7aaa5c2a0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f7d780990b0 tx=0x7f7d78098d10 comp rx=0 tx=0).stop 2024-02-02T15:36:55.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.329+0000 7f7da8c55b80 1 -- 172.21.15.1:0/2946624423 shutdown_connections 2024-02-02T15:36:55.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.329+0000 7f7da8c55b80 1 --2- 172.21.15.1:0/2946624423 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7aaa5c7e0 0x55d7aaa5ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:55.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.329+0000 7f7da8c55b80 1 --2- 172.21.15.1:0/2946624423 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7aaa51010 0x55d7aaa513f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:55.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.329+0000 7f7da8c55b80 1 --2- 172.21.15.1:0/2946624423 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7aaa51930 0x55d7aaa5c2a0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:55.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.329+0000 7f7da8c55b80 1 -- 172.21.15.1:0/2946624423 >> 172.21.15.1:0/2946624423 conn(0x55d7aa9b0db0 msgr2=0x55d7aaa4d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:55.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.330+0000 7f7da8c55b80 1 -- 172.21.15.1:0/2946624423 shutdown_connections 2024-02-02T15:36:55.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.330+0000 7f7da8c55b80 1 -- 172.21.15.1:0/2946624423 wait complete. 2024-02-02T15:36:55.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.330+0000 7f7da8c55b80 1 -- start start 2024-02-02T15:36:55.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.330+0000 7f7da8c55b80 1 Processor -- start 2024-02-02T15:36:55.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.331+0000 7f7da8c55b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7aaa51010 0x55d7aaa5c200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:55.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.331+0000 7f7da8c55b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7aaa51930 0x55d7aaa59480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:55.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.331+0000 7f7da8c55b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7aaa5c7e0 0x55d7aaa59ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:55.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.331+0000 7f7da8c55b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7aa9a2d50 con 0x55d7aaa51010 2024-02-02T15:36:55.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.331+0000 7f7da8c55b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7aa93ed80 con 0x55d7aaa5c7e0 2024-02-02T15:36:55.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.331+0000 7f7da8c55b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7aa93ea20 con 0x55d7aaa51930 2024-02-02T15:36:55.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.331+0000 7f7da8c55b80 10 client.0 _ll_register_callbacks cb 0x55d7aa940290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:36:55.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.331+0000 7f7d8d400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7aaa51930 0x55d7aaa59480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:55.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.331+0000 7f7d8de00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7aaa51010 0x55d7aaa5c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:55.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.331+0000 7f7d8d400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7aaa51930 0x55d7aaa59480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:33974/0 (socket says 172.21.15.1:33974) 2024-02-02T15:36:55.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.331+0000 7f7d8d400700 1 -- 172.21.15.1:0/1003739593 learned_addr learned my addr 172.21.15.1:0/1003739593 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:36:55.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.331+0000 7f7d8e800700 1 --2- 172.21.15.1:0/1003739593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7aaa5c7e0 0x55d7aaa59ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:55.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.331+0000 7f7d8d400700 1 -- 172.21.15.1:0/1003739593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7aaa5c7e0 msgr2=0x55d7aaa59ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:55.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.331+0000 7f7d8d400700 1 --2- 172.21.15.1:0/1003739593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7aaa5c7e0 0x55d7aaa59ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:55.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.331+0000 7f7d8d400700 1 -- 172.21.15.1:0/1003739593 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7aaa51010 msgr2=0x55d7aaa5c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:55.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.331+0000 7f7d8d400700 1 --2- 172.21.15.1:0/1003739593 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7aaa51010 0x55d7aaa5c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:55.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.331+0000 7f7d8d400700 1 -- 172.21.15.1:0/1003739593 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d880112f0 con 0x55d7aaa51930 2024-02-02T15:36:55.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.332+0000 7f7d8d400700 1 --2- 172.21.15.1:0/1003739593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7aaa51930 0x55d7aaa59480 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f7d7800c5b0 tx=0x7f7d780107d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ebfdf2bbaaa1e0e7 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:55.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.332+0000 7f7d77e00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:36:55.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.332+0000 7f7d77e00700 1 -- 172.21.15.1:0/1003739593 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d780993b0 con 0x55d7aaa51930 2024-02-02T15:36:55.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.332+0000 7f7d77e00700 1 -- 172.21.15.1:0/1003739593 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f7d780a1420 con 0x55d7aaa51930 2024-02-02T15:36:55.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.332+0000 7f7da8c55b80 1 -- 172.21.15.1:0/1003739593 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d7aaa450f0 con 0x55d7aaa51930 2024-02-02T15:36:55.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.332+0000 7f7d77e00700 1 -- 172.21.15.1:0/1003739593 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d78099820 con 0x55d7aaa51930 2024-02-02T15:36:55.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.332+0000 7f7da8c55b80 10 client.28210 fetch_fsmap learned FSMap version 45 2024-02-02T15:36:55.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.332+0000 7f7da8c55b80 1 -- 172.21.15.1:0/1003739593 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x7f7d901aa360 con 0x55d7aaa51930 2024-02-02T15:36:55.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.332+0000 7f7d77e00700 1 -- 172.21.15.1:0/1003739593 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d7aaa450f0 con 0x55d7aaa51930 2024-02-02T15:36:55.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.332+0000 7f7d77e00700 1 -- 172.21.15.1:0/1003739593 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 45) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f7d78064f60 con 0x55d7aaa51930 2024-02-02T15:36:55.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.333+0000 7f7da8c55b80 10 client.28210 fetch_fsmap finished waiting for FSMap version 45 2024-02-02T15:36:55.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.333+0000 7f7da8c55b80 10 client.28210 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:36:55.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.333+0000 7f7da8c55b80 20 client.28210 populate_metadata read hostname 'smithi001' 2024-02-02T15:36:55.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.333+0000 7f7da8c55b80 1 --2- 172.21.15.1:0/1003739593 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55d7aaa6bd20 0x55d7aaa6e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:55.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.333+0000 7f7da8c55b80 4 client.28210 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:36:55.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.333+0000 7f7da8c55b80 1 -- 172.21.15.1:0/1003739593 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d7aa99bdd0 con 0x55d7aaa6bd20 2024-02-02T15:36:55.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.333+0000 7f7d8de00700 1 --2- 172.21.15.1:0/1003739593 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55d7aaa6bd20 0x55d7aaa6e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:55.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.333+0000 7f7d8de00700 1 --2- 172.21.15.1:0/1003739593 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55d7aaa6bd20 0x55d7aaa6e140 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b392589fcbbcba6 server_cookie=9ee3687f4c37e1e1 in_seq=0 out_seq=0 2024-02-02T15:36:55.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.333+0000 7f7d77e00700 10 client.28210 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:36:55.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.334+0000 7f7d77e00700 1 -- 172.21.15.1:0/1003739593 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55d7aa99bdd0 con 0x55d7aaa6bd20 2024-02-02T15:36:55.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.334+0000 7f7d77e00700 10 client.28210 handle_command_reply: tid=0 2024-02-02T15:36:55.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.334+0000 7f7da8c55b80 1 client.28210 shutdown 2024-02-02T15:36:55.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.334+0000 7f7da8c55b80 1 -- 172.21.15.1:0/1003739593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7aaa51930 msgr2=0x55d7aaa59480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:55.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.334+0000 7f7da8c55b80 1 --2- 172.21.15.1:0/1003739593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7aaa51930 0x55d7aaa59480 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f7d7800c5b0 tx=0x7f7d780107d0 comp rx=0 tx=0).stop 2024-02-02T15:36:55.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.334+0000 7f7da8c55b80 1 -- 172.21.15.1:0/1003739593 shutdown_connections 2024-02-02T15:36:55.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.334+0000 7f7da8c55b80 1 --2- 172.21.15.1:0/1003739593 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55d7aaa6bd20 0x55d7aaa6e140 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:55.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.334+0000 7f7da8c55b80 1 --2- 172.21.15.1:0/1003739593 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7aaa5c7e0 0x55d7aaa59ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:55.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.334+0000 7f7da8c55b80 1 --2- 172.21.15.1:0/1003739593 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7aaa51010 0x55d7aaa5c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:55.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.334+0000 7f7d77e00700 0 client.28210 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:36:55.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.334+0000 7f7da8c55b80 1 --2- 172.21.15.1:0/1003739593 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7aaa51930 0x55d7aaa59480 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:55.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.334+0000 7f7da8c55b80 1 -- 172.21.15.1:0/1003739593 >> 172.21.15.1:0/1003739593 conn(0x55d7aa9b0db0 msgr2=0x55d7aaa4cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:55.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.335+0000 7f7da8c55b80 1 -- 172.21.15.1:0/1003739593 shutdown_connections 2024-02-02T15:36:55.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.335+0000 7f7da8c55b80 1 -- 172.21.15.1:0/1003739593 wait complete. 2024-02-02T15:36:55.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.335+0000 7f7da8c55b80 20 client.28210 trim_cache size 0 max 16384 2024-02-02T15:36:55.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.352+0000 7f7da8c55b80 1 Processor -- start 2024-02-02T15:36:55.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.352+0000 7f7da8c55b80 1 -- start start 2024-02-02T15:36:55.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.352+0000 7f7da8c55b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7aaa51010 0x55d7aaa5bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:55.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.352+0000 7f7da8c55b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7aaa51930 0x55d7aaa58dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:55.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.352+0000 7f7da8c55b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7aaa5c7e0 0x55d7aaa59300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:55.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.352+0000 7f7da8c55b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7aa8266a0 con 0x55d7aaa51010 2024-02-02T15:36:55.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.352+0000 7f7da8c55b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7aaa59840 con 0x55d7aaa51930 2024-02-02T15:36:55.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.352+0000 7f7da8c55b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7aaa55370 con 0x55d7aaa5c7e0 2024-02-02T15:36:55.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.352+0000 7f7d8de00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7aaa51010 0x55d7aaa5bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:55.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.352+0000 7f7d8de00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7aaa51010 0x55d7aaa5bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48628/0 (socket says 172.21.15.1:48628) 2024-02-02T15:36:55.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.352+0000 7f7d8de00700 1 -- 172.21.15.1:0/2747574901 learned_addr learned my addr 172.21.15.1:0/2747574901 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:36:55.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.352+0000 7f7d8d400700 1 --2- 172.21.15.1:0/2747574901 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7aaa51930 0x55d7aaa58dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:55.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.353+0000 7f7d8e800700 1 --2- 172.21.15.1:0/2747574901 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7aaa5c7e0 0x55d7aaa59300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:55.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.353+0000 7f7d8d400700 1 -- 172.21.15.1:0/2747574901 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7aaa5c7e0 msgr2=0x55d7aaa59300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:55.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.353+0000 7f7d8d400700 1 --2- 172.21.15.1:0/2747574901 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7aaa5c7e0 0x55d7aaa59300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:55.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.353+0000 7f7d8d400700 1 -- 172.21.15.1:0/2747574901 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7aaa51010 msgr2=0x55d7aaa5bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:55.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.353+0000 7f7d8d400700 1 --2- 172.21.15.1:0/2747574901 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7aaa51010 0x55d7aaa5bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:55.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.353+0000 7f7d8d400700 1 -- 172.21.15.1:0/2747574901 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d7aa7d0b60 con 0x55d7aaa51930 2024-02-02T15:36:55.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.353+0000 7f7d8de00700 1 --2- 172.21.15.1:0/2747574901 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7aaa51010 0x55d7aaa5bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:36:55.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.353+0000 7f7d8d400700 1 --2- 172.21.15.1:0/2747574901 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7aaa51930 0x55d7aaa58dc0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f7d78099c60 tx=0x7f7d78095040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f50312416554a6b server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:55.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.353+0000 7f7d7fe00700 1 -- 172.21.15.1:0/2747574901 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d78098d10 con 0x55d7aaa51930 2024-02-02T15:36:55.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.353+0000 7f7d7fe00700 1 -- 172.21.15.1:0/2747574901 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d780a1420 con 0x55d7aaa51930 2024-02-02T15:36:55.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.353+0000 7f7d7fe00700 1 -- 172.21.15.1:0/2747574901 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d780b71a0 con 0x55d7aaa51930 2024-02-02T15:36:55.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.354+0000 7f7da8c55b80 1 -- 172.21.15.1:0/2747574901 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7aaa51930 msgr2=0x55d7aaa58dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:55.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.354+0000 7f7da8c55b80 1 --2- 172.21.15.1:0/2747574901 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7aaa51930 0x55d7aaa58dc0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f7d78099c60 tx=0x7f7d78095040 comp rx=0 tx=0).stop 2024-02-02T15:36:55.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.354+0000 7f7da8c55b80 1 -- 172.21.15.1:0/2747574901 shutdown_connections 2024-02-02T15:36:55.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.354+0000 7f7da8c55b80 1 --2- 172.21.15.1:0/2747574901 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7aaa5c7e0 0x55d7aaa59300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:55.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.354+0000 7f7da8c55b80 1 --2- 172.21.15.1:0/2747574901 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7aaa51010 0x55d7aaa5bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:55.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.354+0000 7f7da8c55b80 1 --2- 172.21.15.1:0/2747574901 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7aaa51930 0x55d7aaa58dc0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:55.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.354+0000 7f7da8c55b80 1 -- 172.21.15.1:0/2747574901 >> 172.21.15.1:0/2747574901 conn(0x55d7aa9b0db0 msgr2=0x55d7aaa67900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:55.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.354+0000 7f7da8c55b80 1 -- 172.21.15.1:0/2747574901 shutdown_connections 2024-02-02T15:36:55.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.354+0000 7f7da8c55b80 1 -- 172.21.15.1:0/2747574901 wait complete. 2024-02-02T15:36:55.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.354+0000 7f7da8c55b80 1 -- start start 2024-02-02T15:36:55.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.355+0000 7f7da8c55b80 1 Processor -- start 2024-02-02T15:36:55.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.355+0000 7f7da8c55b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7aaa51010 0x55d7aaa61040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:55.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.355+0000 7f7da8c55b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7aaa51930 0x55d7aaa61580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:55.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.355+0000 7f7da8c55b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7aaa5c7e0 0x55d7aaab6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:55.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.355+0000 7f7da8c55b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7aaab6d60 con 0x55d7aaa51930 2024-02-02T15:36:55.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.355+0000 7f7da8c55b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7aaab6ed0 con 0x55d7aaa51010 2024-02-02T15:36:55.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.355+0000 7f7da8c55b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7aaab7090 con 0x55d7aaa5c7e0 2024-02-02T15:36:55.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.355+0000 7f7da8c55b80 10 client.0 _ll_register_callbacks cb 0x55d7aa940290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:36:55.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.355+0000 7f7d8d400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7aaa51930 0x55d7aaa61580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:55.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.355+0000 7f7d8d400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7aaa51930 0x55d7aaa61580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48642/0 (socket says 172.21.15.1:48642) 2024-02-02T15:36:55.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.355+0000 7f7d8d400700 1 -- 172.21.15.1:0/76996456 learned_addr learned my addr 172.21.15.1:0/76996456 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:36:55.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.355+0000 7f7d8de00700 1 --2- 172.21.15.1:0/76996456 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7aaa51010 0x55d7aaa61040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:55.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.356+0000 7f7d8e800700 1 --2- 172.21.15.1:0/76996456 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7aaa5c7e0 0x55d7aaab6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:55.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.356+0000 7f7d8d400700 1 -- 172.21.15.1:0/76996456 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7aaa5c7e0 msgr2=0x55d7aaab6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:55.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.356+0000 7f7d8d400700 1 --2- 172.21.15.1:0/76996456 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7aaa5c7e0 0x55d7aaab6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:55.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.356+0000 7f7d8d400700 1 -- 172.21.15.1:0/76996456 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7aaa51010 msgr2=0x55d7aaa61040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:55.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.356+0000 7f7d8d400700 1 --2- 172.21.15.1:0/76996456 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7aaa51010 0x55d7aaa61040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:55.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.356+0000 7f7d8d400700 1 -- 172.21.15.1:0/76996456 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d8001f040 con 0x55d7aaa51930 2024-02-02T15:36:55.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.356+0000 7f7d8e800700 1 --2- 172.21.15.1:0/76996456 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7aaa5c7e0 0x55d7aaab6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:36:55.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.356+0000 7f7d8d400700 1 --2- 172.21.15.1:0/76996456 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7aaa51930 0x55d7aaa61580 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f7d780958b0 tx=0x7f7d780097b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7246d47cc7372099 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:55.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.356+0000 7f7d77e00700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:36:55.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.356+0000 7f7d77e00700 1 -- 172.21.15.1:0/76996456 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d78098d10 con 0x55d7aaa51930 2024-02-02T15:36:55.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.356+0000 7f7d77e00700 1 -- 172.21.15.1:0/76996456 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f7d780993b0 con 0x55d7aaa51930 2024-02-02T15:36:55.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.356+0000 7f7da8c55b80 1 -- 172.21.15.1:0/76996456 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7d7809d1e0 con 0x55d7aaa51930 2024-02-02T15:36:55.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.356+0000 7f7d77e00700 1 -- 172.21.15.1:0/76996456 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d78098ef0 con 0x55d7aaa51930 2024-02-02T15:36:55.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.356+0000 7f7da8c55b80 10 client.18132 fetch_fsmap learned FSMap version 45 2024-02-02T15:36:55.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.356+0000 7f7da8c55b80 1 -- 172.21.15.1:0/76996456 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x7f7d901aa360 con 0x55d7aaa51930 2024-02-02T15:36:55.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.357+0000 7f7d77e00700 1 -- 172.21.15.1:0/76996456 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7d7809d1e0 con 0x55d7aaa51930 2024-02-02T15:36:55.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.357+0000 7f7d77e00700 1 -- 172.21.15.1:0/76996456 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 45) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f7d780b1290 con 0x55d7aaa51930 2024-02-02T15:36:55.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.357+0000 7f7da8c55b80 10 client.18132 fetch_fsmap finished waiting for FSMap version 45 2024-02-02T15:36:55.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.357+0000 7f7da8c55b80 10 client.18132 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:36:55.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.357+0000 7f7da8c55b80 20 client.18132 populate_metadata read hostname 'smithi001' 2024-02-02T15:36:55.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.357+0000 7f7da8c55b80 1 --2- 172.21.15.1:0/76996456 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55d7aaabe3b0 0x55d7aaac07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:55.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.357+0000 7f7da8c55b80 4 client.18132 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2024-02-02T15:36:55.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.357+0000 7f7da8c55b80 1 -- 172.21.15.1:0/76996456 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d7aa961f50 con 0x55d7aaabe3b0 2024-02-02T15:36:55.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.357+0000 7f7d8de00700 1 --2- 172.21.15.1:0/76996456 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55d7aaabe3b0 0x55d7aaac07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:55.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.358+0000 7f7d8de00700 1 --2- 172.21.15.1:0/76996456 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55d7aaabe3b0 0x55d7aaac07d0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb1a92f7aee73765 server_cookie=60ea3de1e161f356 in_seq=0 out_seq=0 2024-02-02T15:36:55.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.358+0000 7f7d77e00700 10 client.18132 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:36:55.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.358+0000 7f7d77e00700 1 -- 172.21.15.1:0/76996456 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55d7aa961f50 con 0x55d7aaabe3b0 2024-02-02T15:36:55.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.358+0000 7f7d77e00700 10 client.18132 handle_command_reply: tid=0 2024-02-02T15:36:55.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.358+0000 7f7da8c55b80 1 client.18132 shutdown 2024-02-02T15:36:55.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.358+0000 7f7da8c55b80 1 -- 172.21.15.1:0/76996456 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7aaa51930 msgr2=0x55d7aaa61580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:55.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.358+0000 7f7da8c55b80 1 --2- 172.21.15.1:0/76996456 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7aaa51930 0x55d7aaa61580 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f7d780958b0 tx=0x7f7d780097b0 comp rx=0 tx=0).stop 2024-02-02T15:36:55.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.358+0000 7f7da8c55b80 1 -- 172.21.15.1:0/76996456 shutdown_connections 2024-02-02T15:36:55.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.358+0000 7f7da8c55b80 1 --2- 172.21.15.1:0/76996456 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55d7aaabe3b0 0x55d7aaac07d0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:55.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.358+0000 7f7da8c55b80 1 --2- 172.21.15.1:0/76996456 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7aaa5c7e0 0x55d7aaab6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:55.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.358+0000 7f7da8c55b80 1 --2- 172.21.15.1:0/76996456 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7aaa51010 0x55d7aaa61040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:55.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.358+0000 7f7d77e00700 0 client.18132 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:36:55.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.358+0000 7f7da8c55b80 1 --2- 172.21.15.1:0/76996456 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7aaa51930 0x55d7aaa61580 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:55.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.358+0000 7f7da8c55b80 1 -- 172.21.15.1:0/76996456 >> 172.21.15.1:0/76996456 conn(0x55d7aa9b0db0 msgr2=0x55d7aaa58dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:55.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.358+0000 7f7da8c55b80 1 -- 172.21.15.1:0/76996456 shutdown_connections 2024-02-02T15:36:55.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.358+0000 7f7da8c55b80 1 -- 172.21.15.1:0/76996456 wait complete. 2024-02-02T15:36:55.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.359+0000 7f7da8c55b80 20 client.18132 trim_cache size 0 max 16384 2024-02-02T15:36:55.364 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:36:55.364 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2024-02-02T15:36:55.364 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2024-02-02T15:36:55.364 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:36:55.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.359+0000 7f7d95200700 1 -- 172.21.15.1:0/2949716896 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f7d700a0690 msgr2=0x7f7d700a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:55.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.359+0000 7f7d95200700 1 --2- 172.21.15.1:0/2949716896 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f7d700a0690 0x7f7d700a2b50 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7d80009fd0 tx=0x7f7d8000f620 comp rx=0 tx=0).stop 2024-02-02T15:36:55.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.359+0000 7f7d95200700 1 -- 172.21.15.1:0/2949716896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d90073a00 msgr2=0x7f7d9019da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:55.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.359+0000 7f7d95200700 1 --2- 172.21.15.1:0/2949716896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d90073a00 0x7f7d9019da70 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f7d7800e9f0 tx=0x7f7d7800ed00 comp rx=0 tx=0).stop 2024-02-02T15:36:55.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.359+0000 7f7d95200700 1 -- 172.21.15.1:0/2949716896 shutdown_connections 2024-02-02T15:36:55.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.359+0000 7f7d95200700 1 --2- 172.21.15.1:0/2949716896 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f7d700a0690 0x7f7d700a2b50 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:55.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.359+0000 7f7d95200700 1 --2- 172.21.15.1:0/2949716896 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7d90111d90 0x7f7d901a1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:55.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.359+0000 7f7d95200700 1 --2- 172.21.15.1:0/2949716896 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d900730e0 0x7f7d9019d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:55.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.359+0000 7f7d95200700 1 --2- 172.21.15.1:0/2949716896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7d90073a00 0x7f7d9019da70 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:55.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.359+0000 7f7d95200700 1 -- 172.21.15.1:0/2949716896 >> 172.21.15.1:0/2949716896 conn(0x7f7d900fc810 msgr2=0x7f7d900fd480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:55.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.359+0000 7f7d95200700 1 -- 172.21.15.1:0/2949716896 shutdown_connections 2024-02-02T15:36:55.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.359+0000 7f7d95200700 1 -- 172.21.15.1:0/2949716896 wait complete. 2024-02-02T15:36:55.377 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-02T15:36:55.377 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T15:36:55.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.577+0000 7f3fbde00700 1 -- 172.21.15.1:0/4027999989 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3fb8100f10 msgr2=0x7f3fb81012f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:55.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.577+0000 7f3fbde00700 1 --2- 172.21.15.1:0/4027999989 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3fb8100f10 0x7f3fb81012f0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f3fa8009990 tx=0x7f3fa8009ca0 comp rx=0 tx=0).stop 2024-02-02T15:36:55.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.577+0000 7f3fbde00700 1 -- 172.21.15.1:0/4027999989 shutdown_connections 2024-02-02T15:36:55.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.577+0000 7f3fbde00700 1 --2- 172.21.15.1:0/4027999989 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3fb8106030 0x7f3fb810dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:55.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.577+0000 7f3fbde00700 1 --2- 172.21.15.1:0/4027999989 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3fb8100f10 0x7f3fb81012f0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:55.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.577+0000 7f3fbde00700 1 --2- 172.21.15.1:0/4027999989 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fb81018c0 0x7f3fb8105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:55.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.577+0000 7f3fbde00700 1 -- 172.21.15.1:0/4027999989 >> 172.21.15.1:0/4027999989 conn(0x7f3fb80fc790 msgr2=0x7f3fb80febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:55.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.577+0000 7f3fbde00700 1 -- 172.21.15.1:0/4027999989 shutdown_connections 2024-02-02T15:36:55.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.577+0000 7f3fbde00700 1 -- 172.21.15.1:0/4027999989 wait complete. 2024-02-02T15:36:55.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.578+0000 7f3fbde00700 1 Processor -- start 2024-02-02T15:36:55.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.578+0000 7f3fbde00700 1 -- start start 2024-02-02T15:36:55.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.578+0000 7f3fbde00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3fb8100f10 0x7f3fb8198fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:55.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.578+0000 7f3fbde00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fb8106030 0x7f3fb8199520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:55.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.578+0000 7f3fbde00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3fb819d980 0x7f3fb819de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:55.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.578+0000 7f3fbde00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb819e400 con 0x7f3fb8100f10 2024-02-02T15:36:55.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.579+0000 7f3fbde00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb819e570 con 0x7f3fb819d980 2024-02-02T15:36:55.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.579+0000 7f3fbde00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fb819e730 con 0x7f3fb8106030 2024-02-02T15:36:55.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.579+0000 7f3fb6a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3fb8100f10 0x7f3fb8198fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:55.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.579+0000 7f3fb7400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3fb819d980 0x7f3fb819de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:55.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.579+0000 7f3fb6000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fb8106030 0x7f3fb8199520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:55.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.579+0000 7f3fb6a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3fb8100f10 0x7f3fb8198fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48656/0 (socket says 172.21.15.1:48656) 2024-02-02T15:36:55.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.579+0000 7f3fb7400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3fb819d980 0x7f3fb819de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59198/0 (socket says 172.21.15.1:59198) 2024-02-02T15:36:55.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.579+0000 7f3fb6a00700 1 -- 172.21.15.1:0/2716882558 learned_addr learned my addr 172.21.15.1:0/2716882558 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:36:55.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.579+0000 7f3fb7400700 1 -- 172.21.15.1:0/2716882558 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fb8106030 msgr2=0x7f3fb8199520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:55.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.579+0000 7f3fb7400700 1 --2- 172.21.15.1:0/2716882558 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fb8106030 0x7f3fb8199520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:55.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.579+0000 7f3fb7400700 1 -- 172.21.15.1:0/2716882558 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3fb8100f10 msgr2=0x7f3fb8198fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:55.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.579+0000 7f3fb7400700 1 --2- 172.21.15.1:0/2716882558 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3fb8100f10 0x7f3fb8198fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:55.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.579+0000 7f3fb7400700 1 -- 172.21.15.1:0/2716882558 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fa80095f0 con 0x7f3fb819d980 2024-02-02T15:36:55.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.579+0000 7f3fb7400700 1 --2- 172.21.15.1:0/2716882558 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3fb819d980 0x7f3fb819de00 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f3fb000efc0 tx=0x7f3fb000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:55.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.580+0000 7f3fbca00700 1 -- 172.21.15.1:0/2716882558 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb0018070 con 0x7f3fb819d980 2024-02-02T15:36:55.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.580+0000 7f3fbde00700 1 -- 172.21.15.1:0/2716882558 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fb81a5c30 con 0x7f3fb819d980 2024-02-02T15:36:55.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.580+0000 7f3fbca00700 1 -- 172.21.15.1:0/2716882558 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3fb000f040 con 0x7f3fb819d980 2024-02-02T15:36:55.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.580+0000 7f3fbca00700 1 -- 172.21.15.1:0/2716882558 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb000cd40 con 0x7f3fb819d980 2024-02-02T15:36:55.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.580+0000 7f3fbde00700 1 -- 172.21.15.1:0/2716882558 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fb81a6160 con 0x7f3fb819d980 2024-02-02T15:36:55.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.581+0000 7f3fbde00700 1 -- 172.21.15.1:0/2716882558 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fb8067bc0 con 0x7f3fb819d980 2024-02-02T15:36:55.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.582+0000 7f3fbca00700 1 -- 172.21.15.1:0/2716882558 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f3fb0003a00 con 0x7f3fb819d980 2024-02-02T15:36:55.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.584+0000 7f3fbca00700 1 --2- 172.21.15.1:0/2716882558 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f3f980a0710 0x7f3f980a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:55.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.584+0000 7f3fbca00700 1 -- 172.21.15.1:0/2716882558 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f3fb00cbbd0 con 0x7f3fb819d980 2024-02-02T15:36:55.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.584+0000 7f3fb6a00700 1 --2- 172.21.15.1:0/2716882558 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f3f980a0710 0x7f3f980a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:55.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.585+0000 7f3fb6a00700 1 --2- 172.21.15.1:0/2716882558 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f3f980a0710 0x7f3f980a2bd0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3fa8006010 tx=0x7f3fa800b260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:55.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.588+0000 7f3fbca00700 1 -- 172.21.15.1:0/2716882558 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f3fb00950c0 con 0x7f3fb819d980 2024-02-02T15:36:55.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:55 smithi001 ceph-mon[35470]: pgmap v306: 129 pgs: 129 active+clean; 435 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 116 op/s 2024-02-02T15:36:55.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.852+0000 7f3fbde00700 1 -- 172.21.15.1:0/2716882558 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3fb819a330 con 0x7f3fb819d980 2024-02-02T15:36:55.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.853+0000 7f3fbca00700 1 -- 172.21.15.1:0/2716882558 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) v1 ==== 94+0+10881 (secure 0 0 0) 0x7f3fb001b320 con 0x7f3fb819d980 2024-02-02T15:36:55.853 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:36:55.853 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":45,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2922240448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2922240448},{"type":"v1","addr":"172.21.15.1:6835","nonce":2922240448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2729497431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2729497431},{"type":"v1","addr":"172.21.15.1:6837","nonce":2729497431}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":45,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-02T15:24:47.761435+0000","modified":"2024-02-02T15:36:49.471896+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24529,"mds_2":24518,"mds_3":24514,"mds_4":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14517":{"gid":14517,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/1218303683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":1218303683},{"type":"v1","addr":"172.21.15.1:6839","nonce":1218303683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14529":{"gid":14529,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/2059197496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2059197496},{"type":"v1","addr":"172.21.15.1:6841","nonce":2059197496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/7648986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":7648986},{"type":"v1","addr":"172.21.15.142:6833","nonce":7648986}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/3482746094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3482746094},{"type":"v1","addr":"172.21.15.71:6833","nonce":3482746094}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/388207814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":388207814},{"type":"v1","addr":"172.21.15.71:6835","nonce":388207814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6835/1052606547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1052606547},{"type":"v1","addr":"172.21.15.142:6835","nonce":1052606547}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.142:6837/2870289019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2870289019},{"type":"v1","addr":"172.21.15.142:6837","nonce":2870289019}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.71:6837/628016199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":628016199},{"type":"v1","addr":"172.21.15.71:6837","nonce":628016199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.142:6839/321675421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":321675421},{"type":"v1","addr":"172.21.15.142:6839","nonce":321675421}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/1268086493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1268086493},{"type":"v1","addr":"172.21.15.71:6839","nonce":1268086493}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-02T15:36:55.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.855+0000 7f3fbde00700 1 -- 172.21.15.1:0/2716882558 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f3f980a0710 msgr2=0x7f3f980a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:55.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.855+0000 7f3fbde00700 1 --2- 172.21.15.1:0/2716882558 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f3f980a0710 0x7f3f980a2bd0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3fa8006010 tx=0x7f3fa800b260 comp rx=0 tx=0).stop 2024-02-02T15:36:55.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.855+0000 7f3fbde00700 1 -- 172.21.15.1:0/2716882558 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3fb819d980 msgr2=0x7f3fb819de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:55.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.855+0000 7f3fbde00700 1 --2- 172.21.15.1:0/2716882558 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3fb819d980 0x7f3fb819de00 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f3fb000efc0 tx=0x7f3fb000c3a0 comp rx=0 tx=0).stop 2024-02-02T15:36:55.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.855+0000 7f3fbde00700 1 -- 172.21.15.1:0/2716882558 shutdown_connections 2024-02-02T15:36:55.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.855+0000 7f3fbde00700 1 --2- 172.21.15.1:0/2716882558 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f3f980a0710 0x7f3f980a2bd0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:55.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.855+0000 7f3fbde00700 1 --2- 172.21.15.1:0/2716882558 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3fb819d980 0x7f3fb819de00 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:55.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.855+0000 7f3fbde00700 1 --2- 172.21.15.1:0/2716882558 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3fb8100f10 0x7f3fb8198fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:55.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.855+0000 7f3fbde00700 1 --2- 172.21.15.1:0/2716882558 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fb8106030 0x7f3fb8199520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:55.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.855+0000 7f3fbde00700 1 -- 172.21.15.1:0/2716882558 >> 172.21.15.1:0/2716882558 conn(0x7f3fb80fc790 msgr2=0x7f3fb81028d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:55.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.855+0000 7f3fbde00700 1 -- 172.21.15.1:0/2716882558 shutdown_connections 2024-02-02T15:36:55.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:55.855+0000 7f3fbde00700 1 -- 172.21.15.1:0/2716882558 wait complete. 2024-02-02T15:36:55.857 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 45 2024-02-02T15:36:55.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:55 smithi071 ceph-mon[44707]: pgmap v306: 129 pgs: 129 active+clean; 435 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 116 op/s 2024-02-02T15:36:55.874 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-02-02T15:36:55.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:55 smithi142 ceph-mon[45728]: pgmap v306: 129 pgs: 129 active+clean; 435 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 116 op/s 2024-02-02T15:36:56.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.100+0000 7ffadce00700 1 -- 172.21.15.1:0/167929390 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffad8073a00 msgr2=0x7ffad8111850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:56.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.100+0000 7ffadce00700 1 --2- 172.21.15.1:0/167929390 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffad8073a00 0x7ffad8111850 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7ffac00098f0 tx=0x7ffac0009c00 comp rx=0 tx=0).stop 2024-02-02T15:36:56.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.100+0000 7ffadce00700 1 -- 172.21.15.1:0/167929390 shutdown_connections 2024-02-02T15:36:56.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.100+0000 7ffadce00700 1 --2- 172.21.15.1:0/167929390 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffad8111d90 0x7ffad8114180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.100+0000 7ffadce00700 1 --2- 172.21.15.1:0/167929390 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffad80730e0 0x7ffad80734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.100+0000 7ffadce00700 1 --2- 172.21.15.1:0/167929390 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffad8073a00 0x7ffad8111850 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.100+0000 7ffadce00700 1 -- 172.21.15.1:0/167929390 >> 172.21.15.1:0/167929390 conn(0x7ffad80fc890 msgr2=0x7ffad80fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:56.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.100+0000 7ffadce00700 1 -- 172.21.15.1:0/167929390 shutdown_connections 2024-02-02T15:36:56.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.100+0000 7ffadce00700 1 -- 172.21.15.1:0/167929390 wait complete. 2024-02-02T15:36:56.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.101+0000 7ffadce00700 1 Processor -- start 2024-02-02T15:36:56.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.101+0000 7ffadce00700 1 -- start start 2024-02-02T15:36:56.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.101+0000 7ffadce00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffad80730e0 0x7ffad819d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:56.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.101+0000 7ffadce00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffad8073a00 0x7ffad819da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:56.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.101+0000 7ffadce00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffad8111d90 0x7ffad81a1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:56.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.101+0000 7ffadce00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad81a2510 con 0x7ffad8111d90 2024-02-02T15:36:56.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.101+0000 7ffadce00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad81a2680 con 0x7ffad80730e0 2024-02-02T15:36:56.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.101+0000 7ffadce00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad81a2840 con 0x7ffad8073a00 2024-02-02T15:36:56.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.102+0000 7ffad5e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffad80730e0 0x7ffad819d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:56.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.102+0000 7ffad5400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffad8073a00 0x7ffad819da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:56.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.102+0000 7ffad5e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffad80730e0 0x7ffad819d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59218/0 (socket says 172.21.15.1:59218) 2024-02-02T15:36:56.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.102+0000 7ffad5e00700 1 -- 172.21.15.1:0/1128087869 learned_addr learned my addr 172.21.15.1:0/1128087869 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:36:56.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.102+0000 7ffad6800700 1 --2- 172.21.15.1:0/1128087869 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffad8111d90 0x7ffad81a1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:56.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.102+0000 7ffad5400700 1 -- 172.21.15.1:0/1128087869 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffad80730e0 msgr2=0x7ffad819d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:56.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.102+0000 7ffad5400700 1 --2- 172.21.15.1:0/1128087869 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffad80730e0 0x7ffad819d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.102+0000 7ffad5400700 1 -- 172.21.15.1:0/1128087869 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffad8111d90 msgr2=0x7ffad81a1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:56.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.102+0000 7ffad5400700 1 --2- 172.21.15.1:0/1128087869 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffad8111d90 0x7ffad81a1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.102+0000 7ffad5400700 1 -- 172.21.15.1:0/1128087869 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffac80095f0 con 0x7ffad8073a00 2024-02-02T15:36:56.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.102+0000 7ffad5400700 1 --2- 172.21.15.1:0/1128087869 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffad8073a00 0x7ffad819da70 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7ffac0009810 tx=0x7ffac000fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:56.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.103+0000 7ffacea00700 1 -- 172.21.15.1:0/1128087869 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac001c070 con 0x7ffad8073a00 2024-02-02T15:36:56.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.103+0000 7ffacea00700 1 -- 172.21.15.1:0/1128087869 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffac000bba0 con 0x7ffad8073a00 2024-02-02T15:36:56.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.103+0000 7ffacea00700 1 -- 172.21.15.1:0/1128087869 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac0003970 con 0x7ffad8073a00 2024-02-02T15:36:56.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.103+0000 7ffadce00700 1 -- 172.21.15.1:0/1128087869 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffac0009520 con 0x7ffad8073a00 2024-02-02T15:36:56.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.103+0000 7ffadce00700 1 -- 172.21.15.1:0/1128087869 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffad81aa180 con 0x7ffad8073a00 2024-02-02T15:36:56.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.106+0000 7ffacea00700 1 -- 172.21.15.1:0/1128087869 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7ffac0020030 con 0x7ffad8073a00 2024-02-02T15:36:56.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.107+0000 7ffacea00700 1 --2- 172.21.15.1:0/1128087869 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ffab80a0710 0x7ffab80a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:56.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.108+0000 7ffaf086fb80 1 Processor -- start 2024-02-02T15:36:56.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.108+0000 7ffacea00700 1 -- 172.21.15.1:0/1128087869 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9453+0+0 (secure 0 0 0) 0x7ffac00ccec0 con 0x7ffad8073a00 2024-02-02T15:36:56.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.108+0000 7ffad5e00700 1 --2- 172.21.15.1:0/1128087869 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ffab80a0710 0x7ffab80a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:56.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.108+0000 7ffaf086fb80 1 -- start start 2024-02-02T15:36:56.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.108+0000 7ffaf086fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55affd409010 0x55affd4093f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:56.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.108+0000 7ffaf086fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55affd409930 0x55affd4142a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:56.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.108+0000 7ffaf086fb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55affd4147e0 0x55affd416bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:56.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.108+0000 7ffaf086fb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55affd37d4c0 con 0x55affd4147e0 2024-02-02T15:36:56.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.108+0000 7ffaf086fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55affd361aa0 con 0x55affd409010 2024-02-02T15:36:56.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.109+0000 7ffaf086fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55affd35f460 con 0x55affd409930 2024-02-02T15:36:56.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.109+0000 7ffad5e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55affd409010 0x55affd4093f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:56.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.109+0000 7ffad5e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55affd409010 0x55affd4093f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59226/0 (socket says 172.21.15.1:59226) 2024-02-02T15:36:56.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.109+0000 7ffad5e00700 1 -- 172.21.15.1:0/462278339 learned_addr learned my addr 172.21.15.1:0/462278339 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:36:56.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.109+0000 7ffad5e00700 1 --2- 172.21.15.1:0/1128087869 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ffab80a0710 0x7ffab80a2bd0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ffac8005d10 tx=0x7ffac8009310 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:56.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.109+0000 7ffad5400700 1 --2- 172.21.15.1:0/462278339 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55affd409930 0x55affd4142a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:56.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.109+0000 7ffad6800700 1 --2- 172.21.15.1:0/462278339 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55affd4147e0 0x55affd416bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:56.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.109+0000 7ffad5e00700 1 -- 172.21.15.1:0/462278339 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55affd409930 msgr2=0x55affd4142a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:56.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.109+0000 7ffad5e00700 1 --2- 172.21.15.1:0/462278339 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55affd409930 0x55affd4142a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.109+0000 7ffad5e00700 1 -- 172.21.15.1:0/462278339 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55affd4147e0 msgr2=0x55affd416bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:56.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.109+0000 7ffad5e00700 1 --2- 172.21.15.1:0/462278339 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55affd4147e0 0x55affd416bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.109+0000 7ffad5e00700 1 -- 172.21.15.1:0/462278339 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55affd1dc3b0 con 0x55affd409010 2024-02-02T15:36:56.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.109+0000 7ffad5e00700 1 --2- 172.21.15.1:0/462278339 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55affd409010 0x55affd4093f0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7ffac800e660 tx=0x7ffac8014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=454fa9e84cfa22c1 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:56.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.110+0000 7ffac7400700 1 -- 172.21.15.1:0/462278339 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac8026430 con 0x55affd409010 2024-02-02T15:36:56.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.110+0000 7ffac7400700 1 -- 172.21.15.1:0/462278339 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffac8021070 con 0x55affd409010 2024-02-02T15:36:56.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.110+0000 7ffac7400700 1 -- 172.21.15.1:0/462278339 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac80258b0 con 0x55affd409010 2024-02-02T15:36:56.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.110+0000 7ffaf086fb80 1 -- 172.21.15.1:0/462278339 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55affd409010 msgr2=0x55affd4093f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:56.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.110+0000 7ffaf086fb80 1 --2- 172.21.15.1:0/462278339 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55affd409010 0x55affd4093f0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7ffac800e660 tx=0x7ffac8014000 comp rx=0 tx=0).stop 2024-02-02T15:36:56.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.111+0000 7ffaf086fb80 1 -- 172.21.15.1:0/462278339 shutdown_connections 2024-02-02T15:36:56.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.111+0000 7ffaf086fb80 1 --2- 172.21.15.1:0/462278339 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55affd4147e0 0x55affd416bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.111+0000 7ffaf086fb80 1 --2- 172.21.15.1:0/462278339 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55affd409010 0x55affd4093f0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.111+0000 7ffaf086fb80 1 --2- 172.21.15.1:0/462278339 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55affd409930 0x55affd4142a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.111+0000 7ffaf086fb80 1 -- 172.21.15.1:0/462278339 >> 172.21.15.1:0/462278339 conn(0x55affd368db0 msgr2=0x55affd4053b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:56.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.111+0000 7ffaf086fb80 1 -- 172.21.15.1:0/462278339 shutdown_connections 2024-02-02T15:36:56.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.111+0000 7ffaf086fb80 1 -- 172.21.15.1:0/462278339 wait complete. 2024-02-02T15:36:56.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.111+0000 7ffaf086fb80 1 -- start start 2024-02-02T15:36:56.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.111+0000 7ffaf086fb80 1 Processor -- start 2024-02-02T15:36:56.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.112+0000 7ffaf086fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55affd409010 0x55affd414190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:56.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.112+0000 7ffaf086fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55affd409930 0x55affd4113a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:56.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.112+0000 7ffaf086fb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55affd4147e0 0x55affd411960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:56.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.112+0000 7ffaf086fb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55affd35ad50 con 0x55affd4147e0 2024-02-02T15:36:56.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.112+0000 7ffaf086fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55affd2f6d80 con 0x55affd409930 2024-02-02T15:36:56.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.112+0000 7ffaf086fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55affd2f6a20 con 0x55affd409010 2024-02-02T15:36:56.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.112+0000 7ffaf086fb80 10 client.0 _ll_register_callbacks cb 0x55affd2f8290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:36:56.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.112+0000 7ffad5e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55affd409010 0x55affd414190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:56.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.112+0000 7ffad6800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55affd4147e0 0x55affd411960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:56.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.112+0000 7ffad5e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55affd409010 0x55affd414190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:34048/0 (socket says 172.21.15.1:34048) 2024-02-02T15:36:56.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.112+0000 7ffad5e00700 1 -- 172.21.15.1:0/3281944448 learned_addr learned my addr 172.21.15.1:0/3281944448 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:36:56.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.113+0000 7ffad5e00700 1 -- 172.21.15.1:0/3281944448 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55affd409930 msgr2=0x55affd4113a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-02T15:36:56.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.113+0000 7ffad5e00700 1 --2- 172.21.15.1:0/3281944448 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55affd409930 0x55affd4113a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.113+0000 7ffad5e00700 1 -- 172.21.15.1:0/3281944448 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55affd4147e0 msgr2=0x55affd411960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:56.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.113+0000 7ffad5e00700 1 --2- 172.21.15.1:0/3281944448 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55affd4147e0 0x55affd411960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.113+0000 7ffad5e00700 1 -- 172.21.15.1:0/3281944448 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffac8017070 con 0x55affd409010 2024-02-02T15:36:56.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.113+0000 7ffad5e00700 1 --2- 172.21.15.1:0/3281944448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55affd409010 0x55affd414190 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7ffac800ec50 tx=0x7ffac800ee00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=784ea3bc84868da3 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:56.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.113+0000 7ffabfe00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:36:56.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.113+0000 7ffabfe00700 1 -- 172.21.15.1:0/3281944448 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac8005410 con 0x55affd409010 2024-02-02T15:36:56.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.113+0000 7ffabfe00700 1 -- 172.21.15.1:0/3281944448 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9453+0+0 (secure 0 0 0) 0x7ffac8005570 con 0x55affd409010 2024-02-02T15:36:56.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.113+0000 7ffaf086fb80 1 -- 172.21.15.1:0/3281944448 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55affd3fd0f0 con 0x55affd409010 2024-02-02T15:36:56.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.114+0000 7ffabfe00700 1 -- 172.21.15.1:0/3281944448 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffac8004750 con 0x55affd409010 2024-02-02T15:36:56.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.114+0000 7ffaf086fb80 10 client.28234 fetch_fsmap learned FSMap version 45 2024-02-02T15:36:56.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.114+0000 7ffaf086fb80 1 -- 172.21.15.1:0/3281944448 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x55affd188b60 con 0x55affd409010 2024-02-02T15:36:56.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.114+0000 7ffabfe00700 1 -- 172.21.15.1:0/3281944448 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55affd3fd0f0 con 0x55affd409010 2024-02-02T15:36:56.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.114+0000 7ffabfe00700 1 -- 172.21.15.1:0/3281944448 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 45) v1 ==== 7506+0+0 (secure 0 0 0) 0x7ffac802d070 con 0x55affd409010 2024-02-02T15:36:56.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.114+0000 7ffaf086fb80 10 client.28234 fetch_fsmap finished waiting for FSMap version 45 2024-02-02T15:36:56.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.114+0000 7ffaf086fb80 10 client.28234 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-02-02T15:36:56.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.114+0000 7ffaf086fb80 20 client.28234 populate_metadata read hostname 'smithi001' 2024-02-02T15:36:56.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.114+0000 7ffaf086fb80 1 --2- 172.21.15.1:0/3281944448 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55affd424090 0x55affd426480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:56.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.114+0000 7ffaf086fb80 4 client.28234 mds_command: new command op to 24502 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:36:56.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.114+0000 7ffaf086fb80 1 -- 172.21.15.1:0/3281944448 --> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55affd353dd0 con 0x55affd424090 2024-02-02T15:36:56.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.114+0000 7ffad5400700 1 --2- 172.21.15.1:0/3281944448 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55affd424090 0x55affd426480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:56.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.115+0000 7ffabfe00700 10 client.28234 ms_handle_connect on v2:172.21.15.142:6834/1052606547 2024-02-02T15:36:56.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.115+0000 7ffad5400700 1 --2- 172.21.15.1:0/3281944448 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55affd424090 0x55affd426480 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1a8553d0646e48a0 server_cookie=f2ccf8509254c3f5 in_seq=0 out_seq=0 2024-02-02T15:36:56.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.115+0000 7ffabfe00700 1 -- 172.21.15.1:0/3281944448 <== mds.4 v2:172.21.15.142:6834/1052606547 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55affd353dd0 con 0x55affd424090 2024-02-02T15:36:56.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.115+0000 7ffabfe00700 10 client.28234 handle_command_reply: tid=0 2024-02-02T15:36:56.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.115+0000 7ffaf086fb80 1 client.28234 shutdown 2024-02-02T15:36:56.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.116+0000 7ffaf086fb80 1 -- 172.21.15.1:0/3281944448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55affd409010 msgr2=0x55affd414190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:56.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.116+0000 7ffaf086fb80 1 --2- 172.21.15.1:0/3281944448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55affd409010 0x55affd414190 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7ffac800ec50 tx=0x7ffac800ee00 comp rx=0 tx=0).stop 2024-02-02T15:36:56.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.116+0000 7ffaf086fb80 1 -- 172.21.15.1:0/3281944448 shutdown_connections 2024-02-02T15:36:56.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.116+0000 7ffaf086fb80 1 --2- 172.21.15.1:0/3281944448 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55affd424090 0x55affd426480 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.116+0000 7ffaf086fb80 1 --2- 172.21.15.1:0/3281944448 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55affd4147e0 0x55affd411960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.116+0000 7ffaf086fb80 1 --2- 172.21.15.1:0/3281944448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55affd409010 0x55affd414190 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.116+0000 7ffaf086fb80 1 --2- 172.21.15.1:0/3281944448 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55affd409930 0x55affd4113a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.116+0000 7ffabfe00700 0 client.28234 ms_handle_reset on v2:172.21.15.142:6834/1052606547 2024-02-02T15:36:56.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.116+0000 7ffaf086fb80 1 -- 172.21.15.1:0/3281944448 >> 172.21.15.1:0/3281944448 conn(0x55affd368db0 msgr2=0x55affd404fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:56.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.116+0000 7ffaf086fb80 1 -- 172.21.15.1:0/3281944448 shutdown_connections 2024-02-02T15:36:56.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.116+0000 7ffaf086fb80 1 -- 172.21.15.1:0/3281944448 wait complete. 2024-02-02T15:36:56.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.116+0000 7ffaf086fb80 20 client.28234 trim_cache size 0 max 16384 2024-02-02T15:36:56.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.130+0000 7ffaf086fb80 1 Processor -- start 2024-02-02T15:36:56.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.131+0000 7ffaf086fb80 1 -- start start 2024-02-02T15:36:56.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.131+0000 7ffaf086fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55affd409010 0x55affd411940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:56.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.131+0000 7ffaf086fb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55affd409930 0x55affd40bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:56.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.131+0000 7ffaf086fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55affd4147e0 0x55affd413cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:56.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.131+0000 7ffaf086fb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55affd1de6a0 con 0x55affd409930 2024-02-02T15:36:56.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.131+0000 7ffaf086fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55affd40cfb0 con 0x55affd4147e0 2024-02-02T15:36:56.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.131+0000 7ffaf086fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55affd4093f0 con 0x55affd409010 2024-02-02T15:36:56.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.131+0000 7ffad5e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55affd409010 0x55affd411940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:56.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.131+0000 7ffad5e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55affd409010 0x55affd411940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:34052/0 (socket says 172.21.15.1:34052) 2024-02-02T15:36:56.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.131+0000 7ffad5e00700 1 -- 172.21.15.1:0/1601125014 learned_addr learned my addr 172.21.15.1:0/1601125014 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:36:56.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.131+0000 7ffad6800700 1 --2- 172.21.15.1:0/1601125014 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55affd4147e0 0x55affd413cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:56.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.131+0000 7ffad5400700 1 --2- 172.21.15.1:0/1601125014 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55affd409930 0x55affd40bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:56.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.132+0000 7ffad5e00700 1 -- 172.21.15.1:0/1601125014 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55affd4147e0 msgr2=0x55affd413cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:56.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.132+0000 7ffad5e00700 1 --2- 172.21.15.1:0/1601125014 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55affd4147e0 0x55affd413cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.132+0000 7ffad5e00700 1 -- 172.21.15.1:0/1601125014 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55affd409930 msgr2=0x55affd40bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:56.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.132+0000 7ffad5e00700 1 --2- 172.21.15.1:0/1601125014 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55affd409930 0x55affd40bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.132+0000 7ffad5e00700 1 -- 172.21.15.1:0/1601125014 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55affd320ea0 con 0x55affd409010 2024-02-02T15:36:56.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.132+0000 7ffad5400700 1 --2- 172.21.15.1:0/1601125014 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55affd409930 0x55affd40bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:36:56.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.132+0000 7ffad5e00700 1 --2- 172.21.15.1:0/1601125014 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55affd409010 0x55affd411940 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7ffac8025440 tx=0x7ffac800ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=574f4a9950197d07 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:56.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.132+0000 7ffac7e00700 1 -- 172.21.15.1:0/1601125014 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac800ec80 con 0x55affd409010 2024-02-02T15:36:56.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.132+0000 7ffac7e00700 1 -- 172.21.15.1:0/1601125014 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffac8009380 con 0x55affd409010 2024-02-02T15:36:56.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.132+0000 7ffac7e00700 1 -- 172.21.15.1:0/1601125014 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac8025ba0 con 0x55affd409010 2024-02-02T15:36:56.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.133+0000 7ffaf086fb80 1 -- 172.21.15.1:0/1601125014 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55affd409010 msgr2=0x55affd411940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:56.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.133+0000 7ffaf086fb80 1 --2- 172.21.15.1:0/1601125014 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55affd409010 0x55affd411940 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7ffac8025440 tx=0x7ffac800ef20 comp rx=0 tx=0).stop 2024-02-02T15:36:56.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.133+0000 7ffaf086fb80 1 -- 172.21.15.1:0/1601125014 shutdown_connections 2024-02-02T15:36:56.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.133+0000 7ffaf086fb80 1 --2- 172.21.15.1:0/1601125014 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55affd4147e0 0x55affd413cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.133+0000 7ffaf086fb80 1 --2- 172.21.15.1:0/1601125014 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55affd409010 0x55affd411940 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.133+0000 7ffaf086fb80 1 --2- 172.21.15.1:0/1601125014 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55affd409930 0x55affd40bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.133+0000 7ffaf086fb80 1 -- 172.21.15.1:0/1601125014 >> 172.21.15.1:0/1601125014 conn(0x55affd368db0 msgr2=0x55affd4204e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:56.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.133+0000 7ffaf086fb80 1 -- 172.21.15.1:0/1601125014 shutdown_connections 2024-02-02T15:36:56.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.133+0000 7ffaf086fb80 1 -- 172.21.15.1:0/1601125014 wait complete. 2024-02-02T15:36:56.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.134+0000 7ffaf086fb80 1 -- start start 2024-02-02T15:36:56.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.134+0000 7ffaf086fb80 1 Processor -- start 2024-02-02T15:36:56.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.134+0000 7ffaf086fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55affd409010 0x55affd40efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:56.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.134+0000 7ffaf086fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55affd409930 0x55affd419310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:56.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.134+0000 7ffaf086fb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55affd4147e0 0x55affd419850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:56.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.134+0000 7ffaf086fb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55affd419f60 con 0x55affd4147e0 2024-02-02T15:36:56.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.134+0000 7ffaf086fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55affd41a0d0 con 0x55affd409010 2024-02-02T15:36:56.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.134+0000 7ffaf086fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55affd474b80 con 0x55affd409930 2024-02-02T15:36:56.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.134+0000 7ffaf086fb80 10 client.0 _ll_register_callbacks cb 0x55affd2f8290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:36:56.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.135+0000 7ffad5e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55affd409010 0x55affd40efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:56.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.135+0000 7ffad5400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55affd409930 0x55affd419310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:56.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.135+0000 7ffad6800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55affd4147e0 0x55affd419850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:56.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.135+0000 7ffad5400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55affd409930 0x55affd419310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:34062/0 (socket says 172.21.15.1:34062) 2024-02-02T15:36:56.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.135+0000 7ffad5400700 1 -- 172.21.15.1:0/3348881 learned_addr learned my addr 172.21.15.1:0/3348881 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:36:56.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.135+0000 7ffad5e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55affd409010 0x55affd40efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59240/0 (socket says 172.21.15.1:59240) 2024-02-02T15:36:56.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.135+0000 7ffad5400700 1 -- 172.21.15.1:0/3348881 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55affd409010 msgr2=0x55affd40efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:56.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.135+0000 7ffad5400700 1 --2- 172.21.15.1:0/3348881 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55affd409010 0x55affd40efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.135+0000 7ffad5400700 1 -- 172.21.15.1:0/3348881 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55affd4147e0 msgr2=0x55affd419850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:56.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.135+0000 7ffad5400700 1 --2- 172.21.15.1:0/3348881 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55affd4147e0 0x55affd419850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.135+0000 7ffad5400700 1 -- 172.21.15.1:0/3348881 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55affd188b60 con 0x55affd409930 2024-02-02T15:36:56.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.135+0000 7ffad5400700 1 --2- 172.21.15.1:0/3348881 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55affd409930 0x55affd419310 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7ffac001f040 tx=0x7ffac0009f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a3e0b2d55a9ed69 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:56.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.135+0000 7ffabfe00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:36:56.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.135+0000 7ffabfe00700 1 -- 172.21.15.1:0/3348881 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac009a070 con 0x55affd409930 2024-02-02T15:36:56.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.136+0000 7ffabfe00700 1 -- 172.21.15.1:0/3348881 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9453+0+0 (secure 0 0 0) 0x7ffac009dea0 con 0x55affd409930 2024-02-02T15:36:56.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.136+0000 7ffaf086fb80 1 -- 172.21.15.1:0/3348881 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffac8021070 con 0x55affd409930 2024-02-02T15:36:56.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.136+0000 7ffabfe00700 1 -- 172.21.15.1:0/3348881 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffac00a2050 con 0x55affd409930 2024-02-02T15:36:56.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.136+0000 7ffaf086fb80 10 client.28240 fetch_fsmap learned FSMap version 45 2024-02-02T15:36:56.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.136+0000 7ffaf086fb80 1 -- 172.21.15.1:0/3348881 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x55affd1e0cd0 con 0x55affd409930 2024-02-02T15:36:56.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.136+0000 7ffabfe00700 1 -- 172.21.15.1:0/3348881 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffac8021070 con 0x55affd409930 2024-02-02T15:36:56.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.136+0000 7ffabfe00700 1 -- 172.21.15.1:0/3348881 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 45) v1 ==== 7506+0+0 (secure 0 0 0) 0x7ffac00ca490 con 0x55affd409930 2024-02-02T15:36:56.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.136+0000 7ffaf086fb80 10 client.28240 fetch_fsmap finished waiting for FSMap version 45 2024-02-02T15:36:56.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.136+0000 7ffaf086fb80 10 client.28240 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-02-02T15:36:56.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.136+0000 7ffaf086fb80 20 client.28240 populate_metadata read hostname 'smithi001' 2024-02-02T15:36:56.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.136+0000 7ffaf086fb80 1 --2- 172.21.15.1:0/3348881 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55affd475eb0 0x55affd4782d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:56.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.136+0000 7ffaf086fb80 4 client.28240 mds_command: new command op to 24502 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:36:56.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.136+0000 7ffaf086fb80 1 -- 172.21.15.1:0/3348881 --> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55affd319f50 con 0x55affd475eb0 2024-02-02T15:36:56.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.137+0000 7ffad5e00700 1 --2- 172.21.15.1:0/3348881 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55affd475eb0 0x55affd4782d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:56.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.137+0000 7ffad5e00700 1 --2- 172.21.15.1:0/3348881 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55affd475eb0 0x55affd4782d0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=103db9c047e58575 server_cookie=83886c0dd4bc76dd in_seq=0 out_seq=0 2024-02-02T15:36:56.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.137+0000 7ffabfe00700 10 client.28240 ms_handle_connect on v2:172.21.15.142:6834/1052606547 2024-02-02T15:36:56.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.137+0000 7ffabfe00700 1 -- 172.21.15.1:0/3348881 <== mds.4 v2:172.21.15.142:6834/1052606547 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55affd319f50 con 0x55affd475eb0 2024-02-02T15:36:56.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.137+0000 7ffabfe00700 10 client.28240 handle_command_reply: tid=0 2024-02-02T15:36:56.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.137+0000 7ffaf086fb80 1 client.28240 shutdown 2024-02-02T15:36:56.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.137+0000 7ffaf086fb80 1 -- 172.21.15.1:0/3348881 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55affd409930 msgr2=0x55affd419310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:56.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.137+0000 7ffaf086fb80 1 --2- 172.21.15.1:0/3348881 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55affd409930 0x55affd419310 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7ffac001f040 tx=0x7ffac0009f50 comp rx=0 tx=0).stop 2024-02-02T15:36:56.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.138+0000 7ffaf086fb80 1 -- 172.21.15.1:0/3348881 shutdown_connections 2024-02-02T15:36:56.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.138+0000 7ffaf086fb80 1 --2- 172.21.15.1:0/3348881 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55affd475eb0 0x55affd4782d0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.138+0000 7ffaf086fb80 1 --2- 172.21.15.1:0/3348881 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55affd4147e0 0x55affd419850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.138+0000 7ffaf086fb80 1 --2- 172.21.15.1:0/3348881 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55affd409010 0x55affd40efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.138+0000 7ffabfe00700 0 client.28240 ms_handle_reset on v2:172.21.15.142:6834/1052606547 2024-02-02T15:36:56.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.138+0000 7ffaf086fb80 1 --2- 172.21.15.1:0/3348881 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55affd409930 0x55affd419310 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.138+0000 7ffaf086fb80 1 -- 172.21.15.1:0/3348881 >> 172.21.15.1:0/3348881 conn(0x55affd368db0 msgr2=0x55affd41f900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:56.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.138+0000 7ffaf086fb80 1 -- 172.21.15.1:0/3348881 shutdown_connections 2024-02-02T15:36:56.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.138+0000 7ffaf086fb80 1 -- 172.21.15.1:0/3348881 wait complete. 2024-02-02T15:36:56.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.138+0000 7ffaf086fb80 20 client.28240 trim_cache size 0 max 16384 2024-02-02T15:36:56.143 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:36:56.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.138+0000 7ffadce00700 1 -- 172.21.15.1:0/1128087869 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ffab80a0710 msgr2=0x7ffab80a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:56.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.138+0000 7ffadce00700 1 --2- 172.21.15.1:0/1128087869 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ffab80a0710 0x7ffab80a2bd0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ffac8005d10 tx=0x7ffac8009310 comp rx=0 tx=0).stop 2024-02-02T15:36:56.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.139+0000 7ffadce00700 1 -- 172.21.15.1:0/1128087869 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffad8073a00 msgr2=0x7ffad819da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:56.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.139+0000 7ffadce00700 1 --2- 172.21.15.1:0/1128087869 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffad8073a00 0x7ffad819da70 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7ffac0009810 tx=0x7ffac000fa00 comp rx=0 tx=0).stop 2024-02-02T15:36:56.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.139+0000 7ffadce00700 1 -- 172.21.15.1:0/1128087869 shutdown_connections 2024-02-02T15:36:56.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.139+0000 7ffadce00700 1 --2- 172.21.15.1:0/1128087869 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ffab80a0710 0x7ffab80a2bd0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.139+0000 7ffadce00700 1 --2- 172.21.15.1:0/1128087869 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffad8111d90 0x7ffad81a1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.139+0000 7ffadce00700 1 --2- 172.21.15.1:0/1128087869 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffad80730e0 0x7ffad819d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.139+0000 7ffadce00700 1 --2- 172.21.15.1:0/1128087869 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffad8073a00 0x7ffad819da70 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.139+0000 7ffadce00700 1 -- 172.21.15.1:0/1128087869 >> 172.21.15.1:0/1128087869 conn(0x7ffad80fc890 msgr2=0x7ffad80fd500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:56.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.139+0000 7ffadce00700 1 -- 172.21.15.1:0/1128087869 shutdown_connections 2024-02-02T15:36:56.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.139+0000 7ffadce00700 1 -- 172.21.15.1:0/1128087869 wait complete. 2024-02-02T15:36:56.158 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-02-02T15:36:56.258 INFO:tasks.workunit.client.0.smithi001.stdout: 1 574318 6.04 MB/sec execute 390 sec latency 831.910 ms 2024-02-02T15:36:56.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.379+0000 7fed6de00700 1 -- 172.21.15.1:0/1305501639 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed68106aa0 msgr2=0x7fed68106e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:56.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.379+0000 7fed6de00700 1 --2- 172.21.15.1:0/1305501639 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed68106aa0 0x7fed68106e80 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fed58009990 tx=0x7fed58009ca0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.379+0000 7fed6de00700 1 -- 172.21.15.1:0/1305501639 shutdown_connections 2024-02-02T15:36:56.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.379+0000 7fed6de00700 1 --2- 172.21.15.1:0/1305501639 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fed68101450 0x7fed6810d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.379+0000 7fed6de00700 1 --2- 172.21.15.1:0/1305501639 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed68106aa0 0x7fed68106e80 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.379+0000 7fed6de00700 1 --2- 172.21.15.1:0/1305501639 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed68100a90 0x7fed68100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.379+0000 7fed6de00700 1 -- 172.21.15.1:0/1305501639 >> 172.21.15.1:0/1305501639 conn(0x7fed680fc580 msgr2=0x7fed680fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:56.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.379+0000 7fed6de00700 1 -- 172.21.15.1:0/1305501639 shutdown_connections 2024-02-02T15:36:56.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.379+0000 7fed6de00700 1 -- 172.21.15.1:0/1305501639 wait complete. 2024-02-02T15:36:56.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.380+0000 7fed6de00700 1 Processor -- start 2024-02-02T15:36:56.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.380+0000 7fed6de00700 1 -- start start 2024-02-02T15:36:56.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.380+0000 7fed6de00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fed68100a90 0x7fed68073040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:56.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.380+0000 7fed6de00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed68101450 0x7fed68073580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:56.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.381+0000 7fed6de00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed68106aa0 0x7fed681a4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:56.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.381+0000 7fed6de00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed68073db0 con 0x7fed68101450 2024-02-02T15:36:56.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.381+0000 7fed6de00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed681a52b0 con 0x7fed68100a90 2024-02-02T15:36:56.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.381+0000 7fed6de00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed681a53f0 con 0x7fed68106aa0 2024-02-02T15:36:56.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.381+0000 7fed6ca00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fed68100a90 0x7fed68073040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:56.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.381+0000 7fed67e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed68101450 0x7fed68073580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:56.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.381+0000 7fed6ca00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fed68100a90 0x7fed68073040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59264/0 (socket says 172.21.15.1:59264) 2024-02-02T15:36:56.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.381+0000 7fed6ca00700 1 -- 172.21.15.1:0/62506864 learned_addr learned my addr 172.21.15.1:0/62506864 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:36:56.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.381+0000 7fed6d400700 1 --2- 172.21.15.1:0/62506864 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed68106aa0 0x7fed681a4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:56.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.381+0000 7fed67e00700 1 -- 172.21.15.1:0/62506864 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed68106aa0 msgr2=0x7fed681a4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:56.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.381+0000 7fed67e00700 1 --2- 172.21.15.1:0/62506864 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed68106aa0 0x7fed681a4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.381+0000 7fed67e00700 1 -- 172.21.15.1:0/62506864 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fed68100a90 msgr2=0x7fed68073040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:56.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.381+0000 7fed67e00700 1 --2- 172.21.15.1:0/62506864 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fed68100a90 0x7fed68073040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.381+0000 7fed67e00700 1 -- 172.21.15.1:0/62506864 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed50009520 con 0x7fed68101450 2024-02-02T15:36:56.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.381+0000 7fed6ca00700 1 --2- 172.21.15.1:0/62506864 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fed68100a90 0x7fed68073040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:36:56.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.381+0000 7fed67e00700 1 --2- 172.21.15.1:0/62506864 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed68101450 0x7fed68073580 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fed5000e9f0 tx=0x7fed5000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:56.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.382+0000 7fed65600700 1 -- 172.21.15.1:0/62506864 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed5000cb70 con 0x7fed68101450 2024-02-02T15:36:56.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.382+0000 7fed65600700 1 -- 172.21.15.1:0/62506864 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed5000ee90 con 0x7fed68101450 2024-02-02T15:36:56.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.382+0000 7fed6de00700 1 -- 172.21.15.1:0/62506864 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed580095f0 con 0x7fed68101450 2024-02-02T15:36:56.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.382+0000 7fed65600700 1 -- 172.21.15.1:0/62506864 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed50003cf0 con 0x7fed68101450 2024-02-02T15:36:56.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.382+0000 7fed6de00700 1 -- 172.21.15.1:0/62506864 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed681a59f0 con 0x7fed68101450 2024-02-02T15:36:56.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.386+0000 7fed65600700 1 -- 172.21.15.1:0/62506864 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7fed5000ccd0 con 0x7fed68101450 2024-02-02T15:36:56.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.387+0000 7fed82e21b80 1 Processor -- start 2024-02-02T15:36:56.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.387+0000 7fed82e21b80 1 -- start start 2024-02-02T15:36:56.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.387+0000 7fed82e21b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588a2ead1f0 0x5588a2ead5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:56.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.387+0000 7fed82e21b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588a2eadb10 0x5588a2eb84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:56.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.387+0000 7fed82e21b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5588a2eb89f0 0x5588a2ebade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:56.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.387+0000 7fed82e21b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588a2e214c0 con 0x5588a2eb89f0 2024-02-02T15:36:56.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.387+0000 7fed82e21b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588a2e05aa0 con 0x5588a2ead1f0 2024-02-02T15:36:56.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.387+0000 7fed82e21b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588a2e03460 con 0x5588a2eadb10 2024-02-02T15:36:56.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.387+0000 7fed6d400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588a2ead1f0 0x5588a2ead5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:56.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.387+0000 7fed6d400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588a2ead1f0 0x5588a2ead5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59276/0 (socket says 172.21.15.1:59276) 2024-02-02T15:36:56.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.387+0000 7fed6d400700 1 -- 172.21.15.1:0/4224575343 learned_addr learned my addr 172.21.15.1:0/4224575343 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:36:56.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.388+0000 7fed67e00700 1 --2- 172.21.15.1:0/4224575343 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5588a2eb89f0 0x5588a2ebade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:56.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.388+0000 7fed6ca00700 1 --2- 172.21.15.1:0/4224575343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588a2eadb10 0x5588a2eb84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:56.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.388+0000 7fed6d400700 1 -- 172.21.15.1:0/4224575343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588a2eadb10 msgr2=0x5588a2eb84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:56.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.388+0000 7fed6d400700 1 --2- 172.21.15.1:0/4224575343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588a2eadb10 0x5588a2eb84b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.388+0000 7fed6d400700 1 -- 172.21.15.1:0/4224575343 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5588a2eb89f0 msgr2=0x5588a2ebade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:56.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.388+0000 7fed6d400700 1 --2- 172.21.15.1:0/4224575343 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5588a2eb89f0 0x5588a2ebade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.388+0000 7fed6d400700 1 -- 172.21.15.1:0/4224575343 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5588a2c803b0 con 0x5588a2ead1f0 2024-02-02T15:36:56.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.388+0000 7fed6ca00700 1 --2- 172.21.15.1:0/4224575343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588a2eadb10 0x5588a2eb84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:36:56.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.388+0000 7fed6d400700 1 --2- 172.21.15.1:0/4224575343 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588a2ead1f0 0x5588a2ead5d0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fed60011600 tx=0x7fed60009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3305dbd53f3083da server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:56.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.388+0000 7fed5dc00700 1 -- 172.21.15.1:0/4224575343 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed6000e040 con 0x5588a2ead1f0 2024-02-02T15:36:56.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.388+0000 7fed5dc00700 1 -- 172.21.15.1:0/4224575343 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed6000f040 con 0x5588a2ead1f0 2024-02-02T15:36:56.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.388+0000 7fed5dc00700 1 -- 172.21.15.1:0/4224575343 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed60004810 con 0x5588a2ead1f0 2024-02-02T15:36:56.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.388+0000 7fed65600700 1 --2- 172.21.15.1:0/62506864 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fed480a0710 0x7fed480a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:56.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.389+0000 7fed65600700 1 -- 172.21.15.1:0/62506864 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fed50014070 con 0x7fed68101450 2024-02-02T15:36:56.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.389+0000 7fed6ca00700 1 --2- 172.21.15.1:0/62506864 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fed480a0710 0x7fed480a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:56.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.389+0000 7fed82e21b80 1 -- 172.21.15.1:0/4224575343 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588a2ead1f0 msgr2=0x5588a2ead5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:56.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.389+0000 7fed82e21b80 1 --2- 172.21.15.1:0/4224575343 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588a2ead1f0 0x5588a2ead5d0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fed60011600 tx=0x7fed60009920 comp rx=0 tx=0).stop 2024-02-02T15:36:56.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.389+0000 7fed82e21b80 1 -- 172.21.15.1:0/4224575343 shutdown_connections 2024-02-02T15:36:56.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.389+0000 7fed82e21b80 1 --2- 172.21.15.1:0/4224575343 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5588a2eb89f0 0x5588a2ebade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.389+0000 7fed82e21b80 1 --2- 172.21.15.1:0/4224575343 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588a2ead1f0 0x5588a2ead5d0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.389+0000 7fed82e21b80 1 --2- 172.21.15.1:0/4224575343 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588a2eadb10 0x5588a2eb84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.389+0000 7fed82e21b80 1 -- 172.21.15.1:0/4224575343 >> 172.21.15.1:0/4224575343 conn(0x5588a2e0cdb0 msgr2=0x5588a2ea95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:56.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.390+0000 7fed6ca00700 1 --2- 172.21.15.1:0/62506864 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fed480a0710 0x7fed480a2bd0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fed58006010 tx=0x7fed5800b260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:56.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.390+0000 7fed82e21b80 1 -- 172.21.15.1:0/4224575343 shutdown_connections 2024-02-02T15:36:56.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.390+0000 7fed82e21b80 1 -- 172.21.15.1:0/4224575343 wait complete. 2024-02-02T15:36:56.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.390+0000 7fed82e21b80 1 -- start start 2024-02-02T15:36:56.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.390+0000 7fed82e21b80 1 Processor -- start 2024-02-02T15:36:56.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.391+0000 7fed82e21b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5588a2ead1f0 0x5588a2eb8440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:56.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.391+0000 7fed82e21b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588a2eadb10 0x5588a2eb5650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:56.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.391+0000 7fed82e21b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588a2eb89f0 0x5588a2eb5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:56.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.391+0000 7fed82e21b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588a2dfed50 con 0x5588a2ead1f0 2024-02-02T15:36:56.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.391+0000 7fed82e21b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588a2d9ad80 con 0x5588a2eadb10 2024-02-02T15:36:56.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.391+0000 7fed82e21b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588a2d9aa20 con 0x5588a2eb89f0 2024-02-02T15:36:56.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.391+0000 7fed82e21b80 10 client.0 _ll_register_callbacks cb 0x5588a2d9c290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:36:56.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.391+0000 7fed67e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588a2eadb10 0x5588a2eb5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:56.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.391+0000 7fed67e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588a2eadb10 0x5588a2eb5650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59282/0 (socket says 172.21.15.1:59282) 2024-02-02T15:36:56.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.391+0000 7fed67e00700 1 -- 172.21.15.1:0/2588579842 learned_addr learned my addr 172.21.15.1:0/2588579842 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:36:56.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.391+0000 7fed6d400700 1 --2- 172.21.15.1:0/2588579842 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588a2eb89f0 0x5588a2eb5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:56.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.391+0000 7fed6ca00700 1 --2- 172.21.15.1:0/2588579842 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5588a2ead1f0 0x5588a2eb8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:56.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.391+0000 7fed67e00700 1 -- 172.21.15.1:0/2588579842 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588a2eb89f0 msgr2=0x5588a2eb5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:56.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.391+0000 7fed67e00700 1 --2- 172.21.15.1:0/2588579842 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588a2eb89f0 0x5588a2eb5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.391+0000 7fed67e00700 1 -- 172.21.15.1:0/2588579842 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5588a2ead1f0 msgr2=0x5588a2eb8440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:56.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.391+0000 7fed67e00700 1 --2- 172.21.15.1:0/2588579842 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5588a2ead1f0 0x5588a2eb8440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.392+0000 7fed67e00700 1 -- 172.21.15.1:0/2588579842 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed600112b0 con 0x5588a2eadb10 2024-02-02T15:36:56.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.392+0000 7fed6d400700 1 --2- 172.21.15.1:0/2588579842 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588a2eb89f0 0x5588a2eb5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:36:56.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.392+0000 7fed6ca00700 1 --2- 172.21.15.1:0/2588579842 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5588a2ead1f0 0x5588a2eb8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:36:56.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.392+0000 7fed67e00700 1 --2- 172.21.15.1:0/2588579842 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588a2eadb10 0x5588a2eb5650 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fed5000e9c0 tx=0x7fed50099020 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bdaac649b6d7ff22 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:56.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.392+0000 7fed56000700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:36:56.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.392+0000 7fed56000700 1 -- 172.21.15.1:0/2588579842 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed5009d150 con 0x5588a2eadb10 2024-02-02T15:36:56.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.392+0000 7fed56000700 1 -- 172.21.15.1:0/2588579842 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fed500994e0 con 0x5588a2eadb10 2024-02-02T15:36:56.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.392+0000 7fed82e21b80 1 -- 172.21.15.1:0/2588579842 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5588a2ea1340 con 0x5588a2eadb10 2024-02-02T15:36:56.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.392+0000 7fed56000700 1 -- 172.21.15.1:0/2588579842 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed5009d310 con 0x5588a2eadb10 2024-02-02T15:36:56.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.392+0000 7fed82e21b80 10 client.28253 fetch_fsmap learned FSMap version 45 2024-02-02T15:36:56.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.392+0000 7fed82e21b80 1 -- 172.21.15.1:0/2588579842 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x5588a2c2cb60 con 0x5588a2eadb10 2024-02-02T15:36:56.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.392+0000 7fed56000700 1 -- 172.21.15.1:0/2588579842 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed500b0030 con 0x5588a2eadb10 2024-02-02T15:36:56.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.393+0000 7fed56000700 1 -- 172.21.15.1:0/2588579842 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 45) v1 ==== 7506+0+0 (secure 0 0 0) 0x7fed50099840 con 0x5588a2eadb10 2024-02-02T15:36:56.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.393+0000 7fed82e21b80 10 client.28253 fetch_fsmap finished waiting for FSMap version 45 2024-02-02T15:36:56.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.393+0000 7fed82e21b80 10 client.28253 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-02-02T15:36:56.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.393+0000 7fed82e21b80 20 client.28253 populate_metadata read hostname 'smithi001' 2024-02-02T15:36:56.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.393+0000 7fed82e21b80 1 --2- 172.21.15.1:0/2588579842 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x5588a2ec8340 0x5588a2eca760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:56.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.393+0000 7fed82e21b80 4 client.28253 mds_command: new command op to 24514 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:36:56.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.393+0000 7fed82e21b80 1 -- 172.21.15.1:0/2588579842 --> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5588a2df7dd0 con 0x5588a2ec8340 2024-02-02T15:36:56.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.393+0000 7fed6ca00700 1 --2- 172.21.15.1:0/2588579842 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x5588a2ec8340 0x5588a2eca760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:56.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.394+0000 7fed6ca00700 1 --2- 172.21.15.1:0/2588579842 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x5588a2ec8340 0x5588a2eca760 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8508a3cce053529d server_cookie=61c93cce3e4aaa18 in_seq=0 out_seq=0 2024-02-02T15:36:56.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.394+0000 7fed56000700 10 client.28253 ms_handle_connect on v2:172.21.15.142:6836/2870289019 2024-02-02T15:36:56.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.394+0000 7fed56000700 1 -- 172.21.15.1:0/2588579842 <== mds.3 v2:172.21.15.142:6836/2870289019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5588a2df7dd0 con 0x5588a2ec8340 2024-02-02T15:36:56.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.394+0000 7fed56000700 10 client.28253 handle_command_reply: tid=0 2024-02-02T15:36:56.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.395+0000 7fed82e21b80 1 client.28253 shutdown 2024-02-02T15:36:56.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.395+0000 7fed82e21b80 1 -- 172.21.15.1:0/2588579842 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588a2eadb10 msgr2=0x5588a2eb5650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:56.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.395+0000 7fed82e21b80 1 --2- 172.21.15.1:0/2588579842 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588a2eadb10 0x5588a2eb5650 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fed5000e9c0 tx=0x7fed50099020 comp rx=0 tx=0).stop 2024-02-02T15:36:56.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.395+0000 7fed82e21b80 1 -- 172.21.15.1:0/2588579842 shutdown_connections 2024-02-02T15:36:56.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.395+0000 7fed82e21b80 1 --2- 172.21.15.1:0/2588579842 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588a2eb89f0 0x5588a2eb5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.395+0000 7fed82e21b80 1 --2- 172.21.15.1:0/2588579842 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x5588a2ec8340 0x5588a2eca760 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.395+0000 7fed82e21b80 1 --2- 172.21.15.1:0/2588579842 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5588a2ead1f0 0x5588a2eb8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.395+0000 7fed82e21b80 1 --2- 172.21.15.1:0/2588579842 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588a2eadb10 0x5588a2eb5650 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.395+0000 7fed56000700 0 client.28253 ms_handle_reset on v2:172.21.15.142:6836/2870289019 2024-02-02T15:36:56.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.395+0000 7fed82e21b80 1 -- 172.21.15.1:0/2588579842 >> 172.21.15.1:0/2588579842 conn(0x5588a2e0cdb0 msgr2=0x5588a2ea91b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:56.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.395+0000 7fed82e21b80 1 -- 172.21.15.1:0/2588579842 shutdown_connections 2024-02-02T15:36:56.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.395+0000 7fed82e21b80 1 -- 172.21.15.1:0/2588579842 wait complete. 2024-02-02T15:36:56.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.395+0000 7fed82e21b80 20 client.28253 trim_cache size 0 max 16384 2024-02-02T15:36:56.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.411+0000 7fed82e21b80 1 Processor -- start 2024-02-02T15:36:56.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.411+0000 7fed82e21b80 1 -- start start 2024-02-02T15:36:56.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.412+0000 7fed82e21b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5588a2ead1f0 0x5588a2eafa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:56.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.412+0000 7fed82e21b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588a2eadb10 0x5588a2eb7f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:56.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.412+0000 7fed82e21b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588a2eb89f0 0x5588a2eb4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:56.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.412+0000 7fed82e21b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588a2c826a0 con 0x5588a2ead1f0 2024-02-02T15:36:56.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.412+0000 7fed82e21b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588a2ec5a90 con 0x5588a2eb89f0 2024-02-02T15:36:56.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.412+0000 7fed82e21b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588a2eae230 con 0x5588a2eadb10 2024-02-02T15:36:56.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.412+0000 7fed6ca00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5588a2ead1f0 0x5588a2eafa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:56.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.412+0000 7fed67e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588a2eadb10 0x5588a2eb7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:56.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.412+0000 7fed6ca00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5588a2ead1f0 0x5588a2eafa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48740/0 (socket says 172.21.15.1:48740) 2024-02-02T15:36:56.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.412+0000 7fed6ca00700 1 -- 172.21.15.1:0/542757066 learned_addr learned my addr 172.21.15.1:0/542757066 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:36:56.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.412+0000 7fed6d400700 1 --2- 172.21.15.1:0/542757066 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588a2eb89f0 0x5588a2eb4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:56.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.412+0000 7fed67e00700 1 -- 172.21.15.1:0/542757066 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588a2eb89f0 msgr2=0x5588a2eb4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:56.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.412+0000 7fed67e00700 1 --2- 172.21.15.1:0/542757066 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588a2eb89f0 0x5588a2eb4ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.413+0000 7fed67e00700 1 -- 172.21.15.1:0/542757066 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5588a2ead1f0 msgr2=0x5588a2eafa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:56.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.413+0000 7fed67e00700 1 --2- 172.21.15.1:0/542757066 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5588a2ead1f0 0x5588a2eafa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.413+0000 7fed67e00700 1 -- 172.21.15.1:0/542757066 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5588a2dc4ea0 con 0x5588a2eadb10 2024-02-02T15:36:56.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.413+0000 7fed6ca00700 1 --2- 172.21.15.1:0/542757066 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5588a2ead1f0 0x5588a2eafa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:36:56.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.413+0000 7fed6d400700 1 --2- 172.21.15.1:0/542757066 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588a2eb89f0 0x5588a2eb4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:36:56.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.413+0000 7fed67e00700 1 --2- 172.21.15.1:0/542757066 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588a2eadb10 0x5588a2eb7f70 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fed500c9d10 tx=0x7fed500cf890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4b6aaf9b64f276b server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:56.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.413+0000 7fed5e600700 1 -- 172.21.15.1:0/542757066 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed5009d150 con 0x5588a2eadb10 2024-02-02T15:36:56.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.413+0000 7fed5e600700 1 -- 172.21.15.1:0/542757066 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed500b0070 con 0x5588a2eadb10 2024-02-02T15:36:56.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.413+0000 7fed5e600700 1 -- 172.21.15.1:0/542757066 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed500a1550 con 0x5588a2eadb10 2024-02-02T15:36:56.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.414+0000 7fed82e21b80 1 -- 172.21.15.1:0/542757066 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588a2eadb10 msgr2=0x5588a2eb7f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:56.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.414+0000 7fed82e21b80 1 --2- 172.21.15.1:0/542757066 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588a2eadb10 0x5588a2eb7f70 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fed500c9d10 tx=0x7fed500cf890 comp rx=0 tx=0).stop 2024-02-02T15:36:56.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.414+0000 7fed82e21b80 1 -- 172.21.15.1:0/542757066 shutdown_connections 2024-02-02T15:36:56.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.414+0000 7fed82e21b80 1 --2- 172.21.15.1:0/542757066 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588a2eb89f0 0x5588a2eb4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.414+0000 7fed82e21b80 1 --2- 172.21.15.1:0/542757066 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5588a2ead1f0 0x5588a2eafa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.414+0000 7fed82e21b80 1 --2- 172.21.15.1:0/542757066 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588a2eadb10 0x5588a2eb7f70 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.414+0000 7fed82e21b80 1 -- 172.21.15.1:0/542757066 >> 172.21.15.1:0/542757066 conn(0x5588a2e0cdb0 msgr2=0x5588a2eb0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:56.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.414+0000 7fed82e21b80 1 -- 172.21.15.1:0/542757066 shutdown_connections 2024-02-02T15:36:56.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.414+0000 7fed82e21b80 1 -- 172.21.15.1:0/542757066 wait complete. 2024-02-02T15:36:56.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.414+0000 7fed82e21b80 1 -- start start 2024-02-02T15:36:56.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.415+0000 7fed82e21b80 1 Processor -- start 2024-02-02T15:36:56.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.415+0000 7fed82e21b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5588a2ead1f0 0x5588a2ebb640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:56.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.415+0000 7fed82e21b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588a2eadb10 0x5588a2ebbb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:56.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.415+0000 7fed82e21b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588a2eb89f0 0x5588a2f12e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:56.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.415+0000 7fed82e21b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588a2f133d0 con 0x5588a2ead1f0 2024-02-02T15:36:56.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.415+0000 7fed82e21b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588a2f13540 con 0x5588a2eadb10 2024-02-02T15:36:56.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.415+0000 7fed82e21b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588a2f13700 con 0x5588a2eb89f0 2024-02-02T15:36:56.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.415+0000 7fed82e21b80 10 client.0 _ll_register_callbacks cb 0x5588a2d9c290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:36:56.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.415+0000 7fed6ca00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5588a2ead1f0 0x5588a2ebb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:56.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.415+0000 7fed67e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588a2eadb10 0x5588a2ebbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:56.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.415+0000 7fed6d400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588a2eb89f0 0x5588a2f12e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:56.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.415+0000 7fed67e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588a2eadb10 0x5588a2ebbb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59298/0 (socket says 172.21.15.1:59298) 2024-02-02T15:36:56.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.415+0000 7fed6ca00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5588a2ead1f0 0x5588a2ebb640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48748/0 (socket says 172.21.15.1:48748) 2024-02-02T15:36:56.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.415+0000 7fed67e00700 1 -- 172.21.15.1:0/895291025 learned_addr learned my addr 172.21.15.1:0/895291025 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:36:56.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.416+0000 7fed67e00700 1 -- 172.21.15.1:0/895291025 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588a2eb89f0 msgr2=0x5588a2f12e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:56.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.416+0000 7fed67e00700 1 --2- 172.21.15.1:0/895291025 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588a2eb89f0 0x5588a2f12e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.416+0000 7fed67e00700 1 -- 172.21.15.1:0/895291025 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5588a2ead1f0 msgr2=0x5588a2ebb640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:56.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.416+0000 7fed67e00700 1 --2- 172.21.15.1:0/895291025 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5588a2ead1f0 0x5588a2ebb640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.416+0000 7fed67e00700 1 -- 172.21.15.1:0/895291025 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed6810e260 con 0x5588a2eadb10 2024-02-02T15:36:56.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.416+0000 7fed6d400700 1 --2- 172.21.15.1:0/895291025 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588a2eb89f0 0x5588a2f12e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:36:56.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.416+0000 7fed67e00700 1 --2- 172.21.15.1:0/895291025 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588a2eadb10 0x5588a2ebbb80 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fed50009230 tx=0x7fed50099020 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eaeec1c716309ba4 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:56.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.416+0000 7fed56000700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:36:56.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.416+0000 7fed56000700 1 -- 172.21.15.1:0/895291025 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed5009d150 con 0x5588a2eadb10 2024-02-02T15:36:56.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.416+0000 7fed56000700 1 -- 172.21.15.1:0/895291025 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fed50098260 con 0x5588a2eadb10 2024-02-02T15:36:56.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.416+0000 7fed82e21b80 1 -- 172.21.15.1:0/895291025 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fed6000f040 con 0x5588a2eadb10 2024-02-02T15:36:56.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.416+0000 7fed56000700 1 -- 172.21.15.1:0/895291025 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed500994e0 con 0x5588a2eadb10 2024-02-02T15:36:56.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.416+0000 7fed82e21b80 10 client.28259 fetch_fsmap learned FSMap version 45 2024-02-02T15:36:56.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.416+0000 7fed82e21b80 1 -- 172.21.15.1:0/895291025 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x5588a2c803b0 con 0x5588a2eadb10 2024-02-02T15:36:56.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.417+0000 7fed56000700 1 -- 172.21.15.1:0/895291025 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fed6000f040 con 0x5588a2eadb10 2024-02-02T15:36:56.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.417+0000 7fed56000700 1 -- 172.21.15.1:0/895291025 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 45) v1 ==== 7506+0+0 (secure 0 0 0) 0x7fed500984d0 con 0x5588a2eadb10 2024-02-02T15:36:56.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.417+0000 7fed82e21b80 10 client.28259 fetch_fsmap finished waiting for FSMap version 45 2024-02-02T15:36:56.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.417+0000 7fed82e21b80 10 client.28259 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-02-02T15:36:56.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.417+0000 7fed82e21b80 20 client.28259 populate_metadata read hostname 'smithi001' 2024-02-02T15:36:56.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.417+0000 7fed82e21b80 1 --2- 172.21.15.1:0/895291025 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x5588a2f1a980 0x5588a2f1cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:56.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.417+0000 7fed82e21b80 4 client.28259 mds_command: new command op to 24514 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:36:56.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.417+0000 7fed82e21b80 1 -- 172.21.15.1:0/895291025 --> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5588a2dbdf50 con 0x5588a2f1a980 2024-02-02T15:36:56.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.417+0000 7fed6ca00700 1 --2- 172.21.15.1:0/895291025 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x5588a2f1a980 0x5588a2f1cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:56.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.418+0000 7fed6ca00700 1 --2- 172.21.15.1:0/895291025 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x5588a2f1a980 0x5588a2f1cd70 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=35498e490b13360f server_cookie=4b5f621818c5f6d9 in_seq=0 out_seq=0 2024-02-02T15:36:56.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.418+0000 7fed56000700 10 client.28259 ms_handle_connect on v2:172.21.15.142:6836/2870289019 2024-02-02T15:36:56.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.418+0000 7fed56000700 1 -- 172.21.15.1:0/895291025 <== mds.3 v2:172.21.15.142:6836/2870289019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5588a2dbdf50 con 0x5588a2f1a980 2024-02-02T15:36:56.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.418+0000 7fed56000700 10 client.28259 handle_command_reply: tid=0 2024-02-02T15:36:56.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.418+0000 7fed82e21b80 1 client.28259 shutdown 2024-02-02T15:36:56.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.419+0000 7fed82e21b80 1 -- 172.21.15.1:0/895291025 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588a2eadb10 msgr2=0x5588a2ebbb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:56.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.419+0000 7fed82e21b80 1 --2- 172.21.15.1:0/895291025 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588a2eadb10 0x5588a2ebbb80 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fed50009230 tx=0x7fed50099020 comp rx=0 tx=0).stop 2024-02-02T15:36:56.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.419+0000 7fed82e21b80 1 -- 172.21.15.1:0/895291025 shutdown_connections 2024-02-02T15:36:56.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.419+0000 7fed82e21b80 1 --2- 172.21.15.1:0/895291025 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5588a2eb89f0 0x5588a2f12e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.419+0000 7fed82e21b80 1 --2- 172.21.15.1:0/895291025 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x5588a2f1a980 0x5588a2f1cd70 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.419+0000 7fed82e21b80 1 --2- 172.21.15.1:0/895291025 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5588a2ead1f0 0x5588a2ebb640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.419+0000 7fed82e21b80 1 --2- 172.21.15.1:0/895291025 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5588a2eadb10 0x5588a2ebbb80 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.419+0000 7fed82e21b80 1 -- 172.21.15.1:0/895291025 >> 172.21.15.1:0/895291025 conn(0x5588a2e0cdb0 msgr2=0x5588a2eb4ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:56.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.419+0000 7fed56000700 0 client.28259 ms_handle_reset on v2:172.21.15.142:6836/2870289019 2024-02-02T15:36:56.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.419+0000 7fed82e21b80 1 -- 172.21.15.1:0/895291025 shutdown_connections 2024-02-02T15:36:56.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.419+0000 7fed82e21b80 1 -- 172.21.15.1:0/895291025 wait complete. 2024-02-02T15:36:56.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.419+0000 7fed82e21b80 20 client.28259 trim_cache size 0 max 16384 2024-02-02T15:36:56.424 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:36:56.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.420+0000 7fed6de00700 1 -- 172.21.15.1:0/62506864 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fed480a0710 msgr2=0x7fed480a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:56.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.420+0000 7fed6de00700 1 --2- 172.21.15.1:0/62506864 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fed480a0710 0x7fed480a2bd0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fed58006010 tx=0x7fed5800b260 comp rx=0 tx=0).stop 2024-02-02T15:36:56.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.420+0000 7fed6de00700 1 -- 172.21.15.1:0/62506864 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed68101450 msgr2=0x7fed68073580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:56.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.420+0000 7fed6de00700 1 --2- 172.21.15.1:0/62506864 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed68101450 0x7fed68073580 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fed5000e9f0 tx=0x7fed5000ed00 comp rx=0 tx=0).stop 2024-02-02T15:36:56.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.420+0000 7fed6de00700 1 -- 172.21.15.1:0/62506864 shutdown_connections 2024-02-02T15:36:56.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.420+0000 7fed6de00700 1 --2- 172.21.15.1:0/62506864 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fed480a0710 0x7fed480a2bd0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.420+0000 7fed6de00700 1 --2- 172.21.15.1:0/62506864 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed68106aa0 0x7fed681a4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.420+0000 7fed6de00700 1 --2- 172.21.15.1:0/62506864 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fed68100a90 0x7fed68073040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.420+0000 7fed6de00700 1 --2- 172.21.15.1:0/62506864 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed68101450 0x7fed68073580 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.420+0000 7fed6de00700 1 -- 172.21.15.1:0/62506864 >> 172.21.15.1:0/62506864 conn(0x7fed680fc580 msgr2=0x7fed680fcc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:56.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.420+0000 7fed6de00700 1 -- 172.21.15.1:0/62506864 shutdown_connections 2024-02-02T15:36:56.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.420+0000 7fed6de00700 1 -- 172.21.15.1:0/62506864 wait complete. 2024-02-02T15:36:56.439 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-02T15:36:56.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.663+0000 7f8d8bc00700 1 -- 172.21.15.1:0/1754432612 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d84073a10 msgr2=0x7f8d8410d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:56.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.663+0000 7f8d8bc00700 1 --2- 172.21.15.1:0/1754432612 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d84073a10 0x7f8d8410d0b0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c009990 tx=0x7f8d6c009ca0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.663+0000 7f8d8bc00700 1 -- 172.21.15.1:0/1754432612 shutdown_connections 2024-02-02T15:36:56.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.663+0000 7f8d8bc00700 1 --2- 172.21.15.1:0/1754432612 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8d8410d5f0 0x7f8d8410f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.663+0000 7f8d8bc00700 1 --2- 172.21.15.1:0/1754432612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d840730f0 0x7f8d840734d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.663+0000 7f8d8bc00700 1 --2- 172.21.15.1:0/1754432612 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d84073a10 0x7f8d8410d0b0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.663+0000 7f8d8bc00700 1 -- 172.21.15.1:0/1754432612 >> 172.21.15.1:0/1754432612 conn(0x7f8d840fc800 msgr2=0x7f8d840fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:56.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.663+0000 7f8d8bc00700 1 -- 172.21.15.1:0/1754432612 shutdown_connections 2024-02-02T15:36:56.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.663+0000 7f8d8bc00700 1 -- 172.21.15.1:0/1754432612 wait complete. 2024-02-02T15:36:56.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.664+0000 7f8d8bc00700 1 Processor -- start 2024-02-02T15:36:56.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.664+0000 7f8d8bc00700 1 -- start start 2024-02-02T15:36:56.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.665+0000 7f8d8bc00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8d840730f0 0x7f8d841990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:56.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.665+0000 7f8d8bc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d84073a10 0x7f8d84199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:56.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.665+0000 7f8d8bc00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d8410d5f0 0x7f8d8419da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:56.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.665+0000 7f8d8bc00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d8419e0b0 con 0x7f8d840730f0 2024-02-02T15:36:56.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.665+0000 7f8d8bc00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d8419e220 con 0x7f8d8410d5f0 2024-02-02T15:36:56.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.665+0000 7f8d8bc00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d8419e3e0 con 0x7f8d84073a10 2024-02-02T15:36:56.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.665+0000 7f8d83e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d84073a10 0x7f8d84199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:56.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.665+0000 7f8d89200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8d840730f0 0x7f8d841990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:56.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.665+0000 7f8d89c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d8410d5f0 0x7f8d8419da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:56.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.665+0000 7f8d83e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d84073a10 0x7f8d84199610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:34116/0 (socket says 172.21.15.1:34116) 2024-02-02T15:36:56.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.665+0000 7f8d83e00700 1 -- 172.21.15.1:0/1177944345 learned_addr learned my addr 172.21.15.1:0/1177944345 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:36:56.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.665+0000 7f8d83e00700 1 -- 172.21.15.1:0/1177944345 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d8410d5f0 msgr2=0x7f8d8419da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:56.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.665+0000 7f8d83e00700 1 --2- 172.21.15.1:0/1177944345 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d8410d5f0 0x7f8d8419da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.665+0000 7f8d83e00700 1 -- 172.21.15.1:0/1177944345 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8d840730f0 msgr2=0x7f8d841990d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:56.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.665+0000 7f8d83e00700 1 --2- 172.21.15.1:0/1177944345 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8d840730f0 0x7f8d841990d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.665+0000 7f8d83e00700 1 -- 172.21.15.1:0/1177944345 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d6c0095f0 con 0x7f8d84073a10 2024-02-02T15:36:56.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.665+0000 7f8d89c00700 1 --2- 172.21.15.1:0/1177944345 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d8410d5f0 0x7f8d8419da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:36:56.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.665+0000 7f8d89200700 1 --2- 172.21.15.1:0/1177944345 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8d840730f0 0x7f8d841990d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:36:56.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.666+0000 7f8d83e00700 1 --2- 172.21.15.1:0/1177944345 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d84073a10 0x7f8d84199610 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c0095c0 tx=0x7f8d6c00f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:56.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.666+0000 7f8d81600700 1 -- 172.21.15.1:0/1177944345 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c01d070 con 0x7f8d84073a10 2024-02-02T15:36:56.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.666+0000 7f8d8bc00700 1 -- 172.21.15.1:0/1177944345 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d8419e660 con 0x7f8d84073a10 2024-02-02T15:36:56.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.666+0000 7f8d81600700 1 -- 172.21.15.1:0/1177944345 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d6c005030 con 0x7f8d84073a10 2024-02-02T15:36:56.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.666+0000 7f8d8bc00700 1 -- 172.21.15.1:0/1177944345 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d841a5f00 con 0x7f8d84073a10 2024-02-02T15:36:56.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.666+0000 7f8d81600700 1 -- 172.21.15.1:0/1177944345 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c004a20 con 0x7f8d84073a10 2024-02-02T15:36:56.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.672+0000 7f8d81600700 1 -- 172.21.15.1:0/1177944345 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f8d6c005430 con 0x7f8d84073a10 2024-02-02T15:36:56.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.673+0000 7f8d9f7b3b80 1 Processor -- start 2024-02-02T15:36:56.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.673+0000 7f8d9f7b3b80 1 -- start start 2024-02-02T15:36:56.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.673+0000 7f8d9f7b3b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621ebe12010 0x5621ebe123f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:56.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.673+0000 7f8d9f7b3b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5621ebe12930 0x5621ebe1d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:56.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.673+0000 7f8d9f7b3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621ebe1d7e0 0x5621ebe1fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:56.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.673+0000 7f8d9f7b3b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621ebd864c0 con 0x5621ebe12010 2024-02-02T15:36:56.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.673+0000 7f8d9f7b3b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621ebd6aaa0 con 0x5621ebe12930 2024-02-02T15:36:56.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.673+0000 7f8d9f7b3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621ebd68460 con 0x5621ebe1d7e0 2024-02-02T15:36:56.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.674+0000 7f8d81600700 1 --2- 172.21.15.1:0/1177944345 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f8d640a0690 0x7f8d640a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:56.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.674+0000 7f8d81600700 1 -- 172.21.15.1:0/1177944345 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f8d6c0cceb0 con 0x7f8d84073a10 2024-02-02T15:36:56.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.674+0000 7f8d83e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621ebe1d7e0 0x5621ebe1fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:56.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.674+0000 7f8d89200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5621ebe12930 0x5621ebe1d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:56.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.674+0000 7f8d83e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621ebe1d7e0 0x5621ebe1fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:34122/0 (socket says 172.21.15.1:34122) 2024-02-02T15:36:56.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.674+0000 7f8d83e00700 1 -- 172.21.15.1:0/956197199 learned_addr learned my addr 172.21.15.1:0/956197199 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:36:56.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.674+0000 7f8d83e00700 1 -- 172.21.15.1:0/956197199 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5621ebe12930 msgr2=0x5621ebe1d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:56.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.674+0000 7f8d83e00700 1 --2- 172.21.15.1:0/956197199 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5621ebe12930 0x5621ebe1d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.674+0000 7f8d89c00700 1 --2- 172.21.15.1:0/1177944345 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f8d640a0690 0x7f8d640a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:56.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.674+0000 7f8d83e00700 1 -- 172.21.15.1:0/956197199 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621ebe12010 msgr2=0x5621ebe123f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:56.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.674+0000 7f8d83e00700 1 --2- 172.21.15.1:0/956197199 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621ebe12010 0x5621ebe123f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.674+0000 7f8d83e00700 1 -- 172.21.15.1:0/956197199 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5621ebbe53b0 con 0x5621ebe1d7e0 2024-02-02T15:36:56.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.675+0000 7f8d83e00700 1 --2- 172.21.15.1:0/956197199 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621ebe1d7e0 0x5621ebe1fbd0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c005f50 tx=0x7f8d6c09a830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc247573caa7c1d5 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:56.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.675+0000 7f8d7a000700 1 -- 172.21.15.1:0/956197199 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c09de90 con 0x5621ebe1d7e0 2024-02-02T15:36:56.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.675+0000 7f8d7a000700 1 -- 172.21.15.1:0/956197199 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d6c0a2040 con 0x5621ebe1d7e0 2024-02-02T15:36:56.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.675+0000 7f8d7a000700 1 -- 172.21.15.1:0/956197199 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6c0997b0 con 0x5621ebe1d7e0 2024-02-02T15:36:56.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.676+0000 7f8d9f7b3b80 1 -- 172.21.15.1:0/956197199 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621ebe1d7e0 msgr2=0x5621ebe1fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:56.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.676+0000 7f8d89c00700 1 --2- 172.21.15.1:0/1177944345 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f8d640a0690 0x7f8d640a2b50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c009550 tx=0x7f8d7c00b040 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:56.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.676+0000 7f8d9f7b3b80 1 --2- 172.21.15.1:0/956197199 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621ebe1d7e0 0x5621ebe1fbd0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c005f50 tx=0x7f8d6c09a830 comp rx=0 tx=0).stop 2024-02-02T15:36:56.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.676+0000 7f8d9f7b3b80 1 -- 172.21.15.1:0/956197199 shutdown_connections 2024-02-02T15:36:56.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.676+0000 7f8d9f7b3b80 1 --2- 172.21.15.1:0/956197199 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621ebe1d7e0 0x5621ebe1fbd0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.676+0000 7f8d9f7b3b80 1 --2- 172.21.15.1:0/956197199 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621ebe12010 0x5621ebe123f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.676+0000 7f8d9f7b3b80 1 --2- 172.21.15.1:0/956197199 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5621ebe12930 0x5621ebe1d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.676+0000 7f8d9f7b3b80 1 -- 172.21.15.1:0/956197199 >> 172.21.15.1:0/956197199 conn(0x5621ebd71db0 msgr2=0x5621ebe0e3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:56.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.676+0000 7f8d9f7b3b80 1 -- 172.21.15.1:0/956197199 shutdown_connections 2024-02-02T15:36:56.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.676+0000 7f8d9f7b3b80 1 -- 172.21.15.1:0/956197199 wait complete. 2024-02-02T15:36:56.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.676+0000 7f8d9f7b3b80 1 -- start start 2024-02-02T15:36:56.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.677+0000 7f8d9f7b3b80 1 Processor -- start 2024-02-02T15:36:56.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.677+0000 7f8d9f7b3b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5621ebe12010 0x5621ebe1dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:56.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.677+0000 7f8d9f7b3b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621ebe12930 0x5621ebe1e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:56.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.677+0000 7f8d9f7b3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621ebe1d7e0 0x5621ebe28c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:56.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.677+0000 7f8d9f7b3b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621ebd63d50 con 0x5621ebe12930 2024-02-02T15:36:56.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.677+0000 7f8d9f7b3b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621ebcffd80 con 0x5621ebe12010 2024-02-02T15:36:56.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.677+0000 7f8d9f7b3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621ebcffa20 con 0x5621ebe1d7e0 2024-02-02T15:36:56.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.677+0000 7f8d9f7b3b80 10 client.0 _ll_register_callbacks cb 0x5621ebd01290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:36:56.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.677+0000 7f8d89200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5621ebe12010 0x5621ebe1dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:56.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.677+0000 7f8d89c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621ebe1d7e0 0x5621ebe28c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:56.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.677+0000 7f8d89200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5621ebe12010 0x5621ebe1dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59336/0 (socket says 172.21.15.1:59336) 2024-02-02T15:36:56.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.677+0000 7f8d89c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621ebe1d7e0 0x5621ebe28c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:34138/0 (socket says 172.21.15.1:34138) 2024-02-02T15:36:56.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.677+0000 7f8d89200700 1 -- 172.21.15.1:0/115768264 learned_addr learned my addr 172.21.15.1:0/115768264 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:36:56.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.678+0000 7f8d83e00700 1 --2- 172.21.15.1:0/115768264 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621ebe12930 0x5621ebe1e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:56.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.678+0000 7f8d89200700 1 -- 172.21.15.1:0/115768264 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621ebe1d7e0 msgr2=0x5621ebe28c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:56.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.678+0000 7f8d89200700 1 --2- 172.21.15.1:0/115768264 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621ebe1d7e0 0x5621ebe28c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.678+0000 7f8d89200700 1 -- 172.21.15.1:0/115768264 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621ebe12930 msgr2=0x5621ebe1e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:56.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.678+0000 7f8d89200700 1 --2- 172.21.15.1:0/115768264 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621ebe12930 0x5621ebe1e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.678+0000 7f8d89200700 1 -- 172.21.15.1:0/115768264 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d841a5f00 con 0x5621ebe12010 2024-02-02T15:36:56.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.678+0000 7f8d89200700 1 --2- 172.21.15.1:0/115768264 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5621ebe12010 0x5621ebe1dbc0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f8d740118c0 tx=0x7f8d74011bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c434b387af40340 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:56.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.678+0000 7f8d72a00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:36:56.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.678+0000 7f8d72a00700 1 -- 172.21.15.1:0/115768264 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d74016070 con 0x5621ebe12010 2024-02-02T15:36:56.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.678+0000 7f8d9f7b3b80 1 -- 172.21.15.1:0/115768264 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5621ebe060f0 con 0x5621ebe12010 2024-02-02T15:36:56.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.678+0000 7f8d72a00700 1 -- 172.21.15.1:0/115768264 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f8d74004830 con 0x5621ebe12010 2024-02-02T15:36:56.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.679+0000 7f8d72a00700 1 -- 172.21.15.1:0/115768264 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d740049f0 con 0x5621ebe12010 2024-02-02T15:36:56.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.679+0000 7f8d9f7b3b80 10 client.28271 fetch_fsmap learned FSMap version 46 2024-02-02T15:36:56.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.679+0000 7f8d9f7b3b80 1 -- 172.21.15.1:0/115768264 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x5621ebb91b60 con 0x5621ebe12010 2024-02-02T15:36:56.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.679+0000 7f8d72a00700 1 -- 172.21.15.1:0/115768264 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5621ebe060f0 con 0x5621ebe12010 2024-02-02T15:36:56.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.679+0000 7f8d72a00700 1 -- 172.21.15.1:0/115768264 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 46) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f8d7401a9a0 con 0x5621ebe12010 2024-02-02T15:36:56.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.679+0000 7f8d9f7b3b80 10 client.28271 fetch_fsmap finished waiting for FSMap version 46 2024-02-02T15:36:56.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.679+0000 7f8d9f7b3b80 10 client.28271 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-02-02T15:36:56.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.679+0000 7f8d9f7b3b80 20 client.28271 populate_metadata read hostname 'smithi001' 2024-02-02T15:36:56.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.679+0000 7f8d9f7b3b80 1 --2- 172.21.15.1:0/115768264 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x5621ebe2cf20 0x5621ebe2f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:56.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.679+0000 7f8d9f7b3b80 4 client.28271 mds_command: new command op to 24518 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:36:56.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.679+0000 7f8d9f7b3b80 1 -- 172.21.15.1:0/115768264 --> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5621ebd5cdd0 con 0x5621ebe2cf20 2024-02-02T15:36:56.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.680+0000 7f8d83e00700 1 --2- 172.21.15.1:0/115768264 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x5621ebe2cf20 0x5621ebe2f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:56.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.680+0000 7f8d83e00700 1 --2- 172.21.15.1:0/115768264 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x5621ebe2cf20 0x5621ebe2f340 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f58e942ef10eb09a server_cookie=54c9a2aa3087a25b in_seq=0 out_seq=0 2024-02-02T15:36:56.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.680+0000 7f8d72a00700 10 client.28271 ms_handle_connect on v2:172.21.15.71:6836/628016199 2024-02-02T15:36:56.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.681+0000 7f8d72a00700 1 -- 172.21.15.1:0/115768264 <== mds.2 v2:172.21.15.71:6836/628016199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5621ebd5cdd0 con 0x5621ebe2cf20 2024-02-02T15:36:56.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.681+0000 7f8d72a00700 10 client.28271 handle_command_reply: tid=0 2024-02-02T15:36:56.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.681+0000 7f8d9f7b3b80 1 client.28271 shutdown 2024-02-02T15:36:56.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.681+0000 7f8d9f7b3b80 1 -- 172.21.15.1:0/115768264 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5621ebe12010 msgr2=0x5621ebe1dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:56.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.681+0000 7f8d9f7b3b80 1 --2- 172.21.15.1:0/115768264 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5621ebe12010 0x5621ebe1dbc0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f8d740118c0 tx=0x7f8d74011bd0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.682+0000 7f8d9f7b3b80 1 -- 172.21.15.1:0/115768264 shutdown_connections 2024-02-02T15:36:56.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.682+0000 7f8d9f7b3b80 1 --2- 172.21.15.1:0/115768264 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x5621ebe2cf20 0x5621ebe2f340 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.682+0000 7f8d9f7b3b80 1 --2- 172.21.15.1:0/115768264 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621ebe1d7e0 0x5621ebe28c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.682+0000 7f8d9f7b3b80 1 --2- 172.21.15.1:0/115768264 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5621ebe12010 0x5621ebe1dbc0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.682+0000 7f8d9f7b3b80 1 --2- 172.21.15.1:0/115768264 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621ebe12930 0x5621ebe1e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.682+0000 7f8d72a00700 0 client.28271 ms_handle_reset on v2:172.21.15.71:6836/628016199 2024-02-02T15:36:56.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.682+0000 7f8d9f7b3b80 1 -- 172.21.15.1:0/115768264 >> 172.21.15.1:0/115768264 conn(0x5621ebd71db0 msgr2=0x5621ebe0dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:56.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.682+0000 7f8d9f7b3b80 1 -- 172.21.15.1:0/115768264 shutdown_connections 2024-02-02T15:36:56.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.682+0000 7f8d9f7b3b80 1 -- 172.21.15.1:0/115768264 wait complete. 2024-02-02T15:36:56.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.682+0000 7f8d9f7b3b80 20 client.28271 trim_cache size 0 max 16384 2024-02-02T15:36:56.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:56 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/2716882558' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:36:56.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.696+0000 7f8d9f7b3b80 1 Processor -- start 2024-02-02T15:36:56.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.696+0000 7f8d9f7b3b80 1 -- start start 2024-02-02T15:36:56.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.697+0000 7f8d9f7b3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621ebe12010 0x5621ebe22bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:56.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.697+0000 7f8d9f7b3b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621ebe12930 0x5621ebe1dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:56.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.697+0000 7f8d9f7b3b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5621ebe1d7e0 0x5621ebe1e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:56.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.697+0000 7f8d9f7b3b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621ebbe76a0 con 0x5621ebe12930 2024-02-02T15:36:56.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.697+0000 7f8d9f7b3b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621ebe0f220 con 0x5621ebe1d7e0 2024-02-02T15:36:56.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.697+0000 7f8d9f7b3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621ebe1ca50 con 0x5621ebe12010 2024-02-02T15:36:56.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.697+0000 7f8d89200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621ebe12010 0x5621ebe22bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:56.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.697+0000 7f8d83e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621ebe12930 0x5621ebe1dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:56.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.697+0000 7f8d89200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621ebe12010 0x5621ebe22bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:34154/0 (socket says 172.21.15.1:34154) 2024-02-02T15:36:56.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.697+0000 7f8d89c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5621ebe1d7e0 0x5621ebe1e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:56.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.697+0000 7f8d89200700 1 -- 172.21.15.1:0/1281640746 learned_addr learned my addr 172.21.15.1:0/1281640746 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:36:56.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.697+0000 7f8d89200700 1 -- 172.21.15.1:0/1281640746 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5621ebe1d7e0 msgr2=0x5621ebe1e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:56.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.697+0000 7f8d89200700 1 --2- 172.21.15.1:0/1281640746 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5621ebe1d7e0 0x5621ebe1e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.697+0000 7f8d89200700 1 -- 172.21.15.1:0/1281640746 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621ebe12930 msgr2=0x5621ebe1dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:56.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.697+0000 7f8d89200700 1 --2- 172.21.15.1:0/1281640746 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621ebe12930 0x5621ebe1dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.698+0000 7f8d89200700 1 -- 172.21.15.1:0/1281640746 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5621ebd29ea0 con 0x5621ebe12010 2024-02-02T15:36:56.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.698+0000 7f8d89200700 1 --2- 172.21.15.1:0/1281640746 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621ebe12010 0x5621ebe22bd0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f8d7401ad00 tx=0x7f8d7400b540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c4df3699218ffec server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:56.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.698+0000 7f8d7aa00700 1 -- 172.21.15.1:0/1281640746 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d74016070 con 0x5621ebe12010 2024-02-02T15:36:56.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.698+0000 7f8d7aa00700 1 -- 172.21.15.1:0/1281640746 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d7400f480 con 0x5621ebe12010 2024-02-02T15:36:56.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.698+0000 7f8d7aa00700 1 -- 172.21.15.1:0/1281640746 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d740052d0 con 0x5621ebe12010 2024-02-02T15:36:56.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.699+0000 7f8d9f7b3b80 1 -- 172.21.15.1:0/1281640746 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621ebe12010 msgr2=0x5621ebe22bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:56.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.699+0000 7f8d9f7b3b80 1 --2- 172.21.15.1:0/1281640746 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621ebe12010 0x5621ebe22bd0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f8d7401ad00 tx=0x7f8d7400b540 comp rx=0 tx=0).stop 2024-02-02T15:36:56.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.699+0000 7f8d9f7b3b80 1 -- 172.21.15.1:0/1281640746 shutdown_connections 2024-02-02T15:36:56.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.699+0000 7f8d9f7b3b80 1 --2- 172.21.15.1:0/1281640746 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5621ebe1d7e0 0x5621ebe1e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.699+0000 7f8d9f7b3b80 1 --2- 172.21.15.1:0/1281640746 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621ebe12010 0x5621ebe22bd0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.699+0000 7f8d9f7b3b80 1 --2- 172.21.15.1:0/1281640746 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621ebe12930 0x5621ebe1dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.699+0000 7f8d9f7b3b80 1 -- 172.21.15.1:0/1281640746 >> 172.21.15.1:0/1281640746 conn(0x5621ebd71db0 msgr2=0x5621ebe17c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:56.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.699+0000 7f8d9f7b3b80 1 -- 172.21.15.1:0/1281640746 shutdown_connections 2024-02-02T15:36:56.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.699+0000 7f8d9f7b3b80 1 -- 172.21.15.1:0/1281640746 wait complete. 2024-02-02T15:36:56.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.700+0000 7f8d9f7b3b80 1 -- start start 2024-02-02T15:36:56.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.700+0000 7f8d9f7b3b80 1 Processor -- start 2024-02-02T15:36:56.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.700+0000 7f8d9f7b3b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5621ebe12010 0x5621ebe77040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:56.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.700+0000 7f8d9f7b3b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621ebe12930 0x5621ebe77580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:56.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.701+0000 7f8d9f7b3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621ebe1d7e0 0x5621ebe729d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:56.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.701+0000 7f8d9f7b3b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621ebe73110 con 0x5621ebe12930 2024-02-02T15:36:56.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.701+0000 7f8d9f7b3b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621ebe73280 con 0x5621ebe12010 2024-02-02T15:36:56.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.701+0000 7f8d9f7b3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621ebe73440 con 0x5621ebe1d7e0 2024-02-02T15:36:56.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.701+0000 7f8d9f7b3b80 10 client.0 _ll_register_callbacks cb 0x5621ebd01290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:36:56.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.701+0000 7f8d89200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5621ebe12010 0x5621ebe77040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:56.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.701+0000 7f8d89200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5621ebe12010 0x5621ebe77040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59364/0 (socket says 172.21.15.1:59364) 2024-02-02T15:36:56.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.701+0000 7f8d83e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621ebe12930 0x5621ebe77580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:56.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.701+0000 7f8d89200700 1 -- 172.21.15.1:0/2236789192 learned_addr learned my addr 172.21.15.1:0/2236789192 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:36:56.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.701+0000 7f8d89c00700 1 --2- 172.21.15.1:0/2236789192 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621ebe1d7e0 0x5621ebe729d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:56.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.701+0000 7f8d89200700 1 -- 172.21.15.1:0/2236789192 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621ebe1d7e0 msgr2=0x5621ebe729d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:56.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.701+0000 7f8d89200700 1 --2- 172.21.15.1:0/2236789192 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621ebe1d7e0 0x5621ebe729d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.701+0000 7f8d89200700 1 -- 172.21.15.1:0/2236789192 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621ebe12930 msgr2=0x5621ebe77580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:56.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.701+0000 7f8d89200700 1 --2- 172.21.15.1:0/2236789192 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621ebe12930 0x5621ebe77580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.701+0000 7f8d89200700 1 -- 172.21.15.1:0/2236789192 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5621ebb91b60 con 0x5621ebe12010 2024-02-02T15:36:56.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.701+0000 7f8d89c00700 1 --2- 172.21.15.1:0/2236789192 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621ebe1d7e0 0x5621ebe729d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:36:56.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.702+0000 7f8d89200700 1 --2- 172.21.15.1:0/2236789192 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5621ebe12010 0x5621ebe77040 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f8d740059d0 tx=0x7f8d74012fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56ee23097cc55d03 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:56.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.702+0000 7f8d72a00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:36:56.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.702+0000 7f8d72a00700 1 -- 172.21.15.1:0/2236789192 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d74016040 con 0x5621ebe12010 2024-02-02T15:36:56.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.702+0000 7f8d72a00700 1 -- 172.21.15.1:0/2236789192 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f8d74000e40 con 0x5621ebe12010 2024-02-02T15:36:56.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.702+0000 7f8d9f7b3b80 1 -- 172.21.15.1:0/2236789192 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5621ebd22f50 con 0x5621ebe12010 2024-02-02T15:36:56.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.702+0000 7f8d72a00700 1 -- 172.21.15.1:0/2236789192 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d74004020 con 0x5621ebe12010 2024-02-02T15:36:56.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.703+0000 7f8d9f7b3b80 10 client.28277 fetch_fsmap learned FSMap version 46 2024-02-02T15:36:56.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.703+0000 7f8d9f7b3b80 1 -- 172.21.15.1:0/2236789192 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x5621ebbe9cd0 con 0x5621ebe12010 2024-02-02T15:36:56.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.703+0000 7f8d72a00700 1 -- 172.21.15.1:0/2236789192 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5621ebd22f50 con 0x5621ebe12010 2024-02-02T15:36:56.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.703+0000 7f8d72a00700 1 -- 172.21.15.1:0/2236789192 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 46) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f8d74003710 con 0x5621ebe12010 2024-02-02T15:36:56.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.703+0000 7f8d9f7b3b80 10 client.28277 fetch_fsmap finished waiting for FSMap version 46 2024-02-02T15:36:56.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.703+0000 7f8d9f7b3b80 10 client.28277 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-02-02T15:36:56.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.703+0000 7f8d9f7b3b80 20 client.28277 populate_metadata read hostname 'smithi001' 2024-02-02T15:36:56.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.703+0000 7f8d9f7b3b80 1 --2- 172.21.15.1:0/2236789192 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x5621ebe7fc20 0x5621ebe82010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:56.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.703+0000 7f8d9f7b3b80 4 client.28277 mds_command: new command op to 24518 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:36:56.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.703+0000 7f8d9f7b3b80 1 -- 172.21.15.1:0/2236789192 --> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5621ebbb2cf0 con 0x5621ebe7fc20 2024-02-02T15:36:56.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.704+0000 7f8d83e00700 1 --2- 172.21.15.1:0/2236789192 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x5621ebe7fc20 0x5621ebe82010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:56.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.704+0000 7f8d83e00700 1 --2- 172.21.15.1:0/2236789192 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x5621ebe7fc20 0x5621ebe82010 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=292a4029050c3460 server_cookie=51972efcf2c0666c in_seq=0 out_seq=0 2024-02-02T15:36:56.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.704+0000 7f8d72a00700 10 client.28277 ms_handle_connect on v2:172.21.15.71:6836/628016199 2024-02-02T15:36:56.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.704+0000 7f8d72a00700 1 -- 172.21.15.1:0/2236789192 <== mds.2 v2:172.21.15.71:6836/628016199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5621ebbb2cf0 con 0x5621ebe7fc20 2024-02-02T15:36:56.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.704+0000 7f8d72a00700 10 client.28277 handle_command_reply: tid=0 2024-02-02T15:36:56.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.704+0000 7f8d9f7b3b80 1 client.28277 shutdown 2024-02-02T15:36:56.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.705+0000 7f8d9f7b3b80 1 -- 172.21.15.1:0/2236789192 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5621ebe12010 msgr2=0x5621ebe77040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:56.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.705+0000 7f8d9f7b3b80 1 --2- 172.21.15.1:0/2236789192 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5621ebe12010 0x5621ebe77040 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f8d740059d0 tx=0x7f8d74012fb0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.705+0000 7f8d9f7b3b80 1 -- 172.21.15.1:0/2236789192 shutdown_connections 2024-02-02T15:36:56.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.705+0000 7f8d9f7b3b80 1 --2- 172.21.15.1:0/2236789192 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x5621ebe7fc20 0x5621ebe82010 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.705+0000 7f8d9f7b3b80 1 --2- 172.21.15.1:0/2236789192 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621ebe1d7e0 0x5621ebe729d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.705+0000 7f8d72a00700 0 client.28277 ms_handle_reset on v2:172.21.15.71:6836/628016199 2024-02-02T15:36:56.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.705+0000 7f8d9f7b3b80 1 --2- 172.21.15.1:0/2236789192 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5621ebe12010 0x5621ebe77040 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.705+0000 7f8d9f7b3b80 1 --2- 172.21.15.1:0/2236789192 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621ebe12930 0x5621ebe77580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.705+0000 7f8d9f7b3b80 1 -- 172.21.15.1:0/2236789192 >> 172.21.15.1:0/2236789192 conn(0x5621ebd71db0 msgr2=0x5621ebe1dbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:56.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.705+0000 7f8d9f7b3b80 1 -- 172.21.15.1:0/2236789192 shutdown_connections 2024-02-02T15:36:56.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.705+0000 7f8d9f7b3b80 1 -- 172.21.15.1:0/2236789192 wait complete. 2024-02-02T15:36:56.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.705+0000 7f8d9f7b3b80 20 client.28277 trim_cache size 0 max 16384 2024-02-02T15:36:56.709 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:36:56.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.706+0000 7f8d8bc00700 1 -- 172.21.15.1:0/1177944345 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f8d640a0690 msgr2=0x7f8d640a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:56.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.706+0000 7f8d8bc00700 1 --2- 172.21.15.1:0/1177944345 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f8d640a0690 0x7f8d640a2b50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c009550 tx=0x7f8d7c00b040 comp rx=0 tx=0).stop 2024-02-02T15:36:56.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.706+0000 7f8d8bc00700 1 -- 172.21.15.1:0/1177944345 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d84073a10 msgr2=0x7f8d84199610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:56.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.706+0000 7f8d8bc00700 1 --2- 172.21.15.1:0/1177944345 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d84073a10 0x7f8d84199610 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c0095c0 tx=0x7f8d6c00f690 comp rx=0 tx=0).stop 2024-02-02T15:36:56.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.706+0000 7f8d8bc00700 1 -- 172.21.15.1:0/1177944345 shutdown_connections 2024-02-02T15:36:56.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.706+0000 7f8d8bc00700 1 --2- 172.21.15.1:0/1177944345 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f8d640a0690 0x7f8d640a2b50 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.706+0000 7f8d8bc00700 1 --2- 172.21.15.1:0/1177944345 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8d8410d5f0 0x7f8d8419da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.706+0000 7f8d8bc00700 1 --2- 172.21.15.1:0/1177944345 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8d840730f0 0x7f8d841990d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.706+0000 7f8d8bc00700 1 --2- 172.21.15.1:0/1177944345 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8d84073a10 0x7f8d84199610 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.706+0000 7f8d8bc00700 1 -- 172.21.15.1:0/1177944345 >> 172.21.15.1:0/1177944345 conn(0x7f8d840fc800 msgr2=0x7f8d8410f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:56.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.706+0000 7f8d8bc00700 1 -- 172.21.15.1:0/1177944345 shutdown_connections 2024-02-02T15:36:56.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.707+0000 7f8d8bc00700 1 -- 172.21.15.1:0/1177944345 wait complete. 2024-02-02T15:36:56.727 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-02T15:36:56.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:56 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/2716882558' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:36:56.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:56 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/2716882558' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:36:56.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.939+0000 7f3045000700 1 -- 172.21.15.1:0/2197225120 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3040102950 msgr2=0x7f304010ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:56.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.939+0000 7f3045000700 1 --2- 172.21.15.1:0/2197225120 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3040102950 0x7f304010ae30 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f3028009990 tx=0x7f3028009ca0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.939+0000 7f3045000700 1 -- 172.21.15.1:0/2197225120 shutdown_connections 2024-02-02T15:36:56.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.939+0000 7f3045000700 1 --2- 172.21.15.1:0/2197225120 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f304010b400 0x7f304010d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.939+0000 7f3045000700 1 --2- 172.21.15.1:0/2197225120 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3040102030 0x7f3040102410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.939+0000 7f3045000700 1 --2- 172.21.15.1:0/2197225120 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3040102950 0x7f304010ae30 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.939+0000 7f3045000700 1 -- 172.21.15.1:0/2197225120 >> 172.21.15.1:0/2197225120 conn(0x7f3040076d70 msgr2=0x7f3040077180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:56.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.939+0000 7f3045000700 1 -- 172.21.15.1:0/2197225120 shutdown_connections 2024-02-02T15:36:56.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.939+0000 7f3045000700 1 -- 172.21.15.1:0/2197225120 wait complete. 2024-02-02T15:36:56.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.940+0000 7f3045000700 1 Processor -- start 2024-02-02T15:36:56.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.940+0000 7f3045000700 1 -- start start 2024-02-02T15:36:56.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.941+0000 7f3045000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3040102030 0x7f30400ffc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:56.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.941+0000 7f3045000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3040102950 0x7f30401001c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:56.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.941+0000 7f3045000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f304010b400 0x7f30401a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:56.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.941+0000 7f3045000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30401009f0 con 0x7f3040102030 2024-02-02T15:36:56.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.941+0000 7f3045000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30401a5620 con 0x7f3040102950 2024-02-02T15:36:56.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.941+0000 7f3045000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30401a5760 con 0x7f304010b400 2024-02-02T15:36:56.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.941+0000 7f303de00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3040102030 0x7f30400ffc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:56.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.941+0000 7f303de00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3040102030 0x7f30400ffc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48806/0 (socket says 172.21.15.1:48806) 2024-02-02T15:36:56.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.941+0000 7f303de00700 1 -- 172.21.15.1:0/475474149 learned_addr learned my addr 172.21.15.1:0/475474149 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:36:56.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.941+0000 7f303e800700 1 --2- 172.21.15.1:0/475474149 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f304010b400 0x7f30401a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:56.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.941+0000 7f303d400700 1 --2- 172.21.15.1:0/475474149 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3040102950 0x7f30401001c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:56.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.941+0000 7f303de00700 1 -- 172.21.15.1:0/475474149 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f304010b400 msgr2=0x7f30401a50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:56.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.941+0000 7f303de00700 1 --2- 172.21.15.1:0/475474149 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f304010b400 0x7f30401a50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.941+0000 7f303de00700 1 -- 172.21.15.1:0/475474149 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3040102950 msgr2=0x7f30401001c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:56.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.941+0000 7f303de00700 1 --2- 172.21.15.1:0/475474149 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3040102950 0x7f30401001c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.942+0000 7f303de00700 1 -- 172.21.15.1:0/475474149 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30280095f0 con 0x7f3040102030 2024-02-02T15:36:56.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.942+0000 7f303de00700 1 --2- 172.21.15.1:0/475474149 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3040102030 0x7f30400ffc80 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f303000e9a0 tx=0x7f303000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:56.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.942+0000 7f3036a00700 1 -- 172.21.15.1:0/475474149 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3030009790 con 0x7f3040102030 2024-02-02T15:36:56.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.942+0000 7f3045000700 1 -- 172.21.15.1:0/475474149 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30401a59c0 con 0x7f3040102030 2024-02-02T15:36:56.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.942+0000 7f3036a00700 1 -- 172.21.15.1:0/475474149 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3030010430 con 0x7f3040102030 2024-02-02T15:36:56.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.942+0000 7f3036a00700 1 -- 172.21.15.1:0/475474149 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3030018620 con 0x7f3040102030 2024-02-02T15:36:56.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.942+0000 7f3045000700 1 -- 172.21.15.1:0/475474149 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30401a5ea0 con 0x7f3040102030 2024-02-02T15:36:56.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.947+0000 7f3036a00700 1 -- 172.21.15.1:0/475474149 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f30300105a0 con 0x7f3040102030 2024-02-02T15:36:56.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.948+0000 7f3058b4eb80 1 Processor -- start 2024-02-02T15:36:56.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.948+0000 7f3058b4eb80 1 -- start start 2024-02-02T15:36:56.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.948+0000 7f3036a00700 1 --2- 172.21.15.1:0/475474149 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f30200a0690 0x7f30200a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:56.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.948+0000 7f3058b4eb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bdfe4f3010 0x55bdfe4f33f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:56.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.948+0000 7f3036a00700 1 -- 172.21.15.1:0/475474149 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f3030014070 con 0x7f3040102030 2024-02-02T15:36:56.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.948+0000 7f3058b4eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bdfe4f3930 0x55bdfe4fe2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:56.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.948+0000 7f3058b4eb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bdfe4fe7e0 0x55bdfe500bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:56.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.948+0000 7f3058b4eb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdfe4674c0 con 0x55bdfe4fe7e0 2024-02-02T15:36:56.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.948+0000 7f3058b4eb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdfe44baa0 con 0x55bdfe4f3010 2024-02-02T15:36:56.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.948+0000 7f3058b4eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdfe449460 con 0x55bdfe4f3930 2024-02-02T15:36:56.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.948+0000 7f303e800700 1 --2- 172.21.15.1:0/475474149 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f30200a0690 0x7f30200a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:56.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.949+0000 7f303d400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bdfe4f3930 0x55bdfe4fe2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:56.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.949+0000 7f303de00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bdfe4f3010 0x55bdfe4f33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:56.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.949+0000 7f303e800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bdfe4fe7e0 0x55bdfe500bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:56.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.949+0000 7f303d400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bdfe4f3930 0x55bdfe4fe2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:34180/0 (socket says 172.21.15.1:34180) 2024-02-02T15:36:56.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.949+0000 7f303de00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bdfe4f3010 0x55bdfe4f33f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59386/0 (socket says 172.21.15.1:59386) 2024-02-02T15:36:56.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.949+0000 7f303d400700 1 -- 172.21.15.1:0/2272429216 learned_addr learned my addr 172.21.15.1:0/2272429216 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:36:56.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.949+0000 7f303de00700 1 -- 172.21.15.1:0/2272429216 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bdfe4f3930 msgr2=0x55bdfe4fe2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:56.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.949+0000 7f303de00700 1 --2- 172.21.15.1:0/2272429216 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bdfe4f3930 0x55bdfe4fe2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.949+0000 7f303de00700 1 -- 172.21.15.1:0/2272429216 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bdfe4fe7e0 msgr2=0x55bdfe500bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:56.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.949+0000 7f303de00700 1 --2- 172.21.15.1:0/2272429216 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bdfe4fe7e0 0x55bdfe500bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.949+0000 7f303de00700 1 -- 172.21.15.1:0/2272429216 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bdfe2c63b0 con 0x55bdfe4f3010 2024-02-02T15:36:56.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.949+0000 7f303e800700 1 --2- 172.21.15.1:0/475474149 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f30200a0690 0x7f30200a2b50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3038009e10 tx=0x7f30380092d0 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:56.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.949+0000 7f303de00700 1 --2- 172.21.15.1:0/2272429216 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bdfe4f3010 0x55bdfe4f33f0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f30300c9630 tx=0x7f3030098d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b06d738501d83cd server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:56.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.949+0000 7f302f400700 1 -- 172.21.15.1:0/2272429216 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30300840b0 con 0x55bdfe4f3010 2024-02-02T15:36:56.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.950+0000 7f302f400700 1 -- 172.21.15.1:0/2272429216 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3030099430 con 0x55bdfe4f3010 2024-02-02T15:36:56.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.950+0000 7f302f400700 1 -- 172.21.15.1:0/2272429216 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30300998f0 con 0x55bdfe4f3010 2024-02-02T15:36:56.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.950+0000 7f3058b4eb80 1 -- 172.21.15.1:0/2272429216 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bdfe4f3010 msgr2=0x55bdfe4f33f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:56.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.950+0000 7f3058b4eb80 1 --2- 172.21.15.1:0/2272429216 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bdfe4f3010 0x55bdfe4f33f0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f30300c9630 tx=0x7f3030098d90 comp rx=0 tx=0).stop 2024-02-02T15:36:56.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.950+0000 7f3058b4eb80 1 -- 172.21.15.1:0/2272429216 shutdown_connections 2024-02-02T15:36:56.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.950+0000 7f3058b4eb80 1 --2- 172.21.15.1:0/2272429216 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bdfe4fe7e0 0x55bdfe500bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.950+0000 7f3058b4eb80 1 --2- 172.21.15.1:0/2272429216 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bdfe4f3010 0x55bdfe4f33f0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.950+0000 7f3058b4eb80 1 --2- 172.21.15.1:0/2272429216 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bdfe4f3930 0x55bdfe4fe2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.950+0000 7f3058b4eb80 1 -- 172.21.15.1:0/2272429216 >> 172.21.15.1:0/2272429216 conn(0x55bdfe452db0 msgr2=0x55bdfe4ef3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:56.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.950+0000 7f3058b4eb80 1 -- 172.21.15.1:0/2272429216 shutdown_connections 2024-02-02T15:36:56.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.950+0000 7f3058b4eb80 1 -- 172.21.15.1:0/2272429216 wait complete. 2024-02-02T15:36:56.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.951+0000 7f3058b4eb80 1 -- start start 2024-02-02T15:36:56.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.951+0000 7f3058b4eb80 1 Processor -- start 2024-02-02T15:36:56.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.952+0000 7f3058b4eb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bdfe4f3010 0x55bdfe4fe220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:56.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.952+0000 7f3058b4eb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bdfe4f3930 0x55bdfe4fb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:56.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.952+0000 7f3058b4eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bdfe4fe7e0 0x55bdfe4fbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:56.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.952+0000 7f3058b4eb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdfe444d50 con 0x55bdfe4f3930 2024-02-02T15:36:56.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.952+0000 7f3058b4eb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdfe3e0d80 con 0x55bdfe4f3010 2024-02-02T15:36:56.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.952+0000 7f3058b4eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdfe3e0a20 con 0x55bdfe4fe7e0 2024-02-02T15:36:56.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.952+0000 7f3058b4eb80 10 client.0 _ll_register_callbacks cb 0x55bdfe3e2290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:36:56.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.952+0000 7f303de00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bdfe4f3010 0x55bdfe4fe220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:56.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.952+0000 7f303de00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bdfe4f3010 0x55bdfe4fe220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59390/0 (socket says 172.21.15.1:59390) 2024-02-02T15:36:56.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.952+0000 7f303de00700 1 -- 172.21.15.1:0/2883237854 learned_addr learned my addr 172.21.15.1:0/2883237854 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:36:56.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.952+0000 7f303d400700 1 --2- 172.21.15.1:0/2883237854 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bdfe4f3930 0x55bdfe4fb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:56.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.952+0000 7f303e800700 1 --2- 172.21.15.1:0/2883237854 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bdfe4fe7e0 0x55bdfe4fbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:56.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.952+0000 7f303de00700 1 -- 172.21.15.1:0/2883237854 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bdfe4fe7e0 msgr2=0x55bdfe4fbbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:56.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.952+0000 7f303de00700 1 --2- 172.21.15.1:0/2883237854 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bdfe4fe7e0 0x55bdfe4fbbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.952+0000 7f303de00700 1 -- 172.21.15.1:0/2883237854 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bdfe4f3930 msgr2=0x55bdfe4fb4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:56.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.952+0000 7f303de00700 1 --2- 172.21.15.1:0/2883237854 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bdfe4f3930 0x55bdfe4fb4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.952+0000 7f303de00700 1 -- 172.21.15.1:0/2883237854 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30401a5ea0 con 0x55bdfe4f3010 2024-02-02T15:36:56.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.952+0000 7f303de00700 1 --2- 172.21.15.1:0/2883237854 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bdfe4f3010 0x55bdfe4fe220 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f30300c9c60 tx=0x7f3030010ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35ed55af89744dd8 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:56.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.952+0000 7f3027e00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:36:56.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.953+0000 7f3027e00700 1 -- 172.21.15.1:0/2883237854 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3030084040 con 0x55bdfe4f3010 2024-02-02T15:36:56.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.953+0000 7f3027e00700 1 -- 172.21.15.1:0/2883237854 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f303009d260 con 0x55bdfe4f3010 2024-02-02T15:36:56.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.953+0000 7f3058b4eb80 1 -- 172.21.15.1:0/2883237854 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bdfe4e70f0 con 0x55bdfe4f3010 2024-02-02T15:36:56.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.953+0000 7f3027e00700 1 -- 172.21.15.1:0/2883237854 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f30300a14a0 con 0x55bdfe4f3010 2024-02-02T15:36:56.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.953+0000 7f3058b4eb80 10 client.28289 fetch_fsmap learned FSMap version 46 2024-02-02T15:36:56.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.953+0000 7f3058b4eb80 1 -- 172.21.15.1:0/2883237854 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x55bdfe272b60 con 0x55bdfe4f3010 2024-02-02T15:36:56.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.953+0000 7f3027e00700 1 -- 172.21.15.1:0/2883237854 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bdfe4e70f0 con 0x55bdfe4f3010 2024-02-02T15:36:56.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.953+0000 7f3027e00700 1 -- 172.21.15.1:0/2883237854 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 46) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f30300a1010 con 0x55bdfe4f3010 2024-02-02T15:36:56.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.953+0000 7f3058b4eb80 10 client.28289 fetch_fsmap finished waiting for FSMap version 46 2024-02-02T15:36:56.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.953+0000 7f3058b4eb80 10 client.28289 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-02T15:36:56.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.953+0000 7f3058b4eb80 20 client.28289 populate_metadata read hostname 'smithi001' 2024-02-02T15:36:56.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.953+0000 7f3058b4eb80 1 --2- 172.21.15.1:0/2883237854 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55bdfe50dde0 0x55bdfe510200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:56.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.953+0000 7f3058b4eb80 4 client.28289 mds_command: new command op to 24529 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:36:56.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.953+0000 7f3058b4eb80 1 -- 172.21.15.1:0/2883237854 --> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bdfe43ddd0 con 0x55bdfe50dde0 2024-02-02T15:36:56.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.954+0000 7f303d400700 1 --2- 172.21.15.1:0/2883237854 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55bdfe50dde0 0x55bdfe510200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:56.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.954+0000 7f303d400700 1 --2- 172.21.15.1:0/2883237854 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55bdfe50dde0 0x55bdfe510200 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=65ee31f9da6ecf6b server_cookie=57644df296e2ffd6 in_seq=0 out_seq=0 2024-02-02T15:36:56.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.954+0000 7f3027e00700 10 client.28289 ms_handle_connect on v2:172.21.15.142:6838/321675421 2024-02-02T15:36:56.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.955+0000 7f3027e00700 1 -- 172.21.15.1:0/2883237854 <== mds.1 v2:172.21.15.142:6838/321675421 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55bdfe43ddd0 con 0x55bdfe50dde0 2024-02-02T15:36:56.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.955+0000 7f3027e00700 10 client.28289 handle_command_reply: tid=0 2024-02-02T15:36:56.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.955+0000 7f3058b4eb80 1 client.28289 shutdown 2024-02-02T15:36:56.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.955+0000 7f3058b4eb80 1 -- 172.21.15.1:0/2883237854 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bdfe4f3010 msgr2=0x55bdfe4fe220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:56.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.955+0000 7f3058b4eb80 1 --2- 172.21.15.1:0/2883237854 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bdfe4f3010 0x55bdfe4fe220 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f30300c9c60 tx=0x7f3030010ab0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.955+0000 7f3058b4eb80 1 -- 172.21.15.1:0/2883237854 shutdown_connections 2024-02-02T15:36:56.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.955+0000 7f3058b4eb80 1 --2- 172.21.15.1:0/2883237854 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55bdfe50dde0 0x55bdfe510200 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.955+0000 7f3058b4eb80 1 --2- 172.21.15.1:0/2883237854 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bdfe4fe7e0 0x55bdfe4fbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.955+0000 7f3027e00700 0 client.28289 ms_handle_reset on v2:172.21.15.142:6838/321675421 2024-02-02T15:36:56.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.955+0000 7f3058b4eb80 1 --2- 172.21.15.1:0/2883237854 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bdfe4f3010 0x55bdfe4fe220 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.955+0000 7f3058b4eb80 1 --2- 172.21.15.1:0/2883237854 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bdfe4f3930 0x55bdfe4fb4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.955+0000 7f3058b4eb80 1 -- 172.21.15.1:0/2883237854 >> 172.21.15.1:0/2883237854 conn(0x55bdfe452db0 msgr2=0x55bdfe4eefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:56.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.955+0000 7f3058b4eb80 1 -- 172.21.15.1:0/2883237854 shutdown_connections 2024-02-02T15:36:56.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.955+0000 7f3058b4eb80 1 -- 172.21.15.1:0/2883237854 wait complete. 2024-02-02T15:36:56.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.956+0000 7f3058b4eb80 20 client.28289 trim_cache size 0 max 16384 2024-02-02T15:36:56.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.970+0000 7f3058b4eb80 1 Processor -- start 2024-02-02T15:36:56.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.970+0000 7f3058b4eb80 1 -- start start 2024-02-02T15:36:56.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.971+0000 7f3058b4eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bdfe4f3010 0x55bdfe500bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:56.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.971+0000 7f3058b4eb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bdfe4f3930 0x55bdfe4fa730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:56.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.971+0000 7f3058b4eb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bdfe4fe7e0 0x55bdfe4fad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:56.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.971+0000 7f3058b4eb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdfe2c86a0 con 0x55bdfe4f3930 2024-02-02T15:36:56.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.971+0000 7f3058b4eb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdfe4fb280 con 0x55bdfe4fe7e0 2024-02-02T15:36:56.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.971+0000 7f3058b4eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdfe4f7070 con 0x55bdfe4f3010 2024-02-02T15:36:56.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.971+0000 7f303e800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bdfe4fe7e0 0x55bdfe4fad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:56.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.971+0000 7f303de00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bdfe4f3010 0x55bdfe500bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:56.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.971+0000 7f303e800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bdfe4fe7e0 0x55bdfe4fad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59404/0 (socket says 172.21.15.1:59404) 2024-02-02T15:36:56.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.971+0000 7f303e800700 1 -- 172.21.15.1:0/1758853480 learned_addr learned my addr 172.21.15.1:0/1758853480 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:36:56.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.971+0000 7f303d400700 1 --2- 172.21.15.1:0/1758853480 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bdfe4f3930 0x55bdfe4fa730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:56.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.971+0000 7f303e800700 1 -- 172.21.15.1:0/1758853480 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bdfe4f3010 msgr2=0x55bdfe500bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:56.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.971+0000 7f303e800700 1 --2- 172.21.15.1:0/1758853480 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bdfe4f3010 0x55bdfe500bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.971+0000 7f303e800700 1 -- 172.21.15.1:0/1758853480 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bdfe4f3930 msgr2=0x55bdfe4fa730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:56.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.971+0000 7f303e800700 1 --2- 172.21.15.1:0/1758853480 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bdfe4f3930 0x55bdfe4fa730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.971+0000 7f303e800700 1 -- 172.21.15.1:0/1758853480 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bdfe40aea0 con 0x55bdfe4fe7e0 2024-02-02T15:36:56.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.971+0000 7f303de00700 1 --2- 172.21.15.1:0/1758853480 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bdfe4f3010 0x55bdfe500bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:36:56.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.971+0000 7f303d400700 1 --2- 172.21.15.1:0/1758853480 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bdfe4f3930 0x55bdfe4fa730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:36:56.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.972+0000 7f303e800700 1 --2- 172.21.15.1:0/1758853480 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bdfe4fe7e0 0x55bdfe4fad40 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f30380091b0 tx=0x7f303801d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2224d5bcb83d8a1 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:56.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.972+0000 7f302fe00700 1 -- 172.21.15.1:0/1758853480 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f303800e070 con 0x55bdfe4fe7e0 2024-02-02T15:36:56.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.972+0000 7f302fe00700 1 -- 172.21.15.1:0/1758853480 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f303800f040 con 0x55bdfe4fe7e0 2024-02-02T15:36:56.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.972+0000 7f302fe00700 1 -- 172.21.15.1:0/1758853480 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3038004020 con 0x55bdfe4fe7e0 2024-02-02T15:36:56.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.972+0000 7f3058b4eb80 1 -- 172.21.15.1:0/1758853480 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bdfe4fe7e0 msgr2=0x55bdfe4fad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:56.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.972+0000 7f3058b4eb80 1 --2- 172.21.15.1:0/1758853480 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bdfe4fe7e0 0x55bdfe4fad40 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f30380091b0 tx=0x7f303801d000 comp rx=0 tx=0).stop 2024-02-02T15:36:56.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.973+0000 7f3058b4eb80 1 -- 172.21.15.1:0/1758853480 shutdown_connections 2024-02-02T15:36:56.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.973+0000 7f3058b4eb80 1 --2- 172.21.15.1:0/1758853480 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bdfe4fe7e0 0x55bdfe4fad40 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.973+0000 7f3058b4eb80 1 --2- 172.21.15.1:0/1758853480 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bdfe4f3010 0x55bdfe500bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.973+0000 7f3058b4eb80 1 --2- 172.21.15.1:0/1758853480 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bdfe4f3930 0x55bdfe4fa730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.973+0000 7f3058b4eb80 1 -- 172.21.15.1:0/1758853480 >> 172.21.15.1:0/1758853480 conn(0x55bdfe452db0 msgr2=0x55bdfe4f4830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:56.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.973+0000 7f3058b4eb80 1 -- 172.21.15.1:0/1758853480 shutdown_connections 2024-02-02T15:36:56.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.973+0000 7f3058b4eb80 1 -- 172.21.15.1:0/1758853480 wait complete. 2024-02-02T15:36:56.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.973+0000 7f3058b4eb80 1 -- start start 2024-02-02T15:36:56.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.973+0000 7f3058b4eb80 1 Processor -- start 2024-02-02T15:36:56.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.974+0000 7f3058b4eb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bdfe4f3010 0x55bdfe5034a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:56.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.974+0000 7f3058b4eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bdfe4f3930 0x55bdfe5039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:56.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.974+0000 7f3058b4eb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bdfe4fe7e0 0x55bdfe558940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:56.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.974+0000 7f3058b4eb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdfe558e80 con 0x55bdfe4fe7e0 2024-02-02T15:36:56.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.974+0000 7f3058b4eb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdfe558ff0 con 0x55bdfe4f3010 2024-02-02T15:36:56.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.974+0000 7f3058b4eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bdfe559160 con 0x55bdfe4f3930 2024-02-02T15:36:56.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.974+0000 7f3058b4eb80 10 client.0 _ll_register_callbacks cb 0x55bdfe3e2290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:36:56.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.974+0000 7f303de00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bdfe4f3010 0x55bdfe5034a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:56.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.974+0000 7f303d400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bdfe4f3930 0x55bdfe5039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:56.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.974+0000 7f303de00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bdfe4f3010 0x55bdfe5034a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59416/0 (socket says 172.21.15.1:59416) 2024-02-02T15:36:56.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.974+0000 7f303de00700 1 -- 172.21.15.1:0/4113770146 learned_addr learned my addr 172.21.15.1:0/4113770146 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:36:56.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.974+0000 7f303e800700 1 --2- 172.21.15.1:0/4113770146 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bdfe4fe7e0 0x55bdfe558940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:56.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.974+0000 7f303d400700 1 -- 172.21.15.1:0/4113770146 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bdfe4f3010 msgr2=0x55bdfe5034a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:56.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.974+0000 7f303d400700 1 --2- 172.21.15.1:0/4113770146 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bdfe4f3010 0x55bdfe5034a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.974+0000 7f303d400700 1 -- 172.21.15.1:0/4113770146 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bdfe4fe7e0 msgr2=0x55bdfe558940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:56.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.974+0000 7f303d400700 1 --2- 172.21.15.1:0/4113770146 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bdfe4fe7e0 0x55bdfe558940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.974+0000 7f303d400700 1 -- 172.21.15.1:0/4113770146 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bdfe272b60 con 0x55bdfe4f3930 2024-02-02T15:36:56.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.975+0000 7f303de00700 1 --2- 172.21.15.1:0/4113770146 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bdfe4f3010 0x55bdfe5034a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:36:56.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.975+0000 7f303d400700 1 --2- 172.21.15.1:0/4113770146 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bdfe4f3930 0x55bdfe5039e0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f302800b2e0 tx=0x7f302800ba10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87f60a5b346de64f server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:56.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.975+0000 7f3027e00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:36:56.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.975+0000 7f3027e00700 1 -- 172.21.15.1:0/4113770146 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f302801d040 con 0x55bdfe4f3930 2024-02-02T15:36:56.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.975+0000 7f3027e00700 1 -- 172.21.15.1:0/4113770146 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f30280290b0 con 0x55bdfe4f3930 2024-02-02T15:36:56.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.975+0000 7f3058b4eb80 1 -- 172.21.15.1:0/4113770146 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3030099430 con 0x55bdfe4f3930 2024-02-02T15:36:56.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.975+0000 7f3027e00700 1 -- 172.21.15.1:0/4113770146 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3028029270 con 0x55bdfe4f3930 2024-02-02T15:36:56.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.975+0000 7f3058b4eb80 10 client.28294 fetch_fsmap learned FSMap version 46 2024-02-02T15:36:56.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.975+0000 7f3058b4eb80 1 -- 172.21.15.1:0/4113770146 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x7f304010e560 con 0x55bdfe4f3930 2024-02-02T15:36:56.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.976+0000 7f3027e00700 1 -- 172.21.15.1:0/4113770146 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3030099430 con 0x55bdfe4f3930 2024-02-02T15:36:56.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.976+0000 7f3027e00700 1 -- 172.21.15.1:0/4113770146 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 46) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f3028004500 con 0x55bdfe4f3930 2024-02-02T15:36:56.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.976+0000 7f3058b4eb80 10 client.28294 fetch_fsmap finished waiting for FSMap version 46 2024-02-02T15:36:56.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.976+0000 7f3058b4eb80 10 client.28294 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-02T15:36:56.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.976+0000 7f3058b4eb80 20 client.28294 populate_metadata read hostname 'smithi001' 2024-02-02T15:36:56.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.976+0000 7f3058b4eb80 1 --2- 172.21.15.1:0/4113770146 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55bdfe5608d0 0x55bdfe562cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:56.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.976+0000 7f3058b4eb80 4 client.28294 mds_command: new command op to 24529 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:36:56.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.976+0000 7f3058b4eb80 1 -- 172.21.15.1:0/4113770146 --> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55bdfe403f50 con 0x55bdfe5608d0 2024-02-02T15:36:56.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.976+0000 7f303de00700 1 --2- 172.21.15.1:0/4113770146 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55bdfe5608d0 0x55bdfe562cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:56.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.977+0000 7f3027e00700 10 client.28294 ms_handle_connect on v2:172.21.15.142:6838/321675421 2024-02-02T15:36:56.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.977+0000 7f303de00700 1 --2- 172.21.15.1:0/4113770146 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55bdfe5608d0 0x55bdfe562cf0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5d06a2ad492b34ac server_cookie=64c264621990d9ed in_seq=0 out_seq=0 2024-02-02T15:36:56.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.977+0000 7f3027e00700 1 -- 172.21.15.1:0/4113770146 <== mds.1 v2:172.21.15.142:6838/321675421 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55bdfe403f50 con 0x55bdfe5608d0 2024-02-02T15:36:56.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.977+0000 7f3027e00700 10 client.28294 handle_command_reply: tid=0 2024-02-02T15:36:56.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.977+0000 7f3058b4eb80 1 client.28294 shutdown 2024-02-02T15:36:56.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.977+0000 7f3058b4eb80 1 -- 172.21.15.1:0/4113770146 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bdfe4f3930 msgr2=0x55bdfe5039e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:56.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.977+0000 7f3058b4eb80 1 --2- 172.21.15.1:0/4113770146 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bdfe4f3930 0x55bdfe5039e0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f302800b2e0 tx=0x7f302800ba10 comp rx=0 tx=0).stop 2024-02-02T15:36:56.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.977+0000 7f3058b4eb80 1 -- 172.21.15.1:0/4113770146 shutdown_connections 2024-02-02T15:36:56.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.977+0000 7f3058b4eb80 1 --2- 172.21.15.1:0/4113770146 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55bdfe5608d0 0x55bdfe562cf0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.977+0000 7f3058b4eb80 1 --2- 172.21.15.1:0/4113770146 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bdfe4fe7e0 0x55bdfe558940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.977+0000 7f3058b4eb80 1 --2- 172.21.15.1:0/4113770146 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bdfe4f3010 0x55bdfe5034a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.977+0000 7f3027e00700 0 client.28294 ms_handle_reset on v2:172.21.15.142:6838/321675421 2024-02-02T15:36:56.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.977+0000 7f3058b4eb80 1 --2- 172.21.15.1:0/4113770146 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bdfe4f3930 0x55bdfe5039e0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.977+0000 7f3058b4eb80 1 -- 172.21.15.1:0/4113770146 >> 172.21.15.1:0/4113770146 conn(0x55bdfe452db0 msgr2=0x55bdfe5098a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:56.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.977+0000 7f3058b4eb80 1 -- 172.21.15.1:0/4113770146 shutdown_connections 2024-02-02T15:36:56.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.977+0000 7f3058b4eb80 1 -- 172.21.15.1:0/4113770146 wait complete. 2024-02-02T15:36:56.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.978+0000 7f3058b4eb80 20 client.28294 trim_cache size 0 max 16384 2024-02-02T15:36:56.983 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:36:56.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.978+0000 7f3045000700 1 -- 172.21.15.1:0/475474149 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f30200a0690 msgr2=0x7f30200a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:56.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.978+0000 7f3045000700 1 --2- 172.21.15.1:0/475474149 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f30200a0690 0x7f30200a2b50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f3038009e10 tx=0x7f30380092d0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.978+0000 7f3045000700 1 -- 172.21.15.1:0/475474149 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3040102030 msgr2=0x7f30400ffc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:56.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.978+0000 7f3045000700 1 --2- 172.21.15.1:0/475474149 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3040102030 0x7f30400ffc80 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f303000e9a0 tx=0x7f303000ecb0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.978+0000 7f3045000700 1 -- 172.21.15.1:0/475474149 shutdown_connections 2024-02-02T15:36:56.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.978+0000 7f3045000700 1 --2- 172.21.15.1:0/475474149 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f30200a0690 0x7f30200a2b50 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.978+0000 7f3045000700 1 --2- 172.21.15.1:0/475474149 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f304010b400 0x7f30401a50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.978+0000 7f3045000700 1 --2- 172.21.15.1:0/475474149 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3040102030 0x7f30400ffc80 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.978+0000 7f3045000700 1 --2- 172.21.15.1:0/475474149 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3040102950 0x7f30401001c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:56.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.978+0000 7f3045000700 1 -- 172.21.15.1:0/475474149 >> 172.21.15.1:0/475474149 conn(0x7f3040076d70 msgr2=0x7f30400fcb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:56.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.978+0000 7f3045000700 1 -- 172.21.15.1:0/475474149 shutdown_connections 2024-02-02T15:36:56.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:56.978+0000 7f3045000700 1 -- 172.21.15.1:0/475474149 wait complete. 2024-02-02T15:36:56.998 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-02T15:36:57.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.206+0000 7f3493e00700 1 -- 172.21.15.1:0/3814195182 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3494101450 msgr2=0x7f349410d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:57.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.206+0000 7f3493e00700 1 --2- 172.21.15.1:0/3814195182 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3494101450 0x7f349410d960 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f348c00b0a0 tx=0x7f348c00b3b0 comp rx=0 tx=0).stop 2024-02-02T15:36:57.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.206+0000 7f3493e00700 1 -- 172.21.15.1:0/3814195182 shutdown_connections 2024-02-02T15:36:57.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.206+0000 7f3493e00700 1 --2- 172.21.15.1:0/3814195182 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3494101450 0x7f349410d960 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:57.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.206+0000 7f3493e00700 1 --2- 172.21.15.1:0/3814195182 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3494106aa0 0x7f3494106e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:57.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.206+0000 7f3493e00700 1 --2- 172.21.15.1:0/3814195182 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3494100a90 0x7f3494100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:57.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.206+0000 7f3493e00700 1 -- 172.21.15.1:0/3814195182 >> 172.21.15.1:0/3814195182 conn(0x7f34940fc520 msgr2=0x7f34940fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:57.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.206+0000 7f3493e00700 1 -- 172.21.15.1:0/3814195182 shutdown_connections 2024-02-02T15:36:57.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.206+0000 7f3493e00700 1 -- 172.21.15.1:0/3814195182 wait complete. 2024-02-02T15:36:57.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.207+0000 7f3493e00700 1 Processor -- start 2024-02-02T15:36:57.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.207+0000 7f3493e00700 1 -- start start 2024-02-02T15:36:57.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.208+0000 7f3493e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3494100a90 0x7f349419c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:57.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.208+0000 7f3493e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3494101450 0x7f349419cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:57.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.208+0000 7f3493e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3494106aa0 0x7f3494196920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:57.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.208+0000 7f3493e00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3494196f90 con 0x7f3494100a90 2024-02-02T15:36:57.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.208+0000 7f3493e00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3494197100 con 0x7f3494106aa0 2024-02-02T15:36:57.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.208+0000 7f3493e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34941972c0 con 0x7f3494101450 2024-02-02T15:36:57.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.208+0000 7f3492a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3494100a90 0x7f349419c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:57.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.208+0000 7f3493400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3494106aa0 0x7f3494196920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:57.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.208+0000 7f348be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3494101450 0x7f349419cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:57.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.208+0000 7f3493400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3494106aa0 0x7f3494196920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59436/0 (socket says 172.21.15.1:59436) 2024-02-02T15:36:57.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.208+0000 7f348be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3494101450 0x7f349419cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:34226/0 (socket says 172.21.15.1:34226) 2024-02-02T15:36:57.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.208+0000 7f348be00700 1 -- 172.21.15.1:0/210871853 learned_addr learned my addr 172.21.15.1:0/210871853 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:36:57.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.208+0000 7f3492a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3494100a90 0x7f349419c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48856/0 (socket says 172.21.15.1:48856) 2024-02-02T15:36:57.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.208+0000 7f3493400700 1 -- 172.21.15.1:0/210871853 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3494101450 msgr2=0x7f349419cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:57.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.208+0000 7f3493400700 1 --2- 172.21.15.1:0/210871853 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3494101450 0x7f349419cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:57.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.208+0000 7f3493400700 1 -- 172.21.15.1:0/210871853 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3494100a90 msgr2=0x7f349419c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:57.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.208+0000 7f3493400700 1 --2- 172.21.15.1:0/210871853 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3494100a90 0x7f349419c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:57.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.208+0000 7f3493400700 1 -- 172.21.15.1:0/210871853 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f347c009520 con 0x7f3494106aa0 2024-02-02T15:36:57.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.208+0000 7f348be00700 1 --2- 172.21.15.1:0/210871853 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3494101450 0x7f349419cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:36:57.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.209+0000 7f3493400700 1 --2- 172.21.15.1:0/210871853 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3494106aa0 0x7f3494196920 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f348c0062a0 tx=0x7f348c0077f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:57.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.209+0000 7f348b400700 1 -- 172.21.15.1:0/210871853 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f348c00e040 con 0x7f3494106aa0 2024-02-02T15:36:57.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.209+0000 7f348b400700 1 -- 172.21.15.1:0/210871853 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f348c0094c0 con 0x7f3494106aa0 2024-02-02T15:36:57.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.209+0000 7f348b400700 1 -- 172.21.15.1:0/210871853 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f348c005040 con 0x7f3494106aa0 2024-02-02T15:36:57.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.209+0000 7f3493e00700 1 -- 172.21.15.1:0/210871853 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f348c009d00 con 0x7f3494106aa0 2024-02-02T15:36:57.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.209+0000 7f3493e00700 1 -- 172.21.15.1:0/210871853 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34941a36c0 con 0x7f3494106aa0 2024-02-02T15:36:57.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.212+0000 7f348b400700 1 -- 172.21.15.1:0/210871853 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f348c01d470 con 0x7f3494106aa0 2024-02-02T15:36:57.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.214+0000 7f34ad13ab80 1 Processor -- start 2024-02-02T15:36:57.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.214+0000 7f348b400700 1 --2- 172.21.15.1:0/210871853 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f34740a0710 0x7f34740a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:57.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.214+0000 7f34ad13ab80 1 -- start start 2024-02-02T15:36:57.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.215+0000 7f348b400700 1 -- 172.21.15.1:0/210871853 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f348c0cd7a0 con 0x7f3494106aa0 2024-02-02T15:36:57.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.215+0000 7f348be00700 1 --2- 172.21.15.1:0/210871853 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f34740a0710 0x7f34740a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:57.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.215+0000 7f34ad13ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581b3b7e1f0 0x5581b3b7e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:57.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.215+0000 7f34ad13ab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5581b3b7eb10 0x5581b3b894b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:57.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.215+0000 7f34ad13ab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5581b3b899f0 0x5581b3b8bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:57.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.215+0000 7f34ad13ab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581b3af24c0 con 0x5581b3b899f0 2024-02-02T15:36:57.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.215+0000 7f34ad13ab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581b3ad6aa0 con 0x5581b3b7eb10 2024-02-02T15:36:57.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.215+0000 7f34ad13ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581b3ad4460 con 0x5581b3b7e1f0 2024-02-02T15:36:57.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.215+0000 7f3492a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581b3b7e1f0 0x5581b3b7e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:57.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.215+0000 7f3492a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581b3b7e1f0 0x5581b3b7e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:34242/0 (socket says 172.21.15.1:34242) 2024-02-02T15:36:57.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.215+0000 7f3492a00700 1 -- 172.21.15.1:0/1652117073 learned_addr learned my addr 172.21.15.1:0/1652117073 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:36:57.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.215+0000 7f3493400700 1 --2- 172.21.15.1:0/1652117073 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5581b3b899f0 0x5581b3b8bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:57.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.215+0000 7f348be00700 1 --2- 172.21.15.1:0/1652117073 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5581b3b7eb10 0x5581b3b894b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:57.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.216+0000 7f3492a00700 1 -- 172.21.15.1:0/1652117073 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5581b3b7eb10 msgr2=0x5581b3b894b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:57.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.216+0000 7f348be00700 1 --2- 172.21.15.1:0/210871853 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f34740a0710 0x7f34740a2bd0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f347c009fd0 tx=0x7f347c009260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:57.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.216+0000 7f3492a00700 1 --2- 172.21.15.1:0/1652117073 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5581b3b7eb10 0x5581b3b894b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:57.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.216+0000 7f3492a00700 1 -- 172.21.15.1:0/1652117073 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5581b3b899f0 msgr2=0x5581b3b8bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:57.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.216+0000 7f3492a00700 1 --2- 172.21.15.1:0/1652117073 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5581b3b899f0 0x5581b3b8bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:57.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.216+0000 7f3492a00700 1 -- 172.21.15.1:0/1652117073 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5581b39513b0 con 0x5581b3b7e1f0 2024-02-02T15:36:57.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.216+0000 7f3492a00700 1 --2- 172.21.15.1:0/1652117073 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581b3b7e1f0 0x5581b3b7e5d0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f3484009d00 tx=0x7f3484011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3041e4dd2b29f3ae server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:57.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.216+0000 7f3483400700 1 -- 172.21.15.1:0/1652117073 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3484004230 con 0x5581b3b7e1f0 2024-02-02T15:36:57.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.216+0000 7f3483400700 1 -- 172.21.15.1:0/1652117073 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3484009e50 con 0x5581b3b7e1f0 2024-02-02T15:36:57.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.216+0000 7f3483400700 1 -- 172.21.15.1:0/1652117073 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34840197a0 con 0x5581b3b7e1f0 2024-02-02T15:36:57.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.217+0000 7f34ad13ab80 1 -- 172.21.15.1:0/1652117073 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581b3b7e1f0 msgr2=0x5581b3b7e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:57.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.217+0000 7f34ad13ab80 1 --2- 172.21.15.1:0/1652117073 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581b3b7e1f0 0x5581b3b7e5d0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f3484009d00 tx=0x7f3484011720 comp rx=0 tx=0).stop 2024-02-02T15:36:57.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.217+0000 7f34ad13ab80 1 -- 172.21.15.1:0/1652117073 shutdown_connections 2024-02-02T15:36:57.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.217+0000 7f34ad13ab80 1 --2- 172.21.15.1:0/1652117073 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5581b3b899f0 0x5581b3b8bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:57.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.217+0000 7f34ad13ab80 1 --2- 172.21.15.1:0/1652117073 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581b3b7e1f0 0x5581b3b7e5d0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:57.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.217+0000 7f34ad13ab80 1 --2- 172.21.15.1:0/1652117073 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5581b3b7eb10 0x5581b3b894b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:57.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.217+0000 7f34ad13ab80 1 -- 172.21.15.1:0/1652117073 >> 172.21.15.1:0/1652117073 conn(0x5581b3adddb0 msgr2=0x5581b3b7a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:57.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.217+0000 7f34ad13ab80 1 -- 172.21.15.1:0/1652117073 shutdown_connections 2024-02-02T15:36:57.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.217+0000 7f34ad13ab80 1 -- 172.21.15.1:0/1652117073 wait complete. 2024-02-02T15:36:57.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.218+0000 7f34ad13ab80 1 -- start start 2024-02-02T15:36:57.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.218+0000 7f34ad13ab80 1 Processor -- start 2024-02-02T15:36:57.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.218+0000 7f34ad13ab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5581b3b7e1f0 0x5581b3b89430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:57.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.218+0000 7f34ad13ab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5581b3b7eb10 0x5581b3b86640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:57.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.218+0000 7f34ad13ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581b3b899f0 0x5581b3b86d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:57.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.218+0000 7f34ad13ab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581b3acfd50 con 0x5581b3b7e1f0 2024-02-02T15:36:57.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.218+0000 7f34ad13ab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581b3a6bd80 con 0x5581b3b7eb10 2024-02-02T15:36:57.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.218+0000 7f34ad13ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581b3a6ba20 con 0x5581b3b899f0 2024-02-02T15:36:57.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.219+0000 7f34ad13ab80 10 client.0 _ll_register_callbacks cb 0x5581b3a6d290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:36:57.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.219+0000 7f3493400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581b3b899f0 0x5581b3b86d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:57.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.219+0000 7f3492a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5581b3b7e1f0 0x5581b3b89430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:57.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.219+0000 7f3493400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581b3b899f0 0x5581b3b86d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:34256/0 (socket says 172.21.15.1:34256) 2024-02-02T15:36:57.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.219+0000 7f3493400700 1 -- 172.21.15.1:0/288446667 learned_addr learned my addr 172.21.15.1:0/288446667 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:36:57.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.219+0000 7f348be00700 1 --2- 172.21.15.1:0/288446667 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5581b3b7eb10 0x5581b3b86640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:57.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.219+0000 7f3493400700 1 -- 172.21.15.1:0/288446667 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5581b3b7eb10 msgr2=0x5581b3b86640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:57.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.219+0000 7f3493400700 1 --2- 172.21.15.1:0/288446667 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5581b3b7eb10 0x5581b3b86640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:57.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.219+0000 7f3493400700 1 -- 172.21.15.1:0/288446667 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5581b3b7e1f0 msgr2=0x5581b3b89430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:57.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.219+0000 7f3493400700 1 --2- 172.21.15.1:0/288446667 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5581b3b7e1f0 0x5581b3b89430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:57.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.219+0000 7f3493400700 1 -- 172.21.15.1:0/288446667 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3484009960 con 0x5581b3b899f0 2024-02-02T15:36:57.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.219+0000 7f3493400700 1 --2- 172.21.15.1:0/288446667 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581b3b899f0 0x5581b3b86d80 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f348c0973c0 tx=0x7f348c00b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e074831b00c86f2d server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:57.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.219+0000 7f347be00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:36:57.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.220+0000 7f347be00700 1 -- 172.21.15.1:0/288446667 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f348c01d920 con 0x5581b3b899f0 2024-02-02T15:36:57.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.220+0000 7f347be00700 1 -- 172.21.15.1:0/288446667 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f348c09a950 con 0x5581b3b899f0 2024-02-02T15:36:57.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.220+0000 7f34ad13ab80 1 -- 172.21.15.1:0/288446667 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5581b3b72340 con 0x5581b3b899f0 2024-02-02T15:36:57.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.220+0000 7f347be00700 1 -- 172.21.15.1:0/288446667 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f348c09e780 con 0x5581b3b899f0 2024-02-02T15:36:57.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.220+0000 7f34ad13ab80 10 client.28306 fetch_fsmap learned FSMap version 46 2024-02-02T15:36:57.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.220+0000 7f34ad13ab80 1 -- 172.21.15.1:0/288446667 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x5581b38fdb60 con 0x5581b3b899f0 2024-02-02T15:36:57.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.220+0000 7f347be00700 1 -- 172.21.15.1:0/288446667 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5581b3b72340 con 0x5581b3b899f0 2024-02-02T15:36:57.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.220+0000 7f347be00700 1 -- 172.21.15.1:0/288446667 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 46) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f348c099e40 con 0x5581b3b899f0 2024-02-02T15:36:57.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.220+0000 7f34ad13ab80 10 client.28306 fetch_fsmap finished waiting for FSMap version 46 2024-02-02T15:36:57.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.220+0000 7f34ad13ab80 10 client.28306 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:36:57.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.220+0000 7f34ad13ab80 20 client.28306 populate_metadata read hostname 'smithi001' 2024-02-02T15:36:57.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.220+0000 7f34ad13ab80 1 --2- 172.21.15.1:0/288446667 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5581b3b99200 0x5581b3b9b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:57.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.220+0000 7f34ad13ab80 4 client.28306 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:36:57.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.220+0000 7f34ad13ab80 1 -- 172.21.15.1:0/288446667 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5581b3ac8dd0 con 0x5581b3b99200 2024-02-02T15:36:57.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.221+0000 7f3492a00700 1 --2- 172.21.15.1:0/288446667 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5581b3b99200 0x5581b3b9b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:57.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.221+0000 7f3492a00700 1 --2- 172.21.15.1:0/288446667 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5581b3b99200 0x5581b3b9b620 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1c581aa547532dc server_cookie=4f5a6670b28e8603 in_seq=0 out_seq=0 2024-02-02T15:36:57.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.221+0000 7f347be00700 10 client.28306 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:36:57.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.222+0000 7f347be00700 1 -- 172.21.15.1:0/288446667 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5581b3ac8dd0 con 0x5581b3b99200 2024-02-02T15:36:57.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.222+0000 7f347be00700 10 client.28306 handle_command_reply: tid=0 2024-02-02T15:36:57.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.222+0000 7f34ad13ab80 1 client.28306 shutdown 2024-02-02T15:36:57.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.222+0000 7f34ad13ab80 1 -- 172.21.15.1:0/288446667 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581b3b899f0 msgr2=0x5581b3b86d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:57.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.222+0000 7f34ad13ab80 1 --2- 172.21.15.1:0/288446667 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581b3b899f0 0x5581b3b86d80 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f348c0973c0 tx=0x7f348c00b040 comp rx=0 tx=0).stop 2024-02-02T15:36:57.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.222+0000 7f34ad13ab80 1 -- 172.21.15.1:0/288446667 shutdown_connections 2024-02-02T15:36:57.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.222+0000 7f34ad13ab80 1 --2- 172.21.15.1:0/288446667 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5581b3b99200 0x5581b3b9b620 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:57.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.222+0000 7f34ad13ab80 1 --2- 172.21.15.1:0/288446667 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581b3b899f0 0x5581b3b86d80 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:57.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.222+0000 7f34ad13ab80 1 --2- 172.21.15.1:0/288446667 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5581b3b7e1f0 0x5581b3b89430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:57.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.222+0000 7f347be00700 0 client.28306 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:36:57.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.222+0000 7f34ad13ab80 1 --2- 172.21.15.1:0/288446667 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5581b3b7eb10 0x5581b3b86640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:57.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.222+0000 7f34ad13ab80 1 -- 172.21.15.1:0/288446667 >> 172.21.15.1:0/288446667 conn(0x5581b3adddb0 msgr2=0x5581b3b7a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:57.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.222+0000 7f34ad13ab80 1 -- 172.21.15.1:0/288446667 shutdown_connections 2024-02-02T15:36:57.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.222+0000 7f34ad13ab80 1 -- 172.21.15.1:0/288446667 wait complete. 2024-02-02T15:36:57.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.222+0000 7f34ad13ab80 20 client.28306 trim_cache size 0 max 16384 2024-02-02T15:36:57.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.236+0000 7f34ad13ab80 1 Processor -- start 2024-02-02T15:36:57.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.236+0000 7f34ad13ab80 1 -- start start 2024-02-02T15:36:57.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.237+0000 7f34ad13ab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5581b3b7e1f0 0x5581b3b88f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:57.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.237+0000 7f34ad13ab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5581b3b7eb10 0x5581b3b94ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:57.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.237+0000 7f34ad13ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581b3b899f0 0x5581b3b94ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:57.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.237+0000 7f34ad13ab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581b39536a0 con 0x5581b3b7eb10 2024-02-02T15:36:57.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.237+0000 7f34ad13ab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581b3b7b450 con 0x5581b3b7e1f0 2024-02-02T15:36:57.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.237+0000 7f34ad13ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581b3b82510 con 0x5581b3b899f0 2024-02-02T15:36:57.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.237+0000 7f3492a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5581b3b7e1f0 0x5581b3b88f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:57.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.237+0000 7f3492a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5581b3b7e1f0 0x5581b3b88f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59468/0 (socket says 172.21.15.1:59468) 2024-02-02T15:36:57.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.237+0000 7f3492a00700 1 -- 172.21.15.1:0/522714307 learned_addr learned my addr 172.21.15.1:0/522714307 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:36:57.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.237+0000 7f3493400700 1 --2- 172.21.15.1:0/522714307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581b3b899f0 0x5581b3b94ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:57.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.237+0000 7f348be00700 1 --2- 172.21.15.1:0/522714307 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5581b3b7eb10 0x5581b3b94ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:57.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.237+0000 7f3492a00700 1 -- 172.21.15.1:0/522714307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581b3b899f0 msgr2=0x5581b3b94ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:57.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.237+0000 7f3492a00700 1 --2- 172.21.15.1:0/522714307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581b3b899f0 0x5581b3b94ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:57.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.237+0000 7f3492a00700 1 -- 172.21.15.1:0/522714307 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5581b3b7eb10 msgr2=0x5581b3b94ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:57.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.237+0000 7f3492a00700 1 --2- 172.21.15.1:0/522714307 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5581b3b7eb10 0x5581b3b94ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:57.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.237+0000 7f3492a00700 1 -- 172.21.15.1:0/522714307 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5581b3a95ea0 con 0x5581b3b7e1f0 2024-02-02T15:36:57.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.238+0000 7f3492a00700 1 --2- 172.21.15.1:0/522714307 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5581b3b7e1f0 0x5581b3b88f60 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f3484009d00 tx=0x7f3484003f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a63aafee150a9e7 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:57.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.238+0000 7f3483e00700 1 -- 172.21.15.1:0/522714307 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3484003680 con 0x5581b3b7e1f0 2024-02-02T15:36:57.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.238+0000 7f3483e00700 1 -- 172.21.15.1:0/522714307 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3484005500 con 0x5581b3b7e1f0 2024-02-02T15:36:57.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.238+0000 7f3483e00700 1 -- 172.21.15.1:0/522714307 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f348402a980 con 0x5581b3b7e1f0 2024-02-02T15:36:57.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.238+0000 7f34ad13ab80 1 -- 172.21.15.1:0/522714307 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5581b3b7e1f0 msgr2=0x5581b3b88f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:57.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.238+0000 7f34ad13ab80 1 --2- 172.21.15.1:0/522714307 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5581b3b7e1f0 0x5581b3b88f60 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f3484009d00 tx=0x7f3484003f00 comp rx=0 tx=0).stop 2024-02-02T15:36:57.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.239+0000 7f34ad13ab80 1 -- 172.21.15.1:0/522714307 shutdown_connections 2024-02-02T15:36:57.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.239+0000 7f34ad13ab80 1 --2- 172.21.15.1:0/522714307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581b3b899f0 0x5581b3b94ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:57.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.239+0000 7f34ad13ab80 1 --2- 172.21.15.1:0/522714307 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5581b3b7e1f0 0x5581b3b88f60 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:57.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.239+0000 7f34ad13ab80 1 --2- 172.21.15.1:0/522714307 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5581b3b7eb10 0x5581b3b94ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:57.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.239+0000 7f34ad13ab80 1 -- 172.21.15.1:0/522714307 >> 172.21.15.1:0/522714307 conn(0x5581b3adddb0 msgr2=0x5581b3b76050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:57.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.239+0000 7f34ad13ab80 1 -- 172.21.15.1:0/522714307 shutdown_connections 2024-02-02T15:36:57.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.239+0000 7f34ad13ab80 1 -- 172.21.15.1:0/522714307 wait complete. 2024-02-02T15:36:57.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.239+0000 7f34ad13ab80 1 -- start start 2024-02-02T15:36:57.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.239+0000 7f34ad13ab80 1 Processor -- start 2024-02-02T15:36:57.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.240+0000 7f34ad13ab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5581b3b7e1f0 0x5581b3b8dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:57.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.240+0000 7f34ad13ab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5581b3b7eb10 0x5581b3b8e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:57.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.240+0000 7f34ad13ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581b3b899f0 0x5581b3bf24f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:57.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.240+0000 7f34ad13ab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581b3b8ebf0 con 0x5581b3b7eb10 2024-02-02T15:36:57.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.240+0000 7f34ad13ab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581b3bf2a30 con 0x5581b3b7e1f0 2024-02-02T15:36:57.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.240+0000 7f34ad13ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581b3bf2b70 con 0x5581b3b899f0 2024-02-02T15:36:57.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.240+0000 7f34ad13ab80 10 client.0 _ll_register_callbacks cb 0x5581b3a6d290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:36:57.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.240+0000 7f3492a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5581b3b7e1f0 0x5581b3b8dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:57.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.240+0000 7f3492a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5581b3b7e1f0 0x5581b3b8dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59470/0 (socket says 172.21.15.1:59470) 2024-02-02T15:36:57.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.240+0000 7f3492a00700 1 -- 172.21.15.1:0/144850476 learned_addr learned my addr 172.21.15.1:0/144850476 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:36:57.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.240+0000 7f348be00700 1 --2- 172.21.15.1:0/144850476 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5581b3b7eb10 0x5581b3b8e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:57.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.240+0000 7f3493400700 1 --2- 172.21.15.1:0/144850476 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581b3b899f0 0x5581b3bf24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:57.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.241+0000 7f3492a00700 1 -- 172.21.15.1:0/144850476 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581b3b899f0 msgr2=0x5581b3bf24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:57.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.241+0000 7f3492a00700 1 --2- 172.21.15.1:0/144850476 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581b3b899f0 0x5581b3bf24f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:57.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.241+0000 7f3492a00700 1 -- 172.21.15.1:0/144850476 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5581b3b7eb10 msgr2=0x5581b3b8e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:57.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.241+0000 7f3492a00700 1 --2- 172.21.15.1:0/144850476 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5581b3b7eb10 0x5581b3b8e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:57.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.241+0000 7f3492a00700 1 -- 172.21.15.1:0/144850476 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5581b39513b0 con 0x5581b3b7e1f0 2024-02-02T15:36:57.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.241+0000 7f3492a00700 1 --2- 172.21.15.1:0/144850476 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5581b3b7e1f0 0x5581b3b8dda0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f348401b040 tx=0x7f3484009440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce8a1bbc6020de56 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:57.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.241+0000 7f347be00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:36:57.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.241+0000 7f347be00700 1 -- 172.21.15.1:0/144850476 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34840052c0 con 0x5581b3b7e1f0 2024-02-02T15:36:57.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.241+0000 7f347be00700 1 -- 172.21.15.1:0/144850476 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f3484034450 con 0x5581b3b7e1f0 2024-02-02T15:36:57.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.241+0000 7f34ad13ab80 1 -- 172.21.15.1:0/144850476 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3484009e50 con 0x5581b3b7e1f0 2024-02-02T15:36:57.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.242+0000 7f347be00700 1 -- 172.21.15.1:0/144850476 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3484034610 con 0x5581b3b7e1f0 2024-02-02T15:36:57.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.242+0000 7f34ad13ab80 10 client.28313 fetch_fsmap learned FSMap version 46 2024-02-02T15:36:57.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.242+0000 7f34ad13ab80 1 -- 172.21.15.1:0/144850476 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x5581b3955cd0 con 0x5581b3b7e1f0 2024-02-02T15:36:57.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.242+0000 7f347be00700 1 -- 172.21.15.1:0/144850476 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3484009e50 con 0x5581b3b7e1f0 2024-02-02T15:36:57.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.242+0000 7f347be00700 1 -- 172.21.15.1:0/144850476 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 46) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f3484011440 con 0x5581b3b7e1f0 2024-02-02T15:36:57.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.242+0000 7f34ad13ab80 10 client.28313 fetch_fsmap finished waiting for FSMap version 46 2024-02-02T15:36:57.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.242+0000 7f34ad13ab80 10 client.28313 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:36:57.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.242+0000 7f34ad13ab80 20 client.28313 populate_metadata read hostname 'smithi001' 2024-02-02T15:36:57.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.242+0000 7f34ad13ab80 1 --2- 172.21.15.1:0/144850476 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5581b3bf4020 0x5581b3bf6440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:57.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.242+0000 7f34ad13ab80 4 client.28313 mds_command: new command op to 24557 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:36:57.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.242+0000 7f34ad13ab80 1 -- 172.21.15.1:0/144850476 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5581b3a8ef50 con 0x5581b3bf4020 2024-02-02T15:36:57.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.243+0000 7f348be00700 1 --2- 172.21.15.1:0/144850476 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5581b3bf4020 0x5581b3bf6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:57.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.243+0000 7f348be00700 1 --2- 172.21.15.1:0/144850476 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5581b3bf4020 0x5581b3bf6440 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12dab9b4604160ab server_cookie=2844f049eae3ffff in_seq=0 out_seq=0 2024-02-02T15:36:57.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.243+0000 7f347be00700 10 client.28313 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:36:57.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.243+0000 7f347be00700 1 -- 172.21.15.1:0/144850476 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5581b3a8ef50 con 0x5581b3bf4020 2024-02-02T15:36:57.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.243+0000 7f347be00700 10 client.28313 handle_command_reply: tid=0 2024-02-02T15:36:57.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.243+0000 7f34ad13ab80 1 client.28313 shutdown 2024-02-02T15:36:57.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.244+0000 7f34ad13ab80 1 -- 172.21.15.1:0/144850476 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5581b3b7e1f0 msgr2=0x5581b3b8dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:57.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.244+0000 7f34ad13ab80 1 --2- 172.21.15.1:0/144850476 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5581b3b7e1f0 0x5581b3b8dda0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f348401b040 tx=0x7f3484009440 comp rx=0 tx=0).stop 2024-02-02T15:36:57.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.244+0000 7f34ad13ab80 1 -- 172.21.15.1:0/144850476 shutdown_connections 2024-02-02T15:36:57.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.244+0000 7f34ad13ab80 1 --2- 172.21.15.1:0/144850476 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5581b3bf4020 0x5581b3bf6440 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:57.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.244+0000 7f34ad13ab80 1 --2- 172.21.15.1:0/144850476 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5581b3b899f0 0x5581b3bf24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:57.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.244+0000 7f34ad13ab80 1 --2- 172.21.15.1:0/144850476 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5581b3b7e1f0 0x5581b3b8dda0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:57.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.244+0000 7f34ad13ab80 1 --2- 172.21.15.1:0/144850476 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5581b3b7eb10 0x5581b3b8e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:57.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.244+0000 7f347be00700 0 client.28313 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:36:57.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.244+0000 7f34ad13ab80 1 -- 172.21.15.1:0/144850476 >> 172.21.15.1:0/144850476 conn(0x5581b3adddb0 msgr2=0x5581b3b94ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:57.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.244+0000 7f34ad13ab80 1 -- 172.21.15.1:0/144850476 shutdown_connections 2024-02-02T15:36:57.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.244+0000 7f34ad13ab80 1 -- 172.21.15.1:0/144850476 wait complete. 2024-02-02T15:36:57.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.244+0000 7f34ad13ab80 20 client.28313 trim_cache size 0 max 16384 2024-02-02T15:36:57.249 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:36:57.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.245+0000 7f3493e00700 1 -- 172.21.15.1:0/210871853 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f34740a0710 msgr2=0x7f34740a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:57.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.245+0000 7f3493e00700 1 --2- 172.21.15.1:0/210871853 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f34740a0710 0x7f34740a2bd0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f347c009fd0 tx=0x7f347c009260 comp rx=0 tx=0).stop 2024-02-02T15:36:57.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.245+0000 7f3493e00700 1 -- 172.21.15.1:0/210871853 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3494106aa0 msgr2=0x7f3494196920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:57.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.245+0000 7f3493e00700 1 --2- 172.21.15.1:0/210871853 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3494106aa0 0x7f3494196920 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f348c0062a0 tx=0x7f348c0077f0 comp rx=0 tx=0).stop 2024-02-02T15:36:57.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.245+0000 7f3493e00700 1 -- 172.21.15.1:0/210871853 shutdown_connections 2024-02-02T15:36:57.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.245+0000 7f3493e00700 1 --2- 172.21.15.1:0/210871853 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f34740a0710 0x7f34740a2bd0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:57.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.245+0000 7f3493e00700 1 --2- 172.21.15.1:0/210871853 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3494106aa0 0x7f3494196920 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:57.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.245+0000 7f3493e00700 1 --2- 172.21.15.1:0/210871853 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3494100a90 0x7f349419c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:57.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.245+0000 7f3493e00700 1 --2- 172.21.15.1:0/210871853 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3494101450 0x7f349419cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:57.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.245+0000 7f3493e00700 1 -- 172.21.15.1:0/210871853 >> 172.21.15.1:0/210871853 conn(0x7f34940fc520 msgr2=0x7f3494107980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:57.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.245+0000 7f3493e00700 1 -- 172.21.15.1:0/210871853 shutdown_connections 2024-02-02T15:36:57.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:57.245+0000 7f3493e00700 1 -- 172.21.15.1:0/210871853 wait complete. 2024-02-02T15:36:57.260 INFO:tasks.workunit.client.0.smithi001.stdout: 1 575743 6.05 MB/sec execute 391 sec latency 43.217 ms 2024-02-02T15:36:57.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:57 smithi001 ceph-mon[35470]: pgmap v307: 129 pgs: 129 active+clean; 440 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 179 op/s 2024-02-02T15:36:57.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:57 smithi001 ceph-mon[35470]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:36:57.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:57 smithi071 ceph-mon[44707]: pgmap v307: 129 pgs: 129 active+clean; 440 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 179 op/s 2024-02-02T15:36:57.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:57 smithi071 ceph-mon[44707]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:36:57.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:57 smithi142 ceph-mon[45728]: pgmap v307: 129 pgs: 129 active+clean; 440 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 179 op/s 2024-02-02T15:36:57.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:57 smithi142 ceph-mon[45728]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:36:58.259 INFO:tasks.workunit.client.0.smithi001.stdout: 1 576869 6.05 MB/sec execute 392 sec latency 19.239 ms 2024-02-02T15:36:58.264 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-02T15:36:58.264 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-02T15:36:58.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.484+0000 7f0e91600700 1 -- 172.21.15.1:0/2359535975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e8c101000 msgr2=0x7f0e8c1013e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:58.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.484+0000 7f0e91600700 1 --2- 172.21.15.1:0/2359535975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e8c101000 0x7f0e8c1013e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c009990 tx=0x7f0e7c009ca0 comp rx=0 tx=0).stop 2024-02-02T15:36:58.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.484+0000 7f0e91600700 1 -- 172.21.15.1:0/2359535975 shutdown_connections 2024-02-02T15:36:58.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.484+0000 7f0e91600700 1 --2- 172.21.15.1:0/2359535975 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e8c106120 0x7f0e8c10dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:58.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.484+0000 7f0e91600700 1 --2- 172.21.15.1:0/2359535975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e8c101000 0x7f0e8c1013e0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:58.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.484+0000 7f0e91600700 1 --2- 172.21.15.1:0/2359535975 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e8c1019b0 0x7f0e8c1059f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:58.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.484+0000 7f0e91600700 1 -- 172.21.15.1:0/2359535975 >> 172.21.15.1:0/2359535975 conn(0x7f0e8c0fc860 msgr2=0x7f0e8c0fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:58.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.485+0000 7f0e91600700 1 -- 172.21.15.1:0/2359535975 shutdown_connections 2024-02-02T15:36:58.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.485+0000 7f0e91600700 1 -- 172.21.15.1:0/2359535975 wait complete. 2024-02-02T15:36:58.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.486+0000 7f0e91600700 1 Processor -- start 2024-02-02T15:36:58.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.486+0000 7f0e91600700 1 -- start start 2024-02-02T15:36:58.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.486+0000 7f0e91600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e8c101000 0x7f0e8c1991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:58.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.486+0000 7f0e91600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e8c106120 0x7f0e8c1996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:58.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.486+0000 7f0e91600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e8c19dad0 0x7f0e8c19df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:58.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.486+0000 7f0e91600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8c19e550 con 0x7f0e8c106120 2024-02-02T15:36:58.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.486+0000 7f0e91600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8c19e6c0 con 0x7f0e8c19dad0 2024-02-02T15:36:58.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.486+0000 7f0e91600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e8c19e880 con 0x7f0e8c101000 2024-02-02T15:36:58.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.486+0000 7f0e8aa00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e8c101000 0x7f0e8c1991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:58.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.486+0000 7f0e8a000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e8c106120 0x7f0e8c1996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:58.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.487+0000 7f0e8aa00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e8c101000 0x7f0e8c1991a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:34306/0 (socket says 172.21.15.1:34306) 2024-02-02T15:36:58.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.487+0000 7f0e8aa00700 1 -- 172.21.15.1:0/2859180377 learned_addr learned my addr 172.21.15.1:0/2859180377 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:36:58.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.487+0000 7f0e8b400700 1 --2- 172.21.15.1:0/2859180377 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e8c19dad0 0x7f0e8c19df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:58.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.487+0000 7f0e8aa00700 1 -- 172.21.15.1:0/2859180377 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e8c19dad0 msgr2=0x7f0e8c19df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:58.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.487+0000 7f0e8aa00700 1 --2- 172.21.15.1:0/2859180377 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e8c19dad0 0x7f0e8c19df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:58.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.487+0000 7f0e8aa00700 1 -- 172.21.15.1:0/2859180377 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e8c106120 msgr2=0x7f0e8c1996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:58.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.487+0000 7f0e8aa00700 1 --2- 172.21.15.1:0/2859180377 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e8c106120 0x7f0e8c1996e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:58.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.487+0000 7f0e8aa00700 1 -- 172.21.15.1:0/2859180377 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e7c0095f0 con 0x7f0e8c101000 2024-02-02T15:36:58.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.487+0000 7f0e8b400700 1 --2- 172.21.15.1:0/2859180377 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e8c19dad0 0x7f0e8c19df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:36:58.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.487+0000 7f0e8a000700 1 --2- 172.21.15.1:0/2859180377 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e8c106120 0x7f0e8c1996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:36:58.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.487+0000 7f0e8aa00700 1 --2- 172.21.15.1:0/2859180377 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e8c101000 0x7f0e8c1991a0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c005910 tx=0x7f0e7c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:58.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.487+0000 7f0e83400700 1 -- 172.21.15.1:0/2859180377 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c01d070 con 0x7f0e8c101000 2024-02-02T15:36:58.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.488+0000 7f0e83400700 1 -- 172.21.15.1:0/2859180377 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e7c022470 con 0x7f0e8c101000 2024-02-02T15:36:58.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.488+0000 7f0e91600700 1 -- 172.21.15.1:0/2859180377 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e8c1a5d80 con 0x7f0e8c101000 2024-02-02T15:36:58.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.488+0000 7f0e83400700 1 -- 172.21.15.1:0/2859180377 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c017650 con 0x7f0e8c101000 2024-02-02T15:36:58.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.488+0000 7f0e91600700 1 -- 172.21.15.1:0/2859180377 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e8c1a61b0 con 0x7f0e8c101000 2024-02-02T15:36:58.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.491+0000 7f0e83400700 1 -- 172.21.15.1:0/2859180377 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f0e7c0225e0 con 0x7f0e8c101000 2024-02-02T15:36:58.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.492+0000 7f0ea51a4b80 1 Processor -- start 2024-02-02T15:36:58.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.492+0000 7f0ea51a4b80 1 -- start start 2024-02-02T15:36:58.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.493+0000 7f0ea51a4b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561be6495090 0x561be6495470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:58.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.493+0000 7f0ea51a4b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561be64959b0 0x561be64a0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:58.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.493+0000 7f0ea51a4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561be64a0860 0x561be64a2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:58.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.493+0000 7f0ea51a4b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x561be6409540 con 0x561be6495090 2024-02-02T15:36:58.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.493+0000 7f0ea51a4b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x561be63edb20 con 0x561be64959b0 2024-02-02T15:36:58.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.493+0000 7f0ea51a4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561be63eb4e0 con 0x561be64a0860 2024-02-02T15:36:58.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.493+0000 7f0e83400700 1 --2- 172.21.15.1:0/2859180377 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f0e6c0a0710 0x7f0e6c0a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:58.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.493+0000 7f0e8aa00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561be64959b0 0x561be64a0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:58.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.493+0000 7f0e8aa00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561be64959b0 0x561be64a0320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59492/0 (socket says 172.21.15.1:59492) 2024-02-02T15:36:58.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.493+0000 7f0e8aa00700 1 -- 172.21.15.1:0/3023394049 learned_addr learned my addr 172.21.15.1:0/3023394049 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:36:58.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.493+0000 7f0e8b400700 1 --2- 172.21.15.1:0/3023394049 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561be6495090 0x561be6495470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:58.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.493+0000 7f0e8a000700 1 --2- 172.21.15.1:0/3023394049 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561be64a0860 0x561be64a2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:58.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.493+0000 7f0e83400700 1 -- 172.21.15.1:0/2859180377 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f0e7c0cd2a0 con 0x7f0e8c101000 2024-02-02T15:36:58.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.493+0000 7f0e8b400700 1 --2- 172.21.15.1:0/2859180377 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f0e6c0a0710 0x7f0e6c0a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:58.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.494+0000 7f0e8aa00700 1 -- 172.21.15.1:0/3023394049 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561be64a0860 msgr2=0x561be64a2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:58.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.494+0000 7f0e8aa00700 1 --2- 172.21.15.1:0/3023394049 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561be64a0860 0x561be64a2c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:58.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.494+0000 7f0e8aa00700 1 -- 172.21.15.1:0/3023394049 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561be6495090 msgr2=0x561be6495470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:58.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.494+0000 7f0e8aa00700 1 --2- 172.21.15.1:0/3023394049 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561be6495090 0x561be6495470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:58.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.494+0000 7f0e8aa00700 1 -- 172.21.15.1:0/3023394049 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561be6268430 con 0x561be64959b0 2024-02-02T15:36:58.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.494+0000 7f0e8a000700 1 --2- 172.21.15.1:0/3023394049 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561be64a0860 0x561be64a2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:36:58.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.494+0000 7f0e8aa00700 1 --2- 172.21.15.1:0/3023394049 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561be64959b0 0x561be64a0320 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c09a3e0 tx=0x7f0e7c06d5e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1fb0f5e0b6f99833 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:58.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.494+0000 7f0e8b400700 1 --2- 172.21.15.1:0/2859180377 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f0e6c0a0710 0x7f0e6c0a2bd0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0e84009fd0 tx=0x7f0e8400b040 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:58.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.494+0000 7f0e7b400700 1 -- 172.21.15.1:0/3023394049 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c0850e0 con 0x561be64959b0 2024-02-02T15:36:58.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.494+0000 7f0e7b400700 1 -- 172.21.15.1:0/3023394049 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e7c09ab50 con 0x561be64959b0 2024-02-02T15:36:58.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.494+0000 7f0e7b400700 1 -- 172.21.15.1:0/3023394049 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c0a2430 con 0x561be64959b0 2024-02-02T15:36:58.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.495+0000 7f0ea51a4b80 1 -- 172.21.15.1:0/3023394049 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561be64959b0 msgr2=0x561be64a0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:58.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.495+0000 7f0ea51a4b80 1 --2- 172.21.15.1:0/3023394049 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561be64959b0 0x561be64a0320 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c09a3e0 tx=0x7f0e7c06d5e0 comp rx=0 tx=0).stop 2024-02-02T15:36:58.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.495+0000 7f0ea51a4b80 1 -- 172.21.15.1:0/3023394049 shutdown_connections 2024-02-02T15:36:58.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.495+0000 7f0ea51a4b80 1 --2- 172.21.15.1:0/3023394049 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561be64a0860 0x561be64a2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:58.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.495+0000 7f0ea51a4b80 1 --2- 172.21.15.1:0/3023394049 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561be6495090 0x561be6495470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:58.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.495+0000 7f0ea51a4b80 1 --2- 172.21.15.1:0/3023394049 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561be64959b0 0x561be64a0320 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:58.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.495+0000 7f0ea51a4b80 1 -- 172.21.15.1:0/3023394049 >> 172.21.15.1:0/3023394049 conn(0x561be63f4e30 msgr2=0x561be6491430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:58.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.495+0000 7f0ea51a4b80 1 -- 172.21.15.1:0/3023394049 shutdown_connections 2024-02-02T15:36:58.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.495+0000 7f0ea51a4b80 1 -- 172.21.15.1:0/3023394049 wait complete. 2024-02-02T15:36:58.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.496+0000 7f0ea51a4b80 1 -- start start 2024-02-02T15:36:58.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.496+0000 7f0ea51a4b80 1 Processor -- start 2024-02-02T15:36:58.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.497+0000 7f0ea51a4b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561be6495090 0x561be64a0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:58.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.497+0000 7f0ea51a4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561be64959b0 0x561be64a1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:58.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.497+0000 7f0ea51a4b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561be64a0860 0x561be64a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:58.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.497+0000 7f0ea51a4b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x561be63e6dd0 con 0x561be6495090 2024-02-02T15:36:58.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.497+0000 7f0ea51a4b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x561be6382e00 con 0x561be64a0860 2024-02-02T15:36:58.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.497+0000 7f0ea51a4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561be6382aa0 con 0x561be64959b0 2024-02-02T15:36:58.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.497+0000 7f0ea51a4b80 10 client.0 _ll_register_callbacks cb 0x561be6384310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:36:58.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.497+0000 7f0e8a000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561be64959b0 0x561be64a1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:58.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.497+0000 7f0e8b400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561be64a0860 0x561be64a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:58.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.497+0000 7f0e8a000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561be64959b0 0x561be64a1180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:34330/0 (socket says 172.21.15.1:34330) 2024-02-02T15:36:58.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.497+0000 7f0e8a000700 1 -- 172.21.15.1:0/489527104 learned_addr learned my addr 172.21.15.1:0/489527104 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:36:58.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.497+0000 7f0e8aa00700 1 --2- 172.21.15.1:0/489527104 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561be6495090 0x561be64a0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:58.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.498+0000 7f0e8a000700 1 -- 172.21.15.1:0/489527104 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561be64a0860 msgr2=0x561be64a16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:58.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.498+0000 7f0e8a000700 1 --2- 172.21.15.1:0/489527104 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561be64a0860 0x561be64a16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:58.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.498+0000 7f0e8a000700 1 -- 172.21.15.1:0/489527104 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561be6495090 msgr2=0x561be64a0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:58.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.498+0000 7f0e8a000700 1 --2- 172.21.15.1:0/489527104 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561be6495090 0x561be64a0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:58.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.498+0000 7f0e8a000700 1 -- 172.21.15.1:0/489527104 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e74009c90 con 0x561be64959b0 2024-02-02T15:36:58.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.498+0000 7f0e8a000700 1 --2- 172.21.15.1:0/489527104 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561be64959b0 0x561be64a1180 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f0e74011d20 tx=0x7f0e7400f6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2cac7516a9c50d20 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:58.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.498+0000 7f0e73e00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:36:58.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.500+0000 7f0e73e00700 1 -- 172.21.15.1:0/489527104 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e74016070 con 0x561be64959b0 2024-02-02T15:36:58.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.500+0000 7f0e73e00700 1 -- 172.21.15.1:0/489527104 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f0e74011020 con 0x561be64959b0 2024-02-02T15:36:58.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.500+0000 7f0ea51a4b80 1 -- 172.21.15.1:0/489527104 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561be6489170 con 0x561be64959b0 2024-02-02T15:36:58.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.500+0000 7f0e73e00700 1 -- 172.21.15.1:0/489527104 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0e740111e0 con 0x561be64959b0 2024-02-02T15:36:58.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.500+0000 7f0ea51a4b80 10 client.28324 fetch_fsmap learned FSMap version 46 2024-02-02T15:36:58.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.500+0000 7f0ea51a4b80 1 -- 172.21.15.1:0/489527104 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x7f0e8c1a61b0 con 0x561be64959b0 2024-02-02T15:36:58.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.500+0000 7f0e73e00700 1 -- 172.21.15.1:0/489527104 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561be6489170 con 0x561be64959b0 2024-02-02T15:36:58.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.500+0000 7f0e73e00700 1 -- 172.21.15.1:0/489527104 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 46) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f0e7401a480 con 0x561be64959b0 2024-02-02T15:36:58.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.501+0000 7f0ea51a4b80 10 client.28324 fetch_fsmap finished waiting for FSMap version 46 2024-02-02T15:36:58.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.501+0000 7f0ea51a4b80 10 client.28324 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:36:58.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.501+0000 7f0ea51a4b80 20 client.28324 populate_metadata read hostname 'smithi001' 2024-02-02T15:36:58.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.501+0000 7f0ea51a4b80 1 --2- 172.21.15.1:0/489527104 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x561be64b0020 0x561be64b2440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:58.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.501+0000 7f0ea51a4b80 4 client.28324 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:36:58.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.501+0000 7f0ea51a4b80 1 -- 172.21.15.1:0/489527104 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561be63dfe50 con 0x561be64b0020 2024-02-02T15:36:58.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.501+0000 7f0e8aa00700 1 --2- 172.21.15.1:0/489527104 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x561be64b0020 0x561be64b2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:58.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.501+0000 7f0e8aa00700 1 --2- 172.21.15.1:0/489527104 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x561be64b0020 0x561be64b2440 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7db8766dbc1507f server_cookie=b7ec90b63872021b in_seq=0 out_seq=0 2024-02-02T15:36:58.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.501+0000 7f0e73e00700 10 client.28324 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:36:58.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.502+0000 7f0e73e00700 1 -- 172.21.15.1:0/489527104 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x561be63dfe50 con 0x561be64b0020 2024-02-02T15:36:58.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.502+0000 7f0e73e00700 10 client.28324 handle_command_reply: tid=0 2024-02-02T15:36:58.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.503+0000 7f0ea51a4b80 1 client.28324 shutdown 2024-02-02T15:36:58.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.503+0000 7f0ea51a4b80 1 -- 172.21.15.1:0/489527104 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561be64959b0 msgr2=0x561be64a1180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:58.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.503+0000 7f0ea51a4b80 1 --2- 172.21.15.1:0/489527104 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561be64959b0 0x561be64a1180 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f0e74011d20 tx=0x7f0e7400f6a0 comp rx=0 tx=0).stop 2024-02-02T15:36:58.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.503+0000 7f0ea51a4b80 1 -- 172.21.15.1:0/489527104 shutdown_connections 2024-02-02T15:36:58.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.503+0000 7f0ea51a4b80 1 --2- 172.21.15.1:0/489527104 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x561be64b0020 0x561be64b2440 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:58.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.503+0000 7f0ea51a4b80 1 --2- 172.21.15.1:0/489527104 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561be64a0860 0x561be64a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:58.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.503+0000 7f0ea51a4b80 1 --2- 172.21.15.1:0/489527104 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561be6495090 0x561be64a0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:58.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.503+0000 7f0e73e00700 0 client.28324 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:36:58.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.503+0000 7f0ea51a4b80 1 --2- 172.21.15.1:0/489527104 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561be64959b0 0x561be64a1180 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:58.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.503+0000 7f0ea51a4b80 1 -- 172.21.15.1:0/489527104 >> 172.21.15.1:0/489527104 conn(0x561be63f4e30 msgr2=0x561be6491020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:58.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.503+0000 7f0ea51a4b80 1 -- 172.21.15.1:0/489527104 shutdown_connections 2024-02-02T15:36:58.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.503+0000 7f0ea51a4b80 1 -- 172.21.15.1:0/489527104 wait complete. 2024-02-02T15:36:58.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.503+0000 7f0ea51a4b80 20 client.28324 trim_cache size 0 max 16384 2024-02-02T15:36:58.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.521+0000 7f0ea51a4b80 1 Processor -- start 2024-02-02T15:36:58.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.521+0000 7f0ea51a4b80 1 -- start start 2024-02-02T15:36:58.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.521+0000 7f0ea51a4b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561be6495090 0x561be64a4c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:58.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.521+0000 7f0ea51a4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561be64a0860 0x561be649baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:58.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.521+0000 7f0ea51a4b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561be649b250 0x561be649c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:58.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.521+0000 7f0ea51a4b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x561be626a720 con 0x561be6495090 2024-02-02T15:36:58.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.521+0000 7f0ea51a4b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x561be648b580 con 0x561be649b250 2024-02-02T15:36:58.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.521+0000 7f0ea51a4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561be649ff10 con 0x561be64a0860 2024-02-02T15:36:58.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.522+0000 7f0e8aa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561be6495090 0x561be64a4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:58.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.522+0000 7f0e8a000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561be64a0860 0x561be649baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:58.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.522+0000 7f0e8b400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561be649b250 0x561be649c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:58.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.522+0000 7f0e8a000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561be64a0860 0x561be649baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:34336/0 (socket says 172.21.15.1:34336) 2024-02-02T15:36:58.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.522+0000 7f0e8aa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561be6495090 0x561be64a4c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48946/0 (socket says 172.21.15.1:48946) 2024-02-02T15:36:58.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.522+0000 7f0e8b400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561be649b250 0x561be649c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59508/0 (socket says 172.21.15.1:59508) 2024-02-02T15:36:58.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.522+0000 7f0e8a000700 1 -- 172.21.15.1:0/827889781 learned_addr learned my addr 172.21.15.1:0/827889781 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:36:58.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.522+0000 7f0e8a000700 1 -- 172.21.15.1:0/827889781 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561be649b250 msgr2=0x561be649c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:58.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.522+0000 7f0e8a000700 1 --2- 172.21.15.1:0/827889781 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561be649b250 0x561be649c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:58.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.522+0000 7f0e8a000700 1 -- 172.21.15.1:0/827889781 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561be6495090 msgr2=0x561be64a4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:58.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.522+0000 7f0e8a000700 1 --2- 172.21.15.1:0/827889781 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561be6495090 0x561be64a4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:58.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.522+0000 7f0e8a000700 1 -- 172.21.15.1:0/827889781 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561be6214be0 con 0x561be64a0860 2024-02-02T15:36:58.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.522+0000 7f0e8aa00700 1 --2- 172.21.15.1:0/827889781 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561be6495090 0x561be64a4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:36:58.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.522+0000 7f0e8a000700 1 --2- 172.21.15.1:0/827889781 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561be64a0860 0x561be649baf0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f0e74013040 tx=0x7f0e7400ff00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4934f7bf4452a71 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:58.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.523+0000 7f0e7be00700 1 -- 172.21.15.1:0/827889781 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e74016040 con 0x561be64a0860 2024-02-02T15:36:58.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.523+0000 7f0e7be00700 1 -- 172.21.15.1:0/827889781 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e740092d0 con 0x561be64a0860 2024-02-02T15:36:58.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.523+0000 7f0e7be00700 1 -- 172.21.15.1:0/827889781 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e74011310 con 0x561be64a0860 2024-02-02T15:36:58.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.523+0000 7f0ea51a4b80 1 -- 172.21.15.1:0/827889781 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561be64a0860 msgr2=0x561be649baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:58.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.523+0000 7f0ea51a4b80 1 --2- 172.21.15.1:0/827889781 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561be64a0860 0x561be649baf0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f0e74013040 tx=0x7f0e7400ff00 comp rx=0 tx=0).stop 2024-02-02T15:36:58.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.523+0000 7f0ea51a4b80 1 -- 172.21.15.1:0/827889781 shutdown_connections 2024-02-02T15:36:58.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.523+0000 7f0ea51a4b80 1 --2- 172.21.15.1:0/827889781 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561be649b250 0x561be649c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:58.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.523+0000 7f0ea51a4b80 1 --2- 172.21.15.1:0/827889781 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561be6495090 0x561be64a4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:58.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.523+0000 7f0ea51a4b80 1 --2- 172.21.15.1:0/827889781 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561be64a0860 0x561be649baf0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:58.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.523+0000 7f0ea51a4b80 1 -- 172.21.15.1:0/827889781 >> 172.21.15.1:0/827889781 conn(0x561be63f4e30 msgr2=0x561be649e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:58.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.523+0000 7f0ea51a4b80 1 -- 172.21.15.1:0/827889781 shutdown_connections 2024-02-02T15:36:58.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.524+0000 7f0ea51a4b80 1 -- 172.21.15.1:0/827889781 wait complete. 2024-02-02T15:36:58.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.524+0000 7f0ea51a4b80 1 -- start start 2024-02-02T15:36:58.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.524+0000 7f0ea51a4b80 1 Processor -- start 2024-02-02T15:36:58.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.525+0000 7f0ea51a4b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561be6495090 0x561be64fbe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:58.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.525+0000 7f0ea51a4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561be649b250 0x561be64fc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:58.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.525+0000 7f0ea51a4b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561be64a0860 0x561be64f6220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:58.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.525+0000 7f0ea51a4b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x561be64f6800 con 0x561be6495090 2024-02-02T15:36:58.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.525+0000 7f0ea51a4b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x561be64f6970 con 0x561be64a0860 2024-02-02T15:36:58.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.525+0000 7f0ea51a4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561be64f6b30 con 0x561be649b250 2024-02-02T15:36:58.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.525+0000 7f0ea51a4b80 10 client.0 _ll_register_callbacks cb 0x561be6384310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:36:58.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.525+0000 7f0e8a000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561be649b250 0x561be64fc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:58.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.525+0000 7f0e8a000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561be649b250 0x561be64fc3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:34346/0 (socket says 172.21.15.1:34346) 2024-02-02T15:36:58.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.525+0000 7f0e8a000700 1 -- 172.21.15.1:0/340219986 learned_addr learned my addr 172.21.15.1:0/340219986 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:36:58.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.525+0000 7f0e8aa00700 1 --2- 172.21.15.1:0/340219986 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561be6495090 0x561be64fbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:58.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.525+0000 7f0e8b400700 1 --2- 172.21.15.1:0/340219986 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561be64a0860 0x561be64f6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:58.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.525+0000 7f0e8a000700 1 -- 172.21.15.1:0/340219986 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561be64a0860 msgr2=0x561be64f6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:58.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.525+0000 7f0e8a000700 1 --2- 172.21.15.1:0/340219986 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561be64a0860 0x561be64f6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:58.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.525+0000 7f0e8a000700 1 -- 172.21.15.1:0/340219986 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561be6495090 msgr2=0x561be64fbe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:58.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.525+0000 7f0e8a000700 1 --2- 172.21.15.1:0/340219986 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561be6495090 0x561be64fbe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:58.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.525+0000 7f0e8a000700 1 -- 172.21.15.1:0/340219986 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561be6268430 con 0x561be649b250 2024-02-02T15:36:58.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.526+0000 7f0e8a000700 1 --2- 172.21.15.1:0/340219986 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561be649b250 0x561be64fc3c0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f0e7400f6e0 tx=0x7f0e740118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c4a8c84537488d28 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:58.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.526+0000 7f0e73e00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:36:58.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.526+0000 7f0e73e00700 1 -- 172.21.15.1:0/340219986 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e74016070 con 0x561be649b250 2024-02-02T15:36:58.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.526+0000 7f0ea51a4b80 1 -- 172.21.15.1:0/340219986 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0e7c09ab50 con 0x561be649b250 2024-02-02T15:36:58.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.526+0000 7f0e73e00700 1 -- 172.21.15.1:0/340219986 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f0e74000e40 con 0x561be649b250 2024-02-02T15:36:58.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.526+0000 7f0e73e00700 1 -- 172.21.15.1:0/340219986 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0e74011020 con 0x561be649b250 2024-02-02T15:36:58.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.526+0000 7f0ea51a4b80 10 client.28330 fetch_fsmap learned FSMap version 46 2024-02-02T15:36:58.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.526+0000 7f0ea51a4b80 1 -- 172.21.15.1:0/340219986 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x7f0e8c1a61b0 con 0x561be649b250 2024-02-02T15:36:58.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.526+0000 7f0e73e00700 1 -- 172.21.15.1:0/340219986 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0e7c09ab50 con 0x561be649b250 2024-02-02T15:36:58.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.527+0000 7f0e73e00700 1 -- 172.21.15.1:0/340219986 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 46) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f0e74021030 con 0x561be649b250 2024-02-02T15:36:58.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.527+0000 7f0ea51a4b80 10 client.28330 fetch_fsmap finished waiting for FSMap version 46 2024-02-02T15:36:58.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.527+0000 7f0ea51a4b80 10 client.28330 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:36:58.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.527+0000 7f0ea51a4b80 20 client.28330 populate_metadata read hostname 'smithi001' 2024-02-02T15:36:58.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.527+0000 7f0ea51a4b80 1 --2- 172.21.15.1:0/340219986 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x561be6503a10 0x561be6505e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:58.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.527+0000 7f0ea51a4b80 4 client.28330 mds_command: new command op to 24557 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-02T15:36:58.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.527+0000 7f0ea51a4b80 1 -- 172.21.15.1:0/340219986 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x561be63a5fd0 con 0x561be6503a10 2024-02-02T15:36:58.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.527+0000 7f0e8aa00700 1 --2- 172.21.15.1:0/340219986 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x561be6503a10 0x561be6505e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:58.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.527+0000 7f0e8aa00700 1 --2- 172.21.15.1:0/340219986 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x561be6503a10 0x561be6505e30 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c64eea03b4d84b server_cookie=e1466d50284f4256 in_seq=0 out_seq=0 2024-02-02T15:36:58.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.527+0000 7f0e73e00700 10 client.28330 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:36:58.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.529+0000 7f0e73e00700 1 -- 172.21.15.1:0/340219986 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x561be63a5fd0 con 0x561be6503a10 2024-02-02T15:36:58.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.529+0000 7f0e73e00700 10 client.28330 handle_command_reply: tid=0 2024-02-02T15:36:58.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.529+0000 7f0ea51a4b80 1 client.28330 shutdown 2024-02-02T15:36:58.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.529+0000 7f0ea51a4b80 1 -- 172.21.15.1:0/340219986 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561be649b250 msgr2=0x561be64fc3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:58.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.529+0000 7f0ea51a4b80 1 --2- 172.21.15.1:0/340219986 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561be649b250 0x561be64fc3c0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f0e7400f6e0 tx=0x7f0e740118d0 comp rx=0 tx=0).stop 2024-02-02T15:36:58.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.529+0000 7f0ea51a4b80 1 -- 172.21.15.1:0/340219986 shutdown_connections 2024-02-02T15:36:58.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.529+0000 7f0ea51a4b80 1 --2- 172.21.15.1:0/340219986 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x561be6503a10 0x561be6505e30 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:58.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.529+0000 7f0ea51a4b80 1 --2- 172.21.15.1:0/340219986 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561be64a0860 0x561be64f6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:58.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.529+0000 7f0ea51a4b80 1 --2- 172.21.15.1:0/340219986 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561be6495090 0x561be64fbe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:58.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.529+0000 7f0e73e00700 0 client.28330 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:36:58.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.529+0000 7f0ea51a4b80 1 --2- 172.21.15.1:0/340219986 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561be649b250 0x561be64fc3c0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:58.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.529+0000 7f0ea51a4b80 1 -- 172.21.15.1:0/340219986 >> 172.21.15.1:0/340219986 conn(0x561be63f4e30 msgr2=0x561be6490640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:58.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.529+0000 7f0ea51a4b80 1 -- 172.21.15.1:0/340219986 shutdown_connections 2024-02-02T15:36:58.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.529+0000 7f0ea51a4b80 1 -- 172.21.15.1:0/340219986 wait complete. 2024-02-02T15:36:58.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.529+0000 7f0ea51a4b80 20 client.28330 trim_cache size 0 max 16384 2024-02-02T15:36:58.533 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:36:58.533 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2024-02-02T15:36:58.533 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "1a9a6929-7be7-4996-94c8-09e64c9b539d", 2024-02-02T15:36:58.533 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2024-02-02T15:36:58.533 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:36:58.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.530+0000 7f0e91600700 1 -- 172.21.15.1:0/2859180377 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f0e6c0a0710 msgr2=0x7f0e6c0a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:58.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.530+0000 7f0e91600700 1 --2- 172.21.15.1:0/2859180377 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f0e6c0a0710 0x7f0e6c0a2bd0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0e84009fd0 tx=0x7f0e8400b040 comp rx=0 tx=0).stop 2024-02-02T15:36:58.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.530+0000 7f0e91600700 1 -- 172.21.15.1:0/2859180377 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e8c101000 msgr2=0x7f0e8c1991a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:58.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.530+0000 7f0e91600700 1 --2- 172.21.15.1:0/2859180377 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e8c101000 0x7f0e8c1991a0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c005910 tx=0x7f0e7c004730 comp rx=0 tx=0).stop 2024-02-02T15:36:58.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.530+0000 7f0e91600700 1 -- 172.21.15.1:0/2859180377 shutdown_connections 2024-02-02T15:36:58.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.530+0000 7f0e91600700 1 --2- 172.21.15.1:0/2859180377 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f0e6c0a0710 0x7f0e6c0a2bd0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:58.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.530+0000 7f0e91600700 1 --2- 172.21.15.1:0/2859180377 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0e8c19dad0 0x7f0e8c19df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:58.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.530+0000 7f0e91600700 1 --2- 172.21.15.1:0/2859180377 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0e8c101000 0x7f0e8c1991a0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:58.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.530+0000 7f0e91600700 1 --2- 172.21.15.1:0/2859180377 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0e8c106120 0x7f0e8c1996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:58.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.530+0000 7f0e91600700 1 -- 172.21.15.1:0/2859180377 >> 172.21.15.1:0/2859180377 conn(0x7f0e8c0fc860 msgr2=0x7f0e8c1029c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:58.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.530+0000 7f0e91600700 1 -- 172.21.15.1:0/2859180377 shutdown_connections 2024-02-02T15:36:58.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.530+0000 7f0e91600700 1 -- 172.21.15.1:0/2859180377 wait complete. 2024-02-02T15:36:58.551 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-02T15:36:58.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.819+0000 7f248f000700 1 -- 172.21.15.1:0/4144069047 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2488106020 msgr2=0x7f248810dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:58.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.819+0000 7f248f000700 1 --2- 172.21.15.1:0/4144069047 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2488106020 0x7f248810dba0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f2480009f90 tx=0x7f248000b330 comp rx=0 tx=0).stop 2024-02-02T15:36:58.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.819+0000 7f248f000700 1 -- 172.21.15.1:0/4144069047 shutdown_connections 2024-02-02T15:36:58.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.819+0000 7f248f000700 1 --2- 172.21.15.1:0/4144069047 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2488106020 0x7f248810dba0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:58.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.819+0000 7f248f000700 1 --2- 172.21.15.1:0/4144069047 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2488100f00 0x7f24881012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:58.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.819+0000 7f248f000700 1 --2- 172.21.15.1:0/4144069047 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f24881018b0 0x7f24881058f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:58.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.819+0000 7f248f000700 1 -- 172.21.15.1:0/4144069047 >> 172.21.15.1:0/4144069047 conn(0x7f24880fc760 msgr2=0x7f24880feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:58.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.820+0000 7f248f000700 1 -- 172.21.15.1:0/4144069047 shutdown_connections 2024-02-02T15:36:58.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.820+0000 7f248f000700 1 -- 172.21.15.1:0/4144069047 wait complete. 2024-02-02T15:36:58.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.820+0000 7f248f000700 1 Processor -- start 2024-02-02T15:36:58.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.821+0000 7f248f000700 1 -- start start 2024-02-02T15:36:58.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.821+0000 7f248f000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2488100f00 0x7f2488073040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:58.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.821+0000 7f248f000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2488106020 0x7f2488073580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:58.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.821+0000 7f248f000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2488073ac0 0x7f24881a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:58.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.821+0000 7f248f000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24881a5540 con 0x7f2488100f00 2024-02-02T15:36:58.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.821+0000 7f248f000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24881a56b0 con 0x7f2488106020 2024-02-02T15:36:58.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.821+0000 7f248f000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24881a5870 con 0x7f2488073ac0 2024-02-02T15:36:58.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.821+0000 7f2487e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2488100f00 0x7f2488073040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:58.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.821+0000 7f2487400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2488106020 0x7f2488073580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:58.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.821+0000 7f2487400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2488106020 0x7f2488073580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59528/0 (socket says 172.21.15.1:59528) 2024-02-02T15:36:58.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.821+0000 7f2487e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2488100f00 0x7f2488073040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48964/0 (socket says 172.21.15.1:48964) 2024-02-02T15:36:58.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.821+0000 7f2487400700 1 -- 172.21.15.1:0/2411236433 learned_addr learned my addr 172.21.15.1:0/2411236433 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:36:58.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.821+0000 7f248d000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2488073ac0 0x7f24881a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:58.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.822+0000 7f2487400700 1 -- 172.21.15.1:0/2411236433 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2488073ac0 msgr2=0x7f24881a4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:58.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.822+0000 7f2487400700 1 --2- 172.21.15.1:0/2411236433 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2488073ac0 0x7f24881a4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:58.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.822+0000 7f2487400700 1 -- 172.21.15.1:0/2411236433 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2488100f00 msgr2=0x7f2488073040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:58.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.822+0000 7f2487400700 1 --2- 172.21.15.1:0/2411236433 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2488100f00 0x7f2488073040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:58.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.822+0000 7f2487400700 1 -- 172.21.15.1:0/2411236433 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2480009c40 con 0x7f2488106020 2024-02-02T15:36:58.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.822+0000 7f248d000700 1 --2- 172.21.15.1:0/2411236433 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2488073ac0 0x7f24881a4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:36:58.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.822+0000 7f2487400700 1 --2- 172.21.15.1:0/2411236433 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2488106020 0x7f2488073580 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f247000e970 tx=0x7f247000ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:58.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.822+0000 7f2484c00700 1 -- 172.21.15.1:0/2411236433 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2470009790 con 0x7f2488106020 2024-02-02T15:36:58.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.822+0000 7f2484c00700 1 -- 172.21.15.1:0/2411236433 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f247000c7c0 con 0x7f2488106020 2024-02-02T15:36:58.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.822+0000 7f248f000700 1 -- 172.21.15.1:0/2411236433 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24881a5b50 con 0x7f2488106020 2024-02-02T15:36:58.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.822+0000 7f2484c00700 1 -- 172.21.15.1:0/2411236433 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24700054b0 con 0x7f2488106020 2024-02-02T15:36:58.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.823+0000 7f248f000700 1 -- 172.21.15.1:0/2411236433 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24881a6060 con 0x7f2488106020 2024-02-02T15:36:58.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.825+0000 7f2484c00700 1 -- 172.21.15.1:0/2411236433 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f247000c930 con 0x7f2488106020 2024-02-02T15:36:58.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.827+0000 7f2484c00700 1 --2- 172.21.15.1:0/2411236433 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f24680a0690 0x7f24680a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:58.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.827+0000 7f24a2c17b80 1 Processor -- start 2024-02-02T15:36:58.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.827+0000 7f2484c00700 1 -- 172.21.15.1:0/2411236433 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f2470014070 con 0x7f2488106020 2024-02-02T15:36:58.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.827+0000 7f24a2c17b80 1 -- start start 2024-02-02T15:36:58.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.827+0000 7f2487e00700 1 --2- 172.21.15.1:0/2411236433 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f24680a0690 0x7f24680a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:58.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.828+0000 7f24a2c17b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556ab4db010 0x5556ab4db3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:58.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.828+0000 7f24a2c17b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5556ab4db930 0x5556ab4e62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:58.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.828+0000 7f24a2c17b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556ab4e67e0 0x5556ab4e8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:58.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.828+0000 7f24a2c17b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556ab44f4c0 con 0x5556ab4db930 2024-02-02T15:36:58.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.828+0000 7f24a2c17b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556ab433aa0 con 0x5556ab4e67e0 2024-02-02T15:36:58.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.828+0000 7f24a2c17b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556ab431460 con 0x5556ab4db010 2024-02-02T15:36:58.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.828+0000 7f2487400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5556ab4db930 0x5556ab4e62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:58.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.828+0000 7f2487e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556ab4db010 0x5556ab4db3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:58.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.828+0000 7f2487400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5556ab4db930 0x5556ab4e62a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48978/0 (socket says 172.21.15.1:48978) 2024-02-02T15:36:58.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.828+0000 7f2487400700 1 -- 172.21.15.1:0/3710996135 learned_addr learned my addr 172.21.15.1:0/3710996135 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:36:58.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.828+0000 7f248d000700 1 --2- 172.21.15.1:0/3710996135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556ab4e67e0 0x5556ab4e8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:58.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.828+0000 7f2487e00700 1 --2- 172.21.15.1:0/2411236433 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f24680a0690 0x7f24680a2b50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2478005fd0 tx=0x7f2478009310 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:58.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.828+0000 7f2487400700 1 -- 172.21.15.1:0/3710996135 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556ab4db010 msgr2=0x5556ab4db3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:58.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.828+0000 7f2487400700 1 --2- 172.21.15.1:0/3710996135 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556ab4db010 0x5556ab4db3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:58.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.828+0000 7f2487400700 1 -- 172.21.15.1:0/3710996135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556ab4e67e0 msgr2=0x5556ab4e8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:58.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.828+0000 7f2487400700 1 --2- 172.21.15.1:0/3710996135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556ab4e67e0 0x5556ab4e8bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:58.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.828+0000 7f2487400700 1 -- 172.21.15.1:0/3710996135 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5556ab2ae3b0 con 0x5556ab4db930 2024-02-02T15:36:58.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.828+0000 7f248d000700 1 --2- 172.21.15.1:0/3710996135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556ab4e67e0 0x5556ab4e8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:36:58.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.829+0000 7f2487400700 1 --2- 172.21.15.1:0/3710996135 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5556ab4db930 0x5556ab4e62a0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f2470095e00 tx=0x7f2470099b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=950aa90a8997a0f5 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:58.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.829+0000 7f247d600700 1 -- 172.21.15.1:0/3710996135 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f247009d1c0 con 0x5556ab4db930 2024-02-02T15:36:58.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.829+0000 7f247d600700 1 -- 172.21.15.1:0/3710996135 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f24700a1400 con 0x5556ab4db930 2024-02-02T15:36:58.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.829+0000 7f247d600700 1 -- 172.21.15.1:0/3710996135 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2470016330 con 0x5556ab4db930 2024-02-02T15:36:58.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.830+0000 7f24a2c17b80 1 -- 172.21.15.1:0/3710996135 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5556ab4db930 msgr2=0x5556ab4e62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:58.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.830+0000 7f24a2c17b80 1 --2- 172.21.15.1:0/3710996135 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5556ab4db930 0x5556ab4e62a0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f2470095e00 tx=0x7f2470099b10 comp rx=0 tx=0).stop 2024-02-02T15:36:58.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.830+0000 7f24a2c17b80 1 -- 172.21.15.1:0/3710996135 shutdown_connections 2024-02-02T15:36:58.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.830+0000 7f24a2c17b80 1 --2- 172.21.15.1:0/3710996135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556ab4e67e0 0x5556ab4e8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:58.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.830+0000 7f24a2c17b80 1 --2- 172.21.15.1:0/3710996135 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556ab4db010 0x5556ab4db3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:58.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.830+0000 7f24a2c17b80 1 --2- 172.21.15.1:0/3710996135 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5556ab4db930 0x5556ab4e62a0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:58.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.830+0000 7f24a2c17b80 1 -- 172.21.15.1:0/3710996135 >> 172.21.15.1:0/3710996135 conn(0x5556ab43adb0 msgr2=0x5556ab4d73b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:58.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.830+0000 7f24a2c17b80 1 -- 172.21.15.1:0/3710996135 shutdown_connections 2024-02-02T15:36:58.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.830+0000 7f24a2c17b80 1 -- 172.21.15.1:0/3710996135 wait complete. 2024-02-02T15:36:58.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.830+0000 7f24a2c17b80 1 -- start start 2024-02-02T15:36:58.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.831+0000 7f24a2c17b80 1 Processor -- start 2024-02-02T15:36:58.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.831+0000 7f24a2c17b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556ab4db010 0x5556ab4e6280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:58.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.831+0000 7f24a2c17b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5556ab4db930 0x5556ab4e3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:58.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.831+0000 7f24a2c17b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556ab4e67e0 0x5556ab4e3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:58.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.831+0000 7f24a2c17b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556ab42cd50 con 0x5556ab4db930 2024-02-02T15:36:58.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.831+0000 7f24a2c17b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556ab3c8d80 con 0x5556ab4db010 2024-02-02T15:36:58.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.831+0000 7f24a2c17b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556ab3c8a20 con 0x5556ab4e67e0 2024-02-02T15:36:58.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.831+0000 7f24a2c17b80 10 client.0 _ll_register_callbacks cb 0x5556ab3ca290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:36:58.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.832+0000 7f2487e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556ab4db010 0x5556ab4e6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:58.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.832+0000 7f2487400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5556ab4db930 0x5556ab4e3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:58.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.832+0000 7f2487e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556ab4db010 0x5556ab4e6280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59550/0 (socket says 172.21.15.1:59550) 2024-02-02T15:36:58.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.832+0000 7f2487e00700 1 -- 172.21.15.1:0/3498357135 learned_addr learned my addr 172.21.15.1:0/3498357135 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:36:58.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.832+0000 7f248d000700 1 --2- 172.21.15.1:0/3498357135 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556ab4e67e0 0x5556ab4e3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:58.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.832+0000 7f2487e00700 1 -- 172.21.15.1:0/3498357135 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556ab4e67e0 msgr2=0x5556ab4e3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:58.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.832+0000 7f2487e00700 1 --2- 172.21.15.1:0/3498357135 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556ab4e67e0 0x5556ab4e3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:58.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.832+0000 7f2487e00700 1 -- 172.21.15.1:0/3498357135 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5556ab4db930 msgr2=0x5556ab4e3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:58.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.832+0000 7f2487e00700 1 --2- 172.21.15.1:0/3498357135 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5556ab4db930 0x5556ab4e3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:58.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.832+0000 7f2487e00700 1 -- 172.21.15.1:0/3498357135 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f248810e410 con 0x5556ab4db010 2024-02-02T15:36:58.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.832+0000 7f2487e00700 1 --2- 172.21.15.1:0/3498357135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556ab4db010 0x5556ab4e6280 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f2478004610 tx=0x7f24780047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44ccaf922103c35f server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:58.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.832+0000 7f2476000700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:36:58.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.833+0000 7f2476000700 1 -- 172.21.15.1:0/3498357135 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2478021070 con 0x5556ab4db010 2024-02-02T15:36:58.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.833+0000 7f24a2c17b80 1 -- 172.21.15.1:0/3498357135 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5556ab4cf0f0 con 0x5556ab4db010 2024-02-02T15:36:58.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.833+0000 7f2476000700 1 -- 172.21.15.1:0/3498357135 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f2478003e20 con 0x5556ab4db010 2024-02-02T15:36:58.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.833+0000 7f2476000700 1 -- 172.21.15.1:0/3498357135 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f247801b670 con 0x5556ab4db010 2024-02-02T15:36:58.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.833+0000 7f24a2c17b80 10 client.28343 fetch_fsmap learned FSMap version 46 2024-02-02T15:36:58.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.833+0000 7f24a2c17b80 1 -- 172.21.15.1:0/3498357135 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x7f24881a6060 con 0x5556ab4db010 2024-02-02T15:36:58.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.833+0000 7f2476000700 1 -- 172.21.15.1:0/3498357135 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5556ab4cf0f0 con 0x5556ab4db010 2024-02-02T15:36:58.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.833+0000 7f2476000700 1 -- 172.21.15.1:0/3498357135 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 46) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f247801c970 con 0x5556ab4db010 2024-02-02T15:36:58.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.833+0000 7f24a2c17b80 10 client.28343 fetch_fsmap finished waiting for FSMap version 46 2024-02-02T15:36:58.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.833+0000 7f24a2c17b80 10 client.28343 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:36:58.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.833+0000 7f24a2c17b80 20 client.28343 populate_metadata read hostname 'smithi001' 2024-02-02T15:36:58.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.833+0000 7f24a2c17b80 1 --2- 172.21.15.1:0/3498357135 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5556ab4f62e0 0x5556ab4f8700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:58.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.833+0000 7f24a2c17b80 4 client.28343 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:36:58.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.833+0000 7f24a2c17b80 1 -- 172.21.15.1:0/3498357135 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5556ab425dd0 con 0x5556ab4f62e0 2024-02-02T15:36:58.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.834+0000 7f2487400700 1 --2- 172.21.15.1:0/3498357135 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5556ab4f62e0 0x5556ab4f8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:58.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.834+0000 7f2487400700 1 --2- 172.21.15.1:0/3498357135 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5556ab4f62e0 0x5556ab4f8700 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5339142c43d3f654 server_cookie=a87eb45a8aed2baf in_seq=0 out_seq=0 2024-02-02T15:36:58.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.834+0000 7f2476000700 10 client.28343 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:36:58.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.835+0000 7f2476000700 1 -- 172.21.15.1:0/3498357135 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5556ab425dd0 con 0x5556ab4f62e0 2024-02-02T15:36:58.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.835+0000 7f2476000700 10 client.28343 handle_command_reply: tid=0 2024-02-02T15:36:58.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.835+0000 7f24a2c17b80 1 client.28343 shutdown 2024-02-02T15:36:58.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.835+0000 7f24a2c17b80 1 -- 172.21.15.1:0/3498357135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556ab4db010 msgr2=0x5556ab4e6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:58.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.835+0000 7f24a2c17b80 1 --2- 172.21.15.1:0/3498357135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556ab4db010 0x5556ab4e6280 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f2478004610 tx=0x7f24780047c0 comp rx=0 tx=0).stop 2024-02-02T15:36:58.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.835+0000 7f24a2c17b80 1 -- 172.21.15.1:0/3498357135 shutdown_connections 2024-02-02T15:36:58.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.835+0000 7f24a2c17b80 1 --2- 172.21.15.1:0/3498357135 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5556ab4f62e0 0x5556ab4f8700 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:58.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.835+0000 7f24a2c17b80 1 --2- 172.21.15.1:0/3498357135 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556ab4e67e0 0x5556ab4e3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:58.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.835+0000 7f24a2c17b80 1 --2- 172.21.15.1:0/3498357135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556ab4db010 0x5556ab4e6280 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:58.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.835+0000 7f24a2c17b80 1 --2- 172.21.15.1:0/3498357135 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5556ab4db930 0x5556ab4e3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:58.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.835+0000 7f24a2c17b80 1 -- 172.21.15.1:0/3498357135 >> 172.21.15.1:0/3498357135 conn(0x5556ab43adb0 msgr2=0x5556ab4d6fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:58.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.835+0000 7f2476000700 0 client.28343 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:36:58.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.835+0000 7f24a2c17b80 1 -- 172.21.15.1:0/3498357135 shutdown_connections 2024-02-02T15:36:58.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.835+0000 7f24a2c17b80 1 -- 172.21.15.1:0/3498357135 wait complete. 2024-02-02T15:36:58.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.836+0000 7f24a2c17b80 20 client.28343 trim_cache size 0 max 16384 2024-02-02T15:36:58.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.854+0000 7f24a2c17b80 1 Processor -- start 2024-02-02T15:36:58.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.854+0000 7f24a2c17b80 1 -- start start 2024-02-02T15:36:58.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.854+0000 7f24a2c17b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556ab4db010 0x5556ab4ed330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:58.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.854+0000 7f24a2c17b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5556ab4db930 0x5556ab4e5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:58.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.854+0000 7f24a2c17b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556ab4e67e0 0x5556ab4f1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:58.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.854+0000 7f24a2c17b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556ab2b06a0 con 0x5556ab4db930 2024-02-02T15:36:58.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.854+0000 7f24a2c17b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556ab4db7d0 con 0x5556ab4e67e0 2024-02-02T15:36:58.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.854+0000 7f24a2c17b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556ab4ced40 con 0x5556ab4db010 2024-02-02T15:36:58.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.855+0000 7f2487e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556ab4db010 0x5556ab4ed330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:58.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.855+0000 7f2487e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556ab4db010 0x5556ab4ed330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:34394/0 (socket says 172.21.15.1:34394) 2024-02-02T15:36:58.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.855+0000 7f2487e00700 1 -- 172.21.15.1:0/2266203423 learned_addr learned my addr 172.21.15.1:0/2266203423 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:36:58.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.855+0000 7f2487400700 1 --2- 172.21.15.1:0/2266203423 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5556ab4db930 0x5556ab4e5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:58.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.855+0000 7f248d000700 1 --2- 172.21.15.1:0/2266203423 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556ab4e67e0 0x5556ab4f1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:58.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.855+0000 7f2487e00700 1 -- 172.21.15.1:0/2266203423 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556ab4e67e0 msgr2=0x5556ab4f1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:58.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.855+0000 7f2487e00700 1 --2- 172.21.15.1:0/2266203423 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556ab4e67e0 0x5556ab4f1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:58.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.855+0000 7f2487e00700 1 -- 172.21.15.1:0/2266203423 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5556ab4db930 msgr2=0x5556ab4e5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:58.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.855+0000 7f2487e00700 1 --2- 172.21.15.1:0/2266203423 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5556ab4db930 0x5556ab4e5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:58.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.855+0000 7f2487e00700 1 -- 172.21.15.1:0/2266203423 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5556ab25ab60 con 0x5556ab4db010 2024-02-02T15:36:58.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.855+0000 7f248d000700 1 --2- 172.21.15.1:0/2266203423 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556ab4e67e0 0x5556ab4f1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:36:58.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.855+0000 7f2487400700 1 --2- 172.21.15.1:0/2266203423 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5556ab4db930 0x5556ab4e5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:36:58.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.855+0000 7f2487e00700 1 --2- 172.21.15.1:0/2266203423 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556ab4db010 0x5556ab4ed330 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f2478009ad0 tx=0x7f2478009e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab6d934ba00ef35b server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:58.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.856+0000 7f247e000700 1 -- 172.21.15.1:0/2266203423 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2478021070 con 0x5556ab4db010 2024-02-02T15:36:58.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.856+0000 7f247e000700 1 -- 172.21.15.1:0/2266203423 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2478003e70 con 0x5556ab4db010 2024-02-02T15:36:58.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.856+0000 7f247e000700 1 -- 172.21.15.1:0/2266203423 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f247801bd10 con 0x5556ab4db010 2024-02-02T15:36:58.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.856+0000 7f24a2c17b80 1 -- 172.21.15.1:0/2266203423 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556ab4db010 msgr2=0x5556ab4ed330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:58.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.856+0000 7f24a2c17b80 1 --2- 172.21.15.1:0/2266203423 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556ab4db010 0x5556ab4ed330 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f2478009ad0 tx=0x7f2478009e40 comp rx=0 tx=0).stop 2024-02-02T15:36:58.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.857+0000 7f24a2c17b80 1 -- 172.21.15.1:0/2266203423 shutdown_connections 2024-02-02T15:36:58.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.857+0000 7f24a2c17b80 1 --2- 172.21.15.1:0/2266203423 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556ab4e67e0 0x5556ab4f1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:58.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.857+0000 7f24a2c17b80 1 --2- 172.21.15.1:0/2266203423 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556ab4db010 0x5556ab4ed330 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:58.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.857+0000 7f24a2c17b80 1 --2- 172.21.15.1:0/2266203423 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5556ab4db930 0x5556ab4e5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:58.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.857+0000 7f24a2c17b80 1 -- 172.21.15.1:0/2266203423 >> 172.21.15.1:0/2266203423 conn(0x5556ab43adb0 msgr2=0x5556ab4f2560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:58.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.857+0000 7f24a2c17b80 1 -- 172.21.15.1:0/2266203423 shutdown_connections 2024-02-02T15:36:58.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.857+0000 7f24a2c17b80 1 -- 172.21.15.1:0/2266203423 wait complete. 2024-02-02T15:36:58.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.857+0000 7f24a2c17b80 1 -- start start 2024-02-02T15:36:58.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.858+0000 7f24a2c17b80 1 Processor -- start 2024-02-02T15:36:58.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.858+0000 7f24a2c17b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556ab4db010 0x5556ab4eb310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:58.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.858+0000 7f24a2c17b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5556ab4db930 0x5556ab4eb850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:58.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.858+0000 7f24a2c17b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556ab4e67e0 0x5556ab4ebd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:58.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.858+0000 7f24a2c17b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556ab54f740 con 0x5556ab4db930 2024-02-02T15:36:58.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.858+0000 7f24a2c17b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556ab54f8b0 con 0x5556ab4e67e0 2024-02-02T15:36:58.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.858+0000 7f24a2c17b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556ab54fa70 con 0x5556ab4db010 2024-02-02T15:36:58.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.858+0000 7f24a2c17b80 10 client.0 _ll_register_callbacks cb 0x5556ab3ca290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:36:58.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.859+0000 7f2487e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556ab4db010 0x5556ab4eb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:58.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.859+0000 7f2487e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556ab4db010 0x5556ab4eb310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:34398/0 (socket says 172.21.15.1:34398) 2024-02-02T15:36:58.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.859+0000 7f2487e00700 1 -- 172.21.15.1:0/561185465 learned_addr learned my addr 172.21.15.1:0/561185465 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:36:58.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.859+0000 7f2487400700 1 --2- 172.21.15.1:0/561185465 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5556ab4db930 0x5556ab4eb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:58.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.859+0000 7f248d000700 1 --2- 172.21.15.1:0/561185465 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556ab4e67e0 0x5556ab4ebd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:58.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.859+0000 7f2487e00700 1 -- 172.21.15.1:0/561185465 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556ab4e67e0 msgr2=0x5556ab4ebd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:58.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.859+0000 7f2487e00700 1 --2- 172.21.15.1:0/561185465 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556ab4e67e0 0x5556ab4ebd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:58.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.859+0000 7f2487e00700 1 -- 172.21.15.1:0/561185465 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5556ab4db930 msgr2=0x5556ab4eb850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:36:58.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.859+0000 7f2487e00700 1 --2- 172.21.15.1:0/561185465 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5556ab4db930 0x5556ab4eb850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:58.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.859+0000 7f2487e00700 1 -- 172.21.15.1:0/561185465 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24881a6060 con 0x5556ab4db010 2024-02-02T15:36:58.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.859+0000 7f248d000700 1 --2- 172.21.15.1:0/561185465 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556ab4e67e0 0x5556ab4ebd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:36:58.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.859+0000 7f2487e00700 1 --2- 172.21.15.1:0/561185465 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556ab4db010 0x5556ab4eb310 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f247801b910 tx=0x7f2478019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4872ef78c1c5333e server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:36:58.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.859+0000 7f2476000700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:36:58.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.860+0000 7f2476000700 1 -- 172.21.15.1:0/561185465 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2478021070 con 0x5556ab4db010 2024-02-02T15:36:58.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.860+0000 7f24a2c17b80 1 -- 172.21.15.1:0/561185465 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f24700a1400 con 0x5556ab4db010 2024-02-02T15:36:58.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.860+0000 7f2476000700 1 -- 172.21.15.1:0/561185465 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f247801c710 con 0x5556ab4db010 2024-02-02T15:36:58.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.860+0000 7f2476000700 1 -- 172.21.15.1:0/561185465 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f247801c8d0 con 0x5556ab4db010 2024-02-02T15:36:58.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.860+0000 7f24a2c17b80 10 client.28348 fetch_fsmap learned FSMap version 46 2024-02-02T15:36:58.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.860+0000 7f24a2c17b80 1 -- 172.21.15.1:0/561185465 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) v3 -- 0x5556ab3f2ea0 con 0x5556ab4db010 2024-02-02T15:36:58.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.860+0000 7f2476000700 1 -- 172.21.15.1:0/561185465 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f24700a1400 con 0x5556ab4db010 2024-02-02T15:36:58.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.860+0000 7f2476000700 1 -- 172.21.15.1:0/561185465 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 46) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f2478026420 con 0x5556ab4db010 2024-02-02T15:36:58.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.860+0000 7f24a2c17b80 10 client.28348 fetch_fsmap finished waiting for FSMap version 46 2024-02-02T15:36:58.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.861+0000 7f24a2c17b80 10 client.28348 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:36:58.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.861+0000 7f24a2c17b80 20 client.28348 populate_metadata read hostname 'smithi001' 2024-02-02T15:36:58.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.861+0000 7f24a2c17b80 1 --2- 172.21.15.1:0/561185465 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5556ab550ea0 0x5556ab5532c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:36:58.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.861+0000 7f24a2c17b80 4 client.28348 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2024-02-02T15:36:58.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.861+0000 7f24a2c17b80 1 -- 172.21.15.1:0/561185465 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5556ab3ebf50 con 0x5556ab550ea0 2024-02-02T15:36:58.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.861+0000 7f2487400700 1 --2- 172.21.15.1:0/561185465 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5556ab550ea0 0x5556ab5532c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:36:58.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.861+0000 7f2487400700 1 --2- 172.21.15.1:0/561185465 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5556ab550ea0 0x5556ab5532c0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=411114880c72d0a7 server_cookie=5374b20d56d9bc9b in_seq=0 out_seq=0 2024-02-02T15:36:58.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.861+0000 7f2476000700 10 client.28348 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:36:58.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.862+0000 7f2476000700 1 -- 172.21.15.1:0/561185465 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5556ab3ebf50 con 0x5556ab550ea0 2024-02-02T15:36:58.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.862+0000 7f2476000700 10 client.28348 handle_command_reply: tid=0 2024-02-02T15:36:58.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.862+0000 7f24a2c17b80 1 client.28348 shutdown 2024-02-02T15:36:58.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.863+0000 7f24a2c17b80 1 -- 172.21.15.1:0/561185465 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556ab4db010 msgr2=0x5556ab4eb310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:58.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.863+0000 7f24a2c17b80 1 --2- 172.21.15.1:0/561185465 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556ab4db010 0x5556ab4eb310 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f247801b910 tx=0x7f2478019000 comp rx=0 tx=0).stop 2024-02-02T15:36:58.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.863+0000 7f24a2c17b80 1 -- 172.21.15.1:0/561185465 shutdown_connections 2024-02-02T15:36:58.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.863+0000 7f24a2c17b80 1 --2- 172.21.15.1:0/561185465 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5556ab550ea0 0x5556ab5532c0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:58.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.863+0000 7f24a2c17b80 1 --2- 172.21.15.1:0/561185465 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5556ab4e67e0 0x5556ab4ebd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:58.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.863+0000 7f24a2c17b80 1 --2- 172.21.15.1:0/561185465 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5556ab4db010 0x5556ab4eb310 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:58.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.863+0000 7f24a2c17b80 1 --2- 172.21.15.1:0/561185465 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5556ab4db930 0x5556ab4eb850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:58.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.863+0000 7f2476000700 0 client.28348 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:36:58.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.863+0000 7f24a2c17b80 1 -- 172.21.15.1:0/561185465 >> 172.21.15.1:0/561185465 conn(0x5556ab43adb0 msgr2=0x5556ab4f1900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:58.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.863+0000 7f24a2c17b80 1 -- 172.21.15.1:0/561185465 shutdown_connections 2024-02-02T15:36:58.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.863+0000 7f24a2c17b80 1 -- 172.21.15.1:0/561185465 wait complete. 2024-02-02T15:36:58.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.863+0000 7f24a2c17b80 20 client.28348 trim_cache size 0 max 16384 2024-02-02T15:36:58.867 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:36:58.868 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-02T15:36:58.868 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2024-02-02T15:36:58.868 INFO:teuthology.orchestra.run.smithi001.stdout: "1a9a6929-7be7-4996-94c8-09e64c9b539d": { 2024-02-02T15:36:58.868 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2024-02-02T15:36:58.868 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "1a9a6929-7be7-4996-94c8-09e64c9b539d", 2024-02-02T15:36:58.868 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,force" 2024-02-02T15:36:58.868 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:36:58.868 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:36:58.868 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:36:58.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.864+0000 7f248f000700 1 -- 172.21.15.1:0/2411236433 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f24680a0690 msgr2=0x7f24680a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:58.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.864+0000 7f248f000700 1 --2- 172.21.15.1:0/2411236433 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f24680a0690 0x7f24680a2b50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2478005fd0 tx=0x7f2478009310 comp rx=0 tx=0).stop 2024-02-02T15:36:58.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.864+0000 7f248f000700 1 -- 172.21.15.1:0/2411236433 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2488106020 msgr2=0x7f2488073580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:36:58.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.864+0000 7f248f000700 1 --2- 172.21.15.1:0/2411236433 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2488106020 0x7f2488073580 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f247000e970 tx=0x7f247000ec80 comp rx=0 tx=0).stop 2024-02-02T15:36:58.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.864+0000 7f248f000700 1 -- 172.21.15.1:0/2411236433 shutdown_connections 2024-02-02T15:36:58.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.864+0000 7f248f000700 1 --2- 172.21.15.1:0/2411236433 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f24680a0690 0x7f24680a2b50 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:58.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.864+0000 7f248f000700 1 --2- 172.21.15.1:0/2411236433 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2488073ac0 0x7f24881a4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:58.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.864+0000 7f248f000700 1 --2- 172.21.15.1:0/2411236433 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2488100f00 0x7f2488073040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:58.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.864+0000 7f248f000700 1 --2- 172.21.15.1:0/2411236433 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2488106020 0x7f2488073580 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:36:58.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.864+0000 7f248f000700 1 -- 172.21.15.1:0/2411236433 >> 172.21.15.1:0/2411236433 conn(0x7f24880fc760 msgr2=0x7f24881048d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:36:58.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.864+0000 7f248f000700 1 -- 172.21.15.1:0/2411236433 shutdown_connections 2024-02-02T15:36:58.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:36:58.864+0000 7f248f000700 1 -- 172.21.15.1:0/2411236433 wait complete. 2024-02-02T15:36:58.883 INFO:tasks.cephfs.filesystem:scrub status for tag:1a9a6929-7be7-4996-94c8-09e64c9b539d - {'path': '/', 'tag': '1a9a6929-7be7-4996-94c8-09e64c9b539d', 'options': 'recursive,force'} 2024-02-02T15:36:59.259 INFO:tasks.workunit.client.0.smithi001.stdout: 1 578411 6.05 MB/sec execute 393 sec latency 39.323 ms 2024-02-02T15:36:59.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:59 smithi071 ceph-mon[44707]: pgmap v308: 129 pgs: 129 active+clean; 440 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 117 op/s 2024-02-02T15:36:59.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:59 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:36:59.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:59 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:36:59.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:59 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:36:59.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:59 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:36:59.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:59 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:36:59.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:59 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:36:59.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:59 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:36:59.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:59 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:36:59.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:59 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:36:59.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:59 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:36:59.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:59 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:36:59.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:59 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:36:59.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:59 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:36:59.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:59 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:36:59.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:59 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:36:59.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:59 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:36:59.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:59 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:36:59.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:36:59 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:36:59.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:59 smithi142 ceph-mon[45728]: pgmap v308: 129 pgs: 129 active+clean; 440 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 117 op/s 2024-02-02T15:36:59.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:59 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:36:59.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:59 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:36:59.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:59 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:36:59.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:59 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:36:59.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:59 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:36:59.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:59 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:36:59.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:59 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:36:59.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:59 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:36:59.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:59 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:36:59.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:59 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:36:59.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:59 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:36:59.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:59 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:36:59.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:59 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:36:59.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:59 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:36:59.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:59 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:36:59.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:59 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:36:59.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:59 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:36:59.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:36:59 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:36:59.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:59 smithi001 ceph-mon[35470]: pgmap v308: 129 pgs: 129 active+clean; 440 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 117 op/s 2024-02-02T15:36:59.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:59 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:36:59.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:59 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:36:59.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:59 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:36:59.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:59 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:36:59.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:59 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:36:59.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:59 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:36:59.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:59 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:36:59.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:59 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:36:59.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:59 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:36:59.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:59 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:36:59.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:59 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:36:59.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:59 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:36:59.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:59 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:36:59.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:59 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:36:59.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:59 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:36:59.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:59 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:36:59.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:59 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:36:59.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:36:59 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:37:00.259 INFO:tasks.workunit.client.0.smithi001.stdout: 1 579580 6.05 MB/sec execute 394 sec latency 16.731 ms 2024-02-02T15:37:00.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:00 smithi071 ceph-mon[44707]: scrub queued for path: / 2024-02-02T15:37:00.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:00 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:37:00.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:00 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:37:00.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:00 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:37:00.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:00 smithi142 ceph-mon[45728]: scrub queued for path: / 2024-02-02T15:37:00.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:00 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:37:00.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:00 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:37:00.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:00 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:37:00.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:00 smithi001 ceph-mon[35470]: scrub queued for path: / 2024-02-02T15:37:00.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:00 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:37:00.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:00 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:37:00.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:00 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:37:01.259 INFO:tasks.workunit.client.0.smithi001.stdout: 1 580132 6.04 MB/sec execute 395 sec latency 48.023 ms 2024-02-02T15:37:01.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:01 smithi071 ceph-mon[44707]: pgmap v309: 129 pgs: 129 active+clean; 440 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 117 op/s 2024-02-02T15:37:01.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:01 smithi142 ceph-mon[45728]: pgmap v309: 129 pgs: 129 active+clean; 440 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 117 op/s 2024-02-02T15:37:01.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:01 smithi001 ceph-mon[35470]: pgmap v309: 129 pgs: 129 active+clean; 440 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 117 op/s 2024-02-02T15:37:02.259 INFO:tasks.workunit.client.0.smithi001.stdout: 1 580669 6.04 MB/sec execute 396 sec latency 344.163 ms 2024-02-02T15:37:03.259 INFO:tasks.workunit.client.0.smithi001.stdout: 1 581273 6.02 MB/sec execute 397 sec latency 725.523 ms 2024-02-02T15:37:03.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:03 smithi142 ceph-mon[45728]: pgmap v310: 129 pgs: 129 active+clean; 447 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 236 op/s 2024-02-02T15:37:03.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:03 smithi142 ceph-mon[45728]: osdmap e96: 12 total, 12 up, 12 in 2024-02-02T15:37:03.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:03 smithi001 ceph-mon[35470]: pgmap v310: 129 pgs: 129 active+clean; 447 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 236 op/s 2024-02-02T15:37:03.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:03 smithi001 ceph-mon[35470]: osdmap e96: 12 total, 12 up, 12 in 2024-02-02T15:37:03.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:03 smithi071 ceph-mon[44707]: pgmap v310: 129 pgs: 129 active+clean; 447 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 236 op/s 2024-02-02T15:37:03.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:03 smithi071 ceph-mon[44707]: osdmap e96: 12 total, 12 up, 12 in 2024-02-02T15:37:04.259 INFO:tasks.workunit.client.0.smithi001.stdout: 1 582805 6.04 MB/sec execute 398 sec latency 24.449 ms 2024-02-02T15:37:05.259 INFO:tasks.workunit.client.0.smithi001.stdout: 1 583911 6.04 MB/sec execute 399 sec latency 21.206 ms 2024-02-02T15:37:05.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:05 smithi142 ceph-mon[45728]: pgmap v312: 129 pgs: 129 active+clean; 447 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 217 op/s 2024-02-02T15:37:05.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:05 smithi001 ceph-mon[35470]: pgmap v312: 129 pgs: 129 active+clean; 447 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 217 op/s 2024-02-02T15:37:05.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:05 smithi071 ceph-mon[44707]: pgmap v312: 129 pgs: 129 active+clean; 447 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 217 op/s 2024-02-02T15:37:06.259 INFO:tasks.workunit.client.0.smithi001.stdout: 1 584222 6.02 MB/sec execute 400 sec latency 562.818 ms 2024-02-02T15:37:07.260 INFO:tasks.workunit.client.0.smithi001.stdout: 1 584222 6.01 MB/sec execute 401 sec latency 1562.971 ms 2024-02-02T15:37:07.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:07 smithi142 ceph-mon[45728]: pgmap v313: 129 pgs: 129 active+clean; 451 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 2.3 MiB/s wr, 240 op/s 2024-02-02T15:37:07.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:07 smithi001 ceph-mon[35470]: pgmap v313: 129 pgs: 129 active+clean; 451 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 2.3 MiB/s wr, 240 op/s 2024-02-02T15:37:07.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:07 smithi071 ceph-mon[44707]: pgmap v313: 129 pgs: 129 active+clean; 451 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 2.3 MiB/s wr, 240 op/s 2024-02-02T15:37:08.259 INFO:tasks.workunit.client.0.smithi001.stdout: 1 585449 6.01 MB/sec execute 402 sec latency 1947.084 ms 2024-02-02T15:37:08.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:08 smithi142 ceph-mon[45728]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:37:08.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:08 smithi001 ceph-mon[35470]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:37:08.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:08 smithi071 ceph-mon[44707]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:37:09.260 INFO:tasks.workunit.client.0.smithi001.stdout: 1 586624 6.01 MB/sec execute 403 sec latency 28.379 ms 2024-02-02T15:37:09.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:09 smithi001 ceph-mon[35470]: pgmap v314: 129 pgs: 129 active+clean; 451 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 2.3 MiB/s wr, 240 op/s 2024-02-02T15:37:09.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:09 smithi071 ceph-mon[44707]: pgmap v314: 129 pgs: 129 active+clean; 451 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 2.3 MiB/s wr, 240 op/s 2024-02-02T15:37:09.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:09 smithi142 ceph-mon[45728]: pgmap v314: 129 pgs: 129 active+clean; 451 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 2.3 MiB/s wr, 240 op/s 2024-02-02T15:37:10.260 INFO:tasks.workunit.client.0.smithi001.stdout: 1 587763 6.01 MB/sec execute 404 sec latency 65.056 ms 2024-02-02T15:37:11.260 INFO:tasks.workunit.client.0.smithi001.stdout: 1 588473 6.00 MB/sec execute 405 sec latency 732.402 ms 2024-02-02T15:37:11.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:11 smithi001 ceph-mon[35470]: pgmap v315: 129 pgs: 129 active+clean; 451 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 2.3 MiB/s wr, 240 op/s 2024-02-02T15:37:11.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:11 smithi001 ceph-mon[35470]: osdmap e97: 12 total, 12 up, 12 in 2024-02-02T15:37:11.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:11 smithi071 ceph-mon[44707]: pgmap v315: 129 pgs: 129 active+clean; 451 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 2.3 MiB/s wr, 240 op/s 2024-02-02T15:37:11.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:11 smithi071 ceph-mon[44707]: osdmap e97: 12 total, 12 up, 12 in 2024-02-02T15:37:11.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:11 smithi142 ceph-mon[45728]: pgmap v315: 129 pgs: 129 active+clean; 451 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 2.3 MiB/s wr, 240 op/s 2024-02-02T15:37:11.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:11 smithi142 ceph-mon[45728]: osdmap e97: 12 total, 12 up, 12 in 2024-02-02T15:37:12.260 INFO:tasks.workunit.client.0.smithi001.stdout: 1 589982 6.02 MB/sec execute 406 sec latency 13.742 ms 2024-02-02T15:37:12.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:12 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:37:12.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:12 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:37:12.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:12 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:37:12.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:12 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:37:12.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:12 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:37:12.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:12 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:37:12.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:12 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:37:12.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:12 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:37:12.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:12 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:37:12.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:12 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:37:12.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:12 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:37:12.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:12 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:37:12.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:12 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:37:12.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:12 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:37:12.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:12 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:37:13.260 INFO:tasks.workunit.client.0.smithi001.stdout: 1 591073 6.01 MB/sec execute 407 sec latency 16.738 ms 2024-02-02T15:37:13.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:13 smithi001 ceph-mon[35470]: pgmap v317: 129 pgs: 129 active+clean; 458 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.0 MiB/s wr, 167 op/s 2024-02-02T15:37:13.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:13 smithi071 ceph-mon[44707]: pgmap v317: 129 pgs: 129 active+clean; 458 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.0 MiB/s wr, 167 op/s 2024-02-02T15:37:13.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:13 smithi142 ceph-mon[45728]: pgmap v317: 129 pgs: 129 active+clean; 458 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.0 MiB/s wr, 167 op/s 2024-02-02T15:37:14.260 INFO:tasks.workunit.client.0.smithi001.stdout: 1 591312 6.00 MB/sec execute 408 sec latency 375.806 ms 2024-02-02T15:37:15.260 INFO:tasks.workunit.client.0.smithi001.stdout: 1 591312 5.98 MB/sec execute 409 sec latency 1375.956 ms 2024-02-02T15:37:15.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:15 smithi001 ceph-mon[35470]: pgmap v318: 129 pgs: 129 active+clean; 458 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.0 MiB/s wr, 165 op/s 2024-02-02T15:37:15.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:15 smithi071 ceph-mon[44707]: pgmap v318: 129 pgs: 129 active+clean; 458 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.0 MiB/s wr, 165 op/s 2024-02-02T15:37:15.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:15 smithi142 ceph-mon[45728]: pgmap v318: 129 pgs: 129 active+clean; 458 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.0 MiB/s wr, 165 op/s 2024-02-02T15:37:16.260 INFO:tasks.workunit.client.0.smithi001.stdout: 1 591312 5.97 MB/sec execute 410 sec latency 2376.117 ms 2024-02-02T15:37:17.261 INFO:tasks.workunit.client.0.smithi001.stdout: 1 591312 5.96 MB/sec execute 411 sec latency 3376.201 ms 2024-02-02T15:37:17.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:17 smithi001 ceph-mon[35470]: pgmap v319: 129 pgs: 129 active+clean; 466 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 1.9 MiB/s wr, 148 op/s 2024-02-02T15:37:17.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:17 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:37:17.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:17 smithi071 ceph-mon[44707]: pgmap v319: 129 pgs: 129 active+clean; 466 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 1.9 MiB/s wr, 148 op/s 2024-02-02T15:37:17.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:17 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:37:17.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:17 smithi142 ceph-mon[45728]: pgmap v319: 129 pgs: 129 active+clean; 466 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 1.9 MiB/s wr, 148 op/s 2024-02-02T15:37:17.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:17 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:37:18.260 INFO:tasks.workunit.client.0.smithi001.stdout: 1 592217 5.95 MB/sec execute 412 sec latency 3761.461 ms 2024-02-02T15:37:18.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:18 smithi071 ceph-mon[44707]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:37:18.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:18 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:37:18.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:18 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:37:18.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:18 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:37:18.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:18 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:37:18.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:18 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:37:18.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:18 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:37:18.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:18 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:37:18.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:18 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:37:18.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:18 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:37:18.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:18 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:37:18.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:18 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:37:18.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:18 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:37:18.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:18 smithi142 ceph-mon[45728]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:37:18.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:18 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:37:18.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:18 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:37:18.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:18 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:37:18.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:18 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:37:18.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:18 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:37:18.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:18 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:37:18.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:18 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:37:18.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:18 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:37:18.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:18 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:37:18.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:18 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:37:18.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:18 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:37:18.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:18 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:37:18.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:18 smithi001 ceph-mon[35470]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:37:18.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:18 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:37:18.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:18 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:37:18.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:18 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:37:18.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:18 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:37:18.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:18 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:37:18.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:18 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:37:18.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:18 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:37:18.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:18 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:37:18.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:18 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:37:18.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:18 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:37:18.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:18 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:37:18.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:18 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:37:19.260 INFO:tasks.workunit.client.0.smithi001.stdout: 1 593554 5.96 MB/sec execute 413 sec latency 25.186 ms 2024-02-02T15:37:19.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:19 smithi071 ceph-mon[44707]: pgmap v320: 129 pgs: 129 active+clean; 466 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 1.9 MiB/s wr, 148 op/s 2024-02-02T15:37:19.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:19 smithi071 ceph-mon[44707]: Adjusting osd_memory_target on smithi071 to 182.4M 2024-02-02T15:37:19.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:19 smithi071 ceph-mon[44707]: Unable to set osd_memory_target on smithi071 to 191358361: error parsing value: Value '191358361' is below minimum 939524096 2024-02-02T15:37:19.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:19 smithi071 ceph-mon[44707]: Adjusting osd_memory_target on smithi142 to 182.4M 2024-02-02T15:37:19.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:19 smithi071 ceph-mon[44707]: Unable to set osd_memory_target on smithi142 to 191358361: error parsing value: Value '191358361' is below minimum 939524096 2024-02-02T15:37:19.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:19 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:37:19.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:19 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:37:19.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:19 smithi142 ceph-mon[45728]: pgmap v320: 129 pgs: 129 active+clean; 466 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 1.9 MiB/s wr, 148 op/s 2024-02-02T15:37:19.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:19 smithi142 ceph-mon[45728]: Adjusting osd_memory_target on smithi071 to 182.4M 2024-02-02T15:37:19.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:19 smithi142 ceph-mon[45728]: Unable to set osd_memory_target on smithi071 to 191358361: error parsing value: Value '191358361' is below minimum 939524096 2024-02-02T15:37:19.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:19 smithi142 ceph-mon[45728]: Adjusting osd_memory_target on smithi142 to 182.4M 2024-02-02T15:37:19.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:19 smithi142 ceph-mon[45728]: Unable to set osd_memory_target on smithi142 to 191358361: error parsing value: Value '191358361' is below minimum 939524096 2024-02-02T15:37:19.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:19 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:37:19.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:19 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:37:19.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:19 smithi001 ceph-mon[35470]: pgmap v320: 129 pgs: 129 active+clean; 466 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 1.9 MiB/s wr, 148 op/s 2024-02-02T15:37:19.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:19 smithi001 ceph-mon[35470]: Adjusting osd_memory_target on smithi071 to 182.4M 2024-02-02T15:37:19.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:19 smithi001 ceph-mon[35470]: Unable to set osd_memory_target on smithi071 to 191358361: error parsing value: Value '191358361' is below minimum 939524096 2024-02-02T15:37:19.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:19 smithi001 ceph-mon[35470]: Adjusting osd_memory_target on smithi142 to 182.4M 2024-02-02T15:37:19.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:19 smithi001 ceph-mon[35470]: Unable to set osd_memory_target on smithi142 to 191358361: error parsing value: Value '191358361' is below minimum 939524096 2024-02-02T15:37:19.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:19 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:37:19.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:19 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:37:20.260 INFO:tasks.workunit.client.0.smithi001.stdout: 1 594694 5.96 MB/sec execute 414 sec latency 19.361 ms 2024-02-02T15:37:20.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:20 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:37:20.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:20 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:37:20.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:20 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:37:20.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:20 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:37:20.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:20 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:37:20.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:20 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:37:20.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:20 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:37:20.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:20 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:37:20.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:20 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:37:20.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:20 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:37:20.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:20 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:37:20.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:20 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:37:20.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:20 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:37:20.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:20 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:37:20.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:20 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:37:20.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:20 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:37:20.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:20 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:37:20.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:20 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-02-02T15:37:20.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:20 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:37:20.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:20 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:37:20.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:20 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:37:21.261 INFO:tasks.workunit.client.0.smithi001.stdout: 1 594856 5.94 MB/sec execute 415 sec latency 873.128 ms 2024-02-02T15:37:21.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:21 smithi071 ceph-mon[44707]: Adjusting osd_memory_target on smithi001 to 223.0M 2024-02-02T15:37:21.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:21 smithi071 ceph-mon[44707]: Unable to set osd_memory_target on smithi001 to 233866752: error parsing value: Value '233866752' is below minimum 939524096 2024-02-02T15:37:21.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:21 smithi071 ceph-mon[44707]: pgmap v321: 129 pgs: 129 active+clean; 466 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 1.9 MiB/s wr, 148 op/s 2024-02-02T15:37:21.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:21 smithi142 ceph-mon[45728]: Adjusting osd_memory_target on smithi001 to 223.0M 2024-02-02T15:37:21.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:21 smithi142 ceph-mon[45728]: Unable to set osd_memory_target on smithi001 to 233866752: error parsing value: Value '233866752' is below minimum 939524096 2024-02-02T15:37:21.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:21 smithi142 ceph-mon[45728]: pgmap v321: 129 pgs: 129 active+clean; 466 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 1.9 MiB/s wr, 148 op/s 2024-02-02T15:37:21.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:21 smithi001 ceph-mon[35470]: Adjusting osd_memory_target on smithi001 to 223.0M 2024-02-02T15:37:21.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:21 smithi001 ceph-mon[35470]: Unable to set osd_memory_target on smithi001 to 233866752: error parsing value: Value '233866752' is below minimum 939524096 2024-02-02T15:37:21.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:21 smithi001 ceph-mon[35470]: pgmap v321: 129 pgs: 129 active+clean; 466 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 1.9 MiB/s wr, 148 op/s 2024-02-02T15:37:22.261 INFO:tasks.workunit.client.0.smithi001.stdout: 1 594856 5.93 MB/sec execute 416 sec latency 1873.201 ms 2024-02-02T15:37:23.261 INFO:tasks.workunit.client.0.smithi001.stdout: 1 596078 5.93 MB/sec execute 417 sec latency 2263.116 ms 2024-02-02T15:37:23.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:23 smithi071 ceph-mon[44707]: pgmap v322: 129 pgs: 129 active+clean; 468 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 1.7 MiB/s wr, 146 op/s 2024-02-02T15:37:23.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:23 smithi071 ceph-mon[44707]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:37:23.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:23 smithi142 ceph-mon[45728]: pgmap v322: 129 pgs: 129 active+clean; 468 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 1.7 MiB/s wr, 146 op/s 2024-02-02T15:37:23.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:23 smithi142 ceph-mon[45728]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:37:23.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:23 smithi001 ceph-mon[35470]: pgmap v322: 129 pgs: 129 active+clean; 468 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 1.7 MiB/s wr, 146 op/s 2024-02-02T15:37:23.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:23 smithi001 ceph-mon[35470]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:37:24.261 INFO:tasks.workunit.client.0.smithi001.stdout: 1 597247 5.93 MB/sec execute 418 sec latency 14.438 ms 2024-02-02T15:37:25.261 INFO:tasks.workunit.client.0.smithi001.stdout: 1 598403 5.93 MB/sec execute 419 sec latency 78.085 ms 2024-02-02T15:37:25.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:25 smithi071 ceph-mon[44707]: pgmap v323: 129 pgs: 129 active+clean; 468 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 1.5 MiB/s wr, 133 op/s 2024-02-02T15:37:25.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:25 smithi142 ceph-mon[45728]: pgmap v323: 129 pgs: 129 active+clean; 468 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 1.5 MiB/s wr, 133 op/s 2024-02-02T15:37:25.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:25 smithi001 ceph-mon[35470]: pgmap v323: 129 pgs: 129 active+clean; 468 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 1.5 MiB/s wr, 133 op/s 2024-02-02T15:37:26.261 INFO:tasks.workunit.client.0.smithi001.stdout: 1 598403 5.92 MB/sec execute 420 sec latency 1078.200 ms 2024-02-02T15:37:27.261 INFO:tasks.workunit.client.0.smithi001.stdout: 1 598403 5.90 MB/sec execute 421 sec latency 2078.344 ms 2024-02-02T15:37:27.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:27 smithi071 ceph-mon[44707]: pgmap v324: 129 pgs: 129 active+clean; 472 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 2.3 MiB/s wr, 191 op/s 2024-02-02T15:37:27.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:27 smithi142 ceph-mon[45728]: pgmap v324: 129 pgs: 129 active+clean; 472 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 2.3 MiB/s wr, 191 op/s 2024-02-02T15:37:27.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:27 smithi001 ceph-mon[35470]: pgmap v324: 129 pgs: 129 active+clean; 472 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 2.3 MiB/s wr, 191 op/s 2024-02-02T15:37:28.261 INFO:tasks.workunit.client.0.smithi001.stdout: 1 599612 5.91 MB/sec execute 422 sec latency 2460.737 ms 2024-02-02T15:37:28.884 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-02T15:37:29.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.096+0000 7fd867c00700 1 -- 172.21.15.1:0/3164921117 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8601019b0 msgr2=0x7fd8601059f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:37:29.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.096+0000 7fd867c00700 1 --2- 172.21.15.1:0/3164921117 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8601019b0 0x7fd8601059f0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fd848009990 tx=0x7fd848009ca0 comp rx=0 tx=0).stop 2024-02-02T15:37:29.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.096+0000 7fd867c00700 1 -- 172.21.15.1:0/3164921117 shutdown_connections 2024-02-02T15:37:29.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.096+0000 7fd867c00700 1 --2- 172.21.15.1:0/3164921117 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd860106120 0x7fd86010dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:29.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.096+0000 7fd867c00700 1 --2- 172.21.15.1:0/3164921117 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd860101000 0x7fd8601013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:29.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.096+0000 7fd867c00700 1 --2- 172.21.15.1:0/3164921117 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd8601019b0 0x7fd8601059f0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:29.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.096+0000 7fd867c00700 1 -- 172.21.15.1:0/3164921117 >> 172.21.15.1:0/3164921117 conn(0x7fd8600fc860 msgr2=0x7fd8600fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:37:29.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.096+0000 7fd867c00700 1 -- 172.21.15.1:0/3164921117 shutdown_connections 2024-02-02T15:37:29.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.096+0000 7fd867c00700 1 -- 172.21.15.1:0/3164921117 wait complete. 2024-02-02T15:37:29.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.098+0000 7fd867c00700 1 Processor -- start 2024-02-02T15:37:29.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.098+0000 7fd867c00700 1 -- start start 2024-02-02T15:37:29.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.098+0000 7fd867c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd860101000 0x7fd860199420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:29.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.098+0000 7fd867c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd860106120 0x7fd860199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:29.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.098+0000 7fd867c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd86019db40 0x7fd86019dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:29.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.098+0000 7fd867c00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd86019e5c0 con 0x7fd860106120 2024-02-02T15:37:29.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.098+0000 7fd867c00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd86019e730 con 0x7fd86019db40 2024-02-02T15:37:29.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.098+0000 7fd867c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd86019e8f0 con 0x7fd860101000 2024-02-02T15:37:29.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.099+0000 7fd865200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd860101000 0x7fd860199420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:29.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.099+0000 7fd865200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd860101000 0x7fd860199420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:41910/0 (socket says 172.21.15.1:41910) 2024-02-02T15:37:29.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.099+0000 7fd865200700 1 -- 172.21.15.1:0/2448170537 learned_addr learned my addr 172.21.15.1:0/2448170537 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:37:29.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.099+0000 7fd865c00700 1 --2- 172.21.15.1:0/2448170537 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd86019db40 0x7fd86019dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:29.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.099+0000 7fd85fe00700 1 --2- 172.21.15.1:0/2448170537 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd860106120 0x7fd860199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:29.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.099+0000 7fd865200700 1 -- 172.21.15.1:0/2448170537 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd86019db40 msgr2=0x7fd86019dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:37:29.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.099+0000 7fd865200700 1 --2- 172.21.15.1:0/2448170537 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd86019db40 0x7fd86019dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:29.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.099+0000 7fd865200700 1 -- 172.21.15.1:0/2448170537 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd860106120 msgr2=0x7fd860199960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:37:29.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.099+0000 7fd865200700 1 --2- 172.21.15.1:0/2448170537 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd860106120 0x7fd860199960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:29.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.099+0000 7fd865200700 1 -- 172.21.15.1:0/2448170537 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8480095f0 con 0x7fd860101000 2024-02-02T15:37:29.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.099+0000 7fd865200700 1 --2- 172.21.15.1:0/2448170537 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd860101000 0x7fd860199420 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fd85000e9a0 tx=0x7fd85000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:37:29.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.099+0000 7fd85d600700 1 -- 172.21.15.1:0/2448170537 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd850009790 con 0x7fd860101000 2024-02-02T15:37:29.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.099+0000 7fd85d600700 1 -- 172.21.15.1:0/2448170537 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd850010430 con 0x7fd860101000 2024-02-02T15:37:29.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.099+0000 7fd85d600700 1 -- 172.21.15.1:0/2448170537 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd850018680 con 0x7fd860101000 2024-02-02T15:37:29.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.099+0000 7fd867c00700 1 -- 172.21.15.1:0/2448170537 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8601a5e30 con 0x7fd860101000 2024-02-02T15:37:29.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.100+0000 7fd867c00700 1 -- 172.21.15.1:0/2448170537 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8601a62b0 con 0x7fd860101000 2024-02-02T15:37:29.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.103+0000 7fd85d600700 1 -- 172.21.15.1:0/2448170537 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7fd8500105a0 con 0x7fd860101000 2024-02-02T15:37:29.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.104+0000 7fd85d600700 1 --2- 172.21.15.1:0/2448170537 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fd8400a0710 0x7fd8400a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:29.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.104+0000 7fd87b6a4b80 1 Processor -- start 2024-02-02T15:37:29.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.104+0000 7fd85d600700 1 -- 172.21.15.1:0/2448170537 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fd850014070 con 0x7fd860101000 2024-02-02T15:37:29.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.104+0000 7fd87b6a4b80 1 -- start start 2024-02-02T15:37:29.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.104+0000 7fd85fe00700 1 --2- 172.21.15.1:0/2448170537 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fd8400a0710 0x7fd8400a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:29.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.105+0000 7fd87b6a4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558cb9631010 0x558cb96313f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:29.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.105+0000 7fd87b6a4b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558cb9631930 0x558cb963c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:29.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.105+0000 7fd87b6a4b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558cb963c7e0 0x558cb963ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:29.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.105+0000 7fd87b6a4b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cb95a54c0 con 0x558cb9631930 2024-02-02T15:37:29.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.105+0000 7fd87b6a4b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cb9589aa0 con 0x558cb963c7e0 2024-02-02T15:37:29.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.105+0000 7fd87b6a4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cb9587460 con 0x558cb9631010 2024-02-02T15:37:29.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.105+0000 7fd865200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558cb9631010 0x558cb96313f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:29.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.105+0000 7fd865200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558cb9631010 0x558cb96313f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:41922/0 (socket says 172.21.15.1:41922) 2024-02-02T15:37:29.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.105+0000 7fd865200700 1 -- 172.21.15.1:0/3506592226 learned_addr learned my addr 172.21.15.1:0/3506592226 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:37:29.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.105+0000 7fd865c00700 1 --2- 172.21.15.1:0/3506592226 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558cb963c7e0 0x558cb963ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:29.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.105+0000 7fd85fe00700 1 --2- 172.21.15.1:0/3506592226 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558cb9631930 0x558cb963c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:29.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.105+0000 7fd865200700 1 -- 172.21.15.1:0/3506592226 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558cb963c7e0 msgr2=0x558cb963ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:37:29.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.105+0000 7fd865200700 1 --2- 172.21.15.1:0/3506592226 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558cb963c7e0 0x558cb963ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:29.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.105+0000 7fd865200700 1 -- 172.21.15.1:0/3506592226 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558cb9631930 msgr2=0x558cb963c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:37:29.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.105+0000 7fd865200700 1 --2- 172.21.15.1:0/3506592226 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558cb9631930 0x558cb963c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:29.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.105+0000 7fd865200700 1 -- 172.21.15.1:0/3506592226 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558cb94043b0 con 0x558cb9631010 2024-02-02T15:37:29.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.105+0000 7fd85fe00700 1 --2- 172.21.15.1:0/3506592226 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558cb9631930 0x558cb963c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:37:29.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.106+0000 7fd85fe00700 1 --2- 172.21.15.1:0/2448170537 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fd8400a0710 0x7fd8400a2bd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd86019a7b0 tx=0x7fd848005040 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:37:29.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.106+0000 7fd865200700 1 --2- 172.21.15.1:0/3506592226 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558cb9631010 0x558cb96313f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fd850099370 tx=0x7fd850098dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=defdf5a36a1f20fc server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:37:29.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.106+0000 7fd856000700 1 -- 172.21.15.1:0/3506592226 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8500acea0 con 0x558cb9631010 2024-02-02T15:37:29.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.106+0000 7fd856000700 1 -- 172.21.15.1:0/3506592226 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd8500993e0 con 0x558cb9631010 2024-02-02T15:37:29.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.106+0000 7fd856000700 1 -- 172.21.15.1:0/3506592226 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd850082e40 con 0x558cb9631010 2024-02-02T15:37:29.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.106+0000 7fd87b6a4b80 1 -- 172.21.15.1:0/3506592226 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558cb9631010 msgr2=0x558cb96313f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:37:29.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.106+0000 7fd87b6a4b80 1 --2- 172.21.15.1:0/3506592226 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558cb9631010 0x558cb96313f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fd850099370 tx=0x7fd850098dc0 comp rx=0 tx=0).stop 2024-02-02T15:37:29.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.107+0000 7fd87b6a4b80 1 -- 172.21.15.1:0/3506592226 shutdown_connections 2024-02-02T15:37:29.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.107+0000 7fd87b6a4b80 1 --2- 172.21.15.1:0/3506592226 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558cb963c7e0 0x558cb963ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:29.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.107+0000 7fd87b6a4b80 1 --2- 172.21.15.1:0/3506592226 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558cb9631010 0x558cb96313f0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:29.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.107+0000 7fd87b6a4b80 1 --2- 172.21.15.1:0/3506592226 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558cb9631930 0x558cb963c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:29.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.107+0000 7fd87b6a4b80 1 -- 172.21.15.1:0/3506592226 >> 172.21.15.1:0/3506592226 conn(0x558cb9590db0 msgr2=0x558cb962d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:37:29.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.107+0000 7fd87b6a4b80 1 -- 172.21.15.1:0/3506592226 shutdown_connections 2024-02-02T15:37:29.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.107+0000 7fd87b6a4b80 1 -- 172.21.15.1:0/3506592226 wait complete. 2024-02-02T15:37:29.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.107+0000 7fd87b6a4b80 1 -- start start 2024-02-02T15:37:29.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.108+0000 7fd87b6a4b80 1 Processor -- start 2024-02-02T15:37:29.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.108+0000 7fd87b6a4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558cb9631010 0x558cb963c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:29.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.108+0000 7fd87b6a4b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558cb9631930 0x558cb9639450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:29.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.108+0000 7fd87b6a4b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558cb963c7e0 0x558cb9639ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:29.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.108+0000 7fd87b6a4b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cb9582d50 con 0x558cb9631930 2024-02-02T15:37:29.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.108+0000 7fd87b6a4b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cb951ed80 con 0x558cb963c7e0 2024-02-02T15:37:29.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.108+0000 7fd87b6a4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cb951ea20 con 0x558cb9631010 2024-02-02T15:37:29.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.108+0000 7fd87b6a4b80 10 client.0 _ll_register_callbacks cb 0x558cb9520290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:37:29.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.108+0000 7fd865200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558cb9631010 0x558cb963c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:29.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.108+0000 7fd865200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558cb9631010 0x558cb963c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:41924/0 (socket says 172.21.15.1:41924) 2024-02-02T15:37:29.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.108+0000 7fd865200700 1 -- 172.21.15.1:0/1478512845 learned_addr learned my addr 172.21.15.1:0/1478512845 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:37:29.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.109+0000 7fd85fe00700 1 --2- 172.21.15.1:0/1478512845 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558cb9631930 0x558cb9639450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:29.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.109+0000 7fd865c00700 1 --2- 172.21.15.1:0/1478512845 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558cb963c7e0 0x558cb9639ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:29.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.109+0000 7fd865200700 1 -- 172.21.15.1:0/1478512845 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558cb963c7e0 msgr2=0x558cb9639ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:37:29.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.109+0000 7fd865200700 1 --2- 172.21.15.1:0/1478512845 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558cb963c7e0 0x558cb9639ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:29.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.109+0000 7fd865200700 1 -- 172.21.15.1:0/1478512845 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558cb9631930 msgr2=0x558cb9639450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:37:29.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.109+0000 7fd865200700 1 --2- 172.21.15.1:0/1478512845 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558cb9631930 0x558cb9639450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:29.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.109+0000 7fd865200700 1 -- 172.21.15.1:0/1478512845 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8601a62b0 con 0x558cb9631010 2024-02-02T15:37:29.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.109+0000 7fd865200700 1 --2- 172.21.15.1:0/1478512845 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558cb9631010 0x558cb963c1d0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fd85000c2d0 tx=0x7fd85000c4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5fb8ce7c3d16ce46 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:37:29.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.109+0000 7fd84ea00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:37:29.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.109+0000 7fd84ea00700 1 -- 172.21.15.1:0/1478512845 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8500acea0 con 0x558cb9631010 2024-02-02T15:37:29.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.109+0000 7fd84ea00700 1 -- 172.21.15.1:0/1478512845 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fd850082df0 con 0x558cb9631010 2024-02-02T15:37:29.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.109+0000 7fd87b6a4b80 1 -- 172.21.15.1:0/1478512845 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558cb96250f0 con 0x558cb9631010 2024-02-02T15:37:29.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.110+0000 7fd84ea00700 1 -- 172.21.15.1:0/1478512845 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd85009d290 con 0x558cb9631010 2024-02-02T15:37:29.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.110+0000 7fd87b6a4b80 10 client.28360 fetch_fsmap learned FSMap version 49 2024-02-02T15:37:29.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.110+0000 7fd87b6a4b80 1 -- 172.21.15.1:0/1478512845 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x558cb93b0b60 con 0x558cb9631010 2024-02-02T15:37:29.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.110+0000 7fd84ea00700 1 -- 172.21.15.1:0/1478512845 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558cb96250f0 con 0x558cb9631010 2024-02-02T15:37:29.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.110+0000 7fd84ea00700 1 -- 172.21.15.1:0/1478512845 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 49) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fd8500a10c0 con 0x558cb9631010 2024-02-02T15:37:29.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.110+0000 7fd87b6a4b80 10 client.28360 fetch_fsmap finished waiting for FSMap version 49 2024-02-02T15:37:29.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.110+0000 7fd87b6a4b80 10 client.28360 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:37:29.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.110+0000 7fd87b6a4b80 20 client.28360 populate_metadata read hostname 'smithi001' 2024-02-02T15:37:29.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.110+0000 7fd87b6a4b80 1 --2- 172.21.15.1:0/1478512845 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x558cb964bdd0 0x558cb964e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:29.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.110+0000 7fd87b6a4b80 4 client.28360 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:37:29.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.110+0000 7fd87b6a4b80 1 -- 172.21.15.1:0/1478512845 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558cb957bdd0 con 0x558cb964bdd0 2024-02-02T15:37:29.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.110+0000 7fd85fe00700 1 --2- 172.21.15.1:0/1478512845 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x558cb964bdd0 0x558cb964e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:29.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.111+0000 7fd85fe00700 1 --2- 172.21.15.1:0/1478512845 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x558cb964bdd0 0x558cb964e1c0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1e0702ba44a68e7 server_cookie=bdb12032f8b03782 in_seq=0 out_seq=0 2024-02-02T15:37:29.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.111+0000 7fd84ea00700 10 client.28360 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:37:29.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.112+0000 7fd84ea00700 1 -- 172.21.15.1:0/1478512845 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x558cb957bdd0 con 0x558cb964bdd0 2024-02-02T15:37:29.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.112+0000 7fd84ea00700 10 client.28360 handle_command_reply: tid=0 2024-02-02T15:37:29.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.112+0000 7fd87b6a4b80 1 client.28360 shutdown 2024-02-02T15:37:29.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.112+0000 7fd87b6a4b80 1 -- 172.21.15.1:0/1478512845 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558cb9631010 msgr2=0x558cb963c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:37:29.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.112+0000 7fd87b6a4b80 1 --2- 172.21.15.1:0/1478512845 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558cb9631010 0x558cb963c1d0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fd85000c2d0 tx=0x7fd85000c4b0 comp rx=0 tx=0).stop 2024-02-02T15:37:29.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.112+0000 7fd87b6a4b80 1 -- 172.21.15.1:0/1478512845 shutdown_connections 2024-02-02T15:37:29.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.112+0000 7fd87b6a4b80 1 --2- 172.21.15.1:0/1478512845 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x558cb964bdd0 0x558cb964e1c0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:29.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.112+0000 7fd87b6a4b80 1 --2- 172.21.15.1:0/1478512845 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558cb963c7e0 0x558cb9639ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:29.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.112+0000 7fd87b6a4b80 1 --2- 172.21.15.1:0/1478512845 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558cb9631010 0x558cb963c1d0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:29.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.112+0000 7fd87b6a4b80 1 --2- 172.21.15.1:0/1478512845 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558cb9631930 0x558cb9639450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:29.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.112+0000 7fd87b6a4b80 1 -- 172.21.15.1:0/1478512845 >> 172.21.15.1:0/1478512845 conn(0x558cb9590db0 msgr2=0x558cb962cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:37:29.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.112+0000 7fd84ea00700 0 client.28360 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:37:29.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.112+0000 7fd87b6a4b80 1 -- 172.21.15.1:0/1478512845 shutdown_connections 2024-02-02T15:37:29.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.112+0000 7fd87b6a4b80 1 -- 172.21.15.1:0/1478512845 wait complete. 2024-02-02T15:37:29.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.113+0000 7fd87b6a4b80 20 client.28360 trim_cache size 0 max 16384 2024-02-02T15:37:29.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.130+0000 7fd87b6a4b80 1 Processor -- start 2024-02-02T15:37:29.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.131+0000 7fd87b6a4b80 1 -- start start 2024-02-02T15:37:29.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.131+0000 7fd87b6a4b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558cb9631010 0x558cb963bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:29.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.131+0000 7fd87b6a4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558cb9631930 0x558cb9634740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:29.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.131+0000 7fd87b6a4b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558cb963c7e0 0x558cb9634c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:29.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.131+0000 7fd87b6a4b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cb94066a0 con 0x558cb963c7e0 2024-02-02T15:37:29.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.131+0000 7fd87b6a4b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cb9635350 con 0x558cb9631010 2024-02-02T15:37:29.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.131+0000 7fd87b6a4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cb9624c80 con 0x558cb9631930 2024-02-02T15:37:29.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.131+0000 7fd865200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558cb9631010 0x558cb963bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:29.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.131+0000 7fd85fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558cb9631930 0x558cb9634740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:29.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.131+0000 7fd865200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558cb9631010 0x558cb963bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:52122/0 (socket says 172.21.15.1:52122) 2024-02-02T15:37:29.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.131+0000 7fd865200700 1 -- 172.21.15.1:0/2674090814 learned_addr learned my addr 172.21.15.1:0/2674090814 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:37:29.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.131+0000 7fd865c00700 1 --2- 172.21.15.1:0/2674090814 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558cb963c7e0 0x558cb9634c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:29.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.131+0000 7fd865200700 1 -- 172.21.15.1:0/2674090814 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558cb9631930 msgr2=0x558cb9634740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:37:29.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.131+0000 7fd865200700 1 --2- 172.21.15.1:0/2674090814 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558cb9631930 0x558cb9634740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:29.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.131+0000 7fd865200700 1 -- 172.21.15.1:0/2674090814 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558cb963c7e0 msgr2=0x558cb9634c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:37:29.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.131+0000 7fd865200700 1 --2- 172.21.15.1:0/2674090814 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558cb963c7e0 0x558cb9634c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:29.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.132+0000 7fd865200700 1 -- 172.21.15.1:0/2674090814 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558cb9548ea0 con 0x558cb9631010 2024-02-02T15:37:29.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.132+0000 7fd865c00700 1 --2- 172.21.15.1:0/2674090814 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558cb963c7e0 0x558cb9634c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:37:29.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.132+0000 7fd865200700 1 --2- 172.21.15.1:0/2674090814 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558cb9631010 0x558cb963bd70 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fd8500958e0 tx=0x7fd850098dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a2599446c4d09d5 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:37:29.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.132+0000 7fd856a00700 1 -- 172.21.15.1:0/2674090814 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8500acea0 con 0x558cb9631010 2024-02-02T15:37:29.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.132+0000 7fd856a00700 1 -- 172.21.15.1:0/2674090814 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd8500985b0 con 0x558cb9631010 2024-02-02T15:37:29.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.132+0000 7fd856a00700 1 -- 172.21.15.1:0/2674090814 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd85009f7f0 con 0x558cb9631010 2024-02-02T15:37:29.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.133+0000 7fd87b6a4b80 1 -- 172.21.15.1:0/2674090814 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558cb9631010 msgr2=0x558cb963bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:37:29.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.133+0000 7fd87b6a4b80 1 --2- 172.21.15.1:0/2674090814 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558cb9631010 0x558cb963bd70 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fd8500958e0 tx=0x7fd850098dc0 comp rx=0 tx=0).stop 2024-02-02T15:37:29.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.133+0000 7fd87b6a4b80 1 -- 172.21.15.1:0/2674090814 shutdown_connections 2024-02-02T15:37:29.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.133+0000 7fd87b6a4b80 1 --2- 172.21.15.1:0/2674090814 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558cb963c7e0 0x558cb9634c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:29.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.133+0000 7fd87b6a4b80 1 --2- 172.21.15.1:0/2674090814 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558cb9631010 0x558cb963bd70 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:29.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.133+0000 7fd87b6a4b80 1 --2- 172.21.15.1:0/2674090814 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558cb9631930 0x558cb9634740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:29.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.133+0000 7fd87b6a4b80 1 -- 172.21.15.1:0/2674090814 >> 172.21.15.1:0/2674090814 conn(0x558cb9590db0 msgr2=0x558cb96485a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:37:29.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.133+0000 7fd87b6a4b80 1 -- 172.21.15.1:0/2674090814 shutdown_connections 2024-02-02T15:37:29.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.133+0000 7fd87b6a4b80 1 -- 172.21.15.1:0/2674090814 wait complete. 2024-02-02T15:37:29.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.133+0000 7fd87b6a4b80 1 -- start start 2024-02-02T15:37:29.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.133+0000 7fd87b6a4b80 1 Processor -- start 2024-02-02T15:37:29.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.134+0000 7fd87b6a4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558cb9631010 0x558cb9695b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:29.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.134+0000 7fd87b6a4b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558cb9631930 0x558cb9696090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:29.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.134+0000 7fd87b6a4b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558cb963c7e0 0x558cb963f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:29.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.134+0000 7fd87b6a4b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cb963f960 con 0x558cb9631930 2024-02-02T15:37:29.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.134+0000 7fd87b6a4b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cb963fad0 con 0x558cb963c7e0 2024-02-02T15:37:29.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.134+0000 7fd87b6a4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cb963fc90 con 0x558cb9631010 2024-02-02T15:37:29.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.134+0000 7fd87b6a4b80 10 client.0 _ll_register_callbacks cb 0x558cb9520290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:37:29.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.134+0000 7fd865200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558cb9631010 0x558cb9695b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:29.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.134+0000 7fd865200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558cb9631010 0x558cb9695b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:41942/0 (socket says 172.21.15.1:41942) 2024-02-02T15:37:29.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.134+0000 7fd865200700 1 -- 172.21.15.1:0/3248657774 learned_addr learned my addr 172.21.15.1:0/3248657774 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:37:29.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.134+0000 7fd85fe00700 1 --2- 172.21.15.1:0/3248657774 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558cb9631930 0x558cb9696090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:29.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.134+0000 7fd865c00700 1 --2- 172.21.15.1:0/3248657774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558cb963c7e0 0x558cb963f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:29.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.135+0000 7fd865200700 1 -- 172.21.15.1:0/3248657774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558cb963c7e0 msgr2=0x558cb963f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:37:29.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.135+0000 7fd865200700 1 --2- 172.21.15.1:0/3248657774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558cb963c7e0 0x558cb963f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:29.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.135+0000 7fd865200700 1 -- 172.21.15.1:0/3248657774 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558cb9631930 msgr2=0x558cb9696090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:37:29.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.135+0000 7fd865200700 1 --2- 172.21.15.1:0/3248657774 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558cb9631930 0x558cb9696090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:29.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.135+0000 7fd865200700 1 -- 172.21.15.1:0/3248657774 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558cb93b0b60 con 0x558cb9631010 2024-02-02T15:37:29.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.135+0000 7fd865200700 1 --2- 172.21.15.1:0/3248657774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558cb9631010 0x558cb9695b50 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fd850000c00 tx=0x7fd8500c9630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ecc06898e5284c36 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:37:29.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.135+0000 7fd84ea00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:37:29.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.135+0000 7fd84ea00700 1 -- 172.21.15.1:0/3248657774 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8500acea0 con 0x558cb9631010 2024-02-02T15:37:29.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.135+0000 7fd84ea00700 1 -- 172.21.15.1:0/3248657774 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fd85009d210 con 0x558cb9631010 2024-02-02T15:37:29.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.135+0000 7fd87b6a4b80 1 -- 172.21.15.1:0/3248657774 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd8500993e0 con 0x558cb9631010 2024-02-02T15:37:29.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.135+0000 7fd84ea00700 1 -- 172.21.15.1:0/3248657774 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd850099ae0 con 0x558cb9631010 2024-02-02T15:37:29.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.135+0000 7fd87b6a4b80 10 client.28366 fetch_fsmap learned FSMap version 49 2024-02-02T15:37:29.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.135+0000 7fd87b6a4b80 1 -- 172.21.15.1:0/3248657774 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x558cb9408cd0 con 0x558cb9631010 2024-02-02T15:37:29.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.135+0000 7fd84ea00700 1 -- 172.21.15.1:0/3248657774 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd8500993e0 con 0x558cb9631010 2024-02-02T15:37:29.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.136+0000 7fd84ea00700 1 -- 172.21.15.1:0/3248657774 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 49) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fd85009e0f0 con 0x558cb9631010 2024-02-02T15:37:29.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.136+0000 7fd87b6a4b80 10 client.28366 fetch_fsmap finished waiting for FSMap version 49 2024-02-02T15:37:29.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.136+0000 7fd87b6a4b80 10 client.28366 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:37:29.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.136+0000 7fd87b6a4b80 20 client.28366 populate_metadata read hostname 'smithi001' 2024-02-02T15:37:29.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.136+0000 7fd87b6a4b80 1 --2- 172.21.15.1:0/3248657774 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x558cb969e540 0x558cb96a0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:29.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.136+0000 7fd87b6a4b80 4 client.28366 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2024-02-02T15:37:29.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.136+0000 7fd87b6a4b80 1 -- 172.21.15.1:0/3248657774 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558cb9541f50 con 0x558cb969e540 2024-02-02T15:37:29.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.136+0000 7fd85fe00700 1 --2- 172.21.15.1:0/3248657774 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x558cb969e540 0x558cb96a0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:29.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.136+0000 7fd85fe00700 1 --2- 172.21.15.1:0/3248657774 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x558cb969e540 0x558cb96a0930 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=161fe78138b08848 server_cookie=c36f4e26732da396 in_seq=0 out_seq=0 2024-02-02T15:37:29.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.136+0000 7fd84ea00700 10 client.28366 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:37:29.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.137+0000 7fd84ea00700 1 -- 172.21.15.1:0/3248657774 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x558cb9541f50 con 0x558cb969e540 2024-02-02T15:37:29.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.137+0000 7fd84ea00700 10 client.28366 handle_command_reply: tid=0 2024-02-02T15:37:29.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.137+0000 7fd87b6a4b80 1 client.28366 shutdown 2024-02-02T15:37:29.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.137+0000 7fd87b6a4b80 1 -- 172.21.15.1:0/3248657774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558cb9631010 msgr2=0x558cb9695b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:37:29.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.137+0000 7fd87b6a4b80 1 --2- 172.21.15.1:0/3248657774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558cb9631010 0x558cb9695b50 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fd850000c00 tx=0x7fd8500c9630 comp rx=0 tx=0).stop 2024-02-02T15:37:29.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.137+0000 7fd87b6a4b80 1 -- 172.21.15.1:0/3248657774 shutdown_connections 2024-02-02T15:37:29.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.137+0000 7fd87b6a4b80 1 --2- 172.21.15.1:0/3248657774 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x558cb969e540 0x558cb96a0930 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:29.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.137+0000 7fd87b6a4b80 1 --2- 172.21.15.1:0/3248657774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558cb963c7e0 0x558cb963f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:29.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.137+0000 7fd87b6a4b80 1 --2- 172.21.15.1:0/3248657774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558cb9631010 0x558cb9695b50 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:29.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.137+0000 7fd87b6a4b80 1 --2- 172.21.15.1:0/3248657774 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558cb9631930 0x558cb9696090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:29.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.137+0000 7fd84ea00700 0 client.28366 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:37:29.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.137+0000 7fd87b6a4b80 1 -- 172.21.15.1:0/3248657774 >> 172.21.15.1:0/3248657774 conn(0x558cb9590db0 msgr2=0x558cb9634740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:37:29.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.137+0000 7fd87b6a4b80 1 -- 172.21.15.1:0/3248657774 shutdown_connections 2024-02-02T15:37:29.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.137+0000 7fd87b6a4b80 1 -- 172.21.15.1:0/3248657774 wait complete. 2024-02-02T15:37:29.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.137+0000 7fd87b6a4b80 20 client.28366 trim_cache size 0 max 16384 2024-02-02T15:37:29.141 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:37:29.141 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2024-02-02T15:37:29.141 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2024-02-02T15:37:29.141 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:37:29.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.138+0000 7fd867c00700 1 -- 172.21.15.1:0/2448170537 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fd8400a0710 msgr2=0x7fd8400a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:37:29.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.138+0000 7fd867c00700 1 --2- 172.21.15.1:0/2448170537 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fd8400a0710 0x7fd8400a2bd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd86019a7b0 tx=0x7fd848005040 comp rx=0 tx=0).stop 2024-02-02T15:37:29.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.138+0000 7fd867c00700 1 -- 172.21.15.1:0/2448170537 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd860101000 msgr2=0x7fd860199420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:37:29.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.138+0000 7fd867c00700 1 --2- 172.21.15.1:0/2448170537 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd860101000 0x7fd860199420 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fd85000e9a0 tx=0x7fd85000ecb0 comp rx=0 tx=0).stop 2024-02-02T15:37:29.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.138+0000 7fd867c00700 1 -- 172.21.15.1:0/2448170537 shutdown_connections 2024-02-02T15:37:29.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.138+0000 7fd867c00700 1 --2- 172.21.15.1:0/2448170537 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fd8400a0710 0x7fd8400a2bd0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:29.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.138+0000 7fd867c00700 1 --2- 172.21.15.1:0/2448170537 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd86019db40 0x7fd86019dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:29.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.138+0000 7fd867c00700 1 --2- 172.21.15.1:0/2448170537 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd860101000 0x7fd860199420 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:29.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.138+0000 7fd867c00700 1 --2- 172.21.15.1:0/2448170537 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd860106120 0x7fd860199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:29.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.138+0000 7fd867c00700 1 -- 172.21.15.1:0/2448170537 >> 172.21.15.1:0/2448170537 conn(0x7fd8600fc860 msgr2=0x7fd8601049d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:37:29.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.138+0000 7fd867c00700 1 -- 172.21.15.1:0/2448170537 shutdown_connections 2024-02-02T15:37:29.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.138+0000 7fd867c00700 1 -- 172.21.15.1:0/2448170537 wait complete. 2024-02-02T15:37:29.160 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-02T15:37:29.160 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T15:37:29.261 INFO:tasks.workunit.client.0.smithi001.stdout: 1 600790 5.91 MB/sec execute 423 sec latency 12.306 ms 2024-02-02T15:37:29.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.394+0000 7facca600700 1 -- 172.21.15.1:0/2527255094 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7facc4103790 msgr2=0x7facc4103c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:37:29.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.394+0000 7facca600700 1 --2- 172.21.15.1:0/2527255094 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7facc4103790 0x7facc4103c10 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7facbc00b0a0 tx=0x7facbc00b3b0 comp rx=0 tx=0).stop 2024-02-02T15:37:29.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.395+0000 7facca600700 1 -- 172.21.15.1:0/2527255094 shutdown_connections 2024-02-02T15:37:29.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.395+0000 7facca600700 1 --2- 172.21.15.1:0/2527255094 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7facc4103790 0x7facc4103c10 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:29.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.395+0000 7facca600700 1 --2- 172.21.15.1:0/2527255094 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7facc4105c70 0x7facc4069280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:29.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.395+0000 7facca600700 1 --2- 172.21.15.1:0/2527255094 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7facc40697c0 0x7facc4069c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:29.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.395+0000 7facca600700 1 -- 172.21.15.1:0/2527255094 >> 172.21.15.1:0/2527255094 conn(0x7facc4076d70 msgr2=0x7facc4077180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:37:29.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.395+0000 7facca600700 1 -- 172.21.15.1:0/2527255094 shutdown_connections 2024-02-02T15:37:29.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.395+0000 7facca600700 1 -- 172.21.15.1:0/2527255094 wait complete. 2024-02-02T15:37:29.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.396+0000 7facca600700 1 Processor -- start 2024-02-02T15:37:29.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.396+0000 7facca600700 1 -- start start 2024-02-02T15:37:29.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.396+0000 7facca600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7facc40697c0 0x7facc419d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:29.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.396+0000 7facca600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7facc4103790 0x7facc419dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:29.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.396+0000 7facca600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7facc4105c70 0x7facc41a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:29.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.397+0000 7facca600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc41a2550 con 0x7facc4103790 2024-02-02T15:37:29.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.397+0000 7facca600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc41a26c0 con 0x7facc4105c70 2024-02-02T15:37:29.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.397+0000 7facca600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc41a2880 con 0x7facc40697c0 2024-02-02T15:37:29.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.397+0000 7facc2a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7facc4103790 0x7facc419dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:29.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.397+0000 7facc3400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7facc40697c0 0x7facc419d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:29.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.397+0000 7facc3e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7facc4105c70 0x7facc41a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:29.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.397+0000 7facc3e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7facc4105c70 0x7facc41a1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:52148/0 (socket says 172.21.15.1:52148) 2024-02-02T15:37:29.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.397+0000 7facc2a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7facc4103790 0x7facc419dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39882/0 (socket says 172.21.15.1:39882) 2024-02-02T15:37:29.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.397+0000 7facc3e00700 1 -- 172.21.15.1:0/80051865 learned_addr learned my addr 172.21.15.1:0/80051865 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:37:29.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.398+0000 7facc3e00700 1 -- 172.21.15.1:0/80051865 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7facc40697c0 msgr2=0x7facc419d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:37:29.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.398+0000 7facc3e00700 1 --2- 172.21.15.1:0/80051865 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7facc40697c0 0x7facc419d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:29.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.398+0000 7facc3e00700 1 -- 172.21.15.1:0/80051865 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7facc4103790 msgr2=0x7facc419dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:37:29.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.398+0000 7facc3e00700 1 --2- 172.21.15.1:0/80051865 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7facc4103790 0x7facc419dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:29.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.398+0000 7facc3e00700 1 -- 172.21.15.1:0/80051865 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facbc009d00 con 0x7facc4105c70 2024-02-02T15:37:29.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.398+0000 7facc3400700 1 --2- 172.21.15.1:0/80051865 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7facc40697c0 0x7facc419d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:37:29.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.398+0000 7facc3e00700 1 --2- 172.21.15.1:0/80051865 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7facc4105c70 0x7facc41a1f10 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7facbc009fd0 tx=0x7facbc00ce90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:37:29.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.398+0000 7facbbe00700 1 -- 172.21.15.1:0/80051865 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facbc00e040 con 0x7facc4105c70 2024-02-02T15:37:29.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.398+0000 7facbbe00700 1 -- 172.21.15.1:0/80051865 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7facbc004ce0 con 0x7facc4105c70 2024-02-02T15:37:29.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.398+0000 7facbbe00700 1 -- 172.21.15.1:0/80051865 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facbc01b450 con 0x7facc4105c70 2024-02-02T15:37:29.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.398+0000 7facca600700 1 -- 172.21.15.1:0/80051865 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facc41a2b00 con 0x7facc4105c70 2024-02-02T15:37:29.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.398+0000 7facca600700 1 -- 172.21.15.1:0/80051865 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facc406a940 con 0x7facc4105c70 2024-02-02T15:37:29.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.399+0000 7facca600700 1 -- 172.21.15.1:0/80051865 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facc41a2d20 con 0x7facc4105c70 2024-02-02T15:37:29.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.401+0000 7facbbe00700 1 -- 172.21.15.1:0/80051865 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7facbc003a00 con 0x7facc4105c70 2024-02-02T15:37:29.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.403+0000 7facbbe00700 1 --2- 172.21.15.1:0/80051865 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7faca40a0690 0x7faca40a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:29.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.403+0000 7facbbe00700 1 -- 172.21.15.1:0/80051865 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9453+0+0 (secure 0 0 0) 0x7facbc0cd5d0 con 0x7facc4105c70 2024-02-02T15:37:29.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.403+0000 7facc3400700 1 --2- 172.21.15.1:0/80051865 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7faca40a0690 0x7faca40a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:29.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.403+0000 7facc3400700 1 --2- 172.21.15.1:0/80051865 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7faca40a0690 0x7faca40a2b50 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7facb4007f50 tx=0x7facb40073d0 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:37:29.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.407+0000 7facbbe00700 1 -- 172.21.15.1:0/80051865 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7facbc096ac0 con 0x7facc4105c70 2024-02-02T15:37:29.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.678+0000 7facca600700 1 -- 172.21.15.1:0/80051865 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7facc4068010 con 0x7facc4105c70 2024-02-02T15:37:29.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.679+0000 7facbbe00700 1 -- 172.21.15.1:0/80051865 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+10873 (secure 0 0 0) 0x7facbc09a0e0 con 0x7facc4105c70 2024-02-02T15:37:29.679 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:37:29.679 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":49,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2922240448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2922240448},{"type":"v1","addr":"172.21.15.1:6835","nonce":2922240448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2729497431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2729497431},{"type":"v1","addr":"172.21.15.1:6837","nonce":2729497431}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":49,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-02T15:24:47.761435+0000","modified":"2024-02-02T15:37:22.447521+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24529,"mds_2":24518,"mds_3":24514,"mds_4":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14517":{"gid":14517,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/1218303683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":1218303683},{"type":"v1","addr":"172.21.15.1:6839","nonce":1218303683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14529":{"gid":14529,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/2059197496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2059197496},{"type":"v1","addr":"172.21.15.1:6841","nonce":2059197496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/7648986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":7648986},{"type":"v1","addr":"172.21.15.142:6833","nonce":7648986}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/3482746094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3482746094},{"type":"v1","addr":"172.21.15.71:6833","nonce":3482746094}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/388207814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":388207814},{"type":"v1","addr":"172.21.15.71:6835","nonce":388207814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6835/1052606547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1052606547},{"type":"v1","addr":"172.21.15.142:6835","nonce":1052606547}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.142:6837/2870289019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2870289019},{"type":"v1","addr":"172.21.15.142:6837","nonce":2870289019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.71:6837/628016199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":628016199},{"type":"v1","addr":"172.21.15.71:6837","nonce":628016199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.142:6839/321675421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":321675421},{"type":"v1","addr":"172.21.15.142:6839","nonce":321675421}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/1268086493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1268086493},{"type":"v1","addr":"172.21.15.71:6839","nonce":1268086493}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-02T15:37:29.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.683+0000 7facca600700 1 -- 172.21.15.1:0/80051865 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7faca40a0690 msgr2=0x7faca40a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:37:29.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.683+0000 7facca600700 1 --2- 172.21.15.1:0/80051865 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7faca40a0690 0x7faca40a2b50 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7facb4007f50 tx=0x7facb40073d0 comp rx=0 tx=0).stop 2024-02-02T15:37:29.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.683+0000 7facca600700 1 -- 172.21.15.1:0/80051865 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7facc4105c70 msgr2=0x7facc41a1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:37:29.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.683+0000 7facca600700 1 --2- 172.21.15.1:0/80051865 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7facc4105c70 0x7facc41a1f10 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7facbc009fd0 tx=0x7facbc00ce90 comp rx=0 tx=0).stop 2024-02-02T15:37:29.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.685+0000 7facca600700 1 -- 172.21.15.1:0/80051865 shutdown_connections 2024-02-02T15:37:29.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.685+0000 7facca600700 1 --2- 172.21.15.1:0/80051865 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7faca40a0690 0x7faca40a2b50 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:29.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.685+0000 7facca600700 1 --2- 172.21.15.1:0/80051865 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7facc4105c70 0x7facc41a1f10 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:29.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.685+0000 7facca600700 1 --2- 172.21.15.1:0/80051865 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7facc40697c0 0x7facc419d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:29.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.685+0000 7facca600700 1 --2- 172.21.15.1:0/80051865 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7facc4103790 0x7facc419dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:29.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.685+0000 7facca600700 1 -- 172.21.15.1:0/80051865 >> 172.21.15.1:0/80051865 conn(0x7facc4076d70 msgr2=0x7facc40feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:37:29.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.685+0000 7facca600700 1 -- 172.21.15.1:0/80051865 shutdown_connections 2024-02-02T15:37:29.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.685+0000 7facca600700 1 -- 172.21.15.1:0/80051865 wait complete. 2024-02-02T15:37:29.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:29 smithi001 ceph-mon[35470]: pgmap v325: 129 pgs: 129 active+clean; 472 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.5 MiB/s wr, 124 op/s 2024-02-02T15:37:29.691 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 49 2024-02-02T15:37:29.708 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-02-02T15:37:29.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:29 smithi071 ceph-mon[44707]: pgmap v325: 129 pgs: 129 active+clean; 472 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.5 MiB/s wr, 124 op/s 2024-02-02T15:37:29.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:29 smithi142 ceph-mon[45728]: pgmap v325: 129 pgs: 129 active+clean; 472 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.5 MiB/s wr, 124 op/s 2024-02-02T15:37:29.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.947+0000 7f89fb000700 1 -- 172.21.15.1:0/284072049 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f89f4100ff0 msgr2=0x7f89f41013d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:37:29.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.947+0000 7f89fb000700 1 --2- 172.21.15.1:0/284072049 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f89f4100ff0 0x7f89f41013d0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f89e4009990 tx=0x7f89e4009ca0 comp rx=0 tx=0).stop 2024-02-02T15:37:29.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.947+0000 7f89fb000700 1 -- 172.21.15.1:0/284072049 shutdown_connections 2024-02-02T15:37:29.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.947+0000 7f89fb000700 1 --2- 172.21.15.1:0/284072049 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f89f4106110 0x7f89f410dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:29.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.947+0000 7f89fb000700 1 --2- 172.21.15.1:0/284072049 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f89f4100ff0 0x7f89f41013d0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:29.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.947+0000 7f89fb000700 1 --2- 172.21.15.1:0/284072049 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89f41019a0 0x7f89f41059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:29.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.947+0000 7f89fb000700 1 -- 172.21.15.1:0/284072049 >> 172.21.15.1:0/284072049 conn(0x7f89f40fc810 msgr2=0x7f89f40fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:37:29.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.947+0000 7f89fb000700 1 -- 172.21.15.1:0/284072049 shutdown_connections 2024-02-02T15:37:29.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.947+0000 7f89fb000700 1 -- 172.21.15.1:0/284072049 wait complete. 2024-02-02T15:37:29.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.948+0000 7f89fb000700 1 Processor -- start 2024-02-02T15:37:29.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.948+0000 7f89fb000700 1 -- start start 2024-02-02T15:37:29.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.949+0000 7f89fb000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f89f4100ff0 0x7f89f41990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:29.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.949+0000 7f89fb000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89f4106110 0x7f89f4199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:29.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.949+0000 7f89fb000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f89f419da60 0x7f89f419dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:29.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.949+0000 7f89fb000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f419e4e0 con 0x7f89f419da60 2024-02-02T15:37:29.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.949+0000 7f89fb000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f419e650 con 0x7f89f4100ff0 2024-02-02T15:37:29.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.949+0000 7f89fb000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f419e810 con 0x7f89f4106110 2024-02-02T15:37:29.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.949+0000 7f89f3e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f89f4100ff0 0x7f89f41990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:29.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.949+0000 7f89f3e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f89f4100ff0 0x7f89f41990c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:52166/0 (socket says 172.21.15.1:52166) 2024-02-02T15:37:29.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.949+0000 7f89f3e00700 1 -- 172.21.15.1:0/619289364 learned_addr learned my addr 172.21.15.1:0/619289364 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:37:29.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.949+0000 7f89f9000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f89f419da60 0x7f89f419dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:29.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.949+0000 7f89f3400700 1 --2- 172.21.15.1:0/619289364 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89f4106110 0x7f89f4199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:29.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.949+0000 7f89f3e00700 1 -- 172.21.15.1:0/619289364 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89f4106110 msgr2=0x7f89f4199600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:37:29.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.949+0000 7f89f3e00700 1 --2- 172.21.15.1:0/619289364 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89f4106110 0x7f89f4199600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:29.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.949+0000 7f89f3e00700 1 -- 172.21.15.1:0/619289364 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f89f419da60 msgr2=0x7f89f419dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:37:29.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.949+0000 7f89f3e00700 1 --2- 172.21.15.1:0/619289364 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f89f419da60 0x7f89f419dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:29.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.949+0000 7f89f3e00700 1 -- 172.21.15.1:0/619289364 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89ec009c40 con 0x7f89f4100ff0 2024-02-02T15:37:29.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.950+0000 7f89f3e00700 1 --2- 172.21.15.1:0/619289364 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f89f4100ff0 0x7f89f41990c0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f89e4005910 tx=0x7f89e4005720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:37:29.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.950+0000 7f89f0c00700 1 -- 172.21.15.1:0/619289364 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e401d070 con 0x7f89f4100ff0 2024-02-02T15:37:29.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.950+0000 7f89f0c00700 1 -- 172.21.15.1:0/619289364 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f89e4022470 con 0x7f89f4100ff0 2024-02-02T15:37:29.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.950+0000 7f89f0c00700 1 -- 172.21.15.1:0/619289364 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e4017650 con 0x7f89f4100ff0 2024-02-02T15:37:29.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.950+0000 7f89fb000700 1 -- 172.21.15.1:0/619289364 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89e40095f0 con 0x7f89f4100ff0 2024-02-02T15:37:29.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.950+0000 7f89fb000700 1 -- 172.21.15.1:0/619289364 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89f41a6000 con 0x7f89f4100ff0 2024-02-02T15:37:29.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.953+0000 7f89f0c00700 1 -- 172.21.15.1:0/619289364 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f89e40225e0 con 0x7f89f4100ff0 2024-02-02T15:37:29.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.954+0000 7f89f0c00700 1 --2- 172.21.15.1:0/619289364 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f89d40a0710 0x7f89d40a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:29.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.954+0000 7f89f0c00700 1 -- 172.21.15.1:0/619289364 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f89e40cd380 con 0x7f89f4100ff0 2024-02-02T15:37:29.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.954+0000 7f8a0ea43b80 1 Processor -- start 2024-02-02T15:37:29.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.954+0000 7f89f3400700 1 --2- 172.21.15.1:0/619289364 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f89d40a0710 0x7f89d40a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:29.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.955+0000 7f8a0ea43b80 1 -- start start 2024-02-02T15:37:29.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.955+0000 7f8a0ea43b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de40749010 0x55de407493f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:29.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.955+0000 7f8a0ea43b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55de40749930 0x55de407542a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:29.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.955+0000 7f8a0ea43b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55de407547e0 0x55de40756bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:29.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.955+0000 7f8a0ea43b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de406bd4c0 con 0x55de407547e0 2024-02-02T15:37:29.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.955+0000 7f8a0ea43b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de406a1aa0 con 0x55de40749930 2024-02-02T15:37:29.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.955+0000 7f8a0ea43b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de4069f460 con 0x55de40749010 2024-02-02T15:37:29.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.955+0000 7f89f3400700 1 --2- 172.21.15.1:0/619289364 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f89d40a0710 0x7f89d40a2bd0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f89f419a6d0 tx=0x7f89dc009260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:37:29.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.955+0000 7f89f3e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de40749010 0x55de407493f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:29.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.955+0000 7f89f9000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55de407547e0 0x55de40756bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:29.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.955+0000 7f89f3e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de40749010 0x55de407493f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:41992/0 (socket says 172.21.15.1:41992) 2024-02-02T15:37:29.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.955+0000 7f89f3e00700 1 -- 172.21.15.1:0/1375263131 learned_addr learned my addr 172.21.15.1:0/1375263131 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:37:29.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.955+0000 7f89f3400700 1 --2- 172.21.15.1:0/1375263131 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55de40749930 0x55de407542a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:29.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.956+0000 7f89f3e00700 1 -- 172.21.15.1:0/1375263131 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55de40749930 msgr2=0x55de407542a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:37:29.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.956+0000 7f89f3e00700 1 --2- 172.21.15.1:0/1375263131 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55de40749930 0x55de407542a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:29.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.956+0000 7f89f3e00700 1 -- 172.21.15.1:0/1375263131 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55de407547e0 msgr2=0x55de40756bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:37:29.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.956+0000 7f89f3e00700 1 --2- 172.21.15.1:0/1375263131 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55de407547e0 0x55de40756bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:29.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.956+0000 7f89f3e00700 1 -- 172.21.15.1:0/1375263131 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de4051c3b0 con 0x55de40749010 2024-02-02T15:37:29.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.956+0000 7f89f3e00700 1 --2- 172.21.15.1:0/1375263131 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de40749010 0x55de407493f0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f89e4006010 tx=0x7f89e4009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7650e8b51a79a4ed server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:37:29.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.956+0000 7f89e9600700 1 -- 172.21.15.1:0/1375263131 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e409ac30 con 0x55de40749010 2024-02-02T15:37:29.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.956+0000 7f89e9600700 1 -- 172.21.15.1:0/1375263131 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f89e409e120 con 0x55de40749010 2024-02-02T15:37:29.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.956+0000 7f89e9600700 1 -- 172.21.15.1:0/1375263131 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e40cd070 con 0x55de40749010 2024-02-02T15:37:29.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.957+0000 7f8a0ea43b80 1 -- 172.21.15.1:0/1375263131 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de40749010 msgr2=0x55de407493f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:37:29.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.957+0000 7f8a0ea43b80 1 --2- 172.21.15.1:0/1375263131 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de40749010 0x55de407493f0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f89e4006010 tx=0x7f89e4009ca0 comp rx=0 tx=0).stop 2024-02-02T15:37:29.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.957+0000 7f8a0ea43b80 1 -- 172.21.15.1:0/1375263131 shutdown_connections 2024-02-02T15:37:29.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.957+0000 7f8a0ea43b80 1 --2- 172.21.15.1:0/1375263131 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55de407547e0 0x55de40756bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:29.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.957+0000 7f8a0ea43b80 1 --2- 172.21.15.1:0/1375263131 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de40749010 0x55de407493f0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:29.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.957+0000 7f8a0ea43b80 1 --2- 172.21.15.1:0/1375263131 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55de40749930 0x55de407542a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:29.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.957+0000 7f8a0ea43b80 1 -- 172.21.15.1:0/1375263131 >> 172.21.15.1:0/1375263131 conn(0x55de406a8db0 msgr2=0x55de407453b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:37:29.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.957+0000 7f8a0ea43b80 1 -- 172.21.15.1:0/1375263131 shutdown_connections 2024-02-02T15:37:29.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.957+0000 7f8a0ea43b80 1 -- 172.21.15.1:0/1375263131 wait complete. 2024-02-02T15:37:29.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.957+0000 7f8a0ea43b80 1 -- start start 2024-02-02T15:37:29.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.958+0000 7f8a0ea43b80 1 Processor -- start 2024-02-02T15:37:29.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.958+0000 7f8a0ea43b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de40749010 0x55de40754bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:29.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.958+0000 7f8a0ea43b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55de40749930 0x55de40755100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:29.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.958+0000 7f8a0ea43b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55de407547e0 0x55de40755640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:29.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.958+0000 7f8a0ea43b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de4069ad50 con 0x55de407547e0 2024-02-02T15:37:29.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.958+0000 7f8a0ea43b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de40636d80 con 0x55de40749930 2024-02-02T15:37:29.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.958+0000 7f8a0ea43b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de40636a20 con 0x55de40749010 2024-02-02T15:37:29.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.958+0000 7f8a0ea43b80 10 client.0 _ll_register_callbacks cb 0x55de40638290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:37:29.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.958+0000 7f89f3400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55de40749930 0x55de40755100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:29.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.958+0000 7f89f9000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55de407547e0 0x55de40755640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:29.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.958+0000 7f89f3400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55de40749930 0x55de40755100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:52182/0 (socket says 172.21.15.1:52182) 2024-02-02T15:37:29.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.958+0000 7f89f3e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de40749010 0x55de40754bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:29.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.958+0000 7f89f3400700 1 -- 172.21.15.1:0/4156216713 learned_addr learned my addr 172.21.15.1:0/4156216713 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:37:29.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.959+0000 7f89f9000700 1 -- 172.21.15.1:0/4156216713 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de40749010 msgr2=0x55de40754bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:37:29.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.959+0000 7f89f9000700 1 --2- 172.21.15.1:0/4156216713 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de40749010 0x55de40754bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:29.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.959+0000 7f89f9000700 1 -- 172.21.15.1:0/4156216713 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55de40749930 msgr2=0x55de40755100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:37:29.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.959+0000 7f89f9000700 1 --2- 172.21.15.1:0/4156216713 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55de40749930 0x55de40755100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:29.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.959+0000 7f89f9000700 1 -- 172.21.15.1:0/4156216713 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89f41a6000 con 0x55de407547e0 2024-02-02T15:37:29.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.959+0000 7f89f3400700 1 --2- 172.21.15.1:0/4156216713 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55de40749930 0x55de40755100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:37:29.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.959+0000 7f89f9000700 1 --2- 172.21.15.1:0/4156216713 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55de407547e0 0x55de40755640 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f89ec00eee0 tx=0x7f89ec014600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e15d4bcd62c5898f server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:37:29.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.959+0000 7f89e2000700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:37:29.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.959+0000 7f89e2000700 1 -- 172.21.15.1:0/4156216713 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ec00e040 con 0x55de407547e0 2024-02-02T15:37:29.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.959+0000 7f89e2000700 1 -- 172.21.15.1:0/4156216713 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f89ec01a070 con 0x55de407547e0 2024-02-02T15:37:29.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.959+0000 7f8a0ea43b80 1 -- 172.21.15.1:0/4156216713 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55de4073d0f0 con 0x55de407547e0 2024-02-02T15:37:29.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.960+0000 7f89e2000700 1 -- 172.21.15.1:0/4156216713 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89ec0093a0 con 0x55de407547e0 2024-02-02T15:37:29.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.960+0000 7f8a0ea43b80 10 client.18291 fetch_fsmap learned FSMap version 49 2024-02-02T15:37:29.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.960+0000 7f8a0ea43b80 1 -- 172.21.15.1:0/4156216713 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55de404c8b60 con 0x55de407547e0 2024-02-02T15:37:29.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.960+0000 7f89e2000700 1 -- 172.21.15.1:0/4156216713 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55de4073d0f0 con 0x55de407547e0 2024-02-02T15:37:29.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.960+0000 7f89e2000700 1 -- 172.21.15.1:0/4156216713 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 49) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f89ec01f480 con 0x55de407547e0 2024-02-02T15:37:29.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.960+0000 7f8a0ea43b80 10 client.18291 fetch_fsmap finished waiting for FSMap version 49 2024-02-02T15:37:29.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.960+0000 7f8a0ea43b80 10 client.18291 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-02-02T15:37:29.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.960+0000 7f8a0ea43b80 20 client.18291 populate_metadata read hostname 'smithi001' 2024-02-02T15:37:29.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.960+0000 7f8a0ea43b80 1 --2- 172.21.15.1:0/4156216713 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55de40763f30 0x55de40766350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:29.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.960+0000 7f8a0ea43b80 4 client.18291 mds_command: new command op to 24502 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:37:29.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.960+0000 7f8a0ea43b80 1 -- 172.21.15.1:0/4156216713 --> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55de40693dd0 con 0x55de40763f30 2024-02-02T15:37:29.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.961+0000 7f89f3e00700 1 --2- 172.21.15.1:0/4156216713 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55de40763f30 0x55de40766350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:29.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.961+0000 7f89f3e00700 1 --2- 172.21.15.1:0/4156216713 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55de40763f30 0x55de40766350 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5decebe6e334ccb7 server_cookie=ef91b77ed108cf96 in_seq=0 out_seq=0 2024-02-02T15:37:29.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.961+0000 7f89e2000700 10 client.18291 ms_handle_connect on v2:172.21.15.142:6834/1052606547 2024-02-02T15:37:29.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.962+0000 7f89e2000700 1 -- 172.21.15.1:0/4156216713 <== mds.4 v2:172.21.15.142:6834/1052606547 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55de40693dd0 con 0x55de40763f30 2024-02-02T15:37:29.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.962+0000 7f89e2000700 10 client.18291 handle_command_reply: tid=0 2024-02-02T15:37:29.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.962+0000 7f8a0ea43b80 1 client.18291 shutdown 2024-02-02T15:37:29.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.962+0000 7f8a0ea43b80 1 -- 172.21.15.1:0/4156216713 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55de407547e0 msgr2=0x55de40755640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:37:29.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.962+0000 7f8a0ea43b80 1 --2- 172.21.15.1:0/4156216713 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55de407547e0 0x55de40755640 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f89ec00eee0 tx=0x7f89ec014600 comp rx=0 tx=0).stop 2024-02-02T15:37:29.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.962+0000 7f8a0ea43b80 1 -- 172.21.15.1:0/4156216713 shutdown_connections 2024-02-02T15:37:29.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.962+0000 7f8a0ea43b80 1 --2- 172.21.15.1:0/4156216713 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55de40763f30 0x55de40766350 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:29.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.962+0000 7f8a0ea43b80 1 --2- 172.21.15.1:0/4156216713 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55de407547e0 0x55de40755640 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:29.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.962+0000 7f8a0ea43b80 1 --2- 172.21.15.1:0/4156216713 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de40749010 0x55de40754bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:29.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.962+0000 7f8a0ea43b80 1 --2- 172.21.15.1:0/4156216713 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55de40749930 0x55de40755100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:29.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.962+0000 7f8a0ea43b80 1 -- 172.21.15.1:0/4156216713 >> 172.21.15.1:0/4156216713 conn(0x55de406a8db0 msgr2=0x55de40744fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:37:29.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.962+0000 7f89e2000700 0 client.18291 ms_handle_reset on v2:172.21.15.142:6834/1052606547 2024-02-02T15:37:29.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.962+0000 7f8a0ea43b80 1 -- 172.21.15.1:0/4156216713 shutdown_connections 2024-02-02T15:37:29.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.962+0000 7f8a0ea43b80 1 -- 172.21.15.1:0/4156216713 wait complete. 2024-02-02T15:37:29.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.962+0000 7f8a0ea43b80 20 client.18291 trim_cache size 0 max 16384 2024-02-02T15:37:29.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.976+0000 7f8a0ea43b80 1 Processor -- start 2024-02-02T15:37:29.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.976+0000 7f8a0ea43b80 1 -- start start 2024-02-02T15:37:29.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.976+0000 7f8a0ea43b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55de40749010 0x55de40758bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:29.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.976+0000 7f8a0ea43b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de40749930 0x55de4074dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:29.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.976+0000 7f8a0ea43b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55de407547e0 0x55de4074dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:29.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.976+0000 7f8a0ea43b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de4051e6a0 con 0x55de407547e0 2024-02-02T15:37:29.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.976+0000 7f8a0ea43b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de40746a30 con 0x55de40749010 2024-02-02T15:37:29.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.976+0000 7f8a0ea43b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de4073f500 con 0x55de40749930 2024-02-02T15:37:29.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.976+0000 7f89f3e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55de40749010 0x55de40758bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:29.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.976+0000 7f89f3e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55de40749010 0x55de40758bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:52184/0 (socket says 172.21.15.1:52184) 2024-02-02T15:37:29.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.976+0000 7f89f3e00700 1 -- 172.21.15.1:0/19668132 learned_addr learned my addr 172.21.15.1:0/19668132 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:37:29.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.976+0000 7f89f9000700 1 --2- 172.21.15.1:0/19668132 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55de407547e0 0x55de4074dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:29.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.976+0000 7f89f3400700 1 --2- 172.21.15.1:0/19668132 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de40749930 0x55de4074dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:29.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.977+0000 7f89f3e00700 1 -- 172.21.15.1:0/19668132 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de40749930 msgr2=0x55de4074dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:37:29.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.977+0000 7f89f3e00700 1 --2- 172.21.15.1:0/19668132 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de40749930 0x55de4074dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:29.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.977+0000 7f89f3e00700 1 -- 172.21.15.1:0/19668132 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55de407547e0 msgr2=0x55de4074dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:37:29.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.977+0000 7f89f3e00700 1 --2- 172.21.15.1:0/19668132 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55de407547e0 0x55de4074dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:29.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.977+0000 7f89f3e00700 1 -- 172.21.15.1:0/19668132 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de40660ea0 con 0x55de40749010 2024-02-02T15:37:29.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.977+0000 7f89f3e00700 1 --2- 172.21.15.1:0/19668132 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55de40749010 0x55de40758bf0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f89e4005f50 tx=0x7f89e4022c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a0393469966a655 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:37:29.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.977+0000 7f89ea000700 1 -- 172.21.15.1:0/19668132 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e409ac30 con 0x55de40749010 2024-02-02T15:37:29.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.977+0000 7f89ea000700 1 -- 172.21.15.1:0/19668132 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f89e409a530 con 0x55de40749010 2024-02-02T15:37:29.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.977+0000 7f89ea000700 1 -- 172.21.15.1:0/19668132 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e40229c0 con 0x55de40749010 2024-02-02T15:37:29.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.978+0000 7f8a0ea43b80 1 -- 172.21.15.1:0/19668132 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55de40749010 msgr2=0x55de40758bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:37:29.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.978+0000 7f8a0ea43b80 1 --2- 172.21.15.1:0/19668132 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55de40749010 0x55de40758bf0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f89e4005f50 tx=0x7f89e4022c90 comp rx=0 tx=0).stop 2024-02-02T15:37:29.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.978+0000 7f8a0ea43b80 1 -- 172.21.15.1:0/19668132 shutdown_connections 2024-02-02T15:37:29.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.978+0000 7f8a0ea43b80 1 --2- 172.21.15.1:0/19668132 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55de407547e0 0x55de4074dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:29.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.978+0000 7f8a0ea43b80 1 --2- 172.21.15.1:0/19668132 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55de40749010 0x55de40758bf0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:29.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.978+0000 7f8a0ea43b80 1 --2- 172.21.15.1:0/19668132 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de40749930 0x55de4074dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:29.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.978+0000 7f8a0ea43b80 1 -- 172.21.15.1:0/19668132 >> 172.21.15.1:0/19668132 conn(0x55de406a8db0 msgr2=0x55de4074ec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:37:29.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.978+0000 7f8a0ea43b80 1 -- 172.21.15.1:0/19668132 shutdown_connections 2024-02-02T15:37:29.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.978+0000 7f8a0ea43b80 1 -- 172.21.15.1:0/19668132 wait complete. 2024-02-02T15:37:29.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.978+0000 7f8a0ea43b80 1 -- start start 2024-02-02T15:37:29.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.978+0000 7f8a0ea43b80 1 Processor -- start 2024-02-02T15:37:29.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.979+0000 7f8a0ea43b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de40749010 0x55de407acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:29.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.979+0000 7f8a0ea43b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55de40749930 0x55de407ad2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:29.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.979+0000 7f8a0ea43b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55de407547e0 0x55de40757990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:29.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.979+0000 7f8a0ea43b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de407580a0 con 0x55de40749930 2024-02-02T15:37:29.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.979+0000 7f8a0ea43b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de40758210 con 0x55de407547e0 2024-02-02T15:37:29.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.979+0000 7f8a0ea43b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de407583d0 con 0x55de40749010 2024-02-02T15:37:29.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.979+0000 7f8a0ea43b80 10 client.0 _ll_register_callbacks cb 0x55de40638290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:37:29.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.979+0000 7f89f9000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55de407547e0 0x55de40757990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:29.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.979+0000 7f89f3400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55de40749930 0x55de407ad2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:29.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.979+0000 7f89f3e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de40749010 0x55de407acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:29.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.979+0000 7f89f9000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55de407547e0 0x55de40757990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:52186/0 (socket says 172.21.15.1:52186) 2024-02-02T15:37:29.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.979+0000 7f89f3400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55de40749930 0x55de407ad2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39936/0 (socket says 172.21.15.1:39936) 2024-02-02T15:37:29.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.979+0000 7f89f9000700 1 -- 172.21.15.1:0/1467166224 learned_addr learned my addr 172.21.15.1:0/1467166224 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:37:29.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.979+0000 7f89f9000700 1 -- 172.21.15.1:0/1467166224 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de40749010 msgr2=0x55de407acdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:37:29.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.979+0000 7f89f9000700 1 --2- 172.21.15.1:0/1467166224 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de40749010 0x55de407acdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:29.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.979+0000 7f89f9000700 1 -- 172.21.15.1:0/1467166224 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55de40749930 msgr2=0x55de407ad2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:37:29.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.979+0000 7f89f9000700 1 --2- 172.21.15.1:0/1467166224 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55de40749930 0x55de407ad2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:29.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.979+0000 7f89f9000700 1 -- 172.21.15.1:0/1467166224 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89e40b1040 con 0x55de407547e0 2024-02-02T15:37:29.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.979+0000 7f89f3400700 1 --2- 172.21.15.1:0/1467166224 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55de40749930 0x55de407ad2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:37:29.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.980+0000 7f89f9000700 1 --2- 172.21.15.1:0/1467166224 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55de407547e0 0x55de40757990 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f89ec000f80 tx=0x7f89ec009450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be17e554ad4599c0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:37:29.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.980+0000 7f89e2000700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:37:29.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.980+0000 7f89e2000700 1 -- 172.21.15.1:0/1467166224 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ec00e040 con 0x55de407547e0 2024-02-02T15:37:29.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.980+0000 7f89e2000700 1 -- 172.21.15.1:0/1467166224 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f89ec01a040 con 0x55de407547e0 2024-02-02T15:37:29.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.980+0000 7f8a0ea43b80 1 -- 172.21.15.1:0/1467166224 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f89e409e120 con 0x55de407547e0 2024-02-02T15:37:29.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.980+0000 7f89e2000700 1 -- 172.21.15.1:0/1467166224 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89ec014a60 con 0x55de407547e0 2024-02-02T15:37:29.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.980+0000 7f8a0ea43b80 10 client.28391 fetch_fsmap learned FSMap version 49 2024-02-02T15:37:29.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.980+0000 7f8a0ea43b80 1 -- 172.21.15.1:0/1467166224 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55de40520cd0 con 0x55de407547e0 2024-02-02T15:37:29.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.981+0000 7f89e2000700 1 -- 172.21.15.1:0/1467166224 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f89e409e120 con 0x55de407547e0 2024-02-02T15:37:29.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.981+0000 7f89e2000700 1 -- 172.21.15.1:0/1467166224 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 49) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f89ec007700 con 0x55de407547e0 2024-02-02T15:37:29.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.981+0000 7f8a0ea43b80 10 client.28391 fetch_fsmap finished waiting for FSMap version 49 2024-02-02T15:37:29.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.981+0000 7f8a0ea43b80 10 client.28391 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-02-02T15:37:29.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.981+0000 7f8a0ea43b80 20 client.28391 populate_metadata read hostname 'smithi001' 2024-02-02T15:37:29.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.981+0000 7f8a0ea43b80 1 --2- 172.21.15.1:0/1467166224 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55de407b5d20 0x55de407b8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:29.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.981+0000 7f8a0ea43b80 4 client.28391 mds_command: new command op to 24502 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:37:29.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.981+0000 7f8a0ea43b80 1 -- 172.21.15.1:0/1467166224 --> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55de40659f50 con 0x55de407b5d20 2024-02-02T15:37:29.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.981+0000 7f89f3e00700 1 --2- 172.21.15.1:0/1467166224 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55de407b5d20 0x55de407b8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:29.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.981+0000 7f89f3e00700 1 --2- 172.21.15.1:0/1467166224 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55de407b5d20 0x55de407b8140 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=683df515d6548bf server_cookie=8e8e24c6f1ead44b in_seq=0 out_seq=0 2024-02-02T15:37:29.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.981+0000 7f89e2000700 10 client.28391 ms_handle_connect on v2:172.21.15.142:6834/1052606547 2024-02-02T15:37:29.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.982+0000 7f89e2000700 1 -- 172.21.15.1:0/1467166224 <== mds.4 v2:172.21.15.142:6834/1052606547 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55de40659f50 con 0x55de407b5d20 2024-02-02T15:37:29.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.982+0000 7f89e2000700 10 client.28391 handle_command_reply: tid=0 2024-02-02T15:37:29.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.982+0000 7f8a0ea43b80 1 client.28391 shutdown 2024-02-02T15:37:29.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.982+0000 7f8a0ea43b80 1 -- 172.21.15.1:0/1467166224 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55de407547e0 msgr2=0x55de40757990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:37:29.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.982+0000 7f8a0ea43b80 1 --2- 172.21.15.1:0/1467166224 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55de407547e0 0x55de40757990 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f89ec000f80 tx=0x7f89ec009450 comp rx=0 tx=0).stop 2024-02-02T15:37:29.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.982+0000 7f8a0ea43b80 1 -- 172.21.15.1:0/1467166224 shutdown_connections 2024-02-02T15:37:29.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.982+0000 7f8a0ea43b80 1 --2- 172.21.15.1:0/1467166224 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55de407b5d20 0x55de407b8140 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:29.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.982+0000 7f8a0ea43b80 1 --2- 172.21.15.1:0/1467166224 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55de407547e0 0x55de40757990 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:29.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.982+0000 7f8a0ea43b80 1 --2- 172.21.15.1:0/1467166224 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55de40749010 0x55de407acdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:29.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.982+0000 7f8a0ea43b80 1 --2- 172.21.15.1:0/1467166224 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55de40749930 0x55de407ad2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:29.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.982+0000 7f89e2000700 0 client.28391 ms_handle_reset on v2:172.21.15.142:6834/1052606547 2024-02-02T15:37:29.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.982+0000 7f8a0ea43b80 1 -- 172.21.15.1:0/1467166224 >> 172.21.15.1:0/1467166224 conn(0x55de406a8db0 msgr2=0x55de407445c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:37:29.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.982+0000 7f8a0ea43b80 1 -- 172.21.15.1:0/1467166224 shutdown_connections 2024-02-02T15:37:29.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.982+0000 7f8a0ea43b80 1 -- 172.21.15.1:0/1467166224 wait complete. 2024-02-02T15:37:29.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.982+0000 7f8a0ea43b80 20 client.28391 trim_cache size 0 max 16384 2024-02-02T15:37:29.988 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:37:29.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.983+0000 7f89fb000700 1 -- 172.21.15.1:0/619289364 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f89d40a0710 msgr2=0x7f89d40a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:37:29.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.983+0000 7f89fb000700 1 --2- 172.21.15.1:0/619289364 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f89d40a0710 0x7f89d40a2bd0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f89f419a6d0 tx=0x7f89dc009260 comp rx=0 tx=0).stop 2024-02-02T15:37:29.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.983+0000 7f89fb000700 1 -- 172.21.15.1:0/619289364 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f89f4100ff0 msgr2=0x7f89f41990c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:37:29.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.983+0000 7f89fb000700 1 --2- 172.21.15.1:0/619289364 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f89f4100ff0 0x7f89f41990c0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f89e4005910 tx=0x7f89e4005720 comp rx=0 tx=0).stop 2024-02-02T15:37:29.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.983+0000 7f89fb000700 1 -- 172.21.15.1:0/619289364 shutdown_connections 2024-02-02T15:37:29.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.983+0000 7f89fb000700 1 --2- 172.21.15.1:0/619289364 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f89d40a0710 0x7f89d40a2bd0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:29.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.983+0000 7f89fb000700 1 --2- 172.21.15.1:0/619289364 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f89f419da60 0x7f89f419dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:29.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.983+0000 7f89fb000700 1 --2- 172.21.15.1:0/619289364 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f89f4100ff0 0x7f89f41990c0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:29.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.983+0000 7f89fb000700 1 --2- 172.21.15.1:0/619289364 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f89f4106110 0x7f89f4199600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:29.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.983+0000 7f89fb000700 1 -- 172.21.15.1:0/619289364 >> 172.21.15.1:0/619289364 conn(0x7f89f40fc810 msgr2=0x7f89f410cf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:37:29.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.983+0000 7f89fb000700 1 -- 172.21.15.1:0/619289364 shutdown_connections 2024-02-02T15:37:29.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:29.983+0000 7f89fb000700 1 -- 172.21.15.1:0/619289364 wait complete. 2024-02-02T15:37:30.000 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-02-02T15:37:30.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.200+0000 7f6797800700 1 -- 172.21.15.1:0/3510262440 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6790111da0 msgr2=0x7f6790114190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:37:30.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.200+0000 7f6797800700 1 --2- 172.21.15.1:0/3510262440 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6790111da0 0x7f6790114190 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f678800b0a0 tx=0x7f678800b3b0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.201+0000 7f6797800700 1 -- 172.21.15.1:0/3510262440 shutdown_connections 2024-02-02T15:37:30.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.201+0000 7f6797800700 1 --2- 172.21.15.1:0/3510262440 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6790111da0 0x7f6790114190 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.201+0000 7f6797800700 1 --2- 172.21.15.1:0/3510262440 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f67900730e0 0x7f67900734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.201+0000 7f6797800700 1 --2- 172.21.15.1:0/3510262440 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6790073a00 0x7f6790111860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.201+0000 7f6797800700 1 -- 172.21.15.1:0/3510262440 >> 172.21.15.1:0/3510262440 conn(0x7f67900fc860 msgr2=0x7f67900fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:37:30.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.201+0000 7f6797800700 1 -- 172.21.15.1:0/3510262440 shutdown_connections 2024-02-02T15:37:30.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.201+0000 7f6797800700 1 -- 172.21.15.1:0/3510262440 wait complete. 2024-02-02T15:37:30.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.202+0000 7f6797800700 1 Processor -- start 2024-02-02T15:37:30.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.202+0000 7f6797800700 1 -- start start 2024-02-02T15:37:30.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.202+0000 7f6797800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67900730e0 0x7f679019d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:30.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.202+0000 7f6797800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6790073a00 0x7f679019db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:30.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.202+0000 7f6797800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6790111da0 0x7f67901a1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:30.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.202+0000 7f6797800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67901a25f0 con 0x7f67900730e0 2024-02-02T15:37:30.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.202+0000 7f6797800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67901a2760 con 0x7f6790073a00 2024-02-02T15:37:30.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.202+0000 7f6797800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67901a2920 con 0x7f6790111da0 2024-02-02T15:37:30.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.202+0000 7f6794e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67900730e0 0x7f679019d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:30.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.203+0000 7f6794e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67900730e0 0x7f679019d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39950/0 (socket says 172.21.15.1:39950) 2024-02-02T15:37:30.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.203+0000 7f6794e00700 1 -- 172.21.15.1:0/1500033491 learned_addr learned my addr 172.21.15.1:0/1500033491 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:37:30.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.203+0000 7f6795800700 1 --2- 172.21.15.1:0/1500033491 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6790111da0 0x7f67901a1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:30.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.203+0000 7f678fe00700 1 --2- 172.21.15.1:0/1500033491 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6790073a00 0x7f679019db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:30.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.203+0000 7f6794e00700 1 -- 172.21.15.1:0/1500033491 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6790111da0 msgr2=0x7f67901a1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:37:30.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.203+0000 7f6794e00700 1 --2- 172.21.15.1:0/1500033491 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6790111da0 0x7f67901a1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.203+0000 7f6794e00700 1 -- 172.21.15.1:0/1500033491 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6790073a00 msgr2=0x7f679019db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:37:30.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.203+0000 7f6794e00700 1 --2- 172.21.15.1:0/1500033491 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6790073a00 0x7f679019db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.203+0000 7f6794e00700 1 -- 172.21.15.1:0/1500033491 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6778009520 con 0x7f67900730e0 2024-02-02T15:37:30.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.203+0000 7f6794e00700 1 --2- 172.21.15.1:0/1500033491 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67900730e0 0x7f679019d610 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f678000e9a0 tx=0x7f678000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:37:30.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.203+0000 7f678d600700 1 -- 172.21.15.1:0/1500033491 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6780009790 con 0x7f67900730e0 2024-02-02T15:37:30.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.203+0000 7f678d600700 1 -- 172.21.15.1:0/1500033491 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6780010430 con 0x7f67900730e0 2024-02-02T15:37:30.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.204+0000 7f6797800700 1 -- 172.21.15.1:0/1500033491 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6788009d00 con 0x7f67900730e0 2024-02-02T15:37:30.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.204+0000 7f678d600700 1 -- 172.21.15.1:0/1500033491 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6780018680 con 0x7f67900730e0 2024-02-02T15:37:30.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.204+0000 7f6797800700 1 -- 172.21.15.1:0/1500033491 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67901a2df0 con 0x7f67900730e0 2024-02-02T15:37:30.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.207+0000 7f678d600700 1 -- 172.21.15.1:0/1500033491 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f67800105a0 con 0x7f67900730e0 2024-02-02T15:37:30.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.208+0000 7f67ab29eb80 1 Processor -- start 2024-02-02T15:37:30.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.208+0000 7f67ab29eb80 1 -- start start 2024-02-02T15:37:30.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.208+0000 7f67ab29eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557901e00010 0x557901e003f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:30.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.208+0000 7f67ab29eb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557901e00930 0x557901e0b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:30.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.208+0000 7f678d600700 1 --2- 172.21.15.1:0/1500033491 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f67700a0710 0x7f67700a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:30.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.208+0000 7f67ab29eb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557901e0b7e0 0x557901e0dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:30.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.208+0000 7f67ab29eb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x557901d744c0 con 0x557901e0b7e0 2024-02-02T15:37:30.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.208+0000 7f67ab29eb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x557901d58aa0 con 0x557901e00930 2024-02-02T15:37:30.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.208+0000 7f67ab29eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557901d56460 con 0x557901e00010 2024-02-02T15:37:30.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.209+0000 7f678d600700 1 -- 172.21.15.1:0/1500033491 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f6780014070 con 0x7f67900730e0 2024-02-02T15:37:30.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.209+0000 7f6795800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557901e00010 0x557901e003f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:30.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.209+0000 7f6795800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557901e00010 0x557901e003f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:42054/0 (socket says 172.21.15.1:42054) 2024-02-02T15:37:30.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.209+0000 7f6795800700 1 -- 172.21.15.1:0/1799207061 learned_addr learned my addr 172.21.15.1:0/1799207061 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:37:30.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.209+0000 7f6794e00700 1 --2- 172.21.15.1:0/1500033491 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f67700a0710 0x7f67700a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:30.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.209+0000 7f6795800700 1 --2- 172.21.15.1:0/1799207061 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557901e0b7e0 0x557901e0dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:30.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.209+0000 7f678fe00700 1 --2- 172.21.15.1:0/1799207061 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557901e00930 0x557901e0b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:30.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.209+0000 7f678fe00700 1 -- 172.21.15.1:0/1799207061 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557901e00010 msgr2=0x557901e003f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:37:30.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.209+0000 7f678fe00700 1 --2- 172.21.15.1:0/1799207061 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557901e00010 0x557901e003f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.209+0000 7f678fe00700 1 -- 172.21.15.1:0/1799207061 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557901e0b7e0 msgr2=0x557901e0dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:37:30.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.209+0000 7f678fe00700 1 --2- 172.21.15.1:0/1799207061 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557901e0b7e0 0x557901e0dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.209+0000 7f678fe00700 1 -- 172.21.15.1:0/1799207061 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557901bd33b0 con 0x557901e00930 2024-02-02T15:37:30.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.209+0000 7f6795800700 1 --2- 172.21.15.1:0/1799207061 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557901e0b7e0 0x557901e0dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:37:30.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.209+0000 7f6794e00700 1 --2- 172.21.15.1:0/1500033491 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f67700a0710 0x7f67700a2bd0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f67800958c0 tx=0x7f6780010b80 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:37:30.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.210+0000 7f678fe00700 1 --2- 172.21.15.1:0/1799207061 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557901e00930 0x557901e0b2a0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f6778009e40 tx=0x7f6778011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15dee83263414e2f server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:37:30.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.210+0000 7f6786000700 1 -- 172.21.15.1:0/1799207061 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6778019630 con 0x557901e00930 2024-02-02T15:37:30.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.210+0000 7f6786000700 1 -- 172.21.15.1:0/1799207061 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6778019c50 con 0x557901e00930 2024-02-02T15:37:30.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.210+0000 7f6786000700 1 -- 172.21.15.1:0/1799207061 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f677800ebf0 con 0x557901e00930 2024-02-02T15:37:30.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.210+0000 7f67ab29eb80 1 -- 172.21.15.1:0/1799207061 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557901e00930 msgr2=0x557901e0b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:37:30.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.210+0000 7f67ab29eb80 1 --2- 172.21.15.1:0/1799207061 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557901e00930 0x557901e0b2a0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f6778009e40 tx=0x7f6778011720 comp rx=0 tx=0).stop 2024-02-02T15:37:30.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.211+0000 7f67ab29eb80 1 -- 172.21.15.1:0/1799207061 shutdown_connections 2024-02-02T15:37:30.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.211+0000 7f67ab29eb80 1 --2- 172.21.15.1:0/1799207061 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557901e0b7e0 0x557901e0dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.211+0000 7f67ab29eb80 1 --2- 172.21.15.1:0/1799207061 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557901e00010 0x557901e003f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.211+0000 7f67ab29eb80 1 --2- 172.21.15.1:0/1799207061 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557901e00930 0x557901e0b2a0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.211+0000 7f67ab29eb80 1 -- 172.21.15.1:0/1799207061 >> 172.21.15.1:0/1799207061 conn(0x557901d5fdb0 msgr2=0x557901dfc3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:37:30.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.211+0000 7f67ab29eb80 1 -- 172.21.15.1:0/1799207061 shutdown_connections 2024-02-02T15:37:30.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.211+0000 7f67ab29eb80 1 -- 172.21.15.1:0/1799207061 wait complete. 2024-02-02T15:37:30.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.211+0000 7f67ab29eb80 1 -- start start 2024-02-02T15:37:30.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.212+0000 7f67ab29eb80 1 Processor -- start 2024-02-02T15:37:30.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.212+0000 7f67ab29eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557901e00010 0x557901e0b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:30.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.212+0000 7f67ab29eb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557901e00930 0x557901e08400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:30.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.212+0000 7f67ab29eb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557901e0b7e0 0x557901e08a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:30.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.212+0000 7f67ab29eb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x557901d51d50 con 0x557901e00930 2024-02-02T15:37:30.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.212+0000 7f67ab29eb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x557901cedd80 con 0x557901e0b7e0 2024-02-02T15:37:30.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.212+0000 7f67ab29eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557901ceda20 con 0x557901e00010 2024-02-02T15:37:30.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.213+0000 7f678fe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557901e00930 0x557901e08400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:30.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.213+0000 7f6795800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557901e00010 0x557901e0b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:30.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.213+0000 7f678fe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557901e00930 0x557901e08400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39980/0 (socket says 172.21.15.1:39980) 2024-02-02T15:37:30.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.213+0000 7f6795800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557901e00010 0x557901e0b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:42056/0 (socket says 172.21.15.1:42056) 2024-02-02T15:37:30.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.213+0000 7f678fe00700 1 -- 172.21.15.1:0/2720600820 learned_addr learned my addr 172.21.15.1:0/2720600820 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:37:30.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.213+0000 7f67ab29eb80 10 client.0 _ll_register_callbacks cb 0x557901cef290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:37:30.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.213+0000 7f6795800700 1 --2- 172.21.15.1:0/2720600820 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557901e0b7e0 0x557901e08a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:30.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.213+0000 7f678fe00700 1 -- 172.21.15.1:0/2720600820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557901e00010 msgr2=0x557901e0b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:37:30.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.213+0000 7f678fe00700 1 --2- 172.21.15.1:0/2720600820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557901e00010 0x557901e0b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.213+0000 7f678fe00700 1 -- 172.21.15.1:0/2720600820 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557901e0b7e0 msgr2=0x557901e08a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:37:30.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.213+0000 7f678fe00700 1 --2- 172.21.15.1:0/2720600820 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557901e0b7e0 0x557901e08a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.213+0000 7f678fe00700 1 -- 172.21.15.1:0/2720600820 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6790114b40 con 0x557901e00930 2024-02-02T15:37:30.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.213+0000 7f6795800700 1 --2- 172.21.15.1:0/2720600820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557901e00010 0x557901e0b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:37:30.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.213+0000 7f678fe00700 1 --2- 172.21.15.1:0/2720600820 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557901e00930 0x557901e08400 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f6778009230 tx=0x7f6778004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fcc1419bf4896ca server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:37:30.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.213+0000 7f677ea00700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:37:30.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.214+0000 7f677ea00700 1 -- 172.21.15.1:0/2720600820 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6778019630 con 0x557901e00930 2024-02-02T15:37:30.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.214+0000 7f677ea00700 1 -- 172.21.15.1:0/2720600820 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f6778019790 con 0x557901e00930 2024-02-02T15:37:30.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.214+0000 7f67ab29eb80 1 -- 172.21.15.1:0/2720600820 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557901df40f0 con 0x557901e00930 2024-02-02T15:37:30.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.214+0000 7f677ea00700 1 -- 172.21.15.1:0/2720600820 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6778019990 con 0x557901e00930 2024-02-02T15:37:30.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.214+0000 7f67ab29eb80 10 client.18309 fetch_fsmap learned FSMap version 49 2024-02-02T15:37:30.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.214+0000 7f67ab29eb80 1 -- 172.21.15.1:0/2720600820 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x7f6778009a70 con 0x557901e00930 2024-02-02T15:37:30.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.214+0000 7f677ea00700 1 -- 172.21.15.1:0/2720600820 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557901df40f0 con 0x557901e00930 2024-02-02T15:37:30.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.214+0000 7f677ea00700 1 -- 172.21.15.1:0/2720600820 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 49) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f67780039e0 con 0x557901e00930 2024-02-02T15:37:30.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.215+0000 7f67ab29eb80 10 client.18309 fetch_fsmap finished waiting for FSMap version 49 2024-02-02T15:37:30.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.215+0000 7f67ab29eb80 10 client.18309 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-02-02T15:37:30.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.215+0000 7f67ab29eb80 20 client.18309 populate_metadata read hostname 'smithi001' 2024-02-02T15:37:30.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.215+0000 7f67ab29eb80 1 --2- 172.21.15.1:0/2720600820 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x557901e1b120 0x557901e1d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:30.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.215+0000 7f67ab29eb80 4 client.18309 mds_command: new command op to 24514 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:37:30.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.215+0000 7f67ab29eb80 1 -- 172.21.15.1:0/2720600820 --> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557901d4add0 con 0x557901e1b120 2024-02-02T15:37:30.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.215+0000 7f6795800700 1 --2- 172.21.15.1:0/2720600820 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x557901e1b120 0x557901e1d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:30.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.215+0000 7f6795800700 1 --2- 172.21.15.1:0/2720600820 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x557901e1b120 0x557901e1d540 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5a6deda16461b88a server_cookie=4af2f05d7b3aa701 in_seq=0 out_seq=0 2024-02-02T15:37:30.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.215+0000 7f677ea00700 10 client.18309 ms_handle_connect on v2:172.21.15.142:6836/2870289019 2024-02-02T15:37:30.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.216+0000 7f677ea00700 1 -- 172.21.15.1:0/2720600820 <== mds.3 v2:172.21.15.142:6836/2870289019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x557901d4add0 con 0x557901e1b120 2024-02-02T15:37:30.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.216+0000 7f677ea00700 10 client.18309 handle_command_reply: tid=0 2024-02-02T15:37:30.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.216+0000 7f67ab29eb80 1 client.18309 shutdown 2024-02-02T15:37:30.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.217+0000 7f67ab29eb80 1 -- 172.21.15.1:0/2720600820 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557901e00930 msgr2=0x557901e08400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:37:30.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.217+0000 7f67ab29eb80 1 --2- 172.21.15.1:0/2720600820 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557901e00930 0x557901e08400 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f6778009230 tx=0x7f6778004270 comp rx=0 tx=0).stop 2024-02-02T15:37:30.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.217+0000 7f67ab29eb80 1 -- 172.21.15.1:0/2720600820 shutdown_connections 2024-02-02T15:37:30.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.217+0000 7f67ab29eb80 1 --2- 172.21.15.1:0/2720600820 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557901e0b7e0 0x557901e08a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.217+0000 7f67ab29eb80 1 --2- 172.21.15.1:0/2720600820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557901e00010 0x557901e0b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.217+0000 7f67ab29eb80 1 --2- 172.21.15.1:0/2720600820 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x557901e1b120 0x557901e1d540 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.217+0000 7f67ab29eb80 1 --2- 172.21.15.1:0/2720600820 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557901e00930 0x557901e08400 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.217+0000 7f67ab29eb80 1 -- 172.21.15.1:0/2720600820 >> 172.21.15.1:0/2720600820 conn(0x557901d5fdb0 msgr2=0x557901dfbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:37:30.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.217+0000 7f677ea00700 0 client.18309 ms_handle_reset on v2:172.21.15.142:6836/2870289019 2024-02-02T15:37:30.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.217+0000 7f67ab29eb80 1 -- 172.21.15.1:0/2720600820 shutdown_connections 2024-02-02T15:37:30.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.217+0000 7f67ab29eb80 1 -- 172.21.15.1:0/2720600820 wait complete. 2024-02-02T15:37:30.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.217+0000 7f67ab29eb80 20 client.18309 trim_cache size 0 max 16384 2024-02-02T15:37:30.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.230+0000 7f67ab29eb80 1 Processor -- start 2024-02-02T15:37:30.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.230+0000 7f67ab29eb80 1 -- start start 2024-02-02T15:37:30.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.231+0000 7f67ab29eb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557901e00010 0x557901e1d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:30.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.231+0000 7f67ab29eb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557901e00930 0x557901dfab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:30.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.231+0000 7f67ab29eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557901e0b7e0 0x557901e02840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:30.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.231+0000 7f67ab29eb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x557901bd56a0 con 0x557901e00930 2024-02-02T15:37:30.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.231+0000 7f67ab29eb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x557901e042b0 con 0x557901e00010 2024-02-02T15:37:30.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.231+0000 7f67ab29eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557901e04030 con 0x557901e0b7e0 2024-02-02T15:37:30.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.231+0000 7f678fe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557901e00930 0x557901dfab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:30.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.231+0000 7f678fe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557901e00930 0x557901dfab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:39996/0 (socket says 172.21.15.1:39996) 2024-02-02T15:37:30.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.231+0000 7f678fe00700 1 -- 172.21.15.1:0/2951459753 learned_addr learned my addr 172.21.15.1:0/2951459753 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:37:30.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.231+0000 7f6795800700 1 --2- 172.21.15.1:0/2951459753 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557901e00010 0x557901e1d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:30.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.231+0000 7f6795800700 1 --2- 172.21.15.1:0/2951459753 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557901e0b7e0 0x557901e02840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:30.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.231+0000 7f678fe00700 1 -- 172.21.15.1:0/2951459753 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557901e0b7e0 msgr2=0x557901e02840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:37:30.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.231+0000 7f678fe00700 1 --2- 172.21.15.1:0/2951459753 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557901e0b7e0 0x557901e02840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.231+0000 7f678fe00700 1 -- 172.21.15.1:0/2951459753 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557901e00010 msgr2=0x557901e1d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:37:30.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.231+0000 7f678fe00700 1 --2- 172.21.15.1:0/2951459753 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557901e00010 0x557901e1d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.231+0000 7f678fe00700 1 -- 172.21.15.1:0/2951459753 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557901b7fb60 con 0x557901e00930 2024-02-02T15:37:30.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.232+0000 7f678fe00700 1 --2- 172.21.15.1:0/2951459753 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557901e00930 0x557901dfab20 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f6778009870 tx=0x7f67780042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1b22ee72b851af2 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:37:30.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.232+0000 7f6786a00700 1 -- 172.21.15.1:0/2951459753 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67780254a0 con 0x557901e00930 2024-02-02T15:37:30.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.232+0000 7f6786a00700 1 -- 172.21.15.1:0/2951459753 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6778003b40 con 0x557901e00930 2024-02-02T15:37:30.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.232+0000 7f6786a00700 1 -- 172.21.15.1:0/2951459753 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6778019820 con 0x557901e00930 2024-02-02T15:37:30.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.232+0000 7f67ab29eb80 1 -- 172.21.15.1:0/2951459753 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557901e00930 msgr2=0x557901dfab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:37:30.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.232+0000 7f67ab29eb80 1 --2- 172.21.15.1:0/2951459753 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557901e00930 0x557901dfab20 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f6778009870 tx=0x7f67780042e0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.232+0000 7f67ab29eb80 1 -- 172.21.15.1:0/2951459753 shutdown_connections 2024-02-02T15:37:30.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.232+0000 7f67ab29eb80 1 --2- 172.21.15.1:0/2951459753 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557901e0b7e0 0x557901e02840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.232+0000 7f67ab29eb80 1 --2- 172.21.15.1:0/2951459753 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557901e00010 0x557901e1d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.232+0000 7f67ab29eb80 1 --2- 172.21.15.1:0/2951459753 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557901e00930 0x557901dfab20 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.232+0000 7f67ab29eb80 1 -- 172.21.15.1:0/2951459753 >> 172.21.15.1:0/2951459753 conn(0x557901d5fdb0 msgr2=0x557901e16900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:37:30.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.233+0000 7f67ab29eb80 1 -- 172.21.15.1:0/2951459753 shutdown_connections 2024-02-02T15:37:30.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.233+0000 7f67ab29eb80 1 -- 172.21.15.1:0/2951459753 wait complete. 2024-02-02T15:37:30.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.233+0000 7f67ab29eb80 1 -- start start 2024-02-02T15:37:30.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.233+0000 7f67ab29eb80 1 Processor -- start 2024-02-02T15:37:30.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.234+0000 7f67ab29eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557901e00010 0x557901e05f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:30.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.234+0000 7f67ab29eb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557901e00930 0x557901e10310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:30.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.234+0000 7f67ab29eb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557901e0b7e0 0x557901e10850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:30.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.234+0000 7f67ab29eb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x557901e10e20 con 0x557901e0b7e0 2024-02-02T15:37:30.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.234+0000 7f67ab29eb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x557901e10f90 con 0x557901e00930 2024-02-02T15:37:30.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.234+0000 7f67ab29eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557901e11150 con 0x557901e00010 2024-02-02T15:37:30.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.234+0000 7f67ab29eb80 10 client.0 _ll_register_callbacks cb 0x557901cef290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:37:30.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.234+0000 7f678fe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557901e00930 0x557901e10310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:30.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.234+0000 7f678fe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557901e00930 0x557901e10310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:52260/0 (socket says 172.21.15.1:52260) 2024-02-02T15:37:30.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.234+0000 7f678fe00700 1 -- 172.21.15.1:0/739641099 learned_addr learned my addr 172.21.15.1:0/739641099 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:37:30.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.234+0000 7f6795800700 1 --2- 172.21.15.1:0/739641099 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557901e00010 0x557901e05f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:30.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.234+0000 7f6795800700 1 --2- 172.21.15.1:0/739641099 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557901e0b7e0 0x557901e10850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:30.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.234+0000 7f678fe00700 1 -- 172.21.15.1:0/739641099 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557901e00010 msgr2=0x557901e05f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:37:30.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.234+0000 7f678fe00700 1 --2- 172.21.15.1:0/739641099 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557901e00010 0x557901e05f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.234+0000 7f678fe00700 1 -- 172.21.15.1:0/739641099 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557901e0b7e0 msgr2=0x557901e10850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:37:30.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.234+0000 7f678fe00700 1 --2- 172.21.15.1:0/739641099 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557901e0b7e0 0x557901e10850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.234+0000 7f678fe00700 1 -- 172.21.15.1:0/739641099 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6778009a70 con 0x557901e00930 2024-02-02T15:37:30.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.234+0000 7f6795800700 1 --2- 172.21.15.1:0/739641099 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557901e0b7e0 0x557901e10850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:37:30.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.234+0000 7f678fe00700 1 --2- 172.21.15.1:0/739641099 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557901e00930 0x557901e10310 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x557901dfffe0 tx=0x7f6778019f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=631afa0cf3c29ca9 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:37:30.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.234+0000 7f677ea00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:37:30.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.235+0000 7f677ea00700 1 -- 172.21.15.1:0/739641099 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6778019430 con 0x557901e00930 2024-02-02T15:37:30.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.235+0000 7f677ea00700 1 -- 172.21.15.1:0/739641099 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f6778000e40 con 0x557901e00930 2024-02-02T15:37:30.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.235+0000 7f67ab29eb80 1 -- 172.21.15.1:0/739641099 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6778019c50 con 0x557901e00930 2024-02-02T15:37:30.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.235+0000 7f677ea00700 1 -- 172.21.15.1:0/739641099 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6778019590 con 0x557901e00930 2024-02-02T15:37:30.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.235+0000 7f67ab29eb80 10 client.28409 fetch_fsmap learned FSMap version 49 2024-02-02T15:37:30.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.235+0000 7f67ab29eb80 1 -- 172.21.15.1:0/739641099 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x557901d17ea0 con 0x557901e00930 2024-02-02T15:37:30.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.235+0000 7f677ea00700 1 -- 172.21.15.1:0/739641099 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6778019c50 con 0x557901e00930 2024-02-02T15:37:30.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.235+0000 7f677ea00700 1 -- 172.21.15.1:0/739641099 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 49) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f677800e6c0 con 0x557901e00930 2024-02-02T15:37:30.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.235+0000 7f67ab29eb80 10 client.28409 fetch_fsmap finished waiting for FSMap version 49 2024-02-02T15:37:30.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.235+0000 7f67ab29eb80 10 client.28409 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-02-02T15:37:30.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.235+0000 7f67ab29eb80 20 client.28409 populate_metadata read hostname 'smithi001' 2024-02-02T15:37:30.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.235+0000 7f67ab29eb80 1 --2- 172.21.15.1:0/739641099 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x557901e6cda0 0x557901e6f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:30.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.235+0000 7f67ab29eb80 4 client.28409 mds_command: new command op to 24514 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:37:30.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.235+0000 7f67ab29eb80 1 -- 172.21.15.1:0/739641099 --> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557901d10f50 con 0x557901e6cda0 2024-02-02T15:37:30.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.236+0000 7f6795800700 1 --2- 172.21.15.1:0/739641099 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x557901e6cda0 0x557901e6f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:30.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.236+0000 7f6795800700 1 --2- 172.21.15.1:0/739641099 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x557901e6cda0 0x557901e6f1c0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=403c74d18144907 server_cookie=1e542bc3d6ba536b in_seq=0 out_seq=0 2024-02-02T15:37:30.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.236+0000 7f677ea00700 10 client.28409 ms_handle_connect on v2:172.21.15.142:6836/2870289019 2024-02-02T15:37:30.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.236+0000 7f677ea00700 1 -- 172.21.15.1:0/739641099 <== mds.3 v2:172.21.15.142:6836/2870289019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557901d10f50 con 0x557901e6cda0 2024-02-02T15:37:30.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.236+0000 7f677ea00700 10 client.28409 handle_command_reply: tid=0 2024-02-02T15:37:30.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.236+0000 7f67ab29eb80 1 client.28409 shutdown 2024-02-02T15:37:30.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.236+0000 7f67ab29eb80 1 -- 172.21.15.1:0/739641099 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557901e00930 msgr2=0x557901e10310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:37:30.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.236+0000 7f67ab29eb80 1 --2- 172.21.15.1:0/739641099 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557901e00930 0x557901e10310 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x557901dfffe0 tx=0x7f6778019f50 comp rx=0 tx=0).stop 2024-02-02T15:37:30.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.236+0000 7f67ab29eb80 1 -- 172.21.15.1:0/739641099 shutdown_connections 2024-02-02T15:37:30.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.236+0000 7f67ab29eb80 1 --2- 172.21.15.1:0/739641099 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x557901e6cda0 0x557901e6f1c0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.237+0000 7f67ab29eb80 1 --2- 172.21.15.1:0/739641099 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557901e0b7e0 0x557901e10850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.237+0000 7f67ab29eb80 1 --2- 172.21.15.1:0/739641099 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557901e00010 0x557901e05f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.237+0000 7f677ea00700 0 client.28409 ms_handle_reset on v2:172.21.15.142:6836/2870289019 2024-02-02T15:37:30.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.237+0000 7f67ab29eb80 1 --2- 172.21.15.1:0/739641099 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557901e00930 0x557901e10310 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.237+0000 7f67ab29eb80 1 -- 172.21.15.1:0/739641099 >> 172.21.15.1:0/739641099 conn(0x557901d5fdb0 msgr2=0x557901e07ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:37:30.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.237+0000 7f67ab29eb80 1 -- 172.21.15.1:0/739641099 shutdown_connections 2024-02-02T15:37:30.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.237+0000 7f67ab29eb80 1 -- 172.21.15.1:0/739641099 wait complete. 2024-02-02T15:37:30.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.237+0000 7f67ab29eb80 20 client.28409 trim_cache size 0 max 16384 2024-02-02T15:37:30.242 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:37:30.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.237+0000 7f6797800700 1 -- 172.21.15.1:0/1500033491 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f67700a0710 msgr2=0x7f67700a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:37:30.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.237+0000 7f6797800700 1 --2- 172.21.15.1:0/1500033491 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f67700a0710 0x7f67700a2bd0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f67800958c0 tx=0x7f6780010b80 comp rx=0 tx=0).stop 2024-02-02T15:37:30.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.237+0000 7f6797800700 1 -- 172.21.15.1:0/1500033491 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67900730e0 msgr2=0x7f679019d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:37:30.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.237+0000 7f6797800700 1 --2- 172.21.15.1:0/1500033491 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67900730e0 0x7f679019d610 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f678000e9a0 tx=0x7f678000ecb0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.237+0000 7f6797800700 1 -- 172.21.15.1:0/1500033491 shutdown_connections 2024-02-02T15:37:30.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.237+0000 7f6797800700 1 --2- 172.21.15.1:0/1500033491 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f67700a0710 0x7f67700a2bd0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.237+0000 7f6797800700 1 --2- 172.21.15.1:0/1500033491 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6790111da0 0x7f67901a1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.237+0000 7f6797800700 1 --2- 172.21.15.1:0/1500033491 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67900730e0 0x7f679019d610 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.237+0000 7f6797800700 1 --2- 172.21.15.1:0/1500033491 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6790073a00 0x7f679019db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.238+0000 7f6797800700 1 -- 172.21.15.1:0/1500033491 >> 172.21.15.1:0/1500033491 conn(0x7f67900fc860 msgr2=0x7f67900fd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:37:30.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.238+0000 7f6797800700 1 -- 172.21.15.1:0/1500033491 shutdown_connections 2024-02-02T15:37:30.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.238+0000 7f6797800700 1 -- 172.21.15.1:0/1500033491 wait complete. 2024-02-02T15:37:30.256 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-02T15:37:30.261 INFO:tasks.workunit.client.0.smithi001.stdout: 1 600850 5.90 MB/sec execute 424 sec latency 925.458 ms 2024-02-02T15:37:30.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.465+0000 7f3363e00700 1 -- 172.21.15.1:0/4113293930 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f335c106110 msgr2=0x7f335c10dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:37:30.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.465+0000 7f3363e00700 1 --2- 172.21.15.1:0/4113293930 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f335c106110 0x7f335c10dc90 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f335400b0a0 tx=0x7f335400b3b0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.465+0000 7f3363e00700 1 -- 172.21.15.1:0/4113293930 shutdown_connections 2024-02-02T15:37:30.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.465+0000 7f3363e00700 1 --2- 172.21.15.1:0/4113293930 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f335c106110 0x7f335c10dc90 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.465+0000 7f3363e00700 1 --2- 172.21.15.1:0/4113293930 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f335c100ff0 0x7f335c1013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.465+0000 7f3363e00700 1 --2- 172.21.15.1:0/4113293930 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f335c1019a0 0x7f335c1059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.465+0000 7f3363e00700 1 -- 172.21.15.1:0/4113293930 >> 172.21.15.1:0/4113293930 conn(0x7f335c0fc810 msgr2=0x7f335c0fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:37:30.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.465+0000 7f3363e00700 1 -- 172.21.15.1:0/4113293930 shutdown_connections 2024-02-02T15:37:30.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.465+0000 7f3363e00700 1 -- 172.21.15.1:0/4113293930 wait complete. 2024-02-02T15:37:30.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.466+0000 7f3363e00700 1 Processor -- start 2024-02-02T15:37:30.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.466+0000 7f3363e00700 1 -- start start 2024-02-02T15:37:30.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.467+0000 7f3363e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f335c100ff0 0x7f335c196ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:30.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.467+0000 7f3363e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f335c106110 0x7f335c197530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:30.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.467+0000 7f3363e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f335c19b990 0x7f335c19be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:30.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.467+0000 7f3363e00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f335c19c410 con 0x7f335c100ff0 2024-02-02T15:37:30.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.467+0000 7f3363e00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f335c19c580 con 0x7f335c106110 2024-02-02T15:37:30.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.467+0000 7f3363e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f335c19c740 con 0x7f335c19b990 2024-02-02T15:37:30.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.467+0000 7f3360a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f335c106110 0x7f335c197530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:30.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.467+0000 7f3361e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f335c19b990 0x7f335c19be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:30.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.467+0000 7f3360a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f335c106110 0x7f335c197530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:52278/0 (socket says 172.21.15.1:52278) 2024-02-02T15:37:30.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.467+0000 7f3361e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f335c19b990 0x7f335c19be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:42094/0 (socket says 172.21.15.1:42094) 2024-02-02T15:37:30.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.467+0000 7f3360a00700 1 -- 172.21.15.1:0/463173295 learned_addr learned my addr 172.21.15.1:0/463173295 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:37:30.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.467+0000 7f3361400700 1 --2- 172.21.15.1:0/463173295 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f335c100ff0 0x7f335c196ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:30.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.468+0000 7f3361e00700 1 -- 172.21.15.1:0/463173295 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f335c106110 msgr2=0x7f335c197530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:37:30.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.468+0000 7f3361e00700 1 --2- 172.21.15.1:0/463173295 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f335c106110 0x7f335c197530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.468+0000 7f3361e00700 1 -- 172.21.15.1:0/463173295 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f335c100ff0 msgr2=0x7f335c196ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:37:30.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.468+0000 7f3361e00700 1 --2- 172.21.15.1:0/463173295 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f335c100ff0 0x7f335c196ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.468+0000 7f3361e00700 1 -- 172.21.15.1:0/463173295 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3354009d00 con 0x7f335c19b990 2024-02-02T15:37:30.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.468+0000 7f3360a00700 1 --2- 172.21.15.1:0/463173295 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f335c106110 0x7f335c197530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:37:30.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.468+0000 7f3361e00700 1 --2- 172.21.15.1:0/463173295 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f335c19b990 0x7f335c19be10 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f3354009fd0 tx=0x7f3354007910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:37:30.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.469+0000 7f335a000700 1 -- 172.21.15.1:0/463173295 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f335400e040 con 0x7f335c19b990 2024-02-02T15:37:30.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.469+0000 7f335a000700 1 -- 172.21.15.1:0/463173295 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f335401d480 con 0x7f335c19b990 2024-02-02T15:37:30.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.469+0000 7f3363e00700 1 -- 172.21.15.1:0/463173295 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f335c1a3c40 con 0x7f335c19b990 2024-02-02T15:37:30.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.469+0000 7f335a000700 1 -- 172.21.15.1:0/463173295 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33540052a0 con 0x7f335c19b990 2024-02-02T15:37:30.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.469+0000 7f3363e00700 1 -- 172.21.15.1:0/463173295 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f335c1a4060 con 0x7f335c19b990 2024-02-02T15:37:30.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.472+0000 7f335a000700 1 -- 172.21.15.1:0/463173295 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f3354003a00 con 0x7f335c19b990 2024-02-02T15:37:30.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.474+0000 7f335a000700 1 --2- 172.21.15.1:0/463173295 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f333c0a0710 0x7f333c0a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:30.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.474+0000 7f335a000700 1 -- 172.21.15.1:0/463173295 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f33540cd150 con 0x7f335c19b990 2024-02-02T15:37:30.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.474+0000 7f3361400700 1 --2- 172.21.15.1:0/463173295 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f333c0a0710 0x7f333c0a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:30.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.475+0000 7f3377845b80 1 Processor -- start 2024-02-02T15:37:30.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.475+0000 7f3361400700 1 --2- 172.21.15.1:0/463173295 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f333c0a0710 0x7f333c0a2bd0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f334c009c60 tx=0x7f334c009260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:37:30.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.475+0000 7f3377845b80 1 -- start start 2024-02-02T15:37:30.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.475+0000 7f3377845b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d50e4dd010 0x55d50e4dd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:30.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.475+0000 7f3377845b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d50e4dd930 0x55d50e4ec6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:30.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.475+0000 7f3377845b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d50e4ecbf0 0x55d50e4eefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:30.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.475+0000 7f3377845b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d50e4514c0 con 0x55d50e4ecbf0 2024-02-02T15:37:30.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.475+0000 7f3377845b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d50e435aa0 con 0x55d50e4dd930 2024-02-02T15:37:30.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.475+0000 7f3377845b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d50e433460 con 0x55d50e4dd010 2024-02-02T15:37:30.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.475+0000 7f3361e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d50e4ecbf0 0x55d50e4eefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:30.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.475+0000 7f3361400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d50e4dd010 0x55d50e4dd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:30.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.475+0000 7f3360a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d50e4dd930 0x55d50e4ec6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:30.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.475+0000 7f3361400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d50e4dd010 0x55d50e4dd3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:42106/0 (socket says 172.21.15.1:42106) 2024-02-02T15:37:30.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.476+0000 7f3361400700 1 -- 172.21.15.1:0/1321261235 learned_addr learned my addr 172.21.15.1:0/1321261235 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:37:30.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.476+0000 7f3361e00700 1 -- 172.21.15.1:0/1321261235 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d50e4dd010 msgr2=0x55d50e4dd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:37:30.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.476+0000 7f3361e00700 1 --2- 172.21.15.1:0/1321261235 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d50e4dd010 0x55d50e4dd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.476+0000 7f3361e00700 1 -- 172.21.15.1:0/1321261235 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d50e4dd930 msgr2=0x55d50e4ec6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:37:30.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.476+0000 7f3361e00700 1 --2- 172.21.15.1:0/1321261235 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d50e4dd930 0x55d50e4ec6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.476+0000 7f3361e00700 1 -- 172.21.15.1:0/1321261235 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d50e2b03b0 con 0x55d50e4ecbf0 2024-02-02T15:37:30.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.476+0000 7f3361e00700 1 --2- 172.21.15.1:0/1321261235 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d50e4ecbf0 0x55d50e4eefe0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f33540ca4a0 tx=0x7f335400b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=624842c96311d628 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:37:30.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.476+0000 7f3352600700 1 -- 172.21.15.1:0/1321261235 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3354011390 con 0x55d50e4ecbf0 2024-02-02T15:37:30.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.476+0000 7f3352600700 1 -- 172.21.15.1:0/1321261235 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f335409aa00 con 0x55d50e4ecbf0 2024-02-02T15:37:30.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.476+0000 7f3352600700 1 -- 172.21.15.1:0/1321261235 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3354096a10 con 0x55d50e4ecbf0 2024-02-02T15:37:30.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.477+0000 7f3377845b80 1 -- 172.21.15.1:0/1321261235 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d50e4ecbf0 msgr2=0x55d50e4eefe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:37:30.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.477+0000 7f3377845b80 1 --2- 172.21.15.1:0/1321261235 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d50e4ecbf0 0x55d50e4eefe0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f33540ca4a0 tx=0x7f335400b3b0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.477+0000 7f3377845b80 1 -- 172.21.15.1:0/1321261235 shutdown_connections 2024-02-02T15:37:30.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.477+0000 7f3377845b80 1 --2- 172.21.15.1:0/1321261235 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d50e4ecbf0 0x55d50e4eefe0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.477+0000 7f3377845b80 1 --2- 172.21.15.1:0/1321261235 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d50e4dd010 0x55d50e4dd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.477+0000 7f3377845b80 1 --2- 172.21.15.1:0/1321261235 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d50e4dd930 0x55d50e4ec6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.477+0000 7f3377845b80 1 -- 172.21.15.1:0/1321261235 >> 172.21.15.1:0/1321261235 conn(0x55d50e43cdb0 msgr2=0x55d50e4d93b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:37:30.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.477+0000 7f3377845b80 1 -- 172.21.15.1:0/1321261235 shutdown_connections 2024-02-02T15:37:30.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.477+0000 7f3377845b80 1 -- 172.21.15.1:0/1321261235 wait complete. 2024-02-02T15:37:30.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.478+0000 7f3377845b80 1 -- start start 2024-02-02T15:37:30.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.478+0000 7f3377845b80 1 Processor -- start 2024-02-02T15:37:30.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.478+0000 7f3377845b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d50e4dd010 0x55d50e4ec580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:30.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.478+0000 7f3377845b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d50e4dd930 0x55d50e4e9790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:30.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.478+0000 7f3377845b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d50e4ecbf0 0x55d50e4e9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:30.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.478+0000 7f3377845b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d50e42ed50 con 0x55d50e4dd010 2024-02-02T15:37:30.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.478+0000 7f3377845b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d50e3cad80 con 0x55d50e4ecbf0 2024-02-02T15:37:30.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.478+0000 7f3377845b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d50e3caa20 con 0x55d50e4dd930 2024-02-02T15:37:30.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.479+0000 7f3377845b80 10 client.0 _ll_register_callbacks cb 0x55d50e3cc290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:37:30.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.479+0000 7f3361400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d50e4dd010 0x55d50e4ec580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:30.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.479+0000 7f3361400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d50e4dd010 0x55d50e4ec580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40064/0 (socket says 172.21.15.1:40064) 2024-02-02T15:37:30.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.479+0000 7f3361400700 1 -- 172.21.15.1:0/854603794 learned_addr learned my addr 172.21.15.1:0/854603794 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:37:30.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.479+0000 7f3361e00700 1 --2- 172.21.15.1:0/854603794 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d50e4ecbf0 0x55d50e4e9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:30.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.479+0000 7f3360a00700 1 --2- 172.21.15.1:0/854603794 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d50e4dd930 0x55d50e4e9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:30.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.479+0000 7f3361400700 1 -- 172.21.15.1:0/854603794 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d50e4dd930 msgr2=0x55d50e4e9790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:37:30.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.479+0000 7f3361400700 1 --2- 172.21.15.1:0/854603794 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d50e4dd930 0x55d50e4e9790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.479+0000 7f3361400700 1 -- 172.21.15.1:0/854603794 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d50e4ecbf0 msgr2=0x55d50e4e9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:37:30.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.479+0000 7f3361400700 1 --2- 172.21.15.1:0/854603794 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d50e4ecbf0 0x55d50e4e9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.479+0000 7f3361400700 1 -- 172.21.15.1:0/854603794 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f335409ee70 con 0x55d50e4dd010 2024-02-02T15:37:30.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.479+0000 7f3361400700 1 --2- 172.21.15.1:0/854603794 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d50e4dd010 0x55d50e4ec580 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f334c003900 tx=0x7f334c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4c20c9b6c8dabfe server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:37:30.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.480+0000 7f334aa00700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:37:30.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.480+0000 7f334aa00700 1 -- 172.21.15.1:0/854603794 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f334c022070 con 0x55d50e4dd010 2024-02-02T15:37:30.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.480+0000 7f334aa00700 1 -- 172.21.15.1:0/854603794 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f334c01d760 con 0x55d50e4dd010 2024-02-02T15:37:30.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.480+0000 7f3377845b80 1 -- 172.21.15.1:0/854603794 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d50e4d10f0 con 0x55d50e4dd010 2024-02-02T15:37:30.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.480+0000 7f334aa00700 1 -- 172.21.15.1:0/854603794 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f334c01d920 con 0x55d50e4dd010 2024-02-02T15:37:30.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.480+0000 7f3377845b80 10 client.18324 fetch_fsmap learned FSMap version 49 2024-02-02T15:37:30.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.480+0000 7f3377845b80 1 -- 172.21.15.1:0/854603794 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55d50e25cb60 con 0x55d50e4dd010 2024-02-02T15:37:30.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.480+0000 7f334aa00700 1 -- 172.21.15.1:0/854603794 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d50e4d10f0 con 0x55d50e4dd010 2024-02-02T15:37:30.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.480+0000 7f334aa00700 1 -- 172.21.15.1:0/854603794 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 49) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f334c027af0 con 0x55d50e4dd010 2024-02-02T15:37:30.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.481+0000 7f3377845b80 10 client.18324 fetch_fsmap finished waiting for FSMap version 49 2024-02-02T15:37:30.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.481+0000 7f3377845b80 10 client.18324 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-02-02T15:37:30.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.481+0000 7f3377845b80 20 client.18324 populate_metadata read hostname 'smithi001' 2024-02-02T15:37:30.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.481+0000 7f3377845b80 1 --2- 172.21.15.1:0/854603794 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55d50e4fc4c0 0x55d50e4fe8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:30.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.481+0000 7f3377845b80 4 client.18324 mds_command: new command op to 24518 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:37:30.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.481+0000 7f3377845b80 1 -- 172.21.15.1:0/854603794 --> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d50e427dd0 con 0x55d50e4fc4c0 2024-02-02T15:37:30.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.481+0000 7f3360a00700 1 --2- 172.21.15.1:0/854603794 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55d50e4fc4c0 0x55d50e4fe8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:30.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.481+0000 7f3360a00700 1 --2- 172.21.15.1:0/854603794 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55d50e4fc4c0 0x55d50e4fe8e0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2392a9e6d556b7ea server_cookie=72a9a1b5f2ca9e78 in_seq=0 out_seq=0 2024-02-02T15:37:30.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.481+0000 7f334aa00700 10 client.18324 ms_handle_connect on v2:172.21.15.71:6836/628016199 2024-02-02T15:37:30.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.482+0000 7f334aa00700 1 -- 172.21.15.1:0/854603794 <== mds.2 v2:172.21.15.71:6836/628016199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55d50e427dd0 con 0x55d50e4fc4c0 2024-02-02T15:37:30.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.482+0000 7f334aa00700 10 client.18324 handle_command_reply: tid=0 2024-02-02T15:37:30.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.482+0000 7f3377845b80 1 client.18324 shutdown 2024-02-02T15:37:30.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.482+0000 7f3377845b80 1 -- 172.21.15.1:0/854603794 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d50e4dd010 msgr2=0x55d50e4ec580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:37:30.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.482+0000 7f3377845b80 1 --2- 172.21.15.1:0/854603794 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d50e4dd010 0x55d50e4ec580 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f334c003900 tx=0x7f334c0047c0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.483+0000 7f3377845b80 1 -- 172.21.15.1:0/854603794 shutdown_connections 2024-02-02T15:37:30.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.483+0000 7f3377845b80 1 --2- 172.21.15.1:0/854603794 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d50e4ecbf0 0x55d50e4e9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.483+0000 7f3377845b80 1 --2- 172.21.15.1:0/854603794 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d50e4dd010 0x55d50e4ec580 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.483+0000 7f3377845b80 1 --2- 172.21.15.1:0/854603794 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55d50e4fc4c0 0x55d50e4fe8e0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.483+0000 7f3377845b80 1 --2- 172.21.15.1:0/854603794 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d50e4dd930 0x55d50e4e9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.483+0000 7f3377845b80 1 -- 172.21.15.1:0/854603794 >> 172.21.15.1:0/854603794 conn(0x55d50e43cdb0 msgr2=0x55d50e4d8fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:37:30.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.483+0000 7f334aa00700 0 client.18324 ms_handle_reset on v2:172.21.15.71:6836/628016199 2024-02-02T15:37:30.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.483+0000 7f3377845b80 1 -- 172.21.15.1:0/854603794 shutdown_connections 2024-02-02T15:37:30.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.483+0000 7f3377845b80 1 -- 172.21.15.1:0/854603794 wait complete. 2024-02-02T15:37:30.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.483+0000 7f3377845b80 20 client.18324 trim_cache size 0 max 16384 2024-02-02T15:37:30.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.496+0000 7f3377845b80 1 Processor -- start 2024-02-02T15:37:30.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.496+0000 7f3377845b80 1 -- start start 2024-02-02T15:37:30.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.496+0000 7f3377845b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d50e4dd010 0x55d50e4ec0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:30.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.496+0000 7f3377845b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d50e4dd930 0x55d50e4e4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:30.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.496+0000 7f3377845b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d50e4ecbf0 0x55d50e4e4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:30.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.496+0000 7f3377845b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d50e2b26a0 con 0x55d50e4ecbf0 2024-02-02T15:37:30.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.496+0000 7f3377845b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d50e427dd0 con 0x55d50e4dd010 2024-02-02T15:37:30.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.496+0000 7f3377845b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d50e4d0fc0 con 0x55d50e4dd930 2024-02-02T15:37:30.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.497+0000 7f3361400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d50e4dd010 0x55d50e4ec0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:30.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.497+0000 7f3361e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d50e4ecbf0 0x55d50e4e4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:30.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.497+0000 7f3361400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d50e4dd010 0x55d50e4ec0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:52322/0 (socket says 172.21.15.1:52322) 2024-02-02T15:37:30.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.497+0000 7f3361400700 1 -- 172.21.15.1:0/1820318466 learned_addr learned my addr 172.21.15.1:0/1820318466 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:37:30.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.497+0000 7f3360a00700 1 --2- 172.21.15.1:0/1820318466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d50e4dd930 0x55d50e4e4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:30.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.497+0000 7f3361e00700 1 -- 172.21.15.1:0/1820318466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d50e4dd930 msgr2=0x55d50e4e4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:37:30.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.497+0000 7f3361e00700 1 --2- 172.21.15.1:0/1820318466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d50e4dd930 0x55d50e4e4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.497+0000 7f3361e00700 1 -- 172.21.15.1:0/1820318466 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d50e4dd010 msgr2=0x55d50e4ec0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:37:30.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.497+0000 7f3361e00700 1 --2- 172.21.15.1:0/1820318466 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d50e4dd010 0x55d50e4ec0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.497+0000 7f3361e00700 1 -- 172.21.15.1:0/1820318466 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d50e4d50d0 con 0x55d50e4ecbf0 2024-02-02T15:37:30.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.497+0000 7f3361400700 1 --2- 172.21.15.1:0/1820318466 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d50e4dd010 0x55d50e4ec0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:37:30.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.497+0000 7f3361e00700 1 --2- 172.21.15.1:0/1820318466 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d50e4ecbf0 0x55d50e4e4ff0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f3354096800 tx=0x7f335400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c41a9e164e9a4e00 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:37:30.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.498+0000 7f3353000700 1 -- 172.21.15.1:0/1820318466 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3354011390 con 0x55d50e4ecbf0 2024-02-02T15:37:30.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.498+0000 7f3353000700 1 -- 172.21.15.1:0/1820318466 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f335409e130 con 0x55d50e4ecbf0 2024-02-02T15:37:30.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.498+0000 7f3353000700 1 -- 172.21.15.1:0/1820318466 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33540a0240 con 0x55d50e4ecbf0 2024-02-02T15:37:30.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.498+0000 7f3377845b80 1 -- 172.21.15.1:0/1820318466 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d50e4ecbf0 msgr2=0x55d50e4e4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:37:30.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.498+0000 7f3377845b80 1 --2- 172.21.15.1:0/1820318466 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d50e4ecbf0 0x55d50e4e4ff0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f3354096800 tx=0x7f335400b040 comp rx=0 tx=0).stop 2024-02-02T15:37:30.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.498+0000 7f3377845b80 1 -- 172.21.15.1:0/1820318466 shutdown_connections 2024-02-02T15:37:30.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.498+0000 7f3377845b80 1 --2- 172.21.15.1:0/1820318466 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d50e4ecbf0 0x55d50e4e4ff0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.498+0000 7f3377845b80 1 --2- 172.21.15.1:0/1820318466 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d50e4dd010 0x55d50e4ec0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.498+0000 7f3377845b80 1 --2- 172.21.15.1:0/1820318466 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d50e4dd930 0x55d50e4e4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.498+0000 7f3377845b80 1 -- 172.21.15.1:0/1820318466 >> 172.21.15.1:0/1820318466 conn(0x55d50e43cdb0 msgr2=0x55d50e4f7bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:37:30.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.498+0000 7f3377845b80 1 -- 172.21.15.1:0/1820318466 shutdown_connections 2024-02-02T15:37:30.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.498+0000 7f3377845b80 1 -- 172.21.15.1:0/1820318466 wait complete. 2024-02-02T15:37:30.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.499+0000 7f3377845b80 1 -- start start 2024-02-02T15:37:30.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.499+0000 7f3377845b80 1 Processor -- start 2024-02-02T15:37:30.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.499+0000 7f3377845b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d50e4dd010 0x55d50e4f0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:30.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.499+0000 7f3377845b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d50e4dd930 0x55d50e4f12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:30.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.499+0000 7f3377845b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d50e4ecbf0 0x55d50e54cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:30.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.499+0000 7f3377845b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d50e4f1a20 con 0x55d50e4dd930 2024-02-02T15:37:30.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.499+0000 7f3377845b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d50e4f1b90 con 0x55d50e4ecbf0 2024-02-02T15:37:30.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.499+0000 7f3377845b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d50e54d4e0 con 0x55d50e4dd010 2024-02-02T15:37:30.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.499+0000 7f3377845b80 10 client.0 _ll_register_callbacks cb 0x55d50e3cc290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:37:30.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.500+0000 7f3361400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d50e4dd010 0x55d50e4f0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:30.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.500+0000 7f3360a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d50e4dd930 0x55d50e4f12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:30.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.500+0000 7f3361400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d50e4dd010 0x55d50e4f0d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:42138/0 (socket says 172.21.15.1:42138) 2024-02-02T15:37:30.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.500+0000 7f3361400700 1 -- 172.21.15.1:0/3672758363 learned_addr learned my addr 172.21.15.1:0/3672758363 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:37:30.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.500+0000 7f3361e00700 1 --2- 172.21.15.1:0/3672758363 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d50e4ecbf0 0x55d50e54cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:30.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.500+0000 7f3361400700 1 -- 172.21.15.1:0/3672758363 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d50e4ecbf0 msgr2=0x55d50e54cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:37:30.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.500+0000 7f3361400700 1 --2- 172.21.15.1:0/3672758363 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d50e4ecbf0 0x55d50e54cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.500+0000 7f3361400700 1 -- 172.21.15.1:0/3672758363 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d50e4dd930 msgr2=0x55d50e4f12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:37:30.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.500+0000 7f3361400700 1 --2- 172.21.15.1:0/3672758363 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d50e4dd930 0x55d50e4f12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.500+0000 7f3361400700 1 -- 172.21.15.1:0/3672758363 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d50e25cb60 con 0x55d50e4dd010 2024-02-02T15:37:30.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.500+0000 7f3361e00700 1 --2- 172.21.15.1:0/3672758363 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d50e4ecbf0 0x55d50e54cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:37:30.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.500+0000 7f3361400700 1 --2- 172.21.15.1:0/3672758363 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d50e4dd010 0x55d50e4f0d70 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f334c000c00 tx=0x7f334c01c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3f2e44b8f0c0a38 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:37:30.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.500+0000 7f334aa00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:37:30.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.501+0000 7f334aa00700 1 -- 172.21.15.1:0/3672758363 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f334c022040 con 0x55d50e4dd010 2024-02-02T15:37:30.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.501+0000 7f3377845b80 1 -- 172.21.15.1:0/3672758363 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f335409aa00 con 0x55d50e4dd010 2024-02-02T15:37:30.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.501+0000 7f334aa00700 1 -- 172.21.15.1:0/3672758363 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f334c009e50 con 0x55d50e4dd010 2024-02-02T15:37:30.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.501+0000 7f334aa00700 1 -- 172.21.15.1:0/3672758363 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f334c01c5b0 con 0x55d50e4dd010 2024-02-02T15:37:30.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.501+0000 7f3377845b80 10 client.28426 fetch_fsmap learned FSMap version 49 2024-02-02T15:37:30.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.501+0000 7f3377845b80 1 -- 172.21.15.1:0/3672758363 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55d50e2b03b0 con 0x55d50e4dd010 2024-02-02T15:37:30.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.501+0000 7f334aa00700 1 -- 172.21.15.1:0/3672758363 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f335409aa00 con 0x55d50e4dd010 2024-02-02T15:37:30.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.501+0000 7f334aa00700 1 -- 172.21.15.1:0/3672758363 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 49) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f334c03e060 con 0x55d50e4dd010 2024-02-02T15:37:30.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.502+0000 7f3377845b80 10 client.28426 fetch_fsmap finished waiting for FSMap version 49 2024-02-02T15:37:30.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.502+0000 7f3377845b80 10 client.28426 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-02-02T15:37:30.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.502+0000 7f3377845b80 20 client.28426 populate_metadata read hostname 'smithi001' 2024-02-02T15:37:30.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.502+0000 7f3377845b80 1 --2- 172.21.15.1:0/3672758363 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55d50e54eb30 0x55d50e550f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:30.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.502+0000 7f3377845b80 4 client.28426 mds_command: new command op to 24518 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:37:30.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.502+0000 7f3377845b80 1 -- 172.21.15.1:0/3672758363 --> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d50e3edf50 con 0x55d50e54eb30 2024-02-02T15:37:30.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.502+0000 7f3360a00700 1 --2- 172.21.15.1:0/3672758363 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55d50e54eb30 0x55d50e550f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:30.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.502+0000 7f3360a00700 1 --2- 172.21.15.1:0/3672758363 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55d50e54eb30 0x55d50e550f50 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6affc2194e429c39 server_cookie=cbf94e0489003c84 in_seq=0 out_seq=0 2024-02-02T15:37:30.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.502+0000 7f334aa00700 10 client.28426 ms_handle_connect on v2:172.21.15.71:6836/628016199 2024-02-02T15:37:30.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.502+0000 7f334aa00700 1 -- 172.21.15.1:0/3672758363 <== mds.2 v2:172.21.15.71:6836/628016199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d50e3edf50 con 0x55d50e54eb30 2024-02-02T15:37:30.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.502+0000 7f334aa00700 10 client.28426 handle_command_reply: tid=0 2024-02-02T15:37:30.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.502+0000 7f3377845b80 1 client.28426 shutdown 2024-02-02T15:37:30.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.503+0000 7f3377845b80 1 -- 172.21.15.1:0/3672758363 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d50e4dd010 msgr2=0x55d50e4f0d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:37:30.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.503+0000 7f3377845b80 1 --2- 172.21.15.1:0/3672758363 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d50e4dd010 0x55d50e4f0d70 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f334c000c00 tx=0x7f334c01c450 comp rx=0 tx=0).stop 2024-02-02T15:37:30.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.503+0000 7f3377845b80 1 -- 172.21.15.1:0/3672758363 shutdown_connections 2024-02-02T15:37:30.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.503+0000 7f3377845b80 1 --2- 172.21.15.1:0/3672758363 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d50e4ecbf0 0x55d50e54cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.503+0000 7f3377845b80 1 --2- 172.21.15.1:0/3672758363 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55d50e54eb30 0x55d50e550f50 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.503+0000 7f3377845b80 1 --2- 172.21.15.1:0/3672758363 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d50e4dd010 0x55d50e4f0d70 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.503+0000 7f3377845b80 1 --2- 172.21.15.1:0/3672758363 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d50e4dd930 0x55d50e4f12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.503+0000 7f3377845b80 1 -- 172.21.15.1:0/3672758363 >> 172.21.15.1:0/3672758363 conn(0x55d50e43cdb0 msgr2=0x55d50e4e4ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:37:30.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.503+0000 7f334aa00700 0 client.28426 ms_handle_reset on v2:172.21.15.71:6836/628016199 2024-02-02T15:37:30.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.503+0000 7f3377845b80 1 -- 172.21.15.1:0/3672758363 shutdown_connections 2024-02-02T15:37:30.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.503+0000 7f3377845b80 1 -- 172.21.15.1:0/3672758363 wait complete. 2024-02-02T15:37:30.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.503+0000 7f3377845b80 20 client.28426 trim_cache size 0 max 16384 2024-02-02T15:37:30.508 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:37:30.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.503+0000 7f3363e00700 1 -- 172.21.15.1:0/463173295 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f333c0a0710 msgr2=0x7f333c0a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:37:30.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.504+0000 7f3363e00700 1 --2- 172.21.15.1:0/463173295 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f333c0a0710 0x7f333c0a2bd0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f334c009c60 tx=0x7f334c009260 comp rx=0 tx=0).stop 2024-02-02T15:37:30.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.504+0000 7f3363e00700 1 -- 172.21.15.1:0/463173295 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f335c19b990 msgr2=0x7f335c19be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:37:30.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.504+0000 7f3363e00700 1 --2- 172.21.15.1:0/463173295 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f335c19b990 0x7f335c19be10 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f3354009fd0 tx=0x7f3354007910 comp rx=0 tx=0).stop 2024-02-02T15:37:30.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.504+0000 7f3363e00700 1 -- 172.21.15.1:0/463173295 shutdown_connections 2024-02-02T15:37:30.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.504+0000 7f3363e00700 1 --2- 172.21.15.1:0/463173295 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f333c0a0710 0x7f333c0a2bd0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.504+0000 7f3363e00700 1 --2- 172.21.15.1:0/463173295 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f335c19b990 0x7f335c19be10 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.504+0000 7f3363e00700 1 --2- 172.21.15.1:0/463173295 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f335c100ff0 0x7f335c196ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.504+0000 7f3363e00700 1 --2- 172.21.15.1:0/463173295 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f335c106110 0x7f335c197530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.504+0000 7f3363e00700 1 -- 172.21.15.1:0/463173295 >> 172.21.15.1:0/463173295 conn(0x7f335c0fc810 msgr2=0x7f335c0fd260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:37:30.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.504+0000 7f3363e00700 1 -- 172.21.15.1:0/463173295 shutdown_connections 2024-02-02T15:37:30.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.504+0000 7f3363e00700 1 -- 172.21.15.1:0/463173295 wait complete. 2024-02-02T15:37:30.520 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:30 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/80051865' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:37:30.522 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-02T15:37:30.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.763+0000 7f8ccd200700 1 -- 172.21.15.1:0/3052062961 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8cc80730e0 msgr2=0x7f8cc80734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:37:30.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.763+0000 7f8ccd200700 1 --2- 172.21.15.1:0/3052062961 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8cc80730e0 0x7f8cc80734c0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f8cb8009990 tx=0x7f8cb8009ca0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.763+0000 7f8ccd200700 1 -- 172.21.15.1:0/3052062961 shutdown_connections 2024-02-02T15:37:30.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.763+0000 7f8ccd200700 1 --2- 172.21.15.1:0/3052062961 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8cc8111d90 0x7f8cc8114180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.763+0000 7f8ccd200700 1 --2- 172.21.15.1:0/3052062961 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8cc80730e0 0x7f8cc80734c0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.763+0000 7f8ccd200700 1 --2- 172.21.15.1:0/3052062961 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8cc8073a00 0x7f8cc8111850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.763+0000 7f8ccd200700 1 -- 172.21.15.1:0/3052062961 >> 172.21.15.1:0/3052062961 conn(0x7f8cc80fc810 msgr2=0x7f8cc80fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:37:30.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.763+0000 7f8ccd200700 1 -- 172.21.15.1:0/3052062961 shutdown_connections 2024-02-02T15:37:30.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.763+0000 7f8ccd200700 1 -- 172.21.15.1:0/3052062961 wait complete. 2024-02-02T15:37:30.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.764+0000 7f8ccd200700 1 Processor -- start 2024-02-02T15:37:30.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.764+0000 7f8ccd200700 1 -- start start 2024-02-02T15:37:30.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.764+0000 7f8ccd200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8cc80730e0 0x7f8cc819d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:30.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.764+0000 7f8ccd200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8cc8073a00 0x7f8cc819dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:30.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.764+0000 7f8ccd200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8cc8111d90 0x7f8cc81a1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:30.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.764+0000 7f8ccd200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc81a24d0 con 0x7f8cc8073a00 2024-02-02T15:37:30.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.764+0000 7f8ccd200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc81a2640 con 0x7f8cc80730e0 2024-02-02T15:37:30.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.764+0000 7f8ccd200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc81a2800 con 0x7f8cc8111d90 2024-02-02T15:37:30.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.765+0000 7f8cc5400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8cc8073a00 0x7f8cc819dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:30.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.765+0000 7f8cc5400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8cc8073a00 0x7f8cc819dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40096/0 (socket says 172.21.15.1:40096) 2024-02-02T15:37:30.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.765+0000 7f8cc5400700 1 -- 172.21.15.1:0/2112073456 learned_addr learned my addr 172.21.15.1:0/2112073456 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:37:30.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.765+0000 7f8cc5e00700 1 --2- 172.21.15.1:0/2112073456 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8cc80730e0 0x7f8cc819d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:30.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.765+0000 7f8cc6800700 1 --2- 172.21.15.1:0/2112073456 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8cc8111d90 0x7f8cc81a1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:30.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.765+0000 7f8cc5400700 1 -- 172.21.15.1:0/2112073456 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8cc8111d90 msgr2=0x7f8cc81a1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:37:30.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.765+0000 7f8cc5400700 1 --2- 172.21.15.1:0/2112073456 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8cc8111d90 0x7f8cc81a1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.765+0000 7f8cc5400700 1 -- 172.21.15.1:0/2112073456 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8cc80730e0 msgr2=0x7f8cc819d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:37:30.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.765+0000 7f8cc5400700 1 --2- 172.21.15.1:0/2112073456 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8cc80730e0 0x7f8cc819d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.765+0000 7f8cc5400700 1 -- 172.21.15.1:0/2112073456 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cc0009c40 con 0x7f8cc8073a00 2024-02-02T15:37:30.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.765+0000 7f8cc5400700 1 --2- 172.21.15.1:0/2112073456 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8cc8073a00 0x7f8cc819dad0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f8cb000d950 tx=0x7f8cb000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:37:30.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.766+0000 7f8cbea00700 1 -- 172.21.15.1:0/2112073456 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb0009930 con 0x7f8cc8073a00 2024-02-02T15:37:30.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.766+0000 7f8cbea00700 1 -- 172.21.15.1:0/2112073456 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8cb0010460 con 0x7f8cc8073a00 2024-02-02T15:37:30.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.766+0000 7f8ccd200700 1 -- 172.21.15.1:0/2112073456 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cb80095f0 con 0x7f8cc8073a00 2024-02-02T15:37:30.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.766+0000 7f8cbea00700 1 -- 172.21.15.1:0/2112073456 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb000f620 con 0x7f8cc8073a00 2024-02-02T15:37:30.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.766+0000 7f8ccd200700 1 -- 172.21.15.1:0/2112073456 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cc81a2cd0 con 0x7f8cc8073a00 2024-02-02T15:37:30.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.769+0000 7f8cbea00700 1 -- 172.21.15.1:0/2112073456 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f8cb00108d0 con 0x7f8cc8073a00 2024-02-02T15:37:30.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.770+0000 7f8cbea00700 1 --2- 172.21.15.1:0/2112073456 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f8ca80a0710 0x7f8ca80a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:30.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.770+0000 7f8ce0df4b80 1 Processor -- start 2024-02-02T15:37:30.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.770+0000 7f8cbea00700 1 -- 172.21.15.1:0/2112073456 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f8cb00cbed0 con 0x7f8cc8073a00 2024-02-02T15:37:30.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.770+0000 7f8ce0df4b80 1 -- start start 2024-02-02T15:37:30.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.770+0000 7f8cc5e00700 1 --2- 172.21.15.1:0/2112073456 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f8ca80a0710 0x7f8ca80a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:30.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.771+0000 7f8ce0df4b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e793ab5010 0x55e793ab53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:30.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.771+0000 7f8ce0df4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e793ab5930 0x55e793ac02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:30.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.771+0000 7f8ce0df4b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e793ac07e0 0x55e793ac2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:30.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.771+0000 7f8ce0df4b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e793a294c0 con 0x55e793ab5010 2024-02-02T15:37:30.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.771+0000 7f8ce0df4b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e793a0daa0 con 0x55e793ac07e0 2024-02-02T15:37:30.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.771+0000 7f8ce0df4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e793a0b460 con 0x55e793ab5930 2024-02-02T15:37:30.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.771+0000 7f8cc5e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e793ab5010 0x55e793ab53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:30.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.771+0000 7f8cc5400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e793ab5930 0x55e793ac02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:30.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.771+0000 7f8cc5400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e793ab5930 0x55e793ac02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:42174/0 (socket says 172.21.15.1:42174) 2024-02-02T15:37:30.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.771+0000 7f8cc5400700 1 -- 172.21.15.1:0/2996739971 learned_addr learned my addr 172.21.15.1:0/2996739971 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:37:30.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.771+0000 7f8cc6800700 1 --2- 172.21.15.1:0/2996739971 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e793ac07e0 0x55e793ac2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:30.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.771+0000 7f8cc5e00700 1 --2- 172.21.15.1:0/2112073456 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f8ca80a0710 0x7f8ca80a2bd0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8cb800b2e0 tx=0x7f8cb8005fd0 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:37:30.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.772+0000 7f8cc5e00700 1 -- 172.21.15.1:0/2996739971 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e793ab5930 msgr2=0x55e793ac02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:37:30.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.772+0000 7f8cc5e00700 1 --2- 172.21.15.1:0/2996739971 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e793ab5930 0x55e793ac02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.772+0000 7f8cc5e00700 1 -- 172.21.15.1:0/2996739971 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e793ac07e0 msgr2=0x55e793ac2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:37:30.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.772+0000 7f8cc5e00700 1 --2- 172.21.15.1:0/2996739971 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e793ac07e0 0x55e793ac2bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.772+0000 7f8cc5e00700 1 -- 172.21.15.1:0/2996739971 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e7938883b0 con 0x55e793ab5010 2024-02-02T15:37:30.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.772+0000 7f8cc5400700 1 --2- 172.21.15.1:0/2996739971 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e793ab5930 0x55e793ac02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:37:30.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.772+0000 7f8cc5e00700 1 --2- 172.21.15.1:0/2996739971 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e793ab5010 0x55e793ab53f0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f8cb800b6f0 tx=0x7f8cb8020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f81a7a0f4421079c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:37:30.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.772+0000 7f8cb7400700 1 -- 172.21.15.1:0/2996739971 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb8004700 con 0x55e793ab5010 2024-02-02T15:37:30.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.772+0000 7f8cb7400700 1 -- 172.21.15.1:0/2996739971 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8cb801e040 con 0x55e793ab5010 2024-02-02T15:37:30.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.772+0000 7f8cb7400700 1 -- 172.21.15.1:0/2996739971 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb800f990 con 0x55e793ab5010 2024-02-02T15:37:30.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.773+0000 7f8ce0df4b80 1 -- 172.21.15.1:0/2996739971 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e793ab5010 msgr2=0x55e793ab53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:37:30.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.773+0000 7f8ce0df4b80 1 --2- 172.21.15.1:0/2996739971 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e793ab5010 0x55e793ab53f0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f8cb800b6f0 tx=0x7f8cb8020000 comp rx=0 tx=0).stop 2024-02-02T15:37:30.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.773+0000 7f8ce0df4b80 1 -- 172.21.15.1:0/2996739971 shutdown_connections 2024-02-02T15:37:30.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.773+0000 7f8ce0df4b80 1 --2- 172.21.15.1:0/2996739971 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e793ac07e0 0x55e793ac2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.773+0000 7f8ce0df4b80 1 --2- 172.21.15.1:0/2996739971 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e793ab5010 0x55e793ab53f0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.773+0000 7f8ce0df4b80 1 --2- 172.21.15.1:0/2996739971 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e793ab5930 0x55e793ac02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.773+0000 7f8ce0df4b80 1 -- 172.21.15.1:0/2996739971 >> 172.21.15.1:0/2996739971 conn(0x55e793a14db0 msgr2=0x55e793ab13b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:37:30.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.773+0000 7f8ce0df4b80 1 -- 172.21.15.1:0/2996739971 shutdown_connections 2024-02-02T15:37:30.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.773+0000 7f8ce0df4b80 1 -- 172.21.15.1:0/2996739971 wait complete. 2024-02-02T15:37:30.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.774+0000 7f8ce0df4b80 1 -- start start 2024-02-02T15:37:30.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.774+0000 7f8ce0df4b80 1 Processor -- start 2024-02-02T15:37:30.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.774+0000 7f8ce0df4b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e793ab5010 0x55e793ac01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:30.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.774+0000 7f8ce0df4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e793ab5930 0x55e793abd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:30.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.774+0000 7f8ce0df4b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e793ac07e0 0x55e793abda00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:30.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.774+0000 7f8ce0df4b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e793a06d50 con 0x55e793ab5010 2024-02-02T15:37:30.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.774+0000 7f8ce0df4b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7939a2d80 con 0x55e793ac07e0 2024-02-02T15:37:30.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.774+0000 7f8ce0df4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7939a2a20 con 0x55e793ab5930 2024-02-02T15:37:30.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.774+0000 7f8ce0df4b80 10 client.0 _ll_register_callbacks cb 0x55e7939a4290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:37:30.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.775+0000 7f8cc5e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e793ab5010 0x55e793ac01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:30.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.775+0000 7f8cc5e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e793ab5010 0x55e793ac01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40110/0 (socket says 172.21.15.1:40110) 2024-02-02T15:37:30.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.775+0000 7f8cc5e00700 1 -- 172.21.15.1:0/1693414546 learned_addr learned my addr 172.21.15.1:0/1693414546 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:37:30.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.775+0000 7f8cc6800700 1 --2- 172.21.15.1:0/1693414546 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e793ac07e0 0x55e793abda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:30.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.775+0000 7f8cc5400700 1 --2- 172.21.15.1:0/1693414546 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e793ab5930 0x55e793abd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:30.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.775+0000 7f8cc5e00700 1 -- 172.21.15.1:0/1693414546 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e793ab5930 msgr2=0x55e793abd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:37:30.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.775+0000 7f8cc5e00700 1 --2- 172.21.15.1:0/1693414546 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e793ab5930 0x55e793abd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.775+0000 7f8cc5e00700 1 -- 172.21.15.1:0/1693414546 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e793ac07e0 msgr2=0x55e793abda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:37:30.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.775+0000 7f8cc5e00700 1 --2- 172.21.15.1:0/1693414546 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e793ac07e0 0x55e793abda00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.775+0000 7f8cc5e00700 1 -- 172.21.15.1:0/1693414546 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cc81a2cd0 con 0x55e793ab5010 2024-02-02T15:37:30.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.775+0000 7f8cc5e00700 1 --2- 172.21.15.1:0/1693414546 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e793ab5010 0x55e793ac01e0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f8cb80098e0 tx=0x7f8cb8004c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf765665e66dfd11 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:37:30.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.775+0000 7f8cafe00700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:37:30.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.776+0000 7f8cafe00700 1 -- 172.21.15.1:0/1693414546 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb800f870 con 0x55e793ab5010 2024-02-02T15:37:30.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.776+0000 7f8cafe00700 1 -- 172.21.15.1:0/1693414546 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f8cb800f9d0 con 0x55e793ab5010 2024-02-02T15:37:30.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.776+0000 7f8ce0df4b80 1 -- 172.21.15.1:0/1693414546 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e793aa90f0 con 0x55e793ab5010 2024-02-02T15:37:30.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.776+0000 7f8cafe00700 1 -- 172.21.15.1:0/1693414546 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8cb800fb90 con 0x55e793ab5010 2024-02-02T15:37:30.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.776+0000 7f8ce0df4b80 10 client.18342 fetch_fsmap learned FSMap version 49 2024-02-02T15:37:30.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.776+0000 7f8ce0df4b80 1 -- 172.21.15.1:0/1693414546 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x7f8cc8114b30 con 0x55e793ab5010 2024-02-02T15:37:30.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.776+0000 7f8cafe00700 1 -- 172.21.15.1:0/1693414546 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e793aa90f0 con 0x55e793ab5010 2024-02-02T15:37:30.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.776+0000 7f8cafe00700 1 -- 172.21.15.1:0/1693414546 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 49) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f8cb8030ca0 con 0x55e793ab5010 2024-02-02T15:37:30.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.776+0000 7f8ce0df4b80 10 client.18342 fetch_fsmap finished waiting for FSMap version 49 2024-02-02T15:37:30.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.776+0000 7f8ce0df4b80 10 client.18342 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-02T15:37:30.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.776+0000 7f8ce0df4b80 20 client.18342 populate_metadata read hostname 'smithi001' 2024-02-02T15:37:30.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.776+0000 7f8ce0df4b80 1 --2- 172.21.15.1:0/1693414546 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55e793acfd20 0x55e793ad2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:30.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.776+0000 7f8ce0df4b80 4 client.18342 mds_command: new command op to 24529 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:37:30.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.776+0000 7f8ce0df4b80 1 -- 172.21.15.1:0/1693414546 --> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e7939ffdd0 con 0x55e793acfd20 2024-02-02T15:37:30.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.777+0000 7f8cc5400700 1 --2- 172.21.15.1:0/1693414546 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55e793acfd20 0x55e793ad2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:30.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.777+0000 7f8cc5400700 1 --2- 172.21.15.1:0/1693414546 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55e793acfd20 0x55e793ad2140 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4b509d5d05d567bc server_cookie=c434807194ea8f0e in_seq=0 out_seq=0 2024-02-02T15:37:30.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.777+0000 7f8cafe00700 10 client.18342 ms_handle_connect on v2:172.21.15.142:6838/321675421 2024-02-02T15:37:30.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.778+0000 7f8cafe00700 1 -- 172.21.15.1:0/1693414546 <== mds.1 v2:172.21.15.142:6838/321675421 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55e7939ffdd0 con 0x55e793acfd20 2024-02-02T15:37:30.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.778+0000 7f8cafe00700 10 client.18342 handle_command_reply: tid=0 2024-02-02T15:37:30.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.778+0000 7f8ce0df4b80 1 client.18342 shutdown 2024-02-02T15:37:30.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.778+0000 7f8ce0df4b80 1 -- 172.21.15.1:0/1693414546 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e793ab5010 msgr2=0x55e793ac01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:37:30.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.778+0000 7f8ce0df4b80 1 --2- 172.21.15.1:0/1693414546 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e793ab5010 0x55e793ac01e0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f8cb80098e0 tx=0x7f8cb8004c90 comp rx=0 tx=0).stop 2024-02-02T15:37:30.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.778+0000 7f8ce0df4b80 1 -- 172.21.15.1:0/1693414546 shutdown_connections 2024-02-02T15:37:30.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.778+0000 7f8ce0df4b80 1 --2- 172.21.15.1:0/1693414546 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55e793acfd20 0x55e793ad2140 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.778+0000 7f8ce0df4b80 1 --2- 172.21.15.1:0/1693414546 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e793ac07e0 0x55e793abda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.778+0000 7f8ce0df4b80 1 --2- 172.21.15.1:0/1693414546 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e793ab5010 0x55e793ac01e0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.778+0000 7f8ce0df4b80 1 --2- 172.21.15.1:0/1693414546 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e793ab5930 0x55e793abd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.778+0000 7f8cafe00700 0 client.18342 ms_handle_reset on v2:172.21.15.142:6838/321675421 2024-02-02T15:37:30.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.778+0000 7f8ce0df4b80 1 -- 172.21.15.1:0/1693414546 >> 172.21.15.1:0/1693414546 conn(0x55e793a14db0 msgr2=0x55e793ab0fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:37:30.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.778+0000 7f8ce0df4b80 1 -- 172.21.15.1:0/1693414546 shutdown_connections 2024-02-02T15:37:30.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.778+0000 7f8ce0df4b80 1 -- 172.21.15.1:0/1693414546 wait complete. 2024-02-02T15:37:30.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.779+0000 7f8ce0df4b80 20 client.18342 trim_cache size 0 max 16384 2024-02-02T15:37:30.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.792+0000 7f8ce0df4b80 1 Processor -- start 2024-02-02T15:37:30.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.792+0000 7f8ce0df4b80 1 -- start start 2024-02-02T15:37:30.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.793+0000 7f8ce0df4b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e793ab5010 0x55e793abfd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:30.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.793+0000 7f8ce0df4b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e793ab5930 0x55e793abcbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:30.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.793+0000 7f8ce0df4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e793ac07e0 0x55e793abd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:30.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.793+0000 7f8ce0df4b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e79388a6a0 con 0x55e793ab5010 2024-02-02T15:37:30.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.793+0000 7f8ce0df4b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e793aab560 con 0x55e793ab5930 2024-02-02T15:37:30.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.793+0000 7f8ce0df4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e793abe030 con 0x55e793ac07e0 2024-02-02T15:37:30.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.793+0000 7f8cc5400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e793ab5930 0x55e793abcbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:30.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.793+0000 7f8cc5400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e793ab5930 0x55e793abcbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:52382/0 (socket says 172.21.15.1:52382) 2024-02-02T15:37:30.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.793+0000 7f8cc5400700 1 -- 172.21.15.1:0/2949074774 learned_addr learned my addr 172.21.15.1:0/2949074774 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:37:30.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.793+0000 7f8cc6800700 1 --2- 172.21.15.1:0/2949074774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e793ac07e0 0x55e793abd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:30.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.793+0000 7f8cc5e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e793ab5010 0x55e793abfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:30.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.794+0000 7f8cc5400700 1 -- 172.21.15.1:0/2949074774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e793ac07e0 msgr2=0x55e793abd0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:37:30.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.794+0000 7f8cc5400700 1 --2- 172.21.15.1:0/2949074774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e793ac07e0 0x55e793abd0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.794+0000 7f8cc5400700 1 -- 172.21.15.1:0/2949074774 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e793ab5010 msgr2=0x55e793abfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:37:30.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.794+0000 7f8cc5400700 1 --2- 172.21.15.1:0/2949074774 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e793ab5010 0x55e793abfd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.794+0000 7f8cc5400700 1 -- 172.21.15.1:0/2949074774 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e793834b60 con 0x55e793ab5930 2024-02-02T15:37:30.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.794+0000 7f8cc5400700 1 --2- 172.21.15.1:0/2949074774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e793ab5930 0x55e793abcbb0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f8cb0098cf0 tx=0x7f8cb0000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1b58322209500e4 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:37:30.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.794+0000 7f8cb7e00700 1 -- 172.21.15.1:0/2949074774 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb0014070 con 0x55e793ab5930 2024-02-02T15:37:30.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.794+0000 7f8cb7e00700 1 -- 172.21.15.1:0/2949074774 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8cb009ce20 con 0x55e793ab5930 2024-02-02T15:37:30.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.794+0000 7f8cb7e00700 1 -- 172.21.15.1:0/2949074774 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb0099250 con 0x55e793ab5930 2024-02-02T15:37:30.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.795+0000 7f8ce0df4b80 1 -- 172.21.15.1:0/2949074774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e793ab5930 msgr2=0x55e793abcbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:37:30.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.795+0000 7f8ce0df4b80 1 --2- 172.21.15.1:0/2949074774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e793ab5930 0x55e793abcbb0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f8cb0098cf0 tx=0x7f8cb0000bc0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.795+0000 7f8ce0df4b80 1 -- 172.21.15.1:0/2949074774 shutdown_connections 2024-02-02T15:37:30.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.795+0000 7f8ce0df4b80 1 --2- 172.21.15.1:0/2949074774 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e793ac07e0 0x55e793abd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.795+0000 7f8ce0df4b80 1 --2- 172.21.15.1:0/2949074774 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e793ab5010 0x55e793abfd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.795+0000 7f8ce0df4b80 1 --2- 172.21.15.1:0/2949074774 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e793ab5930 0x55e793abcbb0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.795+0000 7f8ce0df4b80 1 -- 172.21.15.1:0/2949074774 >> 172.21.15.1:0/2949074774 conn(0x55e793a14db0 msgr2=0x55e793acc4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:37:30.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.795+0000 7f8ce0df4b80 1 -- 172.21.15.1:0/2949074774 shutdown_connections 2024-02-02T15:37:30.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.795+0000 7f8ce0df4b80 1 -- 172.21.15.1:0/2949074774 wait complete. 2024-02-02T15:37:30.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.795+0000 7f8ce0df4b80 1 -- start start 2024-02-02T15:37:30.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.796+0000 7f8ce0df4b80 1 Processor -- start 2024-02-02T15:37:30.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.796+0000 7f8ce0df4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e793ab5010 0x55e793ac34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:30.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.796+0000 7f8ce0df4b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e793ab5930 0x55e793ac3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:30.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.796+0000 7f8ce0df4b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e793ac07e0 0x55e793b207f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:30.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.796+0000 7f8ce0df4b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e793b20d30 con 0x55e793ac07e0 2024-02-02T15:37:30.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.796+0000 7f8ce0df4b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e793b20e70 con 0x55e793ab5930 2024-02-02T15:37:30.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.796+0000 7f8ce0df4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e793b20fb0 con 0x55e793ab5010 2024-02-02T15:37:30.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.796+0000 7f8ce0df4b80 10 client.0 _ll_register_callbacks cb 0x55e7939a4290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:37:30.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.796+0000 7f8cc6800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e793ac07e0 0x55e793b207f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:30.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.796+0000 7f8cc5400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e793ab5930 0x55e793ac3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:30.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.796+0000 7f8cc5e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e793ab5010 0x55e793ac34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:30.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.796+0000 7f8cc6800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e793ac07e0 0x55e793b207f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40130/0 (socket says 172.21.15.1:40130) 2024-02-02T15:37:30.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.797+0000 7f8cc6800700 1 -- 172.21.15.1:0/969449219 learned_addr learned my addr 172.21.15.1:0/969449219 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:37:30.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.797+0000 7f8cc6800700 1 -- 172.21.15.1:0/969449219 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e793ab5010 msgr2=0x55e793ac34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:37:30.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.797+0000 7f8cc6800700 1 --2- 172.21.15.1:0/969449219 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e793ab5010 0x55e793ac34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.797+0000 7f8cc6800700 1 -- 172.21.15.1:0/969449219 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e793ab5930 msgr2=0x55e793ac3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:37:30.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.797+0000 7f8cc6800700 1 --2- 172.21.15.1:0/969449219 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e793ab5930 0x55e793ac3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.797+0000 7f8cc6800700 1 -- 172.21.15.1:0/969449219 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cb80095f0 con 0x55e793ac07e0 2024-02-02T15:37:30.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.797+0000 7f8cc5e00700 1 --2- 172.21.15.1:0/969449219 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e793ab5010 0x55e793ac34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:37:30.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.797+0000 7f8cc5400700 1 --2- 172.21.15.1:0/969449219 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e793ab5930 0x55e793ac3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:37:30.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.797+0000 7f8cc6800700 1 --2- 172.21.15.1:0/969449219 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e793ac07e0 0x55e793b207f0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f8cc0004690 tx=0x7f8cc00049a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93ed1179927a9ce6 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:37:30.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.797+0000 7f8cafe00700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:37:30.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.797+0000 7f8cafe00700 1 -- 172.21.15.1:0/969449219 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc001c070 con 0x55e793ac07e0 2024-02-02T15:37:30.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.797+0000 7f8ce0df4b80 1 -- 172.21.15.1:0/969449219 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8cb801e040 con 0x55e793ac07e0 2024-02-02T15:37:30.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.797+0000 7f8cafe00700 1 -- 172.21.15.1:0/969449219 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f8cc0003e50 con 0x55e793ac07e0 2024-02-02T15:37:30.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.798+0000 7f8cafe00700 1 -- 172.21.15.1:0/969449219 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8cc000e040 con 0x55e793ac07e0 2024-02-02T15:37:30.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.798+0000 7f8ce0df4b80 10 client.18348 fetch_fsmap learned FSMap version 49 2024-02-02T15:37:30.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.798+0000 7f8ce0df4b80 1 -- 172.21.15.1:0/969449219 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55e7939ccea0 con 0x55e793ac07e0 2024-02-02T15:37:30.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.798+0000 7f8cafe00700 1 -- 172.21.15.1:0/969449219 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8cb801e040 con 0x55e793ac07e0 2024-02-02T15:37:30.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.798+0000 7f8cafe00700 1 -- 172.21.15.1:0/969449219 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 49) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f8cc0021c40 con 0x55e793ac07e0 2024-02-02T15:37:30.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.798+0000 7f8ce0df4b80 10 client.18348 fetch_fsmap finished waiting for FSMap version 49 2024-02-02T15:37:30.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.798+0000 7f8ce0df4b80 10 client.18348 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-02T15:37:30.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.798+0000 7f8ce0df4b80 20 client.18348 populate_metadata read hostname 'smithi001' 2024-02-02T15:37:30.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.798+0000 7f8ce0df4b80 1 --2- 172.21.15.1:0/969449219 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55e793b22850 0x55e793b24c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:30.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.798+0000 7f8ce0df4b80 4 client.18348 mds_command: new command op to 24529 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:37:30.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.798+0000 7f8ce0df4b80 1 -- 172.21.15.1:0/969449219 --> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e7939c5f50 con 0x55e793b22850 2024-02-02T15:37:30.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.799+0000 7f8cc5e00700 1 --2- 172.21.15.1:0/969449219 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55e793b22850 0x55e793b24c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:30.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.799+0000 7f8cc5e00700 1 --2- 172.21.15.1:0/969449219 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55e793b22850 0x55e793b24c70 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e271ea7d3fef798b server_cookie=e3b48b49357a59ce in_seq=0 out_seq=0 2024-02-02T15:37:30.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.799+0000 7f8cafe00700 10 client.18348 ms_handle_connect on v2:172.21.15.142:6838/321675421 2024-02-02T15:37:30.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.799+0000 7f8cafe00700 1 -- 172.21.15.1:0/969449219 <== mds.1 v2:172.21.15.142:6838/321675421 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e7939c5f50 con 0x55e793b22850 2024-02-02T15:37:30.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.799+0000 7f8cafe00700 10 client.18348 handle_command_reply: tid=0 2024-02-02T15:37:30.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.799+0000 7f8ce0df4b80 1 client.18348 shutdown 2024-02-02T15:37:30.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.799+0000 7f8ce0df4b80 1 -- 172.21.15.1:0/969449219 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e793ac07e0 msgr2=0x55e793b207f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:37:30.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.799+0000 7f8ce0df4b80 1 --2- 172.21.15.1:0/969449219 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e793ac07e0 0x55e793b207f0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f8cc0004690 tx=0x7f8cc00049a0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.799+0000 7f8ce0df4b80 1 -- 172.21.15.1:0/969449219 shutdown_connections 2024-02-02T15:37:30.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.799+0000 7f8ce0df4b80 1 --2- 172.21.15.1:0/969449219 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55e793b22850 0x55e793b24c70 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.799+0000 7f8ce0df4b80 1 --2- 172.21.15.1:0/969449219 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e793ac07e0 0x55e793b207f0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.799+0000 7f8ce0df4b80 1 --2- 172.21.15.1:0/969449219 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e793ab5010 0x55e793ac34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.799+0000 7f8ce0df4b80 1 --2- 172.21.15.1:0/969449219 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e793ab5930 0x55e793ac3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.799+0000 7f8cafe00700 0 client.18348 ms_handle_reset on v2:172.21.15.142:6838/321675421 2024-02-02T15:37:30.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.799+0000 7f8ce0df4b80 1 -- 172.21.15.1:0/969449219 >> 172.21.15.1:0/969449219 conn(0x55e793a14db0 msgr2=0x55e793ab05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:37:30.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.800+0000 7f8ce0df4b80 1 -- 172.21.15.1:0/969449219 shutdown_connections 2024-02-02T15:37:30.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.800+0000 7f8ce0df4b80 1 -- 172.21.15.1:0/969449219 wait complete. 2024-02-02T15:37:30.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.800+0000 7f8ce0df4b80 20 client.18348 trim_cache size 0 max 16384 2024-02-02T15:37:30.805 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:37:30.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.800+0000 7f8ccd200700 1 -- 172.21.15.1:0/2112073456 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f8ca80a0710 msgr2=0x7f8ca80a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:37:30.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.800+0000 7f8ccd200700 1 --2- 172.21.15.1:0/2112073456 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f8ca80a0710 0x7f8ca80a2bd0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f8cb800b2e0 tx=0x7f8cb8005fd0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.800+0000 7f8ccd200700 1 -- 172.21.15.1:0/2112073456 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8cc8073a00 msgr2=0x7f8cc819dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:37:30.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.800+0000 7f8ccd200700 1 --2- 172.21.15.1:0/2112073456 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8cc8073a00 0x7f8cc819dad0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f8cb000d950 tx=0x7f8cb000dc60 comp rx=0 tx=0).stop 2024-02-02T15:37:30.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.800+0000 7f8ccd200700 1 -- 172.21.15.1:0/2112073456 shutdown_connections 2024-02-02T15:37:30.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.800+0000 7f8ccd200700 1 --2- 172.21.15.1:0/2112073456 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f8ca80a0710 0x7f8ca80a2bd0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.800+0000 7f8ccd200700 1 --2- 172.21.15.1:0/2112073456 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8cc8111d90 0x7f8cc81a1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.800+0000 7f8ccd200700 1 --2- 172.21.15.1:0/2112073456 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8cc80730e0 0x7f8cc819d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.800+0000 7f8ccd200700 1 --2- 172.21.15.1:0/2112073456 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8cc8073a00 0x7f8cc819dad0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:30.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.800+0000 7f8ccd200700 1 -- 172.21.15.1:0/2112073456 >> 172.21.15.1:0/2112073456 conn(0x7f8cc80fc810 msgr2=0x7f8cc80fd480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:37:30.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.801+0000 7f8ccd200700 1 -- 172.21.15.1:0/2112073456 shutdown_connections 2024-02-02T15:37:30.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:30.801+0000 7f8ccd200700 1 -- 172.21.15.1:0/2112073456 wait complete. 2024-02-02T15:37:30.819 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-02T15:37:30.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:30 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/80051865' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:37:30.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:30 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/80051865' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:37:31.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.021+0000 7f6c87c00700 1 -- 172.21.15.1:0/3653639099 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6c80100ff0 msgr2=0x7f6c801013d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:37:31.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.021+0000 7f6c87c00700 1 --2- 172.21.15.1:0/3653639099 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6c80100ff0 0x7f6c801013d0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f6c70009990 tx=0x7f6c70009ca0 comp rx=0 tx=0).stop 2024-02-02T15:37:31.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.021+0000 7f6c87c00700 1 -- 172.21.15.1:0/3653639099 shutdown_connections 2024-02-02T15:37:31.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.021+0000 7f6c87c00700 1 --2- 172.21.15.1:0/3653639099 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c80106110 0x7f6c8010dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:31.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.021+0000 7f6c87c00700 1 --2- 172.21.15.1:0/3653639099 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6c80100ff0 0x7f6c801013d0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:31.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.021+0000 7f6c87c00700 1 --2- 172.21.15.1:0/3653639099 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6c801019a0 0x7f6c801059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:31.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.021+0000 7f6c87c00700 1 -- 172.21.15.1:0/3653639099 >> 172.21.15.1:0/3653639099 conn(0x7f6c800fc810 msgr2=0x7f6c800fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:37:31.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.021+0000 7f6c87c00700 1 -- 172.21.15.1:0/3653639099 shutdown_connections 2024-02-02T15:37:31.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.022+0000 7f6c87c00700 1 -- 172.21.15.1:0/3653639099 wait complete. 2024-02-02T15:37:31.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.023+0000 7f6c87c00700 1 Processor -- start 2024-02-02T15:37:31.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.023+0000 7f6c87c00700 1 -- start start 2024-02-02T15:37:31.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.023+0000 7f6c87c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6c80100ff0 0x7f6c80199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:31.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.023+0000 7f6c87c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6c80106110 0x7f6c80199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:31.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.023+0000 7f6c87c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c8019dac0 0x7f6c8019df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:31.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.023+0000 7f6c87c00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8019e540 con 0x7f6c80100ff0 2024-02-02T15:37:31.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.023+0000 7f6c87c00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8019e6b0 con 0x7f6c80106110 2024-02-02T15:37:31.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.023+0000 7f6c87c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8019e870 con 0x7f6c8019dac0 2024-02-02T15:37:31.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.023+0000 7f6c85200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6c80100ff0 0x7f6c80199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:31.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.023+0000 7f6c85200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6c80100ff0 0x7f6c80199120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40150/0 (socket says 172.21.15.1:40150) 2024-02-02T15:37:31.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.023+0000 7f6c85200700 1 -- 172.21.15.1:0/2829202196 learned_addr learned my addr 172.21.15.1:0/2829202196 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:37:31.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.024+0000 7f6c7fe00700 1 --2- 172.21.15.1:0/2829202196 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6c80106110 0x7f6c80199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:31.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.024+0000 7f6c85c00700 1 --2- 172.21.15.1:0/2829202196 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c8019dac0 0x7f6c8019df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:31.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.024+0000 7f6c85200700 1 -- 172.21.15.1:0/2829202196 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c8019dac0 msgr2=0x7f6c8019df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:37:31.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.024+0000 7f6c85200700 1 --2- 172.21.15.1:0/2829202196 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c8019dac0 0x7f6c8019df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:31.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.024+0000 7f6c85200700 1 -- 172.21.15.1:0/2829202196 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6c80106110 msgr2=0x7f6c80199660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:37:31.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.024+0000 7f6c85200700 1 --2- 172.21.15.1:0/2829202196 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6c80106110 0x7f6c80199660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:31.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.024+0000 7f6c85200700 1 -- 172.21.15.1:0/2829202196 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c700095f0 con 0x7f6c80100ff0 2024-02-02T15:37:31.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.024+0000 7f6c85c00700 1 --2- 172.21.15.1:0/2829202196 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c8019dac0 0x7f6c8019df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:37:31.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.024+0000 7f6c85200700 1 --2- 172.21.15.1:0/2829202196 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6c80100ff0 0x7f6c80199120 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f6c70005910 tx=0x7f6c70005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:37:31.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.024+0000 7f6c7d600700 1 -- 172.21.15.1:0/2829202196 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c7001d070 con 0x7f6c80100ff0 2024-02-02T15:37:31.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.024+0000 7f6c7d600700 1 -- 172.21.15.1:0/2829202196 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c70022470 con 0x7f6c80100ff0 2024-02-02T15:37:31.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.024+0000 7f6c7d600700 1 -- 172.21.15.1:0/2829202196 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c70017650 con 0x7f6c80100ff0 2024-02-02T15:37:31.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.024+0000 7f6c87c00700 1 -- 172.21.15.1:0/2829202196 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c801a5d70 con 0x7f6c80100ff0 2024-02-02T15:37:31.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.025+0000 7f6c87c00700 1 -- 172.21.15.1:0/2829202196 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c801a6140 con 0x7f6c80100ff0 2024-02-02T15:37:31.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.027+0000 7f6c7d600700 1 -- 172.21.15.1:0/2829202196 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f6c700225e0 con 0x7f6c80100ff0 2024-02-02T15:37:31.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.029+0000 7f6c7d600700 1 --2- 172.21.15.1:0/2829202196 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f6c600a0690 0x7f6c600a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:31.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.029+0000 7f6c9b81eb80 1 Processor -- start 2024-02-02T15:37:31.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.029+0000 7f6c7d600700 1 -- 172.21.15.1:0/2829202196 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f6c700cd5d0 con 0x7f6c80100ff0 2024-02-02T15:37:31.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.029+0000 7f6c9b81eb80 1 -- start start 2024-02-02T15:37:31.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.029+0000 7f6c7fe00700 1 --2- 172.21.15.1:0/2829202196 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f6c600a0690 0x7f6c600a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:31.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.030+0000 7f6c9b81eb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dad2464010 0x55dad24643f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:31.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.030+0000 7f6c9b81eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dad2464930 0x55dad246f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:31.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.030+0000 7f6c9b81eb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dad246f7e0 0x55dad2471bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:31.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.030+0000 7f6c9b81eb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dad23d84c0 con 0x55dad2464010 2024-02-02T15:37:31.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.030+0000 7f6c9b81eb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dad23bcaa0 con 0x55dad246f7e0 2024-02-02T15:37:31.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.030+0000 7f6c9b81eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dad23ba460 con 0x55dad2464930 2024-02-02T15:37:31.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.030+0000 7f6c7fe00700 1 --2- 172.21.15.1:0/2829202196 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f6c600a0690 0x7f6c600a2b50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6c8019a730 tx=0x7f6c68009260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:37:31.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.030+0000 7f6c7fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dad2464930 0x55dad246f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:31.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.030+0000 7f6c85200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dad2464010 0x55dad24643f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:31.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.030+0000 7f6c7fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dad2464930 0x55dad246f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:42220/0 (socket says 172.21.15.1:42220) 2024-02-02T15:37:31.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.030+0000 7f6c7fe00700 1 -- 172.21.15.1:0/2715106173 learned_addr learned my addr 172.21.15.1:0/2715106173 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:37:31.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.030+0000 7f6c85c00700 1 --2- 172.21.15.1:0/2715106173 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dad246f7e0 0x55dad2471bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:31.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.030+0000 7f6c7fe00700 1 -- 172.21.15.1:0/2715106173 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dad246f7e0 msgr2=0x55dad2471bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:37:31.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.030+0000 7f6c7fe00700 1 --2- 172.21.15.1:0/2715106173 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dad246f7e0 0x55dad2471bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:31.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.030+0000 7f6c7fe00700 1 -- 172.21.15.1:0/2715106173 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dad2464010 msgr2=0x55dad24643f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:37:31.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.030+0000 7f6c7fe00700 1 --2- 172.21.15.1:0/2715106173 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dad2464010 0x55dad24643f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:31.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.030+0000 7f6c7fe00700 1 -- 172.21.15.1:0/2715106173 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dad22373b0 con 0x55dad2464930 2024-02-02T15:37:31.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.030+0000 7f6c85200700 1 --2- 172.21.15.1:0/2715106173 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dad2464010 0x55dad24643f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:37:31.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.031+0000 7f6c7fe00700 1 --2- 172.21.15.1:0/2715106173 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dad2464930 0x55dad246f2a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f6c68011d40 tx=0x7f6c68014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44359ae725c44856 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:37:31.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.031+0000 7f6c76000700 1 -- 172.21.15.1:0/2715106173 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c680099f0 con 0x55dad2464930 2024-02-02T15:37:31.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.031+0000 7f6c76000700 1 -- 172.21.15.1:0/2715106173 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c68021070 con 0x55dad2464930 2024-02-02T15:37:31.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.031+0000 7f6c76000700 1 -- 172.21.15.1:0/2715106173 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c680054d0 con 0x55dad2464930 2024-02-02T15:37:31.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.032+0000 7f6c9b81eb80 1 -- 172.21.15.1:0/2715106173 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dad2464930 msgr2=0x55dad246f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:37:31.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.032+0000 7f6c9b81eb80 1 --2- 172.21.15.1:0/2715106173 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dad2464930 0x55dad246f2a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f6c68011d40 tx=0x7f6c68014000 comp rx=0 tx=0).stop 2024-02-02T15:37:31.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.032+0000 7f6c9b81eb80 1 -- 172.21.15.1:0/2715106173 shutdown_connections 2024-02-02T15:37:31.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.032+0000 7f6c9b81eb80 1 --2- 172.21.15.1:0/2715106173 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dad246f7e0 0x55dad2471bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:31.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.032+0000 7f6c9b81eb80 1 --2- 172.21.15.1:0/2715106173 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dad2464010 0x55dad24643f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:31.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.032+0000 7f6c9b81eb80 1 --2- 172.21.15.1:0/2715106173 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dad2464930 0x55dad246f2a0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:31.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.032+0000 7f6c9b81eb80 1 -- 172.21.15.1:0/2715106173 >> 172.21.15.1:0/2715106173 conn(0x55dad23c3db0 msgr2=0x55dad24603b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:37:31.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.032+0000 7f6c9b81eb80 1 -- 172.21.15.1:0/2715106173 shutdown_connections 2024-02-02T15:37:31.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.032+0000 7f6c9b81eb80 1 -- 172.21.15.1:0/2715106173 wait complete. 2024-02-02T15:37:31.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.033+0000 7f6c9b81eb80 1 -- start start 2024-02-02T15:37:31.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.033+0000 7f6c9b81eb80 1 Processor -- start 2024-02-02T15:37:31.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.033+0000 7f6c9b81eb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dad2464010 0x55dad246f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:31.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.033+0000 7f6c9b81eb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dad2464930 0x55dad246c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:31.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.033+0000 7f6c9b81eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dad246f7e0 0x55dad246cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:31.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.033+0000 7f6c9b81eb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dad23b5d50 con 0x55dad2464930 2024-02-02T15:37:31.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.033+0000 7f6c9b81eb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dad2351d80 con 0x55dad2464010 2024-02-02T15:37:31.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.033+0000 7f6c9b81eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dad2351a20 con 0x55dad246f7e0 2024-02-02T15:37:31.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.033+0000 7f6c9b81eb80 10 client.0 _ll_register_callbacks cb 0x55dad2353290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:37:31.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.033+0000 7f6c7fe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dad2464930 0x55dad246c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:31.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.034+0000 7f6c7fe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dad2464930 0x55dad246c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40170/0 (socket says 172.21.15.1:40170) 2024-02-02T15:37:31.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.034+0000 7f6c7fe00700 1 -- 172.21.15.1:0/2859868578 learned_addr learned my addr 172.21.15.1:0/2859868578 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:37:31.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.034+0000 7f6c85200700 1 --2- 172.21.15.1:0/2859868578 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dad2464010 0x55dad246f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:31.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.034+0000 7f6c85c00700 1 --2- 172.21.15.1:0/2859868578 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dad246f7e0 0x55dad246cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:31.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.034+0000 7f6c7fe00700 1 -- 172.21.15.1:0/2859868578 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dad246f7e0 msgr2=0x55dad246cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:37:31.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.034+0000 7f6c7fe00700 1 --2- 172.21.15.1:0/2859868578 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dad246f7e0 0x55dad246cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:31.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.034+0000 7f6c7fe00700 1 -- 172.21.15.1:0/2859868578 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dad2464010 msgr2=0x55dad246f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:37:31.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.034+0000 7f6c7fe00700 1 --2- 172.21.15.1:0/2859868578 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dad2464010 0x55dad246f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:31.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.034+0000 7f6c7fe00700 1 -- 172.21.15.1:0/2859868578 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c801a6140 con 0x55dad2464930 2024-02-02T15:37:31.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.034+0000 7f6c7fe00700 1 --2- 172.21.15.1:0/2859868578 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dad2464930 0x55dad246c430 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f6c68004d00 tx=0x7f6c680099f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=165cf57c887e0414 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:37:31.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.034+0000 7f6c6ea00700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:37:31.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.034+0000 7f6c6ea00700 1 -- 172.21.15.1:0/2859868578 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c6802b070 con 0x55dad2464930 2024-02-02T15:37:31.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.034+0000 7f6c6ea00700 1 -- 172.21.15.1:0/2859868578 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f6c6800ee30 con 0x55dad2464930 2024-02-02T15:37:31.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.034+0000 7f6c9b81eb80 1 -- 172.21.15.1:0/2859868578 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dad24580f0 con 0x55dad2464930 2024-02-02T15:37:31.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.035+0000 7f6c6ea00700 1 -- 172.21.15.1:0/2859868578 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c68004750 con 0x55dad2464930 2024-02-02T15:37:31.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.035+0000 7f6c9b81eb80 10 client.18360 fetch_fsmap learned FSMap version 49 2024-02-02T15:37:31.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.035+0000 7f6c9b81eb80 1 -- 172.21.15.1:0/2859868578 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x7f6c68017070 con 0x55dad2464930 2024-02-02T15:37:31.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.035+0000 7f6c6ea00700 1 -- 172.21.15.1:0/2859868578 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55dad24580f0 con 0x55dad2464930 2024-02-02T15:37:31.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.035+0000 7f6c6ea00700 1 -- 172.21.15.1:0/2859868578 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 49) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f6c68026780 con 0x55dad2464930 2024-02-02T15:37:31.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.035+0000 7f6c9b81eb80 10 client.18360 fetch_fsmap finished waiting for FSMap version 49 2024-02-02T15:37:31.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.035+0000 7f6c9b81eb80 10 client.18360 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:37:31.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.035+0000 7f6c9b81eb80 20 client.18360 populate_metadata read hostname 'smithi001' 2024-02-02T15:37:31.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.035+0000 7f6c9b81eb80 1 --2- 172.21.15.1:0/2859868578 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55dad247f130 0x55dad2481550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:31.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.035+0000 7f6c9b81eb80 4 client.18360 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:37:31.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.035+0000 7f6c9b81eb80 1 -- 172.21.15.1:0/2859868578 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dad23aedd0 con 0x55dad247f130 2024-02-02T15:37:31.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.036+0000 7f6c85200700 1 --2- 172.21.15.1:0/2859868578 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55dad247f130 0x55dad2481550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:31.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.036+0000 7f6c85200700 1 --2- 172.21.15.1:0/2859868578 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55dad247f130 0x55dad2481550 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcc30c075f8d2613 server_cookie=b6a2e8862816d2e9 in_seq=0 out_seq=0 2024-02-02T15:37:31.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.036+0000 7f6c6ea00700 10 client.18360 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:37:31.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.037+0000 7f6c6ea00700 1 -- 172.21.15.1:0/2859868578 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x7f6c700b2e90 con 0x55dad247f130 2024-02-02T15:37:31.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.037+0000 7f6c6ea00700 10 client.18360 handle_command_reply: tid=0 2024-02-02T15:37:31.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.037+0000 7f6c9b81eb80 1 client.18360 shutdown 2024-02-02T15:37:31.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.037+0000 7f6c9b81eb80 1 -- 172.21.15.1:0/2859868578 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dad2464930 msgr2=0x55dad246c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:37:31.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.037+0000 7f6c9b81eb80 1 --2- 172.21.15.1:0/2859868578 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dad2464930 0x55dad246c430 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f6c68004d00 tx=0x7f6c680099f0 comp rx=0 tx=0).stop 2024-02-02T15:37:31.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.037+0000 7f6c9b81eb80 1 -- 172.21.15.1:0/2859868578 shutdown_connections 2024-02-02T15:37:31.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.037+0000 7f6c9b81eb80 1 --2- 172.21.15.1:0/2859868578 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55dad247f130 0x55dad2481550 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:31.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.037+0000 7f6c9b81eb80 1 --2- 172.21.15.1:0/2859868578 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dad246f7e0 0x55dad246cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:31.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.037+0000 7f6c9b81eb80 1 --2- 172.21.15.1:0/2859868578 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dad2464010 0x55dad246f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:31.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.037+0000 7f6c9b81eb80 1 --2- 172.21.15.1:0/2859868578 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dad2464930 0x55dad246c430 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:31.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.037+0000 7f6c6ea00700 0 client.18360 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:37:31.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.037+0000 7f6c9b81eb80 1 -- 172.21.15.1:0/2859868578 >> 172.21.15.1:0/2859868578 conn(0x55dad23c3db0 msgr2=0x55dad245ffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:37:31.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.037+0000 7f6c9b81eb80 1 -- 172.21.15.1:0/2859868578 shutdown_connections 2024-02-02T15:37:31.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.037+0000 7f6c9b81eb80 1 -- 172.21.15.1:0/2859868578 wait complete. 2024-02-02T15:37:31.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.038+0000 7f6c9b81eb80 20 client.18360 trim_cache size 0 max 16384 2024-02-02T15:37:31.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.051+0000 7f6c9b81eb80 1 Processor -- start 2024-02-02T15:37:31.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.051+0000 7f6c9b81eb80 1 -- start start 2024-02-02T15:37:31.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.051+0000 7f6c9b81eb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dad2464010 0x55dad246ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:31.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.051+0000 7f6c9b81eb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dad2464930 0x55dad2467740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:31.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.051+0000 7f6c9b81eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dad246f7e0 0x55dad2467c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:31.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.051+0000 7f6c9b81eb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dad22396a0 con 0x55dad2464010 2024-02-02T15:37:31.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.051+0000 7f6c9b81eb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dad247b1f0 con 0x55dad2464930 2024-02-02T15:37:31.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.051+0000 7f6c9b81eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dad2485e70 con 0x55dad246f7e0 2024-02-02T15:37:31.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.052+0000 7f6c85200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dad2464010 0x55dad246ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:31.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.052+0000 7f6c85200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dad2464010 0x55dad246ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40172/0 (socket says 172.21.15.1:40172) 2024-02-02T15:37:31.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.052+0000 7f6c85200700 1 -- 172.21.15.1:0/55628986 learned_addr learned my addr 172.21.15.1:0/55628986 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:37:31.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.052+0000 7f6c7fe00700 1 --2- 172.21.15.1:0/55628986 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dad2464930 0x55dad2467740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:31.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.052+0000 7f6c85c00700 1 --2- 172.21.15.1:0/55628986 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dad246f7e0 0x55dad2467c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:31.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.052+0000 7f6c85200700 1 -- 172.21.15.1:0/55628986 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dad246f7e0 msgr2=0x55dad2467c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:37:31.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.052+0000 7f6c85200700 1 --2- 172.21.15.1:0/55628986 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dad246f7e0 0x55dad2467c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:31.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.052+0000 7f6c85200700 1 -- 172.21.15.1:0/55628986 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dad2464930 msgr2=0x55dad2467740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:37:31.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.052+0000 7f6c85200700 1 --2- 172.21.15.1:0/55628986 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dad2464930 0x55dad2467740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:31.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.052+0000 7f6c85200700 1 -- 172.21.15.1:0/55628986 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dad21e3b60 con 0x55dad2464010 2024-02-02T15:37:31.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.052+0000 7f6c85200700 1 --2- 172.21.15.1:0/55628986 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dad2464010 0x55dad246ed50 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f6c7009a480 tx=0x7f6c70022c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c4aaff325a23ea0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:37:31.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.052+0000 7f6c76a00700 1 -- 172.21.15.1:0/55628986 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c700b6070 con 0x55dad2464010 2024-02-02T15:37:31.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.052+0000 7f6c76a00700 1 -- 172.21.15.1:0/55628986 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c700b2e90 con 0x55dad2464010 2024-02-02T15:37:31.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.052+0000 7f6c76a00700 1 -- 172.21.15.1:0/55628986 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c700cd150 con 0x55dad2464010 2024-02-02T15:37:31.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.053+0000 7f6c9b81eb80 1 -- 172.21.15.1:0/55628986 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dad2464010 msgr2=0x55dad246ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:37:31.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.053+0000 7f6c9b81eb80 1 --2- 172.21.15.1:0/55628986 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dad2464010 0x55dad246ed50 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f6c7009a480 tx=0x7f6c70022c60 comp rx=0 tx=0).stop 2024-02-02T15:37:31.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.053+0000 7f6c9b81eb80 1 -- 172.21.15.1:0/55628986 shutdown_connections 2024-02-02T15:37:31.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.053+0000 7f6c9b81eb80 1 --2- 172.21.15.1:0/55628986 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dad246f7e0 0x55dad2467c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:31.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.053+0000 7f6c9b81eb80 1 --2- 172.21.15.1:0/55628986 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dad2464010 0x55dad246ed50 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:31.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.053+0000 7f6c9b81eb80 1 --2- 172.21.15.1:0/55628986 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dad2464930 0x55dad2467740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:31.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.053+0000 7f6c9b81eb80 1 -- 172.21.15.1:0/55628986 >> 172.21.15.1:0/55628986 conn(0x55dad23c3db0 msgr2=0x55dad247b580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:37:31.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.053+0000 7f6c9b81eb80 1 -- 172.21.15.1:0/55628986 shutdown_connections 2024-02-02T15:37:31.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.053+0000 7f6c9b81eb80 1 -- 172.21.15.1:0/55628986 wait complete. 2024-02-02T15:37:31.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.054+0000 7f6c9b81eb80 1 -- start start 2024-02-02T15:37:31.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.054+0000 7f6c9b81eb80 1 Processor -- start 2024-02-02T15:37:31.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.054+0000 7f6c9b81eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dad2464010 0x55dad2475bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:31.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.054+0000 7f6c9b81eb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dad2464930 0x55dad2476100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:31.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.054+0000 7f6c9b81eb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dad246f7e0 0x55dad2474010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:31.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.054+0000 7f6c9b81eb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dad2474720 con 0x55dad246f7e0 2024-02-02T15:37:31.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.054+0000 7f6c9b81eb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dad2474890 con 0x55dad2464930 2024-02-02T15:37:31.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.054+0000 7f6c9b81eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dad2474a50 con 0x55dad2464010 2024-02-02T15:37:31.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.054+0000 7f6c9b81eb80 10 client.0 _ll_register_callbacks cb 0x55dad2353290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:37:31.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.055+0000 7f6c85200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dad2464010 0x55dad2475bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:31.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.055+0000 7f6c85200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dad2464010 0x55dad2475bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:42248/0 (socket says 172.21.15.1:42248) 2024-02-02T15:37:31.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.055+0000 7f6c85200700 1 -- 172.21.15.1:0/2816292709 learned_addr learned my addr 172.21.15.1:0/2816292709 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:37:31.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.055+0000 7f6c85c00700 1 --2- 172.21.15.1:0/2816292709 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dad246f7e0 0x55dad2474010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:31.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.055+0000 7f6c7fe00700 1 --2- 172.21.15.1:0/2816292709 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dad2464930 0x55dad2476100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:31.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.055+0000 7f6c85200700 1 -- 172.21.15.1:0/2816292709 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dad2464930 msgr2=0x55dad2476100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:37:31.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.055+0000 7f6c85200700 1 --2- 172.21.15.1:0/2816292709 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dad2464930 0x55dad2476100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:31.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.055+0000 7f6c85200700 1 -- 172.21.15.1:0/2816292709 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dad246f7e0 msgr2=0x55dad2474010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:37:31.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.055+0000 7f6c85200700 1 --2- 172.21.15.1:0/2816292709 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dad246f7e0 0x55dad2474010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:31.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.055+0000 7f6c85200700 1 -- 172.21.15.1:0/2816292709 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c801a5d70 con 0x55dad2464010 2024-02-02T15:37:31.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.055+0000 7f6c7fe00700 1 --2- 172.21.15.1:0/2816292709 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dad2464930 0x55dad2476100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:37:31.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.055+0000 7f6c85200700 1 --2- 172.21.15.1:0/2816292709 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dad2464010 0x55dad2475bc0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f6c7009a710 tx=0x7f6c7009e680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11b8d1a032b1cc14 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:37:31.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.055+0000 7f6c6ea00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:37:31.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.055+0000 7f6c6ea00700 1 -- 172.21.15.1:0/2816292709 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c700b6030 con 0x55dad2464010 2024-02-02T15:37:31.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.056+0000 7f6c6ea00700 1 -- 172.21.15.1:0/2816292709 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f6c700b7070 con 0x55dad2464010 2024-02-02T15:37:31.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.056+0000 7f6c9b81eb80 1 -- 172.21.15.1:0/2816292709 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6c68021070 con 0x55dad2464010 2024-02-02T15:37:31.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.056+0000 7f6c6ea00700 1 -- 172.21.15.1:0/2816292709 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c700a27e0 con 0x55dad2464010 2024-02-02T15:37:31.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.056+0000 7f6c9b81eb80 10 client.28462 fetch_fsmap learned FSMap version 49 2024-02-02T15:37:31.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.056+0000 7f6c9b81eb80 1 -- 172.21.15.1:0/2816292709 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55dad237bea0 con 0x55dad2464010 2024-02-02T15:37:31.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.056+0000 7f6c6ea00700 1 -- 172.21.15.1:0/2816292709 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6c68021070 con 0x55dad2464010 2024-02-02T15:37:31.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.056+0000 7f6c6ea00700 1 -- 172.21.15.1:0/2816292709 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 49) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f6c70099c70 con 0x55dad2464010 2024-02-02T15:37:31.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.056+0000 7f6c9b81eb80 10 client.28462 fetch_fsmap finished waiting for FSMap version 49 2024-02-02T15:37:31.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.056+0000 7f6c9b81eb80 10 client.28462 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:37:31.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.056+0000 7f6c9b81eb80 20 client.28462 populate_metadata read hostname 'smithi001' 2024-02-02T15:37:31.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.056+0000 7f6c9b81eb80 1 --2- 172.21.15.1:0/2816292709 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55dad24d1990 0x55dad24d3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:31.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.056+0000 7f6c9b81eb80 4 client.28462 mds_command: new command op to 24557 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:37:31.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.056+0000 7f6c9b81eb80 1 -- 172.21.15.1:0/2816292709 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55dad2374f50 con 0x55dad24d1990 2024-02-02T15:37:31.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.057+0000 7f6c7fe00700 1 --2- 172.21.15.1:0/2816292709 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55dad24d1990 0x55dad24d3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:31.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.057+0000 7f6c7fe00700 1 --2- 172.21.15.1:0/2816292709 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55dad24d1990 0x55dad24d3d80 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be3e245d4a952bdc server_cookie=c8bd18ea4a6c5009 in_seq=0 out_seq=0 2024-02-02T15:37:31.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.057+0000 7f6c6ea00700 10 client.28462 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:37:31.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.057+0000 7f6c6ea00700 1 -- 172.21.15.1:0/2816292709 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55dad2374f50 con 0x55dad24d1990 2024-02-02T15:37:31.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.057+0000 7f6c6ea00700 10 client.28462 handle_command_reply: tid=0 2024-02-02T15:37:31.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.057+0000 7f6c9b81eb80 1 client.28462 shutdown 2024-02-02T15:37:31.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.057+0000 7f6c9b81eb80 1 -- 172.21.15.1:0/2816292709 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dad2464010 msgr2=0x55dad2475bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:37:31.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.057+0000 7f6c9b81eb80 1 --2- 172.21.15.1:0/2816292709 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dad2464010 0x55dad2475bc0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f6c7009a710 tx=0x7f6c7009e680 comp rx=0 tx=0).stop 2024-02-02T15:37:31.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.058+0000 7f6c9b81eb80 1 -- 172.21.15.1:0/2816292709 shutdown_connections 2024-02-02T15:37:31.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.058+0000 7f6c9b81eb80 1 --2- 172.21.15.1:0/2816292709 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55dad24d1990 0x55dad24d3d80 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:31.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.058+0000 7f6c9b81eb80 1 --2- 172.21.15.1:0/2816292709 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55dad246f7e0 0x55dad2474010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:31.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.058+0000 7f6c9b81eb80 1 --2- 172.21.15.1:0/2816292709 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55dad2464010 0x55dad2475bc0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:31.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.058+0000 7f6c6ea00700 0 client.28462 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:37:31.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.058+0000 7f6c9b81eb80 1 --2- 172.21.15.1:0/2816292709 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55dad2464930 0x55dad2476100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:31.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.058+0000 7f6c9b81eb80 1 -- 172.21.15.1:0/2816292709 >> 172.21.15.1:0/2816292709 conn(0x55dad23c3db0 msgr2=0x55dad2467740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:37:31.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.058+0000 7f6c9b81eb80 1 -- 172.21.15.1:0/2816292709 shutdown_connections 2024-02-02T15:37:31.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.058+0000 7f6c9b81eb80 1 -- 172.21.15.1:0/2816292709 wait complete. 2024-02-02T15:37:31.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.058+0000 7f6c9b81eb80 20 client.28462 trim_cache size 0 max 16384 2024-02-02T15:37:31.063 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:37:31.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.058+0000 7f6c87c00700 1 -- 172.21.15.1:0/2829202196 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f6c600a0690 msgr2=0x7f6c600a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:37:31.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.058+0000 7f6c87c00700 1 --2- 172.21.15.1:0/2829202196 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f6c600a0690 0x7f6c600a2b50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6c8019a730 tx=0x7f6c68009260 comp rx=0 tx=0).stop 2024-02-02T15:37:31.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.058+0000 7f6c87c00700 1 -- 172.21.15.1:0/2829202196 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6c80100ff0 msgr2=0x7f6c80199120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:37:31.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.058+0000 7f6c87c00700 1 --2- 172.21.15.1:0/2829202196 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6c80100ff0 0x7f6c80199120 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f6c70005910 tx=0x7f6c70005740 comp rx=0 tx=0).stop 2024-02-02T15:37:31.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.059+0000 7f6c87c00700 1 -- 172.21.15.1:0/2829202196 shutdown_connections 2024-02-02T15:37:31.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.059+0000 7f6c87c00700 1 --2- 172.21.15.1:0/2829202196 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f6c600a0690 0x7f6c600a2b50 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:31.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.059+0000 7f6c87c00700 1 --2- 172.21.15.1:0/2829202196 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6c8019dac0 0x7f6c8019df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:31.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.059+0000 7f6c87c00700 1 --2- 172.21.15.1:0/2829202196 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6c80100ff0 0x7f6c80199120 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:31.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.059+0000 7f6c87c00700 1 --2- 172.21.15.1:0/2829202196 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6c80106110 0x7f6c80199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:31.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.059+0000 7f6c87c00700 1 -- 172.21.15.1:0/2829202196 >> 172.21.15.1:0/2829202196 conn(0x7f6c800fc810 msgr2=0x7f6c800fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:37:31.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.059+0000 7f6c87c00700 1 -- 172.21.15.1:0/2829202196 shutdown_connections 2024-02-02T15:37:31.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:31.059+0000 7f6c87c00700 1 -- 172.21.15.1:0/2829202196 wait complete. 2024-02-02T15:37:31.261 INFO:tasks.workunit.client.0.smithi001.stdout: 1 600850 5.88 MB/sec execute 425 sec latency 1925.546 ms 2024-02-02T15:37:31.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:31 smithi071 ceph-mon[44707]: pgmap v326: 129 pgs: 129 active+clean; 472 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.5 MiB/s wr, 124 op/s 2024-02-02T15:37:31.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:31 smithi142 ceph-mon[45728]: pgmap v326: 129 pgs: 129 active+clean; 472 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.5 MiB/s wr, 124 op/s 2024-02-02T15:37:31.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:31 smithi001 ceph-mon[35470]: pgmap v326: 129 pgs: 129 active+clean; 472 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.5 MiB/s wr, 124 op/s 2024-02-02T15:37:32.077 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-02T15:37:32.077 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-02T15:37:32.262 INFO:tasks.workunit.client.0.smithi001.stdout: 1 600850 5.87 MB/sec execute 426 sec latency 2925.610 ms 2024-02-02T15:37:32.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.292+0000 7f4599200700 1 -- 172.21.15.1:0/1505790599 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4594100ff0 msgr2=0x7f45941013d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:37:32.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.292+0000 7f4599200700 1 --2- 172.21.15.1:0/1505790599 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4594100ff0 0x7f45941013d0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f4584009990 tx=0x7f4584009ca0 comp rx=0 tx=0).stop 2024-02-02T15:37:32.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.292+0000 7f4599200700 1 -- 172.21.15.1:0/1505790599 shutdown_connections 2024-02-02T15:37:32.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.292+0000 7f4599200700 1 --2- 172.21.15.1:0/1505790599 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4594106110 0x7f459410dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:32.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.292+0000 7f4599200700 1 --2- 172.21.15.1:0/1505790599 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4594100ff0 0x7f45941013d0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:32.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.292+0000 7f4599200700 1 --2- 172.21.15.1:0/1505790599 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f45941019a0 0x7f45941059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:32.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.292+0000 7f4599200700 1 -- 172.21.15.1:0/1505790599 >> 172.21.15.1:0/1505790599 conn(0x7f45940fc810 msgr2=0x7f45940fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:37:32.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.292+0000 7f4599200700 1 -- 172.21.15.1:0/1505790599 shutdown_connections 2024-02-02T15:37:32.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.292+0000 7f4599200700 1 -- 172.21.15.1:0/1505790599 wait complete. 2024-02-02T15:37:32.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.293+0000 7f4599200700 1 Processor -- start 2024-02-02T15:37:32.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.293+0000 7f4599200700 1 -- start start 2024-02-02T15:37:32.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.294+0000 7f4599200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4594100ff0 0x7f4594199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:32.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.294+0000 7f4599200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4594106110 0x7f4594199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:32.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.294+0000 7f4599200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f459419dac0 0x7f459419df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:32.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.294+0000 7f4599200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f459419e540 con 0x7f459419dac0 2024-02-02T15:37:32.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.294+0000 7f4599200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f459419e6b0 con 0x7f4594106110 2024-02-02T15:37:32.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.294+0000 7f4599200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f459419e870 con 0x7f4594100ff0 2024-02-02T15:37:32.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.294+0000 7f4591e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4594100ff0 0x7f4594199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:32.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.294+0000 7f4591e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4594100ff0 0x7f4594199120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:42260/0 (socket says 172.21.15.1:42260) 2024-02-02T15:37:32.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.294+0000 7f4591e00700 1 -- 172.21.15.1:0/3063191783 learned_addr learned my addr 172.21.15.1:0/3063191783 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:37:32.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.294+0000 7f4592800700 1 --2- 172.21.15.1:0/3063191783 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f459419dac0 0x7f459419df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:32.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.294+0000 7f458be00700 1 --2- 172.21.15.1:0/3063191783 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4594106110 0x7f4594199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:32.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.294+0000 7f4591e00700 1 -- 172.21.15.1:0/3063191783 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4594106110 msgr2=0x7f4594199660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:37:32.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.295+0000 7f4591e00700 1 --2- 172.21.15.1:0/3063191783 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4594106110 0x7f4594199660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:32.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.295+0000 7f4591e00700 1 -- 172.21.15.1:0/3063191783 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f459419dac0 msgr2=0x7f459419df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:37:32.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.295+0000 7f4591e00700 1 --2- 172.21.15.1:0/3063191783 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f459419dac0 0x7f459419df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:32.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.295+0000 7f4591e00700 1 -- 172.21.15.1:0/3063191783 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45840095f0 con 0x7f4594100ff0 2024-02-02T15:37:32.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.295+0000 7f4592800700 1 --2- 172.21.15.1:0/3063191783 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f459419dac0 0x7f459419df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:37:32.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.295+0000 7f4591e00700 1 --2- 172.21.15.1:0/3063191783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4594100ff0 0x7f4594199120 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f4584005910 tx=0x7f4584009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:37:32.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.295+0000 7f458aa00700 1 -- 172.21.15.1:0/3063191783 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f458401d070 con 0x7f4594100ff0 2024-02-02T15:37:32.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.295+0000 7f4599200700 1 -- 172.21.15.1:0/3063191783 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45941a5d70 con 0x7f4594100ff0 2024-02-02T15:37:32.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.295+0000 7f458aa00700 1 -- 172.21.15.1:0/3063191783 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4584003f60 con 0x7f4594100ff0 2024-02-02T15:37:32.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.295+0000 7f458aa00700 1 -- 172.21.15.1:0/3063191783 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4584022bc0 con 0x7f4594100ff0 2024-02-02T15:37:32.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.295+0000 7f4599200700 1 -- 172.21.15.1:0/3063191783 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45941a6190 con 0x7f4594100ff0 2024-02-02T15:37:32.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.298+0000 7f458aa00700 1 -- 172.21.15.1:0/3063191783 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f45840040d0 con 0x7f4594100ff0 2024-02-02T15:37:32.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.299+0000 7f45acce2b80 1 Processor -- start 2024-02-02T15:37:32.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.300+0000 7f45acce2b80 1 -- start start 2024-02-02T15:37:32.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.300+0000 7f458aa00700 1 --2- 172.21.15.1:0/3063191783 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f45740a0690 0x7f45740a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:32.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.300+0000 7f45acce2b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c5d9f3a090 0x55c5d9f3a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:32.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.300+0000 7f45acce2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c5d9f3a9b0 0x55c5d9f45320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:32.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.300+0000 7f45acce2b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5d9f45860 0x55c5d9f47c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:32.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.300+0000 7f458aa00700 1 -- 172.21.15.1:0/3063191783 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f45840cd5a0 con 0x7f4594100ff0 2024-02-02T15:37:32.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.300+0000 7f45acce2b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5d9eae540 con 0x55c5d9f3a090 2024-02-02T15:37:32.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.300+0000 7f45acce2b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5d9e92b20 con 0x55c5d9f45860 2024-02-02T15:37:32.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.300+0000 7f45acce2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5d9e904e0 con 0x55c5d9f3a9b0 2024-02-02T15:37:32.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.300+0000 7f4591e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c5d9f3a090 0x55c5d9f3a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:32.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.300+0000 7f4591e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c5d9f3a090 0x55c5d9f3a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40218/0 (socket says 172.21.15.1:40218) 2024-02-02T15:37:32.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.300+0000 7f4591e00700 1 -- 172.21.15.1:0/1233358196 learned_addr learned my addr 172.21.15.1:0/1233358196 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:37:32.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.300+0000 7f4592800700 1 --2- 172.21.15.1:0/3063191783 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f45740a0690 0x7f45740a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:32.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.300+0000 7f458be00700 1 --2- 172.21.15.1:0/1233358196 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c5d9f3a9b0 0x55c5d9f45320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:32.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.301+0000 7f4592800700 1 --2- 172.21.15.1:0/1233358196 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5d9f45860 0x55c5d9f47c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:32.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.301+0000 7f4591e00700 1 -- 172.21.15.1:0/1233358196 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c5d9f3a9b0 msgr2=0x55c5d9f45320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:37:32.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.301+0000 7f4591e00700 1 --2- 172.21.15.1:0/1233358196 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c5d9f3a9b0 0x55c5d9f45320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:32.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.301+0000 7f4591e00700 1 -- 172.21.15.1:0/1233358196 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5d9f45860 msgr2=0x55c5d9f47c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:37:32.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.301+0000 7f4591e00700 1 --2- 172.21.15.1:0/1233358196 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5d9f45860 0x55c5d9f47c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:32.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.301+0000 7f4591e00700 1 -- 172.21.15.1:0/1233358196 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5d9d0d430 con 0x55c5d9f3a090 2024-02-02T15:37:32.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.301+0000 7f4592800700 1 --2- 172.21.15.1:0/1233358196 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5d9f45860 0x55c5d9f47c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:37:32.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.301+0000 7f4591e00700 1 --2- 172.21.15.1:0/1233358196 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c5d9f3a090 0x55c5d9f3a470 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f45840971c0 tx=0x7f4584005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f813fbf1efb77a8e server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:37:32.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.301+0000 7f4583400700 1 -- 172.21.15.1:0/1233358196 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f458409a0b0 con 0x55c5d9f3a090 2024-02-02T15:37:32.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.302+0000 7f4583400700 1 -- 172.21.15.1:0/1233358196 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f458409ae50 con 0x55c5d9f3a090 2024-02-02T15:37:32.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.302+0000 7f4583400700 1 -- 172.21.15.1:0/1233358196 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45840cab70 con 0x55c5d9f3a090 2024-02-02T15:37:32.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.302+0000 7f45acce2b80 1 -- 172.21.15.1:0/1233358196 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c5d9f3a090 msgr2=0x55c5d9f3a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:37:32.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.302+0000 7f4592800700 1 --2- 172.21.15.1:0/3063191783 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f45740a0690 0x7f45740a2b50 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f458c007790 tx=0x7f458c00b040 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:37:32.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.302+0000 7f45acce2b80 1 --2- 172.21.15.1:0/1233358196 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c5d9f3a090 0x55c5d9f3a470 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f45840971c0 tx=0x7f4584005fd0 comp rx=0 tx=0).stop 2024-02-02T15:37:32.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.302+0000 7f45acce2b80 1 -- 172.21.15.1:0/1233358196 shutdown_connections 2024-02-02T15:37:32.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.302+0000 7f45acce2b80 1 --2- 172.21.15.1:0/1233358196 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5d9f45860 0x55c5d9f47c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:32.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.302+0000 7f45acce2b80 1 --2- 172.21.15.1:0/1233358196 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c5d9f3a090 0x55c5d9f3a470 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:32.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.302+0000 7f45acce2b80 1 --2- 172.21.15.1:0/1233358196 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c5d9f3a9b0 0x55c5d9f45320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:32.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.302+0000 7f45acce2b80 1 -- 172.21.15.1:0/1233358196 >> 172.21.15.1:0/1233358196 conn(0x55c5d9e99e30 msgr2=0x55c5d9f36430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:37:32.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.302+0000 7f45acce2b80 1 -- 172.21.15.1:0/1233358196 shutdown_connections 2024-02-02T15:37:32.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.302+0000 7f45acce2b80 1 -- 172.21.15.1:0/1233358196 wait complete. 2024-02-02T15:37:32.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.303+0000 7f45acce2b80 1 -- start start 2024-02-02T15:37:32.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.303+0000 7f45acce2b80 1 Processor -- start 2024-02-02T15:37:32.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.304+0000 7f45acce2b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c5d9f3a090 0x55c5d9f45c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:32.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.304+0000 7f45acce2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c5d9f3a9b0 0x55c5d9f46180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:32.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.304+0000 7f45acce2b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5d9f45860 0x55c5d9f466c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:32.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.304+0000 7f45acce2b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5d9e8bdd0 con 0x55c5d9f3a090 2024-02-02T15:37:32.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.304+0000 7f45acce2b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5d9e27e00 con 0x55c5d9f45860 2024-02-02T15:37:32.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.304+0000 7f45acce2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5d9e27aa0 con 0x55c5d9f3a9b0 2024-02-02T15:37:32.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.304+0000 7f45acce2b80 10 client.0 _ll_register_callbacks cb 0x55c5d9e29310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:37:32.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.304+0000 7f458be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c5d9f3a9b0 0x55c5d9f46180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:32.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.304+0000 7f4591e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c5d9f3a090 0x55c5d9f45c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:32.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.304+0000 7f458be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c5d9f3a9b0 0x55c5d9f46180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:42272/0 (socket says 172.21.15.1:42272) 2024-02-02T15:37:32.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.304+0000 7f458be00700 1 -- 172.21.15.1:0/1152840121 learned_addr learned my addr 172.21.15.1:0/1152840121 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:37:32.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.304+0000 7f4592800700 1 --2- 172.21.15.1:0/1152840121 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5d9f45860 0x55c5d9f466c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:32.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.304+0000 7f458be00700 1 -- 172.21.15.1:0/1152840121 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5d9f45860 msgr2=0x55c5d9f466c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:37:32.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.304+0000 7f458be00700 1 --2- 172.21.15.1:0/1152840121 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5d9f45860 0x55c5d9f466c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:32.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.304+0000 7f458be00700 1 -- 172.21.15.1:0/1152840121 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c5d9f3a090 msgr2=0x55c5d9f45c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:37:32.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.304+0000 7f458be00700 1 --2- 172.21.15.1:0/1152840121 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c5d9f3a090 0x55c5d9f45c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:32.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.304+0000 7f458be00700 1 -- 172.21.15.1:0/1152840121 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45941a6190 con 0x55c5d9f3a9b0 2024-02-02T15:37:32.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.305+0000 7f458be00700 1 --2- 172.21.15.1:0/1152840121 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c5d9f3a9b0 0x55c5d9f46180 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f457c007f90 tx=0x7f457c011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f70a3717a3317ae server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:37:32.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.305+0000 7f457be00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:37:32.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.305+0000 7f457be00700 1 -- 172.21.15.1:0/1152840121 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f457c016070 con 0x55c5d9f3a9b0 2024-02-02T15:37:32.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.305+0000 7f457be00700 1 -- 172.21.15.1:0/1152840121 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f457c004830 con 0x55c5d9f3a9b0 2024-02-02T15:37:32.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.305+0000 7f45acce2b80 1 -- 172.21.15.1:0/1152840121 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c5d9f2e170 con 0x55c5d9f3a9b0 2024-02-02T15:37:32.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.305+0000 7f457be00700 1 -- 172.21.15.1:0/1152840121 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f457c0049f0 con 0x55c5d9f3a9b0 2024-02-02T15:37:32.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.305+0000 7f45acce2b80 10 client.28474 fetch_fsmap learned FSMap version 49 2024-02-02T15:37:32.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.305+0000 7f45acce2b80 1 -- 172.21.15.1:0/1152840121 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55c5d9cb9be0 con 0x55c5d9f3a9b0 2024-02-02T15:37:32.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.306+0000 7f457be00700 1 -- 172.21.15.1:0/1152840121 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c5d9f2e170 con 0x55c5d9f3a9b0 2024-02-02T15:37:32.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.306+0000 7f457be00700 1 -- 172.21.15.1:0/1152840121 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 49) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f457c011d20 con 0x55c5d9f3a9b0 2024-02-02T15:37:32.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.306+0000 7f45acce2b80 10 client.28474 fetch_fsmap finished waiting for FSMap version 49 2024-02-02T15:37:32.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.306+0000 7f45acce2b80 10 client.28474 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:37:32.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.306+0000 7f45acce2b80 20 client.28474 populate_metadata read hostname 'smithi001' 2024-02-02T15:37:32.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.306+0000 7f45acce2b80 1 --2- 172.21.15.1:0/1152840121 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55c5d9f54e10 0x55c5d9f57230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:32.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.306+0000 7f45acce2b80 4 client.28474 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:37:32.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.306+0000 7f45acce2b80 1 -- 172.21.15.1:0/1152840121 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c5d9e84e50 con 0x55c5d9f54e10 2024-02-02T15:37:32.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.306+0000 7f4591e00700 1 --2- 172.21.15.1:0/1152840121 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55c5d9f54e10 0x55c5d9f57230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:32.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.307+0000 7f4591e00700 1 --2- 172.21.15.1:0/1152840121 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55c5d9f54e10 0x55c5d9f57230 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c97e87747de72459 server_cookie=c98aefc4dade7d2b in_seq=0 out_seq=0 2024-02-02T15:37:32.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.307+0000 7f457be00700 10 client.28474 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:37:32.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.307+0000 7f457be00700 1 -- 172.21.15.1:0/1152840121 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55c5d9e84e50 con 0x55c5d9f54e10 2024-02-02T15:37:32.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.307+0000 7f457be00700 10 client.28474 handle_command_reply: tid=0 2024-02-02T15:37:32.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.308+0000 7f45acce2b80 1 client.28474 shutdown 2024-02-02T15:37:32.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.308+0000 7f45acce2b80 1 -- 172.21.15.1:0/1152840121 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c5d9f3a9b0 msgr2=0x55c5d9f46180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:37:32.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.308+0000 7f45acce2b80 1 --2- 172.21.15.1:0/1152840121 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c5d9f3a9b0 0x55c5d9f46180 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f457c007f90 tx=0x7f457c011720 comp rx=0 tx=0).stop 2024-02-02T15:37:32.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.308+0000 7f45acce2b80 1 -- 172.21.15.1:0/1152840121 shutdown_connections 2024-02-02T15:37:32.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.308+0000 7f45acce2b80 1 --2- 172.21.15.1:0/1152840121 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55c5d9f54e10 0x55c5d9f57230 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:32.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.308+0000 7f45acce2b80 1 --2- 172.21.15.1:0/1152840121 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5d9f45860 0x55c5d9f466c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:32.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.308+0000 7f45acce2b80 1 --2- 172.21.15.1:0/1152840121 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c5d9f3a090 0x55c5d9f45c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:32.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.308+0000 7f457be00700 0 client.28474 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:37:32.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.308+0000 7f45acce2b80 1 --2- 172.21.15.1:0/1152840121 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c5d9f3a9b0 0x55c5d9f46180 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:32.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.308+0000 7f45acce2b80 1 -- 172.21.15.1:0/1152840121 >> 172.21.15.1:0/1152840121 conn(0x55c5d9e99e30 msgr2=0x55c5d9f36020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:37:32.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.308+0000 7f45acce2b80 1 -- 172.21.15.1:0/1152840121 shutdown_connections 2024-02-02T15:37:32.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.308+0000 7f45acce2b80 1 -- 172.21.15.1:0/1152840121 wait complete. 2024-02-02T15:37:32.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.308+0000 7f45acce2b80 20 client.28474 trim_cache size 0 max 16384 2024-02-02T15:37:32.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.325+0000 7f45acce2b80 1 Processor -- start 2024-02-02T15:37:32.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.325+0000 7f45acce2b80 1 -- start start 2024-02-02T15:37:32.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.326+0000 7f45acce2b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5d9f3a090 0x55c5d9f50230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:32.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.326+0000 7f45acce2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c5d9f3a9b0 0x55c5d9f50770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:32.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.326+0000 7f45acce2b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c5d9f45860 0x55c5d9f3eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:32.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.326+0000 7f45acce2b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5d9d0f720 con 0x55c5d9f45860 2024-02-02T15:37:32.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.326+0000 7f45acce2b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5d9f44f00 con 0x55c5d9f3a090 2024-02-02T15:37:32.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.326+0000 7f45acce2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5d9f40740 con 0x55c5d9f3a9b0 2024-02-02T15:37:32.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.326+0000 7f4592800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c5d9f45860 0x55c5d9f3eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:32.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.326+0000 7f4592800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c5d9f45860 0x55c5d9f3eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40222/0 (socket says 172.21.15.1:40222) 2024-02-02T15:37:32.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.326+0000 7f4592800700 1 -- 172.21.15.1:0/1575149748 learned_addr learned my addr 172.21.15.1:0/1575149748 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:37:32.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.326+0000 7f458be00700 1 --2- 172.21.15.1:0/1575149748 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c5d9f3a9b0 0x55c5d9f50770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:32.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.326+0000 7f4591e00700 1 --2- 172.21.15.1:0/1575149748 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5d9f3a090 0x55c5d9f50230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:32.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.326+0000 7f4592800700 1 -- 172.21.15.1:0/1575149748 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c5d9f3a9b0 msgr2=0x55c5d9f50770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:37:32.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.326+0000 7f4592800700 1 --2- 172.21.15.1:0/1575149748 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c5d9f3a9b0 0x55c5d9f50770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:32.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.326+0000 7f4592800700 1 -- 172.21.15.1:0/1575149748 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5d9f3a090 msgr2=0x55c5d9f50230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:37:32.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.326+0000 7f4592800700 1 --2- 172.21.15.1:0/1575149748 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5d9f3a090 0x55c5d9f50230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:32.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.326+0000 7f4592800700 1 -- 172.21.15.1:0/1575149748 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5d9e51f20 con 0x55c5d9f45860 2024-02-02T15:37:32.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.327+0000 7f4592800700 1 --2- 172.21.15.1:0/1575149748 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c5d9f45860 0x55c5d9f3eaf0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f458c017730 tx=0x7f458c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6e0a41710eaa08c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:37:32.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.327+0000 7f4583e00700 1 -- 172.21.15.1:0/1575149748 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f458c027070 con 0x55c5d9f45860 2024-02-02T15:37:32.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.327+0000 7f4583e00700 1 -- 172.21.15.1:0/1575149748 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f458c0238b0 con 0x55c5d9f45860 2024-02-02T15:37:32.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.327+0000 7f4583e00700 1 -- 172.21.15.1:0/1575149748 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f458c0166f0 con 0x55c5d9f45860 2024-02-02T15:37:32.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.327+0000 7f45acce2b80 1 -- 172.21.15.1:0/1575149748 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c5d9f45860 msgr2=0x55c5d9f3eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:37:32.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.327+0000 7f45acce2b80 1 --2- 172.21.15.1:0/1575149748 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c5d9f45860 0x55c5d9f3eaf0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f458c017730 tx=0x7f458c015000 comp rx=0 tx=0).stop 2024-02-02T15:37:32.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.328+0000 7f45acce2b80 1 -- 172.21.15.1:0/1575149748 shutdown_connections 2024-02-02T15:37:32.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.328+0000 7f45acce2b80 1 --2- 172.21.15.1:0/1575149748 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c5d9f45860 0x55c5d9f3eaf0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:32.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.328+0000 7f45acce2b80 1 --2- 172.21.15.1:0/1575149748 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5d9f3a090 0x55c5d9f50230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:32.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.328+0000 7f45acce2b80 1 --2- 172.21.15.1:0/1575149748 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c5d9f3a9b0 0x55c5d9f50770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:32.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.328+0000 7f45acce2b80 1 -- 172.21.15.1:0/1575149748 >> 172.21.15.1:0/1575149748 conn(0x55c5d9e99e30 msgr2=0x55c5d9f464d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:37:32.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.328+0000 7f45acce2b80 1 -- 172.21.15.1:0/1575149748 shutdown_connections 2024-02-02T15:37:32.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.328+0000 7f45acce2b80 1 -- 172.21.15.1:0/1575149748 wait complete. 2024-02-02T15:37:32.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.328+0000 7f45acce2b80 1 -- start start 2024-02-02T15:37:32.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.328+0000 7f45acce2b80 1 Processor -- start 2024-02-02T15:37:32.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.329+0000 7f45acce2b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5d9f3a090 0x55c5d9f493c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:32.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.329+0000 7f45acce2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c5d9f3a9b0 0x55c5d9f49900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:32.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.329+0000 7f45acce2b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c5d9f45860 0x55c5d9fa6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:32.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.329+0000 7f45acce2b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5d9f4a030 con 0x55c5d9f45860 2024-02-02T15:37:32.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.329+0000 7f45acce2b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5d9f4a1a0 con 0x55c5d9f3a090 2024-02-02T15:37:32.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.329+0000 7f45acce2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5d9fa6ea0 con 0x55c5d9f3a9b0 2024-02-02T15:37:32.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.329+0000 7f45acce2b80 10 client.0 _ll_register_callbacks cb 0x55c5d9e29310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:37:32.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.329+0000 7f4592800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c5d9f45860 0x55c5d9fa6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:32.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.329+0000 7f4592800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c5d9f45860 0x55c5d9fa6960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40224/0 (socket says 172.21.15.1:40224) 2024-02-02T15:37:32.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.329+0000 7f4592800700 1 -- 172.21.15.1:0/2478481333 learned_addr learned my addr 172.21.15.1:0/2478481333 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:37:32.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.329+0000 7f458be00700 1 --2- 172.21.15.1:0/2478481333 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c5d9f3a9b0 0x55c5d9f49900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:32.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.329+0000 7f4591e00700 1 --2- 172.21.15.1:0/2478481333 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5d9f3a090 0x55c5d9f493c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:32.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.329+0000 7f4592800700 1 -- 172.21.15.1:0/2478481333 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c5d9f3a9b0 msgr2=0x55c5d9f49900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:37:32.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.329+0000 7f4592800700 1 --2- 172.21.15.1:0/2478481333 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c5d9f3a9b0 0x55c5d9f49900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:32.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.330+0000 7f4592800700 1 -- 172.21.15.1:0/2478481333 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5d9f3a090 msgr2=0x55c5d9f493c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:37:32.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.330+0000 7f4592800700 1 --2- 172.21.15.1:0/2478481333 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5d9f3a090 0x55c5d9f493c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:32.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.330+0000 7f4592800700 1 -- 172.21.15.1:0/2478481333 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f458c011040 con 0x55c5d9f45860 2024-02-02T15:37:32.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.330+0000 7f4592800700 1 --2- 172.21.15.1:0/2478481333 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c5d9f45860 0x55c5d9fa6960 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f458c0174a0 tx=0x7f458c023620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f933cb4f8a193f4 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:37:32.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.330+0000 7f457be00700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:37:32.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.330+0000 7f457be00700 1 -- 172.21.15.1:0/2478481333 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f458c030070 con 0x55c5d9f45860 2024-02-02T15:37:32.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.330+0000 7f457be00700 1 -- 172.21.15.1:0/2478481333 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f458c004ce0 con 0x55c5d9f45860 2024-02-02T15:37:32.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.330+0000 7f45acce2b80 1 -- 172.21.15.1:0/2478481333 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f458409ae50 con 0x55c5d9f45860 2024-02-02T15:37:32.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.330+0000 7f457be00700 1 -- 172.21.15.1:0/2478481333 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f458c027040 con 0x55c5d9f45860 2024-02-02T15:37:32.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.330+0000 7f45acce2b80 10 client.18384 fetch_fsmap learned FSMap version 49 2024-02-02T15:37:32.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.330+0000 7f45acce2b80 1 -- 172.21.15.1:0/2478481333 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55c5d9d11d50 con 0x55c5d9f45860 2024-02-02T15:37:32.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.331+0000 7f457be00700 1 -- 172.21.15.1:0/2478481333 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f458409ae50 con 0x55c5d9f45860 2024-02-02T15:37:32.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.331+0000 7f457be00700 1 -- 172.21.15.1:0/2478481333 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 49) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f458c033020 con 0x55c5d9f45860 2024-02-02T15:37:32.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.331+0000 7f45acce2b80 10 client.18384 fetch_fsmap finished waiting for FSMap version 49 2024-02-02T15:37:32.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.331+0000 7f45acce2b80 10 client.18384 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:37:32.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.331+0000 7f45acce2b80 20 client.18384 populate_metadata read hostname 'smithi001' 2024-02-02T15:37:32.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.331+0000 7f45acce2b80 1 --2- 172.21.15.1:0/2478481333 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55c5d9fa8520 0x55c5d9faa940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:32.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.331+0000 7f45acce2b80 4 client.18384 mds_command: new command op to 24557 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-02T15:37:32.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.331+0000 7f45acce2b80 1 -- 172.21.15.1:0/2478481333 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55c5d9e4afd0 con 0x55c5d9fa8520 2024-02-02T15:37:32.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.331+0000 7f4591e00700 1 --2- 172.21.15.1:0/2478481333 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55c5d9fa8520 0x55c5d9faa940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:32.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.332+0000 7f4591e00700 1 --2- 172.21.15.1:0/2478481333 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55c5d9fa8520 0x55c5d9faa940 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e93116e706ec951b server_cookie=b19062d8b6541f68 in_seq=0 out_seq=0 2024-02-02T15:37:32.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.332+0000 7f457be00700 10 client.18384 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:37:32.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.333+0000 7f457be00700 1 -- 172.21.15.1:0/2478481333 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55c5d9e4afd0 con 0x55c5d9fa8520 2024-02-02T15:37:32.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.333+0000 7f457be00700 10 client.18384 handle_command_reply: tid=0 2024-02-02T15:37:32.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.333+0000 7f45acce2b80 1 client.18384 shutdown 2024-02-02T15:37:32.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.333+0000 7f45acce2b80 1 -- 172.21.15.1:0/2478481333 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c5d9f45860 msgr2=0x55c5d9fa6960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:37:32.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.333+0000 7f45acce2b80 1 --2- 172.21.15.1:0/2478481333 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c5d9f45860 0x55c5d9fa6960 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f458c0174a0 tx=0x7f458c023620 comp rx=0 tx=0).stop 2024-02-02T15:37:32.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.333+0000 7f45acce2b80 1 -- 172.21.15.1:0/2478481333 shutdown_connections 2024-02-02T15:37:32.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.333+0000 7f45acce2b80 1 --2- 172.21.15.1:0/2478481333 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55c5d9fa8520 0x55c5d9faa940 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:32.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.333+0000 7f45acce2b80 1 --2- 172.21.15.1:0/2478481333 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c5d9f45860 0x55c5d9fa6960 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:32.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.334+0000 7f45acce2b80 1 --2- 172.21.15.1:0/2478481333 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5d9f3a090 0x55c5d9f493c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:32.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.334+0000 7f45acce2b80 1 --2- 172.21.15.1:0/2478481333 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c5d9f3a9b0 0x55c5d9f49900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:32.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.334+0000 7f45acce2b80 1 -- 172.21.15.1:0/2478481333 >> 172.21.15.1:0/2478481333 conn(0x55c5d9e99e30 msgr2=0x55c5d9f35640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:37:32.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.334+0000 7f457be00700 0 client.18384 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:37:32.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.334+0000 7f45acce2b80 1 -- 172.21.15.1:0/2478481333 shutdown_connections 2024-02-02T15:37:32.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.334+0000 7f45acce2b80 1 -- 172.21.15.1:0/2478481333 wait complete. 2024-02-02T15:37:32.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.334+0000 7f45acce2b80 20 client.18384 trim_cache size 0 max 16384 2024-02-02T15:37:32.336 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:37:32.336 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2024-02-02T15:37:32.336 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "7d2c1504-792f-497f-9339-c8522116b63e", 2024-02-02T15:37:32.336 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2024-02-02T15:37:32.336 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:37:32.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.334+0000 7f4599200700 1 -- 172.21.15.1:0/3063191783 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f45740a0690 msgr2=0x7f45740a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:37:32.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.334+0000 7f4599200700 1 --2- 172.21.15.1:0/3063191783 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f45740a0690 0x7f45740a2b50 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f458c007790 tx=0x7f458c00b040 comp rx=0 tx=0).stop 2024-02-02T15:37:32.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.334+0000 7f4599200700 1 -- 172.21.15.1:0/3063191783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4594100ff0 msgr2=0x7f4594199120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:37:32.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.334+0000 7f4599200700 1 --2- 172.21.15.1:0/3063191783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4594100ff0 0x7f4594199120 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f4584005910 tx=0x7f4584009580 comp rx=0 tx=0).stop 2024-02-02T15:37:32.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.335+0000 7f4599200700 1 -- 172.21.15.1:0/3063191783 shutdown_connections 2024-02-02T15:37:32.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.335+0000 7f4599200700 1 --2- 172.21.15.1:0/3063191783 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f45740a0690 0x7f45740a2b50 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:32.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.335+0000 7f4599200700 1 --2- 172.21.15.1:0/3063191783 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f459419dac0 0x7f459419df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:32.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.335+0000 7f4599200700 1 --2- 172.21.15.1:0/3063191783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4594100ff0 0x7f4594199120 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:32.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.335+0000 7f4599200700 1 --2- 172.21.15.1:0/3063191783 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4594106110 0x7f4594199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:32.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.335+0000 7f4599200700 1 -- 172.21.15.1:0/3063191783 >> 172.21.15.1:0/3063191783 conn(0x7f45940fc810 msgr2=0x7f45941029b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:37:32.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.335+0000 7f4599200700 1 -- 172.21.15.1:0/3063191783 shutdown_connections 2024-02-02T15:37:32.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.335+0000 7f4599200700 1 -- 172.21.15.1:0/3063191783 wait complete. 2024-02-02T15:37:32.354 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-02T15:37:32.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.556+0000 7f3c3d000700 1 -- 172.21.15.1:0/474171606 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c381019a0 msgr2=0x7f3c381059e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:37:32.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.556+0000 7f3c3d000700 1 --2- 172.21.15.1:0/474171606 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c381019a0 0x7f3c381059e0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f3c20009990 tx=0x7f3c20009ca0 comp rx=0 tx=0).stop 2024-02-02T15:37:32.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.556+0000 7f3c3d000700 1 -- 172.21.15.1:0/474171606 shutdown_connections 2024-02-02T15:37:32.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.556+0000 7f3c3d000700 1 --2- 172.21.15.1:0/474171606 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3c38106110 0x7f3c3810dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:32.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.556+0000 7f3c3d000700 1 --2- 172.21.15.1:0/474171606 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c38100ff0 0x7f3c381013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:32.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.556+0000 7f3c3d000700 1 --2- 172.21.15.1:0/474171606 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c381019a0 0x7f3c381059e0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:32.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.556+0000 7f3c3d000700 1 -- 172.21.15.1:0/474171606 >> 172.21.15.1:0/474171606 conn(0x7f3c380fc810 msgr2=0x7f3c380fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:37:32.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.556+0000 7f3c3d000700 1 -- 172.21.15.1:0/474171606 shutdown_connections 2024-02-02T15:37:32.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.556+0000 7f3c3d000700 1 -- 172.21.15.1:0/474171606 wait complete. 2024-02-02T15:37:32.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.557+0000 7f3c3d000700 1 Processor -- start 2024-02-02T15:37:32.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.557+0000 7f3c3d000700 1 -- start start 2024-02-02T15:37:32.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.558+0000 7f3c3d000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c38100ff0 0x7f3c38199060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:32.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.558+0000 7f3c3d000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c38106110 0x7f3c381995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:32.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.558+0000 7f3c3d000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3c3819da00 0x7f3c3819de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:32.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.558+0000 7f3c3d000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c3819e480 con 0x7f3c3819da00 2024-02-02T15:37:32.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.558+0000 7f3c3d000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c3819e5f0 con 0x7f3c38106110 2024-02-02T15:37:32.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.558+0000 7f3c3d000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c3819e7b0 con 0x7f3c38100ff0 2024-02-02T15:37:32.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.558+0000 7f3c35e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c38100ff0 0x7f3c38199060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:32.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.558+0000 7f3c36800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3c3819da00 0x7f3c3819de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:32.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.558+0000 7f3c2fe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c38106110 0x7f3c381995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:32.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.558+0000 7f3c35e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c38100ff0 0x7f3c38199060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:42302/0 (socket says 172.21.15.1:42302) 2024-02-02T15:37:32.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.558+0000 7f3c35e00700 1 -- 172.21.15.1:0/702052029 learned_addr learned my addr 172.21.15.1:0/702052029 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:37:32.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.558+0000 7f3c2fe00700 1 -- 172.21.15.1:0/702052029 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c38100ff0 msgr2=0x7f3c38199060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:37:32.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.558+0000 7f3c2fe00700 1 --2- 172.21.15.1:0/702052029 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c38100ff0 0x7f3c38199060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:32.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.558+0000 7f3c2fe00700 1 -- 172.21.15.1:0/702052029 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3c3819da00 msgr2=0x7f3c3819de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:37:32.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.558+0000 7f3c2fe00700 1 --2- 172.21.15.1:0/702052029 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3c3819da00 0x7f3c3819de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:32.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.559+0000 7f3c2fe00700 1 -- 172.21.15.1:0/702052029 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c200095f0 con 0x7f3c38106110 2024-02-02T15:37:32.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.559+0000 7f3c36800700 1 --2- 172.21.15.1:0/702052029 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3c3819da00 0x7f3c3819de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:37:32.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.559+0000 7f3c2fe00700 1 --2- 172.21.15.1:0/702052029 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c38106110 0x7f3c381995a0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f3c3806a330 tx=0x7f3c20009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:37:32.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.559+0000 7f3c2ea00700 1 -- 172.21.15.1:0/702052029 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c2001d070 con 0x7f3c38106110 2024-02-02T15:37:32.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.559+0000 7f3c2ea00700 1 -- 172.21.15.1:0/702052029 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3c20021410 con 0x7f3c38106110 2024-02-02T15:37:32.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.559+0000 7f3c3d000700 1 -- 172.21.15.1:0/702052029 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c381a5cb0 con 0x7f3c38106110 2024-02-02T15:37:32.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.559+0000 7f3c2ea00700 1 -- 172.21.15.1:0/702052029 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c20017650 con 0x7f3c38106110 2024-02-02T15:37:32.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.559+0000 7f3c3d000700 1 -- 172.21.15.1:0/702052029 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c381a6130 con 0x7f3c38106110 2024-02-02T15:37:32.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.562+0000 7f3c2ea00700 1 -- 172.21.15.1:0/702052029 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f3c20021810 con 0x7f3c38106110 2024-02-02T15:37:32.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.563+0000 7f3c2ea00700 1 --2- 172.21.15.1:0/702052029 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f3c180a0690 0x7f3c180a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:32.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.563+0000 7f3c2ea00700 1 -- 172.21.15.1:0/702052029 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f3c200cd3a0 con 0x7f3c38106110 2024-02-02T15:37:32.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.564+0000 7f3c50a32b80 1 Processor -- start 2024-02-02T15:37:32.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.564+0000 7f3c35e00700 1 --2- 172.21.15.1:0/702052029 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f3c180a0690 0x7f3c180a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:32.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.564+0000 7f3c50a32b80 1 -- start start 2024-02-02T15:37:32.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.564+0000 7f3c50a32b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f8fed0a010 0x55f8fed0a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:32.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.564+0000 7f3c50a32b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f8fed0a930 0x55f8fed152a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:32.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.564+0000 7f3c50a32b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8fed157e0 0x55f8fed17bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:32.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.564+0000 7f3c50a32b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8fec7e4c0 con 0x55f8fed157e0 2024-02-02T15:37:32.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.564+0000 7f3c50a32b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8fec62aa0 con 0x55f8fed0a010 2024-02-02T15:37:32.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.564+0000 7f3c50a32b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8fec60460 con 0x55f8fed0a930 2024-02-02T15:37:32.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.564+0000 7f3c35e00700 1 --2- 172.21.15.1:0/702052029 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f3c180a0690 0x7f3c180a2b50 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3c28009d20 tx=0x7f3c28009260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:37:32.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.564+0000 7f3c35e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f8fed0a010 0x55f8fed0a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:32.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.565+0000 7f3c35e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f8fed0a010 0x55f8fed0a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:52504/0 (socket says 172.21.15.1:52504) 2024-02-02T15:37:32.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.565+0000 7f3c36800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8fed157e0 0x55f8fed17bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:32.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.565+0000 7f3c35e00700 1 -- 172.21.15.1:0/1751867596 learned_addr learned my addr 172.21.15.1:0/1751867596 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:37:32.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.565+0000 7f3c2fe00700 1 --2- 172.21.15.1:0/1751867596 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f8fed0a930 0x55f8fed152a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:32.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.565+0000 7f3c35e00700 1 -- 172.21.15.1:0/1751867596 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f8fed0a930 msgr2=0x55f8fed152a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:37:32.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.565+0000 7f3c35e00700 1 --2- 172.21.15.1:0/1751867596 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f8fed0a930 0x55f8fed152a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:32.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.565+0000 7f3c35e00700 1 -- 172.21.15.1:0/1751867596 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8fed157e0 msgr2=0x55f8fed17bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:37:32.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.565+0000 7f3c35e00700 1 --2- 172.21.15.1:0/1751867596 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8fed157e0 0x55f8fed17bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:32.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.565+0000 7f3c35e00700 1 -- 172.21.15.1:0/1751867596 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f8feadd3b0 con 0x55f8fed0a010 2024-02-02T15:37:32.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.565+0000 7f3c2fe00700 1 --2- 172.21.15.1:0/1751867596 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f8fed0a930 0x55f8fed152a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:37:32.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.565+0000 7f3c35e00700 1 --2- 172.21.15.1:0/1751867596 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f8fed0a010 0x55f8fed0a3f0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f3c2800e5a0 tx=0x7f3c28014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b11b2cb599be8fd2 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:37:32.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.565+0000 7f3c27400700 1 -- 172.21.15.1:0/1751867596 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c28026430 con 0x55f8fed0a010 2024-02-02T15:37:32.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.566+0000 7f3c27400700 1 -- 172.21.15.1:0/1751867596 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3c28021070 con 0x55f8fed0a010 2024-02-02T15:37:32.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.566+0000 7f3c27400700 1 -- 172.21.15.1:0/1751867596 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c28025790 con 0x55f8fed0a010 2024-02-02T15:37:32.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.566+0000 7f3c50a32b80 1 -- 172.21.15.1:0/1751867596 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f8fed0a010 msgr2=0x55f8fed0a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:37:32.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.566+0000 7f3c50a32b80 1 --2- 172.21.15.1:0/1751867596 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f8fed0a010 0x55f8fed0a3f0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f3c2800e5a0 tx=0x7f3c28014000 comp rx=0 tx=0).stop 2024-02-02T15:37:32.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.566+0000 7f3c50a32b80 1 -- 172.21.15.1:0/1751867596 shutdown_connections 2024-02-02T15:37:32.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.566+0000 7f3c50a32b80 1 --2- 172.21.15.1:0/1751867596 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8fed157e0 0x55f8fed17bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:32.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.566+0000 7f3c50a32b80 1 --2- 172.21.15.1:0/1751867596 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f8fed0a010 0x55f8fed0a3f0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:32.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.566+0000 7f3c50a32b80 1 --2- 172.21.15.1:0/1751867596 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f8fed0a930 0x55f8fed152a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:32.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.566+0000 7f3c50a32b80 1 -- 172.21.15.1:0/1751867596 >> 172.21.15.1:0/1751867596 conn(0x55f8fec69db0 msgr2=0x55f8fed063b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:37:32.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.566+0000 7f3c50a32b80 1 -- 172.21.15.1:0/1751867596 shutdown_connections 2024-02-02T15:37:32.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.566+0000 7f3c50a32b80 1 -- 172.21.15.1:0/1751867596 wait complete. 2024-02-02T15:37:32.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.567+0000 7f3c50a32b80 1 -- start start 2024-02-02T15:37:32.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.567+0000 7f3c50a32b80 1 Processor -- start 2024-02-02T15:37:32.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.568+0000 7f3c50a32b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f8fed0a010 0x55f8fed15250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:32.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.568+0000 7f3c50a32b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8fed0a930 0x55f8fed12460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:32.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.568+0000 7f3c50a32b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f8fed157e0 0x55f8fed12b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:32.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.568+0000 7f3c50a32b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8fec5bd50 con 0x55f8fed0a930 2024-02-02T15:37:32.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.568+0000 7f3c50a32b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8febf7d80 con 0x55f8fed157e0 2024-02-02T15:37:32.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.568+0000 7f3c50a32b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8febf7a20 con 0x55f8fed0a010 2024-02-02T15:37:32.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.568+0000 7f3c50a32b80 10 client.0 _ll_register_callbacks cb 0x55f8febf9290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:37:32.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.568+0000 7f3c2fe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8fed0a930 0x55f8fed12460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:32.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.568+0000 7f3c2fe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8fed0a930 0x55f8fed12460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40252/0 (socket says 172.21.15.1:40252) 2024-02-02T15:37:32.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.568+0000 7f3c2fe00700 1 -- 172.21.15.1:0/3785079691 learned_addr learned my addr 172.21.15.1:0/3785079691 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:37:32.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.568+0000 7f3c36800700 1 --2- 172.21.15.1:0/3785079691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f8fed157e0 0x55f8fed12b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:32.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.568+0000 7f3c35e00700 1 --2- 172.21.15.1:0/3785079691 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f8fed0a010 0x55f8fed15250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:32.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.568+0000 7f3c2fe00700 1 -- 172.21.15.1:0/3785079691 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f8fed0a010 msgr2=0x55f8fed15250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:37:32.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.568+0000 7f3c2fe00700 1 --2- 172.21.15.1:0/3785079691 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f8fed0a010 0x55f8fed15250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:32.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.568+0000 7f3c2fe00700 1 -- 172.21.15.1:0/3785079691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f8fed157e0 msgr2=0x55f8fed12b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:37:32.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.568+0000 7f3c2fe00700 1 --2- 172.21.15.1:0/3785079691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f8fed157e0 0x55f8fed12b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:32.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.568+0000 7f3c2fe00700 1 -- 172.21.15.1:0/3785079691 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c381a6130 con 0x55f8fed0a930 2024-02-02T15:37:32.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.568+0000 7f3c35e00700 1 --2- 172.21.15.1:0/3785079691 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f8fed0a010 0x55f8fed15250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:37:32.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.568+0000 7f3c2fe00700 1 --2- 172.21.15.1:0/3785079691 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8fed0a930 0x55f8fed12460 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f3c200ca750 tx=0x7f3c2009a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2d93bc4dc63da97 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:37:32.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.568+0000 7f3c1fe00700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:37:32.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.569+0000 7f3c1fe00700 1 -- 172.21.15.1:0/3785079691 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c200cd160 con 0x55f8fed0a930 2024-02-02T15:37:32.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.569+0000 7f3c1fe00700 1 -- 172.21.15.1:0/3785079691 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f3c200cae40 con 0x55f8fed0a930 2024-02-02T15:37:32.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.569+0000 7f3c50a32b80 1 -- 172.21.15.1:0/3785079691 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f8fecfe0f0 con 0x55f8fed0a930 2024-02-02T15:37:32.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.569+0000 7f3c1fe00700 1 -- 172.21.15.1:0/3785079691 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3c2009a550 con 0x55f8fed0a930 2024-02-02T15:37:32.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.569+0000 7f3c50a32b80 10 client.18396 fetch_fsmap learned FSMap version 49 2024-02-02T15:37:32.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.569+0000 7f3c50a32b80 1 -- 172.21.15.1:0/3785079691 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x7f3c28017070 con 0x55f8fed0a930 2024-02-02T15:37:32.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.569+0000 7f3c1fe00700 1 -- 172.21.15.1:0/3785079691 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f8fecfe0f0 con 0x55f8fed0a930 2024-02-02T15:37:32.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.569+0000 7f3c1fe00700 1 -- 172.21.15.1:0/3785079691 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 49) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f3c2009a7b0 con 0x55f8fed0a930 2024-02-02T15:37:32.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.569+0000 7f3c50a32b80 10 client.18396 fetch_fsmap finished waiting for FSMap version 49 2024-02-02T15:37:32.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.569+0000 7f3c50a32b80 10 client.18396 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:37:32.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.569+0000 7f3c50a32b80 20 client.18396 populate_metadata read hostname 'smithi001' 2024-02-02T15:37:32.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.569+0000 7f3c50a32b80 1 --2- 172.21.15.1:0/3785079691 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55f8fed24dd0 0x55f8fed271f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:32.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.569+0000 7f3c50a32b80 4 client.18396 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:37:32.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.570+0000 7f3c50a32b80 1 -- 172.21.15.1:0/3785079691 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f8fec54dd0 con 0x55f8fed24dd0 2024-02-02T15:37:32.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.570+0000 7f3c35e00700 1 --2- 172.21.15.1:0/3785079691 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55f8fed24dd0 0x55f8fed271f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:32.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.570+0000 7f3c35e00700 1 --2- 172.21.15.1:0/3785079691 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55f8fed24dd0 0x55f8fed271f0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10a20ac5ba5a804d server_cookie=a79a893e06e91f08 in_seq=0 out_seq=0 2024-02-02T15:37:32.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.570+0000 7f3c1fe00700 10 client.18396 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:37:32.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.571+0000 7f3c1fe00700 1 -- 172.21.15.1:0/3785079691 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55f8fec54dd0 con 0x55f8fed24dd0 2024-02-02T15:37:32.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.571+0000 7f3c1fe00700 10 client.18396 handle_command_reply: tid=0 2024-02-02T15:37:32.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.571+0000 7f3c50a32b80 1 client.18396 shutdown 2024-02-02T15:37:32.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.571+0000 7f3c50a32b80 1 -- 172.21.15.1:0/3785079691 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8fed0a930 msgr2=0x55f8fed12460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:37:32.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.571+0000 7f3c50a32b80 1 --2- 172.21.15.1:0/3785079691 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8fed0a930 0x55f8fed12460 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f3c200ca750 tx=0x7f3c2009a110 comp rx=0 tx=0).stop 2024-02-02T15:37:32.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.571+0000 7f3c50a32b80 1 -- 172.21.15.1:0/3785079691 shutdown_connections 2024-02-02T15:37:32.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.571+0000 7f3c50a32b80 1 --2- 172.21.15.1:0/3785079691 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55f8fed24dd0 0x55f8fed271f0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:32.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.571+0000 7f3c50a32b80 1 --2- 172.21.15.1:0/3785079691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f8fed157e0 0x55f8fed12b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:32.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.571+0000 7f3c50a32b80 1 --2- 172.21.15.1:0/3785079691 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f8fed0a010 0x55f8fed15250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:32.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.571+0000 7f3c1fe00700 0 client.18396 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:37:32.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.571+0000 7f3c50a32b80 1 --2- 172.21.15.1:0/3785079691 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8fed0a930 0x55f8fed12460 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:32.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.571+0000 7f3c50a32b80 1 -- 172.21.15.1:0/3785079691 >> 172.21.15.1:0/3785079691 conn(0x55f8fec69db0 msgr2=0x55f8fed05fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:37:32.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.571+0000 7f3c50a32b80 1 -- 172.21.15.1:0/3785079691 shutdown_connections 2024-02-02T15:37:32.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.571+0000 7f3c50a32b80 1 -- 172.21.15.1:0/3785079691 wait complete. 2024-02-02T15:37:32.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.572+0000 7f3c50a32b80 20 client.18396 trim_cache size 0 max 16384 2024-02-02T15:37:32.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.588+0000 7f3c50a32b80 1 Processor -- start 2024-02-02T15:37:32.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.589+0000 7f3c50a32b80 1 -- start start 2024-02-02T15:37:32.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.589+0000 7f3c50a32b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f8fed0a010 0x55f8fed1c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:32.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.589+0000 7f3c50a32b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f8fed0a930 0x55f8fed14d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:32.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.589+0000 7f3c50a32b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8fed157e0 0x55f8fed0d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:32.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.589+0000 7f3c50a32b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8feadf6a0 con 0x55f8fed157e0 2024-02-02T15:37:32.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.589+0000 7f3c50a32b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8fed13920 con 0x55f8fed0a930 2024-02-02T15:37:32.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.589+0000 7f3c50a32b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8fed0a6b0 con 0x55f8fed0a010 2024-02-02T15:37:32.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.589+0000 7f3c36800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8fed157e0 0x55f8fed0d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:32.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.589+0000 7f3c36800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8fed157e0 0x55f8fed0d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:40264/0 (socket says 172.21.15.1:40264) 2024-02-02T15:37:32.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.589+0000 7f3c36800700 1 -- 172.21.15.1:0/397533251 learned_addr learned my addr 172.21.15.1:0/397533251 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:37:32.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.589+0000 7f3c2fe00700 1 --2- 172.21.15.1:0/397533251 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f8fed0a930 0x55f8fed14d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:32.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.589+0000 7f3c35e00700 1 --2- 172.21.15.1:0/397533251 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f8fed0a010 0x55f8fed1c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:32.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.589+0000 7f3c36800700 1 -- 172.21.15.1:0/397533251 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f8fed0a010 msgr2=0x55f8fed1c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:37:32.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.589+0000 7f3c36800700 1 --2- 172.21.15.1:0/397533251 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f8fed0a010 0x55f8fed1c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:32.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.590+0000 7f3c36800700 1 -- 172.21.15.1:0/397533251 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f8fed0a930 msgr2=0x55f8fed14d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:37:32.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.590+0000 7f3c36800700 1 --2- 172.21.15.1:0/397533251 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f8fed0a930 0x55f8fed14d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:32.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.590+0000 7f3c36800700 1 -- 172.21.15.1:0/397533251 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f8fea89b60 con 0x55f8fed157e0 2024-02-02T15:37:32.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.590+0000 7f3c36800700 1 --2- 172.21.15.1:0/397533251 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8fed157e0 0x55f8fed0d7e0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f3c3000eb70 tx=0x7f3c30009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4485075ae138bc8a server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:37:32.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.590+0000 7f3c27e00700 1 -- 172.21.15.1:0/397533251 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c30017800 con 0x55f8fed157e0 2024-02-02T15:37:32.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.590+0000 7f3c27e00700 1 -- 172.21.15.1:0/397533251 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3c3000f040 con 0x55f8fed157e0 2024-02-02T15:37:32.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.590+0000 7f3c27e00700 1 -- 172.21.15.1:0/397533251 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c30007d60 con 0x55f8fed157e0 2024-02-02T15:37:32.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.591+0000 7f3c50a32b80 1 -- 172.21.15.1:0/397533251 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8fed157e0 msgr2=0x55f8fed0d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:37:32.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.591+0000 7f3c50a32b80 1 --2- 172.21.15.1:0/397533251 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8fed157e0 0x55f8fed0d7e0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f3c3000eb70 tx=0x7f3c30009920 comp rx=0 tx=0).stop 2024-02-02T15:37:32.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.591+0000 7f3c50a32b80 1 -- 172.21.15.1:0/397533251 shutdown_connections 2024-02-02T15:37:32.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.591+0000 7f3c50a32b80 1 --2- 172.21.15.1:0/397533251 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8fed157e0 0x55f8fed0d7e0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:32.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.591+0000 7f3c50a32b80 1 --2- 172.21.15.1:0/397533251 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f8fed0a010 0x55f8fed1c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:32.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.591+0000 7f3c50a32b80 1 --2- 172.21.15.1:0/397533251 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f8fed0a930 0x55f8fed14d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:32.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.591+0000 7f3c50a32b80 1 -- 172.21.15.1:0/397533251 >> 172.21.15.1:0/397533251 conn(0x55f8fec69db0 msgr2=0x55f8fed04b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:37:32.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.591+0000 7f3c50a32b80 1 -- 172.21.15.1:0/397533251 shutdown_connections 2024-02-02T15:37:32.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.591+0000 7f3c50a32b80 1 -- 172.21.15.1:0/397533251 wait complete. 2024-02-02T15:37:32.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.591+0000 7f3c50a32b80 1 -- start start 2024-02-02T15:37:32.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.591+0000 7f3c50a32b80 1 Processor -- start 2024-02-02T15:37:32.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.592+0000 7f3c50a32b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f8fed0a010 0x55f8fed1c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:32.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.592+0000 7f3c50a32b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f8fed0a930 0x55f8fed1a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:32.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.592+0000 7f3c50a32b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8fed157e0 0x55f8fed1a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:32.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.592+0000 7f3c50a32b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8fed1add0 con 0x55f8fed157e0 2024-02-02T15:37:32.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.592+0000 7f3c50a32b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8fed1af10 con 0x55f8fed0a930 2024-02-02T15:37:32.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.592+0000 7f3c50a32b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8fed1b0d0 con 0x55f8fed0a010 2024-02-02T15:37:32.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.592+0000 7f3c50a32b80 10 client.0 _ll_register_callbacks cb 0x55f8febf9290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:37:32.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.592+0000 7f3c35e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f8fed0a010 0x55f8fed1c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:32.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.592+0000 7f3c36800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8fed157e0 0x55f8fed1a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:32.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.592+0000 7f3c2fe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f8fed0a930 0x55f8fed1a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:32.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.592+0000 7f3c35e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f8fed0a010 0x55f8fed1c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:42334/0 (socket says 172.21.15.1:42334) 2024-02-02T15:37:32.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.592+0000 7f3c35e00700 1 -- 172.21.15.1:0/1441953854 learned_addr learned my addr 172.21.15.1:0/1441953854 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:37:32.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.592+0000 7f3c2fe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f8fed0a930 0x55f8fed1a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:52534/0 (socket says 172.21.15.1:52534) 2024-02-02T15:37:32.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.592+0000 7f3c2fe00700 1 -- 172.21.15.1:0/1441953854 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f8fed0a010 msgr2=0x55f8fed1c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:37:32.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.592+0000 7f3c2fe00700 1 --2- 172.21.15.1:0/1441953854 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f8fed0a010 0x55f8fed1c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:32.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.592+0000 7f3c2fe00700 1 -- 172.21.15.1:0/1441953854 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8fed157e0 msgr2=0x55f8fed1a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:37:32.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.592+0000 7f3c2fe00700 1 --2- 172.21.15.1:0/1441953854 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8fed157e0 0x55f8fed1a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:32.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.592+0000 7f3c2fe00700 1 -- 172.21.15.1:0/1441953854 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c3000e820 con 0x55f8fed0a930 2024-02-02T15:37:32.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.592+0000 7f3c35e00700 1 --2- 172.21.15.1:0/1441953854 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f8fed0a010 0x55f8fed1c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:37:32.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.593+0000 7f3c2fe00700 1 --2- 172.21.15.1:0/1441953854 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f8fed0a930 0x55f8fed1a350 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f3c20012040 tx=0x7f3c200ca7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d5792c5ab6cd385 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:37:32.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.593+0000 7f3c1fe00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:37:32.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.593+0000 7f3c1fe00700 1 -- 172.21.15.1:0/1441953854 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c20099eb0 con 0x55f8fed0a930 2024-02-02T15:37:32.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.593+0000 7f3c1fe00700 1 -- 172.21.15.1:0/1441953854 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f3c200cae40 con 0x55f8fed0a930 2024-02-02T15:37:32.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.593+0000 7f3c50a32b80 1 -- 172.21.15.1:0/1441953854 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3c28021070 con 0x55f8fed0a930 2024-02-02T15:37:32.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.593+0000 7f3c1fe00700 1 -- 172.21.15.1:0/1441953854 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3c200cd160 con 0x55f8fed0a930 2024-02-02T15:37:32.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.593+0000 7f3c50a32b80 10 client.28496 fetch_fsmap learned FSMap version 49 2024-02-02T15:37:32.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.593+0000 7f3c50a32b80 1 -- 172.21.15.1:0/1441953854 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x55f8fec21ea0 con 0x55f8fed0a930 2024-02-02T15:37:32.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.593+0000 7f3c1fe00700 1 -- 172.21.15.1:0/1441953854 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3c28021070 con 0x55f8fed0a930 2024-02-02T15:37:32.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.593+0000 7f3c1fe00700 1 -- 172.21.15.1:0/1441953854 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 49) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f3c2009e040 con 0x55f8fed0a930 2024-02-02T15:37:32.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.594+0000 7f3c50a32b80 10 client.28496 fetch_fsmap finished waiting for FSMap version 49 2024-02-02T15:37:32.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.594+0000 7f3c50a32b80 10 client.28496 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:37:32.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.594+0000 7f3c50a32b80 20 client.28496 populate_metadata read hostname 'smithi001' 2024-02-02T15:37:32.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.594+0000 7f3c50a32b80 1 --2- 172.21.15.1:0/1441953854 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55f8fed77890 0x55f8fed79c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:37:32.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.594+0000 7f3c50a32b80 4 client.28496 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2024-02-02T15:37:32.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.594+0000 7f3c50a32b80 1 -- 172.21.15.1:0/1441953854 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f8fec1af50 con 0x55f8fed77890 2024-02-02T15:37:32.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.594+0000 7f3c35e00700 1 --2- 172.21.15.1:0/1441953854 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55f8fed77890 0x55f8fed79c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:37:32.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.594+0000 7f3c1fe00700 10 client.28496 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:37:32.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.594+0000 7f3c35e00700 1 --2- 172.21.15.1:0/1441953854 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55f8fed77890 0x55f8fed79c80 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3000ed5c59bba6c7 server_cookie=3b8513dfcda41f1a in_seq=0 out_seq=0 2024-02-02T15:37:32.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.595+0000 7f3c1fe00700 1 -- 172.21.15.1:0/1441953854 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55f8fec1af50 con 0x55f8fed77890 2024-02-02T15:37:32.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.595+0000 7f3c1fe00700 10 client.28496 handle_command_reply: tid=0 2024-02-02T15:37:32.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.595+0000 7f3c50a32b80 1 client.28496 shutdown 2024-02-02T15:37:32.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.595+0000 7f3c50a32b80 1 -- 172.21.15.1:0/1441953854 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f8fed0a930 msgr2=0x55f8fed1a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:37:32.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.595+0000 7f3c50a32b80 1 --2- 172.21.15.1:0/1441953854 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f8fed0a930 0x55f8fed1a350 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f3c20012040 tx=0x7f3c200ca7d0 comp rx=0 tx=0).stop 2024-02-02T15:37:32.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.595+0000 7f3c50a32b80 1 -- 172.21.15.1:0/1441953854 shutdown_connections 2024-02-02T15:37:32.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.595+0000 7f3c50a32b80 1 --2- 172.21.15.1:0/1441953854 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55f8fed77890 0x55f8fed79c80 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:32.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.595+0000 7f3c50a32b80 1 --2- 172.21.15.1:0/1441953854 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8fed157e0 0x55f8fed1a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:32.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.595+0000 7f3c50a32b80 1 --2- 172.21.15.1:0/1441953854 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f8fed0a010 0x55f8fed1c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:32.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.595+0000 7f3c50a32b80 1 --2- 172.21.15.1:0/1441953854 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f8fed0a930 0x55f8fed1a350 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:32.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.595+0000 7f3c1fe00700 0 client.28496 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:37:32.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.595+0000 7f3c50a32b80 1 -- 172.21.15.1:0/1441953854 >> 172.21.15.1:0/1441953854 conn(0x55f8fec69db0 msgr2=0x55f8fed0d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:37:32.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.595+0000 7f3c50a32b80 1 -- 172.21.15.1:0/1441953854 shutdown_connections 2024-02-02T15:37:32.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.595+0000 7f3c50a32b80 1 -- 172.21.15.1:0/1441953854 wait complete. 2024-02-02T15:37:32.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.596+0000 7f3c50a32b80 20 client.28496 trim_cache size 0 max 16384 2024-02-02T15:37:32.600 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:37:32.600 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "scrub active (1 inodes in the stack)", 2024-02-02T15:37:32.600 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2024-02-02T15:37:32.600 INFO:teuthology.orchestra.run.smithi001.stdout: "7d2c1504-792f-497f-9339-c8522116b63e": { 2024-02-02T15:37:32.600 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2024-02-02T15:37:32.600 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "7d2c1504-792f-497f-9339-c8522116b63e", 2024-02-02T15:37:32.600 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,force" 2024-02-02T15:37:32.600 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:37:32.600 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:37:32.600 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:37:32.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.596+0000 7f3c3d000700 1 -- 172.21.15.1:0/702052029 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f3c180a0690 msgr2=0x7f3c180a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:37:32.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.596+0000 7f3c3d000700 1 --2- 172.21.15.1:0/702052029 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f3c180a0690 0x7f3c180a2b50 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3c28009d20 tx=0x7f3c28009260 comp rx=0 tx=0).stop 2024-02-02T15:37:32.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.596+0000 7f3c3d000700 1 -- 172.21.15.1:0/702052029 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c38106110 msgr2=0x7f3c381995a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:37:32.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.596+0000 7f3c3d000700 1 --2- 172.21.15.1:0/702052029 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c38106110 0x7f3c381995a0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f3c3806a330 tx=0x7f3c20009580 comp rx=0 tx=0).stop 2024-02-02T15:37:32.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.596+0000 7f3c3d000700 1 -- 172.21.15.1:0/702052029 shutdown_connections 2024-02-02T15:37:32.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.596+0000 7f3c3d000700 1 --2- 172.21.15.1:0/702052029 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f3c180a0690 0x7f3c180a2b50 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:32.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.596+0000 7f3c3d000700 1 --2- 172.21.15.1:0/702052029 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3c3819da00 0x7f3c3819de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:32.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.596+0000 7f3c3d000700 1 --2- 172.21.15.1:0/702052029 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3c38100ff0 0x7f3c38199060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:32.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.596+0000 7f3c3d000700 1 --2- 172.21.15.1:0/702052029 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c38106110 0x7f3c381995a0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:37:32.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.596+0000 7f3c3d000700 1 -- 172.21.15.1:0/702052029 >> 172.21.15.1:0/702052029 conn(0x7f3c380fc810 msgr2=0x7f3c380fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:37:32.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.596+0000 7f3c3d000700 1 -- 172.21.15.1:0/702052029 shutdown_connections 2024-02-02T15:37:32.602 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:37:32.596+0000 7f3c3d000700 1 -- 172.21.15.1:0/702052029 wait complete. 2024-02-02T15:37:32.614 INFO:tasks.cephfs.filesystem:scrub status for tag:7d2c1504-792f-497f-9339-c8522116b63e - {'path': '/', 'tag': '7d2c1504-792f-497f-9339-c8522116b63e', 'options': 'recursive,force'} 2024-02-02T15:37:33.262 INFO:tasks.workunit.client.0.smithi001.stdout: 1 600850 5.85 MB/sec execute 427 sec latency 3925.721 ms 2024-02-02T15:37:33.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:33 smithi071 ceph-mon[44707]: pgmap v327: 129 pgs: 129 active+clean; 477 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.0 MiB/s wr, 168 op/s 2024-02-02T15:37:33.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:33 smithi071 ceph-mon[44707]: scrub queued for path: / 2024-02-02T15:37:33.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:33 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:37:33.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:33 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:37:33.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:33 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:37:33.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:33 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:37:33.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:33 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:37:33.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:33 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:37:33.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:33 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:37:33.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:33 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:37:33.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:33 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:37:33.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:33 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:37:33.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:33 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:37:33.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:33 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:37:33.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:33 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:37:33.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:33 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:37:33.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:33 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:37:33.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:33 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:37:33.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:33 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:37:33.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:33 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:37:33.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:33 smithi142 ceph-mon[45728]: pgmap v327: 129 pgs: 129 active+clean; 477 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.0 MiB/s wr, 168 op/s 2024-02-02T15:37:33.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:33 smithi142 ceph-mon[45728]: scrub queued for path: / 2024-02-02T15:37:33.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:33 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:37:33.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:33 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:37:33.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:33 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:37:33.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:33 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:37:33.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:33 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:37:33.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:33 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:37:33.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:33 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:37:33.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:33 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:37:33.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:33 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:37:33.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:33 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:37:33.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:33 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:37:33.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:33 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:37:33.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:33 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:37:33.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:33 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:37:33.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:33 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:37:33.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:33 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:37:33.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:33 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:37:33.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:33 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:37:33.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:33 smithi001 ceph-mon[35470]: pgmap v327: 129 pgs: 129 active+clean; 477 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.0 MiB/s wr, 168 op/s 2024-02-02T15:37:33.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:33 smithi001 ceph-mon[35470]: scrub queued for path: / 2024-02-02T15:37:33.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:33 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:37:33.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:33 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:37:33.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:33 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:37:33.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:33 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:37:33.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:33 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:37:33.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:33 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:37:33.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:33 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:37:33.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:33 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:37:33.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:33 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:37:33.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:33 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:37:33.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:33 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:37:33.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:33 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:37:33.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:33 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:37:33.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:33 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:37:33.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:33 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:37:33.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:33 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:37:33.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:33 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:37:33.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:33 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:37:34.262 INFO:tasks.workunit.client.0.smithi001.stdout: 1 600850 5.84 MB/sec execute 428 sec latency 4925.810 ms 2024-02-02T15:37:35.262 INFO:tasks.workunit.client.0.smithi001.stdout: 1 600850 5.83 MB/sec execute 429 sec latency 5925.889 ms 2024-02-02T15:37:35.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:35 smithi071 ceph-mon[44707]: pgmap v328: 129 pgs: 129 active+clean; 477 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 969 KiB/s rd, 1.3 MiB/s wr, 102 op/s 2024-02-02T15:37:35.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:35 smithi142 ceph-mon[45728]: pgmap v328: 129 pgs: 129 active+clean; 477 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 969 KiB/s rd, 1.3 MiB/s wr, 102 op/s 2024-02-02T15:37:35.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:35 smithi001 ceph-mon[35470]: pgmap v328: 129 pgs: 129 active+clean; 477 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 969 KiB/s rd, 1.3 MiB/s wr, 102 op/s 2024-02-02T15:37:36.262 INFO:tasks.workunit.client.0.smithi001.stdout: 1 600850 5.81 MB/sec execute 430 sec latency 6925.969 ms 2024-02-02T15:37:37.262 INFO:tasks.workunit.client.0.smithi001.stdout: 1 600850 5.80 MB/sec execute 431 sec latency 7926.093 ms 2024-02-02T15:37:37.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:37 smithi071 ceph-mon[44707]: pgmap v329: 129 pgs: 129 active+clean; 478 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1016 KiB/s rd, 1.6 MiB/s wr, 120 op/s 2024-02-02T15:37:37.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:37 smithi142 ceph-mon[45728]: pgmap v329: 129 pgs: 129 active+clean; 478 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1016 KiB/s rd, 1.6 MiB/s wr, 120 op/s 2024-02-02T15:37:37.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:37 smithi001 ceph-mon[35470]: pgmap v329: 129 pgs: 129 active+clean; 478 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1016 KiB/s rd, 1.6 MiB/s wr, 120 op/s 2024-02-02T15:37:38.262 INFO:tasks.workunit.client.0.smithi001.stdout: 1 600850 5.79 MB/sec execute 432 sec latency 8926.210 ms 2024-02-02T15:37:39.263 INFO:tasks.workunit.client.0.smithi001.stdout: 1 600850 5.77 MB/sec execute 433 sec latency 9926.375 ms 2024-02-02T15:37:39.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:39 smithi071 ceph-mon[44707]: pgmap v330: 129 pgs: 129 active+clean; 478 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 852 KiB/s wr, 61 op/s 2024-02-02T15:37:39.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:39 smithi142 ceph-mon[45728]: pgmap v330: 129 pgs: 129 active+clean; 478 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 852 KiB/s wr, 61 op/s 2024-02-02T15:37:39.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:39 smithi001 ceph-mon[35470]: pgmap v330: 129 pgs: 129 active+clean; 478 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 852 KiB/s wr, 61 op/s 2024-02-02T15:37:40.262 INFO:tasks.workunit.client.0.smithi001.stdout: 1 601778 5.77 MB/sec execute 434 sec latency 10206.462 ms 2024-02-02T15:37:40.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:40 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:37:40.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:40 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:37:40.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:40 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:37:40.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:40 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:37:40.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:40 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:37:40.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:40 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:37:41.263 INFO:tasks.workunit.client.0.smithi001.stdout: 1 601947 5.76 MB/sec execute 435 sec latency 855.017 ms 2024-02-02T15:37:41.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:41 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:37:41.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:41 smithi071 ceph-mon[44707]: pgmap v331: 129 pgs: 129 active+clean; 478 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 852 KiB/s wr, 61 op/s 2024-02-02T15:37:41.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:41 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:37:41.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:41 smithi142 ceph-mon[45728]: pgmap v331: 129 pgs: 129 active+clean; 478 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 852 KiB/s wr, 61 op/s 2024-02-02T15:37:41.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:41 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:37:41.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:41 smithi001 ceph-mon[35470]: pgmap v331: 129 pgs: 129 active+clean; 478 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 852 KiB/s wr, 61 op/s 2024-02-02T15:37:42.263 INFO:tasks.workunit.client.0.smithi001.stdout: 1 601947 5.75 MB/sec execute 436 sec latency 1855.178 ms 2024-02-02T15:37:43.263 INFO:tasks.workunit.client.0.smithi001.stdout: 1 603153 5.75 MB/sec execute 437 sec latency 2237.792 ms 2024-02-02T15:37:43.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:43 smithi071 ceph-mon[44707]: pgmap v332: 129 pgs: 129 active+clean; 476 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 1.3 MiB/s wr, 101 op/s 2024-02-02T15:37:43.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:43 smithi142 ceph-mon[45728]: pgmap v332: 129 pgs: 129 active+clean; 476 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 1.3 MiB/s wr, 101 op/s 2024-02-02T15:37:43.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:43 smithi001 ceph-mon[35470]: pgmap v332: 129 pgs: 129 active+clean; 476 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 1.3 MiB/s wr, 101 op/s 2024-02-02T15:37:44.263 INFO:tasks.workunit.client.0.smithi001.stdout: 1 604360 5.75 MB/sec execute 438 sec latency 12.299 ms 2024-02-02T15:37:45.263 INFO:tasks.workunit.client.0.smithi001.stdout: 1 605471 5.75 MB/sec execute 439 sec latency 71.204 ms 2024-02-02T15:37:45.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:45 smithi071 ceph-mon[44707]: pgmap v333: 129 pgs: 129 active+clean; 476 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 735 KiB/s wr, 57 op/s 2024-02-02T15:37:45.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:45 smithi142 ceph-mon[45728]: pgmap v333: 129 pgs: 129 active+clean; 476 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 735 KiB/s wr, 57 op/s 2024-02-02T15:37:45.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:45 smithi001 ceph-mon[35470]: pgmap v333: 129 pgs: 129 active+clean; 476 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 735 KiB/s wr, 57 op/s 2024-02-02T15:37:46.263 INFO:tasks.workunit.client.0.smithi001.stdout: 1 606234 5.74 MB/sec execute 440 sec latency 652.911 ms 2024-02-02T15:37:47.263 INFO:tasks.workunit.client.0.smithi001.stdout: 1 607716 5.76 MB/sec execute 441 sec latency 14.547 ms 2024-02-02T15:37:47.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:47 smithi071 ceph-mon[44707]: pgmap v334: 129 pgs: 129 active+clean; 481 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 122 op/s 2024-02-02T15:37:47.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:47 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:37:47.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:47 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:37:47.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:47 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:37:47.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:47 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:37:47.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:47 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:37:47.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:47 smithi142 ceph-mon[45728]: pgmap v334: 129 pgs: 129 active+clean; 481 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 122 op/s 2024-02-02T15:37:47.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:47 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:37:47.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:47 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:37:47.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:47 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:37:47.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:47 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:37:47.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:47 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:37:47.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:47 smithi001 ceph-mon[35470]: pgmap v334: 129 pgs: 129 active+clean; 481 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 122 op/s 2024-02-02T15:37:47.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:47 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:37:47.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:47 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:37:47.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:47 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:37:47.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:47 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:37:47.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:47 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:37:48.263 INFO:tasks.workunit.client.0.smithi001.stdout: 1 608812 5.75 MB/sec execute 442 sec latency 19.045 ms 2024-02-02T15:37:49.264 INFO:tasks.workunit.client.0.smithi001.stdout: 1 609036 5.74 MB/sec execute 443 sec latency 764.723 ms 2024-02-02T15:37:49.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:49 smithi071 ceph-mon[44707]: pgmap v335: 129 pgs: 129 active+clean; 481 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 104 op/s 2024-02-02T15:37:49.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:49 smithi142 ceph-mon[45728]: pgmap v335: 129 pgs: 129 active+clean; 481 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 104 op/s 2024-02-02T15:37:49.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:49 smithi001 ceph-mon[35470]: pgmap v335: 129 pgs: 129 active+clean; 481 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 104 op/s 2024-02-02T15:37:50.263 INFO:tasks.workunit.client.0.smithi001.stdout: 1 609036 5.73 MB/sec execute 444 sec latency 1764.864 ms 2024-02-02T15:37:51.264 INFO:tasks.workunit.client.0.smithi001.stdout: 1 609036 5.72 MB/sec execute 445 sec latency 2765.042 ms 2024-02-02T15:37:51.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:51 smithi071 ceph-mon[44707]: pgmap v336: 129 pgs: 129 active+clean; 481 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 104 op/s 2024-02-02T15:37:51.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:51 smithi071 ceph-mon[44707]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:37:51.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:51 smithi142 ceph-mon[45728]: pgmap v336: 129 pgs: 129 active+clean; 481 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 104 op/s 2024-02-02T15:37:51.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:51 smithi142 ceph-mon[45728]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:37:51.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:51 smithi001 ceph-mon[35470]: pgmap v336: 129 pgs: 129 active+clean; 481 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 104 op/s 2024-02-02T15:37:51.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:51 smithi001 ceph-mon[35470]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:37:52.264 INFO:tasks.workunit.client.0.smithi001.stdout: 1 609036 5.70 MB/sec execute 446 sec latency 3765.121 ms 2024-02-02T15:37:52.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:52 smithi071 ceph-mon[44707]: pgmap v337: 129 pgs: 129 active+clean; 488 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 151 op/s 2024-02-02T15:37:52.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:52 smithi142 ceph-mon[45728]: pgmap v337: 129 pgs: 129 active+clean; 488 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 151 op/s 2024-02-02T15:37:52.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:52 smithi001 ceph-mon[35470]: pgmap v337: 129 pgs: 129 active+clean; 488 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 151 op/s 2024-02-02T15:37:53.264 INFO:tasks.workunit.client.0.smithi001.stdout: 1 610260 5.71 MB/sec execute 447 sec latency 4148.081 ms 2024-02-02T15:37:54.264 INFO:tasks.workunit.client.0.smithi001.stdout: 1 611476 5.71 MB/sec execute 448 sec latency 11.054 ms 2024-02-02T15:37:55.264 INFO:tasks.workunit.client.0.smithi001.stdout: 1 612586 5.71 MB/sec execute 449 sec latency 106.081 ms 2024-02-02T15:37:55.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:55 smithi142 ceph-mon[45728]: pgmap v338: 129 pgs: 129 active+clean; 488 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 111 op/s 2024-02-02T15:37:55.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:55 smithi001 ceph-mon[35470]: pgmap v338: 129 pgs: 129 active+clean; 488 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 111 op/s 2024-02-02T15:37:55.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:55 smithi071 ceph-mon[44707]: pgmap v338: 129 pgs: 129 active+clean; 488 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 111 op/s 2024-02-02T15:37:56.264 INFO:tasks.workunit.client.0.smithi001.stdout: 1 612586 5.69 MB/sec execute 450 sec latency 1106.143 ms 2024-02-02T15:37:57.264 INFO:tasks.workunit.client.0.smithi001.stdout: 1 612586 5.68 MB/sec execute 451 sec latency 2106.222 ms 2024-02-02T15:37:57.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:57 smithi001 ceph-mon[35470]: pgmap v339: 129 pgs: 129 active+clean; 489 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 176 op/s 2024-02-02T15:37:57.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:57 smithi071 ceph-mon[44707]: pgmap v339: 129 pgs: 129 active+clean; 489 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 176 op/s 2024-02-02T15:37:57.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:57 smithi142 ceph-mon[45728]: pgmap v339: 129 pgs: 129 active+clean; 489 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 176 op/s 2024-02-02T15:37:58.264 INFO:tasks.workunit.client.0.smithi001.stdout: 1 613807 5.69 MB/sec execute 452 sec latency 2489.189 ms 2024-02-02T15:37:59.264 INFO:tasks.workunit.client.0.smithi001.stdout: 1 615031 5.69 MB/sec execute 453 sec latency 11.499 ms 2024-02-02T15:37:59.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:37:59 smithi001 ceph-mon[35470]: pgmap v340: 129 pgs: 129 active+clean; 489 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 111 op/s 2024-02-02T15:37:59.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:37:59 smithi071 ceph-mon[44707]: pgmap v340: 129 pgs: 129 active+clean; 489 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 111 op/s 2024-02-02T15:37:59.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:37:59 smithi142 ceph-mon[45728]: pgmap v340: 129 pgs: 129 active+clean; 489 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 111 op/s 2024-02-02T15:38:00.264 INFO:tasks.workunit.client.0.smithi001.stdout: 1 616129 5.69 MB/sec execute 454 sec latency 204.317 ms 2024-02-02T15:38:01.264 INFO:tasks.workunit.client.0.smithi001.stdout: 1 616129 5.67 MB/sec execute 455 sec latency 1204.420 ms 2024-02-02T15:38:01.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:01 smithi001 ceph-mon[35470]: pgmap v341: 129 pgs: 129 active+clean; 489 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 111 op/s 2024-02-02T15:38:01.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:01 smithi071 ceph-mon[44707]: pgmap v341: 129 pgs: 129 active+clean; 489 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 111 op/s 2024-02-02T15:38:01.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:01 smithi142 ceph-mon[45728]: pgmap v341: 129 pgs: 129 active+clean; 489 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.3 MiB/s wr, 111 op/s 2024-02-02T15:38:02.264 INFO:tasks.workunit.client.0.smithi001.stdout: 1 616129 5.66 MB/sec execute 456 sec latency 2204.518 ms 2024-02-02T15:38:02.616 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-02T15:38:02.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.874+0000 7fda27e00700 1 -- 172.21.15.1:0/3874585834 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda201019b0 msgr2=0x7fda201059f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:02.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.874+0000 7fda27e00700 1 --2- 172.21.15.1:0/3874585834 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda201019b0 0x7fda201059f0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fda08009990 tx=0x7fda08009ca0 comp rx=0 tx=0).stop 2024-02-02T15:38:02.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.874+0000 7fda27e00700 1 -- 172.21.15.1:0/3874585834 shutdown_connections 2024-02-02T15:38:02.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.874+0000 7fda27e00700 1 --2- 172.21.15.1:0/3874585834 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fda20106120 0x7fda2010dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:02.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.874+0000 7fda27e00700 1 --2- 172.21.15.1:0/3874585834 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda20101000 0x7fda201013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:02.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.874+0000 7fda27e00700 1 --2- 172.21.15.1:0/3874585834 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda201019b0 0x7fda201059f0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:02.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.874+0000 7fda27e00700 1 -- 172.21.15.1:0/3874585834 >> 172.21.15.1:0/3874585834 conn(0x7fda200fc840 msgr2=0x7fda200fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:02.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.874+0000 7fda27e00700 1 -- 172.21.15.1:0/3874585834 shutdown_connections 2024-02-02T15:38:02.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.874+0000 7fda27e00700 1 -- 172.21.15.1:0/3874585834 wait complete. 2024-02-02T15:38:02.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.875+0000 7fda27e00700 1 Processor -- start 2024-02-02T15:38:02.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.875+0000 7fda27e00700 1 -- start start 2024-02-02T15:38:02.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.876+0000 7fda27e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda20101000 0x7fda20073040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:02.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.876+0000 7fda27e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda20106120 0x7fda20073580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:02.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.876+0000 7fda27e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fda20073ac0 0x7fda201a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:02.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.876+0000 7fda27e00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda201a5790 con 0x7fda20106120 2024-02-02T15:38:02.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.876+0000 7fda27e00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda201a5900 con 0x7fda20073ac0 2024-02-02T15:38:02.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.876+0000 7fda27e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda201a5ac0 con 0x7fda20101000 2024-02-02T15:38:02.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.876+0000 7fda25400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda20101000 0x7fda20073040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:02.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.876+0000 7fda25e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fda20073ac0 0x7fda201a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:02.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.876+0000 7fda24a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda20106120 0x7fda20073580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:02.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.876+0000 7fda25400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda20101000 0x7fda20073040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:36632/0 (socket says 172.21.15.1:36632) 2024-02-02T15:38:02.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.876+0000 7fda25400700 1 -- 172.21.15.1:0/2587858595 learned_addr learned my addr 172.21.15.1:0/2587858595 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:38:02.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.876+0000 7fda25e00700 1 -- 172.21.15.1:0/2587858595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda20101000 msgr2=0x7fda20073040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:02.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.876+0000 7fda25e00700 1 --2- 172.21.15.1:0/2587858595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda20101000 0x7fda20073040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:02.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.876+0000 7fda25e00700 1 -- 172.21.15.1:0/2587858595 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda20106120 msgr2=0x7fda20073580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:02.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.876+0000 7fda25e00700 1 --2- 172.21.15.1:0/2587858595 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda20106120 0x7fda20073580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:02.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.876+0000 7fda25e00700 1 -- 172.21.15.1:0/2587858595 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda080095f0 con 0x7fda20073ac0 2024-02-02T15:38:02.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.877+0000 7fda25e00700 1 --2- 172.21.15.1:0/2587858595 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fda20073ac0 0x7fda201a5100 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fda1800efc0 tx=0x7fda1800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:02.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.877+0000 7fda1e000700 1 -- 172.21.15.1:0/2587858595 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda18018070 con 0x7fda20073ac0 2024-02-02T15:38:02.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.877+0000 7fda1e000700 1 -- 172.21.15.1:0/2587858595 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda1800f040 con 0x7fda20073ac0 2024-02-02T15:38:02.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.877+0000 7fda1e000700 1 -- 172.21.15.1:0/2587858595 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda1800cd40 con 0x7fda20073ac0 2024-02-02T15:38:02.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.877+0000 7fda27e00700 1 -- 172.21.15.1:0/2587858595 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda201a5d40 con 0x7fda20073ac0 2024-02-02T15:38:02.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.877+0000 7fda27e00700 1 -- 172.21.15.1:0/2587858595 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda201a6200 con 0x7fda20073ac0 2024-02-02T15:38:02.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.881+0000 7fda1e000700 1 -- 172.21.15.1:0/2587858595 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7fda18003a00 con 0x7fda20073ac0 2024-02-02T15:38:02.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.881+0000 7fda3b887b80 1 Processor -- start 2024-02-02T15:38:02.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.882+0000 7fda3b887b80 1 -- start start 2024-02-02T15:38:02.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.882+0000 7fda3b887b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d4bc9e5010 0x55d4bc9e53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:02.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.882+0000 7fda3b887b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d4bc9e5930 0x55d4bc9f02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:02.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.882+0000 7fda3b887b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d4bc9f07e0 0x55d4bc9f2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:02.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.882+0000 7fda3b887b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4bc9594c0 con 0x55d4bc9f07e0 2024-02-02T15:38:02.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.882+0000 7fda3b887b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4bc93daa0 con 0x55d4bc9e5010 2024-02-02T15:38:02.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.882+0000 7fda3b887b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4bc93b460 con 0x55d4bc9e5930 2024-02-02T15:38:02.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.882+0000 7fda24a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d4bc9e5010 0x55d4bc9e53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:02.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.882+0000 7fda24a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d4bc9e5010 0x55d4bc9e53f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:50036/0 (socket says 172.21.15.1:50036) 2024-02-02T15:38:02.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.882+0000 7fda24a00700 1 -- 172.21.15.1:0/2978380494 learned_addr learned my addr 172.21.15.1:0/2978380494 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:38:02.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.883+0000 7fda24a00700 1 --2- 172.21.15.1:0/2978380494 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d4bc9f07e0 0x55d4bc9f2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:02.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.883+0000 7fda25400700 1 --2- 172.21.15.1:0/2978380494 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d4bc9e5930 0x55d4bc9f02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:02.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.883+0000 7fda1e000700 1 --2- 172.21.15.1:0/2587858595 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fda000a0710 0x7fda000a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:02.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.883+0000 7fda1e000700 1 -- 172.21.15.1:0/2587858595 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fda180cbbd0 con 0x7fda20073ac0 2024-02-02T15:38:02.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.883+0000 7fda24a00700 1 -- 172.21.15.1:0/2978380494 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d4bc9e5930 msgr2=0x55d4bc9f02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:02.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.883+0000 7fda24a00700 1 --2- 172.21.15.1:0/2978380494 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d4bc9e5930 0x55d4bc9f02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:02.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.883+0000 7fda24a00700 1 -- 172.21.15.1:0/2978380494 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d4bc9f07e0 msgr2=0x55d4bc9f2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:02.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.883+0000 7fda24a00700 1 --2- 172.21.15.1:0/2978380494 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d4bc9f07e0 0x55d4bc9f2bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:02.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.883+0000 7fda24a00700 1 -- 172.21.15.1:0/2978380494 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d4bc7b83b0 con 0x55d4bc9e5010 2024-02-02T15:38:02.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.883+0000 7fda25e00700 1 --2- 172.21.15.1:0/2587858595 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fda000a0710 0x7fda000a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:02.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.883+0000 7fda24a00700 1 --2- 172.21.15.1:0/2978380494 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d4bc9e5010 0x55d4bc9e53f0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fda08015000 tx=0x7fda0801a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bac608ee53d36d37 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:02.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.883+0000 7fda16600700 1 -- 172.21.15.1:0/2978380494 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda08005050 con 0x55d4bc9e5010 2024-02-02T15:38:02.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.884+0000 7fda16600700 1 -- 172.21.15.1:0/2978380494 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda080051b0 con 0x55d4bc9e5010 2024-02-02T15:38:02.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.884+0000 7fda16600700 1 -- 172.21.15.1:0/2978380494 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda08004be0 con 0x55d4bc9e5010 2024-02-02T15:38:02.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.885+0000 7fda3b887b80 1 -- 172.21.15.1:0/2978380494 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d4bc9e5010 msgr2=0x55d4bc9e53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:02.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.885+0000 7fda25e00700 1 --2- 172.21.15.1:0/2587858595 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fda000a0710 0x7fda000a2bd0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fda1800ef90 tx=0x7fda180092e0 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:02.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.885+0000 7fda3b887b80 1 --2- 172.21.15.1:0/2978380494 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d4bc9e5010 0x55d4bc9e53f0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fda08015000 tx=0x7fda0801a040 comp rx=0 tx=0).stop 2024-02-02T15:38:02.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.885+0000 7fda3b887b80 1 -- 172.21.15.1:0/2978380494 shutdown_connections 2024-02-02T15:38:02.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.885+0000 7fda3b887b80 1 --2- 172.21.15.1:0/2978380494 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d4bc9f07e0 0x55d4bc9f2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:02.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.885+0000 7fda3b887b80 1 --2- 172.21.15.1:0/2978380494 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d4bc9e5010 0x55d4bc9e53f0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:02.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.885+0000 7fda3b887b80 1 --2- 172.21.15.1:0/2978380494 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d4bc9e5930 0x55d4bc9f02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:02.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.885+0000 7fda3b887b80 1 -- 172.21.15.1:0/2978380494 >> 172.21.15.1:0/2978380494 conn(0x55d4bc944db0 msgr2=0x55d4bc9e13b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:02.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.885+0000 7fda3b887b80 1 -- 172.21.15.1:0/2978380494 shutdown_connections 2024-02-02T15:38:02.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.885+0000 7fda3b887b80 1 -- 172.21.15.1:0/2978380494 wait complete. 2024-02-02T15:38:02.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.885+0000 7fda3b887b80 1 -- start start 2024-02-02T15:38:02.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.886+0000 7fda3b887b80 1 Processor -- start 2024-02-02T15:38:02.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.886+0000 7fda3b887b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d4bc9e5010 0x55d4bc9f01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:02.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.886+0000 7fda3b887b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d4bc9e5930 0x55d4bc9ed3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:02.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.886+0000 7fda3b887b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d4bc9f07e0 0x55d4bc9eda00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:02.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.886+0000 7fda3b887b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4bc936d50 con 0x55d4bc9e5930 2024-02-02T15:38:02.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.886+0000 7fda3b887b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4bc8d2d80 con 0x55d4bc9f07e0 2024-02-02T15:38:02.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.886+0000 7fda3b887b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4bc8d2a20 con 0x55d4bc9e5010 2024-02-02T15:38:02.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.886+0000 7fda3b887b80 10 client.0 _ll_register_callbacks cb 0x55d4bc8d4290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:38:02.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.886+0000 7fda24a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d4bc9e5010 0x55d4bc9f01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:02.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.887+0000 7fda24a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d4bc9e5010 0x55d4bc9f01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:36650/0 (socket says 172.21.15.1:36650) 2024-02-02T15:38:02.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.887+0000 7fda24a00700 1 -- 172.21.15.1:0/975370750 learned_addr learned my addr 172.21.15.1:0/975370750 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:38:02.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.887+0000 7fda24a00700 1 --2- 172.21.15.1:0/975370750 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d4bc9f07e0 0x55d4bc9eda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:02.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.887+0000 7fda25400700 1 --2- 172.21.15.1:0/975370750 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d4bc9e5930 0x55d4bc9ed3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:02.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.887+0000 7fda24a00700 1 -- 172.21.15.1:0/975370750 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d4bc9f07e0 msgr2=0x55d4bc9eda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:02.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.887+0000 7fda24a00700 1 --2- 172.21.15.1:0/975370750 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d4bc9f07e0 0x55d4bc9eda00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:02.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.887+0000 7fda24a00700 1 -- 172.21.15.1:0/975370750 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d4bc9e5930 msgr2=0x55d4bc9ed3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:02.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.887+0000 7fda24a00700 1 --2- 172.21.15.1:0/975370750 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d4bc9e5930 0x55d4bc9ed3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:02.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.887+0000 7fda24a00700 1 -- 172.21.15.1:0/975370750 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda08020040 con 0x55d4bc9e5010 2024-02-02T15:38:02.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.887+0000 7fda24a00700 1 --2- 172.21.15.1:0/975370750 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d4bc9e5010 0x55d4bc9f01e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fda0800b3e0 tx=0x7fda08004760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1dd14148b3215a7 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:02.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.887+0000 7fda0ea00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:38:02.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.887+0000 7fda0ea00700 1 -- 172.21.15.1:0/975370750 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda0802c070 con 0x55d4bc9e5010 2024-02-02T15:38:02.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.887+0000 7fda0ea00700 1 -- 172.21.15.1:0/975370750 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fda08004800 con 0x55d4bc9e5010 2024-02-02T15:38:02.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.888+0000 7fda3b887b80 1 -- 172.21.15.1:0/975370750 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d4bc9d90f0 con 0x55d4bc9e5010 2024-02-02T15:38:02.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.888+0000 7fda0ea00700 1 -- 172.21.15.1:0/975370750 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda080049c0 con 0x55d4bc9e5010 2024-02-02T15:38:02.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.888+0000 7fda3b887b80 10 client.28510 fetch_fsmap learned FSMap version 50 2024-02-02T15:38:02.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.888+0000 7fda3b887b80 1 -- 172.21.15.1:0/975370750 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55d4bc764b60 con 0x55d4bc9e5010 2024-02-02T15:38:02.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.888+0000 7fda0ea00700 1 -- 172.21.15.1:0/975370750 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d4bc9d90f0 con 0x55d4bc9e5010 2024-02-02T15:38:02.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.888+0000 7fda0ea00700 1 -- 172.21.15.1:0/975370750 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 50) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fda0800fa20 con 0x55d4bc9e5010 2024-02-02T15:38:02.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.888+0000 7fda3b887b80 10 client.28510 fetch_fsmap finished waiting for FSMap version 50 2024-02-02T15:38:02.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.888+0000 7fda3b887b80 10 client.28510 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:38:02.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.888+0000 7fda3b887b80 20 client.28510 populate_metadata read hostname 'smithi001' 2024-02-02T15:38:02.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.888+0000 7fda3b887b80 1 --2- 172.21.15.1:0/975370750 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55d4bc9ffca0 0x55d4bca02090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:02.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.888+0000 7fda3b887b80 4 client.28510 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:38:02.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.888+0000 7fda3b887b80 1 -- 172.21.15.1:0/975370750 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d4bc92fdd0 con 0x55d4bc9ffca0 2024-02-02T15:38:02.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.889+0000 7fda25400700 1 --2- 172.21.15.1:0/975370750 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55d4bc9ffca0 0x55d4bca02090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:02.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.889+0000 7fda25400700 1 --2- 172.21.15.1:0/975370750 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55d4bc9ffca0 0x55d4bca02090 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7392fb1b9f3af2e server_cookie=9ff93d3f4865a27e in_seq=0 out_seq=0 2024-02-02T15:38:02.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.889+0000 7fda0ea00700 10 client.28510 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:38:02.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.890+0000 7fda0ea00700 1 -- 172.21.15.1:0/975370750 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55d4bc92fdd0 con 0x55d4bc9ffca0 2024-02-02T15:38:02.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.890+0000 7fda0ea00700 10 client.28510 handle_command_reply: tid=0 2024-02-02T15:38:02.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.890+0000 7fda3b887b80 1 client.28510 shutdown 2024-02-02T15:38:02.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.890+0000 7fda3b887b80 1 -- 172.21.15.1:0/975370750 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d4bc9e5010 msgr2=0x55d4bc9f01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:02.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.890+0000 7fda3b887b80 1 --2- 172.21.15.1:0/975370750 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d4bc9e5010 0x55d4bc9f01e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fda0800b3e0 tx=0x7fda08004760 comp rx=0 tx=0).stop 2024-02-02T15:38:02.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.890+0000 7fda3b887b80 1 -- 172.21.15.1:0/975370750 shutdown_connections 2024-02-02T15:38:02.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.890+0000 7fda3b887b80 1 --2- 172.21.15.1:0/975370750 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55d4bc9ffca0 0x55d4bca02090 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:02.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.890+0000 7fda3b887b80 1 --2- 172.21.15.1:0/975370750 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d4bc9f07e0 0x55d4bc9eda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:02.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.890+0000 7fda3b887b80 1 --2- 172.21.15.1:0/975370750 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d4bc9e5010 0x55d4bc9f01e0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:02.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.890+0000 7fda3b887b80 1 --2- 172.21.15.1:0/975370750 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d4bc9e5930 0x55d4bc9ed3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:02.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.890+0000 7fda0ea00700 0 client.28510 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:38:02.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.890+0000 7fda3b887b80 1 -- 172.21.15.1:0/975370750 >> 172.21.15.1:0/975370750 conn(0x55d4bc944db0 msgr2=0x55d4bc9e0f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:02.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.890+0000 7fda3b887b80 1 -- 172.21.15.1:0/975370750 shutdown_connections 2024-02-02T15:38:02.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.890+0000 7fda3b887b80 1 -- 172.21.15.1:0/975370750 wait complete. 2024-02-02T15:38:02.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.890+0000 7fda3b887b80 20 client.28510 trim_cache size 0 max 16384 2024-02-02T15:38:02.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.909+0000 7fda3b887b80 1 Processor -- start 2024-02-02T15:38:02.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.909+0000 7fda3b887b80 1 -- start start 2024-02-02T15:38:02.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.910+0000 7fda3b887b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d4bc9e5010 0x55d4bc9f72d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:02.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.910+0000 7fda3b887b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d4bc9e5930 0x55d4bc9efd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:02.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.910+0000 7fda3b887b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d4bc9f07e0 0x55d4bc9e0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:02.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.910+0000 7fda3b887b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4bc7ba6a0 con 0x55d4bc9f07e0 2024-02-02T15:38:02.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.910+0000 7fda3b887b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4bc9e6260 con 0x55d4bc9e5930 2024-02-02T15:38:02.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.910+0000 7fda3b887b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4bc9d8c30 con 0x55d4bc9e5010 2024-02-02T15:38:02.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.910+0000 7fda25400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d4bc9e5930 0x55d4bc9efd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:02.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.910+0000 7fda24a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d4bc9e5010 0x55d4bc9f72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:02.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.910+0000 7fda25400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d4bc9e5930 0x55d4bc9efd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:40748/0 (socket says 172.21.15.1:40748) 2024-02-02T15:38:02.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.910+0000 7fda25400700 1 -- 172.21.15.1:0/2700048336 learned_addr learned my addr 172.21.15.1:0/2700048336 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:38:02.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.910+0000 7fda24a00700 1 --2- 172.21.15.1:0/2700048336 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d4bc9f07e0 0x55d4bc9e0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:02.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.910+0000 7fda25400700 1 -- 172.21.15.1:0/2700048336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d4bc9e5010 msgr2=0x55d4bc9f72d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:02.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.910+0000 7fda25400700 1 --2- 172.21.15.1:0/2700048336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d4bc9e5010 0x55d4bc9f72d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:02.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.910+0000 7fda25400700 1 -- 172.21.15.1:0/2700048336 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d4bc9f07e0 msgr2=0x55d4bc9e0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:02.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.910+0000 7fda25400700 1 --2- 172.21.15.1:0/2700048336 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d4bc9f07e0 0x55d4bc9e0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:02.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.910+0000 7fda25400700 1 -- 172.21.15.1:0/2700048336 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d4bc8fcea0 con 0x55d4bc9e5930 2024-02-02T15:38:02.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.910+0000 7fda24a00700 1 --2- 172.21.15.1:0/2700048336 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d4bc9f07e0 0x55d4bc9e0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:38:02.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.911+0000 7fda25400700 1 --2- 172.21.15.1:0/2700048336 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d4bc9e5930 0x55d4bc9efd10 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x55d4bca02930 tx=0x7fda10009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7dbe4a06b8e4c440 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:02.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.911+0000 7fda17000700 1 -- 172.21.15.1:0/2700048336 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda10017030 con 0x55d4bc9e5930 2024-02-02T15:38:02.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.911+0000 7fda17000700 1 -- 172.21.15.1:0/2700048336 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda10018040 con 0x55d4bc9e5930 2024-02-02T15:38:02.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.911+0000 7fda17000700 1 -- 172.21.15.1:0/2700048336 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda10021ae0 con 0x55d4bc9e5930 2024-02-02T15:38:02.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.912+0000 7fda3b887b80 1 -- 172.21.15.1:0/2700048336 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d4bc9e5930 msgr2=0x55d4bc9efd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:02.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.912+0000 7fda3b887b80 1 --2- 172.21.15.1:0/2700048336 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d4bc9e5930 0x55d4bc9efd10 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x55d4bca02930 tx=0x7fda10009260 comp rx=0 tx=0).stop 2024-02-02T15:38:02.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.912+0000 7fda3b887b80 1 -- 172.21.15.1:0/2700048336 shutdown_connections 2024-02-02T15:38:02.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.912+0000 7fda3b887b80 1 --2- 172.21.15.1:0/2700048336 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d4bc9f07e0 0x55d4bc9e0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:02.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.912+0000 7fda3b887b80 1 --2- 172.21.15.1:0/2700048336 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d4bc9e5010 0x55d4bc9f72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:02.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.912+0000 7fda3b887b80 1 --2- 172.21.15.1:0/2700048336 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d4bc9e5930 0x55d4bc9efd10 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:02.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.912+0000 7fda3b887b80 1 -- 172.21.15.1:0/2700048336 >> 172.21.15.1:0/2700048336 conn(0x55d4bc944db0 msgr2=0x55d4bc9fb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:02.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.912+0000 7fda3b887b80 1 -- 172.21.15.1:0/2700048336 shutdown_connections 2024-02-02T15:38:02.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.912+0000 7fda3b887b80 1 -- 172.21.15.1:0/2700048336 wait complete. 2024-02-02T15:38:02.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.912+0000 7fda3b887b80 1 -- start start 2024-02-02T15:38:02.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.913+0000 7fda3b887b80 1 Processor -- start 2024-02-02T15:38:02.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.913+0000 7fda3b887b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d4bc9e5010 0x55d4bc9f7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:02.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.913+0000 7fda3b887b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d4bc9e5930 0x55d4bc9f53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:02.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.913+0000 7fda3b887b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d4bc9f07e0 0x55d4bc9f5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:02.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.913+0000 7fda3b887b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4bc9f6000 con 0x55d4bc9e5930 2024-02-02T15:38:02.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.913+0000 7fda3b887b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4bc9f6170 con 0x55d4bc9f07e0 2024-02-02T15:38:02.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.913+0000 7fda3b887b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4bca51730 con 0x55d4bc9e5010 2024-02-02T15:38:02.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.913+0000 7fda3b887b80 10 client.0 _ll_register_callbacks cb 0x55d4bc8d4290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:38:02.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.914+0000 7fda25400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d4bc9e5930 0x55d4bc9f53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:02.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.914+0000 7fda25400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d4bc9e5930 0x55d4bc9f53f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37204/0 (socket says 172.21.15.1:37204) 2024-02-02T15:38:02.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.914+0000 7fda25400700 1 -- 172.21.15.1:0/3433026137 learned_addr learned my addr 172.21.15.1:0/3433026137 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:38:02.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.914+0000 7fda24a00700 1 --2- 172.21.15.1:0/3433026137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d4bc9e5010 0x55d4bc9f7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:02.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.914+0000 7fda24a00700 1 --2- 172.21.15.1:0/3433026137 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d4bc9f07e0 0x55d4bc9f5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:02.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.914+0000 7fda25400700 1 -- 172.21.15.1:0/3433026137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d4bc9e5010 msgr2=0x55d4bc9f7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:02.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.914+0000 7fda25400700 1 --2- 172.21.15.1:0/3433026137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d4bc9e5010 0x55d4bc9f7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:02.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.914+0000 7fda25400700 1 -- 172.21.15.1:0/3433026137 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d4bc9f07e0 msgr2=0x55d4bc9f5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:02.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.914+0000 7fda25400700 1 --2- 172.21.15.1:0/3433026137 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d4bc9f07e0 0x55d4bc9f5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:02.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.914+0000 7fda25400700 1 -- 172.21.15.1:0/3433026137 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda10009d00 con 0x55d4bc9e5930 2024-02-02T15:38:02.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.914+0000 7fda25400700 1 --2- 172.21.15.1:0/3433026137 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d4bc9e5930 0x55d4bc9f53f0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x55d4bc92fe80 tx=0x7fda10004630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69774a73079612ab server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:02.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.914+0000 7fda0ea00700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:38:02.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.915+0000 7fda0ea00700 1 -- 172.21.15.1:0/3433026137 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda10017040 con 0x55d4bc9e5930 2024-02-02T15:38:02.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.915+0000 7fda0ea00700 1 -- 172.21.15.1:0/3433026137 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fda100210d0 con 0x55d4bc9e5930 2024-02-02T15:38:02.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.915+0000 7fda3b887b80 1 -- 172.21.15.1:0/3433026137 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fda080051b0 con 0x55d4bc9e5930 2024-02-02T15:38:02.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.915+0000 7fda0ea00700 1 -- 172.21.15.1:0/3433026137 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda100041d0 con 0x55d4bc9e5930 2024-02-02T15:38:02.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.915+0000 7fda3b887b80 10 client.18420 fetch_fsmap learned FSMap version 50 2024-02-02T15:38:02.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.915+0000 7fda3b887b80 1 -- 172.21.15.1:0/3433026137 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55d4bc7bccd0 con 0x55d4bc9e5930 2024-02-02T15:38:02.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.915+0000 7fda0ea00700 1 -- 172.21.15.1:0/3433026137 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda080051b0 con 0x55d4bc9e5930 2024-02-02T15:38:02.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.915+0000 7fda0ea00700 1 -- 172.21.15.1:0/3433026137 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 50) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fda10028070 con 0x55d4bc9e5930 2024-02-02T15:38:02.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.915+0000 7fda3b887b80 10 client.18420 fetch_fsmap finished waiting for FSMap version 50 2024-02-02T15:38:02.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.915+0000 7fda3b887b80 10 client.18420 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:38:02.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.915+0000 7fda3b887b80 20 client.18420 populate_metadata read hostname 'smithi001' 2024-02-02T15:38:02.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.915+0000 7fda3b887b80 1 --2- 172.21.15.1:0/3433026137 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55d4bca52cd0 0x55d4bca550f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:02.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.916+0000 7fda3b887b80 4 client.18420 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2024-02-02T15:38:02.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.916+0000 7fda3b887b80 1 -- 172.21.15.1:0/3433026137 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d4bc8f5f50 con 0x55d4bca52cd0 2024-02-02T15:38:02.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.916+0000 7fda24a00700 1 --2- 172.21.15.1:0/3433026137 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55d4bca52cd0 0x55d4bca550f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:02.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.916+0000 7fda24a00700 1 --2- 172.21.15.1:0/3433026137 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55d4bca52cd0 0x55d4bca550f0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd568325a0217c74 server_cookie=33336e189e351466 in_seq=0 out_seq=0 2024-02-02T15:38:02.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.916+0000 7fda0ea00700 10 client.18420 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:38:02.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.916+0000 7fda0ea00700 1 -- 172.21.15.1:0/3433026137 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55d4bc8f5f50 con 0x55d4bca52cd0 2024-02-02T15:38:02.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.916+0000 7fda0ea00700 10 client.18420 handle_command_reply: tid=0 2024-02-02T15:38:02.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.916+0000 7fda3b887b80 1 client.18420 shutdown 2024-02-02T15:38:02.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.917+0000 7fda3b887b80 1 -- 172.21.15.1:0/3433026137 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d4bc9e5930 msgr2=0x55d4bc9f53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:02.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.917+0000 7fda3b887b80 1 --2- 172.21.15.1:0/3433026137 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d4bc9e5930 0x55d4bc9f53f0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x55d4bc92fe80 tx=0x7fda10004630 comp rx=0 tx=0).stop 2024-02-02T15:38:02.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.917+0000 7fda3b887b80 1 -- 172.21.15.1:0/3433026137 shutdown_connections 2024-02-02T15:38:02.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.917+0000 7fda3b887b80 1 --2- 172.21.15.1:0/3433026137 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55d4bca52cd0 0x55d4bca550f0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:02.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.917+0000 7fda3b887b80 1 --2- 172.21.15.1:0/3433026137 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d4bc9f07e0 0x55d4bc9f5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:02.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.917+0000 7fda3b887b80 1 --2- 172.21.15.1:0/3433026137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d4bc9e5010 0x55d4bc9f7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:02.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.917+0000 7fda3b887b80 1 --2- 172.21.15.1:0/3433026137 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d4bc9e5930 0x55d4bc9f53f0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:02.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.917+0000 7fda0ea00700 0 client.18420 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:38:02.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.917+0000 7fda3b887b80 1 -- 172.21.15.1:0/3433026137 >> 172.21.15.1:0/3433026137 conn(0x55d4bc944db0 msgr2=0x55d4bc9e0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:02.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.917+0000 7fda3b887b80 1 -- 172.21.15.1:0/3433026137 shutdown_connections 2024-02-02T15:38:02.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.917+0000 7fda3b887b80 1 -- 172.21.15.1:0/3433026137 wait complete. 2024-02-02T15:38:02.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.917+0000 7fda3b887b80 20 client.18420 trim_cache size 0 max 16384 2024-02-02T15:38:02.924 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:38:02.924 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2024-02-02T15:38:02.924 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2024-02-02T15:38:02.924 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:38:02.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.918+0000 7fda27e00700 1 -- 172.21.15.1:0/2587858595 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fda000a0710 msgr2=0x7fda000a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:02.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.918+0000 7fda27e00700 1 --2- 172.21.15.1:0/2587858595 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fda000a0710 0x7fda000a2bd0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fda1800ef90 tx=0x7fda180092e0 comp rx=0 tx=0).stop 2024-02-02T15:38:02.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.918+0000 7fda27e00700 1 -- 172.21.15.1:0/2587858595 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fda20073ac0 msgr2=0x7fda201a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:02.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.918+0000 7fda27e00700 1 --2- 172.21.15.1:0/2587858595 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fda20073ac0 0x7fda201a5100 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fda1800efc0 tx=0x7fda1800c3a0 comp rx=0 tx=0).stop 2024-02-02T15:38:02.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.918+0000 7fda27e00700 1 -- 172.21.15.1:0/2587858595 shutdown_connections 2024-02-02T15:38:02.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.918+0000 7fda27e00700 1 --2- 172.21.15.1:0/2587858595 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fda000a0710 0x7fda000a2bd0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:02.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.918+0000 7fda27e00700 1 --2- 172.21.15.1:0/2587858595 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fda20073ac0 0x7fda201a5100 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:02.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.918+0000 7fda27e00700 1 --2- 172.21.15.1:0/2587858595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda20101000 0x7fda20073040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:02.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.918+0000 7fda27e00700 1 --2- 172.21.15.1:0/2587858595 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda20106120 0x7fda20073580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:02.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.918+0000 7fda27e00700 1 -- 172.21.15.1:0/2587858595 >> 172.21.15.1:0/2587858595 conn(0x7fda200fc840 msgr2=0x7fda200fd2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:02.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.918+0000 7fda27e00700 1 -- 172.21.15.1:0/2587858595 shutdown_connections 2024-02-02T15:38:02.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:02.918+0000 7fda27e00700 1 -- 172.21.15.1:0/2587858595 wait complete. 2024-02-02T15:38:02.936 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-02T15:38:02.937 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T15:38:03.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.155+0000 7f60a7400700 1 -- 172.21.15.1:0/3101743609 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60a0106b20 msgr2=0x7f60a0106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:03.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.155+0000 7f60a7400700 1 --2- 172.21.15.1:0/3101743609 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60a0106b20 0x7f60a0106f00 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f6090009990 tx=0x7f6090009ca0 comp rx=0 tx=0).stop 2024-02-02T15:38:03.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.155+0000 7f60a7400700 1 -- 172.21.15.1:0/3101743609 shutdown_connections 2024-02-02T15:38:03.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.155+0000 7f60a7400700 1 --2- 172.21.15.1:0/3101743609 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f60a00689d0 0x7f60a010d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:03.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.155+0000 7f60a7400700 1 --2- 172.21.15.1:0/3101743609 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60a0106b20 0x7f60a0106f00 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:03.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.155+0000 7f60a7400700 1 --2- 172.21.15.1:0/3101743609 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60a0068010 0x7f60a0068490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:03.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.155+0000 7f60a7400700 1 -- 172.21.15.1:0/3101743609 >> 172.21.15.1:0/3101743609 conn(0x7f60a0076730 msgr2=0x7f60a0076b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:03.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.155+0000 7f60a7400700 1 -- 172.21.15.1:0/3101743609 shutdown_connections 2024-02-02T15:38:03.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.155+0000 7f60a7400700 1 -- 172.21.15.1:0/3101743609 wait complete. 2024-02-02T15:38:03.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.156+0000 7f60a7400700 1 Processor -- start 2024-02-02T15:38:03.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.156+0000 7f60a7400700 1 -- start start 2024-02-02T15:38:03.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.157+0000 7f60a7400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f60a0068010 0x7f60a019ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:03.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.157+0000 7f60a7400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60a00689d0 0x7f60a019cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:03.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.157+0000 7f60a7400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60a0106b20 0x7f60a0196a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:03.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.157+0000 7f60a7400700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60a01970f0 con 0x7f60a0068010 2024-02-02T15:38:03.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.157+0000 7f60a7400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60a0197260 con 0x7f60a00689d0 2024-02-02T15:38:03.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.157+0000 7f60a7400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60a0197420 con 0x7f60a0106b20 2024-02-02T15:38:03.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.157+0000 7f60a6000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f60a0068010 0x7f60a019ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:03.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.157+0000 7f60a5600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60a00689d0 0x7f60a019cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:03.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.157+0000 7f60a6a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60a0106b20 0x7f60a0196a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:03.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.157+0000 7f60a5600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60a00689d0 0x7f60a019cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:40780/0 (socket says 172.21.15.1:40780) 2024-02-02T15:38:03.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.157+0000 7f60a5600700 1 -- 172.21.15.1:0/1526324719 learned_addr learned my addr 172.21.15.1:0/1526324719 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:38:03.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.157+0000 7f60a6a00700 1 -- 172.21.15.1:0/1526324719 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60a00689d0 msgr2=0x7f60a019cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:03.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.157+0000 7f60a6a00700 1 --2- 172.21.15.1:0/1526324719 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60a00689d0 0x7f60a019cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:03.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.158+0000 7f60a6a00700 1 -- 172.21.15.1:0/1526324719 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f60a0068010 msgr2=0x7f60a019ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:03.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.158+0000 7f60a6a00700 1 --2- 172.21.15.1:0/1526324719 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f60a0068010 0x7f60a019ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:03.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.158+0000 7f60a6a00700 1 -- 172.21.15.1:0/1526324719 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60900095f0 con 0x7f60a0106b20 2024-02-02T15:38:03.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.158+0000 7f60a5600700 1 --2- 172.21.15.1:0/1526324719 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60a00689d0 0x7f60a019cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:38:03.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.158+0000 7f60a6a00700 1 --2- 172.21.15.1:0/1526324719 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60a0106b20 0x7f60a0196a80 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f609800efc0 tx=0x7f609800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:03.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.158+0000 7f609ea00700 1 -- 172.21.15.1:0/1526324719 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6098018070 con 0x7f60a0106b20 2024-02-02T15:38:03.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.158+0000 7f609ea00700 1 -- 172.21.15.1:0/1526324719 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f609800f040 con 0x7f60a0106b20 2024-02-02T15:38:03.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.158+0000 7f60a7400700 1 -- 172.21.15.1:0/1526324719 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60a01976a0 con 0x7f60a0106b20 2024-02-02T15:38:03.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.158+0000 7f609ea00700 1 -- 172.21.15.1:0/1526324719 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f609800cc80 con 0x7f60a0106b20 2024-02-02T15:38:03.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.158+0000 7f60a7400700 1 -- 172.21.15.1:0/1526324719 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60a00ffd70 con 0x7f60a0106b20 2024-02-02T15:38:03.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.159+0000 7f60a7400700 1 -- 172.21.15.1:0/1526324719 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60a004ea40 con 0x7f60a0106b20 2024-02-02T15:38:03.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.161+0000 7f609ea00700 1 -- 172.21.15.1:0/1526324719 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f6098003a00 con 0x7f60a0106b20 2024-02-02T15:38:03.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.163+0000 7f609ea00700 1 --2- 172.21.15.1:0/1526324719 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f60800a0690 0x7f60800a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:03.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.163+0000 7f609ea00700 1 -- 172.21.15.1:0/1526324719 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f60980cc680 con 0x7f60a0106b20 2024-02-02T15:38:03.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.163+0000 7f60a6000700 1 --2- 172.21.15.1:0/1526324719 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f60800a0690 0x7f60800a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:03.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.164+0000 7f60a6000700 1 --2- 172.21.15.1:0/1526324719 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f60800a0690 0x7f60800a2b50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f609000b2e0 tx=0x7f6090005fd0 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:03.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.166+0000 7f609ea00700 1 -- 172.21.15.1:0/1526324719 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f6098095b30 con 0x7f60a0106b20 2024-02-02T15:38:03.264 INFO:tasks.workunit.client.0.smithi001.stdout: 1 616722 5.65 MB/sec execute 457 sec latency 2590.116 ms 2024-02-02T15:38:03.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.502+0000 7f60a7400700 1 -- 172.21.15.1:0/1526324719 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f60a0100030 con 0x7f60a0106b20 2024-02-02T15:38:03.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.502+0000 7f609ea00700 1 -- 172.21.15.1:0/1526324719 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+10869 (secure 0 0 0) 0x7f6098099150 con 0x7f60a0106b20 2024-02-02T15:38:03.503 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:38:03.503 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":50,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2922240448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2922240448},{"type":"v1","addr":"172.21.15.1:6835","nonce":2922240448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2729497431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2729497431},{"type":"v1","addr":"172.21.15.1:6837","nonce":2729497431}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":50,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-02T15:24:47.761435+0000","modified":"2024-02-02T15:37:50.509686+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24529,"mds_2":24518,"mds_3":24514,"mds_4":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14517":{"gid":14517,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/1218303683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":1218303683},{"type":"v1","addr":"172.21.15.1:6839","nonce":1218303683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14529":{"gid":14529,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/2059197496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2059197496},{"type":"v1","addr":"172.21.15.1:6841","nonce":2059197496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/7648986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":7648986},{"type":"v1","addr":"172.21.15.142:6833","nonce":7648986}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/3482746094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3482746094},{"type":"v1","addr":"172.21.15.71:6833","nonce":3482746094}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/388207814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":388207814},{"type":"v1","addr":"172.21.15.71:6835","nonce":388207814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6835/1052606547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1052606547},{"type":"v1","addr":"172.21.15.142:6835","nonce":1052606547}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.142:6837/2870289019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2870289019},{"type":"v1","addr":"172.21.15.142:6837","nonce":2870289019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.71:6837/628016199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":628016199},{"type":"v1","addr":"172.21.15.71:6837","nonce":628016199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.142:6839/321675421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":321675421},{"type":"v1","addr":"172.21.15.142:6839","nonce":321675421}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/1268086493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1268086493},{"type":"v1","addr":"172.21.15.71:6839","nonce":1268086493}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-02T15:38:03.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.505+0000 7f60a7400700 1 -- 172.21.15.1:0/1526324719 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f60800a0690 msgr2=0x7f60800a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:03.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.505+0000 7f60a7400700 1 --2- 172.21.15.1:0/1526324719 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f60800a0690 0x7f60800a2b50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f609000b2e0 tx=0x7f6090005fd0 comp rx=0 tx=0).stop 2024-02-02T15:38:03.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.505+0000 7f60a7400700 1 -- 172.21.15.1:0/1526324719 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60a0106b20 msgr2=0x7f60a0196a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:03.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.505+0000 7f60a7400700 1 --2- 172.21.15.1:0/1526324719 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60a0106b20 0x7f60a0196a80 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f609800efc0 tx=0x7f609800c3a0 comp rx=0 tx=0).stop 2024-02-02T15:38:03.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.505+0000 7f60a7400700 1 -- 172.21.15.1:0/1526324719 shutdown_connections 2024-02-02T15:38:03.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.505+0000 7f60a7400700 1 --2- 172.21.15.1:0/1526324719 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f60800a0690 0x7f60800a2b50 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:03.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.505+0000 7f60a7400700 1 --2- 172.21.15.1:0/1526324719 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f60a0106b20 0x7f60a0196a80 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:03.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.505+0000 7f60a7400700 1 --2- 172.21.15.1:0/1526324719 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f60a0068010 0x7f60a019ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:03.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.505+0000 7f60a7400700 1 --2- 172.21.15.1:0/1526324719 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f60a00689d0 0x7f60a019cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:03.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.505+0000 7f60a7400700 1 -- 172.21.15.1:0/1526324719 >> 172.21.15.1:0/1526324719 conn(0x7f60a0076730 msgr2=0x7f60a00fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:03.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.505+0000 7f60a7400700 1 -- 172.21.15.1:0/1526324719 shutdown_connections 2024-02-02T15:38:03.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.506+0000 7f60a7400700 1 -- 172.21.15.1:0/1526324719 wait complete. 2024-02-02T15:38:03.507 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 50 2024-02-02T15:38:03.528 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-02-02T15:38:03.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:03 smithi001 ceph-mon[35470]: pgmap v342: 129 pgs: 129 active+clean; 496 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 166 op/s 2024-02-02T15:38:03.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.764+0000 7f6574400700 1 -- 172.21.15.1:0/2335999587 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f656c10b400 msgr2=0x7f656c10d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:03.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.764+0000 7f6574400700 1 --2- 172.21.15.1:0/2335999587 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f656c10b400 0x7f656c10d850 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f6564009f90 tx=0x7f656400b330 comp rx=0 tx=0).stop 2024-02-02T15:38:03.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.764+0000 7f6574400700 1 -- 172.21.15.1:0/2335999587 shutdown_connections 2024-02-02T15:38:03.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.764+0000 7f6574400700 1 --2- 172.21.15.1:0/2335999587 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f656c10b400 0x7f656c10d850 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:03.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.764+0000 7f6574400700 1 --2- 172.21.15.1:0/2335999587 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f656c102030 0x7f656c102410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:03.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.764+0000 7f6574400700 1 --2- 172.21.15.1:0/2335999587 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f656c102950 0x7f656c10ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:03.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.764+0000 7f6574400700 1 -- 172.21.15.1:0/2335999587 >> 172.21.15.1:0/2335999587 conn(0x7f656c076d70 msgr2=0x7f656c077180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:03.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.765+0000 7f6574400700 1 -- 172.21.15.1:0/2335999587 shutdown_connections 2024-02-02T15:38:03.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.765+0000 7f6574400700 1 -- 172.21.15.1:0/2335999587 wait complete. 2024-02-02T15:38:03.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.766+0000 7f6574400700 1 Processor -- start 2024-02-02T15:38:03.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.766+0000 7f6574400700 1 -- start start 2024-02-02T15:38:03.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.767+0000 7f6574400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f656c102030 0x7f656c102ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:03.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.767+0000 7f6574400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f656c102950 0x7f656c1033e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:03.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.767+0000 7f6574400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f656c10b400 0x7f656c103920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:03.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.767+0000 7f6574400700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f656c1a9840 con 0x7f656c102950 2024-02-02T15:38:03.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.767+0000 7f6574400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f656c1a99b0 con 0x7f656c10b400 2024-02-02T15:38:03.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.767+0000 7f6574400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f656c1a9b70 con 0x7f656c102030 2024-02-02T15:38:03.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.767+0000 7f6571a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f656c102030 0x7f656c102ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:03.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.767+0000 7f6571a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f656c102030 0x7f656c102ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:45982/0 (socket says 172.21.15.1:45982) 2024-02-02T15:38:03.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.767+0000 7f6571a00700 1 -- 172.21.15.1:0/678140088 learned_addr learned my addr 172.21.15.1:0/678140088 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:38:03.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.767+0000 7f6571000700 1 --2- 172.21.15.1:0/678140088 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f656c102950 0x7f656c1033e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:03.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.767+0000 7f6572400700 1 --2- 172.21.15.1:0/678140088 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f656c10b400 0x7f656c103920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:03.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.768+0000 7f6571a00700 1 -- 172.21.15.1:0/678140088 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f656c10b400 msgr2=0x7f656c103920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:03.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.768+0000 7f6571a00700 1 --2- 172.21.15.1:0/678140088 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f656c10b400 0x7f656c103920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:03.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.768+0000 7f6571a00700 1 -- 172.21.15.1:0/678140088 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f656c102950 msgr2=0x7f656c1033e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:03.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.768+0000 7f6571a00700 1 --2- 172.21.15.1:0/678140088 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f656c102950 0x7f656c1033e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:03.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.768+0000 7f6571a00700 1 -- 172.21.15.1:0/678140088 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65540095f0 con 0x7f656c102030 2024-02-02T15:38:03.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.768+0000 7f6571a00700 1 --2- 172.21.15.1:0/678140088 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f656c102030 0x7f656c102ea0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f655c00e9a0 tx=0x7f655c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:03.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.768+0000 7f656a000700 1 -- 172.21.15.1:0/678140088 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f655c009790 con 0x7f656c102030 2024-02-02T15:38:03.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.768+0000 7f656a000700 1 -- 172.21.15.1:0/678140088 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f655c010430 con 0x7f656c102030 2024-02-02T15:38:03.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.768+0000 7f656a000700 1 -- 172.21.15.1:0/678140088 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f655c018680 con 0x7f656c102030 2024-02-02T15:38:03.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.769+0000 7f6574400700 1 -- 172.21.15.1:0/678140088 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6564009c40 con 0x7f656c102030 2024-02-02T15:38:03.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.769+0000 7f6574400700 1 -- 172.21.15.1:0/678140088 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f656c1aa1d0 con 0x7f656c102030 2024-02-02T15:38:03.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.772+0000 7f656a000700 1 -- 172.21.15.1:0/678140088 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f655c0105a0 con 0x7f656c102030 2024-02-02T15:38:03.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.773+0000 7f6587f10b80 1 Processor -- start 2024-02-02T15:38:03.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.773+0000 7f656a000700 1 --2- 172.21.15.1:0/678140088 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f654c0a0710 0x7f654c0a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:03.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.773+0000 7f6587f10b80 1 -- start start 2024-02-02T15:38:03.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.773+0000 7f656a000700 1 -- 172.21.15.1:0/678140088 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f655c014070 con 0x7f656c102030 2024-02-02T15:38:03.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.774+0000 7f6572400700 1 --2- 172.21.15.1:0/678140088 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f654c0a0710 0x7f654c0a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:03.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.774+0000 7f6587f10b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555898b0a010 0x555898b0a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:03.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.774+0000 7f6587f10b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555898b0a930 0x555898b152a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:03.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.774+0000 7f6587f10b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555898b157e0 0x555898b17bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:03.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.774+0000 7f6587f10b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x555898a7e4c0 con 0x555898b0a010 2024-02-02T15:38:03.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.774+0000 7f6587f10b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x555898a62aa0 con 0x555898b0a930 2024-02-02T15:38:03.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.774+0000 7f6587f10b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x555898a60460 con 0x555898b157e0 2024-02-02T15:38:03.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.774+0000 7f6571a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555898b0a010 0x555898b0a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:03.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.774+0000 7f6571000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555898b0a930 0x555898b152a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:03.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.774+0000 7f6571a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555898b0a010 0x555898b0a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37248/0 (socket says 172.21.15.1:37248) 2024-02-02T15:38:03.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.774+0000 7f6571a00700 1 -- 172.21.15.1:0/1634397473 learned_addr learned my addr 172.21.15.1:0/1634397473 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:38:03.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.774+0000 7f6571000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555898b0a930 0x555898b152a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:40804/0 (socket says 172.21.15.1:40804) 2024-02-02T15:38:03.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.774+0000 7f6572400700 1 --2- 172.21.15.1:0/1634397473 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555898b157e0 0x555898b17bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:03.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.775+0000 7f6571000700 1 -- 172.21.15.1:0/1634397473 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555898b157e0 msgr2=0x555898b17bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:03.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.775+0000 7f6571000700 1 --2- 172.21.15.1:0/1634397473 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555898b157e0 0x555898b17bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:03.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.775+0000 7f6571000700 1 -- 172.21.15.1:0/1634397473 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555898b0a010 msgr2=0x555898b0a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:03.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.775+0000 7f6571000700 1 --2- 172.21.15.1:0/1634397473 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555898b0a010 0x555898b0a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:03.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.775+0000 7f6571000700 1 -- 172.21.15.1:0/1634397473 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5558988dd3b0 con 0x555898b0a930 2024-02-02T15:38:03.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.775+0000 7f6571000700 1 --2- 172.21.15.1:0/1634397473 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555898b0a930 0x555898b152a0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f6554005fd0 tx=0x7f6554011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fad5b3785ca05dc3 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:03.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.775+0000 7f6572400700 1 --2- 172.21.15.1:0/678140088 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f654c0a0710 0x7f654c0a2bd0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6564009790 tx=0x7f6564007e20 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:03.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.775+0000 7f6562a00700 1 -- 172.21.15.1:0/1634397473 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6554004310 con 0x555898b0a930 2024-02-02T15:38:03.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.775+0000 7f6562a00700 1 -- 172.21.15.1:0/1634397473 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6554004470 con 0x555898b0a930 2024-02-02T15:38:03.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.775+0000 7f6562a00700 1 -- 172.21.15.1:0/1634397473 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6554019880 con 0x555898b0a930 2024-02-02T15:38:03.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.776+0000 7f6587f10b80 1 -- 172.21.15.1:0/1634397473 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555898b0a930 msgr2=0x555898b152a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:03.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.776+0000 7f6587f10b80 1 --2- 172.21.15.1:0/1634397473 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555898b0a930 0x555898b152a0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f6554005fd0 tx=0x7f6554011720 comp rx=0 tx=0).stop 2024-02-02T15:38:03.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.776+0000 7f6587f10b80 1 -- 172.21.15.1:0/1634397473 shutdown_connections 2024-02-02T15:38:03.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.776+0000 7f6587f10b80 1 --2- 172.21.15.1:0/1634397473 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555898b157e0 0x555898b17bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:03.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.776+0000 7f6587f10b80 1 --2- 172.21.15.1:0/1634397473 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555898b0a010 0x555898b0a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:03.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.776+0000 7f6587f10b80 1 --2- 172.21.15.1:0/1634397473 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555898b0a930 0x555898b152a0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:03.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.776+0000 7f6587f10b80 1 -- 172.21.15.1:0/1634397473 >> 172.21.15.1:0/1634397473 conn(0x555898a69db0 msgr2=0x555898b063b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:03.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.776+0000 7f6587f10b80 1 -- 172.21.15.1:0/1634397473 shutdown_connections 2024-02-02T15:38:03.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.776+0000 7f6587f10b80 1 -- 172.21.15.1:0/1634397473 wait complete. 2024-02-02T15:38:03.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.777+0000 7f6587f10b80 1 -- start start 2024-02-02T15:38:03.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.777+0000 7f6587f10b80 1 Processor -- start 2024-02-02T15:38:03.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.777+0000 7f6587f10b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555898b0a010 0x555898b15110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:03.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.777+0000 7f6587f10b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555898b0a930 0x555898b12320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:03.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.777+0000 7f6587f10b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555898b157e0 0x555898b12880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:03.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.777+0000 7f6587f10b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x555898a5bd50 con 0x555898b0a930 2024-02-02T15:38:03.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.777+0000 7f6587f10b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558989f7d80 con 0x555898b0a010 2024-02-02T15:38:03.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.777+0000 7f6587f10b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558989f7a20 con 0x555898b157e0 2024-02-02T15:38:03.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.778+0000 7f6587f10b80 10 client.0 _ll_register_callbacks cb 0x5558989f9290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:38:03.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.778+0000 7f6571a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555898b0a010 0x555898b15110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:03.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.778+0000 7f6571a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555898b0a010 0x555898b15110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:40818/0 (socket says 172.21.15.1:40818) 2024-02-02T15:38:03.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.778+0000 7f6571a00700 1 -- 172.21.15.1:0/139369790 learned_addr learned my addr 172.21.15.1:0/139369790 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:38:03.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.778+0000 7f6572400700 1 --2- 172.21.15.1:0/139369790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555898b157e0 0x555898b12880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:03.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.778+0000 7f6571000700 1 --2- 172.21.15.1:0/139369790 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555898b0a930 0x555898b12320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:03.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.778+0000 7f6571a00700 1 -- 172.21.15.1:0/139369790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555898b157e0 msgr2=0x555898b12880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:03.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.778+0000 7f6571a00700 1 --2- 172.21.15.1:0/139369790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555898b157e0 0x555898b12880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:03.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.778+0000 7f6571a00700 1 -- 172.21.15.1:0/139369790 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555898b0a930 msgr2=0x555898b12320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:03.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.778+0000 7f6571a00700 1 --2- 172.21.15.1:0/139369790 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555898b0a930 0x555898b12320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:03.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.778+0000 7f6571a00700 1 -- 172.21.15.1:0/139369790 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6554009b80 con 0x555898b0a010 2024-02-02T15:38:03.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.778+0000 7f6571a00700 1 --2- 172.21.15.1:0/139369790 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555898b0a010 0x555898b15110 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f655c095e30 tx=0x7f655c098d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a937eebc0957f11c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:03.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.778+0000 7f655b400700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:38:03.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.778+0000 7f655b400700 1 -- 172.21.15.1:0/139369790 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f655c098ab0 con 0x555898b0a010 2024-02-02T15:38:03.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.779+0000 7f655b400700 1 -- 172.21.15.1:0/139369790 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f655c099ac0 con 0x555898b0a010 2024-02-02T15:38:03.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.779+0000 7f6587f10b80 1 -- 172.21.15.1:0/139369790 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555898afe0f0 con 0x555898b0a010 2024-02-02T15:38:03.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.779+0000 7f655b400700 1 -- 172.21.15.1:0/139369790 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f655c0993c0 con 0x555898b0a010 2024-02-02T15:38:03.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.779+0000 7f6587f10b80 10 client.28532 fetch_fsmap learned FSMap version 50 2024-02-02T15:38:03.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.779+0000 7f6587f10b80 1 -- 172.21.15.1:0/139369790 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x555898889b60 con 0x555898b0a010 2024-02-02T15:38:03.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.779+0000 7f655b400700 1 -- 172.21.15.1:0/139369790 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555898afe0f0 con 0x555898b0a010 2024-02-02T15:38:03.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.779+0000 7f655b400700 1 -- 172.21.15.1:0/139369790 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 50) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f655c082bc0 con 0x555898b0a010 2024-02-02T15:38:03.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.779+0000 7f6587f10b80 10 client.28532 fetch_fsmap finished waiting for FSMap version 50 2024-02-02T15:38:03.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.779+0000 7f6587f10b80 10 client.28532 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-02-02T15:38:03.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.779+0000 7f6587f10b80 20 client.28532 populate_metadata read hostname 'smithi001' 2024-02-02T15:38:03.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.779+0000 7f6587f10b80 1 --2- 172.21.15.1:0/139369790 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x555898b251b0 0x555898b275d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:03.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.779+0000 7f6587f10b80 4 client.28532 mds_command: new command op to 24502 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:38:03.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.779+0000 7f6587f10b80 1 -- 172.21.15.1:0/139369790 --> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555898a54dd0 con 0x555898b251b0 2024-02-02T15:38:03.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.780+0000 7f6571000700 1 --2- 172.21.15.1:0/139369790 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x555898b251b0 0x555898b275d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:03.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.780+0000 7f6571000700 1 --2- 172.21.15.1:0/139369790 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x555898b251b0 0x555898b275d0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f155ef7e3c68bfc1 server_cookie=bfade8f2e67c810c in_seq=0 out_seq=0 2024-02-02T15:38:03.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.780+0000 7f655b400700 10 client.28532 ms_handle_connect on v2:172.21.15.142:6834/1052606547 2024-02-02T15:38:03.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.781+0000 7f655b400700 1 -- 172.21.15.1:0/139369790 <== mds.4 v2:172.21.15.142:6834/1052606547 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x555898a54dd0 con 0x555898b251b0 2024-02-02T15:38:03.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.781+0000 7f655b400700 10 client.28532 handle_command_reply: tid=0 2024-02-02T15:38:03.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.781+0000 7f6587f10b80 1 client.28532 shutdown 2024-02-02T15:38:03.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.781+0000 7f6587f10b80 1 -- 172.21.15.1:0/139369790 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555898b0a010 msgr2=0x555898b15110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:03.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.781+0000 7f6587f10b80 1 --2- 172.21.15.1:0/139369790 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555898b0a010 0x555898b15110 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f655c095e30 tx=0x7f655c098d20 comp rx=0 tx=0).stop 2024-02-02T15:38:03.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.781+0000 7f6587f10b80 1 -- 172.21.15.1:0/139369790 shutdown_connections 2024-02-02T15:38:03.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.781+0000 7f6587f10b80 1 --2- 172.21.15.1:0/139369790 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x555898b251b0 0x555898b275d0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:03.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.781+0000 7f6587f10b80 1 --2- 172.21.15.1:0/139369790 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555898b157e0 0x555898b12880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:03.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.781+0000 7f6587f10b80 1 --2- 172.21.15.1:0/139369790 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555898b0a010 0x555898b15110 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:03.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.781+0000 7f655b400700 0 client.28532 ms_handle_reset on v2:172.21.15.142:6834/1052606547 2024-02-02T15:38:03.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.781+0000 7f6587f10b80 1 --2- 172.21.15.1:0/139369790 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555898b0a930 0x555898b12320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:03.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.781+0000 7f6587f10b80 1 -- 172.21.15.1:0/139369790 >> 172.21.15.1:0/139369790 conn(0x555898a69db0 msgr2=0x555898b05fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:03.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.781+0000 7f6587f10b80 1 -- 172.21.15.1:0/139369790 shutdown_connections 2024-02-02T15:38:03.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.782+0000 7f6587f10b80 1 -- 172.21.15.1:0/139369790 wait complete. 2024-02-02T15:38:03.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.782+0000 7f6587f10b80 20 client.28532 trim_cache size 0 max 16384 2024-02-02T15:38:03.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.798+0000 7f6587f10b80 1 Processor -- start 2024-02-02T15:38:03.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.798+0000 7f6587f10b80 1 -- start start 2024-02-02T15:38:03.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.798+0000 7f6587f10b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555898b0a010 0x555898b275a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:03.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.798+0000 7f6587f10b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555898b0a930 0x555898b0cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:03.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.798+0000 7f6587f10b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555898b157e0 0x555898b04b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:03.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.798+0000 7f6587f10b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558988df6a0 con 0x555898b157e0 2024-02-02T15:38:03.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.798+0000 7f6587f10b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x555898afdfc0 con 0x555898b0a010 2024-02-02T15:38:03.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.798+0000 7f6587f10b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x555898b07a30 con 0x555898b0a930 2024-02-02T15:38:03.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.798+0000 7f6571a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555898b0a010 0x555898b275a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:03.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.799+0000 7f6571a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555898b0a010 0x555898b275a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:40826/0 (socket says 172.21.15.1:40826) 2024-02-02T15:38:03.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.799+0000 7f6571a00700 1 -- 172.21.15.1:0/2693663763 learned_addr learned my addr 172.21.15.1:0/2693663763 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:38:03.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.799+0000 7f6571000700 1 --2- 172.21.15.1:0/2693663763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555898b0a930 0x555898b0cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:03.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.799+0000 7f6572400700 1 --2- 172.21.15.1:0/2693663763 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555898b157e0 0x555898b04b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:03.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.799+0000 7f6571a00700 1 -- 172.21.15.1:0/2693663763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555898b0a930 msgr2=0x555898b0cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:03.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.799+0000 7f6571a00700 1 --2- 172.21.15.1:0/2693663763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555898b0a930 0x555898b0cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:03.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.799+0000 7f6571a00700 1 -- 172.21.15.1:0/2693663763 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555898b157e0 msgr2=0x555898b04b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:03.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.799+0000 7f6571a00700 1 --2- 172.21.15.1:0/2693663763 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555898b157e0 0x555898b04b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:03.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.799+0000 7f6571a00700 1 -- 172.21.15.1:0/2693663763 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555898a21ea0 con 0x555898b0a010 2024-02-02T15:38:03.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.799+0000 7f6571a00700 1 --2- 172.21.15.1:0/2693663763 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555898b0a010 0x555898b275a0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f655c0958c0 tx=0x7f655c010b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96b8a50c8dd78142 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:03.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.799+0000 7f6563400700 1 -- 172.21.15.1:0/2693663763 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f655c099ac0 con 0x555898b0a010 2024-02-02T15:38:03.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.799+0000 7f6563400700 1 -- 172.21.15.1:0/2693663763 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f655c0993c0 con 0x555898b0a010 2024-02-02T15:38:03.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.799+0000 7f6563400700 1 -- 172.21.15.1:0/2693663763 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f655c0b70d0 con 0x555898b0a010 2024-02-02T15:38:03.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.800+0000 7f6587f10b80 1 -- 172.21.15.1:0/2693663763 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555898b0a010 msgr2=0x555898b275a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:03.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.800+0000 7f6587f10b80 1 --2- 172.21.15.1:0/2693663763 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555898b0a010 0x555898b275a0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f655c0958c0 tx=0x7f655c010b80 comp rx=0 tx=0).stop 2024-02-02T15:38:03.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.800+0000 7f6587f10b80 1 -- 172.21.15.1:0/2693663763 shutdown_connections 2024-02-02T15:38:03.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.800+0000 7f6587f10b80 1 --2- 172.21.15.1:0/2693663763 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555898b157e0 0x555898b04b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:03.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.800+0000 7f6587f10b80 1 --2- 172.21.15.1:0/2693663763 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555898b0a010 0x555898b275a0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:03.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.800+0000 7f6587f10b80 1 --2- 172.21.15.1:0/2693663763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555898b0a930 0x555898b0cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:03.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.800+0000 7f6587f10b80 1 -- 172.21.15.1:0/2693663763 >> 172.21.15.1:0/2693663763 conn(0x555898a69db0 msgr2=0x555898b10ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:03.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.800+0000 7f6587f10b80 1 -- 172.21.15.1:0/2693663763 shutdown_connections 2024-02-02T15:38:03.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.800+0000 7f6587f10b80 1 -- 172.21.15.1:0/2693663763 wait complete. 2024-02-02T15:38:03.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.801+0000 7f6587f10b80 1 -- start start 2024-02-02T15:38:03.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.801+0000 7f6587f10b80 1 Processor -- start 2024-02-02T15:38:03.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.801+0000 7f6587f10b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555898b0a010 0x555898b1c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:03.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.801+0000 7f6587f10b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555898b0a930 0x555898b19410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:03.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.801+0000 7f6587f10b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555898b157e0 0x555898b19950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:03.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.801+0000 7f6587f10b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x555898b19f20 con 0x555898b157e0 2024-02-02T15:38:03.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.801+0000 7f6587f10b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x555898b1a090 con 0x555898b0a010 2024-02-02T15:38:03.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.801+0000 7f6587f10b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x555898b1a250 con 0x555898b0a930 2024-02-02T15:38:03.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.801+0000 7f6587f10b80 10 client.0 _ll_register_callbacks cb 0x5558989f9290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:38:03.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.802+0000 7f6571a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555898b0a010 0x555898b1c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:03.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.802+0000 7f6571000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555898b0a930 0x555898b19410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:03.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.802+0000 7f6571a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555898b0a010 0x555898b1c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:40832/0 (socket says 172.21.15.1:40832) 2024-02-02T15:38:03.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.802+0000 7f6572400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555898b157e0 0x555898b19950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:03.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.802+0000 7f6571a00700 1 -- 172.21.15.1:0/2084449431 learned_addr learned my addr 172.21.15.1:0/2084449431 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:38:03.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.802+0000 7f6571a00700 1 -- 172.21.15.1:0/2084449431 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555898b0a930 msgr2=0x555898b19410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:03.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.802+0000 7f6571a00700 1 --2- 172.21.15.1:0/2084449431 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555898b0a930 0x555898b19410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:03.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.802+0000 7f6571a00700 1 -- 172.21.15.1:0/2084449431 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555898b157e0 msgr2=0x555898b19950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:03.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.802+0000 7f6571a00700 1 --2- 172.21.15.1:0/2084449431 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555898b157e0 0x555898b19950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:03.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.802+0000 7f6571a00700 1 -- 172.21.15.1:0/2084449431 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555898889b60 con 0x555898b0a010 2024-02-02T15:38:03.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.802+0000 7f6571a00700 1 --2- 172.21.15.1:0/2084449431 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555898b0a010 0x555898b1c0f0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f655c0a0f00 tx=0x7f655c0b72e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc310ad9febe85d6 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:03.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.802+0000 7f655b400700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:38:03.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.803+0000 7f655b400700 1 -- 172.21.15.1:0/2084449431 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f655c098ab0 con 0x555898b0a010 2024-02-02T15:38:03.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.803+0000 7f655b400700 1 -- 172.21.15.1:0/2084449431 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f655c09d1f0 con 0x555898b0a010 2024-02-02T15:38:03.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.803+0000 7f6587f10b80 1 -- 172.21.15.1:0/2084449431 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6554004470 con 0x555898b0a010 2024-02-02T15:38:03.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.803+0000 7f655b400700 1 -- 172.21.15.1:0/2084449431 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f655c0a11b0 con 0x555898b0a010 2024-02-02T15:38:03.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.803+0000 7f6587f10b80 10 client.28538 fetch_fsmap learned FSMap version 50 2024-02-02T15:38:03.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.803+0000 7f6587f10b80 1 -- 172.21.15.1:0/2084449431 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x5558988e1cd0 con 0x555898b0a010 2024-02-02T15:38:03.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.803+0000 7f655b400700 1 -- 172.21.15.1:0/2084449431 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6554004470 con 0x555898b0a010 2024-02-02T15:38:03.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.803+0000 7f655b400700 1 -- 172.21.15.1:0/2084449431 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 50) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f655c098320 con 0x555898b0a010 2024-02-02T15:38:03.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.803+0000 7f6587f10b80 10 client.28538 fetch_fsmap finished waiting for FSMap version 50 2024-02-02T15:38:03.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.803+0000 7f6587f10b80 10 client.28538 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-02-02T15:38:03.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.803+0000 7f6587f10b80 20 client.28538 populate_metadata read hostname 'smithi001' 2024-02-02T15:38:03.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.803+0000 7f6587f10b80 1 --2- 172.21.15.1:0/2084449431 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x555898b76d40 0x555898b79130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:03.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.803+0000 7f6587f10b80 4 client.28538 mds_command: new command op to 24502 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:38:03.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.803+0000 7f6587f10b80 1 -- 172.21.15.1:0/2084449431 --> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555898a1af50 con 0x555898b76d40 2024-02-02T15:38:03.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.804+0000 7f6571000700 1 --2- 172.21.15.1:0/2084449431 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x555898b76d40 0x555898b79130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:03.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.804+0000 7f6571000700 1 --2- 172.21.15.1:0/2084449431 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x555898b76d40 0x555898b79130 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=872216868163ac89 server_cookie=8c56e98e8aa5ca30 in_seq=0 out_seq=0 2024-02-02T15:38:03.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.804+0000 7f655b400700 10 client.28538 ms_handle_connect on v2:172.21.15.142:6834/1052606547 2024-02-02T15:38:03.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.804+0000 7f655b400700 1 -- 172.21.15.1:0/2084449431 <== mds.4 v2:172.21.15.142:6834/1052606547 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555898a1af50 con 0x555898b76d40 2024-02-02T15:38:03.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.804+0000 7f655b400700 10 client.28538 handle_command_reply: tid=0 2024-02-02T15:38:03.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.804+0000 7f6587f10b80 1 client.28538 shutdown 2024-02-02T15:38:03.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.804+0000 7f6587f10b80 1 -- 172.21.15.1:0/2084449431 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555898b0a010 msgr2=0x555898b1c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:03.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.804+0000 7f6587f10b80 1 --2- 172.21.15.1:0/2084449431 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555898b0a010 0x555898b1c0f0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f655c0a0f00 tx=0x7f655c0b72e0 comp rx=0 tx=0).stop 2024-02-02T15:38:03.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.805+0000 7f6587f10b80 1 -- 172.21.15.1:0/2084449431 shutdown_connections 2024-02-02T15:38:03.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.805+0000 7f6587f10b80 1 --2- 172.21.15.1:0/2084449431 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x555898b76d40 0x555898b79130 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:03.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.805+0000 7f6587f10b80 1 --2- 172.21.15.1:0/2084449431 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555898b157e0 0x555898b19950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:03.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.805+0000 7f6587f10b80 1 --2- 172.21.15.1:0/2084449431 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555898b0a010 0x555898b1c0f0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:03.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.805+0000 7f6587f10b80 1 --2- 172.21.15.1:0/2084449431 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555898b0a930 0x555898b19410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:03.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.805+0000 7f6587f10b80 1 -- 172.21.15.1:0/2084449431 >> 172.21.15.1:0/2084449431 conn(0x555898a69db0 msgr2=0x555898b11360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:03.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.805+0000 7f655b400700 0 client.28538 ms_handle_reset on v2:172.21.15.142:6834/1052606547 2024-02-02T15:38:03.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.805+0000 7f6587f10b80 1 -- 172.21.15.1:0/2084449431 shutdown_connections 2024-02-02T15:38:03.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.805+0000 7f6587f10b80 1 -- 172.21.15.1:0/2084449431 wait complete. 2024-02-02T15:38:03.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.805+0000 7f6587f10b80 20 client.28538 trim_cache size 0 max 16384 2024-02-02T15:38:03.809 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:38:03.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.805+0000 7f6574400700 1 -- 172.21.15.1:0/678140088 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f654c0a0710 msgr2=0x7f654c0a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:03.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.805+0000 7f6574400700 1 --2- 172.21.15.1:0/678140088 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f654c0a0710 0x7f654c0a2bd0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6564009790 tx=0x7f6564007e20 comp rx=0 tx=0).stop 2024-02-02T15:38:03.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.805+0000 7f6574400700 1 -- 172.21.15.1:0/678140088 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f656c102030 msgr2=0x7f656c102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:03.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.805+0000 7f6574400700 1 --2- 172.21.15.1:0/678140088 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f656c102030 0x7f656c102ea0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f655c00e9a0 tx=0x7f655c00ecb0 comp rx=0 tx=0).stop 2024-02-02T15:38:03.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.806+0000 7f6574400700 1 -- 172.21.15.1:0/678140088 shutdown_connections 2024-02-02T15:38:03.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.806+0000 7f6574400700 1 --2- 172.21.15.1:0/678140088 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f654c0a0710 0x7f654c0a2bd0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:03.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.806+0000 7f6574400700 1 --2- 172.21.15.1:0/678140088 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f656c10b400 0x7f656c103920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:03.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.806+0000 7f6574400700 1 --2- 172.21.15.1:0/678140088 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f656c102030 0x7f656c102ea0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:03.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.806+0000 7f6574400700 1 --2- 172.21.15.1:0/678140088 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f656c102950 0x7f656c1033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:03.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.806+0000 7f6574400700 1 -- 172.21.15.1:0/678140088 >> 172.21.15.1:0/678140088 conn(0x7f656c076d70 msgr2=0x7f656c0ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:03.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.806+0000 7f6574400700 1 -- 172.21.15.1:0/678140088 shutdown_connections 2024-02-02T15:38:03.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:03.806+0000 7f6574400700 1 -- 172.21.15.1:0/678140088 wait complete. 2024-02-02T15:38:03.825 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-02-02T15:38:03.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:03 smithi071 ceph-mon[44707]: pgmap v342: 129 pgs: 129 active+clean; 496 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 166 op/s 2024-02-02T15:38:03.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:03 smithi142 ceph-mon[45728]: pgmap v342: 129 pgs: 129 active+clean; 496 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 166 op/s 2024-02-02T15:38:04.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.043+0000 7ff828200700 1 -- 172.21.15.1:0/3994907462 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff820105cb0 msgr2=0x7ff820068010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:04.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.043+0000 7ff828200700 1 --2- 172.21.15.1:0/3994907462 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff820105cb0 0x7ff820068010 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7ff810009990 tx=0x7ff810009ca0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.043+0000 7ff828200700 1 -- 172.21.15.1:0/3994907462 shutdown_connections 2024-02-02T15:38:04.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.043+0000 7ff828200700 1 --2- 172.21.15.1:0/3994907462 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff820103800 0x7ff820103cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.043+0000 7ff828200700 1 --2- 172.21.15.1:0/3994907462 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff820105cb0 0x7ff820068010 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.043+0000 7ff828200700 1 --2- 172.21.15.1:0/3994907462 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff820068550 0x7ff8200689d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.043+0000 7ff828200700 1 -- 172.21.15.1:0/3994907462 >> 172.21.15.1:0/3994907462 conn(0x7ff8200757d0 msgr2=0x7ff820075be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:04.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.043+0000 7ff828200700 1 -- 172.21.15.1:0/3994907462 shutdown_connections 2024-02-02T15:38:04.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.043+0000 7ff828200700 1 -- 172.21.15.1:0/3994907462 wait complete. 2024-02-02T15:38:04.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.045+0000 7ff828200700 1 Processor -- start 2024-02-02T15:38:04.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.045+0000 7ff828200700 1 -- start start 2024-02-02T15:38:04.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.045+0000 7ff828200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff820068550 0x7ff82019d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:04.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.045+0000 7ff828200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff820103800 0x7ff82019d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:04.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.045+0000 7ff828200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff820105cb0 0x7ff8201a1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:04.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.045+0000 7ff828200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8201a2430 con 0x7ff820103800 2024-02-02T15:38:04.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.045+0000 7ff828200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8201a25a0 con 0x7ff820105cb0 2024-02-02T15:38:04.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.045+0000 7ff828200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8201a2760 con 0x7ff820068550 2024-02-02T15:38:04.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.045+0000 7ff825800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff820068550 0x7ff82019d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:04.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.045+0000 7ff825800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff820068550 0x7ff82019d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:46042/0 (socket says 172.21.15.1:46042) 2024-02-02T15:38:04.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.045+0000 7ff825800700 1 -- 172.21.15.1:0/3644358130 learned_addr learned my addr 172.21.15.1:0/3644358130 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:38:04.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.045+0000 7ff826200700 1 --2- 172.21.15.1:0/3644358130 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff820105cb0 0x7ff8201a1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:04.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.045+0000 7ff824e00700 1 --2- 172.21.15.1:0/3644358130 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff820103800 0x7ff82019d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:04.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.046+0000 7ff825800700 1 -- 172.21.15.1:0/3644358130 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff820105cb0 msgr2=0x7ff8201a1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:04.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.046+0000 7ff825800700 1 --2- 172.21.15.1:0/3644358130 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff820105cb0 0x7ff8201a1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.046+0000 7ff825800700 1 -- 172.21.15.1:0/3644358130 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff820103800 msgr2=0x7ff82019d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:04.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.046+0000 7ff825800700 1 --2- 172.21.15.1:0/3644358130 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff820103800 0x7ff82019d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.046+0000 7ff825800700 1 -- 172.21.15.1:0/3644358130 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8100095f0 con 0x7ff820068550 2024-02-02T15:38:04.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.046+0000 7ff825800700 1 --2- 172.21.15.1:0/3644358130 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff820068550 0x7ff82019d450 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ff810005910 tx=0x7ff810005720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:04.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.046+0000 7ff81e000700 1 -- 172.21.15.1:0/3644358130 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81001d070 con 0x7ff820068550 2024-02-02T15:38:04.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.046+0000 7ff828200700 1 -- 172.21.15.1:0/3644358130 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8201a29e0 con 0x7ff820068550 2024-02-02T15:38:04.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.046+0000 7ff81e000700 1 -- 172.21.15.1:0/3644358130 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff810022470 con 0x7ff820068550 2024-02-02T15:38:04.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.046+0000 7ff81e000700 1 -- 172.21.15.1:0/3644358130 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff810017650 con 0x7ff820068550 2024-02-02T15:38:04.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.047+0000 7ff828200700 1 -- 172.21.15.1:0/3644358130 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8201aa280 con 0x7ff820068550 2024-02-02T15:38:04.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.050+0000 7ff81e000700 1 -- 172.21.15.1:0/3644358130 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7ff8100225e0 con 0x7ff820068550 2024-02-02T15:38:04.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.051+0000 7ff83bcf4b80 1 Processor -- start 2024-02-02T15:38:04.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.051+0000 7ff81e000700 1 --2- 172.21.15.1:0/3644358130 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff8000a0690 0x7ff8000a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:04.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.051+0000 7ff83bcf4b80 1 -- start start 2024-02-02T15:38:04.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.051+0000 7ff81e000700 1 -- 172.21.15.1:0/3644358130 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9453+0+0 (secure 0 0 0) 0x7ff8100cd550 con 0x7ff820068550 2024-02-02T15:38:04.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.052+0000 7ff826200700 1 --2- 172.21.15.1:0/3644358130 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff8000a0690 0x7ff8000a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:04.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.052+0000 7ff83bcf4b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e3d9536010 0x55e3d95363f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:04.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.052+0000 7ff83bcf4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3d9536930 0x55e3d95412a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:04.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.052+0000 7ff83bcf4b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3d95417e0 0x55e3d9543bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:04.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.052+0000 7ff83bcf4b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3d94aa4c0 con 0x55e3d9536010 2024-02-02T15:38:04.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.052+0000 7ff83bcf4b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3d948eaa0 con 0x55e3d95417e0 2024-02-02T15:38:04.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.052+0000 7ff83bcf4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3d948c460 con 0x55e3d9536930 2024-02-02T15:38:04.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.052+0000 7ff825800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e3d9536010 0x55e3d95363f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:04.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.052+0000 7ff825800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e3d9536010 0x55e3d95363f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37300/0 (socket says 172.21.15.1:37300) 2024-02-02T15:38:04.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.052+0000 7ff825800700 1 -- 172.21.15.1:0/2137525820 learned_addr learned my addr 172.21.15.1:0/2137525820 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:38:04.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.052+0000 7ff824e00700 1 --2- 172.21.15.1:0/2137525820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3d9536930 0x55e3d95412a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:04.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.052+0000 7ff825800700 1 -- 172.21.15.1:0/2137525820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3d9536930 msgr2=0x55e3d95412a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:04.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.052+0000 7ff825800700 1 --2- 172.21.15.1:0/2137525820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3d9536930 0x55e3d95412a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.052+0000 7ff825800700 1 -- 172.21.15.1:0/2137525820 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3d95417e0 msgr2=0x55e3d9543bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:04.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.052+0000 7ff825800700 1 --2- 172.21.15.1:0/2137525820 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3d95417e0 0x55e3d9543bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.052+0000 7ff825800700 1 -- 172.21.15.1:0/2137525820 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e3d93093b0 con 0x55e3d9536010 2024-02-02T15:38:04.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.053+0000 7ff825800700 1 --2- 172.21.15.1:0/2137525820 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e3d9536010 0x55e3d95363f0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7ff81009a810 tx=0x7ff810022c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4252066da5012ec7 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:04.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.053+0000 7ff826200700 1 --2- 172.21.15.1:0/3644358130 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff8000a0690 0x7ff8000a2b50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff81800f540 tx=0x7ff818009920 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:04.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.053+0000 7ff816a00700 1 -- 172.21.15.1:0/2137525820 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81009a3c0 con 0x55e3d9536010 2024-02-02T15:38:04.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.053+0000 7ff816a00700 1 -- 172.21.15.1:0/2137525820 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff81009a1e0 con 0x55e3d9536010 2024-02-02T15:38:04.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.053+0000 7ff816a00700 1 -- 172.21.15.1:0/2137525820 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8100cd170 con 0x55e3d9536010 2024-02-02T15:38:04.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.054+0000 7ff83bcf4b80 1 -- 172.21.15.1:0/2137525820 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e3d9536010 msgr2=0x55e3d95363f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:04.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.054+0000 7ff83bcf4b80 1 --2- 172.21.15.1:0/2137525820 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e3d9536010 0x55e3d95363f0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7ff81009a810 tx=0x7ff810022c90 comp rx=0 tx=0).stop 2024-02-02T15:38:04.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.054+0000 7ff83bcf4b80 1 -- 172.21.15.1:0/2137525820 shutdown_connections 2024-02-02T15:38:04.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.054+0000 7ff83bcf4b80 1 --2- 172.21.15.1:0/2137525820 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3d95417e0 0x55e3d9543bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.054+0000 7ff83bcf4b80 1 --2- 172.21.15.1:0/2137525820 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e3d9536010 0x55e3d95363f0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.054+0000 7ff83bcf4b80 1 --2- 172.21.15.1:0/2137525820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3d9536930 0x55e3d95412a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.054+0000 7ff83bcf4b80 1 -- 172.21.15.1:0/2137525820 >> 172.21.15.1:0/2137525820 conn(0x55e3d9495db0 msgr2=0x55e3d95323b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:04.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.054+0000 7ff83bcf4b80 1 -- 172.21.15.1:0/2137525820 shutdown_connections 2024-02-02T15:38:04.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.054+0000 7ff83bcf4b80 1 -- 172.21.15.1:0/2137525820 wait complete. 2024-02-02T15:38:04.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.055+0000 7ff83bcf4b80 1 -- start start 2024-02-02T15:38:04.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.055+0000 7ff83bcf4b80 1 Processor -- start 2024-02-02T15:38:04.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.055+0000 7ff83bcf4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3d9536010 0x55e3d9541bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:04.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.055+0000 7ff83bcf4b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3d9536930 0x55e3d9542100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:04.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.055+0000 7ff83bcf4b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e3d95417e0 0x55e3d9542640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:04.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.055+0000 7ff83bcf4b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3d9487d50 con 0x55e3d95417e0 2024-02-02T15:38:04.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.055+0000 7ff83bcf4b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3d9423d80 con 0x55e3d9536930 2024-02-02T15:38:04.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.055+0000 7ff83bcf4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3d9423a20 con 0x55e3d9536010 2024-02-02T15:38:04.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.055+0000 7ff83bcf4b80 10 client.0 _ll_register_callbacks cb 0x55e3d9425290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:38:04.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.056+0000 7ff825800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3d9536010 0x55e3d9541bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:04.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.056+0000 7ff824e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3d9536930 0x55e3d9542100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:04.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.056+0000 7ff826200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e3d95417e0 0x55e3d9542640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:04.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.056+0000 7ff825800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3d9536010 0x55e3d9541bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:46052/0 (socket says 172.21.15.1:46052) 2024-02-02T15:38:04.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.056+0000 7ff825800700 1 -- 172.21.15.1:0/3504833647 learned_addr learned my addr 172.21.15.1:0/3504833647 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:38:04.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.056+0000 7ff825800700 1 -- 172.21.15.1:0/3504833647 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3d9536930 msgr2=0x55e3d9542100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:04.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.056+0000 7ff825800700 1 --2- 172.21.15.1:0/3504833647 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3d9536930 0x55e3d9542100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.056+0000 7ff825800700 1 -- 172.21.15.1:0/3504833647 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e3d95417e0 msgr2=0x55e3d9542640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:04.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.056+0000 7ff825800700 1 --2- 172.21.15.1:0/3504833647 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e3d95417e0 0x55e3d9542640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.056+0000 7ff825800700 1 -- 172.21.15.1:0/3504833647 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8201aa280 con 0x55e3d9536010 2024-02-02T15:38:04.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.056+0000 7ff825800700 1 --2- 172.21.15.1:0/3504833647 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3d9536010 0x55e3d9541bc0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7ff810000c00 tx=0x7ff810022af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50b278b5a75d1e56 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:04.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.056+0000 7ff80f400700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:38:04.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.057+0000 7ff80f400700 1 -- 172.21.15.1:0/3504833647 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8100a2860 con 0x55e3d9536010 2024-02-02T15:38:04.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.057+0000 7ff80f400700 1 -- 172.21.15.1:0/3504833647 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9453+0+0 (secure 0 0 0) 0x7ff81009a880 con 0x55e3d9536010 2024-02-02T15:38:04.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.057+0000 7ff83bcf4b80 1 -- 172.21.15.1:0/3504833647 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e3d952a0f0 con 0x55e3d9536010 2024-02-02T15:38:04.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.057+0000 7ff80f400700 1 -- 172.21.15.1:0/3504833647 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff81009a3c0 con 0x55e3d9536010 2024-02-02T15:38:04.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.057+0000 7ff83bcf4b80 10 client.28552 fetch_fsmap learned FSMap version 50 2024-02-02T15:38:04.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.057+0000 7ff83bcf4b80 1 -- 172.21.15.1:0/3504833647 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55e3d92b5b60 con 0x55e3d9536010 2024-02-02T15:38:04.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.057+0000 7ff80f400700 1 -- 172.21.15.1:0/3504833647 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e3d952a0f0 con 0x55e3d9536010 2024-02-02T15:38:04.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.057+0000 7ff80f400700 1 -- 172.21.15.1:0/3504833647 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 50) v1 ==== 7478+0+0 (secure 0 0 0) 0x7ff8100af030 con 0x55e3d9536010 2024-02-02T15:38:04.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.057+0000 7ff83bcf4b80 10 client.28552 fetch_fsmap finished waiting for FSMap version 50 2024-02-02T15:38:04.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.057+0000 7ff83bcf4b80 10 client.28552 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-02-02T15:38:04.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.057+0000 7ff83bcf4b80 20 client.28552 populate_metadata read hostname 'smithi001' 2024-02-02T15:38:04.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.058+0000 7ff83bcf4b80 1 --2- 172.21.15.1:0/3504833647 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55e3d9550950 0x55e3d9552d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:04.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.058+0000 7ff83bcf4b80 4 client.28552 mds_command: new command op to 24514 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:38:04.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.058+0000 7ff83bcf4b80 1 -- 172.21.15.1:0/3504833647 --> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e3d9480dd0 con 0x55e3d9550950 2024-02-02T15:38:04.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.058+0000 7ff824e00700 1 --2- 172.21.15.1:0/3504833647 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55e3d9550950 0x55e3d9552d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:04.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.058+0000 7ff824e00700 1 --2- 172.21.15.1:0/3504833647 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55e3d9550950 0x55e3d9552d70 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c4d9086f7610d923 server_cookie=132345ae94dac539 in_seq=0 out_seq=0 2024-02-02T15:38:04.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.058+0000 7ff80f400700 10 client.28552 ms_handle_connect on v2:172.21.15.142:6836/2870289019 2024-02-02T15:38:04.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.059+0000 7ff80f400700 1 -- 172.21.15.1:0/3504833647 <== mds.3 v2:172.21.15.142:6836/2870289019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55e3d9480dd0 con 0x55e3d9550950 2024-02-02T15:38:04.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.059+0000 7ff80f400700 10 client.28552 handle_command_reply: tid=0 2024-02-02T15:38:04.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.059+0000 7ff83bcf4b80 1 client.28552 shutdown 2024-02-02T15:38:04.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.059+0000 7ff83bcf4b80 1 -- 172.21.15.1:0/3504833647 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3d9536010 msgr2=0x55e3d9541bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:04.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.059+0000 7ff83bcf4b80 1 --2- 172.21.15.1:0/3504833647 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3d9536010 0x55e3d9541bc0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7ff810000c00 tx=0x7ff810022af0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.059+0000 7ff83bcf4b80 1 -- 172.21.15.1:0/3504833647 shutdown_connections 2024-02-02T15:38:04.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.059+0000 7ff83bcf4b80 1 --2- 172.21.15.1:0/3504833647 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55e3d9550950 0x55e3d9552d70 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.059+0000 7ff83bcf4b80 1 --2- 172.21.15.1:0/3504833647 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e3d95417e0 0x55e3d9542640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.059+0000 7ff83bcf4b80 1 --2- 172.21.15.1:0/3504833647 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3d9536010 0x55e3d9541bc0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.059+0000 7ff83bcf4b80 1 --2- 172.21.15.1:0/3504833647 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3d9536930 0x55e3d9542100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.059+0000 7ff80f400700 0 client.28552 ms_handle_reset on v2:172.21.15.142:6836/2870289019 2024-02-02T15:38:04.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.059+0000 7ff83bcf4b80 1 -- 172.21.15.1:0/3504833647 >> 172.21.15.1:0/3504833647 conn(0x55e3d9495db0 msgr2=0x55e3d9531fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:04.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.059+0000 7ff83bcf4b80 1 -- 172.21.15.1:0/3504833647 shutdown_connections 2024-02-02T15:38:04.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.059+0000 7ff83bcf4b80 1 -- 172.21.15.1:0/3504833647 wait complete. 2024-02-02T15:38:04.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.060+0000 7ff83bcf4b80 20 client.28552 trim_cache size 0 max 16384 2024-02-02T15:38:04.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.074+0000 7ff83bcf4b80 1 Processor -- start 2024-02-02T15:38:04.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.074+0000 7ff83bcf4b80 1 -- start start 2024-02-02T15:38:04.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.074+0000 7ff83bcf4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3d9536010 0x55e3d954d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:04.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.074+0000 7ff83bcf4b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3d9536930 0x55e3d954c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:04.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.074+0000 7ff83bcf4b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e3d95417e0 0x55e3d953bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:04.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.074+0000 7ff83bcf4b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3d930b6a0 con 0x55e3d95417e0 2024-02-02T15:38:04.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.074+0000 7ff83bcf4b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3d9533a60 con 0x55e3d9536930 2024-02-02T15:38:04.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.074+0000 7ff83bcf4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3d9540f00 con 0x55e3d9536010 2024-02-02T15:38:04.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.075+0000 7ff825800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3d9536010 0x55e3d954d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:04.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.075+0000 7ff825800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3d9536010 0x55e3d954d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:46058/0 (socket says 172.21.15.1:46058) 2024-02-02T15:38:04.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.075+0000 7ff825800700 1 -- 172.21.15.1:0/3953449292 learned_addr learned my addr 172.21.15.1:0/3953449292 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:38:04.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.075+0000 7ff826200700 1 --2- 172.21.15.1:0/3953449292 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e3d95417e0 0x55e3d953bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:04.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.075+0000 7ff825800700 1 -- 172.21.15.1:0/3953449292 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3d9536930 msgr2=0x55e3d954c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:04.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.075+0000 7ff824e00700 1 --2- 172.21.15.1:0/3953449292 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3d9536930 0x55e3d954c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:04.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.075+0000 7ff825800700 1 --2- 172.21.15.1:0/3953449292 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3d9536930 0x55e3d954c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.075+0000 7ff825800700 1 -- 172.21.15.1:0/3953449292 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e3d95417e0 msgr2=0x55e3d953bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:04.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.075+0000 7ff825800700 1 --2- 172.21.15.1:0/3953449292 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e3d95417e0 0x55e3d953bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.075+0000 7ff825800700 1 -- 172.21.15.1:0/3953449292 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e3d944dea0 con 0x55e3d9536010 2024-02-02T15:38:04.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.075+0000 7ff824e00700 1 --2- 172.21.15.1:0/3953449292 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3d9536930 0x55e3d954c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:38:04.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.075+0000 7ff826200700 1 --2- 172.21.15.1:0/3953449292 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e3d95417e0 0x55e3d953bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:38:04.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.076+0000 7ff825800700 1 --2- 172.21.15.1:0/3953449292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3d9536010 0x55e3d954d0b0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ff810098bd0 tx=0x7ff8100ca9b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e32ef0c0b0e48cf server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:04.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.076+0000 7ff817400700 1 -- 172.21.15.1:0/3953449292 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff810096bc0 con 0x55e3d9536010 2024-02-02T15:38:04.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.076+0000 7ff817400700 1 -- 172.21.15.1:0/3953449292 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff81009af80 con 0x55e3d9536010 2024-02-02T15:38:04.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.076+0000 7ff817400700 1 -- 172.21.15.1:0/3953449292 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff810099d70 con 0x55e3d9536010 2024-02-02T15:38:04.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.077+0000 7ff83bcf4b80 1 -- 172.21.15.1:0/3953449292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3d9536010 msgr2=0x55e3d954d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:04.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.077+0000 7ff83bcf4b80 1 --2- 172.21.15.1:0/3953449292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3d9536010 0x55e3d954d0b0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ff810098bd0 tx=0x7ff8100ca9b0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.077+0000 7ff83bcf4b80 1 -- 172.21.15.1:0/3953449292 shutdown_connections 2024-02-02T15:38:04.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.077+0000 7ff83bcf4b80 1 --2- 172.21.15.1:0/3953449292 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e3d95417e0 0x55e3d953bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.077+0000 7ff83bcf4b80 1 --2- 172.21.15.1:0/3953449292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3d9536010 0x55e3d954d0b0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.077+0000 7ff83bcf4b80 1 --2- 172.21.15.1:0/3953449292 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3d9536930 0x55e3d954c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.077+0000 7ff83bcf4b80 1 -- 172.21.15.1:0/3953449292 >> 172.21.15.1:0/3953449292 conn(0x55e3d9495db0 msgr2=0x55e3d953b1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:04.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.078+0000 7ff83bcf4b80 1 -- 172.21.15.1:0/3953449292 shutdown_connections 2024-02-02T15:38:04.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.078+0000 7ff83bcf4b80 1 -- 172.21.15.1:0/3953449292 wait complete. 2024-02-02T15:38:04.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.079+0000 7ff83bcf4b80 1 -- start start 2024-02-02T15:38:04.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.079+0000 7ff83bcf4b80 1 Processor -- start 2024-02-02T15:38:04.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.080+0000 7ff83bcf4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3d9536010 0x55e3d9543380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:04.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.080+0000 7ff83bcf4b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3d9536930 0x55e3d95438c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:04.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.080+0000 7ff83bcf4b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e3d95417e0 0x55e3d95a1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:04.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.080+0000 7ff83bcf4b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3d9544030 con 0x55e3d95417e0 2024-02-02T15:38:04.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.080+0000 7ff83bcf4b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3d95441a0 con 0x55e3d9536930 2024-02-02T15:38:04.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.080+0000 7ff83bcf4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3d95a19b0 con 0x55e3d9536010 2024-02-02T15:38:04.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.080+0000 7ff83bcf4b80 10 client.0 _ll_register_callbacks cb 0x55e3d9425290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:38:04.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.080+0000 7ff824e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3d9536930 0x55e3d95438c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:04.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.080+0000 7ff824e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3d9536930 0x55e3d95438c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:40892/0 (socket says 172.21.15.1:40892) 2024-02-02T15:38:04.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.080+0000 7ff824e00700 1 -- 172.21.15.1:0/4238490300 learned_addr learned my addr 172.21.15.1:0/4238490300 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:38:04.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.080+0000 7ff825800700 1 --2- 172.21.15.1:0/4238490300 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3d9536010 0x55e3d9543380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:04.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.080+0000 7ff826200700 1 --2- 172.21.15.1:0/4238490300 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e3d95417e0 0x55e3d95a1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:04.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.080+0000 7ff824e00700 1 -- 172.21.15.1:0/4238490300 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3d9536010 msgr2=0x55e3d9543380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:04.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.080+0000 7ff824e00700 1 --2- 172.21.15.1:0/4238490300 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3d9536010 0x55e3d9543380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.080+0000 7ff824e00700 1 -- 172.21.15.1:0/4238490300 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e3d95417e0 msgr2=0x55e3d95a1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:04.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.080+0000 7ff824e00700 1 --2- 172.21.15.1:0/4238490300 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e3d95417e0 0x55e3d95a1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.080+0000 7ff824e00700 1 -- 172.21.15.1:0/4238490300 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e3d92b5b60 con 0x55e3d9536930 2024-02-02T15:38:04.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.081+0000 7ff824e00700 1 --2- 172.21.15.1:0/4238490300 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3d9536930 0x55e3d95438c0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x55e3d9540ea0 tx=0x7ff808005e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7696d47bf656a1bb server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:04.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.081+0000 7ff80f400700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:38:04.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.081+0000 7ff80f400700 1 -- 172.21.15.1:0/4238490300 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff808017040 con 0x55e3d9536930 2024-02-02T15:38:04.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.081+0000 7ff80f400700 1 -- 172.21.15.1:0/4238490300 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9453+0+0 (secure 0 0 0) 0x7ff808010d30 con 0x55e3d9536930 2024-02-02T15:38:04.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.081+0000 7ff83bcf4b80 1 -- 172.21.15.1:0/4238490300 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff81009a1e0 con 0x55e3d9536930 2024-02-02T15:38:04.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.081+0000 7ff80f400700 1 -- 172.21.15.1:0/4238490300 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff808004830 con 0x55e3d9536930 2024-02-02T15:38:04.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.081+0000 7ff83bcf4b80 10 client.28550 fetch_fsmap learned FSMap version 50 2024-02-02T15:38:04.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.081+0000 7ff83bcf4b80 1 -- 172.21.15.1:0/4238490300 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55e3d930dcd0 con 0x55e3d9536930 2024-02-02T15:38:04.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.082+0000 7ff80f400700 1 -- 172.21.15.1:0/4238490300 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff81009a1e0 con 0x55e3d9536930 2024-02-02T15:38:04.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.082+0000 7ff80f400700 1 -- 172.21.15.1:0/4238490300 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 50) v1 ==== 7478+0+0 (secure 0 0 0) 0x7ff808024030 con 0x55e3d9536930 2024-02-02T15:38:04.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.082+0000 7ff83bcf4b80 10 client.28550 fetch_fsmap finished waiting for FSMap version 50 2024-02-02T15:38:04.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.082+0000 7ff83bcf4b80 10 client.28550 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-02-02T15:38:04.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.082+0000 7ff83bcf4b80 20 client.28550 populate_metadata read hostname 'smithi001' 2024-02-02T15:38:04.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.082+0000 7ff83bcf4b80 1 --2- 172.21.15.1:0/4238490300 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55e3d95a3150 0x55e3d95a5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:04.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.082+0000 7ff83bcf4b80 4 client.28550 mds_command: new command op to 24514 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:38:04.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.082+0000 7ff83bcf4b80 1 -- 172.21.15.1:0/4238490300 --> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e3d9446f50 con 0x55e3d95a3150 2024-02-02T15:38:04.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.082+0000 7ff825800700 1 --2- 172.21.15.1:0/4238490300 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55e3d95a3150 0x55e3d95a5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:04.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.083+0000 7ff825800700 1 --2- 172.21.15.1:0/4238490300 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55e3d95a3150 0x55e3d95a5570 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3907c178f43f95c6 server_cookie=2dbb59a71709117b in_seq=0 out_seq=0 2024-02-02T15:38:04.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.083+0000 7ff80f400700 10 client.28550 ms_handle_connect on v2:172.21.15.142:6836/2870289019 2024-02-02T15:38:04.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.083+0000 7ff80f400700 1 -- 172.21.15.1:0/4238490300 <== mds.3 v2:172.21.15.142:6836/2870289019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e3d9446f50 con 0x55e3d95a3150 2024-02-02T15:38:04.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.083+0000 7ff80f400700 10 client.28550 handle_command_reply: tid=0 2024-02-02T15:38:04.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.083+0000 7ff83bcf4b80 1 client.28550 shutdown 2024-02-02T15:38:04.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.083+0000 7ff83bcf4b80 1 -- 172.21.15.1:0/4238490300 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3d9536930 msgr2=0x55e3d95438c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:04.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.083+0000 7ff83bcf4b80 1 --2- 172.21.15.1:0/4238490300 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3d9536930 0x55e3d95438c0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x55e3d9540ea0 tx=0x7ff808005e40 comp rx=0 tx=0).stop 2024-02-02T15:38:04.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.083+0000 7ff83bcf4b80 1 -- 172.21.15.1:0/4238490300 shutdown_connections 2024-02-02T15:38:04.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.083+0000 7ff83bcf4b80 1 --2- 172.21.15.1:0/4238490300 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55e3d95a3150 0x55e3d95a5570 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.083+0000 7ff83bcf4b80 1 --2- 172.21.15.1:0/4238490300 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e3d95417e0 0x55e3d95a1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.083+0000 7ff83bcf4b80 1 --2- 172.21.15.1:0/4238490300 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3d9536010 0x55e3d9543380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.083+0000 7ff83bcf4b80 1 --2- 172.21.15.1:0/4238490300 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3d9536930 0x55e3d95438c0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.083+0000 7ff80f400700 0 client.28550 ms_handle_reset on v2:172.21.15.142:6836/2870289019 2024-02-02T15:38:04.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.083+0000 7ff83bcf4b80 1 -- 172.21.15.1:0/4238490300 >> 172.21.15.1:0/4238490300 conn(0x55e3d9495db0 msgr2=0x55e3d954c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:04.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.083+0000 7ff83bcf4b80 1 -- 172.21.15.1:0/4238490300 shutdown_connections 2024-02-02T15:38:04.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.083+0000 7ff83bcf4b80 1 -- 172.21.15.1:0/4238490300 wait complete. 2024-02-02T15:38:04.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.083+0000 7ff83bcf4b80 20 client.28550 trim_cache size 0 max 16384 2024-02-02T15:38:04.086 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:38:04.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.084+0000 7ff828200700 1 -- 172.21.15.1:0/3644358130 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff8000a0690 msgr2=0x7ff8000a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:04.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.084+0000 7ff828200700 1 --2- 172.21.15.1:0/3644358130 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff8000a0690 0x7ff8000a2b50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff81800f540 tx=0x7ff818009920 comp rx=0 tx=0).stop 2024-02-02T15:38:04.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.084+0000 7ff828200700 1 -- 172.21.15.1:0/3644358130 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff820068550 msgr2=0x7ff82019d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:04.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.084+0000 7ff828200700 1 --2- 172.21.15.1:0/3644358130 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff820068550 0x7ff82019d450 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ff810005910 tx=0x7ff810005720 comp rx=0 tx=0).stop 2024-02-02T15:38:04.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.084+0000 7ff828200700 1 -- 172.21.15.1:0/3644358130 shutdown_connections 2024-02-02T15:38:04.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.084+0000 7ff828200700 1 --2- 172.21.15.1:0/3644358130 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff8000a0690 0x7ff8000a2b50 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.084+0000 7ff828200700 1 --2- 172.21.15.1:0/3644358130 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff820105cb0 0x7ff8201a1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.084+0000 7ff828200700 1 --2- 172.21.15.1:0/3644358130 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff820068550 0x7ff82019d450 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.084+0000 7ff828200700 1 --2- 172.21.15.1:0/3644358130 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff820103800 0x7ff82019d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.084+0000 7ff828200700 1 -- 172.21.15.1:0/3644358130 >> 172.21.15.1:0/3644358130 conn(0x7ff8200757d0 msgr2=0x7ff8201027e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:04.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.084+0000 7ff828200700 1 -- 172.21.15.1:0/3644358130 shutdown_connections 2024-02-02T15:38:04.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.084+0000 7ff828200700 1 -- 172.21.15.1:0/3644358130 wait complete. 2024-02-02T15:38:04.105 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-02T15:38:04.264 INFO:tasks.workunit.client.0.smithi001.stdout: 1 617717 5.66 MB/sec execute 458 sec latency 63.699 ms 2024-02-02T15:38:04.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.330+0000 7f4f26e00700 1 -- 172.21.15.1:0/1380727612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f200689b0 msgr2=0x7f4f2010dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:04.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.330+0000 7f4f26e00700 1 --2- 172.21.15.1:0/1380727612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f200689b0 0x7f4f2010dc20 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f4f18009fc0 tx=0x7f4f1800b330 comp rx=0 tx=0).stop 2024-02-02T15:38:04.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.330+0000 7f4f26e00700 1 -- 172.21.15.1:0/1380727612 shutdown_connections 2024-02-02T15:38:04.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.330+0000 7f4f26e00700 1 --2- 172.21.15.1:0/1380727612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f200689b0 0x7f4f2010dc20 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.330+0000 7f4f26e00700 1 --2- 172.21.15.1:0/1380727612 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f20106ed0 0x7f4f201072b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.330+0000 7f4f26e00700 1 --2- 172.21.15.1:0/1380727612 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f20068010 0x7f4f20068470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.330+0000 7f4f26e00700 1 -- 172.21.15.1:0/1380727612 >> 172.21.15.1:0/1380727612 conn(0x7f4f200757d0 msgr2=0x7f4f20075be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:04.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.330+0000 7f4f26e00700 1 -- 172.21.15.1:0/1380727612 shutdown_connections 2024-02-02T15:38:04.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.330+0000 7f4f26e00700 1 -- 172.21.15.1:0/1380727612 wait complete. 2024-02-02T15:38:04.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.331+0000 7f4f26e00700 1 Processor -- start 2024-02-02T15:38:04.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.332+0000 7f4f26e00700 1 -- start start 2024-02-02T15:38:04.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.332+0000 7f4f26e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f20068010 0x7f4f20199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:04.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.332+0000 7f4f26e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f200689b0 0x7f4f20199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:04.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.332+0000 7f4f26e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f20106ed0 0x7f4f2019dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:04.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.332+0000 7f4f26e00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f2019e200 con 0x7f4f20106ed0 2024-02-02T15:38:04.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.332+0000 7f4f26e00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f2019e370 con 0x7f4f20068010 2024-02-02T15:38:04.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.332+0000 7f4f26e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f2019e530 con 0x7f4f200689b0 2024-02-02T15:38:04.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.333+0000 7f4f1fe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f20068010 0x7f4f20199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:04.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.333+0000 7f4f1fe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f20068010 0x7f4f20199220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:40908/0 (socket says 172.21.15.1:40908) 2024-02-02T15:38:04.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.333+0000 7f4f1f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f200689b0 0x7f4f20199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:04.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.333+0000 7f4f1fe00700 1 -- 172.21.15.1:0/2028640366 learned_addr learned my addr 172.21.15.1:0/2028640366 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:38:04.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.333+0000 7f4f24e00700 1 --2- 172.21.15.1:0/2028640366 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f20106ed0 0x7f4f2019dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:04.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.333+0000 7f4f1f400700 1 -- 172.21.15.1:0/2028640366 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f20068010 msgr2=0x7f4f20199220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:04.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.333+0000 7f4f1f400700 1 --2- 172.21.15.1:0/2028640366 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f20068010 0x7f4f20199220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.333+0000 7f4f1f400700 1 -- 172.21.15.1:0/2028640366 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f20106ed0 msgr2=0x7f4f2019dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:04.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.333+0000 7f4f1f400700 1 --2- 172.21.15.1:0/2028640366 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f20106ed0 0x7f4f2019dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.333+0000 7f4f1f400700 1 -- 172.21.15.1:0/2028640366 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f18009c40 con 0x7f4f200689b0 2024-02-02T15:38:04.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.333+0000 7f4f24e00700 1 --2- 172.21.15.1:0/2028640366 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f20106ed0 0x7f4f2019dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:38:04.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.333+0000 7f4f1f400700 1 --2- 172.21.15.1:0/2028640366 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f200689b0 0x7f4f20199760 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f4f0800e9c0 tx=0x7f4f0800ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:04.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.334+0000 7f4f1cc00700 1 -- 172.21.15.1:0/2028640366 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f0800c9c0 con 0x7f4f200689b0 2024-02-02T15:38:04.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.334+0000 7f4f1cc00700 1 -- 172.21.15.1:0/2028640366 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f08010430 con 0x7f4f200689b0 2024-02-02T15:38:04.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.334+0000 7f4f26e00700 1 -- 172.21.15.1:0/2028640366 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f2019e810 con 0x7f4f200689b0 2024-02-02T15:38:04.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.334+0000 7f4f1cc00700 1 -- 172.21.15.1:0/2028640366 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f08018680 con 0x7f4f200689b0 2024-02-02T15:38:04.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.334+0000 7f4f26e00700 1 -- 172.21.15.1:0/2028640366 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f201a6050 con 0x7f4f200689b0 2024-02-02T15:38:04.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.338+0000 7f4f1cc00700 1 -- 172.21.15.1:0/2028640366 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f4f08010860 con 0x7f4f200689b0 2024-02-02T15:38:04.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.339+0000 7f4f3a87bb80 1 Processor -- start 2024-02-02T15:38:04.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.340+0000 7f4f3a87bb80 1 -- start start 2024-02-02T15:38:04.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.340+0000 7f4f3a87bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5575291a1010 0x5575291a13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:04.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.340+0000 7f4f3a87bb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5575291a1930 0x5575291ac2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:04.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.340+0000 7f4f3a87bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5575291ac7e0 0x5575291aebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:04.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.340+0000 7f4f3a87bb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575291154c0 con 0x5575291a1930 2024-02-02T15:38:04.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.340+0000 7f4f3a87bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575290f9aa0 con 0x5575291ac7e0 2024-02-02T15:38:04.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.340+0000 7f4f3a87bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575290f7460 con 0x5575291a1010 2024-02-02T15:38:04.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.340+0000 7f4f1fe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5575291a1930 0x5575291ac2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:04.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.340+0000 7f4f24e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5575291a1010 0x5575291a13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:04.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.340+0000 7f4f1fe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5575291a1930 0x5575291ac2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37370/0 (socket says 172.21.15.1:37370) 2024-02-02T15:38:04.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.340+0000 7f4f1fe00700 1 -- 172.21.15.1:0/1444615237 learned_addr learned my addr 172.21.15.1:0/1444615237 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:38:04.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.340+0000 7f4f1f400700 1 --2- 172.21.15.1:0/1444615237 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5575291ac7e0 0x5575291aebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:04.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.341+0000 7f4f24e00700 1 -- 172.21.15.1:0/1444615237 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5575291ac7e0 msgr2=0x5575291aebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:04.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.341+0000 7f4f24e00700 1 --2- 172.21.15.1:0/1444615237 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5575291ac7e0 0x5575291aebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.341+0000 7f4f24e00700 1 -- 172.21.15.1:0/1444615237 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5575291a1930 msgr2=0x5575291ac2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:04.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.341+0000 7f4f24e00700 1 --2- 172.21.15.1:0/1444615237 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5575291a1930 0x5575291ac2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.341+0000 7f4f1cc00700 1 --2- 172.21.15.1:0/2028640366 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f4f000a0690 0x7f4f000a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:04.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.341+0000 7f4f24e00700 1 -- 172.21.15.1:0/1444615237 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557528f743b0 con 0x5575291a1010 2024-02-02T15:38:04.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.341+0000 7f4f1cc00700 1 -- 172.21.15.1:0/2028640366 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f4f08014070 con 0x7f4f200689b0 2024-02-02T15:38:04.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.341+0000 7f4f24e00700 1 --2- 172.21.15.1:0/1444615237 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5575291a1010 0x5575291a13f0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f4f18015000 tx=0x7f4f18009350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f8f79cf6acfbcb0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:04.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.341+0000 7f4f24e00700 1 --2- 172.21.15.1:0/2028640366 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f4f000a0690 0x7f4f000a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:04.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.342+0000 7f4f15600700 1 -- 172.21.15.1:0/1444615237 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f1800e050 con 0x5575291a1010 2024-02-02T15:38:04.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.342+0000 7f4f15600700 1 -- 172.21.15.1:0/1444615237 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f1802e070 con 0x5575291a1010 2024-02-02T15:38:04.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.342+0000 7f4f15600700 1 -- 172.21.15.1:0/1444615237 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f18028860 con 0x5575291a1010 2024-02-02T15:38:04.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.342+0000 7f4f3a87bb80 1 -- 172.21.15.1:0/1444615237 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5575291a1010 msgr2=0x5575291a13f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:04.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.342+0000 7f4f3a87bb80 1 --2- 172.21.15.1:0/1444615237 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5575291a1010 0x5575291a13f0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f4f18015000 tx=0x7f4f18009350 comp rx=0 tx=0).stop 2024-02-02T15:38:04.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.343+0000 7f4f24e00700 1 --2- 172.21.15.1:0/2028640366 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f4f000a0690 0x7f4f000a2b50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4f18027000 tx=0x7f4f1802a000 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:04.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.343+0000 7f4f3a87bb80 1 -- 172.21.15.1:0/1444615237 shutdown_connections 2024-02-02T15:38:04.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.343+0000 7f4f3a87bb80 1 --2- 172.21.15.1:0/1444615237 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5575291ac7e0 0x5575291aebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.343+0000 7f4f3a87bb80 1 --2- 172.21.15.1:0/1444615237 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5575291a1010 0x5575291a13f0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.343+0000 7f4f3a87bb80 1 --2- 172.21.15.1:0/1444615237 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5575291a1930 0x5575291ac2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.343+0000 7f4f3a87bb80 1 -- 172.21.15.1:0/1444615237 >> 172.21.15.1:0/1444615237 conn(0x557529100db0 msgr2=0x55752919d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:04.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.343+0000 7f4f3a87bb80 1 -- 172.21.15.1:0/1444615237 shutdown_connections 2024-02-02T15:38:04.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.343+0000 7f4f3a87bb80 1 -- 172.21.15.1:0/1444615237 wait complete. 2024-02-02T15:38:04.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.343+0000 7f4f3a87bb80 1 -- start start 2024-02-02T15:38:04.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.343+0000 7f4f3a87bb80 1 Processor -- start 2024-02-02T15:38:04.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.344+0000 7f4f3a87bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5575291a1010 0x5575291ac190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:04.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.344+0000 7f4f3a87bb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5575291a1930 0x5575291a93a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:04.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.344+0000 7f4f3a87bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5575291ac7e0 0x5575291a9960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:04.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.344+0000 7f4f1fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5575291a1010 0x5575291ac190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:04.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.344+0000 7f4f1fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5575291a1010 0x5575291ac190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:46114/0 (socket says 172.21.15.1:46114) 2024-02-02T15:38:04.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.344+0000 7f4f1fe00700 1 -- 172.21.15.1:0/2769353557 learned_addr learned my addr 172.21.15.1:0/2769353557 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:38:04.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.344+0000 7f4f24e00700 1 --2- 172.21.15.1:0/2769353557 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5575291ac7e0 0x5575291a9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:04.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.344+0000 7f4f3a87bb80 1 -- 172.21.15.1:0/2769353557 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575290f2d50 con 0x5575291a1930 2024-02-02T15:38:04.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.344+0000 7f4f3a87bb80 1 -- 172.21.15.1:0/2769353557 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55752908ed80 con 0x5575291ac7e0 2024-02-02T15:38:04.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.344+0000 7f4f3a87bb80 1 -- 172.21.15.1:0/2769353557 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55752908ea20 con 0x5575291a1010 2024-02-02T15:38:04.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.345+0000 7f4f3a87bb80 10 client.0 _ll_register_callbacks cb 0x557529090290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:38:04.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.345+0000 7f4f1f400700 1 --2- 172.21.15.1:0/2769353557 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5575291a1930 0x5575291a93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:04.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.345+0000 7f4f24e00700 1 -- 172.21.15.1:0/2769353557 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5575291a1010 msgr2=0x5575291ac190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:04.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.345+0000 7f4f24e00700 1 --2- 172.21.15.1:0/2769353557 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5575291a1010 0x5575291ac190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.345+0000 7f4f24e00700 1 -- 172.21.15.1:0/2769353557 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5575291a1930 msgr2=0x5575291a93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:04.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.345+0000 7f4f24e00700 1 --2- 172.21.15.1:0/2769353557 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5575291a1930 0x5575291a93a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.345+0000 7f4f24e00700 1 -- 172.21.15.1:0/2769353557 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f2010e5a0 con 0x5575291ac7e0 2024-02-02T15:38:04.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.345+0000 7f4f1fe00700 1 --2- 172.21.15.1:0/2769353557 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5575291a1010 0x5575291ac190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:38:04.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.345+0000 7f4f24e00700 1 --2- 172.21.15.1:0/2769353557 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5575291ac7e0 0x5575291a9960 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f4f180093c0 tx=0x7f4f18007840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c283804d84c77a2 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:04.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.345+0000 7f4f0e000700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:38:04.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.345+0000 7f4f0e000700 1 -- 172.21.15.1:0/2769353557 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f1800e050 con 0x5575291ac7e0 2024-02-02T15:38:04.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.345+0000 7f4f3a87bb80 1 -- 172.21.15.1:0/2769353557 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5575291950f0 con 0x5575291ac7e0 2024-02-02T15:38:04.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.346+0000 7f4f0e000700 1 -- 172.21.15.1:0/2769353557 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f4f1802d070 con 0x5575291ac7e0 2024-02-02T15:38:04.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.346+0000 7f4f0e000700 1 -- 172.21.15.1:0/2769353557 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4f1802c030 con 0x5575291ac7e0 2024-02-02T15:38:04.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.346+0000 7f4f3a87bb80 10 client.28562 fetch_fsmap learned FSMap version 50 2024-02-02T15:38:04.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.346+0000 7f4f3a87bb80 1 -- 172.21.15.1:0/2769353557 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x557528f20b60 con 0x5575291ac7e0 2024-02-02T15:38:04.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.346+0000 7f4f0e000700 1 -- 172.21.15.1:0/2769353557 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5575291950f0 con 0x5575291ac7e0 2024-02-02T15:38:04.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.346+0000 7f4f0e000700 1 -- 172.21.15.1:0/2769353557 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 50) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f4f1801b430 con 0x5575291ac7e0 2024-02-02T15:38:04.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.346+0000 7f4f3a87bb80 10 client.28562 fetch_fsmap finished waiting for FSMap version 50 2024-02-02T15:38:04.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.346+0000 7f4f3a87bb80 10 client.28562 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-02-02T15:38:04.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.346+0000 7f4f3a87bb80 20 client.28562 populate_metadata read hostname 'smithi001' 2024-02-02T15:38:04.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.346+0000 7f4f3a87bb80 1 --2- 172.21.15.1:0/2769353557 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x5575291bbd30 0x5575291be150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:04.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.346+0000 7f4f3a87bb80 4 client.28562 mds_command: new command op to 24518 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:38:04.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.346+0000 7f4f3a87bb80 1 -- 172.21.15.1:0/2769353557 --> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5575290ebdd0 con 0x5575291bbd30 2024-02-02T15:38:04.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.347+0000 7f4f1fe00700 1 --2- 172.21.15.1:0/2769353557 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x5575291bbd30 0x5575291be150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:04.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.347+0000 7f4f1fe00700 1 --2- 172.21.15.1:0/2769353557 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x5575291bbd30 0x5575291be150 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=33b37f0fcbe8a2f5 server_cookie=d9abb46d48f99feb in_seq=0 out_seq=0 2024-02-02T15:38:04.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.347+0000 7f4f0e000700 10 client.28562 ms_handle_connect on v2:172.21.15.71:6836/628016199 2024-02-02T15:38:04.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.348+0000 7f4f0e000700 1 -- 172.21.15.1:0/2769353557 <== mds.2 v2:172.21.15.71:6836/628016199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5575290ebdd0 con 0x5575291bbd30 2024-02-02T15:38:04.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.348+0000 7f4f0e000700 10 client.28562 handle_command_reply: tid=0 2024-02-02T15:38:04.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.348+0000 7f4f3a87bb80 1 client.28562 shutdown 2024-02-02T15:38:04.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.348+0000 7f4f3a87bb80 1 -- 172.21.15.1:0/2769353557 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5575291ac7e0 msgr2=0x5575291a9960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:04.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.348+0000 7f4f3a87bb80 1 --2- 172.21.15.1:0/2769353557 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5575291ac7e0 0x5575291a9960 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f4f180093c0 tx=0x7f4f18007840 comp rx=0 tx=0).stop 2024-02-02T15:38:04.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.348+0000 7f4f3a87bb80 1 -- 172.21.15.1:0/2769353557 shutdown_connections 2024-02-02T15:38:04.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.348+0000 7f4f3a87bb80 1 --2- 172.21.15.1:0/2769353557 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5575291ac7e0 0x5575291a9960 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.348+0000 7f4f3a87bb80 1 --2- 172.21.15.1:0/2769353557 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x5575291bbd30 0x5575291be150 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.348+0000 7f4f3a87bb80 1 --2- 172.21.15.1:0/2769353557 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5575291a1010 0x5575291ac190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.348+0000 7f4f3a87bb80 1 --2- 172.21.15.1:0/2769353557 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5575291a1930 0x5575291a93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.348+0000 7f4f0e000700 0 client.28562 ms_handle_reset on v2:172.21.15.71:6836/628016199 2024-02-02T15:38:04.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.348+0000 7f4f3a87bb80 1 -- 172.21.15.1:0/2769353557 >> 172.21.15.1:0/2769353557 conn(0x557529100db0 msgr2=0x55752919cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:04.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.348+0000 7f4f3a87bb80 1 -- 172.21.15.1:0/2769353557 shutdown_connections 2024-02-02T15:38:04.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.348+0000 7f4f3a87bb80 1 -- 172.21.15.1:0/2769353557 wait complete. 2024-02-02T15:38:04.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.348+0000 7f4f3a87bb80 20 client.28562 trim_cache size 0 max 16384 2024-02-02T15:38:04.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.363+0000 7f4f3a87bb80 1 Processor -- start 2024-02-02T15:38:04.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.363+0000 7f4f3a87bb80 1 -- start start 2024-02-02T15:38:04.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.363+0000 7f4f3a87bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5575291a1010 0x5575291b33b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:04.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.363+0000 7f4f3a87bb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5575291a1930 0x5575291a3840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:04.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.363+0000 7f4f3a87bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5575291ac7e0 0x5575291abcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:04.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.363+0000 7f4f3a87bb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x557528f766a0 con 0x5575291a1930 2024-02-02T15:38:04.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.363+0000 7f4f3a87bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55752919e910 con 0x5575291ac7e0 2024-02-02T15:38:04.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.363+0000 7f4f3a87bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575291be7b0 con 0x5575291a1010 2024-02-02T15:38:04.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.364+0000 7f4f24e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5575291ac7e0 0x5575291abcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:04.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.364+0000 7f4f24e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5575291ac7e0 0x5575291abcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:40924/0 (socket says 172.21.15.1:40924) 2024-02-02T15:38:04.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.364+0000 7f4f24e00700 1 -- 172.21.15.1:0/949467708 learned_addr learned my addr 172.21.15.1:0/949467708 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:38:04.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.364+0000 7f4f1f400700 1 --2- 172.21.15.1:0/949467708 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5575291a1930 0x5575291a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:04.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.364+0000 7f4f24e00700 1 -- 172.21.15.1:0/949467708 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5575291a1010 msgr2=0x5575291b33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:04.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.364+0000 7f4f1fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5575291a1010 0x5575291b33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:04.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.364+0000 7f4f24e00700 1 --2- 172.21.15.1:0/949467708 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5575291a1010 0x5575291b33b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.364+0000 7f4f24e00700 1 -- 172.21.15.1:0/949467708 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5575291a1930 msgr2=0x5575291a3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:04.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.364+0000 7f4f24e00700 1 --2- 172.21.15.1:0/949467708 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5575291a1930 0x5575291a3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.364+0000 7f4f24e00700 1 -- 172.21.15.1:0/949467708 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5575291990d0 con 0x5575291ac7e0 2024-02-02T15:38:04.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.364+0000 7f4f1f400700 1 --2- 172.21.15.1:0/949467708 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5575291a1930 0x5575291a3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:38:04.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.364+0000 7f4f1fe00700 1 --2- 172.21.15.1:0/949467708 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5575291a1010 0x5575291b33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:38:04.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.365+0000 7f4f24e00700 1 --2- 172.21.15.1:0/949467708 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5575291ac7e0 0x5575291abcc0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f4f1801b810 tx=0x7f4f180092e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8fdbfd852880907 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:04.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.365+0000 7f4f16000700 1 -- 172.21.15.1:0/949467708 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f1800e040 con 0x5575291ac7e0 2024-02-02T15:38:04.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.365+0000 7f4f16000700 1 -- 172.21.15.1:0/949467708 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f18007ad0 con 0x5575291ac7e0 2024-02-02T15:38:04.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.365+0000 7f4f16000700 1 -- 172.21.15.1:0/949467708 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f1801bcc0 con 0x5575291ac7e0 2024-02-02T15:38:04.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.366+0000 7f4f3a87bb80 1 -- 172.21.15.1:0/949467708 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5575291ac7e0 msgr2=0x5575291abcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:04.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.366+0000 7f4f3a87bb80 1 --2- 172.21.15.1:0/949467708 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5575291ac7e0 0x5575291abcc0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f4f1801b810 tx=0x7f4f180092e0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.366+0000 7f4f3a87bb80 1 -- 172.21.15.1:0/949467708 shutdown_connections 2024-02-02T15:38:04.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.366+0000 7f4f3a87bb80 1 --2- 172.21.15.1:0/949467708 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5575291ac7e0 0x5575291abcc0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.366+0000 7f4f3a87bb80 1 --2- 172.21.15.1:0/949467708 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5575291a1010 0x5575291b33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.366+0000 7f4f3a87bb80 1 --2- 172.21.15.1:0/949467708 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5575291a1930 0x5575291a3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.366+0000 7f4f3a87bb80 1 -- 172.21.15.1:0/949467708 >> 172.21.15.1:0/949467708 conn(0x557529100db0 msgr2=0x5575291b8530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:04.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.366+0000 7f4f3a87bb80 1 -- 172.21.15.1:0/949467708 shutdown_connections 2024-02-02T15:38:04.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.366+0000 7f4f3a87bb80 1 -- 172.21.15.1:0/949467708 wait complete. 2024-02-02T15:38:04.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.366+0000 7f4f3a87bb80 1 -- start start 2024-02-02T15:38:04.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.367+0000 7f4f3a87bb80 1 Processor -- start 2024-02-02T15:38:04.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.367+0000 7f4f3a87bb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5575291a1010 0x5575291ae320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:04.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.367+0000 7f4f3a87bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5575291a1930 0x5575291a5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:04.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.367+0000 7f4f3a87bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5575291ac7e0 0x5575291a5b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:04.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.367+0000 7f4f3a87bb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575291a6220 con 0x5575291a1010 2024-02-02T15:38:04.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.367+0000 7f4f3a87bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575291a6390 con 0x5575291ac7e0 2024-02-02T15:38:04.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.367+0000 7f4f3a87bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575291a6550 con 0x5575291a1930 2024-02-02T15:38:04.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.367+0000 7f4f3a87bb80 10 client.0 _ll_register_callbacks cb 0x557529090290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:38:04.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.367+0000 7f4f1f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5575291a1930 0x5575291a5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:04.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.367+0000 7f4f1fe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5575291a1010 0x5575291ae320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:04.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.367+0000 7f4f1fe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5575291a1010 0x5575291ae320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37400/0 (socket says 172.21.15.1:37400) 2024-02-02T15:38:04.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.367+0000 7f4f1fe00700 1 -- 172.21.15.1:0/3948460543 learned_addr learned my addr 172.21.15.1:0/3948460543 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:38:04.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.367+0000 7f4f24e00700 1 --2- 172.21.15.1:0/3948460543 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5575291ac7e0 0x5575291a5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:04.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.368+0000 7f4f1fe00700 1 -- 172.21.15.1:0/3948460543 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5575291a1930 msgr2=0x5575291a5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:04.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.368+0000 7f4f1fe00700 1 --2- 172.21.15.1:0/3948460543 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5575291a1930 0x5575291a5620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.368+0000 7f4f1fe00700 1 -- 172.21.15.1:0/3948460543 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5575291ac7e0 msgr2=0x5575291a5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:04.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.368+0000 7f4f1fe00700 1 --2- 172.21.15.1:0/3948460543 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5575291ac7e0 0x5575291a5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.368+0000 7f4f1fe00700 1 -- 172.21.15.1:0/3948460543 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557528f20b60 con 0x5575291a1010 2024-02-02T15:38:04.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.368+0000 7f4f24e00700 1 --2- 172.21.15.1:0/3948460543 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5575291ac7e0 0x5575291a5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:38:04.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.368+0000 7f4f1fe00700 1 --2- 172.21.15.1:0/3948460543 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5575291a1010 0x5575291ae320 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f4f100099a0 tx=0x7f4f1001fbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2027c7db26c57a96 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:04.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.368+0000 7f4f0e000700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:38:04.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.368+0000 7f4f0e000700 1 -- 172.21.15.1:0/3948460543 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f10016040 con 0x5575291a1010 2024-02-02T15:38:04.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.368+0000 7f4f0e000700 1 -- 172.21.15.1:0/3948460543 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f4f10004830 con 0x5575291a1010 2024-02-02T15:38:04.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.368+0000 7f4f3a87bb80 1 -- 172.21.15.1:0/3948460543 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4f1802e070 con 0x5575291a1010 2024-02-02T15:38:04.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.369+0000 7f4f0e000700 1 -- 172.21.15.1:0/3948460543 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4f10005d40 con 0x5575291a1010 2024-02-02T15:38:04.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.369+0000 7f4f3a87bb80 10 client.18477 fetch_fsmap learned FSMap version 50 2024-02-02T15:38:04.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.369+0000 7f4f3a87bb80 1 -- 172.21.15.1:0/3948460543 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x5575290b8ea0 con 0x5575291a1010 2024-02-02T15:38:04.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.369+0000 7f4f0e000700 1 -- 172.21.15.1:0/3948460543 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f1802e070 con 0x5575291a1010 2024-02-02T15:38:04.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.369+0000 7f4f0e000700 1 -- 172.21.15.1:0/3948460543 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 50) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f4f10010830 con 0x5575291a1010 2024-02-02T15:38:04.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.369+0000 7f4f3a87bb80 10 client.18477 fetch_fsmap finished waiting for FSMap version 50 2024-02-02T15:38:04.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.369+0000 7f4f3a87bb80 10 client.18477 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-02-02T15:38:04.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.369+0000 7f4f3a87bb80 20 client.18477 populate_metadata read hostname 'smithi001' 2024-02-02T15:38:04.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.369+0000 7f4f3a87bb80 1 --2- 172.21.15.1:0/3948460543 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55752920de20 0x557529210240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:04.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.369+0000 7f4f3a87bb80 4 client.18477 mds_command: new command op to 24518 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:38:04.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.369+0000 7f4f3a87bb80 1 -- 172.21.15.1:0/3948460543 --> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5575290b1f50 con 0x55752920de20 2024-02-02T15:38:04.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.370+0000 7f4f1f400700 1 --2- 172.21.15.1:0/3948460543 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55752920de20 0x557529210240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:04.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.370+0000 7f4f1f400700 1 --2- 172.21.15.1:0/3948460543 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55752920de20 0x557529210240 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=db11c30b69f4b506 server_cookie=f21b83ca106840de in_seq=0 out_seq=0 2024-02-02T15:38:04.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.370+0000 7f4f0e000700 10 client.18477 ms_handle_connect on v2:172.21.15.71:6836/628016199 2024-02-02T15:38:04.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.370+0000 7f4f0e000700 1 -- 172.21.15.1:0/3948460543 <== mds.2 v2:172.21.15.71:6836/628016199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5575290b1f50 con 0x55752920de20 2024-02-02T15:38:04.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.370+0000 7f4f0e000700 10 client.18477 handle_command_reply: tid=0 2024-02-02T15:38:04.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.370+0000 7f4f3a87bb80 1 client.18477 shutdown 2024-02-02T15:38:04.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.370+0000 7f4f3a87bb80 1 -- 172.21.15.1:0/3948460543 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5575291a1010 msgr2=0x5575291ae320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:04.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.370+0000 7f4f3a87bb80 1 --2- 172.21.15.1:0/3948460543 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5575291a1010 0x5575291ae320 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f4f100099a0 tx=0x7f4f1001fbc0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.370+0000 7f4f3a87bb80 1 -- 172.21.15.1:0/3948460543 shutdown_connections 2024-02-02T15:38:04.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.370+0000 7f4f3a87bb80 1 --2- 172.21.15.1:0/3948460543 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5575291ac7e0 0x5575291a5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.370+0000 7f4f3a87bb80 1 --2- 172.21.15.1:0/3948460543 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5575291a1010 0x5575291ae320 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.370+0000 7f4f3a87bb80 1 --2- 172.21.15.1:0/3948460543 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55752920de20 0x557529210240 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.370+0000 7f4f3a87bb80 1 --2- 172.21.15.1:0/3948460543 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5575291a1930 0x5575291a5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.370+0000 7f4f3a87bb80 1 -- 172.21.15.1:0/3948460543 >> 172.21.15.1:0/3948460543 conn(0x557529100db0 msgr2=0x5575291a46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:04.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.370+0000 7f4f0e000700 0 client.18477 ms_handle_reset on v2:172.21.15.71:6836/628016199 2024-02-02T15:38:04.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.371+0000 7f4f3a87bb80 1 -- 172.21.15.1:0/3948460543 shutdown_connections 2024-02-02T15:38:04.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.371+0000 7f4f3a87bb80 1 -- 172.21.15.1:0/3948460543 wait complete. 2024-02-02T15:38:04.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.371+0000 7f4f3a87bb80 20 client.18477 trim_cache size 0 max 16384 2024-02-02T15:38:04.375 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:38:04.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.371+0000 7f4f26e00700 1 -- 172.21.15.1:0/2028640366 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f4f000a0690 msgr2=0x7f4f000a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:04.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.371+0000 7f4f26e00700 1 --2- 172.21.15.1:0/2028640366 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f4f000a0690 0x7f4f000a2b50 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4f18027000 tx=0x7f4f1802a000 comp rx=0 tx=0).stop 2024-02-02T15:38:04.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.371+0000 7f4f26e00700 1 -- 172.21.15.1:0/2028640366 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f200689b0 msgr2=0x7f4f20199760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:04.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.371+0000 7f4f26e00700 1 --2- 172.21.15.1:0/2028640366 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f200689b0 0x7f4f20199760 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f4f0800e9c0 tx=0x7f4f0800ecd0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.371+0000 7f4f26e00700 1 -- 172.21.15.1:0/2028640366 shutdown_connections 2024-02-02T15:38:04.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.371+0000 7f4f26e00700 1 --2- 172.21.15.1:0/2028640366 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f4f000a0690 0x7f4f000a2b50 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.371+0000 7f4f26e00700 1 --2- 172.21.15.1:0/2028640366 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f20106ed0 0x7f4f2019dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.371+0000 7f4f26e00700 1 --2- 172.21.15.1:0/2028640366 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f20068010 0x7f4f20199220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.371+0000 7f4f26e00700 1 --2- 172.21.15.1:0/2028640366 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f200689b0 0x7f4f20199760 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.371+0000 7f4f26e00700 1 -- 172.21.15.1:0/2028640366 >> 172.21.15.1:0/2028640366 conn(0x7f4f200757d0 msgr2=0x7f4f2010bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:04.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.372+0000 7f4f26e00700 1 -- 172.21.15.1:0/2028640366 shutdown_connections 2024-02-02T15:38:04.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.372+0000 7f4f26e00700 1 -- 172.21.15.1:0/2028640366 wait complete. 2024-02-02T15:38:04.391 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-02T15:38:04.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.609+0000 7fe35da00700 1 -- 172.21.15.1:0/737817975 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe358101950 msgr2=0x7fe358105990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:04.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.609+0000 7fe35da00700 1 --2- 172.21.15.1:0/737817975 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe358101950 0x7fe358105990 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fe340009990 tx=0x7fe340009ca0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.609+0000 7fe35da00700 1 -- 172.21.15.1:0/737817975 shutdown_connections 2024-02-02T15:38:04.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.609+0000 7fe35da00700 1 --2- 172.21.15.1:0/737817975 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe3581060c0 0x7fe35810dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.609+0000 7fe35da00700 1 --2- 172.21.15.1:0/737817975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe358100fa0 0x7fe358101380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.609+0000 7fe35da00700 1 --2- 172.21.15.1:0/737817975 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe358101950 0x7fe358105990 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.609+0000 7fe35da00700 1 -- 172.21.15.1:0/737817975 >> 172.21.15.1:0/737817975 conn(0x7fe3580fc800 msgr2=0x7fe3580fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:04.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.609+0000 7fe35da00700 1 -- 172.21.15.1:0/737817975 shutdown_connections 2024-02-02T15:38:04.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.609+0000 7fe35da00700 1 -- 172.21.15.1:0/737817975 wait complete. 2024-02-02T15:38:04.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.611+0000 7fe35da00700 1 Processor -- start 2024-02-02T15:38:04.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.611+0000 7fe35da00700 1 -- start start 2024-02-02T15:38:04.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.611+0000 7fe35da00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe358100fa0 0x7fe358199070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:04.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.611+0000 7fe35da00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3581060c0 0x7fe3581995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:04.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.612+0000 7fe35da00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe35819da10 0x7fe35819de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:04.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.612+0000 7fe35da00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe35819e490 con 0x7fe35819da10 2024-02-02T15:38:04.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.612+0000 7fe35da00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe35819e600 con 0x7fe3581060c0 2024-02-02T15:38:04.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.612+0000 7fe35da00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe35819e7c0 con 0x7fe358100fa0 2024-02-02T15:38:04.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.612+0000 7fe356a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe358100fa0 0x7fe358199070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:04.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.612+0000 7fe356000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3581060c0 0x7fe3581995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:04.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.612+0000 7fe356000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3581060c0 0x7fe3581995b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:40960/0 (socket says 172.21.15.1:40960) 2024-02-02T15:38:04.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.612+0000 7fe356a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe358100fa0 0x7fe358199070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:46154/0 (socket says 172.21.15.1:46154) 2024-02-02T15:38:04.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.612+0000 7fe356000700 1 -- 172.21.15.1:0/4221000771 learned_addr learned my addr 172.21.15.1:0/4221000771 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:38:04.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.612+0000 7fe357400700 1 --2- 172.21.15.1:0/4221000771 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe35819da10 0x7fe35819de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:04.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.612+0000 7fe356000700 1 -- 172.21.15.1:0/4221000771 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe358100fa0 msgr2=0x7fe358199070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:04.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.612+0000 7fe356000700 1 --2- 172.21.15.1:0/4221000771 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe358100fa0 0x7fe358199070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.612+0000 7fe356000700 1 -- 172.21.15.1:0/4221000771 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe35819da10 msgr2=0x7fe35819de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:04.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.612+0000 7fe356000700 1 --2- 172.21.15.1:0/4221000771 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe35819da10 0x7fe35819de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.612+0000 7fe356000700 1 -- 172.21.15.1:0/4221000771 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3400095f0 con 0x7fe3581060c0 2024-02-02T15:38:04.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.612+0000 7fe356a00700 1 --2- 172.21.15.1:0/4221000771 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe358100fa0 0x7fe358199070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:38:04.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.613+0000 7fe356000700 1 --2- 172.21.15.1:0/4221000771 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3581060c0 0x7fe3581995b0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fe340004740 tx=0x7fe340009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:04.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.613+0000 7fe34f400700 1 -- 172.21.15.1:0/4221000771 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34001d070 con 0x7fe3581060c0 2024-02-02T15:38:04.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.613+0000 7fe34f400700 1 -- 172.21.15.1:0/4221000771 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe340004830 con 0x7fe3581060c0 2024-02-02T15:38:04.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.613+0000 7fe35da00700 1 -- 172.21.15.1:0/4221000771 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3581a5cc0 con 0x7fe3581060c0 2024-02-02T15:38:04.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.613+0000 7fe34f400700 1 -- 172.21.15.1:0/4221000771 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe340005280 con 0x7fe3581060c0 2024-02-02T15:38:04.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.613+0000 7fe35da00700 1 -- 172.21.15.1:0/4221000771 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3581a6140 con 0x7fe3581060c0 2024-02-02T15:38:04.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.616+0000 7fe34f400700 1 -- 172.21.15.1:0/4221000771 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7fe34000fb00 con 0x7fe3581060c0 2024-02-02T15:38:04.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.617+0000 7fe34f400700 1 --2- 172.21.15.1:0/4221000771 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fe3380a0710 0x7fe3380a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:04.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.617+0000 7fe37154fb80 1 Processor -- start 2024-02-02T15:38:04.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.617+0000 7fe34f400700 1 -- 172.21.15.1:0/4221000771 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fe3400cd0e0 con 0x7fe3581060c0 2024-02-02T15:38:04.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.617+0000 7fe37154fb80 1 -- start start 2024-02-02T15:38:04.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.617+0000 7fe356a00700 1 --2- 172.21.15.1:0/4221000771 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fe3380a0710 0x7fe3380a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:04.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.618+0000 7fe37154fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614a18d9010 0x5614a18d93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:04.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.618+0000 7fe37154fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614a18d9930 0x5614a18e42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:04.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.618+0000 7fe37154fb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614a18e47e0 0x5614a18e6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:04.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.618+0000 7fe37154fb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614a184d4c0 con 0x5614a18e47e0 2024-02-02T15:38:04.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.618+0000 7fe37154fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614a1831aa0 con 0x5614a18d9930 2024-02-02T15:38:04.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.618+0000 7fe37154fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614a182f460 con 0x5614a18d9010 2024-02-02T15:38:04.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.618+0000 7fe356a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614a18d9010 0x5614a18d93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:04.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.618+0000 7fe357400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614a18e47e0 0x5614a18e6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:04.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.618+0000 7fe356000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614a18d9930 0x5614a18e42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:04.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.618+0000 7fe356000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614a18d9930 0x5614a18e42a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:40962/0 (socket says 172.21.15.1:40962) 2024-02-02T15:38:04.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.618+0000 7fe357400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614a18e47e0 0x5614a18e6bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37424/0 (socket says 172.21.15.1:37424) 2024-02-02T15:38:04.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.618+0000 7fe356000700 1 -- 172.21.15.1:0/3748645814 learned_addr learned my addr 172.21.15.1:0/3748645814 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:38:04.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.618+0000 7fe356a00700 1 --2- 172.21.15.1:0/4221000771 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fe3380a0710 0x7fe3380a2bd0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe348011550 tx=0x7fe348009260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:04.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.619+0000 7fe356a00700 1 -- 172.21.15.1:0/3748645814 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614a18d9930 msgr2=0x5614a18e42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:04.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.619+0000 7fe356a00700 1 --2- 172.21.15.1:0/3748645814 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614a18d9930 0x5614a18e42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.619+0000 7fe356a00700 1 -- 172.21.15.1:0/3748645814 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614a18e47e0 msgr2=0x5614a18e6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:04.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.619+0000 7fe356a00700 1 --2- 172.21.15.1:0/3748645814 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614a18e47e0 0x5614a18e6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.619+0000 7fe356a00700 1 -- 172.21.15.1:0/3748645814 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5614a16ac3b0 con 0x5614a18d9010 2024-02-02T15:38:04.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.619+0000 7fe357400700 1 --2- 172.21.15.1:0/3748645814 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614a18e47e0 0x5614a18e6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:38:04.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.619+0000 7fe356a00700 1 --2- 172.21.15.1:0/3748645814 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614a18d9010 0x5614a18d93f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fe348009f10 tx=0x7fe348019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6384764e0cd5cadf server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:04.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.619+0000 7fe347e00700 1 -- 172.21.15.1:0/3748645814 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe348003b10 con 0x5614a18d9010 2024-02-02T15:38:04.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.619+0000 7fe347e00700 1 -- 172.21.15.1:0/3748645814 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe348003c70 con 0x5614a18d9010 2024-02-02T15:38:04.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.619+0000 7fe347e00700 1 -- 172.21.15.1:0/3748645814 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34801b6c0 con 0x5614a18d9010 2024-02-02T15:38:04.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.620+0000 7fe37154fb80 1 -- 172.21.15.1:0/3748645814 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614a18d9010 msgr2=0x5614a18d93f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:04.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.620+0000 7fe37154fb80 1 --2- 172.21.15.1:0/3748645814 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614a18d9010 0x5614a18d93f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fe348009f10 tx=0x7fe348019000 comp rx=0 tx=0).stop 2024-02-02T15:38:04.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.620+0000 7fe37154fb80 1 -- 172.21.15.1:0/3748645814 shutdown_connections 2024-02-02T15:38:04.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.620+0000 7fe37154fb80 1 --2- 172.21.15.1:0/3748645814 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614a18e47e0 0x5614a18e6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.620+0000 7fe37154fb80 1 --2- 172.21.15.1:0/3748645814 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614a18d9010 0x5614a18d93f0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.620+0000 7fe37154fb80 1 --2- 172.21.15.1:0/3748645814 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614a18d9930 0x5614a18e42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.620+0000 7fe37154fb80 1 -- 172.21.15.1:0/3748645814 >> 172.21.15.1:0/3748645814 conn(0x5614a1838db0 msgr2=0x5614a18d53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:04.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.620+0000 7fe37154fb80 1 -- 172.21.15.1:0/3748645814 shutdown_connections 2024-02-02T15:38:04.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.620+0000 7fe37154fb80 1 -- 172.21.15.1:0/3748645814 wait complete. 2024-02-02T15:38:04.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.621+0000 7fe37154fb80 1 -- start start 2024-02-02T15:38:04.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.621+0000 7fe37154fb80 1 Processor -- start 2024-02-02T15:38:04.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.622+0000 7fe37154fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614a18d9010 0x5614a18e4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:04.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.622+0000 7fe37154fb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614a18d9930 0x5614a18e1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:04.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.622+0000 7fe37154fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614a18e47e0 0x5614a18e1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:04.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.622+0000 7fe37154fb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614a182ad50 con 0x5614a18d9930 2024-02-02T15:38:04.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.622+0000 7fe37154fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614a17c6d80 con 0x5614a18e47e0 2024-02-02T15:38:04.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.622+0000 7fe37154fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614a17c6a20 con 0x5614a18d9010 2024-02-02T15:38:04.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.622+0000 7fe37154fb80 10 client.0 _ll_register_callbacks cb 0x5614a17c8290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:38:04.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.622+0000 7fe356a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614a18d9010 0x5614a18e4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:04.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.622+0000 7fe356000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614a18d9930 0x5614a18e1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:04.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.622+0000 7fe357400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614a18e47e0 0x5614a18e1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:04.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.622+0000 7fe356a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614a18d9010 0x5614a18e4180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:46158/0 (socket says 172.21.15.1:46158) 2024-02-02T15:38:04.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.622+0000 7fe356a00700 1 -- 172.21.15.1:0/2132123663 learned_addr learned my addr 172.21.15.1:0/2132123663 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:38:04.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.623+0000 7fe356a00700 1 -- 172.21.15.1:0/2132123663 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614a18e47e0 msgr2=0x5614a18e1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:04.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.623+0000 7fe356a00700 1 --2- 172.21.15.1:0/2132123663 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614a18e47e0 0x5614a18e1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.623+0000 7fe356a00700 1 -- 172.21.15.1:0/2132123663 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614a18d9930 msgr2=0x5614a18e1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:04.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.623+0000 7fe356a00700 1 --2- 172.21.15.1:0/2132123663 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614a18d9930 0x5614a18e1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.623+0000 7fe356a00700 1 -- 172.21.15.1:0/2132123663 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe350011230 con 0x5614a18d9010 2024-02-02T15:38:04.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.623+0000 7fe356a00700 1 --2- 172.21.15.1:0/2132123663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614a18d9010 0x5614a18e4180 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fe348003c20 tx=0x7fe34801ce60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca27ca97cc2f7624 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:04.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.623+0000 7fe344c00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:38:04.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.623+0000 7fe344c00700 1 -- 172.21.15.1:0/2132123663 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe348003e60 con 0x5614a18d9010 2024-02-02T15:38:04.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.623+0000 7fe344c00700 1 -- 172.21.15.1:0/2132123663 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fe34801c650 con 0x5614a18d9010 2024-02-02T15:38:04.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.623+0000 7fe37154fb80 1 -- 172.21.15.1:0/2132123663 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5614a18cd0f0 con 0x5614a18d9010 2024-02-02T15:38:04.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.623+0000 7fe344c00700 1 -- 172.21.15.1:0/2132123663 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe34801c810 con 0x5614a18d9010 2024-02-02T15:38:04.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.624+0000 7fe37154fb80 10 client.28585 fetch_fsmap learned FSMap version 50 2024-02-02T15:38:04.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.624+0000 7fe37154fb80 1 -- 172.21.15.1:0/2132123663 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x7fe348016030 con 0x5614a18d9010 2024-02-02T15:38:04.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.624+0000 7fe344c00700 1 -- 172.21.15.1:0/2132123663 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5614a18cd0f0 con 0x5614a18d9010 2024-02-02T15:38:04.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.624+0000 7fe344c00700 1 -- 172.21.15.1:0/2132123663 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 50) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fe34802d420 con 0x5614a18d9010 2024-02-02T15:38:04.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.624+0000 7fe37154fb80 10 client.28585 fetch_fsmap finished waiting for FSMap version 50 2024-02-02T15:38:04.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.624+0000 7fe37154fb80 10 client.28585 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-02T15:38:04.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.624+0000 7fe37154fb80 20 client.28585 populate_metadata read hostname 'smithi001' 2024-02-02T15:38:04.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.624+0000 7fe37154fb80 1 --2- 172.21.15.1:0/2132123663 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x5614a18f4160 0x5614a18f6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:04.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.624+0000 7fe37154fb80 4 client.28585 mds_command: new command op to 24529 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:38:04.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.624+0000 7fe37154fb80 1 -- 172.21.15.1:0/2132123663 --> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5614a1823dd0 con 0x5614a18f4160 2024-02-02T15:38:04.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.624+0000 7fe356000700 1 --2- 172.21.15.1:0/2132123663 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x5614a18f4160 0x5614a18f6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:04.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.625+0000 7fe344c00700 10 client.28585 ms_handle_connect on v2:172.21.15.142:6838/321675421 2024-02-02T15:38:04.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.625+0000 7fe356000700 1 --2- 172.21.15.1:0/2132123663 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x5614a18f4160 0x5614a18f6580 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fb39be942b3ae842 server_cookie=2e9ab6ef2455e130 in_seq=0 out_seq=0 2024-02-02T15:38:04.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.625+0000 7fe344c00700 1 -- 172.21.15.1:0/2132123663 <== mds.1 v2:172.21.15.142:6838/321675421 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5614a1823dd0 con 0x5614a18f4160 2024-02-02T15:38:04.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.625+0000 7fe344c00700 10 client.28585 handle_command_reply: tid=0 2024-02-02T15:38:04.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.626+0000 7fe37154fb80 1 client.28585 shutdown 2024-02-02T15:38:04.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.626+0000 7fe37154fb80 1 -- 172.21.15.1:0/2132123663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614a18d9010 msgr2=0x5614a18e4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:04.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.626+0000 7fe37154fb80 1 --2- 172.21.15.1:0/2132123663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614a18d9010 0x5614a18e4180 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fe348003c20 tx=0x7fe34801ce60 comp rx=0 tx=0).stop 2024-02-02T15:38:04.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.626+0000 7fe37154fb80 1 -- 172.21.15.1:0/2132123663 shutdown_connections 2024-02-02T15:38:04.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.626+0000 7fe37154fb80 1 --2- 172.21.15.1:0/2132123663 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x5614a18f4160 0x5614a18f6580 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.626+0000 7fe37154fb80 1 --2- 172.21.15.1:0/2132123663 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614a18e47e0 0x5614a18e1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.626+0000 7fe37154fb80 1 --2- 172.21.15.1:0/2132123663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614a18d9010 0x5614a18e4180 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.626+0000 7fe37154fb80 1 --2- 172.21.15.1:0/2132123663 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614a18d9930 0x5614a18e1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.626+0000 7fe344c00700 0 client.28585 ms_handle_reset on v2:172.21.15.142:6838/321675421 2024-02-02T15:38:04.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.626+0000 7fe37154fb80 1 -- 172.21.15.1:0/2132123663 >> 172.21.15.1:0/2132123663 conn(0x5614a1838db0 msgr2=0x5614a18d4fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:04.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.626+0000 7fe37154fb80 1 -- 172.21.15.1:0/2132123663 shutdown_connections 2024-02-02T15:38:04.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.626+0000 7fe37154fb80 1 -- 172.21.15.1:0/2132123663 wait complete. 2024-02-02T15:38:04.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.626+0000 7fe37154fb80 20 client.28585 trim_cache size 0 max 16384 2024-02-02T15:38:04.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.640+0000 7fe37154fb80 1 Processor -- start 2024-02-02T15:38:04.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.640+0000 7fe37154fb80 1 -- start start 2024-02-02T15:38:04.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.640+0000 7fe37154fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614a18d9010 0x5614a18f6550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:04.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.641+0000 7fe37154fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614a18d9930 0x5614a18eb330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:04.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.641+0000 7fe37154fb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614a18e47e0 0x5614a18dc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:04.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.641+0000 7fe37154fb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614a16ae6a0 con 0x5614a18e47e0 2024-02-02T15:38:04.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.641+0000 7fe37154fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614a18f3f40 con 0x5614a18d9930 2024-02-02T15:38:04.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.641+0000 7fe37154fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614a18df9d0 con 0x5614a18d9010 2024-02-02T15:38:04.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.641+0000 7fe356a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614a18d9010 0x5614a18f6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:04.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.641+0000 7fe356000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614a18d9930 0x5614a18eb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:04.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.641+0000 7fe356a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614a18d9010 0x5614a18f6550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:46170/0 (socket says 172.21.15.1:46170) 2024-02-02T15:38:04.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.641+0000 7fe356a00700 1 -- 172.21.15.1:0/1637015907 learned_addr learned my addr 172.21.15.1:0/1637015907 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:38:04.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.641+0000 7fe357400700 1 --2- 172.21.15.1:0/1637015907 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614a18e47e0 0x5614a18dc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:04.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.641+0000 7fe356000700 1 -- 172.21.15.1:0/1637015907 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614a18d9010 msgr2=0x5614a18f6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:04.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.641+0000 7fe356000700 1 --2- 172.21.15.1:0/1637015907 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614a18d9010 0x5614a18f6550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.641+0000 7fe356000700 1 -- 172.21.15.1:0/1637015907 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614a18e47e0 msgr2=0x5614a18dc7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:04.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.641+0000 7fe356000700 1 --2- 172.21.15.1:0/1637015907 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614a18e47e0 0x5614a18dc7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.641+0000 7fe356000700 1 -- 172.21.15.1:0/1637015907 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5614a1658b60 con 0x5614a18d9930 2024-02-02T15:38:04.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.641+0000 7fe356a00700 1 --2- 172.21.15.1:0/1637015907 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614a18d9010 0x5614a18f6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:38:04.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.641+0000 7fe356000700 1 --2- 172.21.15.1:0/1637015907 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614a18d9930 0x5614a18eb330 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fe34009a220 tx=0x7fe34002c040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=380a221c62632e85 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:04.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.642+0000 7fe34cc00700 1 -- 172.21.15.1:0/1637015907 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe340094ea0 con 0x5614a18d9930 2024-02-02T15:38:04.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.642+0000 7fe34cc00700 1 -- 172.21.15.1:0/1637015907 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe3400a2270 con 0x5614a18d9930 2024-02-02T15:38:04.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.642+0000 7fe34cc00700 1 -- 172.21.15.1:0/1637015907 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34009a4f0 con 0x5614a18d9930 2024-02-02T15:38:04.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.642+0000 7fe37154fb80 1 -- 172.21.15.1:0/1637015907 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614a18d9930 msgr2=0x5614a18eb330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:04.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.642+0000 7fe37154fb80 1 --2- 172.21.15.1:0/1637015907 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614a18d9930 0x5614a18eb330 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fe34009a220 tx=0x7fe34002c040 comp rx=0 tx=0).stop 2024-02-02T15:38:04.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.642+0000 7fe37154fb80 1 -- 172.21.15.1:0/1637015907 shutdown_connections 2024-02-02T15:38:04.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.642+0000 7fe37154fb80 1 --2- 172.21.15.1:0/1637015907 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614a18e47e0 0x5614a18dc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.642+0000 7fe37154fb80 1 --2- 172.21.15.1:0/1637015907 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614a18d9010 0x5614a18f6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.642+0000 7fe37154fb80 1 --2- 172.21.15.1:0/1637015907 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614a18d9930 0x5614a18eb330 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.642+0000 7fe37154fb80 1 -- 172.21.15.1:0/1637015907 >> 172.21.15.1:0/1637015907 conn(0x5614a1838db0 msgr2=0x5614a18f05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:04.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.642+0000 7fe37154fb80 1 -- 172.21.15.1:0/1637015907 shutdown_connections 2024-02-02T15:38:04.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.642+0000 7fe37154fb80 1 -- 172.21.15.1:0/1637015907 wait complete. 2024-02-02T15:38:04.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.643+0000 7fe37154fb80 1 -- start start 2024-02-02T15:38:04.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.643+0000 7fe37154fb80 1 Processor -- start 2024-02-02T15:38:04.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.643+0000 7fe37154fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614a18d9010 0x5614a18df1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:04.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.643+0000 7fe37154fb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614a18d9930 0x5614a18e9310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:04.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.643+0000 7fe37154fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614a18e47e0 0x5614a18e9a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:04.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.643+0000 7fe37154fb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614a18ea160 con 0x5614a18d9930 2024-02-02T15:38:04.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.643+0000 7fe37154fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614a1944c80 con 0x5614a18d9010 2024-02-02T15:38:04.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.643+0000 7fe37154fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614a1944dc0 con 0x5614a18e47e0 2024-02-02T15:38:04.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.643+0000 7fe37154fb80 10 client.0 _ll_register_callbacks cb 0x5614a17c8290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:38:04.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.643+0000 7fe356a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614a18d9010 0x5614a18df1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:04.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.644+0000 7fe356000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614a18d9930 0x5614a18e9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:04.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.644+0000 7fe356a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614a18d9010 0x5614a18df1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:40986/0 (socket says 172.21.15.1:40986) 2024-02-02T15:38:04.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.644+0000 7fe356a00700 1 -- 172.21.15.1:0/721380357 learned_addr learned my addr 172.21.15.1:0/721380357 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:38:04.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.644+0000 7fe357400700 1 --2- 172.21.15.1:0/721380357 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614a18e47e0 0x5614a18e9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:04.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.644+0000 7fe356a00700 1 -- 172.21.15.1:0/721380357 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614a18e47e0 msgr2=0x5614a18e9a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:04.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.644+0000 7fe356a00700 1 --2- 172.21.15.1:0/721380357 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614a18e47e0 0x5614a18e9a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.644+0000 7fe356a00700 1 -- 172.21.15.1:0/721380357 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614a18d9930 msgr2=0x5614a18e9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:04.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.644+0000 7fe356a00700 1 --2- 172.21.15.1:0/721380357 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614a18d9930 0x5614a18e9310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.644+0000 7fe356a00700 1 -- 172.21.15.1:0/721380357 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe348016030 con 0x5614a18d9010 2024-02-02T15:38:04.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.644+0000 7fe356a00700 1 --2- 172.21.15.1:0/721380357 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614a18d9010 0x5614a18df1b0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fe34802d880 tx=0x7fe348009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=657d10c3d5bc935b server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:04.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.644+0000 7fe344c00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:38:04.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.644+0000 7fe344c00700 1 -- 172.21.15.1:0/721380357 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34802d420 con 0x5614a18d9010 2024-02-02T15:38:04.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.644+0000 7fe37154fb80 1 -- 172.21.15.1:0/721380357 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe348003c70 con 0x5614a18d9010 2024-02-02T15:38:04.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.644+0000 7fe344c00700 1 -- 172.21.15.1:0/721380357 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fe34802d9a0 con 0x5614a18d9010 2024-02-02T15:38:04.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.645+0000 7fe344c00700 1 -- 172.21.15.1:0/721380357 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe34802d580 con 0x5614a18d9010 2024-02-02T15:38:04.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.645+0000 7fe37154fb80 10 client.28586 fetch_fsmap learned FSMap version 50 2024-02-02T15:38:04.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.645+0000 7fe37154fb80 1 -- 172.21.15.1:0/721380357 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x7fe3581a6140 con 0x5614a18d9010 2024-02-02T15:38:04.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.645+0000 7fe344c00700 1 -- 172.21.15.1:0/721380357 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe348003c70 con 0x5614a18d9010 2024-02-02T15:38:04.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.645+0000 7fe344c00700 1 -- 172.21.15.1:0/721380357 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 50) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fe348025880 con 0x5614a18d9010 2024-02-02T15:38:04.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.645+0000 7fe37154fb80 10 client.28586 fetch_fsmap finished waiting for FSMap version 50 2024-02-02T15:38:04.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.645+0000 7fe37154fb80 10 client.28586 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-02T15:38:04.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.645+0000 7fe37154fb80 20 client.28586 populate_metadata read hostname 'smithi001' 2024-02-02T15:38:04.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.645+0000 7fe37154fb80 1 --2- 172.21.15.1:0/721380357 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x5614a1946110 0x5614a1948530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:04.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.645+0000 7fe37154fb80 4 client.28586 mds_command: new command op to 24529 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:38:04.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.645+0000 7fe37154fb80 1 -- 172.21.15.1:0/721380357 --> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5614a17e9f50 con 0x5614a1946110 2024-02-02T15:38:04.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.645+0000 7fe356000700 1 --2- 172.21.15.1:0/721380357 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x5614a1946110 0x5614a1948530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:04.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.646+0000 7fe356000700 1 --2- 172.21.15.1:0/721380357 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x5614a1946110 0x5614a1948530 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fbedfdbd1ba725d5 server_cookie=9cd53abff2b5a7ea in_seq=0 out_seq=0 2024-02-02T15:38:04.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.646+0000 7fe344c00700 10 client.28586 ms_handle_connect on v2:172.21.15.142:6838/321675421 2024-02-02T15:38:04.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.646+0000 7fe344c00700 1 -- 172.21.15.1:0/721380357 <== mds.1 v2:172.21.15.142:6838/321675421 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5614a17e9f50 con 0x5614a1946110 2024-02-02T15:38:04.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.646+0000 7fe344c00700 10 client.28586 handle_command_reply: tid=0 2024-02-02T15:38:04.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.646+0000 7fe37154fb80 1 client.28586 shutdown 2024-02-02T15:38:04.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.646+0000 7fe37154fb80 1 -- 172.21.15.1:0/721380357 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614a18d9010 msgr2=0x5614a18df1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:04.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.646+0000 7fe37154fb80 1 --2- 172.21.15.1:0/721380357 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614a18d9010 0x5614a18df1b0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fe34802d880 tx=0x7fe348009f90 comp rx=0 tx=0).stop 2024-02-02T15:38:04.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.646+0000 7fe37154fb80 1 -- 172.21.15.1:0/721380357 shutdown_connections 2024-02-02T15:38:04.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.646+0000 7fe37154fb80 1 --2- 172.21.15.1:0/721380357 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x5614a1946110 0x5614a1948530 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.646+0000 7fe37154fb80 1 --2- 172.21.15.1:0/721380357 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614a18e47e0 0x5614a18e9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.646+0000 7fe37154fb80 1 --2- 172.21.15.1:0/721380357 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614a18d9010 0x5614a18df1b0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.646+0000 7fe37154fb80 1 --2- 172.21.15.1:0/721380357 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614a18d9930 0x5614a18e9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.646+0000 7fe37154fb80 1 -- 172.21.15.1:0/721380357 >> 172.21.15.1:0/721380357 conn(0x5614a1838db0 msgr2=0x5614a18dc7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:04.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.646+0000 7fe344c00700 0 client.28586 ms_handle_reset on v2:172.21.15.142:6838/321675421 2024-02-02T15:38:04.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.646+0000 7fe37154fb80 1 -- 172.21.15.1:0/721380357 shutdown_connections 2024-02-02T15:38:04.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.646+0000 7fe37154fb80 1 -- 172.21.15.1:0/721380357 wait complete. 2024-02-02T15:38:04.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.646+0000 7fe37154fb80 20 client.28586 trim_cache size 0 max 16384 2024-02-02T15:38:04.651 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:38:04.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.647+0000 7fe35da00700 1 -- 172.21.15.1:0/4221000771 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fe3380a0710 msgr2=0x7fe3380a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:04.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.647+0000 7fe35da00700 1 --2- 172.21.15.1:0/4221000771 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fe3380a0710 0x7fe3380a2bd0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe348011550 tx=0x7fe348009260 comp rx=0 tx=0).stop 2024-02-02T15:38:04.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.647+0000 7fe35da00700 1 -- 172.21.15.1:0/4221000771 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3581060c0 msgr2=0x7fe3581995b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:04.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.647+0000 7fe35da00700 1 --2- 172.21.15.1:0/4221000771 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3581060c0 0x7fe3581995b0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fe340004740 tx=0x7fe340009580 comp rx=0 tx=0).stop 2024-02-02T15:38:04.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.647+0000 7fe35da00700 1 -- 172.21.15.1:0/4221000771 shutdown_connections 2024-02-02T15:38:04.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.647+0000 7fe35da00700 1 --2- 172.21.15.1:0/4221000771 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fe3380a0710 0x7fe3380a2bd0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.647+0000 7fe35da00700 1 --2- 172.21.15.1:0/4221000771 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe35819da10 0x7fe35819de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.647+0000 7fe35da00700 1 --2- 172.21.15.1:0/4221000771 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe358100fa0 0x7fe358199070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.647+0000 7fe35da00700 1 --2- 172.21.15.1:0/4221000771 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3581060c0 0x7fe3581995b0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.647+0000 7fe35da00700 1 -- 172.21.15.1:0/4221000771 >> 172.21.15.1:0/4221000771 conn(0x7fe3580fc800 msgr2=0x7fe3580fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:04.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.647+0000 7fe35da00700 1 -- 172.21.15.1:0/4221000771 shutdown_connections 2024-02-02T15:38:04.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.647+0000 7fe35da00700 1 -- 172.21.15.1:0/4221000771 wait complete. 2024-02-02T15:38:04.663 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:04 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/1526324719' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:38:04.665 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-02T15:38:04.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:04 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/1526324719' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:38:04.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.881+0000 7f3243c00700 1 -- 172.21.15.1:0/2415310321 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f323c1019b0 msgr2=0x7f323c1059f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:04.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.881+0000 7f3243c00700 1 --2- 172.21.15.1:0/2415310321 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f323c1019b0 0x7f323c1059f0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f3224009990 tx=0x7f3224009ca0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.882+0000 7f3243c00700 1 -- 172.21.15.1:0/2415310321 shutdown_connections 2024-02-02T15:38:04.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.882+0000 7f3243c00700 1 --2- 172.21.15.1:0/2415310321 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f323c106120 0x7f323c10dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.882+0000 7f3243c00700 1 --2- 172.21.15.1:0/2415310321 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f323c101000 0x7f323c1013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.882+0000 7f3243c00700 1 --2- 172.21.15.1:0/2415310321 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f323c1019b0 0x7f323c1059f0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.882+0000 7f3243c00700 1 -- 172.21.15.1:0/2415310321 >> 172.21.15.1:0/2415310321 conn(0x7f323c0fc860 msgr2=0x7f323c0fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:04.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.882+0000 7f3243c00700 1 -- 172.21.15.1:0/2415310321 shutdown_connections 2024-02-02T15:38:04.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.882+0000 7f3243c00700 1 -- 172.21.15.1:0/2415310321 wait complete. 2024-02-02T15:38:04.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.883+0000 7f3243c00700 1 Processor -- start 2024-02-02T15:38:04.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.883+0000 7f3243c00700 1 -- start start 2024-02-02T15:38:04.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.883+0000 7f3243c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f323c101000 0x7f323c073040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:04.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.883+0000 7f3243c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f323c106120 0x7f323c073580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:04.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.883+0000 7f3243c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f323c073ac0 0x7f323c1a50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:04.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.883+0000 7f3243c00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f323c1a5740 con 0x7f323c106120 2024-02-02T15:38:04.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.883+0000 7f3243c00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f323c1a58b0 con 0x7f323c073ac0 2024-02-02T15:38:04.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.883+0000 7f3243c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f323c1a5a70 con 0x7f323c101000 2024-02-02T15:38:04.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.883+0000 7f323be00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f323c106120 0x7f323c073580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:04.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.883+0000 7f3241c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f323c073ac0 0x7f323c1a50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:04.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.883+0000 7f3241200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f323c101000 0x7f323c073040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:04.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.883+0000 7f3241c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f323c073ac0 0x7f323c1a50b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:41002/0 (socket says 172.21.15.1:41002) 2024-02-02T15:38:04.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.883+0000 7f3241c00700 1 -- 172.21.15.1:0/1549881079 learned_addr learned my addr 172.21.15.1:0/1549881079 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:38:04.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.884+0000 7f3241c00700 1 -- 172.21.15.1:0/1549881079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f323c101000 msgr2=0x7f323c073040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:04.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.884+0000 7f3241c00700 1 --2- 172.21.15.1:0/1549881079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f323c101000 0x7f323c073040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.884+0000 7f3241c00700 1 -- 172.21.15.1:0/1549881079 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f323c106120 msgr2=0x7f323c073580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:04.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.884+0000 7f3241c00700 1 --2- 172.21.15.1:0/1549881079 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f323c106120 0x7f323c073580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.884+0000 7f3241c00700 1 -- 172.21.15.1:0/1549881079 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32240095f0 con 0x7f323c073ac0 2024-02-02T15:38:04.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.884+0000 7f3241c00700 1 --2- 172.21.15.1:0/1549881079 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f323c073ac0 0x7f323c1a50b0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f323400ef30 tx=0x7f323400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:04.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.884+0000 7f3239600700 1 -- 172.21.15.1:0/1549881079 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3234018070 con 0x7f323c073ac0 2024-02-02T15:38:04.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.884+0000 7f3243c00700 1 -- 172.21.15.1:0/1549881079 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f323c1a5cf0 con 0x7f323c073ac0 2024-02-02T15:38:04.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.884+0000 7f3239600700 1 -- 172.21.15.1:0/1549881079 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f323400f040 con 0x7f323c073ac0 2024-02-02T15:38:04.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.884+0000 7f3239600700 1 -- 172.21.15.1:0/1549881079 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f323400cd00 con 0x7f323c073ac0 2024-02-02T15:38:04.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.884+0000 7f3243c00700 1 -- 172.21.15.1:0/1549881079 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f323c1a6200 con 0x7f323c073ac0 2024-02-02T15:38:04.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.887+0000 7f3239600700 1 -- 172.21.15.1:0/1549881079 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f3234003a00 con 0x7f323c073ac0 2024-02-02T15:38:04.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.889+0000 7f32577d9b80 1 Processor -- start 2024-02-02T15:38:04.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.889+0000 7f32577d9b80 1 -- start start 2024-02-02T15:38:04.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.889+0000 7f3239600700 1 --2- 172.21.15.1:0/1549881079 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f321c0a0690 0x7f321c0a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:04.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.889+0000 7f3239600700 1 -- 172.21.15.1:0/1549881079 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f32340cbbf0 con 0x7f323c073ac0 2024-02-02T15:38:04.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.889+0000 7f32577d9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563461eb7010 0x563461eb73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:04.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.889+0000 7f32577d9b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563461eb7930 0x563461ec22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:04.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.889+0000 7f32577d9b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563461ec27e0 0x563461ec4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:04.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.889+0000 7f323be00700 1 --2- 172.21.15.1:0/1549881079 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f321c0a0690 0x7f321c0a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:04.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.889+0000 7f32577d9b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x563461e2b4c0 con 0x563461ec27e0 2024-02-02T15:38:04.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.889+0000 7f32577d9b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x563461e0faa0 con 0x563461eb7930 2024-02-02T15:38:04.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.889+0000 7f32577d9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563461e0d460 con 0x563461eb7010 2024-02-02T15:38:04.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.889+0000 7f3241200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563461eb7010 0x563461eb73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:04.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.889+0000 7f3241200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563461eb7010 0x563461eb73f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:46196/0 (socket says 172.21.15.1:46196) 2024-02-02T15:38:04.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.889+0000 7f3241200700 1 -- 172.21.15.1:0/1135220548 learned_addr learned my addr 172.21.15.1:0/1135220548 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:38:04.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.889+0000 7f323be00700 1 --2- 172.21.15.1:0/1135220548 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563461eb7930 0x563461ec22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:04.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.889+0000 7f3241c00700 1 --2- 172.21.15.1:0/1135220548 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563461ec27e0 0x563461ec4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:04.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.890+0000 7f3241200700 1 -- 172.21.15.1:0/1135220548 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563461eb7930 msgr2=0x563461ec22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:04.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.890+0000 7f3241200700 1 --2- 172.21.15.1:0/1135220548 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563461eb7930 0x563461ec22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.890+0000 7f3241200700 1 -- 172.21.15.1:0/1135220548 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563461ec27e0 msgr2=0x563461ec4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:04.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.890+0000 7f3241200700 1 --2- 172.21.15.1:0/1135220548 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563461ec27e0 0x563461ec4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.890+0000 7f3241200700 1 -- 172.21.15.1:0/1135220548 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563461c8a3b0 con 0x563461eb7010 2024-02-02T15:38:04.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.890+0000 7f3241200700 1 --2- 172.21.15.1:0/1135220548 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563461eb7010 0x563461eb73f0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f322c009db0 tx=0x7f322c011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d3f9fdce333a2b8 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:04.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.890+0000 7f323be00700 1 --2- 172.21.15.1:0/1549881079 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f321c0a0690 0x7f321c0a2b50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f323c074830 tx=0x7f3224005c00 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:04.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.890+0000 7f3232000700 1 -- 172.21.15.1:0/1135220548 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f322c004230 con 0x563461eb7010 2024-02-02T15:38:04.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.890+0000 7f3232000700 1 -- 172.21.15.1:0/1135220548 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f322c004390 con 0x563461eb7010 2024-02-02T15:38:04.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.890+0000 7f3232000700 1 -- 172.21.15.1:0/1135220548 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f322c0197e0 con 0x563461eb7010 2024-02-02T15:38:04.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.891+0000 7f32577d9b80 1 -- 172.21.15.1:0/1135220548 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563461eb7010 msgr2=0x563461eb73f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:04.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.891+0000 7f32577d9b80 1 --2- 172.21.15.1:0/1135220548 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563461eb7010 0x563461eb73f0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f322c009db0 tx=0x7f322c011720 comp rx=0 tx=0).stop 2024-02-02T15:38:04.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.891+0000 7f32577d9b80 1 -- 172.21.15.1:0/1135220548 shutdown_connections 2024-02-02T15:38:04.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.891+0000 7f32577d9b80 1 --2- 172.21.15.1:0/1135220548 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563461ec27e0 0x563461ec4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.891+0000 7f32577d9b80 1 --2- 172.21.15.1:0/1135220548 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563461eb7010 0x563461eb73f0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.891+0000 7f32577d9b80 1 --2- 172.21.15.1:0/1135220548 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563461eb7930 0x563461ec22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.891+0000 7f32577d9b80 1 -- 172.21.15.1:0/1135220548 >> 172.21.15.1:0/1135220548 conn(0x563461e16db0 msgr2=0x563461eb33b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:04.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.891+0000 7f32577d9b80 1 -- 172.21.15.1:0/1135220548 shutdown_connections 2024-02-02T15:38:04.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.891+0000 7f32577d9b80 1 -- 172.21.15.1:0/1135220548 wait complete. 2024-02-02T15:38:04.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.892+0000 7f32577d9b80 1 -- start start 2024-02-02T15:38:04.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.892+0000 7f32577d9b80 1 Processor -- start 2024-02-02T15:38:04.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.892+0000 7f32577d9b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563461eb7010 0x563461ec2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:04.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.892+0000 7f32577d9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563461eb7930 0x563461ebf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:04.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.893+0000 7f32577d9b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563461ec27e0 0x563461ebfc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:04.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.893+0000 7f32577d9b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x563461e08d50 con 0x563461eb7010 2024-02-02T15:38:04.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.893+0000 7f32577d9b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x563461da4d80 con 0x563461ec27e0 2024-02-02T15:38:04.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.893+0000 7f32577d9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563461da4a20 con 0x563461eb7930 2024-02-02T15:38:04.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.893+0000 7f32577d9b80 10 client.0 _ll_register_callbacks cb 0x563461da6290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:38:04.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.893+0000 7f323be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563461eb7930 0x563461ebf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:04.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.893+0000 7f3241c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563461ec27e0 0x563461ebfc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:04.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.893+0000 7f323be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563461eb7930 0x563461ebf510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:46202/0 (socket says 172.21.15.1:46202) 2024-02-02T15:38:04.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.893+0000 7f3241200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563461eb7010 0x563461ec2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:04.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.893+0000 7f323be00700 1 -- 172.21.15.1:0/1136045624 learned_addr learned my addr 172.21.15.1:0/1136045624 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:38:04.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.893+0000 7f3241c00700 1 -- 172.21.15.1:0/1136045624 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563461eb7930 msgr2=0x563461ebf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:04.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.893+0000 7f3241c00700 1 --2- 172.21.15.1:0/1136045624 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563461eb7930 0x563461ebf510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.893+0000 7f3241c00700 1 -- 172.21.15.1:0/1136045624 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563461eb7010 msgr2=0x563461ec2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:04.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.893+0000 7f3241c00700 1 --2- 172.21.15.1:0/1136045624 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563461eb7010 0x563461ec2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.893+0000 7f3241c00700 1 -- 172.21.15.1:0/1136045624 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f322c0099e0 con 0x563461ec27e0 2024-02-02T15:38:04.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.893+0000 7f3241c00700 1 --2- 172.21.15.1:0/1136045624 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563461ec27e0 0x563461ebfc50 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f32340c8f40 tx=0x7f3234098700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e870a68055c7618f server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:04.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.893+0000 7f322aa00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:38:04.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.894+0000 7f322aa00700 1 -- 172.21.15.1:0/1136045624 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32340994a0 con 0x563461ec27e0 2024-02-02T15:38:04.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.894+0000 7f322aa00700 1 -- 172.21.15.1:0/1136045624 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f323400ce60 con 0x563461ec27e0 2024-02-02T15:38:04.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.894+0000 7f32577d9b80 1 -- 172.21.15.1:0/1136045624 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563461eab0f0 con 0x563461ec27e0 2024-02-02T15:38:04.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.894+0000 7f322aa00700 1 -- 172.21.15.1:0/1136045624 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3234098da0 con 0x563461ec27e0 2024-02-02T15:38:04.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.894+0000 7f32577d9b80 10 client.28598 fetch_fsmap learned FSMap version 50 2024-02-02T15:38:04.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.894+0000 7f32577d9b80 1 -- 172.21.15.1:0/1136045624 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x563461c36b60 con 0x563461ec27e0 2024-02-02T15:38:04.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.894+0000 7f322aa00700 1 -- 172.21.15.1:0/1136045624 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563461eab0f0 con 0x563461ec27e0 2024-02-02T15:38:04.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.894+0000 7f322aa00700 1 -- 172.21.15.1:0/1136045624 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 50) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f3234098290 con 0x563461ec27e0 2024-02-02T15:38:04.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.894+0000 7f32577d9b80 10 client.28598 fetch_fsmap finished waiting for FSMap version 50 2024-02-02T15:38:04.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.894+0000 7f32577d9b80 10 client.28598 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:38:04.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.894+0000 7f32577d9b80 20 client.28598 populate_metadata read hostname 'smithi001' 2024-02-02T15:38:04.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.894+0000 7f32577d9b80 1 --2- 172.21.15.1:0/1136045624 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x563461ed20c0 0x563461ed44e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:04.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.894+0000 7f32577d9b80 4 client.28598 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:38:04.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.894+0000 7f32577d9b80 1 -- 172.21.15.1:0/1136045624 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563461e01dd0 con 0x563461ed20c0 2024-02-02T15:38:04.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.895+0000 7f3241200700 1 --2- 172.21.15.1:0/1136045624 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x563461ed20c0 0x563461ed44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:04.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.895+0000 7f3241200700 1 --2- 172.21.15.1:0/1136045624 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x563461ed20c0 0x563461ed44e0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42ab6f857fa0d813 server_cookie=1f7c00db0e48f3e6 in_seq=0 out_seq=0 2024-02-02T15:38:04.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.895+0000 7f322aa00700 10 client.28598 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:38:04.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.896+0000 7f322aa00700 1 -- 172.21.15.1:0/1136045624 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x563461e01dd0 con 0x563461ed20c0 2024-02-02T15:38:04.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.896+0000 7f322aa00700 10 client.28598 handle_command_reply: tid=0 2024-02-02T15:38:04.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.896+0000 7f32577d9b80 1 client.28598 shutdown 2024-02-02T15:38:04.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.896+0000 7f32577d9b80 1 -- 172.21.15.1:0/1136045624 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563461ec27e0 msgr2=0x563461ebfc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:04.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.896+0000 7f32577d9b80 1 --2- 172.21.15.1:0/1136045624 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563461ec27e0 0x563461ebfc50 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f32340c8f40 tx=0x7f3234098700 comp rx=0 tx=0).stop 2024-02-02T15:38:04.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.896+0000 7f32577d9b80 1 -- 172.21.15.1:0/1136045624 shutdown_connections 2024-02-02T15:38:04.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.896+0000 7f32577d9b80 1 --2- 172.21.15.1:0/1136045624 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x563461ed20c0 0x563461ed44e0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.896+0000 7f32577d9b80 1 --2- 172.21.15.1:0/1136045624 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563461ec27e0 0x563461ebfc50 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.896+0000 7f32577d9b80 1 --2- 172.21.15.1:0/1136045624 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563461eb7010 0x563461ec2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.896+0000 7f322aa00700 0 client.28598 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:38:04.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.896+0000 7f32577d9b80 1 --2- 172.21.15.1:0/1136045624 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563461eb7930 0x563461ebf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.896+0000 7f32577d9b80 1 -- 172.21.15.1:0/1136045624 >> 172.21.15.1:0/1136045624 conn(0x563461e16db0 msgr2=0x563461eb2fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:04.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.896+0000 7f32577d9b80 1 -- 172.21.15.1:0/1136045624 shutdown_connections 2024-02-02T15:38:04.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.896+0000 7f32577d9b80 1 -- 172.21.15.1:0/1136045624 wait complete. 2024-02-02T15:38:04.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.897+0000 7f32577d9b80 20 client.28598 trim_cache size 0 max 16384 2024-02-02T15:38:04.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.910+0000 7f32577d9b80 1 Processor -- start 2024-02-02T15:38:04.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.910+0000 7f32577d9b80 1 -- start start 2024-02-02T15:38:04.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.910+0000 7f32577d9b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563461eb7010 0x563461ebf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:04.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.911+0000 7f32577d9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563461eb7930 0x563461eb1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:04.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.911+0000 7f32577d9b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563461ec27e0 0x563461ec1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:04.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.911+0000 7f32577d9b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x563461c8c6a0 con 0x563461ec27e0 2024-02-02T15:38:04.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.911+0000 7f32577d9b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x563461ebb380 con 0x563461eb7010 2024-02-02T15:38:04.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.911+0000 7f32577d9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563461ebea80 con 0x563461eb7930 2024-02-02T15:38:04.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.911+0000 7f323be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563461eb7930 0x563461eb1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:04.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.911+0000 7f323be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563461eb7930 0x563461eb1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:46216/0 (socket says 172.21.15.1:46216) 2024-02-02T15:38:04.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.911+0000 7f323be00700 1 -- 172.21.15.1:0/418167359 learned_addr learned my addr 172.21.15.1:0/418167359 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:38:04.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.911+0000 7f3241c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563461ec27e0 0x563461ec1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:04.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.911+0000 7f3241200700 1 --2- 172.21.15.1:0/418167359 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563461eb7010 0x563461ebf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:04.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.911+0000 7f323be00700 1 -- 172.21.15.1:0/418167359 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563461eb7010 msgr2=0x563461ebf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:04.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.911+0000 7f323be00700 1 --2- 172.21.15.1:0/418167359 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563461eb7010 0x563461ebf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.911+0000 7f323be00700 1 -- 172.21.15.1:0/418167359 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563461ec27e0 msgr2=0x563461ec1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:04.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.911+0000 7f323be00700 1 --2- 172.21.15.1:0/418167359 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563461ec27e0 0x563461ec1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.911+0000 7f323be00700 1 -- 172.21.15.1:0/418167359 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563461dceea0 con 0x563461eb7930 2024-02-02T15:38:04.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.912+0000 7f323be00700 1 --2- 172.21.15.1:0/418167359 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563461eb7930 0x563461eb1b20 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f32240093f0 tx=0x7f322402c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99dd46cd35340946 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:04.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.912+0000 7f3232a00700 1 -- 172.21.15.1:0/418167359 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f322401d070 con 0x563461eb7930 2024-02-02T15:38:04.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.912+0000 7f3232a00700 1 -- 172.21.15.1:0/418167359 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f322401e040 con 0x563461eb7930 2024-02-02T15:38:04.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.912+0000 7f3232a00700 1 -- 172.21.15.1:0/418167359 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3224029820 con 0x563461eb7930 2024-02-02T15:38:04.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.912+0000 7f32577d9b80 1 -- 172.21.15.1:0/418167359 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563461eb7930 msgr2=0x563461eb1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:04.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.912+0000 7f32577d9b80 1 --2- 172.21.15.1:0/418167359 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563461eb7930 0x563461eb1b20 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f32240093f0 tx=0x7f322402c000 comp rx=0 tx=0).stop 2024-02-02T15:38:04.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.912+0000 7f32577d9b80 1 -- 172.21.15.1:0/418167359 shutdown_connections 2024-02-02T15:38:04.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.912+0000 7f32577d9b80 1 --2- 172.21.15.1:0/418167359 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563461ec27e0 0x563461ec1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.912+0000 7f32577d9b80 1 --2- 172.21.15.1:0/418167359 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563461eb7010 0x563461ebf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.912+0000 7f32577d9b80 1 --2- 172.21.15.1:0/418167359 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563461eb7930 0x563461eb1b20 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.912+0000 7f32577d9b80 1 -- 172.21.15.1:0/418167359 >> 172.21.15.1:0/418167359 conn(0x563461e16db0 msgr2=0x563461eaee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:04.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.913+0000 7f32577d9b80 1 -- 172.21.15.1:0/418167359 shutdown_connections 2024-02-02T15:38:04.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.913+0000 7f32577d9b80 1 -- 172.21.15.1:0/418167359 wait complete. 2024-02-02T15:38:04.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.913+0000 7f32577d9b80 1 -- start start 2024-02-02T15:38:04.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.913+0000 7f32577d9b80 1 Processor -- start 2024-02-02T15:38:04.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.913+0000 7f32577d9b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563461eb7010 0x563461ec7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:04.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.913+0000 7f32577d9b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563461eb7930 0x563461ec7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:04.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.914+0000 7f32577d9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563461ec27e0 0x563461ec7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:04.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.914+0000 7f32577d9b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x563461f22c70 con 0x563461eb7930 2024-02-02T15:38:04.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.914+0000 7f32577d9b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x563461f22de0 con 0x563461eb7010 2024-02-02T15:38:04.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.914+0000 7f32577d9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563461f22fa0 con 0x563461ec27e0 2024-02-02T15:38:04.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.914+0000 7f32577d9b80 10 client.0 _ll_register_callbacks cb 0x563461da6290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:38:04.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.914+0000 7f323be00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563461eb7930 0x563461ec7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:04.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.914+0000 7f3241c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563461ec27e0 0x563461ec7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:04.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.914+0000 7f3241200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563461eb7010 0x563461ec7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:04.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.914+0000 7f3241c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563461ec27e0 0x563461ec7d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:46218/0 (socket says 172.21.15.1:46218) 2024-02-02T15:38:04.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.914+0000 7f3241c00700 1 -- 172.21.15.1:0/2733823580 learned_addr learned my addr 172.21.15.1:0/2733823580 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:38:04.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.914+0000 7f3241200700 1 -- 172.21.15.1:0/2733823580 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563461ec27e0 msgr2=0x563461ec7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:04.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.914+0000 7f3241200700 1 --2- 172.21.15.1:0/2733823580 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563461ec27e0 0x563461ec7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.914+0000 7f3241200700 1 -- 172.21.15.1:0/2733823580 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563461eb7930 msgr2=0x563461ec7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:04.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.914+0000 7f3241200700 1 --2- 172.21.15.1:0/2733823580 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563461eb7930 0x563461ec7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.914+0000 7f3241200700 1 -- 172.21.15.1:0/2733823580 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3224020040 con 0x563461eb7010 2024-02-02T15:38:04.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.914+0000 7f3241200700 1 --2- 172.21.15.1:0/2733823580 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563461eb7010 0x563461ec7310 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f322c011a70 tx=0x7f322c0092d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc4069f3e0d94ad2 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:04.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.914+0000 7f322aa00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:38:04.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.915+0000 7f322aa00700 1 -- 172.21.15.1:0/2733823580 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f322c02a330 con 0x563461eb7010 2024-02-02T15:38:04.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.915+0000 7f32577d9b80 1 -- 172.21.15.1:0/2733823580 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f322c004390 con 0x563461eb7010 2024-02-02T15:38:04.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.915+0000 7f322aa00700 1 -- 172.21.15.1:0/2733823580 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f322c003680 con 0x563461eb7010 2024-02-02T15:38:04.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.915+0000 7f322aa00700 1 -- 172.21.15.1:0/2733823580 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f322c003840 con 0x563461eb7010 2024-02-02T15:38:04.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.915+0000 7f32577d9b80 10 client.28604 fetch_fsmap learned FSMap version 50 2024-02-02T15:38:04.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.915+0000 7f32577d9b80 1 -- 172.21.15.1:0/2733823580 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x563461c8ecd0 con 0x563461eb7010 2024-02-02T15:38:04.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.915+0000 7f322aa00700 1 -- 172.21.15.1:0/2733823580 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f322c004390 con 0x563461eb7010 2024-02-02T15:38:04.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.915+0000 7f322aa00700 1 -- 172.21.15.1:0/2733823580 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 50) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f322c011440 con 0x563461eb7010 2024-02-02T15:38:04.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.915+0000 7f32577d9b80 10 client.28604 fetch_fsmap finished waiting for FSMap version 50 2024-02-02T15:38:04.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.915+0000 7f32577d9b80 10 client.28604 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:38:04.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.915+0000 7f32577d9b80 20 client.28604 populate_metadata read hostname 'smithi001' 2024-02-02T15:38:04.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.915+0000 7f32577d9b80 1 --2- 172.21.15.1:0/2733823580 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x563461f24580 0x563461f269a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:04.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.915+0000 7f32577d9b80 4 client.28604 mds_command: new command op to 24557 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:38:04.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.915+0000 7f32577d9b80 1 -- 172.21.15.1:0/2733823580 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563461dc7f50 con 0x563461f24580 2024-02-02T15:38:04.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.916+0000 7f323be00700 1 --2- 172.21.15.1:0/2733823580 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x563461f24580 0x563461f269a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:04.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.916+0000 7f323be00700 1 --2- 172.21.15.1:0/2733823580 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x563461f24580 0x563461f269a0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a87299d1d5d96508 server_cookie=46e1efa29af0bbc7 in_seq=0 out_seq=0 2024-02-02T15:38:04.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.916+0000 7f322aa00700 10 client.28604 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:38:04.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.916+0000 7f322aa00700 1 -- 172.21.15.1:0/2733823580 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563461dc7f50 con 0x563461f24580 2024-02-02T15:38:04.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.916+0000 7f322aa00700 10 client.28604 handle_command_reply: tid=0 2024-02-02T15:38:04.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.916+0000 7f32577d9b80 1 client.28604 shutdown 2024-02-02T15:38:04.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.917+0000 7f32577d9b80 1 -- 172.21.15.1:0/2733823580 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563461eb7010 msgr2=0x563461ec7310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:04.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.917+0000 7f32577d9b80 1 --2- 172.21.15.1:0/2733823580 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563461eb7010 0x563461ec7310 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f322c011a70 tx=0x7f322c0092d0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.917+0000 7f32577d9b80 1 -- 172.21.15.1:0/2733823580 shutdown_connections 2024-02-02T15:38:04.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.917+0000 7f32577d9b80 1 --2- 172.21.15.1:0/2733823580 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x563461f24580 0x563461f269a0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.917+0000 7f32577d9b80 1 --2- 172.21.15.1:0/2733823580 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563461ec27e0 0x563461ec7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.917+0000 7f32577d9b80 1 --2- 172.21.15.1:0/2733823580 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563461eb7010 0x563461ec7310 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.917+0000 7f32577d9b80 1 --2- 172.21.15.1:0/2733823580 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563461eb7930 0x563461ec7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.917+0000 7f32577d9b80 1 -- 172.21.15.1:0/2733823580 >> 172.21.15.1:0/2733823580 conn(0x563461e16db0 msgr2=0x563461eba840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:04.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.917+0000 7f322aa00700 0 client.28604 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:38:04.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.917+0000 7f32577d9b80 1 -- 172.21.15.1:0/2733823580 shutdown_connections 2024-02-02T15:38:04.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.917+0000 7f32577d9b80 1 -- 172.21.15.1:0/2733823580 wait complete. 2024-02-02T15:38:04.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.917+0000 7f32577d9b80 20 client.28604 trim_cache size 0 max 16384 2024-02-02T15:38:04.921 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:38:04.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:04 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/1526324719' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:38:04.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.917+0000 7f3243c00700 1 -- 172.21.15.1:0/1549881079 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f321c0a0690 msgr2=0x7f321c0a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:04.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.917+0000 7f3243c00700 1 --2- 172.21.15.1:0/1549881079 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f321c0a0690 0x7f321c0a2b50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f323c074830 tx=0x7f3224005c00 comp rx=0 tx=0).stop 2024-02-02T15:38:04.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.917+0000 7f3243c00700 1 -- 172.21.15.1:0/1549881079 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f323c073ac0 msgr2=0x7f323c1a50b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:04.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.917+0000 7f3243c00700 1 --2- 172.21.15.1:0/1549881079 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f323c073ac0 0x7f323c1a50b0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f323400ef30 tx=0x7f323400c3a0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.918+0000 7f3243c00700 1 -- 172.21.15.1:0/1549881079 shutdown_connections 2024-02-02T15:38:04.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.918+0000 7f3243c00700 1 --2- 172.21.15.1:0/1549881079 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f321c0a0690 0x7f321c0a2b50 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.918+0000 7f3243c00700 1 --2- 172.21.15.1:0/1549881079 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f323c073ac0 0x7f323c1a50b0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.918+0000 7f3243c00700 1 --2- 172.21.15.1:0/1549881079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f323c101000 0x7f323c073040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.918+0000 7f3243c00700 1 --2- 172.21.15.1:0/1549881079 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f323c106120 0x7f323c073580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:04.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.918+0000 7f3243c00700 1 -- 172.21.15.1:0/1549881079 >> 172.21.15.1:0/1549881079 conn(0x7f323c0fc860 msgr2=0x7f323c0fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:04.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.918+0000 7f3243c00700 1 -- 172.21.15.1:0/1549881079 shutdown_connections 2024-02-02T15:38:04.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:04.918+0000 7f3243c00700 1 -- 172.21.15.1:0/1549881079 wait complete. 2024-02-02T15:38:05.264 INFO:tasks.workunit.client.0.smithi001.stdout: 1 618706 5.66 MB/sec execute 459 sec latency 14.505 ms 2024-02-02T15:38:05.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:05 smithi001 ceph-mon[35470]: pgmap v343: 129 pgs: 129 active+clean; 496 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 119 op/s 2024-02-02T15:38:05.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:05 smithi071 ceph-mon[44707]: pgmap v343: 129 pgs: 129 active+clean; 496 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 119 op/s 2024-02-02T15:38:05.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:05 smithi142 ceph-mon[45728]: pgmap v343: 129 pgs: 129 active+clean; 496 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 119 op/s 2024-02-02T15:38:05.937 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-02T15:38:05.937 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-02T15:38:06.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.143+0000 7f4efb800700 1 -- 172.21.15.1:0/3508925881 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ef4100ff0 msgr2=0x7f4ef41013d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:06.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.143+0000 7f4efb800700 1 --2- 172.21.15.1:0/3508925881 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ef4100ff0 0x7f4ef41013d0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f4ee40098f0 tx=0x7f4ee4009c00 comp rx=0 tx=0).stop 2024-02-02T15:38:06.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.143+0000 7f4efb800700 1 -- 172.21.15.1:0/3508925881 shutdown_connections 2024-02-02T15:38:06.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.143+0000 7f4efb800700 1 --2- 172.21.15.1:0/3508925881 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ef4106110 0x7f4ef410dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:06.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.143+0000 7f4efb800700 1 --2- 172.21.15.1:0/3508925881 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ef4100ff0 0x7f4ef41013d0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:06.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.143+0000 7f4efb800700 1 --2- 172.21.15.1:0/3508925881 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ef41019a0 0x7f4ef41059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:06.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.143+0000 7f4efb800700 1 -- 172.21.15.1:0/3508925881 >> 172.21.15.1:0/3508925881 conn(0x7f4ef40fc810 msgr2=0x7f4ef40fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:06.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.144+0000 7f4efb800700 1 -- 172.21.15.1:0/3508925881 shutdown_connections 2024-02-02T15:38:06.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.144+0000 7f4efb800700 1 -- 172.21.15.1:0/3508925881 wait complete. 2024-02-02T15:38:06.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.144+0000 7f4efb800700 1 Processor -- start 2024-02-02T15:38:06.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.145+0000 7f4efb800700 1 -- start start 2024-02-02T15:38:06.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.145+0000 7f4efb800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ef4100ff0 0x7f4ef41991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:06.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.145+0000 7f4efb800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ef4106110 0x7f4ef41996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:06.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.145+0000 7f4efb800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ef419db50 0x7f4ef419dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:06.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.145+0000 7f4efb800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef419e5d0 con 0x7f4ef4106110 2024-02-02T15:38:06.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.145+0000 7f4efb800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef419e740 con 0x7f4ef419db50 2024-02-02T15:38:06.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.145+0000 7f4efb800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef419e900 con 0x7f4ef4100ff0 2024-02-02T15:38:06.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.145+0000 7f4ef3e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ef4106110 0x7f4ef41996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:06.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.145+0000 7f4ef3e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ef4106110 0x7f4ef41996f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37518/0 (socket says 172.21.15.1:37518) 2024-02-02T15:38:06.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.145+0000 7f4ef3e00700 1 -- 172.21.15.1:0/1202241409 learned_addr learned my addr 172.21.15.1:0/1202241409 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:38:06.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.145+0000 7f4ef9800700 1 --2- 172.21.15.1:0/1202241409 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ef419db50 0x7f4ef419dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:06.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.145+0000 7f4ef8e00700 1 --2- 172.21.15.1:0/1202241409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ef4100ff0 0x7f4ef41991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:06.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.146+0000 7f4ef3e00700 1 -- 172.21.15.1:0/1202241409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ef4100ff0 msgr2=0x7f4ef41991b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:06.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.146+0000 7f4ef3e00700 1 --2- 172.21.15.1:0/1202241409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ef4100ff0 0x7f4ef41991b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:06.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.146+0000 7f4ef3e00700 1 -- 172.21.15.1:0/1202241409 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ef419db50 msgr2=0x7f4ef419dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:06.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.146+0000 7f4ef3e00700 1 --2- 172.21.15.1:0/1202241409 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ef419db50 0x7f4ef419dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:06.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.146+0000 7f4ef3e00700 1 -- 172.21.15.1:0/1202241409 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4edc009520 con 0x7f4ef4106110 2024-02-02T15:38:06.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.146+0000 7f4ef9800700 1 --2- 172.21.15.1:0/1202241409 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ef419db50 0x7f4ef419dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:38:06.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.146+0000 7f4ef3e00700 1 --2- 172.21.15.1:0/1202241409 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ef4106110 0x7f4ef41996f0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f4edc00ea30 tx=0x7f4edc00ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:06.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.146+0000 7f4ef1600700 1 -- 172.21.15.1:0/1202241409 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4edc00cbd0 con 0x7f4ef4106110 2024-02-02T15:38:06.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.146+0000 7f4ef1600700 1 -- 172.21.15.1:0/1202241409 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4edc00eed0 con 0x7f4ef4106110 2024-02-02T15:38:06.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.146+0000 7f4efb800700 1 -- 172.21.15.1:0/1202241409 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4eec009d00 con 0x7f4ef4106110 2024-02-02T15:38:06.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.146+0000 7f4ef1600700 1 -- 172.21.15.1:0/1202241409 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4edc003d20 con 0x7f4ef4106110 2024-02-02T15:38:06.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.147+0000 7f4efb800700 1 -- 172.21.15.1:0/1202241409 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ee4009520 con 0x7f4ef4106110 2024-02-02T15:38:06.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.150+0000 7f4ef1600700 1 -- 172.21.15.1:0/1202241409 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f4edc003ee0 con 0x7f4ef4106110 2024-02-02T15:38:06.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.151+0000 7f4f0f3f7b80 1 Processor -- start 2024-02-02T15:38:06.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.151+0000 7f4f0f3f7b80 1 -- start start 2024-02-02T15:38:06.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.151+0000 7f4f0f3f7b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5612abce9090 0x5612abce9470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:06.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.151+0000 7f4f0f3f7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5612abce99b0 0x5612abcf4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:06.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.151+0000 7f4f0f3f7b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5612abcf4860 0x5612abcf6c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:06.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.151+0000 7f4f0f3f7b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612abc5d540 con 0x5612abce9090 2024-02-02T15:38:06.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.151+0000 7f4f0f3f7b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612abc41b20 con 0x5612abcf4860 2024-02-02T15:38:06.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.151+0000 7f4f0f3f7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612abc3f4e0 con 0x5612abce99b0 2024-02-02T15:38:06.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.152+0000 7f4ef9800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5612abce9090 0x5612abce9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:06.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.152+0000 7f4ef1600700 1 --2- 172.21.15.1:0/1202241409 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f4ed40a0690 0x7f4ed40a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:06.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.152+0000 7f4ef9800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5612abce9090 0x5612abce9470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37526/0 (socket says 172.21.15.1:37526) 2024-02-02T15:38:06.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.152+0000 7f4ef9800700 1 -- 172.21.15.1:0/2559525617 learned_addr learned my addr 172.21.15.1:0/2559525617 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:38:06.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.152+0000 7f4ef8e00700 1 --2- 172.21.15.1:0/2559525617 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5612abce99b0 0x5612abcf4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:06.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.152+0000 7f4ef3e00700 1 --2- 172.21.15.1:0/2559525617 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5612abcf4860 0x5612abcf6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:06.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.152+0000 7f4ef1600700 1 -- 172.21.15.1:0/1202241409 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f4edc014070 con 0x7f4ef4106110 2024-02-02T15:38:06.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.152+0000 7f4ef9800700 1 --2- 172.21.15.1:0/1202241409 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f4ed40a0690 0x7f4ed40a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:06.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.152+0000 7f4ef8e00700 1 -- 172.21.15.1:0/2559525617 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5612abcf4860 msgr2=0x5612abcf6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:06.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.152+0000 7f4ef8e00700 1 --2- 172.21.15.1:0/2559525617 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5612abcf4860 0x5612abcf6c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:06.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.152+0000 7f4ef8e00700 1 -- 172.21.15.1:0/2559525617 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5612abce9090 msgr2=0x5612abce9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:06.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.152+0000 7f4ef8e00700 1 --2- 172.21.15.1:0/2559525617 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5612abce9090 0x5612abce9470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:06.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.152+0000 7f4ef8e00700 1 -- 172.21.15.1:0/2559525617 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5612ababc430 con 0x5612abce99b0 2024-02-02T15:38:06.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.153+0000 7f4ef8e00700 1 --2- 172.21.15.1:0/2559525617 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5612abce99b0 0x5612abcf4320 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4009230 tx=0x7f4ee4005dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80060c739a2102c0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:06.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.153+0000 7f4ef9800700 1 --2- 172.21.15.1:0/1202241409 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f4ed40a0690 0x7f4ed40a2b50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4eec018440 tx=0x7f4eec004b40 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:06.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.153+0000 7f4eea000700 1 -- 172.21.15.1:0/2559525617 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee400fe70 con 0x5612abce99b0 2024-02-02T15:38:06.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.153+0000 7f4eea000700 1 -- 172.21.15.1:0/2559525617 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ee401d040 con 0x5612abce99b0 2024-02-02T15:38:06.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.153+0000 7f4eea000700 1 -- 172.21.15.1:0/2559525617 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee4005040 con 0x5612abce99b0 2024-02-02T15:38:06.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.153+0000 7f4f0f3f7b80 1 -- 172.21.15.1:0/2559525617 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5612abce99b0 msgr2=0x5612abcf4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:06.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.153+0000 7f4f0f3f7b80 1 --2- 172.21.15.1:0/2559525617 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5612abce99b0 0x5612abcf4320 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4009230 tx=0x7f4ee4005dc0 comp rx=0 tx=0).stop 2024-02-02T15:38:06.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.154+0000 7f4f0f3f7b80 1 -- 172.21.15.1:0/2559525617 shutdown_connections 2024-02-02T15:38:06.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.154+0000 7f4f0f3f7b80 1 --2- 172.21.15.1:0/2559525617 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5612abcf4860 0x5612abcf6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:06.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.154+0000 7f4f0f3f7b80 1 --2- 172.21.15.1:0/2559525617 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5612abce9090 0x5612abce9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:06.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.154+0000 7f4f0f3f7b80 1 --2- 172.21.15.1:0/2559525617 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5612abce99b0 0x5612abcf4320 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:06.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.154+0000 7f4f0f3f7b80 1 -- 172.21.15.1:0/2559525617 >> 172.21.15.1:0/2559525617 conn(0x5612abc48e30 msgr2=0x5612abce5430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:06.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.154+0000 7f4f0f3f7b80 1 -- 172.21.15.1:0/2559525617 shutdown_connections 2024-02-02T15:38:06.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.154+0000 7f4f0f3f7b80 1 -- 172.21.15.1:0/2559525617 wait complete. 2024-02-02T15:38:06.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.154+0000 7f4f0f3f7b80 1 -- start start 2024-02-02T15:38:06.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.154+0000 7f4f0f3f7b80 1 Processor -- start 2024-02-02T15:38:06.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.155+0000 7f4f0f3f7b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5612abce9090 0x5612abcf4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:06.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.155+0000 7f4f0f3f7b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5612abce99b0 0x5612abcf1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:06.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.155+0000 7f4f0f3f7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5612abcf4860 0x5612abcf19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:06.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.155+0000 7f4f0f3f7b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612abc3add0 con 0x5612abce9090 2024-02-02T15:38:06.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.155+0000 7f4f0f3f7b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612abbd6e00 con 0x5612abce99b0 2024-02-02T15:38:06.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.155+0000 7f4f0f3f7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612abbd6aa0 con 0x5612abcf4860 2024-02-02T15:38:06.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.155+0000 7f4f0f3f7b80 10 client.0 _ll_register_callbacks cb 0x5612abbd8310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:38:06.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.155+0000 7f4ef8e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5612abce9090 0x5612abcf4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:06.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.155+0000 7f4ef9800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5612abcf4860 0x5612abcf19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:06.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.155+0000 7f4ef9800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5612abcf4860 0x5612abcf19d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:46246/0 (socket says 172.21.15.1:46246) 2024-02-02T15:38:06.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.155+0000 7f4ef9800700 1 -- 172.21.15.1:0/4131112069 learned_addr learned my addr 172.21.15.1:0/4131112069 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:38:06.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.155+0000 7f4ef3e00700 1 --2- 172.21.15.1:0/4131112069 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5612abce99b0 0x5612abcf1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:06.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.156+0000 7f4ef9800700 1 -- 172.21.15.1:0/4131112069 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5612abce99b0 msgr2=0x5612abcf1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:06.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.156+0000 7f4ef9800700 1 --2- 172.21.15.1:0/4131112069 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5612abce99b0 0x5612abcf1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:06.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.156+0000 7f4ef9800700 1 -- 172.21.15.1:0/4131112069 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5612abce9090 msgr2=0x5612abcf4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:06.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.156+0000 7f4ef9800700 1 --2- 172.21.15.1:0/4131112069 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5612abce9090 0x5612abcf4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:06.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.156+0000 7f4ef9800700 1 -- 172.21.15.1:0/4131112069 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ef410e500 con 0x5612abcf4860 2024-02-02T15:38:06.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.156+0000 7f4ef9800700 1 --2- 172.21.15.1:0/4131112069 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5612abcf4860 0x5612abcf19d0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f4eec009610 tx=0x7f4eec004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=428e2d24fe344690 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:06.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.156+0000 7f4ee2a00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:38:06.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.156+0000 7f4ee2a00700 1 -- 172.21.15.1:0/4131112069 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eec021070 con 0x5612abcf4860 2024-02-02T15:38:06.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.156+0000 7f4ee2a00700 1 -- 172.21.15.1:0/4131112069 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f4eec01a470 con 0x5612abcf4860 2024-02-02T15:38:06.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.156+0000 7f4f0f3f7b80 1 -- 172.21.15.1:0/4131112069 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5612abcdd170 con 0x5612abcf4860 2024-02-02T15:38:06.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.156+0000 7f4ee2a00700 1 -- 172.21.15.1:0/4131112069 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4eec00e040 con 0x5612abcf4860 2024-02-02T15:38:06.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.156+0000 7f4f0f3f7b80 10 client.28621 fetch_fsmap learned FSMap version 51 2024-02-02T15:38:06.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.156+0000 7f4f0f3f7b80 1 -- 172.21.15.1:0/4131112069 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x5612aba68be0 con 0x5612abcf4860 2024-02-02T15:38:06.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.157+0000 7f4ee2a00700 1 -- 172.21.15.1:0/4131112069 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5612abcdd170 con 0x5612abcf4860 2024-02-02T15:38:06.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.157+0000 7f4ee2a00700 1 -- 172.21.15.1:0/4131112069 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 51) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f4eec02d070 con 0x5612abcf4860 2024-02-02T15:38:06.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.157+0000 7f4f0f3f7b80 10 client.28621 fetch_fsmap finished waiting for FSMap version 51 2024-02-02T15:38:06.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.157+0000 7f4f0f3f7b80 10 client.28621 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:38:06.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.157+0000 7f4f0f3f7b80 20 client.28621 populate_metadata read hostname 'smithi001' 2024-02-02T15:38:06.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.157+0000 7f4f0f3f7b80 1 --2- 172.21.15.1:0/4131112069 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5612abd041f0 0x5612abd06610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:06.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.157+0000 7f4f0f3f7b80 4 client.28621 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:38:06.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.157+0000 7f4f0f3f7b80 1 -- 172.21.15.1:0/4131112069 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5612abc33e50 con 0x5612abd041f0 2024-02-02T15:38:06.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.157+0000 7f4ef8e00700 1 --2- 172.21.15.1:0/4131112069 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5612abd041f0 0x5612abd06610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:06.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.158+0000 7f4ef8e00700 1 --2- 172.21.15.1:0/4131112069 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5612abd041f0 0x5612abd06610 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=562940ca8415b903 server_cookie=56ad52e5bf8707fe in_seq=0 out_seq=0 2024-02-02T15:38:06.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.158+0000 7f4ee2a00700 10 client.28621 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:38:06.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.158+0000 7f4ee2a00700 1 -- 172.21.15.1:0/4131112069 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5612abc33e50 con 0x5612abd041f0 2024-02-02T15:38:06.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.158+0000 7f4ee2a00700 10 client.28621 handle_command_reply: tid=0 2024-02-02T15:38:06.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.158+0000 7f4f0f3f7b80 1 client.28621 shutdown 2024-02-02T15:38:06.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.158+0000 7f4f0f3f7b80 1 -- 172.21.15.1:0/4131112069 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5612abcf4860 msgr2=0x5612abcf19d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:06.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.158+0000 7f4f0f3f7b80 1 --2- 172.21.15.1:0/4131112069 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5612abcf4860 0x5612abcf19d0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f4eec009610 tx=0x7f4eec004ad0 comp rx=0 tx=0).stop 2024-02-02T15:38:06.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.159+0000 7f4f0f3f7b80 1 -- 172.21.15.1:0/4131112069 shutdown_connections 2024-02-02T15:38:06.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.159+0000 7f4f0f3f7b80 1 --2- 172.21.15.1:0/4131112069 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5612abd041f0 0x5612abd06610 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:06.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.159+0000 7f4f0f3f7b80 1 --2- 172.21.15.1:0/4131112069 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5612abcf4860 0x5612abcf19d0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:06.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.159+0000 7f4f0f3f7b80 1 --2- 172.21.15.1:0/4131112069 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5612abce9090 0x5612abcf4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:06.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.159+0000 7f4ee2a00700 0 client.28621 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:38:06.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.159+0000 7f4f0f3f7b80 1 --2- 172.21.15.1:0/4131112069 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5612abce99b0 0x5612abcf1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:06.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.159+0000 7f4f0f3f7b80 1 -- 172.21.15.1:0/4131112069 >> 172.21.15.1:0/4131112069 conn(0x5612abc48e30 msgr2=0x5612abce5020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:06.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.159+0000 7f4f0f3f7b80 1 -- 172.21.15.1:0/4131112069 shutdown_connections 2024-02-02T15:38:06.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.159+0000 7f4f0f3f7b80 1 -- 172.21.15.1:0/4131112069 wait complete. 2024-02-02T15:38:06.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.159+0000 7f4f0f3f7b80 20 client.28621 trim_cache size 0 max 16384 2024-02-02T15:38:06.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.176+0000 7f4f0f3f7b80 1 Processor -- start 2024-02-02T15:38:06.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.176+0000 7f4f0f3f7b80 1 -- start start 2024-02-02T15:38:06.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.176+0000 7f4f0f3f7b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5612abce9090 0x5612abcf02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:06.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.176+0000 7f4f0f3f7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5612abce99b0 0x5612abcf3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:06.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.176+0000 7f4f0f3f7b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5612abcf4860 0x5612abcee680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:06.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.177+0000 7f4f0f3f7b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612ababe720 con 0x5612abce9090 2024-02-02T15:38:06.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.177+0000 7f4f0f3f7b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612abce9d90 con 0x5612abcf4860 2024-02-02T15:38:06.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.177+0000 7f4f0f3f7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612abcdd040 con 0x5612abce99b0 2024-02-02T15:38:06.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.177+0000 7f4ef8e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5612abce9090 0x5612abcf02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:06.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.177+0000 7f4ef8e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5612abce9090 0x5612abcf02d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37546/0 (socket says 172.21.15.1:37546) 2024-02-02T15:38:06.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.177+0000 7f4ef8e00700 1 -- 172.21.15.1:0/3113189943 learned_addr learned my addr 172.21.15.1:0/3113189943 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:38:06.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.177+0000 7f4ef3e00700 1 --2- 172.21.15.1:0/3113189943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5612abce99b0 0x5612abcf3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:06.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.177+0000 7f4ef9800700 1 --2- 172.21.15.1:0/3113189943 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5612abcf4860 0x5612abcee680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:06.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.177+0000 7f4ef8e00700 1 -- 172.21.15.1:0/3113189943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5612abce99b0 msgr2=0x5612abcf3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:06.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.177+0000 7f4ef8e00700 1 --2- 172.21.15.1:0/3113189943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5612abce99b0 0x5612abcf3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:06.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.177+0000 7f4ef8e00700 1 -- 172.21.15.1:0/3113189943 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5612abcf4860 msgr2=0x5612abcee680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:06.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.177+0000 7f4ef8e00700 1 --2- 172.21.15.1:0/3113189943 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5612abcf4860 0x5612abcee680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:06.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.177+0000 7f4ef8e00700 1 -- 172.21.15.1:0/3113189943 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5612abce3e70 con 0x5612abce9090 2024-02-02T15:38:06.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.177+0000 7f4ef9800700 1 --2- 172.21.15.1:0/3113189943 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5612abcf4860 0x5612abcee680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:38:06.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.177+0000 7f4ef8e00700 1 --2- 172.21.15.1:0/3113189943 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5612abce9090 0x5612abcf02d0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4000c00 tx=0x7f4ee4009400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4a1df71cc25578d server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:06.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.178+0000 7f4eeaa00700 1 -- 172.21.15.1:0/3113189943 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee40303d0 con 0x5612abce9090 2024-02-02T15:38:06.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.178+0000 7f4eeaa00700 1 -- 172.21.15.1:0/3113189943 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ee4005920 con 0x5612abce9090 2024-02-02T15:38:06.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.178+0000 7f4eeaa00700 1 -- 172.21.15.1:0/3113189943 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee4005310 con 0x5612abce9090 2024-02-02T15:38:06.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.178+0000 7f4f0f3f7b80 1 -- 172.21.15.1:0/3113189943 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5612abce9090 msgr2=0x5612abcf02d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:06.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.178+0000 7f4f0f3f7b80 1 --2- 172.21.15.1:0/3113189943 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5612abce9090 0x5612abcf02d0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4000c00 tx=0x7f4ee4009400 comp rx=0 tx=0).stop 2024-02-02T15:38:06.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.178+0000 7f4f0f3f7b80 1 -- 172.21.15.1:0/3113189943 shutdown_connections 2024-02-02T15:38:06.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.178+0000 7f4f0f3f7b80 1 --2- 172.21.15.1:0/3113189943 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5612abcf4860 0x5612abcee680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:06.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.178+0000 7f4f0f3f7b80 1 --2- 172.21.15.1:0/3113189943 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5612abce9090 0x5612abcf02d0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:06.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.178+0000 7f4f0f3f7b80 1 --2- 172.21.15.1:0/3113189943 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5612abce99b0 0x5612abcf3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:06.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.178+0000 7f4f0f3f7b80 1 -- 172.21.15.1:0/3113189943 >> 172.21.15.1:0/3113189943 conn(0x5612abc48e30 msgr2=0x5612abcefbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:06.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.178+0000 7f4f0f3f7b80 1 -- 172.21.15.1:0/3113189943 shutdown_connections 2024-02-02T15:38:06.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.178+0000 7f4f0f3f7b80 1 -- 172.21.15.1:0/3113189943 wait complete. 2024-02-02T15:38:06.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.179+0000 7f4f0f3f7b80 1 -- start start 2024-02-02T15:38:06.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.179+0000 7f4f0f3f7b80 1 Processor -- start 2024-02-02T15:38:06.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.179+0000 7f4f0f3f7b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5612abce9090 0x5612abcfafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:06.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.179+0000 7f4f0f3f7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5612abce99b0 0x5612abcf9310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:06.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.180+0000 7f4f0f3f7b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5612abcf4860 0x5612abcf9850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:06.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.180+0000 7f4f0f3f7b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612abcf9d90 con 0x5612abcf4860 2024-02-02T15:38:06.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.180+0000 7f4f0f3f7b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612abcf9f00 con 0x5612abce9090 2024-02-02T15:38:06.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.180+0000 7f4f0f3f7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5612abcfa070 con 0x5612abce99b0 2024-02-02T15:38:06.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.180+0000 7f4f0f3f7b80 10 client.0 _ll_register_callbacks cb 0x5612abbd8310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:38:06.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.180+0000 7f4ef3e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5612abce99b0 0x5612abcf9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:06.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.180+0000 7f4ef3e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5612abce99b0 0x5612abcf9310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:46270/0 (socket says 172.21.15.1:46270) 2024-02-02T15:38:06.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.180+0000 7f4ef3e00700 1 -- 172.21.15.1:0/449512712 learned_addr learned my addr 172.21.15.1:0/449512712 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:38:06.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.180+0000 7f4ef9800700 1 --2- 172.21.15.1:0/449512712 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5612abcf4860 0x5612abcf9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:06.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.180+0000 7f4ef8e00700 1 --2- 172.21.15.1:0/449512712 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5612abce9090 0x5612abcfafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:06.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.180+0000 7f4ef3e00700 1 -- 172.21.15.1:0/449512712 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5612abce9090 msgr2=0x5612abcfafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:06.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.180+0000 7f4ef3e00700 1 --2- 172.21.15.1:0/449512712 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5612abce9090 0x5612abcfafe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:06.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.180+0000 7f4ef3e00700 1 -- 172.21.15.1:0/449512712 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5612abcf4860 msgr2=0x5612abcf9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:06.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.180+0000 7f4ef3e00700 1 --2- 172.21.15.1:0/449512712 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5612abcf4860 0x5612abcf9850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:06.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.180+0000 7f4ef3e00700 1 -- 172.21.15.1:0/449512712 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5612ababc430 con 0x5612abce99b0 2024-02-02T15:38:06.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.180+0000 7f4ef3e00700 1 --2- 172.21.15.1:0/449512712 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5612abce99b0 0x5612abcf9310 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f4edc0cc550 tx=0x7f4edc069780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef28a796ef8849b server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:06.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.180+0000 7f4ee2a00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:38:06.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.181+0000 7f4ee2a00700 1 -- 172.21.15.1:0/449512712 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4edc099cd0 con 0x5612abce99b0 2024-02-02T15:38:06.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.181+0000 7f4ee2a00700 1 -- 172.21.15.1:0/449512712 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f4edc0995d0 con 0x5612abce99b0 2024-02-02T15:38:06.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.181+0000 7f4f0f3f7b80 1 -- 172.21.15.1:0/449512712 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4ee401d040 con 0x5612abce99b0 2024-02-02T15:38:06.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.181+0000 7f4ee2a00700 1 -- 172.21.15.1:0/449512712 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4edc09d400 con 0x5612abce99b0 2024-02-02T15:38:06.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.181+0000 7f4f0f3f7b80 10 client.28627 fetch_fsmap learned FSMap version 51 2024-02-02T15:38:06.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.181+0000 7f4f0f3f7b80 1 -- 172.21.15.1:0/449512712 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x5612abce1150 con 0x5612abce99b0 2024-02-02T15:38:06.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.181+0000 7f4ee2a00700 1 -- 172.21.15.1:0/449512712 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ee401d040 con 0x5612abce99b0 2024-02-02T15:38:06.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.181+0000 7f4ee2a00700 1 -- 172.21.15.1:0/449512712 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 51) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f4edc0c9a80 con 0x5612abce99b0 2024-02-02T15:38:06.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.181+0000 7f4f0f3f7b80 10 client.28627 fetch_fsmap finished waiting for FSMap version 51 2024-02-02T15:38:06.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.181+0000 7f4f0f3f7b80 10 client.28627 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:38:06.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.181+0000 7f4f0f3f7b80 20 client.28627 populate_metadata read hostname 'smithi001' 2024-02-02T15:38:06.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.181+0000 7f4f0f3f7b80 1 --2- 172.21.15.1:0/449512712 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5612abd571e0 0x5612abd59600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:06.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.181+0000 7f4f0f3f7b80 4 client.28627 mds_command: new command op to 24557 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-02T15:38:06.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.181+0000 7f4f0f3f7b80 1 -- 172.21.15.1:0/449512712 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5612abbf9fd0 con 0x5612abd571e0 2024-02-02T15:38:06.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.182+0000 7f4ef8e00700 1 --2- 172.21.15.1:0/449512712 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5612abd571e0 0x5612abd59600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:06.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.182+0000 7f4ef8e00700 1 --2- 172.21.15.1:0/449512712 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5612abd571e0 0x5612abd59600 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=261ab6dfaf4de57d server_cookie=d6ddb0728d63057c in_seq=0 out_seq=0 2024-02-02T15:38:06.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.182+0000 7f4ee2a00700 10 client.28627 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:38:06.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.183+0000 7f4ee2a00700 1 -- 172.21.15.1:0/449512712 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5612abbf9fd0 con 0x5612abd571e0 2024-02-02T15:38:06.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.183+0000 7f4ee2a00700 10 client.28627 handle_command_reply: tid=0 2024-02-02T15:38:06.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.183+0000 7f4f0f3f7b80 1 client.28627 shutdown 2024-02-02T15:38:06.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.184+0000 7f4f0f3f7b80 1 -- 172.21.15.1:0/449512712 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5612abce99b0 msgr2=0x5612abcf9310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:06.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.184+0000 7f4f0f3f7b80 1 --2- 172.21.15.1:0/449512712 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5612abce99b0 0x5612abcf9310 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f4edc0cc550 tx=0x7f4edc069780 comp rx=0 tx=0).stop 2024-02-02T15:38:06.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.184+0000 7f4f0f3f7b80 1 -- 172.21.15.1:0/449512712 shutdown_connections 2024-02-02T15:38:06.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.184+0000 7f4f0f3f7b80 1 --2- 172.21.15.1:0/449512712 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5612abd571e0 0x5612abd59600 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:06.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.184+0000 7f4f0f3f7b80 1 --2- 172.21.15.1:0/449512712 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5612abcf4860 0x5612abcf9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:06.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.184+0000 7f4f0f3f7b80 1 --2- 172.21.15.1:0/449512712 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5612abce9090 0x5612abcfafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:06.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.184+0000 7f4f0f3f7b80 1 --2- 172.21.15.1:0/449512712 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5612abce99b0 0x5612abcf9310 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:06.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.184+0000 7f4ee2a00700 0 client.28627 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:38:06.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.184+0000 7f4f0f3f7b80 1 -- 172.21.15.1:0/449512712 >> 172.21.15.1:0/449512712 conn(0x5612abc48e30 msgr2=0x5612abcf0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:06.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.184+0000 7f4f0f3f7b80 1 -- 172.21.15.1:0/449512712 shutdown_connections 2024-02-02T15:38:06.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.184+0000 7f4f0f3f7b80 1 -- 172.21.15.1:0/449512712 wait complete. 2024-02-02T15:38:06.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.184+0000 7f4f0f3f7b80 20 client.28627 trim_cache size 0 max 16384 2024-02-02T15:38:06.188 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:38:06.188 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2024-02-02T15:38:06.188 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "9283363d-61f2-4651-ad5f-58c10d663219", 2024-02-02T15:38:06.188 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2024-02-02T15:38:06.188 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:38:06.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.185+0000 7f4efb800700 1 -- 172.21.15.1:0/1202241409 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f4ed40a0690 msgr2=0x7f4ed40a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:06.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.185+0000 7f4efb800700 1 --2- 172.21.15.1:0/1202241409 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f4ed40a0690 0x7f4ed40a2b50 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4eec018440 tx=0x7f4eec004b40 comp rx=0 tx=0).stop 2024-02-02T15:38:06.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.185+0000 7f4efb800700 1 -- 172.21.15.1:0/1202241409 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ef4106110 msgr2=0x7f4ef41996f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:06.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.185+0000 7f4efb800700 1 --2- 172.21.15.1:0/1202241409 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ef4106110 0x7f4ef41996f0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f4edc00ea30 tx=0x7f4edc00ed40 comp rx=0 tx=0).stop 2024-02-02T15:38:06.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.185+0000 7f4efb800700 1 -- 172.21.15.1:0/1202241409 shutdown_connections 2024-02-02T15:38:06.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.185+0000 7f4efb800700 1 --2- 172.21.15.1:0/1202241409 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f4ed40a0690 0x7f4ed40a2b50 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:06.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.185+0000 7f4efb800700 1 --2- 172.21.15.1:0/1202241409 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ef419db50 0x7f4ef419dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:06.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.185+0000 7f4efb800700 1 --2- 172.21.15.1:0/1202241409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4ef4100ff0 0x7f4ef41991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:06.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.185+0000 7f4efb800700 1 --2- 172.21.15.1:0/1202241409 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4ef4106110 0x7f4ef41996f0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:06.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.185+0000 7f4efb800700 1 -- 172.21.15.1:0/1202241409 >> 172.21.15.1:0/1202241409 conn(0x7f4ef40fc810 msgr2=0x7f4ef40fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:06.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.185+0000 7f4efb800700 1 -- 172.21.15.1:0/1202241409 shutdown_connections 2024-02-02T15:38:06.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.185+0000 7f4efb800700 1 -- 172.21.15.1:0/1202241409 wait complete. 2024-02-02T15:38:06.204 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-02T15:38:06.264 INFO:tasks.workunit.client.0.smithi001.stdout: 1 619759 5.65 MB/sec execute 460 sec latency 258.828 ms 2024-02-02T15:38:06.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.420+0000 7f4a46800700 1 -- 172.21.15.1:0/3172373398 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a4010d5c0 msgr2=0x7f4a4010fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:06.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.420+0000 7f4a46800700 1 --2- 172.21.15.1:0/3172373398 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a4010d5c0 0x7f4a4010fa30 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f4a3800b0a0 tx=0x7f4a3800b3b0 comp rx=0 tx=0).stop 2024-02-02T15:38:06.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.420+0000 7f4a46800700 1 -- 172.21.15.1:0/3172373398 shutdown_connections 2024-02-02T15:38:06.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.420+0000 7f4a46800700 1 --2- 172.21.15.1:0/3172373398 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a4010d5c0 0x7f4a4010fa30 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:06.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.420+0000 7f4a46800700 1 --2- 172.21.15.1:0/3172373398 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a40073170 0x7f4a40073550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:06.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.420+0000 7f4a46800700 1 --2- 172.21.15.1:0/3172373398 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4a40073a90 0x7f4a4010d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:06.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.420+0000 7f4a46800700 1 -- 172.21.15.1:0/3172373398 >> 172.21.15.1:0/3172373398 conn(0x7f4a400fc810 msgr2=0x7f4a400fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:06.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.420+0000 7f4a46800700 1 -- 172.21.15.1:0/3172373398 shutdown_connections 2024-02-02T15:38:06.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.420+0000 7f4a46800700 1 -- 172.21.15.1:0/3172373398 wait complete. 2024-02-02T15:38:06.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.422+0000 7f4a46800700 1 Processor -- start 2024-02-02T15:38:06.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.422+0000 7f4a46800700 1 -- start start 2024-02-02T15:38:06.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.423+0000 7f4a46800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a40073170 0x7f4a401991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:06.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.423+0000 7f4a46800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4a40073a90 0x7f4a401996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:06.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.423+0000 7f4a46800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a4010d5c0 0x7f4a4019db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:06.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.423+0000 7f4a46800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a4019e180 con 0x7f4a40073a90 2024-02-02T15:38:06.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.423+0000 7f4a46800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a4019e2f0 con 0x7f4a4010d5c0 2024-02-02T15:38:06.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.423+0000 7f4a46800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a4019e4b0 con 0x7f4a40073170 2024-02-02T15:38:06.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.423+0000 7f4a3f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a40073170 0x7f4a401991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:06.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.423+0000 7f4a3f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a40073170 0x7f4a401991a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:46280/0 (socket says 172.21.15.1:46280) 2024-02-02T15:38:06.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.423+0000 7f4a3f400700 1 -- 172.21.15.1:0/3213325075 learned_addr learned my addr 172.21.15.1:0/3213325075 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:38:06.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.423+0000 7f4a3fe00700 1 --2- 172.21.15.1:0/3213325075 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a4010d5c0 0x7f4a4019db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:06.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.423+0000 7f4a3ea00700 1 --2- 172.21.15.1:0/3213325075 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4a40073a90 0x7f4a401996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:06.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.423+0000 7f4a3f400700 1 -- 172.21.15.1:0/3213325075 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a4010d5c0 msgr2=0x7f4a4019db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:06.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.423+0000 7f4a3f400700 1 --2- 172.21.15.1:0/3213325075 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a4010d5c0 0x7f4a4019db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:06.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.423+0000 7f4a3f400700 1 -- 172.21.15.1:0/3213325075 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4a40073a90 msgr2=0x7f4a401996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:06.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.423+0000 7f4a3f400700 1 --2- 172.21.15.1:0/3213325075 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4a40073a90 0x7f4a401996e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:06.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.423+0000 7f4a3f400700 1 -- 172.21.15.1:0/3213325075 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a38009d00 con 0x7f4a40073170 2024-02-02T15:38:06.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.423+0000 7f4a3fe00700 1 --2- 172.21.15.1:0/3213325075 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a4010d5c0 0x7f4a4019db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:38:06.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.424+0000 7f4a3f400700 1 --2- 172.21.15.1:0/3213325075 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a40073170 0x7f4a401991a0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f4a3000e9a0 tx=0x7f4a3000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:06.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.424+0000 7f4a37e00700 1 -- 172.21.15.1:0/3213325075 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a30009790 con 0x7f4a40073170 2024-02-02T15:38:06.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.424+0000 7f4a37e00700 1 -- 172.21.15.1:0/3213325075 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a30010430 con 0x7f4a40073170 2024-02-02T15:38:06.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.424+0000 7f4a46800700 1 -- 172.21.15.1:0/3213325075 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a4019e790 con 0x7f4a40073170 2024-02-02T15:38:06.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.424+0000 7f4a37e00700 1 -- 172.21.15.1:0/3213325075 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a30018680 con 0x7f4a40073170 2024-02-02T15:38:06.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.424+0000 7f4a46800700 1 -- 172.21.15.1:0/3213325075 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a401a5fd0 con 0x7f4a40073170 2024-02-02T15:38:06.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.427+0000 7f4a37e00700 1 -- 172.21.15.1:0/3213325075 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f4a3000ca10 con 0x7f4a40073170 2024-02-02T15:38:06.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.428+0000 7f4a37e00700 1 --2- 172.21.15.1:0/3213325075 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f4a200a0710 0x7f4a200a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:06.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.428+0000 7f4a37e00700 1 -- 172.21.15.1:0/3213325075 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f4a30014070 con 0x7f4a40073170 2024-02-02T15:38:06.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.428+0000 7f4a5a252b80 1 Processor -- start 2024-02-02T15:38:06.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.428+0000 7f4a3ea00700 1 --2- 172.21.15.1:0/3213325075 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f4a200a0710 0x7f4a200a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:06.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.429+0000 7f4a5a252b80 1 -- start start 2024-02-02T15:38:06.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.429+0000 7f4a5a252b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ab1e90c010 0x55ab1e90c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:06.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.429+0000 7f4a5a252b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ab1e90c930 0x55ab1e9172a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:06.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.429+0000 7f4a5a252b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab1e9177e0 0x55ab1e919bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:06.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.429+0000 7f4a5a252b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab1e8804c0 con 0x55ab1e90c930 2024-02-02T15:38:06.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.429+0000 7f4a5a252b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab1e864aa0 con 0x55ab1e90c010 2024-02-02T15:38:06.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.429+0000 7f4a5a252b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab1e862460 con 0x55ab1e9177e0 2024-02-02T15:38:06.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.429+0000 7f4a3f400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ab1e90c010 0x55ab1e90c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:06.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.429+0000 7f4a3ea00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ab1e90c930 0x55ab1e9172a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:06.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.429+0000 7f4a3fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab1e9177e0 0x55ab1e919bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:06.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.429+0000 7f4a3f400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ab1e90c010 0x55ab1e90c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:41124/0 (socket says 172.21.15.1:41124) 2024-02-02T15:38:06.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.429+0000 7f4a3f400700 1 -- 172.21.15.1:0/630664611 learned_addr learned my addr 172.21.15.1:0/630664611 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:38:06.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.429+0000 7f4a3ea00700 1 --2- 172.21.15.1:0/3213325075 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f4a200a0710 0x7f4a200a2bd0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4a4019a7b0 tx=0x7f4a28009260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:06.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.430+0000 7f4a3f400700 1 -- 172.21.15.1:0/630664611 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab1e9177e0 msgr2=0x55ab1e919bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:06.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.430+0000 7f4a3f400700 1 --2- 172.21.15.1:0/630664611 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab1e9177e0 0x55ab1e919bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:06.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.430+0000 7f4a3f400700 1 -- 172.21.15.1:0/630664611 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ab1e90c930 msgr2=0x55ab1e9172a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:06.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.430+0000 7f4a3f400700 1 --2- 172.21.15.1:0/630664611 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ab1e90c930 0x55ab1e9172a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:06.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.430+0000 7f4a3f400700 1 -- 172.21.15.1:0/630664611 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab1e6df3b0 con 0x55ab1e90c010 2024-02-02T15:38:06.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.430+0000 7f4a3f400700 1 --2- 172.21.15.1:0/630664611 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ab1e90c010 0x55ab1e90c3f0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f4a30099310 tx=0x7f4a30098fc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54c2a29119748e74 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:06.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.430+0000 7f4a34c00700 1 -- 172.21.15.1:0/630664611 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a30084070 con 0x55ab1e90c010 2024-02-02T15:38:06.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.430+0000 7f4a34c00700 1 -- 172.21.15.1:0/630664611 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a30099380 con 0x55ab1e90c010 2024-02-02T15:38:06.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.430+0000 7f4a34c00700 1 -- 172.21.15.1:0/630664611 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3000ccc0 con 0x55ab1e90c010 2024-02-02T15:38:06.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.431+0000 7f4a5a252b80 1 -- 172.21.15.1:0/630664611 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ab1e90c010 msgr2=0x55ab1e90c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:06.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.431+0000 7f4a5a252b80 1 --2- 172.21.15.1:0/630664611 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ab1e90c010 0x55ab1e90c3f0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f4a30099310 tx=0x7f4a30098fc0 comp rx=0 tx=0).stop 2024-02-02T15:38:06.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.431+0000 7f4a5a252b80 1 -- 172.21.15.1:0/630664611 shutdown_connections 2024-02-02T15:38:06.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.431+0000 7f4a5a252b80 1 --2- 172.21.15.1:0/630664611 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab1e9177e0 0x55ab1e919bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:06.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.431+0000 7f4a5a252b80 1 --2- 172.21.15.1:0/630664611 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ab1e90c010 0x55ab1e90c3f0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:06.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.431+0000 7f4a5a252b80 1 --2- 172.21.15.1:0/630664611 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ab1e90c930 0x55ab1e9172a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:06.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.431+0000 7f4a5a252b80 1 -- 172.21.15.1:0/630664611 >> 172.21.15.1:0/630664611 conn(0x55ab1e86bdb0 msgr2=0x55ab1e9083b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:06.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.431+0000 7f4a5a252b80 1 -- 172.21.15.1:0/630664611 shutdown_connections 2024-02-02T15:38:06.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.431+0000 7f4a5a252b80 1 -- 172.21.15.1:0/630664611 wait complete. 2024-02-02T15:38:06.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.431+0000 7f4a5a252b80 1 -- start start 2024-02-02T15:38:06.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.432+0000 7f4a5a252b80 1 Processor -- start 2024-02-02T15:38:06.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.432+0000 7f4a5a252b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ab1e90c010 0x55ab1e9144d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:06.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.432+0000 7f4a5a252b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab1e90c930 0x55ab1e914a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:06.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.432+0000 7f4a5a252b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ab1e9177e0 0x55ab1e914f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:06.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.432+0000 7f4a5a252b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab1e85dd50 con 0x55ab1e9177e0 2024-02-02T15:38:06.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.432+0000 7f4a5a252b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab1e7f9d80 con 0x55ab1e90c010 2024-02-02T15:38:06.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.432+0000 7f4a5a252b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab1e7f9a20 con 0x55ab1e90c930 2024-02-02T15:38:06.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.432+0000 7f4a5a252b80 10 client.0 _ll_register_callbacks cb 0x55ab1e7fb290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:38:06.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.432+0000 7f4a3f400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ab1e90c010 0x55ab1e9144d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:06.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.432+0000 7f4a3ea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab1e90c930 0x55ab1e914a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:06.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.433+0000 7f4a3f400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ab1e90c010 0x55ab1e9144d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:41138/0 (socket says 172.21.15.1:41138) 2024-02-02T15:38:06.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.433+0000 7f4a3f400700 1 -- 172.21.15.1:0/1747265044 learned_addr learned my addr 172.21.15.1:0/1747265044 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:38:06.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.433+0000 7f4a3fe00700 1 --2- 172.21.15.1:0/1747265044 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ab1e9177e0 0x55ab1e914f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:06.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.433+0000 7f4a3f400700 1 -- 172.21.15.1:0/1747265044 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab1e90c930 msgr2=0x55ab1e914a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:06.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.433+0000 7f4a3f400700 1 --2- 172.21.15.1:0/1747265044 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab1e90c930 0x55ab1e914a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:06.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.433+0000 7f4a3f400700 1 -- 172.21.15.1:0/1747265044 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ab1e9177e0 msgr2=0x55ab1e914f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:06.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.433+0000 7f4a3f400700 1 --2- 172.21.15.1:0/1747265044 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ab1e9177e0 0x55ab1e914f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:06.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.433+0000 7f4a3f400700 1 -- 172.21.15.1:0/1747265044 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a401a5fd0 con 0x55ab1e90c010 2024-02-02T15:38:06.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.433+0000 7f4a3f400700 1 --2- 172.21.15.1:0/1747265044 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ab1e90c010 0x55ab1e9144d0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f4a30095830 tx=0x7f4a300c95c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c814e66887fce14 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:06.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.433+0000 7f4a2d600700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:38:06.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.433+0000 7f4a2d600700 1 -- 172.21.15.1:0/1747265044 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3009d1b0 con 0x55ab1e90c010 2024-02-02T15:38:06.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.433+0000 7f4a2d600700 1 -- 172.21.15.1:0/1747265044 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f4a300ace40 con 0x55ab1e90c010 2024-02-02T15:38:06.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.433+0000 7f4a5a252b80 1 -- 172.21.15.1:0/1747265044 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ab1e9000f0 con 0x55ab1e90c010 2024-02-02T15:38:06.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.434+0000 7f4a2d600700 1 -- 172.21.15.1:0/1747265044 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a300b7300 con 0x55ab1e90c010 2024-02-02T15:38:06.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.434+0000 7f4a5a252b80 10 client.28634 fetch_fsmap learned FSMap version 51 2024-02-02T15:38:06.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.434+0000 7f4a5a252b80 1 -- 172.21.15.1:0/1747265044 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x55ab1e68bb60 con 0x55ab1e90c010 2024-02-02T15:38:06.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.434+0000 7f4a2d600700 1 -- 172.21.15.1:0/1747265044 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ab1e9000f0 con 0x55ab1e90c010 2024-02-02T15:38:06.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.434+0000 7f4a2d600700 1 -- 172.21.15.1:0/1747265044 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 51) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f4a30098900 con 0x55ab1e90c010 2024-02-02T15:38:06.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.434+0000 7f4a5a252b80 10 client.28634 fetch_fsmap finished waiting for FSMap version 51 2024-02-02T15:38:06.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.434+0000 7f4a5a252b80 10 client.28634 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:38:06.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.434+0000 7f4a5a252b80 20 client.28634 populate_metadata read hostname 'smithi001' 2024-02-02T15:38:06.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.434+0000 7f4a5a252b80 1 --2- 172.21.15.1:0/1747265044 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55ab1e926e90 0x55ab1e9292b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:06.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.434+0000 7f4a5a252b80 4 client.28634 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:38:06.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.434+0000 7f4a5a252b80 1 -- 172.21.15.1:0/1747265044 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ab1e856dd0 con 0x55ab1e926e90 2024-02-02T15:38:06.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.434+0000 7f4a3ea00700 1 --2- 172.21.15.1:0/1747265044 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55ab1e926e90 0x55ab1e9292b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:06.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.435+0000 7f4a3ea00700 1 --2- 172.21.15.1:0/1747265044 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55ab1e926e90 0x55ab1e9292b0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f78d08d1ba4bd2c4 server_cookie=6c41f6bb8c54fbfd in_seq=0 out_seq=0 2024-02-02T15:38:06.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.435+0000 7f4a2d600700 10 client.28634 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:38:06.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.436+0000 7f4a2d600700 1 -- 172.21.15.1:0/1747265044 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55ab1e856dd0 con 0x55ab1e926e90 2024-02-02T15:38:06.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.436+0000 7f4a2d600700 10 client.28634 handle_command_reply: tid=0 2024-02-02T15:38:06.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.436+0000 7f4a5a252b80 1 client.28634 shutdown 2024-02-02T15:38:06.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.436+0000 7f4a5a252b80 1 -- 172.21.15.1:0/1747265044 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ab1e90c010 msgr2=0x55ab1e9144d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:06.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.436+0000 7f4a5a252b80 1 --2- 172.21.15.1:0/1747265044 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ab1e90c010 0x55ab1e9144d0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f4a30095830 tx=0x7f4a300c95c0 comp rx=0 tx=0).stop 2024-02-02T15:38:06.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.436+0000 7f4a5a252b80 1 -- 172.21.15.1:0/1747265044 shutdown_connections 2024-02-02T15:38:06.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.436+0000 7f4a5a252b80 1 --2- 172.21.15.1:0/1747265044 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55ab1e926e90 0x55ab1e9292b0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:06.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.436+0000 7f4a5a252b80 1 --2- 172.21.15.1:0/1747265044 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ab1e9177e0 0x55ab1e914f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:06.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.436+0000 7f4a5a252b80 1 --2- 172.21.15.1:0/1747265044 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ab1e90c010 0x55ab1e9144d0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:06.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.436+0000 7f4a2d600700 0 client.28634 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:38:06.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.436+0000 7f4a5a252b80 1 --2- 172.21.15.1:0/1747265044 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab1e90c930 0x55ab1e914a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:06.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.436+0000 7f4a5a252b80 1 -- 172.21.15.1:0/1747265044 >> 172.21.15.1:0/1747265044 conn(0x55ab1e86bdb0 msgr2=0x55ab1e907fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:06.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.436+0000 7f4a5a252b80 1 -- 172.21.15.1:0/1747265044 shutdown_connections 2024-02-02T15:38:06.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.436+0000 7f4a5a252b80 1 -- 172.21.15.1:0/1747265044 wait complete. 2024-02-02T15:38:06.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.436+0000 7f4a5a252b80 20 client.28634 trim_cache size 0 max 16384 2024-02-02T15:38:06.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.455+0000 7f4a5a252b80 1 Processor -- start 2024-02-02T15:38:06.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.455+0000 7f4a5a252b80 1 -- start start 2024-02-02T15:38:06.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.455+0000 7f4a5a252b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ab1e90c010 0x55ab1e90f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:06.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.455+0000 7f4a5a252b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab1e90c930 0x55ab1e90fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:06.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.455+0000 7f4a5a252b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ab1e9177e0 0x55ab1e90e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:06.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.455+0000 7f4a5a252b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab1e6e16a0 con 0x55ab1e90c010 2024-02-02T15:38:06.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.455+0000 7f4a5a252b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab1e856dd0 con 0x55ab1e9177e0 2024-02-02T15:38:06.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.455+0000 7f4a5a252b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab1e90cd10 con 0x55ab1e90c930 2024-02-02T15:38:06.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.456+0000 7f4a3fe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ab1e9177e0 0x55ab1e90e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:06.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.456+0000 7f4a3ea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab1e90c930 0x55ab1e90fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:06.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.456+0000 7f4a3fe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ab1e9177e0 0x55ab1e90e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:41148/0 (socket says 172.21.15.1:41148) 2024-02-02T15:38:06.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.456+0000 7f4a3fe00700 1 -- 172.21.15.1:0/2095269058 learned_addr learned my addr 172.21.15.1:0/2095269058 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:38:06.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.456+0000 7f4a3f400700 1 --2- 172.21.15.1:0/2095269058 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ab1e90c010 0x55ab1e90f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:06.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.456+0000 7f4a3fe00700 1 -- 172.21.15.1:0/2095269058 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab1e90c930 msgr2=0x55ab1e90fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:06.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.456+0000 7f4a3fe00700 1 --2- 172.21.15.1:0/2095269058 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab1e90c930 0x55ab1e90fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:06.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.456+0000 7f4a3fe00700 1 -- 172.21.15.1:0/2095269058 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ab1e90c010 msgr2=0x55ab1e90f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:06.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.456+0000 7f4a3fe00700 1 --2- 172.21.15.1:0/2095269058 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ab1e90c010 0x55ab1e90f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:06.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.456+0000 7f4a3fe00700 1 -- 172.21.15.1:0/2095269058 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab1e823ea0 con 0x55ab1e9177e0 2024-02-02T15:38:06.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.456+0000 7f4a3ea00700 1 --2- 172.21.15.1:0/2095269058 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab1e90c930 0x55ab1e90fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:38:06.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.456+0000 7f4a3fe00700 1 --2- 172.21.15.1:0/2095269058 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ab1e9177e0 0x55ab1e90e840 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f4a38015000 tx=0x7f4a38009540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af0cefa0385ee4ff server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:06.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.457+0000 7f4a35600700 1 -- 172.21.15.1:0/2095269058 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3800e040 con 0x55ab1e9177e0 2024-02-02T15:38:06.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.457+0000 7f4a35600700 1 -- 172.21.15.1:0/2095269058 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a3800be70 con 0x55ab1e9177e0 2024-02-02T15:38:06.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.457+0000 7f4a35600700 1 -- 172.21.15.1:0/2095269058 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a38027800 con 0x55ab1e9177e0 2024-02-02T15:38:06.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.458+0000 7f4a5a252b80 1 -- 172.21.15.1:0/2095269058 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ab1e9177e0 msgr2=0x55ab1e90e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:06.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.458+0000 7f4a5a252b80 1 --2- 172.21.15.1:0/2095269058 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ab1e9177e0 0x55ab1e90e840 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f4a38015000 tx=0x7f4a38009540 comp rx=0 tx=0).stop 2024-02-02T15:38:06.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.458+0000 7f4a5a252b80 1 -- 172.21.15.1:0/2095269058 shutdown_connections 2024-02-02T15:38:06.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.458+0000 7f4a5a252b80 1 --2- 172.21.15.1:0/2095269058 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ab1e9177e0 0x55ab1e90e840 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:06.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.458+0000 7f4a5a252b80 1 --2- 172.21.15.1:0/2095269058 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ab1e90c010 0x55ab1e90f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:06.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.458+0000 7f4a5a252b80 1 --2- 172.21.15.1:0/2095269058 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab1e90c930 0x55ab1e90fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:06.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.458+0000 7f4a5a252b80 1 -- 172.21.15.1:0/2095269058 >> 172.21.15.1:0/2095269058 conn(0x55ab1e86bdb0 msgr2=0x55ab1e922900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:06.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.458+0000 7f4a5a252b80 1 -- 172.21.15.1:0/2095269058 shutdown_connections 2024-02-02T15:38:06.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.458+0000 7f4a5a252b80 1 -- 172.21.15.1:0/2095269058 wait complete. 2024-02-02T15:38:06.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.458+0000 7f4a5a252b80 1 -- start start 2024-02-02T15:38:06.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.458+0000 7f4a5a252b80 1 Processor -- start 2024-02-02T15:38:06.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.459+0000 7f4a5a252b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab1e90c010 0x55ab1e91e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:06.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.459+0000 7f4a5a252b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ab1e90c930 0x55ab1e91c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:06.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.459+0000 7f4a5a252b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ab1e9177e0 0x55ab1e91ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:06.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.459+0000 7f4a5a252b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab1e91d140 con 0x55ab1e90c930 2024-02-02T15:38:06.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.459+0000 7f4a5a252b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab1e91d2b0 con 0x55ab1e9177e0 2024-02-02T15:38:06.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.459+0000 7f4a5a252b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab1e9789c0 con 0x55ab1e90c010 2024-02-02T15:38:06.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.459+0000 7f4a5a252b80 10 client.0 _ll_register_callbacks cb 0x55ab1e7fb290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:38:06.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.459+0000 7f4a3f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab1e90c010 0x55ab1e91e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:06.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.459+0000 7f4a3f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab1e90c010 0x55ab1e91e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:46310/0 (socket says 172.21.15.1:46310) 2024-02-02T15:38:06.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.459+0000 7f4a3f400700 1 -- 172.21.15.1:0/2457071592 learned_addr learned my addr 172.21.15.1:0/2457071592 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:38:06.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.459+0000 7f4a3fe00700 1 --2- 172.21.15.1:0/2457071592 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ab1e9177e0 0x55ab1e91ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:06.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.459+0000 7f4a3ea00700 1 --2- 172.21.15.1:0/2457071592 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ab1e90c930 0x55ab1e91c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:06.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.460+0000 7f4a3f400700 1 -- 172.21.15.1:0/2457071592 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ab1e9177e0 msgr2=0x55ab1e91ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:06.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.460+0000 7f4a3f400700 1 --2- 172.21.15.1:0/2457071592 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ab1e9177e0 0x55ab1e91ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:06.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.460+0000 7f4a3f400700 1 -- 172.21.15.1:0/2457071592 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ab1e90c930 msgr2=0x55ab1e91c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:06.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.460+0000 7f4a3f400700 1 --2- 172.21.15.1:0/2457071592 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ab1e90c930 0x55ab1e91c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:06.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.460+0000 7f4a3f400700 1 -- 172.21.15.1:0/2457071592 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a28017070 con 0x55ab1e90c010 2024-02-02T15:38:06.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.460+0000 7f4a3fe00700 1 --2- 172.21.15.1:0/2457071592 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ab1e9177e0 0x55ab1e91ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:38:06.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.460+0000 7f4a3f400700 1 --2- 172.21.15.1:0/2457071592 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab1e90c010 0x55ab1e91e190 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f4a30009230 tx=0x7f4a30098aa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69cf18ea8f9ee810 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:06.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.460+0000 7f4a2d600700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:38:06.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.460+0000 7f4a2d600700 1 -- 172.21.15.1:0/2457071592 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a30084070 con 0x55ab1e90c010 2024-02-02T15:38:06.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.460+0000 7f4a2d600700 1 -- 172.21.15.1:0/2457071592 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(97..97 src has 1..97) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f4a300ace40 con 0x55ab1e90c010 2024-02-02T15:38:06.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.460+0000 7f4a5a252b80 1 -- 172.21.15.1:0/2457071592 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4a30099380 con 0x55ab1e90c010 2024-02-02T15:38:06.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.461+0000 7f4a2d600700 1 -- 172.21.15.1:0/2457071592 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a300a1150 con 0x55ab1e90c010 2024-02-02T15:38:06.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.461+0000 7f4a5a252b80 10 client.28645 fetch_fsmap learned FSMap version 51 2024-02-02T15:38:06.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.461+0000 7f4a5a252b80 1 -- 172.21.15.1:0/2457071592 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x7f4a401106f0 con 0x55ab1e90c010 2024-02-02T15:38:06.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.461+0000 7f4a2d600700 1 -- 172.21.15.1:0/2457071592 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a30099380 con 0x55ab1e90c010 2024-02-02T15:38:06.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.461+0000 7f4a2d600700 1 -- 172.21.15.1:0/2457071592 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 51) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f4a30095170 con 0x55ab1e90c010 2024-02-02T15:38:06.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.461+0000 7f4a5a252b80 10 client.28645 fetch_fsmap finished waiting for FSMap version 51 2024-02-02T15:38:06.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.461+0000 7f4a5a252b80 10 client.28645 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:38:06.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.461+0000 7f4a5a252b80 20 client.28645 populate_metadata read hostname 'smithi001' 2024-02-02T15:38:06.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.461+0000 7f4a5a252b80 1 --2- 172.21.15.1:0/2457071592 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55ab1e979b10 0x55ab1e97bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:06.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.461+0000 7f4a5a252b80 4 client.28645 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2024-02-02T15:38:06.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.461+0000 7f4a5a252b80 1 -- 172.21.15.1:0/2457071592 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ab1e81cf50 con 0x55ab1e979b10 2024-02-02T15:38:06.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.462+0000 7f4a3ea00700 1 --2- 172.21.15.1:0/2457071592 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55ab1e979b10 0x55ab1e97bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:06.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.462+0000 7f4a3ea00700 1 --2- 172.21.15.1:0/2457071592 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55ab1e979b10 0x55ab1e97bf00 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97dd6f67b606d362 server_cookie=4950a099737e4aac in_seq=0 out_seq=0 2024-02-02T15:38:06.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.462+0000 7f4a2d600700 10 client.28645 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:38:06.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.463+0000 7f4a2d600700 1 -- 172.21.15.1:0/2457071592 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55ab1e81cf50 con 0x55ab1e979b10 2024-02-02T15:38:06.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.463+0000 7f4a2d600700 10 client.28645 handle_command_reply: tid=0 2024-02-02T15:38:06.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.463+0000 7f4a5a252b80 1 client.28645 shutdown 2024-02-02T15:38:06.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.463+0000 7f4a5a252b80 1 -- 172.21.15.1:0/2457071592 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab1e90c010 msgr2=0x55ab1e91e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:06.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.463+0000 7f4a5a252b80 1 --2- 172.21.15.1:0/2457071592 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab1e90c010 0x55ab1e91e190 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f4a30009230 tx=0x7f4a30098aa0 comp rx=0 tx=0).stop 2024-02-02T15:38:06.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.463+0000 7f4a5a252b80 1 -- 172.21.15.1:0/2457071592 shutdown_connections 2024-02-02T15:38:06.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.463+0000 7f4a5a252b80 1 --2- 172.21.15.1:0/2457071592 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55ab1e979b10 0x55ab1e97bf00 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:06.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.463+0000 7f4a5a252b80 1 --2- 172.21.15.1:0/2457071592 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ab1e9177e0 0x55ab1e91ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:06.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.463+0000 7f4a5a252b80 1 --2- 172.21.15.1:0/2457071592 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab1e90c010 0x55ab1e91e190 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:06.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.463+0000 7f4a5a252b80 1 --2- 172.21.15.1:0/2457071592 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ab1e90c930 0x55ab1e91c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:06.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.463+0000 7f4a2d600700 0 client.28645 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:38:06.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.463+0000 7f4a5a252b80 1 -- 172.21.15.1:0/2457071592 >> 172.21.15.1:0/2457071592 conn(0x55ab1e86bdb0 msgr2=0x55ab1e922900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:06.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.463+0000 7f4a5a252b80 1 -- 172.21.15.1:0/2457071592 shutdown_connections 2024-02-02T15:38:06.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.463+0000 7f4a5a252b80 1 -- 172.21.15.1:0/2457071592 wait complete. 2024-02-02T15:38:06.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.464+0000 7f4a5a252b80 20 client.28645 trim_cache size 0 max 16384 2024-02-02T15:38:06.467 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:38:06.467 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-02T15:38:06.467 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2024-02-02T15:38:06.467 INFO:teuthology.orchestra.run.smithi001.stdout: "9283363d-61f2-4651-ad5f-58c10d663219": { 2024-02-02T15:38:06.467 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2024-02-02T15:38:06.467 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "9283363d-61f2-4651-ad5f-58c10d663219", 2024-02-02T15:38:06.467 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,force" 2024-02-02T15:38:06.467 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:38:06.467 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:38:06.467 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:38:06.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.464+0000 7f4a46800700 1 -- 172.21.15.1:0/3213325075 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f4a200a0710 msgr2=0x7f4a200a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:06.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.464+0000 7f4a46800700 1 --2- 172.21.15.1:0/3213325075 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f4a200a0710 0x7f4a200a2bd0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4a4019a7b0 tx=0x7f4a28009260 comp rx=0 tx=0).stop 2024-02-02T15:38:06.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.464+0000 7f4a46800700 1 -- 172.21.15.1:0/3213325075 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a40073170 msgr2=0x7f4a401991a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:06.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.464+0000 7f4a46800700 1 --2- 172.21.15.1:0/3213325075 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a40073170 0x7f4a401991a0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f4a3000e9a0 tx=0x7f4a3000ecb0 comp rx=0 tx=0).stop 2024-02-02T15:38:06.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.464+0000 7f4a46800700 1 -- 172.21.15.1:0/3213325075 shutdown_connections 2024-02-02T15:38:06.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.464+0000 7f4a46800700 1 --2- 172.21.15.1:0/3213325075 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f4a200a0710 0x7f4a200a2bd0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:06.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.464+0000 7f4a46800700 1 --2- 172.21.15.1:0/3213325075 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a4010d5c0 0x7f4a4019db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:06.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.464+0000 7f4a46800700 1 --2- 172.21.15.1:0/3213325075 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a40073170 0x7f4a401991a0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:06.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.464+0000 7f4a46800700 1 --2- 172.21.15.1:0/3213325075 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4a40073a90 0x7f4a401996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:06.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.464+0000 7f4a46800700 1 -- 172.21.15.1:0/3213325075 >> 172.21.15.1:0/3213325075 conn(0x7f4a400fc810 msgr2=0x7f4a40105020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:06.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.464+0000 7f4a46800700 1 -- 172.21.15.1:0/3213325075 shutdown_connections 2024-02-02T15:38:06.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:06.464+0000 7f4a46800700 1 -- 172.21.15.1:0/3213325075 wait complete. 2024-02-02T15:38:06.484 INFO:tasks.cephfs.filesystem:scrub status for tag:9283363d-61f2-4651-ad5f-58c10d663219 - {'path': '/', 'tag': '9283363d-61f2-4651-ad5f-58c10d663219', 'options': 'recursive,force'} 2024-02-02T15:38:06.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:06 smithi001 ceph-mon[35470]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:38:06.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:06 smithi071 ceph-mon[44707]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:38:06.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:06 smithi142 ceph-mon[45728]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:38:07.265 INFO:tasks.workunit.client.0.smithi001.stdout: 1 621744 5.67 MB/sec execute 461 sec latency 47.493 ms 2024-02-02T15:38:07.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:07 smithi071 ceph-mon[44707]: scrub queued for path: / 2024-02-02T15:38:07.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:07 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:38:07.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:07 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:38:07.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:07 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:38:07.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:07 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:38:07.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:07 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:38:07.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:07 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:38:07.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:07 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:38:07.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:07 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:38:07.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:07 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:38:07.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:07 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:38:07.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:07 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:38:07.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:07 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:38:07.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:07 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:38:07.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:07 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:38:07.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:07 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:38:07.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:07 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:38:07.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:07 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:38:07.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:07 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:38:07.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:07 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:38:07.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:07 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:38:07.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:07 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:38:07.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:07 smithi071 ceph-mon[44707]: pgmap v344: 129 pgs: 129 active+clean; 499 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 202 op/s 2024-02-02T15:38:07.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:07 smithi142 ceph-mon[45728]: scrub queued for path: / 2024-02-02T15:38:07.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:07 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:38:07.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:07 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:38:07.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:07 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:38:07.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:07 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:38:07.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:07 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:38:07.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:07 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:38:07.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:07 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:38:07.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:07 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:38:07.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:07 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:38:07.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:07 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:38:07.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:07 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:38:07.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:07 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:38:07.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:07 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:38:07.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:07 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:38:07.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:07 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:38:07.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:07 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:38:07.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:07 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:38:07.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:07 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:38:07.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:07 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:38:07.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:07 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:38:07.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:07 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:38:07.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:07 smithi142 ceph-mon[45728]: pgmap v344: 129 pgs: 129 active+clean; 499 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 202 op/s 2024-02-02T15:38:07.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:07 smithi001 ceph-mon[35470]: scrub queued for path: / 2024-02-02T15:38:07.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:07 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:38:07.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:07 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:38:07.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:07 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:38:07.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:07 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:38:07.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:07 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:38:07.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:07 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:38:07.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:07 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:38:07.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:07 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:38:07.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:07 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:38:07.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:07 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:38:07.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:07 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:38:07.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:07 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:38:07.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:07 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:38:07.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:07 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:38:07.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:07 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:38:07.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:07 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:38:07.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:07 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:38:07.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:07 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:38:07.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:07 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:38:07.942 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:07 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:38:07.942 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:07 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:38:07.942 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:07 smithi001 ceph-mon[35470]: pgmap v344: 129 pgs: 129 active+clean; 499 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 202 op/s 2024-02-02T15:38:08.265 INFO:tasks.workunit.client.0.smithi001.stdout: 1 622441 5.66 MB/sec execute 462 sec latency 67.958 ms 2024-02-02T15:38:08.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:08 smithi071 ceph-mon[44707]: osdmap e98: 12 total, 12 up, 12 in 2024-02-02T15:38:08.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:08 smithi142 ceph-mon[45728]: osdmap e98: 12 total, 12 up, 12 in 2024-02-02T15:38:08.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:08 smithi001 ceph-mon[35470]: osdmap e98: 12 total, 12 up, 12 in 2024-02-02T15:38:09.265 INFO:tasks.workunit.client.0.smithi001.stdout: 1 623219 5.66 MB/sec execute 463 sec latency 217.793 ms 2024-02-02T15:38:09.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:09 smithi071 ceph-mon[44707]: pgmap v346: 129 pgs: 129 active+clean; 499 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 165 op/s 2024-02-02T15:38:09.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:09 smithi142 ceph-mon[45728]: pgmap v346: 129 pgs: 129 active+clean; 499 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 165 op/s 2024-02-02T15:38:09.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:09 smithi001 ceph-mon[35470]: pgmap v346: 129 pgs: 129 active+clean; 499 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 165 op/s 2024-02-02T15:38:10.265 INFO:tasks.workunit.client.0.smithi001.stdout: 1 623219 5.65 MB/sec execute 464 sec latency 1217.866 ms 2024-02-02T15:38:11.265 INFO:tasks.workunit.client.0.smithi001.stdout: 1 623219 5.63 MB/sec execute 465 sec latency 2217.987 ms 2024-02-02T15:38:11.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:11 smithi071 ceph-mon[44707]: pgmap v347: 129 pgs: 129 active+clean; 499 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 165 op/s 2024-02-02T15:38:11.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:11 smithi142 ceph-mon[45728]: pgmap v347: 129 pgs: 129 active+clean; 499 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 165 op/s 2024-02-02T15:38:11.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:11 smithi001 ceph-mon[35470]: pgmap v347: 129 pgs: 129 active+clean; 499 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 165 op/s 2024-02-02T15:38:12.265 INFO:tasks.workunit.client.0.smithi001.stdout: 1 623219 5.62 MB/sec execute 466 sec latency 3218.059 ms 2024-02-02T15:38:13.265 INFO:tasks.workunit.client.0.smithi001.stdout: 1 624440 5.63 MB/sec execute 467 sec latency 3598.666 ms 2024-02-02T15:38:13.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:13 smithi071 ceph-mon[44707]: pgmap v348: 129 pgs: 129 active+clean; 501 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.3 MiB/s wr, 156 op/s 2024-02-02T15:38:13.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:13 smithi142 ceph-mon[45728]: pgmap v348: 129 pgs: 129 active+clean; 501 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.3 MiB/s wr, 156 op/s 2024-02-02T15:38:13.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:13 smithi001 ceph-mon[35470]: pgmap v348: 129 pgs: 129 active+clean; 501 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.3 MiB/s wr, 156 op/s 2024-02-02T15:38:14.265 INFO:tasks.workunit.client.0.smithi001.stdout: 1 625612 5.63 MB/sec execute 468 sec latency 11.882 ms 2024-02-02T15:38:15.265 INFO:tasks.workunit.client.0.smithi001.stdout: 1 626670 5.63 MB/sec execute 469 sec latency 20.522 ms 2024-02-02T15:38:15.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:15 smithi071 ceph-mon[44707]: pgmap v349: 129 pgs: 129 active+clean; 501 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.3 MiB/s wr, 156 op/s 2024-02-02T15:38:15.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:15 smithi142 ceph-mon[45728]: pgmap v349: 129 pgs: 129 active+clean; 501 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.3 MiB/s wr, 156 op/s 2024-02-02T15:38:15.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:15 smithi001 ceph-mon[35470]: pgmap v349: 129 pgs: 129 active+clean; 501 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.3 MiB/s wr, 156 op/s 2024-02-02T15:38:16.266 INFO:tasks.workunit.client.0.smithi001.stdout: 1 626768 5.61 MB/sec execute 470 sec latency 884.475 ms 2024-02-02T15:38:17.266 INFO:tasks.workunit.client.0.smithi001.stdout: 1 626768 5.60 MB/sec execute 471 sec latency 1884.661 ms 2024-02-02T15:38:17.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:17 smithi001 ceph-mon[35470]: pgmap v350: 129 pgs: 129 active+clean; 508 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.4 MiB/s wr, 123 op/s 2024-02-02T15:38:17.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:17 smithi001 ceph-mon[35470]: osdmap e99: 12 total, 12 up, 12 in 2024-02-02T15:38:17.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:17 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:38:17.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:17 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:38:17.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:17 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:38:17.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:17 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:38:17.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:17 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:38:17.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:17 smithi071 ceph-mon[44707]: pgmap v350: 129 pgs: 129 active+clean; 508 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.4 MiB/s wr, 123 op/s 2024-02-02T15:38:17.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:17 smithi071 ceph-mon[44707]: osdmap e99: 12 total, 12 up, 12 in 2024-02-02T15:38:17.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:17 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:38:17.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:17 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:38:17.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:17 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:38:17.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:17 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:38:17.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:17 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:38:17.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:17 smithi142 ceph-mon[45728]: pgmap v350: 129 pgs: 129 active+clean; 508 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.4 MiB/s wr, 123 op/s 2024-02-02T15:38:17.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:17 smithi142 ceph-mon[45728]: osdmap e99: 12 total, 12 up, 12 in 2024-02-02T15:38:17.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:17 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:38:17.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:17 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:38:17.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:17 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:38:17.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:17 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:38:17.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:17 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:38:18.266 INFO:tasks.workunit.client.0.smithi001.stdout: 1 627969 5.60 MB/sec execute 472 sec latency 2273.115 ms 2024-02-02T15:38:19.266 INFO:tasks.workunit.client.0.smithi001.stdout: 1 629095 5.61 MB/sec execute 473 sec latency 15.254 ms 2024-02-02T15:38:19.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:19 smithi001 ceph-mon[35470]: pgmap v352: 129 pgs: 129 active+clean; 508 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.4 MiB/s wr, 123 op/s 2024-02-02T15:38:19.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:19 smithi071 ceph-mon[44707]: pgmap v352: 129 pgs: 129 active+clean; 508 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.4 MiB/s wr, 123 op/s 2024-02-02T15:38:19.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:19 smithi142 ceph-mon[45728]: pgmap v352: 129 pgs: 129 active+clean; 508 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.4 MiB/s wr, 123 op/s 2024-02-02T15:38:20.266 INFO:tasks.workunit.client.0.smithi001.stdout: 1 630147 5.61 MB/sec execute 474 sec latency 31.101 ms 2024-02-02T15:38:20.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:20 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:38:20.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:20 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:38:20.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:20 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:38:21.266 INFO:tasks.workunit.client.0.smithi001.stdout: 1 630309 5.60 MB/sec execute 475 sec latency 774.190 ms 2024-02-02T15:38:21.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:21 smithi001 ceph-mon[35470]: pgmap v353: 129 pgs: 129 active+clean; 508 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.4 MiB/s wr, 123 op/s 2024-02-02T15:38:21.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:21 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:38:21.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:21 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:38:21.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:21 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:38:21.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:21 smithi071 ceph-mon[44707]: pgmap v353: 129 pgs: 129 active+clean; 508 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.4 MiB/s wr, 123 op/s 2024-02-02T15:38:21.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:21 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:38:21.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:21 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:38:21.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:21 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:38:21.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:21 smithi142 ceph-mon[45728]: pgmap v353: 129 pgs: 129 active+clean; 508 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.4 MiB/s wr, 123 op/s 2024-02-02T15:38:21.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:21 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:38:21.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:21 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:38:21.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:21 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:38:22.266 INFO:tasks.workunit.client.0.smithi001.stdout: 1 630309 5.58 MB/sec execute 476 sec latency 1774.240 ms 2024-02-02T15:38:23.266 INFO:tasks.workunit.client.0.smithi001.stdout: 1 631497 5.58 MB/sec execute 477 sec latency 2165.684 ms 2024-02-02T15:38:23.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:23 smithi071 ceph-mon[44707]: pgmap v354: 129 pgs: 129 active+clean; 514 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 136 op/s 2024-02-02T15:38:23.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:23 smithi142 ceph-mon[45728]: pgmap v354: 129 pgs: 129 active+clean; 514 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 136 op/s 2024-02-02T15:38:23.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:23 smithi001 ceph-mon[35470]: pgmap v354: 129 pgs: 129 active+clean; 514 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 136 op/s 2024-02-02T15:38:24.266 INFO:tasks.workunit.client.0.smithi001.stdout: 1 632606 5.59 MB/sec execute 478 sec latency 46.312 ms 2024-02-02T15:38:25.266 INFO:tasks.workunit.client.0.smithi001.stdout: 1 633685 5.59 MB/sec execute 479 sec latency 18.678 ms 2024-02-02T15:38:25.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:25 smithi071 ceph-mon[44707]: pgmap v355: 129 pgs: 129 active+clean; 514 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 136 op/s 2024-02-02T15:38:25.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:25 smithi142 ceph-mon[45728]: pgmap v355: 129 pgs: 129 active+clean; 514 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 136 op/s 2024-02-02T15:38:25.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:25 smithi001 ceph-mon[35470]: pgmap v355: 129 pgs: 129 active+clean; 514 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 136 op/s 2024-02-02T15:38:26.266 INFO:tasks.workunit.client.0.smithi001.stdout: 1 633858 5.58 MB/sec execute 480 sec latency 745.859 ms 2024-02-02T15:38:27.267 INFO:tasks.workunit.client.0.smithi001.stdout: 1 633858 5.57 MB/sec execute 481 sec latency 1745.990 ms 2024-02-02T15:38:27.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:27 smithi071 ceph-mon[44707]: pgmap v356: 129 pgs: 129 active+clean; 519 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 139 op/s 2024-02-02T15:38:27.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:27 smithi142 ceph-mon[45728]: pgmap v356: 129 pgs: 129 active+clean; 519 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 139 op/s 2024-02-02T15:38:27.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:27 smithi001 ceph-mon[35470]: pgmap v356: 129 pgs: 129 active+clean; 519 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 139 op/s 2024-02-02T15:38:28.267 INFO:tasks.workunit.client.0.smithi001.stdout: 1 635041 5.57 MB/sec execute 482 sec latency 2136.868 ms 2024-02-02T15:38:29.267 INFO:tasks.workunit.client.0.smithi001.stdout: 1 636169 5.57 MB/sec execute 483 sec latency 14.507 ms 2024-02-02T15:38:29.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:29 smithi071 ceph-mon[44707]: pgmap v357: 129 pgs: 129 active+clean; 519 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 116 op/s 2024-02-02T15:38:29.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:29 smithi142 ceph-mon[45728]: pgmap v357: 129 pgs: 129 active+clean; 519 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 116 op/s 2024-02-02T15:38:29.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:29 smithi001 ceph-mon[35470]: pgmap v357: 129 pgs: 129 active+clean; 519 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 116 op/s 2024-02-02T15:38:30.267 INFO:tasks.workunit.client.0.smithi001.stdout: 1 637243 5.57 MB/sec execute 484 sec latency 18.001 ms 2024-02-02T15:38:31.267 INFO:tasks.workunit.client.0.smithi001.stdout: 1 638140 5.56 MB/sec execute 485 sec latency 440.648 ms 2024-02-02T15:38:31.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:31 smithi071 ceph-mon[44707]: pgmap v358: 129 pgs: 129 active+clean; 519 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 115 op/s 2024-02-02T15:38:31.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:31 smithi142 ceph-mon[45728]: pgmap v358: 129 pgs: 129 active+clean; 519 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 115 op/s 2024-02-02T15:38:31.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:31 smithi001 ceph-mon[35470]: pgmap v358: 129 pgs: 129 active+clean; 519 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 115 op/s 2024-02-02T15:38:32.267 INFO:tasks.workunit.client.0.smithi001.stdout: 1 639613 5.58 MB/sec execute 486 sec latency 17.129 ms 2024-02-02T15:38:33.267 INFO:tasks.workunit.client.0.smithi001.stdout: 1 640815 5.58 MB/sec execute 487 sec latency 18.318 ms 2024-02-02T15:38:33.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:33 smithi071 ceph-mon[44707]: pgmap v359: 129 pgs: 129 active+clean; 524 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 175 op/s 2024-02-02T15:38:33.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:33 smithi142 ceph-mon[45728]: pgmap v359: 129 pgs: 129 active+clean; 524 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 175 op/s 2024-02-02T15:38:33.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:33 smithi001 ceph-mon[35470]: pgmap v359: 129 pgs: 129 active+clean; 524 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 175 op/s 2024-02-02T15:38:34.267 INFO:tasks.workunit.client.0.smithi001.stdout: 1 640948 5.56 MB/sec execute 488 sec latency 884.241 ms 2024-02-02T15:38:35.267 INFO:tasks.workunit.client.0.smithi001.stdout: 1 640948 5.55 MB/sec execute 489 sec latency 1884.393 ms 2024-02-02T15:38:35.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:35 smithi071 ceph-mon[44707]: pgmap v360: 129 pgs: 129 active+clean; 524 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 116 op/s 2024-02-02T15:38:35.925 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:35 smithi142 ceph-mon[45728]: pgmap v360: 129 pgs: 129 active+clean; 524 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 116 op/s 2024-02-02T15:38:35.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:35 smithi001 ceph-mon[35470]: pgmap v360: 129 pgs: 129 active+clean; 524 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 116 op/s 2024-02-02T15:38:36.267 INFO:tasks.workunit.client.0.smithi001.stdout: 1 640948 5.54 MB/sec execute 490 sec latency 2884.514 ms 2024-02-02T15:38:36.486 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-02T15:38:36.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.718+0000 7fb769e00700 1 -- 172.21.15.1:0/4115711889 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb764101000 msgr2=0x7fb7641013e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:36.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.718+0000 7fb769e00700 1 --2- 172.21.15.1:0/4115711889 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb764101000 0x7fb7641013e0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fb75400faf0 tx=0x7fb75400fe00 comp rx=0 tx=0).stop 2024-02-02T15:38:36.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.718+0000 7fb769e00700 1 -- 172.21.15.1:0/4115711889 shutdown_connections 2024-02-02T15:38:36.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.718+0000 7fb769e00700 1 --2- 172.21.15.1:0/4115711889 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb764106120 0x7fb76410dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:36.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.718+0000 7fb769e00700 1 --2- 172.21.15.1:0/4115711889 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb764101000 0x7fb7641013e0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:36.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.718+0000 7fb769e00700 1 --2- 172.21.15.1:0/4115711889 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7641019b0 0x7fb7641059f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:36.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.718+0000 7fb769e00700 1 -- 172.21.15.1:0/4115711889 >> 172.21.15.1:0/4115711889 conn(0x7fb7640fc860 msgr2=0x7fb7640fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:36.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.719+0000 7fb769e00700 1 -- 172.21.15.1:0/4115711889 shutdown_connections 2024-02-02T15:38:36.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.719+0000 7fb769e00700 1 -- 172.21.15.1:0/4115711889 wait complete. 2024-02-02T15:38:36.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.720+0000 7fb769e00700 1 Processor -- start 2024-02-02T15:38:36.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.720+0000 7fb769e00700 1 -- start start 2024-02-02T15:38:36.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.721+0000 7fb769e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb764101000 0x7fb764194ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:36.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.721+0000 7fb769e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb764106120 0x7fb764195220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:36.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.721+0000 7fb769e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7641985d0 0x7fb764198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:36.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.721+0000 7fb769e00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb764198fc0 con 0x7fb764106120 2024-02-02T15:38:36.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.721+0000 7fb769e00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb764199130 con 0x7fb764101000 2024-02-02T15:38:36.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.721+0000 7fb769e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7641992f0 con 0x7fb7641985d0 2024-02-02T15:38:36.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.721+0000 7fb762000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb764106120 0x7fb764195220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:36.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.721+0000 7fb762000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb764106120 0x7fb764195220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36684/0 (socket says 172.21.15.1:36684) 2024-02-02T15:38:36.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.721+0000 7fb762000700 1 -- 172.21.15.1:0/1991474431 learned_addr learned my addr 172.21.15.1:0/1991474431 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:38:36.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.721+0000 7fb763400700 1 --2- 172.21.15.1:0/1991474431 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7641985d0 0x7fb764198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:36.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.721+0000 7fb762a00700 1 --2- 172.21.15.1:0/1991474431 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb764101000 0x7fb764194ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:36.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.721+0000 7fb762000700 1 -- 172.21.15.1:0/1991474431 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7641985d0 msgr2=0x7fb764198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:36.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.721+0000 7fb762000700 1 --2- 172.21.15.1:0/1991474431 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7641985d0 0x7fb764198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:36.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.721+0000 7fb762000700 1 -- 172.21.15.1:0/1991474431 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb764101000 msgr2=0x7fb764194ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:36.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.721+0000 7fb762000700 1 --2- 172.21.15.1:0/1991474431 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb764101000 0x7fb764194ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:36.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.721+0000 7fb762000700 1 -- 172.21.15.1:0/1991474431 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb75400f750 con 0x7fb764106120 2024-02-02T15:38:36.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.722+0000 7fb762000700 1 --2- 172.21.15.1:0/1991474431 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb764106120 0x7fb764195220 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fb74c00e970 tx=0x7fb74c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:36.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.722+0000 7fb768a00700 1 -- 172.21.15.1:0/1991474431 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb74c009790 con 0x7fb764106120 2024-02-02T15:38:36.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.722+0000 7fb769e00700 1 -- 172.21.15.1:0/1991474431 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7641a1960 con 0x7fb764106120 2024-02-02T15:38:36.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.722+0000 7fb768a00700 1 -- 172.21.15.1:0/1991474431 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb74c00c7e0 con 0x7fb764106120 2024-02-02T15:38:36.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.722+0000 7fb768a00700 1 -- 172.21.15.1:0/1991474431 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb74c0054e0 con 0x7fb764106120 2024-02-02T15:38:36.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.722+0000 7fb769e00700 1 -- 172.21.15.1:0/1991474431 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7641a1ec0 con 0x7fb764106120 2024-02-02T15:38:36.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.726+0000 7fb768a00700 1 -- 172.21.15.1:0/1991474431 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7fb74c004a30 con 0x7fb764106120 2024-02-02T15:38:36.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.727+0000 7fb77d9aab80 1 Processor -- start 2024-02-02T15:38:36.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.727+0000 7fb77d9aab80 1 -- start start 2024-02-02T15:38:36.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.727+0000 7fb768a00700 1 --2- 172.21.15.1:0/1991474431 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fb7440a0740 0x7fb7440a2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:36.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.728+0000 7fb77d9aab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe29ed9010 0x55fe29ed93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:36.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.728+0000 7fb77d9aab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe29ed9930 0x55fe29ee42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:36.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.728+0000 7fb768a00700 1 -- 172.21.15.1:0/1991474431 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fb74c014070 con 0x7fb764106120 2024-02-02T15:38:36.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.728+0000 7fb77d9aab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe29ee47e0 0x55fe29ee6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:36.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.728+0000 7fb77d9aab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe29e4d4c0 con 0x55fe29ee47e0 2024-02-02T15:38:36.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.728+0000 7fb77d9aab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe29e31aa0 con 0x55fe29ed9010 2024-02-02T15:38:36.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.728+0000 7fb77d9aab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe29e2f460 con 0x55fe29ed9930 2024-02-02T15:38:36.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.728+0000 7fb763400700 1 --2- 172.21.15.1:0/1991474431 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fb7440a0740 0x7fb7440a2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:36.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.728+0000 7fb762a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe29ed9010 0x55fe29ed93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:36.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.728+0000 7fb762a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe29ed9010 0x55fe29ed93f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42434/0 (socket says 172.21.15.1:42434) 2024-02-02T15:38:36.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.728+0000 7fb762a00700 1 -- 172.21.15.1:0/1851192533 learned_addr learned my addr 172.21.15.1:0/1851192533 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:38:36.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.728+0000 7fb762000700 1 --2- 172.21.15.1:0/1851192533 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe29ed9930 0x55fe29ee42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:36.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.728+0000 7fb763400700 1 --2- 172.21.15.1:0/1851192533 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe29ee47e0 0x55fe29ee6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:36.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.728+0000 7fb762a00700 1 -- 172.21.15.1:0/1851192533 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe29ed9930 msgr2=0x55fe29ee42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:36.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.728+0000 7fb762a00700 1 --2- 172.21.15.1:0/1851192533 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe29ed9930 0x55fe29ee42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:36.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.728+0000 7fb762a00700 1 -- 172.21.15.1:0/1851192533 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe29ee47e0 msgr2=0x55fe29ee6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:36.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.728+0000 7fb762a00700 1 --2- 172.21.15.1:0/1851192533 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe29ee47e0 0x55fe29ee6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:36.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.728+0000 7fb762a00700 1 -- 172.21.15.1:0/1851192533 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe29cac3b0 con 0x55fe29ed9010 2024-02-02T15:38:36.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.728+0000 7fb762000700 1 --2- 172.21.15.1:0/1851192533 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe29ed9930 0x55fe29ee42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:38:36.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.729+0000 7fb763400700 1 --2- 172.21.15.1:0/1991474431 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fb7440a0740 0x7fb7440a2c00 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb75c013990 tx=0x7fb75c0092b0 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:36.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.729+0000 7fb762a00700 1 --2- 172.21.15.1:0/1851192533 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe29ed9010 0x55fe29ed93f0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fb754014000 tx=0x7fb75400ff90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=295decfa056b1bff server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:36.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.729+0000 7fb753e00700 1 -- 172.21.15.1:0/1851192533 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb75401e040 con 0x55fe29ed9010 2024-02-02T15:38:36.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.729+0000 7fb753e00700 1 -- 172.21.15.1:0/1851192533 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb75400f520 con 0x55fe29ed9010 2024-02-02T15:38:36.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.729+0000 7fb753e00700 1 -- 172.21.15.1:0/1851192533 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb754023bd0 con 0x55fe29ed9010 2024-02-02T15:38:36.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.730+0000 7fb77d9aab80 1 -- 172.21.15.1:0/1851192533 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe29ed9010 msgr2=0x55fe29ed93f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:36.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.730+0000 7fb77d9aab80 1 --2- 172.21.15.1:0/1851192533 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe29ed9010 0x55fe29ed93f0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fb754014000 tx=0x7fb75400ff90 comp rx=0 tx=0).stop 2024-02-02T15:38:36.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.730+0000 7fb77d9aab80 1 -- 172.21.15.1:0/1851192533 shutdown_connections 2024-02-02T15:38:36.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.730+0000 7fb77d9aab80 1 --2- 172.21.15.1:0/1851192533 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe29ee47e0 0x55fe29ee6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:36.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.730+0000 7fb77d9aab80 1 --2- 172.21.15.1:0/1851192533 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe29ed9010 0x55fe29ed93f0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:36.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.730+0000 7fb77d9aab80 1 --2- 172.21.15.1:0/1851192533 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe29ed9930 0x55fe29ee42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:36.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.730+0000 7fb77d9aab80 1 -- 172.21.15.1:0/1851192533 >> 172.21.15.1:0/1851192533 conn(0x55fe29e38db0 msgr2=0x55fe29ed53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:36.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.730+0000 7fb77d9aab80 1 -- 172.21.15.1:0/1851192533 shutdown_connections 2024-02-02T15:38:36.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.730+0000 7fb77d9aab80 1 -- 172.21.15.1:0/1851192533 wait complete. 2024-02-02T15:38:36.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.730+0000 7fb77d9aab80 1 -- start start 2024-02-02T15:38:36.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.731+0000 7fb77d9aab80 1 Processor -- start 2024-02-02T15:38:36.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.731+0000 7fb77d9aab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe29ed9010 0x55fe29ee40e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:36.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.731+0000 7fb77d9aab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe29ed9930 0x55fe29ee1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:36.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.731+0000 7fb77d9aab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe29ee47e0 0x55fe29ee18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:36.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.731+0000 7fb77d9aab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe29e2ad50 con 0x55fe29ed9930 2024-02-02T15:38:36.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.731+0000 7fb77d9aab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe29dc6d80 con 0x55fe29ed9010 2024-02-02T15:38:36.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.731+0000 7fb77d9aab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe29dc6a20 con 0x55fe29ee47e0 2024-02-02T15:38:36.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.731+0000 7fb77d9aab80 10 client.0 _ll_register_callbacks cb 0x55fe29dc8290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:38:36.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.731+0000 7fb762000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe29ed9930 0x55fe29ee1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:36.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.732+0000 7fb762000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe29ed9930 0x55fe29ee1360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36700/0 (socket says 172.21.15.1:36700) 2024-02-02T15:38:36.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.732+0000 7fb762000700 1 -- 172.21.15.1:0/66241585 learned_addr learned my addr 172.21.15.1:0/66241585 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:38:36.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.732+0000 7fb762a00700 1 --2- 172.21.15.1:0/66241585 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe29ed9010 0x55fe29ee40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:36.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.732+0000 7fb763400700 1 --2- 172.21.15.1:0/66241585 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe29ee47e0 0x55fe29ee18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:36.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.732+0000 7fb762000700 1 -- 172.21.15.1:0/66241585 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe29ee47e0 msgr2=0x55fe29ee18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:36.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.732+0000 7fb762000700 1 --2- 172.21.15.1:0/66241585 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe29ee47e0 0x55fe29ee18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:36.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.732+0000 7fb762000700 1 -- 172.21.15.1:0/66241585 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe29ed9010 msgr2=0x55fe29ee40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:36.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.732+0000 7fb762000700 1 --2- 172.21.15.1:0/66241585 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe29ed9010 0x55fe29ee40e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:36.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.732+0000 7fb762000700 1 -- 172.21.15.1:0/66241585 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7641a1ec0 con 0x55fe29ed9930 2024-02-02T15:38:36.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.732+0000 7fb762a00700 1 --2- 172.21.15.1:0/66241585 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe29ed9010 0x55fe29ee40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:38:36.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.732+0000 7fb762000700 1 --2- 172.21.15.1:0/66241585 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe29ed9930 0x55fe29ee1360 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fb74c095960 tx=0x7fb74c098fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f390aff2f1e00137 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:36.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.732+0000 7fb750c00700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:38:36.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.732+0000 7fb750c00700 1 -- 172.21.15.1:0/66241585 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb74c00cc50 con 0x55fe29ed9930 2024-02-02T15:38:36.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.732+0000 7fb750c00700 1 -- 172.21.15.1:0/66241585 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fb74c099b60 con 0x55fe29ed9930 2024-02-02T15:38:36.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.732+0000 7fb77d9aab80 1 -- 172.21.15.1:0/66241585 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fe29ecd0f0 con 0x55fe29ed9930 2024-02-02T15:38:36.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.733+0000 7fb750c00700 1 -- 172.21.15.1:0/66241585 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb74c099460 con 0x55fe29ed9930 2024-02-02T15:38:36.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.733+0000 7fb77d9aab80 10 client.18561 fetch_fsmap learned FSMap version 51 2024-02-02T15:38:36.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.733+0000 7fb77d9aab80 1 -- 172.21.15.1:0/66241585 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x7fb76410e510 con 0x55fe29ed9930 2024-02-02T15:38:36.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.733+0000 7fb750c00700 1 -- 172.21.15.1:0/66241585 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb74c07e080 con 0x55fe29ed9930 2024-02-02T15:38:36.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.733+0000 7fb750c00700 1 -- 172.21.15.1:0/66241585 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 51) v1 ==== 7474+0+0 (secure 0 0 0) 0x7fb74c0b72b0 con 0x55fe29ed9930 2024-02-02T15:38:36.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.733+0000 7fb77d9aab80 10 client.18561 fetch_fsmap finished waiting for FSMap version 51 2024-02-02T15:38:36.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.733+0000 7fb77d9aab80 10 client.18561 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:38:36.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.733+0000 7fb77d9aab80 20 client.18561 populate_metadata read hostname 'smithi001' 2024-02-02T15:38:36.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.733+0000 7fb77d9aab80 1 --2- 172.21.15.1:0/66241585 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55fe29ef4120 0x55fe29ef6540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:36.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.733+0000 7fb77d9aab80 4 client.18561 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:38:36.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.733+0000 7fb77d9aab80 1 -- 172.21.15.1:0/66241585 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fe29e23dd0 con 0x55fe29ef4120 2024-02-02T15:38:36.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.734+0000 7fb762a00700 1 --2- 172.21.15.1:0/66241585 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55fe29ef4120 0x55fe29ef6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:36.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.734+0000 7fb762a00700 1 --2- 172.21.15.1:0/66241585 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55fe29ef4120 0x55fe29ef6540 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=181fa509aa9a2d4a server_cookie=67ceca87d36b9464 in_seq=0 out_seq=0 2024-02-02T15:38:36.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.734+0000 7fb750c00700 10 client.18561 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:38:36.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.735+0000 7fb750c00700 1 -- 172.21.15.1:0/66241585 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55fe29e23dd0 con 0x55fe29ef4120 2024-02-02T15:38:36.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.735+0000 7fb750c00700 10 client.18561 handle_command_reply: tid=0 2024-02-02T15:38:36.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.735+0000 7fb77d9aab80 1 client.18561 shutdown 2024-02-02T15:38:36.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.735+0000 7fb77d9aab80 1 -- 172.21.15.1:0/66241585 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe29ed9930 msgr2=0x55fe29ee1360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:36.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.735+0000 7fb77d9aab80 1 --2- 172.21.15.1:0/66241585 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe29ed9930 0x55fe29ee1360 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fb74c095960 tx=0x7fb74c098fa0 comp rx=0 tx=0).stop 2024-02-02T15:38:36.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.736+0000 7fb77d9aab80 1 -- 172.21.15.1:0/66241585 shutdown_connections 2024-02-02T15:38:36.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.736+0000 7fb77d9aab80 1 --2- 172.21.15.1:0/66241585 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55fe29ef4120 0x55fe29ef6540 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:36.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.736+0000 7fb77d9aab80 1 --2- 172.21.15.1:0/66241585 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe29ee47e0 0x55fe29ee18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:36.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.736+0000 7fb77d9aab80 1 --2- 172.21.15.1:0/66241585 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe29ed9010 0x55fe29ee40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:36.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.736+0000 7fb77d9aab80 1 --2- 172.21.15.1:0/66241585 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe29ed9930 0x55fe29ee1360 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:36.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.736+0000 7fb750c00700 0 client.18561 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:38:36.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.736+0000 7fb77d9aab80 1 -- 172.21.15.1:0/66241585 >> 172.21.15.1:0/66241585 conn(0x55fe29e38db0 msgr2=0x55fe29ed4fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:36.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.736+0000 7fb77d9aab80 1 -- 172.21.15.1:0/66241585 shutdown_connections 2024-02-02T15:38:36.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.736+0000 7fb77d9aab80 1 -- 172.21.15.1:0/66241585 wait complete. 2024-02-02T15:38:36.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.736+0000 7fb77d9aab80 20 client.18561 trim_cache size 0 max 16384 2024-02-02T15:38:36.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.753+0000 7fb77d9aab80 1 Processor -- start 2024-02-02T15:38:36.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.753+0000 7fb77d9aab80 1 -- start start 2024-02-02T15:38:36.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.754+0000 7fb77d9aab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe29ed9010 0x55fe29ee3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:36.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.754+0000 7fb77d9aab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe29ed9930 0x55fe29ee0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:36.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.754+0000 7fb77d9aab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe29ee47e0 0x55fe29ee1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:36.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.754+0000 7fb77d9aab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe29cae6a0 con 0x55fe29ee47e0 2024-02-02T15:38:36.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.754+0000 7fb77d9aab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe29edcfb0 con 0x55fe29ed9010 2024-02-02T15:38:36.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.754+0000 7fb77d9aab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe29ee2720 con 0x55fe29ed9930 2024-02-02T15:38:36.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.754+0000 7fb762a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe29ed9010 0x55fe29ee3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:36.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.754+0000 7fb763400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe29ee47e0 0x55fe29ee1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:36.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.754+0000 7fb762a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe29ed9010 0x55fe29ee3c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42454/0 (socket says 172.21.15.1:42454) 2024-02-02T15:38:36.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.754+0000 7fb762a00700 1 -- 172.21.15.1:0/2860843903 learned_addr learned my addr 172.21.15.1:0/2860843903 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:38:36.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.754+0000 7fb762000700 1 --2- 172.21.15.1:0/2860843903 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe29ed9930 0x55fe29ee0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:36.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.754+0000 7fb762a00700 1 -- 172.21.15.1:0/2860843903 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe29ed9930 msgr2=0x55fe29ee0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:36.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.754+0000 7fb762a00700 1 --2- 172.21.15.1:0/2860843903 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe29ed9930 0x55fe29ee0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:36.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.754+0000 7fb762a00700 1 -- 172.21.15.1:0/2860843903 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe29ee47e0 msgr2=0x55fe29ee1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:36.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.754+0000 7fb762a00700 1 --2- 172.21.15.1:0/2860843903 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe29ee47e0 0x55fe29ee1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:36.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.754+0000 7fb762a00700 1 -- 172.21.15.1:0/2860843903 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe29c58b60 con 0x55fe29ed9010 2024-02-02T15:38:36.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.754+0000 7fb762000700 1 --2- 172.21.15.1:0/2860843903 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe29ed9930 0x55fe29ee0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:38:36.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.755+0000 7fb762a00700 1 --2- 172.21.15.1:0/2860843903 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe29ed9010 0x55fe29ee3c80 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fb754006010 tx=0x7fb754023770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54cb1802efffd771 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:36.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.755+0000 7fb759200700 1 -- 172.21.15.1:0/2860843903 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb75401e040 con 0x55fe29ed9010 2024-02-02T15:38:36.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.755+0000 7fb759200700 1 -- 172.21.15.1:0/2860843903 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb754005500 con 0x55fe29ed9010 2024-02-02T15:38:36.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.755+0000 7fb759200700 1 -- 172.21.15.1:0/2860843903 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb754023ad0 con 0x55fe29ed9010 2024-02-02T15:38:36.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.756+0000 7fb77d9aab80 1 -- 172.21.15.1:0/2860843903 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe29ed9010 msgr2=0x55fe29ee3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:36.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.756+0000 7fb77d9aab80 1 --2- 172.21.15.1:0/2860843903 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe29ed9010 0x55fe29ee3c80 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fb754006010 tx=0x7fb754023770 comp rx=0 tx=0).stop 2024-02-02T15:38:36.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.756+0000 7fb77d9aab80 1 -- 172.21.15.1:0/2860843903 shutdown_connections 2024-02-02T15:38:36.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.756+0000 7fb77d9aab80 1 --2- 172.21.15.1:0/2860843903 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe29ee47e0 0x55fe29ee1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:36.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.756+0000 7fb77d9aab80 1 --2- 172.21.15.1:0/2860843903 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe29ed9010 0x55fe29ee3c80 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:36.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.756+0000 7fb77d9aab80 1 --2- 172.21.15.1:0/2860843903 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe29ed9930 0x55fe29ee0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:36.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.756+0000 7fb77d9aab80 1 -- 172.21.15.1:0/2860843903 >> 172.21.15.1:0/2860843903 conn(0x55fe29e38db0 msgr2=0x55fe29eef8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:36.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.756+0000 7fb77d9aab80 1 -- 172.21.15.1:0/2860843903 shutdown_connections 2024-02-02T15:38:36.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.756+0000 7fb77d9aab80 1 -- 172.21.15.1:0/2860843903 wait complete. 2024-02-02T15:38:36.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.756+0000 7fb77d9aab80 1 -- start start 2024-02-02T15:38:36.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.757+0000 7fb77d9aab80 1 Processor -- start 2024-02-02T15:38:36.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.757+0000 7fb77d9aab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe29ed9010 0x55fe29ee8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:36.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.757+0000 7fb77d9aab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe29ed9930 0x55fe29ee9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:36.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.757+0000 7fb77d9aab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe29ee47e0 0x55fe29f44c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:36.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.757+0000 7fb77d9aab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe29ee9a70 con 0x55fe29ed9010 2024-02-02T15:38:36.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.757+0000 7fb77d9aab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe29f45180 con 0x55fe29ee47e0 2024-02-02T15:38:36.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.757+0000 7fb77d9aab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe29f45310 con 0x55fe29ed9930 2024-02-02T15:38:36.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.757+0000 7fb77d9aab80 10 client.0 _ll_register_callbacks cb 0x55fe29dc8290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:38:36.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.757+0000 7fb762a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe29ed9010 0x55fe29ee8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:36.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.757+0000 7fb763400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe29ee47e0 0x55fe29f44c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:36.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.757+0000 7fb762a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe29ed9010 0x55fe29ee8e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36712/0 (socket says 172.21.15.1:36712) 2024-02-02T15:38:36.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.757+0000 7fb762a00700 1 -- 172.21.15.1:0/1641435228 learned_addr learned my addr 172.21.15.1:0/1641435228 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:38:36.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.757+0000 7fb762000700 1 --2- 172.21.15.1:0/1641435228 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe29ed9930 0x55fe29ee9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:36.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.758+0000 7fb763400700 1 -- 172.21.15.1:0/1641435228 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe29ed9930 msgr2=0x55fe29ee9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:36.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.758+0000 7fb763400700 1 --2- 172.21.15.1:0/1641435228 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe29ed9930 0x55fe29ee9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:36.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.758+0000 7fb763400700 1 -- 172.21.15.1:0/1641435228 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe29ed9010 msgr2=0x55fe29ee8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:36.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.758+0000 7fb763400700 1 --2- 172.21.15.1:0/1641435228 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe29ed9010 0x55fe29ee8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:36.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.758+0000 7fb763400700 1 -- 172.21.15.1:0/1641435228 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fe29cac3b0 con 0x55fe29ee47e0 2024-02-02T15:38:36.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.758+0000 7fb763400700 1 --2- 172.21.15.1:0/1641435228 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe29ee47e0 0x55fe29f44c40 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fb75c004320 tx=0x7fb75c015040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ae31e175e0e0cb5 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:36.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.758+0000 7fb750c00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:38:36.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.758+0000 7fb750c00700 1 -- 172.21.15.1:0/1641435228 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb75c007b70 con 0x55fe29ee47e0 2024-02-02T15:38:36.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.758+0000 7fb77d9aab80 1 -- 172.21.15.1:0/1641435228 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb75400f520 con 0x55fe29ee47e0 2024-02-02T15:38:36.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.758+0000 7fb750c00700 1 -- 172.21.15.1:0/1641435228 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fb75c007cd0 con 0x55fe29ee47e0 2024-02-02T15:38:36.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.758+0000 7fb750c00700 1 -- 172.21.15.1:0/1641435228 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb75c007e90 con 0x55fe29ee47e0 2024-02-02T15:38:36.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.759+0000 7fb77d9aab80 10 client.28658 fetch_fsmap learned FSMap version 51 2024-02-02T15:38:36.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.759+0000 7fb77d9aab80 1 -- 172.21.15.1:0/1641435228 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x55fe29df0ea0 con 0x55fe29ee47e0 2024-02-02T15:38:36.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.759+0000 7fb750c00700 1 -- 172.21.15.1:0/1641435228 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb75400f520 con 0x55fe29ee47e0 2024-02-02T15:38:36.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.759+0000 7fb750c00700 1 -- 172.21.15.1:0/1641435228 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 51) v1 ==== 7474+0+0 (secure 0 0 0) 0x7fb75c0244d0 con 0x55fe29ee47e0 2024-02-02T15:38:36.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.759+0000 7fb77d9aab80 10 client.28658 fetch_fsmap finished waiting for FSMap version 51 2024-02-02T15:38:36.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.759+0000 7fb77d9aab80 10 client.28658 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:38:36.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.759+0000 7fb77d9aab80 20 client.28658 populate_metadata read hostname 'smithi001' 2024-02-02T15:38:36.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.759+0000 7fb77d9aab80 1 --2- 172.21.15.1:0/1641435228 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55fe29f46af0 0x55fe29f48f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:36.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.759+0000 7fb77d9aab80 4 client.28658 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2024-02-02T15:38:36.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.759+0000 7fb77d9aab80 1 -- 172.21.15.1:0/1641435228 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55fe29de9f50 con 0x55fe29f46af0 2024-02-02T15:38:36.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.759+0000 7fb762a00700 1 --2- 172.21.15.1:0/1641435228 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55fe29f46af0 0x55fe29f48f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:36.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.760+0000 7fb762a00700 1 --2- 172.21.15.1:0/1641435228 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55fe29f46af0 0x55fe29f48f10 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fcdfd999a528563 server_cookie=457474b32b62a295 in_seq=0 out_seq=0 2024-02-02T15:38:36.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.760+0000 7fb750c00700 10 client.28658 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:38:36.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.760+0000 7fb750c00700 1 -- 172.21.15.1:0/1641435228 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55fe29de9f50 con 0x55fe29f46af0 2024-02-02T15:38:36.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.760+0000 7fb750c00700 10 client.28658 handle_command_reply: tid=0 2024-02-02T15:38:36.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.760+0000 7fb77d9aab80 1 client.28658 shutdown 2024-02-02T15:38:36.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.760+0000 7fb77d9aab80 1 -- 172.21.15.1:0/1641435228 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe29ee47e0 msgr2=0x55fe29f44c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:36.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.760+0000 7fb77d9aab80 1 --2- 172.21.15.1:0/1641435228 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe29ee47e0 0x55fe29f44c40 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fb75c004320 tx=0x7fb75c015040 comp rx=0 tx=0).stop 2024-02-02T15:38:36.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.760+0000 7fb77d9aab80 1 -- 172.21.15.1:0/1641435228 shutdown_connections 2024-02-02T15:38:36.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.760+0000 7fb77d9aab80 1 --2- 172.21.15.1:0/1641435228 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55fe29f46af0 0x55fe29f48f10 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:36.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.760+0000 7fb77d9aab80 1 --2- 172.21.15.1:0/1641435228 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe29ee47e0 0x55fe29f44c40 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:36.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.760+0000 7fb77d9aab80 1 --2- 172.21.15.1:0/1641435228 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe29ed9010 0x55fe29ee8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:36.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.760+0000 7fb750c00700 0 client.28658 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:38:36.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.760+0000 7fb77d9aab80 1 --2- 172.21.15.1:0/1641435228 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe29ed9930 0x55fe29ee9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:36.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.760+0000 7fb77d9aab80 1 -- 172.21.15.1:0/1641435228 >> 172.21.15.1:0/1641435228 conn(0x55fe29e38db0 msgr2=0x55fe29ed45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:36.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.760+0000 7fb77d9aab80 1 -- 172.21.15.1:0/1641435228 shutdown_connections 2024-02-02T15:38:36.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.761+0000 7fb77d9aab80 1 -- 172.21.15.1:0/1641435228 wait complete. 2024-02-02T15:38:36.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.761+0000 7fb77d9aab80 20 client.28658 trim_cache size 0 max 16384 2024-02-02T15:38:36.763 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:38:36.763 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2024-02-02T15:38:36.763 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2024-02-02T15:38:36.764 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:38:36.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.761+0000 7fb769e00700 1 -- 172.21.15.1:0/1991474431 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fb7440a0740 msgr2=0x7fb7440a2c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:36.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.761+0000 7fb769e00700 1 --2- 172.21.15.1:0/1991474431 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fb7440a0740 0x7fb7440a2c00 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb75c013990 tx=0x7fb75c0092b0 comp rx=0 tx=0).stop 2024-02-02T15:38:36.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.761+0000 7fb769e00700 1 -- 172.21.15.1:0/1991474431 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb764106120 msgr2=0x7fb764195220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:36.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.761+0000 7fb769e00700 1 --2- 172.21.15.1:0/1991474431 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb764106120 0x7fb764195220 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fb74c00e970 tx=0x7fb74c00ec80 comp rx=0 tx=0).stop 2024-02-02T15:38:36.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.761+0000 7fb769e00700 1 -- 172.21.15.1:0/1991474431 shutdown_connections 2024-02-02T15:38:36.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.761+0000 7fb769e00700 1 --2- 172.21.15.1:0/1991474431 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fb7440a0740 0x7fb7440a2c00 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:36.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.761+0000 7fb769e00700 1 --2- 172.21.15.1:0/1991474431 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fb7641985d0 0x7fb764198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:36.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.761+0000 7fb769e00700 1 --2- 172.21.15.1:0/1991474431 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb764101000 0x7fb764194ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:36.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.761+0000 7fb769e00700 1 --2- 172.21.15.1:0/1991474431 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fb764106120 0x7fb764195220 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:36.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.761+0000 7fb769e00700 1 -- 172.21.15.1:0/1991474431 >> 172.21.15.1:0/1991474431 conn(0x7fb7640fc860 msgr2=0x7fb764104a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:36.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.762+0000 7fb769e00700 1 -- 172.21.15.1:0/1991474431 shutdown_connections 2024-02-02T15:38:36.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.762+0000 7fb769e00700 1 -- 172.21.15.1:0/1991474431 wait complete. 2024-02-02T15:38:36.779 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-02T15:38:36.780 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T15:38:36.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.991+0000 7ff554000700 1 -- 172.21.15.1:0/2261920366 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff54c100f10 msgr2=0x7ff54c1012f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:36.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.991+0000 7ff554000700 1 --2- 172.21.15.1:0/2261920366 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff54c100f10 0x7ff54c1012f0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7ff53c009990 tx=0x7ff53c009ca0 comp rx=0 tx=0).stop 2024-02-02T15:38:36.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.991+0000 7ff554000700 1 -- 172.21.15.1:0/2261920366 shutdown_connections 2024-02-02T15:38:36.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.991+0000 7ff554000700 1 --2- 172.21.15.1:0/2261920366 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff54c106030 0x7ff54c10dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:36.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.991+0000 7ff554000700 1 --2- 172.21.15.1:0/2261920366 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff54c100f10 0x7ff54c1012f0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:36.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.991+0000 7ff554000700 1 --2- 172.21.15.1:0/2261920366 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff54c1018c0 0x7ff54c105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:36.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.991+0000 7ff554000700 1 -- 172.21.15.1:0/2261920366 >> 172.21.15.1:0/2261920366 conn(0x7ff54c0fc790 msgr2=0x7ff54c0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:36.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.991+0000 7ff554000700 1 -- 172.21.15.1:0/2261920366 shutdown_connections 2024-02-02T15:38:36.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.991+0000 7ff554000700 1 -- 172.21.15.1:0/2261920366 wait complete. 2024-02-02T15:38:36.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.992+0000 7ff554000700 1 Processor -- start 2024-02-02T15:38:36.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.992+0000 7ff554000700 1 -- start start 2024-02-02T15:38:36.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.993+0000 7ff554000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff54c100f10 0x7ff54c199010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:36.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.993+0000 7ff554000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff54c106030 0x7ff54c199550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:36.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.993+0000 7ff554000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff54c19d9b0 0x7ff54c19de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:36.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.993+0000 7ff554000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54c19e430 con 0x7ff54c19d9b0 2024-02-02T15:38:36.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.993+0000 7ff554000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54c19e5a0 con 0x7ff54c106030 2024-02-02T15:38:36.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.993+0000 7ff554000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54c19e760 con 0x7ff54c100f10 2024-02-02T15:38:36.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.993+0000 7ff552000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff54c19d9b0 0x7ff54c19de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:36.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.993+0000 7ff551600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff54c100f10 0x7ff54c199010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:36.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.993+0000 7ff550c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff54c106030 0x7ff54c199550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:36.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.993+0000 7ff551600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff54c100f10 0x7ff54c199010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:53534/0 (socket says 172.21.15.1:53534) 2024-02-02T15:38:36.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.993+0000 7ff550c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff54c106030 0x7ff54c199550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42474/0 (socket says 172.21.15.1:42474) 2024-02-02T15:38:36.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.993+0000 7ff551600700 1 -- 172.21.15.1:0/3000690820 learned_addr learned my addr 172.21.15.1:0/3000690820 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:38:36.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.993+0000 7ff552000700 1 -- 172.21.15.1:0/3000690820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff54c100f10 msgr2=0x7ff54c199010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:36.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.993+0000 7ff552000700 1 --2- 172.21.15.1:0/3000690820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff54c100f10 0x7ff54c199010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:36.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.993+0000 7ff552000700 1 -- 172.21.15.1:0/3000690820 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff54c106030 msgr2=0x7ff54c199550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:36.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.993+0000 7ff552000700 1 --2- 172.21.15.1:0/3000690820 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff54c106030 0x7ff54c199550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:36.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.993+0000 7ff552000700 1 -- 172.21.15.1:0/3000690820 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff53c0095f0 con 0x7ff54c19d9b0 2024-02-02T15:38:36.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.994+0000 7ff552000700 1 --2- 172.21.15.1:0/3000690820 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff54c19d9b0 0x7ff54c19de30 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7ff54400efc0 tx=0x7ff54400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:36.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.994+0000 7ff54a000700 1 -- 172.21.15.1:0/3000690820 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff544018070 con 0x7ff54c19d9b0 2024-02-02T15:38:36.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.994+0000 7ff54a000700 1 -- 172.21.15.1:0/3000690820 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff54400f040 con 0x7ff54c19d9b0 2024-02-02T15:38:36.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.994+0000 7ff54a000700 1 -- 172.21.15.1:0/3000690820 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff54400cd40 con 0x7ff54c19d9b0 2024-02-02T15:38:36.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.994+0000 7ff554000700 1 -- 172.21.15.1:0/3000690820 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff54c1a5c60 con 0x7ff54c19d9b0 2024-02-02T15:38:36.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.994+0000 7ff554000700 1 -- 172.21.15.1:0/3000690820 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff54c1a60f0 con 0x7ff54c19d9b0 2024-02-02T15:38:36.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.995+0000 7ff554000700 1 -- 172.21.15.1:0/3000690820 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff54c067bc0 con 0x7ff54c19d9b0 2024-02-02T15:38:36.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.997+0000 7ff54a000700 1 -- 172.21.15.1:0/3000690820 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7ff544003a00 con 0x7ff54c19d9b0 2024-02-02T15:38:36.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.998+0000 7ff54a000700 1 --2- 172.21.15.1:0/3000690820 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff52c0a0710 0x7ff52c0a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:36.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.998+0000 7ff54a000700 1 -- 172.21.15.1:0/3000690820 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9453+0+0 (secure 0 0 0) 0x7ff5440cbbd0 con 0x7ff54c19d9b0 2024-02-02T15:38:36.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.998+0000 7ff551600700 1 --2- 172.21.15.1:0/3000690820 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff52c0a0710 0x7ff52c0a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:36.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:36.999+0000 7ff551600700 1 --2- 172.21.15.1:0/3000690820 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff52c0a0710 0x7ff52c0a2bd0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff53c006010 tx=0x7ff53c00b260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:37.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.003+0000 7ff54a000700 1 -- 172.21.15.1:0/3000690820 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7ff5440950c0 con 0x7ff54c19d9b0 2024-02-02T15:38:37.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.260+0000 7ff554000700 1 -- 172.21.15.1:0/3000690820 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff54c19a360 con 0x7ff54c19d9b0 2024-02-02T15:38:37.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.261+0000 7ff54a000700 1 -- 172.21.15.1:0/3000690820 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+10867 (secure 0 0 0) 0x7ff54401b320 con 0x7ff54c19d9b0 2024-02-02T15:38:37.260 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:38:37.260 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":51,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2922240448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2922240448},{"type":"v1","addr":"172.21.15.1:6835","nonce":2922240448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2729497431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2729497431},{"type":"v1","addr":"172.21.15.1:6837","nonce":2729497431}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":51,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-02T15:24:47.761435+0000","modified":"2024-02-02T15:38:05.435016+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24529,"mds_2":24518,"mds_3":24514,"mds_4":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14517":{"gid":14517,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/1218303683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":1218303683},{"type":"v1","addr":"172.21.15.1:6839","nonce":1218303683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14529":{"gid":14529,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/2059197496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2059197496},{"type":"v1","addr":"172.21.15.1:6841","nonce":2059197496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/7648986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":7648986},{"type":"v1","addr":"172.21.15.142:6833","nonce":7648986}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/3482746094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3482746094},{"type":"v1","addr":"172.21.15.71:6833","nonce":3482746094}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/388207814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":388207814},{"type":"v1","addr":"172.21.15.71:6835","nonce":388207814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6835/1052606547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1052606547},{"type":"v1","addr":"172.21.15.142:6835","nonce":1052606547}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.142:6837/2870289019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2870289019},{"type":"v1","addr":"172.21.15.142:6837","nonce":2870289019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.71:6837/628016199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":628016199},{"type":"v1","addr":"172.21.15.71:6837","nonce":628016199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.142:6839/321675421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":321675421},{"type":"v1","addr":"172.21.15.142:6839","nonce":321675421}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/1268086493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1268086493},{"type":"v1","addr":"172.21.15.71:6839","nonce":1268086493}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-02T15:38:37.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.263+0000 7ff554000700 1 -- 172.21.15.1:0/3000690820 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff52c0a0710 msgr2=0x7ff52c0a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:37.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.263+0000 7ff554000700 1 --2- 172.21.15.1:0/3000690820 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff52c0a0710 0x7ff52c0a2bd0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff53c006010 tx=0x7ff53c00b260 comp rx=0 tx=0).stop 2024-02-02T15:38:37.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.263+0000 7ff554000700 1 -- 172.21.15.1:0/3000690820 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff54c19d9b0 msgr2=0x7ff54c19de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:37.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.263+0000 7ff554000700 1 --2- 172.21.15.1:0/3000690820 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff54c19d9b0 0x7ff54c19de30 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7ff54400efc0 tx=0x7ff54400c3a0 comp rx=0 tx=0).stop 2024-02-02T15:38:37.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.263+0000 7ff554000700 1 -- 172.21.15.1:0/3000690820 shutdown_connections 2024-02-02T15:38:37.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.263+0000 7ff554000700 1 --2- 172.21.15.1:0/3000690820 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff52c0a0710 0x7ff52c0a2bd0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:37.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.263+0000 7ff554000700 1 --2- 172.21.15.1:0/3000690820 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff54c19d9b0 0x7ff54c19de30 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:37.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.263+0000 7ff554000700 1 --2- 172.21.15.1:0/3000690820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff54c100f10 0x7ff54c199010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:37.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.263+0000 7ff554000700 1 --2- 172.21.15.1:0/3000690820 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff54c106030 0x7ff54c199550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:37.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.263+0000 7ff554000700 1 -- 172.21.15.1:0/3000690820 >> 172.21.15.1:0/3000690820 conn(0x7ff54c0fc790 msgr2=0x7ff54c0fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:37.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.263+0000 7ff554000700 1 -- 172.21.15.1:0/3000690820 shutdown_connections 2024-02-02T15:38:37.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.264+0000 7ff554000700 1 -- 172.21.15.1:0/3000690820 wait complete. 2024-02-02T15:38:37.263 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 51 2024-02-02T15:38:37.267 INFO:tasks.workunit.client.0.smithi001.stdout: 1 640948 5.53 MB/sec execute 491 sec latency 3884.600 ms 2024-02-02T15:38:37.281 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-02-02T15:38:37.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.489+0000 7f9a19400700 1 -- 172.21.15.1:0/4247931947 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a14111d90 msgr2=0x7f9a14114180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:37.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.489+0000 7f9a19400700 1 --2- 172.21.15.1:0/4247931947 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a14111d90 0x7f9a14114180 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c00b0a0 tx=0x7f9a0c00b3b0 comp rx=0 tx=0).stop 2024-02-02T15:38:37.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.490+0000 7f9a19400700 1 -- 172.21.15.1:0/4247931947 shutdown_connections 2024-02-02T15:38:37.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.490+0000 7f9a19400700 1 --2- 172.21.15.1:0/4247931947 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a14111d90 0x7f9a14114180 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:37.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.490+0000 7f9a19400700 1 --2- 172.21.15.1:0/4247931947 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a140730e0 0x7f9a140734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:37.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.490+0000 7f9a19400700 1 --2- 172.21.15.1:0/4247931947 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9a14073a00 0x7f9a14111850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:37.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.490+0000 7f9a19400700 1 -- 172.21.15.1:0/4247931947 >> 172.21.15.1:0/4247931947 conn(0x7f9a140fc810 msgr2=0x7f9a140fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:37.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.490+0000 7f9a19400700 1 -- 172.21.15.1:0/4247931947 shutdown_connections 2024-02-02T15:38:37.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.490+0000 7f9a19400700 1 -- 172.21.15.1:0/4247931947 wait complete. 2024-02-02T15:38:37.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.491+0000 7f9a19400700 1 Processor -- start 2024-02-02T15:38:37.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.491+0000 7f9a19400700 1 -- start start 2024-02-02T15:38:37.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.491+0000 7f9a19400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a140730e0 0x7f9a1419d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:37.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.491+0000 7f9a19400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a14073a00 0x7f9a1419da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:37.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.491+0000 7f9a19400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9a14111d90 0x7f9a141a1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:37.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.491+0000 7f9a19400700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a141a2530 con 0x7f9a140730e0 2024-02-02T15:38:37.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.491+0000 7f9a19400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a141a26a0 con 0x7f9a14111d90 2024-02-02T15:38:37.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.491+0000 7f9a19400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a141a2860 con 0x7f9a14073a00 2024-02-02T15:38:37.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.492+0000 7f9a12800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a140730e0 0x7f9a1419d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:37.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.492+0000 7f9a12800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a140730e0 0x7f9a1419d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36760/0 (socket says 172.21.15.1:36760) 2024-02-02T15:38:37.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.492+0000 7f9a12800700 1 -- 172.21.15.1:0/828552148 learned_addr learned my addr 172.21.15.1:0/828552148 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:38:37.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.492+0000 7f9a11e00700 1 --2- 172.21.15.1:0/828552148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a14073a00 0x7f9a1419da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:37.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.492+0000 7f9a13200700 1 --2- 172.21.15.1:0/828552148 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9a14111d90 0x7f9a141a1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:37.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.492+0000 7f9a12800700 1 -- 172.21.15.1:0/828552148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a14073a00 msgr2=0x7f9a1419da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:37.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.492+0000 7f9a12800700 1 --2- 172.21.15.1:0/828552148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a14073a00 0x7f9a1419da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:37.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.492+0000 7f9a12800700 1 -- 172.21.15.1:0/828552148 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9a14111d90 msgr2=0x7f9a141a1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:37.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.492+0000 7f9a12800700 1 --2- 172.21.15.1:0/828552148 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9a14111d90 0x7f9a141a1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:37.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.492+0000 7f9a12800700 1 -- 172.21.15.1:0/828552148 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a0c009d00 con 0x7f9a140730e0 2024-02-02T15:38:37.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.492+0000 7f9a12800700 1 --2- 172.21.15.1:0/828552148 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a140730e0 0x7f9a1419d550 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f9a0400e9a0 tx=0x7f9a0400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:37.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.492+0000 7f9a0b400700 1 -- 172.21.15.1:0/828552148 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a04009790 con 0x7f9a140730e0 2024-02-02T15:38:37.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.492+0000 7f9a0b400700 1 -- 172.21.15.1:0/828552148 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a0400cbd0 con 0x7f9a140730e0 2024-02-02T15:38:37.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.492+0000 7f9a0b400700 1 -- 172.21.15.1:0/828552148 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a04018680 con 0x7f9a140730e0 2024-02-02T15:38:37.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.492+0000 7f9a19400700 1 -- 172.21.15.1:0/828552148 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a141a2b40 con 0x7f9a140730e0 2024-02-02T15:38:37.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.493+0000 7f9a19400700 1 -- 172.21.15.1:0/828552148 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a141aa380 con 0x7f9a140730e0 2024-02-02T15:38:37.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.496+0000 7f9a0b400700 1 -- 172.21.15.1:0/828552148 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f9a0400cd40 con 0x7f9a140730e0 2024-02-02T15:38:37.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.497+0000 7f9a2d014b80 1 Processor -- start 2024-02-02T15:38:37.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.497+0000 7f9a0b400700 1 --2- 172.21.15.1:0/828552148 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f99f40a0710 0x7f99f40a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:37.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.497+0000 7f9a2d014b80 1 -- start start 2024-02-02T15:38:37.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.497+0000 7f9a0b400700 1 -- 172.21.15.1:0/828552148 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f9a04014070 con 0x7f9a140730e0 2024-02-02T15:38:37.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.497+0000 7f9a13200700 1 --2- 172.21.15.1:0/828552148 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f99f40a0710 0x7f99f40a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:37.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.497+0000 7f9a2d014b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555bff874010 0x555bff8743f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:37.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.497+0000 7f9a2d014b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555bff874930 0x555bff87f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:37.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.497+0000 7f9a2d014b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555bff87f7e0 0x555bff881bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:37.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.497+0000 7f9a2d014b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bff7e84c0 con 0x555bff874930 2024-02-02T15:38:37.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.497+0000 7f9a2d014b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bff7ccaa0 con 0x555bff874010 2024-02-02T15:38:37.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.497+0000 7f9a2d014b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bff7ca460 con 0x555bff87f7e0 2024-02-02T15:38:37.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.498+0000 7f9a11e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555bff874930 0x555bff87f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:37.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.498+0000 7f9a12800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555bff874010 0x555bff8743f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:37.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.498+0000 7f9a11e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555bff874930 0x555bff87f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36772/0 (socket says 172.21.15.1:36772) 2024-02-02T15:38:37.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.498+0000 7f9a11e00700 1 -- 172.21.15.1:0/4273994825 learned_addr learned my addr 172.21.15.1:0/4273994825 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:38:37.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.498+0000 7f9a13200700 1 --2- 172.21.15.1:0/4273994825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555bff87f7e0 0x555bff881bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:37.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.498+0000 7f9a12800700 1 -- 172.21.15.1:0/4273994825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555bff87f7e0 msgr2=0x555bff881bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:37.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.498+0000 7f9a12800700 1 --2- 172.21.15.1:0/4273994825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555bff87f7e0 0x555bff881bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:37.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.498+0000 7f9a12800700 1 -- 172.21.15.1:0/4273994825 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555bff874930 msgr2=0x555bff87f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:37.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.498+0000 7f9a12800700 1 --2- 172.21.15.1:0/4273994825 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555bff874930 0x555bff87f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:37.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.498+0000 7f9a12800700 1 -- 172.21.15.1:0/4273994825 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555bff6473b0 con 0x555bff874010 2024-02-02T15:38:37.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.498+0000 7f9a13200700 1 --2- 172.21.15.1:0/828552148 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f99f40a0710 0x7f99f40a2bd0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c009fd0 tx=0x7f9a0c009480 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:37.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.498+0000 7f9a12800700 1 --2- 172.21.15.1:0/4273994825 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555bff874010 0x555bff8743f0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f9a040c97a0 tx=0x7f9a04098fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69888c92e1698ffd server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:37.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.499+0000 7f9a03400700 1 -- 172.21.15.1:0/4273994825 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0409d3d0 con 0x555bff874010 2024-02-02T15:38:37.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.499+0000 7f9a03400700 1 -- 172.21.15.1:0/4273994825 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a040a1610 con 0x555bff874010 2024-02-02T15:38:37.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.499+0000 7f9a03400700 1 -- 172.21.15.1:0/4273994825 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a04099a40 con 0x555bff874010 2024-02-02T15:38:37.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.499+0000 7f9a2d014b80 1 -- 172.21.15.1:0/4273994825 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555bff874010 msgr2=0x555bff8743f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:37.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.499+0000 7f9a2d014b80 1 --2- 172.21.15.1:0/4273994825 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555bff874010 0x555bff8743f0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f9a040c97a0 tx=0x7f9a04098fd0 comp rx=0 tx=0).stop 2024-02-02T15:38:37.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.499+0000 7f9a2d014b80 1 -- 172.21.15.1:0/4273994825 shutdown_connections 2024-02-02T15:38:37.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.499+0000 7f9a2d014b80 1 --2- 172.21.15.1:0/4273994825 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555bff87f7e0 0x555bff881bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:37.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.499+0000 7f9a2d014b80 1 --2- 172.21.15.1:0/4273994825 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555bff874010 0x555bff8743f0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:37.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.499+0000 7f9a2d014b80 1 --2- 172.21.15.1:0/4273994825 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555bff874930 0x555bff87f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:37.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.499+0000 7f9a2d014b80 1 -- 172.21.15.1:0/4273994825 >> 172.21.15.1:0/4273994825 conn(0x555bff7d3db0 msgr2=0x555bff8703b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:37.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.499+0000 7f9a2d014b80 1 -- 172.21.15.1:0/4273994825 shutdown_connections 2024-02-02T15:38:37.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.499+0000 7f9a2d014b80 1 -- 172.21.15.1:0/4273994825 wait complete. 2024-02-02T15:38:37.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.500+0000 7f9a2d014b80 1 -- start start 2024-02-02T15:38:37.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.500+0000 7f9a2d014b80 1 Processor -- start 2024-02-02T15:38:37.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.501+0000 7f9a2d014b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555bff874010 0x555bff87f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:37.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.501+0000 7f9a2d014b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555bff874930 0x555bff87c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:37.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.501+0000 7f9a2d014b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555bff87f7e0 0x555bff87cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:37.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.501+0000 7f9a2d014b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bff7c5d50 con 0x555bff874010 2024-02-02T15:38:37.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.501+0000 7f9a2d014b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bff761d80 con 0x555bff87f7e0 2024-02-02T15:38:37.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.501+0000 7f9a2d014b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bff761a20 con 0x555bff874930 2024-02-02T15:38:37.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.501+0000 7f9a2d014b80 10 client.0 _ll_register_callbacks cb 0x555bff763290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:38:37.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.501+0000 7f9a12800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555bff874010 0x555bff87f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:37.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.501+0000 7f9a13200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555bff87f7e0 0x555bff87cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:37.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.501+0000 7f9a12800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555bff874010 0x555bff87f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36776/0 (socket says 172.21.15.1:36776) 2024-02-02T15:38:37.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.501+0000 7f9a12800700 1 -- 172.21.15.1:0/1534043144 learned_addr learned my addr 172.21.15.1:0/1534043144 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:38:37.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.501+0000 7f9a11e00700 1 --2- 172.21.15.1:0/1534043144 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555bff874930 0x555bff87c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:37.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.501+0000 7f9a13200700 1 -- 172.21.15.1:0/1534043144 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555bff874930 msgr2=0x555bff87c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:37.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.501+0000 7f9a13200700 1 --2- 172.21.15.1:0/1534043144 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555bff874930 0x555bff87c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:37.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.501+0000 7f9a13200700 1 -- 172.21.15.1:0/1534043144 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555bff874010 msgr2=0x555bff87f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:37.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.501+0000 7f9a13200700 1 --2- 172.21.15.1:0/1534043144 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555bff874010 0x555bff87f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:37.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.501+0000 7f9a13200700 1 -- 172.21.15.1:0/1534043144 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a141aa380 con 0x555bff87f7e0 2024-02-02T15:38:37.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.501+0000 7f9a12800700 1 --2- 172.21.15.1:0/1534043144 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555bff874010 0x555bff87f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:38:37.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.502+0000 7f9a13200700 1 --2- 172.21.15.1:0/1534043144 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555bff87f7e0 0x555bff87cbd0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c00b070 tx=0x7f9a0c00b3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68fc5b8cfbcd51b1 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:37.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.502+0000 7f99fbe00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:38:37.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.502+0000 7f99fbe00700 1 -- 172.21.15.1:0/1534043144 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c01d860 con 0x555bff87f7e0 2024-02-02T15:38:37.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.502+0000 7f9a2d014b80 1 -- 172.21.15.1:0/1534043144 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555bff8680f0 con 0x555bff87f7e0 2024-02-02T15:38:37.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.502+0000 7f99fbe00700 1 -- 172.21.15.1:0/1534043144 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f9a0c007940 con 0x555bff87f7e0 2024-02-02T15:38:37.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.502+0000 7f99fbe00700 1 -- 172.21.15.1:0/1534043144 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a0c01d9c0 con 0x555bff87f7e0 2024-02-02T15:38:37.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.502+0000 7f9a2d014b80 10 client.28676 fetch_fsmap learned FSMap version 51 2024-02-02T15:38:37.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.502+0000 7f9a2d014b80 1 -- 172.21.15.1:0/1534043144 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x555bff5f3b60 con 0x555bff87f7e0 2024-02-02T15:38:37.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.503+0000 7f99fbe00700 1 -- 172.21.15.1:0/1534043144 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555bff8680f0 con 0x555bff87f7e0 2024-02-02T15:38:37.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.503+0000 7f99fbe00700 1 -- 172.21.15.1:0/1534043144 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 51) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f9a0c026030 con 0x555bff87f7e0 2024-02-02T15:38:37.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.503+0000 7f9a2d014b80 10 client.28676 fetch_fsmap finished waiting for FSMap version 51 2024-02-02T15:38:37.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.503+0000 7f9a2d014b80 10 client.28676 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-02-02T15:38:37.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.503+0000 7f9a2d014b80 20 client.28676 populate_metadata read hostname 'smithi001' 2024-02-02T15:38:37.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.503+0000 7f9a2d014b80 1 --2- 172.21.15.1:0/1534043144 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x555bff88f2e0 0x555bff891700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:37.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.503+0000 7f9a2d014b80 4 client.28676 mds_command: new command op to 24502 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:38:37.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.503+0000 7f9a2d014b80 1 -- 172.21.15.1:0/1534043144 --> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555bff7bedd0 con 0x555bff88f2e0 2024-02-02T15:38:37.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.503+0000 7f9a12800700 1 --2- 172.21.15.1:0/1534043144 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x555bff88f2e0 0x555bff891700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:37.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.504+0000 7f9a12800700 1 --2- 172.21.15.1:0/1534043144 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x555bff88f2e0 0x555bff891700 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=44ccd2d03316ad4b server_cookie=3c02a5b3338f813e in_seq=0 out_seq=0 2024-02-02T15:38:37.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.504+0000 7f99fbe00700 10 client.28676 ms_handle_connect on v2:172.21.15.142:6834/1052606547 2024-02-02T15:38:37.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.504+0000 7f99fbe00700 1 -- 172.21.15.1:0/1534043144 <== mds.4 v2:172.21.15.142:6834/1052606547 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x555bff7bedd0 con 0x555bff88f2e0 2024-02-02T15:38:37.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.505+0000 7f99fbe00700 10 client.28676 handle_command_reply: tid=0 2024-02-02T15:38:37.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.505+0000 7f9a2d014b80 1 client.28676 shutdown 2024-02-02T15:38:37.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.505+0000 7f9a2d014b80 1 -- 172.21.15.1:0/1534043144 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555bff87f7e0 msgr2=0x555bff87cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:37.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.505+0000 7f9a2d014b80 1 --2- 172.21.15.1:0/1534043144 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555bff87f7e0 0x555bff87cbd0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c00b070 tx=0x7f9a0c00b3b0 comp rx=0 tx=0).stop 2024-02-02T15:38:37.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.505+0000 7f9a2d014b80 1 -- 172.21.15.1:0/1534043144 shutdown_connections 2024-02-02T15:38:37.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.505+0000 7f9a2d014b80 1 --2- 172.21.15.1:0/1534043144 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x555bff88f2e0 0x555bff891700 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:37.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.505+0000 7f9a2d014b80 1 --2- 172.21.15.1:0/1534043144 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555bff87f7e0 0x555bff87cbd0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:37.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.505+0000 7f99fbe00700 0 client.28676 ms_handle_reset on v2:172.21.15.142:6834/1052606547 2024-02-02T15:38:37.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.505+0000 7f9a2d014b80 1 --2- 172.21.15.1:0/1534043144 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555bff874010 0x555bff87f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:37.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.505+0000 7f9a2d014b80 1 --2- 172.21.15.1:0/1534043144 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555bff874930 0x555bff87c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:37.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.505+0000 7f9a2d014b80 1 -- 172.21.15.1:0/1534043144 >> 172.21.15.1:0/1534043144 conn(0x555bff7d3db0 msgr2=0x555bff86ffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:37.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.505+0000 7f9a2d014b80 1 -- 172.21.15.1:0/1534043144 shutdown_connections 2024-02-02T15:38:37.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.505+0000 7f9a2d014b80 1 -- 172.21.15.1:0/1534043144 wait complete. 2024-02-02T15:38:37.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.505+0000 7f9a2d014b80 20 client.28676 trim_cache size 0 max 16384 2024-02-02T15:38:37.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.519+0000 7f9a2d014b80 1 Processor -- start 2024-02-02T15:38:37.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.519+0000 7f9a2d014b80 1 -- start start 2024-02-02T15:38:37.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.519+0000 7f9a2d014b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555bff874010 0x555bff8862d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:37.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.519+0000 7f9a2d014b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555bff874930 0x555bff87ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:37.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.519+0000 7f9a2d014b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555bff87f7e0 0x555bff88a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:37.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.519+0000 7f9a2d014b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bff6496a0 con 0x555bff874010 2024-02-02T15:38:37.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.519+0000 7f9a2d014b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bff875260 con 0x555bff874930 2024-02-02T15:38:37.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.519+0000 7f9a2d014b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bff867fc0 con 0x555bff87f7e0 2024-02-02T15:38:37.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.519+0000 7f9a12800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555bff874010 0x555bff8862d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:37.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.519+0000 7f9a12800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555bff874010 0x555bff8862d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36778/0 (socket says 172.21.15.1:36778) 2024-02-02T15:38:37.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.519+0000 7f9a12800700 1 -- 172.21.15.1:0/1608727080 learned_addr learned my addr 172.21.15.1:0/1608727080 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:38:37.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.519+0000 7f9a11e00700 1 --2- 172.21.15.1:0/1608727080 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555bff874930 0x555bff87ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:37.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.520+0000 7f9a13200700 1 --2- 172.21.15.1:0/1608727080 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555bff87f7e0 0x555bff88a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:37.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.520+0000 7f9a12800700 1 -- 172.21.15.1:0/1608727080 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555bff87f7e0 msgr2=0x555bff88a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:37.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.520+0000 7f9a12800700 1 --2- 172.21.15.1:0/1608727080 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555bff87f7e0 0x555bff88a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:37.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.520+0000 7f9a12800700 1 -- 172.21.15.1:0/1608727080 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555bff874930 msgr2=0x555bff87ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:37.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.520+0000 7f9a12800700 1 --2- 172.21.15.1:0/1608727080 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555bff874930 0x555bff87ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:37.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.520+0000 7f9a12800700 1 -- 172.21.15.1:0/1608727080 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555bff86c0d0 con 0x555bff874010 2024-02-02T15:38:37.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.520+0000 7f9a12800700 1 --2- 172.21.15.1:0/1608727080 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555bff874010 0x555bff8862d0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f9a040843d0 tx=0x7f9a0400efa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a388055b6011d23c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:37.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.520+0000 7f9a03e00700 1 -- 172.21.15.1:0/1608727080 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0409d160 con 0x555bff874010 2024-02-02T15:38:37.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.520+0000 7f9a03e00700 1 -- 172.21.15.1:0/1608727080 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a040b0040 con 0x555bff874010 2024-02-02T15:38:37.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.520+0000 7f9a03e00700 1 -- 172.21.15.1:0/1608727080 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a04099a60 con 0x555bff874010 2024-02-02T15:38:37.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.521+0000 7f9a2d014b80 1 -- 172.21.15.1:0/1608727080 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555bff874010 msgr2=0x555bff8862d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:37.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.521+0000 7f9a2d014b80 1 --2- 172.21.15.1:0/1608727080 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555bff874010 0x555bff8862d0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f9a040843d0 tx=0x7f9a0400efa0 comp rx=0 tx=0).stop 2024-02-02T15:38:37.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.521+0000 7f9a2d014b80 1 -- 172.21.15.1:0/1608727080 shutdown_connections 2024-02-02T15:38:37.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.521+0000 7f9a2d014b80 1 --2- 172.21.15.1:0/1608727080 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555bff87f7e0 0x555bff88a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:37.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.521+0000 7f9a2d014b80 1 --2- 172.21.15.1:0/1608727080 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555bff874010 0x555bff8862d0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:37.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.521+0000 7f9a2d014b80 1 --2- 172.21.15.1:0/1608727080 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555bff874930 0x555bff87ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:37.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.521+0000 7f9a2d014b80 1 -- 172.21.15.1:0/1608727080 >> 172.21.15.1:0/1608727080 conn(0x555bff7d3db0 msgr2=0x555bff8777e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:37.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.521+0000 7f9a2d014b80 1 -- 172.21.15.1:0/1608727080 shutdown_connections 2024-02-02T15:38:37.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.521+0000 7f9a2d014b80 1 -- 172.21.15.1:0/1608727080 wait complete. 2024-02-02T15:38:37.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.521+0000 7f9a2d014b80 1 -- start start 2024-02-02T15:38:37.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.522+0000 7f9a2d014b80 1 Processor -- start 2024-02-02T15:38:37.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.522+0000 7f9a2d014b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555bff874010 0x555bff883620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:37.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.522+0000 7f9a2d014b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555bff874930 0x555bff883b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:37.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.522+0000 7f9a2d014b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555bff87f7e0 0x555bff8840a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:37.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.522+0000 7f9a2d014b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bff8866b0 con 0x555bff874010 2024-02-02T15:38:37.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.522+0000 7f9a2d014b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bff8dfdf0 con 0x555bff87f7e0 2024-02-02T15:38:37.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.522+0000 7f9a2d014b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x555bff8dffb0 con 0x555bff874930 2024-02-02T15:38:37.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.522+0000 7f9a2d014b80 10 client.0 _ll_register_callbacks cb 0x555bff763290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:38:37.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.522+0000 7f9a12800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555bff874010 0x555bff883620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:37.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.522+0000 7f9a12800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555bff874010 0x555bff883620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36782/0 (socket says 172.21.15.1:36782) 2024-02-02T15:38:37.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.522+0000 7f9a12800700 1 -- 172.21.15.1:0/682980853 learned_addr learned my addr 172.21.15.1:0/682980853 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:38:37.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.522+0000 7f9a13200700 1 --2- 172.21.15.1:0/682980853 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555bff87f7e0 0x555bff8840a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:37.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.523+0000 7f9a11e00700 1 --2- 172.21.15.1:0/682980853 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555bff874930 0x555bff883b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:37.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.523+0000 7f9a12800700 1 -- 172.21.15.1:0/682980853 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555bff874930 msgr2=0x555bff883b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:37.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.523+0000 7f9a12800700 1 --2- 172.21.15.1:0/682980853 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555bff874930 0x555bff883b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:37.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.523+0000 7f9a12800700 1 -- 172.21.15.1:0/682980853 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555bff87f7e0 msgr2=0x555bff8840a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:37.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.523+0000 7f9a12800700 1 --2- 172.21.15.1:0/682980853 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555bff87f7e0 0x555bff8840a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:37.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.523+0000 7f9a12800700 1 -- 172.21.15.1:0/682980853 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555bff6473b0 con 0x555bff874010 2024-02-02T15:38:37.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.523+0000 7f9a13200700 1 --2- 172.21.15.1:0/682980853 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555bff87f7e0 0x555bff8840a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:38:37.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.523+0000 7f9a12800700 1 --2- 172.21.15.1:0/682980853 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555bff874010 0x555bff883620 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f9a04096010 tx=0x7f9a04010be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bedf9b429918fb59 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:37.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.523+0000 7f99fbe00700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:38:37.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.523+0000 7f99fbe00700 1 -- 172.21.15.1:0/682980853 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a04098c30 con 0x555bff874010 2024-02-02T15:38:37.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.523+0000 7f99fbe00700 1 -- 172.21.15.1:0/682980853 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f9a040cc170 con 0x555bff874010 2024-02-02T15:38:37.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.523+0000 7f9a2d014b80 1 -- 172.21.15.1:0/682980853 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9a040a1610 con 0x555bff874010 2024-02-02T15:38:37.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.524+0000 7f99fbe00700 1 -- 172.21.15.1:0/682980853 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a04099800 con 0x555bff874010 2024-02-02T15:38:37.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.524+0000 7f9a2d014b80 10 client.18591 fetch_fsmap learned FSMap version 51 2024-02-02T15:38:37.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.524+0000 7f9a2d014b80 1 -- 172.21.15.1:0/682980853 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x555bff78bea0 con 0x555bff874010 2024-02-02T15:38:37.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.524+0000 7f99fbe00700 1 -- 172.21.15.1:0/682980853 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a040a1610 con 0x555bff874010 2024-02-02T15:38:37.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.524+0000 7f99fbe00700 1 -- 172.21.15.1:0/682980853 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 51) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f9a04080070 con 0x555bff874010 2024-02-02T15:38:37.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.524+0000 7f9a2d014b80 10 client.18591 fetch_fsmap finished waiting for FSMap version 51 2024-02-02T15:38:37.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.524+0000 7f9a2d014b80 10 client.18591 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-02-02T15:38:37.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.524+0000 7f9a2d014b80 20 client.18591 populate_metadata read hostname 'smithi001' 2024-02-02T15:38:37.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.524+0000 7f9a2d014b80 1 --2- 172.21.15.1:0/682980853 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x555bff8e11c0 0x555bff8e35b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:37.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.524+0000 7f9a2d014b80 4 client.18591 mds_command: new command op to 24502 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:38:37.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.524+0000 7f9a2d014b80 1 -- 172.21.15.1:0/682980853 --> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555bff784f50 con 0x555bff8e11c0 2024-02-02T15:38:37.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.524+0000 7f9a11e00700 1 --2- 172.21.15.1:0/682980853 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x555bff8e11c0 0x555bff8e35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:37.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.525+0000 7f9a11e00700 1 --2- 172.21.15.1:0/682980853 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x555bff8e11c0 0x555bff8e35b0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=12e2e4261055ba02 server_cookie=f6f35512e6e31ff0 in_seq=0 out_seq=0 2024-02-02T15:38:37.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.525+0000 7f99fbe00700 10 client.18591 ms_handle_connect on v2:172.21.15.142:6834/1052606547 2024-02-02T15:38:37.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.525+0000 7f99fbe00700 1 -- 172.21.15.1:0/682980853 <== mds.4 v2:172.21.15.142:6834/1052606547 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555bff784f50 con 0x555bff8e11c0 2024-02-02T15:38:37.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.525+0000 7f99fbe00700 10 client.18591 handle_command_reply: tid=0 2024-02-02T15:38:37.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.525+0000 7f9a2d014b80 1 client.18591 shutdown 2024-02-02T15:38:37.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.525+0000 7f9a2d014b80 1 -- 172.21.15.1:0/682980853 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555bff874010 msgr2=0x555bff883620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:37.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.525+0000 7f9a2d014b80 1 --2- 172.21.15.1:0/682980853 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555bff874010 0x555bff883620 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f9a04096010 tx=0x7f9a04010be0 comp rx=0 tx=0).stop 2024-02-02T15:38:37.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.525+0000 7f9a2d014b80 1 -- 172.21.15.1:0/682980853 shutdown_connections 2024-02-02T15:38:37.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.525+0000 7f9a2d014b80 1 --2- 172.21.15.1:0/682980853 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x555bff8e11c0 0x555bff8e35b0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:37.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.525+0000 7f9a2d014b80 1 --2- 172.21.15.1:0/682980853 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555bff87f7e0 0x555bff8840a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:37.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.525+0000 7f9a2d014b80 1 --2- 172.21.15.1:0/682980853 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x555bff874010 0x555bff883620 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:37.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.525+0000 7f9a2d014b80 1 --2- 172.21.15.1:0/682980853 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x555bff874930 0x555bff883b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:37.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.525+0000 7f99fbe00700 0 client.18591 ms_handle_reset on v2:172.21.15.142:6834/1052606547 2024-02-02T15:38:37.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.525+0000 7f9a2d014b80 1 -- 172.21.15.1:0/682980853 >> 172.21.15.1:0/682980853 conn(0x555bff7d3db0 msgr2=0x555bff88a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:37.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.525+0000 7f9a2d014b80 1 -- 172.21.15.1:0/682980853 shutdown_connections 2024-02-02T15:38:37.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.526+0000 7f9a2d014b80 1 -- 172.21.15.1:0/682980853 wait complete. 2024-02-02T15:38:37.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.526+0000 7f9a2d014b80 20 client.18591 trim_cache size 0 max 16384 2024-02-02T15:38:37.528 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:38:37.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.526+0000 7f9a19400700 1 -- 172.21.15.1:0/828552148 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f99f40a0710 msgr2=0x7f99f40a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:37.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.526+0000 7f9a19400700 1 --2- 172.21.15.1:0/828552148 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f99f40a0710 0x7f99f40a2bd0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c009fd0 tx=0x7f9a0c009480 comp rx=0 tx=0).stop 2024-02-02T15:38:37.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.526+0000 7f9a19400700 1 -- 172.21.15.1:0/828552148 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a140730e0 msgr2=0x7f9a1419d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:37.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.526+0000 7f9a19400700 1 --2- 172.21.15.1:0/828552148 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a140730e0 0x7f9a1419d550 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f9a0400e9a0 tx=0x7f9a0400ed60 comp rx=0 tx=0).stop 2024-02-02T15:38:37.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.526+0000 7f9a19400700 1 -- 172.21.15.1:0/828552148 shutdown_connections 2024-02-02T15:38:37.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.526+0000 7f9a19400700 1 --2- 172.21.15.1:0/828552148 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f99f40a0710 0x7f99f40a2bd0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:37.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.526+0000 7f9a19400700 1 --2- 172.21.15.1:0/828552148 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9a14111d90 0x7f9a141a1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:37.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.526+0000 7f9a19400700 1 --2- 172.21.15.1:0/828552148 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a140730e0 0x7f9a1419d550 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:37.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.526+0000 7f9a19400700 1 --2- 172.21.15.1:0/828552148 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a14073a00 0x7f9a1419da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:37.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.526+0000 7f9a19400700 1 -- 172.21.15.1:0/828552148 >> 172.21.15.1:0/828552148 conn(0x7f9a140fc810 msgr2=0x7f9a140fd480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:37.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.526+0000 7f9a19400700 1 -- 172.21.15.1:0/828552148 shutdown_connections 2024-02-02T15:38:37.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.526+0000 7f9a19400700 1 -- 172.21.15.1:0/828552148 wait complete. 2024-02-02T15:38:37.540 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:37 smithi001 ceph-mon[35470]: pgmap v361: 129 pgs: 129 active+clean; 528 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 172 op/s 2024-02-02T15:38:37.540 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:37 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/3000690820' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:38:37.543 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-02-02T15:38:37.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.764+0000 7fe5df800700 1 -- 172.21.15.1:0/2283269198 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe5d8103b80 msgr2=0x7fe5d8107bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:37.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.764+0000 7fe5df800700 1 --2- 172.21.15.1:0/2283269198 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe5d8103b80 0x7fe5d8107bc0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fe5c0009990 tx=0x7fe5c0009ca0 comp rx=0 tx=0).stop 2024-02-02T15:38:37.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.764+0000 7fe5df800700 1 -- 172.21.15.1:0/2283269198 shutdown_connections 2024-02-02T15:38:37.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.764+0000 7fe5df800700 1 --2- 172.21.15.1:0/2283269198 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe5d81082f0 0x7fe5d810fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:37.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.764+0000 7fe5df800700 1 --2- 172.21.15.1:0/2283269198 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe5d81031d0 0x7fe5d81035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:37.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.764+0000 7fe5df800700 1 --2- 172.21.15.1:0/2283269198 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe5d8103b80 0x7fe5d8107bc0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:37.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.764+0000 7fe5df800700 1 -- 172.21.15.1:0/2283269198 >> 172.21.15.1:0/2283269198 conn(0x7fe5d80fe9f0 msgr2=0x7fe5d8100e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:37.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.764+0000 7fe5df800700 1 -- 172.21.15.1:0/2283269198 shutdown_connections 2024-02-02T15:38:37.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.764+0000 7fe5df800700 1 -- 172.21.15.1:0/2283269198 wait complete. 2024-02-02T15:38:37.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.765+0000 7fe5df800700 1 Processor -- start 2024-02-02T15:38:37.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.765+0000 7fe5df800700 1 -- start start 2024-02-02T15:38:37.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.766+0000 7fe5df800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe5d81031d0 0x7fe5d8199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:37.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.766+0000 7fe5df800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe5d81082f0 0x7fe5d8199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:37.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.766+0000 7fe5df800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe5d819da30 0x7fe5d819deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:37.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.766+0000 7fe5df800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d819e4b0 con 0x7fe5d81031d0 2024-02-02T15:38:37.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.766+0000 7fe5df800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d819e620 con 0x7fe5d819da30 2024-02-02T15:38:37.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.766+0000 7fe5df800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d819e7e0 con 0x7fe5d81082f0 2024-02-02T15:38:37.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.766+0000 7fe5dce00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe5d81031d0 0x7fe5d8199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:37.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.766+0000 7fe5d7e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe5d81082f0 0x7fe5d8199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:37.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.766+0000 7fe5dce00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe5d81031d0 0x7fe5d8199130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36812/0 (socket says 172.21.15.1:36812) 2024-02-02T15:38:37.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.766+0000 7fe5d7e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe5d81082f0 0x7fe5d8199670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:53592/0 (socket says 172.21.15.1:53592) 2024-02-02T15:38:37.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.766+0000 7fe5dce00700 1 -- 172.21.15.1:0/3176457275 learned_addr learned my addr 172.21.15.1:0/3176457275 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:38:37.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.766+0000 7fe5dd800700 1 --2- 172.21.15.1:0/3176457275 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe5d819da30 0x7fe5d819deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:37.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.767+0000 7fe5d7e00700 1 -- 172.21.15.1:0/3176457275 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe5d819da30 msgr2=0x7fe5d819deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:37.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.767+0000 7fe5d7e00700 1 --2- 172.21.15.1:0/3176457275 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe5d819da30 0x7fe5d819deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:37.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.767+0000 7fe5d7e00700 1 -- 172.21.15.1:0/3176457275 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe5d81031d0 msgr2=0x7fe5d8199130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:37.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.767+0000 7fe5d7e00700 1 --2- 172.21.15.1:0/3176457275 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe5d81031d0 0x7fe5d8199130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:37.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.767+0000 7fe5d7e00700 1 -- 172.21.15.1:0/3176457275 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5c00095f0 con 0x7fe5d81082f0 2024-02-02T15:38:37.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.767+0000 7fe5dce00700 1 --2- 172.21.15.1:0/3176457275 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe5d81031d0 0x7fe5d8199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:38:37.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.767+0000 7fe5d7e00700 1 --2- 172.21.15.1:0/3176457275 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe5d81082f0 0x7fe5d8199670 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fe5c0000c00 tx=0x7fe5c0005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:37.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.767+0000 7fe5d5600700 1 -- 172.21.15.1:0/3176457275 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c001d070 con 0x7fe5d81082f0 2024-02-02T15:38:37.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.767+0000 7fe5df800700 1 -- 172.21.15.1:0/3176457275 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5d81a5ce0 con 0x7fe5d81082f0 2024-02-02T15:38:37.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.767+0000 7fe5d5600700 1 -- 172.21.15.1:0/3176457275 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe5c0021410 con 0x7fe5d81082f0 2024-02-02T15:38:37.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.767+0000 7fe5d5600700 1 -- 172.21.15.1:0/3176457275 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c0017650 con 0x7fe5d81082f0 2024-02-02T15:38:37.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.767+0000 7fe5df800700 1 -- 172.21.15.1:0/3176457275 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5d81a6160 con 0x7fe5d81082f0 2024-02-02T15:38:37.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.771+0000 7fe5d5600700 1 -- 172.21.15.1:0/3176457275 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7fe5c00217f0 con 0x7fe5d81082f0 2024-02-02T15:38:37.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.772+0000 7fe5f33eeb80 1 Processor -- start 2024-02-02T15:38:37.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.772+0000 7fe5f33eeb80 1 -- start start 2024-02-02T15:38:37.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.772+0000 7fe5f33eeb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba2eaff010 0x55ba2eaff3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:37.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.772+0000 7fe5f33eeb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba2eaff930 0x55ba2eb0a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:37.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.772+0000 7fe5f33eeb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba2eb0a7e0 0x55ba2eb0cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:37.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.772+0000 7fe5f33eeb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba2ea734c0 con 0x55ba2eaff930 2024-02-02T15:38:37.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.772+0000 7fe5f33eeb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba2ea57aa0 con 0x55ba2eaff010 2024-02-02T15:38:37.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.772+0000 7fe5f33eeb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba2ea55460 con 0x55ba2eb0a7e0 2024-02-02T15:38:37.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.773+0000 7fe5dd800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba2eaff010 0x55ba2eaff3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:37.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.773+0000 7fe5dd800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba2eaff010 0x55ba2eaff3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42546/0 (socket says 172.21.15.1:42546) 2024-02-02T15:38:37.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.773+0000 7fe5dd800700 1 -- 172.21.15.1:0/840330138 learned_addr learned my addr 172.21.15.1:0/840330138 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:38:37.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.773+0000 7fe5dce00700 1 --2- 172.21.15.1:0/840330138 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba2eaff930 0x55ba2eb0a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:37.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.773+0000 7fe5d7e00700 1 --2- 172.21.15.1:0/840330138 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba2eb0a7e0 0x55ba2eb0cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:37.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.773+0000 7fe5dd800700 1 -- 172.21.15.1:0/840330138 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba2eb0a7e0 msgr2=0x55ba2eb0cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:37.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.773+0000 7fe5dd800700 1 --2- 172.21.15.1:0/840330138 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba2eb0a7e0 0x55ba2eb0cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:37.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.773+0000 7fe5dd800700 1 -- 172.21.15.1:0/840330138 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba2eaff930 msgr2=0x55ba2eb0a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:37.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.773+0000 7fe5d5600700 1 --2- 172.21.15.1:0/3176457275 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fe5b80a0690 0x7fe5b80a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:37.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.773+0000 7fe5dd800700 1 --2- 172.21.15.1:0/840330138 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba2eaff930 0x55ba2eb0a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:37.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.773+0000 7fe5dd800700 1 -- 172.21.15.1:0/840330138 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba2e8d23b0 con 0x55ba2eaff010 2024-02-02T15:38:37.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.773+0000 7fe5dce00700 1 --2- 172.21.15.1:0/840330138 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba2eaff930 0x55ba2eb0a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:38:37.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.773+0000 7fe5d5600700 1 -- 172.21.15.1:0/3176457275 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fe5c00cd3f0 con 0x7fe5d81082f0 2024-02-02T15:38:37.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.773+0000 7fe5dd800700 1 --2- 172.21.15.1:0/3176457275 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fe5b80a0690 0x7fe5b80a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:37.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.774+0000 7fe5dd800700 1 --2- 172.21.15.1:0/840330138 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba2eaff010 0x55ba2eaff3f0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fe5d00116b0 tx=0x7fe5d0009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8589ba121436b78 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:37.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.774+0000 7fe5ce000700 1 -- 172.21.15.1:0/840330138 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d000e040 con 0x55ba2eaff010 2024-02-02T15:38:37.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.774+0000 7fe5ce000700 1 -- 172.21.15.1:0/840330138 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe5d000f040 con 0x55ba2eaff010 2024-02-02T15:38:37.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.774+0000 7fe5ce000700 1 -- 172.21.15.1:0/840330138 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d00157b0 con 0x55ba2eaff010 2024-02-02T15:38:37.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.775+0000 7fe5f33eeb80 1 -- 172.21.15.1:0/840330138 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba2eaff010 msgr2=0x55ba2eaff3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:37.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.775+0000 7fe5dd800700 1 --2- 172.21.15.1:0/3176457275 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fe5b80a0690 0x7fe5b80a2b50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe5d0020000 tx=0x7fe5d0023000 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:37.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.775+0000 7fe5f33eeb80 1 --2- 172.21.15.1:0/840330138 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba2eaff010 0x55ba2eaff3f0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fe5d00116b0 tx=0x7fe5d0009920 comp rx=0 tx=0).stop 2024-02-02T15:38:37.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.775+0000 7fe5f33eeb80 1 -- 172.21.15.1:0/840330138 shutdown_connections 2024-02-02T15:38:37.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.775+0000 7fe5f33eeb80 1 --2- 172.21.15.1:0/840330138 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba2eb0a7e0 0x55ba2eb0cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:37.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.775+0000 7fe5f33eeb80 1 --2- 172.21.15.1:0/840330138 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba2eaff010 0x55ba2eaff3f0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:37.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.775+0000 7fe5f33eeb80 1 --2- 172.21.15.1:0/840330138 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba2eaff930 0x55ba2eb0a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:37.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.775+0000 7fe5f33eeb80 1 -- 172.21.15.1:0/840330138 >> 172.21.15.1:0/840330138 conn(0x55ba2ea5edb0 msgr2=0x55ba2eafb3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:37.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.775+0000 7fe5f33eeb80 1 -- 172.21.15.1:0/840330138 shutdown_connections 2024-02-02T15:38:37.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.775+0000 7fe5f33eeb80 1 -- 172.21.15.1:0/840330138 wait complete. 2024-02-02T15:38:37.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.775+0000 7fe5f33eeb80 1 -- start start 2024-02-02T15:38:37.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.776+0000 7fe5f33eeb80 1 Processor -- start 2024-02-02T15:38:37.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.776+0000 7fe5f33eeb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba2eaff010 0x55ba2eb0a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:37.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.776+0000 7fe5f33eeb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba2eaff930 0x55ba2eb07450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:37.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.776+0000 7fe5f33eeb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba2eb0a7e0 0x55ba2eb07b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:37.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.776+0000 7fe5f33eeb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba2ea50d50 con 0x55ba2eb0a7e0 2024-02-02T15:38:37.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.776+0000 7fe5f33eeb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba2e9ecd80 con 0x55ba2eaff010 2024-02-02T15:38:37.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.776+0000 7fe5f33eeb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba2e9eca20 con 0x55ba2eaff930 2024-02-02T15:38:37.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.776+0000 7fe5f33eeb80 10 client.0 _ll_register_callbacks cb 0x55ba2e9ee290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:38:37.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.776+0000 7fe5dce00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba2eaff010 0x55ba2eb0a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:37.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.777+0000 7fe5dce00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba2eaff010 0x55ba2eb0a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42558/0 (socket says 172.21.15.1:42558) 2024-02-02T15:38:37.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.777+0000 7fe5dce00700 1 -- 172.21.15.1:0/2351950347 learned_addr learned my addr 172.21.15.1:0/2351950347 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:38:37.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.777+0000 7fe5d7e00700 1 --2- 172.21.15.1:0/2351950347 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba2eaff930 0x55ba2eb07450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:37.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.777+0000 7fe5dd800700 1 --2- 172.21.15.1:0/2351950347 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba2eb0a7e0 0x55ba2eb07b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:37.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.777+0000 7fe5dce00700 1 -- 172.21.15.1:0/2351950347 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba2eaff930 msgr2=0x55ba2eb07450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:37.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.777+0000 7fe5dce00700 1 --2- 172.21.15.1:0/2351950347 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba2eaff930 0x55ba2eb07450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:37.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.777+0000 7fe5dce00700 1 -- 172.21.15.1:0/2351950347 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba2eb0a7e0 msgr2=0x55ba2eb07b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:37.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.777+0000 7fe5dce00700 1 --2- 172.21.15.1:0/2351950347 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba2eb0a7e0 0x55ba2eb07b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:37.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.777+0000 7fe5dce00700 1 -- 172.21.15.1:0/2351950347 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5c8009d60 con 0x55ba2eaff010 2024-02-02T15:38:37.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.777+0000 7fe5dd800700 1 --2- 172.21.15.1:0/2351950347 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba2eb0a7e0 0x55ba2eb07b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:38:37.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.777+0000 7fe5dce00700 1 --2- 172.21.15.1:0/2351950347 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba2eaff010 0x55ba2eb0a240 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fe5c8011e60 tx=0x7fe5c800f6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a856a596ffef46a server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:37.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.777+0000 7fe5c6a00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:38:37.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.777+0000 7fe5c6a00700 1 -- 172.21.15.1:0/2351950347 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c8016070 con 0x55ba2eaff010 2024-02-02T15:38:37.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.778+0000 7fe5c6a00700 1 -- 172.21.15.1:0/2351950347 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fe5c8011050 con 0x55ba2eaff010 2024-02-02T15:38:37.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.778+0000 7fe5f33eeb80 1 -- 172.21.15.1:0/2351950347 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ba2eaf30f0 con 0x55ba2eaff010 2024-02-02T15:38:37.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.778+0000 7fe5c6a00700 1 -- 172.21.15.1:0/2351950347 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe5c8011210 con 0x55ba2eaff010 2024-02-02T15:38:37.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.778+0000 7fe5f33eeb80 10 client.28694 fetch_fsmap learned FSMap version 51 2024-02-02T15:38:37.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.778+0000 7fe5f33eeb80 1 -- 172.21.15.1:0/2351950347 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x7fe5d0011330 con 0x55ba2eaff010 2024-02-02T15:38:37.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.778+0000 7fe5c6a00700 1 -- 172.21.15.1:0/2351950347 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ba2eaf30f0 con 0x55ba2eaff010 2024-02-02T15:38:37.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.778+0000 7fe5c6a00700 1 -- 172.21.15.1:0/2351950347 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 51) v1 ==== 7474+0+0 (secure 0 0 0) 0x7fe5c8003bb0 con 0x55ba2eaff010 2024-02-02T15:38:37.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.778+0000 7fe5f33eeb80 10 client.28694 fetch_fsmap finished waiting for FSMap version 51 2024-02-02T15:38:37.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.778+0000 7fe5f33eeb80 10 client.28694 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-02-02T15:38:37.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.778+0000 7fe5f33eeb80 20 client.28694 populate_metadata read hostname 'smithi001' 2024-02-02T15:38:37.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.779+0000 7fe5f33eeb80 1 --2- 172.21.15.1:0/2351950347 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55ba2eb1a220 0x55ba2eb1c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:37.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.779+0000 7fe5f33eeb80 4 client.28694 mds_command: new command op to 24514 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:38:37.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.779+0000 7fe5f33eeb80 1 -- 172.21.15.1:0/2351950347 --> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ba2ea49dd0 con 0x55ba2eb1a220 2024-02-02T15:38:37.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.779+0000 7fe5d7e00700 1 --2- 172.21.15.1:0/2351950347 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55ba2eb1a220 0x55ba2eb1c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:37.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.780+0000 7fe5d7e00700 1 --2- 172.21.15.1:0/2351950347 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55ba2eb1a220 0x55ba2eb1c640 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d9bfbcb70f01a1be server_cookie=47bfea3772556abd in_seq=0 out_seq=0 2024-02-02T15:38:37.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.780+0000 7fe5c6a00700 10 client.28694 ms_handle_connect on v2:172.21.15.142:6836/2870289019 2024-02-02T15:38:37.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.780+0000 7fe5c6a00700 1 -- 172.21.15.1:0/2351950347 <== mds.3 v2:172.21.15.142:6836/2870289019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55ba2ea49dd0 con 0x55ba2eb1a220 2024-02-02T15:38:37.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.780+0000 7fe5c6a00700 10 client.28694 handle_command_reply: tid=0 2024-02-02T15:38:37.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.780+0000 7fe5f33eeb80 1 client.28694 shutdown 2024-02-02T15:38:37.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.781+0000 7fe5f33eeb80 1 -- 172.21.15.1:0/2351950347 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba2eaff010 msgr2=0x55ba2eb0a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:37.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.781+0000 7fe5f33eeb80 1 --2- 172.21.15.1:0/2351950347 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba2eaff010 0x55ba2eb0a240 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fe5c8011e60 tx=0x7fe5c800f6a0 comp rx=0 tx=0).stop 2024-02-02T15:38:37.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.781+0000 7fe5f33eeb80 1 -- 172.21.15.1:0/2351950347 shutdown_connections 2024-02-02T15:38:37.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.781+0000 7fe5f33eeb80 1 --2- 172.21.15.1:0/2351950347 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55ba2eb1a220 0x55ba2eb1c640 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:37.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.781+0000 7fe5f33eeb80 1 --2- 172.21.15.1:0/2351950347 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba2eb0a7e0 0x55ba2eb07b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:37.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.781+0000 7fe5f33eeb80 1 --2- 172.21.15.1:0/2351950347 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba2eaff010 0x55ba2eb0a240 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:37.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.781+0000 7fe5f33eeb80 1 --2- 172.21.15.1:0/2351950347 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba2eaff930 0x55ba2eb07450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:37.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.781+0000 7fe5c6a00700 0 client.28694 ms_handle_reset on v2:172.21.15.142:6836/2870289019 2024-02-02T15:38:37.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.781+0000 7fe5f33eeb80 1 -- 172.21.15.1:0/2351950347 >> 172.21.15.1:0/2351950347 conn(0x55ba2ea5edb0 msgr2=0x55ba2eafafa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:37.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.781+0000 7fe5f33eeb80 1 -- 172.21.15.1:0/2351950347 shutdown_connections 2024-02-02T15:38:37.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.781+0000 7fe5f33eeb80 1 -- 172.21.15.1:0/2351950347 wait complete. 2024-02-02T15:38:37.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.781+0000 7fe5f33eeb80 20 client.28694 trim_cache size 0 max 16384 2024-02-02T15:38:37.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.800+0000 7fe5f33eeb80 1 Processor -- start 2024-02-02T15:38:37.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.801+0000 7fe5f33eeb80 1 -- start start 2024-02-02T15:38:37.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.801+0000 7fe5f33eeb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba2eaff010 0x55ba2eb09d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:37.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.801+0000 7fe5f33eeb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba2eaff930 0x55ba2eb06a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:37.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.801+0000 7fe5f33eeb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba2eb0a7e0 0x55ba2eb06fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:37.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.801+0000 7fe5f33eeb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba2e8d46a0 con 0x55ba2eaff010 2024-02-02T15:38:37.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.801+0000 7fe5f33eeb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba2eafc1e0 con 0x55ba2eaff930 2024-02-02T15:38:37.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.801+0000 7fe5f33eeb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba2eb08190 con 0x55ba2eb0a7e0 2024-02-02T15:38:37.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.801+0000 7fe5d7e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba2eaff930 0x55ba2eb06a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:37.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.801+0000 7fe5d7e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba2eaff930 0x55ba2eb06a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42566/0 (socket says 172.21.15.1:42566) 2024-02-02T15:38:37.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.801+0000 7fe5d7e00700 1 -- 172.21.15.1:0/2572735386 learned_addr learned my addr 172.21.15.1:0/2572735386 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:38:37.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.801+0000 7fe5dd800700 1 --2- 172.21.15.1:0/2572735386 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba2eb0a7e0 0x55ba2eb06fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:37.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.802+0000 7fe5dce00700 1 --2- 172.21.15.1:0/2572735386 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba2eaff010 0x55ba2eb09d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:37.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.802+0000 7fe5d7e00700 1 -- 172.21.15.1:0/2572735386 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba2eb0a7e0 msgr2=0x55ba2eb06fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:37.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.802+0000 7fe5d7e00700 1 --2- 172.21.15.1:0/2572735386 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba2eb0a7e0 0x55ba2eb06fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:37.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.802+0000 7fe5d7e00700 1 -- 172.21.15.1:0/2572735386 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba2eaff010 msgr2=0x55ba2eb09d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:37.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.802+0000 7fe5d7e00700 1 --2- 172.21.15.1:0/2572735386 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba2eaff010 0x55ba2eb09d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:37.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.802+0000 7fe5d7e00700 1 -- 172.21.15.1:0/2572735386 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba2e87eb60 con 0x55ba2eaff930 2024-02-02T15:38:37.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.802+0000 7fe5d7e00700 1 --2- 172.21.15.1:0/2572735386 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba2eaff930 0x55ba2eb06a60 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fe5c009a530 tx=0x7fe5c0009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d099254eaa5f5aa6 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:37.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.802+0000 7fe5cea00700 1 -- 172.21.15.1:0/2572735386 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c009a0e0 con 0x55ba2eaff930 2024-02-02T15:38:37.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.802+0000 7fe5cea00700 1 -- 172.21.15.1:0/2572735386 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe5c00cae90 con 0x55ba2eaff930 2024-02-02T15:38:37.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.802+0000 7fe5cea00700 1 -- 172.21.15.1:0/2572735386 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c00a2580 con 0x55ba2eaff930 2024-02-02T15:38:37.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.803+0000 7fe5f33eeb80 1 -- 172.21.15.1:0/2572735386 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba2eaff930 msgr2=0x55ba2eb06a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:37.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.803+0000 7fe5f33eeb80 1 --2- 172.21.15.1:0/2572735386 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba2eaff930 0x55ba2eb06a60 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fe5c009a530 tx=0x7fe5c0009ca0 comp rx=0 tx=0).stop 2024-02-02T15:38:37.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.803+0000 7fe5f33eeb80 1 -- 172.21.15.1:0/2572735386 shutdown_connections 2024-02-02T15:38:37.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.803+0000 7fe5f33eeb80 1 --2- 172.21.15.1:0/2572735386 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba2eb0a7e0 0x55ba2eb06fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:37.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.803+0000 7fe5f33eeb80 1 --2- 172.21.15.1:0/2572735386 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba2eaff010 0x55ba2eb09d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:37.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.803+0000 7fe5f33eeb80 1 --2- 172.21.15.1:0/2572735386 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba2eaff930 0x55ba2eb06a60 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:37.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.803+0000 7fe5f33eeb80 1 -- 172.21.15.1:0/2572735386 >> 172.21.15.1:0/2572735386 conn(0x55ba2ea5edb0 msgr2=0x55ba2eb16560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:37.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.803+0000 7fe5f33eeb80 1 -- 172.21.15.1:0/2572735386 shutdown_connections 2024-02-02T15:38:37.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.803+0000 7fe5f33eeb80 1 -- 172.21.15.1:0/2572735386 wait complete. 2024-02-02T15:38:37.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.804+0000 7fe5f33eeb80 1 -- start start 2024-02-02T15:38:37.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.804+0000 7fe5f33eeb80 1 Processor -- start 2024-02-02T15:38:37.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.806+0000 7fe5f33eeb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba2eaff010 0x55ba2eb10f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:37.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.806+0000 7fe5f33eeb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba2eaff930 0x55ba2eb0f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:37.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.806+0000 7fe5f33eeb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba2eb0a7e0 0x55ba2eb0f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:37.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.806+0000 7fe5f33eeb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba2eb0fd00 con 0x55ba2eaff930 2024-02-02T15:38:37.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.806+0000 7fe5f33eeb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba2eb0fe70 con 0x55ba2eb0a7e0 2024-02-02T15:38:37.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.806+0000 7fe5f33eeb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba2eb10030 con 0x55ba2eaff010 2024-02-02T15:38:37.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.806+0000 7fe5f33eeb80 10 client.0 _ll_register_callbacks cb 0x55ba2e9ee290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:38:37.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.807+0000 7fe5dd800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba2eb0a7e0 0x55ba2eb0f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:37.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.807+0000 7fe5dd800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba2eb0a7e0 0x55ba2eb0f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42572/0 (socket says 172.21.15.1:42572) 2024-02-02T15:38:37.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.807+0000 7fe5dd800700 1 -- 172.21.15.1:0/711485405 learned_addr learned my addr 172.21.15.1:0/711485405 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:38:37.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.808+0000 7fe5dce00700 1 --2- 172.21.15.1:0/711485405 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba2eaff010 0x55ba2eb10f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:37.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.808+0000 7fe5d7e00700 1 --2- 172.21.15.1:0/711485405 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba2eaff930 0x55ba2eb0f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:37.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.808+0000 7fe5dd800700 1 -- 172.21.15.1:0/711485405 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba2eaff010 msgr2=0x55ba2eb10f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:37.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.808+0000 7fe5dd800700 1 --2- 172.21.15.1:0/711485405 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba2eaff010 0x55ba2eb10f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:37.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.808+0000 7fe5dd800700 1 -- 172.21.15.1:0/711485405 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba2eaff930 msgr2=0x55ba2eb0f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:37.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.808+0000 7fe5dd800700 1 --2- 172.21.15.1:0/711485405 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba2eaff930 0x55ba2eb0f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:37.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.808+0000 7fe5dd800700 1 -- 172.21.15.1:0/711485405 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5d81a6160 con 0x55ba2eb0a7e0 2024-02-02T15:38:37.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.809+0000 7fe5dce00700 1 --2- 172.21.15.1:0/711485405 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba2eaff010 0x55ba2eb10f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:38:37.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.810+0000 7fe5dd800700 1 --2- 172.21.15.1:0/711485405 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba2eb0a7e0 0x55ba2eb0f7c0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fe5d00094f0 tx=0x7fe5d0014dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=581663aa6b7a6fbd server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:37.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.810+0000 7fe5c6a00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:38:37.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.811+0000 7fe5c6a00700 1 -- 172.21.15.1:0/711485405 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d000e040 con 0x55ba2eb0a7e0 2024-02-02T15:38:37.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.811+0000 7fe5c6a00700 1 -- 172.21.15.1:0/711485405 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fe5d0015a60 con 0x55ba2eb0a7e0 2024-02-02T15:38:37.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.812+0000 7fe5f33eeb80 1 -- 172.21.15.1:0/711485405 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe5d000f040 con 0x55ba2eb0a7e0 2024-02-02T15:38:37.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.812+0000 7fe5d7e00700 1 --2- 172.21.15.1:0/711485405 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba2eaff930 0x55ba2eb0f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:38:37.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.812+0000 7fe5c6a00700 1 -- 172.21.15.1:0/711485405 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe5d000e040 con 0x55ba2eb0a7e0 2024-02-02T15:38:37.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.813+0000 7fe5f33eeb80 10 client.28700 fetch_fsmap learned FSMap version 51 2024-02-02T15:38:37.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.813+0000 7fe5f33eeb80 1 -- 172.21.15.1:0/711485405 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x55ba2ea16ea0 con 0x55ba2eb0a7e0 2024-02-02T15:38:37.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.814+0000 7fe5c6a00700 1 -- 172.21.15.1:0/711485405 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe5d0015c20 con 0x55ba2eb0a7e0 2024-02-02T15:38:37.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.814+0000 7fe5c6a00700 1 -- 172.21.15.1:0/711485405 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 51) v1 ==== 7474+0+0 (secure 0 0 0) 0x7fe5d0007a70 con 0x55ba2eb0a7e0 2024-02-02T15:38:37.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.814+0000 7fe5f33eeb80 10 client.28700 fetch_fsmap finished waiting for FSMap version 51 2024-02-02T15:38:37.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.814+0000 7fe5f33eeb80 10 client.28700 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-02-02T15:38:37.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.814+0000 7fe5f33eeb80 20 client.28700 populate_metadata read hostname 'smithi001' 2024-02-02T15:38:37.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.814+0000 7fe5f33eeb80 1 --2- 172.21.15.1:0/711485405 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55ba2eb757a0 0x55ba2eb77bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:37.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.814+0000 7fe5f33eeb80 4 client.28700 mds_command: new command op to 24514 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:38:37.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.814+0000 7fe5f33eeb80 1 -- 172.21.15.1:0/711485405 --> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7fe5d000f040 con 0x55ba2eb757a0 2024-02-02T15:38:37.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.815+0000 7fe5dce00700 1 --2- 172.21.15.1:0/711485405 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55ba2eb757a0 0x55ba2eb77bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:37.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.817+0000 7fe5dce00700 1 --2- 172.21.15.1:0/711485405 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55ba2eb757a0 0x55ba2eb77bc0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f8e8634ede01f34a server_cookie=211f5e75a06813bd in_seq=0 out_seq=0 2024-02-02T15:38:37.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.817+0000 7fe5c6a00700 10 client.28700 ms_handle_connect on v2:172.21.15.142:6836/2870289019 2024-02-02T15:38:37.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.818+0000 7fe5c6a00700 1 -- 172.21.15.1:0/711485405 <== mds.3 v2:172.21.15.142:6836/2870289019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7fe5d000f040 con 0x55ba2eb757a0 2024-02-02T15:38:37.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.818+0000 7fe5c6a00700 10 client.28700 handle_command_reply: tid=0 2024-02-02T15:38:37.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.818+0000 7fe5f33eeb80 1 client.28700 shutdown 2024-02-02T15:38:37.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.818+0000 7fe5f33eeb80 1 -- 172.21.15.1:0/711485405 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba2eb0a7e0 msgr2=0x55ba2eb0f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:37.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.818+0000 7fe5f33eeb80 1 --2- 172.21.15.1:0/711485405 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba2eb0a7e0 0x55ba2eb0f7c0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fe5d00094f0 tx=0x7fe5d0014dd0 comp rx=0 tx=0).stop 2024-02-02T15:38:37.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.818+0000 7fe5f33eeb80 1 -- 172.21.15.1:0/711485405 shutdown_connections 2024-02-02T15:38:37.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.818+0000 7fe5f33eeb80 1 --2- 172.21.15.1:0/711485405 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ba2eb0a7e0 0x55ba2eb0f7c0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:37.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.818+0000 7fe5f33eeb80 1 --2- 172.21.15.1:0/711485405 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ba2eaff010 0x55ba2eb10f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:37.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.818+0000 7fe5f33eeb80 1 --2- 172.21.15.1:0/711485405 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55ba2eb757a0 0x55ba2eb77bc0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:37.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.818+0000 7fe5f33eeb80 1 --2- 172.21.15.1:0/711485405 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ba2eaff930 0x55ba2eb0f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:37.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.818+0000 7fe5f33eeb80 1 -- 172.21.15.1:0/711485405 >> 172.21.15.1:0/711485405 conn(0x55ba2ea5edb0 msgr2=0x55ba2eb010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:37.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.819+0000 7fe5c6a00700 0 client.28700 ms_handle_reset on v2:172.21.15.142:6836/2870289019 2024-02-02T15:38:37.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.819+0000 7fe5f33eeb80 1 -- 172.21.15.1:0/711485405 shutdown_connections 2024-02-02T15:38:37.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.819+0000 7fe5f33eeb80 1 -- 172.21.15.1:0/711485405 wait complete. 2024-02-02T15:38:37.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.820+0000 7fe5f33eeb80 20 client.28700 trim_cache size 0 max 16384 2024-02-02T15:38:37.819 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:38:37.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.821+0000 7fe5df800700 1 -- 172.21.15.1:0/3176457275 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fe5b80a0690 msgr2=0x7fe5b80a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:37.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.821+0000 7fe5df800700 1 --2- 172.21.15.1:0/3176457275 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fe5b80a0690 0x7fe5b80a2b50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe5d0020000 tx=0x7fe5d0023000 comp rx=0 tx=0).stop 2024-02-02T15:38:37.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.821+0000 7fe5df800700 1 -- 172.21.15.1:0/3176457275 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe5d81082f0 msgr2=0x7fe5d8199670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:37.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.821+0000 7fe5df800700 1 --2- 172.21.15.1:0/3176457275 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe5d81082f0 0x7fe5d8199670 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fe5c0000c00 tx=0x7fe5c0005740 comp rx=0 tx=0).stop 2024-02-02T15:38:37.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.822+0000 7fe5df800700 1 -- 172.21.15.1:0/3176457275 shutdown_connections 2024-02-02T15:38:37.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.822+0000 7fe5df800700 1 --2- 172.21.15.1:0/3176457275 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fe5b80a0690 0x7fe5b80a2b50 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:37.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.822+0000 7fe5df800700 1 --2- 172.21.15.1:0/3176457275 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe5d819da30 0x7fe5d819deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:37.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.822+0000 7fe5df800700 1 --2- 172.21.15.1:0/3176457275 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe5d81031d0 0x7fe5d8199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:37.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.822+0000 7fe5df800700 1 --2- 172.21.15.1:0/3176457275 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe5d81082f0 0x7fe5d8199670 secure :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fe5c0000c00 tx=0x7fe5c0005740 comp rx=0 tx=0).stop 2024-02-02T15:38:37.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.822+0000 7fe5df800700 1 -- 172.21.15.1:0/3176457275 >> 172.21.15.1:0/3176457275 conn(0x7fe5d80fe9f0 msgr2=0x7fe5d80ff4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:37.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.822+0000 7fe5df800700 1 -- 172.21.15.1:0/3176457275 shutdown_connections 2024-02-02T15:38:37.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:37.822+0000 7fe5df800700 1 -- 172.21.15.1:0/3176457275 wait complete. 2024-02-02T15:38:37.840 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-02T15:38:37.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:37 smithi071 ceph-mon[44707]: pgmap v361: 129 pgs: 129 active+clean; 528 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 172 op/s 2024-02-02T15:38:37.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:37 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/3000690820' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:38:37.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:37 smithi142 ceph-mon[45728]: pgmap v361: 129 pgs: 129 active+clean; 528 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 172 op/s 2024-02-02T15:38:37.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:37 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/3000690820' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:38:38.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.066+0000 7fdfc7800700 1 -- 172.21.15.1:0/2785965671 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdfc0103be0 msgr2=0x7fdfc0107c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:38.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.066+0000 7fdfc7800700 1 --2- 172.21.15.1:0/2785965671 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdfc0103be0 0x7fdfc0107c20 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fdfa80098f0 tx=0x7fdfa8009c00 comp rx=0 tx=0).stop 2024-02-02T15:38:38.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.067+0000 7fdfc7800700 1 -- 172.21.15.1:0/2785965671 shutdown_connections 2024-02-02T15:38:38.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.067+0000 7fdfc7800700 1 --2- 172.21.15.1:0/2785965671 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdfc0108350 0x7fdfc010fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.067+0000 7fdfc7800700 1 --2- 172.21.15.1:0/2785965671 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdfc0103230 0x7fdfc0103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.067+0000 7fdfc7800700 1 --2- 172.21.15.1:0/2785965671 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdfc0103be0 0x7fdfc0107c20 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.067+0000 7fdfc7800700 1 -- 172.21.15.1:0/2785965671 >> 172.21.15.1:0/2785965671 conn(0x7fdfc00fea70 msgr2=0x7fdfc0100e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:38.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.067+0000 7fdfc7800700 1 -- 172.21.15.1:0/2785965671 shutdown_connections 2024-02-02T15:38:38.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.067+0000 7fdfc7800700 1 -- 172.21.15.1:0/2785965671 wait complete. 2024-02-02T15:38:38.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.068+0000 7fdfc7800700 1 Processor -- start 2024-02-02T15:38:38.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.068+0000 7fdfc7800700 1 -- start start 2024-02-02T15:38:38.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.069+0000 7fdfc7800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdfc0103230 0x7fdfc0199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:38.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.069+0000 7fdfc7800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdfc0108350 0x7fdfc01996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:38.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.069+0000 7fdfc7800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdfc019db30 0x7fdfc019dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:38.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.069+0000 7fdfc7800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc019e5b0 con 0x7fdfc0103230 2024-02-02T15:38:38.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.069+0000 7fdfc7800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc019e720 con 0x7fdfc0108350 2024-02-02T15:38:38.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.069+0000 7fdfc7800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc019e8e0 con 0x7fdfc019db30 2024-02-02T15:38:38.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.069+0000 7fdfc4e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdfc0103230 0x7fdfc0199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:38.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.069+0000 7fdfbfe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdfc0108350 0x7fdfc01996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:38.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.069+0000 7fdfc5800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdfc019db30 0x7fdfc019dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:38.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.069+0000 7fdfbfe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdfc0108350 0x7fdfc01996d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42600/0 (socket says 172.21.15.1:42600) 2024-02-02T15:38:38.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.069+0000 7fdfbfe00700 1 -- 172.21.15.1:0/2562894975 learned_addr learned my addr 172.21.15.1:0/2562894975 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:38:38.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.069+0000 7fdfbfe00700 1 -- 172.21.15.1:0/2562894975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdfc019db30 msgr2=0x7fdfc019dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:38.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.069+0000 7fdfbfe00700 1 --2- 172.21.15.1:0/2562894975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdfc019db30 0x7fdfc019dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.069+0000 7fdfbfe00700 1 -- 172.21.15.1:0/2562894975 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdfc0103230 msgr2=0x7fdfc0199190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:38.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.069+0000 7fdfbfe00700 1 --2- 172.21.15.1:0/2562894975 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdfc0103230 0x7fdfc0199190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.069+0000 7fdfbfe00700 1 -- 172.21.15.1:0/2562894975 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfa8009520 con 0x7fdfc0108350 2024-02-02T15:38:38.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.069+0000 7fdfc5800700 1 --2- 172.21.15.1:0/2562894975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdfc019db30 0x7fdfc019dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:38:38.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.070+0000 7fdfbfe00700 1 --2- 172.21.15.1:0/2562894975 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdfc0108350 0x7fdfc01996d0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fdfa8009fd0 tx=0x7fdfa800fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:38.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.070+0000 7fdfbd600700 1 -- 172.21.15.1:0/2562894975 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa801c070 con 0x7fdfc0108350 2024-02-02T15:38:38.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.070+0000 7fdfbd600700 1 -- 172.21.15.1:0/2562894975 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdfa800bac0 con 0x7fdfc0108350 2024-02-02T15:38:38.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.070+0000 7fdfc7800700 1 -- 172.21.15.1:0/2562894975 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfc01a5de0 con 0x7fdfc0108350 2024-02-02T15:38:38.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.070+0000 7fdfbd600700 1 -- 172.21.15.1:0/2562894975 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa8004950 con 0x7fdfc0108350 2024-02-02T15:38:38.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.071+0000 7fdfc7800700 1 -- 172.21.15.1:0/2562894975 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfc01a6200 con 0x7fdfc0108350 2024-02-02T15:38:38.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.074+0000 7fdfbd600700 1 -- 172.21.15.1:0/2562894975 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7fdfa8020070 con 0x7fdfc0108350 2024-02-02T15:38:38.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.075+0000 7fdfdb36ab80 1 Processor -- start 2024-02-02T15:38:38.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.076+0000 7fdfdb36ab80 1 -- start start 2024-02-02T15:38:38.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.076+0000 7fdfdb36ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fd0c7eb010 0x55fd0c7eb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:38.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.076+0000 7fdfdb36ab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd0c7eb930 0x55fd0c7f62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:38.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.076+0000 7fdfdb36ab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fd0c7f67e0 0x55fd0c7f8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:38.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.076+0000 7fdfdb36ab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd0c75f4c0 con 0x55fd0c7eb930 2024-02-02T15:38:38.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.076+0000 7fdfdb36ab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd0c743aa0 con 0x55fd0c7f67e0 2024-02-02T15:38:38.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.076+0000 7fdfdb36ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd0c741460 con 0x55fd0c7eb010 2024-02-02T15:38:38.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.076+0000 7fdfbd600700 1 --2- 172.21.15.1:0/2562894975 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fdfa00a0710 0x7fdfa00a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:38.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.076+0000 7fdfc5800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fd0c7eb010 0x55fd0c7eb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:38.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.076+0000 7fdfc5800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fd0c7eb010 0x55fd0c7eb3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:53662/0 (socket says 172.21.15.1:53662) 2024-02-02T15:38:38.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.076+0000 7fdfc5800700 1 -- 172.21.15.1:0/3406526710 learned_addr learned my addr 172.21.15.1:0/3406526710 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:38:38.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.076+0000 7fdfc4e00700 1 --2- 172.21.15.1:0/3406526710 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd0c7eb930 0x55fd0c7f62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:38.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.076+0000 7fdfbfe00700 1 --2- 172.21.15.1:0/3406526710 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fd0c7f67e0 0x55fd0c7f8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:38.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.077+0000 7fdfbd600700 1 -- 172.21.15.1:0/2562894975 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fdfa80ccd70 con 0x7fdfc0108350 2024-02-02T15:38:38.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.077+0000 7fdfc5800700 1 --2- 172.21.15.1:0/2562894975 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fdfa00a0710 0x7fdfa00a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:38.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.077+0000 7fdfbfe00700 1 -- 172.21.15.1:0/3406526710 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fd0c7eb010 msgr2=0x55fd0c7eb3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:38.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.077+0000 7fdfbfe00700 1 --2- 172.21.15.1:0/3406526710 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fd0c7eb010 0x55fd0c7eb3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.077+0000 7fdfbfe00700 1 -- 172.21.15.1:0/3406526710 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd0c7eb930 msgr2=0x55fd0c7f62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:38.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.077+0000 7fdfbfe00700 1 --2- 172.21.15.1:0/3406526710 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd0c7eb930 0x55fd0c7f62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.077+0000 7fdfbfe00700 1 -- 172.21.15.1:0/3406526710 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd0c5be3b0 con 0x55fd0c7f67e0 2024-02-02T15:38:38.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.077+0000 7fdfc4e00700 1 --2- 172.21.15.1:0/3406526710 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd0c7eb930 0x55fd0c7f62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:38:38.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.077+0000 7fdfbfe00700 1 --2- 172.21.15.1:0/3406526710 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fd0c7f67e0 0x55fd0c7f8bd0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fdfa80ca0f0 tx=0x7fdfa8009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=714e43994d38d357 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:38.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.077+0000 7fdfb6000700 1 -- 172.21.15.1:0/3406526710 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa809a1e0 con 0x55fd0c7f67e0 2024-02-02T15:38:38.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.077+0000 7fdfb6000700 1 -- 172.21.15.1:0/3406526710 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdfa8099f80 con 0x55fd0c7f67e0 2024-02-02T15:38:38.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.077+0000 7fdfb6000700 1 -- 172.21.15.1:0/3406526710 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa8005040 con 0x55fd0c7f67e0 2024-02-02T15:38:38.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.078+0000 7fdfdb36ab80 1 -- 172.21.15.1:0/3406526710 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fd0c7f67e0 msgr2=0x55fd0c7f8bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:38.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.078+0000 7fdfc5800700 1 --2- 172.21.15.1:0/2562894975 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fdfa00a0710 0x7fdfa00a2bd0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fdfb8003fd0 tx=0x7fdfb8004b40 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:38.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.078+0000 7fdfdb36ab80 1 --2- 172.21.15.1:0/3406526710 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fd0c7f67e0 0x55fd0c7f8bd0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fdfa80ca0f0 tx=0x7fdfa8009260 comp rx=0 tx=0).stop 2024-02-02T15:38:38.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.078+0000 7fdfdb36ab80 1 -- 172.21.15.1:0/3406526710 shutdown_connections 2024-02-02T15:38:38.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.078+0000 7fdfdb36ab80 1 --2- 172.21.15.1:0/3406526710 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fd0c7f67e0 0x55fd0c7f8bd0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.078+0000 7fdfdb36ab80 1 --2- 172.21.15.1:0/3406526710 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fd0c7eb010 0x55fd0c7eb3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.078+0000 7fdfdb36ab80 1 --2- 172.21.15.1:0/3406526710 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd0c7eb930 0x55fd0c7f62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.078+0000 7fdfdb36ab80 1 -- 172.21.15.1:0/3406526710 >> 172.21.15.1:0/3406526710 conn(0x55fd0c74adb0 msgr2=0x55fd0c7e73b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:38.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.078+0000 7fdfdb36ab80 1 -- 172.21.15.1:0/3406526710 shutdown_connections 2024-02-02T15:38:38.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.078+0000 7fdfdb36ab80 1 -- 172.21.15.1:0/3406526710 wait complete. 2024-02-02T15:38:38.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.079+0000 7fdfdb36ab80 1 -- start start 2024-02-02T15:38:38.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.079+0000 7fdfdb36ab80 1 Processor -- start 2024-02-02T15:38:38.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.080+0000 7fdfdb36ab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd0c7eb010 0x55fd0c7f6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:38.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.080+0000 7fdfdb36ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fd0c7eb930 0x55fd0c7f7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:38.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.080+0000 7fdfdb36ab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fd0c7f67e0 0x55fd0c7f7640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:38.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.080+0000 7fdfdb36ab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd0c73cd50 con 0x55fd0c7eb010 2024-02-02T15:38:38.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.080+0000 7fdfdb36ab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd0c6d8d80 con 0x55fd0c7f67e0 2024-02-02T15:38:38.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.080+0000 7fdfdb36ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd0c6d8a20 con 0x55fd0c7eb930 2024-02-02T15:38:38.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.080+0000 7fdfdb36ab80 10 client.0 _ll_register_callbacks cb 0x55fd0c6da290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:38:38.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.080+0000 7fdfc5800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fd0c7f67e0 0x55fd0c7f7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:38.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.080+0000 7fdfbfe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fd0c7eb930 0x55fd0c7f7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:38.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.080+0000 7fdfc5800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fd0c7f67e0 0x55fd0c7f7640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42616/0 (socket says 172.21.15.1:42616) 2024-02-02T15:38:38.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.080+0000 7fdfbfe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fd0c7eb930 0x55fd0c7f7100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:53670/0 (socket says 172.21.15.1:53670) 2024-02-02T15:38:38.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.080+0000 7fdfc5800700 1 -- 172.21.15.1:0/2118061544 learned_addr learned my addr 172.21.15.1:0/2118061544 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:38:38.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.080+0000 7fdfc4e00700 1 --2- 172.21.15.1:0/2118061544 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd0c7eb010 0x55fd0c7f6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:38.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.081+0000 7fdfc5800700 1 -- 172.21.15.1:0/2118061544 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fd0c7eb930 msgr2=0x55fd0c7f7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:38.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.081+0000 7fdfc5800700 1 --2- 172.21.15.1:0/2118061544 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fd0c7eb930 0x55fd0c7f7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.081+0000 7fdfc5800700 1 -- 172.21.15.1:0/2118061544 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd0c7eb010 msgr2=0x55fd0c7f6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:38.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.081+0000 7fdfc5800700 1 --2- 172.21.15.1:0/2118061544 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd0c7eb010 0x55fd0c7f6bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.081+0000 7fdfc5800700 1 -- 172.21.15.1:0/2118061544 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfb0009920 con 0x55fd0c7f67e0 2024-02-02T15:38:38.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.081+0000 7fdfbfe00700 1 --2- 172.21.15.1:0/2118061544 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fd0c7eb930 0x55fd0c7f7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:38:38.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.081+0000 7fdfc5800700 1 --2- 172.21.15.1:0/2118061544 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fd0c7f67e0 0x55fd0c7f7640 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fdfb800ee30 tx=0x7fdfb8004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7cc71191ef8a9270 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:38.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.081+0000 7fdfaea00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:38:38.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.081+0000 7fdfaea00700 1 -- 172.21.15.1:0/2118061544 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb8022070 con 0x55fd0c7f67e0 2024-02-02T15:38:38.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.081+0000 7fdfaea00700 1 -- 172.21.15.1:0/2118061544 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fdfb80048c0 con 0x55fd0c7f67e0 2024-02-02T15:38:38.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.081+0000 7fdfdb36ab80 1 -- 172.21.15.1:0/2118061544 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fd0c7df0f0 con 0x55fd0c7f67e0 2024-02-02T15:38:38.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.082+0000 7fdfaea00700 1 -- 172.21.15.1:0/2118061544 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdfb800ee60 con 0x55fd0c7f67e0 2024-02-02T15:38:38.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.082+0000 7fdfdb36ab80 10 client.28712 fetch_fsmap learned FSMap version 51 2024-02-02T15:38:38.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.082+0000 7fdfdb36ab80 1 -- 172.21.15.1:0/2118061544 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x7fdfc01a6200 con 0x55fd0c7f67e0 2024-02-02T15:38:38.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.082+0000 7fdfaea00700 1 -- 172.21.15.1:0/2118061544 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fd0c7df0f0 con 0x55fd0c7f67e0 2024-02-02T15:38:38.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.082+0000 7fdfaea00700 1 -- 172.21.15.1:0/2118061544 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 51) v1 ==== 7474+0+0 (secure 0 0 0) 0x7fdfb801b450 con 0x55fd0c7f67e0 2024-02-02T15:38:38.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.082+0000 7fdfdb36ab80 10 client.28712 fetch_fsmap finished waiting for FSMap version 51 2024-02-02T15:38:38.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.082+0000 7fdfdb36ab80 10 client.28712 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-02-02T15:38:38.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.082+0000 7fdfdb36ab80 20 client.28712 populate_metadata read hostname 'smithi001' 2024-02-02T15:38:38.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.082+0000 7fdfdb36ab80 1 --2- 172.21.15.1:0/2118061544 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55fd0c805ef0 0x55fd0c808310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:38.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.082+0000 7fdfdb36ab80 4 client.28712 mds_command: new command op to 24518 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:38:38.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.082+0000 7fdfdb36ab80 1 -- 172.21.15.1:0/2118061544 --> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fd0c735dd0 con 0x55fd0c805ef0 2024-02-02T15:38:38.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.083+0000 7fdfc4e00700 1 --2- 172.21.15.1:0/2118061544 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55fd0c805ef0 0x55fd0c808310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:38.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.083+0000 7fdfc4e00700 1 --2- 172.21.15.1:0/2118061544 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55fd0c805ef0 0x55fd0c808310 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e598b74c83f4df42 server_cookie=f626b1fef448bf56 in_seq=0 out_seq=0 2024-02-02T15:38:38.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.083+0000 7fdfaea00700 10 client.28712 ms_handle_connect on v2:172.21.15.71:6836/628016199 2024-02-02T15:38:38.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.084+0000 7fdfaea00700 1 -- 172.21.15.1:0/2118061544 <== mds.2 v2:172.21.15.71:6836/628016199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55fd0c735dd0 con 0x55fd0c805ef0 2024-02-02T15:38:38.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.084+0000 7fdfaea00700 10 client.28712 handle_command_reply: tid=0 2024-02-02T15:38:38.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.084+0000 7fdfdb36ab80 1 client.28712 shutdown 2024-02-02T15:38:38.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.084+0000 7fdfdb36ab80 1 -- 172.21.15.1:0/2118061544 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fd0c7f67e0 msgr2=0x55fd0c7f7640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:38.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.084+0000 7fdfdb36ab80 1 --2- 172.21.15.1:0/2118061544 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fd0c7f67e0 0x55fd0c7f7640 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fdfb800ee30 tx=0x7fdfb8004ad0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.084+0000 7fdfdb36ab80 1 -- 172.21.15.1:0/2118061544 shutdown_connections 2024-02-02T15:38:38.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.084+0000 7fdfdb36ab80 1 --2- 172.21.15.1:0/2118061544 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fd0c7f67e0 0x55fd0c7f7640 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.084+0000 7fdfdb36ab80 1 --2- 172.21.15.1:0/2118061544 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd0c7eb010 0x55fd0c7f6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.084+0000 7fdfdb36ab80 1 --2- 172.21.15.1:0/2118061544 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55fd0c805ef0 0x55fd0c808310 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.084+0000 7fdfdb36ab80 1 --2- 172.21.15.1:0/2118061544 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fd0c7eb930 0x55fd0c7f7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.084+0000 7fdfaea00700 0 client.28712 ms_handle_reset on v2:172.21.15.71:6836/628016199 2024-02-02T15:38:38.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.084+0000 7fdfdb36ab80 1 -- 172.21.15.1:0/2118061544 >> 172.21.15.1:0/2118061544 conn(0x55fd0c74adb0 msgr2=0x55fd0c7e6fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:38.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.084+0000 7fdfdb36ab80 1 -- 172.21.15.1:0/2118061544 shutdown_connections 2024-02-02T15:38:38.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.084+0000 7fdfdb36ab80 1 -- 172.21.15.1:0/2118061544 wait complete. 2024-02-02T15:38:38.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.085+0000 7fdfdb36ab80 20 client.28712 trim_cache size 0 max 16384 2024-02-02T15:38:38.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.099+0000 7fdfdb36ab80 1 Processor -- start 2024-02-02T15:38:38.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.099+0000 7fdfdb36ab80 1 -- start start 2024-02-02T15:38:38.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.100+0000 7fdfdb36ab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd0c7eb010 0x55fd0c7fbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:38.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.100+0000 7fdfdb36ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fd0c7eb930 0x55fd0c8011b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:38.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.100+0000 7fdfdb36ab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fd0c7f67e0 0x55fd0c8016f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:38.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.100+0000 7fdfdb36ab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd0c5c06a0 con 0x55fd0c7eb010 2024-02-02T15:38:38.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.100+0000 7fdfdb36ab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd0c7e87e0 con 0x55fd0c7f67e0 2024-02-02T15:38:38.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.100+0000 7fdfdb36ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd0c7ec040 con 0x55fd0c7eb930 2024-02-02T15:38:38.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.100+0000 7fdfc4e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd0c7eb010 0x55fd0c7fbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:38.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.100+0000 7fdfc5800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fd0c7f67e0 0x55fd0c8016f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:38.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.100+0000 7fdfc5800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fd0c7f67e0 0x55fd0c8016f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42626/0 (socket says 172.21.15.1:42626) 2024-02-02T15:38:38.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.100+0000 7fdfc5800700 1 -- 172.21.15.1:0/2348901072 learned_addr learned my addr 172.21.15.1:0/2348901072 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:38:38.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.100+0000 7fdfc4e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd0c7eb010 0x55fd0c7fbbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:36896/0 (socket says 172.21.15.1:36896) 2024-02-02T15:38:38.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.100+0000 7fdfbfe00700 1 --2- 172.21.15.1:0/2348901072 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fd0c7eb930 0x55fd0c8011b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:38.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.101+0000 7fdfc5800700 1 -- 172.21.15.1:0/2348901072 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fd0c7eb930 msgr2=0x55fd0c8011b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:38.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.101+0000 7fdfc5800700 1 --2- 172.21.15.1:0/2348901072 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fd0c7eb930 0x55fd0c8011b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.101+0000 7fdfc5800700 1 -- 172.21.15.1:0/2348901072 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd0c7eb010 msgr2=0x55fd0c7fbbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:38.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.101+0000 7fdfc5800700 1 --2- 172.21.15.1:0/2348901072 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd0c7eb010 0x55fd0c7fbbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.101+0000 7fdfc5800700 1 -- 172.21.15.1:0/2348901072 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd0c56ab60 con 0x55fd0c7f67e0 2024-02-02T15:38:38.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.101+0000 7fdfc5800700 1 --2- 172.21.15.1:0/2348901072 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fd0c7f67e0 0x55fd0c8016f0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fdfb801b830 tx=0x7fdfb800e7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ead2f88bd639c31 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:38.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.101+0000 7fdfb6a00700 1 -- 172.21.15.1:0/2348901072 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb8022040 con 0x55fd0c7f67e0 2024-02-02T15:38:38.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.101+0000 7fdfb6a00700 1 -- 172.21.15.1:0/2348901072 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdfb800f040 con 0x55fd0c7f67e0 2024-02-02T15:38:38.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.101+0000 7fdfb6a00700 1 -- 172.21.15.1:0/2348901072 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb8027b80 con 0x55fd0c7f67e0 2024-02-02T15:38:38.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.102+0000 7fdfdb36ab80 1 -- 172.21.15.1:0/2348901072 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fd0c7f67e0 msgr2=0x55fd0c8016f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:38.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.102+0000 7fdfdb36ab80 1 --2- 172.21.15.1:0/2348901072 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fd0c7f67e0 0x55fd0c8016f0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fdfb801b830 tx=0x7fdfb800e7d0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.102+0000 7fdfdb36ab80 1 -- 172.21.15.1:0/2348901072 shutdown_connections 2024-02-02T15:38:38.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.102+0000 7fdfdb36ab80 1 --2- 172.21.15.1:0/2348901072 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fd0c7f67e0 0x55fd0c8016f0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.102+0000 7fdfdb36ab80 1 --2- 172.21.15.1:0/2348901072 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd0c7eb010 0x55fd0c7fbbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.102+0000 7fdfdb36ab80 1 --2- 172.21.15.1:0/2348901072 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fd0c7eb930 0x55fd0c8011b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.102+0000 7fdfdb36ab80 1 -- 172.21.15.1:0/2348901072 >> 172.21.15.1:0/2348901072 conn(0x55fd0c74adb0 msgr2=0x55fd0c7f0b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:38.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.102+0000 7fdfdb36ab80 1 -- 172.21.15.1:0/2348901072 shutdown_connections 2024-02-02T15:38:38.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.102+0000 7fdfdb36ab80 1 -- 172.21.15.1:0/2348901072 wait complete. 2024-02-02T15:38:38.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.102+0000 7fdfdb36ab80 1 -- start start 2024-02-02T15:38:38.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.103+0000 7fdfdb36ab80 1 Processor -- start 2024-02-02T15:38:38.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.103+0000 7fdfdb36ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fd0c7eb010 0x55fd0c7fb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:38.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.103+0000 7fdfdb36ab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd0c7eb930 0x55fd0c7fb920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:38.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.103+0000 7fdfdb36ab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fd0c7f67e0 0x55fd0c7f9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:38.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.103+0000 7fdfdb36ab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd0c7f9e20 con 0x55fd0c7eb930 2024-02-02T15:38:38.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.103+0000 7fdfdb36ab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd0c7f9f60 con 0x55fd0c7f67e0 2024-02-02T15:38:38.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.103+0000 7fdfdb36ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd0c7fa120 con 0x55fd0c7eb010 2024-02-02T15:38:38.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.103+0000 7fdfdb36ab80 10 client.0 _ll_register_callbacks cb 0x55fd0c6da290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:38:38.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.103+0000 7fdfc4e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fd0c7eb010 0x55fd0c7fb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:38.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.103+0000 7fdfc4e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fd0c7eb010 0x55fd0c7fb3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:53696/0 (socket says 172.21.15.1:53696) 2024-02-02T15:38:38.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.103+0000 7fdfbfe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd0c7eb930 0x55fd0c7fb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:38.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.103+0000 7fdfc4e00700 1 -- 172.21.15.1:0/3724476316 learned_addr learned my addr 172.21.15.1:0/3724476316 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:38:38.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.103+0000 7fdfc5800700 1 --2- 172.21.15.1:0/3724476316 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fd0c7f67e0 0x55fd0c7f9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:38.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.104+0000 7fdfc4e00700 1 -- 172.21.15.1:0/3724476316 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fd0c7f67e0 msgr2=0x55fd0c7f9780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:38.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.104+0000 7fdfc4e00700 1 --2- 172.21.15.1:0/3724476316 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fd0c7f67e0 0x55fd0c7f9780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.104+0000 7fdfc4e00700 1 -- 172.21.15.1:0/3724476316 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd0c7eb930 msgr2=0x55fd0c7fb920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:38.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.104+0000 7fdfc4e00700 1 --2- 172.21.15.1:0/3724476316 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd0c7eb930 0x55fd0c7fb920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.104+0000 7fdfc4e00700 1 -- 172.21.15.1:0/3724476316 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfc01a6200 con 0x55fd0c7eb010 2024-02-02T15:38:38.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.104+0000 7fdfc4e00700 1 --2- 172.21.15.1:0/3724476316 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fd0c7eb010 0x55fd0c7fb3e0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x55fd0c808bf0 tx=0x7fdfb001fc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7fb6edc986c4f45 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:38.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.104+0000 7fdfaea00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:38:38.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.104+0000 7fdfaea00700 1 -- 172.21.15.1:0/3724476316 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb0004520 con 0x55fd0c7eb010 2024-02-02T15:38:38.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.104+0000 7fdfdb36ab80 1 -- 172.21.15.1:0/3724476316 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdfa8099f80 con 0x55fd0c7eb010 2024-02-02T15:38:38.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.104+0000 7fdfaea00700 1 -- 172.21.15.1:0/3724476316 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fdfb00106b0 con 0x55fd0c7eb010 2024-02-02T15:38:38.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.104+0000 7fdfaea00700 1 -- 172.21.15.1:0/3724476316 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdfb00108b0 con 0x55fd0c7eb010 2024-02-02T15:38:38.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.105+0000 7fdfdb36ab80 10 client.28723 fetch_fsmap learned FSMap version 51 2024-02-02T15:38:38.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.105+0000 7fdfdb36ab80 1 -- 172.21.15.1:0/3724476316 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x55fd0c702ea0 con 0x55fd0c7eb010 2024-02-02T15:38:38.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.105+0000 7fdfaea00700 1 -- 172.21.15.1:0/3724476316 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdfa8099f80 con 0x55fd0c7eb010 2024-02-02T15:38:38.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.105+0000 7fdfaea00700 1 -- 172.21.15.1:0/3724476316 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 51) v1 ==== 7474+0+0 (secure 0 0 0) 0x7fdfb00273c0 con 0x55fd0c7eb010 2024-02-02T15:38:38.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.105+0000 7fdfdb36ab80 10 client.28723 fetch_fsmap finished waiting for FSMap version 51 2024-02-02T15:38:38.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.105+0000 7fdfdb36ab80 10 client.28723 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-02-02T15:38:38.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.105+0000 7fdfdb36ab80 20 client.28723 populate_metadata read hostname 'smithi001' 2024-02-02T15:38:38.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.105+0000 7fdfdb36ab80 1 --2- 172.21.15.1:0/3724476316 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55fd0c858b40 0x55fd0c85af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:38.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.105+0000 7fdfdb36ab80 4 client.28723 mds_command: new command op to 24518 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:38:38.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.105+0000 7fdfdb36ab80 1 -- 172.21.15.1:0/3724476316 --> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55fd0c6fbf50 con 0x55fd0c858b40 2024-02-02T15:38:38.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.105+0000 7fdfbfe00700 1 --2- 172.21.15.1:0/3724476316 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55fd0c858b40 0x55fd0c85af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:38.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.106+0000 7fdfbfe00700 1 --2- 172.21.15.1:0/3724476316 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55fd0c858b40 0x55fd0c85af60 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1578bb087c9bc1a2 server_cookie=30e848108005fbbc in_seq=0 out_seq=0 2024-02-02T15:38:38.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.106+0000 7fdfaea00700 10 client.28723 ms_handle_connect on v2:172.21.15.71:6836/628016199 2024-02-02T15:38:38.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.106+0000 7fdfaea00700 1 -- 172.21.15.1:0/3724476316 <== mds.2 v2:172.21.15.71:6836/628016199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55fd0c6fbf50 con 0x55fd0c858b40 2024-02-02T15:38:38.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.106+0000 7fdfaea00700 10 client.28723 handle_command_reply: tid=0 2024-02-02T15:38:38.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.106+0000 7fdfdb36ab80 1 client.28723 shutdown 2024-02-02T15:38:38.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.106+0000 7fdfdb36ab80 1 -- 172.21.15.1:0/3724476316 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fd0c7eb010 msgr2=0x55fd0c7fb3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:38.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.106+0000 7fdfdb36ab80 1 --2- 172.21.15.1:0/3724476316 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fd0c7eb010 0x55fd0c7fb3e0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x55fd0c808bf0 tx=0x7fdfb001fc40 comp rx=0 tx=0).stop 2024-02-02T15:38:38.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.106+0000 7fdfdb36ab80 1 -- 172.21.15.1:0/3724476316 shutdown_connections 2024-02-02T15:38:38.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.106+0000 7fdfdb36ab80 1 --2- 172.21.15.1:0/3724476316 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fd0c7f67e0 0x55fd0c7f9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.106+0000 7fdfdb36ab80 1 --2- 172.21.15.1:0/3724476316 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55fd0c858b40 0x55fd0c85af60 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.106+0000 7fdfaea00700 0 client.28723 ms_handle_reset on v2:172.21.15.71:6836/628016199 2024-02-02T15:38:38.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.106+0000 7fdfdb36ab80 1 --2- 172.21.15.1:0/3724476316 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fd0c7eb010 0x55fd0c7fb3e0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.106+0000 7fdfdb36ab80 1 --2- 172.21.15.1:0/3724476316 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd0c7eb930 0x55fd0c7fb920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.106+0000 7fdfdb36ab80 1 -- 172.21.15.1:0/3724476316 >> 172.21.15.1:0/3724476316 conn(0x55fd0c74adb0 msgr2=0x55fd0c7e6e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:38.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.106+0000 7fdfdb36ab80 1 -- 172.21.15.1:0/3724476316 shutdown_connections 2024-02-02T15:38:38.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.106+0000 7fdfdb36ab80 1 -- 172.21.15.1:0/3724476316 wait complete. 2024-02-02T15:38:38.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.107+0000 7fdfdb36ab80 20 client.28723 trim_cache size 0 max 16384 2024-02-02T15:38:38.109 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:38:38.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.107+0000 7fdfc7800700 1 -- 172.21.15.1:0/2562894975 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fdfa00a0710 msgr2=0x7fdfa00a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:38.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.107+0000 7fdfc7800700 1 --2- 172.21.15.1:0/2562894975 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fdfa00a0710 0x7fdfa00a2bd0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fdfb8003fd0 tx=0x7fdfb8004b40 comp rx=0 tx=0).stop 2024-02-02T15:38:38.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.107+0000 7fdfc7800700 1 -- 172.21.15.1:0/2562894975 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdfc0108350 msgr2=0x7fdfc01996d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:38.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.107+0000 7fdfc7800700 1 --2- 172.21.15.1:0/2562894975 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdfc0108350 0x7fdfc01996d0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fdfa8009fd0 tx=0x7fdfa800fa00 comp rx=0 tx=0).stop 2024-02-02T15:38:38.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.107+0000 7fdfc7800700 1 -- 172.21.15.1:0/2562894975 shutdown_connections 2024-02-02T15:38:38.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.107+0000 7fdfc7800700 1 --2- 172.21.15.1:0/2562894975 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fdfa00a0710 0x7fdfa00a2bd0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.107+0000 7fdfc7800700 1 --2- 172.21.15.1:0/2562894975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdfc019db30 0x7fdfc019dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.107+0000 7fdfc7800700 1 --2- 172.21.15.1:0/2562894975 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdfc0103230 0x7fdfc0199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.107+0000 7fdfc7800700 1 --2- 172.21.15.1:0/2562894975 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdfc0108350 0x7fdfc01996d0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.107+0000 7fdfc7800700 1 -- 172.21.15.1:0/2562894975 >> 172.21.15.1:0/2562894975 conn(0x7fdfc00fea70 msgr2=0x7fdfc00ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:38.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.107+0000 7fdfc7800700 1 -- 172.21.15.1:0/2562894975 shutdown_connections 2024-02-02T15:38:38.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.107+0000 7fdfc7800700 1 -- 172.21.15.1:0/2562894975 wait complete. 2024-02-02T15:38:38.126 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-02T15:38:38.267 INFO:tasks.workunit.client.0.smithi001.stdout: 1 642172 5.53 MB/sec execute 492 sec latency 4269.002 ms 2024-02-02T15:38:38.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.350+0000 7f5558e00700 1 -- 172.21.15.1:0/256795549 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5554106110 msgr2=0x7f555410dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:38.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.350+0000 7f5558e00700 1 --2- 172.21.15.1:0/256795549 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5554106110 0x7f555410dc90 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f554c00b0a0 tx=0x7f554c00b3b0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.350+0000 7f5558e00700 1 -- 172.21.15.1:0/256795549 shutdown_connections 2024-02-02T15:38:38.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.350+0000 7f5558e00700 1 --2- 172.21.15.1:0/256795549 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5554106110 0x7f555410dc90 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.350+0000 7f5558e00700 1 --2- 172.21.15.1:0/256795549 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5554100ff0 0x7f55541013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.350+0000 7f5558e00700 1 --2- 172.21.15.1:0/256795549 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f55541019a0 0x7f55541059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.350+0000 7f5558e00700 1 -- 172.21.15.1:0/256795549 >> 172.21.15.1:0/256795549 conn(0x7f55540fc810 msgr2=0x7f55540fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:38.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.350+0000 7f5558e00700 1 -- 172.21.15.1:0/256795549 shutdown_connections 2024-02-02T15:38:38.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.350+0000 7f5558e00700 1 -- 172.21.15.1:0/256795549 wait complete. 2024-02-02T15:38:38.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.351+0000 7f5558e00700 1 Processor -- start 2024-02-02T15:38:38.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.351+0000 7f5558e00700 1 -- start start 2024-02-02T15:38:38.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.352+0000 7f5558e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5554100ff0 0x7f5554198fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:38.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.352+0000 7f5558e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5554106110 0x7f55541994f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:38.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.352+0000 7f5558e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f555419d950 0x7f555419ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:38.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.352+0000 7f5558e00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f555419e3d0 con 0x7f555419d950 2024-02-02T15:38:38.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.352+0000 7f5558e00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f555419e540 con 0x7f5554106110 2024-02-02T15:38:38.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.352+0000 7f5558e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f555419e700 con 0x7f5554100ff0 2024-02-02T15:38:38.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.352+0000 7f5551e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5554100ff0 0x7f5554198fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:38.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.352+0000 7f5551e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5554100ff0 0x7f5554198fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:53708/0 (socket says 172.21.15.1:53708) 2024-02-02T15:38:38.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.352+0000 7f5551e00700 1 -- 172.21.15.1:0/62673891 learned_addr learned my addr 172.21.15.1:0/62673891 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:38:38.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.352+0000 7f5552800700 1 --2- 172.21.15.1:0/62673891 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f555419d950 0x7f555419ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:38.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.352+0000 7f5551400700 1 --2- 172.21.15.1:0/62673891 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5554106110 0x7f55541994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:38.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.352+0000 7f5551e00700 1 -- 172.21.15.1:0/62673891 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5554106110 msgr2=0x7f55541994f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:38.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.353+0000 7f5551e00700 1 --2- 172.21.15.1:0/62673891 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5554106110 0x7f55541994f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.353+0000 7f5551e00700 1 -- 172.21.15.1:0/62673891 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f555419d950 msgr2=0x7f555419ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:38.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.353+0000 7f5551e00700 1 --2- 172.21.15.1:0/62673891 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f555419d950 0x7f555419ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.353+0000 7f5551e00700 1 -- 172.21.15.1:0/62673891 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f554c009d00 con 0x7f5554100ff0 2024-02-02T15:38:38.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.353+0000 7f5551400700 1 --2- 172.21.15.1:0/62673891 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5554106110 0x7f55541994f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:38:38.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.353+0000 7f5551e00700 1 --2- 172.21.15.1:0/62673891 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5554100ff0 0x7f5554198fb0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f554400e9a0 tx=0x7f554400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:38.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.353+0000 7f554aa00700 1 -- 172.21.15.1:0/62673891 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5544009790 con 0x7f5554100ff0 2024-02-02T15:38:38.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.353+0000 7f5558e00700 1 -- 172.21.15.1:0/62673891 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55541a5c60 con 0x7f5554100ff0 2024-02-02T15:38:38.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.353+0000 7f554aa00700 1 -- 172.21.15.1:0/62673891 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f554400cbd0 con 0x7f5554100ff0 2024-02-02T15:38:38.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.353+0000 7f554aa00700 1 -- 172.21.15.1:0/62673891 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5544018680 con 0x7f5554100ff0 2024-02-02T15:38:38.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.353+0000 7f5558e00700 1 -- 172.21.15.1:0/62673891 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55541a6140 con 0x7f5554100ff0 2024-02-02T15:38:38.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.357+0000 7f554aa00700 1 -- 172.21.15.1:0/62673891 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f554400cd40 con 0x7f5554100ff0 2024-02-02T15:38:38.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.358+0000 7f556c8b6b80 1 Processor -- start 2024-02-02T15:38:38.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.359+0000 7f556c8b6b80 1 -- start start 2024-02-02T15:38:38.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.359+0000 7f554aa00700 1 --2- 172.21.15.1:0/62673891 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f55340a0690 0x7f55340a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:38.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.359+0000 7f556c8b6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609e6445010 0x5609e64453f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:38.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.359+0000 7f556c8b6b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5609e6445930 0x5609e64502a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:38.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.359+0000 7f554aa00700 1 -- 172.21.15.1:0/62673891 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f5544014070 con 0x7f5554100ff0 2024-02-02T15:38:38.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.359+0000 7f556c8b6b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5609e64507e0 0x5609e6452bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:38.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.359+0000 7f556c8b6b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609e63b94c0 con 0x5609e64507e0 2024-02-02T15:38:38.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.359+0000 7f556c8b6b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609e639daa0 con 0x5609e6445930 2024-02-02T15:38:38.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.359+0000 7f556c8b6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609e639b460 con 0x5609e6445010 2024-02-02T15:38:38.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.359+0000 7f5552800700 1 --2- 172.21.15.1:0/62673891 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f55340a0690 0x7f55340a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:38.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.359+0000 7f5551400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5609e6445930 0x5609e64502a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:38.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.359+0000 7f5551e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609e6445010 0x5609e64453f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:38.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.359+0000 7f5551400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5609e6445930 0x5609e64502a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42674/0 (socket says 172.21.15.1:42674) 2024-02-02T15:38:38.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.359+0000 7f5551400700 1 -- 172.21.15.1:0/2869494957 learned_addr learned my addr 172.21.15.1:0/2869494957 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:38:38.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.360+0000 7f5552800700 1 --2- 172.21.15.1:0/2869494957 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5609e64507e0 0x5609e6452bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:38.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.360+0000 7f5551400700 1 -- 172.21.15.1:0/2869494957 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609e6445010 msgr2=0x5609e64453f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:38.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.360+0000 7f5551400700 1 --2- 172.21.15.1:0/2869494957 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609e6445010 0x5609e64453f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.360+0000 7f5551400700 1 -- 172.21.15.1:0/2869494957 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5609e64507e0 msgr2=0x5609e6452bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:38.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.360+0000 7f5551400700 1 --2- 172.21.15.1:0/2869494957 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5609e64507e0 0x5609e6452bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.360+0000 7f5551400700 1 -- 172.21.15.1:0/2869494957 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609e62183b0 con 0x5609e6445930 2024-02-02T15:38:38.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.360+0000 7f5552800700 1 --2- 172.21.15.1:0/2869494957 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5609e64507e0 0x5609e6452bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:38:38.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.360+0000 7f5551e00700 1 --2- 172.21.15.1:0/2869494957 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609e6445010 0x5609e64453f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:38:38.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.360+0000 7f5551400700 1 --2- 172.21.15.1:0/2869494957 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5609e6445930 0x5609e64502a0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f553c009d50 tx=0x7f553c0106b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc2b4e07cd7618a4 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:38.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.360+0000 7f5552800700 1 --2- 172.21.15.1:0/62673891 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f55340a0690 0x7f55340a2b50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f554c009cd0 tx=0x7f554c009c60 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:38.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.360+0000 7f5543400700 1 -- 172.21.15.1:0/2869494957 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f553c009ea0 con 0x5609e6445930 2024-02-02T15:38:38.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.360+0000 7f5543400700 1 -- 172.21.15.1:0/2869494957 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f553c015070 con 0x5609e6445930 2024-02-02T15:38:38.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.360+0000 7f5543400700 1 -- 172.21.15.1:0/2869494957 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f553c003df0 con 0x5609e6445930 2024-02-02T15:38:38.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.361+0000 7f556c8b6b80 1 -- 172.21.15.1:0/2869494957 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5609e6445930 msgr2=0x5609e64502a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:38.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.361+0000 7f556c8b6b80 1 --2- 172.21.15.1:0/2869494957 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5609e6445930 0x5609e64502a0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f553c009d50 tx=0x7f553c0106b0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.361+0000 7f556c8b6b80 1 -- 172.21.15.1:0/2869494957 shutdown_connections 2024-02-02T15:38:38.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.361+0000 7f556c8b6b80 1 --2- 172.21.15.1:0/2869494957 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5609e64507e0 0x5609e6452bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.361+0000 7f556c8b6b80 1 --2- 172.21.15.1:0/2869494957 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609e6445010 0x5609e64453f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.361+0000 7f556c8b6b80 1 --2- 172.21.15.1:0/2869494957 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5609e6445930 0x5609e64502a0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.361+0000 7f556c8b6b80 1 -- 172.21.15.1:0/2869494957 >> 172.21.15.1:0/2869494957 conn(0x5609e63a4db0 msgr2=0x5609e64413b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:38.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.361+0000 7f556c8b6b80 1 -- 172.21.15.1:0/2869494957 shutdown_connections 2024-02-02T15:38:38.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.361+0000 7f556c8b6b80 1 -- 172.21.15.1:0/2869494957 wait complete. 2024-02-02T15:38:38.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.362+0000 7f556c8b6b80 1 -- start start 2024-02-02T15:38:38.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.362+0000 7f556c8b6b80 1 Processor -- start 2024-02-02T15:38:38.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.363+0000 7f556c8b6b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5609e6445010 0x5609e64501c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:38.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.363+0000 7f556c8b6b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5609e6445930 0x5609e644d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:38.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.363+0000 7f556c8b6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609e64507e0 0x5609e644d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:38.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.363+0000 7f556c8b6b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609e6396d50 con 0x5609e6445010 2024-02-02T15:38:38.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.363+0000 7f556c8b6b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609e6332d80 con 0x5609e6445930 2024-02-02T15:38:38.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.363+0000 7f556c8b6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609e6332a20 con 0x5609e64507e0 2024-02-02T15:38:38.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.363+0000 7f556c8b6b80 10 client.0 _ll_register_callbacks cb 0x5609e6334290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:38:38.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.363+0000 7f5551400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5609e6445930 0x5609e644d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:38.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.363+0000 7f5551e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5609e6445010 0x5609e64501c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:38.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.363+0000 7f5551400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5609e6445930 0x5609e644d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42680/0 (socket says 172.21.15.1:42680) 2024-02-02T15:38:38.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.363+0000 7f5551400700 1 -- 172.21.15.1:0/4267857650 learned_addr learned my addr 172.21.15.1:0/4267857650 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:38:38.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.363+0000 7f5552800700 1 --2- 172.21.15.1:0/4267857650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609e64507e0 0x5609e644d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:38.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.363+0000 7f5551400700 1 -- 172.21.15.1:0/4267857650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609e64507e0 msgr2=0x5609e644d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:38.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.363+0000 7f5551400700 1 --2- 172.21.15.1:0/4267857650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609e64507e0 0x5609e644d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.363+0000 7f5551400700 1 -- 172.21.15.1:0/4267857650 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5609e6445010 msgr2=0x5609e64501c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:38.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.363+0000 7f5551400700 1 --2- 172.21.15.1:0/4267857650 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5609e6445010 0x5609e64501c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.363+0000 7f5551400700 1 -- 172.21.15.1:0/4267857650 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55541a6140 con 0x5609e6445930 2024-02-02T15:38:38.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.364+0000 7f5551400700 1 --2- 172.21.15.1:0/4267857650 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5609e6445930 0x5609e644d3d0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f553c000c00 tx=0x7f553c005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92cf4f9005a611c2 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:38.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.364+0000 7f553be00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:38:38.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.364+0000 7f553be00700 1 -- 172.21.15.1:0/4267857650 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f553c004950 con 0x5609e6445930 2024-02-02T15:38:38.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.364+0000 7f556c8b6b80 1 -- 172.21.15.1:0/4267857650 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5609e64390f0 con 0x5609e6445930 2024-02-02T15:38:38.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.364+0000 7f553be00700 1 -- 172.21.15.1:0/4267857650 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f553c017070 con 0x5609e6445930 2024-02-02T15:38:38.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.364+0000 7f553be00700 1 -- 172.21.15.1:0/4267857650 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f553c009ea0 con 0x5609e6445930 2024-02-02T15:38:38.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.364+0000 7f556c8b6b80 10 client.28730 fetch_fsmap learned FSMap version 51 2024-02-02T15:38:38.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.364+0000 7f556c8b6b80 1 -- 172.21.15.1:0/4267857650 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x7f553c009950 con 0x5609e6445930 2024-02-02T15:38:38.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.364+0000 7f553be00700 1 -- 172.21.15.1:0/4267857650 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5609e64390f0 con 0x5609e6445930 2024-02-02T15:38:38.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.364+0000 7f553be00700 1 -- 172.21.15.1:0/4267857650 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 51) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f553c023070 con 0x5609e6445930 2024-02-02T15:38:38.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.365+0000 7f556c8b6b80 10 client.28730 fetch_fsmap finished waiting for FSMap version 51 2024-02-02T15:38:38.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.365+0000 7f556c8b6b80 10 client.28730 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-02T15:38:38.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.365+0000 7f556c8b6b80 20 client.28730 populate_metadata read hostname 'smithi001' 2024-02-02T15:38:38.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.365+0000 7f556c8b6b80 1 --2- 172.21.15.1:0/4267857650 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x5609e64600a0 0x5609e64624c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:38.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.365+0000 7f556c8b6b80 4 client.28730 mds_command: new command op to 24529 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:38:38.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.365+0000 7f556c8b6b80 1 -- 172.21.15.1:0/4267857650 --> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5609e638fdd0 con 0x5609e64600a0 2024-02-02T15:38:38.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.365+0000 7f5551e00700 1 --2- 172.21.15.1:0/4267857650 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x5609e64600a0 0x5609e64624c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:38.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.365+0000 7f5551e00700 1 --2- 172.21.15.1:0/4267857650 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x5609e64600a0 0x5609e64624c0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6891c30350ee9b5f server_cookie=bc7ba267bfdaf897 in_seq=0 out_seq=0 2024-02-02T15:38:38.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.365+0000 7f553be00700 10 client.28730 ms_handle_connect on v2:172.21.15.142:6838/321675421 2024-02-02T15:38:38.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.366+0000 7f553be00700 1 -- 172.21.15.1:0/4267857650 <== mds.1 v2:172.21.15.142:6838/321675421 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5609e638fdd0 con 0x5609e64600a0 2024-02-02T15:38:38.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.366+0000 7f553be00700 10 client.28730 handle_command_reply: tid=0 2024-02-02T15:38:38.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.366+0000 7f556c8b6b80 1 client.28730 shutdown 2024-02-02T15:38:38.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.366+0000 7f556c8b6b80 1 -- 172.21.15.1:0/4267857650 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5609e6445930 msgr2=0x5609e644d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:38.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.366+0000 7f556c8b6b80 1 --2- 172.21.15.1:0/4267857650 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5609e6445930 0x5609e644d3d0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f553c000c00 tx=0x7f553c005270 comp rx=0 tx=0).stop 2024-02-02T15:38:38.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.366+0000 7f556c8b6b80 1 -- 172.21.15.1:0/4267857650 shutdown_connections 2024-02-02T15:38:38.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.366+0000 7f556c8b6b80 1 --2- 172.21.15.1:0/4267857650 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x5609e64600a0 0x5609e64624c0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.366+0000 7f556c8b6b80 1 --2- 172.21.15.1:0/4267857650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609e64507e0 0x5609e644d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.366+0000 7f556c8b6b80 1 --2- 172.21.15.1:0/4267857650 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5609e6445010 0x5609e64501c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.366+0000 7f556c8b6b80 1 --2- 172.21.15.1:0/4267857650 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5609e6445930 0x5609e644d3d0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.366+0000 7f556c8b6b80 1 -- 172.21.15.1:0/4267857650 >> 172.21.15.1:0/4267857650 conn(0x5609e63a4db0 msgr2=0x5609e6440fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:38.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.366+0000 7f553be00700 0 client.28730 ms_handle_reset on v2:172.21.15.142:6838/321675421 2024-02-02T15:38:38.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.366+0000 7f556c8b6b80 1 -- 172.21.15.1:0/4267857650 shutdown_connections 2024-02-02T15:38:38.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.366+0000 7f556c8b6b80 1 -- 172.21.15.1:0/4267857650 wait complete. 2024-02-02T15:38:38.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.367+0000 7f556c8b6b80 20 client.28730 trim_cache size 0 max 16384 2024-02-02T15:38:38.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.381+0000 7f556c8b6b80 1 Processor -- start 2024-02-02T15:38:38.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.381+0000 7f556c8b6b80 1 -- start start 2024-02-02T15:38:38.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.381+0000 7f556c8b6b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5609e6445010 0x5609e644d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:38.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.381+0000 7f556c8b6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609e6445930 0x5609e644fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:38.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.381+0000 7f556c8b6b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5609e64507e0 0x5609e6447840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:38.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.381+0000 7f556c8b6b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609e621a6a0 con 0x5609e64507e0 2024-02-02T15:38:38.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.381+0000 7f556c8b6b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609e6449270 con 0x5609e6445010 2024-02-02T15:38:38.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.381+0000 7f556c8b6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609e644dfc0 con 0x5609e6445930 2024-02-02T15:38:38.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.382+0000 7f5551e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5609e6445010 0x5609e644d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:38.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.382+0000 7f5551e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5609e6445010 0x5609e644d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42688/0 (socket says 172.21.15.1:42688) 2024-02-02T15:38:38.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.382+0000 7f5551e00700 1 -- 172.21.15.1:0/1381956172 learned_addr learned my addr 172.21.15.1:0/1381956172 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:38:38.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.382+0000 7f5551400700 1 --2- 172.21.15.1:0/1381956172 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609e6445930 0x5609e644fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:38.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.382+0000 7f5552800700 1 --2- 172.21.15.1:0/1381956172 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5609e64507e0 0x5609e6447840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:38.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.382+0000 7f5551e00700 1 -- 172.21.15.1:0/1381956172 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609e6445930 msgr2=0x5609e644fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:38.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.382+0000 7f5551e00700 1 --2- 172.21.15.1:0/1381956172 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609e6445930 0x5609e644fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.382+0000 7f5551e00700 1 -- 172.21.15.1:0/1381956172 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5609e64507e0 msgr2=0x5609e6447840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:38.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.382+0000 7f5551e00700 1 --2- 172.21.15.1:0/1381956172 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5609e64507e0 0x5609e6447840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.382+0000 7f5551e00700 1 -- 172.21.15.1:0/1381956172 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5609e61c4b60 con 0x5609e6445010 2024-02-02T15:38:38.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.382+0000 7f5551e00700 1 --2- 172.21.15.1:0/1381956172 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5609e6445010 0x5609e644d380 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f5544095f50 tx=0x7f554401a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54f98783407ad446 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:38.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.383+0000 7f5543e00700 1 -- 172.21.15.1:0/1381956172 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55440994e0 con 0x5609e6445010 2024-02-02T15:38:38.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.383+0000 7f5543e00700 1 -- 172.21.15.1:0/1381956172 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f554409d310 con 0x5609e6445010 2024-02-02T15:38:38.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.383+0000 7f5543e00700 1 -- 172.21.15.1:0/1381956172 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5544099740 con 0x5609e6445010 2024-02-02T15:38:38.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.383+0000 7f556c8b6b80 1 -- 172.21.15.1:0/1381956172 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5609e6445010 msgr2=0x5609e644d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:38.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.383+0000 7f556c8b6b80 1 --2- 172.21.15.1:0/1381956172 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5609e6445010 0x5609e644d380 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f5544095f50 tx=0x7f554401a040 comp rx=0 tx=0).stop 2024-02-02T15:38:38.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.383+0000 7f556c8b6b80 1 -- 172.21.15.1:0/1381956172 shutdown_connections 2024-02-02T15:38:38.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.383+0000 7f556c8b6b80 1 --2- 172.21.15.1:0/1381956172 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5609e64507e0 0x5609e6447840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.383+0000 7f556c8b6b80 1 --2- 172.21.15.1:0/1381956172 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5609e6445010 0x5609e644d380 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.383+0000 7f556c8b6b80 1 --2- 172.21.15.1:0/1381956172 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609e6445930 0x5609e644fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.383+0000 7f556c8b6b80 1 -- 172.21.15.1:0/1381956172 >> 172.21.15.1:0/1381956172 conn(0x5609e63a4db0 msgr2=0x5609e644cc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:38.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.383+0000 7f556c8b6b80 1 -- 172.21.15.1:0/1381956172 shutdown_connections 2024-02-02T15:38:38.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.384+0000 7f556c8b6b80 1 -- 172.21.15.1:0/1381956172 wait complete. 2024-02-02T15:38:38.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.384+0000 7f556c8b6b80 1 -- start start 2024-02-02T15:38:38.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.384+0000 7f556c8b6b80 1 Processor -- start 2024-02-02T15:38:38.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.385+0000 7f556c8b6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609e6445010 0x5609e6457200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:38.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.385+0000 7f556c8b6b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5609e6445930 0x5609e64554f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:38.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.385+0000 7f556c8b6b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5609e64507e0 0x5609e6455a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:38.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.385+0000 7f556c8b6b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609e6456120 con 0x5609e6445930 2024-02-02T15:38:38.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.385+0000 7f556c8b6b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609e6456290 con 0x5609e64507e0 2024-02-02T15:38:38.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.385+0000 7f556c8b6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5609e64b1b90 con 0x5609e6445010 2024-02-02T15:38:38.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.385+0000 7f556c8b6b80 10 client.0 _ll_register_callbacks cb 0x5609e6334290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:38:38.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.385+0000 7f5551e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609e6445010 0x5609e6457200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:38.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.385+0000 7f5552800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5609e64507e0 0x5609e6455a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:38.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.385+0000 7f5551400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5609e6445930 0x5609e64554f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:38.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.385+0000 7f5551e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609e6445010 0x5609e6457200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:53744/0 (socket says 172.21.15.1:53744) 2024-02-02T15:38:38.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.385+0000 7f5552800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5609e64507e0 0x5609e6455a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42704/0 (socket says 172.21.15.1:42704) 2024-02-02T15:38:38.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.385+0000 7f5551e00700 1 -- 172.21.15.1:0/1676337701 learned_addr learned my addr 172.21.15.1:0/1676337701 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:38:38.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.385+0000 7f5552800700 1 -- 172.21.15.1:0/1676337701 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609e6445010 msgr2=0x5609e6457200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:38.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.385+0000 7f5552800700 1 --2- 172.21.15.1:0/1676337701 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609e6445010 0x5609e6457200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.385+0000 7f5552800700 1 -- 172.21.15.1:0/1676337701 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5609e6445930 msgr2=0x5609e64554f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:38.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.385+0000 7f5552800700 1 --2- 172.21.15.1:0/1676337701 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5609e6445930 0x5609e64554f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.385+0000 7f5552800700 1 -- 172.21.15.1:0/1676337701 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55440cae70 con 0x5609e64507e0 2024-02-02T15:38:38.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.385+0000 7f5551e00700 1 --2- 172.21.15.1:0/1676337701 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609e6445010 0x5609e6457200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:38:38.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.386+0000 7f5552800700 1 --2- 172.21.15.1:0/1676337701 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5609e64507e0 0x5609e6455a50 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f554c0062a0 tx=0x7f554c0099e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bdd485351f9bb1e0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:38.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.386+0000 7f553be00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:38:38.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.386+0000 7f553be00700 1 -- 172.21.15.1:0/1676337701 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f554c00e040 con 0x5609e64507e0 2024-02-02T15:38:38.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.386+0000 7f556c8b6b80 1 -- 172.21.15.1:0/1676337701 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f553c015070 con 0x5609e64507e0 2024-02-02T15:38:38.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.386+0000 7f553be00700 1 -- 172.21.15.1:0/1676337701 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f554c00bd30 con 0x5609e64507e0 2024-02-02T15:38:38.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.386+0000 7f553be00700 1 -- 172.21.15.1:0/1676337701 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f554c033460 con 0x5609e64507e0 2024-02-02T15:38:38.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.386+0000 7f556c8b6b80 10 client.28736 fetch_fsmap learned FSMap version 51 2024-02-02T15:38:38.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.386+0000 7f556c8b6b80 1 -- 172.21.15.1:0/1676337701 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x5609e635cea0 con 0x5609e64507e0 2024-02-02T15:38:38.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.386+0000 7f553be00700 1 -- 172.21.15.1:0/1676337701 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f553c015070 con 0x5609e64507e0 2024-02-02T15:38:38.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.386+0000 7f553be00700 1 -- 172.21.15.1:0/1676337701 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 51) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f554c00c170 con 0x5609e64507e0 2024-02-02T15:38:38.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.387+0000 7f556c8b6b80 10 client.28736 fetch_fsmap finished waiting for FSMap version 51 2024-02-02T15:38:38.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.387+0000 7f556c8b6b80 10 client.28736 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-02T15:38:38.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.387+0000 7f556c8b6b80 20 client.28736 populate_metadata read hostname 'smithi001' 2024-02-02T15:38:38.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.387+0000 7f556c8b6b80 1 --2- 172.21.15.1:0/1676337701 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x5609e64b3230 0x5609e64b5650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:38.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.387+0000 7f556c8b6b80 4 client.28736 mds_command: new command op to 24529 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:38:38.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.387+0000 7f556c8b6b80 1 -- 172.21.15.1:0/1676337701 --> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5609e6355f50 con 0x5609e64b3230 2024-02-02T15:38:38.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.387+0000 7f5551e00700 1 --2- 172.21.15.1:0/1676337701 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x5609e64b3230 0x5609e64b5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:38.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.387+0000 7f5551e00700 1 --2- 172.21.15.1:0/1676337701 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x5609e64b3230 0x5609e64b5650 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a2735c0c7d75b4f0 server_cookie=7f8c6ea23b36ed8d in_seq=0 out_seq=0 2024-02-02T15:38:38.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.387+0000 7f553be00700 10 client.28736 ms_handle_connect on v2:172.21.15.142:6838/321675421 2024-02-02T15:38:38.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.387+0000 7f553be00700 1 -- 172.21.15.1:0/1676337701 <== mds.1 v2:172.21.15.142:6838/321675421 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5609e6355f50 con 0x5609e64b3230 2024-02-02T15:38:38.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.387+0000 7f553be00700 10 client.28736 handle_command_reply: tid=0 2024-02-02T15:38:38.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.387+0000 7f556c8b6b80 1 client.28736 shutdown 2024-02-02T15:38:38.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.388+0000 7f556c8b6b80 1 -- 172.21.15.1:0/1676337701 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5609e64507e0 msgr2=0x5609e6455a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:38.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.388+0000 7f556c8b6b80 1 --2- 172.21.15.1:0/1676337701 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5609e64507e0 0x5609e6455a50 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f554c0062a0 tx=0x7f554c0099e0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.388+0000 7f556c8b6b80 1 -- 172.21.15.1:0/1676337701 shutdown_connections 2024-02-02T15:38:38.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.388+0000 7f556c8b6b80 1 --2- 172.21.15.1:0/1676337701 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x5609e64b3230 0x5609e64b5650 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.388+0000 7f556c8b6b80 1 --2- 172.21.15.1:0/1676337701 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5609e64507e0 0x5609e6455a50 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.388+0000 7f556c8b6b80 1 --2- 172.21.15.1:0/1676337701 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5609e6445010 0x5609e6457200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.388+0000 7f556c8b6b80 1 --2- 172.21.15.1:0/1676337701 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5609e6445930 0x5609e64554f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.388+0000 7f553be00700 0 client.28736 ms_handle_reset on v2:172.21.15.142:6838/321675421 2024-02-02T15:38:38.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.388+0000 7f556c8b6b80 1 -- 172.21.15.1:0/1676337701 >> 172.21.15.1:0/1676337701 conn(0x5609e63a4db0 msgr2=0x5609e6440e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:38.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.388+0000 7f556c8b6b80 1 -- 172.21.15.1:0/1676337701 shutdown_connections 2024-02-02T15:38:38.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.388+0000 7f556c8b6b80 1 -- 172.21.15.1:0/1676337701 wait complete. 2024-02-02T15:38:38.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.388+0000 7f556c8b6b80 20 client.28736 trim_cache size 0 max 16384 2024-02-02T15:38:38.391 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:38:38.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.389+0000 7f5558e00700 1 -- 172.21.15.1:0/62673891 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f55340a0690 msgr2=0x7f55340a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:38.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.389+0000 7f5558e00700 1 --2- 172.21.15.1:0/62673891 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f55340a0690 0x7f55340a2b50 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f554c009cd0 tx=0x7f554c009c60 comp rx=0 tx=0).stop 2024-02-02T15:38:38.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.389+0000 7f5558e00700 1 -- 172.21.15.1:0/62673891 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5554100ff0 msgr2=0x7f5554198fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:38.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.389+0000 7f5558e00700 1 --2- 172.21.15.1:0/62673891 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5554100ff0 0x7f5554198fb0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f554400e9a0 tx=0x7f554400ecb0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.389+0000 7f5558e00700 1 -- 172.21.15.1:0/62673891 shutdown_connections 2024-02-02T15:38:38.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.389+0000 7f5558e00700 1 --2- 172.21.15.1:0/62673891 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f55340a0690 0x7f55340a2b50 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.389+0000 7f5558e00700 1 --2- 172.21.15.1:0/62673891 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f555419d950 0x7f555419ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.389+0000 7f5558e00700 1 --2- 172.21.15.1:0/62673891 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5554100ff0 0x7f5554198fb0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.389+0000 7f5558e00700 1 --2- 172.21.15.1:0/62673891 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5554106110 0x7f55541994f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.389+0000 7f5558e00700 1 -- 172.21.15.1:0/62673891 >> 172.21.15.1:0/62673891 conn(0x7f55540fc810 msgr2=0x7f55540fd240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:38.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.389+0000 7f5558e00700 1 -- 172.21.15.1:0/62673891 shutdown_connections 2024-02-02T15:38:38.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.389+0000 7f5558e00700 1 -- 172.21.15.1:0/62673891 wait complete. 2024-02-02T15:38:38.408 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-02T15:38:38.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.633+0000 7f1331400700 1 -- 172.21.15.1:0/3969467688 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f132c0730e0 msgr2=0x7f132c0734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:38.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.633+0000 7f1331400700 1 --2- 172.21.15.1:0/3969467688 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f132c0730e0 0x7f132c0734c0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f131c009990 tx=0x7f131c009ca0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.633+0000 7f1331400700 1 -- 172.21.15.1:0/3969467688 shutdown_connections 2024-02-02T15:38:38.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.633+0000 7f1331400700 1 --2- 172.21.15.1:0/3969467688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f132c111da0 0x7f132c114190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.633+0000 7f1331400700 1 --2- 172.21.15.1:0/3969467688 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f132c0730e0 0x7f132c0734c0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.633+0000 7f1331400700 1 --2- 172.21.15.1:0/3969467688 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f132c073a00 0x7f132c111860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.633+0000 7f1331400700 1 -- 172.21.15.1:0/3969467688 >> 172.21.15.1:0/3969467688 conn(0x7f132c0fc860 msgr2=0x7f132c0fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:38.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.633+0000 7f1331400700 1 -- 172.21.15.1:0/3969467688 shutdown_connections 2024-02-02T15:38:38.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.633+0000 7f1331400700 1 -- 172.21.15.1:0/3969467688 wait complete. 2024-02-02T15:38:38.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.634+0000 7f1331400700 1 Processor -- start 2024-02-02T15:38:38.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.634+0000 7f1331400700 1 -- start start 2024-02-02T15:38:38.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.635+0000 7f1331400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f132c0730e0 0x7f132c19d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:38.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.635+0000 7f1331400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f132c073a00 0x7f132c19da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:38.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.635+0000 7f1331400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f132c111da0 0x7f132c1a1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:38.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.635+0000 7f1331400700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f132c1a2500 con 0x7f132c073a00 2024-02-02T15:38:38.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.635+0000 7f1331400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f132c1a2670 con 0x7f132c0730e0 2024-02-02T15:38:38.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.635+0000 7f1331400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f132c1a2830 con 0x7f132c111da0 2024-02-02T15:38:38.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.635+0000 7f132a800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f132c0730e0 0x7f132c19d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:38.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.635+0000 7f132a800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f132c0730e0 0x7f132c19d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42732/0 (socket says 172.21.15.1:42732) 2024-02-02T15:38:38.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.635+0000 7f132a800700 1 -- 172.21.15.1:0/1869368544 learned_addr learned my addr 172.21.15.1:0/1869368544 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:38:38.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.635+0000 7f132b200700 1 --2- 172.21.15.1:0/1869368544 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f132c111da0 0x7f132c1a1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:38.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.635+0000 7f1329e00700 1 --2- 172.21.15.1:0/1869368544 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f132c073a00 0x7f132c19da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:38.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.635+0000 7f132a800700 1 -- 172.21.15.1:0/1869368544 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f132c111da0 msgr2=0x7f132c1a1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:38.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.635+0000 7f132a800700 1 --2- 172.21.15.1:0/1869368544 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f132c111da0 0x7f132c1a1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.635+0000 7f132a800700 1 -- 172.21.15.1:0/1869368544 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f132c073a00 msgr2=0x7f132c19da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:38.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.635+0000 7f132a800700 1 --2- 172.21.15.1:0/1869368544 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f132c073a00 0x7f132c19da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.635+0000 7f132a800700 1 -- 172.21.15.1:0/1869368544 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1314009520 con 0x7f132c0730e0 2024-02-02T15:38:38.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.636+0000 7f132a800700 1 --2- 172.21.15.1:0/1869368544 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f132c0730e0 0x7f132c19d520 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f131c005910 tx=0x7f131c004710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:38.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.636+0000 7f1323400700 1 -- 172.21.15.1:0/1869368544 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f131c01d070 con 0x7f132c0730e0 2024-02-02T15:38:38.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.636+0000 7f1331400700 1 -- 172.21.15.1:0/1869368544 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f131c0095f0 con 0x7f132c0730e0 2024-02-02T15:38:38.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.636+0000 7f1323400700 1 -- 172.21.15.1:0/1869368544 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f131c022470 con 0x7f132c0730e0 2024-02-02T15:38:38.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.636+0000 7f1323400700 1 -- 172.21.15.1:0/1869368544 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f131c017650 con 0x7f132c0730e0 2024-02-02T15:38:38.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.637+0000 7f1331400700 1 -- 172.21.15.1:0/1869368544 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f132c1aa170 con 0x7f132c0730e0 2024-02-02T15:38:38.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.639+0000 7f1323400700 1 -- 172.21.15.1:0/1869368544 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f131c0225e0 con 0x7f132c0730e0 2024-02-02T15:38:38.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.641+0000 7f1323400700 1 --2- 172.21.15.1:0/1869368544 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f130c0a0690 0x7f130c0a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:38.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.641+0000 7f1323400700 1 -- 172.21.15.1:0/1869368544 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f131c0cd370 con 0x7f132c0730e0 2024-02-02T15:38:38.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.641+0000 7f1344e64b80 1 Processor -- start 2024-02-02T15:38:38.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.641+0000 7f1329e00700 1 --2- 172.21.15.1:0/1869368544 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f130c0a0690 0x7f130c0a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:38.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.641+0000 7f1344e64b80 1 -- start start 2024-02-02T15:38:38.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.642+0000 7f1344e64b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5652773a1010 0x5652773a13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:38.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.642+0000 7f1344e64b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5652773a1930 0x5652773ac2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:38.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.642+0000 7f1344e64b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5652773ac7e0 0x5652773aebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:38.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.642+0000 7f1344e64b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652773154c0 con 0x5652773ac7e0 2024-02-02T15:38:38.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.642+0000 7f1344e64b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652772f9aa0 con 0x5652773a1930 2024-02-02T15:38:38.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.642+0000 7f1344e64b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652772f7460 con 0x5652773a1010 2024-02-02T15:38:38.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.642+0000 7f132a800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5652773a1010 0x5652773a13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:38.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.642+0000 7f132a800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5652773a1010 0x5652773a13f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:53766/0 (socket says 172.21.15.1:53766) 2024-02-02T15:38:38.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.642+0000 7f132a800700 1 -- 172.21.15.1:0/1262083299 learned_addr learned my addr 172.21.15.1:0/1262083299 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:38:38.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.642+0000 7f132b200700 1 --2- 172.21.15.1:0/1262083299 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5652773ac7e0 0x5652773aebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:38.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.643+0000 7f132a800700 1 -- 172.21.15.1:0/1262083299 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5652773a1930 msgr2=0x5652773ac2a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-02T15:38:38.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.643+0000 7f132a800700 1 --2- 172.21.15.1:0/1262083299 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5652773a1930 0x5652773ac2a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.643+0000 7f132a800700 1 -- 172.21.15.1:0/1262083299 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5652773ac7e0 msgr2=0x5652773aebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:38.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.643+0000 7f132a800700 1 --2- 172.21.15.1:0/1262083299 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5652773ac7e0 0x5652773aebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.643+0000 7f132a800700 1 -- 172.21.15.1:0/1262083299 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5652771743b0 con 0x5652773a1010 2024-02-02T15:38:38.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.643+0000 7f132b200700 1 --2- 172.21.15.1:0/1262083299 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5652773ac7e0 0x5652773aebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:38:38.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.643+0000 7f1329e00700 1 --2- 172.21.15.1:0/1869368544 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f130c0a0690 0x7f130c0a2b50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f132c19eb30 tx=0x7f1314009260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:38.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.643+0000 7f132a800700 1 --2- 172.21.15.1:0/1262083299 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5652773a1010 0x5652773a13f0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f131c024000 tx=0x7f131c09a3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=709f493e9087e355 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:38.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.643+0000 7f131b400700 1 -- 172.21.15.1:0/1262083299 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f131c096040 con 0x5652773a1010 2024-02-02T15:38:38.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.643+0000 7f131b400700 1 -- 172.21.15.1:0/1262083299 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f131c09ac20 con 0x5652773a1010 2024-02-02T15:38:38.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.643+0000 7f131b400700 1 -- 172.21.15.1:0/1262083299 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f131c0850e0 con 0x5652773a1010 2024-02-02T15:38:38.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.644+0000 7f1344e64b80 1 -- 172.21.15.1:0/1262083299 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5652773a1010 msgr2=0x5652773a13f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:38.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.644+0000 7f1344e64b80 1 --2- 172.21.15.1:0/1262083299 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5652773a1010 0x5652773a13f0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f131c024000 tx=0x7f131c09a3a0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.644+0000 7f1344e64b80 1 -- 172.21.15.1:0/1262083299 shutdown_connections 2024-02-02T15:38:38.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.644+0000 7f1344e64b80 1 --2- 172.21.15.1:0/1262083299 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5652773ac7e0 0x5652773aebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.644+0000 7f1344e64b80 1 --2- 172.21.15.1:0/1262083299 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5652773a1010 0x5652773a13f0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.644+0000 7f1344e64b80 1 --2- 172.21.15.1:0/1262083299 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5652773a1930 0x5652773ac2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.644+0000 7f1344e64b80 1 -- 172.21.15.1:0/1262083299 >> 172.21.15.1:0/1262083299 conn(0x565277300db0 msgr2=0x56527739d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:38.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.644+0000 7f1344e64b80 1 -- 172.21.15.1:0/1262083299 shutdown_connections 2024-02-02T15:38:38.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.644+0000 7f1344e64b80 1 -- 172.21.15.1:0/1262083299 wait complete. 2024-02-02T15:38:38.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.645+0000 7f1344e64b80 1 -- start start 2024-02-02T15:38:38.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.645+0000 7f1344e64b80 1 Processor -- start 2024-02-02T15:38:38.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.646+0000 7f1344e64b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5652773a1010 0x5652773acbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:38.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.646+0000 7f1344e64b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5652773a1930 0x5652773ad100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:38.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.646+0000 7f1344e64b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5652773ac7e0 0x5652773b7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:38.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.646+0000 7f1344e64b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652772f2d50 con 0x5652773a1930 2024-02-02T15:38:38.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.646+0000 7f1344e64b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56527728ed80 con 0x5652773a1010 2024-02-02T15:38:38.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.646+0000 7f1344e64b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56527728ea20 con 0x5652773ac7e0 2024-02-02T15:38:38.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.646+0000 7f1344e64b80 10 client.0 _ll_register_callbacks cb 0x565277290290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:38:38.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.646+0000 7f132a800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5652773a1010 0x5652773acbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:38.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.646+0000 7f132a800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5652773a1010 0x5652773acbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42746/0 (socket says 172.21.15.1:42746) 2024-02-02T15:38:38.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.646+0000 7f132a800700 1 -- 172.21.15.1:0/428188801 learned_addr learned my addr 172.21.15.1:0/428188801 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:38:38.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.646+0000 7f1329e00700 1 --2- 172.21.15.1:0/428188801 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5652773a1930 0x5652773ad100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:38.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.646+0000 7f132b200700 1 --2- 172.21.15.1:0/428188801 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5652773ac7e0 0x5652773b7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:38.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.646+0000 7f132a800700 1 -- 172.21.15.1:0/428188801 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5652773ac7e0 msgr2=0x5652773b7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:38.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.646+0000 7f132a800700 1 --2- 172.21.15.1:0/428188801 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5652773ac7e0 0x5652773b7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.646+0000 7f132a800700 1 -- 172.21.15.1:0/428188801 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5652773a1930 msgr2=0x5652773ad100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:38.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.646+0000 7f132a800700 1 --2- 172.21.15.1:0/428188801 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5652773a1930 0x5652773ad100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.646+0000 7f132a800700 1 -- 172.21.15.1:0/428188801 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f132c1aa170 con 0x5652773a1010 2024-02-02T15:38:38.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.647+0000 7f132a800700 1 --2- 172.21.15.1:0/428188801 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5652773a1010 0x5652773acbc0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f131c006010 tx=0x7f131c09ad90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81378275c1d0268d server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:38.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.647+0000 7f1313e00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:38:38.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.647+0000 7f1313e00700 1 -- 172.21.15.1:0/428188801 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f131c096040 con 0x5652773a1010 2024-02-02T15:38:38.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.647+0000 7f1313e00700 1 -- 172.21.15.1:0/428188801 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f131c09e350 con 0x5652773a1010 2024-02-02T15:38:38.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.647+0000 7f1344e64b80 1 -- 172.21.15.1:0/428188801 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5652773950f0 con 0x5652773a1010 2024-02-02T15:38:38.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.647+0000 7f1313e00700 1 -- 172.21.15.1:0/428188801 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f131c09e150 con 0x5652773a1010 2024-02-02T15:38:38.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.647+0000 7f1344e64b80 10 client.28745 fetch_fsmap learned FSMap version 51 2024-02-02T15:38:38.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.647+0000 7f1344e64b80 1 -- 172.21.15.1:0/428188801 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x565277120b60 con 0x5652773a1010 2024-02-02T15:38:38.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.647+0000 7f1313e00700 1 -- 172.21.15.1:0/428188801 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5652773950f0 con 0x5652773a1010 2024-02-02T15:38:38.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.647+0000 7f1313e00700 1 -- 172.21.15.1:0/428188801 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 51) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f131c083d20 con 0x5652773a1010 2024-02-02T15:38:38.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.648+0000 7f1344e64b80 10 client.28745 fetch_fsmap finished waiting for FSMap version 51 2024-02-02T15:38:38.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.648+0000 7f1344e64b80 10 client.28745 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:38:38.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.648+0000 7f1344e64b80 20 client.28745 populate_metadata read hostname 'smithi001' 2024-02-02T15:38:38.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.648+0000 7f1344e64b80 1 --2- 172.21.15.1:0/428188801 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5652773bba00 0x5652773bddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:38.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.648+0000 7f1344e64b80 4 client.28745 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:38:38.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.648+0000 7f1344e64b80 1 -- 172.21.15.1:0/428188801 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5652772ebdd0 con 0x5652773bba00 2024-02-02T15:38:38.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.648+0000 7f1329e00700 1 --2- 172.21.15.1:0/428188801 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5652773bba00 0x5652773bddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:38.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.648+0000 7f1329e00700 1 --2- 172.21.15.1:0/428188801 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5652773bba00 0x5652773bddf0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6cdc813a4e0da7c server_cookie=dffe8b3700d51acc in_seq=0 out_seq=0 2024-02-02T15:38:38.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.648+0000 7f1313e00700 10 client.28745 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:38:38.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.649+0000 7f1313e00700 1 -- 172.21.15.1:0/428188801 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5652772ebdd0 con 0x5652773bba00 2024-02-02T15:38:38.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.649+0000 7f1313e00700 10 client.28745 handle_command_reply: tid=0 2024-02-02T15:38:38.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.649+0000 7f1344e64b80 1 client.28745 shutdown 2024-02-02T15:38:38.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.649+0000 7f1344e64b80 1 -- 172.21.15.1:0/428188801 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5652773a1010 msgr2=0x5652773acbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:38.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.649+0000 7f1344e64b80 1 --2- 172.21.15.1:0/428188801 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5652773a1010 0x5652773acbc0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f131c006010 tx=0x7f131c09ad90 comp rx=0 tx=0).stop 2024-02-02T15:38:38.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.649+0000 7f1344e64b80 1 -- 172.21.15.1:0/428188801 shutdown_connections 2024-02-02T15:38:38.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.649+0000 7f1344e64b80 1 --2- 172.21.15.1:0/428188801 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5652773bba00 0x5652773bddf0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.649+0000 7f1344e64b80 1 --2- 172.21.15.1:0/428188801 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5652773ac7e0 0x5652773b7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.649+0000 7f1344e64b80 1 --2- 172.21.15.1:0/428188801 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5652773a1010 0x5652773acbc0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.649+0000 7f1344e64b80 1 --2- 172.21.15.1:0/428188801 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5652773a1930 0x5652773ad100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.649+0000 7f1344e64b80 1 -- 172.21.15.1:0/428188801 >> 172.21.15.1:0/428188801 conn(0x565277300db0 msgr2=0x56527739cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:38.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.649+0000 7f1313e00700 0 client.28745 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:38:38.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.649+0000 7f1344e64b80 1 -- 172.21.15.1:0/428188801 shutdown_connections 2024-02-02T15:38:38.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.649+0000 7f1344e64b80 1 -- 172.21.15.1:0/428188801 wait complete. 2024-02-02T15:38:38.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.650+0000 7f1344e64b80 20 client.28745 trim_cache size 0 max 16384 2024-02-02T15:38:38.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.664+0000 7f1344e64b80 1 Processor -- start 2024-02-02T15:38:38.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.665+0000 7f1344e64b80 1 -- start start 2024-02-02T15:38:38.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.665+0000 7f1344e64b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5652773a1010 0x5652773b1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:38.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.665+0000 7f1344e64b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5652773a1930 0x5652773acbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:38.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.665+0000 7f1344e64b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5652773ac7e0 0x5652773ad100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:38.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.665+0000 7f1344e64b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652771766a0 con 0x5652773a1010 2024-02-02T15:38:38.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.665+0000 7f1344e64b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x565277397500 con 0x5652773ac7e0 2024-02-02T15:38:38.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.665+0000 7f1344e64b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652773abf50 con 0x5652773a1930 2024-02-02T15:38:38.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.665+0000 7f1329e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5652773a1930 0x5652773acbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:38.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.665+0000 7f1329e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5652773a1930 0x5652773acbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:53794/0 (socket says 172.21.15.1:53794) 2024-02-02T15:38:38.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.665+0000 7f1329e00700 1 -- 172.21.15.1:0/2495125514 learned_addr learned my addr 172.21.15.1:0/2495125514 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:38:38.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.665+0000 7f132a800700 1 --2- 172.21.15.1:0/2495125514 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5652773a1010 0x5652773b1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:38.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.665+0000 7f132b200700 1 --2- 172.21.15.1:0/2495125514 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5652773ac7e0 0x5652773ad100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:38.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.666+0000 7f1329e00700 1 -- 172.21.15.1:0/2495125514 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5652773ac7e0 msgr2=0x5652773ad100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:38.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.666+0000 7f1329e00700 1 --2- 172.21.15.1:0/2495125514 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5652773ac7e0 0x5652773ad100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.666+0000 7f1329e00700 1 -- 172.21.15.1:0/2495125514 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5652773a1010 msgr2=0x5652773b1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:38.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.666+0000 7f1329e00700 1 --2- 172.21.15.1:0/2495125514 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5652773a1010 0x5652773b1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.666+0000 7f1329e00700 1 -- 172.21.15.1:0/2495125514 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5652772b8ea0 con 0x5652773a1930 2024-02-02T15:38:38.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.666+0000 7f132a800700 1 --2- 172.21.15.1:0/2495125514 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5652773a1010 0x5652773b1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:38:38.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.666+0000 7f1329e00700 1 --2- 172.21.15.1:0/2495125514 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5652773a1930 0x5652773acbc0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f1314009bd0 tx=0x7f1314014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5c6bca91dffbb16 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:38.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.666+0000 7f131be00700 1 -- 172.21.15.1:0/2495125514 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1314024070 con 0x5652773a1930 2024-02-02T15:38:38.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.666+0000 7f131be00700 1 -- 172.21.15.1:0/2495125514 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1314003e20 con 0x5652773a1930 2024-02-02T15:38:38.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.666+0000 7f131be00700 1 -- 172.21.15.1:0/2495125514 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1314026c50 con 0x5652773a1930 2024-02-02T15:38:38.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.667+0000 7f1344e64b80 1 -- 172.21.15.1:0/2495125514 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5652773a1930 msgr2=0x5652773acbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:38.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.667+0000 7f1344e64b80 1 --2- 172.21.15.1:0/2495125514 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5652773a1930 0x5652773acbc0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f1314009bd0 tx=0x7f1314014000 comp rx=0 tx=0).stop 2024-02-02T15:38:38.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.667+0000 7f1344e64b80 1 -- 172.21.15.1:0/2495125514 shutdown_connections 2024-02-02T15:38:38.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.667+0000 7f1344e64b80 1 --2- 172.21.15.1:0/2495125514 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5652773ac7e0 0x5652773ad100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.667+0000 7f1344e64b80 1 --2- 172.21.15.1:0/2495125514 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5652773a1010 0x5652773b1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.667+0000 7f1344e64b80 1 --2- 172.21.15.1:0/2495125514 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5652773a1930 0x5652773acbc0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.667+0000 7f1344e64b80 1 -- 172.21.15.1:0/2495125514 >> 172.21.15.1:0/2495125514 conn(0x565277300db0 msgr2=0x56527739c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:38.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.667+0000 7f1344e64b80 1 -- 172.21.15.1:0/2495125514 shutdown_connections 2024-02-02T15:38:38.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.667+0000 7f1344e64b80 1 -- 172.21.15.1:0/2495125514 wait complete. 2024-02-02T15:38:38.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.668+0000 7f1344e64b80 1 -- start start 2024-02-02T15:38:38.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.668+0000 7f1344e64b80 1 Processor -- start 2024-02-02T15:38:38.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.669+0000 7f1344e64b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5652773a1010 0x5652773b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:38.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.669+0000 7f1344e64b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5652773a1930 0x5652773b1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:38.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.669+0000 7f1344e64b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5652773ac7e0 0x5652773af7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:38.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.669+0000 7f1344e64b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652773afd80 con 0x5652773a1010 2024-02-02T15:38:38.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.669+0000 7f1344e64b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652773afef0 con 0x5652773ac7e0 2024-02-02T15:38:38.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.669+0000 7f1344e64b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652773b00b0 con 0x5652773a1930 2024-02-02T15:38:38.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.669+0000 7f1344e64b80 10 client.0 _ll_register_callbacks cb 0x565277290290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:38:38.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.669+0000 7f132b200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5652773ac7e0 0x5652773af7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:38.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.669+0000 7f132b200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5652773ac7e0 0x5652773af7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42770/0 (socket says 172.21.15.1:42770) 2024-02-02T15:38:38.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.669+0000 7f132a800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5652773a1010 0x5652773b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:38.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.669+0000 7f132b200700 1 -- 172.21.15.1:0/2935534271 learned_addr learned my addr 172.21.15.1:0/2935534271 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:38:38.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.669+0000 7f1329e00700 1 --2- 172.21.15.1:0/2935534271 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5652773a1930 0x5652773b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:38.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.670+0000 7f132b200700 1 -- 172.21.15.1:0/2935534271 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5652773a1930 msgr2=0x5652773b1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:38.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.670+0000 7f132b200700 1 --2- 172.21.15.1:0/2935534271 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5652773a1930 0x5652773b1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.670+0000 7f132b200700 1 -- 172.21.15.1:0/2935534271 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5652773a1010 msgr2=0x5652773b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:38.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.670+0000 7f132b200700 1 --2- 172.21.15.1:0/2935534271 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5652773a1010 0x5652773b12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.670+0000 7f132b200700 1 -- 172.21.15.1:0/2935534271 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1314017070 con 0x5652773ac7e0 2024-02-02T15:38:38.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.670+0000 7f1329e00700 1 --2- 172.21.15.1:0/2935534271 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5652773a1930 0x5652773b1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:38:38.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.670+0000 7f132b200700 1 --2- 172.21.15.1:0/2935534271 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5652773ac7e0 0x5652773af7a0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f13240092f0 tx=0x7f1324009320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2bb61f9cd4d32a85 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:38.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.670+0000 7f1313e00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:38:38.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.670+0000 7f1313e00700 1 -- 172.21.15.1:0/2935534271 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f132400e040 con 0x5652773ac7e0 2024-02-02T15:38:38.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.670+0000 7f1313e00700 1 -- 172.21.15.1:0/2935534271 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f1324004db0 con 0x5652773ac7e0 2024-02-02T15:38:38.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.670+0000 7f1344e64b80 1 -- 172.21.15.1:0/2935534271 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f131c09ac20 con 0x5652773ac7e0 2024-02-02T15:38:38.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.670+0000 7f1313e00700 1 -- 172.21.15.1:0/2935534271 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1324018b60 con 0x5652773ac7e0 2024-02-02T15:38:38.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.670+0000 7f1344e64b80 10 client.28751 fetch_fsmap learned FSMap version 51 2024-02-02T15:38:38.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.670+0000 7f1344e64b80 1 -- 172.21.15.1:0/2935534271 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x565277178cd0 con 0x5652773ac7e0 2024-02-02T15:38:38.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.671+0000 7f1313e00700 1 -- 172.21.15.1:0/2935534271 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f131c09ac20 con 0x5652773ac7e0 2024-02-02T15:38:38.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.671+0000 7f1313e00700 1 -- 172.21.15.1:0/2935534271 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 51) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f13240044d0 con 0x5652773ac7e0 2024-02-02T15:38:38.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.671+0000 7f1344e64b80 10 client.28751 fetch_fsmap finished waiting for FSMap version 51 2024-02-02T15:38:38.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.671+0000 7f1344e64b80 10 client.28751 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:38:38.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.671+0000 7f1344e64b80 20 client.28751 populate_metadata read hostname 'smithi001' 2024-02-02T15:38:38.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.671+0000 7f1344e64b80 1 --2- 172.21.15.1:0/2935534271 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56527740e6b0 0x565277410ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:38.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.671+0000 7f1344e64b80 4 client.28751 mds_command: new command op to 24557 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:38:38.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.671+0000 7f1344e64b80 1 -- 172.21.15.1:0/2935534271 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5652772b1f50 con 0x56527740e6b0 2024-02-02T15:38:38.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.671+0000 7f132a800700 1 --2- 172.21.15.1:0/2935534271 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56527740e6b0 0x565277410ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:38.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.672+0000 7f1313e00700 10 client.28751 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:38:38.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.672+0000 7f132a800700 1 --2- 172.21.15.1:0/2935534271 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56527740e6b0 0x565277410ad0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df9a088274e4b312 server_cookie=45d08119e056ac14 in_seq=0 out_seq=0 2024-02-02T15:38:38.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.672+0000 7f1313e00700 1 -- 172.21.15.1:0/2935534271 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5652772b1f50 con 0x56527740e6b0 2024-02-02T15:38:38.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.672+0000 7f1313e00700 10 client.28751 handle_command_reply: tid=0 2024-02-02T15:38:38.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.672+0000 7f1344e64b80 1 client.28751 shutdown 2024-02-02T15:38:38.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.672+0000 7f1344e64b80 1 -- 172.21.15.1:0/2935534271 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5652773ac7e0 msgr2=0x5652773af7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:38.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.672+0000 7f1344e64b80 1 --2- 172.21.15.1:0/2935534271 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5652773ac7e0 0x5652773af7a0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f13240092f0 tx=0x7f1324009320 comp rx=0 tx=0).stop 2024-02-02T15:38:38.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.672+0000 7f1344e64b80 1 -- 172.21.15.1:0/2935534271 shutdown_connections 2024-02-02T15:38:38.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.672+0000 7f1344e64b80 1 --2- 172.21.15.1:0/2935534271 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56527740e6b0 0x565277410ad0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.672+0000 7f1344e64b80 1 --2- 172.21.15.1:0/2935534271 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5652773ac7e0 0x5652773af7a0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.672+0000 7f1344e64b80 1 --2- 172.21.15.1:0/2935534271 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5652773a1010 0x5652773b12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.672+0000 7f1344e64b80 1 --2- 172.21.15.1:0/2935534271 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5652773a1930 0x5652773b1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.672+0000 7f1313e00700 0 client.28751 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:38:38.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.672+0000 7f1344e64b80 1 -- 172.21.15.1:0/2935534271 >> 172.21.15.1:0/2935534271 conn(0x565277300db0 msgr2=0x5652773acbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:38.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.672+0000 7f1344e64b80 1 -- 172.21.15.1:0/2935534271 shutdown_connections 2024-02-02T15:38:38.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.672+0000 7f1344e64b80 1 -- 172.21.15.1:0/2935534271 wait complete. 2024-02-02T15:38:38.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.673+0000 7f1344e64b80 20 client.28751 trim_cache size 0 max 16384 2024-02-02T15:38:38.675 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:38:38.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.673+0000 7f1331400700 1 -- 172.21.15.1:0/1869368544 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f130c0a0690 msgr2=0x7f130c0a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:38.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.673+0000 7f1331400700 1 --2- 172.21.15.1:0/1869368544 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f130c0a0690 0x7f130c0a2b50 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f132c19eb30 tx=0x7f1314009260 comp rx=0 tx=0).stop 2024-02-02T15:38:38.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.673+0000 7f1331400700 1 -- 172.21.15.1:0/1869368544 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f132c0730e0 msgr2=0x7f132c19d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:38.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.673+0000 7f1331400700 1 --2- 172.21.15.1:0/1869368544 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f132c0730e0 0x7f132c19d520 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f131c005910 tx=0x7f131c004710 comp rx=0 tx=0).stop 2024-02-02T15:38:38.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.673+0000 7f1331400700 1 -- 172.21.15.1:0/1869368544 shutdown_connections 2024-02-02T15:38:38.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.673+0000 7f1331400700 1 --2- 172.21.15.1:0/1869368544 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f130c0a0690 0x7f130c0a2b50 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.673+0000 7f1331400700 1 --2- 172.21.15.1:0/1869368544 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f132c111da0 0x7f132c1a1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.673+0000 7f1331400700 1 --2- 172.21.15.1:0/1869368544 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f132c0730e0 0x7f132c19d520 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.673+0000 7f1331400700 1 --2- 172.21.15.1:0/1869368544 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f132c073a00 0x7f132c19da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:38.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.673+0000 7f1331400700 1 -- 172.21.15.1:0/1869368544 >> 172.21.15.1:0/1869368544 conn(0x7f132c0fc860 msgr2=0x7f132c0fd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:38.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.673+0000 7f1331400700 1 -- 172.21.15.1:0/1869368544 shutdown_connections 2024-02-02T15:38:38.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:38.673+0000 7f1331400700 1 -- 172.21.15.1:0/1869368544 wait complete. 2024-02-02T15:38:39.268 INFO:tasks.workunit.client.0.smithi001.stdout: 1 643354 5.54 MB/sec execute 493 sec latency 14.761 ms 2024-02-02T15:38:39.692 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-02T15:38:39.693 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-02T15:38:39.755 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:39 smithi001 ceph-mon[35470]: pgmap v362: 129 pgs: 129 active+clean; 528 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 115 op/s 2024-02-02T15:38:39.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:39 smithi071 ceph-mon[44707]: pgmap v362: 129 pgs: 129 active+clean; 528 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 115 op/s 2024-02-02T15:38:39.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:39 smithi142 ceph-mon[45728]: pgmap v362: 129 pgs: 129 active+clean; 528 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 115 op/s 2024-02-02T15:38:39.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.922+0000 7f2317200700 1 -- 172.21.15.1:0/341762542 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2310102030 msgr2=0x7f2310102410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:39.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.922+0000 7f2317200700 1 --2- 172.21.15.1:0/341762542 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2310102030 0x7f2310102410 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f23000098f0 tx=0x7f2300009c00 comp rx=0 tx=0).stop 2024-02-02T15:38:39.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.922+0000 7f2317200700 1 -- 172.21.15.1:0/341762542 shutdown_connections 2024-02-02T15:38:39.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.922+0000 7f2317200700 1 --2- 172.21.15.1:0/341762542 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f231010b400 0x7f231010d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:39.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.922+0000 7f2317200700 1 --2- 172.21.15.1:0/341762542 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2310102030 0x7f2310102410 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:39.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.922+0000 7f2317200700 1 --2- 172.21.15.1:0/341762542 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2310102950 0x7f231010ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:39.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.922+0000 7f2317200700 1 -- 172.21.15.1:0/341762542 >> 172.21.15.1:0/341762542 conn(0x7f2310076d70 msgr2=0x7f2310077180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:39.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.922+0000 7f2317200700 1 -- 172.21.15.1:0/341762542 shutdown_connections 2024-02-02T15:38:39.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.922+0000 7f2317200700 1 -- 172.21.15.1:0/341762542 wait complete. 2024-02-02T15:38:39.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.923+0000 7f2317200700 1 Processor -- start 2024-02-02T15:38:39.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.924+0000 7f2317200700 1 -- start start 2024-02-02T15:38:39.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.924+0000 7f2317200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2310102030 0x7f23100ffc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:39.923 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.924+0000 7f2317200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2310102950 0x7f23101001c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:39.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.924+0000 7f2317200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f231010b400 0x7f23101a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:39.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.924+0000 7f2317200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23101009f0 con 0x7f2310102950 2024-02-02T15:38:39.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.924+0000 7f2317200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23101a5610 con 0x7f2310102030 2024-02-02T15:38:39.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.924+0000 7f2317200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23101a5750 con 0x7f231010b400 2024-02-02T15:38:39.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.924+0000 7f230fe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2310102030 0x7f23100ffc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:39.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.924+0000 7f230f400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2310102950 0x7f23101001c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:39.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.924+0000 7f2315200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f231010b400 0x7f23101a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:39.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.924+0000 7f2315200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f231010b400 0x7f23101a50d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:53812/0 (socket says 172.21.15.1:53812) 2024-02-02T15:38:39.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.924+0000 7f230fe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2310102030 0x7f23100ffc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42794/0 (socket says 172.21.15.1:42794) 2024-02-02T15:38:39.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.924+0000 7f2315200700 1 -- 172.21.15.1:0/2375225057 learned_addr learned my addr 172.21.15.1:0/2375225057 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:38:39.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.925+0000 7f230fe00700 1 -- 172.21.15.1:0/2375225057 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f231010b400 msgr2=0x7f23101a50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:39.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.925+0000 7f230fe00700 1 --2- 172.21.15.1:0/2375225057 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f231010b400 0x7f23101a50d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:39.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.925+0000 7f230fe00700 1 -- 172.21.15.1:0/2375225057 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2310102950 msgr2=0x7f23101001c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:39.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.925+0000 7f230fe00700 1 --2- 172.21.15.1:0/2375225057 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2310102950 0x7f23101001c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:39.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.925+0000 7f230fe00700 1 -- 172.21.15.1:0/2375225057 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2300009520 con 0x7f2310102030 2024-02-02T15:38:39.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.925+0000 7f230fe00700 1 --2- 172.21.15.1:0/2375225057 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2310102030 0x7f23100ffc80 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f2300009810 tx=0x7f230000fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:39.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.925+0000 7f230cc00700 1 -- 172.21.15.1:0/2375225057 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f230001c070 con 0x7f2310102030 2024-02-02T15:38:39.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.925+0000 7f230cc00700 1 -- 172.21.15.1:0/2375225057 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2300005040 con 0x7f2310102030 2024-02-02T15:38:39.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.925+0000 7f230cc00700 1 -- 172.21.15.1:0/2375225057 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23000038e0 con 0x7f2310102030 2024-02-02T15:38:39.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.925+0000 7f2317200700 1 -- 172.21.15.1:0/2375225057 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23101a5950 con 0x7f2310102030 2024-02-02T15:38:39.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.925+0000 7f2317200700 1 -- 172.21.15.1:0/2375225057 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23101a5d80 con 0x7f2310102030 2024-02-02T15:38:39.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.928+0000 7f230cc00700 1 -- 172.21.15.1:0/2375225057 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f2300020030 con 0x7f2310102030 2024-02-02T15:38:39.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.930+0000 7f230cc00700 1 --2- 172.21.15.1:0/2375225057 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f22f00a0710 0x7f22f00a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:39.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.930+0000 7f232acc5b80 1 Processor -- start 2024-02-02T15:38:39.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.930+0000 7f230cc00700 1 -- 172.21.15.1:0/2375225057 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f23000ccfd0 con 0x7f2310102030 2024-02-02T15:38:39.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.930+0000 7f232acc5b80 1 -- start start 2024-02-02T15:38:39.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.931+0000 7f230f400700 1 --2- 172.21.15.1:0/2375225057 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f22f00a0710 0x7f22f00a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:39.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.931+0000 7f232acc5b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a501ffc090 0x55a501ffc470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:39.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.931+0000 7f232acc5b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a501ffc9b0 0x55a50200fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:39.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.931+0000 7f232acc5b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a502010070 0x55a502012460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:39.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.931+0000 7f232acc5b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a501f70540 con 0x55a502010070 2024-02-02T15:38:39.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.931+0000 7f232acc5b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a501f54b20 con 0x55a501ffc090 2024-02-02T15:38:39.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.931+0000 7f232acc5b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a501f524e0 con 0x55a501ffc9b0 2024-02-02T15:38:39.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.931+0000 7f230fe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a501ffc090 0x55a501ffc470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:39.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.931+0000 7f230fe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a501ffc090 0x55a501ffc470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42806/0 (socket says 172.21.15.1:42806) 2024-02-02T15:38:39.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.931+0000 7f230fe00700 1 -- 172.21.15.1:0/733293657 learned_addr learned my addr 172.21.15.1:0/733293657 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:38:39.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.931+0000 7f2315200700 1 --2- 172.21.15.1:0/733293657 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a502010070 0x55a502012460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:39.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.931+0000 7f230f400700 1 --2- 172.21.15.1:0/733293657 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a501ffc9b0 0x55a50200fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:39.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.931+0000 7f230fe00700 1 -- 172.21.15.1:0/733293657 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a501ffc9b0 msgr2=0x55a50200fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:39.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.931+0000 7f230fe00700 1 --2- 172.21.15.1:0/733293657 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a501ffc9b0 0x55a50200fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:39.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.931+0000 7f230fe00700 1 -- 172.21.15.1:0/733293657 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a502010070 msgr2=0x55a502012460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:39.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.931+0000 7f230fe00700 1 --2- 172.21.15.1:0/733293657 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a502010070 0x55a502012460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:39.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.931+0000 7f230fe00700 1 -- 172.21.15.1:0/733293657 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a501dcf430 con 0x55a501ffc090 2024-02-02T15:38:39.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.932+0000 7f230f400700 1 --2- 172.21.15.1:0/2375225057 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f22f00a0710 0x7f22f00a2bd0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2310101470 tx=0x7f22f8009310 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:39.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.932+0000 7f230fe00700 1 --2- 172.21.15.1:0/733293657 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a501ffc090 0x55a501ffc470 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f2300099e80 tx=0x7f2300099bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c974b24272f6eddb server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:39.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.932+0000 7f2305600700 1 -- 172.21.15.1:0/733293657 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23000a00c0 con 0x55a501ffc090 2024-02-02T15:38:39.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.932+0000 7f2305600700 1 -- 172.21.15.1:0/733293657 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f230009a180 con 0x55a501ffc090 2024-02-02T15:38:39.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.932+0000 7f2305600700 1 -- 172.21.15.1:0/733293657 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2300005580 con 0x55a501ffc090 2024-02-02T15:38:39.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.933+0000 7f232acc5b80 1 -- 172.21.15.1:0/733293657 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a501ffc090 msgr2=0x55a501ffc470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:39.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.933+0000 7f232acc5b80 1 --2- 172.21.15.1:0/733293657 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a501ffc090 0x55a501ffc470 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f2300099e80 tx=0x7f2300099bb0 comp rx=0 tx=0).stop 2024-02-02T15:38:39.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.933+0000 7f232acc5b80 1 -- 172.21.15.1:0/733293657 shutdown_connections 2024-02-02T15:38:39.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.933+0000 7f232acc5b80 1 --2- 172.21.15.1:0/733293657 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a502010070 0x55a502012460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:39.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.933+0000 7f232acc5b80 1 --2- 172.21.15.1:0/733293657 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a501ffc090 0x55a501ffc470 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:39.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.933+0000 7f232acc5b80 1 --2- 172.21.15.1:0/733293657 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a501ffc9b0 0x55a50200fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:39.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.933+0000 7f232acc5b80 1 -- 172.21.15.1:0/733293657 >> 172.21.15.1:0/733293657 conn(0x55a501f5be30 msgr2=0x55a501ff8430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:39.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.933+0000 7f232acc5b80 1 -- 172.21.15.1:0/733293657 shutdown_connections 2024-02-02T15:38:39.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.933+0000 7f232acc5b80 1 -- 172.21.15.1:0/733293657 wait complete. 2024-02-02T15:38:39.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.934+0000 7f232acc5b80 1 -- start start 2024-02-02T15:38:39.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.934+0000 7f232acc5b80 1 Processor -- start 2024-02-02T15:38:39.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.934+0000 7f232acc5b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a501ffc090 0x55a502010450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:39.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.935+0000 7f232acc5b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a501ffc9b0 0x55a502010990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:39.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.935+0000 7f232acc5b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a502010070 0x55a502010ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:39.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.935+0000 7f232acc5b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a501f4ddd0 con 0x55a502010070 2024-02-02T15:38:39.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.935+0000 7f232acc5b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a501ee9e00 con 0x55a501ffc9b0 2024-02-02T15:38:39.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.935+0000 7f232acc5b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a501ee9aa0 con 0x55a501ffc090 2024-02-02T15:38:39.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.935+0000 7f232acc5b80 10 client.0 _ll_register_callbacks cb 0x55a501eeb310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:38:39.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.935+0000 7f230f400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a501ffc9b0 0x55a502010990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:39.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.935+0000 7f230fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a501ffc090 0x55a502010450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:39.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.935+0000 7f230f400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a501ffc9b0 0x55a502010990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42816/0 (socket says 172.21.15.1:42816) 2024-02-02T15:38:39.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.935+0000 7f230f400700 1 -- 172.21.15.1:0/393059455 learned_addr learned my addr 172.21.15.1:0/393059455 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:38:39.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.935+0000 7f2315200700 1 --2- 172.21.15.1:0/393059455 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a502010070 0x55a502010ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:39.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.935+0000 7f230fe00700 1 -- 172.21.15.1:0/393059455 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a501ffc9b0 msgr2=0x55a502010990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:39.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.935+0000 7f230fe00700 1 --2- 172.21.15.1:0/393059455 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a501ffc9b0 0x55a502010990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:39.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.935+0000 7f230fe00700 1 -- 172.21.15.1:0/393059455 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a502010070 msgr2=0x55a502010ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:39.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.935+0000 7f230fe00700 1 --2- 172.21.15.1:0/393059455 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a502010070 0x55a502010ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:39.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.935+0000 7f230fe00700 1 -- 172.21.15.1:0/393059455 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23101a5d80 con 0x55a501ffc090 2024-02-02T15:38:39.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.936+0000 7f230fe00700 1 --2- 172.21.15.1:0/393059455 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a501ffc090 0x55a502010450 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f2300099e80 tx=0x7f230000fa70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ec57ba8935dab7b server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:39.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.936+0000 7f22fe000700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:38:39.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.936+0000 7f22fe000700 1 -- 172.21.15.1:0/393059455 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23000b8070 con 0x55a501ffc090 2024-02-02T15:38:39.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.936+0000 7f22fe000700 1 -- 172.21.15.1:0/393059455 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f230009c070 con 0x55a501ffc090 2024-02-02T15:38:39.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.936+0000 7f232acc5b80 1 -- 172.21.15.1:0/393059455 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a501ff0170 con 0x55a501ffc090 2024-02-02T15:38:39.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.936+0000 7f22fe000700 1 -- 172.21.15.1:0/393059455 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2300005460 con 0x55a501ffc090 2024-02-02T15:38:39.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.936+0000 7f232acc5b80 10 client.28768 fetch_fsmap learned FSMap version 51 2024-02-02T15:38:39.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.936+0000 7f232acc5b80 1 -- 172.21.15.1:0/393059455 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x55a501d7bbe0 con 0x55a501ffc090 2024-02-02T15:38:39.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.936+0000 7f22fe000700 1 -- 172.21.15.1:0/393059455 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a501ff0170 con 0x55a501ffc090 2024-02-02T15:38:39.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.936+0000 7f22fe000700 1 -- 172.21.15.1:0/393059455 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 51) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f230009a3e0 con 0x55a501ffc090 2024-02-02T15:38:39.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.937+0000 7f232acc5b80 10 client.28768 fetch_fsmap finished waiting for FSMap version 51 2024-02-02T15:38:39.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.937+0000 7f232acc5b80 10 client.28768 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:38:39.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.937+0000 7f232acc5b80 20 client.28768 populate_metadata read hostname 'smithi001' 2024-02-02T15:38:39.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.937+0000 7f232acc5b80 1 --2- 172.21.15.1:0/393059455 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55a50201f230 0x55a502021620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:39.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.937+0000 7f232acc5b80 4 client.28768 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:38:39.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.937+0000 7f232acc5b80 1 -- 172.21.15.1:0/393059455 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a501f46e50 con 0x55a50201f230 2024-02-02T15:38:39.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.937+0000 7f230f400700 1 --2- 172.21.15.1:0/393059455 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55a50201f230 0x55a502021620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:39.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.937+0000 7f230f400700 1 --2- 172.21.15.1:0/393059455 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55a50201f230 0x55a502021620 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=774a34a4746527a8 server_cookie=93e4f97eebde7ef5 in_seq=0 out_seq=0 2024-02-02T15:38:39.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.937+0000 7f22fe000700 10 client.28768 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:38:39.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.938+0000 7f22fe000700 1 -- 172.21.15.1:0/393059455 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55a501f46e50 con 0x55a50201f230 2024-02-02T15:38:39.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.938+0000 7f22fe000700 10 client.28768 handle_command_reply: tid=0 2024-02-02T15:38:39.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.938+0000 7f232acc5b80 1 client.28768 shutdown 2024-02-02T15:38:39.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.938+0000 7f232acc5b80 1 -- 172.21.15.1:0/393059455 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a501ffc090 msgr2=0x55a502010450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:39.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.938+0000 7f232acc5b80 1 --2- 172.21.15.1:0/393059455 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a501ffc090 0x55a502010450 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f2300099e80 tx=0x7f230000fa70 comp rx=0 tx=0).stop 2024-02-02T15:38:39.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.938+0000 7f232acc5b80 1 -- 172.21.15.1:0/393059455 shutdown_connections 2024-02-02T15:38:39.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.938+0000 7f232acc5b80 1 --2- 172.21.15.1:0/393059455 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55a50201f230 0x55a502021620 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:39.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.938+0000 7f232acc5b80 1 --2- 172.21.15.1:0/393059455 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a502010070 0x55a502010ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:39.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.938+0000 7f232acc5b80 1 --2- 172.21.15.1:0/393059455 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a501ffc090 0x55a502010450 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:39.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.938+0000 7f232acc5b80 1 --2- 172.21.15.1:0/393059455 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a501ffc9b0 0x55a502010990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:39.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.938+0000 7f22fe000700 0 client.28768 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:38:39.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.938+0000 7f232acc5b80 1 -- 172.21.15.1:0/393059455 >> 172.21.15.1:0/393059455 conn(0x55a501f5be30 msgr2=0x55a501ff8020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:39.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.939+0000 7f232acc5b80 1 -- 172.21.15.1:0/393059455 shutdown_connections 2024-02-02T15:38:39.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.939+0000 7f232acc5b80 1 -- 172.21.15.1:0/393059455 wait complete. 2024-02-02T15:38:39.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.939+0000 7f232acc5b80 20 client.28768 trim_cache size 0 max 16384 2024-02-02T15:38:39.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.958+0000 7f232acc5b80 1 Processor -- start 2024-02-02T15:38:39.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.958+0000 7f232acc5b80 1 -- start start 2024-02-02T15:38:39.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.958+0000 7f232acc5b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a501ffc090 0x55a502014480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:39.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.958+0000 7f232acc5b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a501ffc9b0 0x55a50200c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:39.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.959+0000 7f232acc5b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a502010070 0x55a50200c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:39.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.959+0000 7f232acc5b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a501dd1720 con 0x55a501ffc090 2024-02-02T15:38:39.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.959+0000 7f232acc5b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a501ff9ab0 con 0x55a501ffc9b0 2024-02-02T15:38:39.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.959+0000 7f232acc5b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a501ff2580 con 0x55a502010070 2024-02-02T15:38:39.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.959+0000 7f2315200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a502010070 0x55a50200c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:39.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.959+0000 7f230fe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a501ffc090 0x55a502014480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:39.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.959+0000 7f230fe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a501ffc090 0x55a502014480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37042/0 (socket says 172.21.15.1:37042) 2024-02-02T15:38:39.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.959+0000 7f230fe00700 1 -- 172.21.15.1:0/3105939125 learned_addr learned my addr 172.21.15.1:0/3105939125 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:38:39.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.959+0000 7f230f400700 1 --2- 172.21.15.1:0/3105939125 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a501ffc9b0 0x55a50200c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:39.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.959+0000 7f230fe00700 1 -- 172.21.15.1:0/3105939125 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a502010070 msgr2=0x55a50200c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:39.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.959+0000 7f230fe00700 1 --2- 172.21.15.1:0/3105939125 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a502010070 0x55a50200c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:39.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.959+0000 7f230fe00700 1 -- 172.21.15.1:0/3105939125 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a501ffc9b0 msgr2=0x55a50200c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:39.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.959+0000 7f230fe00700 1 --2- 172.21.15.1:0/3105939125 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a501ffc9b0 0x55a50200c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:39.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.959+0000 7f230fe00700 1 -- 172.21.15.1:0/3105939125 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a501f13f20 con 0x55a501ffc090 2024-02-02T15:38:39.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.959+0000 7f230fe00700 1 --2- 172.21.15.1:0/3105939125 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a501ffc090 0x55a502014480 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f230009a110 tx=0x7f2300021430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8a987b2c174ef6a server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:39.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.960+0000 7f2306000700 1 -- 172.21.15.1:0/3105939125 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23000b8050 con 0x55a501ffc090 2024-02-02T15:38:39.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.960+0000 7f2306000700 1 -- 172.21.15.1:0/3105939125 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f23000964c0 con 0x55a501ffc090 2024-02-02T15:38:39.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.960+0000 7f2306000700 1 -- 172.21.15.1:0/3105939125 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f230009a3e0 con 0x55a501ffc090 2024-02-02T15:38:39.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.961+0000 7f232acc5b80 1 -- 172.21.15.1:0/3105939125 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a501ffc090 msgr2=0x55a502014480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:39.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.961+0000 7f232acc5b80 1 --2- 172.21.15.1:0/3105939125 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a501ffc090 0x55a502014480 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f230009a110 tx=0x7f2300021430 comp rx=0 tx=0).stop 2024-02-02T15:38:39.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.961+0000 7f232acc5b80 1 -- 172.21.15.1:0/3105939125 shutdown_connections 2024-02-02T15:38:39.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.961+0000 7f232acc5b80 1 --2- 172.21.15.1:0/3105939125 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a502010070 0x55a50200c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:39.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.961+0000 7f232acc5b80 1 --2- 172.21.15.1:0/3105939125 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a501ffc090 0x55a502014480 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:39.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.961+0000 7f232acc5b80 1 --2- 172.21.15.1:0/3105939125 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a501ffc9b0 0x55a50200c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:39.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.961+0000 7f232acc5b80 1 -- 172.21.15.1:0/3105939125 >> 172.21.15.1:0/3105939125 conn(0x55a501f5be30 msgr2=0x55a501ff7f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:39.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.961+0000 7f232acc5b80 1 -- 172.21.15.1:0/3105939125 shutdown_connections 2024-02-02T15:38:39.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.961+0000 7f232acc5b80 1 -- 172.21.15.1:0/3105939125 wait complete. 2024-02-02T15:38:39.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.961+0000 7f232acc5b80 1 -- start start 2024-02-02T15:38:39.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.961+0000 7f232acc5b80 1 Processor -- start 2024-02-02T15:38:39.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.962+0000 7f232acc5b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a501ffc090 0x55a50206aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:39.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.962+0000 7f232acc5b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a501ffc9b0 0x55a50206b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:39.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.962+0000 7f232acc5b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a502010070 0x55a502065250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:39.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.962+0000 7f232acc5b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a502065830 con 0x55a501ffc090 2024-02-02T15:38:39.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.962+0000 7f232acc5b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5020659a0 con 0x55a501ffc9b0 2024-02-02T15:38:39.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.962+0000 7f232acc5b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a502065b60 con 0x55a502010070 2024-02-02T15:38:39.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.962+0000 7f232acc5b80 10 client.0 _ll_register_callbacks cb 0x55a501eeb310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:38:39.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.962+0000 7f230fe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a501ffc090 0x55a50206aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:39.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.962+0000 7f2315200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a502010070 0x55a502065250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:39.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.962+0000 7f230fe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a501ffc090 0x55a50206aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37048/0 (socket says 172.21.15.1:37048) 2024-02-02T15:38:39.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.962+0000 7f2315200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a502010070 0x55a502065250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:53834/0 (socket says 172.21.15.1:53834) 2024-02-02T15:38:39.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.962+0000 7f230fe00700 1 -- 172.21.15.1:0/1906002660 learned_addr learned my addr 172.21.15.1:0/1906002660 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:38:39.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.962+0000 7f230f400700 1 --2- 172.21.15.1:0/1906002660 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a501ffc9b0 0x55a50206b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:39.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.963+0000 7f2315200700 1 -- 172.21.15.1:0/1906002660 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a501ffc9b0 msgr2=0x55a50206b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:39.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.963+0000 7f2315200700 1 --2- 172.21.15.1:0/1906002660 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a501ffc9b0 0x55a50206b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:39.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.963+0000 7f2315200700 1 -- 172.21.15.1:0/1906002660 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a501ffc090 msgr2=0x55a50206aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:39.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.963+0000 7f2315200700 1 --2- 172.21.15.1:0/1906002660 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a501ffc090 0x55a50206aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:39.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.963+0000 7f2315200700 1 -- 172.21.15.1:0/1906002660 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a501d7bbe0 con 0x55a502010070 2024-02-02T15:38:39.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.963+0000 7f2315200700 1 --2- 172.21.15.1:0/1906002660 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a502010070 0x55a502065250 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x55a502010040 tx=0x7f2308007ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c641e9db09d7b83 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:39.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.963+0000 7f22fe000700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:38:39.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.963+0000 7f22fe000700 1 -- 172.21.15.1:0/1906002660 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f230800e040 con 0x55a502010070 2024-02-02T15:38:39.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.963+0000 7f232acc5b80 1 -- 172.21.15.1:0/1906002660 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f230009a180 con 0x55a502010070 2024-02-02T15:38:39.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.963+0000 7f22fe000700 1 -- 172.21.15.1:0/1906002660 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f2308004db0 con 0x55a502010070 2024-02-02T15:38:39.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.963+0000 7f22fe000700 1 -- 172.21.15.1:0/1906002660 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2308018a70 con 0x55a502010070 2024-02-02T15:38:39.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.963+0000 7f232acc5b80 10 client.28774 fetch_fsmap learned FSMap version 51 2024-02-02T15:38:39.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.963+0000 7f232acc5b80 1 -- 172.21.15.1:0/1906002660 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x55a501dd3d50 con 0x55a502010070 2024-02-02T15:38:39.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.964+0000 7f22fe000700 1 -- 172.21.15.1:0/1906002660 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f230009a180 con 0x55a502010070 2024-02-02T15:38:39.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.964+0000 7f22fe000700 1 -- 172.21.15.1:0/1906002660 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 51) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f2308020c70 con 0x55a502010070 2024-02-02T15:38:39.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.964+0000 7f232acc5b80 10 client.28774 fetch_fsmap finished waiting for FSMap version 51 2024-02-02T15:38:39.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.964+0000 7f232acc5b80 10 client.28774 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:38:39.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.964+0000 7f232acc5b80 20 client.28774 populate_metadata read hostname 'smithi001' 2024-02-02T15:38:39.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.964+0000 7f232acc5b80 1 --2- 172.21.15.1:0/1906002660 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55a502072e20 0x55a502075240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:39.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.964+0000 7f232acc5b80 4 client.28774 mds_command: new command op to 24557 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-02T15:38:39.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.964+0000 7f232acc5b80 1 -- 172.21.15.1:0/1906002660 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55a501f0cfd0 con 0x55a502072e20 2024-02-02T15:38:39.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.964+0000 7f230fe00700 1 --2- 172.21.15.1:0/1906002660 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55a502072e20 0x55a502075240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:39.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.965+0000 7f230fe00700 1 --2- 172.21.15.1:0/1906002660 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55a502072e20 0x55a502075240 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=625891b5402b6a71 server_cookie=f0c4a1cd37a6755c in_seq=0 out_seq=0 2024-02-02T15:38:39.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.965+0000 7f22fe000700 10 client.28774 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:38:39.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.966+0000 7f22fe000700 1 -- 172.21.15.1:0/1906002660 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55a501f0cfd0 con 0x55a502072e20 2024-02-02T15:38:39.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.966+0000 7f22fe000700 10 client.28774 handle_command_reply: tid=0 2024-02-02T15:38:39.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.966+0000 7f232acc5b80 1 client.28774 shutdown 2024-02-02T15:38:39.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.966+0000 7f232acc5b80 1 -- 172.21.15.1:0/1906002660 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a502010070 msgr2=0x55a502065250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:39.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.966+0000 7f232acc5b80 1 --2- 172.21.15.1:0/1906002660 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a502010070 0x55a502065250 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x55a502010040 tx=0x7f2308007ed0 comp rx=0 tx=0).stop 2024-02-02T15:38:39.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.966+0000 7f232acc5b80 1 -- 172.21.15.1:0/1906002660 shutdown_connections 2024-02-02T15:38:39.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.966+0000 7f232acc5b80 1 --2- 172.21.15.1:0/1906002660 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55a502072e20 0x55a502075240 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:39.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.966+0000 7f232acc5b80 1 --2- 172.21.15.1:0/1906002660 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a502010070 0x55a502065250 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:39.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.966+0000 7f232acc5b80 1 --2- 172.21.15.1:0/1906002660 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a501ffc090 0x55a50206aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:39.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.966+0000 7f232acc5b80 1 --2- 172.21.15.1:0/1906002660 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a501ffc9b0 0x55a50206b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:39.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.966+0000 7f232acc5b80 1 -- 172.21.15.1:0/1906002660 >> 172.21.15.1:0/1906002660 conn(0x55a501f5be30 msgr2=0x55a50200d920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:39.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.966+0000 7f22fe000700 0 client.28774 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:38:39.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.966+0000 7f232acc5b80 1 -- 172.21.15.1:0/1906002660 shutdown_connections 2024-02-02T15:38:39.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.966+0000 7f232acc5b80 1 -- 172.21.15.1:0/1906002660 wait complete. 2024-02-02T15:38:39.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.966+0000 7f232acc5b80 20 client.28774 trim_cache size 0 max 16384 2024-02-02T15:38:39.969 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:38:39.970 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2024-02-02T15:38:39.970 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "e2e74fc8-0b02-4099-8bfb-e8f888461bc9", 2024-02-02T15:38:39.970 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2024-02-02T15:38:39.970 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:38:39.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.967+0000 7f2317200700 1 -- 172.21.15.1:0/2375225057 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f22f00a0710 msgr2=0x7f22f00a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:39.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.967+0000 7f2317200700 1 --2- 172.21.15.1:0/2375225057 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f22f00a0710 0x7f22f00a2bd0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2310101470 tx=0x7f22f8009310 comp rx=0 tx=0).stop 2024-02-02T15:38:39.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.967+0000 7f2317200700 1 -- 172.21.15.1:0/2375225057 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2310102030 msgr2=0x7f23100ffc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:39.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.967+0000 7f2317200700 1 --2- 172.21.15.1:0/2375225057 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2310102030 0x7f23100ffc80 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f2300009810 tx=0x7f230000fa00 comp rx=0 tx=0).stop 2024-02-02T15:38:39.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.967+0000 7f2317200700 1 -- 172.21.15.1:0/2375225057 shutdown_connections 2024-02-02T15:38:39.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.967+0000 7f2317200700 1 --2- 172.21.15.1:0/2375225057 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f22f00a0710 0x7f22f00a2bd0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:39.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.967+0000 7f2317200700 1 --2- 172.21.15.1:0/2375225057 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f231010b400 0x7f23101a50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:39.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.967+0000 7f2317200700 1 --2- 172.21.15.1:0/2375225057 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2310102030 0x7f23100ffc80 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:39.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.967+0000 7f2317200700 1 --2- 172.21.15.1:0/2375225057 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2310102950 0x7f23101001c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:39.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.967+0000 7f2317200700 1 -- 172.21.15.1:0/2375225057 >> 172.21.15.1:0/2375225057 conn(0x7f2310076d70 msgr2=0x7f23100fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:39.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.967+0000 7f2317200700 1 -- 172.21.15.1:0/2375225057 shutdown_connections 2024-02-02T15:38:39.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:39.967+0000 7f2317200700 1 -- 172.21.15.1:0/2375225057 wait complete. 2024-02-02T15:38:39.988 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-02T15:38:40.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.222+0000 7f1dce200700 1 -- 172.21.15.1:0/281412003 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1dc8100ff0 msgr2=0x7f1dc81013d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:40.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.222+0000 7f1dce200700 1 --2- 172.21.15.1:0/281412003 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1dc8100ff0 0x7f1dc81013d0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f1db8009990 tx=0x7f1db8009ca0 comp rx=0 tx=0).stop 2024-02-02T15:38:40.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.222+0000 7f1dce200700 1 -- 172.21.15.1:0/281412003 shutdown_connections 2024-02-02T15:38:40.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.222+0000 7f1dce200700 1 --2- 172.21.15.1:0/281412003 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1dc8106110 0x7f1dc810dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:40.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.222+0000 7f1dce200700 1 --2- 172.21.15.1:0/281412003 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1dc8100ff0 0x7f1dc81013d0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:40.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.222+0000 7f1dce200700 1 --2- 172.21.15.1:0/281412003 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1dc81019a0 0x7f1dc81059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:40.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.222+0000 7f1dce200700 1 -- 172.21.15.1:0/281412003 >> 172.21.15.1:0/281412003 conn(0x7f1dc80fc810 msgr2=0x7f1dc80fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:40.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.222+0000 7f1dce200700 1 -- 172.21.15.1:0/281412003 shutdown_connections 2024-02-02T15:38:40.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.223+0000 7f1dce200700 1 -- 172.21.15.1:0/281412003 wait complete. 2024-02-02T15:38:40.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.224+0000 7f1dce200700 1 Processor -- start 2024-02-02T15:38:40.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.224+0000 7f1dce200700 1 -- start start 2024-02-02T15:38:40.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.225+0000 7f1dce200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1dc8100ff0 0x7f1dc81991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:40.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.225+0000 7f1dce200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1dc8106110 0x7f1dc81996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:40.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.225+0000 7f1dce200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1dc819db50 0x7f1dc819dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:40.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.225+0000 7f1dce200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc819e5d0 con 0x7f1dc8106110 2024-02-02T15:38:40.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.225+0000 7f1dce200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc819e740 con 0x7f1dc819db50 2024-02-02T15:38:40.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.225+0000 7f1dce200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc819e900 con 0x7f1dc8100ff0 2024-02-02T15:38:40.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.225+0000 7f1dc7400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1dc8100ff0 0x7f1dc81991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:40.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.225+0000 7f1dc7400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1dc8100ff0 0x7f1dc81991b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:53846/0 (socket says 172.21.15.1:53846) 2024-02-02T15:38:40.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.225+0000 7f1dc7400700 1 -- 172.21.15.1:0/3522541991 learned_addr learned my addr 172.21.15.1:0/3522541991 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:38:40.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.225+0000 7f1dc6a00700 1 --2- 172.21.15.1:0/3522541991 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1dc8106110 0x7f1dc81996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:40.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.225+0000 7f1dc7e00700 1 --2- 172.21.15.1:0/3522541991 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1dc819db50 0x7f1dc819dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:40.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.225+0000 7f1dc7400700 1 -- 172.21.15.1:0/3522541991 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1dc819db50 msgr2=0x7f1dc819dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:40.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.225+0000 7f1dc7400700 1 --2- 172.21.15.1:0/3522541991 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1dc819db50 0x7f1dc819dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:40.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.226+0000 7f1dc7400700 1 -- 172.21.15.1:0/3522541991 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1dc8106110 msgr2=0x7f1dc81996f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:40.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.226+0000 7f1dc7400700 1 --2- 172.21.15.1:0/3522541991 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1dc8106110 0x7f1dc81996f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:40.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.226+0000 7f1dc7400700 1 -- 172.21.15.1:0/3522541991 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1db80095f0 con 0x7f1dc8100ff0 2024-02-02T15:38:40.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.226+0000 7f1dc7400700 1 --2- 172.21.15.1:0/3522541991 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1dc8100ff0 0x7f1dc81991b0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f1db8005910 tx=0x7f1db8009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:40.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.226+0000 7f1dbfe00700 1 -- 172.21.15.1:0/3522541991 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1db801d070 con 0x7f1dc8100ff0 2024-02-02T15:38:40.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.226+0000 7f1dbfe00700 1 -- 172.21.15.1:0/3522541991 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1db8003f60 con 0x7f1dc8100ff0 2024-02-02T15:38:40.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.226+0000 7f1dce200700 1 -- 172.21.15.1:0/3522541991 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dc81a5e00 con 0x7f1dc8100ff0 2024-02-02T15:38:40.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.226+0000 7f1dbfe00700 1 -- 172.21.15.1:0/3522541991 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1db8022c80 con 0x7f1dc8100ff0 2024-02-02T15:38:40.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.226+0000 7f1dce200700 1 -- 172.21.15.1:0/3522541991 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dc81a6220 con 0x7f1dc8100ff0 2024-02-02T15:38:40.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.229+0000 7f1dbfe00700 1 -- 172.21.15.1:0/3522541991 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f1db80040d0 con 0x7f1dc8100ff0 2024-02-02T15:38:40.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.231+0000 7f1de1c62b80 1 Processor -- start 2024-02-02T15:38:40.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.231+0000 7f1de1c62b80 1 -- start start 2024-02-02T15:38:40.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.231+0000 7f1dbfe00700 1 --2- 172.21.15.1:0/3522541991 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f1da80a0710 0x7f1da80a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:40.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.232+0000 7f1de1c62b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7668f0010 0x55d7668f03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:40.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.232+0000 7f1de1c62b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7668f0930 0x55d7668fb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:40.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.232+0000 7f1dbfe00700 1 -- 172.21.15.1:0/3522541991 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f1db80cd550 con 0x7f1dc8100ff0 2024-02-02T15:38:40.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.232+0000 7f1de1c62b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7668fb7e0 0x55d7668fdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:40.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.232+0000 7f1de1c62b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7668644c0 con 0x55d7668f0930 2024-02-02T15:38:40.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.232+0000 7f1de1c62b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d766848aa0 con 0x55d7668f0010 2024-02-02T15:38:40.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.232+0000 7f1de1c62b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d766846460 con 0x55d7668fb7e0 2024-02-02T15:38:40.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.232+0000 7f1dc7e00700 1 --2- 172.21.15.1:0/3522541991 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f1da80a0710 0x7f1da80a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:40.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.232+0000 7f1dc6a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7668f0930 0x55d7668fb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:40.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.232+0000 7f1dc7400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7668f0010 0x55d7668f03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:40.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.232+0000 7f1dc6a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7668f0930 0x55d7668fb2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37082/0 (socket says 172.21.15.1:37082) 2024-02-02T15:38:40.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.232+0000 7f1dc6a00700 1 -- 172.21.15.1:0/2125240619 learned_addr learned my addr 172.21.15.1:0/2125240619 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:38:40.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.232+0000 7f1dc7e00700 1 --2- 172.21.15.1:0/2125240619 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7668fb7e0 0x55d7668fdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:40.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.232+0000 7f1dc7400700 1 -- 172.21.15.1:0/2125240619 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7668fb7e0 msgr2=0x55d7668fdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:40.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.232+0000 7f1dc7400700 1 --2- 172.21.15.1:0/2125240619 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7668fb7e0 0x55d7668fdbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:40.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.233+0000 7f1dc7400700 1 -- 172.21.15.1:0/2125240619 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7668f0930 msgr2=0x55d7668fb2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:40.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.233+0000 7f1dc7400700 1 --2- 172.21.15.1:0/2125240619 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7668f0930 0x55d7668fb2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:40.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.233+0000 7f1dc7400700 1 -- 172.21.15.1:0/2125240619 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d7666c33b0 con 0x55d7668f0010 2024-02-02T15:38:40.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.233+0000 7f1dc7e00700 1 --2- 172.21.15.1:0/3522541991 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f1da80a0710 0x7f1da80a2bd0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0009550 tx=0x7f1dc00079a0 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:40.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.233+0000 7f1dc7400700 1 --2- 172.21.15.1:0/2125240619 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7668f0010 0x55d7668f03f0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f1db80ca870 tx=0x7f1db8009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4fc40bee60f3b0c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:40.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.233+0000 7f1db7e00700 1 -- 172.21.15.1:0/2125240619 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1db809a820 con 0x55d7668f0010 2024-02-02T15:38:40.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.233+0000 7f1db7e00700 1 -- 172.21.15.1:0/2125240619 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1db809e650 con 0x55d7668f0010 2024-02-02T15:38:40.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.233+0000 7f1db7e00700 1 -- 172.21.15.1:0/2125240619 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1db809ace0 con 0x55d7668f0010 2024-02-02T15:38:40.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.234+0000 7f1de1c62b80 1 -- 172.21.15.1:0/2125240619 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7668f0010 msgr2=0x55d7668f03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:40.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.234+0000 7f1de1c62b80 1 --2- 172.21.15.1:0/2125240619 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7668f0010 0x55d7668f03f0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f1db80ca870 tx=0x7f1db8009ca0 comp rx=0 tx=0).stop 2024-02-02T15:38:40.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.234+0000 7f1de1c62b80 1 -- 172.21.15.1:0/2125240619 shutdown_connections 2024-02-02T15:38:40.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.234+0000 7f1de1c62b80 1 --2- 172.21.15.1:0/2125240619 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7668fb7e0 0x55d7668fdbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:40.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.234+0000 7f1de1c62b80 1 --2- 172.21.15.1:0/2125240619 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7668f0010 0x55d7668f03f0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:40.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.234+0000 7f1de1c62b80 1 --2- 172.21.15.1:0/2125240619 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7668f0930 0x55d7668fb2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:40.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.234+0000 7f1de1c62b80 1 -- 172.21.15.1:0/2125240619 >> 172.21.15.1:0/2125240619 conn(0x55d76684fdb0 msgr2=0x55d7668ec3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:40.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.234+0000 7f1de1c62b80 1 -- 172.21.15.1:0/2125240619 shutdown_connections 2024-02-02T15:38:40.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.234+0000 7f1de1c62b80 1 -- 172.21.15.1:0/2125240619 wait complete. 2024-02-02T15:38:40.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.234+0000 7f1de1c62b80 1 -- start start 2024-02-02T15:38:40.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.235+0000 7f1de1c62b80 1 Processor -- start 2024-02-02T15:38:40.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.235+0000 7f1de1c62b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7668f0010 0x55d7668fbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:40.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.235+0000 7f1de1c62b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7668f0930 0x55d7668fc100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:40.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.235+0000 7f1de1c62b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7668fb7e0 0x55d7668fc640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:40.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.235+0000 7f1de1c62b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d766841d50 con 0x55d7668f0010 2024-02-02T15:38:40.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.235+0000 7f1de1c62b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7667ddd80 con 0x55d7668f0930 2024-02-02T15:38:40.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.235+0000 7f1de1c62b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7667dda20 con 0x55d7668fb7e0 2024-02-02T15:38:40.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.235+0000 7f1de1c62b80 10 client.0 _ll_register_callbacks cb 0x55d7667df290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:38:40.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.235+0000 7f1dc7400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7668f0010 0x55d7668fbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:40.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.235+0000 7f1dc7400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7668f0010 0x55d7668fbbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37098/0 (socket says 172.21.15.1:37098) 2024-02-02T15:38:40.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.235+0000 7f1dc7400700 1 -- 172.21.15.1:0/2325708364 learned_addr learned my addr 172.21.15.1:0/2325708364 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:38:40.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.236+0000 7f1dc7e00700 1 --2- 172.21.15.1:0/2325708364 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7668fb7e0 0x55d7668fc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:40.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.236+0000 7f1dc6a00700 1 --2- 172.21.15.1:0/2325708364 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7668f0930 0x55d7668fc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:40.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.236+0000 7f1dc7400700 1 -- 172.21.15.1:0/2325708364 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7668fb7e0 msgr2=0x55d7668fc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:40.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.236+0000 7f1dc7400700 1 --2- 172.21.15.1:0/2325708364 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7668fb7e0 0x55d7668fc640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:40.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.236+0000 7f1dc7400700 1 -- 172.21.15.1:0/2325708364 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7668f0930 msgr2=0x55d7668fc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:40.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.236+0000 7f1dc7400700 1 --2- 172.21.15.1:0/2325708364 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7668f0930 0x55d7668fc100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:40.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.236+0000 7f1dc7400700 1 -- 172.21.15.1:0/2325708364 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dc81a6220 con 0x55d7668f0010 2024-02-02T15:38:40.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.236+0000 7f1dc7400700 1 --2- 172.21.15.1:0/2325708364 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7668f0010 0x55d7668fbbc0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f1db8000c00 tx=0x7f1db80226a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=346b85ad2c7ce413 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:40.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.236+0000 7f1db4c00700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:38:40.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.236+0000 7f1db4c00700 1 -- 172.21.15.1:0/2325708364 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1db809a820 con 0x55d7668f0010 2024-02-02T15:38:40.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.236+0000 7f1db4c00700 1 -- 172.21.15.1:0/2325708364 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f1db80b8070 con 0x55d7668f0010 2024-02-02T15:38:40.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.236+0000 7f1de1c62b80 1 -- 172.21.15.1:0/2325708364 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d7668e40f0 con 0x55d7668f0010 2024-02-02T15:38:40.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.237+0000 7f1db4c00700 1 -- 172.21.15.1:0/2325708364 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1db8099f20 con 0x55d7668f0010 2024-02-02T15:38:40.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.237+0000 7f1de1c62b80 10 client.18687 fetch_fsmap learned FSMap version 51 2024-02-02T15:38:40.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.237+0000 7f1de1c62b80 1 -- 172.21.15.1:0/2325708364 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x55d76666fb60 con 0x55d7668f0010 2024-02-02T15:38:40.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.237+0000 7f1db4c00700 1 -- 172.21.15.1:0/2325708364 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d7668e40f0 con 0x55d7668f0010 2024-02-02T15:38:40.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.237+0000 7f1db4c00700 1 -- 172.21.15.1:0/2325708364 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 51) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f1db8081070 con 0x55d7668f0010 2024-02-02T15:38:40.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.237+0000 7f1de1c62b80 10 client.18687 fetch_fsmap finished waiting for FSMap version 51 2024-02-02T15:38:40.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.237+0000 7f1de1c62b80 10 client.18687 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:38:40.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.237+0000 7f1de1c62b80 20 client.18687 populate_metadata read hostname 'smithi001' 2024-02-02T15:38:40.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.237+0000 7f1de1c62b80 1 --2- 172.21.15.1:0/2325708364 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55d76690aa30 0x55d76690ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:40.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.237+0000 7f1de1c62b80 4 client.18687 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:38:40.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.237+0000 7f1de1c62b80 1 -- 172.21.15.1:0/2325708364 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d76683add0 con 0x55d76690aa30 2024-02-02T15:38:40.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.237+0000 7f1dc6a00700 1 --2- 172.21.15.1:0/2325708364 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55d76690aa30 0x55d76690ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:40.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.238+0000 7f1dc6a00700 1 --2- 172.21.15.1:0/2325708364 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55d76690aa30 0x55d76690ce50 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f683bdf6ac4e1d66 server_cookie=6cd110ce700dd63 in_seq=0 out_seq=0 2024-02-02T15:38:40.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.238+0000 7f1db4c00700 10 client.18687 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:38:40.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.238+0000 7f1db4c00700 1 -- 172.21.15.1:0/2325708364 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55d76683add0 con 0x55d76690aa30 2024-02-02T15:38:40.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.238+0000 7f1db4c00700 10 client.18687 handle_command_reply: tid=0 2024-02-02T15:38:40.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.238+0000 7f1de1c62b80 1 client.18687 shutdown 2024-02-02T15:38:40.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.239+0000 7f1de1c62b80 1 -- 172.21.15.1:0/2325708364 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7668f0010 msgr2=0x55d7668fbbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:40.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.239+0000 7f1de1c62b80 1 --2- 172.21.15.1:0/2325708364 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7668f0010 0x55d7668fbbc0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f1db8000c00 tx=0x7f1db80226a0 comp rx=0 tx=0).stop 2024-02-02T15:38:40.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.239+0000 7f1de1c62b80 1 -- 172.21.15.1:0/2325708364 shutdown_connections 2024-02-02T15:38:40.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.239+0000 7f1de1c62b80 1 --2- 172.21.15.1:0/2325708364 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55d76690aa30 0x55d76690ce50 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:40.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.239+0000 7f1de1c62b80 1 --2- 172.21.15.1:0/2325708364 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7668fb7e0 0x55d7668fc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:40.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.239+0000 7f1de1c62b80 1 --2- 172.21.15.1:0/2325708364 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7668f0010 0x55d7668fbbc0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:40.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.239+0000 7f1de1c62b80 1 --2- 172.21.15.1:0/2325708364 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7668f0930 0x55d7668fc100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:40.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.239+0000 7f1db4c00700 0 client.18687 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:38:40.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.239+0000 7f1de1c62b80 1 -- 172.21.15.1:0/2325708364 >> 172.21.15.1:0/2325708364 conn(0x55d76684fdb0 msgr2=0x55d7668ebfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:40.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.239+0000 7f1de1c62b80 1 -- 172.21.15.1:0/2325708364 shutdown_connections 2024-02-02T15:38:40.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.239+0000 7f1de1c62b80 1 -- 172.21.15.1:0/2325708364 wait complete. 2024-02-02T15:38:40.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.239+0000 7f1de1c62b80 20 client.18687 trim_cache size 0 max 16384 2024-02-02T15:38:40.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.256+0000 7f1de1c62b80 1 Processor -- start 2024-02-02T15:38:40.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.256+0000 7f1de1c62b80 1 -- start start 2024-02-02T15:38:40.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.257+0000 7f1de1c62b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7668f0010 0x55d766900bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:40.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.257+0000 7f1de1c62b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7668f0930 0x55d7668ebeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:40.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.257+0000 7f1de1c62b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7668fb7e0 0x55d7668ec3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:40.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.257+0000 7f1de1c62b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7666c56a0 con 0x55d7668f0930 2024-02-02T15:38:40.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.257+0000 7f1de1c62b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7668fa750 con 0x55d7668fb7e0 2024-02-02T15:38:40.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.257+0000 7f1de1c62b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7668f6680 con 0x55d7668f0010 2024-02-02T15:38:40.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.257+0000 7f1dc7400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7668f0010 0x55d766900bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:40.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.257+0000 7f1dc6a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7668f0930 0x55d7668ebeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:40.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.257+0000 7f1dc7400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7668f0010 0x55d766900bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:53878/0 (socket says 172.21.15.1:53878) 2024-02-02T15:38:40.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.257+0000 7f1dc7400700 1 -- 172.21.15.1:0/3561572743 learned_addr learned my addr 172.21.15.1:0/3561572743 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:38:40.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.257+0000 7f1dc7e00700 1 --2- 172.21.15.1:0/3561572743 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7668fb7e0 0x55d7668ec3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:40.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.257+0000 7f1dc7400700 1 -- 172.21.15.1:0/3561572743 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7668fb7e0 msgr2=0x55d7668ec3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:40.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.257+0000 7f1dc7400700 1 --2- 172.21.15.1:0/3561572743 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7668fb7e0 0x55d7668ec3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:40.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.257+0000 7f1dc7400700 1 -- 172.21.15.1:0/3561572743 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7668f0930 msgr2=0x55d7668ebeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:40.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.257+0000 7f1dc7400700 1 --2- 172.21.15.1:0/3561572743 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7668f0930 0x55d7668ebeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:40.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.257+0000 7f1dc7400700 1 -- 172.21.15.1:0/3561572743 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d766807ea0 con 0x55d7668f0010 2024-02-02T15:38:40.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.257+0000 7f1dc7e00700 1 --2- 172.21.15.1:0/3561572743 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7668fb7e0 0x55d7668ec3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:38:40.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.257+0000 7f1dc6a00700 1 --2- 172.21.15.1:0/3561572743 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7668f0930 0x55d7668ebeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:38:40.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.258+0000 7f1dc7400700 1 --2- 172.21.15.1:0/3561572743 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7668f0010 0x55d766900bd0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f1db80057f0 tx=0x7f1db80cae00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b964efcbbde58054 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:40.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.258+0000 7f1dbd200700 1 -- 172.21.15.1:0/3561572743 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1db80a2880 con 0x55d7668f0010 2024-02-02T15:38:40.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.258+0000 7f1dbd200700 1 -- 172.21.15.1:0/3561572743 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1db809a5f0 con 0x55d7668f0010 2024-02-02T15:38:40.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.258+0000 7f1dbd200700 1 -- 172.21.15.1:0/3561572743 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1db809ace0 con 0x55d7668f0010 2024-02-02T15:38:40.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.258+0000 7f1de1c62b80 1 -- 172.21.15.1:0/3561572743 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7668f0010 msgr2=0x55d766900bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:40.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.258+0000 7f1de1c62b80 1 --2- 172.21.15.1:0/3561572743 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7668f0010 0x55d766900bd0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f1db80057f0 tx=0x7f1db80cae00 comp rx=0 tx=0).stop 2024-02-02T15:38:40.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.258+0000 7f1de1c62b80 1 -- 172.21.15.1:0/3561572743 shutdown_connections 2024-02-02T15:38:40.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.258+0000 7f1de1c62b80 1 --2- 172.21.15.1:0/3561572743 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7668fb7e0 0x55d7668ec3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:40.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.258+0000 7f1de1c62b80 1 --2- 172.21.15.1:0/3561572743 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7668f0010 0x55d766900bd0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:40.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.258+0000 7f1de1c62b80 1 --2- 172.21.15.1:0/3561572743 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7668f0930 0x55d7668ebeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:40.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.258+0000 7f1de1c62b80 1 -- 172.21.15.1:0/3561572743 >> 172.21.15.1:0/3561572743 conn(0x55d76684fdb0 msgr2=0x55d7668f5bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:40.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.259+0000 7f1de1c62b80 1 -- 172.21.15.1:0/3561572743 shutdown_connections 2024-02-02T15:38:40.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.259+0000 7f1de1c62b80 1 -- 172.21.15.1:0/3561572743 wait complete. 2024-02-02T15:38:40.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.259+0000 7f1de1c62b80 1 -- start start 2024-02-02T15:38:40.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.259+0000 7f1de1c62b80 1 Processor -- start 2024-02-02T15:38:40.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.260+0000 7f1de1c62b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7668f0010 0x55d7668fe840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:40.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.260+0000 7f1de1c62b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7668f0930 0x55d7668fed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:40.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.260+0000 7f1de1c62b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7668fb7e0 0x55d766955550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:40.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.260+0000 7f1de1c62b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d766955a90 con 0x55d7668f0010 2024-02-02T15:38:40.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.260+0000 7f1de1c62b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d766955c00 con 0x55d7668fb7e0 2024-02-02T15:38:40.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.260+0000 7f1de1c62b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d766955dc0 con 0x55d7668f0930 2024-02-02T15:38:40.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.260+0000 7f1de1c62b80 10 client.0 _ll_register_callbacks cb 0x55d7667df290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:38:40.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.260+0000 7f1dc7400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7668f0010 0x55d7668fe840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:40.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.260+0000 7f1dc7400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7668f0010 0x55d7668fe840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:37118/0 (socket says 172.21.15.1:37118) 2024-02-02T15:38:40.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.260+0000 7f1dc7400700 1 -- 172.21.15.1:0/4092842101 learned_addr learned my addr 172.21.15.1:0/4092842101 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:38:40.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.260+0000 7f1dc6a00700 1 --2- 172.21.15.1:0/4092842101 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7668f0930 0x55d7668fed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:40.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.260+0000 7f1dc7e00700 1 --2- 172.21.15.1:0/4092842101 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7668fb7e0 0x55d766955550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:40.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.261+0000 7f1dc7400700 1 -- 172.21.15.1:0/4092842101 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7668f0930 msgr2=0x55d7668fed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:40.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.261+0000 7f1dc7400700 1 --2- 172.21.15.1:0/4092842101 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7668f0930 0x55d7668fed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:40.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.261+0000 7f1dc7400700 1 -- 172.21.15.1:0/4092842101 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7668fb7e0 msgr2=0x55d766955550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:38:40.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.261+0000 7f1dc7400700 1 --2- 172.21.15.1:0/4092842101 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7668fb7e0 0x55d766955550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:40.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.261+0000 7f1dc7400700 1 -- 172.21.15.1:0/4092842101 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1db0007bb0 con 0x55d7668f0010 2024-02-02T15:38:40.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.261+0000 7f1dc6a00700 1 --2- 172.21.15.1:0/4092842101 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7668f0930 0x55d7668fed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:38:40.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.261+0000 7f1dc7400700 1 --2- 172.21.15.1:0/4092842101 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7668f0010 0x55d7668fe840 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f1db80a2770 tx=0x7f1db80996d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=217fce8d95a4c310 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:38:40.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.261+0000 7f1db4c00700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:38:40.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.261+0000 7f1db4c00700 1 -- 172.21.15.1:0/4092842101 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1db809e290 con 0x55d7668f0010 2024-02-02T15:38:40.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.261+0000 7f1de1c62b80 1 -- 172.21.15.1:0/4092842101 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1db809e650 con 0x55d7668f0010 2024-02-02T15:38:40.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.261+0000 7f1db4c00700 1 -- 172.21.15.1:0/4092842101 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f1db80b8040 con 0x55d7668f0010 2024-02-02T15:38:40.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.262+0000 7f1db4c00700 1 -- 172.21.15.1:0/4092842101 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1db809a820 con 0x55d7668f0010 2024-02-02T15:38:40.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.262+0000 7f1de1c62b80 10 client.18693 fetch_fsmap learned FSMap version 51 2024-02-02T15:38:40.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.262+0000 7f1de1c62b80 1 -- 172.21.15.1:0/4092842101 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x55d76666fb60 con 0x55d7668f0010 2024-02-02T15:38:40.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.262+0000 7f1db4c00700 1 -- 172.21.15.1:0/4092842101 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1db808b080 con 0x55d7668f0010 2024-02-02T15:38:40.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.262+0000 7f1db4c00700 1 -- 172.21.15.1:0/4092842101 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 51) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f1db8096f30 con 0x55d7668f0010 2024-02-02T15:38:40.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.262+0000 7f1de1c62b80 10 client.18693 fetch_fsmap finished waiting for FSMap version 51 2024-02-02T15:38:40.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.262+0000 7f1de1c62b80 10 client.18693 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:38:40.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.262+0000 7f1de1c62b80 20 client.18693 populate_metadata read hostname 'smithi001' 2024-02-02T15:38:40.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.262+0000 7f1de1c62b80 1 --2- 172.21.15.1:0/4092842101 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55d76695d020 0x55d76695f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:38:40.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.262+0000 7f1de1c62b80 4 client.18693 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2024-02-02T15:38:40.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.262+0000 7f1de1c62b80 1 -- 172.21.15.1:0/4092842101 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d766800f50 con 0x55d76695d020 2024-02-02T15:38:40.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.262+0000 7f1dc6a00700 1 --2- 172.21.15.1:0/4092842101 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55d76695d020 0x55d76695f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:38:40.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.263+0000 7f1dc6a00700 1 --2- 172.21.15.1:0/4092842101 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55d76695d020 0x55d76695f410 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=857e71584a8f1fe9 server_cookie=a88864a582f08267 in_seq=0 out_seq=0 2024-02-02T15:38:40.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.263+0000 7f1db4c00700 10 client.18693 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:38:40.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.263+0000 7f1db4c00700 1 -- 172.21.15.1:0/4092842101 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55d766800f50 con 0x55d76695d020 2024-02-02T15:38:40.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.263+0000 7f1db4c00700 10 client.18693 handle_command_reply: tid=0 2024-02-02T15:38:40.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.263+0000 7f1de1c62b80 1 client.18693 shutdown 2024-02-02T15:38:40.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.263+0000 7f1de1c62b80 1 -- 172.21.15.1:0/4092842101 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7668f0010 msgr2=0x55d7668fe840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:40.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.263+0000 7f1de1c62b80 1 --2- 172.21.15.1:0/4092842101 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7668f0010 0x55d7668fe840 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f1db80a2770 tx=0x7f1db80996d0 comp rx=0 tx=0).stop 2024-02-02T15:38:40.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.263+0000 7f1de1c62b80 1 -- 172.21.15.1:0/4092842101 shutdown_connections 2024-02-02T15:38:40.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.263+0000 7f1de1c62b80 1 --2- 172.21.15.1:0/4092842101 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55d76695d020 0x55d76695f410 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:40.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.263+0000 7f1de1c62b80 1 --2- 172.21.15.1:0/4092842101 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7668fb7e0 0x55d766955550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:40.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.263+0000 7f1de1c62b80 1 --2- 172.21.15.1:0/4092842101 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d7668f0010 0x55d7668fe840 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:40.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.263+0000 7f1db4c00700 0 client.18693 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:38:40.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.263+0000 7f1de1c62b80 1 --2- 172.21.15.1:0/4092842101 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7668f0930 0x55d7668fed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:40.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.263+0000 7f1de1c62b80 1 -- 172.21.15.1:0/4092842101 >> 172.21.15.1:0/4092842101 conn(0x55d76684fdb0 msgr2=0x55d7668f9620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:40.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.263+0000 7f1de1c62b80 1 -- 172.21.15.1:0/4092842101 shutdown_connections 2024-02-02T15:38:40.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.263+0000 7f1de1c62b80 1 -- 172.21.15.1:0/4092842101 wait complete. 2024-02-02T15:38:40.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.264+0000 7f1de1c62b80 20 client.18693 trim_cache size 0 max 16384 2024-02-02T15:38:40.269 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:38:40.269 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-02T15:38:40.269 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2024-02-02T15:38:40.269 INFO:teuthology.orchestra.run.smithi001.stdout: "e2e74fc8-0b02-4099-8bfb-e8f888461bc9": { 2024-02-02T15:38:40.269 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2024-02-02T15:38:40.269 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "e2e74fc8-0b02-4099-8bfb-e8f888461bc9", 2024-02-02T15:38:40.269 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,force" 2024-02-02T15:38:40.269 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:38:40.269 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:38:40.269 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:38:40.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.264+0000 7f1dce200700 1 -- 172.21.15.1:0/3522541991 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f1da80a0710 msgr2=0x7f1da80a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:40.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.264+0000 7f1dce200700 1 --2- 172.21.15.1:0/3522541991 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f1da80a0710 0x7f1da80a2bd0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0009550 tx=0x7f1dc00079a0 comp rx=0 tx=0).stop 2024-02-02T15:38:40.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.264+0000 7f1dce200700 1 -- 172.21.15.1:0/3522541991 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1dc8100ff0 msgr2=0x7f1dc81991b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:38:40.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.264+0000 7f1dce200700 1 --2- 172.21.15.1:0/3522541991 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1dc8100ff0 0x7f1dc81991b0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f1db8005910 tx=0x7f1db8009580 comp rx=0 tx=0).stop 2024-02-02T15:38:40.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.264+0000 7f1dce200700 1 -- 172.21.15.1:0/3522541991 shutdown_connections 2024-02-02T15:38:40.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.264+0000 7f1dce200700 1 --2- 172.21.15.1:0/3522541991 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f1da80a0710 0x7f1da80a2bd0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:40.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.264+0000 7f1dce200700 1 --2- 172.21.15.1:0/3522541991 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1dc819db50 0x7f1dc819dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:40.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.264+0000 7f1dce200700 1 --2- 172.21.15.1:0/3522541991 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1dc8100ff0 0x7f1dc81991b0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:40.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.264+0000 7f1dce200700 1 --2- 172.21.15.1:0/3522541991 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1dc8106110 0x7f1dc81996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:38:40.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.264+0000 7f1dce200700 1 -- 172.21.15.1:0/3522541991 >> 172.21.15.1:0/3522541991 conn(0x7f1dc80fc810 msgr2=0x7f1dc80fd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:38:40.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.264+0000 7f1dce200700 1 -- 172.21.15.1:0/3522541991 shutdown_connections 2024-02-02T15:38:40.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:38:40.264+0000 7f1dce200700 1 -- 172.21.15.1:0/3522541991 wait complete. 2024-02-02T15:38:40.271 INFO:tasks.workunit.client.0.smithi001.stdout: 1 644492 5.54 MB/sec execute 494 sec latency 112.861 ms 2024-02-02T15:38:40.281 INFO:tasks.cephfs.filesystem:scrub status for tag:e2e74fc8-0b02-4099-8bfb-e8f888461bc9 - {'path': '/', 'tag': 'e2e74fc8-0b02-4099-8bfb-e8f888461bc9', 'options': 'recursive,force'} 2024-02-02T15:38:40.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:40 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:38:40.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:40 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:38:40.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:40 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:38:40.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:40 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:38:40.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:40 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:38:40.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:40 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:38:40.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:40 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:38:40.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:40 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:38:40.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:40 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:38:40.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:40 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:38:40.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:40 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:38:40.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:40 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:38:40.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:40 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:38:40.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:40 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:38:40.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:40 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:38:40.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:40 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:38:40.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:40 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:38:40.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:40 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:38:40.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:40 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:38:40.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:40 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:38:40.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:40 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:38:40.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:40 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:38:40.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:40 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:38:40.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:40 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:38:41.268 INFO:tasks.workunit.client.0.smithi001.stdout: 1 644492 5.52 MB/sec execute 495 sec latency 1112.928 ms 2024-02-02T15:38:41.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:41 smithi071 ceph-mon[44707]: scrub queued for path: / 2024-02-02T15:38:41.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:41 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:38:41.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:41 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:38:41.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:41 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:38:41.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:41 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:38:41.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:41 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:38:41.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:41 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:38:41.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:41 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:38:41.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:41 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:38:41.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:41 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:38:41.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:41 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:38:41.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:41 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:38:41.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:41 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:38:41.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:41 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:38:41.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:41 smithi071 ceph-mon[44707]: pgmap v363: 129 pgs: 129 active+clean; 528 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 115 op/s 2024-02-02T15:38:41.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:41 smithi142 ceph-mon[45728]: scrub queued for path: / 2024-02-02T15:38:41.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:41 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:38:41.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:41 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:38:41.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:41 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:38:41.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:41 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:38:41.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:41 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:38:41.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:41 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:38:41.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:41 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:38:41.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:41 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:38:41.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:41 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:38:41.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:41 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:38:41.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:41 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:38:41.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:41 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:38:41.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:41 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:38:41.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:41 smithi142 ceph-mon[45728]: pgmap v363: 129 pgs: 129 active+clean; 528 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 115 op/s 2024-02-02T15:38:41.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:41 smithi001 ceph-mon[35470]: scrub queued for path: / 2024-02-02T15:38:41.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:41 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:38:41.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:41 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:38:41.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:41 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:38:41.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:41 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:38:41.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:41 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:38:41.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:41 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:38:41.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:41 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:38:41.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:41 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:38:41.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:41 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:38:41.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:41 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:38:41.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:41 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:38:41.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:41 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:38:41.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:41 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:38:41.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:41 smithi001 ceph-mon[35470]: pgmap v363: 129 pgs: 129 active+clean; 528 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 115 op/s 2024-02-02T15:38:42.268 INFO:tasks.workunit.client.0.smithi001.stdout: 1 644492 5.51 MB/sec execute 496 sec latency 2113.069 ms 2024-02-02T15:38:43.268 INFO:tasks.workunit.client.0.smithi001.stdout: 1 645714 5.52 MB/sec execute 497 sec latency 2494.868 ms 2024-02-02T15:38:43.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:43 smithi071 ceph-mon[44707]: pgmap v364: 129 pgs: 129 active+clean; 533 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 174 op/s 2024-02-02T15:38:43.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:43 smithi142 ceph-mon[45728]: pgmap v364: 129 pgs: 129 active+clean; 533 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 174 op/s 2024-02-02T15:38:43.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:43 smithi001 ceph-mon[35470]: pgmap v364: 129 pgs: 129 active+clean; 533 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 174 op/s 2024-02-02T15:38:44.268 INFO:tasks.workunit.client.0.smithi001.stdout: 1 646892 5.52 MB/sec execute 498 sec latency 18.164 ms 2024-02-02T15:38:45.268 INFO:tasks.workunit.client.0.smithi001.stdout: 1 648042 5.52 MB/sec execute 499 sec latency 130.808 ms 2024-02-02T15:38:45.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:45 smithi071 ceph-mon[44707]: pgmap v365: 129 pgs: 129 active+clean; 533 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 114 op/s 2024-02-02T15:38:45.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:45 smithi142 ceph-mon[45728]: pgmap v365: 129 pgs: 129 active+clean; 533 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 114 op/s 2024-02-02T15:38:45.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:45 smithi001 ceph-mon[35470]: pgmap v365: 129 pgs: 129 active+clean; 533 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 114 op/s 2024-02-02T15:38:46.268 INFO:tasks.workunit.client.0.smithi001.stdout: 1 648746 5.51 MB/sec execute 500 sec latency 784.379 ms 2024-02-02T15:38:47.268 INFO:tasks.workunit.client.0.smithi001.stdout: 1 650219 5.52 MB/sec execute 501 sec latency 14.526 ms 2024-02-02T15:38:47.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:47 smithi071 ceph-mon[44707]: pgmap v366: 129 pgs: 129 active+clean; 539 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 173 op/s 2024-02-02T15:38:47.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:47 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:38:47.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:47 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:38:47.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:47 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:38:47.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:47 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:38:47.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:47 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:38:47.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:47 smithi142 ceph-mon[45728]: pgmap v366: 129 pgs: 129 active+clean; 539 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 173 op/s 2024-02-02T15:38:47.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:47 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:38:47.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:47 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:38:47.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:47 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:38:47.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:47 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:38:47.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:47 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:38:47.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:47 smithi001 ceph-mon[35470]: pgmap v366: 129 pgs: 129 active+clean; 539 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 173 op/s 2024-02-02T15:38:47.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:47 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:38:47.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:47 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:38:47.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:47 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:38:47.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:47 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:38:47.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:47 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:38:48.268 INFO:tasks.workunit.client.0.smithi001.stdout: 1 651363 5.52 MB/sec execute 502 sec latency 19.607 ms 2024-02-02T15:38:49.268 INFO:tasks.workunit.client.0.smithi001.stdout: 1 651583 5.51 MB/sec execute 503 sec latency 795.440 ms 2024-02-02T15:38:49.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:49 smithi071 ceph-mon[44707]: pgmap v367: 129 pgs: 129 active+clean; 539 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 117 op/s 2024-02-02T15:38:49.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:49 smithi142 ceph-mon[45728]: pgmap v367: 129 pgs: 129 active+clean; 539 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 117 op/s 2024-02-02T15:38:49.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:49 smithi001 ceph-mon[35470]: pgmap v367: 129 pgs: 129 active+clean; 539 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 117 op/s 2024-02-02T15:38:50.269 INFO:tasks.workunit.client.0.smithi001.stdout: 1 651583 5.50 MB/sec execute 504 sec latency 1795.546 ms 2024-02-02T15:38:51.269 INFO:tasks.workunit.client.0.smithi001.stdout: 1 651583 5.49 MB/sec execute 505 sec latency 2795.810 ms 2024-02-02T15:38:51.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:51 smithi071 ceph-mon[44707]: pgmap v368: 129 pgs: 129 active+clean; 539 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 117 op/s 2024-02-02T15:38:51.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:51 smithi142 ceph-mon[45728]: pgmap v368: 129 pgs: 129 active+clean; 539 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 117 op/s 2024-02-02T15:38:51.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:51 smithi001 ceph-mon[35470]: pgmap v368: 129 pgs: 129 active+clean; 539 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 117 op/s 2024-02-02T15:38:52.269 INFO:tasks.workunit.client.0.smithi001.stdout: 1 651583 5.48 MB/sec execute 506 sec latency 3795.938 ms 2024-02-02T15:38:53.269 INFO:tasks.workunit.client.0.smithi001.stdout: 1 652820 5.48 MB/sec execute 507 sec latency 4175.362 ms 2024-02-02T15:38:53.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:53 smithi071 ceph-mon[44707]: pgmap v369: 129 pgs: 129 active+clean; 540 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.9 MiB/s wr, 171 op/s 2024-02-02T15:38:53.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:53 smithi142 ceph-mon[45728]: pgmap v369: 129 pgs: 129 active+clean; 540 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.9 MiB/s wr, 171 op/s 2024-02-02T15:38:53.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:53 smithi001 ceph-mon[35470]: pgmap v369: 129 pgs: 129 active+clean; 540 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.9 MiB/s wr, 171 op/s 2024-02-02T15:38:54.269 INFO:tasks.workunit.client.0.smithi001.stdout: 1 654007 5.49 MB/sec execute 508 sec latency 13.526 ms 2024-02-02T15:38:55.269 INFO:tasks.workunit.client.0.smithi001.stdout: 1 655033 5.48 MB/sec execute 509 sec latency 22.140 ms 2024-02-02T15:38:55.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:55 smithi071 ceph-mon[44707]: pgmap v370: 129 pgs: 129 active+clean; 540 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 113 op/s 2024-02-02T15:38:55.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:55 smithi142 ceph-mon[45728]: pgmap v370: 129 pgs: 129 active+clean; 540 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 113 op/s 2024-02-02T15:38:55.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:55 smithi001 ceph-mon[35470]: pgmap v370: 129 pgs: 129 active+clean; 540 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 113 op/s 2024-02-02T15:38:56.269 INFO:tasks.workunit.client.0.smithi001.stdout: 1 655132 5.47 MB/sec execute 510 sec latency 870.972 ms 2024-02-02T15:38:57.269 INFO:tasks.workunit.client.0.smithi001.stdout: 1 655132 5.46 MB/sec execute 511 sec latency 1871.128 ms 2024-02-02T15:38:57.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:57 smithi071 ceph-mon[44707]: pgmap v371: 129 pgs: 129 active+clean; 545 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.9 MiB/s wr, 171 op/s 2024-02-02T15:38:57.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:57 smithi142 ceph-mon[45728]: pgmap v371: 129 pgs: 129 active+clean; 545 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.9 MiB/s wr, 171 op/s 2024-02-02T15:38:57.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:57 smithi001 ceph-mon[35470]: pgmap v371: 129 pgs: 129 active+clean; 545 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.9 MiB/s wr, 171 op/s 2024-02-02T15:38:58.269 INFO:tasks.workunit.client.0.smithi001.stdout: 1 656317 5.46 MB/sec execute 512 sec latency 2254.694 ms 2024-02-02T15:38:59.270 INFO:tasks.workunit.client.0.smithi001.stdout: 1 657517 5.47 MB/sec execute 513 sec latency 16.967 ms 2024-02-02T15:38:59.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:38:59 smithi071 ceph-mon[44707]: pgmap v372: 129 pgs: 129 active+clean; 545 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 112 op/s 2024-02-02T15:38:59.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:38:59 smithi142 ceph-mon[45728]: pgmap v372: 129 pgs: 129 active+clean; 545 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 112 op/s 2024-02-02T15:38:59.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:38:59 smithi001 ceph-mon[35470]: pgmap v372: 129 pgs: 129 active+clean; 545 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 112 op/s 2024-02-02T15:39:00.270 INFO:tasks.workunit.client.0.smithi001.stdout: 1 658577 5.47 MB/sec execute 514 sec latency 20.173 ms 2024-02-02T15:39:01.270 INFO:tasks.workunit.client.0.smithi001.stdout: 1 658675 5.46 MB/sec execute 515 sec latency 917.029 ms 2024-02-02T15:39:01.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:39:01 smithi071 ceph-mon[44707]: pgmap v373: 129 pgs: 129 active+clean; 545 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 112 op/s 2024-02-02T15:39:01.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:39:01 smithi071 ceph-mon[44707]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:39:01.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:39:01 smithi142 ceph-mon[45728]: pgmap v373: 129 pgs: 129 active+clean; 545 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 112 op/s 2024-02-02T15:39:01.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:39:01 smithi142 ceph-mon[45728]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:39:01.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:39:01 smithi001 ceph-mon[35470]: pgmap v373: 129 pgs: 129 active+clean; 545 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.2 MiB/s wr, 112 op/s 2024-02-02T15:39:01.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:39:01 smithi001 ceph-mon[35470]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:39:02.270 INFO:tasks.workunit.client.0.smithi001.stdout: 1 658675 5.45 MB/sec execute 516 sec latency 1917.182 ms 2024-02-02T15:39:03.270 INFO:tasks.workunit.client.0.smithi001.stdout: 1 659269 5.44 MB/sec execute 517 sec latency 2295.231 ms 2024-02-02T15:39:03.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:39:03 smithi071 ceph-mon[44707]: pgmap v374: 129 pgs: 129 active+clean; 550 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.0 MiB/s wr, 174 op/s 2024-02-02T15:39:03.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:39:03 smithi142 ceph-mon[45728]: pgmap v374: 129 pgs: 129 active+clean; 550 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.0 MiB/s wr, 174 op/s 2024-02-02T15:39:03.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:39:03 smithi001 ceph-mon[35470]: pgmap v374: 129 pgs: 129 active+clean; 550 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.0 MiB/s wr, 174 op/s 2024-02-02T15:39:04.270 INFO:tasks.workunit.client.0.smithi001.stdout: 1 660470 5.45 MB/sec execute 518 sec latency 66.149 ms 2024-02-02T15:39:05.270 INFO:tasks.workunit.client.0.smithi001.stdout: 1 661319 5.44 MB/sec execute 519 sec latency 16.796 ms 2024-02-02T15:39:05.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:39:05 smithi071 ceph-mon[44707]: pgmap v375: 129 pgs: 129 active+clean; 550 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 120 op/s 2024-02-02T15:39:05.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:39:05 smithi142 ceph-mon[45728]: pgmap v375: 129 pgs: 129 active+clean; 550 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 120 op/s 2024-02-02T15:39:05.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:39:05 smithi001 ceph-mon[35470]: pgmap v375: 129 pgs: 129 active+clean; 550 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 120 op/s 2024-02-02T15:39:06.270 INFO:tasks.workunit.client.0.smithi001.stdout: 1 662222 5.44 MB/sec execute 520 sec latency 309.253 ms 2024-02-02T15:39:07.270 INFO:tasks.workunit.client.0.smithi001.stdout: 1 662222 5.43 MB/sec execute 521 sec latency 1309.369 ms 2024-02-02T15:39:07.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:39:07 smithi071 ceph-mon[44707]: pgmap v376: 129 pgs: 129 active+clean; 555 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 214 op/s 2024-02-02T15:39:07.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:39:07 smithi071 ceph-mon[44707]: osdmap e100: 12 total, 12 up, 12 in 2024-02-02T15:39:07.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:39:07 smithi142 ceph-mon[45728]: pgmap v376: 129 pgs: 129 active+clean; 555 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 214 op/s 2024-02-02T15:39:07.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:39:07 smithi142 ceph-mon[45728]: osdmap e100: 12 total, 12 up, 12 in 2024-02-02T15:39:07.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:39:07 smithi001 ceph-mon[35470]: pgmap v376: 129 pgs: 129 active+clean; 555 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.5 MiB/s wr, 214 op/s 2024-02-02T15:39:07.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:39:07 smithi001 ceph-mon[35470]: osdmap e100: 12 total, 12 up, 12 in 2024-02-02T15:39:08.270 INFO:tasks.workunit.client.0.smithi001.stdout: 1 663451 5.44 MB/sec execute 522 sec latency 1688.396 ms 2024-02-02T15:39:09.270 INFO:tasks.workunit.client.0.smithi001.stdout: 1 664611 5.44 MB/sec execute 523 sec latency 14.247 ms 2024-02-02T15:39:09.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:39:09 smithi071 ceph-mon[44707]: pgmap v378: 129 pgs: 129 active+clean; 555 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 186 op/s 2024-02-02T15:39:09.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:39:09 smithi142 ceph-mon[45728]: pgmap v378: 129 pgs: 129 active+clean; 555 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 186 op/s 2024-02-02T15:39:09.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:39:09 smithi001 ceph-mon[35470]: pgmap v378: 129 pgs: 129 active+clean; 555 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 186 op/s 2024-02-02T15:39:10.270 INFO:tasks.workunit.client.0.smithi001.stdout: 1 665713 5.44 MB/sec execute 524 sec latency 19.633 ms 2024-02-02T15:39:10.347 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-02T15:39:10.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.551+0000 7ff79d200700 1 -- 172.21.15.1:0/3160881225 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff7980685e0 msgr2=0x7ff798068a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:10.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.551+0000 7ff79d200700 1 --2- 172.21.15.1:0/3160881225 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff7980685e0 0x7ff798068a60 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7ff780009990 tx=0x7ff780009ca0 comp rx=0 tx=0).stop 2024-02-02T15:39:10.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.551+0000 7ff79d200700 1 -- 172.21.15.1:0/3160881225 shutdown_connections 2024-02-02T15:39:10.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.551+0000 7ff79d200700 1 --2- 172.21.15.1:0/3160881225 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff79810d610 0x7ff79810fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:10.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.551+0000 7ff79d200700 1 --2- 172.21.15.1:0/3160881225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff798101c70 0x7ff798068010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:10.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.551+0000 7ff79d200700 1 --2- 172.21.15.1:0/3160881225 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff7980685e0 0x7ff798068a60 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:10.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.551+0000 7ff79d200700 1 -- 172.21.15.1:0/3160881225 >> 172.21.15.1:0/3160881225 conn(0x7ff7980757d0 msgr2=0x7ff798075be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:10.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.551+0000 7ff79d200700 1 -- 172.21.15.1:0/3160881225 shutdown_connections 2024-02-02T15:39:10.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.551+0000 7ff79d200700 1 -- 172.21.15.1:0/3160881225 wait complete. 2024-02-02T15:39:10.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.552+0000 7ff79d200700 1 Processor -- start 2024-02-02T15:39:10.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.552+0000 7ff79d200700 1 -- start start 2024-02-02T15:39:10.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.553+0000 7ff79d200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff7980685e0 0x7ff7981990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:10.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.553+0000 7ff79d200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff798101c70 0x7ff798199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:10.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.553+0000 7ff79d200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff79810d610 0x7ff79819da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:10.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.553+0000 7ff79d200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff79819e0b0 con 0x7ff7980685e0 2024-02-02T15:39:10.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.553+0000 7ff79d200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff79819e220 con 0x7ff79810d610 2024-02-02T15:39:10.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.553+0000 7ff79d200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff79819e3e0 con 0x7ff798101c70 2024-02-02T15:39:10.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.553+0000 7ff795e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff7980685e0 0x7ff7981990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:10.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.553+0000 7ff795e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff7980685e0 0x7ff7981990d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50524/0 (socket says 172.21.15.1:50524) 2024-02-02T15:39:10.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.553+0000 7ff795e00700 1 -- 172.21.15.1:0/2367120733 learned_addr learned my addr 172.21.15.1:0/2367120733 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:39:10.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.553+0000 7ff795400700 1 --2- 172.21.15.1:0/2367120733 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff798101c70 0x7ff798199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:10.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.553+0000 7ff796800700 1 --2- 172.21.15.1:0/2367120733 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff79810d610 0x7ff79819da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:10.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.554+0000 7ff795e00700 1 -- 172.21.15.1:0/2367120733 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff798101c70 msgr2=0x7ff798199610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:10.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.554+0000 7ff795e00700 1 --2- 172.21.15.1:0/2367120733 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff798101c70 0x7ff798199610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:10.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.554+0000 7ff795e00700 1 -- 172.21.15.1:0/2367120733 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff79810d610 msgr2=0x7ff79819da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:10.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.554+0000 7ff795e00700 1 --2- 172.21.15.1:0/2367120733 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff79810d610 0x7ff79819da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:10.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.554+0000 7ff795e00700 1 -- 172.21.15.1:0/2367120733 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7800095f0 con 0x7ff7980685e0 2024-02-02T15:39:10.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.554+0000 7ff795400700 1 --2- 172.21.15.1:0/2367120733 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff798101c70 0x7ff798199610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:39:10.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.554+0000 7ff796800700 1 --2- 172.21.15.1:0/2367120733 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff79810d610 0x7ff79819da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:39:10.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.554+0000 7ff795e00700 1 --2- 172.21.15.1:0/2367120733 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff7980685e0 0x7ff7981990d0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7ff78800e9a0 tx=0x7ff78800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:10.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.554+0000 7ff78ea00700 1 -- 172.21.15.1:0/2367120733 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff788009790 con 0x7ff7980685e0 2024-02-02T15:39:10.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.554+0000 7ff78ea00700 1 -- 172.21.15.1:0/2367120733 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff78800cbd0 con 0x7ff7980685e0 2024-02-02T15:39:10.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.554+0000 7ff78ea00700 1 -- 172.21.15.1:0/2367120733 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff788018680 con 0x7ff7980685e0 2024-02-02T15:39:10.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.554+0000 7ff79d200700 1 -- 172.21.15.1:0/2367120733 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff79819e6c0 con 0x7ff7980685e0 2024-02-02T15:39:10.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.554+0000 7ff79d200700 1 -- 172.21.15.1:0/2367120733 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7981a5f00 con 0x7ff7980685e0 2024-02-02T15:39:10.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.558+0000 7ff78ea00700 1 -- 172.21.15.1:0/2367120733 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7ff78800cd40 con 0x7ff7980685e0 2024-02-02T15:39:10.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.559+0000 7ff7b0ca2b80 1 Processor -- start 2024-02-02T15:39:10.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.559+0000 7ff7b0ca2b80 1 -- start start 2024-02-02T15:39:10.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.559+0000 7ff78ea00700 1 --2- 172.21.15.1:0/2367120733 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff7780a0710 0x7ff7780a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:10.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.559+0000 7ff7b0ca2b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559da1b62010 0x559da1b623f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:10.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.559+0000 7ff78ea00700 1 -- 172.21.15.1:0/2367120733 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9541+0+0 (secure 0 0 0) 0x7ff788014070 con 0x7ff7980685e0 2024-02-02T15:39:10.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.559+0000 7ff7b0ca2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559da1b62930 0x559da1b6d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:10.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.559+0000 7ff7b0ca2b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559da1b6d7e0 0x559da1b6fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:10.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.559+0000 7ff7b0ca2b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x559da1ad64c0 con 0x559da1b62010 2024-02-02T15:39:10.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.559+0000 7ff7b0ca2b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x559da1abaaa0 con 0x559da1b6d7e0 2024-02-02T15:39:10.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.559+0000 7ff7b0ca2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559da1ab8460 con 0x559da1b62930 2024-02-02T15:39:10.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.559+0000 7ff796800700 1 --2- 172.21.15.1:0/2367120733 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff7780a0710 0x7ff7780a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:10.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.560+0000 7ff795e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559da1b62010 0x559da1b623f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:10.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.560+0000 7ff796800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559da1b6d7e0 0x559da1b6fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:10.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.560+0000 7ff796800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559da1b6d7e0 0x559da1b6fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:60920/0 (socket says 172.21.15.1:60920) 2024-02-02T15:39:10.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.560+0000 7ff795e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559da1b62010 0x559da1b623f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50530/0 (socket says 172.21.15.1:50530) 2024-02-02T15:39:10.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.560+0000 7ff796800700 1 -- 172.21.15.1:0/4180765765 learned_addr learned my addr 172.21.15.1:0/4180765765 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:39:10.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.560+0000 7ff795400700 1 --2- 172.21.15.1:0/4180765765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559da1b62930 0x559da1b6d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:10.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.560+0000 7ff796800700 1 -- 172.21.15.1:0/4180765765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559da1b62930 msgr2=0x559da1b6d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:10.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.560+0000 7ff796800700 1 --2- 172.21.15.1:0/4180765765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559da1b62930 0x559da1b6d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:10.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.560+0000 7ff796800700 1 -- 172.21.15.1:0/4180765765 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559da1b62010 msgr2=0x559da1b623f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:10.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.560+0000 7ff796800700 1 --2- 172.21.15.1:0/4180765765 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559da1b62010 0x559da1b623f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:10.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.560+0000 7ff796800700 1 -- 172.21.15.1:0/4180765765 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559da19353b0 con 0x559da1b6d7e0 2024-02-02T15:39:10.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.560+0000 7ff795400700 1 --2- 172.21.15.1:0/4180765765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559da1b62930 0x559da1b6d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:39:10.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.560+0000 7ff795e00700 1 --2- 172.21.15.1:0/4180765765 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559da1b62010 0x559da1b623f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:39:10.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.560+0000 7ff796800700 1 --2- 172.21.15.1:0/2367120733 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff7780a0710 0x7ff7780a2bd0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff790007cb0 tx=0x7ff7900092d0 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:10.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.560+0000 7ff796800700 1 --2- 172.21.15.1:0/4180765765 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559da1b6d7e0 0x559da1b6fbd0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7ff790007560 tx=0x7ff79001c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2496fba7d8b08 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:10.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.561+0000 7ff787400700 1 -- 172.21.15.1:0/4180765765 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff79000e050 con 0x559da1b6d7e0 2024-02-02T15:39:10.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.561+0000 7ff787400700 1 -- 172.21.15.1:0/4180765765 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff79000f040 con 0x559da1b6d7e0 2024-02-02T15:39:10.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.561+0000 7ff787400700 1 -- 172.21.15.1:0/4180765765 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7900258b0 con 0x559da1b6d7e0 2024-02-02T15:39:10.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.561+0000 7ff7b0ca2b80 1 -- 172.21.15.1:0/4180765765 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559da1b6d7e0 msgr2=0x559da1b6fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:10.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.561+0000 7ff7b0ca2b80 1 --2- 172.21.15.1:0/4180765765 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559da1b6d7e0 0x559da1b6fbd0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7ff790007560 tx=0x7ff79001c000 comp rx=0 tx=0).stop 2024-02-02T15:39:10.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.561+0000 7ff7b0ca2b80 1 -- 172.21.15.1:0/4180765765 shutdown_connections 2024-02-02T15:39:10.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.561+0000 7ff7b0ca2b80 1 --2- 172.21.15.1:0/4180765765 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559da1b6d7e0 0x559da1b6fbd0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:10.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.561+0000 7ff7b0ca2b80 1 --2- 172.21.15.1:0/4180765765 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559da1b62010 0x559da1b623f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:10.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.561+0000 7ff7b0ca2b80 1 --2- 172.21.15.1:0/4180765765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559da1b62930 0x559da1b6d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:10.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.561+0000 7ff7b0ca2b80 1 -- 172.21.15.1:0/4180765765 >> 172.21.15.1:0/4180765765 conn(0x559da1ac1db0 msgr2=0x559da1b5e3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:10.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.561+0000 7ff7b0ca2b80 1 -- 172.21.15.1:0/4180765765 shutdown_connections 2024-02-02T15:39:10.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.562+0000 7ff7b0ca2b80 1 -- 172.21.15.1:0/4180765765 wait complete. 2024-02-02T15:39:10.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.562+0000 7ff7b0ca2b80 1 -- start start 2024-02-02T15:39:10.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.562+0000 7ff7b0ca2b80 1 Processor -- start 2024-02-02T15:39:10.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.563+0000 7ff7b0ca2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559da1b62010 0x559da1b6d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:10.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.563+0000 7ff7b0ca2b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559da1b62930 0x559da1b6a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:10.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.563+0000 7ff7b0ca2b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559da1b6d7e0 0x559da1b6aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:10.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.563+0000 7ff7b0ca2b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x559da1ab3d50 con 0x559da1b6d7e0 2024-02-02T15:39:10.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.563+0000 7ff7b0ca2b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x559da1a4fd80 con 0x559da1b62930 2024-02-02T15:39:10.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.563+0000 7ff7b0ca2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559da1a4fa20 con 0x559da1b62010 2024-02-02T15:39:10.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.563+0000 7ff7b0ca2b80 10 client.0 _ll_register_callbacks cb 0x559da1a51290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:39:10.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.564+0000 7ff795400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559da1b62930 0x559da1b6a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:10.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.564+0000 7ff796800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559da1b6d7e0 0x559da1b6aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:10.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.564+0000 7ff795400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559da1b62930 0x559da1b6a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:60926/0 (socket says 172.21.15.1:60926) 2024-02-02T15:39:10.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.564+0000 7ff795400700 1 -- 172.21.15.1:0/4069300508 learned_addr learned my addr 172.21.15.1:0/4069300508 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:39:10.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.564+0000 7ff795e00700 1 --2- 172.21.15.1:0/4069300508 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559da1b62010 0x559da1b6d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:10.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.564+0000 7ff795400700 1 -- 172.21.15.1:0/4069300508 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559da1b62010 msgr2=0x559da1b6d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:10.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.564+0000 7ff795400700 1 --2- 172.21.15.1:0/4069300508 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559da1b62010 0x559da1b6d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:10.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.564+0000 7ff795400700 1 -- 172.21.15.1:0/4069300508 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559da1b6d7e0 msgr2=0x559da1b6aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:10.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.564+0000 7ff795400700 1 --2- 172.21.15.1:0/4069300508 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559da1b6d7e0 0x559da1b6aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:10.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.564+0000 7ff795400700 1 -- 172.21.15.1:0/4069300508 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7981106e0 con 0x559da1b62930 2024-02-02T15:39:10.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.564+0000 7ff795400700 1 --2- 172.21.15.1:0/4069300508 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559da1b62930 0x559da1b6a460 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7ff798102050 tx=0x7ff78000ba80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=208227937fe08fc8 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:10.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.564+0000 7ff77fe00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:39:10.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.564+0000 7ff77fe00700 1 -- 172.21.15.1:0/4069300508 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78000f870 con 0x559da1b62930 2024-02-02T15:39:10.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.565+0000 7ff7b0ca2b80 1 -- 172.21.15.1:0/4069300508 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559da1b560f0 con 0x559da1b62930 2024-02-02T15:39:10.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.565+0000 7ff77fe00700 1 -- 172.21.15.1:0/4069300508 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9541+0+0 (secure 0 0 0) 0x7ff78000f9d0 con 0x559da1b62930 2024-02-02T15:39:10.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.565+0000 7ff77fe00700 1 -- 172.21.15.1:0/4069300508 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff78000fb90 con 0x559da1b62930 2024-02-02T15:39:10.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.565+0000 7ff7b0ca2b80 10 client.28799 fetch_fsmap learned FSMap version 52 2024-02-02T15:39:10.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.565+0000 7ff7b0ca2b80 1 -- 172.21.15.1:0/4069300508 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x7ff790007df0 con 0x559da1b62930 2024-02-02T15:39:10.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.565+0000 7ff77fe00700 1 -- 172.21.15.1:0/4069300508 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559da1b560f0 con 0x559da1b62930 2024-02-02T15:39:10.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.565+0000 7ff77fe00700 1 -- 172.21.15.1:0/4069300508 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 52) v1 ==== 7486+0+0 (secure 0 0 0) 0x7ff7800173f0 con 0x559da1b62930 2024-02-02T15:39:10.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.566+0000 7ff7b0ca2b80 10 client.28799 fetch_fsmap finished waiting for FSMap version 52 2024-02-02T15:39:10.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.566+0000 7ff7b0ca2b80 10 client.28799 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:39:10.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.566+0000 7ff7b0ca2b80 20 client.28799 populate_metadata read hostname 'smithi001' 2024-02-02T15:39:10.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.566+0000 7ff7b0ca2b80 1 --2- 172.21.15.1:0/4069300508 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x559da1b7d1d0 0x559da1b7f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:10.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.566+0000 7ff7b0ca2b80 4 client.28799 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:39:10.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.566+0000 7ff7b0ca2b80 1 -- 172.21.15.1:0/4069300508 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559da1aacdd0 con 0x559da1b7d1d0 2024-02-02T15:39:10.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.566+0000 7ff795e00700 1 --2- 172.21.15.1:0/4069300508 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x559da1b7d1d0 0x559da1b7f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:10.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.566+0000 7ff795e00700 1 --2- 172.21.15.1:0/4069300508 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x559da1b7d1d0 0x559da1b7f5f0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d27446aed5cb63f3 server_cookie=83c1d7be4770e99d in_seq=0 out_seq=0 2024-02-02T15:39:10.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.566+0000 7ff77fe00700 10 client.28799 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:39:10.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.567+0000 7ff77fe00700 1 -- 172.21.15.1:0/4069300508 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x559da1aacdd0 con 0x559da1b7d1d0 2024-02-02T15:39:10.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.567+0000 7ff77fe00700 10 client.28799 handle_command_reply: tid=0 2024-02-02T15:39:10.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.567+0000 7ff7b0ca2b80 1 client.28799 shutdown 2024-02-02T15:39:10.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.567+0000 7ff7b0ca2b80 1 -- 172.21.15.1:0/4069300508 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559da1b62930 msgr2=0x559da1b6a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:10.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.567+0000 7ff7b0ca2b80 1 --2- 172.21.15.1:0/4069300508 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559da1b62930 0x559da1b6a460 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7ff798102050 tx=0x7ff78000ba80 comp rx=0 tx=0).stop 2024-02-02T15:39:10.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.567+0000 7ff7b0ca2b80 1 -- 172.21.15.1:0/4069300508 shutdown_connections 2024-02-02T15:39:10.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.567+0000 7ff7b0ca2b80 1 --2- 172.21.15.1:0/4069300508 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x559da1b7d1d0 0x559da1b7f5f0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:10.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.567+0000 7ff7b0ca2b80 1 --2- 172.21.15.1:0/4069300508 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559da1b6d7e0 0x559da1b6aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:10.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.567+0000 7ff7b0ca2b80 1 --2- 172.21.15.1:0/4069300508 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559da1b62010 0x559da1b6d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:10.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.567+0000 7ff77fe00700 0 client.28799 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:39:10.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.567+0000 7ff7b0ca2b80 1 --2- 172.21.15.1:0/4069300508 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559da1b62930 0x559da1b6a460 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:10.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.567+0000 7ff7b0ca2b80 1 -- 172.21.15.1:0/4069300508 >> 172.21.15.1:0/4069300508 conn(0x559da1ac1db0 msgr2=0x559da1b5dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:10.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.567+0000 7ff7b0ca2b80 1 -- 172.21.15.1:0/4069300508 shutdown_connections 2024-02-02T15:39:10.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.567+0000 7ff7b0ca2b80 1 -- 172.21.15.1:0/4069300508 wait complete. 2024-02-02T15:39:10.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.568+0000 7ff7b0ca2b80 20 client.28799 trim_cache size 0 max 16384 2024-02-02T15:39:10.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.585+0000 7ff7b0ca2b80 1 Processor -- start 2024-02-02T15:39:10.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.585+0000 7ff7b0ca2b80 1 -- start start 2024-02-02T15:39:10.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.585+0000 7ff7b0ca2b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559da1b62010 0x559da1b65740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:10.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.585+0000 7ff7b0ca2b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559da1b62930 0x559da1b65c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:10.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.585+0000 7ff7b0ca2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559da1b6d7e0 0x559da1b69c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:10.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.585+0000 7ff7b0ca2b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x559da19376a0 con 0x559da1b62010 2024-02-02T15:39:10.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.585+0000 7ff7b0ca2b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x559da1b5f7e0 con 0x559da1b62930 2024-02-02T15:39:10.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.585+0000 7ff7b0ca2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559da1b58500 con 0x559da1b6d7e0 2024-02-02T15:39:10.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.585+0000 7ff795e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559da1b62010 0x559da1b65740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:10.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.585+0000 7ff795e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559da1b62010 0x559da1b65740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50548/0 (socket says 172.21.15.1:50548) 2024-02-02T15:39:10.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.585+0000 7ff795e00700 1 -- 172.21.15.1:0/357588527 learned_addr learned my addr 172.21.15.1:0/357588527 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:39:10.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.585+0000 7ff795400700 1 --2- 172.21.15.1:0/357588527 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559da1b62930 0x559da1b65c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:10.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.585+0000 7ff796800700 1 --2- 172.21.15.1:0/357588527 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559da1b6d7e0 0x559da1b69c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:10.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.585+0000 7ff795e00700 1 -- 172.21.15.1:0/357588527 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559da1b6d7e0 msgr2=0x559da1b69c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:10.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.585+0000 7ff795e00700 1 --2- 172.21.15.1:0/357588527 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559da1b6d7e0 0x559da1b69c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:10.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.585+0000 7ff795e00700 1 -- 172.21.15.1:0/357588527 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559da1b62930 msgr2=0x559da1b65c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:10.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.585+0000 7ff795e00700 1 --2- 172.21.15.1:0/357588527 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559da1b62930 0x559da1b65c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:10.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.586+0000 7ff795e00700 1 -- 172.21.15.1:0/357588527 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559da18e1b60 con 0x559da1b62010 2024-02-02T15:39:10.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.586+0000 7ff796800700 1 --2- 172.21.15.1:0/357588527 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559da1b6d7e0 0x559da1b69c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:39:10.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.586+0000 7ff795e00700 1 --2- 172.21.15.1:0/357588527 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559da1b62010 0x559da1b65740 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7ff788095fb0 tx=0x7ff788010b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=355939409417482a server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:10.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.586+0000 7ff787e00700 1 -- 172.21.15.1:0/357588527 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7880a15b0 con 0x559da1b62010 2024-02-02T15:39:10.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.586+0000 7ff787e00700 1 -- 172.21.15.1:0/357588527 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff78809d120 con 0x559da1b62010 2024-02-02T15:39:10.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.586+0000 7ff787e00700 1 -- 172.21.15.1:0/357588527 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7880997a0 con 0x559da1b62010 2024-02-02T15:39:10.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.586+0000 7ff7b0ca2b80 1 -- 172.21.15.1:0/357588527 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559da1b62010 msgr2=0x559da1b65740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:10.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.586+0000 7ff7b0ca2b80 1 --2- 172.21.15.1:0/357588527 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559da1b62010 0x559da1b65740 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7ff788095fb0 tx=0x7ff788010b50 comp rx=0 tx=0).stop 2024-02-02T15:39:10.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.587+0000 7ff7b0ca2b80 1 -- 172.21.15.1:0/357588527 shutdown_connections 2024-02-02T15:39:10.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.587+0000 7ff7b0ca2b80 1 --2- 172.21.15.1:0/357588527 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559da1b6d7e0 0x559da1b69c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:10.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.587+0000 7ff7b0ca2b80 1 --2- 172.21.15.1:0/357588527 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559da1b62010 0x559da1b65740 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:10.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.587+0000 7ff7b0ca2b80 1 --2- 172.21.15.1:0/357588527 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559da1b62930 0x559da1b65c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:10.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.587+0000 7ff7b0ca2b80 1 -- 172.21.15.1:0/357588527 >> 172.21.15.1:0/357588527 conn(0x559da1ac1db0 msgr2=0x559da1b79420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:10.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.587+0000 7ff7b0ca2b80 1 -- 172.21.15.1:0/357588527 shutdown_connections 2024-02-02T15:39:10.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.587+0000 7ff7b0ca2b80 1 -- 172.21.15.1:0/357588527 wait complete. 2024-02-02T15:39:10.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.587+0000 7ff7b0ca2b80 1 -- start start 2024-02-02T15:39:10.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.587+0000 7ff7b0ca2b80 1 Processor -- start 2024-02-02T15:39:10.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.588+0000 7ff7b0ca2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559da1b62010 0x559da1b73ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:10.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.588+0000 7ff7b0ca2b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559da1b62930 0x559da1b72200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:10.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.588+0000 7ff7b0ca2b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559da1b6d7e0 0x559da1b72740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:10.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.588+0000 7ff7b0ca2b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x559da1b72c80 con 0x559da1b6d7e0 2024-02-02T15:39:10.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.588+0000 7ff7b0ca2b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x559da1b72df0 con 0x559da1b62930 2024-02-02T15:39:10.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.588+0000 7ff7b0ca2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559da1b72fb0 con 0x559da1b62010 2024-02-02T15:39:10.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.588+0000 7ff7b0ca2b80 10 client.0 _ll_register_callbacks cb 0x559da1a51290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:39:10.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.588+0000 7ff796800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559da1b6d7e0 0x559da1b72740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:10.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.589+0000 7ff795400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559da1b62930 0x559da1b72200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:10.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.589+0000 7ff795400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559da1b62930 0x559da1b72200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:60954/0 (socket says 172.21.15.1:60954) 2024-02-02T15:39:10.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.589+0000 7ff795400700 1 -- 172.21.15.1:0/3486333212 learned_addr learned my addr 172.21.15.1:0/3486333212 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:39:10.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.589+0000 7ff795e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559da1b62010 0x559da1b73ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:10.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.589+0000 7ff795400700 1 -- 172.21.15.1:0/3486333212 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559da1b62010 msgr2=0x559da1b73ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:10.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.589+0000 7ff795400700 1 --2- 172.21.15.1:0/3486333212 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559da1b62010 0x559da1b73ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:10.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.589+0000 7ff795400700 1 -- 172.21.15.1:0/3486333212 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559da1b6d7e0 msgr2=0x559da1b72740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:10.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.589+0000 7ff795400700 1 --2- 172.21.15.1:0/3486333212 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559da1b6d7e0 0x559da1b72740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:10.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.589+0000 7ff795400700 1 -- 172.21.15.1:0/3486333212 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff79819e6c0 con 0x559da1b62930 2024-02-02T15:39:10.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.589+0000 7ff795400700 1 --2- 172.21.15.1:0/3486333212 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559da1b62930 0x559da1b72200 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7ff7800098e0 tx=0x7ff780009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b56f9bce0f003ad8 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:10.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.589+0000 7ff77fe00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:39:10.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.589+0000 7ff77fe00700 1 -- 172.21.15.1:0/3486333212 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78000f870 con 0x559da1b62930 2024-02-02T15:39:10.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.589+0000 7ff77fe00700 1 -- 172.21.15.1:0/3486333212 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9541+0+0 (secure 0 0 0) 0x7ff780000e40 con 0x559da1b62930 2024-02-02T15:39:10.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.589+0000 7ff7b0ca2b80 1 -- 172.21.15.1:0/3486333212 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff79000f040 con 0x559da1b62930 2024-02-02T15:39:10.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.590+0000 7ff77fe00700 1 -- 172.21.15.1:0/3486333212 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff78000bdd0 con 0x559da1b62930 2024-02-02T15:39:10.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.590+0000 7ff7b0ca2b80 10 client.28805 fetch_fsmap learned FSMap version 52 2024-02-02T15:39:10.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.590+0000 7ff7b0ca2b80 1 -- 172.21.15.1:0/3486333212 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x559da1a79ea0 con 0x559da1b62930 2024-02-02T15:39:10.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.590+0000 7ff77fe00700 1 -- 172.21.15.1:0/3486333212 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff79000f040 con 0x559da1b62930 2024-02-02T15:39:10.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.590+0000 7ff77fe00700 1 -- 172.21.15.1:0/3486333212 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 52) v1 ==== 7486+0+0 (secure 0 0 0) 0x7ff780023900 con 0x559da1b62930 2024-02-02T15:39:10.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.590+0000 7ff7b0ca2b80 10 client.28805 fetch_fsmap finished waiting for FSMap version 52 2024-02-02T15:39:10.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.590+0000 7ff7b0ca2b80 10 client.28805 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:39:10.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.590+0000 7ff7b0ca2b80 20 client.28805 populate_metadata read hostname 'smithi001' 2024-02-02T15:39:10.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.590+0000 7ff7b0ca2b80 1 --2- 172.21.15.1:0/3486333212 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x559da1bcfc50 0x559da1bd2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:10.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.590+0000 7ff7b0ca2b80 4 client.28805 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2024-02-02T15:39:10.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.590+0000 7ff7b0ca2b80 1 -- 172.21.15.1:0/3486333212 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559da1a72f50 con 0x559da1bcfc50 2024-02-02T15:39:10.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.590+0000 7ff795e00700 1 --2- 172.21.15.1:0/3486333212 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x559da1bcfc50 0x559da1bd2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:10.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.591+0000 7ff795e00700 1 --2- 172.21.15.1:0/3486333212 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x559da1bcfc50 0x559da1bd2040 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75ade20cfef1bfb6 server_cookie=71e68a7865816d69 in_seq=0 out_seq=0 2024-02-02T15:39:10.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.591+0000 7ff77fe00700 10 client.28805 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:39:10.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.591+0000 7ff77fe00700 1 -- 172.21.15.1:0/3486333212 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x559da1a72f50 con 0x559da1bcfc50 2024-02-02T15:39:10.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.591+0000 7ff77fe00700 10 client.28805 handle_command_reply: tid=0 2024-02-02T15:39:10.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.591+0000 7ff7b0ca2b80 1 client.28805 shutdown 2024-02-02T15:39:10.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.591+0000 7ff7b0ca2b80 1 -- 172.21.15.1:0/3486333212 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559da1b62930 msgr2=0x559da1b72200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:10.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.591+0000 7ff7b0ca2b80 1 --2- 172.21.15.1:0/3486333212 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559da1b62930 0x559da1b72200 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7ff7800098e0 tx=0x7ff780009310 comp rx=0 tx=0).stop 2024-02-02T15:39:10.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.591+0000 7ff7b0ca2b80 1 -- 172.21.15.1:0/3486333212 shutdown_connections 2024-02-02T15:39:10.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.591+0000 7ff7b0ca2b80 1 --2- 172.21.15.1:0/3486333212 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x559da1bcfc50 0x559da1bd2040 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:10.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.591+0000 7ff7b0ca2b80 1 --2- 172.21.15.1:0/3486333212 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559da1b6d7e0 0x559da1b72740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:10.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.591+0000 7ff7b0ca2b80 1 --2- 172.21.15.1:0/3486333212 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559da1b62010 0x559da1b73ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:10.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.591+0000 7ff7b0ca2b80 1 --2- 172.21.15.1:0/3486333212 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559da1b62930 0x559da1b72200 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:10.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.591+0000 7ff77fe00700 0 client.28805 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:39:10.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.591+0000 7ff7b0ca2b80 1 -- 172.21.15.1:0/3486333212 >> 172.21.15.1:0/3486333212 conn(0x559da1ac1db0 msgr2=0x559da1b65740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:10.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.591+0000 7ff7b0ca2b80 1 -- 172.21.15.1:0/3486333212 shutdown_connections 2024-02-02T15:39:10.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.591+0000 7ff7b0ca2b80 1 -- 172.21.15.1:0/3486333212 wait complete. 2024-02-02T15:39:10.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.592+0000 7ff7b0ca2b80 20 client.28805 trim_cache size 0 max 16384 2024-02-02T15:39:10.596 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:39:10.596 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2024-02-02T15:39:10.597 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2024-02-02T15:39:10.597 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:39:10.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.592+0000 7ff79d200700 1 -- 172.21.15.1:0/2367120733 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff7780a0710 msgr2=0x7ff7780a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:10.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.592+0000 7ff79d200700 1 --2- 172.21.15.1:0/2367120733 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff7780a0710 0x7ff7780a2bd0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ff790007cb0 tx=0x7ff7900092d0 comp rx=0 tx=0).stop 2024-02-02T15:39:10.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.592+0000 7ff79d200700 1 -- 172.21.15.1:0/2367120733 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff7980685e0 msgr2=0x7ff7981990d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:10.597 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.592+0000 7ff79d200700 1 --2- 172.21.15.1:0/2367120733 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff7980685e0 0x7ff7981990d0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7ff78800e9a0 tx=0x7ff78800ed60 comp rx=0 tx=0).stop 2024-02-02T15:39:10.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.592+0000 7ff79d200700 1 -- 172.21.15.1:0/2367120733 shutdown_connections 2024-02-02T15:39:10.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.592+0000 7ff79d200700 1 --2- 172.21.15.1:0/2367120733 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff7780a0710 0x7ff7780a2bd0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:10.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.592+0000 7ff79d200700 1 --2- 172.21.15.1:0/2367120733 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff79810d610 0x7ff79819da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:10.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.592+0000 7ff79d200700 1 --2- 172.21.15.1:0/2367120733 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff7980685e0 0x7ff7981990d0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:10.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.592+0000 7ff79d200700 1 --2- 172.21.15.1:0/2367120733 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff798101c70 0x7ff798199610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:10.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.592+0000 7ff79d200700 1 -- 172.21.15.1:0/2367120733 >> 172.21.15.1:0/2367120733 conn(0x7ff7980757d0 msgr2=0x7ff7980fee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:10.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.592+0000 7ff79d200700 1 -- 172.21.15.1:0/2367120733 shutdown_connections 2024-02-02T15:39:10.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.592+0000 7ff79d200700 1 -- 172.21.15.1:0/2367120733 wait complete. 2024-02-02T15:39:10.610 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-02T15:39:10.610 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T15:39:10.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.814+0000 7f902fa00700 1 -- 172.21.15.1:0/217286590 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9028073a90 msgr2=0x7f902810cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:10.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.814+0000 7f902fa00700 1 --2- 172.21.15.1:0/217286590 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9028073a90 0x7f902810cfb0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f9010009990 tx=0x7f9010009ca0 comp rx=0 tx=0).stop 2024-02-02T15:39:10.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.815+0000 7f902fa00700 1 -- 172.21.15.1:0/217286590 shutdown_connections 2024-02-02T15:39:10.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.815+0000 7f902fa00700 1 --2- 172.21.15.1:0/217286590 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f902810d4f0 0x7f902810f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:10.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.815+0000 7f902fa00700 1 --2- 172.21.15.1:0/217286590 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9028073170 0x7f9028073550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:10.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.815+0000 7f902fa00700 1 --2- 172.21.15.1:0/217286590 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9028073a90 0x7f902810cfb0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:10.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.815+0000 7f902fa00700 1 -- 172.21.15.1:0/217286590 >> 172.21.15.1:0/217286590 conn(0x7f90280fc7a0 msgr2=0x7f90280febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:10.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.815+0000 7f902fa00700 1 -- 172.21.15.1:0/217286590 shutdown_connections 2024-02-02T15:39:10.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.815+0000 7f902fa00700 1 -- 172.21.15.1:0/217286590 wait complete. 2024-02-02T15:39:10.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.816+0000 7f902fa00700 1 Processor -- start 2024-02-02T15:39:10.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.816+0000 7f902fa00700 1 -- start start 2024-02-02T15:39:10.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.816+0000 7f902fa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9028073170 0x7f9028198fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:10.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.816+0000 7f902fa00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9028073a90 0x7f9028199510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:10.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.816+0000 7f902fa00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f902810d4f0 0x7f902819d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:10.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.816+0000 7f902fa00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f902819dfb0 con 0x7f9028073170 2024-02-02T15:39:10.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.816+0000 7f902fa00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f902819e120 con 0x7f9028073a90 2024-02-02T15:39:10.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.816+0000 7f902fa00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f902819e2e0 con 0x7f902810d4f0 2024-02-02T15:39:10.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.816+0000 7f902d000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9028073170 0x7f9028198fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:10.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.816+0000 7f9027e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9028073a90 0x7f9028199510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:10.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.816+0000 7f902d000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9028073170 0x7f9028198fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50584/0 (socket says 172.21.15.1:50584) 2024-02-02T15:39:10.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.816+0000 7f902d000700 1 -- 172.21.15.1:0/2721152927 learned_addr learned my addr 172.21.15.1:0/2721152927 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:39:10.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.817+0000 7f902da00700 1 --2- 172.21.15.1:0/2721152927 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f902810d4f0 0x7f902819d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:10.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.817+0000 7f9027e00700 1 -- 172.21.15.1:0/2721152927 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f902810d4f0 msgr2=0x7f902819d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:10.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.817+0000 7f9027e00700 1 --2- 172.21.15.1:0/2721152927 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f902810d4f0 0x7f902819d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:10.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.817+0000 7f9027e00700 1 -- 172.21.15.1:0/2721152927 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9028073170 msgr2=0x7f9028198fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:10.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.817+0000 7f9027e00700 1 --2- 172.21.15.1:0/2721152927 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9028073170 0x7f9028198fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:10.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.817+0000 7f9027e00700 1 -- 172.21.15.1:0/2721152927 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90100095f0 con 0x7f9028073a90 2024-02-02T15:39:10.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.817+0000 7f9027e00700 1 --2- 172.21.15.1:0/2721152927 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9028073a90 0x7f9028199510 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f9010000c00 tx=0x7f9010005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:10.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.817+0000 7f9025600700 1 -- 172.21.15.1:0/2721152927 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f901001d070 con 0x7f9028073a90 2024-02-02T15:39:10.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.817+0000 7f902fa00700 1 -- 172.21.15.1:0/2721152927 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f902819e560 con 0x7f9028073a90 2024-02-02T15:39:10.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.817+0000 7f9025600700 1 -- 172.21.15.1:0/2721152927 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9010021410 con 0x7f9028073a90 2024-02-02T15:39:10.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.817+0000 7f9025600700 1 -- 172.21.15.1:0/2721152927 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9010017650 con 0x7f9028073a90 2024-02-02T15:39:10.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.818+0000 7f902fa00700 1 -- 172.21.15.1:0/2721152927 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90281a5e00 con 0x7f9028073a90 2024-02-02T15:39:10.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.819+0000 7f902fa00700 1 -- 172.21.15.1:0/2721152927 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9028067bc0 con 0x7f9028073a90 2024-02-02T15:39:10.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.821+0000 7f9025600700 1 -- 172.21.15.1:0/2721152927 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f9010052020 con 0x7f9028073a90 2024-02-02T15:39:10.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.822+0000 7f9025600700 1 --2- 172.21.15.1:0/2721152927 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f90080a0690 0x7f90080a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:10.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.823+0000 7f9025600700 1 -- 172.21.15.1:0/2721152927 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f90100cd980 con 0x7f9028073a90 2024-02-02T15:39:10.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.823+0000 7f902d000700 1 --2- 172.21.15.1:0/2721152927 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f90080a0690 0x7f90080a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:10.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.823+0000 7f902d000700 1 --2- 172.21.15.1:0/2721152927 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f90080a0690 0x7f90080a2b50 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9018009d20 tx=0x7f9018009260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:10.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:10.825+0000 7f9025600700 1 -- 172.21.15.1:0/2721152927 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f9010096e10 con 0x7f9028073a90 2024-02-02T15:39:11.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.122+0000 7f902fa00700 1 -- 172.21.15.1:0/2721152927 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f902819a320 con 0x7f9028073a90 2024-02-02T15:39:11.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.123+0000 7f9025600700 1 -- 172.21.15.1:0/2721152927 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+10873 (secure 0 0 0) 0x7f901009a430 con 0x7f9028073a90 2024-02-02T15:39:11.123 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:39:11.124 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":52,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2922240448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2922240448},{"type":"v1","addr":"172.21.15.1:6835","nonce":2922240448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2729497431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2729497431},{"type":"v1","addr":"172.21.15.1:6837","nonce":2729497431}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":52,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-02T15:24:47.761435+0000","modified":"2024-02-02T15:39:00.496919+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24529,"mds_2":24518,"mds_3":24514,"mds_4":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14517":{"gid":14517,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/1218303683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":1218303683},{"type":"v1","addr":"172.21.15.1:6839","nonce":1218303683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14529":{"gid":14529,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/2059197496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2059197496},{"type":"v1","addr":"172.21.15.1:6841","nonce":2059197496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/7648986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":7648986},{"type":"v1","addr":"172.21.15.142:6833","nonce":7648986}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/3482746094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3482746094},{"type":"v1","addr":"172.21.15.71:6833","nonce":3482746094}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/388207814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":388207814},{"type":"v1","addr":"172.21.15.71:6835","nonce":388207814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6835/1052606547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1052606547},{"type":"v1","addr":"172.21.15.142:6835","nonce":1052606547}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.142:6837/2870289019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2870289019},{"type":"v1","addr":"172.21.15.142:6837","nonce":2870289019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.71:6837/628016199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":628016199},{"type":"v1","addr":"172.21.15.71:6837","nonce":628016199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.142:6839/321675421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":321675421},{"type":"v1","addr":"172.21.15.142:6839","nonce":321675421}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/1268086493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1268086493},{"type":"v1","addr":"172.21.15.71:6839","nonce":1268086493}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-02T15:39:11.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.126+0000 7f902fa00700 1 -- 172.21.15.1:0/2721152927 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f90080a0690 msgr2=0x7f90080a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:11.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.126+0000 7f902fa00700 1 --2- 172.21.15.1:0/2721152927 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f90080a0690 0x7f90080a2b50 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9018009d20 tx=0x7f9018009260 comp rx=0 tx=0).stop 2024-02-02T15:39:11.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.126+0000 7f902fa00700 1 -- 172.21.15.1:0/2721152927 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9028073a90 msgr2=0x7f9028199510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:11.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.126+0000 7f902fa00700 1 --2- 172.21.15.1:0/2721152927 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9028073a90 0x7f9028199510 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f9010000c00 tx=0x7f9010005740 comp rx=0 tx=0).stop 2024-02-02T15:39:11.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.126+0000 7f902fa00700 1 -- 172.21.15.1:0/2721152927 shutdown_connections 2024-02-02T15:39:11.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.126+0000 7f902fa00700 1 --2- 172.21.15.1:0/2721152927 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f90080a0690 0x7f90080a2b50 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.126+0000 7f902fa00700 1 --2- 172.21.15.1:0/2721152927 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f902810d4f0 0x7f902819d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.126+0000 7f902fa00700 1 --2- 172.21.15.1:0/2721152927 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9028073170 0x7f9028198fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.126+0000 7f902fa00700 1 --2- 172.21.15.1:0/2721152927 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9028073a90 0x7f9028199510 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.126+0000 7f902fa00700 1 -- 172.21.15.1:0/2721152927 >> 172.21.15.1:0/2721152927 conn(0x7f90280fc7a0 msgr2=0x7f902810f630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:11.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.126+0000 7f902fa00700 1 -- 172.21.15.1:0/2721152927 shutdown_connections 2024-02-02T15:39:11.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.126+0000 7f902fa00700 1 -- 172.21.15.1:0/2721152927 wait complete. 2024-02-02T15:39:11.127 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 52 2024-02-02T15:39:11.145 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-02-02T15:39:11.270 INFO:tasks.workunit.client.0.smithi001.stdout: 1 666490 5.43 MB/sec execute 525 sec latency 595.616 ms 2024-02-02T15:39:11.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.359+0000 7f86f9600700 1 -- 172.21.15.1:0/1993632284 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86f40ffec0 msgr2=0x7f86f41002a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:11.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.359+0000 7f86f9600700 1 --2- 172.21.15.1:0/1993632284 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86f40ffec0 0x7f86f41002a0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f86e4009990 tx=0x7f86e4009ca0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.359+0000 7f86f9600700 1 -- 172.21.15.1:0/1993632284 shutdown_connections 2024-02-02T15:39:11.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.359+0000 7f86f9600700 1 --2- 172.21.15.1:0/1993632284 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86f4104fd0 0x7f86f410cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.359+0000 7f86f9600700 1 --2- 172.21.15.1:0/1993632284 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86f40ffec0 0x7f86f41002a0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.359+0000 7f86f9600700 1 --2- 172.21.15.1:0/1993632284 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f86f4100870 0x7f86f41048a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.359+0000 7f86f9600700 1 -- 172.21.15.1:0/1993632284 >> 172.21.15.1:0/1993632284 conn(0x7f86f4076d60 msgr2=0x7f86f4077170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:11.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.360+0000 7f86f9600700 1 -- 172.21.15.1:0/1993632284 shutdown_connections 2024-02-02T15:39:11.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.360+0000 7f86f9600700 1 -- 172.21.15.1:0/1993632284 wait complete. 2024-02-02T15:39:11.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.360+0000 7f86f9600700 1 Processor -- start 2024-02-02T15:39:11.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.361+0000 7f86f9600700 1 -- start start 2024-02-02T15:39:11.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.361+0000 7f86f9600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86f40ffec0 0x7f86f4101940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:11.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.361+0000 7f86f9600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86f4104fd0 0x7f86f4101e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:11.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.361+0000 7f86f9600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f86f41023c0 0x7f86f41a5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:11.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.361+0000 7f86f9600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f41a5710 con 0x7f86f41023c0 2024-02-02T15:39:11.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.361+0000 7f86f9600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f41a5880 con 0x7f86f40ffec0 2024-02-02T15:39:11.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.361+0000 7f86f9600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f41a5a40 con 0x7f86f4104fd0 2024-02-02T15:39:11.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.361+0000 7f86f2a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86f40ffec0 0x7f86f4101940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:11.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.361+0000 7f86f2000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86f4104fd0 0x7f86f4101e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:11.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.361+0000 7f86f2a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86f40ffec0 0x7f86f4101940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:60986/0 (socket says 172.21.15.1:60986) 2024-02-02T15:39:11.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.361+0000 7f86f2000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86f4104fd0 0x7f86f4101e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:53906/0 (socket says 172.21.15.1:53906) 2024-02-02T15:39:11.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.361+0000 7f86f2a00700 1 -- 172.21.15.1:0/2943952295 learned_addr learned my addr 172.21.15.1:0/2943952295 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:39:11.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.361+0000 7f86f3400700 1 --2- 172.21.15.1:0/2943952295 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f86f41023c0 0x7f86f41a5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:11.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.362+0000 7f86f2a00700 1 -- 172.21.15.1:0/2943952295 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86f4104fd0 msgr2=0x7f86f4101e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:11.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.362+0000 7f86f2a00700 1 --2- 172.21.15.1:0/2943952295 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86f4104fd0 0x7f86f4101e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.362+0000 7f86f2a00700 1 -- 172.21.15.1:0/2943952295 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f86f41023c0 msgr2=0x7f86f41a5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:11.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.362+0000 7f86f2a00700 1 --2- 172.21.15.1:0/2943952295 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f86f41023c0 0x7f86f41a5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.362+0000 7f86f2a00700 1 -- 172.21.15.1:0/2943952295 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86dc009520 con 0x7f86f40ffec0 2024-02-02T15:39:11.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.362+0000 7f86f3400700 1 --2- 172.21.15.1:0/2943952295 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f86f41023c0 0x7f86f41a5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:39:11.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.362+0000 7f86f2a00700 1 --2- 172.21.15.1:0/2943952295 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86f40ffec0 0x7f86f4101940 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f86e4004730 tx=0x7f86e4004760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:11.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.362+0000 7f86eb400700 1 -- 172.21.15.1:0/2943952295 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e401d070 con 0x7f86f40ffec0 2024-02-02T15:39:11.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.362+0000 7f86eb400700 1 -- 172.21.15.1:0/2943952295 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86e4005a00 con 0x7f86f40ffec0 2024-02-02T15:39:11.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.362+0000 7f86f9600700 1 -- 172.21.15.1:0/2943952295 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86e40095f0 con 0x7f86f40ffec0 2024-02-02T15:39:11.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.362+0000 7f86eb400700 1 -- 172.21.15.1:0/2943952295 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e4005450 con 0x7f86f40ffec0 2024-02-02T15:39:11.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.362+0000 7f86f9600700 1 -- 172.21.15.1:0/2943952295 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86f41a5fd0 con 0x7f86f40ffec0 2024-02-02T15:39:11.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.366+0000 7f86eb400700 1 -- 172.21.15.1:0/2943952295 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f86e40047d0 con 0x7f86f40ffec0 2024-02-02T15:39:11.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.367+0000 7f86eb400700 1 --2- 172.21.15.1:0/2943952295 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f86d40a0710 0x7f86d40a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:11.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.367+0000 7f86eb400700 1 -- 172.21.15.1:0/2943952295 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f86e40cd280 con 0x7f86f40ffec0 2024-02-02T15:39:11.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.367+0000 7f86f2000700 1 --2- 172.21.15.1:0/2943952295 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f86d40a0710 0x7f86d40a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:11.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.367+0000 7f870d18bb80 1 Processor -- start 2024-02-02T15:39:11.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.368+0000 7f870d18bb80 1 -- start start 2024-02-02T15:39:11.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.368+0000 7f870d18bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56154e23f010 0x56154e23f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:11.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.368+0000 7f870d18bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56154e23f930 0x56154e24a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:11.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.368+0000 7f870d18bb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56154e24a7e0 0x56154e24cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:11.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.368+0000 7f870d18bb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56154e1b34c0 con 0x56154e24a7e0 2024-02-02T15:39:11.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.368+0000 7f870d18bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56154e197aa0 con 0x56154e23f010 2024-02-02T15:39:11.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.368+0000 7f870d18bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56154e195460 con 0x56154e23f930 2024-02-02T15:39:11.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.368+0000 7f86f2000700 1 --2- 172.21.15.1:0/2943952295 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f86d40a0710 0x7f86d40a2bd0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f86f4103130 tx=0x7f86dc009260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:11.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.368+0000 7f86f2a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56154e23f010 0x56154e23f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:11.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.368+0000 7f86f2a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56154e23f010 0x56154e23f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:32770/0 (socket says 172.21.15.1:32770) 2024-02-02T15:39:11.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.368+0000 7f86f2000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56154e23f930 0x56154e24a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:11.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.368+0000 7f86f2a00700 1 -- 172.21.15.1:0/2931206895 learned_addr learned my addr 172.21.15.1:0/2931206895 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:39:11.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.368+0000 7f86f3400700 1 --2- 172.21.15.1:0/2931206895 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56154e24a7e0 0x56154e24cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:11.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.369+0000 7f86f2a00700 1 -- 172.21.15.1:0/2931206895 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56154e23f930 msgr2=0x56154e24a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:11.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.369+0000 7f86f2a00700 1 --2- 172.21.15.1:0/2931206895 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56154e23f930 0x56154e24a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.369+0000 7f86f2a00700 1 -- 172.21.15.1:0/2931206895 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56154e24a7e0 msgr2=0x56154e24cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:11.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.369+0000 7f86f2a00700 1 --2- 172.21.15.1:0/2931206895 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56154e24a7e0 0x56154e24cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.369+0000 7f86f2a00700 1 -- 172.21.15.1:0/2931206895 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56154e0123b0 con 0x56154e23f010 2024-02-02T15:39:11.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.369+0000 7f86f2a00700 1 --2- 172.21.15.1:0/2931206895 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56154e23f010 0x56154e23f3f0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f86e409a0d0 tx=0x7f86e4009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f6adc9b23c86ddd server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:11.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.369+0000 7f86e3400700 1 -- 172.21.15.1:0/2931206895 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e409a3d0 con 0x56154e23f010 2024-02-02T15:39:11.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.369+0000 7f86e3400700 1 -- 172.21.15.1:0/2931206895 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86e409e200 con 0x56154e23f010 2024-02-02T15:39:11.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.369+0000 7f86e3400700 1 -- 172.21.15.1:0/2931206895 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e40a00d0 con 0x56154e23f010 2024-02-02T15:39:11.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.370+0000 7f870d18bb80 1 -- 172.21.15.1:0/2931206895 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56154e23f010 msgr2=0x56154e23f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:11.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.370+0000 7f870d18bb80 1 --2- 172.21.15.1:0/2931206895 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56154e23f010 0x56154e23f3f0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f86e409a0d0 tx=0x7f86e4009ca0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.370+0000 7f870d18bb80 1 -- 172.21.15.1:0/2931206895 shutdown_connections 2024-02-02T15:39:11.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.370+0000 7f870d18bb80 1 --2- 172.21.15.1:0/2931206895 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56154e24a7e0 0x56154e24cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.370+0000 7f870d18bb80 1 --2- 172.21.15.1:0/2931206895 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56154e23f010 0x56154e23f3f0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.370+0000 7f870d18bb80 1 --2- 172.21.15.1:0/2931206895 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56154e23f930 0x56154e24a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.370+0000 7f870d18bb80 1 -- 172.21.15.1:0/2931206895 >> 172.21.15.1:0/2931206895 conn(0x56154e19edb0 msgr2=0x56154e23b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:11.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.370+0000 7f870d18bb80 1 -- 172.21.15.1:0/2931206895 shutdown_connections 2024-02-02T15:39:11.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.370+0000 7f870d18bb80 1 -- 172.21.15.1:0/2931206895 wait complete. 2024-02-02T15:39:11.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.371+0000 7f870d18bb80 1 -- start start 2024-02-02T15:39:11.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.371+0000 7f870d18bb80 1 Processor -- start 2024-02-02T15:39:11.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.371+0000 7f870d18bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56154e23f010 0x56154e24abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:11.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.371+0000 7f870d18bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56154e23f930 0x56154e24b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:11.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.371+0000 7f870d18bb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56154e24a7e0 0x56154e24b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:11.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.371+0000 7f870d18bb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56154e190d50 con 0x56154e24a7e0 2024-02-02T15:39:11.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.371+0000 7f870d18bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56154e12cd80 con 0x56154e23f010 2024-02-02T15:39:11.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.371+0000 7f870d18bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56154e12ca20 con 0x56154e23f930 2024-02-02T15:39:11.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.371+0000 7f870d18bb80 10 client.0 _ll_register_callbacks cb 0x56154e12e290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:39:11.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.372+0000 7f86f2a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56154e23f010 0x56154e24abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:11.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.372+0000 7f86f2000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56154e23f930 0x56154e24b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:11.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.372+0000 7f86f3400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56154e24a7e0 0x56154e24b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:11.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.372+0000 7f86f2a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56154e23f010 0x56154e24abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:32782/0 (socket says 172.21.15.1:32782) 2024-02-02T15:39:11.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.372+0000 7f86f2000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56154e23f930 0x56154e24b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:53912/0 (socket says 172.21.15.1:53912) 2024-02-02T15:39:11.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.372+0000 7f86f2a00700 1 -- 172.21.15.1:0/2254776154 learned_addr learned my addr 172.21.15.1:0/2254776154 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:39:11.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.372+0000 7f86f2a00700 1 -- 172.21.15.1:0/2254776154 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56154e23f930 msgr2=0x56154e24b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:11.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.372+0000 7f86f2a00700 1 --2- 172.21.15.1:0/2254776154 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56154e23f930 0x56154e24b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.372+0000 7f86f2a00700 1 -- 172.21.15.1:0/2254776154 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56154e24a7e0 msgr2=0x56154e24b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:11.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.372+0000 7f86f2a00700 1 --2- 172.21.15.1:0/2254776154 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56154e24a7e0 0x56154e24b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.372+0000 7f86f2a00700 1 -- 172.21.15.1:0/2254776154 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86f41a5fd0 con 0x56154e23f010 2024-02-02T15:39:11.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.372+0000 7f86f2000700 1 --2- 172.21.15.1:0/2254776154 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56154e23f930 0x56154e24b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:39:11.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.372+0000 7f86f3400700 1 --2- 172.21.15.1:0/2254776154 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56154e24a7e0 0x56154e24b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:39:11.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.372+0000 7f86f2a00700 1 --2- 172.21.15.1:0/2254776154 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56154e23f010 0x56154e24abc0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f86e400b690 tx=0x7f86e4028000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=900bf31140d896dc server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:11.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.372+0000 7f86dbe00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:39:11.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.372+0000 7f86dbe00700 1 -- 172.21.15.1:0/2254776154 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e40a23b0 con 0x56154e23f010 2024-02-02T15:39:11.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.373+0000 7f86dbe00700 1 -- 172.21.15.1:0/2254776154 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f86e40a02e0 con 0x56154e23f010 2024-02-02T15:39:11.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.373+0000 7f870d18bb80 1 -- 172.21.15.1:0/2254776154 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56154e2330f0 con 0x56154e23f010 2024-02-02T15:39:11.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.373+0000 7f86dbe00700 1 -- 172.21.15.1:0/2254776154 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86e409a3d0 con 0x56154e23f010 2024-02-02T15:39:11.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.373+0000 7f870d18bb80 10 client.28823 fetch_fsmap learned FSMap version 52 2024-02-02T15:39:11.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.373+0000 7f870d18bb80 1 -- 172.21.15.1:0/2254776154 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x56154dfbeb60 con 0x56154e23f010 2024-02-02T15:39:11.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.373+0000 7f86dbe00700 1 -- 172.21.15.1:0/2254776154 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86e40b6080 con 0x56154e23f010 2024-02-02T15:39:11.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.373+0000 7f86dbe00700 1 -- 172.21.15.1:0/2254776154 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 52) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f86e4083bd0 con 0x56154e23f010 2024-02-02T15:39:11.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.373+0000 7f870d18bb80 10 client.28823 fetch_fsmap finished waiting for FSMap version 52 2024-02-02T15:39:11.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.373+0000 7f870d18bb80 10 client.28823 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-02-02T15:39:11.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.373+0000 7f870d18bb80 20 client.28823 populate_metadata read hostname 'smithi001' 2024-02-02T15:39:11.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.373+0000 7f870d18bb80 1 --2- 172.21.15.1:0/2254776154 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x56154e2599d0 0x56154e25bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:11.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.373+0000 7f870d18bb80 4 client.28823 mds_command: new command op to 24502 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:39:11.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.373+0000 7f870d18bb80 1 -- 172.21.15.1:0/2254776154 --> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56154e189dd0 con 0x56154e2599d0 2024-02-02T15:39:11.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.373+0000 7f86f2000700 1 --2- 172.21.15.1:0/2254776154 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x56154e2599d0 0x56154e25bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:11.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.374+0000 7f86f2000700 1 --2- 172.21.15.1:0/2254776154 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x56154e2599d0 0x56154e25bdc0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8f6e5ac46e5b28f8 server_cookie=d08bf4eee6ce807e in_seq=0 out_seq=0 2024-02-02T15:39:11.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.374+0000 7f86dbe00700 10 client.28823 ms_handle_connect on v2:172.21.15.142:6834/1052606547 2024-02-02T15:39:11.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.375+0000 7f86dbe00700 1 -- 172.21.15.1:0/2254776154 <== mds.4 v2:172.21.15.142:6834/1052606547 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x56154e189dd0 con 0x56154e2599d0 2024-02-02T15:39:11.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.375+0000 7f86dbe00700 10 client.28823 handle_command_reply: tid=0 2024-02-02T15:39:11.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.375+0000 7f870d18bb80 1 client.28823 shutdown 2024-02-02T15:39:11.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.375+0000 7f870d18bb80 1 -- 172.21.15.1:0/2254776154 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56154e23f010 msgr2=0x56154e24abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:11.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.375+0000 7f870d18bb80 1 --2- 172.21.15.1:0/2254776154 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56154e23f010 0x56154e24abc0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f86e400b690 tx=0x7f86e4028000 comp rx=0 tx=0).stop 2024-02-02T15:39:11.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.375+0000 7f870d18bb80 1 -- 172.21.15.1:0/2254776154 shutdown_connections 2024-02-02T15:39:11.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.375+0000 7f870d18bb80 1 --2- 172.21.15.1:0/2254776154 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x56154e2599d0 0x56154e25bdc0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.375+0000 7f870d18bb80 1 --2- 172.21.15.1:0/2254776154 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56154e24a7e0 0x56154e24b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.375+0000 7f870d18bb80 1 --2- 172.21.15.1:0/2254776154 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56154e23f010 0x56154e24abc0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.375+0000 7f870d18bb80 1 --2- 172.21.15.1:0/2254776154 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56154e23f930 0x56154e24b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.375+0000 7f86dbe00700 0 client.28823 ms_handle_reset on v2:172.21.15.142:6834/1052606547 2024-02-02T15:39:11.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.375+0000 7f870d18bb80 1 -- 172.21.15.1:0/2254776154 >> 172.21.15.1:0/2254776154 conn(0x56154e19edb0 msgr2=0x56154e23afa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:11.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.375+0000 7f870d18bb80 1 -- 172.21.15.1:0/2254776154 shutdown_connections 2024-02-02T15:39:11.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.375+0000 7f870d18bb80 1 -- 172.21.15.1:0/2254776154 wait complete. 2024-02-02T15:39:11.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.375+0000 7f870d18bb80 20 client.28823 trim_cache size 0 max 16384 2024-02-02T15:39:11.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.390+0000 7f870d18bb80 1 Processor -- start 2024-02-02T15:39:11.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.390+0000 7f870d18bb80 1 -- start start 2024-02-02T15:39:11.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.390+0000 7f870d18bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56154e23f010 0x56154e241b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:11.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.390+0000 7f870d18bb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56154e23f930 0x56154e2551e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:11.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.390+0000 7f870d18bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56154e24a7e0 0x56154e246ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:11.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.390+0000 7f870d18bb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56154e0146a0 con 0x56154e23f930 2024-02-02T15:39:11.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.390+0000 7f870d18bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56154e232d40 con 0x56154e23f010 2024-02-02T15:39:11.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.390+0000 7f870d18bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56154e2420d0 con 0x56154e24a7e0 2024-02-02T15:39:11.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.390+0000 7f86f2000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56154e23f930 0x56154e2551e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:11.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.390+0000 7f86f2a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56154e23f010 0x56154e241b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:11.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.390+0000 7f86f3400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56154e24a7e0 0x56154e246ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:11.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.390+0000 7f86f2a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56154e23f010 0x56154e241b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:32786/0 (socket says 172.21.15.1:32786) 2024-02-02T15:39:11.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.390+0000 7f86f2000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56154e23f930 0x56154e2551e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50628/0 (socket says 172.21.15.1:50628) 2024-02-02T15:39:11.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.390+0000 7f86f2a00700 1 -- 172.21.15.1:0/67883254 learned_addr learned my addr 172.21.15.1:0/67883254 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:39:11.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.391+0000 7f86f2a00700 1 -- 172.21.15.1:0/67883254 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56154e24a7e0 msgr2=0x56154e246ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:11.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.391+0000 7f86f2a00700 1 --2- 172.21.15.1:0/67883254 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56154e24a7e0 0x56154e246ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.391+0000 7f86f2a00700 1 -- 172.21.15.1:0/67883254 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56154e23f930 msgr2=0x56154e2551e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:11.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.391+0000 7f86f2a00700 1 --2- 172.21.15.1:0/67883254 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56154e23f930 0x56154e2551e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.391+0000 7f86f2a00700 1 -- 172.21.15.1:0/67883254 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56154e156ea0 con 0x56154e23f010 2024-02-02T15:39:11.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.391+0000 7f86f2000700 1 --2- 172.21.15.1:0/67883254 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56154e23f930 0x56154e2551e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:39:11.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.391+0000 7f86f2a00700 1 --2- 172.21.15.1:0/67883254 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56154e23f010 0x56154e241b10 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f86e4096df0 tx=0x7f86e4009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=545e1e570cbca749 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:11.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.391+0000 7f86e3e00700 1 -- 172.21.15.1:0/67883254 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e4099d30 con 0x56154e23f010 2024-02-02T15:39:11.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.391+0000 7f86e3e00700 1 -- 172.21.15.1:0/67883254 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56154e2330f0 con 0x56154e23f010 2024-02-02T15:39:11.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.391+0000 7f86e3e00700 1 -- 172.21.15.1:0/67883254 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e40a2160 con 0x56154e23f010 2024-02-02T15:39:11.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.392+0000 7f870d18bb80 1 -- 172.21.15.1:0/67883254 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56154e23f010 msgr2=0x56154e241b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:11.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.392+0000 7f870d18bb80 1 --2- 172.21.15.1:0/67883254 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56154e23f010 0x56154e241b10 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f86e4096df0 tx=0x7f86e4009ca0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.392+0000 7f870d18bb80 1 -- 172.21.15.1:0/67883254 shutdown_connections 2024-02-02T15:39:11.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.392+0000 7f870d18bb80 1 --2- 172.21.15.1:0/67883254 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56154e24a7e0 0x56154e246ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.392+0000 7f870d18bb80 1 --2- 172.21.15.1:0/67883254 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56154e23f010 0x56154e241b10 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.392+0000 7f870d18bb80 1 --2- 172.21.15.1:0/67883254 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56154e23f930 0x56154e2551e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.392+0000 7f870d18bb80 1 -- 172.21.15.1:0/67883254 >> 172.21.15.1:0/67883254 conn(0x56154e19edb0 msgr2=0x56154e244c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:11.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.392+0000 7f870d18bb80 1 -- 172.21.15.1:0/67883254 shutdown_connections 2024-02-02T15:39:11.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.392+0000 7f870d18bb80 1 -- 172.21.15.1:0/67883254 wait complete. 2024-02-02T15:39:11.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.392+0000 7f870d18bb80 1 -- start start 2024-02-02T15:39:11.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.393+0000 7f870d18bb80 1 Processor -- start 2024-02-02T15:39:11.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.393+0000 7f870d18bb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56154e23f010 0x56154e24d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:11.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.393+0000 7f870d18bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56154e23f930 0x56154e24d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:11.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.393+0000 7f870d18bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56154e24a7e0 0x56154e2aa490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:11.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.393+0000 7f870d18bb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56154e24e070 con 0x56154e23f010 2024-02-02T15:39:11.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.393+0000 7f870d18bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56154e24e1e0 con 0x56154e23f930 2024-02-02T15:39:11.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.393+0000 7f870d18bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56154e2aa9d0 con 0x56154e24a7e0 2024-02-02T15:39:11.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.393+0000 7f870d18bb80 10 client.0 _ll_register_callbacks cb 0x56154e12e290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:39:11.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.393+0000 7f86f2a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56154e23f010 0x56154e24d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:11.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.393+0000 7f86f3400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56154e24a7e0 0x56154e2aa490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:11.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.394+0000 7f86f2a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56154e23f010 0x56154e24d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50638/0 (socket says 172.21.15.1:50638) 2024-02-02T15:39:11.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.394+0000 7f86f3400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56154e24a7e0 0x56154e2aa490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:53924/0 (socket says 172.21.15.1:53924) 2024-02-02T15:39:11.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.394+0000 7f86f2a00700 1 -- 172.21.15.1:0/3219306988 learned_addr learned my addr 172.21.15.1:0/3219306988 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:39:11.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.394+0000 7f86f2000700 1 --2- 172.21.15.1:0/3219306988 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56154e23f930 0x56154e24d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:11.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.394+0000 7f86f2000700 1 -- 172.21.15.1:0/3219306988 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56154e24a7e0 msgr2=0x56154e2aa490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:11.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.394+0000 7f86f2000700 1 --2- 172.21.15.1:0/3219306988 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56154e24a7e0 0x56154e2aa490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.394+0000 7f86f2000700 1 -- 172.21.15.1:0/3219306988 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56154e23f010 msgr2=0x56154e24d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:11.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.394+0000 7f86f2000700 1 --2- 172.21.15.1:0/3219306988 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56154e23f010 0x56154e24d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.394+0000 7f86f2000700 1 -- 172.21.15.1:0/3219306988 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86dc017070 con 0x56154e23f930 2024-02-02T15:39:11.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.394+0000 7f86f2000700 1 --2- 172.21.15.1:0/3219306988 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56154e23f930 0x56154e24d8d0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x56154e25c640 tx=0x7f86dc0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=749363819e87c7c1 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:11.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.394+0000 7f86dbe00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:39:11.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.394+0000 7f86dbe00700 1 -- 172.21.15.1:0/3219306988 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86dc022040 con 0x56154e23f930 2024-02-02T15:39:11.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.395+0000 7f86dbe00700 1 -- 172.21.15.1:0/3219306988 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f86dc00ed90 con 0x56154e23f930 2024-02-02T15:39:11.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.395+0000 7f870d18bb80 1 -- 172.21.15.1:0/3219306988 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f86e409e200 con 0x56154e23f930 2024-02-02T15:39:11.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.395+0000 7f86dbe00700 1 -- 172.21.15.1:0/3219306988 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86dc02be60 con 0x56154e23f930 2024-02-02T15:39:11.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.395+0000 7f870d18bb80 10 client.28829 fetch_fsmap learned FSMap version 52 2024-02-02T15:39:11.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.395+0000 7f870d18bb80 1 -- 172.21.15.1:0/3219306988 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x56154dfbeb60 con 0x56154e23f930 2024-02-02T15:39:11.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.395+0000 7f86dbe00700 1 -- 172.21.15.1:0/3219306988 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86e409e200 con 0x56154e23f930 2024-02-02T15:39:11.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.395+0000 7f86dbe00700 1 -- 172.21.15.1:0/3219306988 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 52) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f86dc0335b0 con 0x56154e23f930 2024-02-02T15:39:11.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.395+0000 7f870d18bb80 10 client.28829 fetch_fsmap finished waiting for FSMap version 52 2024-02-02T15:39:11.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.395+0000 7f870d18bb80 10 client.28829 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-02-02T15:39:11.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.395+0000 7f870d18bb80 20 client.28829 populate_metadata read hostname 'smithi001' 2024-02-02T15:39:11.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.395+0000 7f870d18bb80 1 --2- 172.21.15.1:0/3219306988 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x56154e2ac110 0x56154e2ae530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:11.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.395+0000 7f870d18bb80 4 client.28829 mds_command: new command op to 24502 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:39:11.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.395+0000 7f870d18bb80 1 -- 172.21.15.1:0/3219306988 --> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56154e14ff50 con 0x56154e2ac110 2024-02-02T15:39:11.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.396+0000 7f86f2a00700 1 --2- 172.21.15.1:0/3219306988 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x56154e2ac110 0x56154e2ae530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:11.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.396+0000 7f86f2a00700 1 --2- 172.21.15.1:0/3219306988 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x56154e2ac110 0x56154e2ae530 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=495670a7eea3da63 server_cookie=c736cc3ba05cd888 in_seq=0 out_seq=0 2024-02-02T15:39:11.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.396+0000 7f86dbe00700 10 client.28829 ms_handle_connect on v2:172.21.15.142:6834/1052606547 2024-02-02T15:39:11.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.396+0000 7f86dbe00700 1 -- 172.21.15.1:0/3219306988 <== mds.4 v2:172.21.15.142:6834/1052606547 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56154e14ff50 con 0x56154e2ac110 2024-02-02T15:39:11.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.396+0000 7f86dbe00700 10 client.28829 handle_command_reply: tid=0 2024-02-02T15:39:11.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.396+0000 7f870d18bb80 1 client.28829 shutdown 2024-02-02T15:39:11.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.396+0000 7f870d18bb80 1 -- 172.21.15.1:0/3219306988 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56154e23f930 msgr2=0x56154e24d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:11.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.396+0000 7f870d18bb80 1 --2- 172.21.15.1:0/3219306988 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56154e23f930 0x56154e24d8d0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x56154e25c640 tx=0x7f86dc0047c0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.396+0000 7f870d18bb80 1 -- 172.21.15.1:0/3219306988 shutdown_connections 2024-02-02T15:39:11.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.396+0000 7f870d18bb80 1 --2- 172.21.15.1:0/3219306988 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x56154e2ac110 0x56154e2ae530 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.396+0000 7f870d18bb80 1 --2- 172.21.15.1:0/3219306988 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56154e24a7e0 0x56154e2aa490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.396+0000 7f870d18bb80 1 --2- 172.21.15.1:0/3219306988 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56154e23f010 0x56154e24d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.396+0000 7f86dbe00700 0 client.28829 ms_handle_reset on v2:172.21.15.142:6834/1052606547 2024-02-02T15:39:11.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.396+0000 7f870d18bb80 1 --2- 172.21.15.1:0/3219306988 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56154e23f930 0x56154e24d8d0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.396+0000 7f870d18bb80 1 -- 172.21.15.1:0/3219306988 >> 172.21.15.1:0/3219306988 conn(0x56154e19edb0 msgr2=0x56154e24abc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:11.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.397+0000 7f870d18bb80 1 -- 172.21.15.1:0/3219306988 shutdown_connections 2024-02-02T15:39:11.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.397+0000 7f870d18bb80 1 -- 172.21.15.1:0/3219306988 wait complete. 2024-02-02T15:39:11.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.397+0000 7f870d18bb80 20 client.28829 trim_cache size 0 max 16384 2024-02-02T15:39:11.402 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:39:11.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.397+0000 7f86f9600700 1 -- 172.21.15.1:0/2943952295 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f86d40a0710 msgr2=0x7f86d40a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:11.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.397+0000 7f86f9600700 1 --2- 172.21.15.1:0/2943952295 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f86d40a0710 0x7f86d40a2bd0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f86f4103130 tx=0x7f86dc009260 comp rx=0 tx=0).stop 2024-02-02T15:39:11.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.397+0000 7f86f9600700 1 -- 172.21.15.1:0/2943952295 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86f40ffec0 msgr2=0x7f86f4101940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:11.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.397+0000 7f86f9600700 1 --2- 172.21.15.1:0/2943952295 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86f40ffec0 0x7f86f4101940 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f86e4004730 tx=0x7f86e4004760 comp rx=0 tx=0).stop 2024-02-02T15:39:11.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.397+0000 7f86f9600700 1 -- 172.21.15.1:0/2943952295 shutdown_connections 2024-02-02T15:39:11.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.397+0000 7f86f9600700 1 --2- 172.21.15.1:0/2943952295 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f86d40a0710 0x7f86d40a2bd0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.397+0000 7f86f9600700 1 --2- 172.21.15.1:0/2943952295 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f86f41023c0 0x7f86f41a5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.398+0000 7f86f9600700 1 --2- 172.21.15.1:0/2943952295 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86f40ffec0 0x7f86f4101940 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.398+0000 7f86f9600700 1 --2- 172.21.15.1:0/2943952295 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86f4104fd0 0x7f86f4101e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.398+0000 7f86f9600700 1 -- 172.21.15.1:0/2943952295 >> 172.21.15.1:0/2943952295 conn(0x7f86f4076d60 msgr2=0x7f86f4103b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:11.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.398+0000 7f86f9600700 1 -- 172.21.15.1:0/2943952295 shutdown_connections 2024-02-02T15:39:11.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.398+0000 7f86f9600700 1 -- 172.21.15.1:0/2943952295 wait complete. 2024-02-02T15:39:11.416 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-02-02T15:39:11.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.633+0000 7f102c200700 1 -- 172.21.15.1:0/771290506 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1024102950 msgr2=0x7f102410ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:11.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.633+0000 7f102c200700 1 --2- 172.21.15.1:0/771290506 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1024102950 0x7f102410ae30 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f100c009990 tx=0x7f100c009ca0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.633+0000 7f102c200700 1 -- 172.21.15.1:0/771290506 shutdown_connections 2024-02-02T15:39:11.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.633+0000 7f102c200700 1 --2- 172.21.15.1:0/771290506 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f102410b400 0x7f102410d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.633+0000 7f102c200700 1 --2- 172.21.15.1:0/771290506 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1024102030 0x7f1024102410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.633+0000 7f102c200700 1 --2- 172.21.15.1:0/771290506 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1024102950 0x7f102410ae30 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.633+0000 7f102c200700 1 -- 172.21.15.1:0/771290506 >> 172.21.15.1:0/771290506 conn(0x7f1024076d70 msgr2=0x7f1024077180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:11.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.633+0000 7f102c200700 1 -- 172.21.15.1:0/771290506 shutdown_connections 2024-02-02T15:39:11.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.633+0000 7f102c200700 1 -- 172.21.15.1:0/771290506 wait complete. 2024-02-02T15:39:11.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.635+0000 7f102c200700 1 Processor -- start 2024-02-02T15:39:11.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.635+0000 7f102c200700 1 -- start start 2024-02-02T15:39:11.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.636+0000 7f102c200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1024102030 0x7f10240ffc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:11.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.636+0000 7f102c200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1024102950 0x7f10241001c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:11.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.636+0000 7f102c200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f102410b400 0x7f10241a5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:11.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.636+0000 7f102c200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10241009f0 con 0x7f1024102030 2024-02-02T15:39:11.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.636+0000 7f102c200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10241a55d0 con 0x7f1024102950 2024-02-02T15:39:11.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.636+0000 7f102c200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10241a5710 con 0x7f102410b400 2024-02-02T15:39:11.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.636+0000 7f1028e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1024102950 0x7f10241001c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:11.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.636+0000 7f1029800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1024102030 0x7f10240ffc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:11.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.636+0000 7f1028e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1024102950 0x7f10241001c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:32808/0 (socket says 172.21.15.1:32808) 2024-02-02T15:39:11.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.636+0000 7f1028e00700 1 -- 172.21.15.1:0/424944157 learned_addr learned my addr 172.21.15.1:0/424944157 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:39:11.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.636+0000 7f102a200700 1 --2- 172.21.15.1:0/424944157 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f102410b400 0x7f10241a5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:11.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.636+0000 7f1028e00700 1 -- 172.21.15.1:0/424944157 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f102410b400 msgr2=0x7f10241a5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:11.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.636+0000 7f1028e00700 1 --2- 172.21.15.1:0/424944157 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f102410b400 0x7f10241a5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.636+0000 7f1028e00700 1 -- 172.21.15.1:0/424944157 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1024102030 msgr2=0x7f10240ffc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:11.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.636+0000 7f1028e00700 1 --2- 172.21.15.1:0/424944157 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1024102030 0x7f10240ffc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.636+0000 7f1028e00700 1 -- 172.21.15.1:0/424944157 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f100c0095f0 con 0x7f1024102950 2024-02-02T15:39:11.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.636+0000 7f1029800700 1 --2- 172.21.15.1:0/424944157 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1024102030 0x7f10240ffc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:39:11.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.637+0000 7f1028e00700 1 --2- 172.21.15.1:0/424944157 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1024102950 0x7f10241001c0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f100c000c00 tx=0x7f100c0047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:11.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.637+0000 7f1022000700 1 -- 172.21.15.1:0/424944157 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f100c01d070 con 0x7f1024102950 2024-02-02T15:39:11.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.637+0000 7f1022000700 1 -- 172.21.15.1:0/424944157 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f100c021410 con 0x7f1024102950 2024-02-02T15:39:11.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.637+0000 7f102c200700 1 -- 172.21.15.1:0/424944157 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10241a5910 con 0x7f1024102950 2024-02-02T15:39:11.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.637+0000 7f1022000700 1 -- 172.21.15.1:0/424944157 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f100c017650 con 0x7f1024102950 2024-02-02T15:39:11.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.637+0000 7f102c200700 1 -- 172.21.15.1:0/424944157 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10241a5d90 con 0x7f1024102950 2024-02-02T15:39:11.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.640+0000 7f1022000700 1 -- 172.21.15.1:0/424944157 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f100c0217f0 con 0x7f1024102950 2024-02-02T15:39:11.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.642+0000 7f1022000700 1 --2- 172.21.15.1:0/424944157 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f10040a0690 0x7f10040a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:11.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.642+0000 7f1022000700 1 -- 172.21.15.1:0/424944157 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f100c0cd520 con 0x7f1024102950 2024-02-02T15:39:11.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.642+0000 7f1029800700 1 --2- 172.21.15.1:0/424944157 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f10040a0690 0x7f10040a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:11.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.642+0000 7f103fd95b80 1 Processor -- start 2024-02-02T15:39:11.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.642+0000 7f103fd95b80 1 -- start start 2024-02-02T15:39:11.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.643+0000 7f1029800700 1 --2- 172.21.15.1:0/424944157 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f10040a0690 0x7f10040a2b50 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1014011490 tx=0x7f1014009260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:11.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.643+0000 7f103fd95b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636c9f66010 0x5636c9f663f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:11.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.643+0000 7f103fd95b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5636c9f66930 0x5636c9f712a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:11.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.643+0000 7f103fd95b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636c9f717e0 0x5636c9f73bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:11.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.643+0000 7f103fd95b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636c9eda4c0 con 0x5636c9f66010 2024-02-02T15:39:11.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.643+0000 7f103fd95b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636c9ebeaa0 con 0x5636c9f66930 2024-02-02T15:39:11.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.643+0000 7f103fd95b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636c9ebc460 con 0x5636c9f717e0 2024-02-02T15:39:11.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.643+0000 7f1028e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5636c9f66930 0x5636c9f712a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:11.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.643+0000 7f102a200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636c9f717e0 0x5636c9f73bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:11.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.643+0000 7f1028e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5636c9f66930 0x5636c9f712a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:32816/0 (socket says 172.21.15.1:32816) 2024-02-02T15:39:11.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.643+0000 7f1028e00700 1 -- 172.21.15.1:0/623960588 learned_addr learned my addr 172.21.15.1:0/623960588 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:39:11.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.643+0000 7f1029800700 1 --2- 172.21.15.1:0/623960588 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636c9f66010 0x5636c9f663f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:11.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.643+0000 7f1028e00700 1 -- 172.21.15.1:0/623960588 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636c9f717e0 msgr2=0x5636c9f73bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:11.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.643+0000 7f1028e00700 1 --2- 172.21.15.1:0/623960588 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636c9f717e0 0x5636c9f73bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.643+0000 7f1028e00700 1 -- 172.21.15.1:0/623960588 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636c9f66010 msgr2=0x5636c9f663f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:11.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.643+0000 7f1028e00700 1 --2- 172.21.15.1:0/623960588 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636c9f66010 0x5636c9f663f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.643+0000 7f1028e00700 1 -- 172.21.15.1:0/623960588 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5636c9d393b0 con 0x5636c9f66930 2024-02-02T15:39:11.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.644+0000 7f1028e00700 1 --2- 172.21.15.1:0/623960588 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5636c9f66930 0x5636c9f712a0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f100c005910 tx=0x7f100c09a1b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=862bcdfa3312cd6f server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:11.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.644+0000 7f101aa00700 1 -- 172.21.15.1:0/623960588 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f100c09a670 con 0x5636c9f66930 2024-02-02T15:39:11.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.644+0000 7f101aa00700 1 -- 172.21.15.1:0/623960588 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f100c021e90 con 0x5636c9f66930 2024-02-02T15:39:11.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.644+0000 7f101aa00700 1 -- 172.21.15.1:0/623960588 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f100c09e120 con 0x5636c9f66930 2024-02-02T15:39:11.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.645+0000 7f103fd95b80 1 -- 172.21.15.1:0/623960588 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5636c9f66930 msgr2=0x5636c9f712a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:11.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.645+0000 7f103fd95b80 1 --2- 172.21.15.1:0/623960588 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5636c9f66930 0x5636c9f712a0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f100c005910 tx=0x7f100c09a1b0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.645+0000 7f103fd95b80 1 -- 172.21.15.1:0/623960588 shutdown_connections 2024-02-02T15:39:11.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.645+0000 7f103fd95b80 1 --2- 172.21.15.1:0/623960588 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636c9f717e0 0x5636c9f73bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.645+0000 7f103fd95b80 1 --2- 172.21.15.1:0/623960588 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636c9f66010 0x5636c9f663f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.645+0000 7f103fd95b80 1 --2- 172.21.15.1:0/623960588 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5636c9f66930 0x5636c9f712a0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.645+0000 7f103fd95b80 1 -- 172.21.15.1:0/623960588 >> 172.21.15.1:0/623960588 conn(0x5636c9ec5db0 msgr2=0x5636c9f623b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:11.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.645+0000 7f103fd95b80 1 -- 172.21.15.1:0/623960588 shutdown_connections 2024-02-02T15:39:11.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.645+0000 7f103fd95b80 1 -- 172.21.15.1:0/623960588 wait complete. 2024-02-02T15:39:11.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.645+0000 7f103fd95b80 1 -- start start 2024-02-02T15:39:11.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.646+0000 7f103fd95b80 1 Processor -- start 2024-02-02T15:39:11.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.647+0000 7f103fd95b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636c9f66010 0x5636c9f71bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:11.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.647+0000 7f103fd95b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636c9f66930 0x5636c9f72100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:11.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.647+0000 7f103fd95b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5636c9f717e0 0x5636c9f72640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:11.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.647+0000 7f103fd95b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636c9eb7d50 con 0x5636c9f66930 2024-02-02T15:39:11.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.647+0000 7f103fd95b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636c9e53d80 con 0x5636c9f717e0 2024-02-02T15:39:11.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.647+0000 7f103fd95b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636c9e53a20 con 0x5636c9f66010 2024-02-02T15:39:11.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.647+0000 7f103fd95b80 10 client.0 _ll_register_callbacks cb 0x5636c9e55290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:39:11.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.647+0000 7f1029800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636c9f66010 0x5636c9f71bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:11.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.647+0000 7f1029800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636c9f66010 0x5636c9f71bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:53972/0 (socket says 172.21.15.1:53972) 2024-02-02T15:39:11.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.647+0000 7f1029800700 1 -- 172.21.15.1:0/1190281205 learned_addr learned my addr 172.21.15.1:0/1190281205 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:39:11.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.647+0000 7f102a200700 1 --2- 172.21.15.1:0/1190281205 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5636c9f717e0 0x5636c9f72640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:11.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.647+0000 7f1028e00700 1 --2- 172.21.15.1:0/1190281205 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636c9f66930 0x5636c9f72100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:11.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.647+0000 7f1029800700 1 -- 172.21.15.1:0/1190281205 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5636c9f717e0 msgr2=0x5636c9f72640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:11.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.647+0000 7f1029800700 1 --2- 172.21.15.1:0/1190281205 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5636c9f717e0 0x5636c9f72640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.647+0000 7f1029800700 1 -- 172.21.15.1:0/1190281205 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636c9f66930 msgr2=0x5636c9f72100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:11.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.647+0000 7f1029800700 1 --2- 172.21.15.1:0/1190281205 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636c9f66930 0x5636c9f72100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.647+0000 7f1029800700 1 -- 172.21.15.1:0/1190281205 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f100c083e70 con 0x5636c9f66010 2024-02-02T15:39:11.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.648+0000 7f1029800700 1 --2- 172.21.15.1:0/1190281205 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636c9f66010 0x5636c9f71bc0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f1014003ad0 tx=0x7f10140047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cced294718b0a637 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:11.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.648+0000 7f1013400700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:39:11.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.648+0000 7f1013400700 1 -- 172.21.15.1:0/1190281205 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f101401fca0 con 0x5636c9f66010 2024-02-02T15:39:11.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.648+0000 7f1013400700 1 -- 172.21.15.1:0/1190281205 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f101401fe00 con 0x5636c9f66010 2024-02-02T15:39:11.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.648+0000 7f103fd95b80 1 -- 172.21.15.1:0/1190281205 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5636c9f5a0f0 con 0x5636c9f66010 2024-02-02T15:39:11.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.648+0000 7f1013400700 1 -- 172.21.15.1:0/1190281205 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1014003e90 con 0x5636c9f66010 2024-02-02T15:39:11.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.648+0000 7f103fd95b80 10 client.28843 fetch_fsmap learned FSMap version 52 2024-02-02T15:39:11.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.648+0000 7f103fd95b80 1 -- 172.21.15.1:0/1190281205 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x5636c9ce5b60 con 0x5636c9f66010 2024-02-02T15:39:11.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.648+0000 7f1013400700 1 -- 172.21.15.1:0/1190281205 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5636c9f5a0f0 con 0x5636c9f66010 2024-02-02T15:39:11.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.649+0000 7f1013400700 1 -- 172.21.15.1:0/1190281205 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 52) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f1014027590 con 0x5636c9f66010 2024-02-02T15:39:11.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.649+0000 7f103fd95b80 10 client.28843 fetch_fsmap finished waiting for FSMap version 52 2024-02-02T15:39:11.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.649+0000 7f103fd95b80 10 client.28843 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-02-02T15:39:11.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.649+0000 7f103fd95b80 20 client.28843 populate_metadata read hostname 'smithi001' 2024-02-02T15:39:11.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.649+0000 7f103fd95b80 1 --2- 172.21.15.1:0/1190281205 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x5636c9f80f50 0x5636c9f83370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:11.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.649+0000 7f103fd95b80 4 client.28843 mds_command: new command op to 24514 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:39:11.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.649+0000 7f103fd95b80 1 -- 172.21.15.1:0/1190281205 --> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5636c9eb0dd0 con 0x5636c9f80f50 2024-02-02T15:39:11.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.649+0000 7f1028e00700 1 --2- 172.21.15.1:0/1190281205 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x5636c9f80f50 0x5636c9f83370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:11.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.649+0000 7f1028e00700 1 --2- 172.21.15.1:0/1190281205 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x5636c9f80f50 0x5636c9f83370 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5f1fd02c46ba5a89 server_cookie=f6cb3ed757ec4014 in_seq=0 out_seq=0 2024-02-02T15:39:11.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.649+0000 7f1013400700 10 client.28843 ms_handle_connect on v2:172.21.15.142:6836/2870289019 2024-02-02T15:39:11.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.650+0000 7f1013400700 1 -- 172.21.15.1:0/1190281205 <== mds.3 v2:172.21.15.142:6836/2870289019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5636c9eb0dd0 con 0x5636c9f80f50 2024-02-02T15:39:11.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.650+0000 7f1013400700 10 client.28843 handle_command_reply: tid=0 2024-02-02T15:39:11.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.650+0000 7f103fd95b80 1 client.28843 shutdown 2024-02-02T15:39:11.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.650+0000 7f103fd95b80 1 -- 172.21.15.1:0/1190281205 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636c9f66010 msgr2=0x5636c9f71bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:11.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.650+0000 7f103fd95b80 1 --2- 172.21.15.1:0/1190281205 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636c9f66010 0x5636c9f71bc0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f1014003ad0 tx=0x7f10140047c0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.650+0000 7f103fd95b80 1 -- 172.21.15.1:0/1190281205 shutdown_connections 2024-02-02T15:39:11.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.650+0000 7f103fd95b80 1 --2- 172.21.15.1:0/1190281205 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5636c9f717e0 0x5636c9f72640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.650+0000 7f103fd95b80 1 --2- 172.21.15.1:0/1190281205 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636c9f66010 0x5636c9f71bc0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.650+0000 7f103fd95b80 1 --2- 172.21.15.1:0/1190281205 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x5636c9f80f50 0x5636c9f83370 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.650+0000 7f103fd95b80 1 --2- 172.21.15.1:0/1190281205 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636c9f66930 0x5636c9f72100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.650+0000 7f103fd95b80 1 -- 172.21.15.1:0/1190281205 >> 172.21.15.1:0/1190281205 conn(0x5636c9ec5db0 msgr2=0x5636c9f61fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:11.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.650+0000 7f1013400700 0 client.28843 ms_handle_reset on v2:172.21.15.142:6836/2870289019 2024-02-02T15:39:11.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.651+0000 7f103fd95b80 1 -- 172.21.15.1:0/1190281205 shutdown_connections 2024-02-02T15:39:11.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.651+0000 7f103fd95b80 1 -- 172.21.15.1:0/1190281205 wait complete. 2024-02-02T15:39:11.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.651+0000 7f103fd95b80 20 client.28843 trim_cache size 0 max 16384 2024-02-02T15:39:11.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.664+0000 7f103fd95b80 1 Processor -- start 2024-02-02T15:39:11.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.664+0000 7f103fd95b80 1 -- start start 2024-02-02T15:39:11.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.665+0000 7f103fd95b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636c9f66010 0x5636c9f7d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:11.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.665+0000 7f103fd95b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5636c9f66930 0x5636c9f7c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:11.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.665+0000 7f103fd95b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636c9f717e0 0x5636c9f6bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:11.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.665+0000 7f103fd95b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636c9d3b6a0 con 0x5636c9f66010 2024-02-02T15:39:11.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.665+0000 7f103fd95b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636c9f63a20 con 0x5636c9f66930 2024-02-02T15:39:11.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.665+0000 7f103fd95b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636c9f70f50 con 0x5636c9f717e0 2024-02-02T15:39:11.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.665+0000 7f1028e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5636c9f66930 0x5636c9f7c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:11.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.665+0000 7f1028e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5636c9f66930 0x5636c9f7c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:32840/0 (socket says 172.21.15.1:32840) 2024-02-02T15:39:11.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.665+0000 7f1028e00700 1 -- 172.21.15.1:0/3401288128 learned_addr learned my addr 172.21.15.1:0/3401288128 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:39:11.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.665+0000 7f1029800700 1 --2- 172.21.15.1:0/3401288128 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636c9f66010 0x5636c9f7d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:11.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.665+0000 7f102a200700 1 --2- 172.21.15.1:0/3401288128 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636c9f717e0 0x5636c9f6bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:11.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.665+0000 7f1028e00700 1 -- 172.21.15.1:0/3401288128 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636c9f717e0 msgr2=0x5636c9f6bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:11.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.665+0000 7f1028e00700 1 --2- 172.21.15.1:0/3401288128 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636c9f717e0 0x5636c9f6bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.665+0000 7f1028e00700 1 -- 172.21.15.1:0/3401288128 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636c9f66010 msgr2=0x5636c9f7d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:11.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.665+0000 7f1028e00700 1 --2- 172.21.15.1:0/3401288128 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636c9f66010 0x5636c9f7d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.665+0000 7f1028e00700 1 -- 172.21.15.1:0/3401288128 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5636c9e7dea0 con 0x5636c9f66930 2024-02-02T15:39:11.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.665+0000 7f1029800700 1 --2- 172.21.15.1:0/3401288128 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636c9f66010 0x5636c9f7d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:39:11.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.666+0000 7f1028e00700 1 --2- 172.21.15.1:0/3401288128 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5636c9f66930 0x5636c9f7c1d0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f100c0059d0 tx=0x7f100c085040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f49c06e28cb41b9 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:11.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.666+0000 7f101b400700 1 -- 172.21.15.1:0/3401288128 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f100c0a2650 con 0x5636c9f66930 2024-02-02T15:39:11.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.666+0000 7f101b400700 1 -- 172.21.15.1:0/3401288128 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f100c09a670 con 0x5636c9f66930 2024-02-02T15:39:11.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.666+0000 7f101b400700 1 -- 172.21.15.1:0/3401288128 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f100c0cd120 con 0x5636c9f66930 2024-02-02T15:39:11.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.666+0000 7f103fd95b80 1 -- 172.21.15.1:0/3401288128 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5636c9f66930 msgr2=0x5636c9f7c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:11.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.666+0000 7f103fd95b80 1 --2- 172.21.15.1:0/3401288128 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5636c9f66930 0x5636c9f7c1d0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f100c0059d0 tx=0x7f100c085040 comp rx=0 tx=0).stop 2024-02-02T15:39:11.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.666+0000 7f103fd95b80 1 -- 172.21.15.1:0/3401288128 shutdown_connections 2024-02-02T15:39:11.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.666+0000 7f103fd95b80 1 --2- 172.21.15.1:0/3401288128 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636c9f717e0 0x5636c9f6bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.666+0000 7f103fd95b80 1 --2- 172.21.15.1:0/3401288128 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636c9f66010 0x5636c9f7d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.666+0000 7f103fd95b80 1 --2- 172.21.15.1:0/3401288128 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5636c9f66930 0x5636c9f7c1d0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.666+0000 7f103fd95b80 1 -- 172.21.15.1:0/3401288128 >> 172.21.15.1:0/3401288128 conn(0x5636c9ec5db0 msgr2=0x5636c9f6b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:11.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.667+0000 7f103fd95b80 1 -- 172.21.15.1:0/3401288128 shutdown_connections 2024-02-02T15:39:11.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.667+0000 7f103fd95b80 1 -- 172.21.15.1:0/3401288128 wait complete. 2024-02-02T15:39:11.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.667+0000 7f103fd95b80 1 -- start start 2024-02-02T15:39:11.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.667+0000 7f103fd95b80 1 Processor -- start 2024-02-02T15:39:11.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.668+0000 7f103fd95b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5636c9f66010 0x5636c9f73330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:11.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.668+0000 7f103fd95b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636c9f66930 0x5636c9f73870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:11.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.668+0000 7f103fd95b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636c9f717e0 0x5636c9f73dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:11.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.668+0000 7f103fd95b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636c9fd1c20 con 0x5636c9f66930 2024-02-02T15:39:11.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.668+0000 7f103fd95b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636c9fd1d90 con 0x5636c9f66010 2024-02-02T15:39:11.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.668+0000 7f103fd95b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636c9fd1f50 con 0x5636c9f717e0 2024-02-02T15:39:11.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.668+0000 7f103fd95b80 10 client.0 _ll_register_callbacks cb 0x5636c9e55290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:39:11.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.668+0000 7f1029800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5636c9f66010 0x5636c9f73330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:11.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.668+0000 7f1029800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5636c9f66010 0x5636c9f73330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:32850/0 (socket says 172.21.15.1:32850) 2024-02-02T15:39:11.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.668+0000 7f1029800700 1 -- 172.21.15.1:0/3008120337 learned_addr learned my addr 172.21.15.1:0/3008120337 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:39:11.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.668+0000 7f1028e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636c9f66930 0x5636c9f73870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:11.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.668+0000 7f102a200700 1 --2- 172.21.15.1:0/3008120337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636c9f717e0 0x5636c9f73dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:11.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.669+0000 7f1029800700 1 -- 172.21.15.1:0/3008120337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636c9f717e0 msgr2=0x5636c9f73dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:11.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.669+0000 7f1029800700 1 --2- 172.21.15.1:0/3008120337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636c9f717e0 0x5636c9f73dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.669+0000 7f1029800700 1 -- 172.21.15.1:0/3008120337 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636c9f66930 msgr2=0x5636c9f73870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:11.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.669+0000 7f1029800700 1 --2- 172.21.15.1:0/3008120337 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636c9f66930 0x5636c9f73870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.669+0000 7f1029800700 1 -- 172.21.15.1:0/3008120337 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5636c9d393b0 con 0x5636c9f66010 2024-02-02T15:39:11.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.669+0000 7f1029800700 1 --2- 172.21.15.1:0/3008120337 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5636c9f66010 0x5636c9f73330 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f1014009f10 tx=0x7f1014027e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8319d5c69ddeb0e6 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:11.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.669+0000 7f1013400700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:39:11.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.669+0000 7f1013400700 1 -- 172.21.15.1:0/3008120337 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1014003e90 con 0x5636c9f66010 2024-02-02T15:39:11.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.669+0000 7f103fd95b80 1 -- 172.21.15.1:0/3008120337 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f100c021e90 con 0x5636c9f66010 2024-02-02T15:39:11.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.669+0000 7f1013400700 1 -- 172.21.15.1:0/3008120337 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f101401fcb0 con 0x5636c9f66010 2024-02-02T15:39:11.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.670+0000 7f1013400700 1 -- 172.21.15.1:0/3008120337 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f101401fe70 con 0x5636c9f66010 2024-02-02T15:39:11.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.670+0000 7f103fd95b80 10 client.28847 fetch_fsmap learned FSMap version 52 2024-02-02T15:39:11.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.670+0000 7f103fd95b80 1 -- 172.21.15.1:0/3008120337 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x5636c9d3dcd0 con 0x5636c9f66010 2024-02-02T15:39:11.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.670+0000 7f1013400700 1 -- 172.21.15.1:0/3008120337 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f100c021e90 con 0x5636c9f66010 2024-02-02T15:39:11.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.670+0000 7f1013400700 1 -- 172.21.15.1:0/3008120337 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 52) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f1014028440 con 0x5636c9f66010 2024-02-02T15:39:11.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.670+0000 7f103fd95b80 10 client.28847 fetch_fsmap finished waiting for FSMap version 52 2024-02-02T15:39:11.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.670+0000 7f103fd95b80 10 client.28847 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-02-02T15:39:11.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.670+0000 7f103fd95b80 20 client.28847 populate_metadata read hostname 'smithi001' 2024-02-02T15:39:11.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.670+0000 7f103fd95b80 1 --2- 172.21.15.1:0/3008120337 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x5636c9fd3610 0x5636c9fd5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:11.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.670+0000 7f103fd95b80 4 client.28847 mds_command: new command op to 24514 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:39:11.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.670+0000 7f103fd95b80 1 -- 172.21.15.1:0/3008120337 --> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5636c9e76f50 con 0x5636c9fd3610 2024-02-02T15:39:11.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.670+0000 7f1028e00700 1 --2- 172.21.15.1:0/3008120337 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x5636c9fd3610 0x5636c9fd5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:11.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.671+0000 7f1028e00700 1 --2- 172.21.15.1:0/3008120337 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x5636c9fd3610 0x5636c9fd5a30 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=220ccbc063f6cbb9 server_cookie=8afa584f47904def in_seq=0 out_seq=0 2024-02-02T15:39:11.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.671+0000 7f1013400700 10 client.28847 ms_handle_connect on v2:172.21.15.142:6836/2870289019 2024-02-02T15:39:11.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.671+0000 7f1013400700 1 -- 172.21.15.1:0/3008120337 <== mds.3 v2:172.21.15.142:6836/2870289019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5636c9e76f50 con 0x5636c9fd3610 2024-02-02T15:39:11.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.671+0000 7f1013400700 10 client.28847 handle_command_reply: tid=0 2024-02-02T15:39:11.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.671+0000 7f103fd95b80 1 client.28847 shutdown 2024-02-02T15:39:11.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.671+0000 7f103fd95b80 1 -- 172.21.15.1:0/3008120337 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5636c9f66010 msgr2=0x5636c9f73330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:11.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.671+0000 7f103fd95b80 1 --2- 172.21.15.1:0/3008120337 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5636c9f66010 0x5636c9f73330 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f1014009f10 tx=0x7f1014027e10 comp rx=0 tx=0).stop 2024-02-02T15:39:11.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.671+0000 7f103fd95b80 1 -- 172.21.15.1:0/3008120337 shutdown_connections 2024-02-02T15:39:11.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.671+0000 7f103fd95b80 1 --2- 172.21.15.1:0/3008120337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5636c9f717e0 0x5636c9f73dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.671+0000 7f103fd95b80 1 --2- 172.21.15.1:0/3008120337 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5636c9f66010 0x5636c9f73330 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.671+0000 7f103fd95b80 1 --2- 172.21.15.1:0/3008120337 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x5636c9fd3610 0x5636c9fd5a30 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.672+0000 7f103fd95b80 1 --2- 172.21.15.1:0/3008120337 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5636c9f66930 0x5636c9f73870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.672+0000 7f1013400700 0 client.28847 ms_handle_reset on v2:172.21.15.142:6836/2870289019 2024-02-02T15:39:11.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.672+0000 7f103fd95b80 1 -- 172.21.15.1:0/3008120337 >> 172.21.15.1:0/3008120337 conn(0x5636c9ec5db0 msgr2=0x5636c9f7d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:11.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.672+0000 7f103fd95b80 1 -- 172.21.15.1:0/3008120337 shutdown_connections 2024-02-02T15:39:11.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.672+0000 7f103fd95b80 1 -- 172.21.15.1:0/3008120337 wait complete. 2024-02-02T15:39:11.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.672+0000 7f103fd95b80 20 client.28847 trim_cache size 0 max 16384 2024-02-02T15:39:11.676 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:39:11.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.672+0000 7f102c200700 1 -- 172.21.15.1:0/424944157 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f10040a0690 msgr2=0x7f10040a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:11.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.672+0000 7f102c200700 1 --2- 172.21.15.1:0/424944157 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f10040a0690 0x7f10040a2b50 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1014011490 tx=0x7f1014009260 comp rx=0 tx=0).stop 2024-02-02T15:39:11.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.672+0000 7f102c200700 1 -- 172.21.15.1:0/424944157 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1024102950 msgr2=0x7f10241001c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:11.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.672+0000 7f102c200700 1 --2- 172.21.15.1:0/424944157 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1024102950 0x7f10241001c0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f100c000c00 tx=0x7f100c0047b0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.673+0000 7f102c200700 1 -- 172.21.15.1:0/424944157 shutdown_connections 2024-02-02T15:39:11.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.673+0000 7f102c200700 1 --2- 172.21.15.1:0/424944157 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f10040a0690 0x7f10040a2b50 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.673+0000 7f102c200700 1 --2- 172.21.15.1:0/424944157 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f102410b400 0x7f10241a5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.673+0000 7f102c200700 1 --2- 172.21.15.1:0/424944157 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1024102030 0x7f10240ffc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.673+0000 7f102c200700 1 --2- 172.21.15.1:0/424944157 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1024102950 0x7f10241001c0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.673+0000 7f102c200700 1 -- 172.21.15.1:0/424944157 >> 172.21.15.1:0/424944157 conn(0x7f1024076d70 msgr2=0x7f10240fcb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:11.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.673+0000 7f102c200700 1 -- 172.21.15.1:0/424944157 shutdown_connections 2024-02-02T15:39:11.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.673+0000 7f102c200700 1 -- 172.21.15.1:0/424944157 wait complete. 2024-02-02T15:39:11.689 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:39:11 smithi001 ceph-mon[35470]: pgmap v379: 129 pgs: 129 active+clean; 555 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 186 op/s 2024-02-02T15:39:11.689 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:39:11 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/2721152927' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:39:11.691 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-02T15:39:11.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:39:11 smithi071 ceph-mon[44707]: pgmap v379: 129 pgs: 129 active+clean; 555 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 186 op/s 2024-02-02T15:39:11.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:39:11 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/2721152927' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:39:11.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:39:11 smithi142 ceph-mon[45728]: pgmap v379: 129 pgs: 129 active+clean; 555 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.1 MiB/s wr, 186 op/s 2024-02-02T15:39:11.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:39:11 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/2721152927' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:39:11.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.942+0000 7ff27a800700 1 -- 172.21.15.1:0/2997079101 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff274100fa0 msgr2=0x7ff274101380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:11.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.942+0000 7ff27a800700 1 --2- 172.21.15.1:0/2997079101 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff274100fa0 0x7ff274101380 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7ff264009990 tx=0x7ff264009ca0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.942+0000 7ff27a800700 1 -- 172.21.15.1:0/2997079101 shutdown_connections 2024-02-02T15:39:11.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.942+0000 7ff27a800700 1 --2- 172.21.15.1:0/2997079101 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff2741060c0 0x7ff27410dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.942+0000 7ff27a800700 1 --2- 172.21.15.1:0/2997079101 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff274100fa0 0x7ff274101380 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.942+0000 7ff27a800700 1 --2- 172.21.15.1:0/2997079101 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff274101950 0x7ff274105990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.942+0000 7ff27a800700 1 -- 172.21.15.1:0/2997079101 >> 172.21.15.1:0/2997079101 conn(0x7ff2740fc800 msgr2=0x7ff2740fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:11.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.942+0000 7ff27a800700 1 -- 172.21.15.1:0/2997079101 shutdown_connections 2024-02-02T15:39:11.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.942+0000 7ff27a800700 1 -- 172.21.15.1:0/2997079101 wait complete. 2024-02-02T15:39:11.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.943+0000 7ff27a800700 1 Processor -- start 2024-02-02T15:39:11.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.943+0000 7ff27a800700 1 -- start start 2024-02-02T15:39:11.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.943+0000 7ff27a800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff274100fa0 0x7ff274199010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:11.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.943+0000 7ff27a800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2741060c0 0x7ff274199550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:11.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.944+0000 7ff27a800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff27419d9b0 0x7ff27419de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:11.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.944+0000 7ff27a800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff27419e430 con 0x7ff274100fa0 2024-02-02T15:39:11.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.944+0000 7ff27a800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff27419e5a0 con 0x7ff27419d9b0 2024-02-02T15:39:11.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.944+0000 7ff27a800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff27419e760 con 0x7ff2741060c0 2024-02-02T15:39:11.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.944+0000 7ff273400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff274100fa0 0x7ff274199010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:11.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.944+0000 7ff272a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2741060c0 0x7ff274199550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:11.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.944+0000 7ff273e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff27419d9b0 0x7ff27419de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:11.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.944+0000 7ff272a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2741060c0 0x7ff274199550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:54004/0 (socket says 172.21.15.1:54004) 2024-02-02T15:39:11.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.944+0000 7ff272a00700 1 -- 172.21.15.1:0/2671243939 learned_addr learned my addr 172.21.15.1:0/2671243939 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:39:11.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.944+0000 7ff272a00700 1 -- 172.21.15.1:0/2671243939 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff27419d9b0 msgr2=0x7ff27419de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:11.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.944+0000 7ff272a00700 1 --2- 172.21.15.1:0/2671243939 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff27419d9b0 0x7ff27419de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.944+0000 7ff272a00700 1 -- 172.21.15.1:0/2671243939 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff274100fa0 msgr2=0x7ff274199010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:11.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.944+0000 7ff272a00700 1 --2- 172.21.15.1:0/2671243939 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff274100fa0 0x7ff274199010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.944+0000 7ff272a00700 1 -- 172.21.15.1:0/2671243939 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2640095f0 con 0x7ff2741060c0 2024-02-02T15:39:11.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.944+0000 7ff272a00700 1 --2- 172.21.15.1:0/2671243939 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2741060c0 0x7ff274199550 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7ff25c00d950 tx=0x7ff25c00dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:11.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.945+0000 7ff26be00700 1 -- 172.21.15.1:0/2671243939 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25c004220 con 0x7ff2741060c0 2024-02-02T15:39:11.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.945+0000 7ff27a800700 1 -- 172.21.15.1:0/2671243939 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2741a5cc0 con 0x7ff2741060c0 2024-02-02T15:39:11.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.945+0000 7ff26be00700 1 -- 172.21.15.1:0/2671243939 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff25c007620 con 0x7ff2741060c0 2024-02-02T15:39:11.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.945+0000 7ff26be00700 1 -- 172.21.15.1:0/2671243939 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25c0056c0 con 0x7ff2741060c0 2024-02-02T15:39:11.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.945+0000 7ff27a800700 1 -- 172.21.15.1:0/2671243939 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2741a61a0 con 0x7ff2741060c0 2024-02-02T15:39:11.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.948+0000 7ff26be00700 1 -- 172.21.15.1:0/2671243939 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7ff25c004380 con 0x7ff2741060c0 2024-02-02T15:39:11.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.949+0000 7ff26be00700 1 --2- 172.21.15.1:0/2671243939 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff2540a0690 0x7ff2540a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:11.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.949+0000 7ff26be00700 1 -- 172.21.15.1:0/2671243939 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9541+0+0 (secure 0 0 0) 0x7ff25c0cbef0 con 0x7ff2741060c0 2024-02-02T15:39:11.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.949+0000 7ff273400700 1 --2- 172.21.15.1:0/2671243939 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff2540a0690 0x7ff2540a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:11.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.950+0000 7ff28e3a4b80 1 Processor -- start 2024-02-02T15:39:11.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.950+0000 7ff28e3a4b80 1 -- start start 2024-02-02T15:39:11.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.950+0000 7ff273400700 1 --2- 172.21.15.1:0/2671243939 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff2540a0690 0x7ff2540a2b50 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff26400b5c0 tx=0x7ff264005fb0 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:11.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.950+0000 7ff28e3a4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b795ac0010 0x55b795ac03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:11.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.950+0000 7ff28e3a4b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b795ac0930 0x55b795acb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:11.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.950+0000 7ff28e3a4b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b795acb7e0 0x55b795acdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:11.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.950+0000 7ff28e3a4b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b795a344c0 con 0x55b795acb7e0 2024-02-02T15:39:11.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.950+0000 7ff28e3a4b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b795a18aa0 con 0x55b795ac0930 2024-02-02T15:39:11.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.950+0000 7ff28e3a4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b795a16460 con 0x55b795ac0010 2024-02-02T15:39:11.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.951+0000 7ff273400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b795ac0010 0x55b795ac03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:11.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.951+0000 7ff273400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b795ac0010 0x55b795ac03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:54016/0 (socket says 172.21.15.1:54016) 2024-02-02T15:39:11.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.951+0000 7ff273400700 1 -- 172.21.15.1:0/787428831 learned_addr learned my addr 172.21.15.1:0/787428831 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:39:11.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.951+0000 7ff273e00700 1 --2- 172.21.15.1:0/787428831 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b795acb7e0 0x55b795acdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:11.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.951+0000 7ff272a00700 1 --2- 172.21.15.1:0/787428831 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b795ac0930 0x55b795acb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:11.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.951+0000 7ff273400700 1 -- 172.21.15.1:0/787428831 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b795ac0930 msgr2=0x55b795acb2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:11.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.951+0000 7ff273400700 1 --2- 172.21.15.1:0/787428831 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b795ac0930 0x55b795acb2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.951+0000 7ff273400700 1 -- 172.21.15.1:0/787428831 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b795acb7e0 msgr2=0x55b795acdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:11.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.951+0000 7ff273400700 1 --2- 172.21.15.1:0/787428831 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b795acb7e0 0x55b795acdbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.951+0000 7ff273400700 1 -- 172.21.15.1:0/787428831 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b7958933b0 con 0x55b795ac0010 2024-02-02T15:39:11.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.951+0000 7ff273e00700 1 --2- 172.21.15.1:0/787428831 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b795acb7e0 0x55b795acdbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:39:11.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.952+0000 7ff273400700 1 --2- 172.21.15.1:0/787428831 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b795ac0010 0x55b795ac03f0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7ff26400b960 tx=0x7ff264020000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=466b098920d32b6e server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:11.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.952+0000 7ff268c00700 1 -- 172.21.15.1:0/787428831 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff26402a070 con 0x55b795ac0010 2024-02-02T15:39:11.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.952+0000 7ff268c00700 1 -- 172.21.15.1:0/787428831 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff26401e040 con 0x55b795ac0010 2024-02-02T15:39:11.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.952+0000 7ff268c00700 1 -- 172.21.15.1:0/787428831 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2640239d0 con 0x55b795ac0010 2024-02-02T15:39:11.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.952+0000 7ff28e3a4b80 1 -- 172.21.15.1:0/787428831 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b795ac0010 msgr2=0x55b795ac03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:11.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.953+0000 7ff28e3a4b80 1 --2- 172.21.15.1:0/787428831 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b795ac0010 0x55b795ac03f0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7ff26400b960 tx=0x7ff264020000 comp rx=0 tx=0).stop 2024-02-02T15:39:11.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.953+0000 7ff28e3a4b80 1 -- 172.21.15.1:0/787428831 shutdown_connections 2024-02-02T15:39:11.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.953+0000 7ff28e3a4b80 1 --2- 172.21.15.1:0/787428831 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b795acb7e0 0x55b795acdbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.953+0000 7ff28e3a4b80 1 --2- 172.21.15.1:0/787428831 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b795ac0010 0x55b795ac03f0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.953+0000 7ff28e3a4b80 1 --2- 172.21.15.1:0/787428831 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b795ac0930 0x55b795acb2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.953+0000 7ff28e3a4b80 1 -- 172.21.15.1:0/787428831 >> 172.21.15.1:0/787428831 conn(0x55b795a1fdb0 msgr2=0x55b795abc3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:11.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.953+0000 7ff28e3a4b80 1 -- 172.21.15.1:0/787428831 shutdown_connections 2024-02-02T15:39:11.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.953+0000 7ff28e3a4b80 1 -- 172.21.15.1:0/787428831 wait complete. 2024-02-02T15:39:11.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.953+0000 7ff28e3a4b80 1 -- start start 2024-02-02T15:39:11.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.954+0000 7ff28e3a4b80 1 Processor -- start 2024-02-02T15:39:11.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.954+0000 7ff28e3a4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b795ac0010 0x55b795acb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:11.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.954+0000 7ff28e3a4b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b795ac0930 0x55b795ac8490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:11.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.954+0000 7ff28e3a4b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b795acb7e0 0x55b795ac8b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:11.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.954+0000 7ff28e3a4b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b795a11d50 con 0x55b795ac0930 2024-02-02T15:39:11.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.954+0000 7ff28e3a4b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7959add80 con 0x55b795acb7e0 2024-02-02T15:39:11.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.954+0000 7ff28e3a4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7959ada20 con 0x55b795ac0010 2024-02-02T15:39:11.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.954+0000 7ff28e3a4b80 10 client.0 _ll_register_callbacks cb 0x55b7959af290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:39:11.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.954+0000 7ff273400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b795ac0010 0x55b795acb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:11.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.954+0000 7ff273e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b795acb7e0 0x55b795ac8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:11.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.954+0000 7ff273400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b795ac0010 0x55b795acb1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:54022/0 (socket says 172.21.15.1:54022) 2024-02-02T15:39:11.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.954+0000 7ff272a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b795ac0930 0x55b795ac8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:11.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.954+0000 7ff273400700 1 -- 172.21.15.1:0/1665130390 learned_addr learned my addr 172.21.15.1:0/1665130390 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:39:11.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.955+0000 7ff273e00700 1 -- 172.21.15.1:0/1665130390 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b795ac0010 msgr2=0x55b795acb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:11.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.955+0000 7ff273e00700 1 --2- 172.21.15.1:0/1665130390 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b795ac0010 0x55b795acb1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.955+0000 7ff273e00700 1 -- 172.21.15.1:0/1665130390 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b795ac0930 msgr2=0x55b795ac8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:11.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.955+0000 7ff273e00700 1 --2- 172.21.15.1:0/1665130390 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b795ac0930 0x55b795ac8490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.955+0000 7ff273e00700 1 -- 172.21.15.1:0/1665130390 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff26c007d90 con 0x55b795acb7e0 2024-02-02T15:39:11.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.955+0000 7ff273400700 1 --2- 172.21.15.1:0/1665130390 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b795ac0010 0x55b795acb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:39:11.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.955+0000 7ff272a00700 1 --2- 172.21.15.1:0/1665130390 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b795ac0930 0x55b795ac8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:39:11.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.955+0000 7ff273e00700 1 --2- 172.21.15.1:0/1665130390 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b795acb7e0 0x55b795ac8b10 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7ff26c010f10 tx=0x7ff26c0125a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ba1331ff9c4926e server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:11.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.955+0000 7ff261600700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:39:11.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.955+0000 7ff261600700 1 -- 172.21.15.1:0/1665130390 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff26c012dd0 con 0x55b795acb7e0 2024-02-02T15:39:11.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.955+0000 7ff28e3a4b80 1 -- 172.21.15.1:0/1665130390 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b795ab40f0 con 0x55b795acb7e0 2024-02-02T15:39:11.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.955+0000 7ff261600700 1 -- 172.21.15.1:0/1665130390 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9541+0+0 (secure 0 0 0) 0x7ff26c015440 con 0x55b795acb7e0 2024-02-02T15:39:11.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.956+0000 7ff261600700 1 -- 172.21.15.1:0/1665130390 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff26c015600 con 0x55b795acb7e0 2024-02-02T15:39:11.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.956+0000 7ff28e3a4b80 10 client.28859 fetch_fsmap learned FSMap version 52 2024-02-02T15:39:11.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.956+0000 7ff28e3a4b80 1 -- 172.21.15.1:0/1665130390 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x7ff27410e4b0 con 0x55b795acb7e0 2024-02-02T15:39:11.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.956+0000 7ff261600700 1 -- 172.21.15.1:0/1665130390 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b795ab40f0 con 0x55b795acb7e0 2024-02-02T15:39:11.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.956+0000 7ff261600700 1 -- 172.21.15.1:0/1665130390 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 52) v1 ==== 7486+0+0 (secure 0 0 0) 0x7ff26c01e7c0 con 0x55b795acb7e0 2024-02-02T15:39:11.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.956+0000 7ff28e3a4b80 10 client.28859 fetch_fsmap finished waiting for FSMap version 52 2024-02-02T15:39:11.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.956+0000 7ff28e3a4b80 10 client.28859 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-02-02T15:39:11.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.956+0000 7ff28e3a4b80 20 client.28859 populate_metadata read hostname 'smithi001' 2024-02-02T15:39:11.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.956+0000 7ff28e3a4b80 1 --2- 172.21.15.1:0/1665130390 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55b795adb1e0 0x55b795add600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:11.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.956+0000 7ff28e3a4b80 4 client.28859 mds_command: new command op to 24518 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:39:11.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.956+0000 7ff28e3a4b80 1 -- 172.21.15.1:0/1665130390 --> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b795a0add0 con 0x55b795adb1e0 2024-02-02T15:39:11.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.956+0000 7ff273400700 1 --2- 172.21.15.1:0/1665130390 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55b795adb1e0 0x55b795add600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:11.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.957+0000 7ff261600700 10 client.28859 ms_handle_connect on v2:172.21.15.71:6836/628016199 2024-02-02T15:39:11.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.957+0000 7ff273400700 1 --2- 172.21.15.1:0/1665130390 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55b795adb1e0 0x55b795add600 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1635c92ff180104f server_cookie=3ec7bd035b763135 in_seq=0 out_seq=0 2024-02-02T15:39:11.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.958+0000 7ff261600700 1 -- 172.21.15.1:0/1665130390 <== mds.2 v2:172.21.15.71:6836/628016199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55b795a0add0 con 0x55b795adb1e0 2024-02-02T15:39:11.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.958+0000 7ff261600700 10 client.28859 handle_command_reply: tid=0 2024-02-02T15:39:11.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.958+0000 7ff28e3a4b80 1 client.28859 shutdown 2024-02-02T15:39:11.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.958+0000 7ff28e3a4b80 1 -- 172.21.15.1:0/1665130390 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b795acb7e0 msgr2=0x55b795ac8b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:11.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.958+0000 7ff28e3a4b80 1 --2- 172.21.15.1:0/1665130390 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b795acb7e0 0x55b795ac8b10 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7ff26c010f10 tx=0x7ff26c0125a0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.958+0000 7ff28e3a4b80 1 -- 172.21.15.1:0/1665130390 shutdown_connections 2024-02-02T15:39:11.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.958+0000 7ff28e3a4b80 1 --2- 172.21.15.1:0/1665130390 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b795acb7e0 0x55b795ac8b10 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.958+0000 7ff28e3a4b80 1 --2- 172.21.15.1:0/1665130390 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55b795adb1e0 0x55b795add600 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.958+0000 7ff28e3a4b80 1 --2- 172.21.15.1:0/1665130390 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b795ac0010 0x55b795acb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.958+0000 7ff28e3a4b80 1 --2- 172.21.15.1:0/1665130390 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b795ac0930 0x55b795ac8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.958+0000 7ff28e3a4b80 1 -- 172.21.15.1:0/1665130390 >> 172.21.15.1:0/1665130390 conn(0x55b795a1fdb0 msgr2=0x55b795abbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:11.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.958+0000 7ff261600700 0 client.28859 ms_handle_reset on v2:172.21.15.71:6836/628016199 2024-02-02T15:39:11.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.958+0000 7ff28e3a4b80 1 -- 172.21.15.1:0/1665130390 shutdown_connections 2024-02-02T15:39:11.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.958+0000 7ff28e3a4b80 1 -- 172.21.15.1:0/1665130390 wait complete. 2024-02-02T15:39:11.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.959+0000 7ff28e3a4b80 20 client.28859 trim_cache size 0 max 16384 2024-02-02T15:39:11.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.972+0000 7ff28e3a4b80 1 Processor -- start 2024-02-02T15:39:11.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.972+0000 7ff28e3a4b80 1 -- start start 2024-02-02T15:39:11.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.973+0000 7ff28e3a4b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b795ac0010 0x55b795ad22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:11.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.973+0000 7ff28e3a4b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b795ac0930 0x55b795acad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:11.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.973+0000 7ff28e3a4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b795acb7e0 0x55b795ac37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:11.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.973+0000 7ff28e3a4b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7958956a0 con 0x55b795ac0010 2024-02-02T15:39:11.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.973+0000 7ff28e3a4b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b795ac69f0 con 0x55b795ac0930 2024-02-02T15:39:11.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.973+0000 7ff28e3a4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b795ac42e0 con 0x55b795acb7e0 2024-02-02T15:39:11.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.973+0000 7ff272a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b795ac0930 0x55b795acad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:11.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.973+0000 7ff272a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b795ac0930 0x55b795acad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:32876/0 (socket says 172.21.15.1:32876) 2024-02-02T15:39:11.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.973+0000 7ff272a00700 1 -- 172.21.15.1:0/2216725909 learned_addr learned my addr 172.21.15.1:0/2216725909 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:39:11.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.973+0000 7ff273400700 1 --2- 172.21.15.1:0/2216725909 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b795ac0010 0x55b795ad22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:11.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.973+0000 7ff273e00700 1 --2- 172.21.15.1:0/2216725909 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b795acb7e0 0x55b795ac37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:11.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.973+0000 7ff272a00700 1 -- 172.21.15.1:0/2216725909 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b795acb7e0 msgr2=0x55b795ac37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:11.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.973+0000 7ff272a00700 1 --2- 172.21.15.1:0/2216725909 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b795acb7e0 0x55b795ac37a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.973+0000 7ff272a00700 1 -- 172.21.15.1:0/2216725909 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b795ac0010 msgr2=0x55b795ad22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:11.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.973+0000 7ff272a00700 1 --2- 172.21.15.1:0/2216725909 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b795ac0010 0x55b795ad22b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.973+0000 7ff272a00700 1 -- 172.21.15.1:0/2216725909 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b79583fb60 con 0x55b795ac0930 2024-02-02T15:39:11.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.974+0000 7ff272a00700 1 --2- 172.21.15.1:0/2216725909 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b795ac0930 0x55b795acad40 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7ff25c005cd0 tx=0x7ff25c0996b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3771d5934732f34f server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:11.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.974+0000 7ff269600700 1 -- 172.21.15.1:0/2216725909 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25c09cde0 con 0x55b795ac0930 2024-02-02T15:39:11.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.974+0000 7ff269600700 1 -- 172.21.15.1:0/2216725909 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff25c0a1020 con 0x55b795ac0930 2024-02-02T15:39:11.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.974+0000 7ff269600700 1 -- 172.21.15.1:0/2216725909 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25c098700 con 0x55b795ac0930 2024-02-02T15:39:11.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.974+0000 7ff28e3a4b80 1 -- 172.21.15.1:0/2216725909 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b795ac0930 msgr2=0x55b795acad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:11.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.974+0000 7ff28e3a4b80 1 --2- 172.21.15.1:0/2216725909 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b795ac0930 0x55b795acad40 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7ff25c005cd0 tx=0x7ff25c0996b0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.975+0000 7ff28e3a4b80 1 -- 172.21.15.1:0/2216725909 shutdown_connections 2024-02-02T15:39:11.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.975+0000 7ff28e3a4b80 1 --2- 172.21.15.1:0/2216725909 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b795acb7e0 0x55b795ac37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.975+0000 7ff28e3a4b80 1 --2- 172.21.15.1:0/2216725909 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b795ac0010 0x55b795ad22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.975+0000 7ff28e3a4b80 1 --2- 172.21.15.1:0/2216725909 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b795ac0930 0x55b795acad40 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.975+0000 7ff28e3a4b80 1 -- 172.21.15.1:0/2216725909 >> 172.21.15.1:0/2216725909 conn(0x55b795a1fdb0 msgr2=0x55b795ad6880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:11.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.975+0000 7ff28e3a4b80 1 -- 172.21.15.1:0/2216725909 shutdown_connections 2024-02-02T15:39:11.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.975+0000 7ff28e3a4b80 1 -- 172.21.15.1:0/2216725909 wait complete. 2024-02-02T15:39:11.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.975+0000 7ff28e3a4b80 1 -- start start 2024-02-02T15:39:11.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.975+0000 7ff28e3a4b80 1 Processor -- start 2024-02-02T15:39:11.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.976+0000 7ff28e3a4b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b795ac0010 0x55b795b20c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:11.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.976+0000 7ff28e3a4b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b795ac0930 0x55b795b21190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:11.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.976+0000 7ff28e3a4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b795acb7e0 0x55b795b2bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:11.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.976+0000 7ff28e3a4b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b795b21920 con 0x55b795ac0930 2024-02-02T15:39:11.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.976+0000 7ff28e3a4b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b795b21a90 con 0x55b795ac0010 2024-02-02T15:39:11.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.976+0000 7ff28e3a4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b795b2c210 con 0x55b795acb7e0 2024-02-02T15:39:11.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.976+0000 7ff28e3a4b80 10 client.0 _ll_register_callbacks cb 0x55b7959af290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:39:11.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.976+0000 7ff272a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b795ac0930 0x55b795b21190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:11.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.976+0000 7ff273e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b795acb7e0 0x55b795b2bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:11.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.976+0000 7ff273400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b795ac0010 0x55b795b20c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:11.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.976+0000 7ff273e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b795acb7e0 0x55b795b2bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:54028/0 (socket says 172.21.15.1:54028) 2024-02-02T15:39:11.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.976+0000 7ff273400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b795ac0010 0x55b795b20c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:32886/0 (socket says 172.21.15.1:32886) 2024-02-02T15:39:11.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.976+0000 7ff272a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b795ac0930 0x55b795b21190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50738/0 (socket says 172.21.15.1:50738) 2024-02-02T15:39:11.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.976+0000 7ff273e00700 1 -- 172.21.15.1:0/760086962 learned_addr learned my addr 172.21.15.1:0/760086962 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:39:11.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.977+0000 7ff273e00700 1 -- 172.21.15.1:0/760086962 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b795ac0010 msgr2=0x55b795b20c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:11.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.977+0000 7ff273e00700 1 --2- 172.21.15.1:0/760086962 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b795ac0010 0x55b795b20c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.977+0000 7ff273e00700 1 -- 172.21.15.1:0/760086962 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b795ac0930 msgr2=0x55b795b21190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:11.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.977+0000 7ff273e00700 1 --2- 172.21.15.1:0/760086962 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b795ac0930 0x55b795b21190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.977+0000 7ff273e00700 1 -- 172.21.15.1:0/760086962 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2741a61a0 con 0x55b795acb7e0 2024-02-02T15:39:11.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.977+0000 7ff273e00700 1 --2- 172.21.15.1:0/760086962 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b795acb7e0 0x55b795b2bcd0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7ff26c010f10 tx=0x7ff26c01eec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4fe629d7e02e777 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:11.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.977+0000 7ff261600700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:39:11.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.977+0000 7ff261600700 1 -- 172.21.15.1:0/760086962 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff26c003e10 con 0x55b795acb7e0 2024-02-02T15:39:11.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.977+0000 7ff261600700 1 -- 172.21.15.1:0/760086962 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9541+0+0 (secure 0 0 0) 0x7ff26c01e3b0 con 0x55b795acb7e0 2024-02-02T15:39:11.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.977+0000 7ff28e3a4b80 1 -- 172.21.15.1:0/760086962 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff26401e040 con 0x55b795acb7e0 2024-02-02T15:39:11.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.978+0000 7ff261600700 1 -- 172.21.15.1:0/760086962 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff26c01e570 con 0x55b795acb7e0 2024-02-02T15:39:11.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.978+0000 7ff28e3a4b80 10 client.28867 fetch_fsmap learned FSMap version 52 2024-02-02T15:39:11.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.978+0000 7ff28e3a4b80 1 -- 172.21.15.1:0/760086962 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x55b7959d7ea0 con 0x55b795acb7e0 2024-02-02T15:39:11.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.978+0000 7ff261600700 1 -- 172.21.15.1:0/760086962 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff26401e040 con 0x55b795acb7e0 2024-02-02T15:39:11.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.978+0000 7ff261600700 1 -- 172.21.15.1:0/760086962 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 52) v1 ==== 7486+0+0 (secure 0 0 0) 0x7ff26c0057d0 con 0x55b795acb7e0 2024-02-02T15:39:11.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.978+0000 7ff28e3a4b80 10 client.28867 fetch_fsmap finished waiting for FSMap version 52 2024-02-02T15:39:11.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.978+0000 7ff28e3a4b80 10 client.28867 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-02-02T15:39:11.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.978+0000 7ff28e3a4b80 20 client.28867 populate_metadata read hostname 'smithi001' 2024-02-02T15:39:11.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.978+0000 7ff28e3a4b80 1 --2- 172.21.15.1:0/760086962 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55b795b2d870 0x55b795b2fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:11.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.978+0000 7ff28e3a4b80 4 client.28867 mds_command: new command op to 24518 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:39:11.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.978+0000 7ff28e3a4b80 1 -- 172.21.15.1:0/760086962 --> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b7959d0f50 con 0x55b795b2d870 2024-02-02T15:39:11.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.978+0000 7ff273400700 1 --2- 172.21.15.1:0/760086962 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55b795b2d870 0x55b795b2fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:11.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.979+0000 7ff273400700 1 --2- 172.21.15.1:0/760086962 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55b795b2d870 0x55b795b2fc90 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3a93c3880bc5dcb2 server_cookie=e7f5a2e43f5f81f9 in_seq=0 out_seq=0 2024-02-02T15:39:11.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.979+0000 7ff261600700 10 client.28867 ms_handle_connect on v2:172.21.15.71:6836/628016199 2024-02-02T15:39:11.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.979+0000 7ff261600700 1 -- 172.21.15.1:0/760086962 <== mds.2 v2:172.21.15.71:6836/628016199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b7959d0f50 con 0x55b795b2d870 2024-02-02T15:39:11.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.979+0000 7ff261600700 10 client.28867 handle_command_reply: tid=0 2024-02-02T15:39:11.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.979+0000 7ff28e3a4b80 1 client.28867 shutdown 2024-02-02T15:39:11.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.979+0000 7ff28e3a4b80 1 -- 172.21.15.1:0/760086962 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b795acb7e0 msgr2=0x55b795b2bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:11.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.979+0000 7ff28e3a4b80 1 --2- 172.21.15.1:0/760086962 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b795acb7e0 0x55b795b2bcd0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7ff26c010f10 tx=0x7ff26c01eec0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.979+0000 7ff28e3a4b80 1 -- 172.21.15.1:0/760086962 shutdown_connections 2024-02-02T15:39:11.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.979+0000 7ff28e3a4b80 1 --2- 172.21.15.1:0/760086962 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55b795b2d870 0x55b795b2fc90 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.979+0000 7ff28e3a4b80 1 --2- 172.21.15.1:0/760086962 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b795acb7e0 0x55b795b2bcd0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.979+0000 7ff261600700 0 client.28867 ms_handle_reset on v2:172.21.15.71:6836/628016199 2024-02-02T15:39:11.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.979+0000 7ff28e3a4b80 1 --2- 172.21.15.1:0/760086962 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b795ac0010 0x55b795b20c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.979+0000 7ff28e3a4b80 1 --2- 172.21.15.1:0/760086962 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b795ac0930 0x55b795b21190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.979+0000 7ff28e3a4b80 1 -- 172.21.15.1:0/760086962 >> 172.21.15.1:0/760086962 conn(0x55b795a1fdb0 msgr2=0x55b795ac37a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:11.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.979+0000 7ff28e3a4b80 1 -- 172.21.15.1:0/760086962 shutdown_connections 2024-02-02T15:39:11.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.980+0000 7ff28e3a4b80 1 -- 172.21.15.1:0/760086962 wait complete. 2024-02-02T15:39:11.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.980+0000 7ff28e3a4b80 20 client.28867 trim_cache size 0 max 16384 2024-02-02T15:39:11.984 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:39:11.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.980+0000 7ff27a800700 1 -- 172.21.15.1:0/2671243939 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff2540a0690 msgr2=0x7ff2540a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:11.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.980+0000 7ff27a800700 1 --2- 172.21.15.1:0/2671243939 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff2540a0690 0x7ff2540a2b50 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff26400b5c0 tx=0x7ff264005fb0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.980+0000 7ff27a800700 1 -- 172.21.15.1:0/2671243939 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2741060c0 msgr2=0x7ff274199550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:11.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.980+0000 7ff27a800700 1 --2- 172.21.15.1:0/2671243939 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2741060c0 0x7ff274199550 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7ff25c00d950 tx=0x7ff25c00dc60 comp rx=0 tx=0).stop 2024-02-02T15:39:11.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.980+0000 7ff27a800700 1 -- 172.21.15.1:0/2671243939 shutdown_connections 2024-02-02T15:39:11.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.980+0000 7ff27a800700 1 --2- 172.21.15.1:0/2671243939 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff2540a0690 0x7ff2540a2b50 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.980+0000 7ff27a800700 1 --2- 172.21.15.1:0/2671243939 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff27419d9b0 0x7ff27419de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.980+0000 7ff27a800700 1 --2- 172.21.15.1:0/2671243939 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff274100fa0 0x7ff274199010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.980+0000 7ff27a800700 1 --2- 172.21.15.1:0/2671243939 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2741060c0 0x7ff274199550 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:11.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.980+0000 7ff27a800700 1 -- 172.21.15.1:0/2671243939 >> 172.21.15.1:0/2671243939 conn(0x7ff2740fc800 msgr2=0x7ff2740fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:11.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.980+0000 7ff27a800700 1 -- 172.21.15.1:0/2671243939 shutdown_connections 2024-02-02T15:39:11.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:11.981+0000 7ff27a800700 1 -- 172.21.15.1:0/2671243939 wait complete. 2024-02-02T15:39:11.998 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-02T15:39:12.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.210+0000 7f4147e00700 1 -- 172.21.15.1:0/488897274 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4140068010 msgr2=0x7f4140068470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:12.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.210+0000 7f4147e00700 1 --2- 172.21.15.1:0/488897274 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4140068010 0x7f4140068470 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f4128009920 tx=0x7f4128009c30 comp rx=0 tx=0).stop 2024-02-02T15:39:12.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.211+0000 7f4147e00700 1 -- 172.21.15.1:0/488897274 shutdown_connections 2024-02-02T15:39:12.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.211+0000 7f4147e00700 1 --2- 172.21.15.1:0/488897274 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f41400689b0 0x7f414010dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:12.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.211+0000 7f4147e00700 1 --2- 172.21.15.1:0/488897274 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4140106ed0 0x7f41401072b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:12.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.211+0000 7f4147e00700 1 --2- 172.21.15.1:0/488897274 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4140068010 0x7f4140068470 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:12.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.211+0000 7f4147e00700 1 -- 172.21.15.1:0/488897274 >> 172.21.15.1:0/488897274 conn(0x7f41400757d0 msgr2=0x7f4140075be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:12.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.211+0000 7f4147e00700 1 -- 172.21.15.1:0/488897274 shutdown_connections 2024-02-02T15:39:12.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.211+0000 7f4147e00700 1 -- 172.21.15.1:0/488897274 wait complete. 2024-02-02T15:39:12.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.212+0000 7f4147e00700 1 Processor -- start 2024-02-02T15:39:12.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.212+0000 7f4147e00700 1 -- start start 2024-02-02T15:39:12.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.212+0000 7f4147e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4140068010 0x7f41401991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:12.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.212+0000 7f4147e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f41400689b0 0x7f4140199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:12.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.212+0000 7f4147e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4140106ed0 0x7f414019db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:12.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.212+0000 7f4147e00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f414019e1d0 con 0x7f4140068010 2024-02-02T15:39:12.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.212+0000 7f4147e00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f414019e340 con 0x7f41400689b0 2024-02-02T15:39:12.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.212+0000 7f4147e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f414019e500 con 0x7f4140106ed0 2024-02-02T15:39:12.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.213+0000 7f4145e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4140106ed0 0x7f414019db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:12.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.213+0000 7f4144a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f41400689b0 0x7f4140199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:12.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.213+0000 7f4145e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4140106ed0 0x7f414019db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:54050/0 (socket says 172.21.15.1:54050) 2024-02-02T15:39:12.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.213+0000 7f4145400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4140068010 0x7f41401991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:12.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.213+0000 7f4144a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f41400689b0 0x7f4140199730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:32898/0 (socket says 172.21.15.1:32898) 2024-02-02T15:39:12.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.213+0000 7f4145e00700 1 -- 172.21.15.1:0/1519154659 learned_addr learned my addr 172.21.15.1:0/1519154659 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:39:12.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.213+0000 7f4144a00700 1 -- 172.21.15.1:0/1519154659 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4140106ed0 msgr2=0x7f414019db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:12.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.213+0000 7f4144a00700 1 --2- 172.21.15.1:0/1519154659 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4140106ed0 0x7f414019db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:12.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.213+0000 7f4144a00700 1 -- 172.21.15.1:0/1519154659 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4140068010 msgr2=0x7f41401991f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:12.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.213+0000 7f4144a00700 1 --2- 172.21.15.1:0/1519154659 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4140068010 0x7f41401991f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:12.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.213+0000 7f4144a00700 1 -- 172.21.15.1:0/1519154659 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4138009d00 con 0x7f41400689b0 2024-02-02T15:39:12.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.213+0000 7f4145e00700 1 --2- 172.21.15.1:0/1519154659 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4140106ed0 0x7f414019db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:39:12.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.213+0000 7f4144a00700 1 --2- 172.21.15.1:0/1519154659 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f41400689b0 0x7f4140199730 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f4128009230 tx=0x7f412800fa20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:12.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.214+0000 7f413e000700 1 -- 172.21.15.1:0/1519154659 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f412801c070 con 0x7f41400689b0 2024-02-02T15:39:12.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.214+0000 7f413e000700 1 -- 172.21.15.1:0/1519154659 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f412800baa0 con 0x7f41400689b0 2024-02-02T15:39:12.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.214+0000 7f4147e00700 1 -- 172.21.15.1:0/1519154659 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4128009520 con 0x7f41400689b0 2024-02-02T15:39:12.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.214+0000 7f413e000700 1 -- 172.21.15.1:0/1519154659 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4128004930 con 0x7f41400689b0 2024-02-02T15:39:12.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.214+0000 7f4147e00700 1 -- 172.21.15.1:0/1519154659 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41401a5e40 con 0x7f41400689b0 2024-02-02T15:39:12.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.217+0000 7f413e000700 1 -- 172.21.15.1:0/1519154659 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f4128020070 con 0x7f41400689b0 2024-02-02T15:39:12.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.218+0000 7f413e000700 1 --2- 172.21.15.1:0/1519154659 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f41200a0710 0x7f41200a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:12.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.218+0000 7f413e000700 1 -- 172.21.15.1:0/1519154659 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f41280ccd20 con 0x7f41400689b0 2024-02-02T15:39:12.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.219+0000 7f4145400700 1 --2- 172.21.15.1:0/1519154659 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f41200a0710 0x7f41200a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:12.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.219+0000 7f415b995b80 1 Processor -- start 2024-02-02T15:39:12.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.219+0000 7f415b995b80 1 -- start start 2024-02-02T15:39:12.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.219+0000 7f4145400700 1 --2- 172.21.15.1:0/1519154659 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f41200a0710 0x7f41200a2bd0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4130017f80 tx=0x7f4130009260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:12.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.219+0000 7f415b995b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56216c157010 0x56216c1573f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:12.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.219+0000 7f415b995b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56216c157930 0x56216c1622a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:12.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.219+0000 7f415b995b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56216c1627e0 0x56216c164bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:12.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.219+0000 7f415b995b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56216c0cb4c0 con 0x56216c157010 2024-02-02T15:39:12.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.219+0000 7f415b995b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56216c0afaa0 con 0x56216c157930 2024-02-02T15:39:12.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.219+0000 7f415b995b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56216c0ad460 con 0x56216c1627e0 2024-02-02T15:39:12.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.220+0000 7f4145400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56216c157010 0x56216c1573f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:12.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.220+0000 7f4144a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56216c157930 0x56216c1622a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:12.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.220+0000 7f4145e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56216c1627e0 0x56216c164bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:12.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.220+0000 7f4144a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56216c157930 0x56216c1622a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:32904/0 (socket says 172.21.15.1:32904) 2024-02-02T15:39:12.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.220+0000 7f4144a00700 1 -- 172.21.15.1:0/2254047081 learned_addr learned my addr 172.21.15.1:0/2254047081 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:39:12.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.220+0000 7f4144a00700 1 -- 172.21.15.1:0/2254047081 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56216c1627e0 msgr2=0x56216c164bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:12.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.220+0000 7f4144a00700 1 --2- 172.21.15.1:0/2254047081 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56216c1627e0 0x56216c164bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:12.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.220+0000 7f4144a00700 1 -- 172.21.15.1:0/2254047081 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56216c157010 msgr2=0x56216c1573f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:12.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.220+0000 7f4144a00700 1 --2- 172.21.15.1:0/2254047081 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56216c157010 0x56216c1573f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:12.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.220+0000 7f4144a00700 1 -- 172.21.15.1:0/2254047081 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56216bf2a3b0 con 0x56216c157930 2024-02-02T15:39:12.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.220+0000 7f4144a00700 1 --2- 172.21.15.1:0/2254047081 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56216c157930 0x56216c1622a0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f41280968e0 tx=0x7f4128000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4e165e920517dac server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:12.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.220+0000 7f4136600700 1 -- 172.21.15.1:0/2254047081 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f412809a5d0 con 0x56216c157930 2024-02-02T15:39:12.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.221+0000 7f4136600700 1 -- 172.21.15.1:0/2254047081 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4128099e70 con 0x56216c157930 2024-02-02T15:39:12.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.221+0000 7f4136600700 1 -- 172.21.15.1:0/2254047081 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4128005040 con 0x56216c157930 2024-02-02T15:39:12.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.221+0000 7f415b995b80 1 -- 172.21.15.1:0/2254047081 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56216c157930 msgr2=0x56216c1622a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:12.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.221+0000 7f415b995b80 1 --2- 172.21.15.1:0/2254047081 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56216c157930 0x56216c1622a0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f41280968e0 tx=0x7f4128000bc0 comp rx=0 tx=0).stop 2024-02-02T15:39:12.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.221+0000 7f415b995b80 1 -- 172.21.15.1:0/2254047081 shutdown_connections 2024-02-02T15:39:12.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.221+0000 7f415b995b80 1 --2- 172.21.15.1:0/2254047081 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56216c1627e0 0x56216c164bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:12.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.221+0000 7f415b995b80 1 --2- 172.21.15.1:0/2254047081 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56216c157010 0x56216c1573f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:12.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.221+0000 7f415b995b80 1 --2- 172.21.15.1:0/2254047081 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56216c157930 0x56216c1622a0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:12.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.221+0000 7f415b995b80 1 -- 172.21.15.1:0/2254047081 >> 172.21.15.1:0/2254047081 conn(0x56216c0b6db0 msgr2=0x56216c1533b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:12.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.222+0000 7f415b995b80 1 -- 172.21.15.1:0/2254047081 shutdown_connections 2024-02-02T15:39:12.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.222+0000 7f415b995b80 1 -- 172.21.15.1:0/2254047081 wait complete. 2024-02-02T15:39:12.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.222+0000 7f415b995b80 1 -- start start 2024-02-02T15:39:12.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.222+0000 7f415b995b80 1 Processor -- start 2024-02-02T15:39:12.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.223+0000 7f415b995b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56216c157010 0x56216c162bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:12.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.223+0000 7f415b995b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56216c157930 0x56216c163100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:12.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.223+0000 7f415b995b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56216c1627e0 0x56216c163640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:12.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.223+0000 7f415b995b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56216c0a8d50 con 0x56216c1627e0 2024-02-02T15:39:12.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.223+0000 7f415b995b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56216c044d80 con 0x56216c157010 2024-02-02T15:39:12.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.223+0000 7f415b995b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56216c044a20 con 0x56216c157930 2024-02-02T15:39:12.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.223+0000 7f415b995b80 10 client.0 _ll_register_callbacks cb 0x56216c046290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:39:12.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.223+0000 7f4145400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56216c157010 0x56216c162bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:12.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.223+0000 7f4144a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56216c157930 0x56216c163100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:12.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.223+0000 7f4145400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56216c157010 0x56216c162bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:32914/0 (socket says 172.21.15.1:32914) 2024-02-02T15:39:12.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.223+0000 7f4145400700 1 -- 172.21.15.1:0/3323659008 learned_addr learned my addr 172.21.15.1:0/3323659008 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:39:12.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.223+0000 7f4145e00700 1 --2- 172.21.15.1:0/3323659008 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56216c1627e0 0x56216c163640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:12.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.224+0000 7f4145400700 1 -- 172.21.15.1:0/3323659008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56216c157930 msgr2=0x56216c163100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:12.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.224+0000 7f4145400700 1 --2- 172.21.15.1:0/3323659008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56216c157930 0x56216c163100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:12.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.224+0000 7f4145400700 1 -- 172.21.15.1:0/3323659008 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56216c1627e0 msgr2=0x56216c163640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:12.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.224+0000 7f4145400700 1 --2- 172.21.15.1:0/3323659008 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56216c1627e0 0x56216c163640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:12.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.224+0000 7f4145400700 1 -- 172.21.15.1:0/3323659008 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41401a5e40 con 0x56216c157010 2024-02-02T15:39:12.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.224+0000 7f4145e00700 1 --2- 172.21.15.1:0/3323659008 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56216c1627e0 0x56216c163640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:39:12.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.224+0000 7f4144a00700 1 --2- 172.21.15.1:0/3323659008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56216c157930 0x56216c163100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:39:12.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.224+0000 7f4145400700 1 --2- 172.21.15.1:0/3323659008 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56216c157010 0x56216c162bc0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f413000ee40 tx=0x7f41300047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2bcaec5e7f9d5f09 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:12.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.224+0000 7f412ea00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:39:12.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.224+0000 7f412ea00700 1 -- 172.21.15.1:0/3323659008 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f413002a040 con 0x56216c157010 2024-02-02T15:39:12.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.224+0000 7f415b995b80 1 -- 172.21.15.1:0/3323659008 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56216c14b0f0 con 0x56216c157010 2024-02-02T15:39:12.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.224+0000 7f412ea00700 1 -- 172.21.15.1:0/3323659008 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f413002e420 con 0x56216c157010 2024-02-02T15:39:12.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.225+0000 7f412ea00700 1 -- 172.21.15.1:0/3323659008 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f413001c070 con 0x56216c157010 2024-02-02T15:39:12.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.225+0000 7f415b995b80 10 client.28877 fetch_fsmap learned FSMap version 52 2024-02-02T15:39:12.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.225+0000 7f415b995b80 1 -- 172.21.15.1:0/3323659008 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x56216bed6b60 con 0x56216c157010 2024-02-02T15:39:12.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.225+0000 7f412ea00700 1 -- 172.21.15.1:0/3323659008 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56216c14b0f0 con 0x56216c157010 2024-02-02T15:39:12.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.225+0000 7f412ea00700 1 -- 172.21.15.1:0/3323659008 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 52) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f4130022500 con 0x56216c157010 2024-02-02T15:39:12.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.225+0000 7f415b995b80 10 client.28877 fetch_fsmap finished waiting for FSMap version 52 2024-02-02T15:39:12.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.225+0000 7f415b995b80 10 client.28877 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-02T15:39:12.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.225+0000 7f415b995b80 20 client.28877 populate_metadata read hostname 'smithi001' 2024-02-02T15:39:12.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.225+0000 7f415b995b80 1 --2- 172.21.15.1:0/3323659008 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x56216c171e50 0x56216c174270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:12.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.225+0000 7f415b995b80 4 client.28877 mds_command: new command op to 24529 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:39:12.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.225+0000 7f415b995b80 1 -- 172.21.15.1:0/3323659008 --> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56216c0a1dd0 con 0x56216c171e50 2024-02-02T15:39:12.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.225+0000 7f4144a00700 1 --2- 172.21.15.1:0/3323659008 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x56216c171e50 0x56216c174270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:12.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.226+0000 7f4144a00700 1 --2- 172.21.15.1:0/3323659008 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x56216c171e50 0x56216c174270 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=701a26bc49636339 server_cookie=10b3ffdbeff5dc61 in_seq=0 out_seq=0 2024-02-02T15:39:12.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.226+0000 7f412ea00700 10 client.28877 ms_handle_connect on v2:172.21.15.142:6838/321675421 2024-02-02T15:39:12.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.227+0000 7f412ea00700 1 -- 172.21.15.1:0/3323659008 <== mds.1 v2:172.21.15.142:6838/321675421 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x56216c0a1dd0 con 0x56216c171e50 2024-02-02T15:39:12.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.227+0000 7f412ea00700 10 client.28877 handle_command_reply: tid=0 2024-02-02T15:39:12.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.227+0000 7f415b995b80 1 client.28877 shutdown 2024-02-02T15:39:12.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.227+0000 7f415b995b80 1 -- 172.21.15.1:0/3323659008 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56216c157010 msgr2=0x56216c162bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:12.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.227+0000 7f415b995b80 1 --2- 172.21.15.1:0/3323659008 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56216c157010 0x56216c162bc0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f413000ee40 tx=0x7f41300047c0 comp rx=0 tx=0).stop 2024-02-02T15:39:12.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.227+0000 7f415b995b80 1 -- 172.21.15.1:0/3323659008 shutdown_connections 2024-02-02T15:39:12.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.227+0000 7f415b995b80 1 --2- 172.21.15.1:0/3323659008 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x56216c171e50 0x56216c174270 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:12.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.227+0000 7f415b995b80 1 --2- 172.21.15.1:0/3323659008 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56216c1627e0 0x56216c163640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:12.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.227+0000 7f415b995b80 1 --2- 172.21.15.1:0/3323659008 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56216c157010 0x56216c162bc0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:12.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.227+0000 7f415b995b80 1 --2- 172.21.15.1:0/3323659008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56216c157930 0x56216c163100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:12.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.227+0000 7f412ea00700 0 client.28877 ms_handle_reset on v2:172.21.15.142:6838/321675421 2024-02-02T15:39:12.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.227+0000 7f415b995b80 1 -- 172.21.15.1:0/3323659008 >> 172.21.15.1:0/3323659008 conn(0x56216c0b6db0 msgr2=0x56216c152fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:12.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.227+0000 7f415b995b80 1 -- 172.21.15.1:0/3323659008 shutdown_connections 2024-02-02T15:39:12.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.227+0000 7f415b995b80 1 -- 172.21.15.1:0/3323659008 wait complete. 2024-02-02T15:39:12.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.227+0000 7f415b995b80 20 client.28877 trim_cache size 0 max 16384 2024-02-02T15:39:12.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.241+0000 7f415b995b80 1 Processor -- start 2024-02-02T15:39:12.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.241+0000 7f415b995b80 1 -- start start 2024-02-02T15:39:12.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.241+0000 7f415b995b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56216c157010 0x56216c15cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:12.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.241+0000 7f415b995b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56216c157930 0x56216c15d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:12.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.242+0000 7f415b995b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56216c1627e0 0x56216c161060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:12.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.242+0000 7f415b995b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56216bf2c6a0 con 0x56216c157930 2024-02-02T15:39:12.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.242+0000 7f415b995b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56216c15fe00 con 0x56216c1627e0 2024-02-02T15:39:12.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.242+0000 7f415b995b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56216c15d6f0 con 0x56216c157010 2024-02-02T15:39:12.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.242+0000 7f4145e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56216c1627e0 0x56216c161060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:12.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.242+0000 7f4145400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56216c157010 0x56216c15cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:12.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.242+0000 7f4144a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56216c157930 0x56216c15d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:12.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.242+0000 7f4145e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56216c1627e0 0x56216c161060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:32916/0 (socket says 172.21.15.1:32916) 2024-02-02T15:39:12.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.242+0000 7f4145e00700 1 -- 172.21.15.1:0/3256829397 learned_addr learned my addr 172.21.15.1:0/3256829397 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:39:12.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.242+0000 7f4145400700 1 -- 172.21.15.1:0/3256829397 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56216c1627e0 msgr2=0x56216c161060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:12.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.242+0000 7f4145400700 1 --2- 172.21.15.1:0/3256829397 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56216c1627e0 0x56216c161060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:12.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.242+0000 7f4145400700 1 -- 172.21.15.1:0/3256829397 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56216c157930 msgr2=0x56216c15d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:12.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.242+0000 7f4145400700 1 --2- 172.21.15.1:0/3256829397 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56216c157930 0x56216c15d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:12.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.242+0000 7f4145400700 1 -- 172.21.15.1:0/3256829397 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56216c06eea0 con 0x56216c157010 2024-02-02T15:39:12.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.242+0000 7f4145400700 1 --2- 172.21.15.1:0/3256829397 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56216c157010 0x56216c15cbc0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f413000ee10 tx=0x7f413000ef30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9ce1581aa96895c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:12.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.243+0000 7f4137000700 1 -- 172.21.15.1:0/3256829397 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f413002a070 con 0x56216c157010 2024-02-02T15:39:12.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.243+0000 7f4137000700 1 -- 172.21.15.1:0/3256829397 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4130024aa0 con 0x56216c157010 2024-02-02T15:39:12.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.243+0000 7f4137000700 1 -- 172.21.15.1:0/3256829397 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4130004280 con 0x56216c157010 2024-02-02T15:39:12.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.243+0000 7f415b995b80 1 -- 172.21.15.1:0/3256829397 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56216c157010 msgr2=0x56216c15cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:12.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.243+0000 7f415b995b80 1 --2- 172.21.15.1:0/3256829397 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56216c157010 0x56216c15cbc0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f413000ee10 tx=0x7f413000ef30 comp rx=0 tx=0).stop 2024-02-02T15:39:12.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.244+0000 7f415b995b80 1 -- 172.21.15.1:0/3256829397 shutdown_connections 2024-02-02T15:39:12.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.244+0000 7f415b995b80 1 --2- 172.21.15.1:0/3256829397 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56216c1627e0 0x56216c161060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:12.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.244+0000 7f415b995b80 1 --2- 172.21.15.1:0/3256829397 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56216c157010 0x56216c15cbc0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:12.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.244+0000 7f415b995b80 1 --2- 172.21.15.1:0/3256829397 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56216c157930 0x56216c15d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:12.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.244+0000 7f415b995b80 1 -- 172.21.15.1:0/3256829397 >> 172.21.15.1:0/3256829397 conn(0x56216c0b6db0 msgr2=0x56216c15c060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:12.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.244+0000 7f415b995b80 1 -- 172.21.15.1:0/3256829397 shutdown_connections 2024-02-02T15:39:12.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.244+0000 7f415b995b80 1 -- 172.21.15.1:0/3256829397 wait complete. 2024-02-02T15:39:12.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.244+0000 7f415b995b80 1 -- start start 2024-02-02T15:39:12.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.244+0000 7f415b995b80 1 Processor -- start 2024-02-02T15:39:12.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.245+0000 7f415b995b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56216c157010 0x56216c163770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:12.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.245+0000 7f415b995b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56216c157930 0x56216c163cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:12.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.245+0000 7f415b995b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56216c1627e0 0x56216c1c2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:12.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.245+0000 7f415b995b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56216c1c2ea0 con 0x56216c157010 2024-02-02T15:39:12.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.245+0000 7f415b995b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56216c1c2fe0 con 0x56216c1627e0 2024-02-02T15:39:12.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.245+0000 7f415b995b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56216c1c3120 con 0x56216c157930 2024-02-02T15:39:12.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.245+0000 7f415b995b80 10 client.0 _ll_register_callbacks cb 0x56216c046290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:39:12.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.245+0000 7f4145e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56216c1627e0 0x56216c1c2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:12.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.245+0000 7f4144a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56216c157930 0x56216c163cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:12.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.245+0000 7f4145e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56216c1627e0 0x56216c1c2960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:32926/0 (socket says 172.21.15.1:32926) 2024-02-02T15:39:12.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.245+0000 7f4145e00700 1 -- 172.21.15.1:0/3264950285 learned_addr learned my addr 172.21.15.1:0/3264950285 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:39:12.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.245+0000 7f4145400700 1 --2- 172.21.15.1:0/3264950285 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56216c157010 0x56216c163770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:12.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.246+0000 7f4144a00700 1 -- 172.21.15.1:0/3264950285 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56216c1627e0 msgr2=0x56216c1c2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:12.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.246+0000 7f4144a00700 1 --2- 172.21.15.1:0/3264950285 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56216c1627e0 0x56216c1c2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:12.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.246+0000 7f4144a00700 1 -- 172.21.15.1:0/3264950285 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56216c157010 msgr2=0x56216c163770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:12.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.246+0000 7f4144a00700 1 --2- 172.21.15.1:0/3264950285 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56216c157010 0x56216c163770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:12.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.246+0000 7f4144a00700 1 -- 172.21.15.1:0/3264950285 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56216bed6b60 con 0x56216c157930 2024-02-02T15:39:12.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.246+0000 7f4145e00700 1 --2- 172.21.15.1:0/3264950285 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56216c1627e0 0x56216c1c2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:39:12.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.246+0000 7f4144a00700 1 --2- 172.21.15.1:0/3264950285 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56216c157930 0x56216c163cb0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f41280094f0 tx=0x7f4128005450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2168c20ec3962942 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:12.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.246+0000 7f412ea00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:39:12.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.246+0000 7f412ea00700 1 -- 172.21.15.1:0/3264950285 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41280b2150 con 0x56216c157930 2024-02-02T15:39:12.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.246+0000 7f412ea00700 1 -- 172.21.15.1:0/3264950285 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f41280b22b0 con 0x56216c157930 2024-02-02T15:39:12.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.246+0000 7f415b995b80 1 -- 172.21.15.1:0/3264950285 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4128099e70 con 0x56216c157930 2024-02-02T15:39:12.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.246+0000 7f412ea00700 1 -- 172.21.15.1:0/3264950285 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4128005040 con 0x56216c157930 2024-02-02T15:39:12.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.246+0000 7f415b995b80 10 client.28885 fetch_fsmap learned FSMap version 52 2024-02-02T15:39:12.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.246+0000 7f415b995b80 1 -- 172.21.15.1:0/3264950285 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x56216bf2ecd0 con 0x56216c157930 2024-02-02T15:39:12.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.247+0000 7f412ea00700 1 -- 172.21.15.1:0/3264950285 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4128099e70 con 0x56216c157930 2024-02-02T15:39:12.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.247+0000 7f412ea00700 1 -- 172.21.15.1:0/3264950285 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 52) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f412809a1f0 con 0x56216c157930 2024-02-02T15:39:12.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.247+0000 7f415b995b80 10 client.28885 fetch_fsmap finished waiting for FSMap version 52 2024-02-02T15:39:12.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.247+0000 7f415b995b80 10 client.28885 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-02T15:39:12.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.247+0000 7f415b995b80 20 client.28885 populate_metadata read hostname 'smithi001' 2024-02-02T15:39:12.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.247+0000 7f415b995b80 1 --2- 172.21.15.1:0/3264950285 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x56216c1c4550 0x56216c1c6970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:12.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.247+0000 7f415b995b80 4 client.28885 mds_command: new command op to 24529 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:39:12.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.247+0000 7f415b995b80 1 -- 172.21.15.1:0/3264950285 --> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56216c067f50 con 0x56216c1c4550 2024-02-02T15:39:12.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.247+0000 7f4145400700 1 --2- 172.21.15.1:0/3264950285 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x56216c1c4550 0x56216c1c6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:12.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.248+0000 7f4145400700 1 --2- 172.21.15.1:0/3264950285 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x56216c1c4550 0x56216c1c6970 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3927f4a277bc9c66 server_cookie=fbba7fac334201b6 in_seq=0 out_seq=0 2024-02-02T15:39:12.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.248+0000 7f412ea00700 10 client.28885 ms_handle_connect on v2:172.21.15.142:6838/321675421 2024-02-02T15:39:12.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.248+0000 7f412ea00700 1 -- 172.21.15.1:0/3264950285 <== mds.1 v2:172.21.15.142:6838/321675421 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56216c067f50 con 0x56216c1c4550 2024-02-02T15:39:12.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.248+0000 7f412ea00700 10 client.28885 handle_command_reply: tid=0 2024-02-02T15:39:12.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.248+0000 7f415b995b80 1 client.28885 shutdown 2024-02-02T15:39:12.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.248+0000 7f415b995b80 1 -- 172.21.15.1:0/3264950285 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56216c157930 msgr2=0x56216c163cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:12.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.248+0000 7f415b995b80 1 --2- 172.21.15.1:0/3264950285 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56216c157930 0x56216c163cb0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f41280094f0 tx=0x7f4128005450 comp rx=0 tx=0).stop 2024-02-02T15:39:12.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.248+0000 7f415b995b80 1 -- 172.21.15.1:0/3264950285 shutdown_connections 2024-02-02T15:39:12.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.248+0000 7f415b995b80 1 --2- 172.21.15.1:0/3264950285 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x56216c1c4550 0x56216c1c6970 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:12.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.248+0000 7f415b995b80 1 --2- 172.21.15.1:0/3264950285 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56216c1627e0 0x56216c1c2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:12.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.248+0000 7f415b995b80 1 --2- 172.21.15.1:0/3264950285 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56216c157010 0x56216c163770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:12.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.248+0000 7f415b995b80 1 --2- 172.21.15.1:0/3264950285 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56216c157930 0x56216c163cb0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:12.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.248+0000 7f412ea00700 0 client.28885 ms_handle_reset on v2:172.21.15.142:6838/321675421 2024-02-02T15:39:12.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.248+0000 7f415b995b80 1 -- 172.21.15.1:0/3264950285 >> 172.21.15.1:0/3264950285 conn(0x56216c0b6db0 msgr2=0x56216c1525c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:12.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.248+0000 7f415b995b80 1 -- 172.21.15.1:0/3264950285 shutdown_connections 2024-02-02T15:39:12.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.248+0000 7f415b995b80 1 -- 172.21.15.1:0/3264950285 wait complete. 2024-02-02T15:39:12.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.248+0000 7f415b995b80 20 client.28885 trim_cache size 0 max 16384 2024-02-02T15:39:12.253 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:39:12.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.249+0000 7f4147e00700 1 -- 172.21.15.1:0/1519154659 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f41200a0710 msgr2=0x7f41200a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:12.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.249+0000 7f4147e00700 1 --2- 172.21.15.1:0/1519154659 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f41200a0710 0x7f41200a2bd0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4130017f80 tx=0x7f4130009260 comp rx=0 tx=0).stop 2024-02-02T15:39:12.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.249+0000 7f4147e00700 1 -- 172.21.15.1:0/1519154659 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f41400689b0 msgr2=0x7f4140199730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:12.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.249+0000 7f4147e00700 1 --2- 172.21.15.1:0/1519154659 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f41400689b0 0x7f4140199730 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f4128009230 tx=0x7f412800fa20 comp rx=0 tx=0).stop 2024-02-02T15:39:12.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.249+0000 7f4147e00700 1 -- 172.21.15.1:0/1519154659 shutdown_connections 2024-02-02T15:39:12.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.249+0000 7f4147e00700 1 --2- 172.21.15.1:0/1519154659 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f41200a0710 0x7f41200a2bd0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:12.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.249+0000 7f4147e00700 1 --2- 172.21.15.1:0/1519154659 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4140106ed0 0x7f414019db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:12.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.249+0000 7f4147e00700 1 --2- 172.21.15.1:0/1519154659 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4140068010 0x7f41401991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:12.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.249+0000 7f4147e00700 1 --2- 172.21.15.1:0/1519154659 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f41400689b0 0x7f4140199730 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:12.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.249+0000 7f4147e00700 1 -- 172.21.15.1:0/1519154659 >> 172.21.15.1:0/1519154659 conn(0x7f41400757d0 msgr2=0x7f414010bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:12.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.249+0000 7f4147e00700 1 -- 172.21.15.1:0/1519154659 shutdown_connections 2024-02-02T15:39:12.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.249+0000 7f4147e00700 1 -- 172.21.15.1:0/1519154659 wait complete. 2024-02-02T15:39:12.269 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-02T15:39:12.270 INFO:tasks.workunit.client.0.smithi001.stdout: 1 667940 5.44 MB/sec execute 526 sec latency 46.113 ms 2024-02-02T15:39:12.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.487+0000 7f1f9c600700 1 -- 172.21.15.1:0/234038014 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f94100a90 msgr2=0x7f1f941048f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:12.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.487+0000 7f1f9c600700 1 --2- 172.21.15.1:0/234038014 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f94100a90 0x7f1f941048f0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c0099c0 tx=0x7f1f7c009cd0 comp rx=0 tx=0).stop 2024-02-02T15:39:12.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.487+0000 7f1f9c600700 1 -- 172.21.15.1:0/234038014 shutdown_connections 2024-02-02T15:39:12.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.487+0000 7f1f9c600700 1 --2- 172.21.15.1:0/234038014 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f94105020 0x7f1f9410cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:12.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.487+0000 7f1f9c600700 1 --2- 172.21.15.1:0/234038014 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f941000e0 0x7f1f941004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:12.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.487+0000 7f1f9c600700 1 --2- 172.21.15.1:0/234038014 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f94100a90 0x7f1f941048f0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:12.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.487+0000 7f1f9c600700 1 -- 172.21.15.1:0/234038014 >> 172.21.15.1:0/234038014 conn(0x7f1f940757d0 msgr2=0x7f1f94075be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:12.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.488+0000 7f1f9c600700 1 -- 172.21.15.1:0/234038014 shutdown_connections 2024-02-02T15:39:12.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.488+0000 7f1f9c600700 1 -- 172.21.15.1:0/234038014 wait complete. 2024-02-02T15:39:12.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.489+0000 7f1f9c600700 1 Processor -- start 2024-02-02T15:39:12.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.489+0000 7f1f9c600700 1 -- start start 2024-02-02T15:39:12.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.489+0000 7f1f9c600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f941000e0 0x7f1f94101990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:12.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.489+0000 7f1f9c600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f94105020 0x7f1f94101ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:12.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.489+0000 7f1f9c600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f94102410 0x7f1f941a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:12.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.489+0000 7f1f9c600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f941a57c0 con 0x7f1f94105020 2024-02-02T15:39:12.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.489+0000 7f1f9c600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f941a5930 con 0x7f1f94102410 2024-02-02T15:39:12.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.489+0000 7f1f9c600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f941a5af0 con 0x7f1f941000e0 2024-02-02T15:39:12.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.489+0000 7f1f99c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f941000e0 0x7f1f94101990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:12.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.489+0000 7f1f99200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f94105020 0x7f1f94101ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:12.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.490+0000 7f1f9a600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f94102410 0x7f1f941a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:12.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.490+0000 7f1f99c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f941000e0 0x7f1f94101990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:54104/0 (socket says 172.21.15.1:54104) 2024-02-02T15:39:12.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.490+0000 7f1f99c00700 1 -- 172.21.15.1:0/946447082 learned_addr learned my addr 172.21.15.1:0/946447082 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:39:12.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.490+0000 7f1f99200700 1 -- 172.21.15.1:0/946447082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f941000e0 msgr2=0x7f1f94101990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:12.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.490+0000 7f1f99200700 1 --2- 172.21.15.1:0/946447082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f941000e0 0x7f1f94101990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:12.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.490+0000 7f1f99200700 1 -- 172.21.15.1:0/946447082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f94102410 msgr2=0x7f1f941a5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:12.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.490+0000 7f1f99200700 1 --2- 172.21.15.1:0/946447082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f94102410 0x7f1f941a5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:12.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.490+0000 7f1f99200700 1 -- 172.21.15.1:0/946447082 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f7c0095f0 con 0x7f1f94105020 2024-02-02T15:39:12.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.490+0000 7f1f99200700 1 --2- 172.21.15.1:0/946447082 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f94105020 0x7f1f94101ed0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c00b580 tx=0x7f1f7c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:12.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.490+0000 7f1f92000700 1 -- 172.21.15.1:0/946447082 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c01d070 con 0x7f1f94105020 2024-02-02T15:39:12.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.490+0000 7f1f92000700 1 -- 172.21.15.1:0/946447082 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f7c005050 con 0x7f1f94105020 2024-02-02T15:39:12.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.490+0000 7f1f9c600700 1 -- 172.21.15.1:0/946447082 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f941a5d70 con 0x7f1f94105020 2024-02-02T15:39:12.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.491+0000 7f1f92000700 1 -- 172.21.15.1:0/946447082 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c004a00 con 0x7f1f94105020 2024-02-02T15:39:12.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.491+0000 7f1f9c600700 1 -- 172.21.15.1:0/946447082 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f941a6190 con 0x7f1f94105020 2024-02-02T15:39:12.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.494+0000 7f1f92000700 1 -- 172.21.15.1:0/946447082 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f1f7c005470 con 0x7f1f94105020 2024-02-02T15:39:12.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.496+0000 7f1fb0142b80 1 Processor -- start 2024-02-02T15:39:12.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.496+0000 7f1f92000700 1 --2- 172.21.15.1:0/946447082 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f1f740a0690 0x7f1f740a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:12.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.496+0000 7f1fb0142b80 1 -- start start 2024-02-02T15:39:12.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.496+0000 7f1f92000700 1 -- 172.21.15.1:0/946447082 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f1f7c0cce90 con 0x7f1f94105020 2024-02-02T15:39:12.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.496+0000 7f1f9a600700 1 --2- 172.21.15.1:0/946447082 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f1f740a0690 0x7f1f740a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:12.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.496+0000 7f1fb0142b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c180959010 0x55c1809593f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:12.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.497+0000 7f1fb0142b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c180959930 0x55c1809642a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:12.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.497+0000 7f1fb0142b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c1809647e0 0x55c180966bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:12.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.497+0000 7f1fb0142b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1808cd4c0 con 0x55c1809647e0 2024-02-02T15:39:12.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.497+0000 7f1fb0142b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1808b1aa0 con 0x55c180959010 2024-02-02T15:39:12.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.497+0000 7f1fb0142b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1808af460 con 0x55c180959930 2024-02-02T15:39:12.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.497+0000 7f1f99c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c180959010 0x55c1809593f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:12.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.497+0000 7f1f99c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c180959010 0x55c1809593f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:32960/0 (socket says 172.21.15.1:32960) 2024-02-02T15:39:12.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.497+0000 7f1f99c00700 1 -- 172.21.15.1:0/504151523 learned_addr learned my addr 172.21.15.1:0/504151523 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:39:12.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.497+0000 7f1f99200700 1 --2- 172.21.15.1:0/504151523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c180959930 0x55c1809642a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:12.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.497+0000 7f1f99c00700 1 -- 172.21.15.1:0/504151523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c180959930 msgr2=0x55c1809642a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:12.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.497+0000 7f1f99c00700 1 --2- 172.21.15.1:0/504151523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c180959930 0x55c1809642a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:12.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.497+0000 7f1f9a600700 1 --2- 172.21.15.1:0/504151523 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c1809647e0 0x55c180966bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:12.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.497+0000 7f1f99c00700 1 -- 172.21.15.1:0/504151523 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c1809647e0 msgr2=0x55c180966bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:12.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.497+0000 7f1f99c00700 1 --2- 172.21.15.1:0/504151523 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c1809647e0 0x55c180966bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:12.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.497+0000 7f1f99c00700 1 -- 172.21.15.1:0/504151523 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c18072c3b0 con 0x55c180959010 2024-02-02T15:39:12.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.497+0000 7f1f99200700 1 --2- 172.21.15.1:0/504151523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c180959930 0x55c1809642a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:39:12.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.497+0000 7f1f9a600700 1 --2- 172.21.15.1:0/504151523 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c1809647e0 0x55c180966bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:39:12.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.497+0000 7f1f9a600700 1 --2- 172.21.15.1:0/946447082 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f1f740a0690 0x7f1f740a2b50 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c00f540 tx=0x7f1f8c00b040 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:12.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.498+0000 7f1f99c00700 1 --2- 172.21.15.1:0/504151523 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c180959010 0x55c1809593f0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f1f84007dd0 tx=0x7f1f840106b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bea2713a3ae87b25 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:12.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.498+0000 7f1f8aa00700 1 -- 172.21.15.1:0/504151523 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f84007870 con 0x55c180959010 2024-02-02T15:39:12.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.498+0000 7f1f8aa00700 1 -- 172.21.15.1:0/504151523 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f84010e10 con 0x55c180959010 2024-02-02T15:39:12.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.498+0000 7f1f8aa00700 1 -- 172.21.15.1:0/504151523 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f84011820 con 0x55c180959010 2024-02-02T15:39:12.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.499+0000 7f1fb0142b80 1 -- 172.21.15.1:0/504151523 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c180959010 msgr2=0x55c1809593f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:12.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.499+0000 7f1fb0142b80 1 --2- 172.21.15.1:0/504151523 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c180959010 0x55c1809593f0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f1f84007dd0 tx=0x7f1f840106b0 comp rx=0 tx=0).stop 2024-02-02T15:39:12.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.499+0000 7f1fb0142b80 1 -- 172.21.15.1:0/504151523 shutdown_connections 2024-02-02T15:39:12.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.499+0000 7f1fb0142b80 1 --2- 172.21.15.1:0/504151523 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c1809647e0 0x55c180966bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:12.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.499+0000 7f1fb0142b80 1 --2- 172.21.15.1:0/504151523 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c180959010 0x55c1809593f0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:12.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.499+0000 7f1fb0142b80 1 --2- 172.21.15.1:0/504151523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c180959930 0x55c1809642a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:12.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.499+0000 7f1fb0142b80 1 -- 172.21.15.1:0/504151523 >> 172.21.15.1:0/504151523 conn(0x55c1808b8db0 msgr2=0x55c1809553b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:12.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.499+0000 7f1fb0142b80 1 -- 172.21.15.1:0/504151523 shutdown_connections 2024-02-02T15:39:12.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.499+0000 7f1fb0142b80 1 -- 172.21.15.1:0/504151523 wait complete. 2024-02-02T15:39:12.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.499+0000 7f1fb0142b80 1 -- start start 2024-02-02T15:39:12.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.500+0000 7f1fb0142b80 1 Processor -- start 2024-02-02T15:39:12.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.500+0000 7f1fb0142b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c180959010 0x55c180964160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:12.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.500+0000 7f1fb0142b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c180959930 0x55c180961370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:12.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.500+0000 7f1fb0142b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c1809647e0 0x55c180961910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:12.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.500+0000 7f1fb0142b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1808aad50 con 0x55c1809647e0 2024-02-02T15:39:12.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.500+0000 7f1fb0142b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c180846d80 con 0x55c180959010 2024-02-02T15:39:12.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.500+0000 7f1fb0142b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c180846a20 con 0x55c180959930 2024-02-02T15:39:12.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.500+0000 7f1fb0142b80 10 client.0 _ll_register_callbacks cb 0x55c180848290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:39:12.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.501+0000 7f1f99200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c180959930 0x55c180961370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:12.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.501+0000 7f1f99200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c180959930 0x55c180961370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:54134/0 (socket says 172.21.15.1:54134) 2024-02-02T15:39:12.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.501+0000 7f1f99c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c180959010 0x55c180964160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:12.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.501+0000 7f1f99200700 1 -- 172.21.15.1:0/2514883327 learned_addr learned my addr 172.21.15.1:0/2514883327 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:39:12.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.501+0000 7f1f9a600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c1809647e0 0x55c180961910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:12.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.501+0000 7f1f99c00700 1 -- 172.21.15.1:0/2514883327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c180959930 msgr2=0x55c180961370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:12.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.501+0000 7f1f99c00700 1 --2- 172.21.15.1:0/2514883327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c180959930 0x55c180961370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:12.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.501+0000 7f1f99c00700 1 -- 172.21.15.1:0/2514883327 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c1809647e0 msgr2=0x55c180961910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:12.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.501+0000 7f1f99c00700 1 --2- 172.21.15.1:0/2514883327 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c1809647e0 0x55c180961910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:12.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.501+0000 7f1f99c00700 1 -- 172.21.15.1:0/2514883327 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f941a6190 con 0x55c180959010 2024-02-02T15:39:12.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.501+0000 7f1f99c00700 1 --2- 172.21.15.1:0/2514883327 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c180959010 0x55c180964160 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f1f84005fa0 tx=0x7f1f84005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa8c7008a9ef7816 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:12.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.501+0000 7f1f83400700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:39:12.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.501+0000 7f1f83400700 1 -- 172.21.15.1:0/2514883327 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f8401f070 con 0x55c180959010 2024-02-02T15:39:12.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.502+0000 7f1f83400700 1 -- 172.21.15.1:0/2514883327 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f1f84005670 con 0x55c180959010 2024-02-02T15:39:12.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.502+0000 7f1fb0142b80 1 -- 172.21.15.1:0/2514883327 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c18094d0f0 con 0x55c180959010 2024-02-02T15:39:12.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.502+0000 7f1f83400700 1 -- 172.21.15.1:0/2514883327 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f84010c10 con 0x55c180959010 2024-02-02T15:39:12.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.502+0000 7f1fb0142b80 10 client.28895 fetch_fsmap learned FSMap version 52 2024-02-02T15:39:12.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.502+0000 7f1fb0142b80 1 -- 172.21.15.1:0/2514883327 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x7f1f840079d0 con 0x55c180959010 2024-02-02T15:39:12.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.502+0000 7f1f83400700 1 -- 172.21.15.1:0/2514883327 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c18094d0f0 con 0x55c180959010 2024-02-02T15:39:12.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.502+0000 7f1f83400700 1 -- 172.21.15.1:0/2514883327 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 52) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f1f8402b070 con 0x55c180959010 2024-02-02T15:39:12.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.502+0000 7f1fb0142b80 10 client.28895 fetch_fsmap finished waiting for FSMap version 52 2024-02-02T15:39:12.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.502+0000 7f1fb0142b80 10 client.28895 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:39:12.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.502+0000 7f1fb0142b80 20 client.28895 populate_metadata read hostname 'smithi001' 2024-02-02T15:39:12.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.503+0000 7f1fb0142b80 1 --2- 172.21.15.1:0/2514883327 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55c180973f50 0x55c180976370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:12.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.503+0000 7f1fb0142b80 4 client.28895 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:39:12.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.503+0000 7f1fb0142b80 1 -- 172.21.15.1:0/2514883327 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c1808a3dd0 con 0x55c180973f50 2024-02-02T15:39:12.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.503+0000 7f1f99200700 1 --2- 172.21.15.1:0/2514883327 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55c180973f50 0x55c180976370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:12.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.503+0000 7f1f99200700 1 --2- 172.21.15.1:0/2514883327 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55c180973f50 0x55c180976370 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27adefeefa0f5416 server_cookie=9717d25e5d8b9158 in_seq=0 out_seq=0 2024-02-02T15:39:12.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.503+0000 7f1f83400700 10 client.28895 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:39:12.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.504+0000 7f1f83400700 1 -- 172.21.15.1:0/2514883327 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55c1808a3dd0 con 0x55c180973f50 2024-02-02T15:39:12.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.504+0000 7f1f83400700 10 client.28895 handle_command_reply: tid=0 2024-02-02T15:39:12.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.504+0000 7f1fb0142b80 1 client.28895 shutdown 2024-02-02T15:39:12.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.504+0000 7f1fb0142b80 1 -- 172.21.15.1:0/2514883327 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c180959010 msgr2=0x55c180964160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:12.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.504+0000 7f1fb0142b80 1 --2- 172.21.15.1:0/2514883327 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c180959010 0x55c180964160 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f1f84005fa0 tx=0x7f1f84005270 comp rx=0 tx=0).stop 2024-02-02T15:39:12.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.504+0000 7f1fb0142b80 1 -- 172.21.15.1:0/2514883327 shutdown_connections 2024-02-02T15:39:12.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.504+0000 7f1fb0142b80 1 --2- 172.21.15.1:0/2514883327 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55c180973f50 0x55c180976370 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:12.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.504+0000 7f1fb0142b80 1 --2- 172.21.15.1:0/2514883327 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c1809647e0 0x55c180961910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:12.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.504+0000 7f1fb0142b80 1 --2- 172.21.15.1:0/2514883327 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c180959010 0x55c180964160 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:12.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.504+0000 7f1fb0142b80 1 --2- 172.21.15.1:0/2514883327 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c180959930 0x55c180961370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:12.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.504+0000 7f1f83400700 0 client.28895 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:39:12.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.504+0000 7f1fb0142b80 1 -- 172.21.15.1:0/2514883327 >> 172.21.15.1:0/2514883327 conn(0x55c1808b8db0 msgr2=0x55c180954fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:12.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.505+0000 7f1fb0142b80 1 -- 172.21.15.1:0/2514883327 shutdown_connections 2024-02-02T15:39:12.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.505+0000 7f1fb0142b80 1 -- 172.21.15.1:0/2514883327 wait complete. 2024-02-02T15:39:12.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.505+0000 7f1fb0142b80 20 client.28895 trim_cache size 0 max 16384 2024-02-02T15:39:12.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.519+0000 7f1fb0142b80 1 Processor -- start 2024-02-02T15:39:12.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.519+0000 7f1fb0142b80 1 -- start start 2024-02-02T15:39:12.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.520+0000 7f1fb0142b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c180959010 0x55c180976340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:12.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.520+0000 7f1fb0142b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c180959930 0x55c180963c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:12.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.520+0000 7f1fb0142b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1809647e0 0x55c18096f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:12.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.520+0000 7f1fb0142b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c18072e6a0 con 0x55c180959010 2024-02-02T15:39:12.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.520+0000 7f1fb0142b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c18095f970 con 0x55c180959930 2024-02-02T15:39:12.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.520+0000 7f1fb0142b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c18095d230 con 0x55c1809647e0 2024-02-02T15:39:12.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.520+0000 7f1f9a600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1809647e0 0x55c18096f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:12.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.520+0000 7f1f9a600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1809647e0 0x55c18096f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:54148/0 (socket says 172.21.15.1:54148) 2024-02-02T15:39:12.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.520+0000 7f1f99c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c180959010 0x55c180976340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:12.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.520+0000 7f1f9a600700 1 -- 172.21.15.1:0/626023387 learned_addr learned my addr 172.21.15.1:0/626023387 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:39:12.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.520+0000 7f1f99200700 1 --2- 172.21.15.1:0/626023387 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c180959930 0x55c180963c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:12.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.520+0000 7f1f99200700 1 -- 172.21.15.1:0/626023387 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1809647e0 msgr2=0x55c18096f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:12.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.520+0000 7f1f99200700 1 --2- 172.21.15.1:0/626023387 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1809647e0 0x55c18096f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:12.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.520+0000 7f1f99200700 1 -- 172.21.15.1:0/626023387 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c180959010 msgr2=0x55c180976340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:12.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.520+0000 7f1f99200700 1 --2- 172.21.15.1:0/626023387 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c180959010 0x55c180976340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:12.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.521+0000 7f1f99200700 1 -- 172.21.15.1:0/626023387 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c1806d8b60 con 0x55c180959930 2024-02-02T15:39:12.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.521+0000 7f1f99c00700 1 --2- 172.21.15.1:0/626023387 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c180959010 0x55c180976340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:39:12.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.521+0000 7f1f99200700 1 --2- 172.21.15.1:0/626023387 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c180959930 0x55c180963c90 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c000c00 tx=0x7f1f7c005720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18a35d9cb820c14f server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:12.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.521+0000 7f1f8b400700 1 -- 172.21.15.1:0/626023387 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c0b9040 con 0x55c180959930 2024-02-02T15:39:12.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.521+0000 7f1f8b400700 1 -- 172.21.15.1:0/626023387 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f7c09de10 con 0x55c180959930 2024-02-02T15:39:12.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.521+0000 7f1f8b400700 1 -- 172.21.15.1:0/626023387 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c0b24d0 con 0x55c180959930 2024-02-02T15:39:12.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.522+0000 7f1fb0142b80 1 -- 172.21.15.1:0/626023387 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c180959930 msgr2=0x55c180963c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:12.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.522+0000 7f1fb0142b80 1 --2- 172.21.15.1:0/626023387 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c180959930 0x55c180963c90 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c000c00 tx=0x7f1f7c005720 comp rx=0 tx=0).stop 2024-02-02T15:39:12.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.522+0000 7f1fb0142b80 1 -- 172.21.15.1:0/626023387 shutdown_connections 2024-02-02T15:39:12.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.522+0000 7f1fb0142b80 1 --2- 172.21.15.1:0/626023387 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c1809647e0 0x55c18096f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:12.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.522+0000 7f1fb0142b80 1 --2- 172.21.15.1:0/626023387 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c180959010 0x55c180976340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:12.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.522+0000 7f1fb0142b80 1 --2- 172.21.15.1:0/626023387 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c180959930 0x55c180963c90 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:12.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.522+0000 7f1fb0142b80 1 -- 172.21.15.1:0/626023387 >> 172.21.15.1:0/626023387 conn(0x55c1808b8db0 msgr2=0x55c1809704e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:12.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.522+0000 7f1fb0142b80 1 -- 172.21.15.1:0/626023387 shutdown_connections 2024-02-02T15:39:12.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.522+0000 7f1fb0142b80 1 -- 172.21.15.1:0/626023387 wait complete. 2024-02-02T15:39:12.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.522+0000 7f1fb0142b80 1 -- start start 2024-02-02T15:39:12.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.523+0000 7f1fb0142b80 1 Processor -- start 2024-02-02T15:39:12.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.523+0000 7f1fb0142b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c180959010 0x55c180969310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:12.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.523+0000 7f1fb0142b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c180959930 0x55c180969850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:12.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.523+0000 7f1fb0142b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c1809647e0 0x55c180969d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:12.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.523+0000 7f1fb0142b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1809c4a70 con 0x55c180959010 2024-02-02T15:39:12.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.523+0000 7f1fb0142b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1809c4bb0 con 0x55c1809647e0 2024-02-02T15:39:12.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.523+0000 7f1fb0142b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1809c4d20 con 0x55c180959930 2024-02-02T15:39:12.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.524+0000 7f1fb0142b80 10 client.0 _ll_register_callbacks cb 0x55c180848290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:39:12.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.524+0000 7f1f99200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c180959930 0x55c180969850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:12.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.524+0000 7f1f99c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c180959010 0x55c180969310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:12.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.524+0000 7f1f99200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c180959930 0x55c180969850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:54162/0 (socket says 172.21.15.1:54162) 2024-02-02T15:39:12.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.524+0000 7f1f9a600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c1809647e0 0x55c180969d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:12.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.524+0000 7f1f99200700 1 -- 172.21.15.1:0/982015219 learned_addr learned my addr 172.21.15.1:0/982015219 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:39:12.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.524+0000 7f1f9a600700 1 -- 172.21.15.1:0/982015219 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c180959930 msgr2=0x55c180969850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:12.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.524+0000 7f1f9a600700 1 --2- 172.21.15.1:0/982015219 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c180959930 0x55c180969850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:12.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.524+0000 7f1f9a600700 1 -- 172.21.15.1:0/982015219 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c180959010 msgr2=0x55c180969310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:12.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.524+0000 7f1f9a600700 1 --2- 172.21.15.1:0/982015219 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c180959010 0x55c180969310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:12.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.524+0000 7f1f9a600700 1 -- 172.21.15.1:0/982015219 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f840079d0 con 0x55c1809647e0 2024-02-02T15:39:12.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.524+0000 7f1f9a600700 1 --2- 172.21.15.1:0/982015219 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c1809647e0 0x55c180969d90 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x55c18094f860 tx=0x7f1f8c004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f85f2247fa0e91b7 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:12.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.524+0000 7f1f83400700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:39:12.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.525+0000 7f1f83400700 1 -- 172.21.15.1:0/982015219 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c00e040 con 0x55c1809647e0 2024-02-02T15:39:12.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.525+0000 7f1f83400700 1 -- 172.21.15.1:0/982015219 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f1f8c01c040 con 0x55c1809647e0 2024-02-02T15:39:12.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.525+0000 7f1fb0142b80 1 -- 172.21.15.1:0/982015219 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1f84010e10 con 0x55c1809647e0 2024-02-02T15:39:12.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.525+0000 7f1f83400700 1 -- 172.21.15.1:0/982015219 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f8c008ee0 con 0x55c1809647e0 2024-02-02T15:39:12.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.525+0000 7f1fb0142b80 10 client.28901 fetch_fsmap learned FSMap version 52 2024-02-02T15:39:12.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.525+0000 7f1fb0142b80 1 -- 172.21.15.1:0/982015219 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x7f1f941a5d70 con 0x55c1809647e0 2024-02-02T15:39:12.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.525+0000 7f1f83400700 1 -- 172.21.15.1:0/982015219 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f84010e10 con 0x55c1809647e0 2024-02-02T15:39:12.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.525+0000 7f1f83400700 1 -- 172.21.15.1:0/982015219 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 52) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f1f8c0044b0 con 0x55c1809647e0 2024-02-02T15:39:12.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.525+0000 7f1fb0142b80 10 client.28901 fetch_fsmap finished waiting for FSMap version 52 2024-02-02T15:39:12.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.525+0000 7f1fb0142b80 10 client.28901 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:39:12.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.525+0000 7f1fb0142b80 20 client.28901 populate_metadata read hostname 'smithi001' 2024-02-02T15:39:12.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.525+0000 7f1fb0142b80 1 --2- 172.21.15.1:0/982015219 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55c1809c6480 0x55c1809c88a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:12.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.525+0000 7f1fb0142b80 4 client.28901 mds_command: new command op to 24557 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:39:12.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.525+0000 7f1fb0142b80 1 -- 172.21.15.1:0/982015219 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c180869f50 con 0x55c1809c6480 2024-02-02T15:39:12.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.526+0000 7f1f99c00700 1 --2- 172.21.15.1:0/982015219 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55c1809c6480 0x55c1809c88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:12.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.526+0000 7f1f99c00700 1 --2- 172.21.15.1:0/982015219 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55c1809c6480 0x55c1809c88a0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e0b7fa331a400a5 server_cookie=9ebe3b3fbb514ee0 in_seq=0 out_seq=0 2024-02-02T15:39:12.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.526+0000 7f1f83400700 10 client.28901 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:39:12.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.526+0000 7f1f83400700 1 -- 172.21.15.1:0/982015219 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c180869f50 con 0x55c1809c6480 2024-02-02T15:39:12.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.526+0000 7f1f83400700 10 client.28901 handle_command_reply: tid=0 2024-02-02T15:39:12.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.526+0000 7f1fb0142b80 1 client.28901 shutdown 2024-02-02T15:39:12.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.527+0000 7f1fb0142b80 1 -- 172.21.15.1:0/982015219 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c1809647e0 msgr2=0x55c180969d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:12.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.527+0000 7f1fb0142b80 1 --2- 172.21.15.1:0/982015219 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c1809647e0 0x55c180969d90 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x55c18094f860 tx=0x7f1f8c004ad0 comp rx=0 tx=0).stop 2024-02-02T15:39:12.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.527+0000 7f1fb0142b80 1 -- 172.21.15.1:0/982015219 shutdown_connections 2024-02-02T15:39:12.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.527+0000 7f1fb0142b80 1 --2- 172.21.15.1:0/982015219 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55c1809c6480 0x55c1809c88a0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:12.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.527+0000 7f1fb0142b80 1 --2- 172.21.15.1:0/982015219 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c1809647e0 0x55c180969d90 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:12.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.527+0000 7f1fb0142b80 1 --2- 172.21.15.1:0/982015219 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c180959010 0x55c180969310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:12.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.527+0000 7f1fb0142b80 1 --2- 172.21.15.1:0/982015219 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c180959930 0x55c180969850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:12.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.527+0000 7f1f83400700 0 client.28901 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:39:12.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.527+0000 7f1fb0142b80 1 -- 172.21.15.1:0/982015219 >> 172.21.15.1:0/982015219 conn(0x55c1808b8db0 msgr2=0x55c18096f900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:12.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.527+0000 7f1fb0142b80 1 -- 172.21.15.1:0/982015219 shutdown_connections 2024-02-02T15:39:12.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.527+0000 7f1fb0142b80 1 -- 172.21.15.1:0/982015219 wait complete. 2024-02-02T15:39:12.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.527+0000 7f1fb0142b80 20 client.28901 trim_cache size 0 max 16384 2024-02-02T15:39:12.531 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:39:12.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.528+0000 7f1f9c600700 1 -- 172.21.15.1:0/946447082 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f1f740a0690 msgr2=0x7f1f740a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:12.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.528+0000 7f1f9c600700 1 --2- 172.21.15.1:0/946447082 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f1f740a0690 0x7f1f740a2b50 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c00f540 tx=0x7f1f8c00b040 comp rx=0 tx=0).stop 2024-02-02T15:39:12.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.528+0000 7f1f9c600700 1 -- 172.21.15.1:0/946447082 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f94105020 msgr2=0x7f1f94101ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:12.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.528+0000 7f1f9c600700 1 --2- 172.21.15.1:0/946447082 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f94105020 0x7f1f94101ed0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c00b580 tx=0x7f1f7c00f690 comp rx=0 tx=0).stop 2024-02-02T15:39:12.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.528+0000 7f1f9c600700 1 -- 172.21.15.1:0/946447082 shutdown_connections 2024-02-02T15:39:12.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.528+0000 7f1f9c600700 1 --2- 172.21.15.1:0/946447082 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f1f740a0690 0x7f1f740a2b50 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:12.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.528+0000 7f1f9c600700 1 --2- 172.21.15.1:0/946447082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1f94102410 0x7f1f941a5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:12.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.528+0000 7f1f9c600700 1 --2- 172.21.15.1:0/946447082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1f941000e0 0x7f1f94101990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:12.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.528+0000 7f1f9c600700 1 --2- 172.21.15.1:0/946447082 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1f94105020 0x7f1f94101ed0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:12.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.528+0000 7f1f9c600700 1 -- 172.21.15.1:0/946447082 >> 172.21.15.1:0/946447082 conn(0x7f1f940757d0 msgr2=0x7f1f94068110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:12.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.528+0000 7f1f9c600700 1 -- 172.21.15.1:0/946447082 shutdown_connections 2024-02-02T15:39:12.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:12.528+0000 7f1f9c600700 1 -- 172.21.15.1:0/946447082 wait complete. 2024-02-02T15:39:13.271 INFO:tasks.workunit.client.0.smithi001.stdout: 1 669032 5.44 MB/sec execute 527 sec latency 17.905 ms 2024-02-02T15:39:13.547 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-02T15:39:13.547 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-02T15:39:13.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.760+0000 7fd784000700 1 -- 172.21.15.1:0/1118965209 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd77c073170 msgr2=0x7fd77c073550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:13.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.760+0000 7fd784000700 1 --2- 172.21.15.1:0/1118965209 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd77c073170 0x7fd77c073550 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fd76c009990 tx=0x7fd76c009ca0 comp rx=0 tx=0).stop 2024-02-02T15:39:13.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.761+0000 7fd784000700 1 -- 172.21.15.1:0/1118965209 shutdown_connections 2024-02-02T15:39:13.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.761+0000 7fd784000700 1 --2- 172.21.15.1:0/1118965209 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd77c10d5c0 0x7fd77c10fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:13.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.761+0000 7fd784000700 1 --2- 172.21.15.1:0/1118965209 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd77c073170 0x7fd77c073550 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:13.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.761+0000 7fd784000700 1 --2- 172.21.15.1:0/1118965209 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd77c073a90 0x7fd77c10d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:13.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.761+0000 7fd784000700 1 -- 172.21.15.1:0/1118965209 >> 172.21.15.1:0/1118965209 conn(0x7fd77c0fc810 msgr2=0x7fd77c0fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:13.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.761+0000 7fd784000700 1 -- 172.21.15.1:0/1118965209 shutdown_connections 2024-02-02T15:39:13.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.761+0000 7fd784000700 1 -- 172.21.15.1:0/1118965209 wait complete. 2024-02-02T15:39:13.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.762+0000 7fd784000700 1 Processor -- start 2024-02-02T15:39:13.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.762+0000 7fd784000700 1 -- start start 2024-02-02T15:39:13.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.762+0000 7fd784000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd77c073170 0x7fd77c199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:13.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.762+0000 7fd784000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd77c073a90 0x7fd77c199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:13.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.762+0000 7fd784000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd77c10d5c0 0x7fd77c19dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:13.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.762+0000 7fd784000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd77c19e0f0 con 0x7fd77c073170 2024-02-02T15:39:13.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.762+0000 7fd784000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd77c19e260 con 0x7fd77c10d5c0 2024-02-02T15:39:13.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.762+0000 7fd784000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd77c19e420 con 0x7fd77c073a90 2024-02-02T15:39:13.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.762+0000 7fd781600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd77c073170 0x7fd77c199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:13.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.762+0000 7fd781600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd77c073170 0x7fd77c199110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56846/0 (socket says 172.21.15.1:56846) 2024-02-02T15:39:13.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.762+0000 7fd781600700 1 -- 172.21.15.1:0/1536362369 learned_addr learned my addr 172.21.15.1:0/1536362369 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:39:13.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.763+0000 7fd780c00700 1 --2- 172.21.15.1:0/1536362369 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd77c073a90 0x7fd77c199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:13.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.763+0000 7fd782000700 1 --2- 172.21.15.1:0/1536362369 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd77c10d5c0 0x7fd77c19dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:13.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.763+0000 7fd781600700 1 -- 172.21.15.1:0/1536362369 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd77c073a90 msgr2=0x7fd77c199650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:13.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.763+0000 7fd781600700 1 --2- 172.21.15.1:0/1536362369 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd77c073a90 0x7fd77c199650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:13.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.763+0000 7fd781600700 1 -- 172.21.15.1:0/1536362369 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd77c10d5c0 msgr2=0x7fd77c19dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:13.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.763+0000 7fd781600700 1 --2- 172.21.15.1:0/1536362369 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd77c10d5c0 0x7fd77c19dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:13.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.763+0000 7fd781600700 1 -- 172.21.15.1:0/1536362369 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd764009520 con 0x7fd77c073170 2024-02-02T15:39:13.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.763+0000 7fd781600700 1 --2- 172.21.15.1:0/1536362369 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd77c073170 0x7fd77c199110 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fd76c005910 tx=0x7fd76c005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:13.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.763+0000 7fd77a000700 1 -- 172.21.15.1:0/1536362369 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd76c01d070 con 0x7fd77c073170 2024-02-02T15:39:13.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.763+0000 7fd784000700 1 -- 172.21.15.1:0/1536362369 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd76c0095f0 con 0x7fd77c073170 2024-02-02T15:39:13.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.763+0000 7fd77a000700 1 -- 172.21.15.1:0/1536362369 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd76c022470 con 0x7fd77c073170 2024-02-02T15:39:13.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.763+0000 7fd77a000700 1 -- 172.21.15.1:0/1536362369 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd76c017650 con 0x7fd77c073170 2024-02-02T15:39:13.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.764+0000 7fd784000700 1 -- 172.21.15.1:0/1536362369 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd77c1a5d60 con 0x7fd77c073170 2024-02-02T15:39:13.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.767+0000 7fd77a000700 1 -- 172.21.15.1:0/1536362369 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7fd76c0225e0 con 0x7fd77c073170 2024-02-02T15:39:13.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.768+0000 7fd797a4cb80 1 Processor -- start 2024-02-02T15:39:13.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.768+0000 7fd797a4cb80 1 -- start start 2024-02-02T15:39:13.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.768+0000 7fd77a000700 1 --2- 172.21.15.1:0/1536362369 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fd75c0a0690 0x7fd75c0a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:13.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.768+0000 7fd77a000700 1 -- 172.21.15.1:0/1536362369 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9541+0+0 (secure 0 0 0) 0x7fd76c0cd520 con 0x7fd77c073170 2024-02-02T15:39:13.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.768+0000 7fd797a4cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562517afe090 0x562517afe470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:13.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.768+0000 7fd797a4cb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562517afe9b0 0x562517b09320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:13.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.768+0000 7fd797a4cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562517b09860 0x562517b0bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:13.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.768+0000 7fd797a4cb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x562517a72540 con 0x562517afe9b0 2024-02-02T15:39:13.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.768+0000 7fd797a4cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x562517a56b20 con 0x562517afe090 2024-02-02T15:39:13.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.768+0000 7fd797a4cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x562517a544e0 con 0x562517b09860 2024-02-02T15:39:13.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.768+0000 7fd782000700 1 --2- 172.21.15.1:0/1536362369 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fd75c0a0690 0x7fd75c0a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:13.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.769+0000 7fd780c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562517afe9b0 0x562517b09320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:13.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.769+0000 7fd780c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562517afe9b0 0x562517b09320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56860/0 (socket says 172.21.15.1:56860) 2024-02-02T15:39:13.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.769+0000 7fd781600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562517afe090 0x562517afe470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:13.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.769+0000 7fd780c00700 1 -- 172.21.15.1:0/1767786453 learned_addr learned my addr 172.21.15.1:0/1767786453 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:39:13.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.769+0000 7fd782000700 1 --2- 172.21.15.1:0/1767786453 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562517b09860 0x562517b0bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:13.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.769+0000 7fd781600700 1 -- 172.21.15.1:0/1767786453 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562517b09860 msgr2=0x562517b0bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:13.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.769+0000 7fd781600700 1 --2- 172.21.15.1:0/1767786453 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562517b09860 0x562517b0bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:13.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.769+0000 7fd781600700 1 -- 172.21.15.1:0/1767786453 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562517afe9b0 msgr2=0x562517b09320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:13.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.769+0000 7fd781600700 1 --2- 172.21.15.1:0/1767786453 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562517afe9b0 0x562517b09320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:13.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.769+0000 7fd781600700 1 -- 172.21.15.1:0/1767786453 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5625178d1430 con 0x562517afe090 2024-02-02T15:39:13.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.769+0000 7fd781600700 1 --2- 172.21.15.1:0/1767786453 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562517afe090 0x562517afe470 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fd76c0ca7e0 tx=0x7fd76c022c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f03ad3538b289930 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:13.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.769+0000 7fd782000700 1 --2- 172.21.15.1:0/1536362369 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fd75c0a0690 0x7fd75c0a2b50 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd77400f870 tx=0x7fd77400fc30 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:13.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.770+0000 7fd772600700 1 -- 172.21.15.1:0/1767786453 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd76c09a2d0 con 0x562517afe090 2024-02-02T15:39:13.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.770+0000 7fd772600700 1 -- 172.21.15.1:0/1767786453 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd76c09a0f0 con 0x562517afe090 2024-02-02T15:39:13.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.770+0000 7fd772600700 1 -- 172.21.15.1:0/1767786453 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd76c0cd160 con 0x562517afe090 2024-02-02T15:39:13.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.770+0000 7fd797a4cb80 1 -- 172.21.15.1:0/1767786453 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562517afe090 msgr2=0x562517afe470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:13.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.770+0000 7fd797a4cb80 1 --2- 172.21.15.1:0/1767786453 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562517afe090 0x562517afe470 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fd76c0ca7e0 tx=0x7fd76c022c90 comp rx=0 tx=0).stop 2024-02-02T15:39:13.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.770+0000 7fd797a4cb80 1 -- 172.21.15.1:0/1767786453 shutdown_connections 2024-02-02T15:39:13.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.770+0000 7fd797a4cb80 1 --2- 172.21.15.1:0/1767786453 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562517b09860 0x562517b0bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:13.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.770+0000 7fd797a4cb80 1 --2- 172.21.15.1:0/1767786453 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562517afe090 0x562517afe470 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:13.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.770+0000 7fd797a4cb80 1 --2- 172.21.15.1:0/1767786453 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562517afe9b0 0x562517b09320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:13.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.770+0000 7fd797a4cb80 1 -- 172.21.15.1:0/1767786453 >> 172.21.15.1:0/1767786453 conn(0x562517a5de30 msgr2=0x562517afa430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:13.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.770+0000 7fd797a4cb80 1 -- 172.21.15.1:0/1767786453 shutdown_connections 2024-02-02T15:39:13.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.771+0000 7fd797a4cb80 1 -- 172.21.15.1:0/1767786453 wait complete. 2024-02-02T15:39:13.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.771+0000 7fd797a4cb80 1 -- start start 2024-02-02T15:39:13.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.771+0000 7fd797a4cb80 1 Processor -- start 2024-02-02T15:39:13.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.772+0000 7fd797a4cb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562517afe090 0x562517b09c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:13.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.772+0000 7fd797a4cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562517afe9b0 0x562517b0a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:13.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.772+0000 7fd797a4cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562517b09860 0x562517b0a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:13.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.772+0000 7fd797a4cb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x562517a4fdd0 con 0x562517afe090 2024-02-02T15:39:13.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.772+0000 7fd797a4cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625179ebe00 con 0x562517b09860 2024-02-02T15:39:13.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.772+0000 7fd797a4cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625179ebaa0 con 0x562517afe9b0 2024-02-02T15:39:13.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.772+0000 7fd797a4cb80 10 client.0 _ll_register_callbacks cb 0x5625179ed310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:39:13.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.772+0000 7fd781600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562517afe090 0x562517b09c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:13.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.772+0000 7fd781600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562517afe090 0x562517b09c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56866/0 (socket says 172.21.15.1:56866) 2024-02-02T15:39:13.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.772+0000 7fd781600700 1 -- 172.21.15.1:0/1793206281 learned_addr learned my addr 172.21.15.1:0/1793206281 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:39:13.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.772+0000 7fd782000700 1 --2- 172.21.15.1:0/1793206281 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562517b09860 0x562517b0a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:13.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.772+0000 7fd780c00700 1 --2- 172.21.15.1:0/1793206281 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562517afe9b0 0x562517b0a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:13.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.772+0000 7fd781600700 1 -- 172.21.15.1:0/1793206281 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562517afe9b0 msgr2=0x562517b0a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:13.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.772+0000 7fd781600700 1 --2- 172.21.15.1:0/1793206281 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562517afe9b0 0x562517b0a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:13.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.772+0000 7fd781600700 1 -- 172.21.15.1:0/1793206281 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562517b09860 msgr2=0x562517b0a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:13.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.772+0000 7fd781600700 1 --2- 172.21.15.1:0/1793206281 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562517b09860 0x562517b0a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:13.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.772+0000 7fd781600700 1 -- 172.21.15.1:0/1793206281 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd77c1a5d60 con 0x562517afe090 2024-02-02T15:39:13.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.772+0000 7fd780c00700 1 --2- 172.21.15.1:0/1793206281 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562517afe9b0 0x562517b0a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:39:13.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.773+0000 7fd781600700 1 --2- 172.21.15.1:0/1793206281 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562517afe090 0x562517b09c40 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fd76c029000 tx=0x7fd76c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa9ea31bff55364c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:13.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.773+0000 7fd76aa00700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:39:13.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.773+0000 7fd76aa00700 1 -- 172.21.15.1:0/1793206281 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd76c0b2ea0 con 0x562517afe090 2024-02-02T15:39:13.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.773+0000 7fd76aa00700 1 -- 172.21.15.1:0/1793206281 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9541+0+0 (secure 0 0 0) 0x7fd76c09e5c0 con 0x562517afe090 2024-02-02T15:39:13.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.773+0000 7fd797a4cb80 1 -- 172.21.15.1:0/1793206281 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562517af2170 con 0x562517afe090 2024-02-02T15:39:13.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.773+0000 7fd76aa00700 1 -- 172.21.15.1:0/1793206281 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd76c09ae90 con 0x562517afe090 2024-02-02T15:39:13.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.773+0000 7fd797a4cb80 10 client.18813 fetch_fsmap learned FSMap version 52 2024-02-02T15:39:13.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.773+0000 7fd797a4cb80 1 -- 172.21.15.1:0/1793206281 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x56251787dbe0 con 0x562517afe090 2024-02-02T15:39:13.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.773+0000 7fd76aa00700 1 -- 172.21.15.1:0/1793206281 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562517af2170 con 0x562517afe090 2024-02-02T15:39:13.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.774+0000 7fd76aa00700 1 -- 172.21.15.1:0/1793206281 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 52) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fd76c0a22e0 con 0x562517afe090 2024-02-02T15:39:13.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.774+0000 7fd797a4cb80 10 client.18813 fetch_fsmap finished waiting for FSMap version 52 2024-02-02T15:39:13.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.774+0000 7fd797a4cb80 10 client.18813 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:39:13.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.774+0000 7fd797a4cb80 20 client.18813 populate_metadata read hostname 'smithi001' 2024-02-02T15:39:13.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.774+0000 7fd797a4cb80 1 --2- 172.21.15.1:0/1793206281 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x562517b189d0 0x562517b1adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:13.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.774+0000 7fd797a4cb80 4 client.18813 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:39:13.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.774+0000 7fd797a4cb80 1 -- 172.21.15.1:0/1793206281 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562517a48e50 con 0x562517b189d0 2024-02-02T15:39:13.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.774+0000 7fd780c00700 1 --2- 172.21.15.1:0/1793206281 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x562517b189d0 0x562517b1adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:13.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.774+0000 7fd780c00700 1 --2- 172.21.15.1:0/1793206281 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x562517b189d0 0x562517b1adc0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f4c4774149da6ef server_cookie=84448cfc4ff8080f in_seq=0 out_seq=0 2024-02-02T15:39:13.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.774+0000 7fd76aa00700 10 client.18813 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:39:13.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.775+0000 7fd76aa00700 1 -- 172.21.15.1:0/1793206281 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x562517a48e50 con 0x562517b189d0 2024-02-02T15:39:13.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.775+0000 7fd76aa00700 10 client.18813 handle_command_reply: tid=0 2024-02-02T15:39:13.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.775+0000 7fd797a4cb80 1 client.18813 shutdown 2024-02-02T15:39:13.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.775+0000 7fd797a4cb80 1 -- 172.21.15.1:0/1793206281 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562517afe090 msgr2=0x562517b09c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:13.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.775+0000 7fd797a4cb80 1 --2- 172.21.15.1:0/1793206281 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562517afe090 0x562517b09c40 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fd76c029000 tx=0x7fd76c009310 comp rx=0 tx=0).stop 2024-02-02T15:39:13.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.775+0000 7fd797a4cb80 1 -- 172.21.15.1:0/1793206281 shutdown_connections 2024-02-02T15:39:13.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.775+0000 7fd797a4cb80 1 --2- 172.21.15.1:0/1793206281 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x562517b189d0 0x562517b1adc0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:13.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.775+0000 7fd797a4cb80 1 --2- 172.21.15.1:0/1793206281 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562517b09860 0x562517b0a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:13.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.775+0000 7fd797a4cb80 1 --2- 172.21.15.1:0/1793206281 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562517afe090 0x562517b09c40 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:13.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.775+0000 7fd797a4cb80 1 --2- 172.21.15.1:0/1793206281 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562517afe9b0 0x562517b0a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:13.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.775+0000 7fd76aa00700 0 client.18813 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:39:13.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.775+0000 7fd797a4cb80 1 -- 172.21.15.1:0/1793206281 >> 172.21.15.1:0/1793206281 conn(0x562517a5de30 msgr2=0x562517afa020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:13.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.776+0000 7fd797a4cb80 1 -- 172.21.15.1:0/1793206281 shutdown_connections 2024-02-02T15:39:13.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.776+0000 7fd797a4cb80 1 -- 172.21.15.1:0/1793206281 wait complete. 2024-02-02T15:39:13.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.776+0000 7fd797a4cb80 20 client.18813 trim_cache size 0 max 16384 2024-02-02T15:39:13.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.793+0000 7fd797a4cb80 1 Processor -- start 2024-02-02T15:39:13.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.793+0000 7fd797a4cb80 1 -- start start 2024-02-02T15:39:13.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.793+0000 7fd797a4cb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562517afe090 0x562517b0ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:13.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.793+0000 7fd797a4cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562517afe9b0 0x562517b0a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:13.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.793+0000 7fd797a4cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562517b09860 0x562517b14230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:13.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.793+0000 7fd797a4cb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5625178d3720 con 0x562517afe090 2024-02-02T15:39:13.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.793+0000 7fd797a4cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x562517b08ef0 con 0x562517afe9b0 2024-02-02T15:39:13.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.793+0000 7fd797a4cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x562517b08790 con 0x562517b09860 2024-02-02T15:39:13.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.794+0000 7fd781600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562517afe090 0x562517b0ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:13.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.794+0000 7fd781600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562517afe090 0x562517b0ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:56878/0 (socket says 172.21.15.1:56878) 2024-02-02T15:39:13.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.794+0000 7fd781600700 1 -- 172.21.15.1:0/2385527194 learned_addr learned my addr 172.21.15.1:0/2385527194 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:39:13.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.794+0000 7fd780c00700 1 --2- 172.21.15.1:0/2385527194 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562517afe9b0 0x562517b0a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:13.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.794+0000 7fd782000700 1 --2- 172.21.15.1:0/2385527194 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562517b09860 0x562517b14230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:13.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.794+0000 7fd781600700 1 -- 172.21.15.1:0/2385527194 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562517b09860 msgr2=0x562517b14230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:13.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.794+0000 7fd781600700 1 --2- 172.21.15.1:0/2385527194 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562517b09860 0x562517b14230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:13.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.794+0000 7fd781600700 1 -- 172.21.15.1:0/2385527194 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562517afe9b0 msgr2=0x562517b0a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:13.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.794+0000 7fd781600700 1 --2- 172.21.15.1:0/2385527194 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562517afe9b0 0x562517b0a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:13.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.794+0000 7fd781600700 1 -- 172.21.15.1:0/2385527194 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562517a15f20 con 0x562517afe090 2024-02-02T15:39:13.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.794+0000 7fd780c00700 1 --2- 172.21.15.1:0/2385527194 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562517afe9b0 0x562517b0a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:39:13.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.794+0000 7fd781600700 1 --2- 172.21.15.1:0/2385527194 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562517afe090 0x562517b0ec40 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fd76c09a720 tx=0x7fd76c00b4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9af487baec26578b server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:13.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.794+0000 7fd773000700 1 -- 172.21.15.1:0/2385527194 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd76c0b2ea0 con 0x562517afe090 2024-02-02T15:39:13.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.794+0000 7fd773000700 1 -- 172.21.15.1:0/2385527194 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd76c0b7f00 con 0x562517afe090 2024-02-02T15:39:13.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.794+0000 7fd773000700 1 -- 172.21.15.1:0/2385527194 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd76c0a22e0 con 0x562517afe090 2024-02-02T15:39:13.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.795+0000 7fd797a4cb80 1 -- 172.21.15.1:0/2385527194 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562517afe090 msgr2=0x562517b0ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:13.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.795+0000 7fd797a4cb80 1 --2- 172.21.15.1:0/2385527194 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562517afe090 0x562517b0ec40 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fd76c09a720 tx=0x7fd76c00b4b0 comp rx=0 tx=0).stop 2024-02-02T15:39:13.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.795+0000 7fd797a4cb80 1 -- 172.21.15.1:0/2385527194 shutdown_connections 2024-02-02T15:39:13.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.795+0000 7fd797a4cb80 1 --2- 172.21.15.1:0/2385527194 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562517b09860 0x562517b14230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:13.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.795+0000 7fd797a4cb80 1 --2- 172.21.15.1:0/2385527194 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562517afe090 0x562517b0ec40 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:13.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.795+0000 7fd797a4cb80 1 --2- 172.21.15.1:0/2385527194 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562517afe9b0 0x562517b0a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:13.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.795+0000 7fd797a4cb80 1 -- 172.21.15.1:0/2385527194 >> 172.21.15.1:0/2385527194 conn(0x562517a5de30 msgr2=0x562517b09c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:13.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.795+0000 7fd797a4cb80 1 -- 172.21.15.1:0/2385527194 shutdown_connections 2024-02-02T15:39:13.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.795+0000 7fd797a4cb80 1 -- 172.21.15.1:0/2385527194 wait complete. 2024-02-02T15:39:13.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.795+0000 7fd797a4cb80 1 -- start start 2024-02-02T15:39:13.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.796+0000 7fd797a4cb80 1 Processor -- start 2024-02-02T15:39:13.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.796+0000 7fd797a4cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562517afe090 0x562517b0bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:13.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.796+0000 7fd797a4cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562517afe9b0 0x562517b0c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:13.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.796+0000 7fd797a4cb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562517b09860 0x562517b6a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:13.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.796+0000 7fd797a4cb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x562517b0cad0 con 0x562517b09860 2024-02-02T15:39:13.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.796+0000 7fd797a4cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x562517b6a9a0 con 0x562517afe9b0 2024-02-02T15:39:13.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.796+0000 7fd797a4cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x562517b6aae0 con 0x562517afe090 2024-02-02T15:39:13.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.796+0000 7fd797a4cb80 10 client.0 _ll_register_callbacks cb 0x5625179ed310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:39:13.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.796+0000 7fd781600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562517afe090 0x562517b0bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:13.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.796+0000 7fd781600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562517afe090 0x562517b0bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:37596/0 (socket says 172.21.15.1:37596) 2024-02-02T15:39:13.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.796+0000 7fd780c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562517afe9b0 0x562517b0c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:13.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.796+0000 7fd781600700 1 -- 172.21.15.1:0/988363338 learned_addr learned my addr 172.21.15.1:0/988363338 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:39:13.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.796+0000 7fd782000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562517b09860 0x562517b6a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:13.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.797+0000 7fd780c00700 1 -- 172.21.15.1:0/988363338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562517afe090 msgr2=0x562517b0bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:13.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.797+0000 7fd780c00700 1 --2- 172.21.15.1:0/988363338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562517afe090 0x562517b0bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:13.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.797+0000 7fd780c00700 1 -- 172.21.15.1:0/988363338 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562517b09860 msgr2=0x562517b6a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:13.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.797+0000 7fd780c00700 1 --2- 172.21.15.1:0/988363338 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562517b09860 0x562517b6a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:13.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.797+0000 7fd780c00700 1 -- 172.21.15.1:0/988363338 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56251787dbe0 con 0x562517afe9b0 2024-02-02T15:39:13.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.797+0000 7fd781600700 1 --2- 172.21.15.1:0/988363338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562517afe090 0x562517b0bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:39:13.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.797+0000 7fd780c00700 1 --2- 172.21.15.1:0/988363338 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562517afe9b0 0x562517b0c190 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x562517b1b640 tx=0x7fd76401fcb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b1a43fda664ba01 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:13.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.797+0000 7fd76aa00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:39:13.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.797+0000 7fd76aa00700 1 -- 172.21.15.1:0/988363338 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd764023030 con 0x562517afe9b0 2024-02-02T15:39:13.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.797+0000 7fd797a4cb80 1 -- 172.21.15.1:0/988363338 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd76c09a0f0 con 0x562517afe9b0 2024-02-02T15:39:13.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.797+0000 7fd76aa00700 1 -- 172.21.15.1:0/988363338 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9541+0+0 (secure 0 0 0) 0x7fd764024070 con 0x562517afe9b0 2024-02-02T15:39:13.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.798+0000 7fd76aa00700 1 -- 172.21.15.1:0/988363338 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd764010970 con 0x562517afe9b0 2024-02-02T15:39:13.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.798+0000 7fd797a4cb80 10 client.28919 fetch_fsmap learned FSMap version 52 2024-02-02T15:39:13.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.798+0000 7fd797a4cb80 1 -- 172.21.15.1:0/988363338 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x5625178d5d50 con 0x562517afe9b0 2024-02-02T15:39:13.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.798+0000 7fd76aa00700 1 -- 172.21.15.1:0/988363338 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd76c09a0f0 con 0x562517afe9b0 2024-02-02T15:39:13.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.798+0000 7fd76aa00700 1 -- 172.21.15.1:0/988363338 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 52) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fd764003680 con 0x562517afe9b0 2024-02-02T15:39:13.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.798+0000 7fd797a4cb80 10 client.28919 fetch_fsmap finished waiting for FSMap version 52 2024-02-02T15:39:13.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.798+0000 7fd797a4cb80 10 client.28919 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:39:13.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.798+0000 7fd797a4cb80 20 client.28919 populate_metadata read hostname 'smithi001' 2024-02-02T15:39:13.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.798+0000 7fd797a4cb80 1 --2- 172.21.15.1:0/988363338 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x562517b6c2c0 0x562517b6e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:13.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.798+0000 7fd797a4cb80 4 client.28919 mds_command: new command op to 24557 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-02T15:39:13.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.798+0000 7fd797a4cb80 1 -- 172.21.15.1:0/988363338 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x562517a0efd0 con 0x562517b6c2c0 2024-02-02T15:39:13.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.798+0000 7fd781600700 1 --2- 172.21.15.1:0/988363338 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x562517b6c2c0 0x562517b6e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:13.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.799+0000 7fd781600700 1 --2- 172.21.15.1:0/988363338 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x562517b6c2c0 0x562517b6e6e0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2cfe724a360998d7 server_cookie=64432575d795a38d in_seq=0 out_seq=0 2024-02-02T15:39:13.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.799+0000 7fd76aa00700 10 client.28919 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:39:13.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.800+0000 7fd76aa00700 1 -- 172.21.15.1:0/988363338 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x562517a0efd0 con 0x562517b6c2c0 2024-02-02T15:39:13.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.800+0000 7fd76aa00700 10 client.28919 handle_command_reply: tid=0 2024-02-02T15:39:13.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.800+0000 7fd797a4cb80 1 client.28919 shutdown 2024-02-02T15:39:13.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.800+0000 7fd797a4cb80 1 -- 172.21.15.1:0/988363338 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562517afe9b0 msgr2=0x562517b0c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:13.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.800+0000 7fd797a4cb80 1 --2- 172.21.15.1:0/988363338 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562517afe9b0 0x562517b0c190 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x562517b1b640 tx=0x7fd76401fcb0 comp rx=0 tx=0).stop 2024-02-02T15:39:13.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.800+0000 7fd797a4cb80 1 -- 172.21.15.1:0/988363338 shutdown_connections 2024-02-02T15:39:13.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.800+0000 7fd797a4cb80 1 --2- 172.21.15.1:0/988363338 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x562517b6c2c0 0x562517b6e6e0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:13.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.800+0000 7fd797a4cb80 1 --2- 172.21.15.1:0/988363338 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562517b09860 0x562517b6a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:13.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.800+0000 7fd797a4cb80 1 --2- 172.21.15.1:0/988363338 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562517afe090 0x562517b0bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:13.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.800+0000 7fd76aa00700 0 client.28919 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:39:13.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.800+0000 7fd797a4cb80 1 --2- 172.21.15.1:0/988363338 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562517afe9b0 0x562517b0c190 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:13.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.800+0000 7fd797a4cb80 1 -- 172.21.15.1:0/988363338 >> 172.21.15.1:0/988363338 conn(0x562517a5de30 msgr2=0x562517b00820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:13.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.800+0000 7fd797a4cb80 1 -- 172.21.15.1:0/988363338 shutdown_connections 2024-02-02T15:39:13.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.800+0000 7fd797a4cb80 1 -- 172.21.15.1:0/988363338 wait complete. 2024-02-02T15:39:13.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.800+0000 7fd797a4cb80 20 client.28919 trim_cache size 0 max 16384 2024-02-02T15:39:13.805 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:39:13.805 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2024-02-02T15:39:13.805 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "544a93f4-3aef-43f3-b62b-63b33472beea", 2024-02-02T15:39:13.805 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2024-02-02T15:39:13.805 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:39:13.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.801+0000 7fd784000700 1 -- 172.21.15.1:0/1536362369 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fd75c0a0690 msgr2=0x7fd75c0a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:13.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.801+0000 7fd784000700 1 --2- 172.21.15.1:0/1536362369 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fd75c0a0690 0x7fd75c0a2b50 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fd77400f870 tx=0x7fd77400fc30 comp rx=0 tx=0).stop 2024-02-02T15:39:13.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.801+0000 7fd784000700 1 -- 172.21.15.1:0/1536362369 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd77c073170 msgr2=0x7fd77c199110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:13.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.801+0000 7fd784000700 1 --2- 172.21.15.1:0/1536362369 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd77c073170 0x7fd77c199110 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fd76c005910 tx=0x7fd76c005720 comp rx=0 tx=0).stop 2024-02-02T15:39:13.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.801+0000 7fd784000700 1 -- 172.21.15.1:0/1536362369 shutdown_connections 2024-02-02T15:39:13.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.801+0000 7fd784000700 1 --2- 172.21.15.1:0/1536362369 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fd75c0a0690 0x7fd75c0a2b50 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:13.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.801+0000 7fd784000700 1 --2- 172.21.15.1:0/1536362369 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd77c10d5c0 0x7fd77c19dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:13.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.801+0000 7fd784000700 1 --2- 172.21.15.1:0/1536362369 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd77c073170 0x7fd77c199110 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:13.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.801+0000 7fd784000700 1 --2- 172.21.15.1:0/1536362369 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd77c073a90 0x7fd77c199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:13.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.801+0000 7fd784000700 1 -- 172.21.15.1:0/1536362369 >> 172.21.15.1:0/1536362369 conn(0x7fd77c0fc810 msgr2=0x7fd77c10f790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:13.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.801+0000 7fd784000700 1 -- 172.21.15.1:0/1536362369 shutdown_connections 2024-02-02T15:39:13.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:13.801+0000 7fd784000700 1 -- 172.21.15.1:0/1536362369 wait complete. 2024-02-02T15:39:13.818 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:39:13 smithi001 ceph-mon[35470]: pgmap v380: 129 pgs: 129 active+clean; 557 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 184 op/s 2024-02-02T15:39:13.821 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-02T15:39:13.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:39:13 smithi071 ceph-mon[44707]: pgmap v380: 129 pgs: 129 active+clean; 557 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 184 op/s 2024-02-02T15:39:13.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:39:13 smithi142 ceph-mon[45728]: pgmap v380: 129 pgs: 129 active+clean; 557 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 184 op/s 2024-02-02T15:39:14.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.049+0000 7fa31fa00700 1 -- 172.21.15.1:0/2976707937 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa318073170 msgr2=0x7fa318073550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:14.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.049+0000 7fa31fa00700 1 --2- 172.21.15.1:0/2976707937 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa318073170 0x7fa318073550 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fa308009990 tx=0x7fa308009ca0 comp rx=0 tx=0).stop 2024-02-02T15:39:14.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.049+0000 7fa31fa00700 1 -- 172.21.15.1:0/2976707937 shutdown_connections 2024-02-02T15:39:14.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.049+0000 7fa31fa00700 1 --2- 172.21.15.1:0/2976707937 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa31810d5c0 0x7fa31810fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:14.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.049+0000 7fa31fa00700 1 --2- 172.21.15.1:0/2976707937 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa318073170 0x7fa318073550 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:14.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.049+0000 7fa31fa00700 1 --2- 172.21.15.1:0/2976707937 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa318073a90 0x7fa31810d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:14.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.049+0000 7fa31fa00700 1 -- 172.21.15.1:0/2976707937 >> 172.21.15.1:0/2976707937 conn(0x7fa3180fc810 msgr2=0x7fa3180fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:14.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.050+0000 7fa31fa00700 1 -- 172.21.15.1:0/2976707937 shutdown_connections 2024-02-02T15:39:14.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.050+0000 7fa31fa00700 1 -- 172.21.15.1:0/2976707937 wait complete. 2024-02-02T15:39:14.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.051+0000 7fa31fa00700 1 Processor -- start 2024-02-02T15:39:14.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.051+0000 7fa31fa00700 1 -- start start 2024-02-02T15:39:14.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.051+0000 7fa31fa00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa318073170 0x7fa318199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:14.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.051+0000 7fa31fa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa318073a90 0x7fa3181996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:14.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.051+0000 7fa31fa00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa31810d5c0 0x7fa31819db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:14.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.051+0000 7fa31fa00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa31819e170 con 0x7fa318073a90 2024-02-02T15:39:14.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.051+0000 7fa31fa00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa31819e2e0 con 0x7fa318073170 2024-02-02T15:39:14.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.051+0000 7fa31fa00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa31819e4a0 con 0x7fa31810d5c0 2024-02-02T15:39:14.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.051+0000 7fa31d000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa318073170 0x7fa318199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:14.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.051+0000 7fa31d000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa318073170 0x7fa318199190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:34146/0 (socket says 172.21.15.1:34146) 2024-02-02T15:39:14.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.051+0000 7fa317e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa318073a90 0x7fa3181996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:14.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.051+0000 7fa31d000700 1 -- 172.21.15.1:0/3233550159 learned_addr learned my addr 172.21.15.1:0/3233550159 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:39:14.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.052+0000 7fa31da00700 1 --2- 172.21.15.1:0/3233550159 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa31810d5c0 0x7fa31819db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:14.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.052+0000 7fa31d000700 1 -- 172.21.15.1:0/3233550159 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa31810d5c0 msgr2=0x7fa31819db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:14.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.052+0000 7fa31d000700 1 --2- 172.21.15.1:0/3233550159 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa31810d5c0 0x7fa31819db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:14.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.052+0000 7fa31d000700 1 -- 172.21.15.1:0/3233550159 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa318073a90 msgr2=0x7fa3181996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:14.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.052+0000 7fa31d000700 1 --2- 172.21.15.1:0/3233550159 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa318073a90 0x7fa3181996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:14.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.052+0000 7fa31d000700 1 -- 172.21.15.1:0/3233550159 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3080095f0 con 0x7fa318073170 2024-02-02T15:39:14.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.052+0000 7fa31d000700 1 --2- 172.21.15.1:0/3233550159 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa318073170 0x7fa318199190 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fa308005910 tx=0x7fa308005720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:14.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.052+0000 7fa315600700 1 -- 172.21.15.1:0/3233550159 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa30801d070 con 0x7fa318073170 2024-02-02T15:39:14.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.053+0000 7fa31fa00700 1 -- 172.21.15.1:0/3233550159 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa31819e720 con 0x7fa318073170 2024-02-02T15:39:14.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.053+0000 7fa315600700 1 -- 172.21.15.1:0/3233550159 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa308022470 con 0x7fa318073170 2024-02-02T15:39:14.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.053+0000 7fa315600700 1 -- 172.21.15.1:0/3233550159 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa308017650 con 0x7fa318073170 2024-02-02T15:39:14.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.053+0000 7fa31fa00700 1 -- 172.21.15.1:0/3233550159 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3181a5fc0 con 0x7fa318073170 2024-02-02T15:39:14.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.056+0000 7fa315600700 1 -- 172.21.15.1:0/3233550159 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7fa3080225e0 con 0x7fa318073170 2024-02-02T15:39:14.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.057+0000 7fa315600700 1 --2- 172.21.15.1:0/3233550159 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fa2f80a0690 0x7fa2f80a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:14.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.057+0000 7fa33358ab80 1 Processor -- start 2024-02-02T15:39:14.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.057+0000 7fa315600700 1 -- 172.21.15.1:0/3233550159 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 9541+0+0 (secure 0 0 0) 0x7fa3080cd500 con 0x7fa318073170 2024-02-02T15:39:14.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.057+0000 7fa317e00700 1 --2- 172.21.15.1:0/3233550159 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fa2f80a0690 0x7fa2f80a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:14.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.058+0000 7fa33358ab80 1 -- start start 2024-02-02T15:39:14.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.058+0000 7fa33358ab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561bd68e5010 0x561bd68e53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:14.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.058+0000 7fa33358ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bd68e5930 0x561bd68f02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:14.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.058+0000 7fa33358ab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561bd68f07e0 0x561bd68f2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:14.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.058+0000 7fa33358ab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bd68594c0 con 0x561bd68e5010 2024-02-02T15:39:14.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.058+0000 7fa33358ab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bd683daa0 con 0x561bd68f07e0 2024-02-02T15:39:14.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.058+0000 7fa33358ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bd683b460 con 0x561bd68e5930 2024-02-02T15:39:14.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.058+0000 7fa317e00700 1 --2- 172.21.15.1:0/3233550159 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fa2f80a0690 0x7fa2f80a2b50 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa31819a7a0 tx=0x7fa300009260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:14.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.058+0000 7fa317e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bd68e5930 0x561bd68f02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:14.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.058+0000 7fa31d000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561bd68e5010 0x561bd68e53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:14.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.058+0000 7fa317e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bd68e5930 0x561bd68f02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:37620/0 (socket says 172.21.15.1:37620) 2024-02-02T15:39:14.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.058+0000 7fa317e00700 1 -- 172.21.15.1:0/2402863429 learned_addr learned my addr 172.21.15.1:0/2402863429 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:39:14.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.059+0000 7fa31da00700 1 --2- 172.21.15.1:0/2402863429 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561bd68f07e0 0x561bd68f2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:14.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.059+0000 7fa317e00700 1 -- 172.21.15.1:0/2402863429 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561bd68f07e0 msgr2=0x561bd68f2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:14.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.059+0000 7fa317e00700 1 --2- 172.21.15.1:0/2402863429 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561bd68f07e0 0x561bd68f2bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:14.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.059+0000 7fa317e00700 1 -- 172.21.15.1:0/2402863429 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561bd68e5010 msgr2=0x561bd68e53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:14.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.059+0000 7fa317e00700 1 --2- 172.21.15.1:0/2402863429 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561bd68e5010 0x561bd68e53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:14.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.059+0000 7fa317e00700 1 -- 172.21.15.1:0/2402863429 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561bd66b83b0 con 0x561bd68e5930 2024-02-02T15:39:14.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.059+0000 7fa31d000700 1 --2- 172.21.15.1:0/2402863429 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561bd68e5010 0x561bd68e53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:39:14.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.059+0000 7fa317e00700 1 --2- 172.21.15.1:0/2402863429 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bd68e5930 0x561bd68f02a0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fa300011d50 tx=0x7fa300014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51e1b97d2d2befb8 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:14.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.059+0000 7fa30e000700 1 -- 172.21.15.1:0/2402863429 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa30000eb20 con 0x561bd68e5930 2024-02-02T15:39:14.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.059+0000 7fa30e000700 1 -- 172.21.15.1:0/2402863429 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa300021070 con 0x561bd68e5930 2024-02-02T15:39:14.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.059+0000 7fa30e000700 1 -- 172.21.15.1:0/2402863429 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa300025630 con 0x561bd68e5930 2024-02-02T15:39:14.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.060+0000 7fa33358ab80 1 -- 172.21.15.1:0/2402863429 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bd68e5930 msgr2=0x561bd68f02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:14.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.060+0000 7fa33358ab80 1 --2- 172.21.15.1:0/2402863429 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bd68e5930 0x561bd68f02a0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fa300011d50 tx=0x7fa300014000 comp rx=0 tx=0).stop 2024-02-02T15:39:14.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.060+0000 7fa33358ab80 1 -- 172.21.15.1:0/2402863429 shutdown_connections 2024-02-02T15:39:14.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.060+0000 7fa33358ab80 1 --2- 172.21.15.1:0/2402863429 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561bd68f07e0 0x561bd68f2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:14.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.060+0000 7fa33358ab80 1 --2- 172.21.15.1:0/2402863429 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561bd68e5010 0x561bd68e53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:14.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.060+0000 7fa33358ab80 1 --2- 172.21.15.1:0/2402863429 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bd68e5930 0x561bd68f02a0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:14.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.060+0000 7fa33358ab80 1 -- 172.21.15.1:0/2402863429 >> 172.21.15.1:0/2402863429 conn(0x561bd6844db0 msgr2=0x561bd68e13b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:14.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.060+0000 7fa33358ab80 1 -- 172.21.15.1:0/2402863429 shutdown_connections 2024-02-02T15:39:14.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.060+0000 7fa33358ab80 1 -- 172.21.15.1:0/2402863429 wait complete. 2024-02-02T15:39:14.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.061+0000 7fa33358ab80 1 -- start start 2024-02-02T15:39:14.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.061+0000 7fa33358ab80 1 Processor -- start 2024-02-02T15:39:14.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.062+0000 7fa33358ab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561bd68e5010 0x561bd68f01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:14.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.062+0000 7fa33358ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bd68e5930 0x561bd68ed450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:14.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.062+0000 7fa33358ab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561bd68f07e0 0x561bd68eda60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:14.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.062+0000 7fa33358ab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bd6836d50 con 0x561bd68e5010 2024-02-02T15:39:14.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.062+0000 7fa33358ab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bd67d2d80 con 0x561bd68f07e0 2024-02-02T15:39:14.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.062+0000 7fa33358ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bd67d2a20 con 0x561bd68e5930 2024-02-02T15:39:14.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.062+0000 7fa33358ab80 10 client.0 _ll_register_callbacks cb 0x561bd67d4290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:39:14.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.062+0000 7fa317e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bd68e5930 0x561bd68ed450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:14.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.062+0000 7fa31d000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561bd68e5010 0x561bd68f01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:14.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.062+0000 7fa31da00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561bd68f07e0 0x561bd68eda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:14.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.062+0000 7fa31da00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561bd68f07e0 0x561bd68eda60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:34168/0 (socket says 172.21.15.1:34168) 2024-02-02T15:39:14.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.062+0000 7fa31da00700 1 -- 172.21.15.1:0/2514639125 learned_addr learned my addr 172.21.15.1:0/2514639125 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:39:14.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.062+0000 7fa31da00700 1 -- 172.21.15.1:0/2514639125 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bd68e5930 msgr2=0x561bd68ed450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:14.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.062+0000 7fa31da00700 1 --2- 172.21.15.1:0/2514639125 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bd68e5930 0x561bd68ed450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:14.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.062+0000 7fa31da00700 1 -- 172.21.15.1:0/2514639125 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561bd68e5010 msgr2=0x561bd68f01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:14.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.062+0000 7fa31da00700 1 --2- 172.21.15.1:0/2514639125 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561bd68e5010 0x561bd68f01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:14.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.062+0000 7fa31da00700 1 -- 172.21.15.1:0/2514639125 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3181a5fc0 con 0x561bd68f07e0 2024-02-02T15:39:14.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.062+0000 7fa31d000700 1 --2- 172.21.15.1:0/2514639125 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561bd68e5010 0x561bd68f01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:39:14.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.062+0000 7fa317e00700 1 --2- 172.21.15.1:0/2514639125 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bd68e5930 0x561bd68ed450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:39:14.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.063+0000 7fa31da00700 1 --2- 172.21.15.1:0/2514639125 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561bd68f07e0 0x561bd68eda60 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fa310011f60 tx=0x7fa310014600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9773a82ef681117b server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:14.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.063+0000 7fa306a00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:39:14.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.063+0000 7fa306a00700 1 -- 172.21.15.1:0/2514639125 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa31000e040 con 0x561bd68f07e0 2024-02-02T15:39:14.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.063+0000 7fa306a00700 1 -- 172.21.15.1:0/2514639125 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9541+0+0 (secure 0 0 0) 0x7fa310014b40 con 0x561bd68f07e0 2024-02-02T15:39:14.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.063+0000 7fa33358ab80 1 -- 172.21.15.1:0/2514639125 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561bd68d90f0 con 0x561bd68f07e0 2024-02-02T15:39:14.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.063+0000 7fa306a00700 1 -- 172.21.15.1:0/2514639125 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa310014d00 con 0x561bd68f07e0 2024-02-02T15:39:14.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.063+0000 7fa33358ab80 10 client.28931 fetch_fsmap learned FSMap version 52 2024-02-02T15:39:14.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.063+0000 7fa33358ab80 1 -- 172.21.15.1:0/2514639125 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x7fa300017070 con 0x561bd68f07e0 2024-02-02T15:39:14.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.064+0000 7fa306a00700 1 -- 172.21.15.1:0/2514639125 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561bd68d90f0 con 0x561bd68f07e0 2024-02-02T15:39:14.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.064+0000 7fa306a00700 1 -- 172.21.15.1:0/2514639125 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 52) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fa31001c020 con 0x561bd68f07e0 2024-02-02T15:39:14.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.064+0000 7fa33358ab80 10 client.28931 fetch_fsmap finished waiting for FSMap version 52 2024-02-02T15:39:14.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.064+0000 7fa33358ab80 10 client.28931 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:39:14.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.064+0000 7fa33358ab80 20 client.28931 populate_metadata read hostname 'smithi001' 2024-02-02T15:39:14.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.064+0000 7fa33358ab80 1 --2- 172.21.15.1:0/2514639125 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x561bd6900270 0x561bd6902690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:14.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.064+0000 7fa33358ab80 4 client.28931 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:39:14.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.064+0000 7fa33358ab80 1 -- 172.21.15.1:0/2514639125 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561bd682fdd0 con 0x561bd6900270 2024-02-02T15:39:14.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.064+0000 7fa31d000700 1 --2- 172.21.15.1:0/2514639125 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x561bd6900270 0x561bd6902690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:14.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.065+0000 7fa31d000700 1 --2- 172.21.15.1:0/2514639125 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x561bd6900270 0x561bd6902690 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba447639e04ca705 server_cookie=677fad81c0b31e57 in_seq=0 out_seq=0 2024-02-02T15:39:14.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.065+0000 7fa306a00700 10 client.28931 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:39:14.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.065+0000 7fa306a00700 1 -- 172.21.15.1:0/2514639125 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x561bd682fdd0 con 0x561bd6900270 2024-02-02T15:39:14.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.065+0000 7fa306a00700 10 client.28931 handle_command_reply: tid=0 2024-02-02T15:39:14.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.065+0000 7fa33358ab80 1 client.28931 shutdown 2024-02-02T15:39:14.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.065+0000 7fa33358ab80 1 -- 172.21.15.1:0/2514639125 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561bd68f07e0 msgr2=0x561bd68eda60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:14.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.065+0000 7fa33358ab80 1 --2- 172.21.15.1:0/2514639125 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561bd68f07e0 0x561bd68eda60 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fa310011f60 tx=0x7fa310014600 comp rx=0 tx=0).stop 2024-02-02T15:39:14.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.065+0000 7fa33358ab80 1 -- 172.21.15.1:0/2514639125 shutdown_connections 2024-02-02T15:39:14.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.066+0000 7fa33358ab80 1 --2- 172.21.15.1:0/2514639125 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x561bd6900270 0x561bd6902690 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:14.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.066+0000 7fa33358ab80 1 --2- 172.21.15.1:0/2514639125 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561bd68f07e0 0x561bd68eda60 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:14.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.066+0000 7fa306a00700 0 client.28931 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:39:14.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.066+0000 7fa33358ab80 1 --2- 172.21.15.1:0/2514639125 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561bd68e5010 0x561bd68f01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:14.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.066+0000 7fa33358ab80 1 --2- 172.21.15.1:0/2514639125 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bd68e5930 0x561bd68ed450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:14.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.066+0000 7fa33358ab80 1 -- 172.21.15.1:0/2514639125 >> 172.21.15.1:0/2514639125 conn(0x561bd6844db0 msgr2=0x561bd68e0fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:14.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.066+0000 7fa33358ab80 1 -- 172.21.15.1:0/2514639125 shutdown_connections 2024-02-02T15:39:14.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.066+0000 7fa33358ab80 1 -- 172.21.15.1:0/2514639125 wait complete. 2024-02-02T15:39:14.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.066+0000 7fa33358ab80 20 client.28931 trim_cache size 0 max 16384 2024-02-02T15:39:14.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.085+0000 7fa33358ab80 1 Processor -- start 2024-02-02T15:39:14.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.085+0000 7fa33358ab80 1 -- start start 2024-02-02T15:39:14.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.085+0000 7fa33358ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bd68e5010 0x561bd68f7360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:14.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.085+0000 7fa33358ab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561bd68e5930 0x561bd68ec310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:14.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.085+0000 7fa33358ab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561bd68f07e0 0x561bd68efd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:14.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.085+0000 7fa33358ab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bd66ba6a0 con 0x561bd68f07e0 2024-02-02T15:39:14.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.085+0000 7fa33358ab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bd68ee7a0 con 0x561bd68e5930 2024-02-02T15:39:14.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.085+0000 7fa33358ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bd68fd940 con 0x561bd68e5010 2024-02-02T15:39:14.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.086+0000 7fa317e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561bd68e5930 0x561bd68ec310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:14.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.086+0000 7fa31d000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bd68e5010 0x561bd68f7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:14.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.086+0000 7fa31da00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561bd68f07e0 0x561bd68efd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:14.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.086+0000 7fa31d000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bd68e5010 0x561bd68f7360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:37634/0 (socket says 172.21.15.1:37634) 2024-02-02T15:39:14.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.086+0000 7fa317e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561bd68e5930 0x561bd68ec310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:34174/0 (socket says 172.21.15.1:34174) 2024-02-02T15:39:14.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.086+0000 7fa31d000700 1 -- 172.21.15.1:0/1394296714 learned_addr learned my addr 172.21.15.1:0/1394296714 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:39:14.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.086+0000 7fa31d000700 1 -- 172.21.15.1:0/1394296714 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561bd68e5930 msgr2=0x561bd68ec310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:14.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.086+0000 7fa31d000700 1 --2- 172.21.15.1:0/1394296714 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561bd68e5930 0x561bd68ec310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:14.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.086+0000 7fa31d000700 1 -- 172.21.15.1:0/1394296714 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561bd68f07e0 msgr2=0x561bd68efd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:14.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.086+0000 7fa31d000700 1 --2- 172.21.15.1:0/1394296714 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561bd68f07e0 0x561bd68efd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:14.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.086+0000 7fa31d000700 1 -- 172.21.15.1:0/1394296714 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561bd6664b60 con 0x561bd68e5010 2024-02-02T15:39:14.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.086+0000 7fa317e00700 1 --2- 172.21.15.1:0/1394296714 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561bd68e5930 0x561bd68ec310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:39:14.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.086+0000 7fa31d000700 1 --2- 172.21.15.1:0/1394296714 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bd68e5010 0x561bd68f7360 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fa308096b50 tx=0x7fa30801a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86a1fe2bcfe6941 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:14.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.087+0000 7fa30ea00700 1 -- 172.21.15.1:0/1394296714 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa308099fb0 con 0x561bd68e5010 2024-02-02T15:39:14.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.087+0000 7fa30ea00700 1 -- 172.21.15.1:0/1394296714 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa308096990 con 0x561bd68e5010 2024-02-02T15:39:14.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.087+0000 7fa30ea00700 1 -- 172.21.15.1:0/1394296714 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa308098a80 con 0x561bd68e5010 2024-02-02T15:39:14.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.087+0000 7fa33358ab80 1 -- 172.21.15.1:0/1394296714 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bd68e5010 msgr2=0x561bd68f7360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:14.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.087+0000 7fa33358ab80 1 --2- 172.21.15.1:0/1394296714 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bd68e5010 0x561bd68f7360 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fa308096b50 tx=0x7fa30801a040 comp rx=0 tx=0).stop 2024-02-02T15:39:14.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.087+0000 7fa33358ab80 1 -- 172.21.15.1:0/1394296714 shutdown_connections 2024-02-02T15:39:14.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.087+0000 7fa33358ab80 1 --2- 172.21.15.1:0/1394296714 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561bd68f07e0 0x561bd68efd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:14.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.087+0000 7fa33358ab80 1 --2- 172.21.15.1:0/1394296714 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bd68e5010 0x561bd68f7360 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:14.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.087+0000 7fa33358ab80 1 --2- 172.21.15.1:0/1394296714 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561bd68e5930 0x561bd68ec310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:14.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.088+0000 7fa33358ab80 1 -- 172.21.15.1:0/1394296714 >> 172.21.15.1:0/1394296714 conn(0x561bd6844db0 msgr2=0x561bd68ebc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:14.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.088+0000 7fa33358ab80 1 -- 172.21.15.1:0/1394296714 shutdown_connections 2024-02-02T15:39:14.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.088+0000 7fa33358ab80 1 -- 172.21.15.1:0/1394296714 wait complete. 2024-02-02T15:39:14.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.088+0000 7fa33358ab80 1 -- start start 2024-02-02T15:39:14.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.088+0000 7fa33358ab80 1 Processor -- start 2024-02-02T15:39:14.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.089+0000 7fa33358ab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561bd68e5010 0x561bd68f3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:14.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.089+0000 7fa33358ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bd68e5930 0x561bd68f3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:14.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.089+0000 7fa33358ab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561bd68f07e0 0x561bd694adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:14.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.089+0000 7fa33358ab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bd694b300 con 0x561bd68e5010 2024-02-02T15:39:14.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.089+0000 7fa33358ab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bd694b440 con 0x561bd68f07e0 2024-02-02T15:39:14.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.089+0000 7fa33358ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bd694b600 con 0x561bd68e5930 2024-02-02T15:39:14.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.089+0000 7fa33358ab80 10 client.0 _ll_register_callbacks cb 0x561bd67d4290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:39:14.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.089+0000 7fa317e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bd68e5930 0x561bd68f3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:14.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.089+0000 7fa31da00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561bd68f07e0 0x561bd694adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:14.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.089+0000 7fa317e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bd68e5930 0x561bd68f3a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:37644/0 (socket says 172.21.15.1:37644) 2024-02-02T15:39:14.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.089+0000 7fa317e00700 1 -- 172.21.15.1:0/2411566229 learned_addr learned my addr 172.21.15.1:0/2411566229 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:39:14.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.089+0000 7fa31d000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561bd68e5010 0x561bd68f3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:14.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.089+0000 7fa31da00700 1 -- 172.21.15.1:0/2411566229 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bd68e5930 msgr2=0x561bd68f3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:14.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.089+0000 7fa31da00700 1 --2- 172.21.15.1:0/2411566229 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bd68e5930 0x561bd68f3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:14.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.089+0000 7fa31da00700 1 -- 172.21.15.1:0/2411566229 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561bd68e5010 msgr2=0x561bd68f3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:14.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.089+0000 7fa31da00700 1 --2- 172.21.15.1:0/2411566229 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561bd68e5010 0x561bd68f3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:14.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.089+0000 7fa31da00700 1 -- 172.21.15.1:0/2411566229 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561bd66b83b0 con 0x561bd68f07e0 2024-02-02T15:39:14.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.090+0000 7fa31da00700 1 --2- 172.21.15.1:0/2411566229 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561bd68f07e0 0x561bd694adc0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fa310004720 tx=0x7fa310009510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4f73d3033cafcdf server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:14.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.090+0000 7fa306a00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:39:14.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.090+0000 7fa306a00700 1 -- 172.21.15.1:0/2411566229 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa31000e040 con 0x561bd68f07e0 2024-02-02T15:39:14.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.090+0000 7fa33358ab80 1 -- 172.21.15.1:0/2411566229 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa300021070 con 0x561bd68f07e0 2024-02-02T15:39:14.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.090+0000 7fa306a00700 1 -- 172.21.15.1:0/2411566229 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 9541+0+0 (secure 0 0 0) 0x7fa3100049e0 con 0x561bd68f07e0 2024-02-02T15:39:14.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.090+0000 7fa306a00700 1 -- 172.21.15.1:0/2411566229 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa310004bc0 con 0x561bd68f07e0 2024-02-02T15:39:14.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.090+0000 7fa33358ab80 10 client.28937 fetch_fsmap learned FSMap version 52 2024-02-02T15:39:14.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.090+0000 7fa33358ab80 1 -- 172.21.15.1:0/2411566229 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x7fa31819e720 con 0x561bd68f07e0 2024-02-02T15:39:14.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.091+0000 7fa306a00700 1 -- 172.21.15.1:0/2411566229 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa300021070 con 0x561bd68f07e0 2024-02-02T15:39:14.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.091+0000 7fa306a00700 1 -- 172.21.15.1:0/2411566229 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 52) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fa310014320 con 0x561bd68f07e0 2024-02-02T15:39:14.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.091+0000 7fa33358ab80 10 client.28937 fetch_fsmap finished waiting for FSMap version 52 2024-02-02T15:39:14.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.091+0000 7fa33358ab80 10 client.28937 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:39:14.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.091+0000 7fa33358ab80 20 client.28937 populate_metadata read hostname 'smithi001' 2024-02-02T15:39:14.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.091+0000 7fa33358ab80 1 --2- 172.21.15.1:0/2411566229 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x561bd6900270 0x561bd6954e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:14.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.091+0000 7fa33358ab80 4 client.28937 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2024-02-02T15:39:14.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.091+0000 7fa33358ab80 1 -- 172.21.15.1:0/2411566229 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561bd67f5f50 con 0x561bd6900270 2024-02-02T15:39:14.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.091+0000 7fa31d000700 1 --2- 172.21.15.1:0/2411566229 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x561bd6900270 0x561bd6954e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:14.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.092+0000 7fa31d000700 1 --2- 172.21.15.1:0/2411566229 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x561bd6900270 0x561bd6954e40 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4739e1aab536ca71 server_cookie=e66c19f7ec8d282e in_seq=0 out_seq=0 2024-02-02T15:39:14.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.092+0000 7fa306a00700 10 client.28937 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:39:14.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.092+0000 7fa306a00700 1 -- 172.21.15.1:0/2411566229 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x561bd67f5f50 con 0x561bd6900270 2024-02-02T15:39:14.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.092+0000 7fa306a00700 10 client.28937 handle_command_reply: tid=0 2024-02-02T15:39:14.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.092+0000 7fa33358ab80 1 client.28937 shutdown 2024-02-02T15:39:14.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.092+0000 7fa33358ab80 1 -- 172.21.15.1:0/2411566229 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561bd68f07e0 msgr2=0x561bd694adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:14.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.092+0000 7fa33358ab80 1 --2- 172.21.15.1:0/2411566229 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561bd68f07e0 0x561bd694adc0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fa310004720 tx=0x7fa310009510 comp rx=0 tx=0).stop 2024-02-02T15:39:14.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.092+0000 7fa33358ab80 1 -- 172.21.15.1:0/2411566229 shutdown_connections 2024-02-02T15:39:14.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.092+0000 7fa33358ab80 1 --2- 172.21.15.1:0/2411566229 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x561bd6900270 0x561bd6954e40 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:14.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.092+0000 7fa33358ab80 1 --2- 172.21.15.1:0/2411566229 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561bd68f07e0 0x561bd694adc0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:14.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.092+0000 7fa33358ab80 1 --2- 172.21.15.1:0/2411566229 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561bd68e5010 0x561bd68f3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:14.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.092+0000 7fa33358ab80 1 --2- 172.21.15.1:0/2411566229 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561bd68e5930 0x561bd68f3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:14.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.092+0000 7fa306a00700 0 client.28937 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:39:14.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.092+0000 7fa33358ab80 1 -- 172.21.15.1:0/2411566229 >> 172.21.15.1:0/2411566229 conn(0x561bd6844db0 msgr2=0x561bd68e0e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:14.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.092+0000 7fa33358ab80 1 -- 172.21.15.1:0/2411566229 shutdown_connections 2024-02-02T15:39:14.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.092+0000 7fa33358ab80 1 -- 172.21.15.1:0/2411566229 wait complete. 2024-02-02T15:39:14.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.093+0000 7fa33358ab80 20 client.28937 trim_cache size 0 max 16384 2024-02-02T15:39:14.098 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:39:14.098 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-02T15:39:14.098 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2024-02-02T15:39:14.098 INFO:teuthology.orchestra.run.smithi001.stdout: "544a93f4-3aef-43f3-b62b-63b33472beea": { 2024-02-02T15:39:14.098 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2024-02-02T15:39:14.098 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "544a93f4-3aef-43f3-b62b-63b33472beea", 2024-02-02T15:39:14.098 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,force" 2024-02-02T15:39:14.098 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:39:14.099 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:39:14.099 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:39:14.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.093+0000 7fa31fa00700 1 -- 172.21.15.1:0/3233550159 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fa2f80a0690 msgr2=0x7fa2f80a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:14.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.093+0000 7fa31fa00700 1 --2- 172.21.15.1:0/3233550159 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fa2f80a0690 0x7fa2f80a2b50 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa31819a7a0 tx=0x7fa300009260 comp rx=0 tx=0).stop 2024-02-02T15:39:14.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.093+0000 7fa31fa00700 1 -- 172.21.15.1:0/3233550159 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa318073170 msgr2=0x7fa318199190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:14.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.093+0000 7fa31fa00700 1 --2- 172.21.15.1:0/3233550159 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa318073170 0x7fa318199190 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fa308005910 tx=0x7fa308005720 comp rx=0 tx=0).stop 2024-02-02T15:39:14.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.093+0000 7fa31fa00700 1 -- 172.21.15.1:0/3233550159 shutdown_connections 2024-02-02T15:39:14.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.093+0000 7fa31fa00700 1 --2- 172.21.15.1:0/3233550159 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fa2f80a0690 0x7fa2f80a2b50 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:14.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.093+0000 7fa31fa00700 1 --2- 172.21.15.1:0/3233550159 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa31810d5c0 0x7fa31819db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:14.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.093+0000 7fa31fa00700 1 --2- 172.21.15.1:0/3233550159 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa318073170 0x7fa318199190 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:14.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.093+0000 7fa31fa00700 1 --2- 172.21.15.1:0/3233550159 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa318073a90 0x7fa3181996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:14.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.093+0000 7fa31fa00700 1 -- 172.21.15.1:0/3233550159 >> 172.21.15.1:0/3233550159 conn(0x7fa3180fc810 msgr2=0x7fa31810f760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:14.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.093+0000 7fa31fa00700 1 -- 172.21.15.1:0/3233550159 shutdown_connections 2024-02-02T15:39:14.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:14.094+0000 7fa31fa00700 1 -- 172.21.15.1:0/3233550159 wait complete. 2024-02-02T15:39:14.113 INFO:tasks.cephfs.filesystem:scrub status for tag:544a93f4-3aef-43f3-b62b-63b33472beea - {'path': '/', 'tag': '544a93f4-3aef-43f3-b62b-63b33472beea', 'options': 'recursive,force'} 2024-02-02T15:39:14.271 INFO:tasks.workunit.client.0.smithi001.stdout: 1 670113 5.44 MB/sec execute 528 sec latency 165.930 ms 2024-02-02T15:39:14.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:39:14 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:39:14.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:39:14 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:39:14.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:39:14 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:39:14.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:39:14 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:39:14.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:39:14 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:39:14.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:39:14 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:39:14.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:39:14 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:39:14.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:39:14 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:39:14.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:39:14 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:39:14.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:39:14 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:39:14.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:39:14 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:39:14.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:39:14 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:39:14.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:39:14 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:39:14.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:39:14 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:39:14.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:39:14 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:39:14.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:39:14 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:39:14.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:39:14 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:39:14.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:39:14 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:39:14.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:39:14 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:39:14.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:39:14 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:39:14.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:39:14 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:39:14.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:39:14 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:39:14.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:39:14 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:39:14.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:39:14 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:39:15.271 INFO:tasks.workunit.client.0.smithi001.stdout: 1 671577 5.45 MB/sec execute 529 sec latency 12.571 ms 2024-02-02T15:39:15.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:39:15 smithi071 ceph-mon[44707]: scrub queued for path: / 2024-02-02T15:39:15.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:39:15 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:39:15.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:39:15 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:39:15.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:39:15 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:39:15.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:39:15 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:39:15.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:39:15 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:39:15.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:39:15 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:39:15.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:39:15 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:39:15.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:39:15 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:39:15.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:39:15 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:39:15.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:39:15 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:39:15.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:39:15 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:39:15.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:39:15 smithi071 ceph-mon[44707]: pgmap v381: 129 pgs: 129 active+clean; 557 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 184 op/s 2024-02-02T15:39:15.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:39:15 smithi142 ceph-mon[45728]: scrub queued for path: / 2024-02-02T15:39:15.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:39:15 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:39:15.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:39:15 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:39:15.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:39:15 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:39:15.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:39:15 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:39:15.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:39:15 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:39:15.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:39:15 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:39:15.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:39:15 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:39:15.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:39:15 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:39:15.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:39:15 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:39:15.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:39:15 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:39:15.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:39:15 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:39:15.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:39:15 smithi142 ceph-mon[45728]: pgmap v381: 129 pgs: 129 active+clean; 557 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 184 op/s 2024-02-02T15:39:15.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:39:15 smithi001 ceph-mon[35470]: scrub queued for path: / 2024-02-02T15:39:15.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:39:15 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:39:15.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:39:15 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:39:15.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:39:15 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:39:15.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:39:15 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:39:15.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:39:15 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:39:15.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:39:15 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:39:15.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:39:15 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:39:15.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:39:15 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:39:15.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:39:15 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:39:15.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:39:15 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:39:15.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:39:15 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:39:15.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:39:15 smithi001 ceph-mon[35470]: pgmap v381: 129 pgs: 129 active+clean; 557 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 184 op/s 2024-02-02T15:39:16.271 INFO:tasks.workunit.client.0.smithi001.stdout: 1 672658 5.45 MB/sec execute 530 sec latency 21.824 ms 2024-02-02T15:39:17.271 INFO:tasks.workunit.client.0.smithi001.stdout: 1 672856 5.44 MB/sec execute 531 sec latency 724.911 ms 2024-02-02T15:39:17.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:39:17 smithi071 ceph-mon[44707]: pgmap v382: 129 pgs: 129 active+clean; 567 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 178 op/s 2024-02-02T15:39:17.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:39:17 smithi071 ceph-mon[44707]: osdmap e101: 12 total, 12 up, 12 in 2024-02-02T15:39:17.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:39:17 smithi142 ceph-mon[45728]: pgmap v382: 129 pgs: 129 active+clean; 567 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 178 op/s 2024-02-02T15:39:17.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:39:17 smithi142 ceph-mon[45728]: osdmap e101: 12 total, 12 up, 12 in 2024-02-02T15:39:17.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:39:17 smithi001 ceph-mon[35470]: pgmap v382: 129 pgs: 129 active+clean; 567 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 178 op/s 2024-02-02T15:39:17.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:39:17 smithi001 ceph-mon[35470]: osdmap e101: 12 total, 12 up, 12 in 2024-02-02T15:39:18.271 INFO:tasks.workunit.client.0.smithi001.stdout: 1 674118 5.44 MB/sec execute 532 sec latency 1109.997 ms 2024-02-02T15:39:19.271 INFO:tasks.workunit.client.0.smithi001.stdout: 1 675327 5.44 MB/sec execute 533 sec latency 23.619 ms 2024-02-02T15:39:19.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:39:19 smithi071 ceph-mon[44707]: pgmap v384: 129 pgs: 129 active+clean; 567 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 178 op/s 2024-02-02T15:39:19.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:39:19 smithi142 ceph-mon[45728]: pgmap v384: 129 pgs: 129 active+clean; 567 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 178 op/s 2024-02-02T15:39:19.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:39:19 smithi001 ceph-mon[35470]: pgmap v384: 129 pgs: 129 active+clean; 567 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 178 op/s 2024-02-02T15:39:20.271 INFO:tasks.workunit.client.0.smithi001.stdout: 1 676406 5.44 MB/sec execute 534 sec latency 20.687 ms 2024-02-02T15:39:21.271 INFO:tasks.workunit.client.0.smithi001.stdout: 1 676458 5.43 MB/sec execute 535 sec latency 947.417 ms 2024-02-02T15:39:21.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:39:21 smithi071 ceph-mon[44707]: pgmap v385: 129 pgs: 129 active+clean; 567 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 178 op/s 2024-02-02T15:39:21.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:39:21 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:39:21.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:39:21 smithi142 ceph-mon[45728]: pgmap v385: 129 pgs: 129 active+clean; 567 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 178 op/s 2024-02-02T15:39:21.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:39:21 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:39:21.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:39:21 smithi001 ceph-mon[35470]: pgmap v385: 129 pgs: 129 active+clean; 567 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 178 op/s 2024-02-02T15:39:21.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:39:21 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:39:22.271 INFO:tasks.workunit.client.0.smithi001.stdout: 1 676458 5.42 MB/sec execute 536 sec latency 1947.534 ms 2024-02-02T15:39:22.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:39:22 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:39:22.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:39:22 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:39:22.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:39:22 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:39:22.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:39:22 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:39:22.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:39:22 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:39:22.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:39:22 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:39:22.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:39:22 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:39:22.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:39:22 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:39:22.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:39:22 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:39:23.272 INFO:tasks.workunit.client.0.smithi001.stdout: 1 677698 5.43 MB/sec execute 537 sec latency 2325.161 ms 2024-02-02T15:39:23.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:39:23 smithi071 ceph-mon[44707]: pgmap v386: 129 pgs: 129 active+clean; 572 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 187 op/s 2024-02-02T15:39:23.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:39:23 smithi142 ceph-mon[45728]: pgmap v386: 129 pgs: 129 active+clean; 572 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 187 op/s 2024-02-02T15:39:23.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:39:23 smithi001 ceph-mon[35470]: pgmap v386: 129 pgs: 129 active+clean; 572 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 187 op/s 2024-02-02T15:39:24.271 INFO:tasks.workunit.client.0.smithi001.stdout: 1 678844 5.43 MB/sec execute 538 sec latency 17.435 ms 2024-02-02T15:39:25.271 INFO:tasks.workunit.client.0.smithi001.stdout: 1 679918 5.43 MB/sec execute 539 sec latency 20.032 ms 2024-02-02T15:39:25.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:39:25 smithi071 ceph-mon[44707]: pgmap v387: 129 pgs: 129 active+clean; 572 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 187 op/s 2024-02-02T15:39:25.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:39:25 smithi142 ceph-mon[45728]: pgmap v387: 129 pgs: 129 active+clean; 572 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 187 op/s 2024-02-02T15:39:25.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:39:25 smithi001 ceph-mon[35470]: pgmap v387: 129 pgs: 129 active+clean; 572 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.6 MiB/s wr, 187 op/s 2024-02-02T15:39:26.271 INFO:tasks.workunit.client.0.smithi001.stdout: 1 680001 5.42 MB/sec execute 540 sec latency 886.653 ms 2024-02-02T15:39:27.272 INFO:tasks.workunit.client.0.smithi001.stdout: 1 680001 5.41 MB/sec execute 541 sec latency 1886.769 ms 2024-02-02T15:39:27.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:39:27 smithi071 ceph-mon[44707]: pgmap v388: 129 pgs: 129 active+clean; 578 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 143 op/s 2024-02-02T15:39:27.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:39:27 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:39:27.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:39:27 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:39:27.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:39:27 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:39:27.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:39:27 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:39:27.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:39:27 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:39:27.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:39:27 smithi142 ceph-mon[45728]: pgmap v388: 129 pgs: 129 active+clean; 578 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 143 op/s 2024-02-02T15:39:27.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:39:27 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:39:27.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:39:27 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:39:27.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:39:27 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:39:27.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:39:27 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:39:27.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:39:27 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:39:27.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:39:27 smithi001 ceph-mon[35470]: pgmap v388: 129 pgs: 129 active+clean; 578 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 143 op/s 2024-02-02T15:39:27.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:39:27 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:39:27.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:39:27 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:39:27.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:39:27 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:39:27.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:39:27 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:39:27.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:39:27 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:39:28.272 INFO:tasks.workunit.client.0.smithi001.stdout: 1 681182 5.41 MB/sec execute 542 sec latency 2268.633 ms 2024-02-02T15:39:29.272 INFO:tasks.workunit.client.0.smithi001.stdout: 1 682379 5.41 MB/sec execute 543 sec latency 16.637 ms 2024-02-02T15:39:29.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:39:29 smithi071 ceph-mon[44707]: pgmap v389: 129 pgs: 129 active+clean; 578 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 120 op/s 2024-02-02T15:39:29.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:39:29 smithi142 ceph-mon[45728]: pgmap v389: 129 pgs: 129 active+clean; 578 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 120 op/s 2024-02-02T15:39:29.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:39:29 smithi001 ceph-mon[35470]: pgmap v389: 129 pgs: 129 active+clean; 578 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 120 op/s 2024-02-02T15:39:30.272 INFO:tasks.workunit.client.0.smithi001.stdout: 1 683470 5.41 MB/sec execute 544 sec latency 17.653 ms 2024-02-02T15:39:31.272 INFO:tasks.workunit.client.0.smithi001.stdout: 1 683549 5.40 MB/sec execute 545 sec latency 895.282 ms 2024-02-02T15:39:31.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:39:31 smithi071 ceph-mon[44707]: pgmap v390: 129 pgs: 129 active+clean; 578 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 119 op/s 2024-02-02T15:39:31.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:39:31 smithi142 ceph-mon[45728]: pgmap v390: 129 pgs: 129 active+clean; 578 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 119 op/s 2024-02-02T15:39:31.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:39:31 smithi001 ceph-mon[35470]: pgmap v390: 129 pgs: 129 active+clean; 578 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 119 op/s 2024-02-02T15:39:32.272 INFO:tasks.workunit.client.0.smithi001.stdout: 1 683549 5.39 MB/sec execute 546 sec latency 1895.354 ms 2024-02-02T15:39:33.272 INFO:tasks.workunit.client.0.smithi001.stdout: 1 684735 5.39 MB/sec execute 547 sec latency 2275.679 ms 2024-02-02T15:39:33.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:39:33 smithi071 ceph-mon[44707]: pgmap v391: 129 pgs: 129 active+clean; 583 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 173 op/s 2024-02-02T15:39:33.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:39:33 smithi142 ceph-mon[45728]: pgmap v391: 129 pgs: 129 active+clean; 583 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 173 op/s 2024-02-02T15:39:33.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:39:33 smithi001 ceph-mon[35470]: pgmap v391: 129 pgs: 129 active+clean; 583 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.3 MiB/s wr, 173 op/s 2024-02-02T15:39:34.272 INFO:tasks.workunit.client.0.smithi001.stdout: 1 685935 5.40 MB/sec execute 548 sec latency 15.152 ms 2024-02-02T15:39:35.272 INFO:tasks.workunit.client.0.smithi001.stdout: 1 687009 5.40 MB/sec execute 549 sec latency 19.581 ms 2024-02-02T15:39:35.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:39:35 smithi071 ceph-mon[44707]: pgmap v392: 129 pgs: 129 active+clean; 583 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 994 KiB/s rd, 1.3 MiB/s wr, 105 op/s 2024-02-02T15:39:35.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:39:35 smithi142 ceph-mon[45728]: pgmap v392: 129 pgs: 129 active+clean; 583 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 994 KiB/s rd, 1.3 MiB/s wr, 105 op/s 2024-02-02T15:39:35.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:39:35 smithi001 ceph-mon[35470]: pgmap v392: 129 pgs: 129 active+clean; 583 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 994 KiB/s rd, 1.3 MiB/s wr, 105 op/s 2024-02-02T15:39:36.272 INFO:tasks.workunit.client.0.smithi001.stdout: 1 687092 5.39 MB/sec execute 550 sec latency 888.179 ms 2024-02-02T15:39:37.272 INFO:tasks.workunit.client.0.smithi001.stdout: 1 687092 5.38 MB/sec execute 551 sec latency 1888.309 ms 2024-02-02T15:39:37.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:39:37 smithi071 ceph-mon[44707]: pgmap v393: 129 pgs: 129 active+clean; 589 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.1 MiB/s wr, 160 op/s 2024-02-02T15:39:37.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:39:37 smithi142 ceph-mon[45728]: pgmap v393: 129 pgs: 129 active+clean; 589 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.1 MiB/s wr, 160 op/s 2024-02-02T15:39:37.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:39:37 smithi001 ceph-mon[35470]: pgmap v393: 129 pgs: 129 active+clean; 589 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.1 MiB/s wr, 160 op/s 2024-02-02T15:39:38.272 INFO:tasks.workunit.client.0.smithi001.stdout: 1 688293 5.38 MB/sec execute 552 sec latency 2264.786 ms 2024-02-02T15:39:39.272 INFO:tasks.workunit.client.0.smithi001.stdout: 1 689478 5.38 MB/sec execute 553 sec latency 17.956 ms 2024-02-02T15:39:39.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:39:39 smithi071 ceph-mon[44707]: pgmap v394: 129 pgs: 129 active+clean; 589 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1003 KiB/s rd, 1.6 MiB/s wr, 109 op/s 2024-02-02T15:39:39.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:39:39 smithi142 ceph-mon[45728]: pgmap v394: 129 pgs: 129 active+clean; 589 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1003 KiB/s rd, 1.6 MiB/s wr, 109 op/s 2024-02-02T15:39:39.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:39:39 smithi001 ceph-mon[35470]: pgmap v394: 129 pgs: 129 active+clean; 589 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1003 KiB/s rd, 1.6 MiB/s wr, 109 op/s 2024-02-02T15:39:40.273 INFO:tasks.workunit.client.0.smithi001.stdout: 1 690640 5.38 MB/sec execute 554 sec latency 138.645 ms 2024-02-02T15:39:41.273 INFO:tasks.workunit.client.0.smithi001.stdout: 1 690640 5.37 MB/sec execute 555 sec latency 1138.763 ms 2024-02-02T15:39:41.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:39:41 smithi071 ceph-mon[44707]: pgmap v395: 129 pgs: 129 active+clean; 589 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1003 KiB/s rd, 1.6 MiB/s wr, 109 op/s 2024-02-02T15:39:41.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:39:41 smithi142 ceph-mon[45728]: pgmap v395: 129 pgs: 129 active+clean; 589 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1003 KiB/s rd, 1.6 MiB/s wr, 109 op/s 2024-02-02T15:39:41.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:39:41 smithi001 ceph-mon[35470]: pgmap v395: 129 pgs: 129 active+clean; 589 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1003 KiB/s rd, 1.6 MiB/s wr, 109 op/s 2024-02-02T15:39:42.273 INFO:tasks.workunit.client.0.smithi001.stdout: 1 690640 5.37 MB/sec execute 556 sec latency 2138.898 ms 2024-02-02T15:39:43.273 INFO:tasks.workunit.client.0.smithi001.stdout: 1 691931 5.37 MB/sec execute 557 sec latency 2410.094 ms 2024-02-02T15:39:43.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:39:43 smithi071 ceph-mon[44707]: pgmap v396: 129 pgs: 129 active+clean; 594 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 164 op/s 2024-02-02T15:39:43.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:39:43 smithi142 ceph-mon[45728]: pgmap v396: 129 pgs: 129 active+clean; 594 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 164 op/s 2024-02-02T15:39:43.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:39:43 smithi001 ceph-mon[35470]: pgmap v396: 129 pgs: 129 active+clean; 594 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 164 op/s 2024-02-02T15:39:44.114 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-02T15:39:44.273 INFO:tasks.workunit.client.0.smithi001.stdout: 1 693111 5.37 MB/sec execute 558 sec latency 17.004 ms 2024-02-02T15:39:44.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.328+0000 7fa830e00700 1 -- 172.21.15.1:0/3358075233 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa82c103800 msgr2=0x7fa82c103cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:44.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.328+0000 7fa830e00700 1 --2- 172.21.15.1:0/3358075233 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa82c103800 0x7fa82c103cb0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fa82400b0a0 tx=0x7fa82400b3b0 comp rx=0 tx=0).stop 2024-02-02T15:39:44.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.328+0000 7fa830e00700 1 -- 172.21.15.1:0/3358075233 shutdown_connections 2024-02-02T15:39:44.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.328+0000 7fa830e00700 1 --2- 172.21.15.1:0/3358075233 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa82c103800 0x7fa82c103cb0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:44.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.328+0000 7fa830e00700 1 --2- 172.21.15.1:0/3358075233 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa82c105cb0 0x7fa82c068010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:44.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.328+0000 7fa830e00700 1 --2- 172.21.15.1:0/3358075233 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa82c068550 0x7fa82c0689d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:44.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.328+0000 7fa830e00700 1 -- 172.21.15.1:0/3358075233 >> 172.21.15.1:0/3358075233 conn(0x7fa82c0757d0 msgr2=0x7fa82c075be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:44.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.329+0000 7fa830e00700 1 -- 172.21.15.1:0/3358075233 shutdown_connections 2024-02-02T15:39:44.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.329+0000 7fa830e00700 1 -- 172.21.15.1:0/3358075233 wait complete. 2024-02-02T15:39:44.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.330+0000 7fa830e00700 1 Processor -- start 2024-02-02T15:39:44.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.331+0000 7fa830e00700 1 -- start start 2024-02-02T15:39:44.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.332+0000 7fa830e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa82c068550 0x7fa82c19d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:44.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.332+0000 7fa830e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa82c103800 0x7fa82c19d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:44.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.332+0000 7fa830e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa82c105cb0 0x7fa82c1a1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:44.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.332+0000 7fa830e00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82c1a2300 con 0x7fa82c068550 2024-02-02T15:39:44.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.332+0000 7fa830e00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82c1a2470 con 0x7fa82c103800 2024-02-02T15:39:44.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.332+0000 7fa830e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa82c1a2630 con 0x7fa82c105cb0 2024-02-02T15:39:44.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.332+0000 7fa829400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa82c103800 0x7fa82c19d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:44.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.332+0000 7fa829400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa82c103800 0x7fa82c19d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:41180/0 (socket says 172.21.15.1:41180) 2024-02-02T15:39:44.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.332+0000 7fa829400700 1 -- 172.21.15.1:0/664291346 learned_addr learned my addr 172.21.15.1:0/664291346 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:39:44.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.332+0000 7fa82a800700 1 --2- 172.21.15.1:0/664291346 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa82c105cb0 0x7fa82c1a1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:44.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.332+0000 7fa829e00700 1 --2- 172.21.15.1:0/664291346 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa82c068550 0x7fa82c19d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:44.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.332+0000 7fa829400700 1 -- 172.21.15.1:0/664291346 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa82c105cb0 msgr2=0x7fa82c1a1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:44.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.332+0000 7fa829400700 1 --2- 172.21.15.1:0/664291346 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa82c105cb0 0x7fa82c1a1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:44.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.332+0000 7fa829400700 1 -- 172.21.15.1:0/664291346 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa82c068550 msgr2=0x7fa82c19d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:44.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.332+0000 7fa829400700 1 --2- 172.21.15.1:0/664291346 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa82c068550 0x7fa82c19d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:44.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.332+0000 7fa829400700 1 -- 172.21.15.1:0/664291346 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa824009d00 con 0x7fa82c103800 2024-02-02T15:39:44.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.332+0000 7fa82a800700 1 --2- 172.21.15.1:0/664291346 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa82c105cb0 0x7fa82c1a1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:39:44.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.333+0000 7fa829400700 1 --2- 172.21.15.1:0/664291346 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa82c103800 0x7fa82c19d860 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fa81400e9a0 tx=0x7fa81400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:44.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.333+0000 7fa822a00700 1 -- 172.21.15.1:0/664291346 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa814009790 con 0x7fa82c103800 2024-02-02T15:39:44.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.333+0000 7fa822a00700 1 -- 172.21.15.1:0/664291346 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa814010430 con 0x7fa82c103800 2024-02-02T15:39:44.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.333+0000 7fa830e00700 1 -- 172.21.15.1:0/664291346 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa82c1a2910 con 0x7fa82c103800 2024-02-02T15:39:44.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.333+0000 7fa822a00700 1 -- 172.21.15.1:0/664291346 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa814018680 con 0x7fa82c103800 2024-02-02T15:39:44.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.333+0000 7fa830e00700 1 -- 172.21.15.1:0/664291346 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa82c1aa150 con 0x7fa82c103800 2024-02-02T15:39:44.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.337+0000 7fa822a00700 1 -- 172.21.15.1:0/664291346 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7fa8140105a0 con 0x7fa82c103800 2024-02-02T15:39:44.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.338+0000 7fa84489bb80 1 Processor -- start 2024-02-02T15:39:44.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.338+0000 7fa84489bb80 1 -- start start 2024-02-02T15:39:44.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.339+0000 7fa84489bb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c47eb88010 0x55c47eb883f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:44.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.339+0000 7fa84489bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c47eb88930 0x55c47eb932a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:44.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.339+0000 7fa84489bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c47eb937e0 0x55c47eb95bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:44.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.339+0000 7fa822a00700 1 --2- 172.21.15.1:0/664291346 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fa80c0a0690 0x7fa80c0a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:44.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.339+0000 7fa84489bb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c47eafc4c0 con 0x55c47eb88010 2024-02-02T15:39:44.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.339+0000 7fa84489bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c47eae0aa0 con 0x55c47eb937e0 2024-02-02T15:39:44.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.339+0000 7fa84489bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c47eade460 con 0x55c47eb88930 2024-02-02T15:39:44.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.339+0000 7fa829e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c47eb88930 0x55c47eb932a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:44.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.339+0000 7fa822a00700 1 -- 172.21.15.1:0/664291346 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fa814014070 con 0x7fa82c103800 2024-02-02T15:39:44.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.339+0000 7fa829e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c47eb88930 0x55c47eb932a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:33874/0 (socket says 172.21.15.1:33874) 2024-02-02T15:39:44.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.339+0000 7fa829e00700 1 -- 172.21.15.1:0/3230992273 learned_addr learned my addr 172.21.15.1:0/3230992273 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:39:44.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.339+0000 7fa829400700 1 --2- 172.21.15.1:0/3230992273 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c47eb937e0 0x55c47eb95bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:44.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.339+0000 7fa82a800700 1 --2- 172.21.15.1:0/3230992273 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c47eb88010 0x55c47eb883f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:44.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.340+0000 7fa82a800700 1 --2- 172.21.15.1:0/664291346 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fa80c0a0690 0x7fa80c0a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:44.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.340+0000 7fa829e00700 1 -- 172.21.15.1:0/3230992273 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c47eb937e0 msgr2=0x55c47eb95bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:44.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.340+0000 7fa829e00700 1 --2- 172.21.15.1:0/3230992273 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c47eb937e0 0x55c47eb95bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:44.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.340+0000 7fa829e00700 1 -- 172.21.15.1:0/3230992273 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c47eb88010 msgr2=0x55c47eb883f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:44.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.340+0000 7fa829e00700 1 --2- 172.21.15.1:0/3230992273 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c47eb88010 0x55c47eb883f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:44.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.340+0000 7fa829e00700 1 -- 172.21.15.1:0/3230992273 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c47e95b3b0 con 0x55c47eb88930 2024-02-02T15:39:44.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.340+0000 7fa829e00700 1 --2- 172.21.15.1:0/3230992273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c47eb88930 0x55c47eb932a0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fa81c009d30 tx=0x7fa81c011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d8b3c0da9fca122 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:44.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.340+0000 7fa81b400700 1 -- 172.21.15.1:0/3230992273 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa81c004230 con 0x55c47eb88930 2024-02-02T15:39:44.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.340+0000 7fa82a800700 1 --2- 172.21.15.1:0/664291346 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fa80c0a0690 0x7fa80c0a2b50 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa824009040 tx=0x7fa824006230 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:44.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.340+0000 7fa81b400700 1 -- 172.21.15.1:0/3230992273 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa81c009e80 con 0x55c47eb88930 2024-02-02T15:39:44.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.340+0000 7fa81b400700 1 -- 172.21.15.1:0/3230992273 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa81c019750 con 0x55c47eb88930 2024-02-02T15:39:44.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.341+0000 7fa84489bb80 1 -- 172.21.15.1:0/3230992273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c47eb88930 msgr2=0x55c47eb932a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:44.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.341+0000 7fa84489bb80 1 --2- 172.21.15.1:0/3230992273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c47eb88930 0x55c47eb932a0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fa81c009d30 tx=0x7fa81c011720 comp rx=0 tx=0).stop 2024-02-02T15:39:44.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.341+0000 7fa84489bb80 1 -- 172.21.15.1:0/3230992273 shutdown_connections 2024-02-02T15:39:44.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.341+0000 7fa84489bb80 1 --2- 172.21.15.1:0/3230992273 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c47eb937e0 0x55c47eb95bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:44.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.341+0000 7fa84489bb80 1 --2- 172.21.15.1:0/3230992273 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c47eb88010 0x55c47eb883f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:44.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.341+0000 7fa84489bb80 1 --2- 172.21.15.1:0/3230992273 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c47eb88930 0x55c47eb932a0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:44.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.341+0000 7fa84489bb80 1 -- 172.21.15.1:0/3230992273 >> 172.21.15.1:0/3230992273 conn(0x55c47eae7db0 msgr2=0x55c47eb843b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:44.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.341+0000 7fa84489bb80 1 -- 172.21.15.1:0/3230992273 shutdown_connections 2024-02-02T15:39:44.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.341+0000 7fa84489bb80 1 -- 172.21.15.1:0/3230992273 wait complete. 2024-02-02T15:39:44.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.342+0000 7fa84489bb80 1 -- start start 2024-02-02T15:39:44.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.342+0000 7fa84489bb80 1 Processor -- start 2024-02-02T15:39:44.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.342+0000 7fa84489bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c47eb88010 0x55c47eb93160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:44.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.342+0000 7fa84489bb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c47eb88930 0x55c47eb90370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:44.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.342+0000 7fa84489bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c47eb937e0 0x55c47eb90910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:44.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.342+0000 7fa84489bb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c47ead9d50 con 0x55c47eb88930 2024-02-02T15:39:44.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.342+0000 7fa84489bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c47ea75d80 con 0x55c47eb88010 2024-02-02T15:39:44.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.342+0000 7fa84489bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c47ea75a20 con 0x55c47eb937e0 2024-02-02T15:39:44.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.343+0000 7fa829e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c47eb88010 0x55c47eb93160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:44.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.343+0000 7fa84489bb80 10 client.0 _ll_register_callbacks cb 0x55c47ea77290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:39:44.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.343+0000 7fa829e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c47eb88010 0x55c47eb93160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:41204/0 (socket says 172.21.15.1:41204) 2024-02-02T15:39:44.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.343+0000 7fa829e00700 1 -- 172.21.15.1:0/296716137 learned_addr learned my addr 172.21.15.1:0/296716137 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:39:44.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.343+0000 7fa82a800700 1 --2- 172.21.15.1:0/296716137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c47eb937e0 0x55c47eb90910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:44.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.343+0000 7fa829400700 1 --2- 172.21.15.1:0/296716137 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c47eb88930 0x55c47eb90370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:44.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.343+0000 7fa829e00700 1 -- 172.21.15.1:0/296716137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c47eb937e0 msgr2=0x55c47eb90910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:44.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.343+0000 7fa829e00700 1 --2- 172.21.15.1:0/296716137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c47eb937e0 0x55c47eb90910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:44.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.343+0000 7fa829e00700 1 -- 172.21.15.1:0/296716137 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c47eb88930 msgr2=0x55c47eb90370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:44.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.343+0000 7fa829e00700 1 --2- 172.21.15.1:0/296716137 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c47eb88930 0x55c47eb90370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:44.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.343+0000 7fa829e00700 1 -- 172.21.15.1:0/296716137 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa81c009960 con 0x55c47eb88010 2024-02-02T15:39:44.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.343+0000 7fa829400700 1 --2- 172.21.15.1:0/296716137 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c47eb88930 0x55c47eb90370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:39:44.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.343+0000 7fa829e00700 1 --2- 172.21.15.1:0/296716137 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c47eb88010 0x55c47eb93160 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fa81c005920 tx=0x7fa81c0042b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=268e045f514a8b4e server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:44.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.343+0000 7fa813e00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:39:44.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.344+0000 7fa813e00700 1 -- 172.21.15.1:0/296716137 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa81c005450 con 0x55c47eb88010 2024-02-02T15:39:44.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.344+0000 7fa813e00700 1 -- 172.21.15.1:0/296716137 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fa81c009e80 con 0x55c47eb88010 2024-02-02T15:39:44.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.344+0000 7fa84489bb80 1 -- 172.21.15.1:0/296716137 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c47ead2dd0 con 0x55c47eb88010 2024-02-02T15:39:44.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.344+0000 7fa813e00700 1 -- 172.21.15.1:0/296716137 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa81c0055b0 con 0x55c47eb88010 2024-02-02T15:39:44.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.344+0000 7fa84489bb80 10 client.28949 fetch_fsmap learned FSMap version 52 2024-02-02T15:39:44.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.344+0000 7fa84489bb80 1 -- 172.21.15.1:0/296716137 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x55c47e907b60 con 0x55c47eb88010 2024-02-02T15:39:44.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.344+0000 7fa813e00700 1 -- 172.21.15.1:0/296716137 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c47ead2dd0 con 0x55c47eb88010 2024-02-02T15:39:44.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.344+0000 7fa813e00700 1 -- 172.21.15.1:0/296716137 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 52) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fa81c0226a0 con 0x55c47eb88010 2024-02-02T15:39:44.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.344+0000 7fa84489bb80 10 client.28949 fetch_fsmap finished waiting for FSMap version 52 2024-02-02T15:39:44.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.344+0000 7fa84489bb80 10 client.28949 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:39:44.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.344+0000 7fa84489bb80 20 client.28949 populate_metadata read hostname 'smithi001' 2024-02-02T15:39:44.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.344+0000 7fa84489bb80 1 --2- 172.21.15.1:0/296716137 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55c47eba3080 0x55c47eba5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:44.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.344+0000 7fa84489bb80 4 client.28949 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:39:44.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.344+0000 7fa84489bb80 1 -- 172.21.15.1:0/296716137 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c47ea98f50 con 0x55c47eba3080 2024-02-02T15:39:44.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.345+0000 7fa829400700 1 --2- 172.21.15.1:0/296716137 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55c47eba3080 0x55c47eba5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:44.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.345+0000 7fa829400700 1 --2- 172.21.15.1:0/296716137 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55c47eba3080 0x55c47eba5470 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e125b0555e2430a6 server_cookie=856a93119a12cbf4 in_seq=0 out_seq=0 2024-02-02T15:39:44.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.345+0000 7fa813e00700 10 client.28949 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:39:44.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.346+0000 7fa813e00700 1 -- 172.21.15.1:0/296716137 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55c47ea98f50 con 0x55c47eba3080 2024-02-02T15:39:44.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.346+0000 7fa813e00700 10 client.28949 handle_command_reply: tid=0 2024-02-02T15:39:44.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.346+0000 7fa84489bb80 1 client.28949 shutdown 2024-02-02T15:39:44.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.346+0000 7fa84489bb80 1 -- 172.21.15.1:0/296716137 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c47eb88010 msgr2=0x55c47eb93160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:44.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.346+0000 7fa84489bb80 1 --2- 172.21.15.1:0/296716137 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c47eb88010 0x55c47eb93160 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fa81c005920 tx=0x7fa81c0042b0 comp rx=0 tx=0).stop 2024-02-02T15:39:44.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.346+0000 7fa84489bb80 1 -- 172.21.15.1:0/296716137 shutdown_connections 2024-02-02T15:39:44.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.346+0000 7fa84489bb80 1 --2- 172.21.15.1:0/296716137 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55c47eba3080 0x55c47eba5470 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:44.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.346+0000 7fa84489bb80 1 --2- 172.21.15.1:0/296716137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c47eb937e0 0x55c47eb90910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:44.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.346+0000 7fa84489bb80 1 --2- 172.21.15.1:0/296716137 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c47eb88010 0x55c47eb93160 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:44.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.346+0000 7fa84489bb80 1 --2- 172.21.15.1:0/296716137 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c47eb88930 0x55c47eb90370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:44.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.346+0000 7fa84489bb80 1 -- 172.21.15.1:0/296716137 >> 172.21.15.1:0/296716137 conn(0x55c47eae7db0 msgr2=0x55c47eb83fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:44.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.346+0000 7fa813e00700 0 client.28949 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:39:44.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.346+0000 7fa84489bb80 1 -- 172.21.15.1:0/296716137 shutdown_connections 2024-02-02T15:39:44.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.346+0000 7fa84489bb80 1 -- 172.21.15.1:0/296716137 wait complete. 2024-02-02T15:39:44.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.347+0000 7fa84489bb80 20 client.28949 trim_cache size 0 max 16384 2024-02-02T15:39:44.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.369+0000 7fa84489bb80 1 Processor -- start 2024-02-02T15:39:44.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.370+0000 7fa84489bb80 1 -- start start 2024-02-02T15:39:44.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.370+0000 7fa84489bb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c47eb88010 0x55c47eb92c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:44.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.370+0000 7fa84489bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c47eb88930 0x55c47eb9e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:44.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.370+0000 7fa84489bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c47eb937e0 0x55c47eb9ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:44.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.370+0000 7fa84489bb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c47e95d6a0 con 0x55c47eb88010 2024-02-02T15:39:44.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.370+0000 7fa84489bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c47eb8c1e0 con 0x55c47eb88930 2024-02-02T15:39:44.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.370+0000 7fa84489bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c47eb90f10 con 0x55c47eb937e0 2024-02-02T15:39:44.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.370+0000 7fa829e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c47eb88010 0x55c47eb92c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:44.371 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.370+0000 7fa82a800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c47eb937e0 0x55c47eb9ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:44.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.370+0000 7fa829400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c47eb88930 0x55c47eb9e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:44.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.370+0000 7fa82a800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c47eb937e0 0x55c47eb9ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:33900/0 (socket says 172.21.15.1:33900) 2024-02-02T15:39:44.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.370+0000 7fa82a800700 1 -- 172.21.15.1:0/2461788088 learned_addr learned my addr 172.21.15.1:0/2461788088 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:39:44.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.371+0000 7fa82a800700 1 -- 172.21.15.1:0/2461788088 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c47eb88930 msgr2=0x55c47eb9e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:44.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.371+0000 7fa82a800700 1 --2- 172.21.15.1:0/2461788088 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c47eb88930 0x55c47eb9e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:44.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.371+0000 7fa82a800700 1 -- 172.21.15.1:0/2461788088 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c47eb88010 msgr2=0x55c47eb92c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:44.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.371+0000 7fa82a800700 1 --2- 172.21.15.1:0/2461788088 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c47eb88010 0x55c47eb92c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:44.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.371+0000 7fa82a800700 1 -- 172.21.15.1:0/2461788088 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c47ea9fea0 con 0x55c47eb937e0 2024-02-02T15:39:44.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.371+0000 7fa829e00700 1 --2- 172.21.15.1:0/2461788088 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c47eb88010 0x55c47eb92c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:39:44.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.371+0000 7fa82a800700 1 --2- 172.21.15.1:0/2461788088 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c47eb937e0 0x55c47eb9ee40 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fa82400b0a0 tx=0x7fa824006020 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a40fc9350088fdcc server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:44.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.371+0000 7fa81be00700 1 -- 172.21.15.1:0/2461788088 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82400e040 con 0x55c47eb937e0 2024-02-02T15:39:44.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.371+0000 7fa81be00700 1 -- 172.21.15.1:0/2461788088 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa8240078a0 con 0x55c47eb937e0 2024-02-02T15:39:44.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.371+0000 7fa81be00700 1 -- 172.21.15.1:0/2461788088 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa824005330 con 0x55c47eb937e0 2024-02-02T15:39:44.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.372+0000 7fa84489bb80 1 -- 172.21.15.1:0/2461788088 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c47eb937e0 msgr2=0x55c47eb9ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:44.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.372+0000 7fa84489bb80 1 --2- 172.21.15.1:0/2461788088 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c47eb937e0 0x55c47eb9ee40 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fa82400b0a0 tx=0x7fa824006020 comp rx=0 tx=0).stop 2024-02-02T15:39:44.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.372+0000 7fa84489bb80 1 -- 172.21.15.1:0/2461788088 shutdown_connections 2024-02-02T15:39:44.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.372+0000 7fa84489bb80 1 --2- 172.21.15.1:0/2461788088 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c47eb937e0 0x55c47eb9ee40 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:44.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.372+0000 7fa84489bb80 1 --2- 172.21.15.1:0/2461788088 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c47eb88010 0x55c47eb92c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:44.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.372+0000 7fa84489bb80 1 --2- 172.21.15.1:0/2461788088 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c47eb88930 0x55c47eb9e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:44.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.372+0000 7fa84489bb80 1 -- 172.21.15.1:0/2461788088 >> 172.21.15.1:0/2461788088 conn(0x55c47eae7db0 msgr2=0x55c47eb9f500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:44.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.372+0000 7fa84489bb80 1 -- 172.21.15.1:0/2461788088 shutdown_connections 2024-02-02T15:39:44.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.372+0000 7fa84489bb80 1 -- 172.21.15.1:0/2461788088 wait complete. 2024-02-02T15:39:44.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.373+0000 7fa84489bb80 1 -- start start 2024-02-02T15:39:44.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.373+0000 7fa84489bb80 1 Processor -- start 2024-02-02T15:39:44.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.374+0000 7fa84489bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c47eb88010 0x55c47eb9a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:44.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.374+0000 7fa84489bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c47eb88930 0x55c47eb96750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:44.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.374+0000 7fa84489bb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c47eb937e0 0x55c47eb96d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:44.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.374+0000 7fa84489bb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c47eb97490 con 0x55c47eb937e0 2024-02-02T15:39:44.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.374+0000 7fa84489bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c47eb97600 con 0x55c47eb88010 2024-02-02T15:39:44.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.374+0000 7fa84489bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c47ebf4b90 con 0x55c47eb88930 2024-02-02T15:39:44.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.374+0000 7fa84489bb80 10 client.0 _ll_register_callbacks cb 0x55c47ea77290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:39:44.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.374+0000 7fa829e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c47eb88010 0x55c47eb9a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:44.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.374+0000 7fa829400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c47eb88930 0x55c47eb96750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:44.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.374+0000 7fa829e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c47eb88010 0x55c47eb9a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:41230/0 (socket says 172.21.15.1:41230) 2024-02-02T15:39:44.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.374+0000 7fa829e00700 1 -- 172.21.15.1:0/1783829414 learned_addr learned my addr 172.21.15.1:0/1783829414 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:39:44.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.374+0000 7fa82a800700 1 --2- 172.21.15.1:0/1783829414 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c47eb937e0 0x55c47eb96d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:44.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.374+0000 7fa829400700 1 -- 172.21.15.1:0/1783829414 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c47eb88010 msgr2=0x55c47eb9a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:44.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.374+0000 7fa829400700 1 --2- 172.21.15.1:0/1783829414 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c47eb88010 0x55c47eb9a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:44.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.374+0000 7fa829400700 1 -- 172.21.15.1:0/1783829414 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c47eb937e0 msgr2=0x55c47eb96d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:44.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.374+0000 7fa829400700 1 --2- 172.21.15.1:0/1783829414 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c47eb937e0 0x55c47eb96d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:44.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.374+0000 7fa829400700 1 -- 172.21.15.1:0/1783829414 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c47e907b60 con 0x55c47eb88930 2024-02-02T15:39:44.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.375+0000 7fa829e00700 1 --2- 172.21.15.1:0/1783829414 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c47eb88010 0x55c47eb9a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:39:44.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.375+0000 7fa829400700 1 --2- 172.21.15.1:0/1783829414 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c47eb88930 0x55c47eb96750 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fa8140c9680 tx=0x7fa814084040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dfb59e1a9a6afac4 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:44.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.375+0000 7fa813e00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:39:44.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.375+0000 7fa813e00700 1 -- 172.21.15.1:0/1783829414 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa814099450 con 0x55c47eb88930 2024-02-02T15:39:44.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.375+0000 7fa813e00700 1 -- 172.21.15.1:0/1783829414 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fa81409d280 con 0x55c47eb88930 2024-02-02T15:39:44.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.375+0000 7fa84489bb80 1 -- 172.21.15.1:0/1783829414 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c47eb7c0f0 con 0x55c47eb88930 2024-02-02T15:39:44.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.375+0000 7fa813e00700 1 -- 172.21.15.1:0/1783829414 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa8140a14c0 con 0x55c47eb88930 2024-02-02T15:39:44.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.375+0000 7fa84489bb80 10 client.28957 fetch_fsmap learned FSMap version 52 2024-02-02T15:39:44.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.375+0000 7fa84489bb80 1 -- 172.21.15.1:0/1783829414 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x7fa82c114cd0 con 0x55c47eb88930 2024-02-02T15:39:44.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.375+0000 7fa813e00700 1 -- 172.21.15.1:0/1783829414 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c47eb7c0f0 con 0x55c47eb88930 2024-02-02T15:39:44.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.375+0000 7fa813e00700 1 -- 172.21.15.1:0/1783829414 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 52) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fa814098940 con 0x55c47eb88930 2024-02-02T15:39:44.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.375+0000 7fa84489bb80 10 client.28957 fetch_fsmap finished waiting for FSMap version 52 2024-02-02T15:39:44.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.376+0000 7fa84489bb80 10 client.28957 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:39:44.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.376+0000 7fa84489bb80 20 client.28957 populate_metadata read hostname 'smithi001' 2024-02-02T15:39:44.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.376+0000 7fa84489bb80 1 --2- 172.21.15.1:0/1783829414 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55c47ebf62c0 0x55c47ebf86e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:44.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.376+0000 7fa84489bb80 4 client.28957 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2024-02-02T15:39:44.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.376+0000 7fa84489bb80 1 -- 172.21.15.1:0/1783829414 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c47e928cf0 con 0x55c47ebf62c0 2024-02-02T15:39:44.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.376+0000 7fa829e00700 1 --2- 172.21.15.1:0/1783829414 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55c47ebf62c0 0x55c47ebf86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:44.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.376+0000 7fa829e00700 1 --2- 172.21.15.1:0/1783829414 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55c47ebf62c0 0x55c47ebf86e0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b496fe8177ec0c3 server_cookie=5afb29a252e99465 in_seq=0 out_seq=0 2024-02-02T15:39:44.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.376+0000 7fa813e00700 10 client.28957 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:39:44.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.376+0000 7fa813e00700 1 -- 172.21.15.1:0/1783829414 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55c47e928cf0 con 0x55c47ebf62c0 2024-02-02T15:39:44.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.376+0000 7fa813e00700 10 client.28957 handle_command_reply: tid=0 2024-02-02T15:39:44.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.376+0000 7fa84489bb80 1 client.28957 shutdown 2024-02-02T15:39:44.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.377+0000 7fa84489bb80 1 -- 172.21.15.1:0/1783829414 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c47eb88930 msgr2=0x55c47eb96750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:44.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.377+0000 7fa84489bb80 1 --2- 172.21.15.1:0/1783829414 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c47eb88930 0x55c47eb96750 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fa8140c9680 tx=0x7fa814084040 comp rx=0 tx=0).stop 2024-02-02T15:39:44.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.377+0000 7fa84489bb80 1 -- 172.21.15.1:0/1783829414 shutdown_connections 2024-02-02T15:39:44.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.377+0000 7fa84489bb80 1 --2- 172.21.15.1:0/1783829414 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55c47ebf62c0 0x55c47ebf86e0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:44.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.377+0000 7fa84489bb80 1 --2- 172.21.15.1:0/1783829414 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c47eb937e0 0x55c47eb96d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:44.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.377+0000 7fa84489bb80 1 --2- 172.21.15.1:0/1783829414 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c47eb88010 0x55c47eb9a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:44.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.377+0000 7fa84489bb80 1 --2- 172.21.15.1:0/1783829414 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c47eb88930 0x55c47eb96750 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:44.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.377+0000 7fa813e00700 0 client.28957 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:39:44.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.377+0000 7fa84489bb80 1 -- 172.21.15.1:0/1783829414 >> 172.21.15.1:0/1783829414 conn(0x55c47eae7db0 msgr2=0x55c47eb9e900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:44.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.377+0000 7fa84489bb80 1 -- 172.21.15.1:0/1783829414 shutdown_connections 2024-02-02T15:39:44.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.377+0000 7fa84489bb80 1 -- 172.21.15.1:0/1783829414 wait complete. 2024-02-02T15:39:44.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.377+0000 7fa84489bb80 20 client.28957 trim_cache size 0 max 16384 2024-02-02T15:39:44.382 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:39:44.382 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2024-02-02T15:39:44.382 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2024-02-02T15:39:44.382 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:39:44.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.378+0000 7fa830e00700 1 -- 172.21.15.1:0/664291346 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fa80c0a0690 msgr2=0x7fa80c0a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:44.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.378+0000 7fa830e00700 1 --2- 172.21.15.1:0/664291346 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fa80c0a0690 0x7fa80c0a2b50 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa824009040 tx=0x7fa824006230 comp rx=0 tx=0).stop 2024-02-02T15:39:44.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.378+0000 7fa830e00700 1 -- 172.21.15.1:0/664291346 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa82c103800 msgr2=0x7fa82c19d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:44.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.378+0000 7fa830e00700 1 --2- 172.21.15.1:0/664291346 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa82c103800 0x7fa82c19d860 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fa81400e9a0 tx=0x7fa81400ecb0 comp rx=0 tx=0).stop 2024-02-02T15:39:44.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.378+0000 7fa830e00700 1 -- 172.21.15.1:0/664291346 shutdown_connections 2024-02-02T15:39:44.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.378+0000 7fa830e00700 1 --2- 172.21.15.1:0/664291346 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fa80c0a0690 0x7fa80c0a2b50 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:44.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.378+0000 7fa830e00700 1 --2- 172.21.15.1:0/664291346 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa82c105cb0 0x7fa82c1a1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:44.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.378+0000 7fa830e00700 1 --2- 172.21.15.1:0/664291346 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa82c068550 0x7fa82c19d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:44.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.378+0000 7fa830e00700 1 --2- 172.21.15.1:0/664291346 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa82c103800 0x7fa82c19d860 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:44.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.378+0000 7fa830e00700 1 -- 172.21.15.1:0/664291346 >> 172.21.15.1:0/664291346 conn(0x7fa82c0757d0 msgr2=0x7fa82c1027e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:44.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.378+0000 7fa830e00700 1 -- 172.21.15.1:0/664291346 shutdown_connections 2024-02-02T15:39:44.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.378+0000 7fa830e00700 1 -- 172.21.15.1:0/664291346 wait complete. 2024-02-02T15:39:44.403 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-02T15:39:44.403 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T15:39:44.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.632+0000 7ff263600700 1 -- 172.21.15.1:0/1369360977 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff25c1018d0 msgr2=0x7ff25c105910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:44.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.632+0000 7ff263600700 1 --2- 172.21.15.1:0/1369360977 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff25c1018d0 0x7ff25c105910 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7ff2440098f0 tx=0x7ff244009c00 comp rx=0 tx=0).stop 2024-02-02T15:39:44.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.632+0000 7ff263600700 1 -- 172.21.15.1:0/1369360977 shutdown_connections 2024-02-02T15:39:44.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.632+0000 7ff263600700 1 --2- 172.21.15.1:0/1369360977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff25c106040 0x7ff25c10dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:44.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.632+0000 7ff263600700 1 --2- 172.21.15.1:0/1369360977 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff25c100f20 0x7ff25c101300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:44.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.632+0000 7ff263600700 1 --2- 172.21.15.1:0/1369360977 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff25c1018d0 0x7ff25c105910 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:44.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.632+0000 7ff263600700 1 -- 172.21.15.1:0/1369360977 >> 172.21.15.1:0/1369360977 conn(0x7ff25c0fc7a0 msgr2=0x7ff25c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:44.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.632+0000 7ff263600700 1 -- 172.21.15.1:0/1369360977 shutdown_connections 2024-02-02T15:39:44.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.632+0000 7ff263600700 1 -- 172.21.15.1:0/1369360977 wait complete. 2024-02-02T15:39:44.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.633+0000 7ff263600700 1 Processor -- start 2024-02-02T15:39:44.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.634+0000 7ff263600700 1 -- start start 2024-02-02T15:39:44.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.634+0000 7ff263600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff25c100f20 0x7ff25c198ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:44.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.634+0000 7ff263600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff25c106040 0x7ff25c199530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:44.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.634+0000 7ff263600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff25c19d990 0x7ff25c19de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:44.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.634+0000 7ff263600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff25c19e410 con 0x7ff25c19d990 2024-02-02T15:39:44.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.634+0000 7ff263600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff25c19e580 con 0x7ff25c106040 2024-02-02T15:39:44.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.634+0000 7ff263600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff25c19e740 con 0x7ff25c100f20 2024-02-02T15:39:44.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.634+0000 7ff25be00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff25c106040 0x7ff25c199530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:44.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.634+0000 7ff260c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff25c100f20 0x7ff25c198ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:44.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.634+0000 7ff25be00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff25c106040 0x7ff25c199530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:41256/0 (socket says 172.21.15.1:41256) 2024-02-02T15:39:44.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.634+0000 7ff25be00700 1 -- 172.21.15.1:0/4210301728 learned_addr learned my addr 172.21.15.1:0/4210301728 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:39:44.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.634+0000 7ff261600700 1 --2- 172.21.15.1:0/4210301728 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff25c19d990 0x7ff25c19de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:44.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.635+0000 7ff25be00700 1 -- 172.21.15.1:0/4210301728 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff25c100f20 msgr2=0x7ff25c198ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:44.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.635+0000 7ff25be00700 1 --2- 172.21.15.1:0/4210301728 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff25c100f20 0x7ff25c198ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:44.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.635+0000 7ff25be00700 1 -- 172.21.15.1:0/4210301728 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff25c19d990 msgr2=0x7ff25c19de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:44.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.635+0000 7ff25be00700 1 --2- 172.21.15.1:0/4210301728 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff25c19d990 0x7ff25c19de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:44.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.635+0000 7ff25be00700 1 -- 172.21.15.1:0/4210301728 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff244009520 con 0x7ff25c106040 2024-02-02T15:39:44.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.635+0000 7ff260c00700 1 --2- 172.21.15.1:0/4210301728 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff25c100f20 0x7ff25c198ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:39:44.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.635+0000 7ff25be00700 1 --2- 172.21.15.1:0/4210301728 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff25c106040 0x7ff25c199530 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7ff244009fd0 tx=0x7ff24400fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:44.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.635+0000 7ff259600700 1 -- 172.21.15.1:0/4210301728 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff24401c070 con 0x7ff25c106040 2024-02-02T15:39:44.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.635+0000 7ff259600700 1 -- 172.21.15.1:0/4210301728 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff244005040 con 0x7ff25c106040 2024-02-02T15:39:44.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.635+0000 7ff263600700 1 -- 172.21.15.1:0/4210301728 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff25c1a5c40 con 0x7ff25c106040 2024-02-02T15:39:44.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.635+0000 7ff259600700 1 -- 172.21.15.1:0/4210301728 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff244004910 con 0x7ff25c106040 2024-02-02T15:39:44.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.635+0000 7ff263600700 1 -- 172.21.15.1:0/4210301728 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff25c1a60c0 con 0x7ff25c106040 2024-02-02T15:39:44.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.636+0000 7ff263600700 1 -- 172.21.15.1:0/4210301728 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff25c067bc0 con 0x7ff25c106040 2024-02-02T15:39:44.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.639+0000 7ff259600700 1 -- 172.21.15.1:0/4210301728 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7ff244020070 con 0x7ff25c106040 2024-02-02T15:39:44.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.640+0000 7ff259600700 1 --2- 172.21.15.1:0/4210301728 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff23c0a0690 0x7ff23c0a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:44.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.640+0000 7ff259600700 1 -- 172.21.15.1:0/4210301728 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9453+0+0 (secure 0 0 0) 0x7ff2440ccd60 con 0x7ff25c106040 2024-02-02T15:39:44.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.641+0000 7ff260c00700 1 --2- 172.21.15.1:0/4210301728 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff23c0a0690 0x7ff23c0a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:44.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.641+0000 7ff260c00700 1 --2- 172.21.15.1:0/4210301728 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff23c0a0690 0x7ff23c0a2b50 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff24c005fd0 tx=0x7ff24c009310 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:44.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.644+0000 7ff259600700 1 -- 172.21.15.1:0/4210301728 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7ff244096250 con 0x7ff25c106040 2024-02-02T15:39:44.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.954+0000 7ff263600700 1 -- 172.21.15.1:0/4210301728 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff25c19a340 con 0x7ff25c106040 2024-02-02T15:39:44.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.955+0000 7ff259600700 1 -- 172.21.15.1:0/4210301728 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+10873 (secure 0 0 0) 0x7ff244027090 con 0x7ff25c106040 2024-02-02T15:39:44.956 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:39:44.956 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":52,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2922240448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2922240448},{"type":"v1","addr":"172.21.15.1:6835","nonce":2922240448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2729497431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2729497431},{"type":"v1","addr":"172.21.15.1:6837","nonce":2729497431}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":52,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-02T15:24:47.761435+0000","modified":"2024-02-02T15:39:00.496919+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24529,"mds_2":24518,"mds_3":24514,"mds_4":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14517":{"gid":14517,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/1218303683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":1218303683},{"type":"v1","addr":"172.21.15.1:6839","nonce":1218303683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14529":{"gid":14529,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/2059197496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2059197496},{"type":"v1","addr":"172.21.15.1:6841","nonce":2059197496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/7648986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":7648986},{"type":"v1","addr":"172.21.15.142:6833","nonce":7648986}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/3482746094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3482746094},{"type":"v1","addr":"172.21.15.71:6833","nonce":3482746094}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/388207814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":388207814},{"type":"v1","addr":"172.21.15.71:6835","nonce":388207814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6835/1052606547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1052606547},{"type":"v1","addr":"172.21.15.142:6835","nonce":1052606547}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.142:6837/2870289019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2870289019},{"type":"v1","addr":"172.21.15.142:6837","nonce":2870289019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.71:6837/628016199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":628016199},{"type":"v1","addr":"172.21.15.71:6837","nonce":628016199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.142:6839/321675421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":321675421},{"type":"v1","addr":"172.21.15.142:6839","nonce":321675421}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/1268086493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1268086493},{"type":"v1","addr":"172.21.15.71:6839","nonce":1268086493}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-02T15:39:44.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.958+0000 7ff263600700 1 -- 172.21.15.1:0/4210301728 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff23c0a0690 msgr2=0x7ff23c0a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:44.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.958+0000 7ff263600700 1 --2- 172.21.15.1:0/4210301728 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff23c0a0690 0x7ff23c0a2b50 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff24c005fd0 tx=0x7ff24c009310 comp rx=0 tx=0).stop 2024-02-02T15:39:44.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.958+0000 7ff263600700 1 -- 172.21.15.1:0/4210301728 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff25c106040 msgr2=0x7ff25c199530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:44.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.958+0000 7ff263600700 1 --2- 172.21.15.1:0/4210301728 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff25c106040 0x7ff25c199530 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7ff244009fd0 tx=0x7ff24400fa00 comp rx=0 tx=0).stop 2024-02-02T15:39:44.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.958+0000 7ff263600700 1 -- 172.21.15.1:0/4210301728 shutdown_connections 2024-02-02T15:39:44.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.958+0000 7ff263600700 1 --2- 172.21.15.1:0/4210301728 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff23c0a0690 0x7ff23c0a2b50 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:44.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.958+0000 7ff263600700 1 --2- 172.21.15.1:0/4210301728 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff25c19d990 0x7ff25c19de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:44.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.958+0000 7ff263600700 1 --2- 172.21.15.1:0/4210301728 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff25c100f20 0x7ff25c198ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:44.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.958+0000 7ff263600700 1 --2- 172.21.15.1:0/4210301728 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff25c106040 0x7ff25c199530 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:44.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.958+0000 7ff263600700 1 -- 172.21.15.1:0/4210301728 >> 172.21.15.1:0/4210301728 conn(0x7ff25c0fc7a0 msgr2=0x7ff25c0fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:44.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.958+0000 7ff263600700 1 -- 172.21.15.1:0/4210301728 shutdown_connections 2024-02-02T15:39:44.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:44.958+0000 7ff263600700 1 -- 172.21.15.1:0/4210301728 wait complete. 2024-02-02T15:39:44.960 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 52 2024-02-02T15:39:44.978 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-02-02T15:39:45.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.192+0000 7fc8dbe00700 1 -- 172.21.15.1:0/1517271853 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8d4100ff0 msgr2=0x7fc8d41013d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:45.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.192+0000 7fc8dbe00700 1 --2- 172.21.15.1:0/1517271853 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8d4100ff0 0x7fc8d41013d0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fc8c40098f0 tx=0x7fc8c4009c00 comp rx=0 tx=0).stop 2024-02-02T15:39:45.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.192+0000 7fc8dbe00700 1 -- 172.21.15.1:0/1517271853 shutdown_connections 2024-02-02T15:39:45.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.192+0000 7fc8dbe00700 1 --2- 172.21.15.1:0/1517271853 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8d4106110 0x7fc8d410dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.192+0000 7fc8dbe00700 1 --2- 172.21.15.1:0/1517271853 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8d4100ff0 0x7fc8d41013d0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.192+0000 7fc8dbe00700 1 --2- 172.21.15.1:0/1517271853 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc8d41019a0 0x7fc8d41059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.192+0000 7fc8dbe00700 1 -- 172.21.15.1:0/1517271853 >> 172.21.15.1:0/1517271853 conn(0x7fc8d40fc810 msgr2=0x7fc8d40fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:45.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.192+0000 7fc8dbe00700 1 -- 172.21.15.1:0/1517271853 shutdown_connections 2024-02-02T15:39:45.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.192+0000 7fc8dbe00700 1 -- 172.21.15.1:0/1517271853 wait complete. 2024-02-02T15:39:45.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.194+0000 7fc8dbe00700 1 Processor -- start 2024-02-02T15:39:45.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.194+0000 7fc8dbe00700 1 -- start start 2024-02-02T15:39:45.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.194+0000 7fc8dbe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8d4100ff0 0x7fc8d41990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:45.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.194+0000 7fc8dbe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc8d4106110 0x7fc8d4199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:45.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.194+0000 7fc8dbe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8d419da80 0x7fc8d419df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:45.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.194+0000 7fc8dbe00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d419e500 con 0x7fc8d4106110 2024-02-02T15:39:45.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.194+0000 7fc8dbe00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d419e670 con 0x7fc8d419da80 2024-02-02T15:39:45.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.194+0000 7fc8dbe00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d419e830 con 0x7fc8d4100ff0 2024-02-02T15:39:45.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.195+0000 7fc8d9400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8d4100ff0 0x7fc8d41990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:45.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.195+0000 7fc8d8a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc8d4106110 0x7fc8d4199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:45.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.195+0000 7fc8d9400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8d4100ff0 0x7fc8d41990e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:33946/0 (socket says 172.21.15.1:33946) 2024-02-02T15:39:45.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.195+0000 7fc8d9400700 1 -- 172.21.15.1:0/68422159 learned_addr learned my addr 172.21.15.1:0/68422159 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:39:45.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.195+0000 7fc8d9e00700 1 --2- 172.21.15.1:0/68422159 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8d419da80 0x7fc8d419df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:45.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.195+0000 7fc8d9400700 1 -- 172.21.15.1:0/68422159 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8d419da80 msgr2=0x7fc8d419df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:45.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.195+0000 7fc8d9400700 1 --2- 172.21.15.1:0/68422159 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8d419da80 0x7fc8d419df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.195+0000 7fc8d9400700 1 -- 172.21.15.1:0/68422159 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc8d4106110 msgr2=0x7fc8d4199620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:45.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.195+0000 7fc8d9400700 1 --2- 172.21.15.1:0/68422159 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc8d4106110 0x7fc8d4199620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.195+0000 7fc8d9400700 1 -- 172.21.15.1:0/68422159 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8c4009520 con 0x7fc8d4100ff0 2024-02-02T15:39:45.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.195+0000 7fc8d9400700 1 --2- 172.21.15.1:0/68422159 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8d4100ff0 0x7fc8d41990e0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fc8c4009810 tx=0x7fc8c400fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:45.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.196+0000 7fc8d2000700 1 -- 172.21.15.1:0/68422159 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c401c070 con 0x7fc8d4100ff0 2024-02-02T15:39:45.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.196+0000 7fc8d2000700 1 -- 172.21.15.1:0/68422159 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc8c400bbd0 con 0x7fc8d4100ff0 2024-02-02T15:39:45.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.196+0000 7fc8dbe00700 1 -- 172.21.15.1:0/68422159 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8d41a5d30 con 0x7fc8d4100ff0 2024-02-02T15:39:45.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.196+0000 7fc8d2000700 1 -- 172.21.15.1:0/68422159 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c4004930 con 0x7fc8d4100ff0 2024-02-02T15:39:45.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.196+0000 7fc8dbe00700 1 -- 172.21.15.1:0/68422159 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8d41a61b0 con 0x7fc8d4100ff0 2024-02-02T15:39:45.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.199+0000 7fc8d2000700 1 -- 172.21.15.1:0/68422159 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7fc8c4020070 con 0x7fc8d4100ff0 2024-02-02T15:39:45.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.201+0000 7fc8ef985b80 1 Processor -- start 2024-02-02T15:39:45.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.201+0000 7fc8ef985b80 1 -- start start 2024-02-02T15:39:45.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.201+0000 7fc8d2000700 1 --2- 172.21.15.1:0/68422159 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fc8b40a0690 0x7fc8b40a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:45.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.201+0000 7fc8ef985b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ee4529010 0x564ee45293f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:45.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.201+0000 7fc8d2000700 1 -- 172.21.15.1:0/68422159 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fc8c40ccdf0 con 0x7fc8d4100ff0 2024-02-02T15:39:45.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.201+0000 7fc8ef985b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ee4529930 0x564ee45342a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:45.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.201+0000 7fc8ef985b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ee45347e0 0x564ee4536bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:45.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.201+0000 7fc8ef985b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ee449d4c0 con 0x564ee4529930 2024-02-02T15:39:45.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.201+0000 7fc8ef985b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ee4481aa0 con 0x564ee4529010 2024-02-02T15:39:45.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.201+0000 7fc8ef985b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ee447f460 con 0x564ee45347e0 2024-02-02T15:39:45.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.201+0000 7fc8d9e00700 1 --2- 172.21.15.1:0/68422159 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fc8b40a0690 0x7fc8b40a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:45.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.202+0000 7fc8d8a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ee4529930 0x564ee45342a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:45.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.202+0000 7fc8d8a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ee4529930 0x564ee45342a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55664/0 (socket says 172.21.15.1:55664) 2024-02-02T15:39:45.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.202+0000 7fc8d8a00700 1 -- 172.21.15.1:0/3071288409 learned_addr learned my addr 172.21.15.1:0/3071288409 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:39:45.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.202+0000 7fc8d9400700 1 --2- 172.21.15.1:0/3071288409 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ee4529010 0x564ee45293f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:45.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.202+0000 7fc8d9e00700 1 --2- 172.21.15.1:0/3071288409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ee45347e0 0x564ee4536bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:45.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.202+0000 7fc8d8a00700 1 -- 172.21.15.1:0/3071288409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ee45347e0 msgr2=0x564ee4536bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:45.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.202+0000 7fc8d8a00700 1 --2- 172.21.15.1:0/3071288409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ee45347e0 0x564ee4536bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.202+0000 7fc8d8a00700 1 -- 172.21.15.1:0/3071288409 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ee4529010 msgr2=0x564ee45293f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:45.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.202+0000 7fc8d8a00700 1 --2- 172.21.15.1:0/3071288409 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ee4529010 0x564ee45293f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.202+0000 7fc8d8a00700 1 -- 172.21.15.1:0/3071288409 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564ee42fc3b0 con 0x564ee4529930 2024-02-02T15:39:45.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.202+0000 7fc8d9e00700 1 --2- 172.21.15.1:0/3071288409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ee45347e0 0x564ee4536bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:39:45.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.202+0000 7fc8d9400700 1 --2- 172.21.15.1:0/3071288409 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ee4529010 0x564ee45293f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:39:45.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.202+0000 7fc8d9e00700 1 --2- 172.21.15.1:0/68422159 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fc8b40a0690 0x7fc8b40a2b50 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc00f540 tx=0x7fc8cc009920 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:45.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.202+0000 7fc8d8a00700 1 --2- 172.21.15.1:0/3071288409 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ee4529930 0x564ee45342a0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc009cf0 tx=0x7fc8bc011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d1b0c44c11a410c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:45.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.203+0000 7fc8ca600700 1 -- 172.21.15.1:0/3071288409 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8bc011d80 con 0x564ee4529930 2024-02-02T15:39:45.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.203+0000 7fc8ca600700 1 -- 172.21.15.1:0/3071288409 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc8bc004510 con 0x564ee4529930 2024-02-02T15:39:45.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.203+0000 7fc8ca600700 1 -- 172.21.15.1:0/3071288409 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8bc005280 con 0x564ee4529930 2024-02-02T15:39:45.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.203+0000 7fc8ef985b80 1 -- 172.21.15.1:0/3071288409 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ee4529930 msgr2=0x564ee45342a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:45.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.203+0000 7fc8ef985b80 1 --2- 172.21.15.1:0/3071288409 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ee4529930 0x564ee45342a0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc009cf0 tx=0x7fc8bc011440 comp rx=0 tx=0).stop 2024-02-02T15:39:45.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.203+0000 7fc8ef985b80 1 -- 172.21.15.1:0/3071288409 shutdown_connections 2024-02-02T15:39:45.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.203+0000 7fc8ef985b80 1 --2- 172.21.15.1:0/3071288409 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ee45347e0 0x564ee4536bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.203+0000 7fc8ef985b80 1 --2- 172.21.15.1:0/3071288409 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ee4529010 0x564ee45293f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.203+0000 7fc8ef985b80 1 --2- 172.21.15.1:0/3071288409 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ee4529930 0x564ee45342a0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.203+0000 7fc8ef985b80 1 -- 172.21.15.1:0/3071288409 >> 172.21.15.1:0/3071288409 conn(0x564ee4488db0 msgr2=0x564ee45253b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:45.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.204+0000 7fc8ef985b80 1 -- 172.21.15.1:0/3071288409 shutdown_connections 2024-02-02T15:39:45.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.204+0000 7fc8ef985b80 1 -- 172.21.15.1:0/3071288409 wait complete. 2024-02-02T15:39:45.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.204+0000 7fc8ef985b80 1 -- start start 2024-02-02T15:39:45.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.204+0000 7fc8ef985b80 1 Processor -- start 2024-02-02T15:39:45.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.205+0000 7fc8ef985b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ee4529010 0x564ee45341b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:45.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.205+0000 7fc8ef985b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ee4529930 0x564ee45313c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:45.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.205+0000 7fc8ef985b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ee45347e0 0x564ee45319d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:45.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.205+0000 7fc8ef985b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ee447ad50 con 0x564ee45347e0 2024-02-02T15:39:45.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.205+0000 7fc8ef985b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ee4416d80 con 0x564ee4529930 2024-02-02T15:39:45.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.205+0000 7fc8ef985b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ee4416a20 con 0x564ee4529010 2024-02-02T15:39:45.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.205+0000 7fc8ef985b80 10 client.0 _ll_register_callbacks cb 0x564ee4418290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:39:45.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.205+0000 7fc8d9400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ee4529010 0x564ee45341b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:45.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.205+0000 7fc8d9e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ee45347e0 0x564ee45319d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:45.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.205+0000 7fc8d8a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ee4529930 0x564ee45313c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:45.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.205+0000 7fc8d9400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ee4529010 0x564ee45341b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:33968/0 (socket says 172.21.15.1:33968) 2024-02-02T15:39:45.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.205+0000 7fc8d9400700 1 -- 172.21.15.1:0/3847159864 learned_addr learned my addr 172.21.15.1:0/3847159864 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:39:45.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.205+0000 7fc8d8a00700 1 -- 172.21.15.1:0/3847159864 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ee4529010 msgr2=0x564ee45341b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:45.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.205+0000 7fc8d8a00700 1 --2- 172.21.15.1:0/3847159864 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ee4529010 0x564ee45341b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.205+0000 7fc8d8a00700 1 -- 172.21.15.1:0/3847159864 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ee45347e0 msgr2=0x564ee45319d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:45.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.205+0000 7fc8d8a00700 1 --2- 172.21.15.1:0/3847159864 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ee45347e0 0x564ee45319d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.206+0000 7fc8d8a00700 1 -- 172.21.15.1:0/3847159864 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8d41a61b0 con 0x564ee4529930 2024-02-02T15:39:45.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.206+0000 7fc8d9400700 1 --2- 172.21.15.1:0/3847159864 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ee4529010 0x564ee45341b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:39:45.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.206+0000 7fc8d8a00700 1 --2- 172.21.15.1:0/3847159864 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ee4529930 0x564ee45313c0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc009cf0 tx=0x7fc8bc004bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27a2f48ed84048a5 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:45.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.206+0000 7fc8c2a00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:39:45.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.206+0000 7fc8c2a00700 1 -- 172.21.15.1:0/3847159864 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8bc011e10 con 0x564ee4529930 2024-02-02T15:39:45.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.206+0000 7fc8c2a00700 1 -- 172.21.15.1:0/3847159864 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fc8bc004050 con 0x564ee4529930 2024-02-02T15:39:45.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.206+0000 7fc8ef985b80 1 -- 172.21.15.1:0/3847159864 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564ee451d0f0 con 0x564ee4529930 2024-02-02T15:39:45.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.206+0000 7fc8c2a00700 1 -- 172.21.15.1:0/3847159864 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc8bc004250 con 0x564ee4529930 2024-02-02T15:39:45.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.206+0000 7fc8ef985b80 10 client.28973 fetch_fsmap learned FSMap version 52 2024-02-02T15:39:45.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.206+0000 7fc8ef985b80 1 -- 172.21.15.1:0/3847159864 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x7fc8bc009920 con 0x564ee4529930 2024-02-02T15:39:45.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.207+0000 7fc8c2a00700 1 -- 172.21.15.1:0/3847159864 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564ee451d0f0 con 0x564ee4529930 2024-02-02T15:39:45.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.207+0000 7fc8c2a00700 1 -- 172.21.15.1:0/3847159864 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 52) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fc8bc0051b0 con 0x564ee4529930 2024-02-02T15:39:45.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.207+0000 7fc8ef985b80 10 client.28973 fetch_fsmap finished waiting for FSMap version 52 2024-02-02T15:39:45.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.207+0000 7fc8ef985b80 10 client.28973 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-02-02T15:39:45.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.207+0000 7fc8ef985b80 20 client.28973 populate_metadata read hostname 'smithi001' 2024-02-02T15:39:45.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.207+0000 7fc8ef985b80 1 --2- 172.21.15.1:0/3847159864 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x564ee4544060 0x564ee4546480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:45.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.207+0000 7fc8ef985b80 4 client.28973 mds_command: new command op to 24502 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:39:45.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.207+0000 7fc8ef985b80 1 -- 172.21.15.1:0/3847159864 --> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564ee4473dd0 con 0x564ee4544060 2024-02-02T15:39:45.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.208+0000 7fc8d9400700 1 --2- 172.21.15.1:0/3847159864 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x564ee4544060 0x564ee4546480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:45.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.208+0000 7fc8d9400700 1 --2- 172.21.15.1:0/3847159864 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x564ee4544060 0x564ee4546480 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=da0f89dbfc0cddc2 server_cookie=7750ef35e7cf8a3e in_seq=0 out_seq=0 2024-02-02T15:39:45.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.208+0000 7fc8c2a00700 10 client.28973 ms_handle_connect on v2:172.21.15.142:6834/1052606547 2024-02-02T15:39:45.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.209+0000 7fc8c2a00700 1 -- 172.21.15.1:0/3847159864 <== mds.4 v2:172.21.15.142:6834/1052606547 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x564ee4473dd0 con 0x564ee4544060 2024-02-02T15:39:45.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.209+0000 7fc8c2a00700 10 client.28973 handle_command_reply: tid=0 2024-02-02T15:39:45.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.209+0000 7fc8ef985b80 1 client.28973 shutdown 2024-02-02T15:39:45.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.209+0000 7fc8ef985b80 1 -- 172.21.15.1:0/3847159864 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ee4529930 msgr2=0x564ee45313c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:45.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.209+0000 7fc8ef985b80 1 --2- 172.21.15.1:0/3847159864 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ee4529930 0x564ee45313c0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc009cf0 tx=0x7fc8bc004bc0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.209+0000 7fc8ef985b80 1 -- 172.21.15.1:0/3847159864 shutdown_connections 2024-02-02T15:39:45.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.209+0000 7fc8ef985b80 1 --2- 172.21.15.1:0/3847159864 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x564ee4544060 0x564ee4546480 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.209+0000 7fc8ef985b80 1 --2- 172.21.15.1:0/3847159864 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ee45347e0 0x564ee45319d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.209+0000 7fc8ef985b80 1 --2- 172.21.15.1:0/3847159864 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ee4529010 0x564ee45341b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.209+0000 7fc8c2a00700 0 client.28973 ms_handle_reset on v2:172.21.15.142:6834/1052606547 2024-02-02T15:39:45.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.209+0000 7fc8ef985b80 1 --2- 172.21.15.1:0/3847159864 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ee4529930 0x564ee45313c0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.209+0000 7fc8ef985b80 1 -- 172.21.15.1:0/3847159864 >> 172.21.15.1:0/3847159864 conn(0x564ee4488db0 msgr2=0x564ee4524fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:45.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.209+0000 7fc8ef985b80 1 -- 172.21.15.1:0/3847159864 shutdown_connections 2024-02-02T15:39:45.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.209+0000 7fc8ef985b80 1 -- 172.21.15.1:0/3847159864 wait complete. 2024-02-02T15:39:45.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.210+0000 7fc8ef985b80 20 client.28973 trim_cache size 0 max 16384 2024-02-02T15:39:45.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.222+0000 7fc8ef985b80 1 Processor -- start 2024-02-02T15:39:45.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.222+0000 7fc8ef985b80 1 -- start start 2024-02-02T15:39:45.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.223+0000 7fc8ef985b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ee4488db0 0x564ee452bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:45.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.223+0000 7fc8ef985b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ee4529010 0x564ee4533ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:45.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.223+0000 7fc8ef985b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ee4529930 0x564ee4530c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:45.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.223+0000 7fc8ef985b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ee42fe6a0 con 0x564ee4488db0 2024-02-02T15:39:45.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.223+0000 7fc8ef985b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ee45241a0 con 0x564ee4529930 2024-02-02T15:39:45.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.223+0000 7fc8ef985b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ee452d250 con 0x564ee4529010 2024-02-02T15:39:45.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.223+0000 7fc8d9400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ee4488db0 0x564ee452bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:45.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.223+0000 7fc8d9400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ee4488db0 0x564ee452bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55672/0 (socket says 172.21.15.1:55672) 2024-02-02T15:39:45.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.223+0000 7fc8d9400700 1 -- 172.21.15.1:0/2709827310 learned_addr learned my addr 172.21.15.1:0/2709827310 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:39:45.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.223+0000 7fc8d8a00700 1 --2- 172.21.15.1:0/2709827310 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ee4529010 0x564ee4533ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:45.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.223+0000 7fc8d9e00700 1 --2- 172.21.15.1:0/2709827310 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ee4529930 0x564ee4530c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:45.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.223+0000 7fc8d9400700 1 -- 172.21.15.1:0/2709827310 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ee4529010 msgr2=0x564ee4533ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:45.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.223+0000 7fc8d9400700 1 --2- 172.21.15.1:0/2709827310 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ee4529010 0x564ee4533ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.223+0000 7fc8d9400700 1 -- 172.21.15.1:0/2709827310 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ee4529930 msgr2=0x564ee4530c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:45.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.223+0000 7fc8d9400700 1 --2- 172.21.15.1:0/2709827310 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ee4529930 0x564ee4530c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.223+0000 7fc8d9400700 1 -- 172.21.15.1:0/2709827310 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564ee42a8b60 con 0x564ee4488db0 2024-02-02T15:39:45.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.224+0000 7fc8d9400700 1 --2- 172.21.15.1:0/2709827310 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ee4488db0 0x564ee452bf40 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fc8c40964a0 tx=0x7fc8c402e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a79157afcfe4e04d server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:45.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.224+0000 7fc8cb000700 1 -- 172.21.15.1:0/2709827310 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c409ddd0 con 0x564ee4488db0 2024-02-02T15:39:45.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.224+0000 7fc8cb000700 1 -- 172.21.15.1:0/2709827310 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc8c40a1f80 con 0x564ee4488db0 2024-02-02T15:39:45.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.224+0000 7fc8cb000700 1 -- 172.21.15.1:0/2709827310 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c40996f0 con 0x564ee4488db0 2024-02-02T15:39:45.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.224+0000 7fc8ef985b80 1 -- 172.21.15.1:0/2709827310 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ee4488db0 msgr2=0x564ee452bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:45.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.224+0000 7fc8ef985b80 1 --2- 172.21.15.1:0/2709827310 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ee4488db0 0x564ee452bf40 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fc8c40964a0 tx=0x7fc8c402e040 comp rx=0 tx=0).stop 2024-02-02T15:39:45.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.224+0000 7fc8ef985b80 1 -- 172.21.15.1:0/2709827310 shutdown_connections 2024-02-02T15:39:45.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.224+0000 7fc8ef985b80 1 --2- 172.21.15.1:0/2709827310 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ee4529930 0x564ee4530c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.224+0000 7fc8ef985b80 1 --2- 172.21.15.1:0/2709827310 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ee4488db0 0x564ee452bf40 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.224+0000 7fc8ef985b80 1 --2- 172.21.15.1:0/2709827310 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ee4529010 0x564ee4533ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.224+0000 7fc8ef985b80 1 -- 172.21.15.1:0/2709827310 >> 172.21.15.1:0/2709827310 conn(0x564ee4536df0 msgr2=0x564ee453f900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:45.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.224+0000 7fc8ef985b80 1 -- 172.21.15.1:0/2709827310 shutdown_connections 2024-02-02T15:39:45.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.225+0000 7fc8ef985b80 1 -- 172.21.15.1:0/2709827310 wait complete. 2024-02-02T15:39:45.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.225+0000 7fc8ef985b80 1 -- start start 2024-02-02T15:39:45.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.225+0000 7fc8ef985b80 1 Processor -- start 2024-02-02T15:39:45.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.226+0000 7fc8ef985b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ee4488db0 0x564ee45374f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:45.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.226+0000 7fc8ef985b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ee4529010 0x564ee4537a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:45.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.226+0000 7fc8ef985b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ee4529930 0x564ee4594b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:45.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.226+0000 7fc8ef985b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ee45381a0 con 0x564ee4529010 2024-02-02T15:39:45.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.226+0000 7fc8ef985b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ee45950b0 con 0x564ee4488db0 2024-02-02T15:39:45.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.226+0000 7fc8ef985b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ee4595270 con 0x564ee4529930 2024-02-02T15:39:45.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.226+0000 7fc8ef985b80 10 client.0 _ll_register_callbacks cb 0x564ee4418290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:39:45.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.226+0000 7fc8d9400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ee4488db0 0x564ee45374f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:45.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.226+0000 7fc8d9400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ee4488db0 0x564ee45374f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:41308/0 (socket says 172.21.15.1:41308) 2024-02-02T15:39:45.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.226+0000 7fc8d9400700 1 -- 172.21.15.1:0/317524142 learned_addr learned my addr 172.21.15.1:0/317524142 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:39:45.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.226+0000 7fc8d9e00700 1 --2- 172.21.15.1:0/317524142 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ee4529930 0x564ee4594b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:45.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.226+0000 7fc8d8a00700 1 --2- 172.21.15.1:0/317524142 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ee4529010 0x564ee4537a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:45.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.226+0000 7fc8d9400700 1 -- 172.21.15.1:0/317524142 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ee4529930 msgr2=0x564ee4594b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:45.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.226+0000 7fc8d9400700 1 --2- 172.21.15.1:0/317524142 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ee4529930 0x564ee4594b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.226+0000 7fc8d9400700 1 -- 172.21.15.1:0/317524142 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ee4529010 msgr2=0x564ee4537a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:45.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.226+0000 7fc8d9400700 1 --2- 172.21.15.1:0/317524142 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ee4529010 0x564ee4537a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.226+0000 7fc8d9400700 1 -- 172.21.15.1:0/317524142 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8d41a5d30 con 0x564ee4488db0 2024-02-02T15:39:45.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.226+0000 7fc8d9400700 1 --2- 172.21.15.1:0/317524142 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ee4488db0 0x564ee45374f0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fc8c40a0050 tx=0x7fc8c4009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=432123a2dcd1bfed server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:45.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.226+0000 7fc8c2a00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:39:45.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.227+0000 7fc8c2a00700 1 -- 172.21.15.1:0/317524142 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c4099690 con 0x564ee4488db0 2024-02-02T15:39:45.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.227+0000 7fc8c2a00700 1 -- 172.21.15.1:0/317524142 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fc8c409a200 con 0x564ee4488db0 2024-02-02T15:39:45.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.227+0000 7fc8ef985b80 1 -- 172.21.15.1:0/317524142 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc8bc004510 con 0x564ee4488db0 2024-02-02T15:39:45.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.227+0000 7fc8c2a00700 1 -- 172.21.15.1:0/317524142 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc8c409a3c0 con 0x564ee4488db0 2024-02-02T15:39:45.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.227+0000 7fc8ef985b80 10 client.28979 fetch_fsmap learned FSMap version 52 2024-02-02T15:39:45.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.227+0000 7fc8ef985b80 1 -- 172.21.15.1:0/317524142 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x564ee4440ea0 con 0x564ee4488db0 2024-02-02T15:39:45.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.227+0000 7fc8c2a00700 1 -- 172.21.15.1:0/317524142 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc8bc004510 con 0x564ee4488db0 2024-02-02T15:39:45.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.227+0000 7fc8c2a00700 1 -- 172.21.15.1:0/317524142 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 52) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fc8c4005040 con 0x564ee4488db0 2024-02-02T15:39:45.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.227+0000 7fc8ef985b80 10 client.28979 fetch_fsmap finished waiting for FSMap version 52 2024-02-02T15:39:45.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.227+0000 7fc8ef985b80 10 client.28979 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-02-02T15:39:45.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.227+0000 7fc8ef985b80 20 client.28979 populate_metadata read hostname 'smithi001' 2024-02-02T15:39:45.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.227+0000 7fc8ef985b80 1 --2- 172.21.15.1:0/317524142 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x564ee45347e0 0x564ee4598520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:45.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.227+0000 7fc8ef985b80 4 client.28979 mds_command: new command op to 24502 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:39:45.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.227+0000 7fc8ef985b80 1 -- 172.21.15.1:0/317524142 --> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564ee4439f50 con 0x564ee45347e0 2024-02-02T15:39:45.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.228+0000 7fc8d8a00700 1 --2- 172.21.15.1:0/317524142 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x564ee45347e0 0x564ee4598520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:45.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.228+0000 7fc8d8a00700 1 --2- 172.21.15.1:0/317524142 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x564ee45347e0 0x564ee4598520 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=beafa4587fe19bde server_cookie=3d7cb55016d55d24 in_seq=0 out_seq=0 2024-02-02T15:39:45.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.228+0000 7fc8c2a00700 10 client.28979 ms_handle_connect on v2:172.21.15.142:6834/1052606547 2024-02-02T15:39:45.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.228+0000 7fc8c2a00700 1 -- 172.21.15.1:0/317524142 <== mds.4 v2:172.21.15.142:6834/1052606547 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564ee4439f50 con 0x564ee45347e0 2024-02-02T15:39:45.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.228+0000 7fc8c2a00700 10 client.28979 handle_command_reply: tid=0 2024-02-02T15:39:45.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.228+0000 7fc8ef985b80 1 client.28979 shutdown 2024-02-02T15:39:45.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.228+0000 7fc8ef985b80 1 -- 172.21.15.1:0/317524142 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ee4488db0 msgr2=0x564ee45374f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:45.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.228+0000 7fc8ef985b80 1 --2- 172.21.15.1:0/317524142 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ee4488db0 0x564ee45374f0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fc8c40a0050 tx=0x7fc8c4009c00 comp rx=0 tx=0).stop 2024-02-02T15:39:45.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.228+0000 7fc8ef985b80 1 -- 172.21.15.1:0/317524142 shutdown_connections 2024-02-02T15:39:45.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.228+0000 7fc8ef985b80 1 --2- 172.21.15.1:0/317524142 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x564ee45347e0 0x564ee4598520 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.228+0000 7fc8ef985b80 1 --2- 172.21.15.1:0/317524142 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ee4529930 0x564ee4594b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.228+0000 7fc8ef985b80 1 --2- 172.21.15.1:0/317524142 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ee4488db0 0x564ee45374f0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.228+0000 7fc8c2a00700 0 client.28979 ms_handle_reset on v2:172.21.15.142:6834/1052606547 2024-02-02T15:39:45.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.228+0000 7fc8ef985b80 1 --2- 172.21.15.1:0/317524142 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ee4529010 0x564ee4537a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.228+0000 7fc8ef985b80 1 -- 172.21.15.1:0/317524142 >> 172.21.15.1:0/317524142 conn(0x564ee4536df0 msgr2=0x564ee4530c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:45.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.229+0000 7fc8ef985b80 1 -- 172.21.15.1:0/317524142 shutdown_connections 2024-02-02T15:39:45.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.229+0000 7fc8ef985b80 1 -- 172.21.15.1:0/317524142 wait complete. 2024-02-02T15:39:45.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.229+0000 7fc8ef985b80 20 client.28979 trim_cache size 0 max 16384 2024-02-02T15:39:45.243 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:39:45.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.229+0000 7fc8dbe00700 1 -- 172.21.15.1:0/68422159 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fc8b40a0690 msgr2=0x7fc8b40a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:45.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.229+0000 7fc8dbe00700 1 --2- 172.21.15.1:0/68422159 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fc8b40a0690 0x7fc8b40a2b50 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc00f540 tx=0x7fc8cc009920 comp rx=0 tx=0).stop 2024-02-02T15:39:45.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.229+0000 7fc8dbe00700 1 -- 172.21.15.1:0/68422159 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8d4100ff0 msgr2=0x7fc8d41990e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:45.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.229+0000 7fc8dbe00700 1 --2- 172.21.15.1:0/68422159 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8d4100ff0 0x7fc8d41990e0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fc8c4009810 tx=0x7fc8c400fa00 comp rx=0 tx=0).stop 2024-02-02T15:39:45.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.229+0000 7fc8dbe00700 1 -- 172.21.15.1:0/68422159 shutdown_connections 2024-02-02T15:39:45.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.229+0000 7fc8dbe00700 1 --2- 172.21.15.1:0/68422159 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fc8b40a0690 0x7fc8b40a2b50 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.229+0000 7fc8dbe00700 1 --2- 172.21.15.1:0/68422159 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc8d419da80 0x7fc8d419df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.230+0000 7fc8dbe00700 1 --2- 172.21.15.1:0/68422159 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc8d4100ff0 0x7fc8d41990e0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.230+0000 7fc8dbe00700 1 --2- 172.21.15.1:0/68422159 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc8d4106110 0x7fc8d4199620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.230+0000 7fc8dbe00700 1 -- 172.21.15.1:0/68422159 >> 172.21.15.1:0/68422159 conn(0x7fc8d40fc810 msgr2=0x7fc8d410cf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:45.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.230+0000 7fc8dbe00700 1 -- 172.21.15.1:0/68422159 shutdown_connections 2024-02-02T15:39:45.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.230+0000 7fc8dbe00700 1 -- 172.21.15.1:0/68422159 wait complete. 2024-02-02T15:39:45.247 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-02-02T15:39:45.273 INFO:tasks.workunit.client.0.smithi001.stdout: 1 694183 5.37 MB/sec execute 559 sec latency 156.681 ms 2024-02-02T15:39:45.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.447+0000 7fbc56a00700 1 -- 172.21.15.1:0/2636530096 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc501019a0 msgr2=0x7fbc501059e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:45.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.447+0000 7fbc56a00700 1 --2- 172.21.15.1:0/2636530096 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc501019a0 0x7fbc501059e0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fbc38009990 tx=0x7fbc38009ca0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.448+0000 7fbc56a00700 1 -- 172.21.15.1:0/2636530096 shutdown_connections 2024-02-02T15:39:45.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.448+0000 7fbc56a00700 1 --2- 172.21.15.1:0/2636530096 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc50106110 0x7fbc5010dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.448+0000 7fbc56a00700 1 --2- 172.21.15.1:0/2636530096 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc50100ff0 0x7fbc501013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.448+0000 7fbc56a00700 1 --2- 172.21.15.1:0/2636530096 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc501019a0 0x7fbc501059e0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.448+0000 7fbc56a00700 1 -- 172.21.15.1:0/2636530096 >> 172.21.15.1:0/2636530096 conn(0x7fbc500fc810 msgr2=0x7fbc500fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:45.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.448+0000 7fbc56a00700 1 -- 172.21.15.1:0/2636530096 shutdown_connections 2024-02-02T15:39:45.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.448+0000 7fbc56a00700 1 -- 172.21.15.1:0/2636530096 wait complete. 2024-02-02T15:39:45.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.449+0000 7fbc56a00700 1 Processor -- start 2024-02-02T15:39:45.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.449+0000 7fbc56a00700 1 -- start start 2024-02-02T15:39:45.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.449+0000 7fbc56a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc50100ff0 0x7fbc50073040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:45.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.449+0000 7fbc56a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc50106110 0x7fbc50073580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:45.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.449+0000 7fbc56a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc50073ac0 0x7fbc501a5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:45.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.449+0000 7fbc56a00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc501a56a0 con 0x7fbc50100ff0 2024-02-02T15:39:45.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.449+0000 7fbc56a00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc501a5810 con 0x7fbc50106110 2024-02-02T15:39:45.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.449+0000 7fbc56a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc501a59d0 con 0x7fbc50073ac0 2024-02-02T15:39:45.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.449+0000 7fbc4fe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc50100ff0 0x7fbc50073040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:45.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.449+0000 7fbc4fe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc50100ff0 0x7fbc50073040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55708/0 (socket says 172.21.15.1:55708) 2024-02-02T15:39:45.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.449+0000 7fbc4fe00700 1 -- 172.21.15.1:0/1283589681 learned_addr learned my addr 172.21.15.1:0/1283589681 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:39:45.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.450+0000 7fbc54a00700 1 --2- 172.21.15.1:0/1283589681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc50073ac0 0x7fbc501a5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:45.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.450+0000 7fbc4f400700 1 --2- 172.21.15.1:0/1283589681 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc50106110 0x7fbc50073580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:45.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.450+0000 7fbc4fe00700 1 -- 172.21.15.1:0/1283589681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc50073ac0 msgr2=0x7fbc501a5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:45.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.450+0000 7fbc4fe00700 1 --2- 172.21.15.1:0/1283589681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc50073ac0 0x7fbc501a5010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.450+0000 7fbc4fe00700 1 -- 172.21.15.1:0/1283589681 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc50106110 msgr2=0x7fbc50073580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:45.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.450+0000 7fbc4fe00700 1 --2- 172.21.15.1:0/1283589681 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc50106110 0x7fbc50073580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.450+0000 7fbc4fe00700 1 -- 172.21.15.1:0/1283589681 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc380095f0 con 0x7fbc50100ff0 2024-02-02T15:39:45.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.450+0000 7fbc4fe00700 1 --2- 172.21.15.1:0/1283589681 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc50100ff0 0x7fbc50073040 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fbc4000e9a0 tx=0x7fbc4000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:45.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.450+0000 7fbc4cc00700 1 -- 172.21.15.1:0/1283589681 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc40009790 con 0x7fbc50100ff0 2024-02-02T15:39:45.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.450+0000 7fbc4cc00700 1 -- 172.21.15.1:0/1283589681 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc40010430 con 0x7fbc50100ff0 2024-02-02T15:39:45.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.450+0000 7fbc4cc00700 1 -- 172.21.15.1:0/1283589681 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc40018680 con 0x7fbc50100ff0 2024-02-02T15:39:45.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.450+0000 7fbc56a00700 1 -- 172.21.15.1:0/1283589681 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc501a5cb0 con 0x7fbc50100ff0 2024-02-02T15:39:45.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.451+0000 7fbc56a00700 1 -- 172.21.15.1:0/1283589681 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc501a6170 con 0x7fbc50100ff0 2024-02-02T15:39:45.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.454+0000 7fbc4cc00700 1 -- 172.21.15.1:0/1283589681 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7fbc400105a0 con 0x7fbc50100ff0 2024-02-02T15:39:45.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.455+0000 7fbc4cc00700 1 --2- 172.21.15.1:0/1283589681 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fbc300a0690 0x7fbc300a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:45.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.455+0000 7fbc6a613b80 1 Processor -- start 2024-02-02T15:39:45.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.455+0000 7fbc4cc00700 1 -- 172.21.15.1:0/1283589681 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fbc40014070 con 0x7fbc50100ff0 2024-02-02T15:39:45.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.455+0000 7fbc4f400700 1 --2- 172.21.15.1:0/1283589681 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fbc300a0690 0x7fbc300a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:45.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.455+0000 7fbc6a613b80 1 -- start start 2024-02-02T15:39:45.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.456+0000 7fbc6a613b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a7ccbee010 0x55a7ccbee3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:45.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.456+0000 7fbc6a613b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7ccbee930 0x55a7ccbf92a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:45.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.456+0000 7fbc6a613b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7ccbf97e0 0x55a7ccbfbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:45.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.456+0000 7fbc6a613b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7ccb624c0 con 0x55a7ccbee930 2024-02-02T15:39:45.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.456+0000 7fbc6a613b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7ccb46aa0 con 0x55a7ccbee010 2024-02-02T15:39:45.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.456+0000 7fbc6a613b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7ccb44460 con 0x55a7ccbf97e0 2024-02-02T15:39:45.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.456+0000 7fbc4f400700 1 --2- 172.21.15.1:0/1283589681 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fbc300a0690 0x7fbc300a2b50 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fbc50074830 tx=0x7fbc38005fb0 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:45.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.456+0000 7fbc4fe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a7ccbee010 0x55a7ccbee3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:45.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.456+0000 7fbc4fe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a7ccbee010 0x55a7ccbee3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:41322/0 (socket says 172.21.15.1:41322) 2024-02-02T15:39:45.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.456+0000 7fbc4fe00700 1 -- 172.21.15.1:0/503110150 learned_addr learned my addr 172.21.15.1:0/503110150 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:39:45.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.456+0000 7fbc4f400700 1 --2- 172.21.15.1:0/503110150 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7ccbee930 0x55a7ccbf92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:45.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.456+0000 7fbc54a00700 1 --2- 172.21.15.1:0/503110150 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7ccbf97e0 0x55a7ccbfbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:45.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.456+0000 7fbc4fe00700 1 -- 172.21.15.1:0/503110150 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7ccbf97e0 msgr2=0x55a7ccbfbbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:45.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.456+0000 7fbc4fe00700 1 --2- 172.21.15.1:0/503110150 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7ccbf97e0 0x55a7ccbfbbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.456+0000 7fbc4fe00700 1 -- 172.21.15.1:0/503110150 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7ccbee930 msgr2=0x55a7ccbf92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:45.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.456+0000 7fbc4fe00700 1 --2- 172.21.15.1:0/503110150 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7ccbee930 0x55a7ccbf92a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.456+0000 7fbc4fe00700 1 -- 172.21.15.1:0/503110150 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7cc9c13b0 con 0x55a7ccbee010 2024-02-02T15:39:45.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.457+0000 7fbc4fe00700 1 --2- 172.21.15.1:0/503110150 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a7ccbee010 0x55a7ccbee3f0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fbc400993e0 tx=0x7fbc40098e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbc740fb7ddced20 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:45.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.457+0000 7fbc45200700 1 -- 172.21.15.1:0/503110150 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc40099450 con 0x55a7ccbee010 2024-02-02T15:39:45.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.457+0000 7fbc45200700 1 -- 172.21.15.1:0/503110150 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc4009d280 con 0x55a7ccbee010 2024-02-02T15:39:45.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.457+0000 7fbc45200700 1 -- 172.21.15.1:0/503110150 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc40099910 con 0x55a7ccbee010 2024-02-02T15:39:45.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.457+0000 7fbc6a613b80 1 -- 172.21.15.1:0/503110150 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a7ccbee010 msgr2=0x55a7ccbee3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:45.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.457+0000 7fbc6a613b80 1 --2- 172.21.15.1:0/503110150 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a7ccbee010 0x55a7ccbee3f0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fbc400993e0 tx=0x7fbc40098e30 comp rx=0 tx=0).stop 2024-02-02T15:39:45.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.458+0000 7fbc6a613b80 1 -- 172.21.15.1:0/503110150 shutdown_connections 2024-02-02T15:39:45.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.458+0000 7fbc6a613b80 1 --2- 172.21.15.1:0/503110150 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7ccbf97e0 0x55a7ccbfbbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.458+0000 7fbc6a613b80 1 --2- 172.21.15.1:0/503110150 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a7ccbee010 0x55a7ccbee3f0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.458+0000 7fbc6a613b80 1 --2- 172.21.15.1:0/503110150 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7ccbee930 0x55a7ccbf92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.458+0000 7fbc6a613b80 1 -- 172.21.15.1:0/503110150 >> 172.21.15.1:0/503110150 conn(0x55a7ccb4ddb0 msgr2=0x55a7ccbea3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:45.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.458+0000 7fbc6a613b80 1 -- 172.21.15.1:0/503110150 shutdown_connections 2024-02-02T15:39:45.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.458+0000 7fbc6a613b80 1 -- 172.21.15.1:0/503110150 wait complete. 2024-02-02T15:39:45.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.458+0000 7fbc6a613b80 1 -- start start 2024-02-02T15:39:45.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.458+0000 7fbc6a613b80 1 Processor -- start 2024-02-02T15:39:45.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.459+0000 7fbc6a613b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a7ccbee010 0x55a7ccbf9210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:45.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.459+0000 7fbc6a613b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7ccbee930 0x55a7ccbf6490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:45.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.459+0000 7fbc6a613b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7ccbf97e0 0x55a7ccbf6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:45.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.459+0000 7fbc6a613b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7ccb3fd50 con 0x55a7ccbee930 2024-02-02T15:39:45.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.459+0000 7fbc6a613b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7ccadbd80 con 0x55a7ccbee010 2024-02-02T15:39:45.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.459+0000 7fbc6a613b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7ccadba20 con 0x55a7ccbf97e0 2024-02-02T15:39:45.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.459+0000 7fbc6a613b80 10 client.0 _ll_register_callbacks cb 0x55a7ccadd290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:39:45.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.459+0000 7fbc4fe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a7ccbee010 0x55a7ccbf9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:45.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.459+0000 7fbc4f400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7ccbee930 0x55a7ccbf6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:45.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.459+0000 7fbc54a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7ccbf97e0 0x55a7ccbf6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:45.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.459+0000 7fbc4fe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a7ccbee010 0x55a7ccbf9210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:41326/0 (socket says 172.21.15.1:41326) 2024-02-02T15:39:45.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.459+0000 7fbc4fe00700 1 -- 172.21.15.1:0/504092096 learned_addr learned my addr 172.21.15.1:0/504092096 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:39:45.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.459+0000 7fbc4fe00700 1 -- 172.21.15.1:0/504092096 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7ccbf97e0 msgr2=0x55a7ccbf6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:45.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.459+0000 7fbc4fe00700 1 --2- 172.21.15.1:0/504092096 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7ccbf97e0 0x55a7ccbf6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.459+0000 7fbc4fe00700 1 -- 172.21.15.1:0/504092096 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7ccbee930 msgr2=0x55a7ccbf6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:45.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.459+0000 7fbc4fe00700 1 --2- 172.21.15.1:0/504092096 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7ccbee930 0x55a7ccbf6490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.459+0000 7fbc4fe00700 1 -- 172.21.15.1:0/504092096 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc501a6170 con 0x55a7ccbee010 2024-02-02T15:39:45.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.459+0000 7fbc54a00700 1 --2- 172.21.15.1:0/504092096 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7ccbf97e0 0x55a7ccbf6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:39:45.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.460+0000 7fbc4fe00700 1 --2- 172.21.15.1:0/504092096 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a7ccbee010 0x55a7ccbf9210 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fbc400a0f90 tx=0x7fbc4009d1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65279096fc6ec0a7 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:45.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.460+0000 7fbc3d600700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:39:45.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.460+0000 7fbc3d600700 1 -- 172.21.15.1:0/504092096 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc40099450 con 0x55a7ccbee010 2024-02-02T15:39:45.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.460+0000 7fbc3d600700 1 -- 172.21.15.1:0/504092096 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fbc400996b0 con 0x55a7ccbee010 2024-02-02T15:39:45.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.460+0000 7fbc6a613b80 1 -- 172.21.15.1:0/504092096 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a7ccbe20f0 con 0x55a7ccbee010 2024-02-02T15:39:45.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.460+0000 7fbc3d600700 1 -- 172.21.15.1:0/504092096 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbc40099870 con 0x55a7ccbee010 2024-02-02T15:39:45.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.460+0000 7fbc6a613b80 10 client.28991 fetch_fsmap learned FSMap version 52 2024-02-02T15:39:45.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.460+0000 7fbc6a613b80 1 -- 172.21.15.1:0/504092096 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x55a7cc96db60 con 0x55a7ccbee010 2024-02-02T15:39:45.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.460+0000 7fbc3d600700 1 -- 172.21.15.1:0/504092096 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a7ccbe20f0 con 0x55a7ccbee010 2024-02-02T15:39:45.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.460+0000 7fbc3d600700 1 -- 172.21.15.1:0/504092096 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 52) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fbc400a1110 con 0x55a7ccbee010 2024-02-02T15:39:45.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.460+0000 7fbc6a613b80 10 client.28991 fetch_fsmap finished waiting for FSMap version 52 2024-02-02T15:39:45.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.460+0000 7fbc6a613b80 10 client.28991 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-02-02T15:39:45.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.460+0000 7fbc6a613b80 20 client.28991 populate_metadata read hostname 'smithi001' 2024-02-02T15:39:45.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.461+0000 7fbc6a613b80 1 --2- 172.21.15.1:0/504092096 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55a7ccc08e20 0x55a7ccc0b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:45.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.461+0000 7fbc6a613b80 4 client.28991 mds_command: new command op to 24514 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:39:45.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.461+0000 7fbc6a613b80 1 -- 172.21.15.1:0/504092096 --> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a7ccb38dd0 con 0x55a7ccc08e20 2024-02-02T15:39:45.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.461+0000 7fbc4f400700 1 --2- 172.21.15.1:0/504092096 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55a7ccc08e20 0x55a7ccc0b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:45.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.461+0000 7fbc4f400700 1 --2- 172.21.15.1:0/504092096 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55a7ccc08e20 0x55a7ccc0b240 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b68f7e973344e6bf server_cookie=bd88fbbde26dce87 in_seq=0 out_seq=0 2024-02-02T15:39:45.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.461+0000 7fbc3d600700 10 client.28991 ms_handle_connect on v2:172.21.15.142:6836/2870289019 2024-02-02T15:39:45.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.462+0000 7fbc3d600700 1 -- 172.21.15.1:0/504092096 <== mds.3 v2:172.21.15.142:6836/2870289019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55a7ccb38dd0 con 0x55a7ccc08e20 2024-02-02T15:39:45.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.462+0000 7fbc3d600700 10 client.28991 handle_command_reply: tid=0 2024-02-02T15:39:45.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.462+0000 7fbc6a613b80 1 client.28991 shutdown 2024-02-02T15:39:45.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.463+0000 7fbc6a613b80 1 -- 172.21.15.1:0/504092096 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a7ccbee010 msgr2=0x55a7ccbf9210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:45.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.463+0000 7fbc6a613b80 1 --2- 172.21.15.1:0/504092096 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a7ccbee010 0x55a7ccbf9210 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fbc400a0f90 tx=0x7fbc4009d1a0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.463+0000 7fbc6a613b80 1 -- 172.21.15.1:0/504092096 shutdown_connections 2024-02-02T15:39:45.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.463+0000 7fbc6a613b80 1 --2- 172.21.15.1:0/504092096 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7ccbf97e0 0x55a7ccbf6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.463+0000 7fbc6a613b80 1 --2- 172.21.15.1:0/504092096 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a7ccbee010 0x55a7ccbf9210 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.463+0000 7fbc6a613b80 1 --2- 172.21.15.1:0/504092096 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55a7ccc08e20 0x55a7ccc0b240 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.463+0000 7fbc6a613b80 1 --2- 172.21.15.1:0/504092096 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7ccbee930 0x55a7ccbf6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.463+0000 7fbc6a613b80 1 -- 172.21.15.1:0/504092096 >> 172.21.15.1:0/504092096 conn(0x55a7ccb4ddb0 msgr2=0x55a7ccbe9fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:45.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.463+0000 7fbc3d600700 0 client.28991 ms_handle_reset on v2:172.21.15.142:6836/2870289019 2024-02-02T15:39:45.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.463+0000 7fbc6a613b80 1 -- 172.21.15.1:0/504092096 shutdown_connections 2024-02-02T15:39:45.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.463+0000 7fbc6a613b80 1 -- 172.21.15.1:0/504092096 wait complete. 2024-02-02T15:39:45.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.463+0000 7fbc6a613b80 20 client.28991 trim_cache size 0 max 16384 2024-02-02T15:39:45.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.476+0000 7fbc6a613b80 1 Processor -- start 2024-02-02T15:39:45.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.476+0000 7fbc6a613b80 1 -- start start 2024-02-02T15:39:45.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.477+0000 7fbc6a613b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a7ccbee010 0x55a7ccbf0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:45.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.477+0000 7fbc6a613b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7ccbee930 0x55a7ccc05580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:45.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.477+0000 7fbc6a613b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7ccbf97e0 0x55a7ccbf8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:45.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.477+0000 7fbc6a613b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7cc9c36a0 con 0x55a7ccbf97e0 2024-02-02T15:39:45.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.477+0000 7fbc6a613b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7ccbef260 con 0x55a7ccbee010 2024-02-02T15:39:45.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.477+0000 7fbc6a613b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7ccbeef10 con 0x55a7ccbee930 2024-02-02T15:39:45.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.477+0000 7fbc54a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7ccbf97e0 0x55a7ccbf8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:45.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.477+0000 7fbc4fe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a7ccbee010 0x55a7ccbf0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:45.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.477+0000 7fbc54a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7ccbf97e0 0x55a7ccbf8d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55740/0 (socket says 172.21.15.1:55740) 2024-02-02T15:39:45.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.477+0000 7fbc54a00700 1 -- 172.21.15.1:0/3990978743 learned_addr learned my addr 172.21.15.1:0/3990978743 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:39:45.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.477+0000 7fbc4f400700 1 --2- 172.21.15.1:0/3990978743 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7ccbee930 0x55a7ccc05580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:45.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.477+0000 7fbc4fe00700 1 -- 172.21.15.1:0/3990978743 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7ccbee930 msgr2=0x55a7ccc05580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:45.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.477+0000 7fbc4fe00700 1 --2- 172.21.15.1:0/3990978743 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7ccbee930 0x55a7ccc05580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.477+0000 7fbc4fe00700 1 -- 172.21.15.1:0/3990978743 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7ccbf97e0 msgr2=0x55a7ccbf8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:45.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.477+0000 7fbc4fe00700 1 --2- 172.21.15.1:0/3990978743 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7ccbf97e0 0x55a7ccbf8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.477+0000 7fbc4fe00700 1 -- 172.21.15.1:0/3990978743 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7ccb05ea0 con 0x55a7ccbee010 2024-02-02T15:39:45.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.477+0000 7fbc54a00700 1 --2- 172.21.15.1:0/3990978743 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7ccbf97e0 0x55a7ccbf8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:39:45.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.478+0000 7fbc4fe00700 1 --2- 172.21.15.1:0/3990978743 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a7ccbee010 0x55a7ccbf0fe0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fbc400977a0 tx=0x7fbc40098e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=668b4aaeeb7cd504 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:45.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.478+0000 7fbc45c00700 1 -- 172.21.15.1:0/3990978743 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc40098760 con 0x55a7ccbee010 2024-02-02T15:39:45.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.478+0000 7fbc45c00700 1 -- 172.21.15.1:0/3990978743 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc40065240 con 0x55a7ccbee010 2024-02-02T15:39:45.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.478+0000 7fbc45c00700 1 -- 172.21.15.1:0/3990978743 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc400a14c0 con 0x55a7ccbee010 2024-02-02T15:39:45.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.478+0000 7fbc6a613b80 1 -- 172.21.15.1:0/3990978743 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a7ccbee010 msgr2=0x55a7ccbf0fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:45.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.478+0000 7fbc6a613b80 1 --2- 172.21.15.1:0/3990978743 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a7ccbee010 0x55a7ccbf0fe0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fbc400977a0 tx=0x7fbc40098e30 comp rx=0 tx=0).stop 2024-02-02T15:39:45.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.478+0000 7fbc6a613b80 1 -- 172.21.15.1:0/3990978743 shutdown_connections 2024-02-02T15:39:45.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.478+0000 7fbc6a613b80 1 --2- 172.21.15.1:0/3990978743 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7ccbf97e0 0x55a7ccbf8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.478+0000 7fbc6a613b80 1 --2- 172.21.15.1:0/3990978743 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a7ccbee010 0x55a7ccbf0fe0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.478+0000 7fbc6a613b80 1 --2- 172.21.15.1:0/3990978743 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7ccbee930 0x55a7ccc05580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.478+0000 7fbc6a613b80 1 -- 172.21.15.1:0/3990978743 >> 172.21.15.1:0/3990978743 conn(0x55a7ccb4ddb0 msgr2=0x55a7ccbe8b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:45.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.479+0000 7fbc6a613b80 1 -- 172.21.15.1:0/3990978743 shutdown_connections 2024-02-02T15:39:45.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.479+0000 7fbc6a613b80 1 -- 172.21.15.1:0/3990978743 wait complete. 2024-02-02T15:39:45.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.479+0000 7fbc6a613b80 1 -- start start 2024-02-02T15:39:45.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.479+0000 7fbc6a613b80 1 Processor -- start 2024-02-02T15:39:45.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.479+0000 7fbc6a613b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7ccbee010 0x55a7ccbfd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:45.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.480+0000 7fbc6a613b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7ccbee930 0x55a7ccbfdc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:45.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.480+0000 7fbc6a613b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a7ccbf97e0 0x55a7ccbfe180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:45.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.480+0000 7fbc6a613b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7ccc59940 con 0x55a7ccbee010 2024-02-02T15:39:45.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.480+0000 7fbc6a613b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7ccc59a80 con 0x55a7ccbf97e0 2024-02-02T15:39:45.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.480+0000 7fbc6a613b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7ccc59bc0 con 0x55a7ccbee930 2024-02-02T15:39:45.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.480+0000 7fbc6a613b80 10 client.0 _ll_register_callbacks cb 0x55a7ccadd290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:39:45.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.480+0000 7fbc54a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a7ccbf97e0 0x55a7ccbfe180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:45.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.480+0000 7fbc54a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a7ccbf97e0 0x55a7ccbfe180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:41354/0 (socket says 172.21.15.1:41354) 2024-02-02T15:39:45.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.480+0000 7fbc54a00700 1 -- 172.21.15.1:0/2894234028 learned_addr learned my addr 172.21.15.1:0/2894234028 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:39:45.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.480+0000 7fbc4fe00700 1 --2- 172.21.15.1:0/2894234028 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7ccbee010 0x55a7ccbfd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:45.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.480+0000 7fbc4f400700 1 --2- 172.21.15.1:0/2894234028 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7ccbee930 0x55a7ccbfdc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:45.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.480+0000 7fbc54a00700 1 -- 172.21.15.1:0/2894234028 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7ccbee930 msgr2=0x55a7ccbfdc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:45.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.480+0000 7fbc54a00700 1 --2- 172.21.15.1:0/2894234028 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7ccbee930 0x55a7ccbfdc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.480+0000 7fbc54a00700 1 -- 172.21.15.1:0/2894234028 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7ccbee010 msgr2=0x55a7ccbfd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:45.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.480+0000 7fbc54a00700 1 --2- 172.21.15.1:0/2894234028 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7ccbee010 0x55a7ccbfd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.480+0000 7fbc54a00700 1 -- 172.21.15.1:0/2894234028 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc480076f0 con 0x55a7ccbf97e0 2024-02-02T15:39:45.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.480+0000 7fbc54a00700 1 --2- 172.21.15.1:0/2894234028 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a7ccbf97e0 0x55a7ccbfe180 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fbc480189d0 tx=0x7fbc48018a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23af90432792a445 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:45.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.480+0000 7fbc3d600700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:39:45.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.481+0000 7fbc3d600700 1 -- 172.21.15.1:0/2894234028 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc4800e040 con 0x55a7ccbf97e0 2024-02-02T15:39:45.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.481+0000 7fbc6a613b80 1 -- 172.21.15.1:0/2894234028 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbc4009d280 con 0x55a7ccbf97e0 2024-02-02T15:39:45.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.481+0000 7fbc3d600700 1 -- 172.21.15.1:0/2894234028 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fbc48003c00 con 0x55a7ccbf97e0 2024-02-02T15:39:45.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.481+0000 7fbc3d600700 1 -- 172.21.15.1:0/2894234028 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbc48003dc0 con 0x55a7ccbf97e0 2024-02-02T15:39:45.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.481+0000 7fbc6a613b80 10 client.28997 fetch_fsmap learned FSMap version 52 2024-02-02T15:39:45.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.481+0000 7fbc6a613b80 1 -- 172.21.15.1:0/2894234028 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x55a7cc96db60 con 0x55a7ccbf97e0 2024-02-02T15:39:45.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.481+0000 7fbc3d600700 1 -- 172.21.15.1:0/2894234028 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc4009d280 con 0x55a7ccbf97e0 2024-02-02T15:39:45.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.481+0000 7fbc3d600700 1 -- 172.21.15.1:0/2894234028 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 52) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fbc48004390 con 0x55a7ccbf97e0 2024-02-02T15:39:45.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.482+0000 7fbc6a613b80 10 client.28997 fetch_fsmap finished waiting for FSMap version 52 2024-02-02T15:39:45.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.482+0000 7fbc6a613b80 10 client.28997 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-02-02T15:39:45.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.482+0000 7fbc6a613b80 20 client.28997 populate_metadata read hostname 'smithi001' 2024-02-02T15:39:45.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.482+0000 7fbc6a613b80 1 --2- 172.21.15.1:0/2894234028 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55a7ccc5b320 0x55a7ccc5d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:45.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.482+0000 7fbc6a613b80 4 client.28997 mds_command: new command op to 24514 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:39:45.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.482+0000 7fbc6a613b80 1 -- 172.21.15.1:0/2894234028 --> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a7ccafef50 con 0x55a7ccc5b320 2024-02-02T15:39:45.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.482+0000 7fbc4fe00700 1 --2- 172.21.15.1:0/2894234028 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55a7ccc5b320 0x55a7ccc5d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:45.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.482+0000 7fbc4fe00700 1 --2- 172.21.15.1:0/2894234028 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55a7ccc5b320 0x55a7ccc5d740 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7b0a77cc6c560d11 server_cookie=b92876083a1b4313 in_seq=0 out_seq=0 2024-02-02T15:39:45.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.482+0000 7fbc3d600700 10 client.28997 ms_handle_connect on v2:172.21.15.142:6836/2870289019 2024-02-02T15:39:45.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.482+0000 7fbc3d600700 1 -- 172.21.15.1:0/2894234028 <== mds.3 v2:172.21.15.142:6836/2870289019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a7ccafef50 con 0x55a7ccc5b320 2024-02-02T15:39:45.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.482+0000 7fbc3d600700 10 client.28997 handle_command_reply: tid=0 2024-02-02T15:39:45.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.482+0000 7fbc6a613b80 1 client.28997 shutdown 2024-02-02T15:39:45.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.483+0000 7fbc6a613b80 1 -- 172.21.15.1:0/2894234028 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a7ccbf97e0 msgr2=0x55a7ccbfe180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:45.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.483+0000 7fbc6a613b80 1 --2- 172.21.15.1:0/2894234028 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a7ccbf97e0 0x55a7ccbfe180 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fbc480189d0 tx=0x7fbc48018a00 comp rx=0 tx=0).stop 2024-02-02T15:39:45.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.483+0000 7fbc6a613b80 1 -- 172.21.15.1:0/2894234028 shutdown_connections 2024-02-02T15:39:45.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.483+0000 7fbc6a613b80 1 --2- 172.21.15.1:0/2894234028 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a7ccbf97e0 0x55a7ccbfe180 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.483+0000 7fbc6a613b80 1 --2- 172.21.15.1:0/2894234028 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55a7ccc5b320 0x55a7ccc5d740 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.483+0000 7fbc6a613b80 1 --2- 172.21.15.1:0/2894234028 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a7ccbee010 0x55a7ccbfd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.483+0000 7fbc6a613b80 1 --2- 172.21.15.1:0/2894234028 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a7ccbee930 0x55a7ccbfdc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.483+0000 7fbc3d600700 0 client.28997 ms_handle_reset on v2:172.21.15.142:6836/2870289019 2024-02-02T15:39:45.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.483+0000 7fbc6a613b80 1 -- 172.21.15.1:0/2894234028 >> 172.21.15.1:0/2894234028 conn(0x55a7ccb4ddb0 msgr2=0x55a7ccc048a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:45.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.483+0000 7fbc6a613b80 1 -- 172.21.15.1:0/2894234028 shutdown_connections 2024-02-02T15:39:45.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.483+0000 7fbc6a613b80 1 -- 172.21.15.1:0/2894234028 wait complete. 2024-02-02T15:39:45.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.483+0000 7fbc6a613b80 20 client.28997 trim_cache size 0 max 16384 2024-02-02T15:39:45.488 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:39:45.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.484+0000 7fbc56a00700 1 -- 172.21.15.1:0/1283589681 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fbc300a0690 msgr2=0x7fbc300a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:45.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.484+0000 7fbc56a00700 1 --2- 172.21.15.1:0/1283589681 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fbc300a0690 0x7fbc300a2b50 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fbc50074830 tx=0x7fbc38005fb0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.484+0000 7fbc56a00700 1 -- 172.21.15.1:0/1283589681 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc50100ff0 msgr2=0x7fbc50073040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:45.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.484+0000 7fbc56a00700 1 --2- 172.21.15.1:0/1283589681 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc50100ff0 0x7fbc50073040 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fbc4000e9a0 tx=0x7fbc4000ecb0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.484+0000 7fbc56a00700 1 -- 172.21.15.1:0/1283589681 shutdown_connections 2024-02-02T15:39:45.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.484+0000 7fbc56a00700 1 --2- 172.21.15.1:0/1283589681 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fbc300a0690 0x7fbc300a2b50 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.484+0000 7fbc56a00700 1 --2- 172.21.15.1:0/1283589681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbc50073ac0 0x7fbc501a5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.484+0000 7fbc56a00700 1 --2- 172.21.15.1:0/1283589681 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbc50100ff0 0x7fbc50073040 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.484+0000 7fbc56a00700 1 --2- 172.21.15.1:0/1283589681 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbc50106110 0x7fbc50073580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.484+0000 7fbc56a00700 1 -- 172.21.15.1:0/1283589681 >> 172.21.15.1:0/1283589681 conn(0x7fbc500fc810 msgr2=0x7fbc500fd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:45.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.484+0000 7fbc56a00700 1 -- 172.21.15.1:0/1283589681 shutdown_connections 2024-02-02T15:39:45.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.484+0000 7fbc56a00700 1 -- 172.21.15.1:0/1283589681 wait complete. 2024-02-02T15:39:45.501 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:39:45 smithi001 ceph-mon[35470]: pgmap v397: 129 pgs: 129 active+clean; 594 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1004 KiB/s rd, 1.6 MiB/s wr, 109 op/s 2024-02-02T15:39:45.501 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:39:45 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/4210301728' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:39:45.503 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-02T15:39:45.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.743+0000 7fe467e00700 1 -- 172.21.15.1:0/2093715123 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe460102030 msgr2=0x7fe460102410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:45.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.743+0000 7fe467e00700 1 --2- 172.21.15.1:0/2093715123 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe460102030 0x7fe460102410 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fe4500098f0 tx=0x7fe450009c00 comp rx=0 tx=0).stop 2024-02-02T15:39:45.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.744+0000 7fe467e00700 1 -- 172.21.15.1:0/2093715123 shutdown_connections 2024-02-02T15:39:45.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.744+0000 7fe467e00700 1 --2- 172.21.15.1:0/2093715123 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe46010b400 0x7fe46010d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.744+0000 7fe467e00700 1 --2- 172.21.15.1:0/2093715123 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe460102030 0x7fe460102410 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.744+0000 7fe467e00700 1 --2- 172.21.15.1:0/2093715123 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe460102950 0x7fe46010ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.744+0000 7fe467e00700 1 -- 172.21.15.1:0/2093715123 >> 172.21.15.1:0/2093715123 conn(0x7fe460076d70 msgr2=0x7fe460077180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:45.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.744+0000 7fe467e00700 1 -- 172.21.15.1:0/2093715123 shutdown_connections 2024-02-02T15:39:45.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.744+0000 7fe467e00700 1 -- 172.21.15.1:0/2093715123 wait complete. 2024-02-02T15:39:45.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.745+0000 7fe467e00700 1 Processor -- start 2024-02-02T15:39:45.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.745+0000 7fe467e00700 1 -- start start 2024-02-02T15:39:45.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.745+0000 7fe467e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe460102030 0x7fe460199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:45.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.745+0000 7fe467e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe460102950 0x7fe460199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:45.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.745+0000 7fe467e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe46010b400 0x7fe46019dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:45.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.745+0000 7fe467e00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe46019e110 con 0x7fe460102950 2024-02-02T15:39:45.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.745+0000 7fe467e00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe46019e280 con 0x7fe460102030 2024-02-02T15:39:45.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.745+0000 7fe467e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe46019e440 con 0x7fe46010b400 2024-02-02T15:39:45.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.745+0000 7fe465400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe460102030 0x7fe460199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:45.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.745+0000 7fe464a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe460102950 0x7fe460199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:45.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.745+0000 7fe465400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe460102030 0x7fe460199130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:41368/0 (socket says 172.21.15.1:41368) 2024-02-02T15:39:45.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.745+0000 7fe465400700 1 -- 172.21.15.1:0/3313514457 learned_addr learned my addr 172.21.15.1:0/3313514457 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:39:45.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.746+0000 7fe465e00700 1 --2- 172.21.15.1:0/3313514457 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe46010b400 0x7fe46019dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:45.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.746+0000 7fe465400700 1 -- 172.21.15.1:0/3313514457 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe46010b400 msgr2=0x7fe46019dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:45.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.746+0000 7fe465400700 1 --2- 172.21.15.1:0/3313514457 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe46010b400 0x7fe46019dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.746+0000 7fe465400700 1 -- 172.21.15.1:0/3313514457 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe460102950 msgr2=0x7fe460199670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:45.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.746+0000 7fe465400700 1 --2- 172.21.15.1:0/3313514457 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe460102950 0x7fe460199670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.746+0000 7fe465400700 1 -- 172.21.15.1:0/3313514457 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe450009520 con 0x7fe460102030 2024-02-02T15:39:45.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.746+0000 7fe464a00700 1 --2- 172.21.15.1:0/3313514457 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe460102950 0x7fe460199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:39:45.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.746+0000 7fe465400700 1 --2- 172.21.15.1:0/3313514457 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe460102030 0x7fe460199130 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fe450009810 tx=0x7fe45000fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:45.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.746+0000 7fe45e000700 1 -- 172.21.15.1:0/3313514457 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe45001c070 con 0x7fe460102030 2024-02-02T15:39:45.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.746+0000 7fe45e000700 1 -- 172.21.15.1:0/3313514457 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe45000bb20 con 0x7fe460102030 2024-02-02T15:39:45.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.746+0000 7fe45e000700 1 -- 172.21.15.1:0/3313514457 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe450004930 con 0x7fe460102030 2024-02-02T15:39:45.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.746+0000 7fe467e00700 1 -- 172.21.15.1:0/3313514457 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe46019e6c0 con 0x7fe460102030 2024-02-02T15:39:45.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.747+0000 7fe467e00700 1 -- 172.21.15.1:0/3313514457 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe46019e940 con 0x7fe460102030 2024-02-02T15:39:45.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.749+0000 7fe45e000700 1 -- 172.21.15.1:0/3313514457 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7fe450020070 con 0x7fe460102030 2024-02-02T15:39:45.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.751+0000 7fe45e000700 1 --2- 172.21.15.1:0/3313514457 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fe4400a0710 0x7fe4400a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:45.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.751+0000 7fe45e000700 1 -- 172.21.15.1:0/3313514457 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fe4500ccc00 con 0x7fe460102030 2024-02-02T15:39:45.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.751+0000 7fe47ba08b80 1 Processor -- start 2024-02-02T15:39:45.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.751+0000 7fe464a00700 1 --2- 172.21.15.1:0/3313514457 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fe4400a0710 0x7fe4400a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:45.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.751+0000 7fe47ba08b80 1 -- start start 2024-02-02T15:39:45.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.752+0000 7fe47ba08b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c88483010 0x564c884833f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:45.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.752+0000 7fe47ba08b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564c88483930 0x564c8848e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:45.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.752+0000 7fe47ba08b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564c8848e7e0 0x564c88490bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:45.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.752+0000 7fe464a00700 1 --2- 172.21.15.1:0/3313514457 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fe4400a0710 0x7fe4400a2bd0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe46019a740 tx=0x7fe448009260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:45.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.752+0000 7fe47ba08b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c883f74c0 con 0x564c8848e7e0 2024-02-02T15:39:45.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.752+0000 7fe47ba08b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c883dbaa0 con 0x564c88483930 2024-02-02T15:39:45.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.752+0000 7fe47ba08b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c883d9460 con 0x564c88483010 2024-02-02T15:39:45.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.752+0000 7fe465400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c88483010 0x564c884833f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:45.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.752+0000 7fe465400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c88483010 0x564c884833f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:34080/0 (socket says 172.21.15.1:34080) 2024-02-02T15:39:45.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.752+0000 7fe465400700 1 -- 172.21.15.1:0/132111633 learned_addr learned my addr 172.21.15.1:0/132111633 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:39:45.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.752+0000 7fe464a00700 1 --2- 172.21.15.1:0/132111633 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564c88483930 0x564c8848e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:45.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.752+0000 7fe465e00700 1 --2- 172.21.15.1:0/132111633 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564c8848e7e0 0x564c88490bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:45.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.752+0000 7fe465400700 1 -- 172.21.15.1:0/132111633 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564c88483930 msgr2=0x564c8848e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:45.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.752+0000 7fe465400700 1 --2- 172.21.15.1:0/132111633 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564c88483930 0x564c8848e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.752+0000 7fe465400700 1 -- 172.21.15.1:0/132111633 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564c8848e7e0 msgr2=0x564c88490bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:45.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.752+0000 7fe465400700 1 --2- 172.21.15.1:0/132111633 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564c8848e7e0 0x564c88490bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.752+0000 7fe465400700 1 -- 172.21.15.1:0/132111633 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c882563b0 con 0x564c88483010 2024-02-02T15:39:45.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.753+0000 7fe465400700 1 --2- 172.21.15.1:0/132111633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c88483010 0x564c884833f0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fe450099e60 tx=0x7fe45000fa70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe9415f25e0d46de server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:45.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.753+0000 7fe456600700 1 -- 172.21.15.1:0/132111633 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe45009dd00 con 0x564c88483010 2024-02-02T15:39:45.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.753+0000 7fe456600700 1 -- 172.21.15.1:0/132111633 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe4500a1eb0 con 0x564c88483010 2024-02-02T15:39:45.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.753+0000 7fe456600700 1 -- 172.21.15.1:0/132111633 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe45009a390 con 0x564c88483010 2024-02-02T15:39:45.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.753+0000 7fe47ba08b80 1 -- 172.21.15.1:0/132111633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c88483010 msgr2=0x564c884833f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:45.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.753+0000 7fe47ba08b80 1 --2- 172.21.15.1:0/132111633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c88483010 0x564c884833f0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fe450099e60 tx=0x7fe45000fa70 comp rx=0 tx=0).stop 2024-02-02T15:39:45.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.753+0000 7fe47ba08b80 1 -- 172.21.15.1:0/132111633 shutdown_connections 2024-02-02T15:39:45.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.753+0000 7fe47ba08b80 1 --2- 172.21.15.1:0/132111633 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564c8848e7e0 0x564c88490bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.753+0000 7fe47ba08b80 1 --2- 172.21.15.1:0/132111633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c88483010 0x564c884833f0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.753+0000 7fe47ba08b80 1 --2- 172.21.15.1:0/132111633 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564c88483930 0x564c8848e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.753+0000 7fe47ba08b80 1 -- 172.21.15.1:0/132111633 >> 172.21.15.1:0/132111633 conn(0x564c883e2db0 msgr2=0x564c8847f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:45.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.754+0000 7fe47ba08b80 1 -- 172.21.15.1:0/132111633 shutdown_connections 2024-02-02T15:39:45.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.754+0000 7fe47ba08b80 1 -- 172.21.15.1:0/132111633 wait complete. 2024-02-02T15:39:45.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.754+0000 7fe47ba08b80 1 -- start start 2024-02-02T15:39:45.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.754+0000 7fe47ba08b80 1 Processor -- start 2024-02-02T15:39:45.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.755+0000 7fe47ba08b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c88483010 0x564c8848ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:45.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.755+0000 7fe47ba08b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564c88483930 0x564c8848f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:45.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.755+0000 7fe47ba08b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564c8848e7e0 0x564c8848f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:45.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.755+0000 7fe47ba08b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c883d4d50 con 0x564c88483930 2024-02-02T15:39:45.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.755+0000 7fe47ba08b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c88370d80 con 0x564c8848e7e0 2024-02-02T15:39:45.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.755+0000 7fe47ba08b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c88370a20 con 0x564c88483010 2024-02-02T15:39:45.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.755+0000 7fe47ba08b80 10 client.0 _ll_register_callbacks cb 0x564c88372290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:39:45.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.755+0000 7fe464a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564c88483930 0x564c8848f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:45.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.755+0000 7fe464a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564c88483930 0x564c8848f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55772/0 (socket says 172.21.15.1:55772) 2024-02-02T15:39:45.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.755+0000 7fe464a00700 1 -- 172.21.15.1:0/516792755 learned_addr learned my addr 172.21.15.1:0/516792755 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:39:45.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.755+0000 7fe465400700 1 --2- 172.21.15.1:0/516792755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c88483010 0x564c8848ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:45.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.755+0000 7fe465e00700 1 --2- 172.21.15.1:0/516792755 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564c8848e7e0 0x564c8848f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:45.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.756+0000 7fe464a00700 1 -- 172.21.15.1:0/516792755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c88483010 msgr2=0x564c8848ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:45.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.756+0000 7fe464a00700 1 --2- 172.21.15.1:0/516792755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c88483010 0x564c8848ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.756+0000 7fe464a00700 1 -- 172.21.15.1:0/516792755 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564c8848e7e0 msgr2=0x564c8848f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:45.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.756+0000 7fe464a00700 1 --2- 172.21.15.1:0/516792755 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564c8848e7e0 0x564c8848f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.756+0000 7fe464a00700 1 -- 172.21.15.1:0/516792755 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe46019e6c0 con 0x564c88483930 2024-02-02T15:39:45.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.756+0000 7fe465400700 1 --2- 172.21.15.1:0/516792755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c88483010 0x564c8848ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:39:45.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.756+0000 7fe464a00700 1 --2- 172.21.15.1:0/516792755 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564c88483930 0x564c8848f100 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fe448003ad0 tx=0x7fe4480047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0e324539d42a164 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:45.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.756+0000 7fe44ea00700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:39:45.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.756+0000 7fe44ea00700 1 -- 172.21.15.1:0/516792755 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe448003ea0 con 0x564c88483930 2024-02-02T15:39:45.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.756+0000 7fe44ea00700 1 -- 172.21.15.1:0/516792755 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fe44801fa10 con 0x564c88483930 2024-02-02T15:39:45.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.756+0000 7fe47ba08b80 1 -- 172.21.15.1:0/516792755 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564c884770f0 con 0x564c88483930 2024-02-02T15:39:45.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.756+0000 7fe44ea00700 1 -- 172.21.15.1:0/516792755 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe44801fbd0 con 0x564c88483930 2024-02-02T15:39:45.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.756+0000 7fe47ba08b80 10 client.18909 fetch_fsmap learned FSMap version 52 2024-02-02T15:39:45.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.756+0000 7fe47ba08b80 1 -- 172.21.15.1:0/516792755 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x564c88202b60 con 0x564c88483930 2024-02-02T15:39:45.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.757+0000 7fe44ea00700 1 -- 172.21.15.1:0/516792755 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564c884770f0 con 0x564c88483930 2024-02-02T15:39:45.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.757+0000 7fe44ea00700 1 -- 172.21.15.1:0/516792755 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 52) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fe44800e3b0 con 0x564c88483930 2024-02-02T15:39:45.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.757+0000 7fe47ba08b80 10 client.18909 fetch_fsmap finished waiting for FSMap version 52 2024-02-02T15:39:45.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.757+0000 7fe47ba08b80 10 client.18909 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-02-02T15:39:45.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.757+0000 7fe47ba08b80 20 client.18909 populate_metadata read hostname 'smithi001' 2024-02-02T15:39:45.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.757+0000 7fe47ba08b80 1 --2- 172.21.15.1:0/516792755 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x564c8849dee0 0x564c884a0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:45.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.757+0000 7fe47ba08b80 4 client.18909 mds_command: new command op to 24518 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:39:45.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.757+0000 7fe47ba08b80 1 -- 172.21.15.1:0/516792755 --> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564c883cddd0 con 0x564c8849dee0 2024-02-02T15:39:45.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.757+0000 7fe465400700 1 --2- 172.21.15.1:0/516792755 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x564c8849dee0 0x564c884a0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:45.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.758+0000 7fe465400700 1 --2- 172.21.15.1:0/516792755 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x564c8849dee0 0x564c884a0300 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ebb65e48f4fa6005 server_cookie=dcdb8e813166e8f8 in_seq=0 out_seq=0 2024-02-02T15:39:45.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.758+0000 7fe44ea00700 10 client.18909 ms_handle_connect on v2:172.21.15.71:6836/628016199 2024-02-02T15:39:45.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.758+0000 7fe44ea00700 1 -- 172.21.15.1:0/516792755 <== mds.2 v2:172.21.15.71:6836/628016199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x564c883cddd0 con 0x564c8849dee0 2024-02-02T15:39:45.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.758+0000 7fe44ea00700 10 client.18909 handle_command_reply: tid=0 2024-02-02T15:39:45.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.759+0000 7fe47ba08b80 1 client.18909 shutdown 2024-02-02T15:39:45.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.759+0000 7fe47ba08b80 1 -- 172.21.15.1:0/516792755 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564c88483930 msgr2=0x564c8848f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:45.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.759+0000 7fe47ba08b80 1 --2- 172.21.15.1:0/516792755 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564c88483930 0x564c8848f100 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fe448003ad0 tx=0x7fe4480047c0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.759+0000 7fe47ba08b80 1 -- 172.21.15.1:0/516792755 shutdown_connections 2024-02-02T15:39:45.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.759+0000 7fe47ba08b80 1 --2- 172.21.15.1:0/516792755 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564c8848e7e0 0x564c8848f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.759+0000 7fe47ba08b80 1 --2- 172.21.15.1:0/516792755 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x564c8849dee0 0x564c884a0300 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.759+0000 7fe47ba08b80 1 --2- 172.21.15.1:0/516792755 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c88483010 0x564c8848ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.759+0000 7fe44ea00700 0 client.18909 ms_handle_reset on v2:172.21.15.71:6836/628016199 2024-02-02T15:39:45.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.759+0000 7fe47ba08b80 1 --2- 172.21.15.1:0/516792755 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564c88483930 0x564c8848f100 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.759+0000 7fe47ba08b80 1 -- 172.21.15.1:0/516792755 >> 172.21.15.1:0/516792755 conn(0x564c883e2db0 msgr2=0x564c8847efa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:45.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.759+0000 7fe47ba08b80 1 -- 172.21.15.1:0/516792755 shutdown_connections 2024-02-02T15:39:45.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.759+0000 7fe47ba08b80 1 -- 172.21.15.1:0/516792755 wait complete. 2024-02-02T15:39:45.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.759+0000 7fe47ba08b80 20 client.18909 trim_cache size 0 max 16384 2024-02-02T15:39:45.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.772+0000 7fe47ba08b80 1 Processor -- start 2024-02-02T15:39:45.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.773+0000 7fe47ba08b80 1 -- start start 2024-02-02T15:39:45.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.773+0000 7fe47ba08b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564c88483010 0x564c88492bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:45.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.773+0000 7fe47ba08b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564c88483930 0x564c88488bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:45.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.773+0000 7fe47ba08b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c8848e7e0 0x564c88489100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:45.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.773+0000 7fe47ba08b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c882586a0 con 0x564c88483930 2024-02-02T15:39:45.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.773+0000 7fe47ba08b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c88480240 con 0x564c88483010 2024-02-02T15:39:45.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.773+0000 7fe47ba08b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c88479500 con 0x564c8848e7e0 2024-02-02T15:39:45.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.773+0000 7fe465400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564c88483010 0x564c88492bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:45.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.773+0000 7fe465400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564c88483010 0x564c88492bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:41388/0 (socket says 172.21.15.1:41388) 2024-02-02T15:39:45.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.773+0000 7fe465400700 1 -- 172.21.15.1:0/3374468532 learned_addr learned my addr 172.21.15.1:0/3374468532 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:39:45.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.773+0000 7fe464a00700 1 --2- 172.21.15.1:0/3374468532 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564c88483930 0x564c88488bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:45.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.773+0000 7fe465e00700 1 --2- 172.21.15.1:0/3374468532 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c8848e7e0 0x564c88489100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:45.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.774+0000 7fe465400700 1 -- 172.21.15.1:0/3374468532 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c8848e7e0 msgr2=0x564c88489100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:45.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.774+0000 7fe465400700 1 --2- 172.21.15.1:0/3374468532 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c8848e7e0 0x564c88489100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.774+0000 7fe465400700 1 -- 172.21.15.1:0/3374468532 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564c88483930 msgr2=0x564c88488bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:45.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.774+0000 7fe465400700 1 --2- 172.21.15.1:0/3374468532 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564c88483930 0x564c88488bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.774+0000 7fe465400700 1 -- 172.21.15.1:0/3374468532 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c8839aea0 con 0x564c88483010 2024-02-02T15:39:45.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.774+0000 7fe465400700 1 --2- 172.21.15.1:0/3374468532 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564c88483010 0x564c88492bf0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fe450098220 tx=0x7fe45009a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=237f226b515d5ba3 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:45.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.774+0000 7fe457000700 1 -- 172.21.15.1:0/3374468532 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe450096210 con 0x564c88483010 2024-02-02T15:39:45.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.774+0000 7fe457000700 1 -- 172.21.15.1:0/3374468532 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe45009dd00 con 0x564c88483010 2024-02-02T15:39:45.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.774+0000 7fe457000700 1 -- 172.21.15.1:0/3374468532 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe450099620 con 0x564c88483010 2024-02-02T15:39:45.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.775+0000 7fe47ba08b80 1 -- 172.21.15.1:0/3374468532 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564c88483010 msgr2=0x564c88492bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:45.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.775+0000 7fe47ba08b80 1 --2- 172.21.15.1:0/3374468532 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564c88483010 0x564c88492bf0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fe450098220 tx=0x7fe45009a710 comp rx=0 tx=0).stop 2024-02-02T15:39:45.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.775+0000 7fe47ba08b80 1 -- 172.21.15.1:0/3374468532 shutdown_connections 2024-02-02T15:39:45.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.775+0000 7fe47ba08b80 1 --2- 172.21.15.1:0/3374468532 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c8848e7e0 0x564c88489100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.775+0000 7fe47ba08b80 1 --2- 172.21.15.1:0/3374468532 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564c88483010 0x564c88492bf0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.775+0000 7fe47ba08b80 1 --2- 172.21.15.1:0/3374468532 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564c88483930 0x564c88488bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.775+0000 7fe47ba08b80 1 -- 172.21.15.1:0/3374468532 >> 172.21.15.1:0/3374468532 conn(0x564c883e2db0 msgr2=0x564c884991d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:45.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.775+0000 7fe47ba08b80 1 -- 172.21.15.1:0/3374468532 shutdown_connections 2024-02-02T15:39:45.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.775+0000 7fe47ba08b80 1 -- 172.21.15.1:0/3374468532 wait complete. 2024-02-02T15:39:45.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.775+0000 7fe47ba08b80 1 -- start start 2024-02-02T15:39:45.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.775+0000 7fe47ba08b80 1 Processor -- start 2024-02-02T15:39:45.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.776+0000 7fe47ba08b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564c88483010 0x564c884e3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:45.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.776+0000 7fe47ba08b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c88483930 0x564c884e3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:45.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.776+0000 7fe47ba08b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564c8848e7e0 0x564c884e4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:45.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.776+0000 7fe47ba08b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c884eeb40 con 0x564c8848e7e0 2024-02-02T15:39:45.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.776+0000 7fe47ba08b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c884eec80 con 0x564c88483010 2024-02-02T15:39:45.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.776+0000 7fe47ba08b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c884eee40 con 0x564c88483930 2024-02-02T15:39:45.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.776+0000 7fe47ba08b80 10 client.0 _ll_register_callbacks cb 0x564c88372290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:39:45.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.776+0000 7fe465400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564c88483010 0x564c884e3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:45.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.776+0000 7fe465400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564c88483010 0x564c884e3990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:41402/0 (socket says 172.21.15.1:41402) 2024-02-02T15:39:45.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.776+0000 7fe465400700 1 -- 172.21.15.1:0/2296999063 learned_addr learned my addr 172.21.15.1:0/2296999063 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:39:45.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.777+0000 7fe465e00700 1 --2- 172.21.15.1:0/2296999063 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564c8848e7e0 0x564c884e4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:45.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.777+0000 7fe464a00700 1 --2- 172.21.15.1:0/2296999063 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c88483930 0x564c884e3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:45.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.777+0000 7fe465400700 1 -- 172.21.15.1:0/2296999063 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c88483930 msgr2=0x564c884e3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:45.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.777+0000 7fe465400700 1 --2- 172.21.15.1:0/2296999063 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c88483930 0x564c884e3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.777+0000 7fe465400700 1 -- 172.21.15.1:0/2296999063 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564c8848e7e0 msgr2=0x564c884e4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:45.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.777+0000 7fe465400700 1 --2- 172.21.15.1:0/2296999063 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564c8848e7e0 0x564c884e4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.777+0000 7fe465400700 1 -- 172.21.15.1:0/2296999063 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564c882563b0 con 0x564c88483010 2024-02-02T15:39:45.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.777+0000 7fe465e00700 1 --2- 172.21.15.1:0/2296999063 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564c8848e7e0 0x564c884e4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:39:45.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.777+0000 7fe465400700 1 --2- 172.21.15.1:0/2296999063 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564c88483010 0x564c884e3990 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fe450029000 tx=0x7fe4500054e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24f0928848869c80 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:45.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.777+0000 7fe44ea00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:39:45.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.777+0000 7fe44ea00700 1 -- 172.21.15.1:0/2296999063 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe45007b070 con 0x564c88483010 2024-02-02T15:39:45.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.777+0000 7fe44ea00700 1 -- 172.21.15.1:0/2296999063 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fe450005140 con 0x564c88483010 2024-02-02T15:39:45.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.777+0000 7fe47ba08b80 1 -- 172.21.15.1:0/2296999063 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe4500a1eb0 con 0x564c88483010 2024-02-02T15:39:45.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.777+0000 7fe44ea00700 1 -- 172.21.15.1:0/2296999063 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe450005300 con 0x564c88483010 2024-02-02T15:39:45.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.777+0000 7fe47ba08b80 10 client.29015 fetch_fsmap learned FSMap version 52 2024-02-02T15:39:45.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.777+0000 7fe47ba08b80 1 -- 172.21.15.1:0/2296999063 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x564c8825acd0 con 0x564c88483010 2024-02-02T15:39:45.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.778+0000 7fe44ea00700 1 -- 172.21.15.1:0/2296999063 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe4500a1eb0 con 0x564c88483010 2024-02-02T15:39:45.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.778+0000 7fe44ea00700 1 -- 172.21.15.1:0/2296999063 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 52) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fe45007e020 con 0x564c88483010 2024-02-02T15:39:45.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.778+0000 7fe47ba08b80 10 client.29015 fetch_fsmap finished waiting for FSMap version 52 2024-02-02T15:39:45.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.778+0000 7fe47ba08b80 10 client.29015 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-02-02T15:39:45.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.778+0000 7fe47ba08b80 20 client.29015 populate_metadata read hostname 'smithi001' 2024-02-02T15:39:45.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.778+0000 7fe47ba08b80 1 --2- 172.21.15.1:0/2296999063 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x564c884f0090 0x564c884f2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:45.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.778+0000 7fe47ba08b80 4 client.29015 mds_command: new command op to 24518 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:39:45.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.778+0000 7fe47ba08b80 1 -- 172.21.15.1:0/2296999063 --> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564c88393f50 con 0x564c884f0090 2024-02-02T15:39:45.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.778+0000 7fe464a00700 1 --2- 172.21.15.1:0/2296999063 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x564c884f0090 0x564c884f2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:45.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.778+0000 7fe464a00700 1 --2- 172.21.15.1:0/2296999063 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x564c884f0090 0x564c884f2480 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5437a1c3cfdd6ba4 server_cookie=72e54508889b0c0e in_seq=0 out_seq=0 2024-02-02T15:39:45.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.778+0000 7fe44ea00700 10 client.29015 ms_handle_connect on v2:172.21.15.71:6836/628016199 2024-02-02T15:39:45.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.779+0000 7fe44ea00700 1 -- 172.21.15.1:0/2296999063 <== mds.2 v2:172.21.15.71:6836/628016199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564c88393f50 con 0x564c884f0090 2024-02-02T15:39:45.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.779+0000 7fe44ea00700 10 client.29015 handle_command_reply: tid=0 2024-02-02T15:39:45.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.779+0000 7fe47ba08b80 1 client.29015 shutdown 2024-02-02T15:39:45.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.779+0000 7fe47ba08b80 1 -- 172.21.15.1:0/2296999063 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564c88483010 msgr2=0x564c884e3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:45.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.779+0000 7fe47ba08b80 1 --2- 172.21.15.1:0/2296999063 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564c88483010 0x564c884e3990 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fe450029000 tx=0x7fe4500054e0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.779+0000 7fe47ba08b80 1 -- 172.21.15.1:0/2296999063 shutdown_connections 2024-02-02T15:39:45.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.779+0000 7fe47ba08b80 1 --2- 172.21.15.1:0/2296999063 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564c8848e7e0 0x564c884e4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.779+0000 7fe47ba08b80 1 --2- 172.21.15.1:0/2296999063 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564c88483010 0x564c884e3990 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.779+0000 7fe47ba08b80 1 --2- 172.21.15.1:0/2296999063 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x564c884f0090 0x564c884f2480 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.779+0000 7fe47ba08b80 1 --2- 172.21.15.1:0/2296999063 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c88483930 0x564c884e3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.779+0000 7fe47ba08b80 1 -- 172.21.15.1:0/2296999063 >> 172.21.15.1:0/2296999063 conn(0x564c883e2db0 msgr2=0x564c88488bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:45.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.779+0000 7fe44ea00700 0 client.29015 ms_handle_reset on v2:172.21.15.71:6836/628016199 2024-02-02T15:39:45.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.779+0000 7fe47ba08b80 1 -- 172.21.15.1:0/2296999063 shutdown_connections 2024-02-02T15:39:45.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.779+0000 7fe47ba08b80 1 -- 172.21.15.1:0/2296999063 wait complete. 2024-02-02T15:39:45.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.779+0000 7fe47ba08b80 20 client.29015 trim_cache size 0 max 16384 2024-02-02T15:39:45.783 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:39:45.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.780+0000 7fe467e00700 1 -- 172.21.15.1:0/3313514457 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fe4400a0710 msgr2=0x7fe4400a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:45.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.780+0000 7fe467e00700 1 --2- 172.21.15.1:0/3313514457 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fe4400a0710 0x7fe4400a2bd0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe46019a740 tx=0x7fe448009260 comp rx=0 tx=0).stop 2024-02-02T15:39:45.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.780+0000 7fe467e00700 1 -- 172.21.15.1:0/3313514457 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe460102030 msgr2=0x7fe460199130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:45.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.780+0000 7fe467e00700 1 --2- 172.21.15.1:0/3313514457 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe460102030 0x7fe460199130 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fe450009810 tx=0x7fe45000fa00 comp rx=0 tx=0).stop 2024-02-02T15:39:45.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.780+0000 7fe467e00700 1 -- 172.21.15.1:0/3313514457 shutdown_connections 2024-02-02T15:39:45.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.780+0000 7fe467e00700 1 --2- 172.21.15.1:0/3313514457 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fe4400a0710 0x7fe4400a2bd0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.780+0000 7fe467e00700 1 --2- 172.21.15.1:0/3313514457 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe46010b400 0x7fe46019dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.780+0000 7fe467e00700 1 --2- 172.21.15.1:0/3313514457 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe460102030 0x7fe460199130 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.780+0000 7fe467e00700 1 --2- 172.21.15.1:0/3313514457 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe460102950 0x7fe460199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:45.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.780+0000 7fe467e00700 1 -- 172.21.15.1:0/3313514457 >> 172.21.15.1:0/3313514457 conn(0x7fe460076d70 msgr2=0x7fe4600fcb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:45.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.780+0000 7fe467e00700 1 -- 172.21.15.1:0/3313514457 shutdown_connections 2024-02-02T15:39:45.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.780+0000 7fe467e00700 1 -- 172.21.15.1:0/3313514457 wait complete. 2024-02-02T15:39:45.798 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-02T15:39:45.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:39:45 smithi071 ceph-mon[44707]: pgmap v397: 129 pgs: 129 active+clean; 594 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1004 KiB/s rd, 1.6 MiB/s wr, 109 op/s 2024-02-02T15:39:45.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:39:45 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/4210301728' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:39:45.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:39:45 smithi142 ceph-mon[45728]: pgmap v397: 129 pgs: 129 active+clean; 594 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1004 KiB/s rd, 1.6 MiB/s wr, 109 op/s 2024-02-02T15:39:45.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:39:45 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/4210301728' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:39:46.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.999+0000 7f34eb400700 1 -- 172.21.15.1:0/1175646190 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f34e4100ff0 msgr2=0x7f34e41013d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:46.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.999+0000 7f34eb400700 1 --2- 172.21.15.1:0/1175646190 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f34e4100ff0 0x7f34e41013d0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f34d4009990 tx=0x7f34d4009ca0 comp rx=0 tx=0).stop 2024-02-02T15:39:46.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.999+0000 7f34eb400700 1 -- 172.21.15.1:0/1175646190 shutdown_connections 2024-02-02T15:39:46.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.999+0000 7f34eb400700 1 --2- 172.21.15.1:0/1175646190 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34e4106110 0x7f34e410dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:46.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.999+0000 7f34eb400700 1 --2- 172.21.15.1:0/1175646190 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f34e4100ff0 0x7f34e41013d0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:46.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.999+0000 7f34eb400700 1 --2- 172.21.15.1:0/1175646190 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34e41019a0 0x7f34e41059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:46.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.999+0000 7f34eb400700 1 -- 172.21.15.1:0/1175646190 >> 172.21.15.1:0/1175646190 conn(0x7f34e40fc810 msgr2=0x7f34e40fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:46.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.999+0000 7f34eb400700 1 -- 172.21.15.1:0/1175646190 shutdown_connections 2024-02-02T15:39:46.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:45.999+0000 7f34eb400700 1 -- 172.21.15.1:0/1175646190 wait complete. 2024-02-02T15:39:46.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.000+0000 7f34eb400700 1 Processor -- start 2024-02-02T15:39:46.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.001+0000 7f34eb400700 1 -- start start 2024-02-02T15:39:46.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.001+0000 7f34eb400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34e4100ff0 0x7f34e4073040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:46.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.001+0000 7f34eb400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f34e4106110 0x7f34e4073580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:46.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.001+0000 7f34eb400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34e4073ac0 0x7f34e41a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:46.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.001+0000 7f34eb400700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34e41a57c0 con 0x7f34e4106110 2024-02-02T15:39:46.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.001+0000 7f34eb400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34e41a5930 con 0x7f34e4100ff0 2024-02-02T15:39:46.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.001+0000 7f34eb400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34e41a5af0 con 0x7f34e4073ac0 2024-02-02T15:39:46.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.001+0000 7f34e3e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f34e4106110 0x7f34e4073580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:46.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.001+0000 7f34e3e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f34e4106110 0x7f34e4073580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55818/0 (socket says 172.21.15.1:55818) 2024-02-02T15:39:46.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.001+0000 7f34e3e00700 1 -- 172.21.15.1:0/4017200394 learned_addr learned my addr 172.21.15.1:0/4017200394 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:39:46.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.001+0000 7f34e8a00700 1 --2- 172.21.15.1:0/4017200394 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34e4100ff0 0x7f34e4073040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:46.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.001+0000 7f34e9400700 1 --2- 172.21.15.1:0/4017200394 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34e4073ac0 0x7f34e41a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:46.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.002+0000 7f34e3e00700 1 -- 172.21.15.1:0/4017200394 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34e4073ac0 msgr2=0x7f34e41a5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:46.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.002+0000 7f34e3e00700 1 --2- 172.21.15.1:0/4017200394 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34e4073ac0 0x7f34e41a5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:46.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.002+0000 7f34e3e00700 1 -- 172.21.15.1:0/4017200394 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34e4100ff0 msgr2=0x7f34e4073040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:46.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.002+0000 7f34e3e00700 1 --2- 172.21.15.1:0/4017200394 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34e4100ff0 0x7f34e4073040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:46.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.002+0000 7f34e3e00700 1 -- 172.21.15.1:0/4017200394 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34dc009c40 con 0x7f34e4106110 2024-02-02T15:39:46.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.002+0000 7f34e3e00700 1 --2- 172.21.15.1:0/4017200394 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f34e4106110 0x7f34e4073580 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f34cc00e970 tx=0x7f34cc00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:46.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.002+0000 7f34e1600700 1 -- 172.21.15.1:0/4017200394 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34cc009790 con 0x7f34e4106110 2024-02-02T15:39:46.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.002+0000 7f34e1600700 1 -- 172.21.15.1:0/4017200394 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f34cc00c7c0 con 0x7f34e4106110 2024-02-02T15:39:46.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.002+0000 7f34e1600700 1 -- 172.21.15.1:0/4017200394 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34cc0054b0 con 0x7f34e4106110 2024-02-02T15:39:46.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.002+0000 7f34eb400700 1 -- 172.21.15.1:0/4017200394 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34d40095f0 con 0x7f34e4106110 2024-02-02T15:39:46.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.002+0000 7f34eb400700 1 -- 172.21.15.1:0/4017200394 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34e41a6100 con 0x7f34e4106110 2024-02-02T15:39:46.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.006+0000 7f34e1600700 1 -- 172.21.15.1:0/4017200394 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f34cc00c930 con 0x7f34e4106110 2024-02-02T15:39:46.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.007+0000 7f34fef93b80 1 Processor -- start 2024-02-02T15:39:46.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.007+0000 7f34fef93b80 1 -- start start 2024-02-02T15:39:46.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.007+0000 7f34fef93b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fb9725010 0x558fb97253f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:46.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.007+0000 7f34fef93b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558fb9725930 0x558fb97302a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:46.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.007+0000 7f34fef93b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558fb97307e0 0x558fb9732bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:46.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.007+0000 7f34fef93b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fb96994c0 con 0x558fb9725010 2024-02-02T15:39:46.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.007+0000 7f34fef93b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fb967daa0 con 0x558fb9725930 2024-02-02T15:39:46.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.007+0000 7f34fef93b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fb967b460 con 0x558fb97307e0 2024-02-02T15:39:46.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.008+0000 7f34e9400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fb9725010 0x558fb97253f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:46.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.008+0000 7f34e8a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558fb9725930 0x558fb97302a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:46.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.008+0000 7f34e3e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558fb97307e0 0x558fb9732bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:46.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.008+0000 7f34e9400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fb9725010 0x558fb97253f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55822/0 (socket says 172.21.15.1:55822) 2024-02-02T15:39:46.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.008+0000 7f34e9400700 1 -- 172.21.15.1:0/3835763507 learned_addr learned my addr 172.21.15.1:0/3835763507 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:39:46.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.008+0000 7f34e1600700 1 --2- 172.21.15.1:0/4017200394 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f34c40a0690 0x7f34c40a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:46.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.008+0000 7f34e8a00700 1 -- 172.21.15.1:0/3835763507 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558fb97307e0 msgr2=0x558fb9732bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:46.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.008+0000 7f34e8a00700 1 --2- 172.21.15.1:0/3835763507 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558fb97307e0 0x558fb9732bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:46.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.008+0000 7f34e8a00700 1 -- 172.21.15.1:0/3835763507 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fb9725010 msgr2=0x558fb97253f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:46.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.008+0000 7f34e8a00700 1 --2- 172.21.15.1:0/3835763507 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fb9725010 0x558fb97253f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:46.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.008+0000 7f34e9400700 1 --2- 172.21.15.1:0/4017200394 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f34c40a0690 0x7f34c40a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:46.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.008+0000 7f34e8a00700 1 -- 172.21.15.1:0/3835763507 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558fb94f83b0 con 0x558fb9725930 2024-02-02T15:39:46.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.008+0000 7f34e1600700 1 -- 172.21.15.1:0/4017200394 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f34cc014070 con 0x7f34e4106110 2024-02-02T15:39:46.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.008+0000 7f34e8a00700 1 --2- 172.21.15.1:0/3835763507 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558fb9725930 0x558fb97302a0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f34d40098e0 tx=0x7f34d4005ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df587b49d6adf6fa server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:46.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.009+0000 7f34e9400700 1 --2- 172.21.15.1:0/4017200394 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f34c40a0690 0x7f34c40a2b50 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f34dc009f80 tx=0x7f34dc004b40 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:46.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.009+0000 7f34d9c00700 1 -- 172.21.15.1:0/3835763507 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d401d070 con 0x558fb9725930 2024-02-02T15:39:46.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.009+0000 7f34d9c00700 1 -- 172.21.15.1:0/3835763507 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f34d401e040 con 0x558fb9725930 2024-02-02T15:39:46.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.009+0000 7f34d9c00700 1 -- 172.21.15.1:0/3835763507 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d4004760 con 0x558fb9725930 2024-02-02T15:39:46.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.009+0000 7f34fef93b80 1 -- 172.21.15.1:0/3835763507 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558fb9725930 msgr2=0x558fb97302a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:46.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.009+0000 7f34fef93b80 1 --2- 172.21.15.1:0/3835763507 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558fb9725930 0x558fb97302a0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f34d40098e0 tx=0x7f34d4005ea0 comp rx=0 tx=0).stop 2024-02-02T15:39:46.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.010+0000 7f34fef93b80 1 -- 172.21.15.1:0/3835763507 shutdown_connections 2024-02-02T15:39:46.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.010+0000 7f34fef93b80 1 --2- 172.21.15.1:0/3835763507 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558fb97307e0 0x558fb9732bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:46.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.010+0000 7f34fef93b80 1 --2- 172.21.15.1:0/3835763507 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fb9725010 0x558fb97253f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:46.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.010+0000 7f34fef93b80 1 --2- 172.21.15.1:0/3835763507 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558fb9725930 0x558fb97302a0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:46.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.010+0000 7f34fef93b80 1 -- 172.21.15.1:0/3835763507 >> 172.21.15.1:0/3835763507 conn(0x558fb9684db0 msgr2=0x558fb97213b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:46.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.010+0000 7f34fef93b80 1 -- 172.21.15.1:0/3835763507 shutdown_connections 2024-02-02T15:39:46.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.010+0000 7f34fef93b80 1 -- 172.21.15.1:0/3835763507 wait complete. 2024-02-02T15:39:46.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.010+0000 7f34fef93b80 1 -- start start 2024-02-02T15:39:46.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.011+0000 7f34fef93b80 1 Processor -- start 2024-02-02T15:39:46.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.011+0000 7f34fef93b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558fb9725010 0x558fb97300f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:46.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.011+0000 7f34fef93b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fb9725930 0x558fb972d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:46.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.011+0000 7f34fef93b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558fb97307e0 0x558fb972d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:46.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.011+0000 7f34fef93b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fb9676d50 con 0x558fb9725930 2024-02-02T15:39:46.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.011+0000 7f34fef93b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fb9612d80 con 0x558fb9725010 2024-02-02T15:39:46.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.011+0000 7f34fef93b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fb9612a20 con 0x558fb97307e0 2024-02-02T15:39:46.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.011+0000 7f34fef93b80 10 client.0 _ll_register_callbacks cb 0x558fb9614290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:39:46.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.012+0000 7f34e3e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fb9725930 0x558fb972d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:46.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.012+0000 7f34e3e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fb9725930 0x558fb972d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55824/0 (socket says 172.21.15.1:55824) 2024-02-02T15:39:46.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.012+0000 7f34e3e00700 1 -- 172.21.15.1:0/79572134 learned_addr learned my addr 172.21.15.1:0/79572134 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:39:46.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.012+0000 7f34e8a00700 1 --2- 172.21.15.1:0/79572134 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558fb9725010 0x558fb97300f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:46.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.012+0000 7f34e9400700 1 --2- 172.21.15.1:0/79572134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558fb97307e0 0x558fb972d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:46.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.012+0000 7f34e3e00700 1 -- 172.21.15.1:0/79572134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558fb97307e0 msgr2=0x558fb972d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:46.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.012+0000 7f34e3e00700 1 --2- 172.21.15.1:0/79572134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558fb97307e0 0x558fb972d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:46.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.012+0000 7f34e3e00700 1 -- 172.21.15.1:0/79572134 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558fb9725010 msgr2=0x558fb97300f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:46.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.012+0000 7f34e3e00700 1 --2- 172.21.15.1:0/79572134 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558fb9725010 0x558fb97300f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:46.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.012+0000 7f34e3e00700 1 -- 172.21.15.1:0/79572134 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34e410e500 con 0x558fb9725930 2024-02-02T15:39:46.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.012+0000 7f34e9400700 1 --2- 172.21.15.1:0/79572134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558fb97307e0 0x558fb972d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:39:46.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.012+0000 7f34e3e00700 1 --2- 172.21.15.1:0/79572134 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fb9725930 0x558fb972d300 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f34cc00c5b0 tx=0x7f34cc098f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aece70d02b6ce07 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:46.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.012+0000 7f34d2000700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:39:46.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.012+0000 7f34d2000700 1 -- 172.21.15.1:0/79572134 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34cc099b10 con 0x558fb9725930 2024-02-02T15:39:46.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.012+0000 7f34d2000700 1 -- 172.21.15.1:0/79572134 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f34cc099410 con 0x558fb9725930 2024-02-02T15:39:46.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.012+0000 7f34fef93b80 1 -- 172.21.15.1:0/79572134 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558fb97190f0 con 0x558fb9725930 2024-02-02T15:39:46.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.013+0000 7f34d2000700 1 -- 172.21.15.1:0/79572134 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f34cc09d240 con 0x558fb9725930 2024-02-02T15:39:46.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.013+0000 7f34fef93b80 10 client.18927 fetch_fsmap learned FSMap version 52 2024-02-02T15:39:46.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.013+0000 7f34fef93b80 1 -- 172.21.15.1:0/79572134 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x558fb94a4b60 con 0x558fb9725930 2024-02-02T15:39:46.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.013+0000 7f34d2000700 1 -- 172.21.15.1:0/79572134 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558fb97190f0 con 0x558fb9725930 2024-02-02T15:39:46.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.013+0000 7f34d2000700 1 -- 172.21.15.1:0/79572134 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 52) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f34cc016330 con 0x558fb9725930 2024-02-02T15:39:46.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.013+0000 7f34fef93b80 10 client.18927 fetch_fsmap finished waiting for FSMap version 52 2024-02-02T15:39:46.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.013+0000 7f34fef93b80 10 client.18927 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-02T15:39:46.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.013+0000 7f34fef93b80 20 client.18927 populate_metadata read hostname 'smithi001' 2024-02-02T15:39:46.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.013+0000 7f34fef93b80 1 --2- 172.21.15.1:0/79572134 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x558fb97401d0 0x558fb97425f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:46.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.013+0000 7f34fef93b80 4 client.18927 mds_command: new command op to 24529 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:39:46.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.013+0000 7f34fef93b80 1 -- 172.21.15.1:0/79572134 --> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558fb966fdd0 con 0x558fb97401d0 2024-02-02T15:39:46.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.014+0000 7f34e8a00700 1 --2- 172.21.15.1:0/79572134 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x558fb97401d0 0x558fb97425f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:46.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.014+0000 7f34e8a00700 1 --2- 172.21.15.1:0/79572134 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x558fb97401d0 0x558fb97425f0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=656a37f52b583c17 server_cookie=a52bf07fbac779f8 in_seq=0 out_seq=0 2024-02-02T15:39:46.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.014+0000 7f34d2000700 10 client.18927 ms_handle_connect on v2:172.21.15.142:6838/321675421 2024-02-02T15:39:46.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.015+0000 7f34d2000700 1 -- 172.21.15.1:0/79572134 <== mds.1 v2:172.21.15.142:6838/321675421 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x558fb966fdd0 con 0x558fb97401d0 2024-02-02T15:39:46.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.015+0000 7f34d2000700 10 client.18927 handle_command_reply: tid=0 2024-02-02T15:39:46.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.015+0000 7f34fef93b80 1 client.18927 shutdown 2024-02-02T15:39:46.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.015+0000 7f34fef93b80 1 -- 172.21.15.1:0/79572134 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fb9725930 msgr2=0x558fb972d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:46.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.015+0000 7f34fef93b80 1 --2- 172.21.15.1:0/79572134 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fb9725930 0x558fb972d300 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f34cc00c5b0 tx=0x7f34cc098f50 comp rx=0 tx=0).stop 2024-02-02T15:39:46.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.015+0000 7f34fef93b80 1 -- 172.21.15.1:0/79572134 shutdown_connections 2024-02-02T15:39:46.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.015+0000 7f34fef93b80 1 --2- 172.21.15.1:0/79572134 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x558fb97401d0 0x558fb97425f0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:46.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.015+0000 7f34fef93b80 1 --2- 172.21.15.1:0/79572134 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558fb97307e0 0x558fb972d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:46.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.015+0000 7f34fef93b80 1 --2- 172.21.15.1:0/79572134 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558fb9725010 0x558fb97300f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:46.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.015+0000 7f34fef93b80 1 --2- 172.21.15.1:0/79572134 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fb9725930 0x558fb972d300 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:46.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.015+0000 7f34d2000700 0 client.18927 ms_handle_reset on v2:172.21.15.142:6838/321675421 2024-02-02T15:39:46.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.015+0000 7f34fef93b80 1 -- 172.21.15.1:0/79572134 >> 172.21.15.1:0/79572134 conn(0x558fb9684db0 msgr2=0x558fb9720fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:46.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.015+0000 7f34fef93b80 1 -- 172.21.15.1:0/79572134 shutdown_connections 2024-02-02T15:39:46.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.015+0000 7f34fef93b80 1 -- 172.21.15.1:0/79572134 wait complete. 2024-02-02T15:39:46.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.016+0000 7f34fef93b80 20 client.18927 trim_cache size 0 max 16384 2024-02-02T15:39:46.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.029+0000 7f34fef93b80 1 Processor -- start 2024-02-02T15:39:46.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.029+0000 7f34fef93b80 1 -- start start 2024-02-02T15:39:46.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.029+0000 7f34fef93b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558fb9725010 0x558fb972fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:46.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.029+0000 7f34fef93b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fb9725930 0x558fb9727840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:46.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.029+0000 7f34fef93b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558fb97307e0 0x558fb9727d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:46.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.029+0000 7f34fef93b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fb94fa6a0 con 0x558fb9725930 2024-02-02T15:39:46.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.029+0000 7f34fef93b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fb966fdd0 con 0x558fb97307e0 2024-02-02T15:39:46.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.029+0000 7f34fef93b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fb973cb30 con 0x558fb9725010 2024-02-02T15:39:46.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.029+0000 7f34e3e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fb9725930 0x558fb9727840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:46.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.030+0000 7f34e3e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fb9725930 0x558fb9727840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55834/0 (socket says 172.21.15.1:55834) 2024-02-02T15:39:46.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.030+0000 7f34e3e00700 1 -- 172.21.15.1:0/2417953146 learned_addr learned my addr 172.21.15.1:0/2417953146 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:39:46.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.030+0000 7f34e8a00700 1 --2- 172.21.15.1:0/2417953146 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558fb9725010 0x558fb972fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:46.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.030+0000 7f34e9400700 1 --2- 172.21.15.1:0/2417953146 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558fb97307e0 0x558fb9727d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:46.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.030+0000 7f34e3e00700 1 -- 172.21.15.1:0/2417953146 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558fb9725010 msgr2=0x558fb972fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:46.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.030+0000 7f34e3e00700 1 --2- 172.21.15.1:0/2417953146 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558fb9725010 0x558fb972fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:46.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.030+0000 7f34e3e00700 1 -- 172.21.15.1:0/2417953146 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558fb97307e0 msgr2=0x558fb9727d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:46.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.030+0000 7f34e3e00700 1 --2- 172.21.15.1:0/2417953146 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558fb97307e0 0x558fb9727d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:46.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.030+0000 7f34e3e00700 1 -- 172.21.15.1:0/2417953146 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558fb971d0d0 con 0x558fb9725930 2024-02-02T15:39:46.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.030+0000 7f34e8a00700 1 --2- 172.21.15.1:0/2417953146 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558fb9725010 0x558fb972fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:39:46.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.030+0000 7f34e3e00700 1 --2- 172.21.15.1:0/2417953146 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fb9725930 0x558fb9727840 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f34cc00cbe0 tx=0x7f34cc098e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b00c45322af75839 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:46.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.030+0000 7f34da600700 1 -- 172.21.15.1:0/2417953146 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34cc095750 con 0x558fb9725930 2024-02-02T15:39:46.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.030+0000 7f34da600700 1 -- 172.21.15.1:0/2417953146 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f34cc0cf040 con 0x558fb9725930 2024-02-02T15:39:46.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.030+0000 7f34da600700 1 -- 172.21.15.1:0/2417953146 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34cc016330 con 0x558fb9725930 2024-02-02T15:39:46.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.031+0000 7f34fef93b80 1 -- 172.21.15.1:0/2417953146 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fb9725930 msgr2=0x558fb9727840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:46.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.031+0000 7f34fef93b80 1 --2- 172.21.15.1:0/2417953146 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fb9725930 0x558fb9727840 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f34cc00cbe0 tx=0x7f34cc098e50 comp rx=0 tx=0).stop 2024-02-02T15:39:46.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.031+0000 7f34fef93b80 1 -- 172.21.15.1:0/2417953146 shutdown_connections 2024-02-02T15:39:46.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.031+0000 7f34fef93b80 1 --2- 172.21.15.1:0/2417953146 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558fb97307e0 0x558fb9727d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:46.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.031+0000 7f34fef93b80 1 --2- 172.21.15.1:0/2417953146 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558fb9725010 0x558fb972fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:46.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.031+0000 7f34fef93b80 1 --2- 172.21.15.1:0/2417953146 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fb9725930 0x558fb9727840 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:46.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.031+0000 7f34fef93b80 1 -- 172.21.15.1:0/2417953146 >> 172.21.15.1:0/2417953146 conn(0x558fb9684db0 msgr2=0x558fb973b910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:46.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.031+0000 7f34fef93b80 1 -- 172.21.15.1:0/2417953146 shutdown_connections 2024-02-02T15:39:46.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.031+0000 7f34fef93b80 1 -- 172.21.15.1:0/2417953146 wait complete. 2024-02-02T15:39:46.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.031+0000 7f34fef93b80 1 -- start start 2024-02-02T15:39:46.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.032+0000 7f34fef93b80 1 Processor -- start 2024-02-02T15:39:46.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.032+0000 7f34fef93b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558fb9725010 0x558fb9734b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:46.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.032+0000 7f34fef93b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558fb9725930 0x558fb97350c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:46.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.032+0000 7f34fef93b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fb97307e0 0x558fb9790cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:46.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.032+0000 7f34fef93b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fb9735900 con 0x558fb97307e0 2024-02-02T15:39:46.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.032+0000 7f34fef93b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fb9791230 con 0x558fb9725930 2024-02-02T15:39:46.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.032+0000 7f34fef93b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558fb9791370 con 0x558fb9725010 2024-02-02T15:39:46.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.032+0000 7f34fef93b80 10 client.0 _ll_register_callbacks cb 0x558fb9614290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:39:46.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.032+0000 7f34e9400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fb97307e0 0x558fb9790cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:46.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.032+0000 7f34e8a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558fb9725010 0x558fb9734b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:46.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.032+0000 7f34e9400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fb97307e0 0x558fb9790cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55848/0 (socket says 172.21.15.1:55848) 2024-02-02T15:39:46.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.032+0000 7f34e3e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558fb9725930 0x558fb97350c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:46.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.032+0000 7f34e9400700 1 -- 172.21.15.1:0/875254864 learned_addr learned my addr 172.21.15.1:0/875254864 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:39:46.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.033+0000 7f34e3e00700 1 -- 172.21.15.1:0/875254864 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558fb9725010 msgr2=0x558fb9734b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:46.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.033+0000 7f34e3e00700 1 --2- 172.21.15.1:0/875254864 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558fb9725010 0x558fb9734b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:46.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.033+0000 7f34e3e00700 1 -- 172.21.15.1:0/875254864 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fb97307e0 msgr2=0x558fb9790cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:46.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.033+0000 7f34e3e00700 1 --2- 172.21.15.1:0/875254864 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fb97307e0 0x558fb9790cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:46.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.033+0000 7f34e3e00700 1 -- 172.21.15.1:0/875254864 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558fb94f83b0 con 0x558fb9725930 2024-02-02T15:39:46.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.033+0000 7f34e3e00700 1 --2- 172.21.15.1:0/875254864 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558fb9725930 0x558fb97350c0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x558fb97400c0 tx=0x7f34cc0cc1f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa562aca54d0784d server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:46.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.033+0000 7f34d2000700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:39:46.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.033+0000 7f34d2000700 1 -- 172.21.15.1:0/875254864 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34cc099870 con 0x558fb9725930 2024-02-02T15:39:46.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.033+0000 7f34d2000700 1 -- 172.21.15.1:0/875254864 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f34cc099410 con 0x558fb9725930 2024-02-02T15:39:46.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.033+0000 7f34fef93b80 1 -- 172.21.15.1:0/875254864 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f34d401e040 con 0x558fb9725930 2024-02-02T15:39:46.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.033+0000 7f34d2000700 1 -- 172.21.15.1:0/875254864 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f34cc099b10 con 0x558fb9725930 2024-02-02T15:39:46.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.033+0000 7f34fef93b80 10 client.29033 fetch_fsmap learned FSMap version 52 2024-02-02T15:39:46.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.033+0000 7f34fef93b80 1 -- 172.21.15.1:0/875254864 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x558fb94a4b60 con 0x558fb9725930 2024-02-02T15:39:46.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.034+0000 7f34d2000700 1 -- 172.21.15.1:0/875254864 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f34d401e040 con 0x558fb9725930 2024-02-02T15:39:46.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.034+0000 7f34d2000700 1 -- 172.21.15.1:0/875254864 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 52) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f34cc0a0ff0 con 0x558fb9725930 2024-02-02T15:39:46.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.034+0000 7f34fef93b80 10 client.29033 fetch_fsmap finished waiting for FSMap version 52 2024-02-02T15:39:46.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.034+0000 7f34fef93b80 10 client.29033 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-02T15:39:46.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.034+0000 7f34fef93b80 20 client.29033 populate_metadata read hostname 'smithi001' 2024-02-02T15:39:46.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.034+0000 7f34fef93b80 1 --2- 172.21.15.1:0/875254864 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x558fb9792500 0x558fb97948f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:46.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.034+0000 7f34fef93b80 4 client.29033 mds_command: new command op to 24529 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:39:46.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.034+0000 7f34fef93b80 1 -- 172.21.15.1:0/875254864 --> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558fb9635f50 con 0x558fb9792500 2024-02-02T15:39:46.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.034+0000 7f34e8a00700 1 --2- 172.21.15.1:0/875254864 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x558fb9792500 0x558fb97948f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:46.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.035+0000 7f34e8a00700 1 --2- 172.21.15.1:0/875254864 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x558fb9792500 0x558fb97948f0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=54e5a0f70bc04a14 server_cookie=42a5b1b58dd689ed in_seq=0 out_seq=0 2024-02-02T15:39:46.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.035+0000 7f34d2000700 10 client.29033 ms_handle_connect on v2:172.21.15.142:6838/321675421 2024-02-02T15:39:46.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.035+0000 7f34d2000700 1 -- 172.21.15.1:0/875254864 <== mds.1 v2:172.21.15.142:6838/321675421 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558fb9635f50 con 0x558fb9792500 2024-02-02T15:39:46.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.035+0000 7f34d2000700 10 client.29033 handle_command_reply: tid=0 2024-02-02T15:39:46.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.035+0000 7f34fef93b80 1 client.29033 shutdown 2024-02-02T15:39:46.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.035+0000 7f34fef93b80 1 -- 172.21.15.1:0/875254864 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558fb9725930 msgr2=0x558fb97350c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:46.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.035+0000 7f34fef93b80 1 --2- 172.21.15.1:0/875254864 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558fb9725930 0x558fb97350c0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x558fb97400c0 tx=0x7f34cc0cc1f0 comp rx=0 tx=0).stop 2024-02-02T15:39:46.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.035+0000 7f34fef93b80 1 -- 172.21.15.1:0/875254864 shutdown_connections 2024-02-02T15:39:46.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.035+0000 7f34fef93b80 1 --2- 172.21.15.1:0/875254864 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x558fb9792500 0x558fb97948f0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:46.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.035+0000 7f34fef93b80 1 --2- 172.21.15.1:0/875254864 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558fb97307e0 0x558fb9790cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:46.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.035+0000 7f34fef93b80 1 --2- 172.21.15.1:0/875254864 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558fb9725010 0x558fb9734b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:46.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.035+0000 7f34fef93b80 1 --2- 172.21.15.1:0/875254864 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558fb9725930 0x558fb97350c0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:46.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.035+0000 7f34d2000700 0 client.29033 ms_handle_reset on v2:172.21.15.142:6838/321675421 2024-02-02T15:39:46.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.035+0000 7f34fef93b80 1 -- 172.21.15.1:0/875254864 >> 172.21.15.1:0/875254864 conn(0x558fb9684db0 msgr2=0x558fb9720e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:46.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.035+0000 7f34fef93b80 1 -- 172.21.15.1:0/875254864 shutdown_connections 2024-02-02T15:39:46.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.035+0000 7f34fef93b80 1 -- 172.21.15.1:0/875254864 wait complete. 2024-02-02T15:39:46.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.035+0000 7f34fef93b80 20 client.29033 trim_cache size 0 max 16384 2024-02-02T15:39:46.049 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:39:46.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.036+0000 7f34eb400700 1 -- 172.21.15.1:0/4017200394 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f34c40a0690 msgr2=0x7f34c40a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:46.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.036+0000 7f34eb400700 1 --2- 172.21.15.1:0/4017200394 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f34c40a0690 0x7f34c40a2b50 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f34dc009f80 tx=0x7f34dc004b40 comp rx=0 tx=0).stop 2024-02-02T15:39:46.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.036+0000 7f34eb400700 1 -- 172.21.15.1:0/4017200394 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f34e4106110 msgr2=0x7f34e4073580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:46.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.036+0000 7f34eb400700 1 --2- 172.21.15.1:0/4017200394 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f34e4106110 0x7f34e4073580 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f34cc00e970 tx=0x7f34cc00ec80 comp rx=0 tx=0).stop 2024-02-02T15:39:46.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.036+0000 7f34eb400700 1 -- 172.21.15.1:0/4017200394 shutdown_connections 2024-02-02T15:39:46.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.036+0000 7f34eb400700 1 --2- 172.21.15.1:0/4017200394 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f34c40a0690 0x7f34c40a2b50 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:46.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.036+0000 7f34eb400700 1 --2- 172.21.15.1:0/4017200394 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f34e4073ac0 0x7f34e41a5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:46.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.036+0000 7f34eb400700 1 --2- 172.21.15.1:0/4017200394 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f34e4100ff0 0x7f34e4073040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:46.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.036+0000 7f34eb400700 1 --2- 172.21.15.1:0/4017200394 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f34e4106110 0x7f34e4073580 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:46.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.036+0000 7f34eb400700 1 -- 172.21.15.1:0/4017200394 >> 172.21.15.1:0/4017200394 conn(0x7f34e40fc810 msgr2=0x7f34e4104a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:46.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.036+0000 7f34eb400700 1 -- 172.21.15.1:0/4017200394 shutdown_connections 2024-02-02T15:39:46.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.036+0000 7f34eb400700 1 -- 172.21.15.1:0/4017200394 wait complete. 2024-02-02T15:39:46.054 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-02T15:39:46.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.256+0000 7f7f88200700 1 -- 172.21.15.1:0/1772191984 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f80100ff0 msgr2=0x7f7f801013d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:46.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.256+0000 7f7f88200700 1 --2- 172.21.15.1:0/1772191984 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f80100ff0 0x7f7f801013d0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f7f70009990 tx=0x7f7f70009ca0 comp rx=0 tx=0).stop 2024-02-02T15:39:46.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.256+0000 7f7f88200700 1 -- 172.21.15.1:0/1772191984 shutdown_connections 2024-02-02T15:39:46.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.256+0000 7f7f88200700 1 --2- 172.21.15.1:0/1772191984 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f80106110 0x7f7f8010dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:46.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.256+0000 7f7f88200700 1 --2- 172.21.15.1:0/1772191984 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f80100ff0 0x7f7f801013d0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:46.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.256+0000 7f7f88200700 1 --2- 172.21.15.1:0/1772191984 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7f801019a0 0x7f7f801059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:46.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.256+0000 7f7f88200700 1 -- 172.21.15.1:0/1772191984 >> 172.21.15.1:0/1772191984 conn(0x7f7f800fc810 msgr2=0x7f7f800fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:46.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.256+0000 7f7f88200700 1 -- 172.21.15.1:0/1772191984 shutdown_connections 2024-02-02T15:39:46.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.256+0000 7f7f88200700 1 -- 172.21.15.1:0/1772191984 wait complete. 2024-02-02T15:39:46.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.257+0000 7f7f88200700 1 Processor -- start 2024-02-02T15:39:46.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.257+0000 7f7f88200700 1 -- start start 2024-02-02T15:39:46.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.257+0000 7f7f88200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f80100ff0 0x7f7f80199080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:46.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.257+0000 7f7f88200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f80106110 0x7f7f801995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:46.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.257+0000 7f7f88200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7f8019da20 0x7f7f8019dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:46.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.257+0000 7f7f88200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8019e4a0 con 0x7f7f8019da20 2024-02-02T15:39:46.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.257+0000 7f7f88200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8019e610 con 0x7f7f80100ff0 2024-02-02T15:39:46.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.257+0000 7f7f88200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8019e7d0 con 0x7f7f80106110 2024-02-02T15:39:46.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.258+0000 7f7f86200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7f8019da20 0x7f7f8019dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:46.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.258+0000 7f7f84e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f80106110 0x7f7f801995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:46.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.258+0000 7f7f84e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f80106110 0x7f7f801995c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:34188/0 (socket says 172.21.15.1:34188) 2024-02-02T15:39:46.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.258+0000 7f7f84e00700 1 -- 172.21.15.1:0/3185363911 learned_addr learned my addr 172.21.15.1:0/3185363911 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:39:46.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.258+0000 7f7f86200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7f8019da20 0x7f7f8019dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55860/0 (socket says 172.21.15.1:55860) 2024-02-02T15:39:46.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.258+0000 7f7f85800700 1 --2- 172.21.15.1:0/3185363911 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f80100ff0 0x7f7f80199080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:46.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.258+0000 7f7f84e00700 1 -- 172.21.15.1:0/3185363911 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f80100ff0 msgr2=0x7f7f80199080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:46.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.258+0000 7f7f84e00700 1 --2- 172.21.15.1:0/3185363911 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f80100ff0 0x7f7f80199080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:46.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.258+0000 7f7f84e00700 1 -- 172.21.15.1:0/3185363911 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7f8019da20 msgr2=0x7f7f8019dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:46.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.258+0000 7f7f84e00700 1 --2- 172.21.15.1:0/3185363911 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7f8019da20 0x7f7f8019dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:46.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.258+0000 7f7f84e00700 1 -- 172.21.15.1:0/3185363911 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f700095f0 con 0x7f7f80106110 2024-02-02T15:39:46.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.258+0000 7f7f86200700 1 --2- 172.21.15.1:0/3185363911 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7f8019da20 0x7f7f8019dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:39:46.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.258+0000 7f7f85800700 1 --2- 172.21.15.1:0/3185363911 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f80100ff0 0x7f7f80199080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:39:46.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.258+0000 7f7f84e00700 1 --2- 172.21.15.1:0/3185363911 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f80106110 0x7f7f801995c0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f7f6800e970 tx=0x7f7f6800ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:46.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.259+0000 7f7f7e000700 1 -- 172.21.15.1:0/3185363911 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f68009790 con 0x7f7f80106110 2024-02-02T15:39:46.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.259+0000 7f7f88200700 1 -- 172.21.15.1:0/3185363911 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f801a5d30 con 0x7f7f80106110 2024-02-02T15:39:46.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.259+0000 7f7f7e000700 1 -- 172.21.15.1:0/3185363911 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7f6800c7e0 con 0x7f7f80106110 2024-02-02T15:39:46.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.259+0000 7f7f7e000700 1 -- 172.21.15.1:0/3185363911 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f680054e0 con 0x7f7f80106110 2024-02-02T15:39:46.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.259+0000 7f7f88200700 1 -- 172.21.15.1:0/3185363911 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f801a6200 con 0x7f7f80106110 2024-02-02T15:39:46.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.262+0000 7f7f7e000700 1 -- 172.21.15.1:0/3185363911 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f7f6800c950 con 0x7f7f80106110 2024-02-02T15:39:46.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.263+0000 7f7f9bd91b80 1 Processor -- start 2024-02-02T15:39:46.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.263+0000 7f7f7e000700 1 --2- 172.21.15.1:0/3185363911 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f7f600a0690 0x7f7f600a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:46.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.263+0000 7f7f9bd91b80 1 -- start start 2024-02-02T15:39:46.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.263+0000 7f7f7e000700 1 -- 172.21.15.1:0/3185363911 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f7f68014070 con 0x7f7f80106110 2024-02-02T15:39:46.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.264+0000 7f7f86200700 1 --2- 172.21.15.1:0/3185363911 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f7f600a0690 0x7f7f600a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:46.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.264+0000 7f7f9bd91b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56160c685010 0x56160c6853f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:46.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.264+0000 7f7f9bd91b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56160c685930 0x56160c6902a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:46.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.264+0000 7f7f9bd91b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56160c6907e0 0x56160c692bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:46.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.264+0000 7f7f9bd91b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56160c5f94c0 con 0x56160c6907e0 2024-02-02T15:39:46.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.264+0000 7f7f9bd91b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56160c5ddaa0 con 0x56160c685930 2024-02-02T15:39:46.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.264+0000 7f7f9bd91b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56160c5db460 con 0x56160c685010 2024-02-02T15:39:46.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.264+0000 7f7f85800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56160c685010 0x56160c6853f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:46.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.264+0000 7f7f84e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56160c685930 0x56160c6902a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:46.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.264+0000 7f7f85800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56160c685010 0x56160c6853f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:34198/0 (socket says 172.21.15.1:34198) 2024-02-02T15:39:46.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.264+0000 7f7f85800700 1 -- 172.21.15.1:0/1629236200 learned_addr learned my addr 172.21.15.1:0/1629236200 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:39:46.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.264+0000 7f7f86200700 1 --2- 172.21.15.1:0/1629236200 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56160c6907e0 0x56160c692bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:46.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.265+0000 7f7f84e00700 1 -- 172.21.15.1:0/1629236200 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56160c685010 msgr2=0x56160c6853f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:46.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.265+0000 7f7f84e00700 1 --2- 172.21.15.1:0/1629236200 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56160c685010 0x56160c6853f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:46.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.265+0000 7f7f84e00700 1 -- 172.21.15.1:0/1629236200 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56160c6907e0 msgr2=0x56160c692bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:46.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.265+0000 7f7f84e00700 1 --2- 172.21.15.1:0/1629236200 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56160c6907e0 0x56160c692bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:46.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.265+0000 7f7f84e00700 1 -- 172.21.15.1:0/1629236200 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56160c4583b0 con 0x56160c685930 2024-02-02T15:39:46.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.265+0000 7f7f85800700 1 --2- 172.21.15.1:0/1629236200 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56160c685010 0x56160c6853f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:39:46.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.265+0000 7f7f86200700 1 --2- 172.21.15.1:0/1629236200 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56160c6907e0 0x56160c692bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:39:46.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.265+0000 7f7f86200700 1 --2- 172.21.15.1:0/3185363911 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f7f600a0690 0x7f7f600a2b50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7f78007630 tx=0x7f7f780079f0 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:46.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.265+0000 7f7f84e00700 1 --2- 172.21.15.1:0/1629236200 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56160c685930 0x56160c6902a0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f7f680c96c0 tx=0x7f7f68099130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9aa0d09dfd3f91d3 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:46.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.265+0000 7f7f76a00700 1 -- 172.21.15.1:0/1629236200 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f680cc170 con 0x56160c685930 2024-02-02T15:39:46.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.265+0000 7f7f76a00700 1 -- 172.21.15.1:0/1629236200 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7f680994c0 con 0x56160c685930 2024-02-02T15:39:46.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.265+0000 7f7f76a00700 1 -- 172.21.15.1:0/1629236200 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f680056f0 con 0x56160c685930 2024-02-02T15:39:46.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.266+0000 7f7f9bd91b80 1 -- 172.21.15.1:0/1629236200 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56160c685930 msgr2=0x56160c6902a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:46.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.266+0000 7f7f9bd91b80 1 --2- 172.21.15.1:0/1629236200 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56160c685930 0x56160c6902a0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f7f680c96c0 tx=0x7f7f68099130 comp rx=0 tx=0).stop 2024-02-02T15:39:46.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.266+0000 7f7f9bd91b80 1 -- 172.21.15.1:0/1629236200 shutdown_connections 2024-02-02T15:39:46.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.266+0000 7f7f9bd91b80 1 --2- 172.21.15.1:0/1629236200 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56160c6907e0 0x56160c692bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:46.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.266+0000 7f7f9bd91b80 1 --2- 172.21.15.1:0/1629236200 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56160c685010 0x56160c6853f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:46.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.266+0000 7f7f9bd91b80 1 --2- 172.21.15.1:0/1629236200 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56160c685930 0x56160c6902a0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:46.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.266+0000 7f7f9bd91b80 1 -- 172.21.15.1:0/1629236200 >> 172.21.15.1:0/1629236200 conn(0x56160c5e4db0 msgr2=0x56160c6813b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:46.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.266+0000 7f7f9bd91b80 1 -- 172.21.15.1:0/1629236200 shutdown_connections 2024-02-02T15:39:46.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.266+0000 7f7f9bd91b80 1 -- 172.21.15.1:0/1629236200 wait complete. 2024-02-02T15:39:46.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.267+0000 7f7f9bd91b80 1 -- start start 2024-02-02T15:39:46.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.267+0000 7f7f9bd91b80 1 Processor -- start 2024-02-02T15:39:46.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.267+0000 7f7f9bd91b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56160c685010 0x56160c690240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:46.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.267+0000 7f7f9bd91b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56160c685930 0x56160c68d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:46.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.267+0000 7f7f9bd91b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56160c6907e0 0x56160c68db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:46.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.267+0000 7f7f9bd91b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56160c5d6d50 con 0x56160c6907e0 2024-02-02T15:39:46.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.267+0000 7f7f9bd91b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56160c572d80 con 0x56160c685930 2024-02-02T15:39:46.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.267+0000 7f7f9bd91b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56160c572a20 con 0x56160c685010 2024-02-02T15:39:46.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.268+0000 7f7f9bd91b80 10 client.0 _ll_register_callbacks cb 0x56160c574290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:39:46.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.268+0000 7f7f84e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56160c685930 0x56160c68d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:46.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.268+0000 7f7f86200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56160c6907e0 0x56160c68db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:46.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.268+0000 7f7f84e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56160c685930 0x56160c68d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:41498/0 (socket says 172.21.15.1:41498) 2024-02-02T15:39:46.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.268+0000 7f7f85800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56160c685010 0x56160c690240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:46.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.268+0000 7f7f84e00700 1 -- 172.21.15.1:0/594334079 learned_addr learned my addr 172.21.15.1:0/594334079 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:39:46.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.268+0000 7f7f84e00700 1 -- 172.21.15.1:0/594334079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56160c685010 msgr2=0x56160c690240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:46.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.268+0000 7f7f84e00700 1 --2- 172.21.15.1:0/594334079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56160c685010 0x56160c690240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:46.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.268+0000 7f7f84e00700 1 -- 172.21.15.1:0/594334079 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56160c6907e0 msgr2=0x56160c68db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:46.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.268+0000 7f7f84e00700 1 --2- 172.21.15.1:0/594334079 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56160c6907e0 0x56160c68db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:46.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.268+0000 7f7f84e00700 1 -- 172.21.15.1:0/594334079 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f8010e500 con 0x56160c685930 2024-02-02T15:39:46.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.268+0000 7f7f84e00700 1 --2- 172.21.15.1:0/594334079 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56160c685930 0x56160c68d450 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f7f680991c0 tx=0x7f7f6809d170 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aef0fa9f61ead13e server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:46.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.268+0000 7f7f6f400700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:39:46.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.268+0000 7f7f6f400700 1 -- 172.21.15.1:0/594334079 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f680a1530 con 0x56160c685930 2024-02-02T15:39:46.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.269+0000 7f7f6f400700 1 -- 172.21.15.1:0/594334079 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f7f680c9e40 con 0x56160c685930 2024-02-02T15:39:46.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.269+0000 7f7f9bd91b80 1 -- 172.21.15.1:0/594334079 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56160c6790f0 con 0x56160c685930 2024-02-02T15:39:46.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.269+0000 7f7f6f400700 1 -- 172.21.15.1:0/594334079 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f680cc170 con 0x56160c685930 2024-02-02T15:39:46.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.269+0000 7f7f9bd91b80 10 client.29045 fetch_fsmap learned FSMap version 52 2024-02-02T15:39:46.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.269+0000 7f7f9bd91b80 1 -- 172.21.15.1:0/594334079 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x7f7f801a6200 con 0x56160c685930 2024-02-02T15:39:46.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.269+0000 7f7f6f400700 1 -- 172.21.15.1:0/594334079 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56160c6790f0 con 0x56160c685930 2024-02-02T15:39:46.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.269+0000 7f7f6f400700 1 -- 172.21.15.1:0/594334079 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 52) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f7f680989b0 con 0x56160c685930 2024-02-02T15:39:46.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.269+0000 7f7f9bd91b80 10 client.29045 fetch_fsmap finished waiting for FSMap version 52 2024-02-02T15:39:46.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.269+0000 7f7f9bd91b80 10 client.29045 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:39:46.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.269+0000 7f7f9bd91b80 20 client.29045 populate_metadata read hostname 'smithi001' 2024-02-02T15:39:46.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.269+0000 7f7f9bd91b80 1 --2- 172.21.15.1:0/594334079 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56160c69ff00 0x56160c6a22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:46.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.269+0000 7f7f9bd91b80 4 client.29045 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:39:46.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.269+0000 7f7f9bd91b80 1 -- 172.21.15.1:0/594334079 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56160c5cfdd0 con 0x56160c69ff00 2024-02-02T15:39:46.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.270+0000 7f7f85800700 1 --2- 172.21.15.1:0/594334079 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56160c69ff00 0x56160c6a22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:46.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.270+0000 7f7f6f400700 10 client.29045 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:39:46.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.270+0000 7f7f85800700 1 --2- 172.21.15.1:0/594334079 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56160c69ff00 0x56160c6a22f0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd537e3173b89e33 server_cookie=d8373f96c58614ec in_seq=0 out_seq=0 2024-02-02T15:39:46.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.271+0000 7f7f6f400700 1 -- 172.21.15.1:0/594334079 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x56160c5cfdd0 con 0x56160c69ff00 2024-02-02T15:39:46.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.271+0000 7f7f6f400700 10 client.29045 handle_command_reply: tid=0 2024-02-02T15:39:46.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.271+0000 7f7f9bd91b80 1 client.29045 shutdown 2024-02-02T15:39:46.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.271+0000 7f7f9bd91b80 1 -- 172.21.15.1:0/594334079 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56160c685930 msgr2=0x56160c68d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:46.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.271+0000 7f7f9bd91b80 1 --2- 172.21.15.1:0/594334079 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56160c685930 0x56160c68d450 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f7f680991c0 tx=0x7f7f6809d170 comp rx=0 tx=0).stop 2024-02-02T15:39:46.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.271+0000 7f7f9bd91b80 1 -- 172.21.15.1:0/594334079 shutdown_connections 2024-02-02T15:39:46.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.271+0000 7f7f9bd91b80 1 --2- 172.21.15.1:0/594334079 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56160c69ff00 0x56160c6a22f0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:46.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.271+0000 7f7f9bd91b80 1 --2- 172.21.15.1:0/594334079 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56160c6907e0 0x56160c68db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:46.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.271+0000 7f7f9bd91b80 1 --2- 172.21.15.1:0/594334079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56160c685010 0x56160c690240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:46.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.271+0000 7f7f9bd91b80 1 --2- 172.21.15.1:0/594334079 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56160c685930 0x56160c68d450 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:46.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.271+0000 7f7f9bd91b80 1 -- 172.21.15.1:0/594334079 >> 172.21.15.1:0/594334079 conn(0x56160c5e4db0 msgr2=0x56160c680fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:46.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.271+0000 7f7f6f400700 0 client.29045 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:39:46.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.271+0000 7f7f9bd91b80 1 -- 172.21.15.1:0/594334079 shutdown_connections 2024-02-02T15:39:46.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.271+0000 7f7f9bd91b80 1 -- 172.21.15.1:0/594334079 wait complete. 2024-02-02T15:39:46.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.271+0000 7f7f9bd91b80 20 client.29045 trim_cache size 0 max 16384 2024-02-02T15:39:46.275 INFO:tasks.workunit.client.0.smithi001.stdout: 1 694183 5.36 MB/sec execute 560 sec latency 1156.734 ms 2024-02-02T15:39:46.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.285+0000 7f7f9bd91b80 1 Processor -- start 2024-02-02T15:39:46.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.285+0000 7f7f9bd91b80 1 -- start start 2024-02-02T15:39:46.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.285+0000 7f7f9bd91b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56160c685010 0x56160c687f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:46.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.285+0000 7f7f9bd91b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56160c685930 0x56160c68fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:46.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.285+0000 7f7f9bd91b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56160c6907e0 0x56160c68ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:46.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.285+0000 7f7f9bd91b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56160c45a6a0 con 0x56160c6907e0 2024-02-02T15:39:46.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.285+0000 7f7f9bd91b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56160c6892e0 con 0x56160c685010 2024-02-02T15:39:46.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.285+0000 7f7f9bd91b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56160c67ce20 con 0x56160c685930 2024-02-02T15:39:46.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.286+0000 7f7f86200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56160c6907e0 0x56160c68ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:46.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.286+0000 7f7f86200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56160c6907e0 0x56160c68ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55878/0 (socket says 172.21.15.1:55878) 2024-02-02T15:39:46.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.286+0000 7f7f86200700 1 -- 172.21.15.1:0/1070902872 learned_addr learned my addr 172.21.15.1:0/1070902872 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:39:46.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.286+0000 7f7f84e00700 1 --2- 172.21.15.1:0/1070902872 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56160c685930 0x56160c68fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:46.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.286+0000 7f7f85800700 1 --2- 172.21.15.1:0/1070902872 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56160c685010 0x56160c687f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:46.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.286+0000 7f7f86200700 1 -- 172.21.15.1:0/1070902872 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56160c685930 msgr2=0x56160c68fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:46.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.286+0000 7f7f86200700 1 --2- 172.21.15.1:0/1070902872 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56160c685930 0x56160c68fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:46.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.286+0000 7f7f86200700 1 -- 172.21.15.1:0/1070902872 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56160c685010 msgr2=0x56160c687f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:46.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.286+0000 7f7f86200700 1 --2- 172.21.15.1:0/1070902872 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56160c685010 0x56160c687f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:46.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.286+0000 7f7f86200700 1 -- 172.21.15.1:0/1070902872 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56160c404b60 con 0x56160c6907e0 2024-02-02T15:39:46.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.286+0000 7f7f86200700 1 --2- 172.21.15.1:0/1070902872 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56160c6907e0 0x56160c68ccf0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f7f78018a20 tx=0x7f7f78015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7effb56f8fef603 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:46.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.286+0000 7f7f77400700 1 -- 172.21.15.1:0/1070902872 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f78003c00 con 0x56160c6907e0 2024-02-02T15:39:46.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.287+0000 7f7f77400700 1 -- 172.21.15.1:0/1070902872 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7f7800f040 con 0x56160c6907e0 2024-02-02T15:39:46.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.287+0000 7f7f77400700 1 -- 172.21.15.1:0/1070902872 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f780046a0 con 0x56160c6907e0 2024-02-02T15:39:46.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.287+0000 7f7f9bd91b80 1 -- 172.21.15.1:0/1070902872 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56160c6907e0 msgr2=0x56160c68ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:46.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.287+0000 7f7f9bd91b80 1 --2- 172.21.15.1:0/1070902872 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56160c6907e0 0x56160c68ccf0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f7f78018a20 tx=0x7f7f78015000 comp rx=0 tx=0).stop 2024-02-02T15:39:46.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.287+0000 7f7f9bd91b80 1 -- 172.21.15.1:0/1070902872 shutdown_connections 2024-02-02T15:39:46.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.287+0000 7f7f9bd91b80 1 --2- 172.21.15.1:0/1070902872 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56160c6907e0 0x56160c68ccf0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:46.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.287+0000 7f7f9bd91b80 1 --2- 172.21.15.1:0/1070902872 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56160c685010 0x56160c687f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:46.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.287+0000 7f7f9bd91b80 1 --2- 172.21.15.1:0/1070902872 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56160c685930 0x56160c68fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:46.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.287+0000 7f7f9bd91b80 1 -- 172.21.15.1:0/1070902872 >> 172.21.15.1:0/1070902872 conn(0x56160c5e4db0 msgr2=0x56160c69b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:46.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.287+0000 7f7f9bd91b80 1 -- 172.21.15.1:0/1070902872 shutdown_connections 2024-02-02T15:39:46.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.287+0000 7f7f9bd91b80 1 -- 172.21.15.1:0/1070902872 wait complete. 2024-02-02T15:39:46.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.288+0000 7f7f9bd91b80 1 -- start start 2024-02-02T15:39:46.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.288+0000 7f7f9bd91b80 1 Processor -- start 2024-02-02T15:39:46.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.288+0000 7f7f9bd91b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56160c685010 0x56160c696e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:46.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.288+0000 7f7f9bd91b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56160c685930 0x56160c695310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:46.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.288+0000 7f7f9bd91b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56160c6907e0 0x56160c695850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:46.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.288+0000 7f7f9bd91b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56160c697740 con 0x56160c685930 2024-02-02T15:39:46.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.288+0000 7f7f9bd91b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56160c695d90 con 0x56160c685010 2024-02-02T15:39:46.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.288+0000 7f7f9bd91b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56160c695f20 con 0x56160c6907e0 2024-02-02T15:39:46.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.288+0000 7f7f9bd91b80 10 client.0 _ll_register_callbacks cb 0x56160c574290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:39:46.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.289+0000 7f7f84e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56160c685930 0x56160c695310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:46.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.289+0000 7f7f86200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56160c6907e0 0x56160c695850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:46.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.289+0000 7f7f84e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56160c685930 0x56160c695310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55886/0 (socket says 172.21.15.1:55886) 2024-02-02T15:39:46.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.289+0000 7f7f84e00700 1 -- 172.21.15.1:0/2417928692 learned_addr learned my addr 172.21.15.1:0/2417928692 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:39:46.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.289+0000 7f7f85800700 1 --2- 172.21.15.1:0/2417928692 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56160c685010 0x56160c696e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:46.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.289+0000 7f7f84e00700 1 -- 172.21.15.1:0/2417928692 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56160c6907e0 msgr2=0x56160c695850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:46.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.289+0000 7f7f84e00700 1 --2- 172.21.15.1:0/2417928692 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56160c6907e0 0x56160c695850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:46.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.289+0000 7f7f84e00700 1 -- 172.21.15.1:0/2417928692 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56160c685010 msgr2=0x56160c696e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:46.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.289+0000 7f7f84e00700 1 --2- 172.21.15.1:0/2417928692 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56160c685010 0x56160c696e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:46.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.289+0000 7f7f84e00700 1 -- 172.21.15.1:0/2417928692 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f78018650 con 0x56160c685930 2024-02-02T15:39:46.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.289+0000 7f7f84e00700 1 --2- 172.21.15.1:0/2417928692 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56160c685930 0x56160c695310 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f7f680a1000 tx=0x7f7f68098ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89d61fdaea349f3c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:46.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.289+0000 7f7f6f400700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:39:46.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.290+0000 7f7f6f400700 1 -- 172.21.15.1:0/2417928692 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f680b7460 con 0x56160c685930 2024-02-02T15:39:46.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.290+0000 7f7f6f400700 1 -- 172.21.15.1:0/2417928692 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f7f680c9e40 con 0x56160c685930 2024-02-02T15:39:46.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.290+0000 7f7f9bd91b80 1 -- 172.21.15.1:0/2417928692 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7f680994c0 con 0x56160c685930 2024-02-02T15:39:46.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.290+0000 7f7f6f400700 1 -- 172.21.15.1:0/2417928692 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f680652c0 con 0x56160c685930 2024-02-02T15:39:46.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.290+0000 7f7f9bd91b80 10 client.18951 fetch_fsmap learned FSMap version 52 2024-02-02T15:39:46.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.290+0000 7f7f9bd91b80 1 -- 172.21.15.1:0/2417928692 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x56160c59cea0 con 0x56160c685930 2024-02-02T15:39:46.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.290+0000 7f7f6f400700 1 -- 172.21.15.1:0/2417928692 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7f680994c0 con 0x56160c685930 2024-02-02T15:39:46.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.290+0000 7f7f6f400700 1 -- 172.21.15.1:0/2417928692 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 52) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f7f68099720 con 0x56160c685930 2024-02-02T15:39:46.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.290+0000 7f7f9bd91b80 10 client.18951 fetch_fsmap finished waiting for FSMap version 52 2024-02-02T15:39:46.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.290+0000 7f7f9bd91b80 10 client.18951 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:39:46.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.290+0000 7f7f9bd91b80 20 client.18951 populate_metadata read hostname 'smithi001' 2024-02-02T15:39:46.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.290+0000 7f7f9bd91b80 1 --2- 172.21.15.1:0/2417928692 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56160c6f27a0 0x56160c6f4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:46.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.290+0000 7f7f9bd91b80 4 client.18951 mds_command: new command op to 24557 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:39:46.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.290+0000 7f7f9bd91b80 1 -- 172.21.15.1:0/2417928692 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56160c595f50 con 0x56160c6f27a0 2024-02-02T15:39:46.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.291+0000 7f7f85800700 1 --2- 172.21.15.1:0/2417928692 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56160c6f27a0 0x56160c6f4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:46.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.291+0000 7f7f85800700 1 --2- 172.21.15.1:0/2417928692 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56160c6f27a0 0x56160c6f4b90 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e76a25afb5fb72bd server_cookie=41756de5832f7e95 in_seq=0 out_seq=0 2024-02-02T15:39:46.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.291+0000 7f7f6f400700 10 client.18951 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:39:46.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.291+0000 7f7f6f400700 1 -- 172.21.15.1:0/2417928692 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56160c595f50 con 0x56160c6f27a0 2024-02-02T15:39:46.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.291+0000 7f7f6f400700 10 client.18951 handle_command_reply: tid=0 2024-02-02T15:39:46.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.291+0000 7f7f9bd91b80 1 client.18951 shutdown 2024-02-02T15:39:46.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.292+0000 7f7f9bd91b80 1 -- 172.21.15.1:0/2417928692 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56160c685930 msgr2=0x56160c695310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:46.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.292+0000 7f7f9bd91b80 1 --2- 172.21.15.1:0/2417928692 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56160c685930 0x56160c695310 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f7f680a1000 tx=0x7f7f68098ed0 comp rx=0 tx=0).stop 2024-02-02T15:39:46.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.292+0000 7f7f9bd91b80 1 -- 172.21.15.1:0/2417928692 shutdown_connections 2024-02-02T15:39:46.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.292+0000 7f7f9bd91b80 1 --2- 172.21.15.1:0/2417928692 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56160c6f27a0 0x56160c6f4b90 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:46.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.292+0000 7f7f6f400700 0 client.18951 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:39:46.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.292+0000 7f7f9bd91b80 1 --2- 172.21.15.1:0/2417928692 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56160c6907e0 0x56160c695850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:46.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.292+0000 7f7f9bd91b80 1 --2- 172.21.15.1:0/2417928692 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56160c685010 0x56160c696e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:46.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.292+0000 7f7f9bd91b80 1 --2- 172.21.15.1:0/2417928692 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56160c685930 0x56160c695310 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:46.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.292+0000 7f7f9bd91b80 1 -- 172.21.15.1:0/2417928692 >> 172.21.15.1:0/2417928692 conn(0x56160c5e4db0 msgr2=0x56160c6805c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:46.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.292+0000 7f7f9bd91b80 1 -- 172.21.15.1:0/2417928692 shutdown_connections 2024-02-02T15:39:46.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.292+0000 7f7f9bd91b80 1 -- 172.21.15.1:0/2417928692 wait complete. 2024-02-02T15:39:46.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.292+0000 7f7f9bd91b80 20 client.18951 trim_cache size 0 max 16384 2024-02-02T15:39:46.296 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:39:46.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.292+0000 7f7f88200700 1 -- 172.21.15.1:0/3185363911 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f7f600a0690 msgr2=0x7f7f600a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:46.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.293+0000 7f7f88200700 1 --2- 172.21.15.1:0/3185363911 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f7f600a0690 0x7f7f600a2b50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7f78007630 tx=0x7f7f780079f0 comp rx=0 tx=0).stop 2024-02-02T15:39:46.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.293+0000 7f7f88200700 1 -- 172.21.15.1:0/3185363911 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f80106110 msgr2=0x7f7f801995c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:46.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.293+0000 7f7f88200700 1 --2- 172.21.15.1:0/3185363911 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f80106110 0x7f7f801995c0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f7f6800e970 tx=0x7f7f6800ec80 comp rx=0 tx=0).stop 2024-02-02T15:39:46.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.293+0000 7f7f88200700 1 -- 172.21.15.1:0/3185363911 shutdown_connections 2024-02-02T15:39:46.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.293+0000 7f7f88200700 1 --2- 172.21.15.1:0/3185363911 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f7f600a0690 0x7f7f600a2b50 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:46.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.293+0000 7f7f88200700 1 --2- 172.21.15.1:0/3185363911 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7f8019da20 0x7f7f8019dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:46.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.293+0000 7f7f88200700 1 --2- 172.21.15.1:0/3185363911 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f80100ff0 0x7f7f80199080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:46.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.293+0000 7f7f88200700 1 --2- 172.21.15.1:0/3185363911 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7f80106110 0x7f7f801995c0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:46.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.293+0000 7f7f88200700 1 -- 172.21.15.1:0/3185363911 >> 172.21.15.1:0/3185363911 conn(0x7f7f800fc810 msgr2=0x7f7f8010cf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:46.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.293+0000 7f7f88200700 1 -- 172.21.15.1:0/3185363911 shutdown_connections 2024-02-02T15:39:46.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:46.293+0000 7f7f88200700 1 -- 172.21.15.1:0/3185363911 wait complete. 2024-02-02T15:39:47.273 INFO:tasks.workunit.client.0.smithi001.stdout: 1 694183 5.35 MB/sec execute 561 sec latency 2156.870 ms 2024-02-02T15:39:47.312 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-02T15:39:47.312 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-02T15:39:47.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.526+0000 7f54ce000700 1 -- 172.21.15.1:0/3579898392 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54c81019b0 msgr2=0x7f54c81059f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:47.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.526+0000 7f54ce000700 1 --2- 172.21.15.1:0/3579898392 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54c81019b0 0x7f54c81059f0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f54b0009990 tx=0x7f54b0009ca0 comp rx=0 tx=0).stop 2024-02-02T15:39:47.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.526+0000 7f54ce000700 1 -- 172.21.15.1:0/3579898392 shutdown_connections 2024-02-02T15:39:47.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.526+0000 7f54ce000700 1 --2- 172.21.15.1:0/3579898392 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f54c8106120 0x7f54c810dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:47.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.526+0000 7f54ce000700 1 --2- 172.21.15.1:0/3579898392 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54c8101000 0x7f54c81013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:47.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.526+0000 7f54ce000700 1 --2- 172.21.15.1:0/3579898392 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54c81019b0 0x7f54c81059f0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:47.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.526+0000 7f54ce000700 1 -- 172.21.15.1:0/3579898392 >> 172.21.15.1:0/3579898392 conn(0x7f54c80fc860 msgr2=0x7f54c80fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:47.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.526+0000 7f54ce000700 1 -- 172.21.15.1:0/3579898392 shutdown_connections 2024-02-02T15:39:47.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.526+0000 7f54ce000700 1 -- 172.21.15.1:0/3579898392 wait complete. 2024-02-02T15:39:47.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.527+0000 7f54ce000700 1 Processor -- start 2024-02-02T15:39:47.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.527+0000 7f54ce000700 1 -- start start 2024-02-02T15:39:47.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.527+0000 7f54ce000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54c8101000 0x7f54c81990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:47.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.527+0000 7f54ce000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f54c8106120 0x7f54c8199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:47.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.527+0000 7f54ce000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54c819da90 0x7f54c819df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:47.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.527+0000 7f54ce000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54c819e510 con 0x7f54c819da90 2024-02-02T15:39:47.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.527+0000 7f54ce000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54c819e680 con 0x7f54c8101000 2024-02-02T15:39:47.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.527+0000 7f54ce000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54c819e840 con 0x7f54c8106120 2024-02-02T15:39:47.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.528+0000 7f54c7400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54c8101000 0x7f54c81990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:47.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.528+0000 7f54c7400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54c8101000 0x7f54c81990f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:41540/0 (socket says 172.21.15.1:41540) 2024-02-02T15:39:47.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.528+0000 7f54c7400700 1 -- 172.21.15.1:0/1863583702 learned_addr learned my addr 172.21.15.1:0/1863583702 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:39:47.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.528+0000 7f54c7e00700 1 --2- 172.21.15.1:0/1863583702 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54c819da90 0x7f54c819df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:47.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.528+0000 7f54c6a00700 1 --2- 172.21.15.1:0/1863583702 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f54c8106120 0x7f54c8199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:47.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.528+0000 7f54c7400700 1 -- 172.21.15.1:0/1863583702 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f54c8106120 msgr2=0x7f54c8199630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:47.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.528+0000 7f54c7400700 1 --2- 172.21.15.1:0/1863583702 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f54c8106120 0x7f54c8199630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:47.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.528+0000 7f54c7400700 1 -- 172.21.15.1:0/1863583702 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54c819da90 msgr2=0x7f54c819df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:47.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.528+0000 7f54c7400700 1 --2- 172.21.15.1:0/1863583702 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54c819da90 0x7f54c819df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:47.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.528+0000 7f54c7400700 1 -- 172.21.15.1:0/1863583702 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54b00095f0 con 0x7f54c8101000 2024-02-02T15:39:47.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.528+0000 7f54c7400700 1 --2- 172.21.15.1:0/1863583702 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54c8101000 0x7f54c81990f0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f54b800e9a0 tx=0x7f54b800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:47.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.529+0000 7f54bfe00700 1 -- 172.21.15.1:0/1863583702 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b8009790 con 0x7f54c8101000 2024-02-02T15:39:47.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.529+0000 7f54bfe00700 1 -- 172.21.15.1:0/1863583702 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f54b8010430 con 0x7f54c8101000 2024-02-02T15:39:47.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.529+0000 7f54bfe00700 1 -- 172.21.15.1:0/1863583702 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b8018680 con 0x7f54c8101000 2024-02-02T15:39:47.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.529+0000 7f54ce000700 1 -- 172.21.15.1:0/1863583702 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54c81a5da0 con 0x7f54c8101000 2024-02-02T15:39:47.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.529+0000 7f54ce000700 1 -- 172.21.15.1:0/1863583702 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54c81a61d0 con 0x7f54c8101000 2024-02-02T15:39:47.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.533+0000 7f54bfe00700 1 -- 172.21.15.1:0/1863583702 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f54b80105a0 con 0x7f54c8101000 2024-02-02T15:39:47.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.534+0000 7f54e1b3bb80 1 Processor -- start 2024-02-02T15:39:47.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.535+0000 7f54e1b3bb80 1 -- start start 2024-02-02T15:39:47.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.535+0000 7f54bfe00700 1 --2- 172.21.15.1:0/1863583702 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f54a80a0710 0x7f54a80a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:47.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.535+0000 7f54bfe00700 1 -- 172.21.15.1:0/1863583702 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f54b8014070 con 0x7f54c8101000 2024-02-02T15:39:47.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.535+0000 7f54e1b3bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e385872090 0x55e385872470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:47.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.535+0000 7f54e1b3bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3858729b0 0x55e38587d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:47.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.535+0000 7f54e1b3bb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e38587d860 0x55e38587fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:47.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.535+0000 7f54e1b3bb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3857e6540 con 0x55e38587d860 2024-02-02T15:39:47.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.535+0000 7f54e1b3bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3857cab20 con 0x55e385872090 2024-02-02T15:39:47.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.535+0000 7f54e1b3bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3857c84e0 con 0x55e3858729b0 2024-02-02T15:39:47.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.535+0000 7f54c7e00700 1 --2- 172.21.15.1:0/1863583702 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f54a80a0710 0x7f54a80a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:47.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.535+0000 7f54c7400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e385872090 0x55e385872470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:47.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.536+0000 7f54c7400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e385872090 0x55e385872470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:41544/0 (socket says 172.21.15.1:41544) 2024-02-02T15:39:47.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.536+0000 7f54c7e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e38587d860 0x55e38587fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:47.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.536+0000 7f54c7400700 1 -- 172.21.15.1:0/3698011270 learned_addr learned my addr 172.21.15.1:0/3698011270 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:39:47.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.536+0000 7f54c6a00700 1 --2- 172.21.15.1:0/3698011270 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3858729b0 0x55e38587d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:47.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.536+0000 7f54c7400700 1 -- 172.21.15.1:0/3698011270 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3858729b0 msgr2=0x55e38587d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:47.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.536+0000 7f54c7400700 1 --2- 172.21.15.1:0/3698011270 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3858729b0 0x55e38587d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:47.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.536+0000 7f54c7400700 1 -- 172.21.15.1:0/3698011270 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e38587d860 msgr2=0x55e38587fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:47.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.536+0000 7f54c7400700 1 --2- 172.21.15.1:0/3698011270 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e38587d860 0x55e38587fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:47.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.536+0000 7f54c7400700 1 -- 172.21.15.1:0/3698011270 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e385645430 con 0x55e385872090 2024-02-02T15:39:47.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.536+0000 7f54c7e00700 1 --2- 172.21.15.1:0/3698011270 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e38587d860 0x55e38587fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:39:47.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.536+0000 7f54c7e00700 1 --2- 172.21.15.1:0/1863583702 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f54a80a0710 0x7f54a80a2bd0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f54c000f870 tx=0x7f54c000fc30 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:47.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.536+0000 7f54c7400700 1 --2- 172.21.15.1:0/3698011270 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e385872090 0x55e385872470 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f54b8099370 tx=0x7f54b8098dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1446b26b89bcec06 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:47.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.536+0000 7f54b7e00700 1 -- 172.21.15.1:0/3698011270 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b80acea0 con 0x55e385872090 2024-02-02T15:39:47.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.536+0000 7f54b7e00700 1 -- 172.21.15.1:0/3698011270 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f54b80993e0 con 0x55e385872090 2024-02-02T15:39:47.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.536+0000 7f54b7e00700 1 -- 172.21.15.1:0/3698011270 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b8082e40 con 0x55e385872090 2024-02-02T15:39:47.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.537+0000 7f54e1b3bb80 1 -- 172.21.15.1:0/3698011270 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e385872090 msgr2=0x55e385872470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:47.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.537+0000 7f54e1b3bb80 1 --2- 172.21.15.1:0/3698011270 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e385872090 0x55e385872470 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f54b8099370 tx=0x7f54b8098dc0 comp rx=0 tx=0).stop 2024-02-02T15:39:47.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.537+0000 7f54e1b3bb80 1 -- 172.21.15.1:0/3698011270 shutdown_connections 2024-02-02T15:39:47.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.537+0000 7f54e1b3bb80 1 --2- 172.21.15.1:0/3698011270 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e38587d860 0x55e38587fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:47.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.537+0000 7f54e1b3bb80 1 --2- 172.21.15.1:0/3698011270 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e385872090 0x55e385872470 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:47.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.537+0000 7f54e1b3bb80 1 --2- 172.21.15.1:0/3698011270 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3858729b0 0x55e38587d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:47.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.537+0000 7f54e1b3bb80 1 -- 172.21.15.1:0/3698011270 >> 172.21.15.1:0/3698011270 conn(0x55e3857d1e30 msgr2=0x55e38586e430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:47.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.537+0000 7f54e1b3bb80 1 -- 172.21.15.1:0/3698011270 shutdown_connections 2024-02-02T15:39:47.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.537+0000 7f54e1b3bb80 1 -- 172.21.15.1:0/3698011270 wait complete. 2024-02-02T15:39:47.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.538+0000 7f54e1b3bb80 1 -- start start 2024-02-02T15:39:47.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.538+0000 7f54e1b3bb80 1 Processor -- start 2024-02-02T15:39:47.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.538+0000 7f54e1b3bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e385872090 0x55e38587d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:47.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.538+0000 7f54e1b3bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3858729b0 0x55e38587a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:47.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.538+0000 7f54e1b3bb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e38587d860 0x55e38587aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:47.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.538+0000 7f54e1b3bb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3857c3dd0 con 0x55e38587d860 2024-02-02T15:39:47.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.538+0000 7f54e1b3bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e38575fe00 con 0x55e385872090 2024-02-02T15:39:47.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.538+0000 7f54e1b3bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e38575faa0 con 0x55e3858729b0 2024-02-02T15:39:47.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.538+0000 7f54e1b3bb80 10 client.0 _ll_register_callbacks cb 0x55e385761310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:39:47.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.539+0000 7f54c7400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e385872090 0x55e38587d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:47.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.539+0000 7f54c7400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e385872090 0x55e38587d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:41550/0 (socket says 172.21.15.1:41550) 2024-02-02T15:39:47.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.539+0000 7f54c7400700 1 -- 172.21.15.1:0/2323470248 learned_addr learned my addr 172.21.15.1:0/2323470248 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:39:47.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.539+0000 7f54c7e00700 1 --2- 172.21.15.1:0/2323470248 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e38587d860 0x55e38587aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:47.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.539+0000 7f54c6a00700 1 --2- 172.21.15.1:0/2323470248 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3858729b0 0x55e38587a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:47.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.539+0000 7f54c7400700 1 -- 172.21.15.1:0/2323470248 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3858729b0 msgr2=0x55e38587a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:47.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.539+0000 7f54c7400700 1 --2- 172.21.15.1:0/2323470248 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3858729b0 0x55e38587a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:47.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.539+0000 7f54c7400700 1 -- 172.21.15.1:0/2323470248 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e38587d860 msgr2=0x55e38587aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:47.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.539+0000 7f54c7400700 1 --2- 172.21.15.1:0/2323470248 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e38587d860 0x55e38587aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:47.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.539+0000 7f54c7400700 1 -- 172.21.15.1:0/2323470248 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54c81a61d0 con 0x55e385872090 2024-02-02T15:39:47.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.539+0000 7f54c7400700 1 --2- 172.21.15.1:0/2323470248 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e385872090 0x55e38587d200 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f54b80990e0 tx=0x7f54b80c9630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4aca9b16cbeeab51 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:47.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.539+0000 7f54b4c00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:39:47.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.539+0000 7f54b4c00700 1 -- 172.21.15.1:0/2323470248 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b80acea0 con 0x55e385872090 2024-02-02T15:39:47.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.539+0000 7f54b4c00700 1 -- 172.21.15.1:0/2323470248 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f54b809d210 con 0x55e385872090 2024-02-02T15:39:47.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.539+0000 7f54e1b3bb80 1 -- 172.21.15.1:0/2323470248 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e385866170 con 0x55e385872090 2024-02-02T15:39:47.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.540+0000 7f54b4c00700 1 -- 172.21.15.1:0/2323470248 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f54b8082de0 con 0x55e385872090 2024-02-02T15:39:47.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.540+0000 7f54e1b3bb80 10 client.29063 fetch_fsmap learned FSMap version 52 2024-02-02T15:39:47.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.540+0000 7f54e1b3bb80 1 -- 172.21.15.1:0/2323470248 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x55e3855f1be0 con 0x55e385872090 2024-02-02T15:39:47.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.540+0000 7f54b4c00700 1 -- 172.21.15.1:0/2323470248 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e385866170 con 0x55e385872090 2024-02-02T15:39:47.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.540+0000 7f54b4c00700 1 -- 172.21.15.1:0/2323470248 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 52) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f54b80a0fc0 con 0x55e385872090 2024-02-02T15:39:47.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.540+0000 7f54e1b3bb80 10 client.29063 fetch_fsmap finished waiting for FSMap version 52 2024-02-02T15:39:47.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.540+0000 7f54e1b3bb80 10 client.29063 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:39:47.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.540+0000 7f54e1b3bb80 20 client.29063 populate_metadata read hostname 'smithi001' 2024-02-02T15:39:47.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.540+0000 7f54e1b3bb80 1 --2- 172.21.15.1:0/2323470248 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55e38588cef0 0x55e38588f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:47.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.540+0000 7f54e1b3bb80 4 client.29063 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:39:47.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.540+0000 7f54e1b3bb80 1 -- 172.21.15.1:0/2323470248 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e3857bce50 con 0x55e38588cef0 2024-02-02T15:39:47.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.541+0000 7f54c6a00700 1 --2- 172.21.15.1:0/2323470248 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55e38588cef0 0x55e38588f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:47.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.541+0000 7f54c6a00700 1 --2- 172.21.15.1:0/2323470248 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55e38588cef0 0x55e38588f2e0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb98a18734ca89d2 server_cookie=b45433ebb6eb8c62 in_seq=0 out_seq=0 2024-02-02T15:39:47.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.541+0000 7f54b4c00700 10 client.29063 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:39:47.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.542+0000 7f54b4c00700 1 -- 172.21.15.1:0/2323470248 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55e3857bce50 con 0x55e38588cef0 2024-02-02T15:39:47.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.542+0000 7f54b4c00700 10 client.29063 handle_command_reply: tid=0 2024-02-02T15:39:47.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.542+0000 7f54e1b3bb80 1 client.29063 shutdown 2024-02-02T15:39:47.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.542+0000 7f54e1b3bb80 1 -- 172.21.15.1:0/2323470248 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e385872090 msgr2=0x55e38587d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:47.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.542+0000 7f54e1b3bb80 1 --2- 172.21.15.1:0/2323470248 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e385872090 0x55e38587d200 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f54b80990e0 tx=0x7f54b80c9630 comp rx=0 tx=0).stop 2024-02-02T15:39:47.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.542+0000 7f54e1b3bb80 1 -- 172.21.15.1:0/2323470248 shutdown_connections 2024-02-02T15:39:47.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.542+0000 7f54e1b3bb80 1 --2- 172.21.15.1:0/2323470248 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55e38588cef0 0x55e38588f2e0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:47.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.542+0000 7f54e1b3bb80 1 --2- 172.21.15.1:0/2323470248 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e38587d860 0x55e38587aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:47.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.542+0000 7f54e1b3bb80 1 --2- 172.21.15.1:0/2323470248 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e385872090 0x55e38587d200 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:47.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.542+0000 7f54e1b3bb80 1 --2- 172.21.15.1:0/2323470248 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3858729b0 0x55e38587a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:47.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.542+0000 7f54e1b3bb80 1 -- 172.21.15.1:0/2323470248 >> 172.21.15.1:0/2323470248 conn(0x55e3857d1e30 msgr2=0x55e38586e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:47.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.542+0000 7f54b4c00700 0 client.29063 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:39:47.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.542+0000 7f54e1b3bb80 1 -- 172.21.15.1:0/2323470248 shutdown_connections 2024-02-02T15:39:47.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.542+0000 7f54e1b3bb80 1 -- 172.21.15.1:0/2323470248 wait complete. 2024-02-02T15:39:47.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.542+0000 7f54e1b3bb80 20 client.29063 trim_cache size 0 max 16384 2024-02-02T15:39:47.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.560+0000 7f54e1b3bb80 1 Processor -- start 2024-02-02T15:39:47.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.560+0000 7f54e1b3bb80 1 -- start start 2024-02-02T15:39:47.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.560+0000 7f54e1b3bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e385872090 0x55e385884350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:47.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.561+0000 7f54e1b3bb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e3858729b0 0x55e385874b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:47.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.561+0000 7f54e1b3bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e38587d860 0x55e38587cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:47.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.561+0000 7f54e1b3bb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e385647720 con 0x55e3858729b0 2024-02-02T15:39:47.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.561+0000 7f54e1b3bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e38587b130 con 0x55e38587d860 2024-02-02T15:39:47.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.561+0000 7f54e1b3bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e385873010 con 0x55e385872090 2024-02-02T15:39:47.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.561+0000 7f54c6a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e3858729b0 0x55e385874b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:47.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.561+0000 7f54c6a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e3858729b0 0x55e385874b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55922/0 (socket says 172.21.15.1:55922) 2024-02-02T15:39:47.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.561+0000 7f54c6a00700 1 -- 172.21.15.1:0/179954394 learned_addr learned my addr 172.21.15.1:0/179954394 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:39:47.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.561+0000 7f54c7400700 1 --2- 172.21.15.1:0/179954394 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e385872090 0x55e385884350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:47.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.561+0000 7f54c7e00700 1 --2- 172.21.15.1:0/179954394 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e38587d860 0x55e38587cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:47.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.561+0000 7f54c6a00700 1 -- 172.21.15.1:0/179954394 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e385872090 msgr2=0x55e385884350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:47.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.561+0000 7f54c6a00700 1 --2- 172.21.15.1:0/179954394 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e385872090 0x55e385884350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:47.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.561+0000 7f54c6a00700 1 -- 172.21.15.1:0/179954394 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e38587d860 msgr2=0x55e38587cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:47.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.561+0000 7f54c6a00700 1 --2- 172.21.15.1:0/179954394 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e38587d860 0x55e38587cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:47.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.561+0000 7f54c6a00700 1 -- 172.21.15.1:0/179954394 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e38587fdd0 con 0x55e3858729b0 2024-02-02T15:39:47.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.562+0000 7f54c6a00700 1 --2- 172.21.15.1:0/179954394 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e3858729b0 0x55e385874b90 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f54b000b690 tx=0x7f54b0005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4bcd7e061121592 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:47.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.562+0000 7f54bd200700 1 -- 172.21.15.1:0/179954394 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b001d040 con 0x55e3858729b0 2024-02-02T15:39:47.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.562+0000 7f54bd200700 1 -- 172.21.15.1:0/179954394 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f54b001e040 con 0x55e3858729b0 2024-02-02T15:39:47.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.562+0000 7f54bd200700 1 -- 172.21.15.1:0/179954394 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b000ba50 con 0x55e3858729b0 2024-02-02T15:39:47.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.562+0000 7f54e1b3bb80 1 -- 172.21.15.1:0/179954394 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e3858729b0 msgr2=0x55e385874b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:47.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.562+0000 7f54e1b3bb80 1 --2- 172.21.15.1:0/179954394 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e3858729b0 0x55e385874b90 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f54b000b690 tx=0x7f54b0005fb0 comp rx=0 tx=0).stop 2024-02-02T15:39:47.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.562+0000 7f54e1b3bb80 1 -- 172.21.15.1:0/179954394 shutdown_connections 2024-02-02T15:39:47.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.562+0000 7f54e1b3bb80 1 --2- 172.21.15.1:0/179954394 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e38587d860 0x55e38587cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:47.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.562+0000 7f54e1b3bb80 1 --2- 172.21.15.1:0/179954394 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e385872090 0x55e385884350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:47.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.562+0000 7f54e1b3bb80 1 --2- 172.21.15.1:0/179954394 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e3858729b0 0x55e385874b90 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:47.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.562+0000 7f54e1b3bb80 1 -- 172.21.15.1:0/179954394 >> 172.21.15.1:0/179954394 conn(0x55e3857d1e30 msgr2=0x55e385875780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:47.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.562+0000 7f54e1b3bb80 1 -- 172.21.15.1:0/179954394 shutdown_connections 2024-02-02T15:39:47.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.563+0000 7f54e1b3bb80 1 -- 172.21.15.1:0/179954394 wait complete. 2024-02-02T15:39:47.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.563+0000 7f54e1b3bb80 1 -- start start 2024-02-02T15:39:47.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.563+0000 7f54e1b3bb80 1 Processor -- start 2024-02-02T15:39:47.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.563+0000 7f54e1b3bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e385872090 0x55e385883e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:47.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.564+0000 7f54e1b3bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3858729b0 0x55e385884340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:47.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.564+0000 7f54e1b3bb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e38587d860 0x55e385881320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:47.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.564+0000 7f54e1b3bb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e385881a30 con 0x55e38587d860 2024-02-02T15:39:47.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.564+0000 7f54e1b3bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e385881ba0 con 0x55e385872090 2024-02-02T15:39:47.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.564+0000 7f54e1b3bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e385881d60 con 0x55e3858729b0 2024-02-02T15:39:47.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.564+0000 7f54e1b3bb80 10 client.0 _ll_register_callbacks cb 0x55e385761310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:39:47.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.564+0000 7f54c7e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e38587d860 0x55e385881320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:47.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.564+0000 7f54c7e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e38587d860 0x55e385881320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:55932/0 (socket says 172.21.15.1:55932) 2024-02-02T15:39:47.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.564+0000 7f54c7e00700 1 -- 172.21.15.1:0/51300530 learned_addr learned my addr 172.21.15.1:0/51300530 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:39:47.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.564+0000 7f54c7400700 1 --2- 172.21.15.1:0/51300530 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e385872090 0x55e385883e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:47.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.564+0000 7f54c6a00700 1 --2- 172.21.15.1:0/51300530 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3858729b0 0x55e385884340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:47.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.564+0000 7f54c7400700 1 -- 172.21.15.1:0/51300530 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3858729b0 msgr2=0x55e385884340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:47.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.564+0000 7f54c7400700 1 --2- 172.21.15.1:0/51300530 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3858729b0 0x55e385884340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:47.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.564+0000 7f54c7400700 1 -- 172.21.15.1:0/51300530 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e38587d860 msgr2=0x55e385881320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:47.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.564+0000 7f54c7400700 1 --2- 172.21.15.1:0/51300530 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e38587d860 0x55e385881320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:47.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.564+0000 7f54c7400700 1 -- 172.21.15.1:0/51300530 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54c810e510 con 0x55e385872090 2024-02-02T15:39:47.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.564+0000 7f54c7e00700 1 --2- 172.21.15.1:0/51300530 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e38587d860 0x55e385881320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:39:47.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.565+0000 7f54c7400700 1 --2- 172.21.15.1:0/51300530 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e385872090 0x55e385883e00 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f54b80a13c0 tx=0x7f54b8095720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5ac65e0fbfb01b4 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:47.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.565+0000 7f54b4c00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:39:47.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.565+0000 7f54b4c00700 1 -- 172.21.15.1:0/51300530 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b80acea0 con 0x55e385872090 2024-02-02T15:39:47.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.565+0000 7f54b4c00700 1 -- 172.21.15.1:0/51300530 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f54b809d210 con 0x55e385872090 2024-02-02T15:39:47.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.565+0000 7f54e1b3bb80 1 -- 172.21.15.1:0/51300530 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f54b80993e0 con 0x55e385872090 2024-02-02T15:39:47.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.565+0000 7f54b4c00700 1 -- 172.21.15.1:0/51300530 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f54b8099ae0 con 0x55e385872090 2024-02-02T15:39:47.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.565+0000 7f54e1b3bb80 10 client.29069 fetch_fsmap learned FSMap version 52 2024-02-02T15:39:47.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.565+0000 7f54e1b3bb80 1 -- 172.21.15.1:0/51300530 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x55e385789f20 con 0x55e385872090 2024-02-02T15:39:47.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.565+0000 7f54b4c00700 1 -- 172.21.15.1:0/51300530 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f54b80993e0 con 0x55e385872090 2024-02-02T15:39:47.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.565+0000 7f54b4c00700 1 -- 172.21.15.1:0/51300530 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 52) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f54b80988d0 con 0x55e385872090 2024-02-02T15:39:47.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.566+0000 7f54e1b3bb80 10 client.29069 fetch_fsmap finished waiting for FSMap version 52 2024-02-02T15:39:47.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.566+0000 7f54e1b3bb80 10 client.29069 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:39:47.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.566+0000 7f54e1b3bb80 20 client.29069 populate_metadata read hostname 'smithi001' 2024-02-02T15:39:47.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.566+0000 7f54e1b3bb80 1 --2- 172.21.15.1:0/51300530 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55e3858e07f0 0x55e3858e2be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:47.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.566+0000 7f54e1b3bb80 4 client.29069 mds_command: new command op to 24557 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-02T15:39:47.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.566+0000 7f54e1b3bb80 1 -- 172.21.15.1:0/51300530 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55e385782fd0 con 0x55e3858e07f0 2024-02-02T15:39:47.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.566+0000 7f54c6a00700 1 --2- 172.21.15.1:0/51300530 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55e3858e07f0 0x55e3858e2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:47.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.566+0000 7f54c6a00700 1 --2- 172.21.15.1:0/51300530 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55e3858e07f0 0x55e3858e2be0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81d81a349de2c168 server_cookie=571731478aed3519 in_seq=0 out_seq=0 2024-02-02T15:39:47.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.566+0000 7f54b4c00700 10 client.29069 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:39:47.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.567+0000 7f54b4c00700 1 -- 172.21.15.1:0/51300530 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55e385782fd0 con 0x55e3858e07f0 2024-02-02T15:39:47.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.567+0000 7f54b4c00700 10 client.29069 handle_command_reply: tid=0 2024-02-02T15:39:47.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.568+0000 7f54e1b3bb80 1 client.29069 shutdown 2024-02-02T15:39:47.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.568+0000 7f54e1b3bb80 1 -- 172.21.15.1:0/51300530 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e385872090 msgr2=0x55e385883e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:47.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.568+0000 7f54e1b3bb80 1 --2- 172.21.15.1:0/51300530 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e385872090 0x55e385883e00 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f54b80a13c0 tx=0x7f54b8095720 comp rx=0 tx=0).stop 2024-02-02T15:39:47.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.568+0000 7f54e1b3bb80 1 -- 172.21.15.1:0/51300530 shutdown_connections 2024-02-02T15:39:47.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.568+0000 7f54e1b3bb80 1 --2- 172.21.15.1:0/51300530 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55e3858e07f0 0x55e3858e2be0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:47.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.568+0000 7f54e1b3bb80 1 --2- 172.21.15.1:0/51300530 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e38587d860 0x55e385881320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:47.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.568+0000 7f54e1b3bb80 1 --2- 172.21.15.1:0/51300530 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e385872090 0x55e385883e00 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:47.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.568+0000 7f54b4c00700 0 client.29069 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:39:47.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.568+0000 7f54e1b3bb80 1 --2- 172.21.15.1:0/51300530 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3858729b0 0x55e385884340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:47.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.568+0000 7f54e1b3bb80 1 -- 172.21.15.1:0/51300530 >> 172.21.15.1:0/51300530 conn(0x55e3857d1e30 msgr2=0x55e38586def0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:47.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.568+0000 7f54e1b3bb80 1 -- 172.21.15.1:0/51300530 shutdown_connections 2024-02-02T15:39:47.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.568+0000 7f54e1b3bb80 1 -- 172.21.15.1:0/51300530 wait complete. 2024-02-02T15:39:47.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.568+0000 7f54e1b3bb80 20 client.29069 trim_cache size 0 max 16384 2024-02-02T15:39:47.572 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:39:47.572 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2024-02-02T15:39:47.572 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "a6279fd4-6b03-475b-ad0c-0c5a4605ced6", 2024-02-02T15:39:47.572 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2024-02-02T15:39:47.572 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:39:47.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.569+0000 7f54ce000700 1 -- 172.21.15.1:0/1863583702 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f54a80a0710 msgr2=0x7f54a80a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:47.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.569+0000 7f54ce000700 1 --2- 172.21.15.1:0/1863583702 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f54a80a0710 0x7f54a80a2bd0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f54c000f870 tx=0x7f54c000fc30 comp rx=0 tx=0).stop 2024-02-02T15:39:47.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.569+0000 7f54ce000700 1 -- 172.21.15.1:0/1863583702 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54c8101000 msgr2=0x7f54c81990f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:47.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.569+0000 7f54ce000700 1 --2- 172.21.15.1:0/1863583702 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54c8101000 0x7f54c81990f0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f54b800e9a0 tx=0x7f54b800ecb0 comp rx=0 tx=0).stop 2024-02-02T15:39:47.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.569+0000 7f54ce000700 1 -- 172.21.15.1:0/1863583702 shutdown_connections 2024-02-02T15:39:47.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.569+0000 7f54ce000700 1 --2- 172.21.15.1:0/1863583702 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f54a80a0710 0x7f54a80a2bd0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:47.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.569+0000 7f54ce000700 1 --2- 172.21.15.1:0/1863583702 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f54c819da90 0x7f54c819df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:47.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.569+0000 7f54ce000700 1 --2- 172.21.15.1:0/1863583702 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f54c8101000 0x7f54c81990f0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:47.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.569+0000 7f54ce000700 1 --2- 172.21.15.1:0/1863583702 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f54c8106120 0x7f54c8199630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:47.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.569+0000 7f54ce000700 1 -- 172.21.15.1:0/1863583702 >> 172.21.15.1:0/1863583702 conn(0x7f54c80fc860 msgr2=0x7f54c80fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:47.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.569+0000 7f54ce000700 1 -- 172.21.15.1:0/1863583702 shutdown_connections 2024-02-02T15:39:47.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.569+0000 7f54ce000700 1 -- 172.21.15.1:0/1863583702 wait complete. 2024-02-02T15:39:47.594 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-02T15:39:47.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:39:47 smithi001 ceph-mon[35470]: pgmap v398: 129 pgs: 129 active+clean; 599 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 171 op/s 2024-02-02T15:39:47.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:39:47 smithi071 ceph-mon[44707]: pgmap v398: 129 pgs: 129 active+clean; 599 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 171 op/s 2024-02-02T15:39:47.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.886+0000 7ff498a00700 1 -- 172.21.15.1:0/866310424 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4901096e0 msgr2=0x7ff490109ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:47.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.886+0000 7ff498a00700 1 --2- 172.21.15.1:0/866310424 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4901096e0 0x7ff490109ac0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7ff48800b0a0 tx=0x7ff48800b3b0 comp rx=0 tx=0).stop 2024-02-02T15:39:47.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.888+0000 7ff498a00700 1 -- 172.21.15.1:0/866310424 shutdown_connections 2024-02-02T15:39:47.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.888+0000 7ff498a00700 1 --2- 172.21.15.1:0/866310424 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff490112c90 0x7ff490115080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:47.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.888+0000 7ff498a00700 1 --2- 172.21.15.1:0/866310424 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff4901096e0 0x7ff490109ac0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:47.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.888+0000 7ff498a00700 1 --2- 172.21.15.1:0/866310424 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff49010a000 0x7ff490112750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:47.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.888+0000 7ff498a00700 1 -- 172.21.15.1:0/866310424 >> 172.21.15.1:0/866310424 conn(0x7ff49006d130 msgr2=0x7ff49006d540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:47.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.889+0000 7ff498a00700 1 -- 172.21.15.1:0/866310424 shutdown_connections 2024-02-02T15:39:47.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.889+0000 7ff498a00700 1 -- 172.21.15.1:0/866310424 wait complete. 2024-02-02T15:39:47.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.891+0000 7ff498a00700 1 Processor -- start 2024-02-02T15:39:47.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.891+0000 7ff498a00700 1 -- start start 2024-02-02T15:39:47.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.891+0000 7ff498a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4901096e0 0x7ff49019d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:47.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.891+0000 7ff498a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff49010a000 0x7ff49019d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:47.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.891+0000 7ff498a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff490112c90 0x7ff4901a1d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:47.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.891+0000 7ff498a00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4901a23c0 con 0x7ff49010a000 2024-02-02T15:39:47.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.891+0000 7ff498a00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4901a2530 con 0x7ff490112c90 2024-02-02T15:39:47.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.892+0000 7ff498a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4901a26f0 con 0x7ff4901096e0 2024-02-02T15:39:47.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.892+0000 7ff496000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4901096e0 0x7ff49019d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:47.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.892+0000 7ff496000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4901096e0 0x7ff49019d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:34290/0 (socket says 172.21.15.1:34290) 2024-02-02T15:39:47.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.892+0000 7ff496000700 1 -- 172.21.15.1:0/3299018502 learned_addr learned my addr 172.21.15.1:0/3299018502 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:39:47.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.892+0000 7ff495600700 1 --2- 172.21.15.1:0/3299018502 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff49010a000 0x7ff49019d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:47.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.892+0000 7ff496a00700 1 --2- 172.21.15.1:0/3299018502 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff490112c90 0x7ff4901a1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:47.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.892+0000 7ff496000700 1 -- 172.21.15.1:0/3299018502 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff490112c90 msgr2=0x7ff4901a1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:47.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.892+0000 7ff496000700 1 --2- 172.21.15.1:0/3299018502 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff490112c90 0x7ff4901a1d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:47.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.893+0000 7ff496000700 1 -- 172.21.15.1:0/3299018502 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff49010a000 msgr2=0x7ff49019d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:47.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.893+0000 7ff496000700 1 --2- 172.21.15.1:0/3299018502 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff49010a000 0x7ff49019d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:47.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.893+0000 7ff496000700 1 -- 172.21.15.1:0/3299018502 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff488009d00 con 0x7ff4901096e0 2024-02-02T15:39:47.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.893+0000 7ff496a00700 1 --2- 172.21.15.1:0/3299018502 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff490112c90 0x7ff4901a1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:39:47.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.893+0000 7ff496000700 1 --2- 172.21.15.1:0/3299018502 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4901096e0 0x7ff49019d3e0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ff488015000 tx=0x7ff488003c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:47.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.894+0000 7ff48ea00700 1 -- 172.21.15.1:0/3299018502 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff48800e030 con 0x7ff4901096e0 2024-02-02T15:39:47.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.894+0000 7ff48ea00700 1 -- 172.21.15.1:0/3299018502 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff488007a90 con 0x7ff4901096e0 2024-02-02T15:39:47.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.894+0000 7ff48ea00700 1 -- 172.21.15.1:0/3299018502 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff48801a950 con 0x7ff4901096e0 2024-02-02T15:39:47.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.894+0000 7ff498a00700 1 -- 172.21.15.1:0/3299018502 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4901a2970 con 0x7ff4901096e0 2024-02-02T15:39:47.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.894+0000 7ff498a00700 1 -- 172.21.15.1:0/3299018502 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4901b2a20 con 0x7ff4901096e0 2024-02-02T15:39:47.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.899+0000 7ff48ea00700 1 -- 172.21.15.1:0/3299018502 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7ff488005800 con 0x7ff4901096e0 2024-02-02T15:39:47.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.900+0000 7ff48ea00700 1 --2- 172.21.15.1:0/3299018502 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff4700a0710 0x7ff4700a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:47.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.901+0000 7ff495600700 1 --2- 172.21.15.1:0/3299018502 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff4700a0710 0x7ff4700a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:47.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.901+0000 7ff4ac454b80 1 Processor -- start 2024-02-02T15:39:47.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.900+0000 7ff48ea00700 1 -- 172.21.15.1:0/3299018502 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 9453+0+0 (secure 0 0 0) 0x7ff48800f020 con 0x7ff4901096e0 2024-02-02T15:39:47.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.902+0000 7ff495600700 1 --2- 172.21.15.1:0/3299018502 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff4700a0710 0x7ff4700a2bd0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff49019e9f0 tx=0x7ff48000f040 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:47.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.902+0000 7ff4ac454b80 1 -- start start 2024-02-02T15:39:47.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.903+0000 7ff4ac454b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561319909010 0x5613199093f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:47.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.903+0000 7ff4ac454b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561319909930 0x5613199142a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:47.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.903+0000 7ff4ac454b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5613199147e0 0x561319916bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:47.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.903+0000 7ff4ac454b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56131987d4c0 con 0x561319909930 2024-02-02T15:39:47.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.903+0000 7ff4ac454b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x561319861aa0 con 0x561319909010 2024-02-02T15:39:47.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.903+0000 7ff4ac454b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56131985f460 con 0x5613199147e0 2024-02-02T15:39:47.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.904+0000 7ff496000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561319909010 0x5613199093f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:47.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.904+0000 7ff496000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561319909010 0x5613199093f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:41586/0 (socket says 172.21.15.1:41586) 2024-02-02T15:39:47.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.904+0000 7ff496000700 1 -- 172.21.15.1:0/3920701179 learned_addr learned my addr 172.21.15.1:0/3920701179 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:39:47.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.904+0000 7ff496a00700 1 --2- 172.21.15.1:0/3920701179 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5613199147e0 0x561319916bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:47.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.904+0000 7ff495600700 1 --2- 172.21.15.1:0/3920701179 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561319909930 0x5613199142a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:47.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.904+0000 7ff496000700 1 -- 172.21.15.1:0/3920701179 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5613199147e0 msgr2=0x561319916bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:47.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.904+0000 7ff496000700 1 --2- 172.21.15.1:0/3920701179 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5613199147e0 0x561319916bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:47.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.904+0000 7ff496000700 1 -- 172.21.15.1:0/3920701179 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561319909930 msgr2=0x5613199142a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:47.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.904+0000 7ff496000700 1 --2- 172.21.15.1:0/3920701179 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561319909930 0x5613199142a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:47.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.904+0000 7ff496000700 1 -- 172.21.15.1:0/3920701179 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5613196dc3b0 con 0x561319909010 2024-02-02T15:39:47.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.905+0000 7ff496000700 1 --2- 172.21.15.1:0/3920701179 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561319909010 0x5613199093f0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7ff48809a050 tx=0x7ff488000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e784faba78c1a907 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:47.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.906+0000 7ff487000700 1 -- 172.21.15.1:0/3920701179 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff48809e180 con 0x561319909010 2024-02-02T15:39:47.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.906+0000 7ff487000700 1 -- 172.21.15.1:0/3920701179 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff4880a2330 con 0x561319909010 2024-02-02T15:39:47.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.906+0000 7ff487000700 1 -- 172.21.15.1:0/3920701179 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4880195b0 con 0x561319909010 2024-02-02T15:39:47.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.907+0000 7ff4ac454b80 1 -- 172.21.15.1:0/3920701179 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561319909010 msgr2=0x5613199093f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:47.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.907+0000 7ff4ac454b80 1 --2- 172.21.15.1:0/3920701179 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561319909010 0x5613199093f0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7ff48809a050 tx=0x7ff488000f40 comp rx=0 tx=0).stop 2024-02-02T15:39:47.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.907+0000 7ff4ac454b80 1 -- 172.21.15.1:0/3920701179 shutdown_connections 2024-02-02T15:39:47.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.907+0000 7ff4ac454b80 1 --2- 172.21.15.1:0/3920701179 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5613199147e0 0x561319916bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:47.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.907+0000 7ff4ac454b80 1 --2- 172.21.15.1:0/3920701179 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561319909010 0x5613199093f0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:47.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.907+0000 7ff4ac454b80 1 --2- 172.21.15.1:0/3920701179 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561319909930 0x5613199142a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:47.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.907+0000 7ff4ac454b80 1 -- 172.21.15.1:0/3920701179 >> 172.21.15.1:0/3920701179 conn(0x561319868db0 msgr2=0x5613199053b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:47.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.908+0000 7ff4ac454b80 1 -- 172.21.15.1:0/3920701179 shutdown_connections 2024-02-02T15:39:47.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.908+0000 7ff4ac454b80 1 -- 172.21.15.1:0/3920701179 wait complete. 2024-02-02T15:39:47.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.908+0000 7ff4ac454b80 1 -- start start 2024-02-02T15:39:47.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.909+0000 7ff4ac454b80 1 Processor -- start 2024-02-02T15:39:47.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.909+0000 7ff4ac454b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561319909010 0x561319914bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:47.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.910+0000 7ff4ac454b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561319909930 0x561319915100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:47.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.910+0000 7ff4ac454b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5613199147e0 0x56131991fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:47.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.910+0000 7ff4ac454b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56131985ad50 con 0x561319909010 2024-02-02T15:39:47.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.910+0000 7ff4ac454b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613197f6d80 con 0x561319909930 2024-02-02T15:39:47.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.910+0000 7ff4ac454b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613197f6a20 con 0x5613199147e0 2024-02-02T15:39:47.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.910+0000 7ff4ac454b80 10 client.0 _ll_register_callbacks cb 0x5613197f8290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:39:47.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.910+0000 7ff496a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5613199147e0 0x56131991fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:47.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.910+0000 7ff495600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561319909930 0x561319915100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:47.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.910+0000 7ff495600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561319909930 0x561319915100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:41594/0 (socket says 172.21.15.1:41594) 2024-02-02T15:39:47.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.910+0000 7ff496a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5613199147e0 0x56131991fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:34312/0 (socket says 172.21.15.1:34312) 2024-02-02T15:39:47.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.910+0000 7ff495600700 1 -- 172.21.15.1:0/1185559211 learned_addr learned my addr 172.21.15.1:0/1185559211 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:39:47.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.911+0000 7ff496000700 1 --2- 172.21.15.1:0/1185559211 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561319909010 0x561319914bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:47.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.911+0000 7ff495600700 1 -- 172.21.15.1:0/1185559211 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5613199147e0 msgr2=0x56131991fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:47.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.911+0000 7ff495600700 1 --2- 172.21.15.1:0/1185559211 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5613199147e0 0x56131991fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:47.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.911+0000 7ff495600700 1 -- 172.21.15.1:0/1185559211 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561319909010 msgr2=0x561319914bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:47.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.911+0000 7ff495600700 1 --2- 172.21.15.1:0/1185559211 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561319909010 0x561319914bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:47.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.911+0000 7ff495600700 1 -- 172.21.15.1:0/1185559211 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4901b2a20 con 0x561319909930 2024-02-02T15:39:47.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.911+0000 7ff496a00700 1 --2- 172.21.15.1:0/1185559211 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5613199147e0 0x56131991fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-02-02T15:39:47.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.912+0000 7ff495600700 1 --2- 172.21.15.1:0/1185559211 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561319909930 0x561319915100 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7ff4800045c0 tx=0x7ff480004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60bacbcb8c49b6ca server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:47.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.912+0000 7ff47f400700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:39:47.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.912+0000 7ff47f400700 1 -- 172.21.15.1:0/1185559211 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff480012040 con 0x561319909930 2024-02-02T15:39:47.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.912+0000 7ff47f400700 1 -- 172.21.15.1:0/1185559211 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9453+0+0 (secure 0 0 0) 0x7ff480018820 con 0x561319909930 2024-02-02T15:39:47.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.912+0000 7ff4ac454b80 1 -- 172.21.15.1:0/1185559211 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5613198fd0f0 con 0x561319909930 2024-02-02T15:39:47.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.913+0000 7ff47f400700 1 -- 172.21.15.1:0/1185559211 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff4800189e0 con 0x561319909930 2024-02-02T15:39:47.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.913+0000 7ff4ac454b80 10 client.29081 fetch_fsmap learned FSMap version 52 2024-02-02T15:39:47.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.913+0000 7ff4ac454b80 1 -- 172.21.15.1:0/1185559211 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x561319688b60 con 0x561319909930 2024-02-02T15:39:47.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.914+0000 7ff47f400700 1 -- 172.21.15.1:0/1185559211 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5613198fd0f0 con 0x561319909930 2024-02-02T15:39:47.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.914+0000 7ff47f400700 1 -- 172.21.15.1:0/1185559211 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 52) v1 ==== 7486+0+0 (secure 0 0 0) 0x7ff480024b20 con 0x561319909930 2024-02-02T15:39:47.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.915+0000 7ff4ac454b80 10 client.29081 fetch_fsmap finished waiting for FSMap version 52 2024-02-02T15:39:47.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.915+0000 7ff4ac454b80 10 client.29081 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:39:47.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.915+0000 7ff4ac454b80 20 client.29081 populate_metadata read hostname 'smithi001' 2024-02-02T15:39:47.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.915+0000 7ff4ac454b80 1 --2- 172.21.15.1:0/1185559211 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x561319923f80 0x5613199263a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:47.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.915+0000 7ff4ac454b80 4 client.29081 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:39:47.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.915+0000 7ff4ac454b80 1 -- 172.21.15.1:0/1185559211 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561319853dd0 con 0x561319923f80 2024-02-02T15:39:47.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.915+0000 7ff496000700 1 --2- 172.21.15.1:0/1185559211 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x561319923f80 0x5613199263a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:47.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.916+0000 7ff496000700 1 --2- 172.21.15.1:0/1185559211 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x561319923f80 0x5613199263a0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46618190bd7db40f server_cookie=751f0704bb9156d4 in_seq=0 out_seq=0 2024-02-02T15:39:47.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.916+0000 7ff47f400700 10 client.29081 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:39:47.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.917+0000 7ff47f400700 1 -- 172.21.15.1:0/1185559211 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x561319853dd0 con 0x561319923f80 2024-02-02T15:39:47.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.917+0000 7ff47f400700 10 client.29081 handle_command_reply: tid=0 2024-02-02T15:39:47.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.917+0000 7ff4ac454b80 1 client.29081 shutdown 2024-02-02T15:39:47.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.918+0000 7ff4ac454b80 1 -- 172.21.15.1:0/1185559211 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561319909930 msgr2=0x561319915100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:47.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.918+0000 7ff4ac454b80 1 --2- 172.21.15.1:0/1185559211 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561319909930 0x561319915100 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7ff4800045c0 tx=0x7ff480004750 comp rx=0 tx=0).stop 2024-02-02T15:39:47.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.918+0000 7ff4ac454b80 1 -- 172.21.15.1:0/1185559211 shutdown_connections 2024-02-02T15:39:47.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.918+0000 7ff4ac454b80 1 --2- 172.21.15.1:0/1185559211 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x561319923f80 0x5613199263a0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:47.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.918+0000 7ff4ac454b80 1 --2- 172.21.15.1:0/1185559211 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5613199147e0 0x56131991fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:47.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.918+0000 7ff4ac454b80 1 --2- 172.21.15.1:0/1185559211 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561319909010 0x561319914bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:47.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.918+0000 7ff4ac454b80 1 --2- 172.21.15.1:0/1185559211 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561319909930 0x561319915100 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:47.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.918+0000 7ff4ac454b80 1 -- 172.21.15.1:0/1185559211 >> 172.21.15.1:0/1185559211 conn(0x561319868db0 msgr2=0x561319904fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:47.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.918+0000 7ff47f400700 0 client.29081 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:39:47.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.919+0000 7ff4ac454b80 1 -- 172.21.15.1:0/1185559211 shutdown_connections 2024-02-02T15:39:47.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.919+0000 7ff4ac454b80 1 -- 172.21.15.1:0/1185559211 wait complete. 2024-02-02T15:39:47.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:39:47 smithi142 ceph-mon[45728]: pgmap v398: 129 pgs: 129 active+clean; 599 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 171 op/s 2024-02-02T15:39:47.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.919+0000 7ff4ac454b80 20 client.29081 trim_cache size 0 max 16384 2024-02-02T15:39:47.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.938+0000 7ff4ac454b80 1 Processor -- start 2024-02-02T15:39:47.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.938+0000 7ff4ac454b80 1 -- start start 2024-02-02T15:39:47.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.939+0000 7ff4ac454b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561319909010 0x561319919bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:47.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.939+0000 7ff4ac454b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561319909930 0x561319914bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:47.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.939+0000 7ff4ac454b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5613199147e0 0x561319915100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:47.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.939+0000 7ff4ac454b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613196de6a0 con 0x561319909930 2024-02-02T15:39:47.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.939+0000 7ff4ac454b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x561319921650 con 0x5613199147e0 2024-02-02T15:39:47.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.939+0000 7ff4ac454b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613199157d0 con 0x561319909010 2024-02-02T15:39:47.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.939+0000 7ff496a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5613199147e0 0x561319915100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:47.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.939+0000 7ff496a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5613199147e0 0x561319915100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:41596/0 (socket says 172.21.15.1:41596) 2024-02-02T15:39:47.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.939+0000 7ff496a00700 1 -- 172.21.15.1:0/2357365258 learned_addr learned my addr 172.21.15.1:0/2357365258 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:39:47.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.939+0000 7ff495600700 1 --2- 172.21.15.1:0/2357365258 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561319909930 0x561319914bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:47.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.939+0000 7ff496000700 1 --2- 172.21.15.1:0/2357365258 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561319909010 0x561319919bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:47.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.940+0000 7ff496a00700 1 -- 172.21.15.1:0/2357365258 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561319909010 msgr2=0x561319919bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:47.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.940+0000 7ff496a00700 1 --2- 172.21.15.1:0/2357365258 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561319909010 0x561319919bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:47.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.940+0000 7ff496a00700 1 -- 172.21.15.1:0/2357365258 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561319909930 msgr2=0x561319914bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:47.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.940+0000 7ff496a00700 1 --2- 172.21.15.1:0/2357365258 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561319909930 0x561319914bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:47.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.940+0000 7ff496a00700 1 -- 172.21.15.1:0/2357365258 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5613199010d0 con 0x5613199147e0 2024-02-02T15:39:47.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.940+0000 7ff496000700 1 --2- 172.21.15.1:0/2357365258 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561319909010 0x561319919bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:39:47.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.940+0000 7ff496a00700 1 --2- 172.21.15.1:0/2357365258 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5613199147e0 0x561319915100 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7ff47800dd00 tx=0x7ff4780126d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=932af31104ce586a server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:47.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.940+0000 7ff487a00700 1 -- 172.21.15.1:0/2357365258 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff47801b640 con 0x5613199147e0 2024-02-02T15:39:47.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.940+0000 7ff487a00700 1 -- 172.21.15.1:0/2357365258 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff47800de50 con 0x5613199147e0 2024-02-02T15:39:47.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.940+0000 7ff487a00700 1 -- 172.21.15.1:0/2357365258 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff47801a940 con 0x5613199147e0 2024-02-02T15:39:47.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.941+0000 7ff4ac454b80 1 -- 172.21.15.1:0/2357365258 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5613199147e0 msgr2=0x561319915100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:47.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.941+0000 7ff4ac454b80 1 --2- 172.21.15.1:0/2357365258 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5613199147e0 0x561319915100 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7ff47800dd00 tx=0x7ff4780126d0 comp rx=0 tx=0).stop 2024-02-02T15:39:47.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.941+0000 7ff4ac454b80 1 -- 172.21.15.1:0/2357365258 shutdown_connections 2024-02-02T15:39:47.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.941+0000 7ff4ac454b80 1 --2- 172.21.15.1:0/2357365258 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5613199147e0 0x561319915100 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:47.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.941+0000 7ff4ac454b80 1 --2- 172.21.15.1:0/2357365258 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561319909010 0x561319919bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:47.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.941+0000 7ff4ac454b80 1 --2- 172.21.15.1:0/2357365258 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561319909930 0x561319914bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:47.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.941+0000 7ff4ac454b80 1 -- 172.21.15.1:0/2357365258 >> 172.21.15.1:0/2357365258 conn(0x561319868db0 msgr2=0x56131990ec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:47.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.941+0000 7ff4ac454b80 1 -- 172.21.15.1:0/2357365258 shutdown_connections 2024-02-02T15:39:47.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.941+0000 7ff4ac454b80 1 -- 172.21.15.1:0/2357365258 wait complete. 2024-02-02T15:39:47.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.941+0000 7ff4ac454b80 1 -- start start 2024-02-02T15:39:47.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.942+0000 7ff4ac454b80 1 Processor -- start 2024-02-02T15:39:47.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.942+0000 7ff4ac454b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561319909010 0x561319917110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:47.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.942+0000 7ff4ac454b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561319909930 0x561319917650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:47.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.942+0000 7ff4ac454b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5613199147e0 0x561319917b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:47.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.942+0000 7ff4ac454b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x561319974aa0 con 0x561319909930 2024-02-02T15:39:47.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.942+0000 7ff4ac454b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x561319974be0 con 0x5613199147e0 2024-02-02T15:39:47.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.942+0000 7ff4ac454b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561319974da0 con 0x561319909010 2024-02-02T15:39:47.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.942+0000 7ff4ac454b80 10 client.0 _ll_register_callbacks cb 0x5613197f8290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:39:47.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.943+0000 7ff496000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561319909010 0x561319917110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:47.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.943+0000 7ff496000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561319909010 0x561319917110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:34326/0 (socket says 172.21.15.1:34326) 2024-02-02T15:39:47.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.943+0000 7ff495600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561319909930 0x561319917650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:47.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.943+0000 7ff496000700 1 -- 172.21.15.1:0/3750396393 learned_addr learned my addr 172.21.15.1:0/3750396393 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:39:47.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.943+0000 7ff496a00700 1 --2- 172.21.15.1:0/3750396393 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5613199147e0 0x561319917b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:47.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.943+0000 7ff496000700 1 -- 172.21.15.1:0/3750396393 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5613199147e0 msgr2=0x561319917b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:47.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.943+0000 7ff496000700 1 --2- 172.21.15.1:0/3750396393 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5613199147e0 0x561319917b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:47.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.943+0000 7ff496000700 1 -- 172.21.15.1:0/3750396393 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561319909930 msgr2=0x561319917650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:39:47.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.943+0000 7ff496000700 1 --2- 172.21.15.1:0/3750396393 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561319909930 0x561319917650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:47.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.943+0000 7ff496000700 1 -- 172.21.15.1:0/3750396393 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5613196dc3b0 con 0x561319909010 2024-02-02T15:39:47.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.943+0000 7ff495600700 1 --2- 172.21.15.1:0/3750396393 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561319909930 0x561319917650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:39:47.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.943+0000 7ff496000700 1 --2- 172.21.15.1:0/3750396393 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561319909010 0x561319917110 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ff48800b070 tx=0x7ff48800b3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=772d88fb40440db1 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:39:47.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.943+0000 7ff47f400700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:39:47.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.943+0000 7ff47f400700 1 -- 172.21.15.1:0/3750396393 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4880a00b0 con 0x561319909010 2024-02-02T15:39:47.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.943+0000 7ff47f400700 1 -- 172.21.15.1:0/3750396393 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(101..101 src has 1..101) v4 ==== 9453+0+0 (secure 0 0 0) 0x7ff4880a0210 con 0x561319909010 2024-02-02T15:39:47.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.943+0000 7ff4ac454b80 1 -- 172.21.15.1:0/3750396393 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff4880a2330 con 0x561319909010 2024-02-02T15:39:47.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.944+0000 7ff47f400700 1 -- 172.21.15.1:0/3750396393 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff4880b8250 con 0x561319909010 2024-02-02T15:39:47.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.944+0000 7ff4ac454b80 10 client.29083 fetch_fsmap learned FSMap version 52 2024-02-02T15:39:47.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.944+0000 7ff4ac454b80 1 -- 172.21.15.1:0/3750396393 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x7ff47800d930 con 0x561319909010 2024-02-02T15:39:47.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.944+0000 7ff47f400700 1 -- 172.21.15.1:0/3750396393 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff4880a2330 con 0x561319909010 2024-02-02T15:39:47.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.944+0000 7ff47f400700 1 -- 172.21.15.1:0/3750396393 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 52) v1 ==== 7486+0+0 (secure 0 0 0) 0x7ff488099840 con 0x561319909010 2024-02-02T15:39:47.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.944+0000 7ff4ac454b80 10 client.29083 fetch_fsmap finished waiting for FSMap version 52 2024-02-02T15:39:47.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.944+0000 7ff4ac454b80 10 client.29083 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:39:47.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.944+0000 7ff4ac454b80 20 client.29083 populate_metadata read hostname 'smithi001' 2024-02-02T15:39:47.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.944+0000 7ff4ac454b80 1 --2- 172.21.15.1:0/3750396393 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x561319976000 0x5613199783f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:39:47.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.944+0000 7ff4ac454b80 4 client.29083 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2024-02-02T15:39:47.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.944+0000 7ff4ac454b80 1 -- 172.21.15.1:0/3750396393 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561319819f50 con 0x561319976000 2024-02-02T15:39:47.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.945+0000 7ff495600700 1 --2- 172.21.15.1:0/3750396393 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x561319976000 0x5613199783f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:39:47.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.945+0000 7ff495600700 1 --2- 172.21.15.1:0/3750396393 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x561319976000 0x5613199783f0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=468d702cc54a8154 server_cookie=cee1d51f5d5a1e4e in_seq=0 out_seq=0 2024-02-02T15:39:47.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.945+0000 7ff47f400700 10 client.29083 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:39:47.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.945+0000 7ff47f400700 1 -- 172.21.15.1:0/3750396393 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x561319819f50 con 0x561319976000 2024-02-02T15:39:47.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.945+0000 7ff47f400700 10 client.29083 handle_command_reply: tid=0 2024-02-02T15:39:47.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.945+0000 7ff4ac454b80 1 client.29083 shutdown 2024-02-02T15:39:47.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.945+0000 7ff4ac454b80 1 -- 172.21.15.1:0/3750396393 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561319909010 msgr2=0x561319917110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:47.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.945+0000 7ff4ac454b80 1 --2- 172.21.15.1:0/3750396393 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561319909010 0x561319917110 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ff48800b070 tx=0x7ff48800b3b0 comp rx=0 tx=0).stop 2024-02-02T15:39:47.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.946+0000 7ff4ac454b80 1 -- 172.21.15.1:0/3750396393 shutdown_connections 2024-02-02T15:39:47.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.946+0000 7ff4ac454b80 1 --2- 172.21.15.1:0/3750396393 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x561319976000 0x5613199783f0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:47.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.946+0000 7ff4ac454b80 1 --2- 172.21.15.1:0/3750396393 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5613199147e0 0x561319917b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:47.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.946+0000 7ff4ac454b80 1 --2- 172.21.15.1:0/3750396393 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561319909010 0x561319917110 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:47.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.946+0000 7ff4ac454b80 1 --2- 172.21.15.1:0/3750396393 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561319909930 0x561319917650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:47.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.946+0000 7ff47f400700 0 client.29083 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:39:47.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.946+0000 7ff4ac454b80 1 -- 172.21.15.1:0/3750396393 >> 172.21.15.1:0/3750396393 conn(0x561319868db0 msgr2=0x561319914bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:47.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.946+0000 7ff4ac454b80 1 -- 172.21.15.1:0/3750396393 shutdown_connections 2024-02-02T15:39:47.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.946+0000 7ff4ac454b80 1 -- 172.21.15.1:0/3750396393 wait complete. 2024-02-02T15:39:47.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.946+0000 7ff4ac454b80 20 client.29083 trim_cache size 0 max 16384 2024-02-02T15:39:47.950 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:39:47.950 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-02T15:39:47.950 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2024-02-02T15:39:47.951 INFO:teuthology.orchestra.run.smithi001.stdout: "a6279fd4-6b03-475b-ad0c-0c5a4605ced6": { 2024-02-02T15:39:47.951 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2024-02-02T15:39:47.951 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "a6279fd4-6b03-475b-ad0c-0c5a4605ced6", 2024-02-02T15:39:47.951 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,force" 2024-02-02T15:39:47.951 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:39:47.951 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:39:47.951 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:39:47.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.946+0000 7ff498a00700 1 -- 172.21.15.1:0/3299018502 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff4700a0710 msgr2=0x7ff4700a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:47.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.946+0000 7ff498a00700 1 --2- 172.21.15.1:0/3299018502 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff4700a0710 0x7ff4700a2bd0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff49019e9f0 tx=0x7ff48000f040 comp rx=0 tx=0).stop 2024-02-02T15:39:47.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.947+0000 7ff498a00700 1 -- 172.21.15.1:0/3299018502 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4901096e0 msgr2=0x7ff49019d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:39:47.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.947+0000 7ff498a00700 1 --2- 172.21.15.1:0/3299018502 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4901096e0 0x7ff49019d3e0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ff488015000 tx=0x7ff488003c30 comp rx=0 tx=0).stop 2024-02-02T15:39:47.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.947+0000 7ff498a00700 1 -- 172.21.15.1:0/3299018502 shutdown_connections 2024-02-02T15:39:47.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.947+0000 7ff498a00700 1 --2- 172.21.15.1:0/3299018502 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff4700a0710 0x7ff4700a2bd0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:47.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.947+0000 7ff498a00700 1 --2- 172.21.15.1:0/3299018502 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff490112c90 0x7ff4901a1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:47.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.947+0000 7ff498a00700 1 --2- 172.21.15.1:0/3299018502 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff4901096e0 0x7ff49019d3e0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:47.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.947+0000 7ff498a00700 1 --2- 172.21.15.1:0/3299018502 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff49010a000 0x7ff49019d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:39:47.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.947+0000 7ff498a00700 1 -- 172.21.15.1:0/3299018502 >> 172.21.15.1:0/3299018502 conn(0x7ff49006d130 msgr2=0x7ff490070950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:39:47.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.947+0000 7ff498a00700 1 -- 172.21.15.1:0/3299018502 shutdown_connections 2024-02-02T15:39:47.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:39:47.947+0000 7ff498a00700 1 -- 172.21.15.1:0/3299018502 wait complete. 2024-02-02T15:39:47.965 INFO:tasks.cephfs.filesystem:scrub status for tag:a6279fd4-6b03-475b-ad0c-0c5a4605ced6 - {'path': '/', 'tag': 'a6279fd4-6b03-475b-ad0c-0c5a4605ced6', 'options': 'recursive,force'} 2024-02-02T15:39:48.273 INFO:tasks.workunit.client.0.smithi001.stdout: 1 695258 5.35 MB/sec execute 562 sec latency 2549.403 ms 2024-02-02T15:39:48.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:39:48 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:39:48.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:39:48 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:39:48.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:39:48 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:39:48.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:39:48 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:39:48.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:39:48 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:39:48.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:39:48 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:39:48.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:39:48 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:39:48.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:39:48 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:39:48.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:39:48 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:39:48.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:39:48 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:39:48.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:39:48 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:39:48.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:39:48 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:39:48.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:39:48 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:39:48.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:39:48 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:39:48.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:39:48 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:39:48.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:39:48 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:39:48.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:39:48 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:39:48.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:39:48 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:39:48.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:39:48 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:39:48.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:39:48 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:39:48.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:39:48 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:39:48.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:39:48 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:39:48.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:39:48 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:39:48.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:39:48 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:39:48.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:39:48 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:39:48.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:39:48 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:39:48.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:39:48 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:39:48.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:39:48 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:39:48.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:39:48 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:39:48.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:39:48 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:39:48.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:39:48 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:39:48.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:39:48 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:39:48.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:39:48 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:39:48.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:39:48 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:39:48.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:39:48 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:39:48.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:39:48 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:39:48.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:39:48 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:39:48.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:39:48 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:39:48.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:39:48 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:39:48.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:39:48 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:39:48.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:39:48 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:39:48.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:39:48 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:39:49.273 INFO:tasks.workunit.client.0.smithi001.stdout: 1 696487 5.36 MB/sec execute 563 sec latency 17.010 ms 2024-02-02T15:39:49.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:39:49 smithi071 ceph-mon[44707]: scrub queued for path: / 2024-02-02T15:39:49.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:39:49 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:39:49.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:39:49 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:39:49.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:39:49 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:39:49.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:39:49 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:39:49.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:39:49 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:39:49.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:39:49 smithi071 ceph-mon[44707]: pgmap v399: 129 pgs: 129 active+clean; 599 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.8 MiB/s wr, 117 op/s 2024-02-02T15:39:49.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:39:49 smithi142 ceph-mon[45728]: scrub queued for path: / 2024-02-02T15:39:49.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:39:49 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:39:49.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:39:49 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:39:49.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:39:49 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:39:49.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:39:49 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:39:49.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:39:49 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:39:49.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:39:49 smithi142 ceph-mon[45728]: pgmap v399: 129 pgs: 129 active+clean; 599 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.8 MiB/s wr, 117 op/s 2024-02-02T15:39:49.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:39:49 smithi001 ceph-mon[35470]: scrub queued for path: / 2024-02-02T15:39:49.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:39:49 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:39:49.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:39:49 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:39:49.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:39:49 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:39:49.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:39:49 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:39:49.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:39:49 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:39:49.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:39:49 smithi001 ceph-mon[35470]: pgmap v399: 129 pgs: 129 active+clean; 599 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.8 MiB/s wr, 117 op/s 2024-02-02T15:39:50.273 INFO:tasks.workunit.client.0.smithi001.stdout: 1 697732 5.36 MB/sec execute 564 sec latency 19.106 ms 2024-02-02T15:39:51.273 INFO:tasks.workunit.client.0.smithi001.stdout: 1 697732 5.35 MB/sec execute 565 sec latency 1018.115 ms 2024-02-02T15:39:51.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:39:51 smithi071 ceph-mon[44707]: pgmap v400: 129 pgs: 129 active+clean; 599 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.8 MiB/s wr, 117 op/s 2024-02-02T15:39:51.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:39:51 smithi142 ceph-mon[45728]: pgmap v400: 129 pgs: 129 active+clean; 599 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.8 MiB/s wr, 117 op/s 2024-02-02T15:39:51.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:39:51 smithi001 ceph-mon[35470]: pgmap v400: 129 pgs: 129 active+clean; 599 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.8 MiB/s wr, 117 op/s 2024-02-02T15:39:52.274 INFO:tasks.workunit.client.0.smithi001.stdout: 1 697732 5.34 MB/sec execute 566 sec latency 2018.266 ms 2024-02-02T15:39:53.274 INFO:tasks.workunit.client.0.smithi001.stdout: 1 698799 5.34 MB/sec execute 567 sec latency 2397.634 ms 2024-02-02T15:39:53.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:39:53 smithi071 ceph-mon[44707]: pgmap v401: 129 pgs: 129 active+clean; 604 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 178 op/s 2024-02-02T15:39:53.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:39:53 smithi142 ceph-mon[45728]: pgmap v401: 129 pgs: 129 active+clean; 604 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 178 op/s 2024-02-02T15:39:53.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:39:53 smithi001 ceph-mon[35470]: pgmap v401: 129 pgs: 129 active+clean; 604 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 178 op/s 2024-02-02T15:39:54.274 INFO:tasks.workunit.client.0.smithi001.stdout: 1 700012 5.34 MB/sec execute 568 sec latency 15.881 ms 2024-02-02T15:39:55.274 INFO:tasks.workunit.client.0.smithi001.stdout: 1 701232 5.34 MB/sec execute 569 sec latency 19.655 ms 2024-02-02T15:39:55.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:39:55 smithi071 ceph-mon[44707]: pgmap v402: 129 pgs: 129 active+clean; 604 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.8 MiB/s wr, 123 op/s 2024-02-02T15:39:55.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:39:55 smithi142 ceph-mon[45728]: pgmap v402: 129 pgs: 129 active+clean; 604 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.8 MiB/s wr, 123 op/s 2024-02-02T15:39:55.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:39:55 smithi001 ceph-mon[35470]: pgmap v402: 129 pgs: 129 active+clean; 604 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.8 MiB/s wr, 123 op/s 2024-02-02T15:39:56.274 INFO:tasks.workunit.client.0.smithi001.stdout: 1 701275 5.33 MB/sec execute 570 sec latency 952.190 ms 2024-02-02T15:39:57.274 INFO:tasks.workunit.client.0.smithi001.stdout: 1 701275 5.32 MB/sec execute 571 sec latency 1952.310 ms 2024-02-02T15:39:57.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:39:57 smithi071 ceph-mon[44707]: pgmap v403: 129 pgs: 129 active+clean; 609 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 179 op/s 2024-02-02T15:39:57.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:39:57 smithi142 ceph-mon[45728]: pgmap v403: 129 pgs: 129 active+clean; 609 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 179 op/s 2024-02-02T15:39:57.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:39:57 smithi001 ceph-mon[35470]: pgmap v403: 129 pgs: 129 active+clean; 609 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.6 MiB/s wr, 179 op/s 2024-02-02T15:39:58.274 INFO:tasks.workunit.client.0.smithi001.stdout: 1 702342 5.33 MB/sec execute 572 sec latency 2326.816 ms 2024-02-02T15:39:59.274 INFO:tasks.workunit.client.0.smithi001.stdout: 1 703538 5.33 MB/sec execute 573 sec latency 17.306 ms 2024-02-02T15:39:59.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:39:59 smithi071 ceph-mon[44707]: pgmap v404: 129 pgs: 129 active+clean; 609 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1001 KiB/s rd, 1.5 MiB/s wr, 117 op/s 2024-02-02T15:39:59.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:39:59 smithi142 ceph-mon[45728]: pgmap v404: 129 pgs: 129 active+clean; 609 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1001 KiB/s rd, 1.5 MiB/s wr, 117 op/s 2024-02-02T15:39:59.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:39:59 smithi001 ceph-mon[35470]: pgmap v404: 129 pgs: 129 active+clean; 609 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1001 KiB/s rd, 1.5 MiB/s wr, 117 op/s 2024-02-02T15:40:00.274 INFO:tasks.workunit.client.0.smithi001.stdout: 1 704600 5.33 MB/sec execute 574 sec latency 19.485 ms 2024-02-02T15:40:00.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:00 smithi071 ceph-mon[44707]: overall HEALTH_OK 2024-02-02T15:40:00.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:00 smithi142 ceph-mon[45728]: overall HEALTH_OK 2024-02-02T15:40:00.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:00 smithi001 ceph-mon[35470]: overall HEALTH_OK 2024-02-02T15:40:01.274 INFO:tasks.workunit.client.0.smithi001.stdout: 1 704822 5.32 MB/sec execute 575 sec latency 660.783 ms 2024-02-02T15:40:01.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:01 smithi071 ceph-mon[44707]: pgmap v405: 129 pgs: 129 active+clean; 609 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1001 KiB/s rd, 1.5 MiB/s wr, 117 op/s 2024-02-02T15:40:01.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:01 smithi142 ceph-mon[45728]: pgmap v405: 129 pgs: 129 active+clean; 609 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1001 KiB/s rd, 1.5 MiB/s wr, 117 op/s 2024-02-02T15:40:01.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:01 smithi001 ceph-mon[35470]: pgmap v405: 129 pgs: 129 active+clean; 609 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1001 KiB/s rd, 1.5 MiB/s wr, 117 op/s 2024-02-02T15:40:02.275 INFO:tasks.workunit.client.0.smithi001.stdout: 1 704822 5.31 MB/sec execute 576 sec latency 1660.955 ms 2024-02-02T15:40:02.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:02 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:40:02.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:02 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:40:02.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:02 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:40:02.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:02 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:40:02.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:02 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:40:02.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:02 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:40:02.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:02 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:40:02.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:02 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:40:02.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:02 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:40:02.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:02 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:40:02.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:02 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:40:02.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:02 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:40:02.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:02 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:40:02.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:02 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:40:02.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:02 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:40:03.275 INFO:tasks.workunit.client.0.smithi001.stdout: 1 705572 5.31 MB/sec execute 577 sec latency 2039.201 ms 2024-02-02T15:40:03.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:03 smithi071 ceph-mon[44707]: pgmap v406: 129 pgs: 129 active+clean; 614 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 175 op/s 2024-02-02T15:40:03.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:03 smithi142 ceph-mon[45728]: pgmap v406: 129 pgs: 129 active+clean; 614 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 175 op/s 2024-02-02T15:40:03.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:03 smithi001 ceph-mon[35470]: pgmap v406: 129 pgs: 129 active+clean; 614 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 175 op/s 2024-02-02T15:40:04.275 INFO:tasks.workunit.client.0.smithi001.stdout: 1 707036 5.32 MB/sec execute 578 sec latency 15.889 ms 2024-02-02T15:40:05.275 INFO:tasks.workunit.client.0.smithi001.stdout: 1 708076 5.32 MB/sec execute 579 sec latency 18.716 ms 2024-02-02T15:40:05.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:05 smithi071 ceph-mon[44707]: pgmap v407: 129 pgs: 129 active+clean; 614 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 982 KiB/s rd, 1.5 MiB/s wr, 113 op/s 2024-02-02T15:40:05.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:05 smithi142 ceph-mon[45728]: pgmap v407: 129 pgs: 129 active+clean; 614 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 982 KiB/s rd, 1.5 MiB/s wr, 113 op/s 2024-02-02T15:40:05.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:05 smithi001 ceph-mon[35470]: pgmap v407: 129 pgs: 129 active+clean; 614 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 982 KiB/s rd, 1.5 MiB/s wr, 113 op/s 2024-02-02T15:40:06.275 INFO:tasks.workunit.client.0.smithi001.stdout: 1 708708 5.31 MB/sec execute 580 sec latency 177.682 ms 2024-02-02T15:40:07.275 INFO:tasks.workunit.client.0.smithi001.stdout: 1 710382 5.32 MB/sec execute 581 sec latency 19.839 ms 2024-02-02T15:40:07.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:07 smithi071 ceph-mon[44707]: pgmap v408: 129 pgs: 129 active+clean; 620 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 185 op/s 2024-02-02T15:40:07.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:07 smithi071 ceph-mon[44707]: osdmap e102: 12 total, 12 up, 12 in 2024-02-02T15:40:07.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:07 smithi142 ceph-mon[45728]: pgmap v408: 129 pgs: 129 active+clean; 620 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 185 op/s 2024-02-02T15:40:07.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:07 smithi142 ceph-mon[45728]: osdmap e102: 12 total, 12 up, 12 in 2024-02-02T15:40:07.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:07 smithi001 ceph-mon[35470]: pgmap v408: 129 pgs: 129 active+clean; 620 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.4 MiB/s wr, 185 op/s 2024-02-02T15:40:07.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:07 smithi001 ceph-mon[35470]: osdmap e102: 12 total, 12 up, 12 in 2024-02-02T15:40:08.275 INFO:tasks.workunit.client.0.smithi001.stdout: 1 711107 5.31 MB/sec execute 582 sec latency 39.926 ms 2024-02-02T15:40:09.275 INFO:tasks.workunit.client.0.smithi001.stdout: 1 711916 5.31 MB/sec execute 583 sec latency 327.237 ms 2024-02-02T15:40:09.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:09 smithi071 ceph-mon[44707]: pgmap v410: 129 pgs: 129 active+clean; 620 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 155 op/s 2024-02-02T15:40:09.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:09 smithi142 ceph-mon[45728]: pgmap v410: 129 pgs: 129 active+clean; 620 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 155 op/s 2024-02-02T15:40:09.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:09 smithi001 ceph-mon[35470]: pgmap v410: 129 pgs: 129 active+clean; 620 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 155 op/s 2024-02-02T15:40:10.275 INFO:tasks.workunit.client.0.smithi001.stdout: 1 711916 5.30 MB/sec execute 584 sec latency 1327.333 ms 2024-02-02T15:40:10.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:10 smithi071 ceph-mon[44707]: pgmap v411: 129 pgs: 129 active+clean; 620 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 155 op/s 2024-02-02T15:40:10.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:10 smithi142 ceph-mon[45728]: pgmap v411: 129 pgs: 129 active+clean; 620 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 155 op/s 2024-02-02T15:40:10.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:10 smithi001 ceph-mon[35470]: pgmap v411: 129 pgs: 129 active+clean; 620 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.0 MiB/s wr, 155 op/s 2024-02-02T15:40:11.276 INFO:tasks.workunit.client.0.smithi001.stdout: 1 711916 5.30 MB/sec execute 585 sec latency 2327.465 ms 2024-02-02T15:40:12.276 INFO:tasks.workunit.client.0.smithi001.stdout: 1 711916 5.29 MB/sec execute 586 sec latency 3327.558 ms 2024-02-02T15:40:13.276 INFO:tasks.workunit.client.0.smithi001.stdout: 1 712989 5.29 MB/sec execute 587 sec latency 3700.401 ms 2024-02-02T15:40:13.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:13 smithi071 ceph-mon[44707]: pgmap v412: 129 pgs: 129 active+clean; 602 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 152 op/s 2024-02-02T15:40:13.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:13 smithi142 ceph-mon[45728]: pgmap v412: 129 pgs: 129 active+clean; 602 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 152 op/s 2024-02-02T15:40:13.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:13 smithi001 ceph-mon[35470]: pgmap v412: 129 pgs: 129 active+clean; 602 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 152 op/s 2024-02-02T15:40:14.276 INFO:tasks.workunit.client.0.smithi001.stdout: 1 714206 5.29 MB/sec execute 588 sec latency 17.818 ms 2024-02-02T15:40:15.276 INFO:tasks.workunit.client.0.smithi001.stdout: 1 715424 5.29 MB/sec execute 589 sec latency 19.848 ms 2024-02-02T15:40:15.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:15 smithi071 ceph-mon[44707]: pgmap v413: 129 pgs: 129 active+clean; 602 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 152 op/s 2024-02-02T15:40:15.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:15 smithi142 ceph-mon[45728]: pgmap v413: 129 pgs: 129 active+clean; 602 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 152 op/s 2024-02-02T15:40:15.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:15 smithi001 ceph-mon[35470]: pgmap v413: 129 pgs: 129 active+clean; 602 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.0 MiB/s wr, 152 op/s 2024-02-02T15:40:16.276 INFO:tasks.workunit.client.0.smithi001.stdout: 1 715457 5.28 MB/sec execute 590 sec latency 977.243 ms 2024-02-02T15:40:17.276 INFO:tasks.workunit.client.0.smithi001.stdout: 1 715457 5.27 MB/sec execute 591 sec latency 1977.326 ms 2024-02-02T15:40:17.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:17 smithi071 ceph-mon[44707]: pgmap v414: 129 pgs: 129 active+clean; 607 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 135 op/s 2024-02-02T15:40:17.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:17 smithi071 ceph-mon[44707]: osdmap e103: 12 total, 12 up, 12 in 2024-02-02T15:40:17.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:17 smithi142 ceph-mon[45728]: pgmap v414: 129 pgs: 129 active+clean; 607 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 135 op/s 2024-02-02T15:40:17.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:17 smithi142 ceph-mon[45728]: osdmap e103: 12 total, 12 up, 12 in 2024-02-02T15:40:17.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:17 smithi001 ceph-mon[35470]: pgmap v414: 129 pgs: 129 active+clean; 607 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 135 op/s 2024-02-02T15:40:17.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:17 smithi001 ceph-mon[35470]: osdmap e103: 12 total, 12 up, 12 in 2024-02-02T15:40:17.966 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-02T15:40:18.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.182+0000 7f4a51800700 1 -- 172.21.15.1:0/1826630237 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a4c101c70 msgr2=0x7f4a4c068010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:18.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.182+0000 7f4a51800700 1 --2- 172.21.15.1:0/1826630237 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a4c101c70 0x7f4a4c068010 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c009990 tx=0x7f4a3c009ca0 comp rx=0 tx=0).stop 2024-02-02T15:40:18.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.182+0000 7f4a51800700 1 -- 172.21.15.1:0/1826630237 shutdown_connections 2024-02-02T15:40:18.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.182+0000 7f4a51800700 1 --2- 172.21.15.1:0/1826630237 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a4c10d610 0x7f4a4c10fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:18.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.182+0000 7f4a51800700 1 --2- 172.21.15.1:0/1826630237 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a4c101c70 0x7f4a4c068010 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:18.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.182+0000 7f4a51800700 1 --2- 172.21.15.1:0/1826630237 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4a4c0685e0 0x7f4a4c068a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:18.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.182+0000 7f4a51800700 1 -- 172.21.15.1:0/1826630237 >> 172.21.15.1:0/1826630237 conn(0x7f4a4c0757d0 msgr2=0x7f4a4c075be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:18.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.182+0000 7f4a51800700 1 -- 172.21.15.1:0/1826630237 shutdown_connections 2024-02-02T15:40:18.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.182+0000 7f4a51800700 1 -- 172.21.15.1:0/1826630237 wait complete. 2024-02-02T15:40:18.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.183+0000 7f4a51800700 1 Processor -- start 2024-02-02T15:40:18.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.184+0000 7f4a51800700 1 -- start start 2024-02-02T15:40:18.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.184+0000 7f4a51800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a4c0685e0 0x7f4a4c1990a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:18.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.184+0000 7f4a51800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a4c101c70 0x7f4a4c1995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:18.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.184+0000 7f4a51800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4a4c10d610 0x7f4a4c19da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:18.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.184+0000 7f4a51800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a4c19e080 con 0x7f4a4c10d610 2024-02-02T15:40:18.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.184+0000 7f4a51800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a4c19e1f0 con 0x7f4a4c0685e0 2024-02-02T15:40:18.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.184+0000 7f4a51800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a4c19e3b0 con 0x7f4a4c101c70 2024-02-02T15:40:18.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.184+0000 7f4a4aa00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a4c0685e0 0x7f4a4c1990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:18.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.184+0000 7f4a4aa00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a4c0685e0 0x7f4a4c1990a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:41982/0 (socket says 172.21.15.1:41982) 2024-02-02T15:40:18.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.184+0000 7f4a4a000700 1 --2- 172.21.15.1:0/2584543142 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a4c101c70 0x7f4a4c1995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:18.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.184+0000 7f4a4aa00700 1 -- 172.21.15.1:0/2584543142 learned_addr learned my addr 172.21.15.1:0/2584543142 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:40:18.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.184+0000 7f4a4b400700 1 --2- 172.21.15.1:0/2584543142 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4a4c10d610 0x7f4a4c19da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:18.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.185+0000 7f4a4a000700 1 -- 172.21.15.1:0/2584543142 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a4c0685e0 msgr2=0x7f4a4c1990a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:18.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.185+0000 7f4a4a000700 1 --2- 172.21.15.1:0/2584543142 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a4c0685e0 0x7f4a4c1990a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:18.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.185+0000 7f4a4a000700 1 -- 172.21.15.1:0/2584543142 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4a4c10d610 msgr2=0x7f4a4c19da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:18.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.185+0000 7f4a4a000700 1 --2- 172.21.15.1:0/2584543142 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4a4c10d610 0x7f4a4c19da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:18.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.185+0000 7f4a4a000700 1 -- 172.21.15.1:0/2584543142 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a44009c40 con 0x7f4a4c101c70 2024-02-02T15:40:18.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.185+0000 7f4a4aa00700 1 --2- 172.21.15.1:0/2584543142 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a4c0685e0 0x7f4a4c1990a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:40:18.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.185+0000 7f4a4a000700 1 --2- 172.21.15.1:0/2584543142 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a4c101c70 0x7f4a4c1995e0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f4a3400e9c0 tx=0x7f4a3400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:18.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.185+0000 7f4a43400700 1 -- 172.21.15.1:0/2584543142 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a34004220 con 0x7f4a4c101c70 2024-02-02T15:40:18.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.185+0000 7f4a43400700 1 -- 172.21.15.1:0/2584543142 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a34004380 con 0x7f4a4c101c70 2024-02-02T15:40:18.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.185+0000 7f4a51800700 1 -- 172.21.15.1:0/2584543142 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a3c0095f0 con 0x7f4a4c101c70 2024-02-02T15:40:18.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.185+0000 7f4a43400700 1 -- 172.21.15.1:0/2584543142 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a34003c00 con 0x7f4a4c101c70 2024-02-02T15:40:18.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.185+0000 7f4a51800700 1 -- 172.21.15.1:0/2584543142 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a4c1a5cf0 con 0x7f4a4c101c70 2024-02-02T15:40:18.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.189+0000 7f4a43400700 1 -- 172.21.15.1:0/2584543142 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f4a34003d60 con 0x7f4a4c101c70 2024-02-02T15:40:18.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.190+0000 7f4a653a9b80 1 Processor -- start 2024-02-02T15:40:18.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.190+0000 7f4a653a9b80 1 -- start start 2024-02-02T15:40:18.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.191+0000 7f4a43400700 1 --2- 172.21.15.1:0/2584543142 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f4a2c0a0690 0x7f4a2c0a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:18.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.191+0000 7f4a653a9b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d2403c6010 0x55d2403c63f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:18.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.191+0000 7f4a653a9b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d2403c6930 0x55d2403d12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:18.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.191+0000 7f4a653a9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d2403d17e0 0x55d2403d3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:18.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.191+0000 7f4a43400700 1 -- 172.21.15.1:0/2584543142 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f4a34014070 con 0x7f4a4c101c70 2024-02-02T15:40:18.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.191+0000 7f4a653a9b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d24033a4c0 con 0x55d2403c6930 2024-02-02T15:40:18.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.191+0000 7f4a653a9b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d24031eaa0 con 0x55d2403c6010 2024-02-02T15:40:18.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.191+0000 7f4a653a9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d24031c460 con 0x55d2403d17e0 2024-02-02T15:40:18.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.191+0000 7f4a4b400700 1 --2- 172.21.15.1:0/2584543142 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f4a2c0a0690 0x7f4a2c0a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:18.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.191+0000 7f4a4b400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d2403d17e0 0x55d2403d3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:18.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.191+0000 7f4a4a000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d2403c6930 0x55d2403d12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:18.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.191+0000 7f4a4b400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d2403d17e0 0x55d2403d3bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:40560/0 (socket says 172.21.15.1:40560) 2024-02-02T15:40:18.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.191+0000 7f4a4b400700 1 -- 172.21.15.1:0/1636457394 learned_addr learned my addr 172.21.15.1:0/1636457394 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:40:18.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.191+0000 7f4a4aa00700 1 --2- 172.21.15.1:0/1636457394 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d2403c6010 0x55d2403c63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:18.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.192+0000 7f4a4aa00700 1 -- 172.21.15.1:0/1636457394 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d2403d17e0 msgr2=0x55d2403d3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:18.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.192+0000 7f4a4aa00700 1 --2- 172.21.15.1:0/1636457394 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d2403d17e0 0x55d2403d3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:18.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.192+0000 7f4a4aa00700 1 -- 172.21.15.1:0/1636457394 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d2403c6930 msgr2=0x55d2403d12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:18.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.192+0000 7f4a4aa00700 1 --2- 172.21.15.1:0/1636457394 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d2403c6930 0x55d2403d12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:18.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.192+0000 7f4a4aa00700 1 -- 172.21.15.1:0/1636457394 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d2401993b0 con 0x55d2403c6010 2024-02-02T15:40:18.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.192+0000 7f4a4aa00700 1 --2- 172.21.15.1:0/1636457394 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d2403c6010 0x55d2403c63f0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c005910 tx=0x7f4a3c00b610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d87a03bb03313398 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:18.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.192+0000 7f4a4b400700 1 --2- 172.21.15.1:0/2584543142 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f4a2c0a0690 0x7f4a2c0a2b50 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4a44008ef0 tx=0x7f4a44009920 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:18.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.192+0000 7f4a3be00700 1 -- 172.21.15.1:0/1636457394 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c01d030 con 0x55d2403c6010 2024-02-02T15:40:18.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.192+0000 7f4a3be00700 1 -- 172.21.15.1:0/1636457394 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a3c01e040 con 0x55d2403c6010 2024-02-02T15:40:18.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.192+0000 7f4a3be00700 1 -- 172.21.15.1:0/1636457394 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c004d60 con 0x55d2403c6010 2024-02-02T15:40:18.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.193+0000 7f4a653a9b80 1 -- 172.21.15.1:0/1636457394 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d2403c6010 msgr2=0x55d2403c63f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:18.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.193+0000 7f4a653a9b80 1 --2- 172.21.15.1:0/1636457394 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d2403c6010 0x55d2403c63f0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c005910 tx=0x7f4a3c00b610 comp rx=0 tx=0).stop 2024-02-02T15:40:18.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.193+0000 7f4a653a9b80 1 -- 172.21.15.1:0/1636457394 shutdown_connections 2024-02-02T15:40:18.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.193+0000 7f4a653a9b80 1 --2- 172.21.15.1:0/1636457394 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d2403d17e0 0x55d2403d3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:18.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.193+0000 7f4a653a9b80 1 --2- 172.21.15.1:0/1636457394 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d2403c6010 0x55d2403c63f0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:18.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.193+0000 7f4a653a9b80 1 --2- 172.21.15.1:0/1636457394 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d2403c6930 0x55d2403d12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:18.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.193+0000 7f4a653a9b80 1 -- 172.21.15.1:0/1636457394 >> 172.21.15.1:0/1636457394 conn(0x55d240325db0 msgr2=0x55d2403c23b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:18.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.193+0000 7f4a653a9b80 1 -- 172.21.15.1:0/1636457394 shutdown_connections 2024-02-02T15:40:18.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.193+0000 7f4a653a9b80 1 -- 172.21.15.1:0/1636457394 wait complete. 2024-02-02T15:40:18.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.194+0000 7f4a653a9b80 1 -- start start 2024-02-02T15:40:18.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.194+0000 7f4a653a9b80 1 Processor -- start 2024-02-02T15:40:18.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.194+0000 7f4a653a9b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d2403c6010 0x55d2403d11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:18.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.194+0000 7f4a653a9b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d2403c6930 0x55d2403ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:18.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.194+0000 7f4a653a9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d2403d17e0 0x55d2403ce9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:18.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.194+0000 7f4a653a9b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d240317d50 con 0x55d2403c6930 2024-02-02T15:40:18.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.194+0000 7f4a653a9b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2402b3d80 con 0x55d2403c6010 2024-02-02T15:40:18.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.194+0000 7f4a653a9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2402b3a20 con 0x55d2403d17e0 2024-02-02T15:40:18.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.194+0000 7f4a653a9b80 10 client.0 _ll_register_callbacks cb 0x55d2402b5290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:40:18.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.195+0000 7f4a4aa00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d2403c6010 0x55d2403d11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:18.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.195+0000 7f4a4aa00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d2403c6010 0x55d2403d11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42004/0 (socket says 172.21.15.1:42004) 2024-02-02T15:40:18.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.195+0000 7f4a4aa00700 1 -- 172.21.15.1:0/4119265557 learned_addr learned my addr 172.21.15.1:0/4119265557 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:40:18.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.195+0000 7f4a4b400700 1 --2- 172.21.15.1:0/4119265557 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d2403d17e0 0x55d2403ce9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:18.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.195+0000 7f4a4a000700 1 --2- 172.21.15.1:0/4119265557 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d2403c6930 0x55d2403ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:18.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.195+0000 7f4a4aa00700 1 -- 172.21.15.1:0/4119265557 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d2403d17e0 msgr2=0x55d2403ce9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:18.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.195+0000 7f4a4aa00700 1 --2- 172.21.15.1:0/4119265557 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d2403d17e0 0x55d2403ce9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:18.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.195+0000 7f4a4aa00700 1 -- 172.21.15.1:0/4119265557 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d2403c6930 msgr2=0x55d2403ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:18.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.195+0000 7f4a4aa00700 1 --2- 172.21.15.1:0/4119265557 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d2403c6930 0x55d2403ce3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:18.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.195+0000 7f4a4aa00700 1 -- 172.21.15.1:0/4119265557 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a4c1106e0 con 0x55d2403c6010 2024-02-02T15:40:18.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.195+0000 7f4a4aa00700 1 --2- 172.21.15.1:0/4119265557 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d2403c6010 0x55d2403d11a0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c004920 tx=0x7f4a3c004800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c6ab68a3242b6df server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:18.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.195+0000 7f4a38c00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:40:18.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.195+0000 7f4a38c00700 1 -- 172.21.15.1:0/4119265557 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c01d040 con 0x55d2403c6010 2024-02-02T15:40:18.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.195+0000 7f4a38c00700 1 -- 172.21.15.1:0/4119265557 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f4a3c005460 con 0x55d2403c6010 2024-02-02T15:40:18.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.195+0000 7f4a653a9b80 1 -- 172.21.15.1:0/4119265557 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d2403ba0f0 con 0x55d2403c6010 2024-02-02T15:40:18.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.196+0000 7f4a38c00700 1 -- 172.21.15.1:0/4119265557 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a3c004e70 con 0x55d2403c6010 2024-02-02T15:40:18.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.196+0000 7f4a653a9b80 10 client.29099 fetch_fsmap learned FSMap version 52 2024-02-02T15:40:18.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.196+0000 7f4a653a9b80 1 -- 172.21.15.1:0/4119265557 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x55d240145b60 con 0x55d2403c6010 2024-02-02T15:40:18.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.196+0000 7f4a38c00700 1 -- 172.21.15.1:0/4119265557 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d2403ba0f0 con 0x55d2403c6010 2024-02-02T15:40:18.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.196+0000 7f4a38c00700 1 -- 172.21.15.1:0/4119265557 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 52) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f4a3c023460 con 0x55d2403c6010 2024-02-02T15:40:18.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.196+0000 7f4a653a9b80 10 client.29099 fetch_fsmap finished waiting for FSMap version 52 2024-02-02T15:40:18.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.196+0000 7f4a653a9b80 10 client.29099 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:40:18.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.196+0000 7f4a653a9b80 20 client.29099 populate_metadata read hostname 'smithi001' 2024-02-02T15:40:18.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.196+0000 7f4a653a9b80 1 --2- 172.21.15.1:0/4119265557 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55d2403e0c70 0x55d2403e3060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:18.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.196+0000 7f4a653a9b80 4 client.29099 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:40:18.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.196+0000 7f4a653a9b80 1 -- 172.21.15.1:0/4119265557 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d240310dd0 con 0x55d2403e0c70 2024-02-02T15:40:18.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.197+0000 7f4a4a000700 1 --2- 172.21.15.1:0/4119265557 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55d2403e0c70 0x55d2403e3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:18.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.197+0000 7f4a4a000700 1 --2- 172.21.15.1:0/4119265557 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55d2403e0c70 0x55d2403e3060 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5535436962e5d6ff server_cookie=aa40284c8b2d8770 in_seq=0 out_seq=0 2024-02-02T15:40:18.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.197+0000 7f4a38c00700 10 client.29099 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:40:18.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.197+0000 7f4a38c00700 1 -- 172.21.15.1:0/4119265557 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55d240310dd0 con 0x55d2403e0c70 2024-02-02T15:40:18.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.197+0000 7f4a38c00700 10 client.29099 handle_command_reply: tid=0 2024-02-02T15:40:18.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.198+0000 7f4a653a9b80 1 client.29099 shutdown 2024-02-02T15:40:18.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.198+0000 7f4a653a9b80 1 -- 172.21.15.1:0/4119265557 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d2403c6010 msgr2=0x55d2403d11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:18.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.198+0000 7f4a653a9b80 1 --2- 172.21.15.1:0/4119265557 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d2403c6010 0x55d2403d11a0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c004920 tx=0x7f4a3c004800 comp rx=0 tx=0).stop 2024-02-02T15:40:18.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.198+0000 7f4a653a9b80 1 -- 172.21.15.1:0/4119265557 shutdown_connections 2024-02-02T15:40:18.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.198+0000 7f4a653a9b80 1 --2- 172.21.15.1:0/4119265557 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55d2403e0c70 0x55d2403e3060 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:18.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.198+0000 7f4a653a9b80 1 --2- 172.21.15.1:0/4119265557 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d2403d17e0 0x55d2403ce9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:18.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.198+0000 7f4a38c00700 0 client.29099 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:40:18.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.198+0000 7f4a653a9b80 1 --2- 172.21.15.1:0/4119265557 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d2403c6010 0x55d2403d11a0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:18.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.198+0000 7f4a653a9b80 1 --2- 172.21.15.1:0/4119265557 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d2403c6930 0x55d2403ce3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:18.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.198+0000 7f4a653a9b80 1 -- 172.21.15.1:0/4119265557 >> 172.21.15.1:0/4119265557 conn(0x55d240325db0 msgr2=0x55d2403c1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:18.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.198+0000 7f4a653a9b80 1 -- 172.21.15.1:0/4119265557 shutdown_connections 2024-02-02T15:40:18.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.198+0000 7f4a653a9b80 1 -- 172.21.15.1:0/4119265557 wait complete. 2024-02-02T15:40:18.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.198+0000 7f4a653a9b80 20 client.29099 trim_cache size 0 max 16384 2024-02-02T15:40:18.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.216+0000 7f4a653a9b80 1 Processor -- start 2024-02-02T15:40:18.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.216+0000 7f4a653a9b80 1 -- start start 2024-02-02T15:40:18.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.216+0000 7f4a653a9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d2403c6010 0x55d2403d0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:18.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.216+0000 7f4a653a9b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d2403c6930 0x55d2403c9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:18.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.217+0000 7f4a653a9b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d2403d17e0 0x55d2403c9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:18.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.217+0000 7f4a653a9b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d24019b6a0 con 0x55d2403d17e0 2024-02-02T15:40:18.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.217+0000 7f4a653a9b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2403cc930 con 0x55d2403c6930 2024-02-02T15:40:18.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.217+0000 7f4a653a9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2403ca240 con 0x55d2403c6010 2024-02-02T15:40:18.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.217+0000 7f4a4aa00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d2403c6010 0x55d2403d0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:18.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.217+0000 7f4a4a000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d2403c6930 0x55d2403c9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:18.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.217+0000 7f4a4aa00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d2403c6010 0x55d2403d0cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:40568/0 (socket says 172.21.15.1:40568) 2024-02-02T15:40:18.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.217+0000 7f4a4aa00700 1 -- 172.21.15.1:0/2441843620 learned_addr learned my addr 172.21.15.1:0/2441843620 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:40:18.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.217+0000 7f4a4b400700 1 --2- 172.21.15.1:0/2441843620 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d2403d17e0 0x55d2403c9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:18.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.217+0000 7f4a4aa00700 1 -- 172.21.15.1:0/2441843620 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d2403c6930 msgr2=0x55d2403c9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:18.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.217+0000 7f4a4aa00700 1 --2- 172.21.15.1:0/2441843620 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d2403c6930 0x55d2403c9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:18.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.217+0000 7f4a4aa00700 1 -- 172.21.15.1:0/2441843620 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d2403d17e0 msgr2=0x55d2403c9c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:18.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.217+0000 7f4a4aa00700 1 --2- 172.21.15.1:0/2441843620 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d2403d17e0 0x55d2403c9c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:18.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.217+0000 7f4a4aa00700 1 -- 172.21.15.1:0/2441843620 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d2402ddea0 con 0x55d2403c6010 2024-02-02T15:40:18.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.217+0000 7f4a4a000700 1 --2- 172.21.15.1:0/2441843620 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d2403c6930 0x55d2403c9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:40:18.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.218+0000 7f4a4aa00700 1 --2- 172.21.15.1:0/2441843620 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d2403c6010 0x55d2403d0cd0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c009ce0 tx=0x7f4a3c00bf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bad544e6a3eaec9c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:18.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.218+0000 7f4a40c00700 1 -- 172.21.15.1:0/2441843620 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c005bc0 con 0x55d2403c6010 2024-02-02T15:40:18.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.218+0000 7f4a40c00700 1 -- 172.21.15.1:0/2441843620 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a3c005910 con 0x55d2403c6010 2024-02-02T15:40:18.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.218+0000 7f4a40c00700 1 -- 172.21.15.1:0/2441843620 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c00e3e0 con 0x55d2403c6010 2024-02-02T15:40:18.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.218+0000 7f4a653a9b80 1 -- 172.21.15.1:0/2441843620 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d2403c6010 msgr2=0x55d2403d0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:18.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.218+0000 7f4a653a9b80 1 --2- 172.21.15.1:0/2441843620 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d2403c6010 0x55d2403d0cd0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c009ce0 tx=0x7f4a3c00bf90 comp rx=0 tx=0).stop 2024-02-02T15:40:18.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.219+0000 7f4a653a9b80 1 -- 172.21.15.1:0/2441843620 shutdown_connections 2024-02-02T15:40:18.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.219+0000 7f4a653a9b80 1 --2- 172.21.15.1:0/2441843620 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d2403d17e0 0x55d2403c9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:18.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.219+0000 7f4a653a9b80 1 --2- 172.21.15.1:0/2441843620 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d2403c6010 0x55d2403d0cd0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:18.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.219+0000 7f4a653a9b80 1 --2- 172.21.15.1:0/2441843620 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d2403c6930 0x55d2403c9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:18.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.219+0000 7f4a653a9b80 1 -- 172.21.15.1:0/2441843620 >> 172.21.15.1:0/2441843620 conn(0x55d240325db0 msgr2=0x55d2403c0b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:18.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.219+0000 7f4a653a9b80 1 -- 172.21.15.1:0/2441843620 shutdown_connections 2024-02-02T15:40:18.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.219+0000 7f4a653a9b80 1 -- 172.21.15.1:0/2441843620 wait complete. 2024-02-02T15:40:18.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.219+0000 7f4a653a9b80 1 -- start start 2024-02-02T15:40:18.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.219+0000 7f4a653a9b80 1 Processor -- start 2024-02-02T15:40:18.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.220+0000 7f4a653a9b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d2403c6010 0x55d2403d4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:18.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.220+0000 7f4a653a9b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d2403c6930 0x55d2403d4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:18.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.220+0000 7f4a653a9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d2403d17e0 0x55d240439ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:18.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.220+0000 7f4a653a9b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d24043a4a0 con 0x55d2403c6010 2024-02-02T15:40:18.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.220+0000 7f4a653a9b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d24043a610 con 0x55d2403c6930 2024-02-02T15:40:18.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.220+0000 7f4a653a9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d24043a7d0 con 0x55d2403d17e0 2024-02-02T15:40:18.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.220+0000 7f4a653a9b80 10 client.0 _ll_register_callbacks cb 0x55d2402b5290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:40:18.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.220+0000 7f4a4aa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d2403c6010 0x55d2403d4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:18.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.220+0000 7f4a4a000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d2403c6930 0x55d2403d4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:18.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.220+0000 7f4a4b400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d2403d17e0 0x55d240439ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:18.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.220+0000 7f4a4a000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d2403c6930 0x55d2403d4a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42020/0 (socket says 172.21.15.1:42020) 2024-02-02T15:40:18.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.220+0000 7f4a4a000700 1 -- 172.21.15.1:0/2089867347 learned_addr learned my addr 172.21.15.1:0/2089867347 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:40:18.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.220+0000 7f4a4a000700 1 -- 172.21.15.1:0/2089867347 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d2403d17e0 msgr2=0x55d240439ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:18.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.220+0000 7f4a4a000700 1 --2- 172.21.15.1:0/2089867347 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d2403d17e0 0x55d240439ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:18.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.220+0000 7f4a4a000700 1 -- 172.21.15.1:0/2089867347 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d2403c6010 msgr2=0x55d2403d4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:18.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.220+0000 7f4a4a000700 1 --2- 172.21.15.1:0/2089867347 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d2403c6010 0x55d2403d4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:18.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.220+0000 7f4a4a000700 1 -- 172.21.15.1:0/2089867347 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a4c1a5cf0 con 0x55d2403c6930 2024-02-02T15:40:18.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.221+0000 7f4a4a000700 1 --2- 172.21.15.1:0/2089867347 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d2403c6930 0x55d2403d4a30 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f4a34099210 tx=0x7f4a340958d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7124a57c7585502 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:18.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.221+0000 7f4a38c00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:40:18.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.221+0000 7f4a38c00700 1 -- 172.21.15.1:0/2089867347 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a34099510 con 0x55d2403c6930 2024-02-02T15:40:18.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.221+0000 7f4a38c00700 1 -- 172.21.15.1:0/2089867347 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f4a3409d340 con 0x55d2403c6930 2024-02-02T15:40:18.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.221+0000 7f4a653a9b80 1 -- 172.21.15.1:0/2089867347 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4a3c01e040 con 0x55d2403c6930 2024-02-02T15:40:18.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.221+0000 7f4a38c00700 1 -- 172.21.15.1:0/2089867347 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a340a1580 con 0x55d2403c6930 2024-02-02T15:40:18.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.221+0000 7f4a653a9b80 10 client.29105 fetch_fsmap learned FSMap version 52 2024-02-02T15:40:18.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.221+0000 7f4a653a9b80 1 -- 172.21.15.1:0/2089867347 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x55d240145b60 con 0x55d2403c6930 2024-02-02T15:40:18.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.221+0000 7f4a38c00700 1 -- 172.21.15.1:0/2089867347 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a3c01e040 con 0x55d2403c6930 2024-02-02T15:40:18.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.221+0000 7f4a38c00700 1 -- 172.21.15.1:0/2089867347 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 52) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f4a34099770 con 0x55d2403c6930 2024-02-02T15:40:18.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.221+0000 7f4a653a9b80 10 client.29105 fetch_fsmap finished waiting for FSMap version 52 2024-02-02T15:40:18.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.221+0000 7f4a653a9b80 10 client.29105 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:40:18.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.221+0000 7f4a653a9b80 20 client.29105 populate_metadata read hostname 'smithi001' 2024-02-02T15:40:18.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.222+0000 7f4a653a9b80 1 --2- 172.21.15.1:0/2089867347 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55d24043bf00 0x55d24043e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:18.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.222+0000 7f4a653a9b80 4 client.29105 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2024-02-02T15:40:18.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.222+0000 7f4a653a9b80 1 -- 172.21.15.1:0/2089867347 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d2402d6f50 con 0x55d24043bf00 2024-02-02T15:40:18.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.222+0000 7f4a4aa00700 1 --2- 172.21.15.1:0/2089867347 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55d24043bf00 0x55d24043e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:18.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.222+0000 7f4a4aa00700 1 --2- 172.21.15.1:0/2089867347 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55d24043bf00 0x55d24043e320 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4db89dff7038c0ad server_cookie=ada82e154234d063 in_seq=0 out_seq=0 2024-02-02T15:40:18.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.222+0000 7f4a38c00700 10 client.29105 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:40:18.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.222+0000 7f4a38c00700 1 -- 172.21.15.1:0/2089867347 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55d2402d6f50 con 0x55d24043bf00 2024-02-02T15:40:18.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.222+0000 7f4a38c00700 10 client.29105 handle_command_reply: tid=0 2024-02-02T15:40:18.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.222+0000 7f4a653a9b80 1 client.29105 shutdown 2024-02-02T15:40:18.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.223+0000 7f4a653a9b80 1 -- 172.21.15.1:0/2089867347 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d2403c6930 msgr2=0x55d2403d4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:18.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.223+0000 7f4a653a9b80 1 --2- 172.21.15.1:0/2089867347 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d2403c6930 0x55d2403d4a30 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f4a34099210 tx=0x7f4a340958d0 comp rx=0 tx=0).stop 2024-02-02T15:40:18.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.223+0000 7f4a653a9b80 1 -- 172.21.15.1:0/2089867347 shutdown_connections 2024-02-02T15:40:18.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.223+0000 7f4a653a9b80 1 --2- 172.21.15.1:0/2089867347 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55d24043bf00 0x55d24043e320 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:18.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.223+0000 7f4a653a9b80 1 --2- 172.21.15.1:0/2089867347 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d2403d17e0 0x55d240439ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:18.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.223+0000 7f4a653a9b80 1 --2- 172.21.15.1:0/2089867347 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d2403c6010 0x55d2403d4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:18.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.223+0000 7f4a653a9b80 1 --2- 172.21.15.1:0/2089867347 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d2403c6930 0x55d2403d4a30 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:18.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.223+0000 7f4a653a9b80 1 -- 172.21.15.1:0/2089867347 >> 172.21.15.1:0/2089867347 conn(0x55d240325db0 msgr2=0x55d2403c9700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:18.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.223+0000 7f4a38c00700 0 client.29105 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:40:18.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.223+0000 7f4a653a9b80 1 -- 172.21.15.1:0/2089867347 shutdown_connections 2024-02-02T15:40:18.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.223+0000 7f4a653a9b80 1 -- 172.21.15.1:0/2089867347 wait complete. 2024-02-02T15:40:18.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.223+0000 7f4a653a9b80 20 client.29105 trim_cache size 0 max 16384 2024-02-02T15:40:18.227 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:40:18.227 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2024-02-02T15:40:18.228 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2024-02-02T15:40:18.228 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:40:18.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.224+0000 7f4a51800700 1 -- 172.21.15.1:0/2584543142 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f4a2c0a0690 msgr2=0x7f4a2c0a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:18.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.224+0000 7f4a51800700 1 --2- 172.21.15.1:0/2584543142 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f4a2c0a0690 0x7f4a2c0a2b50 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4a44008ef0 tx=0x7f4a44009920 comp rx=0 tx=0).stop 2024-02-02T15:40:18.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.224+0000 7f4a51800700 1 -- 172.21.15.1:0/2584543142 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a4c101c70 msgr2=0x7f4a4c1995e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:18.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.224+0000 7f4a51800700 1 --2- 172.21.15.1:0/2584543142 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a4c101c70 0x7f4a4c1995e0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f4a3400e9c0 tx=0x7f4a3400ecd0 comp rx=0 tx=0).stop 2024-02-02T15:40:18.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.224+0000 7f4a51800700 1 -- 172.21.15.1:0/2584543142 shutdown_connections 2024-02-02T15:40:18.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.224+0000 7f4a51800700 1 --2- 172.21.15.1:0/2584543142 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f4a2c0a0690 0x7f4a2c0a2b50 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:18.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.224+0000 7f4a51800700 1 --2- 172.21.15.1:0/2584543142 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4a4c10d610 0x7f4a4c19da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:18.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.224+0000 7f4a51800700 1 --2- 172.21.15.1:0/2584543142 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a4c0685e0 0x7f4a4c1990a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:18.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.224+0000 7f4a51800700 1 --2- 172.21.15.1:0/2584543142 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a4c101c70 0x7f4a4c1995e0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:18.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.224+0000 7f4a51800700 1 -- 172.21.15.1:0/2584543142 >> 172.21.15.1:0/2584543142 conn(0x7f4a4c0757d0 msgr2=0x7f4a4c0fee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:18.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.224+0000 7f4a51800700 1 -- 172.21.15.1:0/2584543142 shutdown_connections 2024-02-02T15:40:18.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.224+0000 7f4a51800700 1 -- 172.21.15.1:0/2584543142 wait complete. 2024-02-02T15:40:18.244 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-02T15:40:18.244 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T15:40:18.276 INFO:tasks.workunit.client.0.smithi001.stdout: 1 716639 5.28 MB/sec execute 592 sec latency 2360.972 ms 2024-02-02T15:40:18.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.499+0000 7f83e6e00700 1 -- 172.21.15.1:0/750709056 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83e0073170 msgr2=0x7f83e0073550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:18.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.499+0000 7f83e6e00700 1 --2- 172.21.15.1:0/750709056 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83e0073170 0x7f83e0073550 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f83d0009990 tx=0x7f83d0009ca0 comp rx=0 tx=0).stop 2024-02-02T15:40:18.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.499+0000 7f83e6e00700 1 -- 172.21.15.1:0/750709056 shutdown_connections 2024-02-02T15:40:18.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.499+0000 7f83e6e00700 1 --2- 172.21.15.1:0/750709056 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83e010d4e0 0x7f83e010f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:18.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.499+0000 7f83e6e00700 1 --2- 172.21.15.1:0/750709056 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83e0073170 0x7f83e0073550 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:18.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.499+0000 7f83e6e00700 1 --2- 172.21.15.1:0/750709056 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83e0073a90 0x7f83e010cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:18.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.499+0000 7f83e6e00700 1 -- 172.21.15.1:0/750709056 >> 172.21.15.1:0/750709056 conn(0x7f83e00fc790 msgr2=0x7f83e00febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:18.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.499+0000 7f83e6e00700 1 -- 172.21.15.1:0/750709056 shutdown_connections 2024-02-02T15:40:18.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.499+0000 7f83e6e00700 1 -- 172.21.15.1:0/750709056 wait complete. 2024-02-02T15:40:18.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.500+0000 7f83e6e00700 1 Processor -- start 2024-02-02T15:40:18.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.501+0000 7f83e6e00700 1 -- start start 2024-02-02T15:40:18.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.501+0000 7f83e6e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83e0073170 0x7f83e0199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:18.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.501+0000 7f83e6e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83e0073a90 0x7f83e0199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:18.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.501+0000 7f83e6e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83e010d4e0 0x7f83e019dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:18.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.501+0000 7f83e6e00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83e019e1e0 con 0x7f83e0073170 2024-02-02T15:40:18.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.501+0000 7f83e6e00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83e019e350 con 0x7f83e0073a90 2024-02-02T15:40:18.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.501+0000 7f83e6e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83e019e510 con 0x7f83e010d4e0 2024-02-02T15:40:18.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.501+0000 7f83df400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83e0073a90 0x7f83e0199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:18.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.501+0000 7f83df400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83e0073a90 0x7f83e0199740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42030/0 (socket says 172.21.15.1:42030) 2024-02-02T15:40:18.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.501+0000 7f83dfe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83e0073170 0x7f83e0199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:18.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.501+0000 7f83df400700 1 -- 172.21.15.1:0/3912609337 learned_addr learned my addr 172.21.15.1:0/3912609337 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:40:18.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.501+0000 7f83e4e00700 1 --2- 172.21.15.1:0/3912609337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83e010d4e0 0x7f83e019dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:18.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.502+0000 7f83df400700 1 -- 172.21.15.1:0/3912609337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83e010d4e0 msgr2=0x7f83e019dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:18.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.502+0000 7f83df400700 1 --2- 172.21.15.1:0/3912609337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83e010d4e0 0x7f83e019dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:18.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.502+0000 7f83df400700 1 -- 172.21.15.1:0/3912609337 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83e0073170 msgr2=0x7f83e0199200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:18.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.502+0000 7f83df400700 1 --2- 172.21.15.1:0/3912609337 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83e0073170 0x7f83e0199200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:18.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.502+0000 7f83df400700 1 -- 172.21.15.1:0/3912609337 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83d00095f0 con 0x7f83e0073a90 2024-02-02T15:40:18.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.502+0000 7f83df400700 1 --2- 172.21.15.1:0/3912609337 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83e0073a90 0x7f83e0199740 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f83c800d980 tx=0x7f83c800dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:18.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.502+0000 7f83dcc00700 1 -- 172.21.15.1:0/3912609337 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c8004220 con 0x7f83e0073a90 2024-02-02T15:40:18.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.502+0000 7f83dcc00700 1 -- 172.21.15.1:0/3912609337 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f83c8007640 con 0x7f83e0073a90 2024-02-02T15:40:18.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.502+0000 7f83dcc00700 1 -- 172.21.15.1:0/3912609337 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c8005690 con 0x7f83e0073a90 2024-02-02T15:40:18.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.502+0000 7f83e6e00700 1 -- 172.21.15.1:0/3912609337 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83e019e7f0 con 0x7f83e0073a90 2024-02-02T15:40:18.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.502+0000 7f83e6e00700 1 -- 172.21.15.1:0/3912609337 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83e01a6030 con 0x7f83e0073a90 2024-02-02T15:40:18.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.504+0000 7f83e6e00700 1 -- 172.21.15.1:0/3912609337 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83e019ea10 con 0x7f83e0073a90 2024-02-02T15:40:18.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.506+0000 7f83dcc00700 1 -- 172.21.15.1:0/3912609337 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f83c8004750 con 0x7f83e0073a90 2024-02-02T15:40:18.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.507+0000 7f83dcc00700 1 --2- 172.21.15.1:0/3912609337 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f83c00a0710 0x7f83c00a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:18.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.507+0000 7f83dcc00700 1 -- 172.21.15.1:0/3912609337 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f83c80cbec0 con 0x7f83e0073a90 2024-02-02T15:40:18.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.507+0000 7f83dfe00700 1 --2- 172.21.15.1:0/3912609337 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f83c00a0710 0x7f83c00a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:18.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.508+0000 7f83dfe00700 1 --2- 172.21.15.1:0/3912609337 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f83c00a0710 0x7f83c00a2bd0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f83d000b2e0 tx=0x7f83d0005fd0 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:18.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.512+0000 7f83dcc00700 1 -- 172.21.15.1:0/3912609337 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f83e019ea10 con 0x7f83e0073a90 2024-02-02T15:40:18.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.813+0000 7f83e6e00700 1 -- 172.21.15.1:0/3912609337 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f83e01086f0 con 0x7f83e0073a90 2024-02-02T15:40:18.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.813+0000 7f83dcc00700 1 -- 172.21.15.1:0/3912609337 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+10873 (secure 0 0 0) 0x7f83c8016020 con 0x7f83e0073a90 2024-02-02T15:40:18.813 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:40:18.814 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":52,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2922240448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2922240448},{"type":"v1","addr":"172.21.15.1:6835","nonce":2922240448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2729497431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2729497431},{"type":"v1","addr":"172.21.15.1:6837","nonce":2729497431}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":52,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-02T15:24:47.761435+0000","modified":"2024-02-02T15:39:00.496919+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24529,"mds_2":24518,"mds_3":24514,"mds_4":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14517":{"gid":14517,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/1218303683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":1218303683},{"type":"v1","addr":"172.21.15.1:6839","nonce":1218303683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14529":{"gid":14529,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/2059197496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2059197496},{"type":"v1","addr":"172.21.15.1:6841","nonce":2059197496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/7648986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":7648986},{"type":"v1","addr":"172.21.15.142:6833","nonce":7648986}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/3482746094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3482746094},{"type":"v1","addr":"172.21.15.71:6833","nonce":3482746094}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/388207814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":388207814},{"type":"v1","addr":"172.21.15.71:6835","nonce":388207814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6835/1052606547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1052606547},{"type":"v1","addr":"172.21.15.142:6835","nonce":1052606547}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.142:6837/2870289019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2870289019},{"type":"v1","addr":"172.21.15.142:6837","nonce":2870289019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.71:6837/628016199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":628016199},{"type":"v1","addr":"172.21.15.71:6837","nonce":628016199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.142:6839/321675421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":321675421},{"type":"v1","addr":"172.21.15.142:6839","nonce":321675421}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/1268086493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1268086493},{"type":"v1","addr":"172.21.15.71:6839","nonce":1268086493}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-02T15:40:18.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.816+0000 7f83e6e00700 1 -- 172.21.15.1:0/3912609337 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f83c00a0710 msgr2=0x7f83c00a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:18.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.816+0000 7f83e6e00700 1 --2- 172.21.15.1:0/3912609337 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f83c00a0710 0x7f83c00a2bd0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f83d000b2e0 tx=0x7f83d0005fd0 comp rx=0 tx=0).stop 2024-02-02T15:40:18.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.816+0000 7f83e6e00700 1 -- 172.21.15.1:0/3912609337 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83e0073a90 msgr2=0x7f83e0199740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:18.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.816+0000 7f83e6e00700 1 --2- 172.21.15.1:0/3912609337 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83e0073a90 0x7f83e0199740 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f83c800d980 tx=0x7f83c800dc90 comp rx=0 tx=0).stop 2024-02-02T15:40:18.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.816+0000 7f83e6e00700 1 -- 172.21.15.1:0/3912609337 shutdown_connections 2024-02-02T15:40:18.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.816+0000 7f83e6e00700 1 --2- 172.21.15.1:0/3912609337 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f83c00a0710 0x7f83c00a2bd0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:18.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.816+0000 7f83e6e00700 1 --2- 172.21.15.1:0/3912609337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f83e010d4e0 0x7f83e019dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:18.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.816+0000 7f83e6e00700 1 --2- 172.21.15.1:0/3912609337 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f83e0073170 0x7f83e0199200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:18.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.816+0000 7f83e6e00700 1 --2- 172.21.15.1:0/3912609337 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f83e0073a90 0x7f83e0199740 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:18.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.816+0000 7f83e6e00700 1 -- 172.21.15.1:0/3912609337 >> 172.21.15.1:0/3912609337 conn(0x7f83e00fc790 msgr2=0x7f83e010f720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:18.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.816+0000 7f83e6e00700 1 -- 172.21.15.1:0/3912609337 shutdown_connections 2024-02-02T15:40:18.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:18.816+0000 7f83e6e00700 1 -- 172.21.15.1:0/3912609337 wait complete. 2024-02-02T15:40:18.817 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 52 2024-02-02T15:40:18.835 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-02-02T15:40:19.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.100+0000 7fdd5b800700 1 -- 172.21.15.1:0/3516546879 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd54073a90 msgr2=0x7fdd5410d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:19.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.100+0000 7fdd5b800700 1 --2- 172.21.15.1:0/3516546879 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd54073a90 0x7fdd5410d080 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c009990 tx=0x7fdd3c009ca0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.100+0000 7fdd5b800700 1 -- 172.21.15.1:0/3516546879 shutdown_connections 2024-02-02T15:40:19.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.100+0000 7fdd5b800700 1 --2- 172.21.15.1:0/3516546879 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdd5410d5c0 0x7fdd5410fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.100+0000 7fdd5b800700 1 --2- 172.21.15.1:0/3516546879 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdd54073170 0x7fdd54073550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.100+0000 7fdd5b800700 1 --2- 172.21.15.1:0/3516546879 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd54073a90 0x7fdd5410d080 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.100+0000 7fdd5b800700 1 -- 172.21.15.1:0/3516546879 >> 172.21.15.1:0/3516546879 conn(0x7fdd540fc810 msgr2=0x7fdd540fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:19.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.100+0000 7fdd5b800700 1 -- 172.21.15.1:0/3516546879 shutdown_connections 2024-02-02T15:40:19.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.100+0000 7fdd5b800700 1 -- 172.21.15.1:0/3516546879 wait complete. 2024-02-02T15:40:19.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.101+0000 7fdd5b800700 1 Processor -- start 2024-02-02T15:40:19.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.101+0000 7fdd5b800700 1 -- start start 2024-02-02T15:40:19.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.102+0000 7fdd5b800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdd54073170 0x7fdd541990a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:19.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.102+0000 7fdd5b800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd54073a90 0x7fdd541995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:19.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.102+0000 7fdd5b800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdd5410d5c0 0x7fdd5419da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:19.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.102+0000 7fdd5b800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5419e080 con 0x7fdd54073170 2024-02-02T15:40:19.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.102+0000 7fdd5b800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5419e1f0 con 0x7fdd5410d5c0 2024-02-02T15:40:19.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.102+0000 7fdd5b800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5419e3b0 con 0x7fdd54073a90 2024-02-02T15:40:19.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.102+0000 7fdd4be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd54073a90 0x7fdd541995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:19.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.102+0000 7fdd58e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdd54073170 0x7fdd541990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:19.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.102+0000 7fdd59800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdd5410d5c0 0x7fdd5419da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:19.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.102+0000 7fdd4be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd54073a90 0x7fdd541995e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:40602/0 (socket says 172.21.15.1:40602) 2024-02-02T15:40:19.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.102+0000 7fdd58e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdd54073170 0x7fdd541990a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45716/0 (socket says 172.21.15.1:45716) 2024-02-02T15:40:19.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.102+0000 7fdd4be00700 1 -- 172.21.15.1:0/2167933886 learned_addr learned my addr 172.21.15.1:0/2167933886 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:40:19.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.102+0000 7fdd59800700 1 -- 172.21.15.1:0/2167933886 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd54073a90 msgr2=0x7fdd541995e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:19.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.102+0000 7fdd59800700 1 --2- 172.21.15.1:0/2167933886 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd54073a90 0x7fdd541995e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.102+0000 7fdd59800700 1 -- 172.21.15.1:0/2167933886 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdd54073170 msgr2=0x7fdd541990a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:19.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.102+0000 7fdd59800700 1 --2- 172.21.15.1:0/2167933886 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdd54073170 0x7fdd541990a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.102+0000 7fdd59800700 1 -- 172.21.15.1:0/2167933886 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd44009520 con 0x7fdd5410d5c0 2024-02-02T15:40:19.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.102+0000 7fdd58e00700 1 --2- 172.21.15.1:0/2167933886 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdd54073170 0x7fdd541990a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:40:19.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.103+0000 7fdd59800700 1 --2- 172.21.15.1:0/2167933886 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdd5410d5c0 0x7fdd5419da40 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c00efc0 tx=0x7fdd4c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:19.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.103+0000 7fdd52000700 1 -- 172.21.15.1:0/2167933886 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c018070 con 0x7fdd5410d5c0 2024-02-02T15:40:19.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.103+0000 7fdd5b800700 1 -- 172.21.15.1:0/2167933886 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd3c0095f0 con 0x7fdd5410d5c0 2024-02-02T15:40:19.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.103+0000 7fdd52000700 1 -- 172.21.15.1:0/2167933886 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd4c00f040 con 0x7fdd5410d5c0 2024-02-02T15:40:19.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.103+0000 7fdd52000700 1 -- 172.21.15.1:0/2167933886 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c00cd40 con 0x7fdd5410d5c0 2024-02-02T15:40:19.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.103+0000 7fdd5b800700 1 -- 172.21.15.1:0/2167933886 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd541a5cf0 con 0x7fdd5410d5c0 2024-02-02T15:40:19.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.106+0000 7fdd52000700 1 -- 172.21.15.1:0/2167933886 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7fdd4c003a00 con 0x7fdd5410d5c0 2024-02-02T15:40:19.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.107+0000 7fdd6f31eb80 1 Processor -- start 2024-02-02T15:40:19.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.108+0000 7fdd6f31eb80 1 -- start start 2024-02-02T15:40:19.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.108+0000 7fdd52000700 1 --2- 172.21.15.1:0/2167933886 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fdd340a0690 0x7fdd340a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:19.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.108+0000 7fdd52000700 1 -- 172.21.15.1:0/2167933886 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fdd4c0cbc50 con 0x7fdd5410d5c0 2024-02-02T15:40:19.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.108+0000 7fdd4be00700 1 --2- 172.21.15.1:0/2167933886 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fdd340a0690 0x7fdd340a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:19.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.108+0000 7fdd6f31eb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564bcb62f010 0x564bcb62f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:19.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.108+0000 7fdd6f31eb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564bcb62f930 0x564bcb63a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:19.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.108+0000 7fdd6f31eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564bcb63a7e0 0x564bcb63cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:19.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.108+0000 7fdd6f31eb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bcb5a34c0 con 0x564bcb62f010 2024-02-02T15:40:19.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.108+0000 7fdd6f31eb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bcb587aa0 con 0x564bcb62f930 2024-02-02T15:40:19.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.108+0000 7fdd6f31eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bcb585460 con 0x564bcb63a7e0 2024-02-02T15:40:19.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.108+0000 7fdd59800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564bcb63a7e0 0x564bcb63cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:19.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.108+0000 7fdd58e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564bcb62f010 0x564bcb62f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:19.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.108+0000 7fdd59800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564bcb63a7e0 0x564bcb63cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:40608/0 (socket says 172.21.15.1:40608) 2024-02-02T15:40:19.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.108+0000 7fdd59800700 1 -- 172.21.15.1:0/1635864355 learned_addr learned my addr 172.21.15.1:0/1635864355 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:40:19.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.109+0000 7fdd4be00700 1 --2- 172.21.15.1:0/1635864355 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564bcb62f930 0x564bcb63a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:19.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.109+0000 7fdd59800700 1 -- 172.21.15.1:0/1635864355 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564bcb62f930 msgr2=0x564bcb63a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:19.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.109+0000 7fdd59800700 1 --2- 172.21.15.1:0/1635864355 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564bcb62f930 0x564bcb63a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.109+0000 7fdd59800700 1 -- 172.21.15.1:0/1635864355 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564bcb62f010 msgr2=0x564bcb62f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:19.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.109+0000 7fdd59800700 1 --2- 172.21.15.1:0/1635864355 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564bcb62f010 0x564bcb62f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.109+0000 7fdd59800700 1 -- 172.21.15.1:0/1635864355 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564bcb4023b0 con 0x564bcb63a7e0 2024-02-02T15:40:19.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.109+0000 7fdd59800700 1 --2- 172.21.15.1:0/1635864355 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564bcb63a7e0 0x564bcb63cbd0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c00ef90 tx=0x7fdd4c095210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1237a661be3a92ab server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:19.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.109+0000 7fdd4be00700 1 --2- 172.21.15.1:0/2167933886 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fdd340a0690 0x7fdd340a2b50 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fdd5419a6b0 tx=0x7fdd3c00bb70 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:19.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.109+0000 7fdd4a000700 1 -- 172.21.15.1:0/1635864355 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c098760 con 0x564bcb63a7e0 2024-02-02T15:40:19.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.109+0000 7fdd4a000700 1 -- 172.21.15.1:0/1635864355 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd4c01b320 con 0x564bcb63a7e0 2024-02-02T15:40:19.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.109+0000 7fdd4a000700 1 -- 172.21.15.1:0/1635864355 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c099060 con 0x564bcb63a7e0 2024-02-02T15:40:19.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.110+0000 7fdd6f31eb80 1 -- 172.21.15.1:0/1635864355 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564bcb63a7e0 msgr2=0x564bcb63cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:19.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.110+0000 7fdd6f31eb80 1 --2- 172.21.15.1:0/1635864355 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564bcb63a7e0 0x564bcb63cbd0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c00ef90 tx=0x7fdd4c095210 comp rx=0 tx=0).stop 2024-02-02T15:40:19.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.110+0000 7fdd6f31eb80 1 -- 172.21.15.1:0/1635864355 shutdown_connections 2024-02-02T15:40:19.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.110+0000 7fdd6f31eb80 1 --2- 172.21.15.1:0/1635864355 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564bcb63a7e0 0x564bcb63cbd0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.110+0000 7fdd6f31eb80 1 --2- 172.21.15.1:0/1635864355 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564bcb62f010 0x564bcb62f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.110+0000 7fdd6f31eb80 1 --2- 172.21.15.1:0/1635864355 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564bcb62f930 0x564bcb63a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.110+0000 7fdd6f31eb80 1 -- 172.21.15.1:0/1635864355 >> 172.21.15.1:0/1635864355 conn(0x564bcb58edb0 msgr2=0x564bcb62b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:19.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.110+0000 7fdd6f31eb80 1 -- 172.21.15.1:0/1635864355 shutdown_connections 2024-02-02T15:40:19.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.110+0000 7fdd6f31eb80 1 -- 172.21.15.1:0/1635864355 wait complete. 2024-02-02T15:40:19.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.111+0000 7fdd6f31eb80 1 -- start start 2024-02-02T15:40:19.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.111+0000 7fdd6f31eb80 1 Processor -- start 2024-02-02T15:40:19.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.112+0000 7fdd6f31eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564bcb62f010 0x564bcb637540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:19.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.112+0000 7fdd6f31eb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564bcb62f930 0x564bcb637a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:19.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.112+0000 7fdd6f31eb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564bcb63a7e0 0x564bcb637fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:19.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.112+0000 7fdd6f31eb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bcb580d50 con 0x564bcb63a7e0 2024-02-02T15:40:19.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.112+0000 7fdd6f31eb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bcb51cd80 con 0x564bcb62f930 2024-02-02T15:40:19.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.112+0000 7fdd6f31eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bcb51ca20 con 0x564bcb62f010 2024-02-02T15:40:19.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.112+0000 7fdd6f31eb80 10 client.0 _ll_register_callbacks cb 0x564bcb51e290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:40:19.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.112+0000 7fdd4be00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564bcb62f930 0x564bcb637a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:19.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.112+0000 7fdd4be00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564bcb62f930 0x564bcb637a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42062/0 (socket says 172.21.15.1:42062) 2024-02-02T15:40:19.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.112+0000 7fdd4be00700 1 -- 172.21.15.1:0/3635830223 learned_addr learned my addr 172.21.15.1:0/3635830223 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:40:19.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.112+0000 7fdd58e00700 1 --2- 172.21.15.1:0/3635830223 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564bcb62f010 0x564bcb637540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:19.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.112+0000 7fdd59800700 1 --2- 172.21.15.1:0/3635830223 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564bcb63a7e0 0x564bcb637fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:19.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.112+0000 7fdd4be00700 1 -- 172.21.15.1:0/3635830223 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564bcb62f010 msgr2=0x564bcb637540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:19.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.112+0000 7fdd4be00700 1 --2- 172.21.15.1:0/3635830223 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564bcb62f010 0x564bcb637540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.112+0000 7fdd4be00700 1 -- 172.21.15.1:0/3635830223 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564bcb63a7e0 msgr2=0x564bcb637fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:19.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.112+0000 7fdd4be00700 1 --2- 172.21.15.1:0/3635830223 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564bcb63a7e0 0x564bcb637fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.112+0000 7fdd4be00700 1 -- 172.21.15.1:0/3635830223 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd541a5cf0 con 0x564bcb62f930 2024-02-02T15:40:19.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.112+0000 7fdd59800700 1 --2- 172.21.15.1:0/3635830223 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564bcb63a7e0 0x564bcb637fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:40:19.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.113+0000 7fdd4be00700 1 --2- 172.21.15.1:0/3635830223 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564bcb62f930 0x564bcb637a80 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c0050c0 tx=0x7fdd3c009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99b11a440da4ccd9 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:19.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.113+0000 7fdd42a00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:40:19.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.113+0000 7fdd42a00700 1 -- 172.21.15.1:0/3635830223 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c01d070 con 0x564bcb62f930 2024-02-02T15:40:19.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.113+0000 7fdd42a00700 1 -- 172.21.15.1:0/3635830223 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fdd3c0048d0 con 0x564bcb62f930 2024-02-02T15:40:19.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.113+0000 7fdd6f31eb80 1 -- 172.21.15.1:0/3635830223 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564bcb6230f0 con 0x564bcb62f930 2024-02-02T15:40:19.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.113+0000 7fdd42a00700 1 -- 172.21.15.1:0/3635830223 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd3c004a90 con 0x564bcb62f930 2024-02-02T15:40:19.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.113+0000 7fdd6f31eb80 10 client.29123 fetch_fsmap learned FSMap version 52 2024-02-02T15:40:19.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.113+0000 7fdd6f31eb80 1 -- 172.21.15.1:0/3635830223 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x564bcb3aeb60 con 0x564bcb62f930 2024-02-02T15:40:19.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.113+0000 7fdd42a00700 1 -- 172.21.15.1:0/3635830223 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564bcb6230f0 con 0x564bcb62f930 2024-02-02T15:40:19.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.114+0000 7fdd42a00700 1 -- 172.21.15.1:0/3635830223 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 52) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fdd3c032440 con 0x564bcb62f930 2024-02-02T15:40:19.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.114+0000 7fdd6f31eb80 10 client.29123 fetch_fsmap finished waiting for FSMap version 52 2024-02-02T15:40:19.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.114+0000 7fdd6f31eb80 10 client.29123 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-02-02T15:40:19.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.114+0000 7fdd6f31eb80 20 client.29123 populate_metadata read hostname 'smithi001' 2024-02-02T15:40:19.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.114+0000 7fdd6f31eb80 1 --2- 172.21.15.1:0/3635830223 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x564bcb64a240 0x564bcb64c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:19.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.114+0000 7fdd6f31eb80 4 client.29123 mds_command: new command op to 24502 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:40:19.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.114+0000 7fdd6f31eb80 1 -- 172.21.15.1:0/3635830223 --> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564bcb579dd0 con 0x564bcb64a240 2024-02-02T15:40:19.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.114+0000 7fdd58e00700 1 --2- 172.21.15.1:0/3635830223 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x564bcb64a240 0x564bcb64c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:19.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.115+0000 7fdd58e00700 1 --2- 172.21.15.1:0/3635830223 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x564bcb64a240 0x564bcb64c660 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=83e4fb554ccf7867 server_cookie=4440b763d88e39de in_seq=0 out_seq=0 2024-02-02T15:40:19.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.115+0000 7fdd42a00700 10 client.29123 ms_handle_connect on v2:172.21.15.142:6834/1052606547 2024-02-02T15:40:19.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.115+0000 7fdd42a00700 1 -- 172.21.15.1:0/3635830223 <== mds.4 v2:172.21.15.142:6834/1052606547 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x564bcb579dd0 con 0x564bcb64a240 2024-02-02T15:40:19.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.115+0000 7fdd42a00700 10 client.29123 handle_command_reply: tid=0 2024-02-02T15:40:19.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.115+0000 7fdd6f31eb80 1 client.29123 shutdown 2024-02-02T15:40:19.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.115+0000 7fdd6f31eb80 1 -- 172.21.15.1:0/3635830223 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564bcb62f930 msgr2=0x564bcb637a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:19.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.115+0000 7fdd6f31eb80 1 --2- 172.21.15.1:0/3635830223 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564bcb62f930 0x564bcb637a80 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c0050c0 tx=0x7fdd3c009ca0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.115+0000 7fdd6f31eb80 1 -- 172.21.15.1:0/3635830223 shutdown_connections 2024-02-02T15:40:19.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.115+0000 7fdd6f31eb80 1 --2- 172.21.15.1:0/3635830223 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x564bcb64a240 0x564bcb64c660 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.115+0000 7fdd6f31eb80 1 --2- 172.21.15.1:0/3635830223 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564bcb63a7e0 0x564bcb637fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.115+0000 7fdd42a00700 0 client.29123 ms_handle_reset on v2:172.21.15.142:6834/1052606547 2024-02-02T15:40:19.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.115+0000 7fdd6f31eb80 1 --2- 172.21.15.1:0/3635830223 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564bcb62f010 0x564bcb637540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.115+0000 7fdd6f31eb80 1 --2- 172.21.15.1:0/3635830223 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564bcb62f930 0x564bcb637a80 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.115+0000 7fdd6f31eb80 1 -- 172.21.15.1:0/3635830223 >> 172.21.15.1:0/3635830223 conn(0x564bcb58edb0 msgr2=0x564bcb62afa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:19.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.116+0000 7fdd6f31eb80 1 -- 172.21.15.1:0/3635830223 shutdown_connections 2024-02-02T15:40:19.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.116+0000 7fdd6f31eb80 1 -- 172.21.15.1:0/3635830223 wait complete. 2024-02-02T15:40:19.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.116+0000 7fdd6f31eb80 20 client.29123 trim_cache size 0 max 16384 2024-02-02T15:40:19.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.130+0000 7fdd6f31eb80 1 Processor -- start 2024-02-02T15:40:19.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.130+0000 7fdd6f31eb80 1 -- start start 2024-02-02T15:40:19.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.130+0000 7fdd6f31eb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564bcb62f010 0x564bcb636ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:19.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.131+0000 7fdd6f31eb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564bcb62f930 0x564bcb645900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:19.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.131+0000 7fdd6f31eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564bcb63a7e0 0x564bcb645e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:19.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.131+0000 7fdd6f31eb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bcb4046a0 con 0x564bcb62f930 2024-02-02T15:40:19.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.131+0000 7fdd6f31eb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bcb646c40 con 0x564bcb62f010 2024-02-02T15:40:19.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.131+0000 7fdd6f31eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bcb631f40 con 0x564bcb63a7e0 2024-02-02T15:40:19.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.131+0000 7fdd58e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564bcb62f010 0x564bcb636ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:19.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.131+0000 7fdd58e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564bcb62f010 0x564bcb636ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42068/0 (socket says 172.21.15.1:42068) 2024-02-02T15:40:19.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.131+0000 7fdd58e00700 1 -- 172.21.15.1:0/1781420462 learned_addr learned my addr 172.21.15.1:0/1781420462 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:40:19.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.131+0000 7fdd59800700 1 --2- 172.21.15.1:0/1781420462 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564bcb63a7e0 0x564bcb645e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:19.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.131+0000 7fdd4be00700 1 --2- 172.21.15.1:0/1781420462 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564bcb62f930 0x564bcb645900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:19.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.131+0000 7fdd58e00700 1 -- 172.21.15.1:0/1781420462 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564bcb63a7e0 msgr2=0x564bcb645e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:19.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.131+0000 7fdd58e00700 1 --2- 172.21.15.1:0/1781420462 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564bcb63a7e0 0x564bcb645e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.131+0000 7fdd58e00700 1 -- 172.21.15.1:0/1781420462 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564bcb62f930 msgr2=0x564bcb645900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:19.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.131+0000 7fdd58e00700 1 --2- 172.21.15.1:0/1781420462 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564bcb62f930 0x564bcb645900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.131+0000 7fdd58e00700 1 -- 172.21.15.1:0/1781420462 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564bcb546ea0 con 0x564bcb62f010 2024-02-02T15:40:19.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.132+0000 7fdd58e00700 1 --2- 172.21.15.1:0/1781420462 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564bcb62f010 0x564bcb636ab0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x564bcb64cf60 tx=0x7fdd44009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77e1e147bd94ba07 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:19.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.132+0000 7fdd4aa00700 1 -- 172.21.15.1:0/1781420462 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd44017070 con 0x564bcb62f010 2024-02-02T15:40:19.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.132+0000 7fdd4aa00700 1 -- 172.21.15.1:0/1781420462 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd44018040 con 0x564bcb62f010 2024-02-02T15:40:19.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.132+0000 7fdd4aa00700 1 -- 172.21.15.1:0/1781420462 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd44003680 con 0x564bcb62f010 2024-02-02T15:40:19.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.133+0000 7fdd6f31eb80 1 -- 172.21.15.1:0/1781420462 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564bcb62f010 msgr2=0x564bcb636ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:19.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.133+0000 7fdd6f31eb80 1 --2- 172.21.15.1:0/1781420462 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564bcb62f010 0x564bcb636ab0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x564bcb64cf60 tx=0x7fdd44009260 comp rx=0 tx=0).stop 2024-02-02T15:40:19.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.133+0000 7fdd6f31eb80 1 -- 172.21.15.1:0/1781420462 shutdown_connections 2024-02-02T15:40:19.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.133+0000 7fdd6f31eb80 1 --2- 172.21.15.1:0/1781420462 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564bcb63a7e0 0x564bcb645e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.133+0000 7fdd6f31eb80 1 --2- 172.21.15.1:0/1781420462 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564bcb62f010 0x564bcb636ab0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.133+0000 7fdd6f31eb80 1 --2- 172.21.15.1:0/1781420462 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564bcb62f930 0x564bcb645900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.133+0000 7fdd6f31eb80 1 -- 172.21.15.1:0/1781420462 >> 172.21.15.1:0/1781420462 conn(0x564bcb58edb0 msgr2=0x564bcb637500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:19.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.133+0000 7fdd6f31eb80 1 -- 172.21.15.1:0/1781420462 shutdown_connections 2024-02-02T15:40:19.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.133+0000 7fdd6f31eb80 1 -- 172.21.15.1:0/1781420462 wait complete. 2024-02-02T15:40:19.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.133+0000 7fdd6f31eb80 1 -- start start 2024-02-02T15:40:19.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.134+0000 7fdd6f31eb80 1 Processor -- start 2024-02-02T15:40:19.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.134+0000 7fdd6f31eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564bcb62f010 0x564bcb640ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:19.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.134+0000 7fdd6f31eb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564bcb62f930 0x564bcb63f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:19.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.134+0000 7fdd6f31eb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564bcb63a7e0 0x564bcb63f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:19.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.134+0000 7fdd6f31eb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bcb6417a0 con 0x564bcb63a7e0 2024-02-02T15:40:19.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.134+0000 7fdd6f31eb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bcb63fc00 con 0x564bcb62f930 2024-02-02T15:40:19.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.134+0000 7fdd6f31eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564bcb63fdc0 con 0x564bcb62f010 2024-02-02T15:40:19.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.134+0000 7fdd6f31eb80 10 client.0 _ll_register_callbacks cb 0x564bcb51e290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:40:19.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.134+0000 7fdd58e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564bcb62f010 0x564bcb640ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:19.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.134+0000 7fdd4be00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564bcb62f930 0x564bcb63f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:19.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.134+0000 7fdd58e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564bcb62f010 0x564bcb640ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:40638/0 (socket says 172.21.15.1:40638) 2024-02-02T15:40:19.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.134+0000 7fdd58e00700 1 -- 172.21.15.1:0/1893173926 learned_addr learned my addr 172.21.15.1:0/1893173926 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:40:19.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.134+0000 7fdd59800700 1 --2- 172.21.15.1:0/1893173926 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564bcb63a7e0 0x564bcb63f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:19.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.135+0000 7fdd58e00700 1 -- 172.21.15.1:0/1893173926 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564bcb62f930 msgr2=0x564bcb63f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:19.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.135+0000 7fdd58e00700 1 --2- 172.21.15.1:0/1893173926 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564bcb62f930 0x564bcb63f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.135+0000 7fdd58e00700 1 -- 172.21.15.1:0/1893173926 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564bcb63a7e0 msgr2=0x564bcb63f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:19.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.135+0000 7fdd58e00700 1 --2- 172.21.15.1:0/1893173926 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564bcb63a7e0 0x564bcb63f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.135+0000 7fdd58e00700 1 -- 172.21.15.1:0/1893173926 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd44012a00 con 0x564bcb62f010 2024-02-02T15:40:19.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.135+0000 7fdd58e00700 1 --2- 172.21.15.1:0/1893173926 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564bcb62f010 0x564bcb640ed0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fdd4401fba0 tx=0x7fdd44004610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5047fa900a4a24 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:19.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.135+0000 7fdd42a00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:40:19.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.135+0000 7fdd42a00700 1 -- 172.21.15.1:0/1893173926 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd44017040 con 0x564bcb62f010 2024-02-02T15:40:19.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.135+0000 7fdd42a00700 1 -- 172.21.15.1:0/1893173926 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fdd4401f040 con 0x564bcb62f010 2024-02-02T15:40:19.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.135+0000 7fdd6f31eb80 1 -- 172.21.15.1:0/1893173926 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdd4c01b320 con 0x564bcb62f010 2024-02-02T15:40:19.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.136+0000 7fdd42a00700 1 -- 172.21.15.1:0/1893173926 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd4401f200 con 0x564bcb62f010 2024-02-02T15:40:19.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.136+0000 7fdd6f31eb80 10 client.29125 fetch_fsmap learned FSMap version 52 2024-02-02T15:40:19.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.136+0000 7fdd6f31eb80 1 -- 172.21.15.1:0/1893173926 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x564bcb406cd0 con 0x564bcb62f010 2024-02-02T15:40:19.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.136+0000 7fdd42a00700 1 -- 172.21.15.1:0/1893173926 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdd4c01b320 con 0x564bcb62f010 2024-02-02T15:40:19.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.136+0000 7fdd42a00700 1 -- 172.21.15.1:0/1893173926 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 52) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fdd44026070 con 0x564bcb62f010 2024-02-02T15:40:19.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.136+0000 7fdd6f31eb80 10 client.29125 fetch_fsmap finished waiting for FSMap version 52 2024-02-02T15:40:19.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.136+0000 7fdd6f31eb80 10 client.29125 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-02-02T15:40:19.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.136+0000 7fdd6f31eb80 20 client.29125 populate_metadata read hostname 'smithi001' 2024-02-02T15:40:19.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.136+0000 7fdd6f31eb80 1 --2- 172.21.15.1:0/1893173926 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x564bcb69cf60 0x564bcb69f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:19.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.136+0000 7fdd6f31eb80 4 client.29125 mds_command: new command op to 24502 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:40:19.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.136+0000 7fdd6f31eb80 1 -- 172.21.15.1:0/1893173926 --> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564bcb53ff50 con 0x564bcb69cf60 2024-02-02T15:40:19.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.136+0000 7fdd4be00700 1 --2- 172.21.15.1:0/1893173926 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x564bcb69cf60 0x564bcb69f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:19.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.137+0000 7fdd4be00700 1 --2- 172.21.15.1:0/1893173926 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x564bcb69cf60 0x564bcb69f350 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9557fc28317a01f1 server_cookie=b334a1413bbde672 in_seq=0 out_seq=0 2024-02-02T15:40:19.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.137+0000 7fdd42a00700 10 client.29125 ms_handle_connect on v2:172.21.15.142:6834/1052606547 2024-02-02T15:40:19.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.137+0000 7fdd42a00700 1 -- 172.21.15.1:0/1893173926 <== mds.4 v2:172.21.15.142:6834/1052606547 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564bcb53ff50 con 0x564bcb69cf60 2024-02-02T15:40:19.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.137+0000 7fdd42a00700 10 client.29125 handle_command_reply: tid=0 2024-02-02T15:40:19.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.137+0000 7fdd6f31eb80 1 client.29125 shutdown 2024-02-02T15:40:19.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.137+0000 7fdd6f31eb80 1 -- 172.21.15.1:0/1893173926 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564bcb62f010 msgr2=0x564bcb640ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:19.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.137+0000 7fdd6f31eb80 1 --2- 172.21.15.1:0/1893173926 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564bcb62f010 0x564bcb640ed0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fdd4401fba0 tx=0x7fdd44004610 comp rx=0 tx=0).stop 2024-02-02T15:40:19.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.137+0000 7fdd6f31eb80 1 -- 172.21.15.1:0/1893173926 shutdown_connections 2024-02-02T15:40:19.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.137+0000 7fdd6f31eb80 1 --2- 172.21.15.1:0/1893173926 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x564bcb69cf60 0x564bcb69f350 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.137+0000 7fdd6f31eb80 1 --2- 172.21.15.1:0/1893173926 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564bcb63a7e0 0x564bcb63f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.137+0000 7fdd6f31eb80 1 --2- 172.21.15.1:0/1893173926 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564bcb62f010 0x564bcb640ed0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.137+0000 7fdd42a00700 0 client.29125 ms_handle_reset on v2:172.21.15.142:6834/1052606547 2024-02-02T15:40:19.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.137+0000 7fdd6f31eb80 1 --2- 172.21.15.1:0/1893173926 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564bcb62f930 0x564bcb63f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.137+0000 7fdd6f31eb80 1 -- 172.21.15.1:0/1893173926 >> 172.21.15.1:0/1893173926 conn(0x564bcb58edb0 msgr2=0x564bcb632880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:19.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.137+0000 7fdd6f31eb80 1 -- 172.21.15.1:0/1893173926 shutdown_connections 2024-02-02T15:40:19.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.137+0000 7fdd6f31eb80 1 -- 172.21.15.1:0/1893173926 wait complete. 2024-02-02T15:40:19.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.138+0000 7fdd6f31eb80 20 client.29125 trim_cache size 0 max 16384 2024-02-02T15:40:19.142 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:40:19.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.138+0000 7fdd5b800700 1 -- 172.21.15.1:0/2167933886 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fdd340a0690 msgr2=0x7fdd340a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:19.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.138+0000 7fdd5b800700 1 --2- 172.21.15.1:0/2167933886 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fdd340a0690 0x7fdd340a2b50 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fdd5419a6b0 tx=0x7fdd3c00bb70 comp rx=0 tx=0).stop 2024-02-02T15:40:19.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.138+0000 7fdd5b800700 1 -- 172.21.15.1:0/2167933886 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdd5410d5c0 msgr2=0x7fdd5419da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:19.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.138+0000 7fdd5b800700 1 --2- 172.21.15.1:0/2167933886 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdd5410d5c0 0x7fdd5419da40 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c00efc0 tx=0x7fdd4c00c3a0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.138+0000 7fdd5b800700 1 -- 172.21.15.1:0/2167933886 shutdown_connections 2024-02-02T15:40:19.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.138+0000 7fdd5b800700 1 --2- 172.21.15.1:0/2167933886 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fdd340a0690 0x7fdd340a2b50 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.138+0000 7fdd5b800700 1 --2- 172.21.15.1:0/2167933886 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdd5410d5c0 0x7fdd5419da40 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.138+0000 7fdd5b800700 1 --2- 172.21.15.1:0/2167933886 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdd54073170 0x7fdd541990a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.138+0000 7fdd5b800700 1 --2- 172.21.15.1:0/2167933886 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdd54073a90 0x7fdd541995e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.138+0000 7fdd5b800700 1 -- 172.21.15.1:0/2167933886 >> 172.21.15.1:0/2167933886 conn(0x7fdd540fc810 msgr2=0x7fdd5410f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:19.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.138+0000 7fdd5b800700 1 -- 172.21.15.1:0/2167933886 shutdown_connections 2024-02-02T15:40:19.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.138+0000 7fdd5b800700 1 -- 172.21.15.1:0/2167933886 wait complete. 2024-02-02T15:40:19.156 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-02-02T15:40:19.276 INFO:tasks.workunit.client.0.smithi001.stdout: 1 717730 5.28 MB/sec execute 593 sec latency 41.720 ms 2024-02-02T15:40:19.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.371+0000 7f8b11a00700 1 -- 172.21.15.1:0/1927526116 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b0c103be0 msgr2=0x7f8b0c107c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:19.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.371+0000 7f8b11a00700 1 --2- 172.21.15.1:0/1927526116 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b0c103be0 0x7f8b0c107c20 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f8af40098f0 tx=0x7f8af4009c00 comp rx=0 tx=0).stop 2024-02-02T15:40:19.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.371+0000 7f8b11a00700 1 -- 172.21.15.1:0/1927526116 shutdown_connections 2024-02-02T15:40:19.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.371+0000 7f8b11a00700 1 --2- 172.21.15.1:0/1927526116 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8b0c108350 0x7f8b0c10fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.371+0000 7f8b11a00700 1 --2- 172.21.15.1:0/1927526116 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b0c103230 0x7f8b0c103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.371+0000 7f8b11a00700 1 --2- 172.21.15.1:0/1927526116 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b0c103be0 0x7f8b0c107c20 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.371+0000 7f8b11a00700 1 -- 172.21.15.1:0/1927526116 >> 172.21.15.1:0/1927526116 conn(0x7f8b0c0fea70 msgr2=0x7f8b0c100e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:19.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.371+0000 7f8b11a00700 1 -- 172.21.15.1:0/1927526116 shutdown_connections 2024-02-02T15:40:19.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.372+0000 7f8b11a00700 1 -- 172.21.15.1:0/1927526116 wait complete. 2024-02-02T15:40:19.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.373+0000 7f8b11a00700 1 Processor -- start 2024-02-02T15:40:19.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.373+0000 7f8b11a00700 1 -- start start 2024-02-02T15:40:19.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.374+0000 7f8b11a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8b0c103230 0x7f8b0c199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:19.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.374+0000 7f8b11a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b0c108350 0x7f8b0c1996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:19.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.374+0000 7f8b11a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b0c19db00 0x7f8b0c19df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:19.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.374+0000 7f8b11a00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0c19e580 con 0x7f8b0c103230 2024-02-02T15:40:19.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.374+0000 7f8b11a00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0c19e6f0 con 0x7f8b0c108350 2024-02-02T15:40:19.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.374+0000 7f8b11a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b0c19e8b0 con 0x7f8b0c19db00 2024-02-02T15:40:19.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.374+0000 7f8b0b400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b0c19db00 0x7f8b0c19df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:19.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.375+0000 7f8b0aa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8b0c103230 0x7f8b0c199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:19.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.375+0000 7f8b0b400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b0c19db00 0x7f8b0c19df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:40664/0 (socket says 172.21.15.1:40664) 2024-02-02T15:40:19.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.375+0000 7f8b0b400700 1 -- 172.21.15.1:0/780541095 learned_addr learned my addr 172.21.15.1:0/780541095 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:40:19.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.375+0000 7f8b0b400700 1 -- 172.21.15.1:0/780541095 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b0c108350 msgr2=0x7f8b0c1996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:19.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.375+0000 7f8b0b400700 1 --2- 172.21.15.1:0/780541095 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b0c108350 0x7f8b0c1996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.375+0000 7f8b0b400700 1 -- 172.21.15.1:0/780541095 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8b0c103230 msgr2=0x7f8b0c199160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:19.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.375+0000 7f8b0b400700 1 --2- 172.21.15.1:0/780541095 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8b0c103230 0x7f8b0c199160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.375+0000 7f8b0b400700 1 -- 172.21.15.1:0/780541095 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8af4009520 con 0x7f8b0c19db00 2024-02-02T15:40:19.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.376+0000 7f8b0b400700 1 --2- 172.21.15.1:0/780541095 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b0c19db00 0x7f8b0c19df80 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f8b04007ee0 tx=0x7f8b0400d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:19.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.376+0000 7f8b03400700 1 -- 172.21.15.1:0/780541095 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b0400ddd0 con 0x7f8b0c19db00 2024-02-02T15:40:19.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.376+0000 7f8b03400700 1 -- 172.21.15.1:0/780541095 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b0400f040 con 0x7f8b0c19db00 2024-02-02T15:40:19.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.376+0000 7f8b03400700 1 -- 172.21.15.1:0/780541095 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b04015400 con 0x7f8b0c19db00 2024-02-02T15:40:19.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.376+0000 7f8b11a00700 1 -- 172.21.15.1:0/780541095 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b0c1a5e10 con 0x7f8b0c19db00 2024-02-02T15:40:19.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.377+0000 7f8b11a00700 1 -- 172.21.15.1:0/780541095 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b0c1a6240 con 0x7f8b0c19db00 2024-02-02T15:40:19.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.382+0000 7f8b03400700 1 -- 172.21.15.1:0/780541095 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f8b04004020 con 0x7f8b0c19db00 2024-02-02T15:40:19.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.384+0000 7f8b25590b80 1 Processor -- start 2024-02-02T15:40:19.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.384+0000 7f8b25590b80 1 -- start start 2024-02-02T15:40:19.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.384+0000 7f8b03400700 1 --2- 172.21.15.1:0/780541095 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f8aec0a8f20 0x7f8aec0ab3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:19.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.384+0000 7f8b03400700 1 -- 172.21.15.1:0/780541095 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f8b040cb990 con 0x7f8b0c19db00 2024-02-02T15:40:19.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.384+0000 7f8b25590b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556c6fb97010 0x556c6fb973f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:19.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.384+0000 7f8b25590b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c6fb97930 0x556c6fba22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:19.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.384+0000 7f8b25590b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556c6fba27e0 0x556c6fba4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:19.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.384+0000 7f8b25590b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c6fb0b4c0 con 0x556c6fba27e0 2024-02-02T15:40:19.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.384+0000 7f8b25590b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c6faefaa0 con 0x556c6fb97930 2024-02-02T15:40:19.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.384+0000 7f8b25590b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c6faed460 con 0x556c6fb97010 2024-02-02T15:40:19.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.384+0000 7f8b0a000700 1 --2- 172.21.15.1:0/780541095 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f8aec0a8f20 0x7f8aec0ab3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:19.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.384+0000 7f8b0aa00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556c6fb97010 0x556c6fb973f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:19.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.384+0000 7f8b0a000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c6fb97930 0x556c6fba22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:19.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.384+0000 7f8b0a000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c6fb97930 0x556c6fba22a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42112/0 (socket says 172.21.15.1:42112) 2024-02-02T15:40:19.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.384+0000 7f8b0aa00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556c6fb97010 0x556c6fb973f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:40674/0 (socket says 172.21.15.1:40674) 2024-02-02T15:40:19.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.385+0000 7f8b0a000700 1 -- 172.21.15.1:0/3234880455 learned_addr learned my addr 172.21.15.1:0/3234880455 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:40:19.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.385+0000 7f8b0b400700 1 --2- 172.21.15.1:0/3234880455 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556c6fba27e0 0x556c6fba4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:19.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.385+0000 7f8b0a000700 1 -- 172.21.15.1:0/3234880455 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556c6fb97010 msgr2=0x556c6fb973f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:19.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.385+0000 7f8b0a000700 1 --2- 172.21.15.1:0/3234880455 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556c6fb97010 0x556c6fb973f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.385+0000 7f8b0a000700 1 -- 172.21.15.1:0/3234880455 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556c6fba27e0 msgr2=0x556c6fba4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:19.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.385+0000 7f8b0a000700 1 --2- 172.21.15.1:0/3234880455 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556c6fba27e0 0x556c6fba4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.385+0000 7f8b0a000700 1 -- 172.21.15.1:0/3234880455 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c6f96a3b0 con 0x556c6fb97930 2024-02-02T15:40:19.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.385+0000 7f8b0aa00700 1 --2- 172.21.15.1:0/3234880455 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556c6fb97010 0x556c6fb973f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:40:19.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.385+0000 7f8b0a000700 1 --2- 172.21.15.1:0/3234880455 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c6fb97930 0x556c6fba22a0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f8af4016000 tx=0x7f8af400f460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=237190ac5bdc7d5f server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:19.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.385+0000 7f8b0a000700 1 --2- 172.21.15.1:0/780541095 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f8aec0a8f20 0x7f8aec0ab3e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8af400c010 tx=0x7f8af4025000 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:19.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.386+0000 7f8afbe00700 1 -- 172.21.15.1:0/3234880455 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af400b9e0 con 0x556c6fb97930 2024-02-02T15:40:19.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.386+0000 7f8afbe00700 1 -- 172.21.15.1:0/3234880455 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8af400fbb0 con 0x556c6fb97930 2024-02-02T15:40:19.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.386+0000 7f8afbe00700 1 -- 172.21.15.1:0/3234880455 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af40038c0 con 0x556c6fb97930 2024-02-02T15:40:19.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.386+0000 7f8b25590b80 1 -- 172.21.15.1:0/3234880455 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c6fb97930 msgr2=0x556c6fba22a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:19.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.386+0000 7f8b25590b80 1 --2- 172.21.15.1:0/3234880455 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c6fb97930 0x556c6fba22a0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f8af4016000 tx=0x7f8af400f460 comp rx=0 tx=0).stop 2024-02-02T15:40:19.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.387+0000 7f8b25590b80 1 -- 172.21.15.1:0/3234880455 shutdown_connections 2024-02-02T15:40:19.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.387+0000 7f8b25590b80 1 --2- 172.21.15.1:0/3234880455 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556c6fba27e0 0x556c6fba4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.387+0000 7f8b25590b80 1 --2- 172.21.15.1:0/3234880455 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556c6fb97010 0x556c6fb973f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.387+0000 7f8b25590b80 1 --2- 172.21.15.1:0/3234880455 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c6fb97930 0x556c6fba22a0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.387+0000 7f8b25590b80 1 -- 172.21.15.1:0/3234880455 >> 172.21.15.1:0/3234880455 conn(0x556c6faf6db0 msgr2=0x556c6fb933b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:19.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.387+0000 7f8b25590b80 1 -- 172.21.15.1:0/3234880455 shutdown_connections 2024-02-02T15:40:19.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.387+0000 7f8b25590b80 1 -- 172.21.15.1:0/3234880455 wait complete. 2024-02-02T15:40:19.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.387+0000 7f8b25590b80 1 -- start start 2024-02-02T15:40:19.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.387+0000 7f8b25590b80 1 Processor -- start 2024-02-02T15:40:19.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.388+0000 7f8b25590b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556c6fb97010 0x556c6fba2210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:19.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.388+0000 7f8b25590b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c6fb97930 0x556c6fb9f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:19.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.388+0000 7f8b25590b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556c6fba27e0 0x556c6fb9fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:19.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.388+0000 7f8b25590b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c6fae8d50 con 0x556c6fb97010 2024-02-02T15:40:19.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.388+0000 7f8b25590b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c6fa84d80 con 0x556c6fb97930 2024-02-02T15:40:19.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.388+0000 7f8b25590b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c6fa84a20 con 0x556c6fba27e0 2024-02-02T15:40:19.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.388+0000 7f8b25590b80 10 client.0 _ll_register_callbacks cb 0x556c6fa86290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:40:19.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.388+0000 7f8b0a000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c6fb97930 0x556c6fb9f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:19.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.388+0000 7f8b0aa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556c6fb97010 0x556c6fba2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:19.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.389+0000 7f8b0a000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c6fb97930 0x556c6fb9f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42126/0 (socket says 172.21.15.1:42126) 2024-02-02T15:40:19.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.389+0000 7f8b0a000700 1 -- 172.21.15.1:0/276189710 learned_addr learned my addr 172.21.15.1:0/276189710 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:40:19.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.389+0000 7f8b0a000700 1 -- 172.21.15.1:0/276189710 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556c6fba27e0 msgr2=0x556c6fb9fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:19.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.389+0000 7f8b0b400700 1 --2- 172.21.15.1:0/276189710 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556c6fba27e0 0x556c6fb9fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:19.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.389+0000 7f8b0a000700 1 --2- 172.21.15.1:0/276189710 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556c6fba27e0 0x556c6fb9fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.389+0000 7f8b0a000700 1 -- 172.21.15.1:0/276189710 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556c6fb97010 msgr2=0x556c6fba2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:19.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.389+0000 7f8b0a000700 1 --2- 172.21.15.1:0/276189710 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556c6fb97010 0x556c6fba2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.389+0000 7f8b0a000700 1 -- 172.21.15.1:0/276189710 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8afc009900 con 0x556c6fb97930 2024-02-02T15:40:19.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.389+0000 7f8b0aa00700 1 --2- 172.21.15.1:0/276189710 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556c6fb97010 0x556c6fba2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:40:19.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.389+0000 7f8b0b400700 1 --2- 172.21.15.1:0/276189710 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556c6fba27e0 0x556c6fb9fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:40:19.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.389+0000 7f8b0a000700 1 --2- 172.21.15.1:0/276189710 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c6fb97930 0x556c6fb9f420 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f8af40094f0 tx=0x7f8af40092e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bdbd16af9b476eb1 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:19.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.389+0000 7f8af8c00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:40:19.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.390+0000 7f8af8c00700 1 -- 172.21.15.1:0/276189710 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af4031040 con 0x556c6fb97930 2024-02-02T15:40:19.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.390+0000 7f8b25590b80 1 -- 172.21.15.1:0/276189710 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556c6fb8b0f0 con 0x556c6fb97930 2024-02-02T15:40:19.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.390+0000 7f8af8c00700 1 -- 172.21.15.1:0/276189710 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f8af4003680 con 0x556c6fb97930 2024-02-02T15:40:19.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.390+0000 7f8af8c00700 1 -- 172.21.15.1:0/276189710 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8af4003840 con 0x556c6fb97930 2024-02-02T15:40:19.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.390+0000 7f8b25590b80 10 client.29138 fetch_fsmap learned FSMap version 52 2024-02-02T15:40:19.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.390+0000 7f8b25590b80 1 -- 172.21.15.1:0/276189710 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x7f8b0c110740 con 0x556c6fb97930 2024-02-02T15:40:19.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.390+0000 7f8af8c00700 1 -- 172.21.15.1:0/276189710 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556c6fb8b0f0 con 0x556c6fb97930 2024-02-02T15:40:19.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.390+0000 7f8af8c00700 1 -- 172.21.15.1:0/276189710 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 52) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f8af402da90 con 0x556c6fb97930 2024-02-02T15:40:19.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.390+0000 7f8b25590b80 10 client.29138 fetch_fsmap finished waiting for FSMap version 52 2024-02-02T15:40:19.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.390+0000 7f8b25590b80 10 client.29138 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-02-02T15:40:19.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.390+0000 7f8b25590b80 20 client.29138 populate_metadata read hostname 'smithi001' 2024-02-02T15:40:19.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.390+0000 7f8b25590b80 1 --2- 172.21.15.1:0/276189710 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x556c6fbb1ba0 0x556c6fbb3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:19.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.390+0000 7f8b25590b80 4 client.29138 mds_command: new command op to 24514 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:40:19.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.390+0000 7f8b25590b80 1 -- 172.21.15.1:0/276189710 --> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556c6fae1dd0 con 0x556c6fbb1ba0 2024-02-02T15:40:19.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.391+0000 7f8b0aa00700 1 --2- 172.21.15.1:0/276189710 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x556c6fbb1ba0 0x556c6fbb3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:19.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.391+0000 7f8b0aa00700 1 --2- 172.21.15.1:0/276189710 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x556c6fbb1ba0 0x556c6fbb3fc0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1d860c35a0ff898c server_cookie=1740103c80d58a7 in_seq=0 out_seq=0 2024-02-02T15:40:19.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.391+0000 7f8af8c00700 10 client.29138 ms_handle_connect on v2:172.21.15.142:6836/2870289019 2024-02-02T15:40:19.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.392+0000 7f8af8c00700 1 -- 172.21.15.1:0/276189710 <== mds.3 v2:172.21.15.142:6836/2870289019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x556c6fae1dd0 con 0x556c6fbb1ba0 2024-02-02T15:40:19.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.392+0000 7f8af8c00700 10 client.29138 handle_command_reply: tid=0 2024-02-02T15:40:19.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.392+0000 7f8b25590b80 1 client.29138 shutdown 2024-02-02T15:40:19.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.392+0000 7f8b25590b80 1 -- 172.21.15.1:0/276189710 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c6fb97930 msgr2=0x556c6fb9f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:19.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.392+0000 7f8b25590b80 1 --2- 172.21.15.1:0/276189710 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c6fb97930 0x556c6fb9f420 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f8af40094f0 tx=0x7f8af40092e0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.392+0000 7f8b25590b80 1 -- 172.21.15.1:0/276189710 shutdown_connections 2024-02-02T15:40:19.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.392+0000 7f8b25590b80 1 --2- 172.21.15.1:0/276189710 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556c6fba27e0 0x556c6fb9fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.392+0000 7f8b25590b80 1 --2- 172.21.15.1:0/276189710 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x556c6fbb1ba0 0x556c6fbb3fc0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.392+0000 7f8b25590b80 1 --2- 172.21.15.1:0/276189710 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556c6fb97010 0x556c6fba2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.392+0000 7f8b25590b80 1 --2- 172.21.15.1:0/276189710 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c6fb97930 0x556c6fb9f420 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.392+0000 7f8af8c00700 0 client.29138 ms_handle_reset on v2:172.21.15.142:6836/2870289019 2024-02-02T15:40:19.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.392+0000 7f8b25590b80 1 -- 172.21.15.1:0/276189710 >> 172.21.15.1:0/276189710 conn(0x556c6faf6db0 msgr2=0x556c6fb92fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:19.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.392+0000 7f8b25590b80 1 -- 172.21.15.1:0/276189710 shutdown_connections 2024-02-02T15:40:19.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.393+0000 7f8b25590b80 1 -- 172.21.15.1:0/276189710 wait complete. 2024-02-02T15:40:19.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.393+0000 7f8b25590b80 20 client.29138 trim_cache size 0 max 16384 2024-02-02T15:40:19.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.408+0000 7f8b25590b80 1 Processor -- start 2024-02-02T15:40:19.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.408+0000 7f8b25590b80 1 -- start start 2024-02-02T15:40:19.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.408+0000 7f8b25590b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c6fb97010 0x556c6fb9f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:19.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.408+0000 7f8b25590b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556c6fb97930 0x556c6fba1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:19.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.409+0000 7f8b25590b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556c6fba27e0 0x556c6fb9c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:19.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.409+0000 7f8b25590b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c6f96c6a0 con 0x556c6fb97930 2024-02-02T15:40:19.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.409+0000 7f8b25590b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c6fb94a20 con 0x556c6fb97010 2024-02-02T15:40:19.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.409+0000 7f8b25590b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c6fb94240 con 0x556c6fba27e0 2024-02-02T15:40:19.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.409+0000 7f8b0b400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556c6fba27e0 0x556c6fb9c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:19.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.409+0000 7f8b0aa00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c6fb97010 0x556c6fb9f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:19.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.409+0000 7f8b0a000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556c6fb97930 0x556c6fba1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:19.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.409+0000 7f8b0aa00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c6fb97010 0x556c6fb9f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42136/0 (socket says 172.21.15.1:42136) 2024-02-02T15:40:19.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.409+0000 7f8b0aa00700 1 -- 172.21.15.1:0/2171223851 learned_addr learned my addr 172.21.15.1:0/2171223851 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:40:19.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.409+0000 7f8b0b400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556c6fba27e0 0x556c6fb9c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:40688/0 (socket says 172.21.15.1:40688) 2024-02-02T15:40:19.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.409+0000 7f8b0a000700 1 -- 172.21.15.1:0/2171223851 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556c6fba27e0 msgr2=0x556c6fb9c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:19.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.409+0000 7f8b0a000700 1 --2- 172.21.15.1:0/2171223851 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556c6fba27e0 0x556c6fb9c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.409+0000 7f8b0a000700 1 -- 172.21.15.1:0/2171223851 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c6fb97010 msgr2=0x556c6fb9f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:19.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.409+0000 7f8b0a000700 1 --2- 172.21.15.1:0/2171223851 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c6fb97010 0x556c6fb9f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.409+0000 7f8b0a000700 1 -- 172.21.15.1:0/2171223851 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c6f916b60 con 0x556c6fb97930 2024-02-02T15:40:19.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.409+0000 7f8b0aa00700 1 --2- 172.21.15.1:0/2171223851 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c6fb97010 0x556c6fb9f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:40:19.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.409+0000 7f8b0b400700 1 --2- 172.21.15.1:0/2171223851 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556c6fba27e0 0x556c6fb9c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:40:19.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.409+0000 7f8b0a000700 1 --2- 172.21.15.1:0/2171223851 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556c6fb97930 0x556c6fba1d40 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f8af40098c0 tx=0x7f8af400b340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62d28ac04dde091c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:19.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.410+0000 7f8b00c00700 1 -- 172.21.15.1:0/2171223851 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af4031040 con 0x556c6fb97930 2024-02-02T15:40:19.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.410+0000 7f8b00c00700 1 -- 172.21.15.1:0/2171223851 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8af40057e0 con 0x556c6fb97930 2024-02-02T15:40:19.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.410+0000 7f8b00c00700 1 -- 172.21.15.1:0/2171223851 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af402d6a0 con 0x556c6fb97930 2024-02-02T15:40:19.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.410+0000 7f8b25590b80 1 -- 172.21.15.1:0/2171223851 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556c6fb97930 msgr2=0x556c6fba1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:19.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.410+0000 7f8b25590b80 1 --2- 172.21.15.1:0/2171223851 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556c6fb97930 0x556c6fba1d40 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f8af40098c0 tx=0x7f8af400b340 comp rx=0 tx=0).stop 2024-02-02T15:40:19.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.411+0000 7f8b25590b80 1 -- 172.21.15.1:0/2171223851 shutdown_connections 2024-02-02T15:40:19.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.411+0000 7f8b25590b80 1 --2- 172.21.15.1:0/2171223851 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556c6fba27e0 0x556c6fb9c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.411+0000 7f8b25590b80 1 --2- 172.21.15.1:0/2171223851 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c6fb97010 0x556c6fb9f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.411+0000 7f8b25590b80 1 --2- 172.21.15.1:0/2171223851 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556c6fb97930 0x556c6fba1d40 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.411+0000 7f8b25590b80 1 -- 172.21.15.1:0/2171223851 >> 172.21.15.1:0/2171223851 conn(0x556c6faf6db0 msgr2=0x556c6fb9ecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:19.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.411+0000 7f8b25590b80 1 -- 172.21.15.1:0/2171223851 shutdown_connections 2024-02-02T15:40:19.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.411+0000 7f8b25590b80 1 -- 172.21.15.1:0/2171223851 wait complete. 2024-02-02T15:40:19.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.411+0000 7f8b25590b80 1 -- start start 2024-02-02T15:40:19.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.411+0000 7f8b25590b80 1 Processor -- start 2024-02-02T15:40:19.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.412+0000 7f8b25590b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c6fb97010 0x556c6fba6f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:19.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.412+0000 7f8b25590b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556c6fb97930 0x556c6fba7450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:19.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.412+0000 7f8b25590b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556c6fba27e0 0x556c6fbfc700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:19.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.412+0000 7f8b25590b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c6fbfcc40 con 0x556c6fb97930 2024-02-02T15:40:19.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.412+0000 7f8b25590b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c6fbfcdb0 con 0x556c6fb97010 2024-02-02T15:40:19.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.412+0000 7f8b25590b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c6fbfcf70 con 0x556c6fba27e0 2024-02-02T15:40:19.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.412+0000 7f8b25590b80 10 client.0 _ll_register_callbacks cb 0x556c6fa86290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:40:19.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.412+0000 7f8b0aa00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c6fb97010 0x556c6fba6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:19.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.412+0000 7f8b0a000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556c6fb97930 0x556c6fba7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:19.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.412+0000 7f8b0b400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556c6fba27e0 0x556c6fbfc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:19.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.412+0000 7f8b0aa00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c6fb97010 0x556c6fba6f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42152/0 (socket says 172.21.15.1:42152) 2024-02-02T15:40:19.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.412+0000 7f8b0aa00700 1 -- 172.21.15.1:0/2583763337 learned_addr learned my addr 172.21.15.1:0/2583763337 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:40:19.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.412+0000 7f8b0aa00700 1 -- 172.21.15.1:0/2583763337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556c6fba27e0 msgr2=0x556c6fbfc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:19.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.412+0000 7f8b0aa00700 1 --2- 172.21.15.1:0/2583763337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556c6fba27e0 0x556c6fbfc700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.412+0000 7f8b0aa00700 1 -- 172.21.15.1:0/2583763337 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556c6fb97930 msgr2=0x556c6fba7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:19.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.412+0000 7f8b0aa00700 1 --2- 172.21.15.1:0/2583763337 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556c6fb97930 0x556c6fba7450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.413+0000 7f8b0aa00700 1 -- 172.21.15.1:0/2583763337 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8afc009d90 con 0x556c6fb97010 2024-02-02T15:40:19.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.413+0000 7f8b0aa00700 1 --2- 172.21.15.1:0/2583763337 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c6fb97010 0x556c6fba6f10 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f8afc01f960 tx=0x7f8afc01f540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=511cdb57b6968797 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:19.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.413+0000 7f8af8c00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:40:19.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.413+0000 7f8af8c00700 1 -- 172.21.15.1:0/2583763337 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8afc004520 con 0x556c6fb97010 2024-02-02T15:40:19.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.413+0000 7f8af8c00700 1 -- 172.21.15.1:0/2583763337 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f8afc010730 con 0x556c6fb97010 2024-02-02T15:40:19.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.413+0000 7f8b25590b80 1 -- 172.21.15.1:0/2583763337 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8af400fbb0 con 0x556c6fb97010 2024-02-02T15:40:19.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.413+0000 7f8af8c00700 1 -- 172.21.15.1:0/2583763337 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8afc0108f0 con 0x556c6fb97010 2024-02-02T15:40:19.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.413+0000 7f8b25590b80 10 client.29144 fetch_fsmap learned FSMap version 52 2024-02-02T15:40:19.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.413+0000 7f8b25590b80 1 -- 172.21.15.1:0/2583763337 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x7f8b0c1a6240 con 0x556c6fb97010 2024-02-02T15:40:19.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.414+0000 7f8af8c00700 1 -- 172.21.15.1:0/2583763337 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8af400fbb0 con 0x556c6fb97010 2024-02-02T15:40:19.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.414+0000 7f8af8c00700 1 -- 172.21.15.1:0/2583763337 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 52) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f8afc00f3c0 con 0x556c6fb97010 2024-02-02T15:40:19.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.414+0000 7f8b25590b80 10 client.29144 fetch_fsmap finished waiting for FSMap version 52 2024-02-02T15:40:19.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.414+0000 7f8b25590b80 10 client.29144 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-02-02T15:40:19.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.414+0000 7f8b25590b80 20 client.29144 populate_metadata read hostname 'smithi001' 2024-02-02T15:40:19.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.414+0000 7f8b25590b80 1 --2- 172.21.15.1:0/2583763337 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x556c6fc046e0 0x556c6fc06b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:19.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.414+0000 7f8b25590b80 4 client.29144 mds_command: new command op to 24514 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:40:19.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.414+0000 7f8b25590b80 1 -- 172.21.15.1:0/2583763337 --> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556c6faa7f50 con 0x556c6fc046e0 2024-02-02T15:40:19.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.414+0000 7f8b0a000700 1 --2- 172.21.15.1:0/2583763337 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x556c6fc046e0 0x556c6fc06b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:19.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.415+0000 7f8b0a000700 1 --2- 172.21.15.1:0/2583763337 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x556c6fc046e0 0x556c6fc06b00 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9ad862117cbdb7ef server_cookie=6c2c910c931d0a51 in_seq=0 out_seq=0 2024-02-02T15:40:19.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.415+0000 7f8af8c00700 10 client.29144 ms_handle_connect on v2:172.21.15.142:6836/2870289019 2024-02-02T15:40:19.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.415+0000 7f8af8c00700 1 -- 172.21.15.1:0/2583763337 <== mds.3 v2:172.21.15.142:6836/2870289019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556c6faa7f50 con 0x556c6fc046e0 2024-02-02T15:40:19.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.415+0000 7f8af8c00700 10 client.29144 handle_command_reply: tid=0 2024-02-02T15:40:19.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.415+0000 7f8b25590b80 1 client.29144 shutdown 2024-02-02T15:40:19.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.415+0000 7f8b25590b80 1 -- 172.21.15.1:0/2583763337 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c6fb97010 msgr2=0x556c6fba6f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:19.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.415+0000 7f8b25590b80 1 --2- 172.21.15.1:0/2583763337 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c6fb97010 0x556c6fba6f10 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f8afc01f960 tx=0x7f8afc01f540 comp rx=0 tx=0).stop 2024-02-02T15:40:19.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.415+0000 7f8b25590b80 1 -- 172.21.15.1:0/2583763337 shutdown_connections 2024-02-02T15:40:19.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.415+0000 7f8b25590b80 1 --2- 172.21.15.1:0/2583763337 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556c6fba27e0 0x556c6fbfc700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.415+0000 7f8b25590b80 1 --2- 172.21.15.1:0/2583763337 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556c6fb97010 0x556c6fba6f10 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.415+0000 7f8b25590b80 1 --2- 172.21.15.1:0/2583763337 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x556c6fc046e0 0x556c6fc06b00 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.415+0000 7f8b25590b80 1 --2- 172.21.15.1:0/2583763337 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556c6fb97930 0x556c6fba7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.415+0000 7f8b25590b80 1 -- 172.21.15.1:0/2583763337 >> 172.21.15.1:0/2583763337 conn(0x556c6faf6db0 msgr2=0x556c6fb92e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:19.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.415+0000 7f8af8c00700 0 client.29144 ms_handle_reset on v2:172.21.15.142:6836/2870289019 2024-02-02T15:40:19.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.415+0000 7f8b25590b80 1 -- 172.21.15.1:0/2583763337 shutdown_connections 2024-02-02T15:40:19.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.415+0000 7f8b25590b80 1 -- 172.21.15.1:0/2583763337 wait complete. 2024-02-02T15:40:19.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.416+0000 7f8b25590b80 20 client.29144 trim_cache size 0 max 16384 2024-02-02T15:40:19.419 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:40:19.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.416+0000 7f8b11a00700 1 -- 172.21.15.1:0/780541095 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f8aec0a8f20 msgr2=0x7f8aec0ab3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:19.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.416+0000 7f8b11a00700 1 --2- 172.21.15.1:0/780541095 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f8aec0a8f20 0x7f8aec0ab3e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8af400c010 tx=0x7f8af4025000 comp rx=0 tx=0).stop 2024-02-02T15:40:19.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.416+0000 7f8b11a00700 1 -- 172.21.15.1:0/780541095 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b0c19db00 msgr2=0x7f8b0c19df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:19.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.416+0000 7f8b11a00700 1 --2- 172.21.15.1:0/780541095 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b0c19db00 0x7f8b0c19df80 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f8b04007ee0 tx=0x7f8b0400d3b0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.416+0000 7f8b11a00700 1 -- 172.21.15.1:0/780541095 shutdown_connections 2024-02-02T15:40:19.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.416+0000 7f8b11a00700 1 --2- 172.21.15.1:0/780541095 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f8aec0a8f20 0x7f8aec0ab3e0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.416+0000 7f8b11a00700 1 --2- 172.21.15.1:0/780541095 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8b0c19db00 0x7f8b0c19df80 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.416+0000 7f8b11a00700 1 --2- 172.21.15.1:0/780541095 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8b0c103230 0x7f8b0c199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.416+0000 7f8b11a00700 1 --2- 172.21.15.1:0/780541095 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8b0c108350 0x7f8b0c1996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.416+0000 7f8b11a00700 1 -- 172.21.15.1:0/780541095 >> 172.21.15.1:0/780541095 conn(0x7f8b0c0fea70 msgr2=0x7f8b0c0ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:19.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.417+0000 7f8b11a00700 1 -- 172.21.15.1:0/780541095 shutdown_connections 2024-02-02T15:40:19.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.417+0000 7f8b11a00700 1 -- 172.21.15.1:0/780541095 wait complete. 2024-02-02T15:40:19.435 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-02T15:40:19.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.654+0000 7f8214000700 1 -- 172.21.15.1:0/2349457304 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f820c1031f0 msgr2=0x7f820c1035d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:19.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.654+0000 7f8214000700 1 --2- 172.21.15.1:0/2349457304 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f820c1031f0 0x7f820c1035d0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f81fc009990 tx=0x7f81fc009ca0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.655+0000 7f8214000700 1 -- 172.21.15.1:0/2349457304 shutdown_connections 2024-02-02T15:40:19.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.655+0000 7f8214000700 1 --2- 172.21.15.1:0/2349457304 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f820c108310 0x7f820c10fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.655+0000 7f8214000700 1 --2- 172.21.15.1:0/2349457304 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f820c1031f0 0x7f820c1035d0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.655+0000 7f8214000700 1 --2- 172.21.15.1:0/2349457304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f820c103ba0 0x7f820c107be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.655+0000 7f8214000700 1 -- 172.21.15.1:0/2349457304 >> 172.21.15.1:0/2349457304 conn(0x7f820c0fea70 msgr2=0x7f820c100e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:19.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.655+0000 7f8214000700 1 -- 172.21.15.1:0/2349457304 shutdown_connections 2024-02-02T15:40:19.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.655+0000 7f8214000700 1 -- 172.21.15.1:0/2349457304 wait complete. 2024-02-02T15:40:19.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.656+0000 7f8214000700 1 Processor -- start 2024-02-02T15:40:19.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.656+0000 7f8214000700 1 -- start start 2024-02-02T15:40:19.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.656+0000 7f8214000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f820c1031f0 0x7f820c1990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:19.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.657+0000 7f8214000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f820c108310 0x7f820c199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:19.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.657+0000 7f8214000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f820c19da70 0x7f820c19def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:19.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.657+0000 7f8214000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f820c19e4f0 con 0x7f820c1031f0 2024-02-02T15:40:19.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.657+0000 7f8214000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f820c19e660 con 0x7f820c108310 2024-02-02T15:40:19.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.657+0000 7f8214000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f820c19e820 con 0x7f820c19da70 2024-02-02T15:40:19.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.657+0000 7f8211600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f820c1031f0 0x7f820c1990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:19.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.657+0000 7f8211600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f820c1031f0 0x7f820c1990d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45798/0 (socket says 172.21.15.1:45798) 2024-02-02T15:40:19.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.657+0000 7f8212000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f820c19da70 0x7f820c19def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:19.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.657+0000 7f8211600700 1 -- 172.21.15.1:0/1275432680 learned_addr learned my addr 172.21.15.1:0/1275432680 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:40:19.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.657+0000 7f8210c00700 1 --2- 172.21.15.1:0/1275432680 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f820c108310 0x7f820c199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:19.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.657+0000 7f8212000700 1 -- 172.21.15.1:0/1275432680 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f820c108310 msgr2=0x7f820c199610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:19.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.657+0000 7f8212000700 1 --2- 172.21.15.1:0/1275432680 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f820c108310 0x7f820c199610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.657+0000 7f8212000700 1 -- 172.21.15.1:0/1275432680 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f820c1031f0 msgr2=0x7f820c1990d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:19.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.657+0000 7f8212000700 1 --2- 172.21.15.1:0/1275432680 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f820c1031f0 0x7f820c1990d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.657+0000 7f8212000700 1 -- 172.21.15.1:0/1275432680 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81fc0095f0 con 0x7f820c19da70 2024-02-02T15:40:19.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.657+0000 7f8211600700 1 --2- 172.21.15.1:0/1275432680 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f820c1031f0 0x7f820c1990d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:40:19.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.658+0000 7f8212000700 1 --2- 172.21.15.1:0/1275432680 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f820c19da70 0x7f820c19def0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f820400efc0 tx=0x7f820400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:19.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.658+0000 7f820a000700 1 -- 172.21.15.1:0/1275432680 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8204018070 con 0x7f820c19da70 2024-02-02T15:40:19.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.658+0000 7f8214000700 1 -- 172.21.15.1:0/1275432680 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f820c1a5d20 con 0x7f820c19da70 2024-02-02T15:40:19.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.658+0000 7f820a000700 1 -- 172.21.15.1:0/1275432680 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f820400f040 con 0x7f820c19da70 2024-02-02T15:40:19.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.658+0000 7f820a000700 1 -- 172.21.15.1:0/1275432680 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f820400cd40 con 0x7f820c19da70 2024-02-02T15:40:19.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.658+0000 7f8214000700 1 -- 172.21.15.1:0/1275432680 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f820c1a6250 con 0x7f820c19da70 2024-02-02T15:40:19.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.661+0000 7f820a000700 1 -- 172.21.15.1:0/1275432680 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f8204003a00 con 0x7f820c19da70 2024-02-02T15:40:19.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.663+0000 7f8227c14b80 1 Processor -- start 2024-02-02T15:40:19.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.663+0000 7f8227c14b80 1 -- start start 2024-02-02T15:40:19.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.664+0000 7f8227c14b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b0d659010 0x556b0d6593f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:19.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.664+0000 7f8227c14b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556b0d659930 0x556b0d6642a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:19.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.664+0000 7f820a000700 1 --2- 172.21.15.1:0/1275432680 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f81ec0a0690 0x7f81ec0a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:19.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.664+0000 7f8227c14b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b0d6647e0 0x556b0d666bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:19.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.664+0000 7f8227c14b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b0d5cd4c0 con 0x556b0d659930 2024-02-02T15:40:19.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.664+0000 7f8227c14b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b0d5b1aa0 con 0x556b0d6647e0 2024-02-02T15:40:19.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.664+0000 7f8227c14b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b0d5af460 con 0x556b0d659010 2024-02-02T15:40:19.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.664+0000 7f820a000700 1 -- 172.21.15.1:0/1275432680 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f82040cbc50 con 0x7f820c19da70 2024-02-02T15:40:19.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.664+0000 7f8210c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b0d659010 0x556b0d6593f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:19.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.664+0000 7f8211600700 1 --2- 172.21.15.1:0/1275432680 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f81ec0a0690 0x7f81ec0a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:19.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.664+0000 7f8210c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b0d659010 0x556b0d6593f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:40718/0 (socket says 172.21.15.1:40718) 2024-02-02T15:40:19.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.664+0000 7f8210c00700 1 -- 172.21.15.1:0/1140777015 learned_addr learned my addr 172.21.15.1:0/1140777015 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:40:19.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.664+0000 7f8210c00700 1 --2- 172.21.15.1:0/1140777015 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556b0d659930 0x556b0d6642a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:19.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.664+0000 7f8212000700 1 --2- 172.21.15.1:0/1140777015 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b0d6647e0 0x556b0d666bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:19.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.664+0000 7f8212000700 1 -- 172.21.15.1:0/1140777015 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b0d659010 msgr2=0x556b0d6593f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:19.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.664+0000 7f8212000700 1 --2- 172.21.15.1:0/1140777015 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b0d659010 0x556b0d6593f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.664+0000 7f8212000700 1 -- 172.21.15.1:0/1140777015 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556b0d659930 msgr2=0x556b0d6642a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:19.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.664+0000 7f8212000700 1 --2- 172.21.15.1:0/1140777015 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556b0d659930 0x556b0d6642a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.665+0000 7f8212000700 1 -- 172.21.15.1:0/1140777015 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556b0d42c3b0 con 0x556b0d6647e0 2024-02-02T15:40:19.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.665+0000 7f8212000700 1 --2- 172.21.15.1:0/1140777015 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b0d6647e0 0x556b0d666bd0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f82040c8f70 tx=0x7f8204094af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d575096017a44131 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:19.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.665+0000 7f8211600700 1 --2- 172.21.15.1:0/1275432680 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f81ec0a0690 0x7f81ec0a2b50 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f81fc006010 tx=0x7f81fc00b260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:19.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.665+0000 7f8202600700 1 -- 172.21.15.1:0/1140777015 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8204098760 con 0x556b0d6647e0 2024-02-02T15:40:19.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.665+0000 7f8202600700 1 -- 172.21.15.1:0/1140777015 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f820401b320 con 0x556b0d6647e0 2024-02-02T15:40:19.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.665+0000 7f8202600700 1 -- 172.21.15.1:0/1140777015 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8204098e70 con 0x556b0d6647e0 2024-02-02T15:40:19.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.666+0000 7f8227c14b80 1 -- 172.21.15.1:0/1140777015 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b0d6647e0 msgr2=0x556b0d666bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:19.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.666+0000 7f8227c14b80 1 --2- 172.21.15.1:0/1140777015 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b0d6647e0 0x556b0d666bd0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f82040c8f70 tx=0x7f8204094af0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.666+0000 7f8227c14b80 1 -- 172.21.15.1:0/1140777015 shutdown_connections 2024-02-02T15:40:19.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.666+0000 7f8227c14b80 1 --2- 172.21.15.1:0/1140777015 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b0d6647e0 0x556b0d666bd0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.666+0000 7f8227c14b80 1 --2- 172.21.15.1:0/1140777015 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b0d659010 0x556b0d6593f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.666+0000 7f8227c14b80 1 --2- 172.21.15.1:0/1140777015 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556b0d659930 0x556b0d6642a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.666+0000 7f8227c14b80 1 -- 172.21.15.1:0/1140777015 >> 172.21.15.1:0/1140777015 conn(0x556b0d5b8db0 msgr2=0x556b0d6553b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:19.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.666+0000 7f8227c14b80 1 -- 172.21.15.1:0/1140777015 shutdown_connections 2024-02-02T15:40:19.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.666+0000 7f8227c14b80 1 -- 172.21.15.1:0/1140777015 wait complete. 2024-02-02T15:40:19.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.667+0000 7f8227c14b80 1 -- start start 2024-02-02T15:40:19.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.667+0000 7f8227c14b80 1 Processor -- start 2024-02-02T15:40:19.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.667+0000 7f8227c14b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b0d659010 0x556b0d664180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:19.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.667+0000 7f8227c14b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556b0d659930 0x556b0d661390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:19.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.667+0000 7f8227c14b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b0d6647e0 0x556b0d661950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:19.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.667+0000 7f8227c14b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b0d5aad50 con 0x556b0d659930 2024-02-02T15:40:19.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.667+0000 7f8227c14b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b0d546d80 con 0x556b0d659010 2024-02-02T15:40:19.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.667+0000 7f8227c14b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b0d546a20 con 0x556b0d6647e0 2024-02-02T15:40:19.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.667+0000 7f8227c14b80 10 client.0 _ll_register_callbacks cb 0x556b0d548290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:40:19.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.668+0000 7f8211600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b0d659010 0x556b0d664180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:19.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.668+0000 7f8212000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b0d6647e0 0x556b0d661950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:19.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.668+0000 7f8211600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b0d659010 0x556b0d664180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42174/0 (socket says 172.21.15.1:42174) 2024-02-02T15:40:19.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.668+0000 7f8211600700 1 -- 172.21.15.1:0/892730494 learned_addr learned my addr 172.21.15.1:0/892730494 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:40:19.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.668+0000 7f8210c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556b0d659930 0x556b0d661390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:19.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.668+0000 7f8212000700 1 -- 172.21.15.1:0/892730494 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b0d659010 msgr2=0x556b0d664180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:19.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.668+0000 7f8212000700 1 --2- 172.21.15.1:0/892730494 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b0d659010 0x556b0d664180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.668+0000 7f8212000700 1 -- 172.21.15.1:0/892730494 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556b0d659930 msgr2=0x556b0d661390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:19.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.668+0000 7f8212000700 1 --2- 172.21.15.1:0/892730494 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556b0d659930 0x556b0d661390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.668+0000 7f8212000700 1 -- 172.21.15.1:0/892730494 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f820c1a6250 con 0x556b0d6647e0 2024-02-02T15:40:19.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.668+0000 7f8212000700 1 --2- 172.21.15.1:0/892730494 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b0d6647e0 0x556b0d661950 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f8204098b00 tx=0x7f82040c9450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8db5758ed19e853d server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:19.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.668+0000 7f81faa00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:40:19.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.668+0000 7f81faa00700 1 -- 172.21.15.1:0/892730494 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8204004ad0 con 0x556b0d6647e0 2024-02-02T15:40:19.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.669+0000 7f81faa00700 1 -- 172.21.15.1:0/892730494 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f8204099500 con 0x556b0d6647e0 2024-02-02T15:40:19.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.669+0000 7f8227c14b80 1 -- 172.21.15.1:0/892730494 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556b0d64d0f0 con 0x556b0d6647e0 2024-02-02T15:40:19.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.669+0000 7f81faa00700 1 -- 172.21.15.1:0/892730494 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f820409cc30 con 0x556b0d6647e0 2024-02-02T15:40:19.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.669+0000 7f8227c14b80 10 client.29152 fetch_fsmap learned FSMap version 52 2024-02-02T15:40:19.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.669+0000 7f8227c14b80 1 -- 172.21.15.1:0/892730494 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x556b0d3d8b60 con 0x556b0d6647e0 2024-02-02T15:40:19.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.669+0000 7f81faa00700 1 -- 172.21.15.1:0/892730494 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556b0d64d0f0 con 0x556b0d6647e0 2024-02-02T15:40:19.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.669+0000 7f81faa00700 1 -- 172.21.15.1:0/892730494 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 52) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f82040982f0 con 0x556b0d6647e0 2024-02-02T15:40:19.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.669+0000 7f8227c14b80 10 client.29152 fetch_fsmap finished waiting for FSMap version 52 2024-02-02T15:40:19.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.669+0000 7f8227c14b80 10 client.29152 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-02-02T15:40:19.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.669+0000 7f8227c14b80 20 client.29152 populate_metadata read hostname 'smithi001' 2024-02-02T15:40:19.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.669+0000 7f8227c14b80 1 --2- 172.21.15.1:0/892730494 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x556b0d673d10 0x556b0d676100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:19.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.669+0000 7f8227c14b80 4 client.29152 mds_command: new command op to 24518 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:40:19.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.669+0000 7f8227c14b80 1 -- 172.21.15.1:0/892730494 --> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556b0d5a3dd0 con 0x556b0d673d10 2024-02-02T15:40:19.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.670+0000 7f8211600700 1 --2- 172.21.15.1:0/892730494 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x556b0d673d10 0x556b0d676100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:19.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.670+0000 7f8211600700 1 --2- 172.21.15.1:0/892730494 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x556b0d673d10 0x556b0d676100 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2dd2ad258d80efa4 server_cookie=4068bd0937038844 in_seq=0 out_seq=0 2024-02-02T15:40:19.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.670+0000 7f81faa00700 10 client.29152 ms_handle_connect on v2:172.21.15.71:6836/628016199 2024-02-02T15:40:19.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.671+0000 7f81faa00700 1 -- 172.21.15.1:0/892730494 <== mds.2 v2:172.21.15.71:6836/628016199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x556b0d5a3dd0 con 0x556b0d673d10 2024-02-02T15:40:19.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.671+0000 7f81faa00700 10 client.29152 handle_command_reply: tid=0 2024-02-02T15:40:19.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.671+0000 7f8227c14b80 1 client.29152 shutdown 2024-02-02T15:40:19.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.672+0000 7f8227c14b80 1 -- 172.21.15.1:0/892730494 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b0d6647e0 msgr2=0x556b0d661950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:19.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.672+0000 7f8227c14b80 1 --2- 172.21.15.1:0/892730494 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b0d6647e0 0x556b0d661950 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f8204098b00 tx=0x7f82040c9450 comp rx=0 tx=0).stop 2024-02-02T15:40:19.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.672+0000 7f8227c14b80 1 -- 172.21.15.1:0/892730494 shutdown_connections 2024-02-02T15:40:19.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.672+0000 7f8227c14b80 1 --2- 172.21.15.1:0/892730494 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x556b0d673d10 0x556b0d676100 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.672+0000 7f8227c14b80 1 --2- 172.21.15.1:0/892730494 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b0d6647e0 0x556b0d661950 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.672+0000 7f8227c14b80 1 --2- 172.21.15.1:0/892730494 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b0d659010 0x556b0d664180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.672+0000 7f8227c14b80 1 --2- 172.21.15.1:0/892730494 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556b0d659930 0x556b0d661390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.672+0000 7f81faa00700 0 client.29152 ms_handle_reset on v2:172.21.15.71:6836/628016199 2024-02-02T15:40:19.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.672+0000 7f8227c14b80 1 -- 172.21.15.1:0/892730494 >> 172.21.15.1:0/892730494 conn(0x556b0d5b8db0 msgr2=0x556b0d654fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:19.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.672+0000 7f8227c14b80 1 -- 172.21.15.1:0/892730494 shutdown_connections 2024-02-02T15:40:19.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.672+0000 7f8227c14b80 1 -- 172.21.15.1:0/892730494 wait complete. 2024-02-02T15:40:19.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.672+0000 7f8227c14b80 20 client.29152 trim_cache size 0 max 16384 2024-02-02T15:40:19.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.685+0000 7f8227c14b80 1 Processor -- start 2024-02-02T15:40:19.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.685+0000 7f8227c14b80 1 -- start start 2024-02-02T15:40:19.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.686+0000 7f8227c14b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b0d659010 0x556b0d663cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:19.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.686+0000 7f8227c14b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556b0d659930 0x556b0d65c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:19.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.686+0000 7f8227c14b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b0d6647e0 0x556b0d65cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:19.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.686+0000 7f8227c14b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b0d42e6a0 con 0x556b0d659930 2024-02-02T15:40:19.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.686+0000 7f8227c14b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b0d65d220 con 0x556b0d6647e0 2024-02-02T15:40:19.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.686+0000 7f8227c14b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b0d666bd0 con 0x556b0d659010 2024-02-02T15:40:19.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.686+0000 7f8211600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b0d659010 0x556b0d663cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:19.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.686+0000 7f8212000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b0d6647e0 0x556b0d65cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:19.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.686+0000 7f8211600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b0d659010 0x556b0d663cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:40742/0 (socket says 172.21.15.1:40742) 2024-02-02T15:40:19.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.686+0000 7f8211600700 1 -- 172.21.15.1:0/1710676582 learned_addr learned my addr 172.21.15.1:0/1710676582 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:40:19.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.686+0000 7f8210c00700 1 --2- 172.21.15.1:0/1710676582 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556b0d659930 0x556b0d65c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:19.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.686+0000 7f8212000700 1 -- 172.21.15.1:0/1710676582 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b0d659010 msgr2=0x556b0d663cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:19.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.687+0000 7f8212000700 1 --2- 172.21.15.1:0/1710676582 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b0d659010 0x556b0d663cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.687+0000 7f8212000700 1 -- 172.21.15.1:0/1710676582 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556b0d659930 msgr2=0x556b0d65c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:19.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.687+0000 7f8212000700 1 --2- 172.21.15.1:0/1710676582 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556b0d659930 0x556b0d65c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.687+0000 7f8212000700 1 -- 172.21.15.1:0/1710676582 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556b0d65ab00 con 0x556b0d6647e0 2024-02-02T15:40:19.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.687+0000 7f8211600700 1 --2- 172.21.15.1:0/1710676582 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b0d659010 0x556b0d663cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:40:19.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.687+0000 7f8212000700 1 --2- 172.21.15.1:0/1710676582 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b0d6647e0 0x556b0d65cc80 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f82040a08b0 tx=0x7f82040092e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a258e3df6a4faf2d server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:19.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.687+0000 7f8203000700 1 -- 172.21.15.1:0/1710676582 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8204095140 con 0x556b0d6647e0 2024-02-02T15:40:19.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.687+0000 7f8203000700 1 -- 172.21.15.1:0/1710676582 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8204004ad0 con 0x556b0d6647e0 2024-02-02T15:40:19.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.687+0000 7f8203000700 1 -- 172.21.15.1:0/1710676582 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8204098e00 con 0x556b0d6647e0 2024-02-02T15:40:19.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.688+0000 7f8227c14b80 1 -- 172.21.15.1:0/1710676582 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b0d6647e0 msgr2=0x556b0d65cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:19.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.688+0000 7f8227c14b80 1 --2- 172.21.15.1:0/1710676582 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b0d6647e0 0x556b0d65cc80 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f82040a08b0 tx=0x7f82040092e0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.688+0000 7f8227c14b80 1 -- 172.21.15.1:0/1710676582 shutdown_connections 2024-02-02T15:40:19.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.688+0000 7f8227c14b80 1 --2- 172.21.15.1:0/1710676582 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b0d6647e0 0x556b0d65cc80 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.688+0000 7f8227c14b80 1 --2- 172.21.15.1:0/1710676582 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b0d659010 0x556b0d663cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.688+0000 7f8227c14b80 1 --2- 172.21.15.1:0/1710676582 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556b0d659930 0x556b0d65c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.688+0000 7f8227c14b80 1 -- 172.21.15.1:0/1710676582 >> 172.21.15.1:0/1710676582 conn(0x556b0d5b8db0 msgr2=0x556b0d6704a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:19.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.688+0000 7f8227c14b80 1 -- 172.21.15.1:0/1710676582 shutdown_connections 2024-02-02T15:40:19.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.688+0000 7f8227c14b80 1 -- 172.21.15.1:0/1710676582 wait complete. 2024-02-02T15:40:19.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.688+0000 7f8227c14b80 1 -- start start 2024-02-02T15:40:19.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.689+0000 7f8227c14b80 1 Processor -- start 2024-02-02T15:40:19.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.689+0000 7f8227c14b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b0d659010 0x556b0d667300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:19.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.689+0000 7f8227c14b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b0d659930 0x556b0d667840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:19.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.689+0000 7f8227c14b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556b0d6647e0 0x556b0d667d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:19.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.689+0000 7f8227c14b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b0d6c4850 con 0x556b0d6647e0 2024-02-02T15:40:19.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.689+0000 7f8227c14b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b0d6c4990 con 0x556b0d659930 2024-02-02T15:40:19.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.689+0000 7f8227c14b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556b0d6c4b50 con 0x556b0d659010 2024-02-02T15:40:19.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.689+0000 7f8227c14b80 10 client.0 _ll_register_callbacks cb 0x556b0d548290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:40:19.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.689+0000 7f8210c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b0d659930 0x556b0d667840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:19.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.689+0000 7f8211600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b0d659010 0x556b0d667300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:19.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.689+0000 7f8210c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b0d659930 0x556b0d667840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42202/0 (socket says 172.21.15.1:42202) 2024-02-02T15:40:19.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.689+0000 7f8210c00700 1 -- 172.21.15.1:0/2942395756 learned_addr learned my addr 172.21.15.1:0/2942395756 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:40:19.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.690+0000 7f8212000700 1 --2- 172.21.15.1:0/2942395756 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556b0d6647e0 0x556b0d667d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:19.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.690+0000 7f8211600700 1 -- 172.21.15.1:0/2942395756 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b0d659930 msgr2=0x556b0d667840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:19.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:19 smithi001 ceph-mon[35470]: pgmap v416: 129 pgs: 129 active+clean; 607 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 135 op/s 2024-02-02T15:40:19.694 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:19 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/3912609337' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:40:19.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.690+0000 7f8211600700 1 --2- 172.21.15.1:0/2942395756 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b0d659930 0x556b0d667840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.690+0000 7f8211600700 1 -- 172.21.15.1:0/2942395756 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556b0d6647e0 msgr2=0x556b0d667d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:19.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.690+0000 7f8211600700 1 --2- 172.21.15.1:0/2942395756 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556b0d6647e0 0x556b0d667d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.690+0000 7f8211600700 1 -- 172.21.15.1:0/2942395756 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f820c110700 con 0x556b0d659010 2024-02-02T15:40:19.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.690+0000 7f8212000700 1 --2- 172.21.15.1:0/2942395756 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556b0d6647e0 0x556b0d667d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:40:19.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.690+0000 7f8210c00700 1 --2- 172.21.15.1:0/2942395756 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b0d659930 0x556b0d667840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:40:19.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.690+0000 7f8211600700 1 --2- 172.21.15.1:0/2942395756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b0d659010 0x556b0d667300 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f81fc005930 tx=0x7f81fc01a050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55997dc6c9bc8eed server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:19.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.690+0000 7f81faa00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:40:19.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.691+0000 7f81faa00700 1 -- 172.21.15.1:0/2942395756 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81fc02b040 con 0x556b0d659010 2024-02-02T15:40:19.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.691+0000 7f8227c14b80 1 -- 172.21.15.1:0/2942395756 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f820401b320 con 0x556b0d659010 2024-02-02T15:40:19.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.691+0000 7f81faa00700 1 -- 172.21.15.1:0/2942395756 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f81fc00fa80 con 0x556b0d659010 2024-02-02T15:40:19.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.691+0000 7f81faa00700 1 -- 172.21.15.1:0/2942395756 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f81fc033e80 con 0x556b0d659010 2024-02-02T15:40:19.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.691+0000 7f8227c14b80 10 client.29158 fetch_fsmap learned FSMap version 52 2024-02-02T15:40:19.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.691+0000 7f8227c14b80 1 -- 172.21.15.1:0/2942395756 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x556b0d3d8b60 con 0x556b0d659010 2024-02-02T15:40:19.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.691+0000 7f81faa00700 1 -- 172.21.15.1:0/2942395756 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f820401b320 con 0x556b0d659010 2024-02-02T15:40:19.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.691+0000 7f81faa00700 1 -- 172.21.15.1:0/2942395756 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 52) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f81fc032bc0 con 0x556b0d659010 2024-02-02T15:40:19.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.691+0000 7f8227c14b80 10 client.29158 fetch_fsmap finished waiting for FSMap version 52 2024-02-02T15:40:19.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.691+0000 7f8227c14b80 10 client.29158 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-02-02T15:40:19.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.691+0000 7f8227c14b80 20 client.29158 populate_metadata read hostname 'smithi001' 2024-02-02T15:40:19.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.691+0000 7f8227c14b80 1 --2- 172.21.15.1:0/2942395756 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x556b0d6c6200 0x556b0d6c8620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:19.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.691+0000 7f8227c14b80 4 client.29158 mds_command: new command op to 24518 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:40:19.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.691+0000 7f8227c14b80 1 -- 172.21.15.1:0/2942395756 --> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556b0d569f50 con 0x556b0d6c6200 2024-02-02T15:40:19.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.692+0000 7f8210c00700 1 --2- 172.21.15.1:0/2942395756 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x556b0d6c6200 0x556b0d6c8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:19.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.692+0000 7f8210c00700 1 --2- 172.21.15.1:0/2942395756 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x556b0d6c6200 0x556b0d6c8620 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=90a759867332144d server_cookie=ab539ccecf6bbc1e in_seq=0 out_seq=0 2024-02-02T15:40:19.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.692+0000 7f81faa00700 10 client.29158 ms_handle_connect on v2:172.21.15.71:6836/628016199 2024-02-02T15:40:19.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.692+0000 7f81faa00700 1 -- 172.21.15.1:0/2942395756 <== mds.2 v2:172.21.15.71:6836/628016199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556b0d569f50 con 0x556b0d6c6200 2024-02-02T15:40:19.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.692+0000 7f81faa00700 10 client.29158 handle_command_reply: tid=0 2024-02-02T15:40:19.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.692+0000 7f8227c14b80 1 client.29158 shutdown 2024-02-02T15:40:19.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.693+0000 7f8227c14b80 1 -- 172.21.15.1:0/2942395756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b0d659010 msgr2=0x556b0d667300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:19.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.693+0000 7f8227c14b80 1 --2- 172.21.15.1:0/2942395756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b0d659010 0x556b0d667300 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f81fc005930 tx=0x7f81fc01a050 comp rx=0 tx=0).stop 2024-02-02T15:40:19.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.693+0000 7f8227c14b80 1 -- 172.21.15.1:0/2942395756 shutdown_connections 2024-02-02T15:40:19.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.693+0000 7f8227c14b80 1 --2- 172.21.15.1:0/2942395756 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556b0d6647e0 0x556b0d667d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.693+0000 7f8227c14b80 1 --2- 172.21.15.1:0/2942395756 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x556b0d6c6200 0x556b0d6c8620 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.693+0000 7f8227c14b80 1 --2- 172.21.15.1:0/2942395756 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556b0d659010 0x556b0d667300 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.693+0000 7f8227c14b80 1 --2- 172.21.15.1:0/2942395756 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556b0d659930 0x556b0d667840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.693+0000 7f81faa00700 0 client.29158 ms_handle_reset on v2:172.21.15.71:6836/628016199 2024-02-02T15:40:19.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.693+0000 7f8227c14b80 1 -- 172.21.15.1:0/2942395756 >> 172.21.15.1:0/2942395756 conn(0x556b0d5b8db0 msgr2=0x556b0d65c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:19.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.693+0000 7f8227c14b80 1 -- 172.21.15.1:0/2942395756 shutdown_connections 2024-02-02T15:40:19.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.693+0000 7f8227c14b80 1 -- 172.21.15.1:0/2942395756 wait complete. 2024-02-02T15:40:19.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.693+0000 7f8227c14b80 20 client.29158 trim_cache size 0 max 16384 2024-02-02T15:40:19.700 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:40:19.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.694+0000 7f8214000700 1 -- 172.21.15.1:0/1275432680 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f81ec0a0690 msgr2=0x7f81ec0a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:19.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.694+0000 7f8214000700 1 --2- 172.21.15.1:0/1275432680 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f81ec0a0690 0x7f81ec0a2b50 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f81fc006010 tx=0x7f81fc00b260 comp rx=0 tx=0).stop 2024-02-02T15:40:19.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.694+0000 7f8214000700 1 -- 172.21.15.1:0/1275432680 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f820c19da70 msgr2=0x7f820c19def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:19.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.694+0000 7f8214000700 1 --2- 172.21.15.1:0/1275432680 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f820c19da70 0x7f820c19def0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f820400efc0 tx=0x7f820400c3a0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.694+0000 7f8214000700 1 -- 172.21.15.1:0/1275432680 shutdown_connections 2024-02-02T15:40:19.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.694+0000 7f8214000700 1 --2- 172.21.15.1:0/1275432680 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f81ec0a0690 0x7f81ec0a2b50 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.694+0000 7f8214000700 1 --2- 172.21.15.1:0/1275432680 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f820c19da70 0x7f820c19def0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.694+0000 7f8214000700 1 --2- 172.21.15.1:0/1275432680 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f820c1031f0 0x7f820c1990d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.694+0000 7f8214000700 1 --2- 172.21.15.1:0/1275432680 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f820c108310 0x7f820c199610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.694+0000 7f8214000700 1 -- 172.21.15.1:0/1275432680 >> 172.21.15.1:0/1275432680 conn(0x7f820c0fea70 msgr2=0x7f820c106c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:19.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.694+0000 7f8214000700 1 -- 172.21.15.1:0/1275432680 shutdown_connections 2024-02-02T15:40:19.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.694+0000 7f8214000700 1 -- 172.21.15.1:0/1275432680 wait complete. 2024-02-02T15:40:19.716 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-02T15:40:19.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:19 smithi071 ceph-mon[44707]: pgmap v416: 129 pgs: 129 active+clean; 607 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 135 op/s 2024-02-02T15:40:19.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:19 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/3912609337' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:40:19.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:19 smithi142 ceph-mon[45728]: pgmap v416: 129 pgs: 129 active+clean; 607 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 135 op/s 2024-02-02T15:40:19.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:19 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/3912609337' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:40:19.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.936+0000 7f445f000700 1 -- 172.21.15.1:0/1544201211 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4458068010 msgr2=0x7f4458068470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:19.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.936+0000 7f445f000700 1 --2- 172.21.15.1:0/1544201211 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4458068010 0x7f4458068470 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f44400099c0 tx=0x7f4440009cd0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.936+0000 7f445f000700 1 -- 172.21.15.1:0/1544201211 shutdown_connections 2024-02-02T15:40:19.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.936+0000 7f445f000700 1 --2- 172.21.15.1:0/1544201211 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f44580689b0 0x7f445810dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.936+0000 7f445f000700 1 --2- 172.21.15.1:0/1544201211 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4458106ed0 0x7f44581072b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.936+0000 7f445f000700 1 --2- 172.21.15.1:0/1544201211 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4458068010 0x7f4458068470 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.936+0000 7f445f000700 1 -- 172.21.15.1:0/1544201211 >> 172.21.15.1:0/1544201211 conn(0x7f44580757d0 msgr2=0x7f4458075be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:19.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.937+0000 7f445f000700 1 -- 172.21.15.1:0/1544201211 shutdown_connections 2024-02-02T15:40:19.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.937+0000 7f445f000700 1 -- 172.21.15.1:0/1544201211 wait complete. 2024-02-02T15:40:19.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.938+0000 7f445f000700 1 Processor -- start 2024-02-02T15:40:19.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.938+0000 7f445f000700 1 -- start start 2024-02-02T15:40:19.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.938+0000 7f445f000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4458068010 0x7f44580fff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:19.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.938+0000 7f445f000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44580689b0 0x7f4458100480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:19.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.938+0000 7f445f000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4458106ed0 0x7f44581a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:19.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.938+0000 7f445f000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4458100cb0 con 0x7f4458068010 2024-02-02T15:40:19.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.938+0000 7f445f000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44581a5680 con 0x7f4458106ed0 2024-02-02T15:40:19.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.938+0000 7f445f000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44581a57c0 con 0x7f44580689b0 2024-02-02T15:40:19.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.938+0000 7f4457400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44580689b0 0x7f4458100480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:19.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.938+0000 7f4457e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4458068010 0x7f44580fff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:19.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.938+0000 7f4457400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44580689b0 0x7f4458100480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:40780/0 (socket says 172.21.15.1:40780) 2024-02-02T15:40:19.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.938+0000 7f4457400700 1 -- 172.21.15.1:0/510103944 learned_addr learned my addr 172.21.15.1:0/510103944 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:40:19.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.939+0000 7f445d000700 1 --2- 172.21.15.1:0/510103944 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4458106ed0 0x7f44581a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:19.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.939+0000 7f4457400700 1 -- 172.21.15.1:0/510103944 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4458106ed0 msgr2=0x7f44581a5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:19.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.939+0000 7f4457400700 1 --2- 172.21.15.1:0/510103944 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4458106ed0 0x7f44581a5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.939+0000 7f4457400700 1 -- 172.21.15.1:0/510103944 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4458068010 msgr2=0x7f44580fff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:19.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.939+0000 7f4457400700 1 --2- 172.21.15.1:0/510103944 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4458068010 0x7f44580fff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.939+0000 7f4457400700 1 -- 172.21.15.1:0/510103944 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44400095f0 con 0x7f44580689b0 2024-02-02T15:40:19.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.939+0000 7f445d000700 1 --2- 172.21.15.1:0/510103944 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4458106ed0 0x7f44581a5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:40:19.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.939+0000 7f4457400700 1 --2- 172.21.15.1:0/510103944 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44580689b0 0x7f4458100480 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f444000b580 tx=0x7f444000bfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:19.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.939+0000 7f4454c00700 1 -- 172.21.15.1:0/510103944 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444001d070 con 0x7f44580689b0 2024-02-02T15:40:19.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.939+0000 7f4454c00700 1 -- 172.21.15.1:0/510103944 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4440005050 con 0x7f44580689b0 2024-02-02T15:40:19.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.939+0000 7f445f000700 1 -- 172.21.15.1:0/510103944 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44581a59c0 con 0x7f44580689b0 2024-02-02T15:40:19.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.939+0000 7f4454c00700 1 -- 172.21.15.1:0/510103944 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4440004a40 con 0x7f44580689b0 2024-02-02T15:40:19.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.940+0000 7f445f000700 1 -- 172.21.15.1:0/510103944 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44581a5e90 con 0x7f44580689b0 2024-02-02T15:40:19.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.943+0000 7f4454c00700 1 -- 172.21.15.1:0/510103944 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f44400051c0 con 0x7f44580689b0 2024-02-02T15:40:19.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.944+0000 7f4472a9db80 1 Processor -- start 2024-02-02T15:40:19.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.945+0000 7f4472a9db80 1 -- start start 2024-02-02T15:40:19.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.945+0000 7f4454c00700 1 --2- 172.21.15.1:0/510103944 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f44380a0710 0x7f44380a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:19.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.945+0000 7f4472a9db80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5633d4c4e010 0x5633d4c4e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:19.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.945+0000 7f4472a9db80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5633d4c4e930 0x5633d4c592a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:19.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.945+0000 7f4454c00700 1 -- 172.21.15.1:0/510103944 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f44400ccf70 con 0x7f44580689b0 2024-02-02T15:40:19.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.945+0000 7f4472a9db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633d4c597e0 0x5633d4c5bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:19.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.945+0000 7f4472a9db80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633d4bc24c0 con 0x5633d4c4e930 2024-02-02T15:40:19.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.945+0000 7f4472a9db80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633d4ba6aa0 con 0x5633d4c4e010 2024-02-02T15:40:19.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.945+0000 7f4472a9db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633d4ba4460 con 0x5633d4c597e0 2024-02-02T15:40:19.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.946+0000 7f445d000700 1 --2- 172.21.15.1:0/510103944 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f44380a0710 0x7f44380a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:19.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.946+0000 7f4457e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5633d4c4e010 0x5633d4c4e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:19.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.946+0000 7f4457e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5633d4c4e010 0x5633d4c4e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42242/0 (socket says 172.21.15.1:42242) 2024-02-02T15:40:19.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.946+0000 7f4457e00700 1 -- 172.21.15.1:0/1143132442 learned_addr learned my addr 172.21.15.1:0/1143132442 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:40:19.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.946+0000 7f4457400700 1 --2- 172.21.15.1:0/1143132442 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5633d4c4e930 0x5633d4c592a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:19.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.946+0000 7f445d000700 1 --2- 172.21.15.1:0/1143132442 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633d4c597e0 0x5633d4c5bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:19.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.946+0000 7f4457e00700 1 -- 172.21.15.1:0/1143132442 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633d4c597e0 msgr2=0x5633d4c5bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:19.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.946+0000 7f4457e00700 1 --2- 172.21.15.1:0/1143132442 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633d4c597e0 0x5633d4c5bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.946+0000 7f4457e00700 1 -- 172.21.15.1:0/1143132442 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5633d4c4e930 msgr2=0x5633d4c592a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:19.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.946+0000 7f4457e00700 1 --2- 172.21.15.1:0/1143132442 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5633d4c4e930 0x5633d4c592a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.946+0000 7f4457e00700 1 -- 172.21.15.1:0/1143132442 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5633d4a213b0 con 0x5633d4c4e010 2024-02-02T15:40:19.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.946+0000 7f445d000700 1 --2- 172.21.15.1:0/1143132442 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633d4c597e0 0x5633d4c5bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:40:19.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.946+0000 7f4457e00700 1 --2- 172.21.15.1:0/1143132442 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5633d4c4e010 0x5633d4c4e3f0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f4448009d30 tx=0x7f4448011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8627d440a922b14c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:19.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.946+0000 7f445d000700 1 --2- 172.21.15.1:0/510103944 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f44380a0710 0x7f44380a2bd0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f44500094f0 tx=0x7f4450009920 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:19.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.947+0000 7f444d600700 1 -- 172.21.15.1:0/1143132442 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4448004230 con 0x5633d4c4e010 2024-02-02T15:40:19.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.947+0000 7f444d600700 1 -- 172.21.15.1:0/1143132442 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4448004390 con 0x5633d4c4e010 2024-02-02T15:40:19.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.947+0000 7f444d600700 1 -- 172.21.15.1:0/1143132442 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4448019840 con 0x5633d4c4e010 2024-02-02T15:40:19.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.947+0000 7f4472a9db80 1 -- 172.21.15.1:0/1143132442 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5633d4c4e010 msgr2=0x5633d4c4e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:19.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.947+0000 7f4472a9db80 1 --2- 172.21.15.1:0/1143132442 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5633d4c4e010 0x5633d4c4e3f0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f4448009d30 tx=0x7f4448011720 comp rx=0 tx=0).stop 2024-02-02T15:40:19.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.947+0000 7f4472a9db80 1 -- 172.21.15.1:0/1143132442 shutdown_connections 2024-02-02T15:40:19.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.947+0000 7f4472a9db80 1 --2- 172.21.15.1:0/1143132442 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633d4c597e0 0x5633d4c5bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.947+0000 7f4472a9db80 1 --2- 172.21.15.1:0/1143132442 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5633d4c4e010 0x5633d4c4e3f0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.947+0000 7f4472a9db80 1 --2- 172.21.15.1:0/1143132442 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5633d4c4e930 0x5633d4c592a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.947+0000 7f4472a9db80 1 -- 172.21.15.1:0/1143132442 >> 172.21.15.1:0/1143132442 conn(0x5633d4baddb0 msgr2=0x5633d4c4a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:19.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.948+0000 7f4472a9db80 1 -- 172.21.15.1:0/1143132442 shutdown_connections 2024-02-02T15:40:19.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.948+0000 7f4472a9db80 1 -- 172.21.15.1:0/1143132442 wait complete. 2024-02-02T15:40:19.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.948+0000 7f4472a9db80 1 -- start start 2024-02-02T15:40:19.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.948+0000 7f4472a9db80 1 Processor -- start 2024-02-02T15:40:19.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.949+0000 7f4472a9db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633d4c4e010 0x5633d4c59110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:19.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.949+0000 7f4472a9db80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5633d4c4e930 0x5633d4c56390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:19.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.949+0000 7f4472a9db80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5633d4c597e0 0x5633d4c56950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:19.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.949+0000 7f4472a9db80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633d4b9fd50 con 0x5633d4c4e930 2024-02-02T15:40:19.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.949+0000 7f4472a9db80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633d4b3bd80 con 0x5633d4c597e0 2024-02-02T15:40:19.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.949+0000 7f4472a9db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633d4b3ba20 con 0x5633d4c4e010 2024-02-02T15:40:19.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.949+0000 7f4472a9db80 10 client.0 _ll_register_callbacks cb 0x5633d4b3d290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:40:19.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.949+0000 7f4457e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633d4c4e010 0x5633d4c59110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:19.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.949+0000 7f4457e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633d4c4e010 0x5633d4c59110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:40790/0 (socket says 172.21.15.1:40790) 2024-02-02T15:40:19.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.949+0000 7f4457e00700 1 -- 172.21.15.1:0/1846337465 learned_addr learned my addr 172.21.15.1:0/1846337465 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:40:19.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.949+0000 7f4457400700 1 --2- 172.21.15.1:0/1846337465 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5633d4c4e930 0x5633d4c56390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:19.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.949+0000 7f445d000700 1 --2- 172.21.15.1:0/1846337465 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5633d4c597e0 0x5633d4c56950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:19.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.950+0000 7f445d000700 1 -- 172.21.15.1:0/1846337465 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633d4c4e010 msgr2=0x5633d4c59110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:19.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.950+0000 7f445d000700 1 --2- 172.21.15.1:0/1846337465 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633d4c4e010 0x5633d4c59110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.950+0000 7f445d000700 1 -- 172.21.15.1:0/1846337465 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5633d4c4e930 msgr2=0x5633d4c56390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:19.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.950+0000 7f445d000700 1 --2- 172.21.15.1:0/1846337465 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5633d4c4e930 0x5633d4c56390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.950+0000 7f445d000700 1 -- 172.21.15.1:0/1846337465 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4448009960 con 0x5633d4c597e0 2024-02-02T15:40:19.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.950+0000 7f445d000700 1 --2- 172.21.15.1:0/1846337465 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5633d4c597e0 0x5633d4c56950 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f4450017d80 tx=0x7f4450004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=305706310153920c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:19.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.950+0000 7f4446000700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:40:19.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.950+0000 7f4446000700 1 -- 172.21.15.1:0/1846337465 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445001e040 con 0x5633d4c597e0 2024-02-02T15:40:19.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.950+0000 7f4472a9db80 1 -- 172.21.15.1:0/1846337465 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5633d4c420f0 con 0x5633d4c597e0 2024-02-02T15:40:19.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.950+0000 7f4446000700 1 -- 172.21.15.1:0/1846337465 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f4450019b70 con 0x5633d4c597e0 2024-02-02T15:40:19.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.950+0000 7f4446000700 1 -- 172.21.15.1:0/1846337465 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4450019d30 con 0x5633d4c597e0 2024-02-02T15:40:19.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.950+0000 7f4472a9db80 10 client.29174 fetch_fsmap learned FSMap version 52 2024-02-02T15:40:19.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.950+0000 7f4472a9db80 1 -- 172.21.15.1:0/1846337465 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x5633d49cdb60 con 0x5633d4c597e0 2024-02-02T15:40:19.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.951+0000 7f4446000700 1 -- 172.21.15.1:0/1846337465 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5633d4c420f0 con 0x5633d4c597e0 2024-02-02T15:40:19.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.951+0000 7f4446000700 1 -- 172.21.15.1:0/1846337465 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 52) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f4450004620 con 0x5633d4c597e0 2024-02-02T15:40:19.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.951+0000 7f4472a9db80 10 client.29174 fetch_fsmap finished waiting for FSMap version 52 2024-02-02T15:40:19.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.951+0000 7f4472a9db80 10 client.29174 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-02T15:40:19.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.951+0000 7f4472a9db80 20 client.29174 populate_metadata read hostname 'smithi001' 2024-02-02T15:40:19.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.951+0000 7f4472a9db80 1 --2- 172.21.15.1:0/1846337465 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x5633d4c69220 0x5633d4c6b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:19.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.951+0000 7f4472a9db80 4 client.29174 mds_command: new command op to 24529 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:40:19.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.951+0000 7f4472a9db80 1 -- 172.21.15.1:0/1846337465 --> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5633d4b98dd0 con 0x5633d4c69220 2024-02-02T15:40:19.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.951+0000 7f4457e00700 1 --2- 172.21.15.1:0/1846337465 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x5633d4c69220 0x5633d4c6b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:19.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.952+0000 7f4457e00700 1 --2- 172.21.15.1:0/1846337465 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x5633d4c69220 0x5633d4c6b640 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=40d7a2b8169a7a4c server_cookie=5e6356b61e17dbac in_seq=0 out_seq=0 2024-02-02T15:40:19.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.952+0000 7f4446000700 10 client.29174 ms_handle_connect on v2:172.21.15.142:6838/321675421 2024-02-02T15:40:19.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.953+0000 7f4446000700 1 -- 172.21.15.1:0/1846337465 <== mds.1 v2:172.21.15.142:6838/321675421 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5633d4b98dd0 con 0x5633d4c69220 2024-02-02T15:40:19.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.953+0000 7f4446000700 10 client.29174 handle_command_reply: tid=0 2024-02-02T15:40:19.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.953+0000 7f4472a9db80 1 client.29174 shutdown 2024-02-02T15:40:19.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.953+0000 7f4472a9db80 1 -- 172.21.15.1:0/1846337465 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5633d4c597e0 msgr2=0x5633d4c56950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:19.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.953+0000 7f4472a9db80 1 --2- 172.21.15.1:0/1846337465 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5633d4c597e0 0x5633d4c56950 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f4450017d80 tx=0x7f4450004ad0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.953+0000 7f4472a9db80 1 -- 172.21.15.1:0/1846337465 shutdown_connections 2024-02-02T15:40:19.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.953+0000 7f4472a9db80 1 --2- 172.21.15.1:0/1846337465 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x5633d4c69220 0x5633d4c6b640 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.953+0000 7f4472a9db80 1 --2- 172.21.15.1:0/1846337465 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5633d4c597e0 0x5633d4c56950 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.953+0000 7f4472a9db80 1 --2- 172.21.15.1:0/1846337465 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633d4c4e010 0x5633d4c59110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.953+0000 7f4472a9db80 1 --2- 172.21.15.1:0/1846337465 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5633d4c4e930 0x5633d4c56390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.953+0000 7f4472a9db80 1 -- 172.21.15.1:0/1846337465 >> 172.21.15.1:0/1846337465 conn(0x5633d4baddb0 msgr2=0x5633d4c49fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:19.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.953+0000 7f4446000700 0 client.29174 ms_handle_reset on v2:172.21.15.142:6838/321675421 2024-02-02T15:40:19.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.953+0000 7f4472a9db80 1 -- 172.21.15.1:0/1846337465 shutdown_connections 2024-02-02T15:40:19.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.953+0000 7f4472a9db80 1 -- 172.21.15.1:0/1846337465 wait complete. 2024-02-02T15:40:19.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.953+0000 7f4472a9db80 20 client.29174 trim_cache size 0 max 16384 2024-02-02T15:40:19.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.967+0000 7f4472a9db80 1 Processor -- start 2024-02-02T15:40:19.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.967+0000 7f4472a9db80 1 -- start start 2024-02-02T15:40:19.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.967+0000 7f4472a9db80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5633d4c4e010 0x5633d4c60370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:19.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.967+0000 7f4472a9db80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5633d4c4e930 0x5633d4c58cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:19.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.967+0000 7f4472a9db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633d4c597e0 0x5633d4c53600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:19.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.967+0000 7f4472a9db80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633d4a236a0 con 0x5633d4c4e010 2024-02-02T15:40:19.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.967+0000 7f4472a9db80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633d4c41d20 con 0x5633d4c4e930 2024-02-02T15:40:19.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.967+0000 7f4472a9db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633d4c4e470 con 0x5633d4c597e0 2024-02-02T15:40:19.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.968+0000 7f4457400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5633d4c4e930 0x5633d4c58cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:19.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.968+0000 7f4457e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5633d4c4e010 0x5633d4c60370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:19.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.968+0000 7f4457400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5633d4c4e930 0x5633d4c58cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42252/0 (socket says 172.21.15.1:42252) 2024-02-02T15:40:19.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.968+0000 7f4457400700 1 -- 172.21.15.1:0/1788528818 learned_addr learned my addr 172.21.15.1:0/1788528818 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:40:19.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.968+0000 7f445d000700 1 --2- 172.21.15.1:0/1788528818 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633d4c597e0 0x5633d4c53600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:19.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.968+0000 7f4457400700 1 -- 172.21.15.1:0/1788528818 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633d4c597e0 msgr2=0x5633d4c53600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:19.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.968+0000 7f4457400700 1 --2- 172.21.15.1:0/1788528818 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633d4c597e0 0x5633d4c53600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.968+0000 7f4457400700 1 -- 172.21.15.1:0/1788528818 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5633d4c4e010 msgr2=0x5633d4c60370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:19.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.968+0000 7f4457400700 1 --2- 172.21.15.1:0/1788528818 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5633d4c4e010 0x5633d4c60370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.968+0000 7f4457400700 1 -- 172.21.15.1:0/1788528818 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5633d4b65ea0 con 0x5633d4c4e930 2024-02-02T15:40:19.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.968+0000 7f4457400700 1 --2- 172.21.15.1:0/1788528818 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5633d4c4e930 0x5633d4c58cb0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f4440099e20 tx=0x7f4440096590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffd742bb9b6356e9 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:19.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.968+0000 7f444e000700 1 -- 172.21.15.1:0/1788528818 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444009c070 con 0x5633d4c4e930 2024-02-02T15:40:19.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.968+0000 7f444e000700 1 -- 172.21.15.1:0/1788528818 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f444009df50 con 0x5633d4c4e930 2024-02-02T15:40:19.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.968+0000 7f444e000700 1 -- 172.21.15.1:0/1788528818 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4440004e40 con 0x5633d4c4e930 2024-02-02T15:40:19.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.969+0000 7f4472a9db80 1 -- 172.21.15.1:0/1788528818 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5633d4c4e930 msgr2=0x5633d4c58cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:19.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.969+0000 7f4472a9db80 1 --2- 172.21.15.1:0/1788528818 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5633d4c4e930 0x5633d4c58cb0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f4440099e20 tx=0x7f4440096590 comp rx=0 tx=0).stop 2024-02-02T15:40:19.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.969+0000 7f4472a9db80 1 -- 172.21.15.1:0/1788528818 shutdown_connections 2024-02-02T15:40:19.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.969+0000 7f4472a9db80 1 --2- 172.21.15.1:0/1788528818 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633d4c597e0 0x5633d4c53600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.969+0000 7f4472a9db80 1 --2- 172.21.15.1:0/1788528818 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5633d4c4e010 0x5633d4c60370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.969+0000 7f4472a9db80 1 --2- 172.21.15.1:0/1788528818 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5633d4c4e930 0x5633d4c58cb0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.969+0000 7f4472a9db80 1 -- 172.21.15.1:0/1788528818 >> 172.21.15.1:0/1788528818 conn(0x5633d4baddb0 msgr2=0x5633d4c516a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:19.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.969+0000 7f4472a9db80 1 -- 172.21.15.1:0/1788528818 shutdown_connections 2024-02-02T15:40:19.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.969+0000 7f4472a9db80 1 -- 172.21.15.1:0/1788528818 wait complete. 2024-02-02T15:40:19.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.970+0000 7f4472a9db80 1 -- start start 2024-02-02T15:40:19.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.970+0000 7f4472a9db80 1 Processor -- start 2024-02-02T15:40:19.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.970+0000 7f4472a9db80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5633d4c4e010 0x5633d4caecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:19.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.970+0000 7f4472a9db80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5633d4c4e930 0x5633d4caf210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:19.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.970+0000 7f4472a9db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633d4c597e0 0x5633d4cb9d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:19.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.970+0000 7f4472a9db80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633d4cafb20 con 0x5633d4c4e010 2024-02-02T15:40:19.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.970+0000 7f4472a9db80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633d4cba290 con 0x5633d4c4e930 2024-02-02T15:40:19.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.971+0000 7f4472a9db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633d4cba400 con 0x5633d4c597e0 2024-02-02T15:40:19.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.971+0000 7f4472a9db80 10 client.0 _ll_register_callbacks cb 0x5633d4b3d290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:40:19.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.971+0000 7f4457400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5633d4c4e930 0x5633d4caf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:19.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.971+0000 7f4457400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5633d4c4e930 0x5633d4caf210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42264/0 (socket says 172.21.15.1:42264) 2024-02-02T15:40:19.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.971+0000 7f4457400700 1 -- 172.21.15.1:0/1053887678 learned_addr learned my addr 172.21.15.1:0/1053887678 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:40:19.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.971+0000 7f4457e00700 1 --2- 172.21.15.1:0/1053887678 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5633d4c4e010 0x5633d4caecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:19.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.971+0000 7f445d000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633d4c597e0 0x5633d4cb9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:19.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.971+0000 7f4457400700 1 -- 172.21.15.1:0/1053887678 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633d4c597e0 msgr2=0x5633d4cb9d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:19.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.971+0000 7f4457400700 1 --2- 172.21.15.1:0/1053887678 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633d4c597e0 0x5633d4cb9d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.971+0000 7f4457400700 1 -- 172.21.15.1:0/1053887678 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5633d4c4e010 msgr2=0x5633d4caecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:19.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.971+0000 7f4457400700 1 --2- 172.21.15.1:0/1053887678 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5633d4c4e010 0x5633d4caecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.971+0000 7f4457400700 1 -- 172.21.15.1:0/1053887678 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44581a5e90 con 0x5633d4c4e930 2024-02-02T15:40:19.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.971+0000 7f445d000700 1 --2- 172.21.15.1:0/1053887678 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633d4c597e0 0x5633d4cb9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:40:19.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.971+0000 7f4457e00700 1 --2- 172.21.15.1:0/1053887678 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5633d4c4e010 0x5633d4caecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:40:19.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.971+0000 7f4457400700 1 --2- 172.21.15.1:0/1053887678 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5633d4c4e930 0x5633d4caf210 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f444009a0b0 tx=0x7f4440004e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2192b82748a5d602 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:19.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.971+0000 7f4446000700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:40:19.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.972+0000 7f4446000700 1 -- 172.21.15.1:0/1053887678 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44400b8070 con 0x5633d4c4e930 2024-02-02T15:40:19.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.972+0000 7f4446000700 1 -- 172.21.15.1:0/1053887678 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f444009d070 con 0x5633d4c4e930 2024-02-02T15:40:19.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.972+0000 7f4472a9db80 1 -- 172.21.15.1:0/1053887678 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4448004390 con 0x5633d4c4e930 2024-02-02T15:40:19.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.972+0000 7f4446000700 1 -- 172.21.15.1:0/1053887678 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f444009c030 con 0x5633d4c4e930 2024-02-02T15:40:19.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.972+0000 7f4472a9db80 10 client.29180 fetch_fsmap learned FSMap version 52 2024-02-02T15:40:19.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.972+0000 7f4472a9db80 1 -- 172.21.15.1:0/1053887678 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x5633d4a25cd0 con 0x5633d4c4e930 2024-02-02T15:40:19.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.972+0000 7f4446000700 1 -- 172.21.15.1:0/1053887678 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4440081030 con 0x5633d4c4e930 2024-02-02T15:40:19.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.972+0000 7f4446000700 1 -- 172.21.15.1:0/1053887678 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 52) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f44400a1d00 con 0x5633d4c4e930 2024-02-02T15:40:19.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.972+0000 7f4472a9db80 10 client.29180 fetch_fsmap finished waiting for FSMap version 52 2024-02-02T15:40:19.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.972+0000 7f4472a9db80 10 client.29180 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-02T15:40:19.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.972+0000 7f4472a9db80 20 client.29180 populate_metadata read hostname 'smithi001' 2024-02-02T15:40:19.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.972+0000 7f4472a9db80 1 --2- 172.21.15.1:0/1053887678 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x5633d4cbb7d0 0x5633d4cbdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:19.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.972+0000 7f4472a9db80 4 client.29180 mds_command: new command op to 24529 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:40:19.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.972+0000 7f4472a9db80 1 -- 172.21.15.1:0/1053887678 --> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5633d4b5ef50 con 0x5633d4cbb7d0 2024-02-02T15:40:19.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.973+0000 7f4457e00700 1 --2- 172.21.15.1:0/1053887678 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x5633d4cbb7d0 0x5633d4cbdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:19.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.973+0000 7f4457e00700 1 --2- 172.21.15.1:0/1053887678 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x5633d4cbb7d0 0x5633d4cbdbf0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ade9cf4deaad9621 server_cookie=f5adeae3837c1e67 in_seq=0 out_seq=0 2024-02-02T15:40:19.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.973+0000 7f4446000700 10 client.29180 ms_handle_connect on v2:172.21.15.142:6838/321675421 2024-02-02T15:40:19.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.973+0000 7f4446000700 1 -- 172.21.15.1:0/1053887678 <== mds.1 v2:172.21.15.142:6838/321675421 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5633d4b5ef50 con 0x5633d4cbb7d0 2024-02-02T15:40:19.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.973+0000 7f4446000700 10 client.29180 handle_command_reply: tid=0 2024-02-02T15:40:19.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.973+0000 7f4472a9db80 1 client.29180 shutdown 2024-02-02T15:40:19.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.973+0000 7f4472a9db80 1 -- 172.21.15.1:0/1053887678 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5633d4c4e930 msgr2=0x5633d4caf210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:19.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.973+0000 7f4472a9db80 1 --2- 172.21.15.1:0/1053887678 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5633d4c4e930 0x5633d4caf210 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f444009a0b0 tx=0x7f4440004e10 comp rx=0 tx=0).stop 2024-02-02T15:40:19.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.974+0000 7f4472a9db80 1 -- 172.21.15.1:0/1053887678 shutdown_connections 2024-02-02T15:40:19.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.974+0000 7f4472a9db80 1 --2- 172.21.15.1:0/1053887678 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x5633d4cbb7d0 0x5633d4cbdbf0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.974+0000 7f4472a9db80 1 --2- 172.21.15.1:0/1053887678 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5633d4c597e0 0x5633d4cb9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.974+0000 7f4472a9db80 1 --2- 172.21.15.1:0/1053887678 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5633d4c4e010 0x5633d4caecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.974+0000 7f4446000700 0 client.29180 ms_handle_reset on v2:172.21.15.142:6838/321675421 2024-02-02T15:40:19.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.974+0000 7f4472a9db80 1 --2- 172.21.15.1:0/1053887678 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5633d4c4e930 0x5633d4caf210 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.974+0000 7f4472a9db80 1 -- 172.21.15.1:0/1053887678 >> 172.21.15.1:0/1053887678 conn(0x5633d4baddb0 msgr2=0x5633d4c495c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:19.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.974+0000 7f4472a9db80 1 -- 172.21.15.1:0/1053887678 shutdown_connections 2024-02-02T15:40:19.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.974+0000 7f4472a9db80 1 -- 172.21.15.1:0/1053887678 wait complete. 2024-02-02T15:40:19.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.974+0000 7f4472a9db80 20 client.29180 trim_cache size 0 max 16384 2024-02-02T15:40:19.978 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:40:19.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.975+0000 7f445f000700 1 -- 172.21.15.1:0/510103944 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f44380a0710 msgr2=0x7f44380a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:19.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.975+0000 7f445f000700 1 --2- 172.21.15.1:0/510103944 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f44380a0710 0x7f44380a2bd0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f44500094f0 tx=0x7f4450009920 comp rx=0 tx=0).stop 2024-02-02T15:40:19.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.975+0000 7f445f000700 1 -- 172.21.15.1:0/510103944 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44580689b0 msgr2=0x7f4458100480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:19.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.975+0000 7f445f000700 1 --2- 172.21.15.1:0/510103944 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44580689b0 0x7f4458100480 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f444000b580 tx=0x7f444000bfd0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.975+0000 7f445f000700 1 -- 172.21.15.1:0/510103944 shutdown_connections 2024-02-02T15:40:19.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.975+0000 7f445f000700 1 --2- 172.21.15.1:0/510103944 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f44380a0710 0x7f44380a2bd0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.975+0000 7f445f000700 1 --2- 172.21.15.1:0/510103944 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4458106ed0 0x7f44581a5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.975+0000 7f445f000700 1 --2- 172.21.15.1:0/510103944 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4458068010 0x7f44580fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.975+0000 7f445f000700 1 --2- 172.21.15.1:0/510103944 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f44580689b0 0x7f4458100480 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:19.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.975+0000 7f445f000700 1 -- 172.21.15.1:0/510103944 >> 172.21.15.1:0/510103944 conn(0x7f44580757d0 msgr2=0x7f44580fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:19.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.975+0000 7f445f000700 1 -- 172.21.15.1:0/510103944 shutdown_connections 2024-02-02T15:40:19.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:19.975+0000 7f445f000700 1 -- 172.21.15.1:0/510103944 wait complete. 2024-02-02T15:40:19.995 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-02T15:40:20.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.207+0000 7fd175a00700 1 -- 172.21.15.1:0/975519119 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd170068010 msgr2=0x7fd170068470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:20.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.207+0000 7fd175a00700 1 --2- 172.21.15.1:0/975519119 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd170068010 0x7fd170068470 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fd1580098f0 tx=0x7fd158009c00 comp rx=0 tx=0).stop 2024-02-02T15:40:20.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.207+0000 7fd175a00700 1 -- 172.21.15.1:0/975519119 shutdown_connections 2024-02-02T15:40:20.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.207+0000 7fd175a00700 1 --2- 172.21.15.1:0/975519119 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd1700689b0 0x7fd17010dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:20.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.207+0000 7fd175a00700 1 --2- 172.21.15.1:0/975519119 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd170106ed0 0x7fd1701072b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:20.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.207+0000 7fd175a00700 1 --2- 172.21.15.1:0/975519119 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd170068010 0x7fd170068470 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:20.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.207+0000 7fd175a00700 1 -- 172.21.15.1:0/975519119 >> 172.21.15.1:0/975519119 conn(0x7fd1700757d0 msgr2=0x7fd170075be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:20.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.207+0000 7fd175a00700 1 -- 172.21.15.1:0/975519119 shutdown_connections 2024-02-02T15:40:20.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.207+0000 7fd175a00700 1 -- 172.21.15.1:0/975519119 wait complete. 2024-02-02T15:40:20.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.208+0000 7fd175a00700 1 Processor -- start 2024-02-02T15:40:20.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.208+0000 7fd175a00700 1 -- start start 2024-02-02T15:40:20.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.208+0000 7fd175a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd170068010 0x7fd170196f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:20.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.208+0000 7fd175a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1700689b0 0x7fd170197470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:20.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.208+0000 7fd175a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd170106ed0 0x7fd17019b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:20.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.208+0000 7fd175a00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd17019bf10 con 0x7fd170106ed0 2024-02-02T15:40:20.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.208+0000 7fd175a00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd17019c080 con 0x7fd1700689b0 2024-02-02T15:40:20.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.208+0000 7fd175a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd17019c240 con 0x7fd170068010 2024-02-02T15:40:20.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.209+0000 7fd16e000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1700689b0 0x7fd170197470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:20.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.209+0000 7fd16e000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1700689b0 0x7fd170197470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42274/0 (socket says 172.21.15.1:42274) 2024-02-02T15:40:20.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.209+0000 7fd16ea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd170068010 0x7fd170196f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:20.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.209+0000 7fd16e000700 1 -- 172.21.15.1:0/2346775424 learned_addr learned my addr 172.21.15.1:0/2346775424 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:40:20.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.209+0000 7fd16f400700 1 --2- 172.21.15.1:0/2346775424 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd170106ed0 0x7fd17019b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:20.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.209+0000 7fd16e000700 1 -- 172.21.15.1:0/2346775424 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd170068010 msgr2=0x7fd170196f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:20.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.209+0000 7fd16e000700 1 --2- 172.21.15.1:0/2346775424 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd170068010 0x7fd170196f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:20.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.209+0000 7fd16e000700 1 -- 172.21.15.1:0/2346775424 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd170106ed0 msgr2=0x7fd17019b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:20.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.209+0000 7fd16e000700 1 --2- 172.21.15.1:0/2346775424 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd170106ed0 0x7fd17019b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:20.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.209+0000 7fd16e000700 1 -- 172.21.15.1:0/2346775424 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1600095f0 con 0x7fd1700689b0 2024-02-02T15:40:20.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.209+0000 7fd16e000700 1 --2- 172.21.15.1:0/2346775424 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1700689b0 0x7fd170197470 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fd158009230 tx=0x7fd15800fa80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:20.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.210+0000 7fd167400700 1 -- 172.21.15.1:0/2346775424 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd15801c070 con 0x7fd1700689b0 2024-02-02T15:40:20.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.210+0000 7fd167400700 1 -- 172.21.15.1:0/2346775424 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd15800bba0 con 0x7fd1700689b0 2024-02-02T15:40:20.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.210+0000 7fd175a00700 1 -- 172.21.15.1:0/2346775424 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd158009520 con 0x7fd1700689b0 2024-02-02T15:40:20.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.210+0000 7fd167400700 1 -- 172.21.15.1:0/2346775424 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1580038e0 con 0x7fd1700689b0 2024-02-02T15:40:20.211 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.210+0000 7fd175a00700 1 -- 172.21.15.1:0/2346775424 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1701a3b80 con 0x7fd1700689b0 2024-02-02T15:40:20.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.213+0000 7fd167400700 1 -- 172.21.15.1:0/2346775424 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7fd158020030 con 0x7fd1700689b0 2024-02-02T15:40:20.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.214+0000 7fd167400700 1 --2- 172.21.15.1:0/2346775424 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fd1500a0710 0x7fd1500a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:20.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.214+0000 7fd167400700 1 -- 172.21.15.1:0/2346775424 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fd1580ccfb0 con 0x7fd1700689b0 2024-02-02T15:40:20.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.214+0000 7fd18943bb80 1 Processor -- start 2024-02-02T15:40:20.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.214+0000 7fd16ea00700 1 --2- 172.21.15.1:0/2346775424 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fd1500a0710 0x7fd1500a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:20.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.215+0000 7fd18943bb80 1 -- start start 2024-02-02T15:40:20.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.215+0000 7fd16ea00700 1 --2- 172.21.15.1:0/2346775424 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fd1500a0710 0x7fd1500a2bd0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd160005fd0 tx=0x7fd160009310 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:20.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.215+0000 7fd18943bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5584ce5f3010 0x5584ce5f33f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:20.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.215+0000 7fd18943bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584ce5f3930 0x5584ce5fe2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:20.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.215+0000 7fd18943bb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5584ce5fe7e0 0x5584ce600bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:20.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.215+0000 7fd18943bb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584ce5674c0 con 0x5584ce5fe7e0 2024-02-02T15:40:20.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.215+0000 7fd18943bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584ce54baa0 con 0x5584ce5f3010 2024-02-02T15:40:20.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.215+0000 7fd18943bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584ce549460 con 0x5584ce5f3930 2024-02-02T15:40:20.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.215+0000 7fd16ea00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5584ce5f3010 0x5584ce5f33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:20.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.215+0000 7fd16e000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584ce5f3930 0x5584ce5fe2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:20.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.215+0000 7fd16f400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5584ce5fe7e0 0x5584ce600bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:20.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.215+0000 7fd16e000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584ce5f3930 0x5584ce5fe2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:40832/0 (socket says 172.21.15.1:40832) 2024-02-02T15:40:20.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.215+0000 7fd16e000700 1 -- 172.21.15.1:0/1978611704 learned_addr learned my addr 172.21.15.1:0/1978611704 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:40:20.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.216+0000 7fd16e000700 1 -- 172.21.15.1:0/1978611704 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5584ce5f3010 msgr2=0x5584ce5f33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:20.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.216+0000 7fd16e000700 1 --2- 172.21.15.1:0/1978611704 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5584ce5f3010 0x5584ce5f33f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:20.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.216+0000 7fd16e000700 1 -- 172.21.15.1:0/1978611704 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5584ce5fe7e0 msgr2=0x5584ce600bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:20.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.216+0000 7fd16e000700 1 --2- 172.21.15.1:0/1978611704 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5584ce5fe7e0 0x5584ce600bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:20.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.216+0000 7fd16e000700 1 -- 172.21.15.1:0/1978611704 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5584ce3c63b0 con 0x5584ce5f3930 2024-02-02T15:40:20.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.216+0000 7fd16e000700 1 --2- 172.21.15.1:0/1978611704 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584ce5f3930 0x5584ce5fe2a0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fd15802a030 tx=0x7fd158009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa32459d6c2a73cc server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:20.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.216+0000 7fd15fe00700 1 -- 172.21.15.1:0/1978611704 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1580b6070 con 0x5584ce5f3930 2024-02-02T15:40:20.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.216+0000 7fd15fe00700 1 -- 172.21.15.1:0/1978611704 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd1580a00c0 con 0x5584ce5f3930 2024-02-02T15:40:20.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.216+0000 7fd15fe00700 1 -- 172.21.15.1:0/1978611704 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1580ca580 con 0x5584ce5f3930 2024-02-02T15:40:20.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.217+0000 7fd18943bb80 1 -- 172.21.15.1:0/1978611704 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584ce5f3930 msgr2=0x5584ce5fe2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:20.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.217+0000 7fd18943bb80 1 --2- 172.21.15.1:0/1978611704 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584ce5f3930 0x5584ce5fe2a0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fd15802a030 tx=0x7fd158009c00 comp rx=0 tx=0).stop 2024-02-02T15:40:20.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.217+0000 7fd18943bb80 1 -- 172.21.15.1:0/1978611704 shutdown_connections 2024-02-02T15:40:20.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.217+0000 7fd18943bb80 1 --2- 172.21.15.1:0/1978611704 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5584ce5fe7e0 0x5584ce600bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:20.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.217+0000 7fd18943bb80 1 --2- 172.21.15.1:0/1978611704 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5584ce5f3010 0x5584ce5f33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:20.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.217+0000 7fd18943bb80 1 --2- 172.21.15.1:0/1978611704 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584ce5f3930 0x5584ce5fe2a0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:20.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.217+0000 7fd18943bb80 1 -- 172.21.15.1:0/1978611704 >> 172.21.15.1:0/1978611704 conn(0x5584ce552db0 msgr2=0x5584ce5ef3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:20.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.217+0000 7fd18943bb80 1 -- 172.21.15.1:0/1978611704 shutdown_connections 2024-02-02T15:40:20.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.217+0000 7fd18943bb80 1 -- 172.21.15.1:0/1978611704 wait complete. 2024-02-02T15:40:20.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.218+0000 7fd18943bb80 1 -- start start 2024-02-02T15:40:20.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.218+0000 7fd18943bb80 1 Processor -- start 2024-02-02T15:40:20.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.218+0000 7fd18943bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584ce5f3010 0x5584ce5febc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:20.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.218+0000 7fd18943bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5584ce5f3930 0x5584ce5ff100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:20.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.219+0000 7fd18943bb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5584ce5fe7e0 0x5584ce5ff660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:20.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.219+0000 7fd18943bb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584ce544d50 con 0x5584ce5fe7e0 2024-02-02T15:40:20.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.219+0000 7fd18943bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584ce4e0d80 con 0x5584ce5f3930 2024-02-02T15:40:20.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.219+0000 7fd18943bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584ce4e0a20 con 0x5584ce5f3010 2024-02-02T15:40:20.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.219+0000 7fd18943bb80 10 client.0 _ll_register_callbacks cb 0x5584ce4e2290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:40:20.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.219+0000 7fd16e000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5584ce5f3930 0x5584ce5ff100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:20.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.219+0000 7fd16ea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584ce5f3010 0x5584ce5febc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:20.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.219+0000 7fd16e000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5584ce5f3930 0x5584ce5ff100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42294/0 (socket says 172.21.15.1:42294) 2024-02-02T15:40:20.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.219+0000 7fd16e000700 1 -- 172.21.15.1:0/3936268278 learned_addr learned my addr 172.21.15.1:0/3936268278 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:40:20.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.219+0000 7fd16f400700 1 --2- 172.21.15.1:0/3936268278 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5584ce5fe7e0 0x5584ce5ff660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:20.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.219+0000 7fd16e000700 1 -- 172.21.15.1:0/3936268278 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584ce5f3010 msgr2=0x5584ce5febc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:20.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.219+0000 7fd16e000700 1 --2- 172.21.15.1:0/3936268278 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584ce5f3010 0x5584ce5febc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:20.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.219+0000 7fd16e000700 1 -- 172.21.15.1:0/3936268278 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5584ce5fe7e0 msgr2=0x5584ce5ff660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:20.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.219+0000 7fd16e000700 1 --2- 172.21.15.1:0/3936268278 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5584ce5fe7e0 0x5584ce5ff660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:20.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.219+0000 7fd16e000700 1 -- 172.21.15.1:0/3936268278 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1701a3b80 con 0x5584ce5f3930 2024-02-02T15:40:20.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.219+0000 7fd16f400700 1 --2- 172.21.15.1:0/3936268278 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5584ce5fe7e0 0x5584ce5ff660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:40:20.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.219+0000 7fd16e000700 1 --2- 172.21.15.1:0/3936268278 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5584ce5f3930 0x5584ce5ff100 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fd15809a0a0 tx=0x7fd158005480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f79f09a2f5eb0240 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:20.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.219+0000 7fd15cc00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:40:20.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.220+0000 7fd15cc00700 1 -- 172.21.15.1:0/3936268278 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1580b6040 con 0x5584ce5f3930 2024-02-02T15:40:20.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.220+0000 7fd15cc00700 1 -- 172.21.15.1:0/3936268278 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fd15809df90 con 0x5584ce5f3930 2024-02-02T15:40:20.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.220+0000 7fd18943bb80 1 -- 172.21.15.1:0/3936268278 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5584ce53ddd0 con 0x5584ce5f3930 2024-02-02T15:40:20.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.220+0000 7fd15cc00700 1 -- 172.21.15.1:0/3936268278 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd1580a2140 con 0x5584ce5f3930 2024-02-02T15:40:20.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.220+0000 7fd18943bb80 10 client.29192 fetch_fsmap learned FSMap version 52 2024-02-02T15:40:20.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.220+0000 7fd18943bb80 1 -- 172.21.15.1:0/3936268278 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x5584ce372b60 con 0x5584ce5f3930 2024-02-02T15:40:20.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.220+0000 7fd15cc00700 1 -- 172.21.15.1:0/3936268278 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd1580a8020 con 0x5584ce5f3930 2024-02-02T15:40:20.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.220+0000 7fd15cc00700 1 -- 172.21.15.1:0/3936268278 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 52) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fd158015020 con 0x5584ce5f3930 2024-02-02T15:40:20.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.220+0000 7fd18943bb80 10 client.29192 fetch_fsmap finished waiting for FSMap version 52 2024-02-02T15:40:20.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.220+0000 7fd18943bb80 10 client.29192 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:40:20.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.220+0000 7fd18943bb80 20 client.29192 populate_metadata read hostname 'smithi001' 2024-02-02T15:40:20.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.220+0000 7fd18943bb80 1 --2- 172.21.15.1:0/3936268278 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5584ce60d9c0 0x5584ce60fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:20.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.220+0000 7fd18943bb80 4 client.29192 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:40:20.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.220+0000 7fd18943bb80 1 -- 172.21.15.1:0/3936268278 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5584ce503f50 con 0x5584ce60d9c0 2024-02-02T15:40:20.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.221+0000 7fd16ea00700 1 --2- 172.21.15.1:0/3936268278 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5584ce60d9c0 0x5584ce60fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:20.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.221+0000 7fd16ea00700 1 --2- 172.21.15.1:0/3936268278 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5584ce60d9c0 0x5584ce60fdb0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c8fab83464d440e server_cookie=99e32b6bbae57c5 in_seq=0 out_seq=0 2024-02-02T15:40:20.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.221+0000 7fd15cc00700 10 client.29192 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:40:20.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.222+0000 7fd15cc00700 1 -- 172.21.15.1:0/3936268278 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5584ce503f50 con 0x5584ce60d9c0 2024-02-02T15:40:20.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.222+0000 7fd15cc00700 10 client.29192 handle_command_reply: tid=0 2024-02-02T15:40:20.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.222+0000 7fd18943bb80 1 client.29192 shutdown 2024-02-02T15:40:20.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.222+0000 7fd18943bb80 1 -- 172.21.15.1:0/3936268278 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5584ce5f3930 msgr2=0x5584ce5ff100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:20.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.222+0000 7fd18943bb80 1 --2- 172.21.15.1:0/3936268278 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5584ce5f3930 0x5584ce5ff100 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fd15809a0a0 tx=0x7fd158005480 comp rx=0 tx=0).stop 2024-02-02T15:40:20.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.222+0000 7fd18943bb80 1 -- 172.21.15.1:0/3936268278 shutdown_connections 2024-02-02T15:40:20.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.222+0000 7fd18943bb80 1 --2- 172.21.15.1:0/3936268278 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5584ce60d9c0 0x5584ce60fdb0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:20.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.222+0000 7fd18943bb80 1 --2- 172.21.15.1:0/3936268278 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5584ce5fe7e0 0x5584ce5ff660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:20.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.222+0000 7fd18943bb80 1 --2- 172.21.15.1:0/3936268278 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584ce5f3010 0x5584ce5febc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:20.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.222+0000 7fd18943bb80 1 --2- 172.21.15.1:0/3936268278 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5584ce5f3930 0x5584ce5ff100 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:20.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.222+0000 7fd15cc00700 0 client.29192 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:40:20.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.222+0000 7fd18943bb80 1 -- 172.21.15.1:0/3936268278 >> 172.21.15.1:0/3936268278 conn(0x5584ce552db0 msgr2=0x5584ce5eefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:20.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.222+0000 7fd18943bb80 1 -- 172.21.15.1:0/3936268278 shutdown_connections 2024-02-02T15:40:20.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.222+0000 7fd18943bb80 1 -- 172.21.15.1:0/3936268278 wait complete. 2024-02-02T15:40:20.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.223+0000 7fd18943bb80 20 client.29192 trim_cache size 0 max 16384 2024-02-02T15:40:20.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.238+0000 7fd18943bb80 1 Processor -- start 2024-02-02T15:40:20.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.238+0000 7fd18943bb80 1 -- start start 2024-02-02T15:40:20.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.238+0000 7fd18943bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5584ce5f3010 0x5584ce603c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:20.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.238+0000 7fd18943bb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5584ce5f3930 0x5584ce5eeeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:20.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.238+0000 7fd18943bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584ce5fe7e0 0x5584ce5ef3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:20.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.238+0000 7fd18943bb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584ce3c86a0 con 0x5584ce5f3930 2024-02-02T15:40:20.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.238+0000 7fd18943bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584ce5ef930 con 0x5584ce5f3010 2024-02-02T15:40:20.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.238+0000 7fd18943bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584ce5f4290 con 0x5584ce5fe7e0 2024-02-02T15:40:20.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.238+0000 7fd16e000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5584ce5f3930 0x5584ce5eeeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:20.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.238+0000 7fd16e000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5584ce5f3930 0x5584ce5eeeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45938/0 (socket says 172.21.15.1:45938) 2024-02-02T15:40:20.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.238+0000 7fd16ea00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5584ce5f3010 0x5584ce603c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:20.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.238+0000 7fd16e000700 1 -- 172.21.15.1:0/2384133745 learned_addr learned my addr 172.21.15.1:0/2384133745 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:40:20.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.238+0000 7fd16f400700 1 --2- 172.21.15.1:0/2384133745 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584ce5fe7e0 0x5584ce5ef3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:20.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.239+0000 7fd16e000700 1 -- 172.21.15.1:0/2384133745 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584ce5fe7e0 msgr2=0x5584ce5ef3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:20.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.239+0000 7fd16e000700 1 --2- 172.21.15.1:0/2384133745 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584ce5fe7e0 0x5584ce5ef3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:20.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.239+0000 7fd16e000700 1 -- 172.21.15.1:0/2384133745 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5584ce5f3010 msgr2=0x5584ce603c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:20.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.239+0000 7fd16e000700 1 --2- 172.21.15.1:0/2384133745 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5584ce5f3010 0x5584ce603c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:20.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.239+0000 7fd16e000700 1 -- 172.21.15.1:0/2384133745 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5584ce50aea0 con 0x5584ce5f3930 2024-02-02T15:40:20.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.239+0000 7fd16ea00700 1 --2- 172.21.15.1:0/2384133745 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5584ce5f3010 0x5584ce603c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:40:20.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.239+0000 7fd16e000700 1 --2- 172.21.15.1:0/2384133745 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5584ce5f3930 0x5584ce5eeeb0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fd15802a030 tx=0x7fd158009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40a4157126684e91 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:20.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.239+0000 7fd164c00700 1 -- 172.21.15.1:0/2384133745 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1580b6040 con 0x5584ce5f3930 2024-02-02T15:40:20.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.239+0000 7fd164c00700 1 -- 172.21.15.1:0/2384133745 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd1580a8070 con 0x5584ce5f3930 2024-02-02T15:40:20.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.239+0000 7fd164c00700 1 -- 172.21.15.1:0/2384133745 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1580a2140 con 0x5584ce5f3930 2024-02-02T15:40:20.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.240+0000 7fd18943bb80 1 -- 172.21.15.1:0/2384133745 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5584ce5f3930 msgr2=0x5584ce5eeeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:20.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.240+0000 7fd18943bb80 1 --2- 172.21.15.1:0/2384133745 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5584ce5f3930 0x5584ce5eeeb0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fd15802a030 tx=0x7fd158009c00 comp rx=0 tx=0).stop 2024-02-02T15:40:20.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.240+0000 7fd18943bb80 1 -- 172.21.15.1:0/2384133745 shutdown_connections 2024-02-02T15:40:20.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.240+0000 7fd18943bb80 1 --2- 172.21.15.1:0/2384133745 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584ce5fe7e0 0x5584ce5ef3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:20.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.240+0000 7fd18943bb80 1 --2- 172.21.15.1:0/2384133745 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5584ce5f3010 0x5584ce603c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:20.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.240+0000 7fd18943bb80 1 --2- 172.21.15.1:0/2384133745 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5584ce5f3930 0x5584ce5eeeb0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:20.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.240+0000 7fd18943bb80 1 -- 172.21.15.1:0/2384133745 >> 172.21.15.1:0/2384133745 conn(0x5584ce552db0 msgr2=0x5584ce5edb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:20.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.240+0000 7fd18943bb80 1 -- 172.21.15.1:0/2384133745 shutdown_connections 2024-02-02T15:40:20.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.240+0000 7fd18943bb80 1 -- 172.21.15.1:0/2384133745 wait complete. 2024-02-02T15:40:20.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.240+0000 7fd18943bb80 1 -- start start 2024-02-02T15:40:20.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.241+0000 7fd18943bb80 1 Processor -- start 2024-02-02T15:40:20.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.241+0000 7fd18943bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5584ce5f3010 0x5584ce6533d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:20.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.241+0000 7fd18943bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584ce5f3930 0x5584ce653910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:20.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.241+0000 7fd18943bb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5584ce5fe7e0 0x5584ce65e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:20.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.241+0000 7fd18943bb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584ce6540a0 con 0x5584ce5fe7e0 2024-02-02T15:40:20.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.241+0000 7fd18943bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584ce654210 con 0x5584ce5f3010 2024-02-02T15:40:20.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.241+0000 7fd18943bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584ce65e990 con 0x5584ce5f3930 2024-02-02T15:40:20.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.241+0000 7fd18943bb80 10 client.0 _ll_register_callbacks cb 0x5584ce4e2290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:40:20.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.242+0000 7fd16ea00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5584ce5f3010 0x5584ce6533d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:20.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.242+0000 7fd16ea00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5584ce5f3010 0x5584ce6533d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42308/0 (socket says 172.21.15.1:42308) 2024-02-02T15:40:20.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.242+0000 7fd16ea00700 1 -- 172.21.15.1:0/2223860073 learned_addr learned my addr 172.21.15.1:0/2223860073 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:40:20.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.242+0000 7fd16e000700 1 --2- 172.21.15.1:0/2223860073 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584ce5f3930 0x5584ce653910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:20.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.242+0000 7fd16f400700 1 --2- 172.21.15.1:0/2223860073 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5584ce5fe7e0 0x5584ce65e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:20.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.242+0000 7fd16ea00700 1 -- 172.21.15.1:0/2223860073 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584ce5f3930 msgr2=0x5584ce653910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:20.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.242+0000 7fd16ea00700 1 --2- 172.21.15.1:0/2223860073 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584ce5f3930 0x5584ce653910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:20.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.242+0000 7fd16ea00700 1 -- 172.21.15.1:0/2223860073 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5584ce5fe7e0 msgr2=0x5584ce65e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:20.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.242+0000 7fd16ea00700 1 --2- 172.21.15.1:0/2223860073 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5584ce5fe7e0 0x5584ce65e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:20.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.242+0000 7fd16ea00700 1 -- 172.21.15.1:0/2223860073 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd160017070 con 0x5584ce5f3010 2024-02-02T15:40:20.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.242+0000 7fd16ea00700 1 --2- 172.21.15.1:0/2223860073 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5584ce5f3010 0x5584ce6533d0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x5584ce610620 tx=0x7fd1600047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b19d71f1e8d48955 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:20.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.242+0000 7fd15cc00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:40:20.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.243+0000 7fd15cc00700 1 -- 172.21.15.1:0/2223860073 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd160022040 con 0x5584ce5f3010 2024-02-02T15:40:20.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.243+0000 7fd15cc00700 1 -- 172.21.15.1:0/2223860073 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fd160003680 con 0x5584ce5f3010 2024-02-02T15:40:20.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.243+0000 7fd18943bb80 1 -- 172.21.15.1:0/2223860073 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5584ce5e70f0 con 0x5584ce5f3010 2024-02-02T15:40:20.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.243+0000 7fd15cc00700 1 -- 172.21.15.1:0/2223860073 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd16002b450 con 0x5584ce5f3010 2024-02-02T15:40:20.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.243+0000 7fd18943bb80 10 client.29198 fetch_fsmap learned FSMap version 52 2024-02-02T15:40:20.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.243+0000 7fd18943bb80 1 -- 172.21.15.1:0/2223860073 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x5584ce372b60 con 0x5584ce5f3010 2024-02-02T15:40:20.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.243+0000 7fd15cc00700 1 -- 172.21.15.1:0/2223860073 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5584ce5e70f0 con 0x5584ce5f3010 2024-02-02T15:40:20.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.243+0000 7fd15cc00700 1 -- 172.21.15.1:0/2223860073 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 52) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fd16001d970 con 0x5584ce5f3010 2024-02-02T15:40:20.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.243+0000 7fd18943bb80 10 client.29198 fetch_fsmap finished waiting for FSMap version 52 2024-02-02T15:40:20.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.243+0000 7fd18943bb80 10 client.29198 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:40:20.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.243+0000 7fd18943bb80 20 client.29198 populate_metadata read hostname 'smithi001' 2024-02-02T15:40:20.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.243+0000 7fd18943bb80 1 --2- 172.21.15.1:0/2223860073 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5584ce6600f0 0x5584ce662510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:20.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.243+0000 7fd18943bb80 4 client.29198 mds_command: new command op to 24557 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:40:20.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.243+0000 7fd18943bb80 1 -- 172.21.15.1:0/2223860073 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5584ce393cf0 con 0x5584ce6600f0 2024-02-02T15:40:20.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.244+0000 7fd16e000700 1 --2- 172.21.15.1:0/2223860073 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5584ce6600f0 0x5584ce662510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:20.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.244+0000 7fd15cc00700 10 client.29198 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:40:20.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.244+0000 7fd16e000700 1 --2- 172.21.15.1:0/2223860073 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5584ce6600f0 0x5584ce662510 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd4e0aafe80a9a5e server_cookie=4ebd59362763c2f9 in_seq=0 out_seq=0 2024-02-02T15:40:20.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.244+0000 7fd15cc00700 1 -- 172.21.15.1:0/2223860073 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5584ce393cf0 con 0x5584ce6600f0 2024-02-02T15:40:20.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.244+0000 7fd15cc00700 10 client.29198 handle_command_reply: tid=0 2024-02-02T15:40:20.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.244+0000 7fd18943bb80 1 client.29198 shutdown 2024-02-02T15:40:20.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.244+0000 7fd18943bb80 1 -- 172.21.15.1:0/2223860073 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5584ce5f3010 msgr2=0x5584ce6533d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:20.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.244+0000 7fd18943bb80 1 --2- 172.21.15.1:0/2223860073 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5584ce5f3010 0x5584ce6533d0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x5584ce610620 tx=0x7fd1600047c0 comp rx=0 tx=0).stop 2024-02-02T15:40:20.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.245+0000 7fd18943bb80 1 -- 172.21.15.1:0/2223860073 shutdown_connections 2024-02-02T15:40:20.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.245+0000 7fd18943bb80 1 --2- 172.21.15.1:0/2223860073 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5584ce6600f0 0x5584ce662510 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:20.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.245+0000 7fd18943bb80 1 --2- 172.21.15.1:0/2223860073 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5584ce5fe7e0 0x5584ce65e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:20.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.245+0000 7fd18943bb80 1 --2- 172.21.15.1:0/2223860073 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5584ce5f3010 0x5584ce6533d0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:20.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.245+0000 7fd18943bb80 1 --2- 172.21.15.1:0/2223860073 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584ce5f3930 0x5584ce653910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:20.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.245+0000 7fd15cc00700 0 client.29198 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:40:20.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.245+0000 7fd18943bb80 1 -- 172.21.15.1:0/2223860073 >> 172.21.15.1:0/2223860073 conn(0x5584ce552db0 msgr2=0x5584ce5f8c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:20.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.245+0000 7fd18943bb80 1 -- 172.21.15.1:0/2223860073 shutdown_connections 2024-02-02T15:40:20.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.245+0000 7fd18943bb80 1 -- 172.21.15.1:0/2223860073 wait complete. 2024-02-02T15:40:20.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.245+0000 7fd18943bb80 20 client.29198 trim_cache size 0 max 16384 2024-02-02T15:40:20.249 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:40:20.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.245+0000 7fd175a00700 1 -- 172.21.15.1:0/2346775424 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fd1500a0710 msgr2=0x7fd1500a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:20.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.245+0000 7fd175a00700 1 --2- 172.21.15.1:0/2346775424 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fd1500a0710 0x7fd1500a2bd0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd160005fd0 tx=0x7fd160009310 comp rx=0 tx=0).stop 2024-02-02T15:40:20.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.245+0000 7fd175a00700 1 -- 172.21.15.1:0/2346775424 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1700689b0 msgr2=0x7fd170197470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:20.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.245+0000 7fd175a00700 1 --2- 172.21.15.1:0/2346775424 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1700689b0 0x7fd170197470 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fd158009230 tx=0x7fd15800fa80 comp rx=0 tx=0).stop 2024-02-02T15:40:20.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.246+0000 7fd175a00700 1 -- 172.21.15.1:0/2346775424 shutdown_connections 2024-02-02T15:40:20.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.246+0000 7fd175a00700 1 --2- 172.21.15.1:0/2346775424 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fd1500a0710 0x7fd1500a2bd0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:20.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.246+0000 7fd175a00700 1 --2- 172.21.15.1:0/2346775424 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd170106ed0 0x7fd17019b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:20.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.246+0000 7fd175a00700 1 --2- 172.21.15.1:0/2346775424 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd170068010 0x7fd170196f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:20.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.246+0000 7fd175a00700 1 --2- 172.21.15.1:0/2346775424 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd1700689b0 0x7fd170197470 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:20.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.246+0000 7fd175a00700 1 -- 172.21.15.1:0/2346775424 >> 172.21.15.1:0/2346775424 conn(0x7fd1700757d0 msgr2=0x7fd1700fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:20.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.246+0000 7fd175a00700 1 -- 172.21.15.1:0/2346775424 shutdown_connections 2024-02-02T15:40:20.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:20.246+0000 7fd175a00700 1 -- 172.21.15.1:0/2346775424 wait complete. 2024-02-02T15:40:20.276 INFO:tasks.workunit.client.0.smithi001.stdout: 1 718935 5.28 MB/sec execute 594 sec latency 19.435 ms 2024-02-02T15:40:21.266 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-02T15:40:21.266 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-02T15:40:21.276 INFO:tasks.workunit.client.0.smithi001.stdout: 1 719005 5.27 MB/sec execute 595 sec latency 939.741 ms 2024-02-02T15:40:21.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.484+0000 7f24c7c00700 1 -- 172.21.15.1:0/96017495 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f24c0100ff0 msgr2=0x7f24c01013d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:21.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.484+0000 7f24c7c00700 1 --2- 172.21.15.1:0/96017495 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f24c0100ff0 0x7f24c01013d0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f24b0009990 tx=0x7f24b0009ca0 comp rx=0 tx=0).stop 2024-02-02T15:40:21.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.484+0000 7f24c7c00700 1 -- 172.21.15.1:0/96017495 shutdown_connections 2024-02-02T15:40:21.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.484+0000 7f24c7c00700 1 --2- 172.21.15.1:0/96017495 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f24c0106110 0x7f24c010dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:21.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.484+0000 7f24c7c00700 1 --2- 172.21.15.1:0/96017495 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f24c0100ff0 0x7f24c01013d0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:21.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.484+0000 7f24c7c00700 1 --2- 172.21.15.1:0/96017495 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24c01019a0 0x7f24c01059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:21.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.484+0000 7f24c7c00700 1 -- 172.21.15.1:0/96017495 >> 172.21.15.1:0/96017495 conn(0x7f24c00fc810 msgr2=0x7f24c00fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:21.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.484+0000 7f24c7c00700 1 -- 172.21.15.1:0/96017495 shutdown_connections 2024-02-02T15:40:21.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.484+0000 7f24c7c00700 1 -- 172.21.15.1:0/96017495 wait complete. 2024-02-02T15:40:21.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.485+0000 7f24c7c00700 1 Processor -- start 2024-02-02T15:40:21.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.485+0000 7f24c7c00700 1 -- start start 2024-02-02T15:40:21.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.486+0000 7f24c7c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24c0100ff0 0x7f24c0073040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:21.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.486+0000 7f24c7c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f24c0106110 0x7f24c0073580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:21.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.486+0000 7f24c7c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f24c0073ac0 0x7f24c01a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:21.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.486+0000 7f24c7c00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24c01a5790 con 0x7f24c0073ac0 2024-02-02T15:40:21.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.486+0000 7f24c7c00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24c01a5900 con 0x7f24c0100ff0 2024-02-02T15:40:21.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.486+0000 7f24c7c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24c01a5ac0 con 0x7f24c0106110 2024-02-02T15:40:21.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.486+0000 7f24c5200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24c0100ff0 0x7f24c0073040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:21.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.486+0000 7f24c5c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f24c0073ac0 0x7f24c01a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:21.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.486+0000 7f24c5200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24c0100ff0 0x7f24c0073040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42322/0 (socket says 172.21.15.1:42322) 2024-02-02T15:40:21.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.486+0000 7f24c5200700 1 -- 172.21.15.1:0/1004610850 learned_addr learned my addr 172.21.15.1:0/1004610850 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:40:21.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.486+0000 7f24bfe00700 1 --2- 172.21.15.1:0/1004610850 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f24c0106110 0x7f24c0073580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:21.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.487+0000 7f24c5200700 1 -- 172.21.15.1:0/1004610850 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f24c0106110 msgr2=0x7f24c0073580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:21.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.487+0000 7f24c5200700 1 --2- 172.21.15.1:0/1004610850 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f24c0106110 0x7f24c0073580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:21.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.487+0000 7f24c5200700 1 -- 172.21.15.1:0/1004610850 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f24c0073ac0 msgr2=0x7f24c01a5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:21.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.487+0000 7f24c5200700 1 --2- 172.21.15.1:0/1004610850 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f24c0073ac0 0x7f24c01a5100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:21.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.487+0000 7f24c5200700 1 -- 172.21.15.1:0/1004610850 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24b00095f0 con 0x7f24c0100ff0 2024-02-02T15:40:21.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.487+0000 7f24c5200700 1 --2- 172.21.15.1:0/1004610850 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24c0100ff0 0x7f24c0073040 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f24b0005910 tx=0x7f24b0004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:21.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.487+0000 7f24bd600700 1 -- 172.21.15.1:0/1004610850 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b001d070 con 0x7f24c0100ff0 2024-02-02T15:40:21.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.487+0000 7f24bd600700 1 -- 172.21.15.1:0/1004610850 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f24b0022470 con 0x7f24c0100ff0 2024-02-02T15:40:21.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.487+0000 7f24bd600700 1 -- 172.21.15.1:0/1004610850 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b0017650 con 0x7f24c0100ff0 2024-02-02T15:40:21.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.487+0000 7f24c7c00700 1 -- 172.21.15.1:0/1004610850 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24c01a5d40 con 0x7f24c0100ff0 2024-02-02T15:40:21.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.487+0000 7f24c7c00700 1 -- 172.21.15.1:0/1004610850 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24c01a6110 con 0x7f24c0100ff0 2024-02-02T15:40:21.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.490+0000 7f24bd600700 1 -- 172.21.15.1:0/1004610850 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f24b00225e0 con 0x7f24c0100ff0 2024-02-02T15:40:21.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.491+0000 7f24bd600700 1 --2- 172.21.15.1:0/1004610850 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f24a00a0690 0x7f24a00a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:21.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.491+0000 7f24bd600700 1 -- 172.21.15.1:0/1004610850 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f24b00cd530 con 0x7f24c0100ff0 2024-02-02T15:40:21.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.491+0000 7f24db62bb80 1 Processor -- start 2024-02-02T15:40:21.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.492+0000 7f24bfe00700 1 --2- 172.21.15.1:0/1004610850 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f24a00a0690 0x7f24a00a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:21.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.492+0000 7f24db62bb80 1 -- start start 2024-02-02T15:40:21.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.492+0000 7f24db62bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a01e072090 0x55a01e072470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:21.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.492+0000 7f24db62bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a01e0729b0 0x55a01e07d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:21.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.492+0000 7f24db62bb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a01e07d860 0x55a01e07fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:21.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.492+0000 7f24db62bb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a01dfe6540 con 0x55a01e07d860 2024-02-02T15:40:21.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.492+0000 7f24db62bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a01dfcab20 con 0x55a01e072090 2024-02-02T15:40:21.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.492+0000 7f24db62bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a01dfc84e0 con 0x55a01e0729b0 2024-02-02T15:40:21.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.492+0000 7f24bfe00700 1 --2- 172.21.15.1:0/1004610850 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f24a00a0690 0x7f24a00a2b50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f24c0074830 tx=0x7f24a80073d0 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:21.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.492+0000 7f24bfe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a01e0729b0 0x55a01e07d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:21.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.492+0000 7f24c5c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a01e07d860 0x55a01e07fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:21.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.492+0000 7f24bfe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a01e0729b0 0x55a01e07d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:40896/0 (socket says 172.21.15.1:40896) 2024-02-02T15:40:21.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.493+0000 7f24c5200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a01e072090 0x55a01e072470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:21.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.493+0000 7f24bfe00700 1 -- 172.21.15.1:0/1819461505 learned_addr learned my addr 172.21.15.1:0/1819461505 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:40:21.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.493+0000 7f24bfe00700 1 -- 172.21.15.1:0/1819461505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a01e072090 msgr2=0x55a01e072470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:21.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.493+0000 7f24bfe00700 1 --2- 172.21.15.1:0/1819461505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a01e072090 0x55a01e072470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:21.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.493+0000 7f24bfe00700 1 -- 172.21.15.1:0/1819461505 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a01e07d860 msgr2=0x55a01e07fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:21.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.493+0000 7f24bfe00700 1 --2- 172.21.15.1:0/1819461505 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a01e07d860 0x55a01e07fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:21.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.493+0000 7f24bfe00700 1 -- 172.21.15.1:0/1819461505 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a01de45430 con 0x55a01e0729b0 2024-02-02T15:40:21.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.493+0000 7f24bfe00700 1 --2- 172.21.15.1:0/1819461505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a01e0729b0 0x55a01e07d320 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f24a8004560 tx=0x7f24a8013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2af4060949a108f5 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:21.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.494+0000 7f24b6000700 1 -- 172.21.15.1:0/1819461505 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a8004590 con 0x55a01e0729b0 2024-02-02T15:40:21.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.494+0000 7f24b6000700 1 -- 172.21.15.1:0/1819461505 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f24a8003b40 con 0x55a01e0729b0 2024-02-02T15:40:21.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.494+0000 7f24b6000700 1 -- 172.21.15.1:0/1819461505 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a8011b10 con 0x55a01e0729b0 2024-02-02T15:40:21.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.494+0000 7f24db62bb80 1 -- 172.21.15.1:0/1819461505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a01e0729b0 msgr2=0x55a01e07d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:21.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.494+0000 7f24db62bb80 1 --2- 172.21.15.1:0/1819461505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a01e0729b0 0x55a01e07d320 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f24a8004560 tx=0x7f24a8013000 comp rx=0 tx=0).stop 2024-02-02T15:40:21.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.494+0000 7f24db62bb80 1 -- 172.21.15.1:0/1819461505 shutdown_connections 2024-02-02T15:40:21.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.494+0000 7f24db62bb80 1 --2- 172.21.15.1:0/1819461505 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a01e07d860 0x55a01e07fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:21.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.494+0000 7f24db62bb80 1 --2- 172.21.15.1:0/1819461505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a01e072090 0x55a01e072470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:21.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.494+0000 7f24db62bb80 1 --2- 172.21.15.1:0/1819461505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a01e0729b0 0x55a01e07d320 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:21.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.494+0000 7f24db62bb80 1 -- 172.21.15.1:0/1819461505 >> 172.21.15.1:0/1819461505 conn(0x55a01dfd1e30 msgr2=0x55a01e06e430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:21.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.494+0000 7f24db62bb80 1 -- 172.21.15.1:0/1819461505 shutdown_connections 2024-02-02T15:40:21.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.494+0000 7f24db62bb80 1 -- 172.21.15.1:0/1819461505 wait complete. 2024-02-02T15:40:21.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.495+0000 7f24db62bb80 1 -- start start 2024-02-02T15:40:21.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.495+0000 7f24db62bb80 1 Processor -- start 2024-02-02T15:40:21.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.496+0000 7f24db62bb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a01e072090 0x55a01e07d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:21.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.496+0000 7f24db62bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a01e0729b0 0x55a01e07a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:21.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.496+0000 7f24db62bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a01e07d860 0x55a01e07ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:21.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.496+0000 7f24db62bb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a01dfc3dd0 con 0x55a01e072090 2024-02-02T15:40:21.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.496+0000 7f24db62bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a01df5fe00 con 0x55a01e07d860 2024-02-02T15:40:21.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.496+0000 7f24db62bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a01df5faa0 con 0x55a01e0729b0 2024-02-02T15:40:21.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.496+0000 7f24db62bb80 10 client.0 _ll_register_callbacks cb 0x55a01df61310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:40:21.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.496+0000 7f24c5200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a01e072090 0x55a01e07d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:21.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.496+0000 7f24c5200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a01e072090 0x55a01e07d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46008/0 (socket says 172.21.15.1:46008) 2024-02-02T15:40:21.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.496+0000 7f24c5200700 1 -- 172.21.15.1:0/2519756652 learned_addr learned my addr 172.21.15.1:0/2519756652 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:40:21.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.496+0000 7f24c5c00700 1 --2- 172.21.15.1:0/2519756652 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a01e07d860 0x55a01e07ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:21.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.496+0000 7f24bfe00700 1 --2- 172.21.15.1:0/2519756652 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a01e0729b0 0x55a01e07a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:21.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.496+0000 7f24c5200700 1 -- 172.21.15.1:0/2519756652 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a01e0729b0 msgr2=0x55a01e07a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:21.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.496+0000 7f24c5200700 1 --2- 172.21.15.1:0/2519756652 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a01e0729b0 0x55a01e07a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:21.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.496+0000 7f24c5200700 1 -- 172.21.15.1:0/2519756652 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a01e07d860 msgr2=0x55a01e07ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:21.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.496+0000 7f24c5200700 1 --2- 172.21.15.1:0/2519756652 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a01e07d860 0x55a01e07ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:21.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.496+0000 7f24c5200700 1 -- 172.21.15.1:0/2519756652 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24a8004240 con 0x55a01e072090 2024-02-02T15:40:21.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.496+0000 7f24c5c00700 1 --2- 172.21.15.1:0/2519756652 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a01e07d860 0x55a01e07ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:40:21.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.497+0000 7f24c5200700 1 --2- 172.21.15.1:0/2519756652 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a01e072090 0x55a01e07d200 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f24b00ca8e0 tx=0x7f24b009a5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62a634bf910dff2 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:21.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.497+0000 7f24aea00700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:40:21.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.497+0000 7f24aea00700 1 -- 172.21.15.1:0/2519756652 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b00cd370 con 0x55a01e072090 2024-02-02T15:40:21.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.497+0000 7f24aea00700 1 -- 172.21.15.1:0/2519756652 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f24b009a6e0 con 0x55a01e072090 2024-02-02T15:40:21.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.497+0000 7f24db62bb80 1 -- 172.21.15.1:0/2519756652 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a01e066170 con 0x55a01e072090 2024-02-02T15:40:21.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.497+0000 7f24aea00700 1 -- 172.21.15.1:0/2519756652 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f24b009e510 con 0x55a01e072090 2024-02-02T15:40:21.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.497+0000 7f24db62bb80 10 client.19107 fetch_fsmap learned FSMap version 52 2024-02-02T15:40:21.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.497+0000 7f24db62bb80 1 -- 172.21.15.1:0/2519756652 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x55a01ddf1be0 con 0x55a01e072090 2024-02-02T15:40:21.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.497+0000 7f24aea00700 1 -- 172.21.15.1:0/2519756652 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a01e066170 con 0x55a01e072090 2024-02-02T15:40:21.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.498+0000 7f24aea00700 1 -- 172.21.15.1:0/2519756652 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 52) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f24b0099bd0 con 0x55a01e072090 2024-02-02T15:40:21.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.498+0000 7f24db62bb80 10 client.19107 fetch_fsmap finished waiting for FSMap version 52 2024-02-02T15:40:21.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.498+0000 7f24db62bb80 10 client.19107 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:40:21.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.498+0000 7f24db62bb80 20 client.19107 populate_metadata read hostname 'smithi001' 2024-02-02T15:40:21.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.498+0000 7f24db62bb80 1 --2- 172.21.15.1:0/2519756652 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55a01e08cf50 0x55a01e08f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:21.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.498+0000 7f24db62bb80 4 client.19107 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:40:21.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.498+0000 7f24db62bb80 1 -- 172.21.15.1:0/2519756652 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a01dfbce50 con 0x55a01e08cf50 2024-02-02T15:40:21.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.498+0000 7f24bfe00700 1 --2- 172.21.15.1:0/2519756652 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55a01e08cf50 0x55a01e08f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:21.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.499+0000 7f24bfe00700 1 --2- 172.21.15.1:0/2519756652 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55a01e08cf50 0x55a01e08f340 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e16ce9f665383c79 server_cookie=b8f940668403dc0e in_seq=0 out_seq=0 2024-02-02T15:40:21.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.499+0000 7f24aea00700 10 client.19107 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:40:21.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.499+0000 7f24aea00700 1 -- 172.21.15.1:0/2519756652 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55a01dfbce50 con 0x55a01e08cf50 2024-02-02T15:40:21.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.499+0000 7f24aea00700 10 client.19107 handle_command_reply: tid=0 2024-02-02T15:40:21.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.500+0000 7f24db62bb80 1 client.19107 shutdown 2024-02-02T15:40:21.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.500+0000 7f24db62bb80 1 -- 172.21.15.1:0/2519756652 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a01e072090 msgr2=0x55a01e07d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:21.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.500+0000 7f24db62bb80 1 --2- 172.21.15.1:0/2519756652 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a01e072090 0x55a01e07d200 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f24b00ca8e0 tx=0x7f24b009a5e0 comp rx=0 tx=0).stop 2024-02-02T15:40:21.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.500+0000 7f24db62bb80 1 -- 172.21.15.1:0/2519756652 shutdown_connections 2024-02-02T15:40:21.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.500+0000 7f24db62bb80 1 --2- 172.21.15.1:0/2519756652 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55a01e08cf50 0x55a01e08f340 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:21.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.500+0000 7f24db62bb80 1 --2- 172.21.15.1:0/2519756652 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a01e07d860 0x55a01e07ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:21.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.500+0000 7f24db62bb80 1 --2- 172.21.15.1:0/2519756652 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a01e072090 0x55a01e07d200 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:21.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.500+0000 7f24db62bb80 1 --2- 172.21.15.1:0/2519756652 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a01e0729b0 0x55a01e07a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:21.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.500+0000 7f24aea00700 0 client.19107 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:40:21.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.500+0000 7f24db62bb80 1 -- 172.21.15.1:0/2519756652 >> 172.21.15.1:0/2519756652 conn(0x55a01dfd1e30 msgr2=0x55a01e06e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:21.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.500+0000 7f24db62bb80 1 -- 172.21.15.1:0/2519756652 shutdown_connections 2024-02-02T15:40:21.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.500+0000 7f24db62bb80 1 -- 172.21.15.1:0/2519756652 wait complete. 2024-02-02T15:40:21.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.500+0000 7f24db62bb80 20 client.19107 trim_cache size 0 max 16384 2024-02-02T15:40:21.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.517+0000 7f24db62bb80 1 Processor -- start 2024-02-02T15:40:21.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.517+0000 7f24db62bb80 1 -- start start 2024-02-02T15:40:21.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.518+0000 7f24db62bb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a01e072090 0x55a01e06e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:21.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.518+0000 7f24db62bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a01e0729b0 0x55a01e0843f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:21.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.518+0000 7f24db62bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a01e07d860 0x55a01e07cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:21.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.518+0000 7f24db62bb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a01de47720 con 0x55a01e072090 2024-02-02T15:40:21.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.518+0000 7f24db62bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a01e07ba60 con 0x55a01e07d860 2024-02-02T15:40:21.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.518+0000 7f24db62bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a01e066070 con 0x55a01e0729b0 2024-02-02T15:40:21.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.518+0000 7f24bfe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a01e0729b0 0x55a01e0843f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:21.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.518+0000 7f24bfe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a01e0729b0 0x55a01e0843f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:40912/0 (socket says 172.21.15.1:40912) 2024-02-02T15:40:21.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.518+0000 7f24bfe00700 1 -- 172.21.15.1:0/2955640963 learned_addr learned my addr 172.21.15.1:0/2955640963 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:40:21.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.518+0000 7f24c5200700 1 --2- 172.21.15.1:0/2955640963 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a01e072090 0x55a01e06e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:21.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.518+0000 7f24c5c00700 1 --2- 172.21.15.1:0/2955640963 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a01e07d860 0x55a01e07cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:21.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.518+0000 7f24bfe00700 1 -- 172.21.15.1:0/2955640963 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a01e07d860 msgr2=0x55a01e07cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:21.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.518+0000 7f24bfe00700 1 --2- 172.21.15.1:0/2955640963 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a01e07d860 0x55a01e07cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:21.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.518+0000 7f24bfe00700 1 -- 172.21.15.1:0/2955640963 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a01e072090 msgr2=0x55a01e06e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:21.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.518+0000 7f24bfe00700 1 --2- 172.21.15.1:0/2955640963 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a01e072090 0x55a01e06e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:21.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.518+0000 7f24bfe00700 1 -- 172.21.15.1:0/2955640963 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a01df89f20 con 0x55a01e0729b0 2024-02-02T15:40:21.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.519+0000 7f24bfe00700 1 --2- 172.21.15.1:0/2955640963 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a01e0729b0 0x55a01e0843f0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f24a8004560 tx=0x7f24a80071c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b97dd0f90072fb6 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:21.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.519+0000 7f24b6a00700 1 -- 172.21.15.1:0/2955640963 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a8030040 con 0x55a01e0729b0 2024-02-02T15:40:21.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.519+0000 7f24b6a00700 1 -- 172.21.15.1:0/2955640963 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f24a8034330 con 0x55a01e0729b0 2024-02-02T15:40:21.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.519+0000 7f24b6a00700 1 -- 172.21.15.1:0/2955640963 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a80345b0 con 0x55a01e0729b0 2024-02-02T15:40:21.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.520+0000 7f24db62bb80 1 -- 172.21.15.1:0/2955640963 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a01e0729b0 msgr2=0x55a01e0843f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:21.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.520+0000 7f24db62bb80 1 --2- 172.21.15.1:0/2955640963 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a01e0729b0 0x55a01e0843f0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f24a8004560 tx=0x7f24a80071c0 comp rx=0 tx=0).stop 2024-02-02T15:40:21.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.520+0000 7f24db62bb80 1 -- 172.21.15.1:0/2955640963 shutdown_connections 2024-02-02T15:40:21.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.520+0000 7f24db62bb80 1 --2- 172.21.15.1:0/2955640963 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a01e07d860 0x55a01e07cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:21.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.520+0000 7f24db62bb80 1 --2- 172.21.15.1:0/2955640963 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a01e072090 0x55a01e06e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:21.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.520+0000 7f24db62bb80 1 --2- 172.21.15.1:0/2955640963 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a01e0729b0 0x55a01e0843f0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:21.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.520+0000 7f24db62bb80 1 -- 172.21.15.1:0/2955640963 >> 172.21.15.1:0/2955640963 conn(0x55a01dfd1e30 msgr2=0x55a01e06ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:21.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.520+0000 7f24db62bb80 1 -- 172.21.15.1:0/2955640963 shutdown_connections 2024-02-02T15:40:21.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.520+0000 7f24db62bb80 1 -- 172.21.15.1:0/2955640963 wait complete. 2024-02-02T15:40:21.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.520+0000 7f24db62bb80 1 -- start start 2024-02-02T15:40:21.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.520+0000 7f24db62bb80 1 Processor -- start 2024-02-02T15:40:21.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.521+0000 7f24db62bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a01e072090 0x55a01e0d8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:21.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.521+0000 7f24db62bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a01e0729b0 0x55a01e0d8d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:21.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.521+0000 7f24db62bb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a01e07d860 0x55a01e0d2b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:21.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.521+0000 7f24db62bb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a01e0d3110 con 0x55a01e07d860 2024-02-02T15:40:21.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.521+0000 7f24db62bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a01e0d3280 con 0x55a01e0729b0 2024-02-02T15:40:21.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.521+0000 7f24db62bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a01e0d3440 con 0x55a01e072090 2024-02-02T15:40:21.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.521+0000 7f24db62bb80 10 client.0 _ll_register_callbacks cb 0x55a01df61310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:40:21.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.521+0000 7f24c5c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a01e07d860 0x55a01e0d2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:21.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.521+0000 7f24c5200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a01e072090 0x55a01e0d8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:21.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.521+0000 7f24c5200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a01e072090 0x55a01e0d8840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:40914/0 (socket says 172.21.15.1:40914) 2024-02-02T15:40:21.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.521+0000 7f24c5c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a01e07d860 0x55a01e0d2b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46022/0 (socket says 172.21.15.1:46022) 2024-02-02T15:40:21.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.521+0000 7f24c5200700 1 -- 172.21.15.1:0/3927306808 learned_addr learned my addr 172.21.15.1:0/3927306808 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:40:21.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.521+0000 7f24bfe00700 1 --2- 172.21.15.1:0/3927306808 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a01e0729b0 0x55a01e0d8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:21.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.522+0000 7f24c5200700 1 -- 172.21.15.1:0/3927306808 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a01e0729b0 msgr2=0x55a01e0d8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:21.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.522+0000 7f24c5200700 1 --2- 172.21.15.1:0/3927306808 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a01e0729b0 0x55a01e0d8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:21.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.522+0000 7f24c5200700 1 -- 172.21.15.1:0/3927306808 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a01e07d860 msgr2=0x55a01e0d2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:21.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.522+0000 7f24c5200700 1 --2- 172.21.15.1:0/3927306808 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a01e07d860 0x55a01e0d2b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:21.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.522+0000 7f24c5200700 1 -- 172.21.15.1:0/3927306808 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a01de45430 con 0x55a01e072090 2024-02-02T15:40:21.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.522+0000 7f24c5c00700 1 --2- 172.21.15.1:0/3927306808 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a01e07d860 0x55a01e0d2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:40:21.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.522+0000 7f24c5200700 1 --2- 172.21.15.1:0/3927306808 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a01e072090 0x55a01e0d8840 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f24b0006010 tx=0x7f24b000b390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e20b9f774a1d5c9 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:21.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.522+0000 7f24aea00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:40:21.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.522+0000 7f24aea00700 1 -- 172.21.15.1:0/3927306808 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b00a2230 con 0x55a01e072090 2024-02-02T15:40:21.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.522+0000 7f24aea00700 1 -- 172.21.15.1:0/3927306808 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f24b009a6e0 con 0x55a01e072090 2024-02-02T15:40:21.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.522+0000 7f24db62bb80 1 -- 172.21.15.1:0/3927306808 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f24a8003b40 con 0x55a01e072090 2024-02-02T15:40:21.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.522+0000 7f24aea00700 1 -- 172.21.15.1:0/3927306808 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f24b009a040 con 0x55a01e072090 2024-02-02T15:40:21.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.522+0000 7f24db62bb80 10 client.29212 fetch_fsmap learned FSMap version 52 2024-02-02T15:40:21.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.522+0000 7f24db62bb80 1 -- 172.21.15.1:0/3927306808 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x55a01de49d50 con 0x55a01e072090 2024-02-02T15:40:21.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.522+0000 7f24aea00700 1 -- 172.21.15.1:0/3927306808 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f24b0083030 con 0x55a01e072090 2024-02-02T15:40:21.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.523+0000 7f24aea00700 1 -- 172.21.15.1:0/3927306808 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 52) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f24b009a940 con 0x55a01e072090 2024-02-02T15:40:21.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.523+0000 7f24db62bb80 10 client.29212 fetch_fsmap finished waiting for FSMap version 52 2024-02-02T15:40:21.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.523+0000 7f24db62bb80 10 client.29212 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:40:21.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.523+0000 7f24db62bb80 20 client.29212 populate_metadata read hostname 'smithi001' 2024-02-02T15:40:21.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.523+0000 7f24db62bb80 1 --2- 172.21.15.1:0/3927306808 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55a01e0e01c0 0x55a01e0e25b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:21.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.523+0000 7f24db62bb80 4 client.29212 mds_command: new command op to 24557 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-02T15:40:21.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.523+0000 7f24db62bb80 1 -- 172.21.15.1:0/3927306808 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55a01df82fd0 con 0x55a01e0e01c0 2024-02-02T15:40:21.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.523+0000 7f24bfe00700 1 --2- 172.21.15.1:0/3927306808 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55a01e0e01c0 0x55a01e0e25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:21.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.523+0000 7f24bfe00700 1 --2- 172.21.15.1:0/3927306808 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55a01e0e01c0 0x55a01e0e25b0 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41e996e915c0248f server_cookie=1c8d9a6836f27aa6 in_seq=0 out_seq=0 2024-02-02T15:40:21.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.523+0000 7f24aea00700 10 client.29212 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:40:21.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.525+0000 7f24aea00700 1 -- 172.21.15.1:0/3927306808 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55a01df82fd0 con 0x55a01e0e01c0 2024-02-02T15:40:21.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.525+0000 7f24aea00700 10 client.29212 handle_command_reply: tid=0 2024-02-02T15:40:21.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.525+0000 7f24db62bb80 1 client.29212 shutdown 2024-02-02T15:40:21.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.525+0000 7f24db62bb80 1 -- 172.21.15.1:0/3927306808 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a01e072090 msgr2=0x55a01e0d8840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:21.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.525+0000 7f24db62bb80 1 --2- 172.21.15.1:0/3927306808 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a01e072090 0x55a01e0d8840 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f24b0006010 tx=0x7f24b000b390 comp rx=0 tx=0).stop 2024-02-02T15:40:21.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.525+0000 7f24db62bb80 1 -- 172.21.15.1:0/3927306808 shutdown_connections 2024-02-02T15:40:21.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.525+0000 7f24db62bb80 1 --2- 172.21.15.1:0/3927306808 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55a01e0e01c0 0x55a01e0e25b0 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:21.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.525+0000 7f24db62bb80 1 --2- 172.21.15.1:0/3927306808 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a01e07d860 0x55a01e0d2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:21.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.525+0000 7f24db62bb80 1 --2- 172.21.15.1:0/3927306808 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a01e072090 0x55a01e0d8840 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:21.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.525+0000 7f24db62bb80 1 --2- 172.21.15.1:0/3927306808 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a01e0729b0 0x55a01e0d8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:21.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.525+0000 7f24aea00700 0 client.29212 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:40:21.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.525+0000 7f24db62bb80 1 -- 172.21.15.1:0/3927306808 >> 172.21.15.1:0/3927306808 conn(0x55a01dfd1e30 msgr2=0x55a01e07bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:21.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.525+0000 7f24db62bb80 1 -- 172.21.15.1:0/3927306808 shutdown_connections 2024-02-02T15:40:21.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.525+0000 7f24db62bb80 1 -- 172.21.15.1:0/3927306808 wait complete. 2024-02-02T15:40:21.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.526+0000 7f24db62bb80 20 client.29212 trim_cache size 0 max 16384 2024-02-02T15:40:21.529 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:40:21.529 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2024-02-02T15:40:21.529 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "ff3a6039-bd29-449f-922d-dd77a3b760fa", 2024-02-02T15:40:21.529 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2024-02-02T15:40:21.529 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:40:21.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.526+0000 7f24c7c00700 1 -- 172.21.15.1:0/1004610850 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f24a00a0690 msgr2=0x7f24a00a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:21.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.526+0000 7f24c7c00700 1 --2- 172.21.15.1:0/1004610850 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f24a00a0690 0x7f24a00a2b50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f24c0074830 tx=0x7f24a80073d0 comp rx=0 tx=0).stop 2024-02-02T15:40:21.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.526+0000 7f24c7c00700 1 -- 172.21.15.1:0/1004610850 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24c0100ff0 msgr2=0x7f24c0073040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:21.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.526+0000 7f24c7c00700 1 --2- 172.21.15.1:0/1004610850 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24c0100ff0 0x7f24c0073040 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f24b0005910 tx=0x7f24b0004730 comp rx=0 tx=0).stop 2024-02-02T15:40:21.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.526+0000 7f24c7c00700 1 -- 172.21.15.1:0/1004610850 shutdown_connections 2024-02-02T15:40:21.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.526+0000 7f24c7c00700 1 --2- 172.21.15.1:0/1004610850 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f24a00a0690 0x7f24a00a2b50 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:21.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.526+0000 7f24c7c00700 1 --2- 172.21.15.1:0/1004610850 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f24c0073ac0 0x7f24c01a5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:21.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.526+0000 7f24c7c00700 1 --2- 172.21.15.1:0/1004610850 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f24c0100ff0 0x7f24c0073040 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:21.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.526+0000 7f24c7c00700 1 --2- 172.21.15.1:0/1004610850 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f24c0106110 0x7f24c0073580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:21.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.526+0000 7f24c7c00700 1 -- 172.21.15.1:0/1004610850 >> 172.21.15.1:0/1004610850 conn(0x7f24c00fc810 msgr2=0x7f24c00fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:21.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.526+0000 7f24c7c00700 1 -- 172.21.15.1:0/1004610850 shutdown_connections 2024-02-02T15:40:21.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.526+0000 7f24c7c00700 1 -- 172.21.15.1:0/1004610850 wait complete. 2024-02-02T15:40:21.545 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-02T15:40:21.644 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:21 smithi001 ceph-mon[35470]: pgmap v417: 129 pgs: 129 active+clean; 607 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 135 op/s 2024-02-02T15:40:21.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.801+0000 7f90e4400700 1 -- 172.21.15.1:0/3110295465 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90dc103230 msgr2=0x7f90dc103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:21.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.801+0000 7f90e4400700 1 --2- 172.21.15.1:0/3110295465 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90dc103230 0x7f90dc103610 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f90cc009990 tx=0x7f90cc009ca0 comp rx=0 tx=0).stop 2024-02-02T15:40:21.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.802+0000 7f90e4400700 1 -- 172.21.15.1:0/3110295465 shutdown_connections 2024-02-02T15:40:21.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.802+0000 7f90e4400700 1 --2- 172.21.15.1:0/3110295465 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90dc108350 0x7f90dc10fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:21.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.802+0000 7f90e4400700 1 --2- 172.21.15.1:0/3110295465 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90dc103230 0x7f90dc103610 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:21.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.802+0000 7f90e4400700 1 --2- 172.21.15.1:0/3110295465 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90dc103be0 0x7f90dc107c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:21.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.802+0000 7f90e4400700 1 -- 172.21.15.1:0/3110295465 >> 172.21.15.1:0/3110295465 conn(0x7f90dc0fea70 msgr2=0x7f90dc100e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:21.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.802+0000 7f90e4400700 1 -- 172.21.15.1:0/3110295465 shutdown_connections 2024-02-02T15:40:21.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.802+0000 7f90e4400700 1 -- 172.21.15.1:0/3110295465 wait complete. 2024-02-02T15:40:21.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.803+0000 7f90e4400700 1 Processor -- start 2024-02-02T15:40:21.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.803+0000 7f90e4400700 1 -- start start 2024-02-02T15:40:21.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.803+0000 7f90e4400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90dc103230 0x7f90dc199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:21.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.803+0000 7f90e4400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90dc108350 0x7f90dc199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:21.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.803+0000 7f90e4400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90dc19dad0 0x7f90dc19df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:21.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.803+0000 7f90e4400700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90dc19e550 con 0x7f90dc19dad0 2024-02-02T15:40:21.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.803+0000 7f90e4400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90dc19e6c0 con 0x7f90dc108350 2024-02-02T15:40:21.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.803+0000 7f90e4400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90dc19e880 con 0x7f90dc103230 2024-02-02T15:40:21.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.804+0000 7f90e1a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90dc103230 0x7f90dc199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:21.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.804+0000 7f90e1a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90dc103230 0x7f90dc199130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:40938/0 (socket says 172.21.15.1:40938) 2024-02-02T15:40:21.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.804+0000 7f90e1a00700 1 -- 172.21.15.1:0/1425582201 learned_addr learned my addr 172.21.15.1:0/1425582201 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:40:21.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.804+0000 7f90e1000700 1 --2- 172.21.15.1:0/1425582201 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90dc108350 0x7f90dc199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:21.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.804+0000 7f90e2400700 1 --2- 172.21.15.1:0/1425582201 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90dc19dad0 0x7f90dc19df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:21.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.804+0000 7f90e1000700 1 -- 172.21.15.1:0/1425582201 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90dc103230 msgr2=0x7f90dc199130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:21.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.804+0000 7f90e1000700 1 --2- 172.21.15.1:0/1425582201 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90dc103230 0x7f90dc199130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:21.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.804+0000 7f90e1000700 1 -- 172.21.15.1:0/1425582201 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90dc19dad0 msgr2=0x7f90dc19df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:21.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.804+0000 7f90e1000700 1 --2- 172.21.15.1:0/1425582201 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90dc19dad0 0x7f90dc19df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:21.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.804+0000 7f90e1000700 1 -- 172.21.15.1:0/1425582201 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90cc0095f0 con 0x7f90dc108350 2024-02-02T15:40:21.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.804+0000 7f90e1a00700 1 --2- 172.21.15.1:0/1425582201 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90dc103230 0x7f90dc199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:40:21.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.804+0000 7f90e1000700 1 --2- 172.21.15.1:0/1425582201 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90dc108350 0x7f90dc199670 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f90c400e9c0 tx=0x7f90c400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:21.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.805+0000 7f90da000700 1 -- 172.21.15.1:0/1425582201 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c400c9c0 con 0x7f90dc108350 2024-02-02T15:40:21.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.805+0000 7f90da000700 1 -- 172.21.15.1:0/1425582201 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f90c4010430 con 0x7f90dc108350 2024-02-02T15:40:21.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.805+0000 7f90da000700 1 -- 172.21.15.1:0/1425582201 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c4018680 con 0x7f90dc108350 2024-02-02T15:40:21.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.805+0000 7f90e4400700 1 -- 172.21.15.1:0/1425582201 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90dc1a5de0 con 0x7f90dc108350 2024-02-02T15:40:21.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.805+0000 7f90e4400700 1 -- 172.21.15.1:0/1425582201 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90dc1a6210 con 0x7f90dc108350 2024-02-02T15:40:21.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.808+0000 7f90da000700 1 -- 172.21.15.1:0/1425582201 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f90c40105a0 con 0x7f90dc108350 2024-02-02T15:40:21.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.810+0000 7f90f7ef8b80 1 Processor -- start 2024-02-02T15:40:21.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.810+0000 7f90da000700 1 --2- 172.21.15.1:0/1425582201 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f90bc0a0690 0x7f90bc0a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:21.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.810+0000 7f90f7ef8b80 1 -- start start 2024-02-02T15:40:21.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.810+0000 7f90da000700 1 -- 172.21.15.1:0/1425582201 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f90c4014070 con 0x7f90dc108350 2024-02-02T15:40:21.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.810+0000 7f90e2400700 1 --2- 172.21.15.1:0/1425582201 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f90bc0a0690 0x7f90bc0a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:21.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.810+0000 7f90f7ef8b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56549ea2b010 0x56549ea2b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:21.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.811+0000 7f90f7ef8b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56549ea2b930 0x56549ea362a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:21.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.811+0000 7f90f7ef8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56549ea367e0 0x56549ea38bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:21.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.811+0000 7f90f7ef8b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56549e99f4c0 con 0x56549ea2b010 2024-02-02T15:40:21.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.811+0000 7f90f7ef8b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56549e983aa0 con 0x56549ea2b930 2024-02-02T15:40:21.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.811+0000 7f90f7ef8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56549e981460 con 0x56549ea367e0 2024-02-02T15:40:21.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.811+0000 7f90e1000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56549ea2b930 0x56549ea362a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:21.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.811+0000 7f90e1a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56549ea2b010 0x56549ea2b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:21.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.811+0000 7f90e1000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56549ea2b930 0x56549ea362a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42388/0 (socket says 172.21.15.1:42388) 2024-02-02T15:40:21.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.811+0000 7f90e2400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56549ea367e0 0x56549ea38bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:21.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.811+0000 7f90e1000700 1 -- 172.21.15.1:0/224201304 learned_addr learned my addr 172.21.15.1:0/224201304 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:40:21.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.811+0000 7f90e2400700 1 --2- 172.21.15.1:0/1425582201 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f90bc0a0690 0x7f90bc0a2b50 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f90d400f590 tx=0x7f90d400b040 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:21.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.811+0000 7f90e1000700 1 -- 172.21.15.1:0/224201304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56549ea367e0 msgr2=0x56549ea38bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:21.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.811+0000 7f90e1000700 1 --2- 172.21.15.1:0/224201304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56549ea367e0 0x56549ea38bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:21.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.811+0000 7f90e1000700 1 -- 172.21.15.1:0/224201304 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56549ea2b010 msgr2=0x56549ea2b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:21.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.811+0000 7f90e1000700 1 --2- 172.21.15.1:0/224201304 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56549ea2b010 0x56549ea2b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:21.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.811+0000 7f90e1000700 1 -- 172.21.15.1:0/224201304 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56549e7fe3b0 con 0x56549ea2b930 2024-02-02T15:40:21.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.812+0000 7f90e1000700 1 --2- 172.21.15.1:0/224201304 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56549ea2b930 0x56549ea362a0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f90c4099350 tx=0x7f90c4098f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4df6a0e9bdde101 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:21.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.812+0000 7f90d2a00700 1 -- 172.21.15.1:0/224201304 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c4084070 con 0x56549ea2b930 2024-02-02T15:40:21.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.812+0000 7f90d2a00700 1 -- 172.21.15.1:0/224201304 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f90c4099ac0 con 0x56549ea2b930 2024-02-02T15:40:21.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.812+0000 7f90d2a00700 1 -- 172.21.15.1:0/224201304 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c4099620 con 0x56549ea2b930 2024-02-02T15:40:21.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.812+0000 7f90f7ef8b80 1 -- 172.21.15.1:0/224201304 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56549ea2b930 msgr2=0x56549ea362a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:21.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.812+0000 7f90f7ef8b80 1 --2- 172.21.15.1:0/224201304 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56549ea2b930 0x56549ea362a0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f90c4099350 tx=0x7f90c4098f00 comp rx=0 tx=0).stop 2024-02-02T15:40:21.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.813+0000 7f90f7ef8b80 1 -- 172.21.15.1:0/224201304 shutdown_connections 2024-02-02T15:40:21.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.813+0000 7f90f7ef8b80 1 --2- 172.21.15.1:0/224201304 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56549ea367e0 0x56549ea38bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:21.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.813+0000 7f90f7ef8b80 1 --2- 172.21.15.1:0/224201304 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56549ea2b010 0x56549ea2b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:21.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.813+0000 7f90f7ef8b80 1 --2- 172.21.15.1:0/224201304 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56549ea2b930 0x56549ea362a0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:21.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.813+0000 7f90f7ef8b80 1 -- 172.21.15.1:0/224201304 >> 172.21.15.1:0/224201304 conn(0x56549e98adb0 msgr2=0x56549ea273b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:21.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.813+0000 7f90f7ef8b80 1 -- 172.21.15.1:0/224201304 shutdown_connections 2024-02-02T15:40:21.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.813+0000 7f90f7ef8b80 1 -- 172.21.15.1:0/224201304 wait complete. 2024-02-02T15:40:21.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.813+0000 7f90f7ef8b80 1 -- start start 2024-02-02T15:40:21.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.814+0000 7f90f7ef8b80 1 Processor -- start 2024-02-02T15:40:21.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.814+0000 7f90f7ef8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56549ea2b010 0x56549ea361c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:21.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.814+0000 7f90f7ef8b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56549ea2b930 0x56549ea33440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:21.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.815+0000 7f90f7ef8b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56549ea367e0 0x56549ea33aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:21.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.815+0000 7f90f7ef8b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56549e97cd50 con 0x56549ea367e0 2024-02-02T15:40:21.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.815+0000 7f90f7ef8b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56549e918d80 con 0x56549ea2b930 2024-02-02T15:40:21.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.815+0000 7f90f7ef8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56549e918a20 con 0x56549ea2b010 2024-02-02T15:40:21.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.815+0000 7f90f7ef8b80 10 client.0 _ll_register_callbacks cb 0x56549e91a290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:40:21.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.815+0000 7f90e1a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56549ea2b010 0x56549ea361c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:21.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.815+0000 7f90e1a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56549ea2b010 0x56549ea361c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:40960/0 (socket says 172.21.15.1:40960) 2024-02-02T15:40:21.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.815+0000 7f90e1a00700 1 -- 172.21.15.1:0/2221047744 learned_addr learned my addr 172.21.15.1:0/2221047744 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:40:21.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.815+0000 7f90e1000700 1 --2- 172.21.15.1:0/2221047744 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56549ea2b930 0x56549ea33440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:21.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.815+0000 7f90e2400700 1 --2- 172.21.15.1:0/2221047744 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56549ea367e0 0x56549ea33aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:21.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.815+0000 7f90e1a00700 1 -- 172.21.15.1:0/2221047744 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56549ea2b930 msgr2=0x56549ea33440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:21.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.815+0000 7f90e1a00700 1 --2- 172.21.15.1:0/2221047744 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56549ea2b930 0x56549ea33440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:21.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.815+0000 7f90e1a00700 1 -- 172.21.15.1:0/2221047744 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56549ea367e0 msgr2=0x56549ea33aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:21.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.815+0000 7f90e1a00700 1 --2- 172.21.15.1:0/2221047744 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56549ea367e0 0x56549ea33aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:21.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.815+0000 7f90e1a00700 1 -- 172.21.15.1:0/2221047744 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90dc1a6210 con 0x56549ea2b010 2024-02-02T15:40:21.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.815+0000 7f90e2400700 1 --2- 172.21.15.1:0/2221047744 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56549ea367e0 0x56549ea33aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:40:21.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.815+0000 7f90e1a00700 1 --2- 172.21.15.1:0/2221047744 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56549ea2b010 0x56549ea361c0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f90cc0052a0 tx=0x7f90cc0052d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7a63dbc27a91e27 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:21.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.816+0000 7f90cb400700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:40:21.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.816+0000 7f90cb400700 1 -- 172.21.15.1:0/2221047744 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90cc01d070 con 0x56549ea2b010 2024-02-02T15:40:21.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.816+0000 7f90f7ef8b80 1 -- 172.21.15.1:0/2221047744 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56549ea1f0f0 con 0x56549ea2b010 2024-02-02T15:40:21.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.816+0000 7f90cb400700 1 -- 172.21.15.1:0/2221047744 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f90cc004910 con 0x56549ea2b010 2024-02-02T15:40:21.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.816+0000 7f90cb400700 1 -- 172.21.15.1:0/2221047744 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f90cc004ad0 con 0x56549ea2b010 2024-02-02T15:40:21.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.816+0000 7f90f7ef8b80 10 client.29224 fetch_fsmap learned FSMap version 52 2024-02-02T15:40:21.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.816+0000 7f90f7ef8b80 1 -- 172.21.15.1:0/2221047744 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x56549e7aab60 con 0x56549ea2b010 2024-02-02T15:40:21.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.816+0000 7f90cb400700 1 -- 172.21.15.1:0/2221047744 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56549ea1f0f0 con 0x56549ea2b010 2024-02-02T15:40:21.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.816+0000 7f90cb400700 1 -- 172.21.15.1:0/2221047744 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 52) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f90cc024450 con 0x56549ea2b010 2024-02-02T15:40:21.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.816+0000 7f90f7ef8b80 10 client.29224 fetch_fsmap finished waiting for FSMap version 52 2024-02-02T15:40:21.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.816+0000 7f90f7ef8b80 10 client.29224 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:40:21.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.817+0000 7f90f7ef8b80 20 client.29224 populate_metadata read hostname 'smithi001' 2024-02-02T15:40:21.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.817+0000 7f90f7ef8b80 1 --2- 172.21.15.1:0/2221047744 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56549ea461a0 0x56549ea485c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:21.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.817+0000 7f90f7ef8b80 4 client.29224 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:40:21.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.817+0000 7f90f7ef8b80 1 -- 172.21.15.1:0/2221047744 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56549e975dd0 con 0x56549ea461a0 2024-02-02T15:40:21.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.817+0000 7f90e1000700 1 --2- 172.21.15.1:0/2221047744 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56549ea461a0 0x56549ea485c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:21.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.817+0000 7f90e1000700 1 --2- 172.21.15.1:0/2221047744 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56549ea461a0 0x56549ea485c0 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0f5e3d17aabd85c server_cookie=ae689bcbf1f92d0f in_seq=0 out_seq=0 2024-02-02T15:40:21.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.817+0000 7f90cb400700 10 client.29224 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:40:21.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.818+0000 7f90cb400700 1 -- 172.21.15.1:0/2221047744 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x56549e975dd0 con 0x56549ea461a0 2024-02-02T15:40:21.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.818+0000 7f90cb400700 10 client.29224 handle_command_reply: tid=0 2024-02-02T15:40:21.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.818+0000 7f90f7ef8b80 1 client.29224 shutdown 2024-02-02T15:40:21.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.818+0000 7f90f7ef8b80 1 -- 172.21.15.1:0/2221047744 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56549ea2b010 msgr2=0x56549ea361c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:21.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.818+0000 7f90f7ef8b80 1 --2- 172.21.15.1:0/2221047744 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56549ea2b010 0x56549ea361c0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f90cc0052a0 tx=0x7f90cc0052d0 comp rx=0 tx=0).stop 2024-02-02T15:40:21.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.819+0000 7f90f7ef8b80 1 -- 172.21.15.1:0/2221047744 shutdown_connections 2024-02-02T15:40:21.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.819+0000 7f90f7ef8b80 1 --2- 172.21.15.1:0/2221047744 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56549ea461a0 0x56549ea485c0 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:21.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.819+0000 7f90f7ef8b80 1 --2- 172.21.15.1:0/2221047744 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56549ea367e0 0x56549ea33aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:21.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.819+0000 7f90f7ef8b80 1 --2- 172.21.15.1:0/2221047744 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56549ea2b010 0x56549ea361c0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:21.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.819+0000 7f90cb400700 0 client.29224 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:40:21.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.819+0000 7f90f7ef8b80 1 --2- 172.21.15.1:0/2221047744 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56549ea2b930 0x56549ea33440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:21.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.819+0000 7f90f7ef8b80 1 -- 172.21.15.1:0/2221047744 >> 172.21.15.1:0/2221047744 conn(0x56549e98adb0 msgr2=0x56549ea26fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:21.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.819+0000 7f90f7ef8b80 1 -- 172.21.15.1:0/2221047744 shutdown_connections 2024-02-02T15:40:21.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.819+0000 7f90f7ef8b80 1 -- 172.21.15.1:0/2221047744 wait complete. 2024-02-02T15:40:21.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.819+0000 7f90f7ef8b80 20 client.29224 trim_cache size 0 max 16384 2024-02-02T15:40:21.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.838+0000 7f90f7ef8b80 1 Processor -- start 2024-02-02T15:40:21.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.838+0000 7f90f7ef8b80 1 -- start start 2024-02-02T15:40:21.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.838+0000 7f90f7ef8b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56549ea2b010 0x56549ea3d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:21.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.838+0000 7f90f7ef8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56549ea2b930 0x56549ea2d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:21.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.838+0000 7f90f7ef8b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56549ea367e0 0x56549ea26cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:21.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.838+0000 7f90f7ef8b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56549e8006a0 con 0x56549ea367e0 2024-02-02T15:40:21.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.838+0000 7f90f7ef8b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56549ea34a20 con 0x56549ea2b010 2024-02-02T15:40:21.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.838+0000 7f90f7ef8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56549ea2e2f0 con 0x56549ea2b930 2024-02-02T15:40:21.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.839+0000 7f90e1000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56549ea2b930 0x56549ea2d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:21.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.839+0000 7f90e1a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56549ea2b010 0x56549ea3d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:21.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.839+0000 7f90e1000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56549ea2b930 0x56549ea2d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:40972/0 (socket says 172.21.15.1:40972) 2024-02-02T15:40:21.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.839+0000 7f90e1000700 1 -- 172.21.15.1:0/3564003504 learned_addr learned my addr 172.21.15.1:0/3564003504 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:40:21.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.839+0000 7f90e2400700 1 --2- 172.21.15.1:0/3564003504 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56549ea367e0 0x56549ea26cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:21.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.839+0000 7f90e1000700 1 -- 172.21.15.1:0/3564003504 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56549ea2b010 msgr2=0x56549ea3d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:21.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.839+0000 7f90e1000700 1 --2- 172.21.15.1:0/3564003504 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56549ea2b010 0x56549ea3d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:21.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.839+0000 7f90e1000700 1 -- 172.21.15.1:0/3564003504 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56549ea367e0 msgr2=0x56549ea26cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:21.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.839+0000 7f90e1000700 1 --2- 172.21.15.1:0/3564003504 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56549ea367e0 0x56549ea26cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:21.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.839+0000 7f90e1000700 1 -- 172.21.15.1:0/3564003504 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56549e942ea0 con 0x56549ea2b930 2024-02-02T15:40:21.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.839+0000 7f90e1a00700 1 --2- 172.21.15.1:0/3564003504 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56549ea2b010 0x56549ea3d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:40:21.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.839+0000 7f90e2400700 1 --2- 172.21.15.1:0/3564003504 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56549ea367e0 0x56549ea26cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:40:21.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.839+0000 7f90e1000700 1 --2- 172.21.15.1:0/3564003504 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56549ea2b930 0x56549ea2d840 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f90c4099070 tx=0x7f90c4010c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3ebc72c2915dc76 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:21.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.839+0000 7f90d3400700 1 -- 172.21.15.1:0/3564003504 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c4099440 con 0x56549ea2b930 2024-02-02T15:40:21.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.839+0000 7f90d3400700 1 -- 172.21.15.1:0/3564003504 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f90c4098d20 con 0x56549ea2b930 2024-02-02T15:40:21.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.839+0000 7f90d3400700 1 -- 172.21.15.1:0/3564003504 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c4098b10 con 0x56549ea2b930 2024-02-02T15:40:21.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.840+0000 7f90f7ef8b80 1 -- 172.21.15.1:0/3564003504 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56549ea2b930 msgr2=0x56549ea2d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:21.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.840+0000 7f90f7ef8b80 1 --2- 172.21.15.1:0/3564003504 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56549ea2b930 0x56549ea2d840 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f90c4099070 tx=0x7f90c4010c50 comp rx=0 tx=0).stop 2024-02-02T15:40:21.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.840+0000 7f90f7ef8b80 1 -- 172.21.15.1:0/3564003504 shutdown_connections 2024-02-02T15:40:21.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.840+0000 7f90f7ef8b80 1 --2- 172.21.15.1:0/3564003504 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56549ea367e0 0x56549ea26cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:21.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.840+0000 7f90f7ef8b80 1 --2- 172.21.15.1:0/3564003504 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56549ea2b010 0x56549ea3d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:21.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.840+0000 7f90f7ef8b80 1 --2- 172.21.15.1:0/3564003504 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56549ea2b930 0x56549ea2d840 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:21.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.840+0000 7f90f7ef8b80 1 -- 172.21.15.1:0/3564003504 >> 172.21.15.1:0/3564003504 conn(0x56549e98adb0 msgr2=0x56549ea265c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:21.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.840+0000 7f90f7ef8b80 1 -- 172.21.15.1:0/3564003504 shutdown_connections 2024-02-02T15:40:21.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.841+0000 7f90f7ef8b80 1 -- 172.21.15.1:0/3564003504 wait complete. 2024-02-02T15:40:21.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.841+0000 7f90f7ef8b80 1 -- start start 2024-02-02T15:40:21.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.841+0000 7f90f7ef8b80 1 Processor -- start 2024-02-02T15:40:21.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.842+0000 7f90f7ef8b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56549ea2b010 0x56549ea3d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:21.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.842+0000 7f90f7ef8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56549ea2b930 0x56549ea3b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:21.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.842+0000 7f90f7ef8b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56549ea367e0 0x56549ea3b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:21.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.842+0000 7f90f7ef8b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56549ea3bfd0 con 0x56549ea367e0 2024-02-02T15:40:21.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.842+0000 7f90f7ef8b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56549ea3c140 con 0x56549ea2b010 2024-02-02T15:40:21.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.842+0000 7f90f7ef8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56549ea97c90 con 0x56549ea2b930 2024-02-02T15:40:21.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.842+0000 7f90f7ef8b80 10 client.0 _ll_register_callbacks cb 0x56549e91a290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:40:21.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.842+0000 7f90e1000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56549ea2b930 0x56549ea3b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:21.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.842+0000 7f90e1000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56549ea2b930 0x56549ea3b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:40984/0 (socket says 172.21.15.1:40984) 2024-02-02T15:40:21.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.842+0000 7f90e1000700 1 -- 172.21.15.1:0/3505743113 learned_addr learned my addr 172.21.15.1:0/3505743113 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:40:21.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.842+0000 7f90e1a00700 1 --2- 172.21.15.1:0/3505743113 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56549ea2b010 0x56549ea3d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:21.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.842+0000 7f90e2400700 1 --2- 172.21.15.1:0/3505743113 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56549ea367e0 0x56549ea3b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:21.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.842+0000 7f90e1000700 1 -- 172.21.15.1:0/3505743113 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56549ea2b010 msgr2=0x56549ea3d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:21.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.842+0000 7f90e1000700 1 --2- 172.21.15.1:0/3505743113 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56549ea2b010 0x56549ea3d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:21.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.842+0000 7f90e1000700 1 -- 172.21.15.1:0/3505743113 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56549ea367e0 msgr2=0x56549ea3b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:21.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.842+0000 7f90e1000700 1 --2- 172.21.15.1:0/3505743113 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56549ea367e0 0x56549ea3b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:21.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.842+0000 7f90e1000700 1 -- 172.21.15.1:0/3505743113 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56549e7aab60 con 0x56549ea2b930 2024-02-02T15:40:21.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.842+0000 7f90e1a00700 1 --2- 172.21.15.1:0/3505743113 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56549ea2b010 0x56549ea3d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:40:21.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.843+0000 7f90e1000700 1 --2- 172.21.15.1:0/3505743113 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56549ea2b930 0x56549ea3b3c0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f90c40958c0 tx=0x7f90c4095700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea57b39f9e484d1e server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:21.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.843+0000 7f90cb400700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:40:21.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.843+0000 7f90cb400700 1 -- 172.21.15.1:0/3505743113 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c409d1f0 con 0x56549ea2b930 2024-02-02T15:40:21.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.843+0000 7f90cb400700 1 -- 172.21.15.1:0/3505743113 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f90c4099620 con 0x56549ea2b930 2024-02-02T15:40:21.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.843+0000 7f90f7ef8b80 1 -- 172.21.15.1:0/3505743113 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f90c4099ac0 con 0x56549ea2b930 2024-02-02T15:40:21.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.843+0000 7f90cb400700 1 -- 172.21.15.1:0/3505743113 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f90c4099800 con 0x56549ea2b930 2024-02-02T15:40:21.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.843+0000 7f90f7ef8b80 10 client.29230 fetch_fsmap learned FSMap version 52 2024-02-02T15:40:21.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.843+0000 7f90f7ef8b80 1 -- 172.21.15.1:0/3505743113 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x56549e7fe3b0 con 0x56549ea2b930 2024-02-02T15:40:21.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.843+0000 7f90cb400700 1 -- 172.21.15.1:0/3505743113 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f90c4099ac0 con 0x56549ea2b930 2024-02-02T15:40:21.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.843+0000 7f90cb400700 1 -- 172.21.15.1:0/3505743113 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 52) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f90c40a0fa0 con 0x56549ea2b930 2024-02-02T15:40:21.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.843+0000 7f90f7ef8b80 10 client.29230 fetch_fsmap finished waiting for FSMap version 52 2024-02-02T15:40:21.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.843+0000 7f90f7ef8b80 10 client.29230 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:40:21.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.843+0000 7f90f7ef8b80 20 client.29230 populate_metadata read hostname 'smithi001' 2024-02-02T15:40:21.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.843+0000 7f90f7ef8b80 1 --2- 172.21.15.1:0/3505743113 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56549ea98da0 0x56549ea9b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:21.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.844+0000 7f90f7ef8b80 4 client.29230 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2024-02-02T15:40:21.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.844+0000 7f90f7ef8b80 1 -- 172.21.15.1:0/3505743113 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56549e93bf50 con 0x56549ea98da0 2024-02-02T15:40:21.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.844+0000 7f90e1a00700 1 --2- 172.21.15.1:0/3505743113 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56549ea98da0 0x56549ea9b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:21.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.844+0000 7f90e1a00700 1 --2- 172.21.15.1:0/3505743113 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56549ea98da0 0x56549ea9b1c0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b08527e429e9f629 server_cookie=883c20792601a543 in_seq=0 out_seq=0 2024-02-02T15:40:21.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.844+0000 7f90cb400700 10 client.29230 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:40:21.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.844+0000 7f90cb400700 1 -- 172.21.15.1:0/3505743113 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x56549e93bf50 con 0x56549ea98da0 2024-02-02T15:40:21.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.844+0000 7f90cb400700 10 client.29230 handle_command_reply: tid=0 2024-02-02T15:40:21.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.844+0000 7f90f7ef8b80 1 client.29230 shutdown 2024-02-02T15:40:21.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.845+0000 7f90f7ef8b80 1 -- 172.21.15.1:0/3505743113 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56549ea2b930 msgr2=0x56549ea3b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:21.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.845+0000 7f90f7ef8b80 1 --2- 172.21.15.1:0/3505743113 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56549ea2b930 0x56549ea3b3c0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f90c40958c0 tx=0x7f90c4095700 comp rx=0 tx=0).stop 2024-02-02T15:40:21.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.845+0000 7f90f7ef8b80 1 -- 172.21.15.1:0/3505743113 shutdown_connections 2024-02-02T15:40:21.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.845+0000 7f90f7ef8b80 1 --2- 172.21.15.1:0/3505743113 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56549ea98da0 0x56549ea9b1c0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:21.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.845+0000 7f90f7ef8b80 1 --2- 172.21.15.1:0/3505743113 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56549ea367e0 0x56549ea3b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:21.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.845+0000 7f90f7ef8b80 1 --2- 172.21.15.1:0/3505743113 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56549ea2b010 0x56549ea3d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:21.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.845+0000 7f90f7ef8b80 1 --2- 172.21.15.1:0/3505743113 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56549ea2b930 0x56549ea3b3c0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:21.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.845+0000 7f90f7ef8b80 1 -- 172.21.15.1:0/3505743113 >> 172.21.15.1:0/3505743113 conn(0x56549e98adb0 msgr2=0x56549ea26e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:21.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.845+0000 7f90cb400700 0 client.29230 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:40:21.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.845+0000 7f90f7ef8b80 1 -- 172.21.15.1:0/3505743113 shutdown_connections 2024-02-02T15:40:21.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.845+0000 7f90f7ef8b80 1 -- 172.21.15.1:0/3505743113 wait complete. 2024-02-02T15:40:21.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.845+0000 7f90f7ef8b80 20 client.29230 trim_cache size 0 max 16384 2024-02-02T15:40:21.851 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:40:21.851 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-02T15:40:21.851 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2024-02-02T15:40:21.851 INFO:teuthology.orchestra.run.smithi001.stdout: "ff3a6039-bd29-449f-922d-dd77a3b760fa": { 2024-02-02T15:40:21.851 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2024-02-02T15:40:21.851 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "ff3a6039-bd29-449f-922d-dd77a3b760fa", 2024-02-02T15:40:21.851 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,force" 2024-02-02T15:40:21.851 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:40:21.851 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:40:21.851 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:40:21.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.846+0000 7f90e4400700 1 -- 172.21.15.1:0/1425582201 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f90bc0a0690 msgr2=0x7f90bc0a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:21.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.846+0000 7f90e4400700 1 --2- 172.21.15.1:0/1425582201 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f90bc0a0690 0x7f90bc0a2b50 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f90d400f590 tx=0x7f90d400b040 comp rx=0 tx=0).stop 2024-02-02T15:40:21.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.846+0000 7f90e4400700 1 -- 172.21.15.1:0/1425582201 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90dc108350 msgr2=0x7f90dc199670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:21.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.846+0000 7f90e4400700 1 --2- 172.21.15.1:0/1425582201 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90dc108350 0x7f90dc199670 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f90c400e9c0 tx=0x7f90c400ecd0 comp rx=0 tx=0).stop 2024-02-02T15:40:21.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.846+0000 7f90e4400700 1 -- 172.21.15.1:0/1425582201 shutdown_connections 2024-02-02T15:40:21.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.846+0000 7f90e4400700 1 --2- 172.21.15.1:0/1425582201 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f90bc0a0690 0x7f90bc0a2b50 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:21.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.846+0000 7f90e4400700 1 --2- 172.21.15.1:0/1425582201 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f90dc19dad0 0x7f90dc19df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:21.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.846+0000 7f90e4400700 1 --2- 172.21.15.1:0/1425582201 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f90dc103230 0x7f90dc199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:21.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.846+0000 7f90e4400700 1 --2- 172.21.15.1:0/1425582201 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90dc108350 0x7f90dc199670 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:21.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.846+0000 7f90e4400700 1 -- 172.21.15.1:0/1425582201 >> 172.21.15.1:0/1425582201 conn(0x7f90dc0fea70 msgr2=0x7f90dc0ff580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:21.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.846+0000 7f90e4400700 1 -- 172.21.15.1:0/1425582201 shutdown_connections 2024-02-02T15:40:21.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:21.846+0000 7f90e4400700 1 -- 172.21.15.1:0/1425582201 wait complete. 2024-02-02T15:40:21.864 INFO:tasks.cephfs.filesystem:scrub status for tag:ff3a6039-bd29-449f-922d-dd77a3b760fa - {'path': '/', 'tag': 'ff3a6039-bd29-449f-922d-dd77a3b760fa', 'options': 'recursive,force'} 2024-02-02T15:40:21.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:21 smithi071 ceph-mon[44707]: pgmap v417: 129 pgs: 129 active+clean; 607 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 135 op/s 2024-02-02T15:40:21.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:21 smithi142 ceph-mon[45728]: pgmap v417: 129 pgs: 129 active+clean; 607 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.9 MiB/s wr, 135 op/s 2024-02-02T15:40:22.276 INFO:tasks.workunit.client.0.smithi001.stdout: 1 720311 5.27 MB/sec execute 596 sec latency 1225.667 ms 2024-02-02T15:40:22.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:22 smithi071 ceph-mon[44707]: scrub queued for path: / 2024-02-02T15:40:22.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:22 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:40:22.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:22 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:40:22.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:22 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:40:22.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:22 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:40:22.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:22 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:40:22.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:22 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:40:22.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:22 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:40:22.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:22 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:40:22.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:22 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:40:22.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:22 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:40:22.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:22 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:40:22.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:22 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:40:22.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:22 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:40:22.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:22 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:40:22.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:22 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:40:22.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:22 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:40:22.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:22 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:40:22.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:22 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:40:22.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:22 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:40:22.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:22 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:40:22.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:22 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:40:22.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:22 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:40:22.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:22 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:40:22.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:22 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:40:22.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:22 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:40:22.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:22 smithi142 ceph-mon[45728]: scrub queued for path: / 2024-02-02T15:40:22.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:22 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:40:22.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:22 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:40:22.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:22 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:40:22.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:22 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:40:22.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:22 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:40:22.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:22 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:40:22.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:22 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:40:22.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:22 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:40:22.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:22 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:40:22.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:22 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:40:22.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:22 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:40:22.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:22 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:40:22.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:22 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:40:22.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:22 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:40:22.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:22 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:40:22.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:22 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:40:22.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:22 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:40:22.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:22 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:40:22.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:22 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:40:22.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:22 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:40:22.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:22 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:40:22.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:22 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:40:22.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:22 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:40:22.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:22 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:40:22.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:22 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:40:22.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:22 smithi001 ceph-mon[35470]: scrub queued for path: / 2024-02-02T15:40:22.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:22 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:40:22.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:22 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:40:22.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:22 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:40:22.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:22 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:40:22.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:22 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:40:22.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:22 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:40:22.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:22 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:40:22.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:22 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:40:22.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:22 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:40:22.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:22 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:40:22.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:22 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:40:22.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:22 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:40:22.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:22 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:40:22.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:22 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:40:22.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:22 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:40:22.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:22 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:40:22.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:22 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:40:22.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:22 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:40:22.942 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:22 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:40:22.942 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:22 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:40:22.942 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:22 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:40:22.942 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:22 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:40:22.942 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:22 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:40:22.942 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:22 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:40:22.942 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:22 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:40:23.276 INFO:tasks.workunit.client.0.smithi001.stdout: 1 721345 5.28 MB/sec execute 597 sec latency 16.579 ms 2024-02-02T15:40:23.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:23 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:40:23.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:23 smithi071 ceph-mon[44707]: pgmap v418: 129 pgs: 129 active+clean; 614 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 142 op/s 2024-02-02T15:40:23.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:23 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:40:23.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:23 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:40:23.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:23 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:40:23.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:23 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:40:23.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:23 smithi142 ceph-mon[45728]: pgmap v418: 129 pgs: 129 active+clean; 614 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 142 op/s 2024-02-02T15:40:23.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:23 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:40:23.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:23 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:40:23.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:23 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:40:23.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:23 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:40:23.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:23 smithi001 ceph-mon[35470]: pgmap v418: 129 pgs: 129 active+clean; 614 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 142 op/s 2024-02-02T15:40:23.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:23 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:40:23.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:23 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:40:23.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:23 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:40:24.277 INFO:tasks.workunit.client.0.smithi001.stdout: 1 722548 5.28 MB/sec execute 598 sec latency 43.759 ms 2024-02-02T15:40:25.277 INFO:tasks.workunit.client.0.smithi001.stdout: 1 722548 5.27 MB/sec execute 599 sec latency 1023.370 ms 2024-02-02T15:40:25.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:25 smithi071 ceph-mon[44707]: pgmap v419: 129 pgs: 129 active+clean; 614 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 142 op/s 2024-02-02T15:40:25.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:25 smithi142 ceph-mon[45728]: pgmap v419: 129 pgs: 129 active+clean; 614 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 142 op/s 2024-02-02T15:40:25.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:25 smithi001 ceph-mon[35470]: pgmap v419: 129 pgs: 129 active+clean; 614 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 142 op/s 2024-02-02T15:40:26.277 INFO:tasks.workunit.client.0.smithi001.stdout: 1 cleanup 600 sec 2024-02-02T15:40:27.277 INFO:tasks.workunit.client.0.smithi001.stdout: 1 cleanup 601 sec 2024-02-02T15:40:27.864 INFO:tasks.workunit.client.0.smithi001.stdout: 0 cleanup 602 sec 2024-02-02T15:40:27.864 INFO:tasks.workunit.client.0.smithi001.stdout: 2024-02-02T15:40:27.864 INFO:tasks.workunit.client.0.smithi001.stdout: Operation Count AvgLat MaxLat 2024-02-02T15:40:27.864 INFO:tasks.workunit.client.0.smithi001.stdout: ---------------------------------------- 2024-02-02T15:40:27.865 INFO:tasks.workunit.client.0.smithi001.stdout: NTCreateX 100556 1.985 12788.356 2024-02-02T15:40:27.865 INFO:tasks.workunit.client.0.smithi001.stdout: Close 73841 0.005 1.122 2024-02-02T15:40:27.865 INFO:tasks.workunit.client.0.smithi001.stdout: Rename 4262 42.197 5040.415 2024-02-02T15:40:27.865 INFO:tasks.workunit.client.0.smithi001.stdout: Unlink 20328 2.203 10206.437 2024-02-02T15:40:27.865 INFO:tasks.workunit.client.0.smithi001.stdout: Deltree 2 108.964 155.304 2024-02-02T15:40:27.865 INFO:tasks.workunit.client.0.smithi001.stdout: Mkdir 1 0.015 0.015 2024-02-02T15:40:27.865 INFO:tasks.workunit.client.0.smithi001.stdout: Qpathinfo 91121 1.315 3914.408 2024-02-02T15:40:27.865 INFO:tasks.workunit.client.0.smithi001.stdout: Qfileinfo 15952 0.002 0.068 2024-02-02T15:40:27.865 INFO:tasks.workunit.client.0.smithi001.stdout: Qfsinfo 16731 0.260 28.058 2024-02-02T15:40:27.865 INFO:tasks.workunit.client.0.smithi001.stdout: Sfileinfo 8194 0.033 4.449 2024-02-02T15:40:27.865 INFO:tasks.workunit.client.0.smithi001.stdout: Find 35242 0.260 60.514 2024-02-02T15:40:27.865 INFO:tasks.workunit.client.0.smithi001.stdout: WriteX 50042 0.148 96.338 2024-02-02T15:40:27.865 INFO:tasks.workunit.client.0.smithi001.stdout: ReadX 157761 0.032 229.710 2024-02-02T15:40:27.866 INFO:tasks.workunit.client.0.smithi001.stdout: LockX 328 0.609 2.504 2024-02-02T15:40:27.866 INFO:tasks.workunit.client.0.smithi001.stdout: UnlockX 328 0.593 1.776 2024-02-02T15:40:27.866 INFO:tasks.workunit.client.0.smithi001.stdout: Flush 7052 3.980 63.417 2024-02-02T15:40:27.866 INFO:tasks.workunit.client.0.smithi001.stdout: 2024-02-02T15:40:27.866 INFO:tasks.workunit.client.0.smithi001.stdout:Throughput 5.26743 MB/sec 1 clients 1 procs max_latency=12788.368 ms 2024-02-02T15:40:27.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:27 smithi071 ceph-mon[44707]: pgmap v420: 129 pgs: 129 active+clean; 619 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 142 op/s 2024-02-02T15:40:27.867 INFO:tasks.workunit.client.0.smithi001.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-02-02T15:40:27.867 INFO:tasks.workunit.client.0.smithi001.stdout: 2024-02-02T15:40:27.867 INFO:tasks.workunit.client.0.smithi001.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-02-02T15:40:27.867 INFO:tasks.workunit.client.0.smithi001.stdout:9 of 10 processes prepared for launch 0 sec 2024-02-02T15:40:27.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:27 smithi142 ceph-mon[45728]: pgmap v420: 129 pgs: 129 active+clean; 619 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 142 op/s 2024-02-02T15:40:27.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:27 smithi001 ceph-mon[35470]: pgmap v420: 129 pgs: 129 active+clean; 619 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.1 MiB/s wr, 142 op/s 2024-02-02T15:40:27.966 INFO:tasks.workunit.client.0.smithi001.stdout:10 of 10 processes prepared for launch 0 sec 2024-02-02T15:40:27.966 INFO:tasks.workunit.client.0.smithi001.stdout:releasing clients 2024-02-02T15:40:28.966 INFO:tasks.workunit.client.0.smithi001.stdout: 10 327 119.01 MB/sec warmup 1 sec latency 63.233 ms 2024-02-02T15:40:29.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:29 smithi071 ceph-mon[44707]: pgmap v421: 129 pgs: 129 active+clean; 619 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 119 op/s 2024-02-02T15:40:29.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:29 smithi142 ceph-mon[45728]: pgmap v421: 129 pgs: 129 active+clean; 619 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 119 op/s 2024-02-02T15:40:29.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:29 smithi001 ceph-mon[35470]: pgmap v421: 129 pgs: 129 active+clean; 619 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.8 MiB/s wr, 119 op/s 2024-02-02T15:40:29.966 INFO:tasks.workunit.client.0.smithi001.stdout: 10 704 102.35 MB/sec warmup 2 sec latency 406.569 ms 2024-02-02T15:40:30.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:30 smithi071 ceph-mon[44707]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:40:30.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:30 smithi142 ceph-mon[45728]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:40:30.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:30 smithi001 ceph-mon[35470]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:40:30.966 INFO:tasks.workunit.client.0.smithi001.stdout: 10 1095 74.33 MB/sec warmup 3 sec latency 115.799 ms 2024-02-02T15:40:31.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:31 smithi071 ceph-mon[44707]: pgmap v422: 129 pgs: 129 active+clean; 619 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 118 op/s 2024-02-02T15:40:31.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:31 smithi142 ceph-mon[45728]: pgmap v422: 129 pgs: 129 active+clean; 619 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 118 op/s 2024-02-02T15:40:31.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:31 smithi001 ceph-mon[35470]: pgmap v422: 129 pgs: 129 active+clean; 619 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.7 MiB/s wr, 118 op/s 2024-02-02T15:40:31.966 INFO:tasks.workunit.client.0.smithi001.stdout: 10 1601 64.36 MB/sec warmup 4 sec latency 118.568 ms 2024-02-02T15:40:32.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:32 smithi071 ceph-mon[44707]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:40:32.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:32 smithi142 ceph-mon[45728]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:40:32.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:32 smithi001 ceph-mon[35470]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:40:32.966 INFO:tasks.workunit.client.0.smithi001.stdout: 10 2305 55.17 MB/sec warmup 5 sec latency 60.812 ms 2024-02-02T15:40:33.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:33 smithi071 ceph-mon[44707]: pgmap v423: 129 pgs: 129 active+clean; 630 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.0 MiB/s wr, 214 op/s 2024-02-02T15:40:33.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:33 smithi142 ceph-mon[45728]: pgmap v423: 129 pgs: 129 active+clean; 630 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.0 MiB/s wr, 214 op/s 2024-02-02T15:40:33.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:33 smithi001 ceph-mon[35470]: pgmap v423: 129 pgs: 129 active+clean; 630 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.0 MiB/s wr, 214 op/s 2024-02-02T15:40:33.966 INFO:tasks.workunit.client.0.smithi001.stdout: 10 2720 53.36 MB/sec warmup 6 sec latency 169.816 ms 2024-02-02T15:40:34.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:34 smithi071 ceph-mon[44707]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:40:34.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:34 smithi142 ceph-mon[45728]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:40:34.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:34 smithi001 ceph-mon[35470]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:40:34.966 INFO:tasks.workunit.client.0.smithi001.stdout: 10 3196 49.30 MB/sec warmup 7 sec latency 159.784 ms 2024-02-02T15:40:35.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:35 smithi071 ceph-mon[44707]: pgmap v424: 129 pgs: 129 active+clean; 630 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 153 op/s 2024-02-02T15:40:35.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:35 smithi071 ceph-mon[44707]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:40:35.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:35 smithi142 ceph-mon[45728]: pgmap v424: 129 pgs: 129 active+clean; 630 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 153 op/s 2024-02-02T15:40:35.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:35 smithi142 ceph-mon[45728]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:40:35.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:35 smithi001 ceph-mon[35470]: pgmap v424: 129 pgs: 129 active+clean; 630 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.1 MiB/s wr, 153 op/s 2024-02-02T15:40:35.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:35 smithi001 ceph-mon[35470]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:40:35.966 INFO:tasks.workunit.client.0.smithi001.stdout: 10 3577 48.01 MB/sec warmup 8 sec latency 697.900 ms 2024-02-02T15:40:36.966 INFO:tasks.workunit.client.0.smithi001.stdout: 10 3721 42.96 MB/sec warmup 9 sec latency 1284.209 ms 2024-02-02T15:40:37.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:37 smithi071 ceph-mon[44707]: pgmap v425: 129 pgs: 129 active+clean; 683 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 7.8 MiB/s wr, 280 op/s 2024-02-02T15:40:37.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:37 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:40:37.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:37 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:40:37.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:37 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:40:37.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:37 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:40:37.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:37 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:40:37.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:37 smithi071 ceph-mon[44707]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:40:37.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:37 smithi142 ceph-mon[45728]: pgmap v425: 129 pgs: 129 active+clean; 683 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 7.8 MiB/s wr, 280 op/s 2024-02-02T15:40:37.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:37 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:40:37.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:37 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:40:37.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:37 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:40:37.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:37 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:40:37.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:37 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:40:37.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:37 smithi142 ceph-mon[45728]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:40:37.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:37 smithi001 ceph-mon[35470]: pgmap v425: 129 pgs: 129 active+clean; 683 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 7.8 MiB/s wr, 280 op/s 2024-02-02T15:40:37.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:37 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:40:37.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:37 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:40:37.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:37 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:40:37.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:37 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:40:37.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:37 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:40:37.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:37 smithi001 ceph-mon[35470]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:40:37.966 INFO:tasks.workunit.client.0.smithi001.stdout: 10 3913 38.86 MB/sec warmup 10 sec latency 149.635 ms 2024-02-02T15:40:38.967 INFO:tasks.workunit.client.0.smithi001.stdout: 10 4048 35.47 MB/sec warmup 11 sec latency 168.580 ms 2024-02-02T15:40:39.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:39 smithi071 ceph-mon[44707]: pgmap v426: 129 pgs: 129 active+clean; 683 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.9 MiB/s wr, 223 op/s 2024-02-02T15:40:39.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:39 smithi142 ceph-mon[45728]: pgmap v426: 129 pgs: 129 active+clean; 683 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.9 MiB/s wr, 223 op/s 2024-02-02T15:40:39.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:39 smithi001 ceph-mon[35470]: pgmap v426: 129 pgs: 129 active+clean; 683 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.9 MiB/s wr, 223 op/s 2024-02-02T15:40:39.967 INFO:tasks.workunit.client.0.smithi001.stdout: 10 4080 32.56 MB/sec warmup 12 sec latency 613.257 ms 2024-02-02T15:40:40.967 INFO:tasks.workunit.client.0.smithi001.stdout: 10 4312 30.53 MB/sec warmup 13 sec latency 647.706 ms 2024-02-02T15:40:41.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:41 smithi071 ceph-mon[44707]: pgmap v427: 129 pgs: 129 active+clean; 683 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.9 MiB/s wr, 223 op/s 2024-02-02T15:40:41.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:41 smithi142 ceph-mon[45728]: pgmap v427: 129 pgs: 129 active+clean; 683 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.9 MiB/s wr, 223 op/s 2024-02-02T15:40:41.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:41 smithi001 ceph-mon[35470]: pgmap v427: 129 pgs: 129 active+clean; 683 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.9 MiB/s wr, 223 op/s 2024-02-02T15:40:41.967 INFO:tasks.workunit.client.0.smithi001.stdout: 10 4684 30.55 MB/sec warmup 14 sec latency 69.514 ms 2024-02-02T15:40:42.967 INFO:tasks.workunit.client.0.smithi001.stdout: 10 5179 29.64 MB/sec warmup 15 sec latency 73.640 ms 2024-02-02T15:40:43.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:43 smithi071 ceph-mon[44707]: pgmap v428: 129 pgs: 129 active+clean; 853 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 23 MiB/s wr, 409 op/s 2024-02-02T15:40:43.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:43 smithi142 ceph-mon[45728]: pgmap v428: 129 pgs: 129 active+clean; 853 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 23 MiB/s wr, 409 op/s 2024-02-02T15:40:43.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:43 smithi001 ceph-mon[35470]: pgmap v428: 129 pgs: 129 active+clean; 853 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 23 MiB/s wr, 409 op/s 2024-02-02T15:40:43.967 INFO:tasks.workunit.client.0.smithi001.stdout: 10 5791 29.21 MB/sec warmup 16 sec latency 49.125 ms 2024-02-02T15:40:44.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:44 smithi071 ceph-mon[44707]: pgmap v429: 129 pgs: 129 active+clean; 853 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 22 MiB/s wr, 314 op/s 2024-02-02T15:40:44.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:44 smithi142 ceph-mon[45728]: pgmap v429: 129 pgs: 129 active+clean; 853 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 22 MiB/s wr, 314 op/s 2024-02-02T15:40:44.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:44 smithi001 ceph-mon[35470]: pgmap v429: 129 pgs: 129 active+clean; 853 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 22 MiB/s wr, 314 op/s 2024-02-02T15:40:44.967 INFO:tasks.workunit.client.0.smithi001.stdout: 10 6259 30.06 MB/sec warmup 17 sec latency 49.685 ms 2024-02-02T15:40:45.968 INFO:tasks.workunit.client.0.smithi001.stdout: 10 6732 30.33 MB/sec warmup 18 sec latency 55.698 ms 2024-02-02T15:40:46.967 INFO:tasks.workunit.client.0.smithi001.stdout: 10 6993 29.73 MB/sec warmup 19 sec latency 133.412 ms 2024-02-02T15:40:47.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:47 smithi071 ceph-mon[44707]: pgmap v430: 129 pgs: 129 active+clean; 867 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 23 MiB/s wr, 419 op/s 2024-02-02T15:40:47.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:47 smithi142 ceph-mon[45728]: pgmap v430: 129 pgs: 129 active+clean; 867 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 23 MiB/s wr, 419 op/s 2024-02-02T15:40:47.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:47 smithi001 ceph-mon[35470]: pgmap v430: 129 pgs: 129 active+clean; 867 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 23 MiB/s wr, 419 op/s 2024-02-02T15:40:47.967 INFO:tasks.workunit.client.0.smithi001.stdout: 10 7178 28.68 MB/sec warmup 20 sec latency 140.593 ms 2024-02-02T15:40:48.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:48 smithi071 ceph-mon[44707]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:40:48.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:48 smithi142 ceph-mon[45728]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:40:48.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:48 smithi001 ceph-mon[35470]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:40:48.968 INFO:tasks.workunit.client.0.smithi001.stdout: 10 7487 27.80 MB/sec warmup 21 sec latency 101.922 ms 2024-02-02T15:40:49.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:49 smithi071 ceph-mon[44707]: pgmap v431: 129 pgs: 129 active+clean; 867 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 17 MiB/s wr, 291 op/s 2024-02-02T15:40:49.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:49 smithi142 ceph-mon[45728]: pgmap v431: 129 pgs: 129 active+clean; 867 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 17 MiB/s wr, 291 op/s 2024-02-02T15:40:49.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:49 smithi001 ceph-mon[35470]: pgmap v431: 129 pgs: 129 active+clean; 867 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 17 MiB/s wr, 291 op/s 2024-02-02T15:40:49.968 INFO:tasks.workunit.client.0.smithi001.stdout: 10 7927 27.24 MB/sec warmup 22 sec latency 234.825 ms 2024-02-02T15:40:50.968 INFO:tasks.workunit.client.0.smithi001.stdout: 10 8471 27.35 MB/sec warmup 23 sec latency 1216.809 ms 2024-02-02T15:40:51.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:51 smithi071 ceph-mon[44707]: pgmap v432: 129 pgs: 129 active+clean; 867 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 17 MiB/s wr, 291 op/s 2024-02-02T15:40:51.864 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-02T15:40:51.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:51 smithi142 ceph-mon[45728]: pgmap v432: 129 pgs: 129 active+clean; 867 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 17 MiB/s wr, 291 op/s 2024-02-02T15:40:51.937 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:51 smithi001 ceph-mon[35470]: pgmap v432: 129 pgs: 129 active+clean; 867 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 17 MiB/s wr, 291 op/s 2024-02-02T15:40:51.968 INFO:tasks.workunit.client.0.smithi001.stdout: 10 9112 27.35 MB/sec warmup 24 sec latency 2216.877 ms 2024-02-02T15:40:52.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.127+0000 7fa13a400700 1 -- 172.21.15.1:0/996823503 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa134101950 msgr2=0x7fa134105990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:52.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.127+0000 7fa13a400700 1 --2- 172.21.15.1:0/996823503 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa134101950 0x7fa134105990 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fa11c009990 tx=0x7fa11c009ca0 comp rx=0 tx=0).stop 2024-02-02T15:40:52.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.127+0000 7fa13a400700 1 -- 172.21.15.1:0/996823503 shutdown_connections 2024-02-02T15:40:52.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.127+0000 7fa13a400700 1 --2- 172.21.15.1:0/996823503 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa1341060c0 0x7fa13410dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:52.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.127+0000 7fa13a400700 1 --2- 172.21.15.1:0/996823503 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa134100fa0 0x7fa134101380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:52.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.127+0000 7fa13a400700 1 --2- 172.21.15.1:0/996823503 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa134101950 0x7fa134105990 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:52.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.127+0000 7fa13a400700 1 -- 172.21.15.1:0/996823503 >> 172.21.15.1:0/996823503 conn(0x7fa1340fc800 msgr2=0x7fa1340fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:52.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.127+0000 7fa13a400700 1 -- 172.21.15.1:0/996823503 shutdown_connections 2024-02-02T15:40:52.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.127+0000 7fa13a400700 1 -- 172.21.15.1:0/996823503 wait complete. 2024-02-02T15:40:52.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.128+0000 7fa13a400700 1 Processor -- start 2024-02-02T15:40:52.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.128+0000 7fa13a400700 1 -- start start 2024-02-02T15:40:52.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.129+0000 7fa13a400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa134100fa0 0x7fa134199020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:52.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.129+0000 7fa13a400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa1341060c0 0x7fa134199560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:52.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.129+0000 7fa13a400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa13419d9c0 0x7fa13419de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:52.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.129+0000 7fa13a400700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa13419e440 con 0x7fa13419d9c0 2024-02-02T15:40:52.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.129+0000 7fa13a400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa13419e5b0 con 0x7fa134100fa0 2024-02-02T15:40:52.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.129+0000 7fa13a400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa13419e770 con 0x7fa1341060c0 2024-02-02T15:40:52.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.129+0000 7fa133400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa134100fa0 0x7fa134199020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:52.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.129+0000 7fa133400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa134100fa0 0x7fa134199020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:32860/0 (socket says 172.21.15.1:32860) 2024-02-02T15:40:52.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.129+0000 7fa133400700 1 -- 172.21.15.1:0/3435637655 learned_addr learned my addr 172.21.15.1:0/3435637655 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:40:52.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.129+0000 7fa132a00700 1 --2- 172.21.15.1:0/3435637655 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa1341060c0 0x7fa134199560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:52.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.129+0000 7fa133e00700 1 --2- 172.21.15.1:0/3435637655 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa13419d9c0 0x7fa13419de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:52.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.129+0000 7fa133400700 1 -- 172.21.15.1:0/3435637655 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa1341060c0 msgr2=0x7fa134199560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:52.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.129+0000 7fa133400700 1 --2- 172.21.15.1:0/3435637655 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa1341060c0 0x7fa134199560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:52.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.129+0000 7fa133400700 1 -- 172.21.15.1:0/3435637655 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa13419d9c0 msgr2=0x7fa13419de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:52.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.129+0000 7fa133400700 1 --2- 172.21.15.1:0/3435637655 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa13419d9c0 0x7fa13419de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:52.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.129+0000 7fa133400700 1 -- 172.21.15.1:0/3435637655 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa11c0095f0 con 0x7fa134100fa0 2024-02-02T15:40:52.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.129+0000 7fa133e00700 1 --2- 172.21.15.1:0/3435637655 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa13419d9c0 0x7fa13419de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:40:52.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.130+0000 7fa133400700 1 --2- 172.21.15.1:0/3435637655 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa134100fa0 0x7fa134199020 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fa12400e9a0 tx=0x7fa12400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:52.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.130+0000 7fa12be00700 1 -- 172.21.15.1:0/3435637655 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa124009790 con 0x7fa134100fa0 2024-02-02T15:40:52.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.130+0000 7fa12be00700 1 -- 172.21.15.1:0/3435637655 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa12400cbd0 con 0x7fa134100fa0 2024-02-02T15:40:52.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.130+0000 7fa13a400700 1 -- 172.21.15.1:0/3435637655 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1341a5cd0 con 0x7fa134100fa0 2024-02-02T15:40:52.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.130+0000 7fa12be00700 1 -- 172.21.15.1:0/3435637655 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa124018680 con 0x7fa134100fa0 2024-02-02T15:40:52.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.130+0000 7fa13a400700 1 -- 172.21.15.1:0/3435637655 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1341a61b0 con 0x7fa134100fa0 2024-02-02T15:40:52.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.134+0000 7fa12be00700 1 -- 172.21.15.1:0/3435637655 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7fa12400cd40 con 0x7fa134100fa0 2024-02-02T15:40:52.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.135+0000 7fa12be00700 1 --2- 172.21.15.1:0/3435637655 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fa1140a0710 0x7fa1140a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:52.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.135+0000 7fa12be00700 1 -- 172.21.15.1:0/3435637655 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fa124014070 con 0x7fa134100fa0 2024-02-02T15:40:52.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.135+0000 7fa14dea7b80 1 Processor -- start 2024-02-02T15:40:52.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.135+0000 7fa14dea7b80 1 -- start start 2024-02-02T15:40:52.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.135+0000 7fa132a00700 1 --2- 172.21.15.1:0/3435637655 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fa1140a0710 0x7fa1140a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:52.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.136+0000 7fa14dea7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55781cbcb010 0x55781cbcb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:52.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.136+0000 7fa14dea7b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55781cbcb930 0x55781cbd62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:52.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.136+0000 7fa14dea7b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55781cbd67e0 0x55781cbd8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:52.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.136+0000 7fa14dea7b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55781cb3f4c0 con 0x55781cbcb930 2024-02-02T15:40:52.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.136+0000 7fa14dea7b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55781cb23aa0 con 0x55781cbd67e0 2024-02-02T15:40:52.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.136+0000 7fa14dea7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55781cb21460 con 0x55781cbcb010 2024-02-02T15:40:52.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.136+0000 7fa133400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55781cbcb010 0x55781cbcb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:52.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.136+0000 7fa133e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55781cbd67e0 0x55781cbd8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:52.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.136+0000 7fa133e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55781cbd67e0 0x55781cbd8bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:32862/0 (socket says 172.21.15.1:32862) 2024-02-02T15:40:52.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.136+0000 7fa132a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55781cbcb930 0x55781cbd62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:52.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.136+0000 7fa133e00700 1 -- 172.21.15.1:0/544318639 learned_addr learned my addr 172.21.15.1:0/544318639 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:40:52.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.137+0000 7fa132a00700 1 --2- 172.21.15.1:0/3435637655 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fa1140a0710 0x7fa1140a2bd0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa11c00b2e0 tx=0x7fa11c00b920 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:52.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.137+0000 7fa133400700 1 -- 172.21.15.1:0/544318639 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55781cbd67e0 msgr2=0x55781cbd8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:52.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.137+0000 7fa133400700 1 --2- 172.21.15.1:0/544318639 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55781cbd67e0 0x55781cbd8bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:52.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.137+0000 7fa133400700 1 -- 172.21.15.1:0/544318639 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55781cbcb930 msgr2=0x55781cbd62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:52.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.137+0000 7fa133400700 1 --2- 172.21.15.1:0/544318639 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55781cbcb930 0x55781cbd62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:52.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.137+0000 7fa133400700 1 -- 172.21.15.1:0/544318639 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55781c99e3b0 con 0x55781cbcb010 2024-02-02T15:40:52.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.137+0000 7fa133e00700 1 --2- 172.21.15.1:0/544318639 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55781cbd67e0 0x55781cbd8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:40:52.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.137+0000 7fa133400700 1 --2- 172.21.15.1:0/544318639 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55781cbcb010 0x55781cbcb3f0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fa1240c96b0 tx=0x7fa1240993b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc1a318fc6f3df0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:52.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.137+0000 7fa128c00700 1 -- 172.21.15.1:0/544318639 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa12409d120 con 0x55781cbcb010 2024-02-02T15:40:52.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.137+0000 7fa128c00700 1 -- 172.21.15.1:0/544318639 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa1240cc170 con 0x55781cbcb010 2024-02-02T15:40:52.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.137+0000 7fa128c00700 1 -- 172.21.15.1:0/544318639 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa124099970 con 0x55781cbcb010 2024-02-02T15:40:52.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.138+0000 7fa14dea7b80 1 -- 172.21.15.1:0/544318639 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55781cbcb010 msgr2=0x55781cbcb3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:52.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.138+0000 7fa14dea7b80 1 --2- 172.21.15.1:0/544318639 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55781cbcb010 0x55781cbcb3f0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fa1240c96b0 tx=0x7fa1240993b0 comp rx=0 tx=0).stop 2024-02-02T15:40:52.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.138+0000 7fa14dea7b80 1 -- 172.21.15.1:0/544318639 shutdown_connections 2024-02-02T15:40:52.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.138+0000 7fa14dea7b80 1 --2- 172.21.15.1:0/544318639 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55781cbd67e0 0x55781cbd8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:52.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.138+0000 7fa14dea7b80 1 --2- 172.21.15.1:0/544318639 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55781cbcb010 0x55781cbcb3f0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:52.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.138+0000 7fa14dea7b80 1 --2- 172.21.15.1:0/544318639 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55781cbcb930 0x55781cbd62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:52.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.138+0000 7fa14dea7b80 1 -- 172.21.15.1:0/544318639 >> 172.21.15.1:0/544318639 conn(0x55781cb2adb0 msgr2=0x55781cbc73b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:52.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.138+0000 7fa14dea7b80 1 -- 172.21.15.1:0/544318639 shutdown_connections 2024-02-02T15:40:52.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.138+0000 7fa14dea7b80 1 -- 172.21.15.1:0/544318639 wait complete. 2024-02-02T15:40:52.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.139+0000 7fa14dea7b80 1 -- start start 2024-02-02T15:40:52.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.139+0000 7fa14dea7b80 1 Processor -- start 2024-02-02T15:40:52.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.140+0000 7fa14dea7b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55781cbcb010 0x55781cbd6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:52.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.140+0000 7fa14dea7b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55781cbcb930 0x55781cbd34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:52.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.140+0000 7fa14dea7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55781cbd67e0 0x55781cbd3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:52.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.140+0000 7fa14dea7b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55781cb1cd50 con 0x55781cbcb010 2024-02-02T15:40:52.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.140+0000 7fa14dea7b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55781cab8d80 con 0x55781cbcb930 2024-02-02T15:40:52.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.140+0000 7fa14dea7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55781cab8a20 con 0x55781cbd67e0 2024-02-02T15:40:52.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.140+0000 7fa14dea7b80 10 client.0 _ll_register_callbacks cb 0x55781caba290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:40:52.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.140+0000 7fa132a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55781cbcb930 0x55781cbd34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:52.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.140+0000 7fa133400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55781cbcb010 0x55781cbd6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:52.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.140+0000 7fa133e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55781cbd67e0 0x55781cbd3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:52.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.140+0000 7fa132a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55781cbcb930 0x55781cbd34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:32876/0 (socket says 172.21.15.1:32876) 2024-02-02T15:40:52.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.140+0000 7fa133e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55781cbd67e0 0x55781cbd3bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:36558/0 (socket says 172.21.15.1:36558) 2024-02-02T15:40:52.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.140+0000 7fa132a00700 1 -- 172.21.15.1:0/2190180932 learned_addr learned my addr 172.21.15.1:0/2190180932 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:40:52.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.140+0000 7fa132a00700 1 -- 172.21.15.1:0/2190180932 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55781cbd67e0 msgr2=0x55781cbd3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:52.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.140+0000 7fa132a00700 1 --2- 172.21.15.1:0/2190180932 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55781cbd67e0 0x55781cbd3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:52.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.140+0000 7fa132a00700 1 -- 172.21.15.1:0/2190180932 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55781cbcb010 msgr2=0x55781cbd6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:52.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.140+0000 7fa132a00700 1 --2- 172.21.15.1:0/2190180932 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55781cbcb010 0x55781cbd6250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:52.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.140+0000 7fa132a00700 1 -- 172.21.15.1:0/2190180932 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa12c0090b0 con 0x55781cbcb930 2024-02-02T15:40:52.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.140+0000 7fa133400700 1 --2- 172.21.15.1:0/2190180932 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55781cbcb010 0x55781cbd6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:40:52.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.141+0000 7fa132a00700 1 --2- 172.21.15.1:0/2190180932 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55781cbcb930 0x55781cbd34d0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fa11c009990 tx=0x7fa11c009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed8c6a3a04f62891 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:52.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.141+0000 7fa121600700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:40:52.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.141+0000 7fa121600700 1 -- 172.21.15.1:0/2190180932 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa11c004960 con 0x55781cbcb930 2024-02-02T15:40:52.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.141+0000 7fa14dea7b80 1 -- 172.21.15.1:0/2190180932 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55781cbbf0f0 con 0x55781cbcb930 2024-02-02T15:40:52.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.141+0000 7fa121600700 1 -- 172.21.15.1:0/2190180932 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fa11c02c040 con 0x55781cbcb930 2024-02-02T15:40:52.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.141+0000 7fa121600700 1 -- 172.21.15.1:0/2190180932 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa11c004ac0 con 0x55781cbcb930 2024-02-02T15:40:52.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.141+0000 7fa14dea7b80 10 client.29246 fetch_fsmap learned FSMap version 58 2024-02-02T15:40:52.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.141+0000 7fa14dea7b80 1 -- 172.21.15.1:0/2190180932 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x7fa1341a61b0 con 0x55781cbcb930 2024-02-02T15:40:52.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.141+0000 7fa121600700 1 -- 172.21.15.1:0/2190180932 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55781cbbf0f0 con 0x55781cbcb930 2024-02-02T15:40:52.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.142+0000 7fa121600700 1 -- 172.21.15.1:0/2190180932 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 58) v1 ==== 7526+0+0 (secure 0 0 0) 0x7fa11c030450 con 0x55781cbcb930 2024-02-02T15:40:52.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.142+0000 7fa14dea7b80 10 client.29246 fetch_fsmap finished waiting for FSMap version 58 2024-02-02T15:40:52.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.142+0000 7fa14dea7b80 10 client.29246 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:40:52.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.142+0000 7fa14dea7b80 20 client.29246 populate_metadata read hostname 'smithi001' 2024-02-02T15:40:52.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.142+0000 7fa14dea7b80 1 --2- 172.21.15.1:0/2190180932 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55781cbe6170 0x55781cbe8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:52.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.142+0000 7fa14dea7b80 4 client.29246 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:40:52.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.142+0000 7fa14dea7b80 1 -- 172.21.15.1:0/2190180932 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55781cb15dd0 con 0x55781cbe6170 2024-02-02T15:40:52.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.142+0000 7fa133400700 1 --2- 172.21.15.1:0/2190180932 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55781cbe6170 0x55781cbe8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:52.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.142+0000 7fa133400700 1 --2- 172.21.15.1:0/2190180932 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55781cbe6170 0x55781cbe8590 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3571fbdec7132aa server_cookie=b0bb45cf6ade1ef4 in_seq=0 out_seq=0 2024-02-02T15:40:52.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.142+0000 7fa121600700 10 client.29246 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:40:52.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.145+0000 7fa121600700 1 -- 172.21.15.1:0/2190180932 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55781cb15dd0 con 0x55781cbe6170 2024-02-02T15:40:52.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.145+0000 7fa121600700 10 client.29246 handle_command_reply: tid=0 2024-02-02T15:40:52.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.145+0000 7fa14dea7b80 1 client.29246 shutdown 2024-02-02T15:40:52.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.146+0000 7fa14dea7b80 1 -- 172.21.15.1:0/2190180932 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55781cbcb930 msgr2=0x55781cbd34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:52.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.146+0000 7fa14dea7b80 1 --2- 172.21.15.1:0/2190180932 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55781cbcb930 0x55781cbd34d0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fa11c009990 tx=0x7fa11c009ca0 comp rx=0 tx=0).stop 2024-02-02T15:40:52.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.146+0000 7fa14dea7b80 1 -- 172.21.15.1:0/2190180932 shutdown_connections 2024-02-02T15:40:52.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.146+0000 7fa14dea7b80 1 --2- 172.21.15.1:0/2190180932 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55781cbe6170 0x55781cbe8590 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:52.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.146+0000 7fa14dea7b80 1 --2- 172.21.15.1:0/2190180932 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55781cbd67e0 0x55781cbd3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:52.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.146+0000 7fa14dea7b80 1 --2- 172.21.15.1:0/2190180932 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55781cbcb010 0x55781cbd6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:52.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.146+0000 7fa14dea7b80 1 --2- 172.21.15.1:0/2190180932 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55781cbcb930 0x55781cbd34d0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:52.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.146+0000 7fa121600700 0 client.29246 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:40:52.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.146+0000 7fa14dea7b80 1 -- 172.21.15.1:0/2190180932 >> 172.21.15.1:0/2190180932 conn(0x55781cb2adb0 msgr2=0x55781cbc6fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:52.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.146+0000 7fa14dea7b80 1 -- 172.21.15.1:0/2190180932 shutdown_connections 2024-02-02T15:40:52.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.146+0000 7fa14dea7b80 1 -- 172.21.15.1:0/2190180932 wait complete. 2024-02-02T15:40:52.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.146+0000 7fa14dea7b80 20 client.29246 trim_cache size 0 max 16384 2024-02-02T15:40:52.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.167+0000 7fa14dea7b80 1 Processor -- start 2024-02-02T15:40:52.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.167+0000 7fa14dea7b80 1 -- start start 2024-02-02T15:40:52.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.167+0000 7fa14dea7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55781cbcb010 0x55781cbdd330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:52.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.167+0000 7fa14dea7b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55781cbcb930 0x55781cbd5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:52.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.167+0000 7fa14dea7b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55781cbd67e0 0x55781cbce840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:52.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.167+0000 7fa14dea7b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55781c9a06a0 con 0x55781cbcb930 2024-02-02T15:40:52.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.167+0000 7fa14dea7b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55781cbcbd40 con 0x55781cbd67e0 2024-02-02T15:40:52.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.167+0000 7fa14dea7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55781cbcc880 con 0x55781cbcb010 2024-02-02T15:40:52.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.168+0000 7fa133400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55781cbcb010 0x55781cbdd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:52.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.168+0000 7fa133400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55781cbcb010 0x55781cbdd330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:36564/0 (socket says 172.21.15.1:36564) 2024-02-02T15:40:52.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.168+0000 7fa132a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55781cbcb930 0x55781cbd5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:52.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.168+0000 7fa133400700 1 -- 172.21.15.1:0/2648330401 learned_addr learned my addr 172.21.15.1:0/2648330401 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:40:52.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.168+0000 7fa133e00700 1 --2- 172.21.15.1:0/2648330401 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55781cbd67e0 0x55781cbce840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:52.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.168+0000 7fa133400700 1 -- 172.21.15.1:0/2648330401 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55781cbd67e0 msgr2=0x55781cbce840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:52.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.168+0000 7fa133400700 1 --2- 172.21.15.1:0/2648330401 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55781cbd67e0 0x55781cbce840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:52.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.168+0000 7fa133400700 1 -- 172.21.15.1:0/2648330401 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55781cbcb930 msgr2=0x55781cbd5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:52.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.168+0000 7fa133400700 1 --2- 172.21.15.1:0/2648330401 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55781cbcb930 0x55781cbd5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:52.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.168+0000 7fa133400700 1 -- 172.21.15.1:0/2648330401 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55781c94ab60 con 0x55781cbcb010 2024-02-02T15:40:52.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.168+0000 7fa132a00700 1 --2- 172.21.15.1:0/2648330401 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55781cbcb930 0x55781cbd5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:40:52.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.168+0000 7fa133400700 1 --2- 172.21.15.1:0/2648330401 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55781cbcb010 0x55781cbdd330 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fa12400e970 tx=0x7fa1240951f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bca2d9a9dc77b7d3 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:52.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.169+0000 7fa129600700 1 -- 172.21.15.1:0/2648330401 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa124099530 con 0x55781cbcb010 2024-02-02T15:40:52.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.169+0000 7fa129600700 1 -- 172.21.15.1:0/2648330401 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa12409d2e0 con 0x55781cbcb010 2024-02-02T15:40:52.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.169+0000 7fa129600700 1 -- 172.21.15.1:0/2648330401 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa124099970 con 0x55781cbcb010 2024-02-02T15:40:52.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.169+0000 7fa14dea7b80 1 -- 172.21.15.1:0/2648330401 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55781cbcb010 msgr2=0x55781cbdd330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:52.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.169+0000 7fa14dea7b80 1 --2- 172.21.15.1:0/2648330401 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55781cbcb010 0x55781cbdd330 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fa12400e970 tx=0x7fa1240951f0 comp rx=0 tx=0).stop 2024-02-02T15:40:52.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.169+0000 7fa14dea7b80 1 -- 172.21.15.1:0/2648330401 shutdown_connections 2024-02-02T15:40:52.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.169+0000 7fa14dea7b80 1 --2- 172.21.15.1:0/2648330401 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55781cbd67e0 0x55781cbce840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:52.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.169+0000 7fa14dea7b80 1 --2- 172.21.15.1:0/2648330401 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55781cbcb010 0x55781cbdd330 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:52.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.169+0000 7fa14dea7b80 1 --2- 172.21.15.1:0/2648330401 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55781cbcb930 0x55781cbd5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:52.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.169+0000 7fa14dea7b80 1 -- 172.21.15.1:0/2648330401 >> 172.21.15.1:0/2648330401 conn(0x55781cb2adb0 msgr2=0x55781cbc5b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:52.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.170+0000 7fa14dea7b80 1 -- 172.21.15.1:0/2648330401 shutdown_connections 2024-02-02T15:40:52.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.170+0000 7fa14dea7b80 1 -- 172.21.15.1:0/2648330401 wait complete. 2024-02-02T15:40:52.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.170+0000 7fa14dea7b80 1 -- start start 2024-02-02T15:40:52.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.170+0000 7fa14dea7b80 1 Processor -- start 2024-02-02T15:40:52.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.171+0000 7fa14dea7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55781cbcb010 0x55781cbdac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:52.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.171+0000 7fa14dea7b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55781cbcb930 0x55781cbdb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:52.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.171+0000 7fa14dea7b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55781cbd67e0 0x55781cc36c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:52.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.171+0000 7fa14dea7b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55781cbdbae0 con 0x55781cbd67e0 2024-02-02T15:40:52.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.171+0000 7fa14dea7b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55781cc371d0 con 0x55781cbcb930 2024-02-02T15:40:52.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.171+0000 7fa14dea7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55781cc37390 con 0x55781cbcb010 2024-02-02T15:40:52.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.171+0000 7fa14dea7b80 10 client.0 _ll_register_callbacks cb 0x55781caba290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:40:52.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.171+0000 7fa133400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55781cbcb010 0x55781cbdac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:52.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.171+0000 7fa133400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55781cbcb010 0x55781cbdac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:36574/0 (socket says 172.21.15.1:36574) 2024-02-02T15:40:52.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.171+0000 7fa133400700 1 -- 172.21.15.1:0/1272226687 learned_addr learned my addr 172.21.15.1:0/1272226687 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:40:52.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.171+0000 7fa133e00700 1 --2- 172.21.15.1:0/1272226687 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55781cbd67e0 0x55781cc36c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:52.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.171+0000 7fa132a00700 1 --2- 172.21.15.1:0/1272226687 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55781cbcb930 0x55781cbdb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:52.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.171+0000 7fa133400700 1 -- 172.21.15.1:0/1272226687 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55781cbcb930 msgr2=0x55781cbdb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:52.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.171+0000 7fa133400700 1 --2- 172.21.15.1:0/1272226687 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55781cbcb930 0x55781cbdb1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:52.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.171+0000 7fa133400700 1 -- 172.21.15.1:0/1272226687 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55781cbd67e0 msgr2=0x55781cc36c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:52.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.172+0000 7fa133400700 1 --2- 172.21.15.1:0/1272226687 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55781cbd67e0 0x55781cc36c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:52.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.172+0000 7fa133400700 1 -- 172.21.15.1:0/1272226687 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1341a61b0 con 0x55781cbcb010 2024-02-02T15:40:52.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.172+0000 7fa133400700 1 --2- 172.21.15.1:0/1272226687 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55781cbcb010 0x55781cbdac60 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fa1240cc300 tx=0x7fa124099530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ff00cf472073b5d server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:52.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.172+0000 7fa121600700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:40:52.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.172+0000 7fa121600700 1 -- 172.21.15.1:0/1272226687 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa124099710 con 0x55781cbcb010 2024-02-02T15:40:52.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.172+0000 7fa14dea7b80 1 -- 172.21.15.1:0/1272226687 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa1240cc170 con 0x55781cbcb010 2024-02-02T15:40:52.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.172+0000 7fa121600700 1 -- 172.21.15.1:0/1272226687 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fa124099870 con 0x55781cbcb010 2024-02-02T15:40:52.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.172+0000 7fa121600700 1 -- 172.21.15.1:0/1272226687 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa1240a1090 con 0x55781cbcb010 2024-02-02T15:40:52.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.172+0000 7fa14dea7b80 10 client.29248 fetch_fsmap learned FSMap version 58 2024-02-02T15:40:52.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.172+0000 7fa14dea7b80 1 -- 172.21.15.1:0/1272226687 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x55781c99e3b0 con 0x55781cbcb010 2024-02-02T15:40:52.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.172+0000 7fa121600700 1 -- 172.21.15.1:0/1272226687 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa1240cc170 con 0x55781cbcb010 2024-02-02T15:40:52.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.173+0000 7fa121600700 1 -- 172.21.15.1:0/1272226687 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 58) v1 ==== 7526+0+0 (secure 0 0 0) 0x7fa124098410 con 0x55781cbcb010 2024-02-02T15:40:52.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.173+0000 7fa14dea7b80 10 client.29248 fetch_fsmap finished waiting for FSMap version 58 2024-02-02T15:40:52.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.173+0000 7fa14dea7b80 10 client.29248 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:40:52.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.173+0000 7fa14dea7b80 20 client.29248 populate_metadata read hostname 'smithi001' 2024-02-02T15:40:52.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.173+0000 7fa14dea7b80 1 --2- 172.21.15.1:0/1272226687 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55781cc385b0 0x55781cc3a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:52.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.173+0000 7fa14dea7b80 4 client.29248 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2024-02-02T15:40:52.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.173+0000 7fa14dea7b80 1 -- 172.21.15.1:0/1272226687 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55781cadbf50 con 0x55781cc385b0 2024-02-02T15:40:52.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.173+0000 7fa132a00700 1 --2- 172.21.15.1:0/1272226687 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55781cc385b0 0x55781cc3a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:52.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.174+0000 7fa132a00700 1 --2- 172.21.15.1:0/1272226687 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55781cc385b0 0x55781cc3a9a0 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e21e8d9eff51070a server_cookie=87a7480bd03b0c83 in_seq=0 out_seq=0 2024-02-02T15:40:52.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.174+0000 7fa121600700 10 client.29248 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:40:52.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.178+0000 7fa121600700 1 -- 172.21.15.1:0/1272226687 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55781cadbf50 con 0x55781cc385b0 2024-02-02T15:40:52.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.178+0000 7fa121600700 10 client.29248 handle_command_reply: tid=0 2024-02-02T15:40:52.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.178+0000 7fa14dea7b80 1 client.29248 shutdown 2024-02-02T15:40:52.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.179+0000 7fa14dea7b80 1 -- 172.21.15.1:0/1272226687 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55781cbcb010 msgr2=0x55781cbdac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:52.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.179+0000 7fa14dea7b80 1 --2- 172.21.15.1:0/1272226687 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55781cbcb010 0x55781cbdac60 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fa1240cc300 tx=0x7fa124099530 comp rx=0 tx=0).stop 2024-02-02T15:40:52.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.179+0000 7fa14dea7b80 1 -- 172.21.15.1:0/1272226687 shutdown_connections 2024-02-02T15:40:52.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.179+0000 7fa14dea7b80 1 --2- 172.21.15.1:0/1272226687 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55781cc385b0 0x55781cc3a9a0 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:52.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.179+0000 7fa14dea7b80 1 --2- 172.21.15.1:0/1272226687 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55781cbd67e0 0x55781cc36c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:52.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.179+0000 7fa14dea7b80 1 --2- 172.21.15.1:0/1272226687 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55781cbcb010 0x55781cbdac60 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:52.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.179+0000 7fa121600700 0 client.29248 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:40:52.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.179+0000 7fa14dea7b80 1 --2- 172.21.15.1:0/1272226687 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55781cbcb930 0x55781cbdb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:52.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.179+0000 7fa14dea7b80 1 -- 172.21.15.1:0/1272226687 >> 172.21.15.1:0/1272226687 conn(0x55781cb2adb0 msgr2=0x55781cbce840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:52.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.179+0000 7fa14dea7b80 1 -- 172.21.15.1:0/1272226687 shutdown_connections 2024-02-02T15:40:52.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.179+0000 7fa14dea7b80 1 -- 172.21.15.1:0/1272226687 wait complete. 2024-02-02T15:40:52.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.179+0000 7fa14dea7b80 20 client.29248 trim_cache size 0 max 16384 2024-02-02T15:40:52.180 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:40:52.180 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2024-02-02T15:40:52.180 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2024-02-02T15:40:52.181 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:40:52.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.179+0000 7fa13a400700 1 -- 172.21.15.1:0/3435637655 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fa1140a0710 msgr2=0x7fa1140a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:52.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.179+0000 7fa13a400700 1 --2- 172.21.15.1:0/3435637655 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fa1140a0710 0x7fa1140a2bd0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa11c00b2e0 tx=0x7fa11c00b920 comp rx=0 tx=0).stop 2024-02-02T15:40:52.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.179+0000 7fa13a400700 1 -- 172.21.15.1:0/3435637655 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa134100fa0 msgr2=0x7fa134199020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:52.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.180+0000 7fa13a400700 1 --2- 172.21.15.1:0/3435637655 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa134100fa0 0x7fa134199020 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fa12400e9a0 tx=0x7fa12400ecb0 comp rx=0 tx=0).stop 2024-02-02T15:40:52.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.180+0000 7fa13a400700 1 -- 172.21.15.1:0/3435637655 shutdown_connections 2024-02-02T15:40:52.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.180+0000 7fa13a400700 1 --2- 172.21.15.1:0/3435637655 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fa1140a0710 0x7fa1140a2bd0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:52.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.180+0000 7fa13a400700 1 --2- 172.21.15.1:0/3435637655 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa13419d9c0 0x7fa13419de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:52.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.180+0000 7fa13a400700 1 --2- 172.21.15.1:0/3435637655 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa134100fa0 0x7fa134199020 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:52.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.180+0000 7fa13a400700 1 --2- 172.21.15.1:0/3435637655 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa1341060c0 0x7fa134199560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:52.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.180+0000 7fa13a400700 1 -- 172.21.15.1:0/3435637655 >> 172.21.15.1:0/3435637655 conn(0x7fa1340fc800 msgr2=0x7fa1340fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:52.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.180+0000 7fa13a400700 1 -- 172.21.15.1:0/3435637655 shutdown_connections 2024-02-02T15:40:52.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.180+0000 7fa13a400700 1 -- 172.21.15.1:0/3435637655 wait complete. 2024-02-02T15:40:52.198 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-02T15:40:52.198 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T15:40:52.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.437+0000 7f91bec00700 1 -- 172.21.15.1:0/1590491917 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f91b81018c0 msgr2=0x7f91b8105900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:52.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.437+0000 7f91bec00700 1 --2- 172.21.15.1:0/1590491917 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f91b81018c0 0x7f91b8105900 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f91a0009990 tx=0x7f91a0009ca0 comp rx=0 tx=0).stop 2024-02-02T15:40:52.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.438+0000 7f91bec00700 1 -- 172.21.15.1:0/1590491917 shutdown_connections 2024-02-02T15:40:52.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.438+0000 7f91bec00700 1 --2- 172.21.15.1:0/1590491917 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91b8106030 0x7f91b810dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:52.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.438+0000 7f91bec00700 1 --2- 172.21.15.1:0/1590491917 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f91b8100f10 0x7f91b81012f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:52.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.438+0000 7f91bec00700 1 --2- 172.21.15.1:0/1590491917 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f91b81018c0 0x7f91b8105900 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:52.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.438+0000 7f91bec00700 1 -- 172.21.15.1:0/1590491917 >> 172.21.15.1:0/1590491917 conn(0x7f91b80fc790 msgr2=0x7f91b80febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:52.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.438+0000 7f91bec00700 1 -- 172.21.15.1:0/1590491917 shutdown_connections 2024-02-02T15:40:52.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.438+0000 7f91bec00700 1 -- 172.21.15.1:0/1590491917 wait complete. 2024-02-02T15:40:52.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.439+0000 7f91bec00700 1 Processor -- start 2024-02-02T15:40:52.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.439+0000 7f91bec00700 1 -- start start 2024-02-02T15:40:52.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.439+0000 7f91bec00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91b8100f10 0x7f91b8198f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:52.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.439+0000 7f91bec00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f91b8106030 0x7f91b81994d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:52.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.439+0000 7f91bec00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f91b819d930 0x7f91b819ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:52.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.439+0000 7f91bec00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91b819e3b0 con 0x7f91b819d930 2024-02-02T15:40:52.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.439+0000 7f91bec00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91b819e520 con 0x7f91b8106030 2024-02-02T15:40:52.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.439+0000 7f91bec00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91b819e6e0 con 0x7f91b8100f10 2024-02-02T15:40:52.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.440+0000 7f91b7400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f91b8106030 0x7f91b81994d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:52.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.440+0000 7f91b7e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91b8100f10 0x7f91b8198f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:52.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.440+0000 7f91bcc00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f91b819d930 0x7f91b819ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:52.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.440+0000 7f91b7400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f91b8106030 0x7f91b81994d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:32910/0 (socket says 172.21.15.1:32910) 2024-02-02T15:40:52.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.440+0000 7f91b7400700 1 -- 172.21.15.1:0/36731602 learned_addr learned my addr 172.21.15.1:0/36731602 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:40:52.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.440+0000 7f91b7400700 1 -- 172.21.15.1:0/36731602 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91b8100f10 msgr2=0x7f91b8198f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:52.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.440+0000 7f91b7400700 1 --2- 172.21.15.1:0/36731602 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91b8100f10 0x7f91b8198f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:52.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.440+0000 7f91b7400700 1 -- 172.21.15.1:0/36731602 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f91b819d930 msgr2=0x7f91b819ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:52.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.440+0000 7f91b7400700 1 --2- 172.21.15.1:0/36731602 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f91b819d930 0x7f91b819ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:52.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.440+0000 7f91b7400700 1 -- 172.21.15.1:0/36731602 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91a00095f0 con 0x7f91b8106030 2024-02-02T15:40:52.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.440+0000 7f91b7400700 1 --2- 172.21.15.1:0/36731602 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f91b8106030 0x7f91b81994d0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f91a0004740 tx=0x7f91a0009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:52.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.441+0000 7f91b4c00700 1 -- 172.21.15.1:0/36731602 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a001d070 con 0x7f91b8106030 2024-02-02T15:40:52.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.441+0000 7f91bec00700 1 -- 172.21.15.1:0/36731602 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91b81a5be0 con 0x7f91b8106030 2024-02-02T15:40:52.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.441+0000 7f91b4c00700 1 -- 172.21.15.1:0/36731602 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f91a0004830 con 0x7f91b8106030 2024-02-02T15:40:52.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.441+0000 7f91b4c00700 1 -- 172.21.15.1:0/36731602 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a0005280 con 0x7f91b8106030 2024-02-02T15:40:52.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.441+0000 7f91bec00700 1 -- 172.21.15.1:0/36731602 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91b81a6000 con 0x7f91b8106030 2024-02-02T15:40:52.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.442+0000 7f91bec00700 1 -- 172.21.15.1:0/36731602 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91b8067bc0 con 0x7f91b8106030 2024-02-02T15:40:52.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.444+0000 7f91b4c00700 1 -- 172.21.15.1:0/36731602 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f91a000fb00 con 0x7f91b8106030 2024-02-02T15:40:52.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.445+0000 7f91b4c00700 1 --2- 172.21.15.1:0/36731602 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f91980a0690 0x7f91980a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:52.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.446+0000 7f91b4c00700 1 -- 172.21.15.1:0/36731602 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f91a00cd5f0 con 0x7f91b8106030 2024-02-02T15:40:52.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.446+0000 7f91b7e00700 1 --2- 172.21.15.1:0/36731602 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f91980a0690 0x7f91980a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:52.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.446+0000 7f91b7e00700 1 --2- 172.21.15.1:0/36731602 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f91980a0690 0x7f91980a2b50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f91a8009c60 tx=0x7f91a8009260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:52.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.448+0000 7f91b4c00700 1 -- 172.21.15.1:0/36731602 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f91a0096ae0 con 0x7f91b8106030 2024-02-02T15:40:52.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.794+0000 7f91bec00700 1 -- 172.21.15.1:0/36731602 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f91b819a2e0 con 0x7f91b8106030 2024-02-02T15:40:52.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.795+0000 7f91b4c00700 1 -- 172.21.15.1:0/36731602 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+10895 (secure 0 0 0) 0x7f91a009a100 con 0x7f91b8106030 2024-02-02T15:40:52.795 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:40:52.795 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":59,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2922240448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2922240448},{"type":"v1","addr":"172.21.15.1:6835","nonce":2922240448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2729497431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2729497431},{"type":"v1","addr":"172.21.15.1:6837","nonce":2729497431}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":59,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-02T15:24:47.761435+0000","modified":"2024-02-02T15:40:52.471883+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24529,"mds_2":24518,"mds_3":24514,"mds_4":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14517":{"gid":14517,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/1218303683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":1218303683},{"type":"v1","addr":"172.21.15.1:6839","nonce":1218303683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14529":{"gid":14529,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/2059197496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2059197496},{"type":"v1","addr":"172.21.15.1:6841","nonce":2059197496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/7648986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":7648986},{"type":"v1","addr":"172.21.15.142:6833","nonce":7648986}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/3482746094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3482746094},{"type":"v1","addr":"172.21.15.71:6833","nonce":3482746094}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/388207814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":388207814},{"type":"v1","addr":"172.21.15.71:6835","nonce":388207814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6835/1052606547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1052606547},{"type":"v1","addr":"172.21.15.142:6835","nonce":1052606547}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.142:6837/2870289019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2870289019},{"type":"v1","addr":"172.21.15.142:6837","nonce":2870289019}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.71:6837/628016199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":628016199},{"type":"v1","addr":"172.21.15.71:6837","nonce":628016199}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.142:6839/321675421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":321675421},{"type":"v1","addr":"172.21.15.142:6839","nonce":321675421}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/1268086493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1268086493},{"type":"v1","addr":"172.21.15.71:6839","nonce":1268086493}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-02T15:40:52.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.798+0000 7f91bec00700 1 -- 172.21.15.1:0/36731602 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f91980a0690 msgr2=0x7f91980a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:52.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.798+0000 7f91bec00700 1 --2- 172.21.15.1:0/36731602 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f91980a0690 0x7f91980a2b50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f91a8009c60 tx=0x7f91a8009260 comp rx=0 tx=0).stop 2024-02-02T15:40:52.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.798+0000 7f91bec00700 1 -- 172.21.15.1:0/36731602 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f91b8106030 msgr2=0x7f91b81994d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:52.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.798+0000 7f91bec00700 1 --2- 172.21.15.1:0/36731602 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f91b8106030 0x7f91b81994d0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f91a0004740 tx=0x7f91a0009580 comp rx=0 tx=0).stop 2024-02-02T15:40:52.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.799+0000 7f91bec00700 1 -- 172.21.15.1:0/36731602 shutdown_connections 2024-02-02T15:40:52.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.799+0000 7f91bec00700 1 --2- 172.21.15.1:0/36731602 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f91980a0690 0x7f91980a2b50 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:52.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.799+0000 7f91bec00700 1 --2- 172.21.15.1:0/36731602 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f91b819d930 0x7f91b819ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:52.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.799+0000 7f91bec00700 1 --2- 172.21.15.1:0/36731602 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f91b8100f10 0x7f91b8198f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:52.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.799+0000 7f91bec00700 1 --2- 172.21.15.1:0/36731602 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f91b8106030 0x7f91b81994d0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:52.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.799+0000 7f91bec00700 1 -- 172.21.15.1:0/36731602 >> 172.21.15.1:0/36731602 conn(0x7f91b80fc790 msgr2=0x7f91b81028d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:52.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.799+0000 7f91bec00700 1 -- 172.21.15.1:0/36731602 shutdown_connections 2024-02-02T15:40:52.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:52.799+0000 7f91bec00700 1 -- 172.21.15.1:0/36731602 wait complete. 2024-02-02T15:40:52.800 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 59 2024-02-02T15:40:52.821 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-02-02T15:40:52.968 INFO:tasks.workunit.client.0.smithi001.stdout: 10 9875 27.36 MB/sec warmup 25 sec latency 2804.595 ms 2024-02-02T15:40:53.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.079+0000 7f37cb200700 1 -- 172.21.15.1:0/1384912400 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37cc100550 msgr2=0x7f37cc10c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:53.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.079+0000 7f37cb200700 1 --2- 172.21.15.1:0/1384912400 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37cc100550 0x7f37cc10c810 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f37c4009f90 tx=0x7f37c400b330 comp rx=0 tx=0).stop 2024-02-02T15:40:53.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.079+0000 7f37cb200700 1 -- 172.21.15.1:0/1384912400 shutdown_connections 2024-02-02T15:40:53.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.079+0000 7f37cb200700 1 --2- 172.21.15.1:0/1384912400 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37cc100550 0x7f37cc10c810 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.079+0000 7f37cb200700 1 --2- 172.21.15.1:0/1384912400 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f37cc104980 0x7f37cc104d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.079+0000 7f37cb200700 1 --2- 172.21.15.1:0/1384912400 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37cc0ffb90 0x7f37cc100010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.079+0000 7f37cb200700 1 -- 172.21.15.1:0/1384912400 >> 172.21.15.1:0/1384912400 conn(0x7f37cc076770 msgr2=0x7f37cc076b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:53.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.079+0000 7f37cb200700 1 -- 172.21.15.1:0/1384912400 shutdown_connections 2024-02-02T15:40:53.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.079+0000 7f37cb200700 1 -- 172.21.15.1:0/1384912400 wait complete. 2024-02-02T15:40:53.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.081+0000 7f37cb200700 1 Processor -- start 2024-02-02T15:40:53.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.081+0000 7f37cb200700 1 -- start start 2024-02-02T15:40:53.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.082+0000 7f37cb200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f37cc0ffb90 0x7f37cc198e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:53.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.082+0000 7f37cb200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37cc100550 0x7f37cc1993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:53.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.082+0000 7f37cb200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37cc104980 0x7f37cc19d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:53.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.082+0000 7f37cb200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37cc19de20 con 0x7f37cc0ffb90 2024-02-02T15:40:53.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.082+0000 7f37cb200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37cc19df90 con 0x7f37cc100550 2024-02-02T15:40:53.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.082+0000 7f37cb200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37cc19e150 con 0x7f37cc104980 2024-02-02T15:40:53.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.082+0000 7f37c9400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37cc100550 0x7f37cc1993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:53.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.082+0000 7f37c9400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37cc100550 0x7f37cc1993d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:41606/0 (socket says 172.21.15.1:41606) 2024-02-02T15:40:53.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.082+0000 7f37c9e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f37cc0ffb90 0x7f37cc198e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:53.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.082+0000 7f37c9400700 1 -- 172.21.15.1:0/1937541763 learned_addr learned my addr 172.21.15.1:0/1937541763 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:40:53.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.082+0000 7f37ca800700 1 --2- 172.21.15.1:0/1937541763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37cc104980 0x7f37cc19d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:53.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.082+0000 7f37c9400700 1 -- 172.21.15.1:0/1937541763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37cc104980 msgr2=0x7f37cc19d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:53.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.082+0000 7f37c9400700 1 --2- 172.21.15.1:0/1937541763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37cc104980 0x7f37cc19d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.082+0000 7f37c9400700 1 -- 172.21.15.1:0/1937541763 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f37cc0ffb90 msgr2=0x7f37cc198e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:53.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.082+0000 7f37c9400700 1 --2- 172.21.15.1:0/1937541763 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f37cc0ffb90 0x7f37cc198e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.082+0000 7f37c9400700 1 -- 172.21.15.1:0/1937541763 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37b40095f0 con 0x7f37cc100550 2024-02-02T15:40:53.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.083+0000 7f37c9400700 1 --2- 172.21.15.1:0/1937541763 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37cc100550 0x7f37cc1993d0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f37b400ec10 tx=0x7f37b400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:53.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.083+0000 7f37c2a00700 1 -- 172.21.15.1:0/1937541763 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b400cc90 con 0x7f37cc100550 2024-02-02T15:40:53.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.083+0000 7f37cb200700 1 -- 172.21.15.1:0/1937541763 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37c4009c40 con 0x7f37cc100550 2024-02-02T15:40:53.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.083+0000 7f37cb200700 1 -- 172.21.15.1:0/1937541763 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37cc19e620 con 0x7f37cc100550 2024-02-02T15:40:53.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.084+0000 7f37c2a00700 1 -- 172.21.15.1:0/1937541763 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f37b400cdf0 con 0x7f37cc100550 2024-02-02T15:40:53.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.084+0000 7f37c2a00700 1 -- 172.21.15.1:0/1937541763 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b4010780 con 0x7f37cc100550 2024-02-02T15:40:53.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.088+0000 7f37c2a00700 1 -- 172.21.15.1:0/1937541763 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f37b40108e0 con 0x7f37cc100550 2024-02-02T15:40:53.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.088+0000 7f37e464eb80 1 Processor -- start 2024-02-02T15:40:53.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.088+0000 7f37e464eb80 1 -- start start 2024-02-02T15:40:53.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.089+0000 7f37e464eb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd9cc7d1f0 0x55bd9cc7d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:53.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.089+0000 7f37e464eb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bd9cc7db10 0x55bd9cc884b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:53.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.089+0000 7f37e464eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bd9cc889f0 0x55bd9cc8ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:53.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.089+0000 7f37e464eb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd9cbf14c0 con 0x55bd9cc7d1f0 2024-02-02T15:40:53.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.089+0000 7f37e464eb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd9cbd5aa0 con 0x55bd9cc7db10 2024-02-02T15:40:53.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.089+0000 7f37e464eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd9cbd3460 con 0x55bd9cc889f0 2024-02-02T15:40:53.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.089+0000 7f37ca800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd9cc7d1f0 0x55bd9cc7d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:53.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.089+0000 7f37c9e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bd9cc7db10 0x55bd9cc884b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:53.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.089+0000 7f37ca800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd9cc7d1f0 0x55bd9cc7d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53572/0 (socket says 172.21.15.1:53572) 2024-02-02T15:40:53.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.089+0000 7f37c9e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bd9cc7db10 0x55bd9cc884b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:41612/0 (socket says 172.21.15.1:41612) 2024-02-02T15:40:53.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.089+0000 7f37ca800700 1 -- 172.21.15.1:0/1429817741 learned_addr learned my addr 172.21.15.1:0/1429817741 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:40:53.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.090+0000 7f37c9400700 1 --2- 172.21.15.1:0/1429817741 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bd9cc889f0 0x55bd9cc8ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:53.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.090+0000 7f37c2a00700 1 --2- 172.21.15.1:0/1937541763 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f37ac0a0710 0x7f37ac0a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:53.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.090+0000 7f37c9e00700 1 -- 172.21.15.1:0/1429817741 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bd9cc889f0 msgr2=0x55bd9cc8ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:53.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.090+0000 7f37c9e00700 1 --2- 172.21.15.1:0/1429817741 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bd9cc889f0 0x55bd9cc8ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.090+0000 7f37c9e00700 1 -- 172.21.15.1:0/1429817741 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd9cc7d1f0 msgr2=0x55bd9cc7d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:53.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.090+0000 7f37c9e00700 1 --2- 172.21.15.1:0/1429817741 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd9cc7d1f0 0x55bd9cc7d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.090+0000 7f37c9e00700 1 -- 172.21.15.1:0/1429817741 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bd9ca503b0 con 0x55bd9cc7db10 2024-02-02T15:40:53.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.090+0000 7f37c2a00700 1 -- 172.21.15.1:0/1937541763 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f37b4014070 con 0x7f37cc100550 2024-02-02T15:40:53.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.090+0000 7f37c9e00700 1 --2- 172.21.15.1:0/1429817741 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bd9cc7db10 0x55bd9cc884b0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f37bc009cb0 tx=0x7f37bc011440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5d3f6f106a682f5 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:53.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.090+0000 7f37ca800700 1 --2- 172.21.15.1:0/1937541763 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f37ac0a0710 0x7f37ac0a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:53.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.091+0000 7f37bb000700 1 -- 172.21.15.1:0/1429817741 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37bc011d20 con 0x55bd9cc7db10 2024-02-02T15:40:53.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.091+0000 7f37bb000700 1 -- 172.21.15.1:0/1429817741 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f37bc011e80 con 0x55bd9cc7db10 2024-02-02T15:40:53.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.091+0000 7f37bb000700 1 -- 172.21.15.1:0/1429817741 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37bc005280 con 0x55bd9cc7db10 2024-02-02T15:40:53.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.092+0000 7f37ca800700 1 --2- 172.21.15.1:0/1937541763 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f37ac0a0710 0x7f37ac0a2bd0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f37c4007fd0 tx=0x7f37c400b330 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:53.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.092+0000 7f37e464eb80 1 -- 172.21.15.1:0/1429817741 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bd9cc7db10 msgr2=0x55bd9cc884b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:53.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.092+0000 7f37e464eb80 1 --2- 172.21.15.1:0/1429817741 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bd9cc7db10 0x55bd9cc884b0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f37bc009cb0 tx=0x7f37bc011440 comp rx=0 tx=0).stop 2024-02-02T15:40:53.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.092+0000 7f37e464eb80 1 -- 172.21.15.1:0/1429817741 shutdown_connections 2024-02-02T15:40:53.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.092+0000 7f37e464eb80 1 --2- 172.21.15.1:0/1429817741 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bd9cc889f0 0x55bd9cc8ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.092+0000 7f37e464eb80 1 --2- 172.21.15.1:0/1429817741 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd9cc7d1f0 0x55bd9cc7d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.092+0000 7f37e464eb80 1 --2- 172.21.15.1:0/1429817741 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bd9cc7db10 0x55bd9cc884b0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.092+0000 7f37e464eb80 1 -- 172.21.15.1:0/1429817741 >> 172.21.15.1:0/1429817741 conn(0x55bd9cbdcdb0 msgr2=0x55bd9cc795c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:53.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.092+0000 7f37e464eb80 1 -- 172.21.15.1:0/1429817741 shutdown_connections 2024-02-02T15:40:53.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.092+0000 7f37e464eb80 1 -- 172.21.15.1:0/1429817741 wait complete. 2024-02-02T15:40:53.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.093+0000 7f37e464eb80 1 -- start start 2024-02-02T15:40:53.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.093+0000 7f37e464eb80 1 Processor -- start 2024-02-02T15:40:53.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.094+0000 7f37e464eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bd9cc7d1f0 0x55bd9cc88350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:53.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.094+0000 7f37e464eb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd9cc7db10 0x55bd9cc85640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:53.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.094+0000 7f37e464eb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bd9cc889f0 0x55bd9cc85c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:53.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.094+0000 7f37e464eb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd9cbced50 con 0x55bd9cc7db10 2024-02-02T15:40:53.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.094+0000 7f37e464eb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd9cb6ad80 con 0x55bd9cc889f0 2024-02-02T15:40:53.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.094+0000 7f37e464eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd9cb6aa20 con 0x55bd9cc7d1f0 2024-02-02T15:40:53.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.094+0000 7f37e464eb80 10 client.0 _ll_register_callbacks cb 0x55bd9cb6c290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:40:53.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.094+0000 7f37c9400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd9cc7db10 0x55bd9cc85640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:53.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.094+0000 7f37c9e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bd9cc7d1f0 0x55bd9cc88350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:53.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.094+0000 7f37c9e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bd9cc7d1f0 0x55bd9cc88350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:37418/0 (socket says 172.21.15.1:37418) 2024-02-02T15:40:53.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.094+0000 7f37c9400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd9cc7db10 0x55bd9cc85640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53580/0 (socket says 172.21.15.1:53580) 2024-02-02T15:40:53.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.094+0000 7f37c9e00700 1 -- 172.21.15.1:0/3439546401 learned_addr learned my addr 172.21.15.1:0/3439546401 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:40:53.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.094+0000 7f37ca800700 1 --2- 172.21.15.1:0/3439546401 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bd9cc889f0 0x55bd9cc85c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:53.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.095+0000 7f37c9400700 1 -- 172.21.15.1:0/3439546401 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bd9cc7d1f0 msgr2=0x55bd9cc88350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:53.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.095+0000 7f37c9400700 1 --2- 172.21.15.1:0/3439546401 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bd9cc7d1f0 0x55bd9cc88350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.095+0000 7f37c9400700 1 -- 172.21.15.1:0/3439546401 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bd9cc889f0 msgr2=0x55bd9cc85c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:53.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.095+0000 7f37c9400700 1 --2- 172.21.15.1:0/3439546401 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bd9cc889f0 0x55bd9cc85c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.095+0000 7f37c9400700 1 -- 172.21.15.1:0/3439546401 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37bc0098e0 con 0x55bd9cc7db10 2024-02-02T15:40:53.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.095+0000 7f37ca800700 1 --2- 172.21.15.1:0/3439546401 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bd9cc889f0 0x55bd9cc85c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:40:53.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.095+0000 7f37c9400700 1 --2- 172.21.15.1:0/3439546401 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd9cc7db10 0x55bd9cc85640 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f37b40c9810 tx=0x7f37b4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7ff8418f077c427 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:53.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.095+0000 7f37b3400700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:40:53.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.095+0000 7f37b3400700 1 -- 172.21.15.1:0/3439546401 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b40842d0 con 0x55bd9cc7db10 2024-02-02T15:40:53.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.095+0000 7f37b3400700 1 -- 172.21.15.1:0/3439546401 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f37b4099ce0 con 0x55bd9cc7db10 2024-02-02T15:40:53.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.095+0000 7f37e464eb80 1 -- 172.21.15.1:0/3439546401 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bd9cbc7dd0 con 0x55bd9cc7db10 2024-02-02T15:40:53.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.095+0000 7f37b3400700 1 -- 172.21.15.1:0/3439546401 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7f37b4098f40 con 0x55bd9cc7db10 2024-02-02T15:40:53.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.096+0000 7f37e464eb80 10 client.19164 fetch_fsmap learned FSMap version 59 2024-02-02T15:40:53.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.096+0000 7f37e464eb80 1 -- 172.21.15.1:0/3439546401 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) v3 -- 0x55bd9c9fcb60 con 0x55bd9cc7db10 2024-02-02T15:40:53.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.096+0000 7f37b3400700 1 -- 172.21.15.1:0/3439546401 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bd9cbc7dd0 con 0x55bd9cc7db10 2024-02-02T15:40:53.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.096+0000 7f37b3400700 1 -- 172.21.15.1:0/3439546401 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 59) v1 ==== 7538+0+0 (secure 0 0 0) 0x7f37b40cc170 con 0x55bd9cc7db10 2024-02-02T15:40:53.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.096+0000 7f37e464eb80 10 client.19164 fetch_fsmap finished waiting for FSMap version 59 2024-02-02T15:40:53.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.096+0000 7f37e464eb80 10 client.19164 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-02-02T15:40:53.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.096+0000 7f37e464eb80 20 client.19164 populate_metadata read hostname 'smithi001' 2024-02-02T15:40:53.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.096+0000 7f37e464eb80 1 --2- 172.21.15.1:0/3439546401 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55bd9cc984f0 0x55bd9cc9a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:53.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.096+0000 7f37e464eb80 4 client.19164 mds_command: new command op to 24502 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:40:53.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.096+0000 7f37e464eb80 1 -- 172.21.15.1:0/3439546401 --> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bd9cb8df50 con 0x55bd9cc984f0 2024-02-02T15:40:53.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.097+0000 7f37c9e00700 1 --2- 172.21.15.1:0/3439546401 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55bd9cc984f0 0x55bd9cc9a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:53.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.098+0000 7f37c9e00700 1 --2- 172.21.15.1:0/3439546401 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55bd9cc984f0 0x55bd9cc9a910 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=838161b2b866e788 server_cookie=a40e8616dad2fb79 in_seq=0 out_seq=0 2024-02-02T15:40:53.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.098+0000 7f37b3400700 10 client.19164 ms_handle_connect on v2:172.21.15.142:6834/1052606547 2024-02-02T15:40:53.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.098+0000 7f37b3400700 1 -- 172.21.15.1:0/3439546401 <== mds.4 v2:172.21.15.142:6834/1052606547 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55bd9cb8df50 con 0x55bd9cc984f0 2024-02-02T15:40:53.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.098+0000 7f37b3400700 10 client.19164 handle_command_reply: tid=0 2024-02-02T15:40:53.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.098+0000 7f37e464eb80 1 client.19164 shutdown 2024-02-02T15:40:53.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.098+0000 7f37e464eb80 1 -- 172.21.15.1:0/3439546401 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd9cc7db10 msgr2=0x55bd9cc85640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:53.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.098+0000 7f37e464eb80 1 --2- 172.21.15.1:0/3439546401 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd9cc7db10 0x55bd9cc85640 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f37b40c9810 tx=0x7f37b4013000 comp rx=0 tx=0).stop 2024-02-02T15:40:53.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.099+0000 7f37e464eb80 1 -- 172.21.15.1:0/3439546401 shutdown_connections 2024-02-02T15:40:53.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.099+0000 7f37e464eb80 1 --2- 172.21.15.1:0/3439546401 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55bd9cc984f0 0x55bd9cc9a910 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.099+0000 7f37e464eb80 1 --2- 172.21.15.1:0/3439546401 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bd9cc889f0 0x55bd9cc85c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.099+0000 7f37e464eb80 1 --2- 172.21.15.1:0/3439546401 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bd9cc7d1f0 0x55bd9cc88350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.099+0000 7f37e464eb80 1 --2- 172.21.15.1:0/3439546401 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd9cc7db10 0x55bd9cc85640 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.099+0000 7f37e464eb80 1 -- 172.21.15.1:0/3439546401 >> 172.21.15.1:0/3439546401 conn(0x55bd9cbdcdb0 msgr2=0x55bd9cc791b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:53.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.099+0000 7f37b3400700 0 client.19164 ms_handle_reset on v2:172.21.15.142:6834/1052606547 2024-02-02T15:40:53.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.099+0000 7f37e464eb80 1 -- 172.21.15.1:0/3439546401 shutdown_connections 2024-02-02T15:40:53.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.099+0000 7f37e464eb80 1 -- 172.21.15.1:0/3439546401 wait complete. 2024-02-02T15:40:53.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.099+0000 7f37e464eb80 20 client.19164 trim_cache size 0 max 16384 2024-02-02T15:40:53.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.115+0000 7f37e464eb80 1 Processor -- start 2024-02-02T15:40:53.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.115+0000 7f37e464eb80 1 -- start start 2024-02-02T15:40:53.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.116+0000 7f37e464eb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bd9cc7d1f0 0x55bd9cc93ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:53.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.116+0000 7f37e464eb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd9cc7db10 0x55bd9cc94020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:53.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.116+0000 7f37e464eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bd9cc889f0 0x55bd9cc80950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:53.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.116+0000 7f37e464eb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd9ca526a0 con 0x55bd9cc7db10 2024-02-02T15:40:53.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.116+0000 7f37e464eb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd9cc95a40 con 0x55bd9cc7d1f0 2024-02-02T15:40:53.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.116+0000 7f37e464eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd9cc83ba0 con 0x55bd9cc889f0 2024-02-02T15:40:53.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.116+0000 7f37c9e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bd9cc7d1f0 0x55bd9cc93ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:53.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.116+0000 7f37c9400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd9cc7db10 0x55bd9cc94020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:53.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.116+0000 7f37c9e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bd9cc7d1f0 0x55bd9cc93ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:41634/0 (socket says 172.21.15.1:41634) 2024-02-02T15:40:53.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.116+0000 7f37c9e00700 1 -- 172.21.15.1:0/1095110722 learned_addr learned my addr 172.21.15.1:0/1095110722 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:40:53.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.116+0000 7f37ca800700 1 --2- 172.21.15.1:0/1095110722 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bd9cc889f0 0x55bd9cc80950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:53.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.116+0000 7f37c9400700 1 -- 172.21.15.1:0/1095110722 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bd9cc889f0 msgr2=0x55bd9cc80950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:53.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.116+0000 7f37c9400700 1 --2- 172.21.15.1:0/1095110722 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bd9cc889f0 0x55bd9cc80950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.116+0000 7f37c9400700 1 -- 172.21.15.1:0/1095110722 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bd9cc7d1f0 msgr2=0x55bd9cc93ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:53.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.116+0000 7f37c9400700 1 --2- 172.21.15.1:0/1095110722 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bd9cc7d1f0 0x55bd9cc93ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.116+0000 7f37c9400700 1 -- 172.21.15.1:0/1095110722 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bd9cb94ea0 con 0x55bd9cc7db10 2024-02-02T15:40:53.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.116+0000 7f37c9e00700 1 --2- 172.21.15.1:0/1095110722 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bd9cc7d1f0 0x55bd9cc93ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:40:53.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.117+0000 7f37c9400700 1 --2- 172.21.15.1:0/1095110722 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd9cc7db10 0x55bd9cc94020 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f37b4096000 tx=0x7f37b4099490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=926bfffe17395b28 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:53.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.117+0000 7f37bba00700 1 -- 172.21.15.1:0/1095110722 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b40842d0 con 0x55bd9cc7db10 2024-02-02T15:40:53.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.117+0000 7f37bba00700 1 -- 172.21.15.1:0/1095110722 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f37b4098f40 con 0x55bd9cc7db10 2024-02-02T15:40:53.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.117+0000 7f37bba00700 1 -- 172.21.15.1:0/1095110722 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b400b2c0 con 0x55bd9cc7db10 2024-02-02T15:40:53.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.118+0000 7f37e464eb80 1 -- 172.21.15.1:0/1095110722 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd9cc7db10 msgr2=0x55bd9cc94020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:53.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.118+0000 7f37e464eb80 1 --2- 172.21.15.1:0/1095110722 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd9cc7db10 0x55bd9cc94020 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f37b4096000 tx=0x7f37b4099490 comp rx=0 tx=0).stop 2024-02-02T15:40:53.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.118+0000 7f37e464eb80 1 -- 172.21.15.1:0/1095110722 shutdown_connections 2024-02-02T15:40:53.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.118+0000 7f37e464eb80 1 --2- 172.21.15.1:0/1095110722 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bd9cc889f0 0x55bd9cc80950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.118+0000 7f37e464eb80 1 --2- 172.21.15.1:0/1095110722 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bd9cc7d1f0 0x55bd9cc93ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.118+0000 7f37e464eb80 1 --2- 172.21.15.1:0/1095110722 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd9cc7db10 0x55bd9cc94020 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.118+0000 7f37e464eb80 1 -- 172.21.15.1:0/1095110722 >> 172.21.15.1:0/1095110722 conn(0x55bd9cbdcdb0 msgr2=0x55bd9cc9a500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:53.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.118+0000 7f37e464eb80 1 -- 172.21.15.1:0/1095110722 shutdown_connections 2024-02-02T15:40:53.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.118+0000 7f37e464eb80 1 -- 172.21.15.1:0/1095110722 wait complete. 2024-02-02T15:40:53.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.118+0000 7f37e464eb80 1 -- start start 2024-02-02T15:40:53.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.119+0000 7f37e464eb80 1 Processor -- start 2024-02-02T15:40:53.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.119+0000 7f37e464eb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bd9cc7d1f0 0x55bd9cc8f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:53.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.119+0000 7f37e464eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bd9cc7db10 0x55bd9cc8c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:53.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.119+0000 7f37e464eb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd9cc889f0 0x55bd9cc8ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:53.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.119+0000 7f37e464eb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd9cc8cf90 con 0x55bd9cc889f0 2024-02-02T15:40:53.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.119+0000 7f37e464eb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd9cc8d0d0 con 0x55bd9cc7d1f0 2024-02-02T15:40:53.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.119+0000 7f37e464eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd9cc8d290 con 0x55bd9cc7db10 2024-02-02T15:40:53.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.120+0000 7f37e464eb80 10 client.0 _ll_register_callbacks cb 0x55bd9cb6c290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:40:53.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.120+0000 7f37c9e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bd9cc7d1f0 0x55bd9cc8f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:53.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.120+0000 7f37c9e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bd9cc7d1f0 0x55bd9cc8f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:41638/0 (socket says 172.21.15.1:41638) 2024-02-02T15:40:53.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.120+0000 7f37c9e00700 1 -- 172.21.15.1:0/912325830 learned_addr learned my addr 172.21.15.1:0/912325830 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:40:53.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.120+0000 7f37c9400700 1 --2- 172.21.15.1:0/912325830 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bd9cc7db10 0x55bd9cc8c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:53.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.120+0000 7f37ca800700 1 --2- 172.21.15.1:0/912325830 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd9cc889f0 0x55bd9cc8ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:53.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.120+0000 7f37c9e00700 1 -- 172.21.15.1:0/912325830 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bd9cc7db10 msgr2=0x55bd9cc8c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:53.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.120+0000 7f37c9e00700 1 --2- 172.21.15.1:0/912325830 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bd9cc7db10 0x55bd9cc8c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.120+0000 7f37c9e00700 1 -- 172.21.15.1:0/912325830 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd9cc889f0 msgr2=0x55bd9cc8ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:53.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.120+0000 7f37c9e00700 1 --2- 172.21.15.1:0/912325830 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd9cc889f0 0x55bd9cc8ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.120+0000 7f37c9e00700 1 -- 172.21.15.1:0/912325830 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37bc01d040 con 0x55bd9cc7d1f0 2024-02-02T15:40:53.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.120+0000 7f37c9e00700 1 --2- 172.21.15.1:0/912325830 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bd9cc7d1f0 0x55bd9cc8f1f0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f37bc011810 tx=0x7f37bc005b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14e8d279af019c5e server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:53.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.120+0000 7f37b3400700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:40:53.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.121+0000 7f37b3400700 1 -- 172.21.15.1:0/912325830 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37bc004870 con 0x55bd9cc7d1f0 2024-02-02T15:40:53.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.121+0000 7f37e464eb80 1 -- 172.21.15.1:0/912325830 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bd9cc71340 con 0x55bd9cc7d1f0 2024-02-02T15:40:53.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.121+0000 7f37b3400700 1 -- 172.21.15.1:0/912325830 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f37bc029070 con 0x55bd9cc7d1f0 2024-02-02T15:40:53.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.121+0000 7f37b3400700 1 -- 172.21.15.1:0/912325830 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7f37bc013cc0 con 0x55bd9cc7d1f0 2024-02-02T15:40:53.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.121+0000 7f37e464eb80 10 client.29276 fetch_fsmap learned FSMap version 59 2024-02-02T15:40:53.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.121+0000 7f37e464eb80 1 -- 172.21.15.1:0/912325830 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) v3 -- 0x55bd9c9fcb60 con 0x55bd9cc7d1f0 2024-02-02T15:40:53.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.121+0000 7f37b3400700 1 -- 172.21.15.1:0/912325830 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55bd9cc71340 con 0x55bd9cc7d1f0 2024-02-02T15:40:53.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.122+0000 7f37b3400700 1 -- 172.21.15.1:0/912325830 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 59) v1 ==== 7538+0+0 (secure 0 0 0) 0x7f37bc005250 con 0x55bd9cc7d1f0 2024-02-02T15:40:53.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.122+0000 7f37e464eb80 10 client.29276 fetch_fsmap finished waiting for FSMap version 59 2024-02-02T15:40:53.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.122+0000 7f37e464eb80 10 client.29276 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-02-02T15:40:53.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.122+0000 7f37e464eb80 20 client.29276 populate_metadata read hostname 'smithi001' 2024-02-02T15:40:53.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.122+0000 7f37e464eb80 1 --2- 172.21.15.1:0/912325830 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55bd9ccfb4c0 0x55bd9ccfd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:53.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.122+0000 7f37e464eb80 4 client.29276 mds_command: new command op to 24502 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:40:53.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.122+0000 7f37e464eb80 1 -- 172.21.15.1:0/912325830 --> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55bd9ca1dcf0 con 0x55bd9ccfb4c0 2024-02-02T15:40:53.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.122+0000 7f37c9400700 1 --2- 172.21.15.1:0/912325830 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55bd9ccfb4c0 0x55bd9ccfd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:53.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.123+0000 7f37c9400700 1 --2- 172.21.15.1:0/912325830 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55bd9ccfb4c0 0x55bd9ccfd8e0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=38e61fa334b68d6d server_cookie=5b514955fe079920 in_seq=0 out_seq=0 2024-02-02T15:40:53.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.123+0000 7f37b3400700 10 client.29276 ms_handle_connect on v2:172.21.15.142:6834/1052606547 2024-02-02T15:40:53.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.123+0000 7f37b3400700 1 -- 172.21.15.1:0/912325830 <== mds.4 v2:172.21.15.142:6834/1052606547 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55bd9ca1dcf0 con 0x55bd9ccfb4c0 2024-02-02T15:40:53.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.123+0000 7f37b3400700 10 client.29276 handle_command_reply: tid=0 2024-02-02T15:40:53.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.123+0000 7f37e464eb80 1 client.29276 shutdown 2024-02-02T15:40:53.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.123+0000 7f37e464eb80 1 -- 172.21.15.1:0/912325830 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bd9cc7d1f0 msgr2=0x55bd9cc8f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:53.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.123+0000 7f37e464eb80 1 --2- 172.21.15.1:0/912325830 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bd9cc7d1f0 0x55bd9cc8f1f0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f37bc011810 tx=0x7f37bc005b40 comp rx=0 tx=0).stop 2024-02-02T15:40:53.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.123+0000 7f37e464eb80 1 -- 172.21.15.1:0/912325830 shutdown_connections 2024-02-02T15:40:53.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.123+0000 7f37e464eb80 1 --2- 172.21.15.1:0/912325830 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55bd9ccfb4c0 0x55bd9ccfd8e0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.123+0000 7f37e464eb80 1 --2- 172.21.15.1:0/912325830 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55bd9cc889f0 0x55bd9cc8ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.123+0000 7f37e464eb80 1 --2- 172.21.15.1:0/912325830 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55bd9cc7d1f0 0x55bd9cc8f1f0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.123+0000 7f37e464eb80 1 --2- 172.21.15.1:0/912325830 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55bd9cc7db10 0x55bd9cc8c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.123+0000 7f37b3400700 0 client.29276 ms_handle_reset on v2:172.21.15.142:6834/1052606547 2024-02-02T15:40:53.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.123+0000 7f37e464eb80 1 -- 172.21.15.1:0/912325830 >> 172.21.15.1:0/912325830 conn(0x55bd9cbdcdb0 msgr2=0x55bd9cc93ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:53.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.123+0000 7f37e464eb80 1 -- 172.21.15.1:0/912325830 shutdown_connections 2024-02-02T15:40:53.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.123+0000 7f37e464eb80 1 -- 172.21.15.1:0/912325830 wait complete. 2024-02-02T15:40:53.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.124+0000 7f37e464eb80 20 client.29276 trim_cache size 0 max 16384 2024-02-02T15:40:53.127 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:40:53.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.124+0000 7f37cb200700 1 -- 172.21.15.1:0/1937541763 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f37ac0a0710 msgr2=0x7f37ac0a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:53.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.124+0000 7f37cb200700 1 --2- 172.21.15.1:0/1937541763 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f37ac0a0710 0x7f37ac0a2bd0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f37c4007fd0 tx=0x7f37c400b330 comp rx=0 tx=0).stop 2024-02-02T15:40:53.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.124+0000 7f37cb200700 1 -- 172.21.15.1:0/1937541763 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37cc100550 msgr2=0x7f37cc1993d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:53.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.124+0000 7f37cb200700 1 --2- 172.21.15.1:0/1937541763 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37cc100550 0x7f37cc1993d0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f37b400ec10 tx=0x7f37b400ef20 comp rx=0 tx=0).stop 2024-02-02T15:40:53.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.124+0000 7f37cb200700 1 -- 172.21.15.1:0/1937541763 shutdown_connections 2024-02-02T15:40:53.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.124+0000 7f37cb200700 1 --2- 172.21.15.1:0/1937541763 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f37ac0a0710 0x7f37ac0a2bd0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.124+0000 7f37cb200700 1 --2- 172.21.15.1:0/1937541763 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f37cc104980 0x7f37cc19d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.124+0000 7f37cb200700 1 --2- 172.21.15.1:0/1937541763 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f37cc0ffb90 0x7f37cc198e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.124+0000 7f37cb200700 1 --2- 172.21.15.1:0/1937541763 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f37cc100550 0x7f37cc1993d0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.124+0000 7f37cb200700 1 -- 172.21.15.1:0/1937541763 >> 172.21.15.1:0/1937541763 conn(0x7f37cc076770 msgr2=0x7f37cc10ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:53.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.125+0000 7f37cb200700 1 -- 172.21.15.1:0/1937541763 shutdown_connections 2024-02-02T15:40:53.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.125+0000 7f37cb200700 1 -- 172.21.15.1:0/1937541763 wait complete. 2024-02-02T15:40:53.147 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-02-02T15:40:53.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.402+0000 7fa04da00700 1 -- 172.21.15.1:0/430522752 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa04810d640 msgr2=0x7fa04810fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:53.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.402+0000 7fa04da00700 1 --2- 172.21.15.1:0/430522752 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa04810d640 0x7fa04810fa70 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fa04000b0a0 tx=0x7fa04000b3b0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.402+0000 7fa04da00700 1 -- 172.21.15.1:0/430522752 shutdown_connections 2024-02-02T15:40:53.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.402+0000 7fa04da00700 1 --2- 172.21.15.1:0/430522752 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa04810d640 0x7fa04810fa70 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.402+0000 7fa04da00700 1 --2- 172.21.15.1:0/430522752 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa048073110 0x7fa0480734f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.402+0000 7fa04da00700 1 --2- 172.21.15.1:0/430522752 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa048073a30 0x7fa04810d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.402+0000 7fa04da00700 1 -- 172.21.15.1:0/430522752 >> 172.21.15.1:0/430522752 conn(0x7fa0480fc8b0 msgr2=0x7fa0480fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:53.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.402+0000 7fa04da00700 1 -- 172.21.15.1:0/430522752 shutdown_connections 2024-02-02T15:40:53.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.402+0000 7fa04da00700 1 -- 172.21.15.1:0/430522752 wait complete. 2024-02-02T15:40:53.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.403+0000 7fa04da00700 1 Processor -- start 2024-02-02T15:40:53.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.403+0000 7fa04da00700 1 -- start start 2024-02-02T15:40:53.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.404+0000 7fa04da00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa048073110 0x7fa0481991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:53.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.404+0000 7fa04da00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa048073a30 0x7fa048199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:53.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.404+0000 7fa04da00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa04810d640 0x7fa04819db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:53.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.404+0000 7fa04da00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa04819e1b0 con 0x7fa048073a30 2024-02-02T15:40:53.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.404+0000 7fa04da00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa04819e320 con 0x7fa04810d640 2024-02-02T15:40:53.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.404+0000 7fa04da00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa04819e4e0 con 0x7fa048073110 2024-02-02T15:40:53.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.404+0000 7fa046a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa048073110 0x7fa0481991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:53.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.404+0000 7fa046a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa048073110 0x7fa0481991d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:37444/0 (socket says 172.21.15.1:37444) 2024-02-02T15:40:53.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.404+0000 7fa046a00700 1 -- 172.21.15.1:0/501724908 learned_addr learned my addr 172.21.15.1:0/501724908 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:40:53.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.404+0000 7fa046000700 1 --2- 172.21.15.1:0/501724908 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa048073a30 0x7fa048199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:53.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.404+0000 7fa047400700 1 --2- 172.21.15.1:0/501724908 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa04810d640 0x7fa04819db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:53.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.404+0000 7fa046000700 1 -- 172.21.15.1:0/501724908 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa048073110 msgr2=0x7fa0481991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:53.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.404+0000 7fa046000700 1 --2- 172.21.15.1:0/501724908 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa048073110 0x7fa0481991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.405+0000 7fa046000700 1 -- 172.21.15.1:0/501724908 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa04810d640 msgr2=0x7fa04819db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:53.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.405+0000 7fa046000700 1 --2- 172.21.15.1:0/501724908 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa04810d640 0x7fa04819db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.405+0000 7fa046000700 1 -- 172.21.15.1:0/501724908 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa040009d00 con 0x7fa048073a30 2024-02-02T15:40:53.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.405+0000 7fa047400700 1 --2- 172.21.15.1:0/501724908 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa04810d640 0x7fa04819db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:40:53.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.405+0000 7fa046a00700 1 --2- 172.21.15.1:0/501724908 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa048073110 0x7fa0481991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:40:53.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.405+0000 7fa046000700 1 --2- 172.21.15.1:0/501724908 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa048073a30 0x7fa048199710 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fa03000d750 tx=0x7fa03000da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:53.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.405+0000 7fa03f400700 1 -- 172.21.15.1:0/501724908 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa030004020 con 0x7fa048073a30 2024-02-02T15:40:53.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.405+0000 7fa03f400700 1 -- 172.21.15.1:0/501724908 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa030004830 con 0x7fa048073a30 2024-02-02T15:40:53.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.405+0000 7fa03f400700 1 -- 172.21.15.1:0/501724908 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0300054c0 con 0x7fa048073a30 2024-02-02T15:40:53.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.405+0000 7fa04da00700 1 -- 172.21.15.1:0/501724908 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa04819e7c0 con 0x7fa048073a30 2024-02-02T15:40:53.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.406+0000 7fa04da00700 1 -- 172.21.15.1:0/501724908 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0481a6000 con 0x7fa048073a30 2024-02-02T15:40:53.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.410+0000 7fa03f400700 1 -- 172.21.15.1:0/501724908 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7fa0300049a0 con 0x7fa048073a30 2024-02-02T15:40:53.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.411+0000 7fa03f400700 1 --2- 172.21.15.1:0/501724908 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fa0280a0790 0x7fa0280a2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:53.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.411+0000 7fa03f400700 1 -- 172.21.15.1:0/501724908 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fa0300cb970 con 0x7fa048073a30 2024-02-02T15:40:53.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.411+0000 7fa046a00700 1 --2- 172.21.15.1:0/501724908 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fa0280a0790 0x7fa0280a2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:53.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.412+0000 7fa0614adb80 1 Processor -- start 2024-02-02T15:40:53.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.412+0000 7fa0614adb80 1 -- start start 2024-02-02T15:40:53.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.413+0000 7fa0614adb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d64d91820 0x560d64d91c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:53.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.413+0000 7fa0614adb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560d64d92140 0x560d64d9cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:53.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.413+0000 7fa0614adb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d64d9cff0 0x560d64d9f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:53.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.413+0000 7fa0614adb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d64cfd4c0 con 0x560d64d91820 2024-02-02T15:40:53.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.413+0000 7fa0614adb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d64ce1aa0 con 0x560d64d92140 2024-02-02T15:40:53.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.413+0000 7fa0614adb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d64cdf460 con 0x560d64d9cff0 2024-02-02T15:40:53.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.413+0000 7fa046000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560d64d92140 0x560d64d9cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:53.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.413+0000 7fa046000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560d64d92140 0x560d64d9cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:41664/0 (socket says 172.21.15.1:41664) 2024-02-02T15:40:53.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.413+0000 7fa046000700 1 -- 172.21.15.1:0/4208390321 learned_addr learned my addr 172.21.15.1:0/4208390321 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:40:53.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.413+0000 7fa046000700 1 -- 172.21.15.1:0/4208390321 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d64d9cff0 msgr2=0x560d64d9f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:53.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.413+0000 7fa046000700 1 --2- 172.21.15.1:0/4208390321 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d64d9cff0 0x560d64d9f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.413+0000 7fa046000700 1 -- 172.21.15.1:0/4208390321 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d64d91820 msgr2=0x560d64d91c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:53.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.413+0000 7fa046000700 1 --2- 172.21.15.1:0/4208390321 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d64d91820 0x560d64d91c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.413+0000 7fa046000700 1 -- 172.21.15.1:0/4208390321 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560d64b5c3b0 con 0x560d64d92140 2024-02-02T15:40:53.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.414+0000 7fa046a00700 1 --2- 172.21.15.1:0/501724908 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fa0280a0790 0x7fa0280a2c50 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa038011550 tx=0x7fa038009260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:53.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.414+0000 7fa046000700 1 --2- 172.21.15.1:0/4208390321 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560d64d92140 0x560d64d9cab0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fa030098790 tx=0x7fa030098700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97f3c043a7dac711 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:53.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.414+0000 7fa037e00700 1 -- 172.21.15.1:0/4208390321 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa03009c8c0 con 0x560d64d92140 2024-02-02T15:40:53.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.414+0000 7fa037e00700 1 -- 172.21.15.1:0/4208390321 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0300a0b00 con 0x560d64d92140 2024-02-02T15:40:53.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.414+0000 7fa037e00700 1 -- 172.21.15.1:0/4208390321 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa030098f30 con 0x560d64d92140 2024-02-02T15:40:53.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.415+0000 7fa0614adb80 1 -- 172.21.15.1:0/4208390321 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560d64d92140 msgr2=0x560d64d9cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:53.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.415+0000 7fa0614adb80 1 --2- 172.21.15.1:0/4208390321 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560d64d92140 0x560d64d9cab0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fa030098790 tx=0x7fa030098700 comp rx=0 tx=0).stop 2024-02-02T15:40:53.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.415+0000 7fa0614adb80 1 -- 172.21.15.1:0/4208390321 shutdown_connections 2024-02-02T15:40:53.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.415+0000 7fa0614adb80 1 --2- 172.21.15.1:0/4208390321 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d64d9cff0 0x560d64d9f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.415+0000 7fa0614adb80 1 --2- 172.21.15.1:0/4208390321 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d64d91820 0x560d64d91c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.415+0000 7fa0614adb80 1 --2- 172.21.15.1:0/4208390321 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560d64d92140 0x560d64d9cab0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.415+0000 7fa0614adb80 1 -- 172.21.15.1:0/4208390321 >> 172.21.15.1:0/4208390321 conn(0x560d64ce8db0 msgr2=0x560d64d853b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:53.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.415+0000 7fa0614adb80 1 -- 172.21.15.1:0/4208390321 shutdown_connections 2024-02-02T15:40:53.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.415+0000 7fa0614adb80 1 -- 172.21.15.1:0/4208390321 wait complete. 2024-02-02T15:40:53.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.416+0000 7fa0614adb80 1 -- start start 2024-02-02T15:40:53.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.416+0000 7fa0614adb80 1 Processor -- start 2024-02-02T15:40:53.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.417+0000 7fa0614adb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d64d91820 0x560d64d99db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:53.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.417+0000 7fa0614adb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d64d92140 0x560d64d9a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:53.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.417+0000 7fa0614adb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560d64d9cff0 0x560d64d9a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:53.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.417+0000 7fa0614adb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d64cdad50 con 0x560d64d92140 2024-02-02T15:40:53.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.417+0000 7fa0614adb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d64c76d80 con 0x560d64d9cff0 2024-02-02T15:40:53.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.417+0000 7fa0614adb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d64c76a20 con 0x560d64d91820 2024-02-02T15:40:53.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.417+0000 7fa0614adb80 10 client.0 _ll_register_callbacks cb 0x560d64c78290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:40:53.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.417+0000 7fa046a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d64d91820 0x560d64d99db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:53.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.417+0000 7fa046a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d64d91820 0x560d64d99db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:37464/0 (socket says 172.21.15.1:37464) 2024-02-02T15:40:53.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.417+0000 7fa046a00700 1 -- 172.21.15.1:0/2480017942 learned_addr learned my addr 172.21.15.1:0/2480017942 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:40:53.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.417+0000 7fa047400700 1 --2- 172.21.15.1:0/2480017942 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560d64d9cff0 0x560d64d9a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:53.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.418+0000 7fa046000700 1 --2- 172.21.15.1:0/2480017942 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d64d92140 0x560d64d9a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:53.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.418+0000 7fa046a00700 1 -- 172.21.15.1:0/2480017942 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560d64d9cff0 msgr2=0x560d64d9a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:53.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.418+0000 7fa046a00700 1 --2- 172.21.15.1:0/2480017942 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560d64d9cff0 0x560d64d9a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.418+0000 7fa046a00700 1 -- 172.21.15.1:0/2480017942 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d64d92140 msgr2=0x560d64d9a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:53.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.418+0000 7fa046a00700 1 --2- 172.21.15.1:0/2480017942 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d64d92140 0x560d64d9a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.418+0000 7fa046a00700 1 -- 172.21.15.1:0/2480017942 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0481a6000 con 0x560d64d91820 2024-02-02T15:40:53.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.418+0000 7fa046000700 1 --2- 172.21.15.1:0/2480017942 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d64d92140 0x560d64d9a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:40:53.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.418+0000 7fa046a00700 1 --2- 172.21.15.1:0/2480017942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d64d91820 0x560d64d99db0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fa03800ed60 tx=0x7fa038004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30cfae77e2508c0e server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:53.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.418+0000 7fa034c00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:40:53.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.419+0000 7fa034c00700 1 -- 172.21.15.1:0/2480017942 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa03801cd40 con 0x560d64d91820 2024-02-02T15:40:53.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.419+0000 7fa034c00700 1 -- 172.21.15.1:0/2480017942 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fa038023070 con 0x560d64d91820 2024-02-02T15:40:53.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.419+0000 7fa0614adb80 1 -- 172.21.15.1:0/2480017942 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560d64d7d0f0 con 0x560d64d91820 2024-02-02T15:40:53.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.419+0000 7fa034c00700 1 -- 172.21.15.1:0/2480017942 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa03801cea0 con 0x560d64d91820 2024-02-02T15:40:53.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.419+0000 7fa0614adb80 10 client.29281 fetch_fsmap learned FSMap version 59 2024-02-02T15:40:53.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.419+0000 7fa0614adb80 1 -- 172.21.15.1:0/2480017942 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) v3 -- 0x560d64b08b60 con 0x560d64d91820 2024-02-02T15:40:53.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.421+0000 7fa034c00700 1 -- 172.21.15.1:0/2480017942 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560d64d7d0f0 con 0x560d64d91820 2024-02-02T15:40:53.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.421+0000 7fa034c00700 1 -- 172.21.15.1:0/2480017942 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 59) v1 ==== 7538+0+0 (secure 0 0 0) 0x7fa03801d760 con 0x560d64d91820 2024-02-02T15:40:53.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.421+0000 7fa0614adb80 10 client.29281 fetch_fsmap finished waiting for FSMap version 59 2024-02-02T15:40:53.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.421+0000 7fa0614adb80 10 client.29281 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-02-02T15:40:53.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.421+0000 7fa0614adb80 20 client.29281 populate_metadata read hostname 'smithi001' 2024-02-02T15:40:53.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.421+0000 7fa0614adb80 1 --2- 172.21.15.1:0/2480017942 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x560d64dacbc0 0x560d64daefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:53.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.421+0000 7fa0614adb80 4 client.29281 mds_command: new command op to 24514 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:40:53.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.421+0000 7fa0614adb80 1 -- 172.21.15.1:0/2480017942 --> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560d64cd3dd0 con 0x560d64dacbc0 2024-02-02T15:40:53.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.422+0000 7fa046000700 1 --2- 172.21.15.1:0/2480017942 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x560d64dacbc0 0x560d64daefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:53.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.423+0000 7fa046000700 1 --2- 172.21.15.1:0/2480017942 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x560d64dacbc0 0x560d64daefe0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=48bb627d699201fb server_cookie=c6b2b0475fdea5d3 in_seq=0 out_seq=0 2024-02-02T15:40:53.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.423+0000 7fa034c00700 10 client.29281 ms_handle_connect on v2:172.21.15.142:6836/2870289019 2024-02-02T15:40:53.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.426+0000 7fa034c00700 1 -- 172.21.15.1:0/2480017942 <== mds.3 v2:172.21.15.142:6836/2870289019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x560d64cd3dd0 con 0x560d64dacbc0 2024-02-02T15:40:53.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.426+0000 7fa034c00700 10 client.29281 handle_command_reply: tid=0 2024-02-02T15:40:53.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.426+0000 7fa0614adb80 1 client.29281 shutdown 2024-02-02T15:40:53.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.426+0000 7fa0614adb80 1 -- 172.21.15.1:0/2480017942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d64d91820 msgr2=0x560d64d99db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:53.426 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.426+0000 7fa0614adb80 1 --2- 172.21.15.1:0/2480017942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d64d91820 0x560d64d99db0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fa03800ed60 tx=0x7fa038004750 comp rx=0 tx=0).stop 2024-02-02T15:40:53.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.427+0000 7fa0614adb80 1 -- 172.21.15.1:0/2480017942 shutdown_connections 2024-02-02T15:40:53.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.427+0000 7fa0614adb80 1 --2- 172.21.15.1:0/2480017942 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560d64d9cff0 0x560d64d9a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.427+0000 7fa0614adb80 1 --2- 172.21.15.1:0/2480017942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d64d91820 0x560d64d99db0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.427+0000 7fa0614adb80 1 --2- 172.21.15.1:0/2480017942 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x560d64dacbc0 0x560d64daefe0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.427+0000 7fa0614adb80 1 --2- 172.21.15.1:0/2480017942 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d64d92140 0x560d64d9a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.427+0000 7fa034c00700 0 client.29281 ms_handle_reset on v2:172.21.15.142:6836/2870289019 2024-02-02T15:40:53.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.427+0000 7fa0614adb80 1 -- 172.21.15.1:0/2480017942 >> 172.21.15.1:0/2480017942 conn(0x560d64ce8db0 msgr2=0x560d64d84fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:53.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.427+0000 7fa0614adb80 1 -- 172.21.15.1:0/2480017942 shutdown_connections 2024-02-02T15:40:53.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.427+0000 7fa0614adb80 1 -- 172.21.15.1:0/2480017942 wait complete. 2024-02-02T15:40:53.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.427+0000 7fa0614adb80 20 client.29281 trim_cache size 0 max 16384 2024-02-02T15:40:53.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.448+0000 7fa0614adb80 1 Processor -- start 2024-02-02T15:40:53.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.448+0000 7fa0614adb80 1 -- start start 2024-02-02T15:40:53.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.448+0000 7fa0614adb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560d64d91820 0x560d64da3b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:53.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.448+0000 7fa0614adb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d64d92140 0x560d64d84eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:53.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.448+0000 7fa0614adb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d64d9cff0 0x560d64d9a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:53.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.448+0000 7fa0614adb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d64b5e6a0 con 0x560d64d92140 2024-02-02T15:40:53.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.448+0000 7fa0614adb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d64d869b0 con 0x560d64d91820 2024-02-02T15:40:53.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.448+0000 7fa0614adb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d64d98430 con 0x560d64d9cff0 2024-02-02T15:40:53.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.449+0000 7fa046000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d64d92140 0x560d64d84eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:53.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.449+0000 7fa046000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d64d92140 0x560d64d84eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53646/0 (socket says 172.21.15.1:53646) 2024-02-02T15:40:53.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.449+0000 7fa046000700 1 -- 172.21.15.1:0/3823150966 learned_addr learned my addr 172.21.15.1:0/3823150966 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:40:53.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.449+0000 7fa047400700 1 --2- 172.21.15.1:0/3823150966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d64d9cff0 0x560d64d9a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:53.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.449+0000 7fa046a00700 1 --2- 172.21.15.1:0/3823150966 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560d64d91820 0x560d64da3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:53.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.449+0000 7fa046000700 1 -- 172.21.15.1:0/3823150966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d64d9cff0 msgr2=0x560d64d9a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:53.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.449+0000 7fa046000700 1 --2- 172.21.15.1:0/3823150966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d64d9cff0 0x560d64d9a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.449+0000 7fa046000700 1 -- 172.21.15.1:0/3823150966 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560d64d91820 msgr2=0x560d64da3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:53.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.449+0000 7fa046000700 1 --2- 172.21.15.1:0/3823150966 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560d64d91820 0x560d64da3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.449+0000 7fa046000700 1 -- 172.21.15.1:0/3823150966 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560d64ca0ea0 con 0x560d64d92140 2024-02-02T15:40:53.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.450+0000 7fa046000700 1 --2- 172.21.15.1:0/3823150966 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d64d92140 0x560d64d84eb0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fa030098a20 tx=0x7fa0300cb040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=352f69402290e65a server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:53.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.450+0000 7fa03cc00700 1 -- 172.21.15.1:0/3823150966 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0300b6420 con 0x560d64d92140 2024-02-02T15:40:53.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.450+0000 7fa03cc00700 1 -- 172.21.15.1:0/3823150966 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa030005620 con 0x560d64d92140 2024-02-02T15:40:53.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.450+0000 7fa03cc00700 1 -- 172.21.15.1:0/3823150966 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa030004df0 con 0x560d64d92140 2024-02-02T15:40:53.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.452+0000 7fa0614adb80 1 -- 172.21.15.1:0/3823150966 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d64d92140 msgr2=0x560d64d84eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:53.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.452+0000 7fa0614adb80 1 --2- 172.21.15.1:0/3823150966 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d64d92140 0x560d64d84eb0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fa030098a20 tx=0x7fa0300cb040 comp rx=0 tx=0).stop 2024-02-02T15:40:53.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.452+0000 7fa0614adb80 1 -- 172.21.15.1:0/3823150966 shutdown_connections 2024-02-02T15:40:53.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.452+0000 7fa0614adb80 1 --2- 172.21.15.1:0/3823150966 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d64d9cff0 0x560d64d9a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.452+0000 7fa0614adb80 1 --2- 172.21.15.1:0/3823150966 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560d64d91820 0x560d64da3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.452+0000 7fa0614adb80 1 --2- 172.21.15.1:0/3823150966 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d64d92140 0x560d64d84eb0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.452+0000 7fa0614adb80 1 -- 172.21.15.1:0/3823150966 >> 172.21.15.1:0/3823150966 conn(0x560d64ce8db0 msgr2=0x560d64d83b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:53.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.452+0000 7fa0614adb80 1 -- 172.21.15.1:0/3823150966 shutdown_connections 2024-02-02T15:40:53.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.452+0000 7fa0614adb80 1 -- 172.21.15.1:0/3823150966 wait complete. 2024-02-02T15:40:53.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.453+0000 7fa0614adb80 1 -- start start 2024-02-02T15:40:53.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.453+0000 7fa0614adb80 1 Processor -- start 2024-02-02T15:40:53.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.453+0000 7fa0614adb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d64d91820 0x560d64df2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:53.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.454+0000 7fa0614adb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560d64d92140 0x560d64df2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:53.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.454+0000 7fa0614adb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d64d9cff0 0x560d64dfd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:53.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.454+0000 7fa0614adb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d64df33b0 con 0x560d64d91820 2024-02-02T15:40:53.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.454+0000 7fa0614adb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d64df3520 con 0x560d64d92140 2024-02-02T15:40:53.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.454+0000 7fa0614adb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d64dfdcb0 con 0x560d64d9cff0 2024-02-02T15:40:53.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.454+0000 7fa0614adb80 10 client.0 _ll_register_callbacks cb 0x560d64c78290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:40:53.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.454+0000 7fa046a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d64d91820 0x560d64df2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:53.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.454+0000 7fa046000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560d64d92140 0x560d64df2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:53.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.454+0000 7fa046a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d64d91820 0x560d64df2670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53650/0 (socket says 172.21.15.1:53650) 2024-02-02T15:40:53.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.454+0000 7fa046a00700 1 -- 172.21.15.1:0/1806485360 learned_addr learned my addr 172.21.15.1:0/1806485360 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:40:53.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.454+0000 7fa047400700 1 --2- 172.21.15.1:0/1806485360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d64d9cff0 0x560d64dfd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:53.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.454+0000 7fa046000700 1 -- 172.21.15.1:0/1806485360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d64d9cff0 msgr2=0x560d64dfd6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:53.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.454+0000 7fa046000700 1 --2- 172.21.15.1:0/1806485360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d64d9cff0 0x560d64dfd6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.454+0000 7fa046000700 1 -- 172.21.15.1:0/1806485360 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d64d91820 msgr2=0x560d64df2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:53.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.454+0000 7fa046000700 1 --2- 172.21.15.1:0/1806485360 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d64d91820 0x560d64df2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.454+0000 7fa046000700 1 -- 172.21.15.1:0/1806485360 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560d64b5c3b0 con 0x560d64d92140 2024-02-02T15:40:53.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.454+0000 7fa046a00700 1 --2- 172.21.15.1:0/1806485360 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d64d91820 0x560d64df2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:40:53.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.455+0000 7fa046000700 1 --2- 172.21.15.1:0/1806485360 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560d64d92140 0x560d64df2bb0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fa030095500 tx=0x7fa030005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbd6cce856f59649 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:53.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.455+0000 7fa034c00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:40:53.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.455+0000 7fa034c00700 1 -- 172.21.15.1:0/1806485360 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0300b6420 con 0x560d64d92140 2024-02-02T15:40:53.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.455+0000 7fa0614adb80 1 -- 172.21.15.1:0/1806485360 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa0300a0b00 con 0x560d64d92140 2024-02-02T15:40:53.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.455+0000 7fa034c00700 1 -- 172.21.15.1:0/1806485360 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fa0300b6810 con 0x560d64d92140 2024-02-02T15:40:53.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.455+0000 7fa034c00700 1 -- 172.21.15.1:0/1806485360 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa0300b69d0 con 0x560d64d92140 2024-02-02T15:40:53.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.455+0000 7fa0614adb80 10 client.29294 fetch_fsmap learned FSMap version 59 2024-02-02T15:40:53.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.455+0000 7fa0614adb80 1 -- 172.21.15.1:0/1806485360 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) v3 -- 0x560d64b60cd0 con 0x560d64d92140 2024-02-02T15:40:53.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.455+0000 7fa034c00700 1 -- 172.21.15.1:0/1806485360 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0300a0b00 con 0x560d64d92140 2024-02-02T15:40:53.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.456+0000 7fa034c00700 1 -- 172.21.15.1:0/1806485360 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 59) v1 ==== 7538+0+0 (secure 0 0 0) 0x7fa030098cf0 con 0x560d64d92140 2024-02-02T15:40:53.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.456+0000 7fa0614adb80 10 client.29294 fetch_fsmap finished waiting for FSMap version 59 2024-02-02T15:40:53.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.456+0000 7fa0614adb80 10 client.29294 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-02-02T15:40:53.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.456+0000 7fa0614adb80 20 client.29294 populate_metadata read hostname 'smithi001' 2024-02-02T15:40:53.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.456+0000 7fa0614adb80 1 --2- 172.21.15.1:0/1806485360 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x560d64dfeee0 0x560d64e012d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:53.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.456+0000 7fa0614adb80 4 client.29294 mds_command: new command op to 24514 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:40:53.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.456+0000 7fa0614adb80 1 -- 172.21.15.1:0/1806485360 --> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560d64c99f50 con 0x560d64dfeee0 2024-02-02T15:40:53.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.456+0000 7fa046a00700 1 --2- 172.21.15.1:0/1806485360 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x560d64dfeee0 0x560d64e012d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:53.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.457+0000 7fa034c00700 10 client.29294 ms_handle_connect on v2:172.21.15.142:6836/2870289019 2024-02-02T15:40:53.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.457+0000 7fa046a00700 1 --2- 172.21.15.1:0/1806485360 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x560d64dfeee0 0x560d64e012d0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9a5963d7dbf0ce4e server_cookie=1861a46e835e6e1a in_seq=0 out_seq=0 2024-02-02T15:40:53.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.458+0000 7fa034c00700 1 -- 172.21.15.1:0/1806485360 <== mds.3 v2:172.21.15.142:6836/2870289019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560d64c99f50 con 0x560d64dfeee0 2024-02-02T15:40:53.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.458+0000 7fa034c00700 10 client.29294 handle_command_reply: tid=0 2024-02-02T15:40:53.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.458+0000 7fa0614adb80 1 client.29294 shutdown 2024-02-02T15:40:53.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.458+0000 7fa0614adb80 1 -- 172.21.15.1:0/1806485360 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560d64d92140 msgr2=0x560d64df2bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:53.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.458+0000 7fa0614adb80 1 --2- 172.21.15.1:0/1806485360 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560d64d92140 0x560d64df2bb0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fa030095500 tx=0x7fa030005dc0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.459+0000 7fa0614adb80 1 -- 172.21.15.1:0/1806485360 shutdown_connections 2024-02-02T15:40:53.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.459+0000 7fa0614adb80 1 --2- 172.21.15.1:0/1806485360 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560d64d9cff0 0x560d64dfd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.459+0000 7fa0614adb80 1 --2- 172.21.15.1:0/1806485360 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x560d64dfeee0 0x560d64e012d0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.459+0000 7fa0614adb80 1 --2- 172.21.15.1:0/1806485360 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560d64d91820 0x560d64df2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.459+0000 7fa0614adb80 1 --2- 172.21.15.1:0/1806485360 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560d64d92140 0x560d64df2bb0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.459+0000 7fa0614adb80 1 -- 172.21.15.1:0/1806485360 >> 172.21.15.1:0/1806485360 conn(0x560d64ce8db0 msgr2=0x560d64d95290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:53.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.459+0000 7fa034c00700 0 client.29294 ms_handle_reset on v2:172.21.15.142:6836/2870289019 2024-02-02T15:40:53.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.459+0000 7fa0614adb80 1 -- 172.21.15.1:0/1806485360 shutdown_connections 2024-02-02T15:40:53.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.459+0000 7fa0614adb80 1 -- 172.21.15.1:0/1806485360 wait complete. 2024-02-02T15:40:53.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.459+0000 7fa0614adb80 20 client.29294 trim_cache size 0 max 16384 2024-02-02T15:40:53.460 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:40:53.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.460+0000 7fa04da00700 1 -- 172.21.15.1:0/501724908 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fa0280a0790 msgr2=0x7fa0280a2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:53.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.460+0000 7fa04da00700 1 --2- 172.21.15.1:0/501724908 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fa0280a0790 0x7fa0280a2c50 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa038011550 tx=0x7fa038009260 comp rx=0 tx=0).stop 2024-02-02T15:40:53.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.460+0000 7fa04da00700 1 -- 172.21.15.1:0/501724908 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa048073a30 msgr2=0x7fa048199710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:53.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.460+0000 7fa04da00700 1 --2- 172.21.15.1:0/501724908 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa048073a30 0x7fa048199710 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fa03000d750 tx=0x7fa03000da60 comp rx=0 tx=0).stop 2024-02-02T15:40:53.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.460+0000 7fa04da00700 1 -- 172.21.15.1:0/501724908 shutdown_connections 2024-02-02T15:40:53.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.460+0000 7fa04da00700 1 --2- 172.21.15.1:0/501724908 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fa0280a0790 0x7fa0280a2c50 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.460+0000 7fa04da00700 1 --2- 172.21.15.1:0/501724908 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa04810d640 0x7fa04819db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.460+0000 7fa04da00700 1 --2- 172.21.15.1:0/501724908 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa048073110 0x7fa0481991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.460+0000 7fa04da00700 1 --2- 172.21.15.1:0/501724908 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa048073a30 0x7fa048199710 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.460+0000 7fa04da00700 1 -- 172.21.15.1:0/501724908 >> 172.21.15.1:0/501724908 conn(0x7fa0480fc8b0 msgr2=0x7fa0481050a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:53.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.460+0000 7fa04da00700 1 -- 172.21.15.1:0/501724908 shutdown_connections 2024-02-02T15:40:53.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.460+0000 7fa04da00700 1 -- 172.21.15.1:0/501724908 wait complete. 2024-02-02T15:40:53.483 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-02T15:40:53.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.766+0000 7fc519000700 1 -- 172.21.15.1:0/2476363001 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc514073110 msgr2=0x7fc5140734f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:53.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.766+0000 7fc519000700 1 --2- 172.21.15.1:0/2476363001 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc514073110 0x7fc5140734f0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fc504009990 tx=0x7fc504009ca0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.766+0000 7fc519000700 1 -- 172.21.15.1:0/2476363001 shutdown_connections 2024-02-02T15:40:53.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.766+0000 7fc519000700 1 --2- 172.21.15.1:0/2476363001 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc51410d5d0 0x7fc51410fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.766+0000 7fc519000700 1 --2- 172.21.15.1:0/2476363001 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc514073110 0x7fc5140734f0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.766+0000 7fc519000700 1 --2- 172.21.15.1:0/2476363001 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc514073a30 0x7fc51410d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.766+0000 7fc519000700 1 -- 172.21.15.1:0/2476363001 >> 172.21.15.1:0/2476363001 conn(0x7fc5140fc800 msgr2=0x7fc5140fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:53.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.766+0000 7fc519000700 1 -- 172.21.15.1:0/2476363001 shutdown_connections 2024-02-02T15:40:53.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.766+0000 7fc519000700 1 -- 172.21.15.1:0/2476363001 wait complete. 2024-02-02T15:40:53.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.767+0000 7fc519000700 1 Processor -- start 2024-02-02T15:40:53.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.768+0000 7fc519000700 1 -- start start 2024-02-02T15:40:53.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.768+0000 7fc519000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc514073110 0x7fc514199040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:53.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.768+0000 7fc519000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc514073a30 0x7fc514199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:53.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.768+0000 7fc519000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc51410d5d0 0x7fc51419d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:53.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.768+0000 7fc519000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc51419e020 con 0x7fc514073a30 2024-02-02T15:40:53.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.768+0000 7fc519000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc51419e190 con 0x7fc51410d5d0 2024-02-02T15:40:53.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.768+0000 7fc519000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc51419e350 con 0x7fc514073110 2024-02-02T15:40:53.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.768+0000 7fc511e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc514073110 0x7fc514199040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:53.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.768+0000 7fc511e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc514073110 0x7fc514199040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:37504/0 (socket says 172.21.15.1:37504) 2024-02-02T15:40:53.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.768+0000 7fc511e00700 1 -- 172.21.15.1:0/2047039697 learned_addr learned my addr 172.21.15.1:0/2047039697 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:40:53.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.768+0000 7fc512800700 1 --2- 172.21.15.1:0/2047039697 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc51410d5d0 0x7fc51419d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:53.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.768+0000 7fc511400700 1 --2- 172.21.15.1:0/2047039697 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc514073a30 0x7fc514199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:53.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.769+0000 7fc511e00700 1 -- 172.21.15.1:0/2047039697 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc51410d5d0 msgr2=0x7fc51419d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:53.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.769+0000 7fc511e00700 1 --2- 172.21.15.1:0/2047039697 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc51410d5d0 0x7fc51419d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.769+0000 7fc511e00700 1 -- 172.21.15.1:0/2047039697 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc514073a30 msgr2=0x7fc514199580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:53.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.769+0000 7fc511e00700 1 --2- 172.21.15.1:0/2047039697 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc514073a30 0x7fc514199580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.769+0000 7fc511e00700 1 -- 172.21.15.1:0/2047039697 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5040095f0 con 0x7fc514073110 2024-02-02T15:40:53.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.769+0000 7fc512800700 1 --2- 172.21.15.1:0/2047039697 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc51410d5d0 0x7fc51419d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:40:53.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.769+0000 7fc511400700 1 --2- 172.21.15.1:0/2047039697 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc514073a30 0x7fc514199580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:40:53.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.769+0000 7fc511e00700 1 --2- 172.21.15.1:0/2047039697 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc514073110 0x7fc514199040 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fc504005910 tx=0x7fc5040057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:53.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.769+0000 7fc50aa00700 1 -- 172.21.15.1:0/2047039697 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc50400fb70 con 0x7fc514073110 2024-02-02T15:40:53.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.769+0000 7fc50aa00700 1 -- 172.21.15.1:0/2047039697 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc504027070 con 0x7fc514073110 2024-02-02T15:40:53.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.769+0000 7fc519000700 1 -- 172.21.15.1:0/2047039697 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc51419e5d0 con 0x7fc514073110 2024-02-02T15:40:53.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.769+0000 7fc50aa00700 1 -- 172.21.15.1:0/2047039697 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc504017880 con 0x7fc514073110 2024-02-02T15:40:53.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.770+0000 7fc519000700 1 -- 172.21.15.1:0/2047039697 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc514100e00 con 0x7fc514073110 2024-02-02T15:40:53.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.773+0000 7fc50aa00700 1 -- 172.21.15.1:0/2047039697 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7fc504029030 con 0x7fc514073110 2024-02-02T15:40:53.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.774+0000 7fc52cb83b80 1 Processor -- start 2024-02-02T15:40:53.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.774+0000 7fc52cb83b80 1 -- start start 2024-02-02T15:40:53.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.775+0000 7fc52cb83b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563df8551010 0x563df85513f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:53.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.775+0000 7fc52cb83b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563df8551930 0x563df855c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:53.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.775+0000 7fc52cb83b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563df855c7e0 0x563df855ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:53.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.775+0000 7fc52cb83b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x563df84c54c0 con 0x563df855c7e0 2024-02-02T15:40:53.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.775+0000 7fc52cb83b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x563df84a9aa0 con 0x563df8551010 2024-02-02T15:40:53.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.775+0000 7fc52cb83b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563df84a7460 con 0x563df8551930 2024-02-02T15:40:53.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.775+0000 7fc50aa00700 1 --2- 172.21.15.1:0/2047039697 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fc4f40a0710 0x7fc4f40a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:53.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.775+0000 7fc512800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563df8551010 0x563df85513f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:53.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.775+0000 7fc511e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563df8551930 0x563df855c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:53.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.775+0000 7fc50aa00700 1 -- 172.21.15.1:0/2047039697 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fc5040d5970 con 0x7fc514073110 2024-02-02T15:40:53.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.775+0000 7fc511e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563df8551930 0x563df855c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:37514/0 (socket says 172.21.15.1:37514) 2024-02-02T15:40:53.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.775+0000 7fc511e00700 1 -- 172.21.15.1:0/536740987 learned_addr learned my addr 172.21.15.1:0/536740987 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:40:53.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.776+0000 7fc511400700 1 --2- 172.21.15.1:0/536740987 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563df855c7e0 0x563df855ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:53.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.776+0000 7fc512800700 1 --2- 172.21.15.1:0/2047039697 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fc4f40a0710 0x7fc4f40a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:53.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.776+0000 7fc512800700 1 -- 172.21.15.1:0/536740987 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563df8551930 msgr2=0x563df855c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:53.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.776+0000 7fc512800700 1 --2- 172.21.15.1:0/536740987 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563df8551930 0x563df855c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.776+0000 7fc512800700 1 -- 172.21.15.1:0/536740987 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563df855c7e0 msgr2=0x563df855ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:53.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.776+0000 7fc512800700 1 --2- 172.21.15.1:0/536740987 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563df855c7e0 0x563df855ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.776+0000 7fc512800700 1 -- 172.21.15.1:0/536740987 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563df83243b0 con 0x563df8551010 2024-02-02T15:40:53.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.776+0000 7fc511e00700 1 --2- 172.21.15.1:0/536740987 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563df8551930 0x563df855c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:40:53.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.776+0000 7fc512800700 1 --2- 172.21.15.1:0/536740987 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563df8551010 0x563df85513f0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fc50c012a40 tx=0x7fc50c012d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ade7c9bd613f08da server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:53.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.777+0000 7fc503400700 1 -- 172.21.15.1:0/536740987 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc50c003c00 con 0x563df8551010 2024-02-02T15:40:53.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.777+0000 7fc512800700 1 --2- 172.21.15.1:0/2047039697 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fc4f40a0710 0x7fc4f40a2bd0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc50c015570 tx=0x7fc50c01b000 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:53.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.777+0000 7fc503400700 1 -- 172.21.15.1:0/536740987 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc50c003d60 con 0x563df8551010 2024-02-02T15:40:53.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.777+0000 7fc503400700 1 -- 172.21.15.1:0/536740987 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc50c004290 con 0x563df8551010 2024-02-02T15:40:53.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.778+0000 7fc52cb83b80 1 -- 172.21.15.1:0/536740987 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563df8551010 msgr2=0x563df85513f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:53.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.778+0000 7fc52cb83b80 1 --2- 172.21.15.1:0/536740987 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563df8551010 0x563df85513f0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fc50c012a40 tx=0x7fc50c012d50 comp rx=0 tx=0).stop 2024-02-02T15:40:53.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.778+0000 7fc52cb83b80 1 -- 172.21.15.1:0/536740987 shutdown_connections 2024-02-02T15:40:53.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.778+0000 7fc52cb83b80 1 --2- 172.21.15.1:0/536740987 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563df855c7e0 0x563df855ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.778+0000 7fc52cb83b80 1 --2- 172.21.15.1:0/536740987 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563df8551010 0x563df85513f0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.778+0000 7fc52cb83b80 1 --2- 172.21.15.1:0/536740987 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563df8551930 0x563df855c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.778+0000 7fc52cb83b80 1 -- 172.21.15.1:0/536740987 >> 172.21.15.1:0/536740987 conn(0x563df84b0db0 msgr2=0x563df854d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:53.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.778+0000 7fc52cb83b80 1 -- 172.21.15.1:0/536740987 shutdown_connections 2024-02-02T15:40:53.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.779+0000 7fc52cb83b80 1 -- 172.21.15.1:0/536740987 wait complete. 2024-02-02T15:40:53.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.779+0000 7fc52cb83b80 1 -- start start 2024-02-02T15:40:53.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.779+0000 7fc52cb83b80 1 Processor -- start 2024-02-02T15:40:53.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.780+0000 7fc52cb83b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563df8551010 0x563df855c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:53.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.780+0000 7fc52cb83b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563df8551930 0x563df8559480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:53.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.780+0000 7fc52cb83b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563df855c7e0 0x563df8559bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:53.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.780+0000 7fc52cb83b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x563df84a2d50 con 0x563df855c7e0 2024-02-02T15:40:53.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.780+0000 7fc52cb83b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x563df843ed80 con 0x563df8551010 2024-02-02T15:40:53.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.780+0000 7fc52cb83b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563df843ea20 con 0x563df8551930 2024-02-02T15:40:53.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.780+0000 7fc52cb83b80 10 client.0 _ll_register_callbacks cb 0x563df8440290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:40:53.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.780+0000 7fc511400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563df8551930 0x563df8559480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:53.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.781+0000 7fc511400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563df8551930 0x563df8559480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:37524/0 (socket says 172.21.15.1:37524) 2024-02-02T15:40:53.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.781+0000 7fc511400700 1 -- 172.21.15.1:0/1801611290 learned_addr learned my addr 172.21.15.1:0/1801611290 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:40:53.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.781+0000 7fc511e00700 1 --2- 172.21.15.1:0/1801611290 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563df8551010 0x563df855c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:53.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.781+0000 7fc511400700 1 -- 172.21.15.1:0/1801611290 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563df8551010 msgr2=0x563df855c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:53.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.781+0000 7fc511400700 1 --2- 172.21.15.1:0/1801611290 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563df8551010 0x563df855c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.781+0000 7fc511400700 1 -- 172.21.15.1:0/1801611290 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563df855c7e0 msgr2=0x563df8559bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:53.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.781+0000 7fc511400700 1 --2- 172.21.15.1:0/1801611290 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563df855c7e0 0x563df8559bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.781+0000 7fc511400700 1 -- 172.21.15.1:0/1801611290 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc514100e00 con 0x563df8551930 2024-02-02T15:40:53.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.781+0000 7fc511e00700 1 --2- 172.21.15.1:0/1801611290 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563df8551010 0x563df855c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:40:53.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.781+0000 7fc511400700 1 --2- 172.21.15.1:0/1801611290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563df8551930 0x563df8559480 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc013760 tx=0x7fc4fc013b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd8bf139e6e73862 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:53.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.781+0000 7fc4fbe00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:40:53.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.782+0000 7fc4fbe00700 1 -- 172.21.15.1:0/1801611290 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4fc011a70 con 0x563df8551930 2024-02-02T15:40:53.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.782+0000 7fc4fbe00700 1 -- 172.21.15.1:0/1801611290 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fc4fc011bd0 con 0x563df8551930 2024-02-02T15:40:53.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.782+0000 7fc52cb83b80 1 -- 172.21.15.1:0/1801611290 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563df85450f0 con 0x563df8551930 2024-02-02T15:40:53.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.782+0000 7fc4fbe00700 1 -- 172.21.15.1:0/1801611290 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc4fc011d90 con 0x563df8551930 2024-02-02T15:40:53.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.782+0000 7fc52cb83b80 10 client.29299 fetch_fsmap learned FSMap version 59 2024-02-02T15:40:53.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.782+0000 7fc52cb83b80 1 -- 172.21.15.1:0/1801611290 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) v3 -- 0x7fc50c012620 con 0x563df8551930 2024-02-02T15:40:53.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.782+0000 7fc4fbe00700 1 -- 172.21.15.1:0/1801611290 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563df85450f0 con 0x563df8551930 2024-02-02T15:40:53.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.783+0000 7fc4fbe00700 1 -- 172.21.15.1:0/1801611290 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 59) v1 ==== 7538+0+0 (secure 0 0 0) 0x7fc4fc01d500 con 0x563df8551930 2024-02-02T15:40:53.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.783+0000 7fc52cb83b80 10 client.29299 fetch_fsmap finished waiting for FSMap version 59 2024-02-02T15:40:53.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.783+0000 7fc52cb83b80 10 client.29299 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-02-02T15:40:53.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.783+0000 7fc52cb83b80 20 client.29299 populate_metadata read hostname 'smithi001' 2024-02-02T15:40:53.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.783+0000 7fc52cb83b80 1 --2- 172.21.15.1:0/1801611290 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x563df856c3d0 0x563df856e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:53.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.783+0000 7fc52cb83b80 4 client.29299 mds_command: new command op to 24518 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:40:53.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.783+0000 7fc52cb83b80 1 -- 172.21.15.1:0/1801611290 --> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563df849bdd0 con 0x563df856c3d0 2024-02-02T15:40:53.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.783+0000 7fc511e00700 1 --2- 172.21.15.1:0/1801611290 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x563df856c3d0 0x563df856e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:53.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.784+0000 7fc4fbe00700 10 client.29299 ms_handle_connect on v2:172.21.15.71:6836/628016199 2024-02-02T15:40:53.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.784+0000 7fc511e00700 1 --2- 172.21.15.1:0/1801611290 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x563df856c3d0 0x563df856e7f0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f14fabfc060d7481 server_cookie=732b6aef0246f477 in_seq=0 out_seq=0 2024-02-02T15:40:53.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.785+0000 7fc4fbe00700 1 -- 172.21.15.1:0/1801611290 <== mds.2 v2:172.21.15.71:6836/628016199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x563df849bdd0 con 0x563df856c3d0 2024-02-02T15:40:53.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.786+0000 7fc4fbe00700 10 client.29299 handle_command_reply: tid=0 2024-02-02T15:40:53.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.786+0000 7fc52cb83b80 1 client.29299 shutdown 2024-02-02T15:40:53.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.786+0000 7fc52cb83b80 1 -- 172.21.15.1:0/1801611290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563df8551930 msgr2=0x563df8559480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:53.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.786+0000 7fc52cb83b80 1 --2- 172.21.15.1:0/1801611290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563df8551930 0x563df8559480 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc013760 tx=0x7fc4fc013b20 comp rx=0 tx=0).stop 2024-02-02T15:40:53.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.786+0000 7fc52cb83b80 1 -- 172.21.15.1:0/1801611290 shutdown_connections 2024-02-02T15:40:53.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.786+0000 7fc52cb83b80 1 --2- 172.21.15.1:0/1801611290 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563df855c7e0 0x563df8559bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.786+0000 7fc52cb83b80 1 --2- 172.21.15.1:0/1801611290 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563df8551010 0x563df855c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.786+0000 7fc52cb83b80 1 --2- 172.21.15.1:0/1801611290 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x563df856c3d0 0x563df856e7f0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.786+0000 7fc4fbe00700 0 client.29299 ms_handle_reset on v2:172.21.15.71:6836/628016199 2024-02-02T15:40:53.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.786+0000 7fc52cb83b80 1 --2- 172.21.15.1:0/1801611290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563df8551930 0x563df8559480 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.786+0000 7fc52cb83b80 1 -- 172.21.15.1:0/1801611290 >> 172.21.15.1:0/1801611290 conn(0x563df84b0db0 msgr2=0x563df854cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:53.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.787+0000 7fc52cb83b80 1 -- 172.21.15.1:0/1801611290 shutdown_connections 2024-02-02T15:40:53.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.787+0000 7fc52cb83b80 1 -- 172.21.15.1:0/1801611290 wait complete. 2024-02-02T15:40:53.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.787+0000 7fc52cb83b80 20 client.29299 trim_cache size 0 max 16384 2024-02-02T15:40:53.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.809+0000 7fc52cb83b80 1 Processor -- start 2024-02-02T15:40:53.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.810+0000 7fc52cb83b80 1 -- start start 2024-02-02T15:40:53.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.810+0000 7fc52cb83b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563df8551010 0x563df85632d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:53.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.810+0000 7fc52cb83b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563df8551930 0x563df855bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:53.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.810+0000 7fc52cb83b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563df855c7e0 0x563df854ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:53.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.810+0000 7fc52cb83b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x563df83266a0 con 0x563df8551010 2024-02-02T15:40:53.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.810+0000 7fc52cb83b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x563df8569a10 con 0x563df855c7e0 2024-02-02T15:40:53.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.810+0000 7fc52cb83b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563df855a240 con 0x563df8551930 2024-02-02T15:40:53.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.810+0000 7fc511e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563df8551010 0x563df85632d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:53.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.810+0000 7fc511e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563df8551010 0x563df85632d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53696/0 (socket says 172.21.15.1:53696) 2024-02-02T15:40:53.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.810+0000 7fc511e00700 1 -- 172.21.15.1:0/1539804864 learned_addr learned my addr 172.21.15.1:0/1539804864 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:40:53.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.810+0000 7fc511400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563df8551930 0x563df855bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:53.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.811+0000 7fc512800700 1 --2- 172.21.15.1:0/1539804864 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563df855c7e0 0x563df854ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:53.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.811+0000 7fc512800700 1 -- 172.21.15.1:0/1539804864 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563df8551930 msgr2=0x563df855bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:53.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.811+0000 7fc512800700 1 --2- 172.21.15.1:0/1539804864 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563df8551930 0x563df855bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.811+0000 7fc512800700 1 -- 172.21.15.1:0/1539804864 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563df8551010 msgr2=0x563df85632d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:53.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.811+0000 7fc512800700 1 --2- 172.21.15.1:0/1539804864 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563df8551010 0x563df85632d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.811+0000 7fc512800700 1 -- 172.21.15.1:0/1539804864 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563df82d0b60 con 0x563df855c7e0 2024-02-02T15:40:53.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.811+0000 7fc511e00700 1 --2- 172.21.15.1:0/1539804864 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563df8551010 0x563df85632d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:40:53.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.812+0000 7fc512800700 1 --2- 172.21.15.1:0/1539804864 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563df855c7e0 0x563df854ccd0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fc50c007980 tx=0x7fc50c007d40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89816a65f8dd4c13 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:53.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.812+0000 7fc503e00700 1 -- 172.21.15.1:0/1539804864 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc50c003c00 con 0x563df855c7e0 2024-02-02T15:40:53.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.812+0000 7fc503e00700 1 -- 172.21.15.1:0/1539804864 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc50c007df0 con 0x563df855c7e0 2024-02-02T15:40:53.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.812+0000 7fc503e00700 1 -- 172.21.15.1:0/1539804864 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc50c004710 con 0x563df855c7e0 2024-02-02T15:40:53.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.813+0000 7fc52cb83b80 1 -- 172.21.15.1:0/1539804864 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563df855c7e0 msgr2=0x563df854ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:53.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.813+0000 7fc52cb83b80 1 --2- 172.21.15.1:0/1539804864 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563df855c7e0 0x563df854ccd0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fc50c007980 tx=0x7fc50c007d40 comp rx=0 tx=0).stop 2024-02-02T15:40:53.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.814+0000 7fc52cb83b80 1 -- 172.21.15.1:0/1539804864 shutdown_connections 2024-02-02T15:40:53.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.814+0000 7fc52cb83b80 1 --2- 172.21.15.1:0/1539804864 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563df855c7e0 0x563df854ccd0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.814+0000 7fc52cb83b80 1 --2- 172.21.15.1:0/1539804864 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563df8551010 0x563df85632d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.814+0000 7fc52cb83b80 1 --2- 172.21.15.1:0/1539804864 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563df8551930 0x563df855bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.814+0000 7fc52cb83b80 1 -- 172.21.15.1:0/1539804864 >> 172.21.15.1:0/1539804864 conn(0x563df84b0db0 msgr2=0x563df854c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:53.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.814+0000 7fc52cb83b80 1 -- 172.21.15.1:0/1539804864 shutdown_connections 2024-02-02T15:40:53.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.814+0000 7fc52cb83b80 1 -- 172.21.15.1:0/1539804864 wait complete. 2024-02-02T15:40:53.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.815+0000 7fc52cb83b80 1 -- start start 2024-02-02T15:40:53.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.815+0000 7fc52cb83b80 1 Processor -- start 2024-02-02T15:40:53.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.815+0000 7fc52cb83b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563df8551010 0x563df85614a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:53.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.815+0000 7fc52cb83b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563df8551930 0x563df85619e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:53.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.815+0000 7fc52cb83b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563df855c7e0 0x563df85bcef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:53.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.815+0000 7fc52cb83b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x563df85bd430 con 0x563df8551930 2024-02-02T15:40:53.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.815+0000 7fc52cb83b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x563df85bd570 con 0x563df8551010 2024-02-02T15:40:53.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.815+0000 7fc52cb83b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563df85bd730 con 0x563df855c7e0 2024-02-02T15:40:53.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.815+0000 7fc52cb83b80 10 client.0 _ll_register_callbacks cb 0x563df8440290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:40:53.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.816+0000 7fc511400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563df8551930 0x563df85619e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:53.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.816+0000 7fc511400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563df8551930 0x563df85619e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53712/0 (socket says 172.21.15.1:53712) 2024-02-02T15:40:53.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.816+0000 7fc511400700 1 -- 172.21.15.1:0/1769840283 learned_addr learned my addr 172.21.15.1:0/1769840283 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:40:53.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.816+0000 7fc511e00700 1 --2- 172.21.15.1:0/1769840283 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563df8551010 0x563df85614a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:53.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.816+0000 7fc511400700 1 -- 172.21.15.1:0/1769840283 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563df855c7e0 msgr2=0x563df85bcef0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-02T15:40:53.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.816+0000 7fc511400700 1 --2- 172.21.15.1:0/1769840283 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563df855c7e0 0x563df85bcef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.816+0000 7fc511400700 1 -- 172.21.15.1:0/1769840283 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563df8551010 msgr2=0x563df85614a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:53.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.816+0000 7fc511400700 1 --2- 172.21.15.1:0/1769840283 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563df8551010 0x563df85614a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.816+0000 7fc511400700 1 -- 172.21.15.1:0/1769840283 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc51419e5d0 con 0x563df8551930 2024-02-02T15:40:53.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.816+0000 7fc511400700 1 --2- 172.21.15.1:0/1769840283 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563df8551930 0x563df85619e0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc01d830 tx=0x7fc4fc0059b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f86a8f40564635b server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:53.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.816+0000 7fc4fbe00700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:40:53.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.817+0000 7fc4fbe00700 1 -- 172.21.15.1:0/1769840283 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4fc0054c0 con 0x563df8551930 2024-02-02T15:40:53.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.817+0000 7fc4fbe00700 1 -- 172.21.15.1:0/1769840283 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fc4fc000e40 con 0x563df8551930 2024-02-02T15:40:53.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.817+0000 7fc52cb83b80 1 -- 172.21.15.1:0/1769840283 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563df8461f50 con 0x563df8551930 2024-02-02T15:40:53.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.817+0000 7fc4fbe00700 1 -- 172.21.15.1:0/1769840283 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc4fc01d620 con 0x563df8551930 2024-02-02T15:40:53.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.817+0000 7fc52cb83b80 10 client.19194 fetch_fsmap learned FSMap version 59 2024-02-02T15:40:53.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.817+0000 7fc52cb83b80 1 -- 172.21.15.1:0/1769840283 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) v3 -- 0x7fc50c024070 con 0x563df8551930 2024-02-02T15:40:53.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.817+0000 7fc4fbe00700 1 -- 172.21.15.1:0/1769840283 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563df8461f50 con 0x563df8551930 2024-02-02T15:40:53.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.818+0000 7fc4fbe00700 1 -- 172.21.15.1:0/1769840283 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 59) v1 ==== 7538+0+0 (secure 0 0 0) 0x7fc4fc0038e0 con 0x563df8551930 2024-02-02T15:40:53.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.819+0000 7fc52cb83b80 10 client.19194 fetch_fsmap finished waiting for FSMap version 59 2024-02-02T15:40:53.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.819+0000 7fc52cb83b80 10 client.19194 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-02-02T15:40:53.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.819+0000 7fc52cb83b80 20 client.19194 populate_metadata read hostname 'smithi001' 2024-02-02T15:40:53.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.819+0000 7fc52cb83b80 1 --2- 172.21.15.1:0/1769840283 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x563df85bedd0 0x563df85c11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:53.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.819+0000 7fc52cb83b80 4 client.19194 mds_command: new command op to 24518 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:40:53.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.819+0000 7fc52cb83b80 1 -- 172.21.15.1:0/1769840283 --> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563df82f1cf0 con 0x563df85bedd0 2024-02-02T15:40:53.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.819+0000 7fc511e00700 1 --2- 172.21.15.1:0/1769840283 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x563df85bedd0 0x563df85c11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:53.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.819+0000 7fc511e00700 1 --2- 172.21.15.1:0/1769840283 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x563df85bedd0 0x563df85c11f0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=404fa3ead7e7a9d9 server_cookie=65503c80d828eecf in_seq=0 out_seq=0 2024-02-02T15:40:53.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.819+0000 7fc4fbe00700 10 client.19194 ms_handle_connect on v2:172.21.15.71:6836/628016199 2024-02-02T15:40:53.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.821+0000 7fc4fbe00700 1 -- 172.21.15.1:0/1769840283 <== mds.2 v2:172.21.15.71:6836/628016199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563df82f1cf0 con 0x563df85bedd0 2024-02-02T15:40:53.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.821+0000 7fc4fbe00700 10 client.19194 handle_command_reply: tid=0 2024-02-02T15:40:53.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.821+0000 7fc52cb83b80 1 client.19194 shutdown 2024-02-02T15:40:53.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.821+0000 7fc52cb83b80 1 -- 172.21.15.1:0/1769840283 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563df8551930 msgr2=0x563df85619e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:53.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.821+0000 7fc52cb83b80 1 --2- 172.21.15.1:0/1769840283 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563df8551930 0x563df85619e0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc01d830 tx=0x7fc4fc0059b0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.822+0000 7fc52cb83b80 1 -- 172.21.15.1:0/1769840283 shutdown_connections 2024-02-02T15:40:53.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.822+0000 7fc52cb83b80 1 --2- 172.21.15.1:0/1769840283 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x563df85bedd0 0x563df85c11f0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.822+0000 7fc52cb83b80 1 --2- 172.21.15.1:0/1769840283 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563df855c7e0 0x563df85bcef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.822+0000 7fc52cb83b80 1 --2- 172.21.15.1:0/1769840283 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563df8551010 0x563df85614a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.822+0000 7fc52cb83b80 1 --2- 172.21.15.1:0/1769840283 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563df8551930 0x563df85619e0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.822+0000 7fc4fbe00700 0 client.19194 ms_handle_reset on v2:172.21.15.71:6836/628016199 2024-02-02T15:40:53.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.822+0000 7fc52cb83b80 1 -- 172.21.15.1:0/1769840283 >> 172.21.15.1:0/1769840283 conn(0x563df84b0db0 msgr2=0x563df854ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:53.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.822+0000 7fc52cb83b80 1 -- 172.21.15.1:0/1769840283 shutdown_connections 2024-02-02T15:40:53.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.822+0000 7fc52cb83b80 1 -- 172.21.15.1:0/1769840283 wait complete. 2024-02-02T15:40:53.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.822+0000 7fc52cb83b80 20 client.19194 trim_cache size 0 max 16384 2024-02-02T15:40:53.823 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:40:53.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.823+0000 7fc519000700 1 -- 172.21.15.1:0/2047039697 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fc4f40a0710 msgr2=0x7fc4f40a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:53.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.823+0000 7fc519000700 1 --2- 172.21.15.1:0/2047039697 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fc4f40a0710 0x7fc4f40a2bd0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc50c015570 tx=0x7fc50c01b000 comp rx=0 tx=0).stop 2024-02-02T15:40:53.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.823+0000 7fc519000700 1 -- 172.21.15.1:0/2047039697 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc514073110 msgr2=0x7fc514199040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:53.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.823+0000 7fc519000700 1 --2- 172.21.15.1:0/2047039697 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc514073110 0x7fc514199040 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fc504005910 tx=0x7fc5040057f0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.823+0000 7fc519000700 1 -- 172.21.15.1:0/2047039697 shutdown_connections 2024-02-02T15:40:53.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.823+0000 7fc519000700 1 --2- 172.21.15.1:0/2047039697 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fc4f40a0710 0x7fc4f40a2bd0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.823+0000 7fc519000700 1 --2- 172.21.15.1:0/2047039697 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc51410d5d0 0x7fc51419d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.823+0000 7fc519000700 1 --2- 172.21.15.1:0/2047039697 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc514073110 0x7fc514199040 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.823+0000 7fc519000700 1 --2- 172.21.15.1:0/2047039697 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc514073a30 0x7fc514199580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:53.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.823+0000 7fc519000700 1 -- 172.21.15.1:0/2047039697 >> 172.21.15.1:0/2047039697 conn(0x7fc5140fc800 msgr2=0x7fc51410f6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:53.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.823+0000 7fc519000700 1 -- 172.21.15.1:0/2047039697 shutdown_connections 2024-02-02T15:40:53.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:53.823+0000 7fc519000700 1 -- 172.21.15.1:0/2047039697 wait complete. 2024-02-02T15:40:53.842 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:53 smithi001 ceph-mon[35470]: pgmap v433: 129 pgs: 129 active+clean; 885 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 19 MiB/s wr, 422 op/s 2024-02-02T15:40:53.842 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:53 smithi001 ceph-mon[35470]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:40:53.842 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:53 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/36731602' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:40:53.845 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-02T15:40:53.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:53 smithi071 ceph-mon[44707]: pgmap v433: 129 pgs: 129 active+clean; 885 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 19 MiB/s wr, 422 op/s 2024-02-02T15:40:53.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:53 smithi071 ceph-mon[44707]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:40:53.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:53 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/36731602' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:40:53.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:53 smithi142 ceph-mon[45728]: pgmap v433: 129 pgs: 129 active+clean; 885 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 19 MiB/s wr, 422 op/s 2024-02-02T15:40:53.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:53 smithi142 ceph-mon[45728]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:40:53.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:53 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/36731602' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:40:53.968 INFO:tasks.workunit.client.0.smithi001.stdout: 10 10887 29.63 MB/sec warmup 26 sec latency 1040.119 ms 2024-02-02T15:40:54.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.140+0000 7f7c8fc00700 1 -- 172.21.15.1:0/2451314229 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c88102030 msgr2=0x7f7c88102410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:54.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.140+0000 7f7c8fc00700 1 --2- 172.21.15.1:0/2451314229 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c88102030 0x7f7c88102410 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f7c70009990 tx=0x7f7c70009ca0 comp rx=0 tx=0).stop 2024-02-02T15:40:54.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.140+0000 7f7c8fc00700 1 -- 172.21.15.1:0/2451314229 shutdown_connections 2024-02-02T15:40:54.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.140+0000 7f7c8fc00700 1 --2- 172.21.15.1:0/2451314229 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7c8810b400 0x7f7c8810d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:54.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.140+0000 7f7c8fc00700 1 --2- 172.21.15.1:0/2451314229 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c88102030 0x7f7c88102410 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:54.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.140+0000 7f7c8fc00700 1 --2- 172.21.15.1:0/2451314229 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c88102950 0x7f7c8810ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:54.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.140+0000 7f7c8fc00700 1 -- 172.21.15.1:0/2451314229 >> 172.21.15.1:0/2451314229 conn(0x7f7c88076d70 msgr2=0x7f7c88077180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:54.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.141+0000 7f7c8fc00700 1 -- 172.21.15.1:0/2451314229 shutdown_connections 2024-02-02T15:40:54.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.141+0000 7f7c8fc00700 1 -- 172.21.15.1:0/2451314229 wait complete. 2024-02-02T15:40:54.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.142+0000 7f7c8fc00700 1 Processor -- start 2024-02-02T15:40:54.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.142+0000 7f7c8fc00700 1 -- start start 2024-02-02T15:40:54.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.143+0000 7f7c8fc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c88102030 0x7f7c8806ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:54.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.143+0000 7f7c8fc00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7c88102950 0x7f7c88070450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:54.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.143+0000 7f7c8fc00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c8810b400 0x7f7c8806b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:54.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.143+0000 7f7c8fc00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c8806b6d0 con 0x7f7c88102950 2024-02-02T15:40:54.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.143+0000 7f7c8fc00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c8806b840 con 0x7f7c8810b400 2024-02-02T15:40:54.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.143+0000 7f7c8fc00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c8806ba00 con 0x7f7c88102030 2024-02-02T15:40:54.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.143+0000 7f7c8dc00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c8810b400 0x7f7c8806b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:54.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.143+0000 7f7c8d200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c88102030 0x7f7c8806ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:54.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.143+0000 7f7c8dc00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c8810b400 0x7f7c8806b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:41730/0 (socket says 172.21.15.1:41730) 2024-02-02T15:40:54.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.143+0000 7f7c8d200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c88102030 0x7f7c8806ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:37554/0 (socket says 172.21.15.1:37554) 2024-02-02T15:40:54.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.143+0000 7f7c8dc00700 1 -- 172.21.15.1:0/1010894930 learned_addr learned my addr 172.21.15.1:0/1010894930 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:40:54.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.143+0000 7f7c87e00700 1 --2- 172.21.15.1:0/1010894930 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7c88102950 0x7f7c88070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:54.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.144+0000 7f7c8dc00700 1 -- 172.21.15.1:0/1010894930 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c88102030 msgr2=0x7f7c8806ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:54.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.144+0000 7f7c8dc00700 1 --2- 172.21.15.1:0/1010894930 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c88102030 0x7f7c8806ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:54.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.144+0000 7f7c8dc00700 1 -- 172.21.15.1:0/1010894930 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7c88102950 msgr2=0x7f7c88070450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:54.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.144+0000 7f7c8dc00700 1 --2- 172.21.15.1:0/1010894930 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7c88102950 0x7f7c88070450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:54.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.144+0000 7f7c8dc00700 1 -- 172.21.15.1:0/1010894930 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c80009c40 con 0x7f7c8810b400 2024-02-02T15:40:54.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.144+0000 7f7c8dc00700 1 --2- 172.21.15.1:0/1010894930 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c8810b400 0x7f7c8806b100 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f7c8000c0f0 tx=0x7f7c8000c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:54.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.144+0000 7f7c85600700 1 -- 172.21.15.1:0/1010894930 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c80003a00 con 0x7f7c8810b400 2024-02-02T15:40:54.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.144+0000 7f7c85600700 1 -- 172.21.15.1:0/1010894930 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c8000e030 con 0x7f7c8810b400 2024-02-02T15:40:54.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.144+0000 7f7c8fc00700 1 -- 172.21.15.1:0/1010894930 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c700095f0 con 0x7f7c8810b400 2024-02-02T15:40:54.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.144+0000 7f7c85600700 1 -- 172.21.15.1:0/1010894930 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c800045a0 con 0x7f7c8810b400 2024-02-02T15:40:54.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.145+0000 7f7c8fc00700 1 -- 172.21.15.1:0/1010894930 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c8806ec70 con 0x7f7c8810b400 2024-02-02T15:40:54.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.148+0000 7f7c85600700 1 -- 172.21.15.1:0/1010894930 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f7c80004700 con 0x7f7c8810b400 2024-02-02T15:40:54.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.149+0000 7f7c85600700 1 --2- 172.21.15.1:0/1010894930 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f7c680a0690 0x7f7c680a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:54.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.149+0000 7f7ca369db80 1 Processor -- start 2024-02-02T15:40:54.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.150+0000 7f7c85600700 1 -- 172.21.15.1:0/1010894930 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f7c800cbf50 con 0x7f7c8810b400 2024-02-02T15:40:54.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.150+0000 7f7ca369db80 1 -- start start 2024-02-02T15:40:54.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.150+0000 7f7c8d200700 1 --2- 172.21.15.1:0/1010894930 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f7c680a0690 0x7f7c680a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:54.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.150+0000 7f7ca369db80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5635e6170010 0x5635e61703f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:54.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.150+0000 7f7ca369db80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5635e6170930 0x5635e617b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:54.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.150+0000 7f7ca369db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5635e617b7e0 0x5635e617dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:54.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.150+0000 7f7ca369db80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e60e44c0 con 0x5635e6170930 2024-02-02T15:40:54.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.150+0000 7f7ca369db80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e60c8aa0 con 0x5635e6170010 2024-02-02T15:40:54.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.150+0000 7f7ca369db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e60c6460 con 0x5635e617b7e0 2024-02-02T15:40:54.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.150+0000 7f7c87e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5635e6170930 0x5635e617b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:54.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.151+0000 7f7c87e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5635e6170930 0x5635e617b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53738/0 (socket says 172.21.15.1:53738) 2024-02-02T15:40:54.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.151+0000 7f7c8dc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5635e617b7e0 0x5635e617dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:54.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.151+0000 7f7c87e00700 1 -- 172.21.15.1:0/1664747724 learned_addr learned my addr 172.21.15.1:0/1664747724 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:40:54.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.151+0000 7f7c8d200700 1 --2- 172.21.15.1:0/1664747724 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5635e6170010 0x5635e61703f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:54.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.151+0000 7f7c87e00700 1 -- 172.21.15.1:0/1664747724 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5635e617b7e0 msgr2=0x5635e617dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:54.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.151+0000 7f7c87e00700 1 --2- 172.21.15.1:0/1664747724 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5635e617b7e0 0x5635e617dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:54.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.151+0000 7f7c87e00700 1 -- 172.21.15.1:0/1664747724 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5635e6170010 msgr2=0x5635e61703f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:54.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.151+0000 7f7c87e00700 1 --2- 172.21.15.1:0/1664747724 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5635e6170010 0x5635e61703f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:54.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.151+0000 7f7c87e00700 1 -- 172.21.15.1:0/1664747724 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5635e5f433b0 con 0x5635e6170930 2024-02-02T15:40:54.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.151+0000 7f7c8d200700 1 --2- 172.21.15.1:0/1010894930 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f7c680a0690 0x7f7c680a2b50 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7c7000b2e0 tx=0x7f7c7000b850 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:54.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.151+0000 7f7c87e00700 1 --2- 172.21.15.1:0/1664747724 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5635e6170930 0x5635e617b2a0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f7c78009cf0 tx=0x7f7c78011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a06eb65b87543ab4 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:54.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.152+0000 7f7c7e000700 1 -- 172.21.15.1:0/1664747724 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c78011d80 con 0x5635e6170930 2024-02-02T15:40:54.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.152+0000 7f7c7e000700 1 -- 172.21.15.1:0/1664747724 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c78004510 con 0x5635e6170930 2024-02-02T15:40:54.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.152+0000 7f7c7e000700 1 -- 172.21.15.1:0/1664747724 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c78005280 con 0x5635e6170930 2024-02-02T15:40:54.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.153+0000 7f7ca369db80 1 -- 172.21.15.1:0/1664747724 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5635e6170930 msgr2=0x5635e617b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:54.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.153+0000 7f7ca369db80 1 --2- 172.21.15.1:0/1664747724 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5635e6170930 0x5635e617b2a0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f7c78009cf0 tx=0x7f7c78011440 comp rx=0 tx=0).stop 2024-02-02T15:40:54.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.153+0000 7f7ca369db80 1 -- 172.21.15.1:0/1664747724 shutdown_connections 2024-02-02T15:40:54.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.153+0000 7f7ca369db80 1 --2- 172.21.15.1:0/1664747724 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5635e617b7e0 0x5635e617dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:54.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.153+0000 7f7ca369db80 1 --2- 172.21.15.1:0/1664747724 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5635e6170010 0x5635e61703f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:54.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.153+0000 7f7ca369db80 1 --2- 172.21.15.1:0/1664747724 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5635e6170930 0x5635e617b2a0 secure :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f7c78009cf0 tx=0x7f7c78011440 comp rx=0 tx=0).stop 2024-02-02T15:40:54.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.153+0000 7f7ca369db80 1 -- 172.21.15.1:0/1664747724 >> 172.21.15.1:0/1664747724 conn(0x5635e60cfdb0 msgr2=0x5635e616c3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:54.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.153+0000 7f7ca369db80 1 -- 172.21.15.1:0/1664747724 shutdown_connections 2024-02-02T15:40:54.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.154+0000 7f7ca369db80 1 -- 172.21.15.1:0/1664747724 wait complete. 2024-02-02T15:40:54.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.154+0000 7f7ca369db80 1 -- start start 2024-02-02T15:40:54.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.155+0000 7f7ca369db80 1 Processor -- start 2024-02-02T15:40:54.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.155+0000 7f7ca369db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5635e6170010 0x5635e61786c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:54.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.155+0000 7f7ca369db80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5635e617b7e0 0x5635e6178c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:54.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.155+0000 7f7ca369db80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5635e6179140 0x5635e61873e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:54.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.155+0000 7f7ca369db80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e60c1d50 con 0x5635e6179140 2024-02-02T15:40:54.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.155+0000 7f7ca369db80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e605dd80 con 0x5635e617b7e0 2024-02-02T15:40:54.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.155+0000 7f7ca369db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e605da20 con 0x5635e6170010 2024-02-02T15:40:54.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.155+0000 7f7ca369db80 10 client.0 _ll_register_callbacks cb 0x5635e605f290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:40:54.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.156+0000 7f7c87e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5635e617b7e0 0x5635e6178c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:54.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.156+0000 7f7c87e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5635e617b7e0 0x5635e6178c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:41750/0 (socket says 172.21.15.1:41750) 2024-02-02T15:40:54.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.156+0000 7f7c87e00700 1 -- 172.21.15.1:0/1692203670 learned_addr learned my addr 172.21.15.1:0/1692203670 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:40:54.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.156+0000 7f7c8dc00700 1 --2- 172.21.15.1:0/1692203670 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5635e6179140 0x5635e61873e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:54.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.156+0000 7f7c87e00700 1 -- 172.21.15.1:0/1692203670 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5635e6170010 msgr2=0x5635e61786c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-02T15:40:54.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.156+0000 7f7c87e00700 1 --2- 172.21.15.1:0/1692203670 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5635e6170010 0x5635e61786c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:54.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.156+0000 7f7c87e00700 1 -- 172.21.15.1:0/1692203670 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5635e6179140 msgr2=0x5635e61873e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:54.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.156+0000 7f7c87e00700 1 --2- 172.21.15.1:0/1692203670 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5635e6179140 0x5635e61873e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:54.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.156+0000 7f7c87e00700 1 -- 172.21.15.1:0/1692203670 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c78009920 con 0x5635e617b7e0 2024-02-02T15:40:54.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.156+0000 7f7c8dc00700 1 --2- 172.21.15.1:0/1692203670 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5635e6179140 0x5635e61873e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:40:54.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.157+0000 7f7c87e00700 1 --2- 172.21.15.1:0/1692203670 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5635e617b7e0 0x5635e6178c00 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f7c78011790 tx=0x7f7c78011c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b799435cffc8d966 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:54.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.157+0000 7f7c76a00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:40:54.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.157+0000 7f7c76a00700 1 -- 172.21.15.1:0/1692203670 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c78004260 con 0x5635e617b7e0 2024-02-02T15:40:54.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.158+0000 7f7c76a00700 1 -- 172.21.15.1:0/1692203670 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f7c78003680 con 0x5635e617b7e0 2024-02-02T15:40:54.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.158+0000 7f7ca369db80 1 -- 172.21.15.1:0/1692203670 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5635e61640f0 con 0x5635e617b7e0 2024-02-02T15:40:54.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.159+0000 7f7c76a00700 1 -- 172.21.15.1:0/1692203670 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c78003840 con 0x5635e617b7e0 2024-02-02T15:40:54.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.159+0000 7f7ca369db80 10 client.29324 fetch_fsmap learned FSMap version 59 2024-02-02T15:40:54.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.159+0000 7f7ca369db80 1 -- 172.21.15.1:0/1692203670 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) v3 -- 0x5635e5eefb60 con 0x5635e617b7e0 2024-02-02T15:40:54.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.159+0000 7f7c76a00700 1 -- 172.21.15.1:0/1692203670 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5635e61640f0 con 0x5635e617b7e0 2024-02-02T15:40:54.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.160+0000 7f7c76a00700 1 -- 172.21.15.1:0/1692203670 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 59) v1 ==== 7538+0+0 (secure 0 0 0) 0x7f7c78005400 con 0x5635e617b7e0 2024-02-02T15:40:54.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.160+0000 7f7ca369db80 10 client.29324 fetch_fsmap finished waiting for FSMap version 59 2024-02-02T15:40:54.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.160+0000 7f7ca369db80 10 client.29324 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-02T15:40:54.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.160+0000 7f7ca369db80 20 client.29324 populate_metadata read hostname 'smithi001' 2024-02-02T15:40:54.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.160+0000 7f7ca369db80 1 --2- 172.21.15.1:0/1692203670 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x5635e618b820 0x5635e618dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:54.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.160+0000 7f7ca369db80 4 client.29324 mds_command: new command op to 24529 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:40:54.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.160+0000 7f7ca369db80 1 -- 172.21.15.1:0/1692203670 --> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5635e60badd0 con 0x5635e618b820 2024-02-02T15:40:54.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.161+0000 7f7c8d200700 1 --2- 172.21.15.1:0/1692203670 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x5635e618b820 0x5635e618dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:54.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.162+0000 7f7c8d200700 1 --2- 172.21.15.1:0/1692203670 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x5635e618b820 0x5635e618dc40 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=932f2f1a30c04e08 server_cookie=32baaae9af6b9b54 in_seq=0 out_seq=0 2024-02-02T15:40:54.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.162+0000 7f7c76a00700 10 client.29324 ms_handle_connect on v2:172.21.15.142:6838/321675421 2024-02-02T15:40:54.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.163+0000 7f7c76a00700 1 -- 172.21.15.1:0/1692203670 <== mds.1 v2:172.21.15.142:6838/321675421 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5635e60badd0 con 0x5635e618b820 2024-02-02T15:40:54.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.163+0000 7f7c76a00700 10 client.29324 handle_command_reply: tid=0 2024-02-02T15:40:54.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.164+0000 7f7ca369db80 1 client.29324 shutdown 2024-02-02T15:40:54.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.164+0000 7f7ca369db80 1 -- 172.21.15.1:0/1692203670 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5635e617b7e0 msgr2=0x5635e6178c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:54.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.164+0000 7f7ca369db80 1 --2- 172.21.15.1:0/1692203670 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5635e617b7e0 0x5635e6178c00 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f7c78011790 tx=0x7f7c78011c30 comp rx=0 tx=0).stop 2024-02-02T15:40:54.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.164+0000 7f7ca369db80 1 -- 172.21.15.1:0/1692203670 shutdown_connections 2024-02-02T15:40:54.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.164+0000 7f7ca369db80 1 --2- 172.21.15.1:0/1692203670 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x5635e618b820 0x5635e618dc40 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:54.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.164+0000 7f7ca369db80 1 --2- 172.21.15.1:0/1692203670 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5635e6179140 0x5635e61873e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:54.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.164+0000 7f7ca369db80 1 --2- 172.21.15.1:0/1692203670 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5635e6170010 0x5635e61786c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:54.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.164+0000 7f7c76a00700 0 client.29324 ms_handle_reset on v2:172.21.15.142:6838/321675421 2024-02-02T15:40:54.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.164+0000 7f7ca369db80 1 --2- 172.21.15.1:0/1692203670 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5635e617b7e0 0x5635e6178c00 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:54.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.164+0000 7f7ca369db80 1 -- 172.21.15.1:0/1692203670 >> 172.21.15.1:0/1692203670 conn(0x5635e60cfdb0 msgr2=0x5635e616bfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:54.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.164+0000 7f7ca369db80 1 -- 172.21.15.1:0/1692203670 shutdown_connections 2024-02-02T15:40:54.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.164+0000 7f7ca369db80 1 -- 172.21.15.1:0/1692203670 wait complete. 2024-02-02T15:40:54.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.165+0000 7f7ca369db80 20 client.29324 trim_cache size 0 max 16384 2024-02-02T15:40:54.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.184+0000 7f7ca369db80 1 Processor -- start 2024-02-02T15:40:54.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.184+0000 7f7ca369db80 1 -- start start 2024-02-02T15:40:54.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.185+0000 7f7ca369db80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5635e6170010 0x5635e6182320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:54.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.185+0000 7f7ca369db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5635e6179140 0x5635e6173050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:54.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.185+0000 7f7ca369db80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5635e617b7e0 0x5635e6187ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:54.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.185+0000 7f7ca369db80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e5f456a0 con 0x5635e617b7e0 2024-02-02T15:40:54.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.185+0000 7f7ca369db80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e6178f30 con 0x5635e6170010 2024-02-02T15:40:54.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.185+0000 7f7ca369db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e61744b0 con 0x5635e6179140 2024-02-02T15:40:54.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.185+0000 7f7c8d200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5635e6170010 0x5635e6182320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:54.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.185+0000 7f7c8d200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5635e6170010 0x5635e6182320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:41752/0 (socket says 172.21.15.1:41752) 2024-02-02T15:40:54.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.185+0000 7f7c8d200700 1 -- 172.21.15.1:0/2197201594 learned_addr learned my addr 172.21.15.1:0/2197201594 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:40:54.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.185+0000 7f7c87e00700 1 --2- 172.21.15.1:0/2197201594 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5635e6179140 0x5635e6173050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:54.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.185+0000 7f7c8dc00700 1 --2- 172.21.15.1:0/2197201594 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5635e617b7e0 0x5635e6187ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:54.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.185+0000 7f7c8d200700 1 -- 172.21.15.1:0/2197201594 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5635e6179140 msgr2=0x5635e6173050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:54.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.185+0000 7f7c8d200700 1 --2- 172.21.15.1:0/2197201594 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5635e6179140 0x5635e6173050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:54.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.185+0000 7f7c8d200700 1 -- 172.21.15.1:0/2197201594 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5635e617b7e0 msgr2=0x5635e6187ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:54.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.185+0000 7f7c8d200700 1 --2- 172.21.15.1:0/2197201594 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5635e617b7e0 0x5635e6187ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:54.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.185+0000 7f7c8d200700 1 -- 172.21.15.1:0/2197201594 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5635e6087ea0 con 0x5635e6170010 2024-02-02T15:40:54.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.186+0000 7f7c8d200700 1 --2- 172.21.15.1:0/2197201594 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5635e6170010 0x5635e6182320 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f7c7000b6f0 tx=0x7f7c70022000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9fe7221e7f47e7e1 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:54.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.186+0000 7f7c7ea00700 1 -- 172.21.15.1:0/2197201594 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c7000bca0 con 0x5635e6170010 2024-02-02T15:40:54.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.186+0000 7f7c7ea00700 1 -- 172.21.15.1:0/2197201594 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c7001e040 con 0x5635e6170010 2024-02-02T15:40:54.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.186+0000 7f7c7ea00700 1 -- 172.21.15.1:0/2197201594 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c700048d0 con 0x5635e6170010 2024-02-02T15:40:54.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.187+0000 7f7ca369db80 1 -- 172.21.15.1:0/2197201594 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5635e6170010 msgr2=0x5635e6182320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:54.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.187+0000 7f7ca369db80 1 --2- 172.21.15.1:0/2197201594 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5635e6170010 0x5635e6182320 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f7c7000b6f0 tx=0x7f7c70022000 comp rx=0 tx=0).stop 2024-02-02T15:40:54.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.187+0000 7f7ca369db80 1 -- 172.21.15.1:0/2197201594 shutdown_connections 2024-02-02T15:40:54.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.187+0000 7f7ca369db80 1 --2- 172.21.15.1:0/2197201594 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5635e617b7e0 0x5635e6187ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:54.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.187+0000 7f7ca369db80 1 --2- 172.21.15.1:0/2197201594 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5635e6170010 0x5635e6182320 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:54.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.187+0000 7f7ca369db80 1 --2- 172.21.15.1:0/2197201594 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5635e6179140 0x5635e6173050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:54.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.187+0000 7f7ca369db80 1 -- 172.21.15.1:0/2197201594 >> 172.21.15.1:0/2197201594 conn(0x5635e60cfdb0 msgr2=0x5635e6171830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:54.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.187+0000 7f7ca369db80 1 -- 172.21.15.1:0/2197201594 shutdown_connections 2024-02-02T15:40:54.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.187+0000 7f7ca369db80 1 -- 172.21.15.1:0/2197201594 wait complete. 2024-02-02T15:40:54.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.187+0000 7f7ca369db80 1 -- start start 2024-02-02T15:40:54.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.188+0000 7f7ca369db80 1 Processor -- start 2024-02-02T15:40:54.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.188+0000 7f7ca369db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5635e6170010 0x5635e6182040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:54.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.188+0000 7f7ca369db80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5635e6179140 0x5635e6180350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:54.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.188+0000 7f7ca369db80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5635e617b7e0 0x5635e6180890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:54.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.188+0000 7f7ca369db80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e6180e20 con 0x5635e617b7e0 2024-02-02T15:40:54.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.188+0000 7f7ca369db80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e6180f90 con 0x5635e6179140 2024-02-02T15:40:54.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.188+0000 7f7ca369db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635e6181150 con 0x5635e6170010 2024-02-02T15:40:54.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.188+0000 7f7ca369db80 10 client.0 _ll_register_callbacks cb 0x5635e605f290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:40:54.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.188+0000 7f7c87e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5635e6179140 0x5635e6180350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:54.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.189+0000 7f7c8d200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5635e6170010 0x5635e6182040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:54.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.189+0000 7f7c87e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5635e6179140 0x5635e6180350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:41768/0 (socket says 172.21.15.1:41768) 2024-02-02T15:40:54.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.189+0000 7f7c87e00700 1 -- 172.21.15.1:0/2751970424 learned_addr learned my addr 172.21.15.1:0/2751970424 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:40:54.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.189+0000 7f7c8dc00700 1 --2- 172.21.15.1:0/2751970424 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5635e617b7e0 0x5635e6180890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:54.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.189+0000 7f7c8d200700 1 -- 172.21.15.1:0/2751970424 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5635e6179140 msgr2=0x5635e6180350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:54.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.189+0000 7f7c8d200700 1 --2- 172.21.15.1:0/2751970424 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5635e6179140 0x5635e6180350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:54.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.189+0000 7f7c8d200700 1 -- 172.21.15.1:0/2751970424 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5635e617b7e0 msgr2=0x5635e6180890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:54.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.189+0000 7f7c8d200700 1 --2- 172.21.15.1:0/2751970424 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5635e617b7e0 0x5635e6180890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:54.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.189+0000 7f7c8d200700 1 -- 172.21.15.1:0/2751970424 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c8810e560 con 0x5635e6170010 2024-02-02T15:40:54.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.189+0000 7f7c87e00700 1 --2- 172.21.15.1:0/2751970424 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5635e6179140 0x5635e6180350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:40:54.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.189+0000 7f7c8d200700 1 --2- 172.21.15.1:0/2751970424 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5635e6170010 0x5635e6182040 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f7c70000c00 tx=0x7f7c7000bd20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6772be299e7be6da server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:54.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.189+0000 7f7c76a00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:40:54.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.191+0000 7f7c76a00700 1 -- 172.21.15.1:0/2751970424 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c70005250 con 0x5635e6170010 2024-02-02T15:40:54.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.191+0000 7f7c76a00700 1 -- 172.21.15.1:0/2751970424 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f7c7002c040 con 0x5635e6170010 2024-02-02T15:40:54.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.191+0000 7f7ca369db80 1 -- 172.21.15.1:0/2751970424 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7c78004510 con 0x5635e6170010 2024-02-02T15:40:54.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.192+0000 7f7c76a00700 1 -- 172.21.15.1:0/2751970424 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7c700053b0 con 0x5635e6170010 2024-02-02T15:40:54.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.192+0000 7f7ca369db80 10 client.29317 fetch_fsmap learned FSMap version 59 2024-02-02T15:40:54.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.192+0000 7f7ca369db80 1 -- 172.21.15.1:0/2751970424 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) v3 -- 0x5635e5f47cd0 con 0x5635e6170010 2024-02-02T15:40:54.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.192+0000 7f7c76a00700 1 -- 172.21.15.1:0/2751970424 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c78004510 con 0x5635e6170010 2024-02-02T15:40:54.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.192+0000 7f7c76a00700 1 -- 172.21.15.1:0/2751970424 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 59) v1 ==== 7538+0+0 (secure 0 0 0) 0x7f7c7000f460 con 0x5635e6170010 2024-02-02T15:40:54.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.192+0000 7f7ca369db80 10 client.29317 fetch_fsmap finished waiting for FSMap version 59 2024-02-02T15:40:54.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.192+0000 7f7ca369db80 10 client.29317 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-02T15:40:54.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.192+0000 7f7ca369db80 20 client.29317 populate_metadata read hostname 'smithi001' 2024-02-02T15:40:54.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.192+0000 7f7ca369db80 1 --2- 172.21.15.1:0/2751970424 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x5635e61de550 0x5635e61e0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:54.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.192+0000 7f7ca369db80 4 client.29317 mds_command: new command op to 24529 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:40:54.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.192+0000 7f7ca369db80 1 -- 172.21.15.1:0/2751970424 --> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5635e6080f50 con 0x5635e61de550 2024-02-02T15:40:54.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.193+0000 7f7c87e00700 1 --2- 172.21.15.1:0/2751970424 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x5635e61de550 0x5635e61e0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:54.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.193+0000 7f7c87e00700 1 --2- 172.21.15.1:0/2751970424 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x5635e61de550 0x5635e61e0970 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b8463b63b4b4211e server_cookie=af06ba52894cd412 in_seq=0 out_seq=0 2024-02-02T15:40:54.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.193+0000 7f7c76a00700 10 client.29317 ms_handle_connect on v2:172.21.15.142:6838/321675421 2024-02-02T15:40:54.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.194+0000 7f7c76a00700 1 -- 172.21.15.1:0/2751970424 <== mds.1 v2:172.21.15.142:6838/321675421 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5635e6080f50 con 0x5635e61de550 2024-02-02T15:40:54.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.194+0000 7f7c76a00700 10 client.29317 handle_command_reply: tid=0 2024-02-02T15:40:54.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.194+0000 7f7ca369db80 1 client.29317 shutdown 2024-02-02T15:40:54.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.195+0000 7f7ca369db80 1 -- 172.21.15.1:0/2751970424 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5635e6170010 msgr2=0x5635e6182040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:54.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.195+0000 7f7ca369db80 1 --2- 172.21.15.1:0/2751970424 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5635e6170010 0x5635e6182040 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f7c70000c00 tx=0x7f7c7000bd20 comp rx=0 tx=0).stop 2024-02-02T15:40:54.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.195+0000 7f7ca369db80 1 -- 172.21.15.1:0/2751970424 shutdown_connections 2024-02-02T15:40:54.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.195+0000 7f7ca369db80 1 --2- 172.21.15.1:0/2751970424 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x5635e61de550 0x5635e61e0970 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:54.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.195+0000 7f7ca369db80 1 --2- 172.21.15.1:0/2751970424 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5635e617b7e0 0x5635e6180890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:54.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.195+0000 7f7ca369db80 1 --2- 172.21.15.1:0/2751970424 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5635e6170010 0x5635e6182040 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:54.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.195+0000 7f7ca369db80 1 --2- 172.21.15.1:0/2751970424 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5635e6179140 0x5635e6180350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:54.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.195+0000 7f7c76a00700 0 client.29317 ms_handle_reset on v2:172.21.15.142:6838/321675421 2024-02-02T15:40:54.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.195+0000 7f7ca369db80 1 -- 172.21.15.1:0/2751970424 >> 172.21.15.1:0/2751970424 conn(0x5635e60cfdb0 msgr2=0x5635e6187ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:54.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.195+0000 7f7ca369db80 1 -- 172.21.15.1:0/2751970424 shutdown_connections 2024-02-02T15:40:54.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.195+0000 7f7ca369db80 1 -- 172.21.15.1:0/2751970424 wait complete. 2024-02-02T15:40:54.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.196+0000 7f7ca369db80 20 client.29317 trim_cache size 0 max 16384 2024-02-02T15:40:54.197 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:40:54.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.197+0000 7f7c8fc00700 1 -- 172.21.15.1:0/1010894930 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f7c680a0690 msgr2=0x7f7c680a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:54.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.197+0000 7f7c8fc00700 1 --2- 172.21.15.1:0/1010894930 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f7c680a0690 0x7f7c680a2b50 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7c7000b2e0 tx=0x7f7c7000b850 comp rx=0 tx=0).stop 2024-02-02T15:40:54.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.197+0000 7f7c8fc00700 1 -- 172.21.15.1:0/1010894930 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c8810b400 msgr2=0x7f7c8806b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:54.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.197+0000 7f7c8fc00700 1 --2- 172.21.15.1:0/1010894930 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c8810b400 0x7f7c8806b100 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f7c8000c0f0 tx=0x7f7c8000c400 comp rx=0 tx=0).stop 2024-02-02T15:40:54.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.197+0000 7f7c8fc00700 1 -- 172.21.15.1:0/1010894930 shutdown_connections 2024-02-02T15:40:54.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.197+0000 7f7c8fc00700 1 --2- 172.21.15.1:0/1010894930 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f7c680a0690 0x7f7c680a2b50 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:54.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.197+0000 7f7c8fc00700 1 --2- 172.21.15.1:0/1010894930 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7c8810b400 0x7f7c8806b100 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:54.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.197+0000 7f7c8fc00700 1 --2- 172.21.15.1:0/1010894930 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7c88102030 0x7f7c8806ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:54.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.197+0000 7f7c8fc00700 1 --2- 172.21.15.1:0/1010894930 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7c88102950 0x7f7c88070450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:54.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.197+0000 7f7c8fc00700 1 -- 172.21.15.1:0/1010894930 >> 172.21.15.1:0/1010894930 conn(0x7f7c88076d70 msgr2=0x7f7c880fcb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:54.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.199+0000 7f7c8fc00700 1 -- 172.21.15.1:0/1010894930 shutdown_connections 2024-02-02T15:40:54.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.199+0000 7f7c8fc00700 1 -- 172.21.15.1:0/1010894930 wait complete. 2024-02-02T15:40:54.221 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-02T15:40:54.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.501+0000 7ffb97e00700 1 -- 172.21.15.1:0/3976621270 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffb90108660 msgr2=0x7ffb90113950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:54.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.501+0000 7ffb97e00700 1 --2- 172.21.15.1:0/3976621270 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffb90108660 0x7ffb90113950 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7ffb780098f0 tx=0x7ffb78009c00 comp rx=0 tx=0).stop 2024-02-02T15:40:54.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.502+0000 7ffb97e00700 1 -- 172.21.15.1:0/3976621270 shutdown_connections 2024-02-02T15:40:54.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.502+0000 7ffb97e00700 1 --2- 172.21.15.1:0/3976621270 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb90113e90 0x7ffb90116280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:54.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.502+0000 7ffb97e00700 1 --2- 172.21.15.1:0/3976621270 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb90107d40 0x7ffb90108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:54.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.502+0000 7ffb97e00700 1 --2- 172.21.15.1:0/3976621270 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffb90108660 0x7ffb90113950 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:54.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.502+0000 7ffb97e00700 1 -- 172.21.15.1:0/3976621270 >> 172.21.15.1:0/3976621270 conn(0x7ffb9006d120 msgr2=0x7ffb9006d530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:54.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.502+0000 7ffb97e00700 1 -- 172.21.15.1:0/3976621270 shutdown_connections 2024-02-02T15:40:54.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.502+0000 7ffb97e00700 1 -- 172.21.15.1:0/3976621270 wait complete. 2024-02-02T15:40:54.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.504+0000 7ffb97e00700 1 Processor -- start 2024-02-02T15:40:54.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.504+0000 7ffb97e00700 1 -- start start 2024-02-02T15:40:54.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.505+0000 7ffb97e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb90107d40 0x7ffb901a7720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:54.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.505+0000 7ffb97e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb90108660 0x7ffb901a7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:54.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.505+0000 7ffb97e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffb90113e90 0x7ffb901a1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:54.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.505+0000 7ffb97e00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb901a1e00 con 0x7ffb90108660 2024-02-02T15:40:54.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.505+0000 7ffb97e00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb901a1f70 con 0x7ffb90113e90 2024-02-02T15:40:54.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.505+0000 7ffb97e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb901a2130 con 0x7ffb90107d40 2024-02-02T15:40:54.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.505+0000 7ffb95400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb90107d40 0x7ffb901a7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:54.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.505+0000 7ffb95400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb90107d40 0x7ffb901a7720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:37616/0 (socket says 172.21.15.1:37616) 2024-02-02T15:40:54.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.505+0000 7ffb95400700 1 -- 172.21.15.1:0/1123136563 learned_addr learned my addr 172.21.15.1:0/1123136563 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:40:54.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.506+0000 7ffb95e00700 1 --2- 172.21.15.1:0/1123136563 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffb90113e90 0x7ffb901a1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:54.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.506+0000 7ffb94a00700 1 --2- 172.21.15.1:0/1123136563 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb90108660 0x7ffb901a7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:54.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.507+0000 7ffb95e00700 1 -- 172.21.15.1:0/1123136563 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb90107d40 msgr2=0x7ffb901a7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:54.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.507+0000 7ffb95e00700 1 --2- 172.21.15.1:0/1123136563 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb90107d40 0x7ffb901a7720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:54.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.507+0000 7ffb95e00700 1 -- 172.21.15.1:0/1123136563 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb90108660 msgr2=0x7ffb901a7c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:54.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.507+0000 7ffb95e00700 1 --2- 172.21.15.1:0/1123136563 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb90108660 0x7ffb901a7c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:54.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.507+0000 7ffb95e00700 1 -- 172.21.15.1:0/1123136563 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb78009520 con 0x7ffb90113e90 2024-02-02T15:40:54.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.507+0000 7ffb94a00700 1 --2- 172.21.15.1:0/1123136563 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb90108660 0x7ffb901a7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:40:54.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.507+0000 7ffb95400700 1 --2- 172.21.15.1:0/1123136563 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb90107d40 0x7ffb901a7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:40:54.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.507+0000 7ffb95e00700 1 --2- 172.21.15.1:0/1123136563 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffb90113e90 0x7ffb901a1790 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7ffb8800ef90 tx=0x7ffb8800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:54.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.508+0000 7ffb8e000700 1 -- 172.21.15.1:0/1123136563 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb88018070 con 0x7ffb90113e90 2024-02-02T15:40:54.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.508+0000 7ffb8e000700 1 -- 172.21.15.1:0/1123136563 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffb8800f040 con 0x7ffb90113e90 2024-02-02T15:40:54.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.508+0000 7ffb8e000700 1 -- 172.21.15.1:0/1123136563 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb8800ccd0 con 0x7ffb90113e90 2024-02-02T15:40:54.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.508+0000 7ffb97e00700 1 -- 172.21.15.1:0/1123136563 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb901a23b0 con 0x7ffb90113e90 2024-02-02T15:40:54.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.508+0000 7ffb97e00700 1 -- 172.21.15.1:0/1123136563 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb901ae710 con 0x7ffb90113e90 2024-02-02T15:40:54.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.512+0000 7ffb8e000700 1 -- 172.21.15.1:0/1123136563 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7ffb880075c0 con 0x7ffb90113e90 2024-02-02T15:40:54.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.515+0000 7ffb8e000700 1 --2- 172.21.15.1:0/1123136563 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ffb700a0710 0x7ffb700a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:54.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.516+0000 7ffb8e000700 1 -- 172.21.15.1:0/1123136563 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9453+0+0 (secure 0 0 0) 0x7ffb880cbb60 con 0x7ffb90113e90 2024-02-02T15:40:54.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.516+0000 7ffb95400700 1 --2- 172.21.15.1:0/1123136563 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ffb700a0710 0x7ffb700a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:54.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.518+0000 7ffbab8b0b80 1 Processor -- start 2024-02-02T15:40:54.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.518+0000 7ffb95400700 1 --2- 172.21.15.1:0/1123136563 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ffb700a0710 0x7ffb700a2bd0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ffb80005950 tx=0x7ffb8000b500 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:54.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.519+0000 7ffbab8b0b80 1 -- start start 2024-02-02T15:40:54.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.519+0000 7ffbab8b0b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ab0bd94010 0x55ab0bd943f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:54.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.520+0000 7ffbab8b0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab0bd94930 0x55ab0bda7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:54.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.520+0000 7ffbab8b0b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ab0bda7ff0 0x55ab0bdaa3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:54.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.520+0000 7ffbab8b0b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab0bd084c0 con 0x55ab0bd94010 2024-02-02T15:40:54.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.520+0000 7ffbab8b0b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab0bcecaa0 con 0x55ab0bda7ff0 2024-02-02T15:40:54.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.520+0000 7ffbab8b0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab0bcea460 con 0x55ab0bd94930 2024-02-02T15:40:54.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.521+0000 7ffb95e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ab0bda7ff0 0x55ab0bdaa3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:54.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.521+0000 7ffb95e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ab0bda7ff0 0x55ab0bdaa3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:41808/0 (socket says 172.21.15.1:41808) 2024-02-02T15:40:54.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.521+0000 7ffb95e00700 1 -- 172.21.15.1:0/77831922 learned_addr learned my addr 172.21.15.1:0/77831922 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:40:54.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.521+0000 7ffb95400700 1 --2- 172.21.15.1:0/77831922 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ab0bd94010 0x55ab0bd943f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:54.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.522+0000 7ffb95400700 1 -- 172.21.15.1:0/77831922 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab0bd94930 msgr2=0x55ab0bda7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:54.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.522+0000 7ffb95400700 1 --2- 172.21.15.1:0/77831922 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab0bd94930 0x55ab0bda7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:54.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.522+0000 7ffb95400700 1 -- 172.21.15.1:0/77831922 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ab0bda7ff0 msgr2=0x55ab0bdaa3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:54.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.522+0000 7ffb95400700 1 --2- 172.21.15.1:0/77831922 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ab0bda7ff0 0x55ab0bdaa3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:54.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.522+0000 7ffb95400700 1 -- 172.21.15.1:0/77831922 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab0bb673b0 con 0x55ab0bd94010 2024-02-02T15:40:54.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.522+0000 7ffb95e00700 1 --2- 172.21.15.1:0/77831922 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ab0bda7ff0 0x55ab0bdaa3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:40:54.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.523+0000 7ffb95400700 1 --2- 172.21.15.1:0/77831922 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ab0bd94010 0x55ab0bd943f0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7ffb80011760 tx=0x7ffb80013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34ac77327ada056d server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:54.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.524+0000 7ffb86600700 1 -- 172.21.15.1:0/77831922 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb80004220 con 0x55ab0bd94010 2024-02-02T15:40:54.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.524+0000 7ffb86600700 1 -- 172.21.15.1:0/77831922 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffb80015070 con 0x55ab0bd94010 2024-02-02T15:40:54.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.525+0000 7ffbab8b0b80 1 -- 172.21.15.1:0/77831922 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ab0bd94010 msgr2=0x55ab0bd943f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:54.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.526+0000 7ffbab8b0b80 1 --2- 172.21.15.1:0/77831922 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ab0bd94010 0x55ab0bd943f0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7ffb80011760 tx=0x7ffb80013000 comp rx=0 tx=0).stop 2024-02-02T15:40:54.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.526+0000 7ffbab8b0b80 1 -- 172.21.15.1:0/77831922 shutdown_connections 2024-02-02T15:40:54.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.526+0000 7ffbab8b0b80 1 --2- 172.21.15.1:0/77831922 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ab0bda7ff0 0x55ab0bdaa3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:54.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.526+0000 7ffbab8b0b80 1 --2- 172.21.15.1:0/77831922 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ab0bd94010 0x55ab0bd943f0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:54.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.526+0000 7ffbab8b0b80 1 --2- 172.21.15.1:0/77831922 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab0bd94930 0x55ab0bda7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:54.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.526+0000 7ffbab8b0b80 1 -- 172.21.15.1:0/77831922 >> 172.21.15.1:0/77831922 conn(0x55ab0bcf3db0 msgr2=0x55ab0bd903b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:54.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.527+0000 7ffbab8b0b80 1 -- 172.21.15.1:0/77831922 shutdown_connections 2024-02-02T15:40:54.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.527+0000 7ffbab8b0b80 1 -- 172.21.15.1:0/77831922 wait complete. 2024-02-02T15:40:54.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.528+0000 7ffbab8b0b80 1 -- start start 2024-02-02T15:40:54.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.528+0000 7ffbab8b0b80 1 Processor -- start 2024-02-02T15:40:54.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.529+0000 7ffbab8b0b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ab0bd94010 0x55ab0bda7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:54.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.529+0000 7ffbab8b0b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ab0bd94930 0x55ab0bda4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:54.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.529+0000 7ffbab8b0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab0bda7ff0 0x55ab0bda5010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:54.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.529+0000 7ffbab8b0b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab0bce5d50 con 0x55ab0bd94010 2024-02-02T15:40:54.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.529+0000 7ffbab8b0b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab0bc81d80 con 0x55ab0bd94930 2024-02-02T15:40:54.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.529+0000 7ffbab8b0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab0bc81a20 con 0x55ab0bda7ff0 2024-02-02T15:40:54.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.529+0000 7ffb95e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab0bda7ff0 0x55ab0bda5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:54.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.529+0000 7ffb95e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab0bda7ff0 0x55ab0bda5010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:37640/0 (socket says 172.21.15.1:37640) 2024-02-02T15:40:54.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.530+0000 7ffb95e00700 1 -- 172.21.15.1:0/4222461303 learned_addr learned my addr 172.21.15.1:0/4222461303 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:40:54.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.530+0000 7ffb94a00700 1 --2- 172.21.15.1:0/4222461303 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ab0bd94930 0x55ab0bda4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:54.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.530+0000 7ffbab8b0b80 10 client.0 _ll_register_callbacks cb 0x55ab0bc83290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:40:54.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.530+0000 7ffb95e00700 1 -- 172.21.15.1:0/4222461303 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ab0bd94930 msgr2=0x55ab0bda4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:54.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.530+0000 7ffb95e00700 1 --2- 172.21.15.1:0/4222461303 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ab0bd94930 0x55ab0bda4ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:54.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.530+0000 7ffb95e00700 1 -- 172.21.15.1:0/4222461303 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ab0bd94010 msgr2=0x55ab0bda7850 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-02T15:40:54.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.530+0000 7ffb95400700 1 --2- 172.21.15.1:0/4222461303 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ab0bd94010 0x55ab0bda7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:54.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.530+0000 7ffb95e00700 1 --2- 172.21.15.1:0/4222461303 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ab0bd94010 0x55ab0bda7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:54.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.531+0000 7ffb95e00700 1 -- 172.21.15.1:0/4222461303 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb80011440 con 0x55ab0bda7ff0 2024-02-02T15:40:54.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.531+0000 7ffb95400700 1 --2- 172.21.15.1:0/4222461303 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ab0bd94010 0x55ab0bda7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:40:54.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.533+0000 7ffb95e00700 1 --2- 172.21.15.1:0/4222461303 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab0bda7ff0 0x55ab0bda5010 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7ffb88099990 tx=0x7ffb880b8df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c2453e0396e8c1a server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:54.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.533+0000 7ffb7ea00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:40:54.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.535+0000 7ffb7ea00700 1 -- 172.21.15.1:0/4222461303 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb8809a0f0 con 0x55ab0bda7ff0 2024-02-02T15:40:54.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.535+0000 7ffb7ea00700 1 -- 172.21.15.1:0/4222461303 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9453+0+0 (secure 0 0 0) 0x7ffb8809dac0 con 0x55ab0bda7ff0 2024-02-02T15:40:54.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.536+0000 7ffbab8b0b80 1 -- 172.21.15.1:0/4222461303 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ab0bd880f0 con 0x55ab0bda7ff0 2024-02-02T15:40:54.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.537+0000 7ffb7ea00700 1 -- 172.21.15.1:0/4222461303 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb880a1c70 con 0x55ab0bda7ff0 2024-02-02T15:40:54.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.537+0000 7ffbab8b0b80 10 client.29326 fetch_fsmap learned FSMap version 59 2024-02-02T15:40:54.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.537+0000 7ffbab8b0b80 1 -- 172.21.15.1:0/4222461303 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) v3 -- 0x55ab0bb13b60 con 0x55ab0bda7ff0 2024-02-02T15:40:54.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.541+0000 7ffb7ea00700 1 -- 172.21.15.1:0/4222461303 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ab0bd880f0 con 0x55ab0bda7ff0 2024-02-02T15:40:54.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.541+0000 7ffb7ea00700 1 -- 172.21.15.1:0/4222461303 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 59) v1 ==== 7538+0+0 (secure 0 0 0) 0x7ffb880a18f0 con 0x55ab0bda7ff0 2024-02-02T15:40:54.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.541+0000 7ffbab8b0b80 10 client.29326 fetch_fsmap finished waiting for FSMap version 59 2024-02-02T15:40:54.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.541+0000 7ffbab8b0b80 10 client.29326 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:40:54.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.541+0000 7ffbab8b0b80 20 client.29326 populate_metadata read hostname 'smithi001' 2024-02-02T15:40:54.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.541+0000 7ffbab8b0b80 1 --2- 172.21.15.1:0/4222461303 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55ab0bdb7a20 0x55ab0bdb9e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:54.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.541+0000 7ffbab8b0b80 4 client.29326 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:40:54.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.541+0000 7ffbab8b0b80 1 -- 172.21.15.1:0/4222461303 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ab0bcdedd0 con 0x55ab0bdb7a20 2024-02-02T15:40:54.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.542+0000 7ffb95400700 1 --2- 172.21.15.1:0/4222461303 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55ab0bdb7a20 0x55ab0bdb9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:54.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.543+0000 7ffb95400700 1 --2- 172.21.15.1:0/4222461303 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55ab0bdb7a20 0x55ab0bdb9e40 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73e513be3953efb1 server_cookie=6a6f4717b58d48b6 in_seq=0 out_seq=0 2024-02-02T15:40:54.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.543+0000 7ffb7ea00700 10 client.29326 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:40:54.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.549+0000 7ffb7ea00700 1 -- 172.21.15.1:0/4222461303 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55ab0bcdedd0 con 0x55ab0bdb7a20 2024-02-02T15:40:54.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.549+0000 7ffb7ea00700 10 client.29326 handle_command_reply: tid=0 2024-02-02T15:40:54.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.550+0000 7ffbab8b0b80 1 client.29326 shutdown 2024-02-02T15:40:54.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.550+0000 7ffbab8b0b80 1 -- 172.21.15.1:0/4222461303 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab0bda7ff0 msgr2=0x55ab0bda5010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:54.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.550+0000 7ffbab8b0b80 1 --2- 172.21.15.1:0/4222461303 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab0bda7ff0 0x55ab0bda5010 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7ffb88099990 tx=0x7ffb880b8df0 comp rx=0 tx=0).stop 2024-02-02T15:40:54.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.551+0000 7ffbab8b0b80 1 -- 172.21.15.1:0/4222461303 shutdown_connections 2024-02-02T15:40:54.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.551+0000 7ffbab8b0b80 1 --2- 172.21.15.1:0/4222461303 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55ab0bdb7a20 0x55ab0bdb9e40 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:54.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.551+0000 7ffbab8b0b80 1 --2- 172.21.15.1:0/4222461303 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab0bda7ff0 0x55ab0bda5010 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:54.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.551+0000 7ffbab8b0b80 1 --2- 172.21.15.1:0/4222461303 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ab0bd94010 0x55ab0bda7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:54.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.551+0000 7ffbab8b0b80 1 --2- 172.21.15.1:0/4222461303 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ab0bd94930 0x55ab0bda4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:54.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.551+0000 7ffbab8b0b80 1 -- 172.21.15.1:0/4222461303 >> 172.21.15.1:0/4222461303 conn(0x55ab0bcf3db0 msgr2=0x55ab0bd8ffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:54.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.551+0000 7ffb7ea00700 0 client.29326 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:40:54.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.552+0000 7ffbab8b0b80 1 -- 172.21.15.1:0/4222461303 shutdown_connections 2024-02-02T15:40:54.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.552+0000 7ffbab8b0b80 1 -- 172.21.15.1:0/4222461303 wait complete. 2024-02-02T15:40:54.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.553+0000 7ffbab8b0b80 20 client.29326 trim_cache size 0 max 16384 2024-02-02T15:40:54.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.576+0000 7ffbab8b0b80 1 Processor -- start 2024-02-02T15:40:54.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.576+0000 7ffbab8b0b80 1 -- start start 2024-02-02T15:40:54.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.577+0000 7ffbab8b0b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ab0bd94010 0x55ab0bdaec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:54.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.577+0000 7ffbab8b0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab0bd94930 0x55ab0bda73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:54.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.578+0000 7ffbab8b0b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ab0bda7ff0 0x55ab0bd8fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:54.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.578+0000 7ffbab8b0b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab0bdb44a0 con 0x55ab0bda7ff0 2024-02-02T15:40:54.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.578+0000 7ffbab8b0b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab0bb696a0 con 0x55ab0bd94010 2024-02-02T15:40:54.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.578+0000 7ffbab8b0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab0bd94d10 con 0x55ab0bd94930 2024-02-02T15:40:54.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.578+0000 7ffb95400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ab0bd94010 0x55ab0bdaec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:54.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.578+0000 7ffb95400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ab0bd94010 0x55ab0bdaec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:41826/0 (socket says 172.21.15.1:41826) 2024-02-02T15:40:54.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.578+0000 7ffb95400700 1 -- 172.21.15.1:0/4290063038 learned_addr learned my addr 172.21.15.1:0/4290063038 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:40:54.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.578+0000 7ffb94a00700 1 --2- 172.21.15.1:0/4290063038 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab0bd94930 0x55ab0bda73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:54.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.578+0000 7ffb95e00700 1 --2- 172.21.15.1:0/4290063038 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ab0bda7ff0 0x55ab0bd8fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:54.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.578+0000 7ffb95400700 1 -- 172.21.15.1:0/4290063038 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab0bd94930 msgr2=0x55ab0bda73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:54.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.578+0000 7ffb95400700 1 --2- 172.21.15.1:0/4290063038 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab0bd94930 0x55ab0bda73f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:54.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.578+0000 7ffb95400700 1 -- 172.21.15.1:0/4290063038 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ab0bda7ff0 msgr2=0x55ab0bd8fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:54.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.578+0000 7ffb95400700 1 --2- 172.21.15.1:0/4290063038 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ab0bda7ff0 0x55ab0bd8fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:54.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.578+0000 7ffb95400700 1 -- 172.21.15.1:0/4290063038 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ab0bd8c0d0 con 0x55ab0bd94010 2024-02-02T15:40:54.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.579+0000 7ffb95400700 1 --2- 172.21.15.1:0/4290063038 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ab0bd94010 0x55ab0bdaec60 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7ffb80003ee0 tx=0x7ffb80003f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89b019e9ea2c946b server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:54.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.579+0000 7ffb87000700 1 -- 172.21.15.1:0/4290063038 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb8002d040 con 0x55ab0bd94010 2024-02-02T15:40:54.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.579+0000 7ffb87000700 1 -- 172.21.15.1:0/4290063038 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffb80032420 con 0x55ab0bd94010 2024-02-02T15:40:54.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.579+0000 7ffb87000700 1 -- 172.21.15.1:0/4290063038 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb8002d040 con 0x55ab0bd94010 2024-02-02T15:40:54.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.580+0000 7ffbab8b0b80 1 -- 172.21.15.1:0/4290063038 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ab0bd94010 msgr2=0x55ab0bdaec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:54.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.580+0000 7ffbab8b0b80 1 --2- 172.21.15.1:0/4290063038 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ab0bd94010 0x55ab0bdaec60 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7ffb80003ee0 tx=0x7ffb80003f10 comp rx=0 tx=0).stop 2024-02-02T15:40:54.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.581+0000 7ffbab8b0b80 1 -- 172.21.15.1:0/4290063038 shutdown_connections 2024-02-02T15:40:54.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.581+0000 7ffbab8b0b80 1 --2- 172.21.15.1:0/4290063038 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ab0bda7ff0 0x55ab0bd8fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:54.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.581+0000 7ffbab8b0b80 1 --2- 172.21.15.1:0/4290063038 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ab0bd94010 0x55ab0bdaec60 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:54.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.581+0000 7ffbab8b0b80 1 --2- 172.21.15.1:0/4290063038 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab0bd94930 0x55ab0bda73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:54.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.581+0000 7ffbab8b0b80 1 -- 172.21.15.1:0/4290063038 >> 172.21.15.1:0/4290063038 conn(0x55ab0bcf3db0 msgr2=0x55ab0bd8f5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:54.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.582+0000 7ffbab8b0b80 1 -- 172.21.15.1:0/4290063038 shutdown_connections 2024-02-02T15:40:54.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.582+0000 7ffbab8b0b80 1 -- 172.21.15.1:0/4290063038 wait complete. 2024-02-02T15:40:54.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.583+0000 7ffbab8b0b80 1 -- start start 2024-02-02T15:40:54.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.583+0000 7ffbab8b0b80 1 Processor -- start 2024-02-02T15:40:54.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.584+0000 7ffbab8b0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab0bd94010 0x55ab0bdfd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:54.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.584+0000 7ffbab8b0b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ab0bd94930 0x55ab0bdfdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:54.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.584+0000 7ffbab8b0b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ab0bda7ff0 0x55ab0be08550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:54.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.584+0000 7ffbab8b0b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab0be08a90 con 0x55ab0bd94930 2024-02-02T15:40:54.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.584+0000 7ffbab8b0b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab0be08bd0 con 0x55ab0bda7ff0 2024-02-02T15:40:54.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.584+0000 7ffbab8b0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ab0be08d90 con 0x55ab0bd94010 2024-02-02T15:40:54.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.584+0000 7ffbab8b0b80 10 client.0 _ll_register_callbacks cb 0x55ab0bc83290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:40:54.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.584+0000 7ffb95400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab0bd94010 0x55ab0bdfd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:54.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.584+0000 7ffb95e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ab0bda7ff0 0x55ab0be08550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:54.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.584+0000 7ffb95400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab0bd94010 0x55ab0bdfd6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:37666/0 (socket says 172.21.15.1:37666) 2024-02-02T15:40:54.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.584+0000 7ffb95e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ab0bda7ff0 0x55ab0be08550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:41834/0 (socket says 172.21.15.1:41834) 2024-02-02T15:40:54.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.584+0000 7ffb95400700 1 -- 172.21.15.1:0/3861555951 learned_addr learned my addr 172.21.15.1:0/3861555951 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:40:54.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.584+0000 7ffb94a00700 1 --2- 172.21.15.1:0/3861555951 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ab0bd94930 0x55ab0bdfdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:54.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.585+0000 7ffb95e00700 1 -- 172.21.15.1:0/3861555951 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab0bd94010 msgr2=0x55ab0bdfd6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:54.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.585+0000 7ffb95e00700 1 --2- 172.21.15.1:0/3861555951 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab0bd94010 0x55ab0bdfd6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:54.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.585+0000 7ffb95e00700 1 -- 172.21.15.1:0/3861555951 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ab0bd94930 msgr2=0x55ab0bdfdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:54.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.585+0000 7ffb95e00700 1 --2- 172.21.15.1:0/3861555951 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ab0bd94930 0x55ab0bdfdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:54.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.585+0000 7ffb95e00700 1 -- 172.21.15.1:0/3861555951 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ab0bb673b0 con 0x55ab0bda7ff0 2024-02-02T15:40:54.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.585+0000 7ffb94a00700 1 --2- 172.21.15.1:0/3861555951 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ab0bd94930 0x55ab0bdfdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:40:54.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.585+0000 7ffb95400700 1 --2- 172.21.15.1:0/3861555951 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab0bd94010 0x55ab0bdfd6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:40:54.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.585+0000 7ffb95e00700 1 --2- 172.21.15.1:0/3861555951 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ab0bda7ff0 0x55ab0be08550 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7ffb880090a0 tx=0x7ffb880993b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fff20dbcd903884e server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:54.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.585+0000 7ffb7ea00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:40:54.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.586+0000 7ffb7ea00700 1 -- 172.21.15.1:0/3861555951 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb88098db0 con 0x55ab0bda7ff0 2024-02-02T15:40:54.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.586+0000 7ffb7ea00700 1 -- 172.21.15.1:0/3861555951 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9453+0+0 (secure 0 0 0) 0x7ffb8809dac0 con 0x55ab0bda7ff0 2024-02-02T15:40:54.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.586+0000 7ffbab8b0b80 1 -- 172.21.15.1:0/3861555951 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffb80015070 con 0x55ab0bda7ff0 2024-02-02T15:40:54.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.586+0000 7ffb7ea00700 1 -- 172.21.15.1:0/3861555951 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb88098fa0 con 0x55ab0bda7ff0 2024-02-02T15:40:54.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.587+0000 7ffbab8b0b80 10 client.29348 fetch_fsmap learned FSMap version 59 2024-02-02T15:40:54.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.587+0000 7ffbab8b0b80 1 -- 172.21.15.1:0/3861555951 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) v3 -- 0x55ab0bcabea0 con 0x55ab0bda7ff0 2024-02-02T15:40:54.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.587+0000 7ffb7ea00700 1 -- 172.21.15.1:0/3861555951 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffb80015070 con 0x55ab0bda7ff0 2024-02-02T15:40:54.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.587+0000 7ffb7ea00700 1 -- 172.21.15.1:0/3861555951 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 59) v1 ==== 7538+0+0 (secure 0 0 0) 0x7ffb88095a80 con 0x55ab0bda7ff0 2024-02-02T15:40:54.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.587+0000 7ffbab8b0b80 10 client.29348 fetch_fsmap finished waiting for FSMap version 59 2024-02-02T15:40:54.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.587+0000 7ffbab8b0b80 10 client.29348 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:40:54.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.587+0000 7ffbab8b0b80 20 client.29348 populate_metadata read hostname 'smithi001' 2024-02-02T15:40:54.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.587+0000 7ffbab8b0b80 1 --2- 172.21.15.1:0/3861555951 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55ab0be0a180 0x55ab0be0c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:54.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.587+0000 7ffbab8b0b80 4 client.29348 mds_command: new command op to 24557 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:40:54.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.587+0000 7ffbab8b0b80 1 -- 172.21.15.1:0/3861555951 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ab0bca4f50 con 0x55ab0be0a180 2024-02-02T15:40:54.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.588+0000 7ffb95400700 1 --2- 172.21.15.1:0/3861555951 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55ab0be0a180 0x55ab0be0c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:54.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.588+0000 7ffb7ea00700 10 client.29348 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:40:54.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.588+0000 7ffb95400700 1 --2- 172.21.15.1:0/3861555951 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55ab0be0a180 0x55ab0be0c5a0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb5f65cafaad1b11 server_cookie=201aed2e8f27ff2c in_seq=0 out_seq=0 2024-02-02T15:40:54.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.588+0000 7ffb7ea00700 1 -- 172.21.15.1:0/3861555951 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ab0bca4f50 con 0x55ab0be0a180 2024-02-02T15:40:54.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.588+0000 7ffb7ea00700 10 client.29348 handle_command_reply: tid=0 2024-02-02T15:40:54.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.589+0000 7ffbab8b0b80 1 client.29348 shutdown 2024-02-02T15:40:54.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.589+0000 7ffbab8b0b80 1 -- 172.21.15.1:0/3861555951 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ab0bda7ff0 msgr2=0x55ab0be08550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:54.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.589+0000 7ffbab8b0b80 1 --2- 172.21.15.1:0/3861555951 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ab0bda7ff0 0x55ab0be08550 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7ffb880090a0 tx=0x7ffb880993b0 comp rx=0 tx=0).stop 2024-02-02T15:40:54.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.589+0000 7ffbab8b0b80 1 -- 172.21.15.1:0/3861555951 shutdown_connections 2024-02-02T15:40:54.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.590+0000 7ffbab8b0b80 1 --2- 172.21.15.1:0/3861555951 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55ab0be0a180 0x55ab0be0c5a0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:54.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.590+0000 7ffbab8b0b80 1 --2- 172.21.15.1:0/3861555951 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ab0bda7ff0 0x55ab0be08550 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:54.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.590+0000 7ffbab8b0b80 1 --2- 172.21.15.1:0/3861555951 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ab0bd94010 0x55ab0bdfd6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:54.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.590+0000 7ffbab8b0b80 1 --2- 172.21.15.1:0/3861555951 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ab0bd94930 0x55ab0bdfdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:54.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.590+0000 7ffbab8b0b80 1 -- 172.21.15.1:0/3861555951 >> 172.21.15.1:0/3861555951 conn(0x55ab0bcf3db0 msgr2=0x55ab0bd8fe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:54.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.590+0000 7ffb7ea00700 0 client.29348 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:40:54.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.591+0000 7ffbab8b0b80 1 -- 172.21.15.1:0/3861555951 shutdown_connections 2024-02-02T15:40:54.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.591+0000 7ffbab8b0b80 1 -- 172.21.15.1:0/3861555951 wait complete. 2024-02-02T15:40:54.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.592+0000 7ffbab8b0b80 20 client.29348 trim_cache size 0 max 16384 2024-02-02T15:40:54.593 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:40:54.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.592+0000 7ffb97e00700 1 -- 172.21.15.1:0/1123136563 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ffb700a0710 msgr2=0x7ffb700a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:54.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.592+0000 7ffb97e00700 1 --2- 172.21.15.1:0/1123136563 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ffb700a0710 0x7ffb700a2bd0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ffb80005950 tx=0x7ffb8000b500 comp rx=0 tx=0).stop 2024-02-02T15:40:54.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.592+0000 7ffb97e00700 1 -- 172.21.15.1:0/1123136563 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffb90113e90 msgr2=0x7ffb901a1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:54.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.592+0000 7ffb97e00700 1 --2- 172.21.15.1:0/1123136563 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffb90113e90 0x7ffb901a1790 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7ffb8800ef90 tx=0x7ffb8800c3a0 comp rx=0 tx=0).stop 2024-02-02T15:40:54.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.593+0000 7ffb97e00700 1 -- 172.21.15.1:0/1123136563 shutdown_connections 2024-02-02T15:40:54.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.593+0000 7ffb97e00700 1 --2- 172.21.15.1:0/1123136563 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ffb700a0710 0x7ffb700a2bd0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:54.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.593+0000 7ffb97e00700 1 --2- 172.21.15.1:0/1123136563 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffb90113e90 0x7ffb901a1790 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:54.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.593+0000 7ffb97e00700 1 --2- 172.21.15.1:0/1123136563 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ffb90107d40 0x7ffb901a7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:54.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.593+0000 7ffb97e00700 1 --2- 172.21.15.1:0/1123136563 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ffb90108660 0x7ffb901a7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:54.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.593+0000 7ffb97e00700 1 -- 172.21.15.1:0/1123136563 >> 172.21.15.1:0/1123136563 conn(0x7ffb9006d120 msgr2=0x7ffb901096e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:54.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.593+0000 7ffb97e00700 1 -- 172.21.15.1:0/1123136563 shutdown_connections 2024-02-02T15:40:54.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:54.594+0000 7ffb97e00700 1 -- 172.21.15.1:0/1123136563 wait complete. 2024-02-02T15:40:54.968 INFO:tasks.workunit.client.0.smithi001.stdout: 10 11437 29.61 MB/sec warmup 27 sec latency 1168.997 ms 2024-02-02T15:40:55.617 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-02T15:40:55.617 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-02T15:40:55.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:55 smithi071 ceph-mon[44707]: pgmap v434: 129 pgs: 129 active+clean; 885 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.1 MiB/s wr, 235 op/s 2024-02-02T15:40:55.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:55 smithi142 ceph-mon[45728]: pgmap v434: 129 pgs: 129 active+clean; 885 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.1 MiB/s wr, 235 op/s 2024-02-02T15:40:55.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:55 smithi001 ceph-mon[35470]: pgmap v434: 129 pgs: 129 active+clean; 885 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 3.1 MiB/s wr, 235 op/s 2024-02-02T15:40:55.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:55.963+0000 7f9a2e000700 1 -- 172.21.15.1:0/1432842388 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9a28101000 msgr2=0x7f9a281013e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:55.963 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:55.963+0000 7f9a2e000700 1 --2- 172.21.15.1:0/1432842388 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9a28101000 0x7f9a281013e0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f9a18009990 tx=0x7f9a18009ca0 comp rx=0 tx=0).stop 2024-02-02T15:40:55.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:55.964+0000 7f9a2e000700 1 -- 172.21.15.1:0/1432842388 shutdown_connections 2024-02-02T15:40:55.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:55.964+0000 7f9a2e000700 1 --2- 172.21.15.1:0/1432842388 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a28106120 0x7f9a2810dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:55.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:55.964+0000 7f9a2e000700 1 --2- 172.21.15.1:0/1432842388 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9a28101000 0x7f9a281013e0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:55.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:55.964+0000 7f9a2e000700 1 --2- 172.21.15.1:0/1432842388 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a281019b0 0x7f9a281059f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:55.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:55.964+0000 7f9a2e000700 1 -- 172.21.15.1:0/1432842388 >> 172.21.15.1:0/1432842388 conn(0x7f9a280fc840 msgr2=0x7f9a280fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:55.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:55.964+0000 7f9a2e000700 1 -- 172.21.15.1:0/1432842388 shutdown_connections 2024-02-02T15:40:55.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:55.964+0000 7f9a2e000700 1 -- 172.21.15.1:0/1432842388 wait complete. 2024-02-02T15:40:55.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:55.966+0000 7f9a2e000700 1 Processor -- start 2024-02-02T15:40:55.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:55.968+0000 7f9a2e000700 1 -- start start 2024-02-02T15:40:55.968 INFO:tasks.workunit.client.0.smithi001.stdout: 10 12018 29.31 MB/sec warmup 28 sec latency 55.147 ms 2024-02-02T15:40:55.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:55.968+0000 7f9a2e000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a28101000 0x7f9a2806ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:55.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:55.968+0000 7f9a2e000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a28106120 0x7f9a280704a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:55.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:55.969+0000 7f9a2e000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9a2806b150 0x7f9a2806b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:55.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:55.969+0000 7f9a2e000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a2806bb70 con 0x7f9a28101000 2024-02-02T15:40:55.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:55.969+0000 7f9a2e000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a2806bce0 con 0x7f9a2806b150 2024-02-02T15:40:55.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:55.969+0000 7f9a2e000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a2806bea0 con 0x7f9a28106120 2024-02-02T15:40:55.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:55.969+0000 7f9a26a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a28106120 0x7f9a280704a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:55.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:55.969+0000 7f9a26a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a28106120 0x7f9a280704a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:37694/0 (socket says 172.21.15.1:37694) 2024-02-02T15:40:55.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:55.969+0000 7f9a26a00700 1 -- 172.21.15.1:0/1515713029 learned_addr learned my addr 172.21.15.1:0/1515713029 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:40:55.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:55.969+0000 7f9a27e00700 1 --2- 172.21.15.1:0/1515713029 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9a2806b150 0x7f9a2806b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:55.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:55.969+0000 7f9a26a00700 1 -- 172.21.15.1:0/1515713029 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9a2806b150 msgr2=0x7f9a2806b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:55.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:55.969+0000 7f9a26a00700 1 --2- 172.21.15.1:0/1515713029 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9a2806b150 0x7f9a2806b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:55.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:55.969+0000 7f9a26a00700 1 -- 172.21.15.1:0/1515713029 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a28101000 msgr2=0x7f9a2806ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:55.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:55.969+0000 7f9a26a00700 1 --2- 172.21.15.1:0/1515713029 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a28101000 0x7f9a2806ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:55.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:55.969+0000 7f9a26a00700 1 -- 172.21.15.1:0/1515713029 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a180095f0 con 0x7f9a28106120 2024-02-02T15:40:55.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:55.970+0000 7f9a26a00700 1 --2- 172.21.15.1:0/1515713029 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a28106120 0x7f9a280704a0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f9a1000e970 tx=0x7f9a1000ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:55.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:55.971+0000 7f9a1fe00700 1 -- 172.21.15.1:0/1515713029 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a10009790 con 0x7f9a28106120 2024-02-02T15:40:55.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:55.971+0000 7f9a1fe00700 1 -- 172.21.15.1:0/1515713029 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a10004cf0 con 0x7f9a28106120 2024-02-02T15:40:55.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:55.971+0000 7f9a1fe00700 1 -- 172.21.15.1:0/1515713029 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a10005680 con 0x7f9a28106120 2024-02-02T15:40:55.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:55.971+0000 7f9a2e000700 1 -- 172.21.15.1:0/1515713029 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a2806ed20 con 0x7f9a28106120 2024-02-02T15:40:55.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:55.971+0000 7f9a2e000700 1 -- 172.21.15.1:0/1515713029 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a2806f1e0 con 0x7f9a28106120 2024-02-02T15:40:55.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:55.979+0000 7f9a1fe00700 1 -- 172.21.15.1:0/1515713029 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f9a10004750 con 0x7f9a28106120 2024-02-02T15:40:55.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:55.982+0000 7f9a1fe00700 1 --2- 172.21.15.1:0/1515713029 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f9a080a4bf0 0x7f9a080a70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:55.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:55.982+0000 7f9a1fe00700 1 -- 172.21.15.1:0/1515713029 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f9a10014070 con 0x7f9a28106120 2024-02-02T15:40:55.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:55.983+0000 7f9a27400700 1 --2- 172.21.15.1:0/1515713029 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f9a080a4bf0 0x7f9a080a70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:55.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:55.987+0000 7f9a27400700 1 --2- 172.21.15.1:0/1515713029 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f9a080a4bf0 0x7f9a080a70b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9a180098e0 tx=0x7f9a18005e80 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:55.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:55.997+0000 7f9a41a9eb80 1 Processor -- start 2024-02-02T15:40:55.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:55.998+0000 7f9a41a9eb80 1 -- start start 2024-02-02T15:40:55.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:55.999+0000 7f9a41a9eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a3dae86090 0x55a3dae86470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:56.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.000+0000 7f9a41a9eb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3dae869b0 0x55a3dae91320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:56.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.000+0000 7f9a41a9eb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a3dae91860 0x55a3dae93c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:56.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.000+0000 7f9a41a9eb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3dadfa540 con 0x55a3dae869b0 2024-02-02T15:40:56.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.001+0000 7f9a41a9eb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3daddeb20 con 0x55a3dae91860 2024-02-02T15:40:56.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.001+0000 7f9a41a9eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3daddc4e0 con 0x55a3dae86090 2024-02-02T15:40:56.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.001+0000 7f9a27400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a3dae86090 0x55a3dae86470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:56.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.001+0000 7f9a27400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a3dae86090 0x55a3dae86470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:37696/0 (socket says 172.21.15.1:37696) 2024-02-02T15:40:56.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.001+0000 7f9a27400700 1 -- 172.21.15.1:0/1923197827 learned_addr learned my addr 172.21.15.1:0/1923197827 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:40:56.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.001+0000 7f9a26a00700 1 --2- 172.21.15.1:0/1923197827 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3dae869b0 0x55a3dae91320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:56.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.002+0000 7f9a27e00700 1 --2- 172.21.15.1:0/1923197827 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a3dae91860 0x55a3dae93c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:56.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.002+0000 7f9a26a00700 1 -- 172.21.15.1:0/1923197827 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a3dae86090 msgr2=0x55a3dae86470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:56.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.002+0000 7f9a26a00700 1 --2- 172.21.15.1:0/1923197827 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a3dae86090 0x55a3dae86470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:56.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.002+0000 7f9a26a00700 1 -- 172.21.15.1:0/1923197827 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a3dae91860 msgr2=0x55a3dae93c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:56.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.002+0000 7f9a26a00700 1 --2- 172.21.15.1:0/1923197827 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a3dae91860 0x55a3dae93c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:56.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.002+0000 7f9a26a00700 1 -- 172.21.15.1:0/1923197827 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a3dac59430 con 0x55a3dae869b0 2024-02-02T15:40:56.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.003+0000 7f9a26a00700 1 --2- 172.21.15.1:0/1923197827 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3dae869b0 0x55a3dae91320 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f9a100c96b0 tx=0x7f9a10098de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=564998df9a81ee36 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:56.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.004+0000 7f9a17e00700 1 -- 172.21.15.1:0/1923197827 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a100994e0 con 0x55a3dae869b0 2024-02-02T15:40:56.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.004+0000 7f9a17e00700 1 -- 172.21.15.1:0/1923197827 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a100cc150 con 0x55a3dae869b0 2024-02-02T15:40:56.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.004+0000 7f9a17e00700 1 -- 172.21.15.1:0/1923197827 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a10004e60 con 0x55a3dae869b0 2024-02-02T15:40:56.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.006+0000 7f9a41a9eb80 1 -- 172.21.15.1:0/1923197827 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3dae869b0 msgr2=0x55a3dae91320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:56.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.006+0000 7f9a41a9eb80 1 --2- 172.21.15.1:0/1923197827 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3dae869b0 0x55a3dae91320 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f9a100c96b0 tx=0x7f9a10098de0 comp rx=0 tx=0).stop 2024-02-02T15:40:56.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.008+0000 7f9a41a9eb80 1 -- 172.21.15.1:0/1923197827 shutdown_connections 2024-02-02T15:40:56.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.008+0000 7f9a41a9eb80 1 --2- 172.21.15.1:0/1923197827 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a3dae91860 0x55a3dae93c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:56.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.008+0000 7f9a41a9eb80 1 --2- 172.21.15.1:0/1923197827 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a3dae86090 0x55a3dae86470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:56.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.008+0000 7f9a41a9eb80 1 --2- 172.21.15.1:0/1923197827 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3dae869b0 0x55a3dae91320 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:56.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.008+0000 7f9a41a9eb80 1 -- 172.21.15.1:0/1923197827 >> 172.21.15.1:0/1923197827 conn(0x55a3dade5e30 msgr2=0x55a3dae82430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:56.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.011+0000 7f9a41a9eb80 1 -- 172.21.15.1:0/1923197827 shutdown_connections 2024-02-02T15:40:56.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.011+0000 7f9a41a9eb80 1 -- 172.21.15.1:0/1923197827 wait complete. 2024-02-02T15:40:56.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.012+0000 7f9a41a9eb80 1 -- start start 2024-02-02T15:40:56.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.013+0000 7f9a41a9eb80 1 Processor -- start 2024-02-02T15:40:56.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.013+0000 7f9a41a9eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a3dae86090 0x55a3dae912d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:56.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.014+0000 7f9a41a9eb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a3dae869b0 0x55a3dae8e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:56.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.014+0000 7f9a27400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a3dae86090 0x55a3dae912d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:56.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.014+0000 7f9a41a9eb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3dae91860 0x55a3dae8ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:56.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.014+0000 7f9a27400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a3dae86090 0x55a3dae912d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:37712/0 (socket says 172.21.15.1:37712) 2024-02-02T15:40:56.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.014+0000 7f9a41a9eb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3dadd7dd0 con 0x55a3dae91860 2024-02-02T15:40:56.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.014+0000 7f9a41a9eb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3dad73e00 con 0x55a3dae869b0 2024-02-02T15:40:56.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.014+0000 7f9a41a9eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3dad73aa0 con 0x55a3dae86090 2024-02-02T15:40:56.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.014+0000 7f9a41a9eb80 10 client.0 _ll_register_callbacks cb 0x55a3dad75310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:40:56.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.014+0000 7f9a26a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a3dae869b0 0x55a3dae8e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:56.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.014+0000 7f9a27400700 1 -- 172.21.15.1:0/3628335205 learned_addr learned my addr 172.21.15.1:0/3628335205 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:40:56.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.015+0000 7f9a27e00700 1 --2- 172.21.15.1:0/3628335205 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3dae91860 0x55a3dae8ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:56.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.015+0000 7f9a26a00700 1 -- 172.21.15.1:0/3628335205 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a3dae86090 msgr2=0x55a3dae912d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:56.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.015+0000 7f9a26a00700 1 --2- 172.21.15.1:0/3628335205 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a3dae86090 0x55a3dae912d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:56.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.015+0000 7f9a26a00700 1 -- 172.21.15.1:0/3628335205 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3dae91860 msgr2=0x55a3dae8ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:56.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.015+0000 7f9a26a00700 1 --2- 172.21.15.1:0/3628335205 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3dae91860 0x55a3dae8ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:56.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.015+0000 7f9a26a00700 1 -- 172.21.15.1:0/3628335205 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a2806f1e0 con 0x55a3dae869b0 2024-02-02T15:40:56.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.016+0000 7f9a26a00700 1 --2- 172.21.15.1:0/3628335205 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a3dae869b0 0x55a3dae8e4e0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f9a100a16b0 tx=0x7f9a10099590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27dd2f8bb544cd03 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:56.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.016+0000 7f9a14c00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:40:56.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.017+0000 7f9a14c00700 1 -- 172.21.15.1:0/3628335205 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a100a14f0 con 0x55a3dae869b0 2024-02-02T15:40:56.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.017+0000 7f9a14c00700 1 -- 172.21.15.1:0/3628335205 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f9a100b7030 con 0x55a3dae869b0 2024-02-02T15:40:56.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.017+0000 7f9a41a9eb80 1 -- 172.21.15.1:0/3628335205 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a3dae7a170 con 0x55a3dae869b0 2024-02-02T15:40:56.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.017+0000 7f9a14c00700 1 -- 172.21.15.1:0/3628335205 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a10004e60 con 0x55a3dae869b0 2024-02-02T15:40:56.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.017+0000 7f9a41a9eb80 10 client.29360 fetch_fsmap learned FSMap version 59 2024-02-02T15:40:56.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.017+0000 7f9a41a9eb80 1 -- 172.21.15.1:0/3628335205 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) v3 -- 0x55a3dac05be0 con 0x55a3dae869b0 2024-02-02T15:40:56.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.018+0000 7f9a14c00700 1 -- 172.21.15.1:0/3628335205 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a3dae7a170 con 0x55a3dae869b0 2024-02-02T15:40:56.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.018+0000 7f9a14c00700 1 -- 172.21.15.1:0/3628335205 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 59) v1 ==== 7538+0+0 (secure 0 0 0) 0x7f9a100996e0 con 0x55a3dae869b0 2024-02-02T15:40:56.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.019+0000 7f9a41a9eb80 10 client.29360 fetch_fsmap finished waiting for FSMap version 59 2024-02-02T15:40:56.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.019+0000 7f9a41a9eb80 10 client.29360 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:40:56.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.019+0000 7f9a41a9eb80 20 client.29360 populate_metadata read hostname 'smithi001' 2024-02-02T15:40:56.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.019+0000 7f9a41a9eb80 1 --2- 172.21.15.1:0/3628335205 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55a3daea1050 0x55a3daea3470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:56.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.019+0000 7f9a41a9eb80 4 client.29360 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:40:56.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.019+0000 7f9a41a9eb80 1 -- 172.21.15.1:0/3628335205 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a3dadd0e50 con 0x55a3daea1050 2024-02-02T15:40:56.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.019+0000 7f9a27400700 1 --2- 172.21.15.1:0/3628335205 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55a3daea1050 0x55a3daea3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:56.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.020+0000 7f9a14c00700 10 client.29360 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:40:56.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.020+0000 7f9a27400700 1 --2- 172.21.15.1:0/3628335205 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55a3daea1050 0x55a3daea3470 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5b720489c782c39 server_cookie=4365bafe8a1994cc in_seq=0 out_seq=0 2024-02-02T15:40:56.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.022+0000 7f9a14c00700 1 -- 172.21.15.1:0/3628335205 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55a3dadd0e50 con 0x55a3daea1050 2024-02-02T15:40:56.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.023+0000 7f9a14c00700 10 client.29360 handle_command_reply: tid=0 2024-02-02T15:40:56.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.023+0000 7f9a41a9eb80 1 client.29360 shutdown 2024-02-02T15:40:56.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.025+0000 7f9a41a9eb80 1 -- 172.21.15.1:0/3628335205 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a3dae869b0 msgr2=0x55a3dae8e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:56.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.025+0000 7f9a41a9eb80 1 --2- 172.21.15.1:0/3628335205 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a3dae869b0 0x55a3dae8e4e0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f9a100a16b0 tx=0x7f9a10099590 comp rx=0 tx=0).stop 2024-02-02T15:40:56.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.025+0000 7f9a41a9eb80 1 -- 172.21.15.1:0/3628335205 shutdown_connections 2024-02-02T15:40:56.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.025+0000 7f9a41a9eb80 1 --2- 172.21.15.1:0/3628335205 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55a3daea1050 0x55a3daea3470 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:56.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.025+0000 7f9a41a9eb80 1 --2- 172.21.15.1:0/3628335205 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3dae91860 0x55a3dae8ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:56.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.025+0000 7f9a14c00700 0 client.29360 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:40:56.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.025+0000 7f9a41a9eb80 1 --2- 172.21.15.1:0/3628335205 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a3dae86090 0x55a3dae912d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:56.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.025+0000 7f9a41a9eb80 1 --2- 172.21.15.1:0/3628335205 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a3dae869b0 0x55a3dae8e4e0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:56.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.025+0000 7f9a41a9eb80 1 -- 172.21.15.1:0/3628335205 >> 172.21.15.1:0/3628335205 conn(0x55a3dade5e30 msgr2=0x55a3dae82020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:56.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.026+0000 7f9a41a9eb80 1 -- 172.21.15.1:0/3628335205 shutdown_connections 2024-02-02T15:40:56.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.026+0000 7f9a41a9eb80 1 -- 172.21.15.1:0/3628335205 wait complete. 2024-02-02T15:40:56.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.027+0000 7f9a41a9eb80 20 client.29360 trim_cache size 0 max 16384 2024-02-02T15:40:56.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.065+0000 7f9a41a9eb80 1 Processor -- start 2024-02-02T15:40:56.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.065+0000 7f9a41a9eb80 1 -- start start 2024-02-02T15:40:56.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.066+0000 7f9a41a9eb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3dae86090 0x55a3dae983b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:56.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.066+0000 7f9a41a9eb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a3dae869b0 0x55a3dae8b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:56.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.066+0000 7f9a41a9eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a3dae91860 0x55a3dae8bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:56.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.066+0000 7f9a41a9eb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3dac5b720 con 0x55a3dae86090 2024-02-02T15:40:56.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.066+0000 7f9a41a9eb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3dae79d40 con 0x55a3dae869b0 2024-02-02T15:40:56.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.066+0000 7f9a41a9eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3dae7c580 con 0x55a3dae91860 2024-02-02T15:40:56.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.067+0000 7f9a26a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a3dae869b0 0x55a3dae8b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:56.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.067+0000 7f9a26a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a3dae869b0 0x55a3dae8b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:41878/0 (socket says 172.21.15.1:41878) 2024-02-02T15:40:56.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.067+0000 7f9a26a00700 1 -- 172.21.15.1:0/3244986522 learned_addr learned my addr 172.21.15.1:0/3244986522 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:40:56.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.067+0000 7f9a27e00700 1 --2- 172.21.15.1:0/3244986522 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a3dae91860 0x55a3dae8bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:56.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.067+0000 7f9a26a00700 1 -- 172.21.15.1:0/3244986522 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a3dae91860 msgr2=0x55a3dae8bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:56.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.067+0000 7f9a26a00700 1 --2- 172.21.15.1:0/3244986522 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a3dae91860 0x55a3dae8bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:56.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.067+0000 7f9a26a00700 1 -- 172.21.15.1:0/3244986522 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3dae86090 msgr2=0x55a3dae983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:56.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.067+0000 7f9a26a00700 1 --2- 172.21.15.1:0/3244986522 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3dae86090 0x55a3dae983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:56.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.067+0000 7f9a26a00700 1 -- 172.21.15.1:0/3244986522 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a3dad9df20 con 0x55a3dae869b0 2024-02-02T15:40:56.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.068+0000 7f9a26a00700 1 --2- 172.21.15.1:0/3244986522 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a3dae869b0 0x55a3dae8b790 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f9a10000c00 tx=0x7f9a10017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83504fa78dfc0f28 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:56.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.069+0000 7f9a1d200700 1 -- 172.21.15.1:0/3244986522 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a100a14f0 con 0x55a3dae869b0 2024-02-02T15:40:56.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.069+0000 7f9a1d200700 1 -- 172.21.15.1:0/3244986522 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a10098b60 con 0x55a3dae869b0 2024-02-02T15:40:56.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.069+0000 7f9a1d200700 1 -- 172.21.15.1:0/3244986522 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a1000ccd0 con 0x55a3dae869b0 2024-02-02T15:40:56.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.070+0000 7f9a41a9eb80 1 -- 172.21.15.1:0/3244986522 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a3dae869b0 msgr2=0x55a3dae8b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:56.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.070+0000 7f9a41a9eb80 1 --2- 172.21.15.1:0/3244986522 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a3dae869b0 0x55a3dae8b790 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f9a10000c00 tx=0x7f9a10017000 comp rx=0 tx=0).stop 2024-02-02T15:40:56.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.070+0000 7f9a41a9eb80 1 -- 172.21.15.1:0/3244986522 shutdown_connections 2024-02-02T15:40:56.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.070+0000 7f9a41a9eb80 1 --2- 172.21.15.1:0/3244986522 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a3dae91860 0x55a3dae8bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:56.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.070+0000 7f9a41a9eb80 1 --2- 172.21.15.1:0/3244986522 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3dae86090 0x55a3dae983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:56.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.070+0000 7f9a41a9eb80 1 --2- 172.21.15.1:0/3244986522 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a3dae869b0 0x55a3dae8b790 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:56.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.070+0000 7f9a41a9eb80 1 -- 172.21.15.1:0/3244986522 >> 172.21.15.1:0/3244986522 conn(0x55a3dade5e30 msgr2=0x55a3dae88950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:56.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.070+0000 7f9a41a9eb80 1 -- 172.21.15.1:0/3244986522 shutdown_connections 2024-02-02T15:40:56.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.071+0000 7f9a41a9eb80 1 -- 172.21.15.1:0/3244986522 wait complete. 2024-02-02T15:40:56.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.071+0000 7f9a41a9eb80 1 -- start start 2024-02-02T15:40:56.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.072+0000 7f9a41a9eb80 1 Processor -- start 2024-02-02T15:40:56.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.073+0000 7f9a41a9eb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3dae86090 0x55a3dae973d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:56.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.073+0000 7f9a41a9eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a3dae869b0 0x55a3dae97910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:56.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.073+0000 7f9a41a9eb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a3dae91860 0x55a3daeebc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:56.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.074+0000 7f9a27400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3dae86090 0x55a3dae973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:56.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.074+0000 7f9a27e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a3dae91860 0x55a3daeebc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:56.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.074+0000 7f9a26a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a3dae869b0 0x55a3dae97910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:56.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.074+0000 7f9a26a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a3dae869b0 0x55a3dae97910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:37728/0 (socket says 172.21.15.1:37728) 2024-02-02T15:40:56.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.074+0000 7f9a27e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a3dae91860 0x55a3daeebc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:41892/0 (socket says 172.21.15.1:41892) 2024-02-02T15:40:56.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.074+0000 7f9a41a9eb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3daeec190 con 0x55a3dae86090 2024-02-02T15:40:56.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.074+0000 7f9a41a9eb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3daeec300 con 0x55a3dae91860 2024-02-02T15:40:56.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.074+0000 7f9a41a9eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3daeec4c0 con 0x55a3dae869b0 2024-02-02T15:40:56.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.074+0000 7f9a41a9eb80 10 client.0 _ll_register_callbacks cb 0x55a3dad75310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:40:56.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.074+0000 7f9a27400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3dae86090 0x55a3dae973d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53866/0 (socket says 172.21.15.1:53866) 2024-02-02T15:40:56.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.074+0000 7f9a27400700 1 -- 172.21.15.1:0/397527201 learned_addr learned my addr 172.21.15.1:0/397527201 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:40:56.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.075+0000 7f9a26a00700 1 -- 172.21.15.1:0/397527201 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a3dae91860 msgr2=0x55a3daeebc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:56.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.075+0000 7f9a26a00700 1 --2- 172.21.15.1:0/397527201 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a3dae91860 0x55a3daeebc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:56.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.075+0000 7f9a26a00700 1 -- 172.21.15.1:0/397527201 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3dae86090 msgr2=0x55a3dae973d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:56.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.075+0000 7f9a26a00700 1 --2- 172.21.15.1:0/397527201 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3dae86090 0x55a3dae973d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:56.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.075+0000 7f9a26a00700 1 -- 172.21.15.1:0/397527201 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a3dac05be0 con 0x55a3dae869b0 2024-02-02T15:40:56.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.076+0000 7f9a26a00700 1 --2- 172.21.15.1:0/397527201 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a3dae869b0 0x55a3dae97910 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x55a3daea82f0 tx=0x7f9a10005910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2958753a378f3e79 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:56.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.076+0000 7f9a14c00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:40:56.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.077+0000 7f9a14c00700 1 -- 172.21.15.1:0/397527201 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a10098970 con 0x55a3dae869b0 2024-02-02T15:40:56.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.077+0000 7f9a14c00700 1 -- 172.21.15.1:0/397527201 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f9a100b7040 con 0x55a3dae869b0 2024-02-02T15:40:56.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.077+0000 7f9a41a9eb80 1 -- 172.21.15.1:0/397527201 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9a100cc150 con 0x55a3dae869b0 2024-02-02T15:40:56.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.078+0000 7f9a14c00700 1 -- 172.21.15.1:0/397527201 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a100a14f0 con 0x55a3dae869b0 2024-02-02T15:40:56.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.078+0000 7f9a41a9eb80 10 client.29347 fetch_fsmap learned FSMap version 59 2024-02-02T15:40:56.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.078+0000 7f9a41a9eb80 1 -- 172.21.15.1:0/397527201 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) v3 -- 0x55a3dac5dd50 con 0x55a3dae869b0 2024-02-02T15:40:56.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.078+0000 7f9a14c00700 1 -- 172.21.15.1:0/397527201 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a100cc150 con 0x55a3dae869b0 2024-02-02T15:40:56.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.079+0000 7f9a14c00700 1 -- 172.21.15.1:0/397527201 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 59) v1 ==== 7538+0+0 (secure 0 0 0) 0x7f9a100a1060 con 0x55a3dae869b0 2024-02-02T15:40:56.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.079+0000 7f9a41a9eb80 10 client.29347 fetch_fsmap finished waiting for FSMap version 59 2024-02-02T15:40:56.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.079+0000 7f9a41a9eb80 10 client.29347 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:40:56.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.079+0000 7f9a41a9eb80 20 client.29347 populate_metadata read hostname 'smithi001' 2024-02-02T15:40:56.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.079+0000 7f9a41a9eb80 1 --2- 172.21.15.1:0/397527201 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55a3daef49e0 0x55a3daef6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:56.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.079+0000 7f9a41a9eb80 4 client.29347 mds_command: new command op to 24557 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-02T15:40:56.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.079+0000 7f9a41a9eb80 1 -- 172.21.15.1:0/397527201 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55a3dad96fd0 con 0x55a3daef49e0 2024-02-02T15:40:56.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.080+0000 7f9a27400700 1 --2- 172.21.15.1:0/397527201 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55a3daef49e0 0x55a3daef6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:56.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.081+0000 7f9a27400700 1 --2- 172.21.15.1:0/397527201 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55a3daef49e0 0x55a3daef6dd0 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f066ac2e87cd59ab server_cookie=8b285c1ce0cfd1ab in_seq=0 out_seq=0 2024-02-02T15:40:56.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.081+0000 7f9a14c00700 10 client.29347 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:40:56.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.083+0000 7f9a14c00700 1 -- 172.21.15.1:0/397527201 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55a3dad96fd0 con 0x55a3daef49e0 2024-02-02T15:40:56.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.083+0000 7f9a14c00700 10 client.29347 handle_command_reply: tid=0 2024-02-02T15:40:56.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.083+0000 7f9a41a9eb80 1 client.29347 shutdown 2024-02-02T15:40:56.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.085+0000 7f9a41a9eb80 1 -- 172.21.15.1:0/397527201 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a3dae869b0 msgr2=0x55a3dae97910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:56.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.085+0000 7f9a41a9eb80 1 --2- 172.21.15.1:0/397527201 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a3dae869b0 0x55a3dae97910 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x55a3daea82f0 tx=0x7f9a10005910 comp rx=0 tx=0).stop 2024-02-02T15:40:56.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.086+0000 7f9a41a9eb80 1 -- 172.21.15.1:0/397527201 shutdown_connections 2024-02-02T15:40:56.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.086+0000 7f9a41a9eb80 1 --2- 172.21.15.1:0/397527201 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55a3daef49e0 0x55a3daef6dd0 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:56.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.086+0000 7f9a41a9eb80 1 --2- 172.21.15.1:0/397527201 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a3dae91860 0x55a3daeebc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:56.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.086+0000 7f9a41a9eb80 1 --2- 172.21.15.1:0/397527201 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a3dae86090 0x55a3dae973d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:56.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.086+0000 7f9a41a9eb80 1 --2- 172.21.15.1:0/397527201 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a3dae869b0 0x55a3dae97910 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:56.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.086+0000 7f9a41a9eb80 1 -- 172.21.15.1:0/397527201 >> 172.21.15.1:0/397527201 conn(0x55a3dade5e30 msgr2=0x55a3dae81640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:56.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.086+0000 7f9a14c00700 0 client.29347 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:40:56.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.087+0000 7f9a41a9eb80 1 -- 172.21.15.1:0/397527201 shutdown_connections 2024-02-02T15:40:56.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.087+0000 7f9a41a9eb80 1 -- 172.21.15.1:0/397527201 wait complete. 2024-02-02T15:40:56.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.088+0000 7f9a41a9eb80 20 client.29347 trim_cache size 0 max 16384 2024-02-02T15:40:56.089 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:40:56.089 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2024-02-02T15:40:56.089 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "b678ed4d-56e4-4933-ab32-28b6e1d47a6a", 2024-02-02T15:40:56.089 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2024-02-02T15:40:56.089 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:40:56.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.090+0000 7f9a2e000700 1 -- 172.21.15.1:0/1515713029 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f9a080a4bf0 msgr2=0x7f9a080a70b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:56.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.090+0000 7f9a2e000700 1 --2- 172.21.15.1:0/1515713029 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f9a080a4bf0 0x7f9a080a70b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9a180098e0 tx=0x7f9a18005e80 comp rx=0 tx=0).stop 2024-02-02T15:40:56.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.090+0000 7f9a2e000700 1 -- 172.21.15.1:0/1515713029 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a28106120 msgr2=0x7f9a280704a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:56.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.090+0000 7f9a2e000700 1 --2- 172.21.15.1:0/1515713029 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a28106120 0x7f9a280704a0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f9a1000e970 tx=0x7f9a1000ed30 comp rx=0 tx=0).stop 2024-02-02T15:40:56.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.090+0000 7f9a2e000700 1 -- 172.21.15.1:0/1515713029 shutdown_connections 2024-02-02T15:40:56.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.090+0000 7f9a2e000700 1 --2- 172.21.15.1:0/1515713029 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f9a080a4bf0 0x7f9a080a70b0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:56.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.090+0000 7f9a2e000700 1 --2- 172.21.15.1:0/1515713029 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9a2806b150 0x7f9a2806b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:56.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.090+0000 7f9a2e000700 1 --2- 172.21.15.1:0/1515713029 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a28101000 0x7f9a2806ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:56.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.090+0000 7f9a2e000700 1 --2- 172.21.15.1:0/1515713029 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a28106120 0x7f9a280704a0 secure :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f9a1000e970 tx=0x7f9a1000ed30 comp rx=0 tx=0).stop 2024-02-02T15:40:56.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.090+0000 7f9a2e000700 1 -- 172.21.15.1:0/1515713029 >> 172.21.15.1:0/1515713029 conn(0x7f9a280fc840 msgr2=0x7f9a28104ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:56.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.090+0000 7f9a2e000700 1 -- 172.21.15.1:0/1515713029 shutdown_connections 2024-02-02T15:40:56.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.091+0000 7f9a2e000700 1 -- 172.21.15.1:0/1515713029 wait complete. 2024-02-02T15:40:56.123 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-02T15:40:56.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.456+0000 7ff960000700 1 -- 172.21.15.1:0/1474534140 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff958101010 msgr2=0x7ff9581013f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:56.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.456+0000 7ff960000700 1 --2- 172.21.15.1:0/1474534140 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff958101010 0x7ff9581013f0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7ff948009990 tx=0x7ff948009ca0 comp rx=0 tx=0).stop 2024-02-02T15:40:56.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.457+0000 7ff960000700 1 -- 172.21.15.1:0/1474534140 shutdown_connections 2024-02-02T15:40:56.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.457+0000 7ff960000700 1 --2- 172.21.15.1:0/1474534140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff958106130 0x7ff95810dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:56.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.457+0000 7ff960000700 1 --2- 172.21.15.1:0/1474534140 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff958101010 0x7ff9581013f0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:56.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.457+0000 7ff960000700 1 --2- 172.21.15.1:0/1474534140 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff9581019c0 0x7ff958105a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:56.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.457+0000 7ff960000700 1 -- 172.21.15.1:0/1474534140 >> 172.21.15.1:0/1474534140 conn(0x7ff9580fc8b0 msgr2=0x7ff9580fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:56.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.457+0000 7ff960000700 1 -- 172.21.15.1:0/1474534140 shutdown_connections 2024-02-02T15:40:56.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.457+0000 7ff960000700 1 -- 172.21.15.1:0/1474534140 wait complete. 2024-02-02T15:40:56.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.458+0000 7ff960000700 1 Processor -- start 2024-02-02T15:40:56.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.459+0000 7ff960000700 1 -- start start 2024-02-02T15:40:56.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.459+0000 7ff960000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff958101010 0x7ff95806a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:56.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.459+0000 7ff960000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff958106130 0x7ff95806aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:56.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.459+0000 7ff960000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff95806b1e0 0x7ff95806e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:56.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.459+0000 7ff960000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff95806ed70 con 0x7ff958106130 2024-02-02T15:40:56.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.459+0000 7ff960000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff95806eee0 con 0x7ff958101010 2024-02-02T15:40:56.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.459+0000 7ff960000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff95806f0a0 con 0x7ff95806b1e0 2024-02-02T15:40:56.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.460+0000 7ff95e000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff95806b1e0 0x7ff95806e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:56.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.460+0000 7ff95d600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff958101010 0x7ff95806a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:56.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.460+0000 7ff95e000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff95806b1e0 0x7ff95806e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:37738/0 (socket says 172.21.15.1:37738) 2024-02-02T15:40:56.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.460+0000 7ff95e000700 1 -- 172.21.15.1:0/3073189954 learned_addr learned my addr 172.21.15.1:0/3073189954 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:40:56.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.460+0000 7ff95cc00700 1 --2- 172.21.15.1:0/3073189954 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff958106130 0x7ff95806aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:56.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.460+0000 7ff95e000700 1 -- 172.21.15.1:0/3073189954 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff958101010 msgr2=0x7ff95806a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:56.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.460+0000 7ff95e000700 1 --2- 172.21.15.1:0/3073189954 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff958101010 0x7ff95806a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:56.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.460+0000 7ff95e000700 1 -- 172.21.15.1:0/3073189954 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff958106130 msgr2=0x7ff95806aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:56.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.460+0000 7ff95e000700 1 --2- 172.21.15.1:0/3073189954 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff958106130 0x7ff95806aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:56.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.461+0000 7ff95e000700 1 -- 172.21.15.1:0/3073189954 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9480095f0 con 0x7ff95806b1e0 2024-02-02T15:40:56.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.461+0000 7ff95d600700 1 --2- 172.21.15.1:0/3073189954 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff958101010 0x7ff95806a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:40:56.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.461+0000 7ff95e000700 1 --2- 172.21.15.1:0/3073189954 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff95806b1e0 0x7ff95806e830 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7ff950007ee0 tx=0x7ff95000d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:56.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.462+0000 7ff956000700 1 -- 172.21.15.1:0/3073189954 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff950017070 con 0x7ff95806b1e0 2024-02-02T15:40:56.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.462+0000 7ff956000700 1 -- 172.21.15.1:0/3073189954 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff95000f040 con 0x7ff95806b1e0 2024-02-02T15:40:56.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.462+0000 7ff956000700 1 -- 172.21.15.1:0/3073189954 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9500055d0 con 0x7ff95806b1e0 2024-02-02T15:40:56.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.462+0000 7ff960000700 1 -- 172.21.15.1:0/3073189954 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff95806f380 con 0x7ff95806b1e0 2024-02-02T15:40:56.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.462+0000 7ff960000700 1 -- 172.21.15.1:0/3073189954 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9581ae9e0 con 0x7ff95806b1e0 2024-02-02T15:40:56.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.465+0000 7ff956000700 1 -- 172.21.15.1:0/3073189954 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7ff95000db20 con 0x7ff95806b1e0 2024-02-02T15:40:56.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.468+0000 7ff956000700 1 --2- 172.21.15.1:0/3073189954 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff9380a07e0 0x7ff9380a2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:56.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.469+0000 7ff956000700 1 -- 172.21.15.1:0/3073189954 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 9453+0+0 (secure 0 0 0) 0x7ff9500cb690 con 0x7ff95806b1e0 2024-02-02T15:40:56.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.469+0000 7ff973b89b80 1 Processor -- start 2024-02-02T15:40:56.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.469+0000 7ff95d600700 1 --2- 172.21.15.1:0/3073189954 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff9380a07e0 0x7ff9380a2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:56.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.473+0000 7ff973b89b80 1 -- start start 2024-02-02T15:40:56.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.473+0000 7ff973b89b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55acab6cb010 0x55acab6cb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:56.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.473+0000 7ff973b89b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55acab6cb930 0x55acab6deab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:56.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.473+0000 7ff973b89b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55acab6deff0 0x55acab6e13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:56.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.473+0000 7ff973b89b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acab63f4c0 con 0x55acab6deff0 2024-02-02T15:40:56.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.473+0000 7ff973b89b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acab623aa0 con 0x55acab6cb930 2024-02-02T15:40:56.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.473+0000 7ff973b89b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acab621460 con 0x55acab6cb010 2024-02-02T15:40:56.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.474+0000 7ff95cc00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55acab6cb930 0x55acab6deab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:56.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.474+0000 7ff95cc00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55acab6cb930 0x55acab6deab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:41906/0 (socket says 172.21.15.1:41906) 2024-02-02T15:40:56.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.474+0000 7ff95cc00700 1 -- 172.21.15.1:0/1147320465 learned_addr learned my addr 172.21.15.1:0/1147320465 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:40:56.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.474+0000 7ff95e000700 1 --2- 172.21.15.1:0/1147320465 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55acab6deff0 0x55acab6e13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:56.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.475+0000 7ff95cc00700 1 -- 172.21.15.1:0/1147320465 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55acab6cb010 msgr2=0x55acab6cb3f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-02T15:40:56.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.475+0000 7ff95cc00700 1 --2- 172.21.15.1:0/1147320465 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55acab6cb010 0x55acab6cb3f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:56.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.475+0000 7ff95cc00700 1 -- 172.21.15.1:0/1147320465 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55acab6deff0 msgr2=0x55acab6e13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:56.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.475+0000 7ff95cc00700 1 --2- 172.21.15.1:0/1147320465 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55acab6deff0 0x55acab6e13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:56.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.475+0000 7ff95cc00700 1 -- 172.21.15.1:0/1147320465 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55acab49e3b0 con 0x55acab6cb930 2024-02-02T15:40:56.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.475+0000 7ff95e000700 1 --2- 172.21.15.1:0/1147320465 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55acab6deff0 0x55acab6e13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:40:56.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.475+0000 7ff95d600700 1 --2- 172.21.15.1:0/3073189954 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff9380a07e0 0x7ff9380a2ca0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff94800b5c0 tx=0x7ff948005fb0 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:56.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.475+0000 7ff95cc00700 1 --2- 172.21.15.1:0/1147320465 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55acab6cb930 0x55acab6deab0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7ff94000ad10 tx=0x7ff94000f6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=302381d695b5002e server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:56.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.475+0000 7ff94e600700 1 -- 172.21.15.1:0/1147320465 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff940004220 con 0x55acab6cb930 2024-02-02T15:40:56.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.475+0000 7ff94e600700 1 -- 172.21.15.1:0/1147320465 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff94000fec0 con 0x55acab6cb930 2024-02-02T15:40:56.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.475+0000 7ff94e600700 1 -- 172.21.15.1:0/1147320465 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff940018680 con 0x55acab6cb930 2024-02-02T15:40:56.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.476+0000 7ff973b89b80 1 -- 172.21.15.1:0/1147320465 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55acab6cb930 msgr2=0x55acab6deab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:56.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.476+0000 7ff973b89b80 1 --2- 172.21.15.1:0/1147320465 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55acab6cb930 0x55acab6deab0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7ff94000ad10 tx=0x7ff94000f6a0 comp rx=0 tx=0).stop 2024-02-02T15:40:56.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.477+0000 7ff973b89b80 1 -- 172.21.15.1:0/1147320465 shutdown_connections 2024-02-02T15:40:56.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.477+0000 7ff973b89b80 1 --2- 172.21.15.1:0/1147320465 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55acab6deff0 0x55acab6e13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:56.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.477+0000 7ff973b89b80 1 --2- 172.21.15.1:0/1147320465 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55acab6cb010 0x55acab6cb3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:56.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.477+0000 7ff973b89b80 1 --2- 172.21.15.1:0/1147320465 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55acab6cb930 0x55acab6deab0 secure :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7ff94000ad10 tx=0x7ff94000f6a0 comp rx=0 tx=0).stop 2024-02-02T15:40:56.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.477+0000 7ff973b89b80 1 -- 172.21.15.1:0/1147320465 >> 172.21.15.1:0/1147320465 conn(0x55acab62adb0 msgr2=0x55acab6c73b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:56.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.477+0000 7ff973b89b80 1 -- 172.21.15.1:0/1147320465 shutdown_connections 2024-02-02T15:40:56.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.477+0000 7ff973b89b80 1 -- 172.21.15.1:0/1147320465 wait complete. 2024-02-02T15:40:56.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.478+0000 7ff973b89b80 1 -- start start 2024-02-02T15:40:56.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.478+0000 7ff973b89b80 1 Processor -- start 2024-02-02T15:40:56.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.481+0000 7ff973b89b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55acab6cb010 0x55acab6d8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:56.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.481+0000 7ff973b89b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55acab6deff0 0x55acab6d8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:56.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.481+0000 7ff973b89b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55acab6de3a0 0x55acab6de820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:56.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.482+0000 7ff95d600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55acab6cb010 0x55acab6d8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:56.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.482+0000 7ff95d600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55acab6cb010 0x55acab6d8630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:41922/0 (socket says 172.21.15.1:41922) 2024-02-02T15:40:56.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.482+0000 7ff95d600700 1 -- 172.21.15.1:0/656317826 learned_addr learned my addr 172.21.15.1:0/656317826 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:40:56.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.482+0000 7ff973b89b80 1 -- 172.21.15.1:0/656317826 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acab61cd50 con 0x55acab6de3a0 2024-02-02T15:40:56.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.482+0000 7ff973b89b80 1 -- 172.21.15.1:0/656317826 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acab5b8d80 con 0x55acab6cb010 2024-02-02T15:40:56.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.482+0000 7ff973b89b80 1 -- 172.21.15.1:0/656317826 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acab5b8a20 con 0x55acab6deff0 2024-02-02T15:40:56.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.482+0000 7ff973b89b80 10 client.0 _ll_register_callbacks cb 0x55acab5ba290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:40:56.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.482+0000 7ff95e000700 1 --2- 172.21.15.1:0/656317826 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55acab6de3a0 0x55acab6de820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:56.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.482+0000 7ff95d600700 1 -- 172.21.15.1:0/656317826 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55acab6deff0 msgr2=0x55acab6d8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:56.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.482+0000 7ff95d600700 1 --2- 172.21.15.1:0/656317826 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55acab6deff0 0x55acab6d8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:56.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.482+0000 7ff95d600700 1 -- 172.21.15.1:0/656317826 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55acab6de3a0 msgr2=0x55acab6de820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:56.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.482+0000 7ff95d600700 1 --2- 172.21.15.1:0/656317826 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55acab6de3a0 0x55acab6de820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:56.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.482+0000 7ff95d600700 1 -- 172.21.15.1:0/656317826 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff94000a910 con 0x55acab6cb010 2024-02-02T15:40:56.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.483+0000 7ff95d600700 1 --2- 172.21.15.1:0/656317826 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55acab6cb010 0x55acab6d8630 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7ff948003f70 tx=0x7ff948009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5faed0d81d6890d4 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:56.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.483+0000 7ff946a00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:40:56.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.486+0000 7ff946a00700 1 -- 172.21.15.1:0/656317826 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff94802a070 con 0x55acab6cb010 2024-02-02T15:40:56.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.487+0000 7ff946a00700 1 -- 172.21.15.1:0/656317826 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9453+0+0 (secure 0 0 0) 0x7ff94800f670 con 0x55acab6cb010 2024-02-02T15:40:56.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.487+0000 7ff973b89b80 1 -- 172.21.15.1:0/656317826 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55acab615dd0 con 0x55acab6cb010 2024-02-02T15:40:56.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.492+0000 7ff946a00700 1 -- 172.21.15.1:0/656317826 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff94800f830 con 0x55acab6cb010 2024-02-02T15:40:56.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.492+0000 7ff973b89b80 10 client.29378 fetch_fsmap learned FSMap version 59 2024-02-02T15:40:56.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.492+0000 7ff973b89b80 1 -- 172.21.15.1:0/656317826 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) v3 -- 0x55acab44ab60 con 0x55acab6cb010 2024-02-02T15:40:56.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.492+0000 7ff946a00700 1 -- 172.21.15.1:0/656317826 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff94801e040 con 0x55acab6cb010 2024-02-02T15:40:56.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.492+0000 7ff946a00700 1 -- 172.21.15.1:0/656317826 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 59) v1 ==== 7538+0+0 (secure 0 0 0) 0x7ff948036070 con 0x55acab6cb010 2024-02-02T15:40:56.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.493+0000 7ff973b89b80 10 client.29378 fetch_fsmap finished waiting for FSMap version 59 2024-02-02T15:40:56.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.493+0000 7ff973b89b80 10 client.29378 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:40:56.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.493+0000 7ff973b89b80 20 client.29378 populate_metadata read hostname 'smithi001' 2024-02-02T15:40:56.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.493+0000 7ff973b89b80 1 --2- 172.21.15.1:0/656317826 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55acab6f7750 0x55acab6f9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:56.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.493+0000 7ff973b89b80 4 client.29378 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:40:56.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.493+0000 7ff973b89b80 1 -- 172.21.15.1:0/656317826 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55acab615dd0 con 0x55acab6f7750 2024-02-02T15:40:56.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.493+0000 7ff95cc00700 1 --2- 172.21.15.1:0/656317826 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55acab6f7750 0x55acab6f9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:56.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.494+0000 7ff95cc00700 1 --2- 172.21.15.1:0/656317826 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55acab6f7750 0x55acab6f9bb0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aec20eccd1adeeb5 server_cookie=77d62435386f9b3 in_seq=0 out_seq=0 2024-02-02T15:40:56.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.494+0000 7ff946a00700 10 client.29378 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:40:56.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.496+0000 7ff946a00700 1 -- 172.21.15.1:0/656317826 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55acab615dd0 con 0x55acab6f7750 2024-02-02T15:40:56.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.496+0000 7ff946a00700 10 client.29378 handle_command_reply: tid=0 2024-02-02T15:40:56.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.496+0000 7ff973b89b80 1 client.29378 shutdown 2024-02-02T15:40:56.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.497+0000 7ff973b89b80 1 -- 172.21.15.1:0/656317826 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55acab6cb010 msgr2=0x55acab6d8630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:56.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.497+0000 7ff973b89b80 1 --2- 172.21.15.1:0/656317826 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55acab6cb010 0x55acab6d8630 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7ff948003f70 tx=0x7ff948009ca0 comp rx=0 tx=0).stop 2024-02-02T15:40:56.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.497+0000 7ff973b89b80 1 -- 172.21.15.1:0/656317826 shutdown_connections 2024-02-02T15:40:56.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.497+0000 7ff973b89b80 1 --2- 172.21.15.1:0/656317826 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55acab6f7750 0x55acab6f9bb0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:56.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.497+0000 7ff973b89b80 1 --2- 172.21.15.1:0/656317826 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55acab6de3a0 0x55acab6de820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:56.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.497+0000 7ff973b89b80 1 --2- 172.21.15.1:0/656317826 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55acab6cb010 0x55acab6d8630 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:56.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.497+0000 7ff973b89b80 1 --2- 172.21.15.1:0/656317826 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55acab6deff0 0x55acab6d8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:56.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.497+0000 7ff973b89b80 1 -- 172.21.15.1:0/656317826 >> 172.21.15.1:0/656317826 conn(0x55acab62adb0 msgr2=0x55acab6c6fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:56.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.497+0000 7ff946a00700 0 client.29378 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:40:56.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.499+0000 7ff973b89b80 1 -- 172.21.15.1:0/656317826 shutdown_connections 2024-02-02T15:40:56.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.499+0000 7ff973b89b80 1 -- 172.21.15.1:0/656317826 wait complete. 2024-02-02T15:40:56.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.499+0000 7ff973b89b80 20 client.29378 trim_cache size 0 max 16384 2024-02-02T15:40:56.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.526+0000 7ff973b89b80 1 Processor -- start 2024-02-02T15:40:56.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.526+0000 7ff973b89b80 1 -- start start 2024-02-02T15:40:56.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.527+0000 7ff973b89b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55acab6cb010 0x55acab6ceb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:56.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.527+0000 7ff95d600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55acab6cb010 0x55acab6ceb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:56.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.527+0000 7ff95d600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55acab6cb010 0x55acab6ceb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:41936/0 (socket says 172.21.15.1:41936) 2024-02-02T15:40:56.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.527+0000 7ff973b89b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55acab6de3a0 0x55acab6cf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:56.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.528+0000 7ff95d600700 1 -- 172.21.15.1:0/140478098 learned_addr learned my addr 172.21.15.1:0/140478098 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:40:56.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.528+0000 7ff973b89b80 1 --2- 172.21.15.1:0/140478098 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55acab6deff0 0x55acab6d8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:56.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.528+0000 7ff973b89b80 1 -- 172.21.15.1:0/140478098 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acab4a06a0 con 0x55acab6de3a0 2024-02-02T15:40:56.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.528+0000 7ff973b89b80 1 -- 172.21.15.1:0/140478098 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acab6f7530 con 0x55acab6cb010 2024-02-02T15:40:56.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.528+0000 7ff973b89b80 1 -- 172.21.15.1:0/140478098 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acab6cbf70 con 0x55acab6deff0 2024-02-02T15:40:56.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.529+0000 7ff95e000700 1 --2- 172.21.15.1:0/140478098 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55acab6deff0 0x55acab6d8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:56.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.529+0000 7ff95d600700 1 -- 172.21.15.1:0/140478098 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55acab6deff0 msgr2=0x55acab6d8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:56.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.529+0000 7ff95d600700 1 --2- 172.21.15.1:0/140478098 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55acab6deff0 0x55acab6d8230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:56.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.529+0000 7ff95d600700 1 -- 172.21.15.1:0/140478098 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55acab6de3a0 msgr2=0x55acab6cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:56.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.529+0000 7ff95cc00700 1 --2- 172.21.15.1:0/140478098 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55acab6de3a0 0x55acab6cf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:56.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.529+0000 7ff95d600700 1 --2- 172.21.15.1:0/140478098 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55acab6de3a0 0x55acab6cf0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:56.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.529+0000 7ff95d600700 1 -- 172.21.15.1:0/140478098 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55acab6ccb00 con 0x55acab6cb010 2024-02-02T15:40:56.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.529+0000 7ff95e000700 1 --2- 172.21.15.1:0/140478098 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55acab6deff0 0x55acab6d8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:40:56.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.529+0000 7ff95d600700 1 --2- 172.21.15.1:0/140478098 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55acab6cb010 0x55acab6ceb70 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7ff948000c00 tx=0x7ff94802ece0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e8790afd23a95e7 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:56.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.529+0000 7ff94f000700 1 -- 172.21.15.1:0/140478098 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff94802a070 con 0x55acab6cb010 2024-02-02T15:40:56.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.530+0000 7ff94f000700 1 -- 172.21.15.1:0/140478098 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff94801e040 con 0x55acab6cb010 2024-02-02T15:40:56.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.530+0000 7ff94f000700 1 -- 172.21.15.1:0/140478098 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff94800f9b0 con 0x55acab6cb010 2024-02-02T15:40:56.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.530+0000 7ff973b89b80 1 -- 172.21.15.1:0/140478098 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55acab6cb010 msgr2=0x55acab6ceb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:56.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.530+0000 7ff973b89b80 1 --2- 172.21.15.1:0/140478098 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55acab6cb010 0x55acab6ceb70 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7ff948000c00 tx=0x7ff94802ece0 comp rx=0 tx=0).stop 2024-02-02T15:40:56.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.531+0000 7ff973b89b80 1 -- 172.21.15.1:0/140478098 shutdown_connections 2024-02-02T15:40:56.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.531+0000 7ff973b89b80 1 --2- 172.21.15.1:0/140478098 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55acab6deff0 0x55acab6d8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:56.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.531+0000 7ff973b89b80 1 --2- 172.21.15.1:0/140478098 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55acab6cb010 0x55acab6ceb70 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:56.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.531+0000 7ff973b89b80 1 --2- 172.21.15.1:0/140478098 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55acab6de3a0 0x55acab6cf0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:56.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.531+0000 7ff973b89b80 1 -- 172.21.15.1:0/140478098 >> 172.21.15.1:0/140478098 conn(0x55acab62adb0 msgr2=0x55acab6d3be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:56.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.531+0000 7ff973b89b80 1 -- 172.21.15.1:0/140478098 shutdown_connections 2024-02-02T15:40:56.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.531+0000 7ff973b89b80 1 -- 172.21.15.1:0/140478098 wait complete. 2024-02-02T15:40:56.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.531+0000 7ff973b89b80 1 -- start start 2024-02-02T15:40:56.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.532+0000 7ff973b89b80 1 Processor -- start 2024-02-02T15:40:56.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.533+0000 7ff973b89b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55acab6cb010 0x55acab6db0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:56.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.533+0000 7ff973b89b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55acab6de3a0 0x55acab6db620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:56.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.533+0000 7ff973b89b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55acab6deff0 0x55acab6dbb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:56.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.533+0000 7ff973b89b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acab6de140 con 0x55acab6cb010 2024-02-02T15:40:56.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.533+0000 7ff973b89b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acab7482f0 con 0x55acab6de3a0 2024-02-02T15:40:56.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.533+0000 7ff973b89b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acab748430 con 0x55acab6deff0 2024-02-02T15:40:56.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.533+0000 7ff973b89b80 10 client.0 _ll_register_callbacks cb 0x55acab5ba290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:40:56.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.533+0000 7ff95cc00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55acab6de3a0 0x55acab6db620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:56.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.533+0000 7ff95cc00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55acab6de3a0 0x55acab6db620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:41952/0 (socket says 172.21.15.1:41952) 2024-02-02T15:40:56.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.533+0000 7ff95cc00700 1 -- 172.21.15.1:0/1599940036 learned_addr learned my addr 172.21.15.1:0/1599940036 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:40:56.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.533+0000 7ff95e000700 1 --2- 172.21.15.1:0/1599940036 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55acab6deff0 0x55acab6dbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:56.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.533+0000 7ff95cc00700 1 -- 172.21.15.1:0/1599940036 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55acab6deff0 msgr2=0x55acab6dbb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:56.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.533+0000 7ff95cc00700 1 --2- 172.21.15.1:0/1599940036 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55acab6deff0 0x55acab6dbb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:56.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.533+0000 7ff95cc00700 1 -- 172.21.15.1:0/1599940036 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55acab6cb010 msgr2=0x55acab6db0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:40:56.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.533+0000 7ff95cc00700 1 --2- 172.21.15.1:0/1599940036 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55acab6cb010 0x55acab6db0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:56.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.533+0000 7ff95cc00700 1 -- 172.21.15.1:0/1599940036 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55acab44ab60 con 0x55acab6de3a0 2024-02-02T15:40:56.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.534+0000 7ff95cc00700 1 --2- 172.21.15.1:0/1599940036 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55acab6de3a0 0x55acab6db620 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7ff94000ac00 tx=0x7ff940005ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3635c4b12f6ca0f server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:40:56.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.534+0000 7ff946a00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:40:56.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.534+0000 7ff946a00700 1 -- 172.21.15.1:0/1599940036 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff940026360 con 0x55acab6de3a0 2024-02-02T15:40:56.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.534+0000 7ff946a00700 1 -- 172.21.15.1:0/1599940036 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 9453+0+0 (secure 0 0 0) 0x7ff940003680 con 0x55acab6de3a0 2024-02-02T15:40:56.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.534+0000 7ff973b89b80 1 -- 172.21.15.1:0/1599940036 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55acab6bf0f0 con 0x55acab6de3a0 2024-02-02T15:40:56.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.535+0000 7ff946a00700 1 -- 172.21.15.1:0/1599940036 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff940003880 con 0x55acab6de3a0 2024-02-02T15:40:56.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.535+0000 7ff973b89b80 10 client.29384 fetch_fsmap learned FSMap version 59 2024-02-02T15:40:56.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.535+0000 7ff973b89b80 1 -- 172.21.15.1:0/1599940036 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) v3 -- 0x55acab5e2ea0 con 0x55acab6de3a0 2024-02-02T15:40:56.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.535+0000 7ff946a00700 1 -- 172.21.15.1:0/1599940036 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55acab6bf0f0 con 0x55acab6de3a0 2024-02-02T15:40:56.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.535+0000 7ff946a00700 1 -- 172.21.15.1:0/1599940036 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 59) v1 ==== 7538+0+0 (secure 0 0 0) 0x7ff94000f3c0 con 0x55acab6de3a0 2024-02-02T15:40:56.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.536+0000 7ff973b89b80 10 client.29384 fetch_fsmap finished waiting for FSMap version 59 2024-02-02T15:40:56.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.536+0000 7ff973b89b80 10 client.29384 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:40:56.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.536+0000 7ff973b89b80 20 client.29384 populate_metadata read hostname 'smithi001' 2024-02-02T15:40:56.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.536+0000 7ff973b89b80 1 --2- 172.21.15.1:0/1599940036 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55acab749ba0 0x55acab74bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:40:56.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.536+0000 7ff973b89b80 4 client.29384 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2024-02-02T15:40:56.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.536+0000 7ff973b89b80 1 -- 172.21.15.1:0/1599940036 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55acab5dbf50 con 0x55acab749ba0 2024-02-02T15:40:56.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.537+0000 7ff95d600700 1 --2- 172.21.15.1:0/1599940036 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55acab749ba0 0x55acab74bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:40:56.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.538+0000 7ff95d600700 1 --2- 172.21.15.1:0/1599940036 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55acab749ba0 0x55acab74bfc0 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77d0033093c480a9 server_cookie=6b36ee4699d0aa19 in_seq=0 out_seq=0 2024-02-02T15:40:56.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.538+0000 7ff946a00700 10 client.29384 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:40:56.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.541+0000 7ff946a00700 1 -- 172.21.15.1:0/1599940036 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55acab5dbf50 con 0x55acab749ba0 2024-02-02T15:40:56.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.541+0000 7ff946a00700 10 client.29384 handle_command_reply: tid=0 2024-02-02T15:40:56.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.541+0000 7ff973b89b80 1 client.29384 shutdown 2024-02-02T15:40:56.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.542+0000 7ff973b89b80 1 -- 172.21.15.1:0/1599940036 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55acab6de3a0 msgr2=0x55acab6db620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:56.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.542+0000 7ff973b89b80 1 --2- 172.21.15.1:0/1599940036 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55acab6de3a0 0x55acab6db620 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7ff94000ac00 tx=0x7ff940005ce0 comp rx=0 tx=0).stop 2024-02-02T15:40:56.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.542+0000 7ff973b89b80 1 -- 172.21.15.1:0/1599940036 shutdown_connections 2024-02-02T15:40:56.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.542+0000 7ff973b89b80 1 --2- 172.21.15.1:0/1599940036 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55acab749ba0 0x55acab74bfc0 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:56.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.542+0000 7ff973b89b80 1 --2- 172.21.15.1:0/1599940036 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55acab6deff0 0x55acab6dbb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:56.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.542+0000 7ff946a00700 0 client.29384 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:40:56.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.542+0000 7ff973b89b80 1 --2- 172.21.15.1:0/1599940036 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55acab6cb010 0x55acab6db0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:56.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.542+0000 7ff973b89b80 1 --2- 172.21.15.1:0/1599940036 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55acab6de3a0 0x55acab6db620 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:56.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.542+0000 7ff973b89b80 1 -- 172.21.15.1:0/1599940036 >> 172.21.15.1:0/1599940036 conn(0x55acab62adb0 msgr2=0x55acab6ceb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:56.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.543+0000 7ff973b89b80 1 -- 172.21.15.1:0/1599940036 shutdown_connections 2024-02-02T15:40:56.543 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.543+0000 7ff973b89b80 1 -- 172.21.15.1:0/1599940036 wait complete. 2024-02-02T15:40:56.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.543+0000 7ff973b89b80 20 client.29384 trim_cache size 0 max 16384 2024-02-02T15:40:56.544 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:40:56.544 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-02T15:40:56.544 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2024-02-02T15:40:56.544 INFO:teuthology.orchestra.run.smithi001.stdout: "b678ed4d-56e4-4933-ab32-28b6e1d47a6a": { 2024-02-02T15:40:56.544 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2024-02-02T15:40:56.544 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "b678ed4d-56e4-4933-ab32-28b6e1d47a6a", 2024-02-02T15:40:56.544 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,force" 2024-02-02T15:40:56.544 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:40:56.545 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:40:56.545 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:40:56.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.545+0000 7ff960000700 1 -- 172.21.15.1:0/3073189954 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff9380a07e0 msgr2=0x7ff9380a2ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:56.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.545+0000 7ff960000700 1 --2- 172.21.15.1:0/3073189954 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff9380a07e0 0x7ff9380a2ca0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff94800b5c0 tx=0x7ff948005fb0 comp rx=0 tx=0).stop 2024-02-02T15:40:56.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.545+0000 7ff960000700 1 -- 172.21.15.1:0/3073189954 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff95806b1e0 msgr2=0x7ff95806e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:40:56.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.545+0000 7ff960000700 1 --2- 172.21.15.1:0/3073189954 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff95806b1e0 0x7ff95806e830 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7ff950007ee0 tx=0x7ff95000d3b0 comp rx=0 tx=0).stop 2024-02-02T15:40:56.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.546+0000 7ff960000700 1 -- 172.21.15.1:0/3073189954 shutdown_connections 2024-02-02T15:40:56.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.546+0000 7ff960000700 1 --2- 172.21.15.1:0/3073189954 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff9380a07e0 0x7ff9380a2ca0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:56.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.546+0000 7ff960000700 1 --2- 172.21.15.1:0/3073189954 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff95806b1e0 0x7ff95806e830 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:56.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.546+0000 7ff960000700 1 --2- 172.21.15.1:0/3073189954 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff958101010 0x7ff95806a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:56.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.546+0000 7ff960000700 1 --2- 172.21.15.1:0/3073189954 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff958106130 0x7ff95806aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:40:56.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.546+0000 7ff960000700 1 -- 172.21.15.1:0/3073189954 >> 172.21.15.1:0/3073189954 conn(0x7ff9580fc8b0 msgr2=0x7ff9580fcf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:40:56.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.546+0000 7ff960000700 1 -- 172.21.15.1:0/3073189954 shutdown_connections 2024-02-02T15:40:56.547 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:40:56.546+0000 7ff960000700 1 -- 172.21.15.1:0/3073189954 wait complete. 2024-02-02T15:40:56.573 INFO:tasks.cephfs.filesystem:scrub status for tag:b678ed4d-56e4-4933-ab32-28b6e1d47a6a - {'path': '/', 'tag': 'b678ed4d-56e4-4933-ab32-28b6e1d47a6a', 'options': 'recursive,force'} 2024-02-02T15:40:56.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:56 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:40:56.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:56 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:40:56.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:56 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:40:56.968 INFO:tasks.workunit.client.0.smithi001.stdout: 10 13089 29.78 MB/sec warmup 29 sec latency 98.058 ms 2024-02-02T15:40:57.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:57 smithi071 ceph-mon[44707]: scrub queued for path: / 2024-02-02T15:40:57.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:57 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:40:57.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:57 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:40:57.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:57 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:40:57.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:57 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:40:57.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:57 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:40:57.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:57 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:40:57.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:57 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:40:57.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:57 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:40:57.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:57 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:40:57.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:57 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:40:57.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:57 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:40:57.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:57 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:40:57.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:57 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:40:57.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:57 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:40:57.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:57 smithi071 ceph-mon[44707]: pgmap v435: 129 pgs: 129 active+clean; 911 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 5.6 MiB/s wr, 441 op/s 2024-02-02T15:40:57.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:57 smithi142 ceph-mon[45728]: scrub queued for path: / 2024-02-02T15:40:57.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:57 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:40:57.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:57 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:40:57.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:57 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:40:57.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:57 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:40:57.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:57 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:40:57.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:57 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:40:57.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:57 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:40:57.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:57 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:40:57.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:57 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:40:57.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:57 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:40:57.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:57 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:40:57.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:57 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:40:57.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:57 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:40:57.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:57 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:40:57.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:57 smithi142 ceph-mon[45728]: pgmap v435: 129 pgs: 129 active+clean; 911 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 5.6 MiB/s wr, 441 op/s 2024-02-02T15:40:57.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:57 smithi001 ceph-mon[35470]: scrub queued for path: / 2024-02-02T15:40:57.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:57 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:40:57.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:57 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:40:57.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:57 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:40:57.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:57 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:40:57.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:57 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:40:57.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:57 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:40:57.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:57 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:40:57.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:57 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:40:57.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:57 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:40:57.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:57 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:40:57.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:57 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:40:57.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:57 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:40:57.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:57 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:40:57.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:57 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:40:57.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:57 smithi001 ceph-mon[35470]: pgmap v435: 129 pgs: 129 active+clean; 911 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 5.6 MiB/s wr, 441 op/s 2024-02-02T15:40:57.968 INFO:tasks.workunit.client.0.smithi001.stdout: 10 14177 31.48 MB/sec warmup 30 sec latency 48.311 ms 2024-02-02T15:40:58.969 INFO:tasks.workunit.client.0.smithi001.stdout: 10 14728 31.56 MB/sec warmup 31 sec latency 47.285 ms 2024-02-02T15:40:59.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:40:59 smithi071 ceph-mon[44707]: pgmap v436: 129 pgs: 129 active+clean; 911 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 4.3 MiB/s wr, 336 op/s 2024-02-02T15:40:59.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:40:59 smithi142 ceph-mon[45728]: pgmap v436: 129 pgs: 129 active+clean; 911 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 4.3 MiB/s wr, 336 op/s 2024-02-02T15:40:59.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:40:59 smithi001 ceph-mon[35470]: pgmap v436: 129 pgs: 129 active+clean; 911 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 4.3 MiB/s wr, 336 op/s 2024-02-02T15:40:59.968 INFO:tasks.workunit.client.0.smithi001.stdout: 10 15297 31.15 MB/sec warmup 32 sec latency 74.136 ms 2024-02-02T15:41:00.969 INFO:tasks.workunit.client.0.smithi001.stdout: 10 16165 31.34 MB/sec warmup 33 sec latency 43.646 ms 2024-02-02T15:41:01.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:41:01 smithi071 ceph-mon[44707]: pgmap v437: 129 pgs: 129 active+clean; 911 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 4.3 MiB/s wr, 336 op/s 2024-02-02T15:41:01.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:41:01 smithi142 ceph-mon[45728]: pgmap v437: 129 pgs: 129 active+clean; 911 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 4.3 MiB/s wr, 336 op/s 2024-02-02T15:41:01.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:41:01 smithi001 ceph-mon[35470]: pgmap v437: 129 pgs: 129 active+clean; 911 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 4.3 MiB/s wr, 336 op/s 2024-02-02T15:41:01.969 INFO:tasks.workunit.client.0.smithi001.stdout: 10 16728 31.07 MB/sec warmup 34 sec latency 485.203 ms 2024-02-02T15:41:02.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:41:02 smithi071 ceph-mon[44707]: osdmap e104: 12 total, 12 up, 12 in 2024-02-02T15:41:02.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:41:02 smithi142 ceph-mon[45728]: osdmap e104: 12 total, 12 up, 12 in 2024-02-02T15:41:02.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:41:02 smithi001 ceph-mon[35470]: osdmap e104: 12 total, 12 up, 12 in 2024-02-02T15:41:02.969 INFO:tasks.workunit.client.0.smithi001.stdout: 10 16760 30.31 MB/sec warmup 35 sec latency 1485.261 ms 2024-02-02T15:41:03.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:41:03 smithi071 ceph-mon[44707]: pgmap v439: 129 pgs: 129 active+clean; 941 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 6.5 MiB/s wr, 562 op/s 2024-02-02T15:41:03.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:41:03 smithi142 ceph-mon[45728]: pgmap v439: 129 pgs: 129 active+clean; 941 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 6.5 MiB/s wr, 562 op/s 2024-02-02T15:41:03.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:41:03 smithi001 ceph-mon[35470]: pgmap v439: 129 pgs: 129 active+clean; 941 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 6.5 MiB/s wr, 562 op/s 2024-02-02T15:41:03.969 INFO:tasks.workunit.client.0.smithi001.stdout: 10 17011 30.03 MB/sec warmup 36 sec latency 2485.306 ms 2024-02-02T15:41:04.969 INFO:tasks.workunit.client.0.smithi001.stdout: 10 17696 30.71 MB/sec warmup 37 sec latency 3485.358 ms 2024-02-02T15:41:05.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:41:05 smithi071 ceph-mon[44707]: pgmap v440: 129 pgs: 129 active+clean; 941 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 6.5 MiB/s wr, 563 op/s 2024-02-02T15:41:05.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:41:05 smithi142 ceph-mon[45728]: pgmap v440: 129 pgs: 129 active+clean; 941 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 6.5 MiB/s wr, 563 op/s 2024-02-02T15:41:05.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:41:05 smithi001 ceph-mon[35470]: pgmap v440: 129 pgs: 129 active+clean; 941 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 6.5 MiB/s wr, 563 op/s 2024-02-02T15:41:05.969 INFO:tasks.workunit.client.0.smithi001.stdout: 10 17989 30.25 MB/sec warmup 38 sec latency 4485.408 ms 2024-02-02T15:41:06.969 INFO:tasks.workunit.client.0.smithi001.stdout: 10 18372 29.83 MB/sec warmup 39 sec latency 5485.487 ms 2024-02-02T15:41:07.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:41:07 smithi071 ceph-mon[44707]: pgmap v441: 129 pgs: 129 active+clean; 1001 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 725 op/s 2024-02-02T15:41:07.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:41:07 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:41:07.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:41:07 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:41:07.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:41:07 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:41:07.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:41:07 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:41:07.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:41:07 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:41:07.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:41:07 smithi142 ceph-mon[45728]: pgmap v441: 129 pgs: 129 active+clean; 1001 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 725 op/s 2024-02-02T15:41:07.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:41:07 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:41:07.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:41:07 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:41:07.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:41:07 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:41:07.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:41:07 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:41:07.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:41:07 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:41:07.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:41:07 smithi001 ceph-mon[35470]: pgmap v441: 129 pgs: 129 active+clean; 1001 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 725 op/s 2024-02-02T15:41:07.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:41:07 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:41:07.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:41:07 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:41:07.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:41:07 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:41:07.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:41:07 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:41:07.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:41:07 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:41:07.970 INFO:tasks.workunit.client.0.smithi001.stdout: 10 19042 29.70 MB/sec warmup 40 sec latency 6485.929 ms 2024-02-02T15:41:08.970 INFO:tasks.workunit.client.0.smithi001.stdout: 10 20046 30.54 MB/sec warmup 41 sec latency 7486.001 ms 2024-02-02T15:41:09.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:41:09 smithi071 ceph-mon[44707]: pgmap v442: 129 pgs: 129 active+clean; 1001 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 725 op/s 2024-02-02T15:41:09.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:41:09 smithi142 ceph-mon[45728]: pgmap v442: 129 pgs: 129 active+clean; 1001 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 725 op/s 2024-02-02T15:41:09.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:41:09 smithi001 ceph-mon[35470]: pgmap v442: 129 pgs: 129 active+clean; 1001 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 725 op/s 2024-02-02T15:41:09.970 INFO:tasks.workunit.client.0.smithi001.stdout: 10 20794 30.82 MB/sec warmup 42 sec latency 8486.088 ms 2024-02-02T15:41:10.970 INFO:tasks.workunit.client.0.smithi001.stdout: 10 21423 30.95 MB/sec warmup 43 sec latency 9486.148 ms 2024-02-02T15:41:11.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:41:11 smithi071 ceph-mon[44707]: pgmap v443: 129 pgs: 129 active+clean; 1001 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 725 op/s 2024-02-02T15:41:11.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:41:11 smithi071 ceph-mon[44707]: osdmap e105: 12 total, 12 up, 12 in 2024-02-02T15:41:11.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:41:11 smithi142 ceph-mon[45728]: pgmap v443: 129 pgs: 129 active+clean; 1001 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 725 op/s 2024-02-02T15:41:11.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:41:11 smithi142 ceph-mon[45728]: osdmap e105: 12 total, 12 up, 12 in 2024-02-02T15:41:11.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:41:11 smithi001 ceph-mon[35470]: pgmap v443: 129 pgs: 129 active+clean; 1001 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 725 op/s 2024-02-02T15:41:11.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:41:11 smithi001 ceph-mon[35470]: osdmap e105: 12 total, 12 up, 12 in 2024-02-02T15:41:11.970 INFO:tasks.workunit.client.0.smithi001.stdout: 10 21541 30.39 MB/sec warmup 44 sec latency 10486.213 ms 2024-02-02T15:41:12.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:41:12 smithi071 ceph-mon[44707]: pgmap v445: 129 pgs: 129 active+clean; 1018 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 14 MiB/s wr, 779 op/s 2024-02-02T15:41:12.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:41:12 smithi142 ceph-mon[45728]: pgmap v445: 129 pgs: 129 active+clean; 1018 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 14 MiB/s wr, 779 op/s 2024-02-02T15:41:12.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:41:12 smithi001 ceph-mon[35470]: pgmap v445: 129 pgs: 129 active+clean; 1018 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 14 MiB/s wr, 779 op/s 2024-02-02T15:41:12.970 INFO:tasks.workunit.client.0.smithi001.stdout: 10 21827 29.97 MB/sec warmup 45 sec latency 10788.001 ms 2024-02-02T15:41:13.970 INFO:tasks.workunit.client.0.smithi001.stdout: 10 22283 29.67 MB/sec warmup 46 sec latency 400.084 ms 2024-02-02T15:41:14.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:41:14 smithi071 ceph-mon[44707]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:41:14.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:41:14 smithi071 ceph-mon[44707]: pgmap v446: 129 pgs: 129 active+clean; 1018 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 14 MiB/s wr, 779 op/s 2024-02-02T15:41:14.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:41:14 smithi142 ceph-mon[45728]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:41:14.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:41:14 smithi142 ceph-mon[45728]: pgmap v446: 129 pgs: 129 active+clean; 1018 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 14 MiB/s wr, 779 op/s 2024-02-02T15:41:14.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:41:14 smithi001 ceph-mon[35470]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:41:14.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:41:14 smithi001 ceph-mon[35470]: pgmap v446: 129 pgs: 129 active+clean; 1018 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 14 MiB/s wr, 779 op/s 2024-02-02T15:41:14.970 INFO:tasks.workunit.client.0.smithi001.stdout: 10 23177 30.28 MB/sec warmup 47 sec latency 1400.133 ms 2024-02-02T15:41:15.970 INFO:tasks.workunit.client.0.smithi001.stdout: 10 23954 30.63 MB/sec warmup 48 sec latency 2400.248 ms 2024-02-02T15:41:16.970 INFO:tasks.workunit.client.0.smithi001.stdout: 10 24490 30.59 MB/sec warmup 49 sec latency 3400.302 ms 2024-02-02T15:41:17.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:41:17 smithi071 ceph-mon[44707]: pgmap v447: 129 pgs: 129 active+clean; 1019 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 664 op/s 2024-02-02T15:41:17.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:41:17 smithi142 ceph-mon[45728]: pgmap v447: 129 pgs: 129 active+clean; 1019 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 664 op/s 2024-02-02T15:41:17.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:41:17 smithi001 ceph-mon[35470]: pgmap v447: 129 pgs: 129 active+clean; 1019 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 664 op/s 2024-02-02T15:41:17.970 INFO:tasks.workunit.client.0.smithi001.stdout: 10 25157 30.54 MB/sec warmup 50 sec latency 4400.362 ms 2024-02-02T15:41:18.971 INFO:tasks.workunit.client.0.smithi001.stdout: 10 26111 31.05 MB/sec warmup 51 sec latency 5400.421 ms 2024-02-02T15:41:19.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:41:19 smithi071 ceph-mon[44707]: pgmap v448: 129 pgs: 129 active+clean; 1007 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 16 MiB/s wr, 846 op/s 2024-02-02T15:41:19.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:41:19 smithi142 ceph-mon[45728]: pgmap v448: 129 pgs: 129 active+clean; 1007 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 16 MiB/s wr, 846 op/s 2024-02-02T15:41:19.939 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:41:19 smithi001 ceph-mon[35470]: pgmap v448: 129 pgs: 129 active+clean; 1007 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 16 MiB/s wr, 846 op/s 2024-02-02T15:41:19.970 INFO:tasks.workunit.client.0.smithi001.stdout: 10 26919 31.27 MB/sec warmup 52 sec latency 6400.474 ms 2024-02-02T15:41:20.971 INFO:tasks.workunit.client.0.smithi001.stdout: 10 27579 31.49 MB/sec warmup 53 sec latency 7400.526 ms 2024-02-02T15:41:21.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:41:21 smithi071 ceph-mon[44707]: pgmap v449: 129 pgs: 129 active+clean; 1007 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 16 MiB/s wr, 846 op/s 2024-02-02T15:41:21.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:41:21 smithi142 ceph-mon[45728]: pgmap v449: 129 pgs: 129 active+clean; 1007 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 16 MiB/s wr, 846 op/s 2024-02-02T15:41:21.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:41:21 smithi001 ceph-mon[35470]: pgmap v449: 129 pgs: 129 active+clean; 1007 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 16 MiB/s wr, 846 op/s 2024-02-02T15:41:21.970 INFO:tasks.workunit.client.0.smithi001.stdout: 10 28267 31.53 MB/sec warmup 54 sec latency 8400.576 ms 2024-02-02T15:41:22.971 INFO:tasks.workunit.client.0.smithi001.stdout: 10 28932 31.41 MB/sec warmup 55 sec latency 9270.282 ms 2024-02-02T15:41:23.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:41:23 smithi071 ceph-mon[44707]: pgmap v450: 129 pgs: 129 active+clean; 1.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 14 MiB/s wr, 782 op/s 2024-02-02T15:41:23.923 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:41:23 smithi142 ceph-mon[45728]: pgmap v450: 129 pgs: 129 active+clean; 1.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 14 MiB/s wr, 782 op/s 2024-02-02T15:41:23.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:41:23 smithi001 ceph-mon[35470]: pgmap v450: 129 pgs: 129 active+clean; 1.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 14 MiB/s wr, 782 op/s 2024-02-02T15:41:23.971 INFO:tasks.workunit.client.0.smithi001.stdout: 10 29713 31.66 MB/sec warmup 56 sec latency 58.900 ms 2024-02-02T15:41:24.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:41:24 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:41:24.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:41:24 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:41:24.939 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:41:24 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:41:24.971 INFO:tasks.workunit.client.0.smithi001.stdout: 10 30510 31.94 MB/sec warmup 57 sec latency 55.976 ms 2024-02-02T15:41:25.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:41:25 smithi071 ceph-mon[44707]: pgmap v451: 129 pgs: 129 active+clean; 1.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 644 op/s 2024-02-02T15:41:25.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:41:25 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:41:25.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:41:25 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:41:25.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:41:25 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:41:25.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:41:25 smithi142 ceph-mon[45728]: pgmap v451: 129 pgs: 129 active+clean; 1.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 644 op/s 2024-02-02T15:41:25.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:41:25 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:41:25.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:41:25 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:41:25.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:41:25 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:41:25.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:41:25 smithi001 ceph-mon[35470]: pgmap v451: 129 pgs: 129 active+clean; 1.0 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 644 op/s 2024-02-02T15:41:25.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:41:25 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:41:25.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:41:25 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:41:25.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:41:25 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:41:25.971 INFO:tasks.workunit.client.0.smithi001.stdout: 10 31266 32.05 MB/sec warmup 58 sec latency 47.842 ms 2024-02-02T15:41:26.574 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-02T15:41:26.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.882+0000 7f9a7ce00700 1 -- 172.21.15.1:0/3744062385 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9a78101000 msgr2=0x7f9a781013e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:41:26.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.882+0000 7f9a7ce00700 1 --2- 172.21.15.1:0/3744062385 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9a78101000 0x7f9a781013e0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f9a68009990 tx=0x7f9a68009ca0 comp rx=0 tx=0).stop 2024-02-02T15:41:26.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.883+0000 7f9a7ce00700 1 -- 172.21.15.1:0/3744062385 shutdown_connections 2024-02-02T15:41:26.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.883+0000 7f9a7ce00700 1 --2- 172.21.15.1:0/3744062385 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a78106120 0x7f9a7810dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:26.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.883+0000 7f9a7ce00700 1 --2- 172.21.15.1:0/3744062385 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9a78101000 0x7f9a781013e0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:26.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.883+0000 7f9a7ce00700 1 --2- 172.21.15.1:0/3744062385 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a781019b0 0x7f9a781059f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:26.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.883+0000 7f9a7ce00700 1 -- 172.21.15.1:0/3744062385 >> 172.21.15.1:0/3744062385 conn(0x7f9a780fc840 msgr2=0x7f9a780fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:41:26.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.883+0000 7f9a7ce00700 1 -- 172.21.15.1:0/3744062385 shutdown_connections 2024-02-02T15:41:26.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.884+0000 7f9a7ce00700 1 -- 172.21.15.1:0/3744062385 wait complete. 2024-02-02T15:41:26.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.886+0000 7f9a7ce00700 1 Processor -- start 2024-02-02T15:41:26.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.887+0000 7f9a7ce00700 1 -- start start 2024-02-02T15:41:26.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.887+0000 7f9a7ce00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9a78101000 0x7f9a78196f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:26.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.887+0000 7f9a7ce00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a78106120 0x7f9a78197490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:26.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.887+0000 7f9a7ce00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a7819b8f0 0x7f9a7819bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:26.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.887+0000 7f9a7ce00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7819c370 con 0x7f9a7819b8f0 2024-02-02T15:41:26.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.888+0000 7f9a7ce00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7819c4b0 con 0x7f9a78101000 2024-02-02T15:41:26.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.888+0000 7f9a7ce00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7819c620 con 0x7f9a78106120 2024-02-02T15:41:26.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.888+0000 7f9a75400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a78106120 0x7f9a78197490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:26.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.888+0000 7f9a75400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a78106120 0x7f9a78197490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:35314/0 (socket says 172.21.15.1:35314) 2024-02-02T15:41:26.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.888+0000 7f9a75400700 1 -- 172.21.15.1:0/959908627 learned_addr learned my addr 172.21.15.1:0/959908627 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:41:26.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.889+0000 7f9a76800700 1 --2- 172.21.15.1:0/959908627 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a7819b8f0 0x7f9a7819bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:26.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.889+0000 7f9a75400700 1 -- 172.21.15.1:0/959908627 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9a78101000 msgr2=0x7f9a78196f50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-02T15:41:26.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.889+0000 7f9a75400700 1 --2- 172.21.15.1:0/959908627 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9a78101000 0x7f9a78196f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:26.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.889+0000 7f9a75400700 1 -- 172.21.15.1:0/959908627 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a7819b8f0 msgr2=0x7f9a7819bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:41:26.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.889+0000 7f9a75400700 1 --2- 172.21.15.1:0/959908627 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a7819b8f0 0x7f9a7819bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:26.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.889+0000 7f9a75400700 1 -- 172.21.15.1:0/959908627 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a70009c40 con 0x7f9a78106120 2024-02-02T15:41:26.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.889+0000 7f9a75400700 1 --2- 172.21.15.1:0/959908627 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a78106120 0x7f9a78197490 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f9a6000d980 tx=0x7f9a6000dd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:41:26.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.890+0000 7f9a6ea00700 1 -- 172.21.15.1:0/959908627 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a60009b40 con 0x7f9a78106120 2024-02-02T15:41:26.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.890+0000 7f9a6ea00700 1 -- 172.21.15.1:0/959908627 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a6000de60 con 0x7f9a78106120 2024-02-02T15:41:26.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.890+0000 7f9a6ea00700 1 -- 172.21.15.1:0/959908627 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a6000f620 con 0x7f9a78106120 2024-02-02T15:41:26.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.891+0000 7f9a7ce00700 1 -- 172.21.15.1:0/959908627 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a680095f0 con 0x7f9a78106120 2024-02-02T15:41:26.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.891+0000 7f9a7ce00700 1 -- 172.21.15.1:0/959908627 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a7806aa90 con 0x7f9a78106120 2024-02-02T15:41:26.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.896+0000 7f9a6ea00700 1 -- 172.21.15.1:0/959908627 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f9a60010460 con 0x7f9a78106120 2024-02-02T15:41:26.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.898+0000 7f9a6ea00700 1 --2- 172.21.15.1:0/959908627 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f9a580a0860 0x7f9a580a2d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:26.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.898+0000 7f9a6ea00700 1 -- 172.21.15.1:0/959908627 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f9a600cb720 con 0x7f9a78106120 2024-02-02T15:41:26.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.898+0000 7f9a908a9b80 1 Processor -- start 2024-02-02T15:41:26.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.899+0000 7f9a75e00700 1 --2- 172.21.15.1:0/959908627 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f9a580a0860 0x7f9a580a2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:26.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.899+0000 7f9a908a9b80 1 -- start start 2024-02-02T15:41:26.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.900+0000 7f9a908a9b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5567ae6d3010 0x5567ae6d33f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:26.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.900+0000 7f9a908a9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567ae6d3930 0x5567ae6e26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:26.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.900+0000 7f9a908a9b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5567ae6e2bf0 0x5567ae6e4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:26.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.900+0000 7f9a908a9b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567ae6474c0 con 0x5567ae6e2bf0 2024-02-02T15:41:26.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.900+0000 7f9a908a9b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567ae62baa0 con 0x5567ae6d3010 2024-02-02T15:41:26.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.900+0000 7f9a908a9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567ae629460 con 0x5567ae6d3930 2024-02-02T15:41:26.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.900+0000 7f9a75400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567ae6d3930 0x5567ae6e26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:26.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.900+0000 7f9a75400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567ae6d3930 0x5567ae6e26b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:35324/0 (socket says 172.21.15.1:35324) 2024-02-02T15:41:26.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.900+0000 7f9a75400700 1 -- 172.21.15.1:0/1152672210 learned_addr learned my addr 172.21.15.1:0/1152672210 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:41:26.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.901+0000 7f9a75e00700 1 --2- 172.21.15.1:0/1152672210 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5567ae6d3010 0x5567ae6d33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:26.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.901+0000 7f9a75400700 1 -- 172.21.15.1:0/1152672210 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5567ae6d3010 msgr2=0x5567ae6d33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:41:26.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.901+0000 7f9a75400700 1 --2- 172.21.15.1:0/1152672210 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5567ae6d3010 0x5567ae6d33f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:26.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.901+0000 7f9a75400700 1 -- 172.21.15.1:0/1152672210 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5567ae6e2bf0 msgr2=0x5567ae6e4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:41:26.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.901+0000 7f9a76800700 1 --2- 172.21.15.1:0/1152672210 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5567ae6e2bf0 0x5567ae6e4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:26.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.901+0000 7f9a75400700 1 --2- 172.21.15.1:0/1152672210 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5567ae6e2bf0 0x5567ae6e4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:26.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.901+0000 7f9a75400700 1 -- 172.21.15.1:0/1152672210 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5567ae4a63b0 con 0x5567ae6d3930 2024-02-02T15:41:26.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.901+0000 7f9a75e00700 1 --2- 172.21.15.1:0/1152672210 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5567ae6d3010 0x5567ae6d33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:41:26.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.901+0000 7f9a75e00700 1 --2- 172.21.15.1:0/959908627 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f9a580a0860 0x7f9a580a2d20 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9a6800b2e0 tx=0x7f9a68000bc0 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:41:26.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.901+0000 7f9a76800700 1 --2- 172.21.15.1:0/1152672210 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5567ae6e2bf0 0x5567ae6e4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:41:26.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.902+0000 7f9a75400700 1 --2- 172.21.15.1:0/1152672210 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567ae6d3930 0x5567ae6e26b0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f9a60005cf0 tx=0x7f9a60099040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de9df35b2d335e3 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:41:26.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.902+0000 7f9a67400700 1 -- 172.21.15.1:0/1152672210 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a6009c6f0 con 0x5567ae6d3930 2024-02-02T15:41:26.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.903+0000 7f9a67400700 1 -- 172.21.15.1:0/1152672210 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a600a0930 con 0x5567ae6d3930 2024-02-02T15:41:26.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.903+0000 7f9a67400700 1 -- 172.21.15.1:0/1152672210 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a60097db0 con 0x5567ae6d3930 2024-02-02T15:41:26.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.904+0000 7f9a908a9b80 1 -- 172.21.15.1:0/1152672210 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567ae6d3930 msgr2=0x5567ae6e26b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:41:26.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.904+0000 7f9a908a9b80 1 --2- 172.21.15.1:0/1152672210 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567ae6d3930 0x5567ae6e26b0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f9a60005cf0 tx=0x7f9a60099040 comp rx=0 tx=0).stop 2024-02-02T15:41:26.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.904+0000 7f9a908a9b80 1 -- 172.21.15.1:0/1152672210 shutdown_connections 2024-02-02T15:41:26.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.904+0000 7f9a908a9b80 1 --2- 172.21.15.1:0/1152672210 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5567ae6e2bf0 0x5567ae6e4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:26.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.904+0000 7f9a908a9b80 1 --2- 172.21.15.1:0/1152672210 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5567ae6d3010 0x5567ae6d33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:26.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.904+0000 7f9a908a9b80 1 --2- 172.21.15.1:0/1152672210 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567ae6d3930 0x5567ae6e26b0 secure :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f9a60005cf0 tx=0x7f9a60099040 comp rx=0 tx=0).stop 2024-02-02T15:41:26.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.904+0000 7f9a908a9b80 1 -- 172.21.15.1:0/1152672210 >> 172.21.15.1:0/1152672210 conn(0x5567ae632db0 msgr2=0x5567ae6cf3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:41:26.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.905+0000 7f9a908a9b80 1 -- 172.21.15.1:0/1152672210 shutdown_connections 2024-02-02T15:41:26.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.905+0000 7f9a908a9b80 1 -- 172.21.15.1:0/1152672210 wait complete. 2024-02-02T15:41:26.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.906+0000 7f9a908a9b80 1 -- start start 2024-02-02T15:41:26.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.906+0000 7f9a908a9b80 1 Processor -- start 2024-02-02T15:41:26.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.907+0000 7f9a908a9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567ae6d3010 0x5567ae6dfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:26.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.907+0000 7f9a908a9b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5567ae6e2bf0 0x5567ae6dffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:26.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.907+0000 7f9a908a9b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5567ae6e0520 0x5567ae6ee770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:26.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.907+0000 7f9a75400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5567ae6e2bf0 0x5567ae6dffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:26.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.907+0000 7f9a908a9b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567ae624d50 con 0x5567ae6e2bf0 2024-02-02T15:41:26.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.907+0000 7f9a75400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5567ae6e2bf0 0x5567ae6dffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33092/0 (socket says 172.21.15.1:33092) 2024-02-02T15:41:26.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.907+0000 7f9a908a9b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567ae5c0d80 con 0x5567ae6e0520 2024-02-02T15:41:26.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.907+0000 7f9a75400700 1 -- 172.21.15.1:0/618971733 learned_addr learned my addr 172.21.15.1:0/618971733 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:41:26.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.907+0000 7f9a908a9b80 1 -- 172.21.15.1:0/618971733 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567ae5c0a20 con 0x5567ae6d3010 2024-02-02T15:41:26.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.907+0000 7f9a76800700 1 --2- 172.21.15.1:0/618971733 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5567ae6e0520 0x5567ae6ee770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:26.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.907+0000 7f9a908a9b80 10 client.0 _ll_register_callbacks cb 0x5567ae5c2290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:41:26.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.908+0000 7f9a75e00700 1 --2- 172.21.15.1:0/618971733 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567ae6d3010 0x5567ae6dfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:26.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.908+0000 7f9a75400700 1 -- 172.21.15.1:0/618971733 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567ae6d3010 msgr2=0x5567ae6dfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:41:26.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.908+0000 7f9a75400700 1 --2- 172.21.15.1:0/618971733 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567ae6d3010 0x5567ae6dfaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:26.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.908+0000 7f9a75400700 1 -- 172.21.15.1:0/618971733 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5567ae6e0520 msgr2=0x5567ae6ee770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:41:26.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.908+0000 7f9a75400700 1 --2- 172.21.15.1:0/618971733 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5567ae6e0520 0x5567ae6ee770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:26.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.908+0000 7f9a75400700 1 -- 172.21.15.1:0/618971733 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a7806aa90 con 0x5567ae6e2bf0 2024-02-02T15:41:26.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.909+0000 7f9a75400700 1 --2- 172.21.15.1:0/618971733 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5567ae6e2bf0 0x5567ae6dffe0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f9a600c9070 tx=0x7f9a6009c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d1a45befb4a9e5e server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:41:26.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.909+0000 7f9a5fe00700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:41:26.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.910+0000 7f9a5fe00700 1 -- 172.21.15.1:0/618971733 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a600cb4f0 con 0x5567ae6e2bf0 2024-02-02T15:41:26.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.910+0000 7f9a5fe00700 1 -- 172.21.15.1:0/618971733 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f9a60097db0 con 0x5567ae6e2bf0 2024-02-02T15:41:26.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.910+0000 7f9a908a9b80 1 -- 172.21.15.1:0/618971733 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5567ae6c70f0 con 0x5567ae6e2bf0 2024-02-02T15:41:26.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.910+0000 7f9a5fe00700 1 -- 172.21.15.1:0/618971733 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a600ac120 con 0x5567ae6e2bf0 2024-02-02T15:41:26.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.910+0000 7f9a908a9b80 10 client.19248 fetch_fsmap learned FSMap version 60 2024-02-02T15:41:26.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.910+0000 7f9a908a9b80 1 -- 172.21.15.1:0/618971733 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) v3 -- 0x5567ae452b60 con 0x5567ae6e2bf0 2024-02-02T15:41:26.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.910+0000 7f9a5fe00700 1 -- 172.21.15.1:0/618971733 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5567ae6c70f0 con 0x5567ae6e2bf0 2024-02-02T15:41:26.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.910+0000 7f9a5fe00700 1 -- 172.21.15.1:0/618971733 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 60) v1 ==== 7542+0+0 (secure 0 0 0) 0x7f9a600a04a0 con 0x5567ae6e2bf0 2024-02-02T15:41:26.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.911+0000 7f9a908a9b80 10 client.19248 fetch_fsmap finished waiting for FSMap version 60 2024-02-02T15:41:26.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.911+0000 7f9a908a9b80 10 client.19248 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:41:26.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.911+0000 7f9a908a9b80 20 client.19248 populate_metadata read hostname 'smithi001' 2024-02-02T15:41:26.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.911+0000 7f9a908a9b80 1 --2- 172.21.15.1:0/618971733 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5567ae6d8830 0x5567ae6d8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:26.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.911+0000 7f9a908a9b80 4 client.19248 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:41:26.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.911+0000 7f9a908a9b80 1 -- 172.21.15.1:0/618971733 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5567ae61ddd0 con 0x5567ae6d8830 2024-02-02T15:41:26.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.912+0000 7f9a75e00700 1 --2- 172.21.15.1:0/618971733 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5567ae6d8830 0x5567ae6d8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:26.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.913+0000 7f9a75e00700 1 --2- 172.21.15.1:0/618971733 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5567ae6d8830 0x5567ae6d8c40 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14b72f583a2e4ffc server_cookie=d0db13e1fa83cc0d in_seq=0 out_seq=0 2024-02-02T15:41:26.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.913+0000 7f9a5fe00700 10 client.19248 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:41:26.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.916+0000 7f9a5fe00700 1 -- 172.21.15.1:0/618971733 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5567ae61ddd0 con 0x5567ae6d8830 2024-02-02T15:41:26.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.916+0000 7f9a5fe00700 10 client.19248 handle_command_reply: tid=0 2024-02-02T15:41:26.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.916+0000 7f9a908a9b80 1 client.19248 shutdown 2024-02-02T15:41:26.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.916+0000 7f9a908a9b80 1 -- 172.21.15.1:0/618971733 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5567ae6e2bf0 msgr2=0x5567ae6dffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:41:26.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.916+0000 7f9a908a9b80 1 --2- 172.21.15.1:0/618971733 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5567ae6e2bf0 0x5567ae6dffe0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f9a600c9070 tx=0x7f9a6009c830 comp rx=0 tx=0).stop 2024-02-02T15:41:26.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.916+0000 7f9a908a9b80 1 -- 172.21.15.1:0/618971733 shutdown_connections 2024-02-02T15:41:26.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.916+0000 7f9a908a9b80 1 --2- 172.21.15.1:0/618971733 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5567ae6d8830 0x5567ae6d8c40 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:26.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.917+0000 7f9a908a9b80 1 --2- 172.21.15.1:0/618971733 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5567ae6e0520 0x5567ae6ee770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:26.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.917+0000 7f9a908a9b80 1 --2- 172.21.15.1:0/618971733 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567ae6d3010 0x5567ae6dfaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:26.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.917+0000 7f9a5fe00700 0 client.19248 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:41:26.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.917+0000 7f9a908a9b80 1 --2- 172.21.15.1:0/618971733 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5567ae6e2bf0 0x5567ae6dffe0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:26.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.917+0000 7f9a908a9b80 1 -- 172.21.15.1:0/618971733 >> 172.21.15.1:0/618971733 conn(0x5567ae632db0 msgr2=0x5567ae6cef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:41:26.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.917+0000 7f9a908a9b80 1 -- 172.21.15.1:0/618971733 shutdown_connections 2024-02-02T15:41:26.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.917+0000 7f9a908a9b80 1 -- 172.21.15.1:0/618971733 wait complete. 2024-02-02T15:41:26.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.917+0000 7f9a908a9b80 20 client.19248 trim_cache size 0 max 16384 2024-02-02T15:41:26.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.955+0000 7f9a908a9b80 1 Processor -- start 2024-02-02T15:41:26.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.955+0000 7f9a908a9b80 1 -- start start 2024-02-02T15:41:26.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.956+0000 7f9a908a9b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5567ae6d3010 0x5567ae6ee6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:26.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.956+0000 7f9a908a9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567ae6e0520 0x5567ae6fe9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:26.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.956+0000 7f9a908a9b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5567ae6cae40 0x5567ae6d4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:26.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.956+0000 7f9a908a9b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567ae4a86a0 con 0x5567ae6cae40 2024-02-02T15:41:26.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.956+0000 7f9a908a9b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567ae6de050 con 0x5567ae6d3010 2024-02-02T15:41:26.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.956+0000 7f9a908a9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567ae6db8a0 con 0x5567ae6e0520 2024-02-02T15:41:26.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.956+0000 7f9a75400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567ae6e0520 0x5567ae6fe9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:26.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.956+0000 7f9a75400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567ae6e0520 0x5567ae6fe9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:35342/0 (socket says 172.21.15.1:35342) 2024-02-02T15:41:26.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.956+0000 7f9a75400700 1 -- 172.21.15.1:0/1976144299 learned_addr learned my addr 172.21.15.1:0/1976144299 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:41:26.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.956+0000 7f9a76800700 1 --2- 172.21.15.1:0/1976144299 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5567ae6cae40 0x5567ae6d4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:26.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.957+0000 7f9a75400700 1 -- 172.21.15.1:0/1976144299 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5567ae6d3010 msgr2=0x5567ae6ee6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:41:26.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.957+0000 7f9a75e00700 1 --2- 172.21.15.1:0/1976144299 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5567ae6d3010 0x5567ae6ee6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:26.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.957+0000 7f9a75400700 1 --2- 172.21.15.1:0/1976144299 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5567ae6d3010 0x5567ae6ee6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:26.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.957+0000 7f9a75400700 1 -- 172.21.15.1:0/1976144299 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5567ae6cae40 msgr2=0x5567ae6d4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:41:26.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.957+0000 7f9a75400700 1 --2- 172.21.15.1:0/1976144299 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5567ae6cae40 0x5567ae6d4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:26.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.958+0000 7f9a75400700 1 -- 172.21.15.1:0/1976144299 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5567ae6d4b00 con 0x5567ae6e0520 2024-02-02T15:41:26.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.958+0000 7f9a76800700 1 --2- 172.21.15.1:0/1976144299 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5567ae6cae40 0x5567ae6d4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:41:26.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.958+0000 7f9a75e00700 1 --2- 172.21.15.1:0/1976144299 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5567ae6d3010 0x5567ae6ee6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:41:26.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.958+0000 7f9a75400700 1 --2- 172.21.15.1:0/1976144299 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567ae6e0520 0x5567ae6fe9e0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f9a60095330 tx=0x7f9a60099040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c059dd1e9623dc1c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:41:26.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.959+0000 7f9a67e00700 1 -- 172.21.15.1:0/1976144299 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a600ac120 con 0x5567ae6e0520 2024-02-02T15:41:26.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.959+0000 7f9a67e00700 1 -- 172.21.15.1:0/1976144299 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a60099e60 con 0x5567ae6e0520 2024-02-02T15:41:26.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.959+0000 7f9a67e00700 1 -- 172.21.15.1:0/1976144299 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a60010710 con 0x5567ae6e0520 2024-02-02T15:41:26.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.960+0000 7f9a908a9b80 1 -- 172.21.15.1:0/1976144299 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567ae6e0520 msgr2=0x5567ae6fe9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:41:26.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.960+0000 7f9a908a9b80 1 --2- 172.21.15.1:0/1976144299 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567ae6e0520 0x5567ae6fe9e0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f9a60095330 tx=0x7f9a60099040 comp rx=0 tx=0).stop 2024-02-02T15:41:26.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.966+0000 7f9a908a9b80 1 -- 172.21.15.1:0/1976144299 shutdown_connections 2024-02-02T15:41:26.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.966+0000 7f9a908a9b80 1 --2- 172.21.15.1:0/1976144299 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5567ae6cae40 0x5567ae6d4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:26.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.966+0000 7f9a908a9b80 1 --2- 172.21.15.1:0/1976144299 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5567ae6d3010 0x5567ae6ee6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:26.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.966+0000 7f9a908a9b80 1 --2- 172.21.15.1:0/1976144299 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567ae6e0520 0x5567ae6fe9e0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:26.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.966+0000 7f9a908a9b80 1 -- 172.21.15.1:0/1976144299 >> 172.21.15.1:0/1976144299 conn(0x5567ae632db0 msgr2=0x5567ae6deb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:41:26.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.966+0000 7f9a908a9b80 1 -- 172.21.15.1:0/1976144299 shutdown_connections 2024-02-02T15:41:26.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.966+0000 7f9a908a9b80 1 -- 172.21.15.1:0/1976144299 wait complete. 2024-02-02T15:41:26.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.967+0000 7f9a908a9b80 1 -- start start 2024-02-02T15:41:26.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.968+0000 7f9a908a9b80 1 Processor -- start 2024-02-02T15:41:26.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.968+0000 7f9a908a9b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5567ae6cae40 0x5567ae6f9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:26.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.968+0000 7f9a908a9b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5567ae6d3010 0x5567ae6f9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:26.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.968+0000 7f9a908a9b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567ae6e0520 0x5567ae747760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:26.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.969+0000 7f9a908a9b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567ae747ca0 con 0x5567ae6d3010 2024-02-02T15:41:26.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.969+0000 7f9a908a9b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567ae747de0 con 0x5567ae6cae40 2024-02-02T15:41:26.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.969+0000 7f9a908a9b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567ae747f20 con 0x5567ae6e0520 2024-02-02T15:41:26.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.969+0000 7f9a908a9b80 10 client.0 _ll_register_callbacks cb 0x5567ae5c2290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:41:26.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.969+0000 7f9a76800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567ae6e0520 0x5567ae747760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:26.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.969+0000 7f9a76800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567ae6e0520 0x5567ae747760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:35350/0 (socket says 172.21.15.1:35350) 2024-02-02T15:41:26.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.969+0000 7f9a76800700 1 -- 172.21.15.1:0/2515344526 learned_addr learned my addr 172.21.15.1:0/2515344526 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:41:26.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.969+0000 7f9a75400700 1 --2- 172.21.15.1:0/2515344526 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5567ae6d3010 0x5567ae6f9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:26.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.969+0000 7f9a75e00700 1 --2- 172.21.15.1:0/2515344526 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5567ae6cae40 0x5567ae6f9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:26.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.970+0000 7f9a75400700 1 -- 172.21.15.1:0/2515344526 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567ae6e0520 msgr2=0x5567ae747760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:41:26.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.970+0000 7f9a75400700 1 --2- 172.21.15.1:0/2515344526 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567ae6e0520 0x5567ae747760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:26.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.970+0000 7f9a75400700 1 -- 172.21.15.1:0/2515344526 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5567ae6cae40 msgr2=0x5567ae6f9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:41:26.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.970+0000 7f9a75400700 1 --2- 172.21.15.1:0/2515344526 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5567ae6cae40 0x5567ae6f9500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:26.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.970+0000 7f9a75400700 1 -- 172.21.15.1:0/2515344526 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a70013040 con 0x5567ae6d3010 2024-02-02T15:41:26.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.971+0000 7f9a75400700 1 --2- 172.21.15.1:0/2515344526 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5567ae6d3010 0x5567ae6f9a40 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f9a60095330 tx=0x7f9a60010be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4777f4158a94de9f server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:41:26.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.971+0000 7f9a5fe00700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:41:26.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.971+0000 7f9a5fe00700 1 -- 172.21.15.1:0/2515344526 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a600cb4f0 con 0x5567ae6d3010 2024-02-02T15:41:26.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.971+0000 7f9a908a9b80 1 -- 172.21.15.1:0/2515344526 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9a600a0930 con 0x5567ae6d3010 2024-02-02T15:41:26.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.971+0000 7f9a5fe00700 1 -- 172.21.15.1:0/2515344526 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f9a600a0cf0 con 0x5567ae6d3010 2024-02-02T15:41:26.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.971+0000 7f9a5fe00700 1 -- 172.21.15.1:0/2515344526 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a60098d30 con 0x5567ae6d3010 2024-02-02T15:41:26.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.971+0000 7f9a908a9b80 10 client.19254 fetch_fsmap learned FSMap version 60 2024-02-02T15:41:26.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.971+0000 7f9a908a9b80 1 -- 172.21.15.1:0/2515344526 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) v3 -- 0x5567ae452b60 con 0x5567ae6d3010 2024-02-02T15:41:26.972 INFO:tasks.workunit.client.0.smithi001.stdout: 10 31784 32.03 MB/sec warmup 59 sec latency 252.805 ms 2024-02-02T15:41:26.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.972+0000 7f9a5fe00700 1 -- 172.21.15.1:0/2515344526 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9a600a0930 con 0x5567ae6d3010 2024-02-02T15:41:26.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.972+0000 7f9a5fe00700 1 -- 172.21.15.1:0/2515344526 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 60) v1 ==== 7542+0+0 (secure 0 0 0) 0x7f9a600946b0 con 0x5567ae6d3010 2024-02-02T15:41:26.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.972+0000 7f9a908a9b80 10 client.19254 fetch_fsmap finished waiting for FSMap version 60 2024-02-02T15:41:26.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.972+0000 7f9a908a9b80 10 client.19254 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:41:26.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.972+0000 7f9a908a9b80 20 client.19254 populate_metadata read hostname 'smithi001' 2024-02-02T15:41:26.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.972+0000 7f9a908a9b80 1 --2- 172.21.15.1:0/2515344526 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5567ae7491c0 0x5567ae74b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:26.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.972+0000 7f9a908a9b80 4 client.19254 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2024-02-02T15:41:26.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.972+0000 7f9a908a9b80 1 -- 172.21.15.1:0/2515344526 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5567ae5e3f50 con 0x5567ae7491c0 2024-02-02T15:41:26.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.973+0000 7f9a75e00700 1 --2- 172.21.15.1:0/2515344526 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5567ae7491c0 0x5567ae74b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:26.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.973+0000 7f9a75e00700 1 --2- 172.21.15.1:0/2515344526 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5567ae7491c0 0x5567ae74b5e0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d18407231ff5339 server_cookie=f30414f4d5eb393b in_seq=0 out_seq=0 2024-02-02T15:41:26.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.974+0000 7f9a5fe00700 10 client.19254 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:41:26.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.974+0000 7f9a5fe00700 1 -- 172.21.15.1:0/2515344526 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5567ae5e3f50 con 0x5567ae7491c0 2024-02-02T15:41:26.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.974+0000 7f9a5fe00700 10 client.19254 handle_command_reply: tid=0 2024-02-02T15:41:26.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.974+0000 7f9a908a9b80 1 client.19254 shutdown 2024-02-02T15:41:26.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.978+0000 7f9a908a9b80 1 -- 172.21.15.1:0/2515344526 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5567ae6d3010 msgr2=0x5567ae6f9a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:41:26.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.978+0000 7f9a908a9b80 1 --2- 172.21.15.1:0/2515344526 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5567ae6d3010 0x5567ae6f9a40 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f9a60095330 tx=0x7f9a60010be0 comp rx=0 tx=0).stop 2024-02-02T15:41:26.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.979+0000 7f9a908a9b80 1 -- 172.21.15.1:0/2515344526 shutdown_connections 2024-02-02T15:41:26.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.979+0000 7f9a908a9b80 1 --2- 172.21.15.1:0/2515344526 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5567ae7491c0 0x5567ae74b5e0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:26.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.979+0000 7f9a908a9b80 1 --2- 172.21.15.1:0/2515344526 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5567ae6e0520 0x5567ae747760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:26.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.979+0000 7f9a908a9b80 1 --2- 172.21.15.1:0/2515344526 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5567ae6cae40 0x5567ae6f9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:26.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.979+0000 7f9a908a9b80 1 --2- 172.21.15.1:0/2515344526 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5567ae6d3010 0x5567ae6f9a40 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:26.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.979+0000 7f9a908a9b80 1 -- 172.21.15.1:0/2515344526 >> 172.21.15.1:0/2515344526 conn(0x5567ae632db0 msgr2=0x5567ae6df390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:41:26.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.979+0000 7f9a5fe00700 0 client.19254 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:41:26.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.979+0000 7f9a908a9b80 1 -- 172.21.15.1:0/2515344526 shutdown_connections 2024-02-02T15:41:26.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.979+0000 7f9a908a9b80 1 -- 172.21.15.1:0/2515344526 wait complete. 2024-02-02T15:41:26.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.979+0000 7f9a908a9b80 20 client.19254 trim_cache size 0 max 16384 2024-02-02T15:41:26.980 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:41:26.980 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2024-02-02T15:41:26.980 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2024-02-02T15:41:26.980 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:41:26.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.980+0000 7f9a7ce00700 1 -- 172.21.15.1:0/959908627 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f9a580a0860 msgr2=0x7f9a580a2d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:41:26.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.980+0000 7f9a7ce00700 1 --2- 172.21.15.1:0/959908627 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f9a580a0860 0x7f9a580a2d20 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9a6800b2e0 tx=0x7f9a68000bc0 comp rx=0 tx=0).stop 2024-02-02T15:41:26.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.980+0000 7f9a7ce00700 1 -- 172.21.15.1:0/959908627 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a78106120 msgr2=0x7f9a78197490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:41:26.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.980+0000 7f9a7ce00700 1 --2- 172.21.15.1:0/959908627 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a78106120 0x7f9a78197490 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f9a6000d980 tx=0x7f9a6000dd40 comp rx=0 tx=0).stop 2024-02-02T15:41:26.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.980+0000 7f9a7ce00700 1 -- 172.21.15.1:0/959908627 shutdown_connections 2024-02-02T15:41:26.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.980+0000 7f9a7ce00700 1 --2- 172.21.15.1:0/959908627 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f9a580a0860 0x7f9a580a2d20 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:26.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.980+0000 7f9a7ce00700 1 --2- 172.21.15.1:0/959908627 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9a7819b8f0 0x7f9a7819bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:26.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.980+0000 7f9a7ce00700 1 --2- 172.21.15.1:0/959908627 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9a78101000 0x7f9a78196f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:26.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.980+0000 7f9a7ce00700 1 --2- 172.21.15.1:0/959908627 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9a78106120 0x7f9a78197490 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:26.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.980+0000 7f9a7ce00700 1 -- 172.21.15.1:0/959908627 >> 172.21.15.1:0/959908627 conn(0x7f9a780fc840 msgr2=0x7f9a78104a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:41:26.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.982+0000 7f9a7ce00700 1 -- 172.21.15.1:0/959908627 shutdown_connections 2024-02-02T15:41:26.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:26.982+0000 7f9a7ce00700 1 -- 172.21.15.1:0/959908627 wait complete. 2024-02-02T15:41:27.009 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-02T15:41:27.010 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T15:41:27.306 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:27.306+0000 7f076e200700 1 -- 172.21.15.1:0/3455067676 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0768106e20 msgr2=0x7f0768107200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:41:27.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:27.306+0000 7f076e200700 1 --2- 172.21.15.1:0/3455067676 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0768106e20 0x7f0768107200 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f0758009990 tx=0x7f0758009ca0 comp rx=0 tx=0).stop 2024-02-02T15:41:27.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:27.306+0000 7f076e200700 1 -- 172.21.15.1:0/3455067676 shutdown_connections 2024-02-02T15:41:27.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:27.306+0000 7f076e200700 1 --2- 172.21.15.1:0/3455067676 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0768068a30 0x7f076810dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:27.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:27.306+0000 7f076e200700 1 --2- 172.21.15.1:0/3455067676 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0768106e20 0x7f0768107200 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:27.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:27.306+0000 7f076e200700 1 --2- 172.21.15.1:0/3455067676 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0768068070 0x7f07680684f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:27.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:27.306+0000 7f076e200700 1 -- 172.21.15.1:0/3455067676 >> 172.21.15.1:0/3455067676 conn(0x7f07680769c0 msgr2=0x7f0768076dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:41:27.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:27.306+0000 7f076e200700 1 -- 172.21.15.1:0/3455067676 shutdown_connections 2024-02-02T15:41:27.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:27.306+0000 7f076e200700 1 -- 172.21.15.1:0/3455067676 wait complete. 2024-02-02T15:41:27.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:27.308+0000 7f076e200700 1 Processor -- start 2024-02-02T15:41:27.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:27.308+0000 7f076e200700 1 -- start start 2024-02-02T15:41:27.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:27.309+0000 7f076e200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0768068070 0x7f076819a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:27.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:27.309+0000 7f076e200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0768068a30 0x7f076819ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:27.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:27.309+0000 7f076e200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0768106e20 0x7f07681949a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:27.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:27.309+0000 7f076e200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0768195010 con 0x7f0768068070 2024-02-02T15:41:27.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:27.309+0000 7f076e200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0768195180 con 0x7f0768106e20 2024-02-02T15:41:27.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:27.309+0000 7f076e200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0768195340 con 0x7f0768068a30 2024-02-02T15:41:27.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:27.309+0000 7f0766a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0768068a30 0x7f076819ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:27.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:27.309+0000 7f0766a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0768068a30 0x7f076819ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:35366/0 (socket says 172.21.15.1:35366) 2024-02-02T15:41:27.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:27.309+0000 7f0766a00700 1 -- 172.21.15.1:0/1767583488 learned_addr learned my addr 172.21.15.1:0/1767583488 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:41:27.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:27.309+0000 7f0767400700 1 --2- 172.21.15.1:0/1767583488 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0768068070 0x7f076819a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:27.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:27.310+0000 7f0766a00700 1 -- 172.21.15.1:0/1767583488 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0768106e20 msgr2=0x7f07681949a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:41:27.310 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:27.310+0000 7f0767e00700 1 --2- 172.21.15.1:0/1767583488 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0768106e20 0x7f07681949a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:27.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:27.310+0000 7f0766a00700 1 --2- 172.21.15.1:0/1767583488 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0768106e20 0x7f07681949a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:27.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:27.310+0000 7f0766a00700 1 -- 172.21.15.1:0/1767583488 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0768068070 msgr2=0x7f076819a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:41:27.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:27.310+0000 7f0766a00700 1 --2- 172.21.15.1:0/1767583488 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0768068070 0x7f076819a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:27.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:27.310+0000 7f0766a00700 1 -- 172.21.15.1:0/1767583488 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07580095f0 con 0x7f0768068a30 2024-02-02T15:41:27.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:27.310+0000 7f0767e00700 1 --2- 172.21.15.1:0/1767583488 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0768106e20 0x7f07681949a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:41:27.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:27.310+0000 7f0766a00700 1 --2- 172.21.15.1:0/1767583488 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0768068a30 0x7f076819ae20 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f075000e9a0 tx=0x7f075000ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:41:27.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:27.311+0000 7f075fe00700 1 -- 172.21.15.1:0/1767583488 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0750009790 con 0x7f0768068a30 2024-02-02T15:41:27.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:27.311+0000 7f075fe00700 1 -- 172.21.15.1:0/1767583488 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f075000cbd0 con 0x7f0768068a30 2024-02-02T15:41:27.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:27.311+0000 7f075fe00700 1 -- 172.21.15.1:0/1767583488 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0750018680 con 0x7f0768068a30 2024-02-02T15:41:27.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:27.311+0000 7f076e200700 1 -- 172.21.15.1:0/1767583488 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0768195620 con 0x7f0768068a30 2024-02-02T15:41:27.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:27.311+0000 7f076e200700 1 -- 172.21.15.1:0/1767583488 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07681a18d0 con 0x7f0768068a30 2024-02-02T15:41:27.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:27.314+0000 7f076e200700 1 -- 172.21.15.1:0/1767583488 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0768067bc0 con 0x7f0768068a30 2024-02-02T15:41:27.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:27.317+0000 7f075fe00700 1 -- 172.21.15.1:0/1767583488 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f075000cd40 con 0x7f0768068a30 2024-02-02T15:41:27.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:27.319+0000 7f075fe00700 1 --2- 172.21.15.1:0/1767583488 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f07480a4bf0 0x7f07480a70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:27.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:27.319+0000 7f075fe00700 1 -- 172.21.15.1:0/1767583488 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f0750014070 con 0x7f0768068a30 2024-02-02T15:41:27.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:27.319+0000 7f0767400700 1 --2- 172.21.15.1:0/1767583488 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f07480a4bf0 0x7f07480a70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:27.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:27.320+0000 7f0767400700 1 --2- 172.21.15.1:0/1767583488 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f07480a4bf0 0x7f07480a70b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f075800b2e0 tx=0x7f0758005fb0 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:41:27.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:27.328+0000 7f075fe00700 1 -- 172.21.15.1:0/1767583488 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f0750095920 con 0x7f0768068a30 2024-02-02T15:41:27.781 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:27.781+0000 7f076e200700 1 -- 172.21.15.1:0/1767583488 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0768195e60 con 0x7f0768068a30 2024-02-02T15:41:27.782 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:27.782+0000 7f075fe00700 1 -- 172.21.15.1:0/1767583488 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+10897 (secure 0 0 0) 0x7f0750098f40 con 0x7f0768068a30 2024-02-02T15:41:27.782 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:41:27.783 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":60,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2922240448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2922240448},{"type":"v1","addr":"172.21.15.1:6835","nonce":2922240448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2729497431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2729497431},{"type":"v1","addr":"172.21.15.1:6837","nonce":2729497431}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":60,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-02T15:24:47.761435+0000","modified":"2024-02-02T15:41:13.575501+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24529,"mds_2":24518,"mds_3":24514,"mds_4":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14517":{"gid":14517,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/1218303683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":1218303683},{"type":"v1","addr":"172.21.15.1:6839","nonce":1218303683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14529":{"gid":14529,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/2059197496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2059197496},{"type":"v1","addr":"172.21.15.1:6841","nonce":2059197496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/7648986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":7648986},{"type":"v1","addr":"172.21.15.142:6833","nonce":7648986}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/3482746094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3482746094},{"type":"v1","addr":"172.21.15.71:6833","nonce":3482746094}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/388207814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":388207814},{"type":"v1","addr":"172.21.15.71:6835","nonce":388207814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6835/1052606547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1052606547},{"type":"v1","addr":"172.21.15.142:6835","nonce":1052606547}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.142:6837/2870289019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2870289019},{"type":"v1","addr":"172.21.15.142:6837","nonce":2870289019}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.71:6837/628016199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":628016199},{"type":"v1","addr":"172.21.15.71:6837","nonce":628016199}]},"join_fscid":-1,"export_targets":[0,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.142:6839/321675421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":321675421},{"type":"v1","addr":"172.21.15.142:6839","nonce":321675421}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/1268086493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1268086493},{"type":"v1","addr":"172.21.15.71:6839","nonce":1268086493}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-02T15:41:27.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:27.785+0000 7f076e200700 1 -- 172.21.15.1:0/1767583488 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f07480a4bf0 msgr2=0x7f07480a70b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:41:27.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:27.785+0000 7f076e200700 1 --2- 172.21.15.1:0/1767583488 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f07480a4bf0 0x7f07480a70b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f075800b2e0 tx=0x7f0758005fb0 comp rx=0 tx=0).stop 2024-02-02T15:41:27.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:27.785+0000 7f076e200700 1 -- 172.21.15.1:0/1767583488 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0768068a30 msgr2=0x7f076819ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:41:27.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:27.785+0000 7f076e200700 1 --2- 172.21.15.1:0/1767583488 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0768068a30 0x7f076819ae20 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f075000e9a0 tx=0x7f075000ed60 comp rx=0 tx=0).stop 2024-02-02T15:41:27.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:27.785+0000 7f076e200700 1 -- 172.21.15.1:0/1767583488 shutdown_connections 2024-02-02T15:41:27.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:27.785+0000 7f076e200700 1 --2- 172.21.15.1:0/1767583488 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f07480a4bf0 0x7f07480a70b0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:27.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:27.785+0000 7f076e200700 1 --2- 172.21.15.1:0/1767583488 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0768106e20 0x7f07681949a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:27.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:27.785+0000 7f076e200700 1 --2- 172.21.15.1:0/1767583488 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0768068070 0x7f076819a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:27.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:27.785+0000 7f076e200700 1 --2- 172.21.15.1:0/1767583488 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0768068a30 0x7f076819ae20 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:27.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:27.785+0000 7f076e200700 1 -- 172.21.15.1:0/1767583488 >> 172.21.15.1:0/1767583488 conn(0x7f07680769c0 msgr2=0x7f07680fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:41:27.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:27.786+0000 7f076e200700 1 -- 172.21.15.1:0/1767583488 shutdown_connections 2024-02-02T15:41:27.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:27.786+0000 7f076e200700 1 -- 172.21.15.1:0/1767583488 wait complete. 2024-02-02T15:41:27.787 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 60 2024-02-02T15:41:27.806 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:41:27 smithi001 ceph-mon[35470]: pgmap v452: 129 pgs: 129 active+clean; 1.0 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 13 MiB/s wr, 809 op/s 2024-02-02T15:41:27.809 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-02-02T15:41:27.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:41:27 smithi071 ceph-mon[44707]: pgmap v452: 129 pgs: 129 active+clean; 1.0 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 13 MiB/s wr, 809 op/s 2024-02-02T15:41:27.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:41:27 smithi142 ceph-mon[45728]: pgmap v452: 129 pgs: 129 active+clean; 1.0 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 13 MiB/s wr, 809 op/s 2024-02-02T15:41:27.974 INFO:tasks.workunit.client.0.smithi001.stdout: 10 32545 32.11 MB/sec warmup 60 sec latency 43.796 ms 2024-02-02T15:41:28.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.162+0000 7fba0d600700 1 -- 172.21.15.1:0/2131162193 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fba08107fb0 msgr2=0x7fba0810bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:41:28.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.162+0000 7fba0d600700 1 --2- 172.21.15.1:0/2131162193 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fba08107fb0 0x7fba0810bff0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fb9f00098f0 tx=0x7fb9f0009c00 comp rx=0 tx=0).stop 2024-02-02T15:41:28.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.162+0000 7fba0d600700 1 -- 172.21.15.1:0/2131162193 shutdown_connections 2024-02-02T15:41:28.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.162+0000 7fba0d600700 1 --2- 172.21.15.1:0/2131162193 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba0810c720 0x7fba081142a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:28.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.162+0000 7fba0d600700 1 --2- 172.21.15.1:0/2131162193 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fba08107600 0x7fba081079e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:28.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.162+0000 7fba0d600700 1 --2- 172.21.15.1:0/2131162193 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fba08107fb0 0x7fba0810bff0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:28.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.162+0000 7fba0d600700 1 -- 172.21.15.1:0/2131162193 >> 172.21.15.1:0/2131162193 conn(0x7fba08076b80 msgr2=0x7fba08105b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:41:28.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.163+0000 7fba0d600700 1 -- 172.21.15.1:0/2131162193 shutdown_connections 2024-02-02T15:41:28.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.163+0000 7fba0d600700 1 -- 172.21.15.1:0/2131162193 wait complete. 2024-02-02T15:41:28.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.164+0000 7fba0d600700 1 Processor -- start 2024-02-02T15:41:28.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.164+0000 7fba0d600700 1 -- start start 2024-02-02T15:41:28.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.165+0000 7fba0d600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fba08107600 0x7fba0819d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:28.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.165+0000 7fba0d600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba0810c720 0x7fba0819da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:28.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.165+0000 7fba0d600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fba081a1eb0 0x7fba081a2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:28.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.165+0000 7fba0d600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba081a2930 con 0x7fba08107600 2024-02-02T15:41:28.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.165+0000 7fba0d600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba081a2aa0 con 0x7fba081a1eb0 2024-02-02T15:41:28.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.165+0000 7fba0d600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba081a2c60 con 0x7fba0810c720 2024-02-02T15:41:28.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.166+0000 7fba07400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fba081a1eb0 0x7fba081a2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:28.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.166+0000 7fba07400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fba081a1eb0 0x7fba081a2330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59372/0 (socket says 172.21.15.1:59372) 2024-02-02T15:41:28.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.166+0000 7fba07400700 1 -- 172.21.15.1:0/4108488577 learned_addr learned my addr 172.21.15.1:0/4108488577 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:41:28.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.166+0000 7fba06000700 1 --2- 172.21.15.1:0/4108488577 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba0810c720 0x7fba0819da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:28.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.167+0000 7fba07400700 1 -- 172.21.15.1:0/4108488577 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba0810c720 msgr2=0x7fba0819da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:41:28.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.167+0000 7fba07400700 1 --2- 172.21.15.1:0/4108488577 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba0810c720 0x7fba0819da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:28.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.167+0000 7fba07400700 1 -- 172.21.15.1:0/4108488577 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fba08107600 msgr2=0x7fba0819d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:41:28.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.167+0000 7fba07400700 1 --2- 172.21.15.1:0/4108488577 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fba08107600 0x7fba0819d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:28.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.167+0000 7fba07400700 1 -- 172.21.15.1:0/4108488577 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9f0009520 con 0x7fba081a1eb0 2024-02-02T15:41:28.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.167+0000 7fba07400700 1 --2- 172.21.15.1:0/4108488577 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fba081a1eb0 0x7fba081a2330 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fba0000b780 tx=0x7fba0000bb40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:41:28.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.167+0000 7fb9ff400700 1 -- 172.21.15.1:0/4108488577 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba0000f840 con 0x7fba081a1eb0 2024-02-02T15:41:28.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.168+0000 7fba0d600700 1 -- 172.21.15.1:0/4108488577 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba081aa1c0 con 0x7fba081a1eb0 2024-02-02T15:41:28.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.168+0000 7fb9ff400700 1 -- 172.21.15.1:0/4108488577 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba0000fe60 con 0x7fba081a1eb0 2024-02-02T15:41:28.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.168+0000 7fb9ff400700 1 -- 172.21.15.1:0/4108488577 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba0000d3d0 con 0x7fba081a1eb0 2024-02-02T15:41:28.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.168+0000 7fba0d600700 1 -- 172.21.15.1:0/4108488577 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba081aa6f0 con 0x7fba081a1eb0 2024-02-02T15:41:28.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.173+0000 7fb9ff400700 1 -- 172.21.15.1:0/4108488577 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7fba0000f9a0 con 0x7fba081a1eb0 2024-02-02T15:41:28.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.175+0000 7fb9ff400700 1 --2- 172.21.15.1:0/4108488577 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fb9e80a0790 0x7fb9e80a2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:28.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.176+0000 7fb9ff400700 1 -- 172.21.15.1:0/4108488577 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fba000cbfb0 con 0x7fba081a1eb0 2024-02-02T15:41:28.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.177+0000 7fba06a00700 1 --2- 172.21.15.1:0/4108488577 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fb9e80a0790 0x7fb9e80a2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:28.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.179+0000 7fba21107b80 1 Processor -- start 2024-02-02T15:41:28.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.179+0000 7fba21107b80 1 -- start start 2024-02-02T15:41:28.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.180+0000 7fba21107b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5646fd7b5010 0x5646fd7b53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:28.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.180+0000 7fba21107b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5646fd7b5930 0x5646fd7c02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:28.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.180+0000 7fba21107b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646fd7c07e0 0x5646fd7c2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:28.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.180+0000 7fba21107b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646fd7294c0 con 0x5646fd7b5010 2024-02-02T15:41:28.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.180+0000 7fba21107b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646fd70daa0 con 0x5646fd7b5930 2024-02-02T15:41:28.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.180+0000 7fba21107b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646fd70b460 con 0x5646fd7c07e0 2024-02-02T15:41:28.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.180+0000 7fba06a00700 1 --2- 172.21.15.1:0/4108488577 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fb9e80a0790 0x7fb9e80a2c50 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb9f8005950 tx=0x7fb9f80058e0 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:41:28.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.180+0000 7fba07400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646fd7c07e0 0x5646fd7c2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:28.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.180+0000 7fba07400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646fd7c07e0 0x5646fd7c2bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:35392/0 (socket says 172.21.15.1:35392) 2024-02-02T15:41:28.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.180+0000 7fba07400700 1 -- 172.21.15.1:0/1579132475 learned_addr learned my addr 172.21.15.1:0/1579132475 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:41:28.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.181+0000 7fba06000700 1 --2- 172.21.15.1:0/1579132475 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5646fd7b5930 0x5646fd7c02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:28.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.181+0000 7fba07400700 1 -- 172.21.15.1:0/1579132475 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5646fd7b5930 msgr2=0x5646fd7c02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:41:28.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.181+0000 7fba07400700 1 --2- 172.21.15.1:0/1579132475 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5646fd7b5930 0x5646fd7c02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:28.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.181+0000 7fba07400700 1 -- 172.21.15.1:0/1579132475 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5646fd7b5010 msgr2=0x5646fd7b53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:41:28.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.181+0000 7fba07400700 1 --2- 172.21.15.1:0/1579132475 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5646fd7b5010 0x5646fd7b53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:28.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.181+0000 7fba07400700 1 -- 172.21.15.1:0/1579132475 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5646fd5883b0 con 0x5646fd7c07e0 2024-02-02T15:41:28.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.181+0000 7fba07400700 1 --2- 172.21.15.1:0/1579132475 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646fd7c07e0 0x5646fd7c2bd0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fba000955d0 tx=0x7fba00098ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3254a70fde2b053 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:41:28.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.182+0000 7fb9f7400700 1 -- 172.21.15.1:0/1579132475 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba000990d0 con 0x5646fd7c07e0 2024-02-02T15:41:28.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.182+0000 7fb9f7400700 1 -- 172.21.15.1:0/1579132475 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba0009c070 con 0x5646fd7c07e0 2024-02-02T15:41:28.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.182+0000 7fb9f7400700 1 -- 172.21.15.1:0/1579132475 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba000c9580 con 0x5646fd7c07e0 2024-02-02T15:41:28.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.183+0000 7fba21107b80 1 -- 172.21.15.1:0/1579132475 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646fd7c07e0 msgr2=0x5646fd7c2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:41:28.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.183+0000 7fba21107b80 1 --2- 172.21.15.1:0/1579132475 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646fd7c07e0 0x5646fd7c2bd0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fba000955d0 tx=0x7fba00098ab0 comp rx=0 tx=0).stop 2024-02-02T15:41:28.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.183+0000 7fba21107b80 1 -- 172.21.15.1:0/1579132475 shutdown_connections 2024-02-02T15:41:28.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.184+0000 7fba21107b80 1 --2- 172.21.15.1:0/1579132475 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646fd7c07e0 0x5646fd7c2bd0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:28.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.184+0000 7fba21107b80 1 --2- 172.21.15.1:0/1579132475 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5646fd7b5010 0x5646fd7b53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:28.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.184+0000 7fba21107b80 1 --2- 172.21.15.1:0/1579132475 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5646fd7b5930 0x5646fd7c02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:28.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.184+0000 7fba21107b80 1 -- 172.21.15.1:0/1579132475 >> 172.21.15.1:0/1579132475 conn(0x5646fd714db0 msgr2=0x5646fd7b13b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:41:28.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.184+0000 7fba21107b80 1 -- 172.21.15.1:0/1579132475 shutdown_connections 2024-02-02T15:41:28.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.185+0000 7fba21107b80 1 -- 172.21.15.1:0/1579132475 wait complete. 2024-02-02T15:41:28.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.186+0000 7fba21107b80 1 -- start start 2024-02-02T15:41:28.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.186+0000 7fba21107b80 1 Processor -- start 2024-02-02T15:41:28.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.188+0000 7fba21107b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5646fd7b5010 0x5646fd7bd4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:28.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.189+0000 7fba21107b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5646fd7b5930 0x5646fd7bda00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:28.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.189+0000 7fba21107b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646fd7c07e0 0x5646fd7bdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:28.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.189+0000 7fba21107b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646fd706d50 con 0x5646fd7b5010 2024-02-02T15:41:28.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.189+0000 7fba07400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646fd7c07e0 0x5646fd7bdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:28.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.189+0000 7fba07400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646fd7c07e0 0x5646fd7bdf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:35396/0 (socket says 172.21.15.1:35396) 2024-02-02T15:41:28.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.189+0000 7fba07400700 1 -- 172.21.15.1:0/3828938962 learned_addr learned my addr 172.21.15.1:0/3828938962 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:41:28.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.189+0000 7fba06000700 1 --2- 172.21.15.1:0/3828938962 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5646fd7b5930 0x5646fd7bda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:28.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.189+0000 7fba21107b80 1 -- 172.21.15.1:0/3828938962 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646fd6a2d80 con 0x5646fd7b5930 2024-02-02T15:41:28.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.189+0000 7fba21107b80 1 -- 172.21.15.1:0/3828938962 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646fd6a2a20 con 0x5646fd7c07e0 2024-02-02T15:41:28.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.190+0000 7fba07400700 1 -- 172.21.15.1:0/3828938962 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5646fd7b5930 msgr2=0x5646fd7bda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:41:28.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.190+0000 7fba07400700 1 --2- 172.21.15.1:0/3828938962 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5646fd7b5930 0x5646fd7bda00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:28.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.190+0000 7fba06a00700 1 --2- 172.21.15.1:0/3828938962 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5646fd7b5010 0x5646fd7bd4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:28.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.192+0000 7fba07400700 1 -- 172.21.15.1:0/3828938962 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5646fd7b5010 msgr2=0x5646fd7bd4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:41:28.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.192+0000 7fba07400700 1 --2- 172.21.15.1:0/3828938962 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5646fd7b5010 0x5646fd7bd4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:28.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.192+0000 7fba07400700 1 -- 172.21.15.1:0/3828938962 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba081aa6f0 con 0x5646fd7c07e0 2024-02-02T15:41:28.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.192+0000 7fba21107b80 10 client.0 _ll_register_callbacks cb 0x5646fd6a4290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:41:28.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.192+0000 7fba07400700 1 --2- 172.21.15.1:0/3828938962 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646fd7c07e0 0x5646fd7bdf40 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fba00098dd0 tx=0x7fba000a11b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80943c5905bf17c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:41:28.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.192+0000 7fb9efe00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:41:28.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.193+0000 7fb9efe00700 1 -- 172.21.15.1:0/3828938962 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba000c9580 con 0x5646fd7c07e0 2024-02-02T15:41:28.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.193+0000 7fb9efe00700 1 -- 172.21.15.1:0/3828938962 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fba000c96e0 con 0x5646fd7c07e0 2024-02-02T15:41:28.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.193+0000 7fba21107b80 1 -- 172.21.15.1:0/3828938962 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5646fd7a90f0 con 0x5646fd7c07e0 2024-02-02T15:41:28.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.193+0000 7fb9efe00700 1 -- 172.21.15.1:0/3828938962 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7fba00099330 con 0x5646fd7c07e0 2024-02-02T15:41:28.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.193+0000 7fba21107b80 10 client.29392 fetch_fsmap learned FSMap version 60 2024-02-02T15:41:28.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.193+0000 7fba21107b80 1 -- 172.21.15.1:0/3828938962 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({fsmap=60}) v3 -- 0x5646fd534b60 con 0x5646fd7c07e0 2024-02-02T15:41:28.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.195+0000 7fb9efe00700 1 -- 172.21.15.1:0/3828938962 <== mon.1 v2:172.21.15.142:3300/0 4 ==== fsmap(e 60) v1 ==== 7542+0+0 (secure 0 0 0) 0x7fba000985c0 con 0x5646fd7c07e0 2024-02-02T15:41:28.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.195+0000 7fba21107b80 10 client.29392 fetch_fsmap finished waiting for FSMap version 60 2024-02-02T15:41:28.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.195+0000 7fba21107b80 10 client.29392 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-02-02T15:41:28.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.195+0000 7fba21107b80 20 client.29392 populate_metadata read hostname 'smithi001' 2024-02-02T15:41:28.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.195+0000 7fba21107b80 1 --2- 172.21.15.1:0/3828938962 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x5646fd7cfc20 0x5646fd7d2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:28.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.195+0000 7fba21107b80 4 client.29392 mds_command: new command op to 24502 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:41:28.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.195+0000 7fba21107b80 1 -- 172.21.15.1:0/3828938962 --> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5646fd7a90f0 con 0x5646fd7cfc20 2024-02-02T15:41:28.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.195+0000 7fba06a00700 1 --2- 172.21.15.1:0/3828938962 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x5646fd7cfc20 0x5646fd7d2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:28.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.197+0000 7fb9efe00700 10 client.29392 ms_handle_connect on v2:172.21.15.142:6834/1052606547 2024-02-02T15:41:28.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.197+0000 7fba06a00700 1 --2- 172.21.15.1:0/3828938962 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x5646fd7cfc20 0x5646fd7d2040 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=74ff92be19f9652d server_cookie=bf4d7552ef703469 in_seq=0 out_seq=0 2024-02-02T15:41:28.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.198+0000 7fb9efe00700 1 -- 172.21.15.1:0/3828938962 <== mds.4 v2:172.21.15.142:6834/1052606547 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5646fd7a90f0 con 0x5646fd7cfc20 2024-02-02T15:41:28.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.198+0000 7fb9efe00700 10 client.29392 handle_command_reply: tid=0 2024-02-02T15:41:28.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.199+0000 7fba21107b80 1 client.29392 shutdown 2024-02-02T15:41:28.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.199+0000 7fba21107b80 1 -- 172.21.15.1:0/3828938962 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646fd7c07e0 msgr2=0x5646fd7bdf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:41:28.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.199+0000 7fba21107b80 1 --2- 172.21.15.1:0/3828938962 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646fd7c07e0 0x5646fd7bdf40 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fba00098dd0 tx=0x7fba000a11b0 comp rx=0 tx=0).stop 2024-02-02T15:41:28.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.199+0000 7fba21107b80 1 -- 172.21.15.1:0/3828938962 shutdown_connections 2024-02-02T15:41:28.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.199+0000 7fba21107b80 1 --2- 172.21.15.1:0/3828938962 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x5646fd7cfc20 0x5646fd7d2040 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:28.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.199+0000 7fba21107b80 1 --2- 172.21.15.1:0/3828938962 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646fd7c07e0 0x5646fd7bdf40 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:28.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.199+0000 7fba21107b80 1 --2- 172.21.15.1:0/3828938962 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5646fd7b5010 0x5646fd7bd4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:28.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.199+0000 7fba21107b80 1 --2- 172.21.15.1:0/3828938962 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5646fd7b5930 0x5646fd7bda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:28.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.199+0000 7fba21107b80 1 -- 172.21.15.1:0/3828938962 >> 172.21.15.1:0/3828938962 conn(0x5646fd714db0 msgr2=0x5646fd7b0fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:41:28.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.199+0000 7fb9efe00700 0 client.29392 ms_handle_reset on v2:172.21.15.142:6834/1052606547 2024-02-02T15:41:28.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.200+0000 7fba21107b80 1 -- 172.21.15.1:0/3828938962 shutdown_connections 2024-02-02T15:41:28.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.200+0000 7fba21107b80 1 -- 172.21.15.1:0/3828938962 wait complete. 2024-02-02T15:41:28.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.200+0000 7fba21107b80 20 client.29392 trim_cache size 0 max 16384 2024-02-02T15:41:28.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.226+0000 7fba21107b80 1 Processor -- start 2024-02-02T15:41:28.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.226+0000 7fba21107b80 1 -- start start 2024-02-02T15:41:28.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.226+0000 7fba21107b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646fd7b5010 0x5646fd7b7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:28.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.226+0000 7fba21107b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5646fd7b5930 0x5646fd7bddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:28.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.226+0000 7fba21107b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5646fd7c07e0 0x5646fd7b8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:28.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.228+0000 7fba07400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5646fd7c07e0 0x5646fd7b8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:28.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.228+0000 7fba21107b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646fd58a6a0 con 0x5646fd7b5930 2024-02-02T15:41:28.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.228+0000 7fba21107b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646fd7b6260 con 0x5646fd7c07e0 2024-02-02T15:41:28.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.228+0000 7fba21107b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646fd7bcc00 con 0x5646fd7b5010 2024-02-02T15:41:28.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.228+0000 7fba06000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5646fd7b5930 0x5646fd7bddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:28.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.228+0000 7fba06000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5646fd7b5930 0x5646fd7bddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33174/0 (socket says 172.21.15.1:33174) 2024-02-02T15:41:28.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.228+0000 7fba06000700 1 -- 172.21.15.1:0/2456977215 learned_addr learned my addr 172.21.15.1:0/2456977215 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:41:28.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.228+0000 7fba06a00700 1 --2- 172.21.15.1:0/2456977215 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646fd7b5010 0x5646fd7b7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:28.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.229+0000 7fba06a00700 1 -- 172.21.15.1:0/2456977215 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5646fd7c07e0 msgr2=0x5646fd7b8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:41:28.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.229+0000 7fba06a00700 1 --2- 172.21.15.1:0/2456977215 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5646fd7c07e0 0x5646fd7b8840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:28.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.229+0000 7fba06a00700 1 -- 172.21.15.1:0/2456977215 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5646fd7b5930 msgr2=0x5646fd7bddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:41:28.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.229+0000 7fba06a00700 1 --2- 172.21.15.1:0/2456977215 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5646fd7b5930 0x5646fd7bddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:28.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.229+0000 7fba06a00700 1 -- 172.21.15.1:0/2456977215 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5646fd6ccea0 con 0x5646fd7b5010 2024-02-02T15:41:28.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.230+0000 7fba06a00700 1 --2- 172.21.15.1:0/2456977215 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646fd7b5010 0x5646fd7b7fe0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x5646fd7d0000 tx=0x7fb9f8010000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62c3bf70378d53ed server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:41:28.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.231+0000 7fb9f7e00700 1 -- 172.21.15.1:0/2456977215 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f8023070 con 0x5646fd7b5010 2024-02-02T15:41:28.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.232+0000 7fb9f7e00700 1 -- 172.21.15.1:0/2456977215 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5646fd7a90f0 con 0x5646fd7b5010 2024-02-02T15:41:28.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.233+0000 7fba21107b80 1 -- 172.21.15.1:0/2456977215 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646fd7b5010 msgr2=0x5646fd7b7fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:41:28.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.233+0000 7fb9f7e00700 1 -- 172.21.15.1:0/2456977215 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f8004ea0 con 0x5646fd7b5010 2024-02-02T15:41:28.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.233+0000 7fba21107b80 1 --2- 172.21.15.1:0/2456977215 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646fd7b5010 0x5646fd7b7fe0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x5646fd7d0000 tx=0x7fb9f8010000 comp rx=0 tx=0).stop 2024-02-02T15:41:28.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.233+0000 7fba21107b80 1 -- 172.21.15.1:0/2456977215 shutdown_connections 2024-02-02T15:41:28.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.233+0000 7fba21107b80 1 --2- 172.21.15.1:0/2456977215 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5646fd7c07e0 0x5646fd7b8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:28.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.233+0000 7fba21107b80 1 --2- 172.21.15.1:0/2456977215 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646fd7b5010 0x5646fd7b7fe0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:28.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.233+0000 7fba21107b80 1 --2- 172.21.15.1:0/2456977215 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5646fd7b5930 0x5646fd7bddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:28.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.233+0000 7fba21107b80 1 -- 172.21.15.1:0/2456977215 >> 172.21.15.1:0/2456977215 conn(0x5646fd714db0 msgr2=0x5646fd7bba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:41:28.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.233+0000 7fba21107b80 1 -- 172.21.15.1:0/2456977215 shutdown_connections 2024-02-02T15:41:28.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.233+0000 7fba21107b80 1 -- 172.21.15.1:0/2456977215 wait complete. 2024-02-02T15:41:28.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.234+0000 7fba21107b80 1 -- start start 2024-02-02T15:41:28.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.234+0000 7fba21107b80 1 Processor -- start 2024-02-02T15:41:28.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.236+0000 7fba21107b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5646fd7b5010 0x5646fd7c6e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:28.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.236+0000 7fba21107b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5646fd7b5930 0x5646fd7c5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:28.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.236+0000 7fba21107b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646fd7c07e0 0x5646fd7c5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:28.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.237+0000 7fba21107b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646fd7c5cd0 con 0x5646fd7b5010 2024-02-02T15:41:28.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.237+0000 7fba21107b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646fd7c5e40 con 0x5646fd7b5930 2024-02-02T15:41:28.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.237+0000 7fba21107b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646fd7c6000 con 0x5646fd7c07e0 2024-02-02T15:41:28.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.237+0000 7fba21107b80 10 client.0 _ll_register_callbacks cb 0x5646fd6a4290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:41:28.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.237+0000 7fba06000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5646fd7b5930 0x5646fd7c5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:28.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.237+0000 7fba06a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5646fd7b5010 0x5646fd7c6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:28.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.238+0000 7fba06a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5646fd7b5010 0x5646fd7c6e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33182/0 (socket says 172.21.15.1:33182) 2024-02-02T15:41:28.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.238+0000 7fba06a00700 1 -- 172.21.15.1:0/3201521618 learned_addr learned my addr 172.21.15.1:0/3201521618 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:41:28.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.239+0000 7fba07400700 1 --2- 172.21.15.1:0/3201521618 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646fd7c07e0 0x5646fd7c5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:28.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.239+0000 7fba06000700 1 -- 172.21.15.1:0/3201521618 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646fd7c07e0 msgr2=0x5646fd7c5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:41:28.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.239+0000 7fba06000700 1 --2- 172.21.15.1:0/3201521618 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646fd7c07e0 0x5646fd7c5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:28.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.239+0000 7fba06000700 1 -- 172.21.15.1:0/3201521618 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5646fd7b5010 msgr2=0x5646fd7c6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:41:28.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.239+0000 7fba06000700 1 --2- 172.21.15.1:0/3201521618 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5646fd7b5010 0x5646fd7c6e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:28.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.239+0000 7fba06000700 1 -- 172.21.15.1:0/3201521618 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9f800daa0 con 0x5646fd7b5930 2024-02-02T15:41:28.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.239+0000 7fba07400700 1 --2- 172.21.15.1:0/3201521618 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646fd7c07e0 0x5646fd7c5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:41:28.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.239+0000 7fba06a00700 1 --2- 172.21.15.1:0/3201521618 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5646fd7b5010 0x5646fd7c6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:41:28.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.240+0000 7fb9efe00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:41:28.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.240+0000 7fba06000700 1 --2- 172.21.15.1:0/3201521618 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5646fd7b5930 0x5646fd7c5250 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fb9f0009230 tx=0x7fb9f000b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2332b6e0203e2427 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:41:28.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.241+0000 7fb9efe00700 1 -- 172.21.15.1:0/3201521618 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f0005220 con 0x5646fd7b5930 2024-02-02T15:41:28.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.241+0000 7fb9efe00700 1 -- 172.21.15.1:0/3201521618 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fb9f000bc70 con 0x5646fd7b5930 2024-02-02T15:41:28.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.241+0000 7fba21107b80 1 -- 172.21.15.1:0/3201521618 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fba0009c070 con 0x5646fd7b5930 2024-02-02T15:41:28.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.242+0000 7fb9efe00700 1 -- 172.21.15.1:0/3201521618 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb9f0005380 con 0x5646fd7b5930 2024-02-02T15:41:28.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.242+0000 7fba21107b80 10 client.29414 fetch_fsmap learned FSMap version 60 2024-02-02T15:41:28.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.242+0000 7fba21107b80 1 -- 172.21.15.1:0/3201521618 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) v3 -- 0x5646fd58ccd0 con 0x5646fd7b5930 2024-02-02T15:41:28.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.242+0000 7fb9efe00700 1 -- 172.21.15.1:0/3201521618 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba0009c070 con 0x5646fd7b5930 2024-02-02T15:41:28.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.243+0000 7fb9efe00700 1 -- 172.21.15.1:0/3201521618 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 60) v1 ==== 7542+0+0 (secure 0 0 0) 0x7fb9f0015020 con 0x5646fd7b5930 2024-02-02T15:41:28.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.243+0000 7fba21107b80 10 client.29414 fetch_fsmap finished waiting for FSMap version 60 2024-02-02T15:41:28.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.243+0000 7fba21107b80 10 client.29414 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-02-02T15:41:28.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.243+0000 7fba21107b80 20 client.29414 populate_metadata read hostname 'smithi001' 2024-02-02T15:41:28.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.243+0000 7fba21107b80 1 --2- 172.21.15.1:0/3201521618 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x5646fd822b80 0x5646fd824fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:28.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.243+0000 7fba21107b80 4 client.29414 mds_command: new command op to 24502 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:41:28.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.243+0000 7fba21107b80 1 -- 172.21.15.1:0/3201521618 --> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5646fd6ffdd0 con 0x5646fd822b80 2024-02-02T15:41:28.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.244+0000 7fba06a00700 1 --2- 172.21.15.1:0/3201521618 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x5646fd822b80 0x5646fd824fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:28.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.246+0000 7fba06a00700 1 --2- 172.21.15.1:0/3201521618 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x5646fd822b80 0x5646fd824fa0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=72442e4e6b694ba5 server_cookie=53c85d2fbf9ef04 in_seq=0 out_seq=0 2024-02-02T15:41:28.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.246+0000 7fb9efe00700 10 client.29414 ms_handle_connect on v2:172.21.15.142:6834/1052606547 2024-02-02T15:41:28.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.248+0000 7fb9efe00700 1 -- 172.21.15.1:0/3201521618 <== mds.4 v2:172.21.15.142:6834/1052606547 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5646fd6ffdd0 con 0x5646fd822b80 2024-02-02T15:41:28.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.248+0000 7fb9efe00700 10 client.29414 handle_command_reply: tid=0 2024-02-02T15:41:28.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.248+0000 7fba21107b80 1 client.29414 shutdown 2024-02-02T15:41:28.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.249+0000 7fba21107b80 1 -- 172.21.15.1:0/3201521618 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5646fd7b5930 msgr2=0x5646fd7c5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:41:28.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.249+0000 7fba21107b80 1 --2- 172.21.15.1:0/3201521618 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5646fd7b5930 0x5646fd7c5250 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fb9f0009230 tx=0x7fb9f000b840 comp rx=0 tx=0).stop 2024-02-02T15:41:28.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.250+0000 7fba21107b80 1 -- 172.21.15.1:0/3201521618 shutdown_connections 2024-02-02T15:41:28.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.250+0000 7fba21107b80 1 --2- 172.21.15.1:0/3201521618 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x5646fd822b80 0x5646fd824fa0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:28.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.250+0000 7fba21107b80 1 --2- 172.21.15.1:0/3201521618 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5646fd7c07e0 0x5646fd7c5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:28.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.250+0000 7fba21107b80 1 --2- 172.21.15.1:0/3201521618 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5646fd7b5010 0x5646fd7c6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:28.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.250+0000 7fba21107b80 1 --2- 172.21.15.1:0/3201521618 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5646fd7b5930 0x5646fd7c5250 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:28.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.250+0000 7fba21107b80 1 -- 172.21.15.1:0/3201521618 >> 172.21.15.1:0/3201521618 conn(0x5646fd714db0 msgr2=0x5646fd7b8840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:41:28.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.250+0000 7fb9efe00700 0 client.29414 ms_handle_reset on v2:172.21.15.142:6834/1052606547 2024-02-02T15:41:28.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.250+0000 7fba21107b80 1 -- 172.21.15.1:0/3201521618 shutdown_connections 2024-02-02T15:41:28.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.250+0000 7fba21107b80 1 -- 172.21.15.1:0/3201521618 wait complete. 2024-02-02T15:41:28.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.251+0000 7fba21107b80 20 client.29414 trim_cache size 0 max 16384 2024-02-02T15:41:28.252 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:41:28.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.253+0000 7fba0d600700 1 -- 172.21.15.1:0/4108488577 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fb9e80a0790 msgr2=0x7fb9e80a2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:41:28.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.253+0000 7fba0d600700 1 --2- 172.21.15.1:0/4108488577 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fb9e80a0790 0x7fb9e80a2c50 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb9f8005950 tx=0x7fb9f80058e0 comp rx=0 tx=0).stop 2024-02-02T15:41:28.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.253+0000 7fba0d600700 1 -- 172.21.15.1:0/4108488577 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fba081a1eb0 msgr2=0x7fba081a2330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:41:28.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.253+0000 7fba0d600700 1 --2- 172.21.15.1:0/4108488577 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fba081a1eb0 0x7fba081a2330 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fba0000b780 tx=0x7fba0000bb40 comp rx=0 tx=0).stop 2024-02-02T15:41:28.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.253+0000 7fba0d600700 1 -- 172.21.15.1:0/4108488577 shutdown_connections 2024-02-02T15:41:28.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.253+0000 7fba0d600700 1 --2- 172.21.15.1:0/4108488577 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fb9e80a0790 0x7fb9e80a2c50 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:28.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.253+0000 7fba0d600700 1 --2- 172.21.15.1:0/4108488577 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fba081a1eb0 0x7fba081a2330 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:28.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.253+0000 7fba0d600700 1 --2- 172.21.15.1:0/4108488577 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fba08107600 0x7fba0819d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:28.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.253+0000 7fba0d600700 1 --2- 172.21.15.1:0/4108488577 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fba0810c720 0x7fba0819da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:28.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.253+0000 7fba0d600700 1 -- 172.21.15.1:0/4108488577 >> 172.21.15.1:0/4108488577 conn(0x7fba08076b80 msgr2=0x7fba08076f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:41:28.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.253+0000 7fba0d600700 1 -- 172.21.15.1:0/4108488577 shutdown_connections 2024-02-02T15:41:28.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.253+0000 7fba0d600700 1 -- 172.21.15.1:0/4108488577 wait complete. 2024-02-02T15:41:28.299 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-02-02T15:41:28.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.662+0000 7f5f1c000700 1 -- 172.21.15.1:0/2777122244 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5f14103200 msgr2=0x7f5f141035e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:41:28.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.662+0000 7f5f1c000700 1 --2- 172.21.15.1:0/2777122244 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5f14103200 0x7f5f141035e0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f5f04009990 tx=0x7f5f04009ca0 comp rx=0 tx=0).stop 2024-02-02T15:41:28.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.663+0000 7f5f1c000700 1 -- 172.21.15.1:0/2777122244 shutdown_connections 2024-02-02T15:41:28.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.663+0000 7f5f1c000700 1 --2- 172.21.15.1:0/2777122244 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f14108320 0x7f5f1410fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:28.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.665+0000 7f5f1c000700 1 --2- 172.21.15.1:0/2777122244 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5f14103200 0x7f5f141035e0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:28.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.665+0000 7f5f1c000700 1 --2- 172.21.15.1:0/2777122244 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f14103bb0 0x7f5f14107bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:28.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.665+0000 7f5f1c000700 1 -- 172.21.15.1:0/2777122244 >> 172.21.15.1:0/2777122244 conn(0x7f5f140fea40 msgr2=0x7f5f14100e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:41:28.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.665+0000 7f5f1c000700 1 -- 172.21.15.1:0/2777122244 shutdown_connections 2024-02-02T15:41:28.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.666+0000 7f5f1c000700 1 -- 172.21.15.1:0/2777122244 wait complete. 2024-02-02T15:41:28.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.668+0000 7f5f1c000700 1 Processor -- start 2024-02-02T15:41:28.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.668+0000 7f5f1c000700 1 -- start start 2024-02-02T15:41:28.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.669+0000 7f5f1c000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f14103200 0x7f5f14199090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:28.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.669+0000 7f5f1c000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f14108320 0x7f5f141995d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:28.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.669+0000 7f5f1c000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5f1419da30 0x7f5f1419deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:28.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.669+0000 7f5f1c000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1419e4b0 con 0x7f5f14108320 2024-02-02T15:41:28.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.669+0000 7f5f1c000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1419e620 con 0x7f5f1419da30 2024-02-02T15:41:28.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.669+0000 7f5f1c000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1419e7e0 con 0x7f5f14103200 2024-02-02T15:41:28.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.670+0000 7f5f18c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f14108320 0x7f5f141995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:28.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.670+0000 7f5f18c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f14108320 0x7f5f141995d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33196/0 (socket says 172.21.15.1:33196) 2024-02-02T15:41:28.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.670+0000 7f5f18c00700 1 -- 172.21.15.1:0/4145056602 learned_addr learned my addr 172.21.15.1:0/4145056602 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:41:28.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.671+0000 7f5f18c00700 1 -- 172.21.15.1:0/4145056602 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f14103200 msgr2=0x7f5f14199090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-02T15:41:28.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.671+0000 7f5f18c00700 1 --2- 172.21.15.1:0/4145056602 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f14103200 0x7f5f14199090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:28.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.671+0000 7f5f18c00700 1 -- 172.21.15.1:0/4145056602 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5f1419da30 msgr2=0x7f5f1419deb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-02T15:41:28.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.671+0000 7f5f18c00700 1 --2- 172.21.15.1:0/4145056602 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5f1419da30 0x7f5f1419deb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:28.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.671+0000 7f5f18c00700 1 -- 172.21.15.1:0/4145056602 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f040095f0 con 0x7f5f14108320 2024-02-02T15:41:28.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.672+0000 7f5f18c00700 1 --2- 172.21.15.1:0/4145056602 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f14108320 0x7f5f141995d0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f5efc01f980 tx=0x7f5efc01fd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:41:28.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.673+0000 7f5f12000700 1 -- 172.21.15.1:0/4145056602 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5efc009760 con 0x7f5f14108320 2024-02-02T15:41:28.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.674+0000 7f5f12000700 1 -- 172.21.15.1:0/4145056602 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5efc00cbd0 con 0x7f5f14108320 2024-02-02T15:41:28.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.674+0000 7f5f12000700 1 -- 172.21.15.1:0/4145056602 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5efc0296c0 con 0x7f5f14108320 2024-02-02T15:41:28.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.674+0000 7f5f1c000700 1 -- 172.21.15.1:0/4145056602 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f1406a760 con 0x7f5f14108320 2024-02-02T15:41:28.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.674+0000 7f5f1c000700 1 -- 172.21.15.1:0/4145056602 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f1406ac20 con 0x7f5f14108320 2024-02-02T15:41:28.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.688+0000 7f5f12000700 1 -- 172.21.15.1:0/4145056602 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f5efc021440 con 0x7f5f14108320 2024-02-02T15:41:28.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.690+0000 7f5f12000700 1 --2- 172.21.15.1:0/4145056602 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f5ef40a0820 0x7f5ef40a2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:28.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:41:28 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/1767583488' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:41:28.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.690+0000 7f5f12000700 1 -- 172.21.15.1:0/4145056602 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f5efc025070 con 0x7f5f14108320 2024-02-02T15:41:28.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.691+0000 7f5f19600700 1 --2- 172.21.15.1:0/4145056602 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f5ef40a0820 0x7f5ef40a2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:28.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.693+0000 7f5f19600700 1 --2- 172.21.15.1:0/4145056602 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f5ef40a0820 0x7f5ef40a2ce0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f5f04006010 tx=0x7f5f04005da0 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:41:28.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.699+0000 7f5f2fb69b80 1 Processor -- start 2024-02-02T15:41:28.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.700+0000 7f5f2fb69b80 1 -- start start 2024-02-02T15:41:28.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.702+0000 7f5f2fb69b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557e45f8f010 0x557e45f8f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:28.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.702+0000 7f5f2fb69b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557e45f8f930 0x557e45fa2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:28.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.703+0000 7f5f2fb69b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e45fa2ff0 0x557e45fa53e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:28.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.703+0000 7f5f2fb69b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e45f034c0 con 0x557e45fa2ff0 2024-02-02T15:41:28.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.703+0000 7f5f2fb69b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e45ee7aa0 con 0x557e45f8f010 2024-02-02T15:41:28.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.703+0000 7f5f2fb69b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e45ee5460 con 0x557e45f8f930 2024-02-02T15:41:28.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.704+0000 7f5f18c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557e45f8f930 0x557e45fa2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:28.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.704+0000 7f5f18c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557e45f8f930 0x557e45fa2ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:35436/0 (socket says 172.21.15.1:35436) 2024-02-02T15:41:28.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.704+0000 7f5f18c00700 1 -- 172.21.15.1:0/3621283679 learned_addr learned my addr 172.21.15.1:0/3621283679 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:41:28.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.704+0000 7f5f1a000700 1 --2- 172.21.15.1:0/3621283679 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e45fa2ff0 0x557e45fa53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:28.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.704+0000 7f5f1a000700 1 -- 172.21.15.1:0/3621283679 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557e45f8f930 msgr2=0x557e45fa2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:41:28.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.704+0000 7f5f1a000700 1 --2- 172.21.15.1:0/3621283679 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557e45f8f930 0x557e45fa2ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:28.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.704+0000 7f5f1a000700 1 -- 172.21.15.1:0/3621283679 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557e45f8f010 msgr2=0x557e45f8f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:41:28.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.702+0000 7f5f19600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557e45f8f010 0x557e45f8f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:28.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.706+0000 7f5f1a000700 1 --2- 172.21.15.1:0/3621283679 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557e45f8f010 0x557e45f8f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:28.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.706+0000 7f5f1a000700 1 -- 172.21.15.1:0/3621283679 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e45d623b0 con 0x557e45fa2ff0 2024-02-02T15:41:28.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.706+0000 7f5f19600700 1 --2- 172.21.15.1:0/3621283679 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557e45f8f010 0x557e45f8f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:41:28.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.707+0000 7f5f1a000700 1 --2- 172.21.15.1:0/3621283679 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e45fa2ff0 0x557e45fa53e0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c009aa0 tx=0x7f5f0c009db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2857ea8ba66fdb81 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:41:28.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.707+0000 7f5f0a600700 1 -- 172.21.15.1:0/3621283679 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f0c017070 con 0x557e45fa2ff0 2024-02-02T15:41:28.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.707+0000 7f5f0a600700 1 -- 172.21.15.1:0/3621283679 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f0c00f040 con 0x557e45fa2ff0 2024-02-02T15:41:28.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.707+0000 7f5f0a600700 1 -- 172.21.15.1:0/3621283679 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f0c004580 con 0x557e45fa2ff0 2024-02-02T15:41:28.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.710+0000 7f5f2fb69b80 1 -- 172.21.15.1:0/3621283679 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e45fa2ff0 msgr2=0x557e45fa53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:41:28.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.710+0000 7f5f2fb69b80 1 --2- 172.21.15.1:0/3621283679 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e45fa2ff0 0x557e45fa53e0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c009aa0 tx=0x7f5f0c009db0 comp rx=0 tx=0).stop 2024-02-02T15:41:28.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.711+0000 7f5f2fb69b80 1 -- 172.21.15.1:0/3621283679 shutdown_connections 2024-02-02T15:41:28.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.711+0000 7f5f2fb69b80 1 --2- 172.21.15.1:0/3621283679 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e45fa2ff0 0x557e45fa53e0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:28.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.711+0000 7f5f2fb69b80 1 --2- 172.21.15.1:0/3621283679 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557e45f8f010 0x557e45f8f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:28.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.711+0000 7f5f2fb69b80 1 --2- 172.21.15.1:0/3621283679 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557e45f8f930 0x557e45fa2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:28.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.711+0000 7f5f2fb69b80 1 -- 172.21.15.1:0/3621283679 >> 172.21.15.1:0/3621283679 conn(0x557e45eeedb0 msgr2=0x557e45f8b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:41:28.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.713+0000 7f5f2fb69b80 1 -- 172.21.15.1:0/3621283679 shutdown_connections 2024-02-02T15:41:28.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.713+0000 7f5f2fb69b80 1 -- 172.21.15.1:0/3621283679 wait complete. 2024-02-02T15:41:28.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.715+0000 7f5f2fb69b80 1 -- start start 2024-02-02T15:41:28.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.715+0000 7f5f2fb69b80 1 Processor -- start 2024-02-02T15:41:28.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.716+0000 7f5f2fb69b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e45f8f010 0x557e45fa2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:28.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.716+0000 7f5f2fb69b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557e45f8f930 0x557e45f9fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:28.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.716+0000 7f5f2fb69b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557e45fa2ff0 0x557e45fa0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:28.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.716+0000 7f5f2fb69b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e45ee0d50 con 0x557e45f8f010 2024-02-02T15:41:28.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.716+0000 7f5f2fb69b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e45e7cd80 con 0x557e45fa2ff0 2024-02-02T15:41:28.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.716+0000 7f5f2fb69b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e45e7ca20 con 0x557e45f8f930 2024-02-02T15:41:28.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.716+0000 7f5f2fb69b80 10 client.0 _ll_register_callbacks cb 0x557e45e7e290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:41:28.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.716+0000 7f5f1a000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557e45fa2ff0 0x557e45fa0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:28.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.716+0000 7f5f1a000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557e45fa2ff0 0x557e45fa0390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59422/0 (socket says 172.21.15.1:59422) 2024-02-02T15:41:28.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.716+0000 7f5f1a000700 1 -- 172.21.15.1:0/2116490535 learned_addr learned my addr 172.21.15.1:0/2116490535 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:41:28.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.717+0000 7f5f1a000700 1 -- 172.21.15.1:0/2116490535 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557e45f8f930 msgr2=0x557e45f9fc50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-02T15:41:28.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.717+0000 7f5f1a000700 1 --2- 172.21.15.1:0/2116490535 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557e45f8f930 0x557e45f9fc50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:28.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.717+0000 7f5f1a000700 1 -- 172.21.15.1:0/2116490535 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e45f8f010 msgr2=0x557e45fa2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:41:28.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.717+0000 7f5f1a000700 1 --2- 172.21.15.1:0/2116490535 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e45f8f010 0x557e45fa2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:28.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.717+0000 7f5f1a000700 1 -- 172.21.15.1:0/2116490535 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f0c0093f0 con 0x557e45fa2ff0 2024-02-02T15:41:28.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.717+0000 7f5f1a000700 1 --2- 172.21.15.1:0/2116490535 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557e45fa2ff0 0x557e45fa0390 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c005400 tx=0x7f5f0c0054e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=888b47223dab4d6e server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:41:28.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.717+0000 7f5f02a00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:41:28.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.718+0000 7f5f02a00700 1 -- 172.21.15.1:0/2116490535 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f0c017070 con 0x557e45fa2ff0 2024-02-02T15:41:28.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.718+0000 7f5f02a00700 1 -- 172.21.15.1:0/2116490535 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f5f0c01d450 con 0x557e45fa2ff0 2024-02-02T15:41:28.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.718+0000 7f5f2fb69b80 1 -- 172.21.15.1:0/2116490535 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557e45f830f0 con 0x557e45fa2ff0 2024-02-02T15:41:28.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.718+0000 7f5f02a00700 1 -- 172.21.15.1:0/2116490535 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f0c007c00 con 0x557e45fa2ff0 2024-02-02T15:41:28.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.719+0000 7f5f2fb69b80 10 client.29420 fetch_fsmap learned FSMap version 61 2024-02-02T15:41:28.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.719+0000 7f5f2fb69b80 1 -- 172.21.15.1:0/2116490535 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x557e45d0eb60 con 0x557e45fa2ff0 2024-02-02T15:41:28.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.719+0000 7f5f02a00700 1 -- 172.21.15.1:0/2116490535 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557e45f830f0 con 0x557e45fa2ff0 2024-02-02T15:41:28.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.720+0000 7f5f02a00700 1 -- 172.21.15.1:0/2116490535 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 61) v1 ==== 7550+0+0 (secure 0 0 0) 0x7f5f0c012070 con 0x557e45fa2ff0 2024-02-02T15:41:28.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.720+0000 7f5f2fb69b80 10 client.29420 fetch_fsmap finished waiting for FSMap version 61 2024-02-02T15:41:28.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.720+0000 7f5f2fb69b80 10 client.29420 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-02-02T15:41:28.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.720+0000 7f5f2fb69b80 20 client.29420 populate_metadata read hostname 'smithi001' 2024-02-02T15:41:28.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.720+0000 7f5f2fb69b80 1 --2- 172.21.15.1:0/2116490535 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x557e45fb2840 0x557e45fb4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:28.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.720+0000 7f5f2fb69b80 4 client.29420 mds_command: new command op to 24514 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:41:28.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.720+0000 7f5f2fb69b80 1 -- 172.21.15.1:0/2116490535 --> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557e45ed9dd0 con 0x557e45fb2840 2024-02-02T15:41:28.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.721+0000 7f5f19600700 1 --2- 172.21.15.1:0/2116490535 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x557e45fb2840 0x557e45fb4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:28.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.722+0000 7f5f19600700 1 --2- 172.21.15.1:0/2116490535 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x557e45fb2840 0x557e45fb4c60 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=342acae07a303f34 server_cookie=305d310d2b0a89aa in_seq=0 out_seq=0 2024-02-02T15:41:28.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.722+0000 7f5f02a00700 10 client.29420 ms_handle_connect on v2:172.21.15.142:6836/2870289019 2024-02-02T15:41:28.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.723+0000 7f5f02a00700 1 -- 172.21.15.1:0/2116490535 <== mds.3 v2:172.21.15.142:6836/2870289019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x557e45ed9dd0 con 0x557e45fb2840 2024-02-02T15:41:28.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.723+0000 7f5f02a00700 10 client.29420 handle_command_reply: tid=0 2024-02-02T15:41:28.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.723+0000 7f5f2fb69b80 1 client.29420 shutdown 2024-02-02T15:41:28.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.723+0000 7f5f2fb69b80 1 -- 172.21.15.1:0/2116490535 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557e45fa2ff0 msgr2=0x557e45fa0390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:41:28.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.723+0000 7f5f2fb69b80 1 --2- 172.21.15.1:0/2116490535 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557e45fa2ff0 0x557e45fa0390 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c005400 tx=0x7f5f0c0054e0 comp rx=0 tx=0).stop 2024-02-02T15:41:28.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.723+0000 7f5f2fb69b80 1 -- 172.21.15.1:0/2116490535 shutdown_connections 2024-02-02T15:41:28.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.723+0000 7f5f2fb69b80 1 --2- 172.21.15.1:0/2116490535 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557e45fa2ff0 0x557e45fa0390 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:28.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.723+0000 7f5f2fb69b80 1 --2- 172.21.15.1:0/2116490535 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x557e45fb2840 0x557e45fb4c60 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:28.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.723+0000 7f5f2fb69b80 1 --2- 172.21.15.1:0/2116490535 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e45f8f010 0x557e45fa2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:28.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.723+0000 7f5f2fb69b80 1 --2- 172.21.15.1:0/2116490535 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557e45f8f930 0x557e45f9fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:28.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.723+0000 7f5f02a00700 0 client.29420 ms_handle_reset on v2:172.21.15.142:6836/2870289019 2024-02-02T15:41:28.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.723+0000 7f5f2fb69b80 1 -- 172.21.15.1:0/2116490535 >> 172.21.15.1:0/2116490535 conn(0x557e45eeedb0 msgr2=0x557e45f8afa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:41:28.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.723+0000 7f5f2fb69b80 1 -- 172.21.15.1:0/2116490535 shutdown_connections 2024-02-02T15:41:28.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.723+0000 7f5f2fb69b80 1 -- 172.21.15.1:0/2116490535 wait complete. 2024-02-02T15:41:28.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.724+0000 7f5f2fb69b80 20 client.29420 trim_cache size 0 max 16384 2024-02-02T15:41:28.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.737+0000 7f5f2fb69b80 1 Processor -- start 2024-02-02T15:41:28.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.737+0000 7f5f2fb69b80 1 -- start start 2024-02-02T15:41:28.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.738+0000 7f5f2fb69b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e45f8f010 0x557e45fa2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:28.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.738+0000 7f5f2fb69b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557e45f8f930 0x557e45f9f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:28.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.738+0000 7f5f2fb69b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557e45fa2ff0 0x557e45f9fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:28.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.738+0000 7f5f2fb69b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e45d646a0 con 0x557e45f8f010 2024-02-02T15:41:28.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.738+0000 7f5f2fb69b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e45f9bac0 con 0x557e45f8f930 2024-02-02T15:41:28.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.738+0000 7f5f2fb69b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e45fa0fa0 con 0x557e45fa2ff0 2024-02-02T15:41:28.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.738+0000 7f5f19600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e45f8f010 0x557e45fa2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:28.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.738+0000 7f5f1a000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557e45fa2ff0 0x557e45f9fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:28.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.738+0000 7f5f18c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557e45f8f930 0x557e45f9f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:28.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.738+0000 7f5f1a000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557e45fa2ff0 0x557e45f9fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:35458/0 (socket says 172.21.15.1:35458) 2024-02-02T15:41:28.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.738+0000 7f5f1a000700 1 -- 172.21.15.1:0/140359604 learned_addr learned my addr 172.21.15.1:0/140359604 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:41:28.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.738+0000 7f5f18c00700 1 --2- 172.21.15.1:0/140359604 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557e45f8f930 0x557e45f9f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59424/0 (socket says 172.21.15.1:59424) 2024-02-02T15:41:28.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.738+0000 7f5f18c00700 1 -- 172.21.15.1:0/140359604 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557e45fa2ff0 msgr2=0x557e45f9fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:41:28.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.738+0000 7f5f18c00700 1 --2- 172.21.15.1:0/140359604 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557e45fa2ff0 0x557e45f9fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:28.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.738+0000 7f5f18c00700 1 -- 172.21.15.1:0/140359604 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e45f8f010 msgr2=0x557e45fa2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:41:28.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.738+0000 7f5f18c00700 1 --2- 172.21.15.1:0/140359604 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e45f8f010 0x557e45fa2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:28.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.738+0000 7f5f18c00700 1 -- 172.21.15.1:0/140359604 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e45ea6ea0 con 0x557e45f8f930 2024-02-02T15:41:28.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.738+0000 7f5f19600700 1 --2- 172.21.15.1:0/140359604 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e45f8f010 0x557e45fa2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:41:28.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.739+0000 7f5f18c00700 1 --2- 172.21.15.1:0/140359604 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557e45f8f930 0x557e45f9f4f0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f5efc0a69c0 tx=0x7f5efc0aa290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46578b15a17b8377 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:41:28.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.739+0000 7f5f0b000700 1 -- 172.21.15.1:0/140359604 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5efc0ae2f0 con 0x557e45f8f930 2024-02-02T15:41:28.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.739+0000 7f5f0b000700 1 -- 172.21.15.1:0/140359604 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5efc0b2530 con 0x557e45f8f930 2024-02-02T15:41:28.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.739+0000 7f5f0b000700 1 -- 172.21.15.1:0/140359604 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5efc0a99b0 con 0x557e45f8f930 2024-02-02T15:41:28.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.740+0000 7f5f2fb69b80 1 -- 172.21.15.1:0/140359604 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557e45f8f930 msgr2=0x557e45f9f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:41:28.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.740+0000 7f5f2fb69b80 1 --2- 172.21.15.1:0/140359604 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557e45f8f930 0x557e45f9f4f0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f5efc0a69c0 tx=0x7f5efc0aa290 comp rx=0 tx=0).stop 2024-02-02T15:41:28.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.740+0000 7f5f2fb69b80 1 -- 172.21.15.1:0/140359604 shutdown_connections 2024-02-02T15:41:28.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.740+0000 7f5f2fb69b80 1 --2- 172.21.15.1:0/140359604 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557e45fa2ff0 0x557e45f9fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:28.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.740+0000 7f5f2fb69b80 1 --2- 172.21.15.1:0/140359604 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e45f8f010 0x557e45fa2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:28.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.740+0000 7f5f2fb69b80 1 --2- 172.21.15.1:0/140359604 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557e45f8f930 0x557e45f9f4f0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:28.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.740+0000 7f5f2fb69b80 1 -- 172.21.15.1:0/140359604 >> 172.21.15.1:0/140359604 conn(0x557e45eeedb0 msgr2=0x557e45faed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:41:28.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.740+0000 7f5f2fb69b80 1 -- 172.21.15.1:0/140359604 shutdown_connections 2024-02-02T15:41:28.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.740+0000 7f5f2fb69b80 1 -- 172.21.15.1:0/140359604 wait complete. 2024-02-02T15:41:28.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.740+0000 7f5f2fb69b80 1 -- start start 2024-02-02T15:41:28.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.740+0000 7f5f2fb69b80 1 Processor -- start 2024-02-02T15:41:28.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.741+0000 7f5f2fb69b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557e45f8f010 0x557e45fa9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:28.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.741+0000 7f5f2fb69b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e45f8f930 0x557e45fa7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:28.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.741+0000 7f5f2fb69b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557e45fa2ff0 0x557e45fa80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:28.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.741+0000 7f5f2fb69b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e45fa8640 con 0x557e45f8f930 2024-02-02T15:41:28.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.741+0000 7f5f2fb69b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e45fa8780 con 0x557e45f8f010 2024-02-02T15:41:28.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.741+0000 7f5f2fb69b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e45fa8910 con 0x557e45fa2ff0 2024-02-02T15:41:28.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.741+0000 7f5f2fb69b80 10 client.0 _ll_register_callbacks cb 0x557e45e7e290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:41:28.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.741+0000 7f5f18c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e45f8f930 0x557e45fa7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:28.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.741+0000 7f5f19600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557e45f8f010 0x557e45fa9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:28.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.741+0000 7f5f18c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e45f8f930 0x557e45fa7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33242/0 (socket says 172.21.15.1:33242) 2024-02-02T15:41:28.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.741+0000 7f5f18c00700 1 -- 172.21.15.1:0/2630909628 learned_addr learned my addr 172.21.15.1:0/2630909628 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:41:28.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.741+0000 7f5f1a000700 1 --2- 172.21.15.1:0/2630909628 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557e45fa2ff0 0x557e45fa80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:28.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.742+0000 7f5f19600700 1 -- 172.21.15.1:0/2630909628 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557e45fa2ff0 msgr2=0x557e45fa80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:41:28.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.742+0000 7f5f19600700 1 --2- 172.21.15.1:0/2630909628 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557e45fa2ff0 0x557e45fa80c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:28.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.742+0000 7f5f19600700 1 -- 172.21.15.1:0/2630909628 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e45f8f930 msgr2=0x557e45fa7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:41:28.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.742+0000 7f5f19600700 1 --2- 172.21.15.1:0/2630909628 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e45f8f930 0x557e45fa7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:28.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.742+0000 7f5f19600700 1 -- 172.21.15.1:0/2630909628 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f1406ac20 con 0x557e45f8f010 2024-02-02T15:41:28.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.742+0000 7f5f18c00700 1 --2- 172.21.15.1:0/2630909628 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e45f8f930 0x557e45fa7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:41:28.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.742+0000 7f5f19600700 1 --2- 172.21.15.1:0/2630909628 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557e45f8f010 0x557e45fa9840 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f5f0400b2e0 tx=0x7f5f04009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8e7aeb1e161567e server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:41:28.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.742+0000 7f5f02a00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:41:28.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.742+0000 7f5f02a00700 1 -- 172.21.15.1:0/2630909628 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f0402a040 con 0x557e45f8f010 2024-02-02T15:41:28.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.742+0000 7f5f02a00700 1 -- 172.21.15.1:0/2630909628 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f5f0402f310 con 0x557e45f8f010 2024-02-02T15:41:28.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.742+0000 7f5f2fb69b80 1 -- 172.21.15.1:0/2630909628 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5f0c00f040 con 0x557e45f8f010 2024-02-02T15:41:28.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.742+0000 7f5f02a00700 1 -- 172.21.15.1:0/2630909628 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5f0401d070 con 0x557e45f8f010 2024-02-02T15:41:28.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.743+0000 7f5f2fb69b80 10 client.29426 fetch_fsmap learned FSMap version 61 2024-02-02T15:41:28.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.743+0000 7f5f2fb69b80 1 -- 172.21.15.1:0/2630909628 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x557e45d66cd0 con 0x557e45f8f010 2024-02-02T15:41:28.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.743+0000 7f5f02a00700 1 -- 172.21.15.1:0/2630909628 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f0c00f040 con 0x557e45f8f010 2024-02-02T15:41:28.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.743+0000 7f5f02a00700 1 -- 172.21.15.1:0/2630909628 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 61) v1 ==== 7550+0+0 (secure 0 0 0) 0x7f5f0402fbc0 con 0x557e45f8f010 2024-02-02T15:41:28.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.743+0000 7f5f2fb69b80 10 client.29426 fetch_fsmap finished waiting for FSMap version 61 2024-02-02T15:41:28.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.743+0000 7f5f2fb69b80 10 client.29426 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-02-02T15:41:28.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.743+0000 7f5f2fb69b80 20 client.29426 populate_metadata read hostname 'smithi001' 2024-02-02T15:41:28.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.743+0000 7f5f2fb69b80 1 --2- 172.21.15.1:0/2630909628 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x557e460057d0 0x557e46007bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:28.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.743+0000 7f5f2fb69b80 4 client.29426 mds_command: new command op to 24514 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:41:28.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.743+0000 7f5f2fb69b80 1 -- 172.21.15.1:0/2630909628 --> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557e45e9ff50 con 0x557e460057d0 2024-02-02T15:41:28.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.743+0000 7f5f18c00700 1 --2- 172.21.15.1:0/2630909628 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x557e460057d0 0x557e46007bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:28.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.744+0000 7f5f18c00700 1 --2- 172.21.15.1:0/2630909628 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x557e460057d0 0x557e46007bf0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=64ca981bce67ba14 server_cookie=51814831b7518d02 in_seq=0 out_seq=0 2024-02-02T15:41:28.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.744+0000 7f5f02a00700 10 client.29426 ms_handle_connect on v2:172.21.15.142:6836/2870289019 2024-02-02T15:41:28.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.744+0000 7f5f02a00700 1 -- 172.21.15.1:0/2630909628 <== mds.3 v2:172.21.15.142:6836/2870289019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557e45e9ff50 con 0x557e460057d0 2024-02-02T15:41:28.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.744+0000 7f5f02a00700 10 client.29426 handle_command_reply: tid=0 2024-02-02T15:41:28.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.744+0000 7f5f2fb69b80 1 client.29426 shutdown 2024-02-02T15:41:28.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.744+0000 7f5f2fb69b80 1 -- 172.21.15.1:0/2630909628 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557e45f8f010 msgr2=0x557e45fa9840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:41:28.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.744+0000 7f5f2fb69b80 1 --2- 172.21.15.1:0/2630909628 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557e45f8f010 0x557e45fa9840 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f5f0400b2e0 tx=0x7f5f04009ca0 comp rx=0 tx=0).stop 2024-02-02T15:41:28.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.744+0000 7f5f2fb69b80 1 -- 172.21.15.1:0/2630909628 shutdown_connections 2024-02-02T15:41:28.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.744+0000 7f5f2fb69b80 1 --2- 172.21.15.1:0/2630909628 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557e45fa2ff0 0x557e45fa80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:28.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.744+0000 7f5f2fb69b80 1 --2- 172.21.15.1:0/2630909628 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557e45f8f010 0x557e45fa9840 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:28.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.744+0000 7f5f2fb69b80 1 --2- 172.21.15.1:0/2630909628 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x557e460057d0 0x557e46007bf0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:28.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.744+0000 7f5f2fb69b80 1 --2- 172.21.15.1:0/2630909628 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557e45f8f930 0x557e45fa7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:28.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.744+0000 7f5f02a00700 0 client.29426 ms_handle_reset on v2:172.21.15.142:6836/2870289019 2024-02-02T15:41:28.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.744+0000 7f5f2fb69b80 1 -- 172.21.15.1:0/2630909628 >> 172.21.15.1:0/2630909628 conn(0x557e45eeedb0 msgr2=0x557e45f90ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:41:28.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.744+0000 7f5f2fb69b80 1 -- 172.21.15.1:0/2630909628 shutdown_connections 2024-02-02T15:41:28.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.745+0000 7f5f2fb69b80 1 -- 172.21.15.1:0/2630909628 wait complete. 2024-02-02T15:41:28.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.745+0000 7f5f2fb69b80 20 client.29426 trim_cache size 0 max 16384 2024-02-02T15:41:28.751 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:41:28.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.745+0000 7f5f1c000700 1 -- 172.21.15.1:0/4145056602 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f5ef40a0820 msgr2=0x7f5ef40a2ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:41:28.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.745+0000 7f5f1c000700 1 --2- 172.21.15.1:0/4145056602 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f5ef40a0820 0x7f5ef40a2ce0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f5f04006010 tx=0x7f5f04005da0 comp rx=0 tx=0).stop 2024-02-02T15:41:28.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.745+0000 7f5f1c000700 1 -- 172.21.15.1:0/4145056602 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f14108320 msgr2=0x7f5f141995d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:41:28.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.745+0000 7f5f1c000700 1 --2- 172.21.15.1:0/4145056602 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f14108320 0x7f5f141995d0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f5efc01f980 tx=0x7f5efc01fd40 comp rx=0 tx=0).stop 2024-02-02T15:41:28.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.745+0000 7f5f1c000700 1 -- 172.21.15.1:0/4145056602 shutdown_connections 2024-02-02T15:41:28.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.745+0000 7f5f1c000700 1 --2- 172.21.15.1:0/4145056602 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f5ef40a0820 0x7f5ef40a2ce0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:28.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.745+0000 7f5f1c000700 1 --2- 172.21.15.1:0/4145056602 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5f1419da30 0x7f5f1419deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:28.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.745+0000 7f5f1c000700 1 --2- 172.21.15.1:0/4145056602 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5f14103200 0x7f5f14199090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:28.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.745+0000 7f5f1c000700 1 --2- 172.21.15.1:0/4145056602 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5f14108320 0x7f5f141995d0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:28.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.745+0000 7f5f1c000700 1 -- 172.21.15.1:0/4145056602 >> 172.21.15.1:0/4145056602 conn(0x7f5f140fea40 msgr2=0x7f5f14106bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:41:28.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.745+0000 7f5f1c000700 1 -- 172.21.15.1:0/4145056602 shutdown_connections 2024-02-02T15:41:28.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:28.746+0000 7f5f1c000700 1 -- 172.21.15.1:0/4145056602 wait complete. 2024-02-02T15:41:28.774 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-02T15:41:28.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:41:28 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/1767583488' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:41:28.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:41:28 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/1767583488' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:41:28.974 INFO:tasks.workunit.client.0.smithi001.stdout: 10 33194 32.16 MB/sec warmup 61 sec latency 69.452 ms 2024-02-02T15:41:29.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.062+0000 7f9c33600700 1 -- 172.21.15.1:0/613055131 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9c2c106120 msgr2=0x7f9c2c10dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:41:29.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.062+0000 7f9c33600700 1 --2- 172.21.15.1:0/613055131 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9c2c106120 0x7f9c2c10dca0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f9c24009f90 tx=0x7f9c2400b330 comp rx=0 tx=0).stop 2024-02-02T15:41:29.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.062+0000 7f9c33600700 1 -- 172.21.15.1:0/613055131 shutdown_connections 2024-02-02T15:41:29.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.062+0000 7f9c33600700 1 --2- 172.21.15.1:0/613055131 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9c2c106120 0x7f9c2c10dca0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.062+0000 7f9c33600700 1 --2- 172.21.15.1:0/613055131 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9c2c101000 0x7f9c2c1013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.062+0000 7f9c33600700 1 --2- 172.21.15.1:0/613055131 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9c2c1019b0 0x7f9c2c1059f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.062+0000 7f9c33600700 1 -- 172.21.15.1:0/613055131 >> 172.21.15.1:0/613055131 conn(0x7f9c2c0fc860 msgr2=0x7f9c2c0fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:41:29.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.062+0000 7f9c33600700 1 -- 172.21.15.1:0/613055131 shutdown_connections 2024-02-02T15:41:29.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.062+0000 7f9c33600700 1 -- 172.21.15.1:0/613055131 wait complete. 2024-02-02T15:41:29.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.064+0000 7f9c33600700 1 Processor -- start 2024-02-02T15:41:29.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.064+0000 7f9c33600700 1 -- start start 2024-02-02T15:41:29.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.064+0000 7f9c33600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9c2c101000 0x7f9c2c10db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:29.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.064+0000 7f9c33600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9c2c106120 0x7f9c2c109b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:29.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.064+0000 7f9c33600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9c2c10a0b0 0x7f9c2c10a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:29.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.064+0000 7f9c33600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2c1a9a90 con 0x7f9c2c106120 2024-02-02T15:41:29.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.064+0000 7f9c33600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2c1a9c00 con 0x7f9c2c10a0b0 2024-02-02T15:41:29.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.064+0000 7f9c33600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2c1a9dc0 con 0x7f9c2c101000 2024-02-02T15:41:29.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.065+0000 7f9c2be00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9c2c106120 0x7f9c2c109b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:29.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.065+0000 7f9c2be00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9c2c106120 0x7f9c2c109b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33266/0 (socket says 172.21.15.1:33266) 2024-02-02T15:41:29.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.065+0000 7f9c2be00700 1 -- 172.21.15.1:0/582746872 learned_addr learned my addr 172.21.15.1:0/582746872 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:41:29.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.065+0000 7f9c31600700 1 --2- 172.21.15.1:0/582746872 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9c2c10a0b0 0x7f9c2c10a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:29.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.066+0000 7f9c31600700 1 -- 172.21.15.1:0/582746872 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9c2c101000 msgr2=0x7f9c2c10db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:41:29.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.066+0000 7f9c31600700 1 --2- 172.21.15.1:0/582746872 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9c2c101000 0x7f9c2c10db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.066+0000 7f9c31600700 1 -- 172.21.15.1:0/582746872 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9c2c106120 msgr2=0x7f9c2c109b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:41:29.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.066+0000 7f9c31600700 1 --2- 172.21.15.1:0/582746872 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9c2c106120 0x7f9c2c109b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.066+0000 7f9c31600700 1 -- 172.21.15.1:0/582746872 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c24009c40 con 0x7f9c2c10a0b0 2024-02-02T15:41:29.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.066+0000 7f9c31600700 1 --2- 172.21.15.1:0/582746872 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9c2c10a0b0 0x7f9c2c10a520 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f9c24009f90 tx=0x7f9c240075b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:41:29.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.067+0000 7f9c29600700 1 -- 172.21.15.1:0/582746872 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c2400e040 con 0x7f9c2c10a0b0 2024-02-02T15:41:29.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.067+0000 7f9c33600700 1 -- 172.21.15.1:0/582746872 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c2c1aa040 con 0x7f9c2c10a0b0 2024-02-02T15:41:29.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.067+0000 7f9c33600700 1 -- 172.21.15.1:0/582746872 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c2c1aa4c0 con 0x7f9c2c10a0b0 2024-02-02T15:41:29.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.067+0000 7f9c29600700 1 -- 172.21.15.1:0/582746872 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c2400bbf0 con 0x7f9c2c10a0b0 2024-02-02T15:41:29.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.067+0000 7f9c29600700 1 -- 172.21.15.1:0/582746872 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c240052a0 con 0x7f9c2c10a0b0 2024-02-02T15:41:29.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.073+0000 7f9c4718cb80 1 Processor -- start 2024-02-02T15:41:29.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.073+0000 7f9c4718cb80 1 -- start start 2024-02-02T15:41:29.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.074+0000 7f9c4718cb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56151d23f010 0x56151d23f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:29.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.074+0000 7f9c29600700 1 -- 172.21.15.1:0/582746872 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f9c2401d470 con 0x7f9c2c10a0b0 2024-02-02T15:41:29.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.074+0000 7f9c4718cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56151d23f930 0x56151d24a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:29.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.074+0000 7f9c4718cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56151d24a7e0 0x56151d24cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:29.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.074+0000 7f9c4718cb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56151d1b34c0 con 0x56151d23f010 2024-02-02T15:41:29.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.074+0000 7f9c4718cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56151d197aa0 con 0x56151d24a7e0 2024-02-02T15:41:29.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.074+0000 7f9c4718cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56151d195460 con 0x56151d23f930 2024-02-02T15:41:29.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.074+0000 7f9c30c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56151d23f930 0x56151d24a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:29.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.075+0000 7f9c30c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56151d23f930 0x56151d24a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:35508/0 (socket says 172.21.15.1:35508) 2024-02-02T15:41:29.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.075+0000 7f9c30c00700 1 -- 172.21.15.1:0/2075131607 learned_addr learned my addr 172.21.15.1:0/2075131607 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:41:29.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.075+0000 7f9c30c00700 1 -- 172.21.15.1:0/2075131607 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56151d24a7e0 msgr2=0x56151d24cbd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-02T15:41:29.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.075+0000 7f9c30c00700 1 --2- 172.21.15.1:0/2075131607 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56151d24a7e0 0x56151d24cbd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.075+0000 7f9c30c00700 1 -- 172.21.15.1:0/2075131607 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56151d23f010 msgr2=0x56151d23f3f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-02T15:41:29.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.075+0000 7f9c30c00700 1 --2- 172.21.15.1:0/2075131607 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56151d23f010 0x56151d23f3f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.075+0000 7f9c30c00700 1 -- 172.21.15.1:0/2075131607 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56151d0123b0 con 0x56151d23f930 2024-02-02T15:41:29.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.075+0000 7f9c30c00700 1 --2- 172.21.15.1:0/2075131607 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56151d23f930 0x56151d24a2a0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c009d00 tx=0x7f9c1c00e630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47e96d7bbc81288f server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:41:29.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.076+0000 7f9c21c00700 1 -- 172.21.15.1:0/2075131607 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c1c00ee40 con 0x56151d23f930 2024-02-02T15:41:29.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.076+0000 7f9c21c00700 1 -- 172.21.15.1:0/2075131607 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c1c009e50 con 0x56151d23f930 2024-02-02T15:41:29.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.076+0000 7f9c21c00700 1 -- 172.21.15.1:0/2075131607 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c1c010430 con 0x56151d23f930 2024-02-02T15:41:29.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.076+0000 7f9c4718cb80 1 -- 172.21.15.1:0/2075131607 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56151d23f930 msgr2=0x56151d24a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:41:29.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.076+0000 7f9c4718cb80 1 --2- 172.21.15.1:0/2075131607 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56151d23f930 0x56151d24a2a0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c009d00 tx=0x7f9c1c00e630 comp rx=0 tx=0).stop 2024-02-02T15:41:29.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.079+0000 7f9c4718cb80 1 -- 172.21.15.1:0/2075131607 shutdown_connections 2024-02-02T15:41:29.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.079+0000 7f9c4718cb80 1 --2- 172.21.15.1:0/2075131607 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56151d24a7e0 0x56151d24cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.079+0000 7f9c4718cb80 1 --2- 172.21.15.1:0/2075131607 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56151d23f010 0x56151d23f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.079+0000 7f9c4718cb80 1 --2- 172.21.15.1:0/2075131607 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56151d23f930 0x56151d24a2a0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.079+0000 7f9c4718cb80 1 -- 172.21.15.1:0/2075131607 >> 172.21.15.1:0/2075131607 conn(0x56151d19edb0 msgr2=0x56151d23b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:41:29.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.080+0000 7f9c29600700 1 --2- 172.21.15.1:0/582746872 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f9c0c0a0790 0x7f9c0c0a2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:29.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.080+0000 7f9c4718cb80 1 -- 172.21.15.1:0/2075131607 shutdown_connections 2024-02-02T15:41:29.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.080+0000 7f9c29600700 1 -- 172.21.15.1:0/582746872 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f9c240cd770 con 0x7f9c2c10a0b0 2024-02-02T15:41:29.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.081+0000 7f9c30c00700 1 --2- 172.21.15.1:0/582746872 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f9c0c0a0790 0x7f9c0c0a2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:29.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.081+0000 7f9c4718cb80 1 -- 172.21.15.1:0/2075131607 wait complete. 2024-02-02T15:41:29.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.082+0000 7f9c4718cb80 1 -- start start 2024-02-02T15:41:29.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.083+0000 7f9c4718cb80 1 Processor -- start 2024-02-02T15:41:29.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.084+0000 7f9c4718cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56151d23f010 0x56151d24a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:29.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.084+0000 7f9c4718cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56151d23f930 0x56151d247350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:29.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.085+0000 7f9c4718cb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56151d24a7e0 0x56151d2478d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:29.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.085+0000 7f9c30c00700 1 --2- 172.21.15.1:0/582746872 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f9c0c0a0790 0x7f9c0c0a2c50 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c00ea30 tx=0x7f9c1c000bc0 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:41:29.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.085+0000 7f9c2be00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56151d23f930 0x56151d247350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:29.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.085+0000 7f9c2be00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56151d23f930 0x56151d247350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59452/0 (socket says 172.21.15.1:59452) 2024-02-02T15:41:29.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.085+0000 7f9c30c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56151d23f010 0x56151d24a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:29.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.085+0000 7f9c30c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56151d23f010 0x56151d24a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:35512/0 (socket says 172.21.15.1:35512) 2024-02-02T15:41:29.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.086+0000 7f9c31600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56151d24a7e0 0x56151d2478d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:29.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.086+0000 7f9c31600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56151d24a7e0 0x56151d2478d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33274/0 (socket says 172.21.15.1:33274) 2024-02-02T15:41:29.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.086+0000 7f9c4718cb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56151d190d50 con 0x56151d24a7e0 2024-02-02T15:41:29.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.086+0000 7f9c4718cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56151d12cd80 con 0x56151d23f930 2024-02-02T15:41:29.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.086+0000 7f9c4718cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56151d12ca20 con 0x56151d23f010 2024-02-02T15:41:29.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.086+0000 7f9c4718cb80 10 client.0 _ll_register_callbacks cb 0x56151d12e290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:41:29.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.086+0000 7f9c2be00700 1 -- 172.21.15.1:0/1988263975 learned_addr learned my addr 172.21.15.1:0/1988263975 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:41:29.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.087+0000 7f9c31600700 1 -- 172.21.15.1:0/1988263975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56151d23f010 msgr2=0x56151d24a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:41:29.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.087+0000 7f9c31600700 1 --2- 172.21.15.1:0/1988263975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56151d23f010 0x56151d24a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.087+0000 7f9c31600700 1 -- 172.21.15.1:0/1988263975 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56151d23f930 msgr2=0x56151d247350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:41:29.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.087+0000 7f9c31600700 1 --2- 172.21.15.1:0/1988263975 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56151d23f930 0x56151d247350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.087+0000 7f9c31600700 1 -- 172.21.15.1:0/1988263975 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c1c009980 con 0x56151d24a7e0 2024-02-02T15:41:29.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.087+0000 7f9c31600700 1 --2- 172.21.15.1:0/1988263975 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56151d24a7e0 0x56151d2478d0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f9c240caac0 tx=0x7f9c2409a330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c789963895a7b2ff server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:41:29.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.087+0000 7f9c1a000700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:41:29.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.088+0000 7f9c2be00700 1 --2- 172.21.15.1:0/1988263975 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56151d23f930 0x56151d247350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-02-02T15:41:29.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.088+0000 7f9c1a000700 1 -- 172.21.15.1:0/1988263975 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c2409b020 con 0x56151d24a7e0 2024-02-02T15:41:29.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.088+0000 7f9c1a000700 1 -- 172.21.15.1:0/1988263975 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f9c240094e0 con 0x56151d24a7e0 2024-02-02T15:41:29.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.088+0000 7f9c4718cb80 1 -- 172.21.15.1:0/1988263975 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56151d2330f0 con 0x56151d24a7e0 2024-02-02T15:41:29.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.088+0000 7f9c1a000700 1 -- 172.21.15.1:0/1988263975 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9c2409a920 con 0x56151d24a7e0 2024-02-02T15:41:29.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.088+0000 7f9c4718cb80 10 client.19293 fetch_fsmap learned FSMap version 61 2024-02-02T15:41:29.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.088+0000 7f9c4718cb80 1 -- 172.21.15.1:0/1988263975 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x56151cfbeb60 con 0x56151d24a7e0 2024-02-02T15:41:29.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.089+0000 7f9c1a000700 1 -- 172.21.15.1:0/1988263975 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56151d2330f0 con 0x56151d24a7e0 2024-02-02T15:41:29.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.090+0000 7f9c1a000700 1 -- 172.21.15.1:0/1988263975 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 61) v1 ==== 7550+0+0 (secure 0 0 0) 0x7f9c240cd030 con 0x56151d24a7e0 2024-02-02T15:41:29.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.090+0000 7f9c4718cb80 10 client.19293 fetch_fsmap finished waiting for FSMap version 61 2024-02-02T15:41:29.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.090+0000 7f9c4718cb80 10 client.19293 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-02-02T15:41:29.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.090+0000 7f9c4718cb80 20 client.19293 populate_metadata read hostname 'smithi001' 2024-02-02T15:41:29.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.090+0000 7f9c4718cb80 1 --2- 172.21.15.1:0/1988263975 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x56151d259df0 0x56151d25c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:29.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.090+0000 7f9c4718cb80 4 client.19293 mds_command: new command op to 24518 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:41:29.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.090+0000 7f9c4718cb80 1 -- 172.21.15.1:0/1988263975 --> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56151d189dd0 con 0x56151d259df0 2024-02-02T15:41:29.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.090+0000 7f9c30c00700 1 --2- 172.21.15.1:0/1988263975 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x56151d259df0 0x56151d25c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:29.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.091+0000 7f9c1a000700 10 client.19293 ms_handle_connect on v2:172.21.15.71:6836/628016199 2024-02-02T15:41:29.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.091+0000 7f9c30c00700 1 --2- 172.21.15.1:0/1988263975 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x56151d259df0 0x56151d25c210 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7f858f60b4fbe017 server_cookie=b7ddddee2eb5fc3c in_seq=0 out_seq=0 2024-02-02T15:41:29.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.092+0000 7f9c1a000700 1 -- 172.21.15.1:0/1988263975 <== mds.2 v2:172.21.15.71:6836/628016199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x56151d189dd0 con 0x56151d259df0 2024-02-02T15:41:29.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.092+0000 7f9c1a000700 10 client.19293 handle_command_reply: tid=0 2024-02-02T15:41:29.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.092+0000 7f9c4718cb80 1 client.19293 shutdown 2024-02-02T15:41:29.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.093+0000 7f9c4718cb80 1 -- 172.21.15.1:0/1988263975 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56151d24a7e0 msgr2=0x56151d2478d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:41:29.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.093+0000 7f9c4718cb80 1 --2- 172.21.15.1:0/1988263975 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56151d24a7e0 0x56151d2478d0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f9c240caac0 tx=0x7f9c2409a330 comp rx=0 tx=0).stop 2024-02-02T15:41:29.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.093+0000 7f9c4718cb80 1 -- 172.21.15.1:0/1988263975 shutdown_connections 2024-02-02T15:41:29.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.093+0000 7f9c4718cb80 1 --2- 172.21.15.1:0/1988263975 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56151d24a7e0 0x56151d2478d0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.093+0000 7f9c4718cb80 1 --2- 172.21.15.1:0/1988263975 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x56151d259df0 0x56151d25c210 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.093+0000 7f9c4718cb80 1 --2- 172.21.15.1:0/1988263975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56151d23f010 0x56151d24a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.093+0000 7f9c4718cb80 1 --2- 172.21.15.1:0/1988263975 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56151d23f930 0x56151d247350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.093+0000 7f9c4718cb80 1 -- 172.21.15.1:0/1988263975 >> 172.21.15.1:0/1988263975 conn(0x56151d19edb0 msgr2=0x56151d23afa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:41:29.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.093+0000 7f9c1a000700 0 client.19293 ms_handle_reset on v2:172.21.15.71:6836/628016199 2024-02-02T15:41:29.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.093+0000 7f9c4718cb80 1 -- 172.21.15.1:0/1988263975 shutdown_connections 2024-02-02T15:41:29.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.093+0000 7f9c4718cb80 1 -- 172.21.15.1:0/1988263975 wait complete. 2024-02-02T15:41:29.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.097+0000 7f9c4718cb80 20 client.19293 trim_cache size 0 max 16384 2024-02-02T15:41:29.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.121+0000 7f9c4718cb80 1 Processor -- start 2024-02-02T15:41:29.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.121+0000 7f9c4718cb80 1 -- start start 2024-02-02T15:41:29.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.122+0000 7f9c4718cb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56151d23f010 0x56151d251330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:29.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.122+0000 7f9c4718cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56151d23f930 0x56151d249c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:29.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.122+0000 7f9c4718cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56151d24a7e0 0x56151d244600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:29.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.122+0000 7f9c4718cb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56151d0146a0 con 0x56151d23f010 2024-02-02T15:41:29.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.122+0000 7f9c4718cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56151d256b60 con 0x56151d24a7e0 2024-02-02T15:41:29.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.122+0000 7f9c4718cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56151d247ed0 con 0x56151d23f930 2024-02-02T15:41:29.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.123+0000 7f9c31600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56151d24a7e0 0x56151d244600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:29.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.123+0000 7f9c31600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56151d24a7e0 0x56151d244600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59468/0 (socket says 172.21.15.1:59468) 2024-02-02T15:41:29.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.123+0000 7f9c31600700 1 -- 172.21.15.1:0/272717430 learned_addr learned my addr 172.21.15.1:0/272717430 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:41:29.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.123+0000 7f9c2be00700 1 --2- 172.21.15.1:0/272717430 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56151d23f930 0x56151d249c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:29.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.123+0000 7f9c31600700 1 -- 172.21.15.1:0/272717430 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56151d23f930 msgr2=0x56151d249c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:41:29.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.123+0000 7f9c31600700 1 --2- 172.21.15.1:0/272717430 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56151d23f930 0x56151d249c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.123+0000 7f9c31600700 1 -- 172.21.15.1:0/272717430 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56151d23f010 msgr2=0x56151d251330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:41:29.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.123+0000 7f9c31600700 1 --2- 172.21.15.1:0/272717430 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56151d23f010 0x56151d251330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.123+0000 7f9c31600700 1 -- 172.21.15.1:0/272717430 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56151d156ea0 con 0x56151d24a7e0 2024-02-02T15:41:29.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.124+0000 7f9c31600700 1 --2- 172.21.15.1:0/272717430 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56151d24a7e0 0x56151d244600 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f9c2401daf0 tx=0x7f9c2400bf40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f88f87634bc8de5c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:41:29.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.124+0000 7f9c22600700 1 -- 172.21.15.1:0/272717430 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c240b6020 con 0x56151d24a7e0 2024-02-02T15:41:29.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.125+0000 7f9c22600700 1 -- 172.21.15.1:0/272717430 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c2409dee0 con 0x56151d24a7e0 2024-02-02T15:41:29.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.125+0000 7f9c22600700 1 -- 172.21.15.1:0/272717430 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c240a2900 con 0x56151d24a7e0 2024-02-02T15:41:29.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.126+0000 7f9c4718cb80 1 -- 172.21.15.1:0/272717430 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56151d24a7e0 msgr2=0x56151d244600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:41:29.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.126+0000 7f9c4718cb80 1 --2- 172.21.15.1:0/272717430 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56151d24a7e0 0x56151d244600 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f9c2401daf0 tx=0x7f9c2400bf40 comp rx=0 tx=0).stop 2024-02-02T15:41:29.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.126+0000 7f9c4718cb80 1 -- 172.21.15.1:0/272717430 shutdown_connections 2024-02-02T15:41:29.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.126+0000 7f9c4718cb80 1 --2- 172.21.15.1:0/272717430 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56151d24a7e0 0x56151d244600 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.126+0000 7f9c4718cb80 1 --2- 172.21.15.1:0/272717430 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56151d23f010 0x56151d251330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.126+0000 7f9c4718cb80 1 --2- 172.21.15.1:0/272717430 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56151d23f930 0x56151d249c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.126+0000 7f9c4718cb80 1 -- 172.21.15.1:0/272717430 >> 172.21.15.1:0/272717430 conn(0x56151d19edb0 msgr2=0x56151d242670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:41:29.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.127+0000 7f9c4718cb80 1 -- 172.21.15.1:0/272717430 shutdown_connections 2024-02-02T15:41:29.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.127+0000 7f9c4718cb80 1 -- 172.21.15.1:0/272717430 wait complete. 2024-02-02T15:41:29.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.128+0000 7f9c4718cb80 1 -- start start 2024-02-02T15:41:29.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.128+0000 7f9c4718cb80 1 Processor -- start 2024-02-02T15:41:29.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.129+0000 7f9c4718cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56151d23f010 0x56151d29fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:29.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.129+0000 7f9c4718cb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56151d23f930 0x56151d29ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:29.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.129+0000 7f9c4718cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56151d24a7e0 0x56151d2a3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:29.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.129+0000 7f9c4718cb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56151d2a4120 con 0x56151d23f930 2024-02-02T15:41:29.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.129+0000 7f9c4718cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56151d2a4290 con 0x56151d24a7e0 2024-02-02T15:41:29.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.129+0000 7f9c4718cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56151d2a4450 con 0x56151d23f010 2024-02-02T15:41:29.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.129+0000 7f9c4718cb80 10 client.0 _ll_register_callbacks cb 0x56151d12e290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:41:29.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.130+0000 7f9c31600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56151d24a7e0 0x56151d2a3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:29.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.130+0000 7f9c31600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56151d24a7e0 0x56151d2a3b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59480/0 (socket says 172.21.15.1:59480) 2024-02-02T15:41:29.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.130+0000 7f9c31600700 1 -- 172.21.15.1:0/3785806498 learned_addr learned my addr 172.21.15.1:0/3785806498 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:41:29.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.131+0000 7f9c31600700 1 -- 172.21.15.1:0/3785806498 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56151d23f010 msgr2=0x56151d29fa90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-02T15:41:29.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.131+0000 7f9c31600700 1 --2- 172.21.15.1:0/3785806498 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56151d23f010 0x56151d29fa90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.131+0000 7f9c31600700 1 -- 172.21.15.1:0/3785806498 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56151d23f930 msgr2=0x56151d29ffd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-02T15:41:29.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.132+0000 7f9c2be00700 1 --2- 172.21.15.1:0/3785806498 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56151d23f930 0x56151d29ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:29.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.132+0000 7f9c31600700 1 --2- 172.21.15.1:0/3785806498 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56151d23f930 0x56151d29ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.132+0000 7f9c31600700 1 -- 172.21.15.1:0/3785806498 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56151cfbeb60 con 0x56151d24a7e0 2024-02-02T15:41:29.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.132+0000 7f9c2be00700 1 --2- 172.21.15.1:0/3785806498 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56151d23f930 0x56151d29ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:41:29.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.134+0000 7f9c1a000700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:41:29.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.134+0000 7f9c31600700 1 --2- 172.21.15.1:0/3785806498 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56151d24a7e0 0x56151d2a3b70 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f9c24096ec0 tx=0x7f9c2409a330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8853e936589a62fd server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:41:29.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.134+0000 7f9c1a000700 1 -- 172.21.15.1:0/3785806498 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c240b6070 con 0x56151d24a7e0 2024-02-02T15:41:29.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.135+0000 7f9c1a000700 1 -- 172.21.15.1:0/3785806498 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f9c2409b020 con 0x56151d24a7e0 2024-02-02T15:41:29.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.135+0000 7f9c4718cb80 1 -- 172.21.15.1:0/3785806498 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9c1c009e50 con 0x56151d24a7e0 2024-02-02T15:41:29.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.135+0000 7f9c1a000700 1 -- 172.21.15.1:0/3785806498 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9c240cd530 con 0x56151d24a7e0 2024-02-02T15:41:29.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.135+0000 7f9c4718cb80 10 client.29438 fetch_fsmap learned FSMap version 61 2024-02-02T15:41:29.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.135+0000 7f9c4718cb80 1 -- 172.21.15.1:0/3785806498 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x56151d016cd0 con 0x56151d24a7e0 2024-02-02T15:41:29.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.135+0000 7f9c1a000700 1 -- 172.21.15.1:0/3785806498 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c1c009e50 con 0x56151d24a7e0 2024-02-02T15:41:29.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.136+0000 7f9c1a000700 1 -- 172.21.15.1:0/3785806498 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 61) v1 ==== 7550+0+0 (secure 0 0 0) 0x7f9c24081070 con 0x56151d24a7e0 2024-02-02T15:41:29.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.136+0000 7f9c4718cb80 10 client.29438 fetch_fsmap finished waiting for FSMap version 61 2024-02-02T15:41:29.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.136+0000 7f9c4718cb80 10 client.29438 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-02-02T15:41:29.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.136+0000 7f9c4718cb80 20 client.29438 populate_metadata read hostname 'smithi001' 2024-02-02T15:41:29.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.136+0000 7f9c4718cb80 1 --2- 172.21.15.1:0/3785806498 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x56151d2ac8d0 0x56151d2aecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:29.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.136+0000 7f9c4718cb80 4 client.29438 mds_command: new command op to 24518 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:41:29.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.136+0000 7f9c30c00700 1 --2- 172.21.15.1:0/3785806498 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x56151d2ac8d0 0x56151d2aecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:29.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.137+0000 7f9c4718cb80 1 -- 172.21.15.1:0/3785806498 --> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56151d14ff50 con 0x56151d2ac8d0 2024-02-02T15:41:29.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.137+0000 7f9c30c00700 1 --2- 172.21.15.1:0/3785806498 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x56151d2ac8d0 0x56151d2aecc0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5f4eed24f02be40e server_cookie=8f77ce7dc74e8921 in_seq=0 out_seq=0 2024-02-02T15:41:29.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.137+0000 7f9c1a000700 10 client.29438 ms_handle_connect on v2:172.21.15.71:6836/628016199 2024-02-02T15:41:29.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.137+0000 7f9c1a000700 1 -- 172.21.15.1:0/3785806498 <== mds.2 v2:172.21.15.71:6836/628016199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56151d14ff50 con 0x56151d2ac8d0 2024-02-02T15:41:29.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.137+0000 7f9c1a000700 10 client.29438 handle_command_reply: tid=0 2024-02-02T15:41:29.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.137+0000 7f9c4718cb80 1 client.29438 shutdown 2024-02-02T15:41:29.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.139+0000 7f9c4718cb80 1 -- 172.21.15.1:0/3785806498 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56151d24a7e0 msgr2=0x56151d2a3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:41:29.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.139+0000 7f9c4718cb80 1 --2- 172.21.15.1:0/3785806498 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56151d24a7e0 0x56151d2a3b70 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f9c24096ec0 tx=0x7f9c2409a330 comp rx=0 tx=0).stop 2024-02-02T15:41:29.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.140+0000 7f9c4718cb80 1 -- 172.21.15.1:0/3785806498 shutdown_connections 2024-02-02T15:41:29.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.140+0000 7f9c4718cb80 1 --2- 172.21.15.1:0/3785806498 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56151d24a7e0 0x56151d2a3b70 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.140+0000 7f9c4718cb80 1 --2- 172.21.15.1:0/3785806498 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x56151d2ac8d0 0x56151d2aecc0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.140+0000 7f9c4718cb80 1 --2- 172.21.15.1:0/3785806498 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56151d23f010 0x56151d29fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.140+0000 7f9c4718cb80 1 --2- 172.21.15.1:0/3785806498 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56151d23f930 0x56151d29ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.140+0000 7f9c4718cb80 1 -- 172.21.15.1:0/3785806498 >> 172.21.15.1:0/3785806498 conn(0x56151d19edb0 msgr2=0x56151d240f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:41:29.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.140+0000 7f9c1a000700 0 client.29438 ms_handle_reset on v2:172.21.15.71:6836/628016199 2024-02-02T15:41:29.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.140+0000 7f9c4718cb80 1 -- 172.21.15.1:0/3785806498 shutdown_connections 2024-02-02T15:41:29.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.140+0000 7f9c4718cb80 1 -- 172.21.15.1:0/3785806498 wait complete. 2024-02-02T15:41:29.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.141+0000 7f9c4718cb80 20 client.29438 trim_cache size 0 max 16384 2024-02-02T15:41:29.141 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:41:29.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.142+0000 7f9c33600700 1 -- 172.21.15.1:0/582746872 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f9c0c0a0790 msgr2=0x7f9c0c0a2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:41:29.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.142+0000 7f9c33600700 1 --2- 172.21.15.1:0/582746872 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f9c0c0a0790 0x7f9c0c0a2c50 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c00ea30 tx=0x7f9c1c000bc0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.142+0000 7f9c33600700 1 -- 172.21.15.1:0/582746872 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9c2c10a0b0 msgr2=0x7f9c2c10a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:41:29.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.142+0000 7f9c33600700 1 --2- 172.21.15.1:0/582746872 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9c2c10a0b0 0x7f9c2c10a520 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f9c24009f90 tx=0x7f9c240075b0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.142+0000 7f9c33600700 1 -- 172.21.15.1:0/582746872 shutdown_connections 2024-02-02T15:41:29.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.142+0000 7f9c33600700 1 --2- 172.21.15.1:0/582746872 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f9c0c0a0790 0x7f9c0c0a2c50 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.142+0000 7f9c33600700 1 --2- 172.21.15.1:0/582746872 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9c2c10a0b0 0x7f9c2c10a520 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.142+0000 7f9c33600700 1 --2- 172.21.15.1:0/582746872 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9c2c101000 0x7f9c2c10db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.142+0000 7f9c33600700 1 --2- 172.21.15.1:0/582746872 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9c2c106120 0x7f9c2c109b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.142+0000 7f9c33600700 1 -- 172.21.15.1:0/582746872 >> 172.21.15.1:0/582746872 conn(0x7f9c2c0fc860 msgr2=0x7f9c2c0fde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:41:29.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.142+0000 7f9c33600700 1 -- 172.21.15.1:0/582746872 shutdown_connections 2024-02-02T15:41:29.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.142+0000 7f9c33600700 1 -- 172.21.15.1:0/582746872 wait complete. 2024-02-02T15:41:29.163 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-02T15:41:29.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.429+0000 7f98dbc00700 1 -- 172.21.15.1:0/1013240749 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98d40ffbb0 msgr2=0x7f98d40fff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:41:29.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.429+0000 7f98dbc00700 1 --2- 172.21.15.1:0/1013240749 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98d40ffbb0 0x7f98d40fff90 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f98c4009a10 tx=0x7f98c4009d20 comp rx=0 tx=0).stop 2024-02-02T15:41:29.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.429+0000 7f98dbc00700 1 -- 172.21.15.1:0/1013240749 shutdown_connections 2024-02-02T15:41:29.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.429+0000 7f98dbc00700 1 --2- 172.21.15.1:0/1013240749 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f98d410d760 0x7f98d410fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.430+0000 7f98dbc00700 1 --2- 172.21.15.1:0/1013240749 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98d40ffbb0 0x7f98d40fff90 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.430+0000 7f98dbc00700 1 --2- 172.21.15.1:0/1013240749 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98d41004d0 0x7f98d410d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.430+0000 7f98dbc00700 1 -- 172.21.15.1:0/1013240749 >> 172.21.15.1:0/1013240749 conn(0x7f98d4076d70 msgr2=0x7f98d4077180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:41:29.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.430+0000 7f98dbc00700 1 -- 172.21.15.1:0/1013240749 shutdown_connections 2024-02-02T15:41:29.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.430+0000 7f98dbc00700 1 -- 172.21.15.1:0/1013240749 wait complete. 2024-02-02T15:41:29.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.431+0000 7f98dbc00700 1 Processor -- start 2024-02-02T15:41:29.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.431+0000 7f98dbc00700 1 -- start start 2024-02-02T15:41:29.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.432+0000 7f98dbc00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98d40ffbb0 0x7f98d4108a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:29.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.432+0000 7f98dbc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98d41004d0 0x7f98d4108f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:29.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.432+0000 7f98dbc00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f98d410d760 0x7f98d419c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:29.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.432+0000 7f98dbc00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98d419cc00 con 0x7f98d410d760 2024-02-02T15:41:29.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.432+0000 7f98dbc00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98d419cd40 con 0x7f98d40ffbb0 2024-02-02T15:41:29.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.432+0000 7f98dbc00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98d419ceb0 con 0x7f98d41004d0 2024-02-02T15:41:29.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.432+0000 7f98d9200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98d40ffbb0 0x7f98d4108a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:29.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.432+0000 7f98d9200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98d40ffbb0 0x7f98d4108a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59492/0 (socket says 172.21.15.1:59492) 2024-02-02T15:41:29.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.432+0000 7f98d9200700 1 -- 172.21.15.1:0/2261791143 learned_addr learned my addr 172.21.15.1:0/2261791143 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:41:29.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.432+0000 7f98d9c00700 1 --2- 172.21.15.1:0/2261791143 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f98d410d760 0x7f98d419c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:29.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.432+0000 7f98d9200700 1 -- 172.21.15.1:0/2261791143 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98d41004d0 msgr2=0x7f98d4108f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:41:29.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.432+0000 7f98d9200700 1 --2- 172.21.15.1:0/2261791143 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98d41004d0 0x7f98d4108f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.433+0000 7f98d9200700 1 -- 172.21.15.1:0/2261791143 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f98d410d760 msgr2=0x7f98d419c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:41:29.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.433+0000 7f98d9200700 1 --2- 172.21.15.1:0/2261791143 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f98d410d760 0x7f98d419c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.433+0000 7f98d9200700 1 -- 172.21.15.1:0/2261791143 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98cc009c40 con 0x7f98d40ffbb0 2024-02-02T15:41:29.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.433+0000 7f98d9200700 1 --2- 172.21.15.1:0/2261791143 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98d40ffbb0 0x7f98d4108a10 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f98c4009960 tx=0x7f98c400bf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:41:29.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.433+0000 7f98d1600700 1 -- 172.21.15.1:0/2261791143 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c402f070 con 0x7f98d40ffbb0 2024-02-02T15:41:29.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.433+0000 7f98dbc00700 1 -- 172.21.15.1:0/2261791143 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98c40095f0 con 0x7f98d40ffbb0 2024-02-02T15:41:29.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.433+0000 7f98d1600700 1 -- 172.21.15.1:0/2261791143 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f98c4005da0 con 0x7f98d40ffbb0 2024-02-02T15:41:29.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.433+0000 7f98d1600700 1 -- 172.21.15.1:0/2261791143 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c4033b30 con 0x7f98d40ffbb0 2024-02-02T15:41:29.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.434+0000 7f98dbc00700 1 -- 172.21.15.1:0/2261791143 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98d419d3c0 con 0x7f98d40ffbb0 2024-02-02T15:41:29.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.437+0000 7f98d1600700 1 -- 172.21.15.1:0/2261791143 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f98c4003ab0 con 0x7f98d40ffbb0 2024-02-02T15:41:29.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.438+0000 7f98ef783b80 1 Processor -- start 2024-02-02T15:41:29.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.438+0000 7f98d1600700 1 --2- 172.21.15.1:0/2261791143 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f98b40a0690 0x7f98b40a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:29.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.439+0000 7f98d1600700 1 -- 172.21.15.1:0/2261791143 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f98c40ddc00 con 0x7f98d40ffbb0 2024-02-02T15:41:29.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.439+0000 7f98ef783b80 1 -- start start 2024-02-02T15:41:29.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.439+0000 7f98d3e00700 1 --2- 172.21.15.1:0/2261791143 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f98b40a0690 0x7f98b40a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:29.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.439+0000 7f98ef783b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556fa604b010 0x556fa604b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:29.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.439+0000 7f98ef783b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556fa604b930 0x556fa60562a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:29.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.439+0000 7f98ef783b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556fa60567e0 0x556fa6058bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:29.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.439+0000 7f98ef783b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fa5fbf4c0 con 0x556fa604b930 2024-02-02T15:41:29.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.439+0000 7f98ef783b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fa5fa3aa0 con 0x556fa60567e0 2024-02-02T15:41:29.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.439+0000 7f98ef783b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fa5fa1460 con 0x556fa604b010 2024-02-02T15:41:29.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.439+0000 7f98d9200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556fa604b010 0x556fa604b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:29.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.439+0000 7f98d9c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556fa60567e0 0x556fa6058bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:29.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.439+0000 7f98d9200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556fa604b010 0x556fa604b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:35560/0 (socket says 172.21.15.1:35560) 2024-02-02T15:41:29.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.439+0000 7f98d9200700 1 -- 172.21.15.1:0/4056500650 learned_addr learned my addr 172.21.15.1:0/4056500650 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:41:29.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.440+0000 7f98d9200700 1 -- 172.21.15.1:0/4056500650 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556fa60567e0 msgr2=0x556fa6058bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:41:29.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.440+0000 7f98d9200700 1 --2- 172.21.15.1:0/4056500650 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556fa60567e0 0x556fa6058bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.440+0000 7f98d9200700 1 -- 172.21.15.1:0/4056500650 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556fa604b930 msgr2=0x556fa60562a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:41:29.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.440+0000 7f98d9200700 1 --2- 172.21.15.1:0/4056500650 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556fa604b930 0x556fa60562a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.440+0000 7f98d9200700 1 -- 172.21.15.1:0/4056500650 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556fa5e1e3b0 con 0x556fa604b010 2024-02-02T15:41:29.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.440+0000 7f98d9c00700 1 --2- 172.21.15.1:0/4056500650 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556fa60567e0 0x556fa6058bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:41:29.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.440+0000 7f98d3e00700 1 --2- 172.21.15.1:0/2261791143 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f98b40a0690 0x7f98b40a2b50 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f98bc005950 tx=0x7f98bc0058e0 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:41:29.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.440+0000 7f98d9200700 1 --2- 172.21.15.1:0/4056500650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556fa604b010 0x556fa604b3f0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f98c40aab70 tx=0x7f98c4005f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66cba546ec72cd36 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:41:29.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.441+0000 7f98ca000700 1 -- 172.21.15.1:0/4056500650 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c40ab570 con 0x556fa604b010 2024-02-02T15:41:29.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.441+0000 7f98ca000700 1 -- 172.21.15.1:0/4056500650 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f98c40aae70 con 0x556fa604b010 2024-02-02T15:41:29.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.441+0000 7f98ca000700 1 -- 172.21.15.1:0/4056500650 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c40ab0d0 con 0x556fa604b010 2024-02-02T15:41:29.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.441+0000 7f98ef783b80 1 -- 172.21.15.1:0/4056500650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556fa604b010 msgr2=0x556fa604b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:41:29.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.441+0000 7f98ef783b80 1 --2- 172.21.15.1:0/4056500650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556fa604b010 0x556fa604b3f0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f98c40aab70 tx=0x7f98c4005f10 comp rx=0 tx=0).stop 2024-02-02T15:41:29.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.442+0000 7f98ef783b80 1 -- 172.21.15.1:0/4056500650 shutdown_connections 2024-02-02T15:41:29.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.442+0000 7f98ef783b80 1 --2- 172.21.15.1:0/4056500650 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556fa60567e0 0x556fa6058bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.442+0000 7f98ef783b80 1 --2- 172.21.15.1:0/4056500650 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556fa604b010 0x556fa604b3f0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.442+0000 7f98ef783b80 1 --2- 172.21.15.1:0/4056500650 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556fa604b930 0x556fa60562a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.442+0000 7f98ef783b80 1 -- 172.21.15.1:0/4056500650 >> 172.21.15.1:0/4056500650 conn(0x556fa5faadb0 msgr2=0x556fa60473b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:41:29.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.442+0000 7f98ef783b80 1 -- 172.21.15.1:0/4056500650 shutdown_connections 2024-02-02T15:41:29.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.442+0000 7f98ef783b80 1 -- 172.21.15.1:0/4056500650 wait complete. 2024-02-02T15:41:29.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.442+0000 7f98ef783b80 1 -- start start 2024-02-02T15:41:29.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.443+0000 7f98ef783b80 1 Processor -- start 2024-02-02T15:41:29.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.444+0000 7f98ef783b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556fa604b010 0x556fa6056bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:29.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.444+0000 7f98ef783b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556fa604b930 0x556fa6057100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:29.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.444+0000 7f98ef783b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556fa60567e0 0x556fa6057640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:29.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.444+0000 7f98ef783b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fa5f9cd50 con 0x556fa604b010 2024-02-02T15:41:29.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.444+0000 7f98ef783b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fa5f38d80 con 0x556fa60567e0 2024-02-02T15:41:29.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.444+0000 7f98ef783b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fa5f38a20 con 0x556fa604b930 2024-02-02T15:41:29.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.444+0000 7f98ef783b80 10 client.0 _ll_register_callbacks cb 0x556fa5f3a290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:41:29.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.445+0000 7f98d9c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556fa60567e0 0x556fa6057640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:29.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.445+0000 7f98d9c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556fa60567e0 0x556fa6057640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59512/0 (socket says 172.21.15.1:59512) 2024-02-02T15:41:29.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.445+0000 7f98d9c00700 1 -- 172.21.15.1:0/1867581068 learned_addr learned my addr 172.21.15.1:0/1867581068 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:41:29.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.445+0000 7f98d9200700 1 --2- 172.21.15.1:0/1867581068 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556fa604b010 0x556fa6056bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:29.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.445+0000 7f98d3e00700 1 --2- 172.21.15.1:0/1867581068 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556fa604b930 0x556fa6057100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:29.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.446+0000 7f98d9c00700 1 -- 172.21.15.1:0/1867581068 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556fa604b930 msgr2=0x556fa6057100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:41:29.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.446+0000 7f98d9c00700 1 --2- 172.21.15.1:0/1867581068 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556fa604b930 0x556fa6057100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.446+0000 7f98d9c00700 1 -- 172.21.15.1:0/1867581068 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556fa604b010 msgr2=0x556fa6056bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:41:29.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.446+0000 7f98d9c00700 1 --2- 172.21.15.1:0/1867581068 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556fa604b010 0x556fa6056bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.446+0000 7f98d9c00700 1 -- 172.21.15.1:0/1867581068 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98cc0091b0 con 0x556fa60567e0 2024-02-02T15:41:29.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.446+0000 7f98d9200700 1 --2- 172.21.15.1:0/1867581068 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556fa604b010 0x556fa6056bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:41:29.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.446+0000 7f98d9c00700 1 --2- 172.21.15.1:0/1867581068 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556fa60567e0 0x556fa6057640 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f98cc007f70 tx=0x7f98cc009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3ef996a5ec7422b server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:41:29.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.446+0000 7f98c2a00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:41:29.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.447+0000 7f98c2a00700 1 -- 172.21.15.1:0/1867581068 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98cc00e050 con 0x556fa60567e0 2024-02-02T15:41:29.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.447+0000 7f98c2a00700 1 -- 172.21.15.1:0/1867581068 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f98cc013d60 con 0x556fa60567e0 2024-02-02T15:41:29.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.447+0000 7f98ef783b80 1 -- 172.21.15.1:0/1867581068 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556fa603f0f0 con 0x556fa60567e0 2024-02-02T15:41:29.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.447+0000 7f98c2a00700 1 -- 172.21.15.1:0/1867581068 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f98cc003a00 con 0x556fa60567e0 2024-02-02T15:41:29.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.447+0000 7f98ef783b80 10 client.29450 fetch_fsmap learned FSMap version 61 2024-02-02T15:41:29.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.447+0000 7f98ef783b80 1 -- 172.21.15.1:0/1867581068 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x7f98d419d3c0 con 0x556fa60567e0 2024-02-02T15:41:29.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.447+0000 7f98c2a00700 1 -- 172.21.15.1:0/1867581068 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556fa603f0f0 con 0x556fa60567e0 2024-02-02T15:41:29.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.448+0000 7f98c2a00700 1 -- 172.21.15.1:0/1867581068 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 61) v1 ==== 7550+0+0 (secure 0 0 0) 0x7f98cc016430 con 0x556fa60567e0 2024-02-02T15:41:29.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.448+0000 7f98ef783b80 10 client.29450 fetch_fsmap finished waiting for FSMap version 61 2024-02-02T15:41:29.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.448+0000 7f98ef783b80 10 client.29450 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-02T15:41:29.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.448+0000 7f98ef783b80 20 client.29450 populate_metadata read hostname 'smithi001' 2024-02-02T15:41:29.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.448+0000 7f98ef783b80 1 --2- 172.21.15.1:0/1867581068 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x556fa6065f60 0x556fa6068380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:29.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.448+0000 7f98ef783b80 4 client.29450 mds_command: new command op to 24529 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:41:29.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.448+0000 7f98ef783b80 1 -- 172.21.15.1:0/1867581068 --> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556fa5f95dd0 con 0x556fa6065f60 2024-02-02T15:41:29.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.448+0000 7f98d9200700 1 --2- 172.21.15.1:0/1867581068 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x556fa6065f60 0x556fa6068380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:29.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.449+0000 7f98d9200700 1 --2- 172.21.15.1:0/1867581068 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x556fa6065f60 0x556fa6068380 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a66716f25a8d2fd4 server_cookie=49fbc50a389c6560 in_seq=0 out_seq=0 2024-02-02T15:41:29.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.449+0000 7f98c2a00700 10 client.29450 ms_handle_connect on v2:172.21.15.142:6838/321675421 2024-02-02T15:41:29.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.451+0000 7f98c2a00700 1 -- 172.21.15.1:0/1867581068 <== mds.1 v2:172.21.15.142:6838/321675421 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x556fa5f95dd0 con 0x556fa6065f60 2024-02-02T15:41:29.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.451+0000 7f98c2a00700 10 client.29450 handle_command_reply: tid=0 2024-02-02T15:41:29.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.451+0000 7f98ef783b80 1 client.29450 shutdown 2024-02-02T15:41:29.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.451+0000 7f98ef783b80 1 -- 172.21.15.1:0/1867581068 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556fa60567e0 msgr2=0x556fa6057640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:41:29.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.451+0000 7f98ef783b80 1 --2- 172.21.15.1:0/1867581068 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556fa60567e0 0x556fa6057640 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f98cc007f70 tx=0x7f98cc009500 comp rx=0 tx=0).stop 2024-02-02T15:41:29.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.452+0000 7f98ef783b80 1 -- 172.21.15.1:0/1867581068 shutdown_connections 2024-02-02T15:41:29.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.452+0000 7f98ef783b80 1 --2- 172.21.15.1:0/1867581068 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x556fa6065f60 0x556fa6068380 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.452+0000 7f98ef783b80 1 --2- 172.21.15.1:0/1867581068 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556fa60567e0 0x556fa6057640 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.452+0000 7f98c2a00700 0 client.29450 ms_handle_reset on v2:172.21.15.142:6838/321675421 2024-02-02T15:41:29.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.452+0000 7f98ef783b80 1 --2- 172.21.15.1:0/1867581068 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556fa604b010 0x556fa6056bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.452+0000 7f98ef783b80 1 --2- 172.21.15.1:0/1867581068 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556fa604b930 0x556fa6057100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.452+0000 7f98ef783b80 1 -- 172.21.15.1:0/1867581068 >> 172.21.15.1:0/1867581068 conn(0x556fa5faadb0 msgr2=0x556fa6046fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:41:29.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.452+0000 7f98ef783b80 1 -- 172.21.15.1:0/1867581068 shutdown_connections 2024-02-02T15:41:29.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.452+0000 7f98ef783b80 1 -- 172.21.15.1:0/1867581068 wait complete. 2024-02-02T15:41:29.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.452+0000 7f98ef783b80 20 client.29450 trim_cache size 0 max 16384 2024-02-02T15:41:29.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.472+0000 7f98ef783b80 1 Processor -- start 2024-02-02T15:41:29.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.473+0000 7f98ef783b80 1 -- start start 2024-02-02T15:41:29.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.473+0000 7f98ef783b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556fa604b010 0x556fa605abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:29.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.473+0000 7f98ef783b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556fa604b930 0x556fa604fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:29.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.473+0000 7f98ef783b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556fa60567e0 0x556fa6050240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:29.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.473+0000 7f98ef783b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fa5e206a0 con 0x556fa604b930 2024-02-02T15:41:29.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.473+0000 7f98ef783b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fa60489d0 con 0x556fa604b010 2024-02-02T15:41:29.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.473+0000 7f98ef783b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fa6041500 con 0x556fa60567e0 2024-02-02T15:41:29.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.473+0000 7f98d9200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556fa604b010 0x556fa605abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:29.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.473+0000 7f98d9200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556fa604b010 0x556fa605abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59516/0 (socket says 172.21.15.1:59516) 2024-02-02T15:41:29.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.473+0000 7f98d9200700 1 -- 172.21.15.1:0/119037259 learned_addr learned my addr 172.21.15.1:0/119037259 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:41:29.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.473+0000 7f98d9c00700 1 --2- 172.21.15.1:0/119037259 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556fa60567e0 0x556fa6050240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:29.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.474+0000 7f98d3e00700 1 --2- 172.21.15.1:0/119037259 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556fa604b930 0x556fa604fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:29.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.474+0000 7f98d3e00700 1 -- 172.21.15.1:0/119037259 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556fa60567e0 msgr2=0x556fa6050240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:41:29.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.474+0000 7f98d3e00700 1 --2- 172.21.15.1:0/119037259 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556fa60567e0 0x556fa6050240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.474+0000 7f98d3e00700 1 -- 172.21.15.1:0/119037259 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556fa604b010 msgr2=0x556fa605abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:41:29.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.474+0000 7f98d3e00700 1 --2- 172.21.15.1:0/119037259 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556fa604b010 0x556fa605abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.474+0000 7f98d3e00700 1 -- 172.21.15.1:0/119037259 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556fa5dcab60 con 0x556fa604b930 2024-02-02T15:41:29.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.475+0000 7f98d3e00700 1 --2- 172.21.15.1:0/119037259 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556fa604b930 0x556fa604fd00 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x556fa6055e90 tx=0x7f98bc009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a7904a1745d9d1 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:41:29.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.475+0000 7f98caa00700 1 -- 172.21.15.1:0/119037259 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98bc010890 con 0x556fa604b930 2024-02-02T15:41:29.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.475+0000 7f98caa00700 1 -- 172.21.15.1:0/119037259 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f98bc004830 con 0x556fa604b930 2024-02-02T15:41:29.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.475+0000 7f98caa00700 1 -- 172.21.15.1:0/119037259 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98bc005280 con 0x556fa604b930 2024-02-02T15:41:29.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.477+0000 7f98ef783b80 1 -- 172.21.15.1:0/119037259 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556fa604b930 msgr2=0x556fa604fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:41:29.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.477+0000 7f98ef783b80 1 --2- 172.21.15.1:0/119037259 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556fa604b930 0x556fa604fd00 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x556fa6055e90 tx=0x7f98bc009260 comp rx=0 tx=0).stop 2024-02-02T15:41:29.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.477+0000 7f98ef783b80 1 -- 172.21.15.1:0/119037259 shutdown_connections 2024-02-02T15:41:29.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.477+0000 7f98ef783b80 1 --2- 172.21.15.1:0/119037259 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556fa60567e0 0x556fa6050240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.477+0000 7f98ef783b80 1 --2- 172.21.15.1:0/119037259 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556fa604b010 0x556fa605abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.477+0000 7f98ef783b80 1 --2- 172.21.15.1:0/119037259 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556fa604b930 0x556fa604fd00 secure :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0x556fa6055e90 tx=0x7f98bc009260 comp rx=0 tx=0).stop 2024-02-02T15:41:29.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.477+0000 7f98ef783b80 1 -- 172.21.15.1:0/119037259 >> 172.21.15.1:0/119037259 conn(0x556fa5faadb0 msgr2=0x556fa6062190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:41:29.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.477+0000 7f98ef783b80 1 -- 172.21.15.1:0/119037259 shutdown_connections 2024-02-02T15:41:29.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.477+0000 7f98ef783b80 1 -- 172.21.15.1:0/119037259 wait complete. 2024-02-02T15:41:29.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.478+0000 7f98ef783b80 1 -- start start 2024-02-02T15:41:29.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.478+0000 7f98ef783b80 1 Processor -- start 2024-02-02T15:41:29.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.479+0000 7f98ef783b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556fa604b010 0x556fa60b9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:29.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.479+0000 7f98ef783b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556fa60567e0 0x556fa60b9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:29.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.479+0000 7f98ef783b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556fa60b41e0 0x556fa60b4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:29.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.479+0000 7f98ef783b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fa60b4bf0 con 0x556fa60567e0 2024-02-02T15:41:29.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.479+0000 7f98ef783b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fa60b4d60 con 0x556fa60b41e0 2024-02-02T15:41:29.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.479+0000 7f98ef783b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556fa60b4f20 con 0x556fa604b010 2024-02-02T15:41:29.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.479+0000 7f98ef783b80 10 client.0 _ll_register_callbacks cb 0x556fa5f3a290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:41:29.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.479+0000 7f98d9200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556fa604b010 0x556fa60b9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:29.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.479+0000 7f98d3e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556fa60567e0 0x556fa60b9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:29.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.479+0000 7f98d9c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556fa60b41e0 0x556fa60b4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:29.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.479+0000 7f98d9200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556fa604b010 0x556fa60b9600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:35590/0 (socket says 172.21.15.1:35590) 2024-02-02T15:41:29.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.479+0000 7f98d9c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556fa60b41e0 0x556fa60b4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59518/0 (socket says 172.21.15.1:59518) 2024-02-02T15:41:29.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.479+0000 7f98d9200700 1 -- 172.21.15.1:0/1849915670 learned_addr learned my addr 172.21.15.1:0/1849915670 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:41:29.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.479+0000 7f98d3e00700 1 -- 172.21.15.1:0/1849915670 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556fa604b010 msgr2=0x556fa60b9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:41:29.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.479+0000 7f98d3e00700 1 --2- 172.21.15.1:0/1849915670 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556fa604b010 0x556fa60b9600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.479+0000 7f98d3e00700 1 -- 172.21.15.1:0/1849915670 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556fa60b41e0 msgr2=0x556fa60b4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:41:29.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.479+0000 7f98d3e00700 1 --2- 172.21.15.1:0/1849915670 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556fa60b41e0 0x556fa60b4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.479+0000 7f98d3e00700 1 -- 172.21.15.1:0/1849915670 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98bc00e8f0 con 0x556fa60567e0 2024-02-02T15:41:29.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.479+0000 7f98d9200700 1 --2- 172.21.15.1:0/1849915670 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556fa604b010 0x556fa60b9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:41:29.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.479+0000 7f98d9c00700 1 --2- 172.21.15.1:0/1849915670 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556fa60b41e0 0x556fa60b4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:41:29.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.480+0000 7f98d3e00700 1 --2- 172.21.15.1:0/1849915670 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556fa60567e0 0x556fa60b9b40 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x556fa60628c0 tx=0x7f98bc010ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c8c1290edfef671 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:41:29.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.480+0000 7f98c2a00700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:41:29.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.480+0000 7f98c2a00700 1 -- 172.21.15.1:0/1849915670 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98bc029040 con 0x556fa60567e0 2024-02-02T15:41:29.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.480+0000 7f98ef783b80 1 -- 172.21.15.1:0/1849915670 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f98c40aae70 con 0x556fa60567e0 2024-02-02T15:41:29.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.480+0000 7f98c2a00700 1 -- 172.21.15.1:0/1849915670 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f98bc005000 con 0x556fa60567e0 2024-02-02T15:41:29.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.480+0000 7f98c2a00700 1 -- 172.21.15.1:0/1849915670 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f98bc0051c0 con 0x556fa60567e0 2024-02-02T15:41:29.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.480+0000 7f98ef783b80 10 client.19308 fetch_fsmap learned FSMap version 61 2024-02-02T15:41:29.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.480+0000 7f98ef783b80 1 -- 172.21.15.1:0/1849915670 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x556fa5f62ea0 con 0x556fa60567e0 2024-02-02T15:41:29.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.481+0000 7f98c2a00700 1 -- 172.21.15.1:0/1849915670 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f98c40aae70 con 0x556fa60567e0 2024-02-02T15:41:29.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.481+0000 7f98c2a00700 1 -- 172.21.15.1:0/1849915670 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 61) v1 ==== 7550+0+0 (secure 0 0 0) 0x7f98bc018030 con 0x556fa60567e0 2024-02-02T15:41:29.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.481+0000 7f98ef783b80 10 client.19308 fetch_fsmap finished waiting for FSMap version 61 2024-02-02T15:41:29.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.481+0000 7f98ef783b80 10 client.19308 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-02T15:41:29.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.481+0000 7f98ef783b80 20 client.19308 populate_metadata read hostname 'smithi001' 2024-02-02T15:41:29.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.481+0000 7f98ef783b80 1 --2- 172.21.15.1:0/1849915670 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x556fa60c1690 0x556fa60c3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:29.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.481+0000 7f98ef783b80 4 client.19308 mds_command: new command op to 24529 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:41:29.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.481+0000 7f98ef783b80 1 -- 172.21.15.1:0/1849915670 --> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x556fa5f5bf50 con 0x556fa60c1690 2024-02-02T15:41:29.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.481+0000 7f98d9200700 1 --2- 172.21.15.1:0/1849915670 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x556fa60c1690 0x556fa60c3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:29.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.482+0000 7f98d9200700 1 --2- 172.21.15.1:0/1849915670 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x556fa60c1690 0x556fa60c3ab0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=204ae275a7883ae3 server_cookie=f24914c96ec6e968 in_seq=0 out_seq=0 2024-02-02T15:41:29.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.482+0000 7f98c2a00700 10 client.19308 ms_handle_connect on v2:172.21.15.142:6838/321675421 2024-02-02T15:41:29.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.482+0000 7f98c2a00700 1 -- 172.21.15.1:0/1849915670 <== mds.1 v2:172.21.15.142:6838/321675421 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x556fa5f5bf50 con 0x556fa60c1690 2024-02-02T15:41:29.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.482+0000 7f98c2a00700 10 client.19308 handle_command_reply: tid=0 2024-02-02T15:41:29.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.482+0000 7f98ef783b80 1 client.19308 shutdown 2024-02-02T15:41:29.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.482+0000 7f98ef783b80 1 -- 172.21.15.1:0/1849915670 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556fa60567e0 msgr2=0x556fa60b9b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:41:29.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.482+0000 7f98ef783b80 1 --2- 172.21.15.1:0/1849915670 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556fa60567e0 0x556fa60b9b40 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x556fa60628c0 tx=0x7f98bc010ac0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.482+0000 7f98ef783b80 1 -- 172.21.15.1:0/1849915670 shutdown_connections 2024-02-02T15:41:29.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.482+0000 7f98ef783b80 1 --2- 172.21.15.1:0/1849915670 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x556fa60c1690 0x556fa60c3ab0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.482+0000 7f98ef783b80 1 --2- 172.21.15.1:0/1849915670 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556fa60b41e0 0x556fa60b4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.482+0000 7f98ef783b80 1 --2- 172.21.15.1:0/1849915670 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556fa604b010 0x556fa60b9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.482+0000 7f98ef783b80 1 --2- 172.21.15.1:0/1849915670 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556fa60567e0 0x556fa60b9b40 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.482+0000 7f98c2a00700 0 client.19308 ms_handle_reset on v2:172.21.15.142:6838/321675421 2024-02-02T15:41:29.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.482+0000 7f98ef783b80 1 -- 172.21.15.1:0/1849915670 >> 172.21.15.1:0/1849915670 conn(0x556fa5faadb0 msgr2=0x556fa6054060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:41:29.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.483+0000 7f98ef783b80 1 -- 172.21.15.1:0/1849915670 shutdown_connections 2024-02-02T15:41:29.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.483+0000 7f98ef783b80 1 -- 172.21.15.1:0/1849915670 wait complete. 2024-02-02T15:41:29.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.483+0000 7f98ef783b80 20 client.19308 trim_cache size 0 max 16384 2024-02-02T15:41:29.490 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:41:29.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.483+0000 7f98dbc00700 1 -- 172.21.15.1:0/2261791143 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f98b40a0690 msgr2=0x7f98b40a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:41:29.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.483+0000 7f98dbc00700 1 --2- 172.21.15.1:0/2261791143 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f98b40a0690 0x7f98b40a2b50 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f98bc005950 tx=0x7f98bc0058e0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.483+0000 7f98dbc00700 1 -- 172.21.15.1:0/2261791143 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98d40ffbb0 msgr2=0x7f98d4108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:41:29.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.483+0000 7f98dbc00700 1 --2- 172.21.15.1:0/2261791143 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98d40ffbb0 0x7f98d4108a10 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f98c4009960 tx=0x7f98c400bf20 comp rx=0 tx=0).stop 2024-02-02T15:41:29.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.484+0000 7f98dbc00700 1 -- 172.21.15.1:0/2261791143 shutdown_connections 2024-02-02T15:41:29.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.484+0000 7f98dbc00700 1 --2- 172.21.15.1:0/2261791143 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f98b40a0690 0x7f98b40a2b50 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.484+0000 7f98dbc00700 1 --2- 172.21.15.1:0/2261791143 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f98d410d760 0x7f98d419c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.484+0000 7f98dbc00700 1 --2- 172.21.15.1:0/2261791143 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f98d40ffbb0 0x7f98d4108a10 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.484+0000 7f98dbc00700 1 --2- 172.21.15.1:0/2261791143 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f98d41004d0 0x7f98d4108f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.484+0000 7f98dbc00700 1 -- 172.21.15.1:0/2261791143 >> 172.21.15.1:0/2261791143 conn(0x7f98d4076d70 msgr2=0x7f98d410ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:41:29.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.484+0000 7f98dbc00700 1 -- 172.21.15.1:0/2261791143 shutdown_connections 2024-02-02T15:41:29.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.484+0000 7f98dbc00700 1 -- 172.21.15.1:0/2261791143 wait complete. 2024-02-02T15:41:29.507 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-02T15:41:29.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.788+0000 7f816da00700 1 -- 172.21.15.1:0/314813893 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f81681019b0 msgr2=0x7f81681059f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:41:29.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.788+0000 7f816da00700 1 --2- 172.21.15.1:0/314813893 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f81681019b0 0x7f81681059f0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f81500098f0 tx=0x7f8150009c00 comp rx=0 tx=0).stop 2024-02-02T15:41:29.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.788+0000 7f816da00700 1 -- 172.21.15.1:0/314813893 shutdown_connections 2024-02-02T15:41:29.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.788+0000 7f816da00700 1 --2- 172.21.15.1:0/314813893 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8168106120 0x7f816810dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.788+0000 7f816da00700 1 --2- 172.21.15.1:0/314813893 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8168101000 0x7f81681013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.788+0000 7f816da00700 1 --2- 172.21.15.1:0/314813893 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f81681019b0 0x7f81681059f0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.788+0000 7f816da00700 1 -- 172.21.15.1:0/314813893 >> 172.21.15.1:0/314813893 conn(0x7f81680fc840 msgr2=0x7f81680fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:41:29.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.789+0000 7f816da00700 1 -- 172.21.15.1:0/314813893 shutdown_connections 2024-02-02T15:41:29.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.789+0000 7f816da00700 1 -- 172.21.15.1:0/314813893 wait complete. 2024-02-02T15:41:29.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.790+0000 7f816da00700 1 Processor -- start 2024-02-02T15:41:29.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.791+0000 7f816da00700 1 -- start start 2024-02-02T15:41:29.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.791+0000 7f816da00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8168101000 0x7f8168199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:29.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.791+0000 7f816da00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8168106120 0x7f8168199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:29.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.791+0000 7f816da00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f816819dac0 0x7f816819df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:29.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.791+0000 7f816da00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f816819e540 con 0x7f816819dac0 2024-02-02T15:41:29.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.791+0000 7f816da00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f816819e6b0 con 0x7f8168106120 2024-02-02T15:41:29.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.791+0000 7f816da00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f816819e870 con 0x7f8168101000 2024-02-02T15:41:29.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.792+0000 7f8166a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8168101000 0x7f8168199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:29.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.792+0000 7f8166a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8168101000 0x7f8168199120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:35614/0 (socket says 172.21.15.1:35614) 2024-02-02T15:41:29.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.792+0000 7f8166a00700 1 -- 172.21.15.1:0/1599677470 learned_addr learned my addr 172.21.15.1:0/1599677470 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:41:29.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.792+0000 7f8166000700 1 --2- 172.21.15.1:0/1599677470 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8168106120 0x7f8168199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:29.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.792+0000 7f8167400700 1 --2- 172.21.15.1:0/1599677470 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f816819dac0 0x7f816819df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:29.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.792+0000 7f8166000700 1 -- 172.21.15.1:0/1599677470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8168101000 msgr2=0x7f8168199120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:41:29.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.792+0000 7f8166000700 1 --2- 172.21.15.1:0/1599677470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8168101000 0x7f8168199120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.792+0000 7f8166000700 1 -- 172.21.15.1:0/1599677470 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f816819dac0 msgr2=0x7f816819df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:41:29.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.792+0000 7f8166000700 1 --2- 172.21.15.1:0/1599677470 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f816819dac0 0x7f816819df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.792+0000 7f8166000700 1 -- 172.21.15.1:0/1599677470 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8150009520 con 0x7f8168106120 2024-02-02T15:41:29.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.792+0000 7f8166a00700 1 --2- 172.21.15.1:0/1599677470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8168101000 0x7f8168199120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:41:29.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.792+0000 7f8167400700 1 --2- 172.21.15.1:0/1599677470 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f816819dac0 0x7f816819df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:41:29.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.793+0000 7f8166000700 1 --2- 172.21.15.1:0/1599677470 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8168106120 0x7f8168199660 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f8150009230 tx=0x7f815000fa50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:41:29.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.793+0000 7f815f400700 1 -- 172.21.15.1:0/1599677470 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f815001c070 con 0x7f8168106120 2024-02-02T15:41:29.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.793+0000 7f815f400700 1 -- 172.21.15.1:0/1599677470 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f815000bba0 con 0x7f8168106120 2024-02-02T15:41:29.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.793+0000 7f815f400700 1 -- 172.21.15.1:0/1599677470 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81500038e0 con 0x7f8168106120 2024-02-02T15:41:29.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.793+0000 7f816da00700 1 -- 172.21.15.1:0/1599677470 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81681a5d70 con 0x7f8168106120 2024-02-02T15:41:29.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.793+0000 7f816da00700 1 -- 172.21.15.1:0/1599677470 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81681a61a0 con 0x7f8168106120 2024-02-02T15:41:29.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.798+0000 7f815f400700 1 -- 172.21.15.1:0/1599677470 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f8150020030 con 0x7f8168106120 2024-02-02T15:41:29.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.799+0000 7f81815b7b80 1 Processor -- start 2024-02-02T15:41:29.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.799+0000 7f81815b7b80 1 -- start start 2024-02-02T15:41:29.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.800+0000 7f81815b7b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1bc7bb010 0x55a1bc7bb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:29.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.800+0000 7f81815b7b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1bc7bb930 0x55a1bc7d72b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:29.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.800+0000 7f81815b7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a1bc7d77f0 0x55a1bc7d9be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:29.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.800+0000 7f81815b7b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1bc72f4c0 con 0x55a1bc7bb010 2024-02-02T15:41:29.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.800+0000 7f81815b7b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1bc713aa0 con 0x55a1bc7bb930 2024-02-02T15:41:29.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.800+0000 7f81815b7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1bc711460 con 0x55a1bc7d77f0 2024-02-02T15:41:29.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.800+0000 7f8167400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1bc7bb010 0x55a1bc7bb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:29.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.800+0000 7f8167400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1bc7bb010 0x55a1bc7bb3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33358/0 (socket says 172.21.15.1:33358) 2024-02-02T15:41:29.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.800+0000 7f8167400700 1 -- 172.21.15.1:0/673366236 learned_addr learned my addr 172.21.15.1:0/673366236 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:41:29.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.800+0000 7f8166000700 1 --2- 172.21.15.1:0/673366236 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a1bc7d77f0 0x55a1bc7d9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:29.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.801+0000 7f8166a00700 1 --2- 172.21.15.1:0/673366236 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1bc7bb930 0x55a1bc7d72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:29.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.801+0000 7f8167400700 1 -- 172.21.15.1:0/673366236 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a1bc7d77f0 msgr2=0x55a1bc7d9be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:41:29.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.801+0000 7f8167400700 1 --2- 172.21.15.1:0/673366236 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a1bc7d77f0 0x55a1bc7d9be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.801+0000 7f8167400700 1 -- 172.21.15.1:0/673366236 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1bc7bb930 msgr2=0x55a1bc7d72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:41:29.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.801+0000 7f8167400700 1 --2- 172.21.15.1:0/673366236 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1bc7bb930 0x55a1bc7d72b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.801+0000 7f8167400700 1 -- 172.21.15.1:0/673366236 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a1bc58e3b0 con 0x55a1bc7bb010 2024-02-02T15:41:29.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.801+0000 7f8167400700 1 --2- 172.21.15.1:0/673366236 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1bc7bb010 0x55a1bc7bb3f0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f8160009550 tx=0x7f81600079f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2fe8d29dfd4c578 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:41:29.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.801+0000 7f8157e00700 1 -- 172.21.15.1:0/673366236 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8160015850 con 0x55a1bc7bb010 2024-02-02T15:41:29.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.802+0000 7f8157e00700 1 -- 172.21.15.1:0/673366236 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f816000f040 con 0x55a1bc7bb010 2024-02-02T15:41:29.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.802+0000 7f8157e00700 1 -- 172.21.15.1:0/673366236 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8160014da0 con 0x55a1bc7bb010 2024-02-02T15:41:29.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.802+0000 7f815f400700 1 --2- 172.21.15.1:0/1599677470 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f81480a0790 0x7f81480a2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:29.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.802+0000 7f815f400700 1 -- 172.21.15.1:0/1599677470 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f81500cd010 con 0x7f8168106120 2024-02-02T15:41:29.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.803+0000 7f81815b7b80 1 -- 172.21.15.1:0/673366236 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1bc7bb010 msgr2=0x55a1bc7bb3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:41:29.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.803+0000 7f81815b7b80 1 --2- 172.21.15.1:0/673366236 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1bc7bb010 0x55a1bc7bb3f0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f8160009550 tx=0x7f81600079f0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.803+0000 7f81815b7b80 1 -- 172.21.15.1:0/673366236 shutdown_connections 2024-02-02T15:41:29.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.803+0000 7f81815b7b80 1 --2- 172.21.15.1:0/673366236 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a1bc7d77f0 0x55a1bc7d9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.803+0000 7f81815b7b80 1 --2- 172.21.15.1:0/673366236 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1bc7bb010 0x55a1bc7bb3f0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.803+0000 7f81815b7b80 1 --2- 172.21.15.1:0/673366236 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1bc7bb930 0x55a1bc7d72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.803 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.803+0000 7f81815b7b80 1 -- 172.21.15.1:0/673366236 >> 172.21.15.1:0/673366236 conn(0x55a1bc71adb0 msgr2=0x55a1bc7b73b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:41:29.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.804+0000 7f8166a00700 1 --2- 172.21.15.1:0/1599677470 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f81480a0790 0x7f81480a2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:29.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.804+0000 7f81815b7b80 1 -- 172.21.15.1:0/673366236 shutdown_connections 2024-02-02T15:41:29.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.806+0000 7f8166a00700 1 --2- 172.21.15.1:0/1599677470 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f81480a0790 0x7f81480a2c50 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8158005950 tx=0x7f8158009310 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:41:29.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.806+0000 7f81815b7b80 1 -- 172.21.15.1:0/673366236 wait complete. 2024-02-02T15:41:29.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.807+0000 7f81815b7b80 1 -- start start 2024-02-02T15:41:29.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.807+0000 7f81815b7b80 1 Processor -- start 2024-02-02T15:41:29.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.808+0000 7f81815b7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a1bc7bb010 0x55a1bc7d7190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:29.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.808+0000 7f81815b7b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1bc7bb930 0x55a1bc7d4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:29.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.809+0000 7f81815b7b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1bc7d77f0 0x55a1bc7d49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:29.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.809+0000 7f81815b7b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1bc70cd50 con 0x55a1bc7bb930 2024-02-02T15:41:29.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.809+0000 7f81815b7b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1bc6a8d80 con 0x55a1bc7d77f0 2024-02-02T15:41:29.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.809+0000 7f81815b7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1bc6a8a20 con 0x55a1bc7bb010 2024-02-02T15:41:29.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.810+0000 7f8166a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a1bc7bb010 0x55a1bc7d7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:29.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.810+0000 7f8167400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1bc7d77f0 0x55a1bc7d49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:29.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.810+0000 7f8167400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1bc7d77f0 0x55a1bc7d49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59548/0 (socket says 172.21.15.1:59548) 2024-02-02T15:41:29.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.810+0000 7f8166000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1bc7bb930 0x55a1bc7d4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:29.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.810+0000 7f8167400700 1 -- 172.21.15.1:0/2950550095 learned_addr learned my addr 172.21.15.1:0/2950550095 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:41:29.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.810+0000 7f81815b7b80 10 client.0 _ll_register_callbacks cb 0x55a1bc6aa290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:41:29.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.810+0000 7f8166a00700 1 -- 172.21.15.1:0/2950550095 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1bc7d77f0 msgr2=0x55a1bc7d49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:41:29.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.810+0000 7f8166a00700 1 --2- 172.21.15.1:0/2950550095 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1bc7d77f0 0x55a1bc7d49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.810+0000 7f8166a00700 1 -- 172.21.15.1:0/2950550095 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1bc7bb930 msgr2=0x55a1bc7d4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:41:29.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.810+0000 7f8166a00700 1 --2- 172.21.15.1:0/2950550095 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1bc7bb930 0x55a1bc7d4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.810+0000 7f8166a00700 1 -- 172.21.15.1:0/2950550095 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8160009e30 con 0x55a1bc7bb010 2024-02-02T15:41:29.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.810+0000 7f81815b7b80 1 -- 172.21.15.1:0/2950550095 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a1bc7af0f0 con 0x55a1bc7bb010 2024-02-02T15:41:29.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.810+0000 7f8166000700 1 --2- 172.21.15.1:0/2950550095 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1bc7bb930 0x55a1bc7d4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:41:29.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.811+0000 7f8166a00700 1 --2- 172.21.15.1:0/2950550095 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a1bc7bb010 0x55a1bc7d7190 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f815801e9d0 tx=0x7f81580047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9c05649ddc43f3d server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:41:29.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.811+0000 7f8154c00700 10 client.29443 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:41:29.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.812+0000 7f8154c00700 1 -- 172.21.15.1:0/2950550095 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8158023070 con 0x55a1bc7bb010 2024-02-02T15:41:29.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.812+0000 7f8154c00700 1 -- 172.21.15.1:0/2950550095 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f8158012e20 con 0x55a1bc7bb010 2024-02-02T15:41:29.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.812+0000 7f8154c00700 1 -- 172.21.15.1:0/2950550095 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8158017070 con 0x55a1bc7bb010 2024-02-02T15:41:29.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.812+0000 7f81815b7b80 10 client.29443 fetch_fsmap learned FSMap version 61 2024-02-02T15:41:29.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.812+0000 7f81815b7b80 1 -- 172.21.15.1:0/2950550095 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({fsmap=61}) v3 -- 0x55a1bc53ab60 con 0x55a1bc7bb010 2024-02-02T15:41:29.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.813+0000 7f8154c00700 1 -- 172.21.15.1:0/2950550095 <== mon.1 v2:172.21.15.142:3300/0 4 ==== fsmap(e 61) v1 ==== 7550+0+0 (secure 0 0 0) 0x7f815800f7d0 con 0x55a1bc7bb010 2024-02-02T15:41:29.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.813+0000 7f81815b7b80 10 client.29443 fetch_fsmap finished waiting for FSMap version 61 2024-02-02T15:41:29.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.813+0000 7f81815b7b80 10 client.29443 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:41:29.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.813+0000 7f81815b7b80 20 client.29443 populate_metadata read hostname 'smithi001' 2024-02-02T15:41:29.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.813+0000 7f81815b7b80 1 --2- 172.21.15.1:0/2950550095 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55a1bc7ef840 0x55a1bc7f1c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:29.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.813+0000 7f81815b7b80 4 client.29443 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:41:29.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.813+0000 7f81815b7b80 1 -- 172.21.15.1:0/2950550095 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a1bc705dd0 con 0x55a1bc7ef840 2024-02-02T15:41:29.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.813+0000 7f8166000700 1 --2- 172.21.15.1:0/2950550095 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55a1bc7ef840 0x55a1bc7f1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:29.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.814+0000 7f8166000700 1 --2- 172.21.15.1:0/2950550095 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55a1bc7ef840 0x55a1bc7f1c60 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbd55738774fedb1 server_cookie=233f5e121d750ea8 in_seq=0 out_seq=0 2024-02-02T15:41:29.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.814+0000 7f8154c00700 10 client.29443 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:41:29.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.815+0000 7f8154c00700 1 -- 172.21.15.1:0/2950550095 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55a1bc705dd0 con 0x55a1bc7ef840 2024-02-02T15:41:29.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.815+0000 7f8154c00700 10 client.29443 handle_command_reply: tid=0 2024-02-02T15:41:29.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.815+0000 7f81815b7b80 1 client.29443 shutdown 2024-02-02T15:41:29.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.815+0000 7f81815b7b80 1 -- 172.21.15.1:0/2950550095 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a1bc7bb010 msgr2=0x55a1bc7d7190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:41:29.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.815+0000 7f81815b7b80 1 --2- 172.21.15.1:0/2950550095 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a1bc7bb010 0x55a1bc7d7190 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f815801e9d0 tx=0x7f81580047c0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.815+0000 7f81815b7b80 1 -- 172.21.15.1:0/2950550095 shutdown_connections 2024-02-02T15:41:29.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.815+0000 7f81815b7b80 1 --2- 172.21.15.1:0/2950550095 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55a1bc7ef840 0x55a1bc7f1c60 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.815+0000 7f81815b7b80 1 --2- 172.21.15.1:0/2950550095 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1bc7d77f0 0x55a1bc7d49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.815+0000 7f81815b7b80 1 --2- 172.21.15.1:0/2950550095 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a1bc7bb010 0x55a1bc7d7190 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.815+0000 7f81815b7b80 1 --2- 172.21.15.1:0/2950550095 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1bc7bb930 0x55a1bc7d4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.815+0000 7f8154c00700 0 client.29443 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:41:29.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.815+0000 7f81815b7b80 1 -- 172.21.15.1:0/2950550095 >> 172.21.15.1:0/2950550095 conn(0x55a1bc71adb0 msgr2=0x55a1bc7b6fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:41:29.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.815+0000 7f81815b7b80 1 -- 172.21.15.1:0/2950550095 shutdown_connections 2024-02-02T15:41:29.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.815+0000 7f81815b7b80 1 -- 172.21.15.1:0/2950550095 wait complete. 2024-02-02T15:41:29.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.816+0000 7f81815b7b80 20 client.29443 trim_cache size 0 max 16384 2024-02-02T15:41:29.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.834+0000 7f81815b7b80 1 Processor -- start 2024-02-02T15:41:29.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.834+0000 7f81815b7b80 1 -- start start 2024-02-02T15:41:29.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.834+0000 7f81815b7b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1bc71adb0 0x55a1bc7e6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:29.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.834+0000 7f81815b7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a1bc7bb010 0x55a1bc7d6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:29.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.834+0000 7f81815b7b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1bc7bb930 0x55a1bc7d3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:29.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.834+0000 7f81815b7b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1bc5906a0 con 0x55a1bc7bb930 2024-02-02T15:41:29.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.834+0000 7f81815b7b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1bc7b8a10 con 0x55a1bc71adb0 2024-02-02T15:41:29.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.834+0000 7f81815b7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1bc7b87e0 con 0x55a1bc7bb010 2024-02-02T15:41:29.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.835+0000 7f8166a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1bc71adb0 0x55a1bc7e6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:29.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.835+0000 7f8166a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1bc71adb0 0x55a1bc7e6c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59564/0 (socket says 172.21.15.1:59564) 2024-02-02T15:41:29.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.835+0000 7f8166a00700 1 -- 172.21.15.1:0/3974951569 learned_addr learned my addr 172.21.15.1:0/3974951569 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:41:29.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.835+0000 7f8166000700 1 --2- 172.21.15.1:0/3974951569 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a1bc7bb010 0x55a1bc7d6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:29.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.835+0000 7f8167400700 1 --2- 172.21.15.1:0/3974951569 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1bc7bb930 0x55a1bc7d3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:29.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.835+0000 7f8166000700 1 -- 172.21.15.1:0/3974951569 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1bc71adb0 msgr2=0x55a1bc7e6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:41:29.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.835+0000 7f8166000700 1 --2- 172.21.15.1:0/3974951569 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1bc71adb0 0x55a1bc7e6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.835+0000 7f8166000700 1 -- 172.21.15.1:0/3974951569 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1bc7bb930 msgr2=0x55a1bc7d3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:41:29.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.835+0000 7f8166000700 1 --2- 172.21.15.1:0/3974951569 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1bc7bb930 0x55a1bc7d3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.835+0000 7f8166000700 1 -- 172.21.15.1:0/3974951569 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a1bc6d2ea0 con 0x55a1bc7bb010 2024-02-02T15:41:29.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.835+0000 7f8166a00700 1 --2- 172.21.15.1:0/3974951569 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1bc71adb0 0x55a1bc7e6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:41:29.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.835+0000 7f8167400700 1 --2- 172.21.15.1:0/3974951569 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1bc7bb930 0x55a1bc7d3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:41:29.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.836+0000 7f8166000700 1 --2- 172.21.15.1:0/3974951569 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a1bc7bb010 0x55a1bc7d6cc0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f81500ca330 tx=0x7f8150009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a267544bd219804 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:41:29.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.839+0000 7f815cc00700 1 -- 172.21.15.1:0/3974951569 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8150099b20 con 0x55a1bc7bb010 2024-02-02T15:41:29.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.839+0000 7f815cc00700 1 -- 172.21.15.1:0/3974951569 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8150021380 con 0x55a1bc7bb010 2024-02-02T15:41:29.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.839+0000 7f815cc00700 1 -- 172.21.15.1:0/3974951569 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81500a00b0 con 0x55a1bc7bb010 2024-02-02T15:41:29.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.840+0000 7f81815b7b80 1 -- 172.21.15.1:0/3974951569 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a1bc7bb010 msgr2=0x55a1bc7d6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:41:29.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.840+0000 7f81815b7b80 1 --2- 172.21.15.1:0/3974951569 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a1bc7bb010 0x55a1bc7d6cc0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f81500ca330 tx=0x7f8150009260 comp rx=0 tx=0).stop 2024-02-02T15:41:29.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.840+0000 7f81815b7b80 1 -- 172.21.15.1:0/3974951569 shutdown_connections 2024-02-02T15:41:29.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.840+0000 7f81815b7b80 1 --2- 172.21.15.1:0/3974951569 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1bc7bb930 0x55a1bc7d3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.840+0000 7f81815b7b80 1 --2- 172.21.15.1:0/3974951569 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1bc71adb0 0x55a1bc7e6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.840+0000 7f81815b7b80 1 --2- 172.21.15.1:0/3974951569 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a1bc7bb010 0x55a1bc7d6cc0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.840+0000 7f81815b7b80 1 -- 172.21.15.1:0/3974951569 >> 172.21.15.1:0/3974951569 conn(0x55a1bc7d9e00 msgr2=0x55a1bc7cf7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:41:29.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.840+0000 7f81815b7b80 1 -- 172.21.15.1:0/3974951569 shutdown_connections 2024-02-02T15:41:29.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.840+0000 7f81815b7b80 1 -- 172.21.15.1:0/3974951569 wait complete. 2024-02-02T15:41:29.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.841+0000 7f81815b7b80 1 -- start start 2024-02-02T15:41:29.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.841+0000 7f81815b7b80 1 Processor -- start 2024-02-02T15:41:29.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.842+0000 7f81815b7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a1bc71adb0 0x55a1bc7e6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:29.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.842+0000 7f81815b7b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1bc7bb010 0x55a1bc7e68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:29.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.842+0000 7f81815b7b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1bc7bb930 0x55a1bc7e2be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:29.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.842+0000 7f81815b7b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1bc7e32f0 con 0x55a1bc7bb930 2024-02-02T15:41:29.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.842+0000 7f81815b7b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1bc7e3460 con 0x55a1bc7bb010 2024-02-02T15:41:29.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.842+0000 7f81815b7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1bc7e3620 con 0x55a1bc71adb0 2024-02-02T15:41:29.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.842+0000 7f81815b7b80 10 client.0 _ll_register_callbacks cb 0x55a1bc6aa290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:41:29.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.842+0000 7f8166a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a1bc71adb0 0x55a1bc7e6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:29.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.842+0000 7f8166a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a1bc71adb0 0x55a1bc7e6390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:35654/0 (socket says 172.21.15.1:35654) 2024-02-02T15:41:29.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.842+0000 7f8166a00700 1 -- 172.21.15.1:0/3730354946 learned_addr learned my addr 172.21.15.1:0/3730354946 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:41:29.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.842+0000 7f8166000700 1 --2- 172.21.15.1:0/3730354946 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1bc7bb010 0x55a1bc7e68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:29.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.842+0000 7f8167400700 1 --2- 172.21.15.1:0/3730354946 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1bc7bb930 0x55a1bc7e2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:29.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.842+0000 7f8166000700 1 -- 172.21.15.1:0/3730354946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a1bc71adb0 msgr2=0x55a1bc7e6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:41:29.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.842+0000 7f8166000700 1 --2- 172.21.15.1:0/3730354946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a1bc71adb0 0x55a1bc7e6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.842+0000 7f8166000700 1 -- 172.21.15.1:0/3730354946 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1bc7bb930 msgr2=0x55a1bc7e2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:41:29.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.842+0000 7f8166000700 1 --2- 172.21.15.1:0/3730354946 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1bc7bb930 0x55a1bc7e2be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.842+0000 7f8166000700 1 -- 172.21.15.1:0/3730354946 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a1bc53ab60 con 0x55a1bc7bb010 2024-02-02T15:41:29.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.843+0000 7f8166000700 1 --2- 172.21.15.1:0/3730354946 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1bc7bb010 0x55a1bc7e68d0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x55a1bc7b8970 tx=0x7f815000bd10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=551530326fccef9 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:41:29.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.843+0000 7f8154c00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:41:29.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.843+0000 7f8154c00700 1 -- 172.21.15.1:0/3730354946 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f815009a1c0 con 0x55a1bc7bb010 2024-02-02T15:41:29.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.843+0000 7f81815b7b80 1 -- 172.21.15.1:0/3730354946 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f816000f040 con 0x55a1bc7bb010 2024-02-02T15:41:29.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.843+0000 7f8154c00700 1 -- 172.21.15.1:0/3730354946 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f815009a8c0 con 0x55a1bc7bb010 2024-02-02T15:41:29.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.843+0000 7f8154c00700 1 -- 172.21.15.1:0/3730354946 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f81500a21a0 con 0x55a1bc7bb010 2024-02-02T15:41:29.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.843+0000 7f81815b7b80 10 client.29474 fetch_fsmap learned FSMap version 61 2024-02-02T15:41:29.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.843+0000 7f81815b7b80 1 -- 172.21.15.1:0/3730354946 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x7f81681a61a0 con 0x55a1bc7bb010 2024-02-02T15:41:29.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.844+0000 7f8154c00700 1 -- 172.21.15.1:0/3730354946 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f816000f040 con 0x55a1bc7bb010 2024-02-02T15:41:29.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.844+0000 7f8154c00700 1 -- 172.21.15.1:0/3730354946 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 61) v1 ==== 7550+0+0 (secure 0 0 0) 0x7f81500051b0 con 0x55a1bc7bb010 2024-02-02T15:41:29.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.844+0000 7f81815b7b80 10 client.29474 fetch_fsmap finished waiting for FSMap version 61 2024-02-02T15:41:29.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.844+0000 7f81815b7b80 10 client.29474 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:41:29.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.844+0000 7f81815b7b80 20 client.29474 populate_metadata read hostname 'smithi001' 2024-02-02T15:41:29.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.844+0000 7f81815b7b80 1 --2- 172.21.15.1:0/3730354946 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55a1bc7d77f0 0x55a1bc844140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:29.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.844+0000 7f81815b7b80 4 client.29474 mds_command: new command op to 24557 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:41:29.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.844+0000 7f81815b7b80 1 -- 172.21.15.1:0/3730354946 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a1bc6cbf50 con 0x55a1bc7d77f0 2024-02-02T15:41:29.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.844+0000 7f8166a00700 1 --2- 172.21.15.1:0/3730354946 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55a1bc7d77f0 0x55a1bc844140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:29.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.845+0000 7f8166a00700 1 --2- 172.21.15.1:0/3730354946 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55a1bc7d77f0 0x55a1bc844140 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98170530be51fab7 server_cookie=ae229cd58c455606 in_seq=0 out_seq=0 2024-02-02T15:41:29.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.845+0000 7f8154c00700 10 client.29474 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:41:29.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.845+0000 7f8154c00700 1 -- 172.21.15.1:0/3730354946 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a1bc6cbf50 con 0x55a1bc7d77f0 2024-02-02T15:41:29.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.845+0000 7f8154c00700 10 client.29474 handle_command_reply: tid=0 2024-02-02T15:41:29.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.845+0000 7f81815b7b80 1 client.29474 shutdown 2024-02-02T15:41:29.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.845+0000 7f81815b7b80 1 -- 172.21.15.1:0/3730354946 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1bc7bb010 msgr2=0x55a1bc7e68d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:41:29.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.845+0000 7f81815b7b80 1 --2- 172.21.15.1:0/3730354946 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1bc7bb010 0x55a1bc7e68d0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x55a1bc7b8970 tx=0x7f815000bd10 comp rx=0 tx=0).stop 2024-02-02T15:41:29.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.845+0000 7f81815b7b80 1 -- 172.21.15.1:0/3730354946 shutdown_connections 2024-02-02T15:41:29.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.845+0000 7f81815b7b80 1 --2- 172.21.15.1:0/3730354946 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55a1bc7d77f0 0x55a1bc844140 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.845+0000 7f81815b7b80 1 --2- 172.21.15.1:0/3730354946 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1bc7bb930 0x55a1bc7e2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.845+0000 7f81815b7b80 1 --2- 172.21.15.1:0/3730354946 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a1bc71adb0 0x55a1bc7e6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.845+0000 7f81815b7b80 1 --2- 172.21.15.1:0/3730354946 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1bc7bb010 0x55a1bc7e68d0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.845+0000 7f8154c00700 0 client.29474 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:41:29.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.845+0000 7f81815b7b80 1 -- 172.21.15.1:0/3730354946 >> 172.21.15.1:0/3730354946 conn(0x55a1bc7d9e00 msgr2=0x55a1bc7bccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:41:29.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.846+0000 7f81815b7b80 1 -- 172.21.15.1:0/3730354946 shutdown_connections 2024-02-02T15:41:29.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.846+0000 7f81815b7b80 1 -- 172.21.15.1:0/3730354946 wait complete. 2024-02-02T15:41:29.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.846+0000 7f81815b7b80 20 client.29474 trim_cache size 0 max 16384 2024-02-02T15:41:29.848 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:41:29.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.847+0000 7f816da00700 1 -- 172.21.15.1:0/1599677470 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f81480a0790 msgr2=0x7f81480a2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:41:29.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.847+0000 7f816da00700 1 --2- 172.21.15.1:0/1599677470 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f81480a0790 0x7f81480a2c50 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8158005950 tx=0x7f8158009310 comp rx=0 tx=0).stop 2024-02-02T15:41:29.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.847+0000 7f816da00700 1 -- 172.21.15.1:0/1599677470 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8168106120 msgr2=0x7f8168199660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:41:29.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.847+0000 7f816da00700 1 --2- 172.21.15.1:0/1599677470 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8168106120 0x7f8168199660 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f8150009230 tx=0x7f815000fa50 comp rx=0 tx=0).stop 2024-02-02T15:41:29.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.847+0000 7f816da00700 1 -- 172.21.15.1:0/1599677470 shutdown_connections 2024-02-02T15:41:29.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.847+0000 7f816da00700 1 --2- 172.21.15.1:0/1599677470 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f81480a0790 0x7f81480a2c50 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.847+0000 7f816da00700 1 --2- 172.21.15.1:0/1599677470 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f816819dac0 0x7f816819df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.847+0000 7f816da00700 1 --2- 172.21.15.1:0/1599677470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8168101000 0x7f8168199120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.847+0000 7f816da00700 1 --2- 172.21.15.1:0/1599677470 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8168106120 0x7f8168199660 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:29.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.847+0000 7f816da00700 1 -- 172.21.15.1:0/1599677470 >> 172.21.15.1:0/1599677470 conn(0x7f81680fc840 msgr2=0x7f81681029c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:41:29.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.847+0000 7f816da00700 1 -- 172.21.15.1:0/1599677470 shutdown_connections 2024-02-02T15:41:29.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:29.847+0000 7f816da00700 1 -- 172.21.15.1:0/1599677470 wait complete. 2024-02-02T15:41:29.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:41:29 smithi071 ceph-mon[44707]: pgmap v453: 129 pgs: 129 active+clean; 1.0 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 8.9 MiB/s wr, 618 op/s 2024-02-02T15:41:29.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:41:29 smithi071 ceph-mon[44707]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:41:29.866 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:41:29 smithi001 ceph-mon[35470]: pgmap v453: 129 pgs: 129 active+clean; 1.0 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 8.9 MiB/s wr, 618 op/s 2024-02-02T15:41:29.866 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:41:29 smithi001 ceph-mon[35470]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:41:29.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:41:29 smithi142 ceph-mon[45728]: pgmap v453: 129 pgs: 129 active+clean; 1.0 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 8.9 MiB/s wr, 618 op/s 2024-02-02T15:41:29.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:41:29 smithi142 ceph-mon[45728]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:41:29.974 INFO:tasks.workunit.client.0.smithi001.stdout: 10 33887 32.23 MB/sec warmup 62 sec latency 105.485 ms 2024-02-02T15:41:30.869 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-02T15:41:30.870 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-02T15:41:30.975 INFO:tasks.workunit.client.0.smithi001.stdout: 10 34520 32.29 MB/sec warmup 63 sec latency 261.080 ms 2024-02-02T15:41:31.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.181+0000 7fcfdac00700 1 -- 172.21.15.1:0/1782764598 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcfd410d650 msgr2=0x7fcfd410fa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:41:31.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.181+0000 7fcfdac00700 1 --2- 172.21.15.1:0/1782764598 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcfd410d650 0x7fcfd410fa40 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc00b0a0 tx=0x7fcfcc00b3b0 comp rx=0 tx=0).stop 2024-02-02T15:41:31.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.181+0000 7fcfdac00700 1 -- 172.21.15.1:0/1782764598 shutdown_connections 2024-02-02T15:41:31.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.181+0000 7fcfdac00700 1 --2- 172.21.15.1:0/1782764598 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcfd410d650 0x7fcfd410fa40 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:31.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.181+0000 7fcfdac00700 1 --2- 172.21.15.1:0/1782764598 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcfd40730f0 0x7fcfd40734d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:31.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.181+0000 7fcfdac00700 1 --2- 172.21.15.1:0/1782764598 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcfd4073a10 0x7fcfd410d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:31.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.181+0000 7fcfdac00700 1 -- 172.21.15.1:0/1782764598 >> 172.21.15.1:0/1782764598 conn(0x7fcfd40fc840 msgr2=0x7fcfd40fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:41:31.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.181+0000 7fcfdac00700 1 -- 172.21.15.1:0/1782764598 shutdown_connections 2024-02-02T15:41:31.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.181+0000 7fcfdac00700 1 -- 172.21.15.1:0/1782764598 wait complete. 2024-02-02T15:41:31.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.183+0000 7fcfdac00700 1 Processor -- start 2024-02-02T15:41:31.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.183+0000 7fcfdac00700 1 -- start start 2024-02-02T15:41:31.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.183+0000 7fcfdac00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcfd40730f0 0x7fcfd4199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:31.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.184+0000 7fcfdac00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcfd4073a10 0x7fcfd4199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:31.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.184+0000 7fcfdac00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcfd410d650 0x7fcfd419dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:31.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.184+0000 7fcfdac00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd419e230 con 0x7fcfd410d650 2024-02-02T15:41:31.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.184+0000 7fcfdac00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd419e3a0 con 0x7fcfd40730f0 2024-02-02T15:41:31.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.184+0000 7fcfdac00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd419e560 con 0x7fcfd4073a10 2024-02-02T15:41:31.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.184+0000 7fcfd3e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcfd40730f0 0x7fcfd4199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:31.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.184+0000 7fcfd3400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcfd4073a10 0x7fcfd4199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:31.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.184+0000 7fcfd3400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcfd4073a10 0x7fcfd4199790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:35672/0 (socket says 172.21.15.1:35672) 2024-02-02T15:41:31.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.184+0000 7fcfd3400700 1 -- 172.21.15.1:0/4235346312 learned_addr learned my addr 172.21.15.1:0/4235346312 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:41:31.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.184+0000 7fcfd8c00700 1 --2- 172.21.15.1:0/4235346312 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcfd410d650 0x7fcfd419dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:31.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.185+0000 7fcfd3400700 1 -- 172.21.15.1:0/4235346312 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcfd40730f0 msgr2=0x7fcfd4199250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:41:31.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.185+0000 7fcfd3400700 1 --2- 172.21.15.1:0/4235346312 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcfd40730f0 0x7fcfd4199250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:31.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.185+0000 7fcfd3400700 1 -- 172.21.15.1:0/4235346312 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcfd410d650 msgr2=0x7fcfd419dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:41:31.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.185+0000 7fcfd3400700 1 --2- 172.21.15.1:0/4235346312 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcfd410d650 0x7fcfd419dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:31.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.185+0000 7fcfd3400700 1 -- 172.21.15.1:0/4235346312 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfbc009520 con 0x7fcfd4073a10 2024-02-02T15:41:31.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.185+0000 7fcfd3400700 1 --2- 172.21.15.1:0/4235346312 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcfd4073a10 0x7fcfd4199790 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc00eb30 tx=0x7fcfbc00ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:41:31.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.185+0000 7fcfd0c00700 1 -- 172.21.15.1:0/4235346312 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfbc009880 con 0x7fcfd4073a10 2024-02-02T15:41:31.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.185+0000 7fcfdac00700 1 -- 172.21.15.1:0/4235346312 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfcc009d00 con 0x7fcfd4073a10 2024-02-02T15:41:31.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.185+0000 7fcfdac00700 1 -- 172.21.15.1:0/4235346312 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfd419ea30 con 0x7fcfd4073a10 2024-02-02T15:41:31.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.186+0000 7fcfd0c00700 1 -- 172.21.15.1:0/4235346312 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcfbc0044e0 con 0x7fcfd4073a10 2024-02-02T15:41:31.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.186+0000 7fcfd0c00700 1 -- 172.21.15.1:0/4235346312 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfbc003e90 con 0x7fcfd4073a10 2024-02-02T15:41:31.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.190+0000 7fcfd0c00700 1 -- 172.21.15.1:0/4235346312 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7fcfbc003680 con 0x7fcfd4073a10 2024-02-02T15:41:31.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.192+0000 7fcfee657b80 1 Processor -- start 2024-02-02T15:41:31.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.193+0000 7fcfee657b80 1 -- start start 2024-02-02T15:41:31.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.193+0000 7fcfd0c00700 1 --2- 172.21.15.1:0/4235346312 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fcfb40a06f0 0x7fcfb40a2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:31.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.193+0000 7fcfd0c00700 1 -- 172.21.15.1:0/4235346312 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fcfbc014070 con 0x7fcfd4073a10 2024-02-02T15:41:31.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.194+0000 7fcfee657b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d8c3289090 0x55d8c3289470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:31.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.194+0000 7fcfee657b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d8c32899b0 0x55d8c3294320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:31.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.194+0000 7fcfee657b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d8c3294860 0x55d8c3296c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:31.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.194+0000 7fcfee657b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8c31fd540 con 0x55d8c3289090 2024-02-02T15:41:31.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.194+0000 7fcfee657b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8c31e1b20 con 0x55d8c32899b0 2024-02-02T15:41:31.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.194+0000 7fcfd8c00700 1 --2- 172.21.15.1:0/4235346312 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fcfb40a06f0 0x7fcfb40a2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:31.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.195+0000 7fcfd3400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d8c32899b0 0x55d8c3294320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:31.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.195+0000 7fcfd3400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d8c32899b0 0x55d8c3294320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59606/0 (socket says 172.21.15.1:59606) 2024-02-02T15:41:31.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.195+0000 7fcfd3400700 1 -- 172.21.15.1:0/391281584 learned_addr learned my addr 172.21.15.1:0/391281584 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:41:31.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.195+0000 7fcfee657b80 1 -- 172.21.15.1:0/391281584 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8c31df4e0 con 0x55d8c3294860 2024-02-02T15:41:31.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.195+0000 7fcfd8c00700 1 --2- 172.21.15.1:0/391281584 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d8c3294860 0x55d8c3296c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:31.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.196+0000 7fcfd3e00700 1 --2- 172.21.15.1:0/391281584 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d8c3289090 0x55d8c3289470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:31.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.196+0000 7fcfd3400700 1 -- 172.21.15.1:0/391281584 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d8c3294860 msgr2=0x55d8c3296c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:41:31.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.196+0000 7fcfd3400700 1 --2- 172.21.15.1:0/391281584 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d8c3294860 0x55d8c3296c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:31.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.196+0000 7fcfd3400700 1 -- 172.21.15.1:0/391281584 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d8c3289090 msgr2=0x55d8c3289470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:41:31.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.196+0000 7fcfd3400700 1 --2- 172.21.15.1:0/391281584 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d8c3289090 0x55d8c3289470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:31.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.196+0000 7fcfd3400700 1 -- 172.21.15.1:0/391281584 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d8c305c430 con 0x55d8c32899b0 2024-02-02T15:41:31.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.196+0000 7fcfd8c00700 1 --2- 172.21.15.1:0/4235346312 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fcfb40a06f0 0x7fcfb40a2bb0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc0062a0 tx=0x7fcfcc00bd90 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:41:31.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.197+0000 7fcfd3400700 1 --2- 172.21.15.1:0/391281584 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d8c32899b0 0x55d8c3294320 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc0c9da0 tx=0x7fcfbc098f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff1118cb7aa52518 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:41:31.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.197+0000 7fcfc9200700 1 -- 172.21.15.1:0/391281584 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfbc09d170 con 0x55d8c32899b0 2024-02-02T15:41:31.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.197+0000 7fcfc9200700 1 -- 172.21.15.1:0/391281584 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcfbc003e90 con 0x55d8c32899b0 2024-02-02T15:41:31.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.198+0000 7fcfc9200700 1 -- 172.21.15.1:0/391281584 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfbc0840c0 con 0x55d8c32899b0 2024-02-02T15:41:31.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.199+0000 7fcfee657b80 1 -- 172.21.15.1:0/391281584 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d8c32899b0 msgr2=0x55d8c3294320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:41:31.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.199+0000 7fcfee657b80 1 --2- 172.21.15.1:0/391281584 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d8c32899b0 0x55d8c3294320 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc0c9da0 tx=0x7fcfbc098f50 comp rx=0 tx=0).stop 2024-02-02T15:41:31.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.199+0000 7fcfee657b80 1 -- 172.21.15.1:0/391281584 shutdown_connections 2024-02-02T15:41:31.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.199+0000 7fcfee657b80 1 --2- 172.21.15.1:0/391281584 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d8c3294860 0x55d8c3296c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:31.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.199+0000 7fcfee657b80 1 --2- 172.21.15.1:0/391281584 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d8c3289090 0x55d8c3289470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:31.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.199+0000 7fcfee657b80 1 --2- 172.21.15.1:0/391281584 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d8c32899b0 0x55d8c3294320 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:31.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.199+0000 7fcfee657b80 1 -- 172.21.15.1:0/391281584 >> 172.21.15.1:0/391281584 conn(0x55d8c31e8e30 msgr2=0x55d8c3285430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:41:31.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.199+0000 7fcfee657b80 1 -- 172.21.15.1:0/391281584 shutdown_connections 2024-02-02T15:41:31.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.199+0000 7fcfee657b80 1 -- 172.21.15.1:0/391281584 wait complete. 2024-02-02T15:41:31.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.200+0000 7fcfee657b80 1 -- start start 2024-02-02T15:41:31.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.201+0000 7fcfee657b80 1 Processor -- start 2024-02-02T15:41:31.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.202+0000 7fcfee657b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d8c3289090 0x55d8c3294190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:31.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.202+0000 7fcfee657b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d8c32899b0 0x55d8c3291410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:31.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.202+0000 7fcfee657b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d8c3294860 0x55d8c3291970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:31.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.202+0000 7fcfee657b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8c31dadd0 con 0x55d8c32899b0 2024-02-02T15:41:31.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.202+0000 7fcfee657b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8c3176e00 con 0x55d8c3294860 2024-02-02T15:41:31.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.202+0000 7fcfee657b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8c3176aa0 con 0x55d8c3289090 2024-02-02T15:41:31.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.203+0000 7fcfd3e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d8c3289090 0x55d8c3294190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:31.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.203+0000 7fcfd3e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d8c3289090 0x55d8c3294190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:35688/0 (socket says 172.21.15.1:35688) 2024-02-02T15:41:31.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.203+0000 7fcfd3e00700 1 -- 172.21.15.1:0/3846679523 learned_addr learned my addr 172.21.15.1:0/3846679523 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:41:31.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.203+0000 7fcfd3400700 1 --2- 172.21.15.1:0/3846679523 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d8c32899b0 0x55d8c3291410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:31.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.203+0000 7fcfd8c00700 1 --2- 172.21.15.1:0/3846679523 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d8c3294860 0x55d8c3291970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:31.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.203+0000 7fcfee657b80 10 client.0 _ll_register_callbacks cb 0x55d8c3178310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:41:31.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.203+0000 7fcfd3e00700 1 -- 172.21.15.1:0/3846679523 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d8c3294860 msgr2=0x55d8c3291970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:41:31.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.203+0000 7fcfd3e00700 1 --2- 172.21.15.1:0/3846679523 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d8c3294860 0x55d8c3291970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:31.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.203+0000 7fcfd3e00700 1 -- 172.21.15.1:0/3846679523 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d8c32899b0 msgr2=0x55d8c3291410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:41:31.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.203+0000 7fcfd3e00700 1 --2- 172.21.15.1:0/3846679523 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d8c32899b0 0x55d8c3291410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:31.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.203+0000 7fcfd3e00700 1 -- 172.21.15.1:0/3846679523 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfd419ea30 con 0x55d8c3289090 2024-02-02T15:41:31.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.204+0000 7fcfd3e00700 1 --2- 172.21.15.1:0/3846679523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d8c3289090 0x55d8c3294190 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4010470 tx=0x7fcfc4010780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc74cb29bdbd0d3c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:41:31.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.204+0000 7fcfc1600700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:41:31.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.204+0000 7fcfc1600700 1 -- 172.21.15.1:0/3846679523 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc40145f0 con 0x55d8c3289090 2024-02-02T15:41:31.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.204+0000 7fcfc1600700 1 -- 172.21.15.1:0/3846679523 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fcfc4014750 con 0x55d8c3289090 2024-02-02T15:41:31.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.204+0000 7fcfee657b80 1 -- 172.21.15.1:0/3846679523 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d8c327d170 con 0x55d8c3289090 2024-02-02T15:41:31.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.205+0000 7fcfc1600700 1 -- 172.21.15.1:0/3846679523 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcfc4014910 con 0x55d8c3289090 2024-02-02T15:41:31.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.205+0000 7fcfee657b80 10 client.29458 fetch_fsmap learned FSMap version 61 2024-02-02T15:41:31.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.205+0000 7fcfee657b80 1 -- 172.21.15.1:0/3846679523 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x55d8c3008be0 con 0x55d8c3289090 2024-02-02T15:41:31.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.205+0000 7fcfc1600700 1 -- 172.21.15.1:0/3846679523 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d8c327d170 con 0x55d8c3289090 2024-02-02T15:41:31.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.206+0000 7fcfc1600700 1 -- 172.21.15.1:0/3846679523 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 61) v1 ==== 7550+0+0 (secure 0 0 0) 0x7fcfc401cc20 con 0x55d8c3289090 2024-02-02T15:41:31.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.206+0000 7fcfee657b80 10 client.29458 fetch_fsmap finished waiting for FSMap version 61 2024-02-02T15:41:31.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.206+0000 7fcfee657b80 10 client.29458 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:41:31.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.206+0000 7fcfee657b80 20 client.29458 populate_metadata read hostname 'smithi001' 2024-02-02T15:41:31.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.206+0000 7fcfee657b80 1 --2- 172.21.15.1:0/3846679523 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55d8c32a4370 0x55d8c32a6790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:31.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.206+0000 7fcfee657b80 4 client.29458 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:41:31.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.206+0000 7fcfee657b80 1 -- 172.21.15.1:0/3846679523 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d8c31d3e50 con 0x55d8c32a4370 2024-02-02T15:41:31.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.206+0000 7fcfd3400700 1 --2- 172.21.15.1:0/3846679523 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55d8c32a4370 0x55d8c32a6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:31.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.207+0000 7fcfc1600700 10 client.29458 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:41:31.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.207+0000 7fcfd3400700 1 --2- 172.21.15.1:0/3846679523 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55d8c32a4370 0x55d8c32a6790 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d3cc3794b986fbc server_cookie=4fc76ca11c3d21f8 in_seq=0 out_seq=0 2024-02-02T15:41:31.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.208+0000 7fcfc1600700 1 -- 172.21.15.1:0/3846679523 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55d8c31d3e50 con 0x55d8c32a4370 2024-02-02T15:41:31.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.209+0000 7fcfc1600700 10 client.29458 handle_command_reply: tid=0 2024-02-02T15:41:31.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.209+0000 7fcfee657b80 1 client.29458 shutdown 2024-02-02T15:41:31.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.209+0000 7fcfee657b80 1 -- 172.21.15.1:0/3846679523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d8c3289090 msgr2=0x55d8c3294190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:41:31.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.209+0000 7fcfee657b80 1 --2- 172.21.15.1:0/3846679523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d8c3289090 0x55d8c3294190 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4010470 tx=0x7fcfc4010780 comp rx=0 tx=0).stop 2024-02-02T15:41:31.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.209+0000 7fcfee657b80 1 -- 172.21.15.1:0/3846679523 shutdown_connections 2024-02-02T15:41:31.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.209+0000 7fcfee657b80 1 --2- 172.21.15.1:0/3846679523 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55d8c32a4370 0x55d8c32a6790 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:31.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.209+0000 7fcfee657b80 1 --2- 172.21.15.1:0/3846679523 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d8c3294860 0x55d8c3291970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:31.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.209+0000 7fcfee657b80 1 --2- 172.21.15.1:0/3846679523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d8c3289090 0x55d8c3294190 secure :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4010470 tx=0x7fcfc4010780 comp rx=0 tx=0).stop 2024-02-02T15:41:31.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.209+0000 7fcfee657b80 1 --2- 172.21.15.1:0/3846679523 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d8c32899b0 0x55d8c3291410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:31.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.209+0000 7fcfee657b80 1 -- 172.21.15.1:0/3846679523 >> 172.21.15.1:0/3846679523 conn(0x55d8c31e8e30 msgr2=0x55d8c3285020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:41:31.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.209+0000 7fcfc1600700 0 client.29458 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:41:31.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.210+0000 7fcfee657b80 1 -- 172.21.15.1:0/3846679523 shutdown_connections 2024-02-02T15:41:31.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.210+0000 7fcfee657b80 1 -- 172.21.15.1:0/3846679523 wait complete. 2024-02-02T15:41:31.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.210+0000 7fcfee657b80 20 client.29458 trim_cache size 0 max 16384 2024-02-02T15:41:31.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.233+0000 7fcfee657b80 1 Processor -- start 2024-02-02T15:41:31.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.233+0000 7fcfee657b80 1 -- start start 2024-02-02T15:41:31.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.234+0000 7fcfee657b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d8c32899b0 0x55d8c3293cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:31.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.234+0000 7fcfee657b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d8c3294860 0x55d8c328e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:31.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.234+0000 7fcfee657b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d8c328ce40 0x55d8c328ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:31.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.234+0000 7fcfee657b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8c32a0d30 con 0x55d8c328ce40 2024-02-02T15:41:31.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.234+0000 7fcfee657b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8c305e720 con 0x55d8c32899b0 2024-02-02T15:41:31.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.234+0000 7fcfee657b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8c327f580 con 0x55d8c3294860 2024-02-02T15:41:31.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.234+0000 7fcfd3e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d8c32899b0 0x55d8c3293cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:31.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.234+0000 7fcfd3400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d8c3294860 0x55d8c328e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:31.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.234+0000 7fcfd3400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d8c3294860 0x55d8c328e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:35694/0 (socket says 172.21.15.1:35694) 2024-02-02T15:41:31.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.234+0000 7fcfd3400700 1 -- 172.21.15.1:0/1138907751 learned_addr learned my addr 172.21.15.1:0/1138907751 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:41:31.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.234+0000 7fcfd3e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d8c32899b0 0x55d8c3293cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59630/0 (socket says 172.21.15.1:59630) 2024-02-02T15:41:31.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.234+0000 7fcfd8c00700 1 --2- 172.21.15.1:0/1138907751 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d8c328ce40 0x55d8c328ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:31.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.235+0000 7fcfd3e00700 1 -- 172.21.15.1:0/1138907751 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d8c3294860 msgr2=0x55d8c328e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:41:31.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.235+0000 7fcfd3e00700 1 --2- 172.21.15.1:0/1138907751 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d8c3294860 0x55d8c328e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:31.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.235+0000 7fcfd3e00700 1 -- 172.21.15.1:0/1138907751 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d8c328ce40 msgr2=0x55d8c328ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:41:31.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.235+0000 7fcfd3e00700 1 --2- 172.21.15.1:0/1138907751 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d8c328ce40 0x55d8c328ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:31.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.235+0000 7fcfd3e00700 1 -- 172.21.15.1:0/1138907751 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d8c31a0f20 con 0x55d8c32899b0 2024-02-02T15:41:31.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.235+0000 7fcfd3400700 1 --2- 172.21.15.1:0/1138907751 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d8c3294860 0x55d8c328e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:41:31.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.235+0000 7fcfd3e00700 1 --2- 172.21.15.1:0/1138907751 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d8c32899b0 0x55d8c3293cc0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4011010 tx=0x7fcfc40096f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8e5f9c7214f1b9d server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:41:31.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.235+0000 7fcfc9c00700 1 -- 172.21.15.1:0/1138907751 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc402c070 con 0x55d8c32899b0 2024-02-02T15:41:31.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.235+0000 7fcfc9c00700 1 -- 172.21.15.1:0/1138907751 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcfc4013580 con 0x55d8c32899b0 2024-02-02T15:41:31.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.235+0000 7fcfc9c00700 1 -- 172.21.15.1:0/1138907751 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc401c3b0 con 0x55d8c32899b0 2024-02-02T15:41:31.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.236+0000 7fcfee657b80 1 -- 172.21.15.1:0/1138907751 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d8c32899b0 msgr2=0x55d8c3293cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:41:31.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.236+0000 7fcfee657b80 1 --2- 172.21.15.1:0/1138907751 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d8c32899b0 0x55d8c3293cc0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4011010 tx=0x7fcfc40096f0 comp rx=0 tx=0).stop 2024-02-02T15:41:31.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.236+0000 7fcfee657b80 1 -- 172.21.15.1:0/1138907751 shutdown_connections 2024-02-02T15:41:31.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.236+0000 7fcfee657b80 1 --2- 172.21.15.1:0/1138907751 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d8c328ce40 0x55d8c328ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:31.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.236+0000 7fcfee657b80 1 --2- 172.21.15.1:0/1138907751 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d8c32899b0 0x55d8c3293cc0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:31.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.236+0000 7fcfee657b80 1 --2- 172.21.15.1:0/1138907751 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d8c3294860 0x55d8c328e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:31.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.236+0000 7fcfee657b80 1 -- 172.21.15.1:0/1138907751 >> 172.21.15.1:0/1138907751 conn(0x55d8c31e8e30 msgr2=0x55d8c329fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:41:31.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.237+0000 7fcfee657b80 1 -- 172.21.15.1:0/1138907751 shutdown_connections 2024-02-02T15:41:31.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.237+0000 7fcfee657b80 1 -- 172.21.15.1:0/1138907751 wait complete. 2024-02-02T15:41:31.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.237+0000 7fcfee657b80 1 -- start start 2024-02-02T15:41:31.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.237+0000 7fcfee657b80 1 Processor -- start 2024-02-02T15:41:31.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.238+0000 7fcfee657b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d8c32899b0 0x55d8c329ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:31.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.238+0000 7fcfee657b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d8c328ce40 0x55d8c329b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:31.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.238+0000 7fcfee657b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d8c3294860 0x55d8c3298030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:31.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.238+0000 7fcfee657b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8c3298650 con 0x55d8c328ce40 2024-02-02T15:41:31.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.238+0000 7fcfee657b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8c32987c0 con 0x55d8c3294860 2024-02-02T15:41:31.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.238+0000 7fcfee657b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8c3298980 con 0x55d8c32899b0 2024-02-02T15:41:31.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.238+0000 7fcfee657b80 10 client.0 _ll_register_callbacks cb 0x55d8c3178310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:41:31.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.239+0000 7fcfd3e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d8c32899b0 0x55d8c329ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:31.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.239+0000 7fcfd3400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d8c328ce40 0x55d8c329b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:31.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.239+0000 7fcfd8c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d8c3294860 0x55d8c3298030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:31.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.239+0000 7fcfd3e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d8c32899b0 0x55d8c329ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:35702/0 (socket says 172.21.15.1:35702) 2024-02-02T15:41:31.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.239+0000 7fcfd3e00700 1 -- 172.21.15.1:0/3363606145 learned_addr learned my addr 172.21.15.1:0/3363606145 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:41:31.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.239+0000 7fcfd8c00700 1 -- 172.21.15.1:0/3363606145 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d8c32899b0 msgr2=0x55d8c329ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:41:31.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.239+0000 7fcfd8c00700 1 --2- 172.21.15.1:0/3363606145 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d8c32899b0 0x55d8c329ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:31.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.239+0000 7fcfd8c00700 1 -- 172.21.15.1:0/3363606145 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d8c328ce40 msgr2=0x55d8c329b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:41:31.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.239+0000 7fcfd8c00700 1 --2- 172.21.15.1:0/3363606145 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d8c328ce40 0x55d8c329b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:31.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.239+0000 7fcfd8c00700 1 -- 172.21.15.1:0/3363606145 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d8c305c430 con 0x55d8c3294860 2024-02-02T15:41:31.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.239+0000 7fcfd8c00700 1 --2- 172.21.15.1:0/3363606145 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d8c3294860 0x55d8c3298030 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc00b780 tx=0x7fcfcc00b3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4394e248b909dd1 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:41:31.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.239+0000 7fcfc1600700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:41:31.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.240+0000 7fcfc1600700 1 -- 172.21.15.1:0/3363606145 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc00e040 con 0x55d8c3294860 2024-02-02T15:41:31.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.240+0000 7fcfc1600700 1 -- 172.21.15.1:0/3363606145 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fcfcc020d80 con 0x55d8c3294860 2024-02-02T15:41:31.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.240+0000 7fcfee657b80 1 -- 172.21.15.1:0/3363606145 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcfbc003e90 con 0x55d8c3294860 2024-02-02T15:41:31.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.240+0000 7fcfc1600700 1 -- 172.21.15.1:0/3363606145 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcfcc02c450 con 0x55d8c3294860 2024-02-02T15:41:31.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.240+0000 7fcfee657b80 10 client.29492 fetch_fsmap learned FSMap version 61 2024-02-02T15:41:31.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.240+0000 7fcfee657b80 1 -- 172.21.15.1:0/3363606145 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x55d8c3008be0 con 0x55d8c3294860 2024-02-02T15:41:31.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.240+0000 7fcfc1600700 1 -- 172.21.15.1:0/3363606145 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcfbc003e90 con 0x55d8c3294860 2024-02-02T15:41:31.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.240+0000 7fcfc1600700 1 -- 172.21.15.1:0/3363606145 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 61) v1 ==== 7550+0+0 (secure 0 0 0) 0x7fcfcc0051e0 con 0x55d8c3294860 2024-02-02T15:41:31.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.241+0000 7fcfee657b80 10 client.29492 fetch_fsmap finished waiting for FSMap version 61 2024-02-02T15:41:31.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.241+0000 7fcfee657b80 10 client.29492 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:41:31.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.241+0000 7fcfee657b80 20 client.29492 populate_metadata read hostname 'smithi001' 2024-02-02T15:41:31.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.241+0000 7fcfee657b80 1 --2- 172.21.15.1:0/3363606145 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55d8c3300800 0x55d8c3302c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:31.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.241+0000 7fcfee657b80 4 client.29492 mds_command: new command op to 24557 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-02T15:41:31.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.241+0000 7fcfee657b80 1 -- 172.21.15.1:0/3363606145 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55d8c3199fd0 con 0x55d8c3300800 2024-02-02T15:41:31.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.241+0000 7fcfd3e00700 1 --2- 172.21.15.1:0/3363606145 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55d8c3300800 0x55d8c3302c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:31.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.242+0000 7fcfc1600700 10 client.29492 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:41:31.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.242+0000 7fcfd3e00700 1 --2- 172.21.15.1:0/3363606145 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55d8c3300800 0x55d8c3302c20 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec631eff0d873752 server_cookie=61d5a67b1bf4bd27 in_seq=0 out_seq=0 2024-02-02T15:41:31.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.243+0000 7fcfc1600700 1 -- 172.21.15.1:0/3363606145 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55d8c3199fd0 con 0x55d8c3300800 2024-02-02T15:41:31.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.243+0000 7fcfc1600700 10 client.29492 handle_command_reply: tid=0 2024-02-02T15:41:31.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.243+0000 7fcfee657b80 1 client.29492 shutdown 2024-02-02T15:41:31.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.243+0000 7fcfee657b80 1 -- 172.21.15.1:0/3363606145 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d8c3294860 msgr2=0x55d8c3298030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:41:31.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.243+0000 7fcfee657b80 1 --2- 172.21.15.1:0/3363606145 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d8c3294860 0x55d8c3298030 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc00b780 tx=0x7fcfcc00b3b0 comp rx=0 tx=0).stop 2024-02-02T15:41:31.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.244+0000 7fcfee657b80 1 -- 172.21.15.1:0/3363606145 shutdown_connections 2024-02-02T15:41:31.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.244+0000 7fcfee657b80 1 --2- 172.21.15.1:0/3363606145 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55d8c3300800 0x55d8c3302c20 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:31.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.244+0000 7fcfee657b80 1 --2- 172.21.15.1:0/3363606145 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d8c3294860 0x55d8c3298030 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:31.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.244+0000 7fcfee657b80 1 --2- 172.21.15.1:0/3363606145 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d8c32899b0 0x55d8c329ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:31.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.244+0000 7fcfee657b80 1 --2- 172.21.15.1:0/3363606145 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d8c328ce40 0x55d8c329b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:31.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.244+0000 7fcfee657b80 1 -- 172.21.15.1:0/3363606145 >> 172.21.15.1:0/3363606145 conn(0x55d8c31e8e30 msgr2=0x55d8c328fbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:41:31.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.244+0000 7fcfc1600700 0 client.29492 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:41:31.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.244+0000 7fcfee657b80 1 -- 172.21.15.1:0/3363606145 shutdown_connections 2024-02-02T15:41:31.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.244+0000 7fcfee657b80 1 -- 172.21.15.1:0/3363606145 wait complete. 2024-02-02T15:41:31.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.244+0000 7fcfee657b80 20 client.29492 trim_cache size 0 max 16384 2024-02-02T15:41:31.245 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:41:31.246 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2024-02-02T15:41:31.246 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "8e44cb9e-b6f2-4e86-84f7-b62489d3cde9", 2024-02-02T15:41:31.246 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2024-02-02T15:41:31.246 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:41:31.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.245+0000 7fcfdac00700 1 -- 172.21.15.1:0/4235346312 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fcfb40a06f0 msgr2=0x7fcfb40a2bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:41:31.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.245+0000 7fcfdac00700 1 --2- 172.21.15.1:0/4235346312 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fcfb40a06f0 0x7fcfb40a2bb0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc0062a0 tx=0x7fcfcc00bd90 comp rx=0 tx=0).stop 2024-02-02T15:41:31.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.245+0000 7fcfdac00700 1 -- 172.21.15.1:0/4235346312 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcfd4073a10 msgr2=0x7fcfd4199790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:41:31.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.245+0000 7fcfdac00700 1 --2- 172.21.15.1:0/4235346312 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcfd4073a10 0x7fcfd4199790 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc00eb30 tx=0x7fcfbc00ee40 comp rx=0 tx=0).stop 2024-02-02T15:41:31.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.245+0000 7fcfdac00700 1 -- 172.21.15.1:0/4235346312 shutdown_connections 2024-02-02T15:41:31.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.245+0000 7fcfdac00700 1 --2- 172.21.15.1:0/4235346312 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fcfb40a06f0 0x7fcfb40a2bb0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:31.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.245+0000 7fcfdac00700 1 --2- 172.21.15.1:0/4235346312 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fcfd410d650 0x7fcfd419dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:31.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.245+0000 7fcfdac00700 1 --2- 172.21.15.1:0/4235346312 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcfd40730f0 0x7fcfd4199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:31.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.245+0000 7fcfdac00700 1 --2- 172.21.15.1:0/4235346312 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fcfd4073a10 0x7fcfd4199790 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:31.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.245+0000 7fcfdac00700 1 -- 172.21.15.1:0/4235346312 >> 172.21.15.1:0/4235346312 conn(0x7fcfd40fc840 msgr2=0x7fcfd41050b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:41:31.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.245+0000 7fcfdac00700 1 -- 172.21.15.1:0/4235346312 shutdown_connections 2024-02-02T15:41:31.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.246+0000 7fcfdac00700 1 -- 172.21.15.1:0/4235346312 wait complete. 2024-02-02T15:41:31.267 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-02T15:41:31.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.559+0000 7f9ba2400700 1 -- 172.21.15.1:0/60224831 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b9c107600 msgr2=0x7f9b9c1079e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:41:31.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.559+0000 7f9ba2400700 1 --2- 172.21.15.1:0/60224831 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b9c107600 0x7f9b9c1079e0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c0098f0 tx=0x7f9b8c009c00 comp rx=0 tx=0).stop 2024-02-02T15:41:31.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.559+0000 7f9ba2400700 1 -- 172.21.15.1:0/60224831 shutdown_connections 2024-02-02T15:41:31.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.559+0000 7f9ba2400700 1 --2- 172.21.15.1:0/60224831 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9b9c10c720 0x7f9b9c1142a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:31.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.559+0000 7f9ba2400700 1 --2- 172.21.15.1:0/60224831 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b9c107600 0x7f9b9c1079e0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:31.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.559+0000 7f9ba2400700 1 --2- 172.21.15.1:0/60224831 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b9c107fb0 0x7f9b9c10bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:31.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.559+0000 7f9ba2400700 1 -- 172.21.15.1:0/60224831 >> 172.21.15.1:0/60224831 conn(0x7f9b9c076b80 msgr2=0x7f9b9c105b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:41:31.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.559+0000 7f9ba2400700 1 -- 172.21.15.1:0/60224831 shutdown_connections 2024-02-02T15:41:31.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.559+0000 7f9ba2400700 1 -- 172.21.15.1:0/60224831 wait complete. 2024-02-02T15:41:31.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.560+0000 7f9ba2400700 1 Processor -- start 2024-02-02T15:41:31.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.560+0000 7f9ba2400700 1 -- start start 2024-02-02T15:41:31.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.561+0000 7f9ba2400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b9c107600 0x7f9b9c19d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:31.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.561+0000 7f9ba2400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9b9c10c720 0x7f9b9c19da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:31.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.561+0000 7f9ba2400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b9c1a1e70 0x7f9b9c1a22f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:31.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.561+0000 7f9ba2400700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9c1a28f0 con 0x7f9b9c10c720 2024-02-02T15:41:31.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.561+0000 7f9ba2400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9c1a2a60 con 0x7f9b9c1a1e70 2024-02-02T15:41:31.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.561+0000 7f9ba2400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9c1a2c20 con 0x7f9b9c107600 2024-02-02T15:41:31.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.561+0000 7f9b9b400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b9c107600 0x7f9b9c19d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:31.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.561+0000 7f9b9aa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9b9c10c720 0x7f9b9c19da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:31.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.561+0000 7f9b9be00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b9c1a1e70 0x7f9b9c1a22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:31.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.561+0000 7f9b9be00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b9c1a1e70 0x7f9b9c1a22f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59650/0 (socket says 172.21.15.1:59650) 2024-02-02T15:41:31.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.561+0000 7f9b9be00700 1 -- 172.21.15.1:0/640518120 learned_addr learned my addr 172.21.15.1:0/640518120 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:41:31.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.561+0000 7f9b9aa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9b9c10c720 0x7f9b9c19da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33490/0 (socket says 172.21.15.1:33490) 2024-02-02T15:41:31.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.561+0000 7f9b9b400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b9c107600 0x7f9b9c19d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:35734/0 (socket says 172.21.15.1:35734) 2024-02-02T15:41:31.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.561+0000 7f9b9be00700 1 -- 172.21.15.1:0/640518120 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b9c107600 msgr2=0x7f9b9c19d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:41:31.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.561+0000 7f9b9be00700 1 --2- 172.21.15.1:0/640518120 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b9c107600 0x7f9b9c19d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:31.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.561+0000 7f9b9be00700 1 -- 172.21.15.1:0/640518120 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9b9c10c720 msgr2=0x7f9b9c19da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:41:31.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.561+0000 7f9b9be00700 1 --2- 172.21.15.1:0/640518120 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9b9c10c720 0x7f9b9c19da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:31.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.561+0000 7f9b9be00700 1 -- 172.21.15.1:0/640518120 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b8c009520 con 0x7f9b9c1a1e70 2024-02-02T15:41:31.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.561+0000 7f9b9aa00700 1 --2- 172.21.15.1:0/640518120 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9b9c10c720 0x7f9b9c19da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:41:31.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.562+0000 7f9b9be00700 1 --2- 172.21.15.1:0/640518120 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b9c1a1e70 0x7f9b9c1a22f0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f9b9400ef30 tx=0x7f9b9400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:41:31.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.562+0000 7f9b93e00700 1 -- 172.21.15.1:0/640518120 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b94018070 con 0x7f9b9c1a1e70 2024-02-02T15:41:31.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.562+0000 7f9b93e00700 1 -- 172.21.15.1:0/640518120 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b9400f040 con 0x7f9b9c1a1e70 2024-02-02T15:41:31.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.562+0000 7f9b93e00700 1 -- 172.21.15.1:0/640518120 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b9400ccf0 con 0x7f9b9c1a1e70 2024-02-02T15:41:31.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.562+0000 7f9ba2400700 1 -- 172.21.15.1:0/640518120 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b9c1aa120 con 0x7f9b9c1a1e70 2024-02-02T15:41:31.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.562+0000 7f9ba2400700 1 -- 172.21.15.1:0/640518120 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b9c1aa5b0 con 0x7f9b9c1a1e70 2024-02-02T15:41:31.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.566+0000 7f9b93e00700 1 -- 172.21.15.1:0/640518120 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f9b94003a00 con 0x7f9b9c1a1e70 2024-02-02T15:41:31.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.568+0000 7f9b93e00700 1 --2- 172.21.15.1:0/640518120 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f9b7c0a0710 0x7f9b7c0a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:31.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.568+0000 7f9b93e00700 1 -- 172.21.15.1:0/640518120 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f9b940cbbe0 con 0x7f9b9c1a1e70 2024-02-02T15:41:31.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.569+0000 7f9b9b400700 1 --2- 172.21.15.1:0/640518120 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f9b7c0a0710 0x7f9b7c0a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:31.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.569+0000 7f9bb5fbdb80 1 Processor -- start 2024-02-02T15:41:31.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.570+0000 7f9bb5fbdb80 1 -- start start 2024-02-02T15:41:31.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.571+0000 7f9bb5fbdb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56393e779010 0x56393e7793f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:31.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.571+0000 7f9bb5fbdb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56393e779930 0x56393e7842a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:31.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.571+0000 7f9bb5fbdb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56393e7847e0 0x56393e786bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:31.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.571+0000 7f9bb5fbdb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56393e6ed4c0 con 0x56393e779930 2024-02-02T15:41:31.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.571+0000 7f9bb5fbdb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56393e6d1aa0 con 0x56393e7847e0 2024-02-02T15:41:31.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.571+0000 7f9bb5fbdb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56393e6cf460 con 0x56393e779010 2024-02-02T15:41:31.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.571+0000 7f9b9be00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56393e7847e0 0x56393e786bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:31.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.571+0000 7f9b9be00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56393e7847e0 0x56393e786bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59654/0 (socket says 172.21.15.1:59654) 2024-02-02T15:41:31.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.571+0000 7f9b9be00700 1 -- 172.21.15.1:0/2977490280 learned_addr learned my addr 172.21.15.1:0/2977490280 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:41:31.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.571+0000 7f9b9aa00700 1 --2- 172.21.15.1:0/2977490280 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56393e779930 0x56393e7842a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:31.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.571+0000 7f9b9b400700 1 --2- 172.21.15.1:0/640518120 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f9b7c0a0710 0x7f9b7c0a2bd0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c009fd0 tx=0x7f9b8c019040 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:41:31.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.571+0000 7f9b9be00700 1 -- 172.21.15.1:0/2977490280 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56393e779010 msgr2=0x56393e7793f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-02T15:41:31.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.571+0000 7f9b9be00700 1 --2- 172.21.15.1:0/2977490280 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56393e779010 0x56393e7793f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:31.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.571+0000 7f9b9be00700 1 -- 172.21.15.1:0/2977490280 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56393e779930 msgr2=0x56393e7842a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:41:31.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.571+0000 7f9b9be00700 1 --2- 172.21.15.1:0/2977490280 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56393e779930 0x56393e7842a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:31.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.571+0000 7f9b9be00700 1 -- 172.21.15.1:0/2977490280 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56393e54c3b0 con 0x56393e7847e0 2024-02-02T15:41:31.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.572+0000 7f9b9be00700 1 --2- 172.21.15.1:0/2977490280 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56393e7847e0 0x56393e786bd0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f9b94098d20 tx=0x7f9b940989a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c012b5bc186b11da server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:41:31.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.572+0000 7f9b90c00700 1 -- 172.21.15.1:0/2977490280 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b9400ce70 con 0x56393e7847e0 2024-02-02T15:41:31.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.572+0000 7f9b90c00700 1 -- 172.21.15.1:0/2977490280 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b94099490 con 0x56393e7847e0 2024-02-02T15:41:31.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.572+0000 7f9b90c00700 1 -- 172.21.15.1:0/2977490280 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b9401b320 con 0x56393e7847e0 2024-02-02T15:41:31.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.573+0000 7f9bb5fbdb80 1 -- 172.21.15.1:0/2977490280 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56393e7847e0 msgr2=0x56393e786bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:41:31.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.573+0000 7f9bb5fbdb80 1 --2- 172.21.15.1:0/2977490280 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56393e7847e0 0x56393e786bd0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f9b94098d20 tx=0x7f9b940989a0 comp rx=0 tx=0).stop 2024-02-02T15:41:31.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.573+0000 7f9bb5fbdb80 1 -- 172.21.15.1:0/2977490280 shutdown_connections 2024-02-02T15:41:31.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.573+0000 7f9bb5fbdb80 1 --2- 172.21.15.1:0/2977490280 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56393e7847e0 0x56393e786bd0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:31.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.573+0000 7f9bb5fbdb80 1 --2- 172.21.15.1:0/2977490280 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56393e779010 0x56393e7793f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:31.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.573+0000 7f9bb5fbdb80 1 --2- 172.21.15.1:0/2977490280 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56393e779930 0x56393e7842a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:31.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.573+0000 7f9bb5fbdb80 1 -- 172.21.15.1:0/2977490280 >> 172.21.15.1:0/2977490280 conn(0x56393e6d8db0 msgr2=0x56393e7753b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:41:31.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.574+0000 7f9bb5fbdb80 1 -- 172.21.15.1:0/2977490280 shutdown_connections 2024-02-02T15:41:31.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.574+0000 7f9bb5fbdb80 1 -- 172.21.15.1:0/2977490280 wait complete. 2024-02-02T15:41:31.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.574+0000 7f9bb5fbdb80 1 -- start start 2024-02-02T15:41:31.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.575+0000 7f9bb5fbdb80 1 Processor -- start 2024-02-02T15:41:31.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.575+0000 7f9bb5fbdb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56393e779010 0x56393e7842a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:31.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.575+0000 7f9bb5fbdb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56393e779930 0x56393e781520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:31.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.575+0000 7f9bb5fbdb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56393e7847e0 0x56393e781c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:31.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.575+0000 7f9bb5fbdb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56393e6cad50 con 0x56393e779010 2024-02-02T15:41:31.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.575+0000 7f9bb5fbdb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56393e666d80 con 0x56393e7847e0 2024-02-02T15:41:31.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.575+0000 7f9bb5fbdb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56393e666a20 con 0x56393e779930 2024-02-02T15:41:31.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.575+0000 7f9bb5fbdb80 10 client.0 _ll_register_callbacks cb 0x56393e668290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:41:31.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.576+0000 7f9b9b400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56393e779010 0x56393e7842a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:31.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.576+0000 7f9b9b400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56393e779010 0x56393e7842a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33508/0 (socket says 172.21.15.1:33508) 2024-02-02T15:41:31.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.576+0000 7f9b9b400700 1 -- 172.21.15.1:0/2234072676 learned_addr learned my addr 172.21.15.1:0/2234072676 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:41:31.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.576+0000 7f9b9be00700 1 --2- 172.21.15.1:0/2234072676 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56393e7847e0 0x56393e781c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:31.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.576+0000 7f9b9b400700 1 -- 172.21.15.1:0/2234072676 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56393e779930 msgr2=0x56393e781520 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-02T15:41:31.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.576+0000 7f9b9b400700 1 --2- 172.21.15.1:0/2234072676 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56393e779930 0x56393e781520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:31.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.576+0000 7f9b9b400700 1 -- 172.21.15.1:0/2234072676 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56393e7847e0 msgr2=0x56393e781c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:41:31.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.576+0000 7f9b9b400700 1 --2- 172.21.15.1:0/2234072676 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56393e7847e0 0x56393e781c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:31.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.576+0000 7f9b9b400700 1 -- 172.21.15.1:0/2234072676 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b9c1aa5b0 con 0x56393e779010 2024-02-02T15:41:31.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.576+0000 7f9b9be00700 1 --2- 172.21.15.1:0/2234072676 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56393e7847e0 0x56393e781c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:41:31.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.577+0000 7f9b9b400700 1 --2- 172.21.15.1:0/2234072676 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56393e779010 0x56393e7842a0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c016000 tx=0x7f9b8c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a6d64cf7e37ac04 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:41:31.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.577+0000 7f9b89600700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:41:31.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.578+0000 7f9b89600700 1 -- 172.21.15.1:0/2234072676 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c005040 con 0x56393e779010 2024-02-02T15:41:31.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.578+0000 7f9bb5fbdb80 1 -- 172.21.15.1:0/2234072676 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56393e76d0f0 con 0x56393e779010 2024-02-02T15:41:31.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.578+0000 7f9b89600700 1 -- 172.21.15.1:0/2234072676 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f9b8c0051a0 con 0x56393e779010 2024-02-02T15:41:31.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.578+0000 7f9b89600700 1 -- 172.21.15.1:0/2234072676 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b8c005360 con 0x56393e779010 2024-02-02T15:41:31.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.578+0000 7f9bb5fbdb80 10 client.19353 fetch_fsmap learned FSMap version 61 2024-02-02T15:41:31.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.578+0000 7f9bb5fbdb80 1 -- 172.21.15.1:0/2234072676 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x56393e4f8b60 con 0x56393e779010 2024-02-02T15:41:31.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.579+0000 7f9b89600700 1 -- 172.21.15.1:0/2234072676 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b8c038020 con 0x56393e779010 2024-02-02T15:41:31.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.579+0000 7f9b89600700 1 -- 172.21.15.1:0/2234072676 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 61) v1 ==== 7550+0+0 (secure 0 0 0) 0x7f9b8c02f440 con 0x56393e779010 2024-02-02T15:41:31.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.579+0000 7f9bb5fbdb80 10 client.19353 fetch_fsmap finished waiting for FSMap version 61 2024-02-02T15:41:31.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.579+0000 7f9bb5fbdb80 10 client.19353 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:41:31.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.579+0000 7f9bb5fbdb80 20 client.19353 populate_metadata read hostname 'smithi001' 2024-02-02T15:41:31.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.580+0000 7f9bb5fbdb80 1 --2- 172.21.15.1:0/2234072676 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56393e7a52b0 0x56393e7a76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:31.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.580+0000 7f9bb5fbdb80 4 client.19353 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:41:31.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.580+0000 7f9bb5fbdb80 1 -- 172.21.15.1:0/2234072676 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56393e6c3dd0 con 0x56393e7a52b0 2024-02-02T15:41:31.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.580+0000 7f9b9aa00700 1 --2- 172.21.15.1:0/2234072676 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56393e7a52b0 0x56393e7a76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:31.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.581+0000 7f9b9aa00700 1 --2- 172.21.15.1:0/2234072676 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56393e7a52b0 0x56393e7a76f0 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99a066ee7d0c3591 server_cookie=579992ce68857ba5 in_seq=0 out_seq=0 2024-02-02T15:41:31.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.581+0000 7f9b89600700 10 client.19353 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:41:31.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.581+0000 7f9b89600700 1 -- 172.21.15.1:0/2234072676 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x56393e6c3dd0 con 0x56393e7a52b0 2024-02-02T15:41:31.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.581+0000 7f9b89600700 10 client.19353 handle_command_reply: tid=0 2024-02-02T15:41:31.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.581+0000 7f9bb5fbdb80 1 client.19353 shutdown 2024-02-02T15:41:31.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.582+0000 7f9bb5fbdb80 1 -- 172.21.15.1:0/2234072676 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56393e779010 msgr2=0x56393e7842a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:41:31.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.582+0000 7f9bb5fbdb80 1 --2- 172.21.15.1:0/2234072676 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56393e779010 0x56393e7842a0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c016000 tx=0x7f9b8c009260 comp rx=0 tx=0).stop 2024-02-02T15:41:31.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.582+0000 7f9bb5fbdb80 1 -- 172.21.15.1:0/2234072676 shutdown_connections 2024-02-02T15:41:31.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.582+0000 7f9bb5fbdb80 1 --2- 172.21.15.1:0/2234072676 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56393e7a52b0 0x56393e7a76f0 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:31.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.582+0000 7f9bb5fbdb80 1 --2- 172.21.15.1:0/2234072676 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56393e7847e0 0x56393e781c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:31.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.582+0000 7f9bb5fbdb80 1 --2- 172.21.15.1:0/2234072676 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56393e779010 0x56393e7842a0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:31.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.582+0000 7f9bb5fbdb80 1 --2- 172.21.15.1:0/2234072676 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56393e779930 0x56393e781520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:31.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.582+0000 7f9bb5fbdb80 1 -- 172.21.15.1:0/2234072676 >> 172.21.15.1:0/2234072676 conn(0x56393e6d8db0 msgr2=0x56393e774fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:41:31.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.582+0000 7f9b89600700 0 client.19353 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:41:31.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.582+0000 7f9bb5fbdb80 1 -- 172.21.15.1:0/2234072676 shutdown_connections 2024-02-02T15:41:31.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.583+0000 7f9bb5fbdb80 1 -- 172.21.15.1:0/2234072676 wait complete. 2024-02-02T15:41:31.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.583+0000 7f9bb5fbdb80 20 client.19353 trim_cache size 0 max 16384 2024-02-02T15:41:31.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.607+0000 7f9bb5fbdb80 1 Processor -- start 2024-02-02T15:41:31.607 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.607+0000 7f9bb5fbdb80 1 -- start start 2024-02-02T15:41:31.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.608+0000 7f9bb5fbdb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56393e779930 0x56393e77c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:31.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.608+0000 7f9bb5fbdb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56393e7847e0 0x56393e783dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:31.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.608+0000 7f9bb5fbdb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56393e784310 0x56393e780e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:31.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.608+0000 7f9bb5fbdb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56393e54e6a0 con 0x56393e779930 2024-02-02T15:41:31.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.608+0000 7f9bb5fbdb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56393e7719b0 con 0x56393e784310 2024-02-02T15:41:31.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.608+0000 7f9bb5fbdb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56393e776240 con 0x56393e7847e0 2024-02-02T15:41:31.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.608+0000 7f9b9b400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56393e779930 0x56393e77c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:31.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.608+0000 7f9b9b400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56393e779930 0x56393e77c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:33518/0 (socket says 172.21.15.1:33518) 2024-02-02T15:41:31.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.608+0000 7f9b9b400700 1 -- 172.21.15.1:0/3436074456 learned_addr learned my addr 172.21.15.1:0/3436074456 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:41:31.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.608+0000 7f9b9be00700 1 --2- 172.21.15.1:0/3436074456 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56393e784310 0x56393e780e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:31.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.608+0000 7f9b9aa00700 1 --2- 172.21.15.1:0/3436074456 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56393e7847e0 0x56393e783dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:31.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.609+0000 7f9b9b400700 1 -- 172.21.15.1:0/3436074456 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56393e7847e0 msgr2=0x56393e783dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:41:31.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.609+0000 7f9b9b400700 1 --2- 172.21.15.1:0/3436074456 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56393e7847e0 0x56393e783dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:31.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.609+0000 7f9b9b400700 1 -- 172.21.15.1:0/3436074456 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56393e784310 msgr2=0x56393e780e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:41:31.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.609+0000 7f9b9b400700 1 --2- 172.21.15.1:0/3436074456 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56393e784310 0x56393e780e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:31.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.609+0000 7f9b9b400700 1 -- 172.21.15.1:0/3436074456 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56393e690ea0 con 0x56393e779930 2024-02-02T15:41:31.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.609+0000 7f9b9b400700 1 --2- 172.21.15.1:0/3436074456 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56393e779930 0x56393e77c050 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c009810 tx=0x7f9b8c00fc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c93ca9ce3695489 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:41:31.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.611+0000 7f9b91600700 1 -- 172.21.15.1:0/3436074456 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c00b8e0 con 0x56393e779930 2024-02-02T15:41:31.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.612+0000 7f9b91600700 1 -- 172.21.15.1:0/3436074456 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b8c038030 con 0x56393e779930 2024-02-02T15:41:31.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.612+0000 7f9b91600700 1 -- 172.21.15.1:0/3436074456 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c005640 con 0x56393e779930 2024-02-02T15:41:31.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.613+0000 7f9bb5fbdb80 1 -- 172.21.15.1:0/3436074456 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56393e779930 msgr2=0x56393e77c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:41:31.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.613+0000 7f9bb5fbdb80 1 --2- 172.21.15.1:0/3436074456 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56393e779930 0x56393e77c050 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c009810 tx=0x7f9b8c00fc30 comp rx=0 tx=0).stop 2024-02-02T15:41:31.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.613+0000 7f9bb5fbdb80 1 -- 172.21.15.1:0/3436074456 shutdown_connections 2024-02-02T15:41:31.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.613+0000 7f9bb5fbdb80 1 --2- 172.21.15.1:0/3436074456 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56393e784310 0x56393e780e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:31.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.613+0000 7f9bb5fbdb80 1 --2- 172.21.15.1:0/3436074456 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56393e779930 0x56393e77c050 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:31.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.613+0000 7f9bb5fbdb80 1 --2- 172.21.15.1:0/3436074456 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56393e7847e0 0x56393e783dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:31.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.613+0000 7f9bb5fbdb80 1 -- 172.21.15.1:0/3436074456 >> 172.21.15.1:0/3436074456 conn(0x56393e6d8db0 msgr2=0x56393e7a1560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:41:31.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.613+0000 7f9bb5fbdb80 1 -- 172.21.15.1:0/3436074456 shutdown_connections 2024-02-02T15:41:31.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.613+0000 7f9bb5fbdb80 1 -- 172.21.15.1:0/3436074456 wait complete. 2024-02-02T15:41:31.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.613+0000 7f9bb5fbdb80 1 -- start start 2024-02-02T15:41:31.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.614+0000 7f9bb5fbdb80 1 Processor -- start 2024-02-02T15:41:31.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.614+0000 7f9bb5fbdb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56393e779930 0x56393e78b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:31.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.614+0000 7f9bb5fbdb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56393e784310 0x56393e7893d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:31.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.614+0000 7f9bb5fbdb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56393e7847e0 0x56393e789910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:31.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.614+0000 7f9bb5fbdb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56393e789f30 con 0x56393e7847e0 2024-02-02T15:41:31.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.614+0000 7f9bb5fbdb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56393e78a070 con 0x56393e779930 2024-02-02T15:41:31.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.614+0000 7f9bb5fbdb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56393e78a230 con 0x56393e784310 2024-02-02T15:41:31.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.614+0000 7f9bb5fbdb80 10 client.0 _ll_register_callbacks cb 0x56393e668290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:41:31.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.614+0000 7f9b9b400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56393e779930 0x56393e78b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:31.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.614+0000 7f9b9be00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56393e7847e0 0x56393e789910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:31.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.614+0000 7f9b9b400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56393e779930 0x56393e78b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59676/0 (socket says 172.21.15.1:59676) 2024-02-02T15:41:31.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.614+0000 7f9b9b400700 1 -- 172.21.15.1:0/2459271563 learned_addr learned my addr 172.21.15.1:0/2459271563 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:41:31.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.615+0000 7f9b9b400700 1 -- 172.21.15.1:0/2459271563 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56393e784310 msgr2=0x56393e7893d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:41:31.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.615+0000 7f9b9b400700 1 --2- 172.21.15.1:0/2459271563 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56393e784310 0x56393e7893d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:31.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.615+0000 7f9b9b400700 1 -- 172.21.15.1:0/2459271563 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56393e7847e0 msgr2=0x56393e789910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:41:31.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.615+0000 7f9b9b400700 1 --2- 172.21.15.1:0/2459271563 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56393e7847e0 0x56393e789910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:31.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.615+0000 7f9b9b400700 1 -- 172.21.15.1:0/2459271563 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56393e4f8b60 con 0x56393e779930 2024-02-02T15:41:31.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.615+0000 7f9b9be00700 1 --2- 172.21.15.1:0/2459271563 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56393e7847e0 0x56393e789910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:41:31.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.615+0000 7f9b9b400700 1 --2- 172.21.15.1:0/2459271563 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56393e779930 0x56393e78b0a0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c009c40 tx=0x7f9b8c00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de0eb13e34f5abd5 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:41:31.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.615+0000 7f9b89600700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:41:31.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.615+0000 7f9b89600700 1 -- 172.21.15.1:0/2459271563 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c0056d0 con 0x56393e779930 2024-02-02T15:41:31.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.616+0000 7f9bb5fbdb80 1 -- 172.21.15.1:0/2459271563 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9b94099490 con 0x56393e779930 2024-02-02T15:41:31.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.616+0000 7f9b89600700 1 -- 172.21.15.1:0/2459271563 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f9b8c039080 con 0x56393e779930 2024-02-02T15:41:31.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.616+0000 7f9b89600700 1 -- 172.21.15.1:0/2459271563 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b8c02f580 con 0x56393e779930 2024-02-02T15:41:31.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.616+0000 7f9bb5fbdb80 10 client.29507 fetch_fsmap learned FSMap version 61 2024-02-02T15:41:31.617 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.616+0000 7f9bb5fbdb80 1 -- 172.21.15.1:0/2459271563 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x56393e550cd0 con 0x56393e779930 2024-02-02T15:41:31.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.616+0000 7f9b89600700 1 -- 172.21.15.1:0/2459271563 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9b94099490 con 0x56393e779930 2024-02-02T15:41:31.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.616+0000 7f9b89600700 1 -- 172.21.15.1:0/2459271563 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 61) v1 ==== 7550+0+0 (secure 0 0 0) 0x7f9b8c003d20 con 0x56393e779930 2024-02-02T15:41:31.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.616+0000 7f9bb5fbdb80 10 client.29507 fetch_fsmap finished waiting for FSMap version 61 2024-02-02T15:41:31.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.616+0000 7f9bb5fbdb80 10 client.29507 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:41:31.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.616+0000 7f9bb5fbdb80 20 client.29507 populate_metadata read hostname 'smithi001' 2024-02-02T15:41:31.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.616+0000 7f9bb5fbdb80 1 --2- 172.21.15.1:0/2459271563 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56393e7f7f50 0x56393e7fa370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:41:31.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.617+0000 7f9bb5fbdb80 4 client.29507 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2024-02-02T15:41:31.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.617+0000 7f9bb5fbdb80 1 -- 172.21.15.1:0/2459271563 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56393e689f50 con 0x56393e7f7f50 2024-02-02T15:41:31.618 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.617+0000 7f9b9aa00700 1 --2- 172.21.15.1:0/2459271563 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56393e7f7f50 0x56393e7fa370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:41:31.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.617+0000 7f9b89600700 10 client.29507 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:41:31.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.617+0000 7f9b9aa00700 1 --2- 172.21.15.1:0/2459271563 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56393e7f7f50 0x56393e7fa370 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b89503ab9c0c68c server_cookie=550deab1352009a7 in_seq=0 out_seq=0 2024-02-02T15:41:31.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.618+0000 7f9b89600700 1 -- 172.21.15.1:0/2459271563 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x56393e689f50 con 0x56393e7f7f50 2024-02-02T15:41:31.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.618+0000 7f9b89600700 10 client.29507 handle_command_reply: tid=0 2024-02-02T15:41:31.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.618+0000 7f9bb5fbdb80 1 client.29507 shutdown 2024-02-02T15:41:31.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.618+0000 7f9bb5fbdb80 1 -- 172.21.15.1:0/2459271563 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56393e779930 msgr2=0x56393e78b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:41:31.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.618+0000 7f9bb5fbdb80 1 --2- 172.21.15.1:0/2459271563 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56393e779930 0x56393e78b0a0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c009c40 tx=0x7f9b8c00ef10 comp rx=0 tx=0).stop 2024-02-02T15:41:31.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.618+0000 7f9bb5fbdb80 1 -- 172.21.15.1:0/2459271563 shutdown_connections 2024-02-02T15:41:31.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.618+0000 7f9bb5fbdb80 1 --2- 172.21.15.1:0/2459271563 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56393e7f7f50 0x56393e7fa370 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:31.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.618+0000 7f9bb5fbdb80 1 --2- 172.21.15.1:0/2459271563 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56393e7847e0 0x56393e789910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:31.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.618+0000 7f9bb5fbdb80 1 --2- 172.21.15.1:0/2459271563 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56393e779930 0x56393e78b0a0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:31.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.618+0000 7f9bb5fbdb80 1 --2- 172.21.15.1:0/2459271563 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56393e784310 0x56393e7893d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:31.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.618+0000 7f9bb5fbdb80 1 -- 172.21.15.1:0/2459271563 >> 172.21.15.1:0/2459271563 conn(0x56393e6d8db0 msgr2=0x56393e780e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:41:31.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.618+0000 7f9b89600700 0 client.29507 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:41:31.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.619+0000 7f9bb5fbdb80 1 -- 172.21.15.1:0/2459271563 shutdown_connections 2024-02-02T15:41:31.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.620+0000 7f9bb5fbdb80 1 -- 172.21.15.1:0/2459271563 wait complete. 2024-02-02T15:41:31.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.620+0000 7f9bb5fbdb80 20 client.29507 trim_cache size 0 max 16384 2024-02-02T15:41:31.621 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:41:31.621 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-02T15:41:31.621 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2024-02-02T15:41:31.621 INFO:teuthology.orchestra.run.smithi001.stdout: "8e44cb9e-b6f2-4e86-84f7-b62489d3cde9": { 2024-02-02T15:41:31.621 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2024-02-02T15:41:31.621 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "8e44cb9e-b6f2-4e86-84f7-b62489d3cde9", 2024-02-02T15:41:31.622 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,force" 2024-02-02T15:41:31.622 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:41:31.622 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:41:31.622 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:41:31.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.621+0000 7f9ba2400700 1 -- 172.21.15.1:0/640518120 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f9b7c0a0710 msgr2=0x7f9b7c0a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:41:31.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.621+0000 7f9ba2400700 1 --2- 172.21.15.1:0/640518120 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f9b7c0a0710 0x7f9b7c0a2bd0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c009fd0 tx=0x7f9b8c019040 comp rx=0 tx=0).stop 2024-02-02T15:41:31.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.621+0000 7f9ba2400700 1 -- 172.21.15.1:0/640518120 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b9c1a1e70 msgr2=0x7f9b9c1a22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:41:31.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.621+0000 7f9ba2400700 1 --2- 172.21.15.1:0/640518120 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b9c1a1e70 0x7f9b9c1a22f0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f9b9400ef30 tx=0x7f9b9400c3a0 comp rx=0 tx=0).stop 2024-02-02T15:41:31.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.621+0000 7f9ba2400700 1 -- 172.21.15.1:0/640518120 shutdown_connections 2024-02-02T15:41:31.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.621+0000 7f9ba2400700 1 --2- 172.21.15.1:0/640518120 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f9b7c0a0710 0x7f9b7c0a2bd0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:31.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.621+0000 7f9ba2400700 1 --2- 172.21.15.1:0/640518120 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b9c1a1e70 0x7f9b9c1a22f0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:31.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.621+0000 7f9ba2400700 1 --2- 172.21.15.1:0/640518120 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9b9c107600 0x7f9b9c19d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:31.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.621+0000 7f9ba2400700 1 --2- 172.21.15.1:0/640518120 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9b9c10c720 0x7f9b9c19da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:41:31.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.621+0000 7f9ba2400700 1 -- 172.21.15.1:0/640518120 >> 172.21.15.1:0/640518120 conn(0x7f9b9c076b80 msgr2=0x7f9b9c10b030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:41:31.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.621+0000 7f9ba2400700 1 -- 172.21.15.1:0/640518120 shutdown_connections 2024-02-02T15:41:31.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:41:31.621+0000 7f9ba2400700 1 -- 172.21.15.1:0/640518120 wait complete. 2024-02-02T15:41:31.647 INFO:tasks.cephfs.filesystem:scrub status for tag:8e44cb9e-b6f2-4e86-84f7-b62489d3cde9 - {'path': '/', 'tag': '8e44cb9e-b6f2-4e86-84f7-b62489d3cde9', 'options': 'recursive,force'} 2024-02-02T15:41:31.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:41:31 smithi071 ceph-mon[44707]: pgmap v454: 129 pgs: 129 active+clean; 1.0 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.1 MiB/s wr, 466 op/s 2024-02-02T15:41:31.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:41:31 smithi142 ceph-mon[45728]: pgmap v454: 129 pgs: 129 active+clean; 1.0 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.1 MiB/s wr, 466 op/s 2024-02-02T15:41:31.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:41:31 smithi001 ceph-mon[35470]: pgmap v454: 129 pgs: 129 active+clean; 1.0 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.1 MiB/s wr, 466 op/s 2024-02-02T15:41:31.975 INFO:tasks.workunit.client.0.smithi001.stdout: 10 35123 32.28 MB/sec warmup 64 sec latency 53.291 ms 2024-02-02T15:41:32.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:41:32 smithi071 ceph-mon[44707]: scrub queued for path: / 2024-02-02T15:41:32.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:41:32 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:41:32.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:41:32 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:41:32.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:41:32 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:41:32.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:41:32 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:41:32.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:41:32 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:41:32.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:41:32 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:41:32.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:41:32 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:41:32.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:41:32 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:41:32.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:41:32 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:41:32.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:41:32 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:41:32.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:41:32 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:41:32.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:41:32 smithi142 ceph-mon[45728]: scrub queued for path: / 2024-02-02T15:41:32.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:41:32 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:41:32.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:41:32 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:41:32.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:41:32 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:41:32.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:41:32 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:41:32.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:41:32 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:41:32.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:41:32 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:41:32.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:41:32 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:41:32.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:41:32 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:41:32.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:41:32 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:41:32.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:41:32 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:41:32.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:41:32 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:41:32.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:41:32 smithi001 ceph-mon[35470]: scrub queued for path: / 2024-02-02T15:41:32.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:41:32 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:41:32.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:41:32 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:41:32.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:41:32 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:41:32.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:41:32 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:41:32.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:41:32 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:41:32.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:41:32 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:41:32.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:41:32 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:41:32.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:41:32 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:41:32.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:41:32 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:41:32.942 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:41:32 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:41:32.942 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:41:32 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:41:32.975 INFO:tasks.workunit.client.0.smithi001.stdout: 10 35384 32.06 MB/sec warmup 65 sec latency 505.400 ms 2024-02-02T15:41:33.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:41:33 smithi071 ceph-mon[44707]: pgmap v455: 129 pgs: 129 active+clean; 1.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 8.8 MiB/s wr, 681 op/s 2024-02-02T15:41:33.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:41:33 smithi142 ceph-mon[45728]: pgmap v455: 129 pgs: 129 active+clean; 1.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 8.8 MiB/s wr, 681 op/s 2024-02-02T15:41:33.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:41:33 smithi001 ceph-mon[35470]: pgmap v455: 129 pgs: 129 active+clean; 1.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 8.8 MiB/s wr, 681 op/s 2024-02-02T15:41:33.976 INFO:tasks.workunit.client.0.smithi001.stdout: 10 35751 31.77 MB/sec warmup 66 sec latency 92.825 ms 2024-02-02T15:41:34.979 INFO:tasks.workunit.client.0.smithi001.stdout: 10 36414 31.80 MB/sec warmup 67 sec latency 115.219 ms 2024-02-02T15:41:35.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:41:35 smithi071 ceph-mon[44707]: pgmap v456: 129 pgs: 129 active+clean; 1.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.5 MiB/s wr, 434 op/s 2024-02-02T15:41:35.924 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:41:35 smithi142 ceph-mon[45728]: pgmap v456: 129 pgs: 129 active+clean; 1.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.5 MiB/s wr, 434 op/s 2024-02-02T15:41:35.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:41:35 smithi001 ceph-mon[35470]: pgmap v456: 129 pgs: 129 active+clean; 1.1 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.5 MiB/s wr, 434 op/s 2024-02-02T15:41:35.976 INFO:tasks.workunit.client.0.smithi001.stdout: 10 37308 32.11 MB/sec warmup 68 sec latency 36.859 ms 2024-02-02T15:41:36.977 INFO:tasks.workunit.client.0.smithi001.stdout: 10 38096 32.26 MB/sec warmup 69 sec latency 39.589 ms 2024-02-02T15:41:37.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:41:37 smithi071 ceph-mon[44707]: pgmap v457: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 10 MiB/s wr, 669 op/s 2024-02-02T15:41:37.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:41:37 smithi142 ceph-mon[45728]: pgmap v457: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 10 MiB/s wr, 669 op/s 2024-02-02T15:41:37.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:41:37 smithi001 ceph-mon[35470]: pgmap v457: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 10 MiB/s wr, 669 op/s 2024-02-02T15:41:37.977 INFO:tasks.workunit.client.0.smithi001.stdout: 10 38602 32.23 MB/sec warmup 70 sec latency 273.997 ms 2024-02-02T15:41:38.977 INFO:tasks.workunit.client.0.smithi001.stdout: 10 38844 31.93 MB/sec warmup 71 sec latency 707.032 ms 2024-02-02T15:41:39.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:41:39 smithi071 ceph-mon[44707]: pgmap v458: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 656 op/s 2024-02-02T15:41:39.926 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:41:39 smithi142 ceph-mon[45728]: pgmap v458: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 656 op/s 2024-02-02T15:41:39.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:41:39 smithi001 ceph-mon[35470]: pgmap v458: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 11 MiB/s wr, 656 op/s 2024-02-02T15:41:39.977 INFO:tasks.workunit.client.0.smithi001.stdout: 10 39259 31.71 MB/sec warmup 72 sec latency 94.669 ms 2024-02-02T15:41:40.977 INFO:tasks.workunit.client.0.smithi001.stdout: 10 39851 31.75 MB/sec warmup 73 sec latency 66.742 ms 2024-02-02T15:41:41.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:41:41 smithi071 ceph-mon[44707]: pgmap v459: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 601 op/s 2024-02-02T15:41:41.922 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:41:41 smithi142 ceph-mon[45728]: pgmap v459: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 601 op/s 2024-02-02T15:41:41.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:41:41 smithi001 ceph-mon[35470]: pgmap v459: 129 pgs: 129 active+clean; 1.1 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 601 op/s 2024-02-02T15:41:41.977 INFO:tasks.workunit.client.0.smithi001.stdout: 10 40723 31.99 MB/sec warmup 74 sec latency 49.418 ms 2024-02-02T15:41:42.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:41:42 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:41:42.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:41:42 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:41:42.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:41:42 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:41:42.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:41:42 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:41:42.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:41:42 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:41:42.922 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:41:42 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:41:42.923 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:41:42 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:41:42.923 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:41:42 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:41:42.923 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:41:42 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:41:42.923 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:41:42 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:41:42.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:41:42 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:41:42.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:41:42 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:41:42.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:41:42 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:41:42.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:41:42 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:41:42.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:41:42 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:41:42.977 INFO:tasks.workunit.client.0.smithi001.stdout: 10 41433 32.12 MB/sec warmup 75 sec latency 34.676 ms 2024-02-02T15:41:43.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:41:43 smithi071 ceph-mon[44707]: pgmap v460: 129 pgs: 129 active+clean; 1.1 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 20 MiB/s wr, 998 op/s 2024-02-02T15:41:43.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:41:43 smithi142 ceph-mon[45728]: pgmap v460: 129 pgs: 129 active+clean; 1.1 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 20 MiB/s wr, 998 op/s 2024-02-02T15:41:43.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:41:43 smithi001 ceph-mon[35470]: pgmap v460: 129 pgs: 129 active+clean; 1.1 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 20 MiB/s wr, 998 op/s 2024-02-02T15:41:43.977 INFO:tasks.workunit.client.0.smithi001.stdout: 10 42034 32.07 MB/sec warmup 76 sec latency 81.413 ms 2024-02-02T15:41:44.977 INFO:tasks.workunit.client.0.smithi001.stdout: 10 42830 32.16 MB/sec warmup 77 sec latency 55.243 ms 2024-02-02T15:41:45.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:41:45 smithi071 ceph-mon[44707]: pgmap v461: 129 pgs: 129 active+clean; 1.1 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 17 MiB/s wr, 783 op/s 2024-02-02T15:41:45.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:41:45 smithi142 ceph-mon[45728]: pgmap v461: 129 pgs: 129 active+clean; 1.1 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 17 MiB/s wr, 783 op/s 2024-02-02T15:41:45.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:41:45 smithi001 ceph-mon[35470]: pgmap v461: 129 pgs: 129 active+clean; 1.1 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 17 MiB/s wr, 783 op/s 2024-02-02T15:41:45.977 INFO:tasks.workunit.client.0.smithi001.stdout: 10 43693 32.39 MB/sec warmup 78 sec latency 40.521 ms 2024-02-02T15:41:46.977 INFO:tasks.workunit.client.0.smithi001.stdout: 10 44494 32.53 MB/sec warmup 79 sec latency 68.832 ms 2024-02-02T15:41:47.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:41:47 smithi071 ceph-mon[44707]: pgmap v462: 129 pgs: 129 active+clean; 1.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 19 MiB/s wr, 925 op/s 2024-02-02T15:41:47.923 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:41:47 smithi142 ceph-mon[45728]: pgmap v462: 129 pgs: 129 active+clean; 1.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 19 MiB/s wr, 925 op/s 2024-02-02T15:41:47.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:41:47 smithi001 ceph-mon[35470]: pgmap v462: 129 pgs: 129 active+clean; 1.1 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 19 MiB/s wr, 925 op/s 2024-02-02T15:41:47.977 INFO:tasks.workunit.client.0.smithi001.stdout: 10 45288 32.68 MB/sec warmup 80 sec latency 38.970 ms 2024-02-02T15:41:48.977 INFO:tasks.workunit.client.0.smithi001.stdout: 10 46099 32.85 MB/sec warmup 81 sec latency 39.629 ms 2024-02-02T15:41:49.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:41:49 smithi071 ceph-mon[44707]: pgmap v463: 129 pgs: 129 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 15 MiB/s wr, 781 op/s 2024-02-02T15:41:49.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:41:49 smithi142 ceph-mon[45728]: pgmap v463: 129 pgs: 129 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 15 MiB/s wr, 781 op/s 2024-02-02T15:41:49.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:41:49 smithi001 ceph-mon[35470]: pgmap v463: 129 pgs: 129 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 15 MiB/s wr, 781 op/s 2024-02-02T15:41:49.977 INFO:tasks.workunit.client.0.smithi001.stdout: 10 46890 32.86 MB/sec warmup 82 sec latency 73.289 ms 2024-02-02T15:41:50.977 INFO:tasks.workunit.client.0.smithi001.stdout: 10 47559 33.04 MB/sec warmup 83 sec latency 65.019 ms 2024-02-02T15:41:51.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:41:51 smithi071 ceph-mon[44707]: pgmap v464: 129 pgs: 129 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 12 MiB/s wr, 629 op/s 2024-02-02T15:41:51.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:41:51 smithi142 ceph-mon[45728]: pgmap v464: 129 pgs: 129 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 12 MiB/s wr, 629 op/s 2024-02-02T15:41:51.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:41:51 smithi001 ceph-mon[35470]: pgmap v464: 129 pgs: 129 active+clean; 1.1 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 12 MiB/s wr, 629 op/s 2024-02-02T15:41:51.978 INFO:tasks.workunit.client.0.smithi001.stdout: 10 48256 32.98 MB/sec warmup 84 sec latency 59.309 ms 2024-02-02T15:41:52.978 INFO:tasks.workunit.client.0.smithi001.stdout: 10 49096 33.13 MB/sec warmup 85 sec latency 39.895 ms 2024-02-02T15:41:53.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:41:53 smithi071 ceph-mon[44707]: pgmap v465: 129 pgs: 129 active+clean; 1.2 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 15 MiB/s wr, 864 op/s 2024-02-02T15:41:53.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:41:53 smithi142 ceph-mon[45728]: pgmap v465: 129 pgs: 129 active+clean; 1.2 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 15 MiB/s wr, 864 op/s 2024-02-02T15:41:53.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:41:53 smithi001 ceph-mon[35470]: pgmap v465: 129 pgs: 129 active+clean; 1.2 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 15 MiB/s wr, 864 op/s 2024-02-02T15:41:53.978 INFO:tasks.workunit.client.0.smithi001.stdout: 10 49995 33.37 MB/sec warmup 86 sec latency 70.068 ms 2024-02-02T15:41:54.978 INFO:tasks.workunit.client.0.smithi001.stdout: 10 50752 33.46 MB/sec warmup 87 sec latency 47.640 ms 2024-02-02T15:41:55.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:41:55 smithi071 ceph-mon[44707]: pgmap v466: 129 pgs: 129 active+clean; 1.2 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 6.0 MiB/s wr, 468 op/s 2024-02-02T15:41:55.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:41:55 smithi142 ceph-mon[45728]: pgmap v466: 129 pgs: 129 active+clean; 1.2 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 6.0 MiB/s wr, 468 op/s 2024-02-02T15:41:55.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:41:55 smithi001 ceph-mon[35470]: pgmap v466: 129 pgs: 129 active+clean; 1.2 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 6.0 MiB/s wr, 468 op/s 2024-02-02T15:41:55.978 INFO:tasks.workunit.client.0.smithi001.stdout: 10 51346 33.48 MB/sec warmup 88 sec latency 65.321 ms 2024-02-02T15:41:56.978 INFO:tasks.workunit.client.0.smithi001.stdout: 10 52105 33.43 MB/sec warmup 89 sec latency 48.967 ms 2024-02-02T15:41:57.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:41:57 smithi071 ceph-mon[44707]: pgmap v467: 129 pgs: 129 active+clean; 1.2 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 7.6 MiB/s wr, 602 op/s 2024-02-02T15:41:57.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:41:57 smithi142 ceph-mon[45728]: pgmap v467: 129 pgs: 129 active+clean; 1.2 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 7.6 MiB/s wr, 602 op/s 2024-02-02T15:41:57.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:41:57 smithi001 ceph-mon[35470]: pgmap v467: 129 pgs: 129 active+clean; 1.2 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 7.6 MiB/s wr, 602 op/s 2024-02-02T15:41:57.978 INFO:tasks.workunit.client.0.smithi001.stdout: 10 52979 33.69 MB/sec warmup 90 sec latency 39.537 ms 2024-02-02T15:41:58.978 INFO:tasks.workunit.client.0.smithi001.stdout: 10 53904 33.83 MB/sec warmup 91 sec latency 25.873 ms 2024-02-02T15:41:59.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:41:59 smithi071 ceph-mon[44707]: pgmap v468: 129 pgs: 129 active+clean; 1.2 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.0 MiB/s wr, 551 op/s 2024-02-02T15:41:59.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:41:59 smithi142 ceph-mon[45728]: pgmap v468: 129 pgs: 129 active+clean; 1.2 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.0 MiB/s wr, 551 op/s 2024-02-02T15:41:59.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:41:59 smithi001 ceph-mon[35470]: pgmap v468: 129 pgs: 129 active+clean; 1.2 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.0 MiB/s wr, 551 op/s 2024-02-02T15:41:59.978 INFO:tasks.workunit.client.0.smithi001.stdout: 10 54583 33.94 MB/sec warmup 92 sec latency 43.240 ms 2024-02-02T15:42:00.978 INFO:tasks.workunit.client.0.smithi001.stdout: 10 55127 33.81 MB/sec warmup 93 sec latency 90.175 ms 2024-02-02T15:42:01.648 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-02T15:42:01.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:42:01 smithi071 ceph-mon[44707]: pgmap v469: 129 pgs: 129 active+clean; 1.2 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.9 MiB/s wr, 460 op/s 2024-02-02T15:42:01.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.866+0000 7f305b600700 1 -- 172.21.15.1:0/532568893 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30541019a0 msgr2=0x7f30541059e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:01.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.866+0000 7f305b600700 1 --2- 172.21.15.1:0/532568893 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30541019a0 0x7f30541059e0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f303c009990 tx=0x7f303c009ca0 comp rx=0 tx=0).stop 2024-02-02T15:42:01.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.866+0000 7f305b600700 1 -- 172.21.15.1:0/532568893 shutdown_connections 2024-02-02T15:42:01.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.866+0000 7f305b600700 1 --2- 172.21.15.1:0/532568893 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3054106110 0x7f305410dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:01.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.866+0000 7f305b600700 1 --2- 172.21.15.1:0/532568893 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3054100ff0 0x7f30541013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:01.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.866+0000 7f305b600700 1 --2- 172.21.15.1:0/532568893 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f30541019a0 0x7f30541059e0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:01.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.867+0000 7f305b600700 1 -- 172.21.15.1:0/532568893 >> 172.21.15.1:0/532568893 conn(0x7f30540fc810 msgr2=0x7f30540fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:01.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.867+0000 7f305b600700 1 -- 172.21.15.1:0/532568893 shutdown_connections 2024-02-02T15:42:01.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.867+0000 7f305b600700 1 -- 172.21.15.1:0/532568893 wait complete. 2024-02-02T15:42:01.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.868+0000 7f305b600700 1 Processor -- start 2024-02-02T15:42:01.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.868+0000 7f305b600700 1 -- start start 2024-02-02T15:42:01.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.869+0000 7f305b600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3054100ff0 0x7f3054199060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:01.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.869+0000 7f305b600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3054106110 0x7f30541995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:01.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.869+0000 7f305b600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f305419da00 0x7f305419de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:01.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.869+0000 7f305b600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f305419e480 con 0x7f305419da00 2024-02-02T15:42:01.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.869+0000 7f305b600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f305419e5f0 con 0x7f3054106110 2024-02-02T15:42:01.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.869+0000 7f305b600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f305419e7b0 con 0x7f3054100ff0 2024-02-02T15:42:01.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.869+0000 7f3059600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f305419da00 0x7f305419de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:01.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.869+0000 7f3058c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3054100ff0 0x7f3054199060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:01.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.869+0000 7f3058c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3054100ff0 0x7f3054199060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:42824/0 (socket says 172.21.15.1:42824) 2024-02-02T15:42:01.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.869+0000 7f3058c00700 1 -- 172.21.15.1:0/3119617814 learned_addr learned my addr 172.21.15.1:0/3119617814 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:42:01.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.869+0000 7f3053e00700 1 --2- 172.21.15.1:0/3119617814 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3054106110 0x7f30541995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:01.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.869+0000 7f3058c00700 1 -- 172.21.15.1:0/3119617814 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3054106110 msgr2=0x7f30541995a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:01.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.869+0000 7f3058c00700 1 --2- 172.21.15.1:0/3119617814 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3054106110 0x7f30541995a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:01.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.869+0000 7f3058c00700 1 -- 172.21.15.1:0/3119617814 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f305419da00 msgr2=0x7f305419de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:01.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.869+0000 7f3058c00700 1 --2- 172.21.15.1:0/3119617814 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f305419da00 0x7f305419de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:01.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.869+0000 7f3058c00700 1 -- 172.21.15.1:0/3119617814 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f303c0095f0 con 0x7f3054100ff0 2024-02-02T15:42:01.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.870+0000 7f3058c00700 1 --2- 172.21.15.1:0/3119617814 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3054100ff0 0x7f3054199060 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f304400e9a0 tx=0x7f304400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:01.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.870+0000 7f3051600700 1 -- 172.21.15.1:0/3119617814 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3044009790 con 0x7f3054100ff0 2024-02-02T15:42:01.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.870+0000 7f3051600700 1 -- 172.21.15.1:0/3119617814 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f304400cbd0 con 0x7f3054100ff0 2024-02-02T15:42:01.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.870+0000 7f305b600700 1 -- 172.21.15.1:0/3119617814 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30541a5d10 con 0x7f3054100ff0 2024-02-02T15:42:01.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.870+0000 7f3051600700 1 -- 172.21.15.1:0/3119617814 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3044018680 con 0x7f3054100ff0 2024-02-02T15:42:01.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.870+0000 7f305b600700 1 -- 172.21.15.1:0/3119617814 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30541a61a0 con 0x7f3054100ff0 2024-02-02T15:42:01.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.875+0000 7f3051600700 1 -- 172.21.15.1:0/3119617814 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f304400cd40 con 0x7f3054100ff0 2024-02-02T15:42:01.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.875+0000 7f306f0cbb80 1 Processor -- start 2024-02-02T15:42:01.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.876+0000 7f306f0cbb80 1 -- start start 2024-02-02T15:42:01.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.876+0000 7f306f0cbb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614a1b57010 0x5614a1b573f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:01.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.876+0000 7f3051600700 1 --2- 172.21.15.1:0/3119617814 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f30340a0690 0x7f30340a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:01.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.876+0000 7f306f0cbb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614a1b57930 0x5614a1b622a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:01.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.876+0000 7f306f0cbb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614a1b627e0 0x5614a1b64bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:01.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.876+0000 7f306f0cbb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614a1acb4c0 con 0x5614a1b57010 2024-02-02T15:42:01.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.876+0000 7f306f0cbb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614a1aafaa0 con 0x5614a1b57930 2024-02-02T15:42:01.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.876+0000 7f306f0cbb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614a1aad460 con 0x5614a1b627e0 2024-02-02T15:42:01.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.876+0000 7f3051600700 1 -- 172.21.15.1:0/3119617814 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f3044014070 con 0x7f3054100ff0 2024-02-02T15:42:01.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.876+0000 7f3059600700 1 --2- 172.21.15.1:0/3119617814 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f30340a0690 0x7f30340a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:01.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.876+0000 7f3053e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614a1b57930 0x5614a1b622a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:01.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.876+0000 7f3058c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614a1b57010 0x5614a1b573f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:01.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.876+0000 7f3053e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614a1b57930 0x5614a1b622a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59496/0 (socket says 172.21.15.1:59496) 2024-02-02T15:42:01.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.876+0000 7f3053e00700 1 -- 172.21.15.1:0/1546812219 learned_addr learned my addr 172.21.15.1:0/1546812219 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:42:01.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.877+0000 7f3059600700 1 --2- 172.21.15.1:0/1546812219 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614a1b627e0 0x5614a1b64bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:01.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.877+0000 7f3053e00700 1 -- 172.21.15.1:0/1546812219 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614a1b627e0 msgr2=0x5614a1b64bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:01.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.877+0000 7f3053e00700 1 --2- 172.21.15.1:0/1546812219 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614a1b627e0 0x5614a1b64bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:01.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.877+0000 7f3053e00700 1 -- 172.21.15.1:0/1546812219 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614a1b57010 msgr2=0x5614a1b573f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:01.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.877+0000 7f3053e00700 1 --2- 172.21.15.1:0/1546812219 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614a1b57010 0x5614a1b573f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:01.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.877+0000 7f3053e00700 1 -- 172.21.15.1:0/1546812219 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5614a192a3b0 con 0x5614a1b57930 2024-02-02T15:42:01.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.877+0000 7f3058c00700 1 --2- 172.21.15.1:0/1546812219 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614a1b57010 0x5614a1b573f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:42:01.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.877+0000 7f3059600700 1 --2- 172.21.15.1:0/3119617814 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f30340a0690 0x7f30340a2b50 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f304c009ec0 tx=0x7f304c0134a0 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:01.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.877+0000 7f3053e00700 1 --2- 172.21.15.1:0/1546812219 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614a1b57930 0x5614a1b622a0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f303c015000 tx=0x7f303c005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c92b61b75996778 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:01.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.877+0000 7f3049c00700 1 -- 172.21.15.1:0/1546812219 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f303c00bee0 con 0x5614a1b57930 2024-02-02T15:42:01.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.877+0000 7f3049c00700 1 -- 172.21.15.1:0/1546812219 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f303c01e040 con 0x5614a1b57930 2024-02-02T15:42:01.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.877+0000 7f3049c00700 1 -- 172.21.15.1:0/1546812219 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f303c004a60 con 0x5614a1b57930 2024-02-02T15:42:01.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.878+0000 7f306f0cbb80 1 -- 172.21.15.1:0/1546812219 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614a1b57930 msgr2=0x5614a1b622a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:01.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.878+0000 7f306f0cbb80 1 --2- 172.21.15.1:0/1546812219 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614a1b57930 0x5614a1b622a0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f303c015000 tx=0x7f303c005e80 comp rx=0 tx=0).stop 2024-02-02T15:42:01.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.878+0000 7f306f0cbb80 1 -- 172.21.15.1:0/1546812219 shutdown_connections 2024-02-02T15:42:01.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.878+0000 7f306f0cbb80 1 --2- 172.21.15.1:0/1546812219 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614a1b627e0 0x5614a1b64bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:01.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.878+0000 7f306f0cbb80 1 --2- 172.21.15.1:0/1546812219 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614a1b57010 0x5614a1b573f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:01.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.878+0000 7f306f0cbb80 1 --2- 172.21.15.1:0/1546812219 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614a1b57930 0x5614a1b622a0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:01.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.878+0000 7f306f0cbb80 1 -- 172.21.15.1:0/1546812219 >> 172.21.15.1:0/1546812219 conn(0x5614a1ab6db0 msgr2=0x5614a1b533b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:01.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.878+0000 7f306f0cbb80 1 -- 172.21.15.1:0/1546812219 shutdown_connections 2024-02-02T15:42:01.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.878+0000 7f306f0cbb80 1 -- 172.21.15.1:0/1546812219 wait complete. 2024-02-02T15:42:01.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.879+0000 7f306f0cbb80 1 -- start start 2024-02-02T15:42:01.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.879+0000 7f306f0cbb80 1 Processor -- start 2024-02-02T15:42:01.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.880+0000 7f306f0cbb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614a1b57010 0x5614a1b62110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:01.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.880+0000 7f306f0cbb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614a1b57930 0x5614a1b5f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:01.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.880+0000 7f306f0cbb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614a1b627e0 0x5614a1b5f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:01.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.880+0000 7f306f0cbb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614a1aa8d50 con 0x5614a1b627e0 2024-02-02T15:42:01.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.880+0000 7f306f0cbb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614a1a44d80 con 0x5614a1b57930 2024-02-02T15:42:01.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.880+0000 7f306f0cbb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614a1a44a20 con 0x5614a1b57010 2024-02-02T15:42:01.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.880+0000 7f306f0cbb80 10 client.0 _ll_register_callbacks cb 0x5614a1a46290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:42:01.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.880+0000 7f3059600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614a1b627e0 0x5614a1b5f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:01.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.880+0000 7f3053e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614a1b57930 0x5614a1b5f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:01.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.880+0000 7f3058c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614a1b57010 0x5614a1b62110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:01.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.880+0000 7f3059600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614a1b627e0 0x5614a1b5f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:48988/0 (socket says 172.21.15.1:48988) 2024-02-02T15:42:01.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.880+0000 7f3059600700 1 -- 172.21.15.1:0/3391042818 learned_addr learned my addr 172.21.15.1:0/3391042818 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:42:01.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.881+0000 7f3058c00700 1 -- 172.21.15.1:0/3391042818 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614a1b57930 msgr2=0x5614a1b5f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:01.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.881+0000 7f3058c00700 1 --2- 172.21.15.1:0/3391042818 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614a1b57930 0x5614a1b5f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:01.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.881+0000 7f3058c00700 1 -- 172.21.15.1:0/3391042818 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614a1b627e0 msgr2=0x5614a1b5f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:01.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.881+0000 7f3058c00700 1 --2- 172.21.15.1:0/3391042818 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614a1b627e0 0x5614a1b5f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:01.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.881+0000 7f3058c00700 1 -- 172.21.15.1:0/3391042818 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30541a61a0 con 0x5614a1b57010 2024-02-02T15:42:01.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.881+0000 7f3059600700 1 --2- 172.21.15.1:0/3391042818 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614a1b627e0 0x5614a1b5f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:42:01.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.881+0000 7f3058c00700 1 --2- 172.21.15.1:0/3391042818 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614a1b57010 0x5614a1b62110 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f30440c9710 tx=0x7f30440c9c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3b73e0597b77336 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:01.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.881+0000 7f3042000700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:42:01.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.881+0000 7f3042000700 1 -- 172.21.15.1:0/3391042818 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30440992b0 con 0x5614a1b57010 2024-02-02T15:42:01.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.881+0000 7f3042000700 1 -- 172.21.15.1:0/3391042818 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f30440994e0 con 0x5614a1b57010 2024-02-02T15:42:01.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.881+0000 7f306f0cbb80 1 -- 172.21.15.1:0/3391042818 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5614a1b4b0f0 con 0x5614a1b57010 2024-02-02T15:42:01.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.881+0000 7f3042000700 1 -- 172.21.15.1:0/3391042818 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f304409a4f0 con 0x5614a1b57010 2024-02-02T15:42:01.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.881+0000 7f306f0cbb80 10 client.29485 fetch_fsmap learned FSMap version 61 2024-02-02T15:42:01.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.881+0000 7f306f0cbb80 1 -- 172.21.15.1:0/3391042818 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x7f305410e500 con 0x5614a1b57010 2024-02-02T15:42:01.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.882+0000 7f3042000700 1 -- 172.21.15.1:0/3391042818 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5614a1b4b0f0 con 0x5614a1b57010 2024-02-02T15:42:01.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.882+0000 7f3042000700 1 -- 172.21.15.1:0/3391042818 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 61) v1 ==== 7550+0+0 (secure 0 0 0) 0x7f30440989d0 con 0x5614a1b57010 2024-02-02T15:42:01.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.882+0000 7f306f0cbb80 10 client.29485 fetch_fsmap finished waiting for FSMap version 61 2024-02-02T15:42:01.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.882+0000 7f306f0cbb80 10 client.29485 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:42:01.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.882+0000 7f306f0cbb80 20 client.29485 populate_metadata read hostname 'smithi001' 2024-02-02T15:42:01.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.882+0000 7f306f0cbb80 1 --2- 172.21.15.1:0/3391042818 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5614a1b721a0 0x5614a1b745c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:01.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.882+0000 7f306f0cbb80 4 client.29485 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:42:01.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.882+0000 7f306f0cbb80 1 -- 172.21.15.1:0/3391042818 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5614a1aa1dd0 con 0x5614a1b721a0 2024-02-02T15:42:01.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.882+0000 7f3053e00700 1 --2- 172.21.15.1:0/3391042818 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5614a1b721a0 0x5614a1b745c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:01.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.883+0000 7f3053e00700 1 --2- 172.21.15.1:0/3391042818 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5614a1b721a0 0x5614a1b745c0 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c664bb53e4fd3ca0 server_cookie=c91188e89823eb71 in_seq=0 out_seq=0 2024-02-02T15:42:01.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.883+0000 7f3042000700 10 client.29485 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:42:01.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.883+0000 7f3042000700 1 -- 172.21.15.1:0/3391042818 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5614a1aa1dd0 con 0x5614a1b721a0 2024-02-02T15:42:01.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.883+0000 7f3042000700 10 client.29485 handle_command_reply: tid=0 2024-02-02T15:42:01.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.883+0000 7f306f0cbb80 1 client.29485 shutdown 2024-02-02T15:42:01.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.884+0000 7f306f0cbb80 1 -- 172.21.15.1:0/3391042818 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614a1b57010 msgr2=0x5614a1b62110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:01.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.884+0000 7f306f0cbb80 1 --2- 172.21.15.1:0/3391042818 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614a1b57010 0x5614a1b62110 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f30440c9710 tx=0x7f30440c9c70 comp rx=0 tx=0).stop 2024-02-02T15:42:01.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.884+0000 7f306f0cbb80 1 -- 172.21.15.1:0/3391042818 shutdown_connections 2024-02-02T15:42:01.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.884+0000 7f306f0cbb80 1 --2- 172.21.15.1:0/3391042818 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5614a1b721a0 0x5614a1b745c0 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:01.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.884+0000 7f306f0cbb80 1 --2- 172.21.15.1:0/3391042818 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614a1b627e0 0x5614a1b5f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:01.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.884+0000 7f306f0cbb80 1 --2- 172.21.15.1:0/3391042818 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614a1b57010 0x5614a1b62110 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:01.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.884+0000 7f3042000700 0 client.29485 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:42:01.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.884+0000 7f306f0cbb80 1 --2- 172.21.15.1:0/3391042818 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614a1b57930 0x5614a1b5f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:01.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.884+0000 7f306f0cbb80 1 -- 172.21.15.1:0/3391042818 >> 172.21.15.1:0/3391042818 conn(0x5614a1ab6db0 msgr2=0x5614a1b52fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:01.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.884+0000 7f306f0cbb80 1 -- 172.21.15.1:0/3391042818 shutdown_connections 2024-02-02T15:42:01.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.884+0000 7f306f0cbb80 1 -- 172.21.15.1:0/3391042818 wait complete. 2024-02-02T15:42:01.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.884+0000 7f306f0cbb80 20 client.29485 trim_cache size 0 max 16384 2024-02-02T15:42:01.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.901+0000 7f306f0cbb80 1 Processor -- start 2024-02-02T15:42:01.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.901+0000 7f306f0cbb80 1 -- start start 2024-02-02T15:42:01.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.902+0000 7f306f0cbb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614a1b57010 0x5614a1b69340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:01.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.902+0000 7f306f0cbb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614a1b57930 0x5614a1b59ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:01.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.902+0000 7f306f0cbb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614a1b627e0 0x5614a1b52eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:01.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.902+0000 7f306f0cbb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614a192c6a0 con 0x5614a1b57010 2024-02-02T15:42:01.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.902+0000 7f306f0cbb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614a1b4afc0 con 0x5614a1b57930 2024-02-02T15:42:01.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.902+0000 7f306f0cbb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614a1b54a30 con 0x5614a1b627e0 2024-02-02T15:42:01.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.902+0000 7f3058c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614a1b57010 0x5614a1b69340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:01.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.902+0000 7f3058c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614a1b57010 0x5614a1b69340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49004/0 (socket says 172.21.15.1:49004) 2024-02-02T15:42:01.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.902+0000 7f3058c00700 1 -- 172.21.15.1:0/1204379225 learned_addr learned my addr 172.21.15.1:0/1204379225 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:42:01.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.902+0000 7f3053e00700 1 --2- 172.21.15.1:0/1204379225 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614a1b57930 0x5614a1b59ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:01.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.902+0000 7f3059600700 1 --2- 172.21.15.1:0/1204379225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614a1b627e0 0x5614a1b52eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:01.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.902+0000 7f3058c00700 1 -- 172.21.15.1:0/1204379225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614a1b627e0 msgr2=0x5614a1b52eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:01.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.902+0000 7f3058c00700 1 --2- 172.21.15.1:0/1204379225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614a1b627e0 0x5614a1b52eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:01.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.902+0000 7f3058c00700 1 -- 172.21.15.1:0/1204379225 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614a1b57930 msgr2=0x5614a1b59ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:01.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.902+0000 7f3058c00700 1 --2- 172.21.15.1:0/1204379225 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614a1b57930 0x5614a1b59ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:01.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.902+0000 7f3058c00700 1 -- 172.21.15.1:0/1204379225 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5614a18d6b60 con 0x5614a1b57010 2024-02-02T15:42:01.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.902+0000 7f3053e00700 1 --2- 172.21.15.1:0/1204379225 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614a1b57930 0x5614a1b59ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:42:01.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.903+0000 7f3058c00700 1 --2- 172.21.15.1:0/1204379225 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614a1b57010 0x5614a1b69340 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f3044095f00 tx=0x7f3044097a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=940e6864ab97fdf6 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:01.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.903+0000 7f304a600700 1 -- 172.21.15.1:0/1204379225 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30440992b0 con 0x5614a1b57010 2024-02-02T15:42:01.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.903+0000 7f304a600700 1 -- 172.21.15.1:0/1204379225 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f30440994e0 con 0x5614a1b57010 2024-02-02T15:42:01.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.903+0000 7f304a600700 1 -- 172.21.15.1:0/1204379225 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f304409e450 con 0x5614a1b57010 2024-02-02T15:42:01.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.903+0000 7f306f0cbb80 1 -- 172.21.15.1:0/1204379225 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614a1b57010 msgr2=0x5614a1b69340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:01.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.903+0000 7f306f0cbb80 1 --2- 172.21.15.1:0/1204379225 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614a1b57010 0x5614a1b69340 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f3044095f00 tx=0x7f3044097a90 comp rx=0 tx=0).stop 2024-02-02T15:42:01.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.903+0000 7f306f0cbb80 1 -- 172.21.15.1:0/1204379225 shutdown_connections 2024-02-02T15:42:01.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.903+0000 7f306f0cbb80 1 --2- 172.21.15.1:0/1204379225 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614a1b627e0 0x5614a1b52eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:01.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.904+0000 7f306f0cbb80 1 --2- 172.21.15.1:0/1204379225 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614a1b57010 0x5614a1b69340 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:01.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.904+0000 7f306f0cbb80 1 --2- 172.21.15.1:0/1204379225 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614a1b57930 0x5614a1b59ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:01.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.904+0000 7f306f0cbb80 1 -- 172.21.15.1:0/1204379225 >> 172.21.15.1:0/1204379225 conn(0x5614a1ab6db0 msgr2=0x5614a1b6d910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:01.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.904+0000 7f306f0cbb80 1 -- 172.21.15.1:0/1204379225 shutdown_connections 2024-02-02T15:42:01.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.904+0000 7f306f0cbb80 1 -- 172.21.15.1:0/1204379225 wait complete. 2024-02-02T15:42:01.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.904+0000 7f306f0cbb80 1 -- start start 2024-02-02T15:42:01.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.904+0000 7f306f0cbb80 1 Processor -- start 2024-02-02T15:42:01.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.905+0000 7f306f0cbb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614a1b57010 0x5614a1bb7c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:01.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.905+0000 7f306f0cbb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614a1b57930 0x5614a1bb8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:01.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.905+0000 7f306f0cbb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614a1b627e0 0x5614a1bb86d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:01.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.905+0000 7f306f0cbb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614a1bc2db0 con 0x5614a1b57930 2024-02-02T15:42:01.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.905+0000 7f306f0cbb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614a1bc2ef0 con 0x5614a1b57010 2024-02-02T15:42:01.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.905+0000 7f306f0cbb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614a1bc30b0 con 0x5614a1b627e0 2024-02-02T15:42:01.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.905+0000 7f306f0cbb80 10 client.0 _ll_register_callbacks cb 0x5614a1a46290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:42:01.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.905+0000 7f3058c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614a1b57010 0x5614a1bb7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:01.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.905+0000 7f3053e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614a1b57930 0x5614a1bb8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:01.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.905+0000 7f3058c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614a1b57010 0x5614a1bb7c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59518/0 (socket says 172.21.15.1:59518) 2024-02-02T15:42:01.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.905+0000 7f3058c00700 1 -- 172.21.15.1:0/3854896777 learned_addr learned my addr 172.21.15.1:0/3854896777 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:42:01.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.905+0000 7f3059600700 1 --2- 172.21.15.1:0/3854896777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614a1b627e0 0x5614a1bb86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:01.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.905+0000 7f3058c00700 1 -- 172.21.15.1:0/3854896777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614a1b627e0 msgr2=0x5614a1bb86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:01.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.905+0000 7f3058c00700 1 --2- 172.21.15.1:0/3854896777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614a1b627e0 0x5614a1bb86d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:01.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.905+0000 7f3058c00700 1 -- 172.21.15.1:0/3854896777 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614a1b57930 msgr2=0x5614a1bb8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:01.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.905+0000 7f3058c00700 1 --2- 172.21.15.1:0/3854896777 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614a1b57930 0x5614a1bb8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:01.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.905+0000 7f3058c00700 1 -- 172.21.15.1:0/3854896777 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5614a192a3b0 con 0x5614a1b57010 2024-02-02T15:42:01.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.906+0000 7f3058c00700 1 --2- 172.21.15.1:0/3854896777 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614a1b57010 0x5614a1bb7c50 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f3044097830 tx=0x7f30440c9c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e181daa5226f5c25 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:01.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.906+0000 7f3042000700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:42:01.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.906+0000 7f3042000700 1 -- 172.21.15.1:0/3854896777 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30440992b0 con 0x5614a1b57010 2024-02-02T15:42:01.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.906+0000 7f3042000700 1 -- 172.21.15.1:0/3854896777 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f304409d310 con 0x5614a1b57010 2024-02-02T15:42:01.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.906+0000 7f306f0cbb80 1 -- 172.21.15.1:0/3854896777 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f303c01e040 con 0x5614a1b57010 2024-02-02T15:42:01.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.906+0000 7f3042000700 1 -- 172.21.15.1:0/3854896777 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f30440a1550 con 0x5614a1b57010 2024-02-02T15:42:01.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.906+0000 7f306f0cbb80 10 client.29525 fetch_fsmap learned FSMap version 61 2024-02-02T15:42:01.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.906+0000 7f306f0cbb80 1 -- 172.21.15.1:0/3854896777 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x7f305410e500 con 0x5614a1b57010 2024-02-02T15:42:01.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.906+0000 7f3042000700 1 -- 172.21.15.1:0/3854896777 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f303c01e040 con 0x5614a1b57010 2024-02-02T15:42:01.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.906+0000 7f3042000700 1 -- 172.21.15.1:0/3854896777 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 61) v1 ==== 7550+0+0 (secure 0 0 0) 0x7f3044099e70 con 0x5614a1b57010 2024-02-02T15:42:01.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.906+0000 7f306f0cbb80 10 client.29525 fetch_fsmap finished waiting for FSMap version 61 2024-02-02T15:42:01.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.906+0000 7f306f0cbb80 10 client.29525 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:42:01.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.906+0000 7f306f0cbb80 20 client.29525 populate_metadata read hostname 'smithi001' 2024-02-02T15:42:01.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.906+0000 7f306f0cbb80 1 --2- 172.21.15.1:0/3854896777 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5614a1bc4340 0x5614a1bc6730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:01.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.906+0000 7f306f0cbb80 4 client.29525 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2024-02-02T15:42:01.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.906+0000 7f306f0cbb80 1 -- 172.21.15.1:0/3854896777 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5614a1a67f50 con 0x5614a1bc4340 2024-02-02T15:42:01.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.907+0000 7f3053e00700 1 --2- 172.21.15.1:0/3854896777 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5614a1bc4340 0x5614a1bc6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:01.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.907+0000 7f3053e00700 1 --2- 172.21.15.1:0/3854896777 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5614a1bc4340 0x5614a1bc6730 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b13c5e129f996bd6 server_cookie=c190d0948e9b9f77 in_seq=0 out_seq=0 2024-02-02T15:42:01.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.907+0000 7f3042000700 10 client.29525 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:42:01.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.907+0000 7f3042000700 1 -- 172.21.15.1:0/3854896777 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5614a1a67f50 con 0x5614a1bc4340 2024-02-02T15:42:01.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.907+0000 7f3042000700 10 client.29525 handle_command_reply: tid=0 2024-02-02T15:42:01.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.907+0000 7f306f0cbb80 1 client.29525 shutdown 2024-02-02T15:42:01.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.907+0000 7f306f0cbb80 1 -- 172.21.15.1:0/3854896777 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614a1b57010 msgr2=0x5614a1bb7c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:01.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.907+0000 7f306f0cbb80 1 --2- 172.21.15.1:0/3854896777 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614a1b57010 0x5614a1bb7c50 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f3044097830 tx=0x7f30440c9c70 comp rx=0 tx=0).stop 2024-02-02T15:42:01.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.907+0000 7f306f0cbb80 1 -- 172.21.15.1:0/3854896777 shutdown_connections 2024-02-02T15:42:01.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.907+0000 7f306f0cbb80 1 --2- 172.21.15.1:0/3854896777 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5614a1bc4340 0x5614a1bc6730 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:01.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.907+0000 7f306f0cbb80 1 --2- 172.21.15.1:0/3854896777 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5614a1b627e0 0x5614a1bb86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:01.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.907+0000 7f306f0cbb80 1 --2- 172.21.15.1:0/3854896777 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5614a1b57010 0x5614a1bb7c50 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:01.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.907+0000 7f3042000700 0 client.29525 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:42:01.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.907+0000 7f306f0cbb80 1 --2- 172.21.15.1:0/3854896777 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5614a1b57930 0x5614a1bb8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:01.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.907+0000 7f306f0cbb80 1 -- 172.21.15.1:0/3854896777 >> 172.21.15.1:0/3854896777 conn(0x5614a1ab6db0 msgr2=0x5614a1b52e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:01.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.908+0000 7f306f0cbb80 1 -- 172.21.15.1:0/3854896777 shutdown_connections 2024-02-02T15:42:01.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.908+0000 7f306f0cbb80 1 -- 172.21.15.1:0/3854896777 wait complete. 2024-02-02T15:42:01.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.908+0000 7f306f0cbb80 20 client.29525 trim_cache size 0 max 16384 2024-02-02T15:42:01.913 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:42:01.913 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2024-02-02T15:42:01.913 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2024-02-02T15:42:01.913 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:42:01.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.908+0000 7f305b600700 1 -- 172.21.15.1:0/3119617814 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f30340a0690 msgr2=0x7f30340a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:01.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.908+0000 7f305b600700 1 --2- 172.21.15.1:0/3119617814 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f30340a0690 0x7f30340a2b50 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f304c009ec0 tx=0x7f304c0134a0 comp rx=0 tx=0).stop 2024-02-02T15:42:01.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.908+0000 7f305b600700 1 -- 172.21.15.1:0/3119617814 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3054100ff0 msgr2=0x7f3054199060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:01.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.908+0000 7f305b600700 1 --2- 172.21.15.1:0/3119617814 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3054100ff0 0x7f3054199060 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f304400e9a0 tx=0x7f304400ecb0 comp rx=0 tx=0).stop 2024-02-02T15:42:01.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.908+0000 7f305b600700 1 -- 172.21.15.1:0/3119617814 shutdown_connections 2024-02-02T15:42:01.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.908+0000 7f305b600700 1 --2- 172.21.15.1:0/3119617814 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f30340a0690 0x7f30340a2b50 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:01.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.908+0000 7f305b600700 1 --2- 172.21.15.1:0/3119617814 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f305419da00 0x7f305419de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:01.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.908+0000 7f305b600700 1 --2- 172.21.15.1:0/3119617814 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3054100ff0 0x7f3054199060 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:01.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.908+0000 7f305b600700 1 --2- 172.21.15.1:0/3119617814 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3054106110 0x7f30541995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:01.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.908+0000 7f305b600700 1 -- 172.21.15.1:0/3119617814 >> 172.21.15.1:0/3119617814 conn(0x7f30540fc810 msgr2=0x7f30540fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:01.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.908+0000 7f305b600700 1 -- 172.21.15.1:0/3119617814 shutdown_connections 2024-02-02T15:42:01.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:01.909+0000 7f305b600700 1 -- 172.21.15.1:0/3119617814 wait complete. 2024-02-02T15:42:01.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:42:01 smithi142 ceph-mon[45728]: pgmap v469: 129 pgs: 129 active+clean; 1.2 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.9 MiB/s wr, 460 op/s 2024-02-02T15:42:01.923 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:42:01 smithi001 ceph-mon[35470]: pgmap v469: 129 pgs: 129 active+clean; 1.2 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.9 MiB/s wr, 460 op/s 2024-02-02T15:42:01.926 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-02T15:42:01.926 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T15:42:01.978 INFO:tasks.workunit.client.0.smithi001.stdout: 10 55127 33.45 MB/sec warmup 94 sec latency 1090.247 ms 2024-02-02T15:42:02.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.168+0000 7efc57000700 1 -- 172.21.15.1:0/4201868537 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc500730e0 msgr2=0x7efc500734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:02.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.168+0000 7efc57000700 1 --2- 172.21.15.1:0/4201868537 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc500730e0 0x7efc500734c0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7efc40009990 tx=0x7efc40009ca0 comp rx=0 tx=0).stop 2024-02-02T15:42:02.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.168+0000 7efc57000700 1 -- 172.21.15.1:0/4201868537 shutdown_connections 2024-02-02T15:42:02.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.168+0000 7efc57000700 1 --2- 172.21.15.1:0/4201868537 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efc50111cc0 0x7efc501140b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:02.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.168+0000 7efc57000700 1 --2- 172.21.15.1:0/4201868537 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc500730e0 0x7efc500734c0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:02.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.168+0000 7efc57000700 1 --2- 172.21.15.1:0/4201868537 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc50073a00 0x7efc50111780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:02.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.168+0000 7efc57000700 1 -- 172.21.15.1:0/4201868537 >> 172.21.15.1:0/4201868537 conn(0x7efc500fc760 msgr2=0x7efc500feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:02.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.169+0000 7efc57000700 1 -- 172.21.15.1:0/4201868537 shutdown_connections 2024-02-02T15:42:02.169 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.169+0000 7efc57000700 1 -- 172.21.15.1:0/4201868537 wait complete. 2024-02-02T15:42:02.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.170+0000 7efc57000700 1 Processor -- start 2024-02-02T15:42:02.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.171+0000 7efc57000700 1 -- start start 2024-02-02T15:42:02.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.171+0000 7efc57000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc500730e0 0x7efc5006ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:02.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.171+0000 7efc57000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efc50073a00 0x7efc50070480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:02.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.171+0000 7efc57000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc50111cc0 0x7efc5006b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:02.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.171+0000 7efc57000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc5006b770 con 0x7efc50073a00 2024-02-02T15:42:02.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.171+0000 7efc57000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc5006b8e0 con 0x7efc50111cc0 2024-02-02T15:42:02.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.171+0000 7efc57000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc5006baa0 con 0x7efc500730e0 2024-02-02T15:42:02.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.172+0000 7efc4f400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efc50073a00 0x7efc50070480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:02.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.172+0000 7efc4f400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efc50073a00 0x7efc50070480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49028/0 (socket says 172.21.15.1:49028) 2024-02-02T15:42:02.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.172+0000 7efc4f400700 1 -- 172.21.15.1:0/3074811420 learned_addr learned my addr 172.21.15.1:0/3074811420 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:42:02.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.172+0000 7efc4f400700 1 -- 172.21.15.1:0/3074811420 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc500730e0 msgr2=0x7efc5006ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:02.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.172+0000 7efc4f400700 1 --2- 172.21.15.1:0/3074811420 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc500730e0 0x7efc5006ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:02.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.172+0000 7efc4f400700 1 -- 172.21.15.1:0/3074811420 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc50111cc0 msgr2=0x7efc5006b130 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-02-02T15:42:02.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.172+0000 7efc4f400700 1 --2- 172.21.15.1:0/3074811420 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc50111cc0 0x7efc5006b130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:02.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.172+0000 7efc4f400700 1 -- 172.21.15.1:0/3074811420 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc400095f0 con 0x7efc50073a00 2024-02-02T15:42:02.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.173+0000 7efc4f400700 1 --2- 172.21.15.1:0/3074811420 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efc50073a00 0x7efc50070480 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7efc3800e9a0 tx=0x7efc3800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:02.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.174+0000 7efc4cc00700 1 -- 172.21.15.1:0/3074811420 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc38009790 con 0x7efc50073a00 2024-02-02T15:42:02.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.174+0000 7efc4cc00700 1 -- 172.21.15.1:0/3074811420 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efc3800cbd0 con 0x7efc50073a00 2024-02-02T15:42:02.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.174+0000 7efc4cc00700 1 -- 172.21.15.1:0/3074811420 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc38018680 con 0x7efc50073a00 2024-02-02T15:42:02.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.175+0000 7efc57000700 1 -- 172.21.15.1:0/3074811420 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc5006bd80 con 0x7efc50073a00 2024-02-02T15:42:02.175 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.175+0000 7efc57000700 1 -- 172.21.15.1:0/3074811420 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc5006ee80 con 0x7efc50073a00 2024-02-02T15:42:02.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.180+0000 7efc57000700 1 -- 172.21.15.1:0/3074811420 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc5006bfa0 con 0x7efc50073a00 2024-02-02T15:42:02.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.180+0000 7efc4cc00700 1 -- 172.21.15.1:0/3074811420 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7efc3800cd40 con 0x7efc50073a00 2024-02-02T15:42:02.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.183+0000 7efc4cc00700 1 --2- 172.21.15.1:0/3074811420 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7efc300a08a0 0x7efc300a2d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:02.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.184+0000 7efc4cc00700 1 -- 172.21.15.1:0/3074811420 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9453+0+0 (secure 0 0 0) 0x7efc38014070 con 0x7efc50073a00 2024-02-02T15:42:02.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.185+0000 7efc4fe00700 1 --2- 172.21.15.1:0/3074811420 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7efc300a08a0 0x7efc300a2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:02.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.186+0000 7efc4fe00700 1 --2- 172.21.15.1:0/3074811420 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7efc300a08a0 0x7efc300a2d60 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7efc400098e0 tx=0x7efc4000b690 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:02.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.191+0000 7efc4cc00700 1 -- 172.21.15.1:0/3074811420 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7efc38052110 con 0x7efc50073a00 2024-02-02T15:42:02.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.461+0000 7efc57000700 1 -- 172.21.15.1:0/3074811420 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efc501172d0 con 0x7efc50073a00 2024-02-02T15:42:02.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.462+0000 7efc4cc00700 1 -- 172.21.15.1:0/3074811420 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+10901 (secure 0 0 0) 0x7efc5006bfa0 con 0x7efc50073a00 2024-02-02T15:42:02.462 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:42:02.462 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":61,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2922240448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2922240448},{"type":"v1","addr":"172.21.15.1:6835","nonce":2922240448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2729497431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2729497431},{"type":"v1","addr":"172.21.15.1:6837","nonce":2729497431}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":61,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-02T15:24:47.761435+0000","modified":"2024-02-02T15:41:28.509766+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24529,"mds_2":24518,"mds_3":24514,"mds_4":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14517":{"gid":14517,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/1218303683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":1218303683},{"type":"v1","addr":"172.21.15.1:6839","nonce":1218303683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14529":{"gid":14529,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/2059197496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2059197496},{"type":"v1","addr":"172.21.15.1:6841","nonce":2059197496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/7648986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":7648986},{"type":"v1","addr":"172.21.15.142:6833","nonce":7648986}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/3482746094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3482746094},{"type":"v1","addr":"172.21.15.71:6833","nonce":3482746094}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/388207814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":388207814},{"type":"v1","addr":"172.21.15.71:6835","nonce":388207814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6835/1052606547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1052606547},{"type":"v1","addr":"172.21.15.142:6835","nonce":1052606547}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.142:6837/2870289019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2870289019},{"type":"v1","addr":"172.21.15.142:6837","nonce":2870289019}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.71:6837/628016199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":628016199},{"type":"v1","addr":"172.21.15.71:6837","nonce":628016199}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.142:6839/321675421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":321675421},{"type":"v1","addr":"172.21.15.142:6839","nonce":321675421}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/1268086493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1268086493},{"type":"v1","addr":"172.21.15.71:6839","nonce":1268086493}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-02T15:42:02.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.464+0000 7efc57000700 1 -- 172.21.15.1:0/3074811420 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7efc300a08a0 msgr2=0x7efc300a2d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:02.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.464+0000 7efc57000700 1 --2- 172.21.15.1:0/3074811420 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7efc300a08a0 0x7efc300a2d60 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7efc400098e0 tx=0x7efc4000b690 comp rx=0 tx=0).stop 2024-02-02T15:42:02.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.464+0000 7efc57000700 1 -- 172.21.15.1:0/3074811420 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efc50073a00 msgr2=0x7efc50070480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:02.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.464+0000 7efc57000700 1 --2- 172.21.15.1:0/3074811420 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efc50073a00 0x7efc50070480 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7efc3800e9a0 tx=0x7efc3800ed60 comp rx=0 tx=0).stop 2024-02-02T15:42:02.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.464+0000 7efc57000700 1 -- 172.21.15.1:0/3074811420 shutdown_connections 2024-02-02T15:42:02.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.464+0000 7efc57000700 1 --2- 172.21.15.1:0/3074811420 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7efc300a08a0 0x7efc300a2d60 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:02.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.464+0000 7efc57000700 1 --2- 172.21.15.1:0/3074811420 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc50111cc0 0x7efc5006b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:02.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.464+0000 7efc57000700 1 --2- 172.21.15.1:0/3074811420 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efc500730e0 0x7efc5006ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:02.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.464+0000 7efc57000700 1 --2- 172.21.15.1:0/3074811420 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efc50073a00 0x7efc50070480 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:02.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.464+0000 7efc57000700 1 -- 172.21.15.1:0/3074811420 >> 172.21.15.1:0/3074811420 conn(0x7efc500fc760 msgr2=0x7efc500fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:02.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.464+0000 7efc57000700 1 -- 172.21.15.1:0/3074811420 shutdown_connections 2024-02-02T15:42:02.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.464+0000 7efc57000700 1 -- 172.21.15.1:0/3074811420 wait complete. 2024-02-02T15:42:02.465 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 61 2024-02-02T15:42:02.483 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-02-02T15:42:02.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.692+0000 7eff44a00700 1 -- 172.21.15.1:0/2796029698 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff3c100ff0 msgr2=0x7eff3c1013d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:02.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.692+0000 7eff44a00700 1 --2- 172.21.15.1:0/2796029698 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff3c100ff0 0x7eff3c1013d0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7eff2c009990 tx=0x7eff2c009ca0 comp rx=0 tx=0).stop 2024-02-02T15:42:02.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.692+0000 7eff44a00700 1 -- 172.21.15.1:0/2796029698 shutdown_connections 2024-02-02T15:42:02.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.692+0000 7eff44a00700 1 --2- 172.21.15.1:0/2796029698 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7eff3c106110 0x7eff3c10dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:02.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.692+0000 7eff44a00700 1 --2- 172.21.15.1:0/2796029698 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff3c100ff0 0x7eff3c1013d0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:02.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.692+0000 7eff44a00700 1 --2- 172.21.15.1:0/2796029698 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff3c1019a0 0x7eff3c1059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:02.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.692+0000 7eff44a00700 1 -- 172.21.15.1:0/2796029698 >> 172.21.15.1:0/2796029698 conn(0x7eff3c0fc810 msgr2=0x7eff3c0fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:02.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.692+0000 7eff44a00700 1 -- 172.21.15.1:0/2796029698 shutdown_connections 2024-02-02T15:42:02.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.692+0000 7eff44a00700 1 -- 172.21.15.1:0/2796029698 wait complete. 2024-02-02T15:42:02.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.693+0000 7eff44a00700 1 Processor -- start 2024-02-02T15:42:02.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.693+0000 7eff44a00700 1 -- start start 2024-02-02T15:42:02.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.694+0000 7eff44a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff3c100ff0 0x7eff3c073040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:02.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.694+0000 7eff44a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff3c106110 0x7eff3c073580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:02.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.694+0000 7eff44a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7eff3c073ac0 0x7eff3c1a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:02.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.694+0000 7eff44a00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff3c1a5730 con 0x7eff3c100ff0 2024-02-02T15:42:02.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.694+0000 7eff44a00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff3c1a58a0 con 0x7eff3c073ac0 2024-02-02T15:42:02.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.694+0000 7eff44a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff3c1a5a60 con 0x7eff3c106110 2024-02-02T15:42:02.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.694+0000 7eff42000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff3c100ff0 0x7eff3c073040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:02.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.694+0000 7eff41600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff3c106110 0x7eff3c073580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:02.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.694+0000 7eff41600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff3c106110 0x7eff3c073580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:42882/0 (socket says 172.21.15.1:42882) 2024-02-02T15:42:02.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.694+0000 7eff41600700 1 -- 172.21.15.1:0/1052463747 learned_addr learned my addr 172.21.15.1:0/1052463747 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:42:02.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.694+0000 7eff42a00700 1 --2- 172.21.15.1:0/1052463747 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7eff3c073ac0 0x7eff3c1a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:02.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.694+0000 7eff41600700 1 -- 172.21.15.1:0/1052463747 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7eff3c073ac0 msgr2=0x7eff3c1a50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:02.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.694+0000 7eff41600700 1 --2- 172.21.15.1:0/1052463747 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7eff3c073ac0 0x7eff3c1a50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:02.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.694+0000 7eff41600700 1 -- 172.21.15.1:0/1052463747 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff3c100ff0 msgr2=0x7eff3c073040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:02.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.694+0000 7eff41600700 1 --2- 172.21.15.1:0/1052463747 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff3c100ff0 0x7eff3c073040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:02.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.694+0000 7eff41600700 1 -- 172.21.15.1:0/1052463747 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff2c0095f0 con 0x7eff3c106110 2024-02-02T15:42:02.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.694+0000 7eff42000700 1 --2- 172.21.15.1:0/1052463747 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff3c100ff0 0x7eff3c073040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:42:02.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.695+0000 7eff41600700 1 --2- 172.21.15.1:0/1052463747 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff3c106110 0x7eff3c073580 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7eff2400e970 tx=0x7eff2400ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:02.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.695+0000 7eff3aa00700 1 -- 172.21.15.1:0/1052463747 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff24009790 con 0x7eff3c106110 2024-02-02T15:42:02.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.695+0000 7eff44a00700 1 -- 172.21.15.1:0/1052463747 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff3c1a5d40 con 0x7eff3c106110 2024-02-02T15:42:02.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.695+0000 7eff3aa00700 1 -- 172.21.15.1:0/1052463747 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff2400c7e0 con 0x7eff3c106110 2024-02-02T15:42:02.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.695+0000 7eff3aa00700 1 -- 172.21.15.1:0/1052463747 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff240054e0 con 0x7eff3c106110 2024-02-02T15:42:02.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.695+0000 7eff44a00700 1 -- 172.21.15.1:0/1052463747 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff3c1a61f0 con 0x7eff3c106110 2024-02-02T15:42:02.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.699+0000 7eff3aa00700 1 -- 172.21.15.1:0/1052463747 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7eff2400c950 con 0x7eff3c106110 2024-02-02T15:42:02.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.699+0000 7eff5861fb80 1 Processor -- start 2024-02-02T15:42:02.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.700+0000 7eff5861fb80 1 -- start start 2024-02-02T15:42:02.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.700+0000 7eff5861fb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5584d53c4010 0x5584d53c43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:02.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.700+0000 7eff5861fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584d53c4930 0x5584d53cf2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:02.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.700+0000 7eff5861fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5584d53cf7e0 0x5584d53d1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:02.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.700+0000 7eff5861fb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584d53384c0 con 0x5584d53c4010 2024-02-02T15:42:02.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.700+0000 7eff5861fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584d531caa0 con 0x5584d53cf7e0 2024-02-02T15:42:02.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.700+0000 7eff5861fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584d531a460 con 0x5584d53c4930 2024-02-02T15:42:02.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.700+0000 7eff42a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5584d53c4010 0x5584d53c43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:02.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.700+0000 7eff42000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584d53c4930 0x5584d53cf2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:02.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.700+0000 7eff42a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5584d53c4010 0x5584d53c43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49054/0 (socket says 172.21.15.1:49054) 2024-02-02T15:42:02.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.700+0000 7eff42a00700 1 -- 172.21.15.1:0/2012623281 learned_addr learned my addr 172.21.15.1:0/2012623281 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:42:02.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.701+0000 7eff41600700 1 --2- 172.21.15.1:0/2012623281 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5584d53cf7e0 0x5584d53d1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:02.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.701+0000 7eff42000700 1 --2- 172.21.15.1:0/2012623281 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584d53c4930 0x5584d53cf2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:42890/0 (socket says 172.21.15.1:42890) 2024-02-02T15:42:02.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.701+0000 7eff42000700 1 -- 172.21.15.1:0/2012623281 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5584d53cf7e0 msgr2=0x5584d53d1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:02.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.701+0000 7eff42000700 1 --2- 172.21.15.1:0/2012623281 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5584d53cf7e0 0x5584d53d1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:02.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.701+0000 7eff42000700 1 -- 172.21.15.1:0/2012623281 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5584d53c4010 msgr2=0x5584d53c43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:02.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.701+0000 7eff42000700 1 --2- 172.21.15.1:0/2012623281 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5584d53c4010 0x5584d53c43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:02.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.701+0000 7eff42000700 1 -- 172.21.15.1:0/2012623281 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5584d51973b0 con 0x5584d53c4930 2024-02-02T15:42:02.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.701+0000 7eff42a00700 1 --2- 172.21.15.1:0/2012623281 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5584d53c4010 0x5584d53c43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:42:02.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.701+0000 7eff3aa00700 1 --2- 172.21.15.1:0/1052463747 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7eff1c0a0690 0x7eff1c0a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:02.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.701+0000 7eff42000700 1 --2- 172.21.15.1:0/2012623281 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584d53c4930 0x5584d53cf2a0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7eff2c0098e0 tx=0x7eff2c005ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a8d6db63c0620fa3 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:02.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.701+0000 7eff3aa00700 1 -- 172.21.15.1:0/1052463747 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9453+0+0 (secure 0 0 0) 0x7eff24014070 con 0x7eff3c106110 2024-02-02T15:42:02.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.701+0000 7eff42a00700 1 --2- 172.21.15.1:0/1052463747 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7eff1c0a0690 0x7eff1c0a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:02.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.702+0000 7eff33000700 1 -- 172.21.15.1:0/2012623281 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff2c01d070 con 0x5584d53c4930 2024-02-02T15:42:02.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.702+0000 7eff33000700 1 -- 172.21.15.1:0/2012623281 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff2c01e040 con 0x5584d53c4930 2024-02-02T15:42:02.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.702+0000 7eff33000700 1 -- 172.21.15.1:0/2012623281 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff2c005450 con 0x5584d53c4930 2024-02-02T15:42:02.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.702+0000 7eff42a00700 1 --2- 172.21.15.1:0/1052463747 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7eff1c0a0690 0x7eff1c0a2b50 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7eff34007610 tx=0x7eff3400b040 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:02.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.702+0000 7eff5861fb80 1 -- 172.21.15.1:0/2012623281 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584d53c4930 msgr2=0x5584d53cf2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:02.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.702+0000 7eff5861fb80 1 --2- 172.21.15.1:0/2012623281 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584d53c4930 0x5584d53cf2a0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7eff2c0098e0 tx=0x7eff2c005ea0 comp rx=0 tx=0).stop 2024-02-02T15:42:02.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.702+0000 7eff5861fb80 1 -- 172.21.15.1:0/2012623281 shutdown_connections 2024-02-02T15:42:02.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.702+0000 7eff5861fb80 1 --2- 172.21.15.1:0/2012623281 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5584d53cf7e0 0x5584d53d1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:02.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.702+0000 7eff5861fb80 1 --2- 172.21.15.1:0/2012623281 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5584d53c4010 0x5584d53c43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:02.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.702+0000 7eff5861fb80 1 --2- 172.21.15.1:0/2012623281 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584d53c4930 0x5584d53cf2a0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:02.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.702+0000 7eff5861fb80 1 -- 172.21.15.1:0/2012623281 >> 172.21.15.1:0/2012623281 conn(0x5584d5323db0 msgr2=0x5584d53c03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:02.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.702+0000 7eff5861fb80 1 -- 172.21.15.1:0/2012623281 shutdown_connections 2024-02-02T15:42:02.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.703+0000 7eff5861fb80 1 -- 172.21.15.1:0/2012623281 wait complete. 2024-02-02T15:42:02.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.703+0000 7eff5861fb80 1 -- start start 2024-02-02T15:42:02.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.703+0000 7eff5861fb80 1 Processor -- start 2024-02-02T15:42:02.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.704+0000 7eff5861fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584d53c4010 0x5584d53cf150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:02.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.704+0000 7eff5861fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5584d53c4930 0x5584d53cc360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:02.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.704+0000 7eff5861fb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5584d53cf7e0 0x5584d53cc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:02.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.704+0000 7eff5861fb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584d5315d50 con 0x5584d53cf7e0 2024-02-02T15:42:02.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.704+0000 7eff5861fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584d52b1d80 con 0x5584d53c4930 2024-02-02T15:42:02.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.704+0000 7eff5861fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584d52b1a20 con 0x5584d53c4010 2024-02-02T15:42:02.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.704+0000 7eff5861fb80 10 client.0 _ll_register_callbacks cb 0x5584d52b3290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:42:02.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.704+0000 7eff42a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5584d53cf7e0 0x5584d53cc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:02.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.704+0000 7eff42000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584d53c4010 0x5584d53cf150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:02.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.704+0000 7eff42a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5584d53cf7e0 0x5584d53cc8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49060/0 (socket says 172.21.15.1:49060) 2024-02-02T15:42:02.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.704+0000 7eff42a00700 1 -- 172.21.15.1:0/4121457681 learned_addr learned my addr 172.21.15.1:0/4121457681 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:42:02.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.704+0000 7eff41600700 1 --2- 172.21.15.1:0/4121457681 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5584d53c4930 0x5584d53cc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:02.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.705+0000 7eff42000700 1 -- 172.21.15.1:0/4121457681 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5584d53c4930 msgr2=0x5584d53cc360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:02.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.705+0000 7eff42000700 1 --2- 172.21.15.1:0/4121457681 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5584d53c4930 0x5584d53cc360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:02.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.705+0000 7eff42000700 1 -- 172.21.15.1:0/4121457681 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5584d53cf7e0 msgr2=0x5584d53cc8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:02.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.705+0000 7eff42000700 1 --2- 172.21.15.1:0/4121457681 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5584d53cf7e0 0x5584d53cc8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:02.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.705+0000 7eff42000700 1 -- 172.21.15.1:0/4121457681 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff34011330 con 0x5584d53c4010 2024-02-02T15:42:02.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.705+0000 7eff42a00700 1 --2- 172.21.15.1:0/4121457681 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5584d53cf7e0 0x5584d53cc8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:42:02.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.705+0000 7eff42000700 1 --2- 172.21.15.1:0/4121457681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584d53c4010 0x5584d53cf150 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7eff2c00b2e0 tx=0x7eff2c004a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa16967b598c84dd server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:02.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.705+0000 7eff2b400700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:42:02.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.705+0000 7eff2b400700 1 -- 172.21.15.1:0/4121457681 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff2c01d070 con 0x5584d53c4010 2024-02-02T15:42:02.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.705+0000 7eff2b400700 1 -- 172.21.15.1:0/4121457681 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9453+0+0 (secure 0 0 0) 0x7eff2c005210 con 0x5584d53c4010 2024-02-02T15:42:02.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.705+0000 7eff5861fb80 1 -- 172.21.15.1:0/4121457681 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5584d53b80f0 con 0x5584d53c4010 2024-02-02T15:42:02.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.706+0000 7eff2b400700 1 -- 172.21.15.1:0/4121457681 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff2c004db0 con 0x5584d53c4010 2024-02-02T15:42:02.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.706+0000 7eff5861fb80 10 client.29506 fetch_fsmap learned FSMap version 61 2024-02-02T15:42:02.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.706+0000 7eff5861fb80 1 -- 172.21.15.1:0/4121457681 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x7eff3c10e500 con 0x5584d53c4010 2024-02-02T15:42:02.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.706+0000 7eff2b400700 1 -- 172.21.15.1:0/4121457681 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5584d53b80f0 con 0x5584d53c4010 2024-02-02T15:42:02.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.706+0000 7eff2b400700 1 -- 172.21.15.1:0/4121457681 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 61) v1 ==== 7550+0+0 (secure 0 0 0) 0x7eff2c00fb80 con 0x5584d53c4010 2024-02-02T15:42:02.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.706+0000 7eff5861fb80 10 client.29506 fetch_fsmap finished waiting for FSMap version 61 2024-02-02T15:42:02.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.706+0000 7eff5861fb80 10 client.29506 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-02-02T15:42:02.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.706+0000 7eff5861fb80 20 client.29506 populate_metadata read hostname 'smithi001' 2024-02-02T15:42:02.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.706+0000 7eff5861fb80 1 --2- 172.21.15.1:0/4121457681 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x5584d53deca0 0x5584d53e1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:02.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.706+0000 7eff5861fb80 4 client.29506 mds_command: new command op to 24502 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:42:02.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.706+0000 7eff5861fb80 1 -- 172.21.15.1:0/4121457681 --> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5584d530edd0 con 0x5584d53deca0 2024-02-02T15:42:02.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.706+0000 7eff41600700 1 --2- 172.21.15.1:0/4121457681 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x5584d53deca0 0x5584d53e1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:02.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.707+0000 7eff41600700 1 --2- 172.21.15.1:0/4121457681 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x5584d53deca0 0x5584d53e1090 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b01c874e017ec862 server_cookie=cfaccbe8744e5038 in_seq=0 out_seq=0 2024-02-02T15:42:02.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.707+0000 7eff2b400700 10 client.29506 ms_handle_connect on v2:172.21.15.142:6834/1052606547 2024-02-02T15:42:02.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.708+0000 7eff2b400700 1 -- 172.21.15.1:0/4121457681 <== mds.4 v2:172.21.15.142:6834/1052606547 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5584d530edd0 con 0x5584d53deca0 2024-02-02T15:42:02.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.708+0000 7eff2b400700 10 client.29506 handle_command_reply: tid=0 2024-02-02T15:42:02.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.708+0000 7eff5861fb80 1 client.29506 shutdown 2024-02-02T15:42:02.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.708+0000 7eff5861fb80 1 -- 172.21.15.1:0/4121457681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584d53c4010 msgr2=0x5584d53cf150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:02.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.708+0000 7eff5861fb80 1 --2- 172.21.15.1:0/4121457681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584d53c4010 0x5584d53cf150 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7eff2c00b2e0 tx=0x7eff2c004a90 comp rx=0 tx=0).stop 2024-02-02T15:42:02.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.708+0000 7eff5861fb80 1 -- 172.21.15.1:0/4121457681 shutdown_connections 2024-02-02T15:42:02.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.708+0000 7eff5861fb80 1 --2- 172.21.15.1:0/4121457681 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x5584d53deca0 0x5584d53e1090 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:02.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.708+0000 7eff5861fb80 1 --2- 172.21.15.1:0/4121457681 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5584d53cf7e0 0x5584d53cc8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:02.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.708+0000 7eff5861fb80 1 --2- 172.21.15.1:0/4121457681 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584d53c4010 0x5584d53cf150 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:02.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.708+0000 7eff5861fb80 1 --2- 172.21.15.1:0/4121457681 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5584d53c4930 0x5584d53cc360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:02.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.708+0000 7eff2b400700 0 client.29506 ms_handle_reset on v2:172.21.15.142:6834/1052606547 2024-02-02T15:42:02.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.708+0000 7eff5861fb80 1 -- 172.21.15.1:0/4121457681 >> 172.21.15.1:0/4121457681 conn(0x5584d5323db0 msgr2=0x5584d53bffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:02.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.708+0000 7eff5861fb80 1 -- 172.21.15.1:0/4121457681 shutdown_connections 2024-02-02T15:42:02.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.708+0000 7eff5861fb80 1 -- 172.21.15.1:0/4121457681 wait complete. 2024-02-02T15:42:02.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.708+0000 7eff5861fb80 20 client.29506 trim_cache size 0 max 16384 2024-02-02T15:42:02.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.722+0000 7eff5861fb80 1 Processor -- start 2024-02-02T15:42:02.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.722+0000 7eff5861fb80 1 -- start start 2024-02-02T15:42:02.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.722+0000 7eff5861fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584d53c4010 0x5584d53d6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:02.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.722+0000 7eff5861fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5584d53c4930 0x5584d53cec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:02.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.722+0000 7eff5861fb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5584d53cf7e0 0x5584d53bfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:02.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.722+0000 7eff5861fb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584d51996a0 con 0x5584d53cf7e0 2024-02-02T15:42:02.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.722+0000 7eff5861fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584d53b7fc0 con 0x5584d53c4930 2024-02-02T15:42:02.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.722+0000 7eff5861fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584d53dba90 con 0x5584d53c4010 2024-02-02T15:42:02.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.722+0000 7eff42a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5584d53cf7e0 0x5584d53bfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:02.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.722+0000 7eff41600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5584d53c4930 0x5584d53cec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:02.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.722+0000 7eff42000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584d53c4010 0x5584d53d6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:02.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.722+0000 7eff42a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5584d53cf7e0 0x5584d53bfcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49070/0 (socket says 172.21.15.1:49070) 2024-02-02T15:42:02.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.722+0000 7eff42a00700 1 -- 172.21.15.1:0/1928302045 learned_addr learned my addr 172.21.15.1:0/1928302045 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:42:02.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.723+0000 7eff41600700 1 -- 172.21.15.1:0/1928302045 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584d53c4010 msgr2=0x5584d53d6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:02.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.723+0000 7eff41600700 1 --2- 172.21.15.1:0/1928302045 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584d53c4010 0x5584d53d6340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:02.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.723+0000 7eff41600700 1 -- 172.21.15.1:0/1928302045 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5584d53cf7e0 msgr2=0x5584d53bfcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:02.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.723+0000 7eff41600700 1 --2- 172.21.15.1:0/1928302045 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5584d53cf7e0 0x5584d53bfcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:02.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.723+0000 7eff41600700 1 -- 172.21.15.1:0/1928302045 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5584d5143b60 con 0x5584d53c4930 2024-02-02T15:42:02.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.723+0000 7eff41600700 1 --2- 172.21.15.1:0/1928302045 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5584d53c4930 0x5584d53cec80 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7eff240c96f0 tx=0x7eff24084040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac0137fed8046a56 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:02.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.723+0000 7eff33a00700 1 -- 172.21.15.1:0/1928302045 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff2409d130 con 0x5584d53c4930 2024-02-02T15:42:02.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.723+0000 7eff33a00700 1 -- 172.21.15.1:0/1928302045 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff24099000 con 0x5584d53c4930 2024-02-02T15:42:02.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.723+0000 7eff33a00700 1 -- 172.21.15.1:0/1928302045 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff2409d2f0 con 0x5584d53c4930 2024-02-02T15:42:02.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.724+0000 7eff5861fb80 1 -- 172.21.15.1:0/1928302045 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5584d53c4930 msgr2=0x5584d53cec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:02.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.724+0000 7eff5861fb80 1 --2- 172.21.15.1:0/1928302045 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5584d53c4930 0x5584d53cec80 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7eff240c96f0 tx=0x7eff24084040 comp rx=0 tx=0).stop 2024-02-02T15:42:02.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.724+0000 7eff5861fb80 1 -- 172.21.15.1:0/1928302045 shutdown_connections 2024-02-02T15:42:02.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.724+0000 7eff5861fb80 1 --2- 172.21.15.1:0/1928302045 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5584d53cf7e0 0x5584d53bfcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:02.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.724+0000 7eff5861fb80 1 --2- 172.21.15.1:0/1928302045 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584d53c4010 0x5584d53d6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:02.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.724+0000 7eff5861fb80 1 --2- 172.21.15.1:0/1928302045 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5584d53c4930 0x5584d53cec80 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:02.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.724+0000 7eff5861fb80 1 -- 172.21.15.1:0/1928302045 >> 172.21.15.1:0/1928302045 conn(0x5584d5323db0 msgr2=0x5584d53bf5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:02.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.724+0000 7eff5861fb80 1 -- 172.21.15.1:0/1928302045 shutdown_connections 2024-02-02T15:42:02.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.724+0000 7eff5861fb80 1 -- 172.21.15.1:0/1928302045 wait complete. 2024-02-02T15:42:02.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.724+0000 7eff5861fb80 1 -- start start 2024-02-02T15:42:02.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.725+0000 7eff5861fb80 1 Processor -- start 2024-02-02T15:42:02.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.725+0000 7eff5861fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5584d53c4010 0x5584d54248a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:02.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.725+0000 7eff5861fb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5584d53c4930 0x5584d5424de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:02.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.725+0000 7eff5861fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584d53cf7e0 0x5584d542f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:02.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.725+0000 7eff5861fb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584d542fc70 con 0x5584d53c4930 2024-02-02T15:42:02.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.725+0000 7eff5861fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584d542fdb0 con 0x5584d53c4010 2024-02-02T15:42:02.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.725+0000 7eff5861fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584d542ff20 con 0x5584d53cf7e0 2024-02-02T15:42:02.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.725+0000 7eff5861fb80 10 client.0 _ll_register_callbacks cb 0x5584d52b3290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:42:02.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.725+0000 7eff41600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5584d53c4930 0x5584d5424de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:02.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.725+0000 7eff41600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5584d53c4930 0x5584d5424de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49086/0 (socket says 172.21.15.1:49086) 2024-02-02T15:42:02.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.725+0000 7eff41600700 1 -- 172.21.15.1:0/1481565971 learned_addr learned my addr 172.21.15.1:0/1481565971 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:42:02.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.725+0000 7eff42000700 1 --2- 172.21.15.1:0/1481565971 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5584d53c4010 0x5584d54248a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:02.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.725+0000 7eff42a00700 1 --2- 172.21.15.1:0/1481565971 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584d53cf7e0 0x5584d542f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:02.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.726+0000 7eff41600700 1 -- 172.21.15.1:0/1481565971 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584d53cf7e0 msgr2=0x5584d542f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:02.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.726+0000 7eff41600700 1 --2- 172.21.15.1:0/1481565971 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584d53cf7e0 0x5584d542f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:02.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.726+0000 7eff41600700 1 -- 172.21.15.1:0/1481565971 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5584d53c4010 msgr2=0x5584d54248a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:02.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.726+0000 7eff41600700 1 --2- 172.21.15.1:0/1481565971 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5584d53c4010 0x5584d54248a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:02.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.726+0000 7eff41600700 1 -- 172.21.15.1:0/1481565971 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff3c1a61f0 con 0x5584d53c4930 2024-02-02T15:42:02.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.726+0000 7eff41600700 1 --2- 172.21.15.1:0/1481565971 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5584d53c4930 0x5584d5424de0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7eff24009230 tx=0x7eff24005c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7461caa2aef7bbbe server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:02.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.726+0000 7eff2b400700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:42:02.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.726+0000 7eff2b400700 1 -- 172.21.15.1:0/1481565971 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff24098e20 con 0x5584d53c4930 2024-02-02T15:42:02.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.726+0000 7eff2b400700 1 -- 172.21.15.1:0/1481565971 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9453+0+0 (secure 0 0 0) 0x7eff240c9e40 con 0x5584d53c4930 2024-02-02T15:42:02.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.726+0000 7eff5861fb80 1 -- 172.21.15.1:0/1481565971 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7eff2c01e040 con 0x5584d53c4930 2024-02-02T15:42:02.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.727+0000 7eff2b400700 1 -- 172.21.15.1:0/1481565971 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff2400cc50 con 0x5584d53c4930 2024-02-02T15:42:02.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.727+0000 7eff5861fb80 10 client.19401 fetch_fsmap learned FSMap version 61 2024-02-02T15:42:02.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.727+0000 7eff5861fb80 1 -- 172.21.15.1:0/1481565971 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x5584d52dbea0 con 0x5584d53c4930 2024-02-02T15:42:02.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.727+0000 7eff2b400700 1 -- 172.21.15.1:0/1481565971 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff2c01e040 con 0x5584d53c4930 2024-02-02T15:42:02.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.727+0000 7eff2b400700 1 -- 172.21.15.1:0/1481565971 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 61) v1 ==== 7550+0+0 (secure 0 0 0) 0x7eff240c9970 con 0x5584d53c4930 2024-02-02T15:42:02.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.727+0000 7eff5861fb80 10 client.19401 fetch_fsmap finished waiting for FSMap version 61 2024-02-02T15:42:02.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.727+0000 7eff5861fb80 10 client.19401 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-02-02T15:42:02.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.727+0000 7eff5861fb80 20 client.19401 populate_metadata read hostname 'smithi001' 2024-02-02T15:42:02.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.727+0000 7eff5861fb80 1 --2- 172.21.15.1:0/1481565971 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x5584d54313e0 0x5584d5433800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:02.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.727+0000 7eff5861fb80 4 client.19401 mds_command: new command op to 24502 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:42:02.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.727+0000 7eff5861fb80 1 -- 172.21.15.1:0/1481565971 --> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5584d52d4f50 con 0x5584d54313e0 2024-02-02T15:42:02.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.727+0000 7eff42000700 1 --2- 172.21.15.1:0/1481565971 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x5584d54313e0 0x5584d5433800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:02.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.728+0000 7eff42000700 1 --2- 172.21.15.1:0/1481565971 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x5584d54313e0 0x5584d5433800 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a6c4c036e33e2b81 server_cookie=562e88bdb96e1fec in_seq=0 out_seq=0 2024-02-02T15:42:02.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.728+0000 7eff2b400700 10 client.19401 ms_handle_connect on v2:172.21.15.142:6834/1052606547 2024-02-02T15:42:02.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.728+0000 7eff2b400700 1 -- 172.21.15.1:0/1481565971 <== mds.4 v2:172.21.15.142:6834/1052606547 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5584d52d4f50 con 0x5584d54313e0 2024-02-02T15:42:02.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.728+0000 7eff2b400700 10 client.19401 handle_command_reply: tid=0 2024-02-02T15:42:02.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.728+0000 7eff5861fb80 1 client.19401 shutdown 2024-02-02T15:42:02.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.728+0000 7eff5861fb80 1 -- 172.21.15.1:0/1481565971 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5584d53c4930 msgr2=0x5584d5424de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:02.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.728+0000 7eff5861fb80 1 --2- 172.21.15.1:0/1481565971 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5584d53c4930 0x5584d5424de0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7eff24009230 tx=0x7eff24005c80 comp rx=0 tx=0).stop 2024-02-02T15:42:02.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.728+0000 7eff5861fb80 1 -- 172.21.15.1:0/1481565971 shutdown_connections 2024-02-02T15:42:02.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.728+0000 7eff5861fb80 1 --2- 172.21.15.1:0/1481565971 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x5584d54313e0 0x5584d5433800 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:02.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.728+0000 7eff5861fb80 1 --2- 172.21.15.1:0/1481565971 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5584d53cf7e0 0x5584d542f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:02.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.728+0000 7eff5861fb80 1 --2- 172.21.15.1:0/1481565971 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5584d53c4010 0x5584d54248a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:02.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.728+0000 7eff5861fb80 1 --2- 172.21.15.1:0/1481565971 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5584d53c4930 0x5584d5424de0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:02.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.728+0000 7eff2b400700 0 client.19401 ms_handle_reset on v2:172.21.15.142:6834/1052606547 2024-02-02T15:42:02.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.728+0000 7eff5861fb80 1 -- 172.21.15.1:0/1481565971 >> 172.21.15.1:0/1481565971 conn(0x5584d5323db0 msgr2=0x5584d53bfe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:02.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.728+0000 7eff5861fb80 1 -- 172.21.15.1:0/1481565971 shutdown_connections 2024-02-02T15:42:02.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.728+0000 7eff5861fb80 1 -- 172.21.15.1:0/1481565971 wait complete. 2024-02-02T15:42:02.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.729+0000 7eff5861fb80 20 client.19401 trim_cache size 0 max 16384 2024-02-02T15:42:02.734 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:42:02.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.729+0000 7eff44a00700 1 -- 172.21.15.1:0/1052463747 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7eff1c0a0690 msgr2=0x7eff1c0a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:02.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.729+0000 7eff44a00700 1 --2- 172.21.15.1:0/1052463747 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7eff1c0a0690 0x7eff1c0a2b50 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7eff34007610 tx=0x7eff3400b040 comp rx=0 tx=0).stop 2024-02-02T15:42:02.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.729+0000 7eff44a00700 1 -- 172.21.15.1:0/1052463747 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff3c106110 msgr2=0x7eff3c073580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:02.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.729+0000 7eff44a00700 1 --2- 172.21.15.1:0/1052463747 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff3c106110 0x7eff3c073580 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7eff2400e970 tx=0x7eff2400ec80 comp rx=0 tx=0).stop 2024-02-02T15:42:02.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.729+0000 7eff44a00700 1 -- 172.21.15.1:0/1052463747 shutdown_connections 2024-02-02T15:42:02.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.729+0000 7eff44a00700 1 --2- 172.21.15.1:0/1052463747 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7eff1c0a0690 0x7eff1c0a2b50 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:02.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.729+0000 7eff44a00700 1 --2- 172.21.15.1:0/1052463747 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7eff3c073ac0 0x7eff3c1a50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:02.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.729+0000 7eff44a00700 1 --2- 172.21.15.1:0/1052463747 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff3c100ff0 0x7eff3c073040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:02.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.729+0000 7eff44a00700 1 --2- 172.21.15.1:0/1052463747 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff3c106110 0x7eff3c073580 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:02.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.729+0000 7eff44a00700 1 -- 172.21.15.1:0/1052463747 >> 172.21.15.1:0/1052463747 conn(0x7eff3c0fc810 msgr2=0x7eff3c0fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:02.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.729+0000 7eff44a00700 1 -- 172.21.15.1:0/1052463747 shutdown_connections 2024-02-02T15:42:02.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.729+0000 7eff44a00700 1 -- 172.21.15.1:0/1052463747 wait complete. 2024-02-02T15:42:02.745 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:42:02 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/3074811420' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:42:02.748 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-02-02T15:42:02.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:42:02 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/3074811420' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:42:02.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:42:02 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/3074811420' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:42:02.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.953+0000 7fd40a200700 1 -- 172.21.15.1:0/309444086 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd4041019b0 msgr2=0x7fd4041059f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:02.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.953+0000 7fd40a200700 1 --2- 172.21.15.1:0/309444086 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd4041019b0 0x7fd4041059f0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec009990 tx=0x7fd3ec009ca0 comp rx=0 tx=0).stop 2024-02-02T15:42:02.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.953+0000 7fd40a200700 1 -- 172.21.15.1:0/309444086 shutdown_connections 2024-02-02T15:42:02.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.953+0000 7fd40a200700 1 --2- 172.21.15.1:0/309444086 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd404106120 0x7fd40410dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:02.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.953+0000 7fd40a200700 1 --2- 172.21.15.1:0/309444086 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd404101000 0x7fd4041013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:02.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.953+0000 7fd40a200700 1 --2- 172.21.15.1:0/309444086 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd4041019b0 0x7fd4041059f0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:02.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.953+0000 7fd40a200700 1 -- 172.21.15.1:0/309444086 >> 172.21.15.1:0/309444086 conn(0x7fd4040fc860 msgr2=0x7fd4040fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:02.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.954+0000 7fd40a200700 1 -- 172.21.15.1:0/309444086 shutdown_connections 2024-02-02T15:42:02.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.954+0000 7fd40a200700 1 -- 172.21.15.1:0/309444086 wait complete. 2024-02-02T15:42:02.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.954+0000 7fd40a200700 1 Processor -- start 2024-02-02T15:42:02.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.955+0000 7fd40a200700 1 -- start start 2024-02-02T15:42:02.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.955+0000 7fd40a200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd404101000 0x7fd404073040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:02.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.955+0000 7fd40a200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd404106120 0x7fd404073580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:02.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.955+0000 7fd40a200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd404073ac0 0x7fd4041a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:02.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.955+0000 7fd40a200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4041a57c0 con 0x7fd404101000 2024-02-02T15:42:02.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.955+0000 7fd40a200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4041a5930 con 0x7fd404106120 2024-02-02T15:42:02.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.955+0000 7fd40a200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4041a5af0 con 0x7fd404073ac0 2024-02-02T15:42:02.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.955+0000 7fd403400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd404101000 0x7fd404073040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:02.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.955+0000 7fd403400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd404101000 0x7fd404073040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49732/0 (socket says 172.21.15.1:49732) 2024-02-02T15:42:02.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.955+0000 7fd403400700 1 -- 172.21.15.1:0/2367349574 learned_addr learned my addr 172.21.15.1:0/2367349574 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:42:02.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.955+0000 7fd402a00700 1 --2- 172.21.15.1:0/2367349574 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd404106120 0x7fd404073580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:02.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.956+0000 7fd403e00700 1 --2- 172.21.15.1:0/2367349574 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd404073ac0 0x7fd4041a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:02.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.956+0000 7fd402a00700 1 -- 172.21.15.1:0/2367349574 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd404073ac0 msgr2=0x7fd4041a5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:02.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.956+0000 7fd402a00700 1 --2- 172.21.15.1:0/2367349574 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd404073ac0 0x7fd4041a5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:02.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.956+0000 7fd402a00700 1 -- 172.21.15.1:0/2367349574 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd404101000 msgr2=0x7fd404073040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:02.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.956+0000 7fd402a00700 1 --2- 172.21.15.1:0/2367349574 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd404101000 0x7fd404073040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:02.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.956+0000 7fd402a00700 1 -- 172.21.15.1:0/2367349574 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3ec0095f0 con 0x7fd404106120 2024-02-02T15:42:02.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.956+0000 7fd403400700 1 --2- 172.21.15.1:0/2367349574 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd404101000 0x7fd404073040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:42:02.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.956+0000 7fd402a00700 1 --2- 172.21.15.1:0/2367349574 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd404106120 0x7fd404073580 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec000c00 tx=0x7fd3ec005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:02.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.957+0000 7fd3fbe00700 1 -- 172.21.15.1:0/2367349574 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3ec01d070 con 0x7fd404106120 2024-02-02T15:42:02.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.957+0000 7fd3fbe00700 1 -- 172.21.15.1:0/2367349574 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd3ec021410 con 0x7fd404106120 2024-02-02T15:42:02.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.957+0000 7fd40a200700 1 -- 172.21.15.1:0/2367349574 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4041a5d70 con 0x7fd404106120 2024-02-02T15:42:02.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.957+0000 7fd3fbe00700 1 -- 172.21.15.1:0/2367349574 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3ec017650 con 0x7fd404106120 2024-02-02T15:42:02.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.957+0000 7fd40a200700 1 -- 172.21.15.1:0/2367349574 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4041a6190 con 0x7fd404106120 2024-02-02T15:42:02.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.960+0000 7fd3fbe00700 1 -- 172.21.15.1:0/2367349574 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7fd3ec021580 con 0x7fd404106120 2024-02-02T15:42:02.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.961+0000 7fd41dddeb80 1 Processor -- start 2024-02-02T15:42:02.961 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.961+0000 7fd41dddeb80 1 -- start start 2024-02-02T15:42:02.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.961+0000 7fd41dddeb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560f403fd010 0x560f403fd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:02.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.961+0000 7fd41dddeb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f403fd930 0x560f404082a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:02.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.961+0000 7fd41dddeb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560f404087e0 0x560f4040abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:02.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.961+0000 7fd3fbe00700 1 --2- 172.21.15.1:0/2367349574 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fd3e40a0690 0x7fd3e40a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:02.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.961+0000 7fd41dddeb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f403714c0 con 0x560f404087e0 2024-02-02T15:42:02.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.961+0000 7fd41dddeb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f40355aa0 con 0x560f403fd010 2024-02-02T15:42:02.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.961+0000 7fd41dddeb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f40353460 con 0x560f403fd930 2024-02-02T15:42:02.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.962+0000 7fd3fbe00700 1 -- 172.21.15.1:0/2367349574 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fd3ec0cd430 con 0x7fd404106120 2024-02-02T15:42:02.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.962+0000 7fd403e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560f403fd010 0x560f403fd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:02.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.962+0000 7fd403e00700 1 --2- 172.21.15.1:0/2367349574 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fd3e40a0690 0x7fd3e40a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:02.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.962+0000 7fd403e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560f403fd010 0x560f403fd3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59164/0 (socket says 172.21.15.1:59164) 2024-02-02T15:42:02.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.962+0000 7fd403e00700 1 -- 172.21.15.1:0/65902626 learned_addr learned my addr 172.21.15.1:0/65902626 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:42:02.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.962+0000 7fd402a00700 1 --2- 172.21.15.1:0/65902626 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560f404087e0 0x560f4040abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:02.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.962+0000 7fd403400700 1 --2- 172.21.15.1:0/65902626 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f403fd930 0x560f404082a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:02.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.962+0000 7fd402a00700 1 -- 172.21.15.1:0/65902626 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f403fd930 msgr2=0x560f404082a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:02.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.962+0000 7fd402a00700 1 --2- 172.21.15.1:0/65902626 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f403fd930 0x560f404082a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:02.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.962+0000 7fd402a00700 1 -- 172.21.15.1:0/65902626 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560f403fd010 msgr2=0x560f403fd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:02.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.962+0000 7fd402a00700 1 --2- 172.21.15.1:0/65902626 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560f403fd010 0x560f403fd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:02.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.962+0000 7fd402a00700 1 -- 172.21.15.1:0/65902626 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f401d03b0 con 0x560f404087e0 2024-02-02T15:42:02.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.963+0000 7fd403e00700 1 --2- 172.21.15.1:0/2367349574 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fd3e40a0690 0x7fd3e40a2b50 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc008d00 tx=0x7fd3fc0090c0 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:02.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.963+0000 7fd402a00700 1 --2- 172.21.15.1:0/65902626 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560f404087e0 0x560f4040abd0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec0ca780 tx=0x7fd3ec09a3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e58e44e2a8638e93 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:02.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.963+0000 7fd3f3e00700 1 -- 172.21.15.1:0/65902626 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3ec099f40 con 0x560f404087e0 2024-02-02T15:42:02.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.963+0000 7fd3f3e00700 1 -- 172.21.15.1:0/65902626 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd3ec09ace0 con 0x560f404087e0 2024-02-02T15:42:02.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.963+0000 7fd3f3e00700 1 -- 172.21.15.1:0/65902626 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3ec098a10 con 0x560f404087e0 2024-02-02T15:42:02.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.963+0000 7fd41dddeb80 1 -- 172.21.15.1:0/65902626 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560f404087e0 msgr2=0x560f4040abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:02.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.963+0000 7fd41dddeb80 1 --2- 172.21.15.1:0/65902626 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560f404087e0 0x560f4040abd0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec0ca780 tx=0x7fd3ec09a3b0 comp rx=0 tx=0).stop 2024-02-02T15:42:02.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.964+0000 7fd41dddeb80 1 -- 172.21.15.1:0/65902626 shutdown_connections 2024-02-02T15:42:02.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.964+0000 7fd41dddeb80 1 --2- 172.21.15.1:0/65902626 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560f404087e0 0x560f4040abd0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:02.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.964+0000 7fd41dddeb80 1 --2- 172.21.15.1:0/65902626 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560f403fd010 0x560f403fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:02.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.964+0000 7fd41dddeb80 1 --2- 172.21.15.1:0/65902626 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f403fd930 0x560f404082a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:02.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.964+0000 7fd41dddeb80 1 -- 172.21.15.1:0/65902626 >> 172.21.15.1:0/65902626 conn(0x560f4035cdb0 msgr2=0x560f403f93b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:02.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.964+0000 7fd41dddeb80 1 -- 172.21.15.1:0/65902626 shutdown_connections 2024-02-02T15:42:02.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.964+0000 7fd41dddeb80 1 -- 172.21.15.1:0/65902626 wait complete. 2024-02-02T15:42:02.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.964+0000 7fd41dddeb80 1 -- start start 2024-02-02T15:42:02.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.964+0000 7fd41dddeb80 1 Processor -- start 2024-02-02T15:42:02.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.965+0000 7fd41dddeb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560f403fd010 0x560f40408bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:02.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.965+0000 7fd41dddeb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560f403fd930 0x560f40409100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:02.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.965+0000 7fd41dddeb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f404087e0 0x560f40409640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:02.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.965+0000 7fd41dddeb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f4034ed50 con 0x560f403fd930 2024-02-02T15:42:02.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.965+0000 7fd41dddeb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f402ead80 con 0x560f403fd010 2024-02-02T15:42:02.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.965+0000 7fd41dddeb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f402eaa20 con 0x560f404087e0 2024-02-02T15:42:02.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.965+0000 7fd41dddeb80 10 client.0 _ll_register_callbacks cb 0x560f402ec290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:42:02.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.965+0000 7fd402a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560f403fd930 0x560f40409100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:02.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.965+0000 7fd403400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560f403fd010 0x560f40408bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:02.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.965+0000 7fd403400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560f403fd010 0x560f40408bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59172/0 (socket says 172.21.15.1:59172) 2024-02-02T15:42:02.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.965+0000 7fd403400700 1 -- 172.21.15.1:0/277797595 learned_addr learned my addr 172.21.15.1:0/277797595 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:42:02.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.965+0000 7fd403e00700 1 --2- 172.21.15.1:0/277797595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f404087e0 0x560f40409640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:02.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.965+0000 7fd403400700 1 -- 172.21.15.1:0/277797595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f404087e0 msgr2=0x560f40409640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:02.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.966+0000 7fd403400700 1 --2- 172.21.15.1:0/277797595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f404087e0 0x560f40409640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:02.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.966+0000 7fd403400700 1 -- 172.21.15.1:0/277797595 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560f403fd930 msgr2=0x560f40409100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:02.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.966+0000 7fd403400700 1 --2- 172.21.15.1:0/277797595 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560f403fd930 0x560f40409100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:02.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.966+0000 7fd403400700 1 -- 172.21.15.1:0/277797595 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4041a6190 con 0x560f403fd010 2024-02-02T15:42:02.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.966+0000 7fd403400700 1 --2- 172.21.15.1:0/277797595 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560f403fd010 0x560f40408bc0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fd3f4012cd0 tx=0x7fd3f400f3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8455bd14a08c78b6 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:02.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.966+0000 7fd3f0c00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:42:02.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.966+0000 7fd3f0c00700 1 -- 172.21.15.1:0/277797595 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3f4017070 con 0x560f403fd010 2024-02-02T15:42:02.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.966+0000 7fd41dddeb80 1 -- 172.21.15.1:0/277797595 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560f403f10f0 con 0x560f403fd010 2024-02-02T15:42:02.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.966+0000 7fd3f0c00700 1 -- 172.21.15.1:0/277797595 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fd3f400f9b0 con 0x560f403fd010 2024-02-02T15:42:02.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.966+0000 7fd3f0c00700 1 -- 172.21.15.1:0/277797595 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd3f400fb70 con 0x560f403fd010 2024-02-02T15:42:02.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.967+0000 7fd41dddeb80 10 client.29558 fetch_fsmap learned FSMap version 61 2024-02-02T15:42:02.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.967+0000 7fd41dddeb80 1 -- 172.21.15.1:0/277797595 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x560f4017cb60 con 0x560f403fd010 2024-02-02T15:42:02.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.967+0000 7fd3f0c00700 1 -- 172.21.15.1:0/277797595 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560f403f10f0 con 0x560f403fd010 2024-02-02T15:42:02.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.967+0000 7fd3f0c00700 1 -- 172.21.15.1:0/277797595 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 61) v1 ==== 7550+0+0 (secure 0 0 0) 0x7fd3f40039e0 con 0x560f403fd010 2024-02-02T15:42:02.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.967+0000 7fd41dddeb80 10 client.29558 fetch_fsmap finished waiting for FSMap version 61 2024-02-02T15:42:02.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.967+0000 7fd41dddeb80 10 client.29558 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-02-02T15:42:02.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.967+0000 7fd41dddeb80 20 client.29558 populate_metadata read hostname 'smithi001' 2024-02-02T15:42:02.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.967+0000 7fd41dddeb80 1 --2- 172.21.15.1:0/277797595 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x560f40417ea0 0x560f4041a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:02.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.967+0000 7fd41dddeb80 4 client.29558 mds_command: new command op to 24514 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:42:02.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.967+0000 7fd41dddeb80 1 -- 172.21.15.1:0/277797595 --> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560f40347dd0 con 0x560f40417ea0 2024-02-02T15:42:02.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.968+0000 7fd402a00700 1 --2- 172.21.15.1:0/277797595 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x560f40417ea0 0x560f4041a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:02.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.968+0000 7fd402a00700 1 --2- 172.21.15.1:0/277797595 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x560f40417ea0 0x560f4041a2c0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6d86cf582513b007 server_cookie=9d7f38aea0ca5caf in_seq=0 out_seq=0 2024-02-02T15:42:02.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.968+0000 7fd3f0c00700 10 client.29558 ms_handle_connect on v2:172.21.15.142:6836/2870289019 2024-02-02T15:42:02.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.969+0000 7fd3f0c00700 1 -- 172.21.15.1:0/277797595 <== mds.3 v2:172.21.15.142:6836/2870289019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x560f40347dd0 con 0x560f40417ea0 2024-02-02T15:42:02.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.969+0000 7fd3f0c00700 10 client.29558 handle_command_reply: tid=0 2024-02-02T15:42:02.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.969+0000 7fd41dddeb80 1 client.29558 shutdown 2024-02-02T15:42:02.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.969+0000 7fd41dddeb80 1 -- 172.21.15.1:0/277797595 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560f403fd010 msgr2=0x560f40408bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:02.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.969+0000 7fd41dddeb80 1 --2- 172.21.15.1:0/277797595 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560f403fd010 0x560f40408bc0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fd3f4012cd0 tx=0x7fd3f400f3c0 comp rx=0 tx=0).stop 2024-02-02T15:42:02.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.969+0000 7fd41dddeb80 1 -- 172.21.15.1:0/277797595 shutdown_connections 2024-02-02T15:42:02.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.969+0000 7fd41dddeb80 1 --2- 172.21.15.1:0/277797595 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f404087e0 0x560f40409640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:02.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.969+0000 7fd41dddeb80 1 --2- 172.21.15.1:0/277797595 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560f403fd010 0x560f40408bc0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:02.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.969+0000 7fd41dddeb80 1 --2- 172.21.15.1:0/277797595 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x560f40417ea0 0x560f4041a2c0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:02.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.969+0000 7fd41dddeb80 1 --2- 172.21.15.1:0/277797595 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560f403fd930 0x560f40409100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:02.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.969+0000 7fd3f0c00700 0 client.29558 ms_handle_reset on v2:172.21.15.142:6836/2870289019 2024-02-02T15:42:02.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.969+0000 7fd41dddeb80 1 -- 172.21.15.1:0/277797595 >> 172.21.15.1:0/277797595 conn(0x560f4035cdb0 msgr2=0x560f403f8fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:02.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.969+0000 7fd41dddeb80 1 -- 172.21.15.1:0/277797595 shutdown_connections 2024-02-02T15:42:02.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.969+0000 7fd41dddeb80 1 -- 172.21.15.1:0/277797595 wait complete. 2024-02-02T15:42:02.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.969+0000 7fd41dddeb80 20 client.29558 trim_cache size 0 max 16384 2024-02-02T15:42:02.978 INFO:tasks.workunit.client.0.smithi001.stdout: 10 55127 33.10 MB/sec warmup 95 sec latency 2090.313 ms 2024-02-02T15:42:02.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.983+0000 7fd41dddeb80 1 Processor -- start 2024-02-02T15:42:02.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.983+0000 7fd41dddeb80 1 -- start start 2024-02-02T15:42:02.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.983+0000 7fd41dddeb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560f403fd010 0x560f404092d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:02.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.983+0000 7fd41dddeb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560f403fd930 0x560f404131b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:02.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.983+0000 7fd41dddeb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f404087e0 0x560f404136f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:02.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.983+0000 7fd41dddeb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f401d26a0 con 0x560f403fd930 2024-02-02T15:42:02.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.983+0000 7fd41dddeb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f40407e10 con 0x560f403fd010 2024-02-02T15:42:02.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.983+0000 7fd41dddeb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f40403650 con 0x560f404087e0 2024-02-02T15:42:02.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.983+0000 7fd403400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560f403fd010 0x560f404092d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:02.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.983+0000 7fd402a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560f403fd930 0x560f404131b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:02.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.984+0000 7fd403400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560f403fd010 0x560f404092d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59174/0 (socket says 172.21.15.1:59174) 2024-02-02T15:42:02.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.984+0000 7fd403400700 1 -- 172.21.15.1:0/3262833032 learned_addr learned my addr 172.21.15.1:0/3262833032 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:42:02.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.984+0000 7fd403e00700 1 --2- 172.21.15.1:0/3262833032 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f404087e0 0x560f404136f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:02.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.984+0000 7fd403400700 1 -- 172.21.15.1:0/3262833032 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f404087e0 msgr2=0x560f404136f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:02.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.984+0000 7fd403400700 1 --2- 172.21.15.1:0/3262833032 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f404087e0 0x560f404136f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:02.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.984+0000 7fd403400700 1 -- 172.21.15.1:0/3262833032 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560f403fd930 msgr2=0x560f404131b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:02.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.984+0000 7fd403400700 1 --2- 172.21.15.1:0/3262833032 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560f403fd930 0x560f404131b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:02.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.984+0000 7fd403400700 1 -- 172.21.15.1:0/3262833032 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560f40314ea0 con 0x560f403fd010 2024-02-02T15:42:02.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.984+0000 7fd402a00700 1 --2- 172.21.15.1:0/3262833032 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560f403fd930 0x560f404131b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:42:02.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.984+0000 7fd403400700 1 --2- 172.21.15.1:0/3262833032 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560f403fd010 0x560f404092d0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fd3f4009f80 tx=0x7fd3f4009d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=518c0250b7a93703 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:02.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.984+0000 7fd3f9200700 1 -- 172.21.15.1:0/3262833032 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3f4017070 con 0x560f403fd010 2024-02-02T15:42:02.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.984+0000 7fd3f9200700 1 -- 172.21.15.1:0/3262833032 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd3f4003b40 con 0x560f403fd010 2024-02-02T15:42:02.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.984+0000 7fd3f9200700 1 -- 172.21.15.1:0/3262833032 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3f40052d0 con 0x560f403fd010 2024-02-02T15:42:02.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.985+0000 7fd41dddeb80 1 -- 172.21.15.1:0/3262833032 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560f403fd010 msgr2=0x560f404092d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:02.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.985+0000 7fd41dddeb80 1 --2- 172.21.15.1:0/3262833032 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560f403fd010 0x560f404092d0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fd3f4009f80 tx=0x7fd3f4009d20 comp rx=0 tx=0).stop 2024-02-02T15:42:02.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.985+0000 7fd41dddeb80 1 -- 172.21.15.1:0/3262833032 shutdown_connections 2024-02-02T15:42:02.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.985+0000 7fd41dddeb80 1 --2- 172.21.15.1:0/3262833032 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f404087e0 0x560f404136f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:02.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.985+0000 7fd41dddeb80 1 --2- 172.21.15.1:0/3262833032 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560f403fd010 0x560f404092d0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:02.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.985+0000 7fd41dddeb80 1 --2- 172.21.15.1:0/3262833032 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560f403fd930 0x560f404131b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:02.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.985+0000 7fd41dddeb80 1 -- 172.21.15.1:0/3262833032 >> 172.21.15.1:0/3262833032 conn(0x560f4035cdb0 msgr2=0x560f40408bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:02.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.985+0000 7fd41dddeb80 1 -- 172.21.15.1:0/3262833032 shutdown_connections 2024-02-02T15:42:02.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.985+0000 7fd41dddeb80 1 -- 172.21.15.1:0/3262833032 wait complete. 2024-02-02T15:42:02.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.986+0000 7fd41dddeb80 1 -- start start 2024-02-02T15:42:02.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.986+0000 7fd41dddeb80 1 Processor -- start 2024-02-02T15:42:02.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.986+0000 7fd41dddeb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f403fd010 0x560f4040b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:02.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.986+0000 7fd41dddeb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560f403fd930 0x560f4040bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:02.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.986+0000 7fd41dddeb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560f404087e0 0x560f404689c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:02.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.986+0000 7fd41dddeb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f4040c3c0 con 0x560f404087e0 2024-02-02T15:42:02.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.986+0000 7fd41dddeb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f40468f30 con 0x560f403fd930 2024-02-02T15:42:02.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.986+0000 7fd41dddeb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560f404690f0 con 0x560f403fd010 2024-02-02T15:42:02.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.986+0000 7fd41dddeb80 10 client.0 _ll_register_callbacks cb 0x560f402ec290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:42:02.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.987+0000 7fd403e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560f404087e0 0x560f404689c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:02.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.987+0000 7fd403e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560f404087e0 0x560f404689c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49756/0 (socket says 172.21.15.1:49756) 2024-02-02T15:42:02.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.987+0000 7fd403e00700 1 -- 172.21.15.1:0/2461319484 learned_addr learned my addr 172.21.15.1:0/2461319484 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:42:02.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.987+0000 7fd403400700 1 --2- 172.21.15.1:0/2461319484 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f403fd010 0x560f4040b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:02.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.987+0000 7fd402a00700 1 --2- 172.21.15.1:0/2461319484 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560f403fd930 0x560f4040bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:02.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.987+0000 7fd403e00700 1 -- 172.21.15.1:0/2461319484 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f403fd010 msgr2=0x560f4040b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:02.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.987+0000 7fd403e00700 1 --2- 172.21.15.1:0/2461319484 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f403fd010 0x560f4040b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:02.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.987+0000 7fd403e00700 1 -- 172.21.15.1:0/2461319484 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560f403fd930 msgr2=0x560f4040bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:02.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.987+0000 7fd403e00700 1 --2- 172.21.15.1:0/2461319484 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560f403fd930 0x560f4040bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:02.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.987+0000 7fd403e00700 1 -- 172.21.15.1:0/2461319484 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560f401d03b0 con 0x560f404087e0 2024-02-02T15:42:02.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.987+0000 7fd403e00700 1 --2- 172.21.15.1:0/2461319484 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560f404087e0 0x560f404689c0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc01b700 tx=0x7fd3fc009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8540a25797c40805 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:02.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.987+0000 7fd3f0c00700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:42:02.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.988+0000 7fd3f0c00700 1 -- 172.21.15.1:0/2461319484 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3fc00e070 con 0x560f404087e0 2024-02-02T15:42:02.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.988+0000 7fd41dddeb80 1 -- 172.21.15.1:0/2461319484 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd3ec09ace0 con 0x560f404087e0 2024-02-02T15:42:02.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.988+0000 7fd3f0c00700 1 -- 172.21.15.1:0/2461319484 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fd3fc01be20 con 0x560f404087e0 2024-02-02T15:42:02.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.988+0000 7fd3f0c00700 1 -- 172.21.15.1:0/2461319484 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd3fc008ef0 con 0x560f404087e0 2024-02-02T15:42:02.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.988+0000 7fd41dddeb80 10 client.19419 fetch_fsmap learned FSMap version 61 2024-02-02T15:42:02.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.988+0000 7fd41dddeb80 1 -- 172.21.15.1:0/2461319484 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x560f4017cb60 con 0x560f404087e0 2024-02-02T15:42:02.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.988+0000 7fd3f0c00700 1 -- 172.21.15.1:0/2461319484 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd3ec09ace0 con 0x560f404087e0 2024-02-02T15:42:02.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.988+0000 7fd3f0c00700 1 -- 172.21.15.1:0/2461319484 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 61) v1 ==== 7550+0+0 (secure 0 0 0) 0x7fd3fc0266e0 con 0x560f404087e0 2024-02-02T15:42:02.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.988+0000 7fd41dddeb80 10 client.19419 fetch_fsmap finished waiting for FSMap version 61 2024-02-02T15:42:02.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.988+0000 7fd41dddeb80 10 client.19419 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-02-02T15:42:02.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.988+0000 7fd41dddeb80 20 client.19419 populate_metadata read hostname 'smithi001' 2024-02-02T15:42:02.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.988+0000 7fd41dddeb80 1 --2- 172.21.15.1:0/2461319484 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x560f4046a8a0 0x560f4046ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:02.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.988+0000 7fd41dddeb80 4 client.19419 mds_command: new command op to 24514 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:42:02.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.988+0000 7fd41dddeb80 1 -- 172.21.15.1:0/2461319484 --> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560f4030df50 con 0x560f4046a8a0 2024-02-02T15:42:02.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.989+0000 7fd403400700 1 --2- 172.21.15.1:0/2461319484 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x560f4046a8a0 0x560f4046ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:02.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.989+0000 7fd3f0c00700 10 client.19419 ms_handle_connect on v2:172.21.15.142:6836/2870289019 2024-02-02T15:42:02.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.989+0000 7fd403400700 1 --2- 172.21.15.1:0/2461319484 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x560f4046a8a0 0x560f4046ccc0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=15260fe7972466c3 server_cookie=8435a6bfabd88503 in_seq=0 out_seq=0 2024-02-02T15:42:02.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.990+0000 7fd3f0c00700 1 -- 172.21.15.1:0/2461319484 <== mds.3 v2:172.21.15.142:6836/2870289019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560f4030df50 con 0x560f4046a8a0 2024-02-02T15:42:02.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.990+0000 7fd3f0c00700 10 client.19419 handle_command_reply: tid=0 2024-02-02T15:42:02.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.990+0000 7fd41dddeb80 1 client.19419 shutdown 2024-02-02T15:42:02.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.990+0000 7fd41dddeb80 1 -- 172.21.15.1:0/2461319484 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560f404087e0 msgr2=0x560f404689c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:02.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.990+0000 7fd41dddeb80 1 --2- 172.21.15.1:0/2461319484 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560f404087e0 0x560f404689c0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc01b700 tx=0x7fd3fc009920 comp rx=0 tx=0).stop 2024-02-02T15:42:02.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.990+0000 7fd41dddeb80 1 -- 172.21.15.1:0/2461319484 shutdown_connections 2024-02-02T15:42:02.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.990+0000 7fd41dddeb80 1 --2- 172.21.15.1:0/2461319484 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x560f4046a8a0 0x560f4046ccc0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:02.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.990+0000 7fd41dddeb80 1 --2- 172.21.15.1:0/2461319484 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560f404087e0 0x560f404689c0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:02.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.990+0000 7fd41dddeb80 1 --2- 172.21.15.1:0/2461319484 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560f403fd010 0x560f4040b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:02.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.990+0000 7fd41dddeb80 1 --2- 172.21.15.1:0/2461319484 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560f403fd930 0x560f4040bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:02.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.990+0000 7fd3f0c00700 0 client.19419 ms_handle_reset on v2:172.21.15.142:6836/2870289019 2024-02-02T15:42:02.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.990+0000 7fd41dddeb80 1 -- 172.21.15.1:0/2461319484 >> 172.21.15.1:0/2461319484 conn(0x560f4035cdb0 msgr2=0x560f403f8e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:02.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.990+0000 7fd41dddeb80 1 -- 172.21.15.1:0/2461319484 shutdown_connections 2024-02-02T15:42:02.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.990+0000 7fd41dddeb80 1 -- 172.21.15.1:0/2461319484 wait complete. 2024-02-02T15:42:02.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.990+0000 7fd41dddeb80 20 client.19419 trim_cache size 0 max 16384 2024-02-02T15:42:02.994 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:42:02.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.991+0000 7fd40a200700 1 -- 172.21.15.1:0/2367349574 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fd3e40a0690 msgr2=0x7fd3e40a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:02.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.991+0000 7fd40a200700 1 --2- 172.21.15.1:0/2367349574 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fd3e40a0690 0x7fd3e40a2b50 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc008d00 tx=0x7fd3fc0090c0 comp rx=0 tx=0).stop 2024-02-02T15:42:02.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.991+0000 7fd40a200700 1 -- 172.21.15.1:0/2367349574 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd404106120 msgr2=0x7fd404073580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:02.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.991+0000 7fd40a200700 1 --2- 172.21.15.1:0/2367349574 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd404106120 0x7fd404073580 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec000c00 tx=0x7fd3ec005740 comp rx=0 tx=0).stop 2024-02-02T15:42:02.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.991+0000 7fd40a200700 1 -- 172.21.15.1:0/2367349574 shutdown_connections 2024-02-02T15:42:02.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.991+0000 7fd40a200700 1 --2- 172.21.15.1:0/2367349574 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fd3e40a0690 0x7fd3e40a2b50 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:02.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.991+0000 7fd40a200700 1 --2- 172.21.15.1:0/2367349574 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fd404073ac0 0x7fd4041a5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:02.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.991+0000 7fd40a200700 1 --2- 172.21.15.1:0/2367349574 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fd404101000 0x7fd404073040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:02.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.991+0000 7fd40a200700 1 --2- 172.21.15.1:0/2367349574 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd404106120 0x7fd404073580 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:02.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.991+0000 7fd40a200700 1 -- 172.21.15.1:0/2367349574 >> 172.21.15.1:0/2367349574 conn(0x7fd4040fc860 msgr2=0x7fd4040fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:02.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.991+0000 7fd40a200700 1 -- 172.21.15.1:0/2367349574 shutdown_connections 2024-02-02T15:42:02.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:02.991+0000 7fd40a200700 1 -- 172.21.15.1:0/2367349574 wait complete. 2024-02-02T15:42:03.009 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-02T15:42:03.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.209+0000 7f3fd8200700 1 -- 172.21.15.1:0/1223826850 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3fd0106110 msgr2=0x7f3fd010dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:03.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.209+0000 7f3fd8200700 1 --2- 172.21.15.1:0/1223826850 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3fd0106110 0x7f3fd010dc90 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f3fc800b0a0 tx=0x7f3fc800b3b0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.209+0000 7f3fd8200700 1 -- 172.21.15.1:0/1223826850 shutdown_connections 2024-02-02T15:42:03.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.209+0000 7f3fd8200700 1 --2- 172.21.15.1:0/1223826850 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3fd0106110 0x7f3fd010dc90 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.209+0000 7f3fd8200700 1 --2- 172.21.15.1:0/1223826850 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fd0100ff0 0x7f3fd01013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.209+0000 7f3fd8200700 1 --2- 172.21.15.1:0/1223826850 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3fd01019a0 0x7f3fd01059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.209+0000 7f3fd8200700 1 -- 172.21.15.1:0/1223826850 >> 172.21.15.1:0/1223826850 conn(0x7f3fd00fc810 msgr2=0x7f3fd00fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:03.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.209+0000 7f3fd8200700 1 -- 172.21.15.1:0/1223826850 shutdown_connections 2024-02-02T15:42:03.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.209+0000 7f3fd8200700 1 -- 172.21.15.1:0/1223826850 wait complete. 2024-02-02T15:42:03.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.210+0000 7f3fd8200700 1 Processor -- start 2024-02-02T15:42:03.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.210+0000 7f3fd8200700 1 -- start start 2024-02-02T15:42:03.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.211+0000 7f3fd8200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3fd0100ff0 0x7f3fd0199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:03.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.211+0000 7f3fd8200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fd0106110 0x7f3fd0199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:03.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.211+0000 7f3fd8200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3fd019dbb0 0x7f3fd019e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:03.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.211+0000 7f3fd8200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd019e630 con 0x7f3fd019dbb0 2024-02-02T15:42:03.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.211+0000 7f3fd8200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd019e7a0 con 0x7f3fd0100ff0 2024-02-02T15:42:03.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.211+0000 7f3fd8200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd019e960 con 0x7f3fd0106110 2024-02-02T15:42:03.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.211+0000 7f3fd6200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3fd019dbb0 0x7f3fd019e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:03.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.211+0000 7f3fd5800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3fd0100ff0 0x7f3fd0199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:03.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.211+0000 7f3fd5800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3fd0100ff0 0x7f3fd0199210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59210/0 (socket says 172.21.15.1:59210) 2024-02-02T15:42:03.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.211+0000 7f3fd5800700 1 -- 172.21.15.1:0/3804376095 learned_addr learned my addr 172.21.15.1:0/3804376095 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:42:03.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.211+0000 7f3fd6200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3fd019dbb0 0x7f3fd019e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49770/0 (socket says 172.21.15.1:49770) 2024-02-02T15:42:03.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.211+0000 7f3fd4e00700 1 --2- 172.21.15.1:0/3804376095 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fd0106110 0x7f3fd0199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:03.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.211+0000 7f3fd5800700 1 -- 172.21.15.1:0/3804376095 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fd0106110 msgr2=0x7f3fd0199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:03.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.211+0000 7f3fd5800700 1 --2- 172.21.15.1:0/3804376095 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fd0106110 0x7f3fd0199750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.211+0000 7f3fd5800700 1 -- 172.21.15.1:0/3804376095 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3fd019dbb0 msgr2=0x7f3fd019e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:03.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.211+0000 7f3fd5800700 1 --2- 172.21.15.1:0/3804376095 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3fd019dbb0 0x7f3fd019e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.211+0000 7f3fd5800700 1 -- 172.21.15.1:0/3804376095 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fc8009d00 con 0x7f3fd0100ff0 2024-02-02T15:42:03.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.211+0000 7f3fd4e00700 1 --2- 172.21.15.1:0/3804376095 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fd0106110 0x7f3fd0199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:42:03.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.211+0000 7f3fd6200700 1 --2- 172.21.15.1:0/3804376095 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3fd019dbb0 0x7f3fd019e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:42:03.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.212+0000 7f3fd5800700 1 --2- 172.21.15.1:0/3804376095 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3fd0100ff0 0x7f3fd0199210 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f3fc000e9a0 tx=0x7f3fc000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:03.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.212+0000 7f3fce000700 1 -- 172.21.15.1:0/3804376095 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc0009790 con 0x7f3fd0100ff0 2024-02-02T15:42:03.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.212+0000 7f3fd8200700 1 -- 172.21.15.1:0/3804376095 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fd01a5ec0 con 0x7f3fd0100ff0 2024-02-02T15:42:03.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.212+0000 7f3fce000700 1 -- 172.21.15.1:0/3804376095 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3fc0010430 con 0x7f3fd0100ff0 2024-02-02T15:42:03.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.212+0000 7f3fce000700 1 -- 172.21.15.1:0/3804376095 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc0018680 con 0x7f3fd0100ff0 2024-02-02T15:42:03.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.212+0000 7f3fd8200700 1 -- 172.21.15.1:0/3804376095 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fd01a6340 con 0x7f3fd0100ff0 2024-02-02T15:42:03.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.215+0000 7f3fce000700 1 -- 172.21.15.1:0/3804376095 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f3fc00105a0 con 0x7f3fd0100ff0 2024-02-02T15:42:03.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.216+0000 7f3febd77b80 1 Processor -- start 2024-02-02T15:42:03.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.217+0000 7f3fce000700 1 --2- 172.21.15.1:0/3804376095 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f3fb00a0690 0x7f3fb00a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:03.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.217+0000 7f3febd77b80 1 -- start start 2024-02-02T15:42:03.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.217+0000 7f3fce000700 1 -- 172.21.15.1:0/3804376095 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f3fc0014070 con 0x7f3fd0100ff0 2024-02-02T15:42:03.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.217+0000 7f3fd6200700 1 --2- 172.21.15.1:0/3804376095 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f3fb00a0690 0x7f3fb00a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:03.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.217+0000 7f3febd77b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8d796a010 0x55f8d796a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:03.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.217+0000 7f3febd77b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f8d796a930 0x55f8d79752a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:03.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.217+0000 7f3febd77b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f8d79757e0 0x55f8d7977bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:03.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.217+0000 7f3febd77b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8d78de4c0 con 0x55f8d796a010 2024-02-02T15:42:03.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.217+0000 7f3febd77b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8d78c2aa0 con 0x55f8d79757e0 2024-02-02T15:42:03.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.217+0000 7f3febd77b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8d78c0460 con 0x55f8d796a930 2024-02-02T15:42:03.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.217+0000 7f3fd5800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8d796a010 0x55f8d796a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:03.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.218+0000 7f3fd5800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8d796a010 0x55f8d796a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49774/0 (socket says 172.21.15.1:49774) 2024-02-02T15:42:03.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.218+0000 7f3fd5800700 1 -- 172.21.15.1:0/3884895416 learned_addr learned my addr 172.21.15.1:0/3884895416 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:42:03.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.218+0000 7f3fd6200700 1 --2- 172.21.15.1:0/3884895416 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f8d79757e0 0x55f8d7977bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:03.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.218+0000 7f3fd4e00700 1 --2- 172.21.15.1:0/3884895416 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f8d796a930 0x55f8d79752a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:03.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.218+0000 7f3fd5800700 1 -- 172.21.15.1:0/3884895416 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f8d796a930 msgr2=0x55f8d79752a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:03.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.218+0000 7f3fd5800700 1 --2- 172.21.15.1:0/3884895416 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f8d796a930 0x55f8d79752a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.218+0000 7f3fd5800700 1 -- 172.21.15.1:0/3884895416 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f8d79757e0 msgr2=0x55f8d7977bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:03.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.218+0000 7f3fd5800700 1 --2- 172.21.15.1:0/3884895416 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f8d79757e0 0x55f8d7977bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.218+0000 7f3fd5800700 1 -- 172.21.15.1:0/3884895416 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f8d773d3b0 con 0x55f8d796a010 2024-02-02T15:42:03.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.218+0000 7f3fd6200700 1 --2- 172.21.15.1:0/3884895416 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f8d79757e0 0x55f8d7977bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:42:03.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.218+0000 7f3fd6200700 1 --2- 172.21.15.1:0/3804376095 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f3fb00a0690 0x7f3fb00a2b50 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f3fc80062a0 tx=0x7f3fc8006210 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:03.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.218+0000 7f3fd5800700 1 --2- 172.21.15.1:0/3884895416 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8d796a010 0x55f8d796a3f0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f3fc00a0f40 tx=0x7f3fc0098de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b55e7528c344a015 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:03.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.218+0000 7f3fc6a00700 1 -- 172.21.15.1:0/3884895416 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc0099400 con 0x55f8d796a010 2024-02-02T15:42:03.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.218+0000 7f3fc6a00700 1 -- 172.21.15.1:0/3884895416 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3fc009d230 con 0x55f8d796a010 2024-02-02T15:42:03.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.218+0000 7f3fc6a00700 1 -- 172.21.15.1:0/3884895416 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc00998c0 con 0x55f8d796a010 2024-02-02T15:42:03.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.219+0000 7f3febd77b80 1 -- 172.21.15.1:0/3884895416 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8d796a010 msgr2=0x55f8d796a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:03.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.219+0000 7f3febd77b80 1 --2- 172.21.15.1:0/3884895416 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8d796a010 0x55f8d796a3f0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f3fc00a0f40 tx=0x7f3fc0098de0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.219+0000 7f3febd77b80 1 -- 172.21.15.1:0/3884895416 shutdown_connections 2024-02-02T15:42:03.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.219+0000 7f3febd77b80 1 --2- 172.21.15.1:0/3884895416 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f8d79757e0 0x55f8d7977bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.219+0000 7f3febd77b80 1 --2- 172.21.15.1:0/3884895416 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8d796a010 0x55f8d796a3f0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.219+0000 7f3febd77b80 1 --2- 172.21.15.1:0/3884895416 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f8d796a930 0x55f8d79752a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.219+0000 7f3febd77b80 1 -- 172.21.15.1:0/3884895416 >> 172.21.15.1:0/3884895416 conn(0x55f8d78c9db0 msgr2=0x55f8d79663b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:03.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.219+0000 7f3febd77b80 1 -- 172.21.15.1:0/3884895416 shutdown_connections 2024-02-02T15:42:03.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.219+0000 7f3febd77b80 1 -- 172.21.15.1:0/3884895416 wait complete. 2024-02-02T15:42:03.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.220+0000 7f3febd77b80 1 -- start start 2024-02-02T15:42:03.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.220+0000 7f3febd77b80 1 Processor -- start 2024-02-02T15:42:03.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.221+0000 7f3febd77b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f8d796a010 0x55f8d79751a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:03.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.221+0000 7f3febd77b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8d796a930 0x55f8d7972420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:03.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.221+0000 7f3febd77b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f8d79757e0 0x55f8d7972aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:03.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.221+0000 7f3febd77b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8d78bbd50 con 0x55f8d796a930 2024-02-02T15:42:03.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.221+0000 7f3febd77b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8d7857d80 con 0x55f8d796a010 2024-02-02T15:42:03.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.221+0000 7f3febd77b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8d7857a20 con 0x55f8d79757e0 2024-02-02T15:42:03.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.221+0000 7f3febd77b80 10 client.0 _ll_register_callbacks cb 0x55f8d7859290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:42:03.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.221+0000 7f3fd4e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8d796a930 0x55f8d7972420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:03.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.221+0000 7f3fd5800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f8d796a010 0x55f8d79751a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:03.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.221+0000 7f3fd6200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f8d79757e0 0x55f8d7972aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:03.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.221+0000 7f3fd5800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f8d796a010 0x55f8d79751a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59218/0 (socket says 172.21.15.1:59218) 2024-02-02T15:42:03.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.221+0000 7f3fd5800700 1 -- 172.21.15.1:0/357787560 learned_addr learned my addr 172.21.15.1:0/357787560 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:42:03.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.221+0000 7f3fd6200700 1 -- 172.21.15.1:0/357787560 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f8d796a010 msgr2=0x55f8d79751a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:03.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.221+0000 7f3fd6200700 1 --2- 172.21.15.1:0/357787560 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f8d796a010 0x55f8d79751a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.221+0000 7f3fd6200700 1 -- 172.21.15.1:0/357787560 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8d796a930 msgr2=0x55f8d7972420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:03.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.221+0000 7f3fd6200700 1 --2- 172.21.15.1:0/357787560 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8d796a930 0x55f8d7972420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.221+0000 7f3fd6200700 1 -- 172.21.15.1:0/357787560 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fd01a6340 con 0x55f8d79757e0 2024-02-02T15:42:03.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.222+0000 7f3fd6200700 1 --2- 172.21.15.1:0/357787560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f8d79757e0 0x55f8d7972aa0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f3fc8009fd0 tx=0x7f3fc80099e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3cd46255c3b7308e server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:03.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.222+0000 7f3fbf400700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:42:03.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.222+0000 7f3fbf400700 1 -- 172.21.15.1:0/357787560 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc800e040 con 0x55f8d79757e0 2024-02-02T15:42:03.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.222+0000 7f3febd77b80 1 -- 172.21.15.1:0/357787560 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f8d795e0f0 con 0x55f8d79757e0 2024-02-02T15:42:03.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.222+0000 7f3fbf400700 1 -- 172.21.15.1:0/357787560 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f3fc801c650 con 0x55f8d79757e0 2024-02-02T15:42:03.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.222+0000 7f3fbf400700 1 -- 172.21.15.1:0/357787560 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3fc801c810 con 0x55f8d79757e0 2024-02-02T15:42:03.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.222+0000 7f3febd77b80 10 client.29542 fetch_fsmap learned FSMap version 61 2024-02-02T15:42:03.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.222+0000 7f3febd77b80 1 -- 172.21.15.1:0/357787560 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x55f8d76e9b60 con 0x55f8d79757e0 2024-02-02T15:42:03.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.223+0000 7f3fbf400700 1 -- 172.21.15.1:0/357787560 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f8d795e0f0 con 0x55f8d79757e0 2024-02-02T15:42:03.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.223+0000 7f3fbf400700 1 -- 172.21.15.1:0/357787560 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 61) v1 ==== 7550+0+0 (secure 0 0 0) 0x7f3fc802d030 con 0x55f8d79757e0 2024-02-02T15:42:03.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.223+0000 7f3febd77b80 10 client.29542 fetch_fsmap finished waiting for FSMap version 61 2024-02-02T15:42:03.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.223+0000 7f3febd77b80 10 client.29542 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-02-02T15:42:03.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.223+0000 7f3febd77b80 20 client.29542 populate_metadata read hostname 'smithi001' 2024-02-02T15:42:03.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.223+0000 7f3febd77b80 1 --2- 172.21.15.1:0/357787560 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55f8d79852a0 0x55f8d79876c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:03.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.223+0000 7f3febd77b80 4 client.29542 mds_command: new command op to 24518 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:42:03.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.223+0000 7f3febd77b80 1 -- 172.21.15.1:0/357787560 --> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f8d78b4dd0 con 0x55f8d79852a0 2024-02-02T15:42:03.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.223+0000 7f3fd5800700 1 --2- 172.21.15.1:0/357787560 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55f8d79852a0 0x55f8d79876c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:03.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.224+0000 7f3fd5800700 1 --2- 172.21.15.1:0/357787560 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55f8d79852a0 0x55f8d79876c0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7037d784036e9382 server_cookie=38ae4f831513c24e in_seq=0 out_seq=0 2024-02-02T15:42:03.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.224+0000 7f3fbf400700 10 client.29542 ms_handle_connect on v2:172.21.15.71:6836/628016199 2024-02-02T15:42:03.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.225+0000 7f3fbf400700 1 -- 172.21.15.1:0/357787560 <== mds.2 v2:172.21.15.71:6836/628016199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55f8d78b4dd0 con 0x55f8d79852a0 2024-02-02T15:42:03.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.225+0000 7f3fbf400700 10 client.29542 handle_command_reply: tid=0 2024-02-02T15:42:03.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.225+0000 7f3febd77b80 1 client.29542 shutdown 2024-02-02T15:42:03.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.225+0000 7f3febd77b80 1 -- 172.21.15.1:0/357787560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f8d79757e0 msgr2=0x55f8d7972aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:03.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.225+0000 7f3febd77b80 1 --2- 172.21.15.1:0/357787560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f8d79757e0 0x55f8d7972aa0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f3fc8009fd0 tx=0x7f3fc80099e0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.225+0000 7f3febd77b80 1 -- 172.21.15.1:0/357787560 shutdown_connections 2024-02-02T15:42:03.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.225+0000 7f3febd77b80 1 --2- 172.21.15.1:0/357787560 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55f8d79852a0 0x55f8d79876c0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.225+0000 7f3febd77b80 1 --2- 172.21.15.1:0/357787560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f8d79757e0 0x55f8d7972aa0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.225+0000 7f3febd77b80 1 --2- 172.21.15.1:0/357787560 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f8d796a010 0x55f8d79751a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.225+0000 7f3fbf400700 0 client.29542 ms_handle_reset on v2:172.21.15.71:6836/628016199 2024-02-02T15:42:03.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.225+0000 7f3febd77b80 1 --2- 172.21.15.1:0/357787560 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8d796a930 0x55f8d7972420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.225+0000 7f3febd77b80 1 -- 172.21.15.1:0/357787560 >> 172.21.15.1:0/357787560 conn(0x55f8d78c9db0 msgr2=0x55f8d7965fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:03.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.225+0000 7f3febd77b80 1 -- 172.21.15.1:0/357787560 shutdown_connections 2024-02-02T15:42:03.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.225+0000 7f3febd77b80 1 -- 172.21.15.1:0/357787560 wait complete. 2024-02-02T15:42:03.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.226+0000 7f3febd77b80 20 client.29542 trim_cache size 0 max 16384 2024-02-02T15:42:03.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.239+0000 7f3febd77b80 1 Processor -- start 2024-02-02T15:42:03.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.239+0000 7f3febd77b80 1 -- start start 2024-02-02T15:42:03.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.239+0000 7f3febd77b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8d796a010 0x55f8d796cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:03.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.239+0000 7f3febd77b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f8d796a930 0x55f8d796d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:03.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.239+0000 7f3febd77b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f8d79757e0 0x55f8d796dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:03.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.239+0000 7f3febd77b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8d773f6a0 con 0x55f8d796a010 2024-02-02T15:42:03.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.239+0000 7f3febd77b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8d7967820 con 0x55f8d796a930 2024-02-02T15:42:03.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.239+0000 7f3febd77b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8d79709a0 con 0x55f8d79757e0 2024-02-02T15:42:03.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.239+0000 7f3fd5800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8d796a010 0x55f8d796cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:03.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.240+0000 7f3fd4e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f8d796a930 0x55f8d796d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:03.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.240+0000 7f3fd5800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8d796a010 0x55f8d796cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49790/0 (socket says 172.21.15.1:49790) 2024-02-02T15:42:03.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.240+0000 7f3fd5800700 1 -- 172.21.15.1:0/2088857942 learned_addr learned my addr 172.21.15.1:0/2088857942 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:42:03.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.240+0000 7f3fd4e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f8d796a930 0x55f8d796d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59226/0 (socket says 172.21.15.1:59226) 2024-02-02T15:42:03.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.240+0000 7f3fd6200700 1 --2- 172.21.15.1:0/2088857942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f8d79757e0 0x55f8d796dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:03.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.240+0000 7f3fd5800700 1 -- 172.21.15.1:0/2088857942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f8d79757e0 msgr2=0x55f8d796dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:03.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.240+0000 7f3fd5800700 1 --2- 172.21.15.1:0/2088857942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f8d79757e0 0x55f8d796dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.240+0000 7f3fd5800700 1 -- 172.21.15.1:0/2088857942 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f8d796a930 msgr2=0x55f8d796d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:03.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.240+0000 7f3fd5800700 1 --2- 172.21.15.1:0/2088857942 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f8d796a930 0x55f8d796d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.240+0000 7f3fd5800700 1 -- 172.21.15.1:0/2088857942 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f8d7881ea0 con 0x55f8d796a010 2024-02-02T15:42:03.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.240+0000 7f3fd4e00700 1 --2- 172.21.15.1:0/2088857942 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f8d796a930 0x55f8d796d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:42:03.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.240+0000 7f3fd5800700 1 --2- 172.21.15.1:0/2088857942 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8d796a010 0x55f8d796cab0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f3fc0099100 tx=0x7f3fc0010c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94e223a1419c5572 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:03.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.240+0000 7f3fc7400700 1 -- 172.21.15.1:0/2088857942 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc0099400 con 0x55f8d796a010 2024-02-02T15:42:03.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.240+0000 7f3fc7400700 1 -- 172.21.15.1:0/2088857942 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3fc0010880 con 0x55f8d796a010 2024-02-02T15:42:03.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.241+0000 7f3fc7400700 1 -- 172.21.15.1:0/2088857942 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc00998c0 con 0x55f8d796a010 2024-02-02T15:42:03.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.241+0000 7f3febd77b80 1 -- 172.21.15.1:0/2088857942 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8d796a010 msgr2=0x55f8d796cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:03.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.241+0000 7f3febd77b80 1 --2- 172.21.15.1:0/2088857942 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8d796a010 0x55f8d796cab0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f3fc0099100 tx=0x7f3fc0010c80 comp rx=0 tx=0).stop 2024-02-02T15:42:03.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.241+0000 7f3febd77b80 1 -- 172.21.15.1:0/2088857942 shutdown_connections 2024-02-02T15:42:03.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.241+0000 7f3febd77b80 1 --2- 172.21.15.1:0/2088857942 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f8d79757e0 0x55f8d796dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.241+0000 7f3febd77b80 1 --2- 172.21.15.1:0/2088857942 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8d796a010 0x55f8d796cab0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.241+0000 7f3febd77b80 1 --2- 172.21.15.1:0/2088857942 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f8d796a930 0x55f8d796d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.241+0000 7f3febd77b80 1 -- 172.21.15.1:0/2088857942 >> 172.21.15.1:0/2088857942 conn(0x55f8d78c9db0 msgr2=0x55f8d79815a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:03.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.241+0000 7f3febd77b80 1 -- 172.21.15.1:0/2088857942 shutdown_connections 2024-02-02T15:42:03.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.241+0000 7f3febd77b80 1 -- 172.21.15.1:0/2088857942 wait complete. 2024-02-02T15:42:03.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.242+0000 7f3febd77b80 1 -- start start 2024-02-02T15:42:03.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.242+0000 7f3febd77b80 1 Processor -- start 2024-02-02T15:42:03.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.242+0000 7f3febd77b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8d796a010 0x55f8d797a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:03.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.242+0000 7f3febd77b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f8d796a930 0x55f8d797a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:03.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.242+0000 7f3febd77b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f8d79757e0 0x55f8d797ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:03.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.242+0000 7f3febd77b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8d79d5f40 con 0x55f8d796a010 2024-02-02T15:42:03.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.242+0000 7f3febd77b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8d79d60b0 con 0x55f8d79757e0 2024-02-02T15:42:03.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.242+0000 7f3febd77b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f8d79d6270 con 0x55f8d796a930 2024-02-02T15:42:03.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.242+0000 7f3febd77b80 10 client.0 _ll_register_callbacks cb 0x55f8d7859290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:42:03.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.243+0000 7f3fd5800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8d796a010 0x55f8d797a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:03.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.243+0000 7f3fd5800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8d796a010 0x55f8d797a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49794/0 (socket says 172.21.15.1:49794) 2024-02-02T15:42:03.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.243+0000 7f3fd5800700 1 -- 172.21.15.1:0/2392942410 learned_addr learned my addr 172.21.15.1:0/2392942410 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:42:03.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.243+0000 7f3fd4e00700 1 --2- 172.21.15.1:0/2392942410 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f8d796a930 0x55f8d797a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:03.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.243+0000 7f3fd6200700 1 --2- 172.21.15.1:0/2392942410 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f8d79757e0 0x55f8d797ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:03.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.243+0000 7f3fd5800700 1 -- 172.21.15.1:0/2392942410 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f8d796a930 msgr2=0x55f8d797a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:03.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.243+0000 7f3fd5800700 1 --2- 172.21.15.1:0/2392942410 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f8d796a930 0x55f8d797a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.243+0000 7f3fd5800700 1 -- 172.21.15.1:0/2392942410 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f8d79757e0 msgr2=0x55f8d797ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:03.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.243+0000 7f3fd5800700 1 --2- 172.21.15.1:0/2392942410 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f8d79757e0 0x55f8d797ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.243+0000 7f3fd5800700 1 -- 172.21.15.1:0/2392942410 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fb8011440 con 0x55f8d796a010 2024-02-02T15:42:03.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.243+0000 7f3fd6200700 1 --2- 172.21.15.1:0/2392942410 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f8d79757e0 0x55f8d797ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:42:03.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.243+0000 7f3fd5800700 1 --2- 172.21.15.1:0/2392942410 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8d796a010 0x55f8d797a310 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f3fc0084010 tx=0x7f3fc0099b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a14b82d7c3cacb6e server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:03.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.243+0000 7f3fbf400700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:42:03.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.243+0000 7f3fbf400700 1 -- 172.21.15.1:0/2392942410 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc00b5070 con 0x55f8d796a010 2024-02-02T15:42:03.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.243+0000 7f3fbf400700 1 -- 172.21.15.1:0/2392942410 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f3fc0099400 con 0x55f8d796a010 2024-02-02T15:42:03.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.243+0000 7f3febd77b80 1 -- 172.21.15.1:0/2392942410 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3fc009d230 con 0x55f8d796a010 2024-02-02T15:42:03.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.244+0000 7f3fbf400700 1 -- 172.21.15.1:0/2392942410 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3fc0065170 con 0x55f8d796a010 2024-02-02T15:42:03.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.244+0000 7f3febd77b80 10 client.19437 fetch_fsmap learned FSMap version 61 2024-02-02T15:42:03.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.244+0000 7f3febd77b80 1 -- 172.21.15.1:0/2392942410 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x55f8d773d3b0 con 0x55f8d796a010 2024-02-02T15:42:03.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.244+0000 7f3fbf400700 1 -- 172.21.15.1:0/2392942410 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3fc007c080 con 0x55f8d796a010 2024-02-02T15:42:03.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.244+0000 7f3fbf400700 1 -- 172.21.15.1:0/2392942410 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 61) v1 ==== 7550+0+0 (secure 0 0 0) 0x7f3fc00988f0 con 0x55f8d796a010 2024-02-02T15:42:03.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.244+0000 7f3febd77b80 10 client.19437 fetch_fsmap finished waiting for FSMap version 61 2024-02-02T15:42:03.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.244+0000 7f3febd77b80 10 client.19437 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-02-02T15:42:03.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.244+0000 7f3febd77b80 20 client.19437 populate_metadata read hostname 'smithi001' 2024-02-02T15:42:03.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.244+0000 7f3febd77b80 1 --2- 172.21.15.1:0/2392942410 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55f8d79d7510 0x55f8d79d9930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:03.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.244+0000 7f3febd77b80 4 client.19437 mds_command: new command op to 24518 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:42:03.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.244+0000 7f3febd77b80 1 -- 172.21.15.1:0/2392942410 --> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f8d787af50 con 0x55f8d79d7510 2024-02-02T15:42:03.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.244+0000 7f3fd4e00700 1 --2- 172.21.15.1:0/2392942410 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55f8d79d7510 0x55f8d79d9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:03.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.245+0000 7f3fd4e00700 1 --2- 172.21.15.1:0/2392942410 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55f8d79d7510 0x55f8d79d9930 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ce4ce5ea7be4d963 server_cookie=ab5e284237d4260c in_seq=0 out_seq=0 2024-02-02T15:42:03.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.245+0000 7f3fbf400700 10 client.19437 ms_handle_connect on v2:172.21.15.71:6836/628016199 2024-02-02T15:42:03.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.245+0000 7f3fbf400700 1 -- 172.21.15.1:0/2392942410 <== mds.2 v2:172.21.15.71:6836/628016199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f8d787af50 con 0x55f8d79d7510 2024-02-02T15:42:03.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.245+0000 7f3fbf400700 10 client.19437 handle_command_reply: tid=0 2024-02-02T15:42:03.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.245+0000 7f3febd77b80 1 client.19437 shutdown 2024-02-02T15:42:03.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.245+0000 7f3febd77b80 1 -- 172.21.15.1:0/2392942410 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8d796a010 msgr2=0x55f8d797a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:03.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.245+0000 7f3febd77b80 1 --2- 172.21.15.1:0/2392942410 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8d796a010 0x55f8d797a310 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f3fc0084010 tx=0x7f3fc0099b00 comp rx=0 tx=0).stop 2024-02-02T15:42:03.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.246+0000 7f3febd77b80 1 -- 172.21.15.1:0/2392942410 shutdown_connections 2024-02-02T15:42:03.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.246+0000 7f3febd77b80 1 --2- 172.21.15.1:0/2392942410 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f8d79757e0 0x55f8d797ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.246+0000 7f3febd77b80 1 --2- 172.21.15.1:0/2392942410 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f8d796a010 0x55f8d797a310 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.246+0000 7f3febd77b80 1 --2- 172.21.15.1:0/2392942410 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55f8d79d7510 0x55f8d79d9930 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.246+0000 7f3febd77b80 1 --2- 172.21.15.1:0/2392942410 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f8d796a930 0x55f8d797a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.246+0000 7f3febd77b80 1 -- 172.21.15.1:0/2392942410 >> 172.21.15.1:0/2392942410 conn(0x55f8d78c9db0 msgr2=0x55f8d7980900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:03.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.246+0000 7f3fbf400700 0 client.19437 ms_handle_reset on v2:172.21.15.71:6836/628016199 2024-02-02T15:42:03.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.246+0000 7f3febd77b80 1 -- 172.21.15.1:0/2392942410 shutdown_connections 2024-02-02T15:42:03.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.246+0000 7f3febd77b80 1 -- 172.21.15.1:0/2392942410 wait complete. 2024-02-02T15:42:03.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.246+0000 7f3febd77b80 20 client.19437 trim_cache size 0 max 16384 2024-02-02T15:42:03.252 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:42:03.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.246+0000 7f3fd8200700 1 -- 172.21.15.1:0/3804376095 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f3fb00a0690 msgr2=0x7f3fb00a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:03.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.246+0000 7f3fd8200700 1 --2- 172.21.15.1:0/3804376095 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f3fb00a0690 0x7f3fb00a2b50 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f3fc80062a0 tx=0x7f3fc8006210 comp rx=0 tx=0).stop 2024-02-02T15:42:03.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.246+0000 7f3fd8200700 1 -- 172.21.15.1:0/3804376095 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3fd0100ff0 msgr2=0x7f3fd0199210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:03.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.246+0000 7f3fd8200700 1 --2- 172.21.15.1:0/3804376095 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3fd0100ff0 0x7f3fd0199210 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f3fc000e9a0 tx=0x7f3fc000ecb0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.247+0000 7f3fd8200700 1 -- 172.21.15.1:0/3804376095 shutdown_connections 2024-02-02T15:42:03.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.247+0000 7f3fd8200700 1 --2- 172.21.15.1:0/3804376095 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f3fb00a0690 0x7f3fb00a2b50 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.247+0000 7f3fd8200700 1 --2- 172.21.15.1:0/3804376095 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3fd019dbb0 0x7f3fd019e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.247+0000 7f3fd8200700 1 --2- 172.21.15.1:0/3804376095 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3fd0100ff0 0x7f3fd0199210 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.247+0000 7f3fd8200700 1 --2- 172.21.15.1:0/3804376095 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fd0106110 0x7f3fd0199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.247+0000 7f3fd8200700 1 -- 172.21.15.1:0/3804376095 >> 172.21.15.1:0/3804376095 conn(0x7f3fd00fc810 msgr2=0x7f3fd00fd320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:03.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.247+0000 7f3fd8200700 1 -- 172.21.15.1:0/3804376095 shutdown_connections 2024-02-02T15:42:03.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.247+0000 7f3fd8200700 1 -- 172.21.15.1:0/3804376095 wait complete. 2024-02-02T15:42:03.264 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-02T15:42:03.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.468+0000 7f280f200700 1 -- 172.21.15.1:0/3656147140 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2808106110 msgr2=0x7f280810dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:03.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.468+0000 7f280f200700 1 --2- 172.21.15.1:0/3656147140 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2808106110 0x7f280810dc90 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f280000b0a0 tx=0x7f280000b3b0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.468+0000 7f280f200700 1 -- 172.21.15.1:0/3656147140 shutdown_connections 2024-02-02T15:42:03.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.468+0000 7f280f200700 1 --2- 172.21.15.1:0/3656147140 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2808106110 0x7f280810dc90 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.468+0000 7f280f200700 1 --2- 172.21.15.1:0/3656147140 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2808100ff0 0x7f28081013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.468+0000 7f280f200700 1 --2- 172.21.15.1:0/3656147140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f28081019a0 0x7f28081059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.468+0000 7f280f200700 1 -- 172.21.15.1:0/3656147140 >> 172.21.15.1:0/3656147140 conn(0x7f28080fc810 msgr2=0x7f28080fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:03.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.468+0000 7f280f200700 1 -- 172.21.15.1:0/3656147140 shutdown_connections 2024-02-02T15:42:03.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.468+0000 7f280f200700 1 -- 172.21.15.1:0/3656147140 wait complete. 2024-02-02T15:42:03.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.469+0000 7f280f200700 1 Processor -- start 2024-02-02T15:42:03.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.469+0000 7f280f200700 1 -- start start 2024-02-02T15:42:03.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.469+0000 7f280f200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2808100ff0 0x7f28081991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:03.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.469+0000 7f280f200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2808106110 0x7f28081996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:03.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.469+0000 7f280f200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f280819db50 0x7f280819dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:03.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.469+0000 7f280f200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f280819e5d0 con 0x7f2808100ff0 2024-02-02T15:42:03.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.469+0000 7f280f200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f280819e740 con 0x7f280819db50 2024-02-02T15:42:03.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.469+0000 7f280f200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f280819e900 con 0x7f2808106110 2024-02-02T15:42:03.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.470+0000 7f2807e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2808100ff0 0x7f28081991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:03.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.470+0000 7f2807e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2808100ff0 0x7f28081991b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49814/0 (socket says 172.21.15.1:49814) 2024-02-02T15:42:03.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.470+0000 7f2807e00700 1 -- 172.21.15.1:0/786765961 learned_addr learned my addr 172.21.15.1:0/786765961 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:42:03.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.470+0000 7f2807400700 1 --2- 172.21.15.1:0/786765961 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2808106110 0x7f28081996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:03.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.470+0000 7f280d200700 1 --2- 172.21.15.1:0/786765961 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f280819db50 0x7f280819dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:03.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.470+0000 7f2807e00700 1 -- 172.21.15.1:0/786765961 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2808106110 msgr2=0x7f28081996f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:03.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.470+0000 7f2807e00700 1 --2- 172.21.15.1:0/786765961 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2808106110 0x7f28081996f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.470+0000 7f2807e00700 1 -- 172.21.15.1:0/786765961 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f280819db50 msgr2=0x7f280819dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:03.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.470+0000 7f2807e00700 1 --2- 172.21.15.1:0/786765961 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f280819db50 0x7f280819dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.470+0000 7f2807e00700 1 -- 172.21.15.1:0/786765961 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2800009d00 con 0x7f2808100ff0 2024-02-02T15:42:03.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.470+0000 7f2807e00700 1 --2- 172.21.15.1:0/786765961 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2808100ff0 0x7f28081991b0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f27f800e9a0 tx=0x7f27f800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:03.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.471+0000 7f2804c00700 1 -- 172.21.15.1:0/786765961 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f8009790 con 0x7f2808100ff0 2024-02-02T15:42:03.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.471+0000 7f2804c00700 1 -- 172.21.15.1:0/786765961 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27f8010430 con 0x7f2808100ff0 2024-02-02T15:42:03.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.471+0000 7f2804c00700 1 -- 172.21.15.1:0/786765961 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f8018680 con 0x7f2808100ff0 2024-02-02T15:42:03.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.471+0000 7f280f200700 1 -- 172.21.15.1:0/786765961 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28081a5e60 con 0x7f2808100ff0 2024-02-02T15:42:03.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.471+0000 7f280f200700 1 -- 172.21.15.1:0/786765961 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28081a6230 con 0x7f2808100ff0 2024-02-02T15:42:03.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.474+0000 7f2804c00700 1 -- 172.21.15.1:0/786765961 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f27f80105a0 con 0x7f2808100ff0 2024-02-02T15:42:03.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.475+0000 7f2804c00700 1 --2- 172.21.15.1:0/786765961 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f27e80a0690 0x7f27e80a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:03.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.475+0000 7f2822c8fb80 1 Processor -- start 2024-02-02T15:42:03.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.475+0000 7f2804c00700 1 -- 172.21.15.1:0/786765961 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f27f8014070 con 0x7f2808100ff0 2024-02-02T15:42:03.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.475+0000 7f2822c8fb80 1 -- start start 2024-02-02T15:42:03.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.475+0000 7f2807400700 1 --2- 172.21.15.1:0/786765961 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f27e80a0690 0x7f27e80a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:03.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.476+0000 7f2822c8fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be5f106010 0x55be5f1063f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:03.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.476+0000 7f2822c8fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be5f106930 0x55be5f119ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:03.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.476+0000 7f2822c8fb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55be5f119ff0 0x55be5f11c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:03.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.476+0000 7f2822c8fb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be5f07a4c0 con 0x55be5f119ff0 2024-02-02T15:42:03.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.476+0000 7f2822c8fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be5f05eaa0 con 0x55be5f106930 2024-02-02T15:42:03.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.476+0000 7f2822c8fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be5f05c460 con 0x55be5f106010 2024-02-02T15:42:03.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.476+0000 7f2807e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be5f106010 0x55be5f1063f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:03.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.476+0000 7f2807400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be5f106930 0x55be5f119ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:03.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.476+0000 7f2807e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be5f106010 0x55be5f1063f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:42288/0 (socket says 172.21.15.1:42288) 2024-02-02T15:42:03.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.476+0000 7f2807e00700 1 -- 172.21.15.1:0/1864828402 learned_addr learned my addr 172.21.15.1:0/1864828402 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:42:03.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.476+0000 7f280d200700 1 --2- 172.21.15.1:0/1864828402 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55be5f119ff0 0x55be5f11c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:03.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.476+0000 7f2807400700 1 --2- 172.21.15.1:0/786765961 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f27e80a0690 0x7f27e80a2b50 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f280819a7c0 tx=0x7f27f0009260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:03.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.476+0000 7f2807e00700 1 -- 172.21.15.1:0/1864828402 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be5f106930 msgr2=0x55be5f119ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:03.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.476+0000 7f2807e00700 1 --2- 172.21.15.1:0/1864828402 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be5f106930 0x55be5f119ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.476+0000 7f2807e00700 1 -- 172.21.15.1:0/1864828402 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55be5f119ff0 msgr2=0x55be5f11c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:03.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.476+0000 7f2807e00700 1 --2- 172.21.15.1:0/1864828402 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55be5f119ff0 0x55be5f11c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.476+0000 7f2807e00700 1 -- 172.21.15.1:0/1864828402 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be5eed93b0 con 0x55be5f106010 2024-02-02T15:42:03.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.477+0000 7f2807e00700 1 --2- 172.21.15.1:0/1864828402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be5f106010 0x55be5f1063f0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f27f8099330 tx=0x7f27f8098d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=373b6150d9f7505e server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:03.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.477+0000 7f27fd600700 1 -- 172.21.15.1:0/1864828402 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f80993a0 con 0x55be5f106010 2024-02-02T15:42:03.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.477+0000 7f27fd600700 1 -- 172.21.15.1:0/1864828402 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27f809d1d0 con 0x55be5f106010 2024-02-02T15:42:03.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.477+0000 7f27fd600700 1 -- 172.21.15.1:0/1864828402 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f8099600 con 0x55be5f106010 2024-02-02T15:42:03.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.478+0000 7f2822c8fb80 1 -- 172.21.15.1:0/1864828402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be5f106010 msgr2=0x55be5f1063f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:03.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.478+0000 7f2822c8fb80 1 --2- 172.21.15.1:0/1864828402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be5f106010 0x55be5f1063f0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f27f8099330 tx=0x7f27f8098d80 comp rx=0 tx=0).stop 2024-02-02T15:42:03.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.478+0000 7f2822c8fb80 1 -- 172.21.15.1:0/1864828402 shutdown_connections 2024-02-02T15:42:03.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.478+0000 7f2822c8fb80 1 --2- 172.21.15.1:0/1864828402 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55be5f119ff0 0x55be5f11c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.478+0000 7f2822c8fb80 1 --2- 172.21.15.1:0/1864828402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be5f106010 0x55be5f1063f0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.478+0000 7f2822c8fb80 1 --2- 172.21.15.1:0/1864828402 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be5f106930 0x55be5f119ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.478+0000 7f2822c8fb80 1 -- 172.21.15.1:0/1864828402 >> 172.21.15.1:0/1864828402 conn(0x55be5f065db0 msgr2=0x55be5f1023b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:03.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.478+0000 7f2822c8fb80 1 -- 172.21.15.1:0/1864828402 shutdown_connections 2024-02-02T15:42:03.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.478+0000 7f2822c8fb80 1 -- 172.21.15.1:0/1864828402 wait complete. 2024-02-02T15:42:03.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.478+0000 7f2822c8fb80 1 -- start start 2024-02-02T15:42:03.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.479+0000 7f2822c8fb80 1 Processor -- start 2024-02-02T15:42:03.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.479+0000 7f2822c8fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be5f106010 0x55be5f119a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:03.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.479+0000 7f2822c8fb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55be5f106930 0x55be5f116c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:03.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.479+0000 7f2822c8fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be5f119ff0 0x55be5f117390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:03.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.479+0000 7f2822c8fb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be5f057d50 con 0x55be5f106930 2024-02-02T15:42:03.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.479+0000 7f2822c8fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be5eff3d80 con 0x55be5f119ff0 2024-02-02T15:42:03.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.479+0000 7f2822c8fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be5eff3a20 con 0x55be5f106010 2024-02-02T15:42:03.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.479+0000 7f2822c8fb80 10 client.0 _ll_register_callbacks cb 0x55be5eff5290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:42:03.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.479+0000 7f2807400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55be5f106930 0x55be5f116c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:03.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.479+0000 7f2807400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55be5f106930 0x55be5f116c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49832/0 (socket says 172.21.15.1:49832) 2024-02-02T15:42:03.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.479+0000 7f2807400700 1 -- 172.21.15.1:0/2716229628 learned_addr learned my addr 172.21.15.1:0/2716229628 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:42:03.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.479+0000 7f2807e00700 1 --2- 172.21.15.1:0/2716229628 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be5f106010 0x55be5f119a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:03.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.479+0000 7f280d200700 1 --2- 172.21.15.1:0/2716229628 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be5f119ff0 0x55be5f117390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:03.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.480+0000 7f2807400700 1 -- 172.21.15.1:0/2716229628 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be5f106010 msgr2=0x55be5f119a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:03.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.480+0000 7f2807400700 1 --2- 172.21.15.1:0/2716229628 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be5f106010 0x55be5f119a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.480+0000 7f2807400700 1 -- 172.21.15.1:0/2716229628 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be5f119ff0 msgr2=0x55be5f117390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:03.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.480+0000 7f2807400700 1 --2- 172.21.15.1:0/2716229628 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be5f119ff0 0x55be5f117390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.480+0000 7f2807400700 1 -- 172.21.15.1:0/2716229628 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28081a6230 con 0x55be5f106930 2024-02-02T15:42:03.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.480+0000 7f2807400700 1 --2- 172.21.15.1:0/2716229628 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55be5f106930 0x55be5f116c50 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f27f001c6e0 tx=0x7f27f00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14984ab2e23f8785 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:03.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.480+0000 7f27f6000700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:42:03.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.480+0000 7f27f6000700 1 -- 172.21.15.1:0/2716229628 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f0021070 con 0x55be5f106930 2024-02-02T15:42:03.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.480+0000 7f27f6000700 1 -- 172.21.15.1:0/2716229628 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f27f0011a10 con 0x55be5f106930 2024-02-02T15:42:03.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.480+0000 7f2822c8fb80 1 -- 172.21.15.1:0/2716229628 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55be5f0fa0f0 con 0x55be5f106930 2024-02-02T15:42:03.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.481+0000 7f27f6000700 1 -- 172.21.15.1:0/2716229628 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f27f0011bd0 con 0x55be5f106930 2024-02-02T15:42:03.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.481+0000 7f2822c8fb80 10 client.19449 fetch_fsmap learned FSMap version 61 2024-02-02T15:42:03.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.481+0000 7f2822c8fb80 1 -- 172.21.15.1:0/2716229628 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x55be5ee85b60 con 0x55be5f106930 2024-02-02T15:42:03.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.481+0000 7f27f6000700 1 -- 172.21.15.1:0/2716229628 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55be5f0fa0f0 con 0x55be5f106930 2024-02-02T15:42:03.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.481+0000 7f27f6000700 1 -- 172.21.15.1:0/2716229628 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 61) v1 ==== 7550+0+0 (secure 0 0 0) 0x7f27f0012140 con 0x55be5f106930 2024-02-02T15:42:03.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.481+0000 7f2822c8fb80 10 client.19449 fetch_fsmap finished waiting for FSMap version 61 2024-02-02T15:42:03.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.481+0000 7f2822c8fb80 10 client.19449 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-02T15:42:03.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.481+0000 7f2822c8fb80 20 client.19449 populate_metadata read hostname 'smithi001' 2024-02-02T15:42:03.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.481+0000 7f2822c8fb80 1 --2- 172.21.15.1:0/2716229628 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55be5f129ad0 0x55be5f12bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:03.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.481+0000 7f2822c8fb80 4 client.19449 mds_command: new command op to 24529 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:42:03.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.481+0000 7f2822c8fb80 1 -- 172.21.15.1:0/2716229628 --> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55be5f050dd0 con 0x55be5f129ad0 2024-02-02T15:42:03.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.482+0000 7f2807e00700 1 --2- 172.21.15.1:0/2716229628 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55be5f129ad0 0x55be5f12bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:03.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.482+0000 7f2807e00700 1 --2- 172.21.15.1:0/2716229628 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55be5f129ad0 0x55be5f12bef0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a32f2b464eca08cf server_cookie=b0f932b72cff109e in_seq=0 out_seq=0 2024-02-02T15:42:03.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.482+0000 7f27f6000700 10 client.19449 ms_handle_connect on v2:172.21.15.142:6838/321675421 2024-02-02T15:42:03.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.483+0000 7f27f6000700 1 -- 172.21.15.1:0/2716229628 <== mds.1 v2:172.21.15.142:6838/321675421 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55be5f050dd0 con 0x55be5f129ad0 2024-02-02T15:42:03.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.483+0000 7f27f6000700 10 client.19449 handle_command_reply: tid=0 2024-02-02T15:42:03.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.483+0000 7f2822c8fb80 1 client.19449 shutdown 2024-02-02T15:42:03.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.483+0000 7f2822c8fb80 1 -- 172.21.15.1:0/2716229628 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55be5f106930 msgr2=0x55be5f116c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:03.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.483+0000 7f2822c8fb80 1 --2- 172.21.15.1:0/2716229628 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55be5f106930 0x55be5f116c50 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f27f001c6e0 tx=0x7f27f00047c0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.483+0000 7f2822c8fb80 1 -- 172.21.15.1:0/2716229628 shutdown_connections 2024-02-02T15:42:03.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.483+0000 7f2822c8fb80 1 --2- 172.21.15.1:0/2716229628 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55be5f129ad0 0x55be5f12bef0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.483+0000 7f2822c8fb80 1 --2- 172.21.15.1:0/2716229628 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be5f119ff0 0x55be5f117390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.483+0000 7f2822c8fb80 1 --2- 172.21.15.1:0/2716229628 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be5f106010 0x55be5f119a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.483+0000 7f2822c8fb80 1 --2- 172.21.15.1:0/2716229628 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55be5f106930 0x55be5f116c50 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.483+0000 7f2822c8fb80 1 -- 172.21.15.1:0/2716229628 >> 172.21.15.1:0/2716229628 conn(0x55be5f065db0 msgr2=0x55be5f101fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:03.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.483+0000 7f27f6000700 0 client.19449 ms_handle_reset on v2:172.21.15.142:6838/321675421 2024-02-02T15:42:03.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.483+0000 7f2822c8fb80 1 -- 172.21.15.1:0/2716229628 shutdown_connections 2024-02-02T15:42:03.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.483+0000 7f2822c8fb80 1 -- 172.21.15.1:0/2716229628 wait complete. 2024-02-02T15:42:03.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.484+0000 7f2822c8fb80 20 client.19449 trim_cache size 0 max 16384 2024-02-02T15:42:03.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.497+0000 7f2822c8fb80 1 Processor -- start 2024-02-02T15:42:03.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.497+0000 7f2822c8fb80 1 -- start start 2024-02-02T15:42:03.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.497+0000 7f2822c8fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be5f106010 0x55be5f120b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:03.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.497+0000 7f2822c8fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be5f106930 0x55be5f119570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:03.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.497+0000 7f2822c8fb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55be5f119ff0 0x55be5f111ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:03.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.498+0000 7f2822c8fb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be5eedb6a0 con 0x55be5f119ff0 2024-02-02T15:42:03.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.498+0000 7f2822c8fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be5f127130 con 0x55be5f106930 2024-02-02T15:42:03.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.498+0000 7f2822c8fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be5f117db0 con 0x55be5f106010 2024-02-02T15:42:03.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.498+0000 7f280d200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55be5f119ff0 0x55be5f111ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:03.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.498+0000 7f2807e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be5f106010 0x55be5f120b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:03.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.498+0000 7f2807e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be5f106010 0x55be5f120b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:42302/0 (socket says 172.21.15.1:42302) 2024-02-02T15:42:03.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.498+0000 7f2807e00700 1 -- 172.21.15.1:0/1903863257 learned_addr learned my addr 172.21.15.1:0/1903863257 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:42:03.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.498+0000 7f2807400700 1 --2- 172.21.15.1:0/1903863257 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be5f106930 0x55be5f119570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:03.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.498+0000 7f2807e00700 1 -- 172.21.15.1:0/1903863257 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be5f106930 msgr2=0x55be5f119570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:03.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.498+0000 7f2807e00700 1 --2- 172.21.15.1:0/1903863257 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be5f106930 0x55be5f119570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.498+0000 7f2807e00700 1 -- 172.21.15.1:0/1903863257 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55be5f119ff0 msgr2=0x55be5f111ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:03.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.498+0000 7f2807e00700 1 --2- 172.21.15.1:0/1903863257 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55be5f119ff0 0x55be5f111ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.498+0000 7f2807e00700 1 -- 172.21.15.1:0/1903863257 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be5f1260e0 con 0x55be5f106010 2024-02-02T15:42:03.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.498+0000 7f280d200700 1 --2- 172.21.15.1:0/1903863257 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55be5f119ff0 0x55be5f111ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:42:03.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.499+0000 7f2807e00700 1 --2- 172.21.15.1:0/1903863257 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be5f106010 0x55be5f120b40 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f27f80958a0 tx=0x7f27f8010c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bcc216c5e1c7c9e server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:03.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.499+0000 7f27fe000700 1 -- 172.21.15.1:0/1903863257 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f80993a0 con 0x55be5f106010 2024-02-02T15:42:03.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.499+0000 7f27fe000700 1 -- 172.21.15.1:0/1903863257 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27f8010880 con 0x55be5f106010 2024-02-02T15:42:03.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.499+0000 7f27fe000700 1 -- 172.21.15.1:0/1903863257 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f8099600 con 0x55be5f106010 2024-02-02T15:42:03.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.499+0000 7f2822c8fb80 1 -- 172.21.15.1:0/1903863257 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be5f106010 msgr2=0x55be5f120b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:03.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.499+0000 7f2822c8fb80 1 --2- 172.21.15.1:0/1903863257 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be5f106010 0x55be5f120b40 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f27f80958a0 tx=0x7f27f8010c80 comp rx=0 tx=0).stop 2024-02-02T15:42:03.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.499+0000 7f2822c8fb80 1 -- 172.21.15.1:0/1903863257 shutdown_connections 2024-02-02T15:42:03.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.499+0000 7f2822c8fb80 1 --2- 172.21.15.1:0/1903863257 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55be5f119ff0 0x55be5f111ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.499+0000 7f2822c8fb80 1 --2- 172.21.15.1:0/1903863257 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be5f106010 0x55be5f120b40 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.499+0000 7f2822c8fb80 1 --2- 172.21.15.1:0/1903863257 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be5f106930 0x55be5f119570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.499+0000 7f2822c8fb80 1 -- 172.21.15.1:0/1903863257 >> 172.21.15.1:0/1903863257 conn(0x55be5f065db0 msgr2=0x55be5f100b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:03.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.500+0000 7f2822c8fb80 1 -- 172.21.15.1:0/1903863257 shutdown_connections 2024-02-02T15:42:03.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.500+0000 7f2822c8fb80 1 -- 172.21.15.1:0/1903863257 wait complete. 2024-02-02T15:42:03.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.500+0000 7f2822c8fb80 1 -- start start 2024-02-02T15:42:03.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.500+0000 7f2822c8fb80 1 Processor -- start 2024-02-02T15:42:03.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.501+0000 7f2822c8fb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55be5f106010 0x55be5f11e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:03.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.501+0000 7f2822c8fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be5f106930 0x55be5f11e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:03.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.501+0000 7f2822c8fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be5f119ff0 0x55be5f17a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:03.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.501+0000 7f2822c8fb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be5f11f280 con 0x55be5f106010 2024-02-02T15:42:03.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.501+0000 7f2822c8fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be5f17ab30 con 0x55be5f119ff0 2024-02-02T15:42:03.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.501+0000 7f2822c8fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be5f17aca0 con 0x55be5f106930 2024-02-02T15:42:03.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.501+0000 7f2822c8fb80 10 client.0 _ll_register_callbacks cb 0x55be5eff5290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:42:03.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.501+0000 7f2807e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55be5f106010 0x55be5f11e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:03.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.501+0000 7f2807e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55be5f106010 0x55be5f11e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49850/0 (socket says 172.21.15.1:49850) 2024-02-02T15:42:03.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.501+0000 7f2807e00700 1 -- 172.21.15.1:0/2024526768 learned_addr learned my addr 172.21.15.1:0/2024526768 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:42:03.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.501+0000 7f2807400700 1 --2- 172.21.15.1:0/2024526768 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be5f106930 0x55be5f11e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:03.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.502+0000 7f280d200700 1 --2- 172.21.15.1:0/2024526768 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be5f119ff0 0x55be5f17a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:03.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.502+0000 7f2807e00700 1 -- 172.21.15.1:0/2024526768 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be5f106930 msgr2=0x55be5f11e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:03.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.502+0000 7f2807e00700 1 --2- 172.21.15.1:0/2024526768 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be5f106930 0x55be5f11e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.502+0000 7f2807e00700 1 -- 172.21.15.1:0/2024526768 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be5f119ff0 msgr2=0x55be5f17a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:03.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.502+0000 7f2807e00700 1 --2- 172.21.15.1:0/2024526768 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be5f119ff0 0x55be5f17a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.502+0000 7f2807e00700 1 -- 172.21.15.1:0/2024526768 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f280810e500 con 0x55be5f106010 2024-02-02T15:42:03.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.502+0000 7f280d200700 1 --2- 172.21.15.1:0/2024526768 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be5f119ff0 0x55be5f17a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:42:03.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.502+0000 7f2807400700 1 --2- 172.21.15.1:0/2024526768 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be5f106930 0x55be5f11e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:42:03.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.502+0000 7f2807e00700 1 --2- 172.21.15.1:0/2024526768 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55be5f106010 0x55be5f11e400 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f27f8098e70 tx=0x7f27f8099270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31438333c9a9c8d server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:03.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.502+0000 7f27f6000700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:42:03.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.502+0000 7f27f6000700 1 -- 172.21.15.1:0/2024526768 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f8097800 con 0x55be5f106010 2024-02-02T15:42:03.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.502+0000 7f27f6000700 1 -- 172.21.15.1:0/2024526768 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f27f8099600 con 0x55be5f106010 2024-02-02T15:42:03.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.502+0000 7f2822c8fb80 1 -- 172.21.15.1:0/2024526768 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f27f809d1d0 con 0x55be5f106010 2024-02-02T15:42:03.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.502+0000 7f27f6000700 1 -- 172.21.15.1:0/2024526768 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f27f80997c0 con 0x55be5f106010 2024-02-02T15:42:03.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.502+0000 7f2822c8fb80 10 client.19455 fetch_fsmap learned FSMap version 61 2024-02-02T15:42:03.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.502+0000 7f2822c8fb80 1 -- 172.21.15.1:0/2024526768 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x55be5eed93b0 con 0x55be5f106010 2024-02-02T15:42:03.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.503+0000 7f27f6000700 1 -- 172.21.15.1:0/2024526768 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f27f809d1d0 con 0x55be5f106010 2024-02-02T15:42:03.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.503+0000 7f27f6000700 1 -- 172.21.15.1:0/2024526768 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 61) v1 ==== 7550+0+0 (secure 0 0 0) 0x7f27f80a0f80 con 0x55be5f106010 2024-02-02T15:42:03.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.503+0000 7f2822c8fb80 10 client.19455 fetch_fsmap finished waiting for FSMap version 61 2024-02-02T15:42:03.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.503+0000 7f2822c8fb80 10 client.19455 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-02T15:42:03.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.503+0000 7f2822c8fb80 20 client.19455 populate_metadata read hostname 'smithi001' 2024-02-02T15:42:03.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.503+0000 7f2822c8fb80 1 --2- 172.21.15.1:0/2024526768 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55be5f17bee0 0x55be5f17e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:03.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.503+0000 7f2822c8fb80 4 client.19455 mds_command: new command op to 24529 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:42:03.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.503+0000 7f2822c8fb80 1 -- 172.21.15.1:0/2024526768 --> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55be5f016f50 con 0x55be5f17bee0 2024-02-02T15:42:03.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.503+0000 7f2807400700 1 --2- 172.21.15.1:0/2024526768 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55be5f17bee0 0x55be5f17e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:03.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.504+0000 7f2807400700 1 --2- 172.21.15.1:0/2024526768 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55be5f17bee0 0x55be5f17e300 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6d4ea1e6d595ba24 server_cookie=1e39006fe7b7a9d in_seq=0 out_seq=0 2024-02-02T15:42:03.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.504+0000 7f27f6000700 10 client.19455 ms_handle_connect on v2:172.21.15.142:6838/321675421 2024-02-02T15:42:03.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.504+0000 7f27f6000700 1 -- 172.21.15.1:0/2024526768 <== mds.1 v2:172.21.15.142:6838/321675421 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55be5f016f50 con 0x55be5f17bee0 2024-02-02T15:42:03.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.504+0000 7f27f6000700 10 client.19455 handle_command_reply: tid=0 2024-02-02T15:42:03.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.504+0000 7f2822c8fb80 1 client.19455 shutdown 2024-02-02T15:42:03.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.504+0000 7f2822c8fb80 1 -- 172.21.15.1:0/2024526768 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55be5f106010 msgr2=0x55be5f11e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:03.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.504+0000 7f2822c8fb80 1 --2- 172.21.15.1:0/2024526768 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55be5f106010 0x55be5f11e400 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f27f8098e70 tx=0x7f27f8099270 comp rx=0 tx=0).stop 2024-02-02T15:42:03.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.504+0000 7f2822c8fb80 1 -- 172.21.15.1:0/2024526768 shutdown_connections 2024-02-02T15:42:03.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.504+0000 7f2822c8fb80 1 --2- 172.21.15.1:0/2024526768 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55be5f17bee0 0x55be5f17e300 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.504+0000 7f2822c8fb80 1 --2- 172.21.15.1:0/2024526768 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55be5f119ff0 0x55be5f17a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.504+0000 7f2822c8fb80 1 --2- 172.21.15.1:0/2024526768 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55be5f106010 0x55be5f11e400 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.504+0000 7f27f6000700 0 client.19455 ms_handle_reset on v2:172.21.15.142:6838/321675421 2024-02-02T15:42:03.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.504+0000 7f2822c8fb80 1 --2- 172.21.15.1:0/2024526768 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55be5f106930 0x55be5f11e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.504+0000 7f2822c8fb80 1 -- 172.21.15.1:0/2024526768 >> 172.21.15.1:0/2024526768 conn(0x55be5f065db0 msgr2=0x55be5f111ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:03.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.505+0000 7f2822c8fb80 1 -- 172.21.15.1:0/2024526768 shutdown_connections 2024-02-02T15:42:03.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.505+0000 7f2822c8fb80 1 -- 172.21.15.1:0/2024526768 wait complete. 2024-02-02T15:42:03.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.505+0000 7f2822c8fb80 20 client.19455 trim_cache size 0 max 16384 2024-02-02T15:42:03.510 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:42:03.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.505+0000 7f280f200700 1 -- 172.21.15.1:0/786765961 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f27e80a0690 msgr2=0x7f27e80a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:03.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.505+0000 7f280f200700 1 --2- 172.21.15.1:0/786765961 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f27e80a0690 0x7f27e80a2b50 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f280819a7c0 tx=0x7f27f0009260 comp rx=0 tx=0).stop 2024-02-02T15:42:03.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.505+0000 7f280f200700 1 -- 172.21.15.1:0/786765961 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2808100ff0 msgr2=0x7f28081991b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:03.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.505+0000 7f280f200700 1 --2- 172.21.15.1:0/786765961 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2808100ff0 0x7f28081991b0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f27f800e9a0 tx=0x7f27f800ecb0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.505+0000 7f280f200700 1 -- 172.21.15.1:0/786765961 shutdown_connections 2024-02-02T15:42:03.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.505+0000 7f280f200700 1 --2- 172.21.15.1:0/786765961 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f27e80a0690 0x7f27e80a2b50 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.505+0000 7f280f200700 1 --2- 172.21.15.1:0/786765961 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f280819db50 0x7f280819dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.505+0000 7f280f200700 1 --2- 172.21.15.1:0/786765961 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2808100ff0 0x7f28081991b0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.505+0000 7f280f200700 1 --2- 172.21.15.1:0/786765961 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2808106110 0x7f28081996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.505+0000 7f280f200700 1 -- 172.21.15.1:0/786765961 >> 172.21.15.1:0/786765961 conn(0x7f28080fc810 msgr2=0x7f28080fd2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:03.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.506+0000 7f280f200700 1 -- 172.21.15.1:0/786765961 shutdown_connections 2024-02-02T15:42:03.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.506+0000 7f280f200700 1 -- 172.21.15.1:0/786765961 wait complete. 2024-02-02T15:42:03.523 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-02T15:42:03.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.724+0000 7ff6a4a00700 1 -- 172.21.15.1:0/2897881670 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff6a01038f0 msgr2=0x7ff6a0107930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:03.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.724+0000 7ff6a4a00700 1 --2- 172.21.15.1:0/2897881670 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff6a01038f0 0x7ff6a0107930 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7ff688009990 tx=0x7ff688009ca0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.724+0000 7ff6a4a00700 1 -- 172.21.15.1:0/2897881670 shutdown_connections 2024-02-02T15:42:03.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.724+0000 7ff6a4a00700 1 --2- 172.21.15.1:0/2897881670 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff6a0108060 0x7ff6a010fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.724+0000 7ff6a4a00700 1 --2- 172.21.15.1:0/2897881670 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff6a0102f40 0x7ff6a0103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.724+0000 7ff6a4a00700 1 --2- 172.21.15.1:0/2897881670 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff6a01038f0 0x7ff6a0107930 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.724+0000 7ff6a4a00700 1 -- 172.21.15.1:0/2897881670 >> 172.21.15.1:0/2897881670 conn(0x7ff6a00fe780 msgr2=0x7ff6a0100ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:03.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.724+0000 7ff6a4a00700 1 -- 172.21.15.1:0/2897881670 shutdown_connections 2024-02-02T15:42:03.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.724+0000 7ff6a4a00700 1 -- 172.21.15.1:0/2897881670 wait complete. 2024-02-02T15:42:03.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.726+0000 7ff6a4a00700 1 Processor -- start 2024-02-02T15:42:03.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.726+0000 7ff6a4a00700 1 -- start start 2024-02-02T15:42:03.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.727+0000 7ff6a4a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff6a0102f40 0x7ff6a0198df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:03.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.727+0000 7ff6a4a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff6a0108060 0x7ff6a0199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:03.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.727+0000 7ff6a4a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff6a019d790 0x7ff6a019dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:03.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.727+0000 7ff6a4a00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a019e210 con 0x7ff6a019d790 2024-02-02T15:42:03.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.727+0000 7ff6a4a00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a019e380 con 0x7ff6a0102f40 2024-02-02T15:42:03.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.727+0000 7ff6a4a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6a019e540 con 0x7ff6a0108060 2024-02-02T15:42:03.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.727+0000 7ff69e800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff6a0102f40 0x7ff6a0198df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:03.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.727+0000 7ff69e800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff6a0102f40 0x7ff6a0198df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59320/0 (socket says 172.21.15.1:59320) 2024-02-02T15:42:03.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.727+0000 7ff69e800700 1 -- 172.21.15.1:0/2889634781 learned_addr learned my addr 172.21.15.1:0/2889634781 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:42:03.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.727+0000 7ff69de00700 1 --2- 172.21.15.1:0/2889634781 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff6a0108060 0x7ff6a0199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:03.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.727+0000 7ff69f200700 1 --2- 172.21.15.1:0/2889634781 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff6a019d790 0x7ff6a019dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:03.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.727+0000 7ff69e800700 1 -- 172.21.15.1:0/2889634781 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff6a0108060 msgr2=0x7ff6a0199330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:03.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.727+0000 7ff69e800700 1 --2- 172.21.15.1:0/2889634781 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff6a0108060 0x7ff6a0199330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.727+0000 7ff69e800700 1 -- 172.21.15.1:0/2889634781 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff6a019d790 msgr2=0x7ff6a019dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:03.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.727+0000 7ff69e800700 1 --2- 172.21.15.1:0/2889634781 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff6a019d790 0x7ff6a019dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.727+0000 7ff69e800700 1 -- 172.21.15.1:0/2889634781 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff698009c40 con 0x7ff6a0102f40 2024-02-02T15:42:03.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.727+0000 7ff69f200700 1 --2- 172.21.15.1:0/2889634781 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff6a019d790 0x7ff6a019dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:42:03.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.728+0000 7ff69e800700 1 --2- 172.21.15.1:0/2889634781 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff6a0102f40 0x7ff6a0198df0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7ff69000e9a0 tx=0x7ff69000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:03.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.728+0000 7ff697400700 1 -- 172.21.15.1:0/2889634781 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff690009790 con 0x7ff6a0102f40 2024-02-02T15:42:03.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.728+0000 7ff6a4a00700 1 -- 172.21.15.1:0/2889634781 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6880095f0 con 0x7ff6a0102f40 2024-02-02T15:42:03.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.728+0000 7ff697400700 1 -- 172.21.15.1:0/2889634781 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff69000ce80 con 0x7ff6a0102f40 2024-02-02T15:42:03.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.728+0000 7ff697400700 1 -- 172.21.15.1:0/2889634781 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff690018680 con 0x7ff6a0102f40 2024-02-02T15:42:03.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.728+0000 7ff6a4a00700 1 -- 172.21.15.1:0/2889634781 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6a01a5e40 con 0x7ff6a0102f40 2024-02-02T15:42:03.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.731+0000 7ff697400700 1 -- 172.21.15.1:0/2889634781 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7ff690010430 con 0x7ff6a0102f40 2024-02-02T15:42:03.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.733+0000 7ff6b8fabb80 1 Processor -- start 2024-02-02T15:42:03.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.733+0000 7ff6b8fabb80 1 -- start start 2024-02-02T15:42:03.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.733+0000 7ff6b8fabb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fd34c301f0 0x55fd34c305d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:03.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.733+0000 7ff6b8fabb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fd34c30b10 0x55fd34c3b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:03.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.733+0000 7ff6b8fabb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd34c3b9f0 0x55fd34c3dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:03.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.733+0000 7ff6b8fabb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd34ba44c0 con 0x55fd34c3b9f0 2024-02-02T15:42:03.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.733+0000 7ff6b8fabb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd34b88aa0 con 0x55fd34c30b10 2024-02-02T15:42:03.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.733+0000 7ff6b8fabb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd34b86460 con 0x55fd34c301f0 2024-02-02T15:42:03.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.733+0000 7ff697400700 1 --2- 172.21.15.1:0/2889634781 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff6800a0690 0x7ff6800a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:03.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.734+0000 7ff697400700 1 -- 172.21.15.1:0/2889634781 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9453+0+0 (secure 0 0 0) 0x7ff690014070 con 0x7ff6a0102f40 2024-02-02T15:42:03.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.734+0000 7ff69de00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd34c3b9f0 0x55fd34c3dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:03.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.734+0000 7ff69f200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fd34c301f0 0x55fd34c305d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:03.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.734+0000 7ff69f200700 1 --2- 172.21.15.1:0/2889634781 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff6800a0690 0x7ff6800a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:03.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.734+0000 7ff69de00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd34c3b9f0 0x55fd34c3dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49878/0 (socket says 172.21.15.1:49878) 2024-02-02T15:42:03.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.734+0000 7ff69de00700 1 -- 172.21.15.1:0/3480038704 learned_addr learned my addr 172.21.15.1:0/3480038704 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:42:03.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.734+0000 7ff69e800700 1 --2- 172.21.15.1:0/3480038704 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fd34c30b10 0x55fd34c3b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:03.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.734+0000 7ff69de00700 1 -- 172.21.15.1:0/3480038704 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fd34c301f0 msgr2=0x55fd34c305d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:03.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.734+0000 7ff69de00700 1 --2- 172.21.15.1:0/3480038704 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fd34c301f0 0x55fd34c305d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.734+0000 7ff69de00700 1 -- 172.21.15.1:0/3480038704 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fd34c30b10 msgr2=0x55fd34c3b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:03.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.734+0000 7ff69de00700 1 --2- 172.21.15.1:0/3480038704 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fd34c30b10 0x55fd34c3b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.734+0000 7ff69de00700 1 -- 172.21.15.1:0/3480038704 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd34a033b0 con 0x55fd34c3b9f0 2024-02-02T15:42:03.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.735+0000 7ff69de00700 1 --2- 172.21.15.1:0/3480038704 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd34c3b9f0 0x55fd34c3dde0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7ff688009990 tx=0x7ff688005ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=481291f7d6112891 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:03.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.735+0000 7ff69f200700 1 --2- 172.21.15.1:0/2889634781 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff6800a0690 0x7ff6800a2b50 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ff698009fd0 tx=0x7ff69800b040 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:03.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.735+0000 7ff68f400700 1 -- 172.21.15.1:0/3480038704 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff68801d070 con 0x55fd34c3b9f0 2024-02-02T15:42:03.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.735+0000 7ff68f400700 1 -- 172.21.15.1:0/3480038704 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff68801e040 con 0x55fd34c3b9f0 2024-02-02T15:42:03.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.735+0000 7ff68f400700 1 -- 172.21.15.1:0/3480038704 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff688003c30 con 0x55fd34c3b9f0 2024-02-02T15:42:03.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.736+0000 7ff6b8fabb80 1 -- 172.21.15.1:0/3480038704 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd34c3b9f0 msgr2=0x55fd34c3dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:03.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.736+0000 7ff6b8fabb80 1 --2- 172.21.15.1:0/3480038704 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd34c3b9f0 0x55fd34c3dde0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7ff688009990 tx=0x7ff688005ee0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.736+0000 7ff6b8fabb80 1 -- 172.21.15.1:0/3480038704 shutdown_connections 2024-02-02T15:42:03.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.736+0000 7ff6b8fabb80 1 --2- 172.21.15.1:0/3480038704 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd34c3b9f0 0x55fd34c3dde0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.736+0000 7ff6b8fabb80 1 --2- 172.21.15.1:0/3480038704 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fd34c301f0 0x55fd34c305d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.736+0000 7ff6b8fabb80 1 --2- 172.21.15.1:0/3480038704 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fd34c30b10 0x55fd34c3b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.736+0000 7ff6b8fabb80 1 -- 172.21.15.1:0/3480038704 >> 172.21.15.1:0/3480038704 conn(0x55fd34b8fdb0 msgr2=0x55fd34c2c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:03.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.736+0000 7ff6b8fabb80 1 -- 172.21.15.1:0/3480038704 shutdown_connections 2024-02-02T15:42:03.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.736+0000 7ff6b8fabb80 1 -- 172.21.15.1:0/3480038704 wait complete. 2024-02-02T15:42:03.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.736+0000 7ff6b8fabb80 1 -- start start 2024-02-02T15:42:03.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.737+0000 7ff6b8fabb80 1 Processor -- start 2024-02-02T15:42:03.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.737+0000 7ff6b8fabb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd34c301f0 0x55fd34c3b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:03.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.737+0000 7ff6b8fabb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fd34c30b10 0x55fd34c38600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:03.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.737+0000 7ff6b8fabb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fd34c3b9f0 0x55fd34c38c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:03.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.737+0000 7ff6b8fabb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd34b81d50 con 0x55fd34c301f0 2024-02-02T15:42:03.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.737+0000 7ff6b8fabb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd34b1dd80 con 0x55fd34c30b10 2024-02-02T15:42:03.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.737+0000 7ff6b8fabb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd34b1da20 con 0x55fd34c3b9f0 2024-02-02T15:42:03.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.737+0000 7ff6b8fabb80 10 client.0 _ll_register_callbacks cb 0x55fd34b1f290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:42:03.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.737+0000 7ff69e800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd34c301f0 0x55fd34c3b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:03.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.737+0000 7ff69e800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd34c301f0 0x55fd34c3b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49886/0 (socket says 172.21.15.1:49886) 2024-02-02T15:42:03.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.737+0000 7ff69de00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fd34c30b10 0x55fd34c38600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:03.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.737+0000 7ff69e800700 1 -- 172.21.15.1:0/1928158167 learned_addr learned my addr 172.21.15.1:0/1928158167 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:42:03.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.738+0000 7ff69f200700 1 --2- 172.21.15.1:0/1928158167 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fd34c3b9f0 0x55fd34c38c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:03.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.738+0000 7ff69e800700 1 -- 172.21.15.1:0/1928158167 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fd34c3b9f0 msgr2=0x55fd34c38c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:03.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.738+0000 7ff69e800700 1 --2- 172.21.15.1:0/1928158167 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fd34c3b9f0 0x55fd34c38c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.738+0000 7ff69e800700 1 -- 172.21.15.1:0/1928158167 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fd34c30b10 msgr2=0x55fd34c38600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:03.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.738+0000 7ff69e800700 1 --2- 172.21.15.1:0/1928158167 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fd34c30b10 0x55fd34c38600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.738+0000 7ff69e800700 1 -- 172.21.15.1:0/1928158167 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6a0110450 con 0x55fd34c301f0 2024-02-02T15:42:03.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.738+0000 7ff69f200700 1 --2- 172.21.15.1:0/1928158167 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fd34c3b9f0 0x55fd34c38c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:42:03.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.738+0000 7ff69de00700 1 --2- 172.21.15.1:0/1928158167 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fd34c30b10 0x55fd34c38600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:42:03.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.738+0000 7ff69e800700 1 --2- 172.21.15.1:0/1928158167 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd34c301f0 0x55fd34c3b3f0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7ff69000e970 tx=0x7ff6900651d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a80ae94e289027f server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:03.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.738+0000 7ff687e00700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:42:03.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.738+0000 7ff687e00700 1 -- 172.21.15.1:0/1928158167 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6900cc170 con 0x55fd34c301f0 2024-02-02T15:42:03.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.738+0000 7ff687e00700 1 -- 172.21.15.1:0/1928158167 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9453+0+0 (secure 0 0 0) 0x7ff690065240 con 0x55fd34c301f0 2024-02-02T15:42:03.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.738+0000 7ff6b8fabb80 1 -- 172.21.15.1:0/1928158167 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fd34c24340 con 0x55fd34c301f0 2024-02-02T15:42:03.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.738+0000 7ff687e00700 1 -- 172.21.15.1:0/1928158167 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff690099490 con 0x55fd34c301f0 2024-02-02T15:42:03.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.738+0000 7ff6b8fabb80 10 client.19467 fetch_fsmap learned FSMap version 61 2024-02-02T15:42:03.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.738+0000 7ff6b8fabb80 1 -- 172.21.15.1:0/1928158167 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x55fd349afb60 con 0x55fd34c301f0 2024-02-02T15:42:03.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.739+0000 7ff687e00700 1 -- 172.21.15.1:0/1928158167 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fd34c24340 con 0x55fd34c301f0 2024-02-02T15:42:03.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.739+0000 7ff687e00700 1 -- 172.21.15.1:0/1928158167 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 61) v1 ==== 7550+0+0 (secure 0 0 0) 0x7ff690064da0 con 0x55fd34c301f0 2024-02-02T15:42:03.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.739+0000 7ff6b8fabb80 10 client.19467 fetch_fsmap finished waiting for FSMap version 61 2024-02-02T15:42:03.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.739+0000 7ff6b8fabb80 10 client.19467 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:42:03.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.739+0000 7ff6b8fabb80 20 client.19467 populate_metadata read hostname 'smithi001' 2024-02-02T15:42:03.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.739+0000 7ff6b8fabb80 1 --2- 172.21.15.1:0/1928158167 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55fd34c4b3a0 0x55fd34c4d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:03.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.739+0000 7ff6b8fabb80 4 client.19467 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:42:03.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.739+0000 7ff6b8fabb80 1 -- 172.21.15.1:0/1928158167 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fd34b7add0 con 0x55fd34c4b3a0 2024-02-02T15:42:03.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.739+0000 7ff69de00700 1 --2- 172.21.15.1:0/1928158167 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55fd34c4b3a0 0x55fd34c4d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:03.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.740+0000 7ff69de00700 1 --2- 172.21.15.1:0/1928158167 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55fd34c4b3a0 0x55fd34c4d7c0 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fab8bf6b037a59a server_cookie=9fe4d2f9d01d59d2 in_seq=0 out_seq=0 2024-02-02T15:42:03.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.740+0000 7ff687e00700 10 client.19467 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:42:03.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.741+0000 7ff687e00700 1 -- 172.21.15.1:0/1928158167 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55fd34b7add0 con 0x55fd34c4b3a0 2024-02-02T15:42:03.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.741+0000 7ff687e00700 10 client.19467 handle_command_reply: tid=0 2024-02-02T15:42:03.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.741+0000 7ff6b8fabb80 1 client.19467 shutdown 2024-02-02T15:42:03.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.741+0000 7ff6b8fabb80 1 -- 172.21.15.1:0/1928158167 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd34c301f0 msgr2=0x55fd34c3b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:03.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.741+0000 7ff6b8fabb80 1 --2- 172.21.15.1:0/1928158167 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd34c301f0 0x55fd34c3b3f0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7ff69000e970 tx=0x7ff6900651d0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.741+0000 7ff6b8fabb80 1 -- 172.21.15.1:0/1928158167 shutdown_connections 2024-02-02T15:42:03.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.741+0000 7ff6b8fabb80 1 --2- 172.21.15.1:0/1928158167 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55fd34c4b3a0 0x55fd34c4d7c0 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.741+0000 7ff6b8fabb80 1 --2- 172.21.15.1:0/1928158167 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fd34c3b9f0 0x55fd34c38c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.741+0000 7ff6b8fabb80 1 --2- 172.21.15.1:0/1928158167 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd34c301f0 0x55fd34c3b3f0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.741+0000 7ff6b8fabb80 1 --2- 172.21.15.1:0/1928158167 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fd34c30b10 0x55fd34c38600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.741+0000 7ff687e00700 0 client.19467 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:42:03.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.741+0000 7ff6b8fabb80 1 -- 172.21.15.1:0/1928158167 >> 172.21.15.1:0/1928158167 conn(0x55fd34b8fdb0 msgr2=0x55fd34c2c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:03.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.741+0000 7ff6b8fabb80 1 -- 172.21.15.1:0/1928158167 shutdown_connections 2024-02-02T15:42:03.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.741+0000 7ff6b8fabb80 1 -- 172.21.15.1:0/1928158167 wait complete. 2024-02-02T15:42:03.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.741+0000 7ff6b8fabb80 20 client.19467 trim_cache size 0 max 16384 2024-02-02T15:42:03.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.755+0000 7ff6b8fabb80 1 Processor -- start 2024-02-02T15:42:03.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.755+0000 7ff6b8fabb80 1 -- start start 2024-02-02T15:42:03.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.755+0000 7ff6b8fabb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd34c301f0 0x55fd34c3af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:03.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.755+0000 7ff6b8fabb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fd34c30b10 0x55fd34c33950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:03.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.755+0000 7ff6b8fabb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fd34c3b9f0 0x55fd34c33e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:03.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.755+0000 7ff6b8fabb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd34a056a0 con 0x55fd34c301f0 2024-02-02T15:42:03.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.755+0000 7ff6b8fabb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd34c36bb0 con 0x55fd34c3b9f0 2024-02-02T15:42:03.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.755+0000 7ff6b8fabb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd34c344c0 con 0x55fd34c30b10 2024-02-02T15:42:03.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.756+0000 7ff69e800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd34c301f0 0x55fd34c3af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:03.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.756+0000 7ff69e800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd34c301f0 0x55fd34c3af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49892/0 (socket says 172.21.15.1:49892) 2024-02-02T15:42:03.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.756+0000 7ff69e800700 1 -- 172.21.15.1:0/1042362272 learned_addr learned my addr 172.21.15.1:0/1042362272 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:42:03.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.756+0000 7ff69de00700 1 --2- 172.21.15.1:0/1042362272 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fd34c30b10 0x55fd34c33950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:03.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.756+0000 7ff69f200700 1 --2- 172.21.15.1:0/1042362272 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fd34c3b9f0 0x55fd34c33e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:03.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.756+0000 7ff69e800700 1 -- 172.21.15.1:0/1042362272 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fd34c30b10 msgr2=0x55fd34c33950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:03.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.756+0000 7ff69e800700 1 --2- 172.21.15.1:0/1042362272 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fd34c30b10 0x55fd34c33950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.756+0000 7ff69e800700 1 -- 172.21.15.1:0/1042362272 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fd34c3b9f0 msgr2=0x55fd34c33e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:03.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.756+0000 7ff69e800700 1 --2- 172.21.15.1:0/1042362272 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fd34c3b9f0 0x55fd34c33e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.756+0000 7ff69e800700 1 -- 172.21.15.1:0/1042362272 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fd34b47ea0 con 0x55fd34c301f0 2024-02-02T15:42:03.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.756+0000 7ff69de00700 1 --2- 172.21.15.1:0/1042362272 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fd34c30b10 0x55fd34c33950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:42:03.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.756+0000 7ff69e800700 1 --2- 172.21.15.1:0/1042362272 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd34c301f0 0x55fd34c3af20 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7ff69000c5b0 tx=0x7ff690097a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddad3a24b8d73192 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:03.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.756+0000 7ff68fe00700 1 -- 172.21.15.1:0/1042362272 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6900cc170 con 0x55fd34c301f0 2024-02-02T15:42:03.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.756+0000 7ff68fe00700 1 -- 172.21.15.1:0/1042362272 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff690065240 con 0x55fd34c301f0 2024-02-02T15:42:03.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.756+0000 7ff68fe00700 1 -- 172.21.15.1:0/1042362272 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff690010820 con 0x55fd34c301f0 2024-02-02T15:42:03.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.757+0000 7ff6b8fabb80 1 -- 172.21.15.1:0/1042362272 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd34c301f0 msgr2=0x55fd34c3af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:03.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.757+0000 7ff6b8fabb80 1 --2- 172.21.15.1:0/1042362272 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd34c301f0 0x55fd34c3af20 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7ff69000c5b0 tx=0x7ff690097a40 comp rx=0 tx=0).stop 2024-02-02T15:42:03.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.757+0000 7ff6b8fabb80 1 -- 172.21.15.1:0/1042362272 shutdown_connections 2024-02-02T15:42:03.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.757+0000 7ff6b8fabb80 1 --2- 172.21.15.1:0/1042362272 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fd34c3b9f0 0x55fd34c33e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.757+0000 7ff6b8fabb80 1 --2- 172.21.15.1:0/1042362272 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd34c301f0 0x55fd34c3af20 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.757+0000 7ff6b8fabb80 1 --2- 172.21.15.1:0/1042362272 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fd34c30b10 0x55fd34c33950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.757+0000 7ff6b8fabb80 1 -- 172.21.15.1:0/1042362272 >> 172.21.15.1:0/1042362272 conn(0x55fd34b8fdb0 msgr2=0x55fd34c31a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:03.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.757+0000 7ff6b8fabb80 1 -- 172.21.15.1:0/1042362272 shutdown_connections 2024-02-02T15:42:03.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.757+0000 7ff6b8fabb80 1 -- 172.21.15.1:0/1042362272 wait complete. 2024-02-02T15:42:03.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.758+0000 7ff6b8fabb80 1 -- start start 2024-02-02T15:42:03.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.758+0000 7ff6b8fabb80 1 Processor -- start 2024-02-02T15:42:03.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.758+0000 7ff6b8fabb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fd34c301f0 0x55fd34c40490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:03.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.758+0000 7ff6b8fabb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd34c30b10 0x55fd34c409d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:03.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.758+0000 7ff6b8fabb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fd34c3b9f0 0x55fd34c40f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:03.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.758+0000 7ff6b8fabb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd34c9bf10 con 0x55fd34c30b10 2024-02-02T15:42:03.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.758+0000 7ff6b8fabb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd34c9c050 con 0x55fd34c301f0 2024-02-02T15:42:03.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.758+0000 7ff6b8fabb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fd34c9c210 con 0x55fd34c3b9f0 2024-02-02T15:42:03.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.758+0000 7ff6b8fabb80 10 client.0 _ll_register_callbacks cb 0x55fd34b1f290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:42:03.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.758+0000 7ff69de00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd34c30b10 0x55fd34c409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:03.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.759+0000 7ff69e800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fd34c301f0 0x55fd34c40490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:03.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.759+0000 7ff69de00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd34c30b10 0x55fd34c409d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49900/0 (socket says 172.21.15.1:49900) 2024-02-02T15:42:03.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.759+0000 7ff69de00700 1 -- 172.21.15.1:0/2242889976 learned_addr learned my addr 172.21.15.1:0/2242889976 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:42:03.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.759+0000 7ff69f200700 1 --2- 172.21.15.1:0/2242889976 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fd34c3b9f0 0x55fd34c40f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:03.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.759+0000 7ff69de00700 1 -- 172.21.15.1:0/2242889976 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fd34c3b9f0 msgr2=0x55fd34c40f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:03.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.759+0000 7ff69de00700 1 --2- 172.21.15.1:0/2242889976 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fd34c3b9f0 0x55fd34c40f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.759+0000 7ff69de00700 1 -- 172.21.15.1:0/2242889976 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fd34c301f0 msgr2=0x55fd34c40490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:03.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.759+0000 7ff69de00700 1 --2- 172.21.15.1:0/2242889976 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fd34c301f0 0x55fd34c40490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.759+0000 7ff69de00700 1 -- 172.21.15.1:0/2242889976 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fd349afb60 con 0x55fd34c30b10 2024-02-02T15:42:03.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.759+0000 7ff69e800700 1 --2- 172.21.15.1:0/2242889976 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fd34c301f0 0x55fd34c40490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:42:03.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.759+0000 7ff69de00700 1 --2- 172.21.15.1:0/2242889976 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd34c30b10 0x55fd34c409d0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7ff688005640 tx=0x7ff68800b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f274143c7bec99d2 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:03.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.759+0000 7ff687e00700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:42:03.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.759+0000 7ff687e00700 1 -- 172.21.15.1:0/2242889976 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff688031070 con 0x55fd34c30b10 2024-02-02T15:42:03.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.759+0000 7ff6b8fabb80 1 -- 172.21.15.1:0/2242889976 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff68801e040 con 0x55fd34c30b10 2024-02-02T15:42:03.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.759+0000 7ff687e00700 1 -- 172.21.15.1:0/2242889976 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9453+0+0 (secure 0 0 0) 0x7ff68800bcd0 con 0x55fd34c30b10 2024-02-02T15:42:03.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.760+0000 7ff687e00700 1 -- 172.21.15.1:0/2242889976 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff68801d040 con 0x55fd34c30b10 2024-02-02T15:42:03.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.760+0000 7ff6b8fabb80 10 client.19473 fetch_fsmap learned FSMap version 61 2024-02-02T15:42:03.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.760+0000 7ff6b8fabb80 1 -- 172.21.15.1:0/2242889976 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x55fd34a07cd0 con 0x55fd34c30b10 2024-02-02T15:42:03.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.760+0000 7ff687e00700 1 -- 172.21.15.1:0/2242889976 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff68801e040 con 0x55fd34c30b10 2024-02-02T15:42:03.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.760+0000 7ff687e00700 1 -- 172.21.15.1:0/2242889976 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 61) v1 ==== 7550+0+0 (secure 0 0 0) 0x7ff68803f070 con 0x55fd34c30b10 2024-02-02T15:42:03.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.760+0000 7ff6b8fabb80 10 client.19473 fetch_fsmap finished waiting for FSMap version 61 2024-02-02T15:42:03.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.760+0000 7ff6b8fabb80 10 client.19473 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:42:03.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.760+0000 7ff6b8fabb80 20 client.19473 populate_metadata read hostname 'smithi001' 2024-02-02T15:42:03.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.760+0000 7ff6b8fabb80 1 --2- 172.21.15.1:0/2242889976 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55fd34c9d560 0x55fd34c9f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:03.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.760+0000 7ff6b8fabb80 4 client.19473 mds_command: new command op to 24557 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:42:03.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.760+0000 7ff6b8fabb80 1 -- 172.21.15.1:0/2242889976 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55fd34b40f50 con 0x55fd34c9d560 2024-02-02T15:42:03.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.761+0000 7ff69e800700 1 --2- 172.21.15.1:0/2242889976 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55fd34c9d560 0x55fd34c9f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:03.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.761+0000 7ff69e800700 1 --2- 172.21.15.1:0/2242889976 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55fd34c9d560 0x55fd34c9f980 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d862f886b1cc5a79 server_cookie=e2c0ba29aafcc739 in_seq=0 out_seq=0 2024-02-02T15:42:03.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.761+0000 7ff687e00700 10 client.19473 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:42:03.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.761+0000 7ff687e00700 1 -- 172.21.15.1:0/2242889976 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55fd34b40f50 con 0x55fd34c9d560 2024-02-02T15:42:03.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.761+0000 7ff687e00700 10 client.19473 handle_command_reply: tid=0 2024-02-02T15:42:03.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.761+0000 7ff6b8fabb80 1 client.19473 shutdown 2024-02-02T15:42:03.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.762+0000 7ff6b8fabb80 1 -- 172.21.15.1:0/2242889976 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd34c30b10 msgr2=0x55fd34c409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:03.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.762+0000 7ff6b8fabb80 1 --2- 172.21.15.1:0/2242889976 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd34c30b10 0x55fd34c409d0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7ff688005640 tx=0x7ff68800b390 comp rx=0 tx=0).stop 2024-02-02T15:42:03.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.762+0000 7ff6b8fabb80 1 -- 172.21.15.1:0/2242889976 shutdown_connections 2024-02-02T15:42:03.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.762+0000 7ff6b8fabb80 1 --2- 172.21.15.1:0/2242889976 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55fd34c9d560 0x55fd34c9f980 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.762+0000 7ff6b8fabb80 1 --2- 172.21.15.1:0/2242889976 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fd34c3b9f0 0x55fd34c40f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.762+0000 7ff6b8fabb80 1 --2- 172.21.15.1:0/2242889976 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fd34c301f0 0x55fd34c40490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.762+0000 7ff6b8fabb80 1 --2- 172.21.15.1:0/2242889976 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fd34c30b10 0x55fd34c409d0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.762+0000 7ff687e00700 0 client.19473 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:42:03.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.762+0000 7ff6b8fabb80 1 -- 172.21.15.1:0/2242889976 >> 172.21.15.1:0/2242889976 conn(0x55fd34b8fdb0 msgr2=0x55fd34c46a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:03.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.762+0000 7ff6b8fabb80 1 -- 172.21.15.1:0/2242889976 shutdown_connections 2024-02-02T15:42:03.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.762+0000 7ff6b8fabb80 1 -- 172.21.15.1:0/2242889976 wait complete. 2024-02-02T15:42:03.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.762+0000 7ff6b8fabb80 20 client.19473 trim_cache size 0 max 16384 2024-02-02T15:42:03.770 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:42:03.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.762+0000 7ff6a4a00700 1 -- 172.21.15.1:0/2889634781 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff6800a0690 msgr2=0x7ff6800a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:03.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.762+0000 7ff6a4a00700 1 --2- 172.21.15.1:0/2889634781 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff6800a0690 0x7ff6800a2b50 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ff698009fd0 tx=0x7ff69800b040 comp rx=0 tx=0).stop 2024-02-02T15:42:03.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.762+0000 7ff6a4a00700 1 -- 172.21.15.1:0/2889634781 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff6a0102f40 msgr2=0x7ff6a0198df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:03.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.762+0000 7ff6a4a00700 1 --2- 172.21.15.1:0/2889634781 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff6a0102f40 0x7ff6a0198df0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7ff69000e9a0 tx=0x7ff69000ecb0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.763+0000 7ff6a4a00700 1 -- 172.21.15.1:0/2889634781 shutdown_connections 2024-02-02T15:42:03.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.763+0000 7ff6a4a00700 1 --2- 172.21.15.1:0/2889634781 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff6800a0690 0x7ff6800a2b50 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.763+0000 7ff6a4a00700 1 --2- 172.21.15.1:0/2889634781 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff6a019d790 0x7ff6a019dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.763+0000 7ff6a4a00700 1 --2- 172.21.15.1:0/2889634781 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff6a0102f40 0x7ff6a0198df0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.763+0000 7ff6a4a00700 1 --2- 172.21.15.1:0/2889634781 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff6a0108060 0x7ff6a0199330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:03.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.763+0000 7ff6a4a00700 1 -- 172.21.15.1:0/2889634781 >> 172.21.15.1:0/2889634781 conn(0x7ff6a00fe780 msgr2=0x7ff6a00ff280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:03.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.763+0000 7ff6a4a00700 1 -- 172.21.15.1:0/2889634781 shutdown_connections 2024-02-02T15:42:03.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:03.763+0000 7ff6a4a00700 1 -- 172.21.15.1:0/2889634781 wait complete. 2024-02-02T15:42:03.783 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:42:03 smithi001 ceph-mon[35470]: pgmap v470: 129 pgs: 129 active+clean; 1.2 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 8.6 MiB/s wr, 677 op/s 2024-02-02T15:42:03.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:42:03 smithi071 ceph-mon[44707]: pgmap v470: 129 pgs: 129 active+clean; 1.2 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 8.6 MiB/s wr, 677 op/s 2024-02-02T15:42:03.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:42:03 smithi142 ceph-mon[45728]: pgmap v470: 129 pgs: 129 active+clean; 1.2 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 8.6 MiB/s wr, 677 op/s 2024-02-02T15:42:03.978 INFO:tasks.workunit.client.0.smithi001.stdout: 10 55127 32.75 MB/sec warmup 96 sec latency 3090.364 ms 2024-02-02T15:42:04.786 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-02T15:42:04.787 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-02T15:42:04.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:42:04 smithi071 ceph-mon[44707]: pgmap v471: 129 pgs: 129 active+clean; 1.2 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 5.6 MiB/s wr, 442 op/s 2024-02-02T15:42:04.882 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:42:04 smithi001 ceph-mon[35470]: pgmap v471: 129 pgs: 129 active+clean; 1.2 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 5.6 MiB/s wr, 442 op/s 2024-02-02T15:42:04.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:42:04 smithi142 ceph-mon[45728]: pgmap v471: 129 pgs: 129 active+clean; 1.2 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 5.6 MiB/s wr, 442 op/s 2024-02-02T15:42:04.978 INFO:tasks.workunit.client.0.smithi001.stdout: 10 55148 32.42 MB/sec warmup 97 sec latency 3994.689 ms 2024-02-02T15:42:05.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.109+0000 7fbb81400700 1 -- 172.21.15.1:0/3001896533 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb7c1008c0 msgr2=0x7fbb7c104900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:05.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.109+0000 7fbb81400700 1 --2- 172.21.15.1:0/3001896533 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb7c1008c0 0x7fbb7c104900 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c009990 tx=0x7fbb6c009ca0 comp rx=0 tx=0).stop 2024-02-02T15:42:05.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.110+0000 7fbb81400700 1 -- 172.21.15.1:0/3001896533 shutdown_connections 2024-02-02T15:42:05.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.110+0000 7fbb81400700 1 --2- 172.21.15.1:0/3001896533 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbb7c105030 0x7fbb7c068d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:05.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.111+0000 7fbb81400700 1 --2- 172.21.15.1:0/3001896533 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb7c0fff10 0x7fbb7c1002f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:05.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.111+0000 7fbb81400700 1 --2- 172.21.15.1:0/3001896533 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb7c1008c0 0x7fbb7c104900 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:05.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.111+0000 7fbb81400700 1 -- 172.21.15.1:0/3001896533 >> 172.21.15.1:0/3001896533 conn(0x7fbb7c076e20 msgr2=0x7fbb7c077230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:05.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.112+0000 7fbb81400700 1 -- 172.21.15.1:0/3001896533 shutdown_connections 2024-02-02T15:42:05.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.112+0000 7fbb81400700 1 -- 172.21.15.1:0/3001896533 wait complete. 2024-02-02T15:42:05.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.114+0000 7fbb81400700 1 Processor -- start 2024-02-02T15:42:05.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.114+0000 7fbb81400700 1 -- start start 2024-02-02T15:42:05.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.115+0000 7fbb81400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb7c0fff10 0x7fbb7c10a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:05.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.115+0000 7fbb81400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbb7c105030 0x7fbb7c10a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:05.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.116+0000 7fbb81400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb7c107230 0x7fbb7c1076e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:05.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.116+0000 7fbb81400700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7c107eb0 con 0x7fbb7c105030 2024-02-02T15:42:05.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.116+0000 7fbb81400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7c19f630 con 0x7fbb7c0fff10 2024-02-02T15:42:05.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.116+0000 7fbb81400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7c19f7f0 con 0x7fbb7c107230 2024-02-02T15:42:05.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.116+0000 7fbb7a800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb7c0fff10 0x7fbb7c10a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:05.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.116+0000 7fbb7a800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb7c0fff10 0x7fbb7c10a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59378/0 (socket says 172.21.15.1:59378) 2024-02-02T15:42:05.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.116+0000 7fbb7a800700 1 -- 172.21.15.1:0/3751229234 learned_addr learned my addr 172.21.15.1:0/3751229234 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:42:05.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.116+0000 7fbb79e00700 1 --2- 172.21.15.1:0/3751229234 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbb7c105030 0x7fbb7c10a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:05.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.116+0000 7fbb7b200700 1 --2- 172.21.15.1:0/3751229234 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb7c107230 0x7fbb7c1076e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:05.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.117+0000 7fbb7a800700 1 -- 172.21.15.1:0/3751229234 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb7c107230 msgr2=0x7fbb7c1076e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:05.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.117+0000 7fbb7a800700 1 --2- 172.21.15.1:0/3751229234 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb7c107230 0x7fbb7c1076e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:05.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.117+0000 7fbb7a800700 1 -- 172.21.15.1:0/3751229234 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbb7c105030 msgr2=0x7fbb7c10a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:05.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.117+0000 7fbb7a800700 1 --2- 172.21.15.1:0/3751229234 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbb7c105030 0x7fbb7c10a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:05.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.117+0000 7fbb7a800700 1 -- 172.21.15.1:0/3751229234 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb6c0095f0 con 0x7fbb7c0fff10 2024-02-02T15:42:05.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.117+0000 7fbb7a800700 1 --2- 172.21.15.1:0/3751229234 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb7c0fff10 0x7fbb7c10a0c0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fbb6400d950 tx=0x7fbb6400dd10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:05.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.118+0000 7fbb73400700 1 -- 172.21.15.1:0/3751229234 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb64009b40 con 0x7fbb7c0fff10 2024-02-02T15:42:05.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.118+0000 7fbb73400700 1 -- 172.21.15.1:0/3751229234 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb64009ca0 con 0x7fbb7c0fff10 2024-02-02T15:42:05.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.118+0000 7fbb73400700 1 -- 172.21.15.1:0/3751229234 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb6400f620 con 0x7fbb7c0fff10 2024-02-02T15:42:05.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.118+0000 7fbb81400700 1 -- 172.21.15.1:0/3751229234 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb7c19fa70 con 0x7fbb7c0fff10 2024-02-02T15:42:05.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.118+0000 7fbb79e00700 1 --2- 172.21.15.1:0/3751229234 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbb7c105030 0x7fbb7c10a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:42:05.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.118+0000 7fbb81400700 1 -- 172.21.15.1:0/3751229234 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb7c19ff30 con 0x7fbb7c0fff10 2024-02-02T15:42:05.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.119+0000 7fbb7b200700 1 --2- 172.21.15.1:0/3751229234 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb7c107230 0x7fbb7c1076e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:42:05.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.124+0000 7fbb73400700 1 -- 172.21.15.1:0/3751229234 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7fbb64010460 con 0x7fbb7c0fff10 2024-02-02T15:42:05.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.124+0000 7fbb94fe5b80 1 Processor -- start 2024-02-02T15:42:05.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.127+0000 7fbb94fe5b80 1 -- start start 2024-02-02T15:42:05.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.128+0000 7fbb94fe5b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe316ae090 0x55fe316ae470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:05.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.128+0000 7fbb94fe5b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe316ae9b0 0x55fe316b9320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:05.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.128+0000 7fbb94fe5b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe316b9860 0x55fe316bbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:05.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.128+0000 7fbb94fe5b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe31622540 con 0x55fe316ae9b0 2024-02-02T15:42:05.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.128+0000 7fbb94fe5b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe31606b20 con 0x55fe316ae090 2024-02-02T15:42:05.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.128+0000 7fbb94fe5b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe316044e0 con 0x55fe316b9860 2024-02-02T15:42:05.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.129+0000 7fbb79e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe316b9860 0x55fe316bbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:05.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.129+0000 7fbb79e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe316b9860 0x55fe316bbc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:42386/0 (socket says 172.21.15.1:42386) 2024-02-02T15:42:05.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.129+0000 7fbb79e00700 1 -- 172.21.15.1:0/543902844 learned_addr learned my addr 172.21.15.1:0/543902844 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:42:05.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.129+0000 7fbb73400700 1 --2- 172.21.15.1:0/3751229234 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fbb5c0a0790 0x7fbb5c0a2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:05.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.129+0000 7fbb7a800700 1 --2- 172.21.15.1:0/543902844 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe316ae9b0 0x55fe316b9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:05.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.129+0000 7fbb7b200700 1 --2- 172.21.15.1:0/543902844 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe316ae090 0x55fe316ae470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:05.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.131+0000 7fbb7b200700 1 -- 172.21.15.1:0/543902844 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe316b9860 msgr2=0x55fe316bbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:05.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.131+0000 7fbb7b200700 1 --2- 172.21.15.1:0/543902844 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe316b9860 0x55fe316bbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:05.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.131+0000 7fbb7b200700 1 -- 172.21.15.1:0/543902844 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe316ae9b0 msgr2=0x55fe316b9320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:05.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.131+0000 7fbb7b200700 1 --2- 172.21.15.1:0/543902844 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe316ae9b0 0x55fe316b9320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:05.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.130+0000 7fbb73400700 1 -- 172.21.15.1:0/3751229234 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fbb640cc060 con 0x7fbb7c0fff10 2024-02-02T15:42:05.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.131+0000 7fbb7b200700 1 -- 172.21.15.1:0/543902844 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe31481430 con 0x55fe316ae090 2024-02-02T15:42:05.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.131+0000 7fbb7b200700 1 --2- 172.21.15.1:0/3751229234 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fbb5c0a0790 0x7fbb5c0a2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:05.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.131+0000 7fbb79e00700 1 --2- 172.21.15.1:0/543902844 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe316b9860 0x55fe316bbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:42:05.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.131+0000 7fbb7b200700 1 --2- 172.21.15.1:0/543902844 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe316ae090 0x55fe316ae470 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fbb74009410 tx=0x7fbb740077e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97edbab7914646ed server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:05.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.132+0000 7fbb6b400700 1 -- 172.21.15.1:0/543902844 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb7400e070 con 0x55fe316ae090 2024-02-02T15:42:05.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.132+0000 7fbb6b400700 1 -- 172.21.15.1:0/543902844 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb7400f040 con 0x55fe316ae090 2024-02-02T15:42:05.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.132+0000 7fbb6b400700 1 -- 172.21.15.1:0/543902844 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb74020bf0 con 0x55fe316ae090 2024-02-02T15:42:05.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.133+0000 7fbb7b200700 1 --2- 172.21.15.1:0/3751229234 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fbb5c0a0790 0x7fbb5c0a2c50 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fbb7401f000 tx=0x7fbb740166b0 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:05.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.135+0000 7fbb94fe5b80 1 -- 172.21.15.1:0/543902844 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe316ae090 msgr2=0x55fe316ae470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:05.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.135+0000 7fbb94fe5b80 1 --2- 172.21.15.1:0/543902844 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe316ae090 0x55fe316ae470 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fbb74009410 tx=0x7fbb740077e0 comp rx=0 tx=0).stop 2024-02-02T15:42:05.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.136+0000 7fbb94fe5b80 1 -- 172.21.15.1:0/543902844 shutdown_connections 2024-02-02T15:42:05.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.136+0000 7fbb94fe5b80 1 --2- 172.21.15.1:0/543902844 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe316b9860 0x55fe316bbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:05.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.136+0000 7fbb94fe5b80 1 --2- 172.21.15.1:0/543902844 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe316ae090 0x55fe316ae470 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:05.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.136+0000 7fbb94fe5b80 1 --2- 172.21.15.1:0/543902844 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe316ae9b0 0x55fe316b9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:05.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.136+0000 7fbb94fe5b80 1 -- 172.21.15.1:0/543902844 >> 172.21.15.1:0/543902844 conn(0x55fe3160de30 msgr2=0x55fe316aa430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:05.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.137+0000 7fbb94fe5b80 1 -- 172.21.15.1:0/543902844 shutdown_connections 2024-02-02T15:42:05.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.137+0000 7fbb94fe5b80 1 -- 172.21.15.1:0/543902844 wait complete. 2024-02-02T15:42:05.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.138+0000 7fbb94fe5b80 1 -- start start 2024-02-02T15:42:05.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.138+0000 7fbb94fe5b80 1 Processor -- start 2024-02-02T15:42:05.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.138+0000 7fbb94fe5b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe316ae090 0x55fe316b92e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:05.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.138+0000 7fbb94fe5b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe316ae9b0 0x55fe316b64f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:05.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.139+0000 7fbb94fe5b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe316b9860 0x55fe316b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:05.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.139+0000 7fbb94fe5b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe315ffdd0 con 0x55fe316b9860 2024-02-02T15:42:05.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.139+0000 7fbb94fe5b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe3159be00 con 0x55fe316ae9b0 2024-02-02T15:42:05.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.139+0000 7fbb94fe5b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe3159baa0 con 0x55fe316ae090 2024-02-02T15:42:05.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.139+0000 7fbb94fe5b80 10 client.0 _ll_register_callbacks cb 0x55fe3159d310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:42:05.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.139+0000 7fbb7a800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe316ae090 0x55fe316b92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:05.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.139+0000 7fbb7a800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe316ae090 0x55fe316b92e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:42402/0 (socket says 172.21.15.1:42402) 2024-02-02T15:42:05.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.139+0000 7fbb7a800700 1 -- 172.21.15.1:0/2372841413 learned_addr learned my addr 172.21.15.1:0/2372841413 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:42:05.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.139+0000 7fbb79e00700 1 --2- 172.21.15.1:0/2372841413 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe316ae9b0 0x55fe316b64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:05.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.140+0000 7fbb7a800700 1 -- 172.21.15.1:0/2372841413 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe316ae9b0 msgr2=0x55fe316b64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:05.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.140+0000 7fbb7a800700 1 --2- 172.21.15.1:0/2372841413 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe316ae9b0 0x55fe316b64f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:05.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.140+0000 7fbb7a800700 1 -- 172.21.15.1:0/2372841413 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe316b9860 msgr2=0x55fe316b6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:05.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.140+0000 7fbb7a800700 1 --2- 172.21.15.1:0/2372841413 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe316b9860 0x55fe316b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:05.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.140+0000 7fbb7a800700 1 -- 172.21.15.1:0/2372841413 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb7c0695c0 con 0x55fe316ae090 2024-02-02T15:42:05.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.143+0000 7fbb7a800700 1 --2- 172.21.15.1:0/2372841413 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe316ae090 0x55fe316b92e0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fbb640a0cc0 tx=0x7fbb64098c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d588c445b5fbf9c2 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:05.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.143+0000 7fbb63e00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:42:05.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.144+0000 7fbb63e00700 1 -- 172.21.15.1:0/2372841413 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb6409cfb0 con 0x55fe316ae090 2024-02-02T15:42:05.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.144+0000 7fbb63e00700 1 -- 172.21.15.1:0/2372841413 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fbb6409b070 con 0x55fe316ae090 2024-02-02T15:42:05.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.144+0000 7fbb94fe5b80 1 -- 172.21.15.1:0/2372841413 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fe316a2170 con 0x55fe316ae090 2024-02-02T15:42:05.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.144+0000 7fbb63e00700 1 -- 172.21.15.1:0/2372841413 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb640a11f0 con 0x55fe316ae090 2024-02-02T15:42:05.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.144+0000 7fbb94fe5b80 10 client.29596 fetch_fsmap learned FSMap version 61 2024-02-02T15:42:05.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.144+0000 7fbb94fe5b80 1 -- 172.21.15.1:0/2372841413 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x7fbb74009090 con 0x55fe316ae090 2024-02-02T15:42:05.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.145+0000 7fbb63e00700 1 -- 172.21.15.1:0/2372841413 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55fe316a2170 con 0x55fe316ae090 2024-02-02T15:42:05.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.145+0000 7fbb63e00700 1 -- 172.21.15.1:0/2372841413 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 61) v1 ==== 7550+0+0 (secure 0 0 0) 0x7fbb640a0d60 con 0x55fe316ae090 2024-02-02T15:42:05.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.145+0000 7fbb94fe5b80 10 client.29596 fetch_fsmap finished waiting for FSMap version 61 2024-02-02T15:42:05.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.145+0000 7fbb94fe5b80 10 client.29596 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:42:05.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.145+0000 7fbb94fe5b80 20 client.29596 populate_metadata read hostname 'smithi001' 2024-02-02T15:42:05.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.145+0000 7fbb94fe5b80 1 --2- 172.21.15.1:0/2372841413 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55fe316c9170 0x55fe316cb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:05.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.145+0000 7fbb94fe5b80 4 client.29596 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:42:05.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.145+0000 7fbb94fe5b80 1 -- 172.21.15.1:0/2372841413 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fe315f8e50 con 0x55fe316c9170 2024-02-02T15:42:05.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.147+0000 7fbb79e00700 1 --2- 172.21.15.1:0/2372841413 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55fe316c9170 0x55fe316cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:05.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.147+0000 7fbb79e00700 1 --2- 172.21.15.1:0/2372841413 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55fe316c9170 0x55fe316cb590 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d662444041609ef server_cookie=5564533685964f71 in_seq=0 out_seq=0 2024-02-02T15:42:05.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.147+0000 7fbb63e00700 10 client.29596 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:42:05.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.149+0000 7fbb63e00700 1 -- 172.21.15.1:0/2372841413 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55fe315f8e50 con 0x55fe316c9170 2024-02-02T15:42:05.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.149+0000 7fbb63e00700 10 client.29596 handle_command_reply: tid=0 2024-02-02T15:42:05.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.149+0000 7fbb94fe5b80 1 client.29596 shutdown 2024-02-02T15:42:05.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.150+0000 7fbb94fe5b80 1 -- 172.21.15.1:0/2372841413 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe316ae090 msgr2=0x55fe316b92e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:05.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.150+0000 7fbb94fe5b80 1 --2- 172.21.15.1:0/2372841413 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe316ae090 0x55fe316b92e0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fbb640a0cc0 tx=0x7fbb64098c10 comp rx=0 tx=0).stop 2024-02-02T15:42:05.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.150+0000 7fbb94fe5b80 1 -- 172.21.15.1:0/2372841413 shutdown_connections 2024-02-02T15:42:05.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.150+0000 7fbb94fe5b80 1 --2- 172.21.15.1:0/2372841413 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55fe316c9170 0x55fe316cb590 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:05.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.150+0000 7fbb94fe5b80 1 --2- 172.21.15.1:0/2372841413 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe316b9860 0x55fe316b6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:05.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.150+0000 7fbb63e00700 0 client.29596 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:42:05.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.150+0000 7fbb94fe5b80 1 --2- 172.21.15.1:0/2372841413 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe316ae090 0x55fe316b92e0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:05.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.150+0000 7fbb94fe5b80 1 --2- 172.21.15.1:0/2372841413 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe316ae9b0 0x55fe316b64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:05.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.150+0000 7fbb94fe5b80 1 -- 172.21.15.1:0/2372841413 >> 172.21.15.1:0/2372841413 conn(0x55fe3160de30 msgr2=0x55fe316aa020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:05.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.150+0000 7fbb94fe5b80 1 -- 172.21.15.1:0/2372841413 shutdown_connections 2024-02-02T15:42:05.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.152+0000 7fbb94fe5b80 1 -- 172.21.15.1:0/2372841413 wait complete. 2024-02-02T15:42:05.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.152+0000 7fbb94fe5b80 20 client.29596 trim_cache size 0 max 16384 2024-02-02T15:42:05.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.186+0000 7fbb94fe5b80 1 Processor -- start 2024-02-02T15:42:05.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.186+0000 7fbb94fe5b80 1 -- start start 2024-02-02T15:42:05.186 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.186+0000 7fbb94fe5b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe316ae090 0x55fe316c03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:05.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.186+0000 7fbb94fe5b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe316ae9b0 0x55fe316b8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:05.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.186+0000 7fbb94fe5b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe316b9860 0x55fe316b1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:05.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.186+0000 7fbb94fe5b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe31483720 con 0x55fe316ae9b0 2024-02-02T15:42:05.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.186+0000 7fbb94fe5b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe316aeff0 con 0x55fe316ae090 2024-02-02T15:42:05.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.186+0000 7fbb94fe5b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe316b5160 con 0x55fe316b9860 2024-02-02T15:42:05.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.187+0000 7fbb7a800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe316ae090 0x55fe316c03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:05.187 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.187+0000 7fbb7a800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe316ae090 0x55fe316c03b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59408/0 (socket says 172.21.15.1:59408) 2024-02-02T15:42:05.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.187+0000 7fbb7a800700 1 -- 172.21.15.1:0/211195682 learned_addr learned my addr 172.21.15.1:0/211195682 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:42:05.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.187+0000 7fbb79e00700 1 --2- 172.21.15.1:0/211195682 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe316ae9b0 0x55fe316b8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:05.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.188+0000 7fbb7a800700 1 -- 172.21.15.1:0/211195682 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe316b9860 msgr2=0x55fe316b1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:05.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.188+0000 7fbb7a800700 1 --2- 172.21.15.1:0/211195682 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe316b9860 0x55fe316b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:05.188 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.189+0000 7fbb7a800700 1 -- 172.21.15.1:0/211195682 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe316ae9b0 msgr2=0x55fe316b8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:05.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.189+0000 7fbb7a800700 1 --2- 172.21.15.1:0/211195682 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe316ae9b0 0x55fe316b8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:05.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.189+0000 7fbb7a800700 1 -- 172.21.15.1:0/211195682 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe316b6060 con 0x55fe316ae090 2024-02-02T15:42:05.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.191+0000 7fbb7a800700 1 --2- 172.21.15.1:0/211195682 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe316ae090 0x55fe316c03b0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fbb64010fd0 tx=0x7fbb640954c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=868492e839818496 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:05.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.193+0000 7fbb6be00700 1 -- 172.21.15.1:0/211195682 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb64014070 con 0x55fe316ae090 2024-02-02T15:42:05.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.193+0000 7fbb6be00700 1 -- 172.21.15.1:0/211195682 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb64010860 con 0x55fe316ae090 2024-02-02T15:42:05.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.193+0000 7fbb6be00700 1 -- 172.21.15.1:0/211195682 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb640a0d60 con 0x55fe316ae090 2024-02-02T15:42:05.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.196+0000 7fbb94fe5b80 1 -- 172.21.15.1:0/211195682 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe316ae090 msgr2=0x55fe316c03b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:05.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.196+0000 7fbb94fe5b80 1 --2- 172.21.15.1:0/211195682 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe316ae090 0x55fe316c03b0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fbb64010fd0 tx=0x7fbb640954c0 comp rx=0 tx=0).stop 2024-02-02T15:42:05.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.198+0000 7fbb94fe5b80 1 -- 172.21.15.1:0/211195682 shutdown_connections 2024-02-02T15:42:05.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.198+0000 7fbb94fe5b80 1 --2- 172.21.15.1:0/211195682 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe316b9860 0x55fe316b1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:05.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.198+0000 7fbb94fe5b80 1 --2- 172.21.15.1:0/211195682 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe316ae090 0x55fe316c03b0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:05.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.198+0000 7fbb94fe5b80 1 --2- 172.21.15.1:0/211195682 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe316ae9b0 0x55fe316b8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:05.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.198+0000 7fbb94fe5b80 1 -- 172.21.15.1:0/211195682 >> 172.21.15.1:0/211195682 conn(0x55fe3160de30 msgr2=0x55fe316b4a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:05.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.198+0000 7fbb94fe5b80 1 -- 172.21.15.1:0/211195682 shutdown_connections 2024-02-02T15:42:05.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.198+0000 7fbb94fe5b80 1 -- 172.21.15.1:0/211195682 wait complete. 2024-02-02T15:42:05.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.199+0000 7fbb94fe5b80 1 -- start start 2024-02-02T15:42:05.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.199+0000 7fbb94fe5b80 1 Processor -- start 2024-02-02T15:42:05.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.201+0000 7fbb94fe5b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe316ae090 0x55fe316be940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:05.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.201+0000 7fbb94fe5b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe316ae9b0 0x55fe316bee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:05.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.201+0000 7fbb94fe5b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe316b9860 0x55fe316bf3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:05.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.201+0000 7fbb94fe5b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe31714d80 con 0x55fe316b9860 2024-02-02T15:42:05.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.201+0000 7fbb94fe5b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe31714ef0 con 0x55fe316ae090 2024-02-02T15:42:05.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.201+0000 7fbb94fe5b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe317150b0 con 0x55fe316ae9b0 2024-02-02T15:42:05.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.201+0000 7fbb94fe5b80 10 client.0 _ll_register_callbacks cb 0x55fe3159d310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:42:05.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.202+0000 7fbb7b200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe316b9860 0x55fe316bf3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:05.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.202+0000 7fbb7a800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe316ae090 0x55fe316be940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:05.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.202+0000 7fbb7a800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe316ae090 0x55fe316be940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59418/0 (socket says 172.21.15.1:59418) 2024-02-02T15:42:05.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.202+0000 7fbb7b200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe316b9860 0x55fe316bf3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:49968/0 (socket says 172.21.15.1:49968) 2024-02-02T15:42:05.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.202+0000 7fbb7a800700 1 -- 172.21.15.1:0/3314344402 learned_addr learned my addr 172.21.15.1:0/3314344402 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:42:05.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.202+0000 7fbb79e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe316ae9b0 0x55fe316bee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:05.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.203+0000 7fbb7b200700 1 -- 172.21.15.1:0/3314344402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe316ae9b0 msgr2=0x55fe316bee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:05.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.203+0000 7fbb7b200700 1 --2- 172.21.15.1:0/3314344402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe316ae9b0 0x55fe316bee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:05.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.203+0000 7fbb7b200700 1 -- 172.21.15.1:0/3314344402 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe316ae090 msgr2=0x55fe316be940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:05.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.203+0000 7fbb7b200700 1 --2- 172.21.15.1:0/3314344402 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe316ae090 0x55fe316be940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:05.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.203+0000 7fbb7b200700 1 -- 172.21.15.1:0/3314344402 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb74009090 con 0x55fe316b9860 2024-02-02T15:42:05.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.204+0000 7fbb7b200700 1 --2- 172.21.15.1:0/3314344402 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe316b9860 0x55fe316bf3c0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fbb74000f80 tx=0x7fbb74009480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8439c787c9d056d7 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:05.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.204+0000 7fbb63e00700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:42:05.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.205+0000 7fbb63e00700 1 -- 172.21.15.1:0/3314344402 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb74016eb0 con 0x55fe316b9860 2024-02-02T15:42:05.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.205+0000 7fbb63e00700 1 -- 172.21.15.1:0/3314344402 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fbb74004a70 con 0x55fe316b9860 2024-02-02T15:42:05.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.205+0000 7fbb94fe5b80 1 -- 172.21.15.1:0/3314344402 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbb7400f040 con 0x55fe316b9860 2024-02-02T15:42:05.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.205+0000 7fbb63e00700 1 -- 172.21.15.1:0/3314344402 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb74004c30 con 0x55fe316b9860 2024-02-02T15:42:05.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.205+0000 7fbb94fe5b80 10 client.19488 fetch_fsmap learned FSMap version 61 2024-02-02T15:42:05.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.205+0000 7fbb94fe5b80 1 -- 172.21.15.1:0/3314344402 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x55fe3142dbe0 con 0x55fe316b9860 2024-02-02T15:42:05.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.206+0000 7fbb63e00700 1 -- 172.21.15.1:0/3314344402 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbb7403a030 con 0x55fe316b9860 2024-02-02T15:42:05.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.206+0000 7fbb63e00700 1 -- 172.21.15.1:0/3314344402 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 61) v1 ==== 7550+0+0 (secure 0 0 0) 0x7fbb74012020 con 0x55fe316b9860 2024-02-02T15:42:05.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.208+0000 7fbb94fe5b80 10 client.19488 fetch_fsmap finished waiting for FSMap version 61 2024-02-02T15:42:05.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.208+0000 7fbb94fe5b80 10 client.19488 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:42:05.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.208+0000 7fbb94fe5b80 20 client.19488 populate_metadata read hostname 'smithi001' 2024-02-02T15:42:05.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.208+0000 7fbb94fe5b80 1 --2- 172.21.15.1:0/3314344402 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55fe316c9170 0x55fe3172fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:05.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.208+0000 7fbb94fe5b80 4 client.19488 mds_command: new command op to 24557 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-02T15:42:05.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.208+0000 7fbb94fe5b80 1 -- 172.21.15.1:0/3314344402 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55fe315befd0 con 0x55fe316c9170 2024-02-02T15:42:05.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.209+0000 7fbb7a800700 1 --2- 172.21.15.1:0/3314344402 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55fe316c9170 0x55fe3172fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:05.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.210+0000 7fbb7a800700 1 --2- 172.21.15.1:0/3314344402 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55fe316c9170 0x55fe3172fcb0 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da887422fee2ebd server_cookie=37ddc80c237af5b8 in_seq=0 out_seq=0 2024-02-02T15:42:05.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.210+0000 7fbb63e00700 10 client.19488 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:42:05.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.212+0000 7fbb63e00700 1 -- 172.21.15.1:0/3314344402 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55fe315befd0 con 0x55fe316c9170 2024-02-02T15:42:05.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.212+0000 7fbb63e00700 10 client.19488 handle_command_reply: tid=0 2024-02-02T15:42:05.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.212+0000 7fbb94fe5b80 1 client.19488 shutdown 2024-02-02T15:42:05.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.213+0000 7fbb94fe5b80 1 -- 172.21.15.1:0/3314344402 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe316b9860 msgr2=0x55fe316bf3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:05.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.213+0000 7fbb94fe5b80 1 --2- 172.21.15.1:0/3314344402 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe316b9860 0x55fe316bf3c0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fbb74000f80 tx=0x7fbb74009480 comp rx=0 tx=0).stop 2024-02-02T15:42:05.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.213+0000 7fbb94fe5b80 1 -- 172.21.15.1:0/3314344402 shutdown_connections 2024-02-02T15:42:05.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.213+0000 7fbb94fe5b80 1 --2- 172.21.15.1:0/3314344402 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55fe316c9170 0x55fe3172fcb0 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:05.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.213+0000 7fbb94fe5b80 1 --2- 172.21.15.1:0/3314344402 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55fe316b9860 0x55fe316bf3c0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:05.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.213+0000 7fbb94fe5b80 1 --2- 172.21.15.1:0/3314344402 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fe316ae090 0x55fe316be940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:05.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.213+0000 7fbb94fe5b80 1 --2- 172.21.15.1:0/3314344402 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55fe316ae9b0 0x55fe316bee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:05.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.213+0000 7fbb94fe5b80 1 -- 172.21.15.1:0/3314344402 >> 172.21.15.1:0/3314344402 conn(0x55fe3160de30 msgr2=0x55fe316b1820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:05.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.213+0000 7fbb63e00700 0 client.19488 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:42:05.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.214+0000 7fbb94fe5b80 1 -- 172.21.15.1:0/3314344402 shutdown_connections 2024-02-02T15:42:05.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.214+0000 7fbb94fe5b80 1 -- 172.21.15.1:0/3314344402 wait complete. 2024-02-02T15:42:05.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.215+0000 7fbb94fe5b80 20 client.19488 trim_cache size 0 max 16384 2024-02-02T15:42:05.215 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:42:05.215 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2024-02-02T15:42:05.215 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "56494e46-d137-4867-b269-27c4336fedf4", 2024-02-02T15:42:05.215 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2024-02-02T15:42:05.215 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:42:05.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.216+0000 7fbb81400700 1 -- 172.21.15.1:0/3751229234 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fbb5c0a0790 msgr2=0x7fbb5c0a2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:05.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.216+0000 7fbb81400700 1 --2- 172.21.15.1:0/3751229234 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fbb5c0a0790 0x7fbb5c0a2c50 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fbb7401f000 tx=0x7fbb740166b0 comp rx=0 tx=0).stop 2024-02-02T15:42:05.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.216+0000 7fbb81400700 1 -- 172.21.15.1:0/3751229234 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb7c0fff10 msgr2=0x7fbb7c10a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:05.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.216+0000 7fbb81400700 1 --2- 172.21.15.1:0/3751229234 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb7c0fff10 0x7fbb7c10a0c0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fbb6400d950 tx=0x7fbb6400dd10 comp rx=0 tx=0).stop 2024-02-02T15:42:05.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.217+0000 7fbb81400700 1 -- 172.21.15.1:0/3751229234 shutdown_connections 2024-02-02T15:42:05.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.217+0000 7fbb81400700 1 --2- 172.21.15.1:0/3751229234 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fbb5c0a0790 0x7fbb5c0a2c50 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:05.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.217+0000 7fbb81400700 1 --2- 172.21.15.1:0/3751229234 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbb7c107230 0x7fbb7c1076e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:05.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.217+0000 7fbb81400700 1 --2- 172.21.15.1:0/3751229234 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb7c0fff10 0x7fbb7c10a0c0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:05.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.217+0000 7fbb81400700 1 --2- 172.21.15.1:0/3751229234 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbb7c105030 0x7fbb7c10a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:05.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.217+0000 7fbb81400700 1 -- 172.21.15.1:0/3751229234 >> 172.21.15.1:0/3751229234 conn(0x7fbb7c076e20 msgr2=0x7fbb7c1038e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:05.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.217+0000 7fbb81400700 1 -- 172.21.15.1:0/3751229234 shutdown_connections 2024-02-02T15:42:05.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.218+0000 7fbb81400700 1 -- 172.21.15.1:0/3751229234 wait complete. 2024-02-02T15:42:05.255 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-02T15:42:05.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.638+0000 7fe3c2a00700 1 -- 172.21.15.1:0/3780577945 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3bc100ff0 msgr2=0x7fe3bc1013d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:05.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.638+0000 7fe3c2a00700 1 --2- 172.21.15.1:0/3780577945 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3bc100ff0 0x7fe3bc1013d0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac009990 tx=0x7fe3ac009ca0 comp rx=0 tx=0).stop 2024-02-02T15:42:05.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.638+0000 7fe3c2a00700 1 -- 172.21.15.1:0/3780577945 shutdown_connections 2024-02-02T15:42:05.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.638+0000 7fe3c2a00700 1 --2- 172.21.15.1:0/3780577945 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe3bc106110 0x7fe3bc10dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:05.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.638+0000 7fe3c2a00700 1 --2- 172.21.15.1:0/3780577945 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3bc100ff0 0x7fe3bc1013d0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:05.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.638+0000 7fe3c2a00700 1 --2- 172.21.15.1:0/3780577945 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3bc1019a0 0x7fe3bc1059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:05.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.638+0000 7fe3c2a00700 1 -- 172.21.15.1:0/3780577945 >> 172.21.15.1:0/3780577945 conn(0x7fe3bc0fc810 msgr2=0x7fe3bc0fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:05.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.638+0000 7fe3c2a00700 1 -- 172.21.15.1:0/3780577945 shutdown_connections 2024-02-02T15:42:05.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.639+0000 7fe3c2a00700 1 -- 172.21.15.1:0/3780577945 wait complete. 2024-02-02T15:42:05.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.639+0000 7fe3c2a00700 1 Processor -- start 2024-02-02T15:42:05.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.640+0000 7fe3c2a00700 1 -- start start 2024-02-02T15:42:05.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.640+0000 7fe3c2a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3bc100ff0 0x7fe3bc196eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:05.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.640+0000 7fe3c2a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe3bc106110 0x7fe3bc1973f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:05.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.640+0000 7fe3c2a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3bc19b850 0x7fe3bc19bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:05.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.640+0000 7fe3c2a00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3bc19c2d0 con 0x7fe3bc106110 2024-02-02T15:42:05.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.640+0000 7fe3c2a00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3bc19c440 con 0x7fe3bc100ff0 2024-02-02T15:42:05.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.640+0000 7fe3c2a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3bc19c600 con 0x7fe3bc19b850 2024-02-02T15:42:05.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.641+0000 7fe3bbe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3bc100ff0 0x7fe3bc196eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:05.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.641+0000 7fe3c0a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3bc19b850 0x7fe3bc19bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:05.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.641+0000 7fe3bb400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe3bc106110 0x7fe3bc1973f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:05.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.641+0000 7fe3bbe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3bc100ff0 0x7fe3bc196eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59442/0 (socket says 172.21.15.1:59442) 2024-02-02T15:42:05.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.641+0000 7fe3c0a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3bc19b850 0x7fe3bc19bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:42436/0 (socket says 172.21.15.1:42436) 2024-02-02T15:42:05.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.641+0000 7fe3bbe00700 1 -- 172.21.15.1:0/1892108713 learned_addr learned my addr 172.21.15.1:0/1892108713 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:42:05.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.641+0000 7fe3bbe00700 1 -- 172.21.15.1:0/1892108713 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3bc19b850 msgr2=0x7fe3bc19bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:05.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.641+0000 7fe3bbe00700 1 --2- 172.21.15.1:0/1892108713 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3bc19b850 0x7fe3bc19bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:05.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.641+0000 7fe3bbe00700 1 -- 172.21.15.1:0/1892108713 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe3bc106110 msgr2=0x7fe3bc1973f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:05.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.641+0000 7fe3bbe00700 1 --2- 172.21.15.1:0/1892108713 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe3bc106110 0x7fe3bc1973f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:05.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.641+0000 7fe3bbe00700 1 -- 172.21.15.1:0/1892108713 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3ac0095f0 con 0x7fe3bc100ff0 2024-02-02T15:42:05.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.641+0000 7fe3bbe00700 1 --2- 172.21.15.1:0/1892108713 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3bc100ff0 0x7fe3bc196eb0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac005910 tx=0x7fe3ac0047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:05.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.642+0000 7fe3b8c00700 1 -- 172.21.15.1:0/1892108713 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac00f660 con 0x7fe3bc100ff0 2024-02-02T15:42:05.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.642+0000 7fe3c2a00700 1 -- 172.21.15.1:0/1892108713 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3bc072f70 con 0x7fe3bc100ff0 2024-02-02T15:42:05.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.642+0000 7fe3b8c00700 1 -- 172.21.15.1:0/1892108713 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe3ac00bea0 con 0x7fe3bc100ff0 2024-02-02T15:42:05.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.642+0000 7fe3b8c00700 1 -- 172.21.15.1:0/1892108713 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac02caf0 con 0x7fe3bc100ff0 2024-02-02T15:42:05.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.642+0000 7fe3c2a00700 1 -- 172.21.15.1:0/1892108713 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3bc073410 con 0x7fe3bc100ff0 2024-02-02T15:42:05.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.645+0000 7fe3b8c00700 1 -- 172.21.15.1:0/1892108713 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7fe3ac00f7c0 con 0x7fe3bc100ff0 2024-02-02T15:42:05.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.646+0000 7fe3d6442b80 1 Processor -- start 2024-02-02T15:42:05.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.646+0000 7fe3b8c00700 1 --2- 172.21.15.1:0/1892108713 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fe39c0a0710 0x7fe39c0a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:05.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.646+0000 7fe3b8c00700 1 -- 172.21.15.1:0/1892108713 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fe3ac0d5f90 con 0x7fe3bc100ff0 2024-02-02T15:42:05.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.646+0000 7fe3d6442b80 1 -- start start 2024-02-02T15:42:05.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.647+0000 7fe3bb400700 1 --2- 172.21.15.1:0/1892108713 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fe39c0a0710 0x7fe39c0a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:05.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.647+0000 7fe3d6442b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5571c33e1010 0x5571c33e13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:05.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.647+0000 7fe3d6442b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5571c33e1930 0x5571c33ec2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:05.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.647+0000 7fe3d6442b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5571c33ec7e0 0x5571c33eebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:05.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.647+0000 7fe3d6442b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571c33554c0 con 0x5571c33e1010 2024-02-02T15:42:05.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.647+0000 7fe3d6442b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571c3339aa0 con 0x5571c33e1930 2024-02-02T15:42:05.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.647+0000 7fe3d6442b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571c3337460 con 0x5571c33ec7e0 2024-02-02T15:42:05.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.647+0000 7fe3c0a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5571c33ec7e0 0x5571c33eebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:05.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.647+0000 7fe3c0a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5571c33ec7e0 0x5571c33eebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:42438/0 (socket says 172.21.15.1:42438) 2024-02-02T15:42:05.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.647+0000 7fe3c0a00700 1 -- 172.21.15.1:0/825413412 learned_addr learned my addr 172.21.15.1:0/825413412 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:42:05.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.647+0000 7fe3bb400700 1 --2- 172.21.15.1:0/825413412 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5571c33e1930 0x5571c33ec2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:05.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.647+0000 7fe3bbe00700 1 --2- 172.21.15.1:0/825413412 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5571c33e1010 0x5571c33e13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:05.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.647+0000 7fe3bb400700 1 --2- 172.21.15.1:0/1892108713 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fe39c0a0710 0x7fe39c0a2bd0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc1984c0 tx=0x7fe3a4009260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:05.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.647+0000 7fe3c0a00700 1 -- 172.21.15.1:0/825413412 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5571c33e1930 msgr2=0x5571c33ec2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:05.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.647+0000 7fe3c0a00700 1 --2- 172.21.15.1:0/825413412 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5571c33e1930 0x5571c33ec2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:05.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.648+0000 7fe3c0a00700 1 -- 172.21.15.1:0/825413412 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5571c33e1010 msgr2=0x5571c33e13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:05.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.648+0000 7fe3c0a00700 1 --2- 172.21.15.1:0/825413412 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5571c33e1010 0x5571c33e13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:05.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.648+0000 7fe3c0a00700 1 -- 172.21.15.1:0/825413412 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5571c31b43b0 con 0x5571c33ec7e0 2024-02-02T15:42:05.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.648+0000 7fe3bbe00700 1 --2- 172.21.15.1:0/825413412 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5571c33e1010 0x5571c33e13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:42:05.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.648+0000 7fe3c0a00700 1 --2- 172.21.15.1:0/825413412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5571c33ec7e0 0x5571c33eebd0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fe3b40116b0 tx=0x7fe3b4009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbf895e50a925132 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:05.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.648+0000 7fe3b1200700 1 -- 172.21.15.1:0/825413412 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b400e040 con 0x5571c33ec7e0 2024-02-02T15:42:05.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.648+0000 7fe3b1200700 1 -- 172.21.15.1:0/825413412 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe3b400f040 con 0x5571c33ec7e0 2024-02-02T15:42:05.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.648+0000 7fe3b1200700 1 -- 172.21.15.1:0/825413412 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b40048d0 con 0x5571c33ec7e0 2024-02-02T15:42:05.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.649+0000 7fe3d6442b80 1 -- 172.21.15.1:0/825413412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5571c33ec7e0 msgr2=0x5571c33eebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:05.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.649+0000 7fe3d6442b80 1 --2- 172.21.15.1:0/825413412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5571c33ec7e0 0x5571c33eebd0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fe3b40116b0 tx=0x7fe3b4009920 comp rx=0 tx=0).stop 2024-02-02T15:42:05.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.649+0000 7fe3d6442b80 1 -- 172.21.15.1:0/825413412 shutdown_connections 2024-02-02T15:42:05.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.649+0000 7fe3d6442b80 1 --2- 172.21.15.1:0/825413412 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5571c33ec7e0 0x5571c33eebd0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:05.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.649+0000 7fe3d6442b80 1 --2- 172.21.15.1:0/825413412 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5571c33e1010 0x5571c33e13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:05.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.649+0000 7fe3d6442b80 1 --2- 172.21.15.1:0/825413412 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5571c33e1930 0x5571c33ec2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:05.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.649+0000 7fe3d6442b80 1 -- 172.21.15.1:0/825413412 >> 172.21.15.1:0/825413412 conn(0x5571c3340db0 msgr2=0x5571c33dd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:05.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.649+0000 7fe3d6442b80 1 -- 172.21.15.1:0/825413412 shutdown_connections 2024-02-02T15:42:05.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.649+0000 7fe3d6442b80 1 -- 172.21.15.1:0/825413412 wait complete. 2024-02-02T15:42:05.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.650+0000 7fe3d6442b80 1 -- start start 2024-02-02T15:42:05.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.650+0000 7fe3d6442b80 1 Processor -- start 2024-02-02T15:42:05.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.650+0000 7fe3d6442b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5571c33e1010 0x5571c33ec160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:05.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.650+0000 7fe3d6442b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5571c33e1930 0x5571c33e9370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:05.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.650+0000 7fe3d6442b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5571c33ec7e0 0x5571c33e9910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:05.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.650+0000 7fe3d6442b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571c3332d50 con 0x5571c33e1010 2024-02-02T15:42:05.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.650+0000 7fe3d6442b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571c32ced80 con 0x5571c33e1930 2024-02-02T15:42:05.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.650+0000 7fe3d6442b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571c32cea20 con 0x5571c33ec7e0 2024-02-02T15:42:05.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.650+0000 7fe3d6442b80 10 client.0 _ll_register_callbacks cb 0x5571c32d0290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:42:05.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.651+0000 7fe3bbe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5571c33e1010 0x5571c33ec160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:05.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.651+0000 7fe3c0a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5571c33ec7e0 0x5571c33e9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:05.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.651+0000 7fe3c0a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5571c33ec7e0 0x5571c33e9910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:42440/0 (socket says 172.21.15.1:42440) 2024-02-02T15:42:05.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.651+0000 7fe3bb400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5571c33e1930 0x5571c33e9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:05.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.651+0000 7fe3c0a00700 1 -- 172.21.15.1:0/1684232499 learned_addr learned my addr 172.21.15.1:0/1684232499 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:42:05.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.651+0000 7fe3c0a00700 1 -- 172.21.15.1:0/1684232499 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5571c33e1930 msgr2=0x5571c33e9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:05.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.651+0000 7fe3c0a00700 1 --2- 172.21.15.1:0/1684232499 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5571c33e1930 0x5571c33e9370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:05.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.651+0000 7fe3c0a00700 1 -- 172.21.15.1:0/1684232499 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5571c33e1010 msgr2=0x5571c33ec160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:05.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.651+0000 7fe3c0a00700 1 --2- 172.21.15.1:0/1684232499 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5571c33e1010 0x5571c33ec160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:05.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.651+0000 7fe3c0a00700 1 -- 172.21.15.1:0/1684232499 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3b4011330 con 0x5571c33ec7e0 2024-02-02T15:42:05.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.651+0000 7fe3c0a00700 1 --2- 172.21.15.1:0/1684232499 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5571c33ec7e0 0x5571c33e9910 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fe3b4009e40 tx=0x7fe3b4003cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b27bf9be08a52f31 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:05.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.651+0000 7fe3a9600700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:42:05.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.652+0000 7fe3a9600700 1 -- 172.21.15.1:0/1684232499 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b400e030 con 0x5571c33ec7e0 2024-02-02T15:42:05.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.652+0000 7fe3d6442b80 1 -- 172.21.15.1:0/1684232499 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5571c33d50f0 con 0x5571c33ec7e0 2024-02-02T15:42:05.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.652+0000 7fe3a9600700 1 -- 172.21.15.1:0/1684232499 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fe3b40093a0 con 0x5571c33ec7e0 2024-02-02T15:42:05.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.652+0000 7fe3a9600700 1 -- 172.21.15.1:0/1684232499 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe3b4004210 con 0x5571c33ec7e0 2024-02-02T15:42:05.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.652+0000 7fe3d6442b80 10 client.29611 fetch_fsmap learned FSMap version 61 2024-02-02T15:42:05.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.652+0000 7fe3d6442b80 1 -- 172.21.15.1:0/1684232499 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x5571c3160b60 con 0x5571c33ec7e0 2024-02-02T15:42:05.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.652+0000 7fe3a9600700 1 -- 172.21.15.1:0/1684232499 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5571c33d50f0 con 0x5571c33ec7e0 2024-02-02T15:42:05.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.652+0000 7fe3a9600700 1 -- 172.21.15.1:0/1684232499 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 61) v1 ==== 7550+0+0 (secure 0 0 0) 0x7fe3b4007c30 con 0x5571c33ec7e0 2024-02-02T15:42:05.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.652+0000 7fe3d6442b80 10 client.29611 fetch_fsmap finished waiting for FSMap version 61 2024-02-02T15:42:05.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.652+0000 7fe3d6442b80 10 client.29611 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:42:05.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.652+0000 7fe3d6442b80 20 client.29611 populate_metadata read hostname 'smithi001' 2024-02-02T15:42:05.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.652+0000 7fe3d6442b80 1 --2- 172.21.15.1:0/1684232499 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5571c33fc200 0x5571c33fe620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:05.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.652+0000 7fe3d6442b80 4 client.29611 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:42:05.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.652+0000 7fe3d6442b80 1 -- 172.21.15.1:0/1684232499 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5571c332bdd0 con 0x5571c33fc200 2024-02-02T15:42:05.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.653+0000 7fe3bbe00700 1 --2- 172.21.15.1:0/1684232499 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5571c33fc200 0x5571c33fe620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:05.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.653+0000 7fe3bbe00700 1 --2- 172.21.15.1:0/1684232499 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5571c33fc200 0x5571c33fe620 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f46de6f964fa3d7b server_cookie=d944a5f9eeca579e in_seq=0 out_seq=0 2024-02-02T15:42:05.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.653+0000 7fe3a9600700 10 client.29611 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:42:05.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.654+0000 7fe3a9600700 1 -- 172.21.15.1:0/1684232499 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5571c332bdd0 con 0x5571c33fc200 2024-02-02T15:42:05.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.654+0000 7fe3a9600700 10 client.29611 handle_command_reply: tid=0 2024-02-02T15:42:05.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.654+0000 7fe3d6442b80 1 client.29611 shutdown 2024-02-02T15:42:05.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.654+0000 7fe3d6442b80 1 -- 172.21.15.1:0/1684232499 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5571c33ec7e0 msgr2=0x5571c33e9910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:05.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.654+0000 7fe3d6442b80 1 --2- 172.21.15.1:0/1684232499 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5571c33ec7e0 0x5571c33e9910 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fe3b4009e40 tx=0x7fe3b4003cb0 comp rx=0 tx=0).stop 2024-02-02T15:42:05.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.654+0000 7fe3d6442b80 1 -- 172.21.15.1:0/1684232499 shutdown_connections 2024-02-02T15:42:05.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.654+0000 7fe3d6442b80 1 --2- 172.21.15.1:0/1684232499 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5571c33fc200 0x5571c33fe620 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:05.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.654+0000 7fe3d6442b80 1 --2- 172.21.15.1:0/1684232499 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5571c33ec7e0 0x5571c33e9910 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:05.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.654+0000 7fe3d6442b80 1 --2- 172.21.15.1:0/1684232499 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5571c33e1010 0x5571c33ec160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:05.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.654+0000 7fe3d6442b80 1 --2- 172.21.15.1:0/1684232499 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5571c33e1930 0x5571c33e9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:05.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.654+0000 7fe3a9600700 0 client.29611 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:42:05.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.654+0000 7fe3d6442b80 1 -- 172.21.15.1:0/1684232499 >> 172.21.15.1:0/1684232499 conn(0x5571c3340db0 msgr2=0x5571c33dcfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:05.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.654+0000 7fe3d6442b80 1 -- 172.21.15.1:0/1684232499 shutdown_connections 2024-02-02T15:42:05.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.654+0000 7fe3d6442b80 1 -- 172.21.15.1:0/1684232499 wait complete. 2024-02-02T15:42:05.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.655+0000 7fe3d6442b80 20 client.29611 trim_cache size 0 max 16384 2024-02-02T15:42:05.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.672+0000 7fe3d6442b80 1 Processor -- start 2024-02-02T15:42:05.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.672+0000 7fe3d6442b80 1 -- start start 2024-02-02T15:42:05.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.672+0000 7fe3d6442b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5571c33e1010 0x5571c33f84e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:05.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.672+0000 7fe3d6442b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5571c33e1930 0x5571c33ebc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:05.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.672+0000 7fe3d6442b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5571c33ec7e0 0x5571c33edbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:05.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.672+0000 7fe3d6442b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571c31b66a0 con 0x5571c33ec7e0 2024-02-02T15:42:05.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.672+0000 7fe3d6442b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571c33d7500 con 0x5571c33e1010 2024-02-02T15:42:05.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.672+0000 7fe3d6442b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571c33e78d0 con 0x5571c33e1930 2024-02-02T15:42:05.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.672+0000 7fe3c0a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5571c33ec7e0 0x5571c33edbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:05.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.672+0000 7fe3bbe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5571c33e1010 0x5571c33f84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:05.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.672+0000 7fe3bbe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5571c33e1010 0x5571c33f84e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59462/0 (socket says 172.21.15.1:59462) 2024-02-02T15:42:05.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.672+0000 7fe3c0a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5571c33ec7e0 0x5571c33edbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:50020/0 (socket says 172.21.15.1:50020) 2024-02-02T15:42:05.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.672+0000 7fe3bbe00700 1 -- 172.21.15.1:0/1699572279 learned_addr learned my addr 172.21.15.1:0/1699572279 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:42:05.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.672+0000 7fe3bb400700 1 --2- 172.21.15.1:0/1699572279 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5571c33e1930 0x5571c33ebc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:05.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.673+0000 7fe3bbe00700 1 -- 172.21.15.1:0/1699572279 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5571c33e1930 msgr2=0x5571c33ebc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:05.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.673+0000 7fe3bbe00700 1 --2- 172.21.15.1:0/1699572279 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5571c33e1930 0x5571c33ebc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:05.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.673+0000 7fe3bbe00700 1 -- 172.21.15.1:0/1699572279 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5571c33ec7e0 msgr2=0x5571c33edbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:05.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.673+0000 7fe3bbe00700 1 --2- 172.21.15.1:0/1699572279 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5571c33ec7e0 0x5571c33edbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:05.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.673+0000 7fe3bbe00700 1 -- 172.21.15.1:0/1699572279 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5571c32f8ea0 con 0x5571c33e1010 2024-02-02T15:42:05.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.673+0000 7fe3bbe00700 1 --2- 172.21.15.1:0/1699572279 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5571c33e1010 0x5571c33f84e0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac0a2e40 tx=0x7fe3ac0a9040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9010beabb66b3f64 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:05.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.673+0000 7fe3b1c00700 1 -- 172.21.15.1:0/1699572279 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac0a6f70 con 0x5571c33e1010 2024-02-02T15:42:05.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.673+0000 7fe3b1c00700 1 -- 172.21.15.1:0/1699572279 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe3ac0ab120 con 0x5571c33e1010 2024-02-02T15:42:05.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.673+0000 7fe3b1c00700 1 -- 172.21.15.1:0/1699572279 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac0a33a0 con 0x5571c33e1010 2024-02-02T15:42:05.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.674+0000 7fe3d6442b80 1 -- 172.21.15.1:0/1699572279 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5571c33e1010 msgr2=0x5571c33f84e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:05.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.674+0000 7fe3d6442b80 1 --2- 172.21.15.1:0/1699572279 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5571c33e1010 0x5571c33f84e0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac0a2e40 tx=0x7fe3ac0a9040 comp rx=0 tx=0).stop 2024-02-02T15:42:05.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.674+0000 7fe3d6442b80 1 -- 172.21.15.1:0/1699572279 shutdown_connections 2024-02-02T15:42:05.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.674+0000 7fe3d6442b80 1 --2- 172.21.15.1:0/1699572279 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5571c33ec7e0 0x5571c33edbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:05.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.674+0000 7fe3d6442b80 1 --2- 172.21.15.1:0/1699572279 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5571c33e1010 0x5571c33f84e0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:05.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.674+0000 7fe3d6442b80 1 --2- 172.21.15.1:0/1699572279 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5571c33e1930 0x5571c33ebc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:05.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.674+0000 7fe3d6442b80 1 -- 172.21.15.1:0/1699572279 >> 172.21.15.1:0/1699572279 conn(0x5571c3340db0 msgr2=0x5571c33dbb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:05.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.674+0000 7fe3d6442b80 1 -- 172.21.15.1:0/1699572279 shutdown_connections 2024-02-02T15:42:05.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.674+0000 7fe3d6442b80 1 -- 172.21.15.1:0/1699572279 wait complete. 2024-02-02T15:42:05.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.674+0000 7fe3d6442b80 1 -- start start 2024-02-02T15:42:05.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.674+0000 7fe3d6442b80 1 Processor -- start 2024-02-02T15:42:05.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.675+0000 7fe3d6442b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5571c33e1010 0x5571c33f2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:05.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.675+0000 7fe3d6442b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5571c33e1930 0x5571c33f33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:05.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.675+0000 7fe3d6442b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5571c33ec7e0 0x5571c33f1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:05.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.675+0000 7fe3d6442b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571c33f1c60 con 0x5571c33e1010 2024-02-02T15:42:05.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.675+0000 7fe3d6442b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571c33f1dd0 con 0x5571c33e1930 2024-02-02T15:42:05.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.675+0000 7fe3d6442b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571c33f1f90 con 0x5571c33ec7e0 2024-02-02T15:42:05.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.675+0000 7fe3d6442b80 10 client.0 _ll_register_callbacks cb 0x5571c32d0290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:42:05.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.675+0000 7fe3bb400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5571c33e1930 0x5571c33f33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:05.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.675+0000 7fe3bb400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5571c33e1930 0x5571c33f33d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:59466/0 (socket says 172.21.15.1:59466) 2024-02-02T15:42:05.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.675+0000 7fe3bb400700 1 -- 172.21.15.1:0/1957407046 learned_addr learned my addr 172.21.15.1:0/1957407046 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:42:05.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.675+0000 7fe3bbe00700 1 --2- 172.21.15.1:0/1957407046 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5571c33e1010 0x5571c33f2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:05.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.675+0000 7fe3c0a00700 1 --2- 172.21.15.1:0/1957407046 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5571c33ec7e0 0x5571c33f1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:05.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.676+0000 7fe3bbe00700 1 -- 172.21.15.1:0/1957407046 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5571c33ec7e0 msgr2=0x5571c33f1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:05.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.676+0000 7fe3bbe00700 1 --2- 172.21.15.1:0/1957407046 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5571c33ec7e0 0x5571c33f1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:05.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.676+0000 7fe3bbe00700 1 -- 172.21.15.1:0/1957407046 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5571c33e1930 msgr2=0x5571c33f33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:05.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.676+0000 7fe3bbe00700 1 --2- 172.21.15.1:0/1957407046 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5571c33e1930 0x5571c33f33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:05.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.676+0000 7fe3bbe00700 1 -- 172.21.15.1:0/1957407046 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3bc073410 con 0x5571c33e1010 2024-02-02T15:42:05.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.676+0000 7fe3bb400700 1 --2- 172.21.15.1:0/1957407046 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5571c33e1930 0x5571c33f33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:42:05.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.676+0000 7fe3c0a00700 1 --2- 172.21.15.1:0/1957407046 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5571c33ec7e0 0x5571c33f1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:42:05.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.676+0000 7fe3bbe00700 1 --2- 172.21.15.1:0/1957407046 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5571c33e1010 0x5571c33f2e90 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac006010 tx=0x7fe3ac02c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb1b5f6eb7c28244 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:05.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.676+0000 7fe3a9600700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:42:05.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.676+0000 7fe3a9600700 1 -- 172.21.15.1:0/1957407046 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac0a14e0 con 0x5571c33e1010 2024-02-02T15:42:05.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.676+0000 7fe3a9600700 1 -- 172.21.15.1:0/1957407046 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(105..105 src has 1..105) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fe3ac0a2630 con 0x5571c33e1010 2024-02-02T15:42:05.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.676+0000 7fe3d6442b80 1 -- 172.21.15.1:0/1957407046 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe3b400f040 con 0x5571c33e1010 2024-02-02T15:42:05.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.676+0000 7fe3a9600700 1 -- 172.21.15.1:0/1957407046 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe3ac0a2830 con 0x5571c33e1010 2024-02-02T15:42:05.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.676+0000 7fe3d6442b80 10 client.19506 fetch_fsmap learned FSMap version 61 2024-02-02T15:42:05.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.676+0000 7fe3d6442b80 1 -- 172.21.15.1:0/1957407046 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) v3 -- 0x5571c31b8cd0 con 0x5571c33e1010 2024-02-02T15:42:05.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.677+0000 7fe3a9600700 1 -- 172.21.15.1:0/1957407046 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe3b400f040 con 0x5571c33e1010 2024-02-02T15:42:05.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.677+0000 7fe3a9600700 1 -- 172.21.15.1:0/1957407046 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 61) v1 ==== 7550+0+0 (secure 0 0 0) 0x7fe3ac0b6b70 con 0x5571c33e1010 2024-02-02T15:42:05.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.677+0000 7fe3d6442b80 10 client.19506 fetch_fsmap finished waiting for FSMap version 61 2024-02-02T15:42:05.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.677+0000 7fe3d6442b80 10 client.19506 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:42:05.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.677+0000 7fe3d6442b80 20 client.19506 populate_metadata read hostname 'smithi001' 2024-02-02T15:42:05.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.677+0000 7fe3d6442b80 1 --2- 172.21.15.1:0/1957407046 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5571c344ed40 0x5571c3451160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:05.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.677+0000 7fe3d6442b80 4 client.19506 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2024-02-02T15:42:05.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.677+0000 7fe3d6442b80 1 -- 172.21.15.1:0/1957407046 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5571c32f1f50 con 0x5571c344ed40 2024-02-02T15:42:05.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.677+0000 7fe3bb400700 1 --2- 172.21.15.1:0/1957407046 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5571c344ed40 0x5571c3451160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:05.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.678+0000 7fe3bb400700 1 --2- 172.21.15.1:0/1957407046 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5571c344ed40 0x5571c3451160 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dbeb74908eb91557 server_cookie=f82c4b29e74808d7 in_seq=0 out_seq=0 2024-02-02T15:42:05.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.678+0000 7fe3a9600700 10 client.19506 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:42:05.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.678+0000 7fe3a9600700 1 -- 172.21.15.1:0/1957407046 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5571c32f1f50 con 0x5571c344ed40 2024-02-02T15:42:05.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.678+0000 7fe3a9600700 10 client.19506 handle_command_reply: tid=0 2024-02-02T15:42:05.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.678+0000 7fe3d6442b80 1 client.19506 shutdown 2024-02-02T15:42:05.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.678+0000 7fe3d6442b80 1 -- 172.21.15.1:0/1957407046 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5571c33e1010 msgr2=0x5571c33f2e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:05.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.678+0000 7fe3d6442b80 1 --2- 172.21.15.1:0/1957407046 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5571c33e1010 0x5571c33f2e90 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac006010 tx=0x7fe3ac02c3f0 comp rx=0 tx=0).stop 2024-02-02T15:42:05.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.678+0000 7fe3d6442b80 1 -- 172.21.15.1:0/1957407046 shutdown_connections 2024-02-02T15:42:05.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.678+0000 7fe3d6442b80 1 --2- 172.21.15.1:0/1957407046 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5571c344ed40 0x5571c3451160 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:05.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.678+0000 7fe3d6442b80 1 --2- 172.21.15.1:0/1957407046 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5571c33ec7e0 0x5571c33f1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:05.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.678+0000 7fe3d6442b80 1 --2- 172.21.15.1:0/1957407046 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5571c33e1010 0x5571c33f2e90 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:05.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.678+0000 7fe3d6442b80 1 --2- 172.21.15.1:0/1957407046 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5571c33e1930 0x5571c33f33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:05.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.678+0000 7fe3d6442b80 1 -- 172.21.15.1:0/1957407046 >> 172.21.15.1:0/1957407046 conn(0x5571c3340db0 msgr2=0x5571c33f7940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:05.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.678+0000 7fe3a9600700 0 client.19506 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:42:05.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.678+0000 7fe3d6442b80 1 -- 172.21.15.1:0/1957407046 shutdown_connections 2024-02-02T15:42:05.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.678+0000 7fe3d6442b80 1 -- 172.21.15.1:0/1957407046 wait complete. 2024-02-02T15:42:05.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.678+0000 7fe3d6442b80 20 client.19506 trim_cache size 0 max 16384 2024-02-02T15:42:05.684 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:42:05.684 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-02T15:42:05.684 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2024-02-02T15:42:05.684 INFO:teuthology.orchestra.run.smithi001.stdout: "56494e46-d137-4867-b269-27c4336fedf4": { 2024-02-02T15:42:05.684 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2024-02-02T15:42:05.684 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "56494e46-d137-4867-b269-27c4336fedf4", 2024-02-02T15:42:05.684 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,force" 2024-02-02T15:42:05.684 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:42:05.685 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:42:05.685 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:42:05.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.679+0000 7fe3c2a00700 1 -- 172.21.15.1:0/1892108713 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fe39c0a0710 msgr2=0x7fe39c0a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:05.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.679+0000 7fe3c2a00700 1 --2- 172.21.15.1:0/1892108713 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fe39c0a0710 0x7fe39c0a2bd0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc1984c0 tx=0x7fe3a4009260 comp rx=0 tx=0).stop 2024-02-02T15:42:05.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.679+0000 7fe3c2a00700 1 -- 172.21.15.1:0/1892108713 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3bc100ff0 msgr2=0x7fe3bc196eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:05.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.679+0000 7fe3c2a00700 1 --2- 172.21.15.1:0/1892108713 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3bc100ff0 0x7fe3bc196eb0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac005910 tx=0x7fe3ac0047b0 comp rx=0 tx=0).stop 2024-02-02T15:42:05.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.679+0000 7fe3c2a00700 1 -- 172.21.15.1:0/1892108713 shutdown_connections 2024-02-02T15:42:05.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.679+0000 7fe3c2a00700 1 --2- 172.21.15.1:0/1892108713 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fe39c0a0710 0x7fe39c0a2bd0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:05.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.679+0000 7fe3c2a00700 1 --2- 172.21.15.1:0/1892108713 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe3bc19b850 0x7fe3bc19bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:05.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.679+0000 7fe3c2a00700 1 --2- 172.21.15.1:0/1892108713 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe3bc100ff0 0x7fe3bc196eb0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:05.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.679+0000 7fe3c2a00700 1 --2- 172.21.15.1:0/1892108713 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe3bc106110 0x7fe3bc1973f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:05.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.679+0000 7fe3c2a00700 1 -- 172.21.15.1:0/1892108713 >> 172.21.15.1:0/1892108713 conn(0x7fe3bc0fc810 msgr2=0x7fe3bc1049c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:05.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.679+0000 7fe3c2a00700 1 -- 172.21.15.1:0/1892108713 shutdown_connections 2024-02-02T15:42:05.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:05.679+0000 7fe3c2a00700 1 -- 172.21.15.1:0/1892108713 wait complete. 2024-02-02T15:42:05.697 INFO:tasks.cephfs.filesystem:scrub status for tag:56494e46-d137-4867-b269-27c4336fedf4 - {'path': '/', 'tag': '56494e46-d137-4867-b269-27c4336fedf4', 'options': 'recursive,force'} 2024-02-02T15:42:05.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:42:05 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:42:05.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:42:05 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:42:05.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:42:05 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:42:05.979 INFO:tasks.workunit.client.0.smithi001.stdout: 10 55356 32.21 MB/sec warmup 98 sec latency 405.262 ms 2024-02-02T15:42:06.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:42:06 smithi071 ceph-mon[44707]: scrub queued for path: / 2024-02-02T15:42:06.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:42:06 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:42:06.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:42:06 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:42:06.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:42:06 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:42:06.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:42:06 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:42:06.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:42:06 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:42:06.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:42:06 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:42:06.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:42:06 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:42:06.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:42:06 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:42:06.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:42:06 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:42:06.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:42:06 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:42:06.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:42:06 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:42:06.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:42:06 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:42:06.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:42:06 smithi071 ceph-mon[44707]: pgmap v472: 129 pgs: 129 active+clean; 1.2 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 10 MiB/s wr, 567 op/s 2024-02-02T15:42:06.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:42:06 smithi142 ceph-mon[45728]: scrub queued for path: / 2024-02-02T15:42:06.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:42:06 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:42:06.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:42:06 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:42:06.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:42:06 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:42:06.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:42:06 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:42:06.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:42:06 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:42:06.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:42:06 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:42:06.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:42:06 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:42:06.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:42:06 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:42:06.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:42:06 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:42:06.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:42:06 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:42:06.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:42:06 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:42:06.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:42:06 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:42:06.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:42:06 smithi142 ceph-mon[45728]: pgmap v472: 129 pgs: 129 active+clean; 1.2 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 10 MiB/s wr, 567 op/s 2024-02-02T15:42:06.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:42:06 smithi001 ceph-mon[35470]: scrub queued for path: / 2024-02-02T15:42:06.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:42:06 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:42:06.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:42:06 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:42:06.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:42:06 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:42:06.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:42:06 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:42:06.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:42:06 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:42:06.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:42:06 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:42:06.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:42:06 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:42:06.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:42:06 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:42:06.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:42:06 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:42:06.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:42:06 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:42:06.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:42:06 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:42:06.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:42:06 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:42:06.942 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:42:06 smithi001 ceph-mon[35470]: pgmap v472: 129 pgs: 129 active+clean; 1.2 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 10 MiB/s wr, 567 op/s 2024-02-02T15:42:06.978 INFO:tasks.workunit.client.0.smithi001.stdout: 10 55615 31.98 MB/sec warmup 99 sec latency 366.665 ms 2024-02-02T15:42:07.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:42:07 smithi071 ceph-mon[44707]: osdmap e106: 12 total, 12 up, 12 in 2024-02-02T15:42:07.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:42:07 smithi142 ceph-mon[45728]: osdmap e106: 12 total, 12 up, 12 in 2024-02-02T15:42:07.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:42:07 smithi001 ceph-mon[35470]: osdmap e106: 12 total, 12 up, 12 in 2024-02-02T15:42:07.978 INFO:tasks.workunit.client.0.smithi001.stdout: 10 56251 31.99 MB/sec warmup 100 sec latency 109.582 ms 2024-02-02T15:42:08.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:42:08 smithi142 ceph-mon[45728]: pgmap v474: 129 pgs: 3 active+clean+snaptrim, 114 active+clean, 12 active+clean+snaptrim_wait; 1.2 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 14 MiB/s wr, 569 op/s 2024-02-02T15:42:08.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:42:08 smithi001 ceph-mon[35470]: pgmap v474: 129 pgs: 3 active+clean+snaptrim, 114 active+clean, 12 active+clean+snaptrim_wait; 1.2 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 14 MiB/s wr, 569 op/s 2024-02-02T15:42:08.979 INFO:tasks.workunit.client.0.smithi001.stdout: 10 57161 32.19 MB/sec warmup 101 sec latency 35.119 ms 2024-02-02T15:42:09.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:42:08 smithi071 ceph-mon[44707]: pgmap v474: 129 pgs: 3 active+clean+snaptrim, 114 active+clean, 12 active+clean+snaptrim_wait; 1.2 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 14 MiB/s wr, 569 op/s 2024-02-02T15:42:09.979 INFO:tasks.workunit.client.0.smithi001.stdout: 10 57438 32.05 MB/sec warmup 102 sec latency 546.581 ms 2024-02-02T15:42:10.979 INFO:tasks.workunit.client.0.smithi001.stdout: 10 57673 31.89 MB/sec warmup 103 sec latency 855.232 ms 2024-02-02T15:42:11.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:42:11 smithi071 ceph-mon[44707]: pgmap v475: 129 pgs: 3 active+clean+snaptrim, 114 active+clean, 12 active+clean+snaptrim_wait; 1.2 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 14 MiB/s wr, 569 op/s 2024-02-02T15:42:11.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:42:11 smithi142 ceph-mon[45728]: pgmap v475: 129 pgs: 3 active+clean+snaptrim, 114 active+clean, 12 active+clean+snaptrim_wait; 1.2 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 14 MiB/s wr, 569 op/s 2024-02-02T15:42:11.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:42:11 smithi001 ceph-mon[35470]: pgmap v475: 129 pgs: 3 active+clean+snaptrim, 114 active+clean, 12 active+clean+snaptrim_wait; 1.2 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 14 MiB/s wr, 569 op/s 2024-02-02T15:42:11.979 INFO:tasks.workunit.client.0.smithi001.stdout: 10 58070 31.86 MB/sec warmup 104 sec latency 84.214 ms 2024-02-02T15:42:12.979 INFO:tasks.workunit.client.0.smithi001.stdout: 10 58677 31.74 MB/sec warmup 105 sec latency 59.195 ms 2024-02-02T15:42:13.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:42:13 smithi071 ceph-mon[44707]: pgmap v476: 129 pgs: 129 active+clean; 1.3 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 21 MiB/s wr, 764 op/s 2024-02-02T15:42:13.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:42:13 smithi142 ceph-mon[45728]: pgmap v476: 129 pgs: 129 active+clean; 1.3 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 21 MiB/s wr, 764 op/s 2024-02-02T15:42:13.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:42:13 smithi001 ceph-mon[35470]: pgmap v476: 129 pgs: 129 active+clean; 1.3 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 21 MiB/s wr, 764 op/s 2024-02-02T15:42:13.979 INFO:tasks.workunit.client.0.smithi001.stdout: 10 59301 31.77 MB/sec warmup 106 sec latency 67.251 ms 2024-02-02T15:42:14.979 INFO:tasks.workunit.client.0.smithi001.stdout: 10 60179 31.91 MB/sec warmup 107 sec latency 45.206 ms 2024-02-02T15:42:15.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:42:15 smithi071 ceph-mon[44707]: pgmap v477: 129 pgs: 129 active+clean; 1.3 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 21 MiB/s wr, 764 op/s 2024-02-02T15:42:15.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:42:15 smithi142 ceph-mon[45728]: pgmap v477: 129 pgs: 129 active+clean; 1.3 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 21 MiB/s wr, 764 op/s 2024-02-02T15:42:15.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:42:15 smithi001 ceph-mon[35470]: pgmap v477: 129 pgs: 129 active+clean; 1.3 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 21 MiB/s wr, 764 op/s 2024-02-02T15:42:15.979 INFO:tasks.workunit.client.0.smithi001.stdout: 10 61017 32.09 MB/sec warmup 108 sec latency 43.941 ms 2024-02-02T15:42:16.979 INFO:tasks.workunit.client.0.smithi001.stdout: 10 61820 32.18 MB/sec warmup 109 sec latency 35.050 ms 2024-02-02T15:42:17.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:42:17 smithi071 ceph-mon[44707]: pgmap v478: 129 pgs: 129 active+clean; 1.3 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 18 MiB/s wr, 782 op/s 2024-02-02T15:42:17.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:42:17 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:42:17.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:42:17 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:42:17.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:42:17 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:42:17.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:42:17 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:42:17.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:42:17 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:42:17.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:42:17 smithi071 ceph-mon[44707]: osdmap e107: 12 total, 12 up, 12 in 2024-02-02T15:42:17.923 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:42:17 smithi142 ceph-mon[45728]: pgmap v478: 129 pgs: 129 active+clean; 1.3 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 18 MiB/s wr, 782 op/s 2024-02-02T15:42:17.923 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:42:17 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:42:17.923 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:42:17 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:42:17.924 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:42:17 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:42:17.924 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:42:17 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:42:17.924 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:42:17 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:42:17.924 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:42:17 smithi142 ceph-mon[45728]: osdmap e107: 12 total, 12 up, 12 in 2024-02-02T15:42:17.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:42:17 smithi001 ceph-mon[35470]: pgmap v478: 129 pgs: 129 active+clean; 1.3 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 18 MiB/s wr, 782 op/s 2024-02-02T15:42:17.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:42:17 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:42:17.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:42:17 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:42:17.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:42:17 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:42:17.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:42:17 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:42:17.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:42:17 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:42:17.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:42:17 smithi001 ceph-mon[35470]: osdmap e107: 12 total, 12 up, 12 in 2024-02-02T15:42:17.979 INFO:tasks.workunit.client.0.smithi001.stdout: 10 62522 32.21 MB/sec warmup 110 sec latency 45.398 ms 2024-02-02T15:42:18.979 INFO:tasks.workunit.client.0.smithi001.stdout: 10 63157 32.23 MB/sec warmup 111 sec latency 63.137 ms 2024-02-02T15:42:19.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:42:19 smithi071 ceph-mon[44707]: pgmap v480: 129 pgs: 129 active+clean; 1.3 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 790 op/s 2024-02-02T15:42:19.926 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:42:19 smithi142 ceph-mon[45728]: pgmap v480: 129 pgs: 129 active+clean; 1.3 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 790 op/s 2024-02-02T15:42:19.939 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:42:19 smithi001 ceph-mon[35470]: pgmap v480: 129 pgs: 129 active+clean; 1.3 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 790 op/s 2024-02-02T15:42:19.979 INFO:tasks.workunit.client.0.smithi001.stdout: 10 63895 32.27 MB/sec warmup 112 sec latency 52.857 ms 2024-02-02T15:42:20.979 INFO:tasks.workunit.client.0.smithi001.stdout: 10 64647 32.41 MB/sec warmup 113 sec latency 39.712 ms 2024-02-02T15:42:21.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:42:21 smithi071 ceph-mon[44707]: pgmap v481: 129 pgs: 129 active+clean; 1.3 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 790 op/s 2024-02-02T15:42:21.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:42:21 smithi142 ceph-mon[45728]: pgmap v481: 129 pgs: 129 active+clean; 1.3 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 790 op/s 2024-02-02T15:42:21.939 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:42:21 smithi001 ceph-mon[35470]: pgmap v481: 129 pgs: 129 active+clean; 1.3 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 790 op/s 2024-02-02T15:42:21.980 INFO:tasks.workunit.client.0.smithi001.stdout: 10 65360 32.40 MB/sec warmup 114 sec latency 54.273 ms 2024-02-02T15:42:22.980 INFO:tasks.workunit.client.0.smithi001.stdout: 10 66118 32.51 MB/sec warmup 115 sec latency 51.838 ms 2024-02-02T15:42:23.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:42:23 smithi071 ceph-mon[44707]: pgmap v482: 129 pgs: 129 active+clean; 1.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.3 MiB/s wr, 607 op/s 2024-02-02T15:42:23.923 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:42:23 smithi142 ceph-mon[45728]: pgmap v482: 129 pgs: 129 active+clean; 1.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.3 MiB/s wr, 607 op/s 2024-02-02T15:42:23.939 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:42:23 smithi001 ceph-mon[35470]: pgmap v482: 129 pgs: 129 active+clean; 1.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.3 MiB/s wr, 607 op/s 2024-02-02T15:42:23.980 INFO:tasks.workunit.client.0.smithi001.stdout: 10 66888 32.54 MB/sec warmup 116 sec latency 45.178 ms 2024-02-02T15:42:24.980 INFO:tasks.workunit.client.0.smithi001.stdout: 10 67574 32.62 MB/sec warmup 117 sec latency 92.969 ms 2024-02-02T15:42:25.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:42:25 smithi071 ceph-mon[44707]: pgmap v483: 129 pgs: 129 active+clean; 1.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.3 MiB/s wr, 607 op/s 2024-02-02T15:42:25.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:42:25 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:42:25.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:42:25 smithi142 ceph-mon[45728]: pgmap v483: 129 pgs: 129 active+clean; 1.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.3 MiB/s wr, 607 op/s 2024-02-02T15:42:25.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:42:25 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:42:25.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:42:25 smithi001 ceph-mon[35470]: pgmap v483: 129 pgs: 129 active+clean; 1.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.3 MiB/s wr, 607 op/s 2024-02-02T15:42:25.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:42:25 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:42:25.980 INFO:tasks.workunit.client.0.smithi001.stdout: 10 68251 32.66 MB/sec warmup 118 sec latency 115.927 ms 2024-02-02T15:42:26.980 INFO:tasks.workunit.client.0.smithi001.stdout: 10 68883 32.64 MB/sec warmup 119 sec latency 52.215 ms 2024-02-02T15:42:27.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:42:27 smithi071 ceph-mon[44707]: pgmap v484: 129 pgs: 129 active+clean; 1.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.8 MiB/s wr, 609 op/s 2024-02-02T15:42:27.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:42:27 smithi071 ceph-mon[44707]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:42:27.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:42:27 smithi142 ceph-mon[45728]: pgmap v484: 129 pgs: 129 active+clean; 1.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.8 MiB/s wr, 609 op/s 2024-02-02T15:42:27.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:42:27 smithi142 ceph-mon[45728]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:42:27.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:42:27 smithi001 ceph-mon[35470]: pgmap v484: 129 pgs: 129 active+clean; 1.3 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.8 MiB/s wr, 609 op/s 2024-02-02T15:42:27.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:42:27 smithi001 ceph-mon[35470]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:42:28.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:42:28 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:42:28.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:42:28 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:42:28.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:42:28 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:42:28.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:42:28 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:42:28.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:42:28 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:42:28.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:42:28 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:42:28.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:42:28 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:42:28.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:42:28 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:42:28.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:42:28 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:42:28.980 INFO:tasks.workunit.client.0.smithi001.stdout: 10 70460 34.95 MB/sec execute 1 sec latency 53.387 ms 2024-02-02T15:42:29.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:42:29 smithi071 ceph-mon[44707]: pgmap v485: 129 pgs: 129 active+clean; 1.3 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.4 MiB/s wr, 557 op/s 2024-02-02T15:42:29.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:42:29 smithi142 ceph-mon[45728]: pgmap v485: 129 pgs: 129 active+clean; 1.3 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.4 MiB/s wr, 557 op/s 2024-02-02T15:42:29.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:42:29 smithi001 ceph-mon[35470]: pgmap v485: 129 pgs: 129 active+clean; 1.3 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.4 MiB/s wr, 557 op/s 2024-02-02T15:42:29.983 INFO:tasks.workunit.client.0.smithi001.stdout: 10 71166 39.73 MB/sec execute 2 sec latency 86.372 ms 2024-02-02T15:42:30.981 INFO:tasks.workunit.client.0.smithi001.stdout: 10 71748 37.71 MB/sec execute 3 sec latency 59.043 ms 2024-02-02T15:42:31.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:42:31 smithi071 ceph-mon[44707]: pgmap v486: 129 pgs: 129 active+clean; 1.3 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 6.1 MiB/s wr, 450 op/s 2024-02-02T15:42:31.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:42:31 smithi142 ceph-mon[45728]: pgmap v486: 129 pgs: 129 active+clean; 1.3 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 6.1 MiB/s wr, 450 op/s 2024-02-02T15:42:31.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:42:31 smithi001 ceph-mon[35470]: pgmap v486: 129 pgs: 129 active+clean; 1.3 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 6.1 MiB/s wr, 450 op/s 2024-02-02T15:42:31.981 INFO:tasks.workunit.client.0.smithi001.stdout: 10 72399 35.84 MB/sec execute 4 sec latency 59.308 ms 2024-02-02T15:42:32.981 INFO:tasks.workunit.client.0.smithi001.stdout: 10 73220 37.27 MB/sec execute 5 sec latency 39.129 ms 2024-02-02T15:42:33.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:42:33 smithi071 ceph-mon[44707]: pgmap v487: 129 pgs: 129 active+clean; 1.3 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 9.1 MiB/s wr, 670 op/s 2024-02-02T15:42:33.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:42:33 smithi142 ceph-mon[45728]: pgmap v487: 129 pgs: 129 active+clean; 1.3 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 9.1 MiB/s wr, 670 op/s 2024-02-02T15:42:33.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:42:33 smithi001 ceph-mon[35470]: pgmap v487: 129 pgs: 129 active+clean; 1.3 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 9.1 MiB/s wr, 670 op/s 2024-02-02T15:42:33.981 INFO:tasks.workunit.client.0.smithi001.stdout: 10 74083 38.39 MB/sec execute 6 sec latency 60.309 ms 2024-02-02T15:42:34.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:42:34 smithi071 ceph-mon[44707]: pgmap v488: 129 pgs: 129 active+clean; 1.3 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 6.1 MiB/s wr, 443 op/s 2024-02-02T15:42:34.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:42:34 smithi142 ceph-mon[45728]: pgmap v488: 129 pgs: 129 active+clean; 1.3 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 6.1 MiB/s wr, 443 op/s 2024-02-02T15:42:34.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:42:34 smithi001 ceph-mon[35470]: pgmap v488: 129 pgs: 129 active+clean; 1.3 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 6.1 MiB/s wr, 443 op/s 2024-02-02T15:42:34.981 INFO:tasks.workunit.client.0.smithi001.stdout: 10 74791 39.62 MB/sec execute 7 sec latency 50.571 ms 2024-02-02T15:42:35.697 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-02T15:42:35.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:42:35 smithi071 ceph-mon[44707]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:42:35.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:42:35 smithi142 ceph-mon[45728]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:42:35.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:42:35 smithi001 ceph-mon[35470]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:42:35.983 INFO:tasks.workunit.client.0.smithi001.stdout: 10 75355 38.20 MB/sec execute 8 sec latency 99.356 ms 2024-02-02T15:42:36.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.039+0000 7f448e600700 1 -- 172.21.15.1:0/2637537255 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4488103220 msgr2=0x7f4488103600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:36.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.039+0000 7f448e600700 1 --2- 172.21.15.1:0/2637537255 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4488103220 0x7f4488103600 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f447800b0a0 tx=0x7f447800b3b0 comp rx=0 tx=0).stop 2024-02-02T15:42:36.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.039+0000 7f448e600700 1 -- 172.21.15.1:0/2637537255 shutdown_connections 2024-02-02T15:42:36.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.039+0000 7f448e600700 1 --2- 172.21.15.1:0/2637537255 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4488108340 0x7f448810fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:36.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.039+0000 7f448e600700 1 --2- 172.21.15.1:0/2637537255 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4488103220 0x7f4488103600 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:36.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.039+0000 7f448e600700 1 --2- 172.21.15.1:0/2637537255 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4488103bd0 0x7f4488107c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:36.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.039+0000 7f448e600700 1 -- 172.21.15.1:0/2637537255 >> 172.21.15.1:0/2637537255 conn(0x7f44880fea60 msgr2=0x7f4488100e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:36.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.040+0000 7f448e600700 1 -- 172.21.15.1:0/2637537255 shutdown_connections 2024-02-02T15:42:36.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.041+0000 7f448e600700 1 -- 172.21.15.1:0/2637537255 wait complete. 2024-02-02T15:42:36.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.042+0000 7f448e600700 1 Processor -- start 2024-02-02T15:42:36.041 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.043+0000 7f448e600700 1 -- start start 2024-02-02T15:42:36.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.043+0000 7f448e600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4488103220 0x7f44881990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:36.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.043+0000 7f448e600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4488108340 0x7f4488199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:36.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.043+0000 7f448e600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f448819da70 0x7f448819def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:36.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.043+0000 7f448e600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f448819e4f0 con 0x7f4488108340 2024-02-02T15:42:36.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.043+0000 7f448e600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f448819e660 con 0x7f4488103220 2024-02-02T15:42:36.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.043+0000 7f448e600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f448819e820 con 0x7f448819da70 2024-02-02T15:42:36.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.044+0000 7f447e800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4488108340 0x7f4488199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:36.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.044+0000 7f447e800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4488108340 0x7f4488199610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52916/0 (socket says 172.21.15.1:52916) 2024-02-02T15:42:36.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.044+0000 7f447e800700 1 -- 172.21.15.1:0/1786806437 learned_addr learned my addr 172.21.15.1:0/1786806437 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:42:36.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.044+0000 7f4487400700 1 --2- 172.21.15.1:0/1786806437 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4488103220 0x7f44881990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:36.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.045+0000 7f447e800700 1 -- 172.21.15.1:0/1786806437 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f448819da70 msgr2=0x7f448819def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:36.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.045+0000 7f447e800700 1 --2- 172.21.15.1:0/1786806437 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f448819da70 0x7f448819def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:36.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.045+0000 7f447e800700 1 -- 172.21.15.1:0/1786806437 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4488103220 msgr2=0x7f44881990d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:36.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.045+0000 7f447e800700 1 --2- 172.21.15.1:0/1786806437 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4488103220 0x7f44881990d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:36.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.045+0000 7f447e800700 1 -- 172.21.15.1:0/1786806437 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4478009d00 con 0x7f4488108340 2024-02-02T15:42:36.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.045+0000 7f4487400700 1 --2- 172.21.15.1:0/1786806437 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4488103220 0x7f44881990d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:42:36.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.046+0000 7f447e800700 1 --2- 172.21.15.1:0/1786806437 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4488108340 0x7f4488199610 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f447000e970 tx=0x7f447000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:36.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.047+0000 7f4484c00700 1 -- 172.21.15.1:0/1786806437 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4470009790 con 0x7f4488108340 2024-02-02T15:42:36.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.047+0000 7f4484c00700 1 -- 172.21.15.1:0/1786806437 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f447000eec0 con 0x7f4488108340 2024-02-02T15:42:36.046 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.048+0000 7f448e600700 1 -- 172.21.15.1:0/1786806437 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4488075180 con 0x7f4488108340 2024-02-02T15:42:36.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.048+0000 7f448e600700 1 -- 172.21.15.1:0/1786806437 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4488075690 con 0x7f4488108340 2024-02-02T15:42:36.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.048+0000 7f4484c00700 1 -- 172.21.15.1:0/1786806437 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4470005590 con 0x7f4488108340 2024-02-02T15:42:36.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.057+0000 7f4484c00700 1 -- 172.21.15.1:0/1786806437 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f447000cbd0 con 0x7f4488108340 2024-02-02T15:42:36.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.061+0000 7f4484c00700 1 --2- 172.21.15.1:0/1786806437 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f44680a08a0 0x7f44680a2d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:36.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.062+0000 7f4484c00700 1 -- 172.21.15.1:0/1786806437 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f4470014070 con 0x7f4488108340 2024-02-02T15:42:36.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.062+0000 7f4487400700 1 --2- 172.21.15.1:0/1786806437 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f44680a08a0 0x7f44680a2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:36.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.062+0000 7f44a211ab80 1 Processor -- start 2024-02-02T15:42:36.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.063+0000 7f44a211ab80 1 -- start start 2024-02-02T15:42:36.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.064+0000 7f4487400700 1 --2- 172.21.15.1:0/1786806437 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f44680a08a0 0x7f44680a2d60 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4478000f80 tx=0x7f4478006000 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:36.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.065+0000 7f44a211ab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559b6d745010 0x559b6d7453f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:36.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.065+0000 7f44a211ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b6d745930 0x559b6d7502a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:36.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.065+0000 7f44a211ab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559b6d7507e0 0x559b6d752bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:36.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.065+0000 7f44a211ab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b6d6b94c0 con 0x559b6d745010 2024-02-02T15:42:36.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.065+0000 7f44a211ab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b6d69daa0 con 0x559b6d7507e0 2024-02-02T15:42:36.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.065+0000 7f44a211ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b6d69b460 con 0x559b6d745930 2024-02-02T15:42:36.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.065+0000 7f447e800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b6d745930 0x559b6d7502a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:36.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.065+0000 7f4487e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559b6d7507e0 0x559b6d752bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:36.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.065+0000 7f447e800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b6d745930 0x559b6d7502a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:38062/0 (socket says 172.21.15.1:38062) 2024-02-02T15:42:36.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.065+0000 7f447e800700 1 -- 172.21.15.1:0/2567447324 learned_addr learned my addr 172.21.15.1:0/2567447324 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:42:36.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.065+0000 7f4487400700 1 --2- 172.21.15.1:0/2567447324 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559b6d745010 0x559b6d7453f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:36.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.066+0000 7f4487e00700 1 -- 172.21.15.1:0/2567447324 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b6d745930 msgr2=0x559b6d7502a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:36.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.066+0000 7f4487e00700 1 --2- 172.21.15.1:0/2567447324 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b6d745930 0x559b6d7502a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:36.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.066+0000 7f4487e00700 1 -- 172.21.15.1:0/2567447324 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559b6d745010 msgr2=0x559b6d7453f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:36.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.066+0000 7f4487e00700 1 --2- 172.21.15.1:0/2567447324 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559b6d745010 0x559b6d7453f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:36.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.066+0000 7f4487e00700 1 -- 172.21.15.1:0/2567447324 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b6d5183b0 con 0x559b6d7507e0 2024-02-02T15:42:36.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.066+0000 7f4487e00700 1 --2- 172.21.15.1:0/2567447324 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559b6d7507e0 0x559b6d752bd0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f4480009b80 tx=0x7f4480009e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5853a5d3d60acc7b server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:36.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.067+0000 7f447ca00700 1 -- 172.21.15.1:0/2567447324 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4480004230 con 0x559b6d7507e0 2024-02-02T15:42:36.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.067+0000 7f447ca00700 1 -- 172.21.15.1:0/2567447324 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f448000ed40 con 0x559b6d7507e0 2024-02-02T15:42:36.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.067+0000 7f447ca00700 1 -- 172.21.15.1:0/2567447324 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4480005020 con 0x559b6d7507e0 2024-02-02T15:42:36.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.068+0000 7f44a211ab80 1 -- 172.21.15.1:0/2567447324 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559b6d7507e0 msgr2=0x559b6d752bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:36.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.068+0000 7f44a211ab80 1 --2- 172.21.15.1:0/2567447324 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559b6d7507e0 0x559b6d752bd0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f4480009b80 tx=0x7f4480009e90 comp rx=0 tx=0).stop 2024-02-02T15:42:36.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.069+0000 7f44a211ab80 1 -- 172.21.15.1:0/2567447324 shutdown_connections 2024-02-02T15:42:36.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.069+0000 7f44a211ab80 1 --2- 172.21.15.1:0/2567447324 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559b6d7507e0 0x559b6d752bd0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:36.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.069+0000 7f44a211ab80 1 --2- 172.21.15.1:0/2567447324 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559b6d745010 0x559b6d7453f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:36.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.069+0000 7f44a211ab80 1 --2- 172.21.15.1:0/2567447324 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b6d745930 0x559b6d7502a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:36.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.069+0000 7f44a211ab80 1 -- 172.21.15.1:0/2567447324 >> 172.21.15.1:0/2567447324 conn(0x559b6d6a4db0 msgr2=0x559b6d7413b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:36.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.069+0000 7f44a211ab80 1 -- 172.21.15.1:0/2567447324 shutdown_connections 2024-02-02T15:42:36.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.069+0000 7f44a211ab80 1 -- 172.21.15.1:0/2567447324 wait complete. 2024-02-02T15:42:36.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.071+0000 7f44a211ab80 1 -- start start 2024-02-02T15:42:36.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.071+0000 7f44a211ab80 1 Processor -- start 2024-02-02T15:42:36.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.073+0000 7f44a211ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b6d745010 0x559b6d750260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:36.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.073+0000 7f44a211ab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559b6d745930 0x559b6d74d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:36.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.073+0000 7f44a211ab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559b6d7507e0 0x559b6d74dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:36.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.073+0000 7f44a211ab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b6d696d50 con 0x559b6d7507e0 2024-02-02T15:42:36.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.073+0000 7f44a211ab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b6d632d80 con 0x559b6d745930 2024-02-02T15:42:36.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.073+0000 7f44a211ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b6d632a20 con 0x559b6d745010 2024-02-02T15:42:36.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.073+0000 7f44a211ab80 10 client.0 _ll_register_callbacks cb 0x559b6d634290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:42:36.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.074+0000 7f447e800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559b6d745930 0x559b6d74d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:36.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.074+0000 7f447e800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559b6d745930 0x559b6d74d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:43558/0 (socket says 172.21.15.1:43558) 2024-02-02T15:42:36.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.074+0000 7f447e800700 1 -- 172.21.15.1:0/318486731 learned_addr learned my addr 172.21.15.1:0/318486731 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:42:36.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.075+0000 7f447e800700 1 -- 172.21.15.1:0/318486731 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b6d745010 msgr2=0x559b6d750260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:36.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.075+0000 7f447e800700 1 --2- 172.21.15.1:0/318486731 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b6d745010 0x559b6d750260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:36.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.075+0000 7f447e800700 1 -- 172.21.15.1:0/318486731 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559b6d7507e0 msgr2=0x559b6d74dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:36.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.075+0000 7f447e800700 1 --2- 172.21.15.1:0/318486731 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559b6d7507e0 0x559b6d74dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:36.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.075+0000 7f447e800700 1 -- 172.21.15.1:0/318486731 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4480009800 con 0x559b6d745930 2024-02-02T15:42:36.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.078+0000 7f447e800700 1 --2- 172.21.15.1:0/318486731 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559b6d745930 0x559b6d74d550 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f4470005950 tx=0x7f4470005020 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f07c508521271e42 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:36.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.078+0000 7f4475600700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:42:36.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.078+0000 7f4475600700 1 -- 172.21.15.1:0/318486731 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44700acea0 con 0x559b6d745930 2024-02-02T15:42:36.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.078+0000 7f4475600700 1 -- 172.21.15.1:0/318486731 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f447009d210 con 0x559b6d745930 2024-02-02T15:42:36.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.078+0000 7f44a211ab80 1 -- 172.21.15.1:0/318486731 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559b6d7390f0 con 0x559b6d745930 2024-02-02T15:42:36.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.078+0000 7f4475600700 1 -- 172.21.15.1:0/318486731 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7f44700a1450 con 0x559b6d745930 2024-02-02T15:42:36.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.078+0000 7f44a211ab80 10 client.29666 fetch_fsmap learned FSMap version 63 2024-02-02T15:42:36.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.078+0000 7f44a211ab80 1 -- 172.21.15.1:0/318486731 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) v3 -- 0x559b6d4c4b60 con 0x559b6d745930 2024-02-02T15:42:36.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.078+0000 7f4475600700 1 -- 172.21.15.1:0/318486731 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559b6d7390f0 con 0x559b6d745930 2024-02-02T15:42:36.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.079+0000 7f4475600700 1 -- 172.21.15.1:0/318486731 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 63) v1 ==== 7526+0+0 (secure 0 0 0) 0x7f4470098960 con 0x559b6d745930 2024-02-02T15:42:36.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.079+0000 7f44a211ab80 10 client.29666 fetch_fsmap finished waiting for FSMap version 63 2024-02-02T15:42:36.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.079+0000 7f44a211ab80 10 client.29666 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:42:36.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.079+0000 7f44a211ab80 20 client.29666 populate_metadata read hostname 'smithi001' 2024-02-02T15:42:36.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.079+0000 7f44a211ab80 1 --2- 172.21.15.1:0/318486731 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x559b6d7602f0 0x559b6d762710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:36.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.079+0000 7f44a211ab80 4 client.29666 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:42:36.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.079+0000 7f44a211ab80 1 -- 172.21.15.1:0/318486731 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559b6d68fdd0 con 0x559b6d7602f0 2024-02-02T15:42:36.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.079+0000 7f4487400700 1 --2- 172.21.15.1:0/318486731 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x559b6d7602f0 0x559b6d762710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:36.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.080+0000 7f4475600700 10 client.29666 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:42:36.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.080+0000 7f4487400700 1 --2- 172.21.15.1:0/318486731 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x559b6d7602f0 0x559b6d762710 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e5e386c9e1c87b9 server_cookie=aa897c1726405fc4 in_seq=0 out_seq=0 2024-02-02T15:42:36.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.082+0000 7f4475600700 1 -- 172.21.15.1:0/318486731 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x559b6d68fdd0 con 0x559b6d7602f0 2024-02-02T15:42:36.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.082+0000 7f4475600700 10 client.29666 handle_command_reply: tid=0 2024-02-02T15:42:36.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.082+0000 7f44a211ab80 1 client.29666 shutdown 2024-02-02T15:42:36.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.082+0000 7f44a211ab80 1 -- 172.21.15.1:0/318486731 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559b6d745930 msgr2=0x559b6d74d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:36.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.082+0000 7f44a211ab80 1 --2- 172.21.15.1:0/318486731 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559b6d745930 0x559b6d74d550 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f4470005950 tx=0x7f4470005020 comp rx=0 tx=0).stop 2024-02-02T15:42:36.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.082+0000 7f44a211ab80 1 -- 172.21.15.1:0/318486731 shutdown_connections 2024-02-02T15:42:36.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.082+0000 7f44a211ab80 1 --2- 172.21.15.1:0/318486731 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x559b6d7602f0 0x559b6d762710 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:36.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.082+0000 7f44a211ab80 1 --2- 172.21.15.1:0/318486731 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559b6d7507e0 0x559b6d74dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:36.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.082+0000 7f44a211ab80 1 --2- 172.21.15.1:0/318486731 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b6d745010 0x559b6d750260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:36.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.082+0000 7f44a211ab80 1 --2- 172.21.15.1:0/318486731 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559b6d745930 0x559b6d74d550 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:36.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.082+0000 7f44a211ab80 1 -- 172.21.15.1:0/318486731 >> 172.21.15.1:0/318486731 conn(0x559b6d6a4db0 msgr2=0x559b6d740fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:36.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.083+0000 7f4475600700 0 client.29666 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:42:36.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.083+0000 7f44a211ab80 1 -- 172.21.15.1:0/318486731 shutdown_connections 2024-02-02T15:42:36.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.083+0000 7f44a211ab80 1 -- 172.21.15.1:0/318486731 wait complete. 2024-02-02T15:42:36.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.084+0000 7f44a211ab80 20 client.29666 trim_cache size 0 max 16384 2024-02-02T15:42:36.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.113+0000 7f44a211ab80 1 Processor -- start 2024-02-02T15:42:36.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.113+0000 7f44a211ab80 1 -- start start 2024-02-02T15:42:36.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.114+0000 7f44a211ab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559b6d745010 0x559b6d73fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:36.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.114+0000 7f44a211ab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559b6d7507e0 0x559b6d74cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:36.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.114+0000 7f44a211ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b6d75c010 0x559b6d74d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:36.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.115+0000 7f4487400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559b6d745010 0x559b6d73fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:36.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.115+0000 7f44a211ab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b6d51a6a0 con 0x559b6d7507e0 2024-02-02T15:42:36.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.115+0000 7f44a211ab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b6d7493f0 con 0x559b6d745010 2024-02-02T15:42:36.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.115+0000 7f4487e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b6d75c010 0x559b6d74d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:36.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.115+0000 7f4487e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b6d75c010 0x559b6d74d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:38080/0 (socket says 172.21.15.1:38080) 2024-02-02T15:42:36.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.115+0000 7f4487e00700 1 -- 172.21.15.1:0/2607198200 learned_addr learned my addr 172.21.15.1:0/2607198200 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:42:36.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.115+0000 7f44a211ab80 1 -- 172.21.15.1:0/2607198200 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b6d75cc00 con 0x559b6d75c010 2024-02-02T15:42:36.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.115+0000 7f447e800700 1 --2- 172.21.15.1:0/2607198200 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559b6d7507e0 0x559b6d74cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:36.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.116+0000 7f4487400700 1 -- 172.21.15.1:0/2607198200 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b6d75c010 msgr2=0x559b6d74d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:36.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.116+0000 7f4487400700 1 --2- 172.21.15.1:0/2607198200 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b6d75c010 0x559b6d74d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:36.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.116+0000 7f4487400700 1 -- 172.21.15.1:0/2607198200 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559b6d7507e0 msgr2=0x559b6d74cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:36.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.116+0000 7f4487400700 1 --2- 172.21.15.1:0/2607198200 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559b6d7507e0 0x559b6d74cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:36.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.116+0000 7f4487400700 1 -- 172.21.15.1:0/2607198200 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559b6d65cea0 con 0x559b6d745010 2024-02-02T15:42:36.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.116+0000 7f4487e00700 1 --2- 172.21.15.1:0/2607198200 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b6d75c010 0x559b6d74d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:42:36.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.117+0000 7f4487400700 1 --2- 172.21.15.1:0/2607198200 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559b6d745010 0x559b6d73fd80 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f4478006070 tx=0x7f447800b3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c616c8aeeb6dcb8 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:36.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.119+0000 7f447d400700 1 -- 172.21.15.1:0/2607198200 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447800e040 con 0x559b6d745010 2024-02-02T15:42:36.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.119+0000 7f447d400700 1 -- 172.21.15.1:0/2607198200 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4478034070 con 0x559b6d745010 2024-02-02T15:42:36.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.119+0000 7f447d400700 1 -- 172.21.15.1:0/2607198200 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447800e040 con 0x559b6d745010 2024-02-02T15:42:36.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.119+0000 7f44a211ab80 1 -- 172.21.15.1:0/2607198200 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559b6d745010 msgr2=0x559b6d73fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:36.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.120+0000 7f44a211ab80 1 --2- 172.21.15.1:0/2607198200 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559b6d745010 0x559b6d73fd80 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f4478006070 tx=0x7f447800b3b0 comp rx=0 tx=0).stop 2024-02-02T15:42:36.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.120+0000 7f44a211ab80 1 -- 172.21.15.1:0/2607198200 shutdown_connections 2024-02-02T15:42:36.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.120+0000 7f44a211ab80 1 --2- 172.21.15.1:0/2607198200 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b6d75c010 0x559b6d74d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:36.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.120+0000 7f44a211ab80 1 --2- 172.21.15.1:0/2607198200 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559b6d745010 0x559b6d73fd80 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:36.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.120+0000 7f44a211ab80 1 --2- 172.21.15.1:0/2607198200 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559b6d7507e0 0x559b6d74cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:36.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.120+0000 7f44a211ab80 1 -- 172.21.15.1:0/2607198200 >> 172.21.15.1:0/2607198200 conn(0x559b6d6a4db0 msgr2=0x559b6d75b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:36.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.120+0000 7f44a211ab80 1 -- 172.21.15.1:0/2607198200 shutdown_connections 2024-02-02T15:42:36.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.122+0000 7f44a211ab80 1 -- 172.21.15.1:0/2607198200 wait complete. 2024-02-02T15:42:36.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.123+0000 7f44a211ab80 1 -- start start 2024-02-02T15:42:36.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.123+0000 7f44a211ab80 1 Processor -- start 2024-02-02T15:42:36.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.124+0000 7f44a211ab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b6d745010 0x559b6d754fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:36.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.124+0000 7f44a211ab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559b6d7507e0 0x559b6d755500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:36.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.124+0000 7f44a211ab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559b6d7aee20 0x559b6d7b1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:36.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.124+0000 7f44a211ab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b6d7b1750 con 0x559b6d7aee20 2024-02-02T15:42:36.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.124+0000 7f44a211ab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b6d7b1890 con 0x559b6d7507e0 2024-02-02T15:42:36.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.124+0000 7f44a211ab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559b6d7b1a20 con 0x559b6d745010 2024-02-02T15:42:36.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.124+0000 7f44a211ab80 10 client.0 _ll_register_callbacks cb 0x559b6d634290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:42:36.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.125+0000 7f447e800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559b6d7507e0 0x559b6d755500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:36.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.125+0000 7f447e800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559b6d7507e0 0x559b6d755500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:43572/0 (socket says 172.21.15.1:43572) 2024-02-02T15:42:36.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.125+0000 7f4487e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559b6d7aee20 0x559b6d7b1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:36.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.125+0000 7f4487400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b6d745010 0x559b6d754fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:36.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.126+0000 7f447e800700 1 -- 172.21.15.1:0/818744168 learned_addr learned my addr 172.21.15.1:0/818744168 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:42:36.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.126+0000 7f447e800700 1 -- 172.21.15.1:0/818744168 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b6d745010 msgr2=0x559b6d754fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:36.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.126+0000 7f447e800700 1 --2- 172.21.15.1:0/818744168 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b6d745010 0x559b6d754fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:36.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.126+0000 7f447e800700 1 -- 172.21.15.1:0/818744168 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559b6d7aee20 msgr2=0x559b6d7b1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:36.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.126+0000 7f447e800700 1 --2- 172.21.15.1:0/818744168 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559b6d7aee20 0x559b6d7b1210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:36.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.126+0000 7f447e800700 1 -- 172.21.15.1:0/818744168 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4488110730 con 0x559b6d7507e0 2024-02-02T15:42:36.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.126+0000 7f447e800700 1 --2- 172.21.15.1:0/818744168 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559b6d7507e0 0x559b6d755500 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f447000c5b0 tx=0x7f44700b7480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb9befaa86bb5b02 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:36.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.126+0000 7f4475600700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:42:36.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.127+0000 7f4475600700 1 -- 172.21.15.1:0/818744168 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44700acea0 con 0x559b6d7507e0 2024-02-02T15:42:36.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.127+0000 7f44a211ab80 1 -- 172.21.15.1:0/818744168 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f448000ed40 con 0x559b6d7507e0 2024-02-02T15:42:36.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.127+0000 7f4475600700 1 -- 172.21.15.1:0/818744168 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f447009d210 con 0x559b6d7507e0 2024-02-02T15:42:36.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.127+0000 7f4475600700 1 -- 172.21.15.1:0/818744168 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7f44700a10e0 con 0x559b6d7507e0 2024-02-02T15:42:36.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.127+0000 7f44a211ab80 10 client.29672 fetch_fsmap learned FSMap version 63 2024-02-02T15:42:36.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.127+0000 7f44a211ab80 1 -- 172.21.15.1:0/818744168 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) v3 -- 0x559b6d51ccd0 con 0x559b6d7507e0 2024-02-02T15:42:36.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.128+0000 7f4475600700 1 -- 172.21.15.1:0/818744168 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f448000ed40 con 0x559b6d7507e0 2024-02-02T15:42:36.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.129+0000 7f4475600700 1 -- 172.21.15.1:0/818744168 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 63) v1 ==== 7526+0+0 (secure 0 0 0) 0x7f44700b1b80 con 0x559b6d7507e0 2024-02-02T15:42:36.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.129+0000 7f44a211ab80 10 client.29672 fetch_fsmap finished waiting for FSMap version 63 2024-02-02T15:42:36.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.129+0000 7f44a211ab80 10 client.29672 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:42:36.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.129+0000 7f44a211ab80 20 client.29672 populate_metadata read hostname 'smithi001' 2024-02-02T15:42:36.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.129+0000 7f44a211ab80 1 --2- 172.21.15.1:0/818744168 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x559b6d7c3c90 0x559b6d7c6080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:36.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.129+0000 7f44a211ab80 4 client.29672 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2024-02-02T15:42:36.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.129+0000 7f44a211ab80 1 -- 172.21.15.1:0/818744168 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559b6d655f50 con 0x559b6d7c3c90 2024-02-02T15:42:36.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.129+0000 7f4487400700 1 --2- 172.21.15.1:0/818744168 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x559b6d7c3c90 0x559b6d7c6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:36.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.130+0000 7f4487400700 1 --2- 172.21.15.1:0/818744168 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x559b6d7c3c90 0x559b6d7c6080 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb00554dd9c2f344 server_cookie=698e99c0fbe7e94f in_seq=0 out_seq=0 2024-02-02T15:42:36.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.131+0000 7f4475600700 10 client.29672 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:42:36.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.131+0000 7f4475600700 1 -- 172.21.15.1:0/818744168 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x559b6d655f50 con 0x559b6d7c3c90 2024-02-02T15:42:36.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.131+0000 7f4475600700 10 client.29672 handle_command_reply: tid=0 2024-02-02T15:42:36.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.131+0000 7f44a211ab80 1 client.29672 shutdown 2024-02-02T15:42:36.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.131+0000 7f44a211ab80 1 -- 172.21.15.1:0/818744168 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559b6d7507e0 msgr2=0x559b6d755500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:36.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.131+0000 7f44a211ab80 1 --2- 172.21.15.1:0/818744168 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559b6d7507e0 0x559b6d755500 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f447000c5b0 tx=0x7f44700b7480 comp rx=0 tx=0).stop 2024-02-02T15:42:36.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.133+0000 7f44a211ab80 1 -- 172.21.15.1:0/818744168 shutdown_connections 2024-02-02T15:42:36.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.133+0000 7f44a211ab80 1 --2- 172.21.15.1:0/818744168 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x559b6d7c3c90 0x559b6d7c6080 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:36.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.133+0000 7f44a211ab80 1 --2- 172.21.15.1:0/818744168 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559b6d7aee20 0x559b6d7b1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:36.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.133+0000 7f44a211ab80 1 --2- 172.21.15.1:0/818744168 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559b6d745010 0x559b6d754fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:36.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.133+0000 7f44a211ab80 1 --2- 172.21.15.1:0/818744168 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559b6d7507e0 0x559b6d755500 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:36.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.133+0000 7f44a211ab80 1 -- 172.21.15.1:0/818744168 >> 172.21.15.1:0/818744168 conn(0x559b6d6a4db0 msgr2=0x559b6d7405c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:36.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.133+0000 7f4475600700 0 client.29672 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:42:36.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.134+0000 7f44a211ab80 1 -- 172.21.15.1:0/818744168 shutdown_connections 2024-02-02T15:42:36.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.134+0000 7f44a211ab80 1 -- 172.21.15.1:0/818744168 wait complete. 2024-02-02T15:42:36.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.134+0000 7f44a211ab80 20 client.29672 trim_cache size 0 max 16384 2024-02-02T15:42:36.133 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:42:36.133 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2024-02-02T15:42:36.133 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2024-02-02T15:42:36.134 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:42:36.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.136+0000 7f448e600700 1 -- 172.21.15.1:0/1786806437 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f44680a08a0 msgr2=0x7f44680a2d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:36.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.136+0000 7f448e600700 1 --2- 172.21.15.1:0/1786806437 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f44680a08a0 0x7f44680a2d60 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4478000f80 tx=0x7f4478006000 comp rx=0 tx=0).stop 2024-02-02T15:42:36.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.136+0000 7f448e600700 1 -- 172.21.15.1:0/1786806437 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4488108340 msgr2=0x7f4488199610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:36.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.136+0000 7f448e600700 1 --2- 172.21.15.1:0/1786806437 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4488108340 0x7f4488199610 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f447000e970 tx=0x7f447000ec80 comp rx=0 tx=0).stop 2024-02-02T15:42:36.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.137+0000 7f448e600700 1 -- 172.21.15.1:0/1786806437 shutdown_connections 2024-02-02T15:42:36.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.137+0000 7f448e600700 1 --2- 172.21.15.1:0/1786806437 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f44680a08a0 0x7f44680a2d60 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:36.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.137+0000 7f448e600700 1 --2- 172.21.15.1:0/1786806437 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f448819da70 0x7f448819def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:36.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.137+0000 7f448e600700 1 --2- 172.21.15.1:0/1786806437 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4488103220 0x7f44881990d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:36.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.137+0000 7f448e600700 1 --2- 172.21.15.1:0/1786806437 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4488108340 0x7f4488199610 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:36.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.137+0000 7f448e600700 1 -- 172.21.15.1:0/1786806437 >> 172.21.15.1:0/1786806437 conn(0x7f44880fea60 msgr2=0x7f4488106bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:36.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.137+0000 7f448e600700 1 -- 172.21.15.1:0/1786806437 shutdown_connections 2024-02-02T15:42:36.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.137+0000 7f448e600700 1 -- 172.21.15.1:0/1786806437 wait complete. 2024-02-02T15:42:36.171 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-02T15:42:36.171 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T15:42:36.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.507+0000 7fe41ee00700 1 -- 172.21.15.1:0/2659402609 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe4180ffd20 msgr2=0x7fe41810f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:36.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.507+0000 7fe41ee00700 1 --2- 172.21.15.1:0/2659402609 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe4180ffd20 0x7fe41810f950 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fe41000b0a0 tx=0x7fe41000b3b0 comp rx=0 tx=0).stop 2024-02-02T15:42:36.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.507+0000 7fe41ee00700 1 -- 172.21.15.1:0/2659402609 shutdown_connections 2024-02-02T15:42:36.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.507+0000 7fe41ee00700 1 --2- 172.21.15.1:0/2659402609 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe4180ffd20 0x7fe41810f950 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:36.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.507+0000 7fe41ee00700 1 --2- 172.21.15.1:0/2659402609 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe418101be0 0x7fe4180ff1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:36.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.507+0000 7fe41ee00700 1 --2- 172.21.15.1:0/2659402609 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe418101fc0 0x7fe4180ff700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:36.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.507+0000 7fe41ee00700 1 -- 172.21.15.1:0/2659402609 >> 172.21.15.1:0/2659402609 conn(0x7fe4180760d0 msgr2=0x7fe4180fd450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:36.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.508+0000 7fe41ee00700 1 -- 172.21.15.1:0/2659402609 shutdown_connections 2024-02-02T15:42:36.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.508+0000 7fe41ee00700 1 -- 172.21.15.1:0/2659402609 wait complete. 2024-02-02T15:42:36.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.509+0000 7fe41ee00700 1 Processor -- start 2024-02-02T15:42:36.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.510+0000 7fe41ee00700 1 -- start start 2024-02-02T15:42:36.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.510+0000 7fe41ee00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe4180ffd20 0x7fe4181025b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:36.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.510+0000 7fe41ee00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe418101be0 0x7fe418102af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:36.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.510+0000 7fe41ee00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe418101fc0 0x7fe418103030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:36.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.510+0000 7fe41ee00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4181a97f0 con 0x7fe418101be0 2024-02-02T15:42:36.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.510+0000 7fe41ee00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe418198ec0 con 0x7fe4180ffd20 2024-02-02T15:42:36.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.510+0000 7fe41ee00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe418199080 con 0x7fe418101fc0 2024-02-02T15:42:36.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.510+0000 7fe417e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe4180ffd20 0x7fe4181025b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:36.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.511+0000 7fe417e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe4180ffd20 0x7fe4181025b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:43598/0 (socket says 172.21.15.1:43598) 2024-02-02T15:42:36.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.511+0000 7fe41ce00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe418101fc0 0x7fe418103030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:36.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.511+0000 7fe417e00700 1 -- 172.21.15.1:0/3572776606 learned_addr learned my addr 172.21.15.1:0/3572776606 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:42:36.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.511+0000 7fe417e00700 1 -- 172.21.15.1:0/3572776606 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe418101fc0 msgr2=0x7fe418103030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:36.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.511+0000 7fe417e00700 1 --2- 172.21.15.1:0/3572776606 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe418101fc0 0x7fe418103030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:36.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.511+0000 7fe417e00700 1 -- 172.21.15.1:0/3572776606 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe418101be0 msgr2=0x7fe418102af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:36.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.511+0000 7fe417e00700 1 --2- 172.21.15.1:0/3572776606 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe418101be0 0x7fe418102af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:36.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.511+0000 7fe417e00700 1 -- 172.21.15.1:0/3572776606 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe410009d00 con 0x7fe4180ffd20 2024-02-02T15:42:36.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.511+0000 7fe41ce00700 1 --2- 172.21.15.1:0/3572776606 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe418101fc0 0x7fe418103030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:42:36.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.512+0000 7fe417e00700 1 --2- 172.21.15.1:0/3572776606 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe4180ffd20 0x7fe4181025b0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fe40800e9a0 tx=0x7fe40800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:36.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.512+0000 7fe414c00700 1 -- 172.21.15.1:0/3572776606 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe408009790 con 0x7fe4180ffd20 2024-02-02T15:42:36.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.512+0000 7fe414c00700 1 -- 172.21.15.1:0/3572776606 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe40800cbd0 con 0x7fe4180ffd20 2024-02-02T15:42:36.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.512+0000 7fe414c00700 1 -- 172.21.15.1:0/3572776606 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe408018680 con 0x7fe4180ffd20 2024-02-02T15:42:36.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.512+0000 7fe41ee00700 1 -- 172.21.15.1:0/3572776606 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe418199360 con 0x7fe4180ffd20 2024-02-02T15:42:36.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.513+0000 7fe41ee00700 1 -- 172.21.15.1:0/3572776606 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe418199870 con 0x7fe4180ffd20 2024-02-02T15:42:36.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.514+0000 7fe41ee00700 1 -- 172.21.15.1:0/3572776606 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe418067330 con 0x7fe4180ffd20 2024-02-02T15:42:36.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.517+0000 7fe414c00700 1 -- 172.21.15.1:0/3572776606 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7fe408010430 con 0x7fe4180ffd20 2024-02-02T15:42:36.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.519+0000 7fe414c00700 1 --2- 172.21.15.1:0/3572776606 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fe3f80a07e0 0x7fe3f80a2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:36.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.519+0000 7fe414c00700 1 -- 172.21.15.1:0/3572776606 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fe408014070 con 0x7fe4180ffd20 2024-02-02T15:42:36.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.519+0000 7fe417400700 1 --2- 172.21.15.1:0/3572776606 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fe3f80a07e0 0x7fe3f80a2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:36.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.520+0000 7fe417400700 1 --2- 172.21.15.1:0/3572776606 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fe3f80a07e0 0x7fe3f80a2ca0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fe418103da0 tx=0x7fe400006d20 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:36.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.524+0000 7fe414c00700 1 -- 172.21.15.1:0/3572776606 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7fe408095a00 con 0x7fe4180ffd20 2024-02-02T15:42:36.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.939+0000 7fe41ee00700 1 -- 172.21.15.1:0/3572776606 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe418103ae0 con 0x7fe4180ffd20 2024-02-02T15:42:36.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.940+0000 7fe414c00700 1 -- 172.21.15.1:0/3572776606 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+10889 (secure 0 0 0) 0x7fe408026070 con 0x7fe4180ffd20 2024-02-02T15:42:36.940 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:42:36.940 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":63,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2922240448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2922240448},{"type":"v1","addr":"172.21.15.1:6835","nonce":2922240448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2729497431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2729497431},{"type":"v1","addr":"172.21.15.1:6837","nonce":2729497431}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":63,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-02T15:24:47.761435+0000","modified":"2024-02-02T15:42:34.571858+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24529,"mds_2":24518,"mds_3":24514,"mds_4":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14517":{"gid":14517,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/1218303683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":1218303683},{"type":"v1","addr":"172.21.15.1:6839","nonce":1218303683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14529":{"gid":14529,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/2059197496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2059197496},{"type":"v1","addr":"172.21.15.1:6841","nonce":2059197496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/7648986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":7648986},{"type":"v1","addr":"172.21.15.142:6833","nonce":7648986}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/3482746094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3482746094},{"type":"v1","addr":"172.21.15.71:6833","nonce":3482746094}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/388207814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":388207814},{"type":"v1","addr":"172.21.15.71:6835","nonce":388207814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6835/1052606547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1052606547},{"type":"v1","addr":"172.21.15.142:6835","nonce":1052606547}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.142:6837/2870289019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2870289019},{"type":"v1","addr":"172.21.15.142:6837","nonce":2870289019}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.71:6837/628016199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":628016199},{"type":"v1","addr":"172.21.15.71:6837","nonce":628016199}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.142:6839/321675421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":321675421},{"type":"v1","addr":"172.21.15.142:6839","nonce":321675421}]},"join_fscid":-1,"export_targets":[0,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/1268086493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1268086493},{"type":"v1","addr":"172.21.15.71:6839","nonce":1268086493}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-02T15:42:36.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.943+0000 7fe41ee00700 1 -- 172.21.15.1:0/3572776606 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fe3f80a07e0 msgr2=0x7fe3f80a2ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:36.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.943+0000 7fe41ee00700 1 --2- 172.21.15.1:0/3572776606 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fe3f80a07e0 0x7fe3f80a2ca0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fe418103da0 tx=0x7fe400006d20 comp rx=0 tx=0).stop 2024-02-02T15:42:36.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.943+0000 7fe41ee00700 1 -- 172.21.15.1:0/3572776606 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe4180ffd20 msgr2=0x7fe4181025b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:36.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.943+0000 7fe41ee00700 1 --2- 172.21.15.1:0/3572776606 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe4180ffd20 0x7fe4181025b0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fe40800e9a0 tx=0x7fe40800ecb0 comp rx=0 tx=0).stop 2024-02-02T15:42:36.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.943+0000 7fe41ee00700 1 -- 172.21.15.1:0/3572776606 shutdown_connections 2024-02-02T15:42:36.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.943+0000 7fe41ee00700 1 --2- 172.21.15.1:0/3572776606 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fe3f80a07e0 0x7fe3f80a2ca0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:36.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.943+0000 7fe41ee00700 1 --2- 172.21.15.1:0/3572776606 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe418101fc0 0x7fe418103030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:36.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.943+0000 7fe41ee00700 1 --2- 172.21.15.1:0/3572776606 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe4180ffd20 0x7fe4181025b0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:36.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.943+0000 7fe41ee00700 1 --2- 172.21.15.1:0/3572776606 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe418101be0 0x7fe418102af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:36.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.943+0000 7fe41ee00700 1 -- 172.21.15.1:0/3572776606 >> 172.21.15.1:0/3572776606 conn(0x7fe4180760d0 msgr2=0x7fe4180fc590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:36.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.943+0000 7fe41ee00700 1 -- 172.21.15.1:0/3572776606 shutdown_connections 2024-02-02T15:42:36.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:36.943+0000 7fe41ee00700 1 -- 172.21.15.1:0/3572776606 wait complete. 2024-02-02T15:42:36.943 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 63 2024-02-02T15:42:36.971 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-02-02T15:42:36.983 INFO:tasks.workunit.client.0.smithi001.stdout: 10 75897 37.28 MB/sec execute 9 sec latency 55.139 ms 2024-02-02T15:42:37.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.322+0000 7f1ad6200700 1 -- 172.21.15.1:0/4029081074 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ad0100210 msgr2=0x7f1ad0105a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:37.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.322+0000 7f1ad6200700 1 --2- 172.21.15.1:0/4029081074 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ad0100210 0x7f1ad0105a50 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8009990 tx=0x7f1ab8009ca0 comp rx=0 tx=0).stop 2024-02-02T15:42:37.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.323+0000 7f1ad6200700 1 -- 172.21.15.1:0/4029081074 shutdown_connections 2024-02-02T15:42:37.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.323+0000 7f1ad6200700 1 --2- 172.21.15.1:0/4029081074 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ad0106130 0x7f1ad010dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:37.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.323+0000 7f1ad6200700 1 --2- 172.21.15.1:0/4029081074 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ad00ff860 0x7f1ad00ffc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:37.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.323+0000 7f1ad6200700 1 --2- 172.21.15.1:0/4029081074 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ad0100210 0x7f1ad0105a50 secure :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8009990 tx=0x7f1ab8009ca0 comp rx=0 tx=0).stop 2024-02-02T15:42:37.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.323+0000 7f1ad6200700 1 -- 172.21.15.1:0/4029081074 >> 172.21.15.1:0/4029081074 conn(0x7f1ad0074f30 msgr2=0x7f1ad0075340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:37.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.323+0000 7f1ad6200700 1 -- 172.21.15.1:0/4029081074 shutdown_connections 2024-02-02T15:42:37.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.323+0000 7f1ad6200700 1 -- 172.21.15.1:0/4029081074 wait complete. 2024-02-02T15:42:37.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.325+0000 7f1ad6200700 1 Processor -- start 2024-02-02T15:42:37.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.325+0000 7f1ad6200700 1 -- start start 2024-02-02T15:42:37.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.327+0000 7f1ad6200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ad00ff860 0x7f1ad019d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:37.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.327+0000 7f1ad6200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ad0106130 0x7f1ad019dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:37.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.327+0000 7f1ad6200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ad01a1010 0x7f1ad01a14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:37.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.327+0000 7f1ad6200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad01a1a00 con 0x7f1ad0106130 2024-02-02T15:42:37.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.327+0000 7f1ad6200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad01a1b70 con 0x7f1ad00ff860 2024-02-02T15:42:37.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.327+0000 7f1ad6200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad01a1d30 con 0x7f1ad01a1010 2024-02-02T15:42:37.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.327+0000 7f1acf400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ad00ff860 0x7f1ad019d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:37.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.327+0000 7f1acf400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ad00ff860 0x7f1ad019d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:43606/0 (socket says 172.21.15.1:43606) 2024-02-02T15:42:37.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.327+0000 7f1acf400700 1 -- 172.21.15.1:0/1724138490 learned_addr learned my addr 172.21.15.1:0/1724138490 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:42:37.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.328+0000 7f1acfe00700 1 --2- 172.21.15.1:0/1724138490 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ad01a1010 0x7f1ad01a14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:37.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.328+0000 7f1acea00700 1 --2- 172.21.15.1:0/1724138490 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ad0106130 0x7f1ad019dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:37.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.328+0000 7f1acf400700 1 -- 172.21.15.1:0/1724138490 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ad01a1010 msgr2=0x7f1ad01a14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:37.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.329+0000 7f1acf400700 1 --2- 172.21.15.1:0/1724138490 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ad01a1010 0x7f1ad01a14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:37.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.329+0000 7f1acf400700 1 -- 172.21.15.1:0/1724138490 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ad0106130 msgr2=0x7f1ad019dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:37.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.329+0000 7f1acf400700 1 --2- 172.21.15.1:0/1724138490 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ad0106130 0x7f1ad019dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:37.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.329+0000 7f1acf400700 1 -- 172.21.15.1:0/1724138490 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ab80095f0 con 0x7f1ad00ff860 2024-02-02T15:42:37.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.329+0000 7f1acea00700 1 --2- 172.21.15.1:0/1724138490 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ad0106130 0x7f1ad019dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:42:37.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.329+0000 7f1acf400700 1 --2- 172.21.15.1:0/1724138490 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ad00ff860 0x7f1ad019d720 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f1ac000c9b0 tx=0x7f1ac000ccc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:37.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.329+0000 7f1ac7e00700 1 -- 172.21.15.1:0/1724138490 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac0007b00 con 0x7f1ad00ff860 2024-02-02T15:42:37.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.330+0000 7f1ac7e00700 1 -- 172.21.15.1:0/1724138490 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ac000ce50 con 0x7f1ad00ff860 2024-02-02T15:42:37.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.330+0000 7f1ac7e00700 1 -- 172.21.15.1:0/1724138490 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac0018660 con 0x7f1ad00ff860 2024-02-02T15:42:37.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.330+0000 7f1ad6200700 1 -- 172.21.15.1:0/1724138490 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ad01aa3a0 con 0x7f1ad00ff860 2024-02-02T15:42:37.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.330+0000 7f1ad6200700 1 -- 172.21.15.1:0/1724138490 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ad01aa830 con 0x7f1ad00ff860 2024-02-02T15:42:37.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.336+0000 7f1ac7e00700 1 -- 172.21.15.1:0/1724138490 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f1ac0004830 con 0x7f1ad00ff860 2024-02-02T15:42:37.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.341+0000 7f1ac7e00700 1 --2- 172.21.15.1:0/1724138490 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f1ab00a0720 0x7f1ab00a2be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:37.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.342+0000 7f1ac7e00700 1 -- 172.21.15.1:0/1724138490 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f1ac00cbfe0 con 0x7f1ad00ff860 2024-02-02T15:42:37.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.343+0000 7f1acea00700 1 --2- 172.21.15.1:0/1724138490 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f1ab00a0720 0x7f1ab00a2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:37.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.344+0000 7f1ae9cc0b80 1 Processor -- start 2024-02-02T15:42:37.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.344+0000 7f1ae9cc0b80 1 -- start start 2024-02-02T15:42:37.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.345+0000 7f1ae9cc0b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5639e28e0010 0x5639e28e03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:37.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.346+0000 7f1acf400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5639e28e0010 0x5639e28e03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:37.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.346+0000 7f1acf400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5639e28e0010 0x5639e28e03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:43620/0 (socket says 172.21.15.1:43620) 2024-02-02T15:42:37.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.346+0000 7f1ae9cc0b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5639e28e0930 0x5639e28fc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:37.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.346+0000 7f1acf400700 1 -- 172.21.15.1:0/4088864662 learned_addr learned my addr 172.21.15.1:0/4088864662 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:42:37.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.346+0000 7f1ae9cc0b80 1 --2- 172.21.15.1:0/4088864662 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5639e28fc7f0 0x5639e28febe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:37.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.346+0000 7f1acea00700 1 --2- 172.21.15.1:0/4088864662 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5639e28e0930 0x5639e28fc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:37.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.346+0000 7f1acfe00700 1 --2- 172.21.15.1:0/4088864662 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5639e28fc7f0 0x5639e28febe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:37.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.346+0000 7f1ae9cc0b80 1 -- 172.21.15.1:0/4088864662 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639e28544c0 con 0x5639e28e0930 2024-02-02T15:42:37.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.346+0000 7f1ae9cc0b80 1 -- 172.21.15.1:0/4088864662 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639e2838aa0 con 0x5639e28e0010 2024-02-02T15:42:37.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.346+0000 7f1ae9cc0b80 1 -- 172.21.15.1:0/4088864662 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639e2836460 con 0x5639e28fc7f0 2024-02-02T15:42:37.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.347+0000 7f1acea00700 1 --2- 172.21.15.1:0/1724138490 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f1ab00a0720 0x7f1ab00a2be0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8008000 tx=0x7f1ab8005fb0 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:37.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.347+0000 7f1acf400700 1 -- 172.21.15.1:0/4088864662 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5639e28fc7f0 msgr2=0x5639e28febe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:37.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.347+0000 7f1acf400700 1 --2- 172.21.15.1:0/4088864662 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5639e28fc7f0 0x5639e28febe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:37.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.348+0000 7f1acf400700 1 -- 172.21.15.1:0/4088864662 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5639e28e0930 msgr2=0x5639e28fc2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:37.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.348+0000 7f1acf400700 1 --2- 172.21.15.1:0/4088864662 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5639e28e0930 0x5639e28fc2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:37.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.348+0000 7f1acf400700 1 -- 172.21.15.1:0/4088864662 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5639e26b33b0 con 0x5639e28e0010 2024-02-02T15:42:37.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.348+0000 7f1acfe00700 1 --2- 172.21.15.1:0/4088864662 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5639e28fc7f0 0x5639e28febe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:42:37.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.349+0000 7f1acf400700 1 --2- 172.21.15.1:0/4088864662 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5639e28e0010 0x5639e28e03f0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0095bd0 tx=0x7f1ac0098b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=283db3eb34e7ef81 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:37.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.350+0000 7f1abfe00700 1 -- 172.21.15.1:0/4088864662 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac0099860 con 0x5639e28e0010 2024-02-02T15:42:37.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.350+0000 7f1abfe00700 1 -- 172.21.15.1:0/4088864662 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ac0099160 con 0x5639e28e0010 2024-02-02T15:42:37.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.350+0000 7f1abfe00700 1 -- 172.21.15.1:0/4088864662 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac001f560 con 0x5639e28e0010 2024-02-02T15:42:37.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.351+0000 7f1ae9cc0b80 1 -- 172.21.15.1:0/4088864662 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5639e28e0010 msgr2=0x5639e28e03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:37.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.351+0000 7f1ae9cc0b80 1 --2- 172.21.15.1:0/4088864662 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5639e28e0010 0x5639e28e03f0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0095bd0 tx=0x7f1ac0098b10 comp rx=0 tx=0).stop 2024-02-02T15:42:37.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.352+0000 7f1ae9cc0b80 1 -- 172.21.15.1:0/4088864662 shutdown_connections 2024-02-02T15:42:37.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.352+0000 7f1ae9cc0b80 1 --2- 172.21.15.1:0/4088864662 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5639e28fc7f0 0x5639e28febe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:37.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.352+0000 7f1ae9cc0b80 1 --2- 172.21.15.1:0/4088864662 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5639e28e0010 0x5639e28e03f0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:37.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.352+0000 7f1ae9cc0b80 1 --2- 172.21.15.1:0/4088864662 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5639e28e0930 0x5639e28fc2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:37.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.352+0000 7f1ae9cc0b80 1 -- 172.21.15.1:0/4088864662 >> 172.21.15.1:0/4088864662 conn(0x5639e283fdb0 msgr2=0x5639e28dc3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:37.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.352+0000 7f1ae9cc0b80 1 -- 172.21.15.1:0/4088864662 shutdown_connections 2024-02-02T15:42:37.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.353+0000 7f1ae9cc0b80 1 -- 172.21.15.1:0/4088864662 wait complete. 2024-02-02T15:42:37.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.353+0000 7f1ae9cc0b80 1 -- start start 2024-02-02T15:42:37.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.354+0000 7f1ae9cc0b80 1 Processor -- start 2024-02-02T15:42:37.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.354+0000 7f1ae9cc0b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5639e28e0010 0x5639e28f94e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:37.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.354+0000 7f1ae9cc0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5639e28e0930 0x5639e28f9a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:37.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.354+0000 7f1ae9cc0b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5639e28fc7f0 0x5639e28f9f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:37.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.355+0000 7f1acea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5639e28e0930 0x5639e28f9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:37.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.355+0000 7f1acea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5639e28e0930 0x5639e28f9a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:38154/0 (socket says 172.21.15.1:38154) 2024-02-02T15:42:37.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.355+0000 7f1acf400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5639e28e0010 0x5639e28f94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:37.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.355+0000 7f1acea00700 1 -- 172.21.15.1:0/2738324628 learned_addr learned my addr 172.21.15.1:0/2738324628 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:42:37.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.355+0000 7f1ae9cc0b80 1 -- 172.21.15.1:0/2738324628 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639e2831d50 con 0x5639e28fc7f0 2024-02-02T15:42:37.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.355+0000 7f1ae9cc0b80 1 -- 172.21.15.1:0/2738324628 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639e27cdd80 con 0x5639e28e0010 2024-02-02T15:42:37.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.355+0000 7f1ae9cc0b80 1 -- 172.21.15.1:0/2738324628 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639e27cda20 con 0x5639e28e0930 2024-02-02T15:42:37.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.355+0000 7f1acfe00700 1 --2- 172.21.15.1:0/2738324628 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5639e28fc7f0 0x5639e28f9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:37.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.355+0000 7f1ae9cc0b80 10 client.0 _ll_register_callbacks cb 0x5639e27cf290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:42:37.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.356+0000 7f1acf400700 1 -- 172.21.15.1:0/2738324628 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5639e28e0930 msgr2=0x5639e28f9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:37.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.356+0000 7f1acf400700 1 --2- 172.21.15.1:0/2738324628 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5639e28e0930 0x5639e28f9a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:37.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.356+0000 7f1acf400700 1 -- 172.21.15.1:0/2738324628 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5639e28fc7f0 msgr2=0x5639e28f9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:37.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.356+0000 7f1acf400700 1 --2- 172.21.15.1:0/2738324628 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5639e28fc7f0 0x5639e28f9f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:37.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.356+0000 7f1acf400700 1 -- 172.21.15.1:0/2738324628 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ac8009ba0 con 0x5639e28e0010 2024-02-02T15:42:37.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.356+0000 7f1acea00700 1 --2- 172.21.15.1:0/2738324628 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5639e28e0930 0x5639e28f9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:42:37.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.356+0000 7f1acf400700 1 --2- 172.21.15.1:0/2738324628 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5639e28e0010 0x5639e28f94e0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f1ac000afd0 tx=0x7f1ac000fec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc32f67c7adf822 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:37.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.356+0000 7f1abcc00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:42:37.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.356+0000 7f1abcc00700 1 -- 172.21.15.1:0/2738324628 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac00a11d0 con 0x5639e28e0010 2024-02-02T15:42:37.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.356+0000 7f1abcc00700 1 -- 172.21.15.1:0/2738324628 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f1ac001f510 con 0x5639e28e0010 2024-02-02T15:42:37.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.358+0000 7f1ae9cc0b80 1 -- 172.21.15.1:0/2738324628 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5639e28d40f0 con 0x5639e28e0010 2024-02-02T15:42:37.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.359+0000 7f1abcc00700 1 -- 172.21.15.1:0/2738324628 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ac00993c0 con 0x5639e28e0010 2024-02-02T15:42:37.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.359+0000 7f1ae9cc0b80 10 client.29690 fetch_fsmap learned FSMap version 63 2024-02-02T15:42:37.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.359+0000 7f1ae9cc0b80 1 -- 172.21.15.1:0/2738324628 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) v3 -- 0x7f1ad01aa830 con 0x5639e28e0010 2024-02-02T15:42:37.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.359+0000 7f1abcc00700 1 -- 172.21.15.1:0/2738324628 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5639e28d40f0 con 0x5639e28e0010 2024-02-02T15:42:37.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.359+0000 7f1abcc00700 1 -- 172.21.15.1:0/2738324628 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 63) v1 ==== 7526+0+0 (secure 0 0 0) 0x7f1ac00a0e50 con 0x5639e28e0010 2024-02-02T15:42:37.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.359+0000 7f1ae9cc0b80 10 client.29690 fetch_fsmap finished waiting for FSMap version 63 2024-02-02T15:42:37.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.359+0000 7f1ae9cc0b80 10 client.29690 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-02-02T15:42:37.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.359+0000 7f1ae9cc0b80 20 client.29690 populate_metadata read hostname 'smithi001' 2024-02-02T15:42:37.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.359+0000 7f1ae9cc0b80 1 --2- 172.21.15.1:0/2738324628 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x5639e290bc70 0x5639e290e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:37.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.359+0000 7f1ae9cc0b80 4 client.29690 mds_command: new command op to 24502 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:42:37.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.359+0000 7f1ae9cc0b80 1 -- 172.21.15.1:0/2738324628 --> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5639e282add0 con 0x5639e290bc70 2024-02-02T15:42:37.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.360+0000 7f1acea00700 1 --2- 172.21.15.1:0/2738324628 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x5639e290bc70 0x5639e290e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:37.360 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.360+0000 7f1abcc00700 10 client.29690 ms_handle_connect on v2:172.21.15.142:6834/1052606547 2024-02-02T15:42:37.361 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.362+0000 7f1acea00700 1 --2- 172.21.15.1:0/2738324628 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x5639e290bc70 0x5639e290e060 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a150eeab8b8972bc server_cookie=13b17c3803dee60a in_seq=0 out_seq=0 2024-02-02T15:42:37.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.364+0000 7f1abcc00700 1 -- 172.21.15.1:0/2738324628 <== mds.4 v2:172.21.15.142:6834/1052606547 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5639e282add0 con 0x5639e290bc70 2024-02-02T15:42:37.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.364+0000 7f1abcc00700 10 client.29690 handle_command_reply: tid=0 2024-02-02T15:42:37.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.364+0000 7f1ae9cc0b80 1 client.29690 shutdown 2024-02-02T15:42:37.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.364+0000 7f1ae9cc0b80 1 -- 172.21.15.1:0/2738324628 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5639e28e0010 msgr2=0x5639e28f94e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:37.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.364+0000 7f1ae9cc0b80 1 --2- 172.21.15.1:0/2738324628 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5639e28e0010 0x5639e28f94e0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f1ac000afd0 tx=0x7f1ac000fec0 comp rx=0 tx=0).stop 2024-02-02T15:42:37.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.365+0000 7f1ae9cc0b80 1 -- 172.21.15.1:0/2738324628 shutdown_connections 2024-02-02T15:42:37.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.365+0000 7f1ae9cc0b80 1 --2- 172.21.15.1:0/2738324628 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x5639e290bc70 0x5639e290e060 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:37.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.365+0000 7f1ae9cc0b80 1 --2- 172.21.15.1:0/2738324628 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5639e28fc7f0 0x5639e28f9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:37.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.365+0000 7f1ae9cc0b80 1 --2- 172.21.15.1:0/2738324628 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5639e28e0010 0x5639e28f94e0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:37.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.365+0000 7f1ae9cc0b80 1 --2- 172.21.15.1:0/2738324628 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5639e28e0930 0x5639e28f9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:37.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.365+0000 7f1abcc00700 0 client.29690 ms_handle_reset on v2:172.21.15.142:6834/1052606547 2024-02-02T15:42:37.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.365+0000 7f1ae9cc0b80 1 -- 172.21.15.1:0/2738324628 >> 172.21.15.1:0/2738324628 conn(0x5639e283fdb0 msgr2=0x5639e28dbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:37.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.366+0000 7f1ae9cc0b80 1 -- 172.21.15.1:0/2738324628 shutdown_connections 2024-02-02T15:42:37.365 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.367+0000 7f1ae9cc0b80 1 -- 172.21.15.1:0/2738324628 wait complete. 2024-02-02T15:42:37.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.367+0000 7f1ae9cc0b80 20 client.29690 trim_cache size 0 max 16384 2024-02-02T15:42:37.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.395+0000 7f1ae9cc0b80 1 Processor -- start 2024-02-02T15:42:37.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.395+0000 7f1ae9cc0b80 1 -- start start 2024-02-02T15:42:37.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.395+0000 7f1ae9cc0b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5639e28e0010 0x5639e28f9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:37.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.396+0000 7f1ae9cc0b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5639e28e0930 0x5639e28f4850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:37.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.396+0000 7f1ae9cc0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5639e28fc7f0 0x5639e28f4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:37.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.396+0000 7f1ae9cc0b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639e26b56a0 con 0x5639e28e0010 2024-02-02T15:42:37.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.396+0000 7f1ae9cc0b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639e28dd060 con 0x5639e28e0930 2024-02-02T15:42:37.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.396+0000 7f1ae9cc0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639e28d3cf0 con 0x5639e28fc7f0 2024-02-02T15:42:37.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.397+0000 7f1acf400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5639e28e0010 0x5639e28f9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:37.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.397+0000 7f1acf400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5639e28e0010 0x5639e28f9d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53000/0 (socket says 172.21.15.1:53000) 2024-02-02T15:42:37.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.397+0000 7f1acf400700 1 -- 172.21.15.1:0/3800384477 learned_addr learned my addr 172.21.15.1:0/3800384477 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:42:37.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.398+0000 7f1acfe00700 1 --2- 172.21.15.1:0/3800384477 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5639e28fc7f0 0x5639e28f4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:37.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.398+0000 7f1acf400700 1 -- 172.21.15.1:0/3800384477 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5639e28fc7f0 msgr2=0x5639e28f4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:37.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.398+0000 7f1acf400700 1 --2- 172.21.15.1:0/3800384477 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5639e28fc7f0 0x5639e28f4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:37.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.398+0000 7f1acf400700 1 -- 172.21.15.1:0/3800384477 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5639e28e0930 msgr2=0x5639e28f4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:37.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.398+0000 7f1acf400700 1 --2- 172.21.15.1:0/3800384477 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5639e28e0930 0x5639e28f4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:37.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.398+0000 7f1acf400700 1 -- 172.21.15.1:0/3800384477 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5639e265fb60 con 0x5639e28e0010 2024-02-02T15:42:37.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.399+0000 7f1acf400700 1 --2- 172.21.15.1:0/3800384477 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5639e28e0010 0x5639e28f9d90 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0095660 tx=0x7f1ac0095550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6857e124a19e7553 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:37.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.403+0000 7f1ac5200700 1 -- 172.21.15.1:0/3800384477 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac001f510 con 0x5639e28e0010 2024-02-02T15:42:37.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.403+0000 7f1ac5200700 1 -- 172.21.15.1:0/3800384477 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ac0098ac0 con 0x5639e28e0010 2024-02-02T15:42:37.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.403+0000 7f1ac5200700 1 -- 172.21.15.1:0/3800384477 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac00988b0 con 0x5639e28e0010 2024-02-02T15:42:37.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.404+0000 7f1ae9cc0b80 1 -- 172.21.15.1:0/3800384477 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5639e28e0010 msgr2=0x5639e28f9d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:37.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.404+0000 7f1ae9cc0b80 1 --2- 172.21.15.1:0/3800384477 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5639e28e0010 0x5639e28f9d90 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0095660 tx=0x7f1ac0095550 comp rx=0 tx=0).stop 2024-02-02T15:42:37.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.405+0000 7f1ae9cc0b80 1 -- 172.21.15.1:0/3800384477 shutdown_connections 2024-02-02T15:42:37.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.405+0000 7f1ae9cc0b80 1 --2- 172.21.15.1:0/3800384477 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5639e28fc7f0 0x5639e28f4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:37.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.405+0000 7f1ae9cc0b80 1 --2- 172.21.15.1:0/3800384477 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5639e28e0010 0x5639e28f9d90 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:37.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.405+0000 7f1ae9cc0b80 1 --2- 172.21.15.1:0/3800384477 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5639e28e0930 0x5639e28f4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:37.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.405+0000 7f1ae9cc0b80 1 -- 172.21.15.1:0/3800384477 >> 172.21.15.1:0/3800384477 conn(0x5639e283fdb0 msgr2=0x5639e2907850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:37.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.406+0000 7f1ae9cc0b80 1 -- 172.21.15.1:0/3800384477 shutdown_connections 2024-02-02T15:42:37.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.406+0000 7f1ae9cc0b80 1 -- 172.21.15.1:0/3800384477 wait complete. 2024-02-02T15:42:37.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.407+0000 7f1ae9cc0b80 1 -- start start 2024-02-02T15:42:37.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.407+0000 7f1ae9cc0b80 1 Processor -- start 2024-02-02T15:42:37.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.409+0000 7f1ae9cc0b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5639e28e0010 0x5639e29031d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:37.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.409+0000 7f1ae9cc0b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5639e28e0930 0x5639e29004f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:37.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.409+0000 7f1ae9cc0b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5639e28fc7f0 0x5639e2900ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:37.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.410+0000 7f1ae9cc0b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639e29011c0 con 0x5639e28e0010 2024-02-02T15:42:37.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.410+0000 7f1ae9cc0b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639e2901330 con 0x5639e28e0930 2024-02-02T15:42:37.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.410+0000 7f1ae9cc0b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639e295c6f0 con 0x5639e28fc7f0 2024-02-02T15:42:37.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.410+0000 7f1ae9cc0b80 10 client.0 _ll_register_callbacks cb 0x5639e27cf290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:42:37.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.410+0000 7f1acfe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5639e28fc7f0 0x5639e2900ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:37.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.410+0000 7f1acfe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5639e28fc7f0 0x5639e2900ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:38176/0 (socket says 172.21.15.1:38176) 2024-02-02T15:42:37.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.410+0000 7f1acfe00700 1 -- 172.21.15.1:0/3645089780 learned_addr learned my addr 172.21.15.1:0/3645089780 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:42:37.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.410+0000 7f1acf400700 1 --2- 172.21.15.1:0/3645089780 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5639e28e0010 0x5639e29031d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:37.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.410+0000 7f1acea00700 1 --2- 172.21.15.1:0/3645089780 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5639e28e0930 0x5639e29004f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:37.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.410+0000 7f1acfe00700 1 -- 172.21.15.1:0/3645089780 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5639e28e0930 msgr2=0x5639e29004f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:37.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.410+0000 7f1acfe00700 1 --2- 172.21.15.1:0/3645089780 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5639e28e0930 0x5639e29004f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:37.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.410+0000 7f1acfe00700 1 -- 172.21.15.1:0/3645089780 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5639e28e0010 msgr2=0x5639e29031d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:37.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.410+0000 7f1acfe00700 1 --2- 172.21.15.1:0/3645089780 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5639e28e0010 0x5639e29031d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:37.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.410+0000 7f1acfe00700 1 -- 172.21.15.1:0/3645089780 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ad01aa830 con 0x5639e28fc7f0 2024-02-02T15:42:37.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.411+0000 7f1acea00700 1 --2- 172.21.15.1:0/3645089780 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5639e28e0930 0x5639e29004f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:42:37.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.412+0000 7f1acfe00700 1 --2- 172.21.15.1:0/3645089780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5639e28fc7f0 0x5639e2900ab0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f1ac800af40 tx=0x7f1ac80173e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b68463f4016d88f0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:37.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.412+0000 7f1abcc00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:42:37.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.413+0000 7f1abcc00700 1 -- 172.21.15.1:0/3645089780 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac8010040 con 0x5639e28fc7f0 2024-02-02T15:42:37.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.413+0000 7f1abcc00700 1 -- 172.21.15.1:0/3645089780 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f1ac801d070 con 0x5639e28fc7f0 2024-02-02T15:42:37.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.414+0000 7f1ae9cc0b80 1 -- 172.21.15.1:0/3645089780 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1ac0099160 con 0x5639e28fc7f0 2024-02-02T15:42:37.413 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.415+0000 7f1abcc00700 1 -- 172.21.15.1:0/3645089780 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ac8017b60 con 0x5639e28fc7f0 2024-02-02T15:42:37.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.415+0000 7f1ae9cc0b80 10 client.29647 fetch_fsmap learned FSMap version 63 2024-02-02T15:42:37.414 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.415+0000 7f1ae9cc0b80 1 -- 172.21.15.1:0/3645089780 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) v3 -- 0x5639e27f7ea0 con 0x5639e28fc7f0 2024-02-02T15:42:37.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.417+0000 7f1abcc00700 1 -- 172.21.15.1:0/3645089780 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ac0099160 con 0x5639e28fc7f0 2024-02-02T15:42:37.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.417+0000 7f1abcc00700 1 -- 172.21.15.1:0/3645089780 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 63) v1 ==== 7526+0+0 (secure 0 0 0) 0x7f1ac802c420 con 0x5639e28fc7f0 2024-02-02T15:42:37.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.417+0000 7f1ae9cc0b80 10 client.29647 fetch_fsmap finished waiting for FSMap version 63 2024-02-02T15:42:37.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.417+0000 7f1ae9cc0b80 10 client.29647 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-02-02T15:42:37.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.417+0000 7f1ae9cc0b80 20 client.29647 populate_metadata read hostname 'smithi001' 2024-02-02T15:42:37.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.418+0000 7f1ae9cc0b80 1 --2- 172.21.15.1:0/3645089780 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x5639e295de80 0x5639e29602a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:37.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.418+0000 7f1ae9cc0b80 4 client.29647 mds_command: new command op to 24502 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:42:37.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.418+0000 7f1ae9cc0b80 1 -- 172.21.15.1:0/3645089780 --> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5639e27f0f50 con 0x5639e295de80 2024-02-02T15:42:37.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.420+0000 7f1acf400700 1 --2- 172.21.15.1:0/3645089780 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x5639e295de80 0x5639e29602a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:37.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.423+0000 7f1acf400700 1 --2- 172.21.15.1:0/3645089780 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x5639e295de80 0x5639e29602a0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5f7cb174e8db935d server_cookie=bc2d2160fd231740 in_seq=0 out_seq=0 2024-02-02T15:42:37.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.423+0000 7f1abcc00700 10 client.29647 ms_handle_connect on v2:172.21.15.142:6834/1052606547 2024-02-02T15:42:37.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.428+0000 7f1abcc00700 1 -- 172.21.15.1:0/3645089780 <== mds.4 v2:172.21.15.142:6834/1052606547 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5639e27f0f50 con 0x5639e295de80 2024-02-02T15:42:37.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.428+0000 7f1abcc00700 10 client.29647 handle_command_reply: tid=0 2024-02-02T15:42:37.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.428+0000 7f1ae9cc0b80 1 client.29647 shutdown 2024-02-02T15:42:37.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.428+0000 7f1ae9cc0b80 1 -- 172.21.15.1:0/3645089780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5639e28fc7f0 msgr2=0x5639e2900ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:37.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.428+0000 7f1ae9cc0b80 1 --2- 172.21.15.1:0/3645089780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5639e28fc7f0 0x5639e2900ab0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f1ac800af40 tx=0x7f1ac80173e0 comp rx=0 tx=0).stop 2024-02-02T15:42:37.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.428+0000 7f1ae9cc0b80 1 -- 172.21.15.1:0/3645089780 shutdown_connections 2024-02-02T15:42:37.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.428+0000 7f1ae9cc0b80 1 --2- 172.21.15.1:0/3645089780 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x5639e295de80 0x5639e29602a0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:37.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.428+0000 7f1ae9cc0b80 1 --2- 172.21.15.1:0/3645089780 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5639e28fc7f0 0x5639e2900ab0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:37.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.428+0000 7f1ae9cc0b80 1 --2- 172.21.15.1:0/3645089780 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5639e28e0010 0x5639e29031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:37.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.428+0000 7f1ae9cc0b80 1 --2- 172.21.15.1:0/3645089780 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5639e28e0930 0x5639e29004f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:37.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.428+0000 7f1ae9cc0b80 1 -- 172.21.15.1:0/3645089780 >> 172.21.15.1:0/3645089780 conn(0x5639e283fdb0 msgr2=0x5639e28f4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:37.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.429+0000 7f1abcc00700 0 client.29647 ms_handle_reset on v2:172.21.15.142:6834/1052606547 2024-02-02T15:42:37.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.430+0000 7f1ae9cc0b80 1 -- 172.21.15.1:0/3645089780 shutdown_connections 2024-02-02T15:42:37.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.431+0000 7f1ae9cc0b80 1 -- 172.21.15.1:0/3645089780 wait complete. 2024-02-02T15:42:37.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.432+0000 7f1ae9cc0b80 20 client.29647 trim_cache size 0 max 16384 2024-02-02T15:42:37.431 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:42:37.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.434+0000 7f1ad6200700 1 -- 172.21.15.1:0/1724138490 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f1ab00a0720 msgr2=0x7f1ab00a2be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:37.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.434+0000 7f1ad6200700 1 --2- 172.21.15.1:0/1724138490 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f1ab00a0720 0x7f1ab00a2be0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8008000 tx=0x7f1ab8005fb0 comp rx=0 tx=0).stop 2024-02-02T15:42:37.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.434+0000 7f1ad6200700 1 -- 172.21.15.1:0/1724138490 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ad00ff860 msgr2=0x7f1ad019d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:37.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.434+0000 7f1ad6200700 1 --2- 172.21.15.1:0/1724138490 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ad00ff860 0x7f1ad019d720 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f1ac000c9b0 tx=0x7f1ac000ccc0 comp rx=0 tx=0).stop 2024-02-02T15:42:37.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.434+0000 7f1ad6200700 1 -- 172.21.15.1:0/1724138490 shutdown_connections 2024-02-02T15:42:37.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.434+0000 7f1ad6200700 1 --2- 172.21.15.1:0/1724138490 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f1ab00a0720 0x7f1ab00a2be0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:37.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.434+0000 7f1ad6200700 1 --2- 172.21.15.1:0/1724138490 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1ad01a1010 0x7f1ad01a14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:37.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.434+0000 7f1ad6200700 1 --2- 172.21.15.1:0/1724138490 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ad00ff860 0x7f1ad019d720 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:37.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.434+0000 7f1ad6200700 1 --2- 172.21.15.1:0/1724138490 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1ad0106130 0x7f1ad019dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:37.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.434+0000 7f1ad6200700 1 -- 172.21.15.1:0/1724138490 >> 172.21.15.1:0/1724138490 conn(0x7f1ad0074f30 msgr2=0x7f1ad00fd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:37.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.436+0000 7f1ad6200700 1 -- 172.21.15.1:0/1724138490 shutdown_connections 2024-02-02T15:42:37.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.436+0000 7f1ad6200700 1 -- 172.21.15.1:0/1724138490 wait complete. 2024-02-02T15:42:37.462 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-02-02T15:42:37.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.810+0000 7effb5e00700 1 -- 172.21.15.1:0/3378465751 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7effb0101e80 msgr2=0x7effb0102260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:37.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.810+0000 7effb5e00700 1 --2- 172.21.15.1:0/3378465751 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7effb0101e80 0x7effb0102260 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7effa0009990 tx=0x7effa0009ca0 comp rx=0 tx=0).stop 2024-02-02T15:42:37.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.812+0000 7effb5e00700 1 -- 172.21.15.1:0/3378465751 shutdown_connections 2024-02-02T15:42:37.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.812+0000 7effb5e00700 1 --2- 172.21.15.1:0/3378465751 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7effb010b2a0 0x7effb010d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:37.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.812+0000 7effb5e00700 1 --2- 172.21.15.1:0/3378465751 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7effb0101e80 0x7effb0102260 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:37.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.812+0000 7effb5e00700 1 --2- 172.21.15.1:0/3378465751 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7effb01027a0 0x7effb010ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:37.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.812+0000 7effb5e00700 1 -- 172.21.15.1:0/3378465751 >> 172.21.15.1:0/3378465751 conn(0x7effb00fb6a0 msgr2=0x7effb00fdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:37.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.812+0000 7effb5e00700 1 -- 172.21.15.1:0/3378465751 shutdown_connections 2024-02-02T15:42:37.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.812+0000 7effb5e00700 1 -- 172.21.15.1:0/3378465751 wait complete. 2024-02-02T15:42:37.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.815+0000 7effb5e00700 1 Processor -- start 2024-02-02T15:42:37.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.815+0000 7effb5e00700 1 -- start start 2024-02-02T15:42:37.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.816+0000 7effb5e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7effb0101e80 0x7effb0109a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:37.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.816+0000 7effb5e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7effb01027a0 0x7effb0109f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:37.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.816+0000 7effb5e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7effb010b2a0 0x7effb01a1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:37.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.816+0000 7effb5e00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effb01a2390 con 0x7effb0101e80 2024-02-02T15:42:37.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.816+0000 7effb5e00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effb01a2500 con 0x7effb010b2a0 2024-02-02T15:42:37.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.816+0000 7effb5e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effb01a26c0 con 0x7effb01027a0 2024-02-02T15:42:37.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.817+0000 7effaf400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7effb010b2a0 0x7effb01a1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:37.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.817+0000 7effaf400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7effb010b2a0 0x7effb01a1d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:43644/0 (socket says 172.21.15.1:43644) 2024-02-02T15:42:37.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.817+0000 7effaf400700 1 -- 172.21.15.1:0/2142289663 learned_addr learned my addr 172.21.15.1:0/2142289663 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:42:37.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.817+0000 7effaea00700 1 --2- 172.21.15.1:0/2142289663 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7effb0101e80 0x7effb0109a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:37.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.817+0000 7effaf400700 1 -- 172.21.15.1:0/2142289663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7effb01027a0 msgr2=0x7effb0109f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:37.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.817+0000 7effaf400700 1 --2- 172.21.15.1:0/2142289663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7effb01027a0 0x7effb0109f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:37.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.817+0000 7effaf400700 1 -- 172.21.15.1:0/2142289663 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7effb0101e80 msgr2=0x7effb0109a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:37.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.817+0000 7effaf400700 1 --2- 172.21.15.1:0/2142289663 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7effb0101e80 0x7effb0109a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:37.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.817+0000 7effaf400700 1 -- 172.21.15.1:0/2142289663 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effa00095f0 con 0x7effb010b2a0 2024-02-02T15:42:37.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.817+0000 7effaf400700 1 --2- 172.21.15.1:0/2142289663 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7effb010b2a0 0x7effb01a1d50 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7effa800b970 tx=0x7effa800bd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:37.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.818+0000 7effb4a00700 1 -- 172.21.15.1:0/2142289663 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa800c9b0 con 0x7effb010b2a0 2024-02-02T15:42:37.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.818+0000 7effb4a00700 1 -- 172.21.15.1:0/2142289663 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7effa8014440 con 0x7effb010b2a0 2024-02-02T15:42:37.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.818+0000 7effb4a00700 1 -- 172.21.15.1:0/2142289663 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa80075d0 con 0x7effb010b2a0 2024-02-02T15:42:37.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.818+0000 7effb5e00700 1 -- 172.21.15.1:0/2142289663 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effb01a29a0 con 0x7effb010b2a0 2024-02-02T15:42:37.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.818+0000 7effb5e00700 1 -- 172.21.15.1:0/2142289663 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effb01aa1e0 con 0x7effb010b2a0 2024-02-02T15:42:37.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.822+0000 7effb4a00700 1 -- 172.21.15.1:0/2142289663 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7effa80145b0 con 0x7effb010b2a0 2024-02-02T15:42:37.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.823+0000 7effb4a00700 1 --2- 172.21.15.1:0/2142289663 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7eff900a0710 0x7eff900a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:37.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.824+0000 7effb4a00700 1 -- 172.21.15.1:0/2142289663 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9453+0+0 (secure 0 0 0) 0x7effa8006e40 con 0x7effb010b2a0 2024-02-02T15:42:37.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.824+0000 7effc98e1b80 1 Processor -- start 2024-02-02T15:42:37.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.824+0000 7effaea00700 1 --2- 172.21.15.1:0/2142289663 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7eff900a0710 0x7eff900a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:37.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.824+0000 7effc98e1b80 1 -- start start 2024-02-02T15:42:37.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.825+0000 7effc98e1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557701b2f010 0x557701b2f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:37.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.825+0000 7effc98e1b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557701b2f930 0x557701b3a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:37.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.825+0000 7effc98e1b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557701b3a7e0 0x557701b3cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:37.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.825+0000 7effc98e1b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x557701aa34c0 con 0x557701b2f930 2024-02-02T15:42:37.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.825+0000 7effc98e1b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x557701a87aa0 con 0x557701b3a7e0 2024-02-02T15:42:37.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.825+0000 7effc98e1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557701a85460 con 0x557701b2f010 2024-02-02T15:42:37.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.825+0000 7effae000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557701b2f930 0x557701b3a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:37.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.825+0000 7effae000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557701b2f930 0x557701b3a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53040/0 (socket says 172.21.15.1:53040) 2024-02-02T15:42:37.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.825+0000 7effaf400700 1 --2- 172.21.15.1:0/1863295178 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557701b3a7e0 0x557701b3cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:37.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.825+0000 7effaea00700 1 --2- 172.21.15.1:0/1863295178 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557701b2f010 0x557701b2f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:37.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.825+0000 7effae000700 1 -- 172.21.15.1:0/1863295178 learned_addr learned my addr 172.21.15.1:0/1863295178 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:42:37.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.826+0000 7effaea00700 1 --2- 172.21.15.1:0/2142289663 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7eff900a0710 0x7eff900a2bd0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7effa000b5c0 tx=0x7effa0005fb0 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:37.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.826+0000 7effaf400700 1 -- 172.21.15.1:0/1863295178 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557701b2f010 msgr2=0x557701b2f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:37.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.826+0000 7effaf400700 1 --2- 172.21.15.1:0/1863295178 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557701b2f010 0x557701b2f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:37.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.826+0000 7effaf400700 1 -- 172.21.15.1:0/1863295178 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557701b2f930 msgr2=0x557701b3a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:37.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.826+0000 7effaf400700 1 --2- 172.21.15.1:0/1863295178 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557701b2f930 0x557701b3a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:37.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.826+0000 7effaf400700 1 -- 172.21.15.1:0/1863295178 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5577019023b0 con 0x557701b3a7e0 2024-02-02T15:42:37.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.826+0000 7effaf400700 1 --2- 172.21.15.1:0/1863295178 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557701b3a7e0 0x557701b3cbd0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7effa80c8ba0 tx=0x7effa80985f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b71d7a850450504 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:37.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.826+0000 7eff9fe00700 1 -- 172.21.15.1:0/1863295178 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa80b8070 con 0x557701b3a7e0 2024-02-02T15:42:37.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.827+0000 7eff9fe00700 1 -- 172.21.15.1:0/1863295178 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7effa809c7d0 con 0x557701b3a7e0 2024-02-02T15:42:37.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.827+0000 7eff9fe00700 1 -- 172.21.15.1:0/1863295178 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa8098e60 con 0x557701b3a7e0 2024-02-02T15:42:37.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.828+0000 7effc98e1b80 1 -- 172.21.15.1:0/1863295178 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557701b3a7e0 msgr2=0x557701b3cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:37.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.828+0000 7effc98e1b80 1 --2- 172.21.15.1:0/1863295178 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557701b3a7e0 0x557701b3cbd0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7effa80c8ba0 tx=0x7effa80985f0 comp rx=0 tx=0).stop 2024-02-02T15:42:37.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.828+0000 7effc98e1b80 1 -- 172.21.15.1:0/1863295178 shutdown_connections 2024-02-02T15:42:37.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.828+0000 7effc98e1b80 1 --2- 172.21.15.1:0/1863295178 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557701b3a7e0 0x557701b3cbd0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:37.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.828+0000 7effc98e1b80 1 --2- 172.21.15.1:0/1863295178 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557701b2f010 0x557701b2f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:37.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.828+0000 7effc98e1b80 1 --2- 172.21.15.1:0/1863295178 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557701b2f930 0x557701b3a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:37.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.828+0000 7effc98e1b80 1 -- 172.21.15.1:0/1863295178 >> 172.21.15.1:0/1863295178 conn(0x557701a8edb0 msgr2=0x557701b2b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:37.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.828+0000 7effc98e1b80 1 -- 172.21.15.1:0/1863295178 shutdown_connections 2024-02-02T15:42:37.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.828+0000 7effc98e1b80 1 -- 172.21.15.1:0/1863295178 wait complete. 2024-02-02T15:42:37.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.829+0000 7effc98e1b80 1 -- start start 2024-02-02T15:42:37.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.829+0000 7effc98e1b80 1 Processor -- start 2024-02-02T15:42:37.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.830+0000 7effc98e1b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557701b2f010 0x557701b3a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:37.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.830+0000 7effc98e1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557701b2f930 0x557701b37430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:37.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.830+0000 7effc98e1b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557701b3a7e0 0x557701b37b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:37.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.830+0000 7effc98e1b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x557701a80d50 con 0x557701b3a7e0 2024-02-02T15:42:37.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.830+0000 7effc98e1b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x557701a1cd80 con 0x557701b2f010 2024-02-02T15:42:37.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.830+0000 7effc98e1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557701a1ca20 con 0x557701b2f930 2024-02-02T15:42:37.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.830+0000 7effc98e1b80 10 client.0 _ll_register_callbacks cb 0x557701a1e290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:42:37.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.831+0000 7effae000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557701b2f930 0x557701b37430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:37.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.831+0000 7effae000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557701b2f930 0x557701b37430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:38206/0 (socket says 172.21.15.1:38206) 2024-02-02T15:42:37.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.831+0000 7effae000700 1 -- 172.21.15.1:0/3448937126 learned_addr learned my addr 172.21.15.1:0/3448937126 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:42:37.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.831+0000 7effaea00700 1 --2- 172.21.15.1:0/3448937126 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557701b2f010 0x557701b3a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:37.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.831+0000 7effaf400700 1 --2- 172.21.15.1:0/3448937126 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557701b3a7e0 0x557701b37b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:37.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.831+0000 7effaea00700 1 -- 172.21.15.1:0/3448937126 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557701b2f930 msgr2=0x557701b37430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:37.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.831+0000 7effaea00700 1 --2- 172.21.15.1:0/3448937126 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557701b2f930 0x557701b37430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:37.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.831+0000 7effaea00700 1 -- 172.21.15.1:0/3448937126 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557701b3a7e0 msgr2=0x557701b37b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:37.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.831+0000 7effaea00700 1 --2- 172.21.15.1:0/3448937126 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557701b3a7e0 0x557701b37b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:37.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.831+0000 7effaea00700 1 -- 172.21.15.1:0/3448937126 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effb01aa1e0 con 0x557701b2f010 2024-02-02T15:42:37.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.831+0000 7effaf400700 1 --2- 172.21.15.1:0/3448937126 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557701b3a7e0 0x557701b37b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:42:37.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.832+0000 7effaea00700 1 --2- 172.21.15.1:0/3448937126 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557701b2f010 0x557701b3a220 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7effa0000c00 tx=0x7effa001a050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efcbdd39a326273a server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:37.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.832+0000 7eff9cc00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:42:37.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.832+0000 7eff9cc00700 1 -- 172.21.15.1:0/3448937126 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa002b040 con 0x557701b2f010 2024-02-02T15:42:37.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.832+0000 7eff9cc00700 1 -- 172.21.15.1:0/3448937126 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9453+0+0 (secure 0 0 0) 0x7effa0026de0 con 0x557701b2f010 2024-02-02T15:42:37.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.832+0000 7effc98e1b80 1 -- 172.21.15.1:0/3448937126 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557701b230f0 con 0x557701b2f010 2024-02-02T15:42:37.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.833+0000 7eff9cc00700 1 -- 172.21.15.1:0/3448937126 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7effa000f460 con 0x557701b2f010 2024-02-02T15:42:37.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.833+0000 7effc98e1b80 10 client.29702 fetch_fsmap learned FSMap version 63 2024-02-02T15:42:37.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.833+0000 7effc98e1b80 1 -- 172.21.15.1:0/3448937126 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) v3 -- 0x5577018aeb60 con 0x557701b2f010 2024-02-02T15:42:37.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.833+0000 7eff9cc00700 1 -- 172.21.15.1:0/3448937126 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557701b230f0 con 0x557701b2f010 2024-02-02T15:42:37.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.833+0000 7eff9cc00700 1 -- 172.21.15.1:0/3448937126 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 63) v1 ==== 7526+0+0 (secure 0 0 0) 0x7effa003a6b0 con 0x557701b2f010 2024-02-02T15:42:37.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.833+0000 7effc98e1b80 10 client.29702 fetch_fsmap finished waiting for FSMap version 63 2024-02-02T15:42:37.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.833+0000 7effc98e1b80 10 client.29702 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-02-02T15:42:37.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.833+0000 7effc98e1b80 20 client.29702 populate_metadata read hostname 'smithi001' 2024-02-02T15:42:37.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.833+0000 7effc98e1b80 1 --2- 172.21.15.1:0/3448937126 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x557701b4a250 0x557701b4c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:37.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.833+0000 7effc98e1b80 4 client.29702 mds_command: new command op to 24514 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:42:37.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.833+0000 7effc98e1b80 1 -- 172.21.15.1:0/3448937126 --> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557701a79dd0 con 0x557701b4a250 2024-02-02T15:42:37.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.834+0000 7effae000700 1 --2- 172.21.15.1:0/3448937126 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x557701b4a250 0x557701b4c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:37.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.837+0000 7effae000700 1 --2- 172.21.15.1:0/3448937126 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x557701b4a250 0x557701b4c670 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1bd151f9a774fe74 server_cookie=16d05399b3005192 in_seq=0 out_seq=0 2024-02-02T15:42:37.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.837+0000 7eff9cc00700 10 client.29702 ms_handle_connect on v2:172.21.15.142:6836/2870289019 2024-02-02T15:42:37.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.839+0000 7eff9cc00700 1 -- 172.21.15.1:0/3448937126 <== mds.3 v2:172.21.15.142:6836/2870289019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x557701a79dd0 con 0x557701b4a250 2024-02-02T15:42:37.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.839+0000 7eff9cc00700 10 client.29702 handle_command_reply: tid=0 2024-02-02T15:42:37.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.839+0000 7effc98e1b80 1 client.29702 shutdown 2024-02-02T15:42:37.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.839+0000 7effc98e1b80 1 -- 172.21.15.1:0/3448937126 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557701b2f010 msgr2=0x557701b3a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:37.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.839+0000 7effc98e1b80 1 --2- 172.21.15.1:0/3448937126 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557701b2f010 0x557701b3a220 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7effa0000c00 tx=0x7effa001a050 comp rx=0 tx=0).stop 2024-02-02T15:42:37.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.839+0000 7effc98e1b80 1 -- 172.21.15.1:0/3448937126 shutdown_connections 2024-02-02T15:42:37.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.839+0000 7effc98e1b80 1 --2- 172.21.15.1:0/3448937126 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x557701b4a250 0x557701b4c670 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:37.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.839+0000 7effc98e1b80 1 --2- 172.21.15.1:0/3448937126 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557701b3a7e0 0x557701b37b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:37.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.839+0000 7effc98e1b80 1 --2- 172.21.15.1:0/3448937126 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557701b2f010 0x557701b3a220 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:37.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.839+0000 7eff9cc00700 0 client.29702 ms_handle_reset on v2:172.21.15.142:6836/2870289019 2024-02-02T15:42:37.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.839+0000 7effc98e1b80 1 --2- 172.21.15.1:0/3448937126 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557701b2f930 0x557701b37430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:37.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.839+0000 7effc98e1b80 1 -- 172.21.15.1:0/3448937126 >> 172.21.15.1:0/3448937126 conn(0x557701a8edb0 msgr2=0x557701b2afa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:37.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.839+0000 7effc98e1b80 1 -- 172.21.15.1:0/3448937126 shutdown_connections 2024-02-02T15:42:37.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.840+0000 7effc98e1b80 1 -- 172.21.15.1:0/3448937126 wait complete. 2024-02-02T15:42:37.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.840+0000 7effc98e1b80 20 client.29702 trim_cache size 0 max 16384 2024-02-02T15:42:37.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.858+0000 7effc98e1b80 1 Processor -- start 2024-02-02T15:42:37.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.858+0000 7effc98e1b80 1 -- start start 2024-02-02T15:42:37.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.858+0000 7effc98e1b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557701b2f930 0x557701b2f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:37.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.858+0000 7effc98e1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557701b3a7e0 0x557701b30830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:37.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.858+0000 7effc98e1b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557701b2b5d0 0x557701b39d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:37.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.858+0000 7effc98e1b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577019046a0 con 0x557701b2b5d0 2024-02-02T15:42:37.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.858+0000 7effc98e1b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x557701b25500 con 0x557701b2f930 2024-02-02T15:42:37.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.858+0000 7effc98e1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557701b388c0 con 0x557701b3a7e0 2024-02-02T15:42:37.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.859+0000 7effaea00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557701b2f930 0x557701b2f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:37.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.859+0000 7effaea00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557701b2f930 0x557701b2f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:43678/0 (socket says 172.21.15.1:43678) 2024-02-02T15:42:37.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.859+0000 7effaea00700 1 -- 172.21.15.1:0/4268257069 learned_addr learned my addr 172.21.15.1:0/4268257069 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:42:37.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.859+0000 7effaf400700 1 --2- 172.21.15.1:0/4268257069 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557701b2b5d0 0x557701b39d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:37.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.860+0000 7effaea00700 1 -- 172.21.15.1:0/4268257069 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557701b3a7e0 msgr2=0x557701b30830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:37.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.859+0000 7effae000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557701b3a7e0 0x557701b30830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:37.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.860+0000 7effaea00700 1 --2- 172.21.15.1:0/4268257069 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557701b3a7e0 0x557701b30830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:37.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.860+0000 7effaea00700 1 -- 172.21.15.1:0/4268257069 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557701b2b5d0 msgr2=0x557701b39d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:37.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.860+0000 7effaea00700 1 --2- 172.21.15.1:0/4268257069 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557701b2b5d0 0x557701b39d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:37.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.860+0000 7effaea00700 1 -- 172.21.15.1:0/4268257069 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557701a46ea0 con 0x557701b2f930 2024-02-02T15:42:37.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.860+0000 7effae000700 1 --2- 172.21.15.1:0/4268257069 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557701b3a7e0 0x557701b30830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:42:37.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.860+0000 7effaea00700 1 --2- 172.21.15.1:0/4268257069 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557701b2f930 0x557701b2f3f0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7effa00260a0 tx=0x7effa0009900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79a7ef0699dd720f server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:37.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.860+0000 7effa5200700 1 -- 172.21.15.1:0/4268257069 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa002b040 con 0x557701b2f930 2024-02-02T15:42:37.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.860+0000 7effa5200700 1 -- 172.21.15.1:0/4268257069 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7effa001e040 con 0x557701b2f930 2024-02-02T15:42:37.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.860+0000 7effa5200700 1 -- 172.21.15.1:0/4268257069 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa000f920 con 0x557701b2f930 2024-02-02T15:42:37.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.861+0000 7effc98e1b80 1 -- 172.21.15.1:0/4268257069 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557701b2f930 msgr2=0x557701b2f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:37.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.861+0000 7effc98e1b80 1 --2- 172.21.15.1:0/4268257069 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557701b2f930 0x557701b2f3f0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7effa00260a0 tx=0x7effa0009900 comp rx=0 tx=0).stop 2024-02-02T15:42:37.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.863+0000 7effc98e1b80 1 -- 172.21.15.1:0/4268257069 shutdown_connections 2024-02-02T15:42:37.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.863+0000 7effc98e1b80 1 --2- 172.21.15.1:0/4268257069 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557701b2b5d0 0x557701b39d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:37.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.863+0000 7effc98e1b80 1 --2- 172.21.15.1:0/4268257069 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557701b2f930 0x557701b2f3f0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:37.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.863+0000 7effc98e1b80 1 --2- 172.21.15.1:0/4268257069 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557701b3a7e0 0x557701b30830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:37.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.863+0000 7effc98e1b80 1 -- 172.21.15.1:0/4268257069 >> 172.21.15.1:0/4268257069 conn(0x557701a8edb0 msgr2=0x557701b46580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:37.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.863+0000 7effc98e1b80 1 -- 172.21.15.1:0/4268257069 shutdown_connections 2024-02-02T15:42:37.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.863+0000 7effc98e1b80 1 -- 172.21.15.1:0/4268257069 wait complete. 2024-02-02T15:42:37.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.864+0000 7effc98e1b80 1 -- start start 2024-02-02T15:42:37.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.864+0000 7effc98e1b80 1 Processor -- start 2024-02-02T15:42:37.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.865+0000 7effc98e1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557701b2b5d0 0x557701b3e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:37.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.865+0000 7effc98e1b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557701b2f930 0x557701b3ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:37.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.865+0000 7effc98e1b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557701b3a7e0 0x557701b3f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:37.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.865+0000 7effc98e1b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x557701b9ad70 con 0x557701b2f930 2024-02-02T15:42:37.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.865+0000 7effc98e1b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x557701b9aeb0 con 0x557701b3a7e0 2024-02-02T15:42:37.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.865+0000 7effc98e1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557701b9b040 con 0x557701b2b5d0 2024-02-02T15:42:37.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.865+0000 7effc98e1b80 10 client.0 _ll_register_callbacks cb 0x557701a1e290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:42:37.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.865+0000 7effaf400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557701b3a7e0 0x557701b3f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:37.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.865+0000 7effaf400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557701b3a7e0 0x557701b3f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:43692/0 (socket says 172.21.15.1:43692) 2024-02-02T15:42:37.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.865+0000 7effaf400700 1 -- 172.21.15.1:0/3266998292 learned_addr learned my addr 172.21.15.1:0/3266998292 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:42:37.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.865+0000 7effaea00700 1 --2- 172.21.15.1:0/3266998292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557701b2b5d0 0x557701b3e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:37.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.865+0000 7effae000700 1 --2- 172.21.15.1:0/3266998292 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557701b2f930 0x557701b3ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:37.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.866+0000 7effaf400700 1 -- 172.21.15.1:0/3266998292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557701b2b5d0 msgr2=0x557701b3e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:37.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.866+0000 7effaf400700 1 --2- 172.21.15.1:0/3266998292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557701b2b5d0 0x557701b3e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:37.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.866+0000 7effaf400700 1 -- 172.21.15.1:0/3266998292 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557701b2f930 msgr2=0x557701b3ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:37.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.866+0000 7effaf400700 1 --2- 172.21.15.1:0/3266998292 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557701b2f930 0x557701b3ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:37.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.866+0000 7effaf400700 1 -- 172.21.15.1:0/3266998292 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5577018aeb60 con 0x557701b3a7e0 2024-02-02T15:42:37.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.866+0000 7effaf400700 1 --2- 172.21.15.1:0/3266998292 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557701b3a7e0 0x557701b3f2a0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7effa80c9040 tx=0x7effa8098b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16fe38a03c76d922 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:37.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.866+0000 7eff9cc00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:42:37.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.866+0000 7eff9cc00700 1 -- 172.21.15.1:0/3266998292 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa80a0a10 con 0x557701b3a7e0 2024-02-02T15:42:37.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.866+0000 7eff9cc00700 1 -- 172.21.15.1:0/3266998292 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9453+0+0 (secure 0 0 0) 0x7effa80980a0 con 0x557701b3a7e0 2024-02-02T15:42:37.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.866+0000 7effc98e1b80 1 -- 172.21.15.1:0/3266998292 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7effa809c7d0 con 0x557701b3a7e0 2024-02-02T15:42:37.867 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:42:37 smithi071 ceph-mon[44707]: pgmap v489: 129 pgs: 129 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 8.0 MiB/s wr, 586 op/s 2024-02-02T15:42:37.867 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:42:37 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/3572776606' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:42:37.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.866+0000 7eff9cc00700 1 -- 172.21.15.1:0/3266998292 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7effa8098c00 con 0x557701b3a7e0 2024-02-02T15:42:37.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.867+0000 7effc98e1b80 10 client.29708 fetch_fsmap learned FSMap version 63 2024-02-02T15:42:37.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.867+0000 7effc98e1b80 1 -- 172.21.15.1:0/3266998292 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) v3 -- 0x557701906cd0 con 0x557701b3a7e0 2024-02-02T15:42:37.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.867+0000 7eff9cc00700 1 -- 172.21.15.1:0/3266998292 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7effa809c7d0 con 0x557701b3a7e0 2024-02-02T15:42:37.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.868+0000 7eff9cc00700 1 -- 172.21.15.1:0/3266998292 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 63) v1 ==== 7526+0+0 (secure 0 0 0) 0x7effa807f070 con 0x557701b3a7e0 2024-02-02T15:42:37.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.868+0000 7effc98e1b80 10 client.29708 fetch_fsmap finished waiting for FSMap version 63 2024-02-02T15:42:37.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.868+0000 7effc98e1b80 10 client.29708 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-02-02T15:42:37.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.868+0000 7effc98e1b80 20 client.29708 populate_metadata read hostname 'smithi001' 2024-02-02T15:42:37.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.868+0000 7effc98e1b80 1 --2- 172.21.15.1:0/3266998292 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x557701b9c430 0x557701b9e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:37.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.868+0000 7effc98e1b80 4 client.29708 mds_command: new command op to 24514 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:42:37.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.868+0000 7effc98e1b80 1 -- 172.21.15.1:0/3266998292 --> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557701a3ff50 con 0x557701b9c430 2024-02-02T15:42:37.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.868+0000 7effaea00700 1 --2- 172.21.15.1:0/3266998292 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x557701b9c430 0x557701b9e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:37.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.870+0000 7effaea00700 1 --2- 172.21.15.1:0/3266998292 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x557701b9c430 0x557701b9e820 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8b9d16813deedf0a server_cookie=d847eecd54019e3e in_seq=0 out_seq=0 2024-02-02T15:42:37.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.870+0000 7eff9cc00700 10 client.29708 ms_handle_connect on v2:172.21.15.142:6836/2870289019 2024-02-02T15:42:37.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.871+0000 7eff9cc00700 1 -- 172.21.15.1:0/3266998292 <== mds.3 v2:172.21.15.142:6836/2870289019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557701a3ff50 con 0x557701b9c430 2024-02-02T15:42:37.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.871+0000 7eff9cc00700 10 client.29708 handle_command_reply: tid=0 2024-02-02T15:42:37.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.871+0000 7effc98e1b80 1 client.29708 shutdown 2024-02-02T15:42:37.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.871+0000 7effc98e1b80 1 -- 172.21.15.1:0/3266998292 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557701b3a7e0 msgr2=0x557701b3f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:37.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.871+0000 7effc98e1b80 1 --2- 172.21.15.1:0/3266998292 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557701b3a7e0 0x557701b3f2a0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7effa80c9040 tx=0x7effa8098b20 comp rx=0 tx=0).stop 2024-02-02T15:42:37.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.872+0000 7effc98e1b80 1 -- 172.21.15.1:0/3266998292 shutdown_connections 2024-02-02T15:42:37.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.872+0000 7effc98e1b80 1 --2- 172.21.15.1:0/3266998292 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557701b3a7e0 0x557701b3f2a0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:37.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.872+0000 7effc98e1b80 1 --2- 172.21.15.1:0/3266998292 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557701b2b5d0 0x557701b3e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:37.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.872+0000 7effc98e1b80 1 --2- 172.21.15.1:0/3266998292 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x557701b9c430 0x557701b9e820 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:37.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.872+0000 7effc98e1b80 1 --2- 172.21.15.1:0/3266998292 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557701b2f930 0x557701b3ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:37.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.872+0000 7effc98e1b80 1 -- 172.21.15.1:0/3266998292 >> 172.21.15.1:0/3266998292 conn(0x557701a8edb0 msgr2=0x557701b45900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:37.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.872+0000 7eff9cc00700 0 client.29708 ms_handle_reset on v2:172.21.15.142:6836/2870289019 2024-02-02T15:42:37.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.872+0000 7effc98e1b80 1 -- 172.21.15.1:0/3266998292 shutdown_connections 2024-02-02T15:42:37.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.872+0000 7effc98e1b80 1 -- 172.21.15.1:0/3266998292 wait complete. 2024-02-02T15:42:37.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.872+0000 7effc98e1b80 20 client.29708 trim_cache size 0 max 16384 2024-02-02T15:42:37.872 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:42:37.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.873+0000 7effb5e00700 1 -- 172.21.15.1:0/2142289663 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7eff900a0710 msgr2=0x7eff900a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:37.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.873+0000 7effb5e00700 1 --2- 172.21.15.1:0/2142289663 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7eff900a0710 0x7eff900a2bd0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7effa000b5c0 tx=0x7effa0005fb0 comp rx=0 tx=0).stop 2024-02-02T15:42:37.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.873+0000 7effb5e00700 1 -- 172.21.15.1:0/2142289663 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7effb010b2a0 msgr2=0x7effb01a1d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:37.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.873+0000 7effb5e00700 1 --2- 172.21.15.1:0/2142289663 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7effb010b2a0 0x7effb01a1d50 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7effa800b970 tx=0x7effa800bd30 comp rx=0 tx=0).stop 2024-02-02T15:42:37.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.873+0000 7effb5e00700 1 -- 172.21.15.1:0/2142289663 shutdown_connections 2024-02-02T15:42:37.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.873+0000 7effb5e00700 1 --2- 172.21.15.1:0/2142289663 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7eff900a0710 0x7eff900a2bd0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:37.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.873+0000 7effb5e00700 1 --2- 172.21.15.1:0/2142289663 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7effb010b2a0 0x7effb01a1d50 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:37.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.873+0000 7effb5e00700 1 --2- 172.21.15.1:0/2142289663 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7effb0101e80 0x7effb0109a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:37.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.873+0000 7effb5e00700 1 --2- 172.21.15.1:0/2142289663 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7effb01027a0 0x7effb0109f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:37.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.873+0000 7effb5e00700 1 -- 172.21.15.1:0/2142289663 >> 172.21.15.1:0/2142289663 conn(0x7effb00fb6a0 msgr2=0x7effb00ff9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:37.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.873+0000 7effb5e00700 1 -- 172.21.15.1:0/2142289663 shutdown_connections 2024-02-02T15:42:37.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:37.873+0000 7effb5e00700 1 -- 172.21.15.1:0/2142289663 wait complete. 2024-02-02T15:42:37.895 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:42:37 smithi001 ceph-mon[35470]: pgmap v489: 129 pgs: 129 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 8.0 MiB/s wr, 586 op/s 2024-02-02T15:42:37.896 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:42:37 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/3572776606' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:42:37.898 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-02T15:42:37.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:42:37 smithi142 ceph-mon[45728]: pgmap v489: 129 pgs: 129 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 8.0 MiB/s wr, 586 op/s 2024-02-02T15:42:37.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:42:37 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/3572776606' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:42:37.983 INFO:tasks.workunit.client.0.smithi001.stdout: 10 76660 36.78 MB/sec execute 10 sec latency 70.620 ms 2024-02-02T15:42:38.189 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.190+0000 7fa837a00700 1 -- 172.21.15.1:0/975984671 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa830073a30 msgr2=0x7fa83010d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:38.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.190+0000 7fa837a00700 1 --2- 172.21.15.1:0/975984671 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa830073a30 0x7fa83010d0f0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fa8180098f0 tx=0x7fa818009c00 comp rx=0 tx=0).stop 2024-02-02T15:42:38.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.190+0000 7fa837a00700 1 -- 172.21.15.1:0/975984671 shutdown_connections 2024-02-02T15:42:38.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.190+0000 7fa837a00700 1 --2- 172.21.15.1:0/975984671 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa83010d630 0x7fa83010fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:38.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.190+0000 7fa837a00700 1 --2- 172.21.15.1:0/975984671 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa830073110 0x7fa8300734f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:38.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.190+0000 7fa837a00700 1 --2- 172.21.15.1:0/975984671 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa830073a30 0x7fa83010d0f0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:38.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.190+0000 7fa837a00700 1 -- 172.21.15.1:0/975984671 >> 172.21.15.1:0/975984671 conn(0x7fa8300fc840 msgr2=0x7fa8300fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:38.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.191+0000 7fa837a00700 1 -- 172.21.15.1:0/975984671 shutdown_connections 2024-02-02T15:42:38.190 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.191+0000 7fa837a00700 1 -- 172.21.15.1:0/975984671 wait complete. 2024-02-02T15:42:38.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.193+0000 7fa837a00700 1 Processor -- start 2024-02-02T15:42:38.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.193+0000 7fa837a00700 1 -- start start 2024-02-02T15:42:38.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.194+0000 7fa837a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa830073110 0x7fa830106290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:38.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.194+0000 7fa837a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa830073a30 0x7fa8301067f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:38.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.194+0000 7fa837a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa83010d630 0x7fa830101350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:38.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.194+0000 7fa837a00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa830101b20 con 0x7fa83010d630 2024-02-02T15:42:38.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.194+0000 7fa837a00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa830101c90 con 0x7fa830073110 2024-02-02T15:42:38.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.194+0000 7fa837a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa830101e50 con 0x7fa830073a30 2024-02-02T15:42:38.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.194+0000 7fa835000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa830073110 0x7fa830106290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:38.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.195+0000 7fa835a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa83010d630 0x7fa830101350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:38.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.195+0000 7fa82fe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa830073a30 0x7fa8301067f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:38.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.195+0000 7fa835a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa83010d630 0x7fa830101350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53090/0 (socket says 172.21.15.1:53090) 2024-02-02T15:42:38.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.195+0000 7fa835a00700 1 -- 172.21.15.1:0/595191970 learned_addr learned my addr 172.21.15.1:0/595191970 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:42:38.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.198+0000 7fa835a00700 1 -- 172.21.15.1:0/595191970 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa830073a30 msgr2=0x7fa8301067f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:38.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.198+0000 7fa835a00700 1 --2- 172.21.15.1:0/595191970 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa830073a30 0x7fa8301067f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:38.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.198+0000 7fa835a00700 1 -- 172.21.15.1:0/595191970 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa830073110 msgr2=0x7fa830106290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:38.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.198+0000 7fa835a00700 1 --2- 172.21.15.1:0/595191970 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa830073110 0x7fa830106290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:38.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.198+0000 7fa835a00700 1 -- 172.21.15.1:0/595191970 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8200095f0 con 0x7fa83010d630 2024-02-02T15:42:38.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.198+0000 7fa835a00700 1 --2- 172.21.15.1:0/595191970 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa83010d630 0x7fa830101350 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fa82800b9e0 tx=0x7fa82800bcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:38.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.199+0000 7fa82d600700 1 -- 172.21.15.1:0/595191970 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82800c930 con 0x7fa83010d630 2024-02-02T15:42:38.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.199+0000 7fa82d600700 1 -- 172.21.15.1:0/595191970 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa828014440 con 0x7fa83010d630 2024-02-02T15:42:38.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.199+0000 7fa82d600700 1 -- 172.21.15.1:0/595191970 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8280125a0 con 0x7fa83010d630 2024-02-02T15:42:38.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.199+0000 7fa837a00700 1 -- 172.21.15.1:0/595191970 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa818009520 con 0x7fa83010d630 2024-02-02T15:42:38.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.200+0000 7fa837a00700 1 -- 172.21.15.1:0/595191970 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa83010c1f0 con 0x7fa83010d630 2024-02-02T15:42:38.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.211+0000 7fa82d600700 1 -- 172.21.15.1:0/595191970 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7fa8280145b0 con 0x7fa83010d630 2024-02-02T15:42:38.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.213+0000 7fa82d600700 1 --2- 172.21.15.1:0/595191970 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fa8100a0690 0x7fa8100a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:38.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.213+0000 7fa82d600700 1 -- 172.21.15.1:0/595191970 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fa828006e40 con 0x7fa83010d630 2024-02-02T15:42:38.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.213+0000 7fa835000700 1 --2- 172.21.15.1:0/595191970 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fa8100a0690 0x7fa8100a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:38.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.215+0000 7fa84b50db80 1 Processor -- start 2024-02-02T15:42:38.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.215+0000 7fa835000700 1 --2- 172.21.15.1:0/595191970 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fa8100a0690 0x7fa8100a2b50 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa820005fd0 tx=0x7fa820009310 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:38.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.215+0000 7fa84b50db80 1 -- start start 2024-02-02T15:42:38.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.216+0000 7fa84b50db80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56006581a010 0x56006581a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:38.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.216+0000 7fa84b50db80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56006581a930 0x5600658252a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:38.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.216+0000 7fa84b50db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5600658257e0 0x560065827bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:38.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.217+0000 7fa84b50db80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56006578e4c0 con 0x56006581a010 2024-02-02T15:42:38.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.217+0000 7fa84b50db80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x560065772aa0 con 0x56006581a930 2024-02-02T15:42:38.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.217+0000 7fa84b50db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560065770460 con 0x5600658257e0 2024-02-02T15:42:38.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.217+0000 7fa835a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5600658257e0 0x560065827bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:38.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.217+0000 7fa835a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5600658257e0 0x560065827bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:38258/0 (socket says 172.21.15.1:38258) 2024-02-02T15:42:38.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.217+0000 7fa835a00700 1 -- 172.21.15.1:0/2328087590 learned_addr learned my addr 172.21.15.1:0/2328087590 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:42:38.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.217+0000 7fa82fe00700 1 --2- 172.21.15.1:0/2328087590 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56006581a930 0x5600658252a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:38.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.217+0000 7fa835000700 1 --2- 172.21.15.1:0/2328087590 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56006581a010 0x56006581a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:38.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.218+0000 7fa835000700 1 -- 172.21.15.1:0/2328087590 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5600658257e0 msgr2=0x560065827bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:38.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.218+0000 7fa835000700 1 --2- 172.21.15.1:0/2328087590 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5600658257e0 0x560065827bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:38.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.218+0000 7fa835000700 1 -- 172.21.15.1:0/2328087590 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56006581a930 msgr2=0x5600658252a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:38.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.218+0000 7fa835000700 1 --2- 172.21.15.1:0/2328087590 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56006581a930 0x5600658252a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:38.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.218+0000 7fa835000700 1 -- 172.21.15.1:0/2328087590 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5600655ed3b0 con 0x56006581a010 2024-02-02T15:42:38.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.219+0000 7fa835000700 1 --2- 172.21.15.1:0/2328087590 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56006581a010 0x56006581a3f0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fa820012000 tx=0x7fa820014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5291068594438e61 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:38.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.220+0000 7fa826000700 1 -- 172.21.15.1:0/2328087590 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82001f620 con 0x56006581a010 2024-02-02T15:42:38.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.220+0000 7fa826000700 1 -- 172.21.15.1:0/2328087590 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa820016070 con 0x56006581a010 2024-02-02T15:42:38.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.220+0000 7fa826000700 1 -- 172.21.15.1:0/2328087590 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82001daf0 con 0x56006581a010 2024-02-02T15:42:38.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.222+0000 7fa84b50db80 1 -- 172.21.15.1:0/2328087590 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56006581a010 msgr2=0x56006581a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:38.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.222+0000 7fa84b50db80 1 --2- 172.21.15.1:0/2328087590 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56006581a010 0x56006581a3f0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fa820012000 tx=0x7fa820014000 comp rx=0 tx=0).stop 2024-02-02T15:42:38.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.223+0000 7fa84b50db80 1 -- 172.21.15.1:0/2328087590 shutdown_connections 2024-02-02T15:42:38.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.223+0000 7fa84b50db80 1 --2- 172.21.15.1:0/2328087590 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5600658257e0 0x560065827bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:38.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.223+0000 7fa84b50db80 1 --2- 172.21.15.1:0/2328087590 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56006581a010 0x56006581a3f0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:38.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.223+0000 7fa84b50db80 1 --2- 172.21.15.1:0/2328087590 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56006581a930 0x5600658252a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:38.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.223+0000 7fa84b50db80 1 -- 172.21.15.1:0/2328087590 >> 172.21.15.1:0/2328087590 conn(0x560065779db0 msgr2=0x5600658163b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:38.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.224+0000 7fa84b50db80 1 -- 172.21.15.1:0/2328087590 shutdown_connections 2024-02-02T15:42:38.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.224+0000 7fa84b50db80 1 -- 172.21.15.1:0/2328087590 wait complete. 2024-02-02T15:42:38.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.225+0000 7fa84b50db80 1 -- start start 2024-02-02T15:42:38.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.225+0000 7fa84b50db80 1 Processor -- start 2024-02-02T15:42:38.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.228+0000 7fa84b50db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56006581a010 0x5600658251f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:38.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.228+0000 7fa84b50db80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56006581a930 0x560065822470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:38.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.228+0000 7fa84b50db80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5600658257e0 0x560065822aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:38.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.229+0000 7fa835000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56006581a010 0x5600658251f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:38.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.229+0000 7fa82fe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56006581a930 0x560065822470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:38.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.229+0000 7fa82fe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56006581a930 0x560065822470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:43730/0 (socket says 172.21.15.1:43730) 2024-02-02T15:42:38.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.229+0000 7fa82fe00700 1 -- 172.21.15.1:0/2638290765 learned_addr learned my addr 172.21.15.1:0/2638290765 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:42:38.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.229+0000 7fa835a00700 1 --2- 172.21.15.1:0/2638290765 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5600658257e0 0x560065822aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:38.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.229+0000 7fa835000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56006581a010 0x5600658251f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:38272/0 (socket says 172.21.15.1:38272) 2024-02-02T15:42:38.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.230+0000 7fa84b50db80 1 -- 172.21.15.1:0/2638290765 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56006576bd50 con 0x5600658257e0 2024-02-02T15:42:38.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.230+0000 7fa84b50db80 1 -- 172.21.15.1:0/2638290765 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x560065707d80 con 0x56006581a930 2024-02-02T15:42:38.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.230+0000 7fa84b50db80 1 -- 172.21.15.1:0/2638290765 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560065707a20 con 0x56006581a010 2024-02-02T15:42:38.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.230+0000 7fa84b50db80 10 client.0 _ll_register_callbacks cb 0x560065709290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:42:38.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.230+0000 7fa82fe00700 1 -- 172.21.15.1:0/2638290765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56006581a010 msgr2=0x5600658251f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:38.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.231+0000 7fa82fe00700 1 --2- 172.21.15.1:0/2638290765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56006581a010 0x5600658251f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:38.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.231+0000 7fa82fe00700 1 -- 172.21.15.1:0/2638290765 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5600658257e0 msgr2=0x560065822aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:38.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.231+0000 7fa82fe00700 1 --2- 172.21.15.1:0/2638290765 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5600658257e0 0x560065822aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:38.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.231+0000 7fa82fe00700 1 -- 172.21.15.1:0/2638290765 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa820011ce0 con 0x56006581a930 2024-02-02T15:42:38.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.231+0000 7fa82fe00700 1 --2- 172.21.15.1:0/2638290765 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56006581a930 0x560065822470 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fa818000c00 tx=0x7fa81800b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e47443d98857a5d8 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:38.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.231+0000 7fa81ea00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:42:38.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.232+0000 7fa81ea00700 1 -- 172.21.15.1:0/2638290765 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa81801c070 con 0x56006581a930 2024-02-02T15:42:38.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.232+0000 7fa81ea00700 1 -- 172.21.15.1:0/2638290765 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fa81800bbc0 con 0x56006581a930 2024-02-02T15:42:38.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.232+0000 7fa84b50db80 1 -- 172.21.15.1:0/2638290765 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56006580e0f0 con 0x56006581a930 2024-02-02T15:42:38.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.233+0000 7fa81ea00700 1 -- 172.21.15.1:0/2638290765 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa81800f980 con 0x56006581a930 2024-02-02T15:42:38.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.233+0000 7fa84b50db80 10 client.29720 fetch_fsmap learned FSMap version 63 2024-02-02T15:42:38.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.233+0000 7fa84b50db80 1 -- 172.21.15.1:0/2638290765 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) v3 -- 0x560065599b60 con 0x56006581a930 2024-02-02T15:42:38.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.233+0000 7fa81ea00700 1 -- 172.21.15.1:0/2638290765 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56006580e0f0 con 0x56006581a930 2024-02-02T15:42:38.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.233+0000 7fa81ea00700 1 -- 172.21.15.1:0/2638290765 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 63) v1 ==== 7526+0+0 (secure 0 0 0) 0x7fa818015020 con 0x56006581a930 2024-02-02T15:42:38.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.234+0000 7fa84b50db80 10 client.29720 fetch_fsmap finished waiting for FSMap version 63 2024-02-02T15:42:38.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.234+0000 7fa84b50db80 10 client.29720 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-02-02T15:42:38.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.234+0000 7fa84b50db80 20 client.29720 populate_metadata read hostname 'smithi001' 2024-02-02T15:42:38.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.234+0000 7fa84b50db80 1 --2- 172.21.15.1:0/2638290765 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x560065835350 0x560065837770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:38.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.234+0000 7fa84b50db80 4 client.29720 mds_command: new command op to 24518 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:42:38.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.234+0000 7fa84b50db80 1 -- 172.21.15.1:0/2638290765 --> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560065764dd0 con 0x560065835350 2024-02-02T15:42:38.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.235+0000 7fa835000700 1 --2- 172.21.15.1:0/2638290765 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x560065835350 0x560065837770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:38.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.237+0000 7fa835000700 1 --2- 172.21.15.1:0/2638290765 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x560065835350 0x560065837770 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c1da235aa92920b server_cookie=1165b19191431c0e in_seq=0 out_seq=0 2024-02-02T15:42:38.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.237+0000 7fa81ea00700 10 client.29720 ms_handle_connect on v2:172.21.15.71:6836/628016199 2024-02-02T15:42:38.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.238+0000 7fa81ea00700 1 -- 172.21.15.1:0/2638290765 <== mds.2 v2:172.21.15.71:6836/628016199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x560065764dd0 con 0x560065835350 2024-02-02T15:42:38.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.238+0000 7fa81ea00700 10 client.29720 handle_command_reply: tid=0 2024-02-02T15:42:38.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.238+0000 7fa84b50db80 1 client.29720 shutdown 2024-02-02T15:42:38.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.239+0000 7fa84b50db80 1 -- 172.21.15.1:0/2638290765 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56006581a930 msgr2=0x560065822470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:38.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.239+0000 7fa84b50db80 1 --2- 172.21.15.1:0/2638290765 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56006581a930 0x560065822470 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fa818000c00 tx=0x7fa81800b790 comp rx=0 tx=0).stop 2024-02-02T15:42:38.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.239+0000 7fa84b50db80 1 -- 172.21.15.1:0/2638290765 shutdown_connections 2024-02-02T15:42:38.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.239+0000 7fa84b50db80 1 --2- 172.21.15.1:0/2638290765 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5600658257e0 0x560065822aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:38.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.239+0000 7fa84b50db80 1 --2- 172.21.15.1:0/2638290765 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x560065835350 0x560065837770 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:38.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.239+0000 7fa84b50db80 1 --2- 172.21.15.1:0/2638290765 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56006581a010 0x5600658251f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:38.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.239+0000 7fa81ea00700 0 client.29720 ms_handle_reset on v2:172.21.15.71:6836/628016199 2024-02-02T15:42:38.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.239+0000 7fa84b50db80 1 --2- 172.21.15.1:0/2638290765 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56006581a930 0x560065822470 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:38.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.239+0000 7fa84b50db80 1 -- 172.21.15.1:0/2638290765 >> 172.21.15.1:0/2638290765 conn(0x560065779db0 msgr2=0x560065815fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:38.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.241+0000 7fa84b50db80 1 -- 172.21.15.1:0/2638290765 shutdown_connections 2024-02-02T15:42:38.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.241+0000 7fa84b50db80 1 -- 172.21.15.1:0/2638290765 wait complete. 2024-02-02T15:42:38.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.243+0000 7fa84b50db80 20 client.29720 trim_cache size 0 max 16384 2024-02-02T15:42:38.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.277+0000 7fa84b50db80 1 Processor -- start 2024-02-02T15:42:38.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.278+0000 7fa84b50db80 1 -- start start 2024-02-02T15:42:38.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.278+0000 7fa84b50db80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56006581a010 0x560065837740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:38.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.278+0000 7fa84b50db80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56006581a930 0x5600658315a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:38.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.278+0000 7fa84b50db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5600658257e0 0x560065824d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:38.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.278+0000 7fa84b50db80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600655ef6a0 con 0x56006581a010 2024-02-02T15:42:38.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.278+0000 7fa84b50db80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56006581b8f0 con 0x56006581a930 2024-02-02T15:42:38.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.278+0000 7fa84b50db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560065820a50 con 0x5600658257e0 2024-02-02T15:42:38.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.279+0000 7fa82fe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56006581a930 0x5600658315a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:38.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.279+0000 7fa82fe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56006581a930 0x5600658315a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:43736/0 (socket says 172.21.15.1:43736) 2024-02-02T15:42:38.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.279+0000 7fa82fe00700 1 -- 172.21.15.1:0/4291474980 learned_addr learned my addr 172.21.15.1:0/4291474980 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:42:38.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.279+0000 7fa82fe00700 1 -- 172.21.15.1:0/4291474980 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5600658257e0 msgr2=0x560065824d20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-02T15:42:38.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.279+0000 7fa82fe00700 1 --2- 172.21.15.1:0/4291474980 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5600658257e0 0x560065824d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:38.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.279+0000 7fa82fe00700 1 -- 172.21.15.1:0/4291474980 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56006581a010 msgr2=0x560065837740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:38.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.279+0000 7fa82fe00700 1 --2- 172.21.15.1:0/4291474980 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56006581a010 0x560065837740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:38.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.279+0000 7fa82fe00700 1 -- 172.21.15.1:0/4291474980 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560065731ea0 con 0x56006581a930 2024-02-02T15:42:38.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.279+0000 7fa82fe00700 1 --2- 172.21.15.1:0/4291474980 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56006581a930 0x5600658315a0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fa818003eb0 tx=0x7fa818009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d5015b034415f4e server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:38.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.280+0000 7fa826a00700 1 -- 172.21.15.1:0/4291474980 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa81801c040 con 0x56006581a930 2024-02-02T15:42:38.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.280+0000 7fa826a00700 1 -- 172.21.15.1:0/4291474980 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa81801d040 con 0x56006581a930 2024-02-02T15:42:38.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.280+0000 7fa826a00700 1 -- 172.21.15.1:0/4291474980 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa818005110 con 0x56006581a930 2024-02-02T15:42:38.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.282+0000 7fa84b50db80 1 -- 172.21.15.1:0/4291474980 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56006581a930 msgr2=0x5600658315a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:38.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.282+0000 7fa84b50db80 1 --2- 172.21.15.1:0/4291474980 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56006581a930 0x5600658315a0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fa818003eb0 tx=0x7fa818009c00 comp rx=0 tx=0).stop 2024-02-02T15:42:38.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.282+0000 7fa84b50db80 1 -- 172.21.15.1:0/4291474980 shutdown_connections 2024-02-02T15:42:38.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.282+0000 7fa84b50db80 1 --2- 172.21.15.1:0/4291474980 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5600658257e0 0x560065824d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:38.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.282+0000 7fa84b50db80 1 --2- 172.21.15.1:0/4291474980 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56006581a010 0x560065837740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:38.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.282+0000 7fa84b50db80 1 --2- 172.21.15.1:0/4291474980 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56006581a930 0x5600658315a0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:38.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.282+0000 7fa84b50db80 1 -- 172.21.15.1:0/4291474980 >> 172.21.15.1:0/4291474980 conn(0x560065779db0 msgr2=0x560065811e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:38.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.283+0000 7fa84b50db80 1 -- 172.21.15.1:0/4291474980 shutdown_connections 2024-02-02T15:42:38.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.283+0000 7fa84b50db80 1 -- 172.21.15.1:0/4291474980 wait complete. 2024-02-02T15:42:38.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.283+0000 7fa84b50db80 1 -- start start 2024-02-02T15:42:38.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.284+0000 7fa84b50db80 1 Processor -- start 2024-02-02T15:42:38.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.284+0000 7fa84b50db80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56006581a010 0x56006582c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:38.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.284+0000 7fa84b50db80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56006581a930 0x56006582a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:38.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.284+0000 7fa84b50db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5600658257e0 0x56006582aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:38.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.284+0000 7fa84b50db80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56006582b110 con 0x56006581a930 2024-02-02T15:42:38.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.284+0000 7fa84b50db80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x560065885e70 con 0x56006581a010 2024-02-02T15:42:38.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.284+0000 7fa84b50db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560065885fb0 con 0x5600658257e0 2024-02-02T15:42:38.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.284+0000 7fa84b50db80 10 client.0 _ll_register_callbacks cb 0x560065709290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:42:38.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.285+0000 7fa835a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5600658257e0 0x56006582aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:38.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.285+0000 7fa835a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5600658257e0 0x56006582aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:38280/0 (socket says 172.21.15.1:38280) 2024-02-02T15:42:38.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.285+0000 7fa835a00700 1 -- 172.21.15.1:0/4053504039 learned_addr learned my addr 172.21.15.1:0/4053504039 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:42:38.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.285+0000 7fa835000700 1 --2- 172.21.15.1:0/4053504039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56006581a010 0x56006582c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:38.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.286+0000 7fa835000700 1 -- 172.21.15.1:0/4053504039 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5600658257e0 msgr2=0x56006582aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:38.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.286+0000 7fa835000700 1 --2- 172.21.15.1:0/4053504039 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5600658257e0 0x56006582aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:38.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.286+0000 7fa82fe00700 1 --2- 172.21.15.1:0/4053504039 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56006581a930 0x56006582a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:38.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.286+0000 7fa835000700 1 -- 172.21.15.1:0/4053504039 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56006581a930 msgr2=0x56006582a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:38.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.286+0000 7fa835000700 1 --2- 172.21.15.1:0/4053504039 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56006581a930 0x56006582a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:38.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.286+0000 7fa835000700 1 -- 172.21.15.1:0/4053504039 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560065599b60 con 0x56006581a010 2024-02-02T15:42:38.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.286+0000 7fa82fe00700 1 --2- 172.21.15.1:0/4053504039 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56006581a930 0x56006582a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:42:38.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.286+0000 7fa835000700 1 --2- 172.21.15.1:0/4053504039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56006581a010 0x56006582c330 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x560065838080 tx=0x7fa82001d510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b33d7c3846618a0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:38.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.286+0000 7fa81ea00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:42:38.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.288+0000 7fa81ea00700 1 -- 172.21.15.1:0/4053504039 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82002d040 con 0x56006581a010 2024-02-02T15:42:38.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.288+0000 7fa84b50db80 1 -- 172.21.15.1:0/4053504039 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa820016070 con 0x56006581a010 2024-02-02T15:42:38.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.288+0000 7fa81ea00700 1 -- 172.21.15.1:0/4053504039 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fa820018070 con 0x56006581a010 2024-02-02T15:42:38.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.289+0000 7fa81ea00700 1 -- 172.21.15.1:0/4053504039 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa8200363c0 con 0x56006581a010 2024-02-02T15:42:38.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.290+0000 7fa84b50db80 10 client.29726 fetch_fsmap learned FSMap version 63 2024-02-02T15:42:38.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.290+0000 7fa84b50db80 1 -- 172.21.15.1:0/4053504039 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) v3 -- 0x5600655f1cd0 con 0x56006581a010 2024-02-02T15:42:38.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.290+0000 7fa81ea00700 1 -- 172.21.15.1:0/4053504039 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa820016070 con 0x56006581a010 2024-02-02T15:42:38.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.290+0000 7fa81ea00700 1 -- 172.21.15.1:0/4053504039 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 63) v1 ==== 7526+0+0 (secure 0 0 0) 0x7fa820004710 con 0x56006581a010 2024-02-02T15:42:38.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.291+0000 7fa84b50db80 10 client.29726 fetch_fsmap finished waiting for FSMap version 63 2024-02-02T15:42:38.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.291+0000 7fa84b50db80 10 client.29726 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-02-02T15:42:38.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.291+0000 7fa84b50db80 20 client.29726 populate_metadata read hostname 'smithi001' 2024-02-02T15:42:38.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.291+0000 7fa84b50db80 1 --2- 172.21.15.1:0/4053504039 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x560065887730 0x560065889b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:38.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.291+0000 7fa84b50db80 4 client.29726 mds_command: new command op to 24518 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:42:38.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.291+0000 7fa84b50db80 1 -- 172.21.15.1:0/4053504039 --> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56006572af50 con 0x560065887730 2024-02-02T15:42:38.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.291+0000 7fa82fe00700 1 --2- 172.21.15.1:0/4053504039 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x560065887730 0x560065889b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:38.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.292+0000 7fa82fe00700 1 --2- 172.21.15.1:0/4053504039 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x560065887730 0x560065889b50 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=91b8ab8b9f2cdd3 server_cookie=15ca5257d49f2d71 in_seq=0 out_seq=0 2024-02-02T15:42:38.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.292+0000 7fa81ea00700 10 client.29726 ms_handle_connect on v2:172.21.15.71:6836/628016199 2024-02-02T15:42:38.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.292+0000 7fa81ea00700 1 -- 172.21.15.1:0/4053504039 <== mds.2 v2:172.21.15.71:6836/628016199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56006572af50 con 0x560065887730 2024-02-02T15:42:38.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.292+0000 7fa81ea00700 10 client.29726 handle_command_reply: tid=0 2024-02-02T15:42:38.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.293+0000 7fa84b50db80 1 client.29726 shutdown 2024-02-02T15:42:38.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.293+0000 7fa84b50db80 1 -- 172.21.15.1:0/4053504039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56006581a010 msgr2=0x56006582c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:38.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.293+0000 7fa84b50db80 1 --2- 172.21.15.1:0/4053504039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56006581a010 0x56006582c330 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x560065838080 tx=0x7fa82001d510 comp rx=0 tx=0).stop 2024-02-02T15:42:38.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.293+0000 7fa84b50db80 1 -- 172.21.15.1:0/4053504039 shutdown_connections 2024-02-02T15:42:38.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.293+0000 7fa84b50db80 1 --2- 172.21.15.1:0/4053504039 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x560065887730 0x560065889b50 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:38.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.293+0000 7fa81ea00700 0 client.29726 ms_handle_reset on v2:172.21.15.71:6836/628016199 2024-02-02T15:42:38.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.293+0000 7fa84b50db80 1 --2- 172.21.15.1:0/4053504039 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5600658257e0 0x56006582aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:38.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.293+0000 7fa84b50db80 1 --2- 172.21.15.1:0/4053504039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56006581a010 0x56006582c330 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:38.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.293+0000 7fa84b50db80 1 --2- 172.21.15.1:0/4053504039 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56006581a930 0x56006582a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:38.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.293+0000 7fa84b50db80 1 -- 172.21.15.1:0/4053504039 >> 172.21.15.1:0/4053504039 conn(0x560065779db0 msgr2=0x5600658315a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:38.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.294+0000 7fa84b50db80 1 -- 172.21.15.1:0/4053504039 shutdown_connections 2024-02-02T15:42:38.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.294+0000 7fa84b50db80 1 -- 172.21.15.1:0/4053504039 wait complete. 2024-02-02T15:42:38.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.294+0000 7fa84b50db80 20 client.29726 trim_cache size 0 max 16384 2024-02-02T15:42:38.295 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:42:38.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.295+0000 7fa837a00700 1 -- 172.21.15.1:0/595191970 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fa8100a0690 msgr2=0x7fa8100a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:38.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.295+0000 7fa837a00700 1 --2- 172.21.15.1:0/595191970 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fa8100a0690 0x7fa8100a2b50 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa820005fd0 tx=0x7fa820009310 comp rx=0 tx=0).stop 2024-02-02T15:42:38.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.296+0000 7fa837a00700 1 -- 172.21.15.1:0/595191970 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa83010d630 msgr2=0x7fa830101350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:38.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.296+0000 7fa837a00700 1 --2- 172.21.15.1:0/595191970 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa83010d630 0x7fa830101350 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fa82800b9e0 tx=0x7fa82800bcf0 comp rx=0 tx=0).stop 2024-02-02T15:42:38.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.296+0000 7fa837a00700 1 -- 172.21.15.1:0/595191970 shutdown_connections 2024-02-02T15:42:38.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.296+0000 7fa837a00700 1 --2- 172.21.15.1:0/595191970 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fa8100a0690 0x7fa8100a2b50 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:38.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.296+0000 7fa837a00700 1 --2- 172.21.15.1:0/595191970 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa83010d630 0x7fa830101350 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:38.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.296+0000 7fa837a00700 1 --2- 172.21.15.1:0/595191970 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa830073110 0x7fa830106290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:38.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.296+0000 7fa837a00700 1 --2- 172.21.15.1:0/595191970 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa830073a30 0x7fa8301067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:38.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.296+0000 7fa837a00700 1 -- 172.21.15.1:0/595191970 >> 172.21.15.1:0/595191970 conn(0x7fa8300fc840 msgr2=0x7fa830198630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:38.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.297+0000 7fa837a00700 1 -- 172.21.15.1:0/595191970 shutdown_connections 2024-02-02T15:42:38.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.297+0000 7fa837a00700 1 -- 172.21.15.1:0/595191970 wait complete. 2024-02-02T15:42:38.332 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-02T15:42:38.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.653+0000 7f8968000700 1 -- 172.21.15.1:0/428279852 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8960101000 msgr2=0x7f89601013e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:38.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.653+0000 7f8968000700 1 --2- 172.21.15.1:0/428279852 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8960101000 0x7f89601013e0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f8950009990 tx=0x7f8950009ca0 comp rx=0 tx=0).stop 2024-02-02T15:42:38.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.654+0000 7f8968000700 1 -- 172.21.15.1:0/428279852 shutdown_connections 2024-02-02T15:42:38.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.654+0000 7f8968000700 1 --2- 172.21.15.1:0/428279852 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8960106120 0x7f896010dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:38.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.654+0000 7f8968000700 1 --2- 172.21.15.1:0/428279852 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8960101000 0x7f89601013e0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:38.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.654+0000 7f8968000700 1 --2- 172.21.15.1:0/428279852 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f89601019b0 0x7f89601059f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:38.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.654+0000 7f8968000700 1 -- 172.21.15.1:0/428279852 >> 172.21.15.1:0/428279852 conn(0x7f89600fc840 msgr2=0x7f89600fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:38.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.654+0000 7f8968000700 1 -- 172.21.15.1:0/428279852 shutdown_connections 2024-02-02T15:42:38.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.654+0000 7f8968000700 1 -- 172.21.15.1:0/428279852 wait complete. 2024-02-02T15:42:38.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.655+0000 7f8968000700 1 Processor -- start 2024-02-02T15:42:38.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.655+0000 7f8968000700 1 -- start start 2024-02-02T15:42:38.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.656+0000 7f8968000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8960101000 0x7f8960073040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:38.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.656+0000 7f8968000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8960106120 0x7f8960073580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:38.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.656+0000 7f8968000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8960073ac0 0x7f89601a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:38.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.656+0000 7f8968000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89601a5750 con 0x7f8960106120 2024-02-02T15:42:38.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.656+0000 7f8968000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89601a58c0 con 0x7f8960073ac0 2024-02-02T15:42:38.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.656+0000 7f8968000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89601a5a80 con 0x7f8960101000 2024-02-02T15:42:38.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.656+0000 7f8965600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8960101000 0x7f8960073040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:38.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.656+0000 7f8965600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8960101000 0x7f8960073040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:38298/0 (socket says 172.21.15.1:38298) 2024-02-02T15:42:38.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.656+0000 7f8965600700 1 -- 172.21.15.1:0/1583193585 learned_addr learned my addr 172.21.15.1:0/1583193585 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:42:38.656 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.656+0000 7f8966000700 1 --2- 172.21.15.1:0/1583193585 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8960073ac0 0x7f89601a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:38.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.657+0000 7f8964c00700 1 --2- 172.21.15.1:0/1583193585 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8960106120 0x7f8960073580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:38.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.657+0000 7f8965600700 1 -- 172.21.15.1:0/1583193585 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8960073ac0 msgr2=0x7f89601a50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:38.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.657+0000 7f8965600700 1 --2- 172.21.15.1:0/1583193585 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8960073ac0 0x7f89601a50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:38.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.657+0000 7f8965600700 1 -- 172.21.15.1:0/1583193585 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8960106120 msgr2=0x7f8960073580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:38.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.657+0000 7f8965600700 1 --2- 172.21.15.1:0/1583193585 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8960106120 0x7f8960073580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:38.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.657+0000 7f8965600700 1 -- 172.21.15.1:0/1583193585 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89500095f0 con 0x7f8960101000 2024-02-02T15:42:38.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.658+0000 7f8965600700 1 --2- 172.21.15.1:0/1583193585 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8960101000 0x7f8960073040 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f89500095c0 tx=0x7f895000f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:38.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.658+0000 7f895e000700 1 -- 172.21.15.1:0/1583193585 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f895001d070 con 0x7f8960101000 2024-02-02T15:42:38.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.658+0000 7f8968000700 1 -- 172.21.15.1:0/1583193585 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89601a5d00 con 0x7f8960101000 2024-02-02T15:42:38.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.658+0000 7f8968000700 1 -- 172.21.15.1:0/1583193585 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89601a61d0 con 0x7f8960101000 2024-02-02T15:42:38.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.659+0000 7f895e000700 1 -- 172.21.15.1:0/1583193585 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8950003890 con 0x7f8960101000 2024-02-02T15:42:38.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.659+0000 7f895e000700 1 -- 172.21.15.1:0/1583193585 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8950005470 con 0x7f8960101000 2024-02-02T15:42:38.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.663+0000 7f895e000700 1 -- 172.21.15.1:0/1583193585 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f89500055d0 con 0x7f8960101000 2024-02-02T15:42:38.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.665+0000 7f897ba97b80 1 Processor -- start 2024-02-02T15:42:38.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.667+0000 7f895e000700 1 --2- 172.21.15.1:0/1583193585 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f89400a0790 0x7f89400a2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:38.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.667+0000 7f897ba97b80 1 -- start start 2024-02-02T15:42:38.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.669+0000 7f8966000700 1 --2- 172.21.15.1:0/1583193585 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f89400a0790 0x7f89400a2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:38.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.669+0000 7f895e000700 1 -- 172.21.15.1:0/1583193585 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f89500cce90 con 0x7f8960101000 2024-02-02T15:42:38.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.670+0000 7f897ba97b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ec397be820 0x55ec397bec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:38.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.670+0000 7f897ba97b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ec397bf140 0x55ec397c9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:38.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.670+0000 7f897ba97b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec397c9ff0 0x55ec397cc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:38.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.670+0000 7f897ba97b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec3972a4c0 con 0x55ec397bf140 2024-02-02T15:42:38.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.670+0000 7f897ba97b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec3970eaa0 con 0x55ec397be820 2024-02-02T15:42:38.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.670+0000 7f897ba97b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec3970c460 con 0x55ec397c9ff0 2024-02-02T15:42:38.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.670+0000 7f8965600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ec397be820 0x55ec397bec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:38.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.670+0000 7f8965600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ec397be820 0x55ec397bec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:43756/0 (socket says 172.21.15.1:43756) 2024-02-02T15:42:38.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.670+0000 7f8965600700 1 -- 172.21.15.1:0/726699879 learned_addr learned my addr 172.21.15.1:0/726699879 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:42:38.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.671+0000 7f8965600700 1 -- 172.21.15.1:0/726699879 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec397c9ff0 msgr2=0x55ec397cc3e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-02T15:42:38.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.671+0000 7f8965600700 1 --2- 172.21.15.1:0/726699879 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec397c9ff0 0x55ec397cc3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:38.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.671+0000 7f8965600700 1 -- 172.21.15.1:0/726699879 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ec397bf140 msgr2=0x55ec397c9ab0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-02T15:42:38.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.672+0000 7f8966000700 1 --2- 172.21.15.1:0/1583193585 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f89400a0790 0x7f89400a2c50 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f895800f3c0 tx=0x7f8958007950 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:38.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.672+0000 7f8964c00700 1 --2- 172.21.15.1:0/726699879 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ec397bf140 0x55ec397c9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:38.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.672+0000 7f8965600700 1 --2- 172.21.15.1:0/726699879 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ec397bf140 0x55ec397c9ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:38.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.672+0000 7f8965600700 1 -- 172.21.15.1:0/726699879 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec395893b0 con 0x55ec397be820 2024-02-02T15:42:38.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.673+0000 7f8965600700 1 --2- 172.21.15.1:0/726699879 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ec397be820 0x55ec397bec00 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f8950096ab0 tx=0x7f8950099aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20ef8da01ab87555 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:38.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.673+0000 7f8956600700 1 -- 172.21.15.1:0/726699879 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f895009de70 con 0x55ec397be820 2024-02-02T15:42:38.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.673+0000 7f8956600700 1 -- 172.21.15.1:0/726699879 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f89500a2020 con 0x55ec397be820 2024-02-02T15:42:38.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.673+0000 7f8956600700 1 -- 172.21.15.1:0/726699879 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8950099790 con 0x55ec397be820 2024-02-02T15:42:38.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.675+0000 7f897ba97b80 1 -- 172.21.15.1:0/726699879 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ec397be820 msgr2=0x55ec397bec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:38.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.675+0000 7f897ba97b80 1 --2- 172.21.15.1:0/726699879 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ec397be820 0x55ec397bec00 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f8950096ab0 tx=0x7f8950099aa0 comp rx=0 tx=0).stop 2024-02-02T15:42:38.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.675+0000 7f897ba97b80 1 -- 172.21.15.1:0/726699879 shutdown_connections 2024-02-02T15:42:38.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.675+0000 7f897ba97b80 1 --2- 172.21.15.1:0/726699879 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec397c9ff0 0x55ec397cc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:38.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.675+0000 7f897ba97b80 1 --2- 172.21.15.1:0/726699879 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ec397be820 0x55ec397bec00 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:38.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.675+0000 7f897ba97b80 1 --2- 172.21.15.1:0/726699879 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ec397bf140 0x55ec397c9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:38.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.675+0000 7f897ba97b80 1 -- 172.21.15.1:0/726699879 >> 172.21.15.1:0/726699879 conn(0x55ec39715db0 msgr2=0x55ec397b23b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:38.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.676+0000 7f897ba97b80 1 -- 172.21.15.1:0/726699879 shutdown_connections 2024-02-02T15:42:38.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.676+0000 7f897ba97b80 1 -- 172.21.15.1:0/726699879 wait complete. 2024-02-02T15:42:38.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.676+0000 7f897ba97b80 1 -- start start 2024-02-02T15:42:38.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.677+0000 7f897ba97b80 1 Processor -- start 2024-02-02T15:42:38.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.678+0000 7f897ba97b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ec397be820 0x55ec397ca3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:38.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.678+0000 7f897ba97b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ec397bf140 0x55ec397ca910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:38.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.678+0000 7f897ba97b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec397c9ff0 0x55ec397cae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:38.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.678+0000 7f897ba97b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec39707d50 con 0x55ec397bf140 2024-02-02T15:42:38.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.678+0000 7f897ba97b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec396a3d80 con 0x55ec397be820 2024-02-02T15:42:38.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.678+0000 7f897ba97b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec396a3a20 con 0x55ec397c9ff0 2024-02-02T15:42:38.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.678+0000 7f897ba97b80 10 client.0 _ll_register_callbacks cb 0x55ec396a5290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:42:38.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.678+0000 7f8965600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ec397be820 0x55ec397ca3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:38.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.678+0000 7f8965600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ec397be820 0x55ec397ca3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:43770/0 (socket says 172.21.15.1:43770) 2024-02-02T15:42:38.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.678+0000 7f8965600700 1 -- 172.21.15.1:0/929559880 learned_addr learned my addr 172.21.15.1:0/929559880 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:42:38.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.678+0000 7f8964c00700 1 --2- 172.21.15.1:0/929559880 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ec397bf140 0x55ec397ca910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:38.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.678+0000 7f8966000700 1 --2- 172.21.15.1:0/929559880 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec397c9ff0 0x55ec397cae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:38.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.679+0000 7f8965600700 1 -- 172.21.15.1:0/929559880 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec397c9ff0 msgr2=0x55ec397cae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:38.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.679+0000 7f8965600700 1 --2- 172.21.15.1:0/929559880 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec397c9ff0 0x55ec397cae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:38.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.679+0000 7f8965600700 1 -- 172.21.15.1:0/929559880 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ec397bf140 msgr2=0x55ec397ca910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:38.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.679+0000 7f8965600700 1 --2- 172.21.15.1:0/929559880 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ec397bf140 0x55ec397ca910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:38.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.679+0000 7f8965600700 1 -- 172.21.15.1:0/929559880 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89601a61d0 con 0x55ec397be820 2024-02-02T15:42:38.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.679+0000 7f8964c00700 1 --2- 172.21.15.1:0/929559880 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ec397bf140 0x55ec397ca910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:42:38.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.679+0000 7f8965600700 1 --2- 172.21.15.1:0/929559880 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ec397be820 0x55ec397ca3d0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f8950000c00 tx=0x7f895009a580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eebdea4231c9441a server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:38.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.679+0000 7f894ea00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:42:38.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.679+0000 7f894ea00700 1 -- 172.21.15.1:0/929559880 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f895009de70 con 0x55ec397be820 2024-02-02T15:42:38.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.679+0000 7f897ba97b80 1 -- 172.21.15.1:0/929559880 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ec397aa0f0 con 0x55ec397be820 2024-02-02T15:42:38.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.679+0000 7f894ea00700 1 -- 172.21.15.1:0/929559880 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f89500b9040 con 0x55ec397be820 2024-02-02T15:42:38.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.679+0000 7f894ea00700 1 -- 172.21.15.1:0/929559880 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89500a1da0 con 0x55ec397be820 2024-02-02T15:42:38.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.679+0000 7f897ba97b80 10 client.29738 fetch_fsmap learned FSMap version 63 2024-02-02T15:42:38.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.679+0000 7f897ba97b80 1 -- 172.21.15.1:0/929559880 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) v3 -- 0x55ec39535b60 con 0x55ec397be820 2024-02-02T15:42:38.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.680+0000 7f894ea00700 1 -- 172.21.15.1:0/929559880 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ec397aa0f0 con 0x55ec397be820 2024-02-02T15:42:38.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.681+0000 7f894ea00700 1 -- 172.21.15.1:0/929559880 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 63) v1 ==== 7526+0+0 (secure 0 0 0) 0x7f895000fbf0 con 0x55ec397be820 2024-02-02T15:42:38.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.681+0000 7f897ba97b80 10 client.29738 fetch_fsmap finished waiting for FSMap version 63 2024-02-02T15:42:38.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.681+0000 7f897ba97b80 10 client.29738 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-02T15:42:38.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.681+0000 7f897ba97b80 20 client.29738 populate_metadata read hostname 'smithi001' 2024-02-02T15:42:38.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.681+0000 7f897ba97b80 1 --2- 172.21.15.1:0/929559880 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55ec397d9120 0x55ec397db540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:38.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.681+0000 7f897ba97b80 4 client.29738 mds_command: new command op to 24529 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:42:38.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.681+0000 7f897ba97b80 1 -- 172.21.15.1:0/929559880 --> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ec39700dd0 con 0x55ec397d9120 2024-02-02T15:42:38.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.682+0000 7f8964c00700 1 --2- 172.21.15.1:0/929559880 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55ec397d9120 0x55ec397db540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:38.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.682+0000 7f8964c00700 1 --2- 172.21.15.1:0/929559880 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55ec397d9120 0x55ec397db540 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e8483c251bac1218 server_cookie=31f5a0a4223511f3 in_seq=0 out_seq=0 2024-02-02T15:42:38.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.682+0000 7f894ea00700 10 client.29738 ms_handle_connect on v2:172.21.15.142:6838/321675421 2024-02-02T15:42:38.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.685+0000 7f894ea00700 1 -- 172.21.15.1:0/929559880 <== mds.1 v2:172.21.15.142:6838/321675421 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55ec39700dd0 con 0x55ec397d9120 2024-02-02T15:42:38.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.685+0000 7f894ea00700 10 client.29738 handle_command_reply: tid=0 2024-02-02T15:42:38.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.685+0000 7f897ba97b80 1 client.29738 shutdown 2024-02-02T15:42:38.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.686+0000 7f897ba97b80 1 -- 172.21.15.1:0/929559880 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ec397be820 msgr2=0x55ec397ca3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:38.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.686+0000 7f897ba97b80 1 --2- 172.21.15.1:0/929559880 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ec397be820 0x55ec397ca3d0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f8950000c00 tx=0x7f895009a580 comp rx=0 tx=0).stop 2024-02-02T15:42:38.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.686+0000 7f897ba97b80 1 -- 172.21.15.1:0/929559880 shutdown_connections 2024-02-02T15:42:38.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.686+0000 7f897ba97b80 1 --2- 172.21.15.1:0/929559880 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55ec397d9120 0x55ec397db540 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:38.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.686+0000 7f897ba97b80 1 --2- 172.21.15.1:0/929559880 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec397c9ff0 0x55ec397cae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:38.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.686+0000 7f897ba97b80 1 --2- 172.21.15.1:0/929559880 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ec397be820 0x55ec397ca3d0 secure :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f8950000c00 tx=0x7f895009a580 comp rx=0 tx=0).stop 2024-02-02T15:42:38.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.686+0000 7f897ba97b80 1 --2- 172.21.15.1:0/929559880 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ec397bf140 0x55ec397ca910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:38.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.686+0000 7f894ea00700 0 client.29738 ms_handle_reset on v2:172.21.15.142:6838/321675421 2024-02-02T15:42:38.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.686+0000 7f897ba97b80 1 -- 172.21.15.1:0/929559880 >> 172.21.15.1:0/929559880 conn(0x55ec39715db0 msgr2=0x55ec397b1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:38.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.687+0000 7f897ba97b80 1 -- 172.21.15.1:0/929559880 shutdown_connections 2024-02-02T15:42:38.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.688+0000 7f897ba97b80 1 -- 172.21.15.1:0/929559880 wait complete. 2024-02-02T15:42:38.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.688+0000 7f897ba97b80 20 client.29738 trim_cache size 0 max 16384 2024-02-02T15:42:38.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.709+0000 7f897ba97b80 1 Processor -- start 2024-02-02T15:42:38.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.709+0000 7f897ba97b80 1 -- start start 2024-02-02T15:42:38.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.709+0000 7f897ba97b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ec397bf140 0x55ec397c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:38.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.709+0000 7f897ba97b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec397c9ff0 0x55ec397c3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:38.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.709+0000 7f897ba97b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ec397b25d0 0x55ec397d5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:38.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.709+0000 7f897ba97b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec3958b6a0 con 0x55ec397b25d0 2024-02-02T15:42:38.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.709+0000 7f897ba97b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec397c96c0 con 0x55ec397bf140 2024-02-02T15:42:38.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.709+0000 7f897ba97b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec397dbba0 con 0x55ec397c9ff0 2024-02-02T15:42:38.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.710+0000 7f8964c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec397c9ff0 0x55ec397c3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:38.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.710+0000 7f8964c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec397c9ff0 0x55ec397c3a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:38306/0 (socket says 172.21.15.1:38306) 2024-02-02T15:42:38.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.710+0000 7f8964c00700 1 -- 172.21.15.1:0/248605119 learned_addr learned my addr 172.21.15.1:0/248605119 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:42:38.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.711+0000 7f8964c00700 1 -- 172.21.15.1:0/248605119 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ec397bf140 msgr2=0x55ec397c8a20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-02T15:42:38.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.711+0000 7f8964c00700 1 --2- 172.21.15.1:0/248605119 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ec397bf140 0x55ec397c8a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:38.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.711+0000 7f8964c00700 1 -- 172.21.15.1:0/248605119 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ec397b25d0 msgr2=0x55ec397d5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:38.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.711+0000 7f8964c00700 1 --2- 172.21.15.1:0/248605119 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ec397b25d0 0x55ec397d5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:38.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.711+0000 7f8964c00700 1 -- 172.21.15.1:0/248605119 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec396cdea0 con 0x55ec397c9ff0 2024-02-02T15:42:38.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.711+0000 7f8964c00700 1 --2- 172.21.15.1:0/248605119 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec397c9ff0 0x55ec397c3a60 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f8960067c80 tx=0x7f89480058e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd866fc0940ee063 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:38.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.712+0000 7f8957000700 1 -- 172.21.15.1:0/248605119 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8948017040 con 0x55ec397c9ff0 2024-02-02T15:42:38.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.712+0000 7f8957000700 1 -- 172.21.15.1:0/248605119 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f894801e070 con 0x55ec397c9ff0 2024-02-02T15:42:38.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.712+0000 7f8957000700 1 -- 172.21.15.1:0/248605119 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8948012450 con 0x55ec397c9ff0 2024-02-02T15:42:38.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.713+0000 7f897ba97b80 1 -- 172.21.15.1:0/248605119 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec397c9ff0 msgr2=0x55ec397c3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:38.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.713+0000 7f897ba97b80 1 --2- 172.21.15.1:0/248605119 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec397c9ff0 0x55ec397c3a60 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f8960067c80 tx=0x7f89480058e0 comp rx=0 tx=0).stop 2024-02-02T15:42:38.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.713+0000 7f897ba97b80 1 -- 172.21.15.1:0/248605119 shutdown_connections 2024-02-02T15:42:38.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.713+0000 7f897ba97b80 1 --2- 172.21.15.1:0/248605119 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ec397b25d0 0x55ec397d5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:38.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.713+0000 7f897ba97b80 1 --2- 172.21.15.1:0/248605119 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ec397bf140 0x55ec397c8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:38.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.713+0000 7f897ba97b80 1 --2- 172.21.15.1:0/248605119 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec397c9ff0 0x55ec397c3a60 secure :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f8960067c80 tx=0x7f89480058e0 comp rx=0 tx=0).stop 2024-02-02T15:42:38.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.713+0000 7f897ba97b80 1 -- 172.21.15.1:0/248605119 >> 172.21.15.1:0/248605119 conn(0x55ec39715db0 msgr2=0x55ec397cac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:38.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.715+0000 7f897ba97b80 1 -- 172.21.15.1:0/248605119 shutdown_connections 2024-02-02T15:42:38.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.715+0000 7f897ba97b80 1 -- 172.21.15.1:0/248605119 wait complete. 2024-02-02T15:42:38.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.715+0000 7f897ba97b80 1 -- start start 2024-02-02T15:42:38.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.716+0000 7f897ba97b80 1 Processor -- start 2024-02-02T15:42:38.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.716+0000 7f897ba97b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ec397b25d0 0x55ec397cf360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:38.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.716+0000 7f897ba97b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ec397bf140 0x55ec397cd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:38.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.716+0000 7f897ba97b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec397cdb90 0x55ec397ce010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:38.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.716+0000 7f897ba97b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec3982ad50 con 0x55ec397bf140 2024-02-02T15:42:38.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.716+0000 7f897ba97b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec3982ae90 con 0x55ec397b25d0 2024-02-02T15:42:38.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.716+0000 7f897ba97b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec3982b050 con 0x55ec397cdb90 2024-02-02T15:42:38.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.716+0000 7f897ba97b80 10 client.0 _ll_register_callbacks cb 0x55ec396a5290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:42:38.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.717+0000 7f8966000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec397cdb90 0x55ec397ce010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:38.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.717+0000 7f8966000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec397cdb90 0x55ec397ce010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:38316/0 (socket says 172.21.15.1:38316) 2024-02-02T15:42:38.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.717+0000 7f8966000700 1 -- 172.21.15.1:0/1138421740 learned_addr learned my addr 172.21.15.1:0/1138421740 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:42:38.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.717+0000 7f8964c00700 1 --2- 172.21.15.1:0/1138421740 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ec397bf140 0x55ec397cd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:38.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.717+0000 7f8966000700 1 -- 172.21.15.1:0/1138421740 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ec397b25d0 msgr2=0x55ec397cf360 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-02T15:42:38.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.717+0000 7f8966000700 1 --2- 172.21.15.1:0/1138421740 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ec397b25d0 0x55ec397cf360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:38.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.717+0000 7f8966000700 1 -- 172.21.15.1:0/1138421740 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ec397bf140 msgr2=0x55ec397cd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:38.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.717+0000 7f8966000700 1 --2- 172.21.15.1:0/1138421740 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ec397bf140 0x55ec397cd5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:38.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.717+0000 7f8966000700 1 -- 172.21.15.1:0/1138421740 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f894800dd90 con 0x55ec397cdb90 2024-02-02T15:42:38.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.718+0000 7f8964c00700 1 --2- 172.21.15.1:0/1138421740 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ec397bf140 0x55ec397cd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:42:38.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.718+0000 7f8966000700 1 --2- 172.21.15.1:0/1138421740 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec397cdb90 0x55ec397ce010 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f8958014e50 tx=0x7f8958014f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3633d3483044376 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:38.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.718+0000 7f894ea00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:42:38.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.718+0000 7f894ea00700 1 -- 172.21.15.1:0/1138421740 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8958015a20 con 0x55ec397cdb90 2024-02-02T15:42:38.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.718+0000 7f894ea00700 1 -- 172.21.15.1:0/1138421740 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f8958015b80 con 0x55ec397cdb90 2024-02-02T15:42:38.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.718+0000 7f897ba97b80 1 -- 172.21.15.1:0/1138421740 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f89500a2020 con 0x55ec397cdb90 2024-02-02T15:42:38.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.718+0000 7f894ea00700 1 -- 172.21.15.1:0/1138421740 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8958015d40 con 0x55ec397cdb90 2024-02-02T15:42:38.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.718+0000 7f897ba97b80 10 client.29680 fetch_fsmap learned FSMap version 63 2024-02-02T15:42:38.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.718+0000 7f897ba97b80 1 -- 172.21.15.1:0/1138421740 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) v3 -- 0x55ec3958dcd0 con 0x55ec397cdb90 2024-02-02T15:42:38.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.720+0000 7f894ea00700 1 -- 172.21.15.1:0/1138421740 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f89580218e0 con 0x55ec397cdb90 2024-02-02T15:42:38.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.721+0000 7f894ea00700 1 -- 172.21.15.1:0/1138421740 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 63) v1 ==== 7526+0+0 (secure 0 0 0) 0x7f8958029680 con 0x55ec397cdb90 2024-02-02T15:42:38.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.721+0000 7f897ba97b80 10 client.29680 fetch_fsmap finished waiting for FSMap version 63 2024-02-02T15:42:38.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.721+0000 7f897ba97b80 10 client.29680 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-02T15:42:38.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.721+0000 7f897ba97b80 20 client.29680 populate_metadata read hostname 'smithi001' 2024-02-02T15:42:38.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.721+0000 7f897ba97b80 1 --2- 172.21.15.1:0/1138421740 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55ec3982c910 0x55ec3982ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:38.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.721+0000 7f897ba97b80 4 client.29680 mds_command: new command op to 24529 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:42:38.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.721+0000 7f897ba97b80 1 -- 172.21.15.1:0/1138421740 --> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f89500a2020 con 0x55ec3982c910 2024-02-02T15:42:38.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.721+0000 7f8965600700 1 --2- 172.21.15.1:0/1138421740 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55ec3982c910 0x55ec3982ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:38.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.722+0000 7f894ea00700 10 client.29680 ms_handle_connect on v2:172.21.15.142:6838/321675421 2024-02-02T15:42:38.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.722+0000 7f8965600700 1 --2- 172.21.15.1:0/1138421740 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55ec3982c910 0x55ec3982ed30 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=53cb343cb479ad2f server_cookie=7cd6f0f7937257fd in_seq=0 out_seq=0 2024-02-02T15:42:38.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.724+0000 7f894ea00700 1 -- 172.21.15.1:0/1138421740 <== mds.1 v2:172.21.15.142:6838/321675421 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f89500a2020 con 0x55ec3982c910 2024-02-02T15:42:38.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.724+0000 7f894ea00700 10 client.29680 handle_command_reply: tid=0 2024-02-02T15:42:38.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.724+0000 7f897ba97b80 1 client.29680 shutdown 2024-02-02T15:42:38.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.724+0000 7f897ba97b80 1 -- 172.21.15.1:0/1138421740 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec397cdb90 msgr2=0x55ec397ce010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:38.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.724+0000 7f897ba97b80 1 --2- 172.21.15.1:0/1138421740 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec397cdb90 0x55ec397ce010 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f8958014e50 tx=0x7f8958014f30 comp rx=0 tx=0).stop 2024-02-02T15:42:38.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.724+0000 7f897ba97b80 1 -- 172.21.15.1:0/1138421740 shutdown_connections 2024-02-02T15:42:38.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.724+0000 7f897ba97b80 1 --2- 172.21.15.1:0/1138421740 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55ec3982c910 0x55ec3982ed30 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:38.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.724+0000 7f897ba97b80 1 --2- 172.21.15.1:0/1138421740 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ec397cdb90 0x55ec397ce010 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:38.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.724+0000 7f897ba97b80 1 --2- 172.21.15.1:0/1138421740 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ec397b25d0 0x55ec397cf360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:38.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.724+0000 7f894ea00700 0 client.29680 ms_handle_reset on v2:172.21.15.142:6838/321675421 2024-02-02T15:42:38.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.724+0000 7f897ba97b80 1 --2- 172.21.15.1:0/1138421740 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ec397bf140 0x55ec397cd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:38.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.724+0000 7f897ba97b80 1 -- 172.21.15.1:0/1138421740 >> 172.21.15.1:0/1138421740 conn(0x55ec39715db0 msgr2=0x55ec397c3a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:38.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.725+0000 7f897ba97b80 1 -- 172.21.15.1:0/1138421740 shutdown_connections 2024-02-02T15:42:38.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.725+0000 7f897ba97b80 1 -- 172.21.15.1:0/1138421740 wait complete. 2024-02-02T15:42:38.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.725+0000 7f897ba97b80 20 client.29680 trim_cache size 0 max 16384 2024-02-02T15:42:38.725 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:42:38.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.726+0000 7f8968000700 1 -- 172.21.15.1:0/1583193585 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f89400a0790 msgr2=0x7f89400a2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:38.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.726+0000 7f8968000700 1 --2- 172.21.15.1:0/1583193585 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f89400a0790 0x7f89400a2c50 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f895800f3c0 tx=0x7f8958007950 comp rx=0 tx=0).stop 2024-02-02T15:42:38.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.726+0000 7f8968000700 1 -- 172.21.15.1:0/1583193585 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8960101000 msgr2=0x7f8960073040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:38.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.726+0000 7f8968000700 1 --2- 172.21.15.1:0/1583193585 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8960101000 0x7f8960073040 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f89500095c0 tx=0x7f895000f690 comp rx=0 tx=0).stop 2024-02-02T15:42:38.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.726+0000 7f8968000700 1 -- 172.21.15.1:0/1583193585 shutdown_connections 2024-02-02T15:42:38.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.726+0000 7f8968000700 1 --2- 172.21.15.1:0/1583193585 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f89400a0790 0x7f89400a2c50 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:38.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.726+0000 7f8968000700 1 --2- 172.21.15.1:0/1583193585 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8960073ac0 0x7f89601a50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:38.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.726+0000 7f8968000700 1 --2- 172.21.15.1:0/1583193585 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8960101000 0x7f8960073040 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:38.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.726+0000 7f8968000700 1 --2- 172.21.15.1:0/1583193585 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8960106120 0x7f8960073580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:38.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.726+0000 7f8968000700 1 -- 172.21.15.1:0/1583193585 >> 172.21.15.1:0/1583193585 conn(0x7f89600fc840 msgr2=0x7f89601049d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:38.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.727+0000 7f8968000700 1 -- 172.21.15.1:0/1583193585 shutdown_connections 2024-02-02T15:42:38.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:38.727+0000 7f8968000700 1 -- 172.21.15.1:0/1583193585 wait complete. 2024-02-02T15:42:38.757 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-02T15:42:38.983 INFO:tasks.workunit.client.0.smithi001.stdout: 10 77475 37.65 MB/sec execute 11 sec latency 60.130 ms 2024-02-02T15:42:39.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.060+0000 7f0105a00700 1 -- 172.21.15.1:0/3931187862 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0100101000 msgr2=0x7f01001013e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:39.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.060+0000 7f0105a00700 1 --2- 172.21.15.1:0/3931187862 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0100101000 0x7f01001013e0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f00f0009990 tx=0x7f00f0009ca0 comp rx=0 tx=0).stop 2024-02-02T15:42:39.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.060+0000 7f0105a00700 1 -- 172.21.15.1:0/3931187862 shutdown_connections 2024-02-02T15:42:39.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.060+0000 7f0105a00700 1 --2- 172.21.15.1:0/3931187862 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0100106120 0x7f010010dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:39.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.060+0000 7f0105a00700 1 --2- 172.21.15.1:0/3931187862 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0100101000 0x7f01001013e0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:39.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.060+0000 7f0105a00700 1 --2- 172.21.15.1:0/3931187862 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f01001019b0 0x7f01001059f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:39.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.060+0000 7f0105a00700 1 -- 172.21.15.1:0/3931187862 >> 172.21.15.1:0/3931187862 conn(0x7f01000fc840 msgr2=0x7f01000fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:39.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.060+0000 7f0105a00700 1 -- 172.21.15.1:0/3931187862 shutdown_connections 2024-02-02T15:42:39.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.060+0000 7f0105a00700 1 -- 172.21.15.1:0/3931187862 wait complete. 2024-02-02T15:42:39.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.061+0000 7f0105a00700 1 Processor -- start 2024-02-02T15:42:39.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.062+0000 7f0105a00700 1 -- start start 2024-02-02T15:42:39.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.062+0000 7f0105a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0100101000 0x7f0100194d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:39.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.062+0000 7f0105a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0100106120 0x7f0100195290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:39.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.062+0000 7f0105a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01001996f0 0x7f0100199b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:39.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.062+0000 7f0105a00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f010019a170 con 0x7f0100101000 2024-02-02T15:42:39.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.062+0000 7f00fea00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0100101000 0x7f0100194d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:39.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.063+0000 7f00fea00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0100101000 0x7f0100194d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53210/0 (socket says 172.21.15.1:53210) 2024-02-02T15:42:39.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.063+0000 7f00fea00700 1 -- 172.21.15.1:0/2261928488 learned_addr learned my addr 172.21.15.1:0/2261928488 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:42:39.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.063+0000 7f00fe000700 1 --2- 172.21.15.1:0/2261928488 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0100106120 0x7f0100195290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:39.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.063+0000 7f00ff400700 1 --2- 172.21.15.1:0/2261928488 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01001996f0 0x7f0100199b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:39.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.063+0000 7f0105a00700 1 -- 172.21.15.1:0/2261928488 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f010019a2b0 con 0x7f01001996f0 2024-02-02T15:42:39.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.063+0000 7f0105a00700 1 -- 172.21.15.1:0/2261928488 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f010019a420 con 0x7f0100106120 2024-02-02T15:42:39.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.063+0000 7f00fea00700 1 -- 172.21.15.1:0/2261928488 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0100106120 msgr2=0x7f0100195290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:39.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.063+0000 7f00fea00700 1 --2- 172.21.15.1:0/2261928488 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0100106120 0x7f0100195290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:39.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.064+0000 7f00fea00700 1 -- 172.21.15.1:0/2261928488 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01001996f0 msgr2=0x7f0100199b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:39.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.064+0000 7f00fea00700 1 --2- 172.21.15.1:0/2261928488 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01001996f0 0x7f0100199b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:39.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.064+0000 7f00fea00700 1 -- 172.21.15.1:0/2261928488 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00f00095f0 con 0x7f0100101000 2024-02-02T15:42:39.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.064+0000 7f00fea00700 1 --2- 172.21.15.1:0/2261928488 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0100101000 0x7f0100194d50 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f00f000b2e0 tx=0x7f00f00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:39.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.064+0000 7f00f7400700 1 -- 172.21.15.1:0/2261928488 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f001d070 con 0x7f0100101000 2024-02-02T15:42:39.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.064+0000 7f00ff400700 1 --2- 172.21.15.1:0/2261928488 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01001996f0 0x7f0100199b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:42:39.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.064+0000 7f0105a00700 1 -- 172.21.15.1:0/2261928488 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f010006a760 con 0x7f0100101000 2024-02-02T15:42:39.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.065+0000 7f0105a00700 1 -- 172.21.15.1:0/2261928488 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f010006ac60 con 0x7f0100101000 2024-02-02T15:42:39.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.065+0000 7f00f7400700 1 -- 172.21.15.1:0/2261928488 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f00f000fc40 con 0x7f0100101000 2024-02-02T15:42:39.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.065+0000 7f00f7400700 1 -- 172.21.15.1:0/2261928488 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f002a3b0 con 0x7f0100101000 2024-02-02T15:42:39.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.065+0000 7f00fe000700 1 --2- 172.21.15.1:0/2261928488 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0100106120 0x7f0100195290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:42:39.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.069+0000 7f00f7400700 1 -- 172.21.15.1:0/2261928488 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f00f00173f0 con 0x7f0100101000 2024-02-02T15:42:39.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.071+0000 7f00f7400700 1 --2- 172.21.15.1:0/2261928488 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f00e00a4b20 0x7f00e00a6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:39.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.071+0000 7f00f7400700 1 -- 172.21.15.1:0/2261928488 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f00f00cd600 con 0x7f0100101000 2024-02-02T15:42:39.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.071+0000 7f00fe000700 1 --2- 172.21.15.1:0/2261928488 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f00e00a4b20 0x7f00e00a6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:39.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.073+0000 7f011952cb80 1 Processor -- start 2024-02-02T15:42:39.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.073+0000 7f00fe000700 1 --2- 172.21.15.1:0/2261928488 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f00e00a4b20 0x7f00e00a6fe0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f0100196360 tx=0x7f00e8009260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:39.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.075+0000 7f011952cb80 1 -- start start 2024-02-02T15:42:39.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.076+0000 7f011952cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5634724da010 0x5634724da3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:39.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.076+0000 7f011952cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634724da930 0x5634724edab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:39.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.076+0000 7f011952cb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5634724edff0 0x5634724f03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:39.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.076+0000 7f011952cb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56347244e4c0 con 0x5634724edff0 2024-02-02T15:42:39.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.076+0000 7f011952cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x563472432aa0 con 0x5634724da010 2024-02-02T15:42:39.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.076+0000 7f011952cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563472430460 con 0x5634724da930 2024-02-02T15:42:39.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.076+0000 7f00fe000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634724da930 0x5634724edab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:39.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.076+0000 7f00fe000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634724da930 0x5634724edab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:38334/0 (socket says 172.21.15.1:38334) 2024-02-02T15:42:39.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.076+0000 7f00fea00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5634724da010 0x5634724da3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:39.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.076+0000 7f00fe000700 1 -- 172.21.15.1:0/3444607040 learned_addr learned my addr 172.21.15.1:0/3444607040 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:42:39.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.077+0000 7f00fe000700 1 -- 172.21.15.1:0/3444607040 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5634724da010 msgr2=0x5634724da3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:39.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.077+0000 7f00fe000700 1 --2- 172.21.15.1:0/3444607040 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5634724da010 0x5634724da3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:39.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.077+0000 7f00fe000700 1 -- 172.21.15.1:0/3444607040 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5634724edff0 msgr2=0x5634724f03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:39.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.077+0000 7f00fe000700 1 --2- 172.21.15.1:0/3444607040 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5634724edff0 0x5634724f03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:39.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.077+0000 7f00fe000700 1 -- 172.21.15.1:0/3444607040 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5634722ad3b0 con 0x5634724da930 2024-02-02T15:42:39.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.077+0000 7f00fe000700 1 --2- 172.21.15.1:0/3444607040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634724da930 0x5634724edab0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f00e8010ce0 tx=0x7f00e8013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5567cf2ecf084eca server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:39.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.078+0000 7f00efe00700 1 -- 172.21.15.1:0/3444607040 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00e8012b90 con 0x5634724da930 2024-02-02T15:42:39.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.078+0000 7f00efe00700 1 -- 172.21.15.1:0/3444607040 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f00e8012cf0 con 0x5634724da930 2024-02-02T15:42:39.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.078+0000 7f00efe00700 1 -- 172.21.15.1:0/3444607040 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00e801e680 con 0x5634724da930 2024-02-02T15:42:39.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.079+0000 7f011952cb80 1 -- 172.21.15.1:0/3444607040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634724da930 msgr2=0x5634724edab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:39.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.080+0000 7f011952cb80 1 --2- 172.21.15.1:0/3444607040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634724da930 0x5634724edab0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f00e8010ce0 tx=0x7f00e8013000 comp rx=0 tx=0).stop 2024-02-02T15:42:39.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.081+0000 7f011952cb80 1 -- 172.21.15.1:0/3444607040 shutdown_connections 2024-02-02T15:42:39.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.081+0000 7f011952cb80 1 --2- 172.21.15.1:0/3444607040 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5634724edff0 0x5634724f03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:39.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.081+0000 7f011952cb80 1 --2- 172.21.15.1:0/3444607040 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5634724da010 0x5634724da3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:39.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.081+0000 7f011952cb80 1 --2- 172.21.15.1:0/3444607040 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634724da930 0x5634724edab0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:39.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.081+0000 7f011952cb80 1 -- 172.21.15.1:0/3444607040 >> 172.21.15.1:0/3444607040 conn(0x563472439db0 msgr2=0x5634724d63b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:39.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.081+0000 7f011952cb80 1 -- 172.21.15.1:0/3444607040 shutdown_connections 2024-02-02T15:42:39.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.082+0000 7f011952cb80 1 -- 172.21.15.1:0/3444607040 wait complete. 2024-02-02T15:42:39.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.083+0000 7f011952cb80 1 -- start start 2024-02-02T15:42:39.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.083+0000 7f011952cb80 1 Processor -- start 2024-02-02T15:42:39.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.085+0000 7f011952cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634724da010 0x5634724e7450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:39.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.085+0000 7f011952cb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5634724da930 0x5634724e7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:39.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.085+0000 7f011952cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5634724edff0 0x5634724ed170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:39.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.085+0000 7f011952cb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56347242bd50 con 0x5634724da930 2024-02-02T15:42:39.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.085+0000 7f011952cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634723c7d80 con 0x5634724edff0 2024-02-02T15:42:39.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.085+0000 7f011952cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634723c7a20 con 0x5634724da010 2024-02-02T15:42:39.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.085+0000 7f011952cb80 10 client.0 _ll_register_callbacks cb 0x5634723c9290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:42:39.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.086+0000 7f00ff400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5634724edff0 0x5634724ed170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:39.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.086+0000 7f00fe000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5634724da930 0x5634724e7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:39.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.086+0000 7f00ff400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5634724edff0 0x5634724ed170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:43824/0 (socket says 172.21.15.1:43824) 2024-02-02T15:42:39.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.086+0000 7f00ff400700 1 -- 172.21.15.1:0/878689279 learned_addr learned my addr 172.21.15.1:0/878689279 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:42:39.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.086+0000 7f00fea00700 1 --2- 172.21.15.1:0/878689279 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634724da010 0x5634724e7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:39.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.087+0000 7f00fe000700 1 -- 172.21.15.1:0/878689279 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634724da010 msgr2=0x5634724e7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:39.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.087+0000 7f00fe000700 1 --2- 172.21.15.1:0/878689279 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634724da010 0x5634724e7450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:39.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.087+0000 7f00fe000700 1 -- 172.21.15.1:0/878689279 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5634724edff0 msgr2=0x5634724ed170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:39.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.087+0000 7f00fe000700 1 --2- 172.21.15.1:0/878689279 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5634724edff0 0x5634724ed170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:39.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.087+0000 7f00fe000700 1 -- 172.21.15.1:0/878689279 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00e8010960 con 0x5634724da930 2024-02-02T15:42:39.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.087+0000 7f00fe000700 1 --2- 172.21.15.1:0/878689279 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5634724da930 0x5634724e7990 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f00e8011010 tx=0x7f00e8012c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa17dd3161190079 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:39.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.087+0000 7f00ecc00700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:42:39.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.088+0000 7f00ecc00700 1 -- 172.21.15.1:0/878689279 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00e802e070 con 0x5634724da930 2024-02-02T15:42:39.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.088+0000 7f00ecc00700 1 -- 172.21.15.1:0/878689279 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f00e801f790 con 0x5634724da930 2024-02-02T15:42:39.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.088+0000 7f011952cb80 1 -- 172.21.15.1:0/878689279 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5634724ce0f0 con 0x5634724da930 2024-02-02T15:42:39.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.088+0000 7f00ecc00700 1 -- 172.21.15.1:0/878689279 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00e801f950 con 0x5634724da930 2024-02-02T15:42:39.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.088+0000 7f011952cb80 10 client.19572 fetch_fsmap learned FSMap version 63 2024-02-02T15:42:39.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.088+0000 7f011952cb80 1 -- 172.21.15.1:0/878689279 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) v3 -- 0x563472259b60 con 0x5634724da930 2024-02-02T15:42:39.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.089+0000 7f00ecc00700 1 -- 172.21.15.1:0/878689279 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5634724ce0f0 con 0x5634724da930 2024-02-02T15:42:39.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.089+0000 7f00ecc00700 1 -- 172.21.15.1:0/878689279 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 63) v1 ==== 7526+0+0 (secure 0 0 0) 0x7f00e8003cc0 con 0x5634724da930 2024-02-02T15:42:39.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.089+0000 7f011952cb80 10 client.19572 fetch_fsmap finished waiting for FSMap version 63 2024-02-02T15:42:39.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.089+0000 7f011952cb80 10 client.19572 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:42:39.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.089+0000 7f011952cb80 20 client.19572 populate_metadata read hostname 'smithi001' 2024-02-02T15:42:39.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.089+0000 7f011952cb80 1 --2- 172.21.15.1:0/878689279 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x563472506200 0x563472508620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:39.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.089+0000 7f011952cb80 4 client.19572 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:42:39.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.089+0000 7f011952cb80 1 -- 172.21.15.1:0/878689279 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563472424dd0 con 0x563472506200 2024-02-02T15:42:39.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.089+0000 7f00fea00700 1 --2- 172.21.15.1:0/878689279 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x563472506200 0x563472508620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:39.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.090+0000 7f00fea00700 1 --2- 172.21.15.1:0/878689279 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x563472506200 0x563472508620 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6757d47fd156adf server_cookie=67882e249cfbb67e in_seq=0 out_seq=0 2024-02-02T15:42:39.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.090+0000 7f00ecc00700 10 client.19572 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:42:39.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.093+0000 7f00ecc00700 1 -- 172.21.15.1:0/878689279 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x563472424dd0 con 0x563472506200 2024-02-02T15:42:39.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.093+0000 7f00ecc00700 10 client.19572 handle_command_reply: tid=0 2024-02-02T15:42:39.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.093+0000 7f011952cb80 1 client.19572 shutdown 2024-02-02T15:42:39.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.094+0000 7f011952cb80 1 -- 172.21.15.1:0/878689279 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5634724da930 msgr2=0x5634724e7990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:39.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.094+0000 7f011952cb80 1 --2- 172.21.15.1:0/878689279 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5634724da930 0x5634724e7990 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f00e8011010 tx=0x7f00e8012c30 comp rx=0 tx=0).stop 2024-02-02T15:42:39.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.094+0000 7f011952cb80 1 -- 172.21.15.1:0/878689279 shutdown_connections 2024-02-02T15:42:39.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.094+0000 7f011952cb80 1 --2- 172.21.15.1:0/878689279 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x563472506200 0x563472508620 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:39.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.094+0000 7f011952cb80 1 --2- 172.21.15.1:0/878689279 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5634724edff0 0x5634724ed170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:39.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.094+0000 7f011952cb80 1 --2- 172.21.15.1:0/878689279 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634724da010 0x5634724e7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:39.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.094+0000 7f011952cb80 1 --2- 172.21.15.1:0/878689279 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5634724da930 0x5634724e7990 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:39.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.094+0000 7f011952cb80 1 -- 172.21.15.1:0/878689279 >> 172.21.15.1:0/878689279 conn(0x563472439db0 msgr2=0x5634724d5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:39.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.094+0000 7f00ecc00700 0 client.19572 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:42:39.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.094+0000 7f011952cb80 1 -- 172.21.15.1:0/878689279 shutdown_connections 2024-02-02T15:42:39.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.094+0000 7f011952cb80 1 -- 172.21.15.1:0/878689279 wait complete. 2024-02-02T15:42:39.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.095+0000 7f011952cb80 20 client.19572 trim_cache size 0 max 16384 2024-02-02T15:42:39.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.111+0000 7f011952cb80 1 Processor -- start 2024-02-02T15:42:39.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.111+0000 7f011952cb80 1 -- start start 2024-02-02T15:42:39.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.111+0000 7f011952cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5634724da010 0x5634724d6560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:39.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.111+0000 7f011952cb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5634724da930 0x5634724e1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:39.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.112+0000 7f011952cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634724edff0 0x5634724e7000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:39.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.112+0000 7f011952cb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634722af6a0 con 0x5634724da930 2024-02-02T15:42:39.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.112+0000 7f011952cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634724dd530 con 0x5634724da010 2024-02-02T15:42:39.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.112+0000 7f011952cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634724e0c30 con 0x5634724edff0 2024-02-02T15:42:39.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.112+0000 7f00fea00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5634724da010 0x5634724d6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:39.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.112+0000 7f00ff400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634724edff0 0x5634724e7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:39.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.112+0000 7f00fea00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5634724da010 0x5634724d6560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:43838/0 (socket says 172.21.15.1:43838) 2024-02-02T15:42:39.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.112+0000 7f00fea00700 1 -- 172.21.15.1:0/1076534590 learned_addr learned my addr 172.21.15.1:0/1076534590 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:42:39.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.112+0000 7f00fe000700 1 --2- 172.21.15.1:0/1076534590 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5634724da930 0x5634724e1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:39.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.112+0000 7f00fea00700 1 -- 172.21.15.1:0/1076534590 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634724edff0 msgr2=0x5634724e7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:39.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.112+0000 7f00fea00700 1 --2- 172.21.15.1:0/1076534590 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634724edff0 0x5634724e7000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:39.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.112+0000 7f00fea00700 1 -- 172.21.15.1:0/1076534590 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5634724da930 msgr2=0x5634724e1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:39.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.112+0000 7f00fea00700 1 --2- 172.21.15.1:0/1076534590 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5634724da930 0x5634724e1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:39.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.112+0000 7f00fea00700 1 -- 172.21.15.1:0/1076534590 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5634723f1ea0 con 0x5634724da010 2024-02-02T15:42:39.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.113+0000 7f00fe000700 1 --2- 172.21.15.1:0/1076534590 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5634724da930 0x5634724e1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:42:39.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.113+0000 7f00fea00700 1 --2- 172.21.15.1:0/1076534590 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5634724da010 0x5634724d6560 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f00f0006010 tx=0x7f00f009a7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9fbfca4451f3f4b5 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:39.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.113+0000 7f00f4c00700 1 -- 172.21.15.1:0/1076534590 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f00a2790 con 0x5634724da010 2024-02-02T15:42:39.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.113+0000 7f00f4c00700 1 -- 172.21.15.1:0/1076534590 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f00f00988f0 con 0x5634724da010 2024-02-02T15:42:39.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.113+0000 7f00f4c00700 1 -- 172.21.15.1:0/1076534590 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f0099ca0 con 0x5634724da010 2024-02-02T15:42:39.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.114+0000 7f011952cb80 1 -- 172.21.15.1:0/1076534590 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5634724da010 msgr2=0x5634724d6560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:39.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.114+0000 7f011952cb80 1 --2- 172.21.15.1:0/1076534590 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5634724da010 0x5634724d6560 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f00f0006010 tx=0x7f00f009a7b0 comp rx=0 tx=0).stop 2024-02-02T15:42:39.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.114+0000 7f011952cb80 1 -- 172.21.15.1:0/1076534590 shutdown_connections 2024-02-02T15:42:39.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.114+0000 7f011952cb80 1 --2- 172.21.15.1:0/1076534590 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634724edff0 0x5634724e7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:39.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.114+0000 7f011952cb80 1 --2- 172.21.15.1:0/1076534590 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5634724da010 0x5634724d6560 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:39.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.114+0000 7f011952cb80 1 --2- 172.21.15.1:0/1076534590 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5634724da930 0x5634724e1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:39.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.114+0000 7f011952cb80 1 -- 172.21.15.1:0/1076534590 >> 172.21.15.1:0/1076534590 conn(0x563472439db0 msgr2=0x5634724d5e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:39.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.114+0000 7f011952cb80 1 -- 172.21.15.1:0/1076534590 shutdown_connections 2024-02-02T15:42:39.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.114+0000 7f011952cb80 1 -- 172.21.15.1:0/1076534590 wait complete. 2024-02-02T15:42:39.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.114+0000 7f011952cb80 1 -- start start 2024-02-02T15:42:39.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.115+0000 7f011952cb80 1 Processor -- start 2024-02-02T15:42:39.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.115+0000 7f011952cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634724da010 0x5634724ed050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:39.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.116+0000 7f011952cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5634724da930 0x5634724ea560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:39.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.116+0000 7f011952cb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5634724edff0 0x5634724eab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:39.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.116+0000 7f011952cb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634724eb210 con 0x5634724edff0 2024-02-02T15:42:39.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.116+0000 7f011952cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634724eb380 con 0x5634724da930 2024-02-02T15:42:39.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.116+0000 7f011952cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563472556ce0 con 0x5634724da010 2024-02-02T15:42:39.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.116+0000 7f011952cb80 10 client.0 _ll_register_callbacks cb 0x5634723c9290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:42:39.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.116+0000 7f00fe000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5634724da930 0x5634724ea560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:39.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.116+0000 7f00fe000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5634724da930 0x5634724ea560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:43840/0 (socket says 172.21.15.1:43840) 2024-02-02T15:42:39.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.116+0000 7f00fe000700 1 -- 172.21.15.1:0/1950299138 learned_addr learned my addr 172.21.15.1:0/1950299138 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:42:39.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.117+0000 7f00fea00700 1 --2- 172.21.15.1:0/1950299138 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634724da010 0x5634724ed050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:39.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.117+0000 7f00fea00700 1 -- 172.21.15.1:0/1950299138 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5634724da930 msgr2=0x5634724ea560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:39.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.117+0000 7f00fea00700 1 --2- 172.21.15.1:0/1950299138 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5634724da930 0x5634724ea560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:39.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.117+0000 7f00fea00700 1 -- 172.21.15.1:0/1950299138 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5634724edff0 msgr2=0x5634724eab00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-02T15:42:39.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.117+0000 7f00fea00700 1 --2- 172.21.15.1:0/1950299138 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5634724edff0 0x5634724eab00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:39.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.117+0000 7f00fea00700 1 -- 172.21.15.1:0/1950299138 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f010006ac60 con 0x5634724da010 2024-02-02T15:42:39.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.117+0000 7f00fea00700 1 --2- 172.21.15.1:0/1950299138 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634724da010 0x5634724ed050 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f00f0096cb0 tx=0x7f00f00b2ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0c1da84aadda9e3 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:39.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.117+0000 7f00ecc00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:42:39.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.118+0000 7f00ecc00700 1 -- 172.21.15.1:0/1950299138 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f000fdb0 con 0x5634724da010 2024-02-02T15:42:39.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.118+0000 7f00ecc00700 1 -- 172.21.15.1:0/1950299138 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f00f009e170 con 0x5634724da010 2024-02-02T15:42:39.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.118+0000 7f011952cb80 1 -- 172.21.15.1:0/1950299138 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f00e8012cf0 con 0x5634724da010 2024-02-02T15:42:39.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.120+0000 7f00ecc00700 1 -- 172.21.15.1:0/1950299138 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00f009e370 con 0x5634724da010 2024-02-02T15:42:39.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.120+0000 7f011952cb80 10 client.29692 fetch_fsmap learned FSMap version 63 2024-02-02T15:42:39.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.120+0000 7f011952cb80 1 -- 172.21.15.1:0/1950299138 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) v3 -- 0x5634722b1cd0 con 0x5634724da010 2024-02-02T15:42:39.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.120+0000 7f00ecc00700 1 -- 172.21.15.1:0/1950299138 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f00e8012cf0 con 0x5634724da010 2024-02-02T15:42:39.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.120+0000 7f00ecc00700 1 -- 172.21.15.1:0/1950299138 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 63) v1 ==== 7526+0+0 (secure 0 0 0) 0x7f00f0082070 con 0x5634724da010 2024-02-02T15:42:39.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.120+0000 7f011952cb80 10 client.29692 fetch_fsmap finished waiting for FSMap version 63 2024-02-02T15:42:39.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.120+0000 7f011952cb80 10 client.29692 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:42:39.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.120+0000 7f011952cb80 20 client.29692 populate_metadata read hostname 'smithi001' 2024-02-02T15:42:39.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.120+0000 7f011952cb80 1 --2- 172.21.15.1:0/1950299138 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x563472557f40 0x56347255a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:39.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.120+0000 7f011952cb80 4 client.29692 mds_command: new command op to 24557 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:42:39.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.120+0000 7f011952cb80 1 -- 172.21.15.1:0/1950299138 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5634723eaf50 con 0x563472557f40 2024-02-02T15:42:39.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.121+0000 7f00fe000700 1 --2- 172.21.15.1:0/1950299138 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x563472557f40 0x56347255a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:39.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.121+0000 7f00ecc00700 10 client.29692 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:42:39.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.121+0000 7f00fe000700 1 --2- 172.21.15.1:0/1950299138 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x563472557f40 0x56347255a360 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6383b505a9b5d269 server_cookie=ad8d0627d4ecfec6 in_seq=0 out_seq=0 2024-02-02T15:42:39.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.121+0000 7f00ecc00700 1 -- 172.21.15.1:0/1950299138 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5634723eaf50 con 0x563472557f40 2024-02-02T15:42:39.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.121+0000 7f00ecc00700 10 client.29692 handle_command_reply: tid=0 2024-02-02T15:42:39.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.122+0000 7f011952cb80 1 client.29692 shutdown 2024-02-02T15:42:39.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.122+0000 7f011952cb80 1 -- 172.21.15.1:0/1950299138 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634724da010 msgr2=0x5634724ed050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:39.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.122+0000 7f011952cb80 1 --2- 172.21.15.1:0/1950299138 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634724da010 0x5634724ed050 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f00f0096cb0 tx=0x7f00f00b2ec0 comp rx=0 tx=0).stop 2024-02-02T15:42:39.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.122+0000 7f011952cb80 1 -- 172.21.15.1:0/1950299138 shutdown_connections 2024-02-02T15:42:39.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.122+0000 7f011952cb80 1 --2- 172.21.15.1:0/1950299138 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x563472557f40 0x56347255a360 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:39.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.122+0000 7f011952cb80 1 --2- 172.21.15.1:0/1950299138 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5634724edff0 0x5634724eab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:39.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.122+0000 7f011952cb80 1 --2- 172.21.15.1:0/1950299138 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5634724da010 0x5634724ed050 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:39.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.122+0000 7f011952cb80 1 --2- 172.21.15.1:0/1950299138 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5634724da930 0x5634724ea560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:39.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.122+0000 7f00ecc00700 0 client.29692 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:42:39.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.122+0000 7f011952cb80 1 -- 172.21.15.1:0/1950299138 >> 172.21.15.1:0/1950299138 conn(0x563472439db0 msgr2=0x5634724e0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:39.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.122+0000 7f011952cb80 1 -- 172.21.15.1:0/1950299138 shutdown_connections 2024-02-02T15:42:39.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.122+0000 7f011952cb80 1 -- 172.21.15.1:0/1950299138 wait complete. 2024-02-02T15:42:39.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.123+0000 7f011952cb80 20 client.29692 trim_cache size 0 max 16384 2024-02-02T15:42:39.123 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:42:39.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.124+0000 7f0105a00700 1 -- 172.21.15.1:0/2261928488 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f00e00a4b20 msgr2=0x7f00e00a6fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:39.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.124+0000 7f0105a00700 1 --2- 172.21.15.1:0/2261928488 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f00e00a4b20 0x7f00e00a6fe0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f0100196360 tx=0x7f00e8009260 comp rx=0 tx=0).stop 2024-02-02T15:42:39.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.124+0000 7f0105a00700 1 -- 172.21.15.1:0/2261928488 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0100101000 msgr2=0x7f0100194d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:39.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.124+0000 7f0105a00700 1 --2- 172.21.15.1:0/2261928488 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0100101000 0x7f0100194d50 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f00f000b2e0 tx=0x7f00f00057f0 comp rx=0 tx=0).stop 2024-02-02T15:42:39.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.124+0000 7f0105a00700 1 -- 172.21.15.1:0/2261928488 shutdown_connections 2024-02-02T15:42:39.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.124+0000 7f0105a00700 1 --2- 172.21.15.1:0/2261928488 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f00e00a4b20 0x7f00e00a6fe0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:39.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.124+0000 7f0105a00700 1 --2- 172.21.15.1:0/2261928488 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f01001996f0 0x7f0100199b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:39.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.124+0000 7f0105a00700 1 --2- 172.21.15.1:0/2261928488 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0100101000 0x7f0100194d50 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:39.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.124+0000 7f0105a00700 1 --2- 172.21.15.1:0/2261928488 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0100106120 0x7f0100195290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:39.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.124+0000 7f0105a00700 1 -- 172.21.15.1:0/2261928488 >> 172.21.15.1:0/2261928488 conn(0x7f01000fc840 msgr2=0x7f01001049d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:39.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.125+0000 7f0105a00700 1 -- 172.21.15.1:0/2261928488 shutdown_connections 2024-02-02T15:42:39.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:39.125+0000 7f0105a00700 1 -- 172.21.15.1:0/2261928488 wait complete. 2024-02-02T15:42:39.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:42:39 smithi071 ceph-mon[44707]: pgmap v490: 129 pgs: 129 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.5 MiB/s wr, 541 op/s 2024-02-02T15:42:39.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:42:39 smithi142 ceph-mon[45728]: pgmap v490: 129 pgs: 129 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.5 MiB/s wr, 541 op/s 2024-02-02T15:42:39.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:42:39 smithi001 ceph-mon[35470]: pgmap v490: 129 pgs: 129 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.5 MiB/s wr, 541 op/s 2024-02-02T15:42:39.983 INFO:tasks.workunit.client.0.smithi001.stdout: 10 78312 39.28 MB/sec execute 12 sec latency 39.405 ms 2024-02-02T15:42:40.151 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-02T15:42:40.151 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-02T15:42:40.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.493+0000 7f32b9800700 1 -- 172.21.15.1:0/1932997789 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f32b41005b0 msgr2=0x7f32b410c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:40.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.493+0000 7f32b9800700 1 --2- 172.21.15.1:0/1932997789 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f32b41005b0 0x7f32b410c870 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f32ac00b0d0 tx=0x7f32ac00b3e0 comp rx=0 tx=0).stop 2024-02-02T15:42:40.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.493+0000 7f32b9800700 1 -- 172.21.15.1:0/1932997789 shutdown_connections 2024-02-02T15:42:40.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.493+0000 7f32b9800700 1 --2- 172.21.15.1:0/1932997789 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f32b41005b0 0x7f32b410c870 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:40.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.493+0000 7f32b9800700 1 --2- 172.21.15.1:0/1932997789 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f32b4105a20 0x7f32b4105e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:40.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.493+0000 7f32b9800700 1 --2- 172.21.15.1:0/1932997789 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32b40ffbf0 0x7f32b4100070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:40.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.493+0000 7f32b9800700 1 -- 172.21.15.1:0/1932997789 >> 172.21.15.1:0/1932997789 conn(0x7f32b4076730 msgr2=0x7f32b4076b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:40.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.493+0000 7f32b9800700 1 -- 172.21.15.1:0/1932997789 shutdown_connections 2024-02-02T15:42:40.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.493+0000 7f32b9800700 1 -- 172.21.15.1:0/1932997789 wait complete. 2024-02-02T15:42:40.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.495+0000 7f32b9800700 1 Processor -- start 2024-02-02T15:42:40.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.495+0000 7f32b9800700 1 -- start start 2024-02-02T15:42:40.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.495+0000 7f32b9800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f32b40ffbf0 0x7f32b41951c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:40.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.495+0000 7f32b9800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f32b41005b0 0x7f32b4195700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:40.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.495+0000 7f32b9800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32b4105a20 0x7f32b41993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:40.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.495+0000 7f32b9800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b41998f0 con 0x7f32b41005b0 2024-02-02T15:42:40.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.495+0000 7f32b9800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b4199a60 con 0x7f32b40ffbf0 2024-02-02T15:42:40.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.495+0000 7f32b9800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b4199c20 con 0x7f32b4105a20 2024-02-02T15:42:40.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.496+0000 7f32b3400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f32b41005b0 0x7f32b4195700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:40.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.496+0000 7f32b3e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f32b40ffbf0 0x7f32b41951c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:40.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.496+0000 7f32b3400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f32b41005b0 0x7f32b4195700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53276/0 (socket says 172.21.15.1:53276) 2024-02-02T15:42:40.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.496+0000 7f32b3400700 1 -- 172.21.15.1:0/3539957601 learned_addr learned my addr 172.21.15.1:0/3539957601 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:42:40.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.496+0000 7f32b3400700 1 -- 172.21.15.1:0/3539957601 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32b4105a20 msgr2=0x7f32b41993b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:40.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.496+0000 7f32b8e00700 1 --2- 172.21.15.1:0/3539957601 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32b4105a20 0x7f32b41993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:40.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.497+0000 7f32b3400700 1 --2- 172.21.15.1:0/3539957601 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32b4105a20 0x7f32b41993b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:40.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.497+0000 7f32b3400700 1 -- 172.21.15.1:0/3539957601 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f32b40ffbf0 msgr2=0x7f32b41951c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:40.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.497+0000 7f32b3400700 1 --2- 172.21.15.1:0/3539957601 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f32b40ffbf0 0x7f32b41951c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:40.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.497+0000 7f32b3400700 1 -- 172.21.15.1:0/3539957601 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32ac009d00 con 0x7f32b41005b0 2024-02-02T15:42:40.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.497+0000 7f32b8e00700 1 --2- 172.21.15.1:0/3539957601 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32b4105a20 0x7f32b41993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:42:40.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.497+0000 7f32b3400700 1 --2- 172.21.15.1:0/3539957601 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f32b41005b0 0x7f32b4195700 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f329c00d950 tx=0x7f329c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:40.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.498+0000 7f32b0c00700 1 -- 172.21.15.1:0/3539957601 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329c009b40 con 0x7f32b41005b0 2024-02-02T15:42:40.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.498+0000 7f32b0c00700 1 -- 172.21.15.1:0/3539957601 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f329c009ca0 con 0x7f32b41005b0 2024-02-02T15:42:40.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.498+0000 7f32b0c00700 1 -- 172.21.15.1:0/3539957601 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329c00f620 con 0x7f32b41005b0 2024-02-02T15:42:40.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.498+0000 7f32b9800700 1 -- 172.21.15.1:0/3539957601 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32b4199f00 con 0x7f32b41005b0 2024-02-02T15:42:40.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.498+0000 7f32b9800700 1 -- 172.21.15.1:0/3539957601 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32b419a230 con 0x7f32b41005b0 2024-02-02T15:42:40.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.503+0000 7f32b0c00700 1 -- 172.21.15.1:0/3539957601 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f329c010460 con 0x7f32b41005b0 2024-02-02T15:42:40.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.505+0000 7f32b0c00700 1 --2- 172.21.15.1:0/3539957601 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f32940a0820 0x7f32940a2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:40.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.505+0000 7f32b0c00700 1 -- 172.21.15.1:0/3539957601 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f329c0cc230 con 0x7f32b41005b0 2024-02-02T15:42:40.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.506+0000 7f32b3e00700 1 --2- 172.21.15.1:0/3539957601 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f32940a0820 0x7f32940a2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:40.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.507+0000 7f32b3e00700 1 --2- 172.21.15.1:0/3539957601 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f32940a0820 0x7f32940a2ce0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f32a4005fd0 tx=0x7f32a4005db0 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:40.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.509+0000 7f32ce931b80 1 Processor -- start 2024-02-02T15:42:40.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.510+0000 7f32ce931b80 1 -- start start 2024-02-02T15:42:40.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.512+0000 7f32ce931b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55953260e270 0x55953260e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:40.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.512+0000 7f32ce931b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55953260eb90 0x559532619530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:40.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.512+0000 7f32ce931b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559532619a70 0x55953261be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:40.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.512+0000 7f32ce931b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x559532582540 con 0x559532619a70 2024-02-02T15:42:40.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.512+0000 7f32ce931b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x559532566b20 con 0x55953260eb90 2024-02-02T15:42:40.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.512+0000 7f32ce931b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595325644e0 con 0x55953260e270 2024-02-02T15:42:40.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.512+0000 7f32b3e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55953260e270 0x55953260e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:40.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.512+0000 7f32b3e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55953260e270 0x55953260e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:38406/0 (socket says 172.21.15.1:38406) 2024-02-02T15:42:40.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.512+0000 7f32b3e00700 1 -- 172.21.15.1:0/3690136742 learned_addr learned my addr 172.21.15.1:0/3690136742 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:42:40.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.512+0000 7f32b8e00700 1 --2- 172.21.15.1:0/3690136742 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559532619a70 0x55953261be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:40.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.513+0000 7f32b3e00700 1 -- 172.21.15.1:0/3690136742 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55953260eb90 msgr2=0x559532619530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:40.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.513+0000 7f32b3e00700 1 --2- 172.21.15.1:0/3690136742 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55953260eb90 0x559532619530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:40.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.513+0000 7f32b3e00700 1 -- 172.21.15.1:0/3690136742 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559532619a70 msgr2=0x55953261be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:40.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.513+0000 7f32b3e00700 1 --2- 172.21.15.1:0/3690136742 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559532619a70 0x55953261be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:40.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.513+0000 7f32b3e00700 1 -- 172.21.15.1:0/3690136742 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5595323e1430 con 0x55953260e270 2024-02-02T15:42:40.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.513+0000 7f32b8e00700 1 --2- 172.21.15.1:0/3690136742 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559532619a70 0x55953261be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:42:40.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.515+0000 7f32b3e00700 1 --2- 172.21.15.1:0/3690136742 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55953260e270 0x55953260e650 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f32a400ebd0 tx=0x7f32a4011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55d8c9efc0018e08 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:40.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.515+0000 7f32a9600700 1 -- 172.21.15.1:0/3690136742 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a4010cd0 con 0x55953260e270 2024-02-02T15:42:40.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.516+0000 7f32a9600700 1 -- 172.21.15.1:0/3690136742 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f32a4010e30 con 0x55953260e270 2024-02-02T15:42:40.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.516+0000 7f32a9600700 1 -- 172.21.15.1:0/3690136742 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a400f780 con 0x55953260e270 2024-02-02T15:42:40.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.517+0000 7f32ce931b80 1 -- 172.21.15.1:0/3690136742 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55953260e270 msgr2=0x55953260e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:40.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.517+0000 7f32ce931b80 1 --2- 172.21.15.1:0/3690136742 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55953260e270 0x55953260e650 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f32a400ebd0 tx=0x7f32a4011000 comp rx=0 tx=0).stop 2024-02-02T15:42:40.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.517+0000 7f32ce931b80 1 -- 172.21.15.1:0/3690136742 shutdown_connections 2024-02-02T15:42:40.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.517+0000 7f32ce931b80 1 --2- 172.21.15.1:0/3690136742 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x559532619a70 0x55953261be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:40.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.517+0000 7f32ce931b80 1 --2- 172.21.15.1:0/3690136742 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55953260e270 0x55953260e650 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:40.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.517+0000 7f32ce931b80 1 --2- 172.21.15.1:0/3690136742 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55953260eb90 0x559532619530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:40.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.517+0000 7f32ce931b80 1 -- 172.21.15.1:0/3690136742 >> 172.21.15.1:0/3690136742 conn(0x55953256de30 msgr2=0x55953260a640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:40.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.518+0000 7f32ce931b80 1 -- 172.21.15.1:0/3690136742 shutdown_connections 2024-02-02T15:42:40.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.518+0000 7f32ce931b80 1 -- 172.21.15.1:0/3690136742 wait complete. 2024-02-02T15:42:40.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.518+0000 7f32ce931b80 1 -- start start 2024-02-02T15:42:40.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.519+0000 7f32ce931b80 1 Processor -- start 2024-02-02T15:42:40.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.520+0000 7f32ce931b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55953260e270 0x5595326194e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:40.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.520+0000 7f32ce931b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55953260eb90 0x559532616760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:40.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.520+0000 7f32ce931b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559532619a70 0x559532616e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:40.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.520+0000 7f32ce931b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55953255fdd0 con 0x55953260e270 2024-02-02T15:42:40.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.520+0000 7f32ce931b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595324fbe00 con 0x559532619a70 2024-02-02T15:42:40.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.520+0000 7f32ce931b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595324fbaa0 con 0x55953260eb90 2024-02-02T15:42:40.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.520+0000 7f32ce931b80 10 client.0 _ll_register_callbacks cb 0x5595324fd310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:42:40.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.521+0000 7f32b3e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55953260e270 0x5595326194e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:40.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.521+0000 7f32b3e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55953260e270 0x5595326194e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:53298/0 (socket says 172.21.15.1:53298) 2024-02-02T15:42:40.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.521+0000 7f32b3e00700 1 -- 172.21.15.1:0/1847306063 learned_addr learned my addr 172.21.15.1:0/1847306063 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:42:40.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.521+0000 7f32b8e00700 1 --2- 172.21.15.1:0/1847306063 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559532619a70 0x559532616e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:40.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.522+0000 7f32b3e00700 1 -- 172.21.15.1:0/1847306063 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55953260eb90 msgr2=0x559532616760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:40.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.522+0000 7f32b3e00700 1 --2- 172.21.15.1:0/1847306063 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55953260eb90 0x559532616760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:40.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.522+0000 7f32b3e00700 1 -- 172.21.15.1:0/1847306063 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559532619a70 msgr2=0x559532616e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:40.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.522+0000 7f32b3e00700 1 --2- 172.21.15.1:0/1847306063 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559532619a70 0x559532616e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:40.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.522+0000 7f32b3e00700 1 -- 172.21.15.1:0/1847306063 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32b410d170 con 0x55953260e270 2024-02-02T15:42:40.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.522+0000 7f32b3e00700 1 --2- 172.21.15.1:0/1847306063 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55953260e270 0x5595326194e0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f32a400b010 tx=0x7f32a4005330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3cdde1a522d8f61 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:40.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.522+0000 7f32a2000700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:42:40.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.523+0000 7f32a2000700 1 -- 172.21.15.1:0/1847306063 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a402d070 con 0x55953260e270 2024-02-02T15:42:40.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.523+0000 7f32a2000700 1 -- 172.21.15.1:0/1847306063 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f32a40046f0 con 0x55953260e270 2024-02-02T15:42:40.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.523+0000 7f32ce931b80 1 -- 172.21.15.1:0/1847306063 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5595326023c0 con 0x55953260e270 2024-02-02T15:42:40.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.524+0000 7f32a2000700 1 -- 172.21.15.1:0/1847306063 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7f32a40048b0 con 0x55953260e270 2024-02-02T15:42:40.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.524+0000 7f32ce931b80 10 client.19581 fetch_fsmap learned FSMap version 63 2024-02-02T15:42:40.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.524+0000 7f32ce931b80 1 -- 172.21.15.1:0/1847306063 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) v3 -- 0x7f32a400e8b0 con 0x55953260e270 2024-02-02T15:42:40.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.525+0000 7f32a2000700 1 -- 172.21.15.1:0/1847306063 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5595326023c0 con 0x55953260e270 2024-02-02T15:42:40.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.525+0000 7f32a2000700 1 -- 172.21.15.1:0/1847306063 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 63) v1 ==== 7526+0+0 (secure 0 0 0) 0x7f32a4039070 con 0x55953260e270 2024-02-02T15:42:40.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.525+0000 7f32ce931b80 10 client.19581 fetch_fsmap finished waiting for FSMap version 63 2024-02-02T15:42:40.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.525+0000 7f32ce931b80 10 client.19581 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:42:40.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.525+0000 7f32ce931b80 20 client.19581 populate_metadata read hostname 'smithi001' 2024-02-02T15:42:40.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.525+0000 7f32ce931b80 1 --2- 172.21.15.1:0/1847306063 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55953263a3a0 0x55953263c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:40.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.525+0000 7f32ce931b80 4 client.19581 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:42:40.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.525+0000 7f32ce931b80 1 -- 172.21.15.1:0/1847306063 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559532558e50 con 0x55953263a3a0 2024-02-02T15:42:40.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.527+0000 7f32b3400700 1 --2- 172.21.15.1:0/1847306063 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55953263a3a0 0x55953263c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:40.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.527+0000 7f32b3400700 1 --2- 172.21.15.1:0/1847306063 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55953263a3a0 0x55953263c7c0 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fae446eb566b25d server_cookie=494699e64f2a803e in_seq=0 out_seq=0 2024-02-02T15:42:40.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.527+0000 7f32a2000700 10 client.19581 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:42:40.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.530+0000 7f32a2000700 1 -- 172.21.15.1:0/1847306063 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x559532558e50 con 0x55953263a3a0 2024-02-02T15:42:40.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.530+0000 7f32a2000700 10 client.19581 handle_command_reply: tid=0 2024-02-02T15:42:40.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.531+0000 7f32ce931b80 1 client.19581 shutdown 2024-02-02T15:42:40.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.531+0000 7f32ce931b80 1 -- 172.21.15.1:0/1847306063 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55953260e270 msgr2=0x5595326194e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:40.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.531+0000 7f32ce931b80 1 --2- 172.21.15.1:0/1847306063 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55953260e270 0x5595326194e0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f32a400b010 tx=0x7f32a4005330 comp rx=0 tx=0).stop 2024-02-02T15:42:40.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.531+0000 7f32ce931b80 1 -- 172.21.15.1:0/1847306063 shutdown_connections 2024-02-02T15:42:40.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.531+0000 7f32ce931b80 1 --2- 172.21.15.1:0/1847306063 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55953263a3a0 0x55953263c7c0 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:40.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.531+0000 7f32ce931b80 1 --2- 172.21.15.1:0/1847306063 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559532619a70 0x559532616e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:40.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.531+0000 7f32ce931b80 1 --2- 172.21.15.1:0/1847306063 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55953260e270 0x5595326194e0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:40.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.531+0000 7f32a2000700 0 client.19581 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:42:40.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.531+0000 7f32ce931b80 1 --2- 172.21.15.1:0/1847306063 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55953260eb90 0x559532616760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:40.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.531+0000 7f32ce931b80 1 -- 172.21.15.1:0/1847306063 >> 172.21.15.1:0/1847306063 conn(0x55953256de30 msgr2=0x55953260a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:40.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.532+0000 7f32ce931b80 1 -- 172.21.15.1:0/1847306063 shutdown_connections 2024-02-02T15:42:40.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.532+0000 7f32ce931b80 1 -- 172.21.15.1:0/1847306063 wait complete. 2024-02-02T15:42:40.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.532+0000 7f32ce931b80 20 client.19581 trim_cache size 0 max 16384 2024-02-02T15:42:40.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.557+0000 7f32ce931b80 1 Processor -- start 2024-02-02T15:42:40.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.557+0000 7f32ce931b80 1 -- start start 2024-02-02T15:42:40.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.558+0000 7f32ce931b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55953260e270 0x559532620560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:40.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.558+0000 7f32ce931b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55953260eb90 0x559532610a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:40.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.558+0000 7f32ce931b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559532619a70 0x559532615f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:40.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.558+0000 7f32ce931b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595323e3720 con 0x55953260e270 2024-02-02T15:42:40.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.558+0000 7f32ce931b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55953260ef70 con 0x55953260eb90 2024-02-02T15:42:40.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.558+0000 7f32ce931b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559532636ee0 con 0x559532619a70 2024-02-02T15:42:40.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.558+0000 7f32b3400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55953260eb90 0x559532610a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:40.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.558+0000 7f32b8e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559532619a70 0x559532615f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:40.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.558+0000 7f32b3400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55953260eb90 0x559532610a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:43876/0 (socket says 172.21.15.1:43876) 2024-02-02T15:42:40.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.558+0000 7f32b3400700 1 -- 172.21.15.1:0/3418522350 learned_addr learned my addr 172.21.15.1:0/3418522350 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:42:40.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.559+0000 7f32b3400700 1 -- 172.21.15.1:0/3418522350 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559532619a70 msgr2=0x559532615f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:40.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.559+0000 7f32b3400700 1 --2- 172.21.15.1:0/3418522350 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559532619a70 0x559532615f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:40.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.559+0000 7f32b3400700 1 -- 172.21.15.1:0/3418522350 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55953260e270 msgr2=0x559532620560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:40.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.559+0000 7f32b3400700 1 --2- 172.21.15.1:0/3418522350 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55953260e270 0x559532620560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:40.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.559+0000 7f32b3400700 1 -- 172.21.15.1:0/3418522350 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55953238dbe0 con 0x55953260eb90 2024-02-02T15:42:40.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.559+0000 7f32b3400700 1 --2- 172.21.15.1:0/3418522350 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55953260eb90 0x559532610a90 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f329c0990b0 tx=0x7f329c010f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6d41a10a2a9671e server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:40.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.559+0000 7f32aa000700 1 -- 172.21.15.1:0/3418522350 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329c099ab0 con 0x55953260eb90 2024-02-02T15:42:40.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.560+0000 7f32aa000700 1 -- 172.21.15.1:0/3418522350 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f329c0993b0 con 0x55953260eb90 2024-02-02T15:42:40.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.560+0000 7f32aa000700 1 -- 172.21.15.1:0/3418522350 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329c010710 con 0x55953260eb90 2024-02-02T15:42:40.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.560+0000 7f32ce931b80 1 -- 172.21.15.1:0/3418522350 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55953260eb90 msgr2=0x559532610a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:40.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.560+0000 7f32ce931b80 1 --2- 172.21.15.1:0/3418522350 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55953260eb90 0x559532610a90 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f329c0990b0 tx=0x7f329c010f80 comp rx=0 tx=0).stop 2024-02-02T15:42:40.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.562+0000 7f32ce931b80 1 -- 172.21.15.1:0/3418522350 shutdown_connections 2024-02-02T15:42:40.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.562+0000 7f32ce931b80 1 --2- 172.21.15.1:0/3418522350 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x559532619a70 0x559532615f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:40.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.562+0000 7f32ce931b80 1 --2- 172.21.15.1:0/3418522350 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55953260e270 0x559532620560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:40.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.562+0000 7f32ce931b80 1 --2- 172.21.15.1:0/3418522350 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55953260eb90 0x559532610a90 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:40.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.562+0000 7f32ce931b80 1 -- 172.21.15.1:0/3418522350 >> 172.21.15.1:0/3418522350 conn(0x55953256de30 msgr2=0x559532610380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:40.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.562+0000 7f32ce931b80 1 -- 172.21.15.1:0/3418522350 shutdown_connections 2024-02-02T15:42:40.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.562+0000 7f32ce931b80 1 -- 172.21.15.1:0/3418522350 wait complete. 2024-02-02T15:42:40.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.563+0000 7f32ce931b80 1 -- start start 2024-02-02T15:42:40.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.563+0000 7f32ce931b80 1 Processor -- start 2024-02-02T15:42:40.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.566+0000 7f32ce931b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55953260e270 0x5595326865e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:40.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.566+0000 7f32ce931b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55953260eb90 0x559532686b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:40.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.566+0000 7f32ce931b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559532619a70 0x55953261f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:40.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.566+0000 7f32ce931b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55953261fc00 con 0x55953260e270 2024-02-02T15:42:40.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.566+0000 7f32ce931b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55953261fd70 con 0x559532619a70 2024-02-02T15:42:40.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.566+0000 7f32ce931b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55953261ff30 con 0x55953260eb90 2024-02-02T15:42:40.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.566+0000 7f32ce931b80 10 client.0 _ll_register_callbacks cb 0x5595324fd310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:42:40.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.567+0000 7f32b8e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559532619a70 0x55953261f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:40.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.567+0000 7f32b8e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559532619a70 0x55953261f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:43890/0 (socket says 172.21.15.1:43890) 2024-02-02T15:42:40.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.567+0000 7f32b8e00700 1 -- 172.21.15.1:0/2510494746 learned_addr learned my addr 172.21.15.1:0/2510494746 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:42:40.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.567+0000 7f32b3400700 1 --2- 172.21.15.1:0/2510494746 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55953260eb90 0x559532686b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:40.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.567+0000 7f32b3e00700 1 --2- 172.21.15.1:0/2510494746 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55953260e270 0x5595326865e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:40.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.567+0000 7f32b8e00700 1 -- 172.21.15.1:0/2510494746 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55953260eb90 msgr2=0x559532686b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:40.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.567+0000 7f32b8e00700 1 --2- 172.21.15.1:0/2510494746 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55953260eb90 0x559532686b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:40.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.567+0000 7f32b8e00700 1 -- 172.21.15.1:0/2510494746 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55953260e270 msgr2=0x5595326865e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:40.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.567+0000 7f32b8e00700 1 --2- 172.21.15.1:0/2510494746 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55953260e270 0x5595326865e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:40.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.567+0000 7f32b8e00700 1 -- 172.21.15.1:0/2510494746 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32b4199f00 con 0x559532619a70 2024-02-02T15:42:40.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.568+0000 7f32b8e00700 1 --2- 172.21.15.1:0/2510494746 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559532619a70 0x55953261f4f0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f32ac009610 tx=0x7f32ac004f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7f12a09143c9210 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:40.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.568+0000 7f32a2000700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:42:40.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.568+0000 7f32a2000700 1 -- 172.21.15.1:0/2510494746 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ac00e040 con 0x559532619a70 2024-02-02T15:42:40.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.568+0000 7f32a2000700 1 -- 172.21.15.1:0/2510494746 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f32ac005040 con 0x559532619a70 2024-02-02T15:42:40.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.568+0000 7f32ce931b80 1 -- 172.21.15.1:0/2510494746 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f32a4010e30 con 0x559532619a70 2024-02-02T15:42:40.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.569+0000 7f32a2000700 1 -- 172.21.15.1:0/2510494746 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7f32ac007ad0 con 0x559532619a70 2024-02-02T15:42:40.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.569+0000 7f32ce931b80 10 client.29765 fetch_fsmap learned FSMap version 63 2024-02-02T15:42:40.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.569+0000 7f32ce931b80 1 -- 172.21.15.1:0/2510494746 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) v3 -- 0x559532525f20 con 0x559532619a70 2024-02-02T15:42:40.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.571+0000 7f32a2000700 1 -- 172.21.15.1:0/2510494746 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f32ac02e020 con 0x559532619a70 2024-02-02T15:42:40.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.571+0000 7f32a2000700 1 -- 172.21.15.1:0/2510494746 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 63) v1 ==== 7526+0+0 (secure 0 0 0) 0x7f32ac00b100 con 0x559532619a70 2024-02-02T15:42:40.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.571+0000 7f32ce931b80 10 client.29765 fetch_fsmap finished waiting for FSMap version 63 2024-02-02T15:42:40.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.572+0000 7f32ce931b80 10 client.29765 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:42:40.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.572+0000 7f32ce931b80 20 client.29765 populate_metadata read hostname 'smithi001' 2024-02-02T15:42:40.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.572+0000 7f32ce931b80 1 --2- 172.21.15.1:0/2510494746 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55953268e410 0x559532690830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:40.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.572+0000 7f32ce931b80 4 client.29765 mds_command: new command op to 24557 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-02T15:42:40.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.572+0000 7f32ce931b80 1 -- 172.21.15.1:0/2510494746 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55953251efd0 con 0x55953268e410 2024-02-02T15:42:40.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.572+0000 7f32b3e00700 1 --2- 172.21.15.1:0/2510494746 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55953268e410 0x559532690830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:40.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.573+0000 7f32b3e00700 1 --2- 172.21.15.1:0/2510494746 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55953268e410 0x559532690830 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be57bc936adecd92 server_cookie=c8766ef30d05c95b in_seq=0 out_seq=0 2024-02-02T15:42:40.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.573+0000 7f32a2000700 10 client.29765 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:42:40.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.585+0000 7f32a2000700 1 -- 172.21.15.1:0/2510494746 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55953251efd0 con 0x55953268e410 2024-02-02T15:42:40.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.585+0000 7f32a2000700 10 client.29765 handle_command_reply: tid=0 2024-02-02T15:42:40.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.585+0000 7f32ce931b80 1 client.29765 shutdown 2024-02-02T15:42:40.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.586+0000 7f32ce931b80 1 -- 172.21.15.1:0/2510494746 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559532619a70 msgr2=0x55953261f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:40.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.586+0000 7f32ce931b80 1 --2- 172.21.15.1:0/2510494746 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559532619a70 0x55953261f4f0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f32ac009610 tx=0x7f32ac004f70 comp rx=0 tx=0).stop 2024-02-02T15:42:40.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.587+0000 7f32ce931b80 1 -- 172.21.15.1:0/2510494746 shutdown_connections 2024-02-02T15:42:40.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.587+0000 7f32ce931b80 1 --2- 172.21.15.1:0/2510494746 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55953268e410 0x559532690830 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:40.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.587+0000 7f32ce931b80 1 --2- 172.21.15.1:0/2510494746 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559532619a70 0x55953261f4f0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:40.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.587+0000 7f32ce931b80 1 --2- 172.21.15.1:0/2510494746 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55953260e270 0x5595326865e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:40.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.587+0000 7f32ce931b80 1 --2- 172.21.15.1:0/2510494746 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55953260eb90 0x559532686b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:40.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.587+0000 7f32ce931b80 1 -- 172.21.15.1:0/2510494746 >> 172.21.15.1:0/2510494746 conn(0x55953256de30 msgr2=0x559532610c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:40.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.587+0000 7f32a2000700 0 client.29765 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:42:40.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.587+0000 7f32ce931b80 1 -- 172.21.15.1:0/2510494746 shutdown_connections 2024-02-02T15:42:40.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.588+0000 7f32ce931b80 1 -- 172.21.15.1:0/2510494746 wait complete. 2024-02-02T15:42:40.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.588+0000 7f32ce931b80 20 client.29765 trim_cache size 0 max 16384 2024-02-02T15:42:40.587 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:42:40.588 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2024-02-02T15:42:40.588 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "512abc79-7ad8-4664-8dfb-1b2bb445b29a", 2024-02-02T15:42:40.588 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2024-02-02T15:42:40.588 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:42:40.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.589+0000 7f32b9800700 1 -- 172.21.15.1:0/3539957601 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f32940a0820 msgr2=0x7f32940a2ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:40.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.589+0000 7f32b9800700 1 --2- 172.21.15.1:0/3539957601 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f32940a0820 0x7f32940a2ce0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f32a4005fd0 tx=0x7f32a4005db0 comp rx=0 tx=0).stop 2024-02-02T15:42:40.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.589+0000 7f32b9800700 1 -- 172.21.15.1:0/3539957601 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f32b41005b0 msgr2=0x7f32b4195700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:40.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.589+0000 7f32b9800700 1 --2- 172.21.15.1:0/3539957601 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f32b41005b0 0x7f32b4195700 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f329c00d950 tx=0x7f329c00dd10 comp rx=0 tx=0).stop 2024-02-02T15:42:40.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.589+0000 7f32b9800700 1 -- 172.21.15.1:0/3539957601 shutdown_connections 2024-02-02T15:42:40.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.589+0000 7f32b9800700 1 --2- 172.21.15.1:0/3539957601 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f32940a0820 0x7f32940a2ce0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:40.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.589+0000 7f32b9800700 1 --2- 172.21.15.1:0/3539957601 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f32b4105a20 0x7f32b41993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:40.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.589+0000 7f32b9800700 1 --2- 172.21.15.1:0/3539957601 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f32b40ffbf0 0x7f32b41951c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:40.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.589+0000 7f32b9800700 1 --2- 172.21.15.1:0/3539957601 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f32b41005b0 0x7f32b4195700 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:40.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.589+0000 7f32b9800700 1 -- 172.21.15.1:0/3539957601 >> 172.21.15.1:0/3539957601 conn(0x7f32b4076730 msgr2=0x7f32b41014a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:40.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.589+0000 7f32b9800700 1 -- 172.21.15.1:0/3539957601 shutdown_connections 2024-02-02T15:42:40.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:40.590+0000 7f32b9800700 1 -- 172.21.15.1:0/3539957601 wait complete. 2024-02-02T15:42:40.620 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-02T15:42:40.983 INFO:tasks.workunit.client.0.smithi001.stdout: 10 78656 37.53 MB/sec execute 13 sec latency 284.696 ms 2024-02-02T15:42:41.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.051+0000 7ff558200700 1 -- 172.21.15.1:0/683054704 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff550105020 msgr2=0x7ff55010cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:41.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.051+0000 7ff558200700 1 --2- 172.21.15.1:0/683054704 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff550105020 0x7ff55010cba0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7ff54800b0d0 tx=0x7ff54800b3e0 comp rx=0 tx=0).stop 2024-02-02T15:42:41.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.052+0000 7ff558200700 1 -- 172.21.15.1:0/683054704 shutdown_connections 2024-02-02T15:42:41.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.052+0000 7ff558200700 1 --2- 172.21.15.1:0/683054704 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff550105020 0x7ff55010cba0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:41.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.052+0000 7ff558200700 1 --2- 172.21.15.1:0/683054704 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5501000e0 0x7ff5501004c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:41.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.052+0000 7ff558200700 1 --2- 172.21.15.1:0/683054704 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff550100a90 0x7ff5501048f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:41.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.052+0000 7ff558200700 1 -- 172.21.15.1:0/683054704 >> 172.21.15.1:0/683054704 conn(0x7ff5500757d0 msgr2=0x7ff550075be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:41.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.053+0000 7ff558200700 1 -- 172.21.15.1:0/683054704 shutdown_connections 2024-02-02T15:42:41.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.053+0000 7ff558200700 1 -- 172.21.15.1:0/683054704 wait complete. 2024-02-02T15:42:41.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.055+0000 7ff558200700 1 Processor -- start 2024-02-02T15:42:41.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.056+0000 7ff558200700 1 -- start start 2024-02-02T15:42:41.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.057+0000 7ff558200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5501000e0 0x7ff550199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:41.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.057+0000 7ff558200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff550105020 0x7ff550199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:41.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.057+0000 7ff558200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff55019dbd0 0x7ff55019e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:41.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.057+0000 7ff558200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff55019e650 con 0x7ff550105020 2024-02-02T15:42:41.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.057+0000 7ff558200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff55019e7c0 con 0x7ff5501000e0 2024-02-02T15:42:41.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.057+0000 7ff558200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff55019e980 con 0x7ff55019dbd0 2024-02-02T15:42:41.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.057+0000 7ff556200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff55019dbd0 0x7ff55019e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:41.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.057+0000 7ff556200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff55019dbd0 0x7ff55019e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:38450/0 (socket says 172.21.15.1:38450) 2024-02-02T15:42:41.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.057+0000 7ff556200700 1 -- 172.21.15.1:0/1839452538 learned_addr learned my addr 172.21.15.1:0/1839452538 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:42:41.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.058+0000 7ff554e00700 1 --2- 172.21.15.1:0/1839452538 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff550105020 0x7ff550199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:41.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.058+0000 7ff554e00700 1 -- 172.21.15.1:0/1839452538 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff55019dbd0 msgr2=0x7ff55019e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:41.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.058+0000 7ff554e00700 1 --2- 172.21.15.1:0/1839452538 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff55019dbd0 0x7ff55019e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:41.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.058+0000 7ff554e00700 1 -- 172.21.15.1:0/1839452538 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5501000e0 msgr2=0x7ff550199230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:41.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.058+0000 7ff554e00700 1 --2- 172.21.15.1:0/1839452538 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5501000e0 0x7ff550199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:41.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.058+0000 7ff554e00700 1 -- 172.21.15.1:0/1839452538 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff548009d00 con 0x7ff550105020 2024-02-02T15:42:41.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.059+0000 7ff554e00700 1 --2- 172.21.15.1:0/1839452538 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff550105020 0x7ff550199770 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7ff550073cc0 tx=0x7ff53800dce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:41.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.060+0000 7ff54e000700 1 -- 172.21.15.1:0/1839452538 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff538009940 con 0x7ff550105020 2024-02-02T15:42:41.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.060+0000 7ff558200700 1 -- 172.21.15.1:0/1839452538 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5501a5ee0 con 0x7ff550105020 2024-02-02T15:42:41.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.060+0000 7ff558200700 1 -- 172.21.15.1:0/1839452538 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5501a63b0 con 0x7ff550105020 2024-02-02T15:42:41.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.060+0000 7ff54e000700 1 -- 172.21.15.1:0/1839452538 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff538010460 con 0x7ff550105020 2024-02-02T15:42:41.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.060+0000 7ff54e000700 1 -- 172.21.15.1:0/1839452538 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53800b5d0 con 0x7ff550105020 2024-02-02T15:42:41.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.072+0000 7ff54e000700 1 -- 172.21.15.1:0/1839452538 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7ff538009aa0 con 0x7ff550105020 2024-02-02T15:42:41.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.074+0000 7ff56bd3bb80 1 Processor -- start 2024-02-02T15:42:41.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.075+0000 7ff54e000700 1 --2- 172.21.15.1:0/1839452538 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff5300a0750 0x7ff5300a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:41.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.075+0000 7ff56bd3bb80 1 -- start start 2024-02-02T15:42:41.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.075+0000 7ff54e000700 1 -- 172.21.15.1:0/1839452538 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 9453+0+0 (secure 0 0 0) 0x7ff5380cbbc0 con 0x7ff550105020 2024-02-02T15:42:41.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.076+0000 7ff56bd3bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562068a30820 0x562068a30c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:41.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.076+0000 7ff56bd3bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562068a31140 0x562068a3bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:41.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.076+0000 7ff556200700 1 --2- 172.21.15.1:0/1839452538 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff5300a0750 0x7ff5300a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:41.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.076+0000 7ff56bd3bb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562068a3bff0 0x562068a3e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:41.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.076+0000 7ff56bd3bb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56206899c4c0 con 0x562068a3bff0 2024-02-02T15:42:41.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.076+0000 7ff56bd3bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x562068980aa0 con 0x562068a31140 2024-02-02T15:42:41.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.076+0000 7ff56bd3bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56206897e460 con 0x562068a30820 2024-02-02T15:42:41.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.076+0000 7ff554e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562068a31140 0x562068a3bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:41.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.076+0000 7ff554e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562068a31140 0x562068a3bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:43920/0 (socket says 172.21.15.1:43920) 2024-02-02T15:42:41.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.076+0000 7ff554e00700 1 -- 172.21.15.1:0/1839049308 learned_addr learned my addr 172.21.15.1:0/1839049308 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:42:41.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.076+0000 7ff556200700 1 --2- 172.21.15.1:0/1839049308 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562068a3bff0 0x562068a3e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:41.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.077+0000 7ff554e00700 1 -- 172.21.15.1:0/1839049308 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562068a30820 msgr2=0x562068a30c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:41.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.077+0000 7ff554e00700 1 --2- 172.21.15.1:0/1839049308 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562068a30820 0x562068a30c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:41.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.077+0000 7ff554e00700 1 -- 172.21.15.1:0/1839049308 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562068a3bff0 msgr2=0x562068a3e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:41.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.077+0000 7ff554e00700 1 --2- 172.21.15.1:0/1839049308 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562068a3bff0 0x562068a3e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:41.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.077+0000 7ff554e00700 1 -- 172.21.15.1:0/1839049308 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5620687fb3b0 con 0x562068a31140 2024-02-02T15:42:41.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.077+0000 7ff554e00700 1 --2- 172.21.15.1:0/1839049308 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562068a31140 0x562068a3bab0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7ff538005950 tx=0x7ff538099530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c26d712f34911c82 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:41.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.077+0000 7ff556200700 1 --2- 172.21.15.1:0/1839452538 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff5300a0750 0x7ff5300a2c10 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff5480062a0 tx=0x7ff5480095d0 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:41.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.077+0000 7ff546a00700 1 -- 172.21.15.1:0/1839049308 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5380a0db0 con 0x562068a31140 2024-02-02T15:42:41.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.077+0000 7ff546a00700 1 -- 172.21.15.1:0/1839049308 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff538009d50 con 0x562068a31140 2024-02-02T15:42:41.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.077+0000 7ff546a00700 1 -- 172.21.15.1:0/1839049308 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5380a0db0 con 0x562068a31140 2024-02-02T15:42:41.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.078+0000 7ff56bd3bb80 1 -- 172.21.15.1:0/1839049308 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562068a31140 msgr2=0x562068a3bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:41.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.078+0000 7ff56bd3bb80 1 --2- 172.21.15.1:0/1839049308 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562068a31140 0x562068a3bab0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7ff538005950 tx=0x7ff538099530 comp rx=0 tx=0).stop 2024-02-02T15:42:41.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.079+0000 7ff56bd3bb80 1 -- 172.21.15.1:0/1839049308 shutdown_connections 2024-02-02T15:42:41.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.079+0000 7ff56bd3bb80 1 --2- 172.21.15.1:0/1839049308 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562068a3bff0 0x562068a3e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:41.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.079+0000 7ff56bd3bb80 1 --2- 172.21.15.1:0/1839049308 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562068a30820 0x562068a30c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:41.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.079+0000 7ff56bd3bb80 1 --2- 172.21.15.1:0/1839049308 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562068a31140 0x562068a3bab0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:41.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.079+0000 7ff56bd3bb80 1 -- 172.21.15.1:0/1839049308 >> 172.21.15.1:0/1839049308 conn(0x562068987db0 msgr2=0x562068a243b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:41.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.079+0000 7ff56bd3bb80 1 -- 172.21.15.1:0/1839049308 shutdown_connections 2024-02-02T15:42:41.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.080+0000 7ff56bd3bb80 1 -- 172.21.15.1:0/1839049308 wait complete. 2024-02-02T15:42:41.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.081+0000 7ff56bd3bb80 1 -- start start 2024-02-02T15:42:41.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.082+0000 7ff56bd3bb80 1 Processor -- start 2024-02-02T15:42:41.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.084+0000 7ff56bd3bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562068a30820 0x562068a3b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:41.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.084+0000 7ff56bd3bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562068a31140 0x562068a38be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:41.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.086+0000 7ff555800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562068a30820 0x562068a3b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:41.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.086+0000 7ff555800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562068a30820 0x562068a3b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:43926/0 (socket says 172.21.15.1:43926) 2024-02-02T15:42:41.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.086+0000 7ff56bd3bb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562068a3bff0 0x562068a39210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:41.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.086+0000 7ff56bd3bb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x562068979d50 con 0x562068a3bff0 2024-02-02T15:42:41.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.086+0000 7ff56bd3bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x562068915d80 con 0x562068a30820 2024-02-02T15:42:41.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.086+0000 7ff56bd3bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x562068915a20 con 0x562068a31140 2024-02-02T15:42:41.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.086+0000 7ff56bd3bb80 10 client.0 _ll_register_callbacks cb 0x562068917290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:42:41.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.086+0000 7ff554e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562068a31140 0x562068a38be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:41.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.086+0000 7ff554e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562068a31140 0x562068a38be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:38466/0 (socket says 172.21.15.1:38466) 2024-02-02T15:42:41.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.086+0000 7ff554e00700 1 -- 172.21.15.1:0/1271813599 learned_addr learned my addr 172.21.15.1:0/1271813599 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:42:41.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.087+0000 7ff556200700 1 --2- 172.21.15.1:0/1271813599 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562068a3bff0 0x562068a39210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:41.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.087+0000 7ff554e00700 1 -- 172.21.15.1:0/1271813599 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562068a30820 msgr2=0x562068a3b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:41.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.087+0000 7ff554e00700 1 --2- 172.21.15.1:0/1271813599 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562068a30820 0x562068a3b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:41.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.087+0000 7ff554e00700 1 -- 172.21.15.1:0/1271813599 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562068a3bff0 msgr2=0x562068a39210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:41.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.087+0000 7ff554e00700 1 --2- 172.21.15.1:0/1271813599 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562068a3bff0 0x562068a39210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:41.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.087+0000 7ff554e00700 1 -- 172.21.15.1:0/1271813599 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5501a63b0 con 0x562068a31140 2024-02-02T15:42:41.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.089+0000 7ff554e00700 1 --2- 172.21.15.1:0/1271813599 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562068a31140 0x562068a38be0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7ff5380a0880 tx=0x7ff538012f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99b112d289d3918a server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:41.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.089+0000 7ff53f400700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:42:41.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.089+0000 7ff53f400700 1 -- 172.21.15.1:0/1271813599 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5380a0db0 con 0x562068a31140 2024-02-02T15:42:41.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.089+0000 7ff53f400700 1 -- 172.21.15.1:0/1271813599 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9453+0+0 (secure 0 0 0) 0x7ff5380991e0 con 0x562068a31140 2024-02-02T15:42:41.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.089+0000 7ff56bd3bb80 1 -- 172.21.15.1:0/1271813599 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562068a1c0f0 con 0x562068a31140 2024-02-02T15:42:41.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.089+0000 7ff53f400700 1 -- 172.21.15.1:0/1271813599 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff538098440 con 0x562068a31140 2024-02-02T15:42:41.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.089+0000 7ff56bd3bb80 10 client.29713 fetch_fsmap learned FSMap version 63 2024-02-02T15:42:41.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.089+0000 7ff56bd3bb80 1 -- 172.21.15.1:0/1271813599 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) v3 -- 0x5620687a7b60 con 0x562068a31140 2024-02-02T15:42:41.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.090+0000 7ff53f400700 1 -- 172.21.15.1:0/1271813599 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562068a1c0f0 con 0x562068a31140 2024-02-02T15:42:41.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.090+0000 7ff53f400700 1 -- 172.21.15.1:0/1271813599 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 63) v1 ==== 7526+0+0 (secure 0 0 0) 0x7ff5380b8040 con 0x562068a31140 2024-02-02T15:42:41.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.091+0000 7ff56bd3bb80 10 client.29713 fetch_fsmap finished waiting for FSMap version 63 2024-02-02T15:42:41.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.091+0000 7ff56bd3bb80 10 client.29713 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:42:41.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.091+0000 7ff56bd3bb80 20 client.29713 populate_metadata read hostname 'smithi001' 2024-02-02T15:42:41.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.091+0000 7ff56bd3bb80 1 --2- 172.21.15.1:0/1271813599 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x562068a4b660 0x562068a4da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:41.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.091+0000 7ff56bd3bb80 4 client.29713 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:42:41.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.091+0000 7ff56bd3bb80 1 -- 172.21.15.1:0/1271813599 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562068972dd0 con 0x562068a4b660 2024-02-02T15:42:41.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.092+0000 7ff555800700 1 --2- 172.21.15.1:0/1271813599 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x562068a4b660 0x562068a4da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:41.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.093+0000 7ff53f400700 10 client.29713 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:42:41.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.093+0000 7ff555800700 1 --2- 172.21.15.1:0/1271813599 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x562068a4b660 0x562068a4da80 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5274c08b94800bc server_cookie=e9b32b71bd096529 in_seq=0 out_seq=0 2024-02-02T15:42:41.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.094+0000 7ff53f400700 1 -- 172.21.15.1:0/1271813599 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x562068972dd0 con 0x562068a4b660 2024-02-02T15:42:41.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.094+0000 7ff53f400700 10 client.29713 handle_command_reply: tid=0 2024-02-02T15:42:41.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.095+0000 7ff56bd3bb80 1 client.29713 shutdown 2024-02-02T15:42:41.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.096+0000 7ff56bd3bb80 1 -- 172.21.15.1:0/1271813599 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562068a31140 msgr2=0x562068a38be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:41.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.096+0000 7ff56bd3bb80 1 --2- 172.21.15.1:0/1271813599 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562068a31140 0x562068a38be0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7ff5380a0880 tx=0x7ff538012f20 comp rx=0 tx=0).stop 2024-02-02T15:42:41.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.096+0000 7ff56bd3bb80 1 -- 172.21.15.1:0/1271813599 shutdown_connections 2024-02-02T15:42:41.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.096+0000 7ff56bd3bb80 1 --2- 172.21.15.1:0/1271813599 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x562068a4b660 0x562068a4da80 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:41.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.096+0000 7ff56bd3bb80 1 --2- 172.21.15.1:0/1271813599 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562068a3bff0 0x562068a39210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:41.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.096+0000 7ff56bd3bb80 1 --2- 172.21.15.1:0/1271813599 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562068a30820 0x562068a3b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:41.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.096+0000 7ff53f400700 0 client.29713 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:42:41.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.096+0000 7ff56bd3bb80 1 --2- 172.21.15.1:0/1271813599 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562068a31140 0x562068a38be0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:41.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.096+0000 7ff56bd3bb80 1 -- 172.21.15.1:0/1271813599 >> 172.21.15.1:0/1271813599 conn(0x562068987db0 msgr2=0x562068a23fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:41.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.097+0000 7ff56bd3bb80 1 -- 172.21.15.1:0/1271813599 shutdown_connections 2024-02-02T15:42:41.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.097+0000 7ff56bd3bb80 1 -- 172.21.15.1:0/1271813599 wait complete. 2024-02-02T15:42:41.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.097+0000 7ff56bd3bb80 20 client.29713 trim_cache size 0 max 16384 2024-02-02T15:42:41.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.137+0000 7ff56bd3bb80 1 Processor -- start 2024-02-02T15:42:41.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.138+0000 7ff56bd3bb80 1 -- start start 2024-02-02T15:42:41.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.138+0000 7ff56bd3bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562068a30820 0x562068a42bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:41.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.138+0000 7ff56bd3bb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562068a3bff0 0x562068a47180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:41.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.139+0000 7ff555800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562068a30820 0x562068a42bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:41.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.139+0000 7ff555800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562068a30820 0x562068a42bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:43942/0 (socket says 172.21.15.1:43942) 2024-02-02T15:42:41.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.139+0000 7ff555800700 1 -- 172.21.15.1:0/1478255951 learned_addr learned my addr 172.21.15.1:0/1478255951 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:42:41.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.139+0000 7ff56bd3bb80 1 --2- 172.21.15.1:0/1478255951 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562068a476c0 0x562068a23eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:41.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.139+0000 7ff56bd3bb80 1 -- 172.21.15.1:0/1478255951 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620687fd6a0 con 0x562068a3bff0 2024-02-02T15:42:41.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.139+0000 7ff56bd3bb80 1 -- 172.21.15.1:0/1478255951 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x562068972dd0 con 0x562068a30820 2024-02-02T15:42:41.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.139+0000 7ff56bd3bb80 1 -- 172.21.15.1:0/1478255951 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x562068a250b0 con 0x562068a476c0 2024-02-02T15:42:41.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.139+0000 7ff554e00700 1 --2- 172.21.15.1:0/1478255951 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562068a3bff0 0x562068a47180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:41.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.139+0000 7ff555800700 1 -- 172.21.15.1:0/1478255951 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562068a476c0 msgr2=0x562068a23eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:41.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.139+0000 7ff556200700 1 --2- 172.21.15.1:0/1478255951 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562068a476c0 0x562068a23eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:41.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.140+0000 7ff555800700 1 --2- 172.21.15.1:0/1478255951 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562068a476c0 0x562068a23eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:41.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.140+0000 7ff555800700 1 -- 172.21.15.1:0/1478255951 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562068a3bff0 msgr2=0x562068a47180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:41.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.140+0000 7ff555800700 1 --2- 172.21.15.1:0/1478255951 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562068a3bff0 0x562068a47180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:41.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.140+0000 7ff555800700 1 -- 172.21.15.1:0/1478255951 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56206893fea0 con 0x562068a30820 2024-02-02T15:42:41.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.140+0000 7ff556200700 1 --2- 172.21.15.1:0/1478255951 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562068a476c0 0x562068a23eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:42:41.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.140+0000 7ff555800700 1 --2- 172.21.15.1:0/1478255951 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562068a30820 0x562068a42bb0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7ff540000c00 tx=0x7ff5400169b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c7759bb8d6caa9d server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:41.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.141+0000 7ff547400700 1 -- 172.21.15.1:0/1478255951 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5400161a0 con 0x562068a30820 2024-02-02T15:42:41.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.141+0000 7ff547400700 1 -- 172.21.15.1:0/1478255951 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff540009d90 con 0x562068a30820 2024-02-02T15:42:41.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.141+0000 7ff547400700 1 -- 172.21.15.1:0/1478255951 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff540014bc0 con 0x562068a30820 2024-02-02T15:42:41.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.142+0000 7ff56bd3bb80 1 -- 172.21.15.1:0/1478255951 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562068a30820 msgr2=0x562068a42bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:41.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.142+0000 7ff56bd3bb80 1 --2- 172.21.15.1:0/1478255951 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562068a30820 0x562068a42bb0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7ff540000c00 tx=0x7ff5400169b0 comp rx=0 tx=0).stop 2024-02-02T15:42:41.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.143+0000 7ff56bd3bb80 1 -- 172.21.15.1:0/1478255951 shutdown_connections 2024-02-02T15:42:41.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.143+0000 7ff56bd3bb80 1 --2- 172.21.15.1:0/1478255951 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562068a476c0 0x562068a23eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:41.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.143+0000 7ff56bd3bb80 1 --2- 172.21.15.1:0/1478255951 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562068a30820 0x562068a42bb0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:41.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.143+0000 7ff56bd3bb80 1 --2- 172.21.15.1:0/1478255951 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562068a3bff0 0x562068a47180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:41.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.143+0000 7ff56bd3bb80 1 -- 172.21.15.1:0/1478255951 >> 172.21.15.1:0/1478255951 conn(0x562068987db0 msgr2=0x562068a33f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:41.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.143+0000 7ff56bd3bb80 1 -- 172.21.15.1:0/1478255951 shutdown_connections 2024-02-02T15:42:41.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.143+0000 7ff56bd3bb80 1 -- 172.21.15.1:0/1478255951 wait complete. 2024-02-02T15:42:41.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.144+0000 7ff56bd3bb80 1 -- start start 2024-02-02T15:42:41.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.144+0000 7ff56bd3bb80 1 Processor -- start 2024-02-02T15:42:41.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.145+0000 7ff56bd3bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562068a30820 0x562068a412f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:41.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.145+0000 7ff56bd3bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562068a3bff0 0x562068a41830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:41.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.145+0000 7ff56bd3bb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562068a476c0 0x562068a9d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:41.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.145+0000 7ff555800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562068a30820 0x562068a412f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:41.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.145+0000 7ff555800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562068a30820 0x562068a412f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:38492/0 (socket says 172.21.15.1:38492) 2024-02-02T15:42:41.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.145+0000 7ff555800700 1 -- 172.21.15.1:0/1306632821 learned_addr learned my addr 172.21.15.1:0/1306632821 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:42:41.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.145+0000 7ff56bd3bb80 1 -- 172.21.15.1:0/1306632821 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x562068a41fa0 con 0x562068a476c0 2024-02-02T15:42:41.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.145+0000 7ff56bd3bb80 1 -- 172.21.15.1:0/1306632821 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x562068a42110 con 0x562068a3bff0 2024-02-02T15:42:41.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.145+0000 7ff56bd3bb80 1 -- 172.21.15.1:0/1306632821 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x562068a9d6d0 con 0x562068a30820 2024-02-02T15:42:41.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.145+0000 7ff56bd3bb80 10 client.0 _ll_register_callbacks cb 0x562068a25b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:42:41.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.146+0000 7ff555800700 1 -- 172.21.15.1:0/1306632821 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562068a3bff0 msgr2=0x562068a41830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:41.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.146+0000 7ff554e00700 1 --2- 172.21.15.1:0/1306632821 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562068a3bff0 0x562068a41830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:41.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.146+0000 7ff555800700 1 --2- 172.21.15.1:0/1306632821 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562068a3bff0 0x562068a41830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:41.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.146+0000 7ff555800700 1 -- 172.21.15.1:0/1306632821 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562068a476c0 msgr2=0x562068a9d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:42:41.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.146+0000 7ff555800700 1 --2- 172.21.15.1:0/1306632821 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562068a476c0 0x562068a9d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:41.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.146+0000 7ff555800700 1 -- 172.21.15.1:0/1306632821 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff54000e040 con 0x562068a30820 2024-02-02T15:42:41.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.146+0000 7ff554e00700 1 --2- 172.21.15.1:0/1306632821 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562068a3bff0 0x562068a41830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:42:41.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.147+0000 7ff555800700 1 --2- 172.21.15.1:0/1306632821 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562068a30820 0x562068a412f0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7ff540014740 tx=0x7ff540016250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd5eae60f09d7e99 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:42:41.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.147+0000 7ff53f400700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:42:41.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.147+0000 7ff53f400700 1 -- 172.21.15.1:0/1306632821 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff540009ae0 con 0x562068a30820 2024-02-02T15:42:41.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.147+0000 7ff53f400700 1 -- 172.21.15.1:0/1306632821 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 9453+0+0 (secure 0 0 0) 0x7ff5400147a0 con 0x562068a30820 2024-02-02T15:42:41.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.147+0000 7ff56bd3bb80 1 -- 172.21.15.1:0/1306632821 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff538009d50 con 0x562068a30820 2024-02-02T15:42:41.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.148+0000 7ff53f400700 1 -- 172.21.15.1:0/1306632821 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff540014960 con 0x562068a30820 2024-02-02T15:42:41.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.148+0000 7ff56bd3bb80 10 client.29716 fetch_fsmap learned FSMap version 63 2024-02-02T15:42:41.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.148+0000 7ff56bd3bb80 1 -- 172.21.15.1:0/1306632821 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=4+}) v3 -- 0x5620687ffcd0 con 0x562068a30820 2024-02-02T15:42:41.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.148+0000 7ff53f400700 1 -- 172.21.15.1:0/1306632821 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff538009d50 con 0x562068a30820 2024-02-02T15:42:41.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.149+0000 7ff53f400700 1 -- 172.21.15.1:0/1306632821 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 63) v1 ==== 7526+0+0 (secure 0 0 0) 0x7ff540028020 con 0x562068a30820 2024-02-02T15:42:41.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.149+0000 7ff56bd3bb80 10 client.29716 fetch_fsmap finished waiting for FSMap version 63 2024-02-02T15:42:41.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.149+0000 7ff56bd3bb80 10 client.29716 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:42:41.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.150+0000 7ff56bd3bb80 20 client.29716 populate_metadata read hostname 'smithi001' 2024-02-02T15:42:41.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.150+0000 7ff56bd3bb80 1 --2- 172.21.15.1:0/1306632821 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x562068a9eb60 0x562068aa0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:42:41.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.150+0000 7ff56bd3bb80 4 client.29716 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2024-02-02T15:42:41.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.150+0000 7ff56bd3bb80 1 -- 172.21.15.1:0/1306632821 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562068938f50 con 0x562068a9eb60 2024-02-02T15:42:41.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.150+0000 7ff554e00700 1 --2- 172.21.15.1:0/1306632821 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x562068a9eb60 0x562068aa0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:42:41.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.150+0000 7ff554e00700 1 --2- 172.21.15.1:0/1306632821 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x562068a9eb60 0x562068aa0fa0 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2adc183984051c62 server_cookie=ab5718231889a00f in_seq=0 out_seq=0 2024-02-02T15:42:41.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.151+0000 7ff53f400700 10 client.29716 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:42:41.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.152+0000 7ff53f400700 1 -- 172.21.15.1:0/1306632821 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x562068938f50 con 0x562068a9eb60 2024-02-02T15:42:41.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.152+0000 7ff53f400700 10 client.29716 handle_command_reply: tid=0 2024-02-02T15:42:41.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.152+0000 7ff56bd3bb80 1 client.29716 shutdown 2024-02-02T15:42:41.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.153+0000 7ff56bd3bb80 1 -- 172.21.15.1:0/1306632821 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562068a30820 msgr2=0x562068a412f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:41.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.153+0000 7ff56bd3bb80 1 --2- 172.21.15.1:0/1306632821 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562068a30820 0x562068a412f0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7ff540014740 tx=0x7ff540016250 comp rx=0 tx=0).stop 2024-02-02T15:42:41.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.153+0000 7ff56bd3bb80 1 -- 172.21.15.1:0/1306632821 shutdown_connections 2024-02-02T15:42:41.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.153+0000 7ff56bd3bb80 1 --2- 172.21.15.1:0/1306632821 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x562068a9eb60 0x562068aa0fa0 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:41.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.153+0000 7ff56bd3bb80 1 --2- 172.21.15.1:0/1306632821 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562068a476c0 0x562068a9d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:41.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.153+0000 7ff56bd3bb80 1 --2- 172.21.15.1:0/1306632821 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562068a30820 0x562068a412f0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:41.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.153+0000 7ff53f400700 0 client.29716 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:42:41.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.153+0000 7ff56bd3bb80 1 --2- 172.21.15.1:0/1306632821 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562068a3bff0 0x562068a41830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:41.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.153+0000 7ff56bd3bb80 1 -- 172.21.15.1:0/1306632821 >> 172.21.15.1:0/1306632821 conn(0x562068987db0 msgr2=0x562068a3a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:41.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.154+0000 7ff56bd3bb80 1 -- 172.21.15.1:0/1306632821 shutdown_connections 2024-02-02T15:42:41.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.154+0000 7ff56bd3bb80 1 -- 172.21.15.1:0/1306632821 wait complete. 2024-02-02T15:42:41.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.154+0000 7ff56bd3bb80 20 client.29716 trim_cache size 0 max 16384 2024-02-02T15:42:41.154 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:42:41.154 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-02T15:42:41.154 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2024-02-02T15:42:41.154 INFO:teuthology.orchestra.run.smithi001.stdout: "512abc79-7ad8-4664-8dfb-1b2bb445b29a": { 2024-02-02T15:42:41.154 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2024-02-02T15:42:41.154 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "512abc79-7ad8-4664-8dfb-1b2bb445b29a", 2024-02-02T15:42:41.154 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,force" 2024-02-02T15:42:41.154 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:42:41.154 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:42:41.154 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:42:41.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.156+0000 7ff558200700 1 -- 172.21.15.1:0/1839452538 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff5300a0750 msgr2=0x7ff5300a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:41.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.156+0000 7ff558200700 1 --2- 172.21.15.1:0/1839452538 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff5300a0750 0x7ff5300a2c10 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff5480062a0 tx=0x7ff5480095d0 comp rx=0 tx=0).stop 2024-02-02T15:42:41.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.156+0000 7ff558200700 1 -- 172.21.15.1:0/1839452538 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff550105020 msgr2=0x7ff550199770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:42:41.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.156+0000 7ff558200700 1 --2- 172.21.15.1:0/1839452538 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff550105020 0x7ff550199770 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7ff550073cc0 tx=0x7ff53800dce0 comp rx=0 tx=0).stop 2024-02-02T15:42:41.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.156+0000 7ff558200700 1 -- 172.21.15.1:0/1839452538 shutdown_connections 2024-02-02T15:42:41.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.156+0000 7ff558200700 1 --2- 172.21.15.1:0/1839452538 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff5300a0750 0x7ff5300a2c10 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:41.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.156+0000 7ff558200700 1 --2- 172.21.15.1:0/1839452538 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff55019dbd0 0x7ff55019e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:41.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.156+0000 7ff558200700 1 --2- 172.21.15.1:0/1839452538 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5501000e0 0x7ff550199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:41.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.156+0000 7ff558200700 1 --2- 172.21.15.1:0/1839452538 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff550105020 0x7ff550199770 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:42:41.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.156+0000 7ff558200700 1 -- 172.21.15.1:0/1839452538 >> 172.21.15.1:0/1839452538 conn(0x7ff5500757d0 msgr2=0x7ff550068010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:42:41.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.157+0000 7ff558200700 1 -- 172.21.15.1:0/1839452538 shutdown_connections 2024-02-02T15:42:41.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:42:41.158+0000 7ff558200700 1 -- 172.21.15.1:0/1839452538 wait complete. 2024-02-02T15:42:41.194 INFO:tasks.cephfs.filesystem:scrub status for tag:512abc79-7ad8-4664-8dfb-1b2bb445b29a - {'path': '/', 'tag': '512abc79-7ad8-4664-8dfb-1b2bb445b29a', 'options': 'recursive,force'} 2024-02-02T15:42:41.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:42:41 smithi071 ceph-mon[44707]: pgmap v491: 129 pgs: 129 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 6.3 MiB/s wr, 459 op/s 2024-02-02T15:42:41.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:42:41 smithi071 ceph-mon[44707]: scrub queued for path: / 2024-02-02T15:42:41.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:42:41 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:42:41.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:42:41 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:42:41.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:42:41 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:42:41.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:42:41 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:42:41.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:42:41 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:42:41.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:42:41 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:42:41.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:42:41 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:42:41.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:42:41 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:42:41.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:42:41 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:42:41.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:42:41 smithi142 ceph-mon[45728]: pgmap v491: 129 pgs: 129 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 6.3 MiB/s wr, 459 op/s 2024-02-02T15:42:41.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:42:41 smithi142 ceph-mon[45728]: scrub queued for path: / 2024-02-02T15:42:41.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:42:41 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:42:41.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:42:41 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:42:41.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:42:41 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:42:41.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:42:41 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:42:41.922 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:42:41 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:42:41.922 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:42:41 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:42:41.922 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:42:41 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:42:41.922 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:42:41 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:42:41.922 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:42:41 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:42:41.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:42:41 smithi001 ceph-mon[35470]: pgmap v491: 129 pgs: 129 active+clean; 1.4 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 6.3 MiB/s wr, 459 op/s 2024-02-02T15:42:41.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:42:41 smithi001 ceph-mon[35470]: scrub queued for path: / 2024-02-02T15:42:41.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:42:41 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:42:41.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:42:41 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:42:41.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:42:41 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:42:41.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:42:41 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:42:41.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:42:41 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:42:41.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:42:41 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:42:41.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:42:41 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:42:41.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:42:41 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:42:41.942 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:42:41 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:42:41.984 INFO:tasks.workunit.client.0.smithi001.stdout: 10 78977 35.39 MB/sec execute 14 sec latency 422.518 ms 2024-02-02T15:42:42.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:42:42 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:42:42.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:42:42 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:42:42.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:42:42 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:42:42.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:42:42 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:42:42.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:42:42 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:42:42.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:42:42 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:42:42.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:42:42 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:42:42.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:42:42 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:42:42.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:42:42 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:42:42.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:42:42 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:42:42.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:42:42 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:42:42.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:42:42 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:42:42.984 INFO:tasks.workunit.client.0.smithi001.stdout: 10 79809 36.07 MB/sec execute 15 sec latency 65.863 ms 2024-02-02T15:42:43.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:42:43 smithi071 ceph-mon[44707]: pgmap v492: 129 pgs: 129 active+clean; 1.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 12 MiB/s wr, 702 op/s 2024-02-02T15:42:43.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:42:43 smithi142 ceph-mon[45728]: pgmap v492: 129 pgs: 129 active+clean; 1.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 12 MiB/s wr, 702 op/s 2024-02-02T15:42:43.939 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:42:43 smithi001 ceph-mon[35470]: pgmap v492: 129 pgs: 129 active+clean; 1.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 12 MiB/s wr, 702 op/s 2024-02-02T15:42:43.985 INFO:tasks.workunit.client.0.smithi001.stdout: 10 80758 36.75 MB/sec execute 16 sec latency 162.486 ms 2024-02-02T15:42:44.985 INFO:tasks.workunit.client.0.smithi001.stdout: 10 81482 37.62 MB/sec execute 17 sec latency 39.352 ms 2024-02-02T15:42:45.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:42:45 smithi071 ceph-mon[44707]: pgmap v493: 129 pgs: 129 active+clean; 1.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 8.7 MiB/s wr, 482 op/s 2024-02-02T15:42:45.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:42:45 smithi142 ceph-mon[45728]: pgmap v493: 129 pgs: 129 active+clean; 1.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 8.7 MiB/s wr, 482 op/s 2024-02-02T15:42:45.939 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:42:45 smithi001 ceph-mon[35470]: pgmap v493: 129 pgs: 129 active+clean; 1.4 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 8.7 MiB/s wr, 482 op/s 2024-02-02T15:42:45.985 INFO:tasks.workunit.client.0.smithi001.stdout: 10 82071 37.15 MB/sec execute 18 sec latency 58.767 ms 2024-02-02T15:42:46.985 INFO:tasks.workunit.client.0.smithi001.stdout: 10 82274 35.47 MB/sec execute 19 sec latency 434.008 ms 2024-02-02T15:42:47.867 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:42:47 smithi071 ceph-mon[44707]: pgmap v494: 129 pgs: 129 active+clean; 1.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 10 MiB/s wr, 616 op/s 2024-02-02T15:42:47.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:42:47 smithi142 ceph-mon[45728]: pgmap v494: 129 pgs: 129 active+clean; 1.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 10 MiB/s wr, 616 op/s 2024-02-02T15:42:47.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:42:47 smithi001 ceph-mon[35470]: pgmap v494: 129 pgs: 129 active+clean; 1.4 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 10 MiB/s wr, 616 op/s 2024-02-02T15:42:47.985 INFO:tasks.workunit.client.0.smithi001.stdout: 10 82507 34.38 MB/sec execute 20 sec latency 612.923 ms 2024-02-02T15:42:48.985 INFO:tasks.workunit.client.0.smithi001.stdout: 10 82832 33.38 MB/sec execute 21 sec latency 103.114 ms 2024-02-02T15:42:49.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:42:49 smithi071 ceph-mon[44707]: pgmap v495: 129 pgs: 129 active+clean; 1.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 593 op/s 2024-02-02T15:42:49.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:42:49 smithi001 ceph-mon[35470]: pgmap v495: 129 pgs: 129 active+clean; 1.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 593 op/s 2024-02-02T15:42:49.985 INFO:tasks.workunit.client.0.smithi001.stdout: 10 83403 33.38 MB/sec execute 22 sec latency 82.176 ms 2024-02-02T15:42:50.178 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:42:49 smithi142 ceph-mon[45728]: pgmap v495: 129 pgs: 129 active+clean; 1.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 593 op/s 2024-02-02T15:42:50.985 INFO:tasks.workunit.client.0.smithi001.stdout: 10 84354 34.16 MB/sec execute 23 sec latency 50.377 ms 2024-02-02T15:42:51.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:42:51 smithi071 ceph-mon[44707]: pgmap v496: 129 pgs: 129 active+clean; 1.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 9.5 MiB/s wr, 497 op/s 2024-02-02T15:42:51.922 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:42:51 smithi142 ceph-mon[45728]: pgmap v496: 129 pgs: 129 active+clean; 1.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 9.5 MiB/s wr, 497 op/s 2024-02-02T15:42:51.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:42:51 smithi001 ceph-mon[35470]: pgmap v496: 129 pgs: 129 active+clean; 1.4 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 9.5 MiB/s wr, 497 op/s 2024-02-02T15:42:51.986 INFO:tasks.workunit.client.0.smithi001.stdout: 10 84935 34.28 MB/sec execute 24 sec latency 81.776 ms 2024-02-02T15:42:52.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:42:52 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:42:52.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:42:52 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:42:52.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:42:52 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:42:52.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:42:52 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:42:52.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:42:52 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:42:52.923 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:42:52 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:42:52.923 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:42:52 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:42:52.923 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:42:52 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:42:52.923 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:42:52 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:42:52.923 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:42:52 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:42:52.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:42:52 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:42:52.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:42:52 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:42:52.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:42:52 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:42:52.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:42:52 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:42:52.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:42:52 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:42:52.985 INFO:tasks.workunit.client.0.smithi001.stdout: 10 84959 32.94 MB/sec execute 25 sec latency 1043.106 ms 2024-02-02T15:42:53.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:42:53 smithi071 ceph-mon[44707]: pgmap v497: 129 pgs: 129 active+clean; 1.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 16 MiB/s wr, 814 op/s 2024-02-02T15:42:53.927 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:42:53 smithi142 ceph-mon[45728]: pgmap v497: 129 pgs: 129 active+clean; 1.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 16 MiB/s wr, 814 op/s 2024-02-02T15:42:53.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:42:53 smithi001 ceph-mon[35470]: pgmap v497: 129 pgs: 129 active+clean; 1.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 16 MiB/s wr, 814 op/s 2024-02-02T15:42:53.986 INFO:tasks.workunit.client.0.smithi001.stdout: 10 85114 32.16 MB/sec execute 26 sec latency 1556.572 ms 2024-02-02T15:42:54.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:42:54 smithi142 ceph-mon[45728]: pgmap v498: 129 pgs: 129 active+clean; 1.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 13 MiB/s wr, 644 op/s 2024-02-02T15:42:54.922 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:42:54 smithi142 ceph-mon[45728]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:42:54.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:42:54 smithi001 ceph-mon[35470]: pgmap v498: 129 pgs: 129 active+clean; 1.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 13 MiB/s wr, 644 op/s 2024-02-02T15:42:54.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:42:54 smithi001 ceph-mon[35470]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:42:54.987 INFO:tasks.workunit.client.0.smithi001.stdout: 10 85361 31.46 MB/sec execute 27 sec latency 88.149 ms 2024-02-02T15:42:55.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:42:54 smithi071 ceph-mon[44707]: pgmap v498: 129 pgs: 129 active+clean; 1.4 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 13 MiB/s wr, 644 op/s 2024-02-02T15:42:55.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:42:54 smithi071 ceph-mon[44707]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:42:55.987 INFO:tasks.workunit.client.0.smithi001.stdout: 10 85617 30.71 MB/sec execute 28 sec latency 85.240 ms 2024-02-02T15:42:56.987 INFO:tasks.workunit.client.0.smithi001.stdout: 10 86038 30.08 MB/sec execute 29 sec latency 110.775 ms 2024-02-02T15:42:57.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:42:57 smithi071 ceph-mon[44707]: pgmap v499: 129 pgs: 129 active+clean; 1.4 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 20 MiB/s wr, 887 op/s 2024-02-02T15:42:57.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:42:57 smithi142 ceph-mon[45728]: pgmap v499: 129 pgs: 129 active+clean; 1.4 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 20 MiB/s wr, 887 op/s 2024-02-02T15:42:57.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:42:57 smithi001 ceph-mon[35470]: pgmap v499: 129 pgs: 129 active+clean; 1.4 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 20 MiB/s wr, 887 op/s 2024-02-02T15:42:57.986 INFO:tasks.workunit.client.0.smithi001.stdout: 10 86721 30.45 MB/sec execute 30 sec latency 53.627 ms 2024-02-02T15:42:58.987 INFO:tasks.workunit.client.0.smithi001.stdout: 10 87475 30.67 MB/sec execute 31 sec latency 60.397 ms 2024-02-02T15:42:59.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:42:59 smithi071 ceph-mon[44707]: pgmap v500: 129 pgs: 129 active+clean; 1.4 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 24 MiB/s wr, 947 op/s 2024-02-02T15:42:59.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:42:59 smithi142 ceph-mon[45728]: pgmap v500: 129 pgs: 129 active+clean; 1.4 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 24 MiB/s wr, 947 op/s 2024-02-02T15:42:59.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:42:59 smithi001 ceph-mon[35470]: pgmap v500: 129 pgs: 129 active+clean; 1.4 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 24 MiB/s wr, 947 op/s 2024-02-02T15:42:59.987 INFO:tasks.workunit.client.0.smithi001.stdout: 10 88144 31.17 MB/sec execute 32 sec latency 67.831 ms 2024-02-02T15:43:00.987 INFO:tasks.workunit.client.0.smithi001.stdout: 10 88742 31.23 MB/sec execute 33 sec latency 52.952 ms 2024-02-02T15:43:01.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:43:01 smithi071 ceph-mon[44707]: pgmap v501: 129 pgs: 129 active+clean; 1.4 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 21 MiB/s wr, 827 op/s 2024-02-02T15:43:01.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:43:01 smithi142 ceph-mon[45728]: pgmap v501: 129 pgs: 129 active+clean; 1.4 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 21 MiB/s wr, 827 op/s 2024-02-02T15:43:01.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:43:01 smithi001 ceph-mon[35470]: pgmap v501: 129 pgs: 129 active+clean; 1.4 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 21 MiB/s wr, 827 op/s 2024-02-02T15:43:01.987 INFO:tasks.workunit.client.0.smithi001.stdout: 10 88838 30.38 MB/sec execute 34 sec latency 533.570 ms 2024-02-02T15:43:02.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:43:02 smithi071 ceph-mon[44707]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:43:02.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:43:02 smithi142 ceph-mon[45728]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:43:02.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:43:02 smithi001 ceph-mon[35470]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:43:02.987 INFO:tasks.workunit.client.0.smithi001.stdout: 10 88840 29.53 MB/sec execute 35 sec latency 1310.275 ms 2024-02-02T15:43:03.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:43:03 smithi071 ceph-mon[44707]: pgmap v502: 129 pgs: 129 active+clean; 1.4 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 23 MiB/s wr, 990 op/s 2024-02-02T15:43:03.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:43:03 smithi142 ceph-mon[45728]: pgmap v502: 129 pgs: 129 active+clean; 1.4 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 23 MiB/s wr, 990 op/s 2024-02-02T15:43:03.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:43:03 smithi001 ceph-mon[35470]: pgmap v502: 129 pgs: 129 active+clean; 1.4 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 23 MiB/s wr, 990 op/s 2024-02-02T15:43:03.987 INFO:tasks.workunit.client.0.smithi001.stdout: 10 88840 28.71 MB/sec execute 36 sec latency 2310.395 ms 2024-02-02T15:43:04.988 INFO:tasks.workunit.client.0.smithi001.stdout: 10 88840 27.93 MB/sec execute 37 sec latency 3310.543 ms 2024-02-02T15:43:05.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:43:05 smithi071 ceph-mon[44707]: pgmap v503: 129 pgs: 129 active+clean; 1.5 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 17 MiB/s wr, 722 op/s 2024-02-02T15:43:05.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:43:05 smithi071 ceph-mon[44707]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:43:05.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:43:05 smithi142 ceph-mon[45728]: pgmap v503: 129 pgs: 129 active+clean; 1.5 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 17 MiB/s wr, 722 op/s 2024-02-02T15:43:05.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:43:05 smithi142 ceph-mon[45728]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:43:05.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:43:05 smithi001 ceph-mon[35470]: pgmap v503: 129 pgs: 129 active+clean; 1.5 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 17 MiB/s wr, 722 op/s 2024-02-02T15:43:05.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:43:05 smithi001 ceph-mon[35470]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:43:05.988 INFO:tasks.workunit.client.0.smithi001.stdout: 10 88859 27.20 MB/sec execute 38 sec latency 4277.780 ms 2024-02-02T15:43:06.988 INFO:tasks.workunit.client.0.smithi001.stdout: 10 89224 26.84 MB/sec execute 39 sec latency 378.021 ms 2024-02-02T15:43:07.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:43:07 smithi071 ceph-mon[44707]: pgmap v504: 129 pgs: 129 active+clean; 1.5 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 15 MiB/s wr, 676 op/s 2024-02-02T15:43:07.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:43:07 smithi071 ceph-mon[44707]: osdmap e108: 12 total, 12 up, 12 in 2024-02-02T15:43:07.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:43:07 smithi142 ceph-mon[45728]: pgmap v504: 129 pgs: 129 active+clean; 1.5 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 15 MiB/s wr, 676 op/s 2024-02-02T15:43:07.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:43:07 smithi142 ceph-mon[45728]: osdmap e108: 12 total, 12 up, 12 in 2024-02-02T15:43:07.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:43:07 smithi001 ceph-mon[35470]: pgmap v504: 129 pgs: 129 active+clean; 1.5 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 15 MiB/s wr, 676 op/s 2024-02-02T15:43:07.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:43:07 smithi001 ceph-mon[35470]: osdmap e108: 12 total, 12 up, 12 in 2024-02-02T15:43:07.988 INFO:tasks.workunit.client.0.smithi001.stdout: 10 89337 26.28 MB/sec execute 40 sec latency 355.342 ms 2024-02-02T15:43:08.988 INFO:tasks.workunit.client.0.smithi001.stdout: 10 89348 25.72 MB/sec execute 41 sec latency 245.508 ms 2024-02-02T15:43:09.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:43:09 smithi071 ceph-mon[44707]: pgmap v506: 129 pgs: 1 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 124 active+clean; 1.5 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.7 MiB/s wr, 342 op/s 2024-02-02T15:43:09.923 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:43:09 smithi142 ceph-mon[45728]: pgmap v506: 129 pgs: 1 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 124 active+clean; 1.5 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.7 MiB/s wr, 342 op/s 2024-02-02T15:43:09.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:43:09 smithi001 ceph-mon[35470]: pgmap v506: 129 pgs: 1 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 124 active+clean; 1.5 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.7 MiB/s wr, 342 op/s 2024-02-02T15:43:09.988 INFO:tasks.workunit.client.0.smithi001.stdout: 10 89458 25.22 MB/sec execute 42 sec latency 261.991 ms 2024-02-02T15:43:10.988 INFO:tasks.workunit.client.0.smithi001.stdout: 10 90168 25.59 MB/sec execute 43 sec latency 234.965 ms 2024-02-02T15:43:11.196 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-02T15:43:11.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.508+0000 7ff133200700 1 -- 172.21.15.1:0/54053808 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff12c1019a0 msgr2=0x7ff12c1059e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:11.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.508+0000 7ff133200700 1 --2- 172.21.15.1:0/54053808 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff12c1019a0 0x7ff12c1059e0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7ff114009990 tx=0x7ff114009ca0 comp rx=0 tx=0).stop 2024-02-02T15:43:11.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.508+0000 7ff133200700 1 -- 172.21.15.1:0/54053808 shutdown_connections 2024-02-02T15:43:11.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.508+0000 7ff133200700 1 --2- 172.21.15.1:0/54053808 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff12c106110 0x7ff12c10dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:11.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.508+0000 7ff133200700 1 --2- 172.21.15.1:0/54053808 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff12c100ff0 0x7ff12c1013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:11.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.508+0000 7ff133200700 1 --2- 172.21.15.1:0/54053808 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff12c1019a0 0x7ff12c1059e0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:11.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.508+0000 7ff133200700 1 -- 172.21.15.1:0/54053808 >> 172.21.15.1:0/54053808 conn(0x7ff12c0fc810 msgr2=0x7ff12c0fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:11.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.509+0000 7ff133200700 1 -- 172.21.15.1:0/54053808 shutdown_connections 2024-02-02T15:43:11.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.509+0000 7ff133200700 1 -- 172.21.15.1:0/54053808 wait complete. 2024-02-02T15:43:11.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.510+0000 7ff133200700 1 Processor -- start 2024-02-02T15:43:11.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.511+0000 7ff133200700 1 -- start start 2024-02-02T15:43:11.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.511+0000 7ff133200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff12c100ff0 0x7ff12c114d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:11.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.511+0000 7ff133200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff12c106110 0x7ff12c1152b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:11.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.511+0000 7ff133200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff12c11aad0 0x7ff12c10fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:11.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.511+0000 7ff133200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff12c11b150 con 0x7ff12c106110 2024-02-02T15:43:11.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.511+0000 7ff133200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff12c110330 con 0x7ff12c100ff0 2024-02-02T15:43:11.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.511+0000 7ff133200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff12c1104f0 con 0x7ff12c11aad0 2024-02-02T15:43:11.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.511+0000 7ff12be00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff12c100ff0 0x7ff12c114d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:11.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.511+0000 7ff12be00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff12c100ff0 0x7ff12c114d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:50078/0 (socket says 172.21.15.1:50078) 2024-02-02T15:43:11.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.511+0000 7ff12be00700 1 -- 172.21.15.1:0/2596867290 learned_addr learned my addr 172.21.15.1:0/2596867290 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:43:11.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.511+0000 7ff131200700 1 --2- 172.21.15.1:0/2596867290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff12c11aad0 0x7ff12c10fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:11.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.512+0000 7ff12be00700 1 -- 172.21.15.1:0/2596867290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff12c11aad0 msgr2=0x7ff12c10fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:11.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.512+0000 7ff12be00700 1 --2- 172.21.15.1:0/2596867290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff12c11aad0 0x7ff12c10fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:11.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.512+0000 7ff12be00700 1 -- 172.21.15.1:0/2596867290 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff12c106110 msgr2=0x7ff12c1152b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:11.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.512+0000 7ff12be00700 1 --2- 172.21.15.1:0/2596867290 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff12c106110 0x7ff12c1152b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:11.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.512+0000 7ff12be00700 1 -- 172.21.15.1:0/2596867290 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1140095f0 con 0x7ff12c100ff0 2024-02-02T15:43:11.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.512+0000 7ff12be00700 1 --2- 172.21.15.1:0/2596867290 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff12c100ff0 0x7ff12c114d50 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7ff11c00e9a0 tx=0x7ff11c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:11.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.513+0000 7ff128c00700 1 -- 172.21.15.1:0/2596867290 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff11c009790 con 0x7ff12c100ff0 2024-02-02T15:43:11.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.513+0000 7ff128c00700 1 -- 172.21.15.1:0/2596867290 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff11c00cbd0 con 0x7ff12c100ff0 2024-02-02T15:43:11.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.513+0000 7ff128c00700 1 -- 172.21.15.1:0/2596867290 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff11c018680 con 0x7ff12c100ff0 2024-02-02T15:43:11.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.513+0000 7ff133200700 1 -- 172.21.15.1:0/2596867290 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff12c1107d0 con 0x7ff12c100ff0 2024-02-02T15:43:11.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.513+0000 7ff133200700 1 -- 172.21.15.1:0/2596867290 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff12c1b30a0 con 0x7ff12c100ff0 2024-02-02T15:43:11.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.519+0000 7ff128c00700 1 -- 172.21.15.1:0/2596867290 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7ff11c00cd40 con 0x7ff12c100ff0 2024-02-02T15:43:11.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.520+0000 7ff146d72b80 1 Processor -- start 2024-02-02T15:43:11.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.520+0000 7ff146d72b80 1 -- start start 2024-02-02T15:43:11.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.521+0000 7ff146d72b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b8db09010 0x562b8db093f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:11.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.521+0000 7ff146d72b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562b8db09930 0x562b8db142a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:11.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.521+0000 7ff146d72b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562b8db147e0 0x562b8db16bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:11.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.521+0000 7ff146d72b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b8da7d4c0 con 0x562b8db147e0 2024-02-02T15:43:11.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.521+0000 7ff146d72b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b8da61aa0 con 0x562b8db09010 2024-02-02T15:43:11.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.521+0000 7ff146d72b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b8da5f460 con 0x562b8db09930 2024-02-02T15:43:11.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.521+0000 7ff131200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b8db09010 0x562b8db093f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:11.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.521+0000 7ff12be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562b8db09930 0x562b8db142a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:11.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.521+0000 7ff131200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b8db09010 0x562b8db093f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:50092/0 (socket says 172.21.15.1:50092) 2024-02-02T15:43:11.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.521+0000 7ff131200700 1 -- 172.21.15.1:0/3078997076 learned_addr learned my addr 172.21.15.1:0/3078997076 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:43:11.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.522+0000 7ff12b400700 1 --2- 172.21.15.1:0/3078997076 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562b8db147e0 0x562b8db16bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:11.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.522+0000 7ff131200700 1 -- 172.21.15.1:0/3078997076 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562b8db09930 msgr2=0x562b8db142a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:11.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.522+0000 7ff128c00700 1 --2- 172.21.15.1:0/2596867290 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff10c0a0710 0x7ff10c0a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:11.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.522+0000 7ff131200700 1 --2- 172.21.15.1:0/3078997076 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562b8db09930 0x562b8db142a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:11.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.522+0000 7ff131200700 1 -- 172.21.15.1:0/3078997076 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562b8db147e0 msgr2=0x562b8db16bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:11.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.522+0000 7ff131200700 1 --2- 172.21.15.1:0/3078997076 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562b8db147e0 0x562b8db16bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:11.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.522+0000 7ff131200700 1 -- 172.21.15.1:0/3078997076 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b8d8dc3b0 con 0x562b8db09010 2024-02-02T15:43:11.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.522+0000 7ff128c00700 1 -- 172.21.15.1:0/2596867290 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9541+0+0 (secure 0 0 0) 0x7ff11c014070 con 0x7ff12c100ff0 2024-02-02T15:43:11.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.522+0000 7ff131200700 1 --2- 172.21.15.1:0/2596867290 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff10c0a0710 0x7ff10c0a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:11.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.523+0000 7ff131200700 1 --2- 172.21.15.1:0/3078997076 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b8db09010 0x562b8db093f0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7ff12400a460 tx=0x7ff124009650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1e4b274f370387c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:11.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.523+0000 7ff121600700 1 -- 172.21.15.1:0/3078997076 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff124010040 con 0x562b8db09010 2024-02-02T15:43:11.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.523+0000 7ff121600700 1 -- 172.21.15.1:0/3078997076 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff12401b070 con 0x562b8db09010 2024-02-02T15:43:11.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.523+0000 7ff121600700 1 -- 172.21.15.1:0/3078997076 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff124014680 con 0x562b8db09010 2024-02-02T15:43:11.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.524+0000 7ff146d72b80 1 -- 172.21.15.1:0/3078997076 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b8db09010 msgr2=0x562b8db093f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:11.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.524+0000 7ff131200700 1 --2- 172.21.15.1:0/2596867290 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff10c0a0710 0x7ff10c0a2bd0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff12401e000 tx=0x7ff124021000 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:11.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.524+0000 7ff146d72b80 1 --2- 172.21.15.1:0/3078997076 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b8db09010 0x562b8db093f0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7ff12400a460 tx=0x7ff124009650 comp rx=0 tx=0).stop 2024-02-02T15:43:11.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.524+0000 7ff146d72b80 1 -- 172.21.15.1:0/3078997076 shutdown_connections 2024-02-02T15:43:11.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.524+0000 7ff146d72b80 1 --2- 172.21.15.1:0/3078997076 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562b8db147e0 0x562b8db16bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:11.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.524+0000 7ff146d72b80 1 --2- 172.21.15.1:0/3078997076 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b8db09010 0x562b8db093f0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:11.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.524+0000 7ff146d72b80 1 --2- 172.21.15.1:0/3078997076 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562b8db09930 0x562b8db142a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:11.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.524+0000 7ff146d72b80 1 -- 172.21.15.1:0/3078997076 >> 172.21.15.1:0/3078997076 conn(0x562b8da68db0 msgr2=0x562b8db053b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:11.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.525+0000 7ff146d72b80 1 -- 172.21.15.1:0/3078997076 shutdown_connections 2024-02-02T15:43:11.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.525+0000 7ff146d72b80 1 -- 172.21.15.1:0/3078997076 wait complete. 2024-02-02T15:43:11.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.526+0000 7ff146d72b80 1 -- start start 2024-02-02T15:43:11.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.526+0000 7ff146d72b80 1 Processor -- start 2024-02-02T15:43:11.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.527+0000 7ff146d72b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562b8db09010 0x562b8db14240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:11.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.528+0000 7ff146d72b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b8db09930 0x562b8db11450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:11.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.528+0000 7ff12be00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562b8db09010 0x562b8db14240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:11.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.528+0000 7ff12be00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562b8db09010 0x562b8db14240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60760/0 (socket says 172.21.15.1:60760) 2024-02-02T15:43:11.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.528+0000 7ff12be00700 1 -- 172.21.15.1:0/2553697953 learned_addr learned my addr 172.21.15.1:0/2553697953 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:43:11.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.529+0000 7ff12b400700 1 --2- 172.21.15.1:0/2553697953 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b8db09930 0x562b8db11450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:11.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.529+0000 7ff146d72b80 1 --2- 172.21.15.1:0/2553697953 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562b8db147e0 0x562b8db11b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:11.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.530+0000 7ff146d72b80 1 -- 172.21.15.1:0/2553697953 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b8da5ad50 con 0x562b8db09010 2024-02-02T15:43:11.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.530+0000 7ff146d72b80 1 -- 172.21.15.1:0/2553697953 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b8d9f6d80 con 0x562b8db09930 2024-02-02T15:43:11.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.530+0000 7ff146d72b80 1 -- 172.21.15.1:0/2553697953 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b8d9f6a20 con 0x562b8db147e0 2024-02-02T15:43:11.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.530+0000 7ff146d72b80 10 client.0 _ll_register_callbacks cb 0x562b8d9f8290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:43:11.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.530+0000 7ff12be00700 1 -- 172.21.15.1:0/2553697953 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562b8db147e0 msgr2=0x562b8db11b90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-02T15:43:11.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.530+0000 7ff12be00700 1 --2- 172.21.15.1:0/2553697953 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562b8db147e0 0x562b8db11b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:11.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.530+0000 7ff12be00700 1 -- 172.21.15.1:0/2553697953 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b8db09930 msgr2=0x562b8db11450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:11.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.530+0000 7ff12be00700 1 --2- 172.21.15.1:0/2553697953 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b8db09930 0x562b8db11450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:11.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.530+0000 7ff12be00700 1 -- 172.21.15.1:0/2553697953 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff124009c60 con 0x562b8db09010 2024-02-02T15:43:11.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.530+0000 7ff12b400700 1 --2- 172.21.15.1:0/2553697953 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b8db09930 0x562b8db11450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:43:11.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.531+0000 7ff12be00700 1 --2- 172.21.15.1:0/2553697953 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562b8db09010 0x562b8db14240 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7ff11c095f10 tx=0x7ff11c0c9c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9782396def0a4ac0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:11.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.531+0000 7ff11a000700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:43:11.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.531+0000 7ff11a000700 1 -- 172.21.15.1:0/2553697953 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff11c099ba0 con 0x562b8db09010 2024-02-02T15:43:11.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.531+0000 7ff11a000700 1 -- 172.21.15.1:0/2553697953 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9541+0+0 (secure 0 0 0) 0x7ff11c0994a0 con 0x562b8db09010 2024-02-02T15:43:11.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.532+0000 7ff146d72b80 1 -- 172.21.15.1:0/2553697953 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562b8dafd0f0 con 0x562b8db09010 2024-02-02T15:43:11.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.533+0000 7ff11a000700 1 -- 172.21.15.1:0/2553697953 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff11c09d2d0 con 0x562b8db09010 2024-02-02T15:43:11.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.533+0000 7ff146d72b80 10 client.19608 fetch_fsmap learned FSMap version 67 2024-02-02T15:43:11.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.533+0000 7ff146d72b80 1 -- 172.21.15.1:0/2553697953 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) v3 -- 0x562b8d888b60 con 0x562b8db09010 2024-02-02T15:43:11.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.533+0000 7ff11a000700 1 -- 172.21.15.1:0/2553697953 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562b8dafd0f0 con 0x562b8db09010 2024-02-02T15:43:11.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.533+0000 7ff11a000700 1 -- 172.21.15.1:0/2553697953 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 67) v1 ==== 7506+0+0 (secure 0 0 0) 0x7ff11c0a1080 con 0x562b8db09010 2024-02-02T15:43:11.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.534+0000 7ff146d72b80 10 client.19608 fetch_fsmap finished waiting for FSMap version 67 2024-02-02T15:43:11.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.534+0000 7ff146d72b80 10 client.19608 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:43:11.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.534+0000 7ff146d72b80 20 client.19608 populate_metadata read hostname 'smithi001' 2024-02-02T15:43:11.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.534+0000 7ff146d72b80 1 --2- 172.21.15.1:0/2553697953 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x562b8db24270 0x562b8db26690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:11.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.534+0000 7ff146d72b80 4 client.19608 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:43:11.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.534+0000 7ff146d72b80 1 -- 172.21.15.1:0/2553697953 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562b8da53dd0 con 0x562b8db24270 2024-02-02T15:43:11.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.535+0000 7ff12b400700 1 --2- 172.21.15.1:0/2553697953 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x562b8db24270 0x562b8db26690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:11.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.536+0000 7ff12b400700 1 --2- 172.21.15.1:0/2553697953 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x562b8db24270 0x562b8db26690 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64841fe32a840ff7 server_cookie=db3df698151cc2f6 in_seq=0 out_seq=0 2024-02-02T15:43:11.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.536+0000 7ff11a000700 10 client.19608 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:43:11.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.538+0000 7ff11a000700 1 -- 172.21.15.1:0/2553697953 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x562b8da53dd0 con 0x562b8db24270 2024-02-02T15:43:11.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.538+0000 7ff11a000700 10 client.19608 handle_command_reply: tid=0 2024-02-02T15:43:11.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.539+0000 7ff146d72b80 1 client.19608 shutdown 2024-02-02T15:43:11.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.539+0000 7ff146d72b80 1 -- 172.21.15.1:0/2553697953 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562b8db09010 msgr2=0x562b8db14240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:11.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.539+0000 7ff146d72b80 1 --2- 172.21.15.1:0/2553697953 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562b8db09010 0x562b8db14240 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7ff11c095f10 tx=0x7ff11c0c9c30 comp rx=0 tx=0).stop 2024-02-02T15:43:11.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.540+0000 7ff146d72b80 1 -- 172.21.15.1:0/2553697953 shutdown_connections 2024-02-02T15:43:11.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.540+0000 7ff146d72b80 1 --2- 172.21.15.1:0/2553697953 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x562b8db24270 0x562b8db26690 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:11.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.540+0000 7ff146d72b80 1 --2- 172.21.15.1:0/2553697953 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562b8db147e0 0x562b8db11b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:11.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.540+0000 7ff146d72b80 1 --2- 172.21.15.1:0/2553697953 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562b8db09010 0x562b8db14240 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:11.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.540+0000 7ff146d72b80 1 --2- 172.21.15.1:0/2553697953 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b8db09930 0x562b8db11450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:11.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.540+0000 7ff146d72b80 1 -- 172.21.15.1:0/2553697953 >> 172.21.15.1:0/2553697953 conn(0x562b8da68db0 msgr2=0x562b8db04fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:11.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.540+0000 7ff11a000700 0 client.19608 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:43:11.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.540+0000 7ff146d72b80 1 -- 172.21.15.1:0/2553697953 shutdown_connections 2024-02-02T15:43:11.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.541+0000 7ff146d72b80 1 -- 172.21.15.1:0/2553697953 wait complete. 2024-02-02T15:43:11.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.541+0000 7ff146d72b80 20 client.19608 trim_cache size 0 max 16384 2024-02-02T15:43:11.550 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:43:11 smithi001 ceph-mon[35470]: pgmap v507: 129 pgs: 1 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 124 active+clean; 1.5 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.7 MiB/s wr, 342 op/s 2024-02-02T15:43:11.550 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:43:11 smithi001 ceph-mon[35470]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:43:11.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.571+0000 7ff146d72b80 1 Processor -- start 2024-02-02T15:43:11.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.571+0000 7ff146d72b80 1 -- start start 2024-02-02T15:43:11.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.572+0000 7ff146d72b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562b8db09010 0x562b8db1b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:11.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.572+0000 7ff146d72b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b8db09930 0x562b8db1f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:11.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.572+0000 7ff146d72b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562b8db147e0 0x562b8db1fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:11.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.572+0000 7ff146d72b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b8d8de6a0 con 0x562b8db147e0 2024-02-02T15:43:11.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.572+0000 7ff146d72b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b8db05fb0 con 0x562b8db09930 2024-02-02T15:43:11.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.572+0000 7ff146d72b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b8db12190 con 0x562b8db09010 2024-02-02T15:43:11.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.572+0000 7ff12b400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b8db09930 0x562b8db1f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:11.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.572+0000 7ff12b400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b8db09930 0x562b8db1f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:50098/0 (socket says 172.21.15.1:50098) 2024-02-02T15:43:11.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.572+0000 7ff12b400700 1 -- 172.21.15.1:0/69824574 learned_addr learned my addr 172.21.15.1:0/69824574 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:43:11.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.573+0000 7ff12b400700 1 -- 172.21.15.1:0/69824574 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562b8db09010 msgr2=0x562b8db1b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:11.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.573+0000 7ff12b400700 1 --2- 172.21.15.1:0/69824574 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562b8db09010 0x562b8db1b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:11.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.573+0000 7ff12b400700 1 -- 172.21.15.1:0/69824574 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562b8db147e0 msgr2=0x562b8db1fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:11.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.573+0000 7ff12b400700 1 --2- 172.21.15.1:0/69824574 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562b8db147e0 0x562b8db1fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:11.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.573+0000 7ff12b400700 1 -- 172.21.15.1:0/69824574 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b8da20ea0 con 0x562b8db09930 2024-02-02T15:43:11.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.573+0000 7ff12b400700 1 --2- 172.21.15.1:0/69824574 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b8db09930 0x562b8db1f900 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7ff114006010 tx=0x7ff114005ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39c5a34455adfddc server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:11.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.573+0000 7ff122000700 1 -- 172.21.15.1:0/69824574 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff114026250 con 0x562b8db09930 2024-02-02T15:43:11.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.574+0000 7ff122000700 1 -- 172.21.15.1:0/69824574 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff11401e070 con 0x562b8db09930 2024-02-02T15:43:11.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.574+0000 7ff122000700 1 -- 172.21.15.1:0/69824574 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff114005400 con 0x562b8db09930 2024-02-02T15:43:11.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.574+0000 7ff146d72b80 1 -- 172.21.15.1:0/69824574 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b8db09930 msgr2=0x562b8db1f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:11.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.574+0000 7ff146d72b80 1 --2- 172.21.15.1:0/69824574 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b8db09930 0x562b8db1f900 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7ff114006010 tx=0x7ff114005ec0 comp rx=0 tx=0).stop 2024-02-02T15:43:11.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.575+0000 7ff146d72b80 1 -- 172.21.15.1:0/69824574 shutdown_connections 2024-02-02T15:43:11.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.575+0000 7ff146d72b80 1 --2- 172.21.15.1:0/69824574 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562b8db147e0 0x562b8db1fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:11.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.575+0000 7ff146d72b80 1 --2- 172.21.15.1:0/69824574 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562b8db09010 0x562b8db1b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:11.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.575+0000 7ff146d72b80 1 --2- 172.21.15.1:0/69824574 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b8db09930 0x562b8db1f900 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:11.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.575+0000 7ff146d72b80 1 -- 172.21.15.1:0/69824574 >> 172.21.15.1:0/69824574 conn(0x562b8da68db0 msgr2=0x562b8db0c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:11.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.575+0000 7ff146d72b80 1 -- 172.21.15.1:0/69824574 shutdown_connections 2024-02-02T15:43:11.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.578+0000 7ff146d72b80 1 -- 172.21.15.1:0/69824574 wait complete. 2024-02-02T15:43:11.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.579+0000 7ff146d72b80 1 -- start start 2024-02-02T15:43:11.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.579+0000 7ff146d72b80 1 Processor -- start 2024-02-02T15:43:11.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.580+0000 7ff146d72b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b8db09010 0x562b8db1af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:11.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.580+0000 7ff146d72b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562b8db09930 0x562b8db19210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:11.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.580+0000 7ff146d72b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562b8db147e0 0x562b8db19750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:11.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.580+0000 7ff146d72b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b8db19c90 con 0x562b8db09930 2024-02-02T15:43:11.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.580+0000 7ff146d72b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b8db19e00 con 0x562b8db09010 2024-02-02T15:43:11.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.580+0000 7ff146d72b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b8db19fc0 con 0x562b8db147e0 2024-02-02T15:43:11.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.580+0000 7ff146d72b80 10 client.0 _ll_register_callbacks cb 0x562b8d9f8290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:43:11.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.580+0000 7ff12be00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b8db09010 0x562b8db1af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:11.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.580+0000 7ff12be00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b8db09010 0x562b8db1af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:50102/0 (socket says 172.21.15.1:50102) 2024-02-02T15:43:11.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.581+0000 7ff12be00700 1 -- 172.21.15.1:0/382998810 learned_addr learned my addr 172.21.15.1:0/382998810 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:43:11.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.581+0000 7ff131200700 1 --2- 172.21.15.1:0/382998810 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562b8db147e0 0x562b8db19750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:11.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.581+0000 7ff12b400700 1 --2- 172.21.15.1:0/382998810 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562b8db09930 0x562b8db19210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:11.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.581+0000 7ff131200700 1 -- 172.21.15.1:0/382998810 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b8db09010 msgr2=0x562b8db1af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:11.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.581+0000 7ff131200700 1 --2- 172.21.15.1:0/382998810 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b8db09010 0x562b8db1af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:11.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.581+0000 7ff131200700 1 -- 172.21.15.1:0/382998810 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562b8db09930 msgr2=0x562b8db19210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:11.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.581+0000 7ff131200700 1 --2- 172.21.15.1:0/382998810 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562b8db09930 0x562b8db19210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:11.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.581+0000 7ff131200700 1 -- 172.21.15.1:0/382998810 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff12c10e500 con 0x562b8db147e0 2024-02-02T15:43:11.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.581+0000 7ff12be00700 1 --2- 172.21.15.1:0/382998810 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b8db09010 0x562b8db1af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:43:11.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.582+0000 7ff131200700 1 --2- 172.21.15.1:0/382998810 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562b8db147e0 0x562b8db19750 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x562b8db06800 tx=0x7ff124011f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=50304dffe5e1e4da server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:11.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.582+0000 7ff11a000700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:43:11.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.582+0000 7ff11a000700 1 -- 172.21.15.1:0/382998810 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff124010030 con 0x562b8db147e0 2024-02-02T15:43:11.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.582+0000 7ff11a000700 1 -- 172.21.15.1:0/382998810 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9541+0+0 (secure 0 0 0) 0x7ff124006e40 con 0x562b8db147e0 2024-02-02T15:43:11.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.582+0000 7ff146d72b80 1 -- 172.21.15.1:0/382998810 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff12401b070 con 0x562b8db147e0 2024-02-02T15:43:11.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.583+0000 7ff11a000700 1 -- 172.21.15.1:0/382998810 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff124014640 con 0x562b8db147e0 2024-02-02T15:43:11.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.583+0000 7ff146d72b80 10 client.29725 fetch_fsmap learned FSMap version 67 2024-02-02T15:43:11.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.583+0000 7ff146d72b80 1 -- 172.21.15.1:0/382998810 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) v3 -- 0x562b8d8e0cd0 con 0x562b8db147e0 2024-02-02T15:43:11.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.583+0000 7ff11a000700 1 -- 172.21.15.1:0/382998810 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff12401b070 con 0x562b8db147e0 2024-02-02T15:43:11.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.584+0000 7ff11a000700 1 -- 172.21.15.1:0/382998810 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 67) v1 ==== 7506+0+0 (secure 0 0 0) 0x7ff12402b950 con 0x562b8db147e0 2024-02-02T15:43:11.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.584+0000 7ff146d72b80 10 client.29725 fetch_fsmap finished waiting for FSMap version 67 2024-02-02T15:43:11.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.584+0000 7ff146d72b80 10 client.29725 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:43:11.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.584+0000 7ff146d72b80 20 client.29725 populate_metadata read hostname 'smithi001' 2024-02-02T15:43:11.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.584+0000 7ff146d72b80 1 --2- 172.21.15.1:0/382998810 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x562b8db77260 0x562b8db79680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:11.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.584+0000 7ff146d72b80 4 client.29725 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2024-02-02T15:43:11.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.584+0000 7ff146d72b80 1 -- 172.21.15.1:0/382998810 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562b8da19f50 con 0x562b8db77260 2024-02-02T15:43:11.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.584+0000 7ff12be00700 1 --2- 172.21.15.1:0/382998810 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x562b8db77260 0x562b8db79680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:11.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.585+0000 7ff12be00700 1 --2- 172.21.15.1:0/382998810 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x562b8db77260 0x562b8db79680 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=919c157ff62117e7 server_cookie=3977711cbb6947a1 in_seq=0 out_seq=0 2024-02-02T15:43:11.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.585+0000 7ff11a000700 10 client.29725 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:43:11.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.597+0000 7ff11a000700 1 -- 172.21.15.1:0/382998810 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x562b8da19f50 con 0x562b8db77260 2024-02-02T15:43:11.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.597+0000 7ff11a000700 10 client.29725 handle_command_reply: tid=0 2024-02-02T15:43:11.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.597+0000 7ff146d72b80 1 client.29725 shutdown 2024-02-02T15:43:11.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.597+0000 7ff146d72b80 1 -- 172.21.15.1:0/382998810 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562b8db147e0 msgr2=0x562b8db19750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:11.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.597+0000 7ff146d72b80 1 --2- 172.21.15.1:0/382998810 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562b8db147e0 0x562b8db19750 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x562b8db06800 tx=0x7ff124011f60 comp rx=0 tx=0).stop 2024-02-02T15:43:11.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.597+0000 7ff146d72b80 1 -- 172.21.15.1:0/382998810 shutdown_connections 2024-02-02T15:43:11.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.597+0000 7ff146d72b80 1 --2- 172.21.15.1:0/382998810 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x562b8db77260 0x562b8db79680 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:11.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.597+0000 7ff146d72b80 1 --2- 172.21.15.1:0/382998810 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562b8db147e0 0x562b8db19750 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:11.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.597+0000 7ff146d72b80 1 --2- 172.21.15.1:0/382998810 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562b8db09010 0x562b8db1af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:11.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.597+0000 7ff11a000700 0 client.29725 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:43:11.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.597+0000 7ff146d72b80 1 --2- 172.21.15.1:0/382998810 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562b8db09930 0x562b8db19210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:11.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.597+0000 7ff146d72b80 1 -- 172.21.15.1:0/382998810 >> 172.21.15.1:0/382998810 conn(0x562b8da68db0 msgr2=0x562b8db0c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:11.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.597+0000 7ff146d72b80 1 -- 172.21.15.1:0/382998810 shutdown_connections 2024-02-02T15:43:11.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.597+0000 7ff146d72b80 1 -- 172.21.15.1:0/382998810 wait complete. 2024-02-02T15:43:11.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.598+0000 7ff146d72b80 20 client.29725 trim_cache size 0 max 16384 2024-02-02T15:43:11.599 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:43:11.599 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2024-02-02T15:43:11.599 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2024-02-02T15:43:11.599 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:43:11.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.598+0000 7ff133200700 1 -- 172.21.15.1:0/2596867290 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff10c0a0710 msgr2=0x7ff10c0a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:11.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.598+0000 7ff133200700 1 --2- 172.21.15.1:0/2596867290 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff10c0a0710 0x7ff10c0a2bd0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff12401e000 tx=0x7ff124021000 comp rx=0 tx=0).stop 2024-02-02T15:43:11.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.598+0000 7ff133200700 1 -- 172.21.15.1:0/2596867290 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff12c100ff0 msgr2=0x7ff12c114d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:11.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.598+0000 7ff133200700 1 --2- 172.21.15.1:0/2596867290 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff12c100ff0 0x7ff12c114d50 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7ff11c00e9a0 tx=0x7ff11c00ecb0 comp rx=0 tx=0).stop 2024-02-02T15:43:11.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.598+0000 7ff133200700 1 -- 172.21.15.1:0/2596867290 shutdown_connections 2024-02-02T15:43:11.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.598+0000 7ff133200700 1 --2- 172.21.15.1:0/2596867290 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff10c0a0710 0x7ff10c0a2bd0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:11.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.598+0000 7ff133200700 1 --2- 172.21.15.1:0/2596867290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff12c11aad0 0x7ff12c10fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:11.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.598+0000 7ff133200700 1 --2- 172.21.15.1:0/2596867290 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff12c100ff0 0x7ff12c114d50 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:11.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.598+0000 7ff133200700 1 --2- 172.21.15.1:0/2596867290 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff12c106110 0x7ff12c1152b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:11.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.598+0000 7ff133200700 1 -- 172.21.15.1:0/2596867290 >> 172.21.15.1:0/2596867290 conn(0x7ff12c0fc810 msgr2=0x7ff12c0fd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:11.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.599+0000 7ff133200700 1 -- 172.21.15.1:0/2596867290 shutdown_connections 2024-02-02T15:43:11.601 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.599+0000 7ff133200700 1 -- 172.21.15.1:0/2596867290 wait complete. 2024-02-02T15:43:11.623 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-02T15:43:11.623 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T15:43:11.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:43:11 smithi071 ceph-mon[44707]: pgmap v507: 129 pgs: 1 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 124 active+clean; 1.5 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.7 MiB/s wr, 342 op/s 2024-02-02T15:43:11.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:43:11 smithi071 ceph-mon[44707]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:43:11.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:43:11 smithi142 ceph-mon[45728]: pgmap v507: 129 pgs: 1 active+clean+snaptrim, 4 active+clean+snaptrim_wait, 124 active+clean; 1.5 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.7 MiB/s wr, 342 op/s 2024-02-02T15:43:11.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:43:11 smithi142 ceph-mon[45728]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:43:11.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.940+0000 7f4102600700 1 -- 172.21.15.1:0/1834269407 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40fc100f60 msgr2=0x7f40fc101340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:11.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.940+0000 7f4102600700 1 --2- 172.21.15.1:0/1834269407 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40fc100f60 0x7f40fc101340 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f40ec009990 tx=0x7f40ec009ca0 comp rx=0 tx=0).stop 2024-02-02T15:43:11.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.941+0000 7f4102600700 1 -- 172.21.15.1:0/1834269407 shutdown_connections 2024-02-02T15:43:11.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.941+0000 7f4102600700 1 --2- 172.21.15.1:0/1834269407 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40fc106080 0x7f40fc10dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:11.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.941+0000 7f4102600700 1 --2- 172.21.15.1:0/1834269407 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40fc100f60 0x7f40fc101340 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:11.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.941+0000 7f4102600700 1 --2- 172.21.15.1:0/1834269407 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40fc101910 0x7f40fc105950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:11.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.941+0000 7f4102600700 1 -- 172.21.15.1:0/1834269407 >> 172.21.15.1:0/1834269407 conn(0x7f40fc0fc800 msgr2=0x7f40fc0fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:11.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.942+0000 7f4102600700 1 -- 172.21.15.1:0/1834269407 shutdown_connections 2024-02-02T15:43:11.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.942+0000 7f4102600700 1 -- 172.21.15.1:0/1834269407 wait complete. 2024-02-02T15:43:11.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.943+0000 7f4102600700 1 Processor -- start 2024-02-02T15:43:11.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.943+0000 7f4102600700 1 -- start start 2024-02-02T15:43:11.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.944+0000 7f4102600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40fc100f60 0x7f40fc1990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:11.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.944+0000 7f4102600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40fc106080 0x7f40fc199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:11.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.944+0000 7f4102600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40fc19da90 0x7f40fc19df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:11.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.944+0000 7f4102600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40fc19e510 con 0x7f40fc100f60 2024-02-02T15:43:11.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.944+0000 7f4102600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40fc19e680 con 0x7f40fc19da90 2024-02-02T15:43:11.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.944+0000 7f4102600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40fc19e840 con 0x7f40fc106080 2024-02-02T15:43:11.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.945+0000 7f40f3e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40fc106080 0x7f40fc199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:11.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.945+0000 7f40f3e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40fc106080 0x7f40fc199630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:37784/0 (socket says 172.21.15.1:37784) 2024-02-02T15:43:11.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.945+0000 7f40f3e00700 1 -- 172.21.15.1:0/4105073 learned_addr learned my addr 172.21.15.1:0/4105073 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:43:11.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.945+0000 7f40fbe00700 1 --2- 172.21.15.1:0/4105073 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40fc19da90 0x7f40fc19df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:11.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.945+0000 7f40fb400700 1 --2- 172.21.15.1:0/4105073 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40fc100f60 0x7f40fc1990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:11.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.946+0000 7f40fb400700 1 -- 172.21.15.1:0/4105073 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40fc106080 msgr2=0x7f40fc199630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:11.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.946+0000 7f40fb400700 1 --2- 172.21.15.1:0/4105073 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40fc106080 0x7f40fc199630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:11.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.946+0000 7f40fb400700 1 -- 172.21.15.1:0/4105073 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40fc19da90 msgr2=0x7f40fc19df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:11.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.946+0000 7f40fb400700 1 --2- 172.21.15.1:0/4105073 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40fc19da90 0x7f40fc19df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:11.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.946+0000 7f40fb400700 1 -- 172.21.15.1:0/4105073 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40ec0095f0 con 0x7f40fc100f60 2024-02-02T15:43:11.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.947+0000 7f40fb400700 1 --2- 172.21.15.1:0/4105073 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40fc100f60 0x7f40fc1990f0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f40ec0098e0 tx=0x7f40ec005da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:11.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.948+0000 7f40f8c00700 1 -- 172.21.15.1:0/4105073 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40ec01d070 con 0x7f40fc100f60 2024-02-02T15:43:11.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.948+0000 7f40f8c00700 1 -- 172.21.15.1:0/4105073 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f40ec022470 con 0x7f40fc100f60 2024-02-02T15:43:11.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.948+0000 7f40f8c00700 1 -- 172.21.15.1:0/4105073 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40ec017650 con 0x7f40fc100f60 2024-02-02T15:43:11.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.948+0000 7f4102600700 1 -- 172.21.15.1:0/4105073 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40fc1a5d40 con 0x7f40fc100f60 2024-02-02T15:43:11.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.949+0000 7f4102600700 1 -- 172.21.15.1:0/4105073 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40fc1a6110 con 0x7f40fc100f60 2024-02-02T15:43:11.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.950+0000 7f4102600700 1 -- 172.21.15.1:0/4105073 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40fc067330 con 0x7f40fc100f60 2024-02-02T15:43:11.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.954+0000 7f40f8c00700 1 -- 172.21.15.1:0/4105073 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f40ec0225e0 con 0x7f40fc100f60 2024-02-02T15:43:11.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.956+0000 7f40f8c00700 1 --2- 172.21.15.1:0/4105073 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f40dc0a07e0 0x7f40dc0a2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:11.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.956+0000 7f40f8c00700 1 -- 172.21.15.1:0/4105073 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f40ec0cd780 con 0x7f40fc100f60 2024-02-02T15:43:11.958 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.959+0000 7f40f3e00700 1 --2- 172.21.15.1:0/4105073 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f40dc0a07e0 0x7f40dc0a2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:11.962 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.963+0000 7f40f3e00700 1 --2- 172.21.15.1:0/4105073 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f40dc0a07e0 0x7f40dc0a2ca0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f40fc19a700 tx=0x7f40e4008040 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:11.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:11.964+0000 7f40f8c00700 1 -- 172.21.15.1:0/4105073 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f40ec096db0 con 0x7f40fc100f60 2024-02-02T15:43:11.988 INFO:tasks.workunit.client.0.smithi001.stdout: 10 90949 25.87 MB/sec execute 44 sec latency 54.233 ms 2024-02-02T15:43:12.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.452+0000 7f4102600700 1 -- 172.21.15.1:0/4105073 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f40fc101ae0 con 0x7f40fc100f60 2024-02-02T15:43:12.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.454+0000 7f40f8c00700 1 -- 172.21.15.1:0/4105073 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+10881 (secure 0 0 0) 0x7f40ec027090 con 0x7f40fc100f60 2024-02-02T15:43:12.454 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:43:12.454 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":67,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2922240448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2922240448},{"type":"v1","addr":"172.21.15.1:6835","nonce":2922240448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2729497431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2729497431},{"type":"v1","addr":"172.21.15.1:6837","nonce":2729497431}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":67,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-02T15:24:47.761435+0000","modified":"2024-02-02T15:43:10.541497+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24529,"mds_2":24518,"mds_3":24514,"mds_4":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14517":{"gid":14517,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/1218303683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":1218303683},{"type":"v1","addr":"172.21.15.1:6839","nonce":1218303683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14529":{"gid":14529,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/2059197496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2059197496},{"type":"v1","addr":"172.21.15.1:6841","nonce":2059197496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/7648986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":7648986},{"type":"v1","addr":"172.21.15.142:6833","nonce":7648986}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/3482746094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3482746094},{"type":"v1","addr":"172.21.15.71:6833","nonce":3482746094}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/388207814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":388207814},{"type":"v1","addr":"172.21.15.71:6835","nonce":388207814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6835/1052606547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1052606547},{"type":"v1","addr":"172.21.15.142:6835","nonce":1052606547}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.142:6837/2870289019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2870289019},{"type":"v1","addr":"172.21.15.142:6837","nonce":2870289019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.71:6837/628016199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":628016199},{"type":"v1","addr":"172.21.15.71:6837","nonce":628016199}]},"join_fscid":-1,"export_targets":[0,1,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.142:6839/321675421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":321675421},{"type":"v1","addr":"172.21.15.142:6839","nonce":321675421}]},"join_fscid":-1,"export_targets":[4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/1268086493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1268086493},{"type":"v1","addr":"172.21.15.71:6839","nonce":1268086493}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-02T15:43:12.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.458+0000 7f4102600700 1 -- 172.21.15.1:0/4105073 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f40dc0a07e0 msgr2=0x7f40dc0a2ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:12.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.458+0000 7f4102600700 1 --2- 172.21.15.1:0/4105073 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f40dc0a07e0 0x7f40dc0a2ca0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f40fc19a700 tx=0x7f40e4008040 comp rx=0 tx=0).stop 2024-02-02T15:43:12.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.458+0000 7f4102600700 1 -- 172.21.15.1:0/4105073 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40fc100f60 msgr2=0x7f40fc1990f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:12.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.458+0000 7f4102600700 1 --2- 172.21.15.1:0/4105073 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40fc100f60 0x7f40fc1990f0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f40ec0098e0 tx=0x7f40ec005da0 comp rx=0 tx=0).stop 2024-02-02T15:43:12.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.459+0000 7f4102600700 1 -- 172.21.15.1:0/4105073 shutdown_connections 2024-02-02T15:43:12.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.459+0000 7f4102600700 1 --2- 172.21.15.1:0/4105073 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f40dc0a07e0 0x7f40dc0a2ca0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:12.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.459+0000 7f4102600700 1 --2- 172.21.15.1:0/4105073 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40fc19da90 0x7f40fc19df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:12.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.459+0000 7f4102600700 1 --2- 172.21.15.1:0/4105073 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f40fc100f60 0x7f40fc1990f0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:12.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.459+0000 7f4102600700 1 --2- 172.21.15.1:0/4105073 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f40fc106080 0x7f40fc199630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:12.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.459+0000 7f4102600700 1 -- 172.21.15.1:0/4105073 >> 172.21.15.1:0/4105073 conn(0x7f40fc0fc800 msgr2=0x7f40fc102920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:12.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.459+0000 7f4102600700 1 -- 172.21.15.1:0/4105073 shutdown_connections 2024-02-02T15:43:12.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.459+0000 7f4102600700 1 -- 172.21.15.1:0/4105073 wait complete. 2024-02-02T15:43:12.461 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 67 2024-02-02T15:43:12.487 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-02-02T15:43:12.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.812+0000 7efe33400700 1 -- 172.21.15.1:0/2068252394 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe2c0689b0 msgr2=0x7efe2c10dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:12.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.812+0000 7efe33400700 1 --2- 172.21.15.1:0/2068252394 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe2c0689b0 0x7efe2c10dc20 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7efe2400b0a0 tx=0x7efe2400b3b0 comp rx=0 tx=0).stop 2024-02-02T15:43:12.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.812+0000 7efe33400700 1 -- 172.21.15.1:0/2068252394 shutdown_connections 2024-02-02T15:43:12.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.812+0000 7efe33400700 1 --2- 172.21.15.1:0/2068252394 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe2c0689b0 0x7efe2c10dc20 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:12.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.812+0000 7efe33400700 1 --2- 172.21.15.1:0/2068252394 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe2c106ed0 0x7efe2c1072b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:12.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.812+0000 7efe33400700 1 --2- 172.21.15.1:0/2068252394 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe2c068010 0x7efe2c068470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:12.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.812+0000 7efe33400700 1 -- 172.21.15.1:0/2068252394 >> 172.21.15.1:0/2068252394 conn(0x7efe2c0757d0 msgr2=0x7efe2c075be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:12.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.813+0000 7efe33400700 1 -- 172.21.15.1:0/2068252394 shutdown_connections 2024-02-02T15:43:12.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.813+0000 7efe33400700 1 -- 172.21.15.1:0/2068252394 wait complete. 2024-02-02T15:43:12.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.814+0000 7efe33400700 1 Processor -- start 2024-02-02T15:43:12.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.815+0000 7efe33400700 1 -- start start 2024-02-02T15:43:12.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.815+0000 7efe33400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe2c068010 0x7efe2c10b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:12.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.815+0000 7efe33400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe2c0689b0 0x7efe2c10bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:12.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.815+0000 7efe30a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe2c068010 0x7efe2c10b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:12.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.815+0000 7efe30a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe2c068010 0x7efe2c10b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:50128/0 (socket says 172.21.15.1:50128) 2024-02-02T15:43:12.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.816+0000 7efe33400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe2c106ed0 0x7efe2c1b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:12.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.816+0000 7efe33400700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe2c1b2160 con 0x7efe2c106ed0 2024-02-02T15:43:12.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.816+0000 7efe33400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe2c1b22a0 con 0x7efe2c068010 2024-02-02T15:43:12.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.816+0000 7efe33400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe2c1b23e0 con 0x7efe2c0689b0 2024-02-02T15:43:12.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.816+0000 7efe2be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe2c0689b0 0x7efe2c10bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:12.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.816+0000 7efe30a00700 1 -- 172.21.15.1:0/606025988 learned_addr learned my addr 172.21.15.1:0/606025988 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:43:12.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.816+0000 7efe2be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe2c0689b0 0x7efe2c10bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:37802/0 (socket says 172.21.15.1:37802) 2024-02-02T15:43:12.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.816+0000 7efe31400700 1 --2- 172.21.15.1:0/606025988 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe2c106ed0 0x7efe2c1b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:12.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.816+0000 7efe31400700 1 -- 172.21.15.1:0/606025988 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe2c0689b0 msgr2=0x7efe2c10bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:12.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.816+0000 7efe31400700 1 --2- 172.21.15.1:0/606025988 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe2c0689b0 0x7efe2c10bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:12.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.816+0000 7efe31400700 1 -- 172.21.15.1:0/606025988 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe2c068010 msgr2=0x7efe2c10b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:12.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.816+0000 7efe31400700 1 --2- 172.21.15.1:0/606025988 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe2c068010 0x7efe2c10b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:12.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.816+0000 7efe31400700 1 -- 172.21.15.1:0/606025988 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe24009d00 con 0x7efe2c106ed0 2024-02-02T15:43:12.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.817+0000 7efe31400700 1 --2- 172.21.15.1:0/606025988 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe2c106ed0 0x7efe2c1b1c20 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7efe24009fd0 tx=0x7efe240077f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:12.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.817+0000 7efe29600700 1 -- 172.21.15.1:0/606025988 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2400e040 con 0x7efe2c106ed0 2024-02-02T15:43:12.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.817+0000 7efe29600700 1 -- 172.21.15.1:0/606025988 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe24004e60 con 0x7efe2c106ed0 2024-02-02T15:43:12.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.817+0000 7efe29600700 1 -- 172.21.15.1:0/606025988 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe240052a0 con 0x7efe2c106ed0 2024-02-02T15:43:12.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.817+0000 7efe33400700 1 -- 172.21.15.1:0/606025988 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe2c1b25e0 con 0x7efe2c106ed0 2024-02-02T15:43:12.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.817+0000 7efe33400700 1 -- 172.21.15.1:0/606025988 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe2c1b2aa0 con 0x7efe2c106ed0 2024-02-02T15:43:12.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.822+0000 7efe29600700 1 -- 172.21.15.1:0/606025988 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7efe2401d470 con 0x7efe2c106ed0 2024-02-02T15:43:12.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.823+0000 7efe29600700 1 --2- 172.21.15.1:0/606025988 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7efe0c0a0710 0x7efe0c0a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:12.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.824+0000 7efe29600700 1 -- 172.21.15.1:0/606025988 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9541+0+0 (secure 0 0 0) 0x7efe240cd7f0 con 0x7efe2c106ed0 2024-02-02T15:43:12.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.824+0000 7efe30a00700 1 --2- 172.21.15.1:0/606025988 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7efe0c0a0710 0x7efe0c0a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:12.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.824+0000 7efe46eccb80 1 Processor -- start 2024-02-02T15:43:12.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.824+0000 7efe46eccb80 1 -- start start 2024-02-02T15:43:12.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.825+0000 7efe46eccb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3a695d010 0x55c3a695d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:12.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.825+0000 7efe46eccb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3a695d930 0x55c3a69682a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:12.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.825+0000 7efe46eccb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c3a69687e0 0x55c3a696abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:12.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.825+0000 7efe46eccb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3a68d14c0 con 0x55c3a695d010 2024-02-02T15:43:12.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.825+0000 7efe46eccb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3a68b5aa0 con 0x55c3a69687e0 2024-02-02T15:43:12.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.825+0000 7efe46eccb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3a68b3460 con 0x55c3a695d930 2024-02-02T15:43:12.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.825+0000 7efe30a00700 1 --2- 172.21.15.1:0/606025988 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7efe0c0a0710 0x7efe0c0a2bd0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7efe1c005950 tx=0x7efe1c00b500 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:12.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.825+0000 7efe30a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3a695d010 0x55c3a695d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:12.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.825+0000 7efe2be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3a695d930 0x55c3a69682a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:12.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.825+0000 7efe2be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3a695d930 0x55c3a69682a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:37818/0 (socket says 172.21.15.1:37818) 2024-02-02T15:43:12.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.825+0000 7efe2be00700 1 -- 172.21.15.1:0/190687877 learned_addr learned my addr 172.21.15.1:0/190687877 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:43:12.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.825+0000 7efe30a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3a695d010 0x55c3a695d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60824/0 (socket says 172.21.15.1:60824) 2024-02-02T15:43:12.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.825+0000 7efe31400700 1 --2- 172.21.15.1:0/190687877 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c3a69687e0 0x55c3a696abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:12.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.826+0000 7efe2be00700 1 -- 172.21.15.1:0/190687877 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c3a69687e0 msgr2=0x55c3a696abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:12.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.826+0000 7efe2be00700 1 --2- 172.21.15.1:0/190687877 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c3a69687e0 0x55c3a696abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:12.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.826+0000 7efe2be00700 1 -- 172.21.15.1:0/190687877 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3a695d010 msgr2=0x55c3a695d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:12.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.826+0000 7efe2be00700 1 --2- 172.21.15.1:0/190687877 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3a695d010 0x55c3a695d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:12.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.826+0000 7efe2be00700 1 -- 172.21.15.1:0/190687877 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3a67303b0 con 0x55c3a695d930 2024-02-02T15:43:12.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.826+0000 7efe31400700 1 --2- 172.21.15.1:0/190687877 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c3a69687e0 0x55c3a696abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:43:12.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.826+0000 7efe30a00700 1 --2- 172.21.15.1:0/190687877 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3a695d010 0x55c3a695d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:43:12.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.826+0000 7efe2be00700 1 --2- 172.21.15.1:0/190687877 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3a695d930 0x55c3a69682a0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7efe14009d50 tx=0x7efe14011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=953ce3a2cc189abe server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:12.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.827+0000 7efe21c00700 1 -- 172.21.15.1:0/190687877 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe14019630 con 0x55c3a695d930 2024-02-02T15:43:12.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.827+0000 7efe21c00700 1 -- 172.21.15.1:0/190687877 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe14019c50 con 0x55c3a695d930 2024-02-02T15:43:12.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.827+0000 7efe21c00700 1 -- 172.21.15.1:0/190687877 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe1400eb60 con 0x55c3a695d930 2024-02-02T15:43:12.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.828+0000 7efe46eccb80 1 -- 172.21.15.1:0/190687877 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3a695d930 msgr2=0x55c3a69682a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:12.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.828+0000 7efe46eccb80 1 --2- 172.21.15.1:0/190687877 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3a695d930 0x55c3a69682a0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7efe14009d50 tx=0x7efe14011720 comp rx=0 tx=0).stop 2024-02-02T15:43:12.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.828+0000 7efe46eccb80 1 -- 172.21.15.1:0/190687877 shutdown_connections 2024-02-02T15:43:12.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.828+0000 7efe46eccb80 1 --2- 172.21.15.1:0/190687877 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c3a69687e0 0x55c3a696abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:12.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.828+0000 7efe46eccb80 1 --2- 172.21.15.1:0/190687877 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3a695d010 0x55c3a695d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:12.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.828+0000 7efe46eccb80 1 --2- 172.21.15.1:0/190687877 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3a695d930 0x55c3a69682a0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:12.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.828+0000 7efe46eccb80 1 -- 172.21.15.1:0/190687877 >> 172.21.15.1:0/190687877 conn(0x55c3a68bcdb0 msgr2=0x55c3a69593b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:12.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.828+0000 7efe46eccb80 1 -- 172.21.15.1:0/190687877 shutdown_connections 2024-02-02T15:43:12.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.828+0000 7efe46eccb80 1 -- 172.21.15.1:0/190687877 wait complete. 2024-02-02T15:43:12.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.829+0000 7efe46eccb80 1 -- start start 2024-02-02T15:43:12.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.829+0000 7efe46eccb80 1 Processor -- start 2024-02-02T15:43:12.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.830+0000 7efe46eccb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3a695d010 0x55c3a69681a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:12.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.830+0000 7efe46eccb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3a695d930 0x55c3a6965420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:12.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.830+0000 7efe46eccb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c3a69687e0 0x55c3a6965a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:12.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.830+0000 7efe46eccb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3a68aed50 con 0x55c3a695d010 2024-02-02T15:43:12.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.830+0000 7efe46eccb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3a684ad80 con 0x55c3a69687e0 2024-02-02T15:43:12.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.830+0000 7efe46eccb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3a684aa20 con 0x55c3a695d930 2024-02-02T15:43:12.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.830+0000 7efe46eccb80 10 client.0 _ll_register_callbacks cb 0x55c3a684c290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:43:12.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.830+0000 7efe30a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3a695d010 0x55c3a69681a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:12.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.830+0000 7efe31400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c3a69687e0 0x55c3a6965a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:12.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.831+0000 7efe30a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3a695d010 0x55c3a69681a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60830/0 (socket says 172.21.15.1:60830) 2024-02-02T15:43:12.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.831+0000 7efe30a00700 1 -- 172.21.15.1:0/2112407823 learned_addr learned my addr 172.21.15.1:0/2112407823 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:43:12.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.831+0000 7efe2be00700 1 --2- 172.21.15.1:0/2112407823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3a695d930 0x55c3a6965420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:12.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.831+0000 7efe31400700 1 -- 172.21.15.1:0/2112407823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3a695d930 msgr2=0x55c3a6965420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:12.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.831+0000 7efe31400700 1 --2- 172.21.15.1:0/2112407823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3a695d930 0x55c3a6965420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:12.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.831+0000 7efe31400700 1 -- 172.21.15.1:0/2112407823 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3a695d010 msgr2=0x55c3a69681a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:12.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.831+0000 7efe31400700 1 --2- 172.21.15.1:0/2112407823 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3a695d010 0x55c3a69681a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:12.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.831+0000 7efe31400700 1 -- 172.21.15.1:0/2112407823 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe2c1b2aa0 con 0x55c3a69687e0 2024-02-02T15:43:12.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.831+0000 7efe31400700 1 --2- 172.21.15.1:0/2112407823 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c3a69687e0 0x55c3a6965a30 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7efe240caae0 tx=0x7efe2400b3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b71d1a3bd2d9125d server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:12.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.831+0000 7efe1a000700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:43:12.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.832+0000 7efe1a000700 1 -- 172.21.15.1:0/2112407823 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe240cd570 con 0x55c3a69687e0 2024-02-02T15:43:12.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.832+0000 7efe1a000700 1 -- 172.21.15.1:0/2112407823 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9541+0+0 (secure 0 0 0) 0x7efe2409a940 con 0x55c3a69687e0 2024-02-02T15:43:12.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.832+0000 7efe46eccb80 1 -- 172.21.15.1:0/2112407823 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c3a69510f0 con 0x55c3a69687e0 2024-02-02T15:43:12.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.832+0000 7efe1a000700 1 -- 172.21.15.1:0/2112407823 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe2409e770 con 0x55c3a69687e0 2024-02-02T15:43:12.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.832+0000 7efe46eccb80 10 client.29807 fetch_fsmap learned FSMap version 67 2024-02-02T15:43:12.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.832+0000 7efe46eccb80 1 -- 172.21.15.1:0/2112407823 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) v3 -- 0x7efe14009980 con 0x55c3a69687e0 2024-02-02T15:43:12.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.833+0000 7efe1a000700 1 -- 172.21.15.1:0/2112407823 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c3a69510f0 con 0x55c3a69687e0 2024-02-02T15:43:12.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.833+0000 7efe1a000700 1 -- 172.21.15.1:0/2112407823 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 67) v1 ==== 7506+0+0 (secure 0 0 0) 0x7efe24099f10 con 0x55c3a69687e0 2024-02-02T15:43:12.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.833+0000 7efe46eccb80 10 client.29807 fetch_fsmap finished waiting for FSMap version 67 2024-02-02T15:43:12.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.833+0000 7efe46eccb80 10 client.29807 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-02-02T15:43:12.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.833+0000 7efe46eccb80 20 client.29807 populate_metadata read hostname 'smithi001' 2024-02-02T15:43:12.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.833+0000 7efe46eccb80 1 --2- 172.21.15.1:0/2112407823 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55c3a6978020 0x55c3a697a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:12.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.833+0000 7efe46eccb80 4 client.29807 mds_command: new command op to 24502 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:43:12.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.833+0000 7efe46eccb80 1 -- 172.21.15.1:0/2112407823 --> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c3a68a7dd0 con 0x55c3a6978020 2024-02-02T15:43:12.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.834+0000 7efe30a00700 1 --2- 172.21.15.1:0/2112407823 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55c3a6978020 0x55c3a697a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:12.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.835+0000 7efe1a000700 10 client.29807 ms_handle_connect on v2:172.21.15.142:6834/1052606547 2024-02-02T15:43:12.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.835+0000 7efe30a00700 1 --2- 172.21.15.1:0/2112407823 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55c3a6978020 0x55c3a697a440 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=203841a219d9832d server_cookie=6e603913a5073399 in_seq=0 out_seq=0 2024-02-02T15:43:12.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.836+0000 7efe1a000700 1 -- 172.21.15.1:0/2112407823 <== mds.4 v2:172.21.15.142:6834/1052606547 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55c3a68a7dd0 con 0x55c3a6978020 2024-02-02T15:43:12.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.836+0000 7efe1a000700 10 client.29807 handle_command_reply: tid=0 2024-02-02T15:43:12.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.836+0000 7efe46eccb80 1 client.29807 shutdown 2024-02-02T15:43:12.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.837+0000 7efe46eccb80 1 -- 172.21.15.1:0/2112407823 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c3a69687e0 msgr2=0x55c3a6965a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:12.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.837+0000 7efe46eccb80 1 --2- 172.21.15.1:0/2112407823 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c3a69687e0 0x55c3a6965a30 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7efe240caae0 tx=0x7efe2400b3b0 comp rx=0 tx=0).stop 2024-02-02T15:43:12.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.837+0000 7efe46eccb80 1 -- 172.21.15.1:0/2112407823 shutdown_connections 2024-02-02T15:43:12.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.837+0000 7efe46eccb80 1 --2- 172.21.15.1:0/2112407823 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55c3a6978020 0x55c3a697a440 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:12.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.837+0000 7efe46eccb80 1 --2- 172.21.15.1:0/2112407823 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c3a69687e0 0x55c3a6965a30 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:12.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.837+0000 7efe46eccb80 1 --2- 172.21.15.1:0/2112407823 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3a695d010 0x55c3a69681a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:12.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.837+0000 7efe46eccb80 1 --2- 172.21.15.1:0/2112407823 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3a695d930 0x55c3a6965420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:12.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.837+0000 7efe46eccb80 1 -- 172.21.15.1:0/2112407823 >> 172.21.15.1:0/2112407823 conn(0x55c3a68bcdb0 msgr2=0x55c3a6958fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:12.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.837+0000 7efe1a000700 0 client.29807 ms_handle_reset on v2:172.21.15.142:6834/1052606547 2024-02-02T15:43:12.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.837+0000 7efe46eccb80 1 -- 172.21.15.1:0/2112407823 shutdown_connections 2024-02-02T15:43:12.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.837+0000 7efe46eccb80 1 -- 172.21.15.1:0/2112407823 wait complete. 2024-02-02T15:43:12.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.839+0000 7efe46eccb80 20 client.29807 trim_cache size 0 max 16384 2024-02-02T15:43:12.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:43:12 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/4105073' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:43:12.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.866+0000 7efe46eccb80 1 Processor -- start 2024-02-02T15:43:12.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.866+0000 7efe46eccb80 1 -- start start 2024-02-02T15:43:12.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.867+0000 7efe46eccb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3a695d010 0x55c3a696f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:12.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.867+0000 7efe46eccb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3a695d930 0x55c3a695f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:12.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.867+0000 7efe46eccb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c3a69687e0 0x55c3a6967cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:12.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.867+0000 7efe46eccb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3a67326a0 con 0x55c3a695d930 2024-02-02T15:43:12.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.868+0000 7efe46eccb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3a695a0b0 con 0x55c3a69687e0 2024-02-02T15:43:12.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.868+0000 7efe46eccb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3a6953500 con 0x55c3a695d010 2024-02-02T15:43:12.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.868+0000 7efe30a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3a695d010 0x55c3a696f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:12.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.868+0000 7efe30a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3a695d010 0x55c3a696f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:37840/0 (socket says 172.21.15.1:37840) 2024-02-02T15:43:12.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.868+0000 7efe30a00700 1 -- 172.21.15.1:0/2515974209 learned_addr learned my addr 172.21.15.1:0/2515974209 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:43:12.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.868+0000 7efe31400700 1 --2- 172.21.15.1:0/2515974209 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c3a69687e0 0x55c3a6967cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:12.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.868+0000 7efe31400700 1 -- 172.21.15.1:0/2515974209 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3a695d010 msgr2=0x55c3a696f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:12.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.868+0000 7efe31400700 1 --2- 172.21.15.1:0/2515974209 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3a695d010 0x55c3a696f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:12.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.869+0000 7efe31400700 1 -- 172.21.15.1:0/2515974209 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3a695d930 msgr2=0x55c3a695f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:12.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.869+0000 7efe31400700 1 --2- 172.21.15.1:0/2515974209 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3a695d930 0x55c3a695f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:12.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.869+0000 7efe31400700 1 -- 172.21.15.1:0/2515974209 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3a697a7f0 con 0x55c3a69687e0 2024-02-02T15:43:12.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.869+0000 7efe31400700 1 --2- 172.21.15.1:0/2515974209 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c3a69687e0 0x55c3a6967cd0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7efe240062a0 tx=0x7efe24098ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcaf2e106f393520 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:12.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.871+0000 7efe22600700 1 -- 172.21.15.1:0/2515974209 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe240cd570 con 0x55c3a69687e0 2024-02-02T15:43:12.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.871+0000 7efe22600700 1 -- 172.21.15.1:0/2515974209 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe24085040 con 0x55c3a69687e0 2024-02-02T15:43:12.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.871+0000 7efe22600700 1 -- 172.21.15.1:0/2515974209 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe240b83b0 con 0x55c3a69687e0 2024-02-02T15:43:12.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.873+0000 7efe46eccb80 1 -- 172.21.15.1:0/2515974209 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c3a69687e0 msgr2=0x55c3a6967cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:12.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.873+0000 7efe46eccb80 1 --2- 172.21.15.1:0/2515974209 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c3a69687e0 0x55c3a6967cd0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7efe240062a0 tx=0x7efe24098ef0 comp rx=0 tx=0).stop 2024-02-02T15:43:12.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.875+0000 7efe46eccb80 1 -- 172.21.15.1:0/2515974209 shutdown_connections 2024-02-02T15:43:12.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.875+0000 7efe46eccb80 1 --2- 172.21.15.1:0/2515974209 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c3a69687e0 0x55c3a6967cd0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:12.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.875+0000 7efe46eccb80 1 --2- 172.21.15.1:0/2515974209 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3a695d010 0x55c3a696f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:12.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.875+0000 7efe46eccb80 1 --2- 172.21.15.1:0/2515974209 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3a695d930 0x55c3a695f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:12.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.875+0000 7efe46eccb80 1 -- 172.21.15.1:0/2515974209 >> 172.21.15.1:0/2515974209 conn(0x55c3a68bcdb0 msgr2=0x55c3a6960740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:12.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.876+0000 7efe46eccb80 1 -- 172.21.15.1:0/2515974209 shutdown_connections 2024-02-02T15:43:12.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.877+0000 7efe46eccb80 1 -- 172.21.15.1:0/2515974209 wait complete. 2024-02-02T15:43:12.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.878+0000 7efe46eccb80 1 -- start start 2024-02-02T15:43:12.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.878+0000 7efe46eccb80 1 Processor -- start 2024-02-02T15:43:12.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.879+0000 7efe46eccb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3a695d010 0x55c3a696b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:12.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.879+0000 7efe46eccb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c3a695d930 0x55c3a696b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:12.880 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.880+0000 7efe46eccb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3a69687e0 0x55c3a696bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:12.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.880+0000 7efe46eccb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3a69c8b30 con 0x55c3a695d010 2024-02-02T15:43:12.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.880+0000 7efe46eccb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3a69c8c70 con 0x55c3a695d930 2024-02-02T15:43:12.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.880+0000 7efe46eccb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3a69c8de0 con 0x55c3a69687e0 2024-02-02T15:43:12.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.880+0000 7efe46eccb80 10 client.0 _ll_register_callbacks cb 0x55c3a684c290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:43:12.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.880+0000 7efe30a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3a695d010 0x55c3a696b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:12.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.880+0000 7efe30a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3a695d010 0x55c3a696b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:60854/0 (socket says 172.21.15.1:60854) 2024-02-02T15:43:12.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.880+0000 7efe30a00700 1 -- 172.21.15.1:0/1725326113 learned_addr learned my addr 172.21.15.1:0/1725326113 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:43:12.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.880+0000 7efe2be00700 1 --2- 172.21.15.1:0/1725326113 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c3a695d930 0x55c3a696b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:12.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.881+0000 7efe31400700 1 --2- 172.21.15.1:0/1725326113 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3a69687e0 0x55c3a696bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:12.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.881+0000 7efe2be00700 1 -- 172.21.15.1:0/1725326113 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3a69687e0 msgr2=0x55c3a696bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:12.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.881+0000 7efe2be00700 1 --2- 172.21.15.1:0/1725326113 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3a69687e0 0x55c3a696bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:12.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.881+0000 7efe2be00700 1 -- 172.21.15.1:0/1725326113 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3a695d010 msgr2=0x55c3a696b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:12.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.881+0000 7efe2be00700 1 --2- 172.21.15.1:0/1725326113 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3a695d010 0x55c3a696b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:12.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.881+0000 7efe2be00700 1 -- 172.21.15.1:0/1725326113 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe14009980 con 0x55c3a695d930 2024-02-02T15:43:12.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.881+0000 7efe2be00700 1 --2- 172.21.15.1:0/1725326113 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c3a695d930 0x55c3a696b880 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x55c3a6966a60 tx=0x7efe14019ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de01f2c92bd0bef6 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:12.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.881+0000 7efe30a00700 1 --2- 172.21.15.1:0/1725326113 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3a695d010 0x55c3a696b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:43:12.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.881+0000 7efe1a000700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:43:12.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.882+0000 7efe1a000700 1 -- 172.21.15.1:0/1725326113 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe14005450 con 0x55c3a695d930 2024-02-02T15:43:12.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.882+0000 7efe1a000700 1 -- 172.21.15.1:0/1725326113 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9541+0+0 (secure 0 0 0) 0x7efe140055b0 con 0x55c3a695d930 2024-02-02T15:43:12.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.882+0000 7efe46eccb80 1 -- 172.21.15.1:0/1725326113 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efe14019c50 con 0x55c3a695d930 2024-02-02T15:43:12.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.882+0000 7efe1a000700 1 -- 172.21.15.1:0/1725326113 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe1402d9a0 con 0x55c3a695d930 2024-02-02T15:43:12.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.883+0000 7efe46eccb80 10 client.29813 fetch_fsmap learned FSMap version 67 2024-02-02T15:43:12.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.883+0000 7efe46eccb80 1 -- 172.21.15.1:0/1725326113 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) v3 -- 0x55c3a66dcb60 con 0x55c3a695d930 2024-02-02T15:43:12.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.883+0000 7efe1a000700 1 -- 172.21.15.1:0/1725326113 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efe14019c50 con 0x55c3a695d930 2024-02-02T15:43:12.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.883+0000 7efe1a000700 1 -- 172.21.15.1:0/1725326113 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 67) v1 ==== 7506+0+0 (secure 0 0 0) 0x7efe14003880 con 0x55c3a695d930 2024-02-02T15:43:12.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.884+0000 7efe46eccb80 10 client.29813 fetch_fsmap finished waiting for FSMap version 67 2024-02-02T15:43:12.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.884+0000 7efe46eccb80 10 client.29813 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-02-02T15:43:12.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.884+0000 7efe46eccb80 20 client.29813 populate_metadata read hostname 'smithi001' 2024-02-02T15:43:12.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.884+0000 7efe46eccb80 1 --2- 172.21.15.1:0/1725326113 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55c3a69ca3a0 0x55c3a69cc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:12.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.884+0000 7efe46eccb80 4 client.29813 mds_command: new command op to 24502 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:43:12.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.884+0000 7efe46eccb80 1 -- 172.21.15.1:0/1725326113 --> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c3a686df50 con 0x55c3a69ca3a0 2024-02-02T15:43:12.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.885+0000 7efe30a00700 1 --2- 172.21.15.1:0/1725326113 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55c3a69ca3a0 0x55c3a69cc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:12.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.885+0000 7efe30a00700 1 --2- 172.21.15.1:0/1725326113 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55c3a69ca3a0 0x55c3a69cc7c0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=28621d73616f4a9c server_cookie=980bb910731957c6 in_seq=0 out_seq=0 2024-02-02T15:43:12.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.886+0000 7efe1a000700 10 client.29813 ms_handle_connect on v2:172.21.15.142:6834/1052606547 2024-02-02T15:43:12.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.887+0000 7efe1a000700 1 -- 172.21.15.1:0/1725326113 <== mds.4 v2:172.21.15.142:6834/1052606547 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c3a686df50 con 0x55c3a69ca3a0 2024-02-02T15:43:12.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.887+0000 7efe1a000700 10 client.29813 handle_command_reply: tid=0 2024-02-02T15:43:12.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.887+0000 7efe46eccb80 1 client.29813 shutdown 2024-02-02T15:43:12.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.888+0000 7efe46eccb80 1 -- 172.21.15.1:0/1725326113 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c3a695d930 msgr2=0x55c3a696b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:12.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.888+0000 7efe46eccb80 1 --2- 172.21.15.1:0/1725326113 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c3a695d930 0x55c3a696b880 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x55c3a6966a60 tx=0x7efe14019ab0 comp rx=0 tx=0).stop 2024-02-02T15:43:12.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.888+0000 7efe46eccb80 1 -- 172.21.15.1:0/1725326113 shutdown_connections 2024-02-02T15:43:12.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.888+0000 7efe46eccb80 1 --2- 172.21.15.1:0/1725326113 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55c3a69ca3a0 0x55c3a69cc7c0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:12.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.888+0000 7efe46eccb80 1 --2- 172.21.15.1:0/1725326113 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c3a69687e0 0x55c3a696bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:12.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.888+0000 7efe46eccb80 1 --2- 172.21.15.1:0/1725326113 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c3a695d010 0x55c3a696b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:12.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.888+0000 7efe46eccb80 1 --2- 172.21.15.1:0/1725326113 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c3a695d930 0x55c3a696b880 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:12.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.888+0000 7efe46eccb80 1 -- 172.21.15.1:0/1725326113 >> 172.21.15.1:0/1725326113 conn(0x55c3a68bcdb0 msgr2=0x55c3a6963bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:12.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.888+0000 7efe1a000700 0 client.29813 ms_handle_reset on v2:172.21.15.142:6834/1052606547 2024-02-02T15:43:12.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.889+0000 7efe46eccb80 1 -- 172.21.15.1:0/1725326113 shutdown_connections 2024-02-02T15:43:12.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.889+0000 7efe46eccb80 1 -- 172.21.15.1:0/1725326113 wait complete. 2024-02-02T15:43:12.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.890+0000 7efe46eccb80 20 client.29813 trim_cache size 0 max 16384 2024-02-02T15:43:12.890 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:43:12.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.891+0000 7efe33400700 1 -- 172.21.15.1:0/606025988 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7efe0c0a0710 msgr2=0x7efe0c0a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:12.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.891+0000 7efe33400700 1 --2- 172.21.15.1:0/606025988 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7efe0c0a0710 0x7efe0c0a2bd0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7efe1c005950 tx=0x7efe1c00b500 comp rx=0 tx=0).stop 2024-02-02T15:43:12.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.891+0000 7efe33400700 1 -- 172.21.15.1:0/606025988 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe2c106ed0 msgr2=0x7efe2c1b1c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:12.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.891+0000 7efe33400700 1 --2- 172.21.15.1:0/606025988 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe2c106ed0 0x7efe2c1b1c20 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7efe24009fd0 tx=0x7efe240077f0 comp rx=0 tx=0).stop 2024-02-02T15:43:12.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.891+0000 7efe33400700 1 -- 172.21.15.1:0/606025988 shutdown_connections 2024-02-02T15:43:12.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.891+0000 7efe33400700 1 --2- 172.21.15.1:0/606025988 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7efe0c0a0710 0x7efe0c0a2bd0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:12.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.891+0000 7efe33400700 1 --2- 172.21.15.1:0/606025988 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7efe2c106ed0 0x7efe2c1b1c20 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:12.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.891+0000 7efe33400700 1 --2- 172.21.15.1:0/606025988 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efe2c068010 0x7efe2c10b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:12.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.891+0000 7efe33400700 1 --2- 172.21.15.1:0/606025988 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7efe2c0689b0 0x7efe2c10bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:12.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.891+0000 7efe33400700 1 -- 172.21.15.1:0/606025988 >> 172.21.15.1:0/606025988 conn(0x7efe2c0757d0 msgr2=0x7efe2c100910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:12.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.892+0000 7efe33400700 1 -- 172.21.15.1:0/606025988 shutdown_connections 2024-02-02T15:43:12.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:12.892+0000 7efe33400700 1 -- 172.21.15.1:0/606025988 wait complete. 2024-02-02T15:43:12.918 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:43:12 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/4105073' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:43:12.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:43:12 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/4105073' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:43:12.920 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-02-02T15:43:12.988 INFO:tasks.workunit.client.0.smithi001.stdout: 10 91663 26.33 MB/sec execute 45 sec latency 60.767 ms 2024-02-02T15:43:13.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.226+0000 7f1463400700 1 -- 172.21.15.1:0/95810488 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f145c103bb0 msgr2=0x7f145c107bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:13.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.226+0000 7f1463400700 1 --2- 172.21.15.1:0/95810488 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f145c103bb0 0x7f145c107bf0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f1444009990 tx=0x7f1444009ca0 comp rx=0 tx=0).stop 2024-02-02T15:43:13.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.226+0000 7f1463400700 1 -- 172.21.15.1:0/95810488 shutdown_connections 2024-02-02T15:43:13.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.226+0000 7f1463400700 1 --2- 172.21.15.1:0/95810488 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f145c108320 0x7f145c10fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:13.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.226+0000 7f1463400700 1 --2- 172.21.15.1:0/95810488 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f145c103200 0x7f145c1035e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:13.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.226+0000 7f1463400700 1 --2- 172.21.15.1:0/95810488 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f145c103bb0 0x7f145c107bf0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:13.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.226+0000 7f1463400700 1 -- 172.21.15.1:0/95810488 >> 172.21.15.1:0/95810488 conn(0x7f145c0feaa0 msgr2=0x7f145c100ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:13.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.226+0000 7f1463400700 1 -- 172.21.15.1:0/95810488 shutdown_connections 2024-02-02T15:43:13.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.227+0000 7f1463400700 1 -- 172.21.15.1:0/95810488 wait complete. 2024-02-02T15:43:13.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.228+0000 7f1463400700 1 Processor -- start 2024-02-02T15:43:13.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.228+0000 7f1463400700 1 -- start start 2024-02-02T15:43:13.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.228+0000 7f1463400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f145c103200 0x7f145c199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:13.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.228+0000 7f1463400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f145c108320 0x7f145c199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:13.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.228+0000 7f1463400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f145c19daa0 0x7f145c19df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:13.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.228+0000 7f1463400700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f145c19e520 con 0x7f145c19daa0 2024-02-02T15:43:13.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.228+0000 7f1463400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f145c19e690 con 0x7f145c103200 2024-02-02T15:43:13.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.228+0000 7f1463400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f145c19e850 con 0x7f145c108320 2024-02-02T15:43:13.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.229+0000 7f1460a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f145c103200 0x7f145c199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:13.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.229+0000 7f1460a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f145c103200 0x7f145c199100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:35386/0 (socket says 172.21.15.1:35386) 2024-02-02T15:43:13.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.229+0000 7f1460a00700 1 -- 172.21.15.1:0/3982412801 learned_addr learned my addr 172.21.15.1:0/3982412801 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:43:13.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.229+0000 7f1461400700 1 --2- 172.21.15.1:0/3982412801 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f145c19daa0 0x7f145c19df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:13.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.229+0000 7f145be00700 1 --2- 172.21.15.1:0/3982412801 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f145c108320 0x7f145c199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:13.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.229+0000 7f1460a00700 1 -- 172.21.15.1:0/3982412801 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f145c108320 msgr2=0x7f145c199640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:13.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.229+0000 7f1460a00700 1 --2- 172.21.15.1:0/3982412801 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f145c108320 0x7f145c199640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:13.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.229+0000 7f1460a00700 1 -- 172.21.15.1:0/3982412801 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f145c19daa0 msgr2=0x7f145c19df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:13.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.229+0000 7f1460a00700 1 --2- 172.21.15.1:0/3982412801 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f145c19daa0 0x7f145c19df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:13.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.229+0000 7f1460a00700 1 -- 172.21.15.1:0/3982412801 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14440095f0 con 0x7f145c103200 2024-02-02T15:43:13.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.229+0000 7f1461400700 1 --2- 172.21.15.1:0/3982412801 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f145c19daa0 0x7f145c19df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:43:13.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.230+0000 7f1460a00700 1 --2- 172.21.15.1:0/3982412801 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f145c103200 0x7f145c199100 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f144c00e9a0 tx=0x7f144c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:13.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.230+0000 7f1459600700 1 -- 172.21.15.1:0/3982412801 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f144c009790 con 0x7f145c103200 2024-02-02T15:43:13.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.230+0000 7f1463400700 1 -- 172.21.15.1:0/3982412801 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f145c1a5db0 con 0x7f145c103200 2024-02-02T15:43:13.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.230+0000 7f1463400700 1 -- 172.21.15.1:0/3982412801 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f145c1a6280 con 0x7f145c103200 2024-02-02T15:43:13.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.230+0000 7f1459600700 1 -- 172.21.15.1:0/3982412801 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f144c010430 con 0x7f145c103200 2024-02-02T15:43:13.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.230+0000 7f1459600700 1 -- 172.21.15.1:0/3982412801 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f144c018680 con 0x7f145c103200 2024-02-02T15:43:13.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.236+0000 7f1459600700 1 -- 172.21.15.1:0/3982412801 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f144c00ca10 con 0x7f145c103200 2024-02-02T15:43:13.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.238+0000 7f1476fe3b80 1 Processor -- start 2024-02-02T15:43:13.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.238+0000 7f1476fe3b80 1 -- start start 2024-02-02T15:43:13.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.238+0000 7f1476fe3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5566d4fe0010 0x5566d4fe03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:13.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.239+0000 7f1476fe3b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5566d4fe0930 0x5566d4feb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:13.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.239+0000 7f1476fe3b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566d4feb7e0 0x5566d4fedbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:13.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.239+0000 7f1476fe3b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566d4f544c0 con 0x5566d4fe0930 2024-02-02T15:43:13.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.239+0000 7f1476fe3b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566d4f38aa0 con 0x5566d4feb7e0 2024-02-02T15:43:13.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.239+0000 7f1476fe3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566d4f36460 con 0x5566d4fe0010 2024-02-02T15:43:13.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.239+0000 7f145be00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566d4feb7e0 0x5566d4fedbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:13.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.239+0000 7f145be00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566d4feb7e0 0x5566d4fedbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:35394/0 (socket says 172.21.15.1:35394) 2024-02-02T15:43:13.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.239+0000 7f145be00700 1 -- 172.21.15.1:0/4044166710 learned_addr learned my addr 172.21.15.1:0/4044166710 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:43:13.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.239+0000 7f1461400700 1 --2- 172.21.15.1:0/4044166710 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5566d4fe0010 0x5566d4fe03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:13.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.240+0000 7f145be00700 1 -- 172.21.15.1:0/4044166710 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5566d4fe0010 msgr2=0x5566d4fe03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:13.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.240+0000 7f145be00700 1 --2- 172.21.15.1:0/4044166710 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5566d4fe0010 0x5566d4fe03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:13.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.240+0000 7f145be00700 1 -- 172.21.15.1:0/4044166710 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5566d4fe0930 msgr2=0x5566d4feb2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:13.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.240+0000 7f145be00700 1 --2- 172.21.15.1:0/4044166710 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5566d4fe0930 0x5566d4feb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:13.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.240+0000 7f145be00700 1 -- 172.21.15.1:0/4044166710 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5566d4db33b0 con 0x5566d4feb7e0 2024-02-02T15:43:13.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.240+0000 7f1459600700 1 --2- 172.21.15.1:0/3982412801 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f143c0a0790 0x7f143c0a2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:13.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.240+0000 7f145be00700 1 --2- 172.21.15.1:0/4044166710 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566d4feb7e0 0x5566d4fedbd0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f1444015000 tx=0x7f1444005ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e15c1513436a89ca server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:13.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.241+0000 7f1459600700 1 -- 172.21.15.1:0/3982412801 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f144c014070 con 0x7f145c103200 2024-02-02T15:43:13.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.242+0000 7f1451c00700 1 -- 172.21.15.1:0/4044166710 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f144401d070 con 0x5566d4feb7e0 2024-02-02T15:43:13.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.242+0000 7f1461400700 1 --2- 172.21.15.1:0/3982412801 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f143c0a0790 0x7f143c0a2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:13.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.242+0000 7f1451c00700 1 -- 172.21.15.1:0/4044166710 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f144401e040 con 0x5566d4feb7e0 2024-02-02T15:43:13.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.242+0000 7f1451c00700 1 -- 172.21.15.1:0/4044166710 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14440048a0 con 0x5566d4feb7e0 2024-02-02T15:43:13.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.243+0000 7f1476fe3b80 1 -- 172.21.15.1:0/4044166710 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566d4feb7e0 msgr2=0x5566d4fedbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:13.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.243+0000 7f1476fe3b80 1 --2- 172.21.15.1:0/4044166710 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566d4feb7e0 0x5566d4fedbd0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f1444015000 tx=0x7f1444005ea0 comp rx=0 tx=0).stop 2024-02-02T15:43:13.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.243+0000 7f1461400700 1 --2- 172.21.15.1:0/3982412801 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f143c0a0790 0x7f143c0a2c50 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f1454003b10 tx=0x7f145400b040 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:13.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.243+0000 7f1476fe3b80 1 -- 172.21.15.1:0/4044166710 shutdown_connections 2024-02-02T15:43:13.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.243+0000 7f1476fe3b80 1 --2- 172.21.15.1:0/4044166710 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566d4feb7e0 0x5566d4fedbd0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:13.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.243+0000 7f1476fe3b80 1 --2- 172.21.15.1:0/4044166710 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5566d4fe0010 0x5566d4fe03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:13.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.243+0000 7f1476fe3b80 1 --2- 172.21.15.1:0/4044166710 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5566d4fe0930 0x5566d4feb2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:13.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.243+0000 7f1476fe3b80 1 -- 172.21.15.1:0/4044166710 >> 172.21.15.1:0/4044166710 conn(0x5566d4f3fdb0 msgr2=0x5566d4fdc3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:13.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.243+0000 7f1476fe3b80 1 -- 172.21.15.1:0/4044166710 shutdown_connections 2024-02-02T15:43:13.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.243+0000 7f1476fe3b80 1 -- 172.21.15.1:0/4044166710 wait complete. 2024-02-02T15:43:13.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.244+0000 7f1476fe3b80 1 -- start start 2024-02-02T15:43:13.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.245+0000 7f1476fe3b80 1 Processor -- start 2024-02-02T15:43:13.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.245+0000 7f1476fe3b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566d4fe0010 0x5566d4feb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:13.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.245+0000 7f1476fe3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5566d4fe0930 0x5566d4fe84d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:13.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.246+0000 7f1476fe3b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5566d4feb7e0 0x5566d4fe8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:13.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.246+0000 7f1476fe3b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566d4f31d50 con 0x5566d4feb7e0 2024-02-02T15:43:13.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.246+0000 7f1476fe3b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566d4ecdd80 con 0x5566d4fe0010 2024-02-02T15:43:13.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.246+0000 7f1476fe3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566d4ecda20 con 0x5566d4fe0930 2024-02-02T15:43:13.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.246+0000 7f1476fe3b80 10 client.0 _ll_register_callbacks cb 0x5566d4ecf290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:43:13.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.246+0000 7f145be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5566d4fe0930 0x5566d4fe84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:13.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.246+0000 7f145be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5566d4fe0930 0x5566d4fe84d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:48784/0 (socket says 172.21.15.1:48784) 2024-02-02T15:43:13.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.246+0000 7f145be00700 1 -- 172.21.15.1:0/2851331000 learned_addr learned my addr 172.21.15.1:0/2851331000 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:43:13.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.247+0000 7f1461400700 1 --2- 172.21.15.1:0/2851331000 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5566d4feb7e0 0x5566d4fe8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:13.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.247+0000 7f1460a00700 1 --2- 172.21.15.1:0/2851331000 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566d4fe0010 0x5566d4feb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:13.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.247+0000 7f145be00700 1 -- 172.21.15.1:0/2851331000 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566d4fe0010 msgr2=0x5566d4feb1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:13.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.247+0000 7f145be00700 1 --2- 172.21.15.1:0/2851331000 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566d4fe0010 0x5566d4feb1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:13.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.247+0000 7f145be00700 1 -- 172.21.15.1:0/2851331000 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5566d4feb7e0 msgr2=0x5566d4fe8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:13.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.247+0000 7f145be00700 1 --2- 172.21.15.1:0/2851331000 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5566d4feb7e0 0x5566d4fe8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:13.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.247+0000 7f145be00700 1 -- 172.21.15.1:0/2851331000 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f145c110710 con 0x5566d4fe0930 2024-02-02T15:43:13.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.247+0000 7f1461400700 1 --2- 172.21.15.1:0/2851331000 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5566d4feb7e0 0x5566d4fe8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:43:13.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.247+0000 7f1460a00700 1 --2- 172.21.15.1:0/2851331000 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566d4fe0010 0x5566d4feb1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:43:13.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.248+0000 7f145be00700 1 --2- 172.21.15.1:0/2851331000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5566d4fe0930 0x5566d4fe84d0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f1444005320 tx=0x7f144400ba20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17d28e42a1a9c275 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:13.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.248+0000 7f144a000700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:43:13.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.250+0000 7f144a000700 1 -- 172.21.15.1:0/2851331000 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f144401d030 con 0x5566d4fe0930 2024-02-02T15:43:13.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.250+0000 7f1476fe3b80 1 -- 172.21.15.1:0/2851331000 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5566d4fd40f0 con 0x5566d4fe0930 2024-02-02T15:43:13.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.250+0000 7f144a000700 1 -- 172.21.15.1:0/2851331000 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f1444004710 con 0x5566d4fe0930 2024-02-02T15:43:13.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.251+0000 7f144a000700 1 -- 172.21.15.1:0/2851331000 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7f14440048d0 con 0x5566d4fe0930 2024-02-02T15:43:13.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.251+0000 7f1476fe3b80 10 client.29758 fetch_fsmap learned FSMap version 67 2024-02-02T15:43:13.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.251+0000 7f1476fe3b80 1 -- 172.21.15.1:0/2851331000 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) v3 -- 0x5566d4d5fb60 con 0x5566d4fe0930 2024-02-02T15:43:13.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.251+0000 7f144a000700 1 -- 172.21.15.1:0/2851331000 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5566d4fd40f0 con 0x5566d4fe0930 2024-02-02T15:43:13.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.251+0000 7f144a000700 1 -- 172.21.15.1:0/2851331000 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 67) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f144400fa60 con 0x5566d4fe0930 2024-02-02T15:43:13.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.251+0000 7f1476fe3b80 10 client.29758 fetch_fsmap finished waiting for FSMap version 67 2024-02-02T15:43:13.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.251+0000 7f1476fe3b80 10 client.29758 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-02-02T15:43:13.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.251+0000 7f1476fe3b80 20 client.29758 populate_metadata read hostname 'smithi001' 2024-02-02T15:43:13.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.251+0000 7f1476fe3b80 1 --2- 172.21.15.1:0/2851331000 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x5566d4ffacc0 0x5566d4ffd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:13.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.251+0000 7f1476fe3b80 4 client.29758 mds_command: new command op to 24514 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:43:13.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.251+0000 7f1476fe3b80 1 -- 172.21.15.1:0/2851331000 --> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5566d4f2add0 con 0x5566d4ffacc0 2024-02-02T15:43:13.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.252+0000 7f1460a00700 1 --2- 172.21.15.1:0/2851331000 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x5566d4ffacc0 0x5566d4ffd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:13.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.253+0000 7f1460a00700 1 --2- 172.21.15.1:0/2851331000 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x5566d4ffacc0 0x5566d4ffd0b0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=82addf3cbf80cea server_cookie=7554ac70beb96c43 in_seq=0 out_seq=0 2024-02-02T15:43:13.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.253+0000 7f144a000700 10 client.29758 ms_handle_connect on v2:172.21.15.142:6836/2870289019 2024-02-02T15:43:13.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.253+0000 7f144a000700 1 -- 172.21.15.1:0/2851331000 <== mds.3 v2:172.21.15.142:6836/2870289019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5566d4f2add0 con 0x5566d4ffacc0 2024-02-02T15:43:13.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.253+0000 7f144a000700 10 client.29758 handle_command_reply: tid=0 2024-02-02T15:43:13.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.254+0000 7f1476fe3b80 1 client.29758 shutdown 2024-02-02T15:43:13.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.254+0000 7f1476fe3b80 1 -- 172.21.15.1:0/2851331000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5566d4fe0930 msgr2=0x5566d4fe84d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:13.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.254+0000 7f1476fe3b80 1 --2- 172.21.15.1:0/2851331000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5566d4fe0930 0x5566d4fe84d0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f1444005320 tx=0x7f144400ba20 comp rx=0 tx=0).stop 2024-02-02T15:43:13.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.254+0000 7f1476fe3b80 1 -- 172.21.15.1:0/2851331000 shutdown_connections 2024-02-02T15:43:13.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.254+0000 7f1476fe3b80 1 --2- 172.21.15.1:0/2851331000 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x5566d4ffacc0 0x5566d4ffd0b0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:13.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.254+0000 7f1476fe3b80 1 --2- 172.21.15.1:0/2851331000 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5566d4feb7e0 0x5566d4fe8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:13.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.254+0000 7f1476fe3b80 1 --2- 172.21.15.1:0/2851331000 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566d4fe0010 0x5566d4feb1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:13.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.254+0000 7f1476fe3b80 1 --2- 172.21.15.1:0/2851331000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5566d4fe0930 0x5566d4fe84d0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:13.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.254+0000 7f144a000700 0 client.29758 ms_handle_reset on v2:172.21.15.142:6836/2870289019 2024-02-02T15:43:13.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.254+0000 7f1476fe3b80 1 -- 172.21.15.1:0/2851331000 >> 172.21.15.1:0/2851331000 conn(0x5566d4f3fdb0 msgr2=0x5566d4fdbf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:13.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.254+0000 7f1476fe3b80 1 -- 172.21.15.1:0/2851331000 shutdown_connections 2024-02-02T15:43:13.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.254+0000 7f1476fe3b80 1 -- 172.21.15.1:0/2851331000 wait complete. 2024-02-02T15:43:13.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.254+0000 7f1476fe3b80 20 client.29758 trim_cache size 0 max 16384 2024-02-02T15:43:13.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.274+0000 7f1476fe3b80 1 Processor -- start 2024-02-02T15:43:13.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.274+0000 7f1476fe3b80 1 -- start start 2024-02-02T15:43:13.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.275+0000 7f1476fe3b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566d4fe0010 0x5566d4fe2840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:13.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.275+0000 7f1476fe3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5566d4fe0930 0x5566d4ff6900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:13.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.275+0000 7f1476fe3b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5566d4feb7e0 0x5566d4fe3840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:13.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.275+0000 7f1476fe3b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566d4db56a0 con 0x5566d4feb7e0 2024-02-02T15:43:13.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.275+0000 7f1476fe3b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566d4fe9650 con 0x5566d4fe0010 2024-02-02T15:43:13.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.275+0000 7f1476fe3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566d4fd6580 con 0x5566d4fe0930 2024-02-02T15:43:13.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.276+0000 7f1461400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5566d4feb7e0 0x5566d4fe3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:13.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.276+0000 7f1460a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566d4fe0010 0x5566d4fe2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:13.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.276+0000 7f145be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5566d4fe0930 0x5566d4ff6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:13.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.276+0000 7f1460a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566d4fe0010 0x5566d4fe2840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:35408/0 (socket says 172.21.15.1:35408) 2024-02-02T15:43:13.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.276+0000 7f1461400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5566d4feb7e0 0x5566d4fe3840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46290/0 (socket says 172.21.15.1:46290) 2024-02-02T15:43:13.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.276+0000 7f1460a00700 1 -- 172.21.15.1:0/871995093 learned_addr learned my addr 172.21.15.1:0/871995093 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:43:13.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.276+0000 7f1460a00700 1 -- 172.21.15.1:0/871995093 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5566d4fe0930 msgr2=0x5566d4ff6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:13.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.276+0000 7f1460a00700 1 --2- 172.21.15.1:0/871995093 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5566d4fe0930 0x5566d4ff6900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:13.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.276+0000 7f1460a00700 1 -- 172.21.15.1:0/871995093 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5566d4feb7e0 msgr2=0x5566d4fe3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:13.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.276+0000 7f1460a00700 1 --2- 172.21.15.1:0/871995093 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5566d4feb7e0 0x5566d4fe3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:13.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.276+0000 7f1460a00700 1 -- 172.21.15.1:0/871995093 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5566d4ef7ea0 con 0x5566d4fe0010 2024-02-02T15:43:13.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.278+0000 7f1460a00700 1 --2- 172.21.15.1:0/871995093 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566d4fe0010 0x5566d4fe2840 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f144c00c2d0 tx=0x7f144c084040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8800ff2152d9f996 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:13.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.279+0000 7f1452600700 1 -- 172.21.15.1:0/871995093 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f144c09d150 con 0x5566d4fe0010 2024-02-02T15:43:13.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.280+0000 7f1452600700 1 -- 172.21.15.1:0/871995093 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f144c0a1390 con 0x5566d4fe0010 2024-02-02T15:43:13.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.280+0000 7f1452600700 1 -- 172.21.15.1:0/871995093 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f144c010880 con 0x5566d4fe0010 2024-02-02T15:43:13.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.282+0000 7f1476fe3b80 1 -- 172.21.15.1:0/871995093 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566d4fe0010 msgr2=0x5566d4fe2840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:13.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.282+0000 7f1476fe3b80 1 --2- 172.21.15.1:0/871995093 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566d4fe0010 0x5566d4fe2840 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f144c00c2d0 tx=0x7f144c084040 comp rx=0 tx=0).stop 2024-02-02T15:43:13.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.283+0000 7f1476fe3b80 1 -- 172.21.15.1:0/871995093 shutdown_connections 2024-02-02T15:43:13.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.283+0000 7f1476fe3b80 1 --2- 172.21.15.1:0/871995093 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5566d4feb7e0 0x5566d4fe3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:13.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.283+0000 7f1476fe3b80 1 --2- 172.21.15.1:0/871995093 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566d4fe0010 0x5566d4fe2840 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:13.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.283+0000 7f1476fe3b80 1 --2- 172.21.15.1:0/871995093 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5566d4fe0930 0x5566d4ff6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:13.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.283+0000 7f1476fe3b80 1 -- 172.21.15.1:0/871995093 >> 172.21.15.1:0/871995093 conn(0x5566d4f3fdb0 msgr2=0x5566d4fdab20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:13.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.283+0000 7f1476fe3b80 1 -- 172.21.15.1:0/871995093 shutdown_connections 2024-02-02T15:43:13.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.284+0000 7f1476fe3b80 1 -- 172.21.15.1:0/871995093 wait complete. 2024-02-02T15:43:13.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.284+0000 7f1476fe3b80 1 -- start start 2024-02-02T15:43:13.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.284+0000 7f1476fe3b80 1 Processor -- start 2024-02-02T15:43:13.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.286+0000 7f1476fe3b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5566d4fe0010 0x5566d4fefa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:13.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.286+0000 7f1476fe3b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566d4fe0930 0x5566d4feff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:13.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.286+0000 7f1476fe3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5566d4feb7e0 0x5566d4ff04c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:13.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.286+0000 7f1476fe3b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566d504b8c0 con 0x5566d4fe0010 2024-02-02T15:43:13.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.286+0000 7f1476fe3b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566d504ba30 con 0x5566d4fe0930 2024-02-02T15:43:13.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.287+0000 7f1476fe3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566d504bbf0 con 0x5566d4feb7e0 2024-02-02T15:43:13.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.287+0000 7f1476fe3b80 10 client.0 _ll_register_callbacks cb 0x5566d4ecf290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:43:13.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.287+0000 7f145be00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566d4fe0930 0x5566d4feff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:13.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.287+0000 7f145be00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566d4fe0930 0x5566d4feff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:35418/0 (socket says 172.21.15.1:35418) 2024-02-02T15:43:13.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.287+0000 7f145be00700 1 -- 172.21.15.1:0/3002837268 learned_addr learned my addr 172.21.15.1:0/3002837268 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:43:13.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.288+0000 7f1460a00700 1 --2- 172.21.15.1:0/3002837268 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5566d4fe0010 0x5566d4fefa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:13.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.288+0000 7f1461400700 1 --2- 172.21.15.1:0/3002837268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5566d4feb7e0 0x5566d4ff04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:13.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.288+0000 7f145be00700 1 -- 172.21.15.1:0/3002837268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5566d4feb7e0 msgr2=0x5566d4ff04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:13.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.288+0000 7f145be00700 1 --2- 172.21.15.1:0/3002837268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5566d4feb7e0 0x5566d4ff04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:13.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.288+0000 7f145be00700 1 -- 172.21.15.1:0/3002837268 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5566d4fe0010 msgr2=0x5566d4fefa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:13.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.288+0000 7f145be00700 1 --2- 172.21.15.1:0/3002837268 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5566d4fe0010 0x5566d4fefa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:13.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.288+0000 7f145be00700 1 -- 172.21.15.1:0/3002837268 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f145c1a6280 con 0x5566d4fe0930 2024-02-02T15:43:13.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.289+0000 7f145be00700 1 --2- 172.21.15.1:0/3002837268 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566d4fe0930 0x5566d4feff80 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f144400ebc0 tx=0x7f1444036f30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6670f56dea3fed1f server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:13.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.289+0000 7f144a000700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:43:13.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.290+0000 7f144a000700 1 -- 172.21.15.1:0/3002837268 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f144401d040 con 0x5566d4fe0930 2024-02-02T15:43:13.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.290+0000 7f144a000700 1 -- 172.21.15.1:0/3002837268 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f1444000e40 con 0x5566d4fe0930 2024-02-02T15:43:13.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.290+0000 7f1476fe3b80 1 -- 172.21.15.1:0/3002837268 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f144401e040 con 0x5566d4fe0930 2024-02-02T15:43:13.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.291+0000 7f144a000700 1 -- 172.21.15.1:0/3002837268 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7f14440048d0 con 0x5566d4fe0930 2024-02-02T15:43:13.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.291+0000 7f1476fe3b80 10 client.29831 fetch_fsmap learned FSMap version 67 2024-02-02T15:43:13.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.291+0000 7f1476fe3b80 1 -- 172.21.15.1:0/3002837268 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) v3 -- 0x5566d4db7cd0 con 0x5566d4fe0930 2024-02-02T15:43:13.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.292+0000 7f144a000700 1 -- 172.21.15.1:0/3002837268 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1444046070 con 0x5566d4fe0930 2024-02-02T15:43:13.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.292+0000 7f144a000700 1 -- 172.21.15.1:0/3002837268 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 67) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f144400fa60 con 0x5566d4fe0930 2024-02-02T15:43:13.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.292+0000 7f1476fe3b80 10 client.29831 fetch_fsmap finished waiting for FSMap version 67 2024-02-02T15:43:13.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.292+0000 7f1476fe3b80 10 client.29831 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-02-02T15:43:13.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.292+0000 7f1476fe3b80 20 client.29831 populate_metadata read hostname 'smithi001' 2024-02-02T15:43:13.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.292+0000 7f1476fe3b80 1 --2- 172.21.15.1:0/3002837268 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x5566d504cf40 0x5566d504f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:13.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.292+0000 7f1476fe3b80 4 client.29831 mds_command: new command op to 24514 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:43:13.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.292+0000 7f1476fe3b80 1 -- 172.21.15.1:0/3002837268 --> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5566d4ef0f50 con 0x5566d504cf40 2024-02-02T15:43:13.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.292+0000 7f1460a00700 1 --2- 172.21.15.1:0/3002837268 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x5566d504cf40 0x5566d504f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:13.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.293+0000 7f1460a00700 1 --2- 172.21.15.1:0/3002837268 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x5566d504cf40 0x5566d504f360 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=70aee9d314384bd2 server_cookie=47dc26bb6d767502 in_seq=0 out_seq=0 2024-02-02T15:43:13.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.293+0000 7f144a000700 10 client.29831 ms_handle_connect on v2:172.21.15.142:6836/2870289019 2024-02-02T15:43:13.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.294+0000 7f144a000700 1 -- 172.21.15.1:0/3002837268 <== mds.3 v2:172.21.15.142:6836/2870289019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5566d4ef0f50 con 0x5566d504cf40 2024-02-02T15:43:13.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.294+0000 7f144a000700 10 client.29831 handle_command_reply: tid=0 2024-02-02T15:43:13.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.294+0000 7f1476fe3b80 1 client.29831 shutdown 2024-02-02T15:43:13.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.294+0000 7f1476fe3b80 1 -- 172.21.15.1:0/3002837268 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566d4fe0930 msgr2=0x5566d4feff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:13.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.294+0000 7f1476fe3b80 1 --2- 172.21.15.1:0/3002837268 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566d4fe0930 0x5566d4feff80 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f144400ebc0 tx=0x7f1444036f30 comp rx=0 tx=0).stop 2024-02-02T15:43:13.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.295+0000 7f1476fe3b80 1 -- 172.21.15.1:0/3002837268 shutdown_connections 2024-02-02T15:43:13.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.295+0000 7f1476fe3b80 1 --2- 172.21.15.1:0/3002837268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5566d4feb7e0 0x5566d4ff04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:13.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.295+0000 7f1476fe3b80 1 --2- 172.21.15.1:0/3002837268 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x5566d504cf40 0x5566d504f360 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:13.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.295+0000 7f1476fe3b80 1 --2- 172.21.15.1:0/3002837268 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5566d4fe0010 0x5566d4fefa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:13.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.295+0000 7f1476fe3b80 1 --2- 172.21.15.1:0/3002837268 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566d4fe0930 0x5566d4feff80 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:13.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.295+0000 7f1476fe3b80 1 -- 172.21.15.1:0/3002837268 >> 172.21.15.1:0/3002837268 conn(0x5566d4f3fdb0 msgr2=0x5566d4fe3840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:13.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.295+0000 7f144a000700 0 client.29831 ms_handle_reset on v2:172.21.15.142:6836/2870289019 2024-02-02T15:43:13.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.295+0000 7f1476fe3b80 1 -- 172.21.15.1:0/3002837268 shutdown_connections 2024-02-02T15:43:13.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.295+0000 7f1476fe3b80 1 -- 172.21.15.1:0/3002837268 wait complete. 2024-02-02T15:43:13.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.296+0000 7f1476fe3b80 20 client.29831 trim_cache size 0 max 16384 2024-02-02T15:43:13.296 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:43:13.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.297+0000 7f1463400700 1 -- 172.21.15.1:0/3982412801 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f143c0a0790 msgr2=0x7f143c0a2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:13.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.297+0000 7f1463400700 1 --2- 172.21.15.1:0/3982412801 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f143c0a0790 0x7f143c0a2c50 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f1454003b10 tx=0x7f145400b040 comp rx=0 tx=0).stop 2024-02-02T15:43:13.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.297+0000 7f1463400700 1 -- 172.21.15.1:0/3982412801 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f145c103200 msgr2=0x7f145c199100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:13.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.297+0000 7f1463400700 1 --2- 172.21.15.1:0/3982412801 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f145c103200 0x7f145c199100 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f144c00e9a0 tx=0x7f144c00ecb0 comp rx=0 tx=0).stop 2024-02-02T15:43:13.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.297+0000 7f1463400700 1 -- 172.21.15.1:0/3982412801 shutdown_connections 2024-02-02T15:43:13.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.297+0000 7f1463400700 1 --2- 172.21.15.1:0/3982412801 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f143c0a0790 0x7f143c0a2c50 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:13.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.297+0000 7f1463400700 1 --2- 172.21.15.1:0/3982412801 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f145c19daa0 0x7f145c19df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:13.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.297+0000 7f1463400700 1 --2- 172.21.15.1:0/3982412801 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f145c103200 0x7f145c199100 secure :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f144c00e9a0 tx=0x7f144c00ecb0 comp rx=0 tx=0).stop 2024-02-02T15:43:13.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.297+0000 7f1463400700 1 --2- 172.21.15.1:0/3982412801 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f145c108320 0x7f145c199640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:13.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.297+0000 7f1463400700 1 -- 172.21.15.1:0/3982412801 >> 172.21.15.1:0/3982412801 conn(0x7f145c0feaa0 msgr2=0x7f145c0ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:13.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.299+0000 7f1463400700 1 -- 172.21.15.1:0/3982412801 shutdown_connections 2024-02-02T15:43:13.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.299+0000 7f1463400700 1 -- 172.21.15.1:0/3982412801 wait complete. 2024-02-02T15:43:13.330 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-02T15:43:13.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.671+0000 7f2f22000700 1 -- 172.21.15.1:0/2622069104 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2f1c1019b0 msgr2=0x7f2f1c1059f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:13.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.671+0000 7f2f22000700 1 --2- 172.21.15.1:0/2622069104 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2f1c1019b0 0x7f2f1c1059f0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f2f0400b0a0 tx=0x7f2f0400b3b0 comp rx=0 tx=0).stop 2024-02-02T15:43:13.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.671+0000 7f2f22000700 1 -- 172.21.15.1:0/2622069104 shutdown_connections 2024-02-02T15:43:13.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.671+0000 7f2f22000700 1 --2- 172.21.15.1:0/2622069104 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2f1c106120 0x7f2f1c10dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:13.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.671+0000 7f2f22000700 1 --2- 172.21.15.1:0/2622069104 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2f1c101000 0x7f2f1c1013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:13.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.671+0000 7f2f22000700 1 --2- 172.21.15.1:0/2622069104 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2f1c1019b0 0x7f2f1c1059f0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:13.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.671+0000 7f2f22000700 1 -- 172.21.15.1:0/2622069104 >> 172.21.15.1:0/2622069104 conn(0x7f2f1c0fc840 msgr2=0x7f2f1c0fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:13.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.672+0000 7f2f22000700 1 -- 172.21.15.1:0/2622069104 shutdown_connections 2024-02-02T15:43:13.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.672+0000 7f2f22000700 1 -- 172.21.15.1:0/2622069104 wait complete. 2024-02-02T15:43:13.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.673+0000 7f2f22000700 1 Processor -- start 2024-02-02T15:43:13.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.674+0000 7f2f22000700 1 -- start start 2024-02-02T15:43:13.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.674+0000 7f2f22000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2f1c101000 0x7f2f1c107cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:13.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.674+0000 7f2f22000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2f1c106120 0x7f2f1c108200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:13.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.674+0000 7f2f22000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2f1c108740 0x7f2f1c1a98f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:13.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.674+0000 7f2f22000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f1c1a9e30 con 0x7f2f1c101000 2024-02-02T15:43:13.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.674+0000 7f2f22000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f1c199090 con 0x7f2f1c106120 2024-02-02T15:43:13.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.675+0000 7f2f22000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f1c199200 con 0x7f2f1c108740 2024-02-02T15:43:13.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.675+0000 7f2f1aa00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2f1c106120 0x7f2f1c108200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:13.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.675+0000 7f2f1aa00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2f1c106120 0x7f2f1c108200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:35432/0 (socket says 172.21.15.1:35432) 2024-02-02T15:43:13.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.675+0000 7f2f1aa00700 1 -- 172.21.15.1:0/2107631118 learned_addr learned my addr 172.21.15.1:0/2107631118 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:43:13.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.675+0000 7f2f1be00700 1 --2- 172.21.15.1:0/2107631118 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2f1c108740 0x7f2f1c1a98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:13.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.676+0000 7f2f1be00700 1 -- 172.21.15.1:0/2107631118 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2f1c106120 msgr2=0x7f2f1c108200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:13.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.676+0000 7f2f1be00700 1 --2- 172.21.15.1:0/2107631118 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2f1c106120 0x7f2f1c108200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:13.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.676+0000 7f2f1be00700 1 -- 172.21.15.1:0/2107631118 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2f1c101000 msgr2=0x7f2f1c107cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:13.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.676+0000 7f2f1be00700 1 --2- 172.21.15.1:0/2107631118 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2f1c101000 0x7f2f1c107cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:13.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.676+0000 7f2f1be00700 1 -- 172.21.15.1:0/2107631118 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f0c009520 con 0x7f2f1c108740 2024-02-02T15:43:13.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.676+0000 7f2f1be00700 1 --2- 172.21.15.1:0/2107631118 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2f1c108740 0x7f2f1c1a98f0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c00e900 tx=0x7f2f0c00ec10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:13.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.677+0000 7f2f13e00700 1 -- 172.21.15.1:0/2107631118 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f0c00cbd0 con 0x7f2f1c108740 2024-02-02T15:43:13.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.677+0000 7f2f13e00700 1 -- 172.21.15.1:0/2107631118 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f0c00ee50 con 0x7f2f1c108740 2024-02-02T15:43:13.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.677+0000 7f2f13e00700 1 -- 172.21.15.1:0/2107631118 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f0c003cf0 con 0x7f2f1c108740 2024-02-02T15:43:13.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.677+0000 7f2f22000700 1 -- 172.21.15.1:0/2107631118 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f04009d00 con 0x7f2f1c108740 2024-02-02T15:43:13.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.678+0000 7f2f22000700 1 -- 172.21.15.1:0/2107631118 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f1c199810 con 0x7f2f1c108740 2024-02-02T15:43:13.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.685+0000 7f2f13e00700 1 -- 172.21.15.1:0/2107631118 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f2f0c00cd30 con 0x7f2f1c108740 2024-02-02T15:43:13.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.688+0000 7f2f35a3cb80 1 Processor -- start 2024-02-02T15:43:13.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.688+0000 7f2f35a3cb80 1 -- start start 2024-02-02T15:43:13.689 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:43:13 smithi001 ceph-mon[35470]: pgmap v508: 129 pgs: 129 active+clean; 1.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.4 MiB/s wr, 429 op/s 2024-02-02T15:43:13.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.690+0000 7f2f35a3cb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5601fde4a010 0x5601fde4a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:13.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.691+0000 7f2f35a3cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5601fde4a930 0x5601fde552a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:13.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.691+0000 7f2f35a3cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5601fde557e0 0x5601fde57bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:13.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.691+0000 7f2f35a3cb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601fddbe4c0 con 0x5601fde4a010 2024-02-02T15:43:13.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.691+0000 7f2f35a3cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601fdda2aa0 con 0x5601fde557e0 2024-02-02T15:43:13.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.691+0000 7f2f35a3cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601fdda0460 con 0x5601fde4a930 2024-02-02T15:43:13.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.691+0000 7f2f1b400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5601fde4a930 0x5601fde552a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:13.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.692+0000 7f2f1aa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5601fde4a010 0x5601fde4a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:13.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.692+0000 7f2f1b400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5601fde4a930 0x5601fde552a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:48836/0 (socket says 172.21.15.1:48836) 2024-02-02T15:43:13.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.692+0000 7f2f1b400700 1 -- 172.21.15.1:0/1497075313 learned_addr learned my addr 172.21.15.1:0/1497075313 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:43:13.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.692+0000 7f2f13e00700 1 --2- 172.21.15.1:0/2107631118 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f2efc0a0820 0x7f2efc0a2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:13.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.692+0000 7f2f1aa00700 1 --2- 172.21.15.1:0/1497075313 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5601fde557e0 0x5601fde57bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:13.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.692+0000 7f2f1aa00700 1 -- 172.21.15.1:0/1497075313 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5601fde4a930 msgr2=0x5601fde552a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:13.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.692+0000 7f2f1aa00700 1 --2- 172.21.15.1:0/1497075313 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5601fde4a930 0x5601fde552a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:13.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.692+0000 7f2f13e00700 1 -- 172.21.15.1:0/2107631118 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f2f0c014070 con 0x7f2f1c108740 2024-02-02T15:43:13.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.692+0000 7f2f1aa00700 1 -- 172.21.15.1:0/1497075313 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5601fde557e0 msgr2=0x5601fde57bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:13.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.692+0000 7f2f1aa00700 1 --2- 172.21.15.1:0/1497075313 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5601fde557e0 0x5601fde57bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:13.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.692+0000 7f2f1aa00700 1 -- 172.21.15.1:0/1497075313 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5601fdc1d3b0 con 0x5601fde4a010 2024-02-02T15:43:13.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.692+0000 7f2f1be00700 1 --2- 172.21.15.1:0/2107631118 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f2efc0a0820 0x7f2efc0a2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:13.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.693+0000 7f2f1aa00700 1 --2- 172.21.15.1:0/1497075313 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5601fde4a010 0x5601fde4a3f0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c1094b0 tx=0x7f2f04009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1aae453ce3fbfb7 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:13.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.693+0000 7f2f0be00700 1 -- 172.21.15.1:0/1497075313 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f04007ea0 con 0x5601fde4a010 2024-02-02T15:43:13.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.693+0000 7f2f1be00700 1 --2- 172.21.15.1:0/2107631118 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f2efc0a0820 0x7f2efc0a2ce0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c0c9850 tx=0x7f2f0c003780 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:13.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.693+0000 7f2f0be00700 1 -- 172.21.15.1:0/1497075313 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f0400bc50 con 0x5601fde4a010 2024-02-02T15:43:13.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.693+0000 7f2f0be00700 1 -- 172.21.15.1:0/1497075313 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f0400e040 con 0x5601fde4a010 2024-02-02T15:43:13.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.694+0000 7f2f35a3cb80 1 -- 172.21.15.1:0/1497075313 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5601fde4a010 msgr2=0x5601fde4a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:13.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.694+0000 7f2f35a3cb80 1 --2- 172.21.15.1:0/1497075313 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5601fde4a010 0x5601fde4a3f0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c1094b0 tx=0x7f2f04009360 comp rx=0 tx=0).stop 2024-02-02T15:43:13.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.694+0000 7f2f35a3cb80 1 -- 172.21.15.1:0/1497075313 shutdown_connections 2024-02-02T15:43:13.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.694+0000 7f2f35a3cb80 1 --2- 172.21.15.1:0/1497075313 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5601fde557e0 0x5601fde57bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:13.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.694+0000 7f2f35a3cb80 1 --2- 172.21.15.1:0/1497075313 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5601fde4a010 0x5601fde4a3f0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:13.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.694+0000 7f2f35a3cb80 1 --2- 172.21.15.1:0/1497075313 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5601fde4a930 0x5601fde552a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:13.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.694+0000 7f2f35a3cb80 1 -- 172.21.15.1:0/1497075313 >> 172.21.15.1:0/1497075313 conn(0x5601fdda9db0 msgr2=0x5601fde463b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:13.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.694+0000 7f2f35a3cb80 1 -- 172.21.15.1:0/1497075313 shutdown_connections 2024-02-02T15:43:13.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.694+0000 7f2f35a3cb80 1 -- 172.21.15.1:0/1497075313 wait complete. 2024-02-02T15:43:13.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.695+0000 7f2f35a3cb80 1 -- start start 2024-02-02T15:43:13.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.695+0000 7f2f35a3cb80 1 Processor -- start 2024-02-02T15:43:13.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.696+0000 7f2f35a3cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5601fde4a010 0x5601fde551f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:13.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.696+0000 7f2f35a3cb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5601fde4a930 0x5601fde52400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:13.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.696+0000 7f2f35a3cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5601fde557e0 0x5601fde52a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:13.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.696+0000 7f2f35a3cb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601fdd9bd50 con 0x5601fde4a930 2024-02-02T15:43:13.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.696+0000 7f2f35a3cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601fdd37d80 con 0x5601fde557e0 2024-02-02T15:43:13.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.696+0000 7f2f35a3cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601fdd37a20 con 0x5601fde4a010 2024-02-02T15:43:13.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.696+0000 7f2f35a3cb80 10 client.0 _ll_register_callbacks cb 0x5601fdd39290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:43:13.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.696+0000 7f2f1aa00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5601fde4a010 0x5601fde551f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:13.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.697+0000 7f2f1aa00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5601fde4a010 0x5601fde551f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:48850/0 (socket says 172.21.15.1:48850) 2024-02-02T15:43:13.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.697+0000 7f2f1aa00700 1 -- 172.21.15.1:0/4118737653 learned_addr learned my addr 172.21.15.1:0/4118737653 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:43:13.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.697+0000 7f2f1aa00700 1 --2- 172.21.15.1:0/4118737653 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5601fde557e0 0x5601fde52a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:13.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.697+0000 7f2f1aa00700 1 -- 172.21.15.1:0/4118737653 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5601fde557e0 msgr2=0x5601fde52a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:13.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.697+0000 7f2f1aa00700 1 --2- 172.21.15.1:0/4118737653 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5601fde557e0 0x5601fde52a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:13.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.697+0000 7f2f1aa00700 1 -- 172.21.15.1:0/4118737653 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5601fde4a930 msgr2=0x5601fde52400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:13.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.697+0000 7f2f1aa00700 1 --2- 172.21.15.1:0/4118737653 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5601fde4a930 0x5601fde52400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:13.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.697+0000 7f2f1aa00700 1 -- 172.21.15.1:0/4118737653 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f1c10e510 con 0x5601fde4a010 2024-02-02T15:43:13.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.698+0000 7f2f1aa00700 1 --2- 172.21.15.1:0/4118737653 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5601fde4a010 0x5601fde551f0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f2f040060b0 tx=0x7f2f04006000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67de487845580d79 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:13.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.698+0000 7f2f08c00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:43:13.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.699+0000 7f2f08c00700 1 -- 172.21.15.1:0/4118737653 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f0400e030 con 0x5601fde4a010 2024-02-02T15:43:13.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.699+0000 7f2f08c00700 1 -- 172.21.15.1:0/4118737653 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f2f0401f700 con 0x5601fde4a010 2024-02-02T15:43:13.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.699+0000 7f2f35a3cb80 1 -- 172.21.15.1:0/4118737653 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5601fde3e0f0 con 0x5601fde4a010 2024-02-02T15:43:13.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.699+0000 7f2f08c00700 1 -- 172.21.15.1:0/4118737653 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f04008f70 con 0x5601fde4a010 2024-02-02T15:43:13.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.699+0000 7f2f35a3cb80 10 client.29773 fetch_fsmap learned FSMap version 67 2024-02-02T15:43:13.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.699+0000 7f2f35a3cb80 1 -- 172.21.15.1:0/4118737653 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) v3 -- 0x5601fdbc9b60 con 0x5601fde4a010 2024-02-02T15:43:13.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.700+0000 7f2f08c00700 1 -- 172.21.15.1:0/4118737653 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5601fde3e0f0 con 0x5601fde4a010 2024-02-02T15:43:13.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.700+0000 7f2f08c00700 1 -- 172.21.15.1:0/4118737653 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 67) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f2f04019070 con 0x5601fde4a010 2024-02-02T15:43:13.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.700+0000 7f2f35a3cb80 10 client.29773 fetch_fsmap finished waiting for FSMap version 67 2024-02-02T15:43:13.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.700+0000 7f2f35a3cb80 10 client.29773 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-02-02T15:43:13.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.700+0000 7f2f35a3cb80 20 client.29773 populate_metadata read hostname 'smithi001' 2024-02-02T15:43:13.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.700+0000 7f2f35a3cb80 1 --2- 172.21.15.1:0/4118737653 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x5601fde64f30 0x5601fde67350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:13.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.700+0000 7f2f35a3cb80 4 client.29773 mds_command: new command op to 24518 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:43:13.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.700+0000 7f2f35a3cb80 1 -- 172.21.15.1:0/4118737653 --> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5601fdd94dd0 con 0x5601fde64f30 2024-02-02T15:43:13.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.700+0000 7f2f1b400700 1 --2- 172.21.15.1:0/4118737653 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x5601fde64f30 0x5601fde67350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:13.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.701+0000 7f2f1b400700 1 --2- 172.21.15.1:0/4118737653 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x5601fde64f30 0x5601fde67350 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b6ec58f251d8966c server_cookie=822786dcd9722249 in_seq=0 out_seq=0 2024-02-02T15:43:13.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.701+0000 7f2f08c00700 10 client.29773 ms_handle_connect on v2:172.21.15.71:6836/628016199 2024-02-02T15:43:13.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.702+0000 7f2f08c00700 1 -- 172.21.15.1:0/4118737653 <== mds.2 v2:172.21.15.71:6836/628016199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5601fdd94dd0 con 0x5601fde64f30 2024-02-02T15:43:13.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.702+0000 7f2f08c00700 10 client.29773 handle_command_reply: tid=0 2024-02-02T15:43:13.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.702+0000 7f2f35a3cb80 1 client.29773 shutdown 2024-02-02T15:43:13.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.702+0000 7f2f35a3cb80 1 -- 172.21.15.1:0/4118737653 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5601fde4a010 msgr2=0x5601fde551f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:13.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.702+0000 7f2f35a3cb80 1 --2- 172.21.15.1:0/4118737653 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5601fde4a010 0x5601fde551f0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f2f040060b0 tx=0x7f2f04006000 comp rx=0 tx=0).stop 2024-02-02T15:43:13.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.702+0000 7f2f35a3cb80 1 -- 172.21.15.1:0/4118737653 shutdown_connections 2024-02-02T15:43:13.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.702+0000 7f2f35a3cb80 1 --2- 172.21.15.1:0/4118737653 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5601fde557e0 0x5601fde52a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:13.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.702+0000 7f2f35a3cb80 1 --2- 172.21.15.1:0/4118737653 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x5601fde64f30 0x5601fde67350 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:13.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.702+0000 7f2f35a3cb80 1 --2- 172.21.15.1:0/4118737653 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5601fde4a010 0x5601fde551f0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:13.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.702+0000 7f2f35a3cb80 1 --2- 172.21.15.1:0/4118737653 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5601fde4a930 0x5601fde52400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:13.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.702+0000 7f2f35a3cb80 1 -- 172.21.15.1:0/4118737653 >> 172.21.15.1:0/4118737653 conn(0x5601fdda9db0 msgr2=0x5601fde45fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:13.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.703+0000 7f2f08c00700 0 client.29773 ms_handle_reset on v2:172.21.15.71:6836/628016199 2024-02-02T15:43:13.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.703+0000 7f2f35a3cb80 1 -- 172.21.15.1:0/4118737653 shutdown_connections 2024-02-02T15:43:13.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.703+0000 7f2f35a3cb80 1 -- 172.21.15.1:0/4118737653 wait complete. 2024-02-02T15:43:13.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.704+0000 7f2f35a3cb80 20 client.29773 trim_cache size 0 max 16384 2024-02-02T15:43:13.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.722+0000 7f2f35a3cb80 1 Processor -- start 2024-02-02T15:43:13.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.722+0000 7f2f35a3cb80 1 -- start start 2024-02-02T15:43:13.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.723+0000 7f2f35a3cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5601fde4a010 0x5601fde57bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:13.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.723+0000 7f2f35a3cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5601fde4a930 0x5601fde529f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:13.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.723+0000 7f2f35a3cb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5601fde557e0 0x5601fde51ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:13.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.723+0000 7f2f35a3cb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601fdc1f6a0 con 0x5601fde557e0 2024-02-02T15:43:13.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.723+0000 7f2f35a3cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601fde521e0 con 0x5601fde4a010 2024-02-02T15:43:13.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.723+0000 7f2f35a3cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601fde523c0 con 0x5601fde4a930 2024-02-02T15:43:13.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.724+0000 7f2f1aa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5601fde557e0 0x5601fde51ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:13.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.724+0000 7f2f1aa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5601fde557e0 0x5601fde51ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46344/0 (socket says 172.21.15.1:46344) 2024-02-02T15:43:13.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.724+0000 7f2f1aa00700 1 -- 172.21.15.1:0/2142759472 learned_addr learned my addr 172.21.15.1:0/2142759472 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:43:13.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.724+0000 7f2f1aa00700 1 --2- 172.21.15.1:0/2142759472 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5601fde4a010 0x5601fde57bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:13.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.724+0000 7f2f1b400700 1 --2- 172.21.15.1:0/2142759472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5601fde4a930 0x5601fde529f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:13.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.725+0000 7f2f1aa00700 1 -- 172.21.15.1:0/2142759472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5601fde4a930 msgr2=0x5601fde529f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:13.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.725+0000 7f2f1aa00700 1 --2- 172.21.15.1:0/2142759472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5601fde4a930 0x5601fde529f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:13.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.725+0000 7f2f1aa00700 1 -- 172.21.15.1:0/2142759472 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5601fde4a010 msgr2=0x5601fde57bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:13.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.725+0000 7f2f1aa00700 1 --2- 172.21.15.1:0/2142759472 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5601fde4a010 0x5601fde57bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:13.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.725+0000 7f2f1aa00700 1 -- 172.21.15.1:0/2142759472 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5601fdd61ea0 con 0x5601fde557e0 2024-02-02T15:43:13.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.725+0000 7f2f1aa00700 1 --2- 172.21.15.1:0/2142759472 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5601fde557e0 0x5601fde51ca0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f2f04007a60 tx=0x7f2f040021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1efe208280188eaa server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:13.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.725+0000 7f2f11200700 1 -- 172.21.15.1:0/2142759472 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f0400e040 con 0x5601fde557e0 2024-02-02T15:43:13.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.726+0000 7f2f11200700 1 -- 172.21.15.1:0/2142759472 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f040055d0 con 0x5601fde557e0 2024-02-02T15:43:13.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.727+0000 7f2f35a3cb80 1 -- 172.21.15.1:0/2142759472 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5601fde557e0 msgr2=0x5601fde51ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:13.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.727+0000 7f2f35a3cb80 1 --2- 172.21.15.1:0/2142759472 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5601fde557e0 0x5601fde51ca0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f2f04007a60 tx=0x7f2f040021d0 comp rx=0 tx=0).stop 2024-02-02T15:43:13.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.727+0000 7f2f35a3cb80 1 -- 172.21.15.1:0/2142759472 shutdown_connections 2024-02-02T15:43:13.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.727+0000 7f2f35a3cb80 1 --2- 172.21.15.1:0/2142759472 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5601fde557e0 0x5601fde51ca0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:13.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.727+0000 7f2f35a3cb80 1 --2- 172.21.15.1:0/2142759472 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5601fde4a010 0x5601fde57bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:13.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.727+0000 7f2f35a3cb80 1 --2- 172.21.15.1:0/2142759472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5601fde4a930 0x5601fde529f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:13.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.727+0000 7f2f35a3cb80 1 -- 172.21.15.1:0/2142759472 >> 172.21.15.1:0/2142759472 conn(0x5601fdda9db0 msgr2=0x5601fde66f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:13.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.727+0000 7f2f35a3cb80 1 -- 172.21.15.1:0/2142759472 shutdown_connections 2024-02-02T15:43:13.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.727+0000 7f2f35a3cb80 1 -- 172.21.15.1:0/2142759472 wait complete. 2024-02-02T15:43:13.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.728+0000 7f2f35a3cb80 1 -- start start 2024-02-02T15:43:13.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.729+0000 7f2f35a3cb80 1 Processor -- start 2024-02-02T15:43:13.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.730+0000 7f2f35a3cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5601fde4a010 0x5601fde56c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:13.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.730+0000 7f2f35a3cb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5601fde4a930 0x5601fde57150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:13.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.730+0000 7f2f35a3cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5601fde557e0 0x5601fdeb49f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:13.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.730+0000 7f2f35a3cb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601fde57860 con 0x5601fde4a930 2024-02-02T15:43:13.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.730+0000 7f2f35a3cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601fdeb4f30 con 0x5601fde557e0 2024-02-02T15:43:13.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.730+0000 7f2f35a3cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601fdeb50a0 con 0x5601fde4a010 2024-02-02T15:43:13.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.730+0000 7f2f35a3cb80 10 client.0 _ll_register_callbacks cb 0x5601fdd39290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:43:13.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.730+0000 7f2f1aa00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5601fde4a010 0x5601fde56c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:13.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.731+0000 7f2f1aa00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5601fde4a010 0x5601fde56c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:48870/0 (socket says 172.21.15.1:48870) 2024-02-02T15:43:13.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.731+0000 7f2f1aa00700 1 -- 172.21.15.1:0/3169167807 learned_addr learned my addr 172.21.15.1:0/3169167807 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:43:13.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.731+0000 7f2f1aa00700 1 --2- 172.21.15.1:0/3169167807 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5601fde557e0 0x5601fdeb49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:13.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.731+0000 7f2f1b400700 1 --2- 172.21.15.1:0/3169167807 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5601fde4a930 0x5601fde57150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:13.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.731+0000 7f2f1aa00700 1 -- 172.21.15.1:0/3169167807 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5601fde557e0 msgr2=0x5601fdeb49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:13.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.731+0000 7f2f1aa00700 1 --2- 172.21.15.1:0/3169167807 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5601fde557e0 0x5601fdeb49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:13.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.731+0000 7f2f1aa00700 1 -- 172.21.15.1:0/3169167807 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5601fde4a930 msgr2=0x5601fde57150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:13.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.731+0000 7f2f1aa00700 1 --2- 172.21.15.1:0/3169167807 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5601fde4a930 0x5601fde57150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:13.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.731+0000 7f2f1aa00700 1 -- 172.21.15.1:0/3169167807 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5601fdbc9b60 con 0x5601fde4a010 2024-02-02T15:43:13.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.732+0000 7f2f1aa00700 1 --2- 172.21.15.1:0/3169167807 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5601fde4a010 0x5601fde56c10 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f2f04003cc0 tx=0x7f2f0401fa40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b455d6d50dc1debd server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:13.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.732+0000 7f2f08c00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:43:13.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.732+0000 7f2f08c00700 1 -- 172.21.15.1:0/3169167807 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f0400e070 con 0x5601fde4a010 2024-02-02T15:43:13.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.732+0000 7f2f08c00700 1 -- 172.21.15.1:0/3169167807 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f2f04005110 con 0x5601fde4a010 2024-02-02T15:43:13.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.732+0000 7f2f35a3cb80 1 -- 172.21.15.1:0/3169167807 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2f0400bc50 con 0x5601fde4a010 2024-02-02T15:43:13.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.733+0000 7f2f08c00700 1 -- 172.21.15.1:0/3169167807 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f04008f70 con 0x5601fde4a010 2024-02-02T15:43:13.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.733+0000 7f2f35a3cb80 10 client.29779 fetch_fsmap learned FSMap version 67 2024-02-02T15:43:13.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.733+0000 7f2f35a3cb80 1 -- 172.21.15.1:0/3169167807 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) v3 -- 0x5601fdc21cd0 con 0x5601fde4a010 2024-02-02T15:43:13.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.734+0000 7f2f08c00700 1 -- 172.21.15.1:0/3169167807 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f0400bc50 con 0x5601fde4a010 2024-02-02T15:43:13.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.734+0000 7f2f08c00700 1 -- 172.21.15.1:0/3169167807 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 67) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f2f0403f070 con 0x5601fde4a010 2024-02-02T15:43:13.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.734+0000 7f2f35a3cb80 10 client.29779 fetch_fsmap finished waiting for FSMap version 67 2024-02-02T15:43:13.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.734+0000 7f2f35a3cb80 10 client.29779 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-02-02T15:43:13.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.734+0000 7f2f35a3cb80 20 client.29779 populate_metadata read hostname 'smithi001' 2024-02-02T15:43:13.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.734+0000 7f2f35a3cb80 1 --2- 172.21.15.1:0/3169167807 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x5601fdeb6410 0x5601fdeb8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:13.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.734+0000 7f2f35a3cb80 4 client.29779 mds_command: new command op to 24518 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:43:13.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.734+0000 7f2f35a3cb80 1 -- 172.21.15.1:0/3169167807 --> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5601fdd5af50 con 0x5601fdeb6410 2024-02-02T15:43:13.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.734+0000 7f2f1b400700 1 --2- 172.21.15.1:0/3169167807 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x5601fdeb6410 0x5601fdeb8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:13.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.736+0000 7f2f1b400700 1 --2- 172.21.15.1:0/3169167807 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x5601fdeb6410 0x5601fdeb8830 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=34debbd0c27ef6f2 server_cookie=2e983d4bdf604f10 in_seq=0 out_seq=0 2024-02-02T15:43:13.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.736+0000 7f2f08c00700 10 client.29779 ms_handle_connect on v2:172.21.15.71:6836/628016199 2024-02-02T15:43:13.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.736+0000 7f2f08c00700 1 -- 172.21.15.1:0/3169167807 <== mds.2 v2:172.21.15.71:6836/628016199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5601fdd5af50 con 0x5601fdeb6410 2024-02-02T15:43:13.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.736+0000 7f2f08c00700 10 client.29779 handle_command_reply: tid=0 2024-02-02T15:43:13.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.736+0000 7f2f35a3cb80 1 client.29779 shutdown 2024-02-02T15:43:13.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.738+0000 7f2f35a3cb80 1 -- 172.21.15.1:0/3169167807 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5601fde4a010 msgr2=0x5601fde56c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:13.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.738+0000 7f2f35a3cb80 1 --2- 172.21.15.1:0/3169167807 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5601fde4a010 0x5601fde56c10 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f2f04003cc0 tx=0x7f2f0401fa40 comp rx=0 tx=0).stop 2024-02-02T15:43:13.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.738+0000 7f2f35a3cb80 1 -- 172.21.15.1:0/3169167807 shutdown_connections 2024-02-02T15:43:13.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.738+0000 7f2f35a3cb80 1 --2- 172.21.15.1:0/3169167807 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5601fde557e0 0x5601fdeb49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:13.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.738+0000 7f2f35a3cb80 1 --2- 172.21.15.1:0/3169167807 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x5601fdeb6410 0x5601fdeb8830 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:13.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.738+0000 7f2f35a3cb80 1 --2- 172.21.15.1:0/3169167807 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5601fde4a010 0x5601fde56c10 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:13.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.738+0000 7f2f35a3cb80 1 --2- 172.21.15.1:0/3169167807 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5601fde4a930 0x5601fde57150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:13.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.738+0000 7f2f35a3cb80 1 -- 172.21.15.1:0/3169167807 >> 172.21.15.1:0/3169167807 conn(0x5601fdda9db0 msgr2=0x5601fde51ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:13.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.738+0000 7f2f08c00700 0 client.29779 ms_handle_reset on v2:172.21.15.71:6836/628016199 2024-02-02T15:43:13.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.739+0000 7f2f35a3cb80 1 -- 172.21.15.1:0/3169167807 shutdown_connections 2024-02-02T15:43:13.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.739+0000 7f2f35a3cb80 1 -- 172.21.15.1:0/3169167807 wait complete. 2024-02-02T15:43:13.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.739+0000 7f2f35a3cb80 20 client.29779 trim_cache size 0 max 16384 2024-02-02T15:43:13.740 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:43:13.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.740+0000 7f2f22000700 1 -- 172.21.15.1:0/2107631118 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f2efc0a0820 msgr2=0x7f2efc0a2ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:13.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.740+0000 7f2f22000700 1 --2- 172.21.15.1:0/2107631118 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f2efc0a0820 0x7f2efc0a2ce0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c0c9850 tx=0x7f2f0c003780 comp rx=0 tx=0).stop 2024-02-02T15:43:13.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.740+0000 7f2f22000700 1 -- 172.21.15.1:0/2107631118 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2f1c108740 msgr2=0x7f2f1c1a98f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:13.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.740+0000 7f2f22000700 1 --2- 172.21.15.1:0/2107631118 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2f1c108740 0x7f2f1c1a98f0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c00e900 tx=0x7f2f0c00ec10 comp rx=0 tx=0).stop 2024-02-02T15:43:13.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.740+0000 7f2f22000700 1 -- 172.21.15.1:0/2107631118 shutdown_connections 2024-02-02T15:43:13.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.740+0000 7f2f22000700 1 --2- 172.21.15.1:0/2107631118 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f2efc0a0820 0x7f2efc0a2ce0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:13.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.740+0000 7f2f22000700 1 --2- 172.21.15.1:0/2107631118 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2f1c108740 0x7f2f1c1a98f0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:13.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.740+0000 7f2f22000700 1 --2- 172.21.15.1:0/2107631118 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2f1c101000 0x7f2f1c107cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:13.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.740+0000 7f2f22000700 1 --2- 172.21.15.1:0/2107631118 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2f1c106120 0x7f2f1c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:13.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.740+0000 7f2f22000700 1 -- 172.21.15.1:0/2107631118 >> 172.21.15.1:0/2107631118 conn(0x7f2f1c0fc840 msgr2=0x7f2f1c0fd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:13.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.740+0000 7f2f22000700 1 -- 172.21.15.1:0/2107631118 shutdown_connections 2024-02-02T15:43:13.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:13.740+0000 7f2f22000700 1 -- 172.21.15.1:0/2107631118 wait complete. 2024-02-02T15:43:13.763 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-02T15:43:13.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:43:13 smithi071 ceph-mon[44707]: pgmap v508: 129 pgs: 129 active+clean; 1.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.4 MiB/s wr, 429 op/s 2024-02-02T15:43:13.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:43:13 smithi142 ceph-mon[45728]: pgmap v508: 129 pgs: 129 active+clean; 1.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.4 MiB/s wr, 429 op/s 2024-02-02T15:43:13.988 INFO:tasks.workunit.client.0.smithi001.stdout: 10 92214 26.45 MB/sec execute 46 sec latency 60.009 ms 2024-02-02T15:43:14.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.059+0000 7fe773200700 1 -- 172.21.15.1:0/839430977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe774108700 msgr2=0x7fe774116170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:14.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.059+0000 7fe773200700 1 --2- 172.21.15.1:0/839430977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe774108700 0x7fe774116170 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fe76c009fc0 tx=0x7fe76c00b330 comp rx=0 tx=0).stop 2024-02-02T15:43:14.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.060+0000 7fe773200700 1 -- 172.21.15.1:0/839430977 shutdown_connections 2024-02-02T15:43:14.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.061+0000 7fe773200700 1 --2- 172.21.15.1:0/839430977 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe774108700 0x7fe774116170 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:14.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.061+0000 7fe773200700 1 --2- 172.21.15.1:0/839430977 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe77410f550 0x7fe77410f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:14.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.061+0000 7fe773200700 1 --2- 172.21.15.1:0/839430977 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe774107d40 0x7fe7741081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:14.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.061+0000 7fe773200700 1 -- 172.21.15.1:0/839430977 >> 172.21.15.1:0/839430977 conn(0x7fe77406cfb0 msgr2=0x7fe77406d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:14.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.061+0000 7fe773200700 1 -- 172.21.15.1:0/839430977 shutdown_connections 2024-02-02T15:43:14.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.061+0000 7fe773200700 1 -- 172.21.15.1:0/839430977 wait complete. 2024-02-02T15:43:14.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.063+0000 7fe773200700 1 Processor -- start 2024-02-02T15:43:14.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.066+0000 7fe773200700 1 -- start start 2024-02-02T15:43:14.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.067+0000 7fe773200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe774107d40 0x7fe774108f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:14.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.067+0000 7fe773200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe774108700 0x7fe774109460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:14.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.067+0000 7fe773200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe77410f550 0x7fe7741099a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:14.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.067+0000 7fe773200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7741b1f20 con 0x7fe774108700 2024-02-02T15:43:14.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.067+0000 7fe773200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7741b2060 con 0x7fe774107d40 2024-02-02T15:43:14.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.067+0000 7fe773200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7741b21a0 con 0x7fe77410f550 2024-02-02T15:43:14.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.067+0000 7fe771e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe774107d40 0x7fe774108f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:14.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.067+0000 7fe771e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe774107d40 0x7fe774108f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:35472/0 (socket says 172.21.15.1:35472) 2024-02-02T15:43:14.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.067+0000 7fe771e00700 1 -- 172.21.15.1:0/1223551488 learned_addr learned my addr 172.21.15.1:0/1223551488 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:43:14.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.067+0000 7fe772800700 1 --2- 172.21.15.1:0/1223551488 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe77410f550 0x7fe7741099a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:14.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.068+0000 7fe771e00700 1 -- 172.21.15.1:0/1223551488 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe77410f550 msgr2=0x7fe7741099a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:14.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.068+0000 7fe771e00700 1 --2- 172.21.15.1:0/1223551488 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe77410f550 0x7fe7741099a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:14.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.068+0000 7fe771e00700 1 -- 172.21.15.1:0/1223551488 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe774108700 msgr2=0x7fe774109460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:14.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.068+0000 7fe771e00700 1 --2- 172.21.15.1:0/1223551488 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe774108700 0x7fe774109460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:14.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.068+0000 7fe771e00700 1 -- 172.21.15.1:0/1223551488 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe76c009c40 con 0x7fe774107d40 2024-02-02T15:43:14.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.068+0000 7fe771e00700 1 --2- 172.21.15.1:0/1223551488 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe774107d40 0x7fe774108f20 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fe75c00d980 tx=0x7fe75c00dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:14.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.069+0000 7fe76aa00700 1 -- 172.21.15.1:0/1223551488 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75c0098b0 con 0x7fe774107d40 2024-02-02T15:43:14.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.069+0000 7fe76aa00700 1 -- 172.21.15.1:0/1223551488 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe75c010460 con 0x7fe774107d40 2024-02-02T15:43:14.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.069+0000 7fe76aa00700 1 -- 172.21.15.1:0/1223551488 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75c00f620 con 0x7fe774107d40 2024-02-02T15:43:14.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.069+0000 7fe773200700 1 -- 172.21.15.1:0/1223551488 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7741b2480 con 0x7fe774107d40 2024-02-02T15:43:14.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.069+0000 7fe773200700 1 -- 172.21.15.1:0/1223551488 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7741b2990 con 0x7fe774107d40 2024-02-02T15:43:14.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.075+0000 7fe76aa00700 1 -- 172.21.15.1:0/1223551488 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7fe75c0105d0 con 0x7fe774107d40 2024-02-02T15:43:14.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.076+0000 7fe78cc9eb80 1 Processor -- start 2024-02-02T15:43:14.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.076+0000 7fe78cc9eb80 1 -- start start 2024-02-02T15:43:14.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.077+0000 7fe78cc9eb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a692921f0 0x558a692925d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:14.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.077+0000 7fe78cc9eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558a69292b10 0x558a6929d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:14.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.077+0000 7fe78cc9eb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a6929d9f0 0x558a6929fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:14.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.077+0000 7fe78cc9eb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a692064c0 con 0x558a6929d9f0 2024-02-02T15:43:14.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.077+0000 7fe78cc9eb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a691eaaa0 con 0x558a692921f0 2024-02-02T15:43:14.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.077+0000 7fe78cc9eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a691e8460 con 0x558a69292b10 2024-02-02T15:43:14.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.077+0000 7fe772800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a692921f0 0x558a692925d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:14.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.077+0000 7fe771400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a6929d9f0 0x558a6929fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:14.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.077+0000 7fe771e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558a69292b10 0x558a6929d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:14.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.077+0000 7fe772800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a692921f0 0x558a692925d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:35486/0 (socket says 172.21.15.1:35486) 2024-02-02T15:43:14.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.077+0000 7fe771400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a6929d9f0 0x558a6929fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46386/0 (socket says 172.21.15.1:46386) 2024-02-02T15:43:14.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.077+0000 7fe772800700 1 -- 172.21.15.1:0/1494831105 learned_addr learned my addr 172.21.15.1:0/1494831105 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:43:14.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.077+0000 7fe76aa00700 1 --2- 172.21.15.1:0/1223551488 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fe7540a07d0 0x7fe7540a2c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:14.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.078+0000 7fe76aa00700 1 -- 172.21.15.1:0/1223551488 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9541+0+0 (secure 0 0 0) 0x7fe75c0cbf10 con 0x7fe774107d40 2024-02-02T15:43:14.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.078+0000 7fe772800700 1 -- 172.21.15.1:0/1494831105 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558a69292b10 msgr2=0x558a6929d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:14.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.078+0000 7fe772800700 1 --2- 172.21.15.1:0/1494831105 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558a69292b10 0x558a6929d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:14.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.078+0000 7fe772800700 1 -- 172.21.15.1:0/1494831105 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a6929d9f0 msgr2=0x558a6929fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:14.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.078+0000 7fe772800700 1 --2- 172.21.15.1:0/1494831105 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a6929d9f0 0x558a6929fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:14.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.078+0000 7fe772800700 1 -- 172.21.15.1:0/1494831105 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558a690653b0 con 0x558a692921f0 2024-02-02T15:43:14.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.078+0000 7fe772800700 1 --2- 172.21.15.1:0/1223551488 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fe7540a07d0 0x7fe7540a2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:14.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.078+0000 7fe771400700 1 --2- 172.21.15.1:0/1494831105 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a6929d9f0 0x558a6929fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:43:14.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.079+0000 7fe772800700 1 --2- 172.21.15.1:0/1494831105 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a692921f0 0x558a692925d0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fe76c009c10 tx=0x7fe76c007cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92e715ecb706ffbd server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:14.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.079+0000 7fe772800700 1 --2- 172.21.15.1:0/1223551488 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fe7540a07d0 0x7fe7540a2c90 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe76c007d20 tx=0x7fe76c023000 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:14.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.081+0000 7fe763400700 1 -- 172.21.15.1:0/1494831105 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe76c00e040 con 0x558a692921f0 2024-02-02T15:43:14.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.081+0000 7fe763400700 1 -- 172.21.15.1:0/1494831105 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe76c003e10 con 0x558a692921f0 2024-02-02T15:43:14.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.081+0000 7fe763400700 1 -- 172.21.15.1:0/1494831105 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe76c027e50 con 0x558a692921f0 2024-02-02T15:43:14.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.081+0000 7fe78cc9eb80 1 -- 172.21.15.1:0/1494831105 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a692921f0 msgr2=0x558a692925d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:14.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.082+0000 7fe78cc9eb80 1 --2- 172.21.15.1:0/1494831105 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a692921f0 0x558a692925d0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fe76c009c10 tx=0x7fe76c007cb0 comp rx=0 tx=0).stop 2024-02-02T15:43:14.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.082+0000 7fe78cc9eb80 1 -- 172.21.15.1:0/1494831105 shutdown_connections 2024-02-02T15:43:14.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.082+0000 7fe78cc9eb80 1 --2- 172.21.15.1:0/1494831105 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a6929d9f0 0x558a6929fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:14.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.082+0000 7fe78cc9eb80 1 --2- 172.21.15.1:0/1494831105 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a692921f0 0x558a692925d0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:14.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.082+0000 7fe78cc9eb80 1 --2- 172.21.15.1:0/1494831105 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558a69292b10 0x558a6929d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:14.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.082+0000 7fe78cc9eb80 1 -- 172.21.15.1:0/1494831105 >> 172.21.15.1:0/1494831105 conn(0x558a691f1db0 msgr2=0x558a6928e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:14.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.085+0000 7fe78cc9eb80 1 -- 172.21.15.1:0/1494831105 shutdown_connections 2024-02-02T15:43:14.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.085+0000 7fe78cc9eb80 1 -- 172.21.15.1:0/1494831105 wait complete. 2024-02-02T15:43:14.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.086+0000 7fe78cc9eb80 1 -- start start 2024-02-02T15:43:14.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.086+0000 7fe78cc9eb80 1 Processor -- start 2024-02-02T15:43:14.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.088+0000 7fe78cc9eb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a692921f0 0x558a6929d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:14.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.088+0000 7fe78cc9eb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a69292b10 0x558a6929a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:14.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.088+0000 7fe78cc9eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558a6929d9f0 0x558a6929adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:14.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.088+0000 7fe78cc9eb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a691e3d50 con 0x558a692921f0 2024-02-02T15:43:14.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.088+0000 7fe78cc9eb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a6917fd80 con 0x558a69292b10 2024-02-02T15:43:14.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.088+0000 7fe78cc9eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a6917fa20 con 0x558a6929d9f0 2024-02-02T15:43:14.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.088+0000 7fe78cc9eb80 10 client.0 _ll_register_callbacks cb 0x558a69181290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:43:14.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.088+0000 7fe771400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a69292b10 0x558a6929a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:14.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.088+0000 7fe771400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a69292b10 0x558a6929a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:35496/0 (socket says 172.21.15.1:35496) 2024-02-02T15:43:14.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.088+0000 7fe771400700 1 -- 172.21.15.1:0/3757857877 learned_addr learned my addr 172.21.15.1:0/3757857877 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:43:14.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.088+0000 7fe771400700 1 -- 172.21.15.1:0/3757857877 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558a6929d9f0 msgr2=0x558a6929adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:14.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.088+0000 7fe771400700 1 --2- 172.21.15.1:0/3757857877 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558a6929d9f0 0x558a6929adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:14.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.088+0000 7fe771400700 1 -- 172.21.15.1:0/3757857877 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a692921f0 msgr2=0x558a6929d430 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-02T15:43:14.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.089+0000 7fe771400700 1 --2- 172.21.15.1:0/3757857877 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a692921f0 0x558a6929d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:14.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.089+0000 7fe771400700 1 -- 172.21.15.1:0/3757857877 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe764009420 con 0x558a69292b10 2024-02-02T15:43:14.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.089+0000 7fe771400700 1 --2- 172.21.15.1:0/3757857877 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a69292b10 0x558a6929a6b0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fe76400e4f0 tx=0x7fe76400e8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e54e859b7729f60 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:14.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.089+0000 7fe75be00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:43:14.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.089+0000 7fe75be00700 1 -- 172.21.15.1:0/3757857877 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe764009680 con 0x558a69292b10 2024-02-02T15:43:14.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.089+0000 7fe75be00700 1 -- 172.21.15.1:0/3757857877 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9541+0+0 (secure 0 0 0) 0x7fe764013680 con 0x558a69292b10 2024-02-02T15:43:14.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.090+0000 7fe78cc9eb80 1 -- 172.21.15.1:0/3757857877 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558a69286340 con 0x558a69292b10 2024-02-02T15:43:14.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.090+0000 7fe75be00700 1 -- 172.21.15.1:0/3757857877 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe764013840 con 0x558a69292b10 2024-02-02T15:43:14.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.093+0000 7fe78cc9eb80 10 client.29861 fetch_fsmap learned FSMap version 67 2024-02-02T15:43:14.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.093+0000 7fe78cc9eb80 1 -- 172.21.15.1:0/3757857877 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) v3 -- 0x7fe774116a70 con 0x558a69292b10 2024-02-02T15:43:14.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.093+0000 7fe75be00700 1 -- 172.21.15.1:0/3757857877 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558a69286340 con 0x558a69292b10 2024-02-02T15:43:14.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.094+0000 7fe75be00700 1 -- 172.21.15.1:0/3757857877 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 67) v1 ==== 7506+0+0 (secure 0 0 0) 0x7fe764011070 con 0x558a69292b10 2024-02-02T15:43:14.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.094+0000 7fe78cc9eb80 10 client.29861 fetch_fsmap finished waiting for FSMap version 67 2024-02-02T15:43:14.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.094+0000 7fe78cc9eb80 10 client.29861 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-02T15:43:14.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.094+0000 7fe78cc9eb80 20 client.29861 populate_metadata read hostname 'smithi001' 2024-02-02T15:43:14.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.094+0000 7fe78cc9eb80 1 --2- 172.21.15.1:0/3757857877 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x558a692ad3e0 0x558a692af800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:14.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.094+0000 7fe78cc9eb80 4 client.29861 mds_command: new command op to 24529 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:43:14.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.094+0000 7fe78cc9eb80 1 -- 172.21.15.1:0/3757857877 --> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558a691dcdd0 con 0x558a692ad3e0 2024-02-02T15:43:14.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.094+0000 7fe771e00700 1 --2- 172.21.15.1:0/3757857877 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x558a692ad3e0 0x558a692af800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:14.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.096+0000 7fe771e00700 1 --2- 172.21.15.1:0/3757857877 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x558a692ad3e0 0x558a692af800 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c0aad2ac9b32817b server_cookie=8d806aef9dd9d46b in_seq=0 out_seq=0 2024-02-02T15:43:14.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.096+0000 7fe75be00700 10 client.29861 ms_handle_connect on v2:172.21.15.142:6838/321675421 2024-02-02T15:43:14.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.101+0000 7fe75be00700 1 -- 172.21.15.1:0/3757857877 <== mds.1 v2:172.21.15.142:6838/321675421 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x558a691dcdd0 con 0x558a692ad3e0 2024-02-02T15:43:14.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.101+0000 7fe75be00700 10 client.29861 handle_command_reply: tid=0 2024-02-02T15:43:14.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.101+0000 7fe78cc9eb80 1 client.29861 shutdown 2024-02-02T15:43:14.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.101+0000 7fe78cc9eb80 1 -- 172.21.15.1:0/3757857877 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a69292b10 msgr2=0x558a6929a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:14.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.101+0000 7fe78cc9eb80 1 --2- 172.21.15.1:0/3757857877 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a69292b10 0x558a6929a6b0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fe76400e4f0 tx=0x7fe76400e8b0 comp rx=0 tx=0).stop 2024-02-02T15:43:14.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.101+0000 7fe78cc9eb80 1 -- 172.21.15.1:0/3757857877 shutdown_connections 2024-02-02T15:43:14.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.101+0000 7fe78cc9eb80 1 --2- 172.21.15.1:0/3757857877 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x558a692ad3e0 0x558a692af800 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:14.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.102+0000 7fe75be00700 0 client.29861 ms_handle_reset on v2:172.21.15.142:6838/321675421 2024-02-02T15:43:14.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.102+0000 7fe78cc9eb80 1 --2- 172.21.15.1:0/3757857877 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558a6929d9f0 0x558a6929adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:14.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.102+0000 7fe78cc9eb80 1 --2- 172.21.15.1:0/3757857877 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a692921f0 0x558a6929d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:14.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.102+0000 7fe78cc9eb80 1 --2- 172.21.15.1:0/3757857877 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a69292b10 0x558a6929a6b0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:14.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.102+0000 7fe78cc9eb80 1 -- 172.21.15.1:0/3757857877 >> 172.21.15.1:0/3757857877 conn(0x558a691f1db0 msgr2=0x558a6928e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:14.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.102+0000 7fe78cc9eb80 1 -- 172.21.15.1:0/3757857877 shutdown_connections 2024-02-02T15:43:14.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.102+0000 7fe78cc9eb80 1 -- 172.21.15.1:0/3757857877 wait complete. 2024-02-02T15:43:14.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.102+0000 7fe78cc9eb80 20 client.29861 trim_cache size 0 max 16384 2024-02-02T15:43:14.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.120+0000 7fe78cc9eb80 1 Processor -- start 2024-02-02T15:43:14.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.121+0000 7fe78cc9eb80 1 -- start start 2024-02-02T15:43:14.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.121+0000 7fe78cc9eb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a692921f0 0x558a692951c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:14.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.121+0000 7fe78cc9eb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a69292b10 0x558a6929cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:14.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.121+0000 7fe78cc9eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558a6929d9f0 0x558a69299f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:14.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.121+0000 7fe78cc9eb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a690676a0 con 0x558a692921f0 2024-02-02T15:43:14.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.121+0000 7fe78cc9eb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a6928ac00 con 0x558a69292b10 2024-02-02T15:43:14.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.121+0000 7fe78cc9eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a692aaa10 con 0x558a6929d9f0 2024-02-02T15:43:14.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.122+0000 7fe771400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a69292b10 0x558a6929cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:14.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.122+0000 7fe771400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a69292b10 0x558a6929cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:35502/0 (socket says 172.21.15.1:35502) 2024-02-02T15:43:14.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.122+0000 7fe771400700 1 -- 172.21.15.1:0/768101904 learned_addr learned my addr 172.21.15.1:0/768101904 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:43:14.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.122+0000 7fe771e00700 1 --2- 172.21.15.1:0/768101904 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a692921f0 0x558a692951c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:14.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.122+0000 7fe771400700 1 -- 172.21.15.1:0/768101904 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558a6929d9f0 msgr2=0x558a69299f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:14.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.122+0000 7fe771400700 1 --2- 172.21.15.1:0/768101904 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558a6929d9f0 0x558a69299f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:14.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.122+0000 7fe771400700 1 -- 172.21.15.1:0/768101904 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a692921f0 msgr2=0x558a692951c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:14.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.122+0000 7fe771400700 1 --2- 172.21.15.1:0/768101904 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a692921f0 0x558a692951c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:14.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.122+0000 7fe771400700 1 -- 172.21.15.1:0/768101904 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558a69011b60 con 0x558a69292b10 2024-02-02T15:43:14.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.122+0000 7fe771e00700 1 --2- 172.21.15.1:0/768101904 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a692921f0 0x558a692951c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:43:14.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.122+0000 7fe771400700 1 --2- 172.21.15.1:0/768101904 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a69292b10 0x558a6929cf60 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fe764012020 tx=0x7fe7640096e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6bf9ea4a19575d1 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:14.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.123+0000 7fe763e00700 1 -- 172.21.15.1:0/768101904 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe764005cc0 con 0x558a69292b10 2024-02-02T15:43:14.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.123+0000 7fe763e00700 1 -- 172.21.15.1:0/768101904 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe764012490 con 0x558a69292b10 2024-02-02T15:43:14.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.123+0000 7fe763e00700 1 -- 172.21.15.1:0/768101904 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe764013c70 con 0x558a69292b10 2024-02-02T15:43:14.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.124+0000 7fe78cc9eb80 1 -- 172.21.15.1:0/768101904 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a69292b10 msgr2=0x558a6929cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:14.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.124+0000 7fe78cc9eb80 1 --2- 172.21.15.1:0/768101904 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a69292b10 0x558a6929cf60 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fe764012020 tx=0x7fe7640096e0 comp rx=0 tx=0).stop 2024-02-02T15:43:14.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.124+0000 7fe78cc9eb80 1 -- 172.21.15.1:0/768101904 shutdown_connections 2024-02-02T15:43:14.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.124+0000 7fe78cc9eb80 1 --2- 172.21.15.1:0/768101904 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558a6929d9f0 0x558a69299f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:14.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.124+0000 7fe78cc9eb80 1 --2- 172.21.15.1:0/768101904 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a692921f0 0x558a692951c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:14.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.124+0000 7fe78cc9eb80 1 --2- 172.21.15.1:0/768101904 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a69292b10 0x558a6929cf60 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:14.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.124+0000 7fe78cc9eb80 1 -- 172.21.15.1:0/768101904 >> 172.21.15.1:0/768101904 conn(0x558a691f1db0 msgr2=0x558a692a9630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:14.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.125+0000 7fe78cc9eb80 1 -- 172.21.15.1:0/768101904 shutdown_connections 2024-02-02T15:43:14.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.125+0000 7fe78cc9eb80 1 -- 172.21.15.1:0/768101904 wait complete. 2024-02-02T15:43:14.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.126+0000 7fe78cc9eb80 1 -- start start 2024-02-02T15:43:14.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.126+0000 7fe78cc9eb80 1 Processor -- start 2024-02-02T15:43:14.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.127+0000 7fe78cc9eb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a692921f0 0x558a692a3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:14.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.127+0000 7fe78cc9eb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a69292b10 0x558a692a4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:14.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.127+0000 7fe78cc9eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558a6929d9f0 0x558a692a23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:14.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.127+0000 7fe771e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a692921f0 0x558a692a3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:14.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.127+0000 7fe771e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a692921f0 0x558a692a3ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:35518/0 (socket says 172.21.15.1:35518) 2024-02-02T15:43:14.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.128+0000 7fe771e00700 1 -- 172.21.15.1:0/4069337975 learned_addr learned my addr 172.21.15.1:0/4069337975 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:43:14.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.128+0000 7fe78cc9eb80 1 -- 172.21.15.1:0/4069337975 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a692a2b00 con 0x558a69292b10 2024-02-02T15:43:14.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.128+0000 7fe78cc9eb80 1 -- 172.21.15.1:0/4069337975 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a692a2c70 con 0x558a692921f0 2024-02-02T15:43:14.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.128+0000 7fe78cc9eb80 1 -- 172.21.15.1:0/4069337975 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a692a2e30 con 0x558a6929d9f0 2024-02-02T15:43:14.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.128+0000 7fe78cc9eb80 10 client.0 _ll_register_callbacks cb 0x558a69181290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:43:14.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.128+0000 7fe772800700 1 --2- 172.21.15.1:0/4069337975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558a6929d9f0 0x558a692a23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:14.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.128+0000 7fe771400700 1 --2- 172.21.15.1:0/4069337975 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a69292b10 0x558a692a4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:14.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.128+0000 7fe771e00700 1 -- 172.21.15.1:0/4069337975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558a6929d9f0 msgr2=0x558a692a23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:14.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.129+0000 7fe771e00700 1 --2- 172.21.15.1:0/4069337975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558a6929d9f0 0x558a692a23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:14.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.129+0000 7fe771e00700 1 -- 172.21.15.1:0/4069337975 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a69292b10 msgr2=0x558a692a4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:14.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.129+0000 7fe771e00700 1 --2- 172.21.15.1:0/4069337975 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a69292b10 0x558a692a4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:14.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.129+0000 7fe771e00700 1 -- 172.21.15.1:0/4069337975 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe774116a70 con 0x558a692921f0 2024-02-02T15:43:14.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.129+0000 7fe772800700 1 --2- 172.21.15.1:0/4069337975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558a6929d9f0 0x558a692a23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:43:14.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.129+0000 7fe771400700 1 --2- 172.21.15.1:0/4069337975 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a69292b10 0x558a692a4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:43:14.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.129+0000 7fe771e00700 1 --2- 172.21.15.1:0/4069337975 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a692921f0 0x558a692a3ec0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fe75c010880 tx=0x7fe75c0c9260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41387112bc2e7058 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:14.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.129+0000 7fe75be00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:43:14.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.130+0000 7fe75be00700 1 -- 172.21.15.1:0/4069337975 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75c014020 con 0x558a692921f0 2024-02-02T15:43:14.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.130+0000 7fe75be00700 1 -- 172.21.15.1:0/4069337975 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9541+0+0 (secure 0 0 0) 0x7fe75c093be0 con 0x558a692921f0 2024-02-02T15:43:14.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.130+0000 7fe78cc9eb80 1 -- 172.21.15.1:0/4069337975 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558a691a2f50 con 0x558a692921f0 2024-02-02T15:43:14.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.131+0000 7fe75be00700 1 -- 172.21.15.1:0/4069337975 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe75c093da0 con 0x558a692921f0 2024-02-02T15:43:14.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.131+0000 7fe78cc9eb80 10 client.29867 fetch_fsmap learned FSMap version 67 2024-02-02T15:43:14.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.131+0000 7fe78cc9eb80 1 -- 172.21.15.1:0/4069337975 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) v3 -- 0x558a691a9ea0 con 0x558a692921f0 2024-02-02T15:43:14.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.131+0000 7fe75be00700 1 -- 172.21.15.1:0/4069337975 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe75c0b5020 con 0x558a692921f0 2024-02-02T15:43:14.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.131+0000 7fe75be00700 1 -- 172.21.15.1:0/4069337975 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 67) v1 ==== 7506+0+0 (secure 0 0 0) 0x7fe75c0a0bb0 con 0x558a692921f0 2024-02-02T15:43:14.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.131+0000 7fe78cc9eb80 10 client.29867 fetch_fsmap finished waiting for FSMap version 67 2024-02-02T15:43:14.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.131+0000 7fe78cc9eb80 10 client.29867 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-02T15:43:14.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.131+0000 7fe78cc9eb80 20 client.29867 populate_metadata read hostname 'smithi001' 2024-02-02T15:43:14.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.131+0000 7fe78cc9eb80 1 --2- 172.21.15.1:0/4069337975 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x558a69300020 0x558a69302440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:14.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.131+0000 7fe78cc9eb80 4 client.29867 mds_command: new command op to 24529 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:43:14.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.131+0000 7fe78cc9eb80 1 -- 172.21.15.1:0/4069337975 --> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558a69032cf0 con 0x558a69300020 2024-02-02T15:43:14.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.132+0000 7fe771400700 1 --2- 172.21.15.1:0/4069337975 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x558a69300020 0x558a69302440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:14.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.132+0000 7fe771400700 1 --2- 172.21.15.1:0/4069337975 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x558a69300020 0x558a69302440 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e475be994a951fb4 server_cookie=f790333659e78154 in_seq=0 out_seq=0 2024-02-02T15:43:14.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.132+0000 7fe75be00700 10 client.29867 ms_handle_connect on v2:172.21.15.142:6838/321675421 2024-02-02T15:43:14.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.134+0000 7fe75be00700 1 -- 172.21.15.1:0/4069337975 <== mds.1 v2:172.21.15.142:6838/321675421 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558a69032cf0 con 0x558a69300020 2024-02-02T15:43:14.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.134+0000 7fe75be00700 10 client.29867 handle_command_reply: tid=0 2024-02-02T15:43:14.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.134+0000 7fe78cc9eb80 1 client.29867 shutdown 2024-02-02T15:43:14.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.134+0000 7fe78cc9eb80 1 -- 172.21.15.1:0/4069337975 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a692921f0 msgr2=0x558a692a3ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:14.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.134+0000 7fe78cc9eb80 1 --2- 172.21.15.1:0/4069337975 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a692921f0 0x558a692a3ec0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fe75c010880 tx=0x7fe75c0c9260 comp rx=0 tx=0).stop 2024-02-02T15:43:14.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.134+0000 7fe78cc9eb80 1 -- 172.21.15.1:0/4069337975 shutdown_connections 2024-02-02T15:43:14.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.134+0000 7fe78cc9eb80 1 --2- 172.21.15.1:0/4069337975 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x558a69300020 0x558a69302440 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:14.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.135+0000 7fe78cc9eb80 1 --2- 172.21.15.1:0/4069337975 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558a6929d9f0 0x558a692a23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:14.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.135+0000 7fe78cc9eb80 1 --2- 172.21.15.1:0/4069337975 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558a692921f0 0x558a692a3ec0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:14.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.135+0000 7fe78cc9eb80 1 --2- 172.21.15.1:0/4069337975 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558a69292b10 0x558a692a4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:14.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.135+0000 7fe78cc9eb80 1 -- 172.21.15.1:0/4069337975 >> 172.21.15.1:0/4069337975 conn(0x558a691f1db0 msgr2=0x558a6928d810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:14.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.135+0000 7fe75be00700 0 client.29867 ms_handle_reset on v2:172.21.15.142:6838/321675421 2024-02-02T15:43:14.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.135+0000 7fe78cc9eb80 1 -- 172.21.15.1:0/4069337975 shutdown_connections 2024-02-02T15:43:14.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.135+0000 7fe78cc9eb80 1 -- 172.21.15.1:0/4069337975 wait complete. 2024-02-02T15:43:14.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.135+0000 7fe78cc9eb80 20 client.29867 trim_cache size 0 max 16384 2024-02-02T15:43:14.136 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:43:14.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.136+0000 7fe773200700 1 -- 172.21.15.1:0/1223551488 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fe7540a07d0 msgr2=0x7fe7540a2c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:14.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.136+0000 7fe773200700 1 --2- 172.21.15.1:0/1223551488 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fe7540a07d0 0x7fe7540a2c90 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe76c007d20 tx=0x7fe76c023000 comp rx=0 tx=0).stop 2024-02-02T15:43:14.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.136+0000 7fe773200700 1 -- 172.21.15.1:0/1223551488 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe774107d40 msgr2=0x7fe774108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:14.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.136+0000 7fe773200700 1 --2- 172.21.15.1:0/1223551488 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe774107d40 0x7fe774108f20 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fe75c00d980 tx=0x7fe75c00dc90 comp rx=0 tx=0).stop 2024-02-02T15:43:14.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.136+0000 7fe773200700 1 -- 172.21.15.1:0/1223551488 shutdown_connections 2024-02-02T15:43:14.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.136+0000 7fe773200700 1 --2- 172.21.15.1:0/1223551488 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fe7540a07d0 0x7fe7540a2c90 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:14.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.136+0000 7fe773200700 1 --2- 172.21.15.1:0/1223551488 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe77410f550 0x7fe7741099a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:14.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.136+0000 7fe773200700 1 --2- 172.21.15.1:0/1223551488 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe774107d40 0x7fe774108f20 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:14.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.136+0000 7fe773200700 1 --2- 172.21.15.1:0/1223551488 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe774108700 0x7fe774109460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:14.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.136+0000 7fe773200700 1 -- 172.21.15.1:0/1223551488 >> 172.21.15.1:0/1223551488 conn(0x7fe77406cfb0 msgr2=0x7fe774071040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:14.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.136+0000 7fe773200700 1 -- 172.21.15.1:0/1223551488 shutdown_connections 2024-02-02T15:43:14.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.136+0000 7fe773200700 1 -- 172.21.15.1:0/1223551488 wait complete. 2024-02-02T15:43:14.164 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-02T15:43:14.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.526+0000 7ff29c600700 1 -- 172.21.15.1:0/1371055240 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff294107fb0 msgr2=0x7ff29410bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:14.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.526+0000 7ff29c600700 1 --2- 172.21.15.1:0/1371055240 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff294107fb0 0x7ff29410bff0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7ff27c009990 tx=0x7ff27c009ca0 comp rx=0 tx=0).stop 2024-02-02T15:43:14.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.527+0000 7ff29c600700 1 -- 172.21.15.1:0/1371055240 shutdown_connections 2024-02-02T15:43:14.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.527+0000 7ff29c600700 1 --2- 172.21.15.1:0/1371055240 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff29410c720 0x7ff2941142a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:14.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.527+0000 7ff29c600700 1 --2- 172.21.15.1:0/1371055240 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff294107600 0x7ff2941079e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:14.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.527+0000 7ff29c600700 1 --2- 172.21.15.1:0/1371055240 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff294107fb0 0x7ff29410bff0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:14.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.527+0000 7ff29c600700 1 -- 172.21.15.1:0/1371055240 >> 172.21.15.1:0/1371055240 conn(0x7ff294076b80 msgr2=0x7ff294105b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:14.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.527+0000 7ff29c600700 1 -- 172.21.15.1:0/1371055240 shutdown_connections 2024-02-02T15:43:14.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.528+0000 7ff29c600700 1 -- 172.21.15.1:0/1371055240 wait complete. 2024-02-02T15:43:14.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.530+0000 7ff29c600700 1 Processor -- start 2024-02-02T15:43:14.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.530+0000 7ff29c600700 1 -- start start 2024-02-02T15:43:14.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.531+0000 7ff29c600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff294107600 0x7ff29419d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:14.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.531+0000 7ff29c600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff29410c720 0x7ff29419da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:14.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.531+0000 7ff29c600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2941a1e80 0x7ff2941a2300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:14.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.531+0000 7ff29c600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2941a2900 con 0x7ff29410c720 2024-02-02T15:43:14.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.531+0000 7ff29c600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2941a2a70 con 0x7ff294107600 2024-02-02T15:43:14.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.531+0000 7ff29c600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2941a2c30 con 0x7ff2941a1e80 2024-02-02T15:43:14.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.531+0000 7ff299c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff294107600 0x7ff29419d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:14.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.531+0000 7ff299c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff294107600 0x7ff29419d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:35546/0 (socket says 172.21.15.1:35546) 2024-02-02T15:43:14.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.532+0000 7ff299c00700 1 -- 172.21.15.1:0/3889813129 learned_addr learned my addr 172.21.15.1:0/3889813129 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:43:14.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.532+0000 7ff299c00700 1 -- 172.21.15.1:0/3889813129 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2941a1e80 msgr2=0x7ff2941a2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:14.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.532+0000 7ff299c00700 1 --2- 172.21.15.1:0/3889813129 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2941a1e80 0x7ff2941a2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:14.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.532+0000 7ff299200700 1 --2- 172.21.15.1:0/3889813129 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff29410c720 0x7ff29419da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:14.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.532+0000 7ff299c00700 1 -- 172.21.15.1:0/3889813129 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff29410c720 msgr2=0x7ff29419da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:14.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.532+0000 7ff299c00700 1 --2- 172.21.15.1:0/3889813129 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff29410c720 0x7ff29419da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:14.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.532+0000 7ff299c00700 1 -- 172.21.15.1:0/3889813129 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff27c0095f0 con 0x7ff294107600 2024-02-02T15:43:14.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.532+0000 7ff299200700 1 --2- 172.21.15.1:0/3889813129 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff29410c720 0x7ff29419da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:43:14.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.533+0000 7ff299c00700 1 --2- 172.21.15.1:0/3889813129 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff294107600 0x7ff29419d4e0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7ff28c00ba50 tx=0x7ff28c00be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:14.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.533+0000 7ff292000700 1 -- 172.21.15.1:0/3889813129 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff28c00cab0 con 0x7ff294107600 2024-02-02T15:43:14.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.533+0000 7ff292000700 1 -- 172.21.15.1:0/3889813129 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff28c00cc10 con 0x7ff294107600 2024-02-02T15:43:14.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.533+0000 7ff292000700 1 -- 172.21.15.1:0/3889813129 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff28c0075b0 con 0x7ff294107600 2024-02-02T15:43:14.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.533+0000 7ff29c600700 1 -- 172.21.15.1:0/3889813129 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2941aa190 con 0x7ff294107600 2024-02-02T15:43:14.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.533+0000 7ff29c600700 1 -- 172.21.15.1:0/3889813129 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2941aa620 con 0x7ff294107600 2024-02-02T15:43:14.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.541+0000 7ff292000700 1 -- 172.21.15.1:0/3889813129 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7ff28c014440 con 0x7ff294107600 2024-02-02T15:43:14.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.544+0000 7ff292000700 1 --2- 172.21.15.1:0/3889813129 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff2740a0860 0x7ff2740a2d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:14.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.544+0000 7ff292000700 1 -- 172.21.15.1:0/3889813129 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9541+0+0 (secure 0 0 0) 0x7ff28c0cb800 con 0x7ff294107600 2024-02-02T15:43:14.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.546+0000 7ff299200700 1 --2- 172.21.15.1:0/3889813129 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff2740a0860 0x7ff2740a2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:14.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.548+0000 7ff299200700 1 --2- 172.21.15.1:0/3889813129 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff2740a0860 0x7ff2740a2d20 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff27c006010 tx=0x7ff27c00b540 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:14.550 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.551+0000 7ff2b0125b80 1 Processor -- start 2024-02-02T15:43:14.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.551+0000 7ff2b0125b80 1 -- start start 2024-02-02T15:43:14.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.552+0000 7ff2b0125b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9fcddb010 0x55d9fcddb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:14.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.552+0000 7ff2b0125b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d9fcddb930 0x55d9fcde62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:14.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.552+0000 7ff2b0125b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9fcde67e0 0x55d9fcde8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:14.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.552+0000 7ff2b0125b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9fcd4f4c0 con 0x55d9fcddb010 2024-02-02T15:43:14.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.552+0000 7ff2b0125b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9fcd33aa0 con 0x55d9fcddb930 2024-02-02T15:43:14.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.552+0000 7ff2b0125b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9fcd31460 con 0x55d9fcde67e0 2024-02-02T15:43:14.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.553+0000 7ff299200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d9fcddb930 0x55d9fcde62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:14.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.553+0000 7ff299200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d9fcddb930 0x55d9fcde62a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:35554/0 (socket says 172.21.15.1:35554) 2024-02-02T15:43:14.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.553+0000 7ff299200700 1 -- 172.21.15.1:0/3474434993 learned_addr learned my addr 172.21.15.1:0/3474434993 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:43:14.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.554+0000 7ff299200700 1 -- 172.21.15.1:0/3474434993 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9fcde67e0 msgr2=0x55d9fcde8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:14.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.554+0000 7ff299200700 1 --2- 172.21.15.1:0/3474434993 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9fcde67e0 0x55d9fcde8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:14.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.554+0000 7ff299200700 1 -- 172.21.15.1:0/3474434993 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9fcddb010 msgr2=0x55d9fcddb3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:14.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.554+0000 7ff299200700 1 --2- 172.21.15.1:0/3474434993 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9fcddb010 0x55d9fcddb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:14.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.554+0000 7ff299200700 1 -- 172.21.15.1:0/3474434993 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d9fcbae3b0 con 0x55d9fcddb930 2024-02-02T15:43:14.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.555+0000 7ff299200700 1 --2- 172.21.15.1:0/3474434993 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d9fcddb930 0x55d9fcde62a0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7ff27c0036d0 tx=0x7ff27c01d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=409278751dfbf9a7 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:14.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.555+0000 7ff28aa00700 1 -- 172.21.15.1:0/3474434993 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27c029070 con 0x55d9fcddb930 2024-02-02T15:43:14.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.555+0000 7ff28aa00700 1 -- 172.21.15.1:0/3474434993 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff27c003d70 con 0x55d9fcddb930 2024-02-02T15:43:14.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.555+0000 7ff28aa00700 1 -- 172.21.15.1:0/3474434993 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27c01d070 con 0x55d9fcddb930 2024-02-02T15:43:14.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.557+0000 7ff2b0125b80 1 -- 172.21.15.1:0/3474434993 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d9fcddb930 msgr2=0x55d9fcde62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:14.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.557+0000 7ff2b0125b80 1 --2- 172.21.15.1:0/3474434993 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d9fcddb930 0x55d9fcde62a0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7ff27c0036d0 tx=0x7ff27c01d000 comp rx=0 tx=0).stop 2024-02-02T15:43:14.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.559+0000 7ff2b0125b80 1 -- 172.21.15.1:0/3474434993 shutdown_connections 2024-02-02T15:43:14.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.559+0000 7ff2b0125b80 1 --2- 172.21.15.1:0/3474434993 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9fcde67e0 0x55d9fcde8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:14.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.559+0000 7ff2b0125b80 1 --2- 172.21.15.1:0/3474434993 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9fcddb010 0x55d9fcddb3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:14.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.560+0000 7ff2b0125b80 1 --2- 172.21.15.1:0/3474434993 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d9fcddb930 0x55d9fcde62a0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:14.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.560+0000 7ff2b0125b80 1 -- 172.21.15.1:0/3474434993 >> 172.21.15.1:0/3474434993 conn(0x55d9fcd3adb0 msgr2=0x55d9fcdd73b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:14.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.561+0000 7ff2b0125b80 1 -- 172.21.15.1:0/3474434993 shutdown_connections 2024-02-02T15:43:14.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.561+0000 7ff2b0125b80 1 -- 172.21.15.1:0/3474434993 wait complete. 2024-02-02T15:43:14.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.563+0000 7ff2b0125b80 1 -- start start 2024-02-02T15:43:14.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.563+0000 7ff2b0125b80 1 Processor -- start 2024-02-02T15:43:14.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.568+0000 7ff2b0125b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9fcddb010 0x55d9fcde6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:14.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.568+0000 7ff2b0125b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9fcddb930 0x55d9fcde3470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:14.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.568+0000 7ff2b0125b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d9fcde67e0 0x55d9fcde3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:14.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.568+0000 7ff2b0125b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9fcd2cd50 con 0x55d9fcddb010 2024-02-02T15:43:14.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.568+0000 7ff2b0125b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9fccc8d80 con 0x55d9fcde67e0 2024-02-02T15:43:14.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.568+0000 7ff2b0125b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9fccc8a20 con 0x55d9fcddb930 2024-02-02T15:43:14.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.568+0000 7ff2b0125b80 10 client.0 _ll_register_callbacks cb 0x55d9fccca290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:43:14.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.569+0000 7ff29a600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d9fcde67e0 0x55d9fcde3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:14.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.569+0000 7ff29a600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d9fcde67e0 0x55d9fcde3b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:35564/0 (socket says 172.21.15.1:35564) 2024-02-02T15:43:14.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.569+0000 7ff29a600700 1 -- 172.21.15.1:0/2556531605 learned_addr learned my addr 172.21.15.1:0/2556531605 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:43:14.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.569+0000 7ff299200700 1 --2- 172.21.15.1:0/2556531605 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9fcddb930 0x55d9fcde3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:14.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.570+0000 7ff29a600700 1 -- 172.21.15.1:0/2556531605 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9fcddb930 msgr2=0x55d9fcde3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:14.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.570+0000 7ff29a600700 1 --2- 172.21.15.1:0/2556531605 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9fcddb930 0x55d9fcde3470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:14.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.570+0000 7ff29a600700 1 -- 172.21.15.1:0/2556531605 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9fcddb010 msgr2=0x55d9fcde6260 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-02T15:43:14.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.570+0000 7ff299c00700 1 --2- 172.21.15.1:0/2556531605 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9fcddb010 0x55d9fcde6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:14.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.571+0000 7ff29a600700 1 --2- 172.21.15.1:0/2556531605 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9fcddb010 0x55d9fcde6260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:14.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.571+0000 7ff29a600700 1 -- 172.21.15.1:0/2556531605 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff294114b10 con 0x55d9fcde67e0 2024-02-02T15:43:14.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.572+0000 7ff29a600700 1 --2- 172.21.15.1:0/2556531605 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d9fcde67e0 0x55d9fcde3b60 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7ff28400bca0 tx=0x7ff28400d2e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17ac0b24e16f81a1 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:14.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.572+0000 7ff283400700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:43:14.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.576+0000 7ff283400700 1 -- 172.21.15.1:0/2556531605 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff28400de20 con 0x55d9fcde67e0 2024-02-02T15:43:14.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.576+0000 7ff2b0125b80 1 -- 172.21.15.1:0/2556531605 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d9fcdcf0f0 con 0x55d9fcde67e0 2024-02-02T15:43:14.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.578+0000 7ff283400700 1 -- 172.21.15.1:0/2556531605 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9541+0+0 (secure 0 0 0) 0x7ff284010460 con 0x55d9fcde67e0 2024-02-02T15:43:14.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.580+0000 7ff283400700 1 -- 172.21.15.1:0/2556531605 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff284010620 con 0x55d9fcde67e0 2024-02-02T15:43:14.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.580+0000 7ff2b0125b80 10 client.29879 fetch_fsmap learned FSMap version 67 2024-02-02T15:43:14.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.580+0000 7ff2b0125b80 1 -- 172.21.15.1:0/2556531605 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) v3 -- 0x55d9fcb5ab60 con 0x55d9fcde67e0 2024-02-02T15:43:14.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.580+0000 7ff283400700 1 -- 172.21.15.1:0/2556531605 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d9fcdcf0f0 con 0x55d9fcde67e0 2024-02-02T15:43:14.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.582+0000 7ff283400700 1 -- 172.21.15.1:0/2556531605 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 67) v1 ==== 7506+0+0 (secure 0 0 0) 0x7ff284010780 con 0x55d9fcde67e0 2024-02-02T15:43:14.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.582+0000 7ff2b0125b80 10 client.29879 fetch_fsmap finished waiting for FSMap version 67 2024-02-02T15:43:14.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.582+0000 7ff2b0125b80 10 client.29879 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:43:14.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.582+0000 7ff2b0125b80 20 client.29879 populate_metadata read hostname 'smithi001' 2024-02-02T15:43:14.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.582+0000 7ff2b0125b80 1 --2- 172.21.15.1:0/2556531605 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55d9fce070d0 0x55d9fce094f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:14.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.582+0000 7ff2b0125b80 4 client.29879 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:43:14.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.582+0000 7ff2b0125b80 1 -- 172.21.15.1:0/2556531605 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d9fcd25dd0 con 0x55d9fce070d0 2024-02-02T15:43:14.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.583+0000 7ff299c00700 1 --2- 172.21.15.1:0/2556531605 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55d9fce070d0 0x55d9fce094f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:14.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.584+0000 7ff299c00700 1 --2- 172.21.15.1:0/2556531605 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55d9fce070d0 0x55d9fce094f0 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=234adee983b965b server_cookie=be1d159cd67a6695 in_seq=0 out_seq=0 2024-02-02T15:43:14.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.584+0000 7ff283400700 10 client.29879 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:43:14.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.585+0000 7ff283400700 1 -- 172.21.15.1:0/2556531605 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55d9fcd25dd0 con 0x55d9fce070d0 2024-02-02T15:43:14.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.586+0000 7ff283400700 10 client.29879 handle_command_reply: tid=0 2024-02-02T15:43:14.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.586+0000 7ff2b0125b80 1 client.29879 shutdown 2024-02-02T15:43:14.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.586+0000 7ff2b0125b80 1 -- 172.21.15.1:0/2556531605 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d9fcde67e0 msgr2=0x55d9fcde3b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:14.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.586+0000 7ff2b0125b80 1 --2- 172.21.15.1:0/2556531605 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d9fcde67e0 0x55d9fcde3b60 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7ff28400bca0 tx=0x7ff28400d2e0 comp rx=0 tx=0).stop 2024-02-02T15:43:14.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.587+0000 7ff2b0125b80 1 -- 172.21.15.1:0/2556531605 shutdown_connections 2024-02-02T15:43:14.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.587+0000 7ff2b0125b80 1 --2- 172.21.15.1:0/2556531605 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55d9fce070d0 0x55d9fce094f0 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:14.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.587+0000 7ff2b0125b80 1 --2- 172.21.15.1:0/2556531605 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d9fcde67e0 0x55d9fcde3b60 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:14.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.587+0000 7ff2b0125b80 1 --2- 172.21.15.1:0/2556531605 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9fcddb010 0x55d9fcde6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:14.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.587+0000 7ff2b0125b80 1 --2- 172.21.15.1:0/2556531605 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9fcddb930 0x55d9fcde3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:14.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.587+0000 7ff283400700 0 client.29879 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:43:14.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.587+0000 7ff2b0125b80 1 -- 172.21.15.1:0/2556531605 >> 172.21.15.1:0/2556531605 conn(0x55d9fcd3adb0 msgr2=0x55d9fcdd6f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:14.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.588+0000 7ff2b0125b80 1 -- 172.21.15.1:0/2556531605 shutdown_connections 2024-02-02T15:43:14.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.589+0000 7ff2b0125b80 1 -- 172.21.15.1:0/2556531605 wait complete. 2024-02-02T15:43:14.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.589+0000 7ff2b0125b80 20 client.29879 trim_cache size 0 max 16384 2024-02-02T15:43:14.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.617+0000 7ff2b0125b80 1 Processor -- start 2024-02-02T15:43:14.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.617+0000 7ff2b0125b80 1 -- start start 2024-02-02T15:43:14.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.618+0000 7ff2b0125b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9fcddb010 0x55d9fcde2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:14.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.618+0000 7ff2b0125b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d9fcddb930 0x55d9fcde5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:14.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.618+0000 7ff2b0125b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9fcde67e0 0x55d9fce028f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:14.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.618+0000 7ff2b0125b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9fcbb06a0 con 0x55d9fcddb010 2024-02-02T15:43:14.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.618+0000 7ff2b0125b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9fcddbf10 con 0x55d9fcddb930 2024-02-02T15:43:14.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.618+0000 7ff2b0125b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9fcdd1500 con 0x55d9fcde67e0 2024-02-02T15:43:14.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.618+0000 7ff299200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d9fcddb930 0x55d9fcde5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:14.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.618+0000 7ff299200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d9fcddb930 0x55d9fcde5d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:35570/0 (socket says 172.21.15.1:35570) 2024-02-02T15:43:14.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.618+0000 7ff299200700 1 -- 172.21.15.1:0/1205254717 learned_addr learned my addr 172.21.15.1:0/1205254717 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:43:14.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.618+0000 7ff29a600700 1 --2- 172.21.15.1:0/1205254717 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9fcde67e0 0x55d9fce028f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:14.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.619+0000 7ff299200700 1 -- 172.21.15.1:0/1205254717 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9fcde67e0 msgr2=0x55d9fce028f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:14.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.619+0000 7ff299200700 1 --2- 172.21.15.1:0/1205254717 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9fcde67e0 0x55d9fce028f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:14.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.619+0000 7ff299200700 1 -- 172.21.15.1:0/1205254717 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9fcddb010 msgr2=0x55d9fcde2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:14.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.619+0000 7ff299200700 1 --2- 172.21.15.1:0/1205254717 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9fcddb010 0x55d9fcde2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:14.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.619+0000 7ff299200700 1 -- 172.21.15.1:0/1205254717 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d9fccf2ea0 con 0x55d9fcddb930 2024-02-02T15:43:14.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.619+0000 7ff299200700 1 --2- 172.21.15.1:0/1205254717 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d9fcddb930 0x55d9fcde5d90 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7ff27c003c40 tx=0x7ff27c009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=894576f939f82753 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:14.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.620+0000 7ff28b400700 1 -- 172.21.15.1:0/1205254717 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27c029040 con 0x55d9fcddb930 2024-02-02T15:43:14.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.620+0000 7ff28b400700 1 -- 172.21.15.1:0/1205254717 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff27c032070 con 0x55d9fcddb930 2024-02-02T15:43:14.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.620+0000 7ff28b400700 1 -- 172.21.15.1:0/1205254717 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff27c00f8e0 con 0x55d9fcddb930 2024-02-02T15:43:14.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.621+0000 7ff2b0125b80 1 -- 172.21.15.1:0/1205254717 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d9fcddb930 msgr2=0x55d9fcde5d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:14.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.621+0000 7ff2b0125b80 1 --2- 172.21.15.1:0/1205254717 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d9fcddb930 0x55d9fcde5d90 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7ff27c003c40 tx=0x7ff27c009310 comp rx=0 tx=0).stop 2024-02-02T15:43:14.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.622+0000 7ff2b0125b80 1 -- 172.21.15.1:0/1205254717 shutdown_connections 2024-02-02T15:43:14.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.622+0000 7ff2b0125b80 1 --2- 172.21.15.1:0/1205254717 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9fcde67e0 0x55d9fce028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:14.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.622+0000 7ff2b0125b80 1 --2- 172.21.15.1:0/1205254717 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9fcddb010 0x55d9fcde2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:14.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.622+0000 7ff2b0125b80 1 --2- 172.21.15.1:0/1205254717 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d9fcddb930 0x55d9fcde5d90 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:14.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.622+0000 7ff2b0125b80 1 -- 172.21.15.1:0/1205254717 >> 172.21.15.1:0/1205254717 conn(0x55d9fcd3adb0 msgr2=0x55d9fcdd5b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:14.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.623+0000 7ff2b0125b80 1 -- 172.21.15.1:0/1205254717 shutdown_connections 2024-02-02T15:43:14.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.623+0000 7ff2b0125b80 1 -- 172.21.15.1:0/1205254717 wait complete. 2024-02-02T15:43:14.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.624+0000 7ff2b0125b80 1 -- start start 2024-02-02T15:43:14.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.625+0000 7ff2b0125b80 1 Processor -- start 2024-02-02T15:43:14.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.626+0000 7ff2b0125b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d9fcddb010 0x55d9fcdfc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:14.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.626+0000 7ff2b0125b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9fcddb930 0x55d9fcdfc840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:14.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.626+0000 7ff2b0125b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9fcde67e0 0x55d9fcdfcd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:14.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.626+0000 7ff2b0125b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9fce57bb0 con 0x55d9fcddb930 2024-02-02T15:43:14.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.626+0000 7ff2b0125b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9fce57cf0 con 0x55d9fcddb010 2024-02-02T15:43:14.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.626+0000 7ff2b0125b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9fce57e30 con 0x55d9fcde67e0 2024-02-02T15:43:14.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.626+0000 7ff2b0125b80 10 client.0 _ll_register_callbacks cb 0x55d9fccca290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:43:14.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.627+0000 7ff299c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d9fcddb010 0x55d9fcdfc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:14.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.627+0000 7ff29a600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9fcde67e0 0x55d9fcdfcd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:14.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.627+0000 7ff29a600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9fcde67e0 0x55d9fcdfcd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:48962/0 (socket says 172.21.15.1:48962) 2024-02-02T15:43:14.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.627+0000 7ff29a600700 1 -- 172.21.15.1:0/2169523852 learned_addr learned my addr 172.21.15.1:0/2169523852 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:43:14.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.628+0000 7ff299c00700 1 -- 172.21.15.1:0/2169523852 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9fcde67e0 msgr2=0x55d9fcdfcd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:14.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.628+0000 7ff299c00700 1 --2- 172.21.15.1:0/2169523852 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9fcde67e0 0x55d9fcdfcd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:14.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.628+0000 7ff299c00700 1 -- 172.21.15.1:0/2169523852 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9fcddb930 msgr2=0x55d9fcdfc840 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-02T15:43:14.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.628+0000 7ff299200700 1 --2- 172.21.15.1:0/2169523852 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9fcddb930 0x55d9fcdfc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:14.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.628+0000 7ff299c00700 1 --2- 172.21.15.1:0/2169523852 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9fcddb930 0x55d9fcdfc840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:14.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.629+0000 7ff299c00700 1 -- 172.21.15.1:0/2169523852 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d9fcbae3b0 con 0x55d9fcddb010 2024-02-02T15:43:14.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.629+0000 7ff299c00700 1 --2- 172.21.15.1:0/2169523852 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d9fcddb010 0x55d9fcdfc300 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7ff28c0c8c00 tx=0x7ff28c0c8c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d5b0cfef3f994a8 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:14.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.629+0000 7ff283400700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:43:14.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.629+0000 7ff283400700 1 -- 172.21.15.1:0/2169523852 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff28c0a0a40 con 0x55d9fcddb010 2024-02-02T15:43:14.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.629+0000 7ff283400700 1 -- 172.21.15.1:0/2169523852 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9541+0+0 (secure 0 0 0) 0x7ff28c0c8d70 con 0x55d9fcddb010 2024-02-02T15:43:14.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.629+0000 7ff2b0125b80 1 -- 172.21.15.1:0/2169523852 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff27c003d70 con 0x55d9fcddb010 2024-02-02T15:43:14.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.630+0000 7ff283400700 1 -- 172.21.15.1:0/2169523852 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff28c0c8f30 con 0x55d9fcddb010 2024-02-02T15:43:14.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.630+0000 7ff2b0125b80 10 client.29885 fetch_fsmap learned FSMap version 67 2024-02-02T15:43:14.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.630+0000 7ff2b0125b80 1 -- 172.21.15.1:0/2169523852 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) v3 -- 0x55d9fcbb2cd0 con 0x55d9fcddb010 2024-02-02T15:43:14.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.630+0000 7ff283400700 1 -- 172.21.15.1:0/2169523852 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff27c003d70 con 0x55d9fcddb010 2024-02-02T15:43:14.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.630+0000 7ff283400700 1 -- 172.21.15.1:0/2169523852 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 67) v1 ==== 7506+0+0 (secure 0 0 0) 0x7ff28c098c30 con 0x55d9fcddb010 2024-02-02T15:43:14.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.631+0000 7ff2b0125b80 10 client.29885 fetch_fsmap finished waiting for FSMap version 67 2024-02-02T15:43:14.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.631+0000 7ff2b0125b80 10 client.29885 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:43:14.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.631+0000 7ff2b0125b80 20 client.29885 populate_metadata read hostname 'smithi001' 2024-02-02T15:43:14.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.631+0000 7ff2b0125b80 1 --2- 172.21.15.1:0/2169523852 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55d9fce6a5f0 0x55d9fce6ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:14.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.631+0000 7ff2b0125b80 4 client.29885 mds_command: new command op to 24557 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:43:14.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.631+0000 7ff2b0125b80 1 -- 172.21.15.1:0/2169523852 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d9fccebf50 con 0x55d9fce6a5f0 2024-02-02T15:43:14.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.632+0000 7ff299200700 1 --2- 172.21.15.1:0/2169523852 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55d9fce6a5f0 0x55d9fce6ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:14.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.633+0000 7ff299200700 1 --2- 172.21.15.1:0/2169523852 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55d9fce6a5f0 0x55d9fce6ca10 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e0c15f37ebaa56b server_cookie=56ed1ffe77ebdad4 in_seq=0 out_seq=0 2024-02-02T15:43:14.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.633+0000 7ff283400700 10 client.29885 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:43:14.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.633+0000 7ff283400700 1 -- 172.21.15.1:0/2169523852 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d9fccebf50 con 0x55d9fce6a5f0 2024-02-02T15:43:14.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.633+0000 7ff283400700 10 client.29885 handle_command_reply: tid=0 2024-02-02T15:43:14.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.633+0000 7ff2b0125b80 1 client.29885 shutdown 2024-02-02T15:43:14.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.634+0000 7ff2b0125b80 1 -- 172.21.15.1:0/2169523852 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d9fcddb010 msgr2=0x55d9fcdfc300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:14.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.634+0000 7ff2b0125b80 1 --2- 172.21.15.1:0/2169523852 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d9fcddb010 0x55d9fcdfc300 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7ff28c0c8c00 tx=0x7ff28c0c8c60 comp rx=0 tx=0).stop 2024-02-02T15:43:14.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.637+0000 7ff2b0125b80 1 -- 172.21.15.1:0/2169523852 shutdown_connections 2024-02-02T15:43:14.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.637+0000 7ff2b0125b80 1 --2- 172.21.15.1:0/2169523852 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55d9fce6a5f0 0x55d9fce6ca10 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:14.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.637+0000 7ff2b0125b80 1 --2- 172.21.15.1:0/2169523852 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d9fcde67e0 0x55d9fcdfcd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:14.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.637+0000 7ff2b0125b80 1 --2- 172.21.15.1:0/2169523852 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d9fcddb010 0x55d9fcdfc300 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:14.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.637+0000 7ff2b0125b80 1 --2- 172.21.15.1:0/2169523852 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d9fcddb930 0x55d9fcdfc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:14.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.637+0000 7ff2b0125b80 1 -- 172.21.15.1:0/2169523852 >> 172.21.15.1:0/2169523852 conn(0x55d9fcd3adb0 msgr2=0x55d9fce028f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:14.638 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:43:14.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.637+0000 7ff283400700 0 client.29885 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:43:14.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.637+0000 7ff2b0125b80 1 -- 172.21.15.1:0/2169523852 shutdown_connections 2024-02-02T15:43:14.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.637+0000 7ff2b0125b80 1 -- 172.21.15.1:0/2169523852 wait complete. 2024-02-02T15:43:14.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.638+0000 7ff2b0125b80 20 client.29885 trim_cache size 0 max 16384 2024-02-02T15:43:14.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.639+0000 7ff29c600700 1 -- 172.21.15.1:0/3889813129 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff2740a0860 msgr2=0x7ff2740a2d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:14.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.639+0000 7ff29c600700 1 --2- 172.21.15.1:0/3889813129 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff2740a0860 0x7ff2740a2d20 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff27c006010 tx=0x7ff27c00b540 comp rx=0 tx=0).stop 2024-02-02T15:43:14.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.639+0000 7ff29c600700 1 -- 172.21.15.1:0/3889813129 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff294107600 msgr2=0x7ff29419d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:14.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.639+0000 7ff29c600700 1 --2- 172.21.15.1:0/3889813129 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff294107600 0x7ff29419d4e0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7ff28c00ba50 tx=0x7ff28c00be10 comp rx=0 tx=0).stop 2024-02-02T15:43:14.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.639+0000 7ff29c600700 1 -- 172.21.15.1:0/3889813129 shutdown_connections 2024-02-02T15:43:14.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.639+0000 7ff29c600700 1 --2- 172.21.15.1:0/3889813129 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff2740a0860 0x7ff2740a2d20 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:14.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.639+0000 7ff29c600700 1 --2- 172.21.15.1:0/3889813129 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff2941a1e80 0x7ff2941a2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:14.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.639+0000 7ff29c600700 1 --2- 172.21.15.1:0/3889813129 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff294107600 0x7ff29419d4e0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:14.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.639+0000 7ff29c600700 1 --2- 172.21.15.1:0/3889813129 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff29410c720 0x7ff29419da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:14.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.639+0000 7ff29c600700 1 -- 172.21.15.1:0/3889813129 >> 172.21.15.1:0/3889813129 conn(0x7ff294076b80 msgr2=0x7ff294076f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:14.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.639+0000 7ff29c600700 1 -- 172.21.15.1:0/3889813129 shutdown_connections 2024-02-02T15:43:14.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:14.640+0000 7ff29c600700 1 -- 172.21.15.1:0/3889813129 wait complete. 2024-02-02T15:43:14.988 INFO:tasks.workunit.client.0.smithi001.stdout: 10 92755 26.33 MB/sec execute 47 sec latency 130.978 ms 2024-02-02T15:43:15.677 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-02T15:43:15.677 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-02T15:43:15.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:43:15 smithi071 ceph-mon[44707]: pgmap v509: 129 pgs: 129 active+clean; 1.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.5 MiB/s wr, 432 op/s 2024-02-02T15:43:15.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:43:15 smithi142 ceph-mon[45728]: pgmap v509: 129 pgs: 129 active+clean; 1.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.5 MiB/s wr, 432 op/s 2024-02-02T15:43:15.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:43:15 smithi001 ceph-mon[35470]: pgmap v509: 129 pgs: 129 active+clean; 1.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 3.5 MiB/s wr, 432 op/s 2024-02-02T15:43:15.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.972+0000 7fa279a00700 1 -- 172.21.15.1:0/3377974472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2740ffe20 msgr2=0x7fa274100200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:15.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.972+0000 7fa279a00700 1 --2- 172.21.15.1:0/3377974472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2740ffe20 0x7fa274100200 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fa264009990 tx=0x7fa264009ca0 comp rx=0 tx=0).stop 2024-02-02T15:43:15.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.972+0000 7fa279a00700 1 -- 172.21.15.1:0/3377974472 shutdown_connections 2024-02-02T15:43:15.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.972+0000 7fa279a00700 1 --2- 172.21.15.1:0/3377974472 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa274105220 0x7fa2741056d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:15.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.972+0000 7fa279a00700 1 --2- 172.21.15.1:0/3377974472 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2740ffe20 0x7fa274100200 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:15.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.972+0000 7fa279a00700 1 --2- 172.21.15.1:0/3377974472 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa274100740 0x7fa274104c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:15.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.972+0000 7fa279a00700 1 -- 172.21.15.1:0/3377974472 >> 172.21.15.1:0/3377974472 conn(0x7fa274076d70 msgr2=0x7fa274077180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:15.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.973+0000 7fa279a00700 1 -- 172.21.15.1:0/3377974472 shutdown_connections 2024-02-02T15:43:15.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.973+0000 7fa279a00700 1 -- 172.21.15.1:0/3377974472 wait complete. 2024-02-02T15:43:15.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.974+0000 7fa279a00700 1 Processor -- start 2024-02-02T15:43:15.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.974+0000 7fa279a00700 1 -- start start 2024-02-02T15:43:15.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.975+0000 7fa279a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2740ffe20 0x7fa27419d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:15.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.975+0000 7fa279a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa274100740 0x7fa27419db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:15.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.975+0000 7fa279a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa274105220 0x7fa2741a1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:15.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.975+0000 7fa279a00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2741a2570 con 0x7fa274105220 2024-02-02T15:43:15.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.975+0000 7fa279a00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2741a26e0 con 0x7fa274100740 2024-02-02T15:43:15.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.975+0000 7fa279a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2741a28a0 con 0x7fa2740ffe20 2024-02-02T15:43:15.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.975+0000 7fa272a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2740ffe20 0x7fa27419d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:15.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.975+0000 7fa272000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa274100740 0x7fa27419db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:15.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.975+0000 7fa272a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2740ffe20 0x7fa27419d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:48978/0 (socket says 172.21.15.1:48978) 2024-02-02T15:43:15.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.976+0000 7fa272a00700 1 -- 172.21.15.1:0/461480025 learned_addr learned my addr 172.21.15.1:0/461480025 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:43:15.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.976+0000 7fa273400700 1 --2- 172.21.15.1:0/461480025 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa274105220 0x7fa2741a1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:15.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.976+0000 7fa272000700 1 -- 172.21.15.1:0/461480025 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2740ffe20 msgr2=0x7fa27419d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:15.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.976+0000 7fa272000700 1 --2- 172.21.15.1:0/461480025 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2740ffe20 0x7fa27419d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:15.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.976+0000 7fa272000700 1 -- 172.21.15.1:0/461480025 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa274105220 msgr2=0x7fa2741a1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:15.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.976+0000 7fa272000700 1 --2- 172.21.15.1:0/461480025 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa274105220 0x7fa2741a1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:15.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.976+0000 7fa272000700 1 -- 172.21.15.1:0/461480025 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2640095f0 con 0x7fa274100740 2024-02-02T15:43:15.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.976+0000 7fa273400700 1 --2- 172.21.15.1:0/461480025 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa274105220 0x7fa2741a1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:43:15.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.976+0000 7fa272000700 1 --2- 172.21.15.1:0/461480025 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa274100740 0x7fa27419db20 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fa25c00b450 tx=0x7fa25c00b760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:15.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.977+0000 7fa26b400700 1 -- 172.21.15.1:0/461480025 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa25c011830 con 0x7fa274100740 2024-02-02T15:43:15.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.977+0000 7fa26b400700 1 -- 172.21.15.1:0/461480025 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa25c011e50 con 0x7fa274100740 2024-02-02T15:43:15.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.977+0000 7fa26b400700 1 -- 172.21.15.1:0/461480025 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa25c010de0 con 0x7fa274100740 2024-02-02T15:43:15.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.977+0000 7fa279a00700 1 -- 172.21.15.1:0/461480025 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2741a2b80 con 0x7fa274100740 2024-02-02T15:43:15.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.977+0000 7fa279a00700 1 -- 172.21.15.1:0/461480025 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2741aa3c0 con 0x7fa274100740 2024-02-02T15:43:15.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.982+0000 7fa26b400700 1 -- 172.21.15.1:0/461480025 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7fa25c010480 con 0x7fa274100740 2024-02-02T15:43:15.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.985+0000 7fa26b400700 1 --2- 172.21.15.1:0/461480025 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fa2540a0750 0x7fa2540a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:15.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.985+0000 7fa26b400700 1 -- 172.21.15.1:0/461480025 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9541+0+0 (secure 0 0 0) 0x7fa25c0cbaf0 con 0x7fa274100740 2024-02-02T15:43:15.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.986+0000 7fa272a00700 1 --2- 172.21.15.1:0/461480025 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fa2540a0750 0x7fa2540a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:15.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.986+0000 7fa28d5e4b80 1 Processor -- start 2024-02-02T15:43:15.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.986+0000 7fa272a00700 1 --2- 172.21.15.1:0/461480025 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fa2540a0750 0x7fa2540a2c10 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fa264003760 tx=0x7fa2640058e0 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:15.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.987+0000 7fa28d5e4b80 1 -- start start 2024-02-02T15:43:15.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.987+0000 7fa28d5e4b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5585580cc090 0x5585580cc470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:15.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.987+0000 7fa28d5e4b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5585580cc9b0 0x5585580d7320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:15.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.987+0000 7fa28d5e4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585580d7860 0x5585580d9c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:15.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.987+0000 7fa28d5e4b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x558558040540 con 0x5585580cc9b0 2024-02-02T15:43:15.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.987+0000 7fa28d5e4b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x558558024b20 con 0x5585580cc090 2024-02-02T15:43:15.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.987+0000 7fa28d5e4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585580224e0 con 0x5585580d7860 2024-02-02T15:43:15.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.988+0000 7fa273400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585580d7860 0x5585580d9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:15.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.988+0000 7fa272a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5585580cc090 0x5585580cc470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:15.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.988+0000 7fa273400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585580d7860 0x5585580d9c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:48992/0 (socket says 172.21.15.1:48992) 2024-02-02T15:43:15.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.988+0000 7fa273400700 1 -- 172.21.15.1:0/1702311532 learned_addr learned my addr 172.21.15.1:0/1702311532 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:43:15.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.988+0000 7fa272000700 1 --2- 172.21.15.1:0/1702311532 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5585580cc9b0 0x5585580d7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:15.989 INFO:tasks.workunit.client.0.smithi001.stdout: 10 93473 26.66 MB/sec execute 48 sec latency 42.187 ms 2024-02-02T15:43:15.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.988+0000 7fa273400700 1 -- 172.21.15.1:0/1702311532 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5585580cc090 msgr2=0x5585580cc470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:15.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.988+0000 7fa273400700 1 --2- 172.21.15.1:0/1702311532 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5585580cc090 0x5585580cc470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:15.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.988+0000 7fa273400700 1 -- 172.21.15.1:0/1702311532 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5585580cc9b0 msgr2=0x5585580d7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:15.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.988+0000 7fa273400700 1 --2- 172.21.15.1:0/1702311532 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5585580cc9b0 0x5585580d7320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:15.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.988+0000 7fa273400700 1 -- 172.21.15.1:0/1702311532 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558557e9f430 con 0x5585580d7860 2024-02-02T15:43:15.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.989+0000 7fa273400700 1 --2- 172.21.15.1:0/1702311532 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585580d7860 0x5585580d9c50 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fa26c00c450 tx=0x7fa26c00c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4403a57f06bf2013 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:15.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.989+0000 7fa263e00700 1 -- 172.21.15.1:0/1702311532 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa26c011070 con 0x5585580d7860 2024-02-02T15:43:15.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.989+0000 7fa263e00700 1 -- 172.21.15.1:0/1702311532 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa26c009b10 con 0x5585580d7860 2024-02-02T15:43:15.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.989+0000 7fa263e00700 1 -- 172.21.15.1:0/1702311532 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa26c0153c0 con 0x5585580d7860 2024-02-02T15:43:15.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.990+0000 7fa28d5e4b80 1 -- 172.21.15.1:0/1702311532 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585580d7860 msgr2=0x5585580d9c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:15.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.990+0000 7fa28d5e4b80 1 --2- 172.21.15.1:0/1702311532 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585580d7860 0x5585580d9c50 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fa26c00c450 tx=0x7fa26c00c480 comp rx=0 tx=0).stop 2024-02-02T15:43:15.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.990+0000 7fa28d5e4b80 1 -- 172.21.15.1:0/1702311532 shutdown_connections 2024-02-02T15:43:15.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.990+0000 7fa28d5e4b80 1 --2- 172.21.15.1:0/1702311532 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585580d7860 0x5585580d9c50 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:15.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.990+0000 7fa28d5e4b80 1 --2- 172.21.15.1:0/1702311532 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5585580cc090 0x5585580cc470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:15.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.990+0000 7fa28d5e4b80 1 --2- 172.21.15.1:0/1702311532 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5585580cc9b0 0x5585580d7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:15.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.990+0000 7fa28d5e4b80 1 -- 172.21.15.1:0/1702311532 >> 172.21.15.1:0/1702311532 conn(0x55855802be30 msgr2=0x5585580c8430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:15.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.991+0000 7fa28d5e4b80 1 -- 172.21.15.1:0/1702311532 shutdown_connections 2024-02-02T15:43:15.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.991+0000 7fa28d5e4b80 1 -- 172.21.15.1:0/1702311532 wait complete. 2024-02-02T15:43:15.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.991+0000 7fa28d5e4b80 1 -- start start 2024-02-02T15:43:15.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.992+0000 7fa28d5e4b80 1 Processor -- start 2024-02-02T15:43:15.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.992+0000 7fa28d5e4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585580cc090 0x5585580d7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:15.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.992+0000 7fa28d5e4b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5585580cc9b0 0x5585580d4520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:15.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.992+0000 7fa28d5e4b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5585580d7860 0x5585580d4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:15.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.992+0000 7fa28d5e4b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55855801ddd0 con 0x5585580d7860 2024-02-02T15:43:15.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.992+0000 7fa28d5e4b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x558557fb9e00 con 0x5585580cc9b0 2024-02-02T15:43:15.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.993+0000 7fa28d5e4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558557fb9aa0 con 0x5585580cc090 2024-02-02T15:43:15.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.993+0000 7fa28d5e4b80 10 client.0 _ll_register_callbacks cb 0x558557fbb310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:43:15.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.993+0000 7fa272a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585580cc090 0x5585580d7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:15.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.993+0000 7fa272a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585580cc090 0x5585580d7310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:49006/0 (socket says 172.21.15.1:49006) 2024-02-02T15:43:15.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.993+0000 7fa272a00700 1 -- 172.21.15.1:0/3845810633 learned_addr learned my addr 172.21.15.1:0/3845810633 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:43:15.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.994+0000 7fa273400700 1 --2- 172.21.15.1:0/3845810633 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5585580d7860 0x5585580d4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:15.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.994+0000 7fa272a00700 1 -- 172.21.15.1:0/3845810633 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5585580cc9b0 msgr2=0x5585580d4520 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-02T15:43:15.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.994+0000 7fa272a00700 1 --2- 172.21.15.1:0/3845810633 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5585580cc9b0 0x5585580d4520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:15.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.994+0000 7fa272a00700 1 -- 172.21.15.1:0/3845810633 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5585580d7860 msgr2=0x5585580d4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:15.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.994+0000 7fa272a00700 1 --2- 172.21.15.1:0/3845810633 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5585580d7860 0x5585580d4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:15.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.994+0000 7fa272a00700 1 -- 172.21.15.1:0/3845810633 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa26c009c90 con 0x5585580cc090 2024-02-02T15:43:15.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.997+0000 7fa260c00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:43:15.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.997+0000 7fa272a00700 1 --2- 172.21.15.1:0/3845810633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585580cc090 0x5585580d7310 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fa264005f50 tx=0x7fa264009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cfc49694a7e4ea08 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:15.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.997+0000 7fa260c00700 1 -- 172.21.15.1:0/3845810633 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa264023870 con 0x5585580cc090 2024-02-02T15:43:15.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.997+0000 7fa260c00700 1 -- 172.21.15.1:0/3845810633 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9541+0+0 (secure 0 0 0) 0x7fa2640239d0 con 0x5585580cc090 2024-02-02T15:43:15.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.997+0000 7fa28d5e4b80 1 -- 172.21.15.1:0/3845810633 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558558016e50 con 0x5585580cc090 2024-02-02T15:43:15.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.998+0000 7fa260c00700 1 -- 172.21.15.1:0/3845810633 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa264023870 con 0x5585580cc090 2024-02-02T15:43:15.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.998+0000 7fa28d5e4b80 10 client.29812 fetch_fsmap learned FSMap version 67 2024-02-02T15:43:15.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.998+0000 7fa28d5e4b80 1 -- 172.21.15.1:0/3845810633 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) v3 -- 0x558557e4bbe0 con 0x5585580cc090 2024-02-02T15:43:15.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:15.999+0000 7fa260c00700 1 -- 172.21.15.1:0/3845810633 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558558016e50 con 0x5585580cc090 2024-02-02T15:43:16.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.000+0000 7fa260c00700 1 -- 172.21.15.1:0/3845810633 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 67) v1 ==== 7506+0+0 (secure 0 0 0) 0x7fa26400f460 con 0x5585580cc090 2024-02-02T15:43:16.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.000+0000 7fa28d5e4b80 10 client.29812 fetch_fsmap finished waiting for FSMap version 67 2024-02-02T15:43:16.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.000+0000 7fa28d5e4b80 10 client.29812 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:43:16.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.000+0000 7fa28d5e4b80 20 client.29812 populate_metadata read hostname 'smithi001' 2024-02-02T15:43:16.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.000+0000 7fa28d5e4b80 1 --2- 172.21.15.1:0/3845810633 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5585580f8300 0x5585580fa720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:16.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.000+0000 7fa28d5e4b80 4 client.29812 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:43:16.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.000+0000 7fa28d5e4b80 1 -- 172.21.15.1:0/3845810633 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558557fdcfd0 con 0x5585580f8300 2024-02-02T15:43:16.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.001+0000 7fa272000700 1 --2- 172.21.15.1:0/3845810633 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5585580f8300 0x5585580fa720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:16.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.001+0000 7fa272000700 1 --2- 172.21.15.1:0/3845810633 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5585580f8300 0x5585580fa720 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b51d14ee3c20df69 server_cookie=928a8ace41162d0d in_seq=0 out_seq=0 2024-02-02T15:43:16.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.001+0000 7fa260c00700 10 client.29812 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:43:16.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.002+0000 7fa260c00700 1 -- 172.21.15.1:0/3845810633 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x558557fdcfd0 con 0x5585580f8300 2024-02-02T15:43:16.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.002+0000 7fa260c00700 10 client.29812 handle_command_reply: tid=0 2024-02-02T15:43:16.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.002+0000 7fa28d5e4b80 1 client.29812 shutdown 2024-02-02T15:43:16.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.002+0000 7fa28d5e4b80 1 -- 172.21.15.1:0/3845810633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585580cc090 msgr2=0x5585580d7310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:16.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.002+0000 7fa28d5e4b80 1 --2- 172.21.15.1:0/3845810633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585580cc090 0x5585580d7310 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fa264005f50 tx=0x7fa264009310 comp rx=0 tx=0).stop 2024-02-02T15:43:16.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.003+0000 7fa28d5e4b80 1 -- 172.21.15.1:0/3845810633 shutdown_connections 2024-02-02T15:43:16.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.003+0000 7fa28d5e4b80 1 --2- 172.21.15.1:0/3845810633 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5585580f8300 0x5585580fa720 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:16.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.003+0000 7fa260c00700 0 client.29812 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:43:16.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.003+0000 7fa28d5e4b80 1 --2- 172.21.15.1:0/3845810633 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5585580d7860 0x5585580d4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:16.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.003+0000 7fa28d5e4b80 1 --2- 172.21.15.1:0/3845810633 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585580cc090 0x5585580d7310 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:16.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.003+0000 7fa28d5e4b80 1 --2- 172.21.15.1:0/3845810633 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5585580cc9b0 0x5585580d4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:16.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.003+0000 7fa28d5e4b80 1 -- 172.21.15.1:0/3845810633 >> 172.21.15.1:0/3845810633 conn(0x55855802be30 msgr2=0x5585580c8020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:16.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.003+0000 7fa28d5e4b80 1 -- 172.21.15.1:0/3845810633 shutdown_connections 2024-02-02T15:43:16.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.003+0000 7fa28d5e4b80 1 -- 172.21.15.1:0/3845810633 wait complete. 2024-02-02T15:43:16.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.003+0000 7fa28d5e4b80 20 client.29812 trim_cache size 0 max 16384 2024-02-02T15:43:16.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.025+0000 7fa28d5e4b80 1 Processor -- start 2024-02-02T15:43:16.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.025+0000 7fa28d5e4b80 1 -- start start 2024-02-02T15:43:16.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.025+0000 7fa28d5e4b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5585580cc090 0x5585580de3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:16.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.025+0000 7fa28d5e4b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5585580cc9b0 0x5585580d6e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:16.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.025+0000 7fa28d5e4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585580d7860 0x5585580cf960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:16.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.025+0000 7fa28d5e4b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x558557ea1720 con 0x5585580cc090 2024-02-02T15:43:16.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.025+0000 7fa28d5e4b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585580c4a30 con 0x5585580cc9b0 2024-02-02T15:43:16.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.025+0000 7fa28d5e4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585580f4cb0 con 0x5585580d7860 2024-02-02T15:43:16.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.026+0000 7fa272000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5585580cc9b0 0x5585580d6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:16.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.026+0000 7fa272000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5585580cc9b0 0x5585580d6e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:35626/0 (socket says 172.21.15.1:35626) 2024-02-02T15:43:16.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.026+0000 7fa272000700 1 -- 172.21.15.1:0/3015511815 learned_addr learned my addr 172.21.15.1:0/3015511815 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:43:16.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.026+0000 7fa273400700 1 --2- 172.21.15.1:0/3015511815 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585580d7860 0x5585580cf960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:16.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.026+0000 7fa272a00700 1 --2- 172.21.15.1:0/3015511815 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5585580cc090 0x5585580de3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:16.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.026+0000 7fa272000700 1 -- 172.21.15.1:0/3015511815 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585580d7860 msgr2=0x5585580cf960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:16.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.026+0000 7fa272000700 1 --2- 172.21.15.1:0/3015511815 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585580d7860 0x5585580cf960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:16.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.026+0000 7fa272000700 1 -- 172.21.15.1:0/3015511815 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5585580cc090 msgr2=0x5585580de3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:16.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.026+0000 7fa272000700 1 --2- 172.21.15.1:0/3015511815 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5585580cc090 0x5585580de3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:16.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.026+0000 7fa272000700 1 -- 172.21.15.1:0/3015511815 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5585580c4150 con 0x5585580cc9b0 2024-02-02T15:43:16.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.026+0000 7fa273400700 1 --2- 172.21.15.1:0/3015511815 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585580d7860 0x5585580cf960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:43:16.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.027+0000 7fa272000700 1 --2- 172.21.15.1:0/3015511815 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5585580cc9b0 0x5585580d6e40 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fa25c009fd0 tx=0x7fa25c010880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6de5edf36d3c1bdc server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:16.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.028+0000 7fa268c00700 1 -- 172.21.15.1:0/3015511815 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa25c0b0070 con 0x5585580cc9b0 2024-02-02T15:43:16.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.028+0000 7fa268c00700 1 -- 172.21.15.1:0/3015511815 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa25c0b1270 con 0x5585580cc9b0 2024-02-02T15:43:16.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.028+0000 7fa268c00700 1 -- 172.21.15.1:0/3015511815 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa25c0b6ea0 con 0x5585580cc9b0 2024-02-02T15:43:16.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.029+0000 7fa28d5e4b80 1 -- 172.21.15.1:0/3015511815 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5585580cc9b0 msgr2=0x5585580d6e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:16.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.029+0000 7fa28d5e4b80 1 --2- 172.21.15.1:0/3015511815 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5585580cc9b0 0x5585580d6e40 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fa25c009fd0 tx=0x7fa25c010880 comp rx=0 tx=0).stop 2024-02-02T15:43:16.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.029+0000 7fa28d5e4b80 1 -- 172.21.15.1:0/3015511815 shutdown_connections 2024-02-02T15:43:16.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.029+0000 7fa28d5e4b80 1 --2- 172.21.15.1:0/3015511815 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585580d7860 0x5585580cf960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:16.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.029+0000 7fa28d5e4b80 1 --2- 172.21.15.1:0/3015511815 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5585580cc090 0x5585580de3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:16.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.029+0000 7fa28d5e4b80 1 --2- 172.21.15.1:0/3015511815 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5585580cc9b0 0x5585580d6e40 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:16.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.029+0000 7fa28d5e4b80 1 -- 172.21.15.1:0/3015511815 >> 172.21.15.1:0/3015511815 conn(0x55855802be30 msgr2=0x5585580f45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:16.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.029+0000 7fa28d5e4b80 1 -- 172.21.15.1:0/3015511815 shutdown_connections 2024-02-02T15:43:16.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.029+0000 7fa28d5e4b80 1 -- 172.21.15.1:0/3015511815 wait complete. 2024-02-02T15:43:16.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.030+0000 7fa28d5e4b80 1 -- start start 2024-02-02T15:43:16.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.030+0000 7fa28d5e4b80 1 Processor -- start 2024-02-02T15:43:16.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.031+0000 7fa28d5e4b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585580cc090 0x5585580dc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:16.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.031+0000 7fa28d5e4b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5585580cc9b0 0x5585580dcef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:16.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.031+0000 7fa28d5e4b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5585580d7860 0x5585580dd450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:16.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.031+0000 7fa28d5e4b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x558558149fa0 con 0x5585580cc9b0 2024-02-02T15:43:16.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.031+0000 7fa28d5e4b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55855814a110 con 0x5585580d7860 2024-02-02T15:43:16.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.031+0000 7fa28d5e4b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55855814a2d0 con 0x5585580cc090 2024-02-02T15:43:16.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.031+0000 7fa28d5e4b80 10 client.0 _ll_register_callbacks cb 0x558557fbb310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:43:16.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.031+0000 7fa272000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5585580cc9b0 0x5585580dcef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:16.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.031+0000 7fa272000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5585580cc9b0 0x5585580dcef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:46534/0 (socket says 172.21.15.1:46534) 2024-02-02T15:43:16.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.031+0000 7fa272000700 1 -- 172.21.15.1:0/476579454 learned_addr learned my addr 172.21.15.1:0/476579454 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:43:16.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.031+0000 7fa273400700 1 --2- 172.21.15.1:0/476579454 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5585580d7860 0x5585580dd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:16.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.032+0000 7fa272000700 1 -- 172.21.15.1:0/476579454 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585580cc090 msgr2=0x5585580dc9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:16.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.032+0000 7fa272000700 1 --2- 172.21.15.1:0/476579454 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585580cc090 0x5585580dc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:16.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.032+0000 7fa272000700 1 -- 172.21.15.1:0/476579454 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5585580d7860 msgr2=0x5585580dd450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:16.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.032+0000 7fa272000700 1 --2- 172.21.15.1:0/476579454 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5585580d7860 0x5585580dd450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:16.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.032+0000 7fa272000700 1 -- 172.21.15.1:0/476579454 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558557e9f430 con 0x5585580cc9b0 2024-02-02T15:43:16.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.032+0000 7fa272000700 1 --2- 172.21.15.1:0/476579454 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5585580cc9b0 0x5585580dcef0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fa25c0a0800 tx=0x7fa25c00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c1992daf58b4540 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:16.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.032+0000 7fa260c00700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:43:16.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.032+0000 7fa260c00700 1 -- 172.21.15.1:0/476579454 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa25c0b0050 con 0x5585580cc9b0 2024-02-02T15:43:16.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.032+0000 7fa260c00700 1 -- 172.21.15.1:0/476579454 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9541+0+0 (secure 0 0 0) 0x7fa25c09caf0 con 0x5585580cc9b0 2024-02-02T15:43:16.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.032+0000 7fa28d5e4b80 1 -- 172.21.15.1:0/476579454 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5585580c0170 con 0x5585580cc9b0 2024-02-02T15:43:16.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.035+0000 7fa260c00700 1 -- 172.21.15.1:0/476579454 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa25c098cc0 con 0x5585580cc9b0 2024-02-02T15:43:16.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.035+0000 7fa28d5e4b80 10 client.19677 fetch_fsmap learned FSMap version 67 2024-02-02T15:43:16.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.035+0000 7fa28d5e4b80 1 -- 172.21.15.1:0/476579454 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) v3 -- 0x7fa2741aa3c0 con 0x5585580cc9b0 2024-02-02T15:43:16.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.036+0000 7fa260c00700 1 -- 172.21.15.1:0/476579454 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5585580c0170 con 0x5585580cc9b0 2024-02-02T15:43:16.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.036+0000 7fa260c00700 1 -- 172.21.15.1:0/476579454 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 67) v1 ==== 7506+0+0 (secure 0 0 0) 0x7fa25c0a08a0 con 0x5585580cc9b0 2024-02-02T15:43:16.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.036+0000 7fa28d5e4b80 10 client.19677 fetch_fsmap finished waiting for FSMap version 67 2024-02-02T15:43:16.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.036+0000 7fa28d5e4b80 10 client.19677 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:43:16.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.036+0000 7fa28d5e4b80 20 client.19677 populate_metadata read hostname 'smithi001' 2024-02-02T15:43:16.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.036+0000 7fa28d5e4b80 1 --2- 172.21.15.1:0/476579454 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55855814b810 0x55855814dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:16.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.036+0000 7fa28d5e4b80 4 client.19677 mds_command: new command op to 24557 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-02T15:43:16.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.036+0000 7fa28d5e4b80 1 -- 172.21.15.1:0/476579454 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x558557e6cd70 con 0x55855814b810 2024-02-02T15:43:16.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.037+0000 7fa272a00700 1 --2- 172.21.15.1:0/476579454 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55855814b810 0x55855814dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:16.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.037+0000 7fa260c00700 10 client.19677 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:43:16.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.037+0000 7fa272a00700 1 --2- 172.21.15.1:0/476579454 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55855814b810 0x55855814dc30 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d18ed78eab9ba222 server_cookie=778cf411afa58614 in_seq=0 out_seq=0 2024-02-02T15:43:16.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.038+0000 7fa260c00700 1 -- 172.21.15.1:0/476579454 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x558557e6cd70 con 0x55855814b810 2024-02-02T15:43:16.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.038+0000 7fa260c00700 10 client.19677 handle_command_reply: tid=0 2024-02-02T15:43:16.038 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.038+0000 7fa28d5e4b80 1 client.19677 shutdown 2024-02-02T15:43:16.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.039+0000 7fa28d5e4b80 1 -- 172.21.15.1:0/476579454 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5585580cc9b0 msgr2=0x5585580dcef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:16.039 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.039+0000 7fa28d5e4b80 1 --2- 172.21.15.1:0/476579454 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5585580cc9b0 0x5585580dcef0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fa25c0a0800 tx=0x7fa25c00bf10 comp rx=0 tx=0).stop 2024-02-02T15:43:16.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.039+0000 7fa28d5e4b80 1 -- 172.21.15.1:0/476579454 shutdown_connections 2024-02-02T15:43:16.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.039+0000 7fa28d5e4b80 1 --2- 172.21.15.1:0/476579454 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55855814b810 0x55855814dc30 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:16.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.039+0000 7fa28d5e4b80 1 --2- 172.21.15.1:0/476579454 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5585580d7860 0x5585580dd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:16.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.039+0000 7fa28d5e4b80 1 --2- 172.21.15.1:0/476579454 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5585580cc090 0x5585580dc9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:16.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.039+0000 7fa28d5e4b80 1 --2- 172.21.15.1:0/476579454 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5585580cc9b0 0x5585580dcef0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:16.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.039+0000 7fa28d5e4b80 1 -- 172.21.15.1:0/476579454 >> 172.21.15.1:0/476579454 conn(0x55855802be30 msgr2=0x5585580cdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:16.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.039+0000 7fa260c00700 0 client.19677 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:43:16.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.039+0000 7fa28d5e4b80 1 -- 172.21.15.1:0/476579454 shutdown_connections 2024-02-02T15:43:16.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.040+0000 7fa28d5e4b80 1 -- 172.21.15.1:0/476579454 wait complete. 2024-02-02T15:43:16.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.040+0000 7fa28d5e4b80 20 client.19677 trim_cache size 0 max 16384 2024-02-02T15:43:16.041 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:43:16.041 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2024-02-02T15:43:16.041 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "c09e3d0e-7171-4ece-a97b-02b8fda15974", 2024-02-02T15:43:16.041 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2024-02-02T15:43:16.041 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:43:16.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.041+0000 7fa279a00700 1 -- 172.21.15.1:0/461480025 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fa2540a0750 msgr2=0x7fa2540a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:16.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.041+0000 7fa279a00700 1 --2- 172.21.15.1:0/461480025 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fa2540a0750 0x7fa2540a2c10 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fa264003760 tx=0x7fa2640058e0 comp rx=0 tx=0).stop 2024-02-02T15:43:16.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.041+0000 7fa279a00700 1 -- 172.21.15.1:0/461480025 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa274100740 msgr2=0x7fa27419db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:16.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.041+0000 7fa279a00700 1 --2- 172.21.15.1:0/461480025 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa274100740 0x7fa27419db20 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fa25c00b450 tx=0x7fa25c00b760 comp rx=0 tx=0).stop 2024-02-02T15:43:16.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.041+0000 7fa279a00700 1 -- 172.21.15.1:0/461480025 shutdown_connections 2024-02-02T15:43:16.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.041+0000 7fa279a00700 1 --2- 172.21.15.1:0/461480025 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fa2540a0750 0x7fa2540a2c10 secure :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fa264003760 tx=0x7fa2640058e0 comp rx=0 tx=0).stop 2024-02-02T15:43:16.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.041+0000 7fa279a00700 1 --2- 172.21.15.1:0/461480025 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa274105220 0x7fa2741a1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:16.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.041+0000 7fa279a00700 1 --2- 172.21.15.1:0/461480025 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa2740ffe20 0x7fa27419d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:16.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.041+0000 7fa279a00700 1 --2- 172.21.15.1:0/461480025 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa274100740 0x7fa27419db20 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:16.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.041+0000 7fa279a00700 1 -- 172.21.15.1:0/461480025 >> 172.21.15.1:0/461480025 conn(0x7fa274076d70 msgr2=0x7fa274102bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:16.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.042+0000 7fa279a00700 1 -- 172.21.15.1:0/461480025 shutdown_connections 2024-02-02T15:43:16.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.042+0000 7fa279a00700 1 -- 172.21.15.1:0/461480025 wait complete. 2024-02-02T15:43:16.066 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-02T15:43:16.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.380+0000 7f8c0be00700 1 -- 172.21.15.1:0/4239753514 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c04106120 msgr2=0x7f8c0410dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:16.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.380+0000 7f8c0be00700 1 --2- 172.21.15.1:0/4239753514 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c04106120 0x7f8c0410dca0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc00f530 tx=0x7f8bfc00f840 comp rx=0 tx=0).stop 2024-02-02T15:43:16.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.381+0000 7f8c0be00700 1 -- 172.21.15.1:0/4239753514 shutdown_connections 2024-02-02T15:43:16.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.381+0000 7f8c0be00700 1 --2- 172.21.15.1:0/4239753514 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c04106120 0x7f8c0410dca0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:16.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.381+0000 7f8c0be00700 1 --2- 172.21.15.1:0/4239753514 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8c04101000 0x7f8c041013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:16.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.381+0000 7f8c0be00700 1 --2- 172.21.15.1:0/4239753514 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8c041019b0 0x7f8c041059f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:16.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.381+0000 7f8c0be00700 1 -- 172.21.15.1:0/4239753514 >> 172.21.15.1:0/4239753514 conn(0x7f8c040fc840 msgr2=0x7f8c040fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:16.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.381+0000 7f8c0be00700 1 -- 172.21.15.1:0/4239753514 shutdown_connections 2024-02-02T15:43:16.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.381+0000 7f8c0be00700 1 -- 172.21.15.1:0/4239753514 wait complete. 2024-02-02T15:43:16.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.382+0000 7f8c0be00700 1 Processor -- start 2024-02-02T15:43:16.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.382+0000 7f8c0be00700 1 -- start start 2024-02-02T15:43:16.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.383+0000 7f8c0be00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8c04101000 0x7f8c04194e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:16.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.383+0000 7f8c0be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c04106120 0x7f8c04195350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:16.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.383+0000 7f8c0be00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8c041997b0 0x7f8c04199c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:16.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.383+0000 7f8c0be00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0419a230 con 0x7f8c041997b0 2024-02-02T15:43:16.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.383+0000 7f8c0be00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0419a3a0 con 0x7f8c04101000 2024-02-02T15:43:16.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.383+0000 7f8c0be00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0419a560 con 0x7f8c04106120 2024-02-02T15:43:16.384 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.383+0000 7f8c09400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8c04101000 0x7f8c04194e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:16.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.383+0000 7f8c09400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8c04101000 0x7f8c04194e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:35640/0 (socket says 172.21.15.1:35640) 2024-02-02T15:43:16.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.383+0000 7f8c09400700 1 -- 172.21.15.1:0/1994264442 learned_addr learned my addr 172.21.15.1:0/1994264442 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:43:16.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.383+0000 7f8c08a00700 1 --2- 172.21.15.1:0/1994264442 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c04106120 0x7f8c04195350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:16.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.384+0000 7f8c09e00700 1 --2- 172.21.15.1:0/1994264442 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8c041997b0 0x7f8c04199c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:16.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.384+0000 7f8c09400700 1 -- 172.21.15.1:0/1994264442 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c04106120 msgr2=0x7f8c04195350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:16.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.384+0000 7f8c09400700 1 --2- 172.21.15.1:0/1994264442 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c04106120 0x7f8c04195350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:16.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.384+0000 7f8c09400700 1 -- 172.21.15.1:0/1994264442 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8c041997b0 msgr2=0x7f8c04199c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:16.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.384+0000 7f8c09400700 1 --2- 172.21.15.1:0/1994264442 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8c041997b0 0x7f8c04199c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:16.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.384+0000 7f8c09400700 1 -- 172.21.15.1:0/1994264442 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bf4009520 con 0x7f8c04101000 2024-02-02T15:43:16.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.384+0000 7f8c09400700 1 --2- 172.21.15.1:0/1994264442 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8c04101000 0x7f8c04194e10 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f8bf400eb30 tx=0x7f8bf400ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:16.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.384+0000 7f8c02000700 1 -- 172.21.15.1:0/1994264442 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf4009880 con 0x7f8c04101000 2024-02-02T15:43:16.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.385+0000 7f8c02000700 1 -- 172.21.15.1:0/1994264442 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8bf40044e0 con 0x7f8c04101000 2024-02-02T15:43:16.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.385+0000 7f8c02000700 1 -- 172.21.15.1:0/1994264442 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf4003e90 con 0x7f8c04101000 2024-02-02T15:43:16.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.385+0000 7f8c0be00700 1 -- 172.21.15.1:0/1994264442 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bfc009d00 con 0x7f8c04101000 2024-02-02T15:43:16.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.385+0000 7f8c0be00700 1 -- 172.21.15.1:0/1994264442 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c041a1dc0 con 0x7f8c04101000 2024-02-02T15:43:16.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.389+0000 7f8c02000700 1 -- 172.21.15.1:0/1994264442 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f8bf4004000 con 0x7f8c04101000 2024-02-02T15:43:16.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.392+0000 7f8c1fa1fb80 1 Processor -- start 2024-02-02T15:43:16.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.393+0000 7f8c02000700 1 --2- 172.21.15.1:0/1994264442 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f8be40a0790 0x7f8be40a2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:16.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.393+0000 7f8c02000700 1 -- 172.21.15.1:0/1994264442 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f8bf4014070 con 0x7f8c04101000 2024-02-02T15:43:16.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.394+0000 7f8c09e00700 1 --2- 172.21.15.1:0/1994264442 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f8be40a0790 0x7f8be40a2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:16.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.394+0000 7f8c1fa1fb80 1 -- start start 2024-02-02T15:43:16.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.395+0000 7f8c1fa1fb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558a141a010 0x5558a141a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:16.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.395+0000 7f8c1fa1fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5558a141a930 0x5558a14252a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:16.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.395+0000 7f8c1fa1fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558a14257e0 0x5558a1427bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:16.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.395+0000 7f8c1fa1fb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558a138e4c0 con 0x5558a141a010 2024-02-02T15:43:16.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.395+0000 7f8c1fa1fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558a1372aa0 con 0x5558a141a930 2024-02-02T15:43:16.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.395+0000 7f8c1fa1fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558a1370460 con 0x5558a14257e0 2024-02-02T15:43:16.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.395+0000 7f8c08a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5558a141a930 0x5558a14252a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:16.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.395+0000 7f8c08a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5558a141a930 0x5558a14252a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:35650/0 (socket says 172.21.15.1:35650) 2024-02-02T15:43:16.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.395+0000 7f8c08a00700 1 -- 172.21.15.1:0/1465855773 learned_addr learned my addr 172.21.15.1:0/1465855773 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:43:16.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.395+0000 7f8c09400700 1 --2- 172.21.15.1:0/1465855773 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558a141a010 0x5558a141a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:16.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.396+0000 7f8c08a00700 1 -- 172.21.15.1:0/1465855773 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558a14257e0 msgr2=0x5558a1427bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:16.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.398+0000 7f8c09e00700 1 --2- 172.21.15.1:0/1465855773 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558a14257e0 0x5558a1427bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:16.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.398+0000 7f8c08a00700 1 --2- 172.21.15.1:0/1465855773 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558a14257e0 0x5558a1427bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:16.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.398+0000 7f8c08a00700 1 -- 172.21.15.1:0/1465855773 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558a141a010 msgr2=0x5558a141a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:16.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.398+0000 7f8c08a00700 1 --2- 172.21.15.1:0/1465855773 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558a141a010 0x5558a141a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:16.398 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.398+0000 7f8c08a00700 1 -- 172.21.15.1:0/1465855773 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5558a11ed3b0 con 0x5558a141a930 2024-02-02T15:43:16.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.399+0000 7f8c09400700 1 --2- 172.21.15.1:0/1465855773 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558a141a010 0x5558a141a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:43:16.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.399+0000 7f8c09e00700 1 --2- 172.21.15.1:0/1465855773 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558a14257e0 0x5558a1427bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:43:16.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.400+0000 7f8c08a00700 1 --2- 172.21.15.1:0/1465855773 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5558a141a930 0x5558a14252a0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f8bec009ce0 tx=0x7f8bec011440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=246dd805bd4beac1 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:16.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.400+0000 7f8c09e00700 1 --2- 172.21.15.1:0/1994264442 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f8be40a0790 0x7f8be40a2c50 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc0174f0 tx=0x7f8bfc0094b0 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:16.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.400+0000 7f8bfa600700 1 -- 172.21.15.1:0/1465855773 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bec011d20 con 0x5558a141a930 2024-02-02T15:43:16.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.400+0000 7f8bfa600700 1 -- 172.21.15.1:0/1465855773 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8bec011e80 con 0x5558a141a930 2024-02-02T15:43:16.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.400+0000 7f8bfa600700 1 -- 172.21.15.1:0/1465855773 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bec005280 con 0x5558a141a930 2024-02-02T15:43:16.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.401+0000 7f8c1fa1fb80 1 -- 172.21.15.1:0/1465855773 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5558a141a930 msgr2=0x5558a14252a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:16.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.401+0000 7f8c1fa1fb80 1 --2- 172.21.15.1:0/1465855773 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5558a141a930 0x5558a14252a0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f8bec009ce0 tx=0x7f8bec011440 comp rx=0 tx=0).stop 2024-02-02T15:43:16.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.402+0000 7f8c1fa1fb80 1 -- 172.21.15.1:0/1465855773 shutdown_connections 2024-02-02T15:43:16.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.402+0000 7f8c1fa1fb80 1 --2- 172.21.15.1:0/1465855773 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558a14257e0 0x5558a1427bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:16.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.402+0000 7f8c1fa1fb80 1 --2- 172.21.15.1:0/1465855773 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558a141a010 0x5558a141a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:16.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.402+0000 7f8c1fa1fb80 1 --2- 172.21.15.1:0/1465855773 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5558a141a930 0x5558a14252a0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:16.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.402+0000 7f8c1fa1fb80 1 -- 172.21.15.1:0/1465855773 >> 172.21.15.1:0/1465855773 conn(0x5558a1379db0 msgr2=0x5558a14163b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:16.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.403+0000 7f8c1fa1fb80 1 -- 172.21.15.1:0/1465855773 shutdown_connections 2024-02-02T15:43:16.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.403+0000 7f8c1fa1fb80 1 -- 172.21.15.1:0/1465855773 wait complete. 2024-02-02T15:43:16.403 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.404+0000 7f8c1fa1fb80 1 -- start start 2024-02-02T15:43:16.404 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.404+0000 7f8c1fa1fb80 1 Processor -- start 2024-02-02T15:43:16.405 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.405+0000 7f8c1fa1fb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558a141a010 0x5558a1425190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:16.406 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.405+0000 7f8c1fa1fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558a141a930 0x5558a14223a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:16.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.405+0000 7f8c1fa1fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5558a14257e0 0x5558a1422960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:16.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.405+0000 7f8c1fa1fb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558a136bd50 con 0x5558a141a010 2024-02-02T15:43:16.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.405+0000 7f8c1fa1fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558a1307d80 con 0x5558a14257e0 2024-02-02T15:43:16.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.405+0000 7f8c1fa1fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558a1307a20 con 0x5558a141a930 2024-02-02T15:43:16.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.405+0000 7f8c1fa1fb80 10 client.0 _ll_register_callbacks cb 0x5558a1309290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:43:16.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.406+0000 7f8c08a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558a141a930 0x5558a14223a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:16.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.406+0000 7f8c08a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558a141a930 0x5558a14223a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:49078/0 (socket says 172.21.15.1:49078) 2024-02-02T15:43:16.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.406+0000 7f8c08a00700 1 -- 172.21.15.1:0/151698210 learned_addr learned my addr 172.21.15.1:0/151698210 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:43:16.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.406+0000 7f8c09400700 1 --2- 172.21.15.1:0/151698210 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558a141a010 0x5558a1425190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:16.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.407+0000 7f8c08a00700 1 -- 172.21.15.1:0/151698210 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5558a14257e0 msgr2=0x5558a1422960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:16.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.407+0000 7f8c08a00700 1 --2- 172.21.15.1:0/151698210 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5558a14257e0 0x5558a1422960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:16.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.407+0000 7f8c08a00700 1 -- 172.21.15.1:0/151698210 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558a141a010 msgr2=0x5558a1425190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:16.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.407+0000 7f8c08a00700 1 --2- 172.21.15.1:0/151698210 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558a141a010 0x5558a1425190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:16.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.407+0000 7f8c08a00700 1 -- 172.21.15.1:0/151698210 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c041a1dc0 con 0x5558a141a930 2024-02-02T15:43:16.408 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.407+0000 7f8c09400700 1 --2- 172.21.15.1:0/151698210 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558a141a010 0x5558a1425190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:43:16.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.408+0000 7f8c08a00700 1 --2- 172.21.15.1:0/151698210 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558a141a930 0x5558a14223a0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f8bec009230 tx=0x7f8bec004be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cde748b8df15142c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:16.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.408+0000 7f8bf2a00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:43:16.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.408+0000 7f8bf2a00700 1 -- 172.21.15.1:0/151698210 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bec004800 con 0x5558a141a930 2024-02-02T15:43:16.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.408+0000 7f8c1fa1fb80 1 -- 172.21.15.1:0/151698210 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5558a1364dd0 con 0x5558a141a930 2024-02-02T15:43:16.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.408+0000 7f8bf2a00700 1 -- 172.21.15.1:0/151698210 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f8bec011b20 con 0x5558a141a930 2024-02-02T15:43:16.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.409+0000 7f8bf2a00700 1 -- 172.21.15.1:0/151698210 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8bec011ce0 con 0x5558a141a930 2024-02-02T15:43:16.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.409+0000 7f8c1fa1fb80 10 client.29824 fetch_fsmap learned FSMap version 67 2024-02-02T15:43:16.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.409+0000 7f8c1fa1fb80 1 -- 172.21.15.1:0/151698210 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) v3 -- 0x7f8bec0098e0 con 0x5558a141a930 2024-02-02T15:43:16.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.409+0000 7f8bf2a00700 1 -- 172.21.15.1:0/151698210 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5558a1364dd0 con 0x5558a141a930 2024-02-02T15:43:16.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.410+0000 7f8bf2a00700 1 -- 172.21.15.1:0/151698210 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 67) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f8bec0131b0 con 0x5558a141a930 2024-02-02T15:43:16.410 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.411+0000 7f8c1fa1fb80 10 client.29824 fetch_fsmap finished waiting for FSMap version 67 2024-02-02T15:43:16.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.411+0000 7f8c1fa1fb80 10 client.29824 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:43:16.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.411+0000 7f8c1fa1fb80 20 client.29824 populate_metadata read hostname 'smithi001' 2024-02-02T15:43:16.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.411+0000 7f8c1fa1fb80 1 --2- 172.21.15.1:0/151698210 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5558a143d800 0x5558a143fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:16.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.411+0000 7f8c1fa1fb80 4 client.29824 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:43:16.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.411+0000 7f8c1fa1fb80 1 -- 172.21.15.1:0/151698210 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5558a132af50 con 0x5558a143d800 2024-02-02T15:43:16.411 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.411+0000 7f8c09400700 1 --2- 172.21.15.1:0/151698210 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5558a143d800 0x5558a143fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:16.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.412+0000 7f8c09400700 1 --2- 172.21.15.1:0/151698210 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5558a143d800 0x5558a143fc20 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67688d10646c7b2 server_cookie=8f756f610ff333bc in_seq=0 out_seq=0 2024-02-02T15:43:16.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.412+0000 7f8bf2a00700 10 client.29824 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:43:16.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.415+0000 7f8bf2a00700 1 -- 172.21.15.1:0/151698210 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5558a132af50 con 0x5558a143d800 2024-02-02T15:43:16.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.415+0000 7f8bf2a00700 10 client.29824 handle_command_reply: tid=0 2024-02-02T15:43:16.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.415+0000 7f8c1fa1fb80 1 client.29824 shutdown 2024-02-02T15:43:16.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.416+0000 7f8c1fa1fb80 1 -- 172.21.15.1:0/151698210 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558a141a930 msgr2=0x5558a14223a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:16.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.416+0000 7f8c1fa1fb80 1 --2- 172.21.15.1:0/151698210 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558a141a930 0x5558a14223a0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f8bec009230 tx=0x7f8bec004be0 comp rx=0 tx=0).stop 2024-02-02T15:43:16.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.416+0000 7f8c1fa1fb80 1 -- 172.21.15.1:0/151698210 shutdown_connections 2024-02-02T15:43:16.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.416+0000 7f8c1fa1fb80 1 --2- 172.21.15.1:0/151698210 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5558a143d800 0x5558a143fc20 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:16.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.416+0000 7f8c1fa1fb80 1 --2- 172.21.15.1:0/151698210 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5558a14257e0 0x5558a1422960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:16.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.416+0000 7f8c1fa1fb80 1 --2- 172.21.15.1:0/151698210 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558a141a010 0x5558a1425190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:16.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.416+0000 7f8c1fa1fb80 1 --2- 172.21.15.1:0/151698210 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558a141a930 0x5558a14223a0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:16.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.416+0000 7f8c1fa1fb80 1 -- 172.21.15.1:0/151698210 >> 172.21.15.1:0/151698210 conn(0x5558a1379db0 msgr2=0x5558a1415fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:16.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.416+0000 7f8bf2a00700 0 client.29824 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:43:16.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.416+0000 7f8c1fa1fb80 1 -- 172.21.15.1:0/151698210 shutdown_connections 2024-02-02T15:43:16.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.416+0000 7f8c1fa1fb80 1 -- 172.21.15.1:0/151698210 wait complete. 2024-02-02T15:43:16.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.417+0000 7f8c1fa1fb80 20 client.29824 trim_cache size 0 max 16384 2024-02-02T15:43:16.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.451+0000 7f8c1fa1fb80 1 Processor -- start 2024-02-02T15:43:16.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.451+0000 7f8c1fa1fb80 1 -- start start 2024-02-02T15:43:16.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.452+0000 7f8c1fa1fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558a1379db0 0x5558a1434bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:16.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.452+0000 7f8c1fa1fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5558a141a010 0x5558a1424cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:16.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.452+0000 7f8c1fa1fb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558a141a930 0x5558a1439190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:16.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.452+0000 7f8c1fa1fb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558a11ef6a0 con 0x5558a141a930 2024-02-02T15:43:16.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.452+0000 7f8c1fa1fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558a1410580 con 0x5558a141a010 2024-02-02T15:43:16.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.452+0000 7f8c1fa1fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558a141e2d0 con 0x5558a1379db0 2024-02-02T15:43:16.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.452+0000 7f8c08a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5558a141a010 0x5558a1424cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:16.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.452+0000 7f8c08a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5558a141a010 0x5558a1424cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:35678/0 (socket says 172.21.15.1:35678) 2024-02-02T15:43:16.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.452+0000 7f8c08a00700 1 -- 172.21.15.1:0/1725137570 learned_addr learned my addr 172.21.15.1:0/1725137570 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:43:16.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.452+0000 7f8c09400700 1 --2- 172.21.15.1:0/1725137570 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558a1379db0 0x5558a1434bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:16.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.454+0000 7f8c09400700 1 -- 172.21.15.1:0/1725137570 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5558a141a010 msgr2=0x5558a1424cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:16.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.454+0000 7f8c09400700 1 --2- 172.21.15.1:0/1725137570 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5558a141a010 0x5558a1424cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:16.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.454+0000 7f8c09400700 1 -- 172.21.15.1:0/1725137570 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558a141a930 msgr2=0x5558a1439190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:16.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.454+0000 7f8c09400700 1 --2- 172.21.15.1:0/1725137570 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558a141a930 0x5558a1439190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:16.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.454+0000 7f8c09400700 1 -- 172.21.15.1:0/1725137570 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5558a1199b60 con 0x5558a1379db0 2024-02-02T15:43:16.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.454+0000 7f8c08a00700 1 --2- 172.21.15.1:0/1725137570 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5558a141a010 0x5558a1424cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:43:16.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.455+0000 7f8c09400700 1 --2- 172.21.15.1:0/1725137570 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558a1379db0 0x5558a1434bb0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f8bf40c9870 tx=0x7f8bf4095310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d70912f1cd7b345 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:16.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.456+0000 7f8bfb000700 1 -- 172.21.15.1:0/1725137570 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf400cc90 con 0x5558a1379db0 2024-02-02T15:43:16.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.456+0000 7f8bfb000700 1 -- 172.21.15.1:0/1725137570 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8bf4099640 con 0x5558a1379db0 2024-02-02T15:43:16.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.456+0000 7f8bfb000700 1 -- 172.21.15.1:0/1725137570 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf400cc90 con 0x5558a1379db0 2024-02-02T15:43:16.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.457+0000 7f8c1fa1fb80 1 -- 172.21.15.1:0/1725137570 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558a1379db0 msgr2=0x5558a1434bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:16.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.457+0000 7f8c1fa1fb80 1 --2- 172.21.15.1:0/1725137570 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558a1379db0 0x5558a1434bb0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f8bf40c9870 tx=0x7f8bf4095310 comp rx=0 tx=0).stop 2024-02-02T15:43:16.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.458+0000 7f8c1fa1fb80 1 -- 172.21.15.1:0/1725137570 shutdown_connections 2024-02-02T15:43:16.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.458+0000 7f8c1fa1fb80 1 --2- 172.21.15.1:0/1725137570 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558a141a930 0x5558a1439190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:16.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.458+0000 7f8c1fa1fb80 1 --2- 172.21.15.1:0/1725137570 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558a1379db0 0x5558a1434bb0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:16.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.458+0000 7f8c1fa1fb80 1 --2- 172.21.15.1:0/1725137570 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5558a141a010 0x5558a1424cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:16.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.458+0000 7f8c1fa1fb80 1 -- 172.21.15.1:0/1725137570 >> 172.21.15.1:0/1725137570 conn(0x5558a1427df0 msgr2=0x5558a1439d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:16.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.458+0000 7f8c1fa1fb80 1 -- 172.21.15.1:0/1725137570 shutdown_connections 2024-02-02T15:43:16.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.458+0000 7f8c1fa1fb80 1 -- 172.21.15.1:0/1725137570 wait complete. 2024-02-02T15:43:16.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.459+0000 7f8c1fa1fb80 1 -- start start 2024-02-02T15:43:16.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.459+0000 7f8c1fa1fb80 1 Processor -- start 2024-02-02T15:43:16.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.460+0000 7f8c1fa1fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558a1379db0 0x5558a141fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:16.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.461+0000 7f8c1fa1fb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558a141a010 0x5558a1420080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:16.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.461+0000 7f8c09400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558a1379db0 0x5558a141fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:16.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.461+0000 7f8c09400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558a1379db0 0x5558a141fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:49100/0 (socket says 172.21.15.1:49100) 2024-02-02T15:43:16.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.461+0000 7f8c09400700 1 -- 172.21.15.1:0/4030772607 learned_addr learned my addr 172.21.15.1:0/4030772607 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:43:16.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.461+0000 7f8c1fa1fb80 1 --2- 172.21.15.1:0/4030772607 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5558a141a930 0x5558a141e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:16.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.462+0000 7f8c08a00700 1 --2- 172.21.15.1:0/4030772607 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558a141a010 0x5558a1420080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:16.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.462+0000 7f8c1fa1fb80 1 -- 172.21.15.1:0/4030772607 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558a141ee40 con 0x5558a141a010 2024-02-02T15:43:16.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.462+0000 7f8c1fa1fb80 1 -- 172.21.15.1:0/4030772607 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558a141efb0 con 0x5558a141a930 2024-02-02T15:43:16.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.462+0000 7f8c1fa1fb80 1 -- 172.21.15.1:0/4030772607 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558a141f170 con 0x5558a1379db0 2024-02-02T15:43:16.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.462+0000 7f8c1fa1fb80 10 client.0 _ll_register_callbacks cb 0x5558a1309290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:43:16.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.462+0000 7f8c09400700 1 -- 172.21.15.1:0/4030772607 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5558a141a930 msgr2=0x5558a141e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:16.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.463+0000 7f8c09400700 1 --2- 172.21.15.1:0/4030772607 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5558a141a930 0x5558a141e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:16.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.463+0000 7f8c09400700 1 -- 172.21.15.1:0/4030772607 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558a141a010 msgr2=0x5558a1420080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:16.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.463+0000 7f8c09400700 1 --2- 172.21.15.1:0/4030772607 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558a141a010 0x5558a1420080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:16.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.463+0000 7f8c09400700 1 -- 172.21.15.1:0/4030772607 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bec0098e0 con 0x5558a1379db0 2024-02-02T15:43:16.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.463+0000 7f8c09400700 1 --2- 172.21.15.1:0/4030772607 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558a1379db0 0x5558a141fb40 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4095b40 tx=0x7f8bf4097800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7941ebf848d023f5 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:16.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.463+0000 7f8bf2a00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:43:16.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.465+0000 7f8bf2a00700 1 -- 172.21.15.1:0/4030772607 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf40a16b0 con 0x5558a1379db0 2024-02-02T15:43:16.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.465+0000 7f8bf2a00700 1 -- 172.21.15.1:0/4030772607 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f8bf4082e40 con 0x5558a1379db0 2024-02-02T15:43:16.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.465+0000 7f8c1fa1fb80 1 -- 172.21.15.1:0/4030772607 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5558a140e0f0 con 0x5558a1379db0 2024-02-02T15:43:16.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.467+0000 7f8bf2a00700 1 -- 172.21.15.1:0/4030772607 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8bf409d150 con 0x5558a1379db0 2024-02-02T15:43:16.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.469+0000 7f8c1fa1fb80 10 client.29830 fetch_fsmap learned FSMap version 67 2024-02-02T15:43:16.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.469+0000 7f8c1fa1fb80 1 -- 172.21.15.1:0/4030772607 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) v3 -- 0x7f8bfc009d00 con 0x5558a1379db0 2024-02-02T15:43:16.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.472+0000 7f8bf2a00700 1 -- 172.21.15.1:0/4030772607 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5558a140e0f0 con 0x5558a1379db0 2024-02-02T15:43:16.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.472+0000 7f8bf2a00700 1 -- 172.21.15.1:0/4030772607 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 67) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f8bf40840b0 con 0x5558a1379db0 2024-02-02T15:43:16.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.472+0000 7f8c1fa1fb80 10 client.29830 fetch_fsmap finished waiting for FSMap version 67 2024-02-02T15:43:16.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.472+0000 7f8c1fa1fb80 10 client.29830 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:43:16.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.472+0000 7f8c1fa1fb80 20 client.29830 populate_metadata read hostname 'smithi001' 2024-02-02T15:43:16.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.472+0000 7f8c1fa1fb80 1 --2- 172.21.15.1:0/4030772607 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5558a14257e0 0x5558a1492340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:16.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.472+0000 7f8c1fa1fb80 4 client.29830 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2024-02-02T15:43:16.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.472+0000 7f8c1fa1fb80 1 -- 172.21.15.1:0/4030772607 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5558a11bacf0 con 0x5558a14257e0 2024-02-02T15:43:16.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.473+0000 7f8c08a00700 1 --2- 172.21.15.1:0/4030772607 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5558a14257e0 0x5558a1492340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:16.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.474+0000 7f8c08a00700 1 --2- 172.21.15.1:0/4030772607 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5558a14257e0 0x5558a1492340 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57893ce387d564a6 server_cookie=bea061552c2db3f1 in_seq=0 out_seq=0 2024-02-02T15:43:16.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.474+0000 7f8bf2a00700 10 client.29830 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:43:16.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.474+0000 7f8bf2a00700 1 -- 172.21.15.1:0/4030772607 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5558a11bacf0 con 0x5558a14257e0 2024-02-02T15:43:16.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.475+0000 7f8bf2a00700 10 client.29830 handle_command_reply: tid=0 2024-02-02T15:43:16.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.475+0000 7f8c1fa1fb80 1 client.29830 shutdown 2024-02-02T15:43:16.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.476+0000 7f8c1fa1fb80 1 -- 172.21.15.1:0/4030772607 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558a1379db0 msgr2=0x5558a141fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:16.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.476+0000 7f8c1fa1fb80 1 --2- 172.21.15.1:0/4030772607 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558a1379db0 0x5558a141fb40 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4095b40 tx=0x7f8bf4097800 comp rx=0 tx=0).stop 2024-02-02T15:43:16.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.476+0000 7f8c1fa1fb80 1 -- 172.21.15.1:0/4030772607 shutdown_connections 2024-02-02T15:43:16.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.476+0000 7f8c1fa1fb80 1 --2- 172.21.15.1:0/4030772607 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5558a14257e0 0x5558a1492340 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:16.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.476+0000 7f8c1fa1fb80 1 --2- 172.21.15.1:0/4030772607 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5558a141a930 0x5558a141e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:16.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.476+0000 7f8bf2a00700 0 client.29830 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:43:16.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.476+0000 7f8c1fa1fb80 1 --2- 172.21.15.1:0/4030772607 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5558a1379db0 0x5558a141fb40 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:16.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.476+0000 7f8c1fa1fb80 1 --2- 172.21.15.1:0/4030772607 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5558a141a010 0x5558a1420080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:16.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.476+0000 7f8c1fa1fb80 1 -- 172.21.15.1:0/4030772607 >> 172.21.15.1:0/4030772607 conn(0x5558a1427df0 msgr2=0x5558a141d7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:16.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.477+0000 7f8c1fa1fb80 1 -- 172.21.15.1:0/4030772607 shutdown_connections 2024-02-02T15:43:16.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.477+0000 7f8c1fa1fb80 1 -- 172.21.15.1:0/4030772607 wait complete. 2024-02-02T15:43:16.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.478+0000 7f8c1fa1fb80 20 client.29830 trim_cache size 0 max 16384 2024-02-02T15:43:16.480 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:43:16.480 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-02T15:43:16.480 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2024-02-02T15:43:16.480 INFO:teuthology.orchestra.run.smithi001.stdout: "c09e3d0e-7171-4ece-a97b-02b8fda15974": { 2024-02-02T15:43:16.480 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2024-02-02T15:43:16.481 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "c09e3d0e-7171-4ece-a97b-02b8fda15974", 2024-02-02T15:43:16.481 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,force" 2024-02-02T15:43:16.481 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:43:16.481 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:43:16.481 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:43:16.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.481+0000 7f8c0be00700 1 -- 172.21.15.1:0/1994264442 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f8be40a0790 msgr2=0x7f8be40a2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:16.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.481+0000 7f8c0be00700 1 --2- 172.21.15.1:0/1994264442 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f8be40a0790 0x7f8be40a2c50 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc0174f0 tx=0x7f8bfc0094b0 comp rx=0 tx=0).stop 2024-02-02T15:43:16.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.481+0000 7f8c0be00700 1 -- 172.21.15.1:0/1994264442 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8c04101000 msgr2=0x7f8c04194e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:16.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.481+0000 7f8c0be00700 1 --2- 172.21.15.1:0/1994264442 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8c04101000 0x7f8c04194e10 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f8bf400eb30 tx=0x7f8bf400ee40 comp rx=0 tx=0).stop 2024-02-02T15:43:16.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.481+0000 7f8c0be00700 1 -- 172.21.15.1:0/1994264442 shutdown_connections 2024-02-02T15:43:16.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.481+0000 7f8c0be00700 1 --2- 172.21.15.1:0/1994264442 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f8be40a0790 0x7f8be40a2c50 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:16.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.481+0000 7f8c0be00700 1 --2- 172.21.15.1:0/1994264442 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8c041997b0 0x7f8c04199c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:16.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.481+0000 7f8c0be00700 1 --2- 172.21.15.1:0/1994264442 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8c04101000 0x7f8c04194e10 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:16.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.481+0000 7f8c0be00700 1 --2- 172.21.15.1:0/1994264442 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8c04106120 0x7f8c04195350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:16.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.481+0000 7f8c0be00700 1 -- 172.21.15.1:0/1994264442 >> 172.21.15.1:0/1994264442 conn(0x7f8c040fc840 msgr2=0x7f8c040fd2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:16.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.482+0000 7f8c0be00700 1 -- 172.21.15.1:0/1994264442 shutdown_connections 2024-02-02T15:43:16.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:16.482+0000 7f8c0be00700 1 -- 172.21.15.1:0/1994264442 wait complete. 2024-02-02T15:43:16.530 INFO:tasks.cephfs.filesystem:scrub status for tag:c09e3d0e-7171-4ece-a97b-02b8fda15974 - {'path': '/', 'tag': 'c09e3d0e-7171-4ece-a97b-02b8fda15974', 'options': 'recursive,force'} 2024-02-02T15:43:16.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:43:16 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:43:16.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:43:16 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:43:16.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:43:16 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:43:16.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:43:16 smithi071 ceph-mon[44707]: osdmap e109: 12 total, 12 up, 12 in 2024-02-02T15:43:16.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:43:16 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:43:16.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:43:16 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:43:16.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:43:16 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:43:16.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:43:16 smithi142 ceph-mon[45728]: osdmap e109: 12 total, 12 up, 12 in 2024-02-02T15:43:16.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:43:16 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:43:16.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:43:16 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:43:16.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:43:16 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:43:16.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:43:16 smithi001 ceph-mon[35470]: osdmap e109: 12 total, 12 up, 12 in 2024-02-02T15:43:16.988 INFO:tasks.workunit.client.0.smithi001.stdout: 10 94238 26.94 MB/sec execute 49 sec latency 64.618 ms 2024-02-02T15:43:17.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:43:17 smithi071 ceph-mon[44707]: scrub queued for path: / 2024-02-02T15:43:17.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:43:17 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:43:17.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:43:17 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:43:17.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:43:17 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:43:17.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:43:17 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:43:17.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:43:17 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:43:17.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:43:17 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:43:17.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:43:17 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:43:17.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:43:17 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:43:17.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:43:17 smithi071 ceph-mon[44707]: pgmap v510: 129 pgs: 129 active+clean; 1.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 5.4 MiB/s wr, 556 op/s 2024-02-02T15:43:17.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:43:17 smithi142 ceph-mon[45728]: scrub queued for path: / 2024-02-02T15:43:17.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:43:17 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:43:17.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:43:17 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:43:17.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:43:17 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:43:17.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:43:17 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:43:17.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:43:17 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:43:17.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:43:17 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:43:17.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:43:17 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:43:17.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:43:17 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:43:17.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:43:17 smithi142 ceph-mon[45728]: pgmap v510: 129 pgs: 129 active+clean; 1.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 5.4 MiB/s wr, 556 op/s 2024-02-02T15:43:17.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:43:17 smithi001 ceph-mon[35470]: scrub queued for path: / 2024-02-02T15:43:17.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:43:17 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:43:17.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:43:17 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:43:17.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:43:17 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:43:17.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:43:17 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:43:17.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:43:17 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:43:17.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:43:17 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:43:17.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:43:17 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:43:17.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:43:17 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:43:17.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:43:17 smithi001 ceph-mon[35470]: pgmap v510: 129 pgs: 129 active+clean; 1.5 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 5.4 MiB/s wr, 556 op/s 2024-02-02T15:43:17.988 INFO:tasks.workunit.client.0.smithi001.stdout: 10 95064 27.43 MB/sec execute 50 sec latency 47.211 ms 2024-02-02T15:43:18.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:43:18 smithi071 ceph-mon[44707]: pgmap v512: 129 pgs: 129 active+clean; 1.5 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 6.4 MiB/s wr, 596 op/s 2024-02-02T15:43:18.924 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:43:18 smithi142 ceph-mon[45728]: pgmap v512: 129 pgs: 129 active+clean; 1.5 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 6.4 MiB/s wr, 596 op/s 2024-02-02T15:43:18.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:43:18 smithi001 ceph-mon[35470]: pgmap v512: 129 pgs: 129 active+clean; 1.5 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 6.4 MiB/s wr, 596 op/s 2024-02-02T15:43:18.989 INFO:tasks.workunit.client.0.smithi001.stdout: 10 95184 26.96 MB/sec execute 51 sec latency 564.015 ms 2024-02-02T15:43:19.989 INFO:tasks.workunit.client.0.smithi001.stdout: 10 95490 26.83 MB/sec execute 52 sec latency 618.328 ms 2024-02-02T15:43:20.989 INFO:tasks.workunit.client.0.smithi001.stdout: 10 95956 26.70 MB/sec execute 53 sec latency 107.531 ms 2024-02-02T15:43:21.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:43:21 smithi071 ceph-mon[44707]: pgmap v513: 129 pgs: 129 active+clean; 1.5 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 6.4 MiB/s wr, 596 op/s 2024-02-02T15:43:21.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:43:21 smithi142 ceph-mon[45728]: pgmap v513: 129 pgs: 129 active+clean; 1.5 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 6.4 MiB/s wr, 596 op/s 2024-02-02T15:43:21.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:43:21 smithi001 ceph-mon[35470]: pgmap v513: 129 pgs: 129 active+clean; 1.5 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 6.4 MiB/s wr, 596 op/s 2024-02-02T15:43:21.989 INFO:tasks.workunit.client.0.smithi001.stdout: 10 96606 26.75 MB/sec execute 54 sec latency 73.573 ms 2024-02-02T15:43:22.989 INFO:tasks.workunit.client.0.smithi001.stdout: 10 97454 27.17 MB/sec execute 55 sec latency 41.299 ms 2024-02-02T15:43:23.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:43:23 smithi071 ceph-mon[44707]: pgmap v514: 129 pgs: 129 active+clean; 1.5 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 12 MiB/s wr, 618 op/s 2024-02-02T15:43:23.923 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:43:23 smithi142 ceph-mon[45728]: pgmap v514: 129 pgs: 129 active+clean; 1.5 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 12 MiB/s wr, 618 op/s 2024-02-02T15:43:23.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:43:23 smithi001 ceph-mon[35470]: pgmap v514: 129 pgs: 129 active+clean; 1.5 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 12 MiB/s wr, 618 op/s 2024-02-02T15:43:23.989 INFO:tasks.workunit.client.0.smithi001.stdout: 10 97699 26.88 MB/sec execute 56 sec latency 580.732 ms 2024-02-02T15:43:24.867 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:43:24 smithi071 ceph-mon[44707]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:43:24.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:43:24 smithi142 ceph-mon[45728]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:43:24.942 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:43:24 smithi001 ceph-mon[35470]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:43:24.989 INFO:tasks.workunit.client.0.smithi001.stdout: 10 98007 26.75 MB/sec execute 57 sec latency 673.800 ms 2024-02-02T15:43:25.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:43:25 smithi071 ceph-mon[44707]: pgmap v515: 129 pgs: 129 active+clean; 1.5 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 13 MiB/s wr, 621 op/s 2024-02-02T15:43:25.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:43:25 smithi142 ceph-mon[45728]: pgmap v515: 129 pgs: 129 active+clean; 1.5 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 13 MiB/s wr, 621 op/s 2024-02-02T15:43:25.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:43:25 smithi001 ceph-mon[35470]: pgmap v515: 129 pgs: 129 active+clean; 1.5 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 13 MiB/s wr, 621 op/s 2024-02-02T15:43:25.989 INFO:tasks.workunit.client.0.smithi001.stdout: 10 98476 26.76 MB/sec execute 58 sec latency 75.170 ms 2024-02-02T15:43:26.989 INFO:tasks.workunit.client.0.smithi001.stdout: 10 99233 26.94 MB/sec execute 59 sec latency 44.950 ms 2024-02-02T15:43:27.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:43:27 smithi071 ceph-mon[44707]: pgmap v516: 129 pgs: 129 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 17 MiB/s wr, 719 op/s 2024-02-02T15:43:27.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:43:27 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:43:27.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:43:27 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:43:27.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:43:27 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:43:27.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:43:27 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:43:27.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:43:27 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:43:27.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:43:27 smithi142 ceph-mon[45728]: pgmap v516: 129 pgs: 129 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 17 MiB/s wr, 719 op/s 2024-02-02T15:43:27.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:43:27 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:43:27.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:43:27 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:43:27.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:43:27 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:43:27.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:43:27 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:43:27.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:43:27 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:43:27.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:43:27 smithi001 ceph-mon[35470]: pgmap v516: 129 pgs: 129 active+clean; 1.6 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 17 MiB/s wr, 719 op/s 2024-02-02T15:43:27.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:43:27 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:43:27.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:43:27 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:43:27.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:43:27 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:43:27.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:43:27 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:43:27.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:43:27 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:43:27.989 INFO:tasks.workunit.client.0.smithi001.stdout: 10 99935 27.08 MB/sec execute 60 sec latency 53.749 ms 2024-02-02T15:43:28.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:43:28 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:43:28.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:43:28 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:43:28.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:43:28 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:43:28.989 INFO:tasks.workunit.client.0.smithi001.stdout: 10 100552 27.25 MB/sec execute 61 sec latency 77.747 ms 2024-02-02T15:43:29.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:43:29 smithi071 ceph-mon[44707]: pgmap v517: 129 pgs: 129 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 17 MiB/s wr, 726 op/s 2024-02-02T15:43:29.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:43:29 smithi071 ceph-mon[44707]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:43:29.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:43:29 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:43:29.867 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:43:29 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:43:29.867 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:43:29 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:43:29.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:43:29 smithi142 ceph-mon[45728]: pgmap v517: 129 pgs: 129 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 17 MiB/s wr, 726 op/s 2024-02-02T15:43:29.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:43:29 smithi142 ceph-mon[45728]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:43:29.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:43:29 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:43:29.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:43:29 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:43:29.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:43:29 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:43:29.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:43:29 smithi001 ceph-mon[35470]: pgmap v517: 129 pgs: 129 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 17 MiB/s wr, 726 op/s 2024-02-02T15:43:29.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:43:29 smithi001 ceph-mon[35470]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:43:29.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:43:29 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:43:29.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:43:29 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:43:29.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:43:29 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:43:29.989 INFO:tasks.workunit.client.0.smithi001.stdout: 10 101177 27.27 MB/sec execute 62 sec latency 99.061 ms 2024-02-02T15:43:30.989 INFO:tasks.workunit.client.0.smithi001.stdout: 10 101809 27.49 MB/sec execute 63 sec latency 48.032 ms 2024-02-02T15:43:31.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:43:31 smithi071 ceph-mon[44707]: pgmap v518: 129 pgs: 129 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 15 MiB/s wr, 640 op/s 2024-02-02T15:43:31.922 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:43:31 smithi142 ceph-mon[45728]: pgmap v518: 129 pgs: 129 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 15 MiB/s wr, 640 op/s 2024-02-02T15:43:31.939 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:43:31 smithi001 ceph-mon[35470]: pgmap v518: 129 pgs: 129 active+clean; 1.6 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 15 MiB/s wr, 640 op/s 2024-02-02T15:43:31.989 INFO:tasks.workunit.client.0.smithi001.stdout: 10 102556 27.63 MB/sec execute 64 sec latency 91.161 ms 2024-02-02T15:43:32.990 INFO:tasks.workunit.client.0.smithi001.stdout: 10 103234 27.82 MB/sec execute 65 sec latency 81.700 ms 2024-02-02T15:43:33.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:43:33 smithi071 ceph-mon[44707]: pgmap v519: 129 pgs: 129 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 19 MiB/s wr, 860 op/s 2024-02-02T15:43:33.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:43:33 smithi142 ceph-mon[45728]: pgmap v519: 129 pgs: 129 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 19 MiB/s wr, 860 op/s 2024-02-02T15:43:33.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:43:33 smithi001 ceph-mon[35470]: pgmap v519: 129 pgs: 129 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 19 MiB/s wr, 860 op/s 2024-02-02T15:43:33.990 INFO:tasks.workunit.client.0.smithi001.stdout: 10 103831 27.83 MB/sec execute 66 sec latency 70.345 ms 2024-02-02T15:43:34.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:43:34 smithi142 ceph-mon[45728]: pgmap v520: 129 pgs: 129 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 13 MiB/s wr, 620 op/s 2024-02-02T15:43:34.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:43:34 smithi142 ceph-mon[45728]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:43:34.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:43:34 smithi001 ceph-mon[35470]: pgmap v520: 129 pgs: 129 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 13 MiB/s wr, 620 op/s 2024-02-02T15:43:34.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:43:34 smithi001 ceph-mon[35470]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:43:34.990 INFO:tasks.workunit.client.0.smithi001.stdout: 10 104569 28.01 MB/sec execute 67 sec latency 52.354 ms 2024-02-02T15:43:35.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:43:34 smithi071 ceph-mon[44707]: pgmap v520: 129 pgs: 129 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 13 MiB/s wr, 620 op/s 2024-02-02T15:43:35.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:43:34 smithi071 ceph-mon[44707]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:43:35.990 INFO:tasks.workunit.client.0.smithi001.stdout: 10 105444 28.29 MB/sec execute 68 sec latency 53.308 ms 2024-02-02T15:43:36.990 INFO:tasks.workunit.client.0.smithi001.stdout: 10 105925 28.32 MB/sec execute 69 sec latency 70.697 ms 2024-02-02T15:43:37.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:43:37 smithi071 ceph-mon[44707]: pgmap v521: 129 pgs: 129 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 671 op/s 2024-02-02T15:43:37.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:43:37 smithi142 ceph-mon[45728]: pgmap v521: 129 pgs: 129 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 671 op/s 2024-02-02T15:43:37.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:43:37 smithi001 ceph-mon[35470]: pgmap v521: 129 pgs: 129 active+clean; 1.6 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 671 op/s 2024-02-02T15:43:37.990 INFO:tasks.workunit.client.0.smithi001.stdout: 10 106589 28.45 MB/sec execute 70 sec latency 60.635 ms 2024-02-02T15:43:38.990 INFO:tasks.workunit.client.0.smithi001.stdout: 10 107190 28.39 MB/sec execute 71 sec latency 70.829 ms 2024-02-02T15:43:39.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:43:39 smithi071 ceph-mon[44707]: pgmap v522: 129 pgs: 129 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 10 MiB/s wr, 583 op/s 2024-02-02T15:43:39.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:43:39 smithi142 ceph-mon[45728]: pgmap v522: 129 pgs: 129 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 10 MiB/s wr, 583 op/s 2024-02-02T15:43:39.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:43:39 smithi001 ceph-mon[35470]: pgmap v522: 129 pgs: 129 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 10 MiB/s wr, 583 op/s 2024-02-02T15:43:39.990 INFO:tasks.workunit.client.0.smithi001.stdout: 10 107899 28.52 MB/sec execute 72 sec latency 65.027 ms 2024-02-02T15:43:40.991 INFO:tasks.workunit.client.0.smithi001.stdout: 10 108778 28.80 MB/sec execute 73 sec latency 53.628 ms 2024-02-02T15:43:41.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:43:41 smithi071 ceph-mon[44707]: pgmap v523: 129 pgs: 129 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.6 MiB/s wr, 464 op/s 2024-02-02T15:43:41.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:43:41 smithi142 ceph-mon[45728]: pgmap v523: 129 pgs: 129 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.6 MiB/s wr, 464 op/s 2024-02-02T15:43:41.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:43:41 smithi001 ceph-mon[35470]: pgmap v523: 129 pgs: 129 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.6 MiB/s wr, 464 op/s 2024-02-02T15:43:41.991 INFO:tasks.workunit.client.0.smithi001.stdout: 10 109395 28.96 MB/sec execute 74 sec latency 56.459 ms 2024-02-02T15:43:42.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:43:42 smithi142 ceph-mon[45728]: pgmap v524: 129 pgs: 129 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 9.8 MiB/s wr, 629 op/s 2024-02-02T15:43:42.950 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:43:42 smithi001 ceph-mon[35470]: pgmap v524: 129 pgs: 129 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 9.8 MiB/s wr, 629 op/s 2024-02-02T15:43:42.990 INFO:tasks.workunit.client.0.smithi001.stdout: 10 110004 29.06 MB/sec execute 75 sec latency 49.249 ms 2024-02-02T15:43:43.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:43:42 smithi071 ceph-mon[44707]: pgmap v524: 129 pgs: 129 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 9.8 MiB/s wr, 629 op/s 2024-02-02T15:43:43.991 INFO:tasks.workunit.client.0.smithi001.stdout: 10 110623 28.91 MB/sec execute 76 sec latency 55.780 ms 2024-02-02T15:43:44.991 INFO:tasks.workunit.client.0.smithi001.stdout: 10 111559 29.25 MB/sec execute 77 sec latency 58.362 ms 2024-02-02T15:43:45.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:43:45 smithi071 ceph-mon[44707]: pgmap v525: 129 pgs: 129 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.8 MiB/s wr, 451 op/s 2024-02-02T15:43:45.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:43:45 smithi142 ceph-mon[45728]: pgmap v525: 129 pgs: 129 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.8 MiB/s wr, 451 op/s 2024-02-02T15:43:45.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:43:45 smithi001 ceph-mon[35470]: pgmap v525: 129 pgs: 129 active+clean; 1.6 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.8 MiB/s wr, 451 op/s 2024-02-02T15:43:45.991 INFO:tasks.workunit.client.0.smithi001.stdout: 10 112282 29.36 MB/sec execute 78 sec latency 91.560 ms 2024-02-02T15:43:46.532 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-02T15:43:46.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.927+0000 7f0b06200700 1 -- 172.21.15.1:0/4265693432 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b00073110 msgr2=0x7f0b000734f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:46.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.927+0000 7f0b06200700 1 --2- 172.21.15.1:0/4265693432 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b00073110 0x7f0b000734f0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f0af000b0a0 tx=0x7f0af000b3b0 comp rx=0 tx=0).stop 2024-02-02T15:43:46.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.928+0000 7f0b06200700 1 -- 172.21.15.1:0/4265693432 shutdown_connections 2024-02-02T15:43:46.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.928+0000 7f0b06200700 1 --2- 172.21.15.1:0/4265693432 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b0010d630 0x7f0b0010fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:46.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.928+0000 7f0b06200700 1 --2- 172.21.15.1:0/4265693432 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b00073110 0x7f0b000734f0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:46.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.928+0000 7f0b06200700 1 --2- 172.21.15.1:0/4265693432 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0b00073a30 0x7f0b0010d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:46.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.928+0000 7f0b06200700 1 -- 172.21.15.1:0/4265693432 >> 172.21.15.1:0/4265693432 conn(0x7f0b000fc840 msgr2=0x7f0b000fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:46.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.928+0000 7f0b06200700 1 -- 172.21.15.1:0/4265693432 shutdown_connections 2024-02-02T15:43:46.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.928+0000 7f0b06200700 1 -- 172.21.15.1:0/4265693432 wait complete. 2024-02-02T15:43:46.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.930+0000 7f0b06200700 1 Processor -- start 2024-02-02T15:43:46.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.930+0000 7f0b06200700 1 -- start start 2024-02-02T15:43:46.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.931+0000 7f0b06200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b00073110 0x7f0b00106430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:46.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.931+0000 7f0b06200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0b00073a30 0x7f0b00106990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:46.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.931+0000 7f0b06200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b0010d630 0x7f0b001014f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:46.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.931+0000 7f0b06200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b00101cc0 con 0x7f0b00073a30 2024-02-02T15:43:46.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.931+0000 7f0b06200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b00101e30 con 0x7f0b00073110 2024-02-02T15:43:46.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.931+0000 7f0aff400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b00073110 0x7f0b00106430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:46.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.931+0000 7f0b06200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b00101ff0 con 0x7f0b0010d630 2024-02-02T15:43:46.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.931+0000 7f0aff400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b00073110 0x7f0b00106430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42706/0 (socket says 172.21.15.1:42706) 2024-02-02T15:43:46.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.931+0000 7f0aff400700 1 -- 172.21.15.1:0/2204938463 learned_addr learned my addr 172.21.15.1:0/2204938463 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:43:46.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.932+0000 7f0aff400700 1 -- 172.21.15.1:0/2204938463 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b0010d630 msgr2=0x7f0b001014f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-02-02T15:43:46.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.932+0000 7f0aff400700 1 --2- 172.21.15.1:0/2204938463 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b0010d630 0x7f0b001014f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:46.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.932+0000 7f0aff400700 1 -- 172.21.15.1:0/2204938463 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0b00073a30 msgr2=0x7f0b00106990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:46.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.932+0000 7f0aff400700 1 --2- 172.21.15.1:0/2204938463 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0b00073a30 0x7f0b00106990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:46.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.932+0000 7f0aff400700 1 -- 172.21.15.1:0/2204938463 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0af0009d00 con 0x7f0b00073110 2024-02-02T15:43:46.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.932+0000 7f0aff400700 1 --2- 172.21.15.1:0/2204938463 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b00073110 0x7f0b00106430 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f0af0000f80 tx=0x7f0af000cee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:46.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.932+0000 7f0afcc00700 1 -- 172.21.15.1:0/2204938463 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af000e050 con 0x7f0b00073110 2024-02-02T15:43:46.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.933+0000 7f0b06200700 1 -- 172.21.15.1:0/2204938463 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b00102270 con 0x7f0b00073110 2024-02-02T15:43:46.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.933+0000 7f0afcc00700 1 -- 172.21.15.1:0/2204938463 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0af0005570 con 0x7f0b00073110 2024-02-02T15:43:46.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.933+0000 7f0afcc00700 1 -- 172.21.15.1:0/2204938463 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af001ba50 con 0x7f0b00073110 2024-02-02T15:43:46.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.933+0000 7f0b06200700 1 -- 172.21.15.1:0/2204938463 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b0010c4a0 con 0x7f0b00073110 2024-02-02T15:43:46.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.939+0000 7f0afcc00700 1 -- 172.21.15.1:0/2204938463 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f0af001bc90 con 0x7f0b00073110 2024-02-02T15:43:46.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.941+0000 7f0afcc00700 1 --2- 172.21.15.1:0/2204938463 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f0ae00a0790 0x7f0ae00a2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:46.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.942+0000 7f0af6800700 1 --2- 172.21.15.1:0/2204938463 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f0ae00a0790 0x7f0ae00a2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:46.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.942+0000 7f0b19caab80 1 Processor -- start 2024-02-02T15:43:46.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.943+0000 7f0afcc00700 1 -- 172.21.15.1:0/2204938463 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f0af00cda10 con 0x7f0b00073110 2024-02-02T15:43:46.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.943+0000 7f0b19caab80 1 -- start start 2024-02-02T15:43:46.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.943+0000 7f0b19caab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563dc4cc5010 0x563dc4cc53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:46.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.943+0000 7f0b19caab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563dc4cc5930 0x563dc4cd02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:46.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.943+0000 7f0b19caab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563dc4cd07e0 0x563dc4cd2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:46.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.944+0000 7f0af6800700 1 --2- 172.21.15.1:0/2204938463 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f0ae00a0790 0x7f0ae00a2c50 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0b00102cc0 tx=0x7f0ae8006d20 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:46.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.944+0000 7f0b19caab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x563dc4c394c0 con 0x563dc4cc5930 2024-02-02T15:43:46.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.944+0000 7f0b19caab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x563dc4c1daa0 con 0x563dc4cd07e0 2024-02-02T15:43:46.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.944+0000 7f0b19caab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563dc4c1b460 con 0x563dc4cc5010 2024-02-02T15:43:46.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.944+0000 7f0affe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563dc4cd07e0 0x563dc4cd2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:46.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.944+0000 7f0affe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563dc4cd07e0 0x563dc4cd2bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42710/0 (socket says 172.21.15.1:42710) 2024-02-02T15:43:46.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.944+0000 7f0affe00700 1 -- 172.21.15.1:0/872847610 learned_addr learned my addr 172.21.15.1:0/872847610 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:43:46.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.944+0000 7f0af6800700 1 --2- 172.21.15.1:0/872847610 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563dc4cc5930 0x563dc4cd02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:46.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.945+0000 7f0affe00700 1 -- 172.21.15.1:0/872847610 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563dc4cc5010 msgr2=0x563dc4cc53f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-02T15:43:46.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.945+0000 7f0affe00700 1 --2- 172.21.15.1:0/872847610 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563dc4cc5010 0x563dc4cc53f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:46.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.945+0000 7f0affe00700 1 -- 172.21.15.1:0/872847610 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563dc4cc5930 msgr2=0x563dc4cd02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:46.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.945+0000 7f0affe00700 1 --2- 172.21.15.1:0/872847610 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563dc4cc5930 0x563dc4cd02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:46.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.945+0000 7f0affe00700 1 -- 172.21.15.1:0/872847610 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563dc4a983b0 con 0x563dc4cd07e0 2024-02-02T15:43:46.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.945+0000 7f0affe00700 1 --2- 172.21.15.1:0/872847610 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563dc4cd07e0 0x563dc4cd2bd0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f0af8009c00 tx=0x7f0af8009f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f62f652d6a860e6 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:46.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.946+0000 7f0aefe00700 1 -- 172.21.15.1:0/872847610 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af8009750 con 0x563dc4cd07e0 2024-02-02T15:43:46.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.946+0000 7f0aefe00700 1 -- 172.21.15.1:0/872847610 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0af8004830 con 0x563dc4cd07e0 2024-02-02T15:43:46.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.946+0000 7f0aefe00700 1 -- 172.21.15.1:0/872847610 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af8005650 con 0x563dc4cd07e0 2024-02-02T15:43:46.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.947+0000 7f0b19caab80 1 -- 172.21.15.1:0/872847610 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563dc4cd07e0 msgr2=0x563dc4cd2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:46.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.947+0000 7f0b19caab80 1 --2- 172.21.15.1:0/872847610 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563dc4cd07e0 0x563dc4cd2bd0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f0af8009c00 tx=0x7f0af8009f10 comp rx=0 tx=0).stop 2024-02-02T15:43:46.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.947+0000 7f0b19caab80 1 -- 172.21.15.1:0/872847610 shutdown_connections 2024-02-02T15:43:46.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.947+0000 7f0b19caab80 1 --2- 172.21.15.1:0/872847610 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563dc4cd07e0 0x563dc4cd2bd0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:46.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.947+0000 7f0b19caab80 1 --2- 172.21.15.1:0/872847610 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563dc4cc5010 0x563dc4cc53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:46.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.947+0000 7f0b19caab80 1 --2- 172.21.15.1:0/872847610 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563dc4cc5930 0x563dc4cd02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:46.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.947+0000 7f0b19caab80 1 -- 172.21.15.1:0/872847610 >> 172.21.15.1:0/872847610 conn(0x563dc4c24db0 msgr2=0x563dc4cc13b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:46.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.947+0000 7f0b19caab80 1 -- 172.21.15.1:0/872847610 shutdown_connections 2024-02-02T15:43:46.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.947+0000 7f0b19caab80 1 -- 172.21.15.1:0/872847610 wait complete. 2024-02-02T15:43:46.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.948+0000 7f0b19caab80 1 -- start start 2024-02-02T15:43:46.948 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.948+0000 7f0b19caab80 1 Processor -- start 2024-02-02T15:43:46.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.949+0000 7f0b19caab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563dc4cc5010 0x563dc4ccd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:46.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.949+0000 7f0b19caab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563dc4cc5930 0x563dc4ccda30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:46.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.949+0000 7f0b19caab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563dc4cd07e0 0x563dc4ccdf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:46.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.949+0000 7f0b19caab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x563dc4c16d50 con 0x563dc4cc5010 2024-02-02T15:43:46.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.949+0000 7f0b19caab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x563dc4bb2d80 con 0x563dc4cc5930 2024-02-02T15:43:46.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.949+0000 7f0b19caab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563dc4bb2a20 con 0x563dc4cd07e0 2024-02-02T15:43:46.949 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.949+0000 7f0b19caab80 10 client.0 _ll_register_callbacks cb 0x563dc4bb4290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:43:46.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.950+0000 7f0af6800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563dc4cc5930 0x563dc4ccda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:46.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.950+0000 7f0affe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563dc4cd07e0 0x563dc4ccdf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:46.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.950+0000 7f0affe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563dc4cd07e0 0x563dc4ccdf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:43500/0 (socket says 172.21.15.1:43500) 2024-02-02T15:43:46.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.950+0000 7f0af6800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563dc4cc5930 0x563dc4ccda30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42726/0 (socket says 172.21.15.1:42726) 2024-02-02T15:43:46.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.950+0000 7f0affe00700 1 -- 172.21.15.1:0/565245140 learned_addr learned my addr 172.21.15.1:0/565245140 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:43:46.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.950+0000 7f0aff400700 1 --2- 172.21.15.1:0/565245140 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563dc4cc5010 0x563dc4ccd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:46.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.950+0000 7f0affe00700 1 -- 172.21.15.1:0/565245140 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563dc4cc5930 msgr2=0x563dc4ccda30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:46.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.950+0000 7f0affe00700 1 --2- 172.21.15.1:0/565245140 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563dc4cc5930 0x563dc4ccda30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:46.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.950+0000 7f0affe00700 1 -- 172.21.15.1:0/565245140 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563dc4cc5010 msgr2=0x563dc4ccd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:46.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.950+0000 7f0affe00700 1 --2- 172.21.15.1:0/565245140 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563dc4cc5010 0x563dc4ccd4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:46.951 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.950+0000 7f0affe00700 1 -- 172.21.15.1:0/565245140 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0af80098b0 con 0x563dc4cd07e0 2024-02-02T15:43:46.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.951+0000 7f0aecc00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:43:46.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.951+0000 7f0affe00700 1 --2- 172.21.15.1:0/565245140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563dc4cd07e0 0x563dc4ccdf70 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f0af8004fd0 tx=0x7f0af8009770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f028a39137764364 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:46.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.951+0000 7f0aecc00700 1 -- 172.21.15.1:0/565245140 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af8005580 con 0x563dc4cd07e0 2024-02-02T15:43:46.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.951+0000 7f0aecc00700 1 -- 172.21.15.1:0/565245140 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f0af8014070 con 0x563dc4cd07e0 2024-02-02T15:43:46.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.952+0000 7f0b19caab80 1 -- 172.21.15.1:0/565245140 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563dc4cb90f0 con 0x563dc4cd07e0 2024-02-02T15:43:46.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.952+0000 7f0aecc00700 1 -- 172.21.15.1:0/565245140 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0af8005580 con 0x563dc4cd07e0 2024-02-02T15:43:46.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.952+0000 7f0b19caab80 10 client.29836 fetch_fsmap learned FSMap version 70 2024-02-02T15:43:46.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.952+0000 7f0b19caab80 1 -- 172.21.15.1:0/565245140 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) v3 -- 0x563dc4a44b60 con 0x563dc4cd07e0 2024-02-02T15:43:46.952 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.952+0000 7f0aecc00700 1 -- 172.21.15.1:0/565245140 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0af80056e0 con 0x563dc4cd07e0 2024-02-02T15:43:46.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.953+0000 7f0aecc00700 1 -- 172.21.15.1:0/565245140 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 70) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f0af8003680 con 0x563dc4cd07e0 2024-02-02T15:43:46.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.953+0000 7f0b19caab80 10 client.29836 fetch_fsmap finished waiting for FSMap version 70 2024-02-02T15:43:46.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.953+0000 7f0b19caab80 10 client.29836 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:43:46.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.953+0000 7f0b19caab80 20 client.29836 populate_metadata read hostname 'smithi001' 2024-02-02T15:43:46.953 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.954+0000 7f0b19caab80 1 --2- 172.21.15.1:0/565245140 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x563dc4ce0280 0x563dc4ce26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:46.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.954+0000 7f0b19caab80 4 client.29836 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:43:46.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.954+0000 7f0b19caab80 1 -- 172.21.15.1:0/565245140 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563dc4cb90f0 con 0x563dc4ce0280 2024-02-02T15:43:46.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.954+0000 7f0aff400700 1 --2- 172.21.15.1:0/565245140 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x563dc4ce0280 0x563dc4ce26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:46.954 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.955+0000 7f0aff400700 1 --2- 172.21.15.1:0/565245140 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x563dc4ce0280 0x563dc4ce26a0 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1301a72257014c9f server_cookie=8e5efa9341b00512 in_seq=0 out_seq=0 2024-02-02T15:43:46.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.955+0000 7f0aecc00700 10 client.29836 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:43:46.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.955+0000 7f0aecc00700 1 -- 172.21.15.1:0/565245140 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x563dc4cb90f0 con 0x563dc4ce0280 2024-02-02T15:43:46.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.955+0000 7f0aecc00700 10 client.29836 handle_command_reply: tid=0 2024-02-02T15:43:46.955 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.956+0000 7f0b19caab80 1 client.29836 shutdown 2024-02-02T15:43:46.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.956+0000 7f0b19caab80 1 -- 172.21.15.1:0/565245140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563dc4cd07e0 msgr2=0x563dc4ccdf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:46.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.956+0000 7f0b19caab80 1 --2- 172.21.15.1:0/565245140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563dc4cd07e0 0x563dc4ccdf70 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f0af8004fd0 tx=0x7f0af8009770 comp rx=0 tx=0).stop 2024-02-02T15:43:46.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.956+0000 7f0b19caab80 1 -- 172.21.15.1:0/565245140 shutdown_connections 2024-02-02T15:43:46.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.956+0000 7f0b19caab80 1 --2- 172.21.15.1:0/565245140 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x563dc4ce0280 0x563dc4ce26a0 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:46.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.956+0000 7f0b19caab80 1 --2- 172.21.15.1:0/565245140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563dc4cd07e0 0x563dc4ccdf70 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:46.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.956+0000 7f0b19caab80 1 --2- 172.21.15.1:0/565245140 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563dc4cc5010 0x563dc4ccd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:46.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.956+0000 7f0aecc00700 0 client.29836 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:43:46.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.956+0000 7f0b19caab80 1 --2- 172.21.15.1:0/565245140 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563dc4cc5930 0x563dc4ccda30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:46.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.956+0000 7f0b19caab80 1 -- 172.21.15.1:0/565245140 >> 172.21.15.1:0/565245140 conn(0x563dc4c24db0 msgr2=0x563dc4cc0fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:46.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.957+0000 7f0b19caab80 1 -- 172.21.15.1:0/565245140 shutdown_connections 2024-02-02T15:43:46.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.957+0000 7f0b19caab80 1 -- 172.21.15.1:0/565245140 wait complete. 2024-02-02T15:43:46.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.957+0000 7f0b19caab80 20 client.29836 trim_cache size 0 max 16384 2024-02-02T15:43:46.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.987+0000 7f0b19caab80 1 Processor -- start 2024-02-02T15:43:46.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.987+0000 7f0b19caab80 1 -- start start 2024-02-02T15:43:46.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.988+0000 7f0b19caab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563dc4cc5010 0x563dc4cd7320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:46.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.988+0000 7f0b19caab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563dc4cc5930 0x563dc4cca840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:46.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.988+0000 7f0b19caab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563dc4cd07e0 0x563dc4ccad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:46.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.988+0000 7f0b19caab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x563dc4a9a6a0 con 0x563dc4cc5930 2024-02-02T15:43:46.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.988+0000 7f0b19caab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x563dc4cc01a0 con 0x563dc4cd07e0 2024-02-02T15:43:46.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.988+0000 7f0b19caab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563dc4cdc190 con 0x563dc4cc5010 2024-02-02T15:43:46.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.988+0000 7f0aff400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563dc4cc5010 0x563dc4cd7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:46.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.988+0000 7f0affe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563dc4cd07e0 0x563dc4ccad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:46.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.988+0000 7f0affe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563dc4cd07e0 0x563dc4ccad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42732/0 (socket says 172.21.15.1:42732) 2024-02-02T15:43:46.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.988+0000 7f0affe00700 1 -- 172.21.15.1:0/3425761976 learned_addr learned my addr 172.21.15.1:0/3425761976 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:43:46.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.989+0000 7f0af6800700 1 --2- 172.21.15.1:0/3425761976 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563dc4cc5930 0x563dc4cca840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:46.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.989+0000 7f0affe00700 1 -- 172.21.15.1:0/3425761976 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563dc4cc5010 msgr2=0x563dc4cd7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:46.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.989+0000 7f0affe00700 1 --2- 172.21.15.1:0/3425761976 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563dc4cc5010 0x563dc4cd7320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:46.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.989+0000 7f0affe00700 1 -- 172.21.15.1:0/3425761976 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563dc4cc5930 msgr2=0x563dc4cca840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:46.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.989+0000 7f0affe00700 1 --2- 172.21.15.1:0/3425761976 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563dc4cc5930 0x563dc4cca840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:46.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.989+0000 7f0affe00700 1 -- 172.21.15.1:0/3425761976 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563dc4bdcea0 con 0x563dc4cd07e0 2024-02-02T15:43:46.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.989+0000 7f0affe00700 1 --2- 172.21.15.1:0/3425761976 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563dc4cd07e0 0x563dc4ccad80 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f0af801b040 tx=0x7f0af8005120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b2ab6bff5ddd37d server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:46.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.989+0000 7f0af5000700 1 -- 172.21.15.1:0/3425761976 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af8005580 con 0x563dc4cd07e0 2024-02-02T15:43:46.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.990+0000 7f0af5000700 1 -- 172.21.15.1:0/3425761976 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0af80092a0 con 0x563dc4cd07e0 2024-02-02T15:43:46.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.990+0000 7f0af5000700 1 -- 172.21.15.1:0/3425761976 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af8004000 con 0x563dc4cd07e0 2024-02-02T15:43:46.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.991+0000 7f0b19caab80 1 -- 172.21.15.1:0/3425761976 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563dc4cd07e0 msgr2=0x563dc4ccad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:46.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.991+0000 7f0b19caab80 1 --2- 172.21.15.1:0/3425761976 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563dc4cd07e0 0x563dc4ccad80 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f0af801b040 tx=0x7f0af8005120 comp rx=0 tx=0).stop 2024-02-02T15:43:46.992 INFO:tasks.workunit.client.0.smithi001.stdout: 10 112915 29.53 MB/sec execute 79 sec latency 89.522 ms 2024-02-02T15:43:46.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.991+0000 7f0b19caab80 1 -- 172.21.15.1:0/3425761976 shutdown_connections 2024-02-02T15:43:46.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.991+0000 7f0b19caab80 1 --2- 172.21.15.1:0/3425761976 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563dc4cd07e0 0x563dc4ccad80 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:46.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.991+0000 7f0b19caab80 1 --2- 172.21.15.1:0/3425761976 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563dc4cc5010 0x563dc4cd7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:46.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.991+0000 7f0b19caab80 1 --2- 172.21.15.1:0/3425761976 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563dc4cc5930 0x563dc4cca840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:46.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.991+0000 7f0b19caab80 1 -- 172.21.15.1:0/3425761976 >> 172.21.15.1:0/3425761976 conn(0x563dc4c24db0 msgr2=0x563dc4cc88e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:46.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.991+0000 7f0b19caab80 1 -- 172.21.15.1:0/3425761976 shutdown_connections 2024-02-02T15:43:46.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.991+0000 7f0b19caab80 1 -- 172.21.15.1:0/3425761976 wait complete. 2024-02-02T15:43:46.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.992+0000 7f0b19caab80 1 -- start start 2024-02-02T15:43:46.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.992+0000 7f0b19caab80 1 Processor -- start 2024-02-02T15:43:46.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.993+0000 7f0b19caab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563dc4cc5010 0x563dc4d2b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:46.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.993+0000 7f0b19caab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563dc4cc5930 0x563dc4d29d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:46.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.993+0000 7f0b19caab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563dc4cd07e0 0x563dc4d2a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:46.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.993+0000 7f0b19caab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x563dc4d2a7d0 con 0x563dc4cc5010 2024-02-02T15:43:46.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.993+0000 7f0b19caab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x563dc4d2a940 con 0x563dc4cc5930 2024-02-02T15:43:46.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.993+0000 7f0b19caab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x563dc4d2ab00 con 0x563dc4cd07e0 2024-02-02T15:43:46.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.993+0000 7f0b19caab80 10 client.0 _ll_register_callbacks cb 0x563dc4bb4290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:43:46.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.994+0000 7f0af6800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563dc4cc5930 0x563dc4d29d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:46.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.994+0000 7f0af6800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563dc4cc5930 0x563dc4d29d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42734/0 (socket says 172.21.15.1:42734) 2024-02-02T15:43:46.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.994+0000 7f0af6800700 1 -- 172.21.15.1:0/1324036205 learned_addr learned my addr 172.21.15.1:0/1324036205 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:43:46.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.994+0000 7f0aff400700 1 --2- 172.21.15.1:0/1324036205 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563dc4cc5010 0x563dc4d2b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:46.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.994+0000 7f0aff400700 1 -- 172.21.15.1:0/1324036205 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563dc4cd07e0 msgr2=0x563dc4d2a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:46.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.994+0000 7f0aff400700 1 --2- 172.21.15.1:0/1324036205 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563dc4cd07e0 0x563dc4d2a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:46.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.994+0000 7f0aff400700 1 -- 172.21.15.1:0/1324036205 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563dc4cc5930 msgr2=0x563dc4d29d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:46.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.994+0000 7f0aff400700 1 --2- 172.21.15.1:0/1324036205 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563dc4cc5930 0x563dc4d29d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:46.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.994+0000 7f0aff400700 1 -- 172.21.15.1:0/1324036205 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563dc4a44b60 con 0x563dc4cc5010 2024-02-02T15:43:46.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.995+0000 7f0aff400700 1 --2- 172.21.15.1:0/1324036205 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563dc4cc5010 0x563dc4d2b500 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f0af00cad60 tx=0x7f0af009b370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c499c27182f5a376 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:46.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.995+0000 7f0aecc00700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:43:46.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.995+0000 7f0aecc00700 1 -- 172.21.15.1:0/1324036205 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af00a2ba0 con 0x563dc4cc5010 2024-02-02T15:43:46.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.995+0000 7f0aecc00700 1 -- 172.21.15.1:0/1324036205 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f0af00056e0 con 0x563dc4cc5010 2024-02-02T15:43:46.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.995+0000 7f0b19caab80 1 -- 172.21.15.1:0/1324036205 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0af8004830 con 0x563dc4cc5010 2024-02-02T15:43:46.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.995+0000 7f0aecc00700 1 -- 172.21.15.1:0/1324036205 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0af00b3100 con 0x563dc4cc5010 2024-02-02T15:43:46.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.995+0000 7f0b19caab80 10 client.19698 fetch_fsmap learned FSMap version 70 2024-02-02T15:43:46.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.995+0000 7f0b19caab80 1 -- 172.21.15.1:0/1324036205 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) v3 -- 0x563dc4a9ccd0 con 0x563dc4cc5010 2024-02-02T15:43:46.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.996+0000 7f0aecc00700 1 -- 172.21.15.1:0/1324036205 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0af8004830 con 0x563dc4cc5010 2024-02-02T15:43:46.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.996+0000 7f0aecc00700 1 -- 172.21.15.1:0/1324036205 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 70) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f0af001a3b0 con 0x563dc4cc5010 2024-02-02T15:43:46.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.996+0000 7f0b19caab80 10 client.19698 fetch_fsmap finished waiting for FSMap version 70 2024-02-02T15:43:46.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.996+0000 7f0b19caab80 10 client.19698 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:43:46.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.996+0000 7f0b19caab80 20 client.19698 populate_metadata read hostname 'smithi001' 2024-02-02T15:43:46.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.996+0000 7f0b19caab80 1 --2- 172.21.15.1:0/1324036205 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x563dc4d44160 0x563dc4d46580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:46.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.997+0000 7f0b19caab80 4 client.19698 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2024-02-02T15:43:46.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.997+0000 7f0b19caab80 1 -- 172.21.15.1:0/1324036205 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563dc4c0fdd0 con 0x563dc4d44160 2024-02-02T15:43:46.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.997+0000 7f0af6800700 1 --2- 172.21.15.1:0/1324036205 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x563dc4d44160 0x563dc4d46580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:46.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:46.998+0000 7f0aecc00700 10 client.19698 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:43:47.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:47.000+0000 7f0af6800700 1 --2- 172.21.15.1:0/1324036205 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x563dc4d44160 0x563dc4d46580 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9e964526bd0a75c server_cookie=3c088f8dbbefe9ed in_seq=0 out_seq=0 2024-02-02T15:43:47.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:47.000+0000 7f0aecc00700 1 -- 172.21.15.1:0/1324036205 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x563dc4c0fdd0 con 0x563dc4d44160 2024-02-02T15:43:47.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:47.000+0000 7f0aecc00700 10 client.19698 handle_command_reply: tid=0 2024-02-02T15:43:47.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:47.000+0000 7f0b19caab80 1 client.19698 shutdown 2024-02-02T15:43:47.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:47.002+0000 7f0b19caab80 1 -- 172.21.15.1:0/1324036205 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563dc4cc5010 msgr2=0x563dc4d2b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:47.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:47.002+0000 7f0b19caab80 1 --2- 172.21.15.1:0/1324036205 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563dc4cc5010 0x563dc4d2b500 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f0af00cad60 tx=0x7f0af009b370 comp rx=0 tx=0).stop 2024-02-02T15:43:47.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:47.002+0000 7f0b19caab80 1 -- 172.21.15.1:0/1324036205 shutdown_connections 2024-02-02T15:43:47.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:47.002+0000 7f0b19caab80 1 --2- 172.21.15.1:0/1324036205 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x563dc4d44160 0x563dc4d46580 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:47.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:47.002+0000 7f0b19caab80 1 --2- 172.21.15.1:0/1324036205 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x563dc4cd07e0 0x563dc4d2a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:47.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:47.002+0000 7f0b19caab80 1 --2- 172.21.15.1:0/1324036205 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x563dc4cc5010 0x563dc4d2b500 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:47.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:47.002+0000 7f0b19caab80 1 --2- 172.21.15.1:0/1324036205 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563dc4cc5930 0x563dc4d29d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:47.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:47.002+0000 7f0b19caab80 1 -- 172.21.15.1:0/1324036205 >> 172.21.15.1:0/1324036205 conn(0x563dc4c24db0 msgr2=0x563dc4ccd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:47.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:47.002+0000 7f0aecc00700 0 client.19698 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:43:47.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:47.003+0000 7f0b19caab80 1 -- 172.21.15.1:0/1324036205 shutdown_connections 2024-02-02T15:43:47.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:47.004+0000 7f0b19caab80 1 -- 172.21.15.1:0/1324036205 wait complete. 2024-02-02T15:43:47.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:47.005+0000 7f0b19caab80 20 client.19698 trim_cache size 0 max 16384 2024-02-02T15:43:47.005 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:43:47.005 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2024-02-02T15:43:47.005 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2024-02-02T15:43:47.005 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:43:47.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:47.007+0000 7f0b06200700 1 -- 172.21.15.1:0/2204938463 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f0ae00a0790 msgr2=0x7f0ae00a2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:47.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:47.007+0000 7f0b06200700 1 --2- 172.21.15.1:0/2204938463 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f0ae00a0790 0x7f0ae00a2c50 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0b00102cc0 tx=0x7f0ae8006d20 comp rx=0 tx=0).stop 2024-02-02T15:43:47.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:47.007+0000 7f0b06200700 1 -- 172.21.15.1:0/2204938463 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b00073110 msgr2=0x7f0b00106430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:47.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:47.007+0000 7f0b06200700 1 --2- 172.21.15.1:0/2204938463 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b00073110 0x7f0b00106430 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f0af0000f80 tx=0x7f0af000cee0 comp rx=0 tx=0).stop 2024-02-02T15:43:47.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:47.009+0000 7f0b06200700 1 -- 172.21.15.1:0/2204938463 shutdown_connections 2024-02-02T15:43:47.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:47.009+0000 7f0b06200700 1 --2- 172.21.15.1:0/2204938463 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f0ae00a0790 0x7f0ae00a2c50 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:47.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:47.009+0000 7f0b06200700 1 --2- 172.21.15.1:0/2204938463 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b0010d630 0x7f0b001014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:47.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:47.009+0000 7f0b06200700 1 --2- 172.21.15.1:0/2204938463 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b00073110 0x7f0b00106430 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:47.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:47.009+0000 7f0b06200700 1 --2- 172.21.15.1:0/2204938463 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0b00073a30 0x7f0b00106990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:47.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:47.009+0000 7f0b06200700 1 -- 172.21.15.1:0/2204938463 >> 172.21.15.1:0/2204938463 conn(0x7f0b000fc840 msgr2=0x7f0b0010f790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:47.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:47.009+0000 7f0b06200700 1 -- 172.21.15.1:0/2204938463 shutdown_connections 2024-02-02T15:43:47.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:47.009+0000 7f0b06200700 1 -- 172.21.15.1:0/2204938463 wait complete. 2024-02-02T15:43:47.038 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-02T15:43:47.039 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T15:43:47.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:47.345+0000 7fca45000700 1 -- 172.21.15.1:0/3134390936 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca40100f20 msgr2=0x7fca40101300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:47.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:47.345+0000 7fca45000700 1 --2- 172.21.15.1:0/3134390936 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca40100f20 0x7fca40101300 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fca30009990 tx=0x7fca30009ca0 comp rx=0 tx=0).stop 2024-02-02T15:43:47.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:47.345+0000 7fca45000700 1 -- 172.21.15.1:0/3134390936 shutdown_connections 2024-02-02T15:43:47.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:47.345+0000 7fca45000700 1 --2- 172.21.15.1:0/3134390936 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca40106040 0x7fca4010dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:47.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:47.345+0000 7fca45000700 1 --2- 172.21.15.1:0/3134390936 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca40100f20 0x7fca40101300 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:47.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:47.345+0000 7fca45000700 1 --2- 172.21.15.1:0/3134390936 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca401018d0 0x7fca40105910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:47.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:47.345+0000 7fca45000700 1 -- 172.21.15.1:0/3134390936 >> 172.21.15.1:0/3134390936 conn(0x7fca400fc7a0 msgr2=0x7fca400febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:47.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:47.345+0000 7fca45000700 1 -- 172.21.15.1:0/3134390936 shutdown_connections 2024-02-02T15:43:47.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:47.345+0000 7fca45000700 1 -- 172.21.15.1:0/3134390936 wait complete. 2024-02-02T15:43:47.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:47.346+0000 7fca45000700 1 Processor -- start 2024-02-02T15:43:47.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:47.347+0000 7fca45000700 1 -- start start 2024-02-02T15:43:47.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:47.347+0000 7fca45000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca40100f20 0x7fca4010c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:47.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:47.347+0000 7fca45000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca40106040 0x7fca4010cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:47.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:47.347+0000 7fca45000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca401a1d80 0x7fca401a2230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:47.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:47.347+0000 7fca45000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca401a2770 con 0x7fca401a1d80 2024-02-02T15:43:47.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:47.347+0000 7fca45000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca401a28e0 con 0x7fca40100f20 2024-02-02T15:43:47.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:47.347+0000 7fca45000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca401a2aa0 con 0x7fca40106040 2024-02-02T15:43:47.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:47.348+0000 7fca3de00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca40100f20 0x7fca4010c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:47.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:47.348+0000 7fca3de00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca40100f20 0x7fca4010c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42754/0 (socket says 172.21.15.1:42754) 2024-02-02T15:43:47.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:47.348+0000 7fca3de00700 1 -- 172.21.15.1:0/3848794322 learned_addr learned my addr 172.21.15.1:0/3848794322 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:43:47.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:47.348+0000 7fca3e800700 1 --2- 172.21.15.1:0/3848794322 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca401a1d80 0x7fca401a2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:47.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:47.348+0000 7fca3de00700 1 -- 172.21.15.1:0/3848794322 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca40106040 msgr2=0x7fca4010cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:47.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:47.348+0000 7fca3de00700 1 --2- 172.21.15.1:0/3848794322 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca40106040 0x7fca4010cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:47.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:47.348+0000 7fca3de00700 1 -- 172.21.15.1:0/3848794322 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca401a1d80 msgr2=0x7fca401a2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:47.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:47.348+0000 7fca3de00700 1 --2- 172.21.15.1:0/3848794322 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca401a1d80 0x7fca401a2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:47.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:47.348+0000 7fca3de00700 1 -- 172.21.15.1:0/3848794322 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca38009c40 con 0x7fca40100f20 2024-02-02T15:43:47.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:47.349+0000 7fca3de00700 1 --2- 172.21.15.1:0/3848794322 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca40100f20 0x7fca4010c9b0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fca30000c00 tx=0x7fca300057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:47.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:47.349+0000 7fca36a00700 1 -- 172.21.15.1:0/3848794322 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3001d070 con 0x7fca40100f20 2024-02-02T15:43:47.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:47.349+0000 7fca36a00700 1 -- 172.21.15.1:0/3848794322 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca30022470 con 0x7fca40100f20 2024-02-02T15:43:47.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:47.349+0000 7fca36a00700 1 -- 172.21.15.1:0/3848794322 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca30017650 con 0x7fca40100f20 2024-02-02T15:43:47.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:47.350+0000 7fca45000700 1 -- 172.21.15.1:0/3848794322 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca300095f0 con 0x7fca40100f20 2024-02-02T15:43:47.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:47.350+0000 7fca45000700 1 -- 172.21.15.1:0/3848794322 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca401aa210 con 0x7fca40100f20 2024-02-02T15:43:47.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:47.352+0000 7fca45000700 1 -- 172.21.15.1:0/3848794322 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca40067bc0 con 0x7fca40100f20 2024-02-02T15:43:47.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:47.353+0000 7fca36a00700 1 -- 172.21.15.1:0/3848794322 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7fca30022870 con 0x7fca40100f20 2024-02-02T15:43:47.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:47.355+0000 7fca36a00700 1 --2- 172.21.15.1:0/3848794322 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fca200a0740 0x7fca200a2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:47.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:47.355+0000 7fca36a00700 1 -- 172.21.15.1:0/3848794322 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fca300cd880 con 0x7fca40100f20 2024-02-02T15:43:47.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:47.355+0000 7fca3d400700 1 --2- 172.21.15.1:0/3848794322 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fca200a0740 0x7fca200a2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:47.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:47.356+0000 7fca3d400700 1 --2- 172.21.15.1:0/3848794322 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fca200a0740 0x7fca200a2c00 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fca28005fd0 tx=0x7fca28005d00 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:47.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:47.359+0000 7fca36a00700 1 -- 172.21.15.1:0/3848794322 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7fca30096d70 con 0x7fca40100f20 2024-02-02T15:43:47.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:47.729+0000 7fca45000700 1 -- 172.21.15.1:0/3848794322 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fca40101970 con 0x7fca40100f20 2024-02-02T15:43:47.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:47.730+0000 7fca36a00700 1 -- 172.21.15.1:0/3848794322 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+10874 (secure 0 0 0) 0x7fca3009a390 con 0x7fca40100f20 2024-02-02T15:43:47.730 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:43:47.730 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":70,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2922240448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2922240448},{"type":"v1","addr":"172.21.15.1:6835","nonce":2922240448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2729497431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2729497431},{"type":"v1","addr":"172.21.15.1:6837","nonce":2729497431}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":70,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-02T15:24:47.761435+0000","modified":"2024-02-02T15:43:34.579417+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24529,"mds_2":24518,"mds_3":24514,"mds_4":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14517":{"gid":14517,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/1218303683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":1218303683},{"type":"v1","addr":"172.21.15.1:6839","nonce":1218303683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14529":{"gid":14529,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/2059197496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2059197496},{"type":"v1","addr":"172.21.15.1:6841","nonce":2059197496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/7648986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":7648986},{"type":"v1","addr":"172.21.15.142:6833","nonce":7648986}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/3482746094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3482746094},{"type":"v1","addr":"172.21.15.71:6833","nonce":3482746094}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/388207814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":388207814},{"type":"v1","addr":"172.21.15.71:6835","nonce":388207814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6835/1052606547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1052606547},{"type":"v1","addr":"172.21.15.142:6835","nonce":1052606547}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.142:6837/2870289019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2870289019},{"type":"v1","addr":"172.21.15.142:6837","nonce":2870289019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.71:6837/628016199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":628016199},{"type":"v1","addr":"172.21.15.71:6837","nonce":628016199}]},"join_fscid":-1,"export_targets":[4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.142:6839/321675421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":321675421},{"type":"v1","addr":"172.21.15.142:6839","nonce":321675421}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/1268086493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1268086493},{"type":"v1","addr":"172.21.15.71:6839","nonce":1268086493}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-02T15:43:47.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:47.733+0000 7fca45000700 1 -- 172.21.15.1:0/3848794322 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fca200a0740 msgr2=0x7fca200a2c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:47.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:47.733+0000 7fca45000700 1 --2- 172.21.15.1:0/3848794322 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fca200a0740 0x7fca200a2c00 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fca28005fd0 tx=0x7fca28005d00 comp rx=0 tx=0).stop 2024-02-02T15:43:47.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:47.733+0000 7fca45000700 1 -- 172.21.15.1:0/3848794322 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca40100f20 msgr2=0x7fca4010c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:47.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:47.733+0000 7fca45000700 1 --2- 172.21.15.1:0/3848794322 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca40100f20 0x7fca4010c9b0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fca30000c00 tx=0x7fca300057f0 comp rx=0 tx=0).stop 2024-02-02T15:43:47.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:47.733+0000 7fca45000700 1 -- 172.21.15.1:0/3848794322 shutdown_connections 2024-02-02T15:43:47.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:47.733+0000 7fca45000700 1 --2- 172.21.15.1:0/3848794322 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fca200a0740 0x7fca200a2c00 secure :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fca28005fd0 tx=0x7fca28005d00 comp rx=0 tx=0).stop 2024-02-02T15:43:47.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:47.733+0000 7fca45000700 1 --2- 172.21.15.1:0/3848794322 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca401a1d80 0x7fca401a2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:47.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:47.733+0000 7fca45000700 1 --2- 172.21.15.1:0/3848794322 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca40100f20 0x7fca4010c9b0 secure :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fca30000c00 tx=0x7fca300057f0 comp rx=0 tx=0).stop 2024-02-02T15:43:47.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:47.733+0000 7fca45000700 1 --2- 172.21.15.1:0/3848794322 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca40106040 0x7fca4010cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:47.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:47.733+0000 7fca45000700 1 -- 172.21.15.1:0/3848794322 >> 172.21.15.1:0/3848794322 conn(0x7fca400fc7a0 msgr2=0x7fca401028e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:47.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:47.733+0000 7fca45000700 1 -- 172.21.15.1:0/3848794322 shutdown_connections 2024-02-02T15:43:47.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:47.733+0000 7fca45000700 1 -- 172.21.15.1:0/3848794322 wait complete. 2024-02-02T15:43:47.736 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 70 2024-02-02T15:43:47.757 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-02-02T15:43:47.840 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:43:47 smithi001 ceph-mon[35470]: pgmap v526: 129 pgs: 129 active+clean; 1.7 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 8.3 MiB/s wr, 541 op/s 2024-02-02T15:43:47.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:43:47 smithi071 ceph-mon[44707]: pgmap v526: 129 pgs: 129 active+clean; 1.7 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 8.3 MiB/s wr, 541 op/s 2024-02-02T15:43:47.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:43:47 smithi142 ceph-mon[45728]: pgmap v526: 129 pgs: 129 active+clean; 1.7 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 8.3 MiB/s wr, 541 op/s 2024-02-02T15:43:47.991 INFO:tasks.workunit.client.0.smithi001.stdout: 10 113504 29.57 MB/sec execute 80 sec latency 51.908 ms 2024-02-02T15:43:48.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.053+0000 7faadf400700 1 -- 172.21.15.1:0/4278039814 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faad8102950 msgr2=0x7faad810ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:48.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.053+0000 7faadf400700 1 --2- 172.21.15.1:0/4278039814 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faad8102950 0x7faad810ae30 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7faac00098f0 tx=0x7faac0009c00 comp rx=0 tx=0).stop 2024-02-02T15:43:48.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.053+0000 7faadf400700 1 -- 172.21.15.1:0/4278039814 shutdown_connections 2024-02-02T15:43:48.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.053+0000 7faadf400700 1 --2- 172.21.15.1:0/4278039814 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faad810b400 0x7faad810d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.053+0000 7faadf400700 1 --2- 172.21.15.1:0/4278039814 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faad8102030 0x7faad8102410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.053+0000 7faadf400700 1 --2- 172.21.15.1:0/4278039814 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faad8102950 0x7faad810ae30 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.053+0000 7faadf400700 1 -- 172.21.15.1:0/4278039814 >> 172.21.15.1:0/4278039814 conn(0x7faad8076d70 msgr2=0x7faad8077180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:48.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.053+0000 7faadf400700 1 -- 172.21.15.1:0/4278039814 shutdown_connections 2024-02-02T15:43:48.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.053+0000 7faadf400700 1 -- 172.21.15.1:0/4278039814 wait complete. 2024-02-02T15:43:48.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.054+0000 7faadf400700 1 Processor -- start 2024-02-02T15:43:48.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.055+0000 7faadf400700 1 -- start start 2024-02-02T15:43:48.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.055+0000 7faadf400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faad8102030 0x7faad8102e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:48.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.055+0000 7faadf400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faad8102950 0x7faad8103380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:48.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.055+0000 7faadf400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faad810b400 0x7faad81038c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:48.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.055+0000 7faadf400700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faad81a97d0 con 0x7faad8102030 2024-02-02T15:43:48.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.055+0000 7faadf400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faad81a9940 con 0x7faad8102950 2024-02-02T15:43:48.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.055+0000 7faadf400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faad81a9b00 con 0x7faad810b400 2024-02-02T15:43:48.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.055+0000 7faadca00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faad8102030 0x7faad8102e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:48.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.055+0000 7faadca00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faad8102030 0x7faad8102e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51636/0 (socket says 172.21.15.1:51636) 2024-02-02T15:43:48.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.056+0000 7faadca00700 1 -- 172.21.15.1:0/1618846307 learned_addr learned my addr 172.21.15.1:0/1618846307 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:43:48.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.056+0000 7faad7e00700 1 --2- 172.21.15.1:0/1618846307 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faad8102950 0x7faad8103380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:48.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.056+0000 7faadd400700 1 --2- 172.21.15.1:0/1618846307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faad810b400 0x7faad81038c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:48.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.056+0000 7faadca00700 1 -- 172.21.15.1:0/1618846307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faad810b400 msgr2=0x7faad81038c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:48.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.056+0000 7faadca00700 1 --2- 172.21.15.1:0/1618846307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faad810b400 0x7faad81038c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.056+0000 7faadca00700 1 -- 172.21.15.1:0/1618846307 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faad8102950 msgr2=0x7faad8103380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:48.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.056+0000 7faadca00700 1 --2- 172.21.15.1:0/1618846307 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faad8102950 0x7faad8103380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.056+0000 7faadca00700 1 -- 172.21.15.1:0/1618846307 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faac0009520 con 0x7faad8102030 2024-02-02T15:43:48.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.056+0000 7faadca00700 1 --2- 172.21.15.1:0/1618846307 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faad8102030 0x7faad8102e40 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7faac800e9c0 tx=0x7faac800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:48.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.057+0000 7faad5600700 1 -- 172.21.15.1:0/1618846307 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faac800cbd0 con 0x7faad8102030 2024-02-02T15:43:48.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.057+0000 7faad5600700 1 -- 172.21.15.1:0/1618846307 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faac800eeb0 con 0x7faad8102030 2024-02-02T15:43:48.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.057+0000 7faad5600700 1 -- 172.21.15.1:0/1618846307 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faac8005580 con 0x7faad8102030 2024-02-02T15:43:48.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.057+0000 7faadf400700 1 -- 172.21.15.1:0/1618846307 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faad81a9de0 con 0x7faad8102030 2024-02-02T15:43:48.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.057+0000 7faadf400700 1 -- 172.21.15.1:0/1618846307 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faad81aa210 con 0x7faad8102030 2024-02-02T15:43:48.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.062+0000 7faad5600700 1 -- 172.21.15.1:0/1618846307 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7faac8004750 con 0x7faad8102030 2024-02-02T15:43:48.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.063+0000 7faaf2ec2b80 1 Processor -- start 2024-02-02T15:43:48.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.064+0000 7faaf2ec2b80 1 -- start start 2024-02-02T15:43:48.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.064+0000 7faad5600700 1 --2- 172.21.15.1:0/1618846307 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7faab80a0710 0x7faab80a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:48.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.064+0000 7faaf2ec2b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a9f5434010 0x55a9f54343f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:48.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.064+0000 7faad5600700 1 -- 172.21.15.1:0/1618846307 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 9453+0+0 (secure 0 0 0) 0x7faac8014070 con 0x7faad8102030 2024-02-02T15:43:48.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.064+0000 7faaf2ec2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a9f5434930 0x55a9f543f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:48.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.064+0000 7faadd400700 1 --2- 172.21.15.1:0/1618846307 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7faab80a0710 0x7faab80a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:48.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.065+0000 7faaf2ec2b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a9f543f7e0 0x55a9f5441bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:48.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.065+0000 7faaf2ec2b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f53a84c0 con 0x55a9f543f7e0 2024-02-02T15:43:48.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.065+0000 7faaf2ec2b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f538caa0 con 0x55a9f5434010 2024-02-02T15:43:48.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.065+0000 7faaf2ec2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f538a460 con 0x55a9f5434930 2024-02-02T15:43:48.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.065+0000 7faadd400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a9f543f7e0 0x55a9f5441bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:48.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.066+0000 7faadd400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a9f543f7e0 0x55a9f5441bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51638/0 (socket says 172.21.15.1:51638) 2024-02-02T15:43:48.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.066+0000 7faadd400700 1 -- 172.21.15.1:0/184482117 learned_addr learned my addr 172.21.15.1:0/184482117 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:43:48.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.066+0000 7faad7e00700 1 --2- 172.21.15.1:0/184482117 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a9f5434930 0x55a9f543f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:48.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.066+0000 7faadd400700 1 --2- 172.21.15.1:0/1618846307 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7faab80a0710 0x7faab80a2bd0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7faad0009470 tx=0x7faad0009920 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:48.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.066+0000 7faadca00700 1 --2- 172.21.15.1:0/184482117 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a9f5434010 0x55a9f54343f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:48.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.066+0000 7faad7e00700 1 -- 172.21.15.1:0/184482117 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a9f5434010 msgr2=0x55a9f54343f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:48.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.066+0000 7faad7e00700 1 --2- 172.21.15.1:0/184482117 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a9f5434010 0x55a9f54343f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.066+0000 7faad7e00700 1 -- 172.21.15.1:0/184482117 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a9f543f7e0 msgr2=0x55a9f5441bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:48.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.066+0000 7faad7e00700 1 --2- 172.21.15.1:0/184482117 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a9f543f7e0 0x55a9f5441bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.066+0000 7faad7e00700 1 -- 172.21.15.1:0/184482117 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a9f52073b0 con 0x55a9f5434930 2024-02-02T15:43:48.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.067+0000 7faad7e00700 1 --2- 172.21.15.1:0/184482117 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a9f5434930 0x55a9f543f2a0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7faac0009230 tx=0x7faac0005dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7cff890ee52c63d server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:48.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.067+0000 7faacdc00700 1 -- 172.21.15.1:0/184482117 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faac000f7a0 con 0x55a9f5434930 2024-02-02T15:43:48.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.067+0000 7faacdc00700 1 -- 172.21.15.1:0/184482117 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faac001d040 con 0x55a9f5434930 2024-02-02T15:43:48.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.067+0000 7faacdc00700 1 -- 172.21.15.1:0/184482117 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faac0005600 con 0x55a9f5434930 2024-02-02T15:43:48.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.068+0000 7faaf2ec2b80 1 -- 172.21.15.1:0/184482117 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a9f5434930 msgr2=0x55a9f543f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:48.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.068+0000 7faaf2ec2b80 1 --2- 172.21.15.1:0/184482117 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a9f5434930 0x55a9f543f2a0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7faac0009230 tx=0x7faac0005dc0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.068+0000 7faaf2ec2b80 1 -- 172.21.15.1:0/184482117 shutdown_connections 2024-02-02T15:43:48.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.068+0000 7faaf2ec2b80 1 --2- 172.21.15.1:0/184482117 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a9f543f7e0 0x55a9f5441bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.068+0000 7faaf2ec2b80 1 --2- 172.21.15.1:0/184482117 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a9f5434010 0x55a9f54343f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.068+0000 7faaf2ec2b80 1 --2- 172.21.15.1:0/184482117 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a9f5434930 0x55a9f543f2a0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.068+0000 7faaf2ec2b80 1 -- 172.21.15.1:0/184482117 >> 172.21.15.1:0/184482117 conn(0x55a9f5393db0 msgr2=0x55a9f54303b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:48.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.068+0000 7faaf2ec2b80 1 -- 172.21.15.1:0/184482117 shutdown_connections 2024-02-02T15:43:48.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.069+0000 7faaf2ec2b80 1 -- 172.21.15.1:0/184482117 wait complete. 2024-02-02T15:43:48.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.069+0000 7faaf2ec2b80 1 -- start start 2024-02-02T15:43:48.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.069+0000 7faaf2ec2b80 1 Processor -- start 2024-02-02T15:43:48.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.070+0000 7faaf2ec2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a9f5434010 0x55a9f543f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:48.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.070+0000 7faaf2ec2b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a9f5434930 0x55a9f543c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:48.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.070+0000 7faaf2ec2b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a9f543f7e0 0x55a9f543cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:48.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.070+0000 7faaf2ec2b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f5385d50 con 0x55a9f543f7e0 2024-02-02T15:43:48.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.070+0000 7faaf2ec2b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f5321d80 con 0x55a9f5434930 2024-02-02T15:43:48.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.070+0000 7faaf2ec2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f5321a20 con 0x55a9f5434010 2024-02-02T15:43:48.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.070+0000 7faaf2ec2b80 10 client.0 _ll_register_callbacks cb 0x55a9f5323290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:43:48.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.071+0000 7faad7e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a9f5434930 0x55a9f543c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:48.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.071+0000 7faad7e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a9f5434930 0x55a9f543c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42788/0 (socket says 172.21.15.1:42788) 2024-02-02T15:43:48.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.071+0000 7faad7e00700 1 -- 172.21.15.1:0/1453803922 learned_addr learned my addr 172.21.15.1:0/1453803922 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:43:48.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.071+0000 7faadd400700 1 --2- 172.21.15.1:0/1453803922 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a9f543f7e0 0x55a9f543cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:48.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.071+0000 7faad7e00700 1 -- 172.21.15.1:0/1453803922 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a9f5434010 msgr2=0x55a9f543f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:48.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.071+0000 7faad7e00700 1 --2- 172.21.15.1:0/1453803922 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a9f5434010 0x55a9f543f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.071+0000 7faad7e00700 1 -- 172.21.15.1:0/1453803922 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a9f543f7e0 msgr2=0x55a9f543cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:48.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.071+0000 7faad7e00700 1 --2- 172.21.15.1:0/1453803922 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a9f543f7e0 0x55a9f543cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.071+0000 7faad7e00700 1 -- 172.21.15.1:0/1453803922 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faad810e560 con 0x55a9f5434930 2024-02-02T15:43:48.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.071+0000 7faad7e00700 1 --2- 172.21.15.1:0/1453803922 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a9f5434930 0x55a9f543c440 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7faac0009c40 tx=0x7faac0005990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b865915746be9421 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:48.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.071+0000 7faac6000700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:43:48.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.072+0000 7faac6000700 1 -- 172.21.15.1:0/1453803922 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faac002f070 con 0x55a9f5434930 2024-02-02T15:43:48.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.072+0000 7faac6000700 1 -- 172.21.15.1:0/1453803922 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 9453+0+0 (secure 0 0 0) 0x7faac000b770 con 0x55a9f5434930 2024-02-02T15:43:48.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.072+0000 7faaf2ec2b80 1 -- 172.21.15.1:0/1453803922 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a9f54280f0 con 0x55a9f5434930 2024-02-02T15:43:48.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.072+0000 7faac6000700 1 -- 172.21.15.1:0/1453803922 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7faac000b930 con 0x55a9f5434930 2024-02-02T15:43:48.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.072+0000 7faaf2ec2b80 10 client.29942 fetch_fsmap learned FSMap version 70 2024-02-02T15:43:48.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.072+0000 7faaf2ec2b80 1 -- 172.21.15.1:0/1453803922 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) v3 -- 0x55a9f51b3b60 con 0x55a9f5434930 2024-02-02T15:43:48.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.072+0000 7faac6000700 1 -- 172.21.15.1:0/1453803922 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a9f54280f0 con 0x55a9f5434930 2024-02-02T15:43:48.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.073+0000 7faac6000700 1 -- 172.21.15.1:0/1453803922 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 70) v1 ==== 7490+0+0 (secure 0 0 0) 0x7faac0032030 con 0x55a9f5434930 2024-02-02T15:43:48.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.073+0000 7faaf2ec2b80 10 client.29942 fetch_fsmap finished waiting for FSMap version 70 2024-02-02T15:43:48.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.073+0000 7faaf2ec2b80 10 client.29942 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-02-02T15:43:48.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.073+0000 7faaf2ec2b80 20 client.29942 populate_metadata read hostname 'smithi001' 2024-02-02T15:43:48.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.073+0000 7faaf2ec2b80 1 --2- 172.21.15.1:0/1453803922 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55a9f544ec90 0x55a9f54510b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:48.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.073+0000 7faaf2ec2b80 4 client.29942 mds_command: new command op to 24502 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:43:48.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.073+0000 7faaf2ec2b80 1 -- 172.21.15.1:0/1453803922 --> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a9f537edd0 con 0x55a9f544ec90 2024-02-02T15:43:48.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.073+0000 7faadca00700 1 --2- 172.21.15.1:0/1453803922 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55a9f544ec90 0x55a9f54510b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:48.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.074+0000 7faac6000700 10 client.29942 ms_handle_connect on v2:172.21.15.142:6834/1052606547 2024-02-02T15:43:48.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.074+0000 7faadca00700 1 --2- 172.21.15.1:0/1453803922 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55a9f544ec90 0x55a9f54510b0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=258b46621f472823 server_cookie=42784ad5c5122cfe in_seq=0 out_seq=0 2024-02-02T15:43:48.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.081+0000 7faac6000700 1 -- 172.21.15.1:0/1453803922 <== mds.4 v2:172.21.15.142:6834/1052606547 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55a9f537edd0 con 0x55a9f544ec90 2024-02-02T15:43:48.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.081+0000 7faac6000700 10 client.29942 handle_command_reply: tid=0 2024-02-02T15:43:48.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.082+0000 7faaf2ec2b80 1 client.29942 shutdown 2024-02-02T15:43:48.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.082+0000 7faaf2ec2b80 1 -- 172.21.15.1:0/1453803922 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a9f5434930 msgr2=0x55a9f543c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:48.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.082+0000 7faaf2ec2b80 1 --2- 172.21.15.1:0/1453803922 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a9f5434930 0x55a9f543c440 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7faac0009c40 tx=0x7faac0005990 comp rx=0 tx=0).stop 2024-02-02T15:43:48.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.082+0000 7faaf2ec2b80 1 -- 172.21.15.1:0/1453803922 shutdown_connections 2024-02-02T15:43:48.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.082+0000 7faaf2ec2b80 1 --2- 172.21.15.1:0/1453803922 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55a9f544ec90 0x55a9f54510b0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.082+0000 7faaf2ec2b80 1 --2- 172.21.15.1:0/1453803922 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a9f543f7e0 0x55a9f543cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.082+0000 7faaf2ec2b80 1 --2- 172.21.15.1:0/1453803922 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a9f5434010 0x55a9f543f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.082+0000 7faac6000700 0 client.29942 ms_handle_reset on v2:172.21.15.142:6834/1052606547 2024-02-02T15:43:48.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.082+0000 7faaf2ec2b80 1 --2- 172.21.15.1:0/1453803922 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a9f5434930 0x55a9f543c440 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.082+0000 7faaf2ec2b80 1 -- 172.21.15.1:0/1453803922 >> 172.21.15.1:0/1453803922 conn(0x55a9f5393db0 msgr2=0x55a9f542ffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:48.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.083+0000 7faaf2ec2b80 1 -- 172.21.15.1:0/1453803922 shutdown_connections 2024-02-02T15:43:48.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.083+0000 7faaf2ec2b80 1 -- 172.21.15.1:0/1453803922 wait complete. 2024-02-02T15:43:48.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.083+0000 7faaf2ec2b80 20 client.29942 trim_cache size 0 max 16384 2024-02-02T15:43:48.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.103+0000 7faaf2ec2b80 1 Processor -- start 2024-02-02T15:43:48.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.103+0000 7faaf2ec2b80 1 -- start start 2024-02-02T15:43:48.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.104+0000 7faaf2ec2b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a9f5434010 0x55a9f543cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:48.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.104+0000 7faaf2ec2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a9f5434930 0x55a9f543bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:48.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.104+0000 7faaf2ec2b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a9f543f7e0 0x55a9f542eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:48.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.104+0000 7faaf2ec2b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f52096a0 con 0x55a9f5434010 2024-02-02T15:43:48.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.104+0000 7faaf2ec2b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f543d180 con 0x55a9f543f7e0 2024-02-02T15:43:48.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.104+0000 7faaf2ec2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f544b120 con 0x55a9f5434930 2024-02-02T15:43:48.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.104+0000 7faadd400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a9f543f7e0 0x55a9f542eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:48.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.104+0000 7faadd400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a9f543f7e0 0x55a9f542eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42794/0 (socket says 172.21.15.1:42794) 2024-02-02T15:43:48.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.104+0000 7faadd400700 1 -- 172.21.15.1:0/291215180 learned_addr learned my addr 172.21.15.1:0/291215180 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:43:48.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.104+0000 7faad7e00700 1 --2- 172.21.15.1:0/291215180 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a9f5434930 0x55a9f543bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:48.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.104+0000 7faadca00700 1 --2- 172.21.15.1:0/291215180 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a9f5434010 0x55a9f543cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:48.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.104+0000 7faadd400700 1 -- 172.21.15.1:0/291215180 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a9f5434930 msgr2=0x55a9f543bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:48.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.104+0000 7faadd400700 1 --2- 172.21.15.1:0/291215180 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a9f5434930 0x55a9f543bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.104+0000 7faadd400700 1 -- 172.21.15.1:0/291215180 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a9f5434010 msgr2=0x55a9f543cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:48.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.104+0000 7faadd400700 1 --2- 172.21.15.1:0/291215180 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a9f5434010 0x55a9f543cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.104+0000 7faadd400700 1 -- 172.21.15.1:0/291215180 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a9f534bea0 con 0x55a9f543f7e0 2024-02-02T15:43:48.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.104+0000 7faad7e00700 1 --2- 172.21.15.1:0/291215180 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a9f5434930 0x55a9f543bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:43:48.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.105+0000 7faadd400700 1 --2- 172.21.15.1:0/291215180 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a9f543f7e0 0x55a9f542eb20 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7faad0017600 tx=0x7faad001b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4dd70b9d8b1c4a78 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:48.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.105+0000 7faace600700 1 -- 172.21.15.1:0/291215180 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad000e050 con 0x55a9f543f7e0 2024-02-02T15:43:48.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.105+0000 7faace600700 1 -- 172.21.15.1:0/291215180 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faad000f040 con 0x55a9f543f7e0 2024-02-02T15:43:48.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.105+0000 7faace600700 1 -- 172.21.15.1:0/291215180 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad0018a90 con 0x55a9f543f7e0 2024-02-02T15:43:48.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.106+0000 7faaf2ec2b80 1 -- 172.21.15.1:0/291215180 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a9f543f7e0 msgr2=0x55a9f542eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:48.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.106+0000 7faaf2ec2b80 1 --2- 172.21.15.1:0/291215180 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a9f543f7e0 0x55a9f542eb20 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7faad0017600 tx=0x7faad001b000 comp rx=0 tx=0).stop 2024-02-02T15:43:48.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.106+0000 7faaf2ec2b80 1 -- 172.21.15.1:0/291215180 shutdown_connections 2024-02-02T15:43:48.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.106+0000 7faaf2ec2b80 1 --2- 172.21.15.1:0/291215180 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a9f543f7e0 0x55a9f542eb20 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.106+0000 7faaf2ec2b80 1 --2- 172.21.15.1:0/291215180 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a9f5434010 0x55a9f543cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.106+0000 7faaf2ec2b80 1 --2- 172.21.15.1:0/291215180 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a9f5434930 0x55a9f543bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.106+0000 7faaf2ec2b80 1 -- 172.21.15.1:0/291215180 >> 172.21.15.1:0/291215180 conn(0x55a9f5393db0 msgr2=0x55a9f544b460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:48.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.106+0000 7faaf2ec2b80 1 -- 172.21.15.1:0/291215180 shutdown_connections 2024-02-02T15:43:48.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.106+0000 7faaf2ec2b80 1 -- 172.21.15.1:0/291215180 wait complete. 2024-02-02T15:43:48.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.106+0000 7faaf2ec2b80 1 -- start start 2024-02-02T15:43:48.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.107+0000 7faaf2ec2b80 1 Processor -- start 2024-02-02T15:43:48.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.107+0000 7faaf2ec2b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a9f5434010 0x55a9f5498a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:48.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.107+0000 7faaf2ec2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a9f5434930 0x55a9f5498fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:48.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.107+0000 7faaf2ec2b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a9f543f7e0 0x55a9f5444250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:48.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.107+0000 7faaf2ec2b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f5444850 con 0x55a9f543f7e0 2024-02-02T15:43:48.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.107+0000 7faaf2ec2b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f54449c0 con 0x55a9f5434010 2024-02-02T15:43:48.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.107+0000 7faaf2ec2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9f5444b80 con 0x55a9f5434930 2024-02-02T15:43:48.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.107+0000 7faaf2ec2b80 10 client.0 _ll_register_callbacks cb 0x55a9f5323290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:43:48.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.108+0000 7faadca00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a9f5434010 0x55a9f5498a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:48.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.108+0000 7faadca00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a9f5434010 0x55a9f5498a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42796/0 (socket says 172.21.15.1:42796) 2024-02-02T15:43:48.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.108+0000 7faadca00700 1 -- 172.21.15.1:0/3484823385 learned_addr learned my addr 172.21.15.1:0/3484823385 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:43:48.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.108+0000 7faad7e00700 1 --2- 172.21.15.1:0/3484823385 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a9f5434930 0x55a9f5498fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:48.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.108+0000 7faadd400700 1 --2- 172.21.15.1:0/3484823385 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a9f543f7e0 0x55a9f5444250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:48.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.109+0000 7faadca00700 1 -- 172.21.15.1:0/3484823385 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a9f5434930 msgr2=0x55a9f5498fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:48.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.109+0000 7faadca00700 1 --2- 172.21.15.1:0/3484823385 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a9f5434930 0x55a9f5498fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.109+0000 7faadca00700 1 -- 172.21.15.1:0/3484823385 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a9f543f7e0 msgr2=0x55a9f5444250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:48.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.109+0000 7faadca00700 1 --2- 172.21.15.1:0/3484823385 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a9f543f7e0 0x55a9f5444250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.109+0000 7faadca00700 1 -- 172.21.15.1:0/3484823385 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faad0007e30 con 0x55a9f5434010 2024-02-02T15:43:48.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.109+0000 7faad7e00700 1 --2- 172.21.15.1:0/3484823385 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a9f5434930 0x55a9f5498fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:43:48.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.109+0000 7faadd400700 1 --2- 172.21.15.1:0/3484823385 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a9f543f7e0 0x55a9f5444250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:43:48.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.110+0000 7faadca00700 1 --2- 172.21.15.1:0/3484823385 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a9f5434010 0x55a9f5498a80 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7faac8095ac0 tx=0x7faac8099220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7b5129942deb680 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:48.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.110+0000 7faac6000700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:43:48.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.110+0000 7faac6000700 1 -- 172.21.15.1:0/3484823385 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faac809d3f0 con 0x55a9f5434010 2024-02-02T15:43:48.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.110+0000 7faaf2ec2b80 1 -- 172.21.15.1:0/3484823385 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faac001d040 con 0x55a9f5434010 2024-02-02T15:43:48.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.110+0000 7faac6000700 1 -- 172.21.15.1:0/3484823385 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 9453+0+0 (secure 0 0 0) 0x7faac80a1630 con 0x55a9f5434010 2024-02-02T15:43:48.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.110+0000 7faac6000700 1 -- 172.21.15.1:0/3484823385 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7faac80056e0 con 0x55a9f5434010 2024-02-02T15:43:48.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.110+0000 7faaf2ec2b80 10 client.29948 fetch_fsmap learned FSMap version 70 2024-02-02T15:43:48.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.110+0000 7faaf2ec2b80 1 -- 172.21.15.1:0/3484823385 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) v3 -- 0x55a9f520bcd0 con 0x55a9f5434010 2024-02-02T15:43:48.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.111+0000 7faac6000700 1 -- 172.21.15.1:0/3484823385 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faac001d040 con 0x55a9f5434010 2024-02-02T15:43:48.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.111+0000 7faac6000700 1 -- 172.21.15.1:0/3484823385 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 70) v1 ==== 7490+0+0 (secure 0 0 0) 0x7faac8084140 con 0x55a9f5434010 2024-02-02T15:43:48.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.111+0000 7faaf2ec2b80 10 client.29948 fetch_fsmap finished waiting for FSMap version 70 2024-02-02T15:43:48.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.111+0000 7faaf2ec2b80 10 client.29948 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-02-02T15:43:48.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.111+0000 7faaf2ec2b80 20 client.29948 populate_metadata read hostname 'smithi001' 2024-02-02T15:43:48.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.111+0000 7faaf2ec2b80 1 --2- 172.21.15.1:0/3484823385 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55a9f54a1950 0x55a9f54a3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:48.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.111+0000 7faaf2ec2b80 4 client.29948 mds_command: new command op to 24502 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:43:48.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.111+0000 7faaf2ec2b80 1 -- 172.21.15.1:0/3484823385 --> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a9f5344f50 con 0x55a9f54a1950 2024-02-02T15:43:48.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.111+0000 7faad7e00700 1 --2- 172.21.15.1:0/3484823385 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55a9f54a1950 0x55a9f54a3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:48.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.112+0000 7faad7e00700 1 --2- 172.21.15.1:0/3484823385 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55a9f54a1950 0x55a9f54a3d40 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1fa6923762f90ca9 server_cookie=ac8654859c177d3f in_seq=0 out_seq=0 2024-02-02T15:43:48.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.112+0000 7faac6000700 10 client.29948 ms_handle_connect on v2:172.21.15.142:6834/1052606547 2024-02-02T15:43:48.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.113+0000 7faac6000700 1 -- 172.21.15.1:0/3484823385 <== mds.4 v2:172.21.15.142:6834/1052606547 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a9f5344f50 con 0x55a9f54a1950 2024-02-02T15:43:48.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.113+0000 7faac6000700 10 client.29948 handle_command_reply: tid=0 2024-02-02T15:43:48.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.113+0000 7faaf2ec2b80 1 client.29948 shutdown 2024-02-02T15:43:48.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.113+0000 7faaf2ec2b80 1 -- 172.21.15.1:0/3484823385 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a9f5434010 msgr2=0x55a9f5498a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:48.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.113+0000 7faaf2ec2b80 1 --2- 172.21.15.1:0/3484823385 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a9f5434010 0x55a9f5498a80 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7faac8095ac0 tx=0x7faac8099220 comp rx=0 tx=0).stop 2024-02-02T15:43:48.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.113+0000 7faaf2ec2b80 1 -- 172.21.15.1:0/3484823385 shutdown_connections 2024-02-02T15:43:48.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.113+0000 7faaf2ec2b80 1 --2- 172.21.15.1:0/3484823385 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55a9f54a1950 0x55a9f54a3d40 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.113+0000 7faaf2ec2b80 1 --2- 172.21.15.1:0/3484823385 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a9f543f7e0 0x55a9f5444250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.113+0000 7faaf2ec2b80 1 --2- 172.21.15.1:0/3484823385 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a9f5434010 0x55a9f5498a80 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.113+0000 7faaf2ec2b80 1 --2- 172.21.15.1:0/3484823385 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a9f5434930 0x55a9f5498fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.113+0000 7faac6000700 0 client.29948 ms_handle_reset on v2:172.21.15.142:6834/1052606547 2024-02-02T15:43:48.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.113+0000 7faaf2ec2b80 1 -- 172.21.15.1:0/3484823385 >> 172.21.15.1:0/3484823385 conn(0x55a9f5393db0 msgr2=0x55a9f542f5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:48.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.113+0000 7faaf2ec2b80 1 -- 172.21.15.1:0/3484823385 shutdown_connections 2024-02-02T15:43:48.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.113+0000 7faaf2ec2b80 1 -- 172.21.15.1:0/3484823385 wait complete. 2024-02-02T15:43:48.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.113+0000 7faaf2ec2b80 20 client.29948 trim_cache size 0 max 16384 2024-02-02T15:43:48.117 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:43:48.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.114+0000 7faadf400700 1 -- 172.21.15.1:0/1618846307 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7faab80a0710 msgr2=0x7faab80a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:48.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.114+0000 7faadf400700 1 --2- 172.21.15.1:0/1618846307 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7faab80a0710 0x7faab80a2bd0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7faad0009470 tx=0x7faad0009920 comp rx=0 tx=0).stop 2024-02-02T15:43:48.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.114+0000 7faadf400700 1 -- 172.21.15.1:0/1618846307 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faad8102030 msgr2=0x7faad8102e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:48.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.114+0000 7faadf400700 1 --2- 172.21.15.1:0/1618846307 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faad8102030 0x7faad8102e40 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7faac800e9c0 tx=0x7faac800ecd0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.114+0000 7faadf400700 1 -- 172.21.15.1:0/1618846307 shutdown_connections 2024-02-02T15:43:48.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.114+0000 7faadf400700 1 --2- 172.21.15.1:0/1618846307 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7faab80a0710 0x7faab80a2bd0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.114+0000 7faadf400700 1 --2- 172.21.15.1:0/1618846307 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faad810b400 0x7faad81038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.114+0000 7faadf400700 1 --2- 172.21.15.1:0/1618846307 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faad8102030 0x7faad8102e40 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.114+0000 7faadf400700 1 --2- 172.21.15.1:0/1618846307 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faad8102950 0x7faad8103380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.114+0000 7faadf400700 1 -- 172.21.15.1:0/1618846307 >> 172.21.15.1:0/1618846307 conn(0x7faad8076d70 msgr2=0x7faad80fdc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:48.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.114+0000 7faadf400700 1 -- 172.21.15.1:0/1618846307 shutdown_connections 2024-02-02T15:43:48.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.114+0000 7faadf400700 1 -- 172.21.15.1:0/1618846307 wait complete. 2024-02-02T15:43:48.135 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-02-02T15:43:48.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.395+0000 7f855aa00700 1 -- 172.21.15.1:0/195647540 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f854400fe90 con 0x7f855c100fa0 2024-02-02T15:43:48.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.395+0000 7f8562c00700 1 -- 172.21.15.1:0/195647540 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f855c100fa0 msgr2=0x7f855c101380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:48.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.395+0000 7f8562c00700 1 --2- 172.21.15.1:0/195647540 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f855c100fa0 0x7f855c101380 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f85440098f0 tx=0x7f8544009c00 comp rx=0 tx=0).stop 2024-02-02T15:43:48.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.396+0000 7f8562c00700 1 -- 172.21.15.1:0/195647540 shutdown_connections 2024-02-02T15:43:48.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.396+0000 7f8562c00700 1 --2- 172.21.15.1:0/195647540 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f855c1060c0 0x7f855c10dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.396+0000 7f8562c00700 1 --2- 172.21.15.1:0/195647540 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f855c100fa0 0x7f855c101380 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.396+0000 7f8562c00700 1 --2- 172.21.15.1:0/195647540 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f855c101950 0x7f855c105990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.396+0000 7f8562c00700 1 -- 172.21.15.1:0/195647540 >> 172.21.15.1:0/195647540 conn(0x7f855c0fc800 msgr2=0x7f855c0fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:48.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.396+0000 7f8562c00700 1 -- 172.21.15.1:0/195647540 shutdown_connections 2024-02-02T15:43:48.397 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.398+0000 7f8562c00700 1 -- 172.21.15.1:0/195647540 wait complete. 2024-02-02T15:43:48.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.399+0000 7f8562c00700 1 Processor -- start 2024-02-02T15:43:48.399 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.399+0000 7f8562c00700 1 -- start start 2024-02-02T15:43:48.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.400+0000 7f8562c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f855c100fa0 0x7f855c06ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:48.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.400+0000 7f8562c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f855c1060c0 0x7f855c070440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:48.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.400+0000 7f8562c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f855c06b0f0 0x7f855c06b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:48.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.400+0000 7f8562c00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f855c06bae0 con 0x7f855c100fa0 2024-02-02T15:43:48.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.400+0000 7f8562c00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f855c06bc50 con 0x7f855c1060c0 2024-02-02T15:43:48.400 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.400+0000 7f8562c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f855c06be10 con 0x7f855c06b0f0 2024-02-02T15:43:48.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.400+0000 7f855b400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f855c1060c0 0x7f855c070440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:48.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.400+0000 7f855b400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f855c1060c0 0x7f855c070440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42818/0 (socket says 172.21.15.1:42818) 2024-02-02T15:43:48.401 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.400+0000 7f855b400700 1 -- 172.21.15.1:0/1975451192 learned_addr learned my addr 172.21.15.1:0/1975451192 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:43:48.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.401+0000 7f855b400700 1 -- 172.21.15.1:0/1975451192 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f855c06b0f0 msgr2=0x7f855c06b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:48.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.401+0000 7f855b400700 1 --2- 172.21.15.1:0/1975451192 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f855c06b0f0 0x7f855c06b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.401+0000 7f855b400700 1 -- 172.21.15.1:0/1975451192 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f855c100fa0 msgr2=0x7f855c06ff00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-02T15:43:48.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.401+0000 7f855b400700 1 --2- 172.21.15.1:0/1975451192 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f855c100fa0 0x7f855c06ff00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.401+0000 7f855b400700 1 -- 172.21.15.1:0/1975451192 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8544009520 con 0x7f855c1060c0 2024-02-02T15:43:48.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.401+0000 7f855b400700 1 --2- 172.21.15.1:0/1975451192 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f855c1060c0 0x7f855c070440 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f854c00ff60 tx=0x7f854c015f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:48.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.401+0000 7f8558c00700 1 -- 172.21.15.1:0/1975451192 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f854c013e50 con 0x7f855c1060c0 2024-02-02T15:43:48.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.401+0000 7f8558c00700 1 -- 172.21.15.1:0/1975451192 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f854c017040 con 0x7f855c1060c0 2024-02-02T15:43:48.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.401+0000 7f8558c00700 1 -- 172.21.15.1:0/1975451192 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f854c01c900 con 0x7f855c1060c0 2024-02-02T15:43:48.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.402+0000 7f8562c00700 1 -- 172.21.15.1:0/1975451192 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f855c06ec60 con 0x7f855c1060c0 2024-02-02T15:43:48.402 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.402+0000 7f8562c00700 1 -- 172.21.15.1:0/1975451192 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f855c06f120 con 0x7f855c1060c0 2024-02-02T15:43:48.407 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.407+0000 7f8558c00700 1 -- 172.21.15.1:0/1975451192 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f854c01dbc0 con 0x7f855c1060c0 2024-02-02T15:43:48.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.409+0000 7f85767e3b80 1 Processor -- start 2024-02-02T15:43:48.409 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.410+0000 7f8558c00700 1 --2- 172.21.15.1:0/1975451192 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f853c0a0710 0x7f853c0a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:48.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.410+0000 7f8558c00700 1 -- 172.21.15.1:0/1975451192 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f854c0d3b10 con 0x7f855c1060c0 2024-02-02T15:43:48.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.411+0000 7f85767e3b80 1 -- start start 2024-02-02T15:43:48.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.411+0000 7f8560c00700 1 --2- 172.21.15.1:0/1975451192 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f853c0a0710 0x7f853c0a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:48.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.411+0000 7f85767e3b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561cfb588010 0x561cfb5883f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:48.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.411+0000 7f85767e3b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561cfb588930 0x561cfb5932a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:48.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.411+0000 7f85767e3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561cfb5937e0 0x561cfb595bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:48.412 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.412+0000 7f855be00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561cfb588010 0x561cfb5883f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:48.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.412+0000 7f85767e3b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cfb4fc4c0 con 0x561cfb588930 2024-02-02T15:43:48.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.412+0000 7f85767e3b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cfb4e0aa0 con 0x561cfb588010 2024-02-02T15:43:48.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.412+0000 7f85767e3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cfb4de460 con 0x561cfb5937e0 2024-02-02T15:43:48.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.412+0000 7f855be00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561cfb588010 0x561cfb5883f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42824/0 (socket says 172.21.15.1:42824) 2024-02-02T15:43:48.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.412+0000 7f855be00700 1 -- 172.21.15.1:0/2474399299 learned_addr learned my addr 172.21.15.1:0/2474399299 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:43:48.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.412+0000 7f855be00700 1 -- 172.21.15.1:0/2474399299 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561cfb5937e0 msgr2=0x561cfb595bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:48.415 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.412+0000 7f855be00700 1 --2- 172.21.15.1:0/2474399299 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561cfb5937e0 0x561cfb595bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.412+0000 7f855be00700 1 -- 172.21.15.1:0/2474399299 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561cfb588930 msgr2=0x561cfb5932a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:48.416 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.412+0000 7f855be00700 1 --2- 172.21.15.1:0/2474399299 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561cfb588930 0x561cfb5932a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.412+0000 7f855be00700 1 -- 172.21.15.1:0/2474399299 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561cfb35b3b0 con 0x561cfb588010 2024-02-02T15:43:48.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.413+0000 7f855be00700 1 --2- 172.21.15.1:0/2474399299 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561cfb588010 0x561cfb5883f0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f8544009fd0 tx=0x7f85440058e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8111ba09e154f32e server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:48.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.413+0000 7f8551200700 1 -- 172.21.15.1:0/2474399299 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f854401c070 con 0x561cfb588010 2024-02-02T15:43:48.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.413+0000 7f8551200700 1 -- 172.21.15.1:0/2474399299 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f854400f920 con 0x561cfb588010 2024-02-02T15:43:48.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.413+0000 7f8551200700 1 -- 172.21.15.1:0/2474399299 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8544005040 con 0x561cfb588010 2024-02-02T15:43:48.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.414+0000 7f8560c00700 1 --2- 172.21.15.1:0/1975451192 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f853c0a0710 0x7f853c0a2bd0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8554009c10 tx=0x7f8554009b80 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:48.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.415+0000 7f85767e3b80 1 -- 172.21.15.1:0/2474399299 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561cfb588010 msgr2=0x561cfb5883f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:48.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.415+0000 7f85767e3b80 1 --2- 172.21.15.1:0/2474399299 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561cfb588010 0x561cfb5883f0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f8544009fd0 tx=0x7f85440058e0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.415+0000 7f85767e3b80 1 -- 172.21.15.1:0/2474399299 shutdown_connections 2024-02-02T15:43:48.417 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.415+0000 7f85767e3b80 1 --2- 172.21.15.1:0/2474399299 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561cfb5937e0 0x561cfb595bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.415+0000 7f85767e3b80 1 --2- 172.21.15.1:0/2474399299 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561cfb588010 0x561cfb5883f0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.415+0000 7f85767e3b80 1 --2- 172.21.15.1:0/2474399299 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561cfb588930 0x561cfb5932a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.415+0000 7f85767e3b80 1 -- 172.21.15.1:0/2474399299 >> 172.21.15.1:0/2474399299 conn(0x561cfb4e7db0 msgr2=0x561cfb5843b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:48.418 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.416+0000 7f85767e3b80 1 -- 172.21.15.1:0/2474399299 shutdown_connections 2024-02-02T15:43:48.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.416+0000 7f85767e3b80 1 -- 172.21.15.1:0/2474399299 wait complete. 2024-02-02T15:43:48.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.417+0000 7f85767e3b80 1 -- start start 2024-02-02T15:43:48.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.417+0000 7f85767e3b80 1 Processor -- start 2024-02-02T15:43:48.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.418+0000 7f85767e3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561cfb588010 0x561cfb590590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:48.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.418+0000 7f85767e3b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561cfb588930 0x561cfb590ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:48.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.418+0000 7f85767e3b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561cfb5937e0 0x561cfb591010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:48.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.418+0000 7f85767e3b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cfb4d9d50 con 0x561cfb588930 2024-02-02T15:43:48.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.418+0000 7f85767e3b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cfb475d80 con 0x561cfb5937e0 2024-02-02T15:43:48.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.418+0000 7f85767e3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cfb475a20 con 0x561cfb588010 2024-02-02T15:43:48.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.419+0000 7f85767e3b80 10 client.0 _ll_register_callbacks cb 0x561cfb477290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:43:48.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.419+0000 7f8560c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561cfb5937e0 0x561cfb591010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:48.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.419+0000 7f8560c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561cfb5937e0 0x561cfb591010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42832/0 (socket says 172.21.15.1:42832) 2024-02-02T15:43:48.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.419+0000 7f8560c00700 1 -- 172.21.15.1:0/1106419623 learned_addr learned my addr 172.21.15.1:0/1106419623 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:43:48.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.419+0000 7f8560c00700 1 -- 172.21.15.1:0/1106419623 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561cfb588010 msgr2=0x561cfb590590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:48.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.419+0000 7f8560c00700 1 --2- 172.21.15.1:0/1106419623 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561cfb588010 0x561cfb590590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.419+0000 7f8560c00700 1 -- 172.21.15.1:0/1106419623 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561cfb588930 msgr2=0x561cfb590ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:48.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.419+0000 7f8560c00700 1 --2- 172.21.15.1:0/1106419623 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561cfb588930 0x561cfb590ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.419+0000 7f8560c00700 1 -- 172.21.15.1:0/1106419623 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f855c0732f0 con 0x561cfb5937e0 2024-02-02T15:43:48.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.420+0000 7f8560c00700 1 --2- 172.21.15.1:0/1106419623 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561cfb5937e0 0x561cfb591010 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f85540094f0 tx=0x7f8554004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1d6f1a045c96b72 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:48.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.420+0000 7f8549600700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:43:48.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.420+0000 7f8549600700 1 -- 172.21.15.1:0/1106419623 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8554004db0 con 0x561cfb5937e0 2024-02-02T15:43:48.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.420+0000 7f8549600700 1 -- 172.21.15.1:0/1106419623 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f8554019430 con 0x561cfb5937e0 2024-02-02T15:43:48.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.421+0000 7f85767e3b80 1 -- 172.21.15.1:0/1106419623 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561cfb57c0f0 con 0x561cfb5937e0 2024-02-02T15:43:48.421 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.422+0000 7f8549600700 1 -- 172.21.15.1:0/1106419623 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8554027650 con 0x561cfb5937e0 2024-02-02T15:43:48.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.422+0000 7f85767e3b80 10 client.29960 fetch_fsmap learned FSMap version 70 2024-02-02T15:43:48.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.422+0000 7f85767e3b80 1 -- 172.21.15.1:0/1106419623 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) v3 -- 0x561cfb307b60 con 0x561cfb5937e0 2024-02-02T15:43:48.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.422+0000 7f8549600700 1 -- 172.21.15.1:0/1106419623 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f855402b020 con 0x561cfb5937e0 2024-02-02T15:43:48.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.423+0000 7f8549600700 1 -- 172.21.15.1:0/1106419623 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 70) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f8554005020 con 0x561cfb5937e0 2024-02-02T15:43:48.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.423+0000 7f85767e3b80 10 client.29960 fetch_fsmap finished waiting for FSMap version 70 2024-02-02T15:43:48.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.423+0000 7f85767e3b80 10 client.29960 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-02-02T15:43:48.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.423+0000 7f85767e3b80 20 client.29960 populate_metadata read hostname 'smithi001' 2024-02-02T15:43:48.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.423+0000 7f85767e3b80 1 --2- 172.21.15.1:0/1106419623 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x561cfb5a3380 0x561cfb5a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:48.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.423+0000 7f85767e3b80 4 client.29960 mds_command: new command op to 24514 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:43:48.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.423+0000 7f85767e3b80 1 -- 172.21.15.1:0/1106419623 --> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561cfb4d2dd0 con 0x561cfb5a3380 2024-02-02T15:43:48.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.424+0000 7f855be00700 1 --2- 172.21.15.1:0/1106419623 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x561cfb5a3380 0x561cfb5a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:48.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.425+0000 7f855be00700 1 --2- 172.21.15.1:0/1106419623 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x561cfb5a3380 0x561cfb5a57c0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d1d8f22baca5aad0 server_cookie=b66e3787224b7ae in_seq=0 out_seq=0 2024-02-02T15:43:48.425 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.425+0000 7f8549600700 10 client.29960 ms_handle_connect on v2:172.21.15.142:6836/2870289019 2024-02-02T15:43:48.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.435+0000 7f8549600700 1 -- 172.21.15.1:0/1106419623 <== mds.3 v2:172.21.15.142:6836/2870289019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x561cfb4d2dd0 con 0x561cfb5a3380 2024-02-02T15:43:48.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.435+0000 7f8549600700 10 client.29960 handle_command_reply: tid=0 2024-02-02T15:43:48.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.435+0000 7f85767e3b80 1 client.29960 shutdown 2024-02-02T15:43:48.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.436+0000 7f85767e3b80 1 -- 172.21.15.1:0/1106419623 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561cfb5937e0 msgr2=0x561cfb591010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:48.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.436+0000 7f85767e3b80 1 --2- 172.21.15.1:0/1106419623 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561cfb5937e0 0x561cfb591010 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f85540094f0 tx=0x7f8554004ad0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.436+0000 7f85767e3b80 1 -- 172.21.15.1:0/1106419623 shutdown_connections 2024-02-02T15:43:48.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.436+0000 7f85767e3b80 1 --2- 172.21.15.1:0/1106419623 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561cfb5937e0 0x561cfb591010 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.436+0000 7f85767e3b80 1 --2- 172.21.15.1:0/1106419623 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561cfb588010 0x561cfb590590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.436+0000 7f85767e3b80 1 --2- 172.21.15.1:0/1106419623 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x561cfb5a3380 0x561cfb5a57c0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.436+0000 7f85767e3b80 1 --2- 172.21.15.1:0/1106419623 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561cfb588930 0x561cfb590ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.436+0000 7f85767e3b80 1 -- 172.21.15.1:0/1106419623 >> 172.21.15.1:0/1106419623 conn(0x561cfb4e7db0 msgr2=0x561cfb583fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:48.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.436+0000 7f8549600700 0 client.29960 ms_handle_reset on v2:172.21.15.142:6836/2870289019 2024-02-02T15:43:48.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.438+0000 7f85767e3b80 1 -- 172.21.15.1:0/1106419623 shutdown_connections 2024-02-02T15:43:48.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.440+0000 7f85767e3b80 1 -- 172.21.15.1:0/1106419623 wait complete. 2024-02-02T15:43:48.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.441+0000 7f85767e3b80 20 client.29960 trim_cache size 0 max 16384 2024-02-02T15:43:48.468 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.468+0000 7f85767e3b80 1 Processor -- start 2024-02-02T15:43:48.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.468+0000 7f85767e3b80 1 -- start start 2024-02-02T15:43:48.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.469+0000 7f85767e3b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561cfb588010 0x561cfb59a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:48.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.469+0000 7f85767e3b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561cfb588930 0x561cfb58b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:48.469 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.469+0000 7f85767e3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561cfb5937e0 0x561cfb58be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:48.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.469+0000 7f855b400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561cfb588930 0x561cfb58b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:48.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.470+0000 7f855b400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561cfb588930 0x561cfb58b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42840/0 (socket says 172.21.15.1:42840) 2024-02-02T15:43:48.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.470+0000 7f8560c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561cfb5937e0 0x561cfb58be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:48.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.470+0000 7f8560c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561cfb5937e0 0x561cfb58be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:43660/0 (socket says 172.21.15.1:43660) 2024-02-02T15:43:48.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.470+0000 7f855be00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561cfb588010 0x561cfb59a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:48.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.470+0000 7f855be00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561cfb588010 0x561cfb59a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51702/0 (socket says 172.21.15.1:51702) 2024-02-02T15:43:48.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.470+0000 7f8560c00700 1 -- 172.21.15.1:0/223089521 learned_addr learned my addr 172.21.15.1:0/223089521 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:43:48.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.470+0000 7f85767e3b80 1 -- 172.21.15.1:0/223089521 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cfb35d6a0 con 0x561cfb588010 2024-02-02T15:43:48.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.470+0000 7f85767e3b80 1 -- 172.21.15.1:0/223089521 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cfb57bcc0 con 0x561cfb588930 2024-02-02T15:43:48.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.470+0000 7f85767e3b80 1 -- 172.21.15.1:0/223089521 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cfb57fe00 con 0x561cfb5937e0 2024-02-02T15:43:48.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.471+0000 7f8560c00700 1 -- 172.21.15.1:0/223089521 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561cfb588930 msgr2=0x561cfb58b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:48.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.471+0000 7f8560c00700 1 --2- 172.21.15.1:0/223089521 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561cfb588930 0x561cfb58b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.471+0000 7f8560c00700 1 -- 172.21.15.1:0/223089521 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561cfb588010 msgr2=0x561cfb59a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:48.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.471+0000 7f8560c00700 1 --2- 172.21.15.1:0/223089521 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561cfb588010 0x561cfb59a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.471+0000 7f8560c00700 1 -- 172.21.15.1:0/223089521 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561cfb49fea0 con 0x561cfb5937e0 2024-02-02T15:43:48.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.471+0000 7f855be00700 1 --2- 172.21.15.1:0/223089521 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561cfb588010 0x561cfb59a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:43:48.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.472+0000 7f8560c00700 1 --2- 172.21.15.1:0/223089521 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561cfb5937e0 0x561cfb58be20 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f8554009db0 tx=0x7f855400b2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f54863d2f226332 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:48.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.472+0000 7f8551c00700 1 -- 172.21.15.1:0/223089521 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8554027450 con 0x561cfb5937e0 2024-02-02T15:43:48.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.472+0000 7f8551c00700 1 -- 172.21.15.1:0/223089521 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f855402b030 con 0x561cfb5937e0 2024-02-02T15:43:48.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.472+0000 7f8551c00700 1 -- 172.21.15.1:0/223089521 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8554004c10 con 0x561cfb5937e0 2024-02-02T15:43:48.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.473+0000 7f85767e3b80 1 -- 172.21.15.1:0/223089521 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561cfb5937e0 msgr2=0x561cfb58be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:48.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.473+0000 7f85767e3b80 1 --2- 172.21.15.1:0/223089521 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561cfb5937e0 0x561cfb58be20 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f8554009db0 tx=0x7f855400b2d0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.473+0000 7f85767e3b80 1 -- 172.21.15.1:0/223089521 shutdown_connections 2024-02-02T15:43:48.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.473+0000 7f85767e3b80 1 --2- 172.21.15.1:0/223089521 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561cfb5937e0 0x561cfb58be20 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.473+0000 7f85767e3b80 1 --2- 172.21.15.1:0/223089521 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561cfb588010 0x561cfb59a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.473+0000 7f85767e3b80 1 --2- 172.21.15.1:0/223089521 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561cfb588930 0x561cfb58b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.473+0000 7f85767e3b80 1 -- 172.21.15.1:0/223089521 >> 172.21.15.1:0/223089521 conn(0x561cfb4e7db0 msgr2=0x561cfb59e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:48.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.474+0000 7f85767e3b80 1 -- 172.21.15.1:0/223089521 shutdown_connections 2024-02-02T15:43:48.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.474+0000 7f85767e3b80 1 -- 172.21.15.1:0/223089521 wait complete. 2024-02-02T15:43:48.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.475+0000 7f85767e3b80 1 -- start start 2024-02-02T15:43:48.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.475+0000 7f85767e3b80 1 Processor -- start 2024-02-02T15:43:48.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.476+0000 7f85767e3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561cfb588010 0x561cfb5984a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:48.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.476+0000 7f85767e3b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561cfb588930 0x561cfb5989e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:48.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.476+0000 7f85767e3b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561cfb5937e0 0x561cfb5f3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:48.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.476+0000 7f85767e3b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cfb5f4440 con 0x561cfb5937e0 2024-02-02T15:43:48.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.476+0000 7f85767e3b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cfb5f4580 con 0x561cfb588930 2024-02-02T15:43:48.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.476+0000 7f85767e3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561cfb5f46c0 con 0x561cfb588010 2024-02-02T15:43:48.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.476+0000 7f85767e3b80 10 client.0 _ll_register_callbacks cb 0x561cfb477290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:43:48.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.477+0000 7f855b400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561cfb588930 0x561cfb5989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:48.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.477+0000 7f855b400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561cfb588930 0x561cfb5989e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42850/0 (socket says 172.21.15.1:42850) 2024-02-02T15:43:48.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.477+0000 7f855b400700 1 -- 172.21.15.1:0/1369955646 learned_addr learned my addr 172.21.15.1:0/1369955646 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:43:48.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.477+0000 7f855be00700 1 --2- 172.21.15.1:0/1369955646 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561cfb588010 0x561cfb5984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:48.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.477+0000 7f8560c00700 1 --2- 172.21.15.1:0/1369955646 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561cfb5937e0 0x561cfb5f3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:48.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.478+0000 7f855be00700 1 -- 172.21.15.1:0/1369955646 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561cfb588930 msgr2=0x561cfb5989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:48.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.478+0000 7f855be00700 1 --2- 172.21.15.1:0/1369955646 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561cfb588930 0x561cfb5989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.478+0000 7f855be00700 1 -- 172.21.15.1:0/1369955646 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561cfb5937e0 msgr2=0x561cfb5f3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:48.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.478+0000 7f855be00700 1 --2- 172.21.15.1:0/1369955646 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561cfb5937e0 0x561cfb5f3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.478+0000 7f855be00700 1 -- 172.21.15.1:0/1369955646 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8544027040 con 0x561cfb588010 2024-02-02T15:43:48.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.479+0000 7f855b400700 1 --2- 172.21.15.1:0/1369955646 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561cfb588930 0x561cfb5989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:43:48.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.480+0000 7f855be00700 1 --2- 172.21.15.1:0/1369955646 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561cfb588010 0x561cfb5984a0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f8544009810 tx=0x7f8544010cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7dd43c3efa2f7086 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:48.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.480+0000 7f8549600700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:43:48.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.481+0000 7f8549600700 1 -- 172.21.15.1:0/1369955646 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f854401c040 con 0x561cfb588010 2024-02-02T15:43:48.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.481+0000 7f8549600700 1 -- 172.21.15.1:0/1369955646 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f854402c3d0 con 0x561cfb588010 2024-02-02T15:43:48.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.481+0000 7f85767e3b80 1 -- 172.21.15.1:0/1369955646 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f854400f920 con 0x561cfb588010 2024-02-02T15:43:48.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.486+0000 7f8549600700 1 -- 172.21.15.1:0/1369955646 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7f854400f460 con 0x561cfb588010 2024-02-02T15:43:48.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.486+0000 7f85767e3b80 10 client.29866 fetch_fsmap learned FSMap version 70 2024-02-02T15:43:48.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.486+0000 7f85767e3b80 1 -- 172.21.15.1:0/1369955646 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) v3 -- 0x561cfb307b60 con 0x561cfb588010 2024-02-02T15:43:48.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.486+0000 7f8549600700 1 -- 172.21.15.1:0/1369955646 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f854400f920 con 0x561cfb588010 2024-02-02T15:43:48.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.487+0000 7f8549600700 1 -- 172.21.15.1:0/1369955646 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 70) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f8544015020 con 0x561cfb588010 2024-02-02T15:43:48.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.488+0000 7f85767e3b80 10 client.29866 fetch_fsmap finished waiting for FSMap version 70 2024-02-02T15:43:48.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.488+0000 7f85767e3b80 10 client.29866 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-02-02T15:43:48.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.488+0000 7f85767e3b80 20 client.29866 populate_metadata read hostname 'smithi001' 2024-02-02T15:43:48.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.488+0000 7f85767e3b80 1 --2- 172.21.15.1:0/1369955646 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x561cfb5f5890 0x561cfb5f7c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:48.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.488+0000 7f85767e3b80 4 client.29866 mds_command: new command op to 24514 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:43:48.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.488+0000 7f85767e3b80 1 -- 172.21.15.1:0/1369955646 --> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561cfb498f50 con 0x561cfb5f5890 2024-02-02T15:43:48.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.488+0000 7f855b400700 1 --2- 172.21.15.1:0/1369955646 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x561cfb5f5890 0x561cfb5f7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:48.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.489+0000 7f855b400700 1 --2- 172.21.15.1:0/1369955646 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x561cfb5f5890 0x561cfb5f7c80 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f8a9e0a24f11e7ad server_cookie=f28a4aa2e02e857 in_seq=0 out_seq=0 2024-02-02T15:43:48.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.489+0000 7f8549600700 10 client.29866 ms_handle_connect on v2:172.21.15.142:6836/2870289019 2024-02-02T15:43:48.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.491+0000 7f8549600700 1 -- 172.21.15.1:0/1369955646 <== mds.3 v2:172.21.15.142:6836/2870289019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561cfb498f50 con 0x561cfb5f5890 2024-02-02T15:43:48.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.491+0000 7f8549600700 10 client.29866 handle_command_reply: tid=0 2024-02-02T15:43:48.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.492+0000 7f85767e3b80 1 client.29866 shutdown 2024-02-02T15:43:48.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.492+0000 7f85767e3b80 1 -- 172.21.15.1:0/1369955646 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561cfb588010 msgr2=0x561cfb5984a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:48.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.492+0000 7f85767e3b80 1 --2- 172.21.15.1:0/1369955646 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561cfb588010 0x561cfb5984a0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f8544009810 tx=0x7f8544010cd0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.493+0000 7f85767e3b80 1 -- 172.21.15.1:0/1369955646 shutdown_connections 2024-02-02T15:43:48.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.493+0000 7f85767e3b80 1 --2- 172.21.15.1:0/1369955646 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x561cfb5f5890 0x561cfb5f7c80 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.493+0000 7f85767e3b80 1 --2- 172.21.15.1:0/1369955646 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x561cfb5937e0 0x561cfb5f3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.493+0000 7f85767e3b80 1 --2- 172.21.15.1:0/1369955646 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x561cfb588010 0x561cfb5984a0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.493+0000 7f85767e3b80 1 --2- 172.21.15.1:0/1369955646 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561cfb588930 0x561cfb5989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.493+0000 7f85767e3b80 1 -- 172.21.15.1:0/1369955646 >> 172.21.15.1:0/1369955646 conn(0x561cfb4e7db0 msgr2=0x561cfb58b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:48.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.493+0000 7f8549600700 0 client.29866 ms_handle_reset on v2:172.21.15.142:6836/2870289019 2024-02-02T15:43:48.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.493+0000 7f85767e3b80 1 -- 172.21.15.1:0/1369955646 shutdown_connections 2024-02-02T15:43:48.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.493+0000 7f85767e3b80 1 -- 172.21.15.1:0/1369955646 wait complete. 2024-02-02T15:43:48.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.494+0000 7f85767e3b80 20 client.29866 trim_cache size 0 max 16384 2024-02-02T15:43:48.495 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:43:48.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.495+0000 7f8562c00700 1 -- 172.21.15.1:0/1975451192 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f853c0a0710 msgr2=0x7f853c0a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:48.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.495+0000 7f8562c00700 1 --2- 172.21.15.1:0/1975451192 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f853c0a0710 0x7f853c0a2bd0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8554009c10 tx=0x7f8554009b80 comp rx=0 tx=0).stop 2024-02-02T15:43:48.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.495+0000 7f8562c00700 1 -- 172.21.15.1:0/1975451192 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f855c1060c0 msgr2=0x7f855c070440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:48.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.495+0000 7f8562c00700 1 --2- 172.21.15.1:0/1975451192 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f855c1060c0 0x7f855c070440 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f854c00ff60 tx=0x7f854c015f90 comp rx=0 tx=0).stop 2024-02-02T15:43:48.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.496+0000 7f8562c00700 1 -- 172.21.15.1:0/1975451192 shutdown_connections 2024-02-02T15:43:48.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.496+0000 7f8562c00700 1 --2- 172.21.15.1:0/1975451192 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f853c0a0710 0x7f853c0a2bd0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.496+0000 7f8562c00700 1 --2- 172.21.15.1:0/1975451192 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f855c06b0f0 0x7f855c06b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.496+0000 7f8562c00700 1 --2- 172.21.15.1:0/1975451192 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f855c100fa0 0x7f855c06ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.496+0000 7f8562c00700 1 --2- 172.21.15.1:0/1975451192 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f855c1060c0 0x7f855c070440 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.496+0000 7f8562c00700 1 -- 172.21.15.1:0/1975451192 >> 172.21.15.1:0/1975451192 conn(0x7f855c0fc800 msgr2=0x7f855c0fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:48.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.496+0000 7f8562c00700 1 -- 172.21.15.1:0/1975451192 shutdown_connections 2024-02-02T15:43:48.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.496+0000 7f8562c00700 1 -- 172.21.15.1:0/1975451192 wait complete. 2024-02-02T15:43:48.530 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-02T15:43:48.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.837+0000 7f97f4800700 1 -- 172.21.15.1:0/634092507 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97ec103230 msgr2=0x7f97ec103610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:48.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.837+0000 7f97f4800700 1 --2- 172.21.15.1:0/634092507 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97ec103230 0x7f97ec103610 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f97d4009990 tx=0x7f97d4009ca0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.838+0000 7f97f4800700 1 -- 172.21.15.1:0/634092507 shutdown_connections 2024-02-02T15:43:48.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.838+0000 7f97f4800700 1 --2- 172.21.15.1:0/634092507 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97ec108350 0x7f97ec10fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.838+0000 7f97f4800700 1 --2- 172.21.15.1:0/634092507 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97ec103230 0x7f97ec103610 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.838+0000 7f97f4800700 1 --2- 172.21.15.1:0/634092507 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f97ec103be0 0x7f97ec107c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.838+0000 7f97f4800700 1 -- 172.21.15.1:0/634092507 >> 172.21.15.1:0/634092507 conn(0x7f97ec0fea70 msgr2=0x7f97ec100e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:48.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.838+0000 7f97f4800700 1 -- 172.21.15.1:0/634092507 shutdown_connections 2024-02-02T15:43:48.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.838+0000 7f97f4800700 1 -- 172.21.15.1:0/634092507 wait complete. 2024-02-02T15:43:48.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.839+0000 7f97f4800700 1 Processor -- start 2024-02-02T15:43:48.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.840+0000 7f97f4800700 1 -- start start 2024-02-02T15:43:48.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.840+0000 7f97f4800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97ec103230 0x7f97ec075250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:48.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.840+0000 7f97f4800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97ec108350 0x7f97ec075790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:48.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.840+0000 7f97f4800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f97ec075cd0 0x7f97ec1a96f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:48.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.841+0000 7f97f4800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ec1a9c30 con 0x7f97ec075cd0 2024-02-02T15:43:48.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.841+0000 7f97f4800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ec1a9d70 con 0x7f97ec103230 2024-02-02T15:43:48.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.841+0000 7f97f1e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97ec103230 0x7f97ec075250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:48.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.841+0000 7f97f4800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ec1a9ee0 con 0x7f97ec108350 2024-02-02T15:43:48.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.841+0000 7f97f1e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97ec103230 0x7f97ec075250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42872/0 (socket says 172.21.15.1:42872) 2024-02-02T15:43:48.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.841+0000 7f97f1e00700 1 -- 172.21.15.1:0/2634185079 learned_addr learned my addr 172.21.15.1:0/2634185079 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:43:48.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.841+0000 7f97f2800700 1 --2- 172.21.15.1:0/2634185079 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f97ec075cd0 0x7f97ec1a96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:48.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.842+0000 7f97f1e00700 1 -- 172.21.15.1:0/2634185079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97ec108350 msgr2=0x7f97ec075790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:48.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.842+0000 7f97f1400700 1 --2- 172.21.15.1:0/2634185079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97ec108350 0x7f97ec075790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:48.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.842+0000 7f97f1e00700 1 --2- 172.21.15.1:0/2634185079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97ec108350 0x7f97ec075790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.843+0000 7f97f1e00700 1 -- 172.21.15.1:0/2634185079 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f97ec075cd0 msgr2=0x7f97ec1a96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:48.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.843+0000 7f97f1e00700 1 --2- 172.21.15.1:0/2634185079 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f97ec075cd0 0x7f97ec1a96f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.843+0000 7f97f1e00700 1 -- 172.21.15.1:0/2634185079 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97d40095f0 con 0x7f97ec103230 2024-02-02T15:43:48.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.843+0000 7f97f1400700 1 --2- 172.21.15.1:0/2634185079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97ec108350 0x7f97ec075790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:43:48.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.843+0000 7f97f1e00700 1 --2- 172.21.15.1:0/2634185079 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97ec103230 0x7f97ec075250 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f97d40095c0 tx=0x7f97d400f740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:48.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.843+0000 7f97eaa00700 1 -- 172.21.15.1:0/2634185079 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d401d070 con 0x7f97ec103230 2024-02-02T15:43:48.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.843+0000 7f97f4800700 1 -- 172.21.15.1:0/2634185079 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97ec1aa160 con 0x7f97ec103230 2024-02-02T15:43:48.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.843+0000 7f97f4800700 1 -- 172.21.15.1:0/2634185079 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97ec1aa5e0 con 0x7f97ec103230 2024-02-02T15:43:48.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.843+0000 7f97eaa00700 1 -- 172.21.15.1:0/2634185079 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f97d40054f0 con 0x7f97ec103230 2024-02-02T15:43:48.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.843+0000 7f97eaa00700 1 -- 172.21.15.1:0/2634185079 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d4004bd0 con 0x7f97ec103230 2024-02-02T15:43:48.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.847+0000 7f97eaa00700 1 -- 172.21.15.1:0/2634185079 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f97d4004d30 con 0x7f97ec103230 2024-02-02T15:43:48.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.850+0000 7f97eaa00700 1 --2- 172.21.15.1:0/2634185079 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f97cc0a0710 0x7f97cc0a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:48.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.850+0000 7f97eaa00700 1 -- 172.21.15.1:0/2634185079 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f97d40ccff0 con 0x7f97ec103230 2024-02-02T15:43:48.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.850+0000 7f980836fb80 1 Processor -- start 2024-02-02T15:43:48.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.850+0000 7f980836fb80 1 -- start start 2024-02-02T15:43:48.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.851+0000 7f980836fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3e4d61010 0x55e3e4d613f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:48.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.851+0000 7f980836fb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e3e4d61930 0x55e3e4d6c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:48.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.851+0000 7f97f1400700 1 --2- 172.21.15.1:0/2634185079 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f97cc0a0710 0x7f97cc0a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:48.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.851+0000 7f980836fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3e4d6c7e0 0x55e3e4d6ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:48.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.851+0000 7f980836fb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3e4cd54c0 con 0x55e3e4d61930 2024-02-02T15:43:48.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.851+0000 7f980836fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3e4cb9aa0 con 0x55e3e4d61010 2024-02-02T15:43:48.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.851+0000 7f980836fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3e4cb7460 con 0x55e3e4d6c7e0 2024-02-02T15:43:48.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.851+0000 7f97f2800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3e4d6c7e0 0x55e3e4d6ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:48.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.851+0000 7f97f2800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3e4d6c7e0 0x55e3e4d6ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:43714/0 (socket says 172.21.15.1:43714) 2024-02-02T15:43:48.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.851+0000 7f97f2800700 1 -- 172.21.15.1:0/2438234354 learned_addr learned my addr 172.21.15.1:0/2438234354 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:43:48.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.852+0000 7f97f1e00700 1 --2- 172.21.15.1:0/2438234354 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3e4d61010 0x55e3e4d613f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:48.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.852+0000 7f97f1400700 1 --2- 172.21.15.1:0/2634185079 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f97cc0a0710 0x7f97cc0a2bd0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f97dc005d90 tx=0x7f97dc00e2f0 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:48.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.852+0000 7f97f1e00700 1 -- 172.21.15.1:0/2438234354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3e4d6c7e0 msgr2=0x55e3e4d6ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:48.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.852+0000 7f97f1400700 1 --2- 172.21.15.1:0/2438234354 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e3e4d61930 0x55e3e4d6c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:48.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.852+0000 7f97f1e00700 1 --2- 172.21.15.1:0/2438234354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3e4d6c7e0 0x55e3e4d6ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.852+0000 7f97f1e00700 1 -- 172.21.15.1:0/2438234354 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e3e4d61930 msgr2=0x55e3e4d6c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:48.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.852+0000 7f97f1e00700 1 --2- 172.21.15.1:0/2438234354 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e3e4d61930 0x55e3e4d6c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.852+0000 7f97f1e00700 1 -- 172.21.15.1:0/2438234354 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e3e4b343b0 con 0x55e3e4d61010 2024-02-02T15:43:48.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.852+0000 7f97f1400700 1 --2- 172.21.15.1:0/2438234354 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e3e4d61930 0x55e3e4d6c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:43:48.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.854+0000 7f97f1e00700 1 --2- 172.21.15.1:0/2438234354 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3e4d61010 0x55e3e4d613f0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f97d40ca370 tx=0x7f97d4009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7645672f33dee3cb server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:48.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.854+0000 7f97e3000700 1 -- 172.21.15.1:0/2438234354 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d409a200 con 0x55e3e4d61010 2024-02-02T15:43:48.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.854+0000 7f97e3000700 1 -- 172.21.15.1:0/2438234354 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f97d409e030 con 0x55e3e4d61010 2024-02-02T15:43:48.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.854+0000 7f97e3000700 1 -- 172.21.15.1:0/2438234354 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d409a6c0 con 0x55e3e4d61010 2024-02-02T15:43:48.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.855+0000 7f980836fb80 1 -- 172.21.15.1:0/2438234354 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3e4d61010 msgr2=0x55e3e4d613f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:48.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.856+0000 7f980836fb80 1 --2- 172.21.15.1:0/2438234354 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3e4d61010 0x55e3e4d613f0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f97d40ca370 tx=0x7f97d4009ca0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.856+0000 7f980836fb80 1 -- 172.21.15.1:0/2438234354 shutdown_connections 2024-02-02T15:43:48.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.856+0000 7f980836fb80 1 --2- 172.21.15.1:0/2438234354 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3e4d6c7e0 0x55e3e4d6ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.856+0000 7f980836fb80 1 --2- 172.21.15.1:0/2438234354 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3e4d61010 0x55e3e4d613f0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.856+0000 7f980836fb80 1 --2- 172.21.15.1:0/2438234354 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e3e4d61930 0x55e3e4d6c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.856+0000 7f980836fb80 1 -- 172.21.15.1:0/2438234354 >> 172.21.15.1:0/2438234354 conn(0x55e3e4cc0db0 msgr2=0x55e3e4d5d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:48.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.857+0000 7f980836fb80 1 -- 172.21.15.1:0/2438234354 shutdown_connections 2024-02-02T15:43:48.859 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.859+0000 7f980836fb80 1 -- 172.21.15.1:0/2438234354 wait complete. 2024-02-02T15:43:48.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.860+0000 7f980836fb80 1 -- start start 2024-02-02T15:43:48.860 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.860+0000 7f980836fb80 1 Processor -- start 2024-02-02T15:43:48.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.861+0000 7f980836fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3e4d61010 0x55e3e4d6cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:48.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.861+0000 7f980836fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3e4d61930 0x55e3e4d6d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:48.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.862+0000 7f980836fb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e3e4d6c7e0 0x55e3e4d6d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:48.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.862+0000 7f980836fb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3e4cb2d50 con 0x55e3e4d6c7e0 2024-02-02T15:43:48.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.862+0000 7f980836fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3e4c4ed80 con 0x55e3e4d61930 2024-02-02T15:43:48.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.862+0000 7f980836fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3e4c4ea20 con 0x55e3e4d61010 2024-02-02T15:43:48.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.862+0000 7f980836fb80 10 client.0 _ll_register_callbacks cb 0x55e3e4c50290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:43:48.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.862+0000 7f97f2800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e3e4d6c7e0 0x55e3e4d6d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:48.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.862+0000 7f97f2800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e3e4d6c7e0 0x55e3e4d6d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51740/0 (socket says 172.21.15.1:51740) 2024-02-02T15:43:48.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.862+0000 7f97f2800700 1 -- 172.21.15.1:0/2426566458 learned_addr learned my addr 172.21.15.1:0/2426566458 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:43:48.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.863+0000 7f97f2800700 1 -- 172.21.15.1:0/2426566458 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3e4d61010 msgr2=0x55e3e4d6cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:48.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.863+0000 7f97f2800700 1 --2- 172.21.15.1:0/2426566458 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3e4d61010 0x55e3e4d6cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.863+0000 7f97f2800700 1 -- 172.21.15.1:0/2426566458 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3e4d61930 msgr2=0x55e3e4d6d100 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-02T15:43:48.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.863+0000 7f97f2800700 1 --2- 172.21.15.1:0/2426566458 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3e4d61930 0x55e3e4d6d100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.863+0000 7f97f2800700 1 -- 172.21.15.1:0/2426566458 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97ec1aa5e0 con 0x55e3e4d6c7e0 2024-02-02T15:43:48.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:43:48 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/3848794322' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:43:48.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.863+0000 7f97f2800700 1 --2- 172.21.15.1:0/2426566458 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e3e4d6c7e0 0x55e3e4d6d660 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f97e400be80 tx=0x7f97e400bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6b370dbe2a21d1 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:48.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.863+0000 7f97db400700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:43:48.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.864+0000 7f97db400700 1 -- 172.21.15.1:0/2426566458 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e4007b00 con 0x55e3e4d6c7e0 2024-02-02T15:43:48.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.864+0000 7f97db400700 1 -- 172.21.15.1:0/2426566458 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f97e4006e40 con 0x55e3e4d6c7e0 2024-02-02T15:43:48.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.864+0000 7f980836fb80 1 -- 172.21.15.1:0/2426566458 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e3e4d550f0 con 0x55e3e4d6c7e0 2024-02-02T15:43:48.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.865+0000 7f97db400700 1 -- 172.21.15.1:0/2426566458 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7f97e4007c60 con 0x55e3e4d6c7e0 2024-02-02T15:43:48.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.865+0000 7f980836fb80 10 client.19740 fetch_fsmap learned FSMap version 70 2024-02-02T15:43:48.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.865+0000 7f980836fb80 1 -- 172.21.15.1:0/2426566458 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) v3 -- 0x55e3e4ae0b60 con 0x55e3e4d6c7e0 2024-02-02T15:43:48.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.865+0000 7f97db400700 1 -- 172.21.15.1:0/2426566458 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e3e4d550f0 con 0x55e3e4d6c7e0 2024-02-02T15:43:48.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.866+0000 7f97db400700 1 -- 172.21.15.1:0/2426566458 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 70) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f97e4015070 con 0x55e3e4d6c7e0 2024-02-02T15:43:48.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.866+0000 7f980836fb80 10 client.19740 fetch_fsmap finished waiting for FSMap version 70 2024-02-02T15:43:48.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.866+0000 7f980836fb80 10 client.19740 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-02-02T15:43:48.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.866+0000 7f980836fb80 20 client.19740 populate_metadata read hostname 'smithi001' 2024-02-02T15:43:48.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.866+0000 7f980836fb80 1 --2- 172.21.15.1:0/2426566458 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55e3e4d7bf00 0x55e3e4d7e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:48.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.866+0000 7f980836fb80 4 client.19740 mds_command: new command op to 24518 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:43:48.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.866+0000 7f980836fb80 1 -- 172.21.15.1:0/2426566458 --> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e3e4cabdd0 con 0x55e3e4d7bf00 2024-02-02T15:43:48.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.867+0000 7f97f1e00700 1 --2- 172.21.15.1:0/2426566458 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55e3e4d7bf00 0x55e3e4d7e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:48.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.867+0000 7f97f1e00700 1 --2- 172.21.15.1:0/2426566458 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55e3e4d7bf00 0x55e3e4d7e320 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=45d5b7508cadd4 server_cookie=dd7bbccb57642900 in_seq=0 out_seq=0 2024-02-02T15:43:48.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.868+0000 7f97db400700 10 client.19740 ms_handle_connect on v2:172.21.15.71:6836/628016199 2024-02-02T15:43:48.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.870+0000 7f97db400700 1 -- 172.21.15.1:0/2426566458 <== mds.2 v2:172.21.15.71:6836/628016199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55e3e4cabdd0 con 0x55e3e4d7bf00 2024-02-02T15:43:48.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.870+0000 7f97db400700 10 client.19740 handle_command_reply: tid=0 2024-02-02T15:43:48.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.870+0000 7f980836fb80 1 client.19740 shutdown 2024-02-02T15:43:48.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.870+0000 7f980836fb80 1 -- 172.21.15.1:0/2426566458 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e3e4d6c7e0 msgr2=0x55e3e4d6d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:48.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.870+0000 7f980836fb80 1 --2- 172.21.15.1:0/2426566458 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e3e4d6c7e0 0x55e3e4d6d660 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f97e400be80 tx=0x7f97e400bf60 comp rx=0 tx=0).stop 2024-02-02T15:43:48.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.870+0000 7f980836fb80 1 -- 172.21.15.1:0/2426566458 shutdown_connections 2024-02-02T15:43:48.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.870+0000 7f980836fb80 1 --2- 172.21.15.1:0/2426566458 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e3e4d6c7e0 0x55e3e4d6d660 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.870+0000 7f980836fb80 1 --2- 172.21.15.1:0/2426566458 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55e3e4d7bf00 0x55e3e4d7e320 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.870+0000 7f980836fb80 1 --2- 172.21.15.1:0/2426566458 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3e4d61010 0x55e3e4d6cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.870+0000 7f980836fb80 1 --2- 172.21.15.1:0/2426566458 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3e4d61930 0x55e3e4d6d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.870+0000 7f980836fb80 1 -- 172.21.15.1:0/2426566458 >> 172.21.15.1:0/2426566458 conn(0x55e3e4cc0db0 msgr2=0x55e3e4d5cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:48.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.870+0000 7f97db400700 0 client.19740 ms_handle_reset on v2:172.21.15.71:6836/628016199 2024-02-02T15:43:48.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.871+0000 7f980836fb80 1 -- 172.21.15.1:0/2426566458 shutdown_connections 2024-02-02T15:43:48.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.871+0000 7f980836fb80 1 -- 172.21.15.1:0/2426566458 wait complete. 2024-02-02T15:43:48.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.871+0000 7f980836fb80 20 client.19740 trim_cache size 0 max 16384 2024-02-02T15:43:48.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.895+0000 7f980836fb80 1 Processor -- start 2024-02-02T15:43:48.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.897+0000 7f980836fb80 1 -- start start 2024-02-02T15:43:48.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.898+0000 7f980836fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3e4d61010 0x55e3e4d6b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:48.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.898+0000 7f980836fb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e3e4d61930 0x55e3e4d771e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:48.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.898+0000 7f980836fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3e4d6c7e0 0x55e3e4d68b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:48.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.898+0000 7f980836fb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3e4b366a0 con 0x55e3e4d61930 2024-02-02T15:43:48.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.898+0000 7f97f1e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3e4d61010 0x55e3e4d6b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:48.898 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.898+0000 7f97f1e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3e4d61010 0x55e3e4d6b060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42892/0 (socket says 172.21.15.1:42892) 2024-02-02T15:43:48.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.898+0000 7f97f1e00700 1 -- 172.21.15.1:0/3846397983 learned_addr learned my addr 172.21.15.1:0/3846397983 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:43:48.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.898+0000 7f97f2800700 1 --2- 172.21.15.1:0/3846397983 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3e4d6c7e0 0x55e3e4d68b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:48.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.898+0000 7f980836fb80 1 -- 172.21.15.1:0/3846397983 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3e4d674d0 con 0x55e3e4d61010 2024-02-02T15:43:48.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.898+0000 7f980836fb80 1 -- 172.21.15.1:0/3846397983 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3e4d69e00 con 0x55e3e4d6c7e0 2024-02-02T15:43:48.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.899+0000 7f97f1e00700 1 -- 172.21.15.1:0/3846397983 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3e4d6c7e0 msgr2=0x55e3e4d68b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:48.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.899+0000 7f97f1e00700 1 --2- 172.21.15.1:0/3846397983 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3e4d6c7e0 0x55e3e4d68b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.899+0000 7f97f1e00700 1 -- 172.21.15.1:0/3846397983 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e3e4d61930 msgr2=0x55e3e4d771e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:48.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.899+0000 7f97f1400700 1 --2- 172.21.15.1:0/3846397983 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e3e4d61930 0x55e3e4d771e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:48.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.900+0000 7f97f1e00700 1 --2- 172.21.15.1:0/3846397983 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e3e4d61930 0x55e3e4d771e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.900+0000 7f97f1e00700 1 -- 172.21.15.1:0/3846397983 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e3e4c78ea0 con 0x55e3e4d61010 2024-02-02T15:43:48.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.900+0000 7f97f1400700 1 --2- 172.21.15.1:0/3846397983 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e3e4d61930 0x55e3e4d771e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:43:48.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.900+0000 7f97f1e00700 1 --2- 172.21.15.1:0/3846397983 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3e4d61010 0x55e3e4d6b060 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f97d400ff10 tx=0x7f97d40a21e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2a76c5730d990c6 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:48.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.901+0000 7f97e3a00700 1 -- 172.21.15.1:0/3846397983 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d409a200 con 0x55e3e4d61010 2024-02-02T15:43:48.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.901+0000 7f97e3a00700 1 -- 172.21.15.1:0/3846397983 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f97d40a0040 con 0x55e3e4d61010 2024-02-02T15:43:48.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.901+0000 7f97e3a00700 1 -- 172.21.15.1:0/3846397983 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d409a460 con 0x55e3e4d61010 2024-02-02T15:43:48.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.902+0000 7f980836fb80 1 -- 172.21.15.1:0/3846397983 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3e4d61010 msgr2=0x55e3e4d6b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:48.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.902+0000 7f980836fb80 1 --2- 172.21.15.1:0/3846397983 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3e4d61010 0x55e3e4d6b060 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f97d400ff10 tx=0x7f97d40a21e0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.903+0000 7f980836fb80 1 -- 172.21.15.1:0/3846397983 shutdown_connections 2024-02-02T15:43:48.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.903+0000 7f980836fb80 1 --2- 172.21.15.1:0/3846397983 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3e4d6c7e0 0x55e3e4d68b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.903+0000 7f980836fb80 1 --2- 172.21.15.1:0/3846397983 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3e4d61010 0x55e3e4d6b060 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.903+0000 7f980836fb80 1 --2- 172.21.15.1:0/3846397983 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e3e4d61930 0x55e3e4d771e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.903+0000 7f980836fb80 1 -- 172.21.15.1:0/3846397983 >> 172.21.15.1:0/3846397983 conn(0x55e3e4cc0db0 msgr2=0x55e3e4d661a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:48.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.904+0000 7f980836fb80 1 -- 172.21.15.1:0/3846397983 shutdown_connections 2024-02-02T15:43:48.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.904+0000 7f980836fb80 1 -- 172.21.15.1:0/3846397983 wait complete. 2024-02-02T15:43:48.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.904+0000 7f980836fb80 1 -- start start 2024-02-02T15:43:48.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.905+0000 7f980836fb80 1 Processor -- start 2024-02-02T15:43:48.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.906+0000 7f980836fb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e3e4d61010 0x55e3e4d6ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:48.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.906+0000 7f980836fb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3e4d61930 0x55e3e4d6e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:48.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.906+0000 7f980836fb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3e4d6c7e0 0x55e3e4dcca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:48.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.906+0000 7f980836fb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3e4dccf70 con 0x55e3e4d61010 2024-02-02T15:43:48.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.906+0000 7f980836fb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3e4dcd0b0 con 0x55e3e4d6c7e0 2024-02-02T15:43:48.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.906+0000 7f980836fb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3e4dde280 con 0x55e3e4d61930 2024-02-02T15:43:48.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.907+0000 7f980836fb80 10 client.0 _ll_register_callbacks cb 0x55e3e4c50290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:43:48.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.907+0000 7f97f2800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3e4d6c7e0 0x55e3e4dcca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:48.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.907+0000 7f97f1400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3e4d61930 0x55e3e4d6e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:48.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.907+0000 7f97f2800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3e4d6c7e0 0x55e3e4dcca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42898/0 (socket says 172.21.15.1:42898) 2024-02-02T15:43:48.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.907+0000 7f97f2800700 1 -- 172.21.15.1:0/4082282896 learned_addr learned my addr 172.21.15.1:0/4082282896 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:43:48.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.907+0000 7f97f1e00700 1 --2- 172.21.15.1:0/4082282896 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e3e4d61010 0x55e3e4d6ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:48.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.907+0000 7f97f2800700 1 -- 172.21.15.1:0/4082282896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3e4d61930 msgr2=0x55e3e4d6e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:48.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.907+0000 7f97f2800700 1 --2- 172.21.15.1:0/4082282896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3e4d61930 0x55e3e4d6e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.907+0000 7f97f2800700 1 -- 172.21.15.1:0/4082282896 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e3e4d61010 msgr2=0x55e3e4d6ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:48.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.907+0000 7f97f2800700 1 --2- 172.21.15.1:0/4082282896 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e3e4d61010 0x55e3e4d6ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.907+0000 7f97f2800700 1 -- 172.21.15.1:0/4082282896 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e3e4b343b0 con 0x55e3e4d6c7e0 2024-02-02T15:43:48.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.908+0000 7f97f1e00700 1 --2- 172.21.15.1:0/4082282896 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e3e4d61010 0x55e3e4d6ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:43:48.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.908+0000 7f97f2800700 1 --2- 172.21.15.1:0/4082282896 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3e4d6c7e0 0x55e3e4dcca30 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f97e400bba0 tx=0x7f97e40070f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1f229b79b5c5d62 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:48.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.908+0000 7f97db400700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:43:48.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.909+0000 7f97db400700 1 -- 172.21.15.1:0/4082282896 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e4007e10 con 0x55e3e4d6c7e0 2024-02-02T15:43:48.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.909+0000 7f97db400700 1 -- 172.21.15.1:0/4082282896 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f97e40218f0 con 0x55e3e4d6c7e0 2024-02-02T15:43:48.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.909+0000 7f980836fb80 1 -- 172.21.15.1:0/4082282896 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f97d409e030 con 0x55e3e4d6c7e0 2024-02-02T15:43:48.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.909+0000 7f97db400700 1 -- 172.21.15.1:0/4082282896 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7f97e4021ad0 con 0x55e3e4d6c7e0 2024-02-02T15:43:48.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.909+0000 7f980836fb80 10 client.29981 fetch_fsmap learned FSMap version 70 2024-02-02T15:43:48.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.909+0000 7f980836fb80 1 -- 172.21.15.1:0/4082282896 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) v3 -- 0x55e3e4b38cd0 con 0x55e3e4d6c7e0 2024-02-02T15:43:48.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.910+0000 7f97db400700 1 -- 172.21.15.1:0/4082282896 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f97d409e030 con 0x55e3e4d6c7e0 2024-02-02T15:43:48.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.910+0000 7f97db400700 1 -- 172.21.15.1:0/4082282896 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 70) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f97e4031080 con 0x55e3e4d6c7e0 2024-02-02T15:43:48.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.910+0000 7f980836fb80 10 client.29981 fetch_fsmap finished waiting for FSMap version 70 2024-02-02T15:43:48.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.910+0000 7f980836fb80 10 client.29981 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-02-02T15:43:48.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.910+0000 7f980836fb80 20 client.29981 populate_metadata read hostname 'smithi001' 2024-02-02T15:43:48.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.910+0000 7f980836fb80 1 --2- 172.21.15.1:0/4082282896 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55e3e4ddf930 0x55e3e4de1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:48.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.910+0000 7f980836fb80 4 client.29981 mds_command: new command op to 24518 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:43:48.910 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.910+0000 7f980836fb80 1 -- 172.21.15.1:0/4082282896 --> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e3e4c71f50 con 0x55e3e4ddf930 2024-02-02T15:43:48.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.911+0000 7f97f1e00700 1 --2- 172.21.15.1:0/4082282896 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55e3e4ddf930 0x55e3e4de1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:48.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.911+0000 7f97f1e00700 1 --2- 172.21.15.1:0/4082282896 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55e3e4ddf930 0x55e3e4de1d50 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2e1b4f1bcd60e4b0 server_cookie=3125425645b82637 in_seq=0 out_seq=0 2024-02-02T15:43:48.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.912+0000 7f97db400700 10 client.29981 ms_handle_connect on v2:172.21.15.71:6836/628016199 2024-02-02T15:43:48.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.913+0000 7f97db400700 1 -- 172.21.15.1:0/4082282896 <== mds.2 v2:172.21.15.71:6836/628016199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e3e4c71f50 con 0x55e3e4ddf930 2024-02-02T15:43:48.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.913+0000 7f97db400700 10 client.29981 handle_command_reply: tid=0 2024-02-02T15:43:48.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.913+0000 7f980836fb80 1 client.29981 shutdown 2024-02-02T15:43:48.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.913+0000 7f980836fb80 1 -- 172.21.15.1:0/4082282896 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3e4d6c7e0 msgr2=0x55e3e4dcca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:48.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.913+0000 7f980836fb80 1 --2- 172.21.15.1:0/4082282896 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3e4d6c7e0 0x55e3e4dcca30 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f97e400bba0 tx=0x7f97e40070f0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.913+0000 7f980836fb80 1 -- 172.21.15.1:0/4082282896 shutdown_connections 2024-02-02T15:43:48.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.913+0000 7f980836fb80 1 --2- 172.21.15.1:0/4082282896 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e3e4d6c7e0 0x55e3e4dcca30 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.913+0000 7f980836fb80 1 --2- 172.21.15.1:0/4082282896 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e3e4d61010 0x55e3e4d6ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.913+0000 7f980836fb80 1 --2- 172.21.15.1:0/4082282896 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55e3e4ddf930 0x55e3e4de1d50 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.913+0000 7f97db400700 0 client.29981 ms_handle_reset on v2:172.21.15.71:6836/628016199 2024-02-02T15:43:48.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.913+0000 7f980836fb80 1 --2- 172.21.15.1:0/4082282896 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e3e4d61930 0x55e3e4d6e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.913+0000 7f980836fb80 1 -- 172.21.15.1:0/4082282896 >> 172.21.15.1:0/4082282896 conn(0x55e3e4cc0db0 msgr2=0x55e3e4d6d450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:48.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.914+0000 7f980836fb80 1 -- 172.21.15.1:0/4082282896 shutdown_connections 2024-02-02T15:43:48.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.914+0000 7f980836fb80 1 -- 172.21.15.1:0/4082282896 wait complete. 2024-02-02T15:43:48.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.914+0000 7f980836fb80 20 client.29981 trim_cache size 0 max 16384 2024-02-02T15:43:48.915 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:43:48.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.915+0000 7f97f4800700 1 -- 172.21.15.1:0/2634185079 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f97cc0a0710 msgr2=0x7f97cc0a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:48.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.915+0000 7f97f4800700 1 --2- 172.21.15.1:0/2634185079 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f97cc0a0710 0x7f97cc0a2bd0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f97dc005d90 tx=0x7f97dc00e2f0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.915+0000 7f97f4800700 1 -- 172.21.15.1:0/2634185079 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97ec103230 msgr2=0x7f97ec075250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:48.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.915+0000 7f97f4800700 1 --2- 172.21.15.1:0/2634185079 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97ec103230 0x7f97ec075250 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f97d40095c0 tx=0x7f97d400f740 comp rx=0 tx=0).stop 2024-02-02T15:43:48.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.915+0000 7f97f4800700 1 -- 172.21.15.1:0/2634185079 shutdown_connections 2024-02-02T15:43:48.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.915+0000 7f97f4800700 1 --2- 172.21.15.1:0/2634185079 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f97cc0a0710 0x7f97cc0a2bd0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.915+0000 7f97f4800700 1 --2- 172.21.15.1:0/2634185079 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f97ec075cd0 0x7f97ec1a96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.915+0000 7f97f4800700 1 --2- 172.21.15.1:0/2634185079 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97ec103230 0x7f97ec075250 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.915+0000 7f97f4800700 1 --2- 172.21.15.1:0/2634185079 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97ec108350 0x7f97ec075790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:48.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.915+0000 7f97f4800700 1 -- 172.21.15.1:0/2634185079 >> 172.21.15.1:0/2634185079 conn(0x7f97ec0fea70 msgr2=0x7f97ec106c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:48.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.916+0000 7f97f4800700 1 -- 172.21.15.1:0/2634185079 shutdown_connections 2024-02-02T15:43:48.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:48.916+0000 7f97f4800700 1 -- 172.21.15.1:0/2634185079 wait complete. 2024-02-02T15:43:48.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:43:48 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/3848794322' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:43:48.937 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:43:48 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/3848794322' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:43:48.941 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-02T15:43:48.991 INFO:tasks.workunit.client.0.smithi001.stdout: 10 114066 29.46 MB/sec execute 81 sec latency 62.258 ms 2024-02-02T15:43:49.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.228+0000 7fc59f400700 1 -- 172.21.15.1:0/2335500329 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc598107d40 msgr2=0x7fc5981081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:49.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.228+0000 7fc59f400700 1 --2- 172.21.15.1:0/2335500329 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc598107d40 0x7fc5981081c0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fc580009990 tx=0x7fc580009ca0 comp rx=0 tx=0).stop 2024-02-02T15:43:49.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.228+0000 7fc59f400700 1 -- 172.21.15.1:0/2335500329 shutdown_connections 2024-02-02T15:43:49.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.228+0000 7fc59f400700 1 --2- 172.21.15.1:0/2335500329 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc59806dc40 0x7fc598108700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:49.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.228+0000 7fc59f400700 1 --2- 172.21.15.1:0/2335500329 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc59810d790 0x7fc59810db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:49.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.228+0000 7fc59f400700 1 --2- 172.21.15.1:0/2335500329 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc598107d40 0x7fc5981081c0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:49.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.228+0000 7fc59f400700 1 -- 172.21.15.1:0/2335500329 >> 172.21.15.1:0/2335500329 conn(0x7fc59806d120 msgr2=0x7fc59806d530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:49.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.228+0000 7fc59f400700 1 -- 172.21.15.1:0/2335500329 shutdown_connections 2024-02-02T15:43:49.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.228+0000 7fc59f400700 1 -- 172.21.15.1:0/2335500329 wait complete. 2024-02-02T15:43:49.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.229+0000 7fc59f400700 1 Processor -- start 2024-02-02T15:43:49.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.230+0000 7fc59f400700 1 -- start start 2024-02-02T15:43:49.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.230+0000 7fc59f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc59806dc40 0x7fc5981a7650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:49.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.230+0000 7fc59f400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc598107d40 0x7fc5981a7b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:49.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.230+0000 7fc59f400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc59810d790 0x7fc5981a1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:49.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.230+0000 7fc59f400700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5981a25a0 con 0x7fc59810d790 2024-02-02T15:43:49.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.230+0000 7fc59f400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5981a2710 con 0x7fc598107d40 2024-02-02T15:43:49.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.230+0000 7fc59f400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5981a28d0 con 0x7fc59806dc40 2024-02-02T15:43:49.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.230+0000 7fc597e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc598107d40 0x7fc5981a7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:49.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.231+0000 7fc597e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc598107d40 0x7fc5981a7b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42918/0 (socket says 172.21.15.1:42918) 2024-02-02T15:43:49.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.231+0000 7fc597e00700 1 -- 172.21.15.1:0/2200307204 learned_addr learned my addr 172.21.15.1:0/2200307204 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:43:49.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.231+0000 7fc59d400700 1 --2- 172.21.15.1:0/2200307204 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc59810d790 0x7fc5981a1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:49.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.231+0000 7fc59ca00700 1 --2- 172.21.15.1:0/2200307204 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc59806dc40 0x7fc5981a7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:49.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.231+0000 7fc597e00700 1 -- 172.21.15.1:0/2200307204 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc59806dc40 msgr2=0x7fc5981a7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:49.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.231+0000 7fc597e00700 1 --2- 172.21.15.1:0/2200307204 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc59806dc40 0x7fc5981a7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:49.231 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.231+0000 7fc597e00700 1 -- 172.21.15.1:0/2200307204 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc59810d790 msgr2=0x7fc5981a1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:49.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.231+0000 7fc597e00700 1 --2- 172.21.15.1:0/2200307204 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc59810d790 0x7fc5981a1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:49.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.231+0000 7fc597e00700 1 -- 172.21.15.1:0/2200307204 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5800095f0 con 0x7fc598107d40 2024-02-02T15:43:49.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.231+0000 7fc59d400700 1 --2- 172.21.15.1:0/2200307204 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc59810d790 0x7fc5981a1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:43:49.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.231+0000 7fc59ca00700 1 --2- 172.21.15.1:0/2200307204 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc59806dc40 0x7fc5981a7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:43:49.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.232+0000 7fc597e00700 1 --2- 172.21.15.1:0/2200307204 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc598107d40 0x7fc5981a7b90 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fc580000c00 tx=0x7fc58000f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:49.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.232+0000 7fc595600700 1 -- 172.21.15.1:0/2200307204 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc58001d070 con 0x7fc598107d40 2024-02-02T15:43:49.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.232+0000 7fc59f400700 1 -- 172.21.15.1:0/2200307204 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5981a2af0 con 0x7fc598107d40 2024-02-02T15:43:49.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.232+0000 7fc595600700 1 -- 172.21.15.1:0/2200307204 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc58000be40 con 0x7fc598107d40 2024-02-02T15:43:49.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.232+0000 7fc595600700 1 -- 172.21.15.1:0/2200307204 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc580017650 con 0x7fc598107d40 2024-02-02T15:43:49.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.232+0000 7fc59f400700 1 -- 172.21.15.1:0/2200307204 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5981ae5d0 con 0x7fc598107d40 2024-02-02T15:43:49.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.237+0000 7fc595600700 1 -- 172.21.15.1:0/2200307204 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7fc580022470 con 0x7fc598107d40 2024-02-02T15:43:49.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.239+0000 7fc5b2ee2b80 1 Processor -- start 2024-02-02T15:43:49.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.240+0000 7fc5b2ee2b80 1 -- start start 2024-02-02T15:43:49.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.240+0000 7fc5b2ee2b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b3c8b12010 0x55b3c8b123f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:49.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.240+0000 7fc5b2ee2b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b3c8b12930 0x55b3c8b1d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:49.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.241+0000 7fc59d400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b3c8b12010 0x55b3c8b123f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:49.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.241+0000 7fc59d400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b3c8b12010 0x55b3c8b123f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51784/0 (socket says 172.21.15.1:51784) 2024-02-02T15:43:49.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.241+0000 7fc595600700 1 --2- 172.21.15.1:0/2200307204 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fc5780a0710 0x7fc5780a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:49.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.242+0000 7fc5b2ee2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3c8b1d7e0 0x55b3c8b1fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:49.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.242+0000 7fc5b2ee2b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3c8a864c0 con 0x55b3c8b12010 2024-02-02T15:43:49.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.242+0000 7fc5b2ee2b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3c8a6aaa0 con 0x55b3c8b12930 2024-02-02T15:43:49.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.242+0000 7fc5b2ee2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3c8a68460 con 0x55b3c8b1d7e0 2024-02-02T15:43:49.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.242+0000 7fc595600700 1 -- 172.21.15.1:0/2200307204 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fc5800cd580 con 0x7fc598107d40 2024-02-02T15:43:49.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.242+0000 7fc59d400700 1 -- 172.21.15.1:0/961353316 learned_addr learned my addr 172.21.15.1:0/961353316 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:43:49.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.242+0000 7fc59d400700 1 --2- 172.21.15.1:0/2200307204 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fc5780a0710 0x7fc5780a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:49.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.243+0000 7fc597e00700 1 --2- 172.21.15.1:0/961353316 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3c8b1d7e0 0x55b3c8b1fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:49.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.243+0000 7fc59d400700 1 -- 172.21.15.1:0/961353316 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3c8b1d7e0 msgr2=0x55b3c8b1fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:49.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.243+0000 7fc59d400700 1 --2- 172.21.15.1:0/961353316 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3c8b1d7e0 0x55b3c8b1fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:49.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.243+0000 7fc59d400700 1 -- 172.21.15.1:0/961353316 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b3c8b12930 msgr2=0x55b3c8b1d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:49.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.243+0000 7fc59d400700 1 --2- 172.21.15.1:0/961353316 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b3c8b12930 0x55b3c8b1d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:49.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.243+0000 7fc59d400700 1 -- 172.21.15.1:0/961353316 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b3c88e53b0 con 0x55b3c8b12010 2024-02-02T15:43:49.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.244+0000 7fc59d400700 1 --2- 172.21.15.1:0/961353316 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b3c8b12010 0x55b3c8b123f0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fc590009550 tx=0x7fc5900175d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16d6735e5707914 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:49.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.245+0000 7fc59d400700 1 --2- 172.21.15.1:0/2200307204 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fc5780a0710 0x7fc5780a2bd0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc59001c000 tx=0x7fc59001d000 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:49.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.245+0000 7fc58dc00700 1 -- 172.21.15.1:0/961353316 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc59001aae0 con 0x55b3c8b12010 2024-02-02T15:43:49.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.245+0000 7fc58dc00700 1 -- 172.21.15.1:0/961353316 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc59001e070 con 0x55b3c8b12010 2024-02-02T15:43:49.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.245+0000 7fc58dc00700 1 -- 172.21.15.1:0/961353316 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc590004270 con 0x55b3c8b12010 2024-02-02T15:43:49.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.252+0000 7fc5b2ee2b80 1 -- 172.21.15.1:0/961353316 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b3c8b12010 msgr2=0x55b3c8b123f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:49.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.252+0000 7fc5b2ee2b80 1 --2- 172.21.15.1:0/961353316 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b3c8b12010 0x55b3c8b123f0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fc590009550 tx=0x7fc5900175d0 comp rx=0 tx=0).stop 2024-02-02T15:43:49.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.252+0000 7fc5b2ee2b80 1 -- 172.21.15.1:0/961353316 shutdown_connections 2024-02-02T15:43:49.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.252+0000 7fc5b2ee2b80 1 --2- 172.21.15.1:0/961353316 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3c8b1d7e0 0x55b3c8b1fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:49.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.252+0000 7fc5b2ee2b80 1 --2- 172.21.15.1:0/961353316 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b3c8b12010 0x55b3c8b123f0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:49.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.252+0000 7fc5b2ee2b80 1 --2- 172.21.15.1:0/961353316 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b3c8b12930 0x55b3c8b1d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:49.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.252+0000 7fc5b2ee2b80 1 -- 172.21.15.1:0/961353316 >> 172.21.15.1:0/961353316 conn(0x55b3c8a71db0 msgr2=0x55b3c8b0e3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:49.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.252+0000 7fc5b2ee2b80 1 -- 172.21.15.1:0/961353316 shutdown_connections 2024-02-02T15:43:49.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.253+0000 7fc5b2ee2b80 1 -- 172.21.15.1:0/961353316 wait complete. 2024-02-02T15:43:49.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.253+0000 7fc5b2ee2b80 1 -- start start 2024-02-02T15:43:49.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.254+0000 7fc5b2ee2b80 1 Processor -- start 2024-02-02T15:43:49.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.254+0000 7fc5b2ee2b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b3c8b12010 0x55b3c8b1d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:49.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.254+0000 7fc5b2ee2b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b3c8b12930 0x55b3c8b1a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:49.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.254+0000 7fc5b2ee2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3c8b1d7e0 0x55b3c8b1abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:49.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.254+0000 7fc5b2ee2b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3c8a63d50 con 0x55b3c8b12930 2024-02-02T15:43:49.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.254+0000 7fc5b2ee2b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3c89ffd80 con 0x55b3c8b12010 2024-02-02T15:43:49.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.254+0000 7fc5b2ee2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3c89ffa20 con 0x55b3c8b1d7e0 2024-02-02T15:43:49.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.254+0000 7fc5b2ee2b80 10 client.0 _ll_register_callbacks cb 0x55b3c8a01290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:43:49.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.255+0000 7fc59d400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3c8b1d7e0 0x55b3c8b1abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:49.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.255+0000 7fc59d400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3c8b1d7e0 0x55b3c8b1abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:43778/0 (socket says 172.21.15.1:43778) 2024-02-02T15:43:49.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.255+0000 7fc59d400700 1 -- 172.21.15.1:0/3748140145 learned_addr learned my addr 172.21.15.1:0/3748140145 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:43:49.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.255+0000 7fc597e00700 1 --2- 172.21.15.1:0/3748140145 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b3c8b12930 0x55b3c8b1a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:49.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.256+0000 7fc59ca00700 1 --2- 172.21.15.1:0/3748140145 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b3c8b12010 0x55b3c8b1d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:49.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.256+0000 7fc59d400700 1 -- 172.21.15.1:0/3748140145 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b3c8b12010 msgr2=0x55b3c8b1d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:49.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.257+0000 7fc59d400700 1 --2- 172.21.15.1:0/3748140145 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b3c8b12010 0x55b3c8b1d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:49.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.257+0000 7fc59d400700 1 -- 172.21.15.1:0/3748140145 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b3c8b12930 msgr2=0x55b3c8b1a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:49.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.257+0000 7fc59d400700 1 --2- 172.21.15.1:0/3748140145 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b3c8b12930 0x55b3c8b1a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:49.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.257+0000 7fc59d400700 1 -- 172.21.15.1:0/3748140145 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc590007ac0 con 0x55b3c8b1d7e0 2024-02-02T15:43:49.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.258+0000 7fc59ca00700 1 --2- 172.21.15.1:0/3748140145 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b3c8b12010 0x55b3c8b1d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:43:49.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.262+0000 7fc59d400700 1 --2- 172.21.15.1:0/3748140145 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3c8b1d7e0 0x55b3c8b1abc0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fc590000f80 tx=0x7fc590009e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1af0a0546b99f336 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:49.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.262+0000 7fc586000700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:43:49.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.267+0000 7fc586000700 1 -- 172.21.15.1:0/3748140145 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc590004530 con 0x55b3c8b1d7e0 2024-02-02T15:43:49.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.267+0000 7fc586000700 1 -- 172.21.15.1:0/3748140145 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fc590004690 con 0x55b3c8b1d7e0 2024-02-02T15:43:49.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.267+0000 7fc5b2ee2b80 1 -- 172.21.15.1:0/3748140145 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b3c8b060f0 con 0x55b3c8b1d7e0 2024-02-02T15:43:49.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.275+0000 7fc586000700 1 -- 172.21.15.1:0/3748140145 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc590004850 con 0x55b3c8b1d7e0 2024-02-02T15:43:49.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.275+0000 7fc5b2ee2b80 10 client.29887 fetch_fsmap learned FSMap version 70 2024-02-02T15:43:49.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.275+0000 7fc5b2ee2b80 1 -- 172.21.15.1:0/3748140145 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) v3 -- 0x55b3c8891b60 con 0x55b3c8b1d7e0 2024-02-02T15:43:49.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.275+0000 7fc586000700 1 -- 172.21.15.1:0/3748140145 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b3c8b060f0 con 0x55b3c8b1d7e0 2024-02-02T15:43:49.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.276+0000 7fc586000700 1 -- 172.21.15.1:0/3748140145 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 70) v1 ==== 7490+0+0 (secure 0 0 0) 0x7fc590033a30 con 0x55b3c8b1d7e0 2024-02-02T15:43:49.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.276+0000 7fc5b2ee2b80 10 client.29887 fetch_fsmap finished waiting for FSMap version 70 2024-02-02T15:43:49.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.276+0000 7fc5b2ee2b80 10 client.29887 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-02T15:43:49.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.276+0000 7fc5b2ee2b80 20 client.29887 populate_metadata read hostname 'smithi001' 2024-02-02T15:43:49.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.276+0000 7fc5b2ee2b80 1 --2- 172.21.15.1:0/3748140145 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55b3c8b2d100 0x55b3c8b2f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:49.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.276+0000 7fc5b2ee2b80 4 client.29887 mds_command: new command op to 24529 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:43:49.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.276+0000 7fc5b2ee2b80 1 -- 172.21.15.1:0/3748140145 --> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b3c8a5cdd0 con 0x55b3c8b2d100 2024-02-02T15:43:49.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.277+0000 7fc59ca00700 1 --2- 172.21.15.1:0/3748140145 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55b3c8b2d100 0x55b3c8b2f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:49.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.278+0000 7fc59ca00700 1 --2- 172.21.15.1:0/3748140145 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55b3c8b2d100 0x55b3c8b2f520 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=14a3229d71dff78 server_cookie=d69b22ba6ba0e608 in_seq=0 out_seq=0 2024-02-02T15:43:49.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.278+0000 7fc586000700 10 client.29887 ms_handle_connect on v2:172.21.15.142:6838/321675421 2024-02-02T15:43:49.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.283+0000 7fc586000700 1 -- 172.21.15.1:0/3748140145 <== mds.1 v2:172.21.15.142:6838/321675421 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55b3c8a5cdd0 con 0x55b3c8b2d100 2024-02-02T15:43:49.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.284+0000 7fc586000700 10 client.29887 handle_command_reply: tid=0 2024-02-02T15:43:49.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.284+0000 7fc5b2ee2b80 1 client.29887 shutdown 2024-02-02T15:43:49.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.286+0000 7fc5b2ee2b80 1 -- 172.21.15.1:0/3748140145 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3c8b1d7e0 msgr2=0x55b3c8b1abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:49.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.286+0000 7fc5b2ee2b80 1 --2- 172.21.15.1:0/3748140145 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3c8b1d7e0 0x55b3c8b1abc0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fc590000f80 tx=0x7fc590009e50 comp rx=0 tx=0).stop 2024-02-02T15:43:49.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.287+0000 7fc5b2ee2b80 1 -- 172.21.15.1:0/3748140145 shutdown_connections 2024-02-02T15:43:49.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.287+0000 7fc5b2ee2b80 1 --2- 172.21.15.1:0/3748140145 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55b3c8b2d100 0x55b3c8b2f520 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:49.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.287+0000 7fc5b2ee2b80 1 --2- 172.21.15.1:0/3748140145 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3c8b1d7e0 0x55b3c8b1abc0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:49.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.287+0000 7fc5b2ee2b80 1 --2- 172.21.15.1:0/3748140145 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b3c8b12010 0x55b3c8b1d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:49.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.287+0000 7fc586000700 0 client.29887 ms_handle_reset on v2:172.21.15.142:6838/321675421 2024-02-02T15:43:49.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.287+0000 7fc5b2ee2b80 1 --2- 172.21.15.1:0/3748140145 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b3c8b12930 0x55b3c8b1a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:49.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.287+0000 7fc5b2ee2b80 1 -- 172.21.15.1:0/3748140145 >> 172.21.15.1:0/3748140145 conn(0x55b3c8a71db0 msgr2=0x55b3c8b0dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:49.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.287+0000 7fc5b2ee2b80 1 -- 172.21.15.1:0/3748140145 shutdown_connections 2024-02-02T15:43:49.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.287+0000 7fc5b2ee2b80 1 -- 172.21.15.1:0/3748140145 wait complete. 2024-02-02T15:43:49.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.288+0000 7fc5b2ee2b80 20 client.29887 trim_cache size 0 max 16384 2024-02-02T15:43:49.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.311+0000 7fc5b2ee2b80 1 Processor -- start 2024-02-02T15:43:49.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.311+0000 7fc5b2ee2b80 1 -- start start 2024-02-02T15:43:49.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.311+0000 7fc5b2ee2b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b3c8b12010 0x55b3c8b24330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:49.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.311+0000 7fc5b2ee2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3c8b12930 0x55b3c8b1ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:49.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.311+0000 7fc5b2ee2b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b3c8b1d7e0 0x55b3c8b29560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:49.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.312+0000 7fc5b2ee2b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3c88e76a0 con 0x55b3c8b1d7e0 2024-02-02T15:43:49.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.312+0000 7fc5b2ee2b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3c8b16340 con 0x55b3c8b12010 2024-02-02T15:43:49.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.312+0000 7fc5b2ee2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3c8b0ef80 con 0x55b3c8b12930 2024-02-02T15:43:49.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.312+0000 7fc59ca00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b3c8b12010 0x55b3c8b24330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:49.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.312+0000 7fc597e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3c8b12930 0x55b3c8b1ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:49.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.312+0000 7fc59ca00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b3c8b12010 0x55b3c8b24330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42936/0 (socket says 172.21.15.1:42936) 2024-02-02T15:43:49.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.312+0000 7fc59ca00700 1 -- 172.21.15.1:0/3351757538 learned_addr learned my addr 172.21.15.1:0/3351757538 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:43:49.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.312+0000 7fc59d400700 1 --2- 172.21.15.1:0/3351757538 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b3c8b1d7e0 0x55b3c8b29560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:49.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.312+0000 7fc59ca00700 1 -- 172.21.15.1:0/3351757538 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3c8b12930 msgr2=0x55b3c8b1ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:49.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.312+0000 7fc59ca00700 1 --2- 172.21.15.1:0/3351757538 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3c8b12930 0x55b3c8b1ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:49.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.312+0000 7fc59ca00700 1 -- 172.21.15.1:0/3351757538 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b3c8b1d7e0 msgr2=0x55b3c8b29560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:49.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.312+0000 7fc59ca00700 1 --2- 172.21.15.1:0/3351757538 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b3c8b1d7e0 0x55b3c8b29560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:49.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.312+0000 7fc59ca00700 1 -- 172.21.15.1:0/3351757538 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b3c8a29ea0 con 0x55b3c8b12010 2024-02-02T15:43:49.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.312+0000 7fc597e00700 1 --2- 172.21.15.1:0/3351757538 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3c8b12930 0x55b3c8b1ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:43:49.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.312+0000 7fc59d400700 1 --2- 172.21.15.1:0/3351757538 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b3c8b1d7e0 0x55b3c8b29560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:43:49.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.313+0000 7fc59ca00700 1 --2- 172.21.15.1:0/3351757538 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b3c8b12010 0x55b3c8b24330 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x55b3c8b2fe00 tx=0x7fc588004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6366058ff09336a server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:49.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.313+0000 7fc58e600700 1 -- 172.21.15.1:0/3351757538 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc588017040 con 0x55b3c8b12010 2024-02-02T15:43:49.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.313+0000 7fc58e600700 1 -- 172.21.15.1:0/3351757538 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc58801e070 con 0x55b3c8b12010 2024-02-02T15:43:49.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.313+0000 7fc58e600700 1 -- 172.21.15.1:0/3351757538 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc588009e50 con 0x55b3c8b12010 2024-02-02T15:43:49.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.314+0000 7fc5b2ee2b80 1 -- 172.21.15.1:0/3351757538 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b3c8b12010 msgr2=0x55b3c8b24330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:49.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.314+0000 7fc5b2ee2b80 1 --2- 172.21.15.1:0/3351757538 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b3c8b12010 0x55b3c8b24330 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x55b3c8b2fe00 tx=0x7fc588004750 comp rx=0 tx=0).stop 2024-02-02T15:43:49.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.314+0000 7fc5b2ee2b80 1 -- 172.21.15.1:0/3351757538 shutdown_connections 2024-02-02T15:43:49.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.314+0000 7fc5b2ee2b80 1 --2- 172.21.15.1:0/3351757538 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b3c8b1d7e0 0x55b3c8b29560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:49.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.314+0000 7fc5b2ee2b80 1 --2- 172.21.15.1:0/3351757538 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b3c8b12010 0x55b3c8b24330 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:49.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.314+0000 7fc5b2ee2b80 1 --2- 172.21.15.1:0/3351757538 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3c8b12930 0x55b3c8b1ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:49.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.314+0000 7fc5b2ee2b80 1 -- 172.21.15.1:0/3351757538 >> 172.21.15.1:0/3351757538 conn(0x55b3c8a71db0 msgr2=0x55b3c8b09e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:49.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.314+0000 7fc5b2ee2b80 1 -- 172.21.15.1:0/3351757538 shutdown_connections 2024-02-02T15:43:49.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.314+0000 7fc5b2ee2b80 1 -- 172.21.15.1:0/3351757538 wait complete. 2024-02-02T15:43:49.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.315+0000 7fc5b2ee2b80 1 -- start start 2024-02-02T15:43:49.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.315+0000 7fc5b2ee2b80 1 Processor -- start 2024-02-02T15:43:49.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.315+0000 7fc5b2ee2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3c8b12010 0x55b3c8b78020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:49.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.315+0000 7fc5b2ee2b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b3c8b12930 0x55b3c8b78560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:49.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.315+0000 7fc5b2ee2b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b3c8b1d7e0 0x55b3c8b22310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:49.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.315+0000 7fc5b2ee2b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3c8b22a50 con 0x55b3c8b1d7e0 2024-02-02T15:43:49.317 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.315+0000 7fc5b2ee2b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3c8b22bc0 con 0x55b3c8b12930 2024-02-02T15:43:49.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.315+0000 7fc5b2ee2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b3c8b22d80 con 0x55b3c8b12010 2024-02-02T15:43:49.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.315+0000 7fc5b2ee2b80 10 client.0 _ll_register_callbacks cb 0x55b3c8a01290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:43:49.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.316+0000 7fc597e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b3c8b12930 0x55b3c8b78560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:49.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.316+0000 7fc597e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b3c8b12930 0x55b3c8b78560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42944/0 (socket says 172.21.15.1:42944) 2024-02-02T15:43:49.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.316+0000 7fc59ca00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3c8b12010 0x55b3c8b78020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:49.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.316+0000 7fc597e00700 1 -- 172.21.15.1:0/2380405710 learned_addr learned my addr 172.21.15.1:0/2380405710 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:43:49.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.316+0000 7fc59d400700 1 --2- 172.21.15.1:0/2380405710 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b3c8b1d7e0 0x55b3c8b22310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:49.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.316+0000 7fc597e00700 1 -- 172.21.15.1:0/2380405710 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3c8b12010 msgr2=0x55b3c8b78020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:49.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.316+0000 7fc597e00700 1 --2- 172.21.15.1:0/2380405710 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3c8b12010 0x55b3c8b78020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:49.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.316+0000 7fc597e00700 1 -- 172.21.15.1:0/2380405710 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b3c8b1d7e0 msgr2=0x55b3c8b22310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:49.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.316+0000 7fc597e00700 1 --2- 172.21.15.1:0/2380405710 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b3c8b1d7e0 0x55b3c8b22310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:49.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.316+0000 7fc597e00700 1 -- 172.21.15.1:0/2380405710 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5981ae5d0 con 0x55b3c8b12930 2024-02-02T15:43:49.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.316+0000 7fc597e00700 1 --2- 172.21.15.1:0/2380405710 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b3c8b12930 0x55b3c8b78560 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fc58009a6c0 tx=0x7fc5800cae30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb58c0e83261dc22 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:49.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.316+0000 7fc586000700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:43:49.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.316+0000 7fc586000700 1 -- 172.21.15.1:0/2380405710 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc58009ab90 con 0x55b3c8b12930 2024-02-02T15:43:49.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.317+0000 7fc586000700 1 -- 172.21.15.1:0/2380405710 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fc5800b2e40 con 0x55b3c8b12930 2024-02-02T15:43:49.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.317+0000 7fc5b2ee2b80 1 -- 172.21.15.1:0/2380405710 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc59001e070 con 0x55b3c8b12930 2024-02-02T15:43:49.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.317+0000 7fc586000700 1 -- 172.21.15.1:0/2380405710 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc58009e380 con 0x55b3c8b12930 2024-02-02T15:43:49.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.317+0000 7fc5b2ee2b80 10 client.29993 fetch_fsmap learned FSMap version 70 2024-02-02T15:43:49.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.317+0000 7fc5b2ee2b80 1 -- 172.21.15.1:0/2380405710 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) v3 -- 0x7fc588003810 con 0x55b3c8b12930 2024-02-02T15:43:49.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.317+0000 7fc586000700 1 -- 172.21.15.1:0/2380405710 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc59001e070 con 0x55b3c8b12930 2024-02-02T15:43:49.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.317+0000 7fc586000700 1 -- 172.21.15.1:0/2380405710 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 70) v1 ==== 7490+0+0 (secure 0 0 0) 0x7fc580099c20 con 0x55b3c8b12930 2024-02-02T15:43:49.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.317+0000 7fc5b2ee2b80 10 client.29993 fetch_fsmap finished waiting for FSMap version 70 2024-02-02T15:43:49.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.317+0000 7fc5b2ee2b80 10 client.29993 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-02T15:43:49.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.317+0000 7fc5b2ee2b80 20 client.29993 populate_metadata read hostname 'smithi001' 2024-02-02T15:43:49.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.317+0000 7fc5b2ee2b80 1 --2- 172.21.15.1:0/2380405710 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55b3c8b7f9d0 0x55b3c8b81df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:49.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.317+0000 7fc5b2ee2b80 4 client.29993 mds_command: new command op to 24529 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:43:49.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.317+0000 7fc5b2ee2b80 1 -- 172.21.15.1:0/2380405710 --> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b3c8a22f50 con 0x55b3c8b7f9d0 2024-02-02T15:43:49.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.318+0000 7fc59ca00700 1 --2- 172.21.15.1:0/2380405710 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55b3c8b7f9d0 0x55b3c8b81df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:49.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.319+0000 7fc59ca00700 1 --2- 172.21.15.1:0/2380405710 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55b3c8b7f9d0 0x55b3c8b81df0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ca6ec494b2a0308f server_cookie=6de5ebf8ec3859f3 in_seq=0 out_seq=0 2024-02-02T15:43:49.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.319+0000 7fc586000700 10 client.29993 ms_handle_connect on v2:172.21.15.142:6838/321675421 2024-02-02T15:43:49.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.320+0000 7fc586000700 1 -- 172.21.15.1:0/2380405710 <== mds.1 v2:172.21.15.142:6838/321675421 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b3c8a22f50 con 0x55b3c8b7f9d0 2024-02-02T15:43:49.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.320+0000 7fc586000700 10 client.29993 handle_command_reply: tid=0 2024-02-02T15:43:49.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.320+0000 7fc5b2ee2b80 1 client.29993 shutdown 2024-02-02T15:43:49.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.320+0000 7fc5b2ee2b80 1 -- 172.21.15.1:0/2380405710 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b3c8b12930 msgr2=0x55b3c8b78560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:49.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.320+0000 7fc5b2ee2b80 1 --2- 172.21.15.1:0/2380405710 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b3c8b12930 0x55b3c8b78560 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fc58009a6c0 tx=0x7fc5800cae30 comp rx=0 tx=0).stop 2024-02-02T15:43:49.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.320+0000 7fc5b2ee2b80 1 -- 172.21.15.1:0/2380405710 shutdown_connections 2024-02-02T15:43:49.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.320+0000 7fc5b2ee2b80 1 --2- 172.21.15.1:0/2380405710 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55b3c8b7f9d0 0x55b3c8b81df0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:49.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.320+0000 7fc5b2ee2b80 1 --2- 172.21.15.1:0/2380405710 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b3c8b1d7e0 0x55b3c8b22310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:49.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.320+0000 7fc5b2ee2b80 1 --2- 172.21.15.1:0/2380405710 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b3c8b12010 0x55b3c8b78020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:49.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.320+0000 7fc5b2ee2b80 1 --2- 172.21.15.1:0/2380405710 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b3c8b12930 0x55b3c8b78560 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:49.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.320+0000 7fc586000700 0 client.29993 ms_handle_reset on v2:172.21.15.142:6838/321675421 2024-02-02T15:43:49.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.320+0000 7fc5b2ee2b80 1 -- 172.21.15.1:0/2380405710 >> 172.21.15.1:0/2380405710 conn(0x55b3c8a71db0 msgr2=0x55b3c8b29560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:49.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.320+0000 7fc5b2ee2b80 1 -- 172.21.15.1:0/2380405710 shutdown_connections 2024-02-02T15:43:49.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.321+0000 7fc5b2ee2b80 1 -- 172.21.15.1:0/2380405710 wait complete. 2024-02-02T15:43:49.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.321+0000 7fc5b2ee2b80 20 client.29993 trim_cache size 0 max 16384 2024-02-02T15:43:49.322 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:43:49.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.321+0000 7fc59f400700 1 -- 172.21.15.1:0/2200307204 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fc5780a0710 msgr2=0x7fc5780a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:49.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.321+0000 7fc59f400700 1 --2- 172.21.15.1:0/2200307204 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fc5780a0710 0x7fc5780a2bd0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc59001c000 tx=0x7fc59001d000 comp rx=0 tx=0).stop 2024-02-02T15:43:49.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.321+0000 7fc59f400700 1 -- 172.21.15.1:0/2200307204 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc598107d40 msgr2=0x7fc5981a7b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:49.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.321+0000 7fc59f400700 1 --2- 172.21.15.1:0/2200307204 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc598107d40 0x7fc5981a7b90 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fc580000c00 tx=0x7fc58000f690 comp rx=0 tx=0).stop 2024-02-02T15:43:49.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.322+0000 7fc59f400700 1 -- 172.21.15.1:0/2200307204 shutdown_connections 2024-02-02T15:43:49.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.322+0000 7fc59f400700 1 --2- 172.21.15.1:0/2200307204 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fc5780a0710 0x7fc5780a2bd0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:49.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.322+0000 7fc59f400700 1 --2- 172.21.15.1:0/2200307204 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fc59810d790 0x7fc5981a1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:49.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.322+0000 7fc59f400700 1 --2- 172.21.15.1:0/2200307204 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fc59806dc40 0x7fc5981a7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:49.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.322+0000 7fc59f400700 1 --2- 172.21.15.1:0/2200307204 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc598107d40 0x7fc5981a7b90 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:49.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.322+0000 7fc59f400700 1 -- 172.21.15.1:0/2200307204 >> 172.21.15.1:0/2200307204 conn(0x7fc59806d120 msgr2=0x7fc5980716a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:49.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.322+0000 7fc59f400700 1 -- 172.21.15.1:0/2200307204 shutdown_connections 2024-02-02T15:43:49.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.322+0000 7fc59f400700 1 -- 172.21.15.1:0/2200307204 wait complete. 2024-02-02T15:43:49.348 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-02T15:43:49.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:43:49 smithi001 ceph-mon[35470]: pgmap v527: 129 pgs: 129 active+clean; 1.7 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 8.4 MiB/s wr, 568 op/s 2024-02-02T15:43:49.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.717+0000 7f8700400700 1 -- 172.21.15.1:0/260854480 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86f81083b0 msgr2=0x7f86f810ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:49.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.717+0000 7f8700400700 1 --2- 172.21.15.1:0/260854480 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86f81083b0 0x7f86f810ff30 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f86f000b0a0 tx=0x7f86f000b3b0 comp rx=0 tx=0).stop 2024-02-02T15:43:49.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.718+0000 7f8700400700 1 -- 172.21.15.1:0/260854480 shutdown_connections 2024-02-02T15:43:49.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.718+0000 7f8700400700 1 --2- 172.21.15.1:0/260854480 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86f81083b0 0x7f86f810ff30 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:49.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.718+0000 7f8700400700 1 --2- 172.21.15.1:0/260854480 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86f8103290 0x7f86f8103670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:49.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.718+0000 7f8700400700 1 --2- 172.21.15.1:0/260854480 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f86f8103c40 0x7f86f8107c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:49.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.718+0000 7f8700400700 1 -- 172.21.15.1:0/260854480 >> 172.21.15.1:0/260854480 conn(0x7f86f80feb30 msgr2=0x7f86f8100f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:49.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.719+0000 7f8700400700 1 -- 172.21.15.1:0/260854480 shutdown_connections 2024-02-02T15:43:49.719 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.719+0000 7f8700400700 1 -- 172.21.15.1:0/260854480 wait complete. 2024-02-02T15:43:49.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.720+0000 7f8700400700 1 Processor -- start 2024-02-02T15:43:49.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.721+0000 7f8700400700 1 -- start start 2024-02-02T15:43:49.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.721+0000 7f8700400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f86f8103290 0x7f86f8075250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:49.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.721+0000 7f8700400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86f81083b0 0x7f86f8075790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:49.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.721+0000 7f8700400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86f8075cd0 0x7f86f80710e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:49.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.721+0000 7f8700400700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f8071620 con 0x7f86f8103290 2024-02-02T15:43:49.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.721+0000 7f8700400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f8071790 con 0x7f86f8075cd0 2024-02-02T15:43:49.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.721+0000 7f8700400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f8071950 con 0x7f86f81083b0 2024-02-02T15:43:49.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.722+0000 7f86fd000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86f81083b0 0x7f86f8075790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:49.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.722+0000 7f86fd000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86f81083b0 0x7f86f8075790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:43820/0 (socket says 172.21.15.1:43820) 2024-02-02T15:43:49.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.722+0000 7f86fd000700 1 -- 172.21.15.1:0/1826144137 learned_addr learned my addr 172.21.15.1:0/1826144137 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:43:49.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.722+0000 7f86fe400700 1 --2- 172.21.15.1:0/1826144137 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86f8075cd0 0x7f86f80710e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:49.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.723+0000 7f86fda00700 1 --2- 172.21.15.1:0/1826144137 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f86f8103290 0x7f86f8075250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:49.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.723+0000 7f86fe400700 1 -- 172.21.15.1:0/1826144137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86f81083b0 msgr2=0x7f86f8075790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:49.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.723+0000 7f86fe400700 1 --2- 172.21.15.1:0/1826144137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86f81083b0 0x7f86f8075790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:49.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.723+0000 7f86fe400700 1 -- 172.21.15.1:0/1826144137 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f86f8103290 msgr2=0x7f86f8075250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:49.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.723+0000 7f86fe400700 1 --2- 172.21.15.1:0/1826144137 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f86f8103290 0x7f86f8075250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:49.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.723+0000 7f86fe400700 1 -- 172.21.15.1:0/1826144137 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86f0009d00 con 0x7f86f8075cd0 2024-02-02T15:43:49.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.724+0000 7f86fe400700 1 --2- 172.21.15.1:0/1826144137 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86f8075cd0 0x7f86f80710e0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f86f0009fd0 tx=0x7f86f000bc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:49.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.724+0000 7f86f6000700 1 -- 172.21.15.1:0/1826144137 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f000e040 con 0x7f86f8075cd0 2024-02-02T15:43:49.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.724+0000 7f86f6000700 1 -- 172.21.15.1:0/1826144137 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86f00094a0 con 0x7f86f8075cd0 2024-02-02T15:43:49.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.724+0000 7f86f6000700 1 -- 172.21.15.1:0/1826144137 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f00052a0 con 0x7f86f8075cd0 2024-02-02T15:43:49.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.724+0000 7f8700400700 1 -- 172.21.15.1:0/1826144137 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86f8071bd0 con 0x7f86f8075cd0 2024-02-02T15:43:49.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.725+0000 7f8700400700 1 -- 172.21.15.1:0/1826144137 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86f8072000 con 0x7f86f8075cd0 2024-02-02T15:43:49.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.730+0000 7f86f6000700 1 -- 172.21.15.1:0/1826144137 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f86f001d480 con 0x7f86f8075cd0 2024-02-02T15:43:49.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.734+0000 7f86f6000700 1 --2- 172.21.15.1:0/1826144137 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f86d80a0790 0x7f86d80a2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:49.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.734+0000 7f8713e93b80 1 Processor -- start 2024-02-02T15:43:49.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.734+0000 7f8713e93b80 1 -- start start 2024-02-02T15:43:49.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.735+0000 7f86f6000700 1 -- 172.21.15.1:0/1826144137 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f86f00cd170 con 0x7f86f8075cd0 2024-02-02T15:43:49.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.735+0000 7f86fda00700 1 --2- 172.21.15.1:0/1826144137 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f86d80a0790 0x7f86d80a2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:49.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.735+0000 7f8713e93b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b4d4750010 0x55b4d47503f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:49.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.735+0000 7f8713e93b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b4d4750930 0x55b4d475b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:49.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.735+0000 7f8713e93b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b4d475b7e0 0x55b4d475dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:49.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.735+0000 7f8713e93b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4d46c44c0 con 0x55b4d4750930 2024-02-02T15:43:49.735 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.735+0000 7f8713e93b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4d46a8aa0 con 0x55b4d4750010 2024-02-02T15:43:49.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.735+0000 7f8713e93b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4d46a6460 con 0x55b4d475b7e0 2024-02-02T15:43:49.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.736+0000 7f86fd000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b4d4750930 0x55b4d475b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:49.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.736+0000 7f86fd000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b4d4750930 0x55b4d475b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51852/0 (socket says 172.21.15.1:51852) 2024-02-02T15:43:49.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.736+0000 7f86fd000700 1 -- 172.21.15.1:0/992052066 learned_addr learned my addr 172.21.15.1:0/992052066 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:43:49.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.737+0000 7f86fe400700 1 --2- 172.21.15.1:0/992052066 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b4d475b7e0 0x55b4d475dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:49.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.737+0000 7f86fd000700 1 -- 172.21.15.1:0/992052066 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b4d475b7e0 msgr2=0x55b4d475dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:49.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.737+0000 7f86fd000700 1 --2- 172.21.15.1:0/992052066 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b4d475b7e0 0x55b4d475dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:49.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.738+0000 7f86fd000700 1 -- 172.21.15.1:0/992052066 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b4d4750010 msgr2=0x55b4d47503f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-02T15:43:49.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.738+0000 7f86fd000700 1 --2- 172.21.15.1:0/992052066 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b4d4750010 0x55b4d47503f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:49.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.738+0000 7f86fd000700 1 -- 172.21.15.1:0/992052066 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b4d45233b0 con 0x55b4d4750930 2024-02-02T15:43:49.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.738+0000 7f86fda00700 1 --2- 172.21.15.1:0/1826144137 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f86d80a0790 0x7f86d80a2c50 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f86e8005950 tx=0x7f86e80058e0 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:49.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.738+0000 7f86fd000700 1 --2- 172.21.15.1:0/992052066 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b4d4750930 0x55b4d475b2a0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f86e0009cd0 tx=0x7f86e0011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=447c6278f259aed3 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:49.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.739+0000 7f86eea00700 1 -- 172.21.15.1:0/992052066 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e0011d00 con 0x55b4d4750930 2024-02-02T15:43:49.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.739+0000 7f86eea00700 1 -- 172.21.15.1:0/992052066 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86e0011e60 con 0x55b4d4750930 2024-02-02T15:43:49.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.739+0000 7f86eea00700 1 -- 172.21.15.1:0/992052066 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e00193e0 con 0x55b4d4750930 2024-02-02T15:43:49.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.740+0000 7f8713e93b80 1 -- 172.21.15.1:0/992052066 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b4d4750930 msgr2=0x55b4d475b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:49.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.740+0000 7f8713e93b80 1 --2- 172.21.15.1:0/992052066 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b4d4750930 0x55b4d475b2a0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f86e0009cd0 tx=0x7f86e0011440 comp rx=0 tx=0).stop 2024-02-02T15:43:49.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.740+0000 7f8713e93b80 1 -- 172.21.15.1:0/992052066 shutdown_connections 2024-02-02T15:43:49.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.740+0000 7f8713e93b80 1 --2- 172.21.15.1:0/992052066 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b4d475b7e0 0x55b4d475dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:49.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.740+0000 7f8713e93b80 1 --2- 172.21.15.1:0/992052066 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b4d4750010 0x55b4d47503f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:49.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.740+0000 7f8713e93b80 1 --2- 172.21.15.1:0/992052066 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b4d4750930 0x55b4d475b2a0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:49.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.740+0000 7f8713e93b80 1 -- 172.21.15.1:0/992052066 >> 172.21.15.1:0/992052066 conn(0x55b4d46afdb0 msgr2=0x55b4d474c3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:49.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.741+0000 7f8713e93b80 1 -- 172.21.15.1:0/992052066 shutdown_connections 2024-02-02T15:43:49.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.741+0000 7f8713e93b80 1 -- 172.21.15.1:0/992052066 wait complete. 2024-02-02T15:43:49.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.743+0000 7f8713e93b80 1 -- start start 2024-02-02T15:43:49.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.743+0000 7f8713e93b80 1 Processor -- start 2024-02-02T15:43:49.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.744+0000 7f8713e93b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b4d4750010 0x55b4d47584f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:49.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.744+0000 7f8713e93b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b4d4750930 0x55b4d4758a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:49.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.744+0000 7f8713e93b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b4d475b7e0 0x55b4d4758f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:49.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.744+0000 7f8713e93b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4d46a1d50 con 0x55b4d4750010 2024-02-02T15:43:49.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.744+0000 7f8713e93b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4d463dd80 con 0x55b4d475b7e0 2024-02-02T15:43:49.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.744+0000 7f8713e93b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4d463da20 con 0x55b4d4750930 2024-02-02T15:43:49.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.745+0000 7f86fda00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b4d4750010 0x55b4d47584f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:49.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.745+0000 7f86fda00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b4d4750010 0x55b4d47584f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51864/0 (socket says 172.21.15.1:51864) 2024-02-02T15:43:49.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.745+0000 7f86fe400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b4d475b7e0 0x55b4d4758f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:49.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.745+0000 7f86fda00700 1 -- 172.21.15.1:0/3023925734 learned_addr learned my addr 172.21.15.1:0/3023925734 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:43:49.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.745+0000 7f86fd000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b4d4750930 0x55b4d4758a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:49.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.746+0000 7f8713e93b80 10 client.0 _ll_register_callbacks cb 0x55b4d463f290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:43:49.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.747+0000 7f86fe400700 1 -- 172.21.15.1:0/3023925734 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b4d4750930 msgr2=0x55b4d4758a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:49.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.747+0000 7f86fe400700 1 --2- 172.21.15.1:0/3023925734 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b4d4750930 0x55b4d4758a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:49.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.747+0000 7f86fe400700 1 -- 172.21.15.1:0/3023925734 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b4d4750010 msgr2=0x55b4d47584f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:49.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.747+0000 7f86fe400700 1 --2- 172.21.15.1:0/3023925734 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b4d4750010 0x55b4d47584f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:49.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.747+0000 7f86fe400700 1 -- 172.21.15.1:0/3023925734 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86e0009930 con 0x55b4d475b7e0 2024-02-02T15:43:49.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.748+0000 7f86fda00700 1 --2- 172.21.15.1:0/3023925734 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b4d4750010 0x55b4d47584f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:43:49.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.748+0000 7f86fe400700 1 --2- 172.21.15.1:0/3023925734 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b4d475b7e0 0x55b4d4758f70 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f86f00ca4c0 tx=0x7f86f001dc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3f32e3694f498bd server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:49.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.748+0000 7f86e7400700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:43:49.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.748+0000 7f86e7400700 1 -- 172.21.15.1:0/3023925734 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f0099100 con 0x55b4d475b7e0 2024-02-02T15:43:49.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.748+0000 7f86e7400700 1 -- 172.21.15.1:0/3023925734 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f86f009aa20 con 0x55b4d475b7e0 2024-02-02T15:43:49.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.748+0000 7f8713e93b80 1 -- 172.21.15.1:0/3023925734 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b4d47440f0 con 0x55b4d475b7e0 2024-02-02T15:43:49.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.748+0000 7f86e7400700 1 -- 172.21.15.1:0/3023925734 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86f0098b70 con 0x55b4d475b7e0 2024-02-02T15:43:49.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.748+0000 7f8713e93b80 10 client.30002 fetch_fsmap learned FSMap version 70 2024-02-02T15:43:49.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.748+0000 7f8713e93b80 1 -- 172.21.15.1:0/3023925734 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) v3 -- 0x55b4d44cfb60 con 0x55b4d475b7e0 2024-02-02T15:43:49.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.749+0000 7f86e7400700 1 -- 172.21.15.1:0/3023925734 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b4d47440f0 con 0x55b4d475b7e0 2024-02-02T15:43:49.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.749+0000 7f86e7400700 1 -- 172.21.15.1:0/3023925734 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 70) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f86f0024310 con 0x55b4d475b7e0 2024-02-02T15:43:49.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.749+0000 7f8713e93b80 10 client.30002 fetch_fsmap finished waiting for FSMap version 70 2024-02-02T15:43:49.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.749+0000 7f8713e93b80 10 client.30002 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:43:49.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.749+0000 7f8713e93b80 20 client.30002 populate_metadata read hostname 'smithi001' 2024-02-02T15:43:49.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.749+0000 7f8713e93b80 1 --2- 172.21.15.1:0/3023925734 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55b4d477c080 0x55b4d477e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:49.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.749+0000 7f8713e93b80 4 client.30002 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:43:49.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.749+0000 7f8713e93b80 1 -- 172.21.15.1:0/3023925734 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b4d469add0 con 0x55b4d477c080 2024-02-02T15:43:49.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.750+0000 7f86fda00700 1 --2- 172.21.15.1:0/3023925734 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55b4d477c080 0x55b4d477e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:49.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.751+0000 7f86fda00700 1 --2- 172.21.15.1:0/3023925734 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55b4d477c080 0x55b4d477e4a0 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2fe141051b630edb server_cookie=d04b552a425ec3ef in_seq=0 out_seq=0 2024-02-02T15:43:49.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.751+0000 7f86e7400700 10 client.30002 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:43:49.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.752+0000 7f86e7400700 1 -- 172.21.15.1:0/3023925734 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55b4d469add0 con 0x55b4d477c080 2024-02-02T15:43:49.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.752+0000 7f86e7400700 10 client.30002 handle_command_reply: tid=0 2024-02-02T15:43:49.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.752+0000 7f8713e93b80 1 client.30002 shutdown 2024-02-02T15:43:49.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.752+0000 7f8713e93b80 1 -- 172.21.15.1:0/3023925734 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b4d475b7e0 msgr2=0x55b4d4758f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:49.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.752+0000 7f8713e93b80 1 --2- 172.21.15.1:0/3023925734 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b4d475b7e0 0x55b4d4758f70 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f86f00ca4c0 tx=0x7f86f001dc00 comp rx=0 tx=0).stop 2024-02-02T15:43:49.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.752+0000 7f8713e93b80 1 -- 172.21.15.1:0/3023925734 shutdown_connections 2024-02-02T15:43:49.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.752+0000 7f8713e93b80 1 --2- 172.21.15.1:0/3023925734 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55b4d477c080 0x55b4d477e4a0 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:49.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.752+0000 7f86e7400700 0 client.30002 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:43:49.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.752+0000 7f8713e93b80 1 --2- 172.21.15.1:0/3023925734 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b4d475b7e0 0x55b4d4758f70 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:49.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.752+0000 7f8713e93b80 1 --2- 172.21.15.1:0/3023925734 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b4d4750010 0x55b4d47584f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:49.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.752+0000 7f8713e93b80 1 --2- 172.21.15.1:0/3023925734 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b4d4750930 0x55b4d4758a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:49.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.752+0000 7f8713e93b80 1 -- 172.21.15.1:0/3023925734 >> 172.21.15.1:0/3023925734 conn(0x55b4d46afdb0 msgr2=0x55b4d474bfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:49.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.753+0000 7f8713e93b80 1 -- 172.21.15.1:0/3023925734 shutdown_connections 2024-02-02T15:43:49.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.753+0000 7f8713e93b80 1 -- 172.21.15.1:0/3023925734 wait complete. 2024-02-02T15:43:49.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.753+0000 7f8713e93b80 20 client.30002 trim_cache size 0 max 16384 2024-02-02T15:43:49.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.771+0000 7f8713e93b80 1 Processor -- start 2024-02-02T15:43:49.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.771+0000 7f8713e93b80 1 -- start start 2024-02-02T15:43:49.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.772+0000 7f8713e93b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b4d4750010 0x55b4d4757b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:49.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.772+0000 7f8713e93b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b4d4750930 0x55b4d47538e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:49.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.772+0000 7f8713e93b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b4d475b7e0 0x55b4d4753e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:49.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.772+0000 7f8713e93b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4d45256a0 con 0x55b4d4750010 2024-02-02T15:43:49.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.772+0000 7f8713e93b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4d469add0 con 0x55b4d475b7e0 2024-02-02T15:43:49.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.772+0000 7f8713e93b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4d477bec0 con 0x55b4d4750930 2024-02-02T15:43:49.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.772+0000 7f86fd000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b4d4750930 0x55b4d47538e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:49.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.772+0000 7f86fd000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b4d4750930 0x55b4d47538e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:43854/0 (socket says 172.21.15.1:43854) 2024-02-02T15:43:49.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.772+0000 7f86fd000700 1 -- 172.21.15.1:0/4245294112 learned_addr learned my addr 172.21.15.1:0/4245294112 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:43:49.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.772+0000 7f86fe400700 1 --2- 172.21.15.1:0/4245294112 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b4d475b7e0 0x55b4d4753e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:49.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.772+0000 7f86fda00700 1 --2- 172.21.15.1:0/4245294112 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b4d4750010 0x55b4d4757b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:49.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.773+0000 7f86fd000700 1 -- 172.21.15.1:0/4245294112 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b4d475b7e0 msgr2=0x55b4d4753e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:49.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.773+0000 7f86fd000700 1 --2- 172.21.15.1:0/4245294112 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b4d475b7e0 0x55b4d4753e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:49.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.773+0000 7f86fd000700 1 -- 172.21.15.1:0/4245294112 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b4d4750010 msgr2=0x55b4d4757b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:49.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.773+0000 7f86fd000700 1 --2- 172.21.15.1:0/4245294112 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b4d4750010 0x55b4d4757b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:49.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.773+0000 7f86fd000700 1 -- 172.21.15.1:0/4245294112 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b4d4667ea0 con 0x55b4d4750930 2024-02-02T15:43:49.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.773+0000 7f86fe400700 1 --2- 172.21.15.1:0/4245294112 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b4d475b7e0 0x55b4d4753e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:43:49.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.774+0000 7f86fd000700 1 --2- 172.21.15.1:0/4245294112 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b4d4750930 0x55b4d47538e0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f86e0009cd0 tx=0x7f86e0003730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=602935be626b6dcd server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:49.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.774+0000 7f86ef400700 1 -- 172.21.15.1:0/4245294112 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e0011b00 con 0x55b4d4750930 2024-02-02T15:43:49.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.774+0000 7f86ef400700 1 -- 172.21.15.1:0/4245294112 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86e0003e00 con 0x55b4d4750930 2024-02-02T15:43:49.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.774+0000 7f86ef400700 1 -- 172.21.15.1:0/4245294112 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e0024460 con 0x55b4d4750930 2024-02-02T15:43:49.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.775+0000 7f8713e93b80 1 -- 172.21.15.1:0/4245294112 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b4d4750930 msgr2=0x55b4d47538e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:49.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.775+0000 7f8713e93b80 1 --2- 172.21.15.1:0/4245294112 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b4d4750930 0x55b4d47538e0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f86e0009cd0 tx=0x7f86e0003730 comp rx=0 tx=0).stop 2024-02-02T15:43:49.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.775+0000 7f8713e93b80 1 -- 172.21.15.1:0/4245294112 shutdown_connections 2024-02-02T15:43:49.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.775+0000 7f8713e93b80 1 --2- 172.21.15.1:0/4245294112 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b4d475b7e0 0x55b4d4753e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:49.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.775+0000 7f8713e93b80 1 --2- 172.21.15.1:0/4245294112 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b4d4750010 0x55b4d4757b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:49.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.775+0000 7f8713e93b80 1 --2- 172.21.15.1:0/4245294112 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b4d4750930 0x55b4d47538e0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:49.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.775+0000 7f8713e93b80 1 -- 172.21.15.1:0/4245294112 >> 172.21.15.1:0/4245294112 conn(0x55b4d46afdb0 msgr2=0x55b4d4766940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:49.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.775+0000 7f8713e93b80 1 -- 172.21.15.1:0/4245294112 shutdown_connections 2024-02-02T15:43:49.776 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.775+0000 7f8713e93b80 1 -- 172.21.15.1:0/4245294112 wait complete. 2024-02-02T15:43:49.777 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.776+0000 7f8713e93b80 1 -- start start 2024-02-02T15:43:49.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.776+0000 7f8713e93b80 1 Processor -- start 2024-02-02T15:43:49.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.776+0000 7f8713e93b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b4d4750010 0x55b4d475fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:49.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.776+0000 7f8713e93b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b4d4750930 0x55b4d47603d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:49.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.776+0000 7f8713e93b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b4d475b7e0 0x55b4d47ccb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:49.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.777+0000 7f8713e93b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4d4760b00 con 0x55b4d475b7e0 2024-02-02T15:43:49.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.777+0000 7f8713e93b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4d47cd0a0 con 0x55b4d4750010 2024-02-02T15:43:49.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.777+0000 7f8713e93b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4d47cd230 con 0x55b4d4750930 2024-02-02T15:43:49.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.777+0000 7f8713e93b80 10 client.0 _ll_register_callbacks cb 0x55b4d463f290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:43:49.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.777+0000 7f86fda00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b4d4750010 0x55b4d475fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:49.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.777+0000 7f86fda00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b4d4750010 0x55b4d475fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42998/0 (socket says 172.21.15.1:42998) 2024-02-02T15:43:49.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.777+0000 7f86fda00700 1 -- 172.21.15.1:0/93676482 learned_addr learned my addr 172.21.15.1:0/93676482 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:43:49.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.777+0000 7f86fd000700 1 --2- 172.21.15.1:0/93676482 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b4d4750930 0x55b4d47603d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:49.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.777+0000 7f86fe400700 1 --2- 172.21.15.1:0/93676482 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b4d475b7e0 0x55b4d47ccb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:49.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.778+0000 7f86fd000700 1 -- 172.21.15.1:0/93676482 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b4d4750010 msgr2=0x55b4d475fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:49.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.778+0000 7f86fd000700 1 --2- 172.21.15.1:0/93676482 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b4d4750010 0x55b4d475fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:49.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.778+0000 7f86fd000700 1 -- 172.21.15.1:0/93676482 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b4d475b7e0 msgr2=0x55b4d47ccb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:49.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.778+0000 7f86fd000700 1 --2- 172.21.15.1:0/93676482 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b4d475b7e0 0x55b4d47ccb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:49.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.778+0000 7f86fd000700 1 -- 172.21.15.1:0/93676482 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b4d44cfb60 con 0x55b4d4750930 2024-02-02T15:43:49.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.778+0000 7f86fe400700 1 --2- 172.21.15.1:0/93676482 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b4d475b7e0 0x55b4d47ccb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:43:49.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.778+0000 7f86fd000700 1 --2- 172.21.15.1:0/93676482 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b4d4750930 0x55b4d47603d0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f86e0011790 tx=0x7f86e0005640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd89156a2f992404 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:49.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.778+0000 7f86e7400700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:43:49.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.779+0000 7f86e7400700 1 -- 172.21.15.1:0/93676482 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e0034070 con 0x55b4d4750930 2024-02-02T15:43:49.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.779+0000 7f86e7400700 1 -- 172.21.15.1:0/93676482 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f86e00038a0 con 0x55b4d4750930 2024-02-02T15:43:49.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.779+0000 7f8713e93b80 1 -- 172.21.15.1:0/93676482 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f86e0011e60 con 0x55b4d4750930 2024-02-02T15:43:49.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.779+0000 7f86e7400700 1 -- 172.21.15.1:0/93676482 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7f86e00052a0 con 0x55b4d4750930 2024-02-02T15:43:49.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.779+0000 7f8713e93b80 10 client.29911 fetch_fsmap learned FSMap version 70 2024-02-02T15:43:49.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.779+0000 7f8713e93b80 1 -- 172.21.15.1:0/93676482 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) v3 -- 0x7f86e001e040 con 0x55b4d4750930 2024-02-02T15:43:49.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.782+0000 7f86e7400700 1 -- 172.21.15.1:0/93676482 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86e0011e60 con 0x55b4d4750930 2024-02-02T15:43:49.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.782+0000 7f86e7400700 1 -- 172.21.15.1:0/93676482 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 70) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f86e0037020 con 0x55b4d4750930 2024-02-02T15:43:49.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.782+0000 7f8713e93b80 10 client.29911 fetch_fsmap finished waiting for FSMap version 70 2024-02-02T15:43:49.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.782+0000 7f8713e93b80 10 client.29911 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:43:49.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.782+0000 7f8713e93b80 20 client.29911 populate_metadata read hostname 'smithi001' 2024-02-02T15:43:49.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.782+0000 7f8713e93b80 1 --2- 172.21.15.1:0/93676482 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55b4d477c080 0x55b4d47d05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:49.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.782+0000 7f8713e93b80 4 client.29911 mds_command: new command op to 24557 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:43:49.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.782+0000 7f8713e93b80 1 -- 172.21.15.1:0/93676482 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b4d4660f50 con 0x55b4d477c080 2024-02-02T15:43:49.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.782+0000 7f86fda00700 1 --2- 172.21.15.1:0/93676482 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55b4d477c080 0x55b4d47d05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:49.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.783+0000 7f86fda00700 1 --2- 172.21.15.1:0/93676482 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55b4d477c080 0x55b4d47d05e0 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ba6be5d919de1a9 server_cookie=ff6f041aa534d8f4 in_seq=0 out_seq=0 2024-02-02T15:43:49.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.783+0000 7f86e7400700 10 client.29911 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:43:49.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.788+0000 7f86e7400700 1 -- 172.21.15.1:0/93676482 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b4d4660f50 con 0x55b4d477c080 2024-02-02T15:43:49.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.788+0000 7f86e7400700 10 client.29911 handle_command_reply: tid=0 2024-02-02T15:43:49.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.788+0000 7f8713e93b80 1 client.29911 shutdown 2024-02-02T15:43:49.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.789+0000 7f8713e93b80 1 -- 172.21.15.1:0/93676482 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b4d4750930 msgr2=0x55b4d47603d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:49.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.789+0000 7f8713e93b80 1 --2- 172.21.15.1:0/93676482 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b4d4750930 0x55b4d47603d0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f86e0011790 tx=0x7f86e0005640 comp rx=0 tx=0).stop 2024-02-02T15:43:49.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.789+0000 7f8713e93b80 1 -- 172.21.15.1:0/93676482 shutdown_connections 2024-02-02T15:43:49.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.789+0000 7f8713e93b80 1 --2- 172.21.15.1:0/93676482 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55b4d477c080 0x55b4d47d05e0 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:49.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.789+0000 7f8713e93b80 1 --2- 172.21.15.1:0/93676482 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b4d475b7e0 0x55b4d47ccb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:49.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.789+0000 7f8713e93b80 1 --2- 172.21.15.1:0/93676482 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b4d4750010 0x55b4d475fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:49.789 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.789+0000 7f8713e93b80 1 --2- 172.21.15.1:0/93676482 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b4d4750930 0x55b4d47603d0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:49.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.789+0000 7f8713e93b80 1 -- 172.21.15.1:0/93676482 >> 172.21.15.1:0/93676482 conn(0x55b4d46afdb0 msgr2=0x55b4d47538e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:49.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.789+0000 7f86e7400700 0 client.29911 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:43:49.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.789+0000 7f8713e93b80 1 -- 172.21.15.1:0/93676482 shutdown_connections 2024-02-02T15:43:49.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.789+0000 7f8713e93b80 1 -- 172.21.15.1:0/93676482 wait complete. 2024-02-02T15:43:49.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.789+0000 7f8713e93b80 20 client.29911 trim_cache size 0 max 16384 2024-02-02T15:43:49.791 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:43:49.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.790+0000 7f8700400700 1 -- 172.21.15.1:0/1826144137 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f86d80a0790 msgr2=0x7f86d80a2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:49.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.790+0000 7f8700400700 1 --2- 172.21.15.1:0/1826144137 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f86d80a0790 0x7f86d80a2c50 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f86e8005950 tx=0x7f86e80058e0 comp rx=0 tx=0).stop 2024-02-02T15:43:49.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.790+0000 7f8700400700 1 -- 172.21.15.1:0/1826144137 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86f8075cd0 msgr2=0x7f86f80710e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:49.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.790+0000 7f8700400700 1 --2- 172.21.15.1:0/1826144137 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86f8075cd0 0x7f86f80710e0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f86f0009fd0 tx=0x7f86f000bc80 comp rx=0 tx=0).stop 2024-02-02T15:43:49.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.791+0000 7f8700400700 1 -- 172.21.15.1:0/1826144137 shutdown_connections 2024-02-02T15:43:49.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.791+0000 7f8700400700 1 --2- 172.21.15.1:0/1826144137 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f86d80a0790 0x7f86d80a2c50 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:49.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.791+0000 7f8700400700 1 --2- 172.21.15.1:0/1826144137 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f86f8075cd0 0x7f86f80710e0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:49.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.791+0000 7f8700400700 1 --2- 172.21.15.1:0/1826144137 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f86f8103290 0x7f86f8075250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:49.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.791+0000 7f8700400700 1 --2- 172.21.15.1:0/1826144137 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f86f81083b0 0x7f86f8075790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:49.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.791+0000 7f8700400700 1 -- 172.21.15.1:0/1826144137 >> 172.21.15.1:0/1826144137 conn(0x7f86f80feb30 msgr2=0x7f86f80ff4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:49.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.791+0000 7f8700400700 1 -- 172.21.15.1:0/1826144137 shutdown_connections 2024-02-02T15:43:49.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:49.791+0000 7f8700400700 1 -- 172.21.15.1:0/1826144137 wait complete. 2024-02-02T15:43:49.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:43:49 smithi071 ceph-mon[44707]: pgmap v527: 129 pgs: 129 active+clean; 1.7 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 8.4 MiB/s wr, 568 op/s 2024-02-02T15:43:49.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:43:49 smithi142 ceph-mon[45728]: pgmap v527: 129 pgs: 129 active+clean; 1.7 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 8.4 MiB/s wr, 568 op/s 2024-02-02T15:43:49.991 INFO:tasks.workunit.client.0.smithi001.stdout: 10 114673 29.51 MB/sec execute 82 sec latency 58.411 ms 2024-02-02T15:43:50.818 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-02T15:43:50.818 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-02T15:43:50.991 INFO:tasks.workunit.client.0.smithi001.stdout: 10 115541 29.75 MB/sec execute 83 sec latency 52.729 ms 2024-02-02T15:43:51.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.107+0000 7f2a94200700 1 -- 172.21.15.1:0/1773061277 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a8c1000e0 msgr2=0x7f2a8c1004c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:51.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.107+0000 7f2a94200700 1 --2- 172.21.15.1:0/1773061277 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a8c1000e0 0x7f2a8c1004c0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c009990 tx=0x7f2a7c009ca0 comp rx=0 tx=0).stop 2024-02-02T15:43:51.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.107+0000 7f2a94200700 1 -- 172.21.15.1:0/1773061277 shutdown_connections 2024-02-02T15:43:51.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.107+0000 7f2a94200700 1 --2- 172.21.15.1:0/1773061277 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a8c105020 0x7f2a8c068010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:51.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.107+0000 7f2a94200700 1 --2- 172.21.15.1:0/1773061277 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a8c1000e0 0x7f2a8c1004c0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:51.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.107+0000 7f2a94200700 1 --2- 172.21.15.1:0/1773061277 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a8c100a90 0x7f2a8c1048f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:51.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.107+0000 7f2a94200700 1 -- 172.21.15.1:0/1773061277 >> 172.21.15.1:0/1773061277 conn(0x7f2a8c0757d0 msgr2=0x7f2a8c075be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:51.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.108+0000 7f2a94200700 1 -- 172.21.15.1:0/1773061277 shutdown_connections 2024-02-02T15:43:51.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.108+0000 7f2a94200700 1 -- 172.21.15.1:0/1773061277 wait complete. 2024-02-02T15:43:51.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.109+0000 7f2a94200700 1 Processor -- start 2024-02-02T15:43:51.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.109+0000 7f2a94200700 1 -- start start 2024-02-02T15:43:51.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.110+0000 7f2a94200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a8c1000e0 0x7f2a8c194cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:51.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.110+0000 7f2a94200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a8c105020 0x7f2a8c1951f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:51.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.110+0000 7f2a94200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a8c1985a0 0x7f2a8c198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:51.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.110+0000 7f2a94200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8c198f90 con 0x7f2a8c1985a0 2024-02-02T15:43:51.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.110+0000 7f2a94200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8c199100 con 0x7f2a8c105020 2024-02-02T15:43:51.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.110+0000 7f2a94200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8c1992c0 con 0x7f2a8c1000e0 2024-02-02T15:43:51.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.110+0000 7f2a91800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a8c1000e0 0x7f2a8c194cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:51.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.110+0000 7f2a90e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a8c105020 0x7f2a8c1951f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:51.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.111+0000 7f2a90e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a8c105020 0x7f2a8c1951f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:43020/0 (socket says 172.21.15.1:43020) 2024-02-02T15:43:51.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.111+0000 7f2a90e00700 1 -- 172.21.15.1:0/964957443 learned_addr learned my addr 172.21.15.1:0/964957443 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:43:51.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.111+0000 7f2a92200700 1 --2- 172.21.15.1:0/964957443 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a8c1985a0 0x7f2a8c198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:51.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.111+0000 7f2a90e00700 1 -- 172.21.15.1:0/964957443 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a8c1000e0 msgr2=0x7f2a8c194cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:51.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.111+0000 7f2a90e00700 1 --2- 172.21.15.1:0/964957443 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a8c1000e0 0x7f2a8c194cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:51.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.111+0000 7f2a90e00700 1 -- 172.21.15.1:0/964957443 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a8c1985a0 msgr2=0x7f2a8c198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:51.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.111+0000 7f2a90e00700 1 --2- 172.21.15.1:0/964957443 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a8c1985a0 0x7f2a8c198a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:51.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.111+0000 7f2a90e00700 1 -- 172.21.15.1:0/964957443 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a74009520 con 0x7f2a8c105020 2024-02-02T15:43:51.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.111+0000 7f2a90e00700 1 --2- 172.21.15.1:0/964957443 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a8c105020 0x7f2a8c1951f0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f2a7400ea30 tx=0x7f2a7400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:51.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.112+0000 7f2a8a000700 1 -- 172.21.15.1:0/964957443 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a74009820 con 0x7f2a8c105020 2024-02-02T15:43:51.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.112+0000 7f2a8a000700 1 -- 172.21.15.1:0/964957443 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a740044e0 con 0x7f2a8c105020 2024-02-02T15:43:51.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.112+0000 7f2a8a000700 1 -- 172.21.15.1:0/964957443 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a74003e80 con 0x7f2a8c105020 2024-02-02T15:43:51.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.112+0000 7f2a94200700 1 -- 172.21.15.1:0/964957443 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a7c0095f0 con 0x7f2a8c105020 2024-02-02T15:43:51.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.112+0000 7f2a94200700 1 -- 172.21.15.1:0/964957443 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a8c1a1c60 con 0x7f2a8c105020 2024-02-02T15:43:51.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.116+0000 7f2a8a000700 1 -- 172.21.15.1:0/964957443 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f2a74003fe0 con 0x7f2a8c105020 2024-02-02T15:43:51.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.118+0000 7f2a8a000700 1 --2- 172.21.15.1:0/964957443 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f2a6c0a07c0 0x7f2a6c0a2c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:51.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.118+0000 7f2a8a000700 1 -- 172.21.15.1:0/964957443 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f2a74014070 con 0x7f2a8c105020 2024-02-02T15:43:51.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.119+0000 7f2a91800700 1 --2- 172.21.15.1:0/964957443 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f2a6c0a07c0 0x7f2a6c0a2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:51.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.120+0000 7f2a91800700 1 --2- 172.21.15.1:0/964957443 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f2a6c0a07c0 0x7f2a6c0a2c80 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c00b2e0 tx=0x7f2a7c005da0 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:51.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.120+0000 7f2aa7c8eb80 1 Processor -- start 2024-02-02T15:43:51.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.120+0000 7f2aa7c8eb80 1 -- start start 2024-02-02T15:43:51.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.121+0000 7f2aa7c8eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5561679a1090 0x5561679a1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:51.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.121+0000 7f2aa7c8eb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5561679a19b0 0x5561679bd330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:51.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.121+0000 7f2aa7c8eb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5561679bd870 0x5561679bfc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:51.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.121+0000 7f2aa7c8eb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x556167915540 con 0x5561679a19b0 2024-02-02T15:43:51.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.121+0000 7f2aa7c8eb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561678f9b20 con 0x5561679bd870 2024-02-02T15:43:51.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.121+0000 7f2aa7c8eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561678f74e0 con 0x5561679a1090 2024-02-02T15:43:51.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.121+0000 7f2a92200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5561679bd870 0x5561679bfc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:51.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.121+0000 7f2a92200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5561679bd870 0x5561679bfc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:43024/0 (socket says 172.21.15.1:43024) 2024-02-02T15:43:51.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.121+0000 7f2a92200700 1 -- 172.21.15.1:0/3551410507 learned_addr learned my addr 172.21.15.1:0/3551410507 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:43:51.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.122+0000 7f2a92200700 1 -- 172.21.15.1:0/3551410507 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5561679a1090 msgr2=0x5561679a1470 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-02T15:43:51.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.122+0000 7f2a92200700 1 --2- 172.21.15.1:0/3551410507 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5561679a1090 0x5561679a1470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:51.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.122+0000 7f2a92200700 1 -- 172.21.15.1:0/3551410507 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5561679a19b0 msgr2=0x5561679bd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:51.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.122+0000 7f2a92200700 1 --2- 172.21.15.1:0/3551410507 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5561679a19b0 0x5561679bd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:51.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.122+0000 7f2a92200700 1 -- 172.21.15.1:0/3551410507 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556167774430 con 0x5561679bd870 2024-02-02T15:43:51.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.122+0000 7f2a92200700 1 --2- 172.21.15.1:0/3551410507 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5561679bd870 0x5561679bfc60 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f2a8400bf10 tx=0x7f2a8400b320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=642649dc8aefcc server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:51.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.122+0000 7f2a82a00700 1 -- 172.21.15.1:0/3551410507 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a84012070 con 0x5561679bd870 2024-02-02T15:43:51.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.122+0000 7f2a82a00700 1 -- 172.21.15.1:0/3551410507 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a840084d0 con 0x5561679bd870 2024-02-02T15:43:51.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.122+0000 7f2a82a00700 1 -- 172.21.15.1:0/3551410507 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a84009680 con 0x5561679bd870 2024-02-02T15:43:51.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.123+0000 7f2aa7c8eb80 1 -- 172.21.15.1:0/3551410507 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5561679bd870 msgr2=0x5561679bfc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:51.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.123+0000 7f2aa7c8eb80 1 --2- 172.21.15.1:0/3551410507 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5561679bd870 0x5561679bfc60 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f2a8400bf10 tx=0x7f2a8400b320 comp rx=0 tx=0).stop 2024-02-02T15:43:51.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.124+0000 7f2aa7c8eb80 1 -- 172.21.15.1:0/3551410507 shutdown_connections 2024-02-02T15:43:51.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.124+0000 7f2aa7c8eb80 1 --2- 172.21.15.1:0/3551410507 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5561679bd870 0x5561679bfc60 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:51.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.124+0000 7f2aa7c8eb80 1 --2- 172.21.15.1:0/3551410507 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5561679a1090 0x5561679a1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:51.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.124+0000 7f2aa7c8eb80 1 --2- 172.21.15.1:0/3551410507 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5561679a19b0 0x5561679bd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:51.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.124+0000 7f2aa7c8eb80 1 -- 172.21.15.1:0/3551410507 >> 172.21.15.1:0/3551410507 conn(0x556167900e30 msgr2=0x55616799d430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:51.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.124+0000 7f2aa7c8eb80 1 -- 172.21.15.1:0/3551410507 shutdown_connections 2024-02-02T15:43:51.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.124+0000 7f2aa7c8eb80 1 -- 172.21.15.1:0/3551410507 wait complete. 2024-02-02T15:43:51.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.125+0000 7f2aa7c8eb80 1 -- start start 2024-02-02T15:43:51.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.125+0000 7f2aa7c8eb80 1 Processor -- start 2024-02-02T15:43:51.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.126+0000 7f2aa7c8eb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5561679a1090 0x5561679ba6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:51.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.126+0000 7f2aa7c8eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5561679a19b0 0x5561679bac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:51.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.126+0000 7f2aa7c8eb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5561679bd870 0x5561679bb150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:51.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.126+0000 7f2aa7c8eb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561678f2dd0 con 0x5561679a1090 2024-02-02T15:43:51.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.126+0000 7f2aa7c8eb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55616788ee00 con 0x5561679bd870 2024-02-02T15:43:51.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.126+0000 7f2aa7c8eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55616788eaa0 con 0x5561679a19b0 2024-02-02T15:43:51.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.126+0000 7f2aa7c8eb80 10 client.0 _ll_register_callbacks cb 0x556167890310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:43:51.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.127+0000 7f2a91800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5561679a1090 0x5561679ba6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:51.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.127+0000 7f2a91800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5561679a1090 0x5561679ba6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51914/0 (socket says 172.21.15.1:51914) 2024-02-02T15:43:51.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.127+0000 7f2a91800700 1 -- 172.21.15.1:0/1869388068 learned_addr learned my addr 172.21.15.1:0/1869388068 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:43:51.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.128+0000 7f2a92200700 1 --2- 172.21.15.1:0/1869388068 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5561679bd870 0x5561679bb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:51.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.128+0000 7f2a91800700 1 -- 172.21.15.1:0/1869388068 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5561679a19b0 msgr2=0x5561679bac10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-02T15:43:51.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.128+0000 7f2a91800700 1 --2- 172.21.15.1:0/1869388068 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5561679a19b0 0x5561679bac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:51.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.128+0000 7f2a91800700 1 -- 172.21.15.1:0/1869388068 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5561679bd870 msgr2=0x5561679bb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:51.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.128+0000 7f2a91800700 1 --2- 172.21.15.1:0/1869388068 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5561679bd870 0x5561679bb150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:51.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.128+0000 7f2a91800700 1 -- 172.21.15.1:0/1869388068 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a8400bbc0 con 0x5561679a1090 2024-02-02T15:43:51.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.128+0000 7f2a92200700 1 --2- 172.21.15.1:0/1869388068 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5561679bd870 0x5561679bb150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:43:51.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.129+0000 7f2a91800700 1 --2- 172.21.15.1:0/1869388068 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5561679a1090 0x5561679ba6d0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c015000 tx=0x7f2a7c009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=889a9388962fd69f server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:51.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.129+0000 7f2a7b400700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:43:51.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.130+0000 7f2a7b400700 1 -- 172.21.15.1:0/1869388068 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c02b070 con 0x5561679a1090 2024-02-02T15:43:51.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.130+0000 7f2aa7c8eb80 1 -- 172.21.15.1:0/1869388068 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556167995170 con 0x5561679a1090 2024-02-02T15:43:51.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.130+0000 7f2a7b400700 1 -- 172.21.15.1:0/1869388068 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f2a7c004710 con 0x5561679a1090 2024-02-02T15:43:51.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.131+0000 7f2a7b400700 1 -- 172.21.15.1:0/1869388068 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a7c0048d0 con 0x5561679a1090 2024-02-02T15:43:51.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.131+0000 7f2aa7c8eb80 10 client.19779 fetch_fsmap learned FSMap version 70 2024-02-02T15:43:51.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.131+0000 7f2aa7c8eb80 1 -- 172.21.15.1:0/1869388068 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) v3 -- 0x556167720be0 con 0x5561679a1090 2024-02-02T15:43:51.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.132+0000 7f2a7b400700 1 -- 172.21.15.1:0/1869388068 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556167995170 con 0x5561679a1090 2024-02-02T15:43:51.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.132+0000 7f2a7b400700 1 -- 172.21.15.1:0/1869388068 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 70) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f2a7c00fd10 con 0x5561679a1090 2024-02-02T15:43:51.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.132+0000 7f2aa7c8eb80 10 client.19779 fetch_fsmap finished waiting for FSMap version 70 2024-02-02T15:43:51.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.132+0000 7f2aa7c8eb80 10 client.19779 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:43:51.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.132+0000 7f2aa7c8eb80 20 client.19779 populate_metadata read hostname 'smithi001' 2024-02-02T15:43:51.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.132+0000 7f2aa7c8eb80 1 --2- 172.21.15.1:0/1869388068 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5561679cd290 0x5561679cf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:51.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.132+0000 7f2aa7c8eb80 4 client.19779 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:43:51.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.132+0000 7f2aa7c8eb80 1 -- 172.21.15.1:0/1869388068 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5561678ebe50 con 0x5561679cd290 2024-02-02T15:43:51.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.133+0000 7f2a90e00700 1 --2- 172.21.15.1:0/1869388068 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5561679cd290 0x5561679cf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:51.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.133+0000 7f2a90e00700 1 --2- 172.21.15.1:0/1869388068 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5561679cd290 0x5561679cf6b0 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2031801ea2df48a3 server_cookie=46a204e4d357a699 in_seq=0 out_seq=0 2024-02-02T15:43:51.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.133+0000 7f2a7b400700 10 client.19779 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:43:51.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.137+0000 7f2a7b400700 1 -- 172.21.15.1:0/1869388068 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5561678ebe50 con 0x5561679cd290 2024-02-02T15:43:51.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.137+0000 7f2a7b400700 10 client.19779 handle_command_reply: tid=0 2024-02-02T15:43:51.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.137+0000 7f2aa7c8eb80 1 client.19779 shutdown 2024-02-02T15:43:51.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.138+0000 7f2aa7c8eb80 1 -- 172.21.15.1:0/1869388068 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5561679a1090 msgr2=0x5561679ba6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:51.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.138+0000 7f2aa7c8eb80 1 --2- 172.21.15.1:0/1869388068 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5561679a1090 0x5561679ba6d0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c015000 tx=0x7f2a7c009ca0 comp rx=0 tx=0).stop 2024-02-02T15:43:51.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.138+0000 7f2aa7c8eb80 1 -- 172.21.15.1:0/1869388068 shutdown_connections 2024-02-02T15:43:51.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.138+0000 7f2aa7c8eb80 1 --2- 172.21.15.1:0/1869388068 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5561679cd290 0x5561679cf6b0 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:51.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.138+0000 7f2aa7c8eb80 1 --2- 172.21.15.1:0/1869388068 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5561679bd870 0x5561679bb150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:51.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.138+0000 7f2aa7c8eb80 1 --2- 172.21.15.1:0/1869388068 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5561679a1090 0x5561679ba6d0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:51.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.138+0000 7f2aa7c8eb80 1 --2- 172.21.15.1:0/1869388068 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5561679a19b0 0x5561679bac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:51.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.138+0000 7f2aa7c8eb80 1 -- 172.21.15.1:0/1869388068 >> 172.21.15.1:0/1869388068 conn(0x556167900e30 msgr2=0x55616799d020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:51.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.138+0000 7f2a7b400700 0 client.19779 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:43:51.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.138+0000 7f2aa7c8eb80 1 -- 172.21.15.1:0/1869388068 shutdown_connections 2024-02-02T15:43:51.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.138+0000 7f2aa7c8eb80 1 -- 172.21.15.1:0/1869388068 wait complete. 2024-02-02T15:43:51.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.138+0000 7f2aa7c8eb80 20 client.19779 trim_cache size 0 max 16384 2024-02-02T15:43:51.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.162+0000 7f2aa7c8eb80 1 Processor -- start 2024-02-02T15:43:51.162 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.162+0000 7f2aa7c8eb80 1 -- start start 2024-02-02T15:43:51.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.163+0000 7f2aa7c8eb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5561679a19b0 0x5561679bb000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:51.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.163+0000 7f2aa7c8eb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5561679bd870 0x5561679a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:51.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.163+0000 7f2aa7c8eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5561679b6120 0x5561679b7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:51.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.163+0000 7f2aa7c8eb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x556167776720 con 0x5561679a19b0 2024-02-02T15:43:51.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.163+0000 7f2aa7c8eb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561679c9220 con 0x5561679bd870 2024-02-02T15:43:51.163 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.163+0000 7f2aa7c8eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561679b9300 con 0x5561679b6120 2024-02-02T15:43:51.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.163+0000 7f2a90e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5561679bd870 0x5561679a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:51.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.163+0000 7f2a92200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5561679b6120 0x5561679b7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:51.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.163+0000 7f2a90e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5561679bd870 0x5561679a38c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:43046/0 (socket says 172.21.15.1:43046) 2024-02-02T15:43:51.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.163+0000 7f2a92200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5561679b6120 0x5561679b7990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:43924/0 (socket says 172.21.15.1:43924) 2024-02-02T15:43:51.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.163+0000 7f2a90e00700 1 -- 172.21.15.1:0/3973308990 learned_addr learned my addr 172.21.15.1:0/3973308990 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:43:51.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.163+0000 7f2a91800700 1 --2- 172.21.15.1:0/3973308990 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5561679a19b0 0x5561679bb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:51.164 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.164+0000 7f2a90e00700 1 -- 172.21.15.1:0/3973308990 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5561679b6120 msgr2=0x5561679b7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:51.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.164+0000 7f2a90e00700 1 --2- 172.21.15.1:0/3973308990 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5561679b6120 0x5561679b7990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:51.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.164+0000 7f2a90e00700 1 -- 172.21.15.1:0/3973308990 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5561679a19b0 msgr2=0x5561679bb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:51.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.164+0000 7f2a90e00700 1 --2- 172.21.15.1:0/3973308990 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5561679a19b0 0x5561679bb000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:51.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.164+0000 7f2a90e00700 1 -- 172.21.15.1:0/3973308990 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5561678b8f20 con 0x5561679bd870 2024-02-02T15:43:51.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.164+0000 7f2a90e00700 1 --2- 172.21.15.1:0/3973308990 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5561679bd870 0x5561679a38c0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f2a74009fd0 tx=0x7f2a740990f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c79078f51dabb303 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:51.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.164+0000 7f2a83400700 1 -- 172.21.15.1:0/3973308990 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a74099e10 con 0x5561679bd870 2024-02-02T15:43:51.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.164+0000 7f2a83400700 1 -- 172.21.15.1:0/3973308990 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a7400cc90 con 0x5561679bd870 2024-02-02T15:43:51.165 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.164+0000 7f2a83400700 1 -- 172.21.15.1:0/3973308990 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a74097930 con 0x5561679bd870 2024-02-02T15:43:51.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.165+0000 7f2aa7c8eb80 1 -- 172.21.15.1:0/3973308990 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5561679bd870 msgr2=0x5561679a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:51.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.165+0000 7f2aa7c8eb80 1 --2- 172.21.15.1:0/3973308990 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5561679bd870 0x5561679a38c0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f2a74009fd0 tx=0x7f2a740990f0 comp rx=0 tx=0).stop 2024-02-02T15:43:51.166 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.166+0000 7f2aa7c8eb80 1 -- 172.21.15.1:0/3973308990 shutdown_connections 2024-02-02T15:43:51.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.166+0000 7f2aa7c8eb80 1 --2- 172.21.15.1:0/3973308990 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5561679b6120 0x5561679b7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:51.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.166+0000 7f2aa7c8eb80 1 --2- 172.21.15.1:0/3973308990 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5561679a19b0 0x5561679bb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:51.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.166+0000 7f2aa7c8eb80 1 --2- 172.21.15.1:0/3973308990 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5561679bd870 0x5561679a38c0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:51.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.166+0000 7f2aa7c8eb80 1 -- 172.21.15.1:0/3973308990 >> 172.21.15.1:0/3973308990 conn(0x556167900e30 msgr2=0x5561679b5a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:51.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.167+0000 7f2aa7c8eb80 1 -- 172.21.15.1:0/3973308990 shutdown_connections 2024-02-02T15:43:51.167 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.167+0000 7f2aa7c8eb80 1 -- 172.21.15.1:0/3973308990 wait complete. 2024-02-02T15:43:51.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.168+0000 7f2aa7c8eb80 1 -- start start 2024-02-02T15:43:51.168 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.169+0000 7f2aa7c8eb80 1 Processor -- start 2024-02-02T15:43:51.170 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.170+0000 7f2aa7c8eb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5561679a19b0 0x556167a18370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:51.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.170+0000 7f2aa7c8eb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5561679b6120 0x556167a188b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:51.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.170+0000 7f2aa7c8eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5561679bd870 0x5561679c3470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:51.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.170+0000 7f2aa7c8eb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561679c3bb0 con 0x5561679a19b0 2024-02-02T15:43:51.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.170+0000 7f2aa7c8eb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561679c3d20 con 0x5561679b6120 2024-02-02T15:43:51.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.170+0000 7f2aa7c8eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561679c3ee0 con 0x5561679bd870 2024-02-02T15:43:51.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.170+0000 7f2aa7c8eb80 10 client.0 _ll_register_callbacks cb 0x556167890310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:43:51.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.170+0000 7f2a90e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5561679b6120 0x556167a188b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:51.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.170+0000 7f2a90e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5561679b6120 0x556167a188b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:43062/0 (socket says 172.21.15.1:43062) 2024-02-02T15:43:51.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.170+0000 7f2a90e00700 1 -- 172.21.15.1:0/505286213 learned_addr learned my addr 172.21.15.1:0/505286213 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:43:51.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.170+0000 7f2a91800700 1 --2- 172.21.15.1:0/505286213 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5561679a19b0 0x556167a18370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:51.171 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.170+0000 7f2a92200700 1 --2- 172.21.15.1:0/505286213 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5561679bd870 0x5561679c3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:51.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.171+0000 7f2a92200700 1 -- 172.21.15.1:0/505286213 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5561679b6120 msgr2=0x556167a188b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:51.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.171+0000 7f2a92200700 1 --2- 172.21.15.1:0/505286213 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5561679b6120 0x556167a188b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:51.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.171+0000 7f2a92200700 1 -- 172.21.15.1:0/505286213 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5561679a19b0 msgr2=0x556167a18370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:51.172 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.171+0000 7f2a92200700 1 --2- 172.21.15.1:0/505286213 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5561679a19b0 0x556167a18370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:51.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.171+0000 7f2a92200700 1 -- 172.21.15.1:0/505286213 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a8c1a1c60 con 0x5561679bd870 2024-02-02T15:43:51.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.172+0000 7f2a92200700 1 --2- 172.21.15.1:0/505286213 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5561679bd870 0x5561679c3470 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f2a8400bee0 tx=0x7f2a8400d6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31bfa35bf1fe02ec server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:51.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.172+0000 7f2a7b400700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:43:51.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.172+0000 7f2a7b400700 1 -- 172.21.15.1:0/505286213 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a84012040 con 0x5561679bd870 2024-02-02T15:43:51.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.172+0000 7f2a7b400700 1 -- 172.21.15.1:0/505286213 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f2a84003e10 con 0x5561679bd870 2024-02-02T15:43:51.173 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.172+0000 7f2aa7c8eb80 1 -- 172.21.15.1:0/505286213 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2a840084d0 con 0x5561679bd870 2024-02-02T15:43:51.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.174+0000 7f2a7b400700 1 -- 172.21.15.1:0/505286213 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a84009440 con 0x5561679bd870 2024-02-02T15:43:51.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.174+0000 7f2aa7c8eb80 10 client.29923 fetch_fsmap learned FSMap version 70 2024-02-02T15:43:51.174 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.174+0000 7f2aa7c8eb80 1 -- 172.21.15.1:0/505286213 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) v3 -- 0x556167778d50 con 0x5561679bd870 2024-02-02T15:43:51.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.177+0000 7f2a7b400700 1 -- 172.21.15.1:0/505286213 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a840084d0 con 0x5561679bd870 2024-02-02T15:43:51.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.177+0000 7f2a7b400700 1 -- 172.21.15.1:0/505286213 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 70) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f2a84005280 con 0x5561679bd870 2024-02-02T15:43:51.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.177+0000 7f2aa7c8eb80 10 client.29923 fetch_fsmap finished waiting for FSMap version 70 2024-02-02T15:43:51.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.177+0000 7f2aa7c8eb80 10 client.29923 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:43:51.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.177+0000 7f2aa7c8eb80 20 client.29923 populate_metadata read hostname 'smithi001' 2024-02-02T15:43:51.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.177+0000 7f2aa7c8eb80 1 --2- 172.21.15.1:0/505286213 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x556167a211e0 0x556167a23600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:51.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.178+0000 7f2aa7c8eb80 4 client.29923 mds_command: new command op to 24557 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-02T15:43:51.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.178+0000 7f2aa7c8eb80 1 -- 172.21.15.1:0/505286213 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5561678b1fd0 con 0x556167a211e0 2024-02-02T15:43:51.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.178+0000 7f2a91800700 1 --2- 172.21.15.1:0/505286213 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x556167a211e0 0x556167a23600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:51.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.179+0000 7f2a7b400700 10 client.29923 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:43:51.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.179+0000 7f2a91800700 1 --2- 172.21.15.1:0/505286213 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x556167a211e0 0x556167a23600 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67515cbc7ffb799 server_cookie=3a3f26dc2724d049 in_seq=0 out_seq=0 2024-02-02T15:43:51.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.180+0000 7f2a7b400700 1 -- 172.21.15.1:0/505286213 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5561678b1fd0 con 0x556167a211e0 2024-02-02T15:43:51.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.180+0000 7f2a7b400700 10 client.29923 handle_command_reply: tid=0 2024-02-02T15:43:51.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.180+0000 7f2aa7c8eb80 1 client.29923 shutdown 2024-02-02T15:43:51.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.180+0000 7f2aa7c8eb80 1 -- 172.21.15.1:0/505286213 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5561679bd870 msgr2=0x5561679c3470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:51.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.180+0000 7f2aa7c8eb80 1 --2- 172.21.15.1:0/505286213 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5561679bd870 0x5561679c3470 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f2a8400bee0 tx=0x7f2a8400d6c0 comp rx=0 tx=0).stop 2024-02-02T15:43:51.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.180+0000 7f2aa7c8eb80 1 -- 172.21.15.1:0/505286213 shutdown_connections 2024-02-02T15:43:51.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.180+0000 7f2aa7c8eb80 1 --2- 172.21.15.1:0/505286213 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x556167a211e0 0x556167a23600 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:51.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.181+0000 7f2aa7c8eb80 1 --2- 172.21.15.1:0/505286213 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5561679bd870 0x5561679c3470 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:51.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.181+0000 7f2aa7c8eb80 1 --2- 172.21.15.1:0/505286213 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5561679a19b0 0x556167a18370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:51.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.181+0000 7f2aa7c8eb80 1 --2- 172.21.15.1:0/505286213 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5561679b6120 0x556167a188b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:51.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.181+0000 7f2a7b400700 0 client.29923 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:43:51.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.181+0000 7f2aa7c8eb80 1 -- 172.21.15.1:0/505286213 >> 172.21.15.1:0/505286213 conn(0x556167900e30 msgr2=0x5561679bbf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:51.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.181+0000 7f2aa7c8eb80 1 -- 172.21.15.1:0/505286213 shutdown_connections 2024-02-02T15:43:51.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.181+0000 7f2aa7c8eb80 1 -- 172.21.15.1:0/505286213 wait complete. 2024-02-02T15:43:51.182 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.181+0000 7f2aa7c8eb80 20 client.29923 trim_cache size 0 max 16384 2024-02-02T15:43:51.182 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:43:51.182 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2024-02-02T15:43:51.182 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "3daf8575-1db7-4128-8672-e508a4067b8d", 2024-02-02T15:43:51.182 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2024-02-02T15:43:51.182 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:43:51.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.182+0000 7f2a94200700 1 -- 172.21.15.1:0/964957443 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f2a6c0a07c0 msgr2=0x7f2a6c0a2c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:51.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.182+0000 7f2a94200700 1 --2- 172.21.15.1:0/964957443 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f2a6c0a07c0 0x7f2a6c0a2c80 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c00b2e0 tx=0x7f2a7c005da0 comp rx=0 tx=0).stop 2024-02-02T15:43:51.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.182+0000 7f2a94200700 1 -- 172.21.15.1:0/964957443 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a8c105020 msgr2=0x7f2a8c1951f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:51.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.182+0000 7f2a94200700 1 --2- 172.21.15.1:0/964957443 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a8c105020 0x7f2a8c1951f0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f2a7400ea30 tx=0x7f2a7400edf0 comp rx=0 tx=0).stop 2024-02-02T15:43:51.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.182+0000 7f2a94200700 1 -- 172.21.15.1:0/964957443 shutdown_connections 2024-02-02T15:43:51.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.182+0000 7f2a94200700 1 --2- 172.21.15.1:0/964957443 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f2a6c0a07c0 0x7f2a6c0a2c80 secure :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c00b2e0 tx=0x7f2a7c005da0 comp rx=0 tx=0).stop 2024-02-02T15:43:51.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.182+0000 7f2a94200700 1 --2- 172.21.15.1:0/964957443 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2a8c1985a0 0x7f2a8c198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:51.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.182+0000 7f2a94200700 1 --2- 172.21.15.1:0/964957443 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2a8c1000e0 0x7f2a8c194cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:51.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.182+0000 7f2a94200700 1 --2- 172.21.15.1:0/964957443 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2a8c105020 0x7f2a8c1951f0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:51.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.182+0000 7f2a94200700 1 -- 172.21.15.1:0/964957443 >> 172.21.15.1:0/964957443 conn(0x7f2a8c0757d0 msgr2=0x7f2a8c0fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:51.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.183+0000 7f2a94200700 1 -- 172.21.15.1:0/964957443 shutdown_connections 2024-02-02T15:43:51.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.183+0000 7f2a94200700 1 -- 172.21.15.1:0/964957443 wait complete. 2024-02-02T15:43:51.207 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-02T15:43:51.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.495+0000 7f97f3400700 1 -- 172.21.15.1:0/395257391 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97ec073a30 msgr2=0x7f97ec10d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:51.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.495+0000 7f97f3400700 1 --2- 172.21.15.1:0/395257391 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97ec073a30 0x7f97ec10d0f0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f97d4009990 tx=0x7f97d4009ca0 comp rx=0 tx=0).stop 2024-02-02T15:43:51.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.495+0000 7f97f3400700 1 -- 172.21.15.1:0/395257391 shutdown_connections 2024-02-02T15:43:51.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.495+0000 7f97f3400700 1 --2- 172.21.15.1:0/395257391 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97ec10d630 0x7f97ec10fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:51.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.495+0000 7f97f3400700 1 --2- 172.21.15.1:0/395257391 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f97ec073110 0x7f97ec0734f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:51.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.495+0000 7f97f3400700 1 --2- 172.21.15.1:0/395257391 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97ec073a30 0x7f97ec10d0f0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:51.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.495+0000 7f97f3400700 1 -- 172.21.15.1:0/395257391 >> 172.21.15.1:0/395257391 conn(0x7f97ec0fc840 msgr2=0x7f97ec0fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:51.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.496+0000 7f97f3400700 1 -- 172.21.15.1:0/395257391 shutdown_connections 2024-02-02T15:43:51.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.496+0000 7f97f3400700 1 -- 172.21.15.1:0/395257391 wait complete. 2024-02-02T15:43:51.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.498+0000 7f97f3400700 1 Processor -- start 2024-02-02T15:43:51.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.498+0000 7f97f3400700 1 -- start start 2024-02-02T15:43:51.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.499+0000 7f97f3400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97ec073110 0x7f97ec10cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:51.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.499+0000 7f97f3400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f97ec073a30 0x7f97ec10d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:51.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.499+0000 7f97f3400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97ec10d630 0x7f97ec109d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:51.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.499+0000 7f97f3400700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ec10a540 con 0x7f97ec073a30 2024-02-02T15:43:51.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.499+0000 7f97f3400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ec10a680 con 0x7f97ec073110 2024-02-02T15:43:51.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.499+0000 7f97f3400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ec10a840 con 0x7f97ec10d630 2024-02-02T15:43:51.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.499+0000 7f97ebe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f97ec073a30 0x7f97ec10d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:51.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.499+0000 7f97f0a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97ec073110 0x7f97ec10cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:51.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.499+0000 7f97f0a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97ec073110 0x7f97ec10cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:43080/0 (socket says 172.21.15.1:43080) 2024-02-02T15:43:51.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.499+0000 7f97f0a00700 1 -- 172.21.15.1:0/3121698278 learned_addr learned my addr 172.21.15.1:0/3121698278 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:43:51.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.500+0000 7f97ebe00700 1 -- 172.21.15.1:0/3121698278 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97ec10d630 msgr2=0x7f97ec109d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:51.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.500+0000 7f97ebe00700 1 --2- 172.21.15.1:0/3121698278 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97ec10d630 0x7f97ec109d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:51.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.500+0000 7f97ebe00700 1 -- 172.21.15.1:0/3121698278 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97ec073110 msgr2=0x7f97ec10cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:51.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.500+0000 7f97ebe00700 1 --2- 172.21.15.1:0/3121698278 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97ec073110 0x7f97ec10cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:51.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.500+0000 7f97ebe00700 1 -- 172.21.15.1:0/3121698278 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97dc009520 con 0x7f97ec073a30 2024-02-02T15:43:51.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.501+0000 7f97ebe00700 1 --2- 172.21.15.1:0/3121698278 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f97ec073a30 0x7f97ec10d070 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f97d4009990 tx=0x7f97d4009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:51.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.501+0000 7f97e9600700 1 -- 172.21.15.1:0/3121698278 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d401d070 con 0x7f97ec073a30 2024-02-02T15:43:51.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.502+0000 7f97e9600700 1 -- 172.21.15.1:0/3121698278 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f97d4021410 con 0x7f97ec073a30 2024-02-02T15:43:51.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.502+0000 7f97e9600700 1 -- 172.21.15.1:0/3121698278 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d4017650 con 0x7f97ec073a30 2024-02-02T15:43:51.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.502+0000 7f97f3400700 1 -- 172.21.15.1:0/3121698278 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97e4009c40 con 0x7f97ec073a30 2024-02-02T15:43:51.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.502+0000 7f97f3400700 1 -- 172.21.15.1:0/3121698278 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97d40095f0 con 0x7f97ec073a30 2024-02-02T15:43:51.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.507+0000 7f97e9600700 1 -- 172.21.15.1:0/3121698278 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f97d40217a0 con 0x7f97ec073a30 2024-02-02T15:43:51.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.508+0000 7f9806f27b80 1 Processor -- start 2024-02-02T15:43:51.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.510+0000 7f9806f27b80 1 -- start start 2024-02-02T15:43:51.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.510+0000 7f9806f27b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56314e4de010 0x56314e4de3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:51.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.510+0000 7f9806f27b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56314e4de930 0x56314e4e92a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:51.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.510+0000 7f9806f27b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56314e4e97e0 0x56314e4ebbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:51.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.510+0000 7f9806f27b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56314e4524c0 con 0x56314e4de930 2024-02-02T15:43:51.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.510+0000 7f9806f27b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56314e436aa0 con 0x56314e4de010 2024-02-02T15:43:51.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.510+0000 7f9806f27b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56314e434460 con 0x56314e4e97e0 2024-02-02T15:43:51.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.511+0000 7f97e9600700 1 --2- 172.21.15.1:0/3121698278 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f97cc0a0710 0x7f97cc0a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:51.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.511+0000 7f97f1400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56314e4de010 0x56314e4de3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:51.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.511+0000 7f97f1400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56314e4de010 0x56314e4de3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:43096/0 (socket says 172.21.15.1:43096) 2024-02-02T15:43:51.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.511+0000 7f97f1400700 1 -- 172.21.15.1:0/3722256818 learned_addr learned my addr 172.21.15.1:0/3722256818 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:43:51.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.511+0000 7f97e9600700 1 -- 172.21.15.1:0/3121698278 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f97d4021e40 con 0x7f97ec073a30 2024-02-02T15:43:51.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.511+0000 7f97ebe00700 1 --2- 172.21.15.1:0/3722256818 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56314e4e97e0 0x56314e4ebbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:51.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.511+0000 7f97f0a00700 1 --2- 172.21.15.1:0/3722256818 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56314e4de930 0x56314e4e92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:51.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.511+0000 7f97f1400700 1 --2- 172.21.15.1:0/3121698278 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f97cc0a0710 0x7f97cc0a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:51.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.511+0000 7f97ebe00700 1 -- 172.21.15.1:0/3722256818 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56314e4de010 msgr2=0x56314e4de3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:51.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.511+0000 7f97ebe00700 1 --2- 172.21.15.1:0/3722256818 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56314e4de010 0x56314e4de3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:51.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.512+0000 7f97ebe00700 1 -- 172.21.15.1:0/3722256818 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56314e4de930 msgr2=0x56314e4e92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:51.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.512+0000 7f97ebe00700 1 --2- 172.21.15.1:0/3722256818 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56314e4de930 0x56314e4e92a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:51.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.512+0000 7f97ebe00700 1 -- 172.21.15.1:0/3722256818 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56314e2b13b0 con 0x56314e4e97e0 2024-02-02T15:43:51.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.512+0000 7f97ebe00700 1 --2- 172.21.15.1:0/3722256818 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56314e4e97e0 0x56314e4ebbd0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f97d40ca740 tx=0x7f97d4096990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c8bf0c45ca8853c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:51.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.513+0000 7f97f1400700 1 --2- 172.21.15.1:0/3121698278 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f97cc0a0710 0x7f97cc0a2bd0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f97e4009fd0 tx=0x7f97e4009920 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:51.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.515+0000 7f97e1c00700 1 -- 172.21.15.1:0/3722256818 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d4099f30 con 0x56314e4e97e0 2024-02-02T15:43:51.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.516+0000 7f97e1c00700 1 -- 172.21.15.1:0/3722256818 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f97d40a0040 con 0x56314e4e97e0 2024-02-02T15:43:51.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.516+0000 7f97e1c00700 1 -- 172.21.15.1:0/3722256818 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d4098970 con 0x56314e4e97e0 2024-02-02T15:43:51.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.517+0000 7f9806f27b80 1 -- 172.21.15.1:0/3722256818 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56314e4e97e0 msgr2=0x56314e4ebbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:51.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.517+0000 7f9806f27b80 1 --2- 172.21.15.1:0/3722256818 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56314e4e97e0 0x56314e4ebbd0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f97d40ca740 tx=0x7f97d4096990 comp rx=0 tx=0).stop 2024-02-02T15:43:51.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.518+0000 7f9806f27b80 1 -- 172.21.15.1:0/3722256818 shutdown_connections 2024-02-02T15:43:51.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.518+0000 7f9806f27b80 1 --2- 172.21.15.1:0/3722256818 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56314e4e97e0 0x56314e4ebbd0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:51.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.518+0000 7f9806f27b80 1 --2- 172.21.15.1:0/3722256818 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56314e4de010 0x56314e4de3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:51.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.518+0000 7f9806f27b80 1 --2- 172.21.15.1:0/3722256818 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56314e4de930 0x56314e4e92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:51.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.518+0000 7f9806f27b80 1 -- 172.21.15.1:0/3722256818 >> 172.21.15.1:0/3722256818 conn(0x56314e43ddb0 msgr2=0x56314e4da3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:51.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.518+0000 7f9806f27b80 1 -- 172.21.15.1:0/3722256818 shutdown_connections 2024-02-02T15:43:51.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.518+0000 7f9806f27b80 1 -- 172.21.15.1:0/3722256818 wait complete. 2024-02-02T15:43:51.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.519+0000 7f9806f27b80 1 -- start start 2024-02-02T15:43:51.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.519+0000 7f9806f27b80 1 Processor -- start 2024-02-02T15:43:51.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.521+0000 7f9806f27b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56314e4de010 0x56314e4e9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:51.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.521+0000 7f9806f27b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56314e4de930 0x56314e4ea100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:51.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.521+0000 7f9806f27b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56314e4e97e0 0x56314e4ea640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:51.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.521+0000 7f9806f27b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56314e42fd50 con 0x56314e4e97e0 2024-02-02T15:43:51.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.521+0000 7f9806f27b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56314e3cbd80 con 0x56314e4de930 2024-02-02T15:43:51.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.521+0000 7f9806f27b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56314e3cba20 con 0x56314e4de010 2024-02-02T15:43:51.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.521+0000 7f9806f27b80 10 client.0 _ll_register_callbacks cb 0x56314e3cd290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:43:51.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.522+0000 7f97f1400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56314e4e97e0 0x56314e4ea640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:51.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.522+0000 7f97f1400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56314e4e97e0 0x56314e4ea640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51984/0 (socket says 172.21.15.1:51984) 2024-02-02T15:43:51.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.522+0000 7f97f1400700 1 -- 172.21.15.1:0/2127583328 learned_addr learned my addr 172.21.15.1:0/2127583328 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:43:51.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.523+0000 7f97f1400700 1 -- 172.21.15.1:0/2127583328 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56314e4de010 msgr2=0x56314e4e9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:51.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.523+0000 7f97f1400700 1 --2- 172.21.15.1:0/2127583328 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56314e4de010 0x56314e4e9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:51.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.523+0000 7f97f1400700 1 -- 172.21.15.1:0/2127583328 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56314e4de930 msgr2=0x56314e4ea100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:51.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.523+0000 7f97f1400700 1 --2- 172.21.15.1:0/2127583328 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56314e4de930 0x56314e4ea100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:51.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.523+0000 7f97f1400700 1 -- 172.21.15.1:0/2127583328 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97d40095f0 con 0x56314e4e97e0 2024-02-02T15:43:51.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.524+0000 7f97f1400700 1 --2- 172.21.15.1:0/2127583328 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56314e4e97e0 0x56314e4ea640 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f97e4015ad0 tx=0x7f97e4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca66195cc0810a56 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:51.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.524+0000 7f97da000700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:43:51.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.525+0000 7f97da000700 1 -- 172.21.15.1:0/2127583328 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e4007d20 con 0x56314e4e97e0 2024-02-02T15:43:51.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.525+0000 7f9806f27b80 1 -- 172.21.15.1:0/2127583328 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56314e4d20f0 con 0x56314e4e97e0 2024-02-02T15:43:51.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.525+0000 7f97da000700 1 -- 172.21.15.1:0/2127583328 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f97e401a470 con 0x56314e4e97e0 2024-02-02T15:43:51.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.525+0000 7f97da000700 1 -- 172.21.15.1:0/2127583328 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7f97e4007e80 con 0x56314e4e97e0 2024-02-02T15:43:51.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.525+0000 7f9806f27b80 10 client.19797 fetch_fsmap learned FSMap version 70 2024-02-02T15:43:51.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.525+0000 7f9806f27b80 1 -- 172.21.15.1:0/2127583328 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) v3 -- 0x56314e25db60 con 0x56314e4e97e0 2024-02-02T15:43:51.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.526+0000 7f97da000700 1 -- 172.21.15.1:0/2127583328 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f97e402e080 con 0x56314e4e97e0 2024-02-02T15:43:51.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.526+0000 7f97da000700 1 -- 172.21.15.1:0/2127583328 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 70) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f97e401abf0 con 0x56314e4e97e0 2024-02-02T15:43:51.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.526+0000 7f9806f27b80 10 client.19797 fetch_fsmap finished waiting for FSMap version 70 2024-02-02T15:43:51.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.526+0000 7f9806f27b80 10 client.19797 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:43:51.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.526+0000 7f9806f27b80 20 client.19797 populate_metadata read hostname 'smithi001' 2024-02-02T15:43:51.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.526+0000 7f9806f27b80 1 --2- 172.21.15.1:0/2127583328 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56314e509fa0 0x56314e50c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:51.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.526+0000 7f9806f27b80 4 client.19797 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:43:51.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.526+0000 7f9806f27b80 1 -- 172.21.15.1:0/2127583328 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56314e428dd0 con 0x56314e509fa0 2024-02-02T15:43:51.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.527+0000 7f97f0a00700 1 --2- 172.21.15.1:0/2127583328 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56314e509fa0 0x56314e50c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:51.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.528+0000 7f97f0a00700 1 --2- 172.21.15.1:0/2127583328 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56314e509fa0 0x56314e50c3e0 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bd15b8e0c5f756a server_cookie=11fe35d84909ae31 in_seq=0 out_seq=0 2024-02-02T15:43:51.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.528+0000 7f97da000700 10 client.19797 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:43:51.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.530+0000 7f97da000700 1 -- 172.21.15.1:0/2127583328 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x56314e428dd0 con 0x56314e509fa0 2024-02-02T15:43:51.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.530+0000 7f97da000700 10 client.19797 handle_command_reply: tid=0 2024-02-02T15:43:51.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.531+0000 7f9806f27b80 1 client.19797 shutdown 2024-02-02T15:43:51.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.531+0000 7f9806f27b80 1 -- 172.21.15.1:0/2127583328 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56314e4e97e0 msgr2=0x56314e4ea640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:51.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.531+0000 7f9806f27b80 1 --2- 172.21.15.1:0/2127583328 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56314e4e97e0 0x56314e4ea640 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f97e4015ad0 tx=0x7f97e4004ad0 comp rx=0 tx=0).stop 2024-02-02T15:43:51.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.531+0000 7f9806f27b80 1 -- 172.21.15.1:0/2127583328 shutdown_connections 2024-02-02T15:43:51.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.531+0000 7f9806f27b80 1 --2- 172.21.15.1:0/2127583328 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56314e509fa0 0x56314e50c3e0 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:51.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.531+0000 7f9806f27b80 1 --2- 172.21.15.1:0/2127583328 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56314e4e97e0 0x56314e4ea640 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:51.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.531+0000 7f9806f27b80 1 --2- 172.21.15.1:0/2127583328 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56314e4de010 0x56314e4e9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:51.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.531+0000 7f9806f27b80 1 --2- 172.21.15.1:0/2127583328 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56314e4de930 0x56314e4ea100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:51.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.531+0000 7f9806f27b80 1 -- 172.21.15.1:0/2127583328 >> 172.21.15.1:0/2127583328 conn(0x56314e43ddb0 msgr2=0x56314e4d9f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:51.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.531+0000 7f97da000700 0 client.19797 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:43:51.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.531+0000 7f9806f27b80 1 -- 172.21.15.1:0/2127583328 shutdown_connections 2024-02-02T15:43:51.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.531+0000 7f9806f27b80 1 -- 172.21.15.1:0/2127583328 wait complete. 2024-02-02T15:43:51.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.531+0000 7f9806f27b80 20 client.19797 trim_cache size 0 max 16384 2024-02-02T15:43:51.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.558+0000 7f9806f27b80 1 Processor -- start 2024-02-02T15:43:51.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.558+0000 7f9806f27b80 1 -- start start 2024-02-02T15:43:51.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.558+0000 7f9806f27b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56314e4de010 0x56314e4e3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:51.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.558+0000 7f9806f27b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56314e4de930 0x56314e4e5b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:51.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.558+0000 7f9806f27b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56314e4e97e0 0x56314e4e6050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:51.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.558+0000 7f9806f27b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56314e2b36a0 con 0x56314e4de010 2024-02-02T15:43:51.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.559+0000 7f9806f27b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56314e4d4500 con 0x56314e4e97e0 2024-02-02T15:43:51.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.559+0000 7f9806f27b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56314e4e8eb0 con 0x56314e4de930 2024-02-02T15:43:51.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.559+0000 7f97f0a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56314e4de010 0x56314e4e3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:51.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.559+0000 7f97f1400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56314e4e97e0 0x56314e4e6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:51.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.559+0000 7f97f1400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56314e4e97e0 0x56314e4e6050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:43122/0 (socket says 172.21.15.1:43122) 2024-02-02T15:43:51.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.559+0000 7f97f1400700 1 -- 172.21.15.1:0/3256850361 learned_addr learned my addr 172.21.15.1:0/3256850361 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:43:51.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.559+0000 7f97f0a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56314e4de010 0x56314e4e3c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:51998/0 (socket says 172.21.15.1:51998) 2024-02-02T15:43:51.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.559+0000 7f97ebe00700 1 --2- 172.21.15.1:0/3256850361 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56314e4de930 0x56314e4e5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:51.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.559+0000 7f97f1400700 1 -- 172.21.15.1:0/3256850361 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56314e4de930 msgr2=0x56314e4e5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:51.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.559+0000 7f97f1400700 1 --2- 172.21.15.1:0/3256850361 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56314e4de930 0x56314e4e5b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:51.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.559+0000 7f97f1400700 1 -- 172.21.15.1:0/3256850361 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56314e4de010 msgr2=0x56314e4e3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:51.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.559+0000 7f97f1400700 1 --2- 172.21.15.1:0/3256850361 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56314e4de010 0x56314e4e3c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:51.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.559+0000 7f97f1400700 1 -- 172.21.15.1:0/3256850361 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56314e4e83f0 con 0x56314e4e97e0 2024-02-02T15:43:51.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.559+0000 7f97ebe00700 1 --2- 172.21.15.1:0/3256850361 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56314e4de930 0x56314e4e5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:43:51.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.560+0000 7f97f1400700 1 --2- 172.21.15.1:0/3256850361 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56314e4e97e0 0x56314e4e6050 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f97e401af20 tx=0x7f97e400b380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fffb076d736106dc server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:51.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.560+0000 7f97e2600700 1 -- 172.21.15.1:0/3256850361 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e4007d20 con 0x56314e4e97e0 2024-02-02T15:43:51.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.560+0000 7f97e2600700 1 -- 172.21.15.1:0/3256850361 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56314e4d20f0 con 0x56314e4e97e0 2024-02-02T15:43:51.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.560+0000 7f97e2600700 1 -- 172.21.15.1:0/3256850361 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e4004730 con 0x56314e4e97e0 2024-02-02T15:43:51.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.561+0000 7f9806f27b80 1 -- 172.21.15.1:0/3256850361 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56314e4e97e0 msgr2=0x56314e4e6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:51.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.561+0000 7f9806f27b80 1 --2- 172.21.15.1:0/3256850361 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56314e4e97e0 0x56314e4e6050 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f97e401af20 tx=0x7f97e400b380 comp rx=0 tx=0).stop 2024-02-02T15:43:51.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.561+0000 7f9806f27b80 1 -- 172.21.15.1:0/3256850361 shutdown_connections 2024-02-02T15:43:51.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.561+0000 7f9806f27b80 1 --2- 172.21.15.1:0/3256850361 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56314e4e97e0 0x56314e4e6050 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:51.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.561+0000 7f9806f27b80 1 --2- 172.21.15.1:0/3256850361 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56314e4de010 0x56314e4e3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:51.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.561+0000 7f9806f27b80 1 --2- 172.21.15.1:0/3256850361 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56314e4de930 0x56314e4e5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:51.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.561+0000 7f9806f27b80 1 -- 172.21.15.1:0/3256850361 >> 172.21.15.1:0/3256850361 conn(0x56314e43ddb0 msgr2=0x56314e4f41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:51.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.561+0000 7f9806f27b80 1 -- 172.21.15.1:0/3256850361 shutdown_connections 2024-02-02T15:43:51.562 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.561+0000 7f9806f27b80 1 -- 172.21.15.1:0/3256850361 wait complete. 2024-02-02T15:43:51.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.562+0000 7f9806f27b80 1 -- start start 2024-02-02T15:43:51.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.562+0000 7f9806f27b80 1 Processor -- start 2024-02-02T15:43:51.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.563+0000 7f9806f27b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56314e4de010 0x56314e4eb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:51.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.563+0000 7f9806f27b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56314e4de930 0x56314e4ebde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:51.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.563+0000 7f9806f27b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56314e4e97e0 0x56314e55ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:51.563 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.563+0000 7f9806f27b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56314e55b060 con 0x56314e4de010 2024-02-02T15:43:51.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.563+0000 7f9806f27b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56314e55b1a0 con 0x56314e4e97e0 2024-02-02T15:43:51.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.563+0000 7f9806f27b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56314e55b330 con 0x56314e4de930 2024-02-02T15:43:51.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.563+0000 7f9806f27b80 10 client.0 _ll_register_callbacks cb 0x56314e3cd290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:43:51.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.563+0000 7f97f0a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56314e4de010 0x56314e4eb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:51.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.564+0000 7f97f0a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56314e4de010 0x56314e4eb8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:52004/0 (socket says 172.21.15.1:52004) 2024-02-02T15:43:51.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.564+0000 7f97f0a00700 1 -- 172.21.15.1:0/1279011949 learned_addr learned my addr 172.21.15.1:0/1279011949 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:43:51.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.564+0000 7f97ebe00700 1 --2- 172.21.15.1:0/1279011949 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56314e4de930 0x56314e4ebde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:51.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.564+0000 7f97f1400700 1 --2- 172.21.15.1:0/1279011949 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56314e4e97e0 0x56314e55ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:51.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.564+0000 7f97f0a00700 1 -- 172.21.15.1:0/1279011949 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56314e4de930 msgr2=0x56314e4ebde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:51.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.564+0000 7f97f0a00700 1 --2- 172.21.15.1:0/1279011949 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56314e4de930 0x56314e4ebde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:51.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.564+0000 7f97f0a00700 1 -- 172.21.15.1:0/1279011949 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56314e4e97e0 msgr2=0x56314e55ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:43:51.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.564+0000 7f97f0a00700 1 --2- 172.21.15.1:0/1279011949 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56314e4e97e0 0x56314e55ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:51.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.564+0000 7f97f0a00700 1 -- 172.21.15.1:0/1279011949 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56314e25db60 con 0x56314e4de010 2024-02-02T15:43:51.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.565+0000 7f97f0a00700 1 --2- 172.21.15.1:0/1279011949 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56314e4de010 0x56314e4eb8a0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f97dc012420 tx=0x7f97dc004aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44cb1ba551dcb9db server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:43:51.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.565+0000 7f97da000700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:43:51.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.566+0000 7f97da000700 1 -- 172.21.15.1:0/1279011949 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97dc017040 con 0x56314e4de010 2024-02-02T15:43:51.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.566+0000 7f97da000700 1 -- 172.21.15.1:0/1279011949 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f97dc01eba0 con 0x56314e4de010 2024-02-02T15:43:51.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.566+0000 7f9806f27b80 1 -- 172.21.15.1:0/1279011949 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f97d40a0040 con 0x56314e4de010 2024-02-02T15:43:51.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.566+0000 7f97da000700 1 -- 172.21.15.1:0/1279011949 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7f97dc01ed60 con 0x56314e4de010 2024-02-02T15:43:51.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.566+0000 7f9806f27b80 10 client.19803 fetch_fsmap learned FSMap version 70 2024-02-02T15:43:51.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.566+0000 7f9806f27b80 1 -- 172.21.15.1:0/1279011949 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) v3 -- 0x56314e3f5ea0 con 0x56314e4de010 2024-02-02T15:43:51.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.567+0000 7f97da000700 1 -- 172.21.15.1:0/1279011949 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f97d40a0040 con 0x56314e4de010 2024-02-02T15:43:51.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.567+0000 7f97da000700 1 -- 172.21.15.1:0/1279011949 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 70) v1 ==== 7490+0+0 (secure 0 0 0) 0x7f97dc0127b0 con 0x56314e4de010 2024-02-02T15:43:51.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.567+0000 7f9806f27b80 10 client.19803 fetch_fsmap finished waiting for FSMap version 70 2024-02-02T15:43:51.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.567+0000 7f9806f27b80 10 client.19803 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:43:51.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.567+0000 7f9806f27b80 20 client.19803 populate_metadata read hostname 'smithi001' 2024-02-02T15:43:51.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.567+0000 7f9806f27b80 1 --2- 172.21.15.1:0/1279011949 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56314e55cb10 0x56314e55ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:43:51.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.567+0000 7f9806f27b80 4 client.19803 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2024-02-02T15:43:51.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.567+0000 7f9806f27b80 1 -- 172.21.15.1:0/1279011949 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56314e3eef50 con 0x56314e55cb10 2024-02-02T15:43:51.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.568+0000 7f97ebe00700 1 --2- 172.21.15.1:0/1279011949 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56314e55cb10 0x56314e55ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:43:51.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.568+0000 7f97ebe00700 1 --2- 172.21.15.1:0/1279011949 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56314e55cb10 0x56314e55ef30 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f39ffebd4ba71f6 server_cookie=b860eb0b644d9e24 in_seq=0 out_seq=0 2024-02-02T15:43:51.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.568+0000 7f97da000700 10 client.19803 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:43:51.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.569+0000 7f97da000700 1 -- 172.21.15.1:0/1279011949 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x56314e3eef50 con 0x56314e55cb10 2024-02-02T15:43:51.570 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.569+0000 7f97da000700 10 client.19803 handle_command_reply: tid=0 2024-02-02T15:43:51.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.569+0000 7f9806f27b80 1 client.19803 shutdown 2024-02-02T15:43:51.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.569+0000 7f9806f27b80 1 -- 172.21.15.1:0/1279011949 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56314e4de010 msgr2=0x56314e4eb8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:51.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.569+0000 7f9806f27b80 1 --2- 172.21.15.1:0/1279011949 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56314e4de010 0x56314e4eb8a0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f97dc012420 tx=0x7f97dc004aa0 comp rx=0 tx=0).stop 2024-02-02T15:43:51.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.569+0000 7f9806f27b80 1 -- 172.21.15.1:0/1279011949 shutdown_connections 2024-02-02T15:43:51.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.569+0000 7f9806f27b80 1 --2- 172.21.15.1:0/1279011949 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x56314e55cb10 0x56314e55ef30 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:51.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.569+0000 7f9806f27b80 1 --2- 172.21.15.1:0/1279011949 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56314e4e97e0 0x56314e55ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:51.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.569+0000 7f9806f27b80 1 --2- 172.21.15.1:0/1279011949 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56314e4de010 0x56314e4eb8a0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:51.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.569+0000 7f9806f27b80 1 --2- 172.21.15.1:0/1279011949 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56314e4de930 0x56314e4ebde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:51.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.569+0000 7f9806f27b80 1 -- 172.21.15.1:0/1279011949 >> 172.21.15.1:0/1279011949 conn(0x56314e43ddb0 msgr2=0x56314e4e3c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:51.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.569+0000 7f97da000700 0 client.19803 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:43:51.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.570+0000 7f9806f27b80 1 -- 172.21.15.1:0/1279011949 shutdown_connections 2024-02-02T15:43:51.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.570+0000 7f9806f27b80 1 -- 172.21.15.1:0/1279011949 wait complete. 2024-02-02T15:43:51.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.570+0000 7f9806f27b80 20 client.19803 trim_cache size 0 max 16384 2024-02-02T15:43:51.572 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:43:51.572 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-02T15:43:51.572 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2024-02-02T15:43:51.572 INFO:teuthology.orchestra.run.smithi001.stdout: "3daf8575-1db7-4128-8672-e508a4067b8d": { 2024-02-02T15:43:51.572 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2024-02-02T15:43:51.572 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "3daf8575-1db7-4128-8672-e508a4067b8d", 2024-02-02T15:43:51.572 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,force" 2024-02-02T15:43:51.572 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:43:51.573 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:43:51.573 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:43:51.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.571+0000 7f97f3400700 1 -- 172.21.15.1:0/3121698278 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f97cc0a0710 msgr2=0x7f97cc0a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:51.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.571+0000 7f97f3400700 1 --2- 172.21.15.1:0/3121698278 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f97cc0a0710 0x7f97cc0a2bd0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f97e4009fd0 tx=0x7f97e4009920 comp rx=0 tx=0).stop 2024-02-02T15:43:51.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.571+0000 7f97f3400700 1 -- 172.21.15.1:0/3121698278 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f97ec073a30 msgr2=0x7f97ec10d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:43:51.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.571+0000 7f97f3400700 1 --2- 172.21.15.1:0/3121698278 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f97ec073a30 0x7f97ec10d070 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f97d4009990 tx=0x7f97d4009580 comp rx=0 tx=0).stop 2024-02-02T15:43:51.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.571+0000 7f97f3400700 1 -- 172.21.15.1:0/3121698278 shutdown_connections 2024-02-02T15:43:51.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.571+0000 7f97f3400700 1 --2- 172.21.15.1:0/3121698278 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f97cc0a0710 0x7f97cc0a2bd0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:51.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.571+0000 7f97f3400700 1 --2- 172.21.15.1:0/3121698278 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97ec10d630 0x7f97ec109d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:51.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.571+0000 7f97f3400700 1 --2- 172.21.15.1:0/3121698278 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97ec073110 0x7f97ec10cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:51.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.571+0000 7f97f3400700 1 --2- 172.21.15.1:0/3121698278 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f97ec073a30 0x7f97ec10d070 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:43:51.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.571+0000 7f97f3400700 1 -- 172.21.15.1:0/3121698278 >> 172.21.15.1:0/3121698278 conn(0x7f97ec0fc840 msgr2=0x7f97ec100c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:43:51.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.571+0000 7f97f3400700 1 -- 172.21.15.1:0/3121698278 shutdown_connections 2024-02-02T15:43:51.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:43:51.571+0000 7f97f3400700 1 -- 172.21.15.1:0/3121698278 wait complete. 2024-02-02T15:43:51.599 INFO:tasks.cephfs.filesystem:scrub status for tag:3daf8575-1db7-4128-8672-e508a4067b8d - {'path': '/', 'tag': '3daf8575-1db7-4128-8672-e508a4067b8d', 'options': 'recursive,force'} 2024-02-02T15:43:51.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:43:51 smithi071 ceph-mon[44707]: pgmap v528: 129 pgs: 129 active+clean; 1.7 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.6 MiB/s wr, 444 op/s 2024-02-02T15:43:51.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:43:51 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:43:51.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:43:51 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:43:51.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:43:51 smithi142 ceph-mon[45728]: pgmap v528: 129 pgs: 129 active+clean; 1.7 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.6 MiB/s wr, 444 op/s 2024-02-02T15:43:51.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:43:51 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:43:51.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:43:51 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:43:51.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:43:51 smithi001 ceph-mon[35470]: pgmap v528: 129 pgs: 129 active+clean; 1.7 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.6 MiB/s wr, 444 op/s 2024-02-02T15:43:51.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:43:51 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:43:51.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:43:51 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:43:51.991 INFO:tasks.workunit.client.0.smithi001.stdout: 10 116242 29.85 MB/sec execute 84 sec latency 39.528 ms 2024-02-02T15:43:52.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:43:52 smithi071 ceph-mon[44707]: scrub queued for path: / 2024-02-02T15:43:52.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:43:52 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:43:52.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:43:52 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:43:52.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:43:52 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:43:52.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:43:52 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:43:52.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:43:52 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:43:52.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:43:52 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:43:52.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:43:52 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:43:52.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:43:52 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:43:52.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:43:52 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:43:52.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:43:52 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:43:52.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:43:52 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:43:52.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:43:52 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:43:52.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:43:52 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:43:52.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:43:52 smithi142 ceph-mon[45728]: scrub queued for path: / 2024-02-02T15:43:52.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:43:52 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:43:52.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:43:52 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:43:52.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:43:52 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:43:52.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:43:52 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:43:52.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:43:52 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:43:52.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:43:52 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:43:52.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:43:52 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:43:52.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:43:52 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:43:52.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:43:52 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:43:52.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:43:52 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:43:52.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:43:52 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:43:52.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:43:52 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:43:52.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:43:52 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:43:52.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:43:52 smithi001 ceph-mon[35470]: scrub queued for path: / 2024-02-02T15:43:52.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:43:52 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:43:52.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:43:52 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:43:52.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:43:52 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:43:52.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:43:52 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:43:52.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:43:52 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:43:52.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:43:52 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:43:52.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:43:52 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:43:52.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:43:52 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:43:52.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:43:52 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:43:52.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:43:52 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:43:52.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:43:52 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:43:52.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:43:52 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:43:52.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:43:52 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:43:52.991 INFO:tasks.workunit.client.0.smithi001.stdout: 10 116818 29.92 MB/sec execute 85 sec latency 66.020 ms 2024-02-02T15:43:53.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:43:53 smithi071 ceph-mon[44707]: pgmap v529: 129 pgs: 129 active+clean; 1.7 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 9.3 MiB/s wr, 615 op/s 2024-02-02T15:43:53.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:43:53 smithi142 ceph-mon[45728]: pgmap v529: 129 pgs: 129 active+clean; 1.7 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 9.3 MiB/s wr, 615 op/s 2024-02-02T15:43:53.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:43:53 smithi001 ceph-mon[35470]: pgmap v529: 129 pgs: 129 active+clean; 1.7 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 9.3 MiB/s wr, 615 op/s 2024-02-02T15:43:53.991 INFO:tasks.workunit.client.0.smithi001.stdout: 10 117485 29.90 MB/sec execute 86 sec latency 45.970 ms 2024-02-02T15:43:54.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:43:54 smithi142 ceph-mon[45728]: pgmap v530: 129 pgs: 129 active+clean; 1.7 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 7.6 MiB/s wr, 478 op/s 2024-02-02T15:43:54.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:43:54 smithi001 ceph-mon[35470]: pgmap v530: 129 pgs: 129 active+clean; 1.7 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 7.6 MiB/s wr, 478 op/s 2024-02-02T15:43:54.991 INFO:tasks.workunit.client.0.smithi001.stdout: 10 117927 29.83 MB/sec execute 87 sec latency 353.578 ms 2024-02-02T15:43:55.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:43:54 smithi071 ceph-mon[44707]: pgmap v530: 129 pgs: 129 active+clean; 1.7 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 7.6 MiB/s wr, 478 op/s 2024-02-02T15:43:55.992 INFO:tasks.workunit.client.0.smithi001.stdout: 10 118346 29.82 MB/sec execute 88 sec latency 100.320 ms 2024-02-02T15:43:56.992 INFO:tasks.workunit.client.0.smithi001.stdout: 10 119076 29.86 MB/sec execute 89 sec latency 59.529 ms 2024-02-02T15:43:57.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:43:57 smithi071 ceph-mon[44707]: pgmap v531: 129 pgs: 129 active+clean; 1.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 10 MiB/s wr, 583 op/s 2024-02-02T15:43:57.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:43:57 smithi142 ceph-mon[45728]: pgmap v531: 129 pgs: 129 active+clean; 1.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 10 MiB/s wr, 583 op/s 2024-02-02T15:43:57.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:43:57 smithi001 ceph-mon[35470]: pgmap v531: 129 pgs: 129 active+clean; 1.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 10 MiB/s wr, 583 op/s 2024-02-02T15:43:57.992 INFO:tasks.workunit.client.0.smithi001.stdout: 10 119759 29.95 MB/sec execute 90 sec latency 72.755 ms 2024-02-02T15:43:58.992 INFO:tasks.workunit.client.0.smithi001.stdout: 10 120359 30.03 MB/sec execute 91 sec latency 179.367 ms 2024-02-02T15:43:59.867 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:43:59 smithi071 ceph-mon[44707]: pgmap v532: 129 pgs: 129 active+clean; 1.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 592 op/s 2024-02-02T15:43:59.922 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:43:59 smithi142 ceph-mon[45728]: pgmap v532: 129 pgs: 129 active+clean; 1.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 592 op/s 2024-02-02T15:43:59.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:43:59 smithi001 ceph-mon[35470]: pgmap v532: 129 pgs: 129 active+clean; 1.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 592 op/s 2024-02-02T15:43:59.992 INFO:tasks.workunit.client.0.smithi001.stdout: 10 120725 29.89 MB/sec execute 92 sec latency 438.142 ms 2024-02-02T15:44:00.992 INFO:tasks.workunit.client.0.smithi001.stdout: 10 120815 29.64 MB/sec execute 93 sec latency 1182.599 ms 2024-02-02T15:44:01.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:44:01 smithi071 ceph-mon[44707]: pgmap v533: 129 pgs: 129 active+clean; 1.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 8.4 MiB/s wr, 458 op/s 2024-02-02T15:44:01.924 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:44:01 smithi142 ceph-mon[45728]: pgmap v533: 129 pgs: 129 active+clean; 1.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 8.4 MiB/s wr, 458 op/s 2024-02-02T15:44:01.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:44:01 smithi001 ceph-mon[35470]: pgmap v533: 129 pgs: 129 active+clean; 1.7 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 8.4 MiB/s wr, 458 op/s 2024-02-02T15:44:01.992 INFO:tasks.workunit.client.0.smithi001.stdout: 10 120914 29.35 MB/sec execute 94 sec latency 1319.385 ms 2024-02-02T15:44:02.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:44:02 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:44:02.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:44:02 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:44:02.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:44:02 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:44:02.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:44:02 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:44:02.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:44:02 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:44:02.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:44:02 smithi142 ceph-mon[45728]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:44:02.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:44:02 smithi142 ceph-mon[45728]: pgmap v534: 129 pgs: 129 active+clean; 1.8 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 18 MiB/s wr, 714 op/s 2024-02-02T15:44:02.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:44:02 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:44:02.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:44:02 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:44:02.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:44:02 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:44:02.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:44:02 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:44:02.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:44:02 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:44:02.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:44:02 smithi001 ceph-mon[35470]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:44:02.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:44:02 smithi001 ceph-mon[35470]: pgmap v534: 129 pgs: 129 active+clean; 1.8 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 18 MiB/s wr, 714 op/s 2024-02-02T15:44:02.992 INFO:tasks.workunit.client.0.smithi001.stdout: 10 121152 29.16 MB/sec execute 95 sec latency 595.966 ms 2024-02-02T15:44:03.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:44:02 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:44:03.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:44:02 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:44:03.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:44:02 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:44:03.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:44:02 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:44:03.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:44:02 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:44:03.116 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:44:02 smithi071 ceph-mon[44707]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:44:03.116 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:44:02 smithi071 ceph-mon[44707]: pgmap v534: 129 pgs: 129 active+clean; 1.8 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 18 MiB/s wr, 714 op/s 2024-02-02T15:44:03.992 INFO:tasks.workunit.client.0.smithi001.stdout: 10 121322 28.91 MB/sec execute 96 sec latency 763.282 ms 2024-02-02T15:44:04.992 INFO:tasks.workunit.client.0.smithi001.stdout: 10 121390 28.64 MB/sec execute 97 sec latency 724.878 ms 2024-02-02T15:44:05.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:44:05 smithi071 ceph-mon[44707]: pgmap v535: 129 pgs: 129 active+clean; 1.8 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 18 MiB/s wr, 626 op/s 2024-02-02T15:44:05.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:44:05 smithi142 ceph-mon[45728]: pgmap v535: 129 pgs: 129 active+clean; 1.8 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 18 MiB/s wr, 626 op/s 2024-02-02T15:44:05.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:44:05 smithi001 ceph-mon[35470]: pgmap v535: 129 pgs: 129 active+clean; 1.8 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 18 MiB/s wr, 626 op/s 2024-02-02T15:44:05.992 INFO:tasks.workunit.client.0.smithi001.stdout: 10 121448 28.42 MB/sec execute 98 sec latency 1045.769 ms 2024-02-02T15:44:06.992 INFO:tasks.workunit.client.0.smithi001.stdout: 10 121816 28.40 MB/sec execute 99 sec latency 1199.918 ms 2024-02-02T15:44:07.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:44:07 smithi071 ceph-mon[44707]: pgmap v536: 129 pgs: 129 active+clean; 1.8 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 22 MiB/s wr, 816 op/s 2024-02-02T15:44:07.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:44:07 smithi071 ceph-mon[44707]: osdmap e110: 12 total, 12 up, 12 in 2024-02-02T15:44:07.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:44:07 smithi142 ceph-mon[45728]: pgmap v536: 129 pgs: 129 active+clean; 1.8 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 22 MiB/s wr, 816 op/s 2024-02-02T15:44:07.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:44:07 smithi142 ceph-mon[45728]: osdmap e110: 12 total, 12 up, 12 in 2024-02-02T15:44:07.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:44:07 smithi001 ceph-mon[35470]: pgmap v536: 129 pgs: 129 active+clean; 1.8 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 22 MiB/s wr, 816 op/s 2024-02-02T15:44:07.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:44:07 smithi001 ceph-mon[35470]: osdmap e110: 12 total, 12 up, 12 in 2024-02-02T15:44:07.992 INFO:tasks.workunit.client.0.smithi001.stdout: 10 122112 28.25 MB/sec execute 100 sec latency 123.673 ms 2024-02-02T15:44:08.992 INFO:tasks.workunit.client.0.smithi001.stdout: 10 122685 28.24 MB/sec execute 101 sec latency 75.486 ms 2024-02-02T15:44:09.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:44:09 smithi071 ceph-mon[44707]: pgmap v538: 129 pgs: 1 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 122 active+clean; 1.8 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 25 MiB/s wr, 893 op/s 2024-02-02T15:44:09.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:44:09 smithi071 ceph-mon[44707]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:44:09.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:44:09 smithi142 ceph-mon[45728]: pgmap v538: 129 pgs: 1 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 122 active+clean; 1.8 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 25 MiB/s wr, 893 op/s 2024-02-02T15:44:09.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:44:09 smithi142 ceph-mon[45728]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:44:09.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:44:09 smithi001 ceph-mon[35470]: pgmap v538: 129 pgs: 1 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 122 active+clean; 1.8 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 25 MiB/s wr, 893 op/s 2024-02-02T15:44:09.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:44:09 smithi001 ceph-mon[35470]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:44:09.992 INFO:tasks.workunit.client.0.smithi001.stdout: 10 123323 28.38 MB/sec execute 102 sec latency 91.875 ms 2024-02-02T15:44:10.992 INFO:tasks.workunit.client.0.smithi001.stdout: 10 123944 28.37 MB/sec execute 103 sec latency 794.394 ms 2024-02-02T15:44:11.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:44:11 smithi071 ceph-mon[44707]: pgmap v539: 129 pgs: 1 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 122 active+clean; 1.8 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 25 MiB/s wr, 893 op/s 2024-02-02T15:44:11.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:44:11 smithi142 ceph-mon[45728]: pgmap v539: 129 pgs: 1 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 122 active+clean; 1.8 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 25 MiB/s wr, 893 op/s 2024-02-02T15:44:11.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:44:11 smithi001 ceph-mon[35470]: pgmap v539: 129 pgs: 1 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 122 active+clean; 1.8 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 25 MiB/s wr, 893 op/s 2024-02-02T15:44:11.992 INFO:tasks.workunit.client.0.smithi001.stdout: 10 124488 28.37 MB/sec execute 104 sec latency 1794.455 ms 2024-02-02T15:44:12.993 INFO:tasks.workunit.client.0.smithi001.stdout: 10 125067 28.42 MB/sec execute 105 sec latency 2794.554 ms 2024-02-02T15:44:13.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:44:13 smithi071 ceph-mon[44707]: pgmap v540: 129 pgs: 1 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 122 active+clean; 1.8 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 15 MiB/s wr, 789 op/s 2024-02-02T15:44:13.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:44:13 smithi142 ceph-mon[45728]: pgmap v540: 129 pgs: 1 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 122 active+clean; 1.8 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 15 MiB/s wr, 789 op/s 2024-02-02T15:44:13.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:44:13 smithi001 ceph-mon[35470]: pgmap v540: 129 pgs: 1 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 122 active+clean; 1.8 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 15 MiB/s wr, 789 op/s 2024-02-02T15:44:13.993 INFO:tasks.workunit.client.0.smithi001.stdout: 10 125677 28.46 MB/sec execute 106 sec latency 3794.603 ms 2024-02-02T15:44:14.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:44:14 smithi071 ceph-mon[44707]: pgmap v541: 129 pgs: 129 active+clean; 1.8 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 13 MiB/s wr, 780 op/s 2024-02-02T15:44:14.922 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:44:14 smithi142 ceph-mon[45728]: pgmap v541: 129 pgs: 129 active+clean; 1.8 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 13 MiB/s wr, 780 op/s 2024-02-02T15:44:14.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:44:14 smithi001 ceph-mon[35470]: pgmap v541: 129 pgs: 129 active+clean; 1.8 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 13 MiB/s wr, 780 op/s 2024-02-02T15:44:14.993 INFO:tasks.workunit.client.0.smithi001.stdout: 10 126232 28.44 MB/sec execute 107 sec latency 4794.664 ms 2024-02-02T15:44:15.993 INFO:tasks.workunit.client.0.smithi001.stdout: 10 126887 28.60 MB/sec execute 108 sec latency 5794.730 ms 2024-02-02T15:44:16.993 INFO:tasks.workunit.client.0.smithi001.stdout: 10 127408 28.51 MB/sec execute 109 sec latency 5817.683 ms 2024-02-02T15:44:17.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:44:17 smithi071 ceph-mon[44707]: pgmap v542: 129 pgs: 129 active+clean; 1.8 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 9.8 MiB/s wr, 633 op/s 2024-02-02T15:44:17.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:44:17 smithi071 ceph-mon[44707]: osdmap e111: 12 total, 12 up, 12 in 2024-02-02T15:44:17.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:44:17 smithi071 ceph-mon[44707]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:44:17.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:44:17 smithi142 ceph-mon[45728]: pgmap v542: 129 pgs: 129 active+clean; 1.8 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 9.8 MiB/s wr, 633 op/s 2024-02-02T15:44:17.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:44:17 smithi142 ceph-mon[45728]: osdmap e111: 12 total, 12 up, 12 in 2024-02-02T15:44:17.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:44:17 smithi142 ceph-mon[45728]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:44:17.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:44:17 smithi001 ceph-mon[35470]: pgmap v542: 129 pgs: 129 active+clean; 1.8 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 9.8 MiB/s wr, 633 op/s 2024-02-02T15:44:17.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:44:17 smithi001 ceph-mon[35470]: osdmap e111: 12 total, 12 up, 12 in 2024-02-02T15:44:17.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:44:17 smithi001 ceph-mon[35470]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:44:17.993 INFO:tasks.workunit.client.0.smithi001.stdout: 10 127911 28.52 MB/sec execute 110 sec latency 4085.104 ms 2024-02-02T15:44:18.993 INFO:tasks.workunit.client.0.smithi001.stdout: 10 128626 28.60 MB/sec execute 111 sec latency 5085.157 ms 2024-02-02T15:44:19.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:44:19 smithi071 ceph-mon[44707]: pgmap v544: 129 pgs: 129 active+clean; 1.8 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 7.2 MiB/s wr, 543 op/s 2024-02-02T15:44:19.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:44:19 smithi142 ceph-mon[45728]: pgmap v544: 129 pgs: 129 active+clean; 1.8 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 7.2 MiB/s wr, 543 op/s 2024-02-02T15:44:19.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:44:19 smithi001 ceph-mon[35470]: pgmap v544: 129 pgs: 129 active+clean; 1.8 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 7.2 MiB/s wr, 543 op/s 2024-02-02T15:44:19.993 INFO:tasks.workunit.client.0.smithi001.stdout: 10 129191 28.66 MB/sec execute 112 sec latency 6085.222 ms 2024-02-02T15:44:20.993 INFO:tasks.workunit.client.0.smithi001.stdout: 10 129671 28.62 MB/sec execute 113 sec latency 7085.309 ms 2024-02-02T15:44:21.600 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-02T15:44:21.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:44:21 smithi071 ceph-mon[44707]: pgmap v545: 129 pgs: 129 active+clean; 1.8 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 7.2 MiB/s wr, 543 op/s 2024-02-02T15:44:21.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.894+0000 7f128fc00700 1 -- 172.21.15.1:0/2459596981 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1288100740 msgr2=0x7f1288104c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:21.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.894+0000 7f128fc00700 1 --2- 172.21.15.1:0/2459596981 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1288100740 0x7f1288104c00 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f1270009990 tx=0x7f1270009ca0 comp rx=0 tx=0).stop 2024-02-02T15:44:21.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.895+0000 7f128fc00700 1 -- 172.21.15.1:0/2459596981 shutdown_connections 2024-02-02T15:44:21.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.895+0000 7f128fc00700 1 --2- 172.21.15.1:0/2459596981 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1288105220 0x7f12881056d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:21.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.895+0000 7f128fc00700 1 --2- 172.21.15.1:0/2459596981 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f12880ffe20 0x7f1288100200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:21.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.895+0000 7f128fc00700 1 --2- 172.21.15.1:0/2459596981 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1288100740 0x7f1288104c00 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:21.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.895+0000 7f128fc00700 1 -- 172.21.15.1:0/2459596981 >> 172.21.15.1:0/2459596981 conn(0x7f1288076d70 msgr2=0x7f1288077180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:21.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.895+0000 7f128fc00700 1 -- 172.21.15.1:0/2459596981 shutdown_connections 2024-02-02T15:44:21.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.895+0000 7f128fc00700 1 -- 172.21.15.1:0/2459596981 wait complete. 2024-02-02T15:44:21.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.897+0000 7f128fc00700 1 Processor -- start 2024-02-02T15:44:21.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.898+0000 7f128fc00700 1 -- start start 2024-02-02T15:44:21.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.898+0000 7f128fc00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12880ffe20 0x7f128819d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:21.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.898+0000 7f128fc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1288100740 0x7f128819da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:21.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.898+0000 7f128fc00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1288105220 0x7f12881a1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:21.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.898+0000 7f128fc00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12881a2510 con 0x7f12880ffe20 2024-02-02T15:44:21.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.898+0000 7f128fc00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12881a2680 con 0x7f1288105220 2024-02-02T15:44:21.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.898+0000 7f128fc00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12881a2840 con 0x7f1288100740 2024-02-02T15:44:21.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.899+0000 7f1287e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1288100740 0x7f128819da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:21.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.899+0000 7f1287e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1288100740 0x7f128819da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:58576/0 (socket says 172.21.15.1:58576) 2024-02-02T15:44:21.899 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.899+0000 7f1287e00700 1 -- 172.21.15.1:0/4266572140 learned_addr learned my addr 172.21.15.1:0/4266572140 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:44:21.900 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.900+0000 7f128dc00700 1 --2- 172.21.15.1:0/4266572140 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1288105220 0x7f12881a1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:21.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.901+0000 7f128dc00700 1 -- 172.21.15.1:0/4266572140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1288100740 msgr2=0x7f128819da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:21.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.901+0000 7f128dc00700 1 --2- 172.21.15.1:0/4266572140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1288100740 0x7f128819da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:21.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.901+0000 7f128dc00700 1 -- 172.21.15.1:0/4266572140 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12880ffe20 msgr2=0x7f128819d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:21.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.901+0000 7f128dc00700 1 --2- 172.21.15.1:0/4266572140 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12880ffe20 0x7f128819d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:21.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.901+0000 7f128dc00700 1 -- 172.21.15.1:0/4266572140 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12700095f0 con 0x7f1288105220 2024-02-02T15:44:21.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.901+0000 7f128dc00700 1 --2- 172.21.15.1:0/4266572140 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1288105220 0x7f12881a1ed0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f128000be20 tx=0x7f128000bf00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:21.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.902+0000 7f1285600700 1 -- 172.21.15.1:0/4266572140 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f128000cc30 con 0x7f1288105220 2024-02-02T15:44:21.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.902+0000 7f1285600700 1 -- 172.21.15.1:0/4266572140 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f128000cd90 con 0x7f1288105220 2024-02-02T15:44:21.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.902+0000 7f1285600700 1 -- 172.21.15.1:0/4266572140 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1280012800 con 0x7f1288105220 2024-02-02T15:44:21.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.902+0000 7f128fc00700 1 -- 172.21.15.1:0/4266572140 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12881a2b20 con 0x7f1288105220 2024-02-02T15:44:21.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.903+0000 7f128fc00700 1 -- 172.21.15.1:0/4266572140 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f128806a940 con 0x7f1288105220 2024-02-02T15:44:21.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.909+0000 7f1285600700 1 -- 172.21.15.1:0/4266572140 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f1280014b00 con 0x7f1288105220 2024-02-02T15:44:21.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.911+0000 7f1285600700 1 --2- 172.21.15.1:0/4266572140 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f12680a07e0 0x7f12680a2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:21.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.912+0000 7f128d200700 1 --2- 172.21.15.1:0/4266572140 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f12680a07e0 0x7f12680a2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:21.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.912+0000 7f12a368eb80 1 Processor -- start 2024-02-02T15:44:21.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.912+0000 7f1285600700 1 -- 172.21.15.1:0/4266572140 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f1280006e40 con 0x7f1288105220 2024-02-02T15:44:21.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.912+0000 7f12a368eb80 1 -- start start 2024-02-02T15:44:21.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.913+0000 7f12a368eb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c8e8b96010 0x55c8e8b963f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:21.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.913+0000 7f12a368eb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8e8b96930 0x55c8e8ba12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:21.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.913+0000 7f12a368eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c8e8ba17e0 0x55c8e8ba3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:21.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.913+0000 7f12a368eb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8e8b0a4c0 con 0x55c8e8b96010 2024-02-02T15:44:21.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.913+0000 7f12a368eb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8e8aeeaa0 con 0x55c8e8b96930 2024-02-02T15:44:21.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.913+0000 7f12a368eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8e8aec460 con 0x55c8e8ba17e0 2024-02-02T15:44:21.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.913+0000 7f1287e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8e8b96930 0x55c8e8ba12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:21.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.913+0000 7f1287e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8e8b96930 0x55c8e8ba12a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42280/0 (socket says 172.21.15.1:42280) 2024-02-02T15:44:21.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.913+0000 7f1287e00700 1 -- 172.21.15.1:0/4218578644 learned_addr learned my addr 172.21.15.1:0/4218578644 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:44:21.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.914+0000 7f128d200700 1 --2- 172.21.15.1:0/4266572140 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f12680a07e0 0x7f12680a2ca0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f1278005950 tx=0x7f12780058e0 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:21.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.914+0000 7f128d200700 1 --2- 172.21.15.1:0/4218578644 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c8e8b96010 0x55c8e8b963f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:21.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.915+0000 7f128dc00700 1 --2- 172.21.15.1:0/4218578644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c8e8ba17e0 0x55c8e8ba3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:21.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.915+0000 7f1287e00700 1 -- 172.21.15.1:0/4218578644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c8e8ba17e0 msgr2=0x55c8e8ba3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:21.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.915+0000 7f1287e00700 1 --2- 172.21.15.1:0/4218578644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c8e8ba17e0 0x55c8e8ba3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:21.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.915+0000 7f1287e00700 1 -- 172.21.15.1:0/4218578644 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c8e8b96010 msgr2=0x55c8e8b963f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:21.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.915+0000 7f1287e00700 1 --2- 172.21.15.1:0/4218578644 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c8e8b96010 0x55c8e8b963f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:21.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.915+0000 7f1287e00700 1 -- 172.21.15.1:0/4218578644 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c8e89693b0 con 0x55c8e8b96930 2024-02-02T15:44:21.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.915+0000 7f128dc00700 1 --2- 172.21.15.1:0/4218578644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c8e8ba17e0 0x55c8e8ba3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:44:21.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.917+0000 7f1287e00700 1 --2- 172.21.15.1:0/4218578644 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8e8b96930 0x55c8e8ba12a0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f12700050a0 tx=0x7f12700050d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5c60fe305651bf9 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:21.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:44:21 smithi142 ceph-mon[45728]: pgmap v545: 129 pgs: 129 active+clean; 1.8 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 7.2 MiB/s wr, 543 op/s 2024-02-02T15:44:21.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.918+0000 7f127e000700 1 -- 172.21.15.1:0/4218578644 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f127000b7f0 con 0x55c8e8b96930 2024-02-02T15:44:21.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.918+0000 7f127e000700 1 -- 172.21.15.1:0/4218578644 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f127001e040 con 0x55c8e8b96930 2024-02-02T15:44:21.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.918+0000 7f127e000700 1 -- 172.21.15.1:0/4218578644 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1270004ea0 con 0x55c8e8b96930 2024-02-02T15:44:21.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.919+0000 7f12a368eb80 1 -- 172.21.15.1:0/4218578644 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8e8b96930 msgr2=0x55c8e8ba12a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:21.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.919+0000 7f12a368eb80 1 --2- 172.21.15.1:0/4218578644 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8e8b96930 0x55c8e8ba12a0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f12700050a0 tx=0x7f12700050d0 comp rx=0 tx=0).stop 2024-02-02T15:44:21.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.920+0000 7f12a368eb80 1 -- 172.21.15.1:0/4218578644 shutdown_connections 2024-02-02T15:44:21.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.920+0000 7f12a368eb80 1 --2- 172.21.15.1:0/4218578644 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c8e8ba17e0 0x55c8e8ba3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:21.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.920+0000 7f12a368eb80 1 --2- 172.21.15.1:0/4218578644 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c8e8b96010 0x55c8e8b963f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:21.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.920+0000 7f12a368eb80 1 --2- 172.21.15.1:0/4218578644 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8e8b96930 0x55c8e8ba12a0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:21.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.920+0000 7f12a368eb80 1 -- 172.21.15.1:0/4218578644 >> 172.21.15.1:0/4218578644 conn(0x55c8e8af5db0 msgr2=0x55c8e8b923b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:21.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.920+0000 7f12a368eb80 1 -- 172.21.15.1:0/4218578644 shutdown_connections 2024-02-02T15:44:21.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.921+0000 7f12a368eb80 1 -- 172.21.15.1:0/4218578644 wait complete. 2024-02-02T15:44:21.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.921+0000 7f12a368eb80 1 -- start start 2024-02-02T15:44:21.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.922+0000 7f12a368eb80 1 Processor -- start 2024-02-02T15:44:21.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.922+0000 7f12a368eb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c8e8b96010 0x55c8e8ba1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:21.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.922+0000 7f12a368eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c8e8b96930 0x55c8e8b9e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:21.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.922+0000 7f12a368eb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8e8ba17e0 0x55c8e8b9eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:21.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.923+0000 7f12a368eb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8e8ae7d50 con 0x55c8e8b96010 2024-02-02T15:44:21.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.923+0000 7f12a368eb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8e8a83d80 con 0x55c8e8ba17e0 2024-02-02T15:44:21.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.923+0000 7f12a368eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8e8a83a20 con 0x55c8e8b96930 2024-02-02T15:44:21.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.923+0000 7f12a368eb80 10 client.0 _ll_register_callbacks cb 0x55c8e8a85290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:44:21.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.923+0000 7f128dc00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8e8ba17e0 0x55c8e8b9eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:21.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.923+0000 7f128dc00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8e8ba17e0 0x55c8e8b9eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42296/0 (socket says 172.21.15.1:42296) 2024-02-02T15:44:21.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.923+0000 7f128dc00700 1 -- 172.21.15.1:0/2610200060 learned_addr learned my addr 172.21.15.1:0/2610200060 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:44:21.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.923+0000 7f1287e00700 1 --2- 172.21.15.1:0/2610200060 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c8e8b96930 0x55c8e8b9e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:21.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.923+0000 7f128d200700 1 --2- 172.21.15.1:0/2610200060 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c8e8b96010 0x55c8e8ba1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:21.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.923+0000 7f128dc00700 1 -- 172.21.15.1:0/2610200060 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c8e8b96930 msgr2=0x55c8e8b9e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:21.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.923+0000 7f128dc00700 1 --2- 172.21.15.1:0/2610200060 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c8e8b96930 0x55c8e8b9e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:21.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.923+0000 7f128dc00700 1 -- 172.21.15.1:0/2610200060 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c8e8b96010 msgr2=0x55c8e8ba1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:21.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.923+0000 7f128dc00700 1 --2- 172.21.15.1:0/2610200060 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c8e8b96010 0x55c8e8ba1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:21.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.923+0000 7f128dc00700 1 -- 172.21.15.1:0/2610200060 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1288114b50 con 0x55c8e8ba17e0 2024-02-02T15:44:21.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.923+0000 7f128d200700 1 --2- 172.21.15.1:0/2610200060 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c8e8b96010 0x55c8e8ba1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:44:21.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.924+0000 7f128dc00700 1 --2- 172.21.15.1:0/2610200060 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8e8ba17e0 0x55c8e8b9eb60 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f128001e030 tx=0x7f128009c8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68ca892deebfa37 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:21.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.924+0000 7f1276a00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:44:21.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.924+0000 7f1276a00700 1 -- 172.21.15.1:0/2610200060 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12800b8130 con 0x55c8e8ba17e0 2024-02-02T15:44:21.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.924+0000 7f1276a00700 1 -- 172.21.15.1:0/2610200060 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f1280097fb0 con 0x55c8e8ba17e0 2024-02-02T15:44:21.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.924+0000 7f12a368eb80 1 -- 172.21.15.1:0/2610200060 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c8e8b8a0f0 con 0x55c8e8ba17e0 2024-02-02T15:44:21.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.925+0000 7f1276a00700 1 -- 172.21.15.1:0/2610200060 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1280098170 con 0x55c8e8ba17e0 2024-02-02T15:44:21.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.925+0000 7f12a368eb80 10 client.30053 fetch_fsmap learned FSMap version 73 2024-02-02T15:44:21.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.925+0000 7f12a368eb80 1 -- 172.21.15.1:0/2610200060 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) v3 -- 0x55c8e8915b60 con 0x55c8e8ba17e0 2024-02-02T15:44:21.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.926+0000 7f1276a00700 1 -- 172.21.15.1:0/2610200060 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c8e8b8a0f0 con 0x55c8e8ba17e0 2024-02-02T15:44:21.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.926+0000 7f1276a00700 1 -- 172.21.15.1:0/2610200060 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 73) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f12800b3030 con 0x55c8e8ba17e0 2024-02-02T15:44:21.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.926+0000 7f12a368eb80 10 client.30053 fetch_fsmap finished waiting for FSMap version 73 2024-02-02T15:44:21.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.926+0000 7f12a368eb80 10 client.30053 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:44:21.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.926+0000 7f12a368eb80 20 client.30053 populate_metadata read hostname 'smithi001' 2024-02-02T15:44:21.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.926+0000 7f12a368eb80 1 --2- 172.21.15.1:0/2610200060 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55c8e8bb11c0 0x55c8e8bb35e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:21.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.926+0000 7f12a368eb80 4 client.30053 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:44:21.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.926+0000 7f12a368eb80 1 -- 172.21.15.1:0/2610200060 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c8e8ae0dd0 con 0x55c8e8bb11c0 2024-02-02T15:44:21.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.926+0000 7f128d200700 1 --2- 172.21.15.1:0/2610200060 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55c8e8bb11c0 0x55c8e8bb35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:21.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.928+0000 7f128d200700 1 --2- 172.21.15.1:0/2610200060 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55c8e8bb11c0 0x55c8e8bb35e0 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14397d8e43de8d17 server_cookie=4b7b6df87af5475b in_seq=0 out_seq=0 2024-02-02T15:44:21.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.928+0000 7f1276a00700 10 client.30053 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:44:21.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.930+0000 7f1276a00700 1 -- 172.21.15.1:0/2610200060 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55c8e8ae0dd0 con 0x55c8e8bb11c0 2024-02-02T15:44:21.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.930+0000 7f1276a00700 10 client.30053 handle_command_reply: tid=0 2024-02-02T15:44:21.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.931+0000 7f12a368eb80 1 client.30053 shutdown 2024-02-02T15:44:21.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.931+0000 7f12a368eb80 1 -- 172.21.15.1:0/2610200060 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8e8ba17e0 msgr2=0x55c8e8b9eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:21.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.931+0000 7f12a368eb80 1 --2- 172.21.15.1:0/2610200060 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8e8ba17e0 0x55c8e8b9eb60 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f128001e030 tx=0x7f128009c8f0 comp rx=0 tx=0).stop 2024-02-02T15:44:21.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.931+0000 7f12a368eb80 1 -- 172.21.15.1:0/2610200060 shutdown_connections 2024-02-02T15:44:21.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.931+0000 7f12a368eb80 1 --2- 172.21.15.1:0/2610200060 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55c8e8bb11c0 0x55c8e8bb35e0 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:21.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.931+0000 7f12a368eb80 1 --2- 172.21.15.1:0/2610200060 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8e8ba17e0 0x55c8e8b9eb60 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:21.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.931+0000 7f12a368eb80 1 --2- 172.21.15.1:0/2610200060 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c8e8b96010 0x55c8e8ba1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:21.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.931+0000 7f1276a00700 0 client.30053 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:44:21.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.931+0000 7f12a368eb80 1 --2- 172.21.15.1:0/2610200060 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c8e8b96930 0x55c8e8b9e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:21.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.931+0000 7f12a368eb80 1 -- 172.21.15.1:0/2610200060 >> 172.21.15.1:0/2610200060 conn(0x55c8e8af5db0 msgr2=0x55c8e8b91fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:21.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.934+0000 7f12a368eb80 1 -- 172.21.15.1:0/2610200060 shutdown_connections 2024-02-02T15:44:21.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.935+0000 7f12a368eb80 1 -- 172.21.15.1:0/2610200060 wait complete. 2024-02-02T15:44:21.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.936+0000 7f12a368eb80 20 client.30053 trim_cache size 0 max 16384 2024-02-02T15:44:21.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:44:21 smithi001 ceph-mon[35470]: pgmap v545: 129 pgs: 129 active+clean; 1.8 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 7.2 MiB/s wr, 543 op/s 2024-02-02T15:44:21.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.967+0000 7f12a368eb80 1 Processor -- start 2024-02-02T15:44:21.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.967+0000 7f12a368eb80 1 -- start start 2024-02-02T15:44:21.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.967+0000 7f12a368eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c8e8b96010 0x55c8e8ba82d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:21.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.967+0000 7f12a368eb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8e8b96930 0x55c8e8bad520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:21.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.967+0000 7f12a368eb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c8e8ba17e0 0x55c8e8ba0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:21.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.967+0000 7f12a368eb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8e896b6a0 con 0x55c8e8ba17e0 2024-02-02T15:44:21.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.967+0000 7f12a368eb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8e8b992c0 con 0x55c8e8b96930 2024-02-02T15:44:21.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.968+0000 7f12a368eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8e8bb1020 con 0x55c8e8b96010 2024-02-02T15:44:21.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.968+0000 7f128d200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c8e8b96010 0x55c8e8ba82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:21.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.968+0000 7f128d200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c8e8b96010 0x55c8e8ba82d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:58608/0 (socket says 172.21.15.1:58608) 2024-02-02T15:44:21.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.968+0000 7f128d200700 1 -- 172.21.15.1:0/1757896664 learned_addr learned my addr 172.21.15.1:0/1757896664 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:44:21.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.968+0000 7f1287e00700 1 --2- 172.21.15.1:0/1757896664 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8e8b96930 0x55c8e8bad520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:21.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.968+0000 7f128d200700 1 -- 172.21.15.1:0/1757896664 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8e8b96930 msgr2=0x55c8e8bad520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:21.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.968+0000 7f128d200700 1 --2- 172.21.15.1:0/1757896664 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8e8b96930 0x55c8e8bad520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:21.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.968+0000 7f128d200700 1 -- 172.21.15.1:0/1757896664 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c8e8ba17e0 msgr2=0x55c8e8ba0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:21.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.968+0000 7f128d200700 1 --2- 172.21.15.1:0/1757896664 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c8e8ba17e0 0x55c8e8ba0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:21.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.968+0000 7f128d200700 1 -- 172.21.15.1:0/1757896664 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c8e8aadea0 con 0x55c8e8b96010 2024-02-02T15:44:21.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.969+0000 7f128d200700 1 --2- 172.21.15.1:0/1757896664 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c8e8b96010 0x55c8e8ba82d0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x55c8e8bb3ec0 tx=0x7f1278013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6c45d0f50649da5 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:21.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.969+0000 7f127ea00700 1 -- 172.21.15.1:0/1757896664 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1278022040 con 0x55c8e8b96010 2024-02-02T15:44:21.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.969+0000 7f127ea00700 1 -- 172.21.15.1:0/1757896664 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f127800ee20 con 0x55c8e8b96010 2024-02-02T15:44:21.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.969+0000 7f127ea00700 1 -- 172.21.15.1:0/1757896664 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1278005280 con 0x55c8e8b96010 2024-02-02T15:44:21.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.970+0000 7f12a368eb80 1 -- 172.21.15.1:0/1757896664 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c8e8b96010 msgr2=0x55c8e8ba82d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:21.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.970+0000 7f12a368eb80 1 --2- 172.21.15.1:0/1757896664 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c8e8b96010 0x55c8e8ba82d0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x55c8e8bb3ec0 tx=0x7f1278013000 comp rx=0 tx=0).stop 2024-02-02T15:44:21.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.970+0000 7f12a368eb80 1 -- 172.21.15.1:0/1757896664 shutdown_connections 2024-02-02T15:44:21.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.970+0000 7f12a368eb80 1 --2- 172.21.15.1:0/1757896664 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c8e8ba17e0 0x55c8e8ba0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:21.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.970+0000 7f12a368eb80 1 --2- 172.21.15.1:0/1757896664 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c8e8b96010 0x55c8e8ba82d0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:21.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.970+0000 7f12a368eb80 1 --2- 172.21.15.1:0/1757896664 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8e8b96930 0x55c8e8bad520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:21.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.970+0000 7f12a368eb80 1 -- 172.21.15.1:0/1757896664 >> 172.21.15.1:0/1757896664 conn(0x55c8e8af5db0 msgr2=0x55c8e8b90b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:21.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.970+0000 7f12a368eb80 1 -- 172.21.15.1:0/1757896664 shutdown_connections 2024-02-02T15:44:21.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.970+0000 7f12a368eb80 1 -- 172.21.15.1:0/1757896664 wait complete. 2024-02-02T15:44:21.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.971+0000 7f12a368eb80 1 -- start start 2024-02-02T15:44:21.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.971+0000 7f12a368eb80 1 Processor -- start 2024-02-02T15:44:21.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.972+0000 7f12a368eb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8e8b96010 0x55c8e8ba82d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:21.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.972+0000 7f12a368eb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c8e8b96930 0x55c8e8bfae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:21.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.972+0000 7f12a368eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c8e8ba17e0 0x55c8e8bfb390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:21.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.972+0000 7f12a368eb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8e8bfbaa0 con 0x55c8e8b96930 2024-02-02T15:44:21.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.972+0000 7f12a368eb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8e8bfbc10 con 0x55c8e8b96010 2024-02-02T15:44:21.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.972+0000 7f12a368eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8e8c13ce0 con 0x55c8e8ba17e0 2024-02-02T15:44:21.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.972+0000 7f128d200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8e8b96010 0x55c8e8ba82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:21.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.972+0000 7f128d200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8e8b96010 0x55c8e8ba82d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42318/0 (socket says 172.21.15.1:42318) 2024-02-02T15:44:21.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.972+0000 7f128d200700 1 -- 172.21.15.1:0/3979747275 learned_addr learned my addr 172.21.15.1:0/3979747275 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:44:21.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.972+0000 7f12a368eb80 10 client.0 _ll_register_callbacks cb 0x55c8e8a85290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:44:21.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.972+0000 7f128dc00700 1 --2- 172.21.15.1:0/3979747275 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c8e8ba17e0 0x55c8e8bfb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:21.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.973+0000 7f1287e00700 1 --2- 172.21.15.1:0/3979747275 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c8e8b96930 0x55c8e8bfae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:21.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.973+0000 7f128d200700 1 -- 172.21.15.1:0/3979747275 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c8e8ba17e0 msgr2=0x55c8e8bfb390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:21.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.973+0000 7f128d200700 1 --2- 172.21.15.1:0/3979747275 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c8e8ba17e0 0x55c8e8bfb390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:21.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.973+0000 7f128d200700 1 -- 172.21.15.1:0/3979747275 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c8e8b96930 msgr2=0x55c8e8bfae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:21.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.973+0000 7f128d200700 1 --2- 172.21.15.1:0/3979747275 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c8e8b96930 0x55c8e8bfae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:21.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.973+0000 7f128d200700 1 -- 172.21.15.1:0/3979747275 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c8e896dcd0 con 0x55c8e8b96010 2024-02-02T15:44:21.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.973+0000 7f1287e00700 1 --2- 172.21.15.1:0/3979747275 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c8e8b96930 0x55c8e8bfae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:44:21.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.973+0000 7f128d200700 1 --2- 172.21.15.1:0/3979747275 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8e8b96010 0x55c8e8ba82d0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f12780092a0 tx=0x7f12780242b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7925102d7bffba29 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:21.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.974+0000 7f1276a00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:44:21.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.974+0000 7f1276a00700 1 -- 172.21.15.1:0/3979747275 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1278024c80 con 0x55c8e8b96010 2024-02-02T15:44:21.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.974+0000 7f1276a00700 1 -- 172.21.15.1:0/3979747275 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f1278024de0 con 0x55c8e8b96010 2024-02-02T15:44:21.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.974+0000 7f12a368eb80 1 -- 172.21.15.1:0/3979747275 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f127001e040 con 0x55c8e8b96010 2024-02-02T15:44:21.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.974+0000 7f1276a00700 1 -- 172.21.15.1:0/3979747275 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7f127802e460 con 0x55c8e8b96010 2024-02-02T15:44:21.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.974+0000 7f12a368eb80 10 client.30059 fetch_fsmap learned FSMap version 73 2024-02-02T15:44:21.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.974+0000 7f12a368eb80 1 -- 172.21.15.1:0/3979747275 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) v3 -- 0x55c8e8914030 con 0x55c8e8b96010 2024-02-02T15:44:21.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.975+0000 7f1276a00700 1 -- 172.21.15.1:0/3979747275 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f127001e040 con 0x55c8e8b96010 2024-02-02T15:44:21.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.975+0000 7f1276a00700 1 -- 172.21.15.1:0/3979747275 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 73) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1278034020 con 0x55c8e8b96010 2024-02-02T15:44:21.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.975+0000 7f12a368eb80 10 client.30059 fetch_fsmap finished waiting for FSMap version 73 2024-02-02T15:44:21.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.975+0000 7f12a368eb80 10 client.30059 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:44:21.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.975+0000 7f12a368eb80 20 client.30059 populate_metadata read hostname 'smithi001' 2024-02-02T15:44:21.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.975+0000 7f12a368eb80 1 --2- 172.21.15.1:0/3979747275 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55c8e8c15310 0x55c8e8c17700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:21.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.975+0000 7f12a368eb80 4 client.30059 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2024-02-02T15:44:21.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.975+0000 7f12a368eb80 1 -- 172.21.15.1:0/3979747275 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c8e8aa6f50 con 0x55c8e8c15310 2024-02-02T15:44:21.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.975+0000 7f1287e00700 1 --2- 172.21.15.1:0/3979747275 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55c8e8c15310 0x55c8e8c17700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:21.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.976+0000 7f1287e00700 1 --2- 172.21.15.1:0/3979747275 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55c8e8c15310 0x55c8e8c17700 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21dde6a8d72d8c3e server_cookie=f09eea305fca4c79 in_seq=0 out_seq=0 2024-02-02T15:44:21.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.976+0000 7f1276a00700 10 client.30059 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:44:21.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.976+0000 7f1276a00700 1 -- 172.21.15.1:0/3979747275 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55c8e8aa6f50 con 0x55c8e8c15310 2024-02-02T15:44:21.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.976+0000 7f1276a00700 10 client.30059 handle_command_reply: tid=0 2024-02-02T15:44:21.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.976+0000 7f12a368eb80 1 client.30059 shutdown 2024-02-02T15:44:21.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.976+0000 7f12a368eb80 1 -- 172.21.15.1:0/3979747275 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8e8b96010 msgr2=0x55c8e8ba82d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:21.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.976+0000 7f12a368eb80 1 --2- 172.21.15.1:0/3979747275 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8e8b96010 0x55c8e8ba82d0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f12780092a0 tx=0x7f12780242b0 comp rx=0 tx=0).stop 2024-02-02T15:44:21.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.976+0000 7f12a368eb80 1 -- 172.21.15.1:0/3979747275 shutdown_connections 2024-02-02T15:44:21.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.976+0000 7f12a368eb80 1 --2- 172.21.15.1:0/3979747275 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55c8e8c15310 0x55c8e8c17700 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:21.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.976+0000 7f12a368eb80 1 --2- 172.21.15.1:0/3979747275 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55c8e8ba17e0 0x55c8e8bfb390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:21.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.976+0000 7f12a368eb80 1 --2- 172.21.15.1:0/3979747275 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c8e8b96010 0x55c8e8ba82d0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:21.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.976+0000 7f12a368eb80 1 --2- 172.21.15.1:0/3979747275 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55c8e8b96930 0x55c8e8bfae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:21.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.976+0000 7f12a368eb80 1 -- 172.21.15.1:0/3979747275 >> 172.21.15.1:0/3979747275 conn(0x55c8e8af5db0 msgr2=0x55c8e8bad520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:21.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.976+0000 7f1276a00700 0 client.30059 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:44:21.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.977+0000 7f12a368eb80 1 -- 172.21.15.1:0/3979747275 shutdown_connections 2024-02-02T15:44:21.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.977+0000 7f12a368eb80 1 -- 172.21.15.1:0/3979747275 wait complete. 2024-02-02T15:44:21.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.977+0000 7f12a368eb80 20 client.30059 trim_cache size 0 max 16384 2024-02-02T15:44:21.979 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:44:21.979 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2024-02-02T15:44:21.980 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2024-02-02T15:44:21.980 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:44:21.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.978+0000 7f128fc00700 1 -- 172.21.15.1:0/4266572140 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f12680a07e0 msgr2=0x7f12680a2ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:21.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.978+0000 7f128fc00700 1 --2- 172.21.15.1:0/4266572140 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f12680a07e0 0x7f12680a2ca0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f1278005950 tx=0x7f12780058e0 comp rx=0 tx=0).stop 2024-02-02T15:44:21.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.978+0000 7f128fc00700 1 -- 172.21.15.1:0/4266572140 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1288105220 msgr2=0x7f12881a1ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:21.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.978+0000 7f128fc00700 1 --2- 172.21.15.1:0/4266572140 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1288105220 0x7f12881a1ed0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f128000be20 tx=0x7f128000bf00 comp rx=0 tx=0).stop 2024-02-02T15:44:21.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.979+0000 7f128fc00700 1 -- 172.21.15.1:0/4266572140 shutdown_connections 2024-02-02T15:44:21.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.979+0000 7f128fc00700 1 --2- 172.21.15.1:0/4266572140 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f12680a07e0 0x7f12680a2ca0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:21.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.979+0000 7f128fc00700 1 --2- 172.21.15.1:0/4266572140 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1288105220 0x7f12881a1ed0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:21.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.979+0000 7f128fc00700 1 --2- 172.21.15.1:0/4266572140 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f12880ffe20 0x7f128819d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:21.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.979+0000 7f128fc00700 1 --2- 172.21.15.1:0/4266572140 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1288100740 0x7f128819da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:21.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.979+0000 7f128fc00700 1 -- 172.21.15.1:0/4266572140 >> 172.21.15.1:0/4266572140 conn(0x7f1288076d70 msgr2=0x7f1288103c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:21.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.979+0000 7f128fc00700 1 -- 172.21.15.1:0/4266572140 shutdown_connections 2024-02-02T15:44:21.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:21.979+0000 7f128fc00700 1 -- 172.21.15.1:0/4266572140 wait complete. 2024-02-02T15:44:21.993 INFO:tasks.workunit.client.0.smithi001.stdout: 10 130310 28.66 MB/sec execute 114 sec latency 8085.514 ms 2024-02-02T15:44:22.004 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-02T15:44:22.004 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T15:44:22.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:22.314+0000 7ff633600700 1 -- 172.21.15.1:0/2706946101 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff62c068070 msgr2=0x7ff62c0684f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:22.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:22.314+0000 7ff633600700 1 --2- 172.21.15.1:0/2706946101 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff62c068070 0x7ff62c0684f0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7ff6140098f0 tx=0x7ff614009c00 comp rx=0 tx=0).stop 2024-02-02T15:44:22.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:22.314+0000 7ff633600700 1 -- 172.21.15.1:0/2706946101 shutdown_connections 2024-02-02T15:44:22.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:22.314+0000 7ff633600700 1 --2- 172.21.15.1:0/2706946101 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff62c068a30 0x7ff62c10dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:22.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:22.314+0000 7ff633600700 1 --2- 172.21.15.1:0/2706946101 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff62c106e20 0x7ff62c107200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:22.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:22.314+0000 7ff633600700 1 --2- 172.21.15.1:0/2706946101 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff62c068070 0x7ff62c0684f0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:22.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:22.314+0000 7ff633600700 1 -- 172.21.15.1:0/2706946101 >> 172.21.15.1:0/2706946101 conn(0x7ff62c0769c0 msgr2=0x7ff62c076dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:22.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:22.314+0000 7ff633600700 1 -- 172.21.15.1:0/2706946101 shutdown_connections 2024-02-02T15:44:22.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:22.314+0000 7ff633600700 1 -- 172.21.15.1:0/2706946101 wait complete. 2024-02-02T15:44:22.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:22.315+0000 7ff633600700 1 Processor -- start 2024-02-02T15:44:22.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:22.316+0000 7ff633600700 1 -- start start 2024-02-02T15:44:22.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:22.316+0000 7ff633600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff62c068070 0x7ff62c198fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:22.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:22.316+0000 7ff633600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff62c068a30 0x7ff62c199520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:22.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:22.316+0000 7ff633600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff62c106e20 0x7ff62c19d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:22.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:22.316+0000 7ff633600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff62c19dfc0 con 0x7ff62c106e20 2024-02-02T15:44:22.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:22.316+0000 7ff633600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff62c19e130 con 0x7ff62c068070 2024-02-02T15:44:22.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:22.316+0000 7ff633600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff62c19e2f0 con 0x7ff62c068a30 2024-02-02T15:44:22.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:22.317+0000 7ff630c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff62c068070 0x7ff62c198fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:22.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:22.317+0000 7ff630c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff62c068070 0x7ff62c198fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:42336/0 (socket says 172.21.15.1:42336) 2024-02-02T15:44:22.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:22.317+0000 7ff630c00700 1 -- 172.21.15.1:0/2790619164 learned_addr learned my addr 172.21.15.1:0/2790619164 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:44:22.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:22.317+0000 7ff62be00700 1 --2- 172.21.15.1:0/2790619164 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff62c068a30 0x7ff62c199520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:22.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:22.317+0000 7ff631600700 1 --2- 172.21.15.1:0/2790619164 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff62c106e20 0x7ff62c19d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:22.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:22.317+0000 7ff630c00700 1 -- 172.21.15.1:0/2790619164 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff62c068a30 msgr2=0x7ff62c199520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:22.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:22.317+0000 7ff630c00700 1 --2- 172.21.15.1:0/2790619164 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff62c068a30 0x7ff62c199520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:22.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:22.317+0000 7ff630c00700 1 -- 172.21.15.1:0/2790619164 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff62c106e20 msgr2=0x7ff62c19d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:22.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:22.317+0000 7ff630c00700 1 --2- 172.21.15.1:0/2790619164 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff62c106e20 0x7ff62c19d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:22.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:22.317+0000 7ff630c00700 1 -- 172.21.15.1:0/2790619164 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff614009520 con 0x7ff62c068070 2024-02-02T15:44:22.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:22.317+0000 7ff631600700 1 --2- 172.21.15.1:0/2790619164 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff62c106e20 0x7ff62c19d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:44:22.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:22.317+0000 7ff630c00700 1 --2- 172.21.15.1:0/2790619164 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff62c068070 0x7ff62c198fe0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7ff61c00e9c0 tx=0x7ff61c00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:22.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:22.318+0000 7ff629600700 1 -- 172.21.15.1:0/2790619164 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff61c00cbd0 con 0x7ff62c068070 2024-02-02T15:44:22.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:22.318+0000 7ff633600700 1 -- 172.21.15.1:0/2790619164 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff62c19e5d0 con 0x7ff62c068070 2024-02-02T15:44:22.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:22.318+0000 7ff629600700 1 -- 172.21.15.1:0/2790619164 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff61c00eeb0 con 0x7ff62c068070 2024-02-02T15:44:22.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:22.318+0000 7ff629600700 1 -- 172.21.15.1:0/2790619164 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff61c005580 con 0x7ff62c068070 2024-02-02T15:44:22.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:22.318+0000 7ff633600700 1 -- 172.21.15.1:0/2790619164 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff62c1a5e10 con 0x7ff62c068070 2024-02-02T15:44:22.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:22.319+0000 7ff633600700 1 -- 172.21.15.1:0/2790619164 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff62c067bc0 con 0x7ff62c068070 2024-02-02T15:44:22.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:22.322+0000 7ff629600700 1 -- 172.21.15.1:0/2790619164 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7ff61c004750 con 0x7ff62c068070 2024-02-02T15:44:22.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:22.324+0000 7ff629600700 1 --2- 172.21.15.1:0/2790619164 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff60c0a0710 0x7ff60c0a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:22.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:22.324+0000 7ff629600700 1 -- 172.21.15.1:0/2790619164 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 9453+0+0 (secure 0 0 0) 0x7ff61c014070 con 0x7ff62c068070 2024-02-02T15:44:22.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:22.324+0000 7ff62be00700 1 --2- 172.21.15.1:0/2790619164 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff60c0a0710 0x7ff60c0a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:22.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:22.325+0000 7ff62be00700 1 --2- 172.21.15.1:0/2790619164 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff60c0a0710 0x7ff60c0a2bd0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff614005e90 tx=0x7ff614019040 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:22.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:22.329+0000 7ff629600700 1 -- 172.21.15.1:0/2790619164 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7ff61c0958a0 con 0x7ff62c068070 2024-02-02T15:44:22.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:22.760+0000 7ff633600700 1 -- 172.21.15.1:0/2790619164 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff62c19a330 con 0x7ff62c068070 2024-02-02T15:44:22.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:22.761+0000 7ff629600700 1 -- 172.21.15.1:0/2790619164 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+10866 (secure 0 0 0) 0x7ff61c095020 con 0x7ff62c068070 2024-02-02T15:44:22.762 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:44:22.762 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":73,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2922240448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2922240448},{"type":"v1","addr":"172.21.15.1:6835","nonce":2922240448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2729497431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2729497431},{"type":"v1","addr":"172.21.15.1:6837","nonce":2729497431}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":73,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-02T15:24:47.761435+0000","modified":"2024-02-02T15:44:17.487060+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24529,"mds_2":24518,"mds_3":24514,"mds_4":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14517":{"gid":14517,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/1218303683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":1218303683},{"type":"v1","addr":"172.21.15.1:6839","nonce":1218303683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14529":{"gid":14529,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/2059197496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2059197496},{"type":"v1","addr":"172.21.15.1:6841","nonce":2059197496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/7648986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":7648986},{"type":"v1","addr":"172.21.15.142:6833","nonce":7648986}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/3482746094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3482746094},{"type":"v1","addr":"172.21.15.71:6833","nonce":3482746094}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/388207814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":388207814},{"type":"v1","addr":"172.21.15.71:6835","nonce":388207814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6835/1052606547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1052606547},{"type":"v1","addr":"172.21.15.142:6835","nonce":1052606547}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.142:6837/2870289019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2870289019},{"type":"v1","addr":"172.21.15.142:6837","nonce":2870289019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.71:6837/628016199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":628016199},{"type":"v1","addr":"172.21.15.71:6837","nonce":628016199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.142:6839/321675421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":321675421},{"type":"v1","addr":"172.21.15.142:6839","nonce":321675421}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/1268086493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1268086493},{"type":"v1","addr":"172.21.15.71:6839","nonce":1268086493}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-02T15:44:22.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:22.765+0000 7ff633600700 1 -- 172.21.15.1:0/2790619164 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff60c0a0710 msgr2=0x7ff60c0a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:22.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:22.765+0000 7ff633600700 1 --2- 172.21.15.1:0/2790619164 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff60c0a0710 0x7ff60c0a2bd0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ff614005e90 tx=0x7ff614019040 comp rx=0 tx=0).stop 2024-02-02T15:44:22.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:22.765+0000 7ff633600700 1 -- 172.21.15.1:0/2790619164 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff62c068070 msgr2=0x7ff62c198fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:22.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:22.765+0000 7ff633600700 1 --2- 172.21.15.1:0/2790619164 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff62c068070 0x7ff62c198fe0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7ff61c00e9c0 tx=0x7ff61c00ecd0 comp rx=0 tx=0).stop 2024-02-02T15:44:22.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:22.765+0000 7ff633600700 1 -- 172.21.15.1:0/2790619164 shutdown_connections 2024-02-02T15:44:22.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:22.765+0000 7ff633600700 1 --2- 172.21.15.1:0/2790619164 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff60c0a0710 0x7ff60c0a2bd0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:22.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:22.765+0000 7ff633600700 1 --2- 172.21.15.1:0/2790619164 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff62c106e20 0x7ff62c19d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:22.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:22.765+0000 7ff633600700 1 --2- 172.21.15.1:0/2790619164 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff62c068070 0x7ff62c198fe0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:22.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:22.765+0000 7ff633600700 1 --2- 172.21.15.1:0/2790619164 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff62c068a30 0x7ff62c199520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:22.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:22.765+0000 7ff633600700 1 -- 172.21.15.1:0/2790619164 >> 172.21.15.1:0/2790619164 conn(0x7ff62c0769c0 msgr2=0x7ff62c10be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:22.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:22.766+0000 7ff633600700 1 -- 172.21.15.1:0/2790619164 shutdown_connections 2024-02-02T15:44:22.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:22.766+0000 7ff633600700 1 -- 172.21.15.1:0/2790619164 wait complete. 2024-02-02T15:44:22.767 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 73 2024-02-02T15:44:22.796 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-02-02T15:44:22.993 INFO:tasks.workunit.client.0.smithi001.stdout: 10 130907 28.66 MB/sec execute 115 sec latency 9085.571 ms 2024-02-02T15:44:23.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.067+0000 7fe93c200700 1 -- 172.21.15.1:0/3456141039 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe93410b400 msgr2=0x7fe93410d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:23.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.067+0000 7fe93c200700 1 --2- 172.21.15.1:0/3456141039 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe93410b400 0x7fe93410d850 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fe92c00b0a0 tx=0x7fe92c00b3b0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.067+0000 7fe93c200700 1 -- 172.21.15.1:0/3456141039 shutdown_connections 2024-02-02T15:44:23.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.067+0000 7fe93c200700 1 --2- 172.21.15.1:0/3456141039 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe93410b400 0x7fe93410d850 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.067+0000 7fe93c200700 1 --2- 172.21.15.1:0/3456141039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe934102030 0x7fe934102410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.067+0000 7fe93c200700 1 --2- 172.21.15.1:0/3456141039 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe934102950 0x7fe93410ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.067+0000 7fe93c200700 1 -- 172.21.15.1:0/3456141039 >> 172.21.15.1:0/3456141039 conn(0x7fe934076d70 msgr2=0x7fe934077180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:23.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.067+0000 7fe93c200700 1 -- 172.21.15.1:0/3456141039 shutdown_connections 2024-02-02T15:44:23.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.067+0000 7fe93c200700 1 -- 172.21.15.1:0/3456141039 wait complete. 2024-02-02T15:44:23.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.069+0000 7fe93c200700 1 Processor -- start 2024-02-02T15:44:23.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.070+0000 7fe93c200700 1 -- start start 2024-02-02T15:44:23.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.071+0000 7fe93c200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe934102030 0x7fe9340ffc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:23.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.071+0000 7fe93c200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe934102950 0x7fe9341001c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:23.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.071+0000 7fe93c200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe93410b400 0x7fe934100700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:23.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.071+0000 7fe93c200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9341a97a0 con 0x7fe93410b400 2024-02-02T15:44:23.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.071+0000 7fe93c200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe93419af60 con 0x7fe934102950 2024-02-02T15:44:23.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.071+0000 7fe93c200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe93419b120 con 0x7fe934102030 2024-02-02T15:44:23.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.072+0000 7fe938e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe934102950 0x7fe9341001c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:23.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.072+0000 7fe93a200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe93410b400 0x7fe934100700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:23.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.072+0000 7fe93a200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe93410b400 0x7fe934100700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44926/0 (socket says 172.21.15.1:44926) 2024-02-02T15:44:23.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.072+0000 7fe938e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe934102950 0x7fe9341001c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:51446/0 (socket says 172.21.15.1:51446) 2024-02-02T15:44:23.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.072+0000 7fe93a200700 1 -- 172.21.15.1:0/553638534 learned_addr learned my addr 172.21.15.1:0/553638534 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:44:23.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.072+0000 7fe939800700 1 --2- 172.21.15.1:0/553638534 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe934102030 0x7fe9340ffc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:23.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.072+0000 7fe938e00700 1 -- 172.21.15.1:0/553638534 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe934102030 msgr2=0x7fe9340ffc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:23.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.072+0000 7fe938e00700 1 --2- 172.21.15.1:0/553638534 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe934102030 0x7fe9340ffc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.072+0000 7fe938e00700 1 -- 172.21.15.1:0/553638534 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe93410b400 msgr2=0x7fe934100700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:23.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.072+0000 7fe938e00700 1 --2- 172.21.15.1:0/553638534 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe93410b400 0x7fe934100700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.072+0000 7fe938e00700 1 -- 172.21.15.1:0/553638534 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe92c009d00 con 0x7fe934102950 2024-02-02T15:44:23.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.072+0000 7fe939800700 1 --2- 172.21.15.1:0/553638534 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe934102030 0x7fe9340ffc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:44:23.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.073+0000 7fe938e00700 1 --2- 172.21.15.1:0/553638534 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe934102950 0x7fe9341001c0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fe91c00e9f0 tx=0x7fe91c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:23.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.073+0000 7fe932000700 1 -- 172.21.15.1:0/553638534 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe91c00eee0 con 0x7fe934102950 2024-02-02T15:44:23.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.073+0000 7fe93c200700 1 -- 172.21.15.1:0/553638534 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe93419b400 con 0x7fe934102950 2024-02-02T15:44:23.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.073+0000 7fe932000700 1 -- 172.21.15.1:0/553638534 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe91c00cbd0 con 0x7fe934102950 2024-02-02T15:44:23.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.073+0000 7fe932000700 1 -- 172.21.15.1:0/553638534 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe91c018680 con 0x7fe934102950 2024-02-02T15:44:23.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.073+0000 7fe93c200700 1 -- 172.21.15.1:0/553638534 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe93419b960 con 0x7fe934102950 2024-02-02T15:44:23.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.078+0000 7fe932000700 1 -- 172.21.15.1:0/553638534 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7fe91c00cd40 con 0x7fe934102950 2024-02-02T15:44:23.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.079+0000 7fe94fdc2b80 1 Processor -- start 2024-02-02T15:44:23.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.079+0000 7fe94fdc2b80 1 -- start start 2024-02-02T15:44:23.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.080+0000 7fe94fdc2b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55600c603010 0x55600c6033f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:23.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.080+0000 7fe94fdc2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55600c603930 0x55600c60e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:23.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.080+0000 7fe932000700 1 --2- 172.21.15.1:0/553638534 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fe9140a0690 0x7fe9140a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:23.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.080+0000 7fe932000700 1 -- 172.21.15.1:0/553638534 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fe91c014070 con 0x7fe934102950 2024-02-02T15:44:23.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.080+0000 7fe939800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55600c603930 0x55600c60e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:23.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.081+0000 7fe94fdc2b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55600c60e7e0 0x55600c610bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:23.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.081+0000 7fe938e00700 1 --2- 172.21.15.1:0/553638534 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fe9140a0690 0x7fe9140a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:23.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.081+0000 7fe939800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55600c603930 0x55600c60e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:44602/0 (socket says 172.21.15.1:44602) 2024-02-02T15:44:23.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.081+0000 7fe939800700 1 -- 172.21.15.1:0/995454384 learned_addr learned my addr 172.21.15.1:0/995454384 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:44:23.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.081+0000 7fe94fdc2b80 1 -- 172.21.15.1:0/995454384 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55600c5774c0 con 0x55600c603010 2024-02-02T15:44:23.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.081+0000 7fe94fdc2b80 1 -- 172.21.15.1:0/995454384 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55600c55baa0 con 0x55600c60e7e0 2024-02-02T15:44:23.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.081+0000 7fe94fdc2b80 1 -- 172.21.15.1:0/995454384 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55600c559460 con 0x55600c603930 2024-02-02T15:44:23.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.081+0000 7fe93a200700 1 --2- 172.21.15.1:0/995454384 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55600c60e7e0 0x55600c610bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:23.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.081+0000 7fe939800700 1 -- 172.21.15.1:0/995454384 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55600c60e7e0 msgr2=0x55600c610bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:23.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.081+0000 7fe939800700 1 --2- 172.21.15.1:0/995454384 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55600c60e7e0 0x55600c610bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.081+0000 7fe939800700 1 -- 172.21.15.1:0/995454384 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55600c603010 msgr2=0x55600c6033f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:23.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.081+0000 7fe939800700 1 --2- 172.21.15.1:0/995454384 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55600c603010 0x55600c6033f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.081+0000 7fe939800700 1 -- 172.21.15.1:0/995454384 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55600c3d63b0 con 0x55600c603930 2024-02-02T15:44:23.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.082+0000 7fe938e00700 1 --2- 172.21.15.1:0/553638534 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fe9140a0690 0x7fe9140a2b50 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe91c0a0f10 tx=0x7fe91c010ae0 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:23.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.082+0000 7fe939800700 1 --2- 172.21.15.1:0/995454384 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55600c603930 0x55600c60e2a0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fe924009d10 tx=0x7fe92400e5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cab74de765dd5bd8 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:23.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.082+0000 7fe92aa00700 1 -- 172.21.15.1:0/995454384 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe92400ede0 con 0x55600c603930 2024-02-02T15:44:23.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.083+0000 7fe92aa00700 1 -- 172.21.15.1:0/995454384 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe924009e60 con 0x55600c603930 2024-02-02T15:44:23.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.083+0000 7fe92aa00700 1 -- 172.21.15.1:0/995454384 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9240107d0 con 0x55600c603930 2024-02-02T15:44:23.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.084+0000 7fe94fdc2b80 1 -- 172.21.15.1:0/995454384 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55600c603930 msgr2=0x55600c60e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:23.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.084+0000 7fe94fdc2b80 1 --2- 172.21.15.1:0/995454384 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55600c603930 0x55600c60e2a0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fe924009d10 tx=0x7fe92400e5d0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.084+0000 7fe94fdc2b80 1 -- 172.21.15.1:0/995454384 shutdown_connections 2024-02-02T15:44:23.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.084+0000 7fe94fdc2b80 1 --2- 172.21.15.1:0/995454384 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55600c60e7e0 0x55600c610bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.084+0000 7fe94fdc2b80 1 --2- 172.21.15.1:0/995454384 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55600c603010 0x55600c6033f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.084+0000 7fe94fdc2b80 1 --2- 172.21.15.1:0/995454384 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55600c603930 0x55600c60e2a0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.084+0000 7fe94fdc2b80 1 -- 172.21.15.1:0/995454384 >> 172.21.15.1:0/995454384 conn(0x55600c562db0 msgr2=0x55600c5ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:23.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.084+0000 7fe94fdc2b80 1 -- 172.21.15.1:0/995454384 shutdown_connections 2024-02-02T15:44:23.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.084+0000 7fe94fdc2b80 1 -- 172.21.15.1:0/995454384 wait complete. 2024-02-02T15:44:23.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.085+0000 7fe94fdc2b80 1 -- start start 2024-02-02T15:44:23.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.085+0000 7fe94fdc2b80 1 Processor -- start 2024-02-02T15:44:23.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.086+0000 7fe94fdc2b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55600c603010 0x55600c60e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:23.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.086+0000 7fe94fdc2b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55600c603930 0x55600c60b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:23.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.086+0000 7fe94fdc2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55600c60e7e0 0x55600c60b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:23.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.086+0000 7fe94fdc2b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55600c554d50 con 0x55600c603010 2024-02-02T15:44:23.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.086+0000 7fe94fdc2b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55600c4f0d80 con 0x55600c603930 2024-02-02T15:44:23.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.086+0000 7fe94fdc2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55600c4f0a20 con 0x55600c60e7e0 2024-02-02T15:44:23.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.086+0000 7fe94fdc2b80 10 client.0 _ll_register_callbacks cb 0x55600c4f2290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:44:23.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.087+0000 7fe939800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55600c603930 0x55600c60b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:23.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.087+0000 7fe939800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55600c603930 0x55600c60b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:51462/0 (socket says 172.21.15.1:51462) 2024-02-02T15:44:23.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.087+0000 7fe939800700 1 -- 172.21.15.1:0/89265936 learned_addr learned my addr 172.21.15.1:0/89265936 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:44:23.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.087+0000 7fe93a200700 1 --2- 172.21.15.1:0/89265936 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55600c60e7e0 0x55600c60b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:23.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.087+0000 7fe93a200700 1 --2- 172.21.15.1:0/89265936 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55600c603010 0x55600c60e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:23.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.087+0000 7fe939800700 1 -- 172.21.15.1:0/89265936 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55600c60e7e0 msgr2=0x55600c60b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:23.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.087+0000 7fe939800700 1 --2- 172.21.15.1:0/89265936 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55600c60e7e0 0x55600c60b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.087+0000 7fe939800700 1 -- 172.21.15.1:0/89265936 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55600c603010 msgr2=0x55600c60e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:23.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.087+0000 7fe939800700 1 --2- 172.21.15.1:0/89265936 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55600c603010 0x55600c60e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.087+0000 7fe939800700 1 -- 172.21.15.1:0/89265936 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe924009940 con 0x55600c603930 2024-02-02T15:44:23.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.087+0000 7fe93a200700 1 --2- 172.21.15.1:0/89265936 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55600c603010 0x55600c60e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:44:23.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.088+0000 7fe939800700 1 --2- 172.21.15.1:0/89265936 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55600c603930 0x55600c60b2e0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fe9240052a0 tx=0x7fe9240052d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=957e39dfce1cbfa3 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:23.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.088+0000 7fe923400700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:44:23.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.088+0000 7fe923400700 1 -- 172.21.15.1:0/89265936 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe924020070 con 0x55600c603930 2024-02-02T15:44:23.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.088+0000 7fe923400700 1 -- 172.21.15.1:0/89265936 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fe924004750 con 0x55600c603930 2024-02-02T15:44:23.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.088+0000 7fe94fdc2b80 1 -- 172.21.15.1:0/89265936 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55600c5f70f0 con 0x55600c603930 2024-02-02T15:44:23.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.088+0000 7fe923400700 1 -- 172.21.15.1:0/89265936 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe924004910 con 0x55600c603930 2024-02-02T15:44:23.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.088+0000 7fe94fdc2b80 10 client.30077 fetch_fsmap learned FSMap version 73 2024-02-02T15:44:23.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.089+0000 7fe94fdc2b80 1 -- 172.21.15.1:0/89265936 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) v3 -- 0x55600c382b60 con 0x55600c603930 2024-02-02T15:44:23.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.089+0000 7fe923400700 1 -- 172.21.15.1:0/89265936 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55600c5f70f0 con 0x55600c603930 2024-02-02T15:44:23.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.089+0000 7fe923400700 1 -- 172.21.15.1:0/89265936 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 73) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe92400c070 con 0x55600c603930 2024-02-02T15:44:23.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.089+0000 7fe94fdc2b80 10 client.30077 fetch_fsmap finished waiting for FSMap version 73 2024-02-02T15:44:23.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.089+0000 7fe94fdc2b80 10 client.30077 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-02-02T15:44:23.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.089+0000 7fe94fdc2b80 20 client.30077 populate_metadata read hostname 'smithi001' 2024-02-02T15:44:23.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.089+0000 7fe94fdc2b80 1 --2- 172.21.15.1:0/89265936 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55600c61e0e0 0x55600c620500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:23.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.089+0000 7fe94fdc2b80 4 client.30077 mds_command: new command op to 24502 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:44:23.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.089+0000 7fe94fdc2b80 1 -- 172.21.15.1:0/89265936 --> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55600c54ddd0 con 0x55600c61e0e0 2024-02-02T15:44:23.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.090+0000 7fe93a200700 1 --2- 172.21.15.1:0/89265936 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55600c61e0e0 0x55600c620500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:23.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.091+0000 7fe93a200700 1 --2- 172.21.15.1:0/89265936 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55600c61e0e0 0x55600c620500 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f9b2f4a740f0412f server_cookie=72e117fce4592437 in_seq=0 out_seq=0 2024-02-02T15:44:23.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.091+0000 7fe923400700 10 client.30077 ms_handle_connect on v2:172.21.15.142:6834/1052606547 2024-02-02T15:44:23.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.096+0000 7fe923400700 1 -- 172.21.15.1:0/89265936 <== mds.4 v2:172.21.15.142:6834/1052606547 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55600c54ddd0 con 0x55600c61e0e0 2024-02-02T15:44:23.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.096+0000 7fe923400700 10 client.30077 handle_command_reply: tid=0 2024-02-02T15:44:23.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.096+0000 7fe94fdc2b80 1 client.30077 shutdown 2024-02-02T15:44:23.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.097+0000 7fe94fdc2b80 1 -- 172.21.15.1:0/89265936 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55600c603930 msgr2=0x55600c60b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:23.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.097+0000 7fe94fdc2b80 1 --2- 172.21.15.1:0/89265936 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55600c603930 0x55600c60b2e0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fe9240052a0 tx=0x7fe9240052d0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.097+0000 7fe94fdc2b80 1 -- 172.21.15.1:0/89265936 shutdown_connections 2024-02-02T15:44:23.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.097+0000 7fe94fdc2b80 1 --2- 172.21.15.1:0/89265936 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55600c61e0e0 0x55600c620500 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.097+0000 7fe94fdc2b80 1 --2- 172.21.15.1:0/89265936 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55600c60e7e0 0x55600c60b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.097+0000 7fe94fdc2b80 1 --2- 172.21.15.1:0/89265936 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55600c603010 0x55600c60e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.097+0000 7fe94fdc2b80 1 --2- 172.21.15.1:0/89265936 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55600c603930 0x55600c60b2e0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.097+0000 7fe94fdc2b80 1 -- 172.21.15.1:0/89265936 >> 172.21.15.1:0/89265936 conn(0x55600c562db0 msgr2=0x55600c5fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:23.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.097+0000 7fe923400700 0 client.30077 ms_handle_reset on v2:172.21.15.142:6834/1052606547 2024-02-02T15:44:23.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.098+0000 7fe94fdc2b80 1 -- 172.21.15.1:0/89265936 shutdown_connections 2024-02-02T15:44:23.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.098+0000 7fe94fdc2b80 1 -- 172.21.15.1:0/89265936 wait complete. 2024-02-02T15:44:23.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.098+0000 7fe94fdc2b80 20 client.30077 trim_cache size 0 max 16384 2024-02-02T15:44:23.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.118+0000 7fe94fdc2b80 1 Processor -- start 2024-02-02T15:44:23.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.118+0000 7fe94fdc2b80 1 -- start start 2024-02-02T15:44:23.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.118+0000 7fe94fdc2b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55600c603010 0x55600c610bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:23.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.118+0000 7fe94fdc2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55600c603930 0x55600c615400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:23.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.118+0000 7fe94fdc2b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55600c60e7e0 0x55600c60dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:23.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.118+0000 7fe94fdc2b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55600c3d86a0 con 0x55600c603010 2024-02-02T15:44:23.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.118+0000 7fe94fdc2b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55600c606120 con 0x55600c60e7e0 2024-02-02T15:44:23.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.118+0000 7fe94fdc2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55600c604a80 con 0x55600c603930 2024-02-02T15:44:23.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.119+0000 7fe93a200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55600c603010 0x55600c610bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:23.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.119+0000 7fe93a200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55600c603010 0x55600c610bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44952/0 (socket says 172.21.15.1:44952) 2024-02-02T15:44:23.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.119+0000 7fe93a200700 1 -- 172.21.15.1:0/3241523551 learned_addr learned my addr 172.21.15.1:0/3241523551 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:44:23.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.119+0000 7fe93a200700 1 --2- 172.21.15.1:0/3241523551 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55600c60e7e0 0x55600c60dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:23.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.119+0000 7fe939800700 1 --2- 172.21.15.1:0/3241523551 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55600c603930 0x55600c615400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:23.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.119+0000 7fe93a200700 1 -- 172.21.15.1:0/3241523551 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55600c603930 msgr2=0x55600c615400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:23.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.119+0000 7fe93a200700 1 --2- 172.21.15.1:0/3241523551 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55600c603930 0x55600c615400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.119+0000 7fe93a200700 1 -- 172.21.15.1:0/3241523551 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55600c60e7e0 msgr2=0x55600c60dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:23.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.119+0000 7fe93a200700 1 --2- 172.21.15.1:0/3241523551 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55600c60e7e0 0x55600c60dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.119+0000 7fe93a200700 1 -- 172.21.15.1:0/3241523551 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55600c51aea0 con 0x55600c603010 2024-02-02T15:44:23.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.120+0000 7fe93a200700 1 --2- 172.21.15.1:0/3241523551 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55600c603010 0x55600c610bd0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fe92c000f80 tx=0x7fe92c021040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc976e94015f8803 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:23.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.120+0000 7fe92b400700 1 -- 172.21.15.1:0/3241523551 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe92c00e040 con 0x55600c603010 2024-02-02T15:44:23.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.120+0000 7fe92b400700 1 -- 172.21.15.1:0/3241523551 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe92c0293d0 con 0x55600c603010 2024-02-02T15:44:23.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.120+0000 7fe92b400700 1 -- 172.21.15.1:0/3241523551 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe92c0050d0 con 0x55600c603010 2024-02-02T15:44:23.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.121+0000 7fe94fdc2b80 1 -- 172.21.15.1:0/3241523551 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55600c603010 msgr2=0x55600c610bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:23.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.121+0000 7fe94fdc2b80 1 --2- 172.21.15.1:0/3241523551 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55600c603010 0x55600c610bd0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fe92c000f80 tx=0x7fe92c021040 comp rx=0 tx=0).stop 2024-02-02T15:44:23.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.121+0000 7fe94fdc2b80 1 -- 172.21.15.1:0/3241523551 shutdown_connections 2024-02-02T15:44:23.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.121+0000 7fe94fdc2b80 1 --2- 172.21.15.1:0/3241523551 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55600c60e7e0 0x55600c60dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.121+0000 7fe94fdc2b80 1 --2- 172.21.15.1:0/3241523551 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55600c603010 0x55600c610bd0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.121+0000 7fe94fdc2b80 1 --2- 172.21.15.1:0/3241523551 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55600c603930 0x55600c615400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.121+0000 7fe94fdc2b80 1 -- 172.21.15.1:0/3241523551 >> 172.21.15.1:0/3241523551 conn(0x55600c562db0 msgr2=0x55600c6200f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:23.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.121+0000 7fe94fdc2b80 1 -- 172.21.15.1:0/3241523551 shutdown_connections 2024-02-02T15:44:23.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.121+0000 7fe94fdc2b80 1 -- 172.21.15.1:0/3241523551 wait complete. 2024-02-02T15:44:23.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.121+0000 7fe94fdc2b80 1 -- start start 2024-02-02T15:44:23.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.121+0000 7fe94fdc2b80 1 Processor -- start 2024-02-02T15:44:23.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.122+0000 7fe94fdc2b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55600c603010 0x55600c610c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:23.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.122+0000 7fe94fdc2b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55600c603930 0x55600c6113e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:23.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.122+0000 7fe94fdc2b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55600c60e7e0 0x55600c611a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:23.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.122+0000 7fe94fdc2b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55600c612150 con 0x55600c603930 2024-02-02T15:44:23.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.122+0000 7fe94fdc2b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55600c66ebf0 con 0x55600c603010 2024-02-02T15:44:23.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.122+0000 7fe94fdc2b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55600c66ed30 con 0x55600c60e7e0 2024-02-02T15:44:23.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.122+0000 7fe94fdc2b80 10 client.0 _ll_register_callbacks cb 0x55600c4f2290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:44:23.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.122+0000 7fe93a200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55600c603010 0x55600c610c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:23.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.122+0000 7fe939800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55600c603930 0x55600c6113e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:23.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.122+0000 7fe93a200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55600c603010 0x55600c610c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:51480/0 (socket says 172.21.15.1:51480) 2024-02-02T15:44:23.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.122+0000 7fe93a200700 1 -- 172.21.15.1:0/1861666392 learned_addr learned my addr 172.21.15.1:0/1861666392 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:44:23.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.122+0000 7fe93a200700 1 --2- 172.21.15.1:0/1861666392 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55600c60e7e0 0x55600c611a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:23.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.123+0000 7fe939800700 1 -- 172.21.15.1:0/1861666392 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55600c60e7e0 msgr2=0x55600c611a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:23.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.123+0000 7fe939800700 1 --2- 172.21.15.1:0/1861666392 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55600c60e7e0 0x55600c611a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.123+0000 7fe939800700 1 -- 172.21.15.1:0/1861666392 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55600c603010 msgr2=0x55600c610c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:23.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.123+0000 7fe939800700 1 --2- 172.21.15.1:0/1861666392 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55600c603010 0x55600c610c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.123+0000 7fe939800700 1 -- 172.21.15.1:0/1861666392 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe92c029040 con 0x55600c603930 2024-02-02T15:44:23.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.123+0000 7fe93a200700 1 --2- 172.21.15.1:0/1861666392 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55600c60e7e0 0x55600c611a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:44:23.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.123+0000 7fe939800700 1 --2- 172.21.15.1:0/1861666392 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55600c603930 0x55600c6113e0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fe924022ad0 tx=0x7fe9240093f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e443d98ab41871d server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:23.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.123+0000 7fe923400700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:44:23.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.123+0000 7fe923400700 1 -- 172.21.15.1:0/1861666392 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe924020040 con 0x55600c603930 2024-02-02T15:44:23.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.124+0000 7fe923400700 1 -- 172.21.15.1:0/1861666392 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fe924000e40 con 0x55600c603930 2024-02-02T15:44:23.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.124+0000 7fe94fdc2b80 1 -- 172.21.15.1:0/1861666392 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe924009e60 con 0x55600c603930 2024-02-02T15:44:23.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.124+0000 7fe923400700 1 -- 172.21.15.1:0/1861666392 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe924005ee0 con 0x55600c603930 2024-02-02T15:44:23.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.124+0000 7fe94fdc2b80 10 client.19827 fetch_fsmap learned FSMap version 73 2024-02-02T15:44:23.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.125+0000 7fe94fdc2b80 1 -- 172.21.15.1:0/1861666392 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) v3 -- 0x55600c3dacd0 con 0x55600c603930 2024-02-02T15:44:23.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.125+0000 7fe923400700 1 -- 172.21.15.1:0/1861666392 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe924009e60 con 0x55600c603930 2024-02-02T15:44:23.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.125+0000 7fe923400700 1 -- 172.21.15.1:0/1861666392 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 73) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe92400e6f0 con 0x55600c603930 2024-02-02T15:44:23.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.125+0000 7fe94fdc2b80 10 client.19827 fetch_fsmap finished waiting for FSMap version 73 2024-02-02T15:44:23.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.125+0000 7fe94fdc2b80 10 client.19827 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-02-02T15:44:23.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.125+0000 7fe94fdc2b80 20 client.19827 populate_metadata read hostname 'smithi001' 2024-02-02T15:44:23.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.125+0000 7fe94fdc2b80 1 --2- 172.21.15.1:0/1861666392 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55600c66ff50 0x55600c672370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:23.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.125+0000 7fe94fdc2b80 4 client.19827 mds_command: new command op to 24502 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:44:23.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.125+0000 7fe94fdc2b80 1 -- 172.21.15.1:0/1861666392 --> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55600c513f50 con 0x55600c66ff50 2024-02-02T15:44:23.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.126+0000 7fe93a200700 1 --2- 172.21.15.1:0/1861666392 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55600c66ff50 0x55600c672370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:23.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.126+0000 7fe923400700 10 client.19827 ms_handle_connect on v2:172.21.15.142:6834/1052606547 2024-02-02T15:44:23.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.127+0000 7fe93a200700 1 --2- 172.21.15.1:0/1861666392 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55600c66ff50 0x55600c672370 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=dcec5aba4ec31c83 server_cookie=3a102b270ec22d3 in_seq=0 out_seq=0 2024-02-02T15:44:23.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.127+0000 7fe923400700 1 -- 172.21.15.1:0/1861666392 <== mds.4 v2:172.21.15.142:6834/1052606547 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55600c513f50 con 0x55600c66ff50 2024-02-02T15:44:23.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.127+0000 7fe923400700 10 client.19827 handle_command_reply: tid=0 2024-02-02T15:44:23.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.127+0000 7fe94fdc2b80 1 client.19827 shutdown 2024-02-02T15:44:23.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.127+0000 7fe94fdc2b80 1 -- 172.21.15.1:0/1861666392 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55600c603930 msgr2=0x55600c6113e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:23.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.127+0000 7fe94fdc2b80 1 --2- 172.21.15.1:0/1861666392 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55600c603930 0x55600c6113e0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fe924022ad0 tx=0x7fe9240093f0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.127+0000 7fe94fdc2b80 1 -- 172.21.15.1:0/1861666392 shutdown_connections 2024-02-02T15:44:23.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.127+0000 7fe94fdc2b80 1 --2- 172.21.15.1:0/1861666392 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55600c66ff50 0x55600c672370 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.127+0000 7fe94fdc2b80 1 --2- 172.21.15.1:0/1861666392 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55600c60e7e0 0x55600c611a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.127+0000 7fe94fdc2b80 1 --2- 172.21.15.1:0/1861666392 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55600c603010 0x55600c610c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.127+0000 7fe94fdc2b80 1 --2- 172.21.15.1:0/1861666392 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55600c603930 0x55600c6113e0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.127+0000 7fe94fdc2b80 1 -- 172.21.15.1:0/1861666392 >> 172.21.15.1:0/1861666392 conn(0x55600c562db0 msgr2=0x55600c6199e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:23.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.128+0000 7fe923400700 0 client.19827 ms_handle_reset on v2:172.21.15.142:6834/1052606547 2024-02-02T15:44:23.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.128+0000 7fe94fdc2b80 1 -- 172.21.15.1:0/1861666392 shutdown_connections 2024-02-02T15:44:23.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.128+0000 7fe94fdc2b80 1 -- 172.21.15.1:0/1861666392 wait complete. 2024-02-02T15:44:23.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.128+0000 7fe94fdc2b80 20 client.19827 trim_cache size 0 max 16384 2024-02-02T15:44:23.130 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:44:23.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.129+0000 7fe93c200700 1 -- 172.21.15.1:0/553638534 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fe9140a0690 msgr2=0x7fe9140a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:23.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.129+0000 7fe93c200700 1 --2- 172.21.15.1:0/553638534 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fe9140a0690 0x7fe9140a2b50 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fe91c0a0f10 tx=0x7fe91c010ae0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.129+0000 7fe93c200700 1 -- 172.21.15.1:0/553638534 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe934102950 msgr2=0x7fe9341001c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:23.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.129+0000 7fe93c200700 1 --2- 172.21.15.1:0/553638534 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe934102950 0x7fe9341001c0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fe91c00e9f0 tx=0x7fe91c00ed00 comp rx=0 tx=0).stop 2024-02-02T15:44:23.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.129+0000 7fe93c200700 1 -- 172.21.15.1:0/553638534 shutdown_connections 2024-02-02T15:44:23.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.129+0000 7fe93c200700 1 --2- 172.21.15.1:0/553638534 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fe9140a0690 0x7fe9140a2b50 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.129+0000 7fe93c200700 1 --2- 172.21.15.1:0/553638534 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe93410b400 0x7fe934100700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.129+0000 7fe93c200700 1 --2- 172.21.15.1:0/553638534 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe934102030 0x7fe9340ffc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.129+0000 7fe93c200700 1 --2- 172.21.15.1:0/553638534 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe934102950 0x7fe9341001c0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.129+0000 7fe93c200700 1 -- 172.21.15.1:0/553638534 >> 172.21.15.1:0/553638534 conn(0x7fe934076d70 msgr2=0x7fe934102dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:23.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.129+0000 7fe93c200700 1 -- 172.21.15.1:0/553638534 shutdown_connections 2024-02-02T15:44:23.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.129+0000 7fe93c200700 1 -- 172.21.15.1:0/553638534 wait complete. 2024-02-02T15:44:23.154 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-02-02T15:44:23.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.433+0000 7ff5aa400700 1 -- 172.21.15.1:0/2753258258 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5a4108700 msgr2=0x7ff5a4114650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:23.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.433+0000 7ff5aa400700 1 --2- 172.21.15.1:0/2753258258 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5a4108700 0x7ff5a4114650 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7ff58c009990 tx=0x7ff58c009ca0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.433+0000 7ff5aa400700 1 -- 172.21.15.1:0/2753258258 shutdown_connections 2024-02-02T15:44:23.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.433+0000 7ff5aa400700 1 --2- 172.21.15.1:0/2753258258 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5a4108700 0x7ff5a4114650 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.433+0000 7ff5aa400700 1 --2- 172.21.15.1:0/2753258258 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff5a410d790 0x7ff5a410db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.433+0000 7ff5aa400700 1 --2- 172.21.15.1:0/2753258258 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5a4107d40 0x7ff5a41081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.433+0000 7ff5aa400700 1 -- 172.21.15.1:0/2753258258 >> 172.21.15.1:0/2753258258 conn(0x7ff5a406d120 msgr2=0x7ff5a406d530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:23.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.434+0000 7ff5aa400700 1 -- 172.21.15.1:0/2753258258 shutdown_connections 2024-02-02T15:44:23.436 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.434+0000 7ff5aa400700 1 -- 172.21.15.1:0/2753258258 wait complete. 2024-02-02T15:44:23.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.437+0000 7ff5aa400700 1 Processor -- start 2024-02-02T15:44:23.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.438+0000 7ff5aa400700 1 -- start start 2024-02-02T15:44:23.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.438+0000 7ff5aa400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5a4107d40 0x7ff5a41a2010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:23.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.439+0000 7ff5a3400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5a4107d40 0x7ff5a41a2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:23.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.439+0000 7ff5a3400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5a4107d40 0x7ff5a41a2010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:51498/0 (socket says 172.21.15.1:51498) 2024-02-02T15:44:23.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.439+0000 7ff5aa400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5a4108700 0x7ff5a41a2550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:23.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.439+0000 7ff5aa400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff5a410d790 0x7ff5a419d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:23.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.439+0000 7ff5aa400700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a419d7d0 con 0x7ff5a410d790 2024-02-02T15:44:23.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.439+0000 7ff5aa400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a419d940 con 0x7ff5a4107d40 2024-02-02T15:44:23.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.439+0000 7ff5aa400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a419db00 con 0x7ff5a4108700 2024-02-02T15:44:23.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.439+0000 7ff5a3400700 1 -- 172.21.15.1:0/3303173735 learned_addr learned my addr 172.21.15.1:0/3303173735 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:44:23.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.440+0000 7ff5a2a00700 1 --2- 172.21.15.1:0/3303173735 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5a4108700 0x7ff5a41a2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:23.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.440+0000 7ff5a3400700 1 -- 172.21.15.1:0/3303173735 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5a4108700 msgr2=0x7ff5a41a2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:23.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.440+0000 7ff5a3400700 1 --2- 172.21.15.1:0/3303173735 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5a4108700 0x7ff5a41a2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.440+0000 7ff5a3400700 1 -- 172.21.15.1:0/3303173735 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff5a410d790 msgr2=0x7ff5a419d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:23.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.440+0000 7ff5a3400700 1 --2- 172.21.15.1:0/3303173735 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff5a410d790 0x7ff5a419d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.440+0000 7ff5a3400700 1 -- 172.21.15.1:0/3303173735 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff58c0095f0 con 0x7ff5a4107d40 2024-02-02T15:44:23.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.441+0000 7ff5a3400700 1 --2- 172.21.15.1:0/3303173735 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5a4107d40 0x7ff5a41a2010 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7ff59c007f10 tx=0x7ff59c00d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:23.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.441+0000 7ff59be00700 1 -- 172.21.15.1:0/3303173735 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59c00de90 con 0x7ff5a4107d40 2024-02-02T15:44:23.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.441+0000 7ff5aa400700 1 -- 172.21.15.1:0/3303173735 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5a419dde0 con 0x7ff5a4107d40 2024-02-02T15:44:23.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.442+0000 7ff59be00700 1 -- 172.21.15.1:0/3303173735 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff59c00f040 con 0x7ff5a4107d40 2024-02-02T15:44:23.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.442+0000 7ff5aa400700 1 -- 172.21.15.1:0/3303173735 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5a41a0f50 con 0x7ff5a4107d40 2024-02-02T15:44:23.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.442+0000 7ff59be00700 1 -- 172.21.15.1:0/3303173735 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59c015400 con 0x7ff5a4107d40 2024-02-02T15:44:23.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.446+0000 7ff59be00700 1 -- 172.21.15.1:0/3303173735 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7ff59c004000 con 0x7ff5a4107d40 2024-02-02T15:44:23.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.447+0000 7ff5bdf20b80 1 Processor -- start 2024-02-02T15:44:23.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.448+0000 7ff5bdf20b80 1 -- start start 2024-02-02T15:44:23.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.449+0000 7ff5bdf20b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d3cde91010 0x55d3cde913f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:23.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.449+0000 7ff5bdf20b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d3cde91930 0x55d3cde9c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:23.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.449+0000 7ff5bdf20b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d3cde9c7e0 0x55d3cde9ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:23.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.449+0000 7ff5bdf20b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3cde054c0 con 0x55d3cde91010 2024-02-02T15:44:23.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.449+0000 7ff5bdf20b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3cdde9aa0 con 0x55d3cde91930 2024-02-02T15:44:23.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.449+0000 7ff5bdf20b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3cdde7460 con 0x55d3cde9c7e0 2024-02-02T15:44:23.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.449+0000 7ff59be00700 1 --2- 172.21.15.1:0/3303173735 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff5840a0770 0x7ff5840a2c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:23.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.449+0000 7ff5a2a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d3cde91930 0x55d3cde9c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:23.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.449+0000 7ff5a2a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d3cde91930 0x55d3cde9c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:51512/0 (socket says 172.21.15.1:51512) 2024-02-02T15:44:23.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.449+0000 7ff5a2a00700 1 -- 172.21.15.1:0/3689705054 learned_addr learned my addr 172.21.15.1:0/3689705054 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:44:23.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.449+0000 7ff5a3e00700 1 --2- 172.21.15.1:0/3689705054 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d3cde9c7e0 0x55d3cde9ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:23.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.450+0000 7ff59be00700 1 -- 172.21.15.1:0/3303173735 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 9453+0+0 (secure 0 0 0) 0x7ff59c0cba90 con 0x7ff5a4107d40 2024-02-02T15:44:23.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.450+0000 7ff5a3e00700 1 -- 172.21.15.1:0/3689705054 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d3cde91930 msgr2=0x55d3cde9c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:23.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.450+0000 7ff5a3e00700 1 --2- 172.21.15.1:0/3689705054 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d3cde91930 0x55d3cde9c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.450+0000 7ff5a3e00700 1 -- 172.21.15.1:0/3689705054 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d3cde91010 msgr2=0x55d3cde913f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:23.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.450+0000 7ff5a3e00700 1 --2- 172.21.15.1:0/3689705054 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d3cde91010 0x55d3cde913f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.450+0000 7ff5a3e00700 1 -- 172.21.15.1:0/3689705054 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3cdc643b0 con 0x55d3cde9c7e0 2024-02-02T15:44:23.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.450+0000 7ff5a3e00700 1 --2- 172.21.15.1:0/3689705054 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d3cde9c7e0 0x55d3cde9ebd0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7ff58c009990 tx=0x7ff58c005c20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94c1464266d4bada server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:23.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.451+0000 7ff5a3400700 1 --2- 172.21.15.1:0/3303173735 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff5840a0770 0x7ff5840a2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:23.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.451+0000 7ff598c00700 1 -- 172.21.15.1:0/3689705054 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58c00b7d0 con 0x55d3cde9c7e0 2024-02-02T15:44:23.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.451+0000 7ff598c00700 1 -- 172.21.15.1:0/3689705054 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff58c0055d0 con 0x55d3cde9c7e0 2024-02-02T15:44:23.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.451+0000 7ff598c00700 1 -- 172.21.15.1:0/3689705054 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58c003cc0 con 0x55d3cde9c7e0 2024-02-02T15:44:23.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.451+0000 7ff5bdf20b80 1 -- 172.21.15.1:0/3689705054 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d3cde9c7e0 msgr2=0x55d3cde9ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:23.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.451+0000 7ff5bdf20b80 1 --2- 172.21.15.1:0/3689705054 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d3cde9c7e0 0x55d3cde9ebd0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7ff58c009990 tx=0x7ff58c005c20 comp rx=0 tx=0).stop 2024-02-02T15:44:23.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.452+0000 7ff5a3400700 1 --2- 172.21.15.1:0/3303173735 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff5840a0770 0x7ff5840a2c30 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ff59c0950b0 tx=0x7ff59c009750 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:23.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.452+0000 7ff5bdf20b80 1 -- 172.21.15.1:0/3689705054 shutdown_connections 2024-02-02T15:44:23.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.452+0000 7ff5bdf20b80 1 --2- 172.21.15.1:0/3689705054 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d3cde9c7e0 0x55d3cde9ebd0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.452+0000 7ff5bdf20b80 1 --2- 172.21.15.1:0/3689705054 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d3cde91010 0x55d3cde913f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.452+0000 7ff5bdf20b80 1 --2- 172.21.15.1:0/3689705054 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d3cde91930 0x55d3cde9c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.452+0000 7ff5bdf20b80 1 -- 172.21.15.1:0/3689705054 >> 172.21.15.1:0/3689705054 conn(0x55d3cddf0db0 msgr2=0x55d3cde8d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:23.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.452+0000 7ff5bdf20b80 1 -- 172.21.15.1:0/3689705054 shutdown_connections 2024-02-02T15:44:23.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.452+0000 7ff5bdf20b80 1 -- 172.21.15.1:0/3689705054 wait complete. 2024-02-02T15:44:23.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.453+0000 7ff5bdf20b80 1 -- start start 2024-02-02T15:44:23.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.453+0000 7ff5bdf20b80 1 Processor -- start 2024-02-02T15:44:23.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.453+0000 7ff5bdf20b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d3cde91010 0x55d3cde9c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:23.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.453+0000 7ff5bdf20b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d3cde91930 0x55d3cde994b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:23.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.453+0000 7ff5bdf20b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d3cde9c7e0 0x55d3cde99b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:23.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.453+0000 7ff5bdf20b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3cdde2d50 con 0x55d3cde91010 2024-02-02T15:44:23.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.453+0000 7ff5bdf20b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3cdd7ed80 con 0x55d3cde9c7e0 2024-02-02T15:44:23.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.453+0000 7ff5bdf20b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3cdd7ea20 con 0x55d3cde91930 2024-02-02T15:44:23.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.454+0000 7ff5bdf20b80 10 client.0 _ll_register_callbacks cb 0x55d3cdd80290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:44:23.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.454+0000 7ff5a3e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d3cde9c7e0 0x55d3cde99b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:23.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.454+0000 7ff5a3e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d3cde9c7e0 0x55d3cde99b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:51522/0 (socket says 172.21.15.1:51522) 2024-02-02T15:44:23.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.454+0000 7ff5a3e00700 1 -- 172.21.15.1:0/3489358237 learned_addr learned my addr 172.21.15.1:0/3489358237 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:44:23.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.454+0000 7ff5a3e00700 1 -- 172.21.15.1:0/3489358237 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d3cde91930 msgr2=0x55d3cde994b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-02T15:44:23.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.454+0000 7ff5a3e00700 1 --2- 172.21.15.1:0/3489358237 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d3cde91930 0x55d3cde994b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.454+0000 7ff5a3e00700 1 -- 172.21.15.1:0/3489358237 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d3cde91010 msgr2=0x55d3cde9c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:23.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.454+0000 7ff5a3e00700 1 --2- 172.21.15.1:0/3489358237 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d3cde91010 0x55d3cde9c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.454+0000 7ff5a3e00700 1 -- 172.21.15.1:0/3489358237 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5a4114f50 con 0x55d3cde9c7e0 2024-02-02T15:44:23.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.455+0000 7ff5a3e00700 1 --2- 172.21.15.1:0/3489358237 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d3cde9c7e0 0x55d3cde99b10 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7ff58c00bc60 tx=0x7ff58c00baf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b0cc15c98a44259 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:23.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.455+0000 7ff591600700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:44:23.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.455+0000 7ff591600700 1 -- 172.21.15.1:0/3489358237 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58c003a90 con 0x55d3cde9c7e0 2024-02-02T15:44:23.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.455+0000 7ff591600700 1 -- 172.21.15.1:0/3489358237 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9453+0+0 (secure 0 0 0) 0x7ff58c003bf0 con 0x55d3cde9c7e0 2024-02-02T15:44:23.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.455+0000 7ff5bdf20b80 1 -- 172.21.15.1:0/3489358237 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d3cde850f0 con 0x55d3cde9c7e0 2024-02-02T15:44:23.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.455+0000 7ff591600700 1 -- 172.21.15.1:0/3489358237 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff58c003db0 con 0x55d3cde9c7e0 2024-02-02T15:44:23.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.456+0000 7ff5bdf20b80 10 client.30095 fetch_fsmap learned FSMap version 73 2024-02-02T15:44:23.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.456+0000 7ff5bdf20b80 1 -- 172.21.15.1:0/3489358237 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) v3 -- 0x55d3cdc10b60 con 0x55d3cde9c7e0 2024-02-02T15:44:23.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.456+0000 7ff591600700 1 -- 172.21.15.1:0/3489358237 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d3cde850f0 con 0x55d3cde9c7e0 2024-02-02T15:44:23.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.456+0000 7ff591600700 1 -- 172.21.15.1:0/3489358237 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 73) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff58c022030 con 0x55d3cde9c7e0 2024-02-02T15:44:23.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.456+0000 7ff5bdf20b80 10 client.30095 fetch_fsmap finished waiting for FSMap version 73 2024-02-02T15:44:23.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.456+0000 7ff5bdf20b80 10 client.30095 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-02-02T15:44:23.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.456+0000 7ff5bdf20b80 20 client.30095 populate_metadata read hostname 'smithi001' 2024-02-02T15:44:23.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.456+0000 7ff5bdf20b80 1 --2- 172.21.15.1:0/3489358237 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55d3cdeabd40 0x55d3cdeae130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:23.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.456+0000 7ff5bdf20b80 4 client.30095 mds_command: new command op to 24514 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:44:23.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.456+0000 7ff5bdf20b80 1 -- 172.21.15.1:0/3489358237 --> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d3cdddbdd0 con 0x55d3cdeabd40 2024-02-02T15:44:23.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.457+0000 7ff5a2a00700 1 --2- 172.21.15.1:0/3489358237 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55d3cdeabd40 0x55d3cdeae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:23.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.458+0000 7ff591600700 10 client.30095 ms_handle_connect on v2:172.21.15.142:6836/2870289019 2024-02-02T15:44:23.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.458+0000 7ff5a2a00700 1 --2- 172.21.15.1:0/3489358237 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55d3cdeabd40 0x55d3cdeae130 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1046bae2e61480f7 server_cookie=196bde998c174099 in_seq=0 out_seq=0 2024-02-02T15:44:23.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.460+0000 7ff591600700 1 -- 172.21.15.1:0/3489358237 <== mds.3 v2:172.21.15.142:6836/2870289019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55d3cdddbdd0 con 0x55d3cdeabd40 2024-02-02T15:44:23.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.460+0000 7ff591600700 10 client.30095 handle_command_reply: tid=0 2024-02-02T15:44:23.460 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.461+0000 7ff5bdf20b80 1 client.30095 shutdown 2024-02-02T15:44:23.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.461+0000 7ff5bdf20b80 1 -- 172.21.15.1:0/3489358237 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d3cde9c7e0 msgr2=0x55d3cde99b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:23.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.461+0000 7ff5bdf20b80 1 --2- 172.21.15.1:0/3489358237 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d3cde9c7e0 0x55d3cde99b10 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7ff58c00bc60 tx=0x7ff58c00baf0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.461+0000 7ff5bdf20b80 1 -- 172.21.15.1:0/3489358237 shutdown_connections 2024-02-02T15:44:23.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.461+0000 7ff5bdf20b80 1 --2- 172.21.15.1:0/3489358237 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d3cde9c7e0 0x55d3cde99b10 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.461+0000 7ff5bdf20b80 1 --2- 172.21.15.1:0/3489358237 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55d3cdeabd40 0x55d3cdeae130 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.461+0000 7ff5bdf20b80 1 --2- 172.21.15.1:0/3489358237 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d3cde91010 0x55d3cde9c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.461+0000 7ff5bdf20b80 1 --2- 172.21.15.1:0/3489358237 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d3cde91930 0x55d3cde994b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.461+0000 7ff591600700 0 client.30095 ms_handle_reset on v2:172.21.15.142:6836/2870289019 2024-02-02T15:44:23.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.461+0000 7ff5bdf20b80 1 -- 172.21.15.1:0/3489358237 >> 172.21.15.1:0/3489358237 conn(0x55d3cddf0db0 msgr2=0x55d3cde8cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:23.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.461+0000 7ff5bdf20b80 1 -- 172.21.15.1:0/3489358237 shutdown_connections 2024-02-02T15:44:23.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.461+0000 7ff5bdf20b80 1 -- 172.21.15.1:0/3489358237 wait complete. 2024-02-02T15:44:23.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.462+0000 7ff5bdf20b80 20 client.30095 trim_cache size 0 max 16384 2024-02-02T15:44:23.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.478+0000 7ff5bdf20b80 1 Processor -- start 2024-02-02T15:44:23.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.478+0000 7ff5bdf20b80 1 -- start start 2024-02-02T15:44:23.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.478+0000 7ff5bdf20b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d3cde91010 0x55d3cdea33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:23.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.479+0000 7ff5bdf20b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d3cde91930 0x55d3cde93840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:23.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.479+0000 7ff5bdf20b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d3cde9c7e0 0x55d3cde9bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:23.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.479+0000 7ff5bdf20b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3cdc666a0 con 0x55d3cde91930 2024-02-02T15:44:23.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.479+0000 7ff5bdf20b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3cde94000 con 0x55d3cde9c7e0 2024-02-02T15:44:23.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.479+0000 7ff5bdf20b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3cde9a5f0 con 0x55d3cde91010 2024-02-02T15:44:23.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.479+0000 7ff5a3e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d3cde91010 0x55d3cdea33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:23.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.479+0000 7ff5a3e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d3cde91010 0x55d3cdea33d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:44676/0 (socket says 172.21.15.1:44676) 2024-02-02T15:44:23.481 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.479+0000 7ff5a3e00700 1 -- 172.21.15.1:0/738312621 learned_addr learned my addr 172.21.15.1:0/738312621 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:44:23.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.479+0000 7ff5a3e00700 1 --2- 172.21.15.1:0/738312621 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d3cde9c7e0 0x55d3cde9bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:23.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.482+0000 7ff5a3e00700 1 -- 172.21.15.1:0/738312621 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d3cde91010 msgr2=0x55d3cdea33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:23.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.482+0000 7ff5a3e00700 1 --2- 172.21.15.1:0/738312621 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d3cde91010 0x55d3cdea33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.482+0000 7ff5a3e00700 1 -- 172.21.15.1:0/738312621 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d3cde91930 msgr2=0x55d3cde93840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:23.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.482+0000 7ff5a3e00700 1 --2- 172.21.15.1:0/738312621 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d3cde91930 0x55d3cde93840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.482+0000 7ff5a3e00700 1 -- 172.21.15.1:0/738312621 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3cde890d0 con 0x55d3cde9c7e0 2024-02-02T15:44:23.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.483+0000 7ff5a3e00700 1 --2- 172.21.15.1:0/738312621 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d3cde9c7e0 0x55d3cde9bd60 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7ff58c005f50 tx=0x7ff58c009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7491f95e650b326 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:23.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.483+0000 7ff599600700 1 -- 172.21.15.1:0/738312621 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58c003a90 con 0x55d3cde9c7e0 2024-02-02T15:44:23.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.484+0000 7ff599600700 1 -- 172.21.15.1:0/738312621 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff58c01c070 con 0x55d3cde9c7e0 2024-02-02T15:44:23.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.484+0000 7ff599600700 1 -- 172.21.15.1:0/738312621 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58c024400 con 0x55d3cde9c7e0 2024-02-02T15:44:23.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.485+0000 7ff5bdf20b80 1 -- 172.21.15.1:0/738312621 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d3cde9c7e0 msgr2=0x55d3cde9bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:23.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.485+0000 7ff5bdf20b80 1 --2- 172.21.15.1:0/738312621 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d3cde9c7e0 0x55d3cde9bd60 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7ff58c005f50 tx=0x7ff58c009580 comp rx=0 tx=0).stop 2024-02-02T15:44:23.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.486+0000 7ff5bdf20b80 1 -- 172.21.15.1:0/738312621 shutdown_connections 2024-02-02T15:44:23.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.486+0000 7ff5bdf20b80 1 --2- 172.21.15.1:0/738312621 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d3cde9c7e0 0x55d3cde9bd60 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.486+0000 7ff5bdf20b80 1 --2- 172.21.15.1:0/738312621 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d3cde91010 0x55d3cdea33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.486+0000 7ff5bdf20b80 1 --2- 172.21.15.1:0/738312621 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d3cde91930 0x55d3cde93840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.486+0000 7ff5bdf20b80 1 -- 172.21.15.1:0/738312621 >> 172.21.15.1:0/738312621 conn(0x55d3cddf0db0 msgr2=0x55d3cde94840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:23.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.486+0000 7ff5bdf20b80 1 -- 172.21.15.1:0/738312621 shutdown_connections 2024-02-02T15:44:23.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.486+0000 7ff5bdf20b80 1 -- 172.21.15.1:0/738312621 wait complete. 2024-02-02T15:44:23.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.487+0000 7ff5bdf20b80 1 -- start start 2024-02-02T15:44:23.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.489+0000 7ff5bdf20b80 1 Processor -- start 2024-02-02T15:44:23.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.489+0000 7ff5bdf20b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d3cde91010 0x55d3cdea09b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:23.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.489+0000 7ff5bdf20b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d3cde91930 0x55d3cdea0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:23.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.489+0000 7ff5bdf20b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d3cde9c7e0 0x55d3cdea1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:23.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.489+0000 7ff5bdf20b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3cdefc900 con 0x55d3cde91010 2024-02-02T15:44:23.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.489+0000 7ff5bdf20b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3cdefca70 con 0x55d3cde9c7e0 2024-02-02T15:44:23.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.489+0000 7ff5bdf20b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3cdefcc30 con 0x55d3cde91930 2024-02-02T15:44:23.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.489+0000 7ff5bdf20b80 10 client.0 _ll_register_callbacks cb 0x55d3cdd80290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:44:23.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.490+0000 7ff5a3e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d3cde91010 0x55d3cdea09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:23.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.490+0000 7ff5a3e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d3cde9c7e0 0x55d3cdea1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:23.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.490+0000 7ff5a3e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d3cde9c7e0 0x55d3cdea1430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:51540/0 (socket says 172.21.15.1:51540) 2024-02-02T15:44:23.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.490+0000 7ff5a3e00700 1 -- 172.21.15.1:0/677125728 learned_addr learned my addr 172.21.15.1:0/677125728 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:44:23.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.491+0000 7ff5a2a00700 1 --2- 172.21.15.1:0/677125728 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d3cde91930 0x55d3cdea0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:23.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.491+0000 7ff5a3e00700 1 -- 172.21.15.1:0/677125728 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d3cde91930 msgr2=0x55d3cdea0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:23.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.491+0000 7ff5a3e00700 1 --2- 172.21.15.1:0/677125728 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d3cde91930 0x55d3cdea0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.491+0000 7ff5a3e00700 1 -- 172.21.15.1:0/677125728 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d3cde91010 msgr2=0x55d3cdea09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:23.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.492+0000 7ff5a3e00700 1 --2- 172.21.15.1:0/677125728 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d3cde91010 0x55d3cdea09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.492+0000 7ff5a3e00700 1 -- 172.21.15.1:0/677125728 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d3cdc10b60 con 0x55d3cde9c7e0 2024-02-02T15:44:23.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.492+0000 7ff5a3e00700 1 --2- 172.21.15.1:0/677125728 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d3cde9c7e0 0x55d3cdea1430 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7ff58c005c60 tx=0x7ff58c000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c301529a0bcb5d6 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:23.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.492+0000 7ff591600700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:44:23.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.493+0000 7ff591600700 1 -- 172.21.15.1:0/677125728 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58c040040 con 0x55d3cde9c7e0 2024-02-02T15:44:23.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.493+0000 7ff591600700 1 -- 172.21.15.1:0/677125728 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9453+0+0 (secure 0 0 0) 0x7ff58c000e40 con 0x55d3cde9c7e0 2024-02-02T15:44:23.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.493+0000 7ff5bdf20b80 1 -- 172.21.15.1:0/677125728 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff58c0055d0 con 0x55d3cde9c7e0 2024-02-02T15:44:23.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.493+0000 7ff591600700 1 -- 172.21.15.1:0/677125728 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff58c02d700 con 0x55d3cde9c7e0 2024-02-02T15:44:23.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.494+0000 7ff5bdf20b80 10 client.30101 fetch_fsmap learned FSMap version 73 2024-02-02T15:44:23.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.494+0000 7ff5bdf20b80 1 -- 172.21.15.1:0/677125728 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) v3 -- 0x55d3cdda8ea0 con 0x55d3cde9c7e0 2024-02-02T15:44:23.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.494+0000 7ff591600700 1 -- 172.21.15.1:0/677125728 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff58c0055d0 con 0x55d3cde9c7e0 2024-02-02T15:44:23.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.495+0000 7ff591600700 1 -- 172.21.15.1:0/677125728 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 73) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff58c022070 con 0x55d3cde9c7e0 2024-02-02T15:44:23.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.495+0000 7ff5bdf20b80 10 client.30101 fetch_fsmap finished waiting for FSMap version 73 2024-02-02T15:44:23.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.495+0000 7ff5bdf20b80 10 client.30101 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-02-02T15:44:23.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.495+0000 7ff5bdf20b80 20 client.30101 populate_metadata read hostname 'smithi001' 2024-02-02T15:44:23.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.495+0000 7ff5bdf20b80 1 --2- 172.21.15.1:0/677125728 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55d3cdefde60 0x55d3cdf00250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:23.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.495+0000 7ff5bdf20b80 4 client.30101 mds_command: new command op to 24514 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:44:23.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.495+0000 7ff5bdf20b80 1 -- 172.21.15.1:0/677125728 --> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d3cdda1f50 con 0x55d3cdefde60 2024-02-02T15:44:23.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.496+0000 7ff5a2a00700 1 --2- 172.21.15.1:0/677125728 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55d3cdefde60 0x55d3cdf00250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:23.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.500+0000 7ff5a2a00700 1 --2- 172.21.15.1:0/677125728 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55d3cdefde60 0x55d3cdf00250 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=46722c19c9931aba server_cookie=794a8a06342efe0a in_seq=0 out_seq=0 2024-02-02T15:44:23.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.500+0000 7ff591600700 10 client.30101 ms_handle_connect on v2:172.21.15.142:6836/2870289019 2024-02-02T15:44:23.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.500+0000 7ff591600700 1 -- 172.21.15.1:0/677125728 <== mds.3 v2:172.21.15.142:6836/2870289019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d3cdda1f50 con 0x55d3cdefde60 2024-02-02T15:44:23.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.500+0000 7ff591600700 10 client.30101 handle_command_reply: tid=0 2024-02-02T15:44:23.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.500+0000 7ff5bdf20b80 1 client.30101 shutdown 2024-02-02T15:44:23.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.501+0000 7ff5bdf20b80 1 -- 172.21.15.1:0/677125728 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d3cde9c7e0 msgr2=0x55d3cdea1430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:23.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.501+0000 7ff5bdf20b80 1 --2- 172.21.15.1:0/677125728 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d3cde9c7e0 0x55d3cdea1430 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7ff58c005c60 tx=0x7ff58c000bc0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.501+0000 7ff5bdf20b80 1 -- 172.21.15.1:0/677125728 shutdown_connections 2024-02-02T15:44:23.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.501+0000 7ff5bdf20b80 1 --2- 172.21.15.1:0/677125728 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d3cde9c7e0 0x55d3cdea1430 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.501+0000 7ff5bdf20b80 1 --2- 172.21.15.1:0/677125728 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55d3cdefde60 0x55d3cdf00250 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.501+0000 7ff5bdf20b80 1 --2- 172.21.15.1:0/677125728 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d3cde91010 0x55d3cdea09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.501+0000 7ff5bdf20b80 1 --2- 172.21.15.1:0/677125728 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d3cde91930 0x55d3cdea0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.501+0000 7ff5bdf20b80 1 -- 172.21.15.1:0/677125728 >> 172.21.15.1:0/677125728 conn(0x55d3cddf0db0 msgr2=0x55d3cde8cc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:23.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.502+0000 7ff591600700 0 client.30101 ms_handle_reset on v2:172.21.15.142:6836/2870289019 2024-02-02T15:44:23.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.502+0000 7ff5bdf20b80 1 -- 172.21.15.1:0/677125728 shutdown_connections 2024-02-02T15:44:23.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.502+0000 7ff5bdf20b80 1 -- 172.21.15.1:0/677125728 wait complete. 2024-02-02T15:44:23.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.503+0000 7ff5bdf20b80 20 client.30101 trim_cache size 0 max 16384 2024-02-02T15:44:23.504 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:44:23.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.505+0000 7ff5aa400700 1 -- 172.21.15.1:0/3303173735 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff5840a0770 msgr2=0x7ff5840a2c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:23.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.505+0000 7ff5aa400700 1 --2- 172.21.15.1:0/3303173735 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff5840a0770 0x7ff5840a2c30 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ff59c0950b0 tx=0x7ff59c009750 comp rx=0 tx=0).stop 2024-02-02T15:44:23.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.505+0000 7ff5aa400700 1 -- 172.21.15.1:0/3303173735 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5a4107d40 msgr2=0x7ff5a41a2010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:23.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.505+0000 7ff5aa400700 1 --2- 172.21.15.1:0/3303173735 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5a4107d40 0x7ff5a41a2010 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7ff59c007f10 tx=0x7ff59c00d3b0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.506+0000 7ff5aa400700 1 -- 172.21.15.1:0/3303173735 shutdown_connections 2024-02-02T15:44:23.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.506+0000 7ff5aa400700 1 --2- 172.21.15.1:0/3303173735 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff5840a0770 0x7ff5840a2c30 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.506+0000 7ff5aa400700 1 --2- 172.21.15.1:0/3303173735 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff5a410d790 0x7ff5a419d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.506+0000 7ff5aa400700 1 --2- 172.21.15.1:0/3303173735 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff5a4107d40 0x7ff5a41a2010 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.506+0000 7ff5aa400700 1 --2- 172.21.15.1:0/3303173735 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff5a4108700 0x7ff5a41a2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.506+0000 7ff5aa400700 1 -- 172.21.15.1:0/3303173735 >> 172.21.15.1:0/3303173735 conn(0x7ff5a406d120 msgr2=0x7ff5a410b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:23.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.506+0000 7ff5aa400700 1 -- 172.21.15.1:0/3303173735 shutdown_connections 2024-02-02T15:44:23.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.507+0000 7ff5aa400700 1 -- 172.21.15.1:0/3303173735 wait complete. 2024-02-02T15:44:23.541 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-02T15:44:23.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.825+0000 7f8de9000700 1 -- 172.21.15.1:0/1581458874 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8de4108660 msgr2=0x7f8de4113950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:23.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.825+0000 7f8de9000700 1 --2- 172.21.15.1:0/1581458874 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8de4108660 0x7f8de4113950 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc009990 tx=0x7f8dcc009ca0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.825+0000 7f8de9000700 1 -- 172.21.15.1:0/1581458874 shutdown_connections 2024-02-02T15:44:23.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.825+0000 7f8de9000700 1 --2- 172.21.15.1:0/1581458874 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8de4113e90 0x7f8de4116280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.825+0000 7f8de9000700 1 --2- 172.21.15.1:0/1581458874 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8de4107d40 0x7f8de4108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.825+0000 7f8de9000700 1 --2- 172.21.15.1:0/1581458874 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8de4108660 0x7f8de4113950 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.825+0000 7f8de9000700 1 -- 172.21.15.1:0/1581458874 >> 172.21.15.1:0/1581458874 conn(0x7f8de406d120 msgr2=0x7f8de406d530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:23.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.826+0000 7f8de9000700 1 -- 172.21.15.1:0/1581458874 shutdown_connections 2024-02-02T15:44:23.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.826+0000 7f8de9000700 1 -- 172.21.15.1:0/1581458874 wait complete. 2024-02-02T15:44:23.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.827+0000 7f8de9000700 1 Processor -- start 2024-02-02T15:44:23.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.828+0000 7f8de9000700 1 -- start start 2024-02-02T15:44:23.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.828+0000 7f8de9000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8de4107d40 0x7f8de41097b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:23.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.828+0000 7f8de9000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8de4108660 0x7f8de4109cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:23.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.828+0000 7f8de9000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8de4113e90 0x7f8de410a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:23.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.828+0000 7f8de9000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8de41b2010 con 0x7f8de4107d40 2024-02-02T15:44:23.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.829+0000 7f8de9000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8de41a37d0 con 0x7f8de4108660 2024-02-02T15:44:23.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.829+0000 7f8de9000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8de41a3990 con 0x7f8de4113e90 2024-02-02T15:44:23.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.830+0000 7f8de1e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8de4107d40 0x7f8de41097b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:23.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.830+0000 7f8de1e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8de4107d40 0x7f8de41097b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45014/0 (socket says 172.21.15.1:45014) 2024-02-02T15:44:23.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.830+0000 7f8de1e00700 1 -- 172.21.15.1:0/1059423538 learned_addr learned my addr 172.21.15.1:0/1059423538 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:44:23.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.830+0000 7f8de1400700 1 --2- 172.21.15.1:0/1059423538 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8de4108660 0x7f8de4109cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:23.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.830+0000 7f8de1e00700 1 -- 172.21.15.1:0/1059423538 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8de4113e90 msgr2=0x7f8de410a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:23.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.830+0000 7f8de1e00700 1 --2- 172.21.15.1:0/1059423538 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8de4113e90 0x7f8de410a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.830+0000 7f8de1e00700 1 -- 172.21.15.1:0/1059423538 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8de4108660 msgr2=0x7f8de4109cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:23.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.830+0000 7f8de1e00700 1 --2- 172.21.15.1:0/1059423538 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8de4108660 0x7f8de4109cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.830+0000 7f8de1e00700 1 -- 172.21.15.1:0/1059423538 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dcc0095f0 con 0x7f8de4107d40 2024-02-02T15:44:23.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.831+0000 7f8de1400700 1 --2- 172.21.15.1:0/1059423538 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8de4108660 0x7f8de4109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:44:23.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.831+0000 7f8de1e00700 1 --2- 172.21.15.1:0/1059423538 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8de4107d40 0x7f8de41097b0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f8dd400c9b0 tx=0x7f8dd400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:23.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.832+0000 7f8ddaa00700 1 -- 172.21.15.1:0/1059423538 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd4007b00 con 0x7f8de4107d40 2024-02-02T15:44:23.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.832+0000 7f8ddaa00700 1 -- 172.21.15.1:0/1059423538 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8dd4007c60 con 0x7f8de4107d40 2024-02-02T15:44:23.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.832+0000 7f8ddaa00700 1 -- 172.21.15.1:0/1059423538 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd4018730 con 0x7f8de4107d40 2024-02-02T15:44:23.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.833+0000 7f8de9000700 1 -- 172.21.15.1:0/1059423538 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8de41a3c70 con 0x7f8de4107d40 2024-02-02T15:44:23.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.833+0000 7f8de9000700 1 -- 172.21.15.1:0/1059423538 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8de41a4130 con 0x7f8de4107d40 2024-02-02T15:44:23.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.839+0000 7f8ddaa00700 1 -- 172.21.15.1:0/1059423538 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f8dd400f9a0 con 0x7f8de4107d40 2024-02-02T15:44:23.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.841+0000 7f8dfcaf5b80 1 Processor -- start 2024-02-02T15:44:23.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.841+0000 7f8dfcaf5b80 1 -- start start 2024-02-02T15:44:23.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.841+0000 7f8dfcaf5b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56113578e010 0x56113578e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:23.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.842+0000 7f8dfcaf5b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56113578e930 0x5611357992a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:23.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.842+0000 7f8dfcaf5b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5611357997e0 0x56113579bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:23.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.842+0000 7f8dfcaf5b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611357024c0 con 0x56113578e930 2024-02-02T15:44:23.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.842+0000 7f8dfcaf5b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611356e6aa0 con 0x5611357997e0 2024-02-02T15:44:23.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.842+0000 7f8dfcaf5b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611356e4460 con 0x56113578e010 2024-02-02T15:44:23.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.842+0000 7f8ddaa00700 1 --2- 172.21.15.1:0/1059423538 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f8dc40a0790 0x7f8dc40a2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:23.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.842+0000 7f8de1400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5611357997e0 0x56113579bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:23.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.842+0000 7f8de1e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56113578e930 0x5611357992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:23.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.842+0000 7f8de1400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5611357997e0 0x56113579bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:51562/0 (socket says 172.21.15.1:51562) 2024-02-02T15:44:23.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.842+0000 7f8de1400700 1 -- 172.21.15.1:0/2371775773 learned_addr learned my addr 172.21.15.1:0/2371775773 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:44:23.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.842+0000 7f8de2800700 1 --2- 172.21.15.1:0/1059423538 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f8dc40a0790 0x7f8dc40a2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:23.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.843+0000 7f8de2800700 1 --2- 172.21.15.1:0/2371775773 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56113578e010 0x56113578e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:23.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.843+0000 7f8ddaa00700 1 -- 172.21.15.1:0/1059423538 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f8dd40cc240 con 0x7f8de4107d40 2024-02-02T15:44:23.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.843+0000 7f8de1e00700 1 -- 172.21.15.1:0/2371775773 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56113578e010 msgr2=0x56113578e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:23.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.843+0000 7f8de1e00700 1 --2- 172.21.15.1:0/2371775773 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56113578e010 0x56113578e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.843+0000 7f8de1e00700 1 -- 172.21.15.1:0/2371775773 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5611357997e0 msgr2=0x56113579bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:23.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.843+0000 7f8de1e00700 1 --2- 172.21.15.1:0/2371775773 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5611357997e0 0x56113579bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.843+0000 7f8de1e00700 1 -- 172.21.15.1:0/2371775773 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5611355613b0 con 0x56113578e930 2024-02-02T15:44:23.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.843+0000 7f8de2800700 1 --2- 172.21.15.1:0/2371775773 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56113578e010 0x56113578e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:44:23.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.844+0000 7f8de1e00700 1 --2- 172.21.15.1:0/2371775773 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56113578e930 0x5611357992a0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f8dd40c9590 tx=0x7f8dd4099a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a8dabfa6bf6abdd server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:23.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.844+0000 7f8dd3400700 1 -- 172.21.15.1:0/2371775773 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd4099360 con 0x56113578e930 2024-02-02T15:44:23.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.845+0000 7f8dd3400700 1 -- 172.21.15.1:0/2371775773 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8dd409d190 con 0x56113578e930 2024-02-02T15:44:23.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.845+0000 7f8de2800700 1 --2- 172.21.15.1:0/1059423538 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f8dc40a0790 0x7f8dc40a2c50 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f8de406a150 tx=0x7f8ddc00b040 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:23.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.845+0000 7f8dd3400700 1 -- 172.21.15.1:0/2371775773 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd40a8770 con 0x56113578e930 2024-02-02T15:44:23.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.845+0000 7f8dfcaf5b80 1 -- 172.21.15.1:0/2371775773 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56113578e930 msgr2=0x5611357992a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:23.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.845+0000 7f8dfcaf5b80 1 --2- 172.21.15.1:0/2371775773 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56113578e930 0x5611357992a0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f8dd40c9590 tx=0x7f8dd4099a60 comp rx=0 tx=0).stop 2024-02-02T15:44:23.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.846+0000 7f8dfcaf5b80 1 -- 172.21.15.1:0/2371775773 shutdown_connections 2024-02-02T15:44:23.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.846+0000 7f8dfcaf5b80 1 --2- 172.21.15.1:0/2371775773 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5611357997e0 0x56113579bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.846+0000 7f8dfcaf5b80 1 --2- 172.21.15.1:0/2371775773 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56113578e010 0x56113578e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.846+0000 7f8dfcaf5b80 1 --2- 172.21.15.1:0/2371775773 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56113578e930 0x5611357992a0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.846+0000 7f8dfcaf5b80 1 -- 172.21.15.1:0/2371775773 >> 172.21.15.1:0/2371775773 conn(0x5611356eddb0 msgr2=0x56113578a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:23.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.846+0000 7f8dfcaf5b80 1 -- 172.21.15.1:0/2371775773 shutdown_connections 2024-02-02T15:44:23.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.847+0000 7f8dfcaf5b80 1 -- 172.21.15.1:0/2371775773 wait complete. 2024-02-02T15:44:23.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.847+0000 7f8dfcaf5b80 1 -- start start 2024-02-02T15:44:23.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.848+0000 7f8dfcaf5b80 1 Processor -- start 2024-02-02T15:44:23.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.848+0000 7f8dfcaf5b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56113578e010 0x561135799250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:23.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.848+0000 7f8dfcaf5b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56113578e930 0x561135796460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:23.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.848+0000 7f8dfcaf5b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611357997e0 0x561135796b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:23.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.848+0000 7f8dfcaf5b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611356dfd50 con 0x5611357997e0 2024-02-02T15:44:23.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.848+0000 7f8dfcaf5b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56113567bd80 con 0x56113578e010 2024-02-02T15:44:23.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.848+0000 7f8dfcaf5b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56113567ba20 con 0x56113578e930 2024-02-02T15:44:23.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.848+0000 7f8dfcaf5b80 10 client.0 _ll_register_callbacks cb 0x56113567d290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:44:23.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.849+0000 7f8de1e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56113578e010 0x561135799250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:23.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.849+0000 7f8de1e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56113578e010 0x561135799250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:51578/0 (socket says 172.21.15.1:51578) 2024-02-02T15:44:23.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.849+0000 7f8de1e00700 1 -- 172.21.15.1:0/3909237249 learned_addr learned my addr 172.21.15.1:0/3909237249 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:44:23.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.849+0000 7f8de1400700 1 --2- 172.21.15.1:0/3909237249 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56113578e930 0x561135796460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:23.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.849+0000 7f8de2800700 1 --2- 172.21.15.1:0/3909237249 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611357997e0 0x561135796b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:23.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.850+0000 7f8de1e00700 1 -- 172.21.15.1:0/3909237249 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56113578e930 msgr2=0x561135796460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:23.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.850+0000 7f8de1e00700 1 --2- 172.21.15.1:0/3909237249 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56113578e930 0x561135796460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.850+0000 7f8de1e00700 1 -- 172.21.15.1:0/3909237249 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611357997e0 msgr2=0x561135796b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:23.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.850+0000 7f8de1e00700 1 --2- 172.21.15.1:0/3909237249 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611357997e0 0x561135796b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.850+0000 7f8de1e00700 1 -- 172.21.15.1:0/3909237249 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8de41a4130 con 0x56113578e010 2024-02-02T15:44:23.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.850+0000 7f8de1e00700 1 --2- 172.21.15.1:0/3909237249 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56113578e010 0x561135799250 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f8dd40a0ea0 tx=0x7f8dd40a13d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1e75d5ed1ab5e23 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:23.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.850+0000 7f8de2800700 1 --2- 172.21.15.1:0/3909237249 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611357997e0 0x561135796b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:44:23.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.850+0000 7f8dcbe00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:44:23.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.850+0000 7f8dcbe00700 1 -- 172.21.15.1:0/3909237249 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd40997c0 con 0x56113578e010 2024-02-02T15:44:23.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.850+0000 7f8dcbe00700 1 -- 172.21.15.1:0/3909237249 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f8dd4098850 con 0x56113578e010 2024-02-02T15:44:23.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.850+0000 7f8dfcaf5b80 1 -- 172.21.15.1:0/3909237249 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5611357820f0 con 0x56113578e010 2024-02-02T15:44:23.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.851+0000 7f8de1400700 1 --2- 172.21.15.1:0/3909237249 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56113578e930 0x561135796460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:44:23.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.851+0000 7f8dcbe00700 1 -- 172.21.15.1:0/3909237249 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8dd40b7340 con 0x56113578e010 2024-02-02T15:44:23.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.851+0000 7f8dfcaf5b80 10 client.30110 fetch_fsmap learned FSMap version 73 2024-02-02T15:44:23.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.851+0000 7f8dfcaf5b80 1 -- 172.21.15.1:0/3909237249 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) v3 -- 0x56113550db60 con 0x56113578e010 2024-02-02T15:44:23.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.851+0000 7f8dcbe00700 1 -- 172.21.15.1:0/3909237249 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5611357820f0 con 0x56113578e010 2024-02-02T15:44:23.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.852+0000 7f8dcbe00700 1 -- 172.21.15.1:0/3909237249 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 73) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8dd40a0fd0 con 0x56113578e010 2024-02-02T15:44:23.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.852+0000 7f8dfcaf5b80 10 client.30110 fetch_fsmap finished waiting for FSMap version 73 2024-02-02T15:44:23.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.852+0000 7f8dfcaf5b80 10 client.30110 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-02-02T15:44:23.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.852+0000 7f8dfcaf5b80 20 client.30110 populate_metadata read hostname 'smithi001' 2024-02-02T15:44:23.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.852+0000 7f8dfcaf5b80 1 --2- 172.21.15.1:0/3909237249 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x5611357a8c10 0x5611357ab030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:23.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.852+0000 7f8dfcaf5b80 4 client.30110 mds_command: new command op to 24518 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:44:23.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.852+0000 7f8dfcaf5b80 1 -- 172.21.15.1:0/3909237249 --> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5611356d8dd0 con 0x5611357a8c10 2024-02-02T15:44:23.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.853+0000 7f8de1400700 1 --2- 172.21.15.1:0/3909237249 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x5611357a8c10 0x5611357ab030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:23.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.854+0000 7f8de1400700 1 --2- 172.21.15.1:0/3909237249 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x5611357a8c10 0x5611357ab030 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=df930e90e0fae56 server_cookie=d2d6fd88f5e5cd5c in_seq=0 out_seq=0 2024-02-02T15:44:23.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.854+0000 7f8dcbe00700 10 client.30110 ms_handle_connect on v2:172.21.15.71:6836/628016199 2024-02-02T15:44:23.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.857+0000 7f8dcbe00700 1 -- 172.21.15.1:0/3909237249 <== mds.2 v2:172.21.15.71:6836/628016199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5611356d8dd0 con 0x5611357a8c10 2024-02-02T15:44:23.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.857+0000 7f8dcbe00700 10 client.30110 handle_command_reply: tid=0 2024-02-02T15:44:23.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.857+0000 7f8dfcaf5b80 1 client.30110 shutdown 2024-02-02T15:44:23.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.857+0000 7f8dfcaf5b80 1 -- 172.21.15.1:0/3909237249 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56113578e010 msgr2=0x561135799250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:23.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.857+0000 7f8dfcaf5b80 1 --2- 172.21.15.1:0/3909237249 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56113578e010 0x561135799250 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f8dd40a0ea0 tx=0x7f8dd40a13d0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.857+0000 7f8dfcaf5b80 1 -- 172.21.15.1:0/3909237249 shutdown_connections 2024-02-02T15:44:23.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.857+0000 7f8dfcaf5b80 1 --2- 172.21.15.1:0/3909237249 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611357997e0 0x561135796b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.857+0000 7f8dfcaf5b80 1 --2- 172.21.15.1:0/3909237249 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56113578e010 0x561135799250 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.857+0000 7f8dfcaf5b80 1 --2- 172.21.15.1:0/3909237249 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x5611357a8c10 0x5611357ab030 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.857+0000 7f8dcbe00700 0 client.30110 ms_handle_reset on v2:172.21.15.71:6836/628016199 2024-02-02T15:44:23.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.857+0000 7f8dfcaf5b80 1 --2- 172.21.15.1:0/3909237249 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56113578e930 0x561135796460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.857+0000 7f8dfcaf5b80 1 -- 172.21.15.1:0/3909237249 >> 172.21.15.1:0/3909237249 conn(0x5611356eddb0 msgr2=0x561135789fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:23.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.858+0000 7f8dfcaf5b80 1 -- 172.21.15.1:0/3909237249 shutdown_connections 2024-02-02T15:44:23.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.858+0000 7f8dfcaf5b80 1 -- 172.21.15.1:0/3909237249 wait complete. 2024-02-02T15:44:23.858 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.858+0000 7f8dfcaf5b80 20 client.30110 trim_cache size 0 max 16384 2024-02-02T15:44:23.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:44:23 smithi071 ceph-mon[44707]: pgmap v546: 129 pgs: 129 active+clean; 1.8 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.5 MiB/s wr, 511 op/s 2024-02-02T15:44:23.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:44:23 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/2790619164' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:44:23.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.875+0000 7f8dfcaf5b80 1 Processor -- start 2024-02-02T15:44:23.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.875+0000 7f8dfcaf5b80 1 -- start start 2024-02-02T15:44:23.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.875+0000 7f8dfcaf5b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56113578e010 0x561135790840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:23.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.876+0000 7f8dfcaf5b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56113578e930 0x561135798d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:23.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.876+0000 7f8dfcaf5b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5611357997e0 0x561135795d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:23.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.876+0000 7f8dfcaf5b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611355636a0 con 0x56113578e930 2024-02-02T15:44:23.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.876+0000 7f8dfcaf5b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56113578e770 con 0x5611357997e0 2024-02-02T15:44:23.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.876+0000 7f8dfcaf5b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x561135781cc0 con 0x56113578e010 2024-02-02T15:44:23.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.876+0000 7f8de2800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5611357997e0 0x561135795d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:23.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.876+0000 7f8de1e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56113578e010 0x561135790840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:23.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.876+0000 7f8de2800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5611357997e0 0x561135795d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:51580/0 (socket says 172.21.15.1:51580) 2024-02-02T15:44:23.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.876+0000 7f8de1e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56113578e010 0x561135790840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:44720/0 (socket says 172.21.15.1:44720) 2024-02-02T15:44:23.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.876+0000 7f8de2800700 1 -- 172.21.15.1:0/311968034 learned_addr learned my addr 172.21.15.1:0/311968034 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:44:23.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.876+0000 7f8de1400700 1 --2- 172.21.15.1:0/311968034 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56113578e930 0x561135798d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:23.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.876+0000 7f8de2800700 1 -- 172.21.15.1:0/311968034 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56113578e010 msgr2=0x561135790840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:23.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.876+0000 7f8de2800700 1 --2- 172.21.15.1:0/311968034 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56113578e010 0x561135790840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.876+0000 7f8de2800700 1 -- 172.21.15.1:0/311968034 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56113578e930 msgr2=0x561135798d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:23.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.876+0000 7f8de2800700 1 --2- 172.21.15.1:0/311968034 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56113578e930 0x561135798d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.876+0000 7f8de2800700 1 -- 172.21.15.1:0/311968034 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5611356a5ea0 con 0x5611357997e0 2024-02-02T15:44:23.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.877+0000 7f8de2800700 1 --2- 172.21.15.1:0/311968034 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5611357997e0 0x561135795d00 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc009290 tx=0x7f8ddc013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26356e3462d72185 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:23.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.878+0000 7f8dd3e00700 1 -- 172.21.15.1:0/311968034 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ddc015d90 con 0x5611357997e0 2024-02-02T15:44:23.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.878+0000 7f8dd3e00700 1 -- 172.21.15.1:0/311968034 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ddc01b050 con 0x5611357997e0 2024-02-02T15:44:23.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.878+0000 7f8dd3e00700 1 -- 172.21.15.1:0/311968034 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ddc020790 con 0x5611357997e0 2024-02-02T15:44:23.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.879+0000 7f8dfcaf5b80 1 -- 172.21.15.1:0/311968034 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5611357997e0 msgr2=0x561135795d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:23.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.879+0000 7f8dfcaf5b80 1 --2- 172.21.15.1:0/311968034 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5611357997e0 0x561135795d00 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc009290 tx=0x7f8ddc013000 comp rx=0 tx=0).stop 2024-02-02T15:44:23.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.879+0000 7f8dfcaf5b80 1 -- 172.21.15.1:0/311968034 shutdown_connections 2024-02-02T15:44:23.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.879+0000 7f8dfcaf5b80 1 --2- 172.21.15.1:0/311968034 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5611357997e0 0x561135795d00 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.879+0000 7f8dfcaf5b80 1 --2- 172.21.15.1:0/311968034 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56113578e010 0x561135790840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.879+0000 7f8dfcaf5b80 1 --2- 172.21.15.1:0/311968034 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56113578e930 0x561135798d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.879+0000 7f8dfcaf5b80 1 -- 172.21.15.1:0/311968034 >> 172.21.15.1:0/311968034 conn(0x5611356eddb0 msgr2=0x5611357917e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:23.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.879+0000 7f8dfcaf5b80 1 -- 172.21.15.1:0/311968034 shutdown_connections 2024-02-02T15:44:23.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.879+0000 7f8dfcaf5b80 1 -- 172.21.15.1:0/311968034 wait complete. 2024-02-02T15:44:23.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.880+0000 7f8dfcaf5b80 1 -- start start 2024-02-02T15:44:23.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.880+0000 7f8dfcaf5b80 1 Processor -- start 2024-02-02T15:44:23.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.881+0000 7f8dfcaf5b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56113578e010 0x56113579e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:23.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.881+0000 7f8dfcaf5b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56113578e930 0x56113579e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:23.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.881+0000 7f8dfcaf5b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611357997e0 0x5611357f36c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:23.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.881+0000 7f8dfcaf5b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611357f3c70 con 0x5611357997e0 2024-02-02T15:44:23.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.881+0000 7f8dfcaf5b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611357f3de0 con 0x56113578e010 2024-02-02T15:44:23.882 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.881+0000 7f8dfcaf5b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5611357f3fa0 con 0x56113578e930 2024-02-02T15:44:23.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.881+0000 7f8dfcaf5b80 10 client.0 _ll_register_callbacks cb 0x56113567d290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:44:23.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.881+0000 7f8de1e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56113578e010 0x56113579e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:23.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.881+0000 7f8de1e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56113578e010 0x56113579e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:51594/0 (socket says 172.21.15.1:51594) 2024-02-02T15:44:23.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.881+0000 7f8de1e00700 1 -- 172.21.15.1:0/2291567926 learned_addr learned my addr 172.21.15.1:0/2291567926 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:44:23.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.881+0000 7f8de2800700 1 --2- 172.21.15.1:0/2291567926 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611357997e0 0x5611357f36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:23.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.881+0000 7f8de1400700 1 --2- 172.21.15.1:0/2291567926 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56113578e930 0x56113579e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:23.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.881+0000 7f8de1e00700 1 -- 172.21.15.1:0/2291567926 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56113578e930 msgr2=0x56113579e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:23.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.881+0000 7f8de1e00700 1 --2- 172.21.15.1:0/2291567926 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56113578e930 0x56113579e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.881+0000 7f8de1e00700 1 -- 172.21.15.1:0/2291567926 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611357997e0 msgr2=0x5611357f36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:23.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.881+0000 7f8de1e00700 1 --2- 172.21.15.1:0/2291567926 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611357997e0 0x5611357f36c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.881+0000 7f8de1e00700 1 -- 172.21.15.1:0/2291567926 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ddc00fe30 con 0x56113578e010 2024-02-02T15:44:23.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.882+0000 7f8de1400700 1 --2- 172.21.15.1:0/2291567926 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56113578e930 0x56113579e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:44:23.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.882+0000 7f8de1e00700 1 --2- 172.21.15.1:0/2291567926 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56113578e010 0x56113579e4a0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f8dd400dfd0 tx=0x7f8dd4099260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be8e345cd9e98a89 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:23.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.882+0000 7f8dcbe00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:44:23.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.882+0000 7f8dcbe00700 1 -- 172.21.15.1:0/2291567926 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd40993e0 con 0x56113578e010 2024-02-02T15:44:23.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.882+0000 7f8dfcaf5b80 1 -- 172.21.15.1:0/2291567926 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8dd409d190 con 0x56113578e010 2024-02-02T15:44:23.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.882+0000 7f8dcbe00700 1 -- 172.21.15.1:0/2291567926 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f8dd4082b60 con 0x56113578e010 2024-02-02T15:44:23.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.883+0000 7f8dcbe00700 1 -- 172.21.15.1:0/2291567926 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8dd40b7340 con 0x56113578e010 2024-02-02T15:44:23.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.883+0000 7f8dfcaf5b80 10 client.30116 fetch_fsmap learned FSMap version 73 2024-02-02T15:44:23.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.883+0000 7f8dfcaf5b80 1 -- 172.21.15.1:0/2291567926 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) v3 -- 0x561135565cd0 con 0x56113578e010 2024-02-02T15:44:23.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.883+0000 7f8dcbe00700 1 -- 172.21.15.1:0/2291567926 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8dd409d190 con 0x56113578e010 2024-02-02T15:44:23.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.883+0000 7f8dcbe00700 1 -- 172.21.15.1:0/2291567926 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 73) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8dd400f450 con 0x56113578e010 2024-02-02T15:44:23.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.883+0000 7f8dfcaf5b80 10 client.30116 fetch_fsmap finished waiting for FSMap version 73 2024-02-02T15:44:23.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.883+0000 7f8dfcaf5b80 10 client.30116 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-02-02T15:44:23.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.883+0000 7f8dfcaf5b80 20 client.30116 populate_metadata read hostname 'smithi001' 2024-02-02T15:44:23.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.883+0000 7f8dfcaf5b80 1 --2- 172.21.15.1:0/2291567926 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x5611357fb300 0x5611357fd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:23.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.883+0000 7f8dfcaf5b80 4 client.30116 mds_command: new command op to 24518 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:44:23.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.883+0000 7f8dfcaf5b80 1 -- 172.21.15.1:0/2291567926 --> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56113569ef50 con 0x5611357fb300 2024-02-02T15:44:23.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.884+0000 7f8de1400700 1 --2- 172.21.15.1:0/2291567926 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x5611357fb300 0x5611357fd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:23.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.884+0000 7f8de1400700 1 --2- 172.21.15.1:0/2291567926 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x5611357fb300 0x5611357fd6f0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=90dd4fab1fbfe891 server_cookie=8a6a1f9a50db4a09 in_seq=0 out_seq=0 2024-02-02T15:44:23.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.884+0000 7f8dcbe00700 10 client.30116 ms_handle_connect on v2:172.21.15.71:6836/628016199 2024-02-02T15:44:23.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.886+0000 7f8dcbe00700 1 -- 172.21.15.1:0/2291567926 <== mds.2 v2:172.21.15.71:6836/628016199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56113569ef50 con 0x5611357fb300 2024-02-02T15:44:23.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.887+0000 7f8dcbe00700 10 client.30116 handle_command_reply: tid=0 2024-02-02T15:44:23.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.887+0000 7f8dfcaf5b80 1 client.30116 shutdown 2024-02-02T15:44:23.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.887+0000 7f8dfcaf5b80 1 -- 172.21.15.1:0/2291567926 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56113578e010 msgr2=0x56113579e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:23.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.887+0000 7f8dfcaf5b80 1 --2- 172.21.15.1:0/2291567926 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56113578e010 0x56113579e4a0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f8dd400dfd0 tx=0x7f8dd4099260 comp rx=0 tx=0).stop 2024-02-02T15:44:23.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.887+0000 7f8dfcaf5b80 1 -- 172.21.15.1:0/2291567926 shutdown_connections 2024-02-02T15:44:23.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.887+0000 7f8dfcaf5b80 1 --2- 172.21.15.1:0/2291567926 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5611357997e0 0x5611357f36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.887+0000 7f8dfcaf5b80 1 --2- 172.21.15.1:0/2291567926 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56113578e010 0x56113579e4a0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.887+0000 7f8dfcaf5b80 1 --2- 172.21.15.1:0/2291567926 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x5611357fb300 0x5611357fd6f0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.887+0000 7f8dfcaf5b80 1 --2- 172.21.15.1:0/2291567926 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56113578e930 0x56113579e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.887+0000 7f8dfcaf5b80 1 -- 172.21.15.1:0/2291567926 >> 172.21.15.1:0/2291567926 conn(0x5611356eddb0 msgr2=0x5611357895c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:23.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.887+0000 7f8dcbe00700 0 client.30116 ms_handle_reset on v2:172.21.15.71:6836/628016199 2024-02-02T15:44:23.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.887+0000 7f8dfcaf5b80 1 -- 172.21.15.1:0/2291567926 shutdown_connections 2024-02-02T15:44:23.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.887+0000 7f8dfcaf5b80 1 -- 172.21.15.1:0/2291567926 wait complete. 2024-02-02T15:44:23.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.888+0000 7f8dfcaf5b80 20 client.30116 trim_cache size 0 max 16384 2024-02-02T15:44:23.890 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:44:23.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.888+0000 7f8de9000700 1 -- 172.21.15.1:0/1059423538 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f8dc40a0790 msgr2=0x7f8dc40a2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:23.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.888+0000 7f8de9000700 1 --2- 172.21.15.1:0/1059423538 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f8dc40a0790 0x7f8dc40a2c50 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f8de406a150 tx=0x7f8ddc00b040 comp rx=0 tx=0).stop 2024-02-02T15:44:23.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.888+0000 7f8de9000700 1 -- 172.21.15.1:0/1059423538 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8de4107d40 msgr2=0x7f8de41097b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:23.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.889+0000 7f8de9000700 1 --2- 172.21.15.1:0/1059423538 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8de4107d40 0x7f8de41097b0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f8dd400c9b0 tx=0x7f8dd400cd70 comp rx=0 tx=0).stop 2024-02-02T15:44:23.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.889+0000 7f8de9000700 1 -- 172.21.15.1:0/1059423538 shutdown_connections 2024-02-02T15:44:23.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.889+0000 7f8de9000700 1 --2- 172.21.15.1:0/1059423538 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f8dc40a0790 0x7f8dc40a2c50 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.889+0000 7f8de9000700 1 --2- 172.21.15.1:0/1059423538 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8de4113e90 0x7f8de410a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.889+0000 7f8de9000700 1 --2- 172.21.15.1:0/1059423538 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8de4107d40 0x7f8de41097b0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.889+0000 7f8de9000700 1 --2- 172.21.15.1:0/1059423538 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8de4108660 0x7f8de4109cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:23.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.889+0000 7f8de9000700 1 -- 172.21.15.1:0/1059423538 >> 172.21.15.1:0/1059423538 conn(0x7f8de406d120 msgr2=0x7f8de406f690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:23.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.889+0000 7f8de9000700 1 -- 172.21.15.1:0/1059423538 shutdown_connections 2024-02-02T15:44:23.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:23.889+0000 7f8de9000700 1 -- 172.21.15.1:0/1059423538 wait complete. 2024-02-02T15:44:23.909 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:44:23 smithi001 ceph-mon[35470]: pgmap v546: 129 pgs: 129 active+clean; 1.8 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.5 MiB/s wr, 511 op/s 2024-02-02T15:44:23.909 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:44:23 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/2790619164' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:44:23.912 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-02T15:44:23.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:44:23 smithi142 ceph-mon[45728]: pgmap v546: 129 pgs: 129 active+clean; 1.8 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.5 MiB/s wr, 511 op/s 2024-02-02T15:44:23.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:44:23 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/2790619164' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:44:23.993 INFO:tasks.workunit.client.0.smithi001.stdout: 10 131554 28.77 MB/sec execute 116 sec latency 10085.628 ms 2024-02-02T15:44:24.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.211+0000 7fdb59c00700 1 -- 172.21.15.1:0/2777463913 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdb54106d00 msgr2=0x7fdb541070e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:24.212 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.211+0000 7fdb59c00700 1 --2- 172.21.15.1:0/2777463913 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdb54106d00 0x7fdb541070e0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fdb44009990 tx=0x7fdb44009ca0 comp rx=0 tx=0).stop 2024-02-02T15:44:24.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.212+0000 7fdb59c00700 1 -- 172.21.15.1:0/2777463913 shutdown_connections 2024-02-02T15:44:24.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.212+0000 7fdb59c00700 1 --2- 172.21.15.1:0/2777463913 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb540689d0 0x7fdb5410db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:24.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.212+0000 7fdb59c00700 1 --2- 172.21.15.1:0/2777463913 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdb54106d00 0x7fdb541070e0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:24.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.212+0000 7fdb59c00700 1 --2- 172.21.15.1:0/2777463913 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb54068010 0x7fdb54068490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:24.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.212+0000 7fdb59c00700 1 -- 172.21.15.1:0/2777463913 >> 172.21.15.1:0/2777463913 conn(0x7fdb540756b0 msgr2=0x7fdb54075ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:24.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.212+0000 7fdb59c00700 1 -- 172.21.15.1:0/2777463913 shutdown_connections 2024-02-02T15:44:24.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.212+0000 7fdb59c00700 1 -- 172.21.15.1:0/2777463913 wait complete. 2024-02-02T15:44:24.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.216+0000 7fdb59c00700 1 Processor -- start 2024-02-02T15:44:24.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.217+0000 7fdb59c00700 1 -- start start 2024-02-02T15:44:24.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.218+0000 7fdb59c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb54068010 0x7fdb5419d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:24.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.218+0000 7fdb59c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdb540689d0 0x7fdb5419d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:24.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.218+0000 7fdb59c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb54106d00 0x7fdb541a1d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:24.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.218+0000 7fdb59c00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb541a2340 con 0x7fdb54106d00 2024-02-02T15:44:24.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.218+0000 7fdb59c00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb541a24b0 con 0x7fdb54068010 2024-02-02T15:44:24.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.218+0000 7fdb59c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb541a2670 con 0x7fdb540689d0 2024-02-02T15:44:24.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.218+0000 7fdb52a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb54068010 0x7fdb5419d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:24.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.218+0000 7fdb52a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb54068010 0x7fdb5419d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:51602/0 (socket says 172.21.15.1:51602) 2024-02-02T15:44:24.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.218+0000 7fdb52a00700 1 -- 172.21.15.1:0/3437319816 learned_addr learned my addr 172.21.15.1:0/3437319816 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:44:24.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.218+0000 7fdb52000700 1 --2- 172.21.15.1:0/3437319816 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdb540689d0 0x7fdb5419d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:24.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.218+0000 7fdb53400700 1 --2- 172.21.15.1:0/3437319816 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb54106d00 0x7fdb541a1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:24.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.219+0000 7fdb52000700 1 -- 172.21.15.1:0/3437319816 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb54068010 msgr2=0x7fdb5419d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:24.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.219+0000 7fdb52000700 1 --2- 172.21.15.1:0/3437319816 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb54068010 0x7fdb5419d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:24.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.219+0000 7fdb52000700 1 -- 172.21.15.1:0/3437319816 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb54106d00 msgr2=0x7fdb541a1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:24.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.219+0000 7fdb52000700 1 --2- 172.21.15.1:0/3437319816 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb54106d00 0x7fdb541a1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:24.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.219+0000 7fdb52000700 1 -- 172.21.15.1:0/3437319816 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb440095f0 con 0x7fdb540689d0 2024-02-02T15:44:24.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.219+0000 7fdb52000700 1 --2- 172.21.15.1:0/3437319816 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdb540689d0 0x7fdb5419d8a0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c00d750 tx=0x7fdb3c00da60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:24.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.220+0000 7fdb4b400700 1 -- 172.21.15.1:0/3437319816 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb3c004020 con 0x7fdb540689d0 2024-02-02T15:44:24.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.220+0000 7fdb4b400700 1 -- 172.21.15.1:0/3437319816 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb3c004830 con 0x7fdb540689d0 2024-02-02T15:44:24.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.220+0000 7fdb59c00700 1 -- 172.21.15.1:0/3437319816 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb541a2950 con 0x7fdb540689d0 2024-02-02T15:44:24.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.220+0000 7fdb4b400700 1 -- 172.21.15.1:0/3437319816 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb3c0054c0 con 0x7fdb540689d0 2024-02-02T15:44:24.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.220+0000 7fdb59c00700 1 -- 172.21.15.1:0/3437319816 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb541aa190 con 0x7fdb540689d0 2024-02-02T15:44:24.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.224+0000 7fdb4b400700 1 -- 172.21.15.1:0/3437319816 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7fdb3c0049a0 con 0x7fdb540689d0 2024-02-02T15:44:24.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.226+0000 7fdb6d64db80 1 Processor -- start 2024-02-02T15:44:24.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.226+0000 7fdb4b400700 1 --2- 172.21.15.1:0/3437319816 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fdb340a0670 0x7fdb340a2b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:24.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.226+0000 7fdb6d64db80 1 -- start start 2024-02-02T15:44:24.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.227+0000 7fdb4b400700 1 -- 172.21.15.1:0/3437319816 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fdb3c0cbab0 con 0x7fdb540689d0 2024-02-02T15:44:24.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.227+0000 7fdb53400700 1 --2- 172.21.15.1:0/3437319816 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fdb340a0670 0x7fdb340a2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:24.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.227+0000 7fdb6d64db80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eeec7c8010 0x55eeec7c83f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:24.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.227+0000 7fdb6d64db80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eeec7c8930 0x55eeec7d32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:24.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.227+0000 7fdb6d64db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eeec7d37e0 0x55eeec7d5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:24.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.227+0000 7fdb6d64db80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eeec73c4c0 con 0x55eeec7c8930 2024-02-02T15:44:24.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.227+0000 7fdb6d64db80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eeec720aa0 con 0x55eeec7c8010 2024-02-02T15:44:24.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.227+0000 7fdb6d64db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eeec71e460 con 0x55eeec7d37e0 2024-02-02T15:44:24.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.227+0000 7fdb52a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eeec7c8010 0x55eeec7c83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:24.228 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.227+0000 7fdb52a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eeec7c8010 0x55eeec7c83f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:51604/0 (socket says 172.21.15.1:51604) 2024-02-02T15:44:24.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.227+0000 7fdb52a00700 1 -- 172.21.15.1:0/1066360488 learned_addr learned my addr 172.21.15.1:0/1066360488 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:44:24.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.227+0000 7fdb53400700 1 --2- 172.21.15.1:0/1066360488 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eeec7d37e0 0x55eeec7d5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:24.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.228+0000 7fdb52000700 1 --2- 172.21.15.1:0/1066360488 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eeec7c8930 0x55eeec7d32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:24.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.228+0000 7fdb53400700 1 --2- 172.21.15.1:0/3437319816 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fdb340a0670 0x7fdb340a2b30 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c00f370 tx=0x7fdb4c007950 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:24.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.228+0000 7fdb52a00700 1 -- 172.21.15.1:0/1066360488 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eeec7d37e0 msgr2=0x55eeec7d5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:24.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.228+0000 7fdb52a00700 1 --2- 172.21.15.1:0/1066360488 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eeec7d37e0 0x55eeec7d5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:24.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.228+0000 7fdb52a00700 1 -- 172.21.15.1:0/1066360488 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eeec7c8930 msgr2=0x55eeec7d32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:24.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.228+0000 7fdb52a00700 1 --2- 172.21.15.1:0/1066360488 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eeec7c8930 0x55eeec7d32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:24.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.228+0000 7fdb52a00700 1 -- 172.21.15.1:0/1066360488 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eeec59b3b0 con 0x55eeec7c8010 2024-02-02T15:44:24.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.228+0000 7fdb52000700 1 --2- 172.21.15.1:0/1066360488 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eeec7c8930 0x55eeec7d32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:44:24.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.229+0000 7fdb52a00700 1 --2- 172.21.15.1:0/1066360488 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eeec7c8010 0x55eeec7c83f0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fdb44005cd0 tx=0x7fdb44003680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd8e815d37670d28 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:24.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.229+0000 7fdb43e00700 1 -- 172.21.15.1:0/1066360488 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb4401d070 con 0x55eeec7c8010 2024-02-02T15:44:24.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.229+0000 7fdb43e00700 1 -- 172.21.15.1:0/1066360488 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb4401e040 con 0x55eeec7c8010 2024-02-02T15:44:24.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.229+0000 7fdb43e00700 1 -- 172.21.15.1:0/1066360488 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb44005550 con 0x55eeec7c8010 2024-02-02T15:44:24.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.230+0000 7fdb6d64db80 1 -- 172.21.15.1:0/1066360488 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eeec7c8010 msgr2=0x55eeec7c83f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:24.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.230+0000 7fdb6d64db80 1 --2- 172.21.15.1:0/1066360488 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eeec7c8010 0x55eeec7c83f0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fdb44005cd0 tx=0x7fdb44003680 comp rx=0 tx=0).stop 2024-02-02T15:44:24.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.231+0000 7fdb6d64db80 1 -- 172.21.15.1:0/1066360488 shutdown_connections 2024-02-02T15:44:24.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.231+0000 7fdb6d64db80 1 --2- 172.21.15.1:0/1066360488 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eeec7d37e0 0x55eeec7d5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:24.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.231+0000 7fdb6d64db80 1 --2- 172.21.15.1:0/1066360488 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eeec7c8010 0x55eeec7c83f0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:24.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.231+0000 7fdb6d64db80 1 --2- 172.21.15.1:0/1066360488 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eeec7c8930 0x55eeec7d32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:24.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.231+0000 7fdb6d64db80 1 -- 172.21.15.1:0/1066360488 >> 172.21.15.1:0/1066360488 conn(0x55eeec727db0 msgr2=0x55eeec7c43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:24.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.231+0000 7fdb6d64db80 1 -- 172.21.15.1:0/1066360488 shutdown_connections 2024-02-02T15:44:24.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.231+0000 7fdb6d64db80 1 -- 172.21.15.1:0/1066360488 wait complete. 2024-02-02T15:44:24.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.232+0000 7fdb6d64db80 1 -- start start 2024-02-02T15:44:24.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.233+0000 7fdb6d64db80 1 Processor -- start 2024-02-02T15:44:24.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.234+0000 7fdb6d64db80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eeec7c8010 0x55eeec7d3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:24.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.235+0000 7fdb6d64db80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eeec7c8930 0x55eeec7d03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:24.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.235+0000 7fdb6d64db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eeec7d37e0 0x55eeec7d0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:24.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.235+0000 7fdb6d64db80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eeec719d50 con 0x55eeec7c8010 2024-02-02T15:44:24.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.235+0000 7fdb52a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eeec7c8010 0x55eeec7d3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:24.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.235+0000 7fdb52a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eeec7c8010 0x55eeec7d3150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45076/0 (socket says 172.21.15.1:45076) 2024-02-02T15:44:24.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.235+0000 7fdb52a00700 1 -- 172.21.15.1:0/1288666758 learned_addr learned my addr 172.21.15.1:0/1288666758 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:44:24.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.236+0000 7fdb6d64db80 1 -- 172.21.15.1:0/1288666758 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eeec6b5d80 con 0x55eeec7c8930 2024-02-02T15:44:24.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.236+0000 7fdb6d64db80 1 -- 172.21.15.1:0/1288666758 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eeec6b5a20 con 0x55eeec7d37e0 2024-02-02T15:44:24.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.236+0000 7fdb52000700 1 --2- 172.21.15.1:0/1288666758 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eeec7c8930 0x55eeec7d03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:24.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.236+0000 7fdb52a00700 1 -- 172.21.15.1:0/1288666758 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eeec7d37e0 msgr2=0x55eeec7d0a00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-02T15:44:24.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.236+0000 7fdb52a00700 1 --2- 172.21.15.1:0/1288666758 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eeec7d37e0 0x55eeec7d0a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:24.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.236+0000 7fdb52a00700 1 -- 172.21.15.1:0/1288666758 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eeec7c8930 msgr2=0x55eeec7d03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:24.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.236+0000 7fdb52a00700 1 --2- 172.21.15.1:0/1288666758 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eeec7c8930 0x55eeec7d03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:24.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.236+0000 7fdb52a00700 1 -- 172.21.15.1:0/1288666758 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb5410e410 con 0x55eeec7c8010 2024-02-02T15:44:24.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.237+0000 7fdb6d64db80 10 client.0 _ll_register_callbacks cb 0x55eeec6b7290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:44:24.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.237+0000 7fdb6d64db80 1 -- 172.21.15.1:0/1288666758 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55eeec7bc0f0 con 0x55eeec7c8010 2024-02-02T15:44:24.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.237+0000 7fdb52a00700 1 --2- 172.21.15.1:0/1288666758 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eeec7c8010 0x55eeec7d3150 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fdb44000c00 tx=0x7fdb4400bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd4c6e1ff4759853 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:24.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.237+0000 7fdb40c00700 10 client.19854 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:44:24.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.238+0000 7fdb40c00700 1 -- 172.21.15.1:0/1288666758 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb4401d040 con 0x55eeec7c8010 2024-02-02T15:44:24.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.238+0000 7fdb40c00700 1 -- 172.21.15.1:0/1288666758 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fdb44005980 con 0x55eeec7c8010 2024-02-02T15:44:24.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.239+0000 7fdb40c00700 1 -- 172.21.15.1:0/1288666758 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb44003e60 con 0x55eeec7c8010 2024-02-02T15:44:24.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.239+0000 7fdb6d64db80 10 client.19854 fetch_fsmap learned FSMap version 73 2024-02-02T15:44:24.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.239+0000 7fdb6d64db80 1 -- 172.21.15.1:0/1288666758 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({fsmap=73}) v3 -- 0x55eeec547b60 con 0x55eeec7c8010 2024-02-02T15:44:24.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.240+0000 7fdb40c00700 1 -- 172.21.15.1:0/1288666758 <== mon.0 v2:172.21.15.1:3300/0 4 ==== fsmap(e 73) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fdb4400f460 con 0x55eeec7c8010 2024-02-02T15:44:24.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.240+0000 7fdb6d64db80 10 client.19854 fetch_fsmap finished waiting for FSMap version 73 2024-02-02T15:44:24.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.240+0000 7fdb6d64db80 10 client.19854 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-02T15:44:24.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.240+0000 7fdb6d64db80 20 client.19854 populate_metadata read hostname 'smithi001' 2024-02-02T15:44:24.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.240+0000 7fdb6d64db80 1 --2- 172.21.15.1:0/1288666758 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55eeec7f39f0 0x55eeec7f5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:24.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.240+0000 7fdb6d64db80 4 client.19854 mds_command: new command op to 24529 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:44:24.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.240+0000 7fdb6d64db80 1 -- 172.21.15.1:0/1288666758 --> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eeec712dd0 con 0x55eeec7f39f0 2024-02-02T15:44:24.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.241+0000 7fdb52000700 1 --2- 172.21.15.1:0/1288666758 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55eeec7f39f0 0x55eeec7f5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:24.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.243+0000 7fdb40c00700 10 client.19854 ms_handle_connect on v2:172.21.15.142:6838/321675421 2024-02-02T15:44:24.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.243+0000 7fdb52000700 1 --2- 172.21.15.1:0/1288666758 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55eeec7f39f0 0x55eeec7f5de0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4873995998e47149 server_cookie=7644a647d0f0aa5f in_seq=0 out_seq=0 2024-02-02T15:44:24.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.243+0000 7fdb40c00700 1 -- 172.21.15.1:0/1288666758 <== mds.1 v2:172.21.15.142:6838/321675421 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55eeec712dd0 con 0x55eeec7f39f0 2024-02-02T15:44:24.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.244+0000 7fdb40c00700 10 client.19854 handle_command_reply: tid=0 2024-02-02T15:44:24.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.244+0000 7fdb6d64db80 1 client.19854 shutdown 2024-02-02T15:44:24.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.245+0000 7fdb6d64db80 1 -- 172.21.15.1:0/1288666758 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eeec7c8010 msgr2=0x55eeec7d3150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:24.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.245+0000 7fdb6d64db80 1 --2- 172.21.15.1:0/1288666758 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eeec7c8010 0x55eeec7d3150 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fdb44000c00 tx=0x7fdb4400bf80 comp rx=0 tx=0).stop 2024-02-02T15:44:24.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.245+0000 7fdb6d64db80 1 -- 172.21.15.1:0/1288666758 shutdown_connections 2024-02-02T15:44:24.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.245+0000 7fdb6d64db80 1 --2- 172.21.15.1:0/1288666758 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55eeec7f39f0 0x55eeec7f5de0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:24.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.245+0000 7fdb6d64db80 1 --2- 172.21.15.1:0/1288666758 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eeec7d37e0 0x55eeec7d0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:24.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.245+0000 7fdb40c00700 0 client.19854 ms_handle_reset on v2:172.21.15.142:6838/321675421 2024-02-02T15:44:24.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.245+0000 7fdb6d64db80 1 --2- 172.21.15.1:0/1288666758 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eeec7c8010 0x55eeec7d3150 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:24.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.246+0000 7fdb6d64db80 1 --2- 172.21.15.1:0/1288666758 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eeec7c8930 0x55eeec7d03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:24.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.246+0000 7fdb6d64db80 1 -- 172.21.15.1:0/1288666758 >> 172.21.15.1:0/1288666758 conn(0x55eeec727db0 msgr2=0x55eeec7c3f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:24.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.247+0000 7fdb6d64db80 1 -- 172.21.15.1:0/1288666758 shutdown_connections 2024-02-02T15:44:24.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.247+0000 7fdb6d64db80 1 -- 172.21.15.1:0/1288666758 wait complete. 2024-02-02T15:44:24.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.248+0000 7fdb6d64db80 20 client.19854 trim_cache size 0 max 16384 2024-02-02T15:44:24.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.282+0000 7fdb6d64db80 1 Processor -- start 2024-02-02T15:44:24.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.282+0000 7fdb6d64db80 1 -- start start 2024-02-02T15:44:24.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.282+0000 7fdb6d64db80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eeec7c8010 0x55eeec7ca840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:24.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.282+0000 7fdb6d64db80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eeec7c8930 0x55eeec7cad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:24.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.282+0000 7fdb6d64db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eeec7d37e0 0x55eeec7cfb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:24.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.282+0000 7fdb6d64db80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eeec59d6a0 con 0x55eeec7c8010 2024-02-02T15:44:24.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.282+0000 7fdb6d64db80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eeec7c51f0 con 0x55eeec7c8930 2024-02-02T15:44:24.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.282+0000 7fdb6d64db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eeec7ce930 con 0x55eeec7d37e0 2024-02-02T15:44:24.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.282+0000 7fdb52a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eeec7c8010 0x55eeec7ca840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:24.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.282+0000 7fdb52a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eeec7c8010 0x55eeec7ca840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45092/0 (socket says 172.21.15.1:45092) 2024-02-02T15:44:24.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.282+0000 7fdb52a00700 1 -- 172.21.15.1:0/745148866 learned_addr learned my addr 172.21.15.1:0/745148866 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:44:24.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.282+0000 7fdb53400700 1 --2- 172.21.15.1:0/745148866 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eeec7d37e0 0x55eeec7cfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:24.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.283+0000 7fdb52000700 1 --2- 172.21.15.1:0/745148866 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eeec7c8930 0x55eeec7cad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:24.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.283+0000 7fdb52a00700 1 -- 172.21.15.1:0/745148866 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eeec7d37e0 msgr2=0x55eeec7cfb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:24.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.283+0000 7fdb52a00700 1 --2- 172.21.15.1:0/745148866 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eeec7d37e0 0x55eeec7cfb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:24.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.283+0000 7fdb52a00700 1 -- 172.21.15.1:0/745148866 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eeec7c8930 msgr2=0x55eeec7cad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:24.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.283+0000 7fdb52a00700 1 --2- 172.21.15.1:0/745148866 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eeec7c8930 0x55eeec7cad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:24.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.283+0000 7fdb52a00700 1 -- 172.21.15.1:0/745148866 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eeec7c00d0 con 0x55eeec7c8010 2024-02-02T15:44:24.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.283+0000 7fdb52000700 1 --2- 172.21.15.1:0/745148866 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eeec7c8930 0x55eeec7cad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:44:24.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.283+0000 7fdb52a00700 1 --2- 172.21.15.1:0/745148866 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eeec7c8010 0x55eeec7ca840 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fdb440094b0 tx=0x7fdb44018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e22c4a748aeadd4f server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:24.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.283+0000 7fdb48c00700 1 -- 172.21.15.1:0/745148866 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb4400f8d0 con 0x55eeec7c8010 2024-02-02T15:44:24.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.284+0000 7fdb48c00700 1 -- 172.21.15.1:0/745148866 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55eeec7bc0f0 con 0x55eeec7c8010 2024-02-02T15:44:24.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.284+0000 7fdb48c00700 1 -- 172.21.15.1:0/745148866 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb44035510 con 0x55eeec7c8010 2024-02-02T15:44:24.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.284+0000 7fdb6d64db80 1 -- 172.21.15.1:0/745148866 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eeec7c8010 msgr2=0x55eeec7ca840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:24.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.284+0000 7fdb6d64db80 1 --2- 172.21.15.1:0/745148866 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eeec7c8010 0x55eeec7ca840 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fdb440094b0 tx=0x7fdb44018000 comp rx=0 tx=0).stop 2024-02-02T15:44:24.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.285+0000 7fdb6d64db80 1 -- 172.21.15.1:0/745148866 shutdown_connections 2024-02-02T15:44:24.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.285+0000 7fdb6d64db80 1 --2- 172.21.15.1:0/745148866 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eeec7d37e0 0x55eeec7cfb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:24.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.285+0000 7fdb6d64db80 1 --2- 172.21.15.1:0/745148866 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eeec7c8010 0x55eeec7ca840 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:24.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.285+0000 7fdb6d64db80 1 --2- 172.21.15.1:0/745148866 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eeec7c8930 0x55eeec7cad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:24.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.285+0000 7fdb6d64db80 1 -- 172.21.15.1:0/745148866 >> 172.21.15.1:0/745148866 conn(0x55eeec727db0 msgr2=0x55eeec7c2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:24.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.285+0000 7fdb6d64db80 1 -- 172.21.15.1:0/745148866 shutdown_connections 2024-02-02T15:44:24.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.285+0000 7fdb6d64db80 1 -- 172.21.15.1:0/745148866 wait complete. 2024-02-02T15:44:24.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.285+0000 7fdb6d64db80 1 -- start start 2024-02-02T15:44:24.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.286+0000 7fdb6d64db80 1 Processor -- start 2024-02-02T15:44:24.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.286+0000 7fdb6d64db80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eeec7c8010 0x55eeec7e9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:24.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.286+0000 7fdb6d64db80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eeec7c8930 0x55eeec7e9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:24.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.286+0000 7fdb6d64db80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eeec7d37e0 0x55eeec844430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:24.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.286+0000 7fdb6d64db80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eeec7ea060 con 0x55eeec7c8930 2024-02-02T15:44:24.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.286+0000 7fdb6d64db80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eeec844970 con 0x55eeec7c8010 2024-02-02T15:44:24.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.286+0000 7fdb6d64db80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eeec844ae0 con 0x55eeec7d37e0 2024-02-02T15:44:24.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.286+0000 7fdb6d64db80 10 client.0 _ll_register_callbacks cb 0x55eeec6b7290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:44:24.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.287+0000 7fdb53400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eeec7d37e0 0x55eeec844430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:24.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.287+0000 7fdb53400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eeec7d37e0 0x55eeec844430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:44798/0 (socket says 172.21.15.1:44798) 2024-02-02T15:44:24.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.287+0000 7fdb52000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eeec7c8930 0x55eeec7e9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:24.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.287+0000 7fdb53400700 1 -- 172.21.15.1:0/3148174516 learned_addr learned my addr 172.21.15.1:0/3148174516 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:44:24.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.288+0000 7fdb53400700 1 -- 172.21.15.1:0/3148174516 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eeec7c8010 msgr2=0x55eeec7e9230 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-02T15:44:24.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.288+0000 7fdb53400700 1 --2- 172.21.15.1:0/3148174516 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eeec7c8010 0x55eeec7e9230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:24.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.288+0000 7fdb53400700 1 -- 172.21.15.1:0/3148174516 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eeec7c8930 msgr2=0x55eeec7e9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:24.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.288+0000 7fdb53400700 1 --2- 172.21.15.1:0/3148174516 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eeec7c8930 0x55eeec7e9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:24.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.288+0000 7fdb53400700 1 -- 172.21.15.1:0/3148174516 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eeec547b60 con 0x55eeec7d37e0 2024-02-02T15:44:24.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.288+0000 7fdb52000700 1 --2- 172.21.15.1:0/3148174516 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eeec7c8930 0x55eeec7e9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:44:24.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.289+0000 7fdb53400700 1 --2- 172.21.15.1:0/3148174516 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eeec7d37e0 0x55eeec844430 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c0149e0 tx=0x7fdb4c004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18ef19e18f5f4ec server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:24.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.289+0000 7fdb40c00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:44:24.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.289+0000 7fdb40c00700 1 -- 172.21.15.1:0/3148174516 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c014e50 con 0x55eeec7d37e0 2024-02-02T15:44:24.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.289+0000 7fdb6d64db80 1 -- 172.21.15.1:0/3148174516 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdb4401e040 con 0x55eeec7d37e0 2024-02-02T15:44:24.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.289+0000 7fdb40c00700 1 -- 172.21.15.1:0/3148174516 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fdb4c022650 con 0x55eeec7d37e0 2024-02-02T15:44:24.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.290+0000 7fdb40c00700 1 -- 172.21.15.1:0/3148174516 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb4c022810 con 0x55eeec7d37e0 2024-02-02T15:44:24.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.290+0000 7fdb6d64db80 10 client.30001 fetch_fsmap learned FSMap version 73 2024-02-02T15:44:24.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.290+0000 7fdb6d64db80 1 -- 172.21.15.1:0/3148174516 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) v3 -- 0x55eeec6dfea0 con 0x55eeec7d37e0 2024-02-02T15:44:24.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.290+0000 7fdb40c00700 1 -- 172.21.15.1:0/3148174516 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb4401e040 con 0x55eeec7d37e0 2024-02-02T15:44:24.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.290+0000 7fdb40c00700 1 -- 172.21.15.1:0/3148174516 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 73) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fdb4c0159b0 con 0x55eeec7d37e0 2024-02-02T15:44:24.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.290+0000 7fdb6d64db80 10 client.30001 fetch_fsmap finished waiting for FSMap version 73 2024-02-02T15:44:24.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.290+0000 7fdb6d64db80 10 client.30001 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-02T15:44:24.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.290+0000 7fdb6d64db80 20 client.30001 populate_metadata read hostname 'smithi001' 2024-02-02T15:44:24.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.290+0000 7fdb6d64db80 1 --2- 172.21.15.1:0/3148174516 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55eeec7f39f0 0x55eeec8482b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:24.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.290+0000 7fdb6d64db80 4 client.30001 mds_command: new command op to 24529 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:44:24.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.290+0000 7fdb6d64db80 1 -- 172.21.15.1:0/3148174516 --> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55eeec6d8f50 con 0x55eeec7f39f0 2024-02-02T15:44:24.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.291+0000 7fdb52a00700 1 --2- 172.21.15.1:0/3148174516 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55eeec7f39f0 0x55eeec8482b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:24.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.292+0000 7fdb40c00700 10 client.30001 ms_handle_connect on v2:172.21.15.142:6838/321675421 2024-02-02T15:44:24.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.292+0000 7fdb52a00700 1 --2- 172.21.15.1:0/3148174516 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55eeec7f39f0 0x55eeec8482b0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a59468b64e7e1da4 server_cookie=8889b5b4efcc7ceb in_seq=0 out_seq=0 2024-02-02T15:44:24.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.292+0000 7fdb40c00700 1 -- 172.21.15.1:0/3148174516 <== mds.1 v2:172.21.15.142:6838/321675421 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55eeec6d8f50 con 0x55eeec7f39f0 2024-02-02T15:44:24.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.292+0000 7fdb40c00700 10 client.30001 handle_command_reply: tid=0 2024-02-02T15:44:24.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.292+0000 7fdb6d64db80 1 client.30001 shutdown 2024-02-02T15:44:24.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.293+0000 7fdb6d64db80 1 -- 172.21.15.1:0/3148174516 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eeec7d37e0 msgr2=0x55eeec844430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:24.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.293+0000 7fdb6d64db80 1 --2- 172.21.15.1:0/3148174516 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eeec7d37e0 0x55eeec844430 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c0149e0 tx=0x7fdb4c004ad0 comp rx=0 tx=0).stop 2024-02-02T15:44:24.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.293+0000 7fdb6d64db80 1 -- 172.21.15.1:0/3148174516 shutdown_connections 2024-02-02T15:44:24.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.293+0000 7fdb6d64db80 1 --2- 172.21.15.1:0/3148174516 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x55eeec7f39f0 0x55eeec8482b0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:24.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.293+0000 7fdb6d64db80 1 --2- 172.21.15.1:0/3148174516 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55eeec7d37e0 0x55eeec844430 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:24.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.293+0000 7fdb6d64db80 1 --2- 172.21.15.1:0/3148174516 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55eeec7c8010 0x55eeec7e9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:24.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.293+0000 7fdb6d64db80 1 --2- 172.21.15.1:0/3148174516 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55eeec7c8930 0x55eeec7e9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:24.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.293+0000 7fdb40c00700 0 client.30001 ms_handle_reset on v2:172.21.15.142:6838/321675421 2024-02-02T15:44:24.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.293+0000 7fdb6d64db80 1 -- 172.21.15.1:0/3148174516 >> 172.21.15.1:0/3148174516 conn(0x55eeec727db0 msgr2=0x55eeec7c35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:24.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.295+0000 7fdb6d64db80 1 -- 172.21.15.1:0/3148174516 shutdown_connections 2024-02-02T15:44:24.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.295+0000 7fdb6d64db80 1 -- 172.21.15.1:0/3148174516 wait complete. 2024-02-02T15:44:24.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.296+0000 7fdb6d64db80 20 client.30001 trim_cache size 0 max 16384 2024-02-02T15:44:24.296 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:44:24.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.301+0000 7fdb59c00700 1 -- 172.21.15.1:0/3437319816 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fdb340a0670 msgr2=0x7fdb340a2b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:24.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.301+0000 7fdb59c00700 1 --2- 172.21.15.1:0/3437319816 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fdb340a0670 0x7fdb340a2b30 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c00f370 tx=0x7fdb4c007950 comp rx=0 tx=0).stop 2024-02-02T15:44:24.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.301+0000 7fdb59c00700 1 -- 172.21.15.1:0/3437319816 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdb540689d0 msgr2=0x7fdb5419d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:24.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.301+0000 7fdb59c00700 1 --2- 172.21.15.1:0/3437319816 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdb540689d0 0x7fdb5419d8a0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c00d750 tx=0x7fdb3c00da60 comp rx=0 tx=0).stop 2024-02-02T15:44:24.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.302+0000 7fdb59c00700 1 -- 172.21.15.1:0/3437319816 shutdown_connections 2024-02-02T15:44:24.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.302+0000 7fdb59c00700 1 --2- 172.21.15.1:0/3437319816 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fdb340a0670 0x7fdb340a2b30 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:24.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.302+0000 7fdb59c00700 1 --2- 172.21.15.1:0/3437319816 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdb54106d00 0x7fdb541a1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:24.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.302+0000 7fdb59c00700 1 --2- 172.21.15.1:0/3437319816 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdb54068010 0x7fdb5419d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:24.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.302+0000 7fdb59c00700 1 --2- 172.21.15.1:0/3437319816 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdb540689d0 0x7fdb5419d8a0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:24.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.302+0000 7fdb59c00700 1 -- 172.21.15.1:0/3437319816 >> 172.21.15.1:0/3437319816 conn(0x7fdb540756b0 msgr2=0x7fdb540fe800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:24.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.302+0000 7fdb59c00700 1 -- 172.21.15.1:0/3437319816 shutdown_connections 2024-02-02T15:44:24.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.302+0000 7fdb59c00700 1 -- 172.21.15.1:0/3437319816 wait complete. 2024-02-02T15:44:24.328 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-02T15:44:24.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.609+0000 7f752a600700 1 -- 172.21.15.1:0/912809851 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7524113e90 msgr2=0x7f7524116280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:24.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.609+0000 7f752a600700 1 --2- 172.21.15.1:0/912809851 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7524113e90 0x7f7524116280 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f751c00b0a0 tx=0x7f751c00b3b0 comp rx=0 tx=0).stop 2024-02-02T15:44:24.609 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.609+0000 7f752a600700 1 -- 172.21.15.1:0/912809851 shutdown_connections 2024-02-02T15:44:24.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.609+0000 7f752a600700 1 --2- 172.21.15.1:0/912809851 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7524113e90 0x7f7524116280 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:24.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.609+0000 7f752a600700 1 --2- 172.21.15.1:0/912809851 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7524107d40 0x7f7524108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:24.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.609+0000 7f752a600700 1 --2- 172.21.15.1:0/912809851 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7524108660 0x7f7524113950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:24.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.609+0000 7f752a600700 1 -- 172.21.15.1:0/912809851 >> 172.21.15.1:0/912809851 conn(0x7f752406d120 msgr2=0x7f752406d530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:24.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.609+0000 7f752a600700 1 -- 172.21.15.1:0/912809851 shutdown_connections 2024-02-02T15:44:24.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.609+0000 7f752a600700 1 -- 172.21.15.1:0/912809851 wait complete. 2024-02-02T15:44:24.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.610+0000 7f752a600700 1 Processor -- start 2024-02-02T15:44:24.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.611+0000 7f752a600700 1 -- start start 2024-02-02T15:44:24.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.611+0000 7f752a600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7524107d40 0x7f75241097b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:24.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.612+0000 7f752a600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7524108660 0x7f7524109cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:24.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.612+0000 7f752a600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7524113e90 0x7f752410a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:24.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.612+0000 7f752a600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75241b2180 con 0x7f7524107d40 2024-02-02T15:44:24.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.612+0000 7f752a600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75241b22c0 con 0x7f7524108660 2024-02-02T15:44:24.612 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.612+0000 7f752a600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75241b2400 con 0x7f7524113e90 2024-02-02T15:44:24.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.612+0000 7f7522a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7524108660 0x7f7524109cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:24.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.613+0000 7f7522a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7524108660 0x7f7524109cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:51632/0 (socket says 172.21.15.1:51632) 2024-02-02T15:44:24.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.613+0000 7f7522a00700 1 -- 172.21.15.1:0/2124249513 learned_addr learned my addr 172.21.15.1:0/2124249513 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:44:24.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.613+0000 7f7522a00700 1 -- 172.21.15.1:0/2124249513 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7524113e90 msgr2=0x7f752410a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:24.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.613+0000 7f7522a00700 1 --2- 172.21.15.1:0/2124249513 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7524113e90 0x7f752410a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:24.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.613+0000 7f7522a00700 1 -- 172.21.15.1:0/2124249513 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7524107d40 msgr2=0x7f75241097b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:24.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.613+0000 7f7522a00700 1 --2- 172.21.15.1:0/2124249513 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7524107d40 0x7f75241097b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:24.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.613+0000 7f7522a00700 1 -- 172.21.15.1:0/2124249513 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f751c009d00 con 0x7f7524108660 2024-02-02T15:44:24.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.613+0000 7f7522a00700 1 --2- 172.21.15.1:0/2124249513 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7524108660 0x7f7524109cf0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f750c00e9a0 tx=0x7f750c00ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:24.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.614+0000 7f751be00700 1 -- 172.21.15.1:0/2124249513 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f750c009790 con 0x7f7524108660 2024-02-02T15:44:24.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.614+0000 7f751be00700 1 -- 172.21.15.1:0/2124249513 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f750c00cbd0 con 0x7f7524108660 2024-02-02T15:44:24.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.614+0000 7f751be00700 1 -- 172.21.15.1:0/2124249513 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f750c018680 con 0x7f7524108660 2024-02-02T15:44:24.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.614+0000 7f752a600700 1 -- 172.21.15.1:0/2124249513 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75241b26e0 con 0x7f7524108660 2024-02-02T15:44:24.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.614+0000 7f752a600700 1 -- 172.21.15.1:0/2124249513 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75241b2ba0 con 0x7f7524108660 2024-02-02T15:44:24.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.619+0000 7f751be00700 1 -- 172.21.15.1:0/2124249513 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f750c010430 con 0x7f7524108660 2024-02-02T15:44:24.619 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.620+0000 7f753e1eeb80 1 Processor -- start 2024-02-02T15:44:24.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.620+0000 7f753e1eeb80 1 -- start start 2024-02-02T15:44:24.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.620+0000 7f753e1eeb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55708b026010 0x55708b0263f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:24.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.621+0000 7f753e1eeb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55708b026930 0x55708b0312a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:24.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.621+0000 7f753e1eeb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55708b0317e0 0x55708b033bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:24.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.621+0000 7f753e1eeb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55708af9a4c0 con 0x55708b026010 2024-02-02T15:44:24.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.621+0000 7f753e1eeb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55708af7eaa0 con 0x55708b026930 2024-02-02T15:44:24.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.621+0000 7f753e1eeb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55708af7c460 con 0x55708b0317e0 2024-02-02T15:44:24.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.621+0000 7f7522a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55708b0317e0 0x55708b033bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:24.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.621+0000 7f7522a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55708b0317e0 0x55708b033bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:44810/0 (socket says 172.21.15.1:44810) 2024-02-02T15:44:24.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.621+0000 7f7522a00700 1 -- 172.21.15.1:0/3662222221 learned_addr learned my addr 172.21.15.1:0/3662222221 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:44:24.622 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.621+0000 7f7523e00700 1 --2- 172.21.15.1:0/3662222221 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55708b026010 0x55708b0263f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:24.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.622+0000 7f7523400700 1 --2- 172.21.15.1:0/3662222221 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55708b026930 0x55708b0312a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:24.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.622+0000 7f751be00700 1 --2- 172.21.15.1:0/2124249513 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f75040a0750 0x7f75040a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:24.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.622+0000 7f7522a00700 1 -- 172.21.15.1:0/3662222221 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55708b026930 msgr2=0x55708b0312a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:24.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.622+0000 7f7522a00700 1 --2- 172.21.15.1:0/3662222221 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55708b026930 0x55708b0312a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:24.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.622+0000 7f7522a00700 1 -- 172.21.15.1:0/3662222221 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55708b026010 msgr2=0x55708b0263f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:24.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.622+0000 7f7522a00700 1 --2- 172.21.15.1:0/3662222221 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55708b026010 0x55708b0263f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:24.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.622+0000 7f7522a00700 1 -- 172.21.15.1:0/3662222221 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55708adf93b0 con 0x55708b0317e0 2024-02-02T15:44:24.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.622+0000 7f7523e00700 1 --2- 172.21.15.1:0/3662222221 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55708b026010 0x55708b0263f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:44:24.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.622+0000 7f751be00700 1 -- 172.21.15.1:0/2124249513 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f750c014070 con 0x7f7524108660 2024-02-02T15:44:24.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.622+0000 7f7522a00700 1 --2- 172.21.15.1:0/3662222221 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55708b0317e0 0x55708b033bd0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f750c06cb90 tx=0x7f750c06b670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8395da0c37e77dfc server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:24.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.623+0000 7f7523e00700 1 --2- 172.21.15.1:0/2124249513 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f75040a0750 0x7f75040a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:24.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.623+0000 7f7518c00700 1 -- 172.21.15.1:0/3662222221 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f750c099360 con 0x55708b0317e0 2024-02-02T15:44:24.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.624+0000 7f7523e00700 1 --2- 172.21.15.1:0/2124249513 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f75040a0750 0x7f75040a2c10 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f751c00b780 tx=0x7f751c009bf0 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:24.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.624+0000 7f7518c00700 1 -- 172.21.15.1:0/3662222221 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f750c0990d0 con 0x55708b0317e0 2024-02-02T15:44:24.624 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.624+0000 7f7518c00700 1 -- 172.21.15.1:0/3662222221 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f750c06c790 con 0x55708b0317e0 2024-02-02T15:44:24.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.625+0000 7f753e1eeb80 1 -- 172.21.15.1:0/3662222221 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55708b0317e0 msgr2=0x55708b033bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:24.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.625+0000 7f753e1eeb80 1 --2- 172.21.15.1:0/3662222221 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55708b0317e0 0x55708b033bd0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f750c06cb90 tx=0x7f750c06b670 comp rx=0 tx=0).stop 2024-02-02T15:44:24.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.626+0000 7f753e1eeb80 1 -- 172.21.15.1:0/3662222221 shutdown_connections 2024-02-02T15:44:24.625 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.626+0000 7f753e1eeb80 1 --2- 172.21.15.1:0/3662222221 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55708b0317e0 0x55708b033bd0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:24.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.626+0000 7f753e1eeb80 1 --2- 172.21.15.1:0/3662222221 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55708b026010 0x55708b0263f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:24.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.626+0000 7f753e1eeb80 1 --2- 172.21.15.1:0/3662222221 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55708b026930 0x55708b0312a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:24.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.626+0000 7f753e1eeb80 1 -- 172.21.15.1:0/3662222221 >> 172.21.15.1:0/3662222221 conn(0x55708af85db0 msgr2=0x55708b0223b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:24.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.626+0000 7f753e1eeb80 1 -- 172.21.15.1:0/3662222221 shutdown_connections 2024-02-02T15:44:24.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.626+0000 7f753e1eeb80 1 -- 172.21.15.1:0/3662222221 wait complete. 2024-02-02T15:44:24.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.627+0000 7f753e1eeb80 1 -- start start 2024-02-02T15:44:24.627 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.627+0000 7f753e1eeb80 1 Processor -- start 2024-02-02T15:44:24.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.628+0000 7f753e1eeb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55708b026010 0x55708b031230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:24.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.628+0000 7f753e1eeb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55708b026930 0x55708b02e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:24.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.628+0000 7f753e1eeb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55708b0317e0 0x55708b02eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:24.628 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.628+0000 7f753e1eeb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55708af77d50 con 0x55708b026010 2024-02-02T15:44:24.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.628+0000 7f753e1eeb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55708af13d80 con 0x55708b0317e0 2024-02-02T15:44:24.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.628+0000 7f753e1eeb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55708af13a20 con 0x55708b026930 2024-02-02T15:44:24.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.628+0000 7f753e1eeb80 10 client.0 _ll_register_callbacks cb 0x55708af15290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:44:24.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.629+0000 7f7522a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55708b026930 0x55708b02e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:24.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.629+0000 7f7523400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55708b026010 0x55708b031230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:24.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.629+0000 7f7522a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55708b026930 0x55708b02e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:44818/0 (socket says 172.21.15.1:44818) 2024-02-02T15:44:24.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.629+0000 7f7522a00700 1 -- 172.21.15.1:0/4111218606 learned_addr learned my addr 172.21.15.1:0/4111218606 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:44:24.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.629+0000 7f7523e00700 1 --2- 172.21.15.1:0/4111218606 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55708b0317e0 0x55708b02eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:24.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.630+0000 7f7522a00700 1 -- 172.21.15.1:0/4111218606 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55708b0317e0 msgr2=0x55708b02eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:24.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.630+0000 7f7522a00700 1 --2- 172.21.15.1:0/4111218606 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55708b0317e0 0x55708b02eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:24.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.630+0000 7f7522a00700 1 -- 172.21.15.1:0/4111218606 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55708b026010 msgr2=0x55708b031230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:24.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.630+0000 7f7522a00700 1 --2- 172.21.15.1:0/4111218606 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55708b026010 0x55708b031230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:24.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.630+0000 7f7522a00700 1 -- 172.21.15.1:0/4111218606 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75241b2ba0 con 0x55708b026930 2024-02-02T15:44:24.630 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.630+0000 7f7523e00700 1 --2- 172.21.15.1:0/4111218606 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55708b0317e0 0x55708b02eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:44:24.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.630+0000 7f7522a00700 1 --2- 172.21.15.1:0/4111218606 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55708b026930 0x55708b02e440 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f750c0a10d0 tx=0x7f750c0092d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53f062bbc6041b0b server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:24.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.630+0000 7f7511600700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:44:24.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.630+0000 7f7511600700 1 -- 172.21.15.1:0/4111218606 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f750c098ef0 con 0x55708b026930 2024-02-02T15:44:24.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.631+0000 7f7511600700 1 -- 172.21.15.1:0/4111218606 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f750c0997f0 con 0x55708b026930 2024-02-02T15:44:24.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.631+0000 7f753e1eeb80 1 -- 172.21.15.1:0/4111218606 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55708b01a0f0 con 0x55708b026930 2024-02-02T15:44:24.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.632+0000 7f7511600700 1 -- 172.21.15.1:0/4111218606 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f750c099c90 con 0x55708b026930 2024-02-02T15:44:24.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.632+0000 7f753e1eeb80 10 client.30010 fetch_fsmap learned FSMap version 74 2024-02-02T15:44:24.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.632+0000 7f753e1eeb80 1 -- 172.21.15.1:0/4111218606 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x55708ada5b60 con 0x55708b026930 2024-02-02T15:44:24.632 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.632+0000 7f7511600700 1 -- 172.21.15.1:0/4111218606 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55708b01a0f0 con 0x55708b026930 2024-02-02T15:44:24.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.632+0000 7f7511600700 1 -- 172.21.15.1:0/4111218606 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 74) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f750c084110 con 0x55708b026930 2024-02-02T15:44:24.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.632+0000 7f753e1eeb80 10 client.30010 fetch_fsmap finished waiting for FSMap version 74 2024-02-02T15:44:24.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.632+0000 7f753e1eeb80 10 client.30010 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:44:24.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.633+0000 7f753e1eeb80 20 client.30010 populate_metadata read hostname 'smithi001' 2024-02-02T15:44:24.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.633+0000 7f753e1eeb80 1 --2- 172.21.15.1:0/4111218606 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55708b040d70 0x55708b043160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:24.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.633+0000 7f753e1eeb80 4 client.30010 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:44:24.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.633+0000 7f753e1eeb80 1 -- 172.21.15.1:0/4111218606 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55708af70dd0 con 0x55708b040d70 2024-02-02T15:44:24.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.633+0000 7f7523400700 1 --2- 172.21.15.1:0/4111218606 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55708b040d70 0x55708b043160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:24.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.634+0000 7f7523400700 1 --2- 172.21.15.1:0/4111218606 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55708b040d70 0x55708b043160 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd2b0e4bb6c5d54 server_cookie=5e7b478d65417eea in_seq=0 out_seq=0 2024-02-02T15:44:24.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.635+0000 7f7511600700 10 client.30010 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:44:24.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.637+0000 7f7511600700 1 -- 172.21.15.1:0/4111218606 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55708af70dd0 con 0x55708b040d70 2024-02-02T15:44:24.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.637+0000 7f7511600700 10 client.30010 handle_command_reply: tid=0 2024-02-02T15:44:24.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.637+0000 7f753e1eeb80 1 client.30010 shutdown 2024-02-02T15:44:24.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.638+0000 7f753e1eeb80 1 -- 172.21.15.1:0/4111218606 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55708b026930 msgr2=0x55708b02e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:24.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.638+0000 7f753e1eeb80 1 --2- 172.21.15.1:0/4111218606 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55708b026930 0x55708b02e440 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f750c0a10d0 tx=0x7f750c0092d0 comp rx=0 tx=0).stop 2024-02-02T15:44:24.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.639+0000 7f753e1eeb80 1 -- 172.21.15.1:0/4111218606 shutdown_connections 2024-02-02T15:44:24.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.639+0000 7f753e1eeb80 1 --2- 172.21.15.1:0/4111218606 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55708b040d70 0x55708b043160 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:24.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.639+0000 7f753e1eeb80 1 --2- 172.21.15.1:0/4111218606 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55708b0317e0 0x55708b02eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:24.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.639+0000 7f753e1eeb80 1 --2- 172.21.15.1:0/4111218606 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55708b026010 0x55708b031230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:24.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.639+0000 7f753e1eeb80 1 --2- 172.21.15.1:0/4111218606 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55708b026930 0x55708b02e440 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:24.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.639+0000 7f7511600700 0 client.30010 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:44:24.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.639+0000 7f753e1eeb80 1 -- 172.21.15.1:0/4111218606 >> 172.21.15.1:0/4111218606 conn(0x55708af85db0 msgr2=0x55708b021fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:24.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.640+0000 7f753e1eeb80 1 -- 172.21.15.1:0/4111218606 shutdown_connections 2024-02-02T15:44:24.640 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.640+0000 7f753e1eeb80 1 -- 172.21.15.1:0/4111218606 wait complete. 2024-02-02T15:44:24.641 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.641+0000 7f753e1eeb80 20 client.30010 trim_cache size 0 max 16384 2024-02-02T15:44:24.651 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:44:24 smithi001 ceph-mon[35470]: pgmap v547: 129 pgs: 129 active+clean; 1.8 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 8.2 MiB/s wr, 479 op/s 2024-02-02T15:44:24.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.671+0000 7f753e1eeb80 1 Processor -- start 2024-02-02T15:44:24.671 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.671+0000 7f753e1eeb80 1 -- start start 2024-02-02T15:44:24.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.672+0000 7f753e1eeb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55708b026010 0x55708b047ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:24.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.672+0000 7f753e1eeb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55708b026930 0x55708b038270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:24.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.672+0000 7f753e1eeb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55708b0317e0 0x55708b02dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:24.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.672+0000 7f753e1eeb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55708adfb6a0 con 0x55708b0317e0 2024-02-02T15:44:24.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.672+0000 7f753e1eeb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55708b02e220 con 0x55708b026930 2024-02-02T15:44:24.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.672+0000 7f753e1eeb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55708b02d9b0 con 0x55708b026010 2024-02-02T15:44:24.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.673+0000 7f7523400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55708b026010 0x55708b047ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:24.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.673+0000 7f7522a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55708b026930 0x55708b038270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:24.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.673+0000 7f7522a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55708b026930 0x55708b038270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:51660/0 (socket says 172.21.15.1:51660) 2024-02-02T15:44:24.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.673+0000 7f7522a00700 1 -- 172.21.15.1:0/812659524 learned_addr learned my addr 172.21.15.1:0/812659524 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:44:24.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.674+0000 7f7523e00700 1 --2- 172.21.15.1:0/812659524 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55708b0317e0 0x55708b02dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:24.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.675+0000 7f7522a00700 1 -- 172.21.15.1:0/812659524 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55708b026010 msgr2=0x55708b047ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:24.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.675+0000 7f7522a00700 1 --2- 172.21.15.1:0/812659524 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55708b026010 0x55708b047ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:24.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.675+0000 7f7522a00700 1 -- 172.21.15.1:0/812659524 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55708b0317e0 msgr2=0x55708b02dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:24.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.675+0000 7f7522a00700 1 --2- 172.21.15.1:0/812659524 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55708b0317e0 0x55708b02dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:24.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.675+0000 7f7522a00700 1 -- 172.21.15.1:0/812659524 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55708af3dea0 con 0x55708b026930 2024-02-02T15:44:24.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.675+0000 7f7523e00700 1 --2- 172.21.15.1:0/812659524 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55708b0317e0 0x55708b02dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:44:24.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.676+0000 7f7522a00700 1 --2- 172.21.15.1:0/812659524 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55708b026930 0x55708b038270 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f750c00c5b0 tx=0x7f750c00efa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32b804c553a328ea server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:24.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.676+0000 7f7519600700 1 -- 172.21.15.1:0/812659524 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f750c098ef0 con 0x55708b026930 2024-02-02T15:44:24.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.676+0000 7f7519600700 1 -- 172.21.15.1:0/812659524 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f750c0997f0 con 0x55708b026930 2024-02-02T15:44:24.676 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.676+0000 7f7519600700 1 -- 172.21.15.1:0/812659524 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f750c095400 con 0x55708b026930 2024-02-02T15:44:24.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.678+0000 7f753e1eeb80 1 -- 172.21.15.1:0/812659524 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55708b026930 msgr2=0x55708b038270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:24.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.678+0000 7f753e1eeb80 1 --2- 172.21.15.1:0/812659524 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55708b026930 0x55708b038270 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f750c00c5b0 tx=0x7f750c00efa0 comp rx=0 tx=0).stop 2024-02-02T15:44:24.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.679+0000 7f753e1eeb80 1 -- 172.21.15.1:0/812659524 shutdown_connections 2024-02-02T15:44:24.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.679+0000 7f753e1eeb80 1 --2- 172.21.15.1:0/812659524 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55708b0317e0 0x55708b02dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:24.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.679+0000 7f753e1eeb80 1 --2- 172.21.15.1:0/812659524 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55708b026010 0x55708b047ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:24.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.679+0000 7f753e1eeb80 1 --2- 172.21.15.1:0/812659524 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55708b026930 0x55708b038270 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:24.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.679+0000 7f753e1eeb80 1 -- 172.21.15.1:0/812659524 >> 172.21.15.1:0/812659524 conn(0x55708af85db0 msgr2=0x55708b03d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:24.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.679+0000 7f753e1eeb80 1 -- 172.21.15.1:0/812659524 shutdown_connections 2024-02-02T15:44:24.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.679+0000 7f753e1eeb80 1 -- 172.21.15.1:0/812659524 wait complete. 2024-02-02T15:44:24.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.680+0000 7f753e1eeb80 1 -- start start 2024-02-02T15:44:24.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.680+0000 7f753e1eeb80 1 Processor -- start 2024-02-02T15:44:24.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.681+0000 7f753e1eeb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55708b026010 0x55708b036250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:24.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.681+0000 7f753e1eeb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55708b026930 0x55708b036790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:24.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.681+0000 7f753e1eeb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55708b0317e0 0x55708b036cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:24.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.681+0000 7f753e1eeb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55708b091940 con 0x55708b0317e0 2024-02-02T15:44:24.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.681+0000 7f753e1eeb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55708b091ab0 con 0x55708b026930 2024-02-02T15:44:24.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.681+0000 7f753e1eeb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55708b091c70 con 0x55708b026010 2024-02-02T15:44:24.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.681+0000 7f753e1eeb80 10 client.0 _ll_register_callbacks cb 0x55708af15290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:44:24.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.682+0000 7f7523400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55708b026010 0x55708b036250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:24.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.682+0000 7f7523e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55708b0317e0 0x55708b036cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:24.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.682+0000 7f7522a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55708b026930 0x55708b036790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:24.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.682+0000 7f7522a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55708b026930 0x55708b036790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:51674/0 (socket says 172.21.15.1:51674) 2024-02-02T15:44:24.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.682+0000 7f7522a00700 1 -- 172.21.15.1:0/4103474288 learned_addr learned my addr 172.21.15.1:0/4103474288 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:44:24.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.683+0000 7f7522a00700 1 -- 172.21.15.1:0/4103474288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55708b026010 msgr2=0x55708b036250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:24.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.683+0000 7f7522a00700 1 --2- 172.21.15.1:0/4103474288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55708b026010 0x55708b036250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:24.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.683+0000 7f7522a00700 1 -- 172.21.15.1:0/4103474288 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55708b0317e0 msgr2=0x55708b036cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:24.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.683+0000 7f7522a00700 1 --2- 172.21.15.1:0/4103474288 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55708b0317e0 0x55708b036cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:24.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.683+0000 7f7522a00700 1 -- 172.21.15.1:0/4103474288 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55708ada5b60 con 0x55708b026930 2024-02-02T15:44:24.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.683+0000 7f7523e00700 1 --2- 172.21.15.1:0/4103474288 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55708b0317e0 0x55708b036cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:44:24.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.684+0000 7f7522a00700 1 --2- 172.21.15.1:0/4103474288 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55708b026930 0x55708b036790 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f750c099240 tx=0x7f750c098fc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3196bc6ef7fffb40 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:24.684 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.684+0000 7f7511600700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:44:24.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.687+0000 7f7511600700 1 -- 172.21.15.1:0/4103474288 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f750c09d3c0 con 0x55708b026930 2024-02-02T15:44:24.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.688+0000 7f7511600700 1 -- 172.21.15.1:0/4103474288 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f750c0a1600 con 0x55708b026930 2024-02-02T15:44:24.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.688+0000 7f753e1eeb80 1 -- 172.21.15.1:0/4103474288 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55708b027070 con 0x55708b026930 2024-02-02T15:44:24.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.688+0000 7f7511600700 1 -- 172.21.15.1:0/4103474288 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f750c095400 con 0x55708b026930 2024-02-02T15:44:24.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.688+0000 7f753e1eeb80 10 client.30149 fetch_fsmap learned FSMap version 74 2024-02-02T15:44:24.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.689+0000 7f753e1eeb80 1 -- 172.21.15.1:0/4103474288 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x55708adfdcd0 con 0x55708b026930 2024-02-02T15:44:24.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.690+0000 7f7511600700 1 -- 172.21.15.1:0/4103474288 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55708b027070 con 0x55708b026930 2024-02-02T15:44:24.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.690+0000 7f7511600700 1 -- 172.21.15.1:0/4103474288 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 74) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f750c0a1170 con 0x55708b026930 2024-02-02T15:44:24.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.691+0000 7f753e1eeb80 10 client.30149 fetch_fsmap finished waiting for FSMap version 74 2024-02-02T15:44:24.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.691+0000 7f753e1eeb80 10 client.30149 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:44:24.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.691+0000 7f753e1eeb80 20 client.30149 populate_metadata read hostname 'smithi001' 2024-02-02T15:44:24.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.691+0000 7f753e1eeb80 1 --2- 172.21.15.1:0/4103474288 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55708b092ec0 0x55708b0952b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:24.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.691+0000 7f753e1eeb80 4 client.30149 mds_command: new command op to 24557 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:44:24.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.691+0000 7f753e1eeb80 1 -- 172.21.15.1:0/4103474288 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f750c0990d0 con 0x55708b092ec0 2024-02-02T15:44:24.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.691+0000 7f7523400700 1 --2- 172.21.15.1:0/4103474288 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55708b092ec0 0x55708b0952b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:24.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.692+0000 7f7511600700 10 client.30149 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:44:24.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.692+0000 7f7523400700 1 --2- 172.21.15.1:0/4103474288 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55708b092ec0 0x55708b0952b0 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7113c9f55012bdb3 server_cookie=5d8f9c06885bd3f2 in_seq=0 out_seq=0 2024-02-02T15:44:24.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.693+0000 7f7511600700 1 -- 172.21.15.1:0/4103474288 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f750c0990d0 con 0x55708b092ec0 2024-02-02T15:44:24.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.693+0000 7f7511600700 10 client.30149 handle_command_reply: tid=0 2024-02-02T15:44:24.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.693+0000 7f753e1eeb80 1 client.30149 shutdown 2024-02-02T15:44:24.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.693+0000 7f753e1eeb80 1 -- 172.21.15.1:0/4103474288 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55708b026930 msgr2=0x55708b036790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:24.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.693+0000 7f753e1eeb80 1 --2- 172.21.15.1:0/4103474288 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55708b026930 0x55708b036790 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f750c099240 tx=0x7f750c098fc0 comp rx=0 tx=0).stop 2024-02-02T15:44:24.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.693+0000 7f753e1eeb80 1 -- 172.21.15.1:0/4103474288 shutdown_connections 2024-02-02T15:44:24.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.694+0000 7f753e1eeb80 1 --2- 172.21.15.1:0/4103474288 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55708b092ec0 0x55708b0952b0 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:24.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.694+0000 7f753e1eeb80 1 --2- 172.21.15.1:0/4103474288 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55708b0317e0 0x55708b036cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:24.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.694+0000 7f753e1eeb80 1 --2- 172.21.15.1:0/4103474288 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55708b026010 0x55708b036250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:24.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.694+0000 7f753e1eeb80 1 --2- 172.21.15.1:0/4103474288 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55708b026930 0x55708b036790 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:24.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.694+0000 7f753e1eeb80 1 -- 172.21.15.1:0/4103474288 >> 172.21.15.1:0/4103474288 conn(0x55708af85db0 msgr2=0x55708b02dc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:24.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.694+0000 7f7511600700 0 client.30149 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:44:24.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.694+0000 7f753e1eeb80 1 -- 172.21.15.1:0/4103474288 shutdown_connections 2024-02-02T15:44:24.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.694+0000 7f753e1eeb80 1 -- 172.21.15.1:0/4103474288 wait complete. 2024-02-02T15:44:24.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.695+0000 7f753e1eeb80 20 client.30149 trim_cache size 0 max 16384 2024-02-02T15:44:24.695 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:44:24.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.696+0000 7f752a600700 1 -- 172.21.15.1:0/2124249513 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f75040a0750 msgr2=0x7f75040a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:24.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.696+0000 7f752a600700 1 --2- 172.21.15.1:0/2124249513 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f75040a0750 0x7f75040a2c10 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f751c00b780 tx=0x7f751c009bf0 comp rx=0 tx=0).stop 2024-02-02T15:44:24.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.696+0000 7f752a600700 1 -- 172.21.15.1:0/2124249513 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7524108660 msgr2=0x7f7524109cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:24.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.696+0000 7f752a600700 1 --2- 172.21.15.1:0/2124249513 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7524108660 0x7f7524109cf0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f750c00e9a0 tx=0x7f750c00ed60 comp rx=0 tx=0).stop 2024-02-02T15:44:24.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.696+0000 7f752a600700 1 -- 172.21.15.1:0/2124249513 shutdown_connections 2024-02-02T15:44:24.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.696+0000 7f752a600700 1 --2- 172.21.15.1:0/2124249513 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f75040a0750 0x7f75040a2c10 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:24.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.696+0000 7f752a600700 1 --2- 172.21.15.1:0/2124249513 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7524113e90 0x7f752410a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:24.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.696+0000 7f752a600700 1 --2- 172.21.15.1:0/2124249513 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7524107d40 0x7f75241097b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:24.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.696+0000 7f752a600700 1 --2- 172.21.15.1:0/2124249513 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7524108660 0x7f7524109cf0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:24.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.696+0000 7f752a600700 1 -- 172.21.15.1:0/2124249513 >> 172.21.15.1:0/2124249513 conn(0x7f752406d120 msgr2=0x7f752406f690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:24.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.699+0000 7f752a600700 1 -- 172.21.15.1:0/2124249513 shutdown_connections 2024-02-02T15:44:24.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:24.699+0000 7f752a600700 1 -- 172.21.15.1:0/2124249513 wait complete. 2024-02-02T15:44:24.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:44:24 smithi142 ceph-mon[45728]: pgmap v547: 129 pgs: 129 active+clean; 1.8 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 8.2 MiB/s wr, 479 op/s 2024-02-02T15:44:24.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:44:24 smithi142 ceph-mon[45728]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:44:24.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:44:24 smithi001 ceph-mon[35470]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:44:24.993 INFO:tasks.workunit.client.0.smithi001.stdout: 10 132016 28.72 MB/sec execute 117 sec latency 10765.860 ms 2024-02-02T15:44:25.116 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:44:24 smithi071 ceph-mon[44707]: pgmap v547: 129 pgs: 129 active+clean; 1.8 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 8.2 MiB/s wr, 479 op/s 2024-02-02T15:44:25.116 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:44:24 smithi071 ceph-mon[44707]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:44:25.739 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-02T15:44:25.739 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-02T15:44:25.993 INFO:tasks.workunit.client.0.smithi001.stdout: 10 132546 28.70 MB/sec execute 118 sec latency 76.417 ms 2024-02-02T15:44:26.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.058+0000 7f8425800700 1 -- 172.21.15.1:0/2404630169 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84201019b0 msgr2=0x7f84201059f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:26.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.058+0000 7f8425800700 1 --2- 172.21.15.1:0/2404630169 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84201019b0 0x7f84201059f0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f8408009990 tx=0x7f8408009ca0 comp rx=0 tx=0).stop 2024-02-02T15:44:26.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.059+0000 7f8425800700 1 -- 172.21.15.1:0/2404630169 shutdown_connections 2024-02-02T15:44:26.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.059+0000 7f8425800700 1 --2- 172.21.15.1:0/2404630169 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8420106120 0x7f842010dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:26.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.059+0000 7f8425800700 1 --2- 172.21.15.1:0/2404630169 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8420101000 0x7f84201013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:26.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.059+0000 7f8425800700 1 --2- 172.21.15.1:0/2404630169 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f84201019b0 0x7f84201059f0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:26.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.059+0000 7f8425800700 1 -- 172.21.15.1:0/2404630169 >> 172.21.15.1:0/2404630169 conn(0x7f84200fc840 msgr2=0x7f84200fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:26.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.059+0000 7f8425800700 1 -- 172.21.15.1:0/2404630169 shutdown_connections 2024-02-02T15:44:26.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.059+0000 7f8425800700 1 -- 172.21.15.1:0/2404630169 wait complete. 2024-02-02T15:44:26.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.061+0000 7f8425800700 1 Processor -- start 2024-02-02T15:44:26.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.062+0000 7f8425800700 1 -- start start 2024-02-02T15:44:26.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.062+0000 7f8425800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8420101000 0x7f8420196fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:26.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.062+0000 7f8425800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8420106120 0x7f84201974f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:26.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.062+0000 7f8425800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f842019b950 0x7f842019bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:26.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.062+0000 7f8425800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f842019c3d0 con 0x7f8420101000 2024-02-02T15:44:26.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.062+0000 7f8425800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f842019c540 con 0x7f842019b950 2024-02-02T15:44:26.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.062+0000 7f8425800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f842019c700 con 0x7f8420106120 2024-02-02T15:44:26.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.063+0000 7f841ea00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8420101000 0x7f8420196fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:26.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.063+0000 7f841f400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f842019b950 0x7f842019bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:26.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.063+0000 7f841e000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8420106120 0x7f84201974f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:26.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.063+0000 7f841e000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8420106120 0x7f84201974f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:44856/0 (socket says 172.21.15.1:44856) 2024-02-02T15:44:26.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.063+0000 7f841e000700 1 -- 172.21.15.1:0/206011452 learned_addr learned my addr 172.21.15.1:0/206011452 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:44:26.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.063+0000 7f841f400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f842019b950 0x7f842019bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:51696/0 (socket says 172.21.15.1:51696) 2024-02-02T15:44:26.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.063+0000 7f841e000700 1 -- 172.21.15.1:0/206011452 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f842019b950 msgr2=0x7f842019bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:26.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.063+0000 7f841e000700 1 --2- 172.21.15.1:0/206011452 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f842019b950 0x7f842019bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:26.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.063+0000 7f841e000700 1 -- 172.21.15.1:0/206011452 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8420101000 msgr2=0x7f8420196fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:26.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.063+0000 7f841e000700 1 --2- 172.21.15.1:0/206011452 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8420101000 0x7f8420196fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:26.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.063+0000 7f841e000700 1 -- 172.21.15.1:0/206011452 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84080095f0 con 0x7f8420106120 2024-02-02T15:44:26.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.064+0000 7f841e000700 1 --2- 172.21.15.1:0/206011452 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8420106120 0x7f84201974f0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f840800c010 tx=0x7f8408004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:26.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.064+0000 7f8417400700 1 -- 172.21.15.1:0/206011452 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840801d070 con 0x7f8420106120 2024-02-02T15:44:26.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.064+0000 7f8417400700 1 -- 172.21.15.1:0/206011452 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8408005030 con 0x7f8420106120 2024-02-02T15:44:26.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.064+0000 7f8417400700 1 -- 172.21.15.1:0/206011452 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8408004a90 con 0x7f8420106120 2024-02-02T15:44:26.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.064+0000 7f8425800700 1 -- 172.21.15.1:0/206011452 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8420072f70 con 0x7f8420106120 2024-02-02T15:44:26.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.064+0000 7f8425800700 1 -- 172.21.15.1:0/206011452 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8420073370 con 0x7f8420106120 2024-02-02T15:44:26.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.069+0000 7f8417400700 1 -- 172.21.15.1:0/206011452 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f8408005480 con 0x7f8420106120 2024-02-02T15:44:26.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.071+0000 7f84393b1b80 1 Processor -- start 2024-02-02T15:44:26.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.071+0000 7f84393b1b80 1 -- start start 2024-02-02T15:44:26.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.072+0000 7f84393b1b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557d4d814090 0x557d4d814470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:26.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.072+0000 7f84393b1b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d4d8149b0 0x557d4d81f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:26.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.072+0000 7f84393b1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d4d81f860 0x557d4d821c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:26.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.072+0000 7f84393b1b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d4d788540 con 0x557d4d814090 2024-02-02T15:44:26.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.072+0000 7f84393b1b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d4d76cb20 con 0x557d4d8149b0 2024-02-02T15:44:26.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.072+0000 7f84393b1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d4d76a4e0 con 0x557d4d81f860 2024-02-02T15:44:26.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.072+0000 7f8417400700 1 --2- 172.21.15.1:0/206011452 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f84000a0860 0x7f84000a2d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:26.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.073+0000 7f8417400700 1 -- 172.21.15.1:0/206011452 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f84080cd1f0 con 0x7f8420106120 2024-02-02T15:44:26.073 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.073+0000 7f841f400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557d4d814090 0x557d4d814470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:26.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.073+0000 7f841f400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557d4d814090 0x557d4d814470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45188/0 (socket says 172.21.15.1:45188) 2024-02-02T15:44:26.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.073+0000 7f841ea00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d4d8149b0 0x557d4d81f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:26.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.073+0000 7f841f400700 1 -- 172.21.15.1:0/1391135548 learned_addr learned my addr 172.21.15.1:0/1391135548 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:44:26.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.073+0000 7f841e000700 1 --2- 172.21.15.1:0/1391135548 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d4d81f860 0x557d4d821c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:26.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.073+0000 7f841f400700 1 --2- 172.21.15.1:0/206011452 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f84000a0860 0x7f84000a2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:26.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.074+0000 7f841f400700 1 -- 172.21.15.1:0/1391135548 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d4d81f860 msgr2=0x557d4d821c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:26.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.074+0000 7f841f400700 1 --2- 172.21.15.1:0/1391135548 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d4d81f860 0x557d4d821c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:26.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.074+0000 7f841f400700 1 -- 172.21.15.1:0/1391135548 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d4d8149b0 msgr2=0x557d4d81f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:26.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.074+0000 7f841f400700 1 --2- 172.21.15.1:0/1391135548 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d4d8149b0 0x557d4d81f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:26.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.074+0000 7f841f400700 1 -- 172.21.15.1:0/1391135548 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d4d5e7430 con 0x557d4d814090 2024-02-02T15:44:26.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.075+0000 7f841f400700 1 --2- 172.21.15.1:0/1391135548 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557d4d814090 0x557d4d814470 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f841800ba30 tx=0x7f841800bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebcb85314e00572a server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:26.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.076+0000 7f841f400700 1 --2- 172.21.15.1:0/206011452 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f84000a0860 0x7f84000a2d20 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f841800e000 tx=0x7f841801b000 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:26.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.076+0000 7f840fe00700 1 -- 172.21.15.1:0/1391135548 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8418019620 con 0x557d4d814090 2024-02-02T15:44:26.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.076+0000 7f840fe00700 1 -- 172.21.15.1:0/1391135548 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8418019c40 con 0x557d4d814090 2024-02-02T15:44:26.076 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.076+0000 7f840fe00700 1 -- 172.21.15.1:0/1391135548 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8418019620 con 0x557d4d814090 2024-02-02T15:44:26.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.077+0000 7f84393b1b80 1 -- 172.21.15.1:0/1391135548 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557d4d814090 msgr2=0x557d4d814470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:26.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.077+0000 7f84393b1b80 1 --2- 172.21.15.1:0/1391135548 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557d4d814090 0x557d4d814470 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f841800ba30 tx=0x7f841800bd40 comp rx=0 tx=0).stop 2024-02-02T15:44:26.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.077+0000 7f84393b1b80 1 -- 172.21.15.1:0/1391135548 shutdown_connections 2024-02-02T15:44:26.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.077+0000 7f84393b1b80 1 --2- 172.21.15.1:0/1391135548 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d4d81f860 0x557d4d821c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:26.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.077+0000 7f84393b1b80 1 --2- 172.21.15.1:0/1391135548 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557d4d814090 0x557d4d814470 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:26.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.077+0000 7f84393b1b80 1 --2- 172.21.15.1:0/1391135548 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d4d8149b0 0x557d4d81f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:26.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.077+0000 7f84393b1b80 1 -- 172.21.15.1:0/1391135548 >> 172.21.15.1:0/1391135548 conn(0x557d4d773e30 msgr2=0x557d4d810430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:26.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.077+0000 7f84393b1b80 1 -- 172.21.15.1:0/1391135548 shutdown_connections 2024-02-02T15:44:26.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.078+0000 7f84393b1b80 1 -- 172.21.15.1:0/1391135548 wait complete. 2024-02-02T15:44:26.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.078+0000 7f84393b1b80 1 -- start start 2024-02-02T15:44:26.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.078+0000 7f84393b1b80 1 Processor -- start 2024-02-02T15:44:26.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.079+0000 7f84393b1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d4d814090 0x557d4d81f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:26.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.079+0000 7f84393b1b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d4d8149b0 0x557d4d81c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:26.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.079+0000 7f84393b1b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557d4d81f860 0x557d4d81ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:26.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.079+0000 7f841ea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d4d814090 0x557d4d81f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:26.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.079+0000 7f841ea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d4d814090 0x557d4d81f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:44874/0 (socket says 172.21.15.1:44874) 2024-02-02T15:44:26.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.079+0000 7f841e000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d4d8149b0 0x557d4d81c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:26.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.079+0000 7f841e000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d4d8149b0 0x557d4d81c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:51706/0 (socket says 172.21.15.1:51706) 2024-02-02T15:44:26.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.080+0000 7f841ea00700 1 -- 172.21.15.1:0/2545793549 learned_addr learned my addr 172.21.15.1:0/2545793549 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:44:26.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.080+0000 7f84393b1b80 1 -- 172.21.15.1:0/2545793549 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d4d765dd0 con 0x557d4d81f860 2024-02-02T15:44:26.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.080+0000 7f84393b1b80 1 -- 172.21.15.1:0/2545793549 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d4d701e00 con 0x557d4d8149b0 2024-02-02T15:44:26.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.080+0000 7f84393b1b80 1 -- 172.21.15.1:0/2545793549 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d4d701aa0 con 0x557d4d814090 2024-02-02T15:44:26.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.080+0000 7f84393b1b80 10 client.0 _ll_register_callbacks cb 0x557d4d703310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:44:26.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.080+0000 7f841f400700 1 --2- 172.21.15.1:0/2545793549 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557d4d81f860 0x557d4d81ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:26.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.080+0000 7f841ea00700 1 -- 172.21.15.1:0/2545793549 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d4d8149b0 msgr2=0x557d4d81c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:26.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.080+0000 7f841ea00700 1 --2- 172.21.15.1:0/2545793549 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d4d8149b0 0x557d4d81c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:26.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.080+0000 7f841ea00700 1 -- 172.21.15.1:0/2545793549 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557d4d81f860 msgr2=0x557d4d81ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:26.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.080+0000 7f841ea00700 1 --2- 172.21.15.1:0/2545793549 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557d4d81f860 0x557d4d81ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:26.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.080+0000 7f841ea00700 1 -- 172.21.15.1:0/2545793549 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8418013040 con 0x557d4d814090 2024-02-02T15:44:26.081 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.080+0000 7f841f400700 1 --2- 172.21.15.1:0/2545793549 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557d4d81f860 0x557d4d81ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:44:26.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.081+0000 7f841ea00700 1 --2- 172.21.15.1:0/2545793549 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d4d814090 0x557d4d81f230 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f84100095f0 tx=0x7f84100099b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=847ba1f20f9df964 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:26.082 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.081+0000 7f840cc00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:44:26.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.081+0000 7f840cc00700 1 -- 172.21.15.1:0/2545793549 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8410015070 con 0x557d4d814090 2024-02-02T15:44:26.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.081+0000 7f840cc00700 1 -- 172.21.15.1:0/2545793549 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f8410010520 con 0x557d4d814090 2024-02-02T15:44:26.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.081+0000 7f84393b1b80 1 -- 172.21.15.1:0/2545793549 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557d4d808170 con 0x557d4d814090 2024-02-02T15:44:26.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.082+0000 7f840cc00700 1 -- 172.21.15.1:0/2545793549 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f841000f5d0 con 0x557d4d814090 2024-02-02T15:44:26.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.082+0000 7f84393b1b80 10 client.30025 fetch_fsmap learned FSMap version 74 2024-02-02T15:44:26.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.082+0000 7f84393b1b80 1 -- 172.21.15.1:0/2545793549 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x557d4d593be0 con 0x557d4d814090 2024-02-02T15:44:26.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.082+0000 7f840cc00700 1 -- 172.21.15.1:0/2545793549 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557d4d808170 con 0x557d4d814090 2024-02-02T15:44:26.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.082+0000 7f840cc00700 1 -- 172.21.15.1:0/2545793549 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 74) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f8410019840 con 0x557d4d814090 2024-02-02T15:44:26.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.082+0000 7f84393b1b80 10 client.30025 fetch_fsmap finished waiting for FSMap version 74 2024-02-02T15:44:26.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.082+0000 7f84393b1b80 10 client.30025 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:44:26.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.082+0000 7f84393b1b80 20 client.30025 populate_metadata read hostname 'smithi001' 2024-02-02T15:44:26.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.082+0000 7f84393b1b80 1 --2- 172.21.15.1:0/2545793549 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x557d4d837ad0 0x557d4d839ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:26.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.082+0000 7f84393b1b80 4 client.30025 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:44:26.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.082+0000 7f84393b1b80 1 -- 172.21.15.1:0/2545793549 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557d4d75ee50 con 0x557d4d837ad0 2024-02-02T15:44:26.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.083+0000 7f841e000700 1 --2- 172.21.15.1:0/2545793549 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x557d4d837ad0 0x557d4d839ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:26.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.083+0000 7f841e000700 1 --2- 172.21.15.1:0/2545793549 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x557d4d837ad0 0x557d4d839ef0 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89cf2a4658866322 server_cookie=8c566fdfcc3e1099 in_seq=0 out_seq=0 2024-02-02T15:44:26.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.083+0000 7f840cc00700 10 client.30025 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:44:26.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.085+0000 7f840cc00700 1 -- 172.21.15.1:0/2545793549 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x557d4d75ee50 con 0x557d4d837ad0 2024-02-02T15:44:26.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.085+0000 7f840cc00700 10 client.30025 handle_command_reply: tid=0 2024-02-02T15:44:26.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.085+0000 7f84393b1b80 1 client.30025 shutdown 2024-02-02T15:44:26.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.085+0000 7f84393b1b80 1 -- 172.21.15.1:0/2545793549 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d4d814090 msgr2=0x557d4d81f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:26.085 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.085+0000 7f84393b1b80 1 --2- 172.21.15.1:0/2545793549 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d4d814090 0x557d4d81f230 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f84100095f0 tx=0x7f84100099b0 comp rx=0 tx=0).stop 2024-02-02T15:44:26.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.085+0000 7f84393b1b80 1 -- 172.21.15.1:0/2545793549 shutdown_connections 2024-02-02T15:44:26.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.085+0000 7f84393b1b80 1 --2- 172.21.15.1:0/2545793549 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x557d4d837ad0 0x557d4d839ef0 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:26.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.085+0000 7f84393b1b80 1 --2- 172.21.15.1:0/2545793549 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557d4d81f860 0x557d4d81ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:26.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.085+0000 7f84393b1b80 1 --2- 172.21.15.1:0/2545793549 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d4d814090 0x557d4d81f230 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:26.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.085+0000 7f84393b1b80 1 --2- 172.21.15.1:0/2545793549 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d4d8149b0 0x557d4d81c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:26.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.085+0000 7f84393b1b80 1 -- 172.21.15.1:0/2545793549 >> 172.21.15.1:0/2545793549 conn(0x557d4d773e30 msgr2=0x557d4d810020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:26.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.085+0000 7f840cc00700 0 client.30025 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:44:26.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.085+0000 7f84393b1b80 1 -- 172.21.15.1:0/2545793549 shutdown_connections 2024-02-02T15:44:26.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.086+0000 7f84393b1b80 1 -- 172.21.15.1:0/2545793549 wait complete. 2024-02-02T15:44:26.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.086+0000 7f84393b1b80 20 client.30025 trim_cache size 0 max 16384 2024-02-02T15:44:26.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.109+0000 7f84393b1b80 1 Processor -- start 2024-02-02T15:44:26.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.110+0000 7f84393b1b80 1 -- start start 2024-02-02T15:44:26.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.110+0000 7f84393b1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d4d814090 0x557d4d8263b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:26.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.110+0000 7f84393b1b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557d4d8149b0 0x557d4d816ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:26.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.110+0000 7f84393b1b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d4d81f860 0x557d4d81bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:26.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.110+0000 7f84393b1b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d4d5e9720 con 0x557d4d8149b0 2024-02-02T15:44:26.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.110+0000 7f84393b1b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d4d8158e0 con 0x557d4d81f860 2024-02-02T15:44:26.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.110+0000 7f84393b1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d4d80a600 con 0x557d4d814090 2024-02-02T15:44:26.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.110+0000 7f841e000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557d4d8149b0 0x557d4d816ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:26.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.110+0000 7f841e000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557d4d8149b0 0x557d4d816ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45202/0 (socket says 172.21.15.1:45202) 2024-02-02T15:44:26.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.110+0000 7f841e000700 1 -- 172.21.15.1:0/1218409523 learned_addr learned my addr 172.21.15.1:0/1218409523 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:44:26.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.110+0000 7f841ea00700 1 --2- 172.21.15.1:0/1218409523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d4d814090 0x557d4d8263b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:26.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.110+0000 7f841f400700 1 --2- 172.21.15.1:0/1218409523 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d4d81f860 0x557d4d81bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:26.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.111+0000 7f841e000700 1 -- 172.21.15.1:0/1218409523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d4d814090 msgr2=0x557d4d8263b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:26.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.111+0000 7f841e000700 1 --2- 172.21.15.1:0/1218409523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d4d814090 0x557d4d8263b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:26.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.111+0000 7f841e000700 1 -- 172.21.15.1:0/1218409523 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d4d81f860 msgr2=0x557d4d81bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:26.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.111+0000 7f841e000700 1 --2- 172.21.15.1:0/1218409523 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d4d81f860 0x557d4d81bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:26.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.111+0000 7f841e000700 1 -- 172.21.15.1:0/1218409523 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d4d72bf20 con 0x557d4d8149b0 2024-02-02T15:44:26.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.111+0000 7f841ea00700 1 --2- 172.21.15.1:0/1218409523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d4d814090 0x557d4d8263b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:44:26.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.111+0000 7f841e000700 1 --2- 172.21.15.1:0/1218409523 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557d4d8149b0 0x557d4d816ae0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f840809a330 tx=0x7f84080a0040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2655d2397f62dbbe server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:26.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.112+0000 7f8414c00700 1 -- 172.21.15.1:0/1218409523 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840809a3a0 con 0x557d4d8149b0 2024-02-02T15:44:26.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.112+0000 7f8414c00700 1 -- 172.21.15.1:0/1218409523 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f840809e1d0 con 0x557d4d8149b0 2024-02-02T15:44:26.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.112+0000 7f8414c00700 1 -- 172.21.15.1:0/1218409523 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84080a0310 con 0x557d4d8149b0 2024-02-02T15:44:26.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.113+0000 7f84393b1b80 1 -- 172.21.15.1:0/1218409523 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557d4d8149b0 msgr2=0x557d4d816ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:26.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.113+0000 7f84393b1b80 1 --2- 172.21.15.1:0/1218409523 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557d4d8149b0 0x557d4d816ae0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f840809a330 tx=0x7f84080a0040 comp rx=0 tx=0).stop 2024-02-02T15:44:26.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.113+0000 7f84393b1b80 1 -- 172.21.15.1:0/1218409523 shutdown_connections 2024-02-02T15:44:26.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.113+0000 7f84393b1b80 1 --2- 172.21.15.1:0/1218409523 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d4d81f860 0x557d4d81bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:26.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.113+0000 7f84393b1b80 1 --2- 172.21.15.1:0/1218409523 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d4d814090 0x557d4d8263b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:26.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.113+0000 7f84393b1b80 1 --2- 172.21.15.1:0/1218409523 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557d4d8149b0 0x557d4d816ae0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:26.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.113+0000 7f84393b1b80 1 -- 172.21.15.1:0/1218409523 >> 172.21.15.1:0/1218409523 conn(0x557d4d773e30 msgr2=0x557d4d80f640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:26.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.113+0000 7f84393b1b80 1 -- 172.21.15.1:0/1218409523 shutdown_connections 2024-02-02T15:44:26.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.114+0000 7f84393b1b80 1 -- 172.21.15.1:0/1218409523 wait complete. 2024-02-02T15:44:26.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.114+0000 7f84393b1b80 1 -- start start 2024-02-02T15:44:26.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.115+0000 7f84393b1b80 1 Processor -- start 2024-02-02T15:44:26.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.116+0000 7f84393b1b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557d4d814090 0x557d4d825bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:26.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.116+0000 7f84393b1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d4d8149b0 0x557d4d826110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:26.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.116+0000 7f84393b1b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d4d81f860 0x557d4d824390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:26.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.116+0000 7f84393b1b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d4d824960 con 0x557d4d814090 2024-02-02T15:44:26.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.116+0000 7f84393b1b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d4d824ad0 con 0x557d4d81f860 2024-02-02T15:44:26.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.116+0000 7f84393b1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d4d824c90 con 0x557d4d8149b0 2024-02-02T15:44:26.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.116+0000 7f84393b1b80 10 client.0 _ll_register_callbacks cb 0x557d4d703310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:44:26.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.117+0000 7f841e000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d4d8149b0 0x557d4d826110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:26.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.117+0000 7f841e000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d4d8149b0 0x557d4d826110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:44886/0 (socket says 172.21.15.1:44886) 2024-02-02T15:44:26.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.117+0000 7f841e000700 1 -- 172.21.15.1:0/2988145869 learned_addr learned my addr 172.21.15.1:0/2988145869 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:44:26.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.117+0000 7f841f400700 1 --2- 172.21.15.1:0/2988145869 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d4d81f860 0x557d4d824390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:26.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.117+0000 7f841e000700 1 -- 172.21.15.1:0/2988145869 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d4d81f860 msgr2=0x557d4d824390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:26.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.117+0000 7f841e000700 1 --2- 172.21.15.1:0/2988145869 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d4d81f860 0x557d4d824390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:26.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.117+0000 7f841e000700 1 -- 172.21.15.1:0/2988145869 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557d4d814090 msgr2=0x557d4d825bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:26.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.117+0000 7f841e000700 1 --2- 172.21.15.1:0/2988145869 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557d4d814090 0x557d4d825bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:26.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.118+0000 7f841e000700 1 -- 172.21.15.1:0/2988145869 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557d4d593be0 con 0x557d4d8149b0 2024-02-02T15:44:26.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.118+0000 7f841e000700 1 --2- 172.21.15.1:0/2988145869 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d4d8149b0 0x557d4d826110 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f8408096e10 tx=0x7f840800b3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3d5e235ce987fcd server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:26.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.118+0000 7f840cc00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:44:26.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.118+0000 7f840cc00700 1 -- 172.21.15.1:0/2988145869 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84080b8070 con 0x557d4d8149b0 2024-02-02T15:44:26.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.119+0000 7f84393b1b80 1 -- 172.21.15.1:0/2988145869 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8418019c40 con 0x557d4d8149b0 2024-02-02T15:44:26.119 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.119+0000 7f840cc00700 1 -- 172.21.15.1:0/2988145869 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f840809a3a0 con 0x557d4d8149b0 2024-02-02T15:44:26.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.120+0000 7f840cc00700 1 -- 172.21.15.1:0/2988145869 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84080a00b0 con 0x557d4d8149b0 2024-02-02T15:44:26.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.120+0000 7f84393b1b80 10 client.30031 fetch_fsmap learned FSMap version 74 2024-02-02T15:44:26.120 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.120+0000 7f84393b1b80 1 -- 172.21.15.1:0/2988145869 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x7f8420073370 con 0x557d4d8149b0 2024-02-02T15:44:26.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.122+0000 7f840cc00700 1 -- 172.21.15.1:0/2988145869 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f84080aa080 con 0x557d4d8149b0 2024-02-02T15:44:26.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.122+0000 7f840cc00700 1 -- 172.21.15.1:0/2988145869 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 74) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f840809a600 con 0x557d4d8149b0 2024-02-02T15:44:26.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.122+0000 7f84393b1b80 10 client.30031 fetch_fsmap finished waiting for FSMap version 74 2024-02-02T15:44:26.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.122+0000 7f84393b1b80 10 client.30031 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:44:26.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.122+0000 7f84393b1b80 20 client.30031 populate_metadata read hostname 'smithi001' 2024-02-02T15:44:26.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.122+0000 7f84393b1b80 1 --2- 172.21.15.1:0/2988145869 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x557d4d88a340 0x557d4d88c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:26.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.122+0000 7f84393b1b80 4 client.30031 mds_command: new command op to 24557 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-02T15:44:26.122 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.122+0000 7f84393b1b80 1 -- 172.21.15.1:0/2988145869 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x557d4d724fd0 con 0x557d4d88a340 2024-02-02T15:44:26.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.123+0000 7f841ea00700 1 --2- 172.21.15.1:0/2988145869 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x557d4d88a340 0x557d4d88c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:26.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.124+0000 7f840cc00700 10 client.30031 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:44:26.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.124+0000 7f841ea00700 1 --2- 172.21.15.1:0/2988145869 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x557d4d88a340 0x557d4d88c730 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b44ba726d48d6ff server_cookie=8f1783fb65583785 in_seq=0 out_seq=0 2024-02-02T15:44:26.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.125+0000 7f840cc00700 1 -- 172.21.15.1:0/2988145869 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x557d4d724fd0 con 0x557d4d88a340 2024-02-02T15:44:26.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.125+0000 7f840cc00700 10 client.30031 handle_command_reply: tid=0 2024-02-02T15:44:26.125 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.125+0000 7f84393b1b80 1 client.30031 shutdown 2024-02-02T15:44:26.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.126+0000 7f84393b1b80 1 -- 172.21.15.1:0/2988145869 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d4d8149b0 msgr2=0x557d4d826110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:26.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.126+0000 7f84393b1b80 1 --2- 172.21.15.1:0/2988145869 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d4d8149b0 0x557d4d826110 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f8408096e10 tx=0x7f840800b3a0 comp rx=0 tx=0).stop 2024-02-02T15:44:26.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.126+0000 7f84393b1b80 1 -- 172.21.15.1:0/2988145869 shutdown_connections 2024-02-02T15:44:26.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.126+0000 7f84393b1b80 1 --2- 172.21.15.1:0/2988145869 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x557d4d88a340 0x557d4d88c730 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:26.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.126+0000 7f84393b1b80 1 --2- 172.21.15.1:0/2988145869 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557d4d81f860 0x557d4d824390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:26.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.126+0000 7f84393b1b80 1 --2- 172.21.15.1:0/2988145869 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557d4d814090 0x557d4d825bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:26.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.126+0000 7f84393b1b80 1 --2- 172.21.15.1:0/2988145869 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557d4d8149b0 0x557d4d826110 secure :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f8408096e10 tx=0x7f840800b3a0 comp rx=0 tx=0).stop 2024-02-02T15:44:26.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.126+0000 7f840cc00700 0 client.30031 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:44:26.126 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.126+0000 7f84393b1b80 1 -- 172.21.15.1:0/2988145869 >> 172.21.15.1:0/2988145869 conn(0x557d4d773e30 msgr2=0x557d4d80fed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:26.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.126+0000 7f84393b1b80 1 -- 172.21.15.1:0/2988145869 shutdown_connections 2024-02-02T15:44:26.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.127+0000 7f84393b1b80 1 -- 172.21.15.1:0/2988145869 wait complete. 2024-02-02T15:44:26.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.127+0000 7f84393b1b80 20 client.30031 trim_cache size 0 max 16384 2024-02-02T15:44:26.127 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:44:26.127 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2024-02-02T15:44:26.128 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "4b35180c-5771-4e5f-8253-ac0913eb65d9", 2024-02-02T15:44:26.128 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2024-02-02T15:44:26.128 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:44:26.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.129+0000 7f8425800700 1 -- 172.21.15.1:0/206011452 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f84000a0860 msgr2=0x7f84000a2d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:26.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.129+0000 7f8425800700 1 --2- 172.21.15.1:0/206011452 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f84000a0860 0x7f84000a2d20 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f841800e000 tx=0x7f841801b000 comp rx=0 tx=0).stop 2024-02-02T15:44:26.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.129+0000 7f8425800700 1 -- 172.21.15.1:0/206011452 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8420106120 msgr2=0x7f84201974f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:26.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.129+0000 7f8425800700 1 --2- 172.21.15.1:0/206011452 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8420106120 0x7f84201974f0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f840800c010 tx=0x7f8408004730 comp rx=0 tx=0).stop 2024-02-02T15:44:26.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.129+0000 7f8425800700 1 -- 172.21.15.1:0/206011452 shutdown_connections 2024-02-02T15:44:26.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.129+0000 7f8425800700 1 --2- 172.21.15.1:0/206011452 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f84000a0860 0x7f84000a2d20 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:26.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.129+0000 7f8425800700 1 --2- 172.21.15.1:0/206011452 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f842019b950 0x7f842019bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:26.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.129+0000 7f8425800700 1 --2- 172.21.15.1:0/206011452 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8420101000 0x7f8420196fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:26.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.129+0000 7f8425800700 1 --2- 172.21.15.1:0/206011452 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8420106120 0x7f84201974f0 secure :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f840800c010 tx=0x7f8408004730 comp rx=0 tx=0).stop 2024-02-02T15:44:26.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.129+0000 7f8425800700 1 -- 172.21.15.1:0/206011452 >> 172.21.15.1:0/206011452 conn(0x7f84200fc840 msgr2=0x7f84200fd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:26.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.131+0000 7f8425800700 1 -- 172.21.15.1:0/206011452 shutdown_connections 2024-02-02T15:44:26.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.132+0000 7f8425800700 1 -- 172.21.15.1:0/206011452 wait complete. 2024-02-02T15:44:26.165 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-02T15:44:26.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:44:26 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:44:26.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.451+0000 7f2b7bc00700 1 -- 172.21.15.1:0/3316608727 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2b74068010 msgr2=0x7f2b74068470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:26.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.451+0000 7f2b7bc00700 1 --2- 172.21.15.1:0/3316608727 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2b74068010 0x7f2b74068470 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f2b5c009990 tx=0x7f2b5c009ca0 comp rx=0 tx=0).stop 2024-02-02T15:44:26.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.451+0000 7f2b7bc00700 1 -- 172.21.15.1:0/3316608727 shutdown_connections 2024-02-02T15:44:26.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.451+0000 7f2b7bc00700 1 --2- 172.21.15.1:0/3316608727 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b740689b0 0x7f2b7410dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:26.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.451+0000 7f2b7bc00700 1 --2- 172.21.15.1:0/3316608727 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b74106ed0 0x7f2b741072b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:26.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.451+0000 7f2b7bc00700 1 --2- 172.21.15.1:0/3316608727 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2b74068010 0x7f2b74068470 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:26.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.451+0000 7f2b7bc00700 1 -- 172.21.15.1:0/3316608727 >> 172.21.15.1:0/3316608727 conn(0x7f2b740757d0 msgr2=0x7f2b74075be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:26.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.451+0000 7f2b7bc00700 1 -- 172.21.15.1:0/3316608727 shutdown_connections 2024-02-02T15:44:26.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.452+0000 7f2b7bc00700 1 -- 172.21.15.1:0/3316608727 wait complete. 2024-02-02T15:44:26.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.453+0000 7f2b7bc00700 1 Processor -- start 2024-02-02T15:44:26.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.453+0000 7f2b7bc00700 1 -- start start 2024-02-02T15:44:26.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.454+0000 7f2b7bc00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2b74068010 0x7f2b74199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:26.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.454+0000 7f2b7bc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b740689b0 0x7f2b741996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:26.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.454+0000 7f2b7bc00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b74106ed0 0x7f2b7419db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:26.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.454+0000 7f2b7bc00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7419e150 con 0x7f2b74106ed0 2024-02-02T15:44:26.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.454+0000 7f2b7bc00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7419e2c0 con 0x7f2b74068010 2024-02-02T15:44:26.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.454+0000 7f2b7bc00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7419e480 con 0x7f2b740689b0 2024-02-02T15:44:26.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.454+0000 7f2b79200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2b74068010 0x7f2b74199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:26.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.454+0000 7f2b73e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b740689b0 0x7f2b741996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:26.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.454+0000 7f2b79200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2b74068010 0x7f2b74199170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:51742/0 (socket says 172.21.15.1:51742) 2024-02-02T15:44:26.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.454+0000 7f2b79200700 1 -- 172.21.15.1:0/3061483113 learned_addr learned my addr 172.21.15.1:0/3061483113 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:44:26.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.454+0000 7f2b79c00700 1 --2- 172.21.15.1:0/3061483113 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b74106ed0 0x7f2b7419db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:26.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.455+0000 7f2b79200700 1 -- 172.21.15.1:0/3061483113 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b740689b0 msgr2=0x7f2b741996b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:26.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.455+0000 7f2b79200700 1 --2- 172.21.15.1:0/3061483113 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b740689b0 0x7f2b741996b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:26.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.455+0000 7f2b79200700 1 -- 172.21.15.1:0/3061483113 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b74106ed0 msgr2=0x7f2b7419db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:26.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.455+0000 7f2b79200700 1 --2- 172.21.15.1:0/3061483113 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b74106ed0 0x7f2b7419db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:26.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.455+0000 7f2b79200700 1 -- 172.21.15.1:0/3061483113 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b5c0095f0 con 0x7f2b74068010 2024-02-02T15:44:26.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.455+0000 7f2b79c00700 1 --2- 172.21.15.1:0/3061483113 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b74106ed0 0x7f2b7419db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:44:26.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.456+0000 7f2b79200700 1 --2- 172.21.15.1:0/3061483113 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2b74068010 0x7f2b74199170 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f2b6400e9a0 tx=0x7f2b6400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:26.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.456+0000 7f2b71600700 1 -- 172.21.15.1:0/3061483113 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b64009790 con 0x7f2b74068010 2024-02-02T15:44:26.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.457+0000 7f2b71600700 1 -- 172.21.15.1:0/3061483113 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b64010430 con 0x7f2b74068010 2024-02-02T15:44:26.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.457+0000 7f2b71600700 1 -- 172.21.15.1:0/3061483113 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b64018680 con 0x7f2b74068010 2024-02-02T15:44:26.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.457+0000 7f2b7bc00700 1 -- 172.21.15.1:0/3061483113 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b7419e760 con 0x7f2b74068010 2024-02-02T15:44:26.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.457+0000 7f2b7bc00700 1 -- 172.21.15.1:0/3061483113 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b741a5fa0 con 0x7f2b74068010 2024-02-02T15:44:26.461 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.461+0000 7f2b71600700 1 -- 172.21.15.1:0/3061483113 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f2b640105a0 con 0x7f2b74068010 2024-02-02T15:44:26.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.463+0000 7f2b8f7a7b80 1 Processor -- start 2024-02-02T15:44:26.463 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.463+0000 7f2b8f7a7b80 1 -- start start 2024-02-02T15:44:26.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.463+0000 7f2b8f7a7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b1336a9010 0x55b1336a93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:26.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.463+0000 7f2b8f7a7b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b1336a9930 0x55b1336b42a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:26.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.463+0000 7f2b71600700 1 --2- 172.21.15.1:0/3061483113 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f2b540a0710 0x7f2b540a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:26.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.463+0000 7f2b8f7a7b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b1336b47e0 0x55b1336b6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:26.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.463+0000 7f2b8f7a7b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b13361d4c0 con 0x55b1336b47e0 2024-02-02T15:44:26.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.463+0000 7f2b8f7a7b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b133601aa0 con 0x55b1336a9930 2024-02-02T15:44:26.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.463+0000 7f2b8f7a7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1335ff460 con 0x55b1336a9010 2024-02-02T15:44:26.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.464+0000 7f2b71600700 1 -- 172.21.15.1:0/3061483113 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f2b64014070 con 0x7f2b74068010 2024-02-02T15:44:26.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.464+0000 7f2b73e00700 1 --2- 172.21.15.1:0/3061483113 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f2b540a0710 0x7f2b540a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:26.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.464+0000 7f2b79c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b1336a9010 0x55b1336a93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:26.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.464+0000 7f2b79200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b1336a9930 0x55b1336b42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:26.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.464+0000 7f2b79c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b1336a9010 0x55b1336a93f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:44910/0 (socket says 172.21.15.1:44910) 2024-02-02T15:44:26.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.464+0000 7f2b79c00700 1 -- 172.21.15.1:0/25345335 learned_addr learned my addr 172.21.15.1:0/25345335 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:44:26.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.464+0000 7f2b79c00700 1 --2- 172.21.15.1:0/25345335 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b1336b47e0 0x55b1336b6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:26.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.464+0000 7f2b79c00700 1 -- 172.21.15.1:0/25345335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b1336a9010 msgr2=0x55b1336a93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:26.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.464+0000 7f2b79c00700 1 --2- 172.21.15.1:0/25345335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b1336a9010 0x55b1336a93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:26.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.465+0000 7f2b79c00700 1 -- 172.21.15.1:0/25345335 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b1336a9930 msgr2=0x55b1336b42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:26.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.465+0000 7f2b79c00700 1 --2- 172.21.15.1:0/25345335 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b1336a9930 0x55b1336b42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:26.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.465+0000 7f2b79c00700 1 -- 172.21.15.1:0/25345335 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b13347c3b0 con 0x55b1336b47e0 2024-02-02T15:44:26.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.465+0000 7f2b73e00700 1 --2- 172.21.15.1:0/3061483113 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f2b540a0710 0x7f2b540a2bd0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f2b7419a780 tx=0x7f2b5c005fb0 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:26.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.465+0000 7f2b79c00700 1 --2- 172.21.15.1:0/25345335 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b1336b47e0 0x55b1336b6bd0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c00bd80 tx=0x7f2b6c009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eac41118316ffbe2 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:26.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.465+0000 7f2b6a000700 1 -- 172.21.15.1:0/25345335 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b6c015cf0 con 0x55b1336b47e0 2024-02-02T15:44:26.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.465+0000 7f2b6a000700 1 -- 172.21.15.1:0/25345335 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b6c00be70 con 0x55b1336b47e0 2024-02-02T15:44:26.466 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.465+0000 7f2b6a000700 1 -- 172.21.15.1:0/25345335 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b6c008580 con 0x55b1336b47e0 2024-02-02T15:44:26.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.466+0000 7f2b8f7a7b80 1 -- 172.21.15.1:0/25345335 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b1336b47e0 msgr2=0x55b1336b6bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:26.467 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.466+0000 7f2b8f7a7b80 1 --2- 172.21.15.1:0/25345335 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b1336b47e0 0x55b1336b6bd0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c00bd80 tx=0x7f2b6c009650 comp rx=0 tx=0).stop 2024-02-02T15:44:26.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.466+0000 7f2b8f7a7b80 1 -- 172.21.15.1:0/25345335 shutdown_connections 2024-02-02T15:44:26.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.466+0000 7f2b8f7a7b80 1 --2- 172.21.15.1:0/25345335 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b1336b47e0 0x55b1336b6bd0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:26.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.466+0000 7f2b8f7a7b80 1 --2- 172.21.15.1:0/25345335 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b1336a9010 0x55b1336a93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:26.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.466+0000 7f2b8f7a7b80 1 --2- 172.21.15.1:0/25345335 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b1336a9930 0x55b1336b42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:26.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.466+0000 7f2b8f7a7b80 1 -- 172.21.15.1:0/25345335 >> 172.21.15.1:0/25345335 conn(0x55b133608db0 msgr2=0x55b1336a53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:26.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.466+0000 7f2b8f7a7b80 1 -- 172.21.15.1:0/25345335 shutdown_connections 2024-02-02T15:44:26.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.467+0000 7f2b8f7a7b80 1 -- 172.21.15.1:0/25345335 wait complete. 2024-02-02T15:44:26.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.467+0000 7f2b8f7a7b80 1 -- start start 2024-02-02T15:44:26.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.467+0000 7f2b8f7a7b80 1 Processor -- start 2024-02-02T15:44:26.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.468+0000 7f2b8f7a7b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b1336a9010 0x55b1336b4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:26.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.468+0000 7f2b8f7a7b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b1336a9930 0x55b1336b1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:26.470 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.468+0000 7f2b8f7a7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b1336b47e0 0x55b1336b1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:26.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.468+0000 7f2b8f7a7b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1335fad50 con 0x55b1336a9010 2024-02-02T15:44:26.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.468+0000 7f2b8f7a7b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b133596d80 con 0x55b1336a9930 2024-02-02T15:44:26.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.468+0000 7f2b8f7a7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b133596a20 con 0x55b1336b47e0 2024-02-02T15:44:26.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.468+0000 7f2b8f7a7b80 10 client.0 _ll_register_callbacks cb 0x55b133598290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:44:26.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.468+0000 7f2b79200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b1336a9010 0x55b1336b4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:26.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.468+0000 7f2b79200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b1336a9010 0x55b1336b4260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:45236/0 (socket says 172.21.15.1:45236) 2024-02-02T15:44:26.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.468+0000 7f2b79200700 1 -- 172.21.15.1:0/1931184715 learned_addr learned my addr 172.21.15.1:0/1931184715 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:44:26.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.468+0000 7f2b73e00700 1 --2- 172.21.15.1:0/1931184715 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b1336a9930 0x55b1336b1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:26.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.468+0000 7f2b79c00700 1 --2- 172.21.15.1:0/1931184715 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b1336b47e0 0x55b1336b1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:26.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.468+0000 7f2b79200700 1 -- 172.21.15.1:0/1931184715 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b1336b47e0 msgr2=0x55b1336b1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:26.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.468+0000 7f2b79200700 1 --2- 172.21.15.1:0/1931184715 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b1336b47e0 0x55b1336b1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:26.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.469+0000 7f2b79200700 1 -- 172.21.15.1:0/1931184715 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b1336a9930 msgr2=0x55b1336b1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:26.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.469+0000 7f2b79200700 1 --2- 172.21.15.1:0/1931184715 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b1336a9930 0x55b1336b1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:26.471 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.469+0000 7f2b79200700 1 -- 172.21.15.1:0/1931184715 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b6c00ba30 con 0x55b1336a9010 2024-02-02T15:44:26.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.469+0000 7f2b79200700 1 --2- 172.21.15.1:0/1931184715 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b1336a9010 0x55b1336b4260 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f2b64099350 tx=0x7f2b64095050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d789e0095b830cd3 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:26.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.469+0000 7f2b62a00700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:44:26.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.469+0000 7f2b62a00700 1 -- 172.21.15.1:0/1931184715 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b64098ab0 con 0x55b1336a9010 2024-02-02T15:44:26.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.469+0000 7f2b62a00700 1 -- 172.21.15.1:0/1931184715 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f2b640993c0 con 0x55b1336a9010 2024-02-02T15:44:26.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.469+0000 7f2b8f7a7b80 1 -- 172.21.15.1:0/1931184715 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b13369d0f0 con 0x55b1336a9010 2024-02-02T15:44:26.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.469+0000 7f2b62a00700 1 -- 172.21.15.1:0/1931184715 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2b6409d1f0 con 0x55b1336a9010 2024-02-02T15:44:26.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.469+0000 7f2b8f7a7b80 10 client.19896 fetch_fsmap learned FSMap version 74 2024-02-02T15:44:26.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.470+0000 7f2b8f7a7b80 1 -- 172.21.15.1:0/1931184715 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x55b133428b60 con 0x55b1336a9010 2024-02-02T15:44:26.472 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.470+0000 7f2b62a00700 1 -- 172.21.15.1:0/1931184715 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b640b5080 con 0x55b1336a9010 2024-02-02T15:44:26.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.470+0000 7f2b62a00700 1 -- 172.21.15.1:0/1931184715 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 74) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f2b64082c50 con 0x55b1336a9010 2024-02-02T15:44:26.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.470+0000 7f2b8f7a7b80 10 client.19896 fetch_fsmap finished waiting for FSMap version 74 2024-02-02T15:44:26.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.470+0000 7f2b8f7a7b80 10 client.19896 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:44:26.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.470+0000 7f2b8f7a7b80 20 client.19896 populate_metadata read hostname 'smithi001' 2024-02-02T15:44:26.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.470+0000 7f2b8f7a7b80 1 --2- 172.21.15.1:0/1931184715 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55b1336c4280 0x55b1336c6670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:26.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.470+0000 7f2b8f7a7b80 4 client.19896 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:44:26.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.470+0000 7f2b8f7a7b80 1 -- 172.21.15.1:0/1931184715 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b1335f3dd0 con 0x55b1336c4280 2024-02-02T15:44:26.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.471+0000 7f2b73e00700 1 --2- 172.21.15.1:0/1931184715 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55b1336c4280 0x55b1336c6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:26.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.471+0000 7f2b62a00700 10 client.19896 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:44:26.473 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.472+0000 7f2b73e00700 1 --2- 172.21.15.1:0/1931184715 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55b1336c4280 0x55b1336c6670 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72ab7cc8666e6953 server_cookie=bf062efa61d6950b in_seq=0 out_seq=0 2024-02-02T15:44:26.474 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.475+0000 7f2b62a00700 1 -- 172.21.15.1:0/1931184715 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55b1335f3dd0 con 0x55b1336c4280 2024-02-02T15:44:26.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.475+0000 7f2b62a00700 10 client.19896 handle_command_reply: tid=0 2024-02-02T15:44:26.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.475+0000 7f2b8f7a7b80 1 client.19896 shutdown 2024-02-02T15:44:26.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.475+0000 7f2b8f7a7b80 1 -- 172.21.15.1:0/1931184715 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b1336a9010 msgr2=0x55b1336b4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:26.475 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.475+0000 7f2b8f7a7b80 1 --2- 172.21.15.1:0/1931184715 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b1336a9010 0x55b1336b4260 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f2b64099350 tx=0x7f2b64095050 comp rx=0 tx=0).stop 2024-02-02T15:44:26.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.475+0000 7f2b8f7a7b80 1 -- 172.21.15.1:0/1931184715 shutdown_connections 2024-02-02T15:44:26.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.475+0000 7f2b8f7a7b80 1 --2- 172.21.15.1:0/1931184715 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55b1336c4280 0x55b1336c6670 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:26.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.475+0000 7f2b8f7a7b80 1 --2- 172.21.15.1:0/1931184715 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b1336b47e0 0x55b1336b1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:26.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.475+0000 7f2b8f7a7b80 1 --2- 172.21.15.1:0/1931184715 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b1336a9010 0x55b1336b4260 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:26.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.475+0000 7f2b8f7a7b80 1 --2- 172.21.15.1:0/1931184715 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b1336a9930 0x55b1336b1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:26.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.476+0000 7f2b8f7a7b80 1 -- 172.21.15.1:0/1931184715 >> 172.21.15.1:0/1931184715 conn(0x55b133608db0 msgr2=0x55b1336a4f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:26.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.476+0000 7f2b62a00700 0 client.19896 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:44:26.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.476+0000 7f2b8f7a7b80 1 -- 172.21.15.1:0/1931184715 shutdown_connections 2024-02-02T15:44:26.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.476+0000 7f2b8f7a7b80 1 -- 172.21.15.1:0/1931184715 wait complete. 2024-02-02T15:44:26.476 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.476+0000 7f2b8f7a7b80 20 client.19896 trim_cache size 0 max 16384 2024-02-02T15:44:26.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.505+0000 7f2b8f7a7b80 1 Processor -- start 2024-02-02T15:44:26.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.505+0000 7f2b8f7a7b80 1 -- start start 2024-02-02T15:44:26.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.506+0000 7f2b8f7a7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b1336a9010 0x55b1336bb330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:26.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.506+0000 7f2b8f7a7b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b1336a9930 0x55b1336ab840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:26.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.506+0000 7f2b8f7a7b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b1336b47e0 0x55b1336bf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:26.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.506+0000 7f2b8f7a7b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b13347e6a0 con 0x55b1336b47e0 2024-02-02T15:44:26.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.506+0000 7f2b8f7a7b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1336a61e0 con 0x55b1336a9930 2024-02-02T15:44:26.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.506+0000 7f2b8f7a7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1336ac400 con 0x55b1336a9010 2024-02-02T15:44:26.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.506+0000 7f2b73e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b1336a9930 0x55b1336ab840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:26.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.506+0000 7f2b73e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b1336a9930 0x55b1336ab840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:51776/0 (socket says 172.21.15.1:51776) 2024-02-02T15:44:26.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.506+0000 7f2b73e00700 1 -- 172.21.15.1:0/3613608837 learned_addr learned my addr 172.21.15.1:0/3613608837 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:44:26.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.506+0000 7f2b79c00700 1 --2- 172.21.15.1:0/3613608837 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b1336b47e0 0x55b1336bf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:26.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.506+0000 7f2b79200700 1 --2- 172.21.15.1:0/3613608837 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b1336a9010 0x55b1336bb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:26.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.507+0000 7f2b79c00700 1 -- 172.21.15.1:0/3613608837 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b1336a9010 msgr2=0x55b1336bb330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:26.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.507+0000 7f2b79c00700 1 --2- 172.21.15.1:0/3613608837 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b1336a9010 0x55b1336bb330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:26.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.507+0000 7f2b79c00700 1 -- 172.21.15.1:0/3613608837 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b1336a9930 msgr2=0x55b1336ab840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:26.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.507+0000 7f2b79c00700 1 --2- 172.21.15.1:0/3613608837 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b1336a9930 0x55b1336ab840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:26.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.507+0000 7f2b79c00700 1 -- 172.21.15.1:0/3613608837 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b1335c0ea0 con 0x55b1336b47e0 2024-02-02T15:44:26.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.507+0000 7f2b73e00700 1 --2- 172.21.15.1:0/3613608837 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b1336a9930 0x55b1336ab840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:44:26.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.508+0000 7f2b79c00700 1 --2- 172.21.15.1:0/3613608837 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b1336b47e0 0x55b1336bf900 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c000f80 tx=0x7f2b6c00b5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e3a01c7ebae3c78 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:26.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.509+0000 7f2b6aa00700 1 -- 172.21.15.1:0/3613608837 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b6c015cf0 con 0x55b1336b47e0 2024-02-02T15:44:26.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.509+0000 7f2b6aa00700 1 -- 172.21.15.1:0/3613608837 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b6c015e50 con 0x55b1336b47e0 2024-02-02T15:44:26.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.509+0000 7f2b6aa00700 1 -- 172.21.15.1:0/3613608837 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b6c019660 con 0x55b1336b47e0 2024-02-02T15:44:26.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.510+0000 7f2b8f7a7b80 1 -- 172.21.15.1:0/3613608837 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b1336b47e0 msgr2=0x55b1336bf900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:26.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.510+0000 7f2b8f7a7b80 1 --2- 172.21.15.1:0/3613608837 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b1336b47e0 0x55b1336bf900 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c000f80 tx=0x7f2b6c00b5d0 comp rx=0 tx=0).stop 2024-02-02T15:44:26.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.511+0000 7f2b8f7a7b80 1 -- 172.21.15.1:0/3613608837 shutdown_connections 2024-02-02T15:44:26.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.511+0000 7f2b8f7a7b80 1 --2- 172.21.15.1:0/3613608837 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b1336b47e0 0x55b1336bf900 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:26.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.511+0000 7f2b8f7a7b80 1 --2- 172.21.15.1:0/3613608837 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b1336a9010 0x55b1336bb330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:26.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.511+0000 7f2b8f7a7b80 1 --2- 172.21.15.1:0/3613608837 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b1336a9930 0x55b1336ab840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:26.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.511+0000 7f2b8f7a7b80 1 -- 172.21.15.1:0/3613608837 >> 172.21.15.1:0/3613608837 conn(0x55b133608db0 msgr2=0x55b1336ac8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:26.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.511+0000 7f2b8f7a7b80 1 -- 172.21.15.1:0/3613608837 shutdown_connections 2024-02-02T15:44:26.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.511+0000 7f2b8f7a7b80 1 -- 172.21.15.1:0/3613608837 wait complete. 2024-02-02T15:44:26.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.511+0000 7f2b8f7a7b80 1 -- start start 2024-02-02T15:44:26.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.512+0000 7f2b8f7a7b80 1 Processor -- start 2024-02-02T15:44:26.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.513+0000 7f2b8f7a7b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b1336a9010 0x55b1336b9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:26.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.513+0000 7f2b8f7a7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b1336a9930 0x55b1336b9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:26.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.513+0000 7f2b8f7a7b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b1336b47e0 0x55b13371d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:26.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.513+0000 7f2b8f7a7b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b13371da80 con 0x55b1336b47e0 2024-02-02T15:44:26.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.513+0000 7f2b8f7a7b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b13371dbc0 con 0x55b1336a9010 2024-02-02T15:44:26.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.513+0000 7f2b8f7a7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b13371dd30 con 0x55b1336a9930 2024-02-02T15:44:26.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.513+0000 7f2b8f7a7b80 10 client.0 _ll_register_callbacks cb 0x55b133598290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:44:26.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.514+0000 7f2b79200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b1336a9010 0x55b1336b9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:26.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.514+0000 7f2b73e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b1336a9930 0x55b1336b9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:26.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.514+0000 7f2b79200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b1336a9010 0x55b1336b9500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:51784/0 (socket says 172.21.15.1:51784) 2024-02-02T15:44:26.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.514+0000 7f2b79200700 1 -- 172.21.15.1:0/2794710190 learned_addr learned my addr 172.21.15.1:0/2794710190 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:44:26.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.514+0000 7f2b79200700 1 -- 172.21.15.1:0/2794710190 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b1336a9930 msgr2=0x55b1336b9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:26.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.514+0000 7f2b79200700 1 --2- 172.21.15.1:0/2794710190 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b1336a9930 0x55b1336b9a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:26.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.514+0000 7f2b79200700 1 -- 172.21.15.1:0/2794710190 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b1336b47e0 msgr2=0x55b13371d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:26.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.515+0000 7f2b79c00700 1 --2- 172.21.15.1:0/2794710190 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b1336b47e0 0x55b13371d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:26.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.515+0000 7f2b79200700 1 --2- 172.21.15.1:0/2794710190 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b1336b47e0 0x55b13371d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:26.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.516+0000 7f2b79200700 1 -- 172.21.15.1:0/2794710190 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b7410e5a0 con 0x55b1336a9010 2024-02-02T15:44:26.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.516+0000 7f2b79c00700 1 --2- 172.21.15.1:0/2794710190 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b1336b47e0 0x55b13371d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:44:26.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.517+0000 7f2b79200700 1 --2- 172.21.15.1:0/2794710190 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b1336a9010 0x55b1336b9500 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f2b64000c00 tx=0x7f2b640c9bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51f7a0e968ed5aac server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:26.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.517+0000 7f2b62a00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:44:26.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.517+0000 7f2b62a00700 1 -- 172.21.15.1:0/2794710190 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b64098370 con 0x55b1336a9010 2024-02-02T15:44:26.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.517+0000 7f2b8f7a7b80 1 -- 172.21.15.1:0/2794710190 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2b6c00be70 con 0x55b1336a9010 2024-02-02T15:44:26.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.518+0000 7f2b62a00700 1 -- 172.21.15.1:0/2794710190 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f2b640993c0 con 0x55b1336a9010 2024-02-02T15:44:26.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.519+0000 7f2b62a00700 1 -- 172.21.15.1:0/2794710190 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2b6409d1f0 con 0x55b1336a9010 2024-02-02T15:44:26.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.519+0000 7f2b8f7a7b80 10 client.30182 fetch_fsmap learned FSMap version 74 2024-02-02T15:44:26.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.519+0000 7f2b8f7a7b80 1 -- 172.21.15.1:0/2794710190 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x55b13347c3b0 con 0x55b1336a9010 2024-02-02T15:44:26.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.520+0000 7f2b62a00700 1 -- 172.21.15.1:0/2794710190 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b6c00be70 con 0x55b1336a9010 2024-02-02T15:44:26.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.520+0000 7f2b62a00700 1 -- 172.21.15.1:0/2794710190 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 74) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f2b64098980 con 0x55b1336a9010 2024-02-02T15:44:26.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.520+0000 7f2b8f7a7b80 10 client.30182 fetch_fsmap finished waiting for FSMap version 74 2024-02-02T15:44:26.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.520+0000 7f2b8f7a7b80 10 client.30182 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:44:26.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.520+0000 7f2b8f7a7b80 20 client.30182 populate_metadata read hostname 'smithi001' 2024-02-02T15:44:26.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.520+0000 7f2b8f7a7b80 1 --2- 172.21.15.1:0/2794710190 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55b13371f250 0x55b133721670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:26.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.520+0000 7f2b8f7a7b80 4 client.30182 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2024-02-02T15:44:26.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.520+0000 7f2b8f7a7b80 1 -- 172.21.15.1:0/2794710190 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b1335b9f50 con 0x55b13371f250 2024-02-02T15:44:26.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.521+0000 7f2b73e00700 1 --2- 172.21.15.1:0/2794710190 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55b13371f250 0x55b133721670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:26.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.526+0000 7f2b73e00700 1 --2- 172.21.15.1:0/2794710190 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55b13371f250 0x55b133721670 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=981c03e8ce1bc3ec server_cookie=2c1513c48735f4d1 in_seq=0 out_seq=0 2024-02-02T15:44:26.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.527+0000 7f2b62a00700 10 client.30182 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:44:26.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.527+0000 7f2b62a00700 1 -- 172.21.15.1:0/2794710190 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55b1335b9f50 con 0x55b13371f250 2024-02-02T15:44:26.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.527+0000 7f2b62a00700 10 client.30182 handle_command_reply: tid=0 2024-02-02T15:44:26.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.529+0000 7f2b8f7a7b80 1 client.30182 shutdown 2024-02-02T15:44:26.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.529+0000 7f2b8f7a7b80 1 -- 172.21.15.1:0/2794710190 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b1336a9010 msgr2=0x55b1336b9500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:26.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.529+0000 7f2b8f7a7b80 1 --2- 172.21.15.1:0/2794710190 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b1336a9010 0x55b1336b9500 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f2b64000c00 tx=0x7f2b640c9bb0 comp rx=0 tx=0).stop 2024-02-02T15:44:26.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.530+0000 7f2b8f7a7b80 1 -- 172.21.15.1:0/2794710190 shutdown_connections 2024-02-02T15:44:26.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.530+0000 7f2b8f7a7b80 1 --2- 172.21.15.1:0/2794710190 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55b13371f250 0x55b133721670 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:26.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.530+0000 7f2b8f7a7b80 1 --2- 172.21.15.1:0/2794710190 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55b1336b47e0 0x55b13371d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:26.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.530+0000 7f2b8f7a7b80 1 --2- 172.21.15.1:0/2794710190 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b1336a9010 0x55b1336b9500 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:26.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.530+0000 7f2b8f7a7b80 1 --2- 172.21.15.1:0/2794710190 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55b1336a9930 0x55b1336b9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:26.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.530+0000 7f2b8f7a7b80 1 -- 172.21.15.1:0/2794710190 >> 172.21.15.1:0/2794710190 conn(0x55b133608db0 msgr2=0x55b1336bf900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:26.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.530+0000 7f2b62a00700 0 client.30182 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:44:26.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.532+0000 7f2b8f7a7b80 1 -- 172.21.15.1:0/2794710190 shutdown_connections 2024-02-02T15:44:26.532 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.532+0000 7f2b8f7a7b80 1 -- 172.21.15.1:0/2794710190 wait complete. 2024-02-02T15:44:26.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.533+0000 7f2b8f7a7b80 20 client.30182 trim_cache size 0 max 16384 2024-02-02T15:44:26.534 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:44:26.534 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-02T15:44:26.534 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2024-02-02T15:44:26.534 INFO:teuthology.orchestra.run.smithi001.stdout: "4b35180c-5771-4e5f-8253-ac0913eb65d9": { 2024-02-02T15:44:26.534 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2024-02-02T15:44:26.534 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "4b35180c-5771-4e5f-8253-ac0913eb65d9", 2024-02-02T15:44:26.534 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,force" 2024-02-02T15:44:26.534 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:44:26.535 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:44:26.535 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:44:26.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.534+0000 7f2b7bc00700 1 -- 172.21.15.1:0/3061483113 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f2b540a0710 msgr2=0x7f2b540a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:26.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.534+0000 7f2b7bc00700 1 --2- 172.21.15.1:0/3061483113 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f2b540a0710 0x7f2b540a2bd0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f2b7419a780 tx=0x7f2b5c005fb0 comp rx=0 tx=0).stop 2024-02-02T15:44:26.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.534+0000 7f2b7bc00700 1 -- 172.21.15.1:0/3061483113 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2b74068010 msgr2=0x7f2b74199170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:26.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.534+0000 7f2b7bc00700 1 --2- 172.21.15.1:0/3061483113 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2b74068010 0x7f2b74199170 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f2b6400e9a0 tx=0x7f2b6400ecb0 comp rx=0 tx=0).stop 2024-02-02T15:44:26.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.535+0000 7f2b7bc00700 1 -- 172.21.15.1:0/3061483113 shutdown_connections 2024-02-02T15:44:26.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.535+0000 7f2b7bc00700 1 --2- 172.21.15.1:0/3061483113 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f2b540a0710 0x7f2b540a2bd0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:26.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.535+0000 7f2b7bc00700 1 --2- 172.21.15.1:0/3061483113 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2b74106ed0 0x7f2b7419db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:26.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.535+0000 7f2b7bc00700 1 --2- 172.21.15.1:0/3061483113 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2b74068010 0x7f2b74199170 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:26.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.535+0000 7f2b7bc00700 1 --2- 172.21.15.1:0/3061483113 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2b740689b0 0x7f2b741996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:26.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.535+0000 7f2b7bc00700 1 -- 172.21.15.1:0/3061483113 >> 172.21.15.1:0/3061483113 conn(0x7f2b740757d0 msgr2=0x7f2b740feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:26.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.535+0000 7f2b7bc00700 1 -- 172.21.15.1:0/3061483113 shutdown_connections 2024-02-02T15:44:26.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:26.535+0000 7f2b7bc00700 1 -- 172.21.15.1:0/3061483113 wait complete. 2024-02-02T15:44:26.565 INFO:tasks.cephfs.filesystem:scrub status for tag:4b35180c-5771-4e5f-8253-ac0913eb65d9 - {'path': '/', 'tag': '4b35180c-5771-4e5f-8253-ac0913eb65d9', 'options': 'recursive,force'} 2024-02-02T15:44:26.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:44:26 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:44:26.669 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:44:26 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:44:26.993 INFO:tasks.workunit.client.0.smithi001.stdout: 10 133189 28.72 MB/sec execute 119 sec latency 88.051 ms 2024-02-02T15:44:27.616 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:44:27 smithi071 ceph-mon[44707]: scrub queued for path: / 2024-02-02T15:44:27.616 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:44:27 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:44:27.616 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:44:27 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:44:27.616 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:44:27 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:44:27.616 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:44:27 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:44:27.616 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:44:27 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:44:27.616 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:44:27 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:44:27.616 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:44:27 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:44:27.617 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:44:27 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:44:27.617 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:44:27 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:44:27.617 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:44:27 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:44:27.617 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:44:27 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:44:27.617 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:44:27 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:44:27.617 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:44:27 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:44:27.617 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:44:27 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:44:27.617 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:44:27 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:44:27.617 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:44:27 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:44:27.617 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:44:27 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:44:27.617 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:44:27 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:44:27.617 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:44:27 smithi071 ceph-mon[44707]: pgmap v548: 129 pgs: 129 active+clean; 1.9 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.7 MiB/s wr, 521 op/s 2024-02-02T15:44:27.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:44:27 smithi142 ceph-mon[45728]: scrub queued for path: / 2024-02-02T15:44:27.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:44:27 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:44:27.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:44:27 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:44:27.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:44:27 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:44:27.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:44:27 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:44:27.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:44:27 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:44:27.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:44:27 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:44:27.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:44:27 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:44:27.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:44:27 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:44:27.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:44:27 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:44:27.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:44:27 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:44:27.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:44:27 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:44:27.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:44:27 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:44:27.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:44:27 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:44:27.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:44:27 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:44:27.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:44:27 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:44:27.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:44:27 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:44:27.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:44:27 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:44:27.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:44:27 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:44:27.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:44:27 smithi142 ceph-mon[45728]: pgmap v548: 129 pgs: 129 active+clean; 1.9 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.7 MiB/s wr, 521 op/s 2024-02-02T15:44:27.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:44:27 smithi001 ceph-mon[35470]: scrub queued for path: / 2024-02-02T15:44:27.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:44:27 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:44:27.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:44:27 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:44:27.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:44:27 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:44:27.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:44:27 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:44:27.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:44:27 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:44:27.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:44:27 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:44:27.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:44:27 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:44:27.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:44:27 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:44:27.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:44:27 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:44:27.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:44:27 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:44:27.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:44:27 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:44:27.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:44:27 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:44:27.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:44:27 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:44:27.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:44:27 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:44:27.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:44:27 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:44:27.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:44:27 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:44:27.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:44:27 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:44:27.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:44:27 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:44:27.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:44:27 smithi001 ceph-mon[35470]: pgmap v548: 129 pgs: 129 active+clean; 1.9 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.7 MiB/s wr, 521 op/s 2024-02-02T15:44:27.994 INFO:tasks.workunit.client.0.smithi001.stdout: 10 133935 28.83 MB/sec execute 120 sec latency 65.259 ms 2024-02-02T15:44:28.994 INFO:tasks.workunit.client.0.smithi001.stdout: 10 134538 28.90 MB/sec execute 121 sec latency 65.295 ms 2024-02-02T15:44:29.891 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:44:29 smithi071 ceph-mon[44707]: pgmap v549: 129 pgs: 129 active+clean; 1.9 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.5 MiB/s wr, 511 op/s 2024-02-02T15:44:29.892 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:44:29 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:44:29.922 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:44:29 smithi142 ceph-mon[45728]: pgmap v549: 129 pgs: 129 active+clean; 1.9 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.5 MiB/s wr, 511 op/s 2024-02-02T15:44:29.922 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:44:29 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:44:29.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:44:29 smithi001 ceph-mon[35470]: pgmap v549: 129 pgs: 129 active+clean; 1.9 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.5 MiB/s wr, 511 op/s 2024-02-02T15:44:29.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:44:29 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:44:29.994 INFO:tasks.workunit.client.0.smithi001.stdout: 10 135063 28.93 MB/sec execute 122 sec latency 76.815 ms 2024-02-02T15:44:30.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:44:30 smithi071 ceph-mon[44707]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:44:30.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:44:30 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:44:30.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:44:30 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:44:30.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:44:30 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:44:30.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:44:30 smithi142 ceph-mon[45728]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:44:30.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:44:30 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:44:30.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:44:30 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:44:30.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:44:30 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:44:30.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:44:30 smithi001 ceph-mon[35470]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:44:30.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:44:30 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:44:30.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:44:30 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:44:30.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:44:30 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:44:30.994 INFO:tasks.workunit.client.0.smithi001.stdout: 10 135500 28.86 MB/sec execute 123 sec latency 102.423 ms 2024-02-02T15:44:31.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:44:31 smithi142 ceph-mon[45728]: pgmap v550: 129 pgs: 129 active+clean; 1.9 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 6.5 MiB/s wr, 393 op/s 2024-02-02T15:44:31.923 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:44:31 smithi071 ceph-mon[44707]: pgmap v550: 129 pgs: 129 active+clean; 1.9 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 6.5 MiB/s wr, 393 op/s 2024-02-02T15:44:31.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:44:31 smithi001 ceph-mon[35470]: pgmap v550: 129 pgs: 129 active+clean; 1.9 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 6.5 MiB/s wr, 393 op/s 2024-02-02T15:44:31.994 INFO:tasks.workunit.client.0.smithi001.stdout: 10 135997 28.80 MB/sec execute 124 sec latency 113.826 ms 2024-02-02T15:44:32.994 INFO:tasks.workunit.client.0.smithi001.stdout: 10 136623 28.78 MB/sec execute 125 sec latency 71.629 ms 2024-02-02T15:44:33.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:44:33 smithi142 ceph-mon[45728]: pgmap v551: 129 pgs: 129 active+clean; 1.9 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 9.2 MiB/s wr, 559 op/s 2024-02-02T15:44:33.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:44:33 smithi001 ceph-mon[35470]: pgmap v551: 129 pgs: 129 active+clean; 1.9 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 9.2 MiB/s wr, 559 op/s 2024-02-02T15:44:33.958 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:44:33 smithi071 ceph-mon[44707]: pgmap v551: 129 pgs: 129 active+clean; 1.9 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 9.2 MiB/s wr, 559 op/s 2024-02-02T15:44:33.995 INFO:tasks.workunit.client.0.smithi001.stdout: 10 137337 28.86 MB/sec execute 126 sec latency 92.822 ms 2024-02-02T15:44:34.994 INFO:tasks.workunit.client.0.smithi001.stdout: 10 138071 29.05 MB/sec execute 127 sec latency 87.301 ms 2024-02-02T15:44:35.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:44:35 smithi071 ceph-mon[44707]: pgmap v552: 129 pgs: 129 active+clean; 1.9 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.0 MiB/s wr, 438 op/s 2024-02-02T15:44:35.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:44:35 smithi142 ceph-mon[45728]: pgmap v552: 129 pgs: 129 active+clean; 1.9 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.0 MiB/s wr, 438 op/s 2024-02-02T15:44:35.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:44:35 smithi001 ceph-mon[35470]: pgmap v552: 129 pgs: 129 active+clean; 1.9 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.0 MiB/s wr, 438 op/s 2024-02-02T15:44:35.994 INFO:tasks.workunit.client.0.smithi001.stdout: 10 138491 29.01 MB/sec execute 128 sec latency 174.121 ms 2024-02-02T15:44:36.995 INFO:tasks.workunit.client.0.smithi001.stdout: 10 138621 28.83 MB/sec execute 129 sec latency 471.459 ms 2024-02-02T15:44:37.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:44:36 smithi071 ceph-mon[44707]: pgmap v553: 129 pgs: 129 active+clean; 1.9 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 8.1 MiB/s wr, 516 op/s 2024-02-02T15:44:37.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:44:36 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:44:37.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:44:36 smithi142 ceph-mon[45728]: pgmap v553: 129 pgs: 129 active+clean; 1.9 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 8.1 MiB/s wr, 516 op/s 2024-02-02T15:44:37.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:44:36 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:44:37.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:44:36 smithi001 ceph-mon[35470]: pgmap v553: 129 pgs: 129 active+clean; 1.9 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 8.1 MiB/s wr, 516 op/s 2024-02-02T15:44:37.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:44:36 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:44:37.994 INFO:tasks.workunit.client.0.smithi001.stdout: 10 138777 28.65 MB/sec execute 130 sec latency 431.112 ms 2024-02-02T15:44:38.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:44:37 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:44:38.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:44:37 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:44:38.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:44:37 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:44:38.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:44:37 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:44:38.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:44:37 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:44:38.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:44:37 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:44:38.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:44:37 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:44:38.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:44:37 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:44:38.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:44:37 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:44:38.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:44:37 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:44:38.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:44:37 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:44:38.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:44:37 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:44:38.994 INFO:tasks.workunit.client.0.smithi001.stdout: 10 139014 28.54 MB/sec execute 131 sec latency 135.620 ms 2024-02-02T15:44:39.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:44:38 smithi071 ceph-mon[44707]: pgmap v554: 129 pgs: 129 active+clean; 1.9 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.5 MiB/s wr, 457 op/s 2024-02-02T15:44:39.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:44:38 smithi142 ceph-mon[45728]: pgmap v554: 129 pgs: 129 active+clean; 1.9 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.5 MiB/s wr, 457 op/s 2024-02-02T15:44:39.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:44:38 smithi001 ceph-mon[35470]: pgmap v554: 129 pgs: 129 active+clean; 1.9 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.5 MiB/s wr, 457 op/s 2024-02-02T15:44:39.995 INFO:tasks.workunit.client.0.smithi001.stdout: 10 139557 28.54 MB/sec execute 132 sec latency 105.789 ms 2024-02-02T15:44:40.995 INFO:tasks.workunit.client.0.smithi001.stdout: 10 140175 28.46 MB/sec execute 133 sec latency 100.061 ms 2024-02-02T15:44:41.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:44:41 smithi071 ceph-mon[44707]: pgmap v555: 129 pgs: 129 active+clean; 1.9 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 7.3 MiB/s wr, 386 op/s 2024-02-02T15:44:41.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:44:41 smithi142 ceph-mon[45728]: pgmap v555: 129 pgs: 129 active+clean; 1.9 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 7.3 MiB/s wr, 386 op/s 2024-02-02T15:44:41.942 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:44:41 smithi001 ceph-mon[35470]: pgmap v555: 129 pgs: 129 active+clean; 1.9 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 7.3 MiB/s wr, 386 op/s 2024-02-02T15:44:41.995 INFO:tasks.workunit.client.0.smithi001.stdout: 10 140950 28.68 MB/sec execute 134 sec latency 69.235 ms 2024-02-02T15:44:42.868 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:44:42 smithi071 ceph-mon[44707]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:44:42.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:44:42 smithi142 ceph-mon[45728]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:44:42.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:44:42 smithi001 ceph-mon[35470]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:44:42.995 INFO:tasks.workunit.client.0.smithi001.stdout: 10 141003 28.49 MB/sec execute 135 sec latency 925.405 ms 2024-02-02T15:44:43.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:44:43 smithi071 ceph-mon[44707]: pgmap v556: 129 pgs: 129 active+clean; 1.9 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 590 op/s 2024-02-02T15:44:43.923 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:44:43 smithi142 ceph-mon[45728]: pgmap v556: 129 pgs: 129 active+clean; 1.9 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 590 op/s 2024-02-02T15:44:43.942 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:44:43 smithi001 ceph-mon[35470]: pgmap v556: 129 pgs: 129 active+clean; 1.9 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 590 op/s 2024-02-02T15:44:43.995 INFO:tasks.workunit.client.0.smithi001.stdout: 10 141163 28.33 MB/sec execute 136 sec latency 1175.560 ms 2024-02-02T15:44:44.995 INFO:tasks.workunit.client.0.smithi001.stdout: 10 141433 28.25 MB/sec execute 137 sec latency 169.909 ms 2024-02-02T15:44:45.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:44:45 smithi071 ceph-mon[44707]: pgmap v557: 129 pgs: 129 active+clean; 1.9 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 12 MiB/s wr, 531 op/s 2024-02-02T15:44:45.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:44:45 smithi142 ceph-mon[45728]: pgmap v557: 129 pgs: 129 active+clean; 1.9 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 12 MiB/s wr, 531 op/s 2024-02-02T15:44:45.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:44:45 smithi001 ceph-mon[35470]: pgmap v557: 129 pgs: 129 active+clean; 1.9 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 12 MiB/s wr, 531 op/s 2024-02-02T15:44:45.995 INFO:tasks.workunit.client.0.smithi001.stdout: 10 141689 28.14 MB/sec execute 138 sec latency 156.121 ms 2024-02-02T15:44:46.996 INFO:tasks.workunit.client.0.smithi001.stdout: 10 142140 28.10 MB/sec execute 139 sec latency 111.491 ms 2024-02-02T15:44:47.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:44:47 smithi071 ceph-mon[44707]: pgmap v558: 129 pgs: 129 active+clean; 1.9 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 18 MiB/s wr, 712 op/s 2024-02-02T15:44:47.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:44:47 smithi071 ceph-mon[44707]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:44:47.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:44:47 smithi142 ceph-mon[45728]: pgmap v558: 129 pgs: 129 active+clean; 1.9 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 18 MiB/s wr, 712 op/s 2024-02-02T15:44:47.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:44:47 smithi142 ceph-mon[45728]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:44:47.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:44:47 smithi001 ceph-mon[35470]: pgmap v558: 129 pgs: 129 active+clean; 1.9 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 18 MiB/s wr, 712 op/s 2024-02-02T15:44:47.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:44:47 smithi001 ceph-mon[35470]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:44:47.996 INFO:tasks.workunit.client.0.smithi001.stdout: 10 142607 28.08 MB/sec execute 140 sec latency 100.293 ms 2024-02-02T15:44:48.996 INFO:tasks.workunit.client.0.smithi001.stdout: 10 143120 28.07 MB/sec execute 141 sec latency 75.589 ms 2024-02-02T15:44:49.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:44:49 smithi071 ceph-mon[44707]: pgmap v559: 129 pgs: 129 active+clean; 1.9 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 19 MiB/s wr, 749 op/s 2024-02-02T15:44:49.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:44:49 smithi142 ceph-mon[45728]: pgmap v559: 129 pgs: 129 active+clean; 1.9 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 19 MiB/s wr, 749 op/s 2024-02-02T15:44:49.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:44:49 smithi001 ceph-mon[35470]: pgmap v559: 129 pgs: 129 active+clean; 1.9 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 19 MiB/s wr, 749 op/s 2024-02-02T15:44:49.996 INFO:tasks.workunit.client.0.smithi001.stdout: 10 143706 28.07 MB/sec execute 142 sec latency 92.401 ms 2024-02-02T15:44:50.996 INFO:tasks.workunit.client.0.smithi001.stdout: 10 144325 28.10 MB/sec execute 143 sec latency 87.757 ms 2024-02-02T15:44:51.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:44:51 smithi071 ceph-mon[44707]: pgmap v560: 129 pgs: 129 active+clean; 1.9 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 17 MiB/s wr, 677 op/s 2024-02-02T15:44:51.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:44:51 smithi142 ceph-mon[45728]: pgmap v560: 129 pgs: 129 active+clean; 1.9 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 17 MiB/s wr, 677 op/s 2024-02-02T15:44:51.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:44:51 smithi001 ceph-mon[35470]: pgmap v560: 129 pgs: 129 active+clean; 1.9 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 17 MiB/s wr, 677 op/s 2024-02-02T15:44:51.996 INFO:tasks.workunit.client.0.smithi001.stdout: 10 144912 28.14 MB/sec execute 144 sec latency 124.246 ms 2024-02-02T15:44:52.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:44:52 smithi142 ceph-mon[45728]: pgmap v561: 129 pgs: 129 active+clean; 2.0 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 19 MiB/s wr, 809 op/s 2024-02-02T15:44:52.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:44:52 smithi001 ceph-mon[35470]: pgmap v561: 129 pgs: 129 active+clean; 2.0 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 19 MiB/s wr, 809 op/s 2024-02-02T15:44:52.996 INFO:tasks.workunit.client.0.smithi001.stdout: 10 145431 28.17 MB/sec execute 145 sec latency 86.487 ms 2024-02-02T15:44:53.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:44:52 smithi071 ceph-mon[44707]: pgmap v561: 129 pgs: 129 active+clean; 2.0 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 19 MiB/s wr, 809 op/s 2024-02-02T15:44:53.996 INFO:tasks.workunit.client.0.smithi001.stdout: 10 145993 28.17 MB/sec execute 146 sec latency 90.436 ms 2024-02-02T15:44:54.997 INFO:tasks.workunit.client.0.smithi001.stdout: 10 146521 28.14 MB/sec execute 147 sec latency 75.338 ms 2024-02-02T15:44:55.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:44:55 smithi071 ceph-mon[44707]: pgmap v562: 129 pgs: 129 active+clean; 2.0 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 14 MiB/s wr, 655 op/s 2024-02-02T15:44:55.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:44:55 smithi142 ceph-mon[45728]: pgmap v562: 129 pgs: 129 active+clean; 2.0 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 14 MiB/s wr, 655 op/s 2024-02-02T15:44:55.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:44:55 smithi001 ceph-mon[35470]: pgmap v562: 129 pgs: 129 active+clean; 2.0 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 14 MiB/s wr, 655 op/s 2024-02-02T15:44:55.997 INFO:tasks.workunit.client.0.smithi001.stdout: 10 147128 28.20 MB/sec execute 148 sec latency 75.124 ms 2024-02-02T15:44:56.566 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-02T15:44:56.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.829+0000 7faf05800700 1 -- 172.21.15.1:0/879855451 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf00108320 msgr2=0x7faf0010fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:56.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.829+0000 7faf05800700 1 --2- 172.21.15.1:0/879855451 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf00108320 0x7faf0010fea0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7faef800b0a0 tx=0x7faef800b3b0 comp rx=0 tx=0).stop 2024-02-02T15:44:56.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.830+0000 7faf05800700 1 -- 172.21.15.1:0/879855451 shutdown_connections 2024-02-02T15:44:56.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.830+0000 7faf05800700 1 --2- 172.21.15.1:0/879855451 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf00108320 0x7faf0010fea0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:56.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.830+0000 7faf05800700 1 --2- 172.21.15.1:0/879855451 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf00103200 0x7faf001035e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:56.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.830+0000 7faf05800700 1 --2- 172.21.15.1:0/879855451 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf00103bb0 0x7faf00107bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:56.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.830+0000 7faf05800700 1 -- 172.21.15.1:0/879855451 >> 172.21.15.1:0/879855451 conn(0x7faf000fea40 msgr2=0x7faf00100e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:56.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.830+0000 7faf05800700 1 -- 172.21.15.1:0/879855451 shutdown_connections 2024-02-02T15:44:56.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.830+0000 7faf05800700 1 -- 172.21.15.1:0/879855451 wait complete. 2024-02-02T15:44:56.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.831+0000 7faf05800700 1 Processor -- start 2024-02-02T15:44:56.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.831+0000 7faf05800700 1 -- start start 2024-02-02T15:44:56.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.832+0000 7faf05800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf00103200 0x7faf001991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:56.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.832+0000 7faf05800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf00108320 0x7faf001996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:56.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.832+0000 7faf05800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf0019db40 0x7faf0019dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:56.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.832+0000 7faf05800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf0019e5c0 con 0x7faf00108320 2024-02-02T15:44:56.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.832+0000 7faf05800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf0019e730 con 0x7faf00103200 2024-02-02T15:44:56.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.832+0000 7faf05800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf0019e8f0 con 0x7faf0019db40 2024-02-02T15:44:56.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.832+0000 7faefea00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf00103200 0x7faf001991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:56.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.832+0000 7faefe000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf00108320 0x7faf001996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:56.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.832+0000 7faefea00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf00103200 0x7faf001991a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:35140/0 (socket says 172.21.15.1:35140) 2024-02-02T15:44:56.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.832+0000 7faefea00700 1 -- 172.21.15.1:0/3731174182 learned_addr learned my addr 172.21.15.1:0/3731174182 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:44:56.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.833+0000 7faeff400700 1 --2- 172.21.15.1:0/3731174182 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf0019db40 0x7faf0019dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:56.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.833+0000 7faefea00700 1 -- 172.21.15.1:0/3731174182 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf0019db40 msgr2=0x7faf0019dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:56.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.833+0000 7faefea00700 1 --2- 172.21.15.1:0/3731174182 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf0019db40 0x7faf0019dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:56.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.833+0000 7faefea00700 1 -- 172.21.15.1:0/3731174182 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf00108320 msgr2=0x7faf001996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:56.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.833+0000 7faefea00700 1 --2- 172.21.15.1:0/3731174182 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf00108320 0x7faf001996e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:56.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.833+0000 7faefea00700 1 -- 172.21.15.1:0/3731174182 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faee8009520 con 0x7faf00103200 2024-02-02T15:44:56.834 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.833+0000 7faefe000700 1 --2- 172.21.15.1:0/3731174182 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf00108320 0x7faf001996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:44:56.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.834+0000 7faefea00700 1 --2- 172.21.15.1:0/3731174182 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf00103200 0x7faf001991a0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7faef000e9a0 tx=0x7faef000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:56.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.834+0000 7faef7400700 1 -- 172.21.15.1:0/3731174182 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef0009790 con 0x7faf00103200 2024-02-02T15:44:56.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.834+0000 7faf05800700 1 -- 172.21.15.1:0/3731174182 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faef8009d00 con 0x7faf00103200 2024-02-02T15:44:56.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.834+0000 7faef7400700 1 -- 172.21.15.1:0/3731174182 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faef0010430 con 0x7faf00103200 2024-02-02T15:44:56.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.834+0000 7faef7400700 1 -- 172.21.15.1:0/3731174182 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef0018680 con 0x7faf00103200 2024-02-02T15:44:56.835 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.834+0000 7faf05800700 1 -- 172.21.15.1:0/3731174182 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf001a61a0 con 0x7faf00103200 2024-02-02T15:44:56.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.838+0000 7faef7400700 1 -- 172.21.15.1:0/3731174182 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7faef00105a0 con 0x7faf00103200 2024-02-02T15:44:56.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.839+0000 7faf193a6b80 1 Processor -- start 2024-02-02T15:44:56.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.839+0000 7faf193a6b80 1 -- start start 2024-02-02T15:44:56.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.839+0000 7faf193a6b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1a1a6d010 0x55a1a1a6d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:56.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.839+0000 7faf193a6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a1a1a6d930 0x55a1a1a782a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:56.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.840+0000 7faf193a6b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1a1a787e0 0x55a1a1a7abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:56.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.840+0000 7faf193a6b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1a19e14c0 con 0x55a1a1a6d010 2024-02-02T15:44:56.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.840+0000 7faf193a6b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1a19c5aa0 con 0x55a1a1a787e0 2024-02-02T15:44:56.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.840+0000 7faf193a6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1a19c3460 con 0x55a1a1a6d930 2024-02-02T15:44:56.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.840+0000 7faeff400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1a1a6d010 0x55a1a1a6d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:56.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.840+0000 7faefe000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1a1a787e0 0x55a1a1a7abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:56.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.840+0000 7faefea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a1a1a6d930 0x55a1a1a782a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:56.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.840+0000 7faefe000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1a1a787e0 0x55a1a1a7abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:35150/0 (socket says 172.21.15.1:35150) 2024-02-02T15:44:56.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.840+0000 7faefe000700 1 -- 172.21.15.1:0/1646585103 learned_addr learned my addr 172.21.15.1:0/1646585103 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:44:56.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.840+0000 7faefe000700 1 -- 172.21.15.1:0/1646585103 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a1a1a6d930 msgr2=0x55a1a1a782a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:56.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.840+0000 7faefe000700 1 --2- 172.21.15.1:0/1646585103 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a1a1a6d930 0x55a1a1a782a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:56.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.840+0000 7faefe000700 1 -- 172.21.15.1:0/1646585103 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1a1a6d010 msgr2=0x55a1a1a6d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:56.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.840+0000 7faefe000700 1 --2- 172.21.15.1:0/1646585103 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1a1a6d010 0x55a1a1a6d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:56.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.840+0000 7faefe000700 1 -- 172.21.15.1:0/1646585103 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a1a18403b0 con 0x55a1a1a787e0 2024-02-02T15:44:56.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.841+0000 7faef7400700 1 --2- 172.21.15.1:0/3731174182 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7faee00a0710 0x7faee00a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:56.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.841+0000 7faefe000700 1 --2- 172.21.15.1:0/1646585103 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1a1a787e0 0x55a1a1a7abd0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7faee8011660 tx=0x7faee8011970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8727270e9a500bde server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:56.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.841+0000 7faef7400700 1 -- 172.21.15.1:0/3731174182 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 9453+0+0 (secure 0 0 0) 0x7faef0014070 con 0x7faf00103200 2024-02-02T15:44:56.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.841+0000 7faeefe00700 1 -- 172.21.15.1:0/1646585103 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee8004830 con 0x55a1a1a787e0 2024-02-02T15:44:56.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.841+0000 7faeefe00700 1 -- 172.21.15.1:0/1646585103 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faee8004e50 con 0x55a1a1a787e0 2024-02-02T15:44:56.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.841+0000 7faeefe00700 1 -- 172.21.15.1:0/1646585103 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee800e790 con 0x55a1a1a787e0 2024-02-02T15:44:56.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.841+0000 7faeff400700 1 --2- 172.21.15.1:0/3731174182 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7faee00a0710 0x7faee00a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:56.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.842+0000 7faf193a6b80 1 -- 172.21.15.1:0/1646585103 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1a1a787e0 msgr2=0x55a1a1a7abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:56.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.842+0000 7faf193a6b80 1 --2- 172.21.15.1:0/1646585103 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1a1a787e0 0x55a1a1a7abd0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7faee8011660 tx=0x7faee8011970 comp rx=0 tx=0).stop 2024-02-02T15:44:56.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.842+0000 7faf193a6b80 1 -- 172.21.15.1:0/1646585103 shutdown_connections 2024-02-02T15:44:56.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.842+0000 7faf193a6b80 1 --2- 172.21.15.1:0/1646585103 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1a1a787e0 0x55a1a1a7abd0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:56.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.842+0000 7faf193a6b80 1 --2- 172.21.15.1:0/1646585103 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1a1a6d010 0x55a1a1a6d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:56.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.842+0000 7faf193a6b80 1 --2- 172.21.15.1:0/1646585103 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a1a1a6d930 0x55a1a1a782a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:56.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.842+0000 7faf193a6b80 1 -- 172.21.15.1:0/1646585103 >> 172.21.15.1:0/1646585103 conn(0x55a1a19ccdb0 msgr2=0x55a1a1a693b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:56.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.844+0000 7faeff400700 1 --2- 172.21.15.1:0/3731174182 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7faee00a0710 0x7faee00a2bd0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7faef80062a0 tx=0x7faef8006210 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:56.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.844+0000 7faf193a6b80 1 -- 172.21.15.1:0/1646585103 shutdown_connections 2024-02-02T15:44:56.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.845+0000 7faf193a6b80 1 -- 172.21.15.1:0/1646585103 wait complete. 2024-02-02T15:44:56.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.845+0000 7faf193a6b80 1 -- start start 2024-02-02T15:44:56.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.846+0000 7faf193a6b80 1 Processor -- start 2024-02-02T15:44:56.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.847+0000 7faf193a6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a1a1a6d010 0x55a1a1a78230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:56.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.847+0000 7faf193a6b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1a1a6d930 0x55a1a1a75440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:56.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.847+0000 7faf193a6b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1a1a787e0 0x55a1a1a75b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:56.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.847+0000 7faf193a6b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1a19bed50 con 0x55a1a1a6d930 2024-02-02T15:44:56.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.847+0000 7faf193a6b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1a195ad80 con 0x55a1a1a787e0 2024-02-02T15:44:56.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.847+0000 7faf193a6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1a195aa20 con 0x55a1a1a6d010 2024-02-02T15:44:56.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.847+0000 7faf193a6b80 10 client.0 _ll_register_callbacks cb 0x55a1a195c290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:44:56.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.847+0000 7faeff400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1a1a787e0 0x55a1a1a75b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:56.848 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.847+0000 7faeff400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1a1a787e0 0x55a1a1a75b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:35158/0 (socket says 172.21.15.1:35158) 2024-02-02T15:44:56.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.847+0000 7faeff400700 1 -- 172.21.15.1:0/347183126 learned_addr learned my addr 172.21.15.1:0/347183126 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:44:56.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.847+0000 7faefea00700 1 --2- 172.21.15.1:0/347183126 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a1a1a6d010 0x55a1a1a78230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:56.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.848+0000 7faefe000700 1 --2- 172.21.15.1:0/347183126 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1a1a6d930 0x55a1a1a75440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:56.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.848+0000 7faefea00700 1 -- 172.21.15.1:0/347183126 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1a1a787e0 msgr2=0x55a1a1a75b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:56.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.848+0000 7faefea00700 1 --2- 172.21.15.1:0/347183126 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1a1a787e0 0x55a1a1a75b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:56.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.848+0000 7faefea00700 1 -- 172.21.15.1:0/347183126 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1a1a6d930 msgr2=0x55a1a1a75440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:56.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.848+0000 7faefea00700 1 --2- 172.21.15.1:0/347183126 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1a1a6d930 0x55a1a1a75440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:56.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.848+0000 7faefea00700 1 -- 172.21.15.1:0/347183126 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faee8009e20 con 0x55a1a1a6d010 2024-02-02T15:44:56.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.849+0000 7faefea00700 1 --2- 172.21.15.1:0/347183126 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a1a1a6d010 0x55a1a1a78230 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7faef00c9610 tx=0x7faef0095720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38462989ab8a5cea server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:56.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.849+0000 7faeecc00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:44:56.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.849+0000 7faeecc00700 1 -- 172.21.15.1:0/347183126 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef00ce2d0 con 0x55a1a1a6d010 2024-02-02T15:44:56.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.849+0000 7faeecc00700 1 -- 172.21.15.1:0/347183126 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9453+0+0 (secure 0 0 0) 0x7faef0099ae0 con 0x55a1a1a6d010 2024-02-02T15:44:56.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.849+0000 7faf193a6b80 1 -- 172.21.15.1:0/347183126 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a1a1a610f0 con 0x55a1a1a6d010 2024-02-02T15:44:56.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.849+0000 7faeecc00700 1 -- 172.21.15.1:0/347183126 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) v2 ==== 24+0+0 (secure 0 0 0) 0x7faef00acea0 con 0x55a1a1a6d010 2024-02-02T15:44:56.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.849+0000 7faf193a6b80 10 client.30055 fetch_fsmap learned FSMap version 77 2024-02-02T15:44:56.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.849+0000 7faf193a6b80 1 -- 172.21.15.1:0/347183126 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=4+}) v3 -- 0x55a1a17ecb60 con 0x55a1a1a6d010 2024-02-02T15:44:56.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.852+0000 7faeecc00700 1 -- 172.21.15.1:0/347183126 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a1a1a610f0 con 0x55a1a1a6d010 2024-02-02T15:44:56.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.852+0000 7faeecc00700 1 -- 172.21.15.1:0/347183126 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 77) v1 ==== 7502+0+0 (secure 0 0 0) 0x7faef0082be0 con 0x55a1a1a6d010 2024-02-02T15:44:56.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.853+0000 7faf193a6b80 10 client.30055 fetch_fsmap finished waiting for FSMap version 77 2024-02-02T15:44:56.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.853+0000 7faf193a6b80 10 client.30055 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:44:56.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.853+0000 7faf193a6b80 20 client.30055 populate_metadata read hostname 'smithi001' 2024-02-02T15:44:56.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.853+0000 7faf193a6b80 1 --2- 172.21.15.1:0/347183126 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55a1a1a880f0 0x55a1a1a8a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:56.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.853+0000 7faf193a6b80 4 client.30055 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:44:56.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.853+0000 7faf193a6b80 1 -- 172.21.15.1:0/347183126 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a1a19b7dd0 con 0x55a1a1a880f0 2024-02-02T15:44:56.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.853+0000 7faefe000700 1 --2- 172.21.15.1:0/347183126 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55a1a1a880f0 0x55a1a1a8a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:56.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.854+0000 7faefe000700 1 --2- 172.21.15.1:0/347183126 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55a1a1a880f0 0x55a1a1a8a510 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f66aa7584e7a77fb server_cookie=daa404e021ebbb28 in_seq=0 out_seq=0 2024-02-02T15:44:56.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.854+0000 7faeecc00700 10 client.30055 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:44:56.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.854+0000 7faeecc00700 1 -- 172.21.15.1:0/347183126 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55a1a19b7dd0 con 0x55a1a1a880f0 2024-02-02T15:44:56.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.854+0000 7faeecc00700 10 client.30055 handle_command_reply: tid=0 2024-02-02T15:44:56.854 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.855+0000 7faf193a6b80 1 client.30055 shutdown 2024-02-02T15:44:56.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.855+0000 7faf193a6b80 1 -- 172.21.15.1:0/347183126 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a1a1a6d010 msgr2=0x55a1a1a78230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:56.855 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.855+0000 7faf193a6b80 1 --2- 172.21.15.1:0/347183126 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a1a1a6d010 0x55a1a1a78230 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7faef00c9610 tx=0x7faef0095720 comp rx=0 tx=0).stop 2024-02-02T15:44:56.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.855+0000 7faf193a6b80 1 -- 172.21.15.1:0/347183126 shutdown_connections 2024-02-02T15:44:56.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.855+0000 7faf193a6b80 1 --2- 172.21.15.1:0/347183126 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55a1a1a880f0 0x55a1a1a8a510 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:56.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.855+0000 7faf193a6b80 1 --2- 172.21.15.1:0/347183126 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1a1a787e0 0x55a1a1a75b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:56.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.855+0000 7faf193a6b80 1 --2- 172.21.15.1:0/347183126 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a1a1a6d010 0x55a1a1a78230 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:56.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.855+0000 7faf193a6b80 1 --2- 172.21.15.1:0/347183126 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1a1a6d930 0x55a1a1a75440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:56.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.855+0000 7faf193a6b80 1 -- 172.21.15.1:0/347183126 >> 172.21.15.1:0/347183126 conn(0x55a1a19ccdb0 msgr2=0x55a1a1a68fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:56.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.855+0000 7faeecc00700 0 client.30055 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:44:56.856 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.855+0000 7faf193a6b80 1 -- 172.21.15.1:0/347183126 shutdown_connections 2024-02-02T15:44:56.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.855+0000 7faf193a6b80 1 -- 172.21.15.1:0/347183126 wait complete. 2024-02-02T15:44:56.857 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.856+0000 7faf193a6b80 20 client.30055 trim_cache size 0 max 16384 2024-02-02T15:44:56.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.882+0000 7faf193a6b80 1 Processor -- start 2024-02-02T15:44:56.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.882+0000 7faf193a6b80 1 -- start start 2024-02-02T15:44:56.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.882+0000 7faf193a6b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1a1a6d010 0x55a1a1a77d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:56.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.882+0000 7faf193a6b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1a1a6d930 0x55a1a1a6f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:56.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.882+0000 7faf193a6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a1a1a787e0 0x55a1a1a6fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:56.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.882+0000 7faf193a6b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1a18426a0 con 0x55a1a1a6d010 2024-02-02T15:44:56.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.882+0000 7faf193a6b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1a1a702c0 con 0x55a1a1a6d930 2024-02-02T15:44:56.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.882+0000 7faf193a6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1a1a6dd10 con 0x55a1a1a787e0 2024-02-02T15:44:56.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.883+0000 7faefe000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1a1a6d930 0x55a1a1a6f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:56.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.883+0000 7faeff400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a1a1a787e0 0x55a1a1a6fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:56.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.883+0000 7faeff400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a1a1a787e0 0x55a1a1a6fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:50392/0 (socket says 172.21.15.1:50392) 2024-02-02T15:44:56.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.883+0000 7faeff400700 1 -- 172.21.15.1:0/2059521923 learned_addr learned my addr 172.21.15.1:0/2059521923 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:44:56.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.883+0000 7faefea00700 1 --2- 172.21.15.1:0/2059521923 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1a1a6d010 0x55a1a1a77d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:56.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.883+0000 7faefe000700 1 -- 172.21.15.1:0/2059521923 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a1a1a787e0 msgr2=0x55a1a1a6fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:56.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.883+0000 7faefe000700 1 --2- 172.21.15.1:0/2059521923 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a1a1a787e0 0x55a1a1a6fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:56.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.883+0000 7faefe000700 1 -- 172.21.15.1:0/2059521923 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1a1a6d010 msgr2=0x55a1a1a77d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:56.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.883+0000 7faefe000700 1 --2- 172.21.15.1:0/2059521923 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1a1a6d010 0x55a1a1a77d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:56.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.883+0000 7faefe000700 1 -- 172.21.15.1:0/2059521923 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a1a1984ea0 con 0x55a1a1a6d930 2024-02-02T15:44:56.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.884+0000 7faefe000700 1 --2- 172.21.15.1:0/2059521923 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1a1a6d930 0x55a1a1a6f840 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7faee8009230 tx=0x7faee8005600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=800154372e7ccbbe server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:56.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.884+0000 7faef4c00700 1 -- 172.21.15.1:0/2059521923 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee8005670 con 0x55a1a1a6d930 2024-02-02T15:44:56.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.884+0000 7faef4c00700 1 -- 172.21.15.1:0/2059521923 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faee8028e30 con 0x55a1a1a6d930 2024-02-02T15:44:56.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.884+0000 7faef4c00700 1 -- 172.21.15.1:0/2059521923 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faee8003900 con 0x55a1a1a6d930 2024-02-02T15:44:56.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.885+0000 7faf193a6b80 1 -- 172.21.15.1:0/2059521923 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1a1a6d930 msgr2=0x55a1a1a6f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:56.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.885+0000 7faf193a6b80 1 --2- 172.21.15.1:0/2059521923 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1a1a6d930 0x55a1a1a6f840 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7faee8009230 tx=0x7faee8005600 comp rx=0 tx=0).stop 2024-02-02T15:44:56.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.885+0000 7faf193a6b80 1 -- 172.21.15.1:0/2059521923 shutdown_connections 2024-02-02T15:44:56.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.885+0000 7faf193a6b80 1 --2- 172.21.15.1:0/2059521923 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a1a1a787e0 0x55a1a1a6fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:56.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.885+0000 7faf193a6b80 1 --2- 172.21.15.1:0/2059521923 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1a1a6d010 0x55a1a1a77d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:56.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.885+0000 7faf193a6b80 1 --2- 172.21.15.1:0/2059521923 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1a1a6d930 0x55a1a1a6f840 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:56.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.885+0000 7faf193a6b80 1 -- 172.21.15.1:0/2059521923 >> 172.21.15.1:0/2059521923 conn(0x55a1a19ccdb0 msgr2=0x55a1a1a70740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:56.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.885+0000 7faf193a6b80 1 -- 172.21.15.1:0/2059521923 shutdown_connections 2024-02-02T15:44:56.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.885+0000 7faf193a6b80 1 -- 172.21.15.1:0/2059521923 wait complete. 2024-02-02T15:44:56.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.886+0000 7faf193a6b80 1 -- start start 2024-02-02T15:44:56.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.886+0000 7faf193a6b80 1 Processor -- start 2024-02-02T15:44:56.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.886+0000 7faf193a6b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1a1a6d010 0x55a1a1a7ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:56.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.886+0000 7faf193a6b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1a1a6d930 0x55a1a1a7f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:56.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.886+0000 7faf193a6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a1a1a787e0 0x55a1a1a7d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:56.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.886+0000 7faf193a6b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1a1a7dbc0 con 0x55a1a1a6d930 2024-02-02T15:44:56.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.887+0000 7faf193a6b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1a1a7dd30 con 0x55a1a1a6d010 2024-02-02T15:44:56.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.887+0000 7faf193a6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a1a1a7de70 con 0x55a1a1a787e0 2024-02-02T15:44:56.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.887+0000 7faf193a6b80 10 client.0 _ll_register_callbacks cb 0x55a1a195c290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:44:56.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.887+0000 7faefea00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1a1a6d010 0x55a1a1a7ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:56.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.887+0000 7faefea00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1a1a6d010 0x55a1a1a7ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:35172/0 (socket says 172.21.15.1:35172) 2024-02-02T15:44:56.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.887+0000 7faefea00700 1 -- 172.21.15.1:0/4143212028 learned_addr learned my addr 172.21.15.1:0/4143212028 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:44:56.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.887+0000 7faefea00700 1 -- 172.21.15.1:0/4143212028 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a1a1a787e0 msgr2=0x55a1a1a7d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:56.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.887+0000 7faefea00700 1 --2- 172.21.15.1:0/4143212028 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a1a1a787e0 0x55a1a1a7d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:56.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.887+0000 7faefe000700 1 --2- 172.21.15.1:0/4143212028 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1a1a6d930 0x55a1a1a7f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:56.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.887+0000 7faefea00700 1 -- 172.21.15.1:0/4143212028 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1a1a6d930 msgr2=0x55a1a1a7f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:56.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.887+0000 7faefea00700 1 --2- 172.21.15.1:0/4143212028 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1a1a6d930 0x55a1a1a7f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:56.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.887+0000 7faefea00700 1 -- 172.21.15.1:0/4143212028 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a1a18403b0 con 0x55a1a1a6d010 2024-02-02T15:44:56.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.888+0000 7faefea00700 1 --2- 172.21.15.1:0/4143212028 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1a1a6d010 0x55a1a1a7ee40 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7faef00958e0 tx=0x7faef0010a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4660132581bf655 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:56.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.888+0000 7faeecc00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:44:56.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.888+0000 7faeecc00700 1 -- 172.21.15.1:0/4143212028 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef00acea0 con 0x55a1a1a6d010 2024-02-02T15:44:56.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.888+0000 7faeecc00700 1 -- 172.21.15.1:0/4143212028 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9453+0+0 (secure 0 0 0) 0x7faef0099ae0 con 0x55a1a1a6d010 2024-02-02T15:44:56.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.888+0000 7faf193a6b80 1 -- 172.21.15.1:0/4143212028 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faee8004e50 con 0x55a1a1a6d010 2024-02-02T15:44:56.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.888+0000 7faeecc00700 1 -- 172.21.15.1:0/4143212028 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) v2 ==== 24+0+0 (secure 0 0 0) 0x7faef009d210 con 0x55a1a1a6d010 2024-02-02T15:44:56.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.889+0000 7faf193a6b80 10 client.30200 fetch_fsmap learned FSMap version 77 2024-02-02T15:44:56.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.889+0000 7faf193a6b80 1 -- 172.21.15.1:0/4143212028 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=4+}) v3 -- 0x55a1a1844cd0 con 0x55a1a1a6d010 2024-02-02T15:44:56.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.889+0000 7faeecc00700 1 -- 172.21.15.1:0/4143212028 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faee8004e50 con 0x55a1a1a6d010 2024-02-02T15:44:56.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.890+0000 7faeecc00700 1 -- 172.21.15.1:0/4143212028 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 77) v1 ==== 7502+0+0 (secure 0 0 0) 0x7faef00b7120 con 0x55a1a1a6d010 2024-02-02T15:44:56.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.890+0000 7faf193a6b80 10 client.30200 fetch_fsmap finished waiting for FSMap version 77 2024-02-02T15:44:56.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.890+0000 7faf193a6b80 10 client.30200 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:44:56.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.890+0000 7faf193a6b80 20 client.30200 populate_metadata read hostname 'smithi001' 2024-02-02T15:44:56.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.890+0000 7faf193a6b80 1 --2- 172.21.15.1:0/4143212028 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55a1a1ad0a30 0x55a1a1ad2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:56.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.890+0000 7faf193a6b80 4 client.30200 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2024-02-02T15:44:56.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.890+0000 7faf193a6b80 1 -- 172.21.15.1:0/4143212028 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a1a197df50 con 0x55a1a1ad0a30 2024-02-02T15:44:56.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.890+0000 7faefe000700 1 --2- 172.21.15.1:0/4143212028 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55a1a1ad0a30 0x55a1a1ad2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:56.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.891+0000 7faefe000700 1 --2- 172.21.15.1:0/4143212028 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55a1a1ad0a30 0x55a1a1ad2e20 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec41e17d0171e99c server_cookie=3d9599f35b06dd5b in_seq=0 out_seq=0 2024-02-02T15:44:56.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.891+0000 7faeecc00700 10 client.30200 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:44:56.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.891+0000 7faeecc00700 1 -- 172.21.15.1:0/4143212028 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55a1a197df50 con 0x55a1a1ad0a30 2024-02-02T15:44:56.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.891+0000 7faeecc00700 10 client.30200 handle_command_reply: tid=0 2024-02-02T15:44:56.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.891+0000 7faf193a6b80 1 client.30200 shutdown 2024-02-02T15:44:56.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.891+0000 7faf193a6b80 1 -- 172.21.15.1:0/4143212028 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1a1a6d010 msgr2=0x55a1a1a7ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:56.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.891+0000 7faf193a6b80 1 --2- 172.21.15.1:0/4143212028 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1a1a6d010 0x55a1a1a7ee40 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7faef00958e0 tx=0x7faef0010a90 comp rx=0 tx=0).stop 2024-02-02T15:44:56.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.891+0000 7faf193a6b80 1 -- 172.21.15.1:0/4143212028 shutdown_connections 2024-02-02T15:44:56.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.891+0000 7faf193a6b80 1 --2- 172.21.15.1:0/4143212028 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55a1a1ad0a30 0x55a1a1ad2e20 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:56.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.891+0000 7faf193a6b80 1 --2- 172.21.15.1:0/4143212028 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a1a1a787e0 0x55a1a1a7d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:56.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.891+0000 7faf193a6b80 1 --2- 172.21.15.1:0/4143212028 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1a1a6d010 0x55a1a1a7ee40 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:56.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.891+0000 7faf193a6b80 1 --2- 172.21.15.1:0/4143212028 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a1a1a6d930 0x55a1a1a7f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:56.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.891+0000 7faeecc00700 0 client.30200 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:44:56.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.891+0000 7faf193a6b80 1 -- 172.21.15.1:0/4143212028 >> 172.21.15.1:0/4143212028 conn(0x55a1a19ccdb0 msgr2=0x55a1a1a685c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:56.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.891+0000 7faf193a6b80 1 -- 172.21.15.1:0/4143212028 shutdown_connections 2024-02-02T15:44:56.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.891+0000 7faf193a6b80 1 -- 172.21.15.1:0/4143212028 wait complete. 2024-02-02T15:44:56.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.892+0000 7faf193a6b80 20 client.30200 trim_cache size 0 max 16384 2024-02-02T15:44:56.894 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:44:56.895 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2024-02-02T15:44:56.895 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2024-02-02T15:44:56.895 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:44:56.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.894+0000 7faf05800700 1 -- 172.21.15.1:0/3731174182 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7faee00a0710 msgr2=0x7faee00a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:56.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.894+0000 7faf05800700 1 --2- 172.21.15.1:0/3731174182 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7faee00a0710 0x7faee00a2bd0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7faef80062a0 tx=0x7faef8006210 comp rx=0 tx=0).stop 2024-02-02T15:44:56.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.894+0000 7faf05800700 1 -- 172.21.15.1:0/3731174182 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf00103200 msgr2=0x7faf001991a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:56.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.894+0000 7faf05800700 1 --2- 172.21.15.1:0/3731174182 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf00103200 0x7faf001991a0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7faef000e9a0 tx=0x7faef000ecb0 comp rx=0 tx=0).stop 2024-02-02T15:44:56.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.894+0000 7faf05800700 1 -- 172.21.15.1:0/3731174182 shutdown_connections 2024-02-02T15:44:56.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.894+0000 7faf05800700 1 --2- 172.21.15.1:0/3731174182 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7faee00a0710 0x7faee00a2bd0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:56.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.894+0000 7faf05800700 1 --2- 172.21.15.1:0/3731174182 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faf0019db40 0x7faf0019dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:56.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.894+0000 7faf05800700 1 --2- 172.21.15.1:0/3731174182 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf00103200 0x7faf001991a0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:56.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.894+0000 7faf05800700 1 --2- 172.21.15.1:0/3731174182 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faf00108320 0x7faf001996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:56.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.894+0000 7faf05800700 1 -- 172.21.15.1:0/3731174182 >> 172.21.15.1:0/3731174182 conn(0x7faf000fea40 msgr2=0x7faf000ff420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:56.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.895+0000 7faf05800700 1 -- 172.21.15.1:0/3731174182 shutdown_connections 2024-02-02T15:44:56.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:56.895+0000 7faf05800700 1 -- 172.21.15.1:0/3731174182 wait complete. 2024-02-02T15:44:56.932 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-02T15:44:56.932 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T15:44:56.997 INFO:tasks.workunit.client.0.smithi001.stdout: 10 147749 28.24 MB/sec execute 149 sec latency 92.575 ms 2024-02-02T15:44:57.176 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.176+0000 7f7fe6a00700 1 -- 172.21.15.1:0/3209626491 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7fe0073130 msgr2=0x7f7fe0073510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:57.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.176+0000 7f7fe6a00700 1 --2- 172.21.15.1:0/3209626491 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7fe0073130 0x7f7fe0073510 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f7fd0009990 tx=0x7f7fd0009ca0 comp rx=0 tx=0).stop 2024-02-02T15:44:57.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.176+0000 7f7fe6a00700 1 -- 172.21.15.1:0/3209626491 shutdown_connections 2024-02-02T15:44:57.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.176+0000 7f7fe6a00700 1 --2- 172.21.15.1:0/3209626491 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7fe010d510 0x7f7fe010f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:57.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.176+0000 7f7fe6a00700 1 --2- 172.21.15.1:0/3209626491 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7fe0073130 0x7f7fe0073510 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:57.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.176+0000 7f7fe6a00700 1 --2- 172.21.15.1:0/3209626491 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7fe0073a50 0x7f7fe010cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:57.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.176+0000 7f7fe6a00700 1 -- 172.21.15.1:0/3209626491 >> 172.21.15.1:0/3209626491 conn(0x7f7fe00fc780 msgr2=0x7f7fe00feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:57.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.176+0000 7f7fe6a00700 1 -- 172.21.15.1:0/3209626491 shutdown_connections 2024-02-02T15:44:57.177 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.176+0000 7f7fe6a00700 1 -- 172.21.15.1:0/3209626491 wait complete. 2024-02-02T15:44:57.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.178+0000 7f7fe6a00700 1 Processor -- start 2024-02-02T15:44:57.178 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.178+0000 7f7fe6a00700 1 -- start start 2024-02-02T15:44:57.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.178+0000 7f7fe6a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7fe0073130 0x7f7fe0199000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:57.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.178+0000 7f7fe6a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7fe0073a50 0x7f7fe0199540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:57.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.178+0000 7f7fe6a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7fe010d510 0x7f7fe019d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:57.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.178+0000 7f7fe6a00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe019dfe0 con 0x7f7fe0073130 2024-02-02T15:44:57.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.178+0000 7f7fe6a00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe019e150 con 0x7f7fe0073a50 2024-02-02T15:44:57.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.178+0000 7f7fe6a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe019e310 con 0x7f7fe010d510 2024-02-02T15:44:57.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.179+0000 7f7fdfe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7fe0073130 0x7f7fe0199000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:57.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.179+0000 7f7fdf400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7fe0073a50 0x7f7fe0199540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:57.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.179+0000 7f7fdf400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7fe0073a50 0x7f7fe0199540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:35176/0 (socket says 172.21.15.1:35176) 2024-02-02T15:44:57.179 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.179+0000 7f7fdf400700 1 -- 172.21.15.1:0/3416545446 learned_addr learned my addr 172.21.15.1:0/3416545446 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:44:57.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.179+0000 7f7fdf400700 1 -- 172.21.15.1:0/3416545446 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7fe010d510 msgr2=0x7f7fe019d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:57.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.179+0000 7f7fdf400700 1 --2- 172.21.15.1:0/3416545446 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7fe010d510 0x7f7fe019d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:57.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.179+0000 7f7fdf400700 1 -- 172.21.15.1:0/3416545446 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7fe0073130 msgr2=0x7f7fe0199000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:57.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.179+0000 7f7fdf400700 1 --2- 172.21.15.1:0/3416545446 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7fe0073130 0x7f7fe0199000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:57.180 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.179+0000 7f7fdf400700 1 -- 172.21.15.1:0/3416545446 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fd00095f0 con 0x7f7fe0073a50 2024-02-02T15:44:57.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.180+0000 7f7fdf400700 1 --2- 172.21.15.1:0/3416545446 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7fe0073a50 0x7f7fe0199540 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f7fc800e9a0 tx=0x7f7fc800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:57.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.180+0000 7f7fdcc00700 1 -- 172.21.15.1:0/3416545446 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc8009790 con 0x7f7fe0073a50 2024-02-02T15:44:57.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.180+0000 7f7fe6a00700 1 -- 172.21.15.1:0/3416545446 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fe019e5f0 con 0x7f7fe0073a50 2024-02-02T15:44:57.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.180+0000 7f7fdcc00700 1 -- 172.21.15.1:0/3416545446 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7fc800cbd0 con 0x7f7fe0073a50 2024-02-02T15:44:57.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.180+0000 7f7fdcc00700 1 -- 172.21.15.1:0/3416545446 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc8018680 con 0x7f7fe0073a50 2024-02-02T15:44:57.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.180+0000 7f7fe6a00700 1 -- 172.21.15.1:0/3416545446 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fe01a5e30 con 0x7f7fe0073a50 2024-02-02T15:44:57.181 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.181+0000 7f7fe6a00700 1 -- 172.21.15.1:0/3416545446 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fe0068010 con 0x7f7fe0073a50 2024-02-02T15:44:57.183 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.183+0000 7f7fdcc00700 1 -- 172.21.15.1:0/3416545446 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f7fc800cd40 con 0x7f7fe0073a50 2024-02-02T15:44:57.184 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.184+0000 7f7fdcc00700 1 --2- 172.21.15.1:0/3416545446 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f7fc00a0690 0x7f7fc00a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:57.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.185+0000 7f7fdcc00700 1 -- 172.21.15.1:0/3416545446 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f7fc8014070 con 0x7f7fe0073a50 2024-02-02T15:44:57.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.185+0000 7f7fdfe00700 1 --2- 172.21.15.1:0/3416545446 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f7fc00a0690 0x7f7fc00a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:57.185 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.186+0000 7f7fdfe00700 1 --2- 172.21.15.1:0/3416545446 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f7fc00a0690 0x7f7fc00a2b50 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f7fd000b2e0 tx=0x7f7fd0005fb0 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:57.191 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.192+0000 7f7fdcc00700 1 -- 172.21.15.1:0/3416545446 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f7fc8052110 con 0x7f7fe0073a50 2024-02-02T15:44:57.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.551+0000 7f7fe6a00700 1 -- 172.21.15.1:0/3416545446 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7fe019a350 con 0x7f7fe0073a50 2024-02-02T15:44:57.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.553+0000 7f7fdcc00700 1 -- 172.21.15.1:0/3416545446 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+10880 (secure 0 0 0) 0x7f7fc80958b0 con 0x7f7fe0073a50 2024-02-02T15:44:57.553 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:44:57.553 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":77,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2922240448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2922240448},{"type":"v1","addr":"172.21.15.1:6835","nonce":2922240448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2729497431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2729497431},{"type":"v1","addr":"172.21.15.1:6837","nonce":2729497431}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":77,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-02T15:24:47.761435+0000","modified":"2024-02-02T15:44:46.540986+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24529,"mds_2":24518,"mds_3":24514,"mds_4":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14517":{"gid":14517,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/1218303683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":1218303683},{"type":"v1","addr":"172.21.15.1:6839","nonce":1218303683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14529":{"gid":14529,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/2059197496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2059197496},{"type":"v1","addr":"172.21.15.1:6841","nonce":2059197496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/7648986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":7648986},{"type":"v1","addr":"172.21.15.142:6833","nonce":7648986}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/3482746094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3482746094},{"type":"v1","addr":"172.21.15.71:6833","nonce":3482746094}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/388207814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":388207814},{"type":"v1","addr":"172.21.15.71:6835","nonce":388207814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6835/1052606547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1052606547},{"type":"v1","addr":"172.21.15.142:6835","nonce":1052606547}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.142:6837/2870289019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2870289019},{"type":"v1","addr":"172.21.15.142:6837","nonce":2870289019}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.71:6837/628016199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":628016199},{"type":"v1","addr":"172.21.15.71:6837","nonce":628016199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.142:6839/321675421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":321675421},{"type":"v1","addr":"172.21.15.142:6839","nonce":321675421}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/1268086493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1268086493},{"type":"v1","addr":"172.21.15.71:6839","nonce":1268086493}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-02T15:44:57.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.558+0000 7f7fe6a00700 1 -- 172.21.15.1:0/3416545446 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f7fc00a0690 msgr2=0x7f7fc00a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:57.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.558+0000 7f7fe6a00700 1 --2- 172.21.15.1:0/3416545446 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f7fc00a0690 0x7f7fc00a2b50 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f7fd000b2e0 tx=0x7f7fd0005fb0 comp rx=0 tx=0).stop 2024-02-02T15:44:57.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.558+0000 7f7fe6a00700 1 -- 172.21.15.1:0/3416545446 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7fe0073a50 msgr2=0x7f7fe0199540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:57.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.558+0000 7f7fe6a00700 1 --2- 172.21.15.1:0/3416545446 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7fe0073a50 0x7f7fe0199540 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f7fc800e9a0 tx=0x7f7fc800ecb0 comp rx=0 tx=0).stop 2024-02-02T15:44:57.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.559+0000 7f7fe6a00700 1 -- 172.21.15.1:0/3416545446 shutdown_connections 2024-02-02T15:44:57.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.559+0000 7f7fe6a00700 1 --2- 172.21.15.1:0/3416545446 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f7fc00a0690 0x7f7fc00a2b50 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:57.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.559+0000 7f7fe6a00700 1 --2- 172.21.15.1:0/3416545446 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f7fe010d510 0x7f7fe019d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:57.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.559+0000 7f7fe6a00700 1 --2- 172.21.15.1:0/3416545446 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f7fe0073130 0x7f7fe0199000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:57.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.559+0000 7f7fe6a00700 1 --2- 172.21.15.1:0/3416545446 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7fe0073a50 0x7f7fe0199540 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:57.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.559+0000 7f7fe6a00700 1 -- 172.21.15.1:0/3416545446 >> 172.21.15.1:0/3416545446 conn(0x7f7fe00fc780 msgr2=0x7f7fe010f620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:57.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.559+0000 7f7fe6a00700 1 -- 172.21.15.1:0/3416545446 shutdown_connections 2024-02-02T15:44:57.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.559+0000 7f7fe6a00700 1 -- 172.21.15.1:0/3416545446 wait complete. 2024-02-02T15:44:57.561 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 77 2024-02-02T15:44:57.585 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-02-02T15:44:57.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:44:57 smithi071 ceph-mon[44707]: pgmap v563: 129 pgs: 129 active+clean; 2.0 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 683 op/s 2024-02-02T15:44:57.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.912+0000 7f15f0a00700 1 -- 172.21.15.1:0/1114354363 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15e8067780 msgr2=0x7f15e81054b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:57.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.912+0000 7f15f0a00700 1 --2- 172.21.15.1:0/1114354363 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15e8067780 0x7f15e81054b0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f15e000b0a0 tx=0x7f15e000b3b0 comp rx=0 tx=0).stop 2024-02-02T15:44:57.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.912+0000 7f15f0a00700 1 -- 172.21.15.1:0/1114354363 shutdown_connections 2024-02-02T15:44:57.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.912+0000 7f15f0a00700 1 --2- 172.21.15.1:0/1114354363 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15e80ff5f0 0x7f15e80ffa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:57.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.912+0000 7f15f0a00700 1 --2- 172.21.15.1:0/1114354363 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15e8067780 0x7f15e81054b0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:57.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.912+0000 7f15f0a00700 1 --2- 172.21.15.1:0/1114354363 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15e8067bf0 0x7f15e81059f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:57.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.913+0000 7f15f0a00700 1 -- 172.21.15.1:0/1114354363 >> 172.21.15.1:0/1114354363 conn(0x7f15e8074f30 msgr2=0x7f15e8075340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:57.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.913+0000 7f15f0a00700 1 -- 172.21.15.1:0/1114354363 shutdown_connections 2024-02-02T15:44:57.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.913+0000 7f15f0a00700 1 -- 172.21.15.1:0/1114354363 wait complete. 2024-02-02T15:44:57.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.914+0000 7f15f0a00700 1 Processor -- start 2024-02-02T15:44:57.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.915+0000 7f15f0a00700 1 -- start start 2024-02-02T15:44:57.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.915+0000 7f15f0a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15e8067780 0x7f15e810b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:57.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.915+0000 7f15f0a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15e8067bf0 0x7f15e810b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:57.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.915+0000 7f15f0a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15e80ff5f0 0x7f15e81a51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:57.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.915+0000 7f15f0a00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15e810c220 con 0x7f15e8067780 2024-02-02T15:44:57.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.915+0000 7f15f0a00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15e81a5720 con 0x7f15e8067bf0 2024-02-02T15:44:57.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.915+0000 7f15f0a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15e81a5860 con 0x7f15e80ff5f0 2024-02-02T15:44:57.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.916+0000 7f15ee000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15e8067780 0x7f15e810b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:57.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.916+0000 7f15ed600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15e8067bf0 0x7f15e810b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:57.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.916+0000 7f15eea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15e80ff5f0 0x7f15e81a51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:57.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.916+0000 7f15eea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15e80ff5f0 0x7f15e81a51e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:50408/0 (socket says 172.21.15.1:50408) 2024-02-02T15:44:57.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.916+0000 7f15eea00700 1 -- 172.21.15.1:0/4258128008 learned_addr learned my addr 172.21.15.1:0/4258128008 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:44:57.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.917+0000 7f15ed600700 1 -- 172.21.15.1:0/4258128008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15e80ff5f0 msgr2=0x7f15e81a51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:57.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.917+0000 7f15ed600700 1 --2- 172.21.15.1:0/4258128008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15e80ff5f0 0x7f15e81a51e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:57.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.917+0000 7f15ed600700 1 -- 172.21.15.1:0/4258128008 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15e8067780 msgr2=0x7f15e810b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:57.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.917+0000 7f15ed600700 1 --2- 172.21.15.1:0/4258128008 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15e8067780 0x7f15e810b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:57.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.917+0000 7f15ed600700 1 -- 172.21.15.1:0/4258128008 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15d0009520 con 0x7f15e8067bf0 2024-02-02T15:44:57.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.917+0000 7f15ee000700 1 --2- 172.21.15.1:0/4258128008 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15e8067780 0x7f15e810b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:44:57.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.920+0000 7f15ed600700 1 --2- 172.21.15.1:0/4258128008 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15e8067bf0 0x7f15e810b9f0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f15d800c940 tx=0x7f15d800cc50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:57.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:44:57 smithi142 ceph-mon[45728]: pgmap v563: 129 pgs: 129 active+clean; 2.0 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 683 op/s 2024-02-02T15:44:57.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.920+0000 7f15e6a00700 1 -- 172.21.15.1:0/4258128008 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d8004440 con 0x7f15e8067bf0 2024-02-02T15:44:57.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.920+0000 7f15e6a00700 1 -- 172.21.15.1:0/4258128008 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f15d800ce30 con 0x7f15e8067bf0 2024-02-02T15:44:57.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.920+0000 7f15e6a00700 1 -- 172.21.15.1:0/4258128008 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d8003c70 con 0x7f15e8067bf0 2024-02-02T15:44:57.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.921+0000 7f15f0a00700 1 -- 172.21.15.1:0/4258128008 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15e0009d00 con 0x7f15e8067bf0 2024-02-02T15:44:57.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.921+0000 7f15f0a00700 1 -- 172.21.15.1:0/4258128008 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15e81970c0 con 0x7f15e8067bf0 2024-02-02T15:44:57.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.925+0000 7f15e6a00700 1 -- 172.21.15.1:0/4258128008 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f15d8007cb0 con 0x7f15e8067bf0 2024-02-02T15:44:57.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.927+0000 7f1604600b80 1 Processor -- start 2024-02-02T15:44:57.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.928+0000 7f1604600b80 1 -- start start 2024-02-02T15:44:57.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.928+0000 7f1604600b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557ed387d010 0x557ed387d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:57.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.928+0000 7f1604600b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557ed387d930 0x557ed38882a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:57.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.928+0000 7f1604600b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ed38887e0 0x557ed388abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:57.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.928+0000 7f1604600b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ed37f14c0 con 0x557ed387d010 2024-02-02T15:44:57.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.928+0000 7f1604600b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ed37d5aa0 con 0x557ed387d930 2024-02-02T15:44:57.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.928+0000 7f1604600b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ed37d3460 con 0x557ed38887e0 2024-02-02T15:44:57.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.929+0000 7f15ee000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557ed387d930 0x557ed38882a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:57.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.929+0000 7f15ee000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557ed387d930 0x557ed38882a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:35190/0 (socket says 172.21.15.1:35190) 2024-02-02T15:44:57.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.929+0000 7f15ee000700 1 -- 172.21.15.1:0/2053457459 learned_addr learned my addr 172.21.15.1:0/2053457459 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:44:57.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.929+0000 7f15eea00700 1 --2- 172.21.15.1:0/2053457459 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557ed387d010 0x557ed387d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:57.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.929+0000 7f15e6a00700 1 --2- 172.21.15.1:0/4258128008 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f15c80a0710 0x7f15c80a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:57.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.930+0000 7f15eea00700 1 -- 172.21.15.1:0/2053457459 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ed38887e0 msgr2=0x557ed388abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:57.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.930+0000 7f15eea00700 1 --2- 172.21.15.1:0/2053457459 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ed38887e0 0x557ed388abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:57.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.930+0000 7f15eea00700 1 -- 172.21.15.1:0/2053457459 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557ed387d930 msgr2=0x557ed38882a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:57.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.930+0000 7f15eea00700 1 --2- 172.21.15.1:0/2053457459 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557ed387d930 0x557ed38882a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:57.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.930+0000 7f15eea00700 1 -- 172.21.15.1:0/2053457459 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557ed36503b0 con 0x557ed387d010 2024-02-02T15:44:57.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.930+0000 7f15eea00700 1 --2- 172.21.15.1:0/4258128008 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f15c80a0710 0x7f15c80a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:57.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.931+0000 7f15eea00700 1 --2- 172.21.15.1:0/2053457459 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557ed387d010 0x557ed387d3f0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f15d0005c50 tx=0x7f15d00106b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaae3a0a7e12e2cd server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:57.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.932+0000 7f15e6a00700 1 -- 172.21.15.1:0/4258128008 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f15d80cbd10 con 0x7f15e8067bf0 2024-02-02T15:44:57.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.932+0000 7f15eea00700 1 --2- 172.21.15.1:0/4258128008 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f15c80a0710 0x7f15c80a2bd0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f15d0019000 tx=0x7f15d001b000 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:57.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.932+0000 7f15df000700 1 -- 172.21.15.1:0/2053457459 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d00169c0 con 0x557ed387d010 2024-02-02T15:44:57.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.932+0000 7f15df000700 1 -- 172.21.15.1:0/2053457459 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f15d001f910 con 0x557ed387d010 2024-02-02T15:44:57.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.932+0000 7f15df000700 1 -- 172.21.15.1:0/2053457459 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d00119e0 con 0x557ed387d010 2024-02-02T15:44:57.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.934+0000 7f1604600b80 1 -- 172.21.15.1:0/2053457459 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557ed387d010 msgr2=0x557ed387d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:57.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.934+0000 7f1604600b80 1 --2- 172.21.15.1:0/2053457459 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557ed387d010 0x557ed387d3f0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f15d0005c50 tx=0x7f15d00106b0 comp rx=0 tx=0).stop 2024-02-02T15:44:57.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.934+0000 7f1604600b80 1 -- 172.21.15.1:0/2053457459 shutdown_connections 2024-02-02T15:44:57.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.934+0000 7f1604600b80 1 --2- 172.21.15.1:0/2053457459 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ed38887e0 0x557ed388abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:57.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.934+0000 7f1604600b80 1 --2- 172.21.15.1:0/2053457459 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557ed387d010 0x557ed387d3f0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:57.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.934+0000 7f1604600b80 1 --2- 172.21.15.1:0/2053457459 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557ed387d930 0x557ed38882a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:57.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.934+0000 7f1604600b80 1 -- 172.21.15.1:0/2053457459 >> 172.21.15.1:0/2053457459 conn(0x557ed37dcdb0 msgr2=0x557ed38793b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:57.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.934+0000 7f1604600b80 1 -- 172.21.15.1:0/2053457459 shutdown_connections 2024-02-02T15:44:57.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.935+0000 7f1604600b80 1 -- 172.21.15.1:0/2053457459 wait complete. 2024-02-02T15:44:57.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.935+0000 7f1604600b80 1 -- start start 2024-02-02T15:44:57.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.938+0000 7f1604600b80 1 Processor -- start 2024-02-02T15:44:57.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.939+0000 7f1604600b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557ed387d010 0x557ed3885570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:57.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.939+0000 7f1604600b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ed387d930 0x557ed3885ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:57.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.939+0000 7f1604600b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557ed38887e0 0x557ed3885ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:57.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.939+0000 7f1604600b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ed37ced50 con 0x557ed38887e0 2024-02-02T15:44:57.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.939+0000 7f1604600b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ed376ad80 con 0x557ed387d010 2024-02-02T15:44:57.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.939+0000 7f1604600b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ed376aa20 con 0x557ed387d930 2024-02-02T15:44:57.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.939+0000 7f1604600b80 10 client.0 _ll_register_callbacks cb 0x557ed376c290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:44:57.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.940+0000 7f15ee000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557ed387d010 0x557ed3885570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:57.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.940+0000 7f15ee000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557ed387d010 0x557ed3885570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:35202/0 (socket says 172.21.15.1:35202) 2024-02-02T15:44:57.940 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.940+0000 7f15ee000700 1 -- 172.21.15.1:0/168555135 learned_addr learned my addr 172.21.15.1:0/168555135 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:44:57.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:44:57 smithi001 ceph-mon[35470]: pgmap v563: 129 pgs: 129 active+clean; 2.0 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 683 op/s 2024-02-02T15:44:57.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.940+0000 7f15ed600700 1 --2- 172.21.15.1:0/168555135 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ed387d930 0x557ed3885ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:57.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.940+0000 7f15ee000700 1 -- 172.21.15.1:0/168555135 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ed387d930 msgr2=0x557ed3885ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:57.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.940+0000 7f15ee000700 1 --2- 172.21.15.1:0/168555135 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ed387d930 0x557ed3885ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:57.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.941+0000 7f15ee000700 1 -- 172.21.15.1:0/168555135 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557ed38887e0 msgr2=0x557ed3885ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:57.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.941+0000 7f15eea00700 1 --2- 172.21.15.1:0/168555135 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557ed38887e0 0x557ed3885ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:57.941 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.941+0000 7f15ee000700 1 --2- 172.21.15.1:0/168555135 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557ed38887e0 0x557ed3885ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:57.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.941+0000 7f15ee000700 1 -- 172.21.15.1:0/168555135 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15d0009a80 con 0x557ed387d010 2024-02-02T15:44:57.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.941+0000 7f15eea00700 1 --2- 172.21.15.1:0/168555135 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557ed38887e0 0x557ed3885ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:44:57.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.941+0000 7f15ee000700 1 --2- 172.21.15.1:0/168555135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557ed387d010 0x557ed3885570 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f15e000bfd0 tx=0x7f15e0004cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85c4506d9fe99ce8 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:57.942 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.942+0000 7f15d7400700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:44:57.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.945+0000 7f15d7400700 1 -- 172.21.15.1:0/168555135 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e000e040 con 0x557ed387d010 2024-02-02T15:44:57.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.945+0000 7f15d7400700 1 -- 172.21.15.1:0/168555135 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f15e00078f0 con 0x557ed387d010 2024-02-02T15:44:57.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.945+0000 7f1604600b80 1 -- 172.21.15.1:0/168555135 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557ed38710f0 con 0x557ed387d010 2024-02-02T15:44:57.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.945+0000 7f15d7400700 1 -- 172.21.15.1:0/168555135 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) v2 ==== 24+0+0 (secure 0 0 0) 0x7f15e00094b0 con 0x557ed387d010 2024-02-02T15:44:57.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.945+0000 7f1604600b80 10 client.30215 fetch_fsmap learned FSMap version 77 2024-02-02T15:44:57.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.946+0000 7f1604600b80 1 -- 172.21.15.1:0/168555135 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=4+}) v3 -- 0x557ed35fcb60 con 0x557ed387d010 2024-02-02T15:44:57.946 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.946+0000 7f15d7400700 1 -- 172.21.15.1:0/168555135 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557ed38710f0 con 0x557ed387d010 2024-02-02T15:44:57.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.946+0000 7f15d7400700 1 -- 172.21.15.1:0/168555135 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 77) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f15e0035030 con 0x557ed387d010 2024-02-02T15:44:57.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.946+0000 7f1604600b80 10 client.30215 fetch_fsmap finished waiting for FSMap version 77 2024-02-02T15:44:57.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.947+0000 7f1604600b80 10 client.30215 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-02-02T15:44:57.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.947+0000 7f1604600b80 20 client.30215 populate_metadata read hostname 'smithi001' 2024-02-02T15:44:57.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.947+0000 7f1604600b80 1 --2- 172.21.15.1:0/168555135 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x557ed3898310 0x557ed389a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:57.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.947+0000 7f1604600b80 4 client.30215 mds_command: new command op to 24502 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:44:57.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.947+0000 7f1604600b80 1 -- 172.21.15.1:0/168555135 --> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557ed37c7dd0 con 0x557ed3898310 2024-02-02T15:44:57.947 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.947+0000 7f15ed600700 1 --2- 172.21.15.1:0/168555135 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x557ed3898310 0x557ed389a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:57.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.951+0000 7f15ed600700 1 --2- 172.21.15.1:0/168555135 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x557ed3898310 0x557ed389a730 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a5af7c0d313739e6 server_cookie=61a2ae69f8481363 in_seq=0 out_seq=0 2024-02-02T15:44:57.950 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.951+0000 7f15d7400700 10 client.30215 ms_handle_connect on v2:172.21.15.142:6834/1052606547 2024-02-02T15:44:57.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.955+0000 7f15d7400700 1 -- 172.21.15.1:0/168555135 <== mds.4 v2:172.21.15.142:6834/1052606547 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x557ed37c7dd0 con 0x557ed3898310 2024-02-02T15:44:57.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.955+0000 7f15d7400700 10 client.30215 handle_command_reply: tid=0 2024-02-02T15:44:57.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.956+0000 7f1604600b80 1 client.30215 shutdown 2024-02-02T15:44:57.956 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.956+0000 7f1604600b80 1 -- 172.21.15.1:0/168555135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557ed387d010 msgr2=0x557ed3885570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:57.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.956+0000 7f1604600b80 1 --2- 172.21.15.1:0/168555135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557ed387d010 0x557ed3885570 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f15e000bfd0 tx=0x7f15e0004cf0 comp rx=0 tx=0).stop 2024-02-02T15:44:57.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.957+0000 7f1604600b80 1 -- 172.21.15.1:0/168555135 shutdown_connections 2024-02-02T15:44:57.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.957+0000 7f1604600b80 1 --2- 172.21.15.1:0/168555135 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x557ed3898310 0x557ed389a730 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:57.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.957+0000 7f1604600b80 1 --2- 172.21.15.1:0/168555135 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557ed38887e0 0x557ed3885ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:57.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.957+0000 7f1604600b80 1 --2- 172.21.15.1:0/168555135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557ed387d010 0x557ed3885570 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:57.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.957+0000 7f1604600b80 1 --2- 172.21.15.1:0/168555135 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ed387d930 0x557ed3885ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:57.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.957+0000 7f15d7400700 0 client.30215 ms_handle_reset on v2:172.21.15.142:6834/1052606547 2024-02-02T15:44:57.957 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.957+0000 7f1604600b80 1 -- 172.21.15.1:0/168555135 >> 172.21.15.1:0/168555135 conn(0x557ed37dcdb0 msgr2=0x557ed3878fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:57.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.957+0000 7f1604600b80 1 -- 172.21.15.1:0/168555135 shutdown_connections 2024-02-02T15:44:57.959 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.959+0000 7f1604600b80 1 -- 172.21.15.1:0/168555135 wait complete. 2024-02-02T15:44:57.960 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.960+0000 7f1604600b80 20 client.30215 trim_cache size 0 max 16384 2024-02-02T15:44:57.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.987+0000 7f1604600b80 1 Processor -- start 2024-02-02T15:44:57.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.987+0000 7f1604600b80 1 -- start start 2024-02-02T15:44:57.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.988+0000 7f1604600b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557ed387d010 0x557ed3884ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:57.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.988+0000 7f1604600b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557ed387d930 0x557ed3878eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:57.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.988+0000 7f1604600b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ed38887e0 0x557ed38793f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:57.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.988+0000 7f1604600b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ed36526a0 con 0x557ed387d010 2024-02-02T15:44:57.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.988+0000 7f1604600b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ed3894cb0 con 0x557ed387d930 2024-02-02T15:44:57.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.988+0000 7f1604600b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ed38752e0 con 0x557ed38887e0 2024-02-02T15:44:57.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.989+0000 7f15ed600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557ed387d930 0x557ed3878eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:57.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.990+0000 7f15ed600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557ed387d930 0x557ed3878eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:35218/0 (socket says 172.21.15.1:35218) 2024-02-02T15:44:57.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.990+0000 7f15ed600700 1 -- 172.21.15.1:0/1096994372 learned_addr learned my addr 172.21.15.1:0/1096994372 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:44:57.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.990+0000 7f15eea00700 1 --2- 172.21.15.1:0/1096994372 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ed38887e0 0x557ed38793f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:57.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.990+0000 7f15ee000700 1 --2- 172.21.15.1:0/1096994372 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557ed387d010 0x557ed3884ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:57.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.990+0000 7f15ed600700 1 -- 172.21.15.1:0/1096994372 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ed38887e0 msgr2=0x557ed38793f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:57.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.990+0000 7f15ed600700 1 --2- 172.21.15.1:0/1096994372 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ed38887e0 0x557ed38793f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:57.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.990+0000 7f15ed600700 1 -- 172.21.15.1:0/1096994372 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557ed387d010 msgr2=0x557ed3884ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:57.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.990+0000 7f15ed600700 1 --2- 172.21.15.1:0/1096994372 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557ed387d010 0x557ed3884ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:57.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.990+0000 7f15ed600700 1 -- 172.21.15.1:0/1096994372 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557ed38948d0 con 0x557ed387d930 2024-02-02T15:44:57.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.991+0000 7f15ed600700 1 --2- 172.21.15.1:0/1096994372 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557ed387d930 0x557ed3878eb0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f15d80074a0 tx=0x7f15d8003710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c96ca9d154526883 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:57.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.991+0000 7f15dfa00700 1 -- 172.21.15.1:0/1096994372 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d809cc60 con 0x557ed387d930 2024-02-02T15:44:57.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.991+0000 7f15dfa00700 1 -- 172.21.15.1:0/1096994372 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f15d80a0ea0 con 0x557ed387d930 2024-02-02T15:44:57.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.991+0000 7f15dfa00700 1 -- 172.21.15.1:0/1096994372 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d80c92e0 con 0x557ed387d930 2024-02-02T15:44:57.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.993+0000 7f1604600b80 1 -- 172.21.15.1:0/1096994372 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557ed387d930 msgr2=0x557ed3878eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:57.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.993+0000 7f1604600b80 1 --2- 172.21.15.1:0/1096994372 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557ed387d930 0x557ed3878eb0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f15d80074a0 tx=0x7f15d8003710 comp rx=0 tx=0).stop 2024-02-02T15:44:57.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.995+0000 7f1604600b80 1 -- 172.21.15.1:0/1096994372 shutdown_connections 2024-02-02T15:44:57.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.995+0000 7f1604600b80 1 --2- 172.21.15.1:0/1096994372 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ed38887e0 0x557ed38793f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:57.994 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.995+0000 7f1604600b80 1 --2- 172.21.15.1:0/1096994372 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557ed387d010 0x557ed3884ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:57.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.995+0000 7f1604600b80 1 --2- 172.21.15.1:0/1096994372 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557ed387d930 0x557ed3878eb0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:57.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.995+0000 7f1604600b80 1 -- 172.21.15.1:0/1096994372 >> 172.21.15.1:0/1096994372 conn(0x557ed37dcdb0 msgr2=0x557ed38938a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:57.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.995+0000 7f1604600b80 1 -- 172.21.15.1:0/1096994372 shutdown_connections 2024-02-02T15:44:57.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.995+0000 7f1604600b80 1 -- 172.21.15.1:0/1096994372 wait complete. 2024-02-02T15:44:57.997 INFO:tasks.workunit.client.0.smithi001.stdout: 10 148273 28.25 MB/sec execute 150 sec latency 104.971 ms 2024-02-02T15:44:57.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.996+0000 7f1604600b80 1 -- start start 2024-02-02T15:44:57.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.996+0000 7f1604600b80 1 Processor -- start 2024-02-02T15:44:57.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.997+0000 7f1604600b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ed387d010 0x557ed388ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:57.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.997+0000 7f1604600b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557ed387d930 0x557ed388d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:57.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.997+0000 7f1604600b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557ed38887e0 0x557ed38e8df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:57.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.998+0000 7f1604600b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ed388db30 con 0x557ed387d930 2024-02-02T15:44:57.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.998+0000 7f1604600b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ed38e9360 con 0x557ed38887e0 2024-02-02T15:44:57.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.998+0000 7f1604600b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ed38e9520 con 0x557ed387d010 2024-02-02T15:44:57.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.998+0000 7f1604600b80 10 client.0 _ll_register_callbacks cb 0x557ed376c290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:44:57.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.998+0000 7f15eea00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557ed38887e0 0x557ed38e8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:57.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.998+0000 7f15eea00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557ed38887e0 0x557ed38e8df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:35226/0 (socket says 172.21.15.1:35226) 2024-02-02T15:44:57.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.998+0000 7f15eea00700 1 -- 172.21.15.1:0/312658201 learned_addr learned my addr 172.21.15.1:0/312658201 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:44:57.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.998+0000 7f15ed600700 1 --2- 172.21.15.1:0/312658201 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557ed387d930 0x557ed388d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:58.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.998+0000 7f15ee000700 1 --2- 172.21.15.1:0/312658201 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ed387d010 0x557ed388ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:58.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.999+0000 7f15eea00700 1 -- 172.21.15.1:0/312658201 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ed387d010 msgr2=0x557ed388ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:58.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.999+0000 7f15eea00700 1 --2- 172.21.15.1:0/312658201 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ed387d010 0x557ed388ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:58.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:57.999+0000 7f15eea00700 1 -- 172.21.15.1:0/312658201 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557ed387d930 msgr2=0x557ed388d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:58.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.000+0000 7f15eea00700 1 --2- 172.21.15.1:0/312658201 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557ed387d930 0x557ed388d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:58.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.000+0000 7f15eea00700 1 -- 172.21.15.1:0/312658201 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15e81970c0 con 0x557ed38887e0 2024-02-02T15:44:58.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.000+0000 7f15ed600700 1 --2- 172.21.15.1:0/312658201 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557ed387d930 0x557ed388d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:44:58.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.000+0000 7f15eea00700 1 --2- 172.21.15.1:0/312658201 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557ed38887e0 0x557ed38e8df0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f15d0005c50 tx=0x7f15d0005bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d43f4ed1bfa62c4 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:58.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.000+0000 7f15d7400700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:44:58.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.000+0000 7f15d7400700 1 -- 172.21.15.1:0/312658201 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d001fe90 con 0x557ed38887e0 2024-02-02T15:44:58.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.000+0000 7f15d7400700 1 -- 172.21.15.1:0/312658201 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f15d0016a80 con 0x557ed38887e0 2024-02-02T15:44:58.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.000+0000 7f1604600b80 1 -- 172.21.15.1:0/312658201 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f15d001f910 con 0x557ed38887e0 2024-02-02T15:44:58.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.001+0000 7f15d7400700 1 -- 172.21.15.1:0/312658201 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) v2 ==== 24+0+0 (secure 0 0 0) 0x7f15d0004020 con 0x557ed38887e0 2024-02-02T15:44:58.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.001+0000 7f1604600b80 10 client.30221 fetch_fsmap learned FSMap version 77 2024-02-02T15:44:58.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.001+0000 7f1604600b80 1 -- 172.21.15.1:0/312658201 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=4+}) v3 -- 0x557ed3794ea0 con 0x557ed38887e0 2024-02-02T15:44:58.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.001+0000 7f15d7400700 1 -- 172.21.15.1:0/312658201 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f15d001f910 con 0x557ed38887e0 2024-02-02T15:44:58.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.002+0000 7f15d7400700 1 -- 172.21.15.1:0/312658201 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 77) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f15d0029450 con 0x557ed38887e0 2024-02-02T15:44:58.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.002+0000 7f1604600b80 10 client.30221 fetch_fsmap finished waiting for FSMap version 77 2024-02-02T15:44:58.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.002+0000 7f1604600b80 10 client.30221 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-02-02T15:44:58.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.002+0000 7f1604600b80 20 client.30221 populate_metadata read hostname 'smithi001' 2024-02-02T15:44:58.002 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.002+0000 7f1604600b80 1 --2- 172.21.15.1:0/312658201 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x557ed38eac80 0x557ed38ed0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:58.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.002+0000 7f1604600b80 4 client.30221 mds_command: new command op to 24502 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:44:58.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.002+0000 7f1604600b80 1 -- 172.21.15.1:0/312658201 --> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x557ed378df50 con 0x557ed38eac80 2024-02-02T15:44:58.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.004+0000 7f15ee000700 1 --2- 172.21.15.1:0/312658201 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x557ed38eac80 0x557ed38ed0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:58.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.005+0000 7f15d7400700 10 client.30221 ms_handle_connect on v2:172.21.15.142:6834/1052606547 2024-02-02T15:44:58.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.005+0000 7f15ee000700 1 --2- 172.21.15.1:0/312658201 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x557ed38eac80 0x557ed38ed0a0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=25800ad2dc8d821c server_cookie=891086c693c0e104 in_seq=0 out_seq=0 2024-02-02T15:44:58.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.007+0000 7f15d7400700 1 -- 172.21.15.1:0/312658201 <== mds.4 v2:172.21.15.142:6834/1052606547 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x557ed378df50 con 0x557ed38eac80 2024-02-02T15:44:58.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.007+0000 7f15d7400700 10 client.30221 handle_command_reply: tid=0 2024-02-02T15:44:58.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.007+0000 7f1604600b80 1 client.30221 shutdown 2024-02-02T15:44:58.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.008+0000 7f1604600b80 1 -- 172.21.15.1:0/312658201 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557ed38887e0 msgr2=0x557ed38e8df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:58.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.008+0000 7f1604600b80 1 --2- 172.21.15.1:0/312658201 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557ed38887e0 0x557ed38e8df0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f15d0005c50 tx=0x7f15d0005bb0 comp rx=0 tx=0).stop 2024-02-02T15:44:58.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.009+0000 7f1604600b80 1 -- 172.21.15.1:0/312658201 shutdown_connections 2024-02-02T15:44:58.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.009+0000 7f1604600b80 1 --2- 172.21.15.1:0/312658201 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x557ed38eac80 0x557ed38ed0a0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:58.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.009+0000 7f1604600b80 1 --2- 172.21.15.1:0/312658201 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x557ed38887e0 0x557ed38e8df0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:58.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.009+0000 7f1604600b80 1 --2- 172.21.15.1:0/312658201 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x557ed387d010 0x557ed388ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:58.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.009+0000 7f1604600b80 1 --2- 172.21.15.1:0/312658201 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x557ed387d930 0x557ed388d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:58.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.009+0000 7f1604600b80 1 -- 172.21.15.1:0/312658201 >> 172.21.15.1:0/312658201 conn(0x557ed37dcdb0 msgr2=0x557ed38938a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:58.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.011+0000 7f15d7400700 0 client.30221 ms_handle_reset on v2:172.21.15.142:6834/1052606547 2024-02-02T15:44:58.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.011+0000 7f1604600b80 1 -- 172.21.15.1:0/312658201 shutdown_connections 2024-02-02T15:44:58.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.011+0000 7f1604600b80 1 -- 172.21.15.1:0/312658201 wait complete. 2024-02-02T15:44:58.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.012+0000 7f1604600b80 20 client.30221 trim_cache size 0 max 16384 2024-02-02T15:44:58.012 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:44:58.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.013+0000 7f15f0a00700 1 -- 172.21.15.1:0/4258128008 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f15c80a0710 msgr2=0x7f15c80a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:58.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.013+0000 7f15f0a00700 1 --2- 172.21.15.1:0/4258128008 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f15c80a0710 0x7f15c80a2bd0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f15d0019000 tx=0x7f15d001b000 comp rx=0 tx=0).stop 2024-02-02T15:44:58.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.013+0000 7f15f0a00700 1 -- 172.21.15.1:0/4258128008 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15e8067bf0 msgr2=0x7f15e810b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:58.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.013+0000 7f15f0a00700 1 --2- 172.21.15.1:0/4258128008 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15e8067bf0 0x7f15e810b9f0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f15d800c940 tx=0x7f15d800cc50 comp rx=0 tx=0).stop 2024-02-02T15:44:58.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.013+0000 7f15f0a00700 1 -- 172.21.15.1:0/4258128008 shutdown_connections 2024-02-02T15:44:58.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.013+0000 7f15f0a00700 1 --2- 172.21.15.1:0/4258128008 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f15c80a0710 0x7f15c80a2bd0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:58.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.013+0000 7f15f0a00700 1 --2- 172.21.15.1:0/4258128008 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f15e80ff5f0 0x7f15e81a51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:58.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.013+0000 7f15f0a00700 1 --2- 172.21.15.1:0/4258128008 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f15e8067780 0x7f15e810b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:58.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.013+0000 7f15f0a00700 1 --2- 172.21.15.1:0/4258128008 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15e8067bf0 0x7f15e810b9f0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:58.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.013+0000 7f15f0a00700 1 -- 172.21.15.1:0/4258128008 >> 172.21.15.1:0/4258128008 conn(0x7f15e8074f30 msgr2=0x7f15e80fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:58.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.014+0000 7f15f0a00700 1 -- 172.21.15.1:0/4258128008 shutdown_connections 2024-02-02T15:44:58.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.014+0000 7f15f0a00700 1 -- 172.21.15.1:0/4258128008 wait complete. 2024-02-02T15:44:58.041 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-02-02T15:44:58.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.320+0000 7f3dade00700 1 -- 172.21.15.1:0/64183992 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3da81019b0 msgr2=0x7f3da81059f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:58.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.320+0000 7f3dade00700 1 --2- 172.21.15.1:0/64183992 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3da81019b0 0x7f3da81059f0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f3d90009990 tx=0x7f3d90009ca0 comp rx=0 tx=0).stop 2024-02-02T15:44:58.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.320+0000 7f3dade00700 1 -- 172.21.15.1:0/64183992 shutdown_connections 2024-02-02T15:44:58.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.320+0000 7f3dade00700 1 --2- 172.21.15.1:0/64183992 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3da8106120 0x7f3da810dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:58.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.320+0000 7f3dade00700 1 --2- 172.21.15.1:0/64183992 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3da8101000 0x7f3da81013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:58.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.320+0000 7f3dade00700 1 --2- 172.21.15.1:0/64183992 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3da81019b0 0x7f3da81059f0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:58.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.320+0000 7f3dade00700 1 -- 172.21.15.1:0/64183992 >> 172.21.15.1:0/64183992 conn(0x7f3da80fc840 msgr2=0x7f3da80fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:58.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.320+0000 7f3dade00700 1 -- 172.21.15.1:0/64183992 shutdown_connections 2024-02-02T15:44:58.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.320+0000 7f3dade00700 1 -- 172.21.15.1:0/64183992 wait complete. 2024-02-02T15:44:58.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.322+0000 7f3dade00700 1 Processor -- start 2024-02-02T15:44:58.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.322+0000 7f3dade00700 1 -- start start 2024-02-02T15:44:58.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.323+0000 7f3dade00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3da8101000 0x7f3da810fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:58.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.323+0000 7f3dade00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3da8106120 0x7f3da8110380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:58.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.323+0000 7f3dade00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3da8110960 0x7f3da81149c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:58.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.323+0000 7f3dade00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3da8115150 con 0x7f3da8110960 2024-02-02T15:44:58.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.323+0000 7f3dade00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3da81152c0 con 0x7f3da8101000 2024-02-02T15:44:58.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.323+0000 7f3dade00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3da8115480 con 0x7f3da8106120 2024-02-02T15:44:58.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.324+0000 7f3da6a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3da8101000 0x7f3da810fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:58.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.324+0000 7f3da6a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3da8101000 0x7f3da810fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:35244/0 (socket says 172.21.15.1:35244) 2024-02-02T15:44:58.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.324+0000 7f3da6a00700 1 -- 172.21.15.1:0/2955901564 learned_addr learned my addr 172.21.15.1:0/2955901564 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:44:58.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.324+0000 7f3da7400700 1 --2- 172.21.15.1:0/2955901564 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3da8110960 0x7f3da81149c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:58.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.324+0000 7f3da6a00700 1 -- 172.21.15.1:0/2955901564 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3da8106120 msgr2=0x7f3da8110380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:58.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.324+0000 7f3da6a00700 1 --2- 172.21.15.1:0/2955901564 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3da8106120 0x7f3da8110380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:58.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.324+0000 7f3da6a00700 1 -- 172.21.15.1:0/2955901564 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3da8110960 msgr2=0x7f3da81149c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:58.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.324+0000 7f3da6a00700 1 --2- 172.21.15.1:0/2955901564 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3da8110960 0x7f3da81149c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:58.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.324+0000 7f3da6a00700 1 -- 172.21.15.1:0/2955901564 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d900095f0 con 0x7f3da8101000 2024-02-02T15:44:58.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.324+0000 7f3da6a00700 1 --2- 172.21.15.1:0/2955901564 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3da8101000 0x7f3da810fe40 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f3d9800e9a0 tx=0x7f3d9800ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:58.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.325+0000 7f3daca00700 1 -- 172.21.15.1:0/2955901564 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d98009790 con 0x7f3da8101000 2024-02-02T15:44:58.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.325+0000 7f3daca00700 1 -- 172.21.15.1:0/2955901564 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d9800cbd0 con 0x7f3da8101000 2024-02-02T15:44:58.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.325+0000 7f3daca00700 1 -- 172.21.15.1:0/2955901564 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d98018680 con 0x7f3da8101000 2024-02-02T15:44:58.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.325+0000 7f3dade00700 1 -- 172.21.15.1:0/2955901564 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3da8115760 con 0x7f3da8101000 2024-02-02T15:44:58.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.326+0000 7f3dade00700 1 -- 172.21.15.1:0/2955901564 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3da81b2f60 con 0x7f3da8101000 2024-02-02T15:44:58.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.332+0000 7f3daca00700 1 -- 172.21.15.1:0/2955901564 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f3d9800cd40 con 0x7f3da8101000 2024-02-02T15:44:58.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.334+0000 7f3dc185bb80 1 Processor -- start 2024-02-02T15:44:58.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.334+0000 7f3dc185bb80 1 -- start start 2024-02-02T15:44:58.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.334+0000 7f3dc185bb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d786897010 0x55d7868973f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:58.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.334+0000 7f3dc185bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d786897930 0x55d7868a22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:58.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.334+0000 7f3dc185bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7868a27e0 0x55d7868a4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:58.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.334+0000 7f3dc185bb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d78680b4c0 con 0x55d786897010 2024-02-02T15:44:58.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.334+0000 7f3dc185bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7867efaa0 con 0x55d786897930 2024-02-02T15:44:58.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.334+0000 7f3dc185bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7867ed460 con 0x55d7868a27e0 2024-02-02T15:44:58.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.335+0000 7f3daca00700 1 --2- 172.21.15.1:0/2955901564 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f3d880a0840 0x7f3d880a2d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:58.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.335+0000 7f3da6a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d786897010 0x55d7868973f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:58.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.336+0000 7f3da7400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7868a27e0 0x55d7868a4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:58.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.336+0000 7f3da6000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d786897930 0x55d7868a22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:58.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.336+0000 7f3daca00700 1 -- 172.21.15.1:0/2955901564 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f3d98014070 con 0x7f3da8101000 2024-02-02T15:44:58.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.336+0000 7f3da6000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d786897930 0x55d7868a22a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:35260/0 (socket says 172.21.15.1:35260) 2024-02-02T15:44:58.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.336+0000 7f3da6000700 1 -- 172.21.15.1:0/1856408814 learned_addr learned my addr 172.21.15.1:0/1856408814 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:44:58.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.336+0000 7f3da7400700 1 --2- 172.21.15.1:0/2955901564 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f3d880a0840 0x7f3d880a2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:58.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.337+0000 7f3da6000700 1 -- 172.21.15.1:0/1856408814 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7868a27e0 msgr2=0x55d7868a4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:58.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.337+0000 7f3da6000700 1 --2- 172.21.15.1:0/1856408814 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7868a27e0 0x55d7868a4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:58.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.337+0000 7f3da6000700 1 -- 172.21.15.1:0/1856408814 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d786897010 msgr2=0x55d7868973f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:58.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.337+0000 7f3da6000700 1 --2- 172.21.15.1:0/1856408814 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d786897010 0x55d7868973f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:58.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.337+0000 7f3da6000700 1 -- 172.21.15.1:0/1856408814 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d78666a3b0 con 0x55d786897930 2024-02-02T15:44:58.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.337+0000 7f3da6a00700 1 --2- 172.21.15.1:0/1856408814 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d786897010 0x55d7868973f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:44:58.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.338+0000 7f3da6000700 1 --2- 172.21.15.1:0/1856408814 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d786897930 0x55d7868a22a0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f3d90009530 tx=0x7f3d90009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ee868149c729f7e server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:58.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.338+0000 7f3da7400700 1 --2- 172.21.15.1:0/2955901564 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f3d880a0840 0x7f3d880a2d00 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f3da0012860 tx=0x7f3da000b040 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:58.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.338+0000 7f3d97e00700 1 -- 172.21.15.1:0/1856408814 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d9001d070 con 0x55d786897930 2024-02-02T15:44:58.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.338+0000 7f3d97e00700 1 -- 172.21.15.1:0/1856408814 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d90004cf0 con 0x55d786897930 2024-02-02T15:44:58.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.338+0000 7f3d97e00700 1 -- 172.21.15.1:0/1856408814 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d90005020 con 0x55d786897930 2024-02-02T15:44:58.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.339+0000 7f3dc185bb80 1 -- 172.21.15.1:0/1856408814 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d786897930 msgr2=0x55d7868a22a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:58.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.339+0000 7f3dc185bb80 1 --2- 172.21.15.1:0/1856408814 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d786897930 0x55d7868a22a0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f3d90009530 tx=0x7f3d90009560 comp rx=0 tx=0).stop 2024-02-02T15:44:58.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.339+0000 7f3dc185bb80 1 -- 172.21.15.1:0/1856408814 shutdown_connections 2024-02-02T15:44:58.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.339+0000 7f3dc185bb80 1 --2- 172.21.15.1:0/1856408814 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7868a27e0 0x55d7868a4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:58.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.339+0000 7f3dc185bb80 1 --2- 172.21.15.1:0/1856408814 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d786897010 0x55d7868973f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:58.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.339+0000 7f3dc185bb80 1 --2- 172.21.15.1:0/1856408814 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d786897930 0x55d7868a22a0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:58.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.339+0000 7f3dc185bb80 1 -- 172.21.15.1:0/1856408814 >> 172.21.15.1:0/1856408814 conn(0x55d7867f6db0 msgr2=0x55d7868933b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:58.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.340+0000 7f3dc185bb80 1 -- 172.21.15.1:0/1856408814 shutdown_connections 2024-02-02T15:44:58.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.340+0000 7f3dc185bb80 1 -- 172.21.15.1:0/1856408814 wait complete. 2024-02-02T15:44:58.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.340+0000 7f3dc185bb80 1 -- start start 2024-02-02T15:44:58.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.341+0000 7f3dc185bb80 1 Processor -- start 2024-02-02T15:44:58.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.341+0000 7f3dc185bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d786897010 0x55d7868a21c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:58.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.341+0000 7f3dc185bb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d786897930 0x55d78689f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:58.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.341+0000 7f3dc185bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7868a27e0 0x55d78689f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:58.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.341+0000 7f3dc185bb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7867e8d50 con 0x55d786897930 2024-02-02T15:44:58.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.341+0000 7f3dc185bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d786784d80 con 0x55d7868a27e0 2024-02-02T15:44:58.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.341+0000 7f3dc185bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d786784a20 con 0x55d786897010 2024-02-02T15:44:58.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.341+0000 7f3dc185bb80 10 client.0 _ll_register_callbacks cb 0x55d786786290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:44:58.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.342+0000 7f3da7400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7868a27e0 0x55d78689f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:58.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.342+0000 7f3da6a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d786897010 0x55d7868a21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:58.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.342+0000 7f3da7400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7868a27e0 0x55d78689f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:35274/0 (socket says 172.21.15.1:35274) 2024-02-02T15:44:58.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.342+0000 7f3da6a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d786897010 0x55d7868a21c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:50484/0 (socket says 172.21.15.1:50484) 2024-02-02T15:44:58.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.342+0000 7f3da7400700 1 -- 172.21.15.1:0/3161345657 learned_addr learned my addr 172.21.15.1:0/3161345657 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:44:58.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.342+0000 7f3da7400700 1 -- 172.21.15.1:0/3161345657 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d786897010 msgr2=0x55d7868a21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:58.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.342+0000 7f3da7400700 1 --2- 172.21.15.1:0/3161345657 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d786897010 0x55d7868a21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:58.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.342+0000 7f3da7400700 1 -- 172.21.15.1:0/3161345657 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d786897930 msgr2=0x55d78689f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:58.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.342+0000 7f3da7400700 1 --2- 172.21.15.1:0/3161345657 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d786897930 0x55d78689f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:58.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.342+0000 7f3da7400700 1 -- 172.21.15.1:0/3161345657 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3da81b2f60 con 0x55d7868a27e0 2024-02-02T15:44:58.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.343+0000 7f3da7400700 1 --2- 172.21.15.1:0/3161345657 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7868a27e0 0x55d78689f9c0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f3da0007ed0 tx=0x7f3da0004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb6413af4104012c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:58.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.343+0000 7f3d94c00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:44:58.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.343+0000 7f3d94c00700 1 -- 172.21.15.1:0/3161345657 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da001f070 con 0x55d7868a27e0 2024-02-02T15:44:58.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.343+0000 7f3dc185bb80 1 -- 172.21.15.1:0/3161345657 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d78688b0f0 con 0x55d7868a27e0 2024-02-02T15:44:58.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.343+0000 7f3d94c00700 1 -- 172.21.15.1:0/3161345657 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f3da001a2e0 con 0x55d7868a27e0 2024-02-02T15:44:58.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.344+0000 7f3d94c00700 1 -- 172.21.15.1:0/3161345657 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3da000e040 con 0x55d7868a27e0 2024-02-02T15:44:58.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.344+0000 7f3dc185bb80 10 client.30233 fetch_fsmap learned FSMap version 77 2024-02-02T15:44:58.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.344+0000 7f3dc185bb80 1 -- 172.21.15.1:0/3161345657 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=4+}) v3 -- 0x7f3da810e510 con 0x55d7868a27e0 2024-02-02T15:44:58.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.344+0000 7f3d94c00700 1 -- 172.21.15.1:0/3161345657 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55d78688b0f0 con 0x55d7868a27e0 2024-02-02T15:44:58.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.344+0000 7f3d94c00700 1 -- 172.21.15.1:0/3161345657 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 77) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f3da00040b0 con 0x55d7868a27e0 2024-02-02T15:44:58.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.344+0000 7f3dc185bb80 10 client.30233 fetch_fsmap finished waiting for FSMap version 77 2024-02-02T15:44:58.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.344+0000 7f3dc185bb80 10 client.30233 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-02-02T15:44:58.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.344+0000 7f3dc185bb80 20 client.30233 populate_metadata read hostname 'smithi001' 2024-02-02T15:44:58.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.344+0000 7f3dc185bb80 1 --2- 172.21.15.1:0/3161345657 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55d7868b2240 0x55d7868b4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:58.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.344+0000 7f3dc185bb80 4 client.30233 mds_command: new command op to 24514 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:44:58.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.344+0000 7f3dc185bb80 1 -- 172.21.15.1:0/3161345657 --> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d7867e1dd0 con 0x55d7868b2240 2024-02-02T15:44:58.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.345+0000 7f3da6a00700 1 --2- 172.21.15.1:0/3161345657 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55d7868b2240 0x55d7868b4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:58.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.345+0000 7f3da6a00700 1 --2- 172.21.15.1:0/3161345657 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55d7868b2240 0x55d7868b4660 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2e5c4e2422abb542 server_cookie=4d020181495ac7e0 in_seq=0 out_seq=0 2024-02-02T15:44:58.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.345+0000 7f3d94c00700 10 client.30233 ms_handle_connect on v2:172.21.15.142:6836/2870289019 2024-02-02T15:44:58.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.349+0000 7f3d94c00700 1 -- 172.21.15.1:0/3161345657 <== mds.3 v2:172.21.15.142:6836/2870289019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55d7867e1dd0 con 0x55d7868b2240 2024-02-02T15:44:58.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.349+0000 7f3d94c00700 10 client.30233 handle_command_reply: tid=0 2024-02-02T15:44:58.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.349+0000 7f3dc185bb80 1 client.30233 shutdown 2024-02-02T15:44:58.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.349+0000 7f3dc185bb80 1 -- 172.21.15.1:0/3161345657 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7868a27e0 msgr2=0x55d78689f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:58.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.349+0000 7f3dc185bb80 1 --2- 172.21.15.1:0/3161345657 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7868a27e0 0x55d78689f9c0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f3da0007ed0 tx=0x7f3da0004b40 comp rx=0 tx=0).stop 2024-02-02T15:44:58.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.350+0000 7f3dc185bb80 1 -- 172.21.15.1:0/3161345657 shutdown_connections 2024-02-02T15:44:58.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.350+0000 7f3dc185bb80 1 --2- 172.21.15.1:0/3161345657 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7868a27e0 0x55d78689f9c0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:58.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.350+0000 7f3dc185bb80 1 --2- 172.21.15.1:0/3161345657 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d786897010 0x55d7868a21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:58.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.350+0000 7f3dc185bb80 1 --2- 172.21.15.1:0/3161345657 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55d7868b2240 0x55d7868b4660 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:58.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.350+0000 7f3dc185bb80 1 --2- 172.21.15.1:0/3161345657 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d786897930 0x55d78689f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:58.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.350+0000 7f3dc185bb80 1 -- 172.21.15.1:0/3161345657 >> 172.21.15.1:0/3161345657 conn(0x55d7867f6db0 msgr2=0x55d786892fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:58.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.350+0000 7f3d94c00700 0 client.30233 ms_handle_reset on v2:172.21.15.142:6836/2870289019 2024-02-02T15:44:58.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.350+0000 7f3dc185bb80 1 -- 172.21.15.1:0/3161345657 shutdown_connections 2024-02-02T15:44:58.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.350+0000 7f3dc185bb80 1 -- 172.21.15.1:0/3161345657 wait complete. 2024-02-02T15:44:58.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.350+0000 7f3dc185bb80 20 client.30233 trim_cache size 0 max 16384 2024-02-02T15:44:58.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.372+0000 7f3dc185bb80 1 Processor -- start 2024-02-02T15:44:58.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.372+0000 7f3dc185bb80 1 -- start start 2024-02-02T15:44:58.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.372+0000 7f3dc185bb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d786897010 0x55d7868a92a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:58.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.372+0000 7f3dc185bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d786897930 0x55d7868a1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:58.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.373+0000 7f3dc185bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7868a27e0 0x55d78689a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:58.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.373+0000 7f3dc185bb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d78666c6a0 con 0x55d786897010 2024-02-02T15:44:58.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.373+0000 7f3dc185bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d78688d500 con 0x55d786897930 2024-02-02T15:44:58.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.373+0000 7f3dc185bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d786898d50 con 0x55d7868a27e0 2024-02-02T15:44:58.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.373+0000 7f3da6000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d786897930 0x55d7868a1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:58.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.373+0000 7f3da6000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d786897930 0x55d7868a1cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:35278/0 (socket says 172.21.15.1:35278) 2024-02-02T15:44:58.373 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.373+0000 7f3da6000700 1 -- 172.21.15.1:0/2611647983 learned_addr learned my addr 172.21.15.1:0/2611647983 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:44:58.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.374+0000 7f3da6a00700 1 --2- 172.21.15.1:0/2611647983 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d786897010 0x55d7868a92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:58.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.374+0000 7f3da6000700 1 -- 172.21.15.1:0/2611647983 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7868a27e0 msgr2=0x55d78689a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:58.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.374+0000 7f3da6000700 1 --2- 172.21.15.1:0/2611647983 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7868a27e0 0x55d78689a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:58.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.374+0000 7f3da6000700 1 -- 172.21.15.1:0/2611647983 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d786897010 msgr2=0x55d7868a92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:58.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.374+0000 7f3da6000700 1 --2- 172.21.15.1:0/2611647983 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d786897010 0x55d7868a92a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:58.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.374+0000 7f3da6000700 1 -- 172.21.15.1:0/2611647983 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d786616b60 con 0x55d786897930 2024-02-02T15:44:58.374 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.374+0000 7f3da6a00700 1 --2- 172.21.15.1:0/2611647983 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d786897010 0x55d7868a92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:44:58.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.375+0000 7f3da6000700 1 --2- 172.21.15.1:0/2611647983 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d786897930 0x55d7868a1cf0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f3d90005b30 tx=0x7f3d9000b5b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c8bf7ef0055a914 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:58.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.375+0000 7f3d9d200700 1 -- 172.21.15.1:0/2611647983 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d9001d050 con 0x55d786897930 2024-02-02T15:44:58.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.375+0000 7f3d9d200700 1 -- 172.21.15.1:0/2611647983 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d9001e040 con 0x55d786897930 2024-02-02T15:44:58.375 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.375+0000 7f3d9d200700 1 -- 172.21.15.1:0/2611647983 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d90005350 con 0x55d786897930 2024-02-02T15:44:58.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.376+0000 7f3dc185bb80 1 -- 172.21.15.1:0/2611647983 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d786897930 msgr2=0x55d7868a1cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:58.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.376+0000 7f3dc185bb80 1 --2- 172.21.15.1:0/2611647983 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d786897930 0x55d7868a1cf0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f3d90005b30 tx=0x7f3d9000b5b0 comp rx=0 tx=0).stop 2024-02-02T15:44:58.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.377+0000 7f3dc185bb80 1 -- 172.21.15.1:0/2611647983 shutdown_connections 2024-02-02T15:44:58.376 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.377+0000 7f3dc185bb80 1 --2- 172.21.15.1:0/2611647983 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7868a27e0 0x55d78689a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:58.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.377+0000 7f3dc185bb80 1 --2- 172.21.15.1:0/2611647983 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d786897010 0x55d7868a92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:58.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.377+0000 7f3dc185bb80 1 --2- 172.21.15.1:0/2611647983 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d786897930 0x55d7868a1cf0 secure :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f3d90005b30 tx=0x7f3d9000b5b0 comp rx=0 tx=0).stop 2024-02-02T15:44:58.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.377+0000 7f3dc185bb80 1 -- 172.21.15.1:0/2611647983 >> 172.21.15.1:0/2611647983 conn(0x55d7867f6db0 msgr2=0x55d7868ad880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:58.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.377+0000 7f3dc185bb80 1 -- 172.21.15.1:0/2611647983 shutdown_connections 2024-02-02T15:44:58.377 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.377+0000 7f3dc185bb80 1 -- 172.21.15.1:0/2611647983 wait complete. 2024-02-02T15:44:58.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.377+0000 7f3dc185bb80 1 -- start start 2024-02-02T15:44:58.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.378+0000 7f3dc185bb80 1 Processor -- start 2024-02-02T15:44:58.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.379+0000 7f3dc185bb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d786897010 0x55d7868a91e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:58.378 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.379+0000 7f3dc185bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7868a27e0 0x55d7868a6460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:58.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.379+0000 7f3dc185bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7868a69a0 0x55d7868a6e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:58.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.379+0000 7f3dc185bb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d786902e10 con 0x55d786897010 2024-02-02T15:44:58.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.379+0000 7f3dc185bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d786902f80 con 0x55d7868a27e0 2024-02-02T15:44:58.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.379+0000 7f3dc185bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d786903140 con 0x55d7868a69a0 2024-02-02T15:44:58.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.379+0000 7f3dc185bb80 10 client.0 _ll_register_callbacks cb 0x55d786786290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:44:58.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.379+0000 7f3da6000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7868a27e0 0x55d7868a6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:58.379 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.379+0000 7f3da6a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d786897010 0x55d7868a91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:58.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.379+0000 7f3da6000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7868a27e0 0x55d7868a6460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:35284/0 (socket says 172.21.15.1:35284) 2024-02-02T15:44:58.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.379+0000 7f3da6000700 1 -- 172.21.15.1:0/2836061037 learned_addr learned my addr 172.21.15.1:0/2836061037 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:44:58.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.379+0000 7f3da7400700 1 --2- 172.21.15.1:0/2836061037 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7868a69a0 0x55d7868a6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:58.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.380+0000 7f3da6a00700 1 -- 172.21.15.1:0/2836061037 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7868a69a0 msgr2=0x55d7868a6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:58.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.380+0000 7f3da6a00700 1 --2- 172.21.15.1:0/2836061037 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7868a69a0 0x55d7868a6e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:58.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.380+0000 7f3da6a00700 1 -- 172.21.15.1:0/2836061037 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7868a27e0 msgr2=0x55d7868a6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:58.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.380+0000 7f3da6a00700 1 --2- 172.21.15.1:0/2836061037 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7868a27e0 0x55d7868a6460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:58.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.380+0000 7f3da6a00700 1 -- 172.21.15.1:0/2836061037 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3da8115760 con 0x55d786897010 2024-02-02T15:44:58.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.380+0000 7f3da6000700 1 --2- 172.21.15.1:0/2836061037 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7868a27e0 0x55d7868a6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:44:58.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.381+0000 7f3da6a00700 1 --2- 172.21.15.1:0/2836061037 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d786897010 0x55d7868a91e0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f3d980c9720 tx=0x7f3d98069720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33f07cba1357f457 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:58.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.381+0000 7f3d94c00700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:44:58.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.381+0000 7f3d94c00700 1 -- 172.21.15.1:0/2836061037 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d98099950 con 0x55d786897010 2024-02-02T15:44:58.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.381+0000 7f3d94c00700 1 -- 172.21.15.1:0/2836061037 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f3d980994f0 con 0x55d786897010 2024-02-02T15:44:58.381 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.381+0000 7f3dc185bb80 1 -- 172.21.15.1:0/2836061037 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3d90004cf0 con 0x55d786897010 2024-02-02T15:44:58.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.382+0000 7f3d94c00700 1 -- 172.21.15.1:0/2836061037 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3d9809d130 con 0x55d786897010 2024-02-02T15:44:58.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.382+0000 7f3dc185bb80 10 client.19950 fetch_fsmap learned FSMap version 77 2024-02-02T15:44:58.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.382+0000 7f3dc185bb80 1 -- 172.21.15.1:0/2836061037 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=4+}) v3 -- 0x55d78666a3b0 con 0x55d786897010 2024-02-02T15:44:58.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.382+0000 7f3d94c00700 1 -- 172.21.15.1:0/2836061037 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3d90004cf0 con 0x55d786897010 2024-02-02T15:44:58.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.382+0000 7f3d94c00700 1 -- 172.21.15.1:0/2836061037 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 77) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f3d980a10d0 con 0x55d786897010 2024-02-02T15:44:58.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.383+0000 7f3dc185bb80 10 client.19950 fetch_fsmap finished waiting for FSMap version 77 2024-02-02T15:44:58.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.383+0000 7f3dc185bb80 10 client.19950 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-02-02T15:44:58.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.383+0000 7f3dc185bb80 20 client.19950 populate_metadata read hostname 'smithi001' 2024-02-02T15:44:58.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.383+0000 7f3dc185bb80 1 --2- 172.21.15.1:0/2836061037 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55d786904910 0x55d786906d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:58.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.383+0000 7f3dc185bb80 4 client.19950 mds_command: new command op to 24514 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:44:58.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.383+0000 7f3dc185bb80 1 -- 172.21.15.1:0/2836061037 --> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d7867a7f50 con 0x55d786904910 2024-02-02T15:44:58.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.384+0000 7f3da6000700 1 --2- 172.21.15.1:0/2836061037 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55d786904910 0x55d786906d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:58.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.385+0000 7f3da6000700 1 --2- 172.21.15.1:0/2836061037 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55d786904910 0x55d786906d30 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e55c887f269c2f1d server_cookie=ce8c4e02807d3894 in_seq=0 out_seq=0 2024-02-02T15:44:58.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.385+0000 7f3d94c00700 10 client.19950 ms_handle_connect on v2:172.21.15.142:6836/2870289019 2024-02-02T15:44:58.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.386+0000 7f3d94c00700 1 -- 172.21.15.1:0/2836061037 <== mds.3 v2:172.21.15.142:6836/2870289019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d7867a7f50 con 0x55d786904910 2024-02-02T15:44:58.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.386+0000 7f3d94c00700 10 client.19950 handle_command_reply: tid=0 2024-02-02T15:44:58.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.386+0000 7f3dc185bb80 1 client.19950 shutdown 2024-02-02T15:44:58.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.386+0000 7f3dc185bb80 1 -- 172.21.15.1:0/2836061037 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d786897010 msgr2=0x55d7868a91e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:58.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.386+0000 7f3dc185bb80 1 --2- 172.21.15.1:0/2836061037 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d786897010 0x55d7868a91e0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f3d980c9720 tx=0x7f3d98069720 comp rx=0 tx=0).stop 2024-02-02T15:44:58.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.387+0000 7f3dc185bb80 1 -- 172.21.15.1:0/2836061037 shutdown_connections 2024-02-02T15:44:58.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.387+0000 7f3dc185bb80 1 --2- 172.21.15.1:0/2836061037 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55d7868a69a0 0x55d7868a6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:58.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.387+0000 7f3dc185bb80 1 --2- 172.21.15.1:0/2836061037 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55d786904910 0x55d786906d30 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:58.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.387+0000 7f3dc185bb80 1 --2- 172.21.15.1:0/2836061037 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55d786897010 0x55d7868a91e0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:58.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.387+0000 7f3dc185bb80 1 --2- 172.21.15.1:0/2836061037 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55d7868a27e0 0x55d7868a6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:58.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.387+0000 7f3dc185bb80 1 -- 172.21.15.1:0/2836061037 >> 172.21.15.1:0/2836061037 conn(0x55d7867f6db0 msgr2=0x55d7868ad880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:58.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.387+0000 7f3d94c00700 0 client.19950 ms_handle_reset on v2:172.21.15.142:6836/2870289019 2024-02-02T15:44:58.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.387+0000 7f3dc185bb80 1 -- 172.21.15.1:0/2836061037 shutdown_connections 2024-02-02T15:44:58.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.387+0000 7f3dc185bb80 1 -- 172.21.15.1:0/2836061037 wait complete. 2024-02-02T15:44:58.388 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.388+0000 7f3dc185bb80 20 client.19950 trim_cache size 0 max 16384 2024-02-02T15:44:58.388 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:44:58.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.388+0000 7f3dade00700 1 -- 172.21.15.1:0/2955901564 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f3d880a0840 msgr2=0x7f3d880a2d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:58.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.388+0000 7f3dade00700 1 --2- 172.21.15.1:0/2955901564 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f3d880a0840 0x7f3d880a2d00 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f3da0012860 tx=0x7f3da000b040 comp rx=0 tx=0).stop 2024-02-02T15:44:58.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.388+0000 7f3dade00700 1 -- 172.21.15.1:0/2955901564 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3da8101000 msgr2=0x7f3da810fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:58.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.388+0000 7f3dade00700 1 --2- 172.21.15.1:0/2955901564 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3da8101000 0x7f3da810fe40 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f3d9800e9a0 tx=0x7f3d9800ed60 comp rx=0 tx=0).stop 2024-02-02T15:44:58.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.389+0000 7f3dade00700 1 -- 172.21.15.1:0/2955901564 shutdown_connections 2024-02-02T15:44:58.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.389+0000 7f3dade00700 1 --2- 172.21.15.1:0/2955901564 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f3d880a0840 0x7f3d880a2d00 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:58.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.389+0000 7f3dade00700 1 --2- 172.21.15.1:0/2955901564 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3da8110960 0x7f3da81149c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:58.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.389+0000 7f3dade00700 1 --2- 172.21.15.1:0/2955901564 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3da8101000 0x7f3da810fe40 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:58.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.389+0000 7f3dade00700 1 --2- 172.21.15.1:0/2955901564 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3da8106120 0x7f3da8110380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:58.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.389+0000 7f3dade00700 1 -- 172.21.15.1:0/2955901564 >> 172.21.15.1:0/2955901564 conn(0x7f3da80fc840 msgr2=0x7f3da80fd2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:58.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.389+0000 7f3dade00700 1 -- 172.21.15.1:0/2955901564 shutdown_connections 2024-02-02T15:44:58.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.389+0000 7f3dade00700 1 -- 172.21.15.1:0/2955901564 wait complete. 2024-02-02T15:44:58.411 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-02T15:44:58.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:44:58 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/3416545446' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:44:58.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.718+0000 7f62f9800700 1 -- 172.21.15.1:0/3509539082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f62f4111cc0 msgr2=0x7f62f41140b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:58.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.718+0000 7f62f9800700 1 --2- 172.21.15.1:0/3509539082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f62f4111cc0 0x7f62f41140b0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f62ec009990 tx=0x7f62ec009ca0 comp rx=0 tx=0).stop 2024-02-02T15:44:58.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.720+0000 7f62f9800700 1 -- 172.21.15.1:0/3509539082 shutdown_connections 2024-02-02T15:44:58.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.720+0000 7f62f9800700 1 --2- 172.21.15.1:0/3509539082 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f62f4111cc0 0x7f62f41140b0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:58.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.720+0000 7f62f9800700 1 --2- 172.21.15.1:0/3509539082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f62f4075270 0x7f62f4075650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:58.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.720+0000 7f62f9800700 1 --2- 172.21.15.1:0/3509539082 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f62f4075b90 0x7f62f4111780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:58.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.720+0000 7f62f9800700 1 -- 172.21.15.1:0/3509539082 >> 172.21.15.1:0/3509539082 conn(0x7f62f40feb30 msgr2=0x7f62f4100f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:58.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.721+0000 7f62f9800700 1 -- 172.21.15.1:0/3509539082 shutdown_connections 2024-02-02T15:44:58.725 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.721+0000 7f62f9800700 1 -- 172.21.15.1:0/3509539082 wait complete. 2024-02-02T15:44:58.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.722+0000 7f62f9800700 1 Processor -- start 2024-02-02T15:44:58.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.723+0000 7f62f9800700 1 -- start start 2024-02-02T15:44:58.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.723+0000 7f62f9800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f62f4075270 0x7f62f406b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:58.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.723+0000 7f62f9800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f62f4075b90 0x7f62f406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:58.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.723+0000 7f62f9800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f62f4111cc0 0x7f62f41b1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:58.726 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.723+0000 7f62f9800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62f41b2040 con 0x7f62f4075b90 2024-02-02T15:44:58.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.723+0000 7f62f9800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62f4102f60 con 0x7f62f4111cc0 2024-02-02T15:44:58.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.723+0000 7f62f9800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62f4103120 con 0x7f62f4075270 2024-02-02T15:44:58.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.724+0000 7f62e9e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f62f4075b90 0x7f62f406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:58.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.724+0000 7f62e9e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f62f4075b90 0x7f62f406b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44074/0 (socket says 172.21.15.1:44074) 2024-02-02T15:44:58.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.724+0000 7f62e9e00700 1 -- 172.21.15.1:0/1773978316 learned_addr learned my addr 172.21.15.1:0/1773978316 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:44:58.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.724+0000 7f62f3400700 1 --2- 172.21.15.1:0/1773978316 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f62f4111cc0 0x7f62f41b1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:58.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.725+0000 7f62f3400700 1 -- 172.21.15.1:0/1773978316 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f62f4075270 msgr2=0x7f62f406b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:58.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.725+0000 7f62f3400700 1 --2- 172.21.15.1:0/1773978316 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f62f4075270 0x7f62f406b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:58.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.725+0000 7f62f3400700 1 -- 172.21.15.1:0/1773978316 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f62f4075b90 msgr2=0x7f62f406b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:58.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.725+0000 7f62f3400700 1 --2- 172.21.15.1:0/1773978316 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f62f4075b90 0x7f62f406b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:58.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.725+0000 7f62f3400700 1 -- 172.21.15.1:0/1773978316 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62ec0095f0 con 0x7f62f4111cc0 2024-02-02T15:44:58.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.725+0000 7f62f3400700 1 --2- 172.21.15.1:0/1773978316 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f62f4111cc0 0x7f62f41b1ad0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f62ec0098e0 tx=0x7f62ec00f670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:58.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.726+0000 7f62ebe00700 1 -- 172.21.15.1:0/1773978316 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62ec01d070 con 0x7f62f4111cc0 2024-02-02T15:44:58.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.726+0000 7f62ebe00700 1 -- 172.21.15.1:0/1773978316 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f62ec003680 con 0x7f62f4111cc0 2024-02-02T15:44:58.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.726+0000 7f62ebe00700 1 -- 172.21.15.1:0/1773978316 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62ec021db0 con 0x7f62f4111cc0 2024-02-02T15:44:58.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.726+0000 7f62f9800700 1 -- 172.21.15.1:0/1773978316 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62f41033a0 con 0x7f62f4111cc0 2024-02-02T15:44:58.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.727+0000 7f62f9800700 1 -- 172.21.15.1:0/1773978316 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62f4103800 con 0x7f62f4111cc0 2024-02-02T15:44:58.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.733+0000 7f62ebe00700 1 -- 172.21.15.1:0/1773978316 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f62ec021410 con 0x7f62f4111cc0 2024-02-02T15:44:58.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.733+0000 7f630d3f7b80 1 Processor -- start 2024-02-02T15:44:58.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.736+0000 7f62ebe00700 1 --2- 172.21.15.1:0/1773978316 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f62d40a0820 0x7f62d40a2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:58.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.736+0000 7f62ebe00700 1 -- 172.21.15.1:0/1773978316 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f62ec0ccde0 con 0x7f62f4111cc0 2024-02-02T15:44:58.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.736+0000 7f630d3f7b80 1 -- start start 2024-02-02T15:44:58.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.737+0000 7f630d3f7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aed29f5010 0x55aed29f53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:58.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.737+0000 7f630d3f7b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aed29f5930 0x55aed2a002a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:58.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.737+0000 7f630d3f7b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aed2a007e0 0x55aed2a02bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:58.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.737+0000 7f630d3f7b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aed29694c0 con 0x55aed29f5930 2024-02-02T15:44:58.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.737+0000 7f630d3f7b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aed294daa0 con 0x55aed2a007e0 2024-02-02T15:44:58.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.737+0000 7f630d3f7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aed294b460 con 0x55aed29f5010 2024-02-02T15:44:58.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.737+0000 7f62e9e00700 1 --2- 172.21.15.1:0/1773978316 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f62d40a0820 0x7f62d40a2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:58.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.737+0000 7f62f3400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aed2a007e0 0x55aed2a02bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:58.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.737+0000 7f62f3400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aed2a007e0 0x55aed2a02bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:35298/0 (socket says 172.21.15.1:35298) 2024-02-02T15:44:58.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.737+0000 7f62f3400700 1 -- 172.21.15.1:0/677016266 learned_addr learned my addr 172.21.15.1:0/677016266 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:44:58.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.738+0000 7f62e9e00700 1 --2- 172.21.15.1:0/677016266 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aed29f5930 0x55aed2a002a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:58.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.738+0000 7f62f3400700 1 -- 172.21.15.1:0/677016266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aed29f5010 msgr2=0x55aed29f53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:58.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.738+0000 7f62f3400700 1 --2- 172.21.15.1:0/677016266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aed29f5010 0x55aed29f53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:58.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.738+0000 7f62f3400700 1 -- 172.21.15.1:0/677016266 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aed29f5930 msgr2=0x55aed2a002a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:58.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.738+0000 7f62f3400700 1 --2- 172.21.15.1:0/677016266 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aed29f5930 0x55aed2a002a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:58.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.738+0000 7f62f3400700 1 -- 172.21.15.1:0/677016266 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aed27c83b0 con 0x55aed2a007e0 2024-02-02T15:44:58.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.738+0000 7f62f3400700 1 --2- 172.21.15.1:0/677016266 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aed2a007e0 0x55aed2a02bd0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f62ec099ef0 tx=0x7f62ec021850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1fd897ba0a40adbb server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:58.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.738+0000 7f62e9e00700 1 --2- 172.21.15.1:0/1773978316 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f62d40a0820 0x7f62d40a2ce0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f62dc00b7f0 tx=0x7f62dc0126d0 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:58.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.739+0000 7f62e3e00700 1 -- 172.21.15.1:0/677016266 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62ec0962c0 con 0x55aed2a007e0 2024-02-02T15:44:58.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.739+0000 7f62e3e00700 1 -- 172.21.15.1:0/677016266 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f62ec0b9040 con 0x55aed2a007e0 2024-02-02T15:44:58.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.739+0000 7f62e3e00700 1 -- 172.21.15.1:0/677016266 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62ec09f330 con 0x55aed2a007e0 2024-02-02T15:44:58.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.740+0000 7f630d3f7b80 1 -- 172.21.15.1:0/677016266 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aed2a007e0 msgr2=0x55aed2a02bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:58.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.740+0000 7f630d3f7b80 1 --2- 172.21.15.1:0/677016266 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aed2a007e0 0x55aed2a02bd0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f62ec099ef0 tx=0x7f62ec021850 comp rx=0 tx=0).stop 2024-02-02T15:44:58.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.741+0000 7f630d3f7b80 1 -- 172.21.15.1:0/677016266 shutdown_connections 2024-02-02T15:44:58.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.741+0000 7f630d3f7b80 1 --2- 172.21.15.1:0/677016266 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aed2a007e0 0x55aed2a02bd0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:58.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.741+0000 7f630d3f7b80 1 --2- 172.21.15.1:0/677016266 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aed29f5010 0x55aed29f53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:58.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.741+0000 7f630d3f7b80 1 --2- 172.21.15.1:0/677016266 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aed29f5930 0x55aed2a002a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:58.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.741+0000 7f630d3f7b80 1 -- 172.21.15.1:0/677016266 >> 172.21.15.1:0/677016266 conn(0x55aed2954db0 msgr2=0x55aed29f13b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:58.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.741+0000 7f630d3f7b80 1 -- 172.21.15.1:0/677016266 shutdown_connections 2024-02-02T15:44:58.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.741+0000 7f630d3f7b80 1 -- 172.21.15.1:0/677016266 wait complete. 2024-02-02T15:44:58.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.742+0000 7f630d3f7b80 1 -- start start 2024-02-02T15:44:58.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.742+0000 7f630d3f7b80 1 Processor -- start 2024-02-02T15:44:58.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.746+0000 7f630d3f7b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aed29f5010 0x55aed2a00bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:58.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.746+0000 7f630d3f7b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aed29f5930 0x55aed2a01100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:58.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.747+0000 7f630d3f7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aed2a007e0 0x55aed2a01660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:58.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.748+0000 7f62f2a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aed29f5010 0x55aed2a00bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:58.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.748+0000 7f62f2a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aed29f5010 0x55aed2a00bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44096/0 (socket says 172.21.15.1:44096) 2024-02-02T15:44:58.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.749+0000 7f630d3f7b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aed2946d50 con 0x55aed29f5010 2024-02-02T15:44:58.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.749+0000 7f630d3f7b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aed28e2d80 con 0x55aed29f5930 2024-02-02T15:44:58.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.749+0000 7f630d3f7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aed28e2a20 con 0x55aed2a007e0 2024-02-02T15:44:58.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.749+0000 7f630d3f7b80 10 client.0 _ll_register_callbacks cb 0x55aed28e4290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:44:58.749 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.749+0000 7f62f2a00700 1 -- 172.21.15.1:0/51929828 learned_addr learned my addr 172.21.15.1:0/51929828 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:44:58.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.750+0000 7f62f2a00700 1 -- 172.21.15.1:0/51929828 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aed2a007e0 msgr2=0x55aed2a01660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:58.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.750+0000 7f62f2a00700 1 --2- 172.21.15.1:0/51929828 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aed2a007e0 0x55aed2a01660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:58.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.750+0000 7f62f2a00700 1 -- 172.21.15.1:0/51929828 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aed29f5930 msgr2=0x55aed2a01100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:58.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.750+0000 7f62f2a00700 1 --2- 172.21.15.1:0/51929828 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aed29f5930 0x55aed2a01100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:58.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.750+0000 7f62f2a00700 1 -- 172.21.15.1:0/51929828 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62f4103800 con 0x55aed29f5010 2024-02-02T15:44:58.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.751+0000 7f62f2a00700 1 --2- 172.21.15.1:0/51929828 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aed29f5010 0x55aed2a00bc0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f62e400b460 tx=0x7f62e400b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45523be91e6eda0e server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:58.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.751+0000 7f62e0c00700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:44:58.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.752+0000 7f62e0c00700 1 -- 172.21.15.1:0/51929828 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e400e070 con 0x55aed29f5010 2024-02-02T15:44:58.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.752+0000 7f62e0c00700 1 -- 172.21.15.1:0/51929828 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f62e40115a0 con 0x55aed29f5010 2024-02-02T15:44:58.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.752+0000 7f630d3f7b80 1 -- 172.21.15.1:0/51929828 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aed29e90f0 con 0x55aed29f5010 2024-02-02T15:44:58.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.753+0000 7f62e0c00700 1 -- 172.21.15.1:0/51929828 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) v2 ==== 24+0+0 (secure 0 0 0) 0x7f62e400e070 con 0x55aed29f5010 2024-02-02T15:44:58.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.753+0000 7f630d3f7b80 10 client.19959 fetch_fsmap learned FSMap version 77 2024-02-02T15:44:58.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.753+0000 7f630d3f7b80 1 -- 172.21.15.1:0/51929828 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=4+}) v3 -- 0x55aed2774b60 con 0x55aed29f5010 2024-02-02T15:44:58.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.755+0000 7f62e0c00700 1 -- 172.21.15.1:0/51929828 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55aed29e90f0 con 0x55aed29f5010 2024-02-02T15:44:58.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.757+0000 7f62e0c00700 1 -- 172.21.15.1:0/51929828 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 77) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f62e40129f0 con 0x55aed29f5010 2024-02-02T15:44:58.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.757+0000 7f630d3f7b80 10 client.19959 fetch_fsmap finished waiting for FSMap version 77 2024-02-02T15:44:58.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.757+0000 7f630d3f7b80 10 client.19959 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-02-02T15:44:58.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.757+0000 7f630d3f7b80 20 client.19959 populate_metadata read hostname 'smithi001' 2024-02-02T15:44:58.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.757+0000 7f630d3f7b80 1 --2- 172.21.15.1:0/51929828 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55aed2a0ff60 0x55aed2a123a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:58.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.757+0000 7f630d3f7b80 4 client.19959 mds_command: new command op to 24518 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:44:58.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.757+0000 7f630d3f7b80 1 -- 172.21.15.1:0/51929828 --> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aed293fdd0 con 0x55aed2a0ff60 2024-02-02T15:44:58.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.758+0000 7f62e9e00700 1 --2- 172.21.15.1:0/51929828 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55aed2a0ff60 0x55aed2a123a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:58.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.759+0000 7f62e0c00700 10 client.19959 ms_handle_connect on v2:172.21.15.71:6836/628016199 2024-02-02T15:44:58.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.759+0000 7f62e9e00700 1 --2- 172.21.15.1:0/51929828 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55aed2a0ff60 0x55aed2a123a0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ad2ccf9ffd35a313 server_cookie=f35f54d3a1fa8b69 in_seq=0 out_seq=0 2024-02-02T15:44:58.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.759+0000 7f62e0c00700 1 -- 172.21.15.1:0/51929828 <== mds.2 v2:172.21.15.71:6836/628016199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55aed293fdd0 con 0x55aed2a0ff60 2024-02-02T15:44:58.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.759+0000 7f62e0c00700 10 client.19959 handle_command_reply: tid=0 2024-02-02T15:44:58.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.760+0000 7f630d3f7b80 1 client.19959 shutdown 2024-02-02T15:44:58.760 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.760+0000 7f630d3f7b80 1 -- 172.21.15.1:0/51929828 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aed29f5010 msgr2=0x55aed2a00bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:58.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.761+0000 7f630d3f7b80 1 --2- 172.21.15.1:0/51929828 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aed29f5010 0x55aed2a00bc0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f62e400b460 tx=0x7f62e400b820 comp rx=0 tx=0).stop 2024-02-02T15:44:58.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.761+0000 7f630d3f7b80 1 -- 172.21.15.1:0/51929828 shutdown_connections 2024-02-02T15:44:58.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.761+0000 7f630d3f7b80 1 --2- 172.21.15.1:0/51929828 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55aed2a0ff60 0x55aed2a123a0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:58.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.761+0000 7f630d3f7b80 1 --2- 172.21.15.1:0/51929828 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aed2a007e0 0x55aed2a01660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:58.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.761+0000 7f630d3f7b80 1 --2- 172.21.15.1:0/51929828 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aed29f5010 0x55aed2a00bc0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:58.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.761+0000 7f630d3f7b80 1 --2- 172.21.15.1:0/51929828 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aed29f5930 0x55aed2a01100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:58.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.761+0000 7f62e0c00700 0 client.19959 ms_handle_reset on v2:172.21.15.71:6836/628016199 2024-02-02T15:44:58.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.761+0000 7f630d3f7b80 1 -- 172.21.15.1:0/51929828 >> 172.21.15.1:0/51929828 conn(0x55aed2954db0 msgr2=0x55aed29f0fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:58.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.762+0000 7f630d3f7b80 1 -- 172.21.15.1:0/51929828 shutdown_connections 2024-02-02T15:44:58.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.762+0000 7f630d3f7b80 1 -- 172.21.15.1:0/51929828 wait complete. 2024-02-02T15:44:58.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.763+0000 7f630d3f7b80 20 client.19959 trim_cache size 0 max 16384 2024-02-02T15:44:58.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.798+0000 7f630d3f7b80 1 Processor -- start 2024-02-02T15:44:58.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.798+0000 7f630d3f7b80 1 -- start start 2024-02-02T15:44:58.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.799+0000 7f630d3f7b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aed29f5010 0x55aed29f05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:58.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.799+0000 7f630d3f7b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aed29f5930 0x55aed29face0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:58.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.799+0000 7f630d3f7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aed2a007e0 0x55aed2a0b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:58.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.799+0000 7f630d3f7b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aed27ca6a0 con 0x55aed29f5010 2024-02-02T15:44:58.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.799+0000 7f630d3f7b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aed29fa820 con 0x55aed29f5930 2024-02-02T15:44:58.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.799+0000 7f630d3f7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aed29fff20 con 0x55aed2a007e0 2024-02-02T15:44:58.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.799+0000 7f62f3400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aed2a007e0 0x55aed2a0b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:58.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.799+0000 7f62f3400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aed2a007e0 0x55aed2a0b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:50548/0 (socket says 172.21.15.1:50548) 2024-02-02T15:44:58.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.799+0000 7f62f3400700 1 -- 172.21.15.1:0/3520683579 learned_addr learned my addr 172.21.15.1:0/3520683579 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:44:58.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.800+0000 7f62f2a00700 1 --2- 172.21.15.1:0/3520683579 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aed29f5010 0x55aed29f05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:58.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.800+0000 7f62f3400700 1 -- 172.21.15.1:0/3520683579 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aed29f5930 msgr2=0x55aed29face0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:58.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.800+0000 7f62f3400700 1 --2- 172.21.15.1:0/3520683579 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aed29f5930 0x55aed29face0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:58.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.800+0000 7f62f3400700 1 -- 172.21.15.1:0/3520683579 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aed29f5010 msgr2=0x55aed29f05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:58.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.800+0000 7f62f3400700 1 --2- 172.21.15.1:0/3520683579 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aed29f5010 0x55aed29f05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:58.801 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.800+0000 7f62f3400700 1 -- 172.21.15.1:0/3520683579 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aed290cea0 con 0x55aed2a007e0 2024-02-02T15:44:58.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.802+0000 7f62f3400700 1 --2- 172.21.15.1:0/3520683579 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aed2a007e0 0x55aed2a0b1e0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f62ec0ca710 tx=0x7f62ec009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1bc6ddddcb051a51 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:58.802 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.802+0000 7f62e8a00700 1 -- 172.21.15.1:0/3520683579 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62ec09a660 con 0x55aed2a007e0 2024-02-02T15:44:58.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.802+0000 7f62e8a00700 1 -- 172.21.15.1:0/3520683579 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f62ec0a1fd0 con 0x55aed2a007e0 2024-02-02T15:44:58.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.802+0000 7f62e8a00700 1 -- 172.21.15.1:0/3520683579 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62ec0b24e0 con 0x55aed2a007e0 2024-02-02T15:44:58.804 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.803+0000 7f630d3f7b80 1 -- 172.21.15.1:0/3520683579 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aed2a007e0 msgr2=0x55aed2a0b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:58.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.803+0000 7f630d3f7b80 1 --2- 172.21.15.1:0/3520683579 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aed2a007e0 0x55aed2a0b1e0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f62ec0ca710 tx=0x7f62ec009ca0 comp rx=0 tx=0).stop 2024-02-02T15:44:58.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.804+0000 7f630d3f7b80 1 -- 172.21.15.1:0/3520683579 shutdown_connections 2024-02-02T15:44:58.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.804+0000 7f630d3f7b80 1 --2- 172.21.15.1:0/3520683579 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aed2a007e0 0x55aed2a0b1e0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:58.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.804+0000 7f630d3f7b80 1 --2- 172.21.15.1:0/3520683579 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aed29f5010 0x55aed29f05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:58.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.804+0000 7f630d3f7b80 1 --2- 172.21.15.1:0/3520683579 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aed29f5930 0x55aed29face0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:58.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.804+0000 7f630d3f7b80 1 -- 172.21.15.1:0/3520683579 >> 172.21.15.1:0/3520683579 conn(0x55aed2954db0 msgr2=0x55aed29ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:58.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.804+0000 7f630d3f7b80 1 -- 172.21.15.1:0/3520683579 shutdown_connections 2024-02-02T15:44:58.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.804+0000 7f630d3f7b80 1 -- 172.21.15.1:0/3520683579 wait complete. 2024-02-02T15:44:58.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.805+0000 7f630d3f7b80 1 -- start start 2024-02-02T15:44:58.805 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.805+0000 7f630d3f7b80 1 Processor -- start 2024-02-02T15:44:58.806 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.806+0000 7f630d3f7b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aed29f5010 0x55aed2a5ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:58.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.807+0000 7f630d3f7b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aed29f5930 0x55aed2a5b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:58.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.807+0000 7f630d3f7b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aed2a007e0 0x55aed2a01b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:58.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.807+0000 7f630d3f7b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aed2a02120 con 0x55aed29f5930 2024-02-02T15:44:58.807 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.807+0000 7f630d3f7b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aed2a02290 con 0x55aed2a007e0 2024-02-02T15:44:58.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.807+0000 7f630d3f7b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aed2a02450 con 0x55aed29f5010 2024-02-02T15:44:58.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.807+0000 7f630d3f7b80 10 client.0 _ll_register_callbacks cb 0x55aed28e4290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:44:58.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.807+0000 7f62f3400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aed2a007e0 0x55aed2a01b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:58.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.808+0000 7f62f3400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aed2a007e0 0x55aed2a01b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:35338/0 (socket says 172.21.15.1:35338) 2024-02-02T15:44:58.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.808+0000 7f62f3400700 1 -- 172.21.15.1:0/1697031254 learned_addr learned my addr 172.21.15.1:0/1697031254 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:44:58.808 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.808+0000 7f62f2a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aed29f5010 0x55aed2a5ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:58.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.809+0000 7f62e9e00700 1 --2- 172.21.15.1:0/1697031254 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aed29f5930 0x55aed2a5b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:58.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.809+0000 7f62f3400700 1 -- 172.21.15.1:0/1697031254 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aed29f5010 msgr2=0x55aed2a5ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:58.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.809+0000 7f62f3400700 1 --2- 172.21.15.1:0/1697031254 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aed29f5010 0x55aed2a5ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:58.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.809+0000 7f62f3400700 1 -- 172.21.15.1:0/1697031254 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aed29f5930 msgr2=0x55aed2a5b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:58.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.809+0000 7f62f3400700 1 --2- 172.21.15.1:0/1697031254 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aed29f5930 0x55aed2a5b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:58.809 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.809+0000 7f62f3400700 1 -- 172.21.15.1:0/1697031254 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aed27c83b0 con 0x55aed2a007e0 2024-02-02T15:44:58.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.810+0000 7f62f3400700 1 --2- 172.21.15.1:0/1697031254 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aed2a007e0 0x55aed2a01b00 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f62ec005cd0 tx=0x7f62ec005df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b98cc242aa41b33 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:58.810 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.810+0000 7f62e0c00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:44:58.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.811+0000 7f62e0c00700 1 -- 172.21.15.1:0/1697031254 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62ec0a1d40 con 0x55aed2a007e0 2024-02-02T15:44:58.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.811+0000 7f62e0c00700 1 -- 172.21.15.1:0/1697031254 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f62ec09dd90 con 0x55aed2a007e0 2024-02-02T15:44:58.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.811+0000 7f630d3f7b80 1 -- 172.21.15.1:0/1697031254 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f62ec0b9040 con 0x55aed2a007e0 2024-02-02T15:44:58.811 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.811+0000 7f62e0c00700 1 -- 172.21.15.1:0/1697031254 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) v2 ==== 24+0+0 (secure 0 0 0) 0x7f62ec09ee50 con 0x55aed2a007e0 2024-02-02T15:44:58.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.812+0000 7f630d3f7b80 10 client.30248 fetch_fsmap learned FSMap version 77 2024-02-02T15:44:58.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.813+0000 7f630d3f7b80 1 -- 172.21.15.1:0/1697031254 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=4+}) v3 -- 0x55aed27cccd0 con 0x55aed2a007e0 2024-02-02T15:44:58.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.813+0000 7f62e0c00700 1 -- 172.21.15.1:0/1697031254 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f62ec07e080 con 0x55aed2a007e0 2024-02-02T15:44:58.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.813+0000 7f62e0c00700 1 -- 172.21.15.1:0/1697031254 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 77) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f62ec099450 con 0x55aed2a007e0 2024-02-02T15:44:58.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.814+0000 7f630d3f7b80 10 client.30248 fetch_fsmap finished waiting for FSMap version 77 2024-02-02T15:44:58.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.814+0000 7f630d3f7b80 10 client.30248 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-02-02T15:44:58.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.814+0000 7f630d3f7b80 20 client.30248 populate_metadata read hostname 'smithi001' 2024-02-02T15:44:58.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.814+0000 7f630d3f7b80 1 --2- 172.21.15.1:0/1697031254 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55aed2a62700 0x55aed2a64af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:58.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.814+0000 7f630d3f7b80 4 client.30248 mds_command: new command op to 24518 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:44:58.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.814+0000 7f630d3f7b80 1 -- 172.21.15.1:0/1697031254 --> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55aed2905f50 con 0x55aed2a62700 2024-02-02T15:44:58.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.815+0000 7f62f2a00700 1 --2- 172.21.15.1:0/1697031254 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55aed2a62700 0x55aed2a64af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:58.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.816+0000 7f62f2a00700 1 --2- 172.21.15.1:0/1697031254 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55aed2a62700 0x55aed2a64af0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c5954a27821d0114 server_cookie=57d472829a0341e9 in_seq=0 out_seq=0 2024-02-02T15:44:58.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.817+0000 7f62e0c00700 10 client.30248 ms_handle_connect on v2:172.21.15.71:6836/628016199 2024-02-02T15:44:58.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.821+0000 7f62e0c00700 1 -- 172.21.15.1:0/1697031254 <== mds.2 v2:172.21.15.71:6836/628016199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55aed2905f50 con 0x55aed2a62700 2024-02-02T15:44:58.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.821+0000 7f62e0c00700 10 client.30248 handle_command_reply: tid=0 2024-02-02T15:44:58.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.821+0000 7f630d3f7b80 1 client.30248 shutdown 2024-02-02T15:44:58.821 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.822+0000 7f630d3f7b80 1 -- 172.21.15.1:0/1697031254 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aed2a007e0 msgr2=0x55aed2a01b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:58.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.822+0000 7f630d3f7b80 1 --2- 172.21.15.1:0/1697031254 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aed2a007e0 0x55aed2a01b00 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f62ec005cd0 tx=0x7f62ec005df0 comp rx=0 tx=0).stop 2024-02-02T15:44:58.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.822+0000 7f630d3f7b80 1 -- 172.21.15.1:0/1697031254 shutdown_connections 2024-02-02T15:44:58.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.822+0000 7f630d3f7b80 1 --2- 172.21.15.1:0/1697031254 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55aed2a007e0 0x55aed2a01b00 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:58.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.822+0000 7f630d3f7b80 1 --2- 172.21.15.1:0/1697031254 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55aed2a62700 0x55aed2a64af0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:58.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.822+0000 7f630d3f7b80 1 --2- 172.21.15.1:0/1697031254 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55aed29f5010 0x55aed2a5ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:58.822 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.822+0000 7f630d3f7b80 1 --2- 172.21.15.1:0/1697031254 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55aed29f5930 0x55aed2a5b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:58.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.822+0000 7f62e0c00700 0 client.30248 ms_handle_reset on v2:172.21.15.71:6836/628016199 2024-02-02T15:44:58.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.822+0000 7f630d3f7b80 1 -- 172.21.15.1:0/1697031254 >> 172.21.15.1:0/1697031254 conn(0x55aed2954db0 msgr2=0x55aed29face0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:58.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.823+0000 7f630d3f7b80 1 -- 172.21.15.1:0/1697031254 shutdown_connections 2024-02-02T15:44:58.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.823+0000 7f630d3f7b80 1 -- 172.21.15.1:0/1697031254 wait complete. 2024-02-02T15:44:58.823 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.823+0000 7f630d3f7b80 20 client.30248 trim_cache size 0 max 16384 2024-02-02T15:44:58.824 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:44:58.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.825+0000 7f62f9800700 1 -- 172.21.15.1:0/1773978316 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f62d40a0820 msgr2=0x7f62d40a2ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:58.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.825+0000 7f62f9800700 1 --2- 172.21.15.1:0/1773978316 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f62d40a0820 0x7f62d40a2ce0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f62dc00b7f0 tx=0x7f62dc0126d0 comp rx=0 tx=0).stop 2024-02-02T15:44:58.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.825+0000 7f62f9800700 1 -- 172.21.15.1:0/1773978316 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f62f4111cc0 msgr2=0x7f62f41b1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:58.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.825+0000 7f62f9800700 1 --2- 172.21.15.1:0/1773978316 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f62f4111cc0 0x7f62f41b1ad0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f62ec0098e0 tx=0x7f62ec00f670 comp rx=0 tx=0).stop 2024-02-02T15:44:58.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.825+0000 7f62f9800700 1 -- 172.21.15.1:0/1773978316 shutdown_connections 2024-02-02T15:44:58.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.825+0000 7f62f9800700 1 --2- 172.21.15.1:0/1773978316 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f62d40a0820 0x7f62d40a2ce0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:58.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.825+0000 7f62f9800700 1 --2- 172.21.15.1:0/1773978316 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f62f4111cc0 0x7f62f41b1ad0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:58.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.825+0000 7f62f9800700 1 --2- 172.21.15.1:0/1773978316 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f62f4075270 0x7f62f406b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:58.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.825+0000 7f62f9800700 1 --2- 172.21.15.1:0/1773978316 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f62f4075b90 0x7f62f406b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:58.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.825+0000 7f62f9800700 1 -- 172.21.15.1:0/1773978316 >> 172.21.15.1:0/1773978316 conn(0x7f62f40feb30 msgr2=0x7f62f4100f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:58.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.825+0000 7f62f9800700 1 -- 172.21.15.1:0/1773978316 shutdown_connections 2024-02-02T15:44:58.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:58.825+0000 7f62f9800700 1 -- 172.21.15.1:0/1773978316 wait complete. 2024-02-02T15:44:58.858 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-02T15:44:58.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:44:58 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/3416545446' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:44:58.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:44:58 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/3416545446' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:44:58.998 INFO:tasks.workunit.client.0.smithi001.stdout: 10 148718 28.20 MB/sec execute 151 sec latency 83.362 ms 2024-02-02T15:44:59.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.216+0000 7f95df000700 1 -- 172.21.15.1:0/2281532688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95d81019b0 msgr2=0x7f95d81059f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:59.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.216+0000 7f95df000700 1 --2- 172.21.15.1:0/2281532688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95d81019b0 0x7f95d81059f0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f95c8009990 tx=0x7f95c8009ca0 comp rx=0 tx=0).stop 2024-02-02T15:44:59.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.217+0000 7f95df000700 1 -- 172.21.15.1:0/2281532688 shutdown_connections 2024-02-02T15:44:59.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.217+0000 7f95df000700 1 --2- 172.21.15.1:0/2281532688 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95d8106120 0x7f95d810dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:59.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.217+0000 7f95df000700 1 --2- 172.21.15.1:0/2281532688 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f95d8101000 0x7f95d81013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:59.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.217+0000 7f95df000700 1 --2- 172.21.15.1:0/2281532688 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95d81019b0 0x7f95d81059f0 secure :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f95c8009990 tx=0x7f95c8009ca0 comp rx=0 tx=0).stop 2024-02-02T15:44:59.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.217+0000 7f95df000700 1 -- 172.21.15.1:0/2281532688 >> 172.21.15.1:0/2281532688 conn(0x7f95d80fc840 msgr2=0x7f95d80fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:59.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.218+0000 7f95df000700 1 -- 172.21.15.1:0/2281532688 shutdown_connections 2024-02-02T15:44:59.218 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.218+0000 7f95df000700 1 -- 172.21.15.1:0/2281532688 wait complete. 2024-02-02T15:44:59.219 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.219+0000 7f95df000700 1 Processor -- start 2024-02-02T15:44:59.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.220+0000 7f95df000700 1 -- start start 2024-02-02T15:44:59.220 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.221+0000 7f95df000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95d8101000 0x7f95d8194fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:59.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.221+0000 7f95df000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95d8106120 0x7f95d81954f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:59.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.221+0000 7f95df000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f95d8199950 0x7f95d8199dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:59.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.221+0000 7f95df000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d819a3d0 con 0x7f95d8199950 2024-02-02T15:44:59.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.221+0000 7f95df000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d819a540 con 0x7f95d8101000 2024-02-02T15:44:59.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.221+0000 7f95df000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d819a700 con 0x7f95d8106120 2024-02-02T15:44:59.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.221+0000 7f95d7400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95d8106120 0x7f95d81954f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:59.222 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.221+0000 7f95d7400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95d8106120 0x7f95d81954f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:50586/0 (socket says 172.21.15.1:50586) 2024-02-02T15:44:59.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.221+0000 7f95d7400700 1 -- 172.21.15.1:0/557942130 learned_addr learned my addr 172.21.15.1:0/557942130 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:44:59.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.222+0000 7f95d7400700 1 -- 172.21.15.1:0/557942130 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95d8101000 msgr2=0x7f95d8194fb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-02-02T15:44:59.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.222+0000 7f95d7400700 1 --2- 172.21.15.1:0/557942130 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95d8101000 0x7f95d8194fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:59.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.222+0000 7f95d7400700 1 -- 172.21.15.1:0/557942130 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f95d8199950 msgr2=0x7f95d8199dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:59.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.223+0000 7f95d7400700 1 --2- 172.21.15.1:0/557942130 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f95d8199950 0x7f95d8199dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:59.223 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.224+0000 7f95d7400700 1 -- 172.21.15.1:0/557942130 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95c80095f0 con 0x7f95d8106120 2024-02-02T15:44:59.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.224+0000 7f95d7400700 1 --2- 172.21.15.1:0/557942130 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95d8106120 0x7f95d81954f0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f95c80098e0 tx=0x7f95c8005e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:59.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.225+0000 7f95d4c00700 1 -- 172.21.15.1:0/557942130 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c801d070 con 0x7f95d8106120 2024-02-02T15:44:59.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.225+0000 7f95df000700 1 -- 172.21.15.1:0/557942130 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95d81a1c00 con 0x7f95d8106120 2024-02-02T15:44:59.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.226+0000 7f95df000700 1 -- 172.21.15.1:0/557942130 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95d81a1fc0 con 0x7f95d8106120 2024-02-02T15:44:59.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.227+0000 7f95d4c00700 1 -- 172.21.15.1:0/557942130 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f95c80047e0 con 0x7f95d8106120 2024-02-02T15:44:59.227 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.227+0000 7f95d4c00700 1 -- 172.21.15.1:0/557942130 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c8005290 con 0x7f95d8106120 2024-02-02T15:44:59.230 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.231+0000 7f95f2aeab80 1 Processor -- start 2024-02-02T15:44:59.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.232+0000 7f95f2aeab80 1 -- start start 2024-02-02T15:44:59.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.233+0000 7f95f2aeab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5572850ca010 0x5572850ca3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:59.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.233+0000 7f95f2aeab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5572850ca930 0x5572850d52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:59.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.233+0000 7f95f2aeab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5572850d57e0 0x5572850d7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:59.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.233+0000 7f95f2aeab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55728503e4c0 con 0x5572850d57e0 2024-02-02T15:44:59.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.233+0000 7f95f2aeab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x557285022aa0 con 0x5572850ca010 2024-02-02T15:44:59.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.233+0000 7f95f2aeab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557285020460 con 0x5572850ca930 2024-02-02T15:44:59.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.233+0000 7f95dd000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5572850ca010 0x5572850ca3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:59.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.233+0000 7f95d4c00700 1 -- 172.21.15.1:0/557942130 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f95c80054b0 con 0x7f95d8106120 2024-02-02T15:44:59.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.233+0000 7f95dd000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5572850ca010 0x5572850ca3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:35366/0 (socket says 172.21.15.1:35366) 2024-02-02T15:44:59.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.233+0000 7f95dd000700 1 -- 172.21.15.1:0/2034163490 learned_addr learned my addr 172.21.15.1:0/2034163490 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:44:59.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.234+0000 7f95d7400700 1 --2- 172.21.15.1:0/2034163490 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5572850d57e0 0x5572850d7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:59.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.235+0000 7f95d7e00700 1 --2- 172.21.15.1:0/2034163490 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5572850ca930 0x5572850d52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:59.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.235+0000 7f95d4c00700 1 --2- 172.21.15.1:0/557942130 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f95b80a0710 0x7f95b80a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:59.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.235+0000 7f95dd000700 1 -- 172.21.15.1:0/2034163490 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5572850ca930 msgr2=0x5572850d52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:59.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.235+0000 7f95dd000700 1 --2- 172.21.15.1:0/2034163490 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5572850ca930 0x5572850d52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:59.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.235+0000 7f95dd000700 1 -- 172.21.15.1:0/2034163490 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5572850d57e0 msgr2=0x5572850d7bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:59.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.235+0000 7f95dd000700 1 --2- 172.21.15.1:0/2034163490 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5572850d57e0 0x5572850d7bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:59.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.235+0000 7f95dd000700 1 -- 172.21.15.1:0/2034163490 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557284e9d3b0 con 0x5572850ca010 2024-02-02T15:44:59.235 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.235+0000 7f95d7400700 1 --2- 172.21.15.1:0/2034163490 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5572850d57e0 0x5572850d7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:44:59.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.235+0000 7f95d4c00700 1 -- 172.21.15.1:0/557942130 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f95c80cce80 con 0x7f95d8106120 2024-02-02T15:44:59.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.236+0000 7f95dd000700 1 --2- 172.21.15.1:0/557942130 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f95b80a0710 0x7f95b80a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:59.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.237+0000 7f95dd000700 1 --2- 172.21.15.1:0/2034163490 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5572850ca010 0x5572850ca3f0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f95d00119f0 tx=0x7f95d00152d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85f8ba3981339616 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:59.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.237+0000 7f95cd600700 1 -- 172.21.15.1:0/2034163490 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d0011050 con 0x5572850ca010 2024-02-02T15:44:59.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.238+0000 7f95cd600700 1 -- 172.21.15.1:0/2034163490 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f95d0015ed0 con 0x5572850ca010 2024-02-02T15:44:59.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.238+0000 7f95cd600700 1 -- 172.21.15.1:0/2034163490 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d00173b0 con 0x5572850ca010 2024-02-02T15:44:59.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.238+0000 7f95f2aeab80 1 -- 172.21.15.1:0/2034163490 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5572850ca010 msgr2=0x5572850ca3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:59.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.238+0000 7f95f2aeab80 1 --2- 172.21.15.1:0/2034163490 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5572850ca010 0x5572850ca3f0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f95d00119f0 tx=0x7f95d00152d0 comp rx=0 tx=0).stop 2024-02-02T15:44:59.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.238+0000 7f95dd000700 1 --2- 172.21.15.1:0/557942130 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f95b80a0710 0x7f95b80a2bd0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f95d0026000 tx=0x7f95d0029000 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:59.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.239+0000 7f95f2aeab80 1 -- 172.21.15.1:0/2034163490 shutdown_connections 2024-02-02T15:44:59.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.239+0000 7f95f2aeab80 1 --2- 172.21.15.1:0/2034163490 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5572850d57e0 0x5572850d7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:59.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.239+0000 7f95f2aeab80 1 --2- 172.21.15.1:0/2034163490 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5572850ca010 0x5572850ca3f0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:59.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.239+0000 7f95f2aeab80 1 --2- 172.21.15.1:0/2034163490 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5572850ca930 0x5572850d52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:59.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.239+0000 7f95f2aeab80 1 -- 172.21.15.1:0/2034163490 >> 172.21.15.1:0/2034163490 conn(0x557285029db0 msgr2=0x5572850c63b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:59.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.239+0000 7f95f2aeab80 1 -- 172.21.15.1:0/2034163490 shutdown_connections 2024-02-02T15:44:59.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.239+0000 7f95f2aeab80 1 -- 172.21.15.1:0/2034163490 wait complete. 2024-02-02T15:44:59.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.239+0000 7f95f2aeab80 1 -- start start 2024-02-02T15:44:59.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.240+0000 7f95f2aeab80 1 Processor -- start 2024-02-02T15:44:59.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.241+0000 7f95f2aeab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5572850ca010 0x5572850d5100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:59.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.241+0000 7f95f2aeab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5572850ca930 0x5572850d2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:59.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.241+0000 7f95f2aeab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5572850d57e0 0x5572850d2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:59.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.241+0000 7f95f2aeab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55728501bd50 con 0x5572850ca930 2024-02-02T15:44:59.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.241+0000 7f95f2aeab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x557284fb7d80 con 0x5572850ca010 2024-02-02T15:44:59.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.241+0000 7f95f2aeab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x557284fb7a20 con 0x5572850d57e0 2024-02-02T15:44:59.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.241+0000 7f95f2aeab80 10 client.0 _ll_register_callbacks cb 0x557284fb9290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:44:59.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.241+0000 7f95d7e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5572850ca010 0x5572850d5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:59.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.241+0000 7f95d7e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5572850ca010 0x5572850d5100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:35382/0 (socket says 172.21.15.1:35382) 2024-02-02T15:44:59.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.241+0000 7f95d7e00700 1 -- 172.21.15.1:0/2130319433 learned_addr learned my addr 172.21.15.1:0/2130319433 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:44:59.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.242+0000 7f95d7e00700 1 -- 172.21.15.1:0/2130319433 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5572850d57e0 msgr2=0x5572850d2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:59.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.242+0000 7f95d7e00700 1 --2- 172.21.15.1:0/2130319433 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5572850d57e0 0x5572850d2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:59.242 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.242+0000 7f95d7e00700 1 -- 172.21.15.1:0/2130319433 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5572850ca930 msgr2=0x5572850d2310 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-02T15:44:59.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.242+0000 7f95d7e00700 1 --2- 172.21.15.1:0/2130319433 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5572850ca930 0x5572850d2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:59.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.242+0000 7f95d7e00700 1 -- 172.21.15.1:0/2130319433 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95d81a1fc0 con 0x5572850ca010 2024-02-02T15:44:59.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.243+0000 7f95d7e00700 1 --2- 172.21.15.1:0/2130319433 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5572850ca010 0x5572850d5100 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f95c000c8f0 tx=0x7f95c000ccb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70dc7245ee34963f server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:59.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.243+0000 7f95c6000700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:44:59.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.244+0000 7f95c6000700 1 -- 172.21.15.1:0/2130319433 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c0012070 con 0x5572850ca010 2024-02-02T15:44:59.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.244+0000 7f95c6000700 1 -- 172.21.15.1:0/2130319433 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f95c0009840 con 0x5572850ca010 2024-02-02T15:44:59.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.244+0000 7f95f2aeab80 1 -- 172.21.15.1:0/2130319433 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5572850be0f0 con 0x5572850ca010 2024-02-02T15:44:59.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.245+0000 7f95c6000700 1 -- 172.21.15.1:0/2130319433 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) v2 ==== 24+0+0 (secure 0 0 0) 0x7f95c000ce90 con 0x5572850ca010 2024-02-02T15:44:59.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.245+0000 7f95f2aeab80 10 client.30257 fetch_fsmap learned FSMap version 77 2024-02-02T15:44:59.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.245+0000 7f95f2aeab80 1 -- 172.21.15.1:0/2130319433 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=4+}) v3 -- 0x7f95d00115f0 con 0x5572850ca010 2024-02-02T15:44:59.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.245+0000 7f95c6000700 1 -- 172.21.15.1:0/2130319433 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5572850be0f0 con 0x5572850ca010 2024-02-02T15:44:59.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.246+0000 7f95c6000700 1 -- 172.21.15.1:0/2130319433 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 77) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f95c000b030 con 0x5572850ca010 2024-02-02T15:44:59.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.246+0000 7f95f2aeab80 10 client.30257 fetch_fsmap finished waiting for FSMap version 77 2024-02-02T15:44:59.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.246+0000 7f95f2aeab80 10 client.30257 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-02T15:44:59.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.246+0000 7f95f2aeab80 20 client.30257 populate_metadata read hostname 'smithi001' 2024-02-02T15:44:59.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.246+0000 7f95f2aeab80 1 --2- 172.21.15.1:0/2130319433 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x5572850e52c0 0x5572850e76e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:59.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.246+0000 7f95f2aeab80 4 client.30257 mds_command: new command op to 24529 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:44:59.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.246+0000 7f95f2aeab80 1 -- 172.21.15.1:0/2130319433 --> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557285014dd0 con 0x5572850e52c0 2024-02-02T15:44:59.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.246+0000 7f95d7400700 1 --2- 172.21.15.1:0/2130319433 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x5572850e52c0 0x5572850e76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:59.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.247+0000 7f95d7400700 1 --2- 172.21.15.1:0/2130319433 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x5572850e52c0 0x5572850e76e0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=688f2b357c809176 server_cookie=99e3dc2e158b98b1 in_seq=0 out_seq=0 2024-02-02T15:44:59.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.247+0000 7f95c6000700 10 client.30257 ms_handle_connect on v2:172.21.15.142:6838/321675421 2024-02-02T15:44:59.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.252+0000 7f95c6000700 1 -- 172.21.15.1:0/2130319433 <== mds.1 v2:172.21.15.142:6838/321675421 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x557285014dd0 con 0x5572850e52c0 2024-02-02T15:44:59.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.252+0000 7f95c6000700 10 client.30257 handle_command_reply: tid=0 2024-02-02T15:44:59.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.252+0000 7f95f2aeab80 1 client.30257 shutdown 2024-02-02T15:44:59.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.252+0000 7f95f2aeab80 1 -- 172.21.15.1:0/2130319433 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5572850ca010 msgr2=0x5572850d5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:59.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.252+0000 7f95f2aeab80 1 --2- 172.21.15.1:0/2130319433 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5572850ca010 0x5572850d5100 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f95c000c8f0 tx=0x7f95c000ccb0 comp rx=0 tx=0).stop 2024-02-02T15:44:59.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.252+0000 7f95f2aeab80 1 -- 172.21.15.1:0/2130319433 shutdown_connections 2024-02-02T15:44:59.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.252+0000 7f95f2aeab80 1 --2- 172.21.15.1:0/2130319433 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x5572850e52c0 0x5572850e76e0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:59.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.252+0000 7f95f2aeab80 1 --2- 172.21.15.1:0/2130319433 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5572850d57e0 0x5572850d2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:59.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.252+0000 7f95f2aeab80 1 --2- 172.21.15.1:0/2130319433 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5572850ca010 0x5572850d5100 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:59.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.252+0000 7f95f2aeab80 1 --2- 172.21.15.1:0/2130319433 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5572850ca930 0x5572850d2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:59.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.252+0000 7f95f2aeab80 1 -- 172.21.15.1:0/2130319433 >> 172.21.15.1:0/2130319433 conn(0x557285029db0 msgr2=0x5572850c5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:59.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.252+0000 7f95c6000700 0 client.30257 ms_handle_reset on v2:172.21.15.142:6838/321675421 2024-02-02T15:44:59.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.253+0000 7f95f2aeab80 1 -- 172.21.15.1:0/2130319433 shutdown_connections 2024-02-02T15:44:59.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.253+0000 7f95f2aeab80 1 -- 172.21.15.1:0/2130319433 wait complete. 2024-02-02T15:44:59.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.253+0000 7f95f2aeab80 20 client.30257 trim_cache size 0 max 16384 2024-02-02T15:44:59.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.274+0000 7f95f2aeab80 1 Processor -- start 2024-02-02T15:44:59.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.275+0000 7f95f2aeab80 1 -- start start 2024-02-02T15:44:59.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.275+0000 7f95f2aeab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5572850ca010 0x5572850dc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:59.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.275+0000 7f95f2aeab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5572850ca930 0x5572850cc840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:59.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.275+0000 7f95f2aeab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5572850d57e0 0x5572850c5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:59.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.275+0000 7f95f2aeab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x557284e9f6a0 con 0x5572850d57e0 2024-02-02T15:44:59.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.275+0000 7f95f2aeab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572850e1b90 con 0x5572850ca010 2024-02-02T15:44:59.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.275+0000 7f95f2aeab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572850c7200 con 0x5572850ca930 2024-02-02T15:44:59.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.275+0000 7f95d7400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5572850ca930 0x5572850cc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:59.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.275+0000 7f95d7400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5572850ca930 0x5572850cc840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:50602/0 (socket says 172.21.15.1:50602) 2024-02-02T15:44:59.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.275+0000 7f95d7400700 1 -- 172.21.15.1:0/4234474820 learned_addr learned my addr 172.21.15.1:0/4234474820 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:44:59.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.275+0000 7f95d7e00700 1 --2- 172.21.15.1:0/4234474820 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5572850ca010 0x5572850dc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:59.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.276+0000 7f95dd000700 1 --2- 172.21.15.1:0/4234474820 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5572850d57e0 0x5572850c5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:59.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.276+0000 7f95d7e00700 1 -- 172.21.15.1:0/4234474820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5572850ca930 msgr2=0x5572850cc840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:59.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.276+0000 7f95d7e00700 1 --2- 172.21.15.1:0/4234474820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5572850ca930 0x5572850cc840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:59.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.276+0000 7f95d7e00700 1 -- 172.21.15.1:0/4234474820 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5572850d57e0 msgr2=0x5572850c5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:59.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.276+0000 7f95d7e00700 1 --2- 172.21.15.1:0/4234474820 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5572850d57e0 0x5572850c5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:59.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.276+0000 7f95d7e00700 1 -- 172.21.15.1:0/4234474820 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557284e49b60 con 0x5572850ca010 2024-02-02T15:44:59.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.276+0000 7f95dd000700 1 --2- 172.21.15.1:0/4234474820 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5572850d57e0 0x5572850c5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:44:59.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.277+0000 7f95d7e00700 1 --2- 172.21.15.1:0/4234474820 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5572850ca010 0x5572850dc3d0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f95c0000c00 tx=0x7f95c000d810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19f449e0826b98ab server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:59.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.277+0000 7f95ce000700 1 -- 172.21.15.1:0/4234474820 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c0012040 con 0x5572850ca010 2024-02-02T15:44:59.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.277+0000 7f95ce000700 1 -- 172.21.15.1:0/4234474820 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f95c0005790 con 0x5572850ca010 2024-02-02T15:44:59.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.277+0000 7f95ce000700 1 -- 172.21.15.1:0/4234474820 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c000ce90 con 0x5572850ca010 2024-02-02T15:44:59.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.278+0000 7f95f2aeab80 1 -- 172.21.15.1:0/4234474820 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5572850ca010 msgr2=0x5572850dc3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:59.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.278+0000 7f95f2aeab80 1 --2- 172.21.15.1:0/4234474820 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5572850ca010 0x5572850dc3d0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f95c0000c00 tx=0x7f95c000d810 comp rx=0 tx=0).stop 2024-02-02T15:44:59.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.278+0000 7f95f2aeab80 1 -- 172.21.15.1:0/4234474820 shutdown_connections 2024-02-02T15:44:59.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.278+0000 7f95f2aeab80 1 --2- 172.21.15.1:0/4234474820 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5572850d57e0 0x5572850c5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:59.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.278+0000 7f95f2aeab80 1 --2- 172.21.15.1:0/4234474820 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5572850ca010 0x5572850dc3d0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:59.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.278+0000 7f95f2aeab80 1 --2- 172.21.15.1:0/4234474820 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5572850ca930 0x5572850cc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:59.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.278+0000 7f95f2aeab80 1 -- 172.21.15.1:0/4234474820 >> 172.21.15.1:0/4234474820 conn(0x557285029db0 msgr2=0x5572850c55c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:59.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.278+0000 7f95f2aeab80 1 -- 172.21.15.1:0/4234474820 shutdown_connections 2024-02-02T15:44:59.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.278+0000 7f95f2aeab80 1 -- 172.21.15.1:0/4234474820 wait complete. 2024-02-02T15:44:59.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.279+0000 7f95f2aeab80 1 -- start start 2024-02-02T15:44:59.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.279+0000 7f95f2aeab80 1 Processor -- start 2024-02-02T15:44:59.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.280+0000 7f95f2aeab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5572850ca010 0x55728512adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:59.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.280+0000 7f95f2aeab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5572850ca930 0x55728512b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:59.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.281+0000 7f95f2aeab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5572850d57e0 0x557285135e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:59.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.281+0000 7f95f2aeab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55728512bc00 con 0x5572850d57e0 2024-02-02T15:44:59.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.281+0000 7f95f2aeab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x557285136370 con 0x5572850ca930 2024-02-02T15:44:59.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.281+0000 7f95f2aeab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5572851364b0 con 0x5572850ca010 2024-02-02T15:44:59.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.281+0000 7f95f2aeab80 10 client.0 _ll_register_callbacks cb 0x557284fb9290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:44:59.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.281+0000 7f95d7e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5572850ca010 0x55728512adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:59.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.281+0000 7f95d7e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5572850ca010 0x55728512adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:50604/0 (socket says 172.21.15.1:50604) 2024-02-02T15:44:59.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.281+0000 7f95d7e00700 1 -- 172.21.15.1:0/1295298089 learned_addr learned my addr 172.21.15.1:0/1295298089 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:44:59.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.281+0000 7f95dd000700 1 --2- 172.21.15.1:0/1295298089 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5572850d57e0 0x557285135e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:59.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.281+0000 7f95d7e00700 1 -- 172.21.15.1:0/1295298089 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5572850ca930 msgr2=0x55728512b2f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-02T15:44:59.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.281+0000 7f95d7e00700 1 --2- 172.21.15.1:0/1295298089 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5572850ca930 0x55728512b2f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:59.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.281+0000 7f95d7e00700 1 -- 172.21.15.1:0/1295298089 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5572850d57e0 msgr2=0x557285135e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:59.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.281+0000 7f95d7e00700 1 --2- 172.21.15.1:0/1295298089 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5572850d57e0 0x557285135e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:59.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.281+0000 7f95d7e00700 1 -- 172.21.15.1:0/1295298089 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95d00115f0 con 0x5572850ca010 2024-02-02T15:44:59.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.282+0000 7f95d7e00700 1 --2- 172.21.15.1:0/1295298089 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5572850ca010 0x55728512adb0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f95c000d6b0 tx=0x7f95c0009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ab825c0148cd89e server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:59.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.282+0000 7f95c6000700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:44:59.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.282+0000 7f95c6000700 1 -- 172.21.15.1:0/1295298089 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95c000ce90 con 0x5572850ca010 2024-02-02T15:44:59.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.282+0000 7f95f2aeab80 1 -- 172.21.15.1:0/1295298089 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f95d0015ed0 con 0x5572850ca010 2024-02-02T15:44:59.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.282+0000 7f95c6000700 1 -- 172.21.15.1:0/1295298089 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f95c0000e40 con 0x5572850ca010 2024-02-02T15:44:59.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.283+0000 7f95c6000700 1 -- 172.21.15.1:0/1295298089 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) v2 ==== 24+0+0 (secure 0 0 0) 0x7f95c0003680 con 0x5572850ca010 2024-02-02T15:44:59.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.283+0000 7f95f2aeab80 10 client.30106 fetch_fsmap learned FSMap version 77 2024-02-02T15:44:59.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.283+0000 7f95f2aeab80 1 -- 172.21.15.1:0/1295298089 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=4+}) v3 -- 0x557284fe1ea0 con 0x5572850ca010 2024-02-02T15:44:59.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.283+0000 7f95c6000700 1 -- 172.21.15.1:0/1295298089 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f95c0003ca0 con 0x5572850ca010 2024-02-02T15:44:59.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.284+0000 7f95c6000700 1 -- 172.21.15.1:0/1295298089 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 77) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f95c000b070 con 0x5572850ca010 2024-02-02T15:44:59.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.284+0000 7f95f2aeab80 10 client.30106 fetch_fsmap finished waiting for FSMap version 77 2024-02-02T15:44:59.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.284+0000 7f95f2aeab80 10 client.30106 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-02T15:44:59.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.284+0000 7f95f2aeab80 20 client.30106 populate_metadata read hostname 'smithi001' 2024-02-02T15:44:59.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.284+0000 7f95f2aeab80 1 --2- 172.21.15.1:0/1295298089 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x557285137770 0x557285139b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:59.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.284+0000 7f95f2aeab80 4 client.30106 mds_command: new command op to 24529 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:44:59.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.284+0000 7f95f2aeab80 1 -- 172.21.15.1:0/1295298089 --> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x7f95d0015ed0 con 0x557285137770 2024-02-02T15:44:59.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.284+0000 7f95d7400700 1 --2- 172.21.15.1:0/1295298089 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x557285137770 0x557285139b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:59.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.285+0000 7f95d7400700 1 --2- 172.21.15.1:0/1295298089 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x557285137770 0x557285139b60 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=54dc5e9ecc20149b server_cookie=a59386735a746d45 in_seq=0 out_seq=0 2024-02-02T15:44:59.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.285+0000 7f95c6000700 10 client.30106 ms_handle_connect on v2:172.21.15.142:6838/321675421 2024-02-02T15:44:59.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.291+0000 7f95c6000700 1 -- 172.21.15.1:0/1295298089 <== mds.1 v2:172.21.15.142:6838/321675421 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x7f95d0015ed0 con 0x557285137770 2024-02-02T15:44:59.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.291+0000 7f95c6000700 10 client.30106 handle_command_reply: tid=0 2024-02-02T15:44:59.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.291+0000 7f95f2aeab80 1 client.30106 shutdown 2024-02-02T15:44:59.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.292+0000 7f95f2aeab80 1 -- 172.21.15.1:0/1295298089 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5572850ca010 msgr2=0x55728512adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:59.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.292+0000 7f95f2aeab80 1 --2- 172.21.15.1:0/1295298089 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5572850ca010 0x55728512adb0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f95c000d6b0 tx=0x7f95c0009b20 comp rx=0 tx=0).stop 2024-02-02T15:44:59.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.292+0000 7f95f2aeab80 1 -- 172.21.15.1:0/1295298089 shutdown_connections 2024-02-02T15:44:59.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.292+0000 7f95f2aeab80 1 --2- 172.21.15.1:0/1295298089 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x557285137770 0x557285139b60 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:59.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.292+0000 7f95f2aeab80 1 --2- 172.21.15.1:0/1295298089 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5572850d57e0 0x557285135e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:59.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.292+0000 7f95f2aeab80 1 --2- 172.21.15.1:0/1295298089 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5572850ca010 0x55728512adb0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:59.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.292+0000 7f95c6000700 0 client.30106 ms_handle_reset on v2:172.21.15.142:6838/321675421 2024-02-02T15:44:59.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.292+0000 7f95f2aeab80 1 --2- 172.21.15.1:0/1295298089 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5572850ca930 0x55728512b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:59.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.292+0000 7f95f2aeab80 1 -- 172.21.15.1:0/1295298089 >> 172.21.15.1:0/1295298089 conn(0x557285029db0 msgr2=0x5572850c5e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:59.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.292+0000 7f95f2aeab80 1 -- 172.21.15.1:0/1295298089 shutdown_connections 2024-02-02T15:44:59.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.292+0000 7f95f2aeab80 1 -- 172.21.15.1:0/1295298089 wait complete. 2024-02-02T15:44:59.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.293+0000 7f95f2aeab80 20 client.30106 trim_cache size 0 max 16384 2024-02-02T15:44:59.293 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:44:59.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.293+0000 7f95df000700 1 -- 172.21.15.1:0/557942130 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f95b80a0710 msgr2=0x7f95b80a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:59.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.293+0000 7f95df000700 1 --2- 172.21.15.1:0/557942130 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f95b80a0710 0x7f95b80a2bd0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f95d0026000 tx=0x7f95d0029000 comp rx=0 tx=0).stop 2024-02-02T15:44:59.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.293+0000 7f95df000700 1 -- 172.21.15.1:0/557942130 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95d8106120 msgr2=0x7f95d81954f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:59.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.293+0000 7f95df000700 1 --2- 172.21.15.1:0/557942130 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95d8106120 0x7f95d81954f0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f95c80098e0 tx=0x7f95c8005e80 comp rx=0 tx=0).stop 2024-02-02T15:44:59.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.293+0000 7f95df000700 1 -- 172.21.15.1:0/557942130 shutdown_connections 2024-02-02T15:44:59.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.293+0000 7f95df000700 1 --2- 172.21.15.1:0/557942130 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f95b80a0710 0x7f95b80a2bd0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:59.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.293+0000 7f95df000700 1 --2- 172.21.15.1:0/557942130 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f95d8199950 0x7f95d8199dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:59.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.293+0000 7f95df000700 1 --2- 172.21.15.1:0/557942130 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95d8101000 0x7f95d8194fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:59.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.293+0000 7f95df000700 1 --2- 172.21.15.1:0/557942130 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f95d8106120 0x7f95d81954f0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:59.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.293+0000 7f95df000700 1 -- 172.21.15.1:0/557942130 >> 172.21.15.1:0/557942130 conn(0x7f95d80fc840 msgr2=0x7f95d80fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:59.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.294+0000 7f95df000700 1 -- 172.21.15.1:0/557942130 shutdown_connections 2024-02-02T15:44:59.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.294+0000 7f95df000700 1 -- 172.21.15.1:0/557942130 wait complete. 2024-02-02T15:44:59.321 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-02T15:44:59.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.632+0000 7fe9a1200700 1 -- 172.21.15.1:0/881808493 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe99c073a30 msgr2=0x7fe99c10d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:59.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.632+0000 7fe9a1200700 1 --2- 172.21.15.1:0/881808493 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe99c073a30 0x7fe99c10d0f0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fe9840098f0 tx=0x7fe984009c00 comp rx=0 tx=0).stop 2024-02-02T15:44:59.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.633+0000 7fe9a1200700 1 -- 172.21.15.1:0/881808493 shutdown_connections 2024-02-02T15:44:59.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.633+0000 7fe9a1200700 1 --2- 172.21.15.1:0/881808493 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe99c10d630 0x7fe99c10fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:59.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.633+0000 7fe9a1200700 1 --2- 172.21.15.1:0/881808493 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe99c073110 0x7fe99c0734f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:59.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.633+0000 7fe9a1200700 1 --2- 172.21.15.1:0/881808493 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe99c073a30 0x7fe99c10d0f0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:59.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.633+0000 7fe9a1200700 1 -- 172.21.15.1:0/881808493 >> 172.21.15.1:0/881808493 conn(0x7fe99c0fc840 msgr2=0x7fe99c0fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:59.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.633+0000 7fe9a1200700 1 -- 172.21.15.1:0/881808493 shutdown_connections 2024-02-02T15:44:59.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.633+0000 7fe9a1200700 1 -- 172.21.15.1:0/881808493 wait complete. 2024-02-02T15:44:59.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.634+0000 7fe9a1200700 1 Processor -- start 2024-02-02T15:44:59.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.635+0000 7fe9a1200700 1 -- start start 2024-02-02T15:44:59.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.635+0000 7fe9a1200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe99c073110 0x7fe99c199140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:59.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.635+0000 7fe9a1200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe99c073a30 0x7fe99c199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:59.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.635+0000 7fe9a1200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe99c10d630 0x7fe99c19da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:59.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.635+0000 7fe9a1200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe99c19e0b0 con 0x7fe99c073a30 2024-02-02T15:44:59.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.635+0000 7fe9a1200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe99c19e220 con 0x7fe99c10d630 2024-02-02T15:44:59.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.635+0000 7fe9a1200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe99c19e3e0 con 0x7fe99c073110 2024-02-02T15:44:59.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.636+0000 7fe999e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe99c073110 0x7fe99c199140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:59.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.636+0000 7fe999e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe99c073110 0x7fe99c199140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:50624/0 (socket says 172.21.15.1:50624) 2024-02-02T15:44:59.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.636+0000 7fe999e00700 1 -- 172.21.15.1:0/4056826813 learned_addr learned my addr 172.21.15.1:0/4056826813 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:44:59.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.636+0000 7fe99a800700 1 --2- 172.21.15.1:0/4056826813 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe99c10d630 0x7fe99c19da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:59.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.636+0000 7fe999400700 1 --2- 172.21.15.1:0/4056826813 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe99c073a30 0x7fe99c199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:59.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.636+0000 7fe999e00700 1 -- 172.21.15.1:0/4056826813 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe99c10d630 msgr2=0x7fe99c19da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:59.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.636+0000 7fe999e00700 1 --2- 172.21.15.1:0/4056826813 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe99c10d630 0x7fe99c19da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:59.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.636+0000 7fe999e00700 1 -- 172.21.15.1:0/4056826813 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe99c073a30 msgr2=0x7fe99c199680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:59.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.636+0000 7fe999e00700 1 --2- 172.21.15.1:0/4056826813 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe99c073a30 0x7fe99c199680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:59.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.636+0000 7fe999e00700 1 -- 172.21.15.1:0/4056826813 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe98c0095f0 con 0x7fe99c073110 2024-02-02T15:44:59.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.636+0000 7fe999400700 1 --2- 172.21.15.1:0/4056826813 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe99c073a30 0x7fe99c199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:44:59.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.637+0000 7fe999e00700 1 --2- 172.21.15.1:0/4056826813 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe99c073110 0x7fe99c199140 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fe98c00ec10 tx=0x7fe98c00ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:59.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.637+0000 7fe992a00700 1 -- 172.21.15.1:0/4056826813 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe98c00cd40 con 0x7fe99c073110 2024-02-02T15:44:59.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.637+0000 7fe992a00700 1 -- 172.21.15.1:0/4056826813 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe98c0044e0 con 0x7fe99c073110 2024-02-02T15:44:59.638 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.637+0000 7fe992a00700 1 -- 172.21.15.1:0/4056826813 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe98c010690 con 0x7fe99c073110 2024-02-02T15:44:59.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.638+0000 7fe9a1200700 1 -- 172.21.15.1:0/4056826813 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe984009520 con 0x7fe99c073110 2024-02-02T15:44:59.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.638+0000 7fe9a1200700 1 -- 172.21.15.1:0/4056826813 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe99c1a5d20 con 0x7fe99c073110 2024-02-02T15:44:59.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.643+0000 7fe992a00700 1 -- 172.21.15.1:0/4056826813 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7fe98c003710 con 0x7fe99c073110 2024-02-02T15:44:59.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.646+0000 7fe9b4ce6b80 1 Processor -- start 2024-02-02T15:44:59.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.646+0000 7fe992a00700 1 --2- 172.21.15.1:0/4056826813 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fe97c0a0710 0x7fe97c0a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:59.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.647+0000 7fe992a00700 1 -- 172.21.15.1:0/4056826813 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fe98c014070 con 0x7fe99c073110 2024-02-02T15:44:59.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.648+0000 7fe999400700 1 --2- 172.21.15.1:0/4056826813 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fe97c0a0710 0x7fe97c0a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:59.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.648+0000 7fe9b4ce6b80 1 -- start start 2024-02-02T15:44:59.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.650+0000 7fe9b4ce6b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558af21e2010 0x558af21e23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:59.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.650+0000 7fe9b4ce6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558af21e2930 0x558af21ed2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:59.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.650+0000 7fe9b4ce6b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558af21ed7e0 0x558af21efbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:59.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.650+0000 7fe9b4ce6b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x558af21564c0 con 0x558af21e2010 2024-02-02T15:44:59.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.650+0000 7fe9b4ce6b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x558af213aaa0 con 0x558af21ed7e0 2024-02-02T15:44:59.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.650+0000 7fe9b4ce6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558af2138460 con 0x558af21e2930 2024-02-02T15:44:59.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.651+0000 7fe99a800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558af21ed7e0 0x558af21efbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:59.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.651+0000 7fe99a800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558af21ed7e0 0x558af21efbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:35420/0 (socket says 172.21.15.1:35420) 2024-02-02T15:44:59.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.651+0000 7fe99a800700 1 -- 172.21.15.1:0/3416846132 learned_addr learned my addr 172.21.15.1:0/3416846132 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:44:59.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.651+0000 7fe999400700 1 --2- 172.21.15.1:0/3416846132 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558af21e2930 0x558af21ed2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:59.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.652+0000 7fe999400700 1 --2- 172.21.15.1:0/4056826813 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fe97c0a0710 0x7fe97c0a2bd0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fe98400c010 tx=0x7fe984009f50 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:59.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.652+0000 7fe99a800700 1 -- 172.21.15.1:0/3416846132 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558af21e2930 msgr2=0x558af21ed2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:59.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.652+0000 7fe99a800700 1 --2- 172.21.15.1:0/3416846132 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558af21e2930 0x558af21ed2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:59.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.652+0000 7fe99a800700 1 -- 172.21.15.1:0/3416846132 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558af21e2010 msgr2=0x558af21e23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:59.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.652+0000 7fe99a800700 1 --2- 172.21.15.1:0/3416846132 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558af21e2010 0x558af21e23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:59.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.652+0000 7fe99a800700 1 -- 172.21.15.1:0/3416846132 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558af1fb53b0 con 0x558af21ed7e0 2024-02-02T15:44:59.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.653+0000 7fe99a800700 1 --2- 172.21.15.1:0/3416846132 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558af21ed7e0 0x558af21efbd0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fe994011630 tx=0x7fe994009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24948ccc16fdbfca server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:59.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.653+0000 7fe98b400700 1 -- 172.21.15.1:0/3416846132 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe99400e040 con 0x558af21ed7e0 2024-02-02T15:44:59.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.653+0000 7fe98b400700 1 -- 172.21.15.1:0/3416846132 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe99400f040 con 0x558af21ed7e0 2024-02-02T15:44:59.653 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.653+0000 7fe98b400700 1 -- 172.21.15.1:0/3416846132 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9940047d0 con 0x558af21ed7e0 2024-02-02T15:44:59.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.654+0000 7fe9b4ce6b80 1 -- 172.21.15.1:0/3416846132 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558af21ed7e0 msgr2=0x558af21efbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:59.654 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.654+0000 7fe9b4ce6b80 1 --2- 172.21.15.1:0/3416846132 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558af21ed7e0 0x558af21efbd0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fe994011630 tx=0x7fe994009920 comp rx=0 tx=0).stop 2024-02-02T15:44:59.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.655+0000 7fe9b4ce6b80 1 -- 172.21.15.1:0/3416846132 shutdown_connections 2024-02-02T15:44:59.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.655+0000 7fe9b4ce6b80 1 --2- 172.21.15.1:0/3416846132 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558af21ed7e0 0x558af21efbd0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:59.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.655+0000 7fe9b4ce6b80 1 --2- 172.21.15.1:0/3416846132 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558af21e2010 0x558af21e23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:59.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.655+0000 7fe9b4ce6b80 1 --2- 172.21.15.1:0/3416846132 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558af21e2930 0x558af21ed2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:59.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.655+0000 7fe9b4ce6b80 1 -- 172.21.15.1:0/3416846132 >> 172.21.15.1:0/3416846132 conn(0x558af2141db0 msgr2=0x558af21de3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:59.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.655+0000 7fe9b4ce6b80 1 -- 172.21.15.1:0/3416846132 shutdown_connections 2024-02-02T15:44:59.655 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.655+0000 7fe9b4ce6b80 1 -- 172.21.15.1:0/3416846132 wait complete. 2024-02-02T15:44:59.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.656+0000 7fe9b4ce6b80 1 -- start start 2024-02-02T15:44:59.657 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.657+0000 7fe9b4ce6b80 1 Processor -- start 2024-02-02T15:44:59.658 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.658+0000 7fe9b4ce6b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558af21e2010 0x558af21ed280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:59.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.659+0000 7fe9b4ce6b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558af21e2930 0x558af21ea500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:59.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.659+0000 7fe9b4ce6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558af21ed7e0 0x558af21eac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:59.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.659+0000 7fe999e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558af21e2010 0x558af21ed280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:59.659 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.659+0000 7fe999400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558af21e2930 0x558af21ea500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:59.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.659+0000 7fe9b4ce6b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x558af2133d50 con 0x558af21e2930 2024-02-02T15:44:59.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.659+0000 7fe9b4ce6b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x558af20cfd80 con 0x558af21e2010 2024-02-02T15:44:59.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.659+0000 7fe9b4ce6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558af20cfa20 con 0x558af21ed7e0 2024-02-02T15:44:59.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.659+0000 7fe999e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558af21e2010 0x558af21ed280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:35426/0 (socket says 172.21.15.1:35426) 2024-02-02T15:44:59.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.659+0000 7fe999e00700 1 -- 172.21.15.1:0/1990891505 learned_addr learned my addr 172.21.15.1:0/1990891505 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:44:59.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.660+0000 7fe99a800700 1 --2- 172.21.15.1:0/1990891505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558af21ed7e0 0x558af21eac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:59.660 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.660+0000 7fe9b4ce6b80 10 client.0 _ll_register_callbacks cb 0x558af20d1290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:44:59.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.660+0000 7fe99a800700 1 -- 172.21.15.1:0/1990891505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558af21e2010 msgr2=0x558af21ed280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:59.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.660+0000 7fe99a800700 1 --2- 172.21.15.1:0/1990891505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558af21e2010 0x558af21ed280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:59.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.660+0000 7fe99a800700 1 -- 172.21.15.1:0/1990891505 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558af21e2930 msgr2=0x558af21ea500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:59.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.660+0000 7fe99a800700 1 --2- 172.21.15.1:0/1990891505 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558af21e2930 0x558af21ea500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:59.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.660+0000 7fe99a800700 1 -- 172.21.15.1:0/1990891505 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9940112b0 con 0x558af21ed7e0 2024-02-02T15:44:59.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.660+0000 7fe999e00700 1 --2- 172.21.15.1:0/1990891505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558af21e2010 0x558af21ed280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:44:59.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.661+0000 7fe99a800700 1 --2- 172.21.15.1:0/1990891505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558af21ed7e0 0x558af21eac40 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fe994011630 tx=0x7fe994005b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e8d2b9bb220e717 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:59.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.661+0000 7fe983e00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:44:59.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.661+0000 7fe983e00700 1 -- 172.21.15.1:0/1990891505 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe99400e070 con 0x558af21ed7e0 2024-02-02T15:44:59.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.661+0000 7fe983e00700 1 -- 172.21.15.1:0/1990891505 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fe994003cd0 con 0x558af21ed7e0 2024-02-02T15:44:59.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.661+0000 7fe9b4ce6b80 1 -- 172.21.15.1:0/1990891505 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558af21d60f0 con 0x558af21ed7e0 2024-02-02T15:44:59.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.662+0000 7fe983e00700 1 -- 172.21.15.1:0/1990891505 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe994003e90 con 0x558af21ed7e0 2024-02-02T15:44:59.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.662+0000 7fe9b4ce6b80 10 client.30118 fetch_fsmap learned FSMap version 77 2024-02-02T15:44:59.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.662+0000 7fe9b4ce6b80 1 -- 172.21.15.1:0/1990891505 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=4+}) v3 -- 0x558af1f61b60 con 0x558af21ed7e0 2024-02-02T15:44:59.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.662+0000 7fe983e00700 1 -- 172.21.15.1:0/1990891505 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558af21d60f0 con 0x558af21ed7e0 2024-02-02T15:44:59.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.662+0000 7fe983e00700 1 -- 172.21.15.1:0/1990891505 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 77) v1 ==== 7502+0+0 (secure 0 0 0) 0x7fe994027460 con 0x558af21ed7e0 2024-02-02T15:44:59.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.662+0000 7fe9b4ce6b80 10 client.30118 fetch_fsmap finished waiting for FSMap version 77 2024-02-02T15:44:59.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.662+0000 7fe9b4ce6b80 10 client.30118 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:44:59.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.662+0000 7fe9b4ce6b80 20 client.30118 populate_metadata read hostname 'smithi001' 2024-02-02T15:44:59.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.662+0000 7fe9b4ce6b80 1 --2- 172.21.15.1:0/1990891505 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x558af22059f0 0x558af2207e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:59.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.662+0000 7fe9b4ce6b80 4 client.30118 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:44:59.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.662+0000 7fe9b4ce6b80 1 -- 172.21.15.1:0/1990891505 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558af212cdd0 con 0x558af22059f0 2024-02-02T15:44:59.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.665+0000 7fe999e00700 1 --2- 172.21.15.1:0/1990891505 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x558af22059f0 0x558af2207e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:59.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.665+0000 7fe999e00700 1 --2- 172.21.15.1:0/1990891505 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x558af22059f0 0x558af2207e10 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17985740c6afd5a5 server_cookie=a38fd8bee380d787 in_seq=0 out_seq=0 2024-02-02T15:44:59.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.665+0000 7fe983e00700 10 client.30118 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:44:59.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.666+0000 7fe983e00700 1 -- 172.21.15.1:0/1990891505 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x558af212cdd0 con 0x558af22059f0 2024-02-02T15:44:59.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.666+0000 7fe983e00700 10 client.30118 handle_command_reply: tid=0 2024-02-02T15:44:59.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.666+0000 7fe9b4ce6b80 1 client.30118 shutdown 2024-02-02T15:44:59.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.667+0000 7fe9b4ce6b80 1 -- 172.21.15.1:0/1990891505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558af21ed7e0 msgr2=0x558af21eac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:59.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.667+0000 7fe9b4ce6b80 1 --2- 172.21.15.1:0/1990891505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558af21ed7e0 0x558af21eac40 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fe994011630 tx=0x7fe994005b90 comp rx=0 tx=0).stop 2024-02-02T15:44:59.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.667+0000 7fe9b4ce6b80 1 -- 172.21.15.1:0/1990891505 shutdown_connections 2024-02-02T15:44:59.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.667+0000 7fe9b4ce6b80 1 --2- 172.21.15.1:0/1990891505 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x558af22059f0 0x558af2207e10 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:59.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.667+0000 7fe9b4ce6b80 1 --2- 172.21.15.1:0/1990891505 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558af21ed7e0 0x558af21eac40 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:59.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.667+0000 7fe9b4ce6b80 1 --2- 172.21.15.1:0/1990891505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558af21e2010 0x558af21ed280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:59.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.667+0000 7fe9b4ce6b80 1 --2- 172.21.15.1:0/1990891505 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558af21e2930 0x558af21ea500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:59.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.667+0000 7fe9b4ce6b80 1 -- 172.21.15.1:0/1990891505 >> 172.21.15.1:0/1990891505 conn(0x558af2141db0 msgr2=0x558af21ddfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:59.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.667+0000 7fe983e00700 0 client.30118 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:44:59.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.668+0000 7fe9b4ce6b80 1 -- 172.21.15.1:0/1990891505 shutdown_connections 2024-02-02T15:44:59.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.668+0000 7fe9b4ce6b80 1 -- 172.21.15.1:0/1990891505 wait complete. 2024-02-02T15:44:59.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.668+0000 7fe9b4ce6b80 20 client.30118 trim_cache size 0 max 16384 2024-02-02T15:44:59.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:44:59 smithi001 ceph-mon[35470]: pgmap v564: 129 pgs: 129 active+clean; 2.0 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.4 MiB/s wr, 535 op/s 2024-02-02T15:44:59.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.694+0000 7fe9b4ce6b80 1 Processor -- start 2024-02-02T15:44:59.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.694+0000 7fe9b4ce6b80 1 -- start start 2024-02-02T15:44:59.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.694+0000 7fe9b4ce6b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558af21e2010 0x558af21fcc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:59.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.695+0000 7fe9b4ce6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558af21e2930 0x558af21e5840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:59.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.695+0000 7fe9b4ce6b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558af21ed7e0 0x558af21e5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:59.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.695+0000 7fe9b4ce6b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x558af1fb76a0 con 0x558af21e2010 2024-02-02T15:44:59.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.695+0000 7fe9b4ce6b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x558af22024a0 con 0x558af21ed7e0 2024-02-02T15:44:59.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.695+0000 7fe9b4ce6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558af21eb500 con 0x558af21e2930 2024-02-02T15:44:59.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.695+0000 7fe999400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558af21e2930 0x558af21e5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:59.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.695+0000 7fe999400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558af21e2930 0x558af21e5840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:50658/0 (socket says 172.21.15.1:50658) 2024-02-02T15:44:59.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.695+0000 7fe999400700 1 -- 172.21.15.1:0/3344163917 learned_addr learned my addr 172.21.15.1:0/3344163917 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:44:59.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.696+0000 7fe99a800700 1 --2- 172.21.15.1:0/3344163917 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558af21ed7e0 0x558af21e5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:59.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.696+0000 7fe999400700 1 -- 172.21.15.1:0/3344163917 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558af21ed7e0 msgr2=0x558af21e5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:59.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.696+0000 7fe999400700 1 --2- 172.21.15.1:0/3344163917 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558af21ed7e0 0x558af21e5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:59.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.696+0000 7fe999400700 1 -- 172.21.15.1:0/3344163917 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558af21e2010 msgr2=0x558af21fcc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:59.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.696+0000 7fe999400700 1 --2- 172.21.15.1:0/3344163917 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558af21e2010 0x558af21fcc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:59.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.696+0000 7fe999400700 1 -- 172.21.15.1:0/3344163917 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558af20f9ea0 con 0x558af21e2930 2024-02-02T15:44:59.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.697+0000 7fe999400700 1 --2- 172.21.15.1:0/3344163917 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558af21e2930 0x558af21e5840 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fe9840098f0 tx=0x7fe984022000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf8459fd3651df7b server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:59.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.697+0000 7fe98be00700 1 -- 172.21.15.1:0/3344163917 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe984028050 con 0x558af21e2930 2024-02-02T15:44:59.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.698+0000 7fe98be00700 1 -- 172.21.15.1:0/3344163917 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe984029040 con 0x558af21e2930 2024-02-02T15:44:59.698 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.698+0000 7fe98be00700 1 -- 172.21.15.1:0/3344163917 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe98401c070 con 0x558af21e2930 2024-02-02T15:44:59.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.699+0000 7fe9b4ce6b80 1 -- 172.21.15.1:0/3344163917 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558af21e2930 msgr2=0x558af21e5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:59.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.699+0000 7fe9b4ce6b80 1 --2- 172.21.15.1:0/3344163917 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558af21e2930 0x558af21e5840 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fe9840098f0 tx=0x7fe984022000 comp rx=0 tx=0).stop 2024-02-02T15:44:59.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.699+0000 7fe9b4ce6b80 1 -- 172.21.15.1:0/3344163917 shutdown_connections 2024-02-02T15:44:59.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.699+0000 7fe9b4ce6b80 1 --2- 172.21.15.1:0/3344163917 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558af21ed7e0 0x558af21e5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:59.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.699+0000 7fe9b4ce6b80 1 --2- 172.21.15.1:0/3344163917 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558af21e2010 0x558af21fcc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:59.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.699+0000 7fe9b4ce6b80 1 --2- 172.21.15.1:0/3344163917 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558af21e2930 0x558af21e5840 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:59.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.699+0000 7fe9b4ce6b80 1 -- 172.21.15.1:0/3344163917 >> 172.21.15.1:0/3344163917 conn(0x558af2141db0 msgr2=0x558af22011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:59.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.700+0000 7fe9b4ce6b80 1 -- 172.21.15.1:0/3344163917 shutdown_connections 2024-02-02T15:44:59.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.701+0000 7fe9b4ce6b80 1 -- 172.21.15.1:0/3344163917 wait complete. 2024-02-02T15:44:59.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.701+0000 7fe9b4ce6b80 1 -- start start 2024-02-02T15:44:59.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.702+0000 7fe9b4ce6b80 1 Processor -- start 2024-02-02T15:44:59.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.703+0000 7fe9b4ce6b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558af21e2010 0x558af21fc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:59.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.703+0000 7fe9b4ce6b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558af21e2930 0x558af21fcc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:59.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.703+0000 7fe9b4ce6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558af21ed7e0 0x558af21fab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:59.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.703+0000 7fe9b4ce6b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x558af21fb2a0 con 0x558af21e2930 2024-02-02T15:44:59.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.703+0000 7fe9b4ce6b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x558af21fb410 con 0x558af21e2010 2024-02-02T15:44:59.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.703+0000 7fe9b4ce6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x558af21fb5d0 con 0x558af21ed7e0 2024-02-02T15:44:59.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.703+0000 7fe9b4ce6b80 10 client.0 _ll_register_callbacks cb 0x558af20d1290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:44:59.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.703+0000 7fe99a800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558af21ed7e0 0x558af21fab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:59.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.703+0000 7fe999e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558af21e2010 0x558af21fc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:59.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.703+0000 7fe99a800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558af21ed7e0 0x558af21fab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:50664/0 (socket says 172.21.15.1:50664) 2024-02-02T15:44:59.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.703+0000 7fe99a800700 1 -- 172.21.15.1:0/1478654380 learned_addr learned my addr 172.21.15.1:0/1478654380 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:44:59.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.704+0000 7fe999e00700 1 -- 172.21.15.1:0/1478654380 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558af21ed7e0 msgr2=0x558af21fab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:59.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.704+0000 7fe999e00700 1 --2- 172.21.15.1:0/1478654380 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558af21ed7e0 0x558af21fab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:59.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.704+0000 7fe999e00700 1 -- 172.21.15.1:0/1478654380 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558af21e2930 msgr2=0x558af21fcc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:44:59.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.704+0000 7fe999400700 1 --2- 172.21.15.1:0/1478654380 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558af21e2930 0x558af21fcc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:59.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.705+0000 7fe999e00700 1 --2- 172.21.15.1:0/1478654380 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558af21e2930 0x558af21fcc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:59.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.705+0000 7fe999e00700 1 -- 172.21.15.1:0/1478654380 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe99c110770 con 0x558af21e2010 2024-02-02T15:44:59.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.705+0000 7fe999400700 1 --2- 172.21.15.1:0/1478654380 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558af21e2930 0x558af21fcc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:44:59.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.705+0000 7fe999e00700 1 --2- 172.21.15.1:0/1478654380 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558af21e2010 0x558af21fc6d0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fe98c0994a0 tx=0x7fe98c099f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=240588b9be911a5e server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:44:59.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.705+0000 7fe983e00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:44:59.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.706+0000 7fe983e00700 1 -- 172.21.15.1:0/1478654380 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe98c09d5d0 con 0x558af21e2010 2024-02-02T15:44:59.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.706+0000 7fe983e00700 1 -- 172.21.15.1:0/1478654380 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fe98c0cc040 con 0x558af21e2010 2024-02-02T15:44:59.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.706+0000 7fe9b4ce6b80 1 -- 172.21.15.1:0/1478654380 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe99400f040 con 0x558af21e2010 2024-02-02T15:44:59.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.706+0000 7fe983e00700 1 -- 172.21.15.1:0/1478654380 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe98c0a1810 con 0x558af21e2010 2024-02-02T15:44:59.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.706+0000 7fe9b4ce6b80 10 client.30278 fetch_fsmap learned FSMap version 77 2024-02-02T15:44:59.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.706+0000 7fe9b4ce6b80 1 -- 172.21.15.1:0/1478654380 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=4+}) v3 -- 0x558af1fb9cd0 con 0x558af21e2010 2024-02-02T15:44:59.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.706+0000 7fe983e00700 1 -- 172.21.15.1:0/1478654380 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe99400f040 con 0x558af21e2010 2024-02-02T15:44:59.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.707+0000 7fe983e00700 1 -- 172.21.15.1:0/1478654380 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 77) v1 ==== 7502+0+0 (secure 0 0 0) 0x7fe98c0cc200 con 0x558af21e2010 2024-02-02T15:44:59.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.707+0000 7fe9b4ce6b80 10 client.30278 fetch_fsmap finished waiting for FSMap version 77 2024-02-02T15:44:59.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.707+0000 7fe9b4ce6b80 10 client.30278 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:44:59.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.707+0000 7fe9b4ce6b80 20 client.30278 populate_metadata read hostname 'smithi001' 2024-02-02T15:44:59.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.707+0000 7fe9b4ce6b80 1 --2- 172.21.15.1:0/1478654380 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x558af2258910 0x558af225ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:44:59.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.707+0000 7fe9b4ce6b80 4 client.30278 mds_command: new command op to 24557 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:44:59.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.707+0000 7fe9b4ce6b80 1 -- 172.21.15.1:0/1478654380 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x558af20f2f50 con 0x558af2258910 2024-02-02T15:44:59.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.707+0000 7fe999400700 1 --2- 172.21.15.1:0/1478654380 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x558af2258910 0x558af225ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:44:59.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.708+0000 7fe999400700 1 --2- 172.21.15.1:0/1478654380 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x558af2258910 0x558af225ad30 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47e4aea4cd79f964 server_cookie=293181f6bdd1624e in_seq=0 out_seq=0 2024-02-02T15:44:59.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.708+0000 7fe983e00700 10 client.30278 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:44:59.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.709+0000 7fe983e00700 1 -- 172.21.15.1:0/1478654380 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x558af20f2f50 con 0x558af2258910 2024-02-02T15:44:59.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.709+0000 7fe983e00700 10 client.30278 handle_command_reply: tid=0 2024-02-02T15:44:59.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.709+0000 7fe9b4ce6b80 1 client.30278 shutdown 2024-02-02T15:44:59.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.709+0000 7fe9b4ce6b80 1 -- 172.21.15.1:0/1478654380 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558af21e2010 msgr2=0x558af21fc6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:59.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.709+0000 7fe9b4ce6b80 1 --2- 172.21.15.1:0/1478654380 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558af21e2010 0x558af21fc6d0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fe98c0994a0 tx=0x7fe98c099f50 comp rx=0 tx=0).stop 2024-02-02T15:44:59.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.710+0000 7fe9b4ce6b80 1 -- 172.21.15.1:0/1478654380 shutdown_connections 2024-02-02T15:44:59.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.710+0000 7fe9b4ce6b80 1 --2- 172.21.15.1:0/1478654380 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x558af2258910 0x558af225ad30 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:59.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.710+0000 7fe9b4ce6b80 1 --2- 172.21.15.1:0/1478654380 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x558af21ed7e0 0x558af21fab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:59.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.710+0000 7fe9b4ce6b80 1 --2- 172.21.15.1:0/1478654380 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x558af21e2010 0x558af21fc6d0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:59.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.710+0000 7fe9b4ce6b80 1 --2- 172.21.15.1:0/1478654380 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x558af21e2930 0x558af21fcc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:59.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.710+0000 7fe9b4ce6b80 1 -- 172.21.15.1:0/1478654380 >> 172.21.15.1:0/1478654380 conn(0x558af2141db0 msgr2=0x558af21e5840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:59.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.710+0000 7fe983e00700 0 client.30278 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:44:59.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.710+0000 7fe9b4ce6b80 1 -- 172.21.15.1:0/1478654380 shutdown_connections 2024-02-02T15:44:59.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.710+0000 7fe9b4ce6b80 1 -- 172.21.15.1:0/1478654380 wait complete. 2024-02-02T15:44:59.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.711+0000 7fe9b4ce6b80 20 client.30278 trim_cache size 0 max 16384 2024-02-02T15:44:59.712 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:44:59.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.714+0000 7fe9a1200700 1 -- 172.21.15.1:0/4056826813 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fe97c0a0710 msgr2=0x7fe97c0a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:59.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.714+0000 7fe9a1200700 1 --2- 172.21.15.1:0/4056826813 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fe97c0a0710 0x7fe97c0a2bd0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fe98400c010 tx=0x7fe984009f50 comp rx=0 tx=0).stop 2024-02-02T15:44:59.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.714+0000 7fe9a1200700 1 -- 172.21.15.1:0/4056826813 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe99c073110 msgr2=0x7fe99c199140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:44:59.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.714+0000 7fe9a1200700 1 --2- 172.21.15.1:0/4056826813 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe99c073110 0x7fe99c199140 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fe98c00ec10 tx=0x7fe98c00ef20 comp rx=0 tx=0).stop 2024-02-02T15:44:59.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.714+0000 7fe9a1200700 1 -- 172.21.15.1:0/4056826813 shutdown_connections 2024-02-02T15:44:59.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.714+0000 7fe9a1200700 1 --2- 172.21.15.1:0/4056826813 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fe97c0a0710 0x7fe97c0a2bd0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:59.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.714+0000 7fe9a1200700 1 --2- 172.21.15.1:0/4056826813 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe99c10d630 0x7fe99c19da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:59.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.714+0000 7fe9a1200700 1 --2- 172.21.15.1:0/4056826813 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fe99c073110 0x7fe99c199140 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:59.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.714+0000 7fe9a1200700 1 --2- 172.21.15.1:0/4056826813 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fe99c073a30 0x7fe99c199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:44:59.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.714+0000 7fe9a1200700 1 -- 172.21.15.1:0/4056826813 >> 172.21.15.1:0/4056826813 conn(0x7fe99c0fc840 msgr2=0x7fe99c10f790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:44:59.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.714+0000 7fe9a1200700 1 -- 172.21.15.1:0/4056826813 shutdown_connections 2024-02-02T15:44:59.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:44:59.714+0000 7fe9a1200700 1 -- 172.21.15.1:0/4056826813 wait complete. 2024-02-02T15:44:59.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:44:59 smithi071 ceph-mon[44707]: pgmap v564: 129 pgs: 129 active+clean; 2.0 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.4 MiB/s wr, 535 op/s 2024-02-02T15:44:59.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:44:59 smithi142 ceph-mon[45728]: pgmap v564: 129 pgs: 129 active+clean; 2.0 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.4 MiB/s wr, 535 op/s 2024-02-02T15:44:59.998 INFO:tasks.workunit.client.0.smithi001.stdout: 10 149379 28.25 MB/sec execute 152 sec latency 84.384 ms 2024-02-02T15:45:00.743 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-02T15:45:00.744 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-02T15:45:00.999 INFO:tasks.workunit.client.0.smithi001.stdout: 10 149884 28.21 MB/sec execute 153 sec latency 241.287 ms 2024-02-02T15:45:01.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.110+0000 7f6a1c800700 1 -- 172.21.15.1:0/2478487295 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a14107600 msgr2=0x7f6a141079e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:01.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.110+0000 7f6a1c800700 1 --2- 172.21.15.1:0/2478487295 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a14107600 0x7f6a141079e0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f6a0400b0a0 tx=0x7f6a0400b3b0 comp rx=0 tx=0).stop 2024-02-02T15:45:01.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.110+0000 7f6a1c800700 1 -- 172.21.15.1:0/2478487295 shutdown_connections 2024-02-02T15:45:01.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.110+0000 7f6a1c800700 1 --2- 172.21.15.1:0/2478487295 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a1410c720 0x7f6a141142a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:01.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.110+0000 7f6a1c800700 1 --2- 172.21.15.1:0/2478487295 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a14107600 0x7f6a141079e0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:01.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.110+0000 7f6a1c800700 1 --2- 172.21.15.1:0/2478487295 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a14107fb0 0x7f6a1410bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:01.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.111+0000 7f6a1c800700 1 -- 172.21.15.1:0/2478487295 >> 172.21.15.1:0/2478487295 conn(0x7f6a14076b80 msgr2=0x7f6a14105b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:45:01.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.111+0000 7f6a1c800700 1 -- 172.21.15.1:0/2478487295 shutdown_connections 2024-02-02T15:45:01.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.112+0000 7f6a1c800700 1 -- 172.21.15.1:0/2478487295 wait complete. 2024-02-02T15:45:01.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.114+0000 7f6a1c800700 1 Processor -- start 2024-02-02T15:45:01.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.115+0000 7f6a1c800700 1 -- start start 2024-02-02T15:45:01.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.116+0000 7f6a1c800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a14107600 0x7f6a14081050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:01.116 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.116+0000 7f6a1c800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a1410c720 0x7f6a140815d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:01.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.116+0000 7f6a1c800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a1407a060 0x7f6a1407a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:01.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.116+0000 7f6a1c800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a1407abb0 con 0x7f6a14107600 2024-02-02T15:45:01.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.116+0000 7f6a1c800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a1407ad20 con 0x7f6a1410c720 2024-02-02T15:45:01.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.116+0000 7f6a1c800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a1407aee0 con 0x7f6a1407a060 2024-02-02T15:45:01.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.116+0000 7f6a19400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a1410c720 0x7f6a140815d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:01.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.116+0000 7f6a19400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a1410c720 0x7f6a140815d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:35468/0 (socket says 172.21.15.1:35468) 2024-02-02T15:45:01.117 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.116+0000 7f6a19400700 1 -- 172.21.15.1:0/3256222712 learned_addr learned my addr 172.21.15.1:0/3256222712 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:45:01.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.117+0000 7f6a19400700 1 -- 172.21.15.1:0/3256222712 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a1407a060 msgr2=0x7f6a1407a510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-02T15:45:01.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.117+0000 7f6a19e00700 1 --2- 172.21.15.1:0/3256222712 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a14107600 0x7f6a14081050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:01.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.117+0000 7f6a19400700 1 --2- 172.21.15.1:0/3256222712 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a1407a060 0x7f6a1407a510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:01.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.117+0000 7f6a19400700 1 -- 172.21.15.1:0/3256222712 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a14107600 msgr2=0x7f6a14081050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:01.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.117+0000 7f6a19400700 1 --2- 172.21.15.1:0/3256222712 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a14107600 0x7f6a14081050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:01.118 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.117+0000 7f6a19400700 1 -- 172.21.15.1:0/3256222712 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a04009d00 con 0x7f6a1410c720 2024-02-02T15:45:01.121 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.121+0000 7f6a19400700 1 --2- 172.21.15.1:0/3256222712 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a1410c720 0x7f6a140815d0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f69fc00e970 tx=0x7f69fc00ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:45:01.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.123+0000 7f6a12a00700 1 -- 172.21.15.1:0/3256222712 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69fc009790 con 0x7f6a1410c720 2024-02-02T15:45:01.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.123+0000 7f6a12a00700 1 -- 172.21.15.1:0/3256222712 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f69fc004cf0 con 0x7f6a1410c720 2024-02-02T15:45:01.123 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.123+0000 7f6a12a00700 1 -- 172.21.15.1:0/3256222712 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69fc005680 con 0x7f6a1410c720 2024-02-02T15:45:01.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.123+0000 7f6a1c800700 1 -- 172.21.15.1:0/3256222712 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a1407ed30 con 0x7f6a1410c720 2024-02-02T15:45:01.124 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.124+0000 7f6a1c800700 1 -- 172.21.15.1:0/3256222712 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a1407f1f0 con 0x7f6a1410c720 2024-02-02T15:45:01.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.130+0000 7f6a12a00700 1 -- 172.21.15.1:0/3256222712 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f69fc004750 con 0x7f6a1410c720 2024-02-02T15:45:01.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.132+0000 7f6a12a00700 1 --2- 172.21.15.1:0/3256222712 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f69f40a0860 0x7f69f40a2d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:01.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.132+0000 7f6a12a00700 1 -- 172.21.15.1:0/3256222712 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f69fc014070 con 0x7f6a1410c720 2024-02-02T15:45:01.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.132+0000 7f6a302d6b80 1 Processor -- start 2024-02-02T15:45:01.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.133+0000 7f6a19e00700 1 --2- 172.21.15.1:0/3256222712 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f69f40a0860 0x7f69f40a2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:01.133 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.134+0000 7f6a302d6b80 1 -- start start 2024-02-02T15:45:01.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.134+0000 7f6a302d6b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562915d17090 0x562915d17470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:01.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.134+0000 7f6a302d6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562915d179b0 0x562915d22320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:01.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.134+0000 7f6a302d6b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562915d22860 0x562915d24c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:01.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.135+0000 7f6a19400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562915d179b0 0x562915d22320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:01.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.135+0000 7f6a19400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562915d179b0 0x562915d22320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:50682/0 (socket says 172.21.15.1:50682) 2024-02-02T15:45:01.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.135+0000 7f6a1a800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562915d22860 0x562915d24c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:01.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.135+0000 7f6a1a800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562915d22860 0x562915d24c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44274/0 (socket says 172.21.15.1:44274) 2024-02-02T15:45:01.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.136+0000 7f6a302d6b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x562915c8b540 con 0x562915d22860 2024-02-02T15:45:01.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.136+0000 7f6a302d6b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x562915c6fb20 con 0x562915d17090 2024-02-02T15:45:01.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.136+0000 7f6a302d6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x562915c6d4e0 con 0x562915d179b0 2024-02-02T15:45:01.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.136+0000 7f6a19e00700 1 --2- 172.21.15.1:0/3256222712 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f69f40a0860 0x7f69f40a2d20 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f6a0400b070 tx=0x7f6a0400b700 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:45:01.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.136+0000 7f6a19400700 1 -- 172.21.15.1:0/2445165185 learned_addr learned my addr 172.21.15.1:0/2445165185 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:45:01.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.136+0000 7f6a19e00700 1 --2- 172.21.15.1:0/2445165185 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562915d17090 0x562915d17470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:01.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.137+0000 7f6a1a800700 1 -- 172.21.15.1:0/2445165185 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562915d179b0 msgr2=0x562915d22320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:45:01.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.137+0000 7f6a1a800700 1 --2- 172.21.15.1:0/2445165185 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562915d179b0 0x562915d22320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:01.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.137+0000 7f6a1a800700 1 -- 172.21.15.1:0/2445165185 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562915d17090 msgr2=0x562915d17470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:45:01.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.137+0000 7f6a1a800700 1 --2- 172.21.15.1:0/2445165185 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562915d17090 0x562915d17470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:01.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.137+0000 7f6a1a800700 1 -- 172.21.15.1:0/2445165185 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562915aea430 con 0x562915d22860 2024-02-02T15:45:01.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.137+0000 7f6a19400700 1 --2- 172.21.15.1:0/2445165185 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562915d179b0 0x562915d22320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:45:01.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.137+0000 7f6a19e00700 1 --2- 172.21.15.1:0/2445165185 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562915d17090 0x562915d17470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:45:01.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.138+0000 7f6a1a800700 1 --2- 172.21.15.1:0/2445165185 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562915d22860 0x562915d24c50 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c00b950 tx=0x7f6a0c00bc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8510e5a69c68e9aa server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:45:01.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.139+0000 7f6a0b000700 1 -- 172.21.15.1:0/2445165185 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a0c004220 con 0x562915d22860 2024-02-02T15:45:01.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.139+0000 7f6a0b000700 1 -- 172.21.15.1:0/2445165185 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a0c004cf0 con 0x562915d22860 2024-02-02T15:45:01.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.139+0000 7f6a0b000700 1 -- 172.21.15.1:0/2445165185 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a0c004220 con 0x562915d22860 2024-02-02T15:45:01.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.140+0000 7f6a302d6b80 1 -- 172.21.15.1:0/2445165185 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562915d22860 msgr2=0x562915d24c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:01.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.140+0000 7f6a302d6b80 1 --2- 172.21.15.1:0/2445165185 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562915d22860 0x562915d24c50 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c00b950 tx=0x7f6a0c00bc60 comp rx=0 tx=0).stop 2024-02-02T15:45:01.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.140+0000 7f6a302d6b80 1 -- 172.21.15.1:0/2445165185 shutdown_connections 2024-02-02T15:45:01.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.140+0000 7f6a302d6b80 1 --2- 172.21.15.1:0/2445165185 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562915d22860 0x562915d24c50 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:01.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.140+0000 7f6a302d6b80 1 --2- 172.21.15.1:0/2445165185 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562915d17090 0x562915d17470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:01.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.140+0000 7f6a302d6b80 1 --2- 172.21.15.1:0/2445165185 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562915d179b0 0x562915d22320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:01.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.140+0000 7f6a302d6b80 1 -- 172.21.15.1:0/2445165185 >> 172.21.15.1:0/2445165185 conn(0x562915c76e30 msgr2=0x562915d13430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:45:01.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.141+0000 7f6a302d6b80 1 -- 172.21.15.1:0/2445165185 shutdown_connections 2024-02-02T15:45:01.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.141+0000 7f6a302d6b80 1 -- 172.21.15.1:0/2445165185 wait complete. 2024-02-02T15:45:01.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.141+0000 7f6a302d6b80 1 -- start start 2024-02-02T15:45:01.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.142+0000 7f6a302d6b80 1 Processor -- start 2024-02-02T15:45:01.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.143+0000 7f6a302d6b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562915d17090 0x562915d222b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:01.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.143+0000 7f6a302d6b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562915d179b0 0x562915d1f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:01.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.143+0000 7f6a302d6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562915d22860 0x562915d1fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:01.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.143+0000 7f6a302d6b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x562915c68dd0 con 0x562915d17090 2024-02-02T15:45:01.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.143+0000 7f6a302d6b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x562915c04e00 con 0x562915d179b0 2024-02-02T15:45:01.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.143+0000 7f6a302d6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x562915c04aa0 con 0x562915d22860 2024-02-02T15:45:01.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.143+0000 7f6a302d6b80 10 client.0 _ll_register_callbacks cb 0x562915c06310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:45:01.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.143+0000 7f6a1a800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562915d22860 0x562915d1fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:01.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.143+0000 7f6a1a800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562915d22860 0x562915d1fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:50698/0 (socket says 172.21.15.1:50698) 2024-02-02T15:45:01.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.143+0000 7f6a1a800700 1 -- 172.21.15.1:0/2357772107 learned_addr learned my addr 172.21.15.1:0/2357772107 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:45:01.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.144+0000 7f6a1a800700 1 -- 172.21.15.1:0/2357772107 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562915d179b0 msgr2=0x562915d1f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:45:01.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.144+0000 7f6a1a800700 1 --2- 172.21.15.1:0/2357772107 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562915d179b0 0x562915d1f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:01.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.144+0000 7f6a1a800700 1 -- 172.21.15.1:0/2357772107 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562915d17090 msgr2=0x562915d222b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:45:01.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.144+0000 7f6a1a800700 1 --2- 172.21.15.1:0/2357772107 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562915d17090 0x562915d222b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:01.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.144+0000 7f6a1a800700 1 -- 172.21.15.1:0/2357772107 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a1407f1f0 con 0x562915d22860 2024-02-02T15:45:01.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.144+0000 7f6a1a800700 1 --2- 172.21.15.1:0/2357772107 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562915d22860 0x562915d1fc00 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c00bfb0 tx=0x7f6a0c0049b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2fe5034788e13d98 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:45:01.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.144+0000 7f6a03400700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:45:01.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.145+0000 7f6a03400700 1 -- 172.21.15.1:0/2357772107 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a0c005670 con 0x562915d22860 2024-02-02T15:45:01.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.145+0000 7f6a03400700 1 -- 172.21.15.1:0/2357772107 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f6a0c00d5e0 con 0x562915d22860 2024-02-02T15:45:01.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.146+0000 7f6a302d6b80 1 -- 172.21.15.1:0/2357772107 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562915d0b170 con 0x562915d22860 2024-02-02T15:45:01.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.146+0000 7f6a03400700 1 -- 172.21.15.1:0/2357772107 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a0c00d7a0 con 0x562915d22860 2024-02-02T15:45:01.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.147+0000 7f6a302d6b80 10 client.30133 fetch_fsmap learned FSMap version 77 2024-02-02T15:45:01.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.147+0000 7f6a302d6b80 1 -- 172.21.15.1:0/2357772107 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=4+}) v3 -- 0x7f6a0c00b580 con 0x562915d22860 2024-02-02T15:45:01.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.147+0000 7f6a03400700 1 -- 172.21.15.1:0/2357772107 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x562915d0b170 con 0x562915d22860 2024-02-02T15:45:01.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.147+0000 7f6a03400700 1 -- 172.21.15.1:0/2357772107 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 77) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f6a0c02b070 con 0x562915d22860 2024-02-02T15:45:01.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.147+0000 7f6a302d6b80 10 client.30133 fetch_fsmap finished waiting for FSMap version 77 2024-02-02T15:45:01.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.148+0000 7f6a302d6b80 10 client.30133 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:45:01.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.148+0000 7f6a302d6b80 20 client.30133 populate_metadata read hostname 'smithi001' 2024-02-02T15:45:01.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.148+0000 7f6a302d6b80 1 --2- 172.21.15.1:0/2357772107 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x562915d320f0 0x562915d344e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:01.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.148+0000 7f6a302d6b80 4 client.30133 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:45:01.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.148+0000 7f6a302d6b80 1 -- 172.21.15.1:0/2357772107 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562915c61e50 con 0x562915d320f0 2024-02-02T15:45:01.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.149+0000 7f6a19e00700 1 --2- 172.21.15.1:0/2357772107 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x562915d320f0 0x562915d344e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:01.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.151+0000 7f6a19e00700 1 --2- 172.21.15.1:0/2357772107 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x562915d320f0 0x562915d344e0 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b417ed9ccd84cc4e server_cookie=855632a926d9742a in_seq=0 out_seq=0 2024-02-02T15:45:01.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.151+0000 7f6a03400700 10 client.30133 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:45:01.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.152+0000 7f6a03400700 1 -- 172.21.15.1:0/2357772107 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x562915c61e50 con 0x562915d320f0 2024-02-02T15:45:01.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.152+0000 7f6a03400700 10 client.30133 handle_command_reply: tid=0 2024-02-02T15:45:01.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.152+0000 7f6a302d6b80 1 client.30133 shutdown 2024-02-02T15:45:01.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.153+0000 7f6a302d6b80 1 -- 172.21.15.1:0/2357772107 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562915d22860 msgr2=0x562915d1fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:01.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.153+0000 7f6a302d6b80 1 --2- 172.21.15.1:0/2357772107 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562915d22860 0x562915d1fc00 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c00bfb0 tx=0x7f6a0c0049b0 comp rx=0 tx=0).stop 2024-02-02T15:45:01.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.154+0000 7f6a302d6b80 1 -- 172.21.15.1:0/2357772107 shutdown_connections 2024-02-02T15:45:01.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.154+0000 7f6a302d6b80 1 --2- 172.21.15.1:0/2357772107 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x562915d320f0 0x562915d344e0 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:01.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.154+0000 7f6a302d6b80 1 --2- 172.21.15.1:0/2357772107 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562915d22860 0x562915d1fc00 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:01.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.154+0000 7f6a302d6b80 1 --2- 172.21.15.1:0/2357772107 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562915d17090 0x562915d222b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:01.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.154+0000 7f6a302d6b80 1 --2- 172.21.15.1:0/2357772107 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562915d179b0 0x562915d1f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:01.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.154+0000 7f6a302d6b80 1 -- 172.21.15.1:0/2357772107 >> 172.21.15.1:0/2357772107 conn(0x562915c76e30 msgr2=0x562915d13020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:45:01.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.154+0000 7f6a03400700 0 client.30133 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:45:01.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.154+0000 7f6a302d6b80 1 -- 172.21.15.1:0/2357772107 shutdown_connections 2024-02-02T15:45:01.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.154+0000 7f6a302d6b80 1 -- 172.21.15.1:0/2357772107 wait complete. 2024-02-02T15:45:01.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.154+0000 7f6a302d6b80 20 client.30133 trim_cache size 0 max 16384 2024-02-02T15:45:01.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.193+0000 7f6a302d6b80 1 Processor -- start 2024-02-02T15:45:01.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.193+0000 7f6a302d6b80 1 -- start start 2024-02-02T15:45:01.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.194+0000 7f6a302d6b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562915d17090 0x562915d293f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:01.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.194+0000 7f6a302d6b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562915d179b0 0x562915d21de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:01.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.194+0000 7f6a302d6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562915d22860 0x562915d1e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:01.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.194+0000 7f6a302d6b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x562915aec720 con 0x562915d17090 2024-02-02T15:45:01.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.194+0000 7f6a302d6b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x562915d0fa30 con 0x562915d179b0 2024-02-02T15:45:01.194 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.194+0000 7f6a302d6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x562915d1b330 con 0x562915d22860 2024-02-02T15:45:01.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.195+0000 7f6a19e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562915d17090 0x562915d293f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:01.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.195+0000 7f6a1a800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562915d22860 0x562915d1e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:01.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.195+0000 7f6a1a800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562915d22860 0x562915d1e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:50708/0 (socket says 172.21.15.1:50708) 2024-02-02T15:45:01.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.195+0000 7f6a1a800700 1 -- 172.21.15.1:0/130967973 learned_addr learned my addr 172.21.15.1:0/130967973 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:45:01.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.196+0000 7f6a1a800700 1 -- 172.21.15.1:0/130967973 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562915d179b0 msgr2=0x562915d21de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:45:01.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.196+0000 7f6a1a800700 1 --2- 172.21.15.1:0/130967973 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562915d179b0 0x562915d21de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:01.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.196+0000 7f6a1a800700 1 -- 172.21.15.1:0/130967973 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562915d17090 msgr2=0x562915d293f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:45:01.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.196+0000 7f6a1a800700 1 --2- 172.21.15.1:0/130967973 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562915d17090 0x562915d293f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:01.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.196+0000 7f6a1a800700 1 -- 172.21.15.1:0/130967973 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562915d2e910 con 0x562915d22860 2024-02-02T15:45:01.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.196+0000 7f6a1a800700 1 --2- 172.21.15.1:0/130967973 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562915d22860 0x562915d1e130 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c00bca0 tx=0x7f6a0c004f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3be5f6f37dd8bda0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:45:01.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.196+0000 7f6a0ba00700 1 -- 172.21.15.1:0/130967973 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a0c005b00 con 0x562915d22860 2024-02-02T15:45:01.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.197+0000 7f6a0ba00700 1 -- 172.21.15.1:0/130967973 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a0c02be80 con 0x562915d22860 2024-02-02T15:45:01.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.197+0000 7f6a0ba00700 1 -- 172.21.15.1:0/130967973 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a0c02b710 con 0x562915d22860 2024-02-02T15:45:01.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.198+0000 7f6a302d6b80 1 -- 172.21.15.1:0/130967973 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562915d22860 msgr2=0x562915d1e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:01.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.198+0000 7f6a302d6b80 1 --2- 172.21.15.1:0/130967973 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562915d22860 0x562915d1e130 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c00bca0 tx=0x7f6a0c004f90 comp rx=0 tx=0).stop 2024-02-02T15:45:01.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.199+0000 7f6a302d6b80 1 -- 172.21.15.1:0/130967973 shutdown_connections 2024-02-02T15:45:01.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.199+0000 7f6a302d6b80 1 --2- 172.21.15.1:0/130967973 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562915d22860 0x562915d1e130 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:01.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.199+0000 7f6a302d6b80 1 --2- 172.21.15.1:0/130967973 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562915d17090 0x562915d293f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:01.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.199+0000 7f6a302d6b80 1 --2- 172.21.15.1:0/130967973 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562915d179b0 0x562915d21de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:01.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.199+0000 7f6a302d6b80 1 -- 172.21.15.1:0/130967973 >> 172.21.15.1:0/130967973 conn(0x562915c76e30 msgr2=0x562915d1da20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:45:01.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.199+0000 7f6a302d6b80 1 -- 172.21.15.1:0/130967973 shutdown_connections 2024-02-02T15:45:01.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.199+0000 7f6a302d6b80 1 -- 172.21.15.1:0/130967973 wait complete. 2024-02-02T15:45:01.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.200+0000 7f6a302d6b80 1 -- start start 2024-02-02T15:45:01.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.200+0000 7f6a302d6b80 1 Processor -- start 2024-02-02T15:45:01.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.202+0000 7f6a302d6b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562915d17090 0x562915d27c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:01.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.202+0000 7f6a302d6b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562915d179b0 0x562915d281b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:01.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.202+0000 7f6a19e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562915d17090 0x562915d27c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:01.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.203+0000 7f6a19e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562915d17090 0x562915d27c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:35514/0 (socket says 172.21.15.1:35514) 2024-02-02T15:45:01.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.203+0000 7f6a302d6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562915d22860 0x562915d83c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:01.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.203+0000 7f6a302d6b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x562915d28a90 con 0x562915d179b0 2024-02-02T15:45:01.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.203+0000 7f6a302d6b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x562915d84170 con 0x562915d17090 2024-02-02T15:45:01.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.203+0000 7f6a302d6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x562915d842b0 con 0x562915d22860 2024-02-02T15:45:01.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.203+0000 7f6a302d6b80 10 client.0 _ll_register_callbacks cb 0x562915c06310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:45:01.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.203+0000 7f6a19e00700 1 -- 172.21.15.1:0/3560246147 learned_addr learned my addr 172.21.15.1:0/3560246147 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:45:01.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.203+0000 7f6a19400700 1 --2- 172.21.15.1:0/3560246147 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562915d179b0 0x562915d281b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:01.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.203+0000 7f6a1a800700 1 --2- 172.21.15.1:0/3560246147 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562915d22860 0x562915d83c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:01.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.204+0000 7f6a19e00700 1 -- 172.21.15.1:0/3560246147 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562915d22860 msgr2=0x562915d83c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:45:01.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.204+0000 7f6a19e00700 1 --2- 172.21.15.1:0/3560246147 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562915d22860 0x562915d83c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:01.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.204+0000 7f6a19e00700 1 -- 172.21.15.1:0/3560246147 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562915d179b0 msgr2=0x562915d281b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:45:01.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.204+0000 7f6a19e00700 1 --2- 172.21.15.1:0/3560246147 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562915d179b0 0x562915d281b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:01.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.204+0000 7f6a19e00700 1 -- 172.21.15.1:0/3560246147 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a0c00b580 con 0x562915d17090 2024-02-02T15:45:01.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.204+0000 7f6a19400700 1 --2- 172.21.15.1:0/3560246147 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562915d179b0 0x562915d281b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:45:01.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.204+0000 7f6a19e00700 1 --2- 172.21.15.1:0/3560246147 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562915d17090 0x562915d27c70 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f6a0400b0a0 tx=0x7f6a040099e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e841a0adb834fb06 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:45:01.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.204+0000 7f6a03400700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:45:01.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.205+0000 7f6a03400700 1 -- 172.21.15.1:0/3560246147 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a0402f070 con 0x562915d17090 2024-02-02T15:45:01.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.205+0000 7f6a03400700 1 -- 172.21.15.1:0/3560246147 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(111..111 src has 1..111) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f6a04022040 con 0x562915d17090 2024-02-02T15:45:01.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.205+0000 7f6a302d6b80 1 -- 172.21.15.1:0/3560246147 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6a0c004cf0 con 0x562915d17090 2024-02-02T15:45:01.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.207+0000 7f6a03400700 1 -- 172.21.15.1:0/3560246147 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a0400e040 con 0x562915d17090 2024-02-02T15:45:01.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.207+0000 7f6a302d6b80 10 client.30290 fetch_fsmap learned FSMap version 77 2024-02-02T15:45:01.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.207+0000 7f6a302d6b80 1 -- 172.21.15.1:0/3560246147 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=4+}) v3 -- 0x562915d1f030 con 0x562915d17090 2024-02-02T15:45:01.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.208+0000 7f6a03400700 1 -- 172.21.15.1:0/3560246147 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6a0c004cf0 con 0x562915d17090 2024-02-02T15:45:01.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.208+0000 7f6a03400700 1 -- 172.21.15.1:0/3560246147 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 77) v1 ==== 7502+0+0 (secure 0 0 0) 0x7f6a0402f730 con 0x562915d17090 2024-02-02T15:45:01.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.208+0000 7f6a302d6b80 10 client.30290 fetch_fsmap finished waiting for FSMap version 77 2024-02-02T15:45:01.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.208+0000 7f6a302d6b80 10 client.30290 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:45:01.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.208+0000 7f6a302d6b80 20 client.30290 populate_metadata read hostname 'smithi001' 2024-02-02T15:45:01.208 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.208+0000 7f6a302d6b80 1 --2- 172.21.15.1:0/3560246147 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x562915d859c0 0x562915d87de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:01.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.208+0000 7f6a302d6b80 4 client.30290 mds_command: new command op to 24557 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-02T15:45:01.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.208+0000 7f6a302d6b80 1 -- 172.21.15.1:0/3560246147 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x562915c27fd0 con 0x562915d859c0 2024-02-02T15:45:01.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.209+0000 7f6a19400700 1 --2- 172.21.15.1:0/3560246147 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x562915d859c0 0x562915d87de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:01.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.209+0000 7f6a03400700 10 client.30290 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:45:01.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.211+0000 7f6a19400700 1 --2- 172.21.15.1:0/3560246147 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x562915d859c0 0x562915d87de0 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dff4bb7278ad3ccf server_cookie=3e2c55832893c596 in_seq=0 out_seq=0 2024-02-02T15:45:01.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.212+0000 7f6a03400700 1 -- 172.21.15.1:0/3560246147 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x562915c27fd0 con 0x562915d859c0 2024-02-02T15:45:01.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.212+0000 7f6a03400700 10 client.30290 handle_command_reply: tid=0 2024-02-02T15:45:01.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.212+0000 7f6a302d6b80 1 client.30290 shutdown 2024-02-02T15:45:01.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.213+0000 7f6a302d6b80 1 -- 172.21.15.1:0/3560246147 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562915d17090 msgr2=0x562915d27c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:01.213 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.213+0000 7f6a302d6b80 1 --2- 172.21.15.1:0/3560246147 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562915d17090 0x562915d27c70 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f6a0400b0a0 tx=0x7f6a040099e0 comp rx=0 tx=0).stop 2024-02-02T15:45:01.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.214+0000 7f6a302d6b80 1 -- 172.21.15.1:0/3560246147 shutdown_connections 2024-02-02T15:45:01.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.214+0000 7f6a302d6b80 1 --2- 172.21.15.1:0/3560246147 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x562915d859c0 0x562915d87de0 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:01.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.214+0000 7f6a302d6b80 1 --2- 172.21.15.1:0/3560246147 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x562915d22860 0x562915d83c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:01.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.214+0000 7f6a302d6b80 1 --2- 172.21.15.1:0/3560246147 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562915d17090 0x562915d27c70 secure :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f6a0400b0a0 tx=0x7f6a040099e0 comp rx=0 tx=0).stop 2024-02-02T15:45:01.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.214+0000 7f6a302d6b80 1 --2- 172.21.15.1:0/3560246147 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x562915d179b0 0x562915d281b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:01.214 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.214+0000 7f6a302d6b80 1 -- 172.21.15.1:0/3560246147 >> 172.21.15.1:0/3560246147 conn(0x562915c76e30 msgr2=0x562915d12ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:45:01.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.214+0000 7f6a03400700 0 client.30290 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:45:01.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.214+0000 7f6a302d6b80 1 -- 172.21.15.1:0/3560246147 shutdown_connections 2024-02-02T15:45:01.215 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.214+0000 7f6a302d6b80 1 -- 172.21.15.1:0/3560246147 wait complete. 2024-02-02T15:45:01.215 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:45:01.215 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2024-02-02T15:45:01.215 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "bc1a4ccb-be01-4218-96a2-eeadf3bd02cb", 2024-02-02T15:45:01.215 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2024-02-02T15:45:01.215 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:45:01.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.215+0000 7f6a302d6b80 20 client.30290 trim_cache size 0 max 16384 2024-02-02T15:45:01.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.216+0000 7f6a1c800700 1 -- 172.21.15.1:0/3256222712 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f69f40a0860 msgr2=0x7f69f40a2d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:01.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.216+0000 7f6a1c800700 1 --2- 172.21.15.1:0/3256222712 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f69f40a0860 0x7f69f40a2d20 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f6a0400b070 tx=0x7f6a0400b700 comp rx=0 tx=0).stop 2024-02-02T15:45:01.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.216+0000 7f6a1c800700 1 -- 172.21.15.1:0/3256222712 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a1410c720 msgr2=0x7f6a140815d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:01.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.216+0000 7f6a1c800700 1 --2- 172.21.15.1:0/3256222712 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a1410c720 0x7f6a140815d0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f69fc00e970 tx=0x7f69fc00ed30 comp rx=0 tx=0).stop 2024-02-02T15:45:01.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.216+0000 7f6a1c800700 1 -- 172.21.15.1:0/3256222712 shutdown_connections 2024-02-02T15:45:01.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.216+0000 7f6a1c800700 1 --2- 172.21.15.1:0/3256222712 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f69f40a0860 0x7f69f40a2d20 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:01.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.216+0000 7f6a1c800700 1 --2- 172.21.15.1:0/3256222712 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f6a1407a060 0x7f6a1407a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:01.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.216+0000 7f6a1c800700 1 --2- 172.21.15.1:0/3256222712 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f6a14107600 0x7f6a14081050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:01.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.216+0000 7f6a1c800700 1 --2- 172.21.15.1:0/3256222712 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6a1410c720 0x7f6a140815d0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:01.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.216+0000 7f6a1c800700 1 -- 172.21.15.1:0/3256222712 >> 172.21.15.1:0/3256222712 conn(0x7f6a14076b80 msgr2=0x7f6a14076f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:45:01.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.216+0000 7f6a1c800700 1 -- 172.21.15.1:0/3256222712 shutdown_connections 2024-02-02T15:45:01.217 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.216+0000 7f6a1c800700 1 -- 172.21.15.1:0/3256222712 wait complete. 2024-02-02T15:45:01.250 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-02T15:45:01.551 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:01 smithi001 ceph-mon[35470]: pgmap v565: 129 pgs: 129 active+clean; 2.0 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 5.7 MiB/s wr, 372 op/s 2024-02-02T15:45:01.551 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:01 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:45:01.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.589+0000 7fbd14a00700 1 -- 172.21.15.1:0/1180160648 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd0c10c720 msgr2=0x7fbd0c1142a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:01.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.589+0000 7fbd14a00700 1 --2- 172.21.15.1:0/1180160648 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd0c10c720 0x7fbd0c1142a0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fbd04009f90 tx=0x7fbd0400b330 comp rx=0 tx=0).stop 2024-02-02T15:45:01.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.590+0000 7fbd14a00700 1 -- 172.21.15.1:0/1180160648 shutdown_connections 2024-02-02T15:45:01.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.590+0000 7fbd14a00700 1 --2- 172.21.15.1:0/1180160648 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd0c10c720 0x7fbd0c1142a0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:01.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.590+0000 7fbd14a00700 1 --2- 172.21.15.1:0/1180160648 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbd0c107600 0x7fbd0c1079e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:01.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.590+0000 7fbd14a00700 1 --2- 172.21.15.1:0/1180160648 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd0c107fb0 0x7fbd0c10bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:01.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.590+0000 7fbd14a00700 1 -- 172.21.15.1:0/1180160648 >> 172.21.15.1:0/1180160648 conn(0x7fbd0c076b80 msgr2=0x7fbd0c105b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:45:01.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.591+0000 7fbd14a00700 1 -- 172.21.15.1:0/1180160648 shutdown_connections 2024-02-02T15:45:01.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.591+0000 7fbd14a00700 1 -- 172.21.15.1:0/1180160648 wait complete. 2024-02-02T15:45:01.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.592+0000 7fbd14a00700 1 Processor -- start 2024-02-02T15:45:01.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.593+0000 7fbd14a00700 1 -- start start 2024-02-02T15:45:01.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.593+0000 7fbd14a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd0c107600 0x7fbd0c19d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:01.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.593+0000 7fbd14a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbd0c10c720 0x7fbd0c19d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:01.593 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.593+0000 7fbd14a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd0c1a1d80 0x7fbd0c1a2200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:01.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.593+0000 7fbd14a00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd0c1a2800 con 0x7fbd0c10c720 2024-02-02T15:45:01.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.594+0000 7fbd14a00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd0c1a2970 con 0x7fbd0c1a1d80 2024-02-02T15:45:01.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.594+0000 7fbd14a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd0c1a2b30 con 0x7fbd0c107600 2024-02-02T15:45:01.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.594+0000 7fbd11600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbd0c10c720 0x7fbd0c19d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:01.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.595+0000 7fbd11600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbd0c10c720 0x7fbd0c19d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44314/0 (socket says 172.21.15.1:44314) 2024-02-02T15:45:01.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.595+0000 7fbd11600700 1 -- 172.21.15.1:0/2352492507 learned_addr learned my addr 172.21.15.1:0/2352492507 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:45:01.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.595+0000 7fbd12000700 1 --2- 172.21.15.1:0/2352492507 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd0c107600 0x7fbd0c19d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:01.595 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.595+0000 7fbd12a00700 1 --2- 172.21.15.1:0/2352492507 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd0c1a1d80 0x7fbd0c1a2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:01.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.596+0000 7fbd12000700 1 -- 172.21.15.1:0/2352492507 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd0c1a1d80 msgr2=0x7fbd0c1a2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:01.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.596+0000 7fbd12000700 1 --2- 172.21.15.1:0/2352492507 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd0c1a1d80 0x7fbd0c1a2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:01.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.596+0000 7fbd12000700 1 -- 172.21.15.1:0/2352492507 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbd0c10c720 msgr2=0x7fbd0c19d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:01.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.596+0000 7fbd12000700 1 --2- 172.21.15.1:0/2352492507 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbd0c10c720 0x7fbd0c19d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:01.596 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.596+0000 7fbd12000700 1 -- 172.21.15.1:0/2352492507 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd04009c40 con 0x7fbd0c107600 2024-02-02T15:45:01.598 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.599+0000 7fbd12000700 1 --2- 172.21.15.1:0/2352492507 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd0c107600 0x7fbd0c19d3e0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc00d950 tx=0x7fbcfc00dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:45:01.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.599+0000 7fbd0aa00700 1 -- 172.21.15.1:0/2352492507 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcfc00f890 con 0x7fbd0c107600 2024-02-02T15:45:01.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.599+0000 7fbd0aa00700 1 -- 172.21.15.1:0/2352492507 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbcfc010460 con 0x7fbd0c107600 2024-02-02T15:45:01.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.599+0000 7fbd0aa00700 1 -- 172.21.15.1:0/2352492507 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcfc00e610 con 0x7fbd0c107600 2024-02-02T15:45:01.599 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.599+0000 7fbd14a00700 1 -- 172.21.15.1:0/2352492507 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd0c1aa090 con 0x7fbd0c107600 2024-02-02T15:45:01.600 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.600+0000 7fbd14a00700 1 -- 172.21.15.1:0/2352492507 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd0c1aa520 con 0x7fbd0c107600 2024-02-02T15:45:01.608 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.608+0000 7fbd0aa00700 1 -- 172.21.15.1:0/2352492507 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7fbcfc0108d0 con 0x7fbd0c107600 2024-02-02T15:45:01.610 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.610+0000 7fbd0aa00700 1 --2- 172.21.15.1:0/2352492507 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fbcec0a0860 0x7fbcec0a2d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:01.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.611+0000 7fbd0aa00700 1 -- 172.21.15.1:0/2352492507 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 9541+0+0 (secure 0 0 0) 0x7fbcfc0cbd60 con 0x7fbd0c107600 2024-02-02T15:45:01.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.611+0000 7fbd28440b80 1 Processor -- start 2024-02-02T15:45:01.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.611+0000 7fbd11600700 1 --2- 172.21.15.1:0/2352492507 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fbcec0a0860 0x7fbcec0a2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:01.611 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.611+0000 7fbd28440b80 1 -- start start 2024-02-02T15:45:01.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.613+0000 7fbd28440b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5565d292d010 0x5565d292d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:01.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.613+0000 7fbd28440b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565d292d930 0x5565d29382a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:01.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.613+0000 7fbd28440b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5565d29387e0 0x5565d293abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:01.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.613+0000 7fbd28440b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565d28a14c0 con 0x5565d29387e0 2024-02-02T15:45:01.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.613+0000 7fbd28440b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565d2885aa0 con 0x5565d292d010 2024-02-02T15:45:01.613 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.613+0000 7fbd28440b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565d2883460 con 0x5565d292d930 2024-02-02T15:45:01.614 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.613+0000 7fbd12000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5565d292d010 0x5565d292d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:01.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.613+0000 7fbd11600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565d292d930 0x5565d29382a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:01.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.614+0000 7fbd12000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5565d292d010 0x5565d292d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:35542/0 (socket says 172.21.15.1:35542) 2024-02-02T15:45:01.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.614+0000 7fbd12000700 1 -- 172.21.15.1:0/4147293278 learned_addr learned my addr 172.21.15.1:0/4147293278 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:45:01.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.614+0000 7fbd11600700 1 --2- 172.21.15.1:0/2352492507 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fbcec0a0860 0x7fbcec0a2d20 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c19e9f0 tx=0x7fbcf4009240 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:45:01.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.615+0000 7fbd11600700 1 -- 172.21.15.1:0/4147293278 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5565d292d010 msgr2=0x5565d292d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:45:01.615 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.615+0000 7fbd11600700 1 --2- 172.21.15.1:0/4147293278 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5565d292d010 0x5565d292d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:01.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.615+0000 7fbd11600700 1 -- 172.21.15.1:0/4147293278 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5565d29387e0 msgr2=0x5565d293abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:45:01.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.615+0000 7fbd11600700 1 --2- 172.21.15.1:0/4147293278 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5565d29387e0 0x5565d293abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:01.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.615+0000 7fbd11600700 1 -- 172.21.15.1:0/4147293278 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5565d27003b0 con 0x5565d292d930 2024-02-02T15:45:01.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.615+0000 7fbd11600700 1 --2- 172.21.15.1:0/4147293278 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565d292d930 0x5565d29382a0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fbcf4009880 tx=0x7fbcf4011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8fa6e413ecf6ca6 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:45:01.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.616+0000 7fbd03000700 1 -- 172.21.15.1:0/4147293278 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf4015480 con 0x5565d292d930 2024-02-02T15:45:01.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.616+0000 7fbd03000700 1 -- 172.21.15.1:0/4147293278 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbcf4015aa0 con 0x5565d292d930 2024-02-02T15:45:01.616 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.616+0000 7fbd03000700 1 -- 172.21.15.1:0/4147293278 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf4013b20 con 0x5565d292d930 2024-02-02T15:45:01.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.620+0000 7fbd28440b80 1 -- 172.21.15.1:0/4147293278 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565d292d930 msgr2=0x5565d29382a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:01.620 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.620+0000 7fbd28440b80 1 --2- 172.21.15.1:0/4147293278 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565d292d930 0x5565d29382a0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fbcf4009880 tx=0x7fbcf4011000 comp rx=0 tx=0).stop 2024-02-02T15:45:01.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.621+0000 7fbd28440b80 1 -- 172.21.15.1:0/4147293278 shutdown_connections 2024-02-02T15:45:01.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.621+0000 7fbd28440b80 1 --2- 172.21.15.1:0/4147293278 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5565d29387e0 0x5565d293abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:01.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.621+0000 7fbd28440b80 1 --2- 172.21.15.1:0/4147293278 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5565d292d010 0x5565d292d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:01.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.621+0000 7fbd28440b80 1 --2- 172.21.15.1:0/4147293278 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565d292d930 0x5565d29382a0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:01.621 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.621+0000 7fbd28440b80 1 -- 172.21.15.1:0/4147293278 >> 172.21.15.1:0/4147293278 conn(0x5565d288cdb0 msgr2=0x5565d29293b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:45:01.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.623+0000 7fbd28440b80 1 -- 172.21.15.1:0/4147293278 shutdown_connections 2024-02-02T15:45:01.623 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.623+0000 7fbd28440b80 1 -- 172.21.15.1:0/4147293278 wait complete. 2024-02-02T15:45:01.626 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.627+0000 7fbd28440b80 1 -- start start 2024-02-02T15:45:01.629 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.630+0000 7fbd28440b80 1 Processor -- start 2024-02-02T15:45:01.633 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.634+0000 7fbd28440b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5565d292d010 0x5565d2935530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:01.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.634+0000 7fbd28440b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565d292d930 0x5565d2935a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:01.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.634+0000 7fbd28440b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5565d29387e0 0x5565d2935fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:01.634 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.635+0000 7fbd12000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5565d292d010 0x5565d2935530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:01.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.635+0000 7fbd12000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5565d292d010 0x5565d2935530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:35556/0 (socket says 172.21.15.1:35556) 2024-02-02T15:45:01.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.635+0000 7fbd11600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565d292d930 0x5565d2935a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:01.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.635+0000 7fbd12000700 1 -- 172.21.15.1:0/2633686138 learned_addr learned my addr 172.21.15.1:0/2633686138 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:45:01.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.635+0000 7fbd28440b80 1 -- 172.21.15.1:0/2633686138 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565d287ed50 con 0x5565d29387e0 2024-02-02T15:45:01.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.635+0000 7fbd28440b80 1 -- 172.21.15.1:0/2633686138 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565d281ad80 con 0x5565d292d010 2024-02-02T15:45:01.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.635+0000 7fbd28440b80 1 -- 172.21.15.1:0/2633686138 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565d281aa20 con 0x5565d292d930 2024-02-02T15:45:01.635 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.635+0000 7fbd28440b80 10 client.0 _ll_register_callbacks cb 0x5565d281c290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:45:01.636 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.636+0000 7fbd12a00700 1 --2- 172.21.15.1:0/2633686138 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5565d29387e0 0x5565d2935fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:01.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.637+0000 7fbd12a00700 1 -- 172.21.15.1:0/2633686138 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565d292d930 msgr2=0x5565d2935a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:45:01.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.637+0000 7fbd12a00700 1 --2- 172.21.15.1:0/2633686138 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565d292d930 0x5565d2935a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:01.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.637+0000 7fbd12a00700 1 -- 172.21.15.1:0/2633686138 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5565d292d010 msgr2=0x5565d2935530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:45:01.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.637+0000 7fbd12a00700 1 --2- 172.21.15.1:0/2633686138 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5565d292d010 0x5565d2935530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:01.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.637+0000 7fbd12a00700 1 -- 172.21.15.1:0/2633686138 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcf400e6c0 con 0x5565d29387e0 2024-02-02T15:45:01.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.637+0000 7fbd12a00700 1 --2- 172.21.15.1:0/2633686138 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5565d29387e0 0x5565d2935fb0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fbd04009790 tx=0x7fbd04004cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=862fdfdf496828a5 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:45:01.637 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.637+0000 7fbcfb400700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:45:01.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.638+0000 7fbcfb400700 1 -- 172.21.15.1:0/2633686138 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd0400e040 con 0x5565d29387e0 2024-02-02T15:45:01.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.638+0000 7fbcfb400700 1 -- 172.21.15.1:0/2633686138 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 9541+0+0 (secure 0 0 0) 0x7fbd04007830 con 0x5565d29387e0 2024-02-02T15:45:01.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.638+0000 7fbd28440b80 1 -- 172.21.15.1:0/2633686138 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5565d29210f0 con 0x5565d29387e0 2024-02-02T15:45:01.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.639+0000 7fbcfb400700 1 -- 172.21.15.1:0/2633686138 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbd0401c440 con 0x5565d29387e0 2024-02-02T15:45:01.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.639+0000 7fbd28440b80 10 client.19995 fetch_fsmap learned FSMap version 77 2024-02-02T15:45:01.639 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.639+0000 7fbd28440b80 1 -- 172.21.15.1:0/2633686138 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=4+}) v3 -- 0x5565d26acb60 con 0x5565d29387e0 2024-02-02T15:45:01.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.642+0000 7fbcfb400700 1 -- 172.21.15.1:0/2633686138 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5565d29210f0 con 0x5565d29387e0 2024-02-02T15:45:01.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.643+0000 7fbcfb400700 1 -- 172.21.15.1:0/2633686138 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 77) v1 ==== 7502+0+0 (secure 0 0 0) 0x7fbd04019070 con 0x5565d29387e0 2024-02-02T15:45:01.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.643+0000 7fbd28440b80 10 client.19995 fetch_fsmap finished waiting for FSMap version 77 2024-02-02T15:45:01.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.643+0000 7fbd28440b80 10 client.19995 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:45:01.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.643+0000 7fbd28440b80 20 client.19995 populate_metadata read hostname 'smithi001' 2024-02-02T15:45:01.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.643+0000 7fbd28440b80 1 --2- 172.21.15.1:0/2633686138 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5565d2948260 0x5565d294a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:01.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.643+0000 7fbd28440b80 4 client.19995 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:45:01.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.644+0000 7fbd28440b80 1 -- 172.21.15.1:0/2633686138 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5565d2877dd0 con 0x5565d2948260 2024-02-02T15:45:01.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.647+0000 7fbd12000700 1 --2- 172.21.15.1:0/2633686138 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5565d2948260 0x5565d294a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:01.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.647+0000 7fbd12000700 1 --2- 172.21.15.1:0/2633686138 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5565d2948260 0x5565d294a680 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e82b695153963cb server_cookie=bdbb7ac5c8079f0b in_seq=0 out_seq=0 2024-02-02T15:45:01.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.647+0000 7fbcfb400700 10 client.19995 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:45:01.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.648+0000 7fbcfb400700 1 -- 172.21.15.1:0/2633686138 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5565d2877dd0 con 0x5565d2948260 2024-02-02T15:45:01.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.648+0000 7fbcfb400700 10 client.19995 handle_command_reply: tid=0 2024-02-02T15:45:01.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.648+0000 7fbd28440b80 1 client.19995 shutdown 2024-02-02T15:45:01.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.649+0000 7fbd28440b80 1 -- 172.21.15.1:0/2633686138 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5565d29387e0 msgr2=0x5565d2935fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:01.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.649+0000 7fbd28440b80 1 --2- 172.21.15.1:0/2633686138 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5565d29387e0 0x5565d2935fb0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fbd04009790 tx=0x7fbd04004cd0 comp rx=0 tx=0).stop 2024-02-02T15:45:01.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.649+0000 7fbd28440b80 1 -- 172.21.15.1:0/2633686138 shutdown_connections 2024-02-02T15:45:01.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.649+0000 7fbd28440b80 1 --2- 172.21.15.1:0/2633686138 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5565d2948260 0x5565d294a680 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:01.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.649+0000 7fbd28440b80 1 --2- 172.21.15.1:0/2633686138 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5565d29387e0 0x5565d2935fb0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:01.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.649+0000 7fbd28440b80 1 --2- 172.21.15.1:0/2633686138 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5565d292d010 0x5565d2935530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:01.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.649+0000 7fbd28440b80 1 --2- 172.21.15.1:0/2633686138 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565d292d930 0x5565d2935a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:01.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.649+0000 7fbd28440b80 1 -- 172.21.15.1:0/2633686138 >> 172.21.15.1:0/2633686138 conn(0x5565d288cdb0 msgr2=0x5565d2928fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:45:01.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.649+0000 7fbcfb400700 0 client.19995 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:45:01.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.649+0000 7fbd28440b80 1 -- 172.21.15.1:0/2633686138 shutdown_connections 2024-02-02T15:45:01.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.649+0000 7fbd28440b80 1 -- 172.21.15.1:0/2633686138 wait complete. 2024-02-02T15:45:01.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.650+0000 7fbd28440b80 20 client.19995 trim_cache size 0 max 16384 2024-02-02T15:45:01.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.686+0000 7fbd28440b80 1 Processor -- start 2024-02-02T15:45:01.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.686+0000 7fbd28440b80 1 -- start start 2024-02-02T15:45:01.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.687+0000 7fbd28440b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5565d292d010 0x5565d2935de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:01.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.687+0000 7fbd28440b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5565d292d930 0x5565d29341a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:01.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.687+0000 7fbd28440b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565d29387e0 0x5565d29346e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:01.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.687+0000 7fbd28440b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565d27026a0 con 0x5565d292d010 2024-02-02T15:45:01.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.687+0000 7fbd28440b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565d292df70 con 0x5565d292d930 2024-02-02T15:45:01.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.687+0000 7fbd28440b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565d2936990 con 0x5565d29387e0 2024-02-02T15:45:01.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.687+0000 7fbd11600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5565d292d930 0x5565d29341a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:01.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.687+0000 7fbd11600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5565d292d930 0x5565d29341a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:35562/0 (socket says 172.21.15.1:35562) 2024-02-02T15:45:01.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.687+0000 7fbd11600700 1 -- 172.21.15.1:0/2857121368 learned_addr learned my addr 172.21.15.1:0/2857121368 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:45:01.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.688+0000 7fbd12a00700 1 --2- 172.21.15.1:0/2857121368 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565d29387e0 0x5565d29346e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:01.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.688+0000 7fbd12000700 1 --2- 172.21.15.1:0/2857121368 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5565d292d010 0x5565d2935de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:01.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.689+0000 7fbd11600700 1 -- 172.21.15.1:0/2857121368 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565d29387e0 msgr2=0x5565d29346e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:45:01.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.689+0000 7fbd11600700 1 --2- 172.21.15.1:0/2857121368 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565d29387e0 0x5565d29346e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:01.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.689+0000 7fbd11600700 1 -- 172.21.15.1:0/2857121368 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5565d292d010 msgr2=0x5565d2935de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:45:01.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.689+0000 7fbd11600700 1 --2- 172.21.15.1:0/2857121368 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5565d292d010 0x5565d2935de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:01.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.689+0000 7fbd11600700 1 -- 172.21.15.1:0/2857121368 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5565d2844ea0 con 0x5565d292d930 2024-02-02T15:45:01.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.690+0000 7fbd12000700 1 --2- 172.21.15.1:0/2857121368 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5565d292d010 0x5565d2935de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:45:01.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.691+0000 7fbd11600700 1 --2- 172.21.15.1:0/2857121368 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5565d292d930 0x5565d29341a0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fbcf4009880 tx=0x7fbcf4013660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75b19307e3935948 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:45:01.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.691+0000 7fbd03a00700 1 -- 172.21.15.1:0/2857121368 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf4015780 con 0x5565d292d930 2024-02-02T15:45:01.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.691+0000 7fbd03a00700 1 -- 172.21.15.1:0/2857121368 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbcf40158e0 con 0x5565d292d930 2024-02-02T15:45:01.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.691+0000 7fbd03a00700 1 -- 172.21.15.1:0/2857121368 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf4028a70 con 0x5565d292d930 2024-02-02T15:45:01.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.693+0000 7fbd28440b80 1 -- 172.21.15.1:0/2857121368 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5565d292d930 msgr2=0x5565d29341a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:01.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.693+0000 7fbd28440b80 1 --2- 172.21.15.1:0/2857121368 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5565d292d930 0x5565d29341a0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fbcf4009880 tx=0x7fbcf4013660 comp rx=0 tx=0).stop 2024-02-02T15:45:01.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.693+0000 7fbd28440b80 1 -- 172.21.15.1:0/2857121368 shutdown_connections 2024-02-02T15:45:01.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.693+0000 7fbd28440b80 1 --2- 172.21.15.1:0/2857121368 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565d29387e0 0x5565d29346e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:01.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.693+0000 7fbd28440b80 1 --2- 172.21.15.1:0/2857121368 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5565d292d010 0x5565d2935de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:01.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.693+0000 7fbd28440b80 1 --2- 172.21.15.1:0/2857121368 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5565d292d930 0x5565d29341a0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:01.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.693+0000 7fbd28440b80 1 -- 172.21.15.1:0/2857121368 >> 172.21.15.1:0/2857121368 conn(0x5565d288cdb0 msgr2=0x5565d2933a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:45:01.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.694+0000 7fbd28440b80 1 -- 172.21.15.1:0/2857121368 shutdown_connections 2024-02-02T15:45:01.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.694+0000 7fbd28440b80 1 -- 172.21.15.1:0/2857121368 wait complete. 2024-02-02T15:45:01.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.695+0000 7fbd28440b80 1 -- start start 2024-02-02T15:45:01.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.695+0000 7fbd28440b80 1 Processor -- start 2024-02-02T15:45:01.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.696+0000 7fbd28440b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5565d292d010 0x5565d293eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:01.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.696+0000 7fbd28440b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565d292d930 0x5565d293f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:01.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.696+0000 7fbd28440b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5565d29387e0 0x5565d293cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:01.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.696+0000 7fbd28440b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565d293d4f0 con 0x5565d292d010 2024-02-02T15:45:01.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.696+0000 7fbd28440b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565d293d660 con 0x5565d29387e0 2024-02-02T15:45:01.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.696+0000 7fbd28440b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565d293d820 con 0x5565d292d930 2024-02-02T15:45:01.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.696+0000 7fbd28440b80 10 client.0 _ll_register_callbacks cb 0x5565d281c290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:45:01.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.697+0000 7fbd12000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5565d292d010 0x5565d293eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:01.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.697+0000 7fbd11600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565d292d930 0x5565d293f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:01.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.697+0000 7fbd12000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5565d292d010 0x5565d293eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:44364/0 (socket says 172.21.15.1:44364) 2024-02-02T15:45:01.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.697+0000 7fbd12000700 1 -- 172.21.15.1:0/1087261431 learned_addr learned my addr 172.21.15.1:0/1087261431 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:45:01.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.698+0000 7fbd11600700 1 -- 172.21.15.1:0/1087261431 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5565d29387e0 msgr2=0x5565d293cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:45:01.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.698+0000 7fbd11600700 1 --2- 172.21.15.1:0/1087261431 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5565d29387e0 0x5565d293cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:01.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.698+0000 7fbd11600700 1 -- 172.21.15.1:0/1087261431 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5565d292d010 msgr2=0x5565d293eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:45:01.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.698+0000 7fbd11600700 1 --2- 172.21.15.1:0/1087261431 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5565d292d010 0x5565d293eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:01.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.698+0000 7fbd11600700 1 -- 172.21.15.1:0/1087261431 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5565d27003b0 con 0x5565d292d930 2024-02-02T15:45:01.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.698+0000 7fbd12000700 1 --2- 172.21.15.1:0/1087261431 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5565d292d010 0x5565d293eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:45:01.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.699+0000 7fbd11600700 1 --2- 172.21.15.1:0/1087261431 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565d292d930 0x5565d293f040 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x5565d2936900 tx=0x7fbcf4006e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9bf5f6d6f26bcaab server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:45:01.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.699+0000 7fbcfb400700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:45:01.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.700+0000 7fbcfb400700 1 -- 172.21.15.1:0/1087261431 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf4040070 con 0x5565d292d930 2024-02-02T15:45:01.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.700+0000 7fbcfb400700 1 -- 172.21.15.1:0/1087261431 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 9541+0+0 (secure 0 0 0) 0x7fbcf4013db0 con 0x5565d292d930 2024-02-02T15:45:01.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.700+0000 7fbd28440b80 1 -- 172.21.15.1:0/1087261431 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbcf4015aa0 con 0x5565d292d930 2024-02-02T15:45:01.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.701+0000 7fbcfb400700 1 -- 172.21.15.1:0/1087261431 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbcf4028610 con 0x5565d292d930 2024-02-02T15:45:01.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.701+0000 7fbd28440b80 10 client.30154 fetch_fsmap learned FSMap version 77 2024-02-02T15:45:01.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.701+0000 7fbd28440b80 1 -- 172.21.15.1:0/1087261431 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=4+}) v3 -- 0x5565d2704cd0 con 0x5565d292d930 2024-02-02T15:45:01.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.704+0000 7fbcfb400700 1 -- 172.21.15.1:0/1087261431 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbcf4015aa0 con 0x5565d292d930 2024-02-02T15:45:01.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.704+0000 7fbcfb400700 1 -- 172.21.15.1:0/1087261431 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 77) v1 ==== 7502+0+0 (secure 0 0 0) 0x7fbcf40323f0 con 0x5565d292d930 2024-02-02T15:45:01.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.704+0000 7fbd28440b80 10 client.30154 fetch_fsmap finished waiting for FSMap version 77 2024-02-02T15:45:01.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.704+0000 7fbd28440b80 10 client.30154 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:45:01.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.704+0000 7fbd28440b80 20 client.30154 populate_metadata read hostname 'smithi001' 2024-02-02T15:45:01.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.704+0000 7fbd28440b80 1 --2- 172.21.15.1:0/1087261431 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5565d29aba20 0x5565d29ade10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:01.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.704+0000 7fbd28440b80 4 client.30154 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2024-02-02T15:45:01.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.704+0000 7fbd28440b80 1 -- 172.21.15.1:0/1087261431 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5565d283df50 con 0x5565d29aba20 2024-02-02T15:45:01.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.706+0000 7fbd12000700 1 --2- 172.21.15.1:0/1087261431 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5565d29aba20 0x5565d29ade10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:01.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.707+0000 7fbd12000700 1 --2- 172.21.15.1:0/1087261431 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5565d29aba20 0x5565d29ade10 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a63bdc7eaf3f999 server_cookie=6bba0b9457b60d2c in_seq=0 out_seq=0 2024-02-02T15:45:01.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.707+0000 7fbcfb400700 10 client.30154 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:45:01.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.707+0000 7fbcfb400700 1 -- 172.21.15.1:0/1087261431 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5565d283df50 con 0x5565d29aba20 2024-02-02T15:45:01.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.707+0000 7fbcfb400700 10 client.30154 handle_command_reply: tid=0 2024-02-02T15:45:01.707 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.707+0000 7fbd28440b80 1 client.30154 shutdown 2024-02-02T15:45:01.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.708+0000 7fbd28440b80 1 -- 172.21.15.1:0/1087261431 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565d292d930 msgr2=0x5565d293f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:01.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.708+0000 7fbd28440b80 1 --2- 172.21.15.1:0/1087261431 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565d292d930 0x5565d293f040 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x5565d2936900 tx=0x7fbcf4006e20 comp rx=0 tx=0).stop 2024-02-02T15:45:01.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.708+0000 7fbd28440b80 1 -- 172.21.15.1:0/1087261431 shutdown_connections 2024-02-02T15:45:01.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.708+0000 7fbd28440b80 1 --2- 172.21.15.1:0/1087261431 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5565d29aba20 0x5565d29ade10 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:01.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.708+0000 7fbd28440b80 1 --2- 172.21.15.1:0/1087261431 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5565d29387e0 0x5565d293cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:01.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.708+0000 7fbd28440b80 1 --2- 172.21.15.1:0/1087261431 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5565d292d010 0x5565d293eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:01.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.708+0000 7fbd28440b80 1 --2- 172.21.15.1:0/1087261431 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5565d292d930 0x5565d293f040 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:01.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.708+0000 7fbd28440b80 1 -- 172.21.15.1:0/1087261431 >> 172.21.15.1:0/1087261431 conn(0x5565d288cdb0 msgr2=0x5565d2934320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:45:01.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.708+0000 7fbcfb400700 0 client.30154 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:45:01.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.709+0000 7fbd28440b80 1 -- 172.21.15.1:0/1087261431 shutdown_connections 2024-02-02T15:45:01.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.709+0000 7fbd28440b80 1 -- 172.21.15.1:0/1087261431 wait complete. 2024-02-02T15:45:01.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.710+0000 7fbd28440b80 20 client.30154 trim_cache size 0 max 16384 2024-02-02T15:45:01.710 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:45:01.710 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-02T15:45:01.710 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2024-02-02T15:45:01.710 INFO:teuthology.orchestra.run.smithi001.stdout: "bc1a4ccb-be01-4218-96a2-eeadf3bd02cb": { 2024-02-02T15:45:01.710 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2024-02-02T15:45:01.710 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "bc1a4ccb-be01-4218-96a2-eeadf3bd02cb", 2024-02-02T15:45:01.710 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,force" 2024-02-02T15:45:01.710 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:45:01.710 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:45:01.710 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:45:01.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.711+0000 7fbd14a00700 1 -- 172.21.15.1:0/2352492507 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fbcec0a0860 msgr2=0x7fbcec0a2d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:01.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.711+0000 7fbd14a00700 1 --2- 172.21.15.1:0/2352492507 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fbcec0a0860 0x7fbcec0a2d20 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c19e9f0 tx=0x7fbcf4009240 comp rx=0 tx=0).stop 2024-02-02T15:45:01.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.711+0000 7fbd14a00700 1 -- 172.21.15.1:0/2352492507 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd0c107600 msgr2=0x7fbd0c19d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:01.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.711+0000 7fbd14a00700 1 --2- 172.21.15.1:0/2352492507 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd0c107600 0x7fbd0c19d3e0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc00d950 tx=0x7fbcfc00dc60 comp rx=0 tx=0).stop 2024-02-02T15:45:01.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.711+0000 7fbd14a00700 1 -- 172.21.15.1:0/2352492507 shutdown_connections 2024-02-02T15:45:01.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.711+0000 7fbd14a00700 1 --2- 172.21.15.1:0/2352492507 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fbcec0a0860 0x7fbcec0a2d20 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:01.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.711+0000 7fbd14a00700 1 --2- 172.21.15.1:0/2352492507 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbd0c1a1d80 0x7fbd0c1a2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:01.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.711+0000 7fbd14a00700 1 --2- 172.21.15.1:0/2352492507 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbd0c107600 0x7fbd0c19d3e0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:01.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.711+0000 7fbd14a00700 1 --2- 172.21.15.1:0/2352492507 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbd0c10c720 0x7fbd0c19d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:01.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.711+0000 7fbd14a00700 1 -- 172.21.15.1:0/2352492507 >> 172.21.15.1:0/2352492507 conn(0x7fbd0c076b80 msgr2=0x7fbd0c077f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:45:01.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.711+0000 7fbd14a00700 1 -- 172.21.15.1:0/2352492507 shutdown_connections 2024-02-02T15:45:01.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:01.711+0000 7fbd14a00700 1 -- 172.21.15.1:0/2352492507 wait complete. 2024-02-02T15:45:01.749 INFO:tasks.cephfs.filesystem:scrub status for tag:bc1a4ccb-be01-4218-96a2-eeadf3bd02cb - {'path': '/', 'tag': 'bc1a4ccb-be01-4218-96a2-eeadf3bd02cb', 'options': 'recursive,force'} 2024-02-02T15:45:01.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:01 smithi071 ceph-mon[44707]: pgmap v565: 129 pgs: 129 active+clean; 2.0 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 5.7 MiB/s wr, 372 op/s 2024-02-02T15:45:01.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:01 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:45:01.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:01 smithi142 ceph-mon[45728]: pgmap v565: 129 pgs: 129 active+clean; 2.0 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 5.7 MiB/s wr, 372 op/s 2024-02-02T15:45:01.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:01 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:45:01.999 INFO:tasks.workunit.client.0.smithi001.stdout: 10 150102 28.05 MB/sec execute 154 sec latency 455.838 ms 2024-02-02T15:45:02.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:02 smithi071 ceph-mon[44707]: scrub queued for path: / 2024-02-02T15:45:02.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:02 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:45:02.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:02 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:45:02.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:02 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:45:02.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:02 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:45:02.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:02 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:45:02.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:02 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:45:02.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:02 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:45:02.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:02 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:45:02.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:02 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:45:02.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:02 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:45:02.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:02 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:45:02.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:02 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:45:02.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:02 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:45:02.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:02 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:45:02.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:02 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:45:02.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:02 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:45:02.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:02 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:45:02.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:02 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:45:02.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:02 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:45:02.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:02 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:45:02.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:02 smithi071 ceph-mon[44707]: osdmap e112: 12 total, 12 up, 12 in 2024-02-02T15:45:02.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:02 smithi142 ceph-mon[45728]: scrub queued for path: / 2024-02-02T15:45:02.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:02 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:45:02.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:02 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:45:02.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:02 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:45:02.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:02 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:45:02.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:02 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:45:02.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:02 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:45:02.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:02 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:45:02.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:02 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:45:02.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:02 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:45:02.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:02 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:45:02.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:02 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:45:02.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:02 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:45:02.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:02 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:45:02.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:02 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:45:02.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:02 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:45:02.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:02 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:45:02.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:02 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:45:02.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:02 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:45:02.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:02 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:45:02.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:02 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:45:02.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:02 smithi142 ceph-mon[45728]: osdmap e112: 12 total, 12 up, 12 in 2024-02-02T15:45:02.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:02 smithi001 ceph-mon[35470]: scrub queued for path: / 2024-02-02T15:45:02.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:02 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:45:02.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:02 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:45:02.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:02 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:45:02.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:02 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:45:02.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:02 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:45:02.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:02 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:45:02.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:02 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:45:02.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:02 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:45:02.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:02 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:45:02.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:02 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:45:02.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:02 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:45:02.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:02 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:45:02.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:02 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:45:02.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:02 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:45:02.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:02 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:45:02.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:02 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:45:02.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:02 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:45:02.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:02 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:45:02.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:02 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:45:02.942 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:02 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:45:02.942 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:02 smithi001 ceph-mon[35470]: osdmap e112: 12 total, 12 up, 12 in 2024-02-02T15:45:03.000 INFO:tasks.workunit.client.0.smithi001.stdout: 10 150131 27.89 MB/sec execute 155 sec latency 458.140 ms 2024-02-02T15:45:03.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:03 smithi071 ceph-mon[44707]: pgmap v567: 129 pgs: 129 active+clean; 2.0 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 6.4 MiB/s wr, 422 op/s 2024-02-02T15:45:03.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:03 smithi142 ceph-mon[45728]: pgmap v567: 129 pgs: 129 active+clean; 2.0 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 6.4 MiB/s wr, 422 op/s 2024-02-02T15:45:03.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:03 smithi001 ceph-mon[35470]: pgmap v567: 129 pgs: 129 active+clean; 2.0 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 6.4 MiB/s wr, 422 op/s 2024-02-02T15:45:04.000 INFO:tasks.workunit.client.0.smithi001.stdout: 10 150169 27.75 MB/sec execute 156 sec latency 300.365 ms 2024-02-02T15:45:05.001 INFO:tasks.workunit.client.0.smithi001.stdout: 10 150269 27.61 MB/sec execute 157 sec latency 355.842 ms 2024-02-02T15:45:05.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:05 smithi071 ceph-mon[44707]: pgmap v568: 129 pgs: 129 active+clean; 2.0 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.2 MiB/s wr, 482 op/s 2024-02-02T15:45:05.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:05 smithi142 ceph-mon[45728]: pgmap v568: 129 pgs: 129 active+clean; 2.0 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.2 MiB/s wr, 482 op/s 2024-02-02T15:45:05.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:05 smithi001 ceph-mon[35470]: pgmap v568: 129 pgs: 129 active+clean; 2.0 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.2 MiB/s wr, 482 op/s 2024-02-02T15:45:06.000 INFO:tasks.workunit.client.0.smithi001.stdout: 10 150718 27.62 MB/sec execute 158 sec latency 244.554 ms 2024-02-02T15:45:07.001 INFO:tasks.workunit.client.0.smithi001.stdout: 10 151322 27.70 MB/sec execute 159 sec latency 136.109 ms 2024-02-02T15:45:07.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:07 smithi071 ceph-mon[44707]: pgmap v569: 129 pgs: 129 active+clean; 2.0 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 6.2 MiB/s wr, 535 op/s 2024-02-02T15:45:07.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:07 smithi071 ceph-mon[44707]: osdmap e113: 12 total, 12 up, 12 in 2024-02-02T15:45:07.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:07 smithi142 ceph-mon[45728]: pgmap v569: 129 pgs: 129 active+clean; 2.0 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 6.2 MiB/s wr, 535 op/s 2024-02-02T15:45:07.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:07 smithi142 ceph-mon[45728]: osdmap e113: 12 total, 12 up, 12 in 2024-02-02T15:45:07.944 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:07 smithi001 ceph-mon[35470]: pgmap v569: 129 pgs: 129 active+clean; 2.0 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 6.2 MiB/s wr, 535 op/s 2024-02-02T15:45:07.945 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:07 smithi001 ceph-mon[35470]: osdmap e113: 12 total, 12 up, 12 in 2024-02-02T15:45:08.001 INFO:tasks.workunit.client.0.smithi001.stdout: 10 151894 27.71 MB/sec execute 160 sec latency 75.438 ms 2024-02-02T15:45:09.001 INFO:tasks.workunit.client.0.smithi001.stdout: 10 152018 27.57 MB/sec execute 161 sec latency 789.909 ms 2024-02-02T15:45:09.867 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:09 smithi071 ceph-mon[44707]: pgmap v571: 129 pgs: 129 active+clean; 2.0 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 7.3 MiB/s wr, 682 op/s 2024-02-02T15:45:09.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:09 smithi142 ceph-mon[45728]: pgmap v571: 129 pgs: 129 active+clean; 2.0 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 7.3 MiB/s wr, 682 op/s 2024-02-02T15:45:09.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:09 smithi001 ceph-mon[35470]: pgmap v571: 129 pgs: 129 active+clean; 2.0 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 7.3 MiB/s wr, 682 op/s 2024-02-02T15:45:10.001 INFO:tasks.workunit.client.0.smithi001.stdout: 10 152181 27.45 MB/sec execute 162 sec latency 862.381 ms 2024-02-02T15:45:11.001 INFO:tasks.workunit.client.0.smithi001.stdout: 10 152496 27.34 MB/sec execute 163 sec latency 170.843 ms 2024-02-02T15:45:11.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:11 smithi071 ceph-mon[44707]: pgmap v572: 129 pgs: 129 active+clean; 2.0 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.7 MiB/s wr, 485 op/s 2024-02-02T15:45:11.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:11 smithi142 ceph-mon[45728]: pgmap v572: 129 pgs: 129 active+clean; 2.0 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.7 MiB/s wr, 485 op/s 2024-02-02T15:45:11.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:11 smithi001 ceph-mon[35470]: pgmap v572: 129 pgs: 129 active+clean; 2.0 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.7 MiB/s wr, 485 op/s 2024-02-02T15:45:12.001 INFO:tasks.workunit.client.0.smithi001.stdout: 10 153031 27.41 MB/sec execute 164 sec latency 84.431 ms 2024-02-02T15:45:12.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:12 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:45:12.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:12 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:45:12.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:12 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:45:12.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:12 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:45:12.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:12 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:45:12.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:12 smithi071 ceph-mon[44707]: pgmap v573: 129 pgs: 129 active+clean; 2.0 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 10 MiB/s wr, 657 op/s 2024-02-02T15:45:12.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:12 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:45:12.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:12 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:45:12.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:12 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:45:12.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:12 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:45:12.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:12 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:45:12.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:12 smithi142 ceph-mon[45728]: pgmap v573: 129 pgs: 129 active+clean; 2.0 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 10 MiB/s wr, 657 op/s 2024-02-02T15:45:12.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:12 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:45:12.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:12 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:45:12.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:12 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:45:12.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:12 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:45:12.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:12 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:45:12.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:12 smithi001 ceph-mon[35470]: pgmap v573: 129 pgs: 129 active+clean; 2.0 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 10 MiB/s wr, 657 op/s 2024-02-02T15:45:13.001 INFO:tasks.workunit.client.0.smithi001.stdout: 10 153738 27.44 MB/sec execute 165 sec latency 66.444 ms 2024-02-02T15:45:14.003 INFO:tasks.workunit.client.0.smithi001.stdout: 10 154143 27.40 MB/sec execute 166 sec latency 283.273 ms 2024-02-02T15:45:15.002 INFO:tasks.workunit.client.0.smithi001.stdout: 10 154302 27.30 MB/sec execute 167 sec latency 953.678 ms 2024-02-02T15:45:15.869 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:15 smithi071 ceph-mon[44707]: pgmap v574: 129 pgs: 129 active+clean; 2.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 12 MiB/s wr, 658 op/s 2024-02-02T15:45:15.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:15 smithi142 ceph-mon[45728]: pgmap v574: 129 pgs: 129 active+clean; 2.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 12 MiB/s wr, 658 op/s 2024-02-02T15:45:15.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:15 smithi001 ceph-mon[35470]: pgmap v574: 129 pgs: 129 active+clean; 2.1 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 12 MiB/s wr, 658 op/s 2024-02-02T15:45:16.002 INFO:tasks.workunit.client.0.smithi001.stdout: 10 154558 27.23 MB/sec execute 168 sec latency 165.873 ms 2024-02-02T15:45:16.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:16 smithi071 ceph-mon[44707]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:45:16.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:16 smithi142 ceph-mon[45728]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:45:16.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:16 smithi001 ceph-mon[35470]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:45:17.002 INFO:tasks.workunit.client.0.smithi001.stdout: 10 154863 27.19 MB/sec execute 169 sec latency 177.397 ms 2024-02-02T15:45:17.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:17 smithi071 ceph-mon[44707]: pgmap v575: 129 pgs: 129 active+clean; 2.1 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 14 MiB/s wr, 619 op/s 2024-02-02T15:45:17.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:17 smithi142 ceph-mon[45728]: pgmap v575: 129 pgs: 129 active+clean; 2.1 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 14 MiB/s wr, 619 op/s 2024-02-02T15:45:17.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:17 smithi001 ceph-mon[35470]: pgmap v575: 129 pgs: 129 active+clean; 2.1 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 14 MiB/s wr, 619 op/s 2024-02-02T15:45:18.002 INFO:tasks.workunit.client.0.smithi001.stdout: 10 155464 27.18 MB/sec execute 170 sec latency 71.415 ms 2024-02-02T15:45:19.002 INFO:tasks.workunit.client.0.smithi001.stdout: 10 156077 27.22 MB/sec execute 171 sec latency 72.552 ms 2024-02-02T15:45:19.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:19 smithi071 ceph-mon[44707]: pgmap v576: 129 pgs: 129 active+clean; 2.1 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 16 MiB/s wr, 670 op/s 2024-02-02T15:45:19.925 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:19 smithi142 ceph-mon[45728]: pgmap v576: 129 pgs: 129 active+clean; 2.1 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 16 MiB/s wr, 670 op/s 2024-02-02T15:45:19.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:19 smithi001 ceph-mon[35470]: pgmap v576: 129 pgs: 129 active+clean; 2.1 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 16 MiB/s wr, 670 op/s 2024-02-02T15:45:20.002 INFO:tasks.workunit.client.0.smithi001.stdout: 10 156614 27.26 MB/sec execute 172 sec latency 170.349 ms 2024-02-02T15:45:21.003 INFO:tasks.workunit.client.0.smithi001.stdout: 10 157233 27.24 MB/sec execute 173 sec latency 81.015 ms 2024-02-02T15:45:21.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:21 smithi071 ceph-mon[44707]: pgmap v577: 129 pgs: 129 active+clean; 2.1 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 15 MiB/s wr, 595 op/s 2024-02-02T15:45:21.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:21 smithi142 ceph-mon[45728]: pgmap v577: 129 pgs: 129 active+clean; 2.1 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 15 MiB/s wr, 595 op/s 2024-02-02T15:45:21.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:21 smithi001 ceph-mon[35470]: pgmap v577: 129 pgs: 129 active+clean; 2.1 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 15 MiB/s wr, 595 op/s 2024-02-02T15:45:22.003 INFO:tasks.workunit.client.0.smithi001.stdout: 10 157768 27.30 MB/sec execute 174 sec latency 117.579 ms 2024-02-02T15:45:22.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:22 smithi142 ceph-mon[45728]: pgmap v578: 129 pgs: 129 active+clean; 2.1 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 18 MiB/s wr, 733 op/s 2024-02-02T15:45:22.939 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:22 smithi001 ceph-mon[35470]: pgmap v578: 129 pgs: 129 active+clean; 2.1 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 18 MiB/s wr, 733 op/s 2024-02-02T15:45:23.003 INFO:tasks.workunit.client.0.smithi001.stdout: 10 158260 27.30 MB/sec execute 175 sec latency 85.733 ms 2024-02-02T15:45:23.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:22 smithi071 ceph-mon[44707]: pgmap v578: 129 pgs: 129 active+clean; 2.1 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 18 MiB/s wr, 733 op/s 2024-02-02T15:45:24.003 INFO:tasks.workunit.client.0.smithi001.stdout: 10 158665 27.24 MB/sec execute 176 sec latency 101.247 ms 2024-02-02T15:45:25.003 INFO:tasks.workunit.client.0.smithi001.stdout: 10 159273 27.24 MB/sec execute 177 sec latency 72.376 ms 2024-02-02T15:45:25.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:25 smithi071 ceph-mon[44707]: pgmap v579: 129 pgs: 129 active+clean; 2.1 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 14 MiB/s wr, 596 op/s 2024-02-02T15:45:25.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:25 smithi142 ceph-mon[45728]: pgmap v579: 129 pgs: 129 active+clean; 2.1 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 14 MiB/s wr, 596 op/s 2024-02-02T15:45:25.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:25 smithi001 ceph-mon[35470]: pgmap v579: 129 pgs: 129 active+clean; 2.1 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 14 MiB/s wr, 596 op/s 2024-02-02T15:45:26.004 INFO:tasks.workunit.client.0.smithi001.stdout: 10 159890 27.30 MB/sec execute 178 sec latency 75.835 ms 2024-02-02T15:45:27.003 INFO:tasks.workunit.client.0.smithi001.stdout: 10 160566 27.35 MB/sec execute 179 sec latency 86.429 ms 2024-02-02T15:45:27.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:27 smithi071 ceph-mon[44707]: pgmap v580: 129 pgs: 129 active+clean; 2.1 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 604 op/s 2024-02-02T15:45:27.922 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:27 smithi142 ceph-mon[45728]: pgmap v580: 129 pgs: 129 active+clean; 2.1 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 604 op/s 2024-02-02T15:45:27.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:27 smithi001 ceph-mon[35470]: pgmap v580: 129 pgs: 129 active+clean; 2.1 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 604 op/s 2024-02-02T15:45:28.003 INFO:tasks.workunit.client.0.smithi001.stdout: 10 161170 27.39 MB/sec execute 180 sec latency 100.107 ms 2024-02-02T15:45:29.004 INFO:tasks.workunit.client.0.smithi001.stdout: 10 161756 27.44 MB/sec execute 181 sec latency 73.630 ms 2024-02-02T15:45:29.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:29 smithi071 ceph-mon[44707]: pgmap v581: 129 pgs: 129 active+clean; 2.1 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 542 op/s 2024-02-02T15:45:29.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:29 smithi142 ceph-mon[45728]: pgmap v581: 129 pgs: 129 active+clean; 2.1 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 542 op/s 2024-02-02T15:45:29.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:29 smithi001 ceph-mon[35470]: pgmap v581: 129 pgs: 129 active+clean; 2.1 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 542 op/s 2024-02-02T15:45:30.004 INFO:tasks.workunit.client.0.smithi001.stdout: 10 162251 27.39 MB/sec execute 182 sec latency 111.681 ms 2024-02-02T15:45:31.004 INFO:tasks.workunit.client.0.smithi001.stdout: 10 162844 27.40 MB/sec execute 183 sec latency 68.860 ms 2024-02-02T15:45:31.750 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-02T15:45:31.854 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:31 smithi001 ceph-mon[35470]: pgmap v582: 129 pgs: 129 active+clean; 2.1 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 7.2 MiB/s wr, 399 op/s 2024-02-02T15:45:31.863 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:31 smithi071 ceph-mon[44707]: pgmap v582: 129 pgs: 129 active+clean; 2.1 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 7.2 MiB/s wr, 399 op/s 2024-02-02T15:45:31.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:31 smithi142 ceph-mon[45728]: pgmap v582: 129 pgs: 129 active+clean; 2.1 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 7.2 MiB/s wr, 399 op/s 2024-02-02T15:45:32.004 INFO:tasks.workunit.client.0.smithi001.stdout: 10 163465 27.46 MB/sec execute 184 sec latency 74.323 ms 2024-02-02T15:45:32.127 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.127+0000 7f00e5e00700 1 -- 172.21.15.1:0/921937399 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f00e00ff860 msgr2=0x7f00e00ffc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:32.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.127+0000 7f00e5e00700 1 --2- 172.21.15.1:0/921937399 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f00e00ff860 0x7f00e00ffc40 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f00d0009990 tx=0x7f00d0009ca0 comp rx=0 tx=0).stop 2024-02-02T15:45:32.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.127+0000 7f00e5e00700 1 -- 172.21.15.1:0/921937399 shutdown_connections 2024-02-02T15:45:32.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.127+0000 7f00e5e00700 1 --2- 172.21.15.1:0/921937399 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f00e0106130 0x7f00e010dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:32.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.127+0000 7f00e5e00700 1 --2- 172.21.15.1:0/921937399 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f00e00ff860 0x7f00e00ffc40 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:32.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.127+0000 7f00e5e00700 1 --2- 172.21.15.1:0/921937399 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f00e0100210 0x7f00e0105a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:32.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.127+0000 7f00e5e00700 1 -- 172.21.15.1:0/921937399 >> 172.21.15.1:0/921937399 conn(0x7f00e0074f30 msgr2=0x7f00e0075340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:45:32.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.128+0000 7f00e5e00700 1 -- 172.21.15.1:0/921937399 shutdown_connections 2024-02-02T15:45:32.128 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.129+0000 7f00e5e00700 1 -- 172.21.15.1:0/921937399 wait complete. 2024-02-02T15:45:32.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.130+0000 7f00e5e00700 1 Processor -- start 2024-02-02T15:45:32.130 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.130+0000 7f00e5e00700 1 -- start start 2024-02-02T15:45:32.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.130+0000 7f00e5e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f00e00ff860 0x7f00e0107cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:32.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.130+0000 7f00e5e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f00e0106130 0x7f00e0108230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:32.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.130+0000 7f00e5e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f00e0108770 0x7f00e01a9730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:32.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.130+0000 7f00e5e00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00e01a9c70 con 0x7f00e0106130 2024-02-02T15:45:32.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.130+0000 7f00e5e00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00e0198ed0 con 0x7f00e0108770 2024-02-02T15:45:32.131 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.130+0000 7f00e5e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00e0199040 con 0x7f00e00ff860 2024-02-02T15:45:32.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.131+0000 7f00dea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f00e00ff860 0x7f00e0107cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:32.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.131+0000 7f00de000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f00e0106130 0x7f00e0108230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:32.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.131+0000 7f00dea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f00e00ff860 0x7f00e0107cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:57414/0 (socket says 172.21.15.1:57414) 2024-02-02T15:45:32.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.131+0000 7f00dea00700 1 -- 172.21.15.1:0/1892822103 learned_addr learned my addr 172.21.15.1:0/1892822103 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:45:32.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.132+0000 7f00dea00700 1 -- 172.21.15.1:0/1892822103 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f00e0108770 msgr2=0x7f00e01a9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:32.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.132+0000 7f00dea00700 1 --2- 172.21.15.1:0/1892822103 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f00e0108770 0x7f00e01a9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:32.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.132+0000 7f00dea00700 1 -- 172.21.15.1:0/1892822103 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f00e0106130 msgr2=0x7f00e0108230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:32.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.132+0000 7f00dea00700 1 --2- 172.21.15.1:0/1892822103 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f00e0106130 0x7f00e0108230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:32.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.132+0000 7f00dea00700 1 -- 172.21.15.1:0/1892822103 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00c8009520 con 0x7f00e00ff860 2024-02-02T15:45:32.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.132+0000 7f00dea00700 1 --2- 172.21.15.1:0/1892822103 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f00e00ff860 0x7f00e0107cf0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f00d0005910 tx=0x7f00d0009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:45:32.134 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.132+0000 7f00e4a00700 1 -- 172.21.15.1:0/1892822103 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00d001d070 con 0x7f00e00ff860 2024-02-02T15:45:32.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.132+0000 7f00e4a00700 1 -- 172.21.15.1:0/1892822103 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f00d0021410 con 0x7f00e00ff860 2024-02-02T15:45:32.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.132+0000 7f00e5e00700 1 -- 172.21.15.1:0/1892822103 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00d00095f0 con 0x7f00e00ff860 2024-02-02T15:45:32.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.132+0000 7f00e4a00700 1 -- 172.21.15.1:0/1892822103 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00d0017650 con 0x7f00e00ff860 2024-02-02T15:45:32.135 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.133+0000 7f00e5e00700 1 -- 172.21.15.1:0/1892822103 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00e0199550 con 0x7f00e00ff860 2024-02-02T15:45:32.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.138+0000 7f00e4a00700 1 -- 172.21.15.1:0/1892822103 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f00d00177b0 con 0x7f00e00ff860 2024-02-02T15:45:32.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.140+0000 7f00f99e3b80 1 Processor -- start 2024-02-02T15:45:32.140 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.140+0000 7f00f99e3b80 1 -- start start 2024-02-02T15:45:32.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.141+0000 7f00f99e3b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55899bfa3010 0x55899bfa33f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:32.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.141+0000 7f00f99e3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55899bfa3930 0x55899bfae2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:32.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.141+0000 7f00f99e3b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55899bfae7e0 0x55899bfb0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:32.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.141+0000 7f00f99e3b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55899bf174c0 con 0x55899bfae7e0 2024-02-02T15:45:32.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.141+0000 7f00f99e3b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55899befbaa0 con 0x55899bfa3010 2024-02-02T15:45:32.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.141+0000 7f00f99e3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55899bef9460 con 0x55899bfa3930 2024-02-02T15:45:32.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.141+0000 7f00df400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55899bfa3010 0x55899bfa33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:32.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.142+0000 7f00df400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55899bfa3010 0x55899bfa33f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:47782/0 (socket says 172.21.15.1:47782) 2024-02-02T15:45:32.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.142+0000 7f00df400700 1 -- 172.21.15.1:0/3761981271 learned_addr learned my addr 172.21.15.1:0/3761981271 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:45:32.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.142+0000 7f00de000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55899bfae7e0 0x55899bfb0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:32.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.142+0000 7f00dea00700 1 --2- 172.21.15.1:0/3761981271 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55899bfa3930 0x55899bfae2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:32.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.143+0000 7f00de000700 1 -- 172.21.15.1:0/3761981271 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55899bfa3930 msgr2=0x55899bfae2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:45:32.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.143+0000 7f00de000700 1 --2- 172.21.15.1:0/3761981271 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55899bfa3930 0x55899bfae2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:32.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.143+0000 7f00de000700 1 -- 172.21.15.1:0/3761981271 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55899bfa3010 msgr2=0x55899bfa33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:45:32.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.143+0000 7f00de000700 1 --2- 172.21.15.1:0/3761981271 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55899bfa3010 0x55899bfa33f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:32.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.143+0000 7f00de000700 1 -- 172.21.15.1:0/3761981271 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55899bd763b0 con 0x55899bfae7e0 2024-02-02T15:45:32.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.143+0000 7f00de000700 1 --2- 172.21.15.1:0/3761981271 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55899bfae7e0 0x55899bfb0bd0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f00e0107450 tx=0x7f00c8011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93d1dda71f1bbca server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:45:32.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.144+0000 7f00e4a00700 1 --2- 172.21.15.1:0/1892822103 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f00c00b16e0 0x7f00c00b3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:32.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.144+0000 7f00cfe00700 1 -- 172.21.15.1:0/3761981271 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c8019630 con 0x55899bfae7e0 2024-02-02T15:45:32.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.144+0000 7f00cfe00700 1 -- 172.21.15.1:0/3761981271 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f00c8019c50 con 0x55899bfae7e0 2024-02-02T15:45:32.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.144+0000 7f00cfe00700 1 -- 172.21.15.1:0/3761981271 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c800ec50 con 0x55899bfae7e0 2024-02-02T15:45:32.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.144+0000 7f00e4a00700 1 -- 172.21.15.1:0/1892822103 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f00d00cd3e0 con 0x7f00e00ff860 2024-02-02T15:45:32.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.145+0000 7f00f99e3b80 1 -- 172.21.15.1:0/3761981271 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55899bfae7e0 msgr2=0x55899bfb0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:32.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.145+0000 7f00f99e3b80 1 --2- 172.21.15.1:0/3761981271 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55899bfae7e0 0x55899bfb0bd0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f00e0107450 tx=0x7f00c8011720 comp rx=0 tx=0).stop 2024-02-02T15:45:32.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.146+0000 7f00df400700 1 --2- 172.21.15.1:0/1892822103 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f00c00b16e0 0x7f00c00b3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:32.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.146+0000 7f00f99e3b80 1 -- 172.21.15.1:0/3761981271 shutdown_connections 2024-02-02T15:45:32.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.146+0000 7f00f99e3b80 1 --2- 172.21.15.1:0/3761981271 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55899bfae7e0 0x55899bfb0bd0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:32.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.146+0000 7f00f99e3b80 1 --2- 172.21.15.1:0/3761981271 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55899bfa3010 0x55899bfa33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:32.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.146+0000 7f00f99e3b80 1 --2- 172.21.15.1:0/3761981271 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55899bfa3930 0x55899bfae2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:32.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.146+0000 7f00f99e3b80 1 -- 172.21.15.1:0/3761981271 >> 172.21.15.1:0/3761981271 conn(0x55899bf02db0 msgr2=0x55899bf9f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:45:32.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.146+0000 7f00f99e3b80 1 -- 172.21.15.1:0/3761981271 shutdown_connections 2024-02-02T15:45:32.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.146+0000 7f00f99e3b80 1 -- 172.21.15.1:0/3761981271 wait complete. 2024-02-02T15:45:32.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.147+0000 7f00df400700 1 --2- 172.21.15.1:0/1892822103 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f00c00b16e0 0x7f00c00b3ba0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f00d8009c10 tx=0x7f00d800d040 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:45:32.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.147+0000 7f00f99e3b80 1 -- start start 2024-02-02T15:45:32.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.148+0000 7f00f99e3b80 1 Processor -- start 2024-02-02T15:45:32.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.148+0000 7f00f99e3b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55899bfa3010 0x55899bfae1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:32.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.148+0000 7f00f99e3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55899bfa3930 0x55899bfab3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:32.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.148+0000 7f00f99e3b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55899bfae7e0 0x55899bfab9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:32.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.149+0000 7f00f99e3b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55899bef4d50 con 0x55899bfae7e0 2024-02-02T15:45:32.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.149+0000 7f00f99e3b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55899be90d80 con 0x55899bfa3010 2024-02-02T15:45:32.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.149+0000 7f00f99e3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55899be90a20 con 0x55899bfa3930 2024-02-02T15:45:32.149 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.149+0000 7f00f99e3b80 10 client.0 _ll_register_callbacks cb 0x55899be92290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:45:32.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.149+0000 7f00df400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55899bfae7e0 0x55899bfab9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:32.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.149+0000 7f00df400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55899bfae7e0 0x55899bfab9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57244/0 (socket says 172.21.15.1:57244) 2024-02-02T15:45:32.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.149+0000 7f00df400700 1 -- 172.21.15.1:0/2884530791 learned_addr learned my addr 172.21.15.1:0/2884530791 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:45:32.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.149+0000 7f00df400700 1 -- 172.21.15.1:0/2884530791 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55899bfa3930 msgr2=0x55899bfab3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:45:32.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.149+0000 7f00df400700 1 --2- 172.21.15.1:0/2884530791 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55899bfa3930 0x55899bfab3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:32.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.149+0000 7f00df400700 1 -- 172.21.15.1:0/2884530791 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55899bfa3010 msgr2=0x55899bfae1c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-02T15:45:32.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.149+0000 7f00df400700 1 --2- 172.21.15.1:0/2884530791 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55899bfa3010 0x55899bfae1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:32.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.149+0000 7f00df400700 1 -- 172.21.15.1:0/2884530791 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00c8009a70 con 0x55899bfae7e0 2024-02-02T15:45:32.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.150+0000 7f00df400700 1 --2- 172.21.15.1:0/2884530791 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55899bfae7e0 0x55899bfab9c0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f00d8018e60 tx=0x7f00d8009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3510c0916962d84a server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:45:32.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.150+0000 7f00ccc00700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:45:32.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.150+0000 7f00ccc00700 1 -- 172.21.15.1:0/2884530791 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00d8010040 con 0x55899bfae7e0 2024-02-02T15:45:32.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.150+0000 7f00ccc00700 1 -- 172.21.15.1:0/2884530791 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f00d8016c50 con 0x55899bfae7e0 2024-02-02T15:45:32.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.150+0000 7f00f99e3b80 1 -- 172.21.15.1:0/2884530791 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55899bf970f0 con 0x55899bfae7e0 2024-02-02T15:45:32.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.151+0000 7f00ccc00700 1 -- 172.21.15.1:0/2884530791 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00d8016e10 con 0x55899bfae7e0 2024-02-02T15:45:32.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.151+0000 7f00f99e3b80 10 client.20013 fetch_fsmap learned FSMap version 78 2024-02-02T15:45:32.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.151+0000 7f00f99e3b80 1 -- 172.21.15.1:0/2884530791 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) v3 -- 0x55899bd22b60 con 0x55899bfae7e0 2024-02-02T15:45:32.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.151+0000 7f00ccc00700 1 -- 172.21.15.1:0/2884530791 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55899bf970f0 con 0x55899bfae7e0 2024-02-02T15:45:32.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.151+0000 7f00ccc00700 1 -- 172.21.15.1:0/2884530791 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 78) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f00d8017450 con 0x55899bfae7e0 2024-02-02T15:45:32.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.151+0000 7f00f99e3b80 10 client.20013 fetch_fsmap finished waiting for FSMap version 78 2024-02-02T15:45:32.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.151+0000 7f00f99e3b80 10 client.20013 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:45:32.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.151+0000 7f00f99e3b80 20 client.20013 populate_metadata read hostname 'smithi001' 2024-02-02T15:45:32.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.151+0000 7f00f99e3b80 1 --2- 172.21.15.1:0/2884530791 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55899bfbe2a0 0x55899bfc0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:32.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.151+0000 7f00f99e3b80 4 client.20013 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:45:32.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.151+0000 7f00f99e3b80 1 -- 172.21.15.1:0/2884530791 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55899beeddd0 con 0x55899bfbe2a0 2024-02-02T15:45:32.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.152+0000 7f00dea00700 1 --2- 172.21.15.1:0/2884530791 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55899bfbe2a0 0x55899bfc0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:32.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.153+0000 7f00ccc00700 10 client.20013 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:45:32.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.153+0000 7f00dea00700 1 --2- 172.21.15.1:0/2884530791 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55899bfbe2a0 0x55899bfc0740 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf85c6ff51831600 server_cookie=5d546c44add00147 in_seq=0 out_seq=0 2024-02-02T15:45:32.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.157+0000 7f00ccc00700 1 -- 172.21.15.1:0/2884530791 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55899beeddd0 con 0x55899bfbe2a0 2024-02-02T15:45:32.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.157+0000 7f00ccc00700 10 client.20013 handle_command_reply: tid=0 2024-02-02T15:45:32.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.157+0000 7f00f99e3b80 1 client.20013 shutdown 2024-02-02T15:45:32.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.160+0000 7f00f99e3b80 1 -- 172.21.15.1:0/2884530791 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55899bfae7e0 msgr2=0x55899bfab9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:32.160 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.160+0000 7f00f99e3b80 1 --2- 172.21.15.1:0/2884530791 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55899bfae7e0 0x55899bfab9c0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f00d8018e60 tx=0x7f00d8009920 comp rx=0 tx=0).stop 2024-02-02T15:45:32.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.160+0000 7f00f99e3b80 1 -- 172.21.15.1:0/2884530791 shutdown_connections 2024-02-02T15:45:32.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.160+0000 7f00f99e3b80 1 --2- 172.21.15.1:0/2884530791 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55899bfbe2a0 0x55899bfc0740 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:32.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.160+0000 7f00f99e3b80 1 --2- 172.21.15.1:0/2884530791 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55899bfae7e0 0x55899bfab9c0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:32.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.160+0000 7f00f99e3b80 1 --2- 172.21.15.1:0/2884530791 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55899bfa3010 0x55899bfae1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:32.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.160+0000 7f00f99e3b80 1 --2- 172.21.15.1:0/2884530791 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55899bfa3930 0x55899bfab3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:32.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.160+0000 7f00f99e3b80 1 -- 172.21.15.1:0/2884530791 >> 172.21.15.1:0/2884530791 conn(0x55899bf02db0 msgr2=0x55899bf9efa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:45:32.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.160+0000 7f00ccc00700 0 client.20013 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:45:32.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.161+0000 7f00f99e3b80 1 -- 172.21.15.1:0/2884530791 shutdown_connections 2024-02-02T15:45:32.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.161+0000 7f00f99e3b80 1 -- 172.21.15.1:0/2884530791 wait complete. 2024-02-02T15:45:32.161 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.162+0000 7f00f99e3b80 20 client.20013 trim_cache size 0 max 16384 2024-02-02T15:45:32.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.193+0000 7f00f99e3b80 1 Processor -- start 2024-02-02T15:45:32.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.193+0000 7f00f99e3b80 1 -- start start 2024-02-02T15:45:32.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.193+0000 7f00f99e3b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55899bfa3010 0x55899bfaab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:32.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.193+0000 7f00f99e3b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55899bfa3930 0x55899bfadcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:32.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.193+0000 7f00f99e3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55899bfae7e0 0x55899bf9eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:32.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.193+0000 7f00f99e3b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55899bd786a0 con 0x55899bfa3930 2024-02-02T15:45:32.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.193+0000 7f00f99e3b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55899bfa0910 con 0x55899bfa3010 2024-02-02T15:45:32.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.193+0000 7f00f99e3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55899bfa9930 con 0x55899bfae7e0 2024-02-02T15:45:32.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.194+0000 7f00df400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55899bfae7e0 0x55899bf9eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:32.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.194+0000 7f00dea00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55899bfa3010 0x55899bfaab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:32.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.194+0000 7f00de000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55899bfa3930 0x55899bfadcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:32.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.194+0000 7f00dea00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55899bfa3010 0x55899bfaab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:47792/0 (socket says 172.21.15.1:47792) 2024-02-02T15:45:32.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.194+0000 7f00dea00700 1 -- 172.21.15.1:0/2585223303 learned_addr learned my addr 172.21.15.1:0/2585223303 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:45:32.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.195+0000 7f00dea00700 1 -- 172.21.15.1:0/2585223303 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55899bfae7e0 msgr2=0x55899bf9eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:45:32.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.195+0000 7f00dea00700 1 --2- 172.21.15.1:0/2585223303 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55899bfae7e0 0x55899bf9eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:32.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.195+0000 7f00dea00700 1 -- 172.21.15.1:0/2585223303 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55899bfa3930 msgr2=0x55899bfadcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:45:32.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.195+0000 7f00dea00700 1 --2- 172.21.15.1:0/2585223303 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55899bfa3930 0x55899bfadcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:32.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.195+0000 7f00dea00700 1 -- 172.21.15.1:0/2585223303 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55899bebaea0 con 0x55899bfa3010 2024-02-02T15:45:32.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.195+0000 7f00dea00700 1 --2- 172.21.15.1:0/2585223303 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55899bfa3010 0x55899bfaab90 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f00d0096a90 tx=0x7f00d0009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4200fae314abf8eb server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:45:32.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.196+0000 7f00d5200700 1 -- 172.21.15.1:0/2585223303 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00d009a360 con 0x55899bfa3010 2024-02-02T15:45:32.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.196+0000 7f00d5200700 1 -- 172.21.15.1:0/2585223303 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f00d009a0d0 con 0x55899bfa3010 2024-02-02T15:45:32.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.196+0000 7f00d5200700 1 -- 172.21.15.1:0/2585223303 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00d00a2570 con 0x55899bfa3010 2024-02-02T15:45:32.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.197+0000 7f00f99e3b80 1 -- 172.21.15.1:0/2585223303 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55899bfa3010 msgr2=0x55899bfaab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:32.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.197+0000 7f00f99e3b80 1 --2- 172.21.15.1:0/2585223303 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55899bfa3010 0x55899bfaab90 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f00d0096a90 tx=0x7f00d0009ca0 comp rx=0 tx=0).stop 2024-02-02T15:45:32.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.197+0000 7f00f99e3b80 1 -- 172.21.15.1:0/2585223303 shutdown_connections 2024-02-02T15:45:32.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.197+0000 7f00f99e3b80 1 --2- 172.21.15.1:0/2585223303 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55899bfae7e0 0x55899bf9eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:32.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.197+0000 7f00f99e3b80 1 --2- 172.21.15.1:0/2585223303 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55899bfa3010 0x55899bfaab90 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:32.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.197+0000 7f00f99e3b80 1 --2- 172.21.15.1:0/2585223303 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55899bfa3930 0x55899bfadcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:32.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.197+0000 7f00f99e3b80 1 -- 172.21.15.1:0/2585223303 >> 172.21.15.1:0/2585223303 conn(0x55899bf02db0 msgr2=0x55899bfab390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:45:32.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.198+0000 7f00f99e3b80 1 -- 172.21.15.1:0/2585223303 shutdown_connections 2024-02-02T15:45:32.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.198+0000 7f00f99e3b80 1 -- 172.21.15.1:0/2585223303 wait complete. 2024-02-02T15:45:32.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.198+0000 7f00f99e3b80 1 -- start start 2024-02-02T15:45:32.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.198+0000 7f00f99e3b80 1 Processor -- start 2024-02-02T15:45:32.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.199+0000 7f00f99e3b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55899bfa3010 0x55899bfb2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:32.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.199+0000 7f00f99e3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55899bfa3930 0x55899bfb33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:32.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.199+0000 7f00f99e3b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55899bfae7e0 0x55899c00ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:32.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.199+0000 7f00f99e3b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55899bfb3ab0 con 0x55899bfa3010 2024-02-02T15:45:32.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.199+0000 7f00f99e3b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55899c00f3c0 con 0x55899bfae7e0 2024-02-02T15:45:32.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.199+0000 7f00f99e3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55899c00f500 con 0x55899bfa3930 2024-02-02T15:45:32.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.200+0000 7f00f99e3b80 10 client.0 _ll_register_callbacks cb 0x55899be92290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:45:32.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.200+0000 7f00dea00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55899bfa3010 0x55899bfb2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:32.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.200+0000 7f00df400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55899bfae7e0 0x55899c00ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:32.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.200+0000 7f00df400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55899bfae7e0 0x55899c00ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:47794/0 (socket says 172.21.15.1:47794) 2024-02-02T15:45:32.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.200+0000 7f00df400700 1 -- 172.21.15.1:0/1068941053 learned_addr learned my addr 172.21.15.1:0/1068941053 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:45:32.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.200+0000 7f00df400700 1 -- 172.21.15.1:0/1068941053 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55899bfa3930 msgr2=0x55899bfb33d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-02T15:45:32.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.200+0000 7f00df400700 1 --2- 172.21.15.1:0/1068941053 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55899bfa3930 0x55899bfb33d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:32.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.200+0000 7f00df400700 1 -- 172.21.15.1:0/1068941053 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55899bfa3010 msgr2=0x55899bfb2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:45:32.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.200+0000 7f00df400700 1 --2- 172.21.15.1:0/1068941053 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55899bfa3010 0x55899bfb2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:32.202 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.201+0000 7f00df400700 1 -- 172.21.15.1:0/1068941053 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00e0199550 con 0x55899bfae7e0 2024-02-02T15:45:32.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.201+0000 7f00df400700 1 --2- 172.21.15.1:0/1068941053 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55899bfae7e0 0x55899c00ee80 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f00d801a010 tx=0x7f00d8006190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9cd8de9ad09c163c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:45:32.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.201+0000 7f00ccc00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:45:32.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.202+0000 7f00ccc00700 1 -- 172.21.15.1:0/1068941053 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00d80173c0 con 0x55899bfae7e0 2024-02-02T15:45:32.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.202+0000 7f00ccc00700 1 -- 172.21.15.1:0/1068941053 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f00d803b070 con 0x55899bfae7e0 2024-02-02T15:45:32.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.202+0000 7f00f99e3b80 1 -- 172.21.15.1:0/1068941053 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f00c8019c50 con 0x55899bfae7e0 2024-02-02T15:45:32.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.202+0000 7f00ccc00700 1 -- 172.21.15.1:0/1068941053 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00d8017520 con 0x55899bfae7e0 2024-02-02T15:45:32.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.202+0000 7f00f99e3b80 10 client.30317 fetch_fsmap learned FSMap version 78 2024-02-02T15:45:32.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.203+0000 7f00f99e3b80 1 -- 172.21.15.1:0/1068941053 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) v3 -- 0x55899bd7acd0 con 0x55899bfae7e0 2024-02-02T15:45:32.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.203+0000 7f00ccc00700 1 -- 172.21.15.1:0/1068941053 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f00c8019c50 con 0x55899bfae7e0 2024-02-02T15:45:32.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.203+0000 7f00ccc00700 1 -- 172.21.15.1:0/1068941053 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 78) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f00d8003a00 con 0x55899bfae7e0 2024-02-02T15:45:32.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.203+0000 7f00f99e3b80 10 client.30317 fetch_fsmap finished waiting for FSMap version 78 2024-02-02T15:45:32.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.203+0000 7f00f99e3b80 10 client.30317 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:45:32.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.203+0000 7f00f99e3b80 20 client.30317 populate_metadata read hostname 'smithi001' 2024-02-02T15:45:32.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.203+0000 7f00f99e3b80 1 --2- 172.21.15.1:0/1068941053 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55899c010c00 0x55899c013020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:32.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.203+0000 7f00f99e3b80 4 client.30317 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2024-02-02T15:45:32.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.203+0000 7f00f99e3b80 1 -- 172.21.15.1:0/1068941053 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55899beb3f50 con 0x55899c010c00 2024-02-02T15:45:32.204 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.204+0000 7f00dea00700 1 --2- 172.21.15.1:0/1068941053 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55899c010c00 0x55899c013020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:32.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.204+0000 7f00dea00700 1 --2- 172.21.15.1:0/1068941053 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55899c010c00 0x55899c013020 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f72bfafaf4288ba server_cookie=4595d6b0ca2dae7e in_seq=0 out_seq=0 2024-02-02T15:45:32.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.205+0000 7f00ccc00700 10 client.30317 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:45:32.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.205+0000 7f00ccc00700 1 -- 172.21.15.1:0/1068941053 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55899beb3f50 con 0x55899c010c00 2024-02-02T15:45:32.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.205+0000 7f00ccc00700 10 client.30317 handle_command_reply: tid=0 2024-02-02T15:45:32.205 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.205+0000 7f00f99e3b80 1 client.30317 shutdown 2024-02-02T15:45:32.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.205+0000 7f00f99e3b80 1 -- 172.21.15.1:0/1068941053 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55899bfae7e0 msgr2=0x55899c00ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:32.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.205+0000 7f00f99e3b80 1 --2- 172.21.15.1:0/1068941053 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55899bfae7e0 0x55899c00ee80 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f00d801a010 tx=0x7f00d8006190 comp rx=0 tx=0).stop 2024-02-02T15:45:32.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.206+0000 7f00f99e3b80 1 -- 172.21.15.1:0/1068941053 shutdown_connections 2024-02-02T15:45:32.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.206+0000 7f00f99e3b80 1 --2- 172.21.15.1:0/1068941053 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55899c010c00 0x55899c013020 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:32.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.206+0000 7f00ccc00700 0 client.30317 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:45:32.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.206+0000 7f00f99e3b80 1 --2- 172.21.15.1:0/1068941053 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55899bfae7e0 0x55899c00ee80 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:32.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.206+0000 7f00f99e3b80 1 --2- 172.21.15.1:0/1068941053 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55899bfa3010 0x55899bfb2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:32.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.206+0000 7f00f99e3b80 1 --2- 172.21.15.1:0/1068941053 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55899bfa3930 0x55899bfb33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:32.206 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.206+0000 7f00f99e3b80 1 -- 172.21.15.1:0/1068941053 >> 172.21.15.1:0/1068941053 conn(0x55899bf02db0 msgr2=0x55899bfa4ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:45:32.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.206+0000 7f00f99e3b80 1 -- 172.21.15.1:0/1068941053 shutdown_connections 2024-02-02T15:45:32.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.206+0000 7f00f99e3b80 1 -- 172.21.15.1:0/1068941053 wait complete. 2024-02-02T15:45:32.207 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.207+0000 7f00f99e3b80 20 client.30317 trim_cache size 0 max 16384 2024-02-02T15:45:32.207 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:45:32.208 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2024-02-02T15:45:32.208 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2024-02-02T15:45:32.208 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:45:32.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.209+0000 7f00e5e00700 1 -- 172.21.15.1:0/1892822103 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f00c00b16e0 msgr2=0x7f00c00b3ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:32.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.209+0000 7f00e5e00700 1 --2- 172.21.15.1:0/1892822103 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f00c00b16e0 0x7f00c00b3ba0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f00d8009c10 tx=0x7f00d800d040 comp rx=0 tx=0).stop 2024-02-02T15:45:32.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.209+0000 7f00e5e00700 1 -- 172.21.15.1:0/1892822103 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f00e00ff860 msgr2=0x7f00e0107cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:32.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.209+0000 7f00e5e00700 1 --2- 172.21.15.1:0/1892822103 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f00e00ff860 0x7f00e0107cf0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f00d0005910 tx=0x7f00d0009580 comp rx=0 tx=0).stop 2024-02-02T15:45:32.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.209+0000 7f00e5e00700 1 -- 172.21.15.1:0/1892822103 shutdown_connections 2024-02-02T15:45:32.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.209+0000 7f00e5e00700 1 --2- 172.21.15.1:0/1892822103 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f00c00b16e0 0x7f00c00b3ba0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:32.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.209+0000 7f00e5e00700 1 --2- 172.21.15.1:0/1892822103 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f00e0108770 0x7f00e01a9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:32.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.209+0000 7f00e5e00700 1 --2- 172.21.15.1:0/1892822103 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f00e00ff860 0x7f00e0107cf0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:32.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.209+0000 7f00e5e00700 1 --2- 172.21.15.1:0/1892822103 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f00e0106130 0x7f00e0108230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:32.209 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.209+0000 7f00e5e00700 1 -- 172.21.15.1:0/1892822103 >> 172.21.15.1:0/1892822103 conn(0x7f00e0074f30 msgr2=0x7f00e00fd6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:45:32.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.209+0000 7f00e5e00700 1 -- 172.21.15.1:0/1892822103 shutdown_connections 2024-02-02T15:45:32.210 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.209+0000 7f00e5e00700 1 -- 172.21.15.1:0/1892822103 wait complete. 2024-02-02T15:45:32.240 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-02T15:45:32.241 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T15:45:32.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.572+0000 7ff443a00700 1 -- 172.21.15.1:0/845225021 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff43c105c80 msgr2=0x7ff43c106060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:32.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.572+0000 7ff443a00700 1 --2- 172.21.15.1:0/845225021 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff43c105c80 0x7ff43c106060 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7ff42c0098f0 tx=0x7ff42c009c00 comp rx=0 tx=0).stop 2024-02-02T15:45:32.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.573+0000 7ff443a00700 1 -- 172.21.15.1:0/845225021 shutdown_connections 2024-02-02T15:45:32.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.573+0000 7ff443a00700 1 --2- 172.21.15.1:0/845225021 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff43c100580 0x7ff43c10ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:32.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.573+0000 7ff443a00700 1 --2- 172.21.15.1:0/845225021 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff43c105c80 0x7ff43c106060 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:32.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.573+0000 7ff443a00700 1 --2- 172.21.15.1:0/845225021 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff43c0ffbc0 0x7ff43c100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:32.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.573+0000 7ff443a00700 1 -- 172.21.15.1:0/845225021 >> 172.21.15.1:0/845225021 conn(0x7ff43c0769c0 msgr2=0x7ff43c076dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:45:32.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.573+0000 7ff443a00700 1 -- 172.21.15.1:0/845225021 shutdown_connections 2024-02-02T15:45:32.574 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.573+0000 7ff443a00700 1 -- 172.21.15.1:0/845225021 wait complete. 2024-02-02T15:45:32.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.575+0000 7ff443a00700 1 Processor -- start 2024-02-02T15:45:32.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.575+0000 7ff443a00700 1 -- start start 2024-02-02T15:45:32.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.576+0000 7ff443a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff43c0ffbc0 0x7ff43c196dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:32.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.576+0000 7ff443a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff43c100580 0x7ff43c197310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:32.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.576+0000 7ff443a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff43c105c80 0x7ff43c19b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:32.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.576+0000 7ff443a00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff43c19bdb0 con 0x7ff43c0ffbc0 2024-02-02T15:45:32.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.576+0000 7ff43be00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff43c100580 0x7ff43c197310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:32.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.576+0000 7ff43be00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff43c100580 0x7ff43c197310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:47806/0 (socket says 172.21.15.1:47806) 2024-02-02T15:45:32.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.576+0000 7ff43be00700 1 -- 172.21.15.1:0/3359530732 learned_addr learned my addr 172.21.15.1:0/3359530732 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:45:32.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.577+0000 7ff443a00700 1 -- 172.21.15.1:0/3359530732 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff43c19bf20 con 0x7ff43c100580 2024-02-02T15:45:32.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.577+0000 7ff441000700 1 --2- 172.21.15.1:0/3359530732 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff43c0ffbc0 0x7ff43c196dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:32.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.577+0000 7ff441a00700 1 --2- 172.21.15.1:0/3359530732 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff43c105c80 0x7ff43c19b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:32.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.577+0000 7ff443a00700 1 -- 172.21.15.1:0/3359530732 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff43c19c0e0 con 0x7ff43c105c80 2024-02-02T15:45:32.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.578+0000 7ff441a00700 1 -- 172.21.15.1:0/3359530732 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff43c100580 msgr2=0x7ff43c197310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:32.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.578+0000 7ff441a00700 1 --2- 172.21.15.1:0/3359530732 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff43c100580 0x7ff43c197310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:32.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.578+0000 7ff441a00700 1 -- 172.21.15.1:0/3359530732 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff43c0ffbc0 msgr2=0x7ff43c196dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:32.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.578+0000 7ff441a00700 1 --2- 172.21.15.1:0/3359530732 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff43c0ffbc0 0x7ff43c196dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:32.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.578+0000 7ff441a00700 1 -- 172.21.15.1:0/3359530732 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4240095f0 con 0x7ff43c105c80 2024-02-02T15:45:32.579 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.579+0000 7ff441a00700 1 --2- 172.21.15.1:0/3359530732 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff43c105c80 0x7ff43c19b770 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7ff43400efc0 tx=0x7ff43400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:45:32.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.580+0000 7ff439600700 1 -- 172.21.15.1:0/3359530732 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff434004bb0 con 0x7ff43c105c80 2024-02-02T15:45:32.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.580+0000 7ff439600700 1 -- 172.21.15.1:0/3359530732 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff43400f040 con 0x7ff43c105c80 2024-02-02T15:45:32.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.580+0000 7ff439600700 1 -- 172.21.15.1:0/3359530732 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff434004280 con 0x7ff43c105c80 2024-02-02T15:45:32.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.580+0000 7ff443a00700 1 -- 172.21.15.1:0/3359530732 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff42c009520 con 0x7ff43c105c80 2024-02-02T15:45:32.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.581+0000 7ff443a00700 1 -- 172.21.15.1:0/3359530732 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff43c10d150 con 0x7ff43c105c80 2024-02-02T15:45:32.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.584+0000 7ff443a00700 1 -- 172.21.15.1:0/3359530732 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff43c067bc0 con 0x7ff43c105c80 2024-02-02T15:45:32.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.586+0000 7ff439600700 1 -- 172.21.15.1:0/3359530732 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7ff43400ca50 con 0x7ff43c105c80 2024-02-02T15:45:32.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.589+0000 7ff439600700 1 --2- 172.21.15.1:0/3359530732 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff41c0a07e0 0x7ff41c0a2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:32.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.589+0000 7ff439600700 1 -- 172.21.15.1:0/3359530732 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 9453+0+0 (secure 0 0 0) 0x7ff4340cf890 con 0x7ff43c105c80 2024-02-02T15:45:32.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.591+0000 7ff441000700 1 --2- 172.21.15.1:0/3359530732 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff41c0a07e0 0x7ff41c0a2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:32.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.592+0000 7ff441000700 1 --2- 172.21.15.1:0/3359530732 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff41c0a07e0 0x7ff41c0a2ca0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff42c00c010 tx=0x7ff42c019040 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:45:32.594 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:32.595+0000 7ff439600700 1 -- 172.21.15.1:0/3359530732 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7ff434098e60 con 0x7ff43c105c80 2024-02-02T15:45:32.681 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:32 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:45:32.681 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:32 smithi001 ceph-mon[35470]: pgmap v583: 129 pgs: 129 active+clean; 2.2 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 9.5 MiB/s wr, 530 op/s 2024-02-02T15:45:33.006 INFO:tasks.workunit.client.0.smithi001.stdout: 10 164080 27.50 MB/sec execute 185 sec latency 85.938 ms 2024-02-02T15:45:33.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.087+0000 7ff443a00700 1 -- 172.21.15.1:0/3359530732 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff43c10d560 con 0x7ff43c105c80 2024-02-02T15:45:33.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.088+0000 7ff439600700 1 -- 172.21.15.1:0/3359530732 <== mon.1 v2:172.21.15.142:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+10881 (secure 0 0 0) 0x7ff43409c480 con 0x7ff43c105c80 2024-02-02T15:45:33.088 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:45:33.088 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":78,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2922240448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2922240448},{"type":"v1","addr":"172.21.15.1:6835","nonce":2922240448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2729497431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2729497431},{"type":"v1","addr":"172.21.15.1:6837","nonce":2729497431}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":78,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-02T15:24:47.761435+0000","modified":"2024-02-02T15:45:15.510793+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24529,"mds_2":24518,"mds_3":24514,"mds_4":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14517":{"gid":14517,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/1218303683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":1218303683},{"type":"v1","addr":"172.21.15.1:6839","nonce":1218303683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14529":{"gid":14529,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/2059197496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2059197496},{"type":"v1","addr":"172.21.15.1:6841","nonce":2059197496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/7648986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":7648986},{"type":"v1","addr":"172.21.15.142:6833","nonce":7648986}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/3482746094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3482746094},{"type":"v1","addr":"172.21.15.71:6833","nonce":3482746094}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/388207814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":388207814},{"type":"v1","addr":"172.21.15.71:6835","nonce":388207814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6835/1052606547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1052606547},{"type":"v1","addr":"172.21.15.142:6835","nonce":1052606547}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.142:6837/2870289019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2870289019},{"type":"v1","addr":"172.21.15.142:6837","nonce":2870289019}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.71:6837/628016199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":628016199},{"type":"v1","addr":"172.21.15.71:6837","nonce":628016199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.142:6839/321675421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":321675421},{"type":"v1","addr":"172.21.15.142:6839","nonce":321675421}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/1268086493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1268086493},{"type":"v1","addr":"172.21.15.71:6839","nonce":1268086493}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-02T15:45:33.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.091+0000 7ff443a00700 1 -- 172.21.15.1:0/3359530732 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff41c0a07e0 msgr2=0x7ff41c0a2ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:33.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.091+0000 7ff443a00700 1 --2- 172.21.15.1:0/3359530732 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff41c0a07e0 0x7ff41c0a2ca0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff42c00c010 tx=0x7ff42c019040 comp rx=0 tx=0).stop 2024-02-02T15:45:33.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.091+0000 7ff443a00700 1 -- 172.21.15.1:0/3359530732 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff43c105c80 msgr2=0x7ff43c19b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:33.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.091+0000 7ff443a00700 1 --2- 172.21.15.1:0/3359530732 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff43c105c80 0x7ff43c19b770 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7ff43400efc0 tx=0x7ff43400c3a0 comp rx=0 tx=0).stop 2024-02-02T15:45:33.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.091+0000 7ff443a00700 1 -- 172.21.15.1:0/3359530732 shutdown_connections 2024-02-02T15:45:33.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.091+0000 7ff443a00700 1 --2- 172.21.15.1:0/3359530732 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff41c0a07e0 0x7ff41c0a2ca0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:33.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.091+0000 7ff443a00700 1 --2- 172.21.15.1:0/3359530732 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff43c105c80 0x7ff43c19b770 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:33.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.091+0000 7ff443a00700 1 --2- 172.21.15.1:0/3359530732 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff43c0ffbc0 0x7ff43c196dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:33.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.091+0000 7ff443a00700 1 --2- 172.21.15.1:0/3359530732 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff43c100580 0x7ff43c197310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:33.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.091+0000 7ff443a00700 1 -- 172.21.15.1:0/3359530732 >> 172.21.15.1:0/3359530732 conn(0x7ff43c0769c0 msgr2=0x7ff43c0fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:45:33.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.092+0000 7ff443a00700 1 -- 172.21.15.1:0/3359530732 shutdown_connections 2024-02-02T15:45:33.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.092+0000 7ff443a00700 1 -- 172.21.15.1:0/3359530732 wait complete. 2024-02-02T15:45:33.093 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 78 2024-02-02T15:45:33.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:32 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:45:33.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:32 smithi071 ceph-mon[44707]: pgmap v583: 129 pgs: 129 active+clean; 2.2 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 9.5 MiB/s wr, 530 op/s 2024-02-02T15:45:33.115 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-02-02T15:45:33.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:32 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:45:33.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:32 smithi142 ceph-mon[45728]: pgmap v583: 129 pgs: 129 active+clean; 2.2 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 9.5 MiB/s wr, 530 op/s 2024-02-02T15:45:33.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.438+0000 7f4313e00700 1 -- 172.21.15.1:0/3943552142 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f430c103be0 msgr2=0x7f430c107c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:33.438 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.438+0000 7f4313e00700 1 --2- 172.21.15.1:0/3943552142 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f430c103be0 0x7f430c107c20 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f42f4009990 tx=0x7f42f4009ca0 comp rx=0 tx=0).stop 2024-02-02T15:45:33.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.438+0000 7f4313e00700 1 -- 172.21.15.1:0/3943552142 shutdown_connections 2024-02-02T15:45:33.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.438+0000 7f4313e00700 1 --2- 172.21.15.1:0/3943552142 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f430c108350 0x7f430c10fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:33.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.438+0000 7f4313e00700 1 --2- 172.21.15.1:0/3943552142 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f430c103230 0x7f430c103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:33.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.438+0000 7f4313e00700 1 --2- 172.21.15.1:0/3943552142 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f430c103be0 0x7f430c107c20 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:33.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.438+0000 7f4313e00700 1 -- 172.21.15.1:0/3943552142 >> 172.21.15.1:0/3943552142 conn(0x7f430c0fea70 msgr2=0x7f430c100e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:45:33.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.438+0000 7f4313e00700 1 -- 172.21.15.1:0/3943552142 shutdown_connections 2024-02-02T15:45:33.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.439+0000 7f4313e00700 1 -- 172.21.15.1:0/3943552142 wait complete. 2024-02-02T15:45:33.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.440+0000 7f4313e00700 1 Processor -- start 2024-02-02T15:45:33.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.440+0000 7f4313e00700 1 -- start start 2024-02-02T15:45:33.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.440+0000 7f4313e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f430c103230 0x7f430c10f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:33.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.440+0000 7f4313e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f430c108350 0x7f430c10c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:33.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.441+0000 7f4313e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f430c10cef0 0x7f430c10d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:33.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.441+0000 7f4313e00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f430c1101a0 con 0x7f430c108350 2024-02-02T15:45:33.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.441+0000 7f4313e00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f430c1a9e50 con 0x7f430c10cef0 2024-02-02T15:45:33.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.441+0000 7f4313e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f430c1a9f90 con 0x7f430c103230 2024-02-02T15:45:33.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.441+0000 7f4311400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f430c103230 0x7f430c10f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:33.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.441+0000 7f4311400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f430c103230 0x7f430c10f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:36806/0 (socket says 172.21.15.1:36806) 2024-02-02T15:45:33.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.441+0000 7f4311400700 1 -- 172.21.15.1:0/3922168735 learned_addr learned my addr 172.21.15.1:0/3922168735 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:45:33.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.442+0000 7f4310a00700 1 --2- 172.21.15.1:0/3922168735 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f430c108350 0x7f430c10c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:33.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.442+0000 7f4311400700 1 -- 172.21.15.1:0/3922168735 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f430c10cef0 msgr2=0x7f430c10d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:33.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.442+0000 7f4311400700 1 --2- 172.21.15.1:0/3922168735 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f430c10cef0 0x7f430c10d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:33.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.442+0000 7f4311400700 1 -- 172.21.15.1:0/3922168735 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f430c108350 msgr2=0x7f430c10c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:33.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.442+0000 7f4311400700 1 --2- 172.21.15.1:0/3922168735 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f430c108350 0x7f430c10c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:33.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.442+0000 7f4311400700 1 -- 172.21.15.1:0/3922168735 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42f40095f0 con 0x7f430c103230 2024-02-02T15:45:33.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.443+0000 7f4311400700 1 --2- 172.21.15.1:0/3922168735 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f430c103230 0x7f430c10f9a0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f42fc00b530 tx=0x7f42fc00b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:45:33.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.443+0000 7f430a000700 1 -- 172.21.15.1:0/3922168735 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42fc011830 con 0x7f430c103230 2024-02-02T15:45:33.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.443+0000 7f430a000700 1 -- 172.21.15.1:0/3922168735 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f42fc011e50 con 0x7f430c103230 2024-02-02T15:45:33.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.443+0000 7f4313e00700 1 -- 172.21.15.1:0/3922168735 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f430c1aa1f0 con 0x7f430c103230 2024-02-02T15:45:33.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.443+0000 7f430a000700 1 -- 172.21.15.1:0/3922168735 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42fc010e70 con 0x7f430c103230 2024-02-02T15:45:33.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.443+0000 7f4313e00700 1 -- 172.21.15.1:0/3922168735 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f430c1aa750 con 0x7f430c103230 2024-02-02T15:45:33.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.447+0000 7f430a000700 1 -- 172.21.15.1:0/3922168735 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f42fc010510 con 0x7f430c103230 2024-02-02T15:45:33.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.449+0000 7f4327895b80 1 Processor -- start 2024-02-02T15:45:33.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.449+0000 7f4327895b80 1 -- start start 2024-02-02T15:45:33.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.449+0000 7f430a000700 1 --2- 172.21.15.1:0/3922168735 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f42ec0a0710 0x7f42ec0a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:33.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.449+0000 7f4327895b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5591986d7010 0x5591986d73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:33.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.449+0000 7f4327895b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5591986d7930 0x5591986e22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:33.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.449+0000 7f4327895b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5591986e27e0 0x5591986e4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:33.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.449+0000 7f4327895b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55919864b4c0 con 0x5591986e27e0 2024-02-02T15:45:33.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.449+0000 7f4327895b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55919862faa0 con 0x5591986d7010 2024-02-02T15:45:33.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.449+0000 7f4327895b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55919862d460 con 0x5591986d7930 2024-02-02T15:45:33.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.450+0000 7f430a000700 1 -- 172.21.15.1:0/3922168735 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f42fc0cba80 con 0x7f430c103230 2024-02-02T15:45:33.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.450+0000 7f4311400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5591986d7010 0x5591986d73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:33.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.450+0000 7f4311400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5591986d7010 0x5591986d73f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:49120/0 (socket says 172.21.15.1:49120) 2024-02-02T15:45:33.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.450+0000 7f4311400700 1 -- 172.21.15.1:0/2887297568 learned_addr learned my addr 172.21.15.1:0/2887297568 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:45:33.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.450+0000 7f4310a00700 1 --2- 172.21.15.1:0/2887297568 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5591986d7930 0x5591986e22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:33.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.450+0000 7f4311400700 1 -- 172.21.15.1:0/2887297568 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5591986d7930 msgr2=0x5591986e22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:45:33.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.450+0000 7f4311400700 1 --2- 172.21.15.1:0/2887297568 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5591986d7930 0x5591986e22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:33.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.450+0000 7f4311400700 1 -- 172.21.15.1:0/2887297568 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5591986e27e0 msgr2=0x5591986e4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:45:33.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.450+0000 7f4311400700 1 --2- 172.21.15.1:0/2887297568 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5591986e27e0 0x5591986e4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:33.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.450+0000 7f4311400700 1 -- 172.21.15.1:0/2887297568 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5591984aa3b0 con 0x5591986d7010 2024-02-02T15:45:33.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.450+0000 7f4311e00700 1 --2- 172.21.15.1:0/3922168735 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f42ec0a0710 0x7f42ec0a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:33.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.451+0000 7f4311400700 1 --2- 172.21.15.1:0/2887297568 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5591986d7010 0x5591986d73f0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f42fc098c40 tx=0x7f42fc0bce90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fda2d2cba2f2646 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:45:33.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.451+0000 7f4302600700 1 -- 172.21.15.1:0/2887297568 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42fc0993b0 con 0x5591986d7010 2024-02-02T15:45:33.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.451+0000 7f4302600700 1 -- 172.21.15.1:0/2887297568 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f42fc098cb0 con 0x5591986d7010 2024-02-02T15:45:33.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.451+0000 7f4302600700 1 -- 172.21.15.1:0/2887297568 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42fc0c9070 con 0x5591986d7010 2024-02-02T15:45:33.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.452+0000 7f4327895b80 1 -- 172.21.15.1:0/2887297568 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5591986d7010 msgr2=0x5591986d73f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:33.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.452+0000 7f4327895b80 1 --2- 172.21.15.1:0/2887297568 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5591986d7010 0x5591986d73f0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f42fc098c40 tx=0x7f42fc0bce90 comp rx=0 tx=0).stop 2024-02-02T15:45:33.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.452+0000 7f4311e00700 1 --2- 172.21.15.1:0/3922168735 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f42ec0a0710 0x7f42ec0a2bd0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f4304009690 tx=0x7f43040095e0 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:45:33.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.452+0000 7f4327895b80 1 -- 172.21.15.1:0/2887297568 shutdown_connections 2024-02-02T15:45:33.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.452+0000 7f4327895b80 1 --2- 172.21.15.1:0/2887297568 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5591986e27e0 0x5591986e4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:33.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.452+0000 7f4327895b80 1 --2- 172.21.15.1:0/2887297568 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5591986d7010 0x5591986d73f0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:33.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.452+0000 7f4327895b80 1 --2- 172.21.15.1:0/2887297568 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5591986d7930 0x5591986e22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:33.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.452+0000 7f4327895b80 1 -- 172.21.15.1:0/2887297568 >> 172.21.15.1:0/2887297568 conn(0x559198636db0 msgr2=0x5591986d33b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:45:33.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.452+0000 7f4327895b80 1 -- 172.21.15.1:0/2887297568 shutdown_connections 2024-02-02T15:45:33.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.452+0000 7f4327895b80 1 -- 172.21.15.1:0/2887297568 wait complete. 2024-02-02T15:45:33.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.453+0000 7f4327895b80 1 -- start start 2024-02-02T15:45:33.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.453+0000 7f4327895b80 1 Processor -- start 2024-02-02T15:45:33.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.454+0000 7f4327895b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5591986d7010 0x5591986e21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:33.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.454+0000 7f4327895b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5591986d7930 0x5591986df430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:33.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.454+0000 7f4327895b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5591986e27e0 0x5591986dfab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:33.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.454+0000 7f4327895b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x559198628d50 con 0x5591986d7010 2024-02-02T15:45:33.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.454+0000 7f4327895b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591985c4d80 con 0x5591986e27e0 2024-02-02T15:45:33.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.454+0000 7f4327895b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591985c4a20 con 0x5591986d7930 2024-02-02T15:45:33.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.454+0000 7f4327895b80 10 client.0 _ll_register_callbacks cb 0x5591985c6290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:45:33.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.455+0000 7f4310a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5591986d7930 0x5591986df430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:33.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.455+0000 7f4311e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5591986e27e0 0x5591986dfab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:33.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.455+0000 7f4310a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5591986d7930 0x5591986df430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:36824/0 (socket says 172.21.15.1:36824) 2024-02-02T15:45:33.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.455+0000 7f4310a00700 1 -- 172.21.15.1:0/1830383215 learned_addr learned my addr 172.21.15.1:0/1830383215 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:45:33.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.455+0000 7f4311400700 1 --2- 172.21.15.1:0/1830383215 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5591986d7010 0x5591986e21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:33.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.455+0000 7f4311400700 1 -- 172.21.15.1:0/1830383215 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5591986d7930 msgr2=0x5591986df430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:45:33.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.455+0000 7f4311400700 1 --2- 172.21.15.1:0/1830383215 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5591986d7930 0x5591986df430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:33.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.455+0000 7f4311400700 1 -- 172.21.15.1:0/1830383215 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5591986e27e0 msgr2=0x5591986dfab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:45:33.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.456+0000 7f4311400700 1 --2- 172.21.15.1:0/1830383215 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5591986e27e0 0x5591986dfab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:33.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.456+0000 7f4311400700 1 -- 172.21.15.1:0/1830383215 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f430c1aa750 con 0x5591986d7010 2024-02-02T15:45:33.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.456+0000 7f4311400700 1 --2- 172.21.15.1:0/1830383215 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5591986d7010 0x5591986e21b0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f42fc0972a0 tx=0x7f42fc099430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab723056fdc13b33 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:45:33.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.456+0000 7f42faa00700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:45:33.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.456+0000 7f42faa00700 1 -- 172.21.15.1:0/1830383215 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42fc0c9250 con 0x5591986d7010 2024-02-02T15:45:33.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.456+0000 7f42faa00700 1 -- 172.21.15.1:0/1830383215 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f42fc0a0d20 con 0x5591986d7010 2024-02-02T15:45:33.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.456+0000 7f4327895b80 1 -- 172.21.15.1:0/1830383215 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5591986cb0f0 con 0x5591986d7010 2024-02-02T15:45:33.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.457+0000 7f42faa00700 1 -- 172.21.15.1:0/1830383215 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7f42fc0b8330 con 0x5591986d7010 2024-02-02T15:45:33.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.457+0000 7f4327895b80 10 client.20031 fetch_fsmap learned FSMap version 78 2024-02-02T15:45:33.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.457+0000 7f4327895b80 1 -- 172.21.15.1:0/1830383215 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) v3 -- 0x559198456b60 con 0x5591986d7010 2024-02-02T15:45:33.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.458+0000 7f42faa00700 1 -- 172.21.15.1:0/1830383215 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5591986cb0f0 con 0x5591986d7010 2024-02-02T15:45:33.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.458+0000 7f42faa00700 1 -- 172.21.15.1:0/1830383215 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 78) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f42fc0bd170 con 0x5591986d7010 2024-02-02T15:45:33.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.458+0000 7f4327895b80 10 client.20031 fetch_fsmap finished waiting for FSMap version 78 2024-02-02T15:45:33.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.458+0000 7f4327895b80 10 client.20031 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-02-02T15:45:33.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.458+0000 7f4327895b80 20 client.20031 populate_metadata read hostname 'smithi001' 2024-02-02T15:45:33.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.458+0000 7f4327895b80 1 --2- 172.21.15.1:0/1830383215 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x5591986f1fc0 0x5591986f43e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:33.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.458+0000 7f4327895b80 4 client.20031 mds_command: new command op to 24502 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:45:33.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.458+0000 7f4327895b80 1 -- 172.21.15.1:0/1830383215 --> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559198621dd0 con 0x5591986f1fc0 2024-02-02T15:45:33.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.458+0000 7f4310a00700 1 --2- 172.21.15.1:0/1830383215 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x5591986f1fc0 0x5591986f43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:33.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.459+0000 7f4310a00700 1 --2- 172.21.15.1:0/1830383215 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x5591986f1fc0 0x5591986f43e0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=65b7ec670524bb5b server_cookie=8b890d04f749ad1e in_seq=0 out_seq=0 2024-02-02T15:45:33.459 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.459+0000 7f42faa00700 10 client.20031 ms_handle_connect on v2:172.21.15.142:6834/1052606547 2024-02-02T15:45:33.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.462+0000 7f42faa00700 1 -- 172.21.15.1:0/1830383215 <== mds.4 v2:172.21.15.142:6834/1052606547 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x559198621dd0 con 0x5591986f1fc0 2024-02-02T15:45:33.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.462+0000 7f42faa00700 10 client.20031 handle_command_reply: tid=0 2024-02-02T15:45:33.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.462+0000 7f4327895b80 1 client.20031 shutdown 2024-02-02T15:45:33.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.463+0000 7f4327895b80 1 -- 172.21.15.1:0/1830383215 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5591986d7010 msgr2=0x5591986e21b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:33.462 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.463+0000 7f4327895b80 1 --2- 172.21.15.1:0/1830383215 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5591986d7010 0x5591986e21b0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f42fc0972a0 tx=0x7f42fc099430 comp rx=0 tx=0).stop 2024-02-02T15:45:33.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.463+0000 7f4327895b80 1 -- 172.21.15.1:0/1830383215 shutdown_connections 2024-02-02T15:45:33.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.463+0000 7f4327895b80 1 --2- 172.21.15.1:0/1830383215 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x5591986f1fc0 0x5591986f43e0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:33.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.463+0000 7f4327895b80 1 --2- 172.21.15.1:0/1830383215 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5591986e27e0 0x5591986dfab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:33.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.463+0000 7f4327895b80 1 --2- 172.21.15.1:0/1830383215 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5591986d7010 0x5591986e21b0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:33.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.463+0000 7f42faa00700 0 client.20031 ms_handle_reset on v2:172.21.15.142:6834/1052606547 2024-02-02T15:45:33.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.463+0000 7f4327895b80 1 --2- 172.21.15.1:0/1830383215 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5591986d7930 0x5591986df430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:33.464 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.463+0000 7f4327895b80 1 -- 172.21.15.1:0/1830383215 >> 172.21.15.1:0/1830383215 conn(0x559198636db0 msgr2=0x5591986d2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:45:33.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.465+0000 7f4327895b80 1 -- 172.21.15.1:0/1830383215 shutdown_connections 2024-02-02T15:45:33.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.465+0000 7f4327895b80 1 -- 172.21.15.1:0/1830383215 wait complete. 2024-02-02T15:45:33.465 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.465+0000 7f4327895b80 20 client.20031 trim_cache size 0 max 16384 2024-02-02T15:45:33.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.486+0000 7f4327895b80 1 Processor -- start 2024-02-02T15:45:33.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.486+0000 7f4327895b80 1 -- start start 2024-02-02T15:45:33.486 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.486+0000 7f4327895b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5591986d7930 0x5591986dfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:33.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.486+0000 7f4327895b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5591986e27e0 0x5591986e9330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:33.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.486+0000 7f4327895b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5591986d8830 0x5591986d1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:33.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.486+0000 7f4327895b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591984ac6a0 con 0x5591986d8830 2024-02-02T15:45:33.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.486+0000 7f4327895b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591986d2090 con 0x5591986d7930 2024-02-02T15:45:33.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.486+0000 7f4327895b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5591986d7d40 con 0x5591986e27e0 2024-02-02T15:45:33.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.487+0000 7f4310a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5591986e27e0 0x5591986e9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:33.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.487+0000 7f4310a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5591986e27e0 0x5591986e9330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:36838/0 (socket says 172.21.15.1:36838) 2024-02-02T15:45:33.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.487+0000 7f4310a00700 1 -- 172.21.15.1:0/3529689627 learned_addr learned my addr 172.21.15.1:0/3529689627 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:45:33.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.487+0000 7f4311e00700 1 --2- 172.21.15.1:0/3529689627 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5591986d8830 0x5591986d1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:33.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.487+0000 7f4311400700 1 --2- 172.21.15.1:0/3529689627 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5591986d7930 0x5591986dfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:33.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.487+0000 7f4310a00700 1 -- 172.21.15.1:0/3529689627 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5591986d7930 msgr2=0x5591986dfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:45:33.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.487+0000 7f4310a00700 1 --2- 172.21.15.1:0/3529689627 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5591986d7930 0x5591986dfa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:33.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.487+0000 7f4310a00700 1 -- 172.21.15.1:0/3529689627 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5591986d8830 msgr2=0x5591986d1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:45:33.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.488+0000 7f4310a00700 1 --2- 172.21.15.1:0/3529689627 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5591986d8830 0x5591986d1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:33.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.488+0000 7f4310a00700 1 -- 172.21.15.1:0/3529689627 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5591985eeea0 con 0x5591986e27e0 2024-02-02T15:45:33.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.488+0000 7f4311e00700 1 --2- 172.21.15.1:0/3529689627 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5591986d8830 0x5591986d1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:45:33.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.488+0000 7f4310a00700 1 --2- 172.21.15.1:0/3529689627 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5591986e27e0 0x5591986e9330 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f42f4005830 tx=0x7f42f4005ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=152d1697319865ab server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:45:33.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.488+0000 7f4303000700 1 -- 172.21.15.1:0/3529689627 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f4005630 con 0x5591986e27e0 2024-02-02T15:45:33.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.488+0000 7f4303000700 1 -- 172.21.15.1:0/3529689627 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f42f401f040 con 0x5591986e27e0 2024-02-02T15:45:33.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.488+0000 7f4303000700 1 -- 172.21.15.1:0/3529689627 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f400f6c0 con 0x5591986e27e0 2024-02-02T15:45:33.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.489+0000 7f4327895b80 1 -- 172.21.15.1:0/3529689627 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5591986e27e0 msgr2=0x5591986e9330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:33.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.489+0000 7f4327895b80 1 --2- 172.21.15.1:0/3529689627 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5591986e27e0 0x5591986e9330 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f42f4005830 tx=0x7f42f4005ea0 comp rx=0 tx=0).stop 2024-02-02T15:45:33.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.489+0000 7f4327895b80 1 -- 172.21.15.1:0/3529689627 shutdown_connections 2024-02-02T15:45:33.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.489+0000 7f4327895b80 1 --2- 172.21.15.1:0/3529689627 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5591986d8830 0x5591986d1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:33.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.489+0000 7f4327895b80 1 --2- 172.21.15.1:0/3529689627 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5591986d7930 0x5591986dfa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:33.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.489+0000 7f4327895b80 1 --2- 172.21.15.1:0/3529689627 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5591986e27e0 0x5591986e9330 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:33.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.489+0000 7f4327895b80 1 -- 172.21.15.1:0/3529689627 >> 172.21.15.1:0/3529689627 conn(0x559198636db0 msgr2=0x5591986cee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:45:33.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.489+0000 7f4327895b80 1 -- 172.21.15.1:0/3529689627 shutdown_connections 2024-02-02T15:45:33.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.489+0000 7f4327895b80 1 -- 172.21.15.1:0/3529689627 wait complete. 2024-02-02T15:45:33.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.490+0000 7f4327895b80 1 -- start start 2024-02-02T15:45:33.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.490+0000 7f4327895b80 1 Processor -- start 2024-02-02T15:45:33.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.491+0000 7f4327895b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5591986d7930 0x5591986e8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:33.492 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.491+0000 7f4327895b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5591986d8830 0x5591986e8c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:33.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.491+0000 7f4327895b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5591986e27e0 0x559198742aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:33.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.491+0000 7f4327895b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x559198742fe0 con 0x5591986d7930 2024-02-02T15:45:33.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.491+0000 7f4327895b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x559198743120 con 0x5591986e27e0 2024-02-02T15:45:33.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.491+0000 7f4327895b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x559198743260 con 0x5591986d8830 2024-02-02T15:45:33.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.491+0000 7f4311e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5591986e27e0 0x559198742aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:33.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.491+0000 7f4310a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5591986d8830 0x5591986e8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:33.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.492+0000 7f4311e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5591986e27e0 0x559198742aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:49148/0 (socket says 172.21.15.1:49148) 2024-02-02T15:45:33.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.492+0000 7f4311e00700 1 -- 172.21.15.1:0/2374588267 learned_addr learned my addr 172.21.15.1:0/2374588267 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:45:33.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.492+0000 7f4327895b80 10 client.0 _ll_register_callbacks cb 0x5591986d4b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:45:33.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.492+0000 7f4311400700 1 --2- 172.21.15.1:0/2374588267 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5591986d7930 0x5591986e8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:33.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.492+0000 7f4310a00700 1 -- 172.21.15.1:0/2374588267 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5591986e27e0 msgr2=0x559198742aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:45:33.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.492+0000 7f4310a00700 1 --2- 172.21.15.1:0/2374588267 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5591986e27e0 0x559198742aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:33.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.492+0000 7f4310a00700 1 -- 172.21.15.1:0/2374588267 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5591986d7930 msgr2=0x5591986e8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:45:33.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.492+0000 7f4310a00700 1 --2- 172.21.15.1:0/2374588267 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5591986d7930 0x5591986e8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:33.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.492+0000 7f4310a00700 1 -- 172.21.15.1:0/2374588267 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4304008d00 con 0x5591986d8830 2024-02-02T15:45:33.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.492+0000 7f4310a00700 1 --2- 172.21.15.1:0/2374588267 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5591986d8830 0x5591986e8c80 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f42f4005830 tx=0x7f42f400bcf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=acdff24a91bae78d server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:45:33.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.492+0000 7f42faa00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:45:33.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.493+0000 7f42faa00700 1 -- 172.21.15.1:0/2374588267 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f4036070 con 0x5591986d8830 2024-02-02T15:45:33.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.493+0000 7f4327895b80 1 -- 172.21.15.1:0/2374588267 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f42fc098cb0 con 0x5591986d8830 2024-02-02T15:45:33.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.493+0000 7f42faa00700 1 -- 172.21.15.1:0/2374588267 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f42f400b8d0 con 0x5591986d8830 2024-02-02T15:45:33.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.493+0000 7f42faa00700 1 -- 172.21.15.1:0/2374588267 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7f42f4028e60 con 0x5591986d8830 2024-02-02T15:45:33.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.493+0000 7f4327895b80 10 client.30187 fetch_fsmap learned FSMap version 78 2024-02-02T15:45:33.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.493+0000 7f4327895b80 1 -- 172.21.15.1:0/2374588267 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) v3 -- 0x7f430c110740 con 0x5591986d8830 2024-02-02T15:45:33.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.497+0000 7f42faa00700 1 -- 172.21.15.1:0/2374588267 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f42fc098cb0 con 0x5591986d8830 2024-02-02T15:45:33.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.497+0000 7f42faa00700 1 -- 172.21.15.1:0/2374588267 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 78) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f42f4039020 con 0x5591986d8830 2024-02-02T15:45:33.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.497+0000 7f4327895b80 10 client.30187 fetch_fsmap finished waiting for FSMap version 78 2024-02-02T15:45:33.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.497+0000 7f4327895b80 10 client.30187 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-02-02T15:45:33.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.497+0000 7f4327895b80 20 client.30187 populate_metadata read hostname 'smithi001' 2024-02-02T15:45:33.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.497+0000 7f4327895b80 1 --2- 172.21.15.1:0/2374588267 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x559198744540 0x559198746960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:33.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.497+0000 7f4327895b80 4 client.30187 mds_command: new command op to 24502 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:45:33.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.497+0000 7f4327895b80 1 -- 172.21.15.1:0/2374588267 --> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5591985e7f50 con 0x559198744540 2024-02-02T15:45:33.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.500+0000 7f4311400700 1 --2- 172.21.15.1:0/2374588267 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x559198744540 0x559198746960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:33.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.501+0000 7f4311400700 1 --2- 172.21.15.1:0/2374588267 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x559198744540 0x559198746960 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2d14b88da26e72f1 server_cookie=c6704652d073e762 in_seq=0 out_seq=0 2024-02-02T15:45:33.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.501+0000 7f42faa00700 10 client.30187 ms_handle_connect on v2:172.21.15.142:6834/1052606547 2024-02-02T15:45:33.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.504+0000 7f42faa00700 1 -- 172.21.15.1:0/2374588267 <== mds.4 v2:172.21.15.142:6834/1052606547 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5591985e7f50 con 0x559198744540 2024-02-02T15:45:33.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.504+0000 7f42faa00700 10 client.30187 handle_command_reply: tid=0 2024-02-02T15:45:33.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.504+0000 7f4327895b80 1 client.30187 shutdown 2024-02-02T15:45:33.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.504+0000 7f4327895b80 1 -- 172.21.15.1:0/2374588267 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5591986d8830 msgr2=0x5591986e8c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:33.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.504+0000 7f4327895b80 1 --2- 172.21.15.1:0/2374588267 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5591986d8830 0x5591986e8c80 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f42f4005830 tx=0x7f42f400bcf0 comp rx=0 tx=0).stop 2024-02-02T15:45:33.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.505+0000 7f4327895b80 1 -- 172.21.15.1:0/2374588267 shutdown_connections 2024-02-02T15:45:33.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.505+0000 7f4327895b80 1 --2- 172.21.15.1:0/2374588267 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x559198744540 0x559198746960 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:33.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.505+0000 7f4327895b80 1 --2- 172.21.15.1:0/2374588267 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5591986e27e0 0x559198742aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:33.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.505+0000 7f4327895b80 1 --2- 172.21.15.1:0/2374588267 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5591986d7930 0x5591986e8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:33.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.505+0000 7f4327895b80 1 --2- 172.21.15.1:0/2374588267 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5591986d8830 0x5591986e8c80 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:33.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.505+0000 7f42faa00700 0 client.30187 ms_handle_reset on v2:172.21.15.142:6834/1052606547 2024-02-02T15:45:33.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.505+0000 7f4327895b80 1 -- 172.21.15.1:0/2374588267 >> 172.21.15.1:0/2374588267 conn(0x559198636db0 msgr2=0x5591986da7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:45:33.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.505+0000 7f4327895b80 1 -- 172.21.15.1:0/2374588267 shutdown_connections 2024-02-02T15:45:33.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.505+0000 7f4327895b80 1 -- 172.21.15.1:0/2374588267 wait complete. 2024-02-02T15:45:33.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.505+0000 7f4327895b80 20 client.30187 trim_cache size 0 max 16384 2024-02-02T15:45:33.506 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:45:33.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.506+0000 7f4313e00700 1 -- 172.21.15.1:0/3922168735 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f42ec0a0710 msgr2=0x7f42ec0a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:33.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.506+0000 7f4313e00700 1 --2- 172.21.15.1:0/3922168735 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f42ec0a0710 0x7f42ec0a2bd0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f4304009690 tx=0x7f43040095e0 comp rx=0 tx=0).stop 2024-02-02T15:45:33.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.506+0000 7f4313e00700 1 -- 172.21.15.1:0/3922168735 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f430c103230 msgr2=0x7f430c10f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:33.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.506+0000 7f4313e00700 1 --2- 172.21.15.1:0/3922168735 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f430c103230 0x7f430c10f9a0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f42fc00b530 tx=0x7f42fc00b8f0 comp rx=0 tx=0).stop 2024-02-02T15:45:33.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.506+0000 7f4313e00700 1 -- 172.21.15.1:0/3922168735 shutdown_connections 2024-02-02T15:45:33.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.506+0000 7f4313e00700 1 --2- 172.21.15.1:0/3922168735 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f42ec0a0710 0x7f42ec0a2bd0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:33.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.506+0000 7f4313e00700 1 --2- 172.21.15.1:0/3922168735 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f430c10cef0 0x7f430c10d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:33.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.506+0000 7f4313e00700 1 --2- 172.21.15.1:0/3922168735 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f430c103230 0x7f430c10f9a0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:33.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.506+0000 7f4313e00700 1 --2- 172.21.15.1:0/3922168735 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f430c108350 0x7f430c10c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:33.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.506+0000 7f4313e00700 1 -- 172.21.15.1:0/3922168735 >> 172.21.15.1:0/3922168735 conn(0x7f430c0fea70 msgr2=0x7f430c106ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:45:33.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.506+0000 7f4313e00700 1 -- 172.21.15.1:0/3922168735 shutdown_connections 2024-02-02T15:45:33.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.506+0000 7f4313e00700 1 -- 172.21.15.1:0/3922168735 wait complete. 2024-02-02T15:45:33.531 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-02-02T15:45:33.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.826+0000 7fbedd400700 1 -- 172.21.15.1:0/764465282 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbed8108340 msgr2=0x7fbed810fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:33.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.826+0000 7fbedd400700 1 --2- 172.21.15.1:0/764465282 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbed8108340 0x7fbed810fec0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fbed0009f90 tx=0x7fbed000b330 comp rx=0 tx=0).stop 2024-02-02T15:45:33.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.826+0000 7fbedd400700 1 -- 172.21.15.1:0/764465282 shutdown_connections 2024-02-02T15:45:33.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.826+0000 7fbedd400700 1 --2- 172.21.15.1:0/764465282 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbed8108340 0x7fbed810fec0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:33.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.826+0000 7fbedd400700 1 --2- 172.21.15.1:0/764465282 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbed8103220 0x7fbed8103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:33.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.826+0000 7fbedd400700 1 --2- 172.21.15.1:0/764465282 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbed8103bd0 0x7fbed8107c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:33.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.826+0000 7fbedd400700 1 -- 172.21.15.1:0/764465282 >> 172.21.15.1:0/764465282 conn(0x7fbed80fea60 msgr2=0x7fbed8100e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:45:33.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.827+0000 7fbedd400700 1 -- 172.21.15.1:0/764465282 shutdown_connections 2024-02-02T15:45:33.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.827+0000 7fbedd400700 1 -- 172.21.15.1:0/764465282 wait complete. 2024-02-02T15:45:33.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.828+0000 7fbedd400700 1 Processor -- start 2024-02-02T15:45:33.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.828+0000 7fbedd400700 1 -- start start 2024-02-02T15:45:33.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.830+0000 7fbedd400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbed8103220 0x7fbed8199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:33.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.830+0000 7fbedd400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbed8108340 0x7fbed8199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:33.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.830+0000 7fbedd400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbed819daa0 0x7fbed819df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:33.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.830+0000 7fbedd400700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed819e520 con 0x7fbed8108340 2024-02-02T15:45:33.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.830+0000 7fbedd400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed819e690 con 0x7fbed8103220 2024-02-02T15:45:33.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.830+0000 7fbedd400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed819e850 con 0x7fbed819daa0 2024-02-02T15:45:33.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.831+0000 7fbed6800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbed8103220 0x7fbed8199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:33.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.831+0000 7fbed6800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbed8103220 0x7fbed8199100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:49156/0 (socket says 172.21.15.1:49156) 2024-02-02T15:45:33.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.831+0000 7fbed6800700 1 -- 172.21.15.1:0/432323448 learned_addr learned my addr 172.21.15.1:0/432323448 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:45:33.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.831+0000 7fbed7200700 1 --2- 172.21.15.1:0/432323448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbed819daa0 0x7fbed819df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:33.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.831+0000 7fbed5e00700 1 --2- 172.21.15.1:0/432323448 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbed8108340 0x7fbed8199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:33.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.831+0000 7fbed7200700 1 -- 172.21.15.1:0/432323448 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbed8103220 msgr2=0x7fbed8199100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:33.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.831+0000 7fbed7200700 1 --2- 172.21.15.1:0/432323448 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbed8103220 0x7fbed8199100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:33.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.831+0000 7fbed7200700 1 -- 172.21.15.1:0/432323448 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbed8108340 msgr2=0x7fbed8199640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:33.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.831+0000 7fbed7200700 1 --2- 172.21.15.1:0/432323448 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbed8108340 0x7fbed8199640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:33.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.831+0000 7fbed7200700 1 -- 172.21.15.1:0/432323448 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbed0009c40 con 0x7fbed819daa0 2024-02-02T15:45:33.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.832+0000 7fbed7200700 1 --2- 172.21.15.1:0/432323448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbed819daa0 0x7fbed819df20 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fbed000b770 tx=0x7fbed0007730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:45:33.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.832+0000 7fbecf400700 1 -- 172.21.15.1:0/432323448 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed000e040 con 0x7fbed819daa0 2024-02-02T15:45:33.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.832+0000 7fbecf400700 1 -- 172.21.15.1:0/432323448 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbed000be50 con 0x7fbed819daa0 2024-02-02T15:45:33.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.832+0000 7fbecf400700 1 -- 172.21.15.1:0/432323448 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed00052a0 con 0x7fbed819daa0 2024-02-02T15:45:33.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.832+0000 7fbedd400700 1 -- 172.21.15.1:0/432323448 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbed81a5d50 con 0x7fbed819daa0 2024-02-02T15:45:33.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.832+0000 7fbedd400700 1 -- 172.21.15.1:0/432323448 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbed81a6180 con 0x7fbed819daa0 2024-02-02T15:45:33.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.836+0000 7fbecf400700 1 -- 172.21.15.1:0/432323448 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7fbed001d470 con 0x7fbed819daa0 2024-02-02T15:45:33.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.839+0000 7fbef1012b80 1 Processor -- start 2024-02-02T15:45:33.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.840+0000 7fbef1012b80 1 -- start start 2024-02-02T15:45:33.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.841+0000 7fbef1012b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5647a6d3b010 0x5647a6d3b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:33.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.841+0000 7fbef1012b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5647a6d3b930 0x5647a6d462a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:33.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.841+0000 7fbecf400700 1 --2- 172.21.15.1:0/432323448 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fbeb80a0750 0x7fbeb80a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:33.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.841+0000 7fbef1012b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647a6d467e0 0x5647a6d48bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:33.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.841+0000 7fbef1012b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647a6caf4c0 con 0x5647a6d3b010 2024-02-02T15:45:33.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.841+0000 7fbef1012b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647a6c93aa0 con 0x5647a6d3b930 2024-02-02T15:45:33.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.841+0000 7fbef1012b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647a6c91460 con 0x5647a6d467e0 2024-02-02T15:45:33.842 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.841+0000 7fbecf400700 1 -- 172.21.15.1:0/432323448 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fbed00cd770 con 0x7fbed819daa0 2024-02-02T15:45:33.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.842+0000 7fbed6800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5647a6d3b930 0x5647a6d462a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:33.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.842+0000 7fbed7200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647a6d467e0 0x5647a6d48bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:33.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.842+0000 7fbed6800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5647a6d3b930 0x5647a6d462a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:49160/0 (socket says 172.21.15.1:49160) 2024-02-02T15:45:33.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.842+0000 7fbed6800700 1 -- 172.21.15.1:0/1935113287 learned_addr learned my addr 172.21.15.1:0/1935113287 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:45:33.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.842+0000 7fbed5e00700 1 --2- 172.21.15.1:0/1935113287 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5647a6d3b010 0x5647a6d3b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:33.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.842+0000 7fbed7200700 1 -- 172.21.15.1:0/1935113287 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5647a6d3b930 msgr2=0x5647a6d462a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:45:33.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.842+0000 7fbed7200700 1 --2- 172.21.15.1:0/1935113287 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5647a6d3b930 0x5647a6d462a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:33.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.842+0000 7fbed5e00700 1 --2- 172.21.15.1:0/432323448 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fbeb80a0750 0x7fbeb80a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:33.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.842+0000 7fbed7200700 1 -- 172.21.15.1:0/1935113287 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5647a6d3b010 msgr2=0x5647a6d3b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:45:33.843 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.842+0000 7fbed7200700 1 --2- 172.21.15.1:0/1935113287 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5647a6d3b010 0x5647a6d3b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:33.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.842+0000 7fbed7200700 1 -- 172.21.15.1:0/1935113287 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5647a6b0e3b0 con 0x5647a6d467e0 2024-02-02T15:45:33.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.842+0000 7fbed7200700 1 --2- 172.21.15.1:0/1935113287 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647a6d467e0 0x5647a6d48bd0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fbed00caa90 tx=0x7fbed0009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb6ff6ad199b362c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:45:33.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.843+0000 7fbec7400700 1 -- 172.21.15.1:0/1935113287 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed009b020 con 0x5647a6d467e0 2024-02-02T15:45:33.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.843+0000 7fbec7400700 1 -- 172.21.15.1:0/1935113287 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbed009a920 con 0x5647a6d467e0 2024-02-02T15:45:33.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.843+0000 7fbec7400700 1 -- 172.21.15.1:0/1935113287 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbed00cd310 con 0x5647a6d467e0 2024-02-02T15:45:33.844 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.844+0000 7fbef1012b80 1 -- 172.21.15.1:0/1935113287 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647a6d467e0 msgr2=0x5647a6d48bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:33.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.844+0000 7fbed5e00700 1 --2- 172.21.15.1:0/432323448 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fbeb80a0750 0x7fbeb80a2c10 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fbed819a710 tx=0x7fbec0009650 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:45:33.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.844+0000 7fbef1012b80 1 --2- 172.21.15.1:0/1935113287 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647a6d467e0 0x5647a6d48bd0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fbed00caa90 tx=0x7fbed0009920 comp rx=0 tx=0).stop 2024-02-02T15:45:33.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.844+0000 7fbef1012b80 1 -- 172.21.15.1:0/1935113287 shutdown_connections 2024-02-02T15:45:33.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.844+0000 7fbef1012b80 1 --2- 172.21.15.1:0/1935113287 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647a6d467e0 0x5647a6d48bd0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:33.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.844+0000 7fbef1012b80 1 --2- 172.21.15.1:0/1935113287 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5647a6d3b010 0x5647a6d3b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:33.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.844+0000 7fbef1012b80 1 --2- 172.21.15.1:0/1935113287 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5647a6d3b930 0x5647a6d462a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:33.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.844+0000 7fbef1012b80 1 -- 172.21.15.1:0/1935113287 >> 172.21.15.1:0/1935113287 conn(0x5647a6c9adb0 msgr2=0x5647a6d373b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:45:33.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.844+0000 7fbef1012b80 1 -- 172.21.15.1:0/1935113287 shutdown_connections 2024-02-02T15:45:33.845 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.845+0000 7fbef1012b80 1 -- 172.21.15.1:0/1935113287 wait complete. 2024-02-02T15:45:33.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.845+0000 7fbef1012b80 1 -- start start 2024-02-02T15:45:33.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.845+0000 7fbef1012b80 1 Processor -- start 2024-02-02T15:45:33.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.846+0000 7fbef1012b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5647a6d3b010 0x5647a6d461c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:33.846 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.846+0000 7fbef1012b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647a6d3b930 0x5647a6d43470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:33.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.846+0000 7fbef1012b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5647a6d467e0 0x5647a6d43a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:33.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.846+0000 7fbef1012b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647a6c8cd50 con 0x5647a6d467e0 2024-02-02T15:45:33.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.846+0000 7fbef1012b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647a6c28d80 con 0x5647a6d3b010 2024-02-02T15:45:33.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.846+0000 7fbef1012b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647a6c28a20 con 0x5647a6d3b930 2024-02-02T15:45:33.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.846+0000 7fbef1012b80 10 client.0 _ll_register_callbacks cb 0x5647a6c2a290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:45:33.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.847+0000 7fbed6800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5647a6d3b010 0x5647a6d461c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:33.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.847+0000 7fbed5e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647a6d3b930 0x5647a6d43470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:33.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.847+0000 7fbed6800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5647a6d3b010 0x5647a6d461c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:49162/0 (socket says 172.21.15.1:49162) 2024-02-02T15:45:33.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.847+0000 7fbed6800700 1 -- 172.21.15.1:0/3093898677 learned_addr learned my addr 172.21.15.1:0/3093898677 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:45:33.847 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.847+0000 7fbed7200700 1 --2- 172.21.15.1:0/3093898677 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5647a6d467e0 0x5647a6d43a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:33.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.847+0000 7fbed5e00700 1 -- 172.21.15.1:0/3093898677 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5647a6d3b010 msgr2=0x5647a6d461c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:45:33.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.847+0000 7fbed5e00700 1 --2- 172.21.15.1:0/3093898677 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5647a6d3b010 0x5647a6d461c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:33.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.847+0000 7fbed5e00700 1 -- 172.21.15.1:0/3093898677 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5647a6d467e0 msgr2=0x5647a6d43a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:45:33.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.847+0000 7fbed5e00700 1 --2- 172.21.15.1:0/3093898677 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5647a6d467e0 0x5647a6d43a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:33.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.847+0000 7fbed5e00700 1 -- 172.21.15.1:0/3093898677 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbed81a6180 con 0x5647a6d3b930 2024-02-02T15:45:33.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.847+0000 7fbed5e00700 1 --2- 172.21.15.1:0/3093898677 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647a6d3b930 0x5647a6d43470 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fbed8068d60 tx=0x7fbec0009520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f859e73ecb55e618 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:45:33.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.847+0000 7fbebfe00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:45:33.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.848+0000 7fbebfe00700 1 -- 172.21.15.1:0/3093898677 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec0004020 con 0x5647a6d3b930 2024-02-02T15:45:33.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.848+0000 7fbef1012b80 1 -- 172.21.15.1:0/3093898677 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5647a6d2f0f0 con 0x5647a6d3b930 2024-02-02T15:45:33.849 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.848+0000 7fbebfe00700 1 -- 172.21.15.1:0/3093898677 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fbec0004180 con 0x5647a6d3b930 2024-02-02T15:45:33.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.848+0000 7fbebfe00700 1 -- 172.21.15.1:0/3093898677 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbec0004340 con 0x5647a6d3b930 2024-02-02T15:45:33.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.848+0000 7fbef1012b80 10 client.30199 fetch_fsmap learned FSMap version 78 2024-02-02T15:45:33.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.848+0000 7fbef1012b80 1 -- 172.21.15.1:0/3093898677 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) v3 -- 0x5647a6abab60 con 0x5647a6d3b930 2024-02-02T15:45:33.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.848+0000 7fbebfe00700 1 -- 172.21.15.1:0/3093898677 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5647a6d2f0f0 con 0x5647a6d3b930 2024-02-02T15:45:33.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.849+0000 7fbebfe00700 1 -- 172.21.15.1:0/3093898677 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 78) v1 ==== 7506+0+0 (secure 0 0 0) 0x7fbec00214b0 con 0x5647a6d3b930 2024-02-02T15:45:33.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.849+0000 7fbef1012b80 10 client.30199 fetch_fsmap finished waiting for FSMap version 78 2024-02-02T15:45:33.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.849+0000 7fbef1012b80 10 client.30199 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-02-02T15:45:33.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.849+0000 7fbef1012b80 20 client.30199 populate_metadata read hostname 'smithi001' 2024-02-02T15:45:33.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.849+0000 7fbef1012b80 1 --2- 172.21.15.1:0/3093898677 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x5647a6d5e820 0x5647a6d60c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:33.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.849+0000 7fbef1012b80 4 client.30199 mds_command: new command op to 24514 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:45:33.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.849+0000 7fbef1012b80 1 -- 172.21.15.1:0/3093898677 --> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5647a6c85dd0 con 0x5647a6d5e820 2024-02-02T15:45:33.850 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.849+0000 7fbed6800700 1 --2- 172.21.15.1:0/3093898677 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x5647a6d5e820 0x5647a6d60c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:33.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.850+0000 7fbed6800700 1 --2- 172.21.15.1:0/3093898677 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x5647a6d5e820 0x5647a6d60c40 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=81b299a86243b324 server_cookie=cc94c6434478fac in_seq=0 out_seq=0 2024-02-02T15:45:33.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.850+0000 7fbebfe00700 10 client.30199 ms_handle_connect on v2:172.21.15.142:6836/2870289019 2024-02-02T15:45:33.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.850+0000 7fbebfe00700 1 -- 172.21.15.1:0/3093898677 <== mds.3 v2:172.21.15.142:6836/2870289019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5647a6c85dd0 con 0x5647a6d5e820 2024-02-02T15:45:33.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.850+0000 7fbebfe00700 10 client.30199 handle_command_reply: tid=0 2024-02-02T15:45:33.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.851+0000 7fbef1012b80 1 client.30199 shutdown 2024-02-02T15:45:33.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.851+0000 7fbef1012b80 1 -- 172.21.15.1:0/3093898677 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647a6d3b930 msgr2=0x5647a6d43470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:33.851 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.851+0000 7fbef1012b80 1 --2- 172.21.15.1:0/3093898677 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647a6d3b930 0x5647a6d43470 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fbed8068d60 tx=0x7fbec0009520 comp rx=0 tx=0).stop 2024-02-02T15:45:33.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.852+0000 7fbef1012b80 1 -- 172.21.15.1:0/3093898677 shutdown_connections 2024-02-02T15:45:33.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.852+0000 7fbef1012b80 1 --2- 172.21.15.1:0/3093898677 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x5647a6d5e820 0x5647a6d60c40 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:33.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.852+0000 7fbef1012b80 1 --2- 172.21.15.1:0/3093898677 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5647a6d467e0 0x5647a6d43a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:33.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.852+0000 7fbef1012b80 1 --2- 172.21.15.1:0/3093898677 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5647a6d3b010 0x5647a6d461c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:33.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.852+0000 7fbef1012b80 1 --2- 172.21.15.1:0/3093898677 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647a6d3b930 0x5647a6d43470 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:33.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.852+0000 7fbebfe00700 0 client.30199 ms_handle_reset on v2:172.21.15.142:6836/2870289019 2024-02-02T15:45:33.852 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.852+0000 7fbef1012b80 1 -- 172.21.15.1:0/3093898677 >> 172.21.15.1:0/3093898677 conn(0x5647a6c9adb0 msgr2=0x5647a6d36f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:45:33.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.852+0000 7fbef1012b80 1 -- 172.21.15.1:0/3093898677 shutdown_connections 2024-02-02T15:45:33.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.852+0000 7fbef1012b80 1 -- 172.21.15.1:0/3093898677 wait complete. 2024-02-02T15:45:33.853 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.852+0000 7fbef1012b80 20 client.30199 trim_cache size 0 max 16384 2024-02-02T15:45:33.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.871+0000 7fbef1012b80 1 Processor -- start 2024-02-02T15:45:33.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.872+0000 7fbef1012b80 1 -- start start 2024-02-02T15:45:33.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.872+0000 7fbef1012b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5647a6d3b010 0x5647a6d55950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:33.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.872+0000 7fbef1012b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647a6d3b930 0x5647a6d3df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:33.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.872+0000 7fbef1012b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5647a6d467e0 0x5647a6d45c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:33.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.872+0000 7fbef1012b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647a6b106a0 con 0x5647a6d467e0 2024-02-02T15:45:33.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.872+0000 7fbef1012b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647a6d5b120 con 0x5647a6d3b010 2024-02-02T15:45:33.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.872+0000 7fbef1012b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647a6d31500 con 0x5647a6d3b930 2024-02-02T15:45:33.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.872+0000 7fbed6800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5647a6d3b010 0x5647a6d55950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:33.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.872+0000 7fbed6800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5647a6d3b010 0x5647a6d55950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:49164/0 (socket says 172.21.15.1:49164) 2024-02-02T15:45:33.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.872+0000 7fbed6800700 1 -- 172.21.15.1:0/251940247 learned_addr learned my addr 172.21.15.1:0/251940247 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:45:33.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.872+0000 7fbed5e00700 1 --2- 172.21.15.1:0/251940247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647a6d3b930 0x5647a6d3df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:33.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.873+0000 7fbed7200700 1 --2- 172.21.15.1:0/251940247 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5647a6d467e0 0x5647a6d45c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:33.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.873+0000 7fbed6800700 1 -- 172.21.15.1:0/251940247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647a6d3b930 msgr2=0x5647a6d3df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:45:33.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.873+0000 7fbed6800700 1 --2- 172.21.15.1:0/251940247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647a6d3b930 0x5647a6d3df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:33.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.873+0000 7fbed6800700 1 -- 172.21.15.1:0/251940247 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5647a6d467e0 msgr2=0x5647a6d45c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:45:33.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.873+0000 7fbed6800700 1 --2- 172.21.15.1:0/251940247 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5647a6d467e0 0x5647a6d45c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:33.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.873+0000 7fbed6800700 1 -- 172.21.15.1:0/251940247 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5647a6d330d0 con 0x5647a6d3b010 2024-02-02T15:45:33.876 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.877+0000 7fbed6800700 1 --2- 172.21.15.1:0/251940247 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5647a6d3b010 0x5647a6d55950 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fbec8004310 tx=0x7fbec8004620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22cf1ca68b8822a2 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:45:33.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.877+0000 7fbec7e00700 1 -- 172.21.15.1:0/251940247 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec8017040 con 0x5647a6d3b010 2024-02-02T15:45:33.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.877+0000 7fbec7e00700 1 -- 172.21.15.1:0/251940247 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbec8019040 con 0x5647a6d3b010 2024-02-02T15:45:33.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.877+0000 7fbec7e00700 1 -- 172.21.15.1:0/251940247 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec8012710 con 0x5647a6d3b010 2024-02-02T15:45:33.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.878+0000 7fbef1012b80 1 -- 172.21.15.1:0/251940247 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5647a6d3b010 msgr2=0x5647a6d55950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:33.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.878+0000 7fbef1012b80 1 --2- 172.21.15.1:0/251940247 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5647a6d3b010 0x5647a6d55950 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fbec8004310 tx=0x7fbec8004620 comp rx=0 tx=0).stop 2024-02-02T15:45:33.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.878+0000 7fbef1012b80 1 -- 172.21.15.1:0/251940247 shutdown_connections 2024-02-02T15:45:33.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.878+0000 7fbef1012b80 1 --2- 172.21.15.1:0/251940247 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5647a6d467e0 0x5647a6d45c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:33.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.878+0000 7fbef1012b80 1 --2- 172.21.15.1:0/251940247 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5647a6d3b010 0x5647a6d55950 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:33.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.878+0000 7fbef1012b80 1 --2- 172.21.15.1:0/251940247 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647a6d3b930 0x5647a6d3df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:33.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.878+0000 7fbef1012b80 1 -- 172.21.15.1:0/251940247 >> 172.21.15.1:0/251940247 conn(0x5647a6c9adb0 msgr2=0x5647a6d3e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:45:33.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.878+0000 7fbef1012b80 1 -- 172.21.15.1:0/251940247 shutdown_connections 2024-02-02T15:45:33.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.879+0000 7fbef1012b80 1 -- 172.21.15.1:0/251940247 wait complete. 2024-02-02T15:45:33.879 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.879+0000 7fbef1012b80 1 -- start start 2024-02-02T15:45:33.881 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.882+0000 7fbef1012b80 1 Processor -- start 2024-02-02T15:45:33.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.883+0000 7fbef1012b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5647a6d3b010 0x5647a6d53190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:33.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.883+0000 7fbef1012b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5647a6d3b930 0x5647a6d536d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:33.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.883+0000 7fbef1012b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647a6d467e0 0x5647a6daf300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:33.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.883+0000 7fbef1012b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647a6d53fe0 con 0x5647a6d3b930 2024-02-02T15:45:33.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.883+0000 7fbef1012b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647a6daf840 con 0x5647a6d3b010 2024-02-02T15:45:33.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.883+0000 7fbef1012b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647a6daf980 con 0x5647a6d467e0 2024-02-02T15:45:33.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.883+0000 7fbef1012b80 10 client.0 _ll_register_callbacks cb 0x5647a6c2a290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:45:33.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.883+0000 7fbed6800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5647a6d3b010 0x5647a6d53190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:33.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.883+0000 7fbed6800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5647a6d3b010 0x5647a6d53190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:49174/0 (socket says 172.21.15.1:49174) 2024-02-02T15:45:33.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.883+0000 7fbed6800700 1 -- 172.21.15.1:0/1520167936 learned_addr learned my addr 172.21.15.1:0/1520167936 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:45:33.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.883+0000 7fbed7200700 1 --2- 172.21.15.1:0/1520167936 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647a6d467e0 0x5647a6daf300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:33.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.883+0000 7fbed5e00700 1 --2- 172.21.15.1:0/1520167936 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5647a6d3b930 0x5647a6d536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:33.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.884+0000 7fbed6800700 1 -- 172.21.15.1:0/1520167936 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647a6d467e0 msgr2=0x5647a6daf300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:45:33.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.884+0000 7fbed6800700 1 --2- 172.21.15.1:0/1520167936 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647a6d467e0 0x5647a6daf300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:33.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.884+0000 7fbed6800700 1 -- 172.21.15.1:0/1520167936 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5647a6d3b930 msgr2=0x5647a6d536d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:45:33.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.884+0000 7fbed6800700 1 --2- 172.21.15.1:0/1520167936 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5647a6d3b930 0x5647a6d536d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:33.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.884+0000 7fbed6800700 1 -- 172.21.15.1:0/1520167936 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbec8004040 con 0x5647a6d3b010 2024-02-02T15:45:33.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.884+0000 7fbed6800700 1 --2- 172.21.15.1:0/1520167936 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5647a6d3b010 0x5647a6d53190 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fbec8003db0 tx=0x7fbec8003de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15c13154a616d7e4 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:45:33.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.884+0000 7fbebfe00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:45:33.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.884+0000 7fbebfe00700 1 -- 172.21.15.1:0/1520167936 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec8017030 con 0x5647a6d3b010 2024-02-02T15:45:33.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.884+0000 7fbebfe00700 1 -- 172.21.15.1:0/1520167936 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fbec801e790 con 0x5647a6d3b010 2024-02-02T15:45:33.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.884+0000 7fbef1012b80 1 -- 172.21.15.1:0/1520167936 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbed009a920 con 0x5647a6d3b010 2024-02-02T15:45:33.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.885+0000 7fbebfe00700 1 -- 172.21.15.1:0/1520167936 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbec802c070 con 0x5647a6d3b010 2024-02-02T15:45:33.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.885+0000 7fbef1012b80 10 client.30353 fetch_fsmap learned FSMap version 78 2024-02-02T15:45:33.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.885+0000 7fbef1012b80 1 -- 172.21.15.1:0/1520167936 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) v3 -- 0x5647a6c52ea0 con 0x5647a6d3b010 2024-02-02T15:45:33.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.885+0000 7fbebfe00700 1 -- 172.21.15.1:0/1520167936 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbed009a920 con 0x5647a6d3b010 2024-02-02T15:45:33.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.885+0000 7fbebfe00700 1 -- 172.21.15.1:0/1520167936 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 78) v1 ==== 7506+0+0 (secure 0 0 0) 0x7fbec8004790 con 0x5647a6d3b010 2024-02-02T15:45:33.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.885+0000 7fbef1012b80 10 client.30353 fetch_fsmap finished waiting for FSMap version 78 2024-02-02T15:45:33.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.885+0000 7fbef1012b80 10 client.30353 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-02-02T15:45:33.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.885+0000 7fbef1012b80 20 client.30353 populate_metadata read hostname 'smithi001' 2024-02-02T15:45:33.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.885+0000 7fbef1012b80 1 --2- 172.21.15.1:0/1520167936 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x5647a6db1010 0x5647a6db3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:33.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.885+0000 7fbef1012b80 4 client.30353 mds_command: new command op to 24514 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:45:33.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.885+0000 7fbef1012b80 1 -- 172.21.15.1:0/1520167936 --> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5647a6c4bf50 con 0x5647a6db1010 2024-02-02T15:45:33.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.885+0000 7fbed5e00700 1 --2- 172.21.15.1:0/1520167936 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x5647a6db1010 0x5647a6db3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:33.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.886+0000 7fbed5e00700 1 --2- 172.21.15.1:0/1520167936 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x5647a6db1010 0x5647a6db3430 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b528f9c9d5ad5968 server_cookie=2b761989ff35afc5 in_seq=0 out_seq=0 2024-02-02T15:45:33.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.886+0000 7fbebfe00700 10 client.30353 ms_handle_connect on v2:172.21.15.142:6836/2870289019 2024-02-02T15:45:33.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.886+0000 7fbebfe00700 1 -- 172.21.15.1:0/1520167936 <== mds.3 v2:172.21.15.142:6836/2870289019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5647a6c4bf50 con 0x5647a6db1010 2024-02-02T15:45:33.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.886+0000 7fbebfe00700 10 client.30353 handle_command_reply: tid=0 2024-02-02T15:45:33.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.886+0000 7fbef1012b80 1 client.30353 shutdown 2024-02-02T15:45:33.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.887+0000 7fbef1012b80 1 -- 172.21.15.1:0/1520167936 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5647a6d3b010 msgr2=0x5647a6d53190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:33.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.887+0000 7fbef1012b80 1 --2- 172.21.15.1:0/1520167936 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5647a6d3b010 0x5647a6d53190 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fbec8003db0 tx=0x7fbec8003de0 comp rx=0 tx=0).stop 2024-02-02T15:45:33.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.887+0000 7fbef1012b80 1 -- 172.21.15.1:0/1520167936 shutdown_connections 2024-02-02T15:45:33.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.887+0000 7fbef1012b80 1 --2- 172.21.15.1:0/1520167936 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5647a6d467e0 0x5647a6daf300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:33.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.887+0000 7fbef1012b80 1 --2- 172.21.15.1:0/1520167936 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5647a6d3b010 0x5647a6d53190 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:33.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.887+0000 7fbef1012b80 1 --2- 172.21.15.1:0/1520167936 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x5647a6db1010 0x5647a6db3430 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:33.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.887+0000 7fbef1012b80 1 --2- 172.21.15.1:0/1520167936 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5647a6d3b930 0x5647a6d536d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:33.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.887+0000 7fbef1012b80 1 -- 172.21.15.1:0/1520167936 >> 172.21.15.1:0/1520167936 conn(0x5647a6c9adb0 msgr2=0x5647a6d36e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:45:33.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.887+0000 7fbebfe00700 0 client.30353 ms_handle_reset on v2:172.21.15.142:6836/2870289019 2024-02-02T15:45:33.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.887+0000 7fbef1012b80 1 -- 172.21.15.1:0/1520167936 shutdown_connections 2024-02-02T15:45:33.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.887+0000 7fbef1012b80 1 -- 172.21.15.1:0/1520167936 wait complete. 2024-02-02T15:45:33.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.888+0000 7fbef1012b80 20 client.30353 trim_cache size 0 max 16384 2024-02-02T15:45:33.890 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:45:33.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.889+0000 7fbedd400700 1 -- 172.21.15.1:0/432323448 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fbeb80a0750 msgr2=0x7fbeb80a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:33.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.889+0000 7fbedd400700 1 --2- 172.21.15.1:0/432323448 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fbeb80a0750 0x7fbeb80a2c10 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fbed819a710 tx=0x7fbec0009650 comp rx=0 tx=0).stop 2024-02-02T15:45:33.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.889+0000 7fbedd400700 1 -- 172.21.15.1:0/432323448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbed819daa0 msgr2=0x7fbed819df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:33.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.889+0000 7fbedd400700 1 --2- 172.21.15.1:0/432323448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbed819daa0 0x7fbed819df20 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fbed000b770 tx=0x7fbed0007730 comp rx=0 tx=0).stop 2024-02-02T15:45:33.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.889+0000 7fbedd400700 1 -- 172.21.15.1:0/432323448 shutdown_connections 2024-02-02T15:45:33.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.889+0000 7fbedd400700 1 --2- 172.21.15.1:0/432323448 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fbeb80a0750 0x7fbeb80a2c10 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:33.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.889+0000 7fbedd400700 1 --2- 172.21.15.1:0/432323448 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbed819daa0 0x7fbed819df20 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:33.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.889+0000 7fbedd400700 1 --2- 172.21.15.1:0/432323448 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbed8103220 0x7fbed8199100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:33.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.889+0000 7fbedd400700 1 --2- 172.21.15.1:0/432323448 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbed8108340 0x7fbed8199640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:33.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.889+0000 7fbedd400700 1 -- 172.21.15.1:0/432323448 >> 172.21.15.1:0/432323448 conn(0x7fbed80fea60 msgr2=0x7fbed80ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:45:33.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.889+0000 7fbedd400700 1 -- 172.21.15.1:0/432323448 shutdown_connections 2024-02-02T15:45:33.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:33.889+0000 7fbedd400700 1 -- 172.21.15.1:0/432323448 wait complete. 2024-02-02T15:45:33.911 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:33 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/3359530732' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:45:33.911 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:33 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:45:33.911 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:33 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:45:33.911 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:33 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:45:33.913 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-02T15:45:34.006 INFO:tasks.workunit.client.0.smithi001.stdout: 10 164702 27.57 MB/sec execute 186 sec latency 58.830 ms 2024-02-02T15:45:34.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:33 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/3359530732' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:45:34.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:33 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:45:34.116 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:33 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:45:34.116 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:33 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:45:34.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:33 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/3359530732' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:45:34.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:33 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:45:34.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:33 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:45:34.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:33 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:45:34.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.236+0000 7fdf17e00700 1 -- 172.21.15.1:0/2885498607 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf101000e0 msgr2=0x7fdf101004c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:34.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.236+0000 7fdf17e00700 1 --2- 172.21.15.1:0/2885498607 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf101000e0 0x7fdf101004c0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fdf00009990 tx=0x7fdf00009ca0 comp rx=0 tx=0).stop 2024-02-02T15:45:34.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.236+0000 7fdf17e00700 1 -- 172.21.15.1:0/2885498607 shutdown_connections 2024-02-02T15:45:34.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.236+0000 7fdf17e00700 1 --2- 172.21.15.1:0/2885498607 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf10105020 0x7fdf1010cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:34.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.236+0000 7fdf17e00700 1 --2- 172.21.15.1:0/2885498607 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf101000e0 0x7fdf101004c0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:34.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.236+0000 7fdf17e00700 1 --2- 172.21.15.1:0/2885498607 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf10100a90 0x7fdf101048f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:34.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.236+0000 7fdf17e00700 1 -- 172.21.15.1:0/2885498607 >> 172.21.15.1:0/2885498607 conn(0x7fdf100757d0 msgr2=0x7fdf10075be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:45:34.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.236+0000 7fdf17e00700 1 -- 172.21.15.1:0/2885498607 shutdown_connections 2024-02-02T15:45:34.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.237+0000 7fdf17e00700 1 -- 172.21.15.1:0/2885498607 wait complete. 2024-02-02T15:45:34.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.238+0000 7fdf17e00700 1 Processor -- start 2024-02-02T15:45:34.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.238+0000 7fdf17e00700 1 -- start start 2024-02-02T15:45:34.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.238+0000 7fdf17e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf101000e0 0x7fdf10196eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:34.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.238+0000 7fdf17e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf10105020 0x7fdf101973f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:34.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.238+0000 7fdf17e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf1019b850 0x7fdf1019bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:34.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.238+0000 7fdf17e00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1019c2d0 con 0x7fdf1019b850 2024-02-02T15:45:34.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.238+0000 7fdf17e00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1019c440 con 0x7fdf10105020 2024-02-02T15:45:34.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.238+0000 7fdf17e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1019c600 con 0x7fdf101000e0 2024-02-02T15:45:34.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.239+0000 7fdf15e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf1019b850 0x7fdf1019bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:34.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.239+0000 7fdf14a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf10105020 0x7fdf101973f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:34.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.239+0000 7fdf15e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf1019b850 0x7fdf1019bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57700/0 (socket says 172.21.15.1:57700) 2024-02-02T15:45:34.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.239+0000 7fdf14a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf10105020 0x7fdf101973f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:49196/0 (socket says 172.21.15.1:49196) 2024-02-02T15:45:34.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.239+0000 7fdf15e00700 1 -- 172.21.15.1:0/267026538 learned_addr learned my addr 172.21.15.1:0/267026538 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:45:34.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.239+0000 7fdf15400700 1 --2- 172.21.15.1:0/267026538 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf101000e0 0x7fdf10196eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:34.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.239+0000 7fdf15e00700 1 -- 172.21.15.1:0/267026538 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf101000e0 msgr2=0x7fdf10196eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:34.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.239+0000 7fdf15e00700 1 --2- 172.21.15.1:0/267026538 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf101000e0 0x7fdf10196eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:34.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.239+0000 7fdf15e00700 1 -- 172.21.15.1:0/267026538 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf10105020 msgr2=0x7fdf101973f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:34.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.239+0000 7fdf15e00700 1 --2- 172.21.15.1:0/267026538 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf10105020 0x7fdf101973f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:34.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.239+0000 7fdf15e00700 1 -- 172.21.15.1:0/267026538 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf000095f0 con 0x7fdf1019b850 2024-02-02T15:45:34.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.239+0000 7fdf15400700 1 --2- 172.21.15.1:0/267026538 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf101000e0 0x7fdf10196eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:45:34.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.239+0000 7fdf15e00700 1 --2- 172.21.15.1:0/267026538 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf1019b850 0x7fdf1019bcd0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fdf0800ef30 tx=0x7fdf0800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:45:34.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.239+0000 7fdf0e000700 1 -- 172.21.15.1:0/267026538 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf08018070 con 0x7fdf1019b850 2024-02-02T15:45:34.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.239+0000 7fdf0e000700 1 -- 172.21.15.1:0/267026538 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf0800f040 con 0x7fdf1019b850 2024-02-02T15:45:34.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.240+0000 7fdf0e000700 1 -- 172.21.15.1:0/267026538 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf0800cdb0 con 0x7fdf1019b850 2024-02-02T15:45:34.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.240+0000 7fdf17e00700 1 -- 172.21.15.1:0/267026538 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf101a3b00 con 0x7fdf1019b850 2024-02-02T15:45:34.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.240+0000 7fdf17e00700 1 -- 172.21.15.1:0/267026538 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf101a3f90 con 0x7fdf1019b850 2024-02-02T15:45:34.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.243+0000 7fdf0e000700 1 -- 172.21.15.1:0/267026538 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7fdf08003a00 con 0x7fdf1019b850 2024-02-02T15:45:34.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.245+0000 7fdf2b83cb80 1 Processor -- start 2024-02-02T15:45:34.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.245+0000 7fdf2b83cb80 1 -- start start 2024-02-02T15:45:34.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.245+0000 7fdf0e000700 1 --2- 172.21.15.1:0/267026538 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fdef00a06d0 0x7fdef00a2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:34.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.245+0000 7fdf2b83cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6dc6e2010 0x55f6dc6e23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:34.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.245+0000 7fdf2b83cb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6dc6e2930 0x55f6dc6ed2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:34.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.245+0000 7fdf2b83cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6dc6ed7e0 0x55f6dc6efbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:34.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.246+0000 7fdf2b83cb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6dc6564c0 con 0x55f6dc6e2930 2024-02-02T15:45:34.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.246+0000 7fdf2b83cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6dc63aaa0 con 0x55f6dc6e2010 2024-02-02T15:45:34.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.246+0000 7fdf2b83cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6dc638460 con 0x55f6dc6ed7e0 2024-02-02T15:45:34.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.246+0000 7fdf0e000700 1 -- 172.21.15.1:0/267026538 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fdf080cbca0 con 0x7fdf1019b850 2024-02-02T15:45:34.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.246+0000 7fdf15400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6dc6e2010 0x55f6dc6e23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:34.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.246+0000 7fdf15400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6dc6e2010 0x55f6dc6e23f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:49210/0 (socket says 172.21.15.1:49210) 2024-02-02T15:45:34.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.246+0000 7fdf15e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6dc6ed7e0 0x55f6dc6efbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:34.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.246+0000 7fdf15e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6dc6ed7e0 0x55f6dc6efbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:36924/0 (socket says 172.21.15.1:36924) 2024-02-02T15:45:34.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.246+0000 7fdf15e00700 1 -- 172.21.15.1:0/3610697759 learned_addr learned my addr 172.21.15.1:0/3610697759 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:45:34.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.247+0000 7fdf15e00700 1 -- 172.21.15.1:0/3610697759 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6dc6e2010 msgr2=0x55f6dc6e23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:45:34.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.247+0000 7fdf15e00700 1 --2- 172.21.15.1:0/3610697759 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6dc6e2010 0x55f6dc6e23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:34.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.247+0000 7fdf15e00700 1 -- 172.21.15.1:0/3610697759 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6dc6e2930 msgr2=0x55f6dc6ed2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:45:34.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.247+0000 7fdf15e00700 1 --2- 172.21.15.1:0/3610697759 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6dc6e2930 0x55f6dc6ed2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:34.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.247+0000 7fdf15e00700 1 -- 172.21.15.1:0/3610697759 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f6dc4b53b0 con 0x55f6dc6ed7e0 2024-02-02T15:45:34.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.247+0000 7fdf15e00700 1 --2- 172.21.15.1:0/3610697759 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6dc6ed7e0 0x55f6dc6efbd0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fdf080090a0 tx=0x7fdf08098ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76bec9c3f46760b4 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:45:34.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.247+0000 7fdf14a00700 1 --2- 172.21.15.1:0/267026538 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fdef00a06d0 0x7fdef00a2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:34.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.247+0000 7fdf06600700 1 -- 172.21.15.1:0/3610697759 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf08004ad0 con 0x55f6dc6ed7e0 2024-02-02T15:45:34.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.247+0000 7fdf06600700 1 -- 172.21.15.1:0/3610697759 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf0801b320 con 0x55f6dc6ed7e0 2024-02-02T15:45:34.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.247+0000 7fdf06600700 1 -- 172.21.15.1:0/3610697759 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf080990b0 con 0x55f6dc6ed7e0 2024-02-02T15:45:34.248 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.248+0000 7fdf2b83cb80 1 -- 172.21.15.1:0/3610697759 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6dc6ed7e0 msgr2=0x55f6dc6efbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:34.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.248+0000 7fdf2b83cb80 1 --2- 172.21.15.1:0/3610697759 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6dc6ed7e0 0x55f6dc6efbd0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fdf080090a0 tx=0x7fdf08098ca0 comp rx=0 tx=0).stop 2024-02-02T15:45:34.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.248+0000 7fdf14a00700 1 --2- 172.21.15.1:0/267026538 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fdef00a06d0 0x7fdef00a2b90 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fdf101984c0 tx=0x7fdef80099b0 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:45:34.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.248+0000 7fdf2b83cb80 1 -- 172.21.15.1:0/3610697759 shutdown_connections 2024-02-02T15:45:34.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.248+0000 7fdf2b83cb80 1 --2- 172.21.15.1:0/3610697759 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6dc6ed7e0 0x55f6dc6efbd0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:34.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.248+0000 7fdf2b83cb80 1 --2- 172.21.15.1:0/3610697759 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6dc6e2010 0x55f6dc6e23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:34.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.249+0000 7fdf2b83cb80 1 --2- 172.21.15.1:0/3610697759 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6dc6e2930 0x55f6dc6ed2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:34.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.249+0000 7fdf2b83cb80 1 -- 172.21.15.1:0/3610697759 >> 172.21.15.1:0/3610697759 conn(0x55f6dc641db0 msgr2=0x55f6dc6de3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:45:34.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.249+0000 7fdf2b83cb80 1 -- 172.21.15.1:0/3610697759 shutdown_connections 2024-02-02T15:45:34.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.249+0000 7fdf2b83cb80 1 -- 172.21.15.1:0/3610697759 wait complete. 2024-02-02T15:45:34.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.250+0000 7fdf2b83cb80 1 -- start start 2024-02-02T15:45:34.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.250+0000 7fdf2b83cb80 1 Processor -- start 2024-02-02T15:45:34.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.251+0000 7fdf2b83cb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6dc6e2010 0x55f6dc6ecf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:34.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.252+0000 7fdf2b83cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6dc6ed7e0 0x55f6dc6e61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:34.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.252+0000 7fdf2b83cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6dc6e6700 0x55f6dc6e6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:34.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.252+0000 7fdf2b83cb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6dc633d50 con 0x55f6dc6e2010 2024-02-02T15:45:34.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.252+0000 7fdf2b83cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6dc5cfd80 con 0x55f6dc6ed7e0 2024-02-02T15:45:34.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.252+0000 7fdf2b83cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6dc5cfa20 con 0x55f6dc6e6700 2024-02-02T15:45:34.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.252+0000 7fdf2b83cb80 10 client.0 _ll_register_callbacks cb 0x55f6dc5d1290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:45:34.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.252+0000 7fdf15400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6dc6e2010 0x55f6dc6ecf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:34.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.252+0000 7fdf15400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6dc6e2010 0x55f6dc6ecf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57708/0 (socket says 172.21.15.1:57708) 2024-02-02T15:45:34.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.252+0000 7fdf15400700 1 -- 172.21.15.1:0/3206701741 learned_addr learned my addr 172.21.15.1:0/3206701741 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:45:34.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.253+0000 7fdf15400700 1 -- 172.21.15.1:0/3206701741 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6dc6e6700 msgr2=0x55f6dc6e6b80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-02T15:45:34.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.253+0000 7fdf15400700 1 --2- 172.21.15.1:0/3206701741 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6dc6e6700 0x55f6dc6e6b80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:34.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.253+0000 7fdf15400700 1 -- 172.21.15.1:0/3206701741 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6dc6ed7e0 msgr2=0x55f6dc6e61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:45:34.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.253+0000 7fdf15400700 1 --2- 172.21.15.1:0/3206701741 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6dc6ed7e0 0x55f6dc6e61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:34.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.253+0000 7fdf15400700 1 -- 172.21.15.1:0/3206701741 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf101a3f90 con 0x55f6dc6e2010 2024-02-02T15:45:34.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.253+0000 7fdf15400700 1 --2- 172.21.15.1:0/3206701741 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6dc6e2010 0x55f6dc6ecf80 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fdf0000beb0 tx=0x7fdf0000bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=631345327ffc85f8 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:45:34.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.253+0000 7fdefea00700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:45:34.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.254+0000 7fdefea00700 1 -- 172.21.15.1:0/3206701741 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf00004500 con 0x55f6dc6e2010 2024-02-02T15:45:34.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.254+0000 7fdefea00700 1 -- 172.21.15.1:0/3206701741 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fdf00004660 con 0x55f6dc6e2010 2024-02-02T15:45:34.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.254+0000 7fdf2b83cb80 1 -- 172.21.15.1:0/3206701741 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f6dc6d60f0 con 0x55f6dc6e2010 2024-02-02T15:45:34.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.254+0000 7fdefea00700 1 -- 172.21.15.1:0/3206701741 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf00004820 con 0x55f6dc6e2010 2024-02-02T15:45:34.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.254+0000 7fdf2b83cb80 10 client.20064 fetch_fsmap learned FSMap version 78 2024-02-02T15:45:34.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.254+0000 7fdf2b83cb80 1 -- 172.21.15.1:0/3206701741 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) v3 -- 0x55f6dc461b60 con 0x55f6dc6e2010 2024-02-02T15:45:34.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.255+0000 7fdefea00700 1 -- 172.21.15.1:0/3206701741 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f6dc6d60f0 con 0x55f6dc6e2010 2024-02-02T15:45:34.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.255+0000 7fdefea00700 1 -- 172.21.15.1:0/3206701741 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 78) v1 ==== 7506+0+0 (secure 0 0 0) 0x7fdf0000fae0 con 0x55f6dc6e2010 2024-02-02T15:45:34.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.255+0000 7fdf2b83cb80 10 client.20064 fetch_fsmap finished waiting for FSMap version 78 2024-02-02T15:45:34.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.255+0000 7fdf2b83cb80 10 client.20064 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-02-02T15:45:34.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.255+0000 7fdf2b83cb80 20 client.20064 populate_metadata read hostname 'smithi001' 2024-02-02T15:45:34.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.255+0000 7fdf2b83cb80 1 --2- 172.21.15.1:0/3206701741 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55f6dc7060a0 0x55f6dc7084c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:34.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.255+0000 7fdf2b83cb80 4 client.20064 mds_command: new command op to 24518 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:45:34.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.256+0000 7fdf2b83cb80 1 -- 172.21.15.1:0/3206701741 --> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f6dc62cdd0 con 0x55f6dc7060a0 2024-02-02T15:45:34.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.256+0000 7fdf14a00700 1 --2- 172.21.15.1:0/3206701741 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55f6dc7060a0 0x55f6dc7084c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:34.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.256+0000 7fdefea00700 10 client.20064 ms_handle_connect on v2:172.21.15.71:6836/628016199 2024-02-02T15:45:34.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.256+0000 7fdf14a00700 1 --2- 172.21.15.1:0/3206701741 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55f6dc7060a0 0x55f6dc7084c0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a9f846e0ae0c19ee server_cookie=bf1af68489f422aa in_seq=0 out_seq=0 2024-02-02T15:45:34.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.258+0000 7fdefea00700 1 -- 172.21.15.1:0/3206701741 <== mds.2 v2:172.21.15.71:6836/628016199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55f6dc62cdd0 con 0x55f6dc7060a0 2024-02-02T15:45:34.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.258+0000 7fdefea00700 10 client.20064 handle_command_reply: tid=0 2024-02-02T15:45:34.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.258+0000 7fdf2b83cb80 1 client.20064 shutdown 2024-02-02T15:45:34.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.259+0000 7fdf2b83cb80 1 -- 172.21.15.1:0/3206701741 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6dc6e2010 msgr2=0x55f6dc6ecf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:34.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.259+0000 7fdf2b83cb80 1 --2- 172.21.15.1:0/3206701741 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6dc6e2010 0x55f6dc6ecf80 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fdf0000beb0 tx=0x7fdf0000bf90 comp rx=0 tx=0).stop 2024-02-02T15:45:34.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.259+0000 7fdf2b83cb80 1 -- 172.21.15.1:0/3206701741 shutdown_connections 2024-02-02T15:45:34.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.259+0000 7fdf2b83cb80 1 --2- 172.21.15.1:0/3206701741 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55f6dc7060a0 0x55f6dc7084c0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:34.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.259+0000 7fdf2b83cb80 1 --2- 172.21.15.1:0/3206701741 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6dc6e6700 0x55f6dc6e6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:34.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.259+0000 7fdf2b83cb80 1 --2- 172.21.15.1:0/3206701741 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6dc6e2010 0x55f6dc6ecf80 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:34.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.259+0000 7fdf2b83cb80 1 --2- 172.21.15.1:0/3206701741 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6dc6ed7e0 0x55f6dc6e61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:34.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.259+0000 7fdf2b83cb80 1 -- 172.21.15.1:0/3206701741 >> 172.21.15.1:0/3206701741 conn(0x55f6dc641db0 msgr2=0x55f6dc6ddf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:45:34.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.259+0000 7fdefea00700 0 client.20064 ms_handle_reset on v2:172.21.15.71:6836/628016199 2024-02-02T15:45:34.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.259+0000 7fdf2b83cb80 1 -- 172.21.15.1:0/3206701741 shutdown_connections 2024-02-02T15:45:34.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.259+0000 7fdf2b83cb80 1 -- 172.21.15.1:0/3206701741 wait complete. 2024-02-02T15:45:34.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.259+0000 7fdf2b83cb80 20 client.20064 trim_cache size 0 max 16384 2024-02-02T15:45:34.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.284+0000 7fdf2b83cb80 1 Processor -- start 2024-02-02T15:45:34.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.284+0000 7fdf2b83cb80 1 -- start start 2024-02-02T15:45:34.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.284+0000 7fdf2b83cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6dc6e6700 0x55f6dc6efbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:34.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.284+0000 7fdf2b83cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6dc6ed7e0 0x55f6dc6ecb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:34.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.284+0000 7fdf2b83cb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6dc6d9e00 0x55f6dc6f3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:34.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.284+0000 7fdf2b83cb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6dc4b76a0 con 0x55f6dc6d9e00 2024-02-02T15:45:34.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.284+0000 7fdf2b83cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6dc6f1250 con 0x55f6dc6ed7e0 2024-02-02T15:45:34.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.285+0000 7fdf2b83cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6dc6de5d0 con 0x55f6dc6e6700 2024-02-02T15:45:34.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.285+0000 7fdf15e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6dc6d9e00 0x55f6dc6f3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:34.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.285+0000 7fdf15e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6dc6d9e00 0x55f6dc6f3c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57712/0 (socket says 172.21.15.1:57712) 2024-02-02T15:45:34.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.285+0000 7fdf15e00700 1 -- 172.21.15.1:0/2682686673 learned_addr learned my addr 172.21.15.1:0/2682686673 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:45:34.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.286+0000 7fdf15400700 1 --2- 172.21.15.1:0/2682686673 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6dc6e6700 0x55f6dc6efbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:34.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.287+0000 7fdf15e00700 1 -- 172.21.15.1:0/2682686673 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6dc6e6700 msgr2=0x55f6dc6efbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:45:34.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.287+0000 7fdf15e00700 1 --2- 172.21.15.1:0/2682686673 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6dc6e6700 0x55f6dc6efbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:34.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.287+0000 7fdf15e00700 1 -- 172.21.15.1:0/2682686673 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6dc6ed7e0 msgr2=0x55f6dc6ecb00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-02T15:45:34.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.287+0000 7fdf15e00700 1 --2- 172.21.15.1:0/2682686673 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6dc6ed7e0 0x55f6dc6ecb00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:34.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.287+0000 7fdf15e00700 1 -- 172.21.15.1:0/2682686673 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f6dc5f9ea0 con 0x55f6dc6d9e00 2024-02-02T15:45:34.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.287+0000 7fdf15400700 1 --2- 172.21.15.1:0/2682686673 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6dc6e6700 0x55f6dc6efbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:45:34.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.288+0000 7fdf15e00700 1 --2- 172.21.15.1:0/2682686673 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6dc6d9e00 0x55f6dc6f3c50 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fdf0800c7a0 tx=0x7fdf080092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=787c41f269ad756 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:45:34.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.289+0000 7fdf07000700 1 -- 172.21.15.1:0/2682686673 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf08004ad0 con 0x55f6dc6d9e00 2024-02-02T15:45:34.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.289+0000 7fdf07000700 1 -- 172.21.15.1:0/2682686673 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf08099550 con 0x55f6dc6d9e00 2024-02-02T15:45:34.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.290+0000 7fdf07000700 1 -- 172.21.15.1:0/2682686673 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf080b11a0 con 0x55f6dc6d9e00 2024-02-02T15:45:34.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.294+0000 7fdf2b83cb80 1 -- 172.21.15.1:0/2682686673 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6dc6d9e00 msgr2=0x55f6dc6f3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:34.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.294+0000 7fdf2b83cb80 1 --2- 172.21.15.1:0/2682686673 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6dc6d9e00 0x55f6dc6f3c50 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fdf0800c7a0 tx=0x7fdf080092e0 comp rx=0 tx=0).stop 2024-02-02T15:45:34.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.294+0000 7fdf2b83cb80 1 -- 172.21.15.1:0/2682686673 shutdown_connections 2024-02-02T15:45:34.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.294+0000 7fdf2b83cb80 1 --2- 172.21.15.1:0/2682686673 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6dc6d9e00 0x55f6dc6f3c50 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:34.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.294+0000 7fdf2b83cb80 1 --2- 172.21.15.1:0/2682686673 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6dc6e6700 0x55f6dc6efbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:34.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.294+0000 7fdf2b83cb80 1 --2- 172.21.15.1:0/2682686673 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6dc6ed7e0 0x55f6dc6ecb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:34.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.294+0000 7fdf2b83cb80 1 -- 172.21.15.1:0/2682686673 >> 172.21.15.1:0/2682686673 conn(0x55f6dc641db0 msgr2=0x55f6dc6eae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:45:34.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.294+0000 7fdf2b83cb80 1 -- 172.21.15.1:0/2682686673 shutdown_connections 2024-02-02T15:45:34.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.295+0000 7fdf2b83cb80 1 -- 172.21.15.1:0/2682686673 wait complete. 2024-02-02T15:45:34.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.295+0000 7fdf2b83cb80 1 -- start start 2024-02-02T15:45:34.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.296+0000 7fdf2b83cb80 1 Processor -- start 2024-02-02T15:45:34.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.296+0000 7fdf2b83cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6dc6d9e00 0x55f6dc750c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:34.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.296+0000 7fdf2b83cb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6dc6e6700 0x55f6dc7511b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:34.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.296+0000 7fdf2b83cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6dc6ed7e0 0x55f6dc7516f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:34.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.296+0000 7fdf2b83cb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6dc6fd900 con 0x55f6dc6e6700 2024-02-02T15:45:34.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.296+0000 7fdf2b83cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6dc6fda70 con 0x55f6dc6ed7e0 2024-02-02T15:45:34.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.296+0000 7fdf2b83cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f6dc6fdc30 con 0x55f6dc6d9e00 2024-02-02T15:45:34.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.297+0000 7fdf2b83cb80 10 client.0 _ll_register_callbacks cb 0x55f6dc5d1290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:45:34.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.297+0000 7fdf15400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6dc6d9e00 0x55f6dc750c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:34.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.297+0000 7fdf15400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6dc6d9e00 0x55f6dc750c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:36942/0 (socket says 172.21.15.1:36942) 2024-02-02T15:45:34.300 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.297+0000 7fdf15400700 1 -- 172.21.15.1:0/2935298363 learned_addr learned my addr 172.21.15.1:0/2935298363 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:45:34.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.298+0000 7fdf15400700 1 -- 172.21.15.1:0/2935298363 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6dc6ed7e0 msgr2=0x55f6dc7516f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-02T15:45:34.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.298+0000 7fdf15e00700 1 --2- 172.21.15.1:0/2935298363 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6dc6ed7e0 0x55f6dc7516f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:34.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.298+0000 7fdf14a00700 1 --2- 172.21.15.1:0/2935298363 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6dc6e6700 0x55f6dc7511b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:34.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.298+0000 7fdf15400700 1 --2- 172.21.15.1:0/2935298363 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6dc6ed7e0 0x55f6dc7516f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:34.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.298+0000 7fdf15400700 1 -- 172.21.15.1:0/2935298363 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6dc6e6700 msgr2=0x55f6dc7511b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:45:34.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.298+0000 7fdf15400700 1 --2- 172.21.15.1:0/2935298363 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6dc6e6700 0x55f6dc7511b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:34.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.298+0000 7fdf15400700 1 -- 172.21.15.1:0/2935298363 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f6dc4b53b0 con 0x55f6dc6d9e00 2024-02-02T15:45:34.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.298+0000 7fdf14a00700 1 --2- 172.21.15.1:0/2935298363 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6dc6e6700 0x55f6dc7511b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:45:34.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.299+0000 7fdf15400700 1 --2- 172.21.15.1:0/2935298363 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6dc6d9e00 0x55f6dc750c70 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fdf00009990 tx=0x7fdf0000b380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90ecb8c606fd1b6f server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:45:34.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.299+0000 7fdf15e00700 1 --2- 172.21.15.1:0/2935298363 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6dc6ed7e0 0x55f6dc7516f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:45:34.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.299+0000 7fdefea00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:45:34.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.299+0000 7fdefea00700 1 -- 172.21.15.1:0/2935298363 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf000239a0 con 0x55f6dc6d9e00 2024-02-02T15:45:34.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.299+0000 7fdefea00700 1 -- 172.21.15.1:0/2935298363 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fdf00000e40 con 0x55f6dc6d9e00 2024-02-02T15:45:34.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.300+0000 7fdf2b83cb80 1 -- 172.21.15.1:0/2935298363 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdf0801b320 con 0x55f6dc6d9e00 2024-02-02T15:45:34.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.300+0000 7fdefea00700 1 -- 172.21.15.1:0/2935298363 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf00023b00 con 0x55f6dc6d9e00 2024-02-02T15:45:34.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.300+0000 7fdf2b83cb80 10 client.30220 fetch_fsmap learned FSMap version 78 2024-02-02T15:45:34.302 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.300+0000 7fdf2b83cb80 1 -- 172.21.15.1:0/2935298363 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) v3 -- 0x55f6dc4b9cd0 con 0x55f6dc6d9e00 2024-02-02T15:45:34.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.303+0000 7fdefea00700 1 -- 172.21.15.1:0/2935298363 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdf00040070 con 0x55f6dc6d9e00 2024-02-02T15:45:34.303 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.303+0000 7fdefea00700 1 -- 172.21.15.1:0/2935298363 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 78) v1 ==== 7506+0+0 (secure 0 0 0) 0x7fdf0000f460 con 0x55f6dc6d9e00 2024-02-02T15:45:34.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.304+0000 7fdf2b83cb80 10 client.30220 fetch_fsmap finished waiting for FSMap version 78 2024-02-02T15:45:34.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.304+0000 7fdf2b83cb80 10 client.30220 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-02-02T15:45:34.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.304+0000 7fdf2b83cb80 20 client.30220 populate_metadata read hostname 'smithi001' 2024-02-02T15:45:34.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.304+0000 7fdf2b83cb80 1 --2- 172.21.15.1:0/2935298363 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55f6dc6fef20 0x55f6dc701310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:34.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.304+0000 7fdf2b83cb80 4 client.30220 mds_command: new command op to 24518 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:45:34.304 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.304+0000 7fdf2b83cb80 1 -- 172.21.15.1:0/2935298363 --> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f6dc5f2f50 con 0x55f6dc6fef20 2024-02-02T15:45:34.307 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.307+0000 7fdf14a00700 1 --2- 172.21.15.1:0/2935298363 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55f6dc6fef20 0x55f6dc701310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:34.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.308+0000 7fdf14a00700 1 --2- 172.21.15.1:0/2935298363 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55f6dc6fef20 0x55f6dc701310 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=80db3953edb69c5a server_cookie=7d479a4fdaae261 in_seq=0 out_seq=0 2024-02-02T15:45:34.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.308+0000 7fdefea00700 10 client.30220 ms_handle_connect on v2:172.21.15.71:6836/628016199 2024-02-02T15:45:34.308 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.309+0000 7fdefea00700 1 -- 172.21.15.1:0/2935298363 <== mds.2 v2:172.21.15.71:6836/628016199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55f6dc5f2f50 con 0x55f6dc6fef20 2024-02-02T15:45:34.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.309+0000 7fdefea00700 10 client.30220 handle_command_reply: tid=0 2024-02-02T15:45:34.309 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.309+0000 7fdf2b83cb80 1 client.30220 shutdown 2024-02-02T15:45:34.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.311+0000 7fdf2b83cb80 1 -- 172.21.15.1:0/2935298363 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6dc6d9e00 msgr2=0x55f6dc750c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:34.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.311+0000 7fdf2b83cb80 1 --2- 172.21.15.1:0/2935298363 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6dc6d9e00 0x55f6dc750c70 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fdf00009990 tx=0x7fdf0000b380 comp rx=0 tx=0).stop 2024-02-02T15:45:34.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.311+0000 7fdf2b83cb80 1 -- 172.21.15.1:0/2935298363 shutdown_connections 2024-02-02T15:45:34.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.311+0000 7fdf2b83cb80 1 --2- 172.21.15.1:0/2935298363 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f6dc6ed7e0 0x55f6dc7516f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:34.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.311+0000 7fdf2b83cb80 1 --2- 172.21.15.1:0/2935298363 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x55f6dc6fef20 0x55f6dc701310 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:34.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.311+0000 7fdf2b83cb80 1 --2- 172.21.15.1:0/2935298363 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55f6dc6d9e00 0x55f6dc750c70 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:34.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.311+0000 7fdf2b83cb80 1 --2- 172.21.15.1:0/2935298363 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55f6dc6e6700 0x55f6dc7511b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:34.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.311+0000 7fdf2b83cb80 1 -- 172.21.15.1:0/2935298363 >> 172.21.15.1:0/2935298363 conn(0x55f6dc641db0 msgr2=0x55f6dc6efbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:45:34.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.311+0000 7fdefea00700 0 client.30220 ms_handle_reset on v2:172.21.15.71:6836/628016199 2024-02-02T15:45:34.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.312+0000 7fdf2b83cb80 1 -- 172.21.15.1:0/2935298363 shutdown_connections 2024-02-02T15:45:34.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.312+0000 7fdf2b83cb80 1 -- 172.21.15.1:0/2935298363 wait complete. 2024-02-02T15:45:34.313 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.313+0000 7fdf2b83cb80 20 client.30220 trim_cache size 0 max 16384 2024-02-02T15:45:34.313 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:45:34.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.313+0000 7fdf17e00700 1 -- 172.21.15.1:0/267026538 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fdef00a06d0 msgr2=0x7fdef00a2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:34.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.313+0000 7fdf17e00700 1 --2- 172.21.15.1:0/267026538 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fdef00a06d0 0x7fdef00a2b90 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fdf101984c0 tx=0x7fdef80099b0 comp rx=0 tx=0).stop 2024-02-02T15:45:34.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.313+0000 7fdf17e00700 1 -- 172.21.15.1:0/267026538 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf1019b850 msgr2=0x7fdf1019bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:34.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.313+0000 7fdf17e00700 1 --2- 172.21.15.1:0/267026538 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf1019b850 0x7fdf1019bcd0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fdf0800ef30 tx=0x7fdf0800c3a0 comp rx=0 tx=0).stop 2024-02-02T15:45:34.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.314+0000 7fdf17e00700 1 -- 172.21.15.1:0/267026538 shutdown_connections 2024-02-02T15:45:34.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.314+0000 7fdf17e00700 1 --2- 172.21.15.1:0/267026538 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fdef00a06d0 0x7fdef00a2b90 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:34.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.314+0000 7fdf17e00700 1 --2- 172.21.15.1:0/267026538 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fdf1019b850 0x7fdf1019bcd0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:34.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.314+0000 7fdf17e00700 1 --2- 172.21.15.1:0/267026538 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fdf101000e0 0x7fdf10196eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:34.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.314+0000 7fdf17e00700 1 --2- 172.21.15.1:0/267026538 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdf10105020 0x7fdf101973f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:34.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.314+0000 7fdf17e00700 1 -- 172.21.15.1:0/267026538 >> 172.21.15.1:0/267026538 conn(0x7fdf100757d0 msgr2=0x7fdf10068110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:45:34.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.314+0000 7fdf17e00700 1 -- 172.21.15.1:0/267026538 shutdown_connections 2024-02-02T15:45:34.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.315+0000 7fdf17e00700 1 -- 172.21.15.1:0/267026538 wait complete. 2024-02-02T15:45:34.347 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-02T15:45:34.661 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.661+0000 7f8abfc00700 1 -- 172.21.15.1:0/2015779027 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8ab810b4d0 msgr2=0x7f8ab8107d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:34.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.661+0000 7f8abfc00700 1 --2- 172.21.15.1:0/2015779027 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8ab810b4d0 0x7f8ab8107d40 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f8ab000b0a0 tx=0x7f8ab000b3b0 comp rx=0 tx=0).stop 2024-02-02T15:45:34.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.661+0000 7f8abfc00700 1 -- 172.21.15.1:0/2015779027 shutdown_connections 2024-02-02T15:45:34.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.661+0000 7f8abfc00700 1 --2- 172.21.15.1:0/2015779027 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ab8115e70 0x7f8ab8118260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:34.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.661+0000 7f8abfc00700 1 --2- 172.21.15.1:0/2015779027 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8ab810b4d0 0x7f8ab8107d40 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:34.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.661+0000 7f8abfc00700 1 --2- 172.21.15.1:0/2015779027 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ab8108310 0x7f8ab8108790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:34.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.661+0000 7f8abfc00700 1 -- 172.21.15.1:0/2015779027 >> 172.21.15.1:0/2015779027 conn(0x7f8ab806d120 msgr2=0x7f8ab806d530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:45:34.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.661+0000 7f8abfc00700 1 -- 172.21.15.1:0/2015779027 shutdown_connections 2024-02-02T15:45:34.662 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.662+0000 7f8abfc00700 1 -- 172.21.15.1:0/2015779027 wait complete. 2024-02-02T15:45:34.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.663+0000 7f8abfc00700 1 Processor -- start 2024-02-02T15:45:34.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.664+0000 7f8abfc00700 1 -- start start 2024-02-02T15:45:34.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.665+0000 7f8abfc00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ab8108310 0x7f8ab81153f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:34.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.665+0000 7f8abfc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ab810b4d0 0x7f8ab8112400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:34.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.665+0000 7f8abfc00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8ab8115e70 0x7f8ab8112940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:34.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.665+0000 7f8abfc00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab8112f40 con 0x7f8ab8115e70 2024-02-02T15:45:34.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.665+0000 7f8abfc00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab81130b0 con 0x7f8ab8108310 2024-02-02T15:45:34.665 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.665+0000 7f8abfc00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab8113270 con 0x7f8ab810b4d0 2024-02-02T15:45:34.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.666+0000 7f8ab7e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ab810b4d0 0x7f8ab8112400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:34.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.666+0000 7f8abdc00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8ab8115e70 0x7f8ab8112940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:34.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.666+0000 7f8ab7e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ab810b4d0 0x7f8ab8112400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:36968/0 (socket says 172.21.15.1:36968) 2024-02-02T15:45:34.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.666+0000 7f8ab7e00700 1 -- 172.21.15.1:0/3466623508 learned_addr learned my addr 172.21.15.1:0/3466623508 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:45:34.666 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.666+0000 7f8abd200700 1 --2- 172.21.15.1:0/3466623508 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ab8108310 0x7f8ab81153f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:34.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.666+0000 7f8abd200700 1 -- 172.21.15.1:0/3466623508 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ab810b4d0 msgr2=0x7f8ab8112400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:34.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.666+0000 7f8abd200700 1 --2- 172.21.15.1:0/3466623508 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ab810b4d0 0x7f8ab8112400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:34.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.666+0000 7f8abd200700 1 -- 172.21.15.1:0/3466623508 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8ab8115e70 msgr2=0x7f8ab8112940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:34.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.666+0000 7f8abd200700 1 --2- 172.21.15.1:0/3466623508 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8ab8115e70 0x7f8ab8112940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:34.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.666+0000 7f8abd200700 1 -- 172.21.15.1:0/3466623508 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ab0009d00 con 0x7f8ab8108310 2024-02-02T15:45:34.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.667+0000 7f8abdc00700 1 --2- 172.21.15.1:0/3466623508 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8ab8115e70 0x7f8ab8112940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:45:34.669 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.669+0000 7f8abd200700 1 --2- 172.21.15.1:0/3466623508 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ab8108310 0x7f8ab81153f0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f8ab00062a0 tx=0x7f8ab00079b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:45:34.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.670+0000 7f8ab5600700 1 -- 172.21.15.1:0/3466623508 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab000e040 con 0x7f8ab8108310 2024-02-02T15:45:34.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.670+0000 7f8abfc00700 1 -- 172.21.15.1:0/3466623508 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ab81b2720 con 0x7f8ab8108310 2024-02-02T15:45:34.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.670+0000 7f8ab5600700 1 -- 172.21.15.1:0/3466623508 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8ab0004ce0 con 0x7f8ab8108310 2024-02-02T15:45:34.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.670+0000 7f8ab5600700 1 -- 172.21.15.1:0/3466623508 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab00048b0 con 0x7f8ab8108310 2024-02-02T15:45:34.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.670+0000 7f8abfc00700 1 -- 172.21.15.1:0/3466623508 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ab81b2ba0 con 0x7f8ab8108310 2024-02-02T15:45:34.677 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.678+0000 7f8ad36aeb80 1 Processor -- start 2024-02-02T15:45:34.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.678+0000 7f8ab5600700 1 -- 172.21.15.1:0/3466623508 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f8ab0004020 con 0x7f8ab8108310 2024-02-02T15:45:34.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.678+0000 7f8ad36aeb80 1 -- start start 2024-02-02T15:45:34.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.679+0000 7f8ad36aeb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621d910d010 0x5621d910d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:34.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.679+0000 7f8ad36aeb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5621d910d930 0x5621d91182a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:34.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.679+0000 7f8ad36aeb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621d91187e0 0x5621d911abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:34.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.679+0000 7f8ad36aeb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621d90814c0 con 0x5621d910d010 2024-02-02T15:45:34.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.679+0000 7f8ad36aeb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621d9065aa0 con 0x5621d910d930 2024-02-02T15:45:34.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.679+0000 7f8ad36aeb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621d9063460 con 0x5621d91187e0 2024-02-02T15:45:34.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.679+0000 7f8abdc00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621d910d010 0x5621d910d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:34.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.679+0000 7f8abdc00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621d910d010 0x5621d910d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57750/0 (socket says 172.21.15.1:57750) 2024-02-02T15:45:34.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.679+0000 7f8abdc00700 1 -- 172.21.15.1:0/1120677560 learned_addr learned my addr 172.21.15.1:0/1120677560 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:45:34.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.680+0000 7f8ab7e00700 1 --2- 172.21.15.1:0/1120677560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621d91187e0 0x5621d911abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:34.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.680+0000 7f8abdc00700 1 -- 172.21.15.1:0/1120677560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621d91187e0 msgr2=0x5621d911abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:45:34.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.680+0000 7f8abdc00700 1 --2- 172.21.15.1:0/1120677560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621d91187e0 0x5621d911abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:34.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.680+0000 7f8abdc00700 1 -- 172.21.15.1:0/1120677560 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5621d910d930 msgr2=0x5621d91182a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:45:34.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.680+0000 7f8abdc00700 1 --2- 172.21.15.1:0/1120677560 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5621d910d930 0x5621d91182a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:34.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.680+0000 7f8abdc00700 1 -- 172.21.15.1:0/1120677560 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5621d8ee03b0 con 0x5621d910d010 2024-02-02T15:45:34.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.681+0000 7f8abdc00700 1 --2- 172.21.15.1:0/1120677560 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621d910d010 0x5621d910d3f0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f8aa8009e10 tx=0x7f8aa800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccd0d71f12766e8a server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:45:34.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.681+0000 7f8ab5600700 1 --2- 172.21.15.1:0/3466623508 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f8a980a0770 0x7f8a980a2c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:34.681 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.681+0000 7f8aae000700 1 -- 172.21.15.1:0/1120677560 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa80098e0 con 0x5621d910d010 2024-02-02T15:45:34.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.682+0000 7f8aae000700 1 -- 172.21.15.1:0/1120677560 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8aa800ecd0 con 0x5621d910d010 2024-02-02T15:45:34.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.682+0000 7f8ab5600700 1 -- 172.21.15.1:0/3466623508 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f8ab00cd740 con 0x7f8ab8108310 2024-02-02T15:45:34.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.682+0000 7f8aae000700 1 -- 172.21.15.1:0/1120677560 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa8012750 con 0x5621d910d010 2024-02-02T15:45:34.682 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.682+0000 7f8abd200700 1 --2- 172.21.15.1:0/3466623508 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f8a980a0770 0x7f8a980a2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:34.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.683+0000 7f8ad36aeb80 1 -- 172.21.15.1:0/1120677560 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621d910d010 msgr2=0x5621d910d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:34.683 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.683+0000 7f8ad36aeb80 1 --2- 172.21.15.1:0/1120677560 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621d910d010 0x5621d910d3f0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f8aa8009e10 tx=0x7f8aa800e4b0 comp rx=0 tx=0).stop 2024-02-02T15:45:34.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.684+0000 7f8abd200700 1 --2- 172.21.15.1:0/3466623508 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f8a980a0770 0x7f8a980a2c30 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f8ab0007b20 tx=0x7f8ab0009040 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:45:34.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.684+0000 7f8ad36aeb80 1 -- 172.21.15.1:0/1120677560 shutdown_connections 2024-02-02T15:45:34.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.684+0000 7f8ad36aeb80 1 --2- 172.21.15.1:0/1120677560 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621d91187e0 0x5621d911abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:34.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.684+0000 7f8ad36aeb80 1 --2- 172.21.15.1:0/1120677560 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621d910d010 0x5621d910d3f0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:34.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.684+0000 7f8ad36aeb80 1 --2- 172.21.15.1:0/1120677560 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5621d910d930 0x5621d91182a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:34.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.684+0000 7f8ad36aeb80 1 -- 172.21.15.1:0/1120677560 >> 172.21.15.1:0/1120677560 conn(0x5621d906cdb0 msgr2=0x5621d91093b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:45:34.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.684+0000 7f8ad36aeb80 1 -- 172.21.15.1:0/1120677560 shutdown_connections 2024-02-02T15:45:34.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.684+0000 7f8ad36aeb80 1 -- 172.21.15.1:0/1120677560 wait complete. 2024-02-02T15:45:34.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.685+0000 7f8ad36aeb80 1 -- start start 2024-02-02T15:45:34.685 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.685+0000 7f8ad36aeb80 1 Processor -- start 2024-02-02T15:45:34.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.686+0000 7f8ad36aeb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621d910d010 0x5621d9118230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:34.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.686+0000 7f8ad36aeb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5621d910d930 0x5621d9115440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:34.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.686+0000 7f8ad36aeb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621d91187e0 0x5621d9115b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:34.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.686+0000 7f8ad36aeb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621d905ed50 con 0x5621d91187e0 2024-02-02T15:45:34.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.686+0000 7f8ad36aeb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621d8ffad80 con 0x5621d910d930 2024-02-02T15:45:34.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.686+0000 7f8ad36aeb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621d8ffaa20 con 0x5621d910d010 2024-02-02T15:45:34.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.686+0000 7f8ad36aeb80 10 client.0 _ll_register_callbacks cb 0x5621d8ffc290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:45:34.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.686+0000 7f8abdc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621d910d010 0x5621d9118230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:34.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.686+0000 7f8abdc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621d910d010 0x5621d9118230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:36976/0 (socket says 172.21.15.1:36976) 2024-02-02T15:45:34.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.686+0000 7f8abdc00700 1 -- 172.21.15.1:0/2303624622 learned_addr learned my addr 172.21.15.1:0/2303624622 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:45:34.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.686+0000 7f8abdc00700 1 --2- 172.21.15.1:0/2303624622 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621d91187e0 0x5621d9115b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:34.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.687+0000 7f8ab7e00700 1 --2- 172.21.15.1:0/2303624622 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5621d910d930 0x5621d9115440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:34.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.687+0000 7f8abdc00700 1 -- 172.21.15.1:0/2303624622 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621d910d010 msgr2=0x5621d9118230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:45:34.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.687+0000 7f8abdc00700 1 --2- 172.21.15.1:0/2303624622 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621d910d010 0x5621d9118230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:34.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.687+0000 7f8abdc00700 1 -- 172.21.15.1:0/2303624622 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5621d910d930 msgr2=0x5621d9115440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:45:34.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.687+0000 7f8abdc00700 1 --2- 172.21.15.1:0/2303624622 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5621d910d930 0x5621d9115440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:34.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.687+0000 7f8abdc00700 1 -- 172.21.15.1:0/2303624622 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8aa8009a40 con 0x5621d91187e0 2024-02-02T15:45:34.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.687+0000 7f8ab7e00700 1 --2- 172.21.15.1:0/2303624622 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5621d910d930 0x5621d9115440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:45:34.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.688+0000 7f8abdc00700 1 --2- 172.21.15.1:0/2303624622 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621d91187e0 0x5621d9115b80 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f8aa8012840 tx=0x7f8aa8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cade04ce40aee63 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:45:34.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.688+0000 7f8aa6a00700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:45:34.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.690+0000 7f8aa6a00700 1 -- 172.21.15.1:0/2303624622 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa80138b0 con 0x5621d91187e0 2024-02-02T15:45:34.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.690+0000 7f8aa6a00700 1 -- 172.21.15.1:0/2303624622 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f8aa8012870 con 0x5621d91187e0 2024-02-02T15:45:34.690 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.690+0000 7f8ad36aeb80 1 -- 172.21.15.1:0/2303624622 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5621d91010f0 con 0x5621d91187e0 2024-02-02T15:45:34.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.692+0000 7f8aa6a00700 1 -- 172.21.15.1:0/2303624622 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8aa8013a10 con 0x5621d91187e0 2024-02-02T15:45:34.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.692+0000 7f8ad36aeb80 10 client.20079 fetch_fsmap learned FSMap version 78 2024-02-02T15:45:34.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.692+0000 7f8ad36aeb80 1 -- 172.21.15.1:0/2303624622 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) v3 -- 0x5621d8e8cb60 con 0x5621d91187e0 2024-02-02T15:45:34.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.696+0000 7f8aa6a00700 1 -- 172.21.15.1:0/2303624622 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5621d91010f0 con 0x5621d91187e0 2024-02-02T15:45:34.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.696+0000 7f8aa6a00700 1 -- 172.21.15.1:0/2303624622 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 78) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f8aa8011070 con 0x5621d91187e0 2024-02-02T15:45:34.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.696+0000 7f8ad36aeb80 10 client.20079 fetch_fsmap finished waiting for FSMap version 78 2024-02-02T15:45:34.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.696+0000 7f8ad36aeb80 10 client.20079 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-02T15:45:34.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.696+0000 7f8ad36aeb80 20 client.20079 populate_metadata read hostname 'smithi001' 2024-02-02T15:45:34.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.696+0000 7f8ad36aeb80 1 --2- 172.21.15.1:0/2303624622 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x5621d9139260 0x5621d913b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:34.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.696+0000 7f8ad36aeb80 4 client.20079 mds_command: new command op to 24529 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:45:34.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.696+0000 7f8ad36aeb80 1 -- 172.21.15.1:0/2303624622 --> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5621d9057dd0 con 0x5621d9139260 2024-02-02T15:45:34.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.697+0000 7f8ab7e00700 1 --2- 172.21.15.1:0/2303624622 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x5621d9139260 0x5621d913b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:34.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.697+0000 7f8ab7e00700 1 --2- 172.21.15.1:0/2303624622 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x5621d9139260 0x5621d913b680 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=135afce8a7b86a36 server_cookie=fb0708b0a97ee9cb in_seq=0 out_seq=0 2024-02-02T15:45:34.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.697+0000 7f8aa6a00700 10 client.20079 ms_handle_connect on v2:172.21.15.142:6838/321675421 2024-02-02T15:45:34.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.700+0000 7f8aa6a00700 1 -- 172.21.15.1:0/2303624622 <== mds.1 v2:172.21.15.142:6838/321675421 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5621d9057dd0 con 0x5621d9139260 2024-02-02T15:45:34.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.700+0000 7f8aa6a00700 10 client.20079 handle_command_reply: tid=0 2024-02-02T15:45:34.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.700+0000 7f8ad36aeb80 1 client.20079 shutdown 2024-02-02T15:45:34.700 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.700+0000 7f8ad36aeb80 1 -- 172.21.15.1:0/2303624622 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621d91187e0 msgr2=0x5621d9115b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:34.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.700+0000 7f8ad36aeb80 1 --2- 172.21.15.1:0/2303624622 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621d91187e0 0x5621d9115b80 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f8aa8012840 tx=0x7f8aa8004750 comp rx=0 tx=0).stop 2024-02-02T15:45:34.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.701+0000 7f8ad36aeb80 1 -- 172.21.15.1:0/2303624622 shutdown_connections 2024-02-02T15:45:34.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.701+0000 7f8ad36aeb80 1 --2- 172.21.15.1:0/2303624622 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x5621d9139260 0x5621d913b680 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:34.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.701+0000 7f8ad36aeb80 1 --2- 172.21.15.1:0/2303624622 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621d91187e0 0x5621d9115b80 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:34.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.701+0000 7f8ad36aeb80 1 --2- 172.21.15.1:0/2303624622 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621d910d010 0x5621d9118230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:34.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.701+0000 7f8ad36aeb80 1 --2- 172.21.15.1:0/2303624622 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5621d910d930 0x5621d9115440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:34.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.701+0000 7f8aa6a00700 0 client.20079 ms_handle_reset on v2:172.21.15.142:6838/321675421 2024-02-02T15:45:34.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.701+0000 7f8ad36aeb80 1 -- 172.21.15.1:0/2303624622 >> 172.21.15.1:0/2303624622 conn(0x5621d906cdb0 msgr2=0x5621d9108fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:45:34.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.701+0000 7f8ad36aeb80 1 -- 172.21.15.1:0/2303624622 shutdown_connections 2024-02-02T15:45:34.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.701+0000 7f8ad36aeb80 1 -- 172.21.15.1:0/2303624622 wait complete. 2024-02-02T15:45:34.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.701+0000 7f8ad36aeb80 20 client.20079 trim_cache size 0 max 16384 2024-02-02T15:45:34.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.719+0000 7f8ad36aeb80 1 Processor -- start 2024-02-02T15:45:34.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.720+0000 7f8ad36aeb80 1 -- start start 2024-02-02T15:45:34.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.720+0000 7f8ad36aeb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621d910d010 0x5621d9130380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:34.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.720+0000 7f8ad36aeb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621d910d930 0x5621d9117d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:34.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.720+0000 7f8ad36aeb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5621d91187e0 0x5621d9114630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:34.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.720+0000 7f8ad36aeb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621d8ee26a0 con 0x5621d910d930 2024-02-02T15:45:34.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.720+0000 7f8ad36aeb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621d91112e0 con 0x5621d91187e0 2024-02-02T15:45:34.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.720+0000 7f8ad36aeb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621d91168d0 con 0x5621d910d010 2024-02-02T15:45:34.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.720+0000 7f8abdc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621d910d010 0x5621d9130380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:34.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.720+0000 7f8abdc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621d910d010 0x5621d9130380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:36978/0 (socket says 172.21.15.1:36978) 2024-02-02T15:45:34.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.720+0000 7f8abdc00700 1 -- 172.21.15.1:0/3078384255 learned_addr learned my addr 172.21.15.1:0/3078384255 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:45:34.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.720+0000 7f8ab7e00700 1 --2- 172.21.15.1:0/3078384255 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621d910d930 0x5621d9117d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:34.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.720+0000 7f8abdc00700 1 --2- 172.21.15.1:0/3078384255 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5621d91187e0 0x5621d9114630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:34.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.721+0000 7f8abdc00700 1 -- 172.21.15.1:0/3078384255 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5621d91187e0 msgr2=0x5621d9114630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:45:34.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.721+0000 7f8abdc00700 1 --2- 172.21.15.1:0/3078384255 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5621d91187e0 0x5621d9114630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:34.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.721+0000 7f8abdc00700 1 -- 172.21.15.1:0/3078384255 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621d910d930 msgr2=0x5621d9117d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:45:34.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.721+0000 7f8abdc00700 1 --2- 172.21.15.1:0/3078384255 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621d910d930 0x5621d9117d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:34.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.721+0000 7f8abdc00700 1 -- 172.21.15.1:0/3078384255 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5621d9135920 con 0x5621d910d010 2024-02-02T15:45:34.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.721+0000 7f8abdc00700 1 --2- 172.21.15.1:0/3078384255 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621d910d010 0x5621d9130380 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f8aa8001eb0 tx=0x7f8aa800e040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a41f153b58dc8cf server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:45:34.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.721+0000 7f8aaea00700 1 -- 172.21.15.1:0/3078384255 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa80138b0 con 0x5621d910d010 2024-02-02T15:45:34.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.721+0000 7f8aaea00700 1 -- 172.21.15.1:0/3078384255 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8aa8020070 con 0x5621d910d010 2024-02-02T15:45:34.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.721+0000 7f8aaea00700 1 -- 172.21.15.1:0/3078384255 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa801c450 con 0x5621d910d010 2024-02-02T15:45:34.723 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.722+0000 7f8ad36aeb80 1 -- 172.21.15.1:0/3078384255 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621d910d010 msgr2=0x5621d9130380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:34.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.722+0000 7f8ad36aeb80 1 --2- 172.21.15.1:0/3078384255 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621d910d010 0x5621d9130380 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f8aa8001eb0 tx=0x7f8aa800e040 comp rx=0 tx=0).stop 2024-02-02T15:45:34.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.722+0000 7f8ad36aeb80 1 -- 172.21.15.1:0/3078384255 shutdown_connections 2024-02-02T15:45:34.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.722+0000 7f8ad36aeb80 1 --2- 172.21.15.1:0/3078384255 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5621d91187e0 0x5621d9114630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:34.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.722+0000 7f8ad36aeb80 1 --2- 172.21.15.1:0/3078384255 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621d910d010 0x5621d9130380 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:34.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.722+0000 7f8ad36aeb80 1 --2- 172.21.15.1:0/3078384255 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621d910d930 0x5621d9117d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:34.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.722+0000 7f8ad36aeb80 1 -- 172.21.15.1:0/3078384255 >> 172.21.15.1:0/3078384255 conn(0x5621d906cdb0 msgr2=0x5621d9108eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:45:34.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.722+0000 7f8ad36aeb80 1 -- 172.21.15.1:0/3078384255 shutdown_connections 2024-02-02T15:45:34.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.722+0000 7f8ad36aeb80 1 -- 172.21.15.1:0/3078384255 wait complete. 2024-02-02T15:45:34.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.723+0000 7f8ad36aeb80 1 -- start start 2024-02-02T15:45:34.724 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.723+0000 7f8ad36aeb80 1 Processor -- start 2024-02-02T15:45:34.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.724+0000 7f8ad36aeb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5621d910d010 0x5621d912e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:34.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.724+0000 7f8ad36aeb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621d910d930 0x5621d912e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:34.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.724+0000 7f8ad36aeb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621d91187e0 0x5621d9189d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:34.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.724+0000 7f8ad36aeb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621d912f1b0 con 0x5621d910d930 2024-02-02T15:45:34.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.724+0000 7f8ad36aeb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621d918a280 con 0x5621d910d010 2024-02-02T15:45:34.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.724+0000 7f8ad36aeb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621d918a3c0 con 0x5621d91187e0 2024-02-02T15:45:34.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.724+0000 7f8ad36aeb80 10 client.0 _ll_register_callbacks cb 0x5621d8ffc290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:45:34.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.724+0000 7f8ab7e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621d910d930 0x5621d912e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:34.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.724+0000 7f8abdc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621d91187e0 0x5621d9189d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:34.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.724+0000 7f8ab7e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621d910d930 0x5621d912e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57774/0 (socket says 172.21.15.1:57774) 2024-02-02T15:45:34.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.724+0000 7f8ab7e00700 1 -- 172.21.15.1:0/656615449 learned_addr learned my addr 172.21.15.1:0/656615449 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:45:34.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.724+0000 7f8abdc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621d91187e0 0x5621d9189d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:36994/0 (socket says 172.21.15.1:36994) 2024-02-02T15:45:34.727 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.724+0000 7f8ab7e00700 1 -- 172.21.15.1:0/656615449 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621d91187e0 msgr2=0x5621d9189d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:45:34.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.724+0000 7f8ab7e00700 1 --2- 172.21.15.1:0/656615449 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621d91187e0 0x5621d9189d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:34.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.724+0000 7f8ab7e00700 1 -- 172.21.15.1:0/656615449 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5621d910d010 msgr2=0x5621d912e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:45:34.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.724+0000 7f8ab7e00700 1 --2- 172.21.15.1:0/656615449 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5621d910d010 0x5621d912e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:34.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.724+0000 7f8ab7e00700 1 -- 172.21.15.1:0/656615449 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5621d8e8cb60 con 0x5621d910d930 2024-02-02T15:45:34.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.724+0000 7f8abdc00700 1 --2- 172.21.15.1:0/656615449 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621d91187e0 0x5621d9189d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:45:34.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.725+0000 7f8ab7e00700 1 --2- 172.21.15.1:0/656615449 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621d910d930 0x5621d912e8a0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f8aa0005950 tx=0x7f8aa0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6451dc16e37562d server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:45:34.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.725+0000 7f8aa6a00700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:45:34.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.725+0000 7f8aa6a00700 1 -- 172.21.15.1:0/656615449 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa0017040 con 0x5621d910d930 2024-02-02T15:45:34.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.725+0000 7f8aa6a00700 1 -- 172.21.15.1:0/656615449 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f8aa0011910 con 0x5621d910d930 2024-02-02T15:45:34.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.725+0000 7f8ad36aeb80 1 -- 172.21.15.1:0/656615449 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8aa800ecd0 con 0x5621d910d930 2024-02-02T15:45:34.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.726+0000 7f8aa6a00700 1 -- 172.21.15.1:0/656615449 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8aa001b5b0 con 0x5621d910d930 2024-02-02T15:45:34.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.726+0000 7f8ad36aeb80 10 client.20085 fetch_fsmap learned FSMap version 78 2024-02-02T15:45:34.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.726+0000 7f8ad36aeb80 1 -- 172.21.15.1:0/656615449 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) v3 -- 0x5621d9024ea0 con 0x5621d910d930 2024-02-02T15:45:34.728 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.726+0000 7f8aa6a00700 1 -- 172.21.15.1:0/656615449 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8aa800ecd0 con 0x5621d910d930 2024-02-02T15:45:34.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.727+0000 7f8aa6a00700 1 -- 172.21.15.1:0/656615449 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 78) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f8aa0024440 con 0x5621d910d930 2024-02-02T15:45:34.729 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.727+0000 7f8ad36aeb80 10 client.20085 fetch_fsmap finished waiting for FSMap version 78 2024-02-02T15:45:34.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.727+0000 7f8ad36aeb80 10 client.20085 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-02T15:45:34.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.727+0000 7f8ad36aeb80 20 client.20085 populate_metadata read hostname 'smithi001' 2024-02-02T15:45:34.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.727+0000 7f8ad36aeb80 1 --2- 172.21.15.1:0/656615449 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x5621d918bbb0 0x5621d918dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:34.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.727+0000 7f8ad36aeb80 4 client.20085 mds_command: new command op to 24529 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:45:34.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.727+0000 7f8ad36aeb80 1 -- 172.21.15.1:0/656615449 --> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5621d901df50 con 0x5621d918bbb0 2024-02-02T15:45:34.730 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.728+0000 7f8abdc00700 1 --2- 172.21.15.1:0/656615449 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x5621d918bbb0 0x5621d918dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:34.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.728+0000 7f8abdc00700 1 --2- 172.21.15.1:0/656615449 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x5621d918bbb0 0x5621d918dfd0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=853d0b05c04b544d server_cookie=7c31ef2673d19c44 in_seq=0 out_seq=0 2024-02-02T15:45:34.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.728+0000 7f8aa6a00700 10 client.20085 ms_handle_connect on v2:172.21.15.142:6838/321675421 2024-02-02T15:45:34.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.728+0000 7f8aa6a00700 1 -- 172.21.15.1:0/656615449 <== mds.1 v2:172.21.15.142:6838/321675421 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5621d901df50 con 0x5621d918bbb0 2024-02-02T15:45:34.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.728+0000 7f8aa6a00700 10 client.20085 handle_command_reply: tid=0 2024-02-02T15:45:34.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.728+0000 7f8ad36aeb80 1 client.20085 shutdown 2024-02-02T15:45:34.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.729+0000 7f8ad36aeb80 1 -- 172.21.15.1:0/656615449 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621d910d930 msgr2=0x5621d912e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:34.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.729+0000 7f8ad36aeb80 1 --2- 172.21.15.1:0/656615449 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621d910d930 0x5621d912e8a0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f8aa0005950 tx=0x7f8aa0004750 comp rx=0 tx=0).stop 2024-02-02T15:45:34.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.729+0000 7f8ad36aeb80 1 -- 172.21.15.1:0/656615449 shutdown_connections 2024-02-02T15:45:34.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.729+0000 7f8ad36aeb80 1 --2- 172.21.15.1:0/656615449 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x5621d918bbb0 0x5621d918dfd0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:34.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.729+0000 7f8ad36aeb80 1 --2- 172.21.15.1:0/656615449 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5621d91187e0 0x5621d9189d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:34.731 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.729+0000 7f8ad36aeb80 1 --2- 172.21.15.1:0/656615449 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5621d910d010 0x5621d912e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:34.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.729+0000 7f8aa6a00700 0 client.20085 ms_handle_reset on v2:172.21.15.142:6838/321675421 2024-02-02T15:45:34.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.729+0000 7f8ad36aeb80 1 --2- 172.21.15.1:0/656615449 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5621d910d930 0x5621d912e8a0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:34.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.729+0000 7f8ad36aeb80 1 -- 172.21.15.1:0/656615449 >> 172.21.15.1:0/656615449 conn(0x5621d906cdb0 msgr2=0x5621d9116d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:45:34.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.729+0000 7f8ad36aeb80 1 -- 172.21.15.1:0/656615449 shutdown_connections 2024-02-02T15:45:34.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.729+0000 7f8ad36aeb80 1 -- 172.21.15.1:0/656615449 wait complete. 2024-02-02T15:45:34.732 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.729+0000 7f8ad36aeb80 20 client.20085 trim_cache size 0 max 16384 2024-02-02T15:45:34.732 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:45:34.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.730+0000 7f8abfc00700 1 -- 172.21.15.1:0/3466623508 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f8a980a0770 msgr2=0x7f8a980a2c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:34.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.730+0000 7f8abfc00700 1 --2- 172.21.15.1:0/3466623508 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f8a980a0770 0x7f8a980a2c30 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f8ab0007b20 tx=0x7f8ab0009040 comp rx=0 tx=0).stop 2024-02-02T15:45:34.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.730+0000 7f8abfc00700 1 -- 172.21.15.1:0/3466623508 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ab8108310 msgr2=0x7f8ab81153f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:34.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.730+0000 7f8abfc00700 1 --2- 172.21.15.1:0/3466623508 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ab8108310 0x7f8ab81153f0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f8ab00062a0 tx=0x7f8ab00079b0 comp rx=0 tx=0).stop 2024-02-02T15:45:34.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.731+0000 7f8abfc00700 1 -- 172.21.15.1:0/3466623508 shutdown_connections 2024-02-02T15:45:34.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.731+0000 7f8abfc00700 1 --2- 172.21.15.1:0/3466623508 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f8a980a0770 0x7f8a980a2c30 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:34.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.731+0000 7f8abfc00700 1 --2- 172.21.15.1:0/3466623508 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8ab8115e70 0x7f8ab8112940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:34.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.731+0000 7f8abfc00700 1 --2- 172.21.15.1:0/3466623508 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ab8108310 0x7f8ab81153f0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:34.733 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.731+0000 7f8abfc00700 1 --2- 172.21.15.1:0/3466623508 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ab810b4d0 0x7f8ab8112400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:34.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.731+0000 7f8abfc00700 1 -- 172.21.15.1:0/3466623508 >> 172.21.15.1:0/3466623508 conn(0x7f8ab806d120 msgr2=0x7f8ab8117430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:45:34.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.731+0000 7f8abfc00700 1 -- 172.21.15.1:0/3466623508 shutdown_connections 2024-02-02T15:45:34.734 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:34.731+0000 7f8abfc00700 1 -- 172.21.15.1:0/3466623508 wait complete. 2024-02-02T15:45:34.755 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-02T15:45:35.006 INFO:tasks.workunit.client.0.smithi001.stdout: 10 165131 27.52 MB/sec execute 187 sec latency 69.850 ms 2024-02-02T15:45:35.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.042+0000 7faa31c00700 1 -- 172.21.15.1:0/1131259759 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa2c111d90 msgr2=0x7faa2c114180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:35.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.042+0000 7faa31c00700 1 --2- 172.21.15.1:0/1131259759 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa2c111d90 0x7faa2c114180 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7faa2400b0a0 tx=0x7faa2400b3b0 comp rx=0 tx=0).stop 2024-02-02T15:45:35.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.042+0000 7faa31c00700 1 -- 172.21.15.1:0/1131259759 shutdown_connections 2024-02-02T15:45:35.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.042+0000 7faa31c00700 1 --2- 172.21.15.1:0/1131259759 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa2c111d90 0x7faa2c114180 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:35.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.042+0000 7faa31c00700 1 --2- 172.21.15.1:0/1131259759 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa2c0730e0 0x7faa2c0734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:35.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.042+0000 7faa31c00700 1 --2- 172.21.15.1:0/1131259759 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa2c073a00 0x7faa2c111850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:35.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.042+0000 7faa31c00700 1 -- 172.21.15.1:0/1131259759 >> 172.21.15.1:0/1131259759 conn(0x7faa2c0fc810 msgr2=0x7faa2c0fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:45:35.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.042+0000 7faa31c00700 1 -- 172.21.15.1:0/1131259759 shutdown_connections 2024-02-02T15:45:35.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.042+0000 7faa31c00700 1 -- 172.21.15.1:0/1131259759 wait complete. 2024-02-02T15:45:35.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.044+0000 7faa31c00700 1 Processor -- start 2024-02-02T15:45:35.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.044+0000 7faa31c00700 1 -- start start 2024-02-02T15:45:35.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.044+0000 7faa31c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa2c0730e0 0x7faa2c1a3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:35.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.044+0000 7faa31c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa2c073a00 0x7faa2c1a3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:35.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.045+0000 7faa31c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa2c111d90 0x7faa2c19d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:35.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.045+0000 7faa31c00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa2c19dc70 con 0x7faa2c073a00 2024-02-02T15:45:35.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.045+0000 7faa31c00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa2c19dde0 con 0x7faa2c0730e0 2024-02-02T15:45:35.045 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.045+0000 7faa31c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa2c19dfa0 con 0x7faa2c111d90 2024-02-02T15:45:35.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.045+0000 7faa2aa00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa2c0730e0 0x7faa2c1a3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:35.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.045+0000 7faa2aa00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa2c0730e0 0x7faa2c1a3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:49308/0 (socket says 172.21.15.1:49308) 2024-02-02T15:45:35.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.045+0000 7faa2aa00700 1 -- 172.21.15.1:0/229772450 learned_addr learned my addr 172.21.15.1:0/229772450 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:45:35.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.045+0000 7faa2a000700 1 --2- 172.21.15.1:0/229772450 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa2c073a00 0x7faa2c1a3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:35.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.046+0000 7faa2a000700 1 -- 172.21.15.1:0/229772450 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa2c111d90 msgr2=0x7faa2c19d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:35.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.046+0000 7faa2a000700 1 --2- 172.21.15.1:0/229772450 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa2c111d90 0x7faa2c19d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:35.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.046+0000 7faa2a000700 1 -- 172.21.15.1:0/229772450 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa2c0730e0 msgr2=0x7faa2c1a3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:35.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.046+0000 7faa2a000700 1 --2- 172.21.15.1:0/229772450 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa2c0730e0 0x7faa2c1a3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:35.047 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.046+0000 7faa2a000700 1 -- 172.21.15.1:0/229772450 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa24009d00 con 0x7faa2c073a00 2024-02-02T15:45:35.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.046+0000 7faa2a000700 1 --2- 172.21.15.1:0/229772450 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa2c073a00 0x7faa2c1a3ad0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7faa1400b450 tx=0x7faa1400b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:45:35.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.046+0000 7faa23400700 1 -- 172.21.15.1:0/229772450 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa14011830 con 0x7faa2c073a00 2024-02-02T15:45:35.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.046+0000 7faa23400700 1 -- 172.21.15.1:0/229772450 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa14011e50 con 0x7faa2c073a00 2024-02-02T15:45:35.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.046+0000 7faa23400700 1 -- 172.21.15.1:0/229772450 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa14010e40 con 0x7faa2c073a00 2024-02-02T15:45:35.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.047+0000 7faa31c00700 1 -- 172.21.15.1:0/229772450 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa2c19e280 con 0x7faa2c073a00 2024-02-02T15:45:35.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.047+0000 7faa31c00700 1 -- 172.21.15.1:0/229772450 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa2c1aa580 con 0x7faa2c073a00 2024-02-02T15:45:35.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.051+0000 7faa23400700 1 -- 172.21.15.1:0/229772450 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7faa14011990 con 0x7faa2c073a00 2024-02-02T15:45:35.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.052+0000 7faa23400700 1 --2- 172.21.15.1:0/229772450 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7faa0c0a0750 0x7faa0c0a2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:35.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.052+0000 7faa23400700 1 -- 172.21.15.1:0/229772450 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 9453+0+0 (secure 0 0 0) 0x7faa140cba10 con 0x7faa2c073a00 2024-02-02T15:45:35.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.053+0000 7faa2aa00700 1 --2- 172.21.15.1:0/229772450 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7faa0c0a0750 0x7faa0c0a2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:35.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.053+0000 7faa2aa00700 1 --2- 172.21.15.1:0/229772450 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7faa0c0a0750 0x7faa0c0a2c10 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7faa1c0098f0 tx=0x7faa1c008040 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:45:35.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.054+0000 7faa4579cb80 1 Processor -- start 2024-02-02T15:45:35.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.054+0000 7faa4579cb80 1 -- start start 2024-02-02T15:45:35.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.054+0000 7faa4579cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5610d2c96010 0x5610d2c963f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:35.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.054+0000 7faa4579cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5610d2c96930 0x5610d2ca12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:35.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.054+0000 7faa4579cb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5610d2ca17e0 0x5610d2ca3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:35.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.055+0000 7faa2a000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5610d2c96930 0x5610d2ca12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:35.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.055+0000 7faa4579cb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610d2c0a4c0 con 0x5610d2ca17e0 2024-02-02T15:45:35.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.055+0000 7faa4579cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610d2beeaa0 con 0x5610d2c96010 2024-02-02T15:45:35.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.055+0000 7faa4579cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610d2bec460 con 0x5610d2c96930 2024-02-02T15:45:35.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.055+0000 7faa2a000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5610d2c96930 0x5610d2ca12a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:37010/0 (socket says 172.21.15.1:37010) 2024-02-02T15:45:35.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.055+0000 7faa2a000700 1 -- 172.21.15.1:0/422571998 learned_addr learned my addr 172.21.15.1:0/422571998 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:45:35.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.055+0000 7faa2aa00700 1 --2- 172.21.15.1:0/422571998 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5610d2c96010 0x5610d2c963f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:35.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.055+0000 7faa2b400700 1 --2- 172.21.15.1:0/422571998 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5610d2ca17e0 0x5610d2ca3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:35.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.055+0000 7faa2a000700 1 -- 172.21.15.1:0/422571998 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5610d2c96010 msgr2=0x5610d2c963f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:45:35.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.055+0000 7faa2a000700 1 --2- 172.21.15.1:0/422571998 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5610d2c96010 0x5610d2c963f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:35.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.055+0000 7faa2a000700 1 -- 172.21.15.1:0/422571998 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5610d2ca17e0 msgr2=0x5610d2ca3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:45:35.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.055+0000 7faa2a000700 1 --2- 172.21.15.1:0/422571998 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5610d2ca17e0 0x5610d2ca3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:35.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.055+0000 7faa2a000700 1 -- 172.21.15.1:0/422571998 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5610d2a693b0 con 0x5610d2c96930 2024-02-02T15:45:35.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.055+0000 7faa2b400700 1 --2- 172.21.15.1:0/422571998 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5610d2ca17e0 0x5610d2ca3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:45:35.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.056+0000 7faa2a000700 1 --2- 172.21.15.1:0/422571998 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5610d2c96930 0x5610d2ca12a0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7faa14095140 tx=0x7faa140985a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1cd42bd4e7eb4ca server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:45:35.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.056+0000 7faa1be00700 1 -- 172.21.15.1:0/422571998 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa14099340 con 0x5610d2c96930 2024-02-02T15:45:35.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.056+0000 7faa1be00700 1 -- 172.21.15.1:0/422571998 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa14098c40 con 0x5610d2c96930 2024-02-02T15:45:35.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.056+0000 7faa1be00700 1 -- 172.21.15.1:0/422571998 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa140b80b0 con 0x5610d2c96930 2024-02-02T15:45:35.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.057+0000 7faa4579cb80 1 -- 172.21.15.1:0/422571998 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5610d2c96930 msgr2=0x5610d2ca12a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:35.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.057+0000 7faa4579cb80 1 --2- 172.21.15.1:0/422571998 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5610d2c96930 0x5610d2ca12a0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7faa14095140 tx=0x7faa140985a0 comp rx=0 tx=0).stop 2024-02-02T15:45:35.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.057+0000 7faa4579cb80 1 -- 172.21.15.1:0/422571998 shutdown_connections 2024-02-02T15:45:35.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.057+0000 7faa4579cb80 1 --2- 172.21.15.1:0/422571998 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5610d2ca17e0 0x5610d2ca3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:35.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.057+0000 7faa4579cb80 1 --2- 172.21.15.1:0/422571998 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5610d2c96010 0x5610d2c963f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:35.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.057+0000 7faa4579cb80 1 --2- 172.21.15.1:0/422571998 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5610d2c96930 0x5610d2ca12a0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:35.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.057+0000 7faa4579cb80 1 -- 172.21.15.1:0/422571998 >> 172.21.15.1:0/422571998 conn(0x5610d2bf5db0 msgr2=0x5610d2c923b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:45:35.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.057+0000 7faa4579cb80 1 -- 172.21.15.1:0/422571998 shutdown_connections 2024-02-02T15:45:35.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.057+0000 7faa4579cb80 1 -- 172.21.15.1:0/422571998 wait complete. 2024-02-02T15:45:35.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.058+0000 7faa4579cb80 1 -- start start 2024-02-02T15:45:35.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.058+0000 7faa4579cb80 1 Processor -- start 2024-02-02T15:45:35.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.059+0000 7faa4579cb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5610d2c96010 0x5610d2c9e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:35.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.059+0000 7faa4579cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5610d2c96930 0x5610d2c9eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:35.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.059+0000 7faa4579cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5610d2ca17e0 0x5610d2c9f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:35.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.059+0000 7faa4579cb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610d2be7d50 con 0x5610d2c96010 2024-02-02T15:45:35.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.059+0000 7faa4579cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610d2b83d80 con 0x5610d2ca17e0 2024-02-02T15:45:35.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.059+0000 7faa4579cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610d2b83a20 con 0x5610d2c96930 2024-02-02T15:45:35.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.059+0000 7faa4579cb80 10 client.0 _ll_register_callbacks cb 0x5610d2b85290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:45:35.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.059+0000 7faa2b400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5610d2ca17e0 0x5610d2c9f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:35.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.059+0000 7faa2b400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5610d2ca17e0 0x5610d2c9f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:49322/0 (socket says 172.21.15.1:49322) 2024-02-02T15:45:35.060 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.059+0000 7faa2b400700 1 -- 172.21.15.1:0/2026759889 learned_addr learned my addr 172.21.15.1:0/2026759889 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:45:35.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.059+0000 7faa2a000700 1 --2- 172.21.15.1:0/2026759889 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5610d2c96930 0x5610d2c9eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:35.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.059+0000 7faa2aa00700 1 --2- 172.21.15.1:0/2026759889 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5610d2c96010 0x5610d2c9e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:35.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.060+0000 7faa2a000700 1 -- 172.21.15.1:0/2026759889 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5610d2ca17e0 msgr2=0x5610d2c9f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:45:35.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.060+0000 7faa2a000700 1 --2- 172.21.15.1:0/2026759889 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5610d2ca17e0 0x5610d2c9f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:35.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.060+0000 7faa2a000700 1 -- 172.21.15.1:0/2026759889 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5610d2c96010 msgr2=0x5610d2c9e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:45:35.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.060+0000 7faa2a000700 1 --2- 172.21.15.1:0/2026759889 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5610d2c96010 0x5610d2c9e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:35.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.060+0000 7faa2a000700 1 -- 172.21.15.1:0/2026759889 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa2c1aa580 con 0x5610d2c96930 2024-02-02T15:45:35.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.060+0000 7faa2a000700 1 --2- 172.21.15.1:0/2026759889 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5610d2c96930 0x5610d2c9eaf0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7faa14098bd0 tx=0x7faa14099340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=328118fea91c6de server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:45:35.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.060+0000 7faa18c00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:45:35.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.060+0000 7faa18c00700 1 -- 172.21.15.1:0/2026759889 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa140b82b0 con 0x5610d2c96930 2024-02-02T15:45:35.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.060+0000 7faa18c00700 1 -- 172.21.15.1:0/2026759889 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 9453+0+0 (secure 0 0 0) 0x7faa140a0cb0 con 0x5610d2c96930 2024-02-02T15:45:35.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.060+0000 7faa4579cb80 1 -- 172.21.15.1:0/2026759889 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5610d2c8a0f0 con 0x5610d2c96930 2024-02-02T15:45:35.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.060+0000 7faa18c00700 1 -- 172.21.15.1:0/2026759889 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7faa14098ea0 con 0x5610d2c96930 2024-02-02T15:45:35.061 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.060+0000 7faa4579cb80 10 client.30244 fetch_fsmap learned FSMap version 78 2024-02-02T15:45:35.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.061+0000 7faa4579cb80 1 -- 172.21.15.1:0/2026759889 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) v3 -- 0x5610d2a15b60 con 0x5610d2c96930 2024-02-02T15:45:35.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.061+0000 7faa18c00700 1 -- 172.21.15.1:0/2026759889 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa1407b080 con 0x5610d2c96930 2024-02-02T15:45:35.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.061+0000 7faa18c00700 1 -- 172.21.15.1:0/2026759889 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 78) v1 ==== 7506+0+0 (secure 0 0 0) 0x7faa14077d20 con 0x5610d2c96930 2024-02-02T15:45:35.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.061+0000 7faa4579cb80 10 client.30244 fetch_fsmap finished waiting for FSMap version 78 2024-02-02T15:45:35.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.061+0000 7faa4579cb80 10 client.30244 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:45:35.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.061+0000 7faa4579cb80 20 client.30244 populate_metadata read hostname 'smithi001' 2024-02-02T15:45:35.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.061+0000 7faa4579cb80 1 --2- 172.21.15.1:0/2026759889 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5610d2cb0d50 0x5610d2cb3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:35.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.061+0000 7faa4579cb80 4 client.30244 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:45:35.063 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.061+0000 7faa4579cb80 1 -- 172.21.15.1:0/2026759889 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5610d2be0dd0 con 0x5610d2cb0d50 2024-02-02T15:45:35.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.064+0000 7faa2aa00700 1 --2- 172.21.15.1:0/2026759889 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5610d2cb0d50 0x5610d2cb3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:35.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.065+0000 7faa2aa00700 1 --2- 172.21.15.1:0/2026759889 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5610d2cb0d50 0x5610d2cb3140 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6085610aa73817c9 server_cookie=8670a68081c63817 in_seq=0 out_seq=0 2024-02-02T15:45:35.065 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.065+0000 7faa18c00700 10 client.30244 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:45:35.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.066+0000 7faa18c00700 1 -- 172.21.15.1:0/2026759889 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5610d2be0dd0 con 0x5610d2cb0d50 2024-02-02T15:45:35.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.066+0000 7faa18c00700 10 client.30244 handle_command_reply: tid=0 2024-02-02T15:45:35.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.067+0000 7faa4579cb80 1 client.30244 shutdown 2024-02-02T15:45:35.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.067+0000 7faa4579cb80 1 -- 172.21.15.1:0/2026759889 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5610d2c96930 msgr2=0x5610d2c9eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:35.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.067+0000 7faa4579cb80 1 --2- 172.21.15.1:0/2026759889 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5610d2c96930 0x5610d2c9eaf0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7faa14098bd0 tx=0x7faa14099340 comp rx=0 tx=0).stop 2024-02-02T15:45:35.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.067+0000 7faa4579cb80 1 -- 172.21.15.1:0/2026759889 shutdown_connections 2024-02-02T15:45:35.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.067+0000 7faa4579cb80 1 --2- 172.21.15.1:0/2026759889 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5610d2cb0d50 0x5610d2cb3140 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:35.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.067+0000 7faa4579cb80 1 --2- 172.21.15.1:0/2026759889 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5610d2ca17e0 0x5610d2c9f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:35.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.067+0000 7faa18c00700 0 client.30244 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:45:35.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.067+0000 7faa4579cb80 1 --2- 172.21.15.1:0/2026759889 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5610d2c96010 0x5610d2c9e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:35.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.067+0000 7faa4579cb80 1 --2- 172.21.15.1:0/2026759889 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5610d2c96930 0x5610d2c9eaf0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:35.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.067+0000 7faa4579cb80 1 -- 172.21.15.1:0/2026759889 >> 172.21.15.1:0/2026759889 conn(0x5610d2bf5db0 msgr2=0x5610d2c91fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:45:35.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.067+0000 7faa4579cb80 1 -- 172.21.15.1:0/2026759889 shutdown_connections 2024-02-02T15:45:35.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.067+0000 7faa4579cb80 1 -- 172.21.15.1:0/2026759889 wait complete. 2024-02-02T15:45:35.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.068+0000 7faa4579cb80 20 client.30244 trim_cache size 0 max 16384 2024-02-02T15:45:35.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.083+0000 7faa4579cb80 1 Processor -- start 2024-02-02T15:45:35.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.083+0000 7faa4579cb80 1 -- start start 2024-02-02T15:45:35.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.083+0000 7faa4579cb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5610d2c96010 0x5610d2ca8320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:35.083 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.084+0000 7faa4579cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5610d2c96930 0x5610d2c9d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:35.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.084+0000 7faa4579cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5610d2ca17e0 0x5610d2c998e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:35.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.084+0000 7faa4579cb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610d2a6b6a0 con 0x5610d2c96010 2024-02-02T15:45:35.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.084+0000 7faa4579cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610d2c8e9e0 con 0x5610d2ca17e0 2024-02-02T15:45:35.084 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.084+0000 7faa4579cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610d2c89ce0 con 0x5610d2c96930 2024-02-02T15:45:35.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.084+0000 7faa2a000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5610d2c96930 0x5610d2c9d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:35.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.084+0000 7faa2a000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5610d2c96930 0x5610d2c9d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:37030/0 (socket says 172.21.15.1:37030) 2024-02-02T15:45:35.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.084+0000 7faa2a000700 1 -- 172.21.15.1:0/1348955251 learned_addr learned my addr 172.21.15.1:0/1348955251 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:45:35.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.084+0000 7faa2aa00700 1 --2- 172.21.15.1:0/1348955251 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5610d2c96010 0x5610d2ca8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:35.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.084+0000 7faa2b400700 1 --2- 172.21.15.1:0/1348955251 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5610d2ca17e0 0x5610d2c998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:35.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.084+0000 7faa2a000700 1 -- 172.21.15.1:0/1348955251 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5610d2ca17e0 msgr2=0x5610d2c998e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:45:35.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.084+0000 7faa2a000700 1 --2- 172.21.15.1:0/1348955251 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5610d2ca17e0 0x5610d2c998e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:35.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.084+0000 7faa2a000700 1 -- 172.21.15.1:0/1348955251 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5610d2c96010 msgr2=0x5610d2ca8320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:45:35.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.084+0000 7faa2a000700 1 --2- 172.21.15.1:0/1348955251 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5610d2c96010 0x5610d2ca8320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:35.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.084+0000 7faa2a000700 1 -- 172.21.15.1:0/1348955251 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5610d2c9e120 con 0x5610d2c96930 2024-02-02T15:45:35.086 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.085+0000 7faa2a000700 1 --2- 172.21.15.1:0/1348955251 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5610d2c96930 0x5610d2c9d840 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7faa14095140 tx=0x7faa140985a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2540e966fc5ad0e server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:45:35.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.085+0000 7faa20c00700 1 -- 172.21.15.1:0/1348955251 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa140b82e0 con 0x5610d2c96930 2024-02-02T15:45:35.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.085+0000 7faa20c00700 1 -- 172.21.15.1:0/1348955251 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5610d2c8a0f0 con 0x5610d2c96930 2024-02-02T15:45:35.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.085+0000 7faa20c00700 1 -- 172.21.15.1:0/1348955251 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa14098370 con 0x5610d2c96930 2024-02-02T15:45:35.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.086+0000 7faa4579cb80 1 -- 172.21.15.1:0/1348955251 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5610d2c96930 msgr2=0x5610d2c9d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:35.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.086+0000 7faa4579cb80 1 --2- 172.21.15.1:0/1348955251 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5610d2c96930 0x5610d2c9d840 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7faa14095140 tx=0x7faa140985a0 comp rx=0 tx=0).stop 2024-02-02T15:45:35.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.086+0000 7faa4579cb80 1 -- 172.21.15.1:0/1348955251 shutdown_connections 2024-02-02T15:45:35.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.086+0000 7faa4579cb80 1 --2- 172.21.15.1:0/1348955251 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5610d2ca17e0 0x5610d2c998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:35.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.086+0000 7faa4579cb80 1 --2- 172.21.15.1:0/1348955251 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5610d2c96010 0x5610d2ca8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:35.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.086+0000 7faa4579cb80 1 --2- 172.21.15.1:0/1348955251 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5610d2c96930 0x5610d2c9d840 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:35.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.086+0000 7faa4579cb80 1 -- 172.21.15.1:0/1348955251 >> 172.21.15.1:0/1348955251 conn(0x5610d2bf5db0 msgr2=0x5610d2cac900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:45:35.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.086+0000 7faa4579cb80 1 -- 172.21.15.1:0/1348955251 shutdown_connections 2024-02-02T15:45:35.087 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.086+0000 7faa4579cb80 1 -- 172.21.15.1:0/1348955251 wait complete. 2024-02-02T15:45:35.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.086+0000 7faa4579cb80 1 -- start start 2024-02-02T15:45:35.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.087+0000 7faa4579cb80 1 Processor -- start 2024-02-02T15:45:35.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.087+0000 7faa4579cb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5610d2c96010 0x5610d2ca6310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:35.090 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.087+0000 7faa4579cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5610d2c96930 0x5610d2ca6850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:35.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.087+0000 7faa4579cb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5610d2ca17e0 0x5610d2ca6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:35.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.087+0000 7faa4579cb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610d2d01890 con 0x5610d2ca17e0 2024-02-02T15:45:35.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.087+0000 7faa4579cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610d2d01a00 con 0x5610d2c96930 2024-02-02T15:45:35.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.087+0000 7faa4579cb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5610d2d01bc0 con 0x5610d2c96010 2024-02-02T15:45:35.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.087+0000 7faa4579cb80 10 client.0 _ll_register_callbacks cb 0x5610d2b85290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:45:35.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.088+0000 7faa2aa00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5610d2c96010 0x5610d2ca6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:35.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.088+0000 7faa2a000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5610d2c96930 0x5610d2ca6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:35.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.088+0000 7faa2a000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5610d2c96930 0x5610d2ca6850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:49350/0 (socket says 172.21.15.1:49350) 2024-02-02T15:45:35.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.088+0000 7faa2a000700 1 -- 172.21.15.1:0/3799542099 learned_addr learned my addr 172.21.15.1:0/3799542099 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:45:35.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.088+0000 7faa2aa00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5610d2c96010 0x5610d2ca6310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:37040/0 (socket says 172.21.15.1:37040) 2024-02-02T15:45:35.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.088+0000 7faa2b400700 1 --2- 172.21.15.1:0/3799542099 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5610d2ca17e0 0x5610d2ca6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:35.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.088+0000 7faa2a000700 1 -- 172.21.15.1:0/3799542099 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5610d2c96010 msgr2=0x5610d2ca6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:45:35.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.088+0000 7faa2a000700 1 --2- 172.21.15.1:0/3799542099 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5610d2c96010 0x5610d2ca6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:35.091 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.088+0000 7faa2a000700 1 -- 172.21.15.1:0/3799542099 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5610d2ca17e0 msgr2=0x5610d2ca6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:45:35.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.088+0000 7faa2a000700 1 --2- 172.21.15.1:0/3799542099 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5610d2ca17e0 0x5610d2ca6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:35.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.088+0000 7faa2a000700 1 -- 172.21.15.1:0/3799542099 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5610d2a15b60 con 0x5610d2c96930 2024-02-02T15:45:35.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.088+0000 7faa2b400700 1 --2- 172.21.15.1:0/3799542099 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5610d2ca17e0 0x5610d2ca6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:45:35.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.088+0000 7faa2a000700 1 --2- 172.21.15.1:0/3799542099 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5610d2c96930 0x5610d2ca6850 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7faa14098bd0 tx=0x7faa14099340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7941a0e3a24ef288 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:45:35.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.088+0000 7faa18c00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:45:35.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.088+0000 7faa18c00700 1 -- 172.21.15.1:0/3799542099 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa140990a0 con 0x5610d2c96930 2024-02-02T15:45:35.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.088+0000 7faa4579cb80 1 -- 172.21.15.1:0/3799542099 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faa14098c40 con 0x5610d2c96930 2024-02-02T15:45:35.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.088+0000 7faa18c00700 1 -- 172.21.15.1:0/3799542099 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 9453+0+0 (secure 0 0 0) 0x7faa140b11c0 con 0x5610d2c96930 2024-02-02T15:45:35.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.089+0000 7faa18c00700 1 -- 172.21.15.1:0/3799542099 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7faa140b82c0 con 0x5610d2c96930 2024-02-02T15:45:35.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.089+0000 7faa4579cb80 10 client.30386 fetch_fsmap learned FSMap version 78 2024-02-02T15:45:35.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.089+0000 7faa4579cb80 1 -- 172.21.15.1:0/3799542099 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) v3 -- 0x5610d2badea0 con 0x5610d2c96930 2024-02-02T15:45:35.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.089+0000 7faa18c00700 1 -- 172.21.15.1:0/3799542099 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faa14098c40 con 0x5610d2c96930 2024-02-02T15:45:35.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.089+0000 7faa18c00700 1 -- 172.21.15.1:0/3799542099 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 78) v1 ==== 7506+0+0 (secure 0 0 0) 0x7faa14095350 con 0x5610d2c96930 2024-02-02T15:45:35.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.089+0000 7faa4579cb80 10 client.30386 fetch_fsmap finished waiting for FSMap version 78 2024-02-02T15:45:35.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.089+0000 7faa4579cb80 10 client.30386 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:45:35.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.089+0000 7faa4579cb80 20 client.30386 populate_metadata read hostname 'smithi001' 2024-02-02T15:45:35.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.089+0000 7faa4579cb80 1 --2- 172.21.15.1:0/3799542099 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5610d2d02e60 0x5610d2d05250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:35.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.089+0000 7faa4579cb80 4 client.30386 mds_command: new command op to 24557 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:45:35.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.089+0000 7faa4579cb80 1 -- 172.21.15.1:0/3799542099 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5610d2ba6f50 con 0x5610d2d02e60 2024-02-02T15:45:35.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.089+0000 7faa2aa00700 1 --2- 172.21.15.1:0/3799542099 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5610d2d02e60 0x5610d2d05250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:35.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.090+0000 7faa2aa00700 1 --2- 172.21.15.1:0/3799542099 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5610d2d02e60 0x5610d2d05250 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bea1c77063d508c0 server_cookie=934f3d5ca15074e2 in_seq=0 out_seq=0 2024-02-02T15:45:35.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.090+0000 7faa18c00700 10 client.30386 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:45:35.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.091+0000 7faa18c00700 1 -- 172.21.15.1:0/3799542099 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5610d2ba6f50 con 0x5610d2d02e60 2024-02-02T15:45:35.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.091+0000 7faa18c00700 10 client.30386 handle_command_reply: tid=0 2024-02-02T15:45:35.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.091+0000 7faa4579cb80 1 client.30386 shutdown 2024-02-02T15:45:35.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.091+0000 7faa4579cb80 1 -- 172.21.15.1:0/3799542099 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5610d2c96930 msgr2=0x5610d2ca6850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:35.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.091+0000 7faa4579cb80 1 --2- 172.21.15.1:0/3799542099 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5610d2c96930 0x5610d2ca6850 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7faa14098bd0 tx=0x7faa14099340 comp rx=0 tx=0).stop 2024-02-02T15:45:35.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.091+0000 7faa4579cb80 1 -- 172.21.15.1:0/3799542099 shutdown_connections 2024-02-02T15:45:35.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.091+0000 7faa4579cb80 1 --2- 172.21.15.1:0/3799542099 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x5610d2d02e60 0x5610d2d05250 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:35.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.091+0000 7faa4579cb80 1 --2- 172.21.15.1:0/3799542099 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5610d2ca17e0 0x5610d2ca6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:35.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.091+0000 7faa4579cb80 1 --2- 172.21.15.1:0/3799542099 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5610d2c96010 0x5610d2ca6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:35.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.091+0000 7faa18c00700 0 client.30386 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:45:35.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.091+0000 7faa4579cb80 1 --2- 172.21.15.1:0/3799542099 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5610d2c96930 0x5610d2ca6850 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:35.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.091+0000 7faa4579cb80 1 -- 172.21.15.1:0/3799542099 >> 172.21.15.1:0/3799542099 conn(0x5610d2bf5db0 msgr2=0x5610d2c998e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:45:35.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.091+0000 7faa4579cb80 1 -- 172.21.15.1:0/3799542099 shutdown_connections 2024-02-02T15:45:35.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.091+0000 7faa4579cb80 1 -- 172.21.15.1:0/3799542099 wait complete. 2024-02-02T15:45:35.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.092+0000 7faa4579cb80 20 client.30386 trim_cache size 0 max 16384 2024-02-02T15:45:35.096 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:45:35.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.092+0000 7faa31c00700 1 -- 172.21.15.1:0/229772450 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7faa0c0a0750 msgr2=0x7faa0c0a2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:35.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.092+0000 7faa31c00700 1 --2- 172.21.15.1:0/229772450 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7faa0c0a0750 0x7faa0c0a2c10 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7faa1c0098f0 tx=0x7faa1c008040 comp rx=0 tx=0).stop 2024-02-02T15:45:35.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.092+0000 7faa31c00700 1 -- 172.21.15.1:0/229772450 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa2c073a00 msgr2=0x7faa2c1a3ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:35.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.092+0000 7faa31c00700 1 --2- 172.21.15.1:0/229772450 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa2c073a00 0x7faa2c1a3ad0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7faa1400b450 tx=0x7faa1400b810 comp rx=0 tx=0).stop 2024-02-02T15:45:35.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.092+0000 7faa31c00700 1 -- 172.21.15.1:0/229772450 shutdown_connections 2024-02-02T15:45:35.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.092+0000 7faa31c00700 1 --2- 172.21.15.1:0/229772450 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7faa0c0a0750 0x7faa0c0a2c10 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:35.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.092+0000 7faa31c00700 1 --2- 172.21.15.1:0/229772450 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7faa2c111d90 0x7faa2c19d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:35.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.092+0000 7faa31c00700 1 --2- 172.21.15.1:0/229772450 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faa2c0730e0 0x7faa2c1a3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:35.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.092+0000 7faa31c00700 1 --2- 172.21.15.1:0/229772450 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7faa2c073a00 0x7faa2c1a3ad0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:35.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.092+0000 7faa31c00700 1 -- 172.21.15.1:0/229772450 >> 172.21.15.1:0/229772450 conn(0x7faa2c0fc810 msgr2=0x7faa2c0fd480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:45:35.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.092+0000 7faa31c00700 1 -- 172.21.15.1:0/229772450 shutdown_connections 2024-02-02T15:45:35.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:35.093+0000 7faa31c00700 1 -- 172.21.15.1:0/229772450 wait complete. 2024-02-02T15:45:35.112 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:34 smithi001 ceph-mon[35470]: pgmap v584: 129 pgs: 129 active+clean; 2.2 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.8 MiB/s wr, 445 op/s 2024-02-02T15:45:35.113 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:34 smithi071 ceph-mon[44707]: pgmap v584: 129 pgs: 129 active+clean; 2.2 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.8 MiB/s wr, 445 op/s 2024-02-02T15:45:35.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:34 smithi142 ceph-mon[45728]: pgmap v584: 129 pgs: 129 active+clean; 2.2 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.8 MiB/s wr, 445 op/s 2024-02-02T15:45:36.006 INFO:tasks.workunit.client.0.smithi001.stdout: 10 165579 27.46 MB/sec execute 188 sec latency 112.435 ms 2024-02-02T15:45:36.116 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-02T15:45:36.116 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-02T15:45:36.427 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.427+0000 7f3229000700 1 -- 172.21.15.1:0/1564247630 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3224101c70 msgr2=0x7f3224102050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:36.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.427+0000 7f3229000700 1 --2- 172.21.15.1:0/1564247630 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3224101c70 0x7f3224102050 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f321400b0a0 tx=0x7f321400b3b0 comp rx=0 tx=0).stop 2024-02-02T15:45:36.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.427+0000 7f3229000700 1 -- 172.21.15.1:0/1564247630 shutdown_connections 2024-02-02T15:45:36.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.427+0000 7f3229000700 1 --2- 172.21.15.1:0/1564247630 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f322410b090 0x7f322410d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:36.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.427+0000 7f3229000700 1 --2- 172.21.15.1:0/1564247630 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3224101c70 0x7f3224102050 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:36.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.427+0000 7f3229000700 1 --2- 172.21.15.1:0/1564247630 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3224102590 0x7f322410aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:36.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.427+0000 7f3229000700 1 -- 172.21.15.1:0/1564247630 >> 172.21.15.1:0/1564247630 conn(0x7f32240fb490 msgr2=0x7f32240fd8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:45:36.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.427+0000 7f3229000700 1 -- 172.21.15.1:0/1564247630 shutdown_connections 2024-02-02T15:45:36.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.427+0000 7f3229000700 1 -- 172.21.15.1:0/1564247630 wait complete. 2024-02-02T15:45:36.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.428+0000 7f3229000700 1 Processor -- start 2024-02-02T15:45:36.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.428+0000 7f3229000700 1 -- start start 2024-02-02T15:45:36.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.429+0000 7f3229000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3224101c70 0x7f3224196d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:36.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.429+0000 7f3229000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3224102590 0x7f32241972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:36.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.429+0000 7f3229000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f322410b090 0x7f322419b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:36.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.429+0000 7f3229000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322419bd40 con 0x7f3224102590 2024-02-02T15:45:36.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.429+0000 7f3229000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322419beb0 con 0x7f322410b090 2024-02-02T15:45:36.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.429+0000 7f3229000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322419c070 con 0x7f3224101c70 2024-02-02T15:45:36.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.429+0000 7f3223e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f322410b090 0x7f322419b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:36.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.429+0000 7f3223e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f322410b090 0x7f322419b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:49376/0 (socket says 172.21.15.1:49376) 2024-02-02T15:45:36.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.429+0000 7f3223e00700 1 -- 172.21.15.1:0/2511975779 learned_addr learned my addr 172.21.15.1:0/2511975779 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:45:36.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.429+0000 7f321a800700 1 --2- 172.21.15.1:0/2511975779 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3224102590 0x7f32241972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:36.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.430+0000 7f3223e00700 1 -- 172.21.15.1:0/2511975779 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3224101c70 msgr2=0x7f3224196d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:36.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.430+0000 7f3223e00700 1 --2- 172.21.15.1:0/2511975779 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3224101c70 0x7f3224196d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:36.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.430+0000 7f3223e00700 1 -- 172.21.15.1:0/2511975779 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3224102590 msgr2=0x7f32241972a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:36.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.430+0000 7f3223e00700 1 --2- 172.21.15.1:0/2511975779 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3224102590 0x7f32241972a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:36.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.430+0000 7f3223e00700 1 -- 172.21.15.1:0/2511975779 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3214009d00 con 0x7f322410b090 2024-02-02T15:45:36.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.430+0000 7f3223e00700 1 --2- 172.21.15.1:0/2511975779 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f322410b090 0x7f322419b700 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f321c00e940 tx=0x7f321c00ec50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:45:36.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.430+0000 7f3220c00700 1 -- 172.21.15.1:0/2511975779 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321c0097b0 con 0x7f322410b090 2024-02-02T15:45:36.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.430+0000 7f3229000700 1 -- 172.21.15.1:0/2511975779 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f322419c350 con 0x7f322410b090 2024-02-02T15:45:36.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.430+0000 7f3220c00700 1 -- 172.21.15.1:0/2511975779 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f321c00ee90 con 0x7f322410b090 2024-02-02T15:45:36.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.430+0000 7f3220c00700 1 -- 172.21.15.1:0/2511975779 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321c005520 con 0x7f322410b090 2024-02-02T15:45:36.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.431+0000 7f3229000700 1 -- 172.21.15.1:0/2511975779 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32241a3b90 con 0x7f322410b090 2024-02-02T15:45:36.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.435+0000 7f3220c00700 1 -- 172.21.15.1:0/2511975779 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f321c004750 con 0x7f322410b090 2024-02-02T15:45:36.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.437+0000 7f323e128b80 1 Processor -- start 2024-02-02T15:45:36.437 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.437+0000 7f3220c00700 1 --2- 172.21.15.1:0/2511975779 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f32040a0710 0x7f32040a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:36.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.437+0000 7f3220c00700 1 -- 172.21.15.1:0/2511975779 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f321c014070 con 0x7f322410b090 2024-02-02T15:45:36.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.437+0000 7f323e128b80 1 -- start start 2024-02-02T15:45:36.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.438+0000 7f321a800700 1 --2- 172.21.15.1:0/2511975779 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f32040a0710 0x7f32040a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:36.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.438+0000 7f323e128b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6695c8e50 0x55e6695c7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:36.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.438+0000 7f323e128b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e6695c77b0 0x55e6695c7c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:36.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.438+0000 7f323e128b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e6695db110 0x55e6695dd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:36.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.438+0000 7f323e128b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e66953b540 con 0x55e6695db110 2024-02-02T15:45:36.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.438+0000 7f323e128b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e66951fb20 con 0x55e6695c77b0 2024-02-02T15:45:36.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.438+0000 7f323e128b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e66951d4e0 con 0x55e6695c8e50 2024-02-02T15:45:36.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.438+0000 7f3223e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e6695db110 0x55e6695dd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:36.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.438+0000 7f321a800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e6695c77b0 0x55e6695c7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:36.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.438+0000 7f321a800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e6695c77b0 0x55e6695c7c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:49382/0 (socket says 172.21.15.1:49382) 2024-02-02T15:45:36.439 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.439+0000 7f321a800700 1 -- 172.21.15.1:0/1627716196 learned_addr learned my addr 172.21.15.1:0/1627716196 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:45:36.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.439+0000 7f321a800700 1 --2- 172.21.15.1:0/2511975779 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f32040a0710 0x7f32040a2bd0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f3224198370 tx=0x7f320c007320 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:45:36.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.438+0000 7f3223e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e6695db110 0x55e6695dd500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:57850/0 (socket says 172.21.15.1:57850) 2024-02-02T15:45:36.440 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.439+0000 7f321a800700 1 -- 172.21.15.1:0/1627716196 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6695c8e50 msgr2=0x55e6695c7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:45:36.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.439+0000 7f321a800700 1 --2- 172.21.15.1:0/1627716196 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6695c8e50 0x55e6695c7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:36.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.439+0000 7f321a800700 1 -- 172.21.15.1:0/1627716196 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e6695db110 msgr2=0x55e6695dd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:45:36.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.439+0000 7f321a800700 1 --2- 172.21.15.1:0/1627716196 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e6695db110 0x55e6695dd500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:36.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.439+0000 7f321a800700 1 -- 172.21.15.1:0/1627716196 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e66939a430 con 0x55e6695c77b0 2024-02-02T15:45:36.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.440+0000 7f321a800700 1 --2- 172.21.15.1:0/1627716196 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e6695c77b0 0x55e6695c7c30 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f320c010e40 tx=0x7f320c013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9cdf6675aa046c91 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:45:36.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.440+0000 7f3218a00700 1 -- 172.21.15.1:0/1627716196 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f320c003680 con 0x55e6695c77b0 2024-02-02T15:45:36.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.440+0000 7f3218a00700 1 -- 172.21.15.1:0/1627716196 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f320c003ca0 con 0x55e6695c77b0 2024-02-02T15:45:36.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.440+0000 7f3218a00700 1 -- 172.21.15.1:0/1627716196 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f320c011c60 con 0x55e6695c77b0 2024-02-02T15:45:36.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.441+0000 7f323e128b80 1 -- 172.21.15.1:0/1627716196 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e6695c77b0 msgr2=0x55e6695c7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:36.441 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.441+0000 7f323e128b80 1 --2- 172.21.15.1:0/1627716196 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e6695c77b0 0x55e6695c7c30 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f320c010e40 tx=0x7f320c013000 comp rx=0 tx=0).stop 2024-02-02T15:45:36.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.441+0000 7f323e128b80 1 -- 172.21.15.1:0/1627716196 shutdown_connections 2024-02-02T15:45:36.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.441+0000 7f323e128b80 1 --2- 172.21.15.1:0/1627716196 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e6695db110 0x55e6695dd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:36.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.441+0000 7f323e128b80 1 --2- 172.21.15.1:0/1627716196 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6695c8e50 0x55e6695c7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:36.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.441+0000 7f323e128b80 1 --2- 172.21.15.1:0/1627716196 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e6695c77b0 0x55e6695c7c30 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:36.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.441+0000 7f323e128b80 1 -- 172.21.15.1:0/1627716196 >> 172.21.15.1:0/1627716196 conn(0x55e669526e30 msgr2=0x55e6695c3640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:45:36.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.441+0000 7f323e128b80 1 -- 172.21.15.1:0/1627716196 shutdown_connections 2024-02-02T15:45:36.442 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.442+0000 7f323e128b80 1 -- 172.21.15.1:0/1627716196 wait complete. 2024-02-02T15:45:36.443 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.443+0000 7f323e128b80 1 -- start start 2024-02-02T15:45:36.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.444+0000 7f323e128b80 1 Processor -- start 2024-02-02T15:45:36.446 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.446+0000 7f323e128b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e6695c77b0 0x55e6695d4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:36.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.446+0000 7f323e128b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6695c8e50 0x55e6695d4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:36.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.446+0000 7f323e128b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e6695db110 0x55e6695da3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:36.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.446+0000 7f323e128b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e669518dd0 con 0x55e6695db110 2024-02-02T15:45:36.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.446+0000 7f323e128b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6694b4e00 con 0x55e6695c77b0 2024-02-02T15:45:36.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.446+0000 7f323e128b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6694b4aa0 con 0x55e6695c8e50 2024-02-02T15:45:36.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.446+0000 7f323e128b80 10 client.0 _ll_register_callbacks cb 0x55e6694b6310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:45:36.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.446+0000 7f3223400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e6695c77b0 0x55e6695d4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:36.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.446+0000 7f3223400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e6695c77b0 0x55e6695d4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:49392/0 (socket says 172.21.15.1:49392) 2024-02-02T15:45:36.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.446+0000 7f3223400700 1 -- 172.21.15.1:0/2238658093 learned_addr learned my addr 172.21.15.1:0/2238658093 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:45:36.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.446+0000 7f3223400700 1 -- 172.21.15.1:0/2238658093 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6695c8e50 msgr2=0x55e6695d4bb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-02T15:45:36.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.447+0000 7f3223400700 1 --2- 172.21.15.1:0/2238658093 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6695c8e50 0x55e6695d4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:36.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.447+0000 7f3223400700 1 -- 172.21.15.1:0/2238658093 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e6695db110 msgr2=0x55e6695da3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:45:36.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.447+0000 7f3223400700 1 --2- 172.21.15.1:0/2238658093 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e6695db110 0x55e6695da3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:36.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.447+0000 7f3223400700 1 -- 172.21.15.1:0/2238658093 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f320c010b20 con 0x55e6695c77b0 2024-02-02T15:45:36.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.447+0000 7f3223400700 1 --2- 172.21.15.1:0/2238658093 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e6695c77b0 0x55e6695d4670 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f321400b070 tx=0x7f321400bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9bb7b27a039dd45 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:45:36.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.447+0000 7f3211600700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:45:36.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.447+0000 7f3211600700 1 -- 172.21.15.1:0/2238658093 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321400e040 con 0x55e6695c77b0 2024-02-02T15:45:36.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.447+0000 7f3211600700 1 -- 172.21.15.1:0/2238658093 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f3214004c50 con 0x55e6695c77b0 2024-02-02T15:45:36.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.448+0000 7f323e128b80 1 -- 172.21.15.1:0/2238658093 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e6695bb3c0 con 0x55e6695c77b0 2024-02-02T15:45:36.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.448+0000 7f3211600700 1 -- 172.21.15.1:0/2238658093 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3214004e10 con 0x55e6695c77b0 2024-02-02T15:45:36.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.448+0000 7f323e128b80 10 client.30398 fetch_fsmap learned FSMap version 78 2024-02-02T15:45:36.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.448+0000 7f323e128b80 1 -- 172.21.15.1:0/2238658093 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) v3 -- 0x55e669346be0 con 0x55e6695c77b0 2024-02-02T15:45:36.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.448+0000 7f3211600700 1 -- 172.21.15.1:0/2238658093 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f321402a030 con 0x55e6695c77b0 2024-02-02T15:45:36.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.449+0000 7f3211600700 1 -- 172.21.15.1:0/2238658093 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 78) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f3214019040 con 0x55e6695c77b0 2024-02-02T15:45:36.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.449+0000 7f323e128b80 10 client.30398 fetch_fsmap finished waiting for FSMap version 78 2024-02-02T15:45:36.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.449+0000 7f323e128b80 10 client.30398 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:45:36.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.449+0000 7f323e128b80 20 client.30398 populate_metadata read hostname 'smithi001' 2024-02-02T15:45:36.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.449+0000 7f323e128b80 1 --2- 172.21.15.1:0/2238658093 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55e6695f35e0 0x55e6695f5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:36.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.449+0000 7f323e128b80 4 client.30398 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:45:36.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.449+0000 7f323e128b80 1 -- 172.21.15.1:0/2238658093 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e669511e50 con 0x55e6695f35e0 2024-02-02T15:45:36.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.451+0000 7f321a800700 1 --2- 172.21.15.1:0/2238658093 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55e6695f35e0 0x55e6695f5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:36.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.451+0000 7f321a800700 1 --2- 172.21.15.1:0/2238658093 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55e6695f35e0 0x55e6695f5a00 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=999aa70f8a33a215 server_cookie=2a56569d6b4850c1 in_seq=0 out_seq=0 2024-02-02T15:45:36.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.451+0000 7f3211600700 10 client.30398 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:45:36.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.456+0000 7f3211600700 1 -- 172.21.15.1:0/2238658093 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55e669511e50 con 0x55e6695f35e0 2024-02-02T15:45:36.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.456+0000 7f3211600700 10 client.30398 handle_command_reply: tid=0 2024-02-02T15:45:36.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.456+0000 7f323e128b80 1 client.30398 shutdown 2024-02-02T15:45:36.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.456+0000 7f323e128b80 1 -- 172.21.15.1:0/2238658093 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e6695c77b0 msgr2=0x55e6695d4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:36.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.456+0000 7f323e128b80 1 --2- 172.21.15.1:0/2238658093 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e6695c77b0 0x55e6695d4670 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f321400b070 tx=0x7f321400bab0 comp rx=0 tx=0).stop 2024-02-02T15:45:36.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.456+0000 7f323e128b80 1 -- 172.21.15.1:0/2238658093 shutdown_connections 2024-02-02T15:45:36.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.456+0000 7f323e128b80 1 --2- 172.21.15.1:0/2238658093 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55e6695f35e0 0x55e6695f5a00 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:36.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.456+0000 7f323e128b80 1 --2- 172.21.15.1:0/2238658093 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e6695db110 0x55e6695da3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:36.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.456+0000 7f323e128b80 1 --2- 172.21.15.1:0/2238658093 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e6695c77b0 0x55e6695d4670 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:36.456 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.456+0000 7f323e128b80 1 --2- 172.21.15.1:0/2238658093 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6695c8e50 0x55e6695d4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:36.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.456+0000 7f323e128b80 1 -- 172.21.15.1:0/2238658093 >> 172.21.15.1:0/2238658093 conn(0x55e669526e30 msgr2=0x55e6695c2890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:45:36.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.456+0000 7f3211600700 0 client.30398 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:45:36.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.457+0000 7f323e128b80 1 -- 172.21.15.1:0/2238658093 shutdown_connections 2024-02-02T15:45:36.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.457+0000 7f323e128b80 1 -- 172.21.15.1:0/2238658093 wait complete. 2024-02-02T15:45:36.457 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.457+0000 7f323e128b80 20 client.30398 trim_cache size 0 max 16384 2024-02-02T15:45:36.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.477+0000 7f323e128b80 1 Processor -- start 2024-02-02T15:45:36.477 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.477+0000 7f323e128b80 1 -- start start 2024-02-02T15:45:36.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.478+0000 7f323e128b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e6695c77b0 0x55e6695da320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:36.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.478+0000 7f323e128b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6695c8e50 0x55e6695cf960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:36.478 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.478+0000 7f323e128b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e6695db110 0x55e6695c6200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:36.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.478+0000 7f323e128b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e66939c720 con 0x55e6695db110 2024-02-02T15:45:36.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.478+0000 7f323e128b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6695c83a0 con 0x55e6695c77b0 2024-02-02T15:45:36.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.478+0000 7f323e128b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6695dd8a0 con 0x55e6695c8e50 2024-02-02T15:45:36.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.478+0000 7f3223400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e6695c77b0 0x55e6695da320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:36.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.478+0000 7f321a800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6695c8e50 0x55e6695cf960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:36.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.478+0000 7f3223400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e6695c77b0 0x55e6695da320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:49404/0 (socket says 172.21.15.1:49404) 2024-02-02T15:45:36.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.478+0000 7f3223400700 1 -- 172.21.15.1:0/2913161224 learned_addr learned my addr 172.21.15.1:0/2913161224 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:45:36.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.478+0000 7f3223e00700 1 --2- 172.21.15.1:0/2913161224 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e6695db110 0x55e6695c6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:36.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.478+0000 7f3223400700 1 -- 172.21.15.1:0/2913161224 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6695c8e50 msgr2=0x55e6695cf960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:45:36.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.478+0000 7f3223400700 1 --2- 172.21.15.1:0/2913161224 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6695c8e50 0x55e6695cf960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:36.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.478+0000 7f3223400700 1 -- 172.21.15.1:0/2913161224 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e6695db110 msgr2=0x55e6695c6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:45:36.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.478+0000 7f3223400700 1 --2- 172.21.15.1:0/2913161224 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e6695db110 0x55e6695c6200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:36.479 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.478+0000 7f3223400700 1 -- 172.21.15.1:0/2913161224 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e6694def20 con 0x55e6695c77b0 2024-02-02T15:45:36.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.478+0000 7f3223e00700 1 --2- 172.21.15.1:0/2913161224 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e6695db110 0x55e6695c6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:45:36.480 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.479+0000 7f3223400700 1 --2- 172.21.15.1:0/2913161224 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e6695c77b0 0x55e6695da320 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f3214000f80 tx=0x7f321401dcf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3cb104e220e0a48 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:45:36.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.479+0000 7f3219400700 1 -- 172.21.15.1:0/2913161224 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321400e040 con 0x55e6695c77b0 2024-02-02T15:45:36.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.479+0000 7f3219400700 1 -- 172.21.15.1:0/2913161224 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f321402a040 con 0x55e6695c77b0 2024-02-02T15:45:36.482 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.479+0000 7f3219400700 1 -- 172.21.15.1:0/2913161224 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f321401d8d0 con 0x55e6695c77b0 2024-02-02T15:45:36.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.480+0000 7f323e128b80 1 -- 172.21.15.1:0/2913161224 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e6695c77b0 msgr2=0x55e6695da320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:36.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.480+0000 7f323e128b80 1 --2- 172.21.15.1:0/2913161224 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e6695c77b0 0x55e6695da320 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f3214000f80 tx=0x7f321401dcf0 comp rx=0 tx=0).stop 2024-02-02T15:45:36.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.480+0000 7f323e128b80 1 -- 172.21.15.1:0/2913161224 shutdown_connections 2024-02-02T15:45:36.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.480+0000 7f323e128b80 1 --2- 172.21.15.1:0/2913161224 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e6695db110 0x55e6695c6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:36.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.480+0000 7f323e128b80 1 --2- 172.21.15.1:0/2913161224 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e6695c77b0 0x55e6695da320 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:36.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.480+0000 7f323e128b80 1 --2- 172.21.15.1:0/2913161224 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6695c8e50 0x55e6695cf960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:36.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.480+0000 7f323e128b80 1 -- 172.21.15.1:0/2913161224 >> 172.21.15.1:0/2913161224 conn(0x55e669526e30 msgr2=0x55e6695cf250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:45:36.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.480+0000 7f323e128b80 1 -- 172.21.15.1:0/2913161224 shutdown_connections 2024-02-02T15:45:36.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.480+0000 7f323e128b80 1 -- 172.21.15.1:0/2913161224 wait complete. 2024-02-02T15:45:36.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.480+0000 7f323e128b80 1 -- start start 2024-02-02T15:45:36.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.481+0000 7f323e128b80 1 Processor -- start 2024-02-02T15:45:36.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.481+0000 7f323e128b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6695c77b0 0x55e6695da180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:36.483 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.481+0000 7f323e128b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e6695c8e50 0x55e6695d6420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:36.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.481+0000 7f323e128b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e6695db110 0x55e6695d6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:36.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.481+0000 7f323e128b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6695d7020 con 0x55e6695db110 2024-02-02T15:45:36.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.481+0000 7f323e128b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6695d7190 con 0x55e6695c8e50 2024-02-02T15:45:36.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.481+0000 7f323e128b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e669645100 con 0x55e6695c77b0 2024-02-02T15:45:36.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.481+0000 7f323e128b80 10 client.0 _ll_register_callbacks cb 0x55e6694b6310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:45:36.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.481+0000 7f3223400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6695c77b0 0x55e6695da180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:36.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.481+0000 7f321a800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e6695c8e50 0x55e6695d6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:36.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.481+0000 7f3223400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6695c77b0 0x55e6695da180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:37094/0 (socket says 172.21.15.1:37094) 2024-02-02T15:45:36.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.481+0000 7f3223400700 1 -- 172.21.15.1:0/2098864850 learned_addr learned my addr 172.21.15.1:0/2098864850 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:45:36.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.481+0000 7f3223e00700 1 --2- 172.21.15.1:0/2098864850 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e6695db110 0x55e6695d6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:36.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.482+0000 7f321a800700 1 -- 172.21.15.1:0/2098864850 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6695c77b0 msgr2=0x55e6695da180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:45:36.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.482+0000 7f321a800700 1 --2- 172.21.15.1:0/2098864850 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6695c77b0 0x55e6695da180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:36.484 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.482+0000 7f321a800700 1 -- 172.21.15.1:0/2098864850 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e6695db110 msgr2=0x55e6695d6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:45:36.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.482+0000 7f321a800700 1 --2- 172.21.15.1:0/2098864850 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e6695db110 0x55e6695d6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:36.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.482+0000 7f321a800700 1 -- 172.21.15.1:0/2098864850 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e669346be0 con 0x55e6695c8e50 2024-02-02T15:45:36.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.482+0000 7f321a800700 1 --2- 172.21.15.1:0/2098864850 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e6695c8e50 0x55e6695d6420 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f320c004000 tx=0x7f320c0117a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7986f3872fa28092 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:45:36.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.482+0000 7f3211600700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:45:36.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.483+0000 7f3211600700 1 -- 172.21.15.1:0/2098864850 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f320c030350 con 0x55e6695c8e50 2024-02-02T15:45:36.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.483+0000 7f323e128b80 1 -- 172.21.15.1:0/2098864850 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f320c003ca0 con 0x55e6695c8e50 2024-02-02T15:45:36.485 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.483+0000 7f3211600700 1 -- 172.21.15.1:0/2098864850 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f320c017070 con 0x55e6695c8e50 2024-02-02T15:45:36.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.483+0000 7f3211600700 1 -- 172.21.15.1:0/2098864850 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7f320c003680 con 0x55e6695c8e50 2024-02-02T15:45:36.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.483+0000 7f323e128b80 10 client.30404 fetch_fsmap learned FSMap version 78 2024-02-02T15:45:36.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.483+0000 7f323e128b80 1 -- 172.21.15.1:0/2098864850 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) v3 -- 0x55e66939ed50 con 0x55e6695c8e50 2024-02-02T15:45:36.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.483+0000 7f3211600700 1 -- 172.21.15.1:0/2098864850 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f320c003ca0 con 0x55e6695c8e50 2024-02-02T15:45:36.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.483+0000 7f3211600700 1 -- 172.21.15.1:0/2098864850 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 78) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f320c005050 con 0x55e6695c8e50 2024-02-02T15:45:36.487 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.484+0000 7f323e128b80 10 client.30404 fetch_fsmap finished waiting for FSMap version 78 2024-02-02T15:45:36.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.484+0000 7f323e128b80 10 client.30404 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:45:36.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.484+0000 7f323e128b80 20 client.30404 populate_metadata read hostname 'smithi001' 2024-02-02T15:45:36.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.484+0000 7f323e128b80 1 --2- 172.21.15.1:0/2098864850 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55e6696466f0 0x55e669648ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:36.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.484+0000 7f323e128b80 4 client.30404 mds_command: new command op to 24557 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-02T15:45:36.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.484+0000 7f323e128b80 1 -- 172.21.15.1:0/2098864850 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55e6694d7fd0 con 0x55e6696466f0 2024-02-02T15:45:36.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.484+0000 7f3223400700 1 --2- 172.21.15.1:0/2098864850 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55e6696466f0 0x55e669648ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:36.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.484+0000 7f3223400700 1 --2- 172.21.15.1:0/2098864850 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55e6696466f0 0x55e669648ae0 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46dd31edac7e4670 server_cookie=20acebec6eb69b08 in_seq=0 out_seq=0 2024-02-02T15:45:36.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.484+0000 7f3211600700 10 client.30404 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:45:36.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.487+0000 7f3211600700 1 -- 172.21.15.1:0/2098864850 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55e6694d7fd0 con 0x55e6696466f0 2024-02-02T15:45:36.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.487+0000 7f3211600700 10 client.30404 handle_command_reply: tid=0 2024-02-02T15:45:36.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.487+0000 7f323e128b80 1 client.30404 shutdown 2024-02-02T15:45:36.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.487+0000 7f323e128b80 1 -- 172.21.15.1:0/2098864850 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e6695c8e50 msgr2=0x55e6695d6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:36.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.487+0000 7f323e128b80 1 --2- 172.21.15.1:0/2098864850 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e6695c8e50 0x55e6695d6420 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f320c004000 tx=0x7f320c0117a0 comp rx=0 tx=0).stop 2024-02-02T15:45:36.488 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.487+0000 7f323e128b80 1 -- 172.21.15.1:0/2098864850 shutdown_connections 2024-02-02T15:45:36.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.487+0000 7f323e128b80 1 --2- 172.21.15.1:0/2098864850 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55e6696466f0 0x55e669648ae0 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:36.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.487+0000 7f323e128b80 1 --2- 172.21.15.1:0/2098864850 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e6695db110 0x55e6695d6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:36.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.487+0000 7f323e128b80 1 --2- 172.21.15.1:0/2098864850 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e6695c77b0 0x55e6695da180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:36.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.487+0000 7f323e128b80 1 --2- 172.21.15.1:0/2098864850 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e6695c8e50 0x55e6695d6420 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:36.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.487+0000 7f323e128b80 1 -- 172.21.15.1:0/2098864850 >> 172.21.15.1:0/2098864850 conn(0x55e669526e30 msgr2=0x55e6695c9360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:45:36.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.487+0000 7f3211600700 0 client.30404 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:45:36.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.487+0000 7f323e128b80 1 -- 172.21.15.1:0/2098864850 shutdown_connections 2024-02-02T15:45:36.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.487+0000 7f323e128b80 1 -- 172.21.15.1:0/2098864850 wait complete. 2024-02-02T15:45:36.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.487+0000 7f323e128b80 20 client.30404 trim_cache size 0 max 16384 2024-02-02T15:45:36.489 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:45:36.490 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2024-02-02T15:45:36.490 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "ea43b95a-27e9-4be7-a487-e68daa4971bc", 2024-02-02T15:45:36.490 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2024-02-02T15:45:36.490 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:45:36.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.488+0000 7f3229000700 1 -- 172.21.15.1:0/2511975779 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f32040a0710 msgr2=0x7f32040a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:36.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.488+0000 7f3229000700 1 --2- 172.21.15.1:0/2511975779 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f32040a0710 0x7f32040a2bd0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f3224198370 tx=0x7f320c007320 comp rx=0 tx=0).stop 2024-02-02T15:45:36.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.488+0000 7f3229000700 1 -- 172.21.15.1:0/2511975779 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f322410b090 msgr2=0x7f322419b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:36.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.488+0000 7f3229000700 1 --2- 172.21.15.1:0/2511975779 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f322410b090 0x7f322419b700 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f321c00e940 tx=0x7f321c00ec50 comp rx=0 tx=0).stop 2024-02-02T15:45:36.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.488+0000 7f3229000700 1 -- 172.21.15.1:0/2511975779 shutdown_connections 2024-02-02T15:45:36.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.488+0000 7f3229000700 1 --2- 172.21.15.1:0/2511975779 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f32040a0710 0x7f32040a2bd0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:36.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.488+0000 7f3229000700 1 --2- 172.21.15.1:0/2511975779 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f322410b090 0x7f322419b700 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:36.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.488+0000 7f3229000700 1 --2- 172.21.15.1:0/2511975779 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3224101c70 0x7f3224196d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:36.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.488+0000 7f3229000700 1 --2- 172.21.15.1:0/2511975779 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3224102590 0x7f32241972a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:36.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.488+0000 7f3229000700 1 -- 172.21.15.1:0/2511975779 >> 172.21.15.1:0/2511975779 conn(0x7f32240fb490 msgr2=0x7f322410d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:45:36.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.489+0000 7f3229000700 1 -- 172.21.15.1:0/2511975779 shutdown_connections 2024-02-02T15:45:36.491 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.489+0000 7f3229000700 1 -- 172.21.15.1:0/2511975779 wait complete. 2024-02-02T15:45:36.515 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-02T15:45:36.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.812+0000 7fed0b000700 1 -- 172.21.15.1:0/820184984 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fed04106120 msgr2=0x7fed0410dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:36.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.812+0000 7fed0b000700 1 --2- 172.21.15.1:0/820184984 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fed04106120 0x7fed0410dca0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fecfc00b0a0 tx=0x7fecfc00b3b0 comp rx=0 tx=0).stop 2024-02-02T15:45:36.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.812+0000 7fed0b000700 1 -- 172.21.15.1:0/820184984 shutdown_connections 2024-02-02T15:45:36.812 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.812+0000 7fed0b000700 1 --2- 172.21.15.1:0/820184984 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fed04106120 0x7fed0410dca0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:36.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.812+0000 7fed0b000700 1 --2- 172.21.15.1:0/820184984 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed04101000 0x7fed041013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:36.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.812+0000 7fed0b000700 1 --2- 172.21.15.1:0/820184984 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed041019b0 0x7fed041059f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:36.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.812+0000 7fed0b000700 1 -- 172.21.15.1:0/820184984 >> 172.21.15.1:0/820184984 conn(0x7fed040fc840 msgr2=0x7fed040fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:45:36.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.812+0000 7fed0b000700 1 -- 172.21.15.1:0/820184984 shutdown_connections 2024-02-02T15:45:36.813 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.813+0000 7fed0b000700 1 -- 172.21.15.1:0/820184984 wait complete. 2024-02-02T15:45:36.814 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.814+0000 7fed0b000700 1 Processor -- start 2024-02-02T15:45:36.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.815+0000 7fed0b000700 1 -- start start 2024-02-02T15:45:36.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.815+0000 7fed0b000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fed04101000 0x7fed041a6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:36.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.815+0000 7fed0b000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed04106120 0x7fed041a6980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:36.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.815+0000 7fed0b000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed041aa570 0x7fed041aaa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:36.815 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.815+0000 7fed0b000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed041aaf60 con 0x7fed041aa570 2024-02-02T15:45:36.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.815+0000 7fed0b000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed041ab0d0 con 0x7fed04101000 2024-02-02T15:45:36.816 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.815+0000 7fed0b000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed041ab290 con 0x7fed04106120 2024-02-02T15:45:36.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.816+0000 7fed03400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed04106120 0x7fed041a6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:36.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.816+0000 7fed03400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed04106120 0x7fed041a6980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:37118/0 (socket says 172.21.15.1:37118) 2024-02-02T15:45:36.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.816+0000 7fed03400700 1 -- 172.21.15.1:0/1642721568 learned_addr learned my addr 172.21.15.1:0/1642721568 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:45:36.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.816+0000 7fed09000700 1 --2- 172.21.15.1:0/1642721568 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed041aa570 0x7fed041aaa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:36.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.817+0000 7fed03400700 1 -- 172.21.15.1:0/1642721568 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fed04101000 msgr2=0x7fed041a6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:36.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.817+0000 7fed03400700 1 --2- 172.21.15.1:0/1642721568 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fed04101000 0x7fed041a6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:36.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.817+0000 7fed03400700 1 -- 172.21.15.1:0/1642721568 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed041aa570 msgr2=0x7fed041aaa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:36.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.817+0000 7fed03400700 1 --2- 172.21.15.1:0/1642721568 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed041aa570 0x7fed041aaa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:36.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.817+0000 7fed03400700 1 -- 172.21.15.1:0/1642721568 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecfc009d00 con 0x7fed04106120 2024-02-02T15:45:36.818 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.817+0000 7fed03400700 1 --2- 172.21.15.1:0/1642721568 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed04106120 0x7fed041a6980 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fecec00e970 tx=0x7fecec00ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:45:36.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.819+0000 7fed00c00700 1 -- 172.21.15.1:0/1642721568 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecec009790 con 0x7fed04106120 2024-02-02T15:45:36.819 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.819+0000 7fed0b000700 1 -- 172.21.15.1:0/1642721568 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed041b2820 con 0x7fed04106120 2024-02-02T15:45:36.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.819+0000 7fed0b000700 1 -- 172.21.15.1:0/1642721568 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed041b2d80 con 0x7fed04106120 2024-02-02T15:45:36.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.819+0000 7fed00c00700 1 -- 172.21.15.1:0/1642721568 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fecec004cf0 con 0x7fed04106120 2024-02-02T15:45:36.820 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.819+0000 7fed00c00700 1 -- 172.21.15.1:0/1642721568 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecec005680 con 0x7fed04106120 2024-02-02T15:45:36.824 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.824+0000 7fed00c00700 1 -- 172.21.15.1:0/1642721568 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7fecec004e60 con 0x7fed04106120 2024-02-02T15:45:36.825 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.825+0000 7fed1eb19b80 1 Processor -- start 2024-02-02T15:45:36.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.826+0000 7fed1eb19b80 1 -- start start 2024-02-02T15:45:36.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.826+0000 7fed00c00700 1 --2- 172.21.15.1:0/1642721568 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fece40a0790 0x7fece40a2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:36.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.826+0000 7fed1eb19b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ea9b089010 0x55ea9b0893f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:36.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.826+0000 7fed1eb19b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ea9b089930 0x55ea9b0942a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:36.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.826+0000 7fed1eb19b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ea9b0947e0 0x55ea9b096bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:36.826 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.826+0000 7fed1eb19b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea9affd4c0 con 0x55ea9b0947e0 2024-02-02T15:45:36.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.826+0000 7fed1eb19b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea9afe1aa0 con 0x55ea9b089010 2024-02-02T15:45:36.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.826+0000 7fed1eb19b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea9afdf460 con 0x55ea9b089930 2024-02-02T15:45:36.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.826+0000 7fed00c00700 1 -- 172.21.15.1:0/1642721568 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fecec014070 con 0x7fed04106120 2024-02-02T15:45:36.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.827+0000 7fed03e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ea9b089010 0x55ea9b0893f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:36.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.827+0000 7fed03e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ea9b089010 0x55ea9b0893f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:49444/0 (socket says 172.21.15.1:49444) 2024-02-02T15:45:36.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.827+0000 7fed03e00700 1 -- 172.21.15.1:0/2454331449 learned_addr learned my addr 172.21.15.1:0/2454331449 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:45:36.827 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.827+0000 7fed03400700 1 --2- 172.21.15.1:0/2454331449 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ea9b089930 0x55ea9b0942a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:36.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.828+0000 7fed09000700 1 --2- 172.21.15.1:0/1642721568 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fece40a0790 0x7fece40a2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:36.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.828+0000 7fed03e00700 1 -- 172.21.15.1:0/2454331449 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ea9b089930 msgr2=0x55ea9b0942a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:45:36.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.828+0000 7fed03e00700 1 --2- 172.21.15.1:0/2454331449 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ea9b089930 0x55ea9b0942a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:36.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.828+0000 7fed03e00700 1 -- 172.21.15.1:0/2454331449 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ea9b0947e0 msgr2=0x55ea9b096bd0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-02T15:45:36.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.828+0000 7fed03e00700 1 --2- 172.21.15.1:0/2454331449 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ea9b0947e0 0x55ea9b096bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:36.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.828+0000 7fed03e00700 1 -- 172.21.15.1:0/2454331449 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea9ae5c3b0 con 0x55ea9b089010 2024-02-02T15:45:36.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.828+0000 7fed03e00700 1 --2- 172.21.15.1:0/2454331449 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ea9b089010 0x55ea9b0893f0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fecf4007b80 tx=0x7fecf4007e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd8c833bd1c906ad server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:45:36.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.828+0000 7fed09000700 1 --2- 172.21.15.1:0/1642721568 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fece40a0790 0x7fece40a2c50 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fecfc0060b0 tx=0x7fecfc006020 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:45:36.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.829+0000 7fecf9600700 1 -- 172.21.15.1:0/2454331449 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf4004230 con 0x55ea9b089010 2024-02-02T15:45:36.829 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.829+0000 7fecf9600700 1 -- 172.21.15.1:0/2454331449 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fecf400dd50 con 0x55ea9b089010 2024-02-02T15:45:36.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.829+0000 7fecf9600700 1 -- 172.21.15.1:0/2454331449 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf4004e60 con 0x55ea9b089010 2024-02-02T15:45:36.830 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.830+0000 7fed1eb19b80 1 -- 172.21.15.1:0/2454331449 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ea9b089010 msgr2=0x55ea9b0893f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:36.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.830+0000 7fed1eb19b80 1 --2- 172.21.15.1:0/2454331449 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ea9b089010 0x55ea9b0893f0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fecf4007b80 tx=0x7fecf4007e90 comp rx=0 tx=0).stop 2024-02-02T15:45:36.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.830+0000 7fed1eb19b80 1 -- 172.21.15.1:0/2454331449 shutdown_connections 2024-02-02T15:45:36.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.830+0000 7fed1eb19b80 1 --2- 172.21.15.1:0/2454331449 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ea9b0947e0 0x55ea9b096bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:36.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.830+0000 7fed1eb19b80 1 --2- 172.21.15.1:0/2454331449 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ea9b089010 0x55ea9b0893f0 secure :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fecf4007b80 tx=0x7fecf4007e90 comp rx=0 tx=0).stop 2024-02-02T15:45:36.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.830+0000 7fed1eb19b80 1 --2- 172.21.15.1:0/2454331449 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ea9b089930 0x55ea9b0942a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:36.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.830+0000 7fed1eb19b80 1 -- 172.21.15.1:0/2454331449 >> 172.21.15.1:0/2454331449 conn(0x55ea9afe8db0 msgr2=0x55ea9b0853b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:45:36.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.830+0000 7fed1eb19b80 1 -- 172.21.15.1:0/2454331449 shutdown_connections 2024-02-02T15:45:36.831 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.831+0000 7fed1eb19b80 1 -- 172.21.15.1:0/2454331449 wait complete. 2024-02-02T15:45:36.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.831+0000 7fed1eb19b80 1 -- start start 2024-02-02T15:45:36.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.832+0000 7fed1eb19b80 1 Processor -- start 2024-02-02T15:45:36.832 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.832+0000 7fed1eb19b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ea9b089930 0x55ea9b0915c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:36.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.833+0000 7fed1eb19b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ea9b0947e0 0x55ea9b091b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:36.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.833+0000 7fed1eb19b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ea9b092040 0x55ea9b0a0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:36.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.833+0000 7fed1eb19b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea9afdad50 con 0x55ea9b092040 2024-02-02T15:45:36.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.833+0000 7fed1eb19b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea9af76d80 con 0x55ea9b0947e0 2024-02-02T15:45:36.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.833+0000 7fed1eb19b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea9af76a20 con 0x55ea9b089930 2024-02-02T15:45:36.833 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.833+0000 7fed1eb19b80 10 client.0 _ll_register_callbacks cb 0x55ea9af78290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:45:36.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.833+0000 7fed03e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ea9b089930 0x55ea9b0915c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:36.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.833+0000 7fed03400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ea9b0947e0 0x55ea9b091b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:36.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.833+0000 7fed03400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ea9b0947e0 0x55ea9b091b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:49446/0 (socket says 172.21.15.1:49446) 2024-02-02T15:45:36.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.833+0000 7fed03400700 1 -- 172.21.15.1:0/4259295916 learned_addr learned my addr 172.21.15.1:0/4259295916 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:45:36.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.833+0000 7fed09000700 1 --2- 172.21.15.1:0/4259295916 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ea9b092040 0x55ea9b0a0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:36.836 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.834+0000 7fed03e00700 1 -- 172.21.15.1:0/4259295916 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ea9b0947e0 msgr2=0x55ea9b091b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:45:36.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.834+0000 7fed03e00700 1 --2- 172.21.15.1:0/4259295916 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ea9b0947e0 0x55ea9b091b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:36.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.834+0000 7fed03e00700 1 -- 172.21.15.1:0/4259295916 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ea9b092040 msgr2=0x55ea9b0a0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:45:36.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.834+0000 7fed03e00700 1 --2- 172.21.15.1:0/4259295916 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ea9b092040 0x55ea9b0a0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:36.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.834+0000 7fed03e00700 1 -- 172.21.15.1:0/4259295916 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecf4007780 con 0x55ea9b089930 2024-02-02T15:45:36.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.834+0000 7fed03e00700 1 --2- 172.21.15.1:0/4259295916 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ea9b089930 0x55ea9b0915c0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fecf4011000 tx=0x7fecf400dac0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52148382393b2604 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:45:36.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.834+0000 7fecf2000700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:45:36.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.834+0000 7fecf2000700 1 -- 172.21.15.1:0/4259295916 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf4010660 con 0x55ea9b089930 2024-02-02T15:45:36.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.834+0000 7fed1eb19b80 1 -- 172.21.15.1:0/4259295916 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ea9b07d0f0 con 0x55ea9b089930 2024-02-02T15:45:36.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.835+0000 7fecf2000700 1 -- 172.21.15.1:0/4259295916 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fecf40107c0 con 0x55ea9b089930 2024-02-02T15:45:36.837 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.835+0000 7fecf2000700 1 -- 172.21.15.1:0/4259295916 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7fecf400dec0 con 0x55ea9b089930 2024-02-02T15:45:36.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.835+0000 7fed1eb19b80 10 client.30271 fetch_fsmap learned FSMap version 78 2024-02-02T15:45:36.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.835+0000 7fed1eb19b80 1 -- 172.21.15.1:0/4259295916 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) v3 -- 0x55ea9ae08b60 con 0x55ea9b089930 2024-02-02T15:45:36.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.835+0000 7fecf2000700 1 -- 172.21.15.1:0/4259295916 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ea9b07d0f0 con 0x55ea9b089930 2024-02-02T15:45:36.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.835+0000 7fecf2000700 1 -- 172.21.15.1:0/4259295916 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 78) v1 ==== 7506+0+0 (secure 0 0 0) 0x7fecf4018960 con 0x55ea9b089930 2024-02-02T15:45:36.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.835+0000 7fed1eb19b80 10 client.30271 fetch_fsmap finished waiting for FSMap version 78 2024-02-02T15:45:36.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.835+0000 7fed1eb19b80 10 client.30271 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:45:36.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.835+0000 7fed1eb19b80 20 client.30271 populate_metadata read hostname 'smithi001' 2024-02-02T15:45:36.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.835+0000 7fed1eb19b80 1 --2- 172.21.15.1:0/4259295916 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55ea9b0a46b0 0x55ea9b0a6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:36.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.836+0000 7fed1eb19b80 4 client.30271 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:45:36.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.836+0000 7fed1eb19b80 1 -- 172.21.15.1:0/4259295916 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ea9afd3dd0 con 0x55ea9b0a46b0 2024-02-02T15:45:36.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.836+0000 7fed03400700 1 --2- 172.21.15.1:0/4259295916 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55ea9b0a46b0 0x55ea9b0a6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:36.838 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.836+0000 7fed03400700 1 --2- 172.21.15.1:0/4259295916 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55ea9b0a46b0 0x55ea9b0a6ad0 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42fd04625662b5a3 server_cookie=223bc7b97c7c4e89 in_seq=0 out_seq=0 2024-02-02T15:45:36.839 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.836+0000 7fecf2000700 10 client.30271 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:45:36.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.837+0000 7fecf2000700 1 -- 172.21.15.1:0/4259295916 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55ea9afd3dd0 con 0x55ea9b0a46b0 2024-02-02T15:45:36.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.837+0000 7fecf2000700 10 client.30271 handle_command_reply: tid=0 2024-02-02T15:45:36.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.837+0000 7fed1eb19b80 1 client.30271 shutdown 2024-02-02T15:45:36.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.837+0000 7fed1eb19b80 1 -- 172.21.15.1:0/4259295916 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ea9b089930 msgr2=0x55ea9b0915c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:36.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.837+0000 7fed1eb19b80 1 --2- 172.21.15.1:0/4259295916 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ea9b089930 0x55ea9b0915c0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fecf4011000 tx=0x7fecf400dac0 comp rx=0 tx=0).stop 2024-02-02T15:45:36.840 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.837+0000 7fed1eb19b80 1 -- 172.21.15.1:0/4259295916 shutdown_connections 2024-02-02T15:45:36.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.837+0000 7fed1eb19b80 1 --2- 172.21.15.1:0/4259295916 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55ea9b0a46b0 0x55ea9b0a6ad0 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:36.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.837+0000 7fed1eb19b80 1 --2- 172.21.15.1:0/4259295916 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ea9b092040 0x55ea9b0a0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:36.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.837+0000 7fed1eb19b80 1 --2- 172.21.15.1:0/4259295916 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ea9b089930 0x55ea9b0915c0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:36.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.837+0000 7fecf2000700 0 client.30271 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:45:36.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.837+0000 7fed1eb19b80 1 --2- 172.21.15.1:0/4259295916 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ea9b0947e0 0x55ea9b091b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:36.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.837+0000 7fed1eb19b80 1 -- 172.21.15.1:0/4259295916 >> 172.21.15.1:0/4259295916 conn(0x55ea9afe8db0 msgr2=0x55ea9b084f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:45:36.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.838+0000 7fed1eb19b80 1 -- 172.21.15.1:0/4259295916 shutdown_connections 2024-02-02T15:45:36.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.838+0000 7fed1eb19b80 1 -- 172.21.15.1:0/4259295916 wait complete. 2024-02-02T15:45:36.841 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.838+0000 7fed1eb19b80 20 client.30271 trim_cache size 0 max 16384 2024-02-02T15:45:36.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.861+0000 7fed1eb19b80 1 Processor -- start 2024-02-02T15:45:36.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.861+0000 7fed1eb19b80 1 -- start start 2024-02-02T15:45:36.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.861+0000 7fed1eb19b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ea9b089930 0x55ea9b09b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:36.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.861+0000 7fed1eb19b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ea9b092040 0x55ea9b08b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:36.861 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.861+0000 7fed1eb19b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ea9b0947e0 0x55ea9b0a0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:36.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.861+0000 7fed1eb19b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea9ae5e6a0 con 0x55ea9b092040 2024-02-02T15:45:36.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.861+0000 7fed1eb19b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea9b08bd80 con 0x55ea9b089930 2024-02-02T15:45:36.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.861+0000 7fed1eb19b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea9b080e00 con 0x55ea9b0947e0 2024-02-02T15:45:36.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.861+0000 7fed09000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ea9b0947e0 0x55ea9b0a0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:36.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.862+0000 7fed03e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ea9b089930 0x55ea9b09b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:36.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.862+0000 7fed09000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ea9b0947e0 0x55ea9b0a0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:37152/0 (socket says 172.21.15.1:37152) 2024-02-02T15:45:36.862 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.862+0000 7fed09000700 1 -- 172.21.15.1:0/3530316940 learned_addr learned my addr 172.21.15.1:0/3530316940 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:45:36.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.862+0000 7fed03400700 1 --2- 172.21.15.1:0/3530316940 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ea9b092040 0x55ea9b08b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:36.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.862+0000 7fed03e00700 1 -- 172.21.15.1:0/3530316940 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ea9b0947e0 msgr2=0x55ea9b0a0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:45:36.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.862+0000 7fed03e00700 1 --2- 172.21.15.1:0/3530316940 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ea9b0947e0 0x55ea9b0a0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:36.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.862+0000 7fed03e00700 1 -- 172.21.15.1:0/3530316940 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ea9b092040 msgr2=0x55ea9b08b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:45:36.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.862+0000 7fed03e00700 1 --2- 172.21.15.1:0/3530316940 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ea9b092040 0x55ea9b08b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:36.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.862+0000 7fed03e00700 1 -- 172.21.15.1:0/3530316940 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea9afa0ea0 con 0x55ea9b089930 2024-02-02T15:45:36.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.862+0000 7fed03400700 1 --2- 172.21.15.1:0/3530316940 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ea9b092040 0x55ea9b08b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:45:36.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.863+0000 7fed03e00700 1 --2- 172.21.15.1:0/3530316940 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ea9b089930 0x55ea9b09b3c0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fecf4018d20 tx=0x7fecf4005f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd197684b105209b server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:45:36.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.863+0000 7fecfa000700 1 -- 172.21.15.1:0/3530316940 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf4037070 con 0x55ea9b089930 2024-02-02T15:45:36.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.863+0000 7fecfa000700 1 -- 172.21.15.1:0/3530316940 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fecf400dbe0 con 0x55ea9b089930 2024-02-02T15:45:36.864 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.863+0000 7fecfa000700 1 -- 172.21.15.1:0/3530316940 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf40187e0 con 0x55ea9b089930 2024-02-02T15:45:36.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.864+0000 7fed1eb19b80 1 -- 172.21.15.1:0/3530316940 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ea9b089930 msgr2=0x55ea9b09b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:36.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.864+0000 7fed1eb19b80 1 --2- 172.21.15.1:0/3530316940 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ea9b089930 0x55ea9b09b3c0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fecf4018d20 tx=0x7fecf4005f60 comp rx=0 tx=0).stop 2024-02-02T15:45:36.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.864+0000 7fed1eb19b80 1 -- 172.21.15.1:0/3530316940 shutdown_connections 2024-02-02T15:45:36.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.864+0000 7fed1eb19b80 1 --2- 172.21.15.1:0/3530316940 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ea9b0947e0 0x55ea9b0a0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:36.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.864+0000 7fed1eb19b80 1 --2- 172.21.15.1:0/3530316940 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ea9b089930 0x55ea9b09b3c0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:36.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.864+0000 7fed1eb19b80 1 --2- 172.21.15.1:0/3530316940 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ea9b092040 0x55ea9b08b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:36.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.864+0000 7fed1eb19b80 1 -- 172.21.15.1:0/3530316940 >> 172.21.15.1:0/3530316940 conn(0x55ea9afe8db0 msgr2=0x55ea9b083b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:45:36.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.865+0000 7fed1eb19b80 1 -- 172.21.15.1:0/3530316940 shutdown_connections 2024-02-02T15:45:36.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.865+0000 7fed1eb19b80 1 -- 172.21.15.1:0/3530316940 wait complete. 2024-02-02T15:45:36.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.865+0000 7fed1eb19b80 1 -- start start 2024-02-02T15:45:36.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.865+0000 7fed1eb19b80 1 Processor -- start 2024-02-02T15:45:36.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.866+0000 7fed1eb19b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ea9b089930 0x55ea9b0995a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:36.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.866+0000 7fed1eb19b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ea9b092040 0x55ea9b099ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:36.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.866+0000 7fed1eb19b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ea9b0947e0 0x55ea9b0ef1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:36.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.866+0000 7fed1eb19b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea9b0ef6e0 con 0x55ea9b092040 2024-02-02T15:45:36.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.866+0000 7fed1eb19b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea9b0ef850 con 0x55ea9b0947e0 2024-02-02T15:45:36.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.866+0000 7fed1eb19b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea9b0efa10 con 0x55ea9b089930 2024-02-02T15:45:36.867 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.866+0000 7fed1eb19b80 10 client.0 _ll_register_callbacks cb 0x55ea9af78290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:45:36.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.867+0000 7fed03e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ea9b089930 0x55ea9b0995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:36.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.867+0000 7fed09000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ea9b0947e0 0x55ea9b0ef1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:36.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.867+0000 7fed03e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ea9b089930 0x55ea9b0995a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:37154/0 (socket says 172.21.15.1:37154) 2024-02-02T15:45:36.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.867+0000 7fed03e00700 1 -- 172.21.15.1:0/3361249897 learned_addr learned my addr 172.21.15.1:0/3361249897 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:45:36.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.867+0000 7fed03400700 1 --2- 172.21.15.1:0/3361249897 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ea9b092040 0x55ea9b099ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:36.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.868+0000 7fed09000700 1 -- 172.21.15.1:0/3361249897 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ea9b089930 msgr2=0x55ea9b0995a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:45:36.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.868+0000 7fed09000700 1 --2- 172.21.15.1:0/3361249897 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ea9b089930 0x55ea9b0995a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:36.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.868+0000 7fed09000700 1 -- 172.21.15.1:0/3361249897 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ea9b092040 msgr2=0x55ea9b099ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:45:36.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.868+0000 7fed09000700 1 --2- 172.21.15.1:0/3361249897 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ea9b092040 0x55ea9b099ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:36.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.868+0000 7fed09000700 1 -- 172.21.15.1:0/3361249897 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ea9ae08b60 con 0x55ea9b0947e0 2024-02-02T15:45:36.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.868+0000 7fed03e00700 1 --2- 172.21.15.1:0/3361249897 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ea9b089930 0x55ea9b0995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:45:36.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.868+0000 7fed03400700 1 --2- 172.21.15.1:0/3361249897 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ea9b092040 0x55ea9b099ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:45:36.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.869+0000 7fed09000700 1 --2- 172.21.15.1:0/3361249897 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ea9b0947e0 0x55ea9b0ef1a0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x55ea9b09b950 tx=0x7fecfc00b3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef8046095eeee30d server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:45:36.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.869+0000 7fecf2000700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:45:36.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.869+0000 7fecf2000700 1 -- 172.21.15.1:0/3361249897 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecfc004c50 con 0x55ea9b0947e0 2024-02-02T15:45:36.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.869+0000 7fecf2000700 1 -- 172.21.15.1:0/3361249897 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fecfc023040 con 0x55ea9b0947e0 2024-02-02T15:45:36.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.869+0000 7fed1eb19b80 1 -- 172.21.15.1:0/3361249897 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fecf400dd50 con 0x55ea9b0947e0 2024-02-02T15:45:36.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.869+0000 7fecf2000700 1 -- 172.21.15.1:0/3361249897 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7fecfc004db0 con 0x55ea9b0947e0 2024-02-02T15:45:36.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.870+0000 7fed1eb19b80 10 client.30419 fetch_fsmap learned FSMap version 78 2024-02-02T15:45:36.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.870+0000 7fed1eb19b80 1 -- 172.21.15.1:0/3361249897 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) v3 -- 0x55ea9ae60cd0 con 0x55ea9b0947e0 2024-02-02T15:45:36.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.870+0000 7fecf2000700 1 -- 172.21.15.1:0/3361249897 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fecf400dd50 con 0x55ea9b0947e0 2024-02-02T15:45:36.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.871+0000 7fecf2000700 1 -- 172.21.15.1:0/3361249897 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 78) v1 ==== 7506+0+0 (secure 0 0 0) 0x7fecfc0054f0 con 0x55ea9b0947e0 2024-02-02T15:45:36.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.871+0000 7fed1eb19b80 10 client.30419 fetch_fsmap finished waiting for FSMap version 78 2024-02-02T15:45:36.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.871+0000 7fed1eb19b80 10 client.30419 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:45:36.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.871+0000 7fed1eb19b80 20 client.30419 populate_metadata read hostname 'smithi001' 2024-02-02T15:45:36.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.871+0000 7fed1eb19b80 1 --2- 172.21.15.1:0/3361249897 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55ea9b0f70d0 0x55ea9b0f94f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:45:36.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.871+0000 7fed1eb19b80 4 client.30419 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2024-02-02T15:45:36.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.871+0000 7fed1eb19b80 1 -- 172.21.15.1:0/3361249897 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ea9af99f50 con 0x55ea9b0f70d0 2024-02-02T15:45:36.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.871+0000 7fed03e00700 1 --2- 172.21.15.1:0/3361249897 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55ea9b0f70d0 0x55ea9b0f94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:45:36.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.872+0000 7fed03e00700 1 --2- 172.21.15.1:0/3361249897 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55ea9b0f70d0 0x55ea9b0f94f0 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fd12265b70f8c5e server_cookie=ff0d108ce942c783 in_seq=0 out_seq=0 2024-02-02T15:45:36.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.872+0000 7fecf2000700 10 client.30419 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:45:36.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.872+0000 7fecf2000700 1 -- 172.21.15.1:0/3361249897 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55ea9af99f50 con 0x55ea9b0f70d0 2024-02-02T15:45:36.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.872+0000 7fecf2000700 10 client.30419 handle_command_reply: tid=0 2024-02-02T15:45:36.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.872+0000 7fed1eb19b80 1 client.30419 shutdown 2024-02-02T15:45:36.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.873+0000 7fed1eb19b80 1 -- 172.21.15.1:0/3361249897 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ea9b0947e0 msgr2=0x55ea9b0ef1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:36.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.873+0000 7fed1eb19b80 1 --2- 172.21.15.1:0/3361249897 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ea9b0947e0 0x55ea9b0ef1a0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x55ea9b09b950 tx=0x7fecfc00b3b0 comp rx=0 tx=0).stop 2024-02-02T15:45:36.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.873+0000 7fed1eb19b80 1 -- 172.21.15.1:0/3361249897 shutdown_connections 2024-02-02T15:45:36.873 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.873+0000 7fed1eb19b80 1 --2- 172.21.15.1:0/3361249897 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55ea9b0f70d0 0x55ea9b0f94f0 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:36.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.873+0000 7fed1eb19b80 1 --2- 172.21.15.1:0/3361249897 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ea9b0947e0 0x55ea9b0ef1a0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:36.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.873+0000 7fed1eb19b80 1 --2- 172.21.15.1:0/3361249897 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55ea9b089930 0x55ea9b0995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:36.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.873+0000 7fed1eb19b80 1 --2- 172.21.15.1:0/3361249897 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55ea9b092040 0x55ea9b099ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:36.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.873+0000 7fed1eb19b80 1 -- 172.21.15.1:0/3361249897 >> 172.21.15.1:0/3361249897 conn(0x55ea9afe8db0 msgr2=0x55ea9b0a0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:45:36.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.873+0000 7fecf2000700 0 client.30419 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:45:36.874 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.874+0000 7fed1eb19b80 1 -- 172.21.15.1:0/3361249897 shutdown_connections 2024-02-02T15:45:36.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.874+0000 7fed1eb19b80 1 -- 172.21.15.1:0/3361249897 wait complete. 2024-02-02T15:45:36.875 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.874+0000 7fed1eb19b80 20 client.30419 trim_cache size 0 max 16384 2024-02-02T15:45:36.875 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:45:36.876 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-02T15:45:36.876 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2024-02-02T15:45:36.876 INFO:teuthology.orchestra.run.smithi001.stdout: "ea43b95a-27e9-4be7-a487-e68daa4971bc": { 2024-02-02T15:45:36.876 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2024-02-02T15:45:36.876 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "ea43b95a-27e9-4be7-a487-e68daa4971bc", 2024-02-02T15:45:36.876 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,force" 2024-02-02T15:45:36.876 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:45:36.876 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:45:36.876 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:45:36.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.876+0000 7fed0b000700 1 -- 172.21.15.1:0/1642721568 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fece40a0790 msgr2=0x7fece40a2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:36.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.876+0000 7fed0b000700 1 --2- 172.21.15.1:0/1642721568 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fece40a0790 0x7fece40a2c50 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fecfc0060b0 tx=0x7fecfc006020 comp rx=0 tx=0).stop 2024-02-02T15:45:36.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.876+0000 7fed0b000700 1 -- 172.21.15.1:0/1642721568 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed04106120 msgr2=0x7fed041a6980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:45:36.877 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.876+0000 7fed0b000700 1 --2- 172.21.15.1:0/1642721568 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed04106120 0x7fed041a6980 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fecec00e970 tx=0x7fecec00ed30 comp rx=0 tx=0).stop 2024-02-02T15:45:36.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.877+0000 7fed0b000700 1 -- 172.21.15.1:0/1642721568 shutdown_connections 2024-02-02T15:45:36.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.877+0000 7fed0b000700 1 --2- 172.21.15.1:0/1642721568 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fece40a0790 0x7fece40a2c50 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:36.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.877+0000 7fed0b000700 1 --2- 172.21.15.1:0/1642721568 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fed041aa570 0x7fed041aaa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:36.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.877+0000 7fed0b000700 1 --2- 172.21.15.1:0/1642721568 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fed04101000 0x7fed041a6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:36.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.877+0000 7fed0b000700 1 --2- 172.21.15.1:0/1642721568 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fed04106120 0x7fed041a6980 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:45:36.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.877+0000 7fed0b000700 1 -- 172.21.15.1:0/1642721568 >> 172.21.15.1:0/1642721568 conn(0x7fed040fc840 msgr2=0x7fed040fd260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:45:36.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.877+0000 7fed0b000700 1 -- 172.21.15.1:0/1642721568 shutdown_connections 2024-02-02T15:45:36.878 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:45:36.878+0000 7fed0b000700 1 -- 172.21.15.1:0/1642721568 wait complete. 2024-02-02T15:45:36.902 INFO:tasks.cephfs.filesystem:scrub status for tag:ea43b95a-27e9-4be7-a487-e68daa4971bc - {'path': '/', 'tag': 'ea43b95a-27e9-4be7-a487-e68daa4971bc', 'options': 'recursive,force'} 2024-02-02T15:45:37.006 INFO:tasks.workunit.client.0.smithi001.stdout: 10 166208 27.49 MB/sec execute 189 sec latency 137.089 ms 2024-02-02T15:45:37.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:37 smithi071 ceph-mon[44707]: pgmap v585: 129 pgs: 129 active+clean; 2.2 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.8 MiB/s wr, 495 op/s 2024-02-02T15:45:37.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:37 smithi071 ceph-mon[44707]: scrub queued for path: / 2024-02-02T15:45:37.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:37 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:45:37.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:37 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:45:37.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:37 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:45:37.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:37 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:45:37.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:37 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:45:37.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:37 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:45:37.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:37 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:45:37.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:37 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:45:37.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:37 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:45:37.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:37 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:45:37.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:37 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:45:37.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:37 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:45:37.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:37 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:45:37.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:37 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:45:37.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:37 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:45:37.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:37 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:45:37.867 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:37 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:45:37.867 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:37 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:45:37.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:37 smithi142 ceph-mon[45728]: pgmap v585: 129 pgs: 129 active+clean; 2.2 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.8 MiB/s wr, 495 op/s 2024-02-02T15:45:37.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:37 smithi142 ceph-mon[45728]: scrub queued for path: / 2024-02-02T15:45:37.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:37 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:45:37.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:37 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:45:37.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:37 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:45:37.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:37 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:45:37.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:37 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:45:37.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:37 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:45:37.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:37 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:45:37.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:37 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:45:37.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:37 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:45:37.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:37 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:45:37.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:37 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:45:37.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:37 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:45:37.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:37 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:45:37.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:37 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:45:37.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:37 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:45:37.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:37 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:45:37.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:37 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:45:37.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:37 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:45:37.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:37 smithi001 ceph-mon[35470]: pgmap v585: 129 pgs: 129 active+clean; 2.2 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.8 MiB/s wr, 495 op/s 2024-02-02T15:45:37.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:37 smithi001 ceph-mon[35470]: scrub queued for path: / 2024-02-02T15:45:37.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:37 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:45:37.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:37 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:45:37.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:37 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:45:37.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:37 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:45:37.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:37 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:45:37.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:37 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:45:37.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:37 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:45:37.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:37 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:45:37.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:37 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:45:37.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:37 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:45:37.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:37 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:45:37.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:37 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:45:37.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:37 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:45:37.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:37 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:45:37.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:37 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:45:37.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:37 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:45:37.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:37 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:45:37.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:37 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:45:38.006 INFO:tasks.workunit.client.0.smithi001.stdout: 10 166867 27.54 MB/sec execute 190 sec latency 82.370 ms 2024-02-02T15:45:38.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:38 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:45:38.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:38 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:45:38.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:38 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:45:39.006 INFO:tasks.workunit.client.0.smithi001.stdout: 10 167550 27.62 MB/sec execute 191 sec latency 62.236 ms 2024-02-02T15:45:39.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:39 smithi071 ceph-mon[44707]: pgmap v586: 129 pgs: 129 active+clean; 2.2 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.8 MiB/s wr, 447 op/s 2024-02-02T15:45:39.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:39 smithi142 ceph-mon[45728]: pgmap v586: 129 pgs: 129 active+clean; 2.2 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.8 MiB/s wr, 447 op/s 2024-02-02T15:45:39.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:39 smithi001 ceph-mon[35470]: pgmap v586: 129 pgs: 129 active+clean; 2.2 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.8 MiB/s wr, 447 op/s 2024-02-02T15:45:40.006 INFO:tasks.workunit.client.0.smithi001.stdout: 10 168089 27.67 MB/sec execute 192 sec latency 116.104 ms 2024-02-02T15:45:41.007 INFO:tasks.workunit.client.0.smithi001.stdout: 10 168448 27.61 MB/sec execute 193 sec latency 65.228 ms 2024-02-02T15:45:41.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:41 smithi071 ceph-mon[44707]: pgmap v587: 129 pgs: 129 active+clean; 2.2 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 6.4 MiB/s wr, 362 op/s 2024-02-02T15:45:41.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:41 smithi142 ceph-mon[45728]: pgmap v587: 129 pgs: 129 active+clean; 2.2 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 6.4 MiB/s wr, 362 op/s 2024-02-02T15:45:41.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:41 smithi001 ceph-mon[35470]: pgmap v587: 129 pgs: 129 active+clean; 2.2 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 6.4 MiB/s wr, 362 op/s 2024-02-02T15:45:42.006 INFO:tasks.workunit.client.0.smithi001.stdout: 10 168971 27.56 MB/sec execute 194 sec latency 78.429 ms 2024-02-02T15:45:42.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:42 smithi001 ceph-mon[35470]: pgmap v588: 129 pgs: 129 active+clean; 2.2 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.1 MiB/s wr, 517 op/s 2024-02-02T15:45:43.007 INFO:tasks.workunit.client.0.smithi001.stdout: 10 169546 27.58 MB/sec execute 195 sec latency 79.102 ms 2024-02-02T15:45:43.116 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:42 smithi071 ceph-mon[44707]: pgmap v588: 129 pgs: 129 active+clean; 2.2 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.1 MiB/s wr, 517 op/s 2024-02-02T15:45:43.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:42 smithi142 ceph-mon[45728]: pgmap v588: 129 pgs: 129 active+clean; 2.2 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.1 MiB/s wr, 517 op/s 2024-02-02T15:45:44.007 INFO:tasks.workunit.client.0.smithi001.stdout: 10 170212 27.59 MB/sec execute 196 sec latency 92.838 ms 2024-02-02T15:45:45.007 INFO:tasks.workunit.client.0.smithi001.stdout: 10 170969 27.70 MB/sec execute 197 sec latency 78.779 ms 2024-02-02T15:45:45.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:45 smithi071 ceph-mon[44707]: pgmap v589: 129 pgs: 129 active+clean; 2.2 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.3 MiB/s wr, 461 op/s 2024-02-02T15:45:45.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:45 smithi142 ceph-mon[45728]: pgmap v589: 129 pgs: 129 active+clean; 2.2 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.3 MiB/s wr, 461 op/s 2024-02-02T15:45:45.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:45 smithi001 ceph-mon[35470]: pgmap v589: 129 pgs: 129 active+clean; 2.2 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.3 MiB/s wr, 461 op/s 2024-02-02T15:45:46.007 INFO:tasks.workunit.client.0.smithi001.stdout: 10 171019 27.59 MB/sec execute 198 sec latency 944.316 ms 2024-02-02T15:45:47.009 INFO:tasks.workunit.client.0.smithi001.stdout: 10 171256 27.53 MB/sec execute 199 sec latency 829.789 ms 2024-02-02T15:45:47.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:47 smithi071 ceph-mon[44707]: pgmap v590: 129 pgs: 129 active+clean; 2.3 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 511 op/s 2024-02-02T15:45:47.922 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:47 smithi142 ceph-mon[45728]: pgmap v590: 129 pgs: 129 active+clean; 2.3 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 511 op/s 2024-02-02T15:45:47.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:47 smithi001 ceph-mon[35470]: pgmap v590: 129 pgs: 129 active+clean; 2.3 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 511 op/s 2024-02-02T15:45:48.007 INFO:tasks.workunit.client.0.smithi001.stdout: 10 171491 27.46 MB/sec execute 200 sec latency 424.255 ms 2024-02-02T15:45:49.007 INFO:tasks.workunit.client.0.smithi001.stdout: 10 171753 27.43 MB/sec execute 201 sec latency 189.278 ms 2024-02-02T15:45:49.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:49 smithi071 ceph-mon[44707]: pgmap v591: 129 pgs: 129 active+clean; 2.3 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 539 op/s 2024-02-02T15:45:49.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:49 smithi142 ceph-mon[45728]: pgmap v591: 129 pgs: 129 active+clean; 2.3 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 539 op/s 2024-02-02T15:45:49.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:49 smithi001 ceph-mon[35470]: pgmap v591: 129 pgs: 129 active+clean; 2.3 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 539 op/s 2024-02-02T15:45:50.007 INFO:tasks.workunit.client.0.smithi001.stdout: 10 172154 27.35 MB/sec execute 202 sec latency 96.157 ms 2024-02-02T15:45:51.007 INFO:tasks.workunit.client.0.smithi001.stdout: 10 172407 27.26 MB/sec execute 203 sec latency 411.704 ms 2024-02-02T15:45:51.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:51 smithi071 ceph-mon[44707]: pgmap v592: 129 pgs: 129 active+clean; 2.3 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 13 MiB/s wr, 476 op/s 2024-02-02T15:45:51.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:51 smithi142 ceph-mon[45728]: pgmap v592: 129 pgs: 129 active+clean; 2.3 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 13 MiB/s wr, 476 op/s 2024-02-02T15:45:51.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:51 smithi001 ceph-mon[35470]: pgmap v592: 129 pgs: 129 active+clean; 2.3 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 13 MiB/s wr, 476 op/s 2024-02-02T15:45:52.008 INFO:tasks.workunit.client.0.smithi001.stdout: 10 172483 27.16 MB/sec execute 204 sec latency 1326.092 ms 2024-02-02T15:45:52.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:52 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:45:52.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:52 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:45:52.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:52 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:45:52.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:52 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:45:52.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:52 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:45:52.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:52 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:45:52.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:52 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:45:52.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:52 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:45:52.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:52 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:45:52.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:52 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:45:52.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:52 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:45:52.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:52 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:45:52.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:52 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:45:52.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:52 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:45:52.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:52 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:45:53.010 INFO:tasks.workunit.client.0.smithi001.stdout: 10 172689 27.05 MB/sec execute 205 sec latency 1012.475 ms 2024-02-02T15:45:53.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:53 smithi071 ceph-mon[44707]: pgmap v593: 129 pgs: 129 active+clean; 2.3 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 19 MiB/s wr, 716 op/s 2024-02-02T15:45:53.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:53 smithi142 ceph-mon[45728]: pgmap v593: 129 pgs: 129 active+clean; 2.3 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 19 MiB/s wr, 716 op/s 2024-02-02T15:45:53.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:53 smithi001 ceph-mon[35470]: pgmap v593: 129 pgs: 129 active+clean; 2.3 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 19 MiB/s wr, 716 op/s 2024-02-02T15:45:54.009 INFO:tasks.workunit.client.0.smithi001.stdout: 10 172994 27.00 MB/sec execute 206 sec latency 172.386 ms 2024-02-02T15:45:55.008 INFO:tasks.workunit.client.0.smithi001.stdout: 10 173435 26.99 MB/sec execute 207 sec latency 135.228 ms 2024-02-02T15:45:55.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:55 smithi071 ceph-mon[44707]: pgmap v594: 129 pgs: 129 active+clean; 2.3 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 19 MiB/s wr, 651 op/s 2024-02-02T15:45:55.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:55 smithi142 ceph-mon[45728]: pgmap v594: 129 pgs: 129 active+clean; 2.3 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 19 MiB/s wr, 651 op/s 2024-02-02T15:45:55.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:55 smithi001 ceph-mon[35470]: pgmap v594: 129 pgs: 129 active+clean; 2.3 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 19 MiB/s wr, 651 op/s 2024-02-02T15:45:56.009 INFO:tasks.workunit.client.0.smithi001.stdout: 10 174189 27.05 MB/sec execute 208 sec latency 73.404 ms 2024-02-02T15:45:57.009 INFO:tasks.workunit.client.0.smithi001.stdout: 10 174930 27.17 MB/sec execute 209 sec latency 69.781 ms 2024-02-02T15:45:57.866 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:57 smithi071 ceph-mon[44707]: pgmap v595: 129 pgs: 129 active+clean; 2.3 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 20 MiB/s wr, 760 op/s 2024-02-02T15:45:57.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:57 smithi142 ceph-mon[45728]: pgmap v595: 129 pgs: 129 active+clean; 2.3 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 20 MiB/s wr, 760 op/s 2024-02-02T15:45:57.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:57 smithi001 ceph-mon[35470]: pgmap v595: 129 pgs: 129 active+clean; 2.3 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 20 MiB/s wr, 760 op/s 2024-02-02T15:45:58.009 INFO:tasks.workunit.client.0.smithi001.stdout: 10 175478 27.17 MB/sec execute 210 sec latency 91.505 ms 2024-02-02T15:45:59.009 INFO:tasks.workunit.client.0.smithi001.stdout: 10 175921 27.16 MB/sec execute 211 sec latency 67.494 ms 2024-02-02T15:45:59.867 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:45:59 smithi071 ceph-mon[44707]: pgmap v596: 129 pgs: 129 active+clean; 2.3 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 16 MiB/s wr, 731 op/s 2024-02-02T15:45:59.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:45:59 smithi142 ceph-mon[45728]: pgmap v596: 129 pgs: 129 active+clean; 2.3 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 16 MiB/s wr, 731 op/s 2024-02-02T15:45:59.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:45:59 smithi001 ceph-mon[35470]: pgmap v596: 129 pgs: 129 active+clean; 2.3 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 16 MiB/s wr, 731 op/s 2024-02-02T15:46:00.009 INFO:tasks.workunit.client.0.smithi001.stdout: 10 176307 27.11 MB/sec execute 212 sec latency 93.265 ms 2024-02-02T15:46:01.010 INFO:tasks.workunit.client.0.smithi001.stdout: 10 176534 27.02 MB/sec execute 213 sec latency 593.164 ms 2024-02-02T15:46:01.794 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:46:01 smithi142 ceph-mon[45728]: pgmap v597: 129 pgs: 129 active+clean; 2.3 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 12 MiB/s wr, 587 op/s 2024-02-02T15:46:01.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:46:01 smithi071 ceph-mon[44707]: pgmap v597: 129 pgs: 129 active+clean; 2.3 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 12 MiB/s wr, 587 op/s 2024-02-02T15:46:01.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:46:01 smithi001 ceph-mon[35470]: pgmap v597: 129 pgs: 129 active+clean; 2.3 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 12 MiB/s wr, 587 op/s 2024-02-02T15:46:02.009 INFO:tasks.workunit.client.0.smithi001.stdout: 10 176534 26.90 MB/sec execute 214 sec latency 1593.249 ms 2024-02-02T15:46:02.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:46:02 smithi001 ceph-mon[35470]: pgmap v598: 129 pgs: 129 active+clean; 2.3 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 707 op/s 2024-02-02T15:46:03.009 INFO:tasks.workunit.client.0.smithi001.stdout: 10 176677 26.79 MB/sec execute 215 sec latency 2298.660 ms 2024-02-02T15:46:03.117 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:46:02 smithi071 ceph-mon[44707]: pgmap v598: 129 pgs: 129 active+clean; 2.3 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 707 op/s 2024-02-02T15:46:03.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:46:02 smithi142 ceph-mon[45728]: pgmap v598: 129 pgs: 129 active+clean; 2.3 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 14 MiB/s wr, 707 op/s 2024-02-02T15:46:04.010 INFO:tasks.workunit.client.0.smithi001.stdout: 10 177075 26.73 MB/sec execute 216 sec latency 289.187 ms 2024-02-02T15:46:05.010 INFO:tasks.workunit.client.0.smithi001.stdout: 10 177221 26.63 MB/sec execute 217 sec latency 338.028 ms 2024-02-02T15:46:05.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:46:05 smithi071 ceph-mon[44707]: pgmap v599: 129 pgs: 129 active+clean; 2.3 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 8.4 MiB/s wr, 509 op/s 2024-02-02T15:46:05.922 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:46:05 smithi142 ceph-mon[45728]: pgmap v599: 129 pgs: 129 active+clean; 2.3 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 8.4 MiB/s wr, 509 op/s 2024-02-02T15:46:05.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:46:05 smithi001 ceph-mon[35470]: pgmap v599: 129 pgs: 129 active+clean; 2.3 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 8.4 MiB/s wr, 509 op/s 2024-02-02T15:46:06.010 INFO:tasks.workunit.client.0.smithi001.stdout: 10 177242 26.52 MB/sec execute 218 sec latency 353.825 ms 2024-02-02T15:46:06.903 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-02T15:46:07.010 INFO:tasks.workunit.client.0.smithi001.stdout: 10 177246 26.40 MB/sec execute 219 sec latency 458.097 ms 2024-02-02T15:46:07.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.244+0000 7f2f0e600700 1 -- 172.21.15.1:0/2530282704 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2f08108660 msgr2=0x7f2f08113950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:07.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.244+0000 7f2f0e600700 1 --2- 172.21.15.1:0/2530282704 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2f08108660 0x7f2f08113950 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8009990 tx=0x7f2ef8009ca0 comp rx=0 tx=0).stop 2024-02-02T15:46:07.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.246+0000 7f2f0e600700 1 -- 172.21.15.1:0/2530282704 shutdown_connections 2024-02-02T15:46:07.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.246+0000 7f2f0e600700 1 --2- 172.21.15.1:0/2530282704 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2f08113e90 0x7f2f08116280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:07.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.246+0000 7f2f0e600700 1 --2- 172.21.15.1:0/2530282704 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2f08107d40 0x7f2f08108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:07.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.246+0000 7f2f0e600700 1 --2- 172.21.15.1:0/2530282704 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2f08108660 0x7f2f08113950 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:07.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.246+0000 7f2f0e600700 1 -- 172.21.15.1:0/2530282704 >> 172.21.15.1:0/2530282704 conn(0x7f2f0806d120 msgr2=0x7f2f0806d530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:07.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.247+0000 7f2f0e600700 1 -- 172.21.15.1:0/2530282704 shutdown_connections 2024-02-02T15:46:07.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.247+0000 7f2f0e600700 1 -- 172.21.15.1:0/2530282704 wait complete. 2024-02-02T15:46:07.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.249+0000 7f2f0e600700 1 Processor -- start 2024-02-02T15:46:07.249 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.249+0000 7f2f0e600700 1 -- start start 2024-02-02T15:46:07.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.250+0000 7f2f0e600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2f08107d40 0x7f2f081a19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:07.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.250+0000 7f2f0e600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2f08108660 0x7f2f081a1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:07.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.250+0000 7f2f0e600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2f08113e90 0x7f2f081a6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:07.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.250+0000 7f2f0e600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f081a6940 con 0x7f2f08108660 2024-02-02T15:46:07.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.250+0000 7f2f0e600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f081a6ab0 con 0x7f2f08113e90 2024-02-02T15:46:07.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.250+0000 7f2f0e600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f081a6c70 con 0x7f2f08107d40 2024-02-02T15:46:07.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.251+0000 7f2f06a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2f08108660 0x7f2f081a1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:07.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.251+0000 7f2f06a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2f08108660 0x7f2f081a1ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:34446/0 (socket says 172.21.15.1:34446) 2024-02-02T15:46:07.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.251+0000 7f2f06a00700 1 -- 172.21.15.1:0/1049649959 learned_addr learned my addr 172.21.15.1:0/1049649959 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:46:07.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.251+0000 7f2f07e00700 1 --2- 172.21.15.1:0/1049649959 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2f08113e90 0x7f2f081a6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:07.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.251+0000 7f2f06a00700 1 -- 172.21.15.1:0/1049649959 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2f08107d40 msgr2=0x7f2f081a19a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-02T15:46:07.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.251+0000 7f2f06a00700 1 --2- 172.21.15.1:0/1049649959 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2f08107d40 0x7f2f081a19a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:07.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.251+0000 7f2f06a00700 1 -- 172.21.15.1:0/1049649959 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2f08113e90 msgr2=0x7f2f081a6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:07.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.251+0000 7f2f06a00700 1 --2- 172.21.15.1:0/1049649959 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2f08113e90 0x7f2f081a6370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:07.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.251+0000 7f2f06a00700 1 -- 172.21.15.1:0/1049649959 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ef80095f0 con 0x7f2f08108660 2024-02-02T15:46:07.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.252+0000 7f2f06a00700 1 --2- 172.21.15.1:0/1049649959 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2f08108660 0x7f2f081a1ee0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8006010 tx=0x7f2ef800f7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:07.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.252+0000 7f2effe00700 1 -- 172.21.15.1:0/1049649959 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef801d070 con 0x7f2f08108660 2024-02-02T15:46:07.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.252+0000 7f2effe00700 1 -- 172.21.15.1:0/1049649959 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ef800fca0 con 0x7f2f08108660 2024-02-02T15:46:07.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.252+0000 7f2f0e600700 1 -- 172.21.15.1:0/1049649959 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f081a6ef0 con 0x7f2f08108660 2024-02-02T15:46:07.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.252+0000 7f2effe00700 1 -- 172.21.15.1:0/1049649959 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef80176e0 con 0x7f2f08108660 2024-02-02T15:46:07.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.253+0000 7f2f0e600700 1 -- 172.21.15.1:0/1049649959 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f0810dae0 con 0x7f2f08108660 2024-02-02T15:46:07.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.258+0000 7f2effe00700 1 -- 172.21.15.1:0/1049649959 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f2ef80229c0 con 0x7f2f08108660 2024-02-02T15:46:07.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.259+0000 7f2f2207eb80 1 Processor -- start 2024-02-02T15:46:07.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.260+0000 7f2f2207eb80 1 -- start start 2024-02-02T15:46:07.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.261+0000 7f2f2207eb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556953913010 0x5569539133f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:07.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.261+0000 7f2f2207eb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556953913930 0x55695391e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:07.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.261+0000 7f2f2207eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55695391e7e0 0x556953920bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:07.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.261+0000 7f2f2207eb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569538874c0 con 0x556953913010 2024-02-02T15:46:07.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.261+0000 7f2f2207eb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55695386baa0 con 0x556953913930 2024-02-02T15:46:07.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.261+0000 7f2f2207eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556953869460 con 0x55695391e7e0 2024-02-02T15:46:07.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.262+0000 7f2f07e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556953913010 0x5569539133f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:07.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.262+0000 7f2f07e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556953913010 0x5569539133f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:34462/0 (socket says 172.21.15.1:34462) 2024-02-02T15:46:07.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.262+0000 7f2f07e00700 1 -- 172.21.15.1:0/1082037491 learned_addr learned my addr 172.21.15.1:0/1082037491 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:46:07.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.262+0000 7f2effe00700 1 --2- 172.21.15.1:0/1049649959 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f2ee80a0790 0x7f2ee80a2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:07.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.262+0000 7f2f07400700 1 --2- 172.21.15.1:0/1082037491 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556953913930 0x55695391e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:07.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.262+0000 7f2effe00700 1 -- 172.21.15.1:0/1049649959 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f2ef80cd790 con 0x7f2f08108660 2024-02-02T15:46:07.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.263+0000 7f2f07e00700 1 -- 172.21.15.1:0/1082037491 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55695391e7e0 msgr2=0x556953920bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:07.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.263+0000 7f2f07e00700 1 --2- 172.21.15.1:0/1082037491 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55695391e7e0 0x556953920bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:07.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.263+0000 7f2f07e00700 1 -- 172.21.15.1:0/1082037491 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556953913930 msgr2=0x55695391e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:07.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.263+0000 7f2f07e00700 1 --2- 172.21.15.1:0/1082037491 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556953913930 0x55695391e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:07.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.263+0000 7f2f07e00700 1 -- 172.21.15.1:0/1082037491 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5569536e63b0 con 0x556953913010 2024-02-02T15:46:07.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.263+0000 7f2f07e00700 1 --2- 172.21.15.1:0/1049649959 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f2ee80a0790 0x7f2ee80a2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:07.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.264+0000 7f2f07e00700 1 --2- 172.21.15.1:0/1082037491 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556953913010 0x5569539133f0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f2f0000c470 tx=0x7f2f0000c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e2682016ba9c01b server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:07.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.265+0000 7f2f07e00700 1 --2- 172.21.15.1:0/1049649959 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f2ee80a0790 0x7f2ee80a2c50 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f2f00016000 tx=0x7f2f00018000 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:07.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.265+0000 7f2efcc00700 1 -- 172.21.15.1:0/1082037491 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f0000acf0 con 0x556953913010 2024-02-02T15:46:07.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.265+0000 7f2efcc00700 1 -- 172.21.15.1:0/1082037491 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2f0000ae50 con 0x556953913010 2024-02-02T15:46:07.265 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.265+0000 7f2efcc00700 1 -- 172.21.15.1:0/1082037491 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f0000acf0 con 0x556953913010 2024-02-02T15:46:07.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.266+0000 7f2f2207eb80 1 -- 172.21.15.1:0/1082037491 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556953913010 msgr2=0x5569539133f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:07.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.266+0000 7f2f2207eb80 1 --2- 172.21.15.1:0/1082037491 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556953913010 0x5569539133f0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f2f0000c470 tx=0x7f2f0000c4a0 comp rx=0 tx=0).stop 2024-02-02T15:46:07.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.267+0000 7f2f2207eb80 1 -- 172.21.15.1:0/1082037491 shutdown_connections 2024-02-02T15:46:07.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.267+0000 7f2f2207eb80 1 --2- 172.21.15.1:0/1082037491 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55695391e7e0 0x556953920bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:07.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.267+0000 7f2f2207eb80 1 --2- 172.21.15.1:0/1082037491 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556953913010 0x5569539133f0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:07.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.267+0000 7f2f2207eb80 1 --2- 172.21.15.1:0/1082037491 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556953913930 0x55695391e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:07.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.267+0000 7f2f2207eb80 1 -- 172.21.15.1:0/1082037491 >> 172.21.15.1:0/1082037491 conn(0x556953872db0 msgr2=0x55695390f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:07.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.267+0000 7f2f2207eb80 1 -- 172.21.15.1:0/1082037491 shutdown_connections 2024-02-02T15:46:07.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.268+0000 7f2f2207eb80 1 -- 172.21.15.1:0/1082037491 wait complete. 2024-02-02T15:46:07.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.268+0000 7f2f2207eb80 1 -- start start 2024-02-02T15:46:07.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.270+0000 7f2f2207eb80 1 Processor -- start 2024-02-02T15:46:07.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.272+0000 7f2f2207eb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556953913010 0x55695391b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:07.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.272+0000 7f2f2207eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556953913930 0x55695391bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:07.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.272+0000 7f2f2207eb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55695391e7e0 0x55695391c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:07.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.272+0000 7f2f2207eb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x556953864d50 con 0x556953913010 2024-02-02T15:46:07.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.272+0000 7f2f2207eb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x556953800d80 con 0x55695391e7e0 2024-02-02T15:46:07.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.272+0000 7f2f2207eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556953800a20 con 0x556953913930 2024-02-02T15:46:07.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.272+0000 7f2f2207eb80 10 client.0 _ll_register_callbacks cb 0x556953802290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:46:07.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.273+0000 7f2f06a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556953913930 0x55695391bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:07.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.273+0000 7f2f06a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556953913930 0x55695391bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:44532/0 (socket says 172.21.15.1:44532) 2024-02-02T15:46:07.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.273+0000 7f2f06a00700 1 -- 172.21.15.1:0/3227559952 learned_addr learned my addr 172.21.15.1:0/3227559952 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:46:07.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.273+0000 7f2f07400700 1 --2- 172.21.15.1:0/3227559952 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556953913010 0x55695391b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:07.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.273+0000 7f2f07e00700 1 --2- 172.21.15.1:0/3227559952 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55695391e7e0 0x55695391c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:07.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.273+0000 7f2f06a00700 1 -- 172.21.15.1:0/3227559952 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55695391e7e0 msgr2=0x55695391c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:07.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.273+0000 7f2f06a00700 1 --2- 172.21.15.1:0/3227559952 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55695391e7e0 0x55695391c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:07.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.273+0000 7f2f06a00700 1 -- 172.21.15.1:0/3227559952 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556953913010 msgr2=0x55695391b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:07.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.273+0000 7f2f06a00700 1 --2- 172.21.15.1:0/3227559952 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556953913010 0x55695391b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:07.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.273+0000 7f2f06a00700 1 -- 172.21.15.1:0/3227559952 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f00009bf0 con 0x556953913930 2024-02-02T15:46:07.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.273+0000 7f2ef5600700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:46:07.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.273+0000 7f2f06a00700 1 --2- 172.21.15.1:0/3227559952 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556953913930 0x55695391bb60 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f2ef80a2390 tx=0x7f2ef80cafe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e21cb3c80c5158c server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:07.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.274+0000 7f2ef5600700 1 -- 172.21.15.1:0/3227559952 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef80a28c0 con 0x556953913930 2024-02-02T15:46:07.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.274+0000 7f2ef5600700 1 -- 172.21.15.1:0/3227559952 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f2ef800fe10 con 0x556953913930 2024-02-02T15:46:07.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.274+0000 7f2f2207eb80 1 -- 172.21.15.1:0/3227559952 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5569539070f0 con 0x556953913930 2024-02-02T15:46:07.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.276+0000 7f2ef5600700 1 -- 172.21.15.1:0/3227559952 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ef80cd520 con 0x556953913930 2024-02-02T15:46:07.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.276+0000 7f2f2207eb80 10 client.30280 fetch_fsmap learned FSMap version 78 2024-02-02T15:46:07.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.276+0000 7f2f2207eb80 1 -- 172.21.15.1:0/3227559952 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) v3 -- 0x556953692b60 con 0x556953913930 2024-02-02T15:46:07.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.284+0000 7f2ef5600700 1 -- 172.21.15.1:0/3227559952 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5569539070f0 con 0x556953913930 2024-02-02T15:46:07.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.284+0000 7f2ef5600700 1 -- 172.21.15.1:0/3227559952 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 78) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f2ef8022470 con 0x556953913930 2024-02-02T15:46:07.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.285+0000 7f2f2207eb80 10 client.30280 fetch_fsmap finished waiting for FSMap version 78 2024-02-02T15:46:07.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.285+0000 7f2f2207eb80 10 client.30280 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:46:07.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.285+0000 7f2f2207eb80 20 client.30280 populate_metadata read hostname 'smithi001' 2024-02-02T15:46:07.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.285+0000 7f2f2207eb80 1 --2- 172.21.15.1:0/3227559952 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55695392def0 0x556953930310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:07.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.285+0000 7f2f2207eb80 4 client.30280 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:46:07.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.285+0000 7f2f2207eb80 1 -- 172.21.15.1:0/3227559952 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55695385ddd0 con 0x55695392def0 2024-02-02T15:46:07.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.285+0000 7f2f07400700 1 --2- 172.21.15.1:0/3227559952 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55695392def0 0x556953930310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:07.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.286+0000 7f2f07400700 1 --2- 172.21.15.1:0/3227559952 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55695392def0 0x556953930310 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1dc31aa1714235d server_cookie=7967a5582193d9dc in_seq=0 out_seq=0 2024-02-02T15:46:07.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.286+0000 7f2ef5600700 10 client.30280 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:46:07.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.287+0000 7f2ef5600700 1 -- 172.21.15.1:0/3227559952 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55695385ddd0 con 0x55695392def0 2024-02-02T15:46:07.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.287+0000 7f2ef5600700 10 client.30280 handle_command_reply: tid=0 2024-02-02T15:46:07.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.287+0000 7f2f2207eb80 1 client.30280 shutdown 2024-02-02T15:46:07.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.287+0000 7f2f2207eb80 1 -- 172.21.15.1:0/3227559952 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556953913930 msgr2=0x55695391bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:07.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.287+0000 7f2f2207eb80 1 --2- 172.21.15.1:0/3227559952 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556953913930 0x55695391bb60 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f2ef80a2390 tx=0x7f2ef80cafe0 comp rx=0 tx=0).stop 2024-02-02T15:46:07.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.287+0000 7f2f2207eb80 1 -- 172.21.15.1:0/3227559952 shutdown_connections 2024-02-02T15:46:07.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.287+0000 7f2f2207eb80 1 --2- 172.21.15.1:0/3227559952 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55695392def0 0x556953930310 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:07.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.287+0000 7f2f2207eb80 1 --2- 172.21.15.1:0/3227559952 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55695391e7e0 0x55695391c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:07.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.287+0000 7f2f2207eb80 1 --2- 172.21.15.1:0/3227559952 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556953913010 0x55695391b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:07.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.287+0000 7f2f2207eb80 1 --2- 172.21.15.1:0/3227559952 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556953913930 0x55695391bb60 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:07.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.287+0000 7f2f2207eb80 1 -- 172.21.15.1:0/3227559952 >> 172.21.15.1:0/3227559952 conn(0x556953872db0 msgr2=0x55695390efa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:07.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.288+0000 7f2ef5600700 0 client.30280 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:46:07.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.288+0000 7f2f2207eb80 1 -- 172.21.15.1:0/3227559952 shutdown_connections 2024-02-02T15:46:07.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.288+0000 7f2f2207eb80 1 -- 172.21.15.1:0/3227559952 wait complete. 2024-02-02T15:46:07.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.288+0000 7f2f2207eb80 20 client.30280 trim_cache size 0 max 16384 2024-02-02T15:46:07.318 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.319+0000 7f2f2207eb80 1 Processor -- start 2024-02-02T15:46:07.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.319+0000 7f2f2207eb80 1 -- start start 2024-02-02T15:46:07.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.319+0000 7f2f2207eb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556953913010 0x556953925320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:07.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.319+0000 7f2f2207eb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556953913930 0x556953915b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:07.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.319+0000 7f2f2207eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55695391e7e0 0x556953916050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:07.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.319+0000 7f2f2207eb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569536e86a0 con 0x556953913930 2024-02-02T15:46:07.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.319+0000 7f2f2207eb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5569539107e0 con 0x556953913010 2024-02-02T15:46:07.319 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.319+0000 7f2f2207eb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x556953906fc0 con 0x55695391e7e0 2024-02-02T15:46:07.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.320+0000 7f2f06a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556953913930 0x556953915b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:07.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.320+0000 7f2f07400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556953913010 0x556953925320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:07.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.320+0000 7f2f06a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556953913930 0x556953915b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:34488/0 (socket says 172.21.15.1:34488) 2024-02-02T15:46:07.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.320+0000 7f2f06a00700 1 -- 172.21.15.1:0/360626965 learned_addr learned my addr 172.21.15.1:0/360626965 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:46:07.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.320+0000 7f2f06a00700 1 -- 172.21.15.1:0/360626965 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55695391e7e0 msgr2=0x556953916050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:07.320 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.320+0000 7f2f06a00700 1 --2- 172.21.15.1:0/360626965 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55695391e7e0 0x556953916050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:07.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.320+0000 7f2f06a00700 1 -- 172.21.15.1:0/360626965 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556953913010 msgr2=0x556953925320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:07.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.320+0000 7f2f06a00700 1 --2- 172.21.15.1:0/360626965 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556953913010 0x556953925320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:07.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.320+0000 7f2f06a00700 1 -- 172.21.15.1:0/360626965 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55695382aea0 con 0x556953913930 2024-02-02T15:46:07.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.320+0000 7f2f06a00700 1 --2- 172.21.15.1:0/360626965 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556953913930 0x556953915b10 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8024000 tx=0x7f2ef8009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75b26adf52450414 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:07.321 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.320+0000 7f2f07400700 1 --2- 172.21.15.1:0/360626965 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556953913010 0x556953925320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:46:07.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.321+0000 7f2efd600700 1 -- 172.21.15.1:0/360626965 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef8099c00 con 0x556953913930 2024-02-02T15:46:07.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.321+0000 7f2efd600700 1 -- 172.21.15.1:0/360626965 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ef800fe10 con 0x556953913930 2024-02-02T15:46:07.322 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.321+0000 7f2efd600700 1 -- 172.21.15.1:0/360626965 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef8099dd0 con 0x556953913930 2024-02-02T15:46:07.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.322+0000 7f2f2207eb80 1 -- 172.21.15.1:0/360626965 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556953913930 msgr2=0x556953915b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:07.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.322+0000 7f2f2207eb80 1 --2- 172.21.15.1:0/360626965 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556953913930 0x556953915b10 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8024000 tx=0x7f2ef8009ca0 comp rx=0 tx=0).stop 2024-02-02T15:46:07.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.322+0000 7f2f2207eb80 1 -- 172.21.15.1:0/360626965 shutdown_connections 2024-02-02T15:46:07.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.322+0000 7f2f2207eb80 1 --2- 172.21.15.1:0/360626965 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55695391e7e0 0x556953916050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:07.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.322+0000 7f2f2207eb80 1 --2- 172.21.15.1:0/360626965 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556953913010 0x556953925320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:07.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.322+0000 7f2f2207eb80 1 --2- 172.21.15.1:0/360626965 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x556953913930 0x556953915b10 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:07.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.322+0000 7f2f2207eb80 1 -- 172.21.15.1:0/360626965 >> 172.21.15.1:0/360626965 conn(0x556953872db0 msgr2=0x556953916980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:07.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.322+0000 7f2f2207eb80 1 -- 172.21.15.1:0/360626965 shutdown_connections 2024-02-02T15:46:07.323 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.322+0000 7f2f2207eb80 1 -- 172.21.15.1:0/360626965 wait complete. 2024-02-02T15:46:07.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.323+0000 7f2f2207eb80 1 -- start start 2024-02-02T15:46:07.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.323+0000 7f2f2207eb80 1 Processor -- start 2024-02-02T15:46:07.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.324+0000 7f2f2207eb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556953913010 0x556953922a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:07.324 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.324+0000 7f2f2207eb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556953913930 0x556953922f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:07.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.325+0000 7f2f2207eb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55695391e7e0 0x5569539234e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:07.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.325+0000 7f2f06a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556953913930 0x556953922f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:07.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.325+0000 7f2f07400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556953913010 0x556953922a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:07.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.325+0000 7f2f06a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556953913930 0x556953922f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:39482/0 (socket says 172.21.15.1:39482) 2024-02-02T15:46:07.325 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.325+0000 7f2f07400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556953913010 0x556953922a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:44552/0 (socket says 172.21.15.1:44552) 2024-02-02T15:46:07.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.325+0000 7f2f06a00700 1 -- 172.21.15.1:0/2701905478 learned_addr learned my addr 172.21.15.1:0/2701905478 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:46:07.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.325+0000 7f2f2207eb80 1 -- 172.21.15.1:0/2701905478 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55695397ebc0 con 0x55695391e7e0 2024-02-02T15:46:07.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.325+0000 7f2f2207eb80 1 -- 172.21.15.1:0/2701905478 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55695397ed30 con 0x556953913930 2024-02-02T15:46:07.326 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.325+0000 7f2f2207eb80 1 -- 172.21.15.1:0/2701905478 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55695397eef0 con 0x556953913010 2024-02-02T15:46:07.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.325+0000 7f2f07e00700 1 --2- 172.21.15.1:0/2701905478 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55695391e7e0 0x5569539234e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:07.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.325+0000 7f2f2207eb80 10 client.0 _ll_register_callbacks cb 0x556953802290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:46:07.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.325+0000 7f2f06a00700 1 -- 172.21.15.1:0/2701905478 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556953913010 msgr2=0x556953922a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:07.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.325+0000 7f2f06a00700 1 --2- 172.21.15.1:0/2701905478 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556953913010 0x556953922a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:07.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.326+0000 7f2f06a00700 1 -- 172.21.15.1:0/2701905478 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55695391e7e0 msgr2=0x5569539234e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:07.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.326+0000 7f2f06a00700 1 --2- 172.21.15.1:0/2701905478 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55695391e7e0 0x5569539234e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:07.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.326+0000 7f2f06a00700 1 -- 172.21.15.1:0/2701905478 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556953692b60 con 0x556953913930 2024-02-02T15:46:07.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.326+0000 7f2f07e00700 1 --2- 172.21.15.1:0/2701905478 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55695391e7e0 0x5569539234e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:46:07.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.326+0000 7f2f2207eb80 1 -- 172.21.15.1:0/2701905478 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2f0000ae50 con 0x556953913930 2024-02-02T15:46:07.327 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.326+0000 7f2f06a00700 1 --2- 172.21.15.1:0/2701905478 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556953913930 0x556953922f80 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8097350 tx=0x7f2ef80cafe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3579497905a8e220 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:07.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.326+0000 7f2ef5600700 10 client.30434 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:46:07.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.326+0000 7f2ef5600700 1 -- 172.21.15.1:0/2701905478 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef80a2430 con 0x556953913930 2024-02-02T15:46:07.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.327+0000 7f2ef5600700 1 -- 172.21.15.1:0/2701905478 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f2ef80a2590 con 0x556953913930 2024-02-02T15:46:07.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.327+0000 7f2ef5600700 1 -- 172.21.15.1:0/2701905478 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ef80966d0 con 0x556953913930 2024-02-02T15:46:07.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.327+0000 7f2f2207eb80 10 client.30434 fetch_fsmap learned FSMap version 78 2024-02-02T15:46:07.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.327+0000 7f2f2207eb80 1 -- 172.21.15.1:0/2701905478 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({fsmap=78}) v3 -- 0x5569536eacd0 con 0x556953913930 2024-02-02T15:46:07.328 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.327+0000 7f2ef5600700 1 -- 172.21.15.1:0/2701905478 <== mon.2 v2:172.21.15.71:3300/0 4 ==== fsmap(e 78) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f2ef8096830 con 0x556953913930 2024-02-02T15:46:07.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.327+0000 7f2f2207eb80 10 client.30434 fetch_fsmap finished waiting for FSMap version 78 2024-02-02T15:46:07.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.328+0000 7f2f2207eb80 10 client.30434 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:46:07.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.328+0000 7f2f2207eb80 20 client.30434 populate_metadata read hostname 'smithi001' 2024-02-02T15:46:07.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.328+0000 7f2f2207eb80 1 --2- 172.21.15.1:0/2701905478 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55695397ffd0 0x5569539823c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:07.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.328+0000 7f2f2207eb80 4 client.30434 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2024-02-02T15:46:07.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.328+0000 7f2f2207eb80 1 -- 172.21.15.1:0/2701905478 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556953823f50 con 0x55695397ffd0 2024-02-02T15:46:07.329 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.329+0000 7f2f07400700 1 --2- 172.21.15.1:0/2701905478 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55695397ffd0 0x5569539823c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:07.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.329+0000 7f2ef5600700 10 client.30434 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:46:07.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.329+0000 7f2f07400700 1 --2- 172.21.15.1:0/2701905478 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55695397ffd0 0x5569539823c0 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4166afd473a998f6 server_cookie=3bbd40459c9926d0 in_seq=0 out_seq=0 2024-02-02T15:46:07.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.330+0000 7f2ef5600700 1 -- 172.21.15.1:0/2701905478 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x556953823f50 con 0x55695397ffd0 2024-02-02T15:46:07.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.330+0000 7f2ef5600700 10 client.30434 handle_command_reply: tid=0 2024-02-02T15:46:07.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.330+0000 7f2f2207eb80 1 client.30434 shutdown 2024-02-02T15:46:07.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.332+0000 7f2f2207eb80 1 -- 172.21.15.1:0/2701905478 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556953913930 msgr2=0x556953922f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:07.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.332+0000 7f2f2207eb80 1 --2- 172.21.15.1:0/2701905478 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556953913930 0x556953922f80 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8097350 tx=0x7f2ef80cafe0 comp rx=0 tx=0).stop 2024-02-02T15:46:07.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.333+0000 7f2f2207eb80 1 -- 172.21.15.1:0/2701905478 shutdown_connections 2024-02-02T15:46:07.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.333+0000 7f2f2207eb80 1 --2- 172.21.15.1:0/2701905478 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55695397ffd0 0x5569539823c0 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:07.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.333+0000 7f2f2207eb80 1 --2- 172.21.15.1:0/2701905478 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55695391e7e0 0x5569539234e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:07.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.333+0000 7f2f2207eb80 1 --2- 172.21.15.1:0/2701905478 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x556953913010 0x556953922a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:07.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.333+0000 7f2f2207eb80 1 --2- 172.21.15.1:0/2701905478 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x556953913930 0x556953922f80 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:07.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.333+0000 7f2ef5600700 0 client.30434 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:46:07.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.333+0000 7f2f2207eb80 1 -- 172.21.15.1:0/2701905478 >> 172.21.15.1:0/2701905478 conn(0x556953872db0 msgr2=0x556953919e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:07.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.334+0000 7f2f2207eb80 1 -- 172.21.15.1:0/2701905478 shutdown_connections 2024-02-02T15:46:07.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.334+0000 7f2f2207eb80 1 -- 172.21.15.1:0/2701905478 wait complete. 2024-02-02T15:46:07.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.334+0000 7f2f2207eb80 20 client.30434 trim_cache size 0 max 16384 2024-02-02T15:46:07.334 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:46:07.335 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2024-02-02T15:46:07.335 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2024-02-02T15:46:07.335 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:46:07.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.335+0000 7f2f0e600700 1 -- 172.21.15.1:0/1049649959 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f2ee80a0790 msgr2=0x7f2ee80a2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:07.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.335+0000 7f2f0e600700 1 --2- 172.21.15.1:0/1049649959 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f2ee80a0790 0x7f2ee80a2c50 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f2f00016000 tx=0x7f2f00018000 comp rx=0 tx=0).stop 2024-02-02T15:46:07.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.335+0000 7f2f0e600700 1 -- 172.21.15.1:0/1049649959 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2f08108660 msgr2=0x7f2f081a1ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:07.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.335+0000 7f2f0e600700 1 --2- 172.21.15.1:0/1049649959 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2f08108660 0x7f2f081a1ee0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8006010 tx=0x7f2ef800f7b0 comp rx=0 tx=0).stop 2024-02-02T15:46:07.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.336+0000 7f2f0e600700 1 -- 172.21.15.1:0/1049649959 shutdown_connections 2024-02-02T15:46:07.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.336+0000 7f2f0e600700 1 --2- 172.21.15.1:0/1049649959 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f2ee80a0790 0x7f2ee80a2c50 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:07.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.336+0000 7f2f0e600700 1 --2- 172.21.15.1:0/1049649959 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2f08113e90 0x7f2f081a6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:07.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.336+0000 7f2f0e600700 1 --2- 172.21.15.1:0/1049649959 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2f08107d40 0x7f2f081a19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:07.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.336+0000 7f2f0e600700 1 --2- 172.21.15.1:0/1049649959 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2f08108660 0x7f2f081a1ee0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:07.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.336+0000 7f2f0e600700 1 -- 172.21.15.1:0/1049649959 >> 172.21.15.1:0/1049649959 conn(0x7f2f0806d120 msgr2=0x7f2f081096e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:07.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.336+0000 7f2f0e600700 1 -- 172.21.15.1:0/1049649959 shutdown_connections 2024-02-02T15:46:07.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.336+0000 7f2f0e600700 1 -- 172.21.15.1:0/1049649959 wait complete. 2024-02-02T15:46:07.366 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-02T15:46:07.366 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T15:46:07.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.667+0000 7f8ff5400700 1 -- 172.21.15.1:0/320547601 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ff0107d40 msgr2=0x7f8ff01081c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:07.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.667+0000 7f8ff5400700 1 --2- 172.21.15.1:0/320547601 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ff0107d40 0x7f8ff01081c0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f8fe800b0a0 tx=0x7f8fe800b3b0 comp rx=0 tx=0).stop 2024-02-02T15:46:07.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.668+0000 7f8ff5400700 1 -- 172.21.15.1:0/320547601 shutdown_connections 2024-02-02T15:46:07.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.668+0000 7f8ff5400700 1 --2- 172.21.15.1:0/320547601 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ff006dbd0 0x7f8ff0108700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:07.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.668+0000 7f8ff5400700 1 --2- 172.21.15.1:0/320547601 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8ff010d730 0x7f8ff010db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:07.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.668+0000 7f8ff5400700 1 --2- 172.21.15.1:0/320547601 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ff0107d40 0x7f8ff01081c0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:07.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.668+0000 7f8ff5400700 1 -- 172.21.15.1:0/320547601 >> 172.21.15.1:0/320547601 conn(0x7f8ff006d0b0 msgr2=0x7f8ff006d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:07.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.668+0000 7f8ff5400700 1 -- 172.21.15.1:0/320547601 shutdown_connections 2024-02-02T15:46:07.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.668+0000 7f8ff5400700 1 -- 172.21.15.1:0/320547601 wait complete. 2024-02-02T15:46:07.670 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.670+0000 7f8ff5400700 1 Processor -- start 2024-02-02T15:46:07.671 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:46:07 smithi142 ceph-mon[45728]: pgmap v600: 129 pgs: 129 active+clean; 2.3 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 6.6 MiB/s wr, 542 op/s 2024-02-02T15:46:07.672 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:46:07 smithi142 ceph-mon[45728]: osdmap e114: 12 total, 12 up, 12 in 2024-02-02T15:46:07.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.671+0000 7f8ff5400700 1 -- start start 2024-02-02T15:46:07.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.672+0000 7f8ff5400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ff006dbd0 0x7f8ff01aba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:07.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.672+0000 7f8ff5400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ff0107d40 0x7f8ff01abf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:07.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.672+0000 7f8ff5400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8ff010d730 0x7f8ff01a5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:07.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.672+0000 7f8ff5400700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff01a6130 con 0x7f8ff010d730 2024-02-02T15:46:07.672 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.672+0000 7f8ff5400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff01a62a0 con 0x7f8ff006dbd0 2024-02-02T15:46:07.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.672+0000 7f8ff5400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff01a6460 con 0x7f8ff0107d40 2024-02-02T15:46:07.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.673+0000 7f8fede00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ff0107d40 0x7f8ff01abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:07.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.673+0000 7f8fede00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ff0107d40 0x7f8ff01abf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:44570/0 (socket says 172.21.15.1:44570) 2024-02-02T15:46:07.673 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.673+0000 7f8fede00700 1 -- 172.21.15.1:0/3289949364 learned_addr learned my addr 172.21.15.1:0/3289949364 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:46:07.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.673+0000 7f8fef200700 1 --2- 172.21.15.1:0/3289949364 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8ff010d730 0x7f8ff01a5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:07.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.674+0000 7f8fef200700 1 -- 172.21.15.1:0/3289949364 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ff0107d40 msgr2=0x7f8ff01abf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:07.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.674+0000 7f8fef200700 1 --2- 172.21.15.1:0/3289949364 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ff0107d40 0x7f8ff01abf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:07.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.674+0000 7f8fef200700 1 -- 172.21.15.1:0/3289949364 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ff006dbd0 msgr2=0x7f8ff01aba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:07.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.674+0000 7f8fef200700 1 --2- 172.21.15.1:0/3289949364 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ff006dbd0 0x7f8ff01aba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:07.674 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.674+0000 7f8fef200700 1 -- 172.21.15.1:0/3289949364 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fe8009d00 con 0x7f8ff010d730 2024-02-02T15:46:07.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.675+0000 7f8fede00700 1 --2- 172.21.15.1:0/3289949364 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ff0107d40 0x7f8ff01abf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:46:07.675 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.675+0000 7f8fef200700 1 --2- 172.21.15.1:0/3289949364 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8ff010d730 0x7f8ff01a5ac0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f8fe000e970 tx=0x7f8fe000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:07.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.678+0000 7f8fe7400700 1 -- 172.21.15.1:0/3289949364 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe000cbd0 con 0x7f8ff010d730 2024-02-02T15:46:07.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.678+0000 7f8fe7400700 1 -- 172.21.15.1:0/3289949364 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8fe000ee60 con 0x7f8ff010d730 2024-02-02T15:46:07.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.678+0000 7f8fe7400700 1 -- 172.21.15.1:0/3289949364 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe0005020 con 0x7f8ff010d730 2024-02-02T15:46:07.678 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.678+0000 7f8ff5400700 1 -- 172.21.15.1:0/3289949364 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ff01a6740 con 0x7f8ff010d730 2024-02-02T15:46:07.679 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.678+0000 7f8ff5400700 1 -- 172.21.15.1:0/3289949364 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ff019d3a0 con 0x7f8ff010d730 2024-02-02T15:46:07.680 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.680+0000 7f8ff5400700 1 -- 172.21.15.1:0/3289949364 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ff010c110 con 0x7f8ff010d730 2024-02-02T15:46:07.686 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.685+0000 7f8fe7400700 1 -- 172.21.15.1:0/3289949364 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f8fe0004020 con 0x7f8ff010d730 2024-02-02T15:46:07.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.687+0000 7f8fe7400700 1 --2- 172.21.15.1:0/3289949364 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f8fd00a07c0 0x7f8fd00a2c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:07.687 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.687+0000 7f8fee800700 1 --2- 172.21.15.1:0/3289949364 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f8fd00a07c0 0x7f8fd00a2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:07.688 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.689+0000 7f8fee800700 1 --2- 172.21.15.1:0/3289949364 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f8fd00a07c0 0x7f8fd00a2c80 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8fd800a9b0 tx=0x7f8fd8005c90 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:07.689 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.689+0000 7f8fe7400700 1 -- 172.21.15.1:0/3289949364 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f8fe0014070 con 0x7f8ff010d730 2024-02-02T15:46:07.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:46:07 smithi001 ceph-mon[35470]: pgmap v600: 129 pgs: 129 active+clean; 2.3 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 6.6 MiB/s wr, 542 op/s 2024-02-02T15:46:07.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:46:07 smithi001 ceph-mon[35470]: osdmap e114: 12 total, 12 up, 12 in 2024-02-02T15:46:07.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:07.696+0000 7f8fe7400700 1 -- 172.21.15.1:0/3289949364 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f8fe0095520 con 0x7f8ff010d730 2024-02-02T15:46:07.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:46:07 smithi071 ceph-mon[44707]: pgmap v600: 129 pgs: 129 active+clean; 2.3 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 6.6 MiB/s wr, 542 op/s 2024-02-02T15:46:07.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:46:07 smithi071 ceph-mon[44707]: osdmap e114: 12 total, 12 up, 12 in 2024-02-02T15:46:08.011 INFO:tasks.workunit.client.0.smithi001.stdout: 10 177385 26.34 MB/sec execute 220 sec latency 409.799 ms 2024-02-02T15:46:08.129 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.129+0000 7f8ff5400700 1 -- 172.21.15.1:0/3289949364 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8ff002cc70 con 0x7f8ff010d730 2024-02-02T15:46:08.132 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.132+0000 7f8fe7400700 1 -- 172.21.15.1:0/3289949364 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+10881 (secure 0 0 0) 0x7f8fe0098b40 con 0x7f8ff010d730 2024-02-02T15:46:08.132 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:46:08.132 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":78,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2922240448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2922240448},{"type":"v1","addr":"172.21.15.1:6835","nonce":2922240448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2729497431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2729497431},{"type":"v1","addr":"172.21.15.1:6837","nonce":2729497431}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":78,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-02T15:24:47.761435+0000","modified":"2024-02-02T15:45:15.510793+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24529,"mds_2":24518,"mds_3":24514,"mds_4":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14517":{"gid":14517,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/1218303683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":1218303683},{"type":"v1","addr":"172.21.15.1:6839","nonce":1218303683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14529":{"gid":14529,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/2059197496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2059197496},{"type":"v1","addr":"172.21.15.1:6841","nonce":2059197496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/7648986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":7648986},{"type":"v1","addr":"172.21.15.142:6833","nonce":7648986}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/3482746094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3482746094},{"type":"v1","addr":"172.21.15.71:6833","nonce":3482746094}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/388207814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":388207814},{"type":"v1","addr":"172.21.15.71:6835","nonce":388207814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6835/1052606547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1052606547},{"type":"v1","addr":"172.21.15.142:6835","nonce":1052606547}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.142:6837/2870289019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2870289019},{"type":"v1","addr":"172.21.15.142:6837","nonce":2870289019}]},"join_fscid":-1,"export_targets":[0,1,2,4],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.71:6837/628016199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":628016199},{"type":"v1","addr":"172.21.15.71:6837","nonce":628016199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.142:6839/321675421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":321675421},{"type":"v1","addr":"172.21.15.142:6839","nonce":321675421}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/1268086493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1268086493},{"type":"v1","addr":"172.21.15.71:6839","nonce":1268086493}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-02T15:46:08.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.136+0000 7f8ff5400700 1 -- 172.21.15.1:0/3289949364 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f8fd00a07c0 msgr2=0x7f8fd00a2c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:08.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.136+0000 7f8ff5400700 1 --2- 172.21.15.1:0/3289949364 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f8fd00a07c0 0x7f8fd00a2c80 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8fd800a9b0 tx=0x7f8fd8005c90 comp rx=0 tx=0).stop 2024-02-02T15:46:08.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.136+0000 7f8ff5400700 1 -- 172.21.15.1:0/3289949364 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8ff010d730 msgr2=0x7f8ff01a5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:08.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.136+0000 7f8ff5400700 1 --2- 172.21.15.1:0/3289949364 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8ff010d730 0x7f8ff01a5ac0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f8fe000e970 tx=0x7f8fe000ec80 comp rx=0 tx=0).stop 2024-02-02T15:46:08.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.136+0000 7f8ff5400700 1 -- 172.21.15.1:0/3289949364 shutdown_connections 2024-02-02T15:46:08.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.136+0000 7f8ff5400700 1 --2- 172.21.15.1:0/3289949364 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f8fd00a07c0 0x7f8fd00a2c80 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:08.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.136+0000 7f8ff5400700 1 --2- 172.21.15.1:0/3289949364 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8ff010d730 0x7f8ff01a5ac0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:08.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.136+0000 7f8ff5400700 1 --2- 172.21.15.1:0/3289949364 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8ff006dbd0 0x7f8ff01aba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:08.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.136+0000 7f8ff5400700 1 --2- 172.21.15.1:0/3289949364 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8ff0107d40 0x7f8ff01abf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:08.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.136+0000 7f8ff5400700 1 -- 172.21.15.1:0/3289949364 >> 172.21.15.1:0/3289949364 conn(0x7f8ff006d0b0 msgr2=0x7f8ff0109250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:08.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.136+0000 7f8ff5400700 1 -- 172.21.15.1:0/3289949364 shutdown_connections 2024-02-02T15:46:08.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.139+0000 7f8ff5400700 1 -- 172.21.15.1:0/3289949364 wait complete. 2024-02-02T15:46:08.141 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 78 2024-02-02T15:46:08.169 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-02-02T15:46:08.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.493+0000 7eff2c200700 1 -- 172.21.15.1:0/403781281 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7eff2410d630 msgr2=0x7eff2410fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:08.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.493+0000 7eff2c200700 1 --2- 172.21.15.1:0/403781281 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7eff2410d630 0x7eff2410fa60 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7eff1c00b0a0 tx=0x7eff1c00b3b0 comp rx=0 tx=0).stop 2024-02-02T15:46:08.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.494+0000 7eff2c200700 1 -- 172.21.15.1:0/403781281 shutdown_connections 2024-02-02T15:46:08.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.494+0000 7eff2c200700 1 --2- 172.21.15.1:0/403781281 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7eff2410d630 0x7eff2410fa60 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:08.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.494+0000 7eff2c200700 1 --2- 172.21.15.1:0/403781281 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff24073110 0x7eff240734f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:08.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.494+0000 7eff2c200700 1 --2- 172.21.15.1:0/403781281 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff24073a30 0x7eff2410d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:08.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.494+0000 7eff2c200700 1 -- 172.21.15.1:0/403781281 >> 172.21.15.1:0/403781281 conn(0x7eff240fc840 msgr2=0x7eff240fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:08.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.496+0000 7eff2c200700 1 -- 172.21.15.1:0/403781281 shutdown_connections 2024-02-02T15:46:08.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.496+0000 7eff2c200700 1 -- 172.21.15.1:0/403781281 wait complete. 2024-02-02T15:46:08.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.498+0000 7eff2c200700 1 Processor -- start 2024-02-02T15:46:08.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.499+0000 7eff2c200700 1 -- start start 2024-02-02T15:46:08.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.500+0000 7eff2c200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7eff24073110 0x7eff24199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:08.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.500+0000 7eff2c200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff24073a30 0x7eff24199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:08.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.500+0000 7eff2c200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff2410d630 0x7eff2419dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:08.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.500+0000 7eff2c200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff2419e230 con 0x7eff2410d630 2024-02-02T15:46:08.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.500+0000 7eff2c200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff2419e3a0 con 0x7eff24073110 2024-02-02T15:46:08.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.500+0000 7eff2c200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff2419e560 con 0x7eff24073a30 2024-02-02T15:46:08.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.501+0000 7eff29800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7eff24073110 0x7eff24199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:08.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.501+0000 7eff29800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7eff24073110 0x7eff24199250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:39532/0 (socket says 172.21.15.1:39532) 2024-02-02T15:46:08.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.501+0000 7eff29800700 1 -- 172.21.15.1:0/843308916 learned_addr learned my addr 172.21.15.1:0/843308916 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:46:08.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.501+0000 7eff28e00700 1 --2- 172.21.15.1:0/843308916 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff24073a30 0x7eff24199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:08.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.502+0000 7eff29800700 1 -- 172.21.15.1:0/843308916 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff24073a30 msgr2=0x7eff24199790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:08.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.502+0000 7eff29800700 1 --2- 172.21.15.1:0/843308916 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff24073a30 0x7eff24199790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:08.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.502+0000 7eff29800700 1 -- 172.21.15.1:0/843308916 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff2410d630 msgr2=0x7eff2419dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:08.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.502+0000 7eff29800700 1 --2- 172.21.15.1:0/843308916 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff2410d630 0x7eff2419dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:08.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.502+0000 7eff29800700 1 -- 172.21.15.1:0/843308916 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff1c009d00 con 0x7eff24073110 2024-02-02T15:46:08.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.502+0000 7eff29800700 1 --2- 172.21.15.1:0/843308916 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7eff24073110 0x7eff24199250 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7eff0c00b530 tx=0x7eff0c00b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:08.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.503+0000 7eff22000700 1 -- 172.21.15.1:0/843308916 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff0c011830 con 0x7eff24073110 2024-02-02T15:46:08.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.503+0000 7eff22000700 1 -- 172.21.15.1:0/843308916 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff0c011e50 con 0x7eff24073110 2024-02-02T15:46:08.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.503+0000 7eff2c200700 1 -- 172.21.15.1:0/843308916 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff2419e840 con 0x7eff24073110 2024-02-02T15:46:08.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.503+0000 7eff22000700 1 -- 172.21.15.1:0/843308916 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff0c00f2e0 con 0x7eff24073110 2024-02-02T15:46:08.503 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.503+0000 7eff2c200700 1 -- 172.21.15.1:0/843308916 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff241a6080 con 0x7eff24073110 2024-02-02T15:46:08.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.507+0000 7eff22000700 1 -- 172.21.15.1:0/843308916 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7eff0c010580 con 0x7eff24073110 2024-02-02T15:46:08.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.509+0000 7eff3fd3bb80 1 Processor -- start 2024-02-02T15:46:08.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.509+0000 7eff3fd3bb80 1 -- start start 2024-02-02T15:46:08.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.510+0000 7eff3fd3bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e1ab564010 0x55e1ab5643f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:08.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.510+0000 7eff3fd3bb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e1ab564930 0x55e1ab56f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:08.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.510+0000 7eff3fd3bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e1ab56f7e0 0x55e1ab571bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:08.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.510+0000 7eff3fd3bb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1ab4d84c0 con 0x55e1ab564930 2024-02-02T15:46:08.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.510+0000 7eff3fd3bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1ab4bcaa0 con 0x55e1ab564010 2024-02-02T15:46:08.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.510+0000 7eff3fd3bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1ab4ba460 con 0x55e1ab56f7e0 2024-02-02T15:46:08.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.510+0000 7eff2a200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e1ab564010 0x55e1ab5643f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:08.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.511+0000 7eff22000700 1 --2- 172.21.15.1:0/843308916 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7eff040a0790 0x7eff040a2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:08.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.512+0000 7eff22000700 1 -- 172.21.15.1:0/843308916 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 9541+0+0 (secure 0 0 0) 0x7eff0c0cbb80 con 0x7eff24073110 2024-02-02T15:46:08.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.513+0000 7eff28e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e1ab56f7e0 0x55e1ab571bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:08.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.513+0000 7eff29800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e1ab564930 0x55e1ab56f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:08.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.513+0000 7eff28e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e1ab56f7e0 0x55e1ab571bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:44596/0 (socket says 172.21.15.1:44596) 2024-02-02T15:46:08.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.513+0000 7eff28e00700 1 -- 172.21.15.1:0/3041182952 learned_addr learned my addr 172.21.15.1:0/3041182952 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:46:08.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.514+0000 7eff28e00700 1 -- 172.21.15.1:0/3041182952 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e1ab564010 msgr2=0x55e1ab5643f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:08.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.514+0000 7eff28e00700 1 --2- 172.21.15.1:0/3041182952 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e1ab564010 0x55e1ab5643f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:08.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.514+0000 7eff28e00700 1 -- 172.21.15.1:0/3041182952 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e1ab564930 msgr2=0x55e1ab56f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:08.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.514+0000 7eff28e00700 1 --2- 172.21.15.1:0/3041182952 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e1ab564930 0x55e1ab56f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:08.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.514+0000 7eff28e00700 1 -- 172.21.15.1:0/3041182952 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e1ab3373b0 con 0x55e1ab56f7e0 2024-02-02T15:46:08.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.514+0000 7eff29800700 1 --2- 172.21.15.1:0/3041182952 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e1ab564930 0x55e1ab56f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:46:08.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.514+0000 7eff2a200700 1 --2- 172.21.15.1:0/843308916 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7eff040a0790 0x7eff040a2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:08.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.515+0000 7eff28e00700 1 --2- 172.21.15.1:0/3041182952 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e1ab56f7e0 0x55e1ab571bd0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7eff14009ce0 tx=0x7eff140113e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb5956677f0d517e server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:08.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.516+0000 7eff1aa00700 1 -- 172.21.15.1:0/3041182952 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff14011c60 con 0x55e1ab56f7e0 2024-02-02T15:46:08.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.516+0000 7eff2a200700 1 --2- 172.21.15.1:0/843308916 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7eff040a0790 0x7eff040a2c50 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7eff1c009cd0 tx=0x7eff1c009be0 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:08.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.516+0000 7eff1aa00700 1 -- 172.21.15.1:0/3041182952 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff14011dc0 con 0x55e1ab56f7e0 2024-02-02T15:46:08.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.516+0000 7eff1aa00700 1 -- 172.21.15.1:0/3041182952 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1400f7c0 con 0x55e1ab56f7e0 2024-02-02T15:46:08.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.517+0000 7eff3fd3bb80 1 -- 172.21.15.1:0/3041182952 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e1ab56f7e0 msgr2=0x55e1ab571bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:08.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.517+0000 7eff3fd3bb80 1 --2- 172.21.15.1:0/3041182952 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e1ab56f7e0 0x55e1ab571bd0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7eff14009ce0 tx=0x7eff140113e0 comp rx=0 tx=0).stop 2024-02-02T15:46:08.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.517+0000 7eff3fd3bb80 1 -- 172.21.15.1:0/3041182952 shutdown_connections 2024-02-02T15:46:08.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.517+0000 7eff3fd3bb80 1 --2- 172.21.15.1:0/3041182952 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e1ab56f7e0 0x55e1ab571bd0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:08.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.517+0000 7eff3fd3bb80 1 --2- 172.21.15.1:0/3041182952 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e1ab564010 0x55e1ab5643f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:08.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.517+0000 7eff3fd3bb80 1 --2- 172.21.15.1:0/3041182952 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e1ab564930 0x55e1ab56f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:08.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.517+0000 7eff3fd3bb80 1 -- 172.21.15.1:0/3041182952 >> 172.21.15.1:0/3041182952 conn(0x55e1ab4c3db0 msgr2=0x55e1ab5603b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:08.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.518+0000 7eff3fd3bb80 1 -- 172.21.15.1:0/3041182952 shutdown_connections 2024-02-02T15:46:08.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.518+0000 7eff3fd3bb80 1 -- 172.21.15.1:0/3041182952 wait complete. 2024-02-02T15:46:08.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.519+0000 7eff3fd3bb80 1 -- start start 2024-02-02T15:46:08.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.520+0000 7eff3fd3bb80 1 Processor -- start 2024-02-02T15:46:08.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.521+0000 7eff3fd3bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e1ab564010 0x55e1ab56f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:08.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.521+0000 7eff3fd3bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e1ab564930 0x55e1ab56c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:08.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.521+0000 7eff3fd3bb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e1ab56f7e0 0x55e1ab56cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:08.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.521+0000 7eff28e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e1ab564930 0x55e1ab56c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:08.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.521+0000 7eff29800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e1ab564010 0x55e1ab56f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:08.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.521+0000 7eff3fd3bb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1ab4b5d50 con 0x55e1ab56f7e0 2024-02-02T15:46:08.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.521+0000 7eff3fd3bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1ab451d80 con 0x55e1ab564010 2024-02-02T15:46:08.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.521+0000 7eff3fd3bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1ab451a20 con 0x55e1ab564930 2024-02-02T15:46:08.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.521+0000 7eff28e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e1ab564930 0x55e1ab56c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:44604/0 (socket says 172.21.15.1:44604) 2024-02-02T15:46:08.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.521+0000 7eff28e00700 1 -- 172.21.15.1:0/2122451990 learned_addr learned my addr 172.21.15.1:0/2122451990 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:46:08.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.521+0000 7eff3fd3bb80 10 client.0 _ll_register_callbacks cb 0x55e1ab453290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:46:08.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.522+0000 7eff28e00700 1 -- 172.21.15.1:0/2122451990 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e1ab564010 msgr2=0x55e1ab56f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:08.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.521+0000 7eff29800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e1ab564010 0x55e1ab56f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:39554/0 (socket says 172.21.15.1:39554) 2024-02-02T15:46:08.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.522+0000 7eff28e00700 1 --2- 172.21.15.1:0/2122451990 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e1ab564010 0x55e1ab56f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:08.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.523+0000 7eff28e00700 1 -- 172.21.15.1:0/2122451990 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e1ab56f7e0 msgr2=0x55e1ab56cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:08.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.523+0000 7eff28e00700 1 --2- 172.21.15.1:0/2122451990 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e1ab56f7e0 0x55e1ab56cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:08.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.523+0000 7eff28e00700 1 -- 172.21.15.1:0/2122451990 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff241a6080 con 0x55e1ab564930 2024-02-02T15:46:08.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.523+0000 7eff29800700 1 --2- 172.21.15.1:0/2122451990 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e1ab564010 0x55e1ab56f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:46:08.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.525+0000 7eff28e00700 1 --2- 172.21.15.1:0/2122451990 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e1ab564930 0x55e1ab56c480 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7eff14011fe0 tx=0x7eff14004320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2755867c22bdc5f server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:08.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.525+0000 7eff13400700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:46:08.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.530+0000 7eff13400700 1 -- 172.21.15.1:0/2122451990 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff14012830 con 0x55e1ab564930 2024-02-02T15:46:08.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.530+0000 7eff13400700 1 -- 172.21.15.1:0/2122451990 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 9541+0+0 (secure 0 0 0) 0x7eff14012990 con 0x55e1ab564930 2024-02-02T15:46:08.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.530+0000 7eff3fd3bb80 1 -- 172.21.15.1:0/2122451990 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e1ab5580f0 con 0x55e1ab564930 2024-02-02T15:46:08.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.531+0000 7eff13400700 1 -- 172.21.15.1:0/2122451990 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff14012b50 con 0x55e1ab564930 2024-02-02T15:46:08.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.531+0000 7eff3fd3bb80 10 client.30298 fetch_fsmap learned FSMap version 78 2024-02-02T15:46:08.531 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.531+0000 7eff3fd3bb80 1 -- 172.21.15.1:0/2122451990 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) v3 -- 0x7eff14009910 con 0x55e1ab564930 2024-02-02T15:46:08.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.536+0000 7eff13400700 1 -- 172.21.15.1:0/2122451990 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e1ab5580f0 con 0x55e1ab564930 2024-02-02T15:46:08.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.536+0000 7eff13400700 1 -- 172.21.15.1:0/2122451990 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 78) v1 ==== 7506+0+0 (secure 0 0 0) 0x7eff14022070 con 0x55e1ab564930 2024-02-02T15:46:08.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.536+0000 7eff3fd3bb80 10 client.30298 fetch_fsmap finished waiting for FSMap version 78 2024-02-02T15:46:08.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.536+0000 7eff3fd3bb80 10 client.30298 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-02-02T15:46:08.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.536+0000 7eff3fd3bb80 20 client.30298 populate_metadata read hostname 'smithi001' 2024-02-02T15:46:08.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.536+0000 7eff3fd3bb80 1 --2- 172.21.15.1:0/2122451990 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55e1ab57f170 0x55e1ab581590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:08.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.537+0000 7eff3fd3bb80 4 client.30298 mds_command: new command op to 24502 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:46:08.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.537+0000 7eff3fd3bb80 1 -- 172.21.15.1:0/2122451990 --> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e1ab4aedd0 con 0x55e1ab57f170 2024-02-02T15:46:08.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.537+0000 7eff29800700 1 --2- 172.21.15.1:0/2122451990 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55e1ab57f170 0x55e1ab581590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:08.545 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.546+0000 7eff29800700 1 --2- 172.21.15.1:0/2122451990 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55e1ab57f170 0x55e1ab581590 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=567b9cfd41127916 server_cookie=92ebdf65ab81e616 in_seq=0 out_seq=0 2024-02-02T15:46:08.546 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.546+0000 7eff13400700 10 client.30298 ms_handle_connect on v2:172.21.15.142:6834/1052606547 2024-02-02T15:46:08.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.551+0000 7eff13400700 1 -- 172.21.15.1:0/2122451990 <== mds.4 v2:172.21.15.142:6834/1052606547 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55e1ab4aedd0 con 0x55e1ab57f170 2024-02-02T15:46:08.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.551+0000 7eff13400700 10 client.30298 handle_command_reply: tid=0 2024-02-02T15:46:08.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.551+0000 7eff3fd3bb80 1 client.30298 shutdown 2024-02-02T15:46:08.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.552+0000 7eff3fd3bb80 1 -- 172.21.15.1:0/2122451990 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e1ab564930 msgr2=0x55e1ab56c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:08.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.552+0000 7eff3fd3bb80 1 --2- 172.21.15.1:0/2122451990 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e1ab564930 0x55e1ab56c480 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7eff14011fe0 tx=0x7eff14004320 comp rx=0 tx=0).stop 2024-02-02T15:46:08.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.552+0000 7eff3fd3bb80 1 -- 172.21.15.1:0/2122451990 shutdown_connections 2024-02-02T15:46:08.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.552+0000 7eff3fd3bb80 1 --2- 172.21.15.1:0/2122451990 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55e1ab57f170 0x55e1ab581590 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:08.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.552+0000 7eff3fd3bb80 1 --2- 172.21.15.1:0/2122451990 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e1ab56f7e0 0x55e1ab56cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:08.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.552+0000 7eff3fd3bb80 1 --2- 172.21.15.1:0/2122451990 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e1ab564010 0x55e1ab56f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:08.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.552+0000 7eff3fd3bb80 1 --2- 172.21.15.1:0/2122451990 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e1ab564930 0x55e1ab56c480 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:08.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.552+0000 7eff3fd3bb80 1 -- 172.21.15.1:0/2122451990 >> 172.21.15.1:0/2122451990 conn(0x55e1ab4c3db0 msgr2=0x55e1ab55ffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:08.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.552+0000 7eff13400700 0 client.30298 ms_handle_reset on v2:172.21.15.142:6834/1052606547 2024-02-02T15:46:08.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.552+0000 7eff3fd3bb80 1 -- 172.21.15.1:0/2122451990 shutdown_connections 2024-02-02T15:46:08.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.552+0000 7eff3fd3bb80 1 -- 172.21.15.1:0/2122451990 wait complete. 2024-02-02T15:46:08.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.553+0000 7eff3fd3bb80 20 client.30298 trim_cache size 0 max 16384 2024-02-02T15:46:08.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.575+0000 7eff3fd3bb80 1 Processor -- start 2024-02-02T15:46:08.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.575+0000 7eff3fd3bb80 1 -- start start 2024-02-02T15:46:08.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.576+0000 7eff3fd3bb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e1ab564010 0x55e1ab56eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:08.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.576+0000 7eff3fd3bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e1ab564930 0x55e1ab566840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:08.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.576+0000 7eff3fd3bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e1ab56f7e0 0x55e1ab566d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:08.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.576+0000 7eff3fd3bb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1ab3396a0 con 0x55e1ab564010 2024-02-02T15:46:08.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.576+0000 7eff3fd3bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1ab57bc40 con 0x55e1ab564930 2024-02-02T15:46:08.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.576+0000 7eff3fd3bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1ab5672c0 con 0x55e1ab56f7e0 2024-02-02T15:46:08.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.576+0000 7eff28e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e1ab564930 0x55e1ab566840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:08.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.577+0000 7eff28e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e1ab564930 0x55e1ab566840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:39564/0 (socket says 172.21.15.1:39564) 2024-02-02T15:46:08.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.577+0000 7eff28e00700 1 -- 172.21.15.1:0/449109948 learned_addr learned my addr 172.21.15.1:0/449109948 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:46:08.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.577+0000 7eff2a200700 1 --2- 172.21.15.1:0/449109948 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e1ab56f7e0 0x55e1ab566d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:08.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.577+0000 7eff29800700 1 --2- 172.21.15.1:0/449109948 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e1ab564010 0x55e1ab56eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:08.577 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.577+0000 7eff28e00700 1 -- 172.21.15.1:0/449109948 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e1ab56f7e0 msgr2=0x55e1ab566d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:08.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.577+0000 7eff28e00700 1 --2- 172.21.15.1:0/449109948 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e1ab56f7e0 0x55e1ab566d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:08.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.577+0000 7eff28e00700 1 -- 172.21.15.1:0/449109948 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e1ab564010 msgr2=0x55e1ab56eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:08.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.577+0000 7eff28e00700 1 --2- 172.21.15.1:0/449109948 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e1ab564010 0x55e1ab56eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:08.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.577+0000 7eff28e00700 1 -- 172.21.15.1:0/449109948 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e1ab55c0d0 con 0x55e1ab564930 2024-02-02T15:46:08.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.578+0000 7eff28e00700 1 --2- 172.21.15.1:0/449109948 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e1ab564930 0x55e1ab566840 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7eff14012e70 tx=0x7eff140113e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34188efaa0443ca5 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:08.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.578+0000 7eff1b400700 1 -- 172.21.15.1:0/449109948 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1402cd10 con 0x55e1ab564930 2024-02-02T15:46:08.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.578+0000 7eff1b400700 1 -- 172.21.15.1:0/449109948 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff14005900 con 0x55e1ab564930 2024-02-02T15:46:08.578 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.578+0000 7eff1b400700 1 -- 172.21.15.1:0/449109948 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff14012ba0 con 0x55e1ab564930 2024-02-02T15:46:08.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.579+0000 7eff3fd3bb80 1 -- 172.21.15.1:0/449109948 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e1ab564930 msgr2=0x55e1ab566840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:08.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.579+0000 7eff3fd3bb80 1 --2- 172.21.15.1:0/449109948 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e1ab564930 0x55e1ab566840 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7eff14012e70 tx=0x7eff140113e0 comp rx=0 tx=0).stop 2024-02-02T15:46:08.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.580+0000 7eff3fd3bb80 1 -- 172.21.15.1:0/449109948 shutdown_connections 2024-02-02T15:46:08.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.580+0000 7eff3fd3bb80 1 --2- 172.21.15.1:0/449109948 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e1ab56f7e0 0x55e1ab566d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:08.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.580+0000 7eff3fd3bb80 1 --2- 172.21.15.1:0/449109948 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e1ab564010 0x55e1ab56eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:08.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.580+0000 7eff3fd3bb80 1 --2- 172.21.15.1:0/449109948 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e1ab564930 0x55e1ab566840 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:08.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.580+0000 7eff3fd3bb80 1 -- 172.21.15.1:0/449109948 >> 172.21.15.1:0/449109948 conn(0x55e1ab4c3db0 msgr2=0x55e1ab5677a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:08.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.580+0000 7eff3fd3bb80 1 -- 172.21.15.1:0/449109948 shutdown_connections 2024-02-02T15:46:08.580 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.580+0000 7eff3fd3bb80 1 -- 172.21.15.1:0/449109948 wait complete. 2024-02-02T15:46:08.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.581+0000 7eff3fd3bb80 1 -- start start 2024-02-02T15:46:08.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.581+0000 7eff3fd3bb80 1 Processor -- start 2024-02-02T15:46:08.581 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.582+0000 7eff3fd3bb80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e1ab564010 0x55e1ab576390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:08.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.582+0000 7eff3fd3bb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e1ab564930 0x55e1ab573720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:08.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.582+0000 7eff3fd3bb80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e1ab56f7e0 0x55e1ab573e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:08.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.582+0000 7eff3fd3bb80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1ab574550 con 0x55e1ab564010 2024-02-02T15:46:08.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.582+0000 7eff3fd3bb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1ab5cfcb0 con 0x55e1ab564930 2024-02-02T15:46:08.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.582+0000 7eff3fd3bb80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1ab5cfe20 con 0x55e1ab56f7e0 2024-02-02T15:46:08.582 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.582+0000 7eff3fd3bb80 10 client.0 _ll_register_callbacks cb 0x55e1ab453290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:46:08.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.582+0000 7eff28e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e1ab564930 0x55e1ab573720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:08.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.583+0000 7eff28e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e1ab564930 0x55e1ab573720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:39578/0 (socket says 172.21.15.1:39578) 2024-02-02T15:46:08.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.583+0000 7eff28e00700 1 -- 172.21.15.1:0/341636138 learned_addr learned my addr 172.21.15.1:0/341636138 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:46:08.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.583+0000 7eff29800700 1 --2- 172.21.15.1:0/341636138 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e1ab564010 0x55e1ab576390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:08.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.583+0000 7eff28e00700 1 -- 172.21.15.1:0/341636138 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e1ab56f7e0 msgr2=0x55e1ab573e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:08.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.583+0000 7eff28e00700 1 --2- 172.21.15.1:0/341636138 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e1ab56f7e0 0x55e1ab573e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:08.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.583+0000 7eff28e00700 1 -- 172.21.15.1:0/341636138 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e1ab564010 msgr2=0x55e1ab576390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:08.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.583+0000 7eff28e00700 1 --2- 172.21.15.1:0/341636138 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e1ab564010 0x55e1ab576390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:08.583 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.583+0000 7eff28e00700 1 -- 172.21.15.1:0/341636138 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff14009910 con 0x55e1ab564930 2024-02-02T15:46:08.584 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.583+0000 7eff28e00700 1 --2- 172.21.15.1:0/341636138 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e1ab564930 0x55e1ab573720 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7eff14012e70 tx=0x7eff14004930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43f08f47bcdfa2f server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:08.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.583+0000 7eff13400700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:46:08.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.584+0000 7eff13400700 1 -- 172.21.15.1:0/341636138 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff14012ea0 con 0x55e1ab564930 2024-02-02T15:46:08.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.584+0000 7eff13400700 1 -- 172.21.15.1:0/341636138 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 9541+0+0 (secure 0 0 0) 0x7eff14000e40 con 0x55e1ab564930 2024-02-02T15:46:08.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.584+0000 7eff3fd3bb80 1 -- 172.21.15.1:0/341636138 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7eff14011dc0 con 0x55e1ab564930 2024-02-02T15:46:08.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.584+0000 7eff13400700 1 -- 172.21.15.1:0/341636138 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff14012730 con 0x55e1ab564930 2024-02-02T15:46:08.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.584+0000 7eff3fd3bb80 10 client.30449 fetch_fsmap learned FSMap version 78 2024-02-02T15:46:08.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.585+0000 7eff3fd3bb80 1 -- 172.21.15.1:0/341636138 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) v3 -- 0x55e1ab2e3b60 con 0x55e1ab564930 2024-02-02T15:46:08.585 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.585+0000 7eff13400700 1 -- 172.21.15.1:0/341636138 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7eff14011dc0 con 0x55e1ab564930 2024-02-02T15:46:08.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.585+0000 7eff13400700 1 -- 172.21.15.1:0/341636138 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 78) v1 ==== 7506+0+0 (secure 0 0 0) 0x7eff14011450 con 0x55e1ab564930 2024-02-02T15:46:08.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.585+0000 7eff3fd3bb80 10 client.30449 fetch_fsmap finished waiting for FSMap version 78 2024-02-02T15:46:08.586 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.585+0000 7eff3fd3bb80 10 client.30449 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-02-02T15:46:08.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.585+0000 7eff3fd3bb80 20 client.30449 populate_metadata read hostname 'smithi001' 2024-02-02T15:46:08.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.585+0000 7eff3fd3bb80 1 --2- 172.21.15.1:0/341636138 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55e1ab5d11a0 0x55e1ab5d35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:08.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.585+0000 7eff3fd3bb80 4 client.30449 mds_command: new command op to 24502 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:46:08.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.585+0000 7eff3fd3bb80 1 -- 172.21.15.1:0/341636138 --> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e1ab474f50 con 0x55e1ab5d11a0 2024-02-02T15:46:08.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.586+0000 7eff29800700 1 --2- 172.21.15.1:0/341636138 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55e1ab5d11a0 0x55e1ab5d35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:08.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.586+0000 7eff29800700 1 --2- 172.21.15.1:0/341636138 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55e1ab5d11a0 0x55e1ab5d35c0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e8410dcf6ce2ecda server_cookie=871da72bbb1feb36 in_seq=0 out_seq=0 2024-02-02T15:46:08.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.586+0000 7eff13400700 10 client.30449 ms_handle_connect on v2:172.21.15.142:6834/1052606547 2024-02-02T15:46:08.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.586+0000 7eff13400700 1 -- 172.21.15.1:0/341636138 <== mds.4 v2:172.21.15.142:6834/1052606547 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e1ab474f50 con 0x55e1ab5d11a0 2024-02-02T15:46:08.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.586+0000 7eff13400700 10 client.30449 handle_command_reply: tid=0 2024-02-02T15:46:08.587 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.586+0000 7eff3fd3bb80 1 client.30449 shutdown 2024-02-02T15:46:08.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.589+0000 7eff3fd3bb80 1 -- 172.21.15.1:0/341636138 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e1ab564930 msgr2=0x55e1ab573720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:08.588 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.589+0000 7eff3fd3bb80 1 --2- 172.21.15.1:0/341636138 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e1ab564930 0x55e1ab573720 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7eff14012e70 tx=0x7eff14004930 comp rx=0 tx=0).stop 2024-02-02T15:46:08.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.589+0000 7eff3fd3bb80 1 -- 172.21.15.1:0/341636138 shutdown_connections 2024-02-02T15:46:08.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.589+0000 7eff3fd3bb80 1 --2- 172.21.15.1:0/341636138 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55e1ab5d11a0 0x55e1ab5d35c0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:08.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.589+0000 7eff3fd3bb80 1 --2- 172.21.15.1:0/341636138 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e1ab56f7e0 0x55e1ab573e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:08.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.589+0000 7eff3fd3bb80 1 --2- 172.21.15.1:0/341636138 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e1ab564010 0x55e1ab576390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:08.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.589+0000 7eff3fd3bb80 1 --2- 172.21.15.1:0/341636138 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e1ab564930 0x55e1ab573720 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:08.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.589+0000 7eff13400700 0 client.30449 ms_handle_reset on v2:172.21.15.142:6834/1052606547 2024-02-02T15:46:08.589 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.589+0000 7eff3fd3bb80 1 -- 172.21.15.1:0/341636138 >> 172.21.15.1:0/341636138 conn(0x55e1ab4c3db0 msgr2=0x55e1ab55f5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:08.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.589+0000 7eff3fd3bb80 1 -- 172.21.15.1:0/341636138 shutdown_connections 2024-02-02T15:46:08.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.589+0000 7eff3fd3bb80 1 -- 172.21.15.1:0/341636138 wait complete. 2024-02-02T15:46:08.590 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.590+0000 7eff3fd3bb80 20 client.30449 trim_cache size 0 max 16384 2024-02-02T15:46:08.590 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:46:08.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.590+0000 7eff2c200700 1 -- 172.21.15.1:0/843308916 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7eff040a0790 msgr2=0x7eff040a2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:08.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.590+0000 7eff2c200700 1 --2- 172.21.15.1:0/843308916 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7eff040a0790 0x7eff040a2c50 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7eff1c009cd0 tx=0x7eff1c009be0 comp rx=0 tx=0).stop 2024-02-02T15:46:08.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.591+0000 7eff2c200700 1 -- 172.21.15.1:0/843308916 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7eff24073110 msgr2=0x7eff24199250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:08.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.591+0000 7eff2c200700 1 --2- 172.21.15.1:0/843308916 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7eff24073110 0x7eff24199250 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7eff0c00b530 tx=0x7eff0c00b8f0 comp rx=0 tx=0).stop 2024-02-02T15:46:08.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.592+0000 7eff2c200700 1 -- 172.21.15.1:0/843308916 shutdown_connections 2024-02-02T15:46:08.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.592+0000 7eff2c200700 1 --2- 172.21.15.1:0/843308916 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7eff040a0790 0x7eff040a2c50 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:08.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.592+0000 7eff2c200700 1 --2- 172.21.15.1:0/843308916 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7eff2410d630 0x7eff2419dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:08.591 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.592+0000 7eff2c200700 1 --2- 172.21.15.1:0/843308916 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7eff24073110 0x7eff24199250 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:08.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.592+0000 7eff2c200700 1 --2- 172.21.15.1:0/843308916 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7eff24073a30 0x7eff24199790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:08.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.592+0000 7eff2c200700 1 -- 172.21.15.1:0/843308916 >> 172.21.15.1:0/843308916 conn(0x7eff240fc840 msgr2=0x7eff2410f730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:08.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.592+0000 7eff2c200700 1 -- 172.21.15.1:0/843308916 shutdown_connections 2024-02-02T15:46:08.592 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.592+0000 7eff2c200700 1 -- 172.21.15.1:0/843308916 wait complete. 2024-02-02T15:46:08.617 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-02-02T15:46:08.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:46:08 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/3289949364' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:46:08.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.912+0000 7f0c83600700 1 -- 172.21.15.1:0/3107462916 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c7c075270 msgr2=0x7f0c7c075650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:08.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.912+0000 7f0c83600700 1 --2- 172.21.15.1:0/3107462916 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c7c075270 0x7f0c7c075650 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c009990 tx=0x7f0c6c009ca0 comp rx=0 tx=0).stop 2024-02-02T15:46:08.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.912+0000 7f0c83600700 1 -- 172.21.15.1:0/3107462916 shutdown_connections 2024-02-02T15:46:08.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.912+0000 7f0c83600700 1 --2- 172.21.15.1:0/3107462916 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c7c111c30 0x7f0c7c114020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:08.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.912+0000 7f0c83600700 1 --2- 172.21.15.1:0/3107462916 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c7c075270 0x7f0c7c075650 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:08.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.912+0000 7f0c83600700 1 --2- 172.21.15.1:0/3107462916 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c7c075b90 0x7f0c7c1116f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:08.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.912+0000 7f0c83600700 1 -- 172.21.15.1:0/3107462916 >> 172.21.15.1:0/3107462916 conn(0x7f0c7c0feaa0 msgr2=0x7f0c7c100ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:08.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.913+0000 7f0c83600700 1 -- 172.21.15.1:0/3107462916 shutdown_connections 2024-02-02T15:46:08.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.913+0000 7f0c83600700 1 -- 172.21.15.1:0/3107462916 wait complete. 2024-02-02T15:46:08.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.915+0000 7f0c83600700 1 Processor -- start 2024-02-02T15:46:08.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.915+0000 7f0c83600700 1 -- start start 2024-02-02T15:46:08.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.915+0000 7f0c83600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c7c075270 0x7f0c7c19d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:08.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.915+0000 7f0c83600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c7c075b90 0x7f0c7c19d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:08.915 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.915+0000 7f0c83600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c7c111c30 0x7f0c7c1a1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:08.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.915+0000 7f0c83600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7c1a2480 con 0x7f0c7c075270 2024-02-02T15:46:08.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.915+0000 7f0c83600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7c1a25f0 con 0x7f0c7c075b90 2024-02-02T15:46:08.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.915+0000 7f0c83600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7c1a27b0 con 0x7f0c7c111c30 2024-02-02T15:46:08.916 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.916+0000 7f0c7be00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c7c075b90 0x7f0c7c19d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:08.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.916+0000 7f0c7be00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c7c075b90 0x7f0c7c19d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:39606/0 (socket says 172.21.15.1:39606) 2024-02-02T15:46:08.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.916+0000 7f0c7be00700 1 -- 172.21.15.1:0/1163160751 learned_addr learned my addr 172.21.15.1:0/1163160751 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:46:08.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.916+0000 7f0c80c00700 1 --2- 172.21.15.1:0/1163160751 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c7c075270 0x7f0c7c19d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:08.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.916+0000 7f0c81600700 1 --2- 172.21.15.1:0/1163160751 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c7c111c30 0x7f0c7c1a1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:08.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.916+0000 7f0c7be00700 1 -- 172.21.15.1:0/1163160751 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c7c111c30 msgr2=0x7f0c7c1a1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:08.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.916+0000 7f0c7be00700 1 --2- 172.21.15.1:0/1163160751 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c7c111c30 0x7f0c7c1a1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:08.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.916+0000 7f0c7be00700 1 -- 172.21.15.1:0/1163160751 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c7c075270 msgr2=0x7f0c7c19d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:08.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.916+0000 7f0c7be00700 1 --2- 172.21.15.1:0/1163160751 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c7c075270 0x7f0c7c19d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:08.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.916+0000 7f0c7be00700 1 -- 172.21.15.1:0/1163160751 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c6c0095f0 con 0x7f0c7c075b90 2024-02-02T15:46:08.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.917+0000 7f0c7be00700 1 --2- 172.21.15.1:0/1163160751 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c7c075b90 0x7f0c7c19d9e0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f0c6400d950 tx=0x7f0c6400dd10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:08.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.917+0000 7f0c79600700 1 -- 172.21.15.1:0/1163160751 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c64009b40 con 0x7f0c7c075b90 2024-02-02T15:46:08.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.917+0000 7f0c79600700 1 -- 172.21.15.1:0/1163160751 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0c64009ca0 con 0x7f0c7c075b90 2024-02-02T15:46:08.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.917+0000 7f0c79600700 1 -- 172.21.15.1:0/1163160751 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c6400f620 con 0x7f0c7c075b90 2024-02-02T15:46:08.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.917+0000 7f0c83600700 1 -- 172.21.15.1:0/1163160751 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c7c1a2a90 con 0x7f0c7c075b90 2024-02-02T15:46:08.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.917+0000 7f0c83600700 1 -- 172.21.15.1:0/1163160751 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c7c1aa2d0 con 0x7f0c7c075b90 2024-02-02T15:46:08.919 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:46:08 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/3289949364' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:46:08.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.922+0000 7f0c79600700 1 -- 172.21.15.1:0/1163160751 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f0c64010460 con 0x7f0c7c075b90 2024-02-02T15:46:08.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.924+0000 7f0c79600700 1 --2- 172.21.15.1:0/1163160751 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f0c5c0a0710 0x7f0c5c0a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:08.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.924+0000 7f0c79600700 1 -- 172.21.15.1:0/1163160751 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f0c640cc4e0 con 0x7f0c7c075b90 2024-02-02T15:46:08.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.925+0000 7f0c80c00700 1 --2- 172.21.15.1:0/1163160751 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f0c5c0a0710 0x7f0c5c0a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:08.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.925+0000 7f0c97103b80 1 Processor -- start 2024-02-02T15:46:08.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.925+0000 7f0c97103b80 1 -- start start 2024-02-02T15:46:08.925 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.926+0000 7f0c97103b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e4889b7010 0x55e4889b73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:08.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.926+0000 7f0c80c00700 1 --2- 172.21.15.1:0/1163160751 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f0c5c0a0710 0x7f0c5c0a2bd0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c006010 tx=0x7f0c6c00b260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:08.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.926+0000 7f0c97103b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e4889b7930 0x55e4889c22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:08.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.926+0000 7f0c97103b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e4889c27e0 0x55e4889c4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:08.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.926+0000 7f0c97103b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e48892b4c0 con 0x55e4889b7930 2024-02-02T15:46:08.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.926+0000 7f0c97103b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e48890faa0 con 0x55e4889b7010 2024-02-02T15:46:08.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.926+0000 7f0c97103b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e48890d460 con 0x55e4889c27e0 2024-02-02T15:46:08.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.926+0000 7f0c80c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e4889b7010 0x55e4889b73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:08.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.926+0000 7f0c80c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e4889b7010 0x55e4889b73f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:39608/0 (socket says 172.21.15.1:39608) 2024-02-02T15:46:08.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.926+0000 7f0c80c00700 1 -- 172.21.15.1:0/507161781 learned_addr learned my addr 172.21.15.1:0/507161781 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:46:08.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.927+0000 7f0c7be00700 1 --2- 172.21.15.1:0/507161781 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e4889b7930 0x55e4889c22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:08.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.927+0000 7f0c80c00700 1 -- 172.21.15.1:0/507161781 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e4889c27e0 msgr2=0x55e4889c4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:08.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.927+0000 7f0c80c00700 1 --2- 172.21.15.1:0/507161781 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e4889c27e0 0x55e4889c4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:08.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.927+0000 7f0c80c00700 1 -- 172.21.15.1:0/507161781 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e4889b7930 msgr2=0x55e4889c22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:08.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.927+0000 7f0c80c00700 1 --2- 172.21.15.1:0/507161781 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e4889b7930 0x55e4889c22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:08.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.927+0000 7f0c80c00700 1 -- 172.21.15.1:0/507161781 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e48878a3b0 con 0x55e4889b7010 2024-02-02T15:46:08.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.927+0000 7f0c7be00700 1 --2- 172.21.15.1:0/507161781 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e4889b7930 0x55e4889c22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:46:08.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.928+0000 7f0c80c00700 1 --2- 172.21.15.1:0/507161781 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e4889b7010 0x55e4889b73f0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c00be90 tx=0x7f0c6c01f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb8d9edeaf3fdda1 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:08.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.928+0000 7f0c71c00700 1 -- 172.21.15.1:0/507161781 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c6c02b070 con 0x55e4889b7010 2024-02-02T15:46:08.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.928+0000 7f0c71c00700 1 -- 172.21.15.1:0/507161781 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0c6c01e040 con 0x55e4889b7010 2024-02-02T15:46:08.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.928+0000 7f0c71c00700 1 -- 172.21.15.1:0/507161781 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c6c00f460 con 0x55e4889b7010 2024-02-02T15:46:08.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.929+0000 7f0c97103b80 1 -- 172.21.15.1:0/507161781 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e4889b7010 msgr2=0x55e4889b73f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:08.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.929+0000 7f0c97103b80 1 --2- 172.21.15.1:0/507161781 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e4889b7010 0x55e4889b73f0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c00be90 tx=0x7f0c6c01f000 comp rx=0 tx=0).stop 2024-02-02T15:46:08.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.929+0000 7f0c97103b80 1 -- 172.21.15.1:0/507161781 shutdown_connections 2024-02-02T15:46:08.929 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.929+0000 7f0c97103b80 1 --2- 172.21.15.1:0/507161781 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e4889c27e0 0x55e4889c4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:08.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.929+0000 7f0c97103b80 1 --2- 172.21.15.1:0/507161781 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e4889b7010 0x55e4889b73f0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:08.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.929+0000 7f0c97103b80 1 --2- 172.21.15.1:0/507161781 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e4889b7930 0x55e4889c22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:08.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.929+0000 7f0c97103b80 1 -- 172.21.15.1:0/507161781 >> 172.21.15.1:0/507161781 conn(0x55e488916db0 msgr2=0x55e4889b33b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:08.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.929+0000 7f0c97103b80 1 -- 172.21.15.1:0/507161781 shutdown_connections 2024-02-02T15:46:08.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.929+0000 7f0c97103b80 1 -- 172.21.15.1:0/507161781 wait complete. 2024-02-02T15:46:08.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.930+0000 7f0c97103b80 1 -- start start 2024-02-02T15:46:08.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.930+0000 7f0c97103b80 1 Processor -- start 2024-02-02T15:46:08.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.931+0000 7f0c97103b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e4889b7010 0x55e4889c2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:08.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.931+0000 7f0c97103b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e4889b7930 0x55e4889bf480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:08.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.931+0000 7f0c97103b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e4889c27e0 0x55e4889bfab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:08.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.931+0000 7f0c97103b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e488908d50 con 0x55e4889b7010 2024-02-02T15:46:08.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.931+0000 7f0c97103b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4888a4d80 con 0x55e4889b7930 2024-02-02T15:46:08.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.931+0000 7f0c97103b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4888a4a20 con 0x55e4889c27e0 2024-02-02T15:46:08.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.931+0000 7f0c97103b80 10 client.0 _ll_register_callbacks cb 0x55e4888a6290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:46:08.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.932+0000 7f0c81600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e4889c27e0 0x55e4889bfab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:08.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.932+0000 7f0c7be00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e4889b7930 0x55e4889bf480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:08.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.932+0000 7f0c81600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e4889c27e0 0x55e4889bfab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:44646/0 (socket says 172.21.15.1:44646) 2024-02-02T15:46:08.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.932+0000 7f0c81600700 1 -- 172.21.15.1:0/771026883 learned_addr learned my addr 172.21.15.1:0/771026883 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:46:08.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.933+0000 7f0c7be00700 1 -- 172.21.15.1:0/771026883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e4889c27e0 msgr2=0x55e4889bfab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:08.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.933+0000 7f0c7be00700 1 --2- 172.21.15.1:0/771026883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e4889c27e0 0x55e4889bfab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:08.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.933+0000 7f0c7be00700 1 -- 172.21.15.1:0/771026883 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e4889b7010 msgr2=0x55e4889c2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:08.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.933+0000 7f0c7be00700 1 --2- 172.21.15.1:0/771026883 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e4889b7010 0x55e4889c2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:08.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.933+0000 7f0c7be00700 1 -- 172.21.15.1:0/771026883 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c7c1aa2d0 con 0x55e4889b7930 2024-02-02T15:46:08.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.933+0000 7f0c7be00700 1 --2- 172.21.15.1:0/771026883 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e4889b7930 0x55e4889bf480 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f0c640c9830 tx=0x7f0c64099010 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97deaf2f5a06598e server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:08.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.934+0000 7f0c6a000700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:46:08.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.934+0000 7f0c6a000700 1 -- 172.21.15.1:0/771026883 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c6409d430 con 0x55e4889b7930 2024-02-02T15:46:08.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.935+0000 7f0c6a000700 1 -- 172.21.15.1:0/771026883 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f0c640a1670 con 0x55e4889b7930 2024-02-02T15:46:08.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.935+0000 7f0c97103b80 1 -- 172.21.15.1:0/771026883 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e4889ab0f0 con 0x55e4889b7930 2024-02-02T15:46:08.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.935+0000 7f0c6a000700 1 -- 172.21.15.1:0/771026883 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c64095d10 con 0x55e4889b7930 2024-02-02T15:46:08.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.936+0000 7f0c97103b80 10 client.30461 fetch_fsmap learned FSMap version 78 2024-02-02T15:46:08.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.936+0000 7f0c97103b80 1 -- 172.21.15.1:0/771026883 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) v3 -- 0x7f0c7c114b10 con 0x55e4889b7930 2024-02-02T15:46:08.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.936+0000 7f0c6a000700 1 -- 172.21.15.1:0/771026883 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e4889ab0f0 con 0x55e4889b7930 2024-02-02T15:46:08.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.936+0000 7f0c6a000700 1 -- 172.21.15.1:0/771026883 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 78) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f0c64010710 con 0x55e4889b7930 2024-02-02T15:46:08.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.937+0000 7f0c97103b80 10 client.30461 fetch_fsmap finished waiting for FSMap version 78 2024-02-02T15:46:08.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.937+0000 7f0c97103b80 10 client.30461 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-02-02T15:46:08.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.937+0000 7f0c97103b80 20 client.30461 populate_metadata read hostname 'smithi001' 2024-02-02T15:46:08.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.937+0000 7f0c97103b80 1 --2- 172.21.15.1:0/771026883 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55e4889d20f0 0x55e4889d4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:08.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.937+0000 7f0c97103b80 4 client.30461 mds_command: new command op to 24514 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:46:08.937 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.937+0000 7f0c97103b80 1 -- 172.21.15.1:0/771026883 --> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e488901dd0 con 0x55e4889d20f0 2024-02-02T15:46:08.938 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.938+0000 7f0c80c00700 1 --2- 172.21.15.1:0/771026883 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55e4889d20f0 0x55e4889d4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:08.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.940+0000 7f0c80c00700 1 --2- 172.21.15.1:0/771026883 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55e4889d20f0 0x55e4889d4510 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a537e5025c5aef3d server_cookie=24fdf4aba59d2551 in_seq=0 out_seq=0 2024-02-02T15:46:08.939 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.940+0000 7f0c6a000700 10 client.30461 ms_handle_connect on v2:172.21.15.142:6836/2870289019 2024-02-02T15:46:08.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:46:08 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/3289949364' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:46:08.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.943+0000 7f0c6a000700 1 -- 172.21.15.1:0/771026883 <== mds.3 v2:172.21.15.142:6836/2870289019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55e488901dd0 con 0x55e4889d20f0 2024-02-02T15:46:08.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.943+0000 7f0c6a000700 10 client.30461 handle_command_reply: tid=0 2024-02-02T15:46:08.943 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.944+0000 7f0c97103b80 1 client.30461 shutdown 2024-02-02T15:46:08.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.944+0000 7f0c97103b80 1 -- 172.21.15.1:0/771026883 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e4889b7930 msgr2=0x55e4889bf480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:08.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.944+0000 7f0c97103b80 1 --2- 172.21.15.1:0/771026883 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e4889b7930 0x55e4889bf480 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f0c640c9830 tx=0x7f0c64099010 comp rx=0 tx=0).stop 2024-02-02T15:46:08.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.944+0000 7f0c97103b80 1 -- 172.21.15.1:0/771026883 shutdown_connections 2024-02-02T15:46:08.944 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.944+0000 7f0c97103b80 1 --2- 172.21.15.1:0/771026883 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e4889c27e0 0x55e4889bfab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:08.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.944+0000 7f0c97103b80 1 --2- 172.21.15.1:0/771026883 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55e4889d20f0 0x55e4889d4510 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:08.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.944+0000 7f0c97103b80 1 --2- 172.21.15.1:0/771026883 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e4889b7010 0x55e4889c2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:08.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.944+0000 7f0c97103b80 1 --2- 172.21.15.1:0/771026883 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e4889b7930 0x55e4889bf480 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:08.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.944+0000 7f0c6a000700 0 client.30461 ms_handle_reset on v2:172.21.15.142:6836/2870289019 2024-02-02T15:46:08.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.944+0000 7f0c97103b80 1 -- 172.21.15.1:0/771026883 >> 172.21.15.1:0/771026883 conn(0x55e488916db0 msgr2=0x55e4889b2fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:08.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.945+0000 7f0c97103b80 1 -- 172.21.15.1:0/771026883 shutdown_connections 2024-02-02T15:46:08.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.945+0000 7f0c97103b80 1 -- 172.21.15.1:0/771026883 wait complete. 2024-02-02T15:46:08.945 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.945+0000 7f0c97103b80 20 client.30461 trim_cache size 0 max 16384 2024-02-02T15:46:08.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.969+0000 7f0c97103b80 1 Processor -- start 2024-02-02T15:46:08.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.969+0000 7f0c97103b80 1 -- start start 2024-02-02T15:46:08.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.969+0000 7f0c97103b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e4889b7010 0x55e4889bf440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:08.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.970+0000 7f0c97103b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e4889b7930 0x55e4889bed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:08.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.970+0000 7f0c97103b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e4889c27e0 0x55e4889c1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:08.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.970+0000 7f0c97103b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e48878c6a0 con 0x55e4889b7010 2024-02-02T15:46:08.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.970+0000 7f0c97103b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4889c0090 con 0x55e4889b7930 2024-02-02T15:46:08.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.970+0000 7f0c97103b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4889b8830 con 0x55e4889c27e0 2024-02-02T15:46:08.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.970+0000 7f0c7be00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e4889b7930 0x55e4889bed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:08.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.970+0000 7f0c7be00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e4889b7930 0x55e4889bed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:39620/0 (socket says 172.21.15.1:39620) 2024-02-02T15:46:08.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.970+0000 7f0c7be00700 1 -- 172.21.15.1:0/1801817265 learned_addr learned my addr 172.21.15.1:0/1801817265 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:46:08.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.970+0000 7f0c80c00700 1 --2- 172.21.15.1:0/1801817265 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e4889b7010 0x55e4889bf440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:08.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.970+0000 7f0c81600700 1 --2- 172.21.15.1:0/1801817265 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e4889c27e0 0x55e4889c1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:08.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.971+0000 7f0c7be00700 1 -- 172.21.15.1:0/1801817265 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e4889c27e0 msgr2=0x55e4889c1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:08.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.971+0000 7f0c7be00700 1 --2- 172.21.15.1:0/1801817265 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e4889c27e0 0x55e4889c1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:08.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.971+0000 7f0c7be00700 1 -- 172.21.15.1:0/1801817265 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e4889b7010 msgr2=0x55e4889bf440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:08.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.971+0000 7f0c7be00700 1 --2- 172.21.15.1:0/1801817265 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e4889b7010 0x55e4889bf440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:08.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.971+0000 7f0c7be00700 1 -- 172.21.15.1:0/1801817265 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e488736b60 con 0x55e4889b7930 2024-02-02T15:46:08.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.971+0000 7f0c80c00700 1 --2- 172.21.15.1:0/1801817265 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e4889b7010 0x55e4889bf440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:46:08.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.971+0000 7f0c7be00700 1 --2- 172.21.15.1:0/1801817265 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e4889b7930 0x55e4889bed20 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f0c64096070 tx=0x7f0c64099780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44d2fd8f5cbec587 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:08.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.972+0000 7f0c72600700 1 -- 172.21.15.1:0/1801817265 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c64084220 con 0x55e4889b7930 2024-02-02T15:46:08.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.972+0000 7f0c72600700 1 -- 172.21.15.1:0/1801817265 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0c64095940 con 0x55e4889b7930 2024-02-02T15:46:08.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.972+0000 7f0c72600700 1 -- 172.21.15.1:0/1801817265 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c64010710 con 0x55e4889b7930 2024-02-02T15:46:08.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.973+0000 7f0c97103b80 1 -- 172.21.15.1:0/1801817265 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e4889b7930 msgr2=0x55e4889bed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:08.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.973+0000 7f0c97103b80 1 --2- 172.21.15.1:0/1801817265 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e4889b7930 0x55e4889bed20 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f0c64096070 tx=0x7f0c64099780 comp rx=0 tx=0).stop 2024-02-02T15:46:08.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.973+0000 7f0c97103b80 1 -- 172.21.15.1:0/1801817265 shutdown_connections 2024-02-02T15:46:08.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.973+0000 7f0c97103b80 1 --2- 172.21.15.1:0/1801817265 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e4889c27e0 0x55e4889c1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:08.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.973+0000 7f0c97103b80 1 --2- 172.21.15.1:0/1801817265 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e4889b7010 0x55e4889bf440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:08.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.973+0000 7f0c97103b80 1 --2- 172.21.15.1:0/1801817265 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e4889b7930 0x55e4889bed20 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:08.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.973+0000 7f0c97103b80 1 -- 172.21.15.1:0/1801817265 >> 172.21.15.1:0/1801817265 conn(0x55e488916db0 msgr2=0x55e4889b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:08.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.973+0000 7f0c97103b80 1 -- 172.21.15.1:0/1801817265 shutdown_connections 2024-02-02T15:46:08.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.973+0000 7f0c97103b80 1 -- 172.21.15.1:0/1801817265 wait complete. 2024-02-02T15:46:08.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.974+0000 7f0c97103b80 1 -- start start 2024-02-02T15:46:08.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.974+0000 7f0c97103b80 1 Processor -- start 2024-02-02T15:46:08.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.975+0000 7f0c97103b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e4889b7010 0x55e4889c8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:08.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.975+0000 7f0c97103b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e4889b7930 0x55e4889c8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:08.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.975+0000 7f0c97103b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e4889c27e0 0x55e488a22bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:08.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.975+0000 7f0c97103b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e488a23110 con 0x55e4889c27e0 2024-02-02T15:46:08.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.975+0000 7f0c97103b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e488a23250 con 0x55e4889b7930 2024-02-02T15:46:08.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.975+0000 7f0c97103b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e488a23390 con 0x55e4889b7010 2024-02-02T15:46:08.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.975+0000 7f0c97103b80 10 client.0 _ll_register_callbacks cb 0x55e4888a6290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:46:08.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.975+0000 7f0c7be00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e4889b7930 0x55e4889c8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:08.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.976+0000 7f0c7be00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e4889b7930 0x55e4889c8bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:39628/0 (socket says 172.21.15.1:39628) 2024-02-02T15:46:08.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.976+0000 7f0c7be00700 1 -- 172.21.15.1:0/3223040839 learned_addr learned my addr 172.21.15.1:0/3223040839 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:46:08.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.976+0000 7f0c81600700 1 --2- 172.21.15.1:0/3223040839 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e4889c27e0 0x55e488a22bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:08.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.976+0000 7f0c7be00700 1 -- 172.21.15.1:0/3223040839 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e4889b7010 msgr2=0x55e4889c8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:08.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.976+0000 7f0c7be00700 1 --2- 172.21.15.1:0/3223040839 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e4889b7010 0x55e4889c8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:08.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.976+0000 7f0c7be00700 1 -- 172.21.15.1:0/3223040839 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e4889c27e0 msgr2=0x55e488a22bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:08.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.976+0000 7f0c7be00700 1 --2- 172.21.15.1:0/3223040839 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e4889c27e0 0x55e488a22bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:08.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.976+0000 7f0c7be00700 1 -- 172.21.15.1:0/3223040839 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e48878a3b0 con 0x55e4889b7930 2024-02-02T15:46:08.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.976+0000 7f0c81600700 1 --2- 172.21.15.1:0/3223040839 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e4889c27e0 0x55e488a22bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:46:08.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.977+0000 7f0c7be00700 1 --2- 172.21.15.1:0/3223040839 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e4889b7930 0x55e4889c8bc0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f0c64099300 tx=0x7f0c64095f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57a4e0fc34cb9efd server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:08.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.977+0000 7f0c6a000700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:46:08.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.977+0000 7f0c6a000700 1 -- 172.21.15.1:0/3223040839 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c640a1670 con 0x55e4889b7930 2024-02-02T15:46:08.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.977+0000 7f0c6a000700 1 -- 172.21.15.1:0/3223040839 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f0c6409d430 con 0x55e4889b7930 2024-02-02T15:46:08.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.978+0000 7f0c97103b80 1 -- 172.21.15.1:0/3223040839 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0c6c01e040 con 0x55e4889b7930 2024-02-02T15:46:08.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.980+0000 7f0c6a000700 1 -- 172.21.15.1:0/3223040839 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c640cc170 con 0x55e4889b7930 2024-02-02T15:46:08.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.980+0000 7f0c97103b80 10 client.30467 fetch_fsmap learned FSMap version 78 2024-02-02T15:46:08.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.980+0000 7f0c97103b80 1 -- 172.21.15.1:0/3223040839 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) v3 -- 0x7f0c7c114b10 con 0x55e4889b7930 2024-02-02T15:46:08.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.981+0000 7f0c6a000700 1 -- 172.21.15.1:0/3223040839 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0c6c01e040 con 0x55e4889b7930 2024-02-02T15:46:08.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.981+0000 7f0c6a000700 1 -- 172.21.15.1:0/3223040839 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 78) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f0c640a11e0 con 0x55e4889b7930 2024-02-02T15:46:08.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.981+0000 7f0c97103b80 10 client.30467 fetch_fsmap finished waiting for FSMap version 78 2024-02-02T15:46:08.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.981+0000 7f0c97103b80 10 client.30467 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-02-02T15:46:08.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.981+0000 7f0c97103b80 20 client.30467 populate_metadata read hostname 'smithi001' 2024-02-02T15:46:08.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.982+0000 7f0c97103b80 1 --2- 172.21.15.1:0/3223040839 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55e488a35600 0x55e488a37a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:08.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.982+0000 7f0c97103b80 4 client.30467 mds_command: new command op to 24514 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:46:08.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.982+0000 7f0c97103b80 1 -- 172.21.15.1:0/3223040839 --> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e4888c7f50 con 0x55e488a35600 2024-02-02T15:46:08.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.985+0000 7f0c80c00700 1 --2- 172.21.15.1:0/3223040839 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55e488a35600 0x55e488a37a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:08.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.986+0000 7f0c80c00700 1 --2- 172.21.15.1:0/3223040839 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55e488a35600 0x55e488a37a20 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=97031bc8be20498c server_cookie=eeeb58ba7e72dc9e in_seq=0 out_seq=0 2024-02-02T15:46:08.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.986+0000 7f0c6a000700 10 client.30467 ms_handle_connect on v2:172.21.15.142:6836/2870289019 2024-02-02T15:46:08.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.986+0000 7f0c6a000700 1 -- 172.21.15.1:0/3223040839 <== mds.3 v2:172.21.15.142:6836/2870289019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e4888c7f50 con 0x55e488a35600 2024-02-02T15:46:08.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.986+0000 7f0c6a000700 10 client.30467 handle_command_reply: tid=0 2024-02-02T15:46:08.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.986+0000 7f0c97103b80 1 client.30467 shutdown 2024-02-02T15:46:08.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.987+0000 7f0c97103b80 1 -- 172.21.15.1:0/3223040839 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e4889b7930 msgr2=0x55e4889c8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:08.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.987+0000 7f0c97103b80 1 --2- 172.21.15.1:0/3223040839 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e4889b7930 0x55e4889c8bc0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f0c64099300 tx=0x7f0c64095f40 comp rx=0 tx=0).stop 2024-02-02T15:46:08.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.987+0000 7f0c97103b80 1 -- 172.21.15.1:0/3223040839 shutdown_connections 2024-02-02T15:46:08.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.987+0000 7f0c97103b80 1 --2- 172.21.15.1:0/3223040839 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55e488a35600 0x55e488a37a20 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:08.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.988+0000 7f0c6a000700 0 client.30467 ms_handle_reset on v2:172.21.15.142:6836/2870289019 2024-02-02T15:46:08.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.988+0000 7f0c97103b80 1 --2- 172.21.15.1:0/3223040839 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e4889c27e0 0x55e488a22bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:08.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.988+0000 7f0c97103b80 1 --2- 172.21.15.1:0/3223040839 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e4889b7010 0x55e4889c8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:08.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.988+0000 7f0c97103b80 1 --2- 172.21.15.1:0/3223040839 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e4889b7930 0x55e4889c8bc0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:08.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.988+0000 7f0c97103b80 1 -- 172.21.15.1:0/3223040839 >> 172.21.15.1:0/3223040839 conn(0x55e488916db0 msgr2=0x55e4889b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:08.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.988+0000 7f0c97103b80 1 -- 172.21.15.1:0/3223040839 shutdown_connections 2024-02-02T15:46:08.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.988+0000 7f0c97103b80 1 -- 172.21.15.1:0/3223040839 wait complete. 2024-02-02T15:46:08.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.990+0000 7f0c97103b80 20 client.30467 trim_cache size 0 max 16384 2024-02-02T15:46:08.990 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:46:08.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.990+0000 7f0c83600700 1 -- 172.21.15.1:0/1163160751 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f0c5c0a0710 msgr2=0x7f0c5c0a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:08.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.990+0000 7f0c83600700 1 --2- 172.21.15.1:0/1163160751 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f0c5c0a0710 0x7f0c5c0a2bd0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c006010 tx=0x7f0c6c00b260 comp rx=0 tx=0).stop 2024-02-02T15:46:08.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.990+0000 7f0c83600700 1 -- 172.21.15.1:0/1163160751 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c7c075b90 msgr2=0x7f0c7c19d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:08.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.990+0000 7f0c83600700 1 --2- 172.21.15.1:0/1163160751 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c7c075b90 0x7f0c7c19d9e0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f0c6400d950 tx=0x7f0c6400dd10 comp rx=0 tx=0).stop 2024-02-02T15:46:08.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.991+0000 7f0c83600700 1 -- 172.21.15.1:0/1163160751 shutdown_connections 2024-02-02T15:46:08.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.991+0000 7f0c83600700 1 --2- 172.21.15.1:0/1163160751 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f0c5c0a0710 0x7f0c5c0a2bd0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:08.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.991+0000 7f0c83600700 1 --2- 172.21.15.1:0/1163160751 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0c7c111c30 0x7f0c7c1a1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:08.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.991+0000 7f0c83600700 1 --2- 172.21.15.1:0/1163160751 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0c7c075270 0x7f0c7c19d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:08.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.991+0000 7f0c83600700 1 --2- 172.21.15.1:0/1163160751 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c7c075b90 0x7f0c7c19d9e0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:08.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.991+0000 7f0c83600700 1 -- 172.21.15.1:0/1163160751 >> 172.21.15.1:0/1163160751 conn(0x7f0c7c0feaa0 msgr2=0x7f0c7c100e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:08.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.991+0000 7f0c83600700 1 -- 172.21.15.1:0/1163160751 shutdown_connections 2024-02-02T15:46:08.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:08.991+0000 7f0c83600700 1 -- 172.21.15.1:0/1163160751 wait complete. 2024-02-02T15:46:09.010 INFO:tasks.workunit.client.0.smithi001.stdout: 10 177839 26.36 MB/sec execute 221 sec latency 374.254 ms 2024-02-02T15:46:09.020 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-02T15:46:09.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.329+0000 7f5bfc800700 1 -- 172.21.15.1:0/2962264214 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5bf41083b0 msgr2=0x7f5bf410ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:09.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.329+0000 7f5bfc800700 1 --2- 172.21.15.1:0/2962264214 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5bf41083b0 0x7f5bf410ff30 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f5bec00b0a0 tx=0x7f5bec00b3b0 comp rx=0 tx=0).stop 2024-02-02T15:46:09.330 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.330+0000 7f5bfc800700 1 -- 172.21.15.1:0/2962264214 shutdown_connections 2024-02-02T15:46:09.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.330+0000 7f5bfc800700 1 --2- 172.21.15.1:0/2962264214 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5bf41083b0 0x7f5bf410ff30 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:09.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.330+0000 7f5bfc800700 1 --2- 172.21.15.1:0/2962264214 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5bf4103290 0x7f5bf4103670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:09.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.330+0000 7f5bfc800700 1 --2- 172.21.15.1:0/2962264214 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5bf4103c40 0x7f5bf4107c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:09.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.330+0000 7f5bfc800700 1 -- 172.21.15.1:0/2962264214 >> 172.21.15.1:0/2962264214 conn(0x7f5bf40feb30 msgr2=0x7f5bf4100f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:09.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.330+0000 7f5bfc800700 1 -- 172.21.15.1:0/2962264214 shutdown_connections 2024-02-02T15:46:09.331 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.331+0000 7f5bfc800700 1 -- 172.21.15.1:0/2962264214 wait complete. 2024-02-02T15:46:09.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.332+0000 7f5bfc800700 1 Processor -- start 2024-02-02T15:46:09.332 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.333+0000 7f5bfc800700 1 -- start start 2024-02-02T15:46:09.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.333+0000 7f5bfc800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5bf4103290 0x7f5bf4199150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:09.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.333+0000 7f5bfc800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5bf41083b0 0x7f5bf4199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:09.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.333+0000 7f5bfc800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5bf419daf0 0x7f5bf419df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:09.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.333+0000 7f5bfc800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf419e570 con 0x7f5bf4103290 2024-02-02T15:46:09.333 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.333+0000 7f5bfc800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf419e6e0 con 0x7f5bf41083b0 2024-02-02T15:46:09.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.333+0000 7f5bfc800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf419e8a0 con 0x7f5bf419daf0 2024-02-02T15:46:09.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.334+0000 7f5bfa800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5bf419daf0 0x7f5bf419df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:09.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.334+0000 7f5bf9400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5bf41083b0 0x7f5bf4199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:09.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.334+0000 7f5bfa800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5bf419daf0 0x7f5bf419df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:44668/0 (socket says 172.21.15.1:44668) 2024-02-02T15:46:09.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.334+0000 7f5bfa800700 1 -- 172.21.15.1:0/2532761847 learned_addr learned my addr 172.21.15.1:0/2532761847 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:46:09.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.334+0000 7f5bf9400700 1 -- 172.21.15.1:0/2532761847 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5bf419daf0 msgr2=0x7f5bf419df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:09.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.334+0000 7f5bf9400700 1 --2- 172.21.15.1:0/2532761847 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5bf419daf0 0x7f5bf419df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:09.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.334+0000 7f5bf9400700 1 -- 172.21.15.1:0/2532761847 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5bf4103290 msgr2=0x7f5bf4199150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:09.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.334+0000 7f5bf9400700 1 --2- 172.21.15.1:0/2532761847 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5bf4103290 0x7f5bf4199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:09.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.334+0000 7f5bf9400700 1 -- 172.21.15.1:0/2532761847 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bec009d00 con 0x7f5bf41083b0 2024-02-02T15:46:09.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.335+0000 7f5bf9400700 1 --2- 172.21.15.1:0/2532761847 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5bf41083b0 0x7f5bf4199690 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f5bf406a570 tx=0x7f5be400b750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:09.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.335+0000 7f5bf2a00700 1 -- 172.21.15.1:0/2532761847 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be4011830 con 0x7f5bf41083b0 2024-02-02T15:46:09.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.335+0000 7f5bfc800700 1 -- 172.21.15.1:0/2532761847 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bf41a5e00 con 0x7f5bf41083b0 2024-02-02T15:46:09.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.335+0000 7f5bf2a00700 1 -- 172.21.15.1:0/2532761847 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5be4011e50 con 0x7f5bf41083b0 2024-02-02T15:46:09.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.336+0000 7f5bfc800700 1 -- 172.21.15.1:0/2532761847 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bf41a6280 con 0x7f5bf41083b0 2024-02-02T15:46:09.336 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.336+0000 7f5bf2a00700 1 -- 172.21.15.1:0/2532761847 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be4010de0 con 0x7f5bf41083b0 2024-02-02T15:46:09.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.341+0000 7f5bf2a00700 1 -- 172.21.15.1:0/2532761847 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f5be4011990 con 0x7f5bf41083b0 2024-02-02T15:46:09.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.342+0000 7f5bf2a00700 1 --2- 172.21.15.1:0/2532761847 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f5bd40a07d0 0x7f5bd40a2c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:09.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.342+0000 7f5bf2a00700 1 -- 172.21.15.1:0/2532761847 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f5be40cb900 con 0x7f5bf41083b0 2024-02-02T15:46:09.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.343+0000 7f5bf9e00700 1 --2- 172.21.15.1:0/2532761847 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f5bd40a07d0 0x7f5bd40a2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:09.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.343+0000 7f5c10345b80 1 Processor -- start 2024-02-02T15:46:09.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.343+0000 7f5c10345b80 1 -- start start 2024-02-02T15:46:09.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.344+0000 7f5bf9e00700 1 --2- 172.21.15.1:0/2532761847 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f5bd40a07d0 0x7f5bd40a2c90 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc005950 tx=0x7f5bdc0058e0 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:09.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.344+0000 7f5c10345b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56339e139010 0x56339e1393f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:09.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.344+0000 7f5c10345b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56339e139930 0x56339e14cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:09.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.344+0000 7f5c10345b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56339e14d010 0x56339e14f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:09.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.345+0000 7f5c10345b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56339e0ad4c0 con 0x56339e139010 2024-02-02T15:46:09.344 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.345+0000 7f5c10345b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56339e091aa0 con 0x56339e14d010 2024-02-02T15:46:09.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.345+0000 7f5c10345b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56339e08f460 con 0x56339e139930 2024-02-02T15:46:09.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.345+0000 7f5bfa800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56339e14d010 0x56339e14f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:09.345 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.345+0000 7f5bf9400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56339e139930 0x56339e14cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:09.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.345+0000 7f5bfa800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56339e14d010 0x56339e14f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:39662/0 (socket says 172.21.15.1:39662) 2024-02-02T15:46:09.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.345+0000 7f5bfa800700 1 -- 172.21.15.1:0/648186866 learned_addr learned my addr 172.21.15.1:0/648186866 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:46:09.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.345+0000 7f5bf9400700 1 -- 172.21.15.1:0/648186866 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56339e14d010 msgr2=0x56339e14f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:09.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.345+0000 7f5bf9400700 1 --2- 172.21.15.1:0/648186866 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56339e14d010 0x56339e14f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:09.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.345+0000 7f5bf9400700 1 -- 172.21.15.1:0/648186866 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56339e139010 msgr2=0x56339e1393f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:09.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.345+0000 7f5bf9400700 1 --2- 172.21.15.1:0/648186866 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56339e139010 0x56339e1393f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:09.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.346+0000 7f5bf9400700 1 -- 172.21.15.1:0/648186866 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56339df0c3b0 con 0x56339e139930 2024-02-02T15:46:09.346 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.346+0000 7f5bf9400700 1 --2- 172.21.15.1:0/648186866 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56339e139930 0x56339e14cad0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f5be40c8d00 tx=0x7f5be4098430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9226c6b396e5abd8 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:09.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.346+0000 7f5beb000700 1 -- 172.21.15.1:0/648186866 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be409c900 con 0x56339e139930 2024-02-02T15:46:09.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.346+0000 7f5beb000700 1 -- 172.21.15.1:0/648186866 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5be40a0b40 con 0x56339e139930 2024-02-02T15:46:09.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.346+0000 7f5beb000700 1 -- 172.21.15.1:0/648186866 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be40cb6e0 con 0x56339e139930 2024-02-02T15:46:09.347 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.347+0000 7f5c10345b80 1 -- 172.21.15.1:0/648186866 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56339e139930 msgr2=0x56339e14cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:09.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.347+0000 7f5c10345b80 1 --2- 172.21.15.1:0/648186866 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56339e139930 0x56339e14cad0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f5be40c8d00 tx=0x7f5be4098430 comp rx=0 tx=0).stop 2024-02-02T15:46:09.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.347+0000 7f5c10345b80 1 -- 172.21.15.1:0/648186866 shutdown_connections 2024-02-02T15:46:09.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.347+0000 7f5c10345b80 1 --2- 172.21.15.1:0/648186866 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56339e14d010 0x56339e14f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:09.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.347+0000 7f5c10345b80 1 --2- 172.21.15.1:0/648186866 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56339e139010 0x56339e1393f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:09.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.347+0000 7f5c10345b80 1 --2- 172.21.15.1:0/648186866 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56339e139930 0x56339e14cad0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:09.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.348+0000 7f5c10345b80 1 -- 172.21.15.1:0/648186866 >> 172.21.15.1:0/648186866 conn(0x56339e098db0 msgr2=0x56339e1352f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:09.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.348+0000 7f5c10345b80 1 -- 172.21.15.1:0/648186866 shutdown_connections 2024-02-02T15:46:09.348 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.348+0000 7f5c10345b80 1 -- 172.21.15.1:0/648186866 wait complete. 2024-02-02T15:46:09.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.348+0000 7f5c10345b80 1 -- start start 2024-02-02T15:46:09.349 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.349+0000 7f5c10345b80 1 Processor -- start 2024-02-02T15:46:09.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.350+0000 7f5c10345b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56339e139010 0x56339e14ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:09.350 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.350+0000 7f5c10345b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56339e139930 0x56339e149c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:09.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.350+0000 7f5c10345b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56339e14d010 0x56339e14a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:09.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.350+0000 7f5c10345b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56339e08ad50 con 0x56339e14d010 2024-02-02T15:46:09.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.350+0000 7f5c10345b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56339e026d80 con 0x56339e139010 2024-02-02T15:46:09.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.350+0000 7f5c10345b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56339e026a20 con 0x56339e139930 2024-02-02T15:46:09.351 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.350+0000 7f5c10345b80 10 client.0 _ll_register_callbacks cb 0x56339e028290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:46:09.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.351+0000 7f5bf9e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56339e139010 0x56339e14ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:09.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.351+0000 7f5bf9e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56339e139010 0x56339e14ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:39664/0 (socket says 172.21.15.1:39664) 2024-02-02T15:46:09.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.351+0000 7f5bf9e00700 1 -- 172.21.15.1:0/1988722190 learned_addr learned my addr 172.21.15.1:0/1988722190 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:46:09.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.351+0000 7f5bf9400700 1 --2- 172.21.15.1:0/1988722190 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56339e139930 0x56339e149c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:09.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.351+0000 7f5bfa800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56339e14d010 0x56339e14a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:09.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.352+0000 7f5bf9400700 1 -- 172.21.15.1:0/1988722190 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56339e139010 msgr2=0x56339e14ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:09.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.352+0000 7f5bf9400700 1 --2- 172.21.15.1:0/1988722190 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56339e139010 0x56339e14ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:09.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.352+0000 7f5bf9400700 1 -- 172.21.15.1:0/1988722190 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56339e14d010 msgr2=0x56339e14a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:09.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.352+0000 7f5bf9400700 1 --2- 172.21.15.1:0/1988722190 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56339e14d010 0x56339e14a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:09.352 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.352+0000 7f5bf9400700 1 -- 172.21.15.1:0/1988722190 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bf41a6280 con 0x56339e139930 2024-02-02T15:46:09.353 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.352+0000 7f5bfa800700 1 --2- 172.21.15.1:0/1988722190 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56339e14d010 0x56339e14a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:46:09.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.352+0000 7f5bf9400700 1 --2- 172.21.15.1:0/1988722190 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56339e139930 0x56339e149c60 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f5be40b82e0 tx=0x7f5be4010f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e602adc3dbbfc85 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:09.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.352+0000 7f5be3400700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:46:09.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.353+0000 7f5be3400700 1 -- 172.21.15.1:0/1988722190 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be4018510 con 0x56339e139930 2024-02-02T15:46:09.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.353+0000 7f5be3400700 1 -- 172.21.15.1:0/1988722190 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f5be40cb480 con 0x56339e139930 2024-02-02T15:46:09.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.353+0000 7f5c10345b80 1 -- 172.21.15.1:0/1988722190 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56339e12d0f0 con 0x56339e139930 2024-02-02T15:46:09.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.353+0000 7f5be3400700 1 -- 172.21.15.1:0/1988722190 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5be4096cd0 con 0x56339e139930 2024-02-02T15:46:09.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.353+0000 7f5c10345b80 10 client.30322 fetch_fsmap learned FSMap version 78 2024-02-02T15:46:09.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.353+0000 7f5c10345b80 1 -- 172.21.15.1:0/1988722190 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) v3 -- 0x56339deb8b60 con 0x56339e139930 2024-02-02T15:46:09.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.354+0000 7f5be3400700 1 -- 172.21.15.1:0/1988722190 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56339e12d0f0 con 0x56339e139930 2024-02-02T15:46:09.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.354+0000 7f5be3400700 1 -- 172.21.15.1:0/1988722190 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 78) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f5be4098030 con 0x56339e139930 2024-02-02T15:46:09.354 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.354+0000 7f5c10345b80 10 client.30322 fetch_fsmap finished waiting for FSMap version 78 2024-02-02T15:46:09.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.354+0000 7f5c10345b80 10 client.30322 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-02-02T15:46:09.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.354+0000 7f5c10345b80 20 client.30322 populate_metadata read hostname 'smithi001' 2024-02-02T15:46:09.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.354+0000 7f5c10345b80 1 --2- 172.21.15.1:0/1988722190 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x56339e15c650 0x56339e15ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:09.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.354+0000 7f5c10345b80 4 client.30322 mds_command: new command op to 24518 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:46:09.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.354+0000 7f5c10345b80 1 -- 172.21.15.1:0/1988722190 --> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56339e083dd0 con 0x56339e15c650 2024-02-02T15:46:09.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.354+0000 7f5bf9e00700 1 --2- 172.21.15.1:0/1988722190 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x56339e15c650 0x56339e15ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:09.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.355+0000 7f5be3400700 10 client.30322 ms_handle_connect on v2:172.21.15.71:6836/628016199 2024-02-02T15:46:09.355 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.355+0000 7f5bf9e00700 1 --2- 172.21.15.1:0/1988722190 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x56339e15c650 0x56339e15ea70 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e4fe6bc6e2b577e7 server_cookie=300f66dcc617877b in_seq=0 out_seq=0 2024-02-02T15:46:09.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.356+0000 7f5be3400700 1 -- 172.21.15.1:0/1988722190 <== mds.2 v2:172.21.15.71:6836/628016199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x56339e083dd0 con 0x56339e15c650 2024-02-02T15:46:09.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.356+0000 7f5be3400700 10 client.30322 handle_command_reply: tid=0 2024-02-02T15:46:09.356 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.357+0000 7f5c10345b80 1 client.30322 shutdown 2024-02-02T15:46:09.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.357+0000 7f5c10345b80 1 -- 172.21.15.1:0/1988722190 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56339e139930 msgr2=0x56339e149c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:09.357 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.357+0000 7f5c10345b80 1 --2- 172.21.15.1:0/1988722190 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56339e139930 0x56339e149c60 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f5be40b82e0 tx=0x7f5be4010f60 comp rx=0 tx=0).stop 2024-02-02T15:46:09.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.358+0000 7f5c10345b80 1 -- 172.21.15.1:0/1988722190 shutdown_connections 2024-02-02T15:46:09.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.358+0000 7f5c10345b80 1 --2- 172.21.15.1:0/1988722190 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56339e14d010 0x56339e14a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:09.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.358+0000 7f5c10345b80 1 --2- 172.21.15.1:0/1988722190 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56339e139010 0x56339e14ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:09.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.358+0000 7f5c10345b80 1 --2- 172.21.15.1:0/1988722190 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x56339e15c650 0x56339e15ea70 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:09.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.358+0000 7f5c10345b80 1 --2- 172.21.15.1:0/1988722190 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56339e139930 0x56339e149c60 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:09.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.358+0000 7f5be3400700 0 client.30322 ms_handle_reset on v2:172.21.15.71:6836/628016199 2024-02-02T15:46:09.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.358+0000 7f5c10345b80 1 -- 172.21.15.1:0/1988722190 >> 172.21.15.1:0/1988722190 conn(0x56339e098db0 msgr2=0x56339e134eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:09.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.358+0000 7f5c10345b80 1 -- 172.21.15.1:0/1988722190 shutdown_connections 2024-02-02T15:46:09.358 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.358+0000 7f5c10345b80 1 -- 172.21.15.1:0/1988722190 wait complete. 2024-02-02T15:46:09.359 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.358+0000 7f5c10345b80 20 client.30322 trim_cache size 0 max 16384 2024-02-02T15:46:09.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.380+0000 7f5c10345b80 1 Processor -- start 2024-02-02T15:46:09.380 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.381+0000 7f5c10345b80 1 -- start start 2024-02-02T15:46:09.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.381+0000 7f5c10345b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56339e139010 0x56339e153ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:09.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.381+0000 7f5c10345b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56339e139930 0x56339e148ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:09.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.381+0000 7f5c10345b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56339e14d010 0x56339e14c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:09.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.381+0000 7f5c10345b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56339df0e6a0 con 0x56339e14d010 2024-02-02T15:46:09.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.381+0000 7f5c10345b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56339e13c0b0 con 0x56339e139930 2024-02-02T15:46:09.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.381+0000 7f5c10345b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56339e139d50 con 0x56339e139010 2024-02-02T15:46:09.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.381+0000 7f5bf9400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56339e139930 0x56339e148ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:09.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.382+0000 7f5bf9400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56339e139930 0x56339e148ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:39680/0 (socket says 172.21.15.1:39680) 2024-02-02T15:46:09.382 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.382+0000 7f5bf9400700 1 -- 172.21.15.1:0/1712150612 learned_addr learned my addr 172.21.15.1:0/1712150612 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:46:09.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.382+0000 7f5bf9e00700 1 --2- 172.21.15.1:0/1712150612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56339e139010 0x56339e153ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:09.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.382+0000 7f5bfa800700 1 --2- 172.21.15.1:0/1712150612 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56339e14d010 0x56339e14c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:09.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.382+0000 7f5bf9400700 1 -- 172.21.15.1:0/1712150612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56339e139010 msgr2=0x56339e153ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:09.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.382+0000 7f5bf9400700 1 --2- 172.21.15.1:0/1712150612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56339e139010 0x56339e153ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:09.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.382+0000 7f5bf9400700 1 -- 172.21.15.1:0/1712150612 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56339e14d010 msgr2=0x56339e14c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:09.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.382+0000 7f5bf9400700 1 --2- 172.21.15.1:0/1712150612 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56339e14d010 0x56339e14c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:09.383 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.382+0000 7f5bf9400700 1 -- 172.21.15.1:0/1712150612 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56339e050ea0 con 0x56339e139930 2024-02-02T15:46:09.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.382+0000 7f5bf9400700 1 --2- 172.21.15.1:0/1712150612 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56339e139930 0x56339e148ef0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f5be4094fd0 tx=0x7f5be40b11a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30680afc3bb2d5cb server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:09.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.382+0000 7f5beba00700 1 -- 172.21.15.1:0/1712150612 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be40a08f0 con 0x56339e139930 2024-02-02T15:46:09.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.383+0000 7f5beba00700 1 -- 172.21.15.1:0/1712150612 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5be409c900 con 0x56339e139930 2024-02-02T15:46:09.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.383+0000 7f5beba00700 1 -- 172.21.15.1:0/1712150612 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be4018310 con 0x56339e139930 2024-02-02T15:46:09.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.383+0000 7f5c10345b80 1 -- 172.21.15.1:0/1712150612 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56339e139930 msgr2=0x56339e148ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:09.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.383+0000 7f5c10345b80 1 --2- 172.21.15.1:0/1712150612 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56339e139930 0x56339e148ef0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f5be4094fd0 tx=0x7f5be40b11a0 comp rx=0 tx=0).stop 2024-02-02T15:46:09.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.384+0000 7f5c10345b80 1 -- 172.21.15.1:0/1712150612 shutdown_connections 2024-02-02T15:46:09.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.384+0000 7f5c10345b80 1 --2- 172.21.15.1:0/1712150612 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56339e14d010 0x56339e14c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:09.385 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.384+0000 7f5c10345b80 1 --2- 172.21.15.1:0/1712150612 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56339e139010 0x56339e153ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:09.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.384+0000 7f5c10345b80 1 --2- 172.21.15.1:0/1712150612 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56339e139930 0x56339e148ef0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:09.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.384+0000 7f5c10345b80 1 -- 172.21.15.1:0/1712150612 >> 172.21.15.1:0/1712150612 conn(0x56339e098db0 msgr2=0x56339e1345c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:09.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.384+0000 7f5c10345b80 1 -- 172.21.15.1:0/1712150612 shutdown_connections 2024-02-02T15:46:09.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.384+0000 7f5c10345b80 1 -- 172.21.15.1:0/1712150612 wait complete. 2024-02-02T15:46:09.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.384+0000 7f5c10345b80 1 -- start start 2024-02-02T15:46:09.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.384+0000 7f5c10345b80 1 Processor -- start 2024-02-02T15:46:09.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.385+0000 7f5c10345b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56339e139010 0x56339e151d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:09.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.385+0000 7f5c10345b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56339e139930 0x56339e1522b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:09.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.385+0000 7f5c10345b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56339e14d010 0x56339e1a7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:09.386 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.385+0000 7f5c10345b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x56339e1a7640 con 0x56339e14d010 2024-02-02T15:46:09.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.385+0000 7f5c10345b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x56339e1a77b0 con 0x56339e139930 2024-02-02T15:46:09.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.385+0000 7f5c10345b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x56339e1a7970 con 0x56339e139010 2024-02-02T15:46:09.387 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.385+0000 7f5c10345b80 10 client.0 _ll_register_callbacks cb 0x56339e028290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:46:09.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.385+0000 7f5bf9400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56339e139930 0x56339e1522b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:09.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.386+0000 7f5bf9400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56339e139930 0x56339e1522b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:39686/0 (socket says 172.21.15.1:39686) 2024-02-02T15:46:09.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.386+0000 7f5bf9400700 1 -- 172.21.15.1:0/2728657374 learned_addr learned my addr 172.21.15.1:0/2728657374 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:46:09.389 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.386+0000 7f5bfa800700 1 --2- 172.21.15.1:0/2728657374 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56339e14d010 0x56339e1a7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:09.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.386+0000 7f5bf9400700 1 -- 172.21.15.1:0/2728657374 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56339e139010 msgr2=0x56339e151d70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-02-02T15:46:09.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.386+0000 7f5bf9400700 1 --2- 172.21.15.1:0/2728657374 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56339e139010 0x56339e151d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:09.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.386+0000 7f5bf9400700 1 -- 172.21.15.1:0/2728657374 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56339e14d010 msgr2=0x56339e1a7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:09.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.386+0000 7f5bf9400700 1 --2- 172.21.15.1:0/2728657374 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56339e14d010 0x56339e1a7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:09.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.386+0000 7f5bf9400700 1 -- 172.21.15.1:0/2728657374 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56339deb8b60 con 0x56339e139930 2024-02-02T15:46:09.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.386+0000 7f5bf9400700 1 --2- 172.21.15.1:0/2728657374 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56339e139930 0x56339e1522b0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f5be40a0610 tx=0x7f5be4098300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a32b61a7dd85a57a server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:09.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.386+0000 7f5be3400700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:46:09.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.387+0000 7f5be3400700 1 -- 172.21.15.1:0/2728657374 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be40cb680 con 0x56339e139930 2024-02-02T15:46:09.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.387+0000 7f5be3400700 1 -- 172.21.15.1:0/2728657374 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f5be4018520 con 0x56339e139930 2024-02-02T15:46:09.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.387+0000 7f5c10345b80 1 -- 172.21.15.1:0/2728657374 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5be40a0b40 con 0x56339e139930 2024-02-02T15:46:09.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.387+0000 7f5be3400700 1 -- 172.21.15.1:0/2728657374 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5be40a08f0 con 0x56339e139930 2024-02-02T15:46:09.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.387+0000 7f5c10345b80 10 client.30485 fetch_fsmap learned FSMap version 78 2024-02-02T15:46:09.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.387+0000 7f5c10345b80 1 -- 172.21.15.1:0/2728657374 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) v3 -- 0x56339df10cd0 con 0x56339e139930 2024-02-02T15:46:09.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.387+0000 7f5be3400700 1 -- 172.21.15.1:0/2728657374 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5be40a0b40 con 0x56339e139930 2024-02-02T15:46:09.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.387+0000 7f5be3400700 1 -- 172.21.15.1:0/2728657374 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 78) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f5be4077bb0 con 0x56339e139930 2024-02-02T15:46:09.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.388+0000 7f5c10345b80 10 client.30485 fetch_fsmap finished waiting for FSMap version 78 2024-02-02T15:46:09.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.388+0000 7f5c10345b80 10 client.30485 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-02-02T15:46:09.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.388+0000 7f5c10345b80 20 client.30485 populate_metadata read hostname 'smithi001' 2024-02-02T15:46:09.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.388+0000 7f5c10345b80 1 --2- 172.21.15.1:0/2728657374 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x56339e1aec30 0x56339e1b1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:09.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.388+0000 7f5c10345b80 4 client.30485 mds_command: new command op to 24518 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:46:09.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.388+0000 7f5c10345b80 1 -- 172.21.15.1:0/2728657374 --> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56339e049f50 con 0x56339e1aec30 2024-02-02T15:46:09.391 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.388+0000 7f5bf9e00700 1 --2- 172.21.15.1:0/2728657374 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x56339e1aec30 0x56339e1b1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:09.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.389+0000 7f5be3400700 10 client.30485 ms_handle_connect on v2:172.21.15.71:6836/628016199 2024-02-02T15:46:09.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.389+0000 7f5bf9e00700 1 --2- 172.21.15.1:0/2728657374 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x56339e1aec30 0x56339e1b1020 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3729d1e1d87e0d36 server_cookie=832dde3be3fd01cc in_seq=0 out_seq=0 2024-02-02T15:46:09.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.389+0000 7f5be3400700 1 -- 172.21.15.1:0/2728657374 <== mds.2 v2:172.21.15.71:6836/628016199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56339e049f50 con 0x56339e1aec30 2024-02-02T15:46:09.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.389+0000 7f5be3400700 10 client.30485 handle_command_reply: tid=0 2024-02-02T15:46:09.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.389+0000 7f5c10345b80 1 client.30485 shutdown 2024-02-02T15:46:09.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.389+0000 7f5c10345b80 1 -- 172.21.15.1:0/2728657374 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56339e139930 msgr2=0x56339e1522b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:09.392 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.389+0000 7f5c10345b80 1 --2- 172.21.15.1:0/2728657374 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56339e139930 0x56339e1522b0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f5be40a0610 tx=0x7f5be4098300 comp rx=0 tx=0).stop 2024-02-02T15:46:09.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.392+0000 7f5c10345b80 1 -- 172.21.15.1:0/2728657374 shutdown_connections 2024-02-02T15:46:09.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.392+0000 7f5c10345b80 1 --2- 172.21.15.1:0/2728657374 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56339e14d010 0x56339e1a7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:09.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.392+0000 7f5c10345b80 1 --2- 172.21.15.1:0/2728657374 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x56339e1aec30 0x56339e1b1020 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:09.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.392+0000 7f5c10345b80 1 --2- 172.21.15.1:0/2728657374 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56339e139010 0x56339e151d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:09.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.392+0000 7f5c10345b80 1 --2- 172.21.15.1:0/2728657374 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56339e139930 0x56339e1522b0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:09.393 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.392+0000 7f5be3400700 0 client.30485 ms_handle_reset on v2:172.21.15.71:6836/628016199 2024-02-02T15:46:09.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.392+0000 7f5c10345b80 1 -- 172.21.15.1:0/2728657374 >> 172.21.15.1:0/2728657374 conn(0x56339e098db0 msgr2=0x56339e134e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:09.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.392+0000 7f5c10345b80 1 -- 172.21.15.1:0/2728657374 shutdown_connections 2024-02-02T15:46:09.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.392+0000 7f5c10345b80 1 -- 172.21.15.1:0/2728657374 wait complete. 2024-02-02T15:46:09.394 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.393+0000 7f5c10345b80 20 client.30485 trim_cache size 0 max 16384 2024-02-02T15:46:09.394 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:46:09.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.395+0000 7f5bfc800700 1 -- 172.21.15.1:0/2532761847 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f5bd40a07d0 msgr2=0x7f5bd40a2c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:09.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.395+0000 7f5bfc800700 1 --2- 172.21.15.1:0/2532761847 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f5bd40a07d0 0x7f5bd40a2c90 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc005950 tx=0x7f5bdc0058e0 comp rx=0 tx=0).stop 2024-02-02T15:46:09.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.395+0000 7f5bfc800700 1 -- 172.21.15.1:0/2532761847 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5bf41083b0 msgr2=0x7f5bf4199690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:09.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.395+0000 7f5bfc800700 1 --2- 172.21.15.1:0/2532761847 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5bf41083b0 0x7f5bf4199690 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f5bf406a570 tx=0x7f5be400b750 comp rx=0 tx=0).stop 2024-02-02T15:46:09.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.395+0000 7f5bfc800700 1 -- 172.21.15.1:0/2532761847 shutdown_connections 2024-02-02T15:46:09.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.395+0000 7f5bfc800700 1 --2- 172.21.15.1:0/2532761847 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f5bd40a07d0 0x7f5bd40a2c90 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:09.395 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.395+0000 7f5bfc800700 1 --2- 172.21.15.1:0/2532761847 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5bf419daf0 0x7f5bf419df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:09.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.395+0000 7f5bfc800700 1 --2- 172.21.15.1:0/2532761847 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5bf4103290 0x7f5bf4199150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:09.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.395+0000 7f5bfc800700 1 --2- 172.21.15.1:0/2532761847 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5bf41083b0 0x7f5bf4199690 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:09.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.395+0000 7f5bfc800700 1 -- 172.21.15.1:0/2532761847 >> 172.21.15.1:0/2532761847 conn(0x7f5bf40feb30 msgr2=0x7f5bf40ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:09.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.396+0000 7f5bfc800700 1 -- 172.21.15.1:0/2532761847 shutdown_connections 2024-02-02T15:46:09.396 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.396+0000 7f5bfc800700 1 -- 172.21.15.1:0/2532761847 wait complete. 2024-02-02T15:46:09.423 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-02T15:46:09.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:46:09 smithi001 ceph-mon[35470]: pgmap v602: 129 pgs: 129 active+clean; 2.3 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 4.8 MiB/s wr, 416 op/s 2024-02-02T15:46:09.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.691+0000 7fbea0600700 1 -- 172.21.15.1:0/3871130582 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe98105cb0 msgr2=0x7fbe98068010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:09.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.691+0000 7fbea0600700 1 --2- 172.21.15.1:0/3871130582 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe98105cb0 0x7fbe98068010 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fbe88009990 tx=0x7fbe88009ca0 comp rx=0 tx=0).stop 2024-02-02T15:46:09.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.691+0000 7fbea0600700 1 -- 172.21.15.1:0/3871130582 shutdown_connections 2024-02-02T15:46:09.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.691+0000 7fbea0600700 1 --2- 172.21.15.1:0/3871130582 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe98103800 0x7fbe98103cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:09.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.691+0000 7fbea0600700 1 --2- 172.21.15.1:0/3871130582 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe98105cb0 0x7fbe98068010 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:09.691 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.691+0000 7fbea0600700 1 --2- 172.21.15.1:0/3871130582 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe98068550 0x7fbe980689d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:09.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.691+0000 7fbea0600700 1 -- 172.21.15.1:0/3871130582 >> 172.21.15.1:0/3871130582 conn(0x7fbe980757d0 msgr2=0x7fbe98075be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:09.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.691+0000 7fbea0600700 1 -- 172.21.15.1:0/3871130582 shutdown_connections 2024-02-02T15:46:09.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.691+0000 7fbea0600700 1 -- 172.21.15.1:0/3871130582 wait complete. 2024-02-02T15:46:09.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.693+0000 7fbea0600700 1 Processor -- start 2024-02-02T15:46:09.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.693+0000 7fbea0600700 1 -- start start 2024-02-02T15:46:09.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.693+0000 7fbea0600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe98068550 0x7fbe9810dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:09.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.693+0000 7fbea0600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe98103800 0x7fbe9810e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:09.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.693+0000 7fbea0600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe98105cb0 0x7fbe981127c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:09.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.693+0000 7fbea0600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe98112d00 con 0x7fbe98103800 2024-02-02T15:46:09.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.693+0000 7fbea0600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe98112e70 con 0x7fbe98105cb0 2024-02-02T15:46:09.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.693+0000 7fbea0600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe98113030 con 0x7fbe98068550 2024-02-02T15:46:09.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.694+0000 7fbe9e600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe98105cb0 0x7fbe981127c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:09.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.694+0000 7fbe9e600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe98105cb0 0x7fbe981127c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:39716/0 (socket says 172.21.15.1:39716) 2024-02-02T15:46:09.694 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.694+0000 7fbe9e600700 1 -- 172.21.15.1:0/2967648978 learned_addr learned my addr 172.21.15.1:0/2967648978 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:46:09.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.694+0000 7fbe9d200700 1 --2- 172.21.15.1:0/2967648978 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe98103800 0x7fbe9810e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:09.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.694+0000 7fbe9e600700 1 -- 172.21.15.1:0/2967648978 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe98068550 msgr2=0x7fbe9810dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:09.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.694+0000 7fbe9e600700 1 --2- 172.21.15.1:0/2967648978 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe98068550 0x7fbe9810dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:09.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.694+0000 7fbe9e600700 1 -- 172.21.15.1:0/2967648978 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe98103800 msgr2=0x7fbe9810e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:09.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.694+0000 7fbe9e600700 1 --2- 172.21.15.1:0/2967648978 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe98103800 0x7fbe9810e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:09.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.694+0000 7fbe9e600700 1 -- 172.21.15.1:0/2967648978 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe880095f0 con 0x7fbe98105cb0 2024-02-02T15:46:09.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.694+0000 7fbe9d200700 1 --2- 172.21.15.1:0/2967648978 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe98103800 0x7fbe9810e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-02-02T15:46:09.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.694+0000 7fbe9e600700 1 --2- 172.21.15.1:0/2967648978 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe98105cb0 0x7fbe981127c0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fbe9000ef30 tx=0x7fbe9000c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:09.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.695+0000 7fbe96000700 1 -- 172.21.15.1:0/2967648978 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe90018070 con 0x7fbe98105cb0 2024-02-02T15:46:09.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.695+0000 7fbe96000700 1 -- 172.21.15.1:0/2967648978 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe9000f040 con 0x7fbe98105cb0 2024-02-02T15:46:09.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.695+0000 7fbe96000700 1 -- 172.21.15.1:0/2967648978 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe90003dc0 con 0x7fbe98105cb0 2024-02-02T15:46:09.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.695+0000 7fbea0600700 1 -- 172.21.15.1:0/2967648978 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe98113250 con 0x7fbe98105cb0 2024-02-02T15:46:09.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.695+0000 7fbea0600700 1 -- 172.21.15.1:0/2967648978 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe981b2af0 con 0x7fbe98105cb0 2024-02-02T15:46:09.699 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.699+0000 7fbe96000700 1 -- 172.21.15.1:0/2967648978 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7fbe9000caa0 con 0x7fbe98105cb0 2024-02-02T15:46:09.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.701+0000 7fbe96000700 1 --2- 172.21.15.1:0/2967648978 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fbe780a0710 0x7fbe780a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:09.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.701+0000 7fbe96000700 1 -- 172.21.15.1:0/2967648978 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 9541+0+0 (secure 0 0 0) 0x7fbe900cbdd0 con 0x7fbe98105cb0 2024-02-02T15:46:09.701 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.702+0000 7fbe9dc00700 1 --2- 172.21.15.1:0/2967648978 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fbe780a0710 0x7fbe780a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:09.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.702+0000 7fbe9dc00700 1 --2- 172.21.15.1:0/2967648978 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fbe780a0710 0x7fbe780a2bd0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fbe880098e0 tx=0x7fbe88005e80 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:09.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.702+0000 7fbeb41d6b80 1 Processor -- start 2024-02-02T15:46:09.702 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.703+0000 7fbeb41d6b80 1 -- start start 2024-02-02T15:46:09.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.703+0000 7fbeb41d6b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564091301010 0x5640913013f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:09.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.703+0000 7fbeb41d6b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564091301930 0x56409130c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:09.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.703+0000 7fbeb41d6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56409130c7e0 0x56409130ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:09.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.703+0000 7fbeb41d6b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640912754c0 con 0x564091301010 2024-02-02T15:46:09.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.703+0000 7fbeb41d6b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x564091259aa0 con 0x564091301930 2024-02-02T15:46:09.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.703+0000 7fbeb41d6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564091257460 con 0x56409130c7e0 2024-02-02T15:46:09.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.703+0000 7fbe9dc00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564091301010 0x5640913013f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:09.703 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.703+0000 7fbe9e600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56409130c7e0 0x56409130ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:09.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.703+0000 7fbe9e600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56409130c7e0 0x56409130ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:44760/0 (socket says 172.21.15.1:44760) 2024-02-02T15:46:09.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.703+0000 7fbe9e600700 1 -- 172.21.15.1:0/2042393081 learned_addr learned my addr 172.21.15.1:0/2042393081 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:46:09.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.703+0000 7fbe9dc00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564091301010 0x5640913013f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:34678/0 (socket says 172.21.15.1:34678) 2024-02-02T15:46:09.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.703+0000 7fbe9d200700 1 --2- 172.21.15.1:0/2042393081 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564091301930 0x56409130c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:09.704 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.704+0000 7fbe9e600700 1 -- 172.21.15.1:0/2042393081 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564091301930 msgr2=0x56409130c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:09.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.704+0000 7fbe9e600700 1 --2- 172.21.15.1:0/2042393081 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564091301930 0x56409130c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:09.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.704+0000 7fbe9e600700 1 -- 172.21.15.1:0/2042393081 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564091301010 msgr2=0x5640913013f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:09.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.704+0000 7fbe9e600700 1 --2- 172.21.15.1:0/2042393081 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564091301010 0x5640913013f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:09.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.704+0000 7fbe9e600700 1 -- 172.21.15.1:0/2042393081 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5640910d43b0 con 0x56409130c7e0 2024-02-02T15:46:09.706 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.706+0000 7fbe9e600700 1 --2- 172.21.15.1:0/2042393081 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56409130c7e0 0x56409130ebd0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fbe90098eb0 tx=0x7fbe90098ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4fe62dd5e2367e8 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:09.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.708+0000 7fbe8ea00700 1 -- 172.21.15.1:0/2042393081 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe90098f20 con 0x56409130c7e0 2024-02-02T15:46:09.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.708+0000 7fbe8ea00700 1 -- 172.21.15.1:0/2042393081 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe9009cd50 con 0x56409130c7e0 2024-02-02T15:46:09.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.708+0000 7fbe8ea00700 1 -- 172.21.15.1:0/2042393081 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe9001b580 con 0x56409130c7e0 2024-02-02T15:46:09.708 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.709+0000 7fbeb41d6b80 1 -- 172.21.15.1:0/2042393081 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56409130c7e0 msgr2=0x56409130ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:09.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.709+0000 7fbeb41d6b80 1 --2- 172.21.15.1:0/2042393081 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56409130c7e0 0x56409130ebd0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fbe90098eb0 tx=0x7fbe90098ae0 comp rx=0 tx=0).stop 2024-02-02T15:46:09.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.709+0000 7fbeb41d6b80 1 -- 172.21.15.1:0/2042393081 shutdown_connections 2024-02-02T15:46:09.709 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.709+0000 7fbeb41d6b80 1 --2- 172.21.15.1:0/2042393081 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x56409130c7e0 0x56409130ebd0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:09.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.709+0000 7fbeb41d6b80 1 --2- 172.21.15.1:0/2042393081 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564091301010 0x5640913013f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:09.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.709+0000 7fbeb41d6b80 1 --2- 172.21.15.1:0/2042393081 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564091301930 0x56409130c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:09.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.709+0000 7fbeb41d6b80 1 -- 172.21.15.1:0/2042393081 >> 172.21.15.1:0/2042393081 conn(0x564091260db0 msgr2=0x5640912fd3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:09.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.709+0000 7fbeb41d6b80 1 -- 172.21.15.1:0/2042393081 shutdown_connections 2024-02-02T15:46:09.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.709+0000 7fbeb41d6b80 1 -- 172.21.15.1:0/2042393081 wait complete. 2024-02-02T15:46:09.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.710+0000 7fbeb41d6b80 1 -- start start 2024-02-02T15:46:09.710 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.710+0000 7fbeb41d6b80 1 Processor -- start 2024-02-02T15:46:09.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.711+0000 7fbeb41d6b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564091301010 0x56409130c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:09.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.711+0000 7fbeb41d6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564091301930 0x5640913094b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:09.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.711+0000 7fbeb41d6b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56409130c7e0 0x564091309bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:09.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.711+0000 7fbeb41d6b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x564091252d50 con 0x56409130c7e0 2024-02-02T15:46:09.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.711+0000 7fbeb41d6b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640911eed80 con 0x564091301010 2024-02-02T15:46:09.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.711+0000 7fbeb41d6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640911eea20 con 0x564091301930 2024-02-02T15:46:09.711 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.711+0000 7fbeb41d6b80 10 client.0 _ll_register_callbacks cb 0x5640911f0290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:46:09.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.711+0000 7fbe9d200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564091301930 0x5640913094b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:09.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.711+0000 7fbe9dc00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564091301010 0x56409130c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:09.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.711+0000 7fbe9d200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564091301930 0x5640913094b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:44772/0 (socket says 172.21.15.1:44772) 2024-02-02T15:46:09.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.711+0000 7fbe9dc00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564091301010 0x56409130c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:39736/0 (socket says 172.21.15.1:39736) 2024-02-02T15:46:09.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.711+0000 7fbe9d200700 1 -- 172.21.15.1:0/1564816233 learned_addr learned my addr 172.21.15.1:0/1564816233 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:46:09.712 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.712+0000 7fbe9e600700 1 --2- 172.21.15.1:0/1564816233 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56409130c7e0 0x564091309bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:09.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.712+0000 7fbe9dc00700 1 -- 172.21.15.1:0/1564816233 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564091301930 msgr2=0x5640913094b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:09.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.712+0000 7fbe9dc00700 1 --2- 172.21.15.1:0/1564816233 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564091301930 0x5640913094b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:09.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.712+0000 7fbe9dc00700 1 -- 172.21.15.1:0/1564816233 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56409130c7e0 msgr2=0x564091309bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:09.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.712+0000 7fbe9dc00700 1 --2- 172.21.15.1:0/1564816233 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56409130c7e0 0x564091309bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:09.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.712+0000 7fbe9dc00700 1 -- 172.21.15.1:0/1564816233 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe981b2af0 con 0x564091301010 2024-02-02T15:46:09.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.712+0000 7fbe9dc00700 1 --2- 172.21.15.1:0/1564816233 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564091301010 0x56409130c230 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fbe88005f50 tx=0x7fbe88000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39055b453941b061 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:09.713 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.712+0000 7fbe87400700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:46:09.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.713+0000 7fbe87400700 1 -- 172.21.15.1:0/1564816233 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe8802b040 con 0x564091301010 2024-02-02T15:46:09.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.713+0000 7fbe87400700 1 -- 172.21.15.1:0/1564816233 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 9541+0+0 (secure 0 0 0) 0x7fbe8800fdb0 con 0x564091301010 2024-02-02T15:46:09.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.713+0000 7fbeb41d6b80 1 -- 172.21.15.1:0/1564816233 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5640912f50f0 con 0x564091301010 2024-02-02T15:46:09.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.713+0000 7fbe87400700 1 -- 172.21.15.1:0/1564816233 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe8801d070 con 0x564091301010 2024-02-02T15:46:09.714 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.713+0000 7fbeb41d6b80 10 client.30497 fetch_fsmap learned FSMap version 78 2024-02-02T15:46:09.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.713+0000 7fbeb41d6b80 1 -- 172.21.15.1:0/1564816233 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) v3 -- 0x564091080b60 con 0x564091301010 2024-02-02T15:46:09.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.714+0000 7fbe87400700 1 -- 172.21.15.1:0/1564816233 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5640912f50f0 con 0x564091301010 2024-02-02T15:46:09.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.714+0000 7fbe87400700 1 -- 172.21.15.1:0/1564816233 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 78) v1 ==== 7506+0+0 (secure 0 0 0) 0x7fbe88025540 con 0x564091301010 2024-02-02T15:46:09.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.714+0000 7fbeb41d6b80 10 client.30497 fetch_fsmap finished waiting for FSMap version 78 2024-02-02T15:46:09.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.714+0000 7fbeb41d6b80 10 client.30497 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-02T15:46:09.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.714+0000 7fbeb41d6b80 20 client.30497 populate_metadata read hostname 'smithi001' 2024-02-02T15:46:09.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.714+0000 7fbeb41d6b80 1 --2- 172.21.15.1:0/1564816233 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x56409131c2c0 0x56409131e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:09.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.714+0000 7fbeb41d6b80 4 client.30497 mds_command: new command op to 24529 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:46:09.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.714+0000 7fbeb41d6b80 1 -- 172.21.15.1:0/1564816233 --> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56409124bdd0 con 0x56409131c2c0 2024-02-02T15:46:09.715 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.714+0000 7fbe9d200700 1 --2- 172.21.15.1:0/1564816233 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x56409131c2c0 0x56409131e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:09.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.715+0000 7fbe9d200700 1 --2- 172.21.15.1:0/1564816233 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x56409131c2c0 0x56409131e6e0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9f97871eabf45b35 server_cookie=729f039880d261f in_seq=0 out_seq=0 2024-02-02T15:46:09.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.715+0000 7fbe87400700 10 client.30497 ms_handle_connect on v2:172.21.15.142:6838/321675421 2024-02-02T15:46:09.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.716+0000 7fbe87400700 1 -- 172.21.15.1:0/1564816233 <== mds.1 v2:172.21.15.142:6838/321675421 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x56409124bdd0 con 0x56409131c2c0 2024-02-02T15:46:09.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.716+0000 7fbe87400700 10 client.30497 handle_command_reply: tid=0 2024-02-02T15:46:09.716 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.716+0000 7fbeb41d6b80 1 client.30497 shutdown 2024-02-02T15:46:09.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.717+0000 7fbeb41d6b80 1 -- 172.21.15.1:0/1564816233 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564091301010 msgr2=0x56409130c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:09.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.717+0000 7fbeb41d6b80 1 --2- 172.21.15.1:0/1564816233 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564091301010 0x56409130c230 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fbe88005f50 tx=0x7fbe88000bc0 comp rx=0 tx=0).stop 2024-02-02T15:46:09.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.717+0000 7fbeb41d6b80 1 -- 172.21.15.1:0/1564816233 shutdown_connections 2024-02-02T15:46:09.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.717+0000 7fbeb41d6b80 1 --2- 172.21.15.1:0/1564816233 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x56409131c2c0 0x56409131e6e0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:09.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.717+0000 7fbeb41d6b80 1 --2- 172.21.15.1:0/1564816233 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56409130c7e0 0x564091309bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:09.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.717+0000 7fbeb41d6b80 1 --2- 172.21.15.1:0/1564816233 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564091301010 0x56409130c230 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:09.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.717+0000 7fbeb41d6b80 1 --2- 172.21.15.1:0/1564816233 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564091301930 0x5640913094b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:09.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.717+0000 7fbe87400700 0 client.30497 ms_handle_reset on v2:172.21.15.142:6838/321675421 2024-02-02T15:46:09.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.717+0000 7fbeb41d6b80 1 -- 172.21.15.1:0/1564816233 >> 172.21.15.1:0/1564816233 conn(0x564091260db0 msgr2=0x5640912fcfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:09.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.717+0000 7fbeb41d6b80 1 -- 172.21.15.1:0/1564816233 shutdown_connections 2024-02-02T15:46:09.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.717+0000 7fbeb41d6b80 1 -- 172.21.15.1:0/1564816233 wait complete. 2024-02-02T15:46:09.718 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.718+0000 7fbeb41d6b80 20 client.30497 trim_cache size 0 max 16384 2024-02-02T15:46:09.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.736+0000 7fbeb41d6b80 1 Processor -- start 2024-02-02T15:46:09.736 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.736+0000 7fbeb41d6b80 1 -- start start 2024-02-02T15:46:09.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.737+0000 7fbeb41d6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564091301930 0x5640913132d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:09.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.737+0000 7fbeb41d6b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56409130c7e0 0x56409130bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:09.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.737+0000 7fbeb41d6b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56409130c2a0 0x564091304840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:09.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.737+0000 7fbeb41d6b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640910d66a0 con 0x56409130c7e0 2024-02-02T15:46:09.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.737+0000 7fbeb41d6b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640912f4cc0 con 0x56409130c2a0 2024-02-02T15:46:09.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.737+0000 7fbeb41d6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640912f7500 con 0x564091301930 2024-02-02T15:46:09.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.737+0000 7fbe9d200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56409130c7e0 0x56409130bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:09.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.737+0000 7fbe9d200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56409130c7e0 0x56409130bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:34702/0 (socket says 172.21.15.1:34702) 2024-02-02T15:46:09.737 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.737+0000 7fbe9d200700 1 -- 172.21.15.1:0/1385486063 learned_addr learned my addr 172.21.15.1:0/1385486063 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:46:09.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.738+0000 7fbe9e600700 1 --2- 172.21.15.1:0/1385486063 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56409130c2a0 0x564091304840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:09.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.738+0000 7fbe9d200700 1 -- 172.21.15.1:0/1385486063 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564091301930 msgr2=0x5640913132d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:09.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.738+0000 7fbe9d200700 1 --2- 172.21.15.1:0/1385486063 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564091301930 0x5640913132d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:09.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.738+0000 7fbe9d200700 1 -- 172.21.15.1:0/1385486063 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56409130c2a0 msgr2=0x564091304840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:09.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.738+0000 7fbe9d200700 1 --2- 172.21.15.1:0/1385486063 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56409130c2a0 0x564091304840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:09.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.738+0000 7fbe9d200700 1 -- 172.21.15.1:0/1385486063 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564091302b00 con 0x56409130c7e0 2024-02-02T15:46:09.738 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.738+0000 7fbe9e600700 1 --2- 172.21.15.1:0/1385486063 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56409130c2a0 0x564091304840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:46:09.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.738+0000 7fbe9d200700 1 --2- 172.21.15.1:0/1385486063 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56409130c7e0 0x56409130bd60 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x56409131ef80 tx=0x7fbe8001da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d14c2b0d7a0fc194 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:09.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.739+0000 7fbe8f400700 1 -- 172.21.15.1:0/1385486063 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe80021030 con 0x56409130c7e0 2024-02-02T15:46:09.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.739+0000 7fbe8f400700 1 -- 172.21.15.1:0/1385486063 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe80003680 con 0x56409130c7e0 2024-02-02T15:46:09.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.739+0000 7fbe8f400700 1 -- 172.21.15.1:0/1385486063 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe80018040 con 0x56409130c7e0 2024-02-02T15:46:09.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.740+0000 7fbeb41d6b80 1 -- 172.21.15.1:0/1385486063 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56409130c7e0 msgr2=0x56409130bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:09.739 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.740+0000 7fbeb41d6b80 1 --2- 172.21.15.1:0/1385486063 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56409130c7e0 0x56409130bd60 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x56409131ef80 tx=0x7fbe8001da60 comp rx=0 tx=0).stop 2024-02-02T15:46:09.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.740+0000 7fbeb41d6b80 1 -- 172.21.15.1:0/1385486063 shutdown_connections 2024-02-02T15:46:09.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.740+0000 7fbeb41d6b80 1 --2- 172.21.15.1:0/1385486063 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56409130c2a0 0x564091304840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:09.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.740+0000 7fbeb41d6b80 1 --2- 172.21.15.1:0/1385486063 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564091301930 0x5640913132d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:09.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.740+0000 7fbeb41d6b80 1 --2- 172.21.15.1:0/1385486063 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56409130c7e0 0x56409130bd60 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:09.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.740+0000 7fbeb41d6b80 1 -- 172.21.15.1:0/1385486063 >> 172.21.15.1:0/1385486063 conn(0x564091260db0 msgr2=0x5640913178a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:09.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.740+0000 7fbeb41d6b80 1 -- 172.21.15.1:0/1385486063 shutdown_connections 2024-02-02T15:46:09.740 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.740+0000 7fbeb41d6b80 1 -- 172.21.15.1:0/1385486063 wait complete. 2024-02-02T15:46:09.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.741+0000 7fbeb41d6b80 1 -- start start 2024-02-02T15:46:09.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.741+0000 7fbeb41d6b80 1 Processor -- start 2024-02-02T15:46:09.741 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.742+0000 7fbeb41d6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564091301930 0x564091312de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:09.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.742+0000 7fbeb41d6b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56409130c2a0 0x5640913112b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:09.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.742+0000 7fbeb41d6b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56409130c7e0 0x5640913117f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:09.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.742+0000 7fbe9dc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564091301930 0x564091312de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:09.742 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.742+0000 7fbe9dc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564091301930 0x564091312de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:44780/0 (socket says 172.21.15.1:44780) 2024-02-02T15:46:09.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.743+0000 7fbeb41d6b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x564091311d30 con 0x56409130c7e0 2024-02-02T15:46:09.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.743+0000 7fbeb41d6b80 1 -- 172.21.15.1:0/4144429268 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x564091311ea0 con 0x56409130c2a0 2024-02-02T15:46:09.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.743+0000 7fbe9dc00700 1 -- 172.21.15.1:0/4144429268 learned_addr learned my addr 172.21.15.1:0/4144429268 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:46:09.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.743+0000 7fbeb41d6b80 1 -- 172.21.15.1:0/4144429268 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564091312060 con 0x564091301930 2024-02-02T15:46:09.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.743+0000 7fbeb41d6b80 10 client.0 _ll_register_callbacks cb 0x5640911f0290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:46:09.743 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.743+0000 7fbe9e600700 1 --2- 172.21.15.1:0/4144429268 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56409130c7e0 0x5640913117f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:09.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.743+0000 7fbe9d200700 1 --2- 172.21.15.1:0/4144429268 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56409130c2a0 0x5640913112b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:09.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.744+0000 7fbe9e600700 1 -- 172.21.15.1:0/4144429268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564091301930 msgr2=0x564091312de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:09.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.744+0000 7fbe9e600700 1 --2- 172.21.15.1:0/4144429268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564091301930 0x564091312de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:09.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.744+0000 7fbe9e600700 1 -- 172.21.15.1:0/4144429268 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56409130c2a0 msgr2=0x5640913112b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:09.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.744+0000 7fbe9e600700 1 --2- 172.21.15.1:0/4144429268 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56409130c2a0 0x5640913112b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:09.744 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.744+0000 7fbe9e600700 1 -- 172.21.15.1:0/4144429268 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5640910d43b0 con 0x56409130c7e0 2024-02-02T15:46:09.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.745+0000 7fbe9e600700 1 --2- 172.21.15.1:0/4144429268 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56409130c7e0 0x5640913117f0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x564091301e80 tx=0x7fbe90095260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50a3c28c9be57b6d server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:09.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.745+0000 7fbe87400700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:46:09.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.745+0000 7fbe87400700 1 -- 172.21.15.1:0/4144429268 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe9001b320 con 0x56409130c7e0 2024-02-02T15:46:09.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.745+0000 7fbeb41d6b80 1 -- 172.21.15.1:0/4144429268 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbe9009cd50 con 0x56409130c7e0 2024-02-02T15:46:09.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.745+0000 7fbe87400700 1 -- 172.21.15.1:0/4144429268 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 9541+0+0 (secure 0 0 0) 0x7fbe9001b480 con 0x56409130c7e0 2024-02-02T15:46:09.745 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.745+0000 7fbe87400700 1 -- 172.21.15.1:0/4144429268 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe90099180 con 0x56409130c7e0 2024-02-02T15:46:09.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.746+0000 7fbeb41d6b80 10 client.20202 fetch_fsmap learned FSMap version 78 2024-02-02T15:46:09.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.746+0000 7fbeb41d6b80 1 -- 172.21.15.1:0/4144429268 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) v3 -- 0x7fbe80009c10 con 0x56409130c7e0 2024-02-02T15:46:09.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.746+0000 7fbe87400700 1 -- 172.21.15.1:0/4144429268 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe9009cd50 con 0x56409130c7e0 2024-02-02T15:46:09.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.746+0000 7fbe87400700 1 -- 172.21.15.1:0/4144429268 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 78) v1 ==== 7506+0+0 (secure 0 0 0) 0x7fbe900a0a70 con 0x56409130c7e0 2024-02-02T15:46:09.746 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.746+0000 7fbeb41d6b80 10 client.20202 fetch_fsmap finished waiting for FSMap version 78 2024-02-02T15:46:09.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.746+0000 7fbeb41d6b80 10 client.20202 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-02T15:46:09.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.746+0000 7fbeb41d6b80 20 client.20202 populate_metadata read hostname 'smithi001' 2024-02-02T15:46:09.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.746+0000 7fbeb41d6b80 1 --2- 172.21.15.1:0/4144429268 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x56409136ecf0 0x564091371110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:09.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.746+0000 7fbeb41d6b80 4 client.20202 mds_command: new command op to 24529 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:46:09.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.746+0000 7fbeb41d6b80 1 -- 172.21.15.1:0/4144429268 --> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564091211f50 con 0x56409136ecf0 2024-02-02T15:46:09.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.747+0000 7fbe9dc00700 1 --2- 172.21.15.1:0/4144429268 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x56409136ecf0 0x564091371110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:09.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.747+0000 7fbe87400700 10 client.20202 ms_handle_connect on v2:172.21.15.142:6838/321675421 2024-02-02T15:46:09.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.747+0000 7fbe9dc00700 1 --2- 172.21.15.1:0/4144429268 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x56409136ecf0 0x564091371110 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=820c91bd86386a38 server_cookie=fed354efcd9134e6 in_seq=0 out_seq=0 2024-02-02T15:46:09.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.753+0000 7fbe87400700 1 -- 172.21.15.1:0/4144429268 <== mds.1 v2:172.21.15.142:6838/321675421 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564091211f50 con 0x56409136ecf0 2024-02-02T15:46:09.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.753+0000 7fbe87400700 10 client.20202 handle_command_reply: tid=0 2024-02-02T15:46:09.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.753+0000 7fbeb41d6b80 1 client.20202 shutdown 2024-02-02T15:46:09.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.753+0000 7fbeb41d6b80 1 -- 172.21.15.1:0/4144429268 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56409130c7e0 msgr2=0x5640913117f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:09.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.753+0000 7fbeb41d6b80 1 --2- 172.21.15.1:0/4144429268 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56409130c7e0 0x5640913117f0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x564091301e80 tx=0x7fbe90095260 comp rx=0 tx=0).stop 2024-02-02T15:46:09.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.753+0000 7fbeb41d6b80 1 -- 172.21.15.1:0/4144429268 shutdown_connections 2024-02-02T15:46:09.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.753+0000 7fbeb41d6b80 1 --2- 172.21.15.1:0/4144429268 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x56409136ecf0 0x564091371110 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:09.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.753+0000 7fbeb41d6b80 1 --2- 172.21.15.1:0/4144429268 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x56409130c7e0 0x5640913117f0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:09.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.753+0000 7fbeb41d6b80 1 --2- 172.21.15.1:0/4144429268 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564091301930 0x564091312de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:09.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.753+0000 7fbeb41d6b80 1 --2- 172.21.15.1:0/4144429268 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56409130c2a0 0x5640913112b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:09.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.753+0000 7fbe87400700 0 client.20202 ms_handle_reset on v2:172.21.15.142:6838/321675421 2024-02-02T15:46:09.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.753+0000 7fbeb41d6b80 1 -- 172.21.15.1:0/4144429268 >> 172.21.15.1:0/4144429268 conn(0x564091260db0 msgr2=0x564091304840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:09.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.753+0000 7fbeb41d6b80 1 -- 172.21.15.1:0/4144429268 shutdown_connections 2024-02-02T15:46:09.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.754+0000 7fbeb41d6b80 1 -- 172.21.15.1:0/4144429268 wait complete. 2024-02-02T15:46:09.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.754+0000 7fbeb41d6b80 20 client.20202 trim_cache size 0 max 16384 2024-02-02T15:46:09.755 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:46:09.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.755+0000 7fbea0600700 1 -- 172.21.15.1:0/2967648978 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fbe780a0710 msgr2=0x7fbe780a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:09.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.755+0000 7fbea0600700 1 --2- 172.21.15.1:0/2967648978 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fbe780a0710 0x7fbe780a2bd0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fbe880098e0 tx=0x7fbe88005e80 comp rx=0 tx=0).stop 2024-02-02T15:46:09.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.755+0000 7fbea0600700 1 -- 172.21.15.1:0/2967648978 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe98105cb0 msgr2=0x7fbe981127c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:09.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.755+0000 7fbea0600700 1 --2- 172.21.15.1:0/2967648978 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe98105cb0 0x7fbe981127c0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fbe9000ef30 tx=0x7fbe9000c450 comp rx=0 tx=0).stop 2024-02-02T15:46:09.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.755+0000 7fbea0600700 1 -- 172.21.15.1:0/2967648978 shutdown_connections 2024-02-02T15:46:09.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.755+0000 7fbea0600700 1 --2- 172.21.15.1:0/2967648978 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fbe780a0710 0x7fbe780a2bd0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:09.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.755+0000 7fbea0600700 1 --2- 172.21.15.1:0/2967648978 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe98105cb0 0x7fbe981127c0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:09.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.755+0000 7fbea0600700 1 --2- 172.21.15.1:0/2967648978 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe98068550 0x7fbe9810dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:09.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.755+0000 7fbea0600700 1 --2- 172.21.15.1:0/2967648978 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe98103800 0x7fbe9810e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:09.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.755+0000 7fbea0600700 1 -- 172.21.15.1:0/2967648978 >> 172.21.15.1:0/2967648978 conn(0x7fbe980757d0 msgr2=0x7fbe981027e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:09.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.755+0000 7fbea0600700 1 -- 172.21.15.1:0/2967648978 shutdown_connections 2024-02-02T15:46:09.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:09.755+0000 7fbea0600700 1 -- 172.21.15.1:0/2967648978 wait complete. 2024-02-02T15:46:09.780 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-02T15:46:09.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:46:09 smithi071 ceph-mon[44707]: pgmap v602: 129 pgs: 129 active+clean; 2.3 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 4.8 MiB/s wr, 416 op/s 2024-02-02T15:46:09.922 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:46:09 smithi142 ceph-mon[45728]: pgmap v602: 129 pgs: 129 active+clean; 2.3 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 4.8 MiB/s wr, 416 op/s 2024-02-02T15:46:10.011 INFO:tasks.workunit.client.0.smithi001.stdout: 10 178532 26.43 MB/sec execute 222 sec latency 198.427 ms 2024-02-02T15:46:10.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.054+0000 7ff08ce00700 1 -- 172.21.15.1:0/1118282787 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff088073a10 msgr2=0x7ff08810d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:10.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.054+0000 7ff08ce00700 1 --2- 172.21.15.1:0/1118282787 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff088073a10 0x7ff08810d110 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7ff070009990 tx=0x7ff070009ca0 comp rx=0 tx=0).stop 2024-02-02T15:46:10.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.054+0000 7ff08ce00700 1 -- 172.21.15.1:0/1118282787 shutdown_connections 2024-02-02T15:46:10.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.054+0000 7ff08ce00700 1 --2- 172.21.15.1:0/1118282787 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff08810d650 0x7ff08810fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:10.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.054+0000 7ff08ce00700 1 --2- 172.21.15.1:0/1118282787 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff0880730f0 0x7ff0880734d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:10.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.054+0000 7ff08ce00700 1 --2- 172.21.15.1:0/1118282787 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff088073a10 0x7ff08810d110 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:10.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.054+0000 7ff08ce00700 1 -- 172.21.15.1:0/1118282787 >> 172.21.15.1:0/1118282787 conn(0x7ff0880fc840 msgr2=0x7ff0880fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:10.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.054+0000 7ff08ce00700 1 -- 172.21.15.1:0/1118282787 shutdown_connections 2024-02-02T15:46:10.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.054+0000 7ff08ce00700 1 -- 172.21.15.1:0/1118282787 wait complete. 2024-02-02T15:46:10.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.056+0000 7ff08ce00700 1 Processor -- start 2024-02-02T15:46:10.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.056+0000 7ff08ce00700 1 -- start start 2024-02-02T15:46:10.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.056+0000 7ff08ce00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0880730f0 0x7ff0881990b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:10.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.056+0000 7ff08ce00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff088073a10 0x7ff0881995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:10.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.056+0000 7ff08ce00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff08810d650 0x7ff08819da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:10.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.056+0000 7ff08ce00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff08819e090 con 0x7ff088073a10 2024-02-02T15:46:10.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.056+0000 7ff08ce00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff08819e200 con 0x7ff0880730f0 2024-02-02T15:46:10.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.056+0000 7ff08ce00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff08819e3c0 con 0x7ff08810d650 2024-02-02T15:46:10.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.057+0000 7ff085e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0880730f0 0x7ff0881990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:10.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.057+0000 7ff085e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0880730f0 0x7ff0881990b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:39788/0 (socket says 172.21.15.1:39788) 2024-02-02T15:46:10.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.057+0000 7ff085e00700 1 -- 172.21.15.1:0/3031678007 learned_addr learned my addr 172.21.15.1:0/3031678007 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:46:10.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.057+0000 7ff085400700 1 --2- 172.21.15.1:0/3031678007 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff088073a10 0x7ff0881995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:10.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.057+0000 7ff086800700 1 --2- 172.21.15.1:0/3031678007 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff08810d650 0x7ff08819da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:10.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.057+0000 7ff085e00700 1 -- 172.21.15.1:0/3031678007 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff08810d650 msgr2=0x7ff08819da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:10.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.057+0000 7ff085e00700 1 --2- 172.21.15.1:0/3031678007 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff08810d650 0x7ff08819da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:10.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.057+0000 7ff085e00700 1 -- 172.21.15.1:0/3031678007 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff088073a10 msgr2=0x7ff0881995f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:10.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.057+0000 7ff085e00700 1 --2- 172.21.15.1:0/3031678007 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff088073a10 0x7ff0881995f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:10.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.057+0000 7ff085e00700 1 -- 172.21.15.1:0/3031678007 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0700095f0 con 0x7ff0880730f0 2024-02-02T15:46:10.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.058+0000 7ff085e00700 1 --2- 172.21.15.1:0/3031678007 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0880730f0 0x7ff0881990b0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7ff07800e9a0 tx=0x7ff07800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:10.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.058+0000 7ff07ea00700 1 -- 172.21.15.1:0/3031678007 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff078009790 con 0x7ff0880730f0 2024-02-02T15:46:10.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.058+0000 7ff07ea00700 1 -- 172.21.15.1:0/3031678007 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff078010430 con 0x7ff0880730f0 2024-02-02T15:46:10.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.058+0000 7ff08ce00700 1 -- 172.21.15.1:0/3031678007 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff08819e6a0 con 0x7ff0880730f0 2024-02-02T15:46:10.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.058+0000 7ff07ea00700 1 -- 172.21.15.1:0/3031678007 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff078018680 con 0x7ff0880730f0 2024-02-02T15:46:10.059 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.058+0000 7ff08ce00700 1 -- 172.21.15.1:0/3031678007 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0881a5ee0 con 0x7ff0880730f0 2024-02-02T15:46:10.062 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.062+0000 7ff07ea00700 1 -- 172.21.15.1:0/3031678007 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7ff0780105a0 con 0x7ff0880730f0 2024-02-02T15:46:10.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.064+0000 7ff07ea00700 1 --2- 172.21.15.1:0/3031678007 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff0680a0710 0x7ff0680a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:10.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.064+0000 7ff0a0878b80 1 Processor -- start 2024-02-02T15:46:10.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.064+0000 7ff07ea00700 1 -- 172.21.15.1:0/3031678007 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 9541+0+0 (secure 0 0 0) 0x7ff078014070 con 0x7ff0880730f0 2024-02-02T15:46:10.064 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.064+0000 7ff0a0878b80 1 -- start start 2024-02-02T15:46:10.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.065+0000 7ff0a0878b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55df60d37010 0x55df60d373f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:10.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.065+0000 7ff0a0878b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55df60d37930 0x55df60d422a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:10.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.065+0000 7ff0a0878b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df60d427e0 0x55df60d44bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:10.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.065+0000 7ff0a0878b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df60cab4c0 con 0x55df60d37930 2024-02-02T15:46:10.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.065+0000 7ff0a0878b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df60c8faa0 con 0x55df60d37010 2024-02-02T15:46:10.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.065+0000 7ff0a0878b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df60c8d460 con 0x55df60d427e0 2024-02-02T15:46:10.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.065+0000 7ff085e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55df60d37010 0x55df60d373f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:10.066 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.065+0000 7ff085e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55df60d37010 0x55df60d373f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:39802/0 (socket says 172.21.15.1:39802) 2024-02-02T15:46:10.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.065+0000 7ff085e00700 1 -- 172.21.15.1:0/2230996771 learned_addr learned my addr 172.21.15.1:0/2230996771 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:46:10.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.065+0000 7ff086800700 1 --2- 172.21.15.1:0/2230996771 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df60d427e0 0x55df60d44bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:10.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.065+0000 7ff085400700 1 --2- 172.21.15.1:0/3031678007 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff0680a0710 0x7ff0680a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:10.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.066+0000 7ff085e00700 1 -- 172.21.15.1:0/2230996771 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df60d427e0 msgr2=0x55df60d44bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:10.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.066+0000 7ff085e00700 1 --2- 172.21.15.1:0/2230996771 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df60d427e0 0x55df60d44bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:10.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.066+0000 7ff085e00700 1 -- 172.21.15.1:0/2230996771 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55df60d37930 msgr2=0x55df60d422a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-02-02T15:46:10.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.066+0000 7ff085e00700 1 --2- 172.21.15.1:0/2230996771 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55df60d37930 0x55df60d422a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:10.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.066+0000 7ff085e00700 1 -- 172.21.15.1:0/2230996771 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df60b0a3b0 con 0x55df60d37010 2024-02-02T15:46:10.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.066+0000 7ff085e00700 1 --2- 172.21.15.1:0/2230996771 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55df60d37010 0x55df60d373f0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7ff078099350 tx=0x7ff078098e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=358892b45c8ec83 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:10.067 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.067+0000 7ff085400700 1 --2- 172.21.15.1:0/3031678007 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff0680a0710 0x7ff0680a2bd0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff08819a6c0 tx=0x7ff07000b610 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:10.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.068+0000 7ff077400700 1 -- 172.21.15.1:0/2230996771 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff07809d1f0 con 0x55df60d37010 2024-02-02T15:46:10.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.069+0000 7ff077400700 1 -- 172.21.15.1:0/2230996771 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff0780a1430 con 0x55df60d37010 2024-02-02T15:46:10.068 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.069+0000 7ff077400700 1 -- 172.21.15.1:0/2230996771 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0780988b0 con 0x55df60d37010 2024-02-02T15:46:10.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.070+0000 7ff0a0878b80 1 -- 172.21.15.1:0/2230996771 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55df60d37010 msgr2=0x55df60d373f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:10.069 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.070+0000 7ff0a0878b80 1 --2- 172.21.15.1:0/2230996771 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55df60d37010 0x55df60d373f0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7ff078099350 tx=0x7ff078098e10 comp rx=0 tx=0).stop 2024-02-02T15:46:10.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.070+0000 7ff0a0878b80 1 -- 172.21.15.1:0/2230996771 shutdown_connections 2024-02-02T15:46:10.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.070+0000 7ff0a0878b80 1 --2- 172.21.15.1:0/2230996771 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df60d427e0 0x55df60d44bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:10.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.070+0000 7ff0a0878b80 1 --2- 172.21.15.1:0/2230996771 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55df60d37010 0x55df60d373f0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:10.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.070+0000 7ff0a0878b80 1 --2- 172.21.15.1:0/2230996771 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55df60d37930 0x55df60d422a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:10.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.070+0000 7ff0a0878b80 1 -- 172.21.15.1:0/2230996771 >> 172.21.15.1:0/2230996771 conn(0x55df60c96db0 msgr2=0x55df60d333b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:10.070 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.070+0000 7ff0a0878b80 1 -- 172.21.15.1:0/2230996771 shutdown_connections 2024-02-02T15:46:10.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.070+0000 7ff0a0878b80 1 -- 172.21.15.1:0/2230996771 wait complete. 2024-02-02T15:46:10.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.071+0000 7ff0a0878b80 1 -- start start 2024-02-02T15:46:10.071 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.071+0000 7ff0a0878b80 1 Processor -- start 2024-02-02T15:46:10.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.072+0000 7ff0a0878b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55df60d37010 0x55df60d42290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:10.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.072+0000 7ff0a0878b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df60d37930 0x55df60d3f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:10.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.072+0000 7ff0a0878b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55df60d427e0 0x55df60d3fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:10.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.072+0000 7ff0a0878b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df60c88d50 con 0x55df60d37010 2024-02-02T15:46:10.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.072+0000 7ff0a0878b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df60c24d80 con 0x55df60d427e0 2024-02-02T15:46:10.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.072+0000 7ff0a0878b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df60c24a20 con 0x55df60d37930 2024-02-02T15:46:10.072 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.072+0000 7ff0a0878b80 10 client.0 _ll_register_callbacks cb 0x55df60c26290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:46:10.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.072+0000 7ff085400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df60d37930 0x55df60d3f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:10.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.072+0000 7ff085400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df60d37930 0x55df60d3f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:44808/0 (socket says 172.21.15.1:44808) 2024-02-02T15:46:10.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.072+0000 7ff085400700 1 -- 172.21.15.1:0/751537708 learned_addr learned my addr 172.21.15.1:0/751537708 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:46:10.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.073+0000 7ff085e00700 1 --2- 172.21.15.1:0/751537708 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55df60d37010 0x55df60d42290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:10.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.073+0000 7ff086800700 1 --2- 172.21.15.1:0/751537708 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55df60d427e0 0x55df60d3fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:10.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.073+0000 7ff085e00700 1 -- 172.21.15.1:0/751537708 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df60d37930 msgr2=0x55df60d3f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:10.074 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.073+0000 7ff085e00700 1 --2- 172.21.15.1:0/751537708 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df60d37930 0x55df60d3f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:10.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.073+0000 7ff085e00700 1 -- 172.21.15.1:0/751537708 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55df60d427e0 msgr2=0x55df60d3fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:10.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.073+0000 7ff085e00700 1 --2- 172.21.15.1:0/751537708 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55df60d427e0 0x55df60d3fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:10.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.073+0000 7ff085e00700 1 -- 172.21.15.1:0/751537708 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0881a5ee0 con 0x55df60d37010 2024-02-02T15:46:10.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.073+0000 7ff085e00700 1 --2- 172.21.15.1:0/751537708 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55df60d37010 0x55df60d42290 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7ff07809d160 tx=0x7ff078010a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bbac0d854ac4f3e server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:10.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.074+0000 7ff06fe00700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:46:10.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.074+0000 7ff06fe00700 1 -- 172.21.15.1:0/751537708 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0780840b0 con 0x55df60d37010 2024-02-02T15:46:10.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.074+0000 7ff06fe00700 1 -- 172.21.15.1:0/751537708 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 9541+0+0 (secure 0 0 0) 0x7ff07809d1f0 con 0x55df60d37010 2024-02-02T15:46:10.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.074+0000 7ff0a0878b80 1 -- 172.21.15.1:0/751537708 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55df60d2b0f0 con 0x55df60d37010 2024-02-02T15:46:10.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.074+0000 7ff06fe00700 1 -- 172.21.15.1:0/751537708 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff078099830 con 0x55df60d37010 2024-02-02T15:46:10.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.074+0000 7ff0a0878b80 10 client.20211 fetch_fsmap learned FSMap version 78 2024-02-02T15:46:10.075 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.074+0000 7ff0a0878b80 1 -- 172.21.15.1:0/751537708 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) v3 -- 0x55df60ab6b60 con 0x55df60d37010 2024-02-02T15:46:10.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.075+0000 7ff06fe00700 1 -- 172.21.15.1:0/751537708 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55df60d2b0f0 con 0x55df60d37010 2024-02-02T15:46:10.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.075+0000 7ff06fe00700 1 -- 172.21.15.1:0/751537708 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 78) v1 ==== 7506+0+0 (secure 0 0 0) 0x7ff078098320 con 0x55df60d37010 2024-02-02T15:46:10.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.075+0000 7ff0a0878b80 10 client.20211 fetch_fsmap finished waiting for FSMap version 78 2024-02-02T15:46:10.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.075+0000 7ff0a0878b80 10 client.20211 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:46:10.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.075+0000 7ff0a0878b80 20 client.20211 populate_metadata read hostname 'smithi001' 2024-02-02T15:46:10.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.075+0000 7ff0a0878b80 1 --2- 172.21.15.1:0/751537708 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55df60d51e50 0x55df60d54270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:10.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.075+0000 7ff0a0878b80 4 client.20211 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:46:10.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.075+0000 7ff0a0878b80 1 -- 172.21.15.1:0/751537708 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55df60c81dd0 con 0x55df60d51e50 2024-02-02T15:46:10.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.075+0000 7ff085400700 1 --2- 172.21.15.1:0/751537708 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55df60d51e50 0x55df60d54270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:10.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.076+0000 7ff085400700 1 --2- 172.21.15.1:0/751537708 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55df60d51e50 0x55df60d54270 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a239fa1a8fe226c server_cookie=4a7fd99f7552d99f in_seq=0 out_seq=0 2024-02-02T15:46:10.077 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.076+0000 7ff06fe00700 10 client.20211 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:46:10.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.077+0000 7ff06fe00700 1 -- 172.21.15.1:0/751537708 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55df60c81dd0 con 0x55df60d51e50 2024-02-02T15:46:10.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.077+0000 7ff06fe00700 10 client.20211 handle_command_reply: tid=0 2024-02-02T15:46:10.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.077+0000 7ff0a0878b80 1 client.20211 shutdown 2024-02-02T15:46:10.078 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.077+0000 7ff0a0878b80 1 -- 172.21.15.1:0/751537708 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55df60d37010 msgr2=0x55df60d42290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:10.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.077+0000 7ff0a0878b80 1 --2- 172.21.15.1:0/751537708 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55df60d37010 0x55df60d42290 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7ff07809d160 tx=0x7ff078010a90 comp rx=0 tx=0).stop 2024-02-02T15:46:10.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.078+0000 7ff0a0878b80 1 -- 172.21.15.1:0/751537708 shutdown_connections 2024-02-02T15:46:10.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.078+0000 7ff0a0878b80 1 --2- 172.21.15.1:0/751537708 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55df60d51e50 0x55df60d54270 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:10.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.078+0000 7ff0a0878b80 1 --2- 172.21.15.1:0/751537708 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55df60d427e0 0x55df60d3fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:10.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.078+0000 7ff0a0878b80 1 --2- 172.21.15.1:0/751537708 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55df60d37010 0x55df60d42290 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:10.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.078+0000 7ff0a0878b80 1 --2- 172.21.15.1:0/751537708 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df60d37930 0x55df60d3f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:10.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.078+0000 7ff0a0878b80 1 -- 172.21.15.1:0/751537708 >> 172.21.15.1:0/751537708 conn(0x55df60c96db0 msgr2=0x55df60d32fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:10.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.078+0000 7ff06fe00700 0 client.20211 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:46:10.079 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.079+0000 7ff0a0878b80 1 -- 172.21.15.1:0/751537708 shutdown_connections 2024-02-02T15:46:10.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.079+0000 7ff0a0878b80 1 -- 172.21.15.1:0/751537708 wait complete. 2024-02-02T15:46:10.080 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.080+0000 7ff0a0878b80 20 client.20211 trim_cache size 0 max 16384 2024-02-02T15:46:10.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.098+0000 7ff0a0878b80 1 Processor -- start 2024-02-02T15:46:10.098 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.098+0000 7ff0a0878b80 1 -- start start 2024-02-02T15:46:10.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.098+0000 7ff0a0878b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df60d37010 0x55df60d3fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:10.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.098+0000 7ff0a0878b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55df60d37930 0x55df60d3e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:10.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.098+0000 7ff0a0878b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55df60d427e0 0x55df60d31b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:10.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.098+0000 7ff0a0878b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df60b0c6a0 con 0x55df60d427e0 2024-02-02T15:46:10.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.098+0000 7ff0a0878b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df60d2d540 con 0x55df60d37930 2024-02-02T15:46:10.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.098+0000 7ff0a0878b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df60d34a40 con 0x55df60d37010 2024-02-02T15:46:10.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.098+0000 7ff085400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55df60d37930 0x55df60d3e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:10.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.098+0000 7ff085400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55df60d37930 0x55df60d3e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:39820/0 (socket says 172.21.15.1:39820) 2024-02-02T15:46:10.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.099+0000 7ff085400700 1 -- 172.21.15.1:0/2522350841 learned_addr learned my addr 172.21.15.1:0/2522350841 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:46:10.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.099+0000 7ff085e00700 1 --2- 172.21.15.1:0/2522350841 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df60d37010 0x55df60d3fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:10.100 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.099+0000 7ff086800700 1 --2- 172.21.15.1:0/2522350841 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55df60d427e0 0x55df60d31b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:10.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.099+0000 7ff085e00700 1 -- 172.21.15.1:0/2522350841 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55df60d37930 msgr2=0x55df60d3e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:10.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.099+0000 7ff085e00700 1 --2- 172.21.15.1:0/2522350841 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55df60d37930 0x55df60d3e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:10.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.099+0000 7ff085e00700 1 -- 172.21.15.1:0/2522350841 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55df60d427e0 msgr2=0x55df60d31b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:10.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.099+0000 7ff085e00700 1 --2- 172.21.15.1:0/2522350841 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55df60d427e0 0x55df60d31b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:10.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.099+0000 7ff085e00700 1 -- 172.21.15.1:0/2522350841 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55df60c4eea0 con 0x55df60d37010 2024-02-02T15:46:10.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.099+0000 7ff086800700 1 --2- 172.21.15.1:0/2522350841 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55df60d427e0 0x55df60d31b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:46:10.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.099+0000 7ff085e00700 1 --2- 172.21.15.1:0/2522350841 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df60d37010 0x55df60d3fba0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7ff0780990c0 tx=0x7ff078010c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c65aa37e37e0185f server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:10.101 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.100+0000 7ff077e00700 1 -- 172.21.15.1:0/2522350841 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff078082ea0 con 0x55df60d37010 2024-02-02T15:46:10.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.100+0000 7ff077e00700 1 -- 172.21.15.1:0/2522350841 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff078099190 con 0x55df60d37010 2024-02-02T15:46:10.102 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.100+0000 7ff077e00700 1 -- 172.21.15.1:0/2522350841 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff078098320 con 0x55df60d37010 2024-02-02T15:46:10.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.100+0000 7ff0a0878b80 1 -- 172.21.15.1:0/2522350841 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df60d37010 msgr2=0x55df60d3fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:10.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.100+0000 7ff0a0878b80 1 --2- 172.21.15.1:0/2522350841 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df60d37010 0x55df60d3fba0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7ff0780990c0 tx=0x7ff078010c80 comp rx=0 tx=0).stop 2024-02-02T15:46:10.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.101+0000 7ff0a0878b80 1 -- 172.21.15.1:0/2522350841 shutdown_connections 2024-02-02T15:46:10.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.101+0000 7ff0a0878b80 1 --2- 172.21.15.1:0/2522350841 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55df60d427e0 0x55df60d31b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:10.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.101+0000 7ff0a0878b80 1 --2- 172.21.15.1:0/2522350841 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df60d37010 0x55df60d3fba0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:10.103 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.101+0000 7ff0a0878b80 1 --2- 172.21.15.1:0/2522350841 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55df60d37930 0x55df60d3e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:10.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.101+0000 7ff0a0878b80 1 -- 172.21.15.1:0/2522350841 >> 172.21.15.1:0/2522350841 conn(0x55df60c96db0 msgr2=0x55df60d2ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:10.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.101+0000 7ff0a0878b80 1 -- 172.21.15.1:0/2522350841 shutdown_connections 2024-02-02T15:46:10.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.101+0000 7ff0a0878b80 1 -- 172.21.15.1:0/2522350841 wait complete. 2024-02-02T15:46:10.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.101+0000 7ff0a0878b80 1 -- start start 2024-02-02T15:46:10.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.102+0000 7ff0a0878b80 1 Processor -- start 2024-02-02T15:46:10.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.102+0000 7ff0a0878b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55df60d37010 0x55df60d3cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:10.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.102+0000 7ff0a0878b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df60d37930 0x55df60d47310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:10.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.102+0000 7ff0a0878b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55df60d427e0 0x55df60d47850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:10.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.102+0000 7ff0a0878b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df60d47d90 con 0x55df60d427e0 2024-02-02T15:46:10.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.102+0000 7ff0a0878b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df60d47f00 con 0x55df60d37010 2024-02-02T15:46:10.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.102+0000 7ff0a0878b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55df60d480c0 con 0x55df60d37930 2024-02-02T15:46:10.104 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.103+0000 7ff085e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55df60d37010 0x55df60d3cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:10.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.103+0000 7ff085e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55df60d37010 0x55df60d3cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:39824/0 (socket says 172.21.15.1:39824) 2024-02-02T15:46:10.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.103+0000 7ff085e00700 1 -- 172.21.15.1:0/1221141227 learned_addr learned my addr 172.21.15.1:0/1221141227 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:46:10.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.103+0000 7ff0a0878b80 10 client.0 _ll_register_callbacks cb 0x55df60c26290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:46:10.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.103+0000 7ff086800700 1 --2- 172.21.15.1:0/1221141227 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55df60d427e0 0x55df60d47850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:10.105 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.103+0000 7ff085400700 1 --2- 172.21.15.1:0/1221141227 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df60d37930 0x55df60d47310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:10.106 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.103+0000 7ff086800700 1 -- 172.21.15.1:0/1221141227 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df60d37930 msgr2=0x55df60d47310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:10.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.103+0000 7ff086800700 1 --2- 172.21.15.1:0/1221141227 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df60d37930 0x55df60d47310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:10.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.103+0000 7ff086800700 1 -- 172.21.15.1:0/1221141227 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55df60d37010 msgr2=0x55df60d3cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:10.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.103+0000 7ff086800700 1 --2- 172.21.15.1:0/1221141227 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55df60d37010 0x55df60d3cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:10.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.103+0000 7ff086800700 1 -- 172.21.15.1:0/1221141227 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55df60ab6b60 con 0x55df60d427e0 2024-02-02T15:46:10.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.103+0000 7ff085400700 1 --2- 172.21.15.1:0/1221141227 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df60d37930 0x55df60d47310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:46:10.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.104+0000 7ff086800700 1 --2- 172.21.15.1:0/1221141227 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55df60d427e0 0x55df60d47850 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7ff080007b40 tx=0x7ff080007b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaa1c42f1a2a8565 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:10.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.104+0000 7ff06fe00700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:46:10.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.104+0000 7ff06fe00700 1 -- 172.21.15.1:0/1221141227 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff08000e040 con 0x55df60d427e0 2024-02-02T15:46:10.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.105+0000 7ff06fe00700 1 -- 172.21.15.1:0/1221141227 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 9541+0+0 (secure 0 0 0) 0x7ff080004bb0 con 0x55df60d427e0 2024-02-02T15:46:10.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.105+0000 7ff0a0878b80 1 -- 172.21.15.1:0/1221141227 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff0780a1430 con 0x55df60d427e0 2024-02-02T15:46:10.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.105+0000 7ff06fe00700 1 -- 172.21.15.1:0/1221141227 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff080004d70 con 0x55df60d427e0 2024-02-02T15:46:10.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.105+0000 7ff0a0878b80 10 client.20217 fetch_fsmap learned FSMap version 78 2024-02-02T15:46:10.107 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.105+0000 7ff0a0878b80 1 -- 172.21.15.1:0/1221141227 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) v3 -- 0x55df60b0ecd0 con 0x55df60d427e0 2024-02-02T15:46:10.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.105+0000 7ff06fe00700 1 -- 172.21.15.1:0/1221141227 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff0780a1430 con 0x55df60d427e0 2024-02-02T15:46:10.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.106+0000 7ff06fe00700 1 -- 172.21.15.1:0/1221141227 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 78) v1 ==== 7506+0+0 (secure 0 0 0) 0x7ff080004400 con 0x55df60d427e0 2024-02-02T15:46:10.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.106+0000 7ff0a0878b80 10 client.20217 fetch_fsmap finished waiting for FSMap version 78 2024-02-02T15:46:10.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.106+0000 7ff0a0878b80 10 client.20217 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:46:10.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.106+0000 7ff0a0878b80 20 client.20217 populate_metadata read hostname 'smithi001' 2024-02-02T15:46:10.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.106+0000 7ff0a0878b80 1 --2- 172.21.15.1:0/1221141227 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55df60dac6e0 0x55df60daeb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:10.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.106+0000 7ff0a0878b80 4 client.20217 mds_command: new command op to 24557 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:46:10.108 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.106+0000 7ff0a0878b80 1 -- 172.21.15.1:0/1221141227 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55df60c47f50 con 0x55df60dac6e0 2024-02-02T15:46:10.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.108+0000 7ff085e00700 1 --2- 172.21.15.1:0/1221141227 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55df60dac6e0 0x55df60daeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:10.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.108+0000 7ff085e00700 1 --2- 172.21.15.1:0/1221141227 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55df60dac6e0 0x55df60daeb00 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca84aa74ed96850b server_cookie=e755467b01bb0ffb in_seq=0 out_seq=0 2024-02-02T15:46:10.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.108+0000 7ff06fe00700 10 client.20217 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:46:10.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.109+0000 7ff06fe00700 1 -- 172.21.15.1:0/1221141227 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55df60c47f50 con 0x55df60dac6e0 2024-02-02T15:46:10.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.109+0000 7ff06fe00700 10 client.20217 handle_command_reply: tid=0 2024-02-02T15:46:10.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.109+0000 7ff0a0878b80 1 client.20217 shutdown 2024-02-02T15:46:10.110 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.109+0000 7ff0a0878b80 1 -- 172.21.15.1:0/1221141227 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55df60d427e0 msgr2=0x55df60d47850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:10.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.109+0000 7ff0a0878b80 1 --2- 172.21.15.1:0/1221141227 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55df60d427e0 0x55df60d47850 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7ff080007b40 tx=0x7ff080007b70 comp rx=0 tx=0).stop 2024-02-02T15:46:10.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.109+0000 7ff0a0878b80 1 -- 172.21.15.1:0/1221141227 shutdown_connections 2024-02-02T15:46:10.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.109+0000 7ff0a0878b80 1 --2- 172.21.15.1:0/1221141227 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55df60dac6e0 0x55df60daeb00 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:10.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.109+0000 7ff0a0878b80 1 --2- 172.21.15.1:0/1221141227 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55df60d427e0 0x55df60d47850 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:10.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.109+0000 7ff0a0878b80 1 --2- 172.21.15.1:0/1221141227 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55df60d37010 0x55df60d3cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:10.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.109+0000 7ff0a0878b80 1 --2- 172.21.15.1:0/1221141227 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55df60d37930 0x55df60d47310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:10.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.109+0000 7ff0a0878b80 1 -- 172.21.15.1:0/1221141227 >> 172.21.15.1:0/1221141227 conn(0x55df60c96db0 msgr2=0x55df60d4e520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:10.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.110+0000 7ff06fe00700 0 client.20217 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:46:10.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.110+0000 7ff0a0878b80 1 -- 172.21.15.1:0/1221141227 shutdown_connections 2024-02-02T15:46:10.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.110+0000 7ff0a0878b80 1 -- 172.21.15.1:0/1221141227 wait complete. 2024-02-02T15:46:10.111 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.110+0000 7ff0a0878b80 20 client.20217 trim_cache size 0 max 16384 2024-02-02T15:46:10.112 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:46:10.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.111+0000 7ff08ce00700 1 -- 172.21.15.1:0/3031678007 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff0680a0710 msgr2=0x7ff0680a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:10.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.111+0000 7ff08ce00700 1 --2- 172.21.15.1:0/3031678007 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff0680a0710 0x7ff0680a2bd0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff08819a6c0 tx=0x7ff07000b610 comp rx=0 tx=0).stop 2024-02-02T15:46:10.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.111+0000 7ff08ce00700 1 -- 172.21.15.1:0/3031678007 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0880730f0 msgr2=0x7ff0881990b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:10.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.111+0000 7ff08ce00700 1 --2- 172.21.15.1:0/3031678007 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0880730f0 0x7ff0881990b0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7ff07800e9a0 tx=0x7ff07800ecb0 comp rx=0 tx=0).stop 2024-02-02T15:46:10.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.111+0000 7ff08ce00700 1 -- 172.21.15.1:0/3031678007 shutdown_connections 2024-02-02T15:46:10.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.111+0000 7ff08ce00700 1 --2- 172.21.15.1:0/3031678007 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff0680a0710 0x7ff0680a2bd0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:10.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.111+0000 7ff08ce00700 1 --2- 172.21.15.1:0/3031678007 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff08810d650 0x7ff08819da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:10.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.111+0000 7ff08ce00700 1 --2- 172.21.15.1:0/3031678007 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff0880730f0 0x7ff0881990b0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:10.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.111+0000 7ff08ce00700 1 --2- 172.21.15.1:0/3031678007 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff088073a10 0x7ff0881995f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:10.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.111+0000 7ff08ce00700 1 -- 172.21.15.1:0/3031678007 >> 172.21.15.1:0/3031678007 conn(0x7ff0880fc840 msgr2=0x7ff0881050b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:10.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.112+0000 7ff08ce00700 1 -- 172.21.15.1:0/3031678007 shutdown_connections 2024-02-02T15:46:10.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:10.112+0000 7ff08ce00700 1 -- 172.21.15.1:0/3031678007 wait complete. 2024-02-02T15:46:11.011 INFO:tasks.workunit.client.0.smithi001.stdout: 10 179027 26.45 MB/sec execute 223 sec latency 67.300 ms 2024-02-02T15:46:11.137 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-02-02T15:46:11.137 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-02-02T15:46:11.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.493+0000 7f9205200700 1 -- 172.21.15.1:0/3581281537 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9200100030 msgr2=0x7f9200111540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:11.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.493+0000 7f9205200700 1 --2- 172.21.15.1:0/3581281537 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9200100030 0x7f9200111540 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f91e80098f0 tx=0x7f91e8009c00 comp rx=0 tx=0).stop 2024-02-02T15:46:11.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.493+0000 7f9205200700 1 -- 172.21.15.1:0/3581281537 shutdown_connections 2024-02-02T15:46:11.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.493+0000 7f9205200700 1 --2- 172.21.15.1:0/3581281537 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9200111a80 0x7f9200113e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:11.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.493+0000 7f9205200700 1 --2- 172.21.15.1:0/3581281537 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f92000ff680 0x7f92000ffa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:11.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.493+0000 7f9205200700 1 --2- 172.21.15.1:0/3581281537 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9200100030 0x7f9200111540 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:11.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.493+0000 7f9205200700 1 -- 172.21.15.1:0/3581281537 >> 172.21.15.1:0/3581281537 conn(0x7f9200074f30 msgr2=0x7f9200075340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:11.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.494+0000 7f9205200700 1 -- 172.21.15.1:0/3581281537 shutdown_connections 2024-02-02T15:46:11.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.494+0000 7f9205200700 1 -- 172.21.15.1:0/3581281537 wait complete. 2024-02-02T15:46:11.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.496+0000 7f9205200700 1 Processor -- start 2024-02-02T15:46:11.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.497+0000 7f9205200700 1 -- start start 2024-02-02T15:46:11.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.497+0000 7f9205200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92000ff680 0x7f920019d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:11.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.497+0000 7f9205200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9200100030 0x7f920019da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:11.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.497+0000 7f9205200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9200111a80 0x7f92001a1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:11.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.498+0000 7f9205200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92001a24f0 con 0x7f9200100030 2024-02-02T15:46:11.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.498+0000 7f9205200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92001a2660 con 0x7f9200111a80 2024-02-02T15:46:11.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.498+0000 7f9205200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92001a2820 con 0x7f92000ff680 2024-02-02T15:46:11.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.498+0000 7f91fde00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92000ff680 0x7f920019d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:11.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.498+0000 7f91fe800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9200111a80 0x7f92001a1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:11.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.498+0000 7f91fde00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92000ff680 0x7f920019d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:44860/0 (socket says 172.21.15.1:44860) 2024-02-02T15:46:11.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.498+0000 7f91fde00700 1 -- 172.21.15.1:0/648387549 learned_addr learned my addr 172.21.15.1:0/648387549 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:46:11.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.498+0000 7f91fd400700 1 --2- 172.21.15.1:0/648387549 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9200100030 0x7f920019da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:11.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.498+0000 7f91fe800700 1 -- 172.21.15.1:0/648387549 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92000ff680 msgr2=0x7f920019d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:11.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.498+0000 7f91fe800700 1 --2- 172.21.15.1:0/648387549 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92000ff680 0x7f920019d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:11.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.498+0000 7f91fe800700 1 -- 172.21.15.1:0/648387549 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9200100030 msgr2=0x7f920019da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:11.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.498+0000 7f91fe800700 1 --2- 172.21.15.1:0/648387549 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9200100030 0x7f920019da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:11.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.498+0000 7f91fe800700 1 -- 172.21.15.1:0/648387549 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91e8009520 con 0x7f9200111a80 2024-02-02T15:46:11.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.499+0000 7f91fe800700 1 --2- 172.21.15.1:0/648387549 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9200111a80 0x7f92001a1eb0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f91f0005fd0 tx=0x7f91f000ec20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:11.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.499+0000 7f91f6a00700 1 -- 172.21.15.1:0/648387549 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f0004a30 con 0x7f9200111a80 2024-02-02T15:46:11.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.499+0000 7f91f6a00700 1 -- 172.21.15.1:0/648387549 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f91f00054e0 con 0x7f9200111a80 2024-02-02T15:46:11.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.499+0000 7f91f6a00700 1 -- 172.21.15.1:0/648387549 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91f0010be0 con 0x7f9200111a80 2024-02-02T15:46:11.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.499+0000 7f9205200700 1 -- 172.21.15.1:0/648387549 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92001a2b00 con 0x7f9200111a80 2024-02-02T15:46:11.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.500+0000 7f9205200700 1 -- 172.21.15.1:0/648387549 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92001aa340 con 0x7f9200111a80 2024-02-02T15:46:11.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.506+0000 7f91f6a00700 1 -- 172.21.15.1:0/648387549 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f91f0004b90 con 0x7f9200111a80 2024-02-02T15:46:11.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.507+0000 7f9218daab80 1 Processor -- start 2024-02-02T15:46:11.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.507+0000 7f9218daab80 1 -- start start 2024-02-02T15:46:11.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.508+0000 7f9218daab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ebd89c090 0x564ebd89c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:11.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.508+0000 7f9218daab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ebd89c9b0 0x564ebd8a7320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:11.508 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.508+0000 7f9218daab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ebd8a7860 0x564ebd8a9c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:11.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.508+0000 7f9218daab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ebd810540 con 0x564ebd89c090 2024-02-02T15:46:11.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.508+0000 7f9218daab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ebd7f4b20 con 0x564ebd89c9b0 2024-02-02T15:46:11.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.508+0000 7f9218daab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ebd7f24e0 con 0x564ebd8a7860 2024-02-02T15:46:11.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.508+0000 7f91fd400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ebd8a7860 0x564ebd8a9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:11.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.509+0000 7f91fd400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ebd8a7860 0x564ebd8a9c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:44866/0 (socket says 172.21.15.1:44866) 2024-02-02T15:46:11.509 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.509+0000 7f91fd400700 1 -- 172.21.15.1:0/2918589484 learned_addr learned my addr 172.21.15.1:0/2918589484 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:46:11.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.509+0000 7f91fd400700 1 --2- 172.21.15.1:0/2918589484 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ebd89c090 0x564ebd89c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:11.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.510+0000 7f91fde00700 1 --2- 172.21.15.1:0/2918589484 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ebd89c9b0 0x564ebd8a7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:11.510 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.510+0000 7f91f6a00700 1 --2- 172.21.15.1:0/648387549 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f91e00a0790 0x7f91e00a2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:11.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.510+0000 7f91f6a00700 1 -- 172.21.15.1:0/648387549 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f91f0014070 con 0x7f9200111a80 2024-02-02T15:46:11.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.510+0000 7f91fd400700 1 -- 172.21.15.1:0/2918589484 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ebd89c9b0 msgr2=0x564ebd8a7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:11.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.510+0000 7f91fd400700 1 --2- 172.21.15.1:0/2918589484 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ebd89c9b0 0x564ebd8a7320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:11.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.510+0000 7f91fd400700 1 -- 172.21.15.1:0/2918589484 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ebd89c090 msgr2=0x564ebd89c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:11.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.510+0000 7f91fd400700 1 --2- 172.21.15.1:0/2918589484 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ebd89c090 0x564ebd89c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:11.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.510+0000 7f91fd400700 1 -- 172.21.15.1:0/2918589484 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564ebd66f430 con 0x564ebd8a7860 2024-02-02T15:46:11.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.512+0000 7f91fe800700 1 --2- 172.21.15.1:0/648387549 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f91e00a0790 0x7f91e00a2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:11.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.513+0000 7f91fd400700 1 --2- 172.21.15.1:0/2918589484 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ebd8a7860 0x564ebd8a9c50 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f91e8016000 tx=0x7f91e8019040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ecd270d4a4951ffb server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:11.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.514+0000 7f91fe800700 1 --2- 172.21.15.1:0/648387549 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f91e00a0790 0x7f91e00a2c50 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f91f0009740 tx=0x7f91f000ef70 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:11.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.514+0000 7f91ef400700 1 -- 172.21.15.1:0/2918589484 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e800f490 con 0x564ebd8a7860 2024-02-02T15:46:11.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.514+0000 7f91ef400700 1 -- 172.21.15.1:0/2918589484 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f91e800b560 con 0x564ebd8a7860 2024-02-02T15:46:11.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.514+0000 7f91ef400700 1 -- 172.21.15.1:0/2918589484 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e8005310 con 0x564ebd8a7860 2024-02-02T15:46:11.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.515+0000 7f9218daab80 1 -- 172.21.15.1:0/2918589484 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ebd8a7860 msgr2=0x564ebd8a9c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:11.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.515+0000 7f9218daab80 1 --2- 172.21.15.1:0/2918589484 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ebd8a7860 0x564ebd8a9c50 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f91e8016000 tx=0x7f91e8019040 comp rx=0 tx=0).stop 2024-02-02T15:46:11.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.516+0000 7f9218daab80 1 -- 172.21.15.1:0/2918589484 shutdown_connections 2024-02-02T15:46:11.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.516+0000 7f9218daab80 1 --2- 172.21.15.1:0/2918589484 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ebd8a7860 0x564ebd8a9c50 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:11.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.516+0000 7f9218daab80 1 --2- 172.21.15.1:0/2918589484 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ebd89c090 0x564ebd89c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:11.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.516+0000 7f9218daab80 1 --2- 172.21.15.1:0/2918589484 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ebd89c9b0 0x564ebd8a7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:11.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.516+0000 7f9218daab80 1 -- 172.21.15.1:0/2918589484 >> 172.21.15.1:0/2918589484 conn(0x564ebd7fbe30 msgr2=0x564ebd898430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:11.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.516+0000 7f9218daab80 1 -- 172.21.15.1:0/2918589484 shutdown_connections 2024-02-02T15:46:11.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.516+0000 7f9218daab80 1 -- 172.21.15.1:0/2918589484 wait complete. 2024-02-02T15:46:11.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.517+0000 7f9218daab80 1 -- start start 2024-02-02T15:46:11.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.517+0000 7f9218daab80 1 Processor -- start 2024-02-02T15:46:11.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.518+0000 7f9218daab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ebd89c090 0x564ebd8a7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:11.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.518+0000 7f9218daab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ebd89c9b0 0x564ebd8a4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:11.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.518+0000 7f9218daab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ebd8a7860 0x564ebd8a4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:11.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.518+0000 7f9218daab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ebd7eddd0 con 0x564ebd8a7860 2024-02-02T15:46:11.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.518+0000 7f9218daab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ebd789e00 con 0x564ebd89c9b0 2024-02-02T15:46:11.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.518+0000 7f9218daab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ebd789aa0 con 0x564ebd89c090 2024-02-02T15:46:11.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.518+0000 7f9218daab80 10 client.0 _ll_register_callbacks cb 0x564ebd78b310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:46:11.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.518+0000 7f91fd400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ebd8a7860 0x564ebd8a4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:11.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.518+0000 7f91fd400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ebd8a7860 0x564ebd8a4a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:34804/0 (socket says 172.21.15.1:34804) 2024-02-02T15:46:11.520 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.518+0000 7f91fd400700 1 -- 172.21.15.1:0/694003620 learned_addr learned my addr 172.21.15.1:0/694003620 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:46:11.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.519+0000 7f91fde00700 1 --2- 172.21.15.1:0/694003620 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ebd89c9b0 0x564ebd8a4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:11.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.520+0000 7f91fd400700 1 -- 172.21.15.1:0/694003620 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ebd89c090 msgr2=0x564ebd8a7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:11.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.520+0000 7f91fd400700 1 --2- 172.21.15.1:0/694003620 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ebd89c090 0x564ebd8a7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:11.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.520+0000 7f91fd400700 1 -- 172.21.15.1:0/694003620 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ebd89c9b0 msgr2=0x564ebd8a4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:11.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.520+0000 7f91fd400700 1 --2- 172.21.15.1:0/694003620 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ebd89c9b0 0x564ebd8a4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:11.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.520+0000 7f91fd400700 1 -- 172.21.15.1:0/694003620 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91e801f040 con 0x564ebd8a7860 2024-02-02T15:46:11.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.521+0000 7f91fd400700 1 --2- 172.21.15.1:0/694003620 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ebd8a7860 0x564ebd8a4a80 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f91e80098f0 tx=0x7f91e8005940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee8ac912f9069f50 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:11.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.521+0000 7f91e7e00700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:46:11.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.521+0000 7f91e7e00700 1 -- 172.21.15.1:0/694003620 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e8005250 con 0x564ebd8a7860 2024-02-02T15:46:11.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.522+0000 7f91e7e00700 1 -- 172.21.15.1:0/694003620 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f91e800f490 con 0x564ebd8a7860 2024-02-02T15:46:11.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.522+0000 7f9218daab80 1 -- 172.21.15.1:0/694003620 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564ebd890170 con 0x564ebd8a7860 2024-02-02T15:46:11.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.522+0000 7f91e7e00700 1 -- 172.21.15.1:0/694003620 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7f91e800f650 con 0x564ebd8a7860 2024-02-02T15:46:11.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.522+0000 7f9218daab80 10 client.20229 fetch_fsmap learned FSMap version 78 2024-02-02T15:46:11.522 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.522+0000 7f9218daab80 1 -- 172.21.15.1:0/694003620 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) v3 -- 0x564ebd61bbe0 con 0x564ebd8a7860 2024-02-02T15:46:11.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.523+0000 7f91e7e00700 1 -- 172.21.15.1:0/694003620 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564ebd890170 con 0x564ebd8a7860 2024-02-02T15:46:11.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.523+0000 7f91e7e00700 1 -- 172.21.15.1:0/694003620 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 78) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f91e8015020 con 0x564ebd8a7860 2024-02-02T15:46:11.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.523+0000 7f9218daab80 10 client.20229 fetch_fsmap finished waiting for FSMap version 78 2024-02-02T15:46:11.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.523+0000 7f9218daab80 10 client.20229 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:46:11.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.523+0000 7f9218daab80 20 client.20229 populate_metadata read hostname 'smithi001' 2024-02-02T15:46:11.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.523+0000 7f9218daab80 1 --2- 172.21.15.1:0/694003620 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x564ebd8c7d90 0x564ebd8ca1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:11.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.523+0000 7f9218daab80 4 client.20229 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:46:11.523 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.523+0000 7f9218daab80 1 -- 172.21.15.1:0/694003620 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564ebd7e6e50 con 0x564ebd8c7d90 2024-02-02T15:46:11.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.524+0000 7f91fde00700 1 --2- 172.21.15.1:0/694003620 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x564ebd8c7d90 0x564ebd8ca1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:11.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.524+0000 7f91fde00700 1 --2- 172.21.15.1:0/694003620 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x564ebd8c7d90 0x564ebd8ca1b0 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f891134616e9154 server_cookie=e74e82148945d6d3 in_seq=0 out_seq=0 2024-02-02T15:46:11.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.525+0000 7f91e7e00700 10 client.20229 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:46:11.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.528+0000 7f91e7e00700 1 -- 172.21.15.1:0/694003620 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x564ebd7e6e50 con 0x564ebd8c7d90 2024-02-02T15:46:11.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.528+0000 7f91e7e00700 10 client.20229 handle_command_reply: tid=0 2024-02-02T15:46:11.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.528+0000 7f9218daab80 1 client.20229 shutdown 2024-02-02T15:46:11.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.528+0000 7f9218daab80 1 -- 172.21.15.1:0/694003620 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ebd8a7860 msgr2=0x564ebd8a4a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:11.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.528+0000 7f9218daab80 1 --2- 172.21.15.1:0/694003620 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ebd8a7860 0x564ebd8a4a80 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f91e80098f0 tx=0x7f91e8005940 comp rx=0 tx=0).stop 2024-02-02T15:46:11.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.529+0000 7f9218daab80 1 -- 172.21.15.1:0/694003620 shutdown_connections 2024-02-02T15:46:11.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.529+0000 7f9218daab80 1 --2- 172.21.15.1:0/694003620 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x564ebd8c7d90 0x564ebd8ca1b0 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:11.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.529+0000 7f9218daab80 1 --2- 172.21.15.1:0/694003620 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ebd8a7860 0x564ebd8a4a80 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:11.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.529+0000 7f9218daab80 1 --2- 172.21.15.1:0/694003620 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ebd89c090 0x564ebd8a7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:11.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.529+0000 7f9218daab80 1 --2- 172.21.15.1:0/694003620 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ebd89c9b0 0x564ebd8a4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:11.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.529+0000 7f9218daab80 1 -- 172.21.15.1:0/694003620 >> 172.21.15.1:0/694003620 conn(0x564ebd7fbe30 msgr2=0x564ebd898020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:11.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.529+0000 7f91e7e00700 0 client.20229 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:46:11.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.529+0000 7f9218daab80 1 -- 172.21.15.1:0/694003620 shutdown_connections 2024-02-02T15:46:11.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.529+0000 7f9218daab80 1 -- 172.21.15.1:0/694003620 wait complete. 2024-02-02T15:46:11.530 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.529+0000 7f9218daab80 20 client.20229 trim_cache size 0 max 16384 2024-02-02T15:46:11.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.554+0000 7f9218daab80 1 Processor -- start 2024-02-02T15:46:11.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.554+0000 7f9218daab80 1 -- start start 2024-02-02T15:46:11.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.554+0000 7f9218daab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ebd89c090 0x564ebd8a3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:11.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.554+0000 7f9218daab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ebd89c9b0 0x564ebd8a6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:11.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.554+0000 7f9218daab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ebd8a7860 0x564ebd8a8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:11.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.554+0000 7f9218daab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ebd671720 con 0x564ebd89c9b0 2024-02-02T15:46:11.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.554+0000 7f9218daab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ebd892580 con 0x564ebd8a7860 2024-02-02T15:46:11.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.554+0000 7f9218daab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ebd8a02c0 con 0x564ebd89c090 2024-02-02T15:46:11.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.555+0000 7f91fd400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ebd89c090 0x564ebd8a3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:11.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.555+0000 7f91fd400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ebd89c090 0x564ebd8a3330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:44884/0 (socket says 172.21.15.1:44884) 2024-02-02T15:46:11.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.555+0000 7f91fd400700 1 -- 172.21.15.1:0/298530477 learned_addr learned my addr 172.21.15.1:0/298530477 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:46:11.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.555+0000 7f91fd400700 1 --2- 172.21.15.1:0/298530477 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ebd8a7860 0x564ebd8a8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:11.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.555+0000 7f91fde00700 1 --2- 172.21.15.1:0/298530477 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ebd89c9b0 0x564ebd8a6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:11.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.555+0000 7f91fd400700 1 -- 172.21.15.1:0/298530477 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ebd8a7860 msgr2=0x564ebd8a8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:11.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.555+0000 7f91fd400700 1 --2- 172.21.15.1:0/298530477 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ebd8a7860 0x564ebd8a8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:11.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.555+0000 7f91fd400700 1 -- 172.21.15.1:0/298530477 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ebd89c9b0 msgr2=0x564ebd8a6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:11.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.555+0000 7f91fd400700 1 --2- 172.21.15.1:0/298530477 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ebd89c9b0 0x564ebd8a6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:11.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.555+0000 7f91fd400700 1 -- 172.21.15.1:0/298530477 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564ebd7b3f20 con 0x564ebd89c090 2024-02-02T15:46:11.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.555+0000 7f91fd400700 1 --2- 172.21.15.1:0/298530477 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ebd89c090 0x564ebd8a3330 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f91e8005850 tx=0x7f91e8005e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=688a13915feed009 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:11.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.557+0000 7f91efe00700 1 -- 172.21.15.1:0/298530477 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e8005250 con 0x564ebd89c090 2024-02-02T15:46:11.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.557+0000 7f91efe00700 1 -- 172.21.15.1:0/298530477 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f91e800f920 con 0x564ebd89c090 2024-02-02T15:46:11.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.557+0000 7f91efe00700 1 -- 172.21.15.1:0/298530477 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e800ba00 con 0x564ebd89c090 2024-02-02T15:46:11.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.558+0000 7f9218daab80 1 -- 172.21.15.1:0/298530477 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ebd89c090 msgr2=0x564ebd8a3330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:11.558 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.558+0000 7f9218daab80 1 --2- 172.21.15.1:0/298530477 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ebd89c090 0x564ebd8a3330 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f91e8005850 tx=0x7f91e8005e50 comp rx=0 tx=0).stop 2024-02-02T15:46:11.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.559+0000 7f9218daab80 1 -- 172.21.15.1:0/298530477 shutdown_connections 2024-02-02T15:46:11.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.559+0000 7f9218daab80 1 --2- 172.21.15.1:0/298530477 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ebd8a7860 0x564ebd8a8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:11.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.559+0000 7f9218daab80 1 --2- 172.21.15.1:0/298530477 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ebd89c090 0x564ebd8a3330 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:11.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.559+0000 7f9218daab80 1 --2- 172.21.15.1:0/298530477 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ebd89c9b0 0x564ebd8a6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:11.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.559+0000 7f9218daab80 1 -- 172.21.15.1:0/298530477 >> 172.21.15.1:0/298530477 conn(0x564ebd7fbe30 msgr2=0x564ebd8a2c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:11.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.559+0000 7f9218daab80 1 -- 172.21.15.1:0/298530477 shutdown_connections 2024-02-02T15:46:11.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.559+0000 7f9218daab80 1 -- 172.21.15.1:0/298530477 wait complete. 2024-02-02T15:46:11.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.560+0000 7f9218daab80 1 -- start start 2024-02-02T15:46:11.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.561+0000 7f9218daab80 1 Processor -- start 2024-02-02T15:46:11.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.563+0000 7f9218daab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ebd89c090 0x564ebd8bf130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:11.564 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.563+0000 7f9218daab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ebd89c9b0 0x564ebd9168a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:11.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.563+0000 7f9218daab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ebd8a7860 0x564ebd916de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:11.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.563+0000 7f9218daab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ebd917480 con 0x564ebd89c090 2024-02-02T15:46:11.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.563+0000 7f9218daab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ebd9175f0 con 0x564ebd8a7860 2024-02-02T15:46:11.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.563+0000 7f9218daab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564ebd8bc4c0 con 0x564ebd89c9b0 2024-02-02T15:46:11.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.563+0000 7f9218daab80 10 client.0 _ll_register_callbacks cb 0x564ebd78b310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:46:11.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.564+0000 7f91fd400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ebd8a7860 0x564ebd916de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:11.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.564+0000 7f91fd400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ebd8a7860 0x564ebd916de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:39866/0 (socket says 172.21.15.1:39866) 2024-02-02T15:46:11.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.564+0000 7f91fd400700 1 -- 172.21.15.1:0/682950705 learned_addr learned my addr 172.21.15.1:0/682950705 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:46:11.565 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.565+0000 7f91fd400700 1 --2- 172.21.15.1:0/682950705 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ebd89c090 0x564ebd8bf130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:11.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.565+0000 7f91fde00700 1 --2- 172.21.15.1:0/682950705 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ebd89c9b0 0x564ebd9168a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:11.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.565+0000 7f91fd400700 1 -- 172.21.15.1:0/682950705 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ebd89c9b0 msgr2=0x564ebd9168a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:11.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.565+0000 7f91fd400700 1 --2- 172.21.15.1:0/682950705 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ebd89c9b0 0x564ebd9168a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:11.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.565+0000 7f91fd400700 1 -- 172.21.15.1:0/682950705 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ebd89c090 msgr2=0x564ebd8bf130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:11.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.565+0000 7f91fd400700 1 --2- 172.21.15.1:0/682950705 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ebd89c090 0x564ebd8bf130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:11.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.565+0000 7f91fd400700 1 -- 172.21.15.1:0/682950705 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564ebd61bbe0 con 0x564ebd8a7860 2024-02-02T15:46:11.566 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.565+0000 7f91fd400700 1 --2- 172.21.15.1:0/682950705 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ebd8a7860 0x564ebd916de0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f91e800b390 tx=0x7f91e800b7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a3aad9bfbce55d5 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:11.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.567+0000 7f91e7e00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:46:11.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.567+0000 7f91e7e00700 1 -- 172.21.15.1:0/682950705 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e8005920 con 0x564ebd8a7860 2024-02-02T15:46:11.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.567+0000 7f91e7e00700 1 -- 172.21.15.1:0/682950705 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f91e8000e40 con 0x564ebd8a7860 2024-02-02T15:46:11.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.567+0000 7f9218daab80 1 -- 172.21.15.1:0/682950705 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f91e800b560 con 0x564ebd8a7860 2024-02-02T15:46:11.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.567+0000 7f91e7e00700 1 -- 172.21.15.1:0/682950705 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7f91e800b880 con 0x564ebd8a7860 2024-02-02T15:46:11.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.567+0000 7f9218daab80 10 client.30539 fetch_fsmap learned FSMap version 78 2024-02-02T15:46:11.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.567+0000 7f9218daab80 1 -- 172.21.15.1:0/682950705 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) v3 -- 0x564ebd673d50 con 0x564ebd8a7860 2024-02-02T15:46:11.567 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.567+0000 7f91e7e00700 1 -- 172.21.15.1:0/682950705 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f91e800b560 con 0x564ebd8a7860 2024-02-02T15:46:11.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.567+0000 7f91e7e00700 1 -- 172.21.15.1:0/682950705 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 78) v1 ==== 7506+0+0 (secure 0 0 0) 0x7f91e8015070 con 0x564ebd8a7860 2024-02-02T15:46:11.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.568+0000 7f9218daab80 10 client.30539 fetch_fsmap finished waiting for FSMap version 78 2024-02-02T15:46:11.568 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.568+0000 7f9218daab80 10 client.30539 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:46:11.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.568+0000 7f9218daab80 20 client.30539 populate_metadata read hostname 'smithi001' 2024-02-02T15:46:11.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.568+0000 7f9218daab80 1 --2- 172.21.15.1:0/682950705 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x564ebd92ca70 0x564ebd92ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:11.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.568+0000 7f9218daab80 4 client.30539 mds_command: new command op to 24557 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-02-02T15:46:11.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.568+0000 7f9218daab80 1 -- 172.21.15.1:0/682950705 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x564ebd7acfd0 con 0x564ebd92ca70 2024-02-02T15:46:11.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.568+0000 7f91fde00700 1 --2- 172.21.15.1:0/682950705 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x564ebd92ca70 0x564ebd92ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:11.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.568+0000 7f91fde00700 1 --2- 172.21.15.1:0/682950705 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x564ebd92ca70 0x564ebd92ee60 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f743dcb9339a2e3 server_cookie=528c36559c6b97 in_seq=0 out_seq=0 2024-02-02T15:46:11.569 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.568+0000 7f91e7e00700 10 client.30539 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:46:11.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.571+0000 7f91e7e00700 1 -- 172.21.15.1:0/682950705 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x564ebd7acfd0 con 0x564ebd92ca70 2024-02-02T15:46:11.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.571+0000 7f91e7e00700 10 client.30539 handle_command_reply: tid=0 2024-02-02T15:46:11.571 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.571+0000 7f9218daab80 1 client.30539 shutdown 2024-02-02T15:46:11.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.572+0000 7f9218daab80 1 -- 172.21.15.1:0/682950705 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ebd8a7860 msgr2=0x564ebd916de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:11.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.572+0000 7f9218daab80 1 --2- 172.21.15.1:0/682950705 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ebd8a7860 0x564ebd916de0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f91e800b390 tx=0x7f91e800b7e0 comp rx=0 tx=0).stop 2024-02-02T15:46:11.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.572+0000 7f9218daab80 1 -- 172.21.15.1:0/682950705 shutdown_connections 2024-02-02T15:46:11.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.572+0000 7f9218daab80 1 --2- 172.21.15.1:0/682950705 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x564ebd92ca70 0x564ebd92ee60 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:11.572 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.572+0000 7f9218daab80 1 --2- 172.21.15.1:0/682950705 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564ebd8a7860 0x564ebd916de0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:11.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.572+0000 7f9218daab80 1 --2- 172.21.15.1:0/682950705 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564ebd89c090 0x564ebd8bf130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:11.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.572+0000 7f9218daab80 1 --2- 172.21.15.1:0/682950705 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564ebd89c9b0 0x564ebd9168a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:11.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.572+0000 7f91e7e00700 0 client.30539 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:46:11.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.573+0000 7f9218daab80 1 -- 172.21.15.1:0/682950705 >> 172.21.15.1:0/682950705 conn(0x564ebd7fbe30 msgr2=0x564ebd8a34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:11.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.573+0000 7f9218daab80 1 -- 172.21.15.1:0/682950705 shutdown_connections 2024-02-02T15:46:11.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.573+0000 7f9218daab80 1 -- 172.21.15.1:0/682950705 wait complete. 2024-02-02T15:46:11.573 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.574+0000 7f9218daab80 20 client.30539 trim_cache size 0 max 16384 2024-02-02T15:46:11.574 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:46:11.574 INFO:teuthology.orchestra.run.smithi001.stdout: "return_code": 0, 2024-02-02T15:46:11.574 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_tag": "7ecf2941-68ef-4dfb-887a-e158a5606caf", 2024-02-02T15:46:11.574 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": "asynchronous" 2024-02-02T15:46:11.574 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:46:11.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.575+0000 7f9205200700 1 -- 172.21.15.1:0/648387549 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f91e00a0790 msgr2=0x7f91e00a2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:11.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.575+0000 7f9205200700 1 --2- 172.21.15.1:0/648387549 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f91e00a0790 0x7f91e00a2c50 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f91f0009740 tx=0x7f91f000ef70 comp rx=0 tx=0).stop 2024-02-02T15:46:11.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.575+0000 7f9205200700 1 -- 172.21.15.1:0/648387549 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9200111a80 msgr2=0x7f92001a1eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:11.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.575+0000 7f9205200700 1 --2- 172.21.15.1:0/648387549 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9200111a80 0x7f92001a1eb0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f91f0005fd0 tx=0x7f91f000ec20 comp rx=0 tx=0).stop 2024-02-02T15:46:11.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.575+0000 7f9205200700 1 -- 172.21.15.1:0/648387549 shutdown_connections 2024-02-02T15:46:11.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.575+0000 7f9205200700 1 --2- 172.21.15.1:0/648387549 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f91e00a0790 0x7f91e00a2c50 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:11.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.575+0000 7f9205200700 1 --2- 172.21.15.1:0/648387549 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9200111a80 0x7f92001a1eb0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:11.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.575+0000 7f9205200700 1 --2- 172.21.15.1:0/648387549 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f92000ff680 0x7f920019d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:11.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.575+0000 7f9205200700 1 --2- 172.21.15.1:0/648387549 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9200100030 0x7f920019da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:11.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.575+0000 7f9205200700 1 -- 172.21.15.1:0/648387549 >> 172.21.15.1:0/648387549 conn(0x7f9200074f30 msgr2=0x7f920010f530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:11.575 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.575+0000 7f9205200700 1 -- 172.21.15.1:0/648387549 shutdown_connections 2024-02-02T15:46:11.576 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.575+0000 7f9205200700 1 -- 172.21.15.1:0/648387549 wait complete. 2024-02-02T15:46:11.602 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-02T15:46:11.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:46:11 smithi071 ceph-mon[44707]: pgmap v603: 129 pgs: 129 active+clean; 2.3 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 4.8 MiB/s wr, 416 op/s 2024-02-02T15:46:11.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.887+0000 7ff838200700 1 -- 172.21.15.1:0/3083621290 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff830107600 msgr2=0x7ff8301079e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:11.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.887+0000 7ff838200700 1 --2- 172.21.15.1:0/3083621290 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff830107600 0x7ff8301079e0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7ff82800b0a0 tx=0x7ff82800b3b0 comp rx=0 tx=0).stop 2024-02-02T15:46:11.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.888+0000 7ff838200700 1 -- 172.21.15.1:0/3083621290 shutdown_connections 2024-02-02T15:46:11.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.888+0000 7ff838200700 1 --2- 172.21.15.1:0/3083621290 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff83010c720 0x7ff8301142a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:11.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.888+0000 7ff838200700 1 --2- 172.21.15.1:0/3083621290 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff830107600 0x7ff8301079e0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:11.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.888+0000 7ff838200700 1 --2- 172.21.15.1:0/3083621290 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff830107fb0 0x7ff83010bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:11.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.888+0000 7ff838200700 1 -- 172.21.15.1:0/3083621290 >> 172.21.15.1:0/3083621290 conn(0x7ff830076b80 msgr2=0x7ff830105b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:11.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.888+0000 7ff838200700 1 -- 172.21.15.1:0/3083621290 shutdown_connections 2024-02-02T15:46:11.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.889+0000 7ff838200700 1 -- 172.21.15.1:0/3083621290 wait complete. 2024-02-02T15:46:11.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.890+0000 7ff838200700 1 Processor -- start 2024-02-02T15:46:11.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.892+0000 7ff838200700 1 -- start start 2024-02-02T15:46:11.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.893+0000 7ff838200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff830107600 0x7ff83019d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:11.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.893+0000 7ff838200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff83010c720 0x7ff83019d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:11.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.893+0000 7ff838200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8301a1da0 0x7ff8301a2220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:11.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.893+0000 7ff838200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8301a2820 con 0x7ff83010c720 2024-02-02T15:46:11.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.893+0000 7ff838200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8301a2990 con 0x7ff830107600 2024-02-02T15:46:11.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.893+0000 7ff838200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8301a2b50 con 0x7ff8301a1da0 2024-02-02T15:46:11.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.894+0000 7ff836200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8301a1da0 0x7ff8301a2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:11.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.894+0000 7ff836200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8301a1da0 0x7ff8301a2220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:44906/0 (socket says 172.21.15.1:44906) 2024-02-02T15:46:11.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.894+0000 7ff836200700 1 -- 172.21.15.1:0/2246816470 learned_addr learned my addr 172.21.15.1:0/2246816470 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:46:11.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.894+0000 7ff835800700 1 --2- 172.21.15.1:0/2246816470 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff830107600 0x7ff83019d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:11.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.894+0000 7ff836200700 1 -- 172.21.15.1:0/2246816470 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff830107600 msgr2=0x7ff83019d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:11.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.894+0000 7ff836200700 1 --2- 172.21.15.1:0/2246816470 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff830107600 0x7ff83019d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:11.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.894+0000 7ff836200700 1 -- 172.21.15.1:0/2246816470 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff83010c720 msgr2=0x7ff83019d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:11.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.894+0000 7ff836200700 1 --2- 172.21.15.1:0/2246816470 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff83010c720 0x7ff83019d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:11.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.894+0000 7ff836200700 1 -- 172.21.15.1:0/2246816470 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff828009d00 con 0x7ff8301a1da0 2024-02-02T15:46:11.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.895+0000 7ff836200700 1 --2- 172.21.15.1:0/2246816470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8301a1da0 0x7ff8301a2220 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7ff81800e940 tx=0x7ff81800ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:11.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.895+0000 7ff82e000700 1 -- 172.21.15.1:0/2246816470 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81800cbd0 con 0x7ff8301a1da0 2024-02-02T15:46:11.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.895+0000 7ff82e000700 1 -- 172.21.15.1:0/2246816470 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff81800ee90 con 0x7ff8301a1da0 2024-02-02T15:46:11.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.895+0000 7ff82e000700 1 -- 172.21.15.1:0/2246816470 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8180055c0 con 0x7ff8301a1da0 2024-02-02T15:46:11.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.895+0000 7ff838200700 1 -- 172.21.15.1:0/2246816470 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8301aa0b0 con 0x7ff8301a1da0 2024-02-02T15:46:11.896 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.895+0000 7ff838200700 1 -- 172.21.15.1:0/2246816470 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8301aa540 con 0x7ff8301a1da0 2024-02-02T15:46:11.901 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.900+0000 7ff82e000700 1 -- 172.21.15.1:0/2246816470 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7ff818004750 con 0x7ff8301a1da0 2024-02-02T15:46:11.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.903+0000 7ff82e000700 1 --2- 172.21.15.1:0/2246816470 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff8100a07d0 0x7ff8100a2c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:11.902 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.903+0000 7ff82e000700 1 -- 172.21.15.1:0/2246816470 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 9541+0+0 (secure 0 0 0) 0x7ff818014070 con 0x7ff8301a1da0 2024-02-02T15:46:11.903 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.903+0000 7ff835800700 1 --2- 172.21.15.1:0/2246816470 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff8100a07d0 0x7ff8100a2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:11.904 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.904+0000 7ff84bde6b80 1 Processor -- start 2024-02-02T15:46:11.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.904+0000 7ff835800700 1 --2- 172.21.15.1:0/2246816470 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff8100a07d0 0x7ff8100a2c90 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ff828009610 tx=0x7ff82800b700 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:11.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.904+0000 7ff84bde6b80 1 -- start start 2024-02-02T15:46:11.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.904+0000 7ff84bde6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c8cd21010 0x564c8cd213f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:11.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.904+0000 7ff84bde6b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564c8cd21930 0x564c8cd2c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:11.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.904+0000 7ff84bde6b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564c8cd2c7e0 0x564c8cd2ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:11.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.904+0000 7ff84bde6b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c8cc954c0 con 0x564c8cd2c7e0 2024-02-02T15:46:11.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.904+0000 7ff84bde6b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c8cc79aa0 con 0x564c8cd21930 2024-02-02T15:46:11.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.904+0000 7ff84bde6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c8cc77460 con 0x564c8cd21010 2024-02-02T15:46:11.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.905+0000 7ff834e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564c8cd21930 0x564c8cd2c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:11.905 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.905+0000 7ff834e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564c8cd21930 0x564c8cd2c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:39900/0 (socket says 172.21.15.1:39900) 2024-02-02T15:46:11.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.905+0000 7ff834e00700 1 -- 172.21.15.1:0/3325078713 learned_addr learned my addr 172.21.15.1:0/3325078713 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:46:11.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.905+0000 7ff836200700 1 --2- 172.21.15.1:0/3325078713 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564c8cd2c7e0 0x564c8cd2ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:11.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.905+0000 7ff834e00700 1 -- 172.21.15.1:0/3325078713 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c8cd21010 msgr2=0x564c8cd213f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:11.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.905+0000 7ff834e00700 1 --2- 172.21.15.1:0/3325078713 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c8cd21010 0x564c8cd213f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:11.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.905+0000 7ff834e00700 1 -- 172.21.15.1:0/3325078713 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564c8cd2c7e0 msgr2=0x564c8cd2ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:11.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.905+0000 7ff834e00700 1 --2- 172.21.15.1:0/3325078713 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564c8cd2c7e0 0x564c8cd2ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:11.906 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.905+0000 7ff834e00700 1 -- 172.21.15.1:0/3325078713 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c8caf43b0 con 0x564c8cd21930 2024-02-02T15:46:11.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.906+0000 7ff834e00700 1 --2- 172.21.15.1:0/3325078713 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564c8cd21930 0x564c8cd2c2a0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7ff820009c70 tx=0x7ff820009f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f8291b8cc5122a2 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:11.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.906+0000 7ff826a00700 1 -- 172.21.15.1:0/3325078713 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8200097c0 con 0x564c8cd21930 2024-02-02T15:46:11.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.906+0000 7ff826a00700 1 -- 172.21.15.1:0/3325078713 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff820004830 con 0x564c8cd21930 2024-02-02T15:46:11.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.906+0000 7ff826a00700 1 -- 172.21.15.1:0/3325078713 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8200056c0 con 0x564c8cd21930 2024-02-02T15:46:11.907 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.907+0000 7ff84bde6b80 1 -- 172.21.15.1:0/3325078713 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564c8cd21930 msgr2=0x564c8cd2c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:11.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.907+0000 7ff84bde6b80 1 --2- 172.21.15.1:0/3325078713 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564c8cd21930 0x564c8cd2c2a0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7ff820009c70 tx=0x7ff820009f80 comp rx=0 tx=0).stop 2024-02-02T15:46:11.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.907+0000 7ff84bde6b80 1 -- 172.21.15.1:0/3325078713 shutdown_connections 2024-02-02T15:46:11.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.907+0000 7ff84bde6b80 1 --2- 172.21.15.1:0/3325078713 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564c8cd2c7e0 0x564c8cd2ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:11.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.907+0000 7ff84bde6b80 1 --2- 172.21.15.1:0/3325078713 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c8cd21010 0x564c8cd213f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:11.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.907+0000 7ff84bde6b80 1 --2- 172.21.15.1:0/3325078713 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564c8cd21930 0x564c8cd2c2a0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:11.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.907+0000 7ff84bde6b80 1 -- 172.21.15.1:0/3325078713 >> 172.21.15.1:0/3325078713 conn(0x564c8cc80db0 msgr2=0x564c8cd1d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:11.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.907+0000 7ff84bde6b80 1 -- 172.21.15.1:0/3325078713 shutdown_connections 2024-02-02T15:46:11.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.907+0000 7ff84bde6b80 1 -- 172.21.15.1:0/3325078713 wait complete. 2024-02-02T15:46:11.908 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.909+0000 7ff84bde6b80 1 -- start start 2024-02-02T15:46:11.909 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.909+0000 7ff84bde6b80 1 Processor -- start 2024-02-02T15:46:11.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.911+0000 7ff84bde6b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564c8cd21010 0x564c8cd29730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:11.911 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.911+0000 7ff84bde6b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564c8cd2c7e0 0x564c8cd29c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:11.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.912+0000 7ff84bde6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c8cd2a1b0 0x564c8cd383a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:11.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.912+0000 7ff84bde6b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c8cc72d50 con 0x564c8cd2c7e0 2024-02-02T15:46:11.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.912+0000 7ff84bde6b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c8cc0ed80 con 0x564c8cd21010 2024-02-02T15:46:11.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.912+0000 7ff84bde6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c8cc0ea20 con 0x564c8cd2a1b0 2024-02-02T15:46:11.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.912+0000 7ff84bde6b80 10 client.0 _ll_register_callbacks cb 0x564c8cc10290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:46:11.912 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.913+0000 7ff834e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564c8cd2c7e0 0x564c8cd29c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:11.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.913+0000 7ff834e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564c8cd2c7e0 0x564c8cd29c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:34860/0 (socket says 172.21.15.1:34860) 2024-02-02T15:46:11.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.913+0000 7ff834e00700 1 -- 172.21.15.1:0/2658134067 learned_addr learned my addr 172.21.15.1:0/2658134067 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:46:11.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.913+0000 7ff836200700 1 --2- 172.21.15.1:0/2658134067 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c8cd2a1b0 0x564c8cd383a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:11.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.913+0000 7ff836200700 1 -- 172.21.15.1:0/2658134067 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564c8cd21010 msgr2=0x564c8cd29730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:11.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.913+0000 7ff836200700 1 --2- 172.21.15.1:0/2658134067 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564c8cd21010 0x564c8cd29730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:11.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.913+0000 7ff836200700 1 -- 172.21.15.1:0/2658134067 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564c8cd2c7e0 msgr2=0x564c8cd29c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:11.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.913+0000 7ff836200700 1 --2- 172.21.15.1:0/2658134067 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564c8cd2c7e0 0x564c8cd29c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:11.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.913+0000 7ff836200700 1 -- 172.21.15.1:0/2658134067 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff820009920 con 0x564c8cd2a1b0 2024-02-02T15:46:11.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.913+0000 7ff834e00700 1 --2- 172.21.15.1:0/2658134067 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564c8cd2c7e0 0x564c8cd29c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:46:11.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.914+0000 7ff836200700 1 --2- 172.21.15.1:0/2658134067 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c8cd2a1b0 0x564c8cd383a0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7ff8180c9720 tx=0x7ff818005720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9c5243e748c377e server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:11.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.914+0000 7ff81f400700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:46:11.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.917+0000 7ff81f400700 1 -- 172.21.15.1:0/2658134067 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8180994f0 con 0x564c8cd2a1b0 2024-02-02T15:46:11.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.917+0000 7ff81f400700 1 -- 172.21.15.1:0/2658134067 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 9541+0+0 (secure 0 0 0) 0x7ff81809d320 con 0x564c8cd2a1b0 2024-02-02T15:46:11.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.917+0000 7ff84bde6b80 1 -- 172.21.15.1:0/2658134067 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564c8cd150f0 con 0x564c8cd2a1b0 2024-02-02T15:46:11.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.918+0000 7ff81f400700 1 -- 172.21.15.1:0/2658134067 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff8180a1560 con 0x564c8cd2a1b0 2024-02-02T15:46:11.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.918+0000 7ff84bde6b80 10 client.30379 fetch_fsmap learned FSMap version 78 2024-02-02T15:46:11.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.918+0000 7ff84bde6b80 1 -- 172.21.15.1:0/2658134067 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) v3 -- 0x564c8caa0b60 con 0x564c8cd2a1b0 2024-02-02T15:46:11.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:46:11 smithi142 ceph-mon[45728]: pgmap v603: 129 pgs: 129 active+clean; 2.3 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 4.8 MiB/s wr, 416 op/s 2024-02-02T15:46:11.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.921+0000 7ff81f400700 1 -- 172.21.15.1:0/2658134067 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x564c8cd150f0 con 0x564c8cd2a1b0 2024-02-02T15:46:11.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.921+0000 7ff81f400700 1 -- 172.21.15.1:0/2658134067 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 78) v1 ==== 7506+0+0 (secure 0 0 0) 0x7ff818099750 con 0x564c8cd2a1b0 2024-02-02T15:46:11.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.921+0000 7ff84bde6b80 10 client.30379 fetch_fsmap finished waiting for FSMap version 78 2024-02-02T15:46:11.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.921+0000 7ff84bde6b80 10 client.30379 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:46:11.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.922+0000 7ff84bde6b80 20 client.30379 populate_metadata read hostname 'smithi001' 2024-02-02T15:46:11.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.922+0000 7ff84bde6b80 1 --2- 172.21.15.1:0/2658134067 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x564c8cd3c820 0x564c8cd3ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:11.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.922+0000 7ff84bde6b80 4 client.30379 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:46:11.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.922+0000 7ff84bde6b80 1 -- 172.21.15.1:0/2658134067 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564c8cc6bdd0 con 0x564c8cd3c820 2024-02-02T15:46:11.924 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.924+0000 7ff835800700 1 --2- 172.21.15.1:0/2658134067 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x564c8cd3c820 0x564c8cd3ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:11.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.928+0000 7ff81f400700 10 client.30379 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:46:11.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.928+0000 7ff835800700 1 --2- 172.21.15.1:0/2658134067 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x564c8cd3c820 0x564c8cd3ec40 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7efb5e815607606e server_cookie=a76157ca32127745 in_seq=0 out_seq=0 2024-02-02T15:46:11.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.929+0000 7ff81f400700 1 -- 172.21.15.1:0/2658134067 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x564c8cc6bdd0 con 0x564c8cd3c820 2024-02-02T15:46:11.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.929+0000 7ff81f400700 10 client.30379 handle_command_reply: tid=0 2024-02-02T15:46:11.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.931+0000 7ff84bde6b80 1 client.30379 shutdown 2024-02-02T15:46:11.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.931+0000 7ff84bde6b80 1 -- 172.21.15.1:0/2658134067 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c8cd2a1b0 msgr2=0x564c8cd383a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:11.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.931+0000 7ff84bde6b80 1 --2- 172.21.15.1:0/2658134067 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c8cd2a1b0 0x564c8cd383a0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7ff8180c9720 tx=0x7ff818005720 comp rx=0 tx=0).stop 2024-02-02T15:46:11.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.932+0000 7ff84bde6b80 1 -- 172.21.15.1:0/2658134067 shutdown_connections 2024-02-02T15:46:11.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.932+0000 7ff84bde6b80 1 --2- 172.21.15.1:0/2658134067 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x564c8cd3c820 0x564c8cd3ec40 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:11.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.932+0000 7ff84bde6b80 1 --2- 172.21.15.1:0/2658134067 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c8cd2a1b0 0x564c8cd383a0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:11.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.932+0000 7ff84bde6b80 1 --2- 172.21.15.1:0/2658134067 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564c8cd21010 0x564c8cd29730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:11.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.932+0000 7ff84bde6b80 1 --2- 172.21.15.1:0/2658134067 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564c8cd2c7e0 0x564c8cd29c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:11.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.932+0000 7ff84bde6b80 1 -- 172.21.15.1:0/2658134067 >> 172.21.15.1:0/2658134067 conn(0x564c8cc80db0 msgr2=0x564c8cd1cfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:11.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.932+0000 7ff81f400700 0 client.30379 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:46:11.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.932+0000 7ff84bde6b80 1 -- 172.21.15.1:0/2658134067 shutdown_connections 2024-02-02T15:46:11.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.932+0000 7ff84bde6b80 1 -- 172.21.15.1:0/2658134067 wait complete. 2024-02-02T15:46:11.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.933+0000 7ff84bde6b80 20 client.30379 trim_cache size 0 max 16384 2024-02-02T15:46:11.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:46:11 smithi001 ceph-mon[35470]: pgmap v603: 129 pgs: 129 active+clean; 2.3 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 4.8 MiB/s wr, 416 op/s 2024-02-02T15:46:11.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.966+0000 7ff84bde6b80 1 Processor -- start 2024-02-02T15:46:11.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.967+0000 7ff84bde6b80 1 -- start start 2024-02-02T15:46:11.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.967+0000 7ff84bde6b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564c8cd21010 0x564c8cd23a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:11.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.967+0000 7ff84bde6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c8cd2a1b0 0x564c8cd33330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:11.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.967+0000 7ff84bde6b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564c8cd2c7e0 0x564c8cd269c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:11.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.967+0000 7ff84bde6b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c8caf66a0 con 0x564c8cd21010 2024-02-02T15:46:11.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.967+0000 7ff84bde6b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c8cd199f0 con 0x564c8cd2c7e0 2024-02-02T15:46:11.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.967+0000 7ff84bde6b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c8cd22830 con 0x564c8cd2a1b0 2024-02-02T15:46:11.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.968+0000 7ff834e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c8cd2a1b0 0x564c8cd33330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:11.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.968+0000 7ff835800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564c8cd21010 0x564c8cd23a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:11.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.968+0000 7ff836200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564c8cd2c7e0 0x564c8cd269c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:11.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.968+0000 7ff834e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c8cd2a1b0 0x564c8cd33330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:44940/0 (socket says 172.21.15.1:44940) 2024-02-02T15:46:11.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.968+0000 7ff834e00700 1 -- 172.21.15.1:0/751963956 learned_addr learned my addr 172.21.15.1:0/751963956 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:46:11.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.968+0000 7ff835800700 1 -- 172.21.15.1:0/751963956 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c8cd2a1b0 msgr2=0x564c8cd33330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:11.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.968+0000 7ff835800700 1 --2- 172.21.15.1:0/751963956 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c8cd2a1b0 0x564c8cd33330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:11.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.968+0000 7ff835800700 1 -- 172.21.15.1:0/751963956 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564c8cd2c7e0 msgr2=0x564c8cd269c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:11.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.968+0000 7ff835800700 1 --2- 172.21.15.1:0/751963956 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564c8cd2c7e0 0x564c8cd269c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:11.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.968+0000 7ff835800700 1 -- 172.21.15.1:0/751963956 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c8cc38ea0 con 0x564c8cd21010 2024-02-02T15:46:11.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.969+0000 7ff834e00700 1 --2- 172.21.15.1:0/751963956 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c8cd2a1b0 0x564c8cd33330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:46:11.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.969+0000 7ff835800700 1 --2- 172.21.15.1:0/751963956 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564c8cd21010 0x564c8cd23a40 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7ff828009fd0 tx=0x7ff8280099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45c4089d7b146430 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:11.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.969+0000 7ff827400700 1 -- 172.21.15.1:0/751963956 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff82800e040 con 0x564c8cd21010 2024-02-02T15:46:11.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.969+0000 7ff827400700 1 -- 172.21.15.1:0/751963956 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff828007800 con 0x564c8cd21010 2024-02-02T15:46:11.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.969+0000 7ff827400700 1 -- 172.21.15.1:0/751963956 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8280286a0 con 0x564c8cd21010 2024-02-02T15:46:11.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.970+0000 7ff84bde6b80 1 -- 172.21.15.1:0/751963956 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564c8cd21010 msgr2=0x564c8cd23a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:11.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.970+0000 7ff84bde6b80 1 --2- 172.21.15.1:0/751963956 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564c8cd21010 0x564c8cd23a40 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7ff828009fd0 tx=0x7ff8280099e0 comp rx=0 tx=0).stop 2024-02-02T15:46:11.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.970+0000 7ff84bde6b80 1 -- 172.21.15.1:0/751963956 shutdown_connections 2024-02-02T15:46:11.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.971+0000 7ff84bde6b80 1 --2- 172.21.15.1:0/751963956 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564c8cd2c7e0 0x564c8cd269c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:11.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.971+0000 7ff84bde6b80 1 --2- 172.21.15.1:0/751963956 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564c8cd21010 0x564c8cd23a40 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:11.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.971+0000 7ff84bde6b80 1 --2- 172.21.15.1:0/751963956 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c8cd2a1b0 0x564c8cd33330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:11.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.971+0000 7ff84bde6b80 1 -- 172.21.15.1:0/751963956 >> 172.21.15.1:0/751963956 conn(0x564c8cc80db0 msgr2=0x564c8cd38190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:11.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.971+0000 7ff84bde6b80 1 -- 172.21.15.1:0/751963956 shutdown_connections 2024-02-02T15:46:11.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.971+0000 7ff84bde6b80 1 -- 172.21.15.1:0/751963956 wait complete. 2024-02-02T15:46:11.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.972+0000 7ff84bde6b80 1 -- start start 2024-02-02T15:46:11.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.972+0000 7ff84bde6b80 1 Processor -- start 2024-02-02T15:46:11.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.974+0000 7ff84bde6b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564c8cd21010 0x564c8cd88030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:11.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.974+0000 7ff84bde6b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c8cd2a1b0 0x564c8cd88570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:11.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.974+0000 7ff84bde6b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564c8cd2c7e0 0x564c8cd82390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:11.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.975+0000 7ff84bde6b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c8cd82970 con 0x564c8cd21010 2024-02-02T15:46:11.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.975+0000 7ff835800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564c8cd21010 0x564c8cd88030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:11.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.975+0000 7ff835800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564c8cd21010 0x564c8cd88030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:34872/0 (socket says 172.21.15.1:34872) 2024-02-02T15:46:11.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.975+0000 7ff835800700 1 -- 172.21.15.1:0/3691094590 learned_addr learned my addr 172.21.15.1:0/3691094590 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:46:11.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.976+0000 7ff834e00700 1 --2- 172.21.15.1:0/3691094590 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c8cd2a1b0 0x564c8cd88570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:11.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.976+0000 7ff84bde6b80 1 -- 172.21.15.1:0/3691094590 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c8cd82ae0 con 0x564c8cd2c7e0 2024-02-02T15:46:11.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.976+0000 7ff84bde6b80 1 -- 172.21.15.1:0/3691094590 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c8cd82ca0 con 0x564c8cd2a1b0 2024-02-02T15:46:11.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.976+0000 7ff84bde6b80 10 client.0 _ll_register_callbacks cb 0x564c8cc10290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:46:11.976 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.976+0000 7ff836200700 1 --2- 172.21.15.1:0/3691094590 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564c8cd2c7e0 0x564c8cd82390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:11.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.977+0000 7ff835800700 1 -- 172.21.15.1:0/3691094590 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c8cd2a1b0 msgr2=0x564c8cd88570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:11.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.977+0000 7ff835800700 1 --2- 172.21.15.1:0/3691094590 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c8cd2a1b0 0x564c8cd88570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:11.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.977+0000 7ff835800700 1 -- 172.21.15.1:0/3691094590 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564c8cd2c7e0 msgr2=0x564c8cd82390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:11.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.977+0000 7ff835800700 1 --2- 172.21.15.1:0/3691094590 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564c8cd2c7e0 0x564c8cd82390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:11.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.977+0000 7ff835800700 1 -- 172.21.15.1:0/3691094590 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564c8caf43b0 con 0x564c8cd21010 2024-02-02T15:46:11.977 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.977+0000 7ff836200700 1 --2- 172.21.15.1:0/3691094590 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564c8cd2c7e0 0x564c8cd82390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:46:11.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.978+0000 7ff835800700 1 --2- 172.21.15.1:0/3691094590 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564c8cd21010 0x564c8cd88030 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7ff82800bc00 tx=0x7ff828007970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17d835b587fbc113 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:11.978 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.978+0000 7ff81f400700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:46:11.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.979+0000 7ff81f400700 1 -- 172.21.15.1:0/3691094590 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff82801ce90 con 0x564c8cd21010 2024-02-02T15:46:11.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.979+0000 7ff81f400700 1 -- 172.21.15.1:0/3691094590 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 9541+0+0 (secure 0 0 0) 0x7ff82801c440 con 0x564c8cd21010 2024-02-02T15:46:11.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.979+0000 7ff84bde6b80 1 -- 172.21.15.1:0/3691094590 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff820004830 con 0x564c8cd21010 2024-02-02T15:46:11.979 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.979+0000 7ff81f400700 1 -- 172.21.15.1:0/3691094590 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff82801c600 con 0x564c8cd21010 2024-02-02T15:46:11.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.980+0000 7ff84bde6b80 10 client.20250 fetch_fsmap learned FSMap version 78 2024-02-02T15:46:11.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.980+0000 7ff84bde6b80 1 -- 172.21.15.1:0/3691094590 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) v3 -- 0x7ff830114b10 con 0x564c8cd21010 2024-02-02T15:46:11.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.980+0000 7ff81f400700 1 -- 172.21.15.1:0/3691094590 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff820004830 con 0x564c8cd21010 2024-02-02T15:46:11.980 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.980+0000 7ff81f400700 1 -- 172.21.15.1:0/3691094590 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 78) v1 ==== 7506+0+0 (secure 0 0 0) 0x7ff82803f070 con 0x564c8cd21010 2024-02-02T15:46:11.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.981+0000 7ff84bde6b80 10 client.20250 fetch_fsmap finished waiting for FSMap version 78 2024-02-02T15:46:11.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.981+0000 7ff84bde6b80 10 client.20250 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:46:11.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.981+0000 7ff84bde6b80 20 client.20250 populate_metadata read hostname 'smithi001' 2024-02-02T15:46:11.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.981+0000 7ff84bde6b80 1 --2- 172.21.15.1:0/3691094590 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x564c8cd8f960 0x564c8cd91d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:11.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.981+0000 7ff84bde6b80 4 client.20250 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2024-02-02T15:46:11.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.981+0000 7ff84bde6b80 1 -- 172.21.15.1:0/3691094590 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564c8cc31f50 con 0x564c8cd8f960 2024-02-02T15:46:11.981 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.981+0000 7ff834e00700 1 --2- 172.21.15.1:0/3691094590 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x564c8cd8f960 0x564c8cd91d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:11.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.982+0000 7ff81f400700 10 client.20250 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:46:11.982 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.983+0000 7ff834e00700 1 --2- 172.21.15.1:0/3691094590 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x564c8cd8f960 0x564c8cd91d50 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=635cd9e1bc570dd9 server_cookie=74cd9279756243ad in_seq=0 out_seq=0 2024-02-02T15:46:11.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.983+0000 7ff81f400700 1 -- 172.21.15.1:0/3691094590 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x564c8cc31f50 con 0x564c8cd8f960 2024-02-02T15:46:11.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.983+0000 7ff81f400700 10 client.20250 handle_command_reply: tid=0 2024-02-02T15:46:11.983 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.983+0000 7ff84bde6b80 1 client.20250 shutdown 2024-02-02T15:46:11.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.984+0000 7ff84bde6b80 1 -- 172.21.15.1:0/3691094590 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564c8cd21010 msgr2=0x564c8cd88030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:11.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.984+0000 7ff84bde6b80 1 --2- 172.21.15.1:0/3691094590 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564c8cd21010 0x564c8cd88030 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7ff82800bc00 tx=0x7ff828007970 comp rx=0 tx=0).stop 2024-02-02T15:46:11.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.984+0000 7ff84bde6b80 1 -- 172.21.15.1:0/3691094590 shutdown_connections 2024-02-02T15:46:11.984 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.984+0000 7ff84bde6b80 1 --2- 172.21.15.1:0/3691094590 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x564c8cd8f960 0x564c8cd91d50 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:11.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.984+0000 7ff84bde6b80 1 --2- 172.21.15.1:0/3691094590 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564c8cd2c7e0 0x564c8cd82390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:11.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.984+0000 7ff84bde6b80 1 --2- 172.21.15.1:0/3691094590 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x564c8cd21010 0x564c8cd88030 secure :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7ff82800bc00 tx=0x7ff828007970 comp rx=0 tx=0).stop 2024-02-02T15:46:11.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.984+0000 7ff84bde6b80 1 --2- 172.21.15.1:0/3691094590 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x564c8cd2a1b0 0x564c8cd88570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:11.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.984+0000 7ff84bde6b80 1 -- 172.21.15.1:0/3691094590 >> 172.21.15.1:0/3691094590 conn(0x564c8cc80db0 msgr2=0x564c8cd2afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:11.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.984+0000 7ff81f400700 0 client.20250 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:46:11.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.984+0000 7ff84bde6b80 1 -- 172.21.15.1:0/3691094590 shutdown_connections 2024-02-02T15:46:11.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.984+0000 7ff84bde6b80 1 -- 172.21.15.1:0/3691094590 wait complete. 2024-02-02T15:46:11.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.985+0000 7ff84bde6b80 20 client.20250 trim_cache size 0 max 16384 2024-02-02T15:46:11.986 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:46:11.986 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "IDLE (waiting for more scrubs)", 2024-02-02T15:46:11.986 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": { 2024-02-02T15:46:11.986 INFO:teuthology.orchestra.run.smithi001.stdout: "7ecf2941-68ef-4dfb-887a-e158a5606caf": { 2024-02-02T15:46:11.986 INFO:teuthology.orchestra.run.smithi001.stdout: "path": "/", 2024-02-02T15:46:11.986 INFO:teuthology.orchestra.run.smithi001.stdout: "tag": "7ecf2941-68ef-4dfb-887a-e158a5606caf", 2024-02-02T15:46:11.986 INFO:teuthology.orchestra.run.smithi001.stdout: "options": "recursive,force" 2024-02-02T15:46:11.986 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:11.986 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:11.986 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:46:11.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.985+0000 7ff838200700 1 -- 172.21.15.1:0/2246816470 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff8100a07d0 msgr2=0x7ff8100a2c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:11.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.985+0000 7ff838200700 1 --2- 172.21.15.1:0/2246816470 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff8100a07d0 0x7ff8100a2c90 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ff828009610 tx=0x7ff82800b700 comp rx=0 tx=0).stop 2024-02-02T15:46:11.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.985+0000 7ff838200700 1 -- 172.21.15.1:0/2246816470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8301a1da0 msgr2=0x7ff8301a2220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:11.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.985+0000 7ff838200700 1 --2- 172.21.15.1:0/2246816470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8301a1da0 0x7ff8301a2220 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7ff81800e940 tx=0x7ff81800ec50 comp rx=0 tx=0).stop 2024-02-02T15:46:11.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.986+0000 7ff838200700 1 -- 172.21.15.1:0/2246816470 shutdown_connections 2024-02-02T15:46:11.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.986+0000 7ff838200700 1 --2- 172.21.15.1:0/2246816470 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff8100a07d0 0x7ff8100a2c90 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:11.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.986+0000 7ff838200700 1 --2- 172.21.15.1:0/2246816470 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff8301a1da0 0x7ff8301a2220 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:11.987 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.986+0000 7ff838200700 1 --2- 172.21.15.1:0/2246816470 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff830107600 0x7ff83019d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:11.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.986+0000 7ff838200700 1 --2- 172.21.15.1:0/2246816470 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff83010c720 0x7ff83019d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:11.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.986+0000 7ff838200700 1 -- 172.21.15.1:0/2246816470 >> 172.21.15.1:0/2246816470 conn(0x7ff830076b80 msgr2=0x7ff830076f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:11.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.986+0000 7ff838200700 1 -- 172.21.15.1:0/2246816470 shutdown_connections 2024-02-02T15:46:11.988 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:11.986+0000 7ff838200700 1 -- 172.21.15.1:0/2246816470 wait complete. 2024-02-02T15:46:12.011 INFO:tasks.workunit.client.0.smithi001.stdout: 10 179463 26.43 MB/sec execute 224 sec latency 103.866 ms 2024-02-02T15:46:12.017 INFO:tasks.cephfs.filesystem:scrub status for tag:7ecf2941-68ef-4dfb-887a-e158a5606caf - {'path': '/', 'tag': '7ecf2941-68ef-4dfb-887a-e158a5606caf', 'options': 'recursive,force'} 2024-02-02T15:46:12.297 INFO:tasks.workunit.client.0.smithi001.stdout:[185981] write failed on handle 12281 (Resource temporarily unavailable) 2024-02-02T15:46:12.297 INFO:tasks.workunit.client.0.smithi001.stdout:Child failed with status 1 2024-02-02T15:46:12.298 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-02T15:46:12.314 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2024-02-02T15:46:12.314 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-02-02T15:46:12.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:46:12 smithi001 ceph-mon[35470]: scrub queued for path: / 2024-02-02T15:46:12.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:46:12 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:46:12.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:46:12 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:46:12.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:46:12 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:46:12.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:46:12 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:46:12.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:46:12 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:46:12.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:46:12 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:46:12.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:46:12 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:46:12.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:46:12 smithi001 ceph-mon[35470]: scrub summary: active paths [/] 2024-02-02T15:46:12.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:46:12 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:46:12.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:46:12 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:46:12.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:46:12 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:46:12.691 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:46:12 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:46:12.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:46:12 smithi071 ceph-mon[44707]: scrub queued for path: / 2024-02-02T15:46:12.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:46:12 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:46:12.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:46:12 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:46:12.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:46:12 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:46:12.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:46:12 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:46:12.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:46:12 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:46:12.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:46:12 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:46:12.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:46:12 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:46:12.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:46:12 smithi071 ceph-mon[44707]: scrub summary: active paths [/] 2024-02-02T15:46:12.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:46:12 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:46:12.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:46:12 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:46:12.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:46:12 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:46:12.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:46:12 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:46:12.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:46:12 smithi142 ceph-mon[45728]: scrub queued for path: / 2024-02-02T15:46:12.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:46:12 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:46:12.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:46:12 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:46:12.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:46:12 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:46:12.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:46:12 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:46:12.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:46:12 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:46:12.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:46:12 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:46:12.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:46:12 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:46:12.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:46:12 smithi142 ceph-mon[45728]: scrub summary: active paths [/] 2024-02-02T15:46:12.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:46:12 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:46:12.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:46:12 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:46:12.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:46:12 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:46:12.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:46:12 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:46:13.342 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_0fb327883c124fd83155c94db0804f823b805a12/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_0fb327883c124fd83155c94db0804f823b805a12/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_0fb327883c124fd83155c94db0804f823b805a12/qa/tasks/workunit.py", line 424, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi001 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=0fb327883c124fd83155c94db0804f823b805a12 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-02-02T15:46:13.597 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=6e95807c1d0f4617a7189c92498fb1ac Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_0fb327883c124fd83155c94db0804f823b805a12/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph-c_0fb327883c124fd83155c94db0804f823b805a12/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph-c_0fb327883c124fd83155c94db0804f823b805a12/qa/tasks/workunit.py", line 424, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_d9fdb2209e15b39d9f061fd85399f352ce0f0894/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi001 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=0fb327883c124fd83155c94db0804f823b805a12 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-02-02T15:46:13.600 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-02-02T15:46:13.613 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-02-02T15:46:13.613 INFO:teuthology.task.exec:Executing custom commands... 2024-02-02T15:46:13.613 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi001.front.sepia.ceph.com 2024-02-02T15:46:13.614 DEBUG:teuthology.orchestra.run.smithi001:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-02-02T15:46:13.687 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:46:13 smithi001 ceph-mon[35470]: scrub summary: idle+waiting paths [/] 2024-02-02T15:46:13.687 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:46:13 smithi001 ceph-mon[35470]: pgmap v604: 129 pgs: 129 active+clean; 2.3 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 4.7 MiB/s wr, 460 op/s 2024-02-02T15:46:13.698 DEBUG:teuthology.orchestra.run.smithi001:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-02-02T15:46:13.770 DEBUG:teuthology.orchestra.run.smithi001:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-02-02T15:46:13.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:46:13 smithi071 ceph-mon[44707]: scrub summary: idle+waiting paths [/] 2024-02-02T15:46:13.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:46:13 smithi071 ceph-mon[44707]: pgmap v604: 129 pgs: 129 active+clean; 2.3 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 4.7 MiB/s wr, 460 op/s 2024-02-02T15:46:13.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:46:13 smithi142 ceph-mon[45728]: scrub summary: idle+waiting paths [/] 2024-02-02T15:46:13.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:46:13 smithi142 ceph-mon[45728]: pgmap v604: 129 pgs: 129 active+clean; 2.3 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 4.7 MiB/s wr, 460 op/s 2024-02-02T15:46:13.964 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:13.964+0000 7f4d37400700 1 -- 172.21.15.1:0/3681754344 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4d30101900 msgr2=0x7f4d30105940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:13.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:13.964+0000 7f4d37400700 1 --2- 172.21.15.1:0/3681754344 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4d30101900 0x7f4d30105940 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f4d18009990 tx=0x7f4d18009ca0 comp rx=0 tx=0).stop 2024-02-02T15:46:13.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:13.964+0000 7f4d37400700 1 -- 172.21.15.1:0/3681754344 shutdown_connections 2024-02-02T15:46:13.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:13.964+0000 7f4d37400700 1 --2- 172.21.15.1:0/3681754344 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d30106070 0x7f4d3010dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:13.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:13.964+0000 7f4d37400700 1 --2- 172.21.15.1:0/3681754344 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4d30100f50 0x7f4d30101330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:13.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:13.964+0000 7f4d37400700 1 --2- 172.21.15.1:0/3681754344 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4d30101900 0x7f4d30105940 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:13.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:13.964+0000 7f4d37400700 1 -- 172.21.15.1:0/3681754344 >> 172.21.15.1:0/3681754344 conn(0x7f4d300fc7f0 msgr2=0x7f4d300fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:13.965 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:13.964+0000 7f4d37400700 1 -- 172.21.15.1:0/3681754344 shutdown_connections 2024-02-02T15:46:13.966 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:13.964+0000 7f4d37400700 1 -- 172.21.15.1:0/3681754344 wait complete. 2024-02-02T15:46:13.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:13.965+0000 7f4d37400700 1 Processor -- start 2024-02-02T15:46:13.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:13.965+0000 7f4d37400700 1 -- start start 2024-02-02T15:46:13.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:13.966+0000 7f4d37400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d30100f50 0x7f4d301990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:13.968 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:13.966+0000 7f4d37400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4d30106070 0x7f4d30199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:13.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:13.966+0000 7f4d37400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4d3019da70 0x7f4d3019def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:13.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:13.966+0000 7f4d37400700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3019e4f0 con 0x7f4d30106070 2024-02-02T15:46:13.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:13.966+0000 7f4d37400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3019e660 con 0x7f4d30100f50 2024-02-02T15:46:13.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:13.966+0000 7f4d37400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3019e820 con 0x7f4d3019da70 2024-02-02T15:46:13.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:13.966+0000 7f4d2fe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4d30106070 0x7f4d30199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:13.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:13.966+0000 7f4d35400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4d3019da70 0x7f4d3019def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:13.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:13.966+0000 7f4d35400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4d3019da70 0x7f4d3019def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:35510/0 (socket says 172.21.15.1:35510) 2024-02-02T15:46:13.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:13.966+0000 7f4d2fe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4d30106070 0x7f4d30199610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:34024/0 (socket says 172.21.15.1:34024) 2024-02-02T15:46:13.969 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:13.966+0000 7f4d35400700 1 -- 172.21.15.1:0/636171894 learned_addr learned my addr 172.21.15.1:0/636171894 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:46:13.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:13.966+0000 7f4d34a00700 1 --2- 172.21.15.1:0/636171894 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d30100f50 0x7f4d301990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:13.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:13.967+0000 7f4d35400700 1 -- 172.21.15.1:0/636171894 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d30100f50 msgr2=0x7f4d301990d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:13.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:13.967+0000 7f4d35400700 1 --2- 172.21.15.1:0/636171894 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d30100f50 0x7f4d301990d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:13.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:13.967+0000 7f4d35400700 1 -- 172.21.15.1:0/636171894 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4d30106070 msgr2=0x7f4d30199610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:13.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:13.967+0000 7f4d35400700 1 --2- 172.21.15.1:0/636171894 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4d30106070 0x7f4d30199610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:13.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:13.967+0000 7f4d35400700 1 -- 172.21.15.1:0/636171894 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d180095f0 con 0x7f4d3019da70 2024-02-02T15:46:13.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:13.967+0000 7f4d2fe00700 1 --2- 172.21.15.1:0/636171894 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4d30106070 0x7f4d30199610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:46:13.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:13.967+0000 7f4d35400700 1 --2- 172.21.15.1:0/636171894 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4d3019da70 0x7f4d3019def0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f4d2800ef30 tx=0x7f4d2800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:13.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:13.967+0000 7f4d2d600700 1 -- 172.21.15.1:0/636171894 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d28018070 con 0x7f4d3019da70 2024-02-02T15:46:13.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:13.967+0000 7f4d37400700 1 -- 172.21.15.1:0/636171894 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d301a5d20 con 0x7f4d3019da70 2024-02-02T15:46:13.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:13.967+0000 7f4d2d600700 1 -- 172.21.15.1:0/636171894 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4d2800f040 con 0x7f4d3019da70 2024-02-02T15:46:13.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:13.967+0000 7f4d2d600700 1 -- 172.21.15.1:0/636171894 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d2800cd00 con 0x7f4d3019da70 2024-02-02T15:46:13.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:13.967+0000 7f4d37400700 1 -- 172.21.15.1:0/636171894 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d301a6200 con 0x7f4d3019da70 2024-02-02T15:46:13.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:13.968+0000 7f4d37400700 1 -- 172.21.15.1:0/636171894 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d3004ea40 con 0x7f4d3019da70 2024-02-02T15:46:13.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:13.970+0000 7f4d2d600700 1 -- 172.21.15.1:0/636171894 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f4d28003a00 con 0x7f4d3019da70 2024-02-02T15:46:13.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:13.972+0000 7f4d2d600700 1 --2- 172.21.15.1:0/636171894 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f4d100a0690 0x7f4d100a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:13.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:13.972+0000 7f4d2d600700 1 -- 172.21.15.1:0/636171894 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f4d280cc5e0 con 0x7f4d3019da70 2024-02-02T15:46:13.972 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:13.972+0000 7f4d34a00700 1 --2- 172.21.15.1:0/636171894 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f4d100a0690 0x7f4d100a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:13.973 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:13.972+0000 7f4d34a00700 1 --2- 172.21.15.1:0/636171894 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f4d100a0690 0x7f4d100a2b50 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f4d20009c60 tx=0x7f4d20009260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:13.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:13.974+0000 7f4d2d600700 1 -- 172.21.15.1:0/636171894 <== mon.1 v2:172.21.15.142:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f4d28095a70 con 0x7f4d3019da70 2024-02-02T15:46:14.216 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:14.216+0000 7f4d37400700 1 -- 172.21.15.1:0/636171894 --> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7f4d30101a70 con 0x7f4d100a0690 2024-02-02T15:46:14.229 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:14.229+0000 7f4d2d600700 1 -- 172.21.15.1:0/636171894 <== mgr.25603 v2:172.21.15.1:6800/167617173 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f4d30101a70 con 0x7f4d100a0690 2024-02-02T15:46:14.229 INFO:teuthology.orchestra.run.smithi001.stdout:{"fs": "cephfs", "subvol": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-02-02T00:00:00", "created": "2024-02-02T15:27:01", "first": "2024-02-02T15:28:00", "last": "2024-02-02T15:46:00", "last_pruned": "2024-02-02T15:46:02", "created_count": 19, "pruned_count": 13, "active": true} 2024-02-02T15:46:14.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:14.232+0000 7f4d37400700 1 -- 172.21.15.1:0/636171894 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f4d100a0690 msgr2=0x7f4d100a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:14.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:14.232+0000 7f4d37400700 1 --2- 172.21.15.1:0/636171894 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f4d100a0690 0x7f4d100a2b50 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f4d20009c60 tx=0x7f4d20009260 comp rx=0 tx=0).stop 2024-02-02T15:46:14.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:14.232+0000 7f4d37400700 1 -- 172.21.15.1:0/636171894 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4d3019da70 msgr2=0x7f4d3019def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:14.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:14.232+0000 7f4d37400700 1 --2- 172.21.15.1:0/636171894 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4d3019da70 0x7f4d3019def0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f4d2800ef30 tx=0x7f4d2800c3a0 comp rx=0 tx=0).stop 2024-02-02T15:46:14.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:14.232+0000 7f4d37400700 1 -- 172.21.15.1:0/636171894 shutdown_connections 2024-02-02T15:46:14.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:14.232+0000 7f4d37400700 1 --2- 172.21.15.1:0/636171894 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f4d100a0690 0x7f4d100a2b50 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:14.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:14.232+0000 7f4d37400700 1 --2- 172.21.15.1:0/636171894 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4d3019da70 0x7f4d3019def0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:14.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:14.232+0000 7f4d37400700 1 --2- 172.21.15.1:0/636171894 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d30100f50 0x7f4d301990d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:14.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:14.232+0000 7f4d37400700 1 --2- 172.21.15.1:0/636171894 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4d30106070 0x7f4d30199610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:14.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:14.232+0000 7f4d37400700 1 -- 172.21.15.1:0/636171894 >> 172.21.15.1:0/636171894 conn(0x7f4d300fc7f0 msgr2=0x7f4d300fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:14.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:14.232+0000 7f4d37400700 1 -- 172.21.15.1:0/636171894 shutdown_connections 2024-02-02T15:46:14.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:14.232+0000 7f4d37400700 1 -- 172.21.15.1:0/636171894 wait complete. 2024-02-02T15:46:14.250 DEBUG:teuthology.orchestra.run.smithi001:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-02-02T15:46:14.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:14.446+0000 7f67cbe00700 1 -- 172.21.15.1:0/2890252589 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67c4101900 msgr2=0x7f67c4105940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:14.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:14.446+0000 7f67cbe00700 1 --2- 172.21.15.1:0/2890252589 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67c4101900 0x7f67c4105940 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f67ac009990 tx=0x7f67ac009ca0 comp rx=0 tx=0).stop 2024-02-02T15:46:14.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:14.447+0000 7f67cbe00700 1 -- 172.21.15.1:0/2890252589 shutdown_connections 2024-02-02T15:46:14.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:14.447+0000 7f67cbe00700 1 --2- 172.21.15.1:0/2890252589 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67c4106070 0x7f67c410dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:14.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:14.447+0000 7f67cbe00700 1 --2- 172.21.15.1:0/2890252589 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f67c4100f50 0x7f67c4101330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:14.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:14.447+0000 7f67cbe00700 1 --2- 172.21.15.1:0/2890252589 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67c4101900 0x7f67c4105940 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:14.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:14.447+0000 7f67cbe00700 1 -- 172.21.15.1:0/2890252589 >> 172.21.15.1:0/2890252589 conn(0x7f67c40fc7f0 msgr2=0x7f67c40fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:14.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:14.447+0000 7f67cbe00700 1 -- 172.21.15.1:0/2890252589 shutdown_connections 2024-02-02T15:46:14.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:14.447+0000 7f67cbe00700 1 -- 172.21.15.1:0/2890252589 wait complete. 2024-02-02T15:46:14.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:14.448+0000 7f67cbe00700 1 Processor -- start 2024-02-02T15:46:14.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:14.448+0000 7f67cbe00700 1 -- start start 2024-02-02T15:46:14.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:14.448+0000 7f67cbe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67c4100f50 0x7f67c4073040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:14.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:14.448+0000 7f67cbe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67c4106070 0x7f67c4073580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:14.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:14.448+0000 7f67cbe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f67c4073ac0 0x7f67c41a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:14.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:14.448+0000 7f67cbe00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c41a56e0 con 0x7f67c4100f50 2024-02-02T15:46:14.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:14.448+0000 7f67cbe00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c41a5850 con 0x7f67c4073ac0 2024-02-02T15:46:14.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:14.448+0000 7f67cbe00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c41a5a10 con 0x7f67c4106070 2024-02-02T15:46:14.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:14.448+0000 7f67c9400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67c4100f50 0x7f67c4073040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:14.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:14.448+0000 7f67c9400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67c4100f50 0x7f67c4073040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:34042/0 (socket says 172.21.15.1:34042) 2024-02-02T15:46:14.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:14.448+0000 7f67c9400700 1 -- 172.21.15.1:0/339086462 learned_addr learned my addr 172.21.15.1:0/339086462 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:46:14.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:14.448+0000 7f67c9e00700 1 --2- 172.21.15.1:0/339086462 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f67c4073ac0 0x7f67c41a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:14.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:14.449+0000 7f67c8a00700 1 --2- 172.21.15.1:0/339086462 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67c4106070 0x7f67c4073580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:14.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:14.449+0000 7f67c9400700 1 -- 172.21.15.1:0/339086462 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67c4106070 msgr2=0x7f67c4073580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:14.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:14.449+0000 7f67c9400700 1 --2- 172.21.15.1:0/339086462 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67c4106070 0x7f67c4073580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:14.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:14.449+0000 7f67c9400700 1 -- 172.21.15.1:0/339086462 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f67c4073ac0 msgr2=0x7f67c41a5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:14.452 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:14.449+0000 7f67c9400700 1 --2- 172.21.15.1:0/339086462 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f67c4073ac0 0x7f67c41a5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:14.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:14.449+0000 7f67c9400700 1 -- 172.21.15.1:0/339086462 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67ac0095f0 con 0x7f67c4100f50 2024-02-02T15:46:14.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:14.449+0000 7f67c9400700 1 --2- 172.21.15.1:0/339086462 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67c4100f50 0x7f67c4073040 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f67b400e9a0 tx=0x7f67b400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:14.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:14.449+0000 7f67c2000700 1 -- 172.21.15.1:0/339086462 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b4009790 con 0x7f67c4100f50 2024-02-02T15:46:14.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:14.449+0000 7f67cbe00700 1 -- 172.21.15.1:0/339086462 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67c41a5cf0 con 0x7f67c4100f50 2024-02-02T15:46:14.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:14.449+0000 7f67c2000700 1 -- 172.21.15.1:0/339086462 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f67b4010430 con 0x7f67c4100f50 2024-02-02T15:46:14.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:14.449+0000 7f67c2000700 1 -- 172.21.15.1:0/339086462 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67b4018680 con 0x7f67c4100f50 2024-02-02T15:46:14.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:14.450+0000 7f67cbe00700 1 -- 172.21.15.1:0/339086462 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67c41a61f0 con 0x7f67c4100f50 2024-02-02T15:46:14.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:14.451+0000 7f67cbe00700 1 -- 172.21.15.1:0/339086462 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67c404ea40 con 0x7f67c4100f50 2024-02-02T15:46:14.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:14.452+0000 7f67c2000700 1 -- 172.21.15.1:0/339086462 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f67b40105a0 con 0x7f67c4100f50 2024-02-02T15:46:14.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:14.454+0000 7f67c2000700 1 --2- 172.21.15.1:0/339086462 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f67a40a0690 0x7f67a40a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:14.454 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:14.454+0000 7f67c2000700 1 -- 172.21.15.1:0/339086462 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f67b4014070 con 0x7f67c4100f50 2024-02-02T15:46:14.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:14.454+0000 7f67c8a00700 1 --2- 172.21.15.1:0/339086462 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f67a40a0690 0x7f67a40a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:14.455 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:14.455+0000 7f67c8a00700 1 --2- 172.21.15.1:0/339086462 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f67a40a0690 0x7f67a40a2b50 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f67c4074830 tx=0x7f67ac005fb0 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:14.458 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:14.458+0000 7f67c2000700 1 -- 172.21.15.1:0/339086462 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f67b4052110 con 0x7f67c4100f50 2024-02-02T15:46:14.705 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:14.705+0000 7f67cbe00700 1 -- 172.21.15.1:0/339086462 --> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f67c4101b30 con 0x7f67a40a0690 2024-02-02T15:46:14.717 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:14.717+0000 7f67c2000700 1 -- 172.21.15.1:0/339086462 <== mgr.25603 v2:172.21.15.1:6800/167617173 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f67c4101b30 con 0x7f67a40a0690 2024-02-02T15:46:14.718 INFO:teuthology.orchestra.run.smithi001.stdout:/ 1m 6m3h 2024-02-02T15:46:14.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:14.720+0000 7f67cbe00700 1 -- 172.21.15.1:0/339086462 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f67a40a0690 msgr2=0x7f67a40a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:14.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:14.720+0000 7f67cbe00700 1 --2- 172.21.15.1:0/339086462 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f67a40a0690 0x7f67a40a2b50 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f67c4074830 tx=0x7f67ac005fb0 comp rx=0 tx=0).stop 2024-02-02T15:46:14.720 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:14.720+0000 7f67cbe00700 1 -- 172.21.15.1:0/339086462 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67c4100f50 msgr2=0x7f67c4073040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:14.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:14.720+0000 7f67cbe00700 1 --2- 172.21.15.1:0/339086462 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67c4100f50 0x7f67c4073040 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f67b400e9a0 tx=0x7f67b400ecb0 comp rx=0 tx=0).stop 2024-02-02T15:46:14.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:14.720+0000 7f67cbe00700 1 -- 172.21.15.1:0/339086462 shutdown_connections 2024-02-02T15:46:14.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:14.720+0000 7f67cbe00700 1 --2- 172.21.15.1:0/339086462 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f67a40a0690 0x7f67a40a2b50 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:14.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:14.720+0000 7f67cbe00700 1 --2- 172.21.15.1:0/339086462 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f67c4073ac0 0x7f67c41a5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:14.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:14.720+0000 7f67cbe00700 1 --2- 172.21.15.1:0/339086462 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f67c4100f50 0x7f67c4073040 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:14.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:14.720+0000 7f67cbe00700 1 --2- 172.21.15.1:0/339086462 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f67c4106070 0x7f67c4073580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:14.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:14.720+0000 7f67cbe00700 1 -- 172.21.15.1:0/339086462 >> 172.21.15.1:0/339086462 conn(0x7f67c40fc7f0 msgr2=0x7f67c40fd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:14.721 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:14.720+0000 7f67cbe00700 1 -- 172.21.15.1:0/339086462 shutdown_connections 2024-02-02T15:46:14.722 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:14.720+0000 7f67cbe00700 1 -- 172.21.15.1:0/339086462 wait complete. 2024-02-02T15:46:14.738 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-02-02T15:46:14.749 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-02-02T15:46:15.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:46:15 smithi071 ceph-mon[44707]: from='client.30391 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:46:15.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:46:15 smithi071 ceph-mon[44707]: pgmap v605: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.0 MiB/s wr, 465 op/s 2024-02-02T15:46:15.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:46:15 smithi142 ceph-mon[45728]: from='client.30391 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:46:15.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:46:15 smithi142 ceph-mon[45728]: pgmap v605: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.0 MiB/s wr, 465 op/s 2024-02-02T15:46:15.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:46:15 smithi001 ceph-mon[35470]: from='client.30391 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:46:15.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:46:15 smithi001 ceph-mon[35470]: pgmap v605: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.0 MiB/s wr, 465 op/s 2024-02-02T15:46:16.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:46:16 smithi071 ceph-mon[44707]: from='client.20262 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:46:16.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:46:16 smithi142 ceph-mon[45728]: from='client.20262 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:46:16.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:46:16 smithi001 ceph-mon[35470]: from='client.20262 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-02T15:46:17.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:46:17 smithi071 ceph-mon[44707]: pgmap v606: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.4 MiB/s wr, 353 op/s 2024-02-02T15:46:17.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:46:17 smithi071 ceph-mon[44707]: osdmap e115: 12 total, 12 up, 12 in 2024-02-02T15:46:17.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:46:17 smithi142 ceph-mon[45728]: pgmap v606: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.4 MiB/s wr, 353 op/s 2024-02-02T15:46:17.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:46:17 smithi142 ceph-mon[45728]: osdmap e115: 12 total, 12 up, 12 in 2024-02-02T15:46:17.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:46:17 smithi001 ceph-mon[35470]: pgmap v606: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 4.4 MiB/s wr, 353 op/s 2024-02-02T15:46:17.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:46:17 smithi001 ceph-mon[35470]: osdmap e115: 12 total, 12 up, 12 in 2024-02-02T15:46:19.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:46:19 smithi142 ceph-mon[45728]: pgmap v608: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.4 MiB/s wr, 293 op/s 2024-02-02T15:46:19.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:46:19 smithi071 ceph-mon[44707]: pgmap v608: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.4 MiB/s wr, 293 op/s 2024-02-02T15:46:19.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:46:19 smithi001 ceph-mon[35470]: pgmap v608: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.4 MiB/s wr, 293 op/s 2024-02-02T15:46:21.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:46:21 smithi071 ceph-mon[44707]: pgmap v609: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.4 MiB/s wr, 293 op/s 2024-02-02T15:46:21.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:46:21 smithi142 ceph-mon[45728]: pgmap v609: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.4 MiB/s wr, 293 op/s 2024-02-02T15:46:21.942 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:46:21 smithi001 ceph-mon[35470]: pgmap v609: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.4 MiB/s wr, 293 op/s 2024-02-02T15:46:23.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:46:22 smithi071 ceph-mon[44707]: pgmap v610: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 128 op/s 2024-02-02T15:46:23.169 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:46:22 smithi142 ceph-mon[45728]: pgmap v610: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 128 op/s 2024-02-02T15:46:23.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:46:22 smithi001 ceph-mon[35470]: pgmap v610: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 128 op/s 2024-02-02T15:46:25.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:46:25 smithi071 ceph-mon[44707]: pgmap v611: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.2 MiB/s wr, 103 op/s 2024-02-02T15:46:25.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:46:25 smithi142 ceph-mon[45728]: pgmap v611: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.2 MiB/s wr, 103 op/s 2024-02-02T15:46:25.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:46:25 smithi001 ceph-mon[35470]: pgmap v611: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.2 MiB/s wr, 103 op/s 2024-02-02T15:46:27.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:46:27 smithi071 ceph-mon[44707]: pgmap v612: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 845 KiB/s rd, 5.6 MiB/s wr, 149 op/s 2024-02-02T15:46:27.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:46:27 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:46:27.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:46:27 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:46:27.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:46:27 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:46:27.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:46:27 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:46:27.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:46:27 smithi071 ceph-mon[44707]: scrub summary: idle 2024-02-02T15:46:27.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:46:27 smithi142 ceph-mon[45728]: pgmap v612: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 845 KiB/s rd, 5.6 MiB/s wr, 149 op/s 2024-02-02T15:46:27.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:46:27 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:46:27.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:46:27 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:46:27.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:46:27 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:46:27.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:46:27 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:46:27.921 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:46:27 smithi142 ceph-mon[45728]: scrub summary: idle 2024-02-02T15:46:27.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:46:27 smithi001 ceph-mon[35470]: pgmap v612: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 845 KiB/s rd, 5.6 MiB/s wr, 149 op/s 2024-02-02T15:46:27.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:46:27 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:46:27.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:46:27 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:46:27.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:46:27 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:46:27.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:46:27 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:46:27.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:46:27 smithi001 ceph-mon[35470]: scrub summary: idle 2024-02-02T15:46:29.865 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:46:29 smithi071 ceph-mon[44707]: pgmap v613: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 983 KiB/s rd, 5.7 MiB/s wr, 169 op/s 2024-02-02T15:46:29.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:46:29 smithi142 ceph-mon[45728]: pgmap v613: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 983 KiB/s rd, 5.7 MiB/s wr, 169 op/s 2024-02-02T15:46:29.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:46:29 smithi001 ceph-mon[35470]: pgmap v613: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 983 KiB/s rd, 5.7 MiB/s wr, 169 op/s 2024-02-02T15:46:30.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:46:30 smithi071 ceph-mon[44707]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:46:30.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:46:30 smithi142 ceph-mon[45728]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:46:30.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:46:30 smithi001 ceph-mon[35470]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:46:31.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:46:31 smithi071 ceph-mon[44707]: pgmap v614: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 823 KiB/s rd, 5.2 MiB/s wr, 155 op/s 2024-02-02T15:46:31.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:46:31 smithi142 ceph-mon[45728]: pgmap v614: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 823 KiB/s rd, 5.2 MiB/s wr, 155 op/s 2024-02-02T15:46:31.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:46:31 smithi001 ceph-mon[35470]: pgmap v614: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 823 KiB/s rd, 5.2 MiB/s wr, 155 op/s 2024-02-02T15:46:33.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:46:32 smithi071 ceph-mon[44707]: pgmap v615: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 7.2 MiB/s wr, 221 op/s 2024-02-02T15:46:33.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:46:32 smithi142 ceph-mon[45728]: pgmap v615: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 7.2 MiB/s wr, 221 op/s 2024-02-02T15:46:33.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:46:32 smithi001 ceph-mon[35470]: pgmap v615: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 7.2 MiB/s wr, 221 op/s 2024-02-02T15:46:34.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:46:33 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:46:34.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:46:33 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:46:34.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:46:33 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-02T15:46:35.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:46:34 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:46:35.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:46:34 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:46:35.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:46:34 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:46:35.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:46:34 smithi071 ceph-mon[44707]: pgmap v616: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 6.5 MiB/s wr, 216 op/s 2024-02-02T15:46:35.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:46:34 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:46:35.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:46:34 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:46:35.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:46:34 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:46:35.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:46:34 smithi142 ceph-mon[45728]: pgmap v616: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 6.5 MiB/s wr, 216 op/s 2024-02-02T15:46:35.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:46:34 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-02T15:46:35.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:46:34 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-02T15:46:35.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:46:34 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' 2024-02-02T15:46:35.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:46:34 smithi001 ceph-mon[35470]: pgmap v616: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 6.5 MiB/s wr, 216 op/s 2024-02-02T15:46:36.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:46:35 smithi071 ceph-mon[44707]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:46:36.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:46:35 smithi142 ceph-mon[45728]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:46:36.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:46:35 smithi001 ceph-mon[35470]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:46:37.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:46:36 smithi071 ceph-mon[44707]: pgmap v617: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.4 MiB/s wr, 199 op/s 2024-02-02T15:46:37.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:46:36 smithi142 ceph-mon[45728]: pgmap v617: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.4 MiB/s wr, 199 op/s 2024-02-02T15:46:37.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:46:36 smithi001 ceph-mon[35470]: pgmap v617: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.4 MiB/s wr, 199 op/s 2024-02-02T15:46:38.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:46:37 smithi071 ceph-mon[44707]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:46:38.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:46:37 smithi142 ceph-mon[45728]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:46:38.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:46:37 smithi001 ceph-mon[35470]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:46:39.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:46:38 smithi071 ceph-mon[44707]: pgmap v618: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 790 KiB/s rd, 3.0 MiB/s wr, 134 op/s 2024-02-02T15:46:39.169 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:46:38 smithi142 ceph-mon[45728]: pgmap v618: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 790 KiB/s rd, 3.0 MiB/s wr, 134 op/s 2024-02-02T15:46:39.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:46:38 smithi001 ceph-mon[35470]: pgmap v618: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 790 KiB/s rd, 3.0 MiB/s wr, 134 op/s 2024-02-02T15:46:41.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:46:41 smithi071 ceph-mon[44707]: pgmap v619: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 2.1 MiB/s wr, 91 op/s 2024-02-02T15:46:41.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:46:41 smithi142 ceph-mon[45728]: pgmap v619: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 2.1 MiB/s wr, 91 op/s 2024-02-02T15:46:41.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:46:41 smithi001 ceph-mon[35470]: pgmap v619: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 2.1 MiB/s wr, 91 op/s 2024-02-02T15:46:42.018 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2024-02-02T15:46:42.232 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.233+0000 7f8f65600700 1 -- 172.21.15.1:0/4110648670 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8f601019a0 msgr2=0x7f8f601059e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:42.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.233+0000 7f8f65600700 1 --2- 172.21.15.1:0/4110648670 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8f601019a0 0x7f8f601059e0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f8f48009990 tx=0x7f8f48009ca0 comp rx=0 tx=0).stop 2024-02-02T15:46:42.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.233+0000 7f8f65600700 1 -- 172.21.15.1:0/4110648670 shutdown_connections 2024-02-02T15:46:42.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.233+0000 7f8f65600700 1 --2- 172.21.15.1:0/4110648670 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f60106110 0x7f8f6010dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:42.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.233+0000 7f8f65600700 1 --2- 172.21.15.1:0/4110648670 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f60100ff0 0x7f8f601013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:42.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.233+0000 7f8f65600700 1 --2- 172.21.15.1:0/4110648670 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8f601019a0 0x7f8f601059e0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:42.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.233+0000 7f8f65600700 1 -- 172.21.15.1:0/4110648670 >> 172.21.15.1:0/4110648670 conn(0x7f8f600fc810 msgr2=0x7f8f600fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:42.233 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.233+0000 7f8f65600700 1 -- 172.21.15.1:0/4110648670 shutdown_connections 2024-02-02T15:46:42.234 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.233+0000 7f8f65600700 1 -- 172.21.15.1:0/4110648670 wait complete. 2024-02-02T15:46:42.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.234+0000 7f8f65600700 1 Processor -- start 2024-02-02T15:46:42.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.234+0000 7f8f65600700 1 -- start start 2024-02-02T15:46:42.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.235+0000 7f8f65600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f60100ff0 0x7f8f60073040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:42.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.235+0000 7f8f65600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f60106110 0x7f8f60073580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:42.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.235+0000 7f8f65600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8f60073ac0 0x7f8f601a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:42.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.235+0000 7f8f65600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f601a56f0 con 0x7f8f60073ac0 2024-02-02T15:46:42.236 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.235+0000 7f8f65600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f601a5860 con 0x7f8f60106110 2024-02-02T15:46:42.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.235+0000 7f8f65600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f601a5a20 con 0x7f8f60100ff0 2024-02-02T15:46:42.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.235+0000 7f8f5ea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f60100ff0 0x7f8f60073040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:42.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.235+0000 7f8f5ea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f60100ff0 0x7f8f60073040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:54254/0 (socket says 172.21.15.1:54254) 2024-02-02T15:46:42.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.235+0000 7f8f5ea00700 1 -- 172.21.15.1:0/1062361891 learned_addr learned my addr 172.21.15.1:0/1062361891 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:46:42.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.235+0000 7f8f5f400700 1 --2- 172.21.15.1:0/1062361891 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8f60073ac0 0x7f8f601a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:42.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.235+0000 7f8f5e000700 1 --2- 172.21.15.1:0/1062361891 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f60106110 0x7f8f60073580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:42.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.236+0000 7f8f5ea00700 1 -- 172.21.15.1:0/1062361891 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f60106110 msgr2=0x7f8f60073580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:42.237 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.236+0000 7f8f5ea00700 1 --2- 172.21.15.1:0/1062361891 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f60106110 0x7f8f60073580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:42.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.236+0000 7f8f5ea00700 1 -- 172.21.15.1:0/1062361891 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8f60073ac0 msgr2=0x7f8f601a50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:42.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.236+0000 7f8f5ea00700 1 --2- 172.21.15.1:0/1062361891 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8f60073ac0 0x7f8f601a50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:42.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.236+0000 7f8f5ea00700 1 -- 172.21.15.1:0/1062361891 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f480095f0 con 0x7f8f60100ff0 2024-02-02T15:46:42.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.236+0000 7f8f5ea00700 1 --2- 172.21.15.1:0/1062361891 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f60100ff0 0x7f8f60073040 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f8f5000e9a0 tx=0x7f8f5000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:42.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.236+0000 7f8f57400700 1 -- 172.21.15.1:0/1062361891 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f50009790 con 0x7f8f60100ff0 2024-02-02T15:46:42.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.236+0000 7f8f65600700 1 -- 172.21.15.1:0/1062361891 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f601a5d00 con 0x7f8f60100ff0 2024-02-02T15:46:42.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.236+0000 7f8f57400700 1 -- 172.21.15.1:0/1062361891 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f50010430 con 0x7f8f60100ff0 2024-02-02T15:46:42.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.236+0000 7f8f57400700 1 -- 172.21.15.1:0/1062361891 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f50018680 con 0x7f8f60100ff0 2024-02-02T15:46:42.238 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.236+0000 7f8f65600700 1 -- 172.21.15.1:0/1062361891 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f601a6200 con 0x7f8f60100ff0 2024-02-02T15:46:42.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.239+0000 7f8f57400700 1 -- 172.21.15.1:0/1062361891 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f8f50010810 con 0x7f8f60100ff0 2024-02-02T15:46:42.239 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.241+0000 7f8f790bab80 1 Processor -- start 2024-02-02T15:46:42.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.241+0000 7f8f57400700 1 --2- 172.21.15.1:0/1062361891 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f8f400a0710 0x7f8f400a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:42.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.241+0000 7f8f57400700 1 -- 172.21.15.1:0/1062361891 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f8f50014070 con 0x7f8f60100ff0 2024-02-02T15:46:42.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.241+0000 7f8f790bab80 1 -- start start 2024-02-02T15:46:42.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.241+0000 7f8f5e000700 1 --2- 172.21.15.1:0/1062361891 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f8f400a0710 0x7f8f400a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:42.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.241+0000 7f8f790bab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e4a2d7d010 0x55e4a2d7d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:42.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.241+0000 7f8f790bab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e4a2d7d930 0x55e4a2d882a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:42.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.241+0000 7f8f790bab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e4a2d887e0 0x55e4a2d8abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:42.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.241+0000 7f8f790bab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4a2cf14c0 con 0x55e4a2d7d010 2024-02-02T15:46:42.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.241+0000 7f8f790bab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4a2cd5aa0 con 0x55e4a2d7d930 2024-02-02T15:46:42.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.241+0000 7f8f790bab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4a2cd3460 con 0x55e4a2d887e0 2024-02-02T15:46:42.243 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.242+0000 7f8f5ea00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e4a2d7d010 0x55e4a2d7d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:42.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.242+0000 7f8f5f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e4a2d887e0 0x55e4a2d8abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:42.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.242+0000 7f8f5ea00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e4a2d7d010 0x55e4a2d7d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59628/0 (socket says 172.21.15.1:59628) 2024-02-02T15:46:42.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.242+0000 7f8f5ea00700 1 -- 172.21.15.1:0/2292167206 learned_addr learned my addr 172.21.15.1:0/2292167206 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:46:42.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.242+0000 7f8f5f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e4a2d887e0 0x55e4a2d8abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:54262/0 (socket says 172.21.15.1:54262) 2024-02-02T15:46:42.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.242+0000 7f8f5e000700 1 --2- 172.21.15.1:0/2292167206 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e4a2d7d930 0x55e4a2d882a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:42.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.242+0000 7f8f5f400700 1 -- 172.21.15.1:0/2292167206 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e4a2d7d930 msgr2=0x55e4a2d882a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:42.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.242+0000 7f8f5f400700 1 --2- 172.21.15.1:0/2292167206 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e4a2d7d930 0x55e4a2d882a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:42.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.242+0000 7f8f5f400700 1 -- 172.21.15.1:0/2292167206 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e4a2d7d010 msgr2=0x55e4a2d7d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:42.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.242+0000 7f8f5f400700 1 --2- 172.21.15.1:0/2292167206 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e4a2d7d010 0x55e4a2d7d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:42.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.242+0000 7f8f5f400700 1 -- 172.21.15.1:0/2292167206 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e4a2b503b0 con 0x55e4a2d887e0 2024-02-02T15:46:42.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.242+0000 7f8f5ea00700 1 --2- 172.21.15.1:0/2292167206 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e4a2d7d010 0x55e4a2d7d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:46:42.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.242+0000 7f8f5e000700 1 --2- 172.21.15.1:0/1062361891 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f8f400a0710 0x7f8f400a2bd0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f8f600747c0 tx=0x7f8f48005040 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:42.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.242+0000 7f8f5f400700 1 --2- 172.21.15.1:0/2292167206 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e4a2d887e0 0x55e4a2d8abd0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f8f580115b0 tx=0x7f8f58009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d04d6b5d545547af server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:42.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.243+0000 7f8f4f400700 1 -- 172.21.15.1:0/2292167206 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f5800e040 con 0x55e4a2d887e0 2024-02-02T15:46:42.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.243+0000 7f8f4f400700 1 -- 172.21.15.1:0/2292167206 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f5800f040 con 0x55e4a2d887e0 2024-02-02T15:46:42.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.243+0000 7f8f4f400700 1 -- 172.21.15.1:0/2292167206 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f58004430 con 0x55e4a2d887e0 2024-02-02T15:46:42.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.243+0000 7f8f790bab80 1 -- 172.21.15.1:0/2292167206 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e4a2d887e0 msgr2=0x55e4a2d8abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:42.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.243+0000 7f8f790bab80 1 --2- 172.21.15.1:0/2292167206 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e4a2d887e0 0x55e4a2d8abd0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f8f580115b0 tx=0x7f8f58009920 comp rx=0 tx=0).stop 2024-02-02T15:46:42.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.244+0000 7f8f790bab80 1 -- 172.21.15.1:0/2292167206 shutdown_connections 2024-02-02T15:46:42.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.244+0000 7f8f790bab80 1 --2- 172.21.15.1:0/2292167206 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e4a2d887e0 0x55e4a2d8abd0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:42.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.244+0000 7f8f790bab80 1 --2- 172.21.15.1:0/2292167206 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e4a2d7d010 0x55e4a2d7d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:42.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.244+0000 7f8f790bab80 1 --2- 172.21.15.1:0/2292167206 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e4a2d7d930 0x55e4a2d882a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:42.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.244+0000 7f8f790bab80 1 -- 172.21.15.1:0/2292167206 >> 172.21.15.1:0/2292167206 conn(0x55e4a2cdcdb0 msgr2=0x55e4a2d793b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:42.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.244+0000 7f8f790bab80 1 -- 172.21.15.1:0/2292167206 shutdown_connections 2024-02-02T15:46:42.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.244+0000 7f8f790bab80 1 -- 172.21.15.1:0/2292167206 wait complete. 2024-02-02T15:46:42.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.244+0000 7f8f790bab80 1 -- start start 2024-02-02T15:46:42.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.244+0000 7f8f790bab80 1 Processor -- start 2024-02-02T15:46:42.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.245+0000 7f8f790bab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e4a2d7d010 0x55e4a2d88140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:42.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.245+0000 7f8f790bab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e4a2d7d930 0x55e4a2d85430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:42.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.245+0000 7f8f790bab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e4a2d887e0 0x55e4a2d85a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:42.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.245+0000 7f8f790bab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4a2cced50 con 0x55e4a2d887e0 2024-02-02T15:46:42.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.245+0000 7f8f790bab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4a2c6ad80 con 0x55e4a2d7d010 2024-02-02T15:46:42.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.245+0000 7f8f790bab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4a2c6aa20 con 0x55e4a2d7d930 2024-02-02T15:46:42.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.245+0000 7f8f790bab80 10 client.0 _ll_register_callbacks cb 0x55e4a2c6c290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:46:42.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.245+0000 7f8f5f400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e4a2d887e0 0x55e4a2d85a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:42.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.245+0000 7f8f5ea00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e4a2d7d010 0x55e4a2d88140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:42.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.245+0000 7f8f5ea00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e4a2d7d010 0x55e4a2d88140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:44408/0 (socket says 172.21.15.1:44408) 2024-02-02T15:46:42.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.245+0000 7f8f5ea00700 1 -- 172.21.15.1:0/3561528358 learned_addr learned my addr 172.21.15.1:0/3561528358 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:46:42.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.245+0000 7f8f5f400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e4a2d887e0 0x55e4a2d85a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59644/0 (socket says 172.21.15.1:59644) 2024-02-02T15:46:42.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.245+0000 7f8f5e000700 1 --2- 172.21.15.1:0/3561528358 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e4a2d7d930 0x55e4a2d85430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:42.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.246+0000 7f8f5ea00700 1 -- 172.21.15.1:0/3561528358 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e4a2d7d930 msgr2=0x55e4a2d85430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:42.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.246+0000 7f8f5ea00700 1 --2- 172.21.15.1:0/3561528358 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e4a2d7d930 0x55e4a2d85430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:42.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.246+0000 7f8f5ea00700 1 -- 172.21.15.1:0/3561528358 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e4a2d887e0 msgr2=0x55e4a2d85a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:42.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.246+0000 7f8f5ea00700 1 --2- 172.21.15.1:0/3561528358 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e4a2d887e0 0x55e4a2d85a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:42.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.246+0000 7f8f5ea00700 1 -- 172.21.15.1:0/3561528358 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f601a6200 con 0x55e4a2d7d010 2024-02-02T15:46:42.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.246+0000 7f8f5f400700 1 --2- 172.21.15.1:0/3561528358 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e4a2d887e0 0x55e4a2d85a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:46:42.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.246+0000 7f8f5ea00700 1 --2- 172.21.15.1:0/3561528358 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e4a2d7d010 0x55e4a2d88140 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f8f50099100 tx=0x7f8f50098de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=853cb2d49310c33 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:42.257 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.246+0000 7f8f47e00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:46:42.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.246+0000 7f8f47e00700 1 -- 172.21.15.1:0/3561528358 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f50098af0 con 0x55e4a2d7d010 2024-02-02T15:46:42.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.246+0000 7f8f790bab80 1 -- 172.21.15.1:0/3561528358 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e4a2d710f0 con 0x55e4a2d7d010 2024-02-02T15:46:42.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.246+0000 7f8f47e00700 1 -- 172.21.15.1:0/3561528358 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f8f50099b00 con 0x55e4a2d7d010 2024-02-02T15:46:42.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.246+0000 7f8f47e00700 1 -- 172.21.15.1:0/3561528358 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f50099400 con 0x55e4a2d7d010 2024-02-02T15:46:42.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.246+0000 7f8f790bab80 10 client.30578 fetch_fsmap learned FSMap version 81 2024-02-02T15:46:42.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.246+0000 7f8f790bab80 1 -- 172.21.15.1:0/3561528358 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) v3 -- 0x7f8f58011230 con 0x55e4a2d7d010 2024-02-02T15:46:42.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.247+0000 7f8f47e00700 1 -- 172.21.15.1:0/3561528358 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e4a2d710f0 con 0x55e4a2d7d010 2024-02-02T15:46:42.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.247+0000 7f8f47e00700 1 -- 172.21.15.1:0/3561528358 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 81) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f8f500a0fe0 con 0x55e4a2d7d010 2024-02-02T15:46:42.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.247+0000 7f8f790bab80 10 client.30578 fetch_fsmap finished waiting for FSMap version 81 2024-02-02T15:46:42.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.247+0000 7f8f790bab80 10 client.30578 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:46:42.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.247+0000 7f8f790bab80 20 client.30578 populate_metadata read hostname 'smithi001' 2024-02-02T15:46:42.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.247+0000 7f8f790bab80 1 --2- 172.21.15.1:0/3561528358 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55e4a2d98260 0x55e4a2d9a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:42.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.247+0000 7f8f790bab80 4 client.30578 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:46:42.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.247+0000 7f8f790bab80 1 -- 172.21.15.1:0/3561528358 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e4a2cc7dd0 con 0x55e4a2d98260 2024-02-02T15:46:42.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.247+0000 7f8f5e000700 1 --2- 172.21.15.1:0/3561528358 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55e4a2d98260 0x55e4a2d9a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:42.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.248+0000 7f8f5e000700 1 --2- 172.21.15.1:0/3561528358 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55e4a2d98260 0x55e4a2d9a680 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4d91c8b8060fd3b server_cookie=2f92449a1963569c in_seq=0 out_seq=0 2024-02-02T15:46:42.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.248+0000 7f8f47e00700 10 client.30578 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:46:42.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.248+0000 7f8f47e00700 1 -- 172.21.15.1:0/3561528358 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55e4a2cc7dd0 con 0x55e4a2d98260 2024-02-02T15:46:42.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.248+0000 7f8f47e00700 10 client.30578 handle_command_reply: tid=0 2024-02-02T15:46:42.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.249+0000 7f8f790bab80 1 client.30578 shutdown 2024-02-02T15:46:42.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.249+0000 7f8f790bab80 1 -- 172.21.15.1:0/3561528358 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e4a2d7d010 msgr2=0x55e4a2d88140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:42.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.249+0000 7f8f790bab80 1 --2- 172.21.15.1:0/3561528358 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e4a2d7d010 0x55e4a2d88140 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f8f50099100 tx=0x7f8f50098de0 comp rx=0 tx=0).stop 2024-02-02T15:46:42.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.249+0000 7f8f790bab80 1 -- 172.21.15.1:0/3561528358 shutdown_connections 2024-02-02T15:46:42.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.249+0000 7f8f790bab80 1 --2- 172.21.15.1:0/3561528358 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55e4a2d98260 0x55e4a2d9a680 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:42.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.249+0000 7f8f790bab80 1 --2- 172.21.15.1:0/3561528358 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e4a2d887e0 0x55e4a2d85a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:42.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.249+0000 7f8f790bab80 1 --2- 172.21.15.1:0/3561528358 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e4a2d7d010 0x55e4a2d88140 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:42.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.249+0000 7f8f47e00700 0 client.30578 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:46:42.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.249+0000 7f8f790bab80 1 --2- 172.21.15.1:0/3561528358 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e4a2d7d930 0x55e4a2d85430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:42.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.249+0000 7f8f790bab80 1 -- 172.21.15.1:0/3561528358 >> 172.21.15.1:0/3561528358 conn(0x55e4a2cdcdb0 msgr2=0x55e4a2d78fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:42.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.249+0000 7f8f790bab80 1 -- 172.21.15.1:0/3561528358 shutdown_connections 2024-02-02T15:46:42.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.249+0000 7f8f790bab80 1 -- 172.21.15.1:0/3561528358 wait complete. 2024-02-02T15:46:42.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.249+0000 7f8f790bab80 20 client.30578 trim_cache size 0 max 16384 2024-02-02T15:46:42.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.266+0000 7f8f790bab80 1 Processor -- start 2024-02-02T15:46:42.266 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.266+0000 7f8f790bab80 1 -- start start 2024-02-02T15:46:42.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.267+0000 7f8f790bab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e4a2d7d010 0x55e4a2d84bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:42.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.267+0000 7f8f790bab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e4a2d7d930 0x55e4a2d87ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:42.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.267+0000 7f8f790bab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e4a2d887e0 0x55e4a2d826a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:42.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.267+0000 7f8f790bab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4a2b526a0 con 0x55e4a2d7d930 2024-02-02T15:46:42.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.267+0000 7f8f790bab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4a2d81280 con 0x55e4a2d7d010 2024-02-02T15:46:42.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.267+0000 7f8f790bab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4a2d94220 con 0x55e4a2d887e0 2024-02-02T15:46:42.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.267+0000 7f8f5e000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e4a2d7d930 0x55e4a2d87ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:42.267 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.267+0000 7f8f5f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e4a2d887e0 0x55e4a2d826a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:42.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.267+0000 7f8f5e000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e4a2d7d930 0x55e4a2d87ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59660/0 (socket says 172.21.15.1:59660) 2024-02-02T15:46:42.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.267+0000 7f8f5f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e4a2d887e0 0x55e4a2d826a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:54278/0 (socket says 172.21.15.1:54278) 2024-02-02T15:46:42.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.267+0000 7f8f5e000700 1 -- 172.21.15.1:0/2499886565 learned_addr learned my addr 172.21.15.1:0/2499886565 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:46:42.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.267+0000 7f8f5ea00700 1 --2- 172.21.15.1:0/2499886565 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e4a2d7d010 0x55e4a2d84bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:42.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.267+0000 7f8f5e000700 1 -- 172.21.15.1:0/2499886565 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e4a2d887e0 msgr2=0x55e4a2d826a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:42.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.267+0000 7f8f5e000700 1 --2- 172.21.15.1:0/2499886565 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e4a2d887e0 0x55e4a2d826a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:42.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.267+0000 7f8f5e000700 1 -- 172.21.15.1:0/2499886565 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e4a2d7d010 msgr2=0x55e4a2d84bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:42.268 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.267+0000 7f8f5e000700 1 --2- 172.21.15.1:0/2499886565 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e4a2d7d010 0x55e4a2d84bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:42.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.268+0000 7f8f5e000700 1 -- 172.21.15.1:0/2499886565 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e4a2afcb60 con 0x55e4a2d7d930 2024-02-02T15:46:42.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.268+0000 7f8f5f400700 1 --2- 172.21.15.1:0/2499886565 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e4a2d887e0 0x55e4a2d826a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:46:42.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.268+0000 7f8f5e000700 1 --2- 172.21.15.1:0/2499886565 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e4a2d7d930 0x55e4a2d87ce0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f8f48009990 tx=0x7f8f48022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d06e66bfbe8159ca server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:42.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.268+0000 7f8f4fe00700 1 -- 172.21.15.1:0/2499886565 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f48035170 con 0x55e4a2d7d930 2024-02-02T15:46:42.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.268+0000 7f8f4fe00700 1 -- 172.21.15.1:0/2499886565 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f4801e040 con 0x55e4a2d7d930 2024-02-02T15:46:42.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.268+0000 7f8f4fe00700 1 -- 172.21.15.1:0/2499886565 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f480048c0 con 0x55e4a2d7d930 2024-02-02T15:46:42.273 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.269+0000 7f8f790bab80 1 -- 172.21.15.1:0/2499886565 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e4a2d7d930 msgr2=0x55e4a2d87ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:42.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.269+0000 7f8f790bab80 1 --2- 172.21.15.1:0/2499886565 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e4a2d7d930 0x55e4a2d87ce0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f8f48009990 tx=0x7f8f48022000 comp rx=0 tx=0).stop 2024-02-02T15:46:42.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.269+0000 7f8f790bab80 1 -- 172.21.15.1:0/2499886565 shutdown_connections 2024-02-02T15:46:42.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.269+0000 7f8f790bab80 1 --2- 172.21.15.1:0/2499886565 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e4a2d887e0 0x55e4a2d826a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:42.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.269+0000 7f8f790bab80 1 --2- 172.21.15.1:0/2499886565 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e4a2d7d010 0x55e4a2d84bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:42.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.269+0000 7f8f790bab80 1 --2- 172.21.15.1:0/2499886565 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e4a2d7d930 0x55e4a2d87ce0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:42.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.269+0000 7f8f790bab80 1 -- 172.21.15.1:0/2499886565 >> 172.21.15.1:0/2499886565 conn(0x55e4a2cdcdb0 msgr2=0x55e4a2d80740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:42.274 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.269+0000 7f8f790bab80 1 -- 172.21.15.1:0/2499886565 shutdown_connections 2024-02-02T15:46:42.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.269+0000 7f8f790bab80 1 -- 172.21.15.1:0/2499886565 wait complete. 2024-02-02T15:46:42.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.269+0000 7f8f790bab80 1 -- start start 2024-02-02T15:46:42.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.269+0000 7f8f790bab80 1 Processor -- start 2024-02-02T15:46:42.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.270+0000 7f8f790bab80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e4a2d7d010 0x55e4a2d8f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:42.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.270+0000 7f8f790bab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e4a2d7d930 0x55e4a2d8d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:42.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.270+0000 7f8f790bab80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e4a2d887e0 0x55e4a2d8de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:42.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.270+0000 7f8f790bab80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4a2d8e590 con 0x55e4a2d7d010 2024-02-02T15:46:42.275 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.270+0000 7f8f790bab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4a2de9d90 con 0x55e4a2d7d930 2024-02-02T15:46:42.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.270+0000 7f8f790bab80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e4a2de9ed0 con 0x55e4a2d887e0 2024-02-02T15:46:42.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.270+0000 7f8f790bab80 10 client.0 _ll_register_callbacks cb 0x55e4a2c6c290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:46:42.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.270+0000 7f8f5ea00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e4a2d7d010 0x55e4a2d8f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:42.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.270+0000 7f8f5ea00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e4a2d7d010 0x55e4a2d8f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59676/0 (socket says 172.21.15.1:59676) 2024-02-02T15:46:42.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.270+0000 7f8f5ea00700 1 -- 172.21.15.1:0/1759616674 learned_addr learned my addr 172.21.15.1:0/1759616674 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:46:42.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.270+0000 7f8f5e000700 1 --2- 172.21.15.1:0/1759616674 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e4a2d7d930 0x55e4a2d8d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:42.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.270+0000 7f8f5f400700 1 --2- 172.21.15.1:0/1759616674 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e4a2d887e0 0x55e4a2d8de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:42.276 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.271+0000 7f8f5ea00700 1 -- 172.21.15.1:0/1759616674 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e4a2d887e0 msgr2=0x55e4a2d8de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:42.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.271+0000 7f8f5ea00700 1 --2- 172.21.15.1:0/1759616674 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e4a2d887e0 0x55e4a2d8de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:42.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.271+0000 7f8f5ea00700 1 -- 172.21.15.1:0/1759616674 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e4a2d7d930 msgr2=0x55e4a2d8d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:42.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.271+0000 7f8f5ea00700 1 --2- 172.21.15.1:0/1759616674 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e4a2d7d930 0x55e4a2d8d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:42.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.271+0000 7f8f5ea00700 1 -- 172.21.15.1:0/1759616674 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e4a2b503b0 con 0x55e4a2d7d010 2024-02-02T15:46:42.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.271+0000 7f8f5f400700 1 --2- 172.21.15.1:0/1759616674 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e4a2d887e0 0x55e4a2d8de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:46:42.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.271+0000 7f8f5ea00700 1 --2- 172.21.15.1:0/1759616674 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e4a2d7d010 0x55e4a2d8f410 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f8f500a1340 tx=0x7f8f50010bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b142610634e322b server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:42.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.271+0000 7f8f47e00700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:46:42.277 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.271+0000 7f8f47e00700 1 -- 172.21.15.1:0/1759616674 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f5007c070 con 0x55e4a2d7d010 2024-02-02T15:46:42.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.271+0000 7f8f47e00700 1 -- 172.21.15.1:0/1759616674 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f8f500b7210 con 0x55e4a2d7d010 2024-02-02T15:46:42.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.271+0000 7f8f790bab80 1 -- 172.21.15.1:0/1759616674 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8f5800f040 con 0x55e4a2d7d010 2024-02-02T15:46:42.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.271+0000 7f8f47e00700 1 -- 172.21.15.1:0/1759616674 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f50095740 con 0x55e4a2d7d010 2024-02-02T15:46:42.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.271+0000 7f8f790bab80 10 client.20280 fetch_fsmap learned FSMap version 81 2024-02-02T15:46:42.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.271+0000 7f8f790bab80 1 -- 172.21.15.1:0/1759616674 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) v3 -- 0x7f8f6010e500 con 0x55e4a2d7d010 2024-02-02T15:46:42.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.272+0000 7f8f47e00700 1 -- 172.21.15.1:0/1759616674 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8f500a5020 con 0x55e4a2d7d010 2024-02-02T15:46:42.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.272+0000 7f8f47e00700 1 -- 172.21.15.1:0/1759616674 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 81) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f8f5007f030 con 0x55e4a2d7d010 2024-02-02T15:46:42.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.272+0000 7f8f790bab80 10 client.20280 fetch_fsmap finished waiting for FSMap version 81 2024-02-02T15:46:42.278 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.272+0000 7f8f790bab80 10 client.20280 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:46:42.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.272+0000 7f8f790bab80 20 client.20280 populate_metadata read hostname 'smithi001' 2024-02-02T15:46:42.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.272+0000 7f8f790bab80 1 --2- 172.21.15.1:0/1759616674 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55e4a2deb250 0x55e4a2ded670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:42.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.272+0000 7f8f790bab80 4 client.20280 mds_command: new command op to 24557 tid=0[{"prefix": "scrub status"}] 2024-02-02T15:46:42.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.272+0000 7f8f790bab80 1 -- 172.21.15.1:0/1759616674 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e4a2c8df50 con 0x55e4a2deb250 2024-02-02T15:46:42.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.272+0000 7f8f5e000700 1 --2- 172.21.15.1:0/1759616674 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55e4a2deb250 0x55e4a2ded670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:42.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.273+0000 7f8f5e000700 1 --2- 172.21.15.1:0/1759616674 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55e4a2deb250 0x55e4a2ded670 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc055bca1788fcaf server_cookie=9e54d2552c28fb2b in_seq=0 out_seq=0 2024-02-02T15:46:42.279 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.273+0000 7f8f47e00700 10 client.20280 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:46:42.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.273+0000 7f8f47e00700 1 -- 172.21.15.1:0/1759616674 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55e4a2c8df50 con 0x55e4a2deb250 2024-02-02T15:46:42.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.273+0000 7f8f47e00700 10 client.20280 handle_command_reply: tid=0 2024-02-02T15:46:42.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.273+0000 7f8f790bab80 1 client.20280 shutdown 2024-02-02T15:46:42.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.273+0000 7f8f790bab80 1 -- 172.21.15.1:0/1759616674 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e4a2d7d010 msgr2=0x55e4a2d8f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:42.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.273+0000 7f8f790bab80 1 --2- 172.21.15.1:0/1759616674 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e4a2d7d010 0x55e4a2d8f410 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f8f500a1340 tx=0x7f8f50010bb0 comp rx=0 tx=0).stop 2024-02-02T15:46:42.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.273+0000 7f8f790bab80 1 -- 172.21.15.1:0/1759616674 shutdown_connections 2024-02-02T15:46:42.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.273+0000 7f8f790bab80 1 --2- 172.21.15.1:0/1759616674 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x55e4a2deb250 0x55e4a2ded670 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:42.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.273+0000 7f8f790bab80 1 --2- 172.21.15.1:0/1759616674 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55e4a2d887e0 0x55e4a2d8de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:42.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.273+0000 7f8f790bab80 1 --2- 172.21.15.1:0/1759616674 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55e4a2d7d010 0x55e4a2d8f410 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:42.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.273+0000 7f8f47e00700 0 client.20280 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:46:42.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.273+0000 7f8f790bab80 1 --2- 172.21.15.1:0/1759616674 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e4a2d7d930 0x55e4a2d8d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:42.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.273+0000 7f8f790bab80 1 -- 172.21.15.1:0/1759616674 >> 172.21.15.1:0/1759616674 conn(0x55e4a2cdcdb0 msgr2=0x55e4a2d785c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:42.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.273+0000 7f8f790bab80 1 -- 172.21.15.1:0/1759616674 shutdown_connections 2024-02-02T15:46:42.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.273+0000 7f8f790bab80 1 -- 172.21.15.1:0/1759616674 wait complete. 2024-02-02T15:46:42.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.274+0000 7f8f790bab80 20 client.20280 trim_cache size 0 max 16384 2024-02-02T15:46:42.282 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:46:42.282 INFO:teuthology.orchestra.run.smithi001.stdout: "status": "no active scrubs running", 2024-02-02T15:46:42.282 INFO:teuthology.orchestra.run.smithi001.stdout: "scrubs": {} 2024-02-02T15:46:42.282 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:46:42.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.274+0000 7f8f65600700 1 -- 172.21.15.1:0/1062361891 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f8f400a0710 msgr2=0x7f8f400a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:42.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.274+0000 7f8f65600700 1 --2- 172.21.15.1:0/1062361891 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f8f400a0710 0x7f8f400a2bd0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f8f600747c0 tx=0x7f8f48005040 comp rx=0 tx=0).stop 2024-02-02T15:46:42.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.274+0000 7f8f65600700 1 -- 172.21.15.1:0/1062361891 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f60100ff0 msgr2=0x7f8f60073040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:42.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.274+0000 7f8f65600700 1 --2- 172.21.15.1:0/1062361891 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f60100ff0 0x7f8f60073040 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f8f5000e9a0 tx=0x7f8f5000ecb0 comp rx=0 tx=0).stop 2024-02-02T15:46:42.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.274+0000 7f8f65600700 1 -- 172.21.15.1:0/1062361891 shutdown_connections 2024-02-02T15:46:42.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.274+0000 7f8f65600700 1 --2- 172.21.15.1:0/1062361891 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f8f400a0710 0x7f8f400a2bd0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:42.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.274+0000 7f8f65600700 1 --2- 172.21.15.1:0/1062361891 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f8f60073ac0 0x7f8f601a50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:42.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.274+0000 7f8f65600700 1 --2- 172.21.15.1:0/1062361891 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8f60100ff0 0x7f8f60073040 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:42.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.274+0000 7f8f65600700 1 --2- 172.21.15.1:0/1062361891 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8f60106110 0x7f8f60073580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:42.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.274+0000 7f8f65600700 1 -- 172.21.15.1:0/1062361891 >> 172.21.15.1:0/1062361891 conn(0x7f8f600fc810 msgr2=0x7f8f600fd2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:42.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.274+0000 7f8f65600700 1 -- 172.21.15.1:0/1062361891 shutdown_connections 2024-02-02T15:46:42.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.274+0000 7f8f65600700 1 -- 172.21.15.1:0/1062361891 wait complete. 2024-02-02T15:46:42.291 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-02-02T15:46:42.291 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T15:46:42.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.490+0000 7f3fc1200700 1 -- 172.21.15.1:0/1330438566 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3fbc101920 msgr2=0x7f3fbc105960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:42.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.490+0000 7f3fc1200700 1 --2- 172.21.15.1:0/1330438566 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3fbc101920 0x7f3fbc105960 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f3fa4009990 tx=0x7f3fa4009ca0 comp rx=0 tx=0).stop 2024-02-02T15:46:42.489 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.490+0000 7f3fc1200700 1 -- 172.21.15.1:0/1330438566 shutdown_connections 2024-02-02T15:46:42.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.490+0000 7f3fc1200700 1 --2- 172.21.15.1:0/1330438566 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3fbc106090 0x7f3fbc10dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:42.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.490+0000 7f3fc1200700 1 --2- 172.21.15.1:0/1330438566 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fbc100f70 0x7f3fbc101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:42.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.490+0000 7f3fc1200700 1 --2- 172.21.15.1:0/1330438566 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3fbc101920 0x7f3fbc105960 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:42.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.490+0000 7f3fc1200700 1 -- 172.21.15.1:0/1330438566 >> 172.21.15.1:0/1330438566 conn(0x7f3fbc0fc810 msgr2=0x7f3fbc0fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:42.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.490+0000 7f3fc1200700 1 -- 172.21.15.1:0/1330438566 shutdown_connections 2024-02-02T15:46:42.490 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.490+0000 7f3fc1200700 1 -- 172.21.15.1:0/1330438566 wait complete. 2024-02-02T15:46:42.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.491+0000 7f3fc1200700 1 Processor -- start 2024-02-02T15:46:42.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.492+0000 7f3fc1200700 1 -- start start 2024-02-02T15:46:42.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.492+0000 7f3fc1200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fbc100f70 0x7f3fbc199000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:42.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.492+0000 7f3fc1200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3fbc106090 0x7f3fbc199540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:42.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.492+0000 7f3fc1200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3fbc19d9a0 0x7f3fbc19de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:42.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.492+0000 7f3fc1200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fbc19e420 con 0x7f3fbc106090 2024-02-02T15:46:42.493 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.492+0000 7f3fc1200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fbc19e590 con 0x7f3fbc19d9a0 2024-02-02T15:46:42.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.492+0000 7f3fc1200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fbc19e750 con 0x7f3fbc100f70 2024-02-02T15:46:42.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.492+0000 7f3fb9400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3fbc106090 0x7f3fbc199540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:42.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.492+0000 7f3fb9400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3fbc106090 0x7f3fbc199540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:59688/0 (socket says 172.21.15.1:59688) 2024-02-02T15:46:42.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.493+0000 7f3fb9400700 1 -- 172.21.15.1:0/4271448383 learned_addr learned my addr 172.21.15.1:0/4271448383 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:46:42.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.492+0000 7f3fb9e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fbc100f70 0x7f3fbc199000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:42.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.493+0000 7f3fba800700 1 --2- 172.21.15.1:0/4271448383 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3fbc19d9a0 0x7f3fbc19de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:42.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.493+0000 7f3fb9400700 1 -- 172.21.15.1:0/4271448383 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fbc100f70 msgr2=0x7f3fbc199000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:42.494 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.493+0000 7f3fb9400700 1 --2- 172.21.15.1:0/4271448383 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fbc100f70 0x7f3fbc199000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:42.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.493+0000 7f3fb9400700 1 -- 172.21.15.1:0/4271448383 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3fbc19d9a0 msgr2=0x7f3fbc19de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:42.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.493+0000 7f3fb9400700 1 --2- 172.21.15.1:0/4271448383 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3fbc19d9a0 0x7f3fbc19de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:42.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.493+0000 7f3fb9400700 1 -- 172.21.15.1:0/4271448383 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fa40095f0 con 0x7f3fbc106090 2024-02-02T15:46:42.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.493+0000 7f3fb9400700 1 --2- 172.21.15.1:0/4271448383 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3fbc106090 0x7f3fbc199540 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f3fa4000c00 tx=0x7f3fa4005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:42.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.493+0000 7f3fb2a00700 1 -- 172.21.15.1:0/4271448383 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa401d070 con 0x7f3fbc106090 2024-02-02T15:46:42.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.493+0000 7f3fb2a00700 1 -- 172.21.15.1:0/4271448383 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3fa4021410 con 0x7f3fbc106090 2024-02-02T15:46:42.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.493+0000 7f3fc1200700 1 -- 172.21.15.1:0/4271448383 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fbc1a5c50 con 0x7f3fbc106090 2024-02-02T15:46:42.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.494+0000 7f3fb2a00700 1 -- 172.21.15.1:0/4271448383 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa4017650 con 0x7f3fbc106090 2024-02-02T15:46:42.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.494+0000 7f3fc1200700 1 -- 172.21.15.1:0/4271448383 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fbc1a60d0 con 0x7f3fbc106090 2024-02-02T15:46:42.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.495+0000 7f3fc1200700 1 -- 172.21.15.1:0/4271448383 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fbc067bc0 con 0x7f3fbc106090 2024-02-02T15:46:42.497 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.498+0000 7f3fb2a00700 1 -- 172.21.15.1:0/4271448383 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f3fa4021580 con 0x7f3fbc106090 2024-02-02T15:46:42.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.499+0000 7f3fb2a00700 1 --2- 172.21.15.1:0/4271448383 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f3f9c0a0710 0x7f3f9c0a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:42.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.499+0000 7f3fb2a00700 1 -- 172.21.15.1:0/4271448383 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f3fa40cd9f0 con 0x7f3fbc106090 2024-02-02T15:46:42.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.500+0000 7f3fb9e00700 1 --2- 172.21.15.1:0/4271448383 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f3f9c0a0710 0x7f3f9c0a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:42.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.500+0000 7f3fb9e00700 1 --2- 172.21.15.1:0/4271448383 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f3f9c0a0710 0x7f3f9c0a2bd0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f3fac009c60 tx=0x7f3fac009260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:42.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.502+0000 7f3fb2a00700 1 -- 172.21.15.1:0/4271448383 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f3fa4096ee0 con 0x7f3fbc106090 2024-02-02T15:46:42.761 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.763+0000 7f3fc1200700 1 -- 172.21.15.1:0/4271448383 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3fbc19a350 con 0x7f3fbc106090 2024-02-02T15:46:42.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.763+0000 7f3fb2a00700 1 -- 172.21.15.1:0/4271448383 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+10869 (secure 0 0 0) 0x7f3fa409a500 con 0x7f3fbc106090 2024-02-02T15:46:42.763 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:46:42.763 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":81,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2922240448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2922240448},{"type":"v1","addr":"172.21.15.1:6835","nonce":2922240448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2729497431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2729497431},{"type":"v1","addr":"172.21.15.1:6837","nonce":2729497431}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":81,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-02T15:24:47.761435+0000","modified":"2024-02-02T15:46:36.705531+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24529,"mds_2":24518,"mds_3":24514,"mds_4":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14517":{"gid":14517,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/1218303683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":1218303683},{"type":"v1","addr":"172.21.15.1:6839","nonce":1218303683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14529":{"gid":14529,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/2059197496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2059197496},{"type":"v1","addr":"172.21.15.1:6841","nonce":2059197496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/7648986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":7648986},{"type":"v1","addr":"172.21.15.142:6833","nonce":7648986}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/3482746094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3482746094},{"type":"v1","addr":"172.21.15.71:6833","nonce":3482746094}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/388207814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":388207814},{"type":"v1","addr":"172.21.15.71:6835","nonce":388207814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6835/1052606547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1052606547},{"type":"v1","addr":"172.21.15.142:6835","nonce":1052606547}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.142:6837/2870289019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2870289019},{"type":"v1","addr":"172.21.15.142:6837","nonce":2870289019}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.71:6837/628016199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":628016199},{"type":"v1","addr":"172.21.15.71:6837","nonce":628016199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.142:6839/321675421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":321675421},{"type":"v1","addr":"172.21.15.142:6839","nonce":321675421}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/1268086493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1268086493},{"type":"v1","addr":"172.21.15.71:6839","nonce":1268086493}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-02T15:46:42.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.766+0000 7f3fc1200700 1 -- 172.21.15.1:0/4271448383 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f3f9c0a0710 msgr2=0x7f3f9c0a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:42.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.766+0000 7f3fc1200700 1 --2- 172.21.15.1:0/4271448383 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f3f9c0a0710 0x7f3f9c0a2bd0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f3fac009c60 tx=0x7f3fac009260 comp rx=0 tx=0).stop 2024-02-02T15:46:42.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.766+0000 7f3fc1200700 1 -- 172.21.15.1:0/4271448383 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3fbc106090 msgr2=0x7f3fbc199540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:42.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.766+0000 7f3fc1200700 1 --2- 172.21.15.1:0/4271448383 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3fbc106090 0x7f3fbc199540 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f3fa4000c00 tx=0x7f3fa4005740 comp rx=0 tx=0).stop 2024-02-02T15:46:42.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.766+0000 7f3fc1200700 1 -- 172.21.15.1:0/4271448383 shutdown_connections 2024-02-02T15:46:42.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.766+0000 7f3fc1200700 1 --2- 172.21.15.1:0/4271448383 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f3f9c0a0710 0x7f3f9c0a2bd0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:42.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.766+0000 7f3fc1200700 1 --2- 172.21.15.1:0/4271448383 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3fbc19d9a0 0x7f3fbc19de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:42.766 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.766+0000 7f3fc1200700 1 --2- 172.21.15.1:0/4271448383 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f3fbc100f70 0x7f3fbc199000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:42.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.766+0000 7f3fc1200700 1 --2- 172.21.15.1:0/4271448383 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f3fbc106090 0x7f3fbc199540 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:42.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.766+0000 7f3fc1200700 1 -- 172.21.15.1:0/4271448383 >> 172.21.15.1:0/4271448383 conn(0x7f3fbc0fc810 msgr2=0x7f3fbc0fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:42.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.766+0000 7f3fc1200700 1 -- 172.21.15.1:0/4271448383 shutdown_connections 2024-02-02T15:46:42.767 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.766+0000 7f3fc1200700 1 -- 172.21.15.1:0/4271448383 wait complete. 2024-02-02T15:46:42.767 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 81 2024-02-02T15:46:42.785 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-02-02T15:46:42.985 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.986+0000 7f5e42e00700 1 -- 172.21.15.1:0/3310606869 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5e3c101950 msgr2=0x7f5e3c105990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:42.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.986+0000 7f5e42e00700 1 --2- 172.21.15.1:0/3310606869 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5e3c101950 0x7f5e3c105990 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f5e24009990 tx=0x7f5e24009ca0 comp rx=0 tx=0).stop 2024-02-02T15:46:42.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.986+0000 7f5e42e00700 1 -- 172.21.15.1:0/3310606869 shutdown_connections 2024-02-02T15:46:42.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.986+0000 7f5e42e00700 1 --2- 172.21.15.1:0/3310606869 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e3c1060c0 0x7f5e3c10dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:42.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.986+0000 7f5e42e00700 1 --2- 172.21.15.1:0/3310606869 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e3c100fa0 0x7f5e3c101380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:42.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.986+0000 7f5e42e00700 1 --2- 172.21.15.1:0/3310606869 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5e3c101950 0x7f5e3c105990 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:42.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.986+0000 7f5e42e00700 1 -- 172.21.15.1:0/3310606869 >> 172.21.15.1:0/3310606869 conn(0x7f5e3c0fc800 msgr2=0x7f5e3c0fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:42.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.986+0000 7f5e42e00700 1 -- 172.21.15.1:0/3310606869 shutdown_connections 2024-02-02T15:46:42.986 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.986+0000 7f5e42e00700 1 -- 172.21.15.1:0/3310606869 wait complete. 2024-02-02T15:46:42.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.987+0000 7f5e42e00700 1 Processor -- start 2024-02-02T15:46:42.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.987+0000 7f5e42e00700 1 -- start start 2024-02-02T15:46:42.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.988+0000 7f5e42e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e3c100fa0 0x7f5e3c1990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:42.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.988+0000 7f5e42e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e3c1060c0 0x7f5e3c199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:42.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.988+0000 7f5e42e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5e3c19da60 0x7f5e3c19dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:42.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.988+0000 7f5e42e00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3c19e4e0 con 0x7f5e3c19da60 2024-02-02T15:46:42.989 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.988+0000 7f5e42e00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3c19e650 con 0x7f5e3c1060c0 2024-02-02T15:46:42.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.988+0000 7f5e42e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3c19e810 con 0x7f5e3c100fa0 2024-02-02T15:46:42.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.988+0000 7f5e3be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e3c100fa0 0x7f5e3c1990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:42.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.988+0000 7f5e3be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e3c100fa0 0x7f5e3c1990c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:51690/0 (socket says 172.21.15.1:51690) 2024-02-02T15:46:42.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.988+0000 7f5e3be00700 1 -- 172.21.15.1:0/3388203770 learned_addr learned my addr 172.21.15.1:0/3388203770 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:46:42.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.988+0000 7f5e3b400700 1 --2- 172.21.15.1:0/3388203770 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e3c1060c0 0x7f5e3c199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:42.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.988+0000 7f5e40e00700 1 --2- 172.21.15.1:0/3388203770 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5e3c19da60 0x7f5e3c19dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:42.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.988+0000 7f5e3be00700 1 -- 172.21.15.1:0/3388203770 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e3c1060c0 msgr2=0x7f5e3c199600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:42.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.988+0000 7f5e3be00700 1 --2- 172.21.15.1:0/3388203770 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e3c1060c0 0x7f5e3c199600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:42.990 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.988+0000 7f5e3be00700 1 -- 172.21.15.1:0/3388203770 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5e3c19da60 msgr2=0x7f5e3c19dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:42.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.988+0000 7f5e3be00700 1 --2- 172.21.15.1:0/3388203770 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5e3c19da60 0x7f5e3c19dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:42.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.988+0000 7f5e3be00700 1 -- 172.21.15.1:0/3388203770 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e240095f0 con 0x7f5e3c100fa0 2024-02-02T15:46:42.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.989+0000 7f5e3be00700 1 --2- 172.21.15.1:0/3388203770 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e3c100fa0 0x7f5e3c1990c0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c00d950 tx=0x7f5e2c00dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:42.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.989+0000 7f5e38c00700 1 -- 172.21.15.1:0/3388203770 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c00f890 con 0x7f5e3c100fa0 2024-02-02T15:46:42.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.989+0000 7f5e38c00700 1 -- 172.21.15.1:0/3388203770 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e2c010460 con 0x7f5e3c100fa0 2024-02-02T15:46:42.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.989+0000 7f5e42e00700 1 -- 172.21.15.1:0/3388203770 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e3c1a5d70 con 0x7f5e3c100fa0 2024-02-02T15:46:42.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.989+0000 7f5e38c00700 1 -- 172.21.15.1:0/3388203770 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c00e610 con 0x7f5e3c100fa0 2024-02-02T15:46:42.991 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.989+0000 7f5e42e00700 1 -- 172.21.15.1:0/3388203770 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e3c1a61f0 con 0x7f5e3c100fa0 2024-02-02T15:46:42.992 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.992+0000 7f5e38c00700 1 -- 172.21.15.1:0/3388203770 <== mon.1 v2:172.21.15.142:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f5e2c0108d0 con 0x7f5e3c100fa0 2024-02-02T15:46:42.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.993+0000 7f5e38c00700 1 --2- 172.21.15.1:0/3388203770 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f5e1c0a0690 0x7f5e1c0a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:42.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.994+0000 7f5e38c00700 1 -- 172.21.15.1:0/3388203770 <== mon.1 v2:172.21.15.142:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f5e2c0cbf00 con 0x7f5e3c100fa0 2024-02-02T15:46:42.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.994+0000 7f5e568e1b80 1 Processor -- start 2024-02-02T15:46:42.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.994+0000 7f5e3b400700 1 --2- 172.21.15.1:0/3388203770 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f5e1c0a0690 0x7f5e1c0a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:42.993 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.994+0000 7f5e568e1b80 1 -- start start 2024-02-02T15:46:42.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.994+0000 7f5e568e1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4b8ea6010 0x55a4b8ea63f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:42.995 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.994+0000 7f5e568e1b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a4b8ea6930 0x55a4b8eb12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:42.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.994+0000 7f5e568e1b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a4b8eb17e0 0x55a4b8eb3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:42.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.995+0000 7f5e568e1b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4b8e1a4c0 con 0x55a4b8eb17e0 2024-02-02T15:46:42.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.995+0000 7f5e568e1b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4b8dfeaa0 con 0x55a4b8ea6930 2024-02-02T15:46:42.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.995+0000 7f5e568e1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4b8dfc460 con 0x55a4b8ea6010 2024-02-02T15:46:42.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.995+0000 7f5e3b400700 1 --2- 172.21.15.1:0/3388203770 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f5e1c0a0690 0x7f5e1c0a2b50 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c19a6d0 tx=0x7f5e24005fd0 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:42.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.995+0000 7f5e40e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a4b8eb17e0 0x55a4b8eb3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:42.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.995+0000 7f5e3be00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4b8ea6010 0x55a4b8ea63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:42.996 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.995+0000 7f5e40e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a4b8eb17e0 0x55a4b8eb3bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41428/0 (socket says 172.21.15.1:41428) 2024-02-02T15:46:42.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.995+0000 7f5e40e00700 1 -- 172.21.15.1:0/2037141806 learned_addr learned my addr 172.21.15.1:0/2037141806 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:46:42.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.995+0000 7f5e3b400700 1 --2- 172.21.15.1:0/2037141806 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a4b8ea6930 0x55a4b8eb12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:42.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.995+0000 7f5e40e00700 1 -- 172.21.15.1:0/2037141806 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4b8ea6010 msgr2=0x55a4b8ea63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:42.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.995+0000 7f5e40e00700 1 --2- 172.21.15.1:0/2037141806 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4b8ea6010 0x55a4b8ea63f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:42.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.995+0000 7f5e40e00700 1 -- 172.21.15.1:0/2037141806 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a4b8ea6930 msgr2=0x55a4b8eb12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:42.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.995+0000 7f5e40e00700 1 --2- 172.21.15.1:0/2037141806 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a4b8ea6930 0x55a4b8eb12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:42.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.995+0000 7f5e40e00700 1 -- 172.21.15.1:0/2037141806 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a4b8c793b0 con 0x55a4b8eb17e0 2024-02-02T15:46:42.997 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.995+0000 7f5e3be00700 1 --2- 172.21.15.1:0/2037141806 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4b8ea6010 0x55a4b8ea63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:46:42.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.996+0000 7f5e40e00700 1 --2- 172.21.15.1:0/2037141806 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a4b8eb17e0 0x55a4b8eb3bd0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f5e34003f30 tx=0x7f5e34009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8452bad1158546b0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:42.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.996+0000 7f5e31600700 1 -- 172.21.15.1:0/2037141806 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e34003a00 con 0x55a4b8eb17e0 2024-02-02T15:46:42.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.996+0000 7f5e31600700 1 -- 172.21.15.1:0/2037141806 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e3400f040 con 0x55a4b8eb17e0 2024-02-02T15:46:42.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.996+0000 7f5e31600700 1 -- 172.21.15.1:0/2037141806 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e340156e0 con 0x55a4b8eb17e0 2024-02-02T15:46:42.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.996+0000 7f5e568e1b80 1 -- 172.21.15.1:0/2037141806 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a4b8eb17e0 msgr2=0x55a4b8eb3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:42.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.996+0000 7f5e568e1b80 1 --2- 172.21.15.1:0/2037141806 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a4b8eb17e0 0x55a4b8eb3bd0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f5e34003f30 tx=0x7f5e34009c20 comp rx=0 tx=0).stop 2024-02-02T15:46:42.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.997+0000 7f5e568e1b80 1 -- 172.21.15.1:0/2037141806 shutdown_connections 2024-02-02T15:46:42.998 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.997+0000 7f5e568e1b80 1 --2- 172.21.15.1:0/2037141806 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a4b8eb17e0 0x55a4b8eb3bd0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:42.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.997+0000 7f5e568e1b80 1 --2- 172.21.15.1:0/2037141806 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4b8ea6010 0x55a4b8ea63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:42.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.997+0000 7f5e568e1b80 1 --2- 172.21.15.1:0/2037141806 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a4b8ea6930 0x55a4b8eb12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:42.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.997+0000 7f5e568e1b80 1 -- 172.21.15.1:0/2037141806 >> 172.21.15.1:0/2037141806 conn(0x55a4b8e05db0 msgr2=0x55a4b8ea23b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:42.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.997+0000 7f5e568e1b80 1 -- 172.21.15.1:0/2037141806 shutdown_connections 2024-02-02T15:46:42.999 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.997+0000 7f5e568e1b80 1 -- 172.21.15.1:0/2037141806 wait complete. 2024-02-02T15:46:43.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.997+0000 7f5e568e1b80 1 -- start start 2024-02-02T15:46:43.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.997+0000 7f5e568e1b80 1 Processor -- start 2024-02-02T15:46:43.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.998+0000 7f5e568e1b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a4b8ea6010 0x55a4b8eb1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:43.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.998+0000 7f5e568e1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4b8ea6930 0x55a4b8eae3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:43.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.998+0000 7f5e568e1b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a4b8eb17e0 0x55a4b8eae960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:43.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.998+0000 7f5e568e1b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4b8df7d50 con 0x55a4b8ea6010 2024-02-02T15:46:43.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.998+0000 7f5e568e1b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4b8d93d80 con 0x55a4b8eb17e0 2024-02-02T15:46:43.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.998+0000 7f5e568e1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4b8d93a20 con 0x55a4b8ea6930 2024-02-02T15:46:43.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.998+0000 7f5e568e1b80 10 client.0 _ll_register_callbacks cb 0x55a4b8d95290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:46:43.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.998+0000 7f5e3be00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a4b8ea6010 0x55a4b8eb1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:43.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.998+0000 7f5e3be00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a4b8ea6010 0x55a4b8eb1120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41432/0 (socket says 172.21.15.1:41432) 2024-02-02T15:46:43.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.998+0000 7f5e3be00700 1 -- 172.21.15.1:0/2985785282 learned_addr learned my addr 172.21.15.1:0/2985785282 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:46:43.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.998+0000 7f5e40e00700 1 --2- 172.21.15.1:0/2985785282 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a4b8eb17e0 0x55a4b8eae960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:43.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.998+0000 7f5e3b400700 1 --2- 172.21.15.1:0/2985785282 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4b8ea6930 0x55a4b8eae3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:43.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.998+0000 7f5e3be00700 1 -- 172.21.15.1:0/2985785282 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4b8ea6930 msgr2=0x55a4b8eae3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:43.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.998+0000 7f5e3be00700 1 --2- 172.21.15.1:0/2985785282 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4b8ea6930 0x55a4b8eae3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.998+0000 7f5e3be00700 1 -- 172.21.15.1:0/2985785282 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a4b8eb17e0 msgr2=0x55a4b8eae960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:43.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.998+0000 7f5e3be00700 1 --2- 172.21.15.1:0/2985785282 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a4b8eb17e0 0x55a4b8eae960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.999+0000 7f5e3be00700 1 -- 172.21.15.1:0/2985785282 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e3c1a61f0 con 0x55a4b8ea6010 2024-02-02T15:46:43.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.999+0000 7f5e3be00700 1 --2- 172.21.15.1:0/2985785282 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a4b8ea6010 0x55a4b8eb1120 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c00b2f0 tx=0x7f5e2c0990a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=241048a741e31d90 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:43.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.999+0000 7f5e2a000700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:46:43.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.999+0000 7f5e2a000700 1 -- 172.21.15.1:0/2985785282 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c099480 con 0x55a4b8ea6010 2024-02-02T15:46:43.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.999+0000 7f5e2a000700 1 -- 172.21.15.1:0/2985785282 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f5e2c098510 con 0x55a4b8ea6010 2024-02-02T15:46:43.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.999+0000 7f5e568e1b80 1 -- 172.21.15.1:0/2985785282 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a4b8e9a0f0 con 0x55a4b8ea6010 2024-02-02T15:46:43.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.999+0000 7f5e2a000700 1 -- 172.21.15.1:0/2985785282 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5e2c0986d0 con 0x55a4b8ea6010 2024-02-02T15:46:43.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.999+0000 7f5e568e1b80 10 client.20298 fetch_fsmap learned FSMap version 81 2024-02-02T15:46:43.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.999+0000 7f5e568e1b80 1 -- 172.21.15.1:0/2985785282 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) v3 -- 0x7f5e34003b60 con 0x55a4b8ea6010 2024-02-02T15:46:43.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:42.999+0000 7f5e2a000700 1 -- 172.21.15.1:0/2985785282 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a4b8e9a0f0 con 0x55a4b8ea6010 2024-02-02T15:46:43.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.000+0000 7f5e2a000700 1 -- 172.21.15.1:0/2985785282 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 81) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f5e2c094e10 con 0x55a4b8ea6010 2024-02-02T15:46:43.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.000+0000 7f5e568e1b80 10 client.20298 fetch_fsmap finished waiting for FSMap version 81 2024-02-02T15:46:43.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.000+0000 7f5e568e1b80 10 client.20298 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-02-02T15:46:43.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.000+0000 7f5e568e1b80 20 client.20298 populate_metadata read hostname 'smithi001' 2024-02-02T15:46:43.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.000+0000 7f5e568e1b80 1 --2- 172.21.15.1:0/2985785282 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55a4b8ec10f0 0x55a4b8ec3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:43.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.000+0000 7f5e568e1b80 4 client.20298 mds_command: new command op to 24502 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:46:43.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.000+0000 7f5e568e1b80 1 -- 172.21.15.1:0/2985785282 --> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a4b8df0dd0 con 0x55a4b8ec10f0 2024-02-02T15:46:43.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.000+0000 7f5e3b400700 1 --2- 172.21.15.1:0/2985785282 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55a4b8ec10f0 0x55a4b8ec3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:43.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.000+0000 7f5e3b400700 1 --2- 172.21.15.1:0/2985785282 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55a4b8ec10f0 0x55a4b8ec3510 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b8820974607023a0 server_cookie=fd276b0d5930b036 in_seq=0 out_seq=0 2024-02-02T15:46:43.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.000+0000 7f5e2a000700 10 client.20298 ms_handle_connect on v2:172.21.15.142:6834/1052606547 2024-02-02T15:46:43.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.001+0000 7f5e2a000700 1 -- 172.21.15.1:0/2985785282 <== mds.4 v2:172.21.15.142:6834/1052606547 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55a4b8df0dd0 con 0x55a4b8ec10f0 2024-02-02T15:46:43.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.001+0000 7f5e2a000700 10 client.20298 handle_command_reply: tid=0 2024-02-02T15:46:43.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.001+0000 7f5e568e1b80 1 client.20298 shutdown 2024-02-02T15:46:43.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.001+0000 7f5e568e1b80 1 -- 172.21.15.1:0/2985785282 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a4b8ea6010 msgr2=0x55a4b8eb1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:43.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.001+0000 7f5e568e1b80 1 --2- 172.21.15.1:0/2985785282 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a4b8ea6010 0x55a4b8eb1120 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c00b2f0 tx=0x7f5e2c0990a0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.001+0000 7f5e568e1b80 1 -- 172.21.15.1:0/2985785282 shutdown_connections 2024-02-02T15:46:43.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.002+0000 7f5e568e1b80 1 --2- 172.21.15.1:0/2985785282 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55a4b8ec10f0 0x55a4b8ec3510 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.002+0000 7f5e568e1b80 1 --2- 172.21.15.1:0/2985785282 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a4b8eb17e0 0x55a4b8eae960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.002+0000 7f5e568e1b80 1 --2- 172.21.15.1:0/2985785282 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a4b8ea6010 0x55a4b8eb1120 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.002+0000 7f5e568e1b80 1 --2- 172.21.15.1:0/2985785282 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4b8ea6930 0x55a4b8eae3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.002+0000 7f5e2a000700 0 client.20298 ms_handle_reset on v2:172.21.15.142:6834/1052606547 2024-02-02T15:46:43.009 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.002+0000 7f5e568e1b80 1 -- 172.21.15.1:0/2985785282 >> 172.21.15.1:0/2985785282 conn(0x55a4b8e05db0 msgr2=0x55a4b8ea1f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:43.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.002+0000 7f5e568e1b80 1 -- 172.21.15.1:0/2985785282 shutdown_connections 2024-02-02T15:46:43.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.002+0000 7f5e568e1b80 1 -- 172.21.15.1:0/2985785282 wait complete. 2024-02-02T15:46:43.010 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.002+0000 7f5e568e1b80 20 client.20298 trim_cache size 0 max 16384 2024-02-02T15:46:43.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.015+0000 7f5e568e1b80 1 Processor -- start 2024-02-02T15:46:43.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.015+0000 7f5e568e1b80 1 -- start start 2024-02-02T15:46:43.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.015+0000 7f5e568e1b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a4b8ea6930 0x55a4b8eb0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:43.014 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.015+0000 7f5e568e1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4b8eb17e0 0x55a4b8eae360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:43.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.015+0000 7f5e568e1b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a4b8eb1200 0x55a4b8eae8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:43.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.015+0000 7f5e568e1b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4b8c7b6a0 con 0x55a4b8ea6930 2024-02-02T15:46:43.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.015+0000 7f5e568e1b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4b8ec0f20 con 0x55a4b8eb1200 2024-02-02T15:46:43.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.015+0000 7f5e568e1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4b8ea6f70 con 0x55a4b8eb17e0 2024-02-02T15:46:43.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.016+0000 7f5e3be00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a4b8ea6930 0x55a4b8eb0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:43.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.016+0000 7f5e3be00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a4b8ea6930 0x55a4b8eb0cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41442/0 (socket says 172.21.15.1:41442) 2024-02-02T15:46:43.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.016+0000 7f5e3be00700 1 -- 172.21.15.1:0/3745173216 learned_addr learned my addr 172.21.15.1:0/3745173216 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:46:43.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.016+0000 7f5e40e00700 1 --2- 172.21.15.1:0/3745173216 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a4b8eb1200 0x55a4b8eae8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:43.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.016+0000 7f5e3b400700 1 --2- 172.21.15.1:0/3745173216 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4b8eb17e0 0x55a4b8eae360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:43.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.016+0000 7f5e3be00700 1 -- 172.21.15.1:0/3745173216 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4b8eb17e0 msgr2=0x55a4b8eae360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:43.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.016+0000 7f5e3be00700 1 --2- 172.21.15.1:0/3745173216 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4b8eb17e0 0x55a4b8eae360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.016+0000 7f5e3be00700 1 -- 172.21.15.1:0/3745173216 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a4b8eb1200 msgr2=0x55a4b8eae8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:43.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.016+0000 7f5e3be00700 1 --2- 172.21.15.1:0/3745173216 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a4b8eb1200 0x55a4b8eae8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.016+0000 7f5e3be00700 1 -- 172.21.15.1:0/3745173216 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a4b8c25b60 con 0x55a4b8ea6930 2024-02-02T15:46:43.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.016+0000 7f5e3be00700 1 --2- 172.21.15.1:0/3745173216 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a4b8ea6930 0x55a4b8eb0cc0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c095140 tx=0x7f5e2c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=666d51890c545de9 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:43.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.017+0000 7f5e32000700 1 -- 172.21.15.1:0/3745173216 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c014070 con 0x55a4b8ea6930 2024-02-02T15:46:43.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.017+0000 7f5e32000700 1 -- 172.21.15.1:0/3745173216 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e2c09ce50 con 0x55a4b8ea6930 2024-02-02T15:46:43.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.017+0000 7f5e32000700 1 -- 172.21.15.1:0/3745173216 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c0b7020 con 0x55a4b8ea6930 2024-02-02T15:46:43.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.017+0000 7f5e568e1b80 1 -- 172.21.15.1:0/3745173216 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a4b8ea6930 msgr2=0x55a4b8eb0cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:43.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.017+0000 7f5e568e1b80 1 --2- 172.21.15.1:0/3745173216 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a4b8ea6930 0x55a4b8eb0cc0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c095140 tx=0x7f5e2c000bc0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.017+0000 7f5e568e1b80 1 -- 172.21.15.1:0/3745173216 shutdown_connections 2024-02-02T15:46:43.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.017+0000 7f5e568e1b80 1 --2- 172.21.15.1:0/3745173216 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a4b8eb1200 0x55a4b8eae8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.017+0000 7f5e568e1b80 1 --2- 172.21.15.1:0/3745173216 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a4b8ea6930 0x55a4b8eb0cc0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.017+0000 7f5e568e1b80 1 --2- 172.21.15.1:0/3745173216 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4b8eb17e0 0x55a4b8eae360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.020 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.017+0000 7f5e568e1b80 1 -- 172.21.15.1:0/3745173216 >> 172.21.15.1:0/3745173216 conn(0x55a4b8e05db0 msgr2=0x55a4b8ebd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:43.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.017+0000 7f5e568e1b80 1 -- 172.21.15.1:0/3745173216 shutdown_connections 2024-02-02T15:46:43.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.017+0000 7f5e568e1b80 1 -- 172.21.15.1:0/3745173216 wait complete. 2024-02-02T15:46:43.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.018+0000 7f5e568e1b80 1 -- start start 2024-02-02T15:46:43.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.018+0000 7f5e568e1b80 1 Processor -- start 2024-02-02T15:46:43.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.018+0000 7f5e568e1b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a4b8ea6930 0x55a4b8eb8060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:43.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.018+0000 7f5e568e1b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a4b8eb1200 0x55a4b8eb6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:43.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.018+0000 7f5e568e1b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4b8eb17e0 0x55a4b8eb68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:43.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.018+0000 7f5e568e1b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4b8eb6e10 con 0x55a4b8eb1200 2024-02-02T15:46:43.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.018+0000 7f5e568e1b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4b8eb6f50 con 0x55a4b8ea6930 2024-02-02T15:46:43.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.018+0000 7f5e568e1b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4b8eb7110 con 0x55a4b8eb17e0 2024-02-02T15:46:43.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.019+0000 7f5e568e1b80 10 client.0 _ll_register_callbacks cb 0x55a4b8d95290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:46:43.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.019+0000 7f5e3b400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a4b8eb1200 0x55a4b8eb6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:43.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.019+0000 7f5e3b400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a4b8eb1200 0x55a4b8eb6390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41446/0 (socket says 172.21.15.1:41446) 2024-02-02T15:46:43.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.019+0000 7f5e3b400700 1 -- 172.21.15.1:0/184687838 learned_addr learned my addr 172.21.15.1:0/184687838 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:46:43.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.019+0000 7f5e40e00700 1 --2- 172.21.15.1:0/184687838 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4b8eb17e0 0x55a4b8eb68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:43.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.019+0000 7f5e3be00700 1 --2- 172.21.15.1:0/184687838 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a4b8ea6930 0x55a4b8eb8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:43.022 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.019+0000 7f5e3b400700 1 -- 172.21.15.1:0/184687838 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4b8eb17e0 msgr2=0x55a4b8eb68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:43.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.019+0000 7f5e3b400700 1 --2- 172.21.15.1:0/184687838 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4b8eb17e0 0x55a4b8eb68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.019+0000 7f5e3b400700 1 -- 172.21.15.1:0/184687838 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a4b8ea6930 msgr2=0x55a4b8eb8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:43.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.019+0000 7f5e3b400700 1 --2- 172.21.15.1:0/184687838 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a4b8ea6930 0x55a4b8eb8060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.019+0000 7f5e3b400700 1 -- 172.21.15.1:0/184687838 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e34003b60 con 0x55a4b8eb1200 2024-02-02T15:46:43.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.019+0000 7f5e3b400700 1 --2- 172.21.15.1:0/184687838 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a4b8eb1200 0x55a4b8eb6390 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f5e240036d0 tx=0x7f5e24009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a72857a5f68f3c0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:43.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.019+0000 7f5e2a000700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:46:43.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.020+0000 7f5e2a000700 1 -- 172.21.15.1:0/184687838 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e24034dd0 con 0x55a4b8eb1200 2024-02-02T15:46:43.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.020+0000 7f5e2a000700 1 -- 172.21.15.1:0/184687838 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f5e2400b770 con 0x55a4b8eb1200 2024-02-02T15:46:43.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.020+0000 7f5e568e1b80 1 -- 172.21.15.1:0/184687838 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5e3400f040 con 0x55a4b8eb1200 2024-02-02T15:46:43.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.020+0000 7f5e2a000700 1 -- 172.21.15.1:0/184687838 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5e2400b930 con 0x55a4b8eb1200 2024-02-02T15:46:43.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.020+0000 7f5e568e1b80 10 client.20304 fetch_fsmap learned FSMap version 81 2024-02-02T15:46:43.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.020+0000 7f5e568e1b80 1 -- 172.21.15.1:0/184687838 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) v3 -- 0x55a4b8dbdea0 con 0x55a4b8eb1200 2024-02-02T15:46:43.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.020+0000 7f5e2a000700 1 -- 172.21.15.1:0/184687838 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e3400f040 con 0x55a4b8eb1200 2024-02-02T15:46:43.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.020+0000 7f5e2a000700 1 -- 172.21.15.1:0/184687838 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 81) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f5e24003de0 con 0x55a4b8eb1200 2024-02-02T15:46:43.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.020+0000 7f5e568e1b80 10 client.20304 fetch_fsmap finished waiting for FSMap version 81 2024-02-02T15:46:43.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.020+0000 7f5e568e1b80 10 client.20304 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.f 2024-02-02T15:46:43.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.020+0000 7f5e568e1b80 20 client.20304 populate_metadata read hostname 'smithi001' 2024-02-02T15:46:43.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.020+0000 7f5e568e1b80 1 --2- 172.21.15.1:0/184687838 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55a4b8f14180 0x55a4b8f165a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:43.027 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.020+0000 7f5e568e1b80 4 client.20304 mds_command: new command op to 24502 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:46:43.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.020+0000 7f5e568e1b80 1 -- 172.21.15.1:0/184687838 --> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a4b8db6f50 con 0x55a4b8f14180 2024-02-02T15:46:43.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.021+0000 7f5e3be00700 1 --2- 172.21.15.1:0/184687838 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55a4b8f14180 0x55a4b8f165a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:43.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.021+0000 7f5e3be00700 1 --2- 172.21.15.1:0/184687838 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55a4b8f14180 0x55a4b8f165a0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1e8ae603e7daea23 server_cookie=7c385b9c3f2d5867 in_seq=0 out_seq=0 2024-02-02T15:46:43.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.021+0000 7f5e2a000700 10 client.20304 ms_handle_connect on v2:172.21.15.142:6834/1052606547 2024-02-02T15:46:43.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.021+0000 7f5e2a000700 1 -- 172.21.15.1:0/184687838 <== mds.4 v2:172.21.15.142:6834/1052606547 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a4b8db6f50 con 0x55a4b8f14180 2024-02-02T15:46:43.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.021+0000 7f5e2a000700 10 client.20304 handle_command_reply: tid=0 2024-02-02T15:46:43.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.021+0000 7f5e568e1b80 1 client.20304 shutdown 2024-02-02T15:46:43.028 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.022+0000 7f5e568e1b80 1 -- 172.21.15.1:0/184687838 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a4b8eb1200 msgr2=0x55a4b8eb6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:43.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.022+0000 7f5e568e1b80 1 --2- 172.21.15.1:0/184687838 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a4b8eb1200 0x55a4b8eb6390 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f5e240036d0 tx=0x7f5e24009ca0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.022+0000 7f5e568e1b80 1 -- 172.21.15.1:0/184687838 shutdown_connections 2024-02-02T15:46:43.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.022+0000 7f5e568e1b80 1 --2- 172.21.15.1:0/184687838 >> [v2:172.21.15.142:6834/1052606547,v1:172.21.15.142:6835/1052606547] conn(0x55a4b8f14180 0x55a4b8f165a0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.022+0000 7f5e568e1b80 1 --2- 172.21.15.1:0/184687838 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a4b8eb17e0 0x55a4b8eb68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.022+0000 7f5e568e1b80 1 --2- 172.21.15.1:0/184687838 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a4b8ea6930 0x55a4b8eb8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.022+0000 7f5e2a000700 0 client.20304 ms_handle_reset on v2:172.21.15.142:6834/1052606547 2024-02-02T15:46:43.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.022+0000 7f5e568e1b80 1 --2- 172.21.15.1:0/184687838 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a4b8eb1200 0x55a4b8eb6390 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.022+0000 7f5e568e1b80 1 -- 172.21.15.1:0/184687838 >> 172.21.15.1:0/184687838 conn(0x55a4b8e05db0 msgr2=0x55a4b8eae340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:43.029 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.022+0000 7f5e568e1b80 1 -- 172.21.15.1:0/184687838 shutdown_connections 2024-02-02T15:46:43.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.022+0000 7f5e568e1b80 1 -- 172.21.15.1:0/184687838 wait complete. 2024-02-02T15:46:43.030 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.022+0000 7f5e568e1b80 20 client.20304 trim_cache size 0 max 16384 2024-02-02T15:46:43.030 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:46:43.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.022+0000 7f5e42e00700 1 -- 172.21.15.1:0/3388203770 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f5e1c0a0690 msgr2=0x7f5e1c0a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:43.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.022+0000 7f5e42e00700 1 --2- 172.21.15.1:0/3388203770 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f5e1c0a0690 0x7f5e1c0a2b50 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c19a6d0 tx=0x7f5e24005fd0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.022+0000 7f5e42e00700 1 -- 172.21.15.1:0/3388203770 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e3c100fa0 msgr2=0x7f5e3c1990c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:43.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.022+0000 7f5e42e00700 1 --2- 172.21.15.1:0/3388203770 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e3c100fa0 0x7f5e3c1990c0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c00d950 tx=0x7f5e2c00dc60 comp rx=0 tx=0).stop 2024-02-02T15:46:43.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.023+0000 7f5e42e00700 1 -- 172.21.15.1:0/3388203770 shutdown_connections 2024-02-02T15:46:43.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.023+0000 7f5e42e00700 1 --2- 172.21.15.1:0/3388203770 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f5e1c0a0690 0x7f5e1c0a2b50 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.023+0000 7f5e42e00700 1 --2- 172.21.15.1:0/3388203770 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f5e3c19da60 0x7f5e3c19dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.023+0000 7f5e42e00700 1 --2- 172.21.15.1:0/3388203770 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f5e3c100fa0 0x7f5e3c1990c0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.023+0000 7f5e42e00700 1 --2- 172.21.15.1:0/3388203770 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5e3c1060c0 0x7f5e3c199600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.023+0000 7f5e42e00700 1 -- 172.21.15.1:0/3388203770 >> 172.21.15.1:0/3388203770 conn(0x7f5e3c0fc800 msgr2=0x7f5e3c0fd2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:43.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.023+0000 7f5e42e00700 1 -- 172.21.15.1:0/3388203770 shutdown_connections 2024-02-02T15:46:43.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.023+0000 7f5e42e00700 1 -- 172.21.15.1:0/3388203770 wait complete. 2024-02-02T15:46:43.037 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:46:42 smithi001 ceph-mon[35470]: pgmap v620: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 2.1 MiB/s wr, 96 op/s 2024-02-02T15:46:43.039 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-02-02T15:46:43.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:46:42 smithi071 ceph-mon[44707]: pgmap v620: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 2.1 MiB/s wr, 96 op/s 2024-02-02T15:46:43.169 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:46:42 smithi142 ceph-mon[45728]: pgmap v620: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 2.1 MiB/s wr, 96 op/s 2024-02-02T15:46:43.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.240+0000 7fa3ffe00700 1 -- 172.21.15.1:0/1289695330 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa3f8100ff0 msgr2=0x7fa3f81013d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:43.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.240+0000 7fa3ffe00700 1 --2- 172.21.15.1:0/1289695330 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa3f8100ff0 0x7fa3f81013d0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fa3e8009990 tx=0x7fa3e8009ca0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.240+0000 7fa3ffe00700 1 -- 172.21.15.1:0/1289695330 shutdown_connections 2024-02-02T15:46:43.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.241+0000 7fa3ffe00700 1 --2- 172.21.15.1:0/1289695330 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3f8106110 0x7fa3f810dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.240 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.241+0000 7fa3ffe00700 1 --2- 172.21.15.1:0/1289695330 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa3f8100ff0 0x7fa3f81013d0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.241+0000 7fa3ffe00700 1 --2- 172.21.15.1:0/1289695330 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3f81019a0 0x7fa3f81059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.241+0000 7fa3ffe00700 1 -- 172.21.15.1:0/1289695330 >> 172.21.15.1:0/1289695330 conn(0x7fa3f80fc810 msgr2=0x7fa3f80fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:43.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.241+0000 7fa3ffe00700 1 -- 172.21.15.1:0/1289695330 shutdown_connections 2024-02-02T15:46:43.241 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.241+0000 7fa3ffe00700 1 -- 172.21.15.1:0/1289695330 wait complete. 2024-02-02T15:46:43.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.242+0000 7fa3ffe00700 1 Processor -- start 2024-02-02T15:46:43.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.242+0000 7fa3ffe00700 1 -- start start 2024-02-02T15:46:43.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.242+0000 7fa3ffe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa3f8100ff0 0x7fa3f8073040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:43.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.242+0000 7fa3ffe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3f8106110 0x7fa3f8073580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:43.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.242+0000 7fa3ffe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3f8073ac0 0x7fa3f81a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:43.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.242+0000 7fa3ffe00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f81a5730 con 0x7fa3f8100ff0 2024-02-02T15:46:43.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.242+0000 7fa3ffe00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f81a58a0 con 0x7fa3f8073ac0 2024-02-02T15:46:43.244 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.242+0000 7fa3ffe00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f81a5a60 con 0x7fa3f8106110 2024-02-02T15:46:43.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.242+0000 7fa3fca00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3f8106110 0x7fa3f8073580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:43.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.242+0000 7fa3fd400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa3f8100ff0 0x7fa3f8073040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:43.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.242+0000 7fa3fca00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3f8106110 0x7fa3f8073580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:51740/0 (socket says 172.21.15.1:51740) 2024-02-02T15:46:43.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.242+0000 7fa3fca00700 1 -- 172.21.15.1:0/3836308651 learned_addr learned my addr 172.21.15.1:0/3836308651 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:46:43.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.242+0000 7fa3fde00700 1 --2- 172.21.15.1:0/3836308651 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3f8073ac0 0x7fa3f81a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:43.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.243+0000 7fa3fd400700 1 -- 172.21.15.1:0/3836308651 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3f8106110 msgr2=0x7fa3f8073580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:43.245 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.243+0000 7fa3fd400700 1 --2- 172.21.15.1:0/3836308651 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3f8106110 0x7fa3f8073580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.243+0000 7fa3fd400700 1 -- 172.21.15.1:0/3836308651 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3f8073ac0 msgr2=0x7fa3f81a50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:43.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.243+0000 7fa3fd400700 1 --2- 172.21.15.1:0/3836308651 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3f8073ac0 0x7fa3f81a50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.243+0000 7fa3fd400700 1 -- 172.21.15.1:0/3836308651 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3e80095f0 con 0x7fa3f8100ff0 2024-02-02T15:46:43.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.243+0000 7fa3fde00700 1 --2- 172.21.15.1:0/3836308651 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3f8073ac0 0x7fa3f81a50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:46:43.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.243+0000 7fa3fd400700 1 --2- 172.21.15.1:0/3836308651 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa3f8100ff0 0x7fa3f8073040 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fa3e80059d0 tx=0x7fa3e8005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:43.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.243+0000 7fa3f6000700 1 -- 172.21.15.1:0/3836308651 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e801d070 con 0x7fa3f8100ff0 2024-02-02T15:46:43.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.243+0000 7fa3f6000700 1 -- 172.21.15.1:0/3836308651 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa3e8022470 con 0x7fa3f8100ff0 2024-02-02T15:46:43.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.243+0000 7fa3ffe00700 1 -- 172.21.15.1:0/3836308651 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3f81a5ce0 con 0x7fa3f8100ff0 2024-02-02T15:46:43.246 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.243+0000 7fa3f6000700 1 -- 172.21.15.1:0/3836308651 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e8017670 con 0x7fa3f8100ff0 2024-02-02T15:46:43.247 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.244+0000 7fa3ffe00700 1 -- 172.21.15.1:0/3836308651 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3f81a6100 con 0x7fa3f8100ff0 2024-02-02T15:46:43.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.247+0000 7fa3f6000700 1 -- 172.21.15.1:0/3836308651 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7fa3e80225e0 con 0x7fa3f8100ff0 2024-02-02T15:46:43.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.248+0000 7fa4138b5b80 1 Processor -- start 2024-02-02T15:46:43.250 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.248+0000 7fa3f6000700 1 --2- 172.21.15.1:0/3836308651 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fa3d80a0690 0x7fa3d80a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:43.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.248+0000 7fa3f6000700 1 -- 172.21.15.1:0/3836308651 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fa3e80cd290 con 0x7fa3f8100ff0 2024-02-02T15:46:43.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.248+0000 7fa4138b5b80 1 -- start start 2024-02-02T15:46:43.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.248+0000 7fa3fde00700 1 --2- 172.21.15.1:0/3836308651 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fa3d80a0690 0x7fa3d80a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:43.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.248+0000 7fa4138b5b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a810b4a010 0x55a810b4a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:43.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.248+0000 7fa4138b5b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a810b4a930 0x55a810b552a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:43.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.248+0000 7fa4138b5b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a810b557e0 0x55a810b57bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:43.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.248+0000 7fa4138b5b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a810abe4c0 con 0x55a810b557e0 2024-02-02T15:46:43.251 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.248+0000 7fa4138b5b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a810aa2aa0 con 0x55a810b4a930 2024-02-02T15:46:43.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.248+0000 7fa4138b5b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a810aa0460 con 0x55a810b4a010 2024-02-02T15:46:43.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.249+0000 7fa3fd400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a810b4a010 0x55a810b4a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:43.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.249+0000 7fa3fca00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a810b4a930 0x55a810b552a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:43.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.249+0000 7fa3fd400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a810b4a010 0x55a810b4a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:51746/0 (socket says 172.21.15.1:51746) 2024-02-02T15:46:43.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.249+0000 7fa3fca00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a810b4a930 0x55a810b552a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:33112/0 (socket says 172.21.15.1:33112) 2024-02-02T15:46:43.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.249+0000 7fa3fde00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a810b557e0 0x55a810b57bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:43.252 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.249+0000 7fa3fd400700 1 -- 172.21.15.1:0/197012997 learned_addr learned my addr 172.21.15.1:0/197012997 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:46:43.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.249+0000 7fa3fde00700 1 --2- 172.21.15.1:0/3836308651 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fa3d80a0690 0x7fa3d80a2b50 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fa3f0003e80 tx=0x7fa3f0009920 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:43.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.249+0000 7fa3fd400700 1 -- 172.21.15.1:0/197012997 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a810b4a930 msgr2=0x55a810b552a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:43.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.249+0000 7fa3fd400700 1 --2- 172.21.15.1:0/197012997 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a810b4a930 0x55a810b552a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.249+0000 7fa3fd400700 1 -- 172.21.15.1:0/197012997 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a810b557e0 msgr2=0x55a810b57bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:43.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.249+0000 7fa3fd400700 1 --2- 172.21.15.1:0/197012997 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a810b557e0 0x55a810b57bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.249+0000 7fa3fd400700 1 -- 172.21.15.1:0/197012997 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a81091d3b0 con 0x55a810b4a010 2024-02-02T15:46:43.253 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.249+0000 7fa3fca00700 1 --2- 172.21.15.1:0/197012997 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a810b4a930 0x55a810b552a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:46:43.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.249+0000 7fa3fde00700 1 --2- 172.21.15.1:0/197012997 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a810b557e0 0x55a810b57bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:46:43.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.250+0000 7fa3fd400700 1 --2- 172.21.15.1:0/197012997 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a810b4a010 0x55a810b4a3f0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fa3e8009990 tx=0x7fa3e800b5b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70bbe0e13a63dfeb server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:43.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.250+0000 7fa3ee600700 1 -- 172.21.15.1:0/197012997 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e80057b0 con 0x55a810b4a010 2024-02-02T15:46:43.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.250+0000 7fa3ee600700 1 -- 172.21.15.1:0/197012997 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa3e80850f0 con 0x55a810b4a010 2024-02-02T15:46:43.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.250+0000 7fa3ee600700 1 -- 172.21.15.1:0/197012997 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e8022900 con 0x55a810b4a010 2024-02-02T15:46:43.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.250+0000 7fa4138b5b80 1 -- 172.21.15.1:0/197012997 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a810b4a010 msgr2=0x55a810b4a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:43.254 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.250+0000 7fa4138b5b80 1 --2- 172.21.15.1:0/197012997 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a810b4a010 0x55a810b4a3f0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fa3e8009990 tx=0x7fa3e800b5b0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.251+0000 7fa4138b5b80 1 -- 172.21.15.1:0/197012997 shutdown_connections 2024-02-02T15:46:43.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.251+0000 7fa4138b5b80 1 --2- 172.21.15.1:0/197012997 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a810b557e0 0x55a810b57bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.251+0000 7fa4138b5b80 1 --2- 172.21.15.1:0/197012997 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a810b4a010 0x55a810b4a3f0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.251+0000 7fa4138b5b80 1 --2- 172.21.15.1:0/197012997 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a810b4a930 0x55a810b552a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.251+0000 7fa4138b5b80 1 -- 172.21.15.1:0/197012997 >> 172.21.15.1:0/197012997 conn(0x55a810aa9db0 msgr2=0x55a810b463b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:43.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.251+0000 7fa4138b5b80 1 -- 172.21.15.1:0/197012997 shutdown_connections 2024-02-02T15:46:43.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.251+0000 7fa4138b5b80 1 -- 172.21.15.1:0/197012997 wait complete. 2024-02-02T15:46:43.255 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.251+0000 7fa4138b5b80 1 -- start start 2024-02-02T15:46:43.256 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.252+0000 7fa4138b5b80 1 Processor -- start 2024-02-02T15:46:43.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.252+0000 7fa4138b5b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a810b4a010 0x55a810b55bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:43.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.252+0000 7fa4138b5b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a810b4a930 0x55a810b56100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:43.258 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.252+0000 7fa4138b5b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a810b557e0 0x55a810b56640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:43.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.252+0000 7fa4138b5b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a810a9bd50 con 0x55a810b557e0 2024-02-02T15:46:43.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.252+0000 7fa4138b5b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a810a37d80 con 0x55a810b4a010 2024-02-02T15:46:43.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.252+0000 7fa4138b5b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a810a37a20 con 0x55a810b4a930 2024-02-02T15:46:43.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.252+0000 7fa4138b5b80 10 client.0 _ll_register_callbacks cb 0x55a810a39290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:46:43.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.252+0000 7fa3fde00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a810b557e0 0x55a810b56640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:43.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.252+0000 7fa3fca00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a810b4a930 0x55a810b56100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:43.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.252+0000 7fa3fde00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a810b557e0 0x55a810b56640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41492/0 (socket says 172.21.15.1:41492) 2024-02-02T15:46:43.259 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.252+0000 7fa3fde00700 1 -- 172.21.15.1:0/2688488314 learned_addr learned my addr 172.21.15.1:0/2688488314 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:46:43.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.252+0000 7fa3fd400700 1 --2- 172.21.15.1:0/2688488314 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a810b4a010 0x55a810b55bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:43.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.253+0000 7fa3fde00700 1 -- 172.21.15.1:0/2688488314 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a810b4a930 msgr2=0x55a810b56100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:43.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.253+0000 7fa3fde00700 1 --2- 172.21.15.1:0/2688488314 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a810b4a930 0x55a810b56100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.253+0000 7fa3fde00700 1 -- 172.21.15.1:0/2688488314 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a810b4a010 msgr2=0x55a810b55bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:43.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.253+0000 7fa3fde00700 1 --2- 172.21.15.1:0/2688488314 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a810b4a010 0x55a810b55bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.253+0000 7fa3fde00700 1 -- 172.21.15.1:0/2688488314 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3e0009990 con 0x55a810b557e0 2024-02-02T15:46:43.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.253+0000 7fa3fde00700 1 --2- 172.21.15.1:0/2688488314 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a810b557e0 0x55a810b56640 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fa3f000efa0 tx=0x7fa3f0004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e993fe8a400bf39e server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:43.260 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.253+0000 7fa3e6a00700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:46:43.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.253+0000 7fa3e6a00700 1 -- 172.21.15.1:0/2688488314 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f0019b90 con 0x55a810b557e0 2024-02-02T15:46:43.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.253+0000 7fa4138b5b80 1 -- 172.21.15.1:0/2688488314 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a810b3e0f0 con 0x55a810b557e0 2024-02-02T15:46:43.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.253+0000 7fa3e6a00700 1 -- 172.21.15.1:0/2688488314 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fa3f0019cf0 con 0x55a810b557e0 2024-02-02T15:46:43.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.254+0000 7fa3e6a00700 1 -- 172.21.15.1:0/2688488314 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa3f0009020 con 0x55a810b557e0 2024-02-02T15:46:43.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.254+0000 7fa4138b5b80 10 client.20316 fetch_fsmap learned FSMap version 81 2024-02-02T15:46:43.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.254+0000 7fa4138b5b80 1 -- 172.21.15.1:0/2688488314 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) v3 -- 0x7fa3f81a6100 con 0x55a810b557e0 2024-02-02T15:46:43.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.254+0000 7fa3e6a00700 1 -- 172.21.15.1:0/2688488314 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a810b3e0f0 con 0x55a810b557e0 2024-02-02T15:46:43.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.254+0000 7fa3e6a00700 1 -- 172.21.15.1:0/2688488314 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 81) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fa3f0008cd0 con 0x55a810b557e0 2024-02-02T15:46:43.261 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.254+0000 7fa4138b5b80 10 client.20316 fetch_fsmap finished waiting for FSMap version 81 2024-02-02T15:46:43.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.254+0000 7fa4138b5b80 10 client.20316 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-02-02T15:46:43.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.254+0000 7fa4138b5b80 20 client.20316 populate_metadata read hostname 'smithi001' 2024-02-02T15:46:43.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.254+0000 7fa4138b5b80 1 --2- 172.21.15.1:0/2688488314 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55a810b64f30 0x55a810b67350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:43.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.254+0000 7fa4138b5b80 4 client.20316 mds_command: new command op to 24514 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:46:43.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.254+0000 7fa4138b5b80 1 -- 172.21.15.1:0/2688488314 --> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a810a94dd0 con 0x55a810b64f30 2024-02-02T15:46:43.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.255+0000 7fa3fd400700 1 --2- 172.21.15.1:0/2688488314 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55a810b64f30 0x55a810b67350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:43.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.255+0000 7fa3fd400700 1 --2- 172.21.15.1:0/2688488314 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55a810b64f30 0x55a810b67350 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5a5a953cf88ebde9 server_cookie=4ae694692499f4c7 in_seq=0 out_seq=0 2024-02-02T15:46:43.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.255+0000 7fa3e6a00700 10 client.20316 ms_handle_connect on v2:172.21.15.142:6836/2870289019 2024-02-02T15:46:43.262 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.256+0000 7fa3e6a00700 1 -- 172.21.15.1:0/2688488314 <== mds.3 v2:172.21.15.142:6836/2870289019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55a810a94dd0 con 0x55a810b64f30 2024-02-02T15:46:43.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.256+0000 7fa3e6a00700 10 client.20316 handle_command_reply: tid=0 2024-02-02T15:46:43.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.256+0000 7fa4138b5b80 1 client.20316 shutdown 2024-02-02T15:46:43.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.256+0000 7fa4138b5b80 1 -- 172.21.15.1:0/2688488314 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a810b557e0 msgr2=0x55a810b56640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:43.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.256+0000 7fa4138b5b80 1 --2- 172.21.15.1:0/2688488314 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a810b557e0 0x55a810b56640 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fa3f000efa0 tx=0x7fa3f0004b40 comp rx=0 tx=0).stop 2024-02-02T15:46:43.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.256+0000 7fa4138b5b80 1 -- 172.21.15.1:0/2688488314 shutdown_connections 2024-02-02T15:46:43.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.256+0000 7fa4138b5b80 1 --2- 172.21.15.1:0/2688488314 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55a810b64f30 0x55a810b67350 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.256+0000 7fa4138b5b80 1 --2- 172.21.15.1:0/2688488314 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a810b557e0 0x55a810b56640 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.263 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.256+0000 7fa4138b5b80 1 --2- 172.21.15.1:0/2688488314 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a810b4a010 0x55a810b55bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.256+0000 7fa4138b5b80 1 --2- 172.21.15.1:0/2688488314 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a810b4a930 0x55a810b56100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.256+0000 7fa3e6a00700 0 client.20316 ms_handle_reset on v2:172.21.15.142:6836/2870289019 2024-02-02T15:46:43.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.256+0000 7fa4138b5b80 1 -- 172.21.15.1:0/2688488314 >> 172.21.15.1:0/2688488314 conn(0x55a810aa9db0 msgr2=0x55a810b45fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:43.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.256+0000 7fa4138b5b80 1 -- 172.21.15.1:0/2688488314 shutdown_connections 2024-02-02T15:46:43.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.256+0000 7fa4138b5b80 1 -- 172.21.15.1:0/2688488314 wait complete. 2024-02-02T15:46:43.264 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.256+0000 7fa4138b5b80 20 client.20316 trim_cache size 0 max 16384 2024-02-02T15:46:43.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.269+0000 7fa4138b5b80 1 Processor -- start 2024-02-02T15:46:43.269 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.270+0000 7fa4138b5b80 1 -- start start 2024-02-02T15:46:43.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.270+0000 7fa4138b5b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a810b4a010 0x55a810b5abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:43.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.270+0000 7fa4138b5b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a810b4a930 0x55a810b4cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:43.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.270+0000 7fa4138b5b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a810b557e0 0x55a810b601b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:43.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.270+0000 7fa4138b5b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a810b4f770 con 0x55a810b4a930 2024-02-02T15:46:43.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.270+0000 7fa4138b5b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a81091f6a0 con 0x55a810b557e0 2024-02-02T15:46:43.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.270+0000 7fa4138b5b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a810b506c0 con 0x55a810b4a010 2024-02-02T15:46:43.270 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.270+0000 7fa3fca00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a810b4a930 0x55a810b4cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:43.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.270+0000 7fa3fca00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a810b4a930 0x55a810b4cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41504/0 (socket says 172.21.15.1:41504) 2024-02-02T15:46:43.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.270+0000 7fa3fca00700 1 -- 172.21.15.1:0/3592115404 learned_addr learned my addr 172.21.15.1:0/3592115404 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:46:43.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.270+0000 7fa3fd400700 1 --2- 172.21.15.1:0/3592115404 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a810b4a010 0x55a810b5abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:43.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.270+0000 7fa3fde00700 1 --2- 172.21.15.1:0/3592115404 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a810b557e0 0x55a810b601b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:43.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.271+0000 7fa3fca00700 1 -- 172.21.15.1:0/3592115404 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a810b4a010 msgr2=0x55a810b5abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:43.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.271+0000 7fa3fca00700 1 --2- 172.21.15.1:0/3592115404 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a810b4a010 0x55a810b5abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.271+0000 7fa3fca00700 1 -- 172.21.15.1:0/3592115404 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a810b557e0 msgr2=0x55a810b601b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:43.271 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.271+0000 7fa3fca00700 1 --2- 172.21.15.1:0/3592115404 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a810b557e0 0x55a810b601b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.271+0000 7fa3fca00700 1 -- 172.21.15.1:0/3592115404 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8108c9b60 con 0x55a810b4a930 2024-02-02T15:46:43.272 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.271+0000 7fa3fca00700 1 --2- 172.21.15.1:0/3592115404 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a810b4a930 0x55a810b4cab0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0012590 tx=0x7fa3e0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fe18dbba2ba8eb3 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:43.280 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.271+0000 7fa3ef000700 1 -- 172.21.15.1:0/3592115404 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e0017050 con 0x55a810b4a930 2024-02-02T15:46:43.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.271+0000 7fa3ef000700 1 -- 172.21.15.1:0/3592115404 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa3e0018040 con 0x55a810b4a930 2024-02-02T15:46:43.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.271+0000 7fa3ef000700 1 -- 172.21.15.1:0/3592115404 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e0003dc0 con 0x55a810b4a930 2024-02-02T15:46:43.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.272+0000 7fa4138b5b80 1 -- 172.21.15.1:0/3592115404 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a810b4a930 msgr2=0x55a810b4cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:43.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.272+0000 7fa4138b5b80 1 --2- 172.21.15.1:0/3592115404 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a810b4a930 0x55a810b4cab0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0012590 tx=0x7fa3e0009260 comp rx=0 tx=0).stop 2024-02-02T15:46:43.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.272+0000 7fa4138b5b80 1 -- 172.21.15.1:0/3592115404 shutdown_connections 2024-02-02T15:46:43.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.272+0000 7fa4138b5b80 1 --2- 172.21.15.1:0/3592115404 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a810b557e0 0x55a810b601b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.281 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.272+0000 7fa4138b5b80 1 --2- 172.21.15.1:0/3592115404 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a810b4a010 0x55a810b5abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.272+0000 7fa4138b5b80 1 --2- 172.21.15.1:0/3592115404 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a810b4a930 0x55a810b4cab0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.272+0000 7fa4138b5b80 1 -- 172.21.15.1:0/3592115404 >> 172.21.15.1:0/3592115404 conn(0x55a810aa9db0 msgr2=0x55a810b4fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:43.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.272+0000 7fa4138b5b80 1 -- 172.21.15.1:0/3592115404 shutdown_connections 2024-02-02T15:46:43.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.272+0000 7fa4138b5b80 1 -- 172.21.15.1:0/3592115404 wait complete. 2024-02-02T15:46:43.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.272+0000 7fa4138b5b80 1 -- start start 2024-02-02T15:46:43.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.272+0000 7fa4138b5b80 1 Processor -- start 2024-02-02T15:46:43.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.273+0000 7fa4138b5b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a810b4a010 0x55a810b5a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:43.282 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.273+0000 7fa4138b5b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a810b4a930 0x55a810b5a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:43.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.273+0000 7fa4138b5b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a810b557e0 0x55a810b587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:43.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.273+0000 7fa4138b5b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a810b58de0 con 0x55a810b4a930 2024-02-02T15:46:43.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.273+0000 7fa4138b5b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a810b58f50 con 0x55a810b557e0 2024-02-02T15:46:43.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.273+0000 7fa4138b5b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a810b59110 con 0x55a810b4a010 2024-02-02T15:46:43.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.273+0000 7fa4138b5b80 10 client.0 _ll_register_callbacks cb 0x55a810b47b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:46:43.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.273+0000 7fa3fd400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a810b4a010 0x55a810b5a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:43.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.273+0000 7fa3fca00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a810b4a930 0x55a810b5a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:43.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.273+0000 7fa3fd400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a810b4a010 0x55a810b5a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:51786/0 (socket says 172.21.15.1:51786) 2024-02-02T15:46:43.283 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.273+0000 7fa3fd400700 1 -- 172.21.15.1:0/4153976378 learned_addr learned my addr 172.21.15.1:0/4153976378 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:46:43.284 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.273+0000 7fa3fde00700 1 --2- 172.21.15.1:0/4153976378 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a810b557e0 0x55a810b587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:43.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.274+0000 7fa3fd400700 1 -- 172.21.15.1:0/4153976378 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a810b557e0 msgr2=0x55a810b587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:43.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.274+0000 7fa3fd400700 1 --2- 172.21.15.1:0/4153976378 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a810b557e0 0x55a810b587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.274+0000 7fa3fd400700 1 -- 172.21.15.1:0/4153976378 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a810b4a930 msgr2=0x55a810b5a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:43.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.274+0000 7fa3fd400700 1 --2- 172.21.15.1:0/4153976378 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a810b4a930 0x55a810b5a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.274+0000 7fa3fd400700 1 -- 172.21.15.1:0/4153976378 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3e0009e20 con 0x55a810b4a010 2024-02-02T15:46:43.285 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.274+0000 7fa3fd400700 1 --2- 172.21.15.1:0/4153976378 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a810b4a010 0x55a810b5a310 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fa3e800c010 tx=0x7fa3e8022ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7376a5cf9726db29 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:43.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.274+0000 7fa3e6a00700 10 client.0 ms_handle_connect on v2:172.21.15.142:3300/0 2024-02-02T15:46:43.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.274+0000 7fa3e6a00700 1 -- 172.21.15.1:0/4153976378 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e80057b0 con 0x55a810b4a010 2024-02-02T15:46:43.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.274+0000 7fa3e6a00700 1 -- 172.21.15.1:0/4153976378 <== mon.1 v2:172.21.15.142:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fa3e809e270 con 0x55a810b4a010 2024-02-02T15:46:43.286 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.274+0000 7fa4138b5b80 1 -- 172.21.15.1:0/4153976378 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa3e80850f0 con 0x55a810b4a010 2024-02-02T15:46:43.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.274+0000 7fa3e6a00700 1 -- 172.21.15.1:0/4153976378 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa3e809ab40 con 0x55a810b4a010 2024-02-02T15:46:43.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.274+0000 7fa4138b5b80 10 client.30457 fetch_fsmap learned FSMap version 81 2024-02-02T15:46:43.287 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.274+0000 7fa4138b5b80 1 -- 172.21.15.1:0/4153976378 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) v3 -- 0x55a810a61ea0 con 0x55a810b4a010 2024-02-02T15:46:43.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.275+0000 7fa3e6a00700 1 -- 172.21.15.1:0/4153976378 <== mon.1 v2:172.21.15.142:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa3e80850f0 con 0x55a810b4a010 2024-02-02T15:46:43.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.275+0000 7fa3e6a00700 1 -- 172.21.15.1:0/4153976378 <== mon.1 v2:172.21.15.142:3300/0 5 ==== fsmap(e 81) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fa3e8022900 con 0x55a810b4a010 2024-02-02T15:46:43.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.275+0000 7fa4138b5b80 10 client.30457 fetch_fsmap finished waiting for FSMap version 81 2024-02-02T15:46:43.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.275+0000 7fa4138b5b80 10 client.30457 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-02-02T15:46:43.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.275+0000 7fa4138b5b80 20 client.30457 populate_metadata read hostname 'smithi001' 2024-02-02T15:46:43.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.275+0000 7fa4138b5b80 1 --2- 172.21.15.1:0/4153976378 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55a810bb7640 0x55a810bb9a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:43.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.275+0000 7fa4138b5b80 4 client.30457 mds_command: new command op to 24514 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:46:43.288 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.275+0000 7fa4138b5b80 1 -- 172.21.15.1:0/4153976378 --> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a810a5af50 con 0x55a810bb7640 2024-02-02T15:46:43.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.275+0000 7fa3fca00700 1 --2- 172.21.15.1:0/4153976378 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55a810bb7640 0x55a810bb9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:43.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.276+0000 7fa3fca00700 1 --2- 172.21.15.1:0/4153976378 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55a810bb7640 0x55a810bb9a30 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f24f4c4e9efc04e0 server_cookie=1a411ccfe89120b5 in_seq=0 out_seq=0 2024-02-02T15:46:43.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.276+0000 7fa3e6a00700 10 client.30457 ms_handle_connect on v2:172.21.15.142:6836/2870289019 2024-02-02T15:46:43.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.276+0000 7fa3e6a00700 1 -- 172.21.15.1:0/4153976378 <== mds.3 v2:172.21.15.142:6836/2870289019 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a810a5af50 con 0x55a810bb7640 2024-02-02T15:46:43.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.276+0000 7fa3e6a00700 10 client.30457 handle_command_reply: tid=0 2024-02-02T15:46:43.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.276+0000 7fa4138b5b80 1 client.30457 shutdown 2024-02-02T15:46:43.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.276+0000 7fa4138b5b80 1 -- 172.21.15.1:0/4153976378 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a810b4a010 msgr2=0x55a810b5a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:43.289 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.276+0000 7fa4138b5b80 1 --2- 172.21.15.1:0/4153976378 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a810b4a010 0x55a810b5a310 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fa3e800c010 tx=0x7fa3e8022ca0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.276+0000 7fa4138b5b80 1 -- 172.21.15.1:0/4153976378 shutdown_connections 2024-02-02T15:46:43.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.276+0000 7fa4138b5b80 1 --2- 172.21.15.1:0/4153976378 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a810b557e0 0x55a810b587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.276+0000 7fa4138b5b80 1 --2- 172.21.15.1:0/4153976378 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x55a810b4a010 0x55a810b5a310 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.276+0000 7fa4138b5b80 1 --2- 172.21.15.1:0/4153976378 >> [v2:172.21.15.142:6836/2870289019,v1:172.21.15.142:6837/2870289019] conn(0x55a810bb7640 0x55a810bb9a30 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.276+0000 7fa4138b5b80 1 --2- 172.21.15.1:0/4153976378 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x55a810b4a930 0x55a810b5a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.276+0000 7fa4138b5b80 1 -- 172.21.15.1:0/4153976378 >> 172.21.15.1:0/4153976378 conn(0x55a810aa9db0 msgr2=0x55a810b45e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:43.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.276+0000 7fa3e6a00700 0 client.30457 ms_handle_reset on v2:172.21.15.142:6836/2870289019 2024-02-02T15:46:43.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.276+0000 7fa4138b5b80 1 -- 172.21.15.1:0/4153976378 shutdown_connections 2024-02-02T15:46:43.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.276+0000 7fa4138b5b80 1 -- 172.21.15.1:0/4153976378 wait complete. 2024-02-02T15:46:43.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.276+0000 7fa4138b5b80 20 client.30457 trim_cache size 0 max 16384 2024-02-02T15:46:43.291 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:46:43.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.277+0000 7fa3ffe00700 1 -- 172.21.15.1:0/3836308651 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fa3d80a0690 msgr2=0x7fa3d80a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:43.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.277+0000 7fa3ffe00700 1 --2- 172.21.15.1:0/3836308651 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fa3d80a0690 0x7fa3d80a2b50 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fa3f0003e80 tx=0x7fa3f0009920 comp rx=0 tx=0).stop 2024-02-02T15:46:43.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.277+0000 7fa3ffe00700 1 -- 172.21.15.1:0/3836308651 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa3f8100ff0 msgr2=0x7fa3f8073040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:43.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.277+0000 7fa3ffe00700 1 --2- 172.21.15.1:0/3836308651 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa3f8100ff0 0x7fa3f8073040 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fa3e80059d0 tx=0x7fa3e8005740 comp rx=0 tx=0).stop 2024-02-02T15:46:43.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.277+0000 7fa3ffe00700 1 -- 172.21.15.1:0/3836308651 shutdown_connections 2024-02-02T15:46:43.292 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.277+0000 7fa3ffe00700 1 --2- 172.21.15.1:0/3836308651 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fa3d80a0690 0x7fa3d80a2b50 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.277+0000 7fa3ffe00700 1 --2- 172.21.15.1:0/3836308651 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3f8073ac0 0x7fa3f81a50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.277+0000 7fa3ffe00700 1 --2- 172.21.15.1:0/3836308651 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fa3f8100ff0 0x7fa3f8073040 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.277+0000 7fa3ffe00700 1 --2- 172.21.15.1:0/3836308651 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fa3f8106110 0x7fa3f8073580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.277+0000 7fa3ffe00700 1 -- 172.21.15.1:0/3836308651 >> 172.21.15.1:0/3836308651 conn(0x7fa3f80fc810 msgr2=0x7fa3f80fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:43.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.277+0000 7fa3ffe00700 1 -- 172.21.15.1:0/3836308651 shutdown_connections 2024-02-02T15:46:43.293 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.277+0000 7fa3ffe00700 1 -- 172.21.15.1:0/3836308651 wait complete. 2024-02-02T15:46:43.294 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-02-02T15:46:43.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.496+0000 7fca1f600700 1 -- 172.21.15.1:0/1523014175 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca18101000 msgr2=0x7fca181013e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:43.495 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.496+0000 7fca1f600700 1 --2- 172.21.15.1:0/1523014175 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca18101000 0x7fca181013e0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fca08009990 tx=0x7fca08009ca0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.496+0000 7fca1f600700 1 -- 172.21.15.1:0/1523014175 shutdown_connections 2024-02-02T15:46:43.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.496+0000 7fca1f600700 1 --2- 172.21.15.1:0/1523014175 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca18106120 0x7fca1810dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.496+0000 7fca1f600700 1 --2- 172.21.15.1:0/1523014175 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca18101000 0x7fca181013e0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.496+0000 7fca1f600700 1 --2- 172.21.15.1:0/1523014175 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca181019b0 0x7fca181059f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.496+0000 7fca1f600700 1 -- 172.21.15.1:0/1523014175 >> 172.21.15.1:0/1523014175 conn(0x7fca180fc860 msgr2=0x7fca180fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:43.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.496+0000 7fca1f600700 1 -- 172.21.15.1:0/1523014175 shutdown_connections 2024-02-02T15:46:43.496 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.496+0000 7fca1f600700 1 -- 172.21.15.1:0/1523014175 wait complete. 2024-02-02T15:46:43.498 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.497+0000 7fca1f600700 1 Processor -- start 2024-02-02T15:46:43.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.497+0000 7fca1f600700 1 -- start start 2024-02-02T15:46:43.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.498+0000 7fca1f600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca18101000 0x7fca18073040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:43.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.498+0000 7fca1f600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca18106120 0x7fca18073580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:43.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.498+0000 7fca1f600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca18073ac0 0x7fca181a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:43.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.498+0000 7fca1f600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca181a5750 con 0x7fca18106120 2024-02-02T15:46:43.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.498+0000 7fca1f600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca181a58c0 con 0x7fca18101000 2024-02-02T15:46:43.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.498+0000 7fca1f600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca181a5a80 con 0x7fca18073ac0 2024-02-02T15:46:43.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.498+0000 7fca17e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca18106120 0x7fca18073580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:43.499 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.498+0000 7fca17e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca18106120 0x7fca18073580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41532/0 (socket says 172.21.15.1:41532) 2024-02-02T15:46:43.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.498+0000 7fca17e00700 1 -- 172.21.15.1:0/1353883453 learned_addr learned my addr 172.21.15.1:0/1353883453 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:46:43.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.498+0000 7fca1cc00700 1 --2- 172.21.15.1:0/1353883453 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca18101000 0x7fca18073040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:43.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.498+0000 7fca1d600700 1 --2- 172.21.15.1:0/1353883453 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca18073ac0 0x7fca181a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:43.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.498+0000 7fca17e00700 1 -- 172.21.15.1:0/1353883453 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca18073ac0 msgr2=0x7fca181a50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:43.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.498+0000 7fca17e00700 1 --2- 172.21.15.1:0/1353883453 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca18073ac0 0x7fca181a50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.498+0000 7fca17e00700 1 -- 172.21.15.1:0/1353883453 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca18101000 msgr2=0x7fca18073040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:43.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.498+0000 7fca17e00700 1 --2- 172.21.15.1:0/1353883453 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca18101000 0x7fca18073040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.500 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.498+0000 7fca17e00700 1 -- 172.21.15.1:0/1353883453 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca080095f0 con 0x7fca18106120 2024-02-02T15:46:43.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.499+0000 7fca17e00700 1 --2- 172.21.15.1:0/1353883453 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca18106120 0x7fca18073580 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fca0000e970 tx=0x7fca0000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:43.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.499+0000 7fca15600700 1 -- 172.21.15.1:0/1353883453 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca00009790 con 0x7fca18106120 2024-02-02T15:46:43.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.499+0000 7fca15600700 1 -- 172.21.15.1:0/1353883453 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca0000eec0 con 0x7fca18106120 2024-02-02T15:46:43.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.499+0000 7fca1f600700 1 -- 172.21.15.1:0/1353883453 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca181a5d60 con 0x7fca18106120 2024-02-02T15:46:43.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.499+0000 7fca15600700 1 -- 172.21.15.1:0/1353883453 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca00005590 con 0x7fca18106120 2024-02-02T15:46:43.501 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.499+0000 7fca1f600700 1 -- 172.21.15.1:0/1353883453 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca181a6210 con 0x7fca18106120 2024-02-02T15:46:43.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.503+0000 7fca15600700 1 -- 172.21.15.1:0/1353883453 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7fca0000cbd0 con 0x7fca18106120 2024-02-02T15:46:43.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.503+0000 7fca33111b80 1 Processor -- start 2024-02-02T15:46:43.502 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.503+0000 7fca33111b80 1 -- start start 2024-02-02T15:46:43.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.504+0000 7fca33111b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5599fdc1d010 0x5599fdc1d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:43.504 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.504+0000 7fca33111b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5599fdc1d930 0x5599fdc282a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:43.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.504+0000 7fca33111b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5599fdc287e0 0x5599fdc2abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:43.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.504+0000 7fca33111b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599fdb914c0 con 0x5599fdc1d930 2024-02-02T15:46:43.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.504+0000 7fca33111b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599fdb75aa0 con 0x5599fdc287e0 2024-02-02T15:46:43.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.504+0000 7fca33111b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599fdb73460 con 0x5599fdc1d010 2024-02-02T15:46:43.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.504+0000 7fca1cc00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5599fdc1d930 0x5599fdc282a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:43.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.504+0000 7fca1d600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5599fdc1d010 0x5599fdc1d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:43.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.504+0000 7fca1d600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5599fdc1d010 0x5599fdc1d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:51828/0 (socket says 172.21.15.1:51828) 2024-02-02T15:46:43.505 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.504+0000 7fca1d600700 1 -- 172.21.15.1:0/2342333063 learned_addr learned my addr 172.21.15.1:0/2342333063 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:46:43.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.504+0000 7fca17e00700 1 --2- 172.21.15.1:0/2342333063 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5599fdc287e0 0x5599fdc2abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:43.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.505+0000 7fca15600700 1 --2- 172.21.15.1:0/1353883453 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fc9f80a0690 0x7fc9f80a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:43.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.505+0000 7fca1d600700 1 -- 172.21.15.1:0/2342333063 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5599fdc287e0 msgr2=0x5599fdc2abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:43.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.505+0000 7fca1d600700 1 --2- 172.21.15.1:0/2342333063 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5599fdc287e0 0x5599fdc2abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.505+0000 7fca1d600700 1 -- 172.21.15.1:0/2342333063 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5599fdc1d930 msgr2=0x5599fdc282a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:43.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.505+0000 7fca1d600700 1 --2- 172.21.15.1:0/2342333063 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5599fdc1d930 0x5599fdc282a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.505+0000 7fca1d600700 1 -- 172.21.15.1:0/2342333063 --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599fd9f03b0 con 0x5599fdc1d010 2024-02-02T15:46:43.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.505+0000 7fca1cc00700 1 --2- 172.21.15.1:0/2342333063 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5599fdc1d930 0x5599fdc282a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:46:43.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.505+0000 7fca15600700 1 -- 172.21.15.1:0/1353883453 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fca00014070 con 0x7fca18106120 2024-02-02T15:46:43.506 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.505+0000 7fca1d600700 1 --2- 172.21.15.1:0/1353883453 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fc9f80a0690 0x7fc9f80a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:43.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.505+0000 7fca1d600700 1 --2- 172.21.15.1:0/2342333063 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5599fdc1d010 0x5599fdc1d3f0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fca10009470 tx=0x7fca10009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9bff2aa6aad371a server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:43.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.505+0000 7fca0dc00700 1 -- 172.21.15.1:0/2342333063 <== mon.1 v2:172.21.15.142:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1000e070 con 0x5599fdc1d010 2024-02-02T15:46:43.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.506+0000 7fca0dc00700 1 -- 172.21.15.1:0/2342333063 <== mon.1 v2:172.21.15.142:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca1000f040 con 0x5599fdc1d010 2024-02-02T15:46:43.507 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.506+0000 7fca0dc00700 1 -- 172.21.15.1:0/2342333063 <== mon.1 v2:172.21.15.142:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca10014760 con 0x5599fdc1d010 2024-02-02T15:46:43.511 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.506+0000 7fca33111b80 1 -- 172.21.15.1:0/2342333063 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5599fdc1d010 msgr2=0x5599fdc1d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:43.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.506+0000 7fca1d600700 1 --2- 172.21.15.1:0/1353883453 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fc9f80a0690 0x7fc9f80a2b50 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fca1001e000 tx=0x7fca10021000 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:43.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.506+0000 7fca33111b80 1 --2- 172.21.15.1:0/2342333063 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5599fdc1d010 0x5599fdc1d3f0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fca10009470 tx=0x7fca10009920 comp rx=0 tx=0).stop 2024-02-02T15:46:43.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.506+0000 7fca33111b80 1 -- 172.21.15.1:0/2342333063 shutdown_connections 2024-02-02T15:46:43.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.506+0000 7fca33111b80 1 --2- 172.21.15.1:0/2342333063 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5599fdc287e0 0x5599fdc2abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.506+0000 7fca33111b80 1 --2- 172.21.15.1:0/2342333063 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5599fdc1d010 0x5599fdc1d3f0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.506+0000 7fca33111b80 1 --2- 172.21.15.1:0/2342333063 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5599fdc1d930 0x5599fdc282a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.506+0000 7fca33111b80 1 -- 172.21.15.1:0/2342333063 >> 172.21.15.1:0/2342333063 conn(0x5599fdb7cdb0 msgr2=0x5599fdc193b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:43.512 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.506+0000 7fca33111b80 1 -- 172.21.15.1:0/2342333063 shutdown_connections 2024-02-02T15:46:43.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.507+0000 7fca33111b80 1 -- 172.21.15.1:0/2342333063 wait complete. 2024-02-02T15:46:43.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.507+0000 7fca33111b80 1 -- start start 2024-02-02T15:46:43.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.507+0000 7fca33111b80 1 Processor -- start 2024-02-02T15:46:43.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.508+0000 7fca33111b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5599fdc1d010 0x5599fdc28190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:43.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.508+0000 7fca33111b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5599fdc1d930 0x5599fdc253a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:43.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.508+0000 7fca33111b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5599fdc287e0 0x5599fdc25960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:43.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.508+0000 7fca33111b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599fdb6ed50 con 0x5599fdc287e0 2024-02-02T15:46:43.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.508+0000 7fca33111b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599fdb0ad80 con 0x5599fdc1d010 2024-02-02T15:46:43.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.508+0000 7fca33111b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599fdb0aa20 con 0x5599fdc1d930 2024-02-02T15:46:43.513 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.508+0000 7fca33111b80 10 client.0 _ll_register_callbacks cb 0x5599fdb0c290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:46:43.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.508+0000 7fca1d600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5599fdc287e0 0x5599fdc25960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:43.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.508+0000 7fca1cc00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5599fdc1d010 0x5599fdc28190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:43.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.508+0000 7fca1cc00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5599fdc1d010 0x5599fdc28190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:33166/0 (socket says 172.21.15.1:33166) 2024-02-02T15:46:43.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.508+0000 7fca1cc00700 1 -- 172.21.15.1:0/17523585 learned_addr learned my addr 172.21.15.1:0/17523585 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:46:43.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.508+0000 7fca17e00700 1 --2- 172.21.15.1:0/17523585 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5599fdc1d930 0x5599fdc253a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:43.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.508+0000 7fca1d600700 1 -- 172.21.15.1:0/17523585 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5599fdc1d930 msgr2=0x5599fdc253a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:43.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.508+0000 7fca1d600700 1 --2- 172.21.15.1:0/17523585 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5599fdc1d930 0x5599fdc253a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.508+0000 7fca1d600700 1 -- 172.21.15.1:0/17523585 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5599fdc1d010 msgr2=0x5599fdc28190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:43.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.508+0000 7fca1d600700 1 --2- 172.21.15.1:0/17523585 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5599fdc1d010 0x5599fdc28190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.514 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.508+0000 7fca1d600700 1 -- 172.21.15.1:0/17523585 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca1810e510 con 0x5599fdc287e0 2024-02-02T15:46:43.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.508+0000 7fca1cc00700 1 --2- 172.21.15.1:0/17523585 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5599fdc1d010 0x5599fdc28190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:46:43.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.509+0000 7fca1d600700 1 --2- 172.21.15.1:0/17523585 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5599fdc287e0 0x5599fdc25960 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fca10009410 tx=0x7fca100146b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=396ed675d1c91e52 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:43.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.509+0000 7fca06000700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:46:43.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.509+0000 7fca06000700 1 -- 172.21.15.1:0/17523585 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1000e070 con 0x5599fdc287e0 2024-02-02T15:46:43.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.509+0000 7fca06000700 1 -- 172.21.15.1:0/17523585 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fca10014ac0 con 0x5599fdc287e0 2024-02-02T15:46:43.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.509+0000 7fca33111b80 1 -- 172.21.15.1:0/17523585 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5599fdc110f0 con 0x5599fdc287e0 2024-02-02T15:46:43.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.509+0000 7fca06000700 1 -- 172.21.15.1:0/17523585 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca10014c80 con 0x5599fdc287e0 2024-02-02T15:46:43.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.509+0000 7fca33111b80 10 client.20334 fetch_fsmap learned FSMap version 81 2024-02-02T15:46:43.515 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.509+0000 7fca33111b80 1 -- 172.21.15.1:0/17523585 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) v3 -- 0x7fca100090f0 con 0x5599fdc287e0 2024-02-02T15:46:43.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.510+0000 7fca06000700 1 -- 172.21.15.1:0/17523585 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5599fdc110f0 con 0x5599fdc287e0 2024-02-02T15:46:43.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.510+0000 7fca06000700 1 -- 172.21.15.1:0/17523585 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 81) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fca10022410 con 0x5599fdc287e0 2024-02-02T15:46:43.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.510+0000 7fca33111b80 10 client.20334 fetch_fsmap finished waiting for FSMap version 81 2024-02-02T15:46:43.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.510+0000 7fca33111b80 10 client.20334 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-02-02T15:46:43.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.510+0000 7fca33111b80 20 client.20334 populate_metadata read hostname 'smithi001' 2024-02-02T15:46:43.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.510+0000 7fca33111b80 1 --2- 172.21.15.1:0/17523585 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x5599fdc38080 0x5599fdc3a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:43.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.510+0000 7fca33111b80 4 client.20334 mds_command: new command op to 24518 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:46:43.516 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.510+0000 7fca33111b80 1 -- 172.21.15.1:0/17523585 --> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5599fdb67dd0 con 0x5599fdc38080 2024-02-02T15:46:43.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.510+0000 7fca1cc00700 1 --2- 172.21.15.1:0/17523585 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x5599fdc38080 0x5599fdc3a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:43.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.511+0000 7fca1cc00700 1 --2- 172.21.15.1:0/17523585 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x5599fdc38080 0x5599fdc3a4a0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=74851370c76c316e server_cookie=e55b5ac78be01191 in_seq=0 out_seq=0 2024-02-02T15:46:43.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.511+0000 7fca06000700 10 client.20334 ms_handle_connect on v2:172.21.15.71:6836/628016199 2024-02-02T15:46:43.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.512+0000 7fca06000700 1 -- 172.21.15.1:0/17523585 <== mds.2 v2:172.21.15.71:6836/628016199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5599fdb67dd0 con 0x5599fdc38080 2024-02-02T15:46:43.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.512+0000 7fca06000700 10 client.20334 handle_command_reply: tid=0 2024-02-02T15:46:43.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.512+0000 7fca33111b80 1 client.20334 shutdown 2024-02-02T15:46:43.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.512+0000 7fca33111b80 1 -- 172.21.15.1:0/17523585 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5599fdc287e0 msgr2=0x5599fdc25960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:43.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.512+0000 7fca33111b80 1 --2- 172.21.15.1:0/17523585 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5599fdc287e0 0x5599fdc25960 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fca10009410 tx=0x7fca100146b0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.517 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.512+0000 7fca33111b80 1 -- 172.21.15.1:0/17523585 shutdown_connections 2024-02-02T15:46:43.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.512+0000 7fca33111b80 1 --2- 172.21.15.1:0/17523585 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5599fdc287e0 0x5599fdc25960 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.512+0000 7fca33111b80 1 --2- 172.21.15.1:0/17523585 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5599fdc1d010 0x5599fdc28190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.512+0000 7fca33111b80 1 --2- 172.21.15.1:0/17523585 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x5599fdc38080 0x5599fdc3a4a0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.512+0000 7fca33111b80 1 --2- 172.21.15.1:0/17523585 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5599fdc1d930 0x5599fdc253a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.512+0000 7fca33111b80 1 -- 172.21.15.1:0/17523585 >> 172.21.15.1:0/17523585 conn(0x5599fdb7cdb0 msgr2=0x5599fdc18fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:43.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.512+0000 7fca06000700 0 client.20334 ms_handle_reset on v2:172.21.15.71:6836/628016199 2024-02-02T15:46:43.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.512+0000 7fca33111b80 1 -- 172.21.15.1:0/17523585 shutdown_connections 2024-02-02T15:46:43.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.512+0000 7fca33111b80 1 -- 172.21.15.1:0/17523585 wait complete. 2024-02-02T15:46:43.518 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.513+0000 7fca33111b80 20 client.20334 trim_cache size 0 max 16384 2024-02-02T15:46:43.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.526+0000 7fca33111b80 1 Processor -- start 2024-02-02T15:46:43.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.526+0000 7fca33111b80 1 -- start start 2024-02-02T15:46:43.526 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.526+0000 7fca33111b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5599fdc1d010 0x5599fdc2f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:43.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.526+0000 7fca33111b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5599fdc1d930 0x5599fdc34510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:43.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.526+0000 7fca33111b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5599fdc287e0 0x5599fdc27cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:43.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.526+0000 7fca33111b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599fd9f26a0 con 0x5599fdc1d930 2024-02-02T15:46:43.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.526+0000 7fca33111b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599fdc1e2d0 con 0x5599fdc287e0 2024-02-02T15:46:43.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.526+0000 7fca33111b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599fdc25fe0 con 0x5599fdc1d010 2024-02-02T15:46:43.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.527+0000 7fca17e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5599fdc1d930 0x5599fdc34510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:43.527 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.527+0000 7fca1cc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5599fdc1d010 0x5599fdc2f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:43.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.527+0000 7fca17e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5599fdc1d930 0x5599fdc34510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41568/0 (socket says 172.21.15.1:41568) 2024-02-02T15:46:43.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.527+0000 7fca17e00700 1 -- 172.21.15.1:0/573523874 learned_addr learned my addr 172.21.15.1:0/573523874 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:46:43.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.527+0000 7fca1d600700 1 --2- 172.21.15.1:0/573523874 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5599fdc287e0 0x5599fdc27cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:43.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.527+0000 7fca17e00700 1 -- 172.21.15.1:0/573523874 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5599fdc1d010 msgr2=0x5599fdc2f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:43.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.527+0000 7fca17e00700 1 --2- 172.21.15.1:0/573523874 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5599fdc1d010 0x5599fdc2f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.527+0000 7fca17e00700 1 -- 172.21.15.1:0/573523874 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5599fdc287e0 msgr2=0x5599fdc27cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:43.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.527+0000 7fca17e00700 1 --2- 172.21.15.1:0/573523874 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5599fdc287e0 0x5599fdc27cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.527+0000 7fca17e00700 1 -- 172.21.15.1:0/573523874 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5599fd99cb60 con 0x5599fdc1d930 2024-02-02T15:46:43.528 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.527+0000 7fca17e00700 1 --2- 172.21.15.1:0/573523874 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5599fdc1d930 0x5599fdc34510 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fca000991e0 tx=0x7fca000ce670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d786cab1c6b31b05 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:43.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.528+0000 7fca0e600700 1 -- 172.21.15.1:0/573523874 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca00099020 con 0x5599fdc1d930 2024-02-02T15:46:43.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.528+0000 7fca0e600700 1 -- 172.21.15.1:0/573523874 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca00098e40 con 0x5599fdc1d930 2024-02-02T15:46:43.529 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.528+0000 7fca0e600700 1 -- 172.21.15.1:0/573523874 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca0009d3a0 con 0x5599fdc1d930 2024-02-02T15:46:43.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.528+0000 7fca33111b80 1 -- 172.21.15.1:0/573523874 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5599fdc1d930 msgr2=0x5599fdc34510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:43.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.528+0000 7fca33111b80 1 --2- 172.21.15.1:0/573523874 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5599fdc1d930 0x5599fdc34510 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fca000991e0 tx=0x7fca000ce670 comp rx=0 tx=0).stop 2024-02-02T15:46:43.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.528+0000 7fca33111b80 1 -- 172.21.15.1:0/573523874 shutdown_connections 2024-02-02T15:46:43.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.528+0000 7fca33111b80 1 --2- 172.21.15.1:0/573523874 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5599fdc287e0 0x5599fdc27cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.528+0000 7fca33111b80 1 --2- 172.21.15.1:0/573523874 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5599fdc1d010 0x5599fdc2f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.528+0000 7fca33111b80 1 --2- 172.21.15.1:0/573523874 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5599fdc1d930 0x5599fdc34510 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.528+0000 7fca33111b80 1 -- 172.21.15.1:0/573523874 >> 172.21.15.1:0/573523874 conn(0x5599fdb7cdb0 msgr2=0x5599fdc17b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:43.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.528+0000 7fca33111b80 1 -- 172.21.15.1:0/573523874 shutdown_connections 2024-02-02T15:46:43.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.528+0000 7fca33111b80 1 -- 172.21.15.1:0/573523874 wait complete. 2024-02-02T15:46:43.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.529+0000 7fca33111b80 1 -- start start 2024-02-02T15:46:43.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.529+0000 7fca33111b80 1 Processor -- start 2024-02-02T15:46:43.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.529+0000 7fca33111b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5599fdc1d010 0x5599fdc2b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:43.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.529+0000 7fca33111b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5599fdc1d930 0x5599fdc2baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:43.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.529+0000 7fca33111b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5599fdc287e0 0x5599fdc88b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:43.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.529+0000 7fca33111b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599fdc2c200 con 0x5599fdc287e0 2024-02-02T15:46:43.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.529+0000 7fca33111b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599fdc89090 con 0x5599fdc1d010 2024-02-02T15:46:43.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.529+0000 7fca33111b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599fdc89200 con 0x5599fdc1d930 2024-02-02T15:46:43.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.529+0000 7fca33111b80 10 client.0 _ll_register_callbacks cb 0x5599fdb0c290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:46:43.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.530+0000 7fca1cc00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5599fdc1d010 0x5599fdc2b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:43.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.530+0000 7fca1d600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5599fdc287e0 0x5599fdc88b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:43.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.530+0000 7fca1cc00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5599fdc1d010 0x5599fdc2b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:33182/0 (socket says 172.21.15.1:33182) 2024-02-02T15:46:43.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.530+0000 7fca1cc00700 1 -- 172.21.15.1:0/143197889 learned_addr learned my addr 172.21.15.1:0/143197889 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:46:43.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.530+0000 7fca17e00700 1 --2- 172.21.15.1:0/143197889 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5599fdc1d930 0x5599fdc2baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:43.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.530+0000 7fca1d600700 1 -- 172.21.15.1:0/143197889 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5599fdc1d930 msgr2=0x5599fdc2baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:43.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.530+0000 7fca1d600700 1 --2- 172.21.15.1:0/143197889 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5599fdc1d930 0x5599fdc2baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.530+0000 7fca1d600700 1 -- 172.21.15.1:0/143197889 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5599fdc1d010 msgr2=0x5599fdc2b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:43.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.530+0000 7fca1d600700 1 --2- 172.21.15.1:0/143197889 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5599fdc1d010 0x5599fdc2b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.530+0000 7fca1d600700 1 -- 172.21.15.1:0/143197889 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca181a6210 con 0x5599fdc287e0 2024-02-02T15:46:43.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.530+0000 7fca1d600700 1 --2- 172.21.15.1:0/143197889 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5599fdc287e0 0x5599fdc88b50 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fca10012ba0 tx=0x7fca10009990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=955a8fd883cbc2d3 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:43.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.530+0000 7fca06000700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:46:43.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.530+0000 7fca06000700 1 -- 172.21.15.1:0/143197889 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1000e040 con 0x5599fdc287e0 2024-02-02T15:46:43.536 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.531+0000 7fca06000700 1 -- 172.21.15.1:0/143197889 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fca10014a20 con 0x5599fdc287e0 2024-02-02T15:46:43.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.531+0000 7fca33111b80 1 -- 172.21.15.1:0/143197889 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fca1000f040 con 0x5599fdc287e0 2024-02-02T15:46:43.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.531+0000 7fca06000700 1 -- 172.21.15.1:0/143197889 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca10009ce0 con 0x5599fdc287e0 2024-02-02T15:46:43.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.531+0000 7fca33111b80 10 client.20340 fetch_fsmap learned FSMap version 81 2024-02-02T15:46:43.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.531+0000 7fca33111b80 1 -- 172.21.15.1:0/143197889 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) v3 -- 0x5599fdb34ea0 con 0x5599fdc287e0 2024-02-02T15:46:43.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.531+0000 7fca06000700 1 -- 172.21.15.1:0/143197889 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fca1000f040 con 0x5599fdc287e0 2024-02-02T15:46:43.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.531+0000 7fca06000700 1 -- 172.21.15.1:0/143197889 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 81) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fca10016b20 con 0x5599fdc287e0 2024-02-02T15:46:43.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.531+0000 7fca33111b80 10 client.20340 fetch_fsmap finished waiting for FSMap version 81 2024-02-02T15:46:43.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.531+0000 7fca33111b80 10 client.20340 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-02-02T15:46:43.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.531+0000 7fca33111b80 20 client.20340 populate_metadata read hostname 'smithi001' 2024-02-02T15:46:43.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.531+0000 7fca33111b80 1 --2- 172.21.15.1:0/143197889 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x5599fdc8a580 0x5599fdc8c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:43.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.531+0000 7fca33111b80 4 client.20340 mds_command: new command op to 24518 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:46:43.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.531+0000 7fca33111b80 1 -- 172.21.15.1:0/143197889 --> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5599fdb2df50 con 0x5599fdc8a580 2024-02-02T15:46:43.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.532+0000 7fca1cc00700 1 --2- 172.21.15.1:0/143197889 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x5599fdc8a580 0x5599fdc8c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:43.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.532+0000 7fca1cc00700 1 --2- 172.21.15.1:0/143197889 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x5599fdc8a580 0x5599fdc8c970 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2f947566b90aa9 server_cookie=c14a562d699946c3 in_seq=0 out_seq=0 2024-02-02T15:46:43.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.532+0000 7fca06000700 10 client.20340 ms_handle_connect on v2:172.21.15.71:6836/628016199 2024-02-02T15:46:43.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.533+0000 7fca06000700 1 -- 172.21.15.1:0/143197889 <== mds.2 v2:172.21.15.71:6836/628016199 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5599fdb2df50 con 0x5599fdc8a580 2024-02-02T15:46:43.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.533+0000 7fca06000700 10 client.20340 handle_command_reply: tid=0 2024-02-02T15:46:43.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.533+0000 7fca33111b80 1 client.20340 shutdown 2024-02-02T15:46:43.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.533+0000 7fca33111b80 1 -- 172.21.15.1:0/143197889 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5599fdc287e0 msgr2=0x5599fdc88b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:43.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.533+0000 7fca33111b80 1 --2- 172.21.15.1:0/143197889 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5599fdc287e0 0x5599fdc88b50 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fca10012ba0 tx=0x7fca10009990 comp rx=0 tx=0).stop 2024-02-02T15:46:43.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.533+0000 7fca33111b80 1 -- 172.21.15.1:0/143197889 shutdown_connections 2024-02-02T15:46:43.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.533+0000 7fca33111b80 1 --2- 172.21.15.1:0/143197889 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x5599fdc287e0 0x5599fdc88b50 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.533+0000 7fca33111b80 1 --2- 172.21.15.1:0/143197889 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5599fdc1d010 0x5599fdc2b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.533+0000 7fca33111b80 1 --2- 172.21.15.1:0/143197889 >> [v2:172.21.15.71:6836/628016199,v1:172.21.15.71:6837/628016199] conn(0x5599fdc8a580 0x5599fdc8c970 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.533+0000 7fca33111b80 1 --2- 172.21.15.1:0/143197889 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x5599fdc1d930 0x5599fdc2baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.533+0000 7fca06000700 0 client.20340 ms_handle_reset on v2:172.21.15.71:6836/628016199 2024-02-02T15:46:43.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.533+0000 7fca33111b80 1 -- 172.21.15.1:0/143197889 >> 172.21.15.1:0/143197889 conn(0x5599fdb7cdb0 msgr2=0x5599fdc206c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:43.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.533+0000 7fca33111b80 1 -- 172.21.15.1:0/143197889 shutdown_connections 2024-02-02T15:46:43.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.533+0000 7fca33111b80 1 -- 172.21.15.1:0/143197889 wait complete. 2024-02-02T15:46:43.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.533+0000 7fca33111b80 20 client.20340 trim_cache size 0 max 16384 2024-02-02T15:46:43.540 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:46:43.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.534+0000 7fca1f600700 1 -- 172.21.15.1:0/1353883453 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fc9f80a0690 msgr2=0x7fc9f80a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:43.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.534+0000 7fca1f600700 1 --2- 172.21.15.1:0/1353883453 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fc9f80a0690 0x7fc9f80a2b50 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fca1001e000 tx=0x7fca10021000 comp rx=0 tx=0).stop 2024-02-02T15:46:43.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.534+0000 7fca1f600700 1 -- 172.21.15.1:0/1353883453 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca18106120 msgr2=0x7fca18073580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:43.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.534+0000 7fca1f600700 1 --2- 172.21.15.1:0/1353883453 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca18106120 0x7fca18073580 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fca0000e970 tx=0x7fca0000ec80 comp rx=0 tx=0).stop 2024-02-02T15:46:43.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.534+0000 7fca1f600700 1 -- 172.21.15.1:0/1353883453 shutdown_connections 2024-02-02T15:46:43.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.534+0000 7fca1f600700 1 --2- 172.21.15.1:0/1353883453 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fc9f80a0690 0x7fc9f80a2b50 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.534+0000 7fca1f600700 1 --2- 172.21.15.1:0/1353883453 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fca18073ac0 0x7fca181a50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.534+0000 7fca1f600700 1 --2- 172.21.15.1:0/1353883453 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca18101000 0x7fca18073040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.534+0000 7fca1f600700 1 --2- 172.21.15.1:0/1353883453 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fca18106120 0x7fca18073580 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.534+0000 7fca1f600700 1 -- 172.21.15.1:0/1353883453 >> 172.21.15.1:0/1353883453 conn(0x7fca180fc860 msgr2=0x7fca181049d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:43.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.534+0000 7fca1f600700 1 -- 172.21.15.1:0/1353883453 shutdown_connections 2024-02-02T15:46:43.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.534+0000 7fca1f600700 1 -- 172.21.15.1:0/1353883453 wait complete. 2024-02-02T15:46:43.551 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-02-02T15:46:43.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.753+0000 7f64b7600700 1 -- 172.21.15.1:0/1735954517 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f64b01019c0 msgr2=0x7f64b0105a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:43.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.753+0000 7f64b7600700 1 --2- 172.21.15.1:0/1735954517 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f64b01019c0 0x7f64b0105a00 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f64980098f0 tx=0x7f6498009c00 comp rx=0 tx=0).stop 2024-02-02T15:46:43.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.753+0000 7f64b7600700 1 -- 172.21.15.1:0/1735954517 shutdown_connections 2024-02-02T15:46:43.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.753+0000 7f64b7600700 1 --2- 172.21.15.1:0/1735954517 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64b0106130 0x7f64b010dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.753+0000 7f64b7600700 1 --2- 172.21.15.1:0/1735954517 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64b0101010 0x7f64b01013f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.753+0000 7f64b7600700 1 --2- 172.21.15.1:0/1735954517 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f64b01019c0 0x7f64b0105a00 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.753+0000 7f64b7600700 1 -- 172.21.15.1:0/1735954517 >> 172.21.15.1:0/1735954517 conn(0x7f64b00fc8b0 msgr2=0x7f64b00fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:43.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.753+0000 7f64b7600700 1 -- 172.21.15.1:0/1735954517 shutdown_connections 2024-02-02T15:46:43.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.753+0000 7f64b7600700 1 -- 172.21.15.1:0/1735954517 wait complete. 2024-02-02T15:46:43.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.754+0000 7f64b7600700 1 Processor -- start 2024-02-02T15:46:43.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.754+0000 7f64b7600700 1 -- start start 2024-02-02T15:46:43.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.755+0000 7f64b7600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64b0101010 0x7f64b0196fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:43.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.755+0000 7f64b7600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64b0106130 0x7f64b0197510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:43.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.755+0000 7f64b7600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f64b019b8d0 0x7f64b019bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:43.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.755+0000 7f64b7600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64b019c350 con 0x7f64b019b8d0 2024-02-02T15:46:43.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.755+0000 7f64b7600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64b019c4c0 con 0x7f64b0101010 2024-02-02T15:46:43.756 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.755+0000 7f64b7600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64b019c680 con 0x7f64b0106130 2024-02-02T15:46:43.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.755+0000 7f64b5600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f64b019b8d0 0x7f64b019bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:43.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.755+0000 7f64b5600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f64b019b8d0 0x7f64b019bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41590/0 (socket says 172.21.15.1:41590) 2024-02-02T15:46:43.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.755+0000 7f64b5600700 1 -- 172.21.15.1:0/2359361101 learned_addr learned my addr 172.21.15.1:0/2359361101 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:46:43.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.755+0000 7f64a7e00700 1 --2- 172.21.15.1:0/2359361101 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64b0106130 0x7f64b0197510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:43.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.755+0000 7f64b4c00700 1 --2- 172.21.15.1:0/2359361101 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64b0101010 0x7f64b0196fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:43.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.755+0000 7f64b5600700 1 -- 172.21.15.1:0/2359361101 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64b0106130 msgr2=0x7f64b0197510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:43.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.755+0000 7f64b5600700 1 --2- 172.21.15.1:0/2359361101 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64b0106130 0x7f64b0197510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.757 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.755+0000 7f64b5600700 1 -- 172.21.15.1:0/2359361101 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64b0101010 msgr2=0x7f64b0196fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:43.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.755+0000 7f64b5600700 1 --2- 172.21.15.1:0/2359361101 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64b0101010 0x7f64b0196fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.755+0000 7f64b5600700 1 -- 172.21.15.1:0/2359361101 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6498009520 con 0x7f64b019b8d0 2024-02-02T15:46:43.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.756+0000 7f64b5600700 1 --2- 172.21.15.1:0/2359361101 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f64b019b8d0 0x7f64b019bd50 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f64a800ef30 tx=0x7f64a800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:43.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.756+0000 7f64ae000700 1 -- 172.21.15.1:0/2359361101 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a8018070 con 0x7f64b019b8d0 2024-02-02T15:46:43.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.756+0000 7f64ae000700 1 -- 172.21.15.1:0/2359361101 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f64a800f040 con 0x7f64b019b8d0 2024-02-02T15:46:43.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.756+0000 7f64b7600700 1 -- 172.21.15.1:0/2359361101 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64b01a3b80 con 0x7f64b019b8d0 2024-02-02T15:46:43.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.756+0000 7f64ae000700 1 -- 172.21.15.1:0/2359361101 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a800ccf0 con 0x7f64b019b8d0 2024-02-02T15:46:43.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.756+0000 7f64b7600700 1 -- 172.21.15.1:0/2359361101 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64b01a40b0 con 0x7f64b019b8d0 2024-02-02T15:46:43.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.759+0000 7f64ae000700 1 -- 172.21.15.1:0/2359361101 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f64a8003a00 con 0x7f64b019b8d0 2024-02-02T15:46:43.759 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.761+0000 7f64cb0d8b80 1 Processor -- start 2024-02-02T15:46:43.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.761+0000 7f64cb0d8b80 1 -- start start 2024-02-02T15:46:43.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.761+0000 7f64ae000700 1 --2- 172.21.15.1:0/2359361101 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f64900a0710 0x7f64900a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:43.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.762+0000 7f64ae000700 1 -- 172.21.15.1:0/2359361101 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f64a80cbb60 con 0x7f64b019b8d0 2024-02-02T15:46:43.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.762+0000 7f64cb0d8b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e0fef1010 0x560e0fef13f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:43.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.762+0000 7f64cb0d8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e0fef1930 0x560e0ff006b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:43.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.762+0000 7f64a7e00700 1 --2- 172.21.15.1:0/2359361101 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f64900a0710 0x7f64900a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:43.762 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.762+0000 7f64cb0d8b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e0ff00bf0 0x560e0ff02fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:43.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.762+0000 7f64cb0d8b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e0fe654c0 con 0x560e0fef1010 2024-02-02T15:46:43.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.762+0000 7f64cb0d8b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e0fe49aa0 con 0x560e0ff00bf0 2024-02-02T15:46:43.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.762+0000 7f64cb0d8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e0fe47460 con 0x560e0fef1930 2024-02-02T15:46:43.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.762+0000 7f64b4c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e0fef1010 0x560e0fef13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:43.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.762+0000 7f64a7e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e0fef1930 0x560e0ff006b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:43.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.762+0000 7f64b4c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e0fef1010 0x560e0fef13f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41604/0 (socket says 172.21.15.1:41604) 2024-02-02T15:46:43.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.762+0000 7f64b4c00700 1 -- 172.21.15.1:0/2741749119 learned_addr learned my addr 172.21.15.1:0/2741749119 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:46:43.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.762+0000 7f64a7e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e0fef1930 0x560e0ff006b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:51886/0 (socket says 172.21.15.1:51886) 2024-02-02T15:46:43.763 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.762+0000 7f64b5600700 1 --2- 172.21.15.1:0/2741749119 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e0ff00bf0 0x560e0ff02fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:43.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.762+0000 7f64b4c00700 1 -- 172.21.15.1:0/2741749119 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e0fef1930 msgr2=0x560e0ff006b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:43.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.762+0000 7f64b4c00700 1 --2- 172.21.15.1:0/2741749119 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e0fef1930 0x560e0ff006b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.762+0000 7f64b4c00700 1 -- 172.21.15.1:0/2741749119 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e0ff00bf0 msgr2=0x560e0ff02fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:43.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.762+0000 7f64b4c00700 1 --2- 172.21.15.1:0/2741749119 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e0ff00bf0 0x560e0ff02fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.762+0000 7f64b4c00700 1 -- 172.21.15.1:0/2741749119 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e0fcc43b0 con 0x560e0fef1010 2024-02-02T15:46:43.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.763+0000 7f64a7e00700 1 --2- 172.21.15.1:0/2359361101 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f64900a0710 0x7f64900a2bd0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f6498009fd0 tx=0x7f649800fed0 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:43.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.763+0000 7f64b4c00700 1 --2- 172.21.15.1:0/2741749119 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e0fef1010 0x560e0fef13f0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f64a0009d60 tx=0x7f64a0011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b060ee649604d71 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:43.764 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.763+0000 7f64a5c00700 1 -- 172.21.15.1:0/2741749119 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a0019630 con 0x560e0fef1010 2024-02-02T15:46:43.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.763+0000 7f64a5c00700 1 -- 172.21.15.1:0/2741749119 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f64a0019c50 con 0x560e0fef1010 2024-02-02T15:46:43.765 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.763+0000 7f64a5c00700 1 -- 172.21.15.1:0/2741749119 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a000eba0 con 0x560e0fef1010 2024-02-02T15:46:43.768 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.764+0000 7f64cb0d8b80 1 -- 172.21.15.1:0/2741749119 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e0fef1010 msgr2=0x560e0fef13f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:43.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.764+0000 7f64cb0d8b80 1 --2- 172.21.15.1:0/2741749119 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e0fef1010 0x560e0fef13f0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f64a0009d60 tx=0x7f64a0011720 comp rx=0 tx=0).stop 2024-02-02T15:46:43.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.764+0000 7f64cb0d8b80 1 -- 172.21.15.1:0/2741749119 shutdown_connections 2024-02-02T15:46:43.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.764+0000 7f64cb0d8b80 1 --2- 172.21.15.1:0/2741749119 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e0ff00bf0 0x560e0ff02fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.764+0000 7f64cb0d8b80 1 --2- 172.21.15.1:0/2741749119 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e0fef1010 0x560e0fef13f0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.764+0000 7f64cb0d8b80 1 --2- 172.21.15.1:0/2741749119 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e0fef1930 0x560e0ff006b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.764+0000 7f64cb0d8b80 1 -- 172.21.15.1:0/2741749119 >> 172.21.15.1:0/2741749119 conn(0x560e0fe50db0 msgr2=0x560e0feed3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:43.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.764+0000 7f64cb0d8b80 1 -- 172.21.15.1:0/2741749119 shutdown_connections 2024-02-02T15:46:43.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.764+0000 7f64cb0d8b80 1 -- 172.21.15.1:0/2741749119 wait complete. 2024-02-02T15:46:43.769 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.764+0000 7f64cb0d8b80 1 -- start start 2024-02-02T15:46:43.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.765+0000 7f64cb0d8b80 1 Processor -- start 2024-02-02T15:46:43.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.765+0000 7f64cb0d8b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e0fef1010 0x560e0ff005d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:43.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.765+0000 7f64cb0d8b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e0fef1930 0x560e0fefd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:43.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.765+0000 7f64cb0d8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e0ff00bf0 0x560e0fefddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:43.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.765+0000 7f64cb0d8b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e0fe42d50 con 0x560e0fef1010 2024-02-02T15:46:43.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.765+0000 7f64cb0d8b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e0fdded80 con 0x560e0fef1930 2024-02-02T15:46:43.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.765+0000 7f64cb0d8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e0fddea20 con 0x560e0ff00bf0 2024-02-02T15:46:43.770 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.765+0000 7f64cb0d8b80 10 client.0 _ll_register_callbacks cb 0x560e0fde0290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:46:43.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.765+0000 7f64b4c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e0fef1010 0x560e0ff005d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:43.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.766+0000 7f64b4c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e0fef1010 0x560e0ff005d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41614/0 (socket says 172.21.15.1:41614) 2024-02-02T15:46:43.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.766+0000 7f64b4c00700 1 -- 172.21.15.1:0/447915892 learned_addr learned my addr 172.21.15.1:0/447915892 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:46:43.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.766+0000 7f64a7e00700 1 --2- 172.21.15.1:0/447915892 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e0fef1930 0x560e0fefd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:43.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.766+0000 7f64b5600700 1 --2- 172.21.15.1:0/447915892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e0ff00bf0 0x560e0fefddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:43.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.766+0000 7f64b4c00700 1 -- 172.21.15.1:0/447915892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e0ff00bf0 msgr2=0x560e0fefddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:43.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.766+0000 7f64b4c00700 1 --2- 172.21.15.1:0/447915892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e0ff00bf0 0x560e0fefddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.766+0000 7f64b4c00700 1 -- 172.21.15.1:0/447915892 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e0fef1930 msgr2=0x560e0fefd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:43.771 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.766+0000 7f64b4c00700 1 --2- 172.21.15.1:0/447915892 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e0fef1930 0x560e0fefd7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.766+0000 7f64b4c00700 1 -- 172.21.15.1:0/447915892 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64a00099c0 con 0x560e0fef1010 2024-02-02T15:46:43.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.766+0000 7f64b4c00700 1 --2- 172.21.15.1:0/447915892 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e0fef1010 0x560e0ff005d0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f64a0009700 tx=0x7f64a0019b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=173cd775d9d80399 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:43.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.766+0000 7f649e000700 10 client.0 ms_handle_connect on v2:172.21.15.1:3300/0 2024-02-02T15:46:43.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.766+0000 7f649e000700 1 -- 172.21.15.1:0/447915892 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a0004220 con 0x560e0fef1010 2024-02-02T15:46:43.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.766+0000 7f649e000700 1 -- 172.21.15.1:0/447915892 <== mon.0 v2:172.21.15.1:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f64a0026070 con 0x560e0fef1010 2024-02-02T15:46:43.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.766+0000 7f64cb0d8b80 1 -- 172.21.15.1:0/447915892 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560e0fee50f0 con 0x560e0fef1010 2024-02-02T15:46:43.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.767+0000 7f649e000700 1 -- 172.21.15.1:0/447915892 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7f64a0004380 con 0x560e0fef1010 2024-02-02T15:46:43.772 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.767+0000 7f64cb0d8b80 10 client.20352 fetch_fsmap learned FSMap version 81 2024-02-02T15:46:43.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.767+0000 7f64cb0d8b80 1 -- 172.21.15.1:0/447915892 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) v3 -- 0x560e0fc70b60 con 0x560e0fef1010 2024-02-02T15:46:43.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.767+0000 7f649e000700 1 -- 172.21.15.1:0/447915892 <== mon.0 v2:172.21.15.1:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x560e0fee50f0 con 0x560e0fef1010 2024-02-02T15:46:43.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.767+0000 7f649e000700 1 -- 172.21.15.1:0/447915892 <== mon.0 v2:172.21.15.1:3300/0 5 ==== fsmap(e 81) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f64a0004800 con 0x560e0fef1010 2024-02-02T15:46:43.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.767+0000 7f64cb0d8b80 10 client.20352 fetch_fsmap finished waiting for FSMap version 81 2024-02-02T15:46:43.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.767+0000 7f64cb0d8b80 10 client.20352 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-02T15:46:43.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.767+0000 7f64cb0d8b80 20 client.20352 populate_metadata read hostname 'smithi001' 2024-02-02T15:46:43.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.767+0000 7f64cb0d8b80 1 --2- 172.21.15.1:0/447915892 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x560e0ff103d0 0x560e0ff127f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:43.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.767+0000 7f64cb0d8b80 4 client.20352 mds_command: new command op to 24529 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:46:43.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.767+0000 7f64cb0d8b80 1 -- 172.21.15.1:0/447915892 --> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560e0fe3bdd0 con 0x560e0ff103d0 2024-02-02T15:46:43.773 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.768+0000 7f64a7e00700 1 --2- 172.21.15.1:0/447915892 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x560e0ff103d0 0x560e0ff127f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:43.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.768+0000 7f64a7e00700 1 --2- 172.21.15.1:0/447915892 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x560e0ff103d0 0x560e0ff127f0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f54317403a4e87dc server_cookie=a42a0120f2bc761 in_seq=0 out_seq=0 2024-02-02T15:46:43.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.768+0000 7f649e000700 10 client.20352 ms_handle_connect on v2:172.21.15.142:6838/321675421 2024-02-02T15:46:43.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.769+0000 7f649e000700 1 -- 172.21.15.1:0/447915892 <== mds.1 v2:172.21.15.142:6838/321675421 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x560e0fe3bdd0 con 0x560e0ff103d0 2024-02-02T15:46:43.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.769+0000 7f649e000700 10 client.20352 handle_command_reply: tid=0 2024-02-02T15:46:43.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.769+0000 7f64cb0d8b80 1 client.20352 shutdown 2024-02-02T15:46:43.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.769+0000 7f64cb0d8b80 1 -- 172.21.15.1:0/447915892 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e0fef1010 msgr2=0x560e0ff005d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:43.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.769+0000 7f64cb0d8b80 1 --2- 172.21.15.1:0/447915892 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e0fef1010 0x560e0ff005d0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f64a0009700 tx=0x7f64a0019b10 comp rx=0 tx=0).stop 2024-02-02T15:46:43.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.770+0000 7f64cb0d8b80 1 -- 172.21.15.1:0/447915892 shutdown_connections 2024-02-02T15:46:43.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.770+0000 7f64cb0d8b80 1 --2- 172.21.15.1:0/447915892 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x560e0ff103d0 0x560e0ff127f0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.770+0000 7f64cb0d8b80 1 --2- 172.21.15.1:0/447915892 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e0ff00bf0 0x560e0fefddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.770+0000 7f64cb0d8b80 1 --2- 172.21.15.1:0/447915892 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e0fef1010 0x560e0ff005d0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.770+0000 7f64cb0d8b80 1 --2- 172.21.15.1:0/447915892 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e0fef1930 0x560e0fefd7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.770+0000 7f649e000700 0 client.20352 ms_handle_reset on v2:172.21.15.142:6838/321675421 2024-02-02T15:46:43.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.770+0000 7f64cb0d8b80 1 -- 172.21.15.1:0/447915892 >> 172.21.15.1:0/447915892 conn(0x560e0fe50db0 msgr2=0x560e0feecfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:43.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.770+0000 7f64cb0d8b80 1 -- 172.21.15.1:0/447915892 shutdown_connections 2024-02-02T15:46:43.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.770+0000 7f64cb0d8b80 1 -- 172.21.15.1:0/447915892 wait complete. 2024-02-02T15:46:43.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.770+0000 7f64cb0d8b80 20 client.20352 trim_cache size 0 max 16384 2024-02-02T15:46:43.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.783+0000 7f64cb0d8b80 1 Processor -- start 2024-02-02T15:46:43.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.783+0000 7f64cb0d8b80 1 -- start start 2024-02-02T15:46:43.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.783+0000 7f64cb0d8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e0fef1010 0x560e0fef3840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:43.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.783+0000 7f64cb0d8b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e0fef1930 0x560e0ff00100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:43.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.783+0000 7f64cb0d8b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e0ff00bf0 0x560e0fefd080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:43.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.783+0000 7f64cb0d8b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e0fcc66a0 con 0x560e0ff00bf0 2024-02-02T15:46:43.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.783+0000 7f64cb0d8b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e0fef1d10 con 0x560e0fef1930 2024-02-02T15:46:43.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.783+0000 7f64cb0d8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e0fefbd40 con 0x560e0fef1010 2024-02-02T15:46:43.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.784+0000 7f64b5600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e0ff00bf0 0x560e0fefd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:43.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.784+0000 7f64a7e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e0fef1930 0x560e0ff00100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:43.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.784+0000 7f64b5600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e0ff00bf0 0x560e0fefd080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41620/0 (socket says 172.21.15.1:41620) 2024-02-02T15:46:43.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.784+0000 7f64b5600700 1 -- 172.21.15.1:0/767304010 learned_addr learned my addr 172.21.15.1:0/767304010 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:46:43.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.784+0000 7f64b4c00700 1 --2- 172.21.15.1:0/767304010 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e0fef1010 0x560e0fef3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:43.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.784+0000 7f64b5600700 1 -- 172.21.15.1:0/767304010 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e0fef1010 msgr2=0x560e0fef3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:43.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.784+0000 7f64b5600700 1 --2- 172.21.15.1:0/767304010 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e0fef1010 0x560e0fef3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.784+0000 7f64b5600700 1 -- 172.21.15.1:0/767304010 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e0fef1930 msgr2=0x560e0ff00100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:43.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.784+0000 7f64b5600700 1 --2- 172.21.15.1:0/767304010 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e0fef1930 0x560e0ff00100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.784+0000 7f64b5600700 1 -- 172.21.15.1:0/767304010 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e0fe08ea0 con 0x560e0ff00bf0 2024-02-02T15:46:43.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.784+0000 7f64b4c00700 1 --2- 172.21.15.1:0/767304010 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e0fef1010 0x560e0fef3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:46:43.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.784+0000 7f64a7e00700 1 --2- 172.21.15.1:0/767304010 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e0fef1930 0x560e0ff00100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:46:43.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.784+0000 7f64b5600700 1 --2- 172.21.15.1:0/767304010 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e0ff00bf0 0x560e0fefd080 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f64a8095210 tx=0x7f64a8009370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee82381f145fdcbf server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:43.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.785+0000 7f64a6600700 1 -- 172.21.15.1:0/767304010 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a8098670 con 0x560e0ff00bf0 2024-02-02T15:46:43.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.785+0000 7f64a6600700 1 -- 172.21.15.1:0/767304010 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f64a80b1210 con 0x560e0ff00bf0 2024-02-02T15:46:43.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.785+0000 7f64a6600700 1 -- 172.21.15.1:0/767304010 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a80b80b0 con 0x560e0ff00bf0 2024-02-02T15:46:43.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.785+0000 7f64cb0d8b80 1 -- 172.21.15.1:0/767304010 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e0ff00bf0 msgr2=0x560e0fefd080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:43.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.785+0000 7f64cb0d8b80 1 --2- 172.21.15.1:0/767304010 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e0ff00bf0 0x560e0fefd080 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f64a8095210 tx=0x7f64a8009370 comp rx=0 tx=0).stop 2024-02-02T15:46:43.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.785+0000 7f64cb0d8b80 1 -- 172.21.15.1:0/767304010 shutdown_connections 2024-02-02T15:46:43.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.785+0000 7f64cb0d8b80 1 --2- 172.21.15.1:0/767304010 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e0ff00bf0 0x560e0fefd080 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.790 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.785+0000 7f64cb0d8b80 1 --2- 172.21.15.1:0/767304010 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e0fef1010 0x560e0fef3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.785+0000 7f64cb0d8b80 1 --2- 172.21.15.1:0/767304010 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e0fef1930 0x560e0ff00100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.785+0000 7f64cb0d8b80 1 -- 172.21.15.1:0/767304010 >> 172.21.15.1:0/767304010 conn(0x560e0fe50db0 msgr2=0x560e0feebb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:43.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.785+0000 7f64cb0d8b80 1 -- 172.21.15.1:0/767304010 shutdown_connections 2024-02-02T15:46:43.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.785+0000 7f64cb0d8b80 1 -- 172.21.15.1:0/767304010 wait complete. 2024-02-02T15:46:43.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.786+0000 7f64cb0d8b80 1 -- start start 2024-02-02T15:46:43.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.786+0000 7f64cb0d8b80 1 Processor -- start 2024-02-02T15:46:43.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.786+0000 7f64cb0d8b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e0fef1010 0x560e0ff07030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:43.791 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.786+0000 7f64cb0d8b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e0fef1930 0x560e0ff07570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:43.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.786+0000 7f64cb0d8b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e0ff00bf0 0x560e0ff03710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:43.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.786+0000 7f64cb0d8b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e0ff03e50 con 0x560e0fef1930 2024-02-02T15:46:43.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.786+0000 7f64cb0d8b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e0ff03fc0 con 0x560e0ff00bf0 2024-02-02T15:46:43.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.786+0000 7f64cb0d8b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e0ff04180 con 0x560e0fef1010 2024-02-02T15:46:43.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.787+0000 7f64cb0d8b80 10 client.0 _ll_register_callbacks cb 0x560e0fde0290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:46:43.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.787+0000 7f64a7e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e0fef1930 0x560e0ff07570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:43.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.787+0000 7f64b4c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e0fef1010 0x560e0ff07030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:43.792 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.787+0000 7f64a7e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e0fef1930 0x560e0ff07570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41632/0 (socket says 172.21.15.1:41632) 2024-02-02T15:46:43.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.787+0000 7f64b5600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e0ff00bf0 0x560e0ff03710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:43.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.787+0000 7f64a7e00700 1 -- 172.21.15.1:0/4170382496 learned_addr learned my addr 172.21.15.1:0/4170382496 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:46:43.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.787+0000 7f64b4c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e0fef1010 0x560e0ff07030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:51896/0 (socket says 172.21.15.1:51896) 2024-02-02T15:46:43.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.787+0000 7f64b5600700 1 -- 172.21.15.1:0/4170382496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e0fef1010 msgr2=0x560e0ff07030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:43.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.787+0000 7f64b5600700 1 --2- 172.21.15.1:0/4170382496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e0fef1010 0x560e0ff07030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.787+0000 7f64b5600700 1 -- 172.21.15.1:0/4170382496 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e0fef1930 msgr2=0x560e0ff07570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:43.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.787+0000 7f64b5600700 1 --2- 172.21.15.1:0/4170382496 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e0fef1930 0x560e0ff07570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.793 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.787+0000 7f64b5600700 1 -- 172.21.15.1:0/4170382496 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f649801f040 con 0x560e0ff00bf0 2024-02-02T15:46:43.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.787+0000 7f64a7e00700 1 --2- 172.21.15.1:0/4170382496 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e0fef1930 0x560e0ff07570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:46:43.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.787+0000 7f64b5600700 1 --2- 172.21.15.1:0/4170382496 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e0ff00bf0 0x560e0ff03710 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f64a8095210 tx=0x7f64a8098670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc9f338bfd685923 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:43.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.787+0000 7f649e000700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:46:43.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.788+0000 7f649e000700 1 -- 172.21.15.1:0/4170382496 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a8098d10 con 0x560e0ff00bf0 2024-02-02T15:46:43.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.788+0000 7f64cb0d8b80 1 -- 172.21.15.1:0/4170382496 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f64a0019c50 con 0x560e0ff00bf0 2024-02-02T15:46:43.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.788+0000 7f649e000700 1 -- 172.21.15.1:0/4170382496 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f64a8099410 con 0x560e0ff00bf0 2024-02-02T15:46:43.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.788+0000 7f649e000700 1 -- 172.21.15.1:0/4170382496 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7f64a80a0cf0 con 0x560e0ff00bf0 2024-02-02T15:46:43.794 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.788+0000 7f64cb0d8b80 10 client.30662 fetch_fsmap learned FSMap version 81 2024-02-02T15:46:43.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.788+0000 7f64cb0d8b80 1 -- 172.21.15.1:0/4170382496 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) v3 -- 0x7f64b01a40b0 con 0x560e0ff00bf0 2024-02-02T15:46:43.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.788+0000 7f649e000700 1 -- 172.21.15.1:0/4170382496 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f64a0019c50 con 0x560e0ff00bf0 2024-02-02T15:46:43.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.788+0000 7f649e000700 1 -- 172.21.15.1:0/4170382496 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 81) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f64a8099d20 con 0x560e0ff00bf0 2024-02-02T15:46:43.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.788+0000 7f64cb0d8b80 10 client.30662 fetch_fsmap finished waiting for FSMap version 81 2024-02-02T15:46:43.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.788+0000 7f64cb0d8b80 10 client.30662 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-02-02T15:46:43.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.788+0000 7f64cb0d8b80 20 client.30662 populate_metadata read hostname 'smithi001' 2024-02-02T15:46:43.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.788+0000 7f64cb0d8b80 1 --2- 172.21.15.1:0/4170382496 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x560e0ff62bc0 0x560e0ff64fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:43.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.788+0000 7f64cb0d8b80 4 client.30662 mds_command: new command op to 24529 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:46:43.795 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.788+0000 7f64cb0d8b80 1 -- 172.21.15.1:0/4170382496 --> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560e0fe01f50 con 0x560e0ff62bc0 2024-02-02T15:46:43.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.789+0000 7f64b4c00700 1 --2- 172.21.15.1:0/4170382496 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x560e0ff62bc0 0x560e0ff64fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:43.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.789+0000 7f64b4c00700 1 --2- 172.21.15.1:0/4170382496 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x560e0ff62bc0 0x560e0ff64fb0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7ee0cb44350099b1 server_cookie=e944641e6d044dba in_seq=0 out_seq=0 2024-02-02T15:46:43.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.789+0000 7f649e000700 10 client.30662 ms_handle_connect on v2:172.21.15.142:6838/321675421 2024-02-02T15:46:43.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.789+0000 7f649e000700 1 -- 172.21.15.1:0/4170382496 <== mds.1 v2:172.21.15.142:6838/321675421 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560e0fe01f50 con 0x560e0ff62bc0 2024-02-02T15:46:43.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.789+0000 7f649e000700 10 client.30662 handle_command_reply: tid=0 2024-02-02T15:46:43.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.790+0000 7f64cb0d8b80 1 client.30662 shutdown 2024-02-02T15:46:43.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.790+0000 7f64cb0d8b80 1 -- 172.21.15.1:0/4170382496 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e0ff00bf0 msgr2=0x560e0ff03710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:43.796 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.790+0000 7f64cb0d8b80 1 --2- 172.21.15.1:0/4170382496 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e0ff00bf0 0x560e0ff03710 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f64a8095210 tx=0x7f64a8098670 comp rx=0 tx=0).stop 2024-02-02T15:46:43.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.790+0000 7f64cb0d8b80 1 -- 172.21.15.1:0/4170382496 shutdown_connections 2024-02-02T15:46:43.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.790+0000 7f64cb0d8b80 1 --2- 172.21.15.1:0/4170382496 >> [v2:172.21.15.142:6838/321675421,v1:172.21.15.142:6839/321675421] conn(0x560e0ff62bc0 0x560e0ff64fb0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.790+0000 7f64cb0d8b80 1 --2- 172.21.15.1:0/4170382496 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560e0ff00bf0 0x560e0ff03710 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.790+0000 7f64cb0d8b80 1 --2- 172.21.15.1:0/4170382496 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560e0fef1010 0x560e0ff07030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.790+0000 7f64cb0d8b80 1 --2- 172.21.15.1:0/4170382496 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560e0fef1930 0x560e0ff07570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.790+0000 7f649e000700 0 client.30662 ms_handle_reset on v2:172.21.15.142:6838/321675421 2024-02-02T15:46:43.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.790+0000 7f64cb0d8b80 1 -- 172.21.15.1:0/4170382496 >> 172.21.15.1:0/4170382496 conn(0x560e0fe50db0 msgr2=0x560e0fefd050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:43.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.790+0000 7f64cb0d8b80 1 -- 172.21.15.1:0/4170382496 shutdown_connections 2024-02-02T15:46:43.797 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.790+0000 7f64cb0d8b80 1 -- 172.21.15.1:0/4170382496 wait complete. 2024-02-02T15:46:43.798 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.790+0000 7f64cb0d8b80 20 client.30662 trim_cache size 0 max 16384 2024-02-02T15:46:43.798 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:46:43.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.791+0000 7f64b7600700 1 -- 172.21.15.1:0/2359361101 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f64900a0710 msgr2=0x7f64900a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:43.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.791+0000 7f64b7600700 1 --2- 172.21.15.1:0/2359361101 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f64900a0710 0x7f64900a2bd0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f6498009fd0 tx=0x7f649800fed0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.791+0000 7f64b7600700 1 -- 172.21.15.1:0/2359361101 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f64b019b8d0 msgr2=0x7f64b019bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:43.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.791+0000 7f64b7600700 1 --2- 172.21.15.1:0/2359361101 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f64b019b8d0 0x7f64b019bd50 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f64a800ef30 tx=0x7f64a800c3a0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.791+0000 7f64b7600700 1 -- 172.21.15.1:0/2359361101 shutdown_connections 2024-02-02T15:46:43.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.791+0000 7f64b7600700 1 --2- 172.21.15.1:0/2359361101 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f64900a0710 0x7f64900a2bd0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.791+0000 7f64b7600700 1 --2- 172.21.15.1:0/2359361101 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f64b019b8d0 0x7f64b019bd50 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.799 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.791+0000 7f64b7600700 1 --2- 172.21.15.1:0/2359361101 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f64b0101010 0x7f64b0196fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.791+0000 7f64b7600700 1 --2- 172.21.15.1:0/2359361101 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f64b0106130 0x7f64b0197510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:43.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.791+0000 7f64b7600700 1 -- 172.21.15.1:0/2359361101 >> 172.21.15.1:0/2359361101 conn(0x7f64b00fc8b0 msgr2=0x7f64b00fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:43.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.791+0000 7f64b7600700 1 -- 172.21.15.1:0/2359361101 shutdown_connections 2024-02-02T15:46:43.800 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:43.791+0000 7f64b7600700 1 -- 172.21.15.1:0/2359361101 wait complete. 2024-02-02T15:46:43.807 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:46:43 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/4271448383' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:46:43.808 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-02-02T15:46:44.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.012+0000 7febfea00700 1 -- 172.21.15.1:0/1326369901 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7febf8103c40 msgr2=0x7febf8107c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:44.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.012+0000 7febfea00700 1 --2- 172.21.15.1:0/1326369901 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7febf8103c40 0x7febf8107c80 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7febe0009990 tx=0x7febe0009ca0 comp rx=0 tx=0).stop 2024-02-02T15:46:44.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.012+0000 7febfea00700 1 -- 172.21.15.1:0/1326369901 shutdown_connections 2024-02-02T15:46:44.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.012+0000 7febfea00700 1 --2- 172.21.15.1:0/1326369901 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7febf81083b0 0x7febf810ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:44.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.012+0000 7febfea00700 1 --2- 172.21.15.1:0/1326369901 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febf8103290 0x7febf8103670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:44.012 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.012+0000 7febfea00700 1 --2- 172.21.15.1:0/1326369901 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7febf8103c40 0x7febf8107c80 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:44.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.012+0000 7febfea00700 1 -- 172.21.15.1:0/1326369901 >> 172.21.15.1:0/1326369901 conn(0x7febf80feb30 msgr2=0x7febf8100f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:44.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.012+0000 7febfea00700 1 -- 172.21.15.1:0/1326369901 shutdown_connections 2024-02-02T15:46:44.013 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.012+0000 7febfea00700 1 -- 172.21.15.1:0/1326369901 wait complete. 2024-02-02T15:46:44.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.013+0000 7febfea00700 1 Processor -- start 2024-02-02T15:46:44.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.014+0000 7febfea00700 1 -- start start 2024-02-02T15:46:44.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.014+0000 7febfea00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7febf8103290 0x7febf8199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:44.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.014+0000 7febfea00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febf81083b0 0x7febf81996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:44.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.014+0000 7febfea00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7febf819db00 0x7febf819df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:44.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.014+0000 7febfea00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf819e580 con 0x7febf8103290 2024-02-02T15:46:44.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.014+0000 7febfea00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf819e6f0 con 0x7febf81083b0 2024-02-02T15:46:44.015 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.014+0000 7febfea00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf819e8b0 con 0x7febf819db00 2024-02-02T15:46:44.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.014+0000 7febf7e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7febf8103290 0x7febf8199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:44.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.014+0000 7febf7e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7febf8103290 0x7febf8199160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41652/0 (socket says 172.21.15.1:41652) 2024-02-02T15:46:44.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.014+0000 7febf7e00700 1 -- 172.21.15.1:0/879727734 learned_addr learned my addr 172.21.15.1:0/879727734 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:46:44.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.014+0000 7febfca00700 1 --2- 172.21.15.1:0/879727734 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7febf819db00 0x7febf819df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:44.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.014+0000 7febf7400700 1 --2- 172.21.15.1:0/879727734 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febf81083b0 0x7febf81996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:44.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.015+0000 7febf7e00700 1 -- 172.21.15.1:0/879727734 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7febf819db00 msgr2=0x7febf819df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:44.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.015+0000 7febf7e00700 1 --2- 172.21.15.1:0/879727734 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7febf819db00 0x7febf819df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:44.016 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.015+0000 7febf7e00700 1 -- 172.21.15.1:0/879727734 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febf81083b0 msgr2=0x7febf81996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:44.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.015+0000 7febf7e00700 1 --2- 172.21.15.1:0/879727734 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febf81083b0 0x7febf81996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:44.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.015+0000 7febf7e00700 1 -- 172.21.15.1:0/879727734 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febe00095f0 con 0x7febf8103290 2024-02-02T15:46:44.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.015+0000 7febf7400700 1 --2- 172.21.15.1:0/879727734 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febf81083b0 0x7febf81996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:46:44.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.015+0000 7febf7e00700 1 --2- 172.21.15.1:0/879727734 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7febf8103290 0x7febf8199160 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7febe800e9a0 tx=0x7febe800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:44.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.015+0000 7febf4c00700 1 -- 172.21.15.1:0/879727734 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe8009790 con 0x7febf8103290 2024-02-02T15:46:44.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.015+0000 7febf4c00700 1 -- 172.21.15.1:0/879727734 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7febe8010430 con 0x7febf8103290 2024-02-02T15:46:44.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.015+0000 7febfea00700 1 -- 172.21.15.1:0/879727734 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febf81a5e10 con 0x7febf8103290 2024-02-02T15:46:44.017 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.015+0000 7febf4c00700 1 -- 172.21.15.1:0/879727734 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe8018680 con 0x7febf8103290 2024-02-02T15:46:44.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.015+0000 7febfea00700 1 -- 172.21.15.1:0/879727734 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febf81a6290 con 0x7febf8103290 2024-02-02T15:46:44.018 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.018+0000 7febf4c00700 1 -- 172.21.15.1:0/879727734 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7febe80105a0 con 0x7febf8103290 2024-02-02T15:46:44.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.020+0000 7febf4c00700 1 --2- 172.21.15.1:0/879727734 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7febd80a0690 0x7febd80a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:44.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.020+0000 7fec125e3b80 1 Processor -- start 2024-02-02T15:46:44.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.020+0000 7febf4c00700 1 -- 172.21.15.1:0/879727734 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 9453+0+0 (secure 0 0 0) 0x7febe8014070 con 0x7febf8103290 2024-02-02T15:46:44.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.020+0000 7febf7400700 1 --2- 172.21.15.1:0/879727734 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7febd80a0690 0x7febd80a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:44.019 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.020+0000 7fec125e3b80 1 -- start start 2024-02-02T15:46:44.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.020+0000 7fec125e3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560fde528010 0x560fde5283f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:44.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.020+0000 7fec125e3b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560fde528930 0x560fde5332a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:44.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.020+0000 7fec125e3b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560fde5337e0 0x560fde535bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:44.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.020+0000 7fec125e3b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fde49c4c0 con 0x560fde528930 2024-02-02T15:46:44.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.020+0000 7fec125e3b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fde480aa0 con 0x560fde5337e0 2024-02-02T15:46:44.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.020+0000 7fec125e3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fde47e460 con 0x560fde528010 2024-02-02T15:46:44.023 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.021+0000 7febf7400700 1 --2- 172.21.15.1:0/879727734 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7febd80a0690 0x7febd80a2b50 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7febf819a770 tx=0x7febe0005fd0 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:44.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.021+0000 7febf7400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560fde528930 0x560fde5332a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:44.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.021+0000 7febfca00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560fde5337e0 0x560fde535bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:44.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.021+0000 7febfca00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560fde5337e0 0x560fde535bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:33256/0 (socket says 172.21.15.1:33256) 2024-02-02T15:46:44.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.021+0000 7febfca00700 1 -- 172.21.15.1:0/3862293377 learned_addr learned my addr 172.21.15.1:0/3862293377 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:46:44.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.021+0000 7febf7e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560fde528010 0x560fde5283f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:44.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.021+0000 7febf7400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560fde528930 0x560fde5332a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41662/0 (socket says 172.21.15.1:41662) 2024-02-02T15:46:44.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.021+0000 7febfca00700 1 -- 172.21.15.1:0/3862293377 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560fde528010 msgr2=0x560fde5283f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:44.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.021+0000 7febfca00700 1 --2- 172.21.15.1:0/3862293377 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560fde528010 0x560fde5283f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:44.024 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.021+0000 7febfca00700 1 -- 172.21.15.1:0/3862293377 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560fde528930 msgr2=0x560fde5332a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:44.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.021+0000 7febfca00700 1 --2- 172.21.15.1:0/3862293377 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560fde528930 0x560fde5332a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:44.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.021+0000 7febfca00700 1 -- 172.21.15.1:0/3862293377 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560fde2fb3b0 con 0x560fde5337e0 2024-02-02T15:46:44.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.021+0000 7febfca00700 1 --2- 172.21.15.1:0/3862293377 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560fde5337e0 0x560fde535bd0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7febf0003f60 tx=0x7febf0003f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76f811fb54eaa77a server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:44.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.022+0000 7febed200700 1 -- 172.21.15.1:0/3862293377 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febf0003a00 con 0x560fde5337e0 2024-02-02T15:46:44.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.022+0000 7febed200700 1 -- 172.21.15.1:0/3862293377 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7febf000f040 con 0x560fde5337e0 2024-02-02T15:46:44.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.022+0000 7febed200700 1 -- 172.21.15.1:0/3862293377 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febf00156e0 con 0x560fde5337e0 2024-02-02T15:46:44.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.022+0000 7fec125e3b80 1 -- 172.21.15.1:0/3862293377 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560fde5337e0 msgr2=0x560fde535bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:44.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.022+0000 7fec125e3b80 1 --2- 172.21.15.1:0/3862293377 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560fde5337e0 0x560fde535bd0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7febf0003f60 tx=0x7febf0003f90 comp rx=0 tx=0).stop 2024-02-02T15:46:44.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.022+0000 7fec125e3b80 1 -- 172.21.15.1:0/3862293377 shutdown_connections 2024-02-02T15:46:44.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.022+0000 7fec125e3b80 1 --2- 172.21.15.1:0/3862293377 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560fde5337e0 0x560fde535bd0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:44.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.022+0000 7fec125e3b80 1 --2- 172.21.15.1:0/3862293377 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560fde528010 0x560fde5283f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:44.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.022+0000 7fec125e3b80 1 --2- 172.21.15.1:0/3862293377 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560fde528930 0x560fde5332a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:44.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.022+0000 7fec125e3b80 1 -- 172.21.15.1:0/3862293377 >> 172.21.15.1:0/3862293377 conn(0x560fde487db0 msgr2=0x560fde5243b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:44.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.023+0000 7fec125e3b80 1 -- 172.21.15.1:0/3862293377 shutdown_connections 2024-02-02T15:46:44.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.023+0000 7fec125e3b80 1 -- 172.21.15.1:0/3862293377 wait complete. 2024-02-02T15:46:44.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.023+0000 7fec125e3b80 1 -- start start 2024-02-02T15:46:44.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.023+0000 7fec125e3b80 1 Processor -- start 2024-02-02T15:46:44.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.024+0000 7fec125e3b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560fde528010 0x560fde5331b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:44.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.024+0000 7fec125e3b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560fde528930 0x560fde5303c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:44.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.024+0000 7fec125e3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560fde5337e0 0x560fde5309d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:44.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.024+0000 7fec125e3b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fde479d50 con 0x560fde528930 2024-02-02T15:46:44.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.024+0000 7fec125e3b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fde415d80 con 0x560fde528010 2024-02-02T15:46:44.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.024+0000 7fec125e3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fde415a20 con 0x560fde5337e0 2024-02-02T15:46:44.031 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.024+0000 7fec125e3b80 10 client.0 _ll_register_callbacks cb 0x560fde417290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:46:44.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.024+0000 7febf7e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560fde528010 0x560fde5331b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:44.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.024+0000 7febf7400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560fde528930 0x560fde5303c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:44.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.024+0000 7febfca00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560fde5337e0 0x560fde5309d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:44.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.024+0000 7febf7e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560fde528010 0x560fde5331b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:33258/0 (socket says 172.21.15.1:33258) 2024-02-02T15:46:44.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.024+0000 7febf7e00700 1 -- 172.21.15.1:0/489990039 learned_addr learned my addr 172.21.15.1:0/489990039 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:46:44.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.025+0000 7febf7e00700 1 -- 172.21.15.1:0/489990039 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560fde5337e0 msgr2=0x560fde5309d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:44.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.025+0000 7febf7e00700 1 --2- 172.21.15.1:0/489990039 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560fde5337e0 0x560fde5309d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:44.032 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.025+0000 7febf7e00700 1 -- 172.21.15.1:0/489990039 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560fde528930 msgr2=0x560fde5303c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:44.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.025+0000 7febf7e00700 1 --2- 172.21.15.1:0/489990039 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560fde528930 0x560fde5303c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:44.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.025+0000 7febf7e00700 1 -- 172.21.15.1:0/489990039 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febf0003b60 con 0x560fde528010 2024-02-02T15:46:44.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.025+0000 7febf7400700 1 --2- 172.21.15.1:0/489990039 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560fde528930 0x560fde5303c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:46:44.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.025+0000 7febf7e00700 1 --2- 172.21.15.1:0/489990039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560fde528010 0x560fde5331b0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7febe800e970 tx=0x7febe8099070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6123a6c352dd4f0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:44.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.025+0000 7febe5600700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:46:44.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.025+0000 7febe5600700 1 -- 172.21.15.1:0/489990039 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe8098af0 con 0x560fde528010 2024-02-02T15:46:44.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.025+0000 7febe5600700 1 -- 172.21.15.1:0/489990039 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 9453+0+0 (secure 0 0 0) 0x7febe8099400 con 0x560fde528010 2024-02-02T15:46:44.033 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.025+0000 7fec125e3b80 1 -- 172.21.15.1:0/489990039 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560fde51c0f0 con 0x560fde528010 2024-02-02T15:46:44.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.025+0000 7febe5600700 1 -- 172.21.15.1:0/489990039 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7febe809d230 con 0x560fde528010 2024-02-02T15:46:44.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.025+0000 7fec125e3b80 10 client.30674 fetch_fsmap learned FSMap version 81 2024-02-02T15:46:44.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.025+0000 7fec125e3b80 1 -- 172.21.15.1:0/489990039 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) v3 -- 0x560fde2a7b60 con 0x560fde528010 2024-02-02T15:46:44.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.026+0000 7febe5600700 1 -- 172.21.15.1:0/489990039 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7febe8063020 con 0x560fde528010 2024-02-02T15:46:44.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.026+0000 7febe5600700 1 -- 172.21.15.1:0/489990039 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 81) v1 ==== 7482+0+0 (secure 0 0 0) 0x7febe80b7120 con 0x560fde528010 2024-02-02T15:46:44.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.026+0000 7fec125e3b80 10 client.30674 fetch_fsmap finished waiting for FSMap version 81 2024-02-02T15:46:44.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.026+0000 7fec125e3b80 10 client.30674 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:46:44.034 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.026+0000 7fec125e3b80 20 client.30674 populate_metadata read hostname 'smithi001' 2024-02-02T15:46:44.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.026+0000 7fec125e3b80 1 --2- 172.21.15.1:0/489990039 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x560fde5431e0 0x560fde5455d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:44.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.026+0000 7fec125e3b80 4 client.30674 mds_command: new command op to 24557 tid=0[{"prefix": "get_command_descriptions"}] 2024-02-02T15:46:44.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.026+0000 7fec125e3b80 1 -- 172.21.15.1:0/489990039 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560fde472dd0 con 0x560fde5431e0 2024-02-02T15:46:44.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.026+0000 7febf7400700 1 --2- 172.21.15.1:0/489990039 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x560fde5431e0 0x560fde5455d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:44.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.027+0000 7febf7400700 1 --2- 172.21.15.1:0/489990039 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x560fde5431e0 0x560fde5455d0 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87f67fc0f0a307cc server_cookie=287bea469cb478b4 in_seq=0 out_seq=0 2024-02-02T15:46:44.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.027+0000 7febe5600700 10 client.30674 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:46:44.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.028+0000 7febe5600700 1 -- 172.21.15.1:0/489990039 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x560fde472dd0 con 0x560fde5431e0 2024-02-02T15:46:44.035 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.028+0000 7febe5600700 10 client.30674 handle_command_reply: tid=0 2024-02-02T15:46:44.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.028+0000 7fec125e3b80 1 client.30674 shutdown 2024-02-02T15:46:44.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.028+0000 7fec125e3b80 1 -- 172.21.15.1:0/489990039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560fde528010 msgr2=0x560fde5331b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:44.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.028+0000 7fec125e3b80 1 --2- 172.21.15.1:0/489990039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560fde528010 0x560fde5331b0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7febe800e970 tx=0x7febe8099070 comp rx=0 tx=0).stop 2024-02-02T15:46:44.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.028+0000 7fec125e3b80 1 -- 172.21.15.1:0/489990039 shutdown_connections 2024-02-02T15:46:44.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.028+0000 7fec125e3b80 1 --2- 172.21.15.1:0/489990039 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x560fde5431e0 0x560fde5455d0 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:44.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.028+0000 7fec125e3b80 1 --2- 172.21.15.1:0/489990039 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560fde5337e0 0x560fde5309d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:44.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.028+0000 7fec125e3b80 1 --2- 172.21.15.1:0/489990039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560fde528010 0x560fde5331b0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:44.036 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.028+0000 7fec125e3b80 1 --2- 172.21.15.1:0/489990039 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560fde528930 0x560fde5303c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:44.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.028+0000 7febe5600700 0 client.30674 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:46:44.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.028+0000 7fec125e3b80 1 -- 172.21.15.1:0/489990039 >> 172.21.15.1:0/489990039 conn(0x560fde487db0 msgr2=0x560fde523fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:44.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.028+0000 7fec125e3b80 1 -- 172.21.15.1:0/489990039 shutdown_connections 2024-02-02T15:46:44.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.028+0000 7fec125e3b80 1 -- 172.21.15.1:0/489990039 wait complete. 2024-02-02T15:46:44.037 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.028+0000 7fec125e3b80 20 client.30674 trim_cache size 0 max 16384 2024-02-02T15:46:44.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.042+0000 7fec125e3b80 1 Processor -- start 2024-02-02T15:46:44.040 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.042+0000 7fec125e3b80 1 -- start start 2024-02-02T15:46:44.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.042+0000 7fec125e3b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560fde528010 0x560fde53a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:44.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.042+0000 7fec125e3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560fde528930 0x560fde522b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:44.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.042+0000 7fec125e3b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560fde5337e0 0x560fde532ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:44.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.042+0000 7fec125e3b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fde2fd6a0 con 0x560fde528010 2024-02-02T15:46:44.042 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.042+0000 7fec125e3b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fde542fc0 con 0x560fde5337e0 2024-02-02T15:46:44.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.042+0000 7fec125e3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fde53f190 con 0x560fde528930 2024-02-02T15:46:44.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.042+0000 7febf7e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560fde528010 0x560fde53a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:44.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.042+0000 7febf7e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560fde528010 0x560fde53a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41688/0 (socket says 172.21.15.1:41688) 2024-02-02T15:46:44.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.042+0000 7febf7e00700 1 -- 172.21.15.1:0/3821965246 learned_addr learned my addr 172.21.15.1:0/3821965246 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:46:44.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.043+0000 7febf7400700 1 --2- 172.21.15.1:0/3821965246 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560fde528930 0x560fde522b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:44.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.043+0000 7febfca00700 1 --2- 172.21.15.1:0/3821965246 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560fde5337e0 0x560fde532ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:44.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.043+0000 7febf7e00700 1 -- 172.21.15.1:0/3821965246 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560fde528930 msgr2=0x560fde522b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:44.043 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.043+0000 7febf7e00700 1 --2- 172.21.15.1:0/3821965246 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560fde528930 0x560fde522b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:44.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.043+0000 7febf7e00700 1 -- 172.21.15.1:0/3821965246 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560fde5337e0 msgr2=0x560fde532ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:44.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.043+0000 7febf7e00700 1 --2- 172.21.15.1:0/3821965246 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560fde5337e0 0x560fde532ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:44.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.043+0000 7febf7e00700 1 -- 172.21.15.1:0/3821965246 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560fde43fea0 con 0x560fde528010 2024-02-02T15:46:44.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.043+0000 7febf7e00700 1 --2- 172.21.15.1:0/3821965246 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560fde528010 0x560fde53a320 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7febe80b7510 tx=0x7febe8095870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43fec6b287541371 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:44.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.043+0000 7febedc00700 1 -- 172.21.15.1:0/3821965246 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe80991d0 con 0x560fde528010 2024-02-02T15:46:44.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.043+0000 7febedc00700 1 -- 172.21.15.1:0/3821965246 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7febe8063030 con 0x560fde528010 2024-02-02T15:46:44.044 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.043+0000 7febedc00700 1 -- 172.21.15.1:0/3821965246 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe80a0fe0 con 0x560fde528010 2024-02-02T15:46:44.048 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.044+0000 7fec125e3b80 1 -- 172.21.15.1:0/3821965246 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560fde528010 msgr2=0x560fde53a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:44.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.044+0000 7fec125e3b80 1 --2- 172.21.15.1:0/3821965246 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560fde528010 0x560fde53a320 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7febe80b7510 tx=0x7febe8095870 comp rx=0 tx=0).stop 2024-02-02T15:46:44.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.044+0000 7fec125e3b80 1 -- 172.21.15.1:0/3821965246 shutdown_connections 2024-02-02T15:46:44.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.044+0000 7fec125e3b80 1 --2- 172.21.15.1:0/3821965246 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560fde5337e0 0x560fde532ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:44.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.044+0000 7fec125e3b80 1 --2- 172.21.15.1:0/3821965246 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560fde528010 0x560fde53a320 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:44.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.044+0000 7fec125e3b80 1 --2- 172.21.15.1:0/3821965246 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560fde528930 0x560fde522b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:44.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.044+0000 7fec125e3b80 1 -- 172.21.15.1:0/3821965246 >> 172.21.15.1:0/3821965246 conn(0x560fde487db0 msgr2=0x560fde51fe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:44.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.044+0000 7fec125e3b80 1 -- 172.21.15.1:0/3821965246 shutdown_connections 2024-02-02T15:46:44.049 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.044+0000 7fec125e3b80 1 -- 172.21.15.1:0/3821965246 wait complete. 2024-02-02T15:46:44.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.044+0000 7fec125e3b80 1 -- start start 2024-02-02T15:46:44.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.045+0000 7fec125e3b80 1 Processor -- start 2024-02-02T15:46:44.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.045+0000 7fec125e3b80 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560fde528010 0x560fde52c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:44.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.045+0000 7fec125e3b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560fde528930 0x560fde52cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:44.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.045+0000 7fec125e3b80 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560fde5337e0 0x560fde52d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:44.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.045+0000 7fec125e3b80 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fde52d640 con 0x560fde5337e0 2024-02-02T15:46:44.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.045+0000 7fec125e3b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fde52d7b0 con 0x560fde528930 2024-02-02T15:46:44.050 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.045+0000 7fec125e3b80 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fde52d970 con 0x560fde528010 2024-02-02T15:46:44.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.045+0000 7fec125e3b80 10 client.0 _ll_register_callbacks cb 0x560fde417290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-02-02T15:46:44.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.045+0000 7febfca00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560fde5337e0 0x560fde52d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:44.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.045+0000 7febf7400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560fde528930 0x560fde52cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:44.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.045+0000 7febf7400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560fde528930 0x560fde52cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:33290/0 (socket says 172.21.15.1:33290) 2024-02-02T15:46:44.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.045+0000 7febf7400700 1 -- 172.21.15.1:0/3577027609 learned_addr learned my addr 172.21.15.1:0/3577027609 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:46:44.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.045+0000 7febf7e00700 1 --2- 172.21.15.1:0/3577027609 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560fde528010 0x560fde52c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:44.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.046+0000 7febf7400700 1 -- 172.21.15.1:0/3577027609 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560fde528010 msgr2=0x560fde52c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:44.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.046+0000 7febf7400700 1 --2- 172.21.15.1:0/3577027609 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560fde528010 0x560fde52c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:44.051 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.046+0000 7febf7400700 1 -- 172.21.15.1:0/3577027609 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560fde5337e0 msgr2=0x560fde52d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-02-02T15:46:44.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.046+0000 7febf7400700 1 --2- 172.21.15.1:0/3577027609 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560fde5337e0 0x560fde52d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:44.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.046+0000 7febf7400700 1 -- 172.21.15.1:0/3577027609 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560fde2a7b60 con 0x560fde528930 2024-02-02T15:46:44.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.046+0000 7febfca00700 1 --2- 172.21.15.1:0/3577027609 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560fde5337e0 0x560fde52d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:46:44.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.046+0000 7febf7400700 1 --2- 172.21.15.1:0/3577027609 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560fde528930 0x560fde52cbc0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7febf806a570 tx=0x7febe0009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1bfe602f42f7deb server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:44.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.046+0000 7febe5600700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-02-02T15:46:44.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.047+0000 7febe5600700 1 -- 172.21.15.1:0/3577027609 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe0034dd0 con 0x560fde528930 2024-02-02T15:46:44.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.047+0000 7febe5600700 1 -- 172.21.15.1:0/3577027609 <== mon.2 v2:172.21.15.71:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 9453+0+0 (secure 0 0 0) 0x7febe0004820 con 0x560fde528930 2024-02-02T15:46:44.052 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.047+0000 7fec125e3b80 1 -- 172.21.15.1:0/3577027609 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7febf000f040 con 0x560fde528930 2024-02-02T15:46:44.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.047+0000 7febe5600700 1 -- 172.21.15.1:0/3577027609 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) v2 ==== 24+0+0 (secure 0 0 0) 0x7febe00049e0 con 0x560fde528930 2024-02-02T15:46:44.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.047+0000 7fec125e3b80 10 client.30680 fetch_fsmap learned FSMap version 81 2024-02-02T15:46:44.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.047+0000 7fec125e3b80 1 -- 172.21.15.1:0/3577027609 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=4+}) v3 -- 0x560fde2ffcd0 con 0x560fde528930 2024-02-02T15:46:44.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.047+0000 7febe5600700 1 -- 172.21.15.1:0/3577027609 <== mon.2 v2:172.21.15.71:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7febf000f040 con 0x560fde528930 2024-02-02T15:46:44.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.047+0000 7febe5600700 1 -- 172.21.15.1:0/3577027609 <== mon.2 v2:172.21.15.71:3300/0 5 ==== fsmap(e 81) v1 ==== 7482+0+0 (secure 0 0 0) 0x7febe003da40 con 0x560fde528930 2024-02-02T15:46:44.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.047+0000 7fec125e3b80 10 client.30680 fetch_fsmap finished waiting for FSMap version 81 2024-02-02T15:46:44.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.047+0000 7fec125e3b80 10 client.30680 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-02-02T15:46:44.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.047+0000 7fec125e3b80 20 client.30680 populate_metadata read hostname 'smithi001' 2024-02-02T15:46:44.053 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.047+0000 7fec125e3b80 1 --2- 172.21.15.1:0/3577027609 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x560fde595580 0x560fde5979a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:44.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.047+0000 7fec125e3b80 4 client.30680 mds_command: new command op to 24557 tid=0[{"prefix": "damage ls"}] 2024-02-02T15:46:44.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.047+0000 7fec125e3b80 1 -- 172.21.15.1:0/3577027609 --> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560fde438f50 con 0x560fde595580 2024-02-02T15:46:44.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.048+0000 7febf7e00700 1 --2- 172.21.15.1:0/3577027609 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x560fde595580 0x560fde5979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:44.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.048+0000 7febe5600700 10 client.30680 ms_handle_connect on v2:172.21.15.71:6838/1268086493 2024-02-02T15:46:44.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.048+0000 7febf7e00700 1 --2- 172.21.15.1:0/3577027609 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x560fde595580 0x560fde5979a0 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95f51a47c551251 server_cookie=3a252823d248c559 in_seq=0 out_seq=0 2024-02-02T15:46:44.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.048+0000 7febe5600700 1 -- 172.21.15.1:0/3577027609 <== mds.0 v2:172.21.15.71:6838/1268086493 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560fde438f50 con 0x560fde595580 2024-02-02T15:46:44.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.048+0000 7febe5600700 10 client.30680 handle_command_reply: tid=0 2024-02-02T15:46:44.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.048+0000 7fec125e3b80 1 client.30680 shutdown 2024-02-02T15:46:44.054 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.048+0000 7fec125e3b80 1 -- 172.21.15.1:0/3577027609 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560fde528930 msgr2=0x560fde52cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:44.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.048+0000 7fec125e3b80 1 --2- 172.21.15.1:0/3577027609 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560fde528930 0x560fde52cbc0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7febf806a570 tx=0x7febe0009ca0 comp rx=0 tx=0).stop 2024-02-02T15:46:44.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.048+0000 7fec125e3b80 1 -- 172.21.15.1:0/3577027609 shutdown_connections 2024-02-02T15:46:44.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.048+0000 7fec125e3b80 1 --2- 172.21.15.1:0/3577027609 >> [v2:172.21.15.71:6838/1268086493,v1:172.21.15.71:6839/1268086493] conn(0x560fde595580 0x560fde5979a0 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:44.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.048+0000 7fec125e3b80 1 --2- 172.21.15.1:0/3577027609 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x560fde5337e0 0x560fde52d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:44.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.048+0000 7fec125e3b80 1 --2- 172.21.15.1:0/3577027609 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x560fde528010 0x560fde52c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:44.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.048+0000 7fec125e3b80 1 --2- 172.21.15.1:0/3577027609 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560fde528930 0x560fde52cbc0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:44.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.048+0000 7febe5600700 0 client.30680 ms_handle_reset on v2:172.21.15.71:6838/1268086493 2024-02-02T15:46:44.055 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.048+0000 7fec125e3b80 1 -- 172.21.15.1:0/3577027609 >> 172.21.15.1:0/3577027609 conn(0x560fde487db0 msgr2=0x560fde522b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:44.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.049+0000 7fec125e3b80 1 -- 172.21.15.1:0/3577027609 shutdown_connections 2024-02-02T15:46:44.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.049+0000 7fec125e3b80 1 -- 172.21.15.1:0/3577027609 wait complete. 2024-02-02T15:46:44.056 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.049+0000 7fec125e3b80 20 client.30680 trim_cache size 0 max 16384 2024-02-02T15:46:44.056 INFO:teuthology.orchestra.run.smithi001.stdout:[] 2024-02-02T15:46:44.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.049+0000 7febfea00700 1 -- 172.21.15.1:0/879727734 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7febd80a0690 msgr2=0x7febd80a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:44.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.049+0000 7febfea00700 1 --2- 172.21.15.1:0/879727734 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7febd80a0690 0x7febd80a2b50 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7febf819a770 tx=0x7febe0005fd0 comp rx=0 tx=0).stop 2024-02-02T15:46:44.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.049+0000 7febfea00700 1 -- 172.21.15.1:0/879727734 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7febf8103290 msgr2=0x7febf8199160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:44.057 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.049+0000 7febfea00700 1 --2- 172.21.15.1:0/879727734 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7febf8103290 0x7febf8199160 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7febe800e9a0 tx=0x7febe800ecb0 comp rx=0 tx=0).stop 2024-02-02T15:46:44.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.049+0000 7febfea00700 1 -- 172.21.15.1:0/879727734 shutdown_connections 2024-02-02T15:46:44.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.049+0000 7febfea00700 1 --2- 172.21.15.1:0/879727734 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7febd80a0690 0x7febd80a2b50 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:44.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.049+0000 7febfea00700 1 --2- 172.21.15.1:0/879727734 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7febf819db00 0x7febf819df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:44.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.049+0000 7febfea00700 1 --2- 172.21.15.1:0/879727734 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7febf8103290 0x7febf8199160 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:44.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.049+0000 7febfea00700 1 --2- 172.21.15.1:0/879727734 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febf81083b0 0x7febf81996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:44.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.049+0000 7febfea00700 1 -- 172.21.15.1:0/879727734 >> 172.21.15.1:0/879727734 conn(0x7febf80feb30 msgr2=0x7febf80ff660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:44.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.050+0000 7febfea00700 1 -- 172.21.15.1:0/879727734 shutdown_connections 2024-02-02T15:46:44.058 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:44.050+0000 7febfea00700 1 -- 172.21.15.1:0/879727734 wait complete. 2024-02-02T15:46:44.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:46:43 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/4271448383' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:46:44.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:46:43 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/4271448383' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:46:45.067 INFO:tasks.fwd_scrub.fs.[cephfs]:end scrubbing fs: cephfs 2024-02-02T15:46:45.068 INFO:tasks.fwd_scrub:done joining 2024-02-02T15:46:45.068 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-02-02T15:46:45.079 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T15:46:45.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:46:44 smithi071 ceph-mon[44707]: pgmap v621: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 149 KiB/s wr, 31 op/s 2024-02-02T15:46:45.143 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:46:44 smithi001 ceph-mon[35470]: pgmap v621: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 149 KiB/s wr, 31 op/s 2024-02-02T15:46:45.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:46:44 smithi142 ceph-mon[45728]: pgmap v621: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 149 KiB/s wr, 31 op/s 2024-02-02T15:46:45.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.291+0000 7f4a30a00700 1 -- 172.21.15.1:0/1597149903 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4a28100f20 msgr2=0x7f4a28101300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:45.290 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.291+0000 7f4a30a00700 1 --2- 172.21.15.1:0/1597149903 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4a28100f20 0x7f4a28101300 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f4a18009990 tx=0x7f4a18009ca0 comp rx=0 tx=0).stop 2024-02-02T15:46:45.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.291+0000 7f4a30a00700 1 -- 172.21.15.1:0/1597149903 shutdown_connections 2024-02-02T15:46:45.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.291+0000 7f4a30a00700 1 --2- 172.21.15.1:0/1597149903 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a28106040 0x7f4a2810dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:45.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.291+0000 7f4a30a00700 1 --2- 172.21.15.1:0/1597149903 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4a28100f20 0x7f4a28101300 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:45.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.291+0000 7f4a30a00700 1 --2- 172.21.15.1:0/1597149903 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a281018d0 0x7f4a28105910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:45.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.291+0000 7f4a30a00700 1 -- 172.21.15.1:0/1597149903 >> 172.21.15.1:0/1597149903 conn(0x7f4a280fc7a0 msgr2=0x7f4a280febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:45.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.291+0000 7f4a30a00700 1 -- 172.21.15.1:0/1597149903 shutdown_connections 2024-02-02T15:46:45.291 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.291+0000 7f4a30a00700 1 -- 172.21.15.1:0/1597149903 wait complete. 2024-02-02T15:46:45.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.292+0000 7f4a30a00700 1 Processor -- start 2024-02-02T15:46:45.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.292+0000 7f4a30a00700 1 -- start start 2024-02-02T15:46:45.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.293+0000 7f4a30a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a28100f20 0x7f4a28073040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:45.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.293+0000 7f4a30a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4a28106040 0x7f4a28073580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:45.294 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.293+0000 7f4a30a00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a28073ac0 0x7f4a281a4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:45.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.293+0000 7f4a30a00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a281a5670 con 0x7f4a28106040 2024-02-02T15:46:45.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.293+0000 7f4a30a00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a281a57e0 con 0x7f4a28100f20 2024-02-02T15:46:45.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.293+0000 7f4a30a00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a281a59a0 con 0x7f4a28073ac0 2024-02-02T15:46:45.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.293+0000 7f4a2e000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a28100f20 0x7f4a28073040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:45.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.293+0000 7f4a2d600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4a28106040 0x7f4a28073580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:45.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.293+0000 7f4a2e000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a28100f20 0x7f4a28073040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:33304/0 (socket says 172.21.15.1:33304) 2024-02-02T15:46:45.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.293+0000 7f4a2e000700 1 -- 172.21.15.1:0/3160251016 learned_addr learned my addr 172.21.15.1:0/3160251016 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:46:45.295 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.293+0000 7f4a2ea00700 1 --2- 172.21.15.1:0/3160251016 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a28073ac0 0x7f4a281a4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:45.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.293+0000 7f4a2d600700 1 -- 172.21.15.1:0/3160251016 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a28073ac0 msgr2=0x7f4a281a4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:45.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.293+0000 7f4a2d600700 1 --2- 172.21.15.1:0/3160251016 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a28073ac0 0x7f4a281a4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:45.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.293+0000 7f4a2d600700 1 -- 172.21.15.1:0/3160251016 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a28100f20 msgr2=0x7f4a28073040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:45.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.293+0000 7f4a2d600700 1 --2- 172.21.15.1:0/3160251016 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a28100f20 0x7f4a28073040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:45.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.293+0000 7f4a2d600700 1 -- 172.21.15.1:0/3160251016 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a180095f0 con 0x7f4a28106040 2024-02-02T15:46:45.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.293+0000 7f4a2e000700 1 --2- 172.21.15.1:0/3160251016 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a28100f20 0x7f4a28073040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:46:45.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.294+0000 7f4a2d600700 1 --2- 172.21.15.1:0/3160251016 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4a28106040 0x7f4a28073580 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f4a1000e9c0 tx=0x7f4a1000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:45.296 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.294+0000 7f4a26a00700 1 -- 172.21.15.1:0/3160251016 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1000c9c0 con 0x7f4a28106040 2024-02-02T15:46:45.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.294+0000 7f4a26a00700 1 -- 172.21.15.1:0/3160251016 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4a10010430 con 0x7f4a28106040 2024-02-02T15:46:45.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.294+0000 7f4a30a00700 1 -- 172.21.15.1:0/3160251016 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a281a5c80 con 0x7f4a28106040 2024-02-02T15:46:45.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.294+0000 7f4a26a00700 1 -- 172.21.15.1:0/3160251016 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a10018680 con 0x7f4a28106040 2024-02-02T15:46:45.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.294+0000 7f4a30a00700 1 -- 172.21.15.1:0/3160251016 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a281a6190 con 0x7f4a28106040 2024-02-02T15:46:45.297 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.295+0000 7f4a30a00700 1 -- 172.21.15.1:0/3160251016 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a28067bc0 con 0x7f4a28106040 2024-02-02T15:46:45.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.297+0000 7f4a26a00700 1 -- 172.21.15.1:0/3160251016 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f4a1000cb20 con 0x7f4a28106040 2024-02-02T15:46:45.298 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.298+0000 7f4a26a00700 1 --2- 172.21.15.1:0/3160251016 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f4a080a0710 0x7f4a080a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:45.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.298+0000 7f4a26a00700 1 -- 172.21.15.1:0/3160251016 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f4a10014070 con 0x7f4a28106040 2024-02-02T15:46:45.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.299+0000 7f4a2e000700 1 --2- 172.21.15.1:0/3160251016 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f4a080a0710 0x7f4a080a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:45.299 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.299+0000 7f4a2e000700 1 --2- 172.21.15.1:0/3160251016 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f4a080a0710 0x7f4a080a2bd0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f4a18006010 tx=0x7f4a1800b260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:45.301 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.302+0000 7f4a26a00700 1 -- 172.21.15.1:0/3160251016 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f4a10052110 con 0x7f4a28106040 2024-02-02T15:46:45.519 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.521+0000 7f4a30a00700 1 -- 172.21.15.1:0/3160251016 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4a281019d0 con 0x7f4a28106040 2024-02-02T15:46:45.521 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.522+0000 7f4a26a00700 1 -- 172.21.15.1:0/3160251016 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v115) v1 ==== 74+0+25013 (secure 0 0 0) 0x7f4a10095660 con 0x7f4a28106040 2024-02-02T15:46:45.522 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:46:45.522 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":115,"fsid":"371a409c-c1de-11ee-95b6-87774f69a715","created":"2024-02-02T15:18:26.746506+0000","modified":"2024-02-02T15:46:16.678513+0000","last_up_change":"2024-02-02T15:24:08.800951+0000","last_in_change":"2024-02-02T15:23:52.784280+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T15:21:22.471754+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-02T15:24:46.195779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-02T15:24:47.226403+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":33,"snap_epoch":114,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-02T15:24:50.300462+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":33,"snap_epoch":114,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"73e8a07f-3560-4c19-85ed-030d4d8f0f23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6803","nonce":2591906423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6805","nonce":2591906423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6809","nonce":2591906423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6807","nonce":2591906423}]},"public_addr":"172.21.15.1:6803/2591906423","cluster_addr":"172.21.15.1:6805/2591906423","heartbeat_back_addr":"172.21.15.1:6809/2591906423","heartbeat_front_addr":"172.21.15.1:6807/2591906423","state":["exists","up"]},{"osd":1,"uuid":"e53cffec-0ff9-4ed9-8dcf-ff47d04db2bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6801","nonce":747581719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6803","nonce":747581719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6807","nonce":747581719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6805","nonce":747581719}]},"public_addr":"172.21.15.71:6801/747581719","cluster_addr":"172.21.15.71:6803/747581719","heartbeat_back_addr":"172.21.15.71:6807/747581719","heartbeat_front_addr":"172.21.15.71:6805/747581719","state":["exists","up"]},{"osd":2,"uuid":"856c2347-294c-482b-8595-ad77f4792db4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6801","nonce":377511414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6803","nonce":377511414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6807","nonce":377511414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6805","nonce":377511414}]},"public_addr":"172.21.15.142:6801/377511414","cluster_addr":"172.21.15.142:6803/377511414","heartbeat_back_addr":"172.21.15.142:6807/377511414","heartbeat_front_addr":"172.21.15.142:6805/377511414","state":["exists","up"]},{"osd":3,"uuid":"4b946ad2-2da9-4e0b-9d61-d039d8669a42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6811","nonce":1086124477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6813","nonce":1086124477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6817","nonce":1086124477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6815","nonce":1086124477}]},"public_addr":"172.21.15.1:6811/1086124477","cluster_addr":"172.21.15.1:6813/1086124477","heartbeat_back_addr":"172.21.15.1:6817/1086124477","heartbeat_front_addr":"172.21.15.1:6815/1086124477","state":["exists","up"]},{"osd":4,"uuid":"43e36c43-47a0-4634-a45b-1da14a0e19d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6809","nonce":3095380305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6811","nonce":3095380305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6815","nonce":3095380305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6813","nonce":3095380305}]},"public_addr":"172.21.15.71:6809/3095380305","cluster_addr":"172.21.15.71:6811/3095380305","heartbeat_back_addr":"172.21.15.71:6815/3095380305","heartbeat_front_addr":"172.21.15.71:6813/3095380305","state":["exists","up"]},{"osd":5,"uuid":"eb48160e-ea7c-42f7-9477-be110bc8a34e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6809","nonce":4195149262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6811","nonce":4195149262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6815","nonce":4195149262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6813","nonce":4195149262}]},"public_addr":"172.21.15.142:6809/4195149262","cluster_addr":"172.21.15.142:6811/4195149262","heartbeat_back_addr":"172.21.15.142:6815/4195149262","heartbeat_front_addr":"172.21.15.142:6813/4195149262","state":["exists","up"]},{"osd":6,"uuid":"0ec4501a-5853-4ca6-b325-231f5b72335e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6819","nonce":1560153428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6821","nonce":1560153428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6825","nonce":1560153428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6823","nonce":1560153428}]},"public_addr":"172.21.15.1:6819/1560153428","cluster_addr":"172.21.15.1:6821/1560153428","heartbeat_back_addr":"172.21.15.1:6825/1560153428","heartbeat_front_addr":"172.21.15.1:6823/1560153428","state":["exists","up"]},{"osd":7,"uuid":"029debe8-9cdf-44f6-9aea-cabd1d185f93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6817","nonce":4080955538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6819","nonce":4080955538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6823","nonce":4080955538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6821","nonce":4080955538}]},"public_addr":"172.21.15.71:6817/4080955538","cluster_addr":"172.21.15.71:6819/4080955538","heartbeat_back_addr":"172.21.15.71:6823/4080955538","heartbeat_front_addr":"172.21.15.71:6821/4080955538","state":["exists","up"]},{"osd":8,"uuid":"7c90896c-9955-4c1b-aaca-7531dd5457f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6817","nonce":3942038209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6819","nonce":3942038209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6823","nonce":3942038209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6821","nonce":3942038209}]},"public_addr":"172.21.15.142:6817/3942038209","cluster_addr":"172.21.15.142:6819/3942038209","heartbeat_back_addr":"172.21.15.142:6823/3942038209","heartbeat_front_addr":"172.21.15.142:6821/3942038209","state":["exists","up"]},{"osd":9,"uuid":"7e165e68-31a8-4598-a577-de3e90ae5c5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6827","nonce":2716544850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6829","nonce":2716544850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6833","nonce":2716544850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6831","nonce":2716544850}]},"public_addr":"172.21.15.1:6827/2716544850","cluster_addr":"172.21.15.1:6829/2716544850","heartbeat_back_addr":"172.21.15.1:6833/2716544850","heartbeat_front_addr":"172.21.15.1:6831/2716544850","state":["exists","up"]},{"osd":10,"uuid":"48cd5fdb-0887-444a-aab8-a1de24282bb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6825","nonce":84691841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6827","nonce":84691841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6831","nonce":84691841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6829","nonce":84691841}]},"public_addr":"172.21.15.71:6825/84691841","cluster_addr":"172.21.15.71:6827/84691841","heartbeat_back_addr":"172.21.15.71:6831/84691841","heartbeat_front_addr":"172.21.15.71:6829/84691841","state":["exists","up"]},{"osd":11,"uuid":"f011dd27-11b0-4166-adfe-76756533163a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6825","nonce":2290179934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6827","nonce":2290179934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6831","nonce":2290179934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6829","nonce":2290179934}]},"public_addr":"172.21.15.142:6825/2290179934","cluster_addr":"172.21.15.142:6827/2290179934","heartbeat_back_addr":"172.21.15.142:6831/2290179934","heartbeat_front_addr":"172.21.15.142:6829/2290179934","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:20:44.060692+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:01.903817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:19.368258+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:40.028037+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:57.513361+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:22:14.805944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:22:35.188439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:22:53.014412+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:23:10.507420+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:23:31.849765+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:23:50.070612+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:24:07.388625+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":5,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/3997770901":"2024-02-03T15:27:00.115055+0000","172.21.15.1:0/3236204176":"2024-02-03T15:27:00.115055+0000","172.21.15.1:0/2256116842":"2024-02-03T15:27:00.115055+0000","172.21.15.1:0/4092532060":"2024-02-03T15:19:25.836171+0000","172.21.15.1:6801/3768989981":"2024-02-03T15:27:00.115055+0000","172.21.15.1:6800/3768989981":"2024-02-03T15:27:00.115055+0000","172.21.15.1:6801/395498296":"2024-02-03T15:19:25.836171+0000","172.21.15.1:6800/395498296":"2024-02-03T15:19:25.836171+0000","172.21.15.1:0/1574620681":"2024-02-03T15:27:00.115055+0000","172.21.15.1:0/1575373489":"2024-02-03T15:19:25.836171+0000","172.21.15.1:0/865983374":"2024-02-03T15:19:01.431596+0000","172.21.15.1:0/3165960227":"2024-02-03T15:19:01.431596+0000","172.21.15.1:0/2350830293":"2024-02-03T15:19:01.431596+0000","172.21.15.1:0/2959690250":"2024-02-03T15:19:25.836171+0000","172.21.15.1:6800/1162530732":"2024-02-03T15:19:01.431596+0000","172.21.15.1:6801/1162530732":"2024-02-03T15:19:01.431596+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":20,"length":1},{"begin":33,"length":1}]},{"pool":4,"snaps":[{"begin":20,"length":1},{"begin":33,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T15:46:45.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.525+0000 7f4a30a00700 1 -- 172.21.15.1:0/3160251016 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f4a080a0710 msgr2=0x7f4a080a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:45.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.525+0000 7f4a30a00700 1 --2- 172.21.15.1:0/3160251016 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f4a080a0710 0x7f4a080a2bd0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f4a18006010 tx=0x7f4a1800b260 comp rx=0 tx=0).stop 2024-02-02T15:46:45.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.525+0000 7f4a30a00700 1 -- 172.21.15.1:0/3160251016 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4a28106040 msgr2=0x7f4a28073580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:45.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.525+0000 7f4a30a00700 1 --2- 172.21.15.1:0/3160251016 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4a28106040 0x7f4a28073580 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f4a1000e9c0 tx=0x7f4a1000ecd0 comp rx=0 tx=0).stop 2024-02-02T15:46:45.524 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.525+0000 7f4a30a00700 1 -- 172.21.15.1:0/3160251016 shutdown_connections 2024-02-02T15:46:45.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.525+0000 7f4a30a00700 1 --2- 172.21.15.1:0/3160251016 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f4a080a0710 0x7f4a080a2bd0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:45.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.525+0000 7f4a30a00700 1 --2- 172.21.15.1:0/3160251016 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4a28073ac0 0x7f4a281a4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:45.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.525+0000 7f4a30a00700 1 --2- 172.21.15.1:0/3160251016 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a28100f20 0x7f4a28073040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:45.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.525+0000 7f4a30a00700 1 --2- 172.21.15.1:0/3160251016 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4a28106040 0x7f4a28073580 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:45.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.525+0000 7f4a30a00700 1 -- 172.21.15.1:0/3160251016 >> 172.21.15.1:0/3160251016 conn(0x7f4a280fc7a0 msgr2=0x7f4a280fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:45.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.525+0000 7f4a30a00700 1 -- 172.21.15.1:0/3160251016 shutdown_connections 2024-02-02T15:46:45.525 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.525+0000 7f4a30a00700 1 -- 172.21.15.1:0/3160251016 wait complete. 2024-02-02T15:46:45.543 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-02T15:21:22.471754+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-02T15:24:46.195779+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-02T15:24:47.226403+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '114', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 33, 'snap_epoch': 114, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-02-02T15:24:50.300462+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '114', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 33, 'snap_epoch': 114, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-02-02T15:46:45.544 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-02T15:46:45.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.748+0000 7f3143000700 1 -- 172.21.15.1:0/3447305822 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f313c1019a0 msgr2=0x7f313c1059e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:45.747 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.748+0000 7f3143000700 1 --2- 172.21.15.1:0/3447305822 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f313c1019a0 0x7f313c1059e0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f3124009990 tx=0x7f3124009ca0 comp rx=0 tx=0).stop 2024-02-02T15:46:45.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.748+0000 7f3143000700 1 -- 172.21.15.1:0/3447305822 shutdown_connections 2024-02-02T15:46:45.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.748+0000 7f3143000700 1 --2- 172.21.15.1:0/3447305822 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f313c106110 0x7f313c10dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:45.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.748+0000 7f3143000700 1 --2- 172.21.15.1:0/3447305822 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f313c100ff0 0x7f313c1013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:45.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.748+0000 7f3143000700 1 --2- 172.21.15.1:0/3447305822 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f313c1019a0 0x7f313c1059e0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:45.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.748+0000 7f3143000700 1 -- 172.21.15.1:0/3447305822 >> 172.21.15.1:0/3447305822 conn(0x7f313c0fc810 msgr2=0x7f313c0fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:45.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.748+0000 7f3143000700 1 -- 172.21.15.1:0/3447305822 shutdown_connections 2024-02-02T15:46:45.748 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.748+0000 7f3143000700 1 -- 172.21.15.1:0/3447305822 wait complete. 2024-02-02T15:46:45.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.749+0000 7f3143000700 1 Processor -- start 2024-02-02T15:46:45.750 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.750+0000 7f3143000700 1 -- start start 2024-02-02T15:46:45.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.750+0000 7f3143000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f313c100ff0 0x7f313c196f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:45.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.750+0000 7f3143000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f313c106110 0x7f313c1974a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:45.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.750+0000 7f3143000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f313c19b900 0x7f313c19bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:45.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.750+0000 7f3143000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f313c19c380 con 0x7f313c19b900 2024-02-02T15:46:45.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.750+0000 7f3143000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f313c19c4f0 con 0x7f313c106110 2024-02-02T15:46:45.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.750+0000 7f3143000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f313c19c6b0 con 0x7f313c100ff0 2024-02-02T15:46:45.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.751+0000 7f3141000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f313c19b900 0x7f313c19bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:45.751 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.751+0000 7f3141000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f313c19b900 0x7f313c19bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41716/0 (socket says 172.21.15.1:41716) 2024-02-02T15:46:45.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.751+0000 7f3141000700 1 -- 172.21.15.1:0/2298468716 learned_addr learned my addr 172.21.15.1:0/2298468716 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:46:45.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.751+0000 7f313be00700 1 --2- 172.21.15.1:0/2298468716 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f313c100ff0 0x7f313c196f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:45.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.751+0000 7f3133e00700 1 --2- 172.21.15.1:0/2298468716 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f313c106110 0x7f313c1974a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:45.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.751+0000 7f3141000700 1 -- 172.21.15.1:0/2298468716 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f313c100ff0 msgr2=0x7f313c196f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:45.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.751+0000 7f3141000700 1 --2- 172.21.15.1:0/2298468716 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f313c100ff0 0x7f313c196f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:45.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.751+0000 7f3141000700 1 -- 172.21.15.1:0/2298468716 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f313c106110 msgr2=0x7f313c1974a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:45.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.751+0000 7f3141000700 1 --2- 172.21.15.1:0/2298468716 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f313c106110 0x7f313c1974a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:45.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.751+0000 7f3141000700 1 -- 172.21.15.1:0/2298468716 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f312c009520 con 0x7f313c19b900 2024-02-02T15:46:45.752 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.751+0000 7f3141000700 1 --2- 172.21.15.1:0/2298468716 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f313c19b900 0x7f313c19bd80 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f313400ef30 tx=0x7f313400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:45.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.752+0000 7f3139600700 1 -- 172.21.15.1:0/2298468716 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3134018070 con 0x7f313c19b900 2024-02-02T15:46:45.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.752+0000 7f3143000700 1 -- 172.21.15.1:0/2298468716 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31240095f0 con 0x7f313c19b900 2024-02-02T15:46:45.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.752+0000 7f3139600700 1 -- 172.21.15.1:0/2298468716 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f313400f040 con 0x7f313c19b900 2024-02-02T15:46:45.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.752+0000 7f3139600700 1 -- 172.21.15.1:0/2298468716 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f313400cd00 con 0x7f313c19b900 2024-02-02T15:46:45.753 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.752+0000 7f3143000700 1 -- 172.21.15.1:0/2298468716 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f313c1a3f00 con 0x7f313c19b900 2024-02-02T15:46:45.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.753+0000 7f3143000700 1 -- 172.21.15.1:0/2298468716 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f313c067bc0 con 0x7f313c19b900 2024-02-02T15:46:45.754 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.755+0000 7f3139600700 1 -- 172.21.15.1:0/2298468716 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f3134003a00 con 0x7f313c19b900 2024-02-02T15:46:45.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.756+0000 7f3139600700 1 --2- 172.21.15.1:0/2298468716 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f311c0a0690 0x7f311c0a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:45.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.756+0000 7f3139600700 1 -- 172.21.15.1:0/2298468716 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f31340cbc30 con 0x7f313c19b900 2024-02-02T15:46:45.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.756+0000 7f313be00700 1 --2- 172.21.15.1:0/2298468716 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f311c0a0690 0x7f311c0a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:45.755 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.757+0000 7f313be00700 1 --2- 172.21.15.1:0/2298468716 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f311c0a0690 0x7f311c0a2b50 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f312c009df0 tx=0x7f312c009260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:45.758 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.760+0000 7f3139600700 1 -- 172.21.15.1:0/2298468716 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f3134095180 con 0x7f313c19b900 2024-02-02T15:46:45.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:46:45 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/3160251016' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-02T15:46:45.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.968+0000 7f3143000700 1 -- 172.21.15.1:0/2298468716 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f313c1019a0 con 0x7f313c19b900 2024-02-02T15:46:45.967 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.968+0000 7f3139600700 1 -- 172.21.15.1:0/2298468716 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v115) v1 ==== 93+0+10 (secure 0 0 0) 0x7f31340987a0 con 0x7f313c19b900 2024-02-02T15:46:45.967 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-02-02T15:46:45.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.970+0000 7f3143000700 1 -- 172.21.15.1:0/2298468716 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f311c0a0690 msgr2=0x7f311c0a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:45.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.970+0000 7f3143000700 1 --2- 172.21.15.1:0/2298468716 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f311c0a0690 0x7f311c0a2b50 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f312c009df0 tx=0x7f312c009260 comp rx=0 tx=0).stop 2024-02-02T15:46:45.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.971+0000 7f3143000700 1 -- 172.21.15.1:0/2298468716 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f313c19b900 msgr2=0x7f313c19bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:45.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.971+0000 7f3143000700 1 --2- 172.21.15.1:0/2298468716 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f313c19b900 0x7f313c19bd80 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f313400ef30 tx=0x7f313400c3a0 comp rx=0 tx=0).stop 2024-02-02T15:46:45.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.971+0000 7f3143000700 1 -- 172.21.15.1:0/2298468716 shutdown_connections 2024-02-02T15:46:45.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.971+0000 7f3143000700 1 --2- 172.21.15.1:0/2298468716 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f311c0a0690 0x7f311c0a2b50 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:45.970 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.971+0000 7f3143000700 1 --2- 172.21.15.1:0/2298468716 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f313c19b900 0x7f313c19bd80 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:45.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.971+0000 7f3143000700 1 --2- 172.21.15.1:0/2298468716 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f313c100ff0 0x7f313c196f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:45.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.971+0000 7f3143000700 1 --2- 172.21.15.1:0/2298468716 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f313c106110 0x7f313c1974a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:45.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.971+0000 7f3143000700 1 -- 172.21.15.1:0/2298468716 >> 172.21.15.1:0/2298468716 conn(0x7f313c0fc810 msgr2=0x7f313c0fd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:45.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.971+0000 7f3143000700 1 -- 172.21.15.1:0/2298468716 shutdown_connections 2024-02-02T15:46:45.971 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:45.971+0000 7f3143000700 1 -- 172.21.15.1:0/2298468716 wait complete. 2024-02-02T15:46:45.990 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-02T15:46:46.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:46:45 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/3160251016' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-02T15:46:46.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:46:45 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/3160251016' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-02T15:46:46.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.192+0000 7f2add600700 1 -- 172.21.15.1:0/3373613900 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2ad8106aa0 msgr2=0x7f2ad8106e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:46.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.192+0000 7f2add600700 1 --2- 172.21.15.1:0/3373613900 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2ad8106aa0 0x7f2ad8106e80 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f2ac8009990 tx=0x7f2ac8009ca0 comp rx=0 tx=0).stop 2024-02-02T15:46:46.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.193+0000 7f2add600700 1 -- 172.21.15.1:0/3373613900 shutdown_connections 2024-02-02T15:46:46.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.193+0000 7f2add600700 1 --2- 172.21.15.1:0/3373613900 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ad8101450 0x7f2ad810d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:46.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.193+0000 7f2add600700 1 --2- 172.21.15.1:0/3373613900 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2ad8106aa0 0x7f2ad8106e80 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:46.192 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.193+0000 7f2add600700 1 --2- 172.21.15.1:0/3373613900 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ad8100a90 0x7f2ad8100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:46.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.193+0000 7f2add600700 1 -- 172.21.15.1:0/3373613900 >> 172.21.15.1:0/3373613900 conn(0x7f2ad80fc580 msgr2=0x7f2ad80fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:46.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.193+0000 7f2add600700 1 -- 172.21.15.1:0/3373613900 shutdown_connections 2024-02-02T15:46:46.193 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.193+0000 7f2add600700 1 -- 172.21.15.1:0/3373613900 wait complete. 2024-02-02T15:46:46.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.194+0000 7f2add600700 1 Processor -- start 2024-02-02T15:46:46.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.194+0000 7f2add600700 1 -- start start 2024-02-02T15:46:46.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.194+0000 7f2add600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2ad8100a90 0x7f2ad8198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:46.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.194+0000 7f2add600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ad8101450 0x7f2ad8199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:46.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.194+0000 7f2add600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ad8106aa0 0x7f2ad819d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:46.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.194+0000 7f2add600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad819ddb0 con 0x7f2ad8100a90 2024-02-02T15:46:46.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.194+0000 7f2add600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad819df20 con 0x7f2ad8106aa0 2024-02-02T15:46:46.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.194+0000 7f2add600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad819e0e0 con 0x7f2ad8101450 2024-02-02T15:46:46.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.194+0000 7f2ad7e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2ad8100a90 0x7f2ad8198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:46.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.194+0000 7f2ad7e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2ad8100a90 0x7f2ad8198dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41732/0 (socket says 172.21.15.1:41732) 2024-02-02T15:46:46.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.194+0000 7f2ad7e00700 1 -- 172.21.15.1:0/3348966536 learned_addr learned my addr 172.21.15.1:0/3348966536 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:46:46.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.195+0000 7f2ad7400700 1 --2- 172.21.15.1:0/3348966536 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ad8101450 0x7f2ad8199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:46.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.195+0000 7f2adcc00700 1 --2- 172.21.15.1:0/3348966536 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ad8106aa0 0x7f2ad819d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:46.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.195+0000 7f2ad7e00700 1 -- 172.21.15.1:0/3348966536 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ad8101450 msgr2=0x7f2ad8199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:46.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.195+0000 7f2ad7e00700 1 --2- 172.21.15.1:0/3348966536 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ad8101450 0x7f2ad8199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:46.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.195+0000 7f2ad7e00700 1 -- 172.21.15.1:0/3348966536 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ad8106aa0 msgr2=0x7f2ad819d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:46.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.195+0000 7f2ad7e00700 1 --2- 172.21.15.1:0/3348966536 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ad8106aa0 0x7f2ad819d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:46.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.195+0000 7f2ad7e00700 1 -- 172.21.15.1:0/3348966536 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ac80095f0 con 0x7f2ad8100a90 2024-02-02T15:46:46.197 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.195+0000 7f2ad7e00700 1 --2- 172.21.15.1:0/3348966536 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2ad8100a90 0x7f2ad8198dd0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f2ac8005910 tx=0x7f2ac8004710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:46.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.195+0000 7f2ad4c00700 1 -- 172.21.15.1:0/3348966536 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac801d070 con 0x7f2ad8100a90 2024-02-02T15:46:46.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.195+0000 7f2ad4c00700 1 -- 172.21.15.1:0/3348966536 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ac8022470 con 0x7f2ad8100a90 2024-02-02T15:46:46.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.195+0000 7f2ad4c00700 1 -- 172.21.15.1:0/3348966536 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac8017650 con 0x7f2ad8100a90 2024-02-02T15:46:46.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.196+0000 7f2add600700 1 -- 172.21.15.1:0/3348966536 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ad819e360 con 0x7f2ad8100a90 2024-02-02T15:46:46.198 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.196+0000 7f2add600700 1 -- 172.21.15.1:0/3348966536 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ad81a5c00 con 0x7f2ad8100a90 2024-02-02T15:46:46.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.197+0000 7f2add600700 1 -- 172.21.15.1:0/3348966536 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ad804ea40 con 0x7f2ad8100a90 2024-02-02T15:46:46.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.199+0000 7f2ad4c00700 1 -- 172.21.15.1:0/3348966536 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f2ac80225e0 con 0x7f2ad8100a90 2024-02-02T15:46:46.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.201+0000 7f2ad4c00700 1 --2- 172.21.15.1:0/3348966536 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f2ab80a0690 0x7f2ab80a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:46.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.201+0000 7f2ad4c00700 1 -- 172.21.15.1:0/3348966536 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f2ac80cd350 con 0x7f2ad8100a90 2024-02-02T15:46:46.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.201+0000 7f2ad7400700 1 --2- 172.21.15.1:0/3348966536 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f2ab80a0690 0x7f2ab80a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:46.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.202+0000 7f2ad7400700 1 --2- 172.21.15.1:0/3348966536 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f2ab80a0690 0x7f2ab80a2b50 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f2ad819a3e0 tx=0x7f2ac0009260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:46.203 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.205+0000 7f2ad4c00700 1 -- 172.21.15.1:0/3348966536 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f2ac805d110 con 0x7f2ad8100a90 2024-02-02T15:46:46.419 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.420+0000 7f2add600700 1 -- 172.21.15.1:0/3348966536 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2ad81a5dc0 con 0x7f2ad8100a90 2024-02-02T15:46:46.420 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.421+0000 7f2ad4c00700 1 -- 172.21.15.1:0/3348966536 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v115) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2ac8096840 con 0x7f2ad8100a90 2024-02-02T15:46:46.420 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-02-02T15:46:46.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.423+0000 7f2add600700 1 -- 172.21.15.1:0/3348966536 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f2ab80a0690 msgr2=0x7f2ab80a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:46.422 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.423+0000 7f2add600700 1 --2- 172.21.15.1:0/3348966536 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f2ab80a0690 0x7f2ab80a2b50 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f2ad819a3e0 tx=0x7f2ac0009260 comp rx=0 tx=0).stop 2024-02-02T15:46:46.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.423+0000 7f2add600700 1 -- 172.21.15.1:0/3348966536 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2ad8100a90 msgr2=0x7f2ad8198dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:46.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.423+0000 7f2add600700 1 --2- 172.21.15.1:0/3348966536 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2ad8100a90 0x7f2ad8198dd0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f2ac8005910 tx=0x7f2ac8004710 comp rx=0 tx=0).stop 2024-02-02T15:46:46.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.423+0000 7f2add600700 1 -- 172.21.15.1:0/3348966536 shutdown_connections 2024-02-02T15:46:46.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.423+0000 7f2add600700 1 --2- 172.21.15.1:0/3348966536 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f2ab80a0690 0x7f2ab80a2b50 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:46.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.423+0000 7f2add600700 1 --2- 172.21.15.1:0/3348966536 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ad8106aa0 0x7f2ad819d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:46.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.423+0000 7f2add600700 1 --2- 172.21.15.1:0/3348966536 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2ad8100a90 0x7f2ad8198dd0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:46.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.423+0000 7f2add600700 1 --2- 172.21.15.1:0/3348966536 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2ad8101450 0x7f2ad8199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:46.423 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.423+0000 7f2add600700 1 -- 172.21.15.1:0/3348966536 >> 172.21.15.1:0/3348966536 conn(0x7f2ad80fc580 msgr2=0x7f2ad810bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:46.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.423+0000 7f2add600700 1 -- 172.21.15.1:0/3348966536 shutdown_connections 2024-02-02T15:46:46.424 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.423+0000 7f2add600700 1 -- 172.21.15.1:0/3348966536 wait complete. 2024-02-02T15:46:46.441 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-02T15:46:46.642 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.643+0000 7f38f8c00700 1 -- 172.21.15.1:0/551441061 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f38f4106110 msgr2=0x7f38f410dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:46.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.643+0000 7f38f8c00700 1 --2- 172.21.15.1:0/551441061 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f38f4106110 0x7f38f410dc90 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f38ec00b0a0 tx=0x7f38ec00b3b0 comp rx=0 tx=0).stop 2024-02-02T15:46:46.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.643+0000 7f38f8c00700 1 -- 172.21.15.1:0/551441061 shutdown_connections 2024-02-02T15:46:46.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.643+0000 7f38f8c00700 1 --2- 172.21.15.1:0/551441061 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f38f4106110 0x7f38f410dc90 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:46.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.643+0000 7f38f8c00700 1 --2- 172.21.15.1:0/551441061 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f38f4100ff0 0x7f38f41013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:46.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.643+0000 7f38f8c00700 1 --2- 172.21.15.1:0/551441061 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38f41019a0 0x7f38f41059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:46.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.643+0000 7f38f8c00700 1 -- 172.21.15.1:0/551441061 >> 172.21.15.1:0/551441061 conn(0x7f38f40fc810 msgr2=0x7f38f40fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:46.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.643+0000 7f38f8c00700 1 -- 172.21.15.1:0/551441061 shutdown_connections 2024-02-02T15:46:46.644 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.643+0000 7f38f8c00700 1 -- 172.21.15.1:0/551441061 wait complete. 2024-02-02T15:46:46.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.645+0000 7f38f8c00700 1 Processor -- start 2024-02-02T15:46:46.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.645+0000 7f38f8c00700 1 -- start start 2024-02-02T15:46:46.646 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.646+0000 7f38f8c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f38f4100ff0 0x7f38f4073040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:46.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.646+0000 7f38f8c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38f4106110 0x7f38f4073580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:46.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.646+0000 7f38f8c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f38f4073ac0 0x7f38f41a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:46.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.646+0000 7f38f8c00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f41a5840 con 0x7f38f4100ff0 2024-02-02T15:46:46.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.646+0000 7f38f8c00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f41a59b0 con 0x7f38f4073ac0 2024-02-02T15:46:46.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.646+0000 7f38f8c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f41a5b70 con 0x7f38f4106110 2024-02-02T15:46:46.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.646+0000 7f38f1e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f38f4100ff0 0x7f38f4073040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:46.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.646+0000 7f38f1e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f38f4100ff0 0x7f38f4073040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41764/0 (socket says 172.21.15.1:41764) 2024-02-02T15:46:46.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.646+0000 7f38f1e00700 1 -- 172.21.15.1:0/2126011963 learned_addr learned my addr 172.21.15.1:0/2126011963 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:46:46.647 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.646+0000 7f38f2800700 1 --2- 172.21.15.1:0/2126011963 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f38f4073ac0 0x7f38f41a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:46.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.646+0000 7f38f1400700 1 --2- 172.21.15.1:0/2126011963 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38f4106110 0x7f38f4073580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:46.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.646+0000 7f38f1e00700 1 -- 172.21.15.1:0/2126011963 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38f4106110 msgr2=0x7f38f4073580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:46.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.646+0000 7f38f1e00700 1 --2- 172.21.15.1:0/2126011963 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38f4106110 0x7f38f4073580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:46.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.646+0000 7f38f1e00700 1 -- 172.21.15.1:0/2126011963 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f38f4073ac0 msgr2=0x7f38f41a51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:46.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.646+0000 7f38f1e00700 1 --2- 172.21.15.1:0/2126011963 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f38f4073ac0 0x7f38f41a51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:46.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.646+0000 7f38f1e00700 1 -- 172.21.15.1:0/2126011963 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38ec009d00 con 0x7f38f4100ff0 2024-02-02T15:46:46.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.646+0000 7f38f2800700 1 --2- 172.21.15.1:0/2126011963 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f38f4073ac0 0x7f38f41a51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:46:46.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.647+0000 7f38f1e00700 1 --2- 172.21.15.1:0/2126011963 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f38f4100ff0 0x7f38f4073040 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f38e400e9a0 tx=0x7f38e400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:46.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.647+0000 7f38eaa00700 1 -- 172.21.15.1:0/2126011963 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e4009790 con 0x7f38f4100ff0 2024-02-02T15:46:46.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.647+0000 7f38eaa00700 1 -- 172.21.15.1:0/2126011963 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f38e4010430 con 0x7f38f4100ff0 2024-02-02T15:46:46.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.647+0000 7f38f8c00700 1 -- 172.21.15.1:0/2126011963 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38f41a5e50 con 0x7f38f4100ff0 2024-02-02T15:46:46.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.647+0000 7f38eaa00700 1 -- 172.21.15.1:0/2126011963 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e4018680 con 0x7f38f4100ff0 2024-02-02T15:46:46.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.647+0000 7f38f8c00700 1 -- 172.21.15.1:0/2126011963 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38f41a62d0 con 0x7f38f4100ff0 2024-02-02T15:46:46.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.648+0000 7f38f8c00700 1 -- 172.21.15.1:0/2126011963 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38f4067bc0 con 0x7f38f4100ff0 2024-02-02T15:46:46.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.650+0000 7f38eaa00700 1 -- 172.21.15.1:0/2126011963 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f38e40105a0 con 0x7f38f4100ff0 2024-02-02T15:46:46.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.651+0000 7f38eaa00700 1 --2- 172.21.15.1:0/2126011963 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f38d40a0690 0x7f38d40a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:46.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.651+0000 7f38eaa00700 1 -- 172.21.15.1:0/2126011963 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f38e4014070 con 0x7f38f4100ff0 2024-02-02T15:46:46.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.652+0000 7f38f1400700 1 --2- 172.21.15.1:0/2126011963 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f38d40a0690 0x7f38d40a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:46.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.652+0000 7f38f1400700 1 --2- 172.21.15.1:0/2126011963 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f38d40a0690 0x7f38d40a2b50 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f38f4074830 tx=0x7f38dc009260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:46.652 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.653+0000 7f38eaa00700 1 -- 172.21.15.1:0/2126011963 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f38e4095930 con 0x7f38f4100ff0 2024-02-02T15:46:46.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.866+0000 7f38f8c00700 1 -- 172.21.15.1:0/2126011963 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f38f4101aa0 con 0x7f38f4100ff0 2024-02-02T15:46:46.866 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.867+0000 7f38eaa00700 1 -- 172.21.15.1:0/2126011963 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v115) v1 ==== 100+0+11 (secure 0 0 0) 0x7f38e40260a0 con 0x7f38f4100ff0 2024-02-02T15:46:46.866 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 64 2024-02-02T15:46:46.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.869+0000 7f38f8c00700 1 -- 172.21.15.1:0/2126011963 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f38d40a0690 msgr2=0x7f38d40a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:46.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.869+0000 7f38f8c00700 1 --2- 172.21.15.1:0/2126011963 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f38d40a0690 0x7f38d40a2b50 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f38f4074830 tx=0x7f38dc009260 comp rx=0 tx=0).stop 2024-02-02T15:46:46.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.869+0000 7f38f8c00700 1 -- 172.21.15.1:0/2126011963 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f38f4100ff0 msgr2=0x7f38f4073040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:46.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.869+0000 7f38f8c00700 1 --2- 172.21.15.1:0/2126011963 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f38f4100ff0 0x7f38f4073040 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f38e400e9a0 tx=0x7f38e400ecb0 comp rx=0 tx=0).stop 2024-02-02T15:46:46.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.869+0000 7f38f8c00700 1 -- 172.21.15.1:0/2126011963 shutdown_connections 2024-02-02T15:46:46.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.869+0000 7f38f8c00700 1 --2- 172.21.15.1:0/2126011963 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f38d40a0690 0x7f38d40a2b50 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:46.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.869+0000 7f38f8c00700 1 --2- 172.21.15.1:0/2126011963 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f38f4073ac0 0x7f38f41a51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:46.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.869+0000 7f38f8c00700 1 --2- 172.21.15.1:0/2126011963 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f38f4100ff0 0x7f38f4073040 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:46.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.869+0000 7f38f8c00700 1 --2- 172.21.15.1:0/2126011963 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f38f4106110 0x7f38f4073580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:46.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.869+0000 7f38f8c00700 1 -- 172.21.15.1:0/2126011963 >> 172.21.15.1:0/2126011963 conn(0x7f38f40fc810 msgr2=0x7f38f40fd260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:46.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.869+0000 7f38f8c00700 1 -- 172.21.15.1:0/2126011963 shutdown_connections 2024-02-02T15:46:46.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:46.869+0000 7f38f8c00700 1 -- 172.21.15.1:0/2126011963 wait complete. 2024-02-02T15:46:46.887 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-02-02T15:46:47.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.089+0000 7fda6d200700 1 -- 172.21.15.1:0/1992867375 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda681019a0 msgr2=0x7fda681059e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:47.088 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.089+0000 7fda6d200700 1 --2- 172.21.15.1:0/1992867375 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda681019a0 0x7fda681059e0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fda50009990 tx=0x7fda50009ca0 comp rx=0 tx=0).stop 2024-02-02T15:46:47.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.089+0000 7fda6d200700 1 -- 172.21.15.1:0/1992867375 shutdown_connections 2024-02-02T15:46:47.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.089+0000 7fda6d200700 1 --2- 172.21.15.1:0/1992867375 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda68106110 0x7fda6810dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:47.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.089+0000 7fda6d200700 1 --2- 172.21.15.1:0/1992867375 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fda68100ff0 0x7fda681013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:47.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.089+0000 7fda6d200700 1 --2- 172.21.15.1:0/1992867375 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda681019a0 0x7fda681059e0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:47.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.089+0000 7fda6d200700 1 -- 172.21.15.1:0/1992867375 >> 172.21.15.1:0/1992867375 conn(0x7fda680fc810 msgr2=0x7fda680fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:47.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.089+0000 7fda6d200700 1 -- 172.21.15.1:0/1992867375 shutdown_connections 2024-02-02T15:46:47.089 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.089+0000 7fda6d200700 1 -- 172.21.15.1:0/1992867375 wait complete. 2024-02-02T15:46:47.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.090+0000 7fda6d200700 1 Processor -- start 2024-02-02T15:46:47.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.090+0000 7fda6d200700 1 -- start start 2024-02-02T15:46:47.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.090+0000 7fda6d200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda68100ff0 0x7fda68199080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:47.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.090+0000 7fda6d200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda68106110 0x7fda681995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:47.092 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.091+0000 7fda6d200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fda6819da20 0x7fda6819dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:47.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.091+0000 7fda6d200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda6819e4a0 con 0x7fda68100ff0 2024-02-02T15:46:47.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.091+0000 7fda6d200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda6819e610 con 0x7fda6819da20 2024-02-02T15:46:47.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.091+0000 7fda6d200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda6819e7d0 con 0x7fda68106110 2024-02-02T15:46:47.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.091+0000 7fda65e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda68100ff0 0x7fda68199080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:47.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.091+0000 7fda65e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda68100ff0 0x7fda68199080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41784/0 (socket says 172.21.15.1:41784) 2024-02-02T15:46:47.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.091+0000 7fda65e00700 1 -- 172.21.15.1:0/907938665 learned_addr learned my addr 172.21.15.1:0/907938665 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:46:47.093 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.091+0000 7fda65400700 1 --2- 172.21.15.1:0/907938665 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda68106110 0x7fda681995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:47.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.091+0000 7fda66800700 1 --2- 172.21.15.1:0/907938665 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fda6819da20 0x7fda6819dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:47.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.091+0000 7fda65e00700 1 -- 172.21.15.1:0/907938665 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda68106110 msgr2=0x7fda681995c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:47.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.091+0000 7fda65e00700 1 --2- 172.21.15.1:0/907938665 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda68106110 0x7fda681995c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:47.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.091+0000 7fda65e00700 1 -- 172.21.15.1:0/907938665 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fda6819da20 msgr2=0x7fda6819dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:47.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.091+0000 7fda65e00700 1 --2- 172.21.15.1:0/907938665 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fda6819da20 0x7fda6819dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:47.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.091+0000 7fda65e00700 1 -- 172.21.15.1:0/907938665 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda500095f0 con 0x7fda68100ff0 2024-02-02T15:46:47.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.092+0000 7fda65e00700 1 --2- 172.21.15.1:0/907938665 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda68100ff0 0x7fda68199080 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fda5800e9a0 tx=0x7fda5800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:47.094 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.092+0000 7fda5ea00700 1 -- 172.21.15.1:0/907938665 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda58009790 con 0x7fda68100ff0 2024-02-02T15:46:47.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.092+0000 7fda5ea00700 1 -- 172.21.15.1:0/907938665 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda58010430 con 0x7fda68100ff0 2024-02-02T15:46:47.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.092+0000 7fda5ea00700 1 -- 172.21.15.1:0/907938665 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda58018680 con 0x7fda68100ff0 2024-02-02T15:46:47.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.092+0000 7fda6d200700 1 -- 172.21.15.1:0/907938665 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda681a5d30 con 0x7fda68100ff0 2024-02-02T15:46:47.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.092+0000 7fda6d200700 1 -- 172.21.15.1:0/907938665 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda681a61c0 con 0x7fda68100ff0 2024-02-02T15:46:47.095 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.093+0000 7fda6d200700 1 -- 172.21.15.1:0/907938665 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda68067bc0 con 0x7fda68100ff0 2024-02-02T15:46:47.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.095+0000 7fda5ea00700 1 -- 172.21.15.1:0/907938665 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7fda580105a0 con 0x7fda68100ff0 2024-02-02T15:46:47.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.096+0000 7fda5ea00700 1 --2- 172.21.15.1:0/907938665 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fda480a0690 0x7fda480a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:47.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.096+0000 7fda5ea00700 1 -- 172.21.15.1:0/907938665 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 9453+0+0 (secure 0 0 0) 0x7fda58014070 con 0x7fda68100ff0 2024-02-02T15:46:47.096 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.096+0000 7fda65400700 1 --2- 172.21.15.1:0/907938665 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fda480a0690 0x7fda480a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:47.097 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.097+0000 7fda65400700 1 --2- 172.21.15.1:0/907938665 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fda480a0690 0x7fda480a2b50 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fda5000b2e0 tx=0x7fda50005960 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:47.099 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.100+0000 7fda5ea00700 1 -- 172.21.15.1:0/907938665 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7fda58052110 con 0x7fda68100ff0 2024-02-02T15:46:47.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:46:46 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/2298468716' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-02T15:46:47.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:46:46 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/3348966536' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-02T15:46:47.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:46:46 smithi071 ceph-mon[44707]: pgmap v622: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 76 KiB/s wr, 20 op/s 2024-02-02T15:46:47.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:46:46 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/2298468716' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-02T15:46:47.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:46:46 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/3348966536' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-02T15:46:47.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:46:46 smithi142 ceph-mon[45728]: pgmap v622: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 76 KiB/s wr, 20 op/s 2024-02-02T15:46:47.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:46:46 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/2298468716' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-02T15:46:47.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:46:46 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/3348966536' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-02T15:46:47.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:46:46 smithi001 ceph-mon[35470]: pgmap v622: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 76 KiB/s wr, 20 op/s 2024-02-02T15:46:47.311 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.312+0000 7fda6d200700 1 -- 172.21.15.1:0/907938665 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fda68101aa0 con 0x7fda68100ff0 2024-02-02T15:46:47.312 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.313+0000 7fda5ea00700 1 -- 172.21.15.1:0/907938665 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v115) v1 ==== 103+0+11 (secure 0 0 0) 0x7fda58095790 con 0x7fda68100ff0 2024-02-02T15:46:47.312 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-02-02T15:46:47.314 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.315+0000 7fda6d200700 1 -- 172.21.15.1:0/907938665 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fda480a0690 msgr2=0x7fda480a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:47.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.315+0000 7fda6d200700 1 --2- 172.21.15.1:0/907938665 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fda480a0690 0x7fda480a2b50 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fda5000b2e0 tx=0x7fda50005960 comp rx=0 tx=0).stop 2024-02-02T15:46:47.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.315+0000 7fda6d200700 1 -- 172.21.15.1:0/907938665 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda68100ff0 msgr2=0x7fda68199080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:47.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.315+0000 7fda6d200700 1 --2- 172.21.15.1:0/907938665 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda68100ff0 0x7fda68199080 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fda5800e9a0 tx=0x7fda5800ecb0 comp rx=0 tx=0).stop 2024-02-02T15:46:47.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.315+0000 7fda6d200700 1 -- 172.21.15.1:0/907938665 shutdown_connections 2024-02-02T15:46:47.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.315+0000 7fda6d200700 1 --2- 172.21.15.1:0/907938665 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fda480a0690 0x7fda480a2b50 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:47.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.315+0000 7fda6d200700 1 --2- 172.21.15.1:0/907938665 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fda6819da20 0x7fda6819dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:47.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.316+0000 7fda6d200700 1 --2- 172.21.15.1:0/907938665 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fda68100ff0 0x7fda68199080 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:47.315 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.316+0000 7fda6d200700 1 --2- 172.21.15.1:0/907938665 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fda68106110 0x7fda681995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:47.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.316+0000 7fda6d200700 1 -- 172.21.15.1:0/907938665 >> 172.21.15.1:0/907938665 conn(0x7fda680fc810 msgr2=0x7fda680fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:47.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.316+0000 7fda6d200700 1 -- 172.21.15.1:0/907938665 shutdown_connections 2024-02-02T15:46:47.316 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.316+0000 7fda6d200700 1 -- 172.21.15.1:0/907938665 wait complete. 2024-02-02T15:46:47.334 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-02-02T15:46:47.533 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.534+0000 7f4f6b000700 1 -- 172.21.15.1:0/895724657 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f641018c0 msgr2=0x7f4f64105900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:47.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.534+0000 7f4f6b000700 1 --2- 172.21.15.1:0/895724657 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f641018c0 0x7f4f64105900 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c009990 tx=0x7f4f4c009ca0 comp rx=0 tx=0).stop 2024-02-02T15:46:47.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.534+0000 7f4f6b000700 1 -- 172.21.15.1:0/895724657 shutdown_connections 2024-02-02T15:46:47.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.534+0000 7f4f6b000700 1 --2- 172.21.15.1:0/895724657 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f64106030 0x7f4f6410dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:47.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.534+0000 7f4f6b000700 1 --2- 172.21.15.1:0/895724657 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f64100f10 0x7f4f641012f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:47.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.534+0000 7f4f6b000700 1 --2- 172.21.15.1:0/895724657 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f641018c0 0x7f4f64105900 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:47.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.534+0000 7f4f6b000700 1 -- 172.21.15.1:0/895724657 >> 172.21.15.1:0/895724657 conn(0x7f4f640fc790 msgr2=0x7f4f640febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:47.534 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.534+0000 7f4f6b000700 1 -- 172.21.15.1:0/895724657 shutdown_connections 2024-02-02T15:46:47.535 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.534+0000 7f4f6b000700 1 -- 172.21.15.1:0/895724657 wait complete. 2024-02-02T15:46:47.537 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.535+0000 7f4f6b000700 1 Processor -- start 2024-02-02T15:46:47.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.536+0000 7f4f6b000700 1 -- start start 2024-02-02T15:46:47.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.536+0000 7f4f6b000700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f64100f10 0x7f4f64073040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:47.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.536+0000 7f4f6b000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f64106030 0x7f4f64073580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:47.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.536+0000 7f4f6b000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f64073ac0 0x7f4f641a5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:47.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.536+0000 7f4f6b000700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f641a5710 con 0x7f4f64073ac0 2024-02-02T15:46:47.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.536+0000 7f4f6b000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f641a5880 con 0x7f4f64106030 2024-02-02T15:46:47.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.536+0000 7f4f6b000700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f641a5a40 con 0x7f4f64100f10 2024-02-02T15:46:47.538 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.536+0000 7f4f69000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f64073ac0 0x7f4f641a5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:47.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.536+0000 7f4f69000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f64073ac0 0x7f4f641a5080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:41796/0 (socket says 172.21.15.1:41796) 2024-02-02T15:46:47.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.536+0000 7f4f69000700 1 -- 172.21.15.1:0/3305643017 learned_addr learned my addr 172.21.15.1:0/3305643017 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:46:47.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.536+0000 7f4f63400700 1 --2- 172.21.15.1:0/3305643017 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f64106030 0x7f4f64073580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:47.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.536+0000 7f4f63e00700 1 --2- 172.21.15.1:0/3305643017 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f64100f10 0x7f4f64073040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:47.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.537+0000 7f4f69000700 1 -- 172.21.15.1:0/3305643017 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f64100f10 msgr2=0x7f4f64073040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:47.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.537+0000 7f4f69000700 1 --2- 172.21.15.1:0/3305643017 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f64100f10 0x7f4f64073040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:47.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.537+0000 7f4f69000700 1 -- 172.21.15.1:0/3305643017 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f64106030 msgr2=0x7f4f64073580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:47.539 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.537+0000 7f4f69000700 1 --2- 172.21.15.1:0/3305643017 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f64106030 0x7f4f64073580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:47.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.537+0000 7f4f69000700 1 -- 172.21.15.1:0/3305643017 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f4c0095f0 con 0x7f4f64073ac0 2024-02-02T15:46:47.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.537+0000 7f4f69000700 1 --2- 172.21.15.1:0/3305643017 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f64073ac0 0x7f4f641a5080 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c00ef30 tx=0x7f4f5c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:47.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.537+0000 7f4f60c00700 1 -- 172.21.15.1:0/3305643017 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f5c018070 con 0x7f4f64073ac0 2024-02-02T15:46:47.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.537+0000 7f4f60c00700 1 -- 172.21.15.1:0/3305643017 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f5c00f040 con 0x7f4f64073ac0 2024-02-02T15:46:47.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.537+0000 7f4f60c00700 1 -- 172.21.15.1:0/3305643017 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f5c00cd00 con 0x7f4f64073ac0 2024-02-02T15:46:47.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.537+0000 7f4f6b000700 1 -- 172.21.15.1:0/3305643017 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f641a5cc0 con 0x7f4f64073ac0 2024-02-02T15:46:47.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.537+0000 7f4f6b000700 1 -- 172.21.15.1:0/3305643017 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f641a6180 con 0x7f4f64073ac0 2024-02-02T15:46:47.540 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.538+0000 7f4f6b000700 1 -- 172.21.15.1:0/3305643017 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f64067bc0 con 0x7f4f64073ac0 2024-02-02T15:46:47.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.540+0000 7f4f60c00700 1 -- 172.21.15.1:0/3305643017 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f4f5c003a00 con 0x7f4f64073ac0 2024-02-02T15:46:47.541 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.541+0000 7f4f60c00700 1 --2- 172.21.15.1:0/3305643017 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f4f440a0690 0x7f4f440a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:46:47.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.542+0000 7f4f60c00700 1 -- 172.21.15.1:0/3305643017 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f4f5c0cbbf0 con 0x7f4f64073ac0 2024-02-02T15:46:47.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.542+0000 7f4f63e00700 1 --2- 172.21.15.1:0/3305643017 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f4f440a0690 0x7f4f440a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:46:47.542 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.542+0000 7f4f63e00700 1 --2- 172.21.15.1:0/3305643017 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f4f440a0690 0x7f4f440a2b50 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f4f54009c60 tx=0x7f4f54009260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:46:47.544 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.545+0000 7f4f60c00700 1 -- 172.21.15.1:0/3305643017 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f4f5c0950e0 con 0x7f4f64073ac0 2024-02-02T15:46:47.817 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.818+0000 7f4f6b000700 1 -- 172.21.15.1:0/3305643017 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7f4f641019c0 con 0x7f4f64073ac0 2024-02-02T15:46:47.828 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.830+0000 7f4f60c00700 1 -- 172.21.15.1:0/3305643017 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v28) v1 ==== 81+0+730224 (secure 0 0 0) 0x7f4f5c098700 con 0x7f4f64073ac0 2024-02-02T15:46:47.839 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:46:47.839 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:46:47.840 INFO:teuthology.orchestra.run.smithi001.stdout: "epoch": 28, 2024-02-02T15:46:47.840 INFO:teuthology.orchestra.run.smithi001.stdout: "active_gid": 25603, 2024-02-02T15:46:47.840 INFO:teuthology.orchestra.run.smithi001.stdout: "active_name": "x", 2024-02-02T15:46:47.840 INFO:teuthology.orchestra.run.smithi001.stdout: "active_addrs": { 2024-02-02T15:46:47.840 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-02-02T15:46:47.840 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:47.840 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-02-02T15:46:47.840 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6800", 2024-02-02T15:46:47.841 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 167617173 2024-02-02T15:46:47.841 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.841 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:47.841 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v1", 2024-02-02T15:46:47.841 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:6801", 2024-02-02T15:46:47.841 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 167617173 2024-02-02T15:46:47.841 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:47.841 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-02-02T15:46:47.842 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.842 INFO:teuthology.orchestra.run.smithi001.stdout: "active_addr": "172.21.15.1:6801/167617173", 2024-02-02T15:46:47.842 INFO:teuthology.orchestra.run.smithi001.stdout: "active_change": "2024-02-02T15:27:00.115304+0000", 2024-02-02T15:46:47.842 INFO:teuthology.orchestra.run.smithi001.stdout: "active_mgr_features": 4540138322906710015, 2024-02-02T15:46:47.842 INFO:teuthology.orchestra.run.smithi001.stdout: "available": true, 2024-02-02T15:46:47.842 INFO:teuthology.orchestra.run.smithi001.stdout: "standbys": [ 2024-02-02T15:46:47.842 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:47.842 INFO:teuthology.orchestra.run.smithi001.stdout: "gid": 25595, 2024-02-02T15:46:47.842 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "y", 2024-02-02T15:46:47.843 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_features": 4540138322906710015, 2024-02-02T15:46:47.843 INFO:teuthology.orchestra.run.smithi001.stdout: "available_modules": [ 2024-02-02T15:46:47.843 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:47.843 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "alerts", 2024-02-02T15:46:47.843 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:47.843 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:47.843 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:47.843 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-02-02T15:46:47.844 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-02-02T15:46:47.844 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-02-02T15:46:47.844 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.844 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:47.844 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-02-02T15:46:47.844 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:47.844 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:47.844 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:47.845 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "How frequently to reexamine health status", 2024-02-02T15:46:47.845 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:47.845 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.845 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.845 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.845 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:47.845 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:47.845 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:47.846 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.846 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:47.846 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:47.846 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:47.846 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:47.846 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:47.846 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:47.846 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:47.846 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:47.847 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:47.847 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:47.847 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:47.847 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:47.847 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:47.847 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:47.847 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.847 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.848 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.848 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:47.848 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:47.848 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:47.848 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.848 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:47.848 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:47.848 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:47.848 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:47.849 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:47.849 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:47.849 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:47.849 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.849 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.849 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.849 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:47.849 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:47.849 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:47.850 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.850 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:47.850 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:47.850 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:47.850 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:47.850 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:47.850 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:47.850 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:47.851 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:47.851 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:47.851 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:47.851 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:47.851 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:47.851 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:47.851 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:47.851 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.851 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.852 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.852 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:47.852 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:47.852 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:47.852 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.852 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:47.852 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:47.852 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:47.853 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:47.853 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:47.853 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:47.853 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:47.853 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.853 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.853 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.853 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_destination": { 2024-02-02T15:46:47.853 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_destination", 2024-02-02T15:46:47.854 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:47.854 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.854 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:47.854 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:47.854 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:47.854 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:47.855 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:47.855 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Email address to send alerts to", 2024-02-02T15:46:47.855 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:47.855 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.855 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.855 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.855 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_from_name": { 2024-02-02T15:46:47.855 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_from_name", 2024-02-02T15:46:47.856 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:47.856 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.856 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:47.856 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "Ceph", 2024-02-02T15:46:47.856 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:47.856 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:47.856 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:47.856 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Email From: name", 2024-02-02T15:46:47.857 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:47.857 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.857 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.857 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.857 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_host": { 2024-02-02T15:46:47.857 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_host", 2024-02-02T15:46:47.857 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:47.857 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.858 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:47.858 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:47.858 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:47.858 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:47.858 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:47.858 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "SMTP server", 2024-02-02T15:46:47.858 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:47.858 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.858 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.859 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.859 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_password": { 2024-02-02T15:46:47.859 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_password", 2024-02-02T15:46:47.859 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:47.859 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.859 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:47.859 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:47.859 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:47.860 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:47.860 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:47.860 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Password to authenticate with", 2024-02-02T15:46:47.860 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:47.860 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.860 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.860 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.860 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_port": { 2024-02-02T15:46:47.860 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_port", 2024-02-02T15:46:47.861 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:47.861 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.861 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:47.861 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "465", 2024-02-02T15:46:47.861 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:47.861 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:47.861 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:47.862 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "SMTP port", 2024-02-02T15:46:47.862 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:47.862 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.862 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.862 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.862 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_sender": { 2024-02-02T15:46:47.862 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_sender", 2024-02-02T15:46:47.862 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:47.863 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.863 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:47.863 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:47.863 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:47.863 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:47.863 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:47.863 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "SMTP envelope sender", 2024-02-02T15:46:47.863 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:47.863 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.864 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.864 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.864 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_ssl": { 2024-02-02T15:46:47.864 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_ssl", 2024-02-02T15:46:47.864 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:47.864 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.865 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:47.865 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:47.865 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:47.865 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:47.865 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:47.865 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Use SSL to connect to SMTP server", 2024-02-02T15:46:47.865 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:47.865 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.865 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.866 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.866 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_user": { 2024-02-02T15:46:47.866 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_user", 2024-02-02T15:46:47.866 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:47.866 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.866 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:47.866 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:47.866 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:47.867 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:47.867 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:47.867 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "User to authenticate as", 2024-02-02T15:46:47.867 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:47.867 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.867 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.867 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:47.867 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:47.868 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.868 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:47.868 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "balancer", 2024-02-02T15:46:47.868 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:47.868 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:47.868 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:47.868 INFO:teuthology.orchestra.run.smithi001.stdout: "active": { 2024-02-02T15:46:47.868 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "active", 2024-02-02T15:46:47.869 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:47.869 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.869 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:47.869 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:47.869 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:47.869 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:47.869 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:47.869 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "automatically balance PGs across cluster", 2024-02-02T15:46:47.869 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:47.870 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.870 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.870 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.870 INFO:teuthology.orchestra.run.smithi001.stdout: "begin_time": { 2024-02-02T15:46:47.870 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "begin_time", 2024-02-02T15:46:47.870 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:47.870 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.870 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:47.871 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0000", 2024-02-02T15:46:47.871 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:47.871 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:47.871 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:47.871 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "beginning time of day to automatically balance", 2024-02-02T15:46:47.871 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-02-02T15:46:47.871 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.871 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.871 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.872 INFO:teuthology.orchestra.run.smithi001.stdout: "begin_weekday": { 2024-02-02T15:46:47.872 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "begin_weekday", 2024-02-02T15:46:47.872 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-02-02T15:46:47.872 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.872 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:47.872 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-02-02T15:46:47.872 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "0", 2024-02-02T15:46:47.872 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "6", 2024-02-02T15:46:47.873 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:47.873 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-02-02T15:46:47.873 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-02-02T15:46:47.873 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.873 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.873 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.873 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_compat_max_iterations": { 2024-02-02T15:46:47.873 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crush_compat_max_iterations", 2024-02-02T15:46:47.873 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-02-02T15:46:47.873 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.874 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:47.874 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "25", 2024-02-02T15:46:47.874 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1", 2024-02-02T15:46:47.874 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "250", 2024-02-02T15:46:47.874 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:47.874 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-02-02T15:46:47.874 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:47.874 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.874 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.875 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.875 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_compat_metrics": { 2024-02-02T15:46:47.875 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crush_compat_metrics", 2024-02-02T15:46:47.875 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:47.875 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.875 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:47.875 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "pgs,objects,bytes", 2024-02-02T15:46:47.875 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:47.875 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:47.876 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:47.876 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-02-02T15:46:47.876 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-02-02T15:46:47.876 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.876 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.876 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.876 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_compat_step": { 2024-02-02T15:46:47.876 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crush_compat_step", 2024-02-02T15:46:47.876 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-02-02T15:46:47.877 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.877 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:47.877 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0.5", 2024-02-02T15:46:47.877 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "0.001", 2024-02-02T15:46:47.877 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "0.999", 2024-02-02T15:46:47.877 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:47.877 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "aggressiveness of optimization", 2024-02-02T15:46:47.877 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-02-02T15:46:47.877 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.878 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.878 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.878 INFO:teuthology.orchestra.run.smithi001.stdout: "end_time": { 2024-02-02T15:46:47.878 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "end_time", 2024-02-02T15:46:47.878 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:47.878 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.878 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:47.878 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "2359", 2024-02-02T15:46:47.878 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:47.878 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:47.879 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:47.879 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "ending time of day to automatically balance", 2024-02-02T15:46:47.879 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-02-02T15:46:47.879 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.879 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.879 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.879 INFO:teuthology.orchestra.run.smithi001.stdout: "end_weekday": { 2024-02-02T15:46:47.879 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "end_weekday", 2024-02-02T15:46:47.879 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-02-02T15:46:47.880 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.880 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:47.880 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-02-02T15:46:47.880 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "0", 2024-02-02T15:46:47.880 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "6", 2024-02-02T15:46:47.880 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:47.880 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-02-02T15:46:47.880 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-02-02T15:46:47.880 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.881 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.881 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.881 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:47.881 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:47.881 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:47.881 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.881 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:47.881 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:47.881 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:47.882 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:47.882 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:47.882 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:47.882 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:47.882 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:47.882 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:47.882 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:47.882 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:47.883 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:47.883 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:47.883 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:47.883 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.883 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.883 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.883 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:47.883 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:47.884 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:47.884 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.884 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:47.884 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:47.884 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:47.885 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:47.885 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:47.885 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:47.885 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:47.885 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.885 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.885 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.885 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:47.886 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:47.886 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:47.886 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.886 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:47.886 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:47.886 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:47.886 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:47.886 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:47.887 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:47.887 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:47.887 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:47.887 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:47.887 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:47.887 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:47.887 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:47.887 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:47.888 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:47.888 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.888 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.888 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.888 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:47.888 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:47.888 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:47.888 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.888 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:47.889 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:47.889 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:47.889 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:47.889 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:47.889 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:47.889 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:47.889 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.889 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.890 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.890 INFO:teuthology.orchestra.run.smithi001.stdout: "min_score": { 2024-02-02T15:46:47.890 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "min_score", 2024-02-02T15:46:47.890 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-02-02T15:46:47.890 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.890 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:47.890 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-02-02T15:46:47.890 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:47.891 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:47.891 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:47.891 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-02-02T15:46:47.891 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:47.891 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.891 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.891 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.891 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": { 2024-02-02T15:46:47.892 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mode", 2024-02-02T15:46:47.892 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:47.892 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.892 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:47.892 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "upmap", 2024-02-02T15:46:47.892 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:47.892 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:47.892 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:47.892 INFO:teuthology.orchestra.run.smithi001.stdout: "crush-compat", 2024-02-02T15:46:47.892 INFO:teuthology.orchestra.run.smithi001.stdout: "none", 2024-02-02T15:46:47.893 INFO:teuthology.orchestra.run.smithi001.stdout: "upmap" 2024-02-02T15:46:47.893 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:47.893 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Balancer mode", 2024-02-02T15:46:47.893 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:47.893 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.893 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.893 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.893 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_ids": { 2024-02-02T15:46:47.894 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "pool_ids", 2024-02-02T15:46:47.894 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:47.894 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.894 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:47.894 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:47.894 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:47.894 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:47.894 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:47.895 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-02-02T15:46:47.895 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:47.895 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.895 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.895 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.895 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-02-02T15:46:47.895 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-02-02T15:46:47.895 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-02-02T15:46:47.895 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.896 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:47.896 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-02-02T15:46:47.896 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:47.896 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:47.896 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:47.896 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-02-02T15:46:47.896 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:47.897 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.897 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.897 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.897 INFO:teuthology.orchestra.run.smithi001.stdout: "upmap_max_deviation": { 2024-02-02T15:46:47.897 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "upmap_max_deviation", 2024-02-02T15:46:47.897 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:47.897 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.897 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:47.898 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2024-02-02T15:46:47.898 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1", 2024-02-02T15:46:47.898 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:47.898 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:47.898 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "deviation below which no optimization is attempted", 2024-02-02T15:46:47.898 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-02-02T15:46:47.898 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.898 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.899 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.899 INFO:teuthology.orchestra.run.smithi001.stdout: "upmap_max_optimizations": { 2024-02-02T15:46:47.899 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "upmap_max_optimizations", 2024-02-02T15:46:47.899 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-02-02T15:46:47.899 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.899 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:47.899 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-02-02T15:46:47.899 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:47.899 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:47.900 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:47.900 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-02-02T15:46:47.900 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:47.900 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.900 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.900 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:47.900 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:47.900 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.900 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:47.901 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cephadm", 2024-02-02T15:46:47.901 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:47.901 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:47.901 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:47.901 INFO:teuthology.orchestra.run.smithi001.stdout: "agent_down_multiplier": { 2024-02-02T15:46:47.901 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "agent_down_multiplier", 2024-02-02T15:46:47.901 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-02-02T15:46:47.901 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.902 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:47.902 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "3.0", 2024-02-02T15:46:47.902 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:47.902 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:47.902 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:47.902 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-02-02T15:46:47.902 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:47.902 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.902 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.903 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.903 INFO:teuthology.orchestra.run.smithi001.stdout: "agent_refresh_rate": { 2024-02-02T15:46:47.903 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "agent_refresh_rate", 2024-02-02T15:46:47.903 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-02-02T15:46:47.903 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.903 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:47.903 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "20", 2024-02-02T15:46:47.903 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:47.903 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:47.904 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:47.904 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-02-02T15:46:47.904 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:47.904 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.904 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.904 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.904 INFO:teuthology.orchestra.run.smithi001.stdout: "agent_starting_port": { 2024-02-02T15:46:47.904 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "agent_starting_port", 2024-02-02T15:46:47.905 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:47.905 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.905 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:47.905 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "4721", 2024-02-02T15:46:47.905 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:47.905 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:47.905 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:47.905 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-02-02T15:46:47.905 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:47.905 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.906 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.906 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.906 INFO:teuthology.orchestra.run.smithi001.stdout: "allow_ptrace": { 2024-02-02T15:46:47.906 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "allow_ptrace", 2024-02-02T15:46:47.906 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:47.906 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.906 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:47.906 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:47.906 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:47.907 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:47.907 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:47.907 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-02-02T15:46:47.907 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-02-02T15:46:47.907 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.907 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.907 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.907 INFO:teuthology.orchestra.run.smithi001.stdout: "autotune_interval": { 2024-02-02T15:46:47.907 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "autotune_interval", 2024-02-02T15:46:47.907 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-02-02T15:46:47.908 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.908 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:47.908 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-02-02T15:46:47.908 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:47.908 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:47.908 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:47.908 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to autotune daemon memory", 2024-02-02T15:46:47.908 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:47.908 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.909 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.909 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.909 INFO:teuthology.orchestra.run.smithi001.stdout: "autotune_memory_target_ratio": { 2024-02-02T15:46:47.909 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "autotune_memory_target_ratio", 2024-02-02T15:46:47.909 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-02-02T15:46:47.909 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.909 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:47.909 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0.7", 2024-02-02T15:46:47.909 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:47.910 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:47.910 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:47.910 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-02-02T15:46:47.910 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:47.910 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.910 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.910 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.910 INFO:teuthology.orchestra.run.smithi001.stdout: "cgroups_split": { 2024-02-02T15:46:47.911 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cgroups_split", 2024-02-02T15:46:47.911 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:47.911 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.911 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:47.911 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:47.911 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:47.911 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:47.911 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:47.912 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-02-02T15:46:47.912 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:47.912 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.912 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.912 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.912 INFO:teuthology.orchestra.run.smithi001.stdout: "config_checks_enabled": { 2024-02-02T15:46:47.912 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "config_checks_enabled", 2024-02-02T15:46:47.913 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:47.913 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.913 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:47.913 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:47.913 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:47.913 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:47.913 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:47.914 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-02-02T15:46:47.914 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:47.914 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.914 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.914 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.914 INFO:teuthology.orchestra.run.smithi001.stdout: "config_dashboard": { 2024-02-02T15:46:47.914 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "config_dashboard", 2024-02-02T15:46:47.914 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:47.915 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.915 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:47.915 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:47.915 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:47.915 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:47.915 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:47.915 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-02-02T15:46:47.915 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:47.916 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.916 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.916 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.916 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_alertmanager": { 2024-02-02T15:46:47.916 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_alertmanager", 2024-02-02T15:46:47.916 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:47.916 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.916 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:47.917 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-02-02T15:46:47.917 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:47.917 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:47.917 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:47.917 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus container image", 2024-02-02T15:46:47.917 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:47.917 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.917 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.917 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.918 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_base": { 2024-02-02T15:46:47.918 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_base", 2024-02-02T15:46:47.918 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:47.918 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.918 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:47.918 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/ceph", 2024-02-02T15:46:47.918 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:47.918 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:47.918 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:47.919 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Container image name, without the tag", 2024-02-02T15:46:47.919 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:47.919 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.919 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.919 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.919 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_elasticsearch": { 2024-02-02T15:46:47.919 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_elasticsearch", 2024-02-02T15:46:47.920 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:47.920 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.920 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:47.920 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-02-02T15:46:47.920 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:47.920 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:47.920 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:47.920 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "elasticsearch container image", 2024-02-02T15:46:47.921 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:47.921 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.921 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.921 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.921 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_grafana": { 2024-02-02T15:46:47.921 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_grafana", 2024-02-02T15:46:47.921 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:47.921 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.921 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:47.922 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2024-02-02T15:46:47.922 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:47.922 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:47.922 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:47.922 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus container image", 2024-02-02T15:46:47.922 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:47.922 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.922 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.922 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.923 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_haproxy": { 2024-02-02T15:46:47.923 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_haproxy", 2024-02-02T15:46:47.923 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:47.923 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.923 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:47.923 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-02-02T15:46:47.923 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:47.923 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:47.924 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:47.924 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "HAproxy container image", 2024-02-02T15:46:47.924 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:47.924 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.924 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.924 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.924 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_jaeger_agent": { 2024-02-02T15:46:47.924 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_jaeger_agent", 2024-02-02T15:46:47.924 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:47.925 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.925 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:47.925 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-02-02T15:46:47.925 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:47.925 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:47.925 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:47.925 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Jaeger agent container image", 2024-02-02T15:46:47.925 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:47.926 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.926 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.926 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.926 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_jaeger_collector": { 2024-02-02T15:46:47.926 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_jaeger_collector", 2024-02-02T15:46:47.926 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:47.926 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.926 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:47.927 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-02-02T15:46:47.927 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:47.927 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:47.927 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:47.927 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Jaeger collector container image", 2024-02-02T15:46:47.927 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:47.927 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.928 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.928 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.928 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_jaeger_query": { 2024-02-02T15:46:47.928 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_jaeger_query", 2024-02-02T15:46:47.928 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:47.928 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.928 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:47.928 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-02-02T15:46:47.929 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:47.929 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:47.929 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:47.929 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Jaeger query container image", 2024-02-02T15:46:47.929 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:47.929 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.929 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.929 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.930 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_keepalived": { 2024-02-02T15:46:47.930 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_keepalived", 2024-02-02T15:46:47.930 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:47.930 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.930 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:47.930 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-02-02T15:46:47.930 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:47.930 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:47.930 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:47.931 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Keepalived container image", 2024-02-02T15:46:47.931 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:47.931 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.931 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.931 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.931 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_loki": { 2024-02-02T15:46:47.931 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_loki", 2024-02-02T15:46:47.932 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:47.932 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.932 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:47.932 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-02-02T15:46:47.932 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:47.932 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:47.932 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:47.933 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Loki container image", 2024-02-02T15:46:47.933 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:47.933 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.933 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.933 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.933 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_node_exporter": { 2024-02-02T15:46:47.933 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_node_exporter", 2024-02-02T15:46:47.933 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:47.933 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.934 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:47.934 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-02-02T15:46:47.934 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:47.934 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:47.934 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:47.934 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus container image", 2024-02-02T15:46:47.934 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:47.934 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.934 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.934 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.935 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_nvmeof": { 2024-02-02T15:46:47.935 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_nvmeof", 2024-02-02T15:46:47.935 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:47.935 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.935 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:47.935 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/nvmeof:0.0.2", 2024-02-02T15:46:47.935 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:47.935 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:47.935 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:47.936 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Nvme-of container image", 2024-02-02T15:46:47.936 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:47.936 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.936 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.936 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.936 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_prometheus": { 2024-02-02T15:46:47.936 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_prometheus", 2024-02-02T15:46:47.936 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:47.936 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.937 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:47.937 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-02-02T15:46:47.937 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:47.937 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:47.937 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:47.937 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus container image", 2024-02-02T15:46:47.937 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:47.937 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.937 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.937 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.938 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_promtail": { 2024-02-02T15:46:47.938 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_promtail", 2024-02-02T15:46:47.938 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:47.938 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.938 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:47.938 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-02-02T15:46:47.938 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:47.938 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:47.938 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:47.938 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Promtail container image", 2024-02-02T15:46:47.939 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:47.939 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.939 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.939 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.939 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_snmp_gateway": { 2024-02-02T15:46:47.939 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_snmp_gateway", 2024-02-02T15:46:47.939 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:47.939 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.939 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:47.939 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-02-02T15:46:47.940 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:47.940 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:47.940 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:47.940 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "SNMP Gateway container image", 2024-02-02T15:46:47.940 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:47.940 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.940 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.940 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.940 INFO:teuthology.orchestra.run.smithi001.stdout: "container_init": { 2024-02-02T15:46:47.941 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_init", 2024-02-02T15:46:47.941 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:47.941 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.941 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:47.941 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:47.941 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:47.941 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:47.941 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:47.941 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Run podman/docker with `--init`", 2024-02-02T15:46:47.941 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:47.942 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.942 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.942 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.942 INFO:teuthology.orchestra.run.smithi001.stdout: "daemon_cache_timeout": { 2024-02-02T15:46:47.942 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "daemon_cache_timeout", 2024-02-02T15:46:47.942 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-02-02T15:46:47.942 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.942 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:47.942 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-02-02T15:46:47.942 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:47.943 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:47.943 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:47.943 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-02-02T15:46:47.943 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:47.943 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.943 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.943 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.943 INFO:teuthology.orchestra.run.smithi001.stdout: "default_cephadm_command_timeout": { 2024-02-02T15:46:47.943 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "default_cephadm_command_timeout", 2024-02-02T15:46:47.943 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-02-02T15:46:47.944 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.944 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:47.944 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "900", 2024-02-02T15:46:47.944 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:47.944 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:47.944 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:47.944 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-02-02T15:46:47.944 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:47.945 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.945 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.945 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.945 INFO:teuthology.orchestra.run.smithi001.stdout: "default_registry": { 2024-02-02T15:46:47.945 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "default_registry", 2024-02-02T15:46:47.945 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:47.945 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.946 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:47.946 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "docker.io", 2024-02-02T15:46:47.946 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:47.946 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:47.946 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:47.946 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-02-02T15:46:47.946 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:47.946 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.946 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.947 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.947 INFO:teuthology.orchestra.run.smithi001.stdout: "device_cache_timeout": { 2024-02-02T15:46:47.947 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "device_cache_timeout", 2024-02-02T15:46:47.947 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-02-02T15:46:47.947 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.947 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:47.947 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "1800", 2024-02-02T15:46:47.947 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:47.948 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:47.948 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:47.948 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "seconds to cache device inventory", 2024-02-02T15:46:47.948 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:47.948 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.948 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.948 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.949 INFO:teuthology.orchestra.run.smithi001.stdout: "device_enhanced_scan": { 2024-02-02T15:46:47.949 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "device_enhanced_scan", 2024-02-02T15:46:47.949 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:47.949 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.949 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:47.949 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:47.949 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:47.949 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:47.949 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:47.950 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Use libstoragemgmt during device scans", 2024-02-02T15:46:47.950 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:47.950 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.950 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.950 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.950 INFO:teuthology.orchestra.run.smithi001.stdout: "facts_cache_timeout": { 2024-02-02T15:46:47.950 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "facts_cache_timeout", 2024-02-02T15:46:47.951 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-02-02T15:46:47.951 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.951 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:47.951 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-02-02T15:46:47.951 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:47.951 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:47.951 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:47.952 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "seconds to cache host facts data", 2024-02-02T15:46:47.952 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:47.952 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.952 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.952 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.952 INFO:teuthology.orchestra.run.smithi001.stdout: "host_check_interval": { 2024-02-02T15:46:47.952 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "host_check_interval", 2024-02-02T15:46:47.952 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-02-02T15:46:47.953 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.953 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:47.953 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-02-02T15:46:47.953 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:47.953 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:47.953 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:47.953 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to perform a host check", 2024-02-02T15:46:47.953 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:47.954 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.954 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.954 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.954 INFO:teuthology.orchestra.run.smithi001.stdout: "inventory_list_all": { 2024-02-02T15:46:47.954 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "inventory_list_all", 2024-02-02T15:46:47.954 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:47.954 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.954 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:47.955 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:47.955 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:47.955 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:47.955 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:47.955 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-02-02T15:46:47.955 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:47.955 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.955 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.956 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.956 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:47.956 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:47.956 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:47.956 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.956 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:47.956 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:47.956 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:47.957 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:47.957 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:47.957 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:47.957 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:47.957 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:47.957 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:47.957 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:47.958 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:47.958 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:47.958 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:47.958 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:47.958 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.958 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.958 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.958 INFO:teuthology.orchestra.run.smithi001.stdout: "log_refresh_metadata": { 2024-02-02T15:46:47.959 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_refresh_metadata", 2024-02-02T15:46:47.959 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:47.959 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.959 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:47.959 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:47.959 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:47.959 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:47.959 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:47.960 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-02-02T15:46:47.960 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:47.960 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.960 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.960 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.960 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:47.960 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:47.960 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:47.961 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.961 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:47.961 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:47.961 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:47.961 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:47.961 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:47.961 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-02-02T15:46:47.961 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:47.962 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.962 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.962 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.962 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:47.962 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:47.962 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:47.962 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.962 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:47.963 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:47.963 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:47.963 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:47.963 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:47.963 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:47.963 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:47.963 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:47.963 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:47.964 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:47.964 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:47.964 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:47.964 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:47.964 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:47.964 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.964 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.965 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.965 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:47.965 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:47.965 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:47.965 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.965 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:47.965 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:47.965 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:47.966 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:47.966 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:47.966 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:47.966 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:47.966 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.966 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.966 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.966 INFO:teuthology.orchestra.run.smithi001.stdout: "manage_etc_ceph_ceph_conf": { 2024-02-02T15:46:47.966 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-02-02T15:46:47.967 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:47.967 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.967 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:47.967 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:47.967 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:47.967 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:47.967 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:47.967 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-02-02T15:46:47.968 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:47.968 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.968 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.968 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.968 INFO:teuthology.orchestra.run.smithi001.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-02-02T15:46:47.968 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-02-02T15:46:47.968 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:47.968 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.969 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:47.969 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "*", 2024-02-02T15:46:47.969 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:47.969 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:47.969 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:47.969 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-02-02T15:46:47.969 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:47.969 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.970 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.970 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.970 INFO:teuthology.orchestra.run.smithi001.stdout: "max_count_per_host": { 2024-02-02T15:46:47.970 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_count_per_host", 2024-02-02T15:46:47.970 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:47.970 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.970 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:47.970 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-02-02T15:46:47.970 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:47.971 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:47.971 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:47.971 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "max number of daemons per service per host", 2024-02-02T15:46:47.971 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:47.971 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.971 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.971 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.971 INFO:teuthology.orchestra.run.smithi001.stdout: "max_osd_draining_count": { 2024-02-02T15:46:47.971 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_osd_draining_count", 2024-02-02T15:46:47.971 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:47.972 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.972 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:47.972 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-02-02T15:46:47.972 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:47.972 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:47.972 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:47.972 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-02-02T15:46:47.972 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:47.972 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.973 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.973 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.973 INFO:teuthology.orchestra.run.smithi001.stdout: "migration_current": { 2024-02-02T15:46:47.973 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "migration_current", 2024-02-02T15:46:47.973 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:47.973 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.973 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:47.973 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:47.973 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:47.974 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:47.974 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:47.974 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "internal - do not modify", 2024-02-02T15:46:47.974 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:47.974 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.974 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.974 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.974 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": { 2024-02-02T15:46:47.974 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mode", 2024-02-02T15:46:47.974 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:47.975 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.975 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:47.975 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "root", 2024-02-02T15:46:47.975 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:47.975 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:47.975 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:47.975 INFO:teuthology.orchestra.run.smithi001.stdout: "cephadm-package", 2024-02-02T15:46:47.975 INFO:teuthology.orchestra.run.smithi001.stdout: "root" 2024-02-02T15:46:47.975 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:47.976 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "mode for remote execution of cephadm", 2024-02-02T15:46:47.976 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:47.976 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.976 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.976 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.976 INFO:teuthology.orchestra.run.smithi001.stdout: "prometheus_alerts_path": { 2024-02-02T15:46:47.976 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "prometheus_alerts_path", 2024-02-02T15:46:47.976 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:47.976 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.977 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:47.977 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-02-02T15:46:47.977 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:47.977 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:47.977 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:47.977 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-02-02T15:46:47.977 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:47.977 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.977 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.978 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.978 INFO:teuthology.orchestra.run.smithi001.stdout: "registry_insecure": { 2024-02-02T15:46:47.978 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "registry_insecure", 2024-02-02T15:46:47.978 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:47.978 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.978 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:47.978 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:47.978 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:47.978 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:47.978 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:47.979 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-02-02T15:46:47.979 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:47.979 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.979 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.979 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.979 INFO:teuthology.orchestra.run.smithi001.stdout: "registry_password": { 2024-02-02T15:46:47.979 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "registry_password", 2024-02-02T15:46:47.979 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:47.979 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.979 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:47.980 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:47.980 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:47.980 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:47.980 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:47.980 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-02-02T15:46:47.980 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:47.980 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.980 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.980 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.981 INFO:teuthology.orchestra.run.smithi001.stdout: "registry_url": { 2024-02-02T15:46:47.981 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "registry_url", 2024-02-02T15:46:47.981 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:47.981 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.981 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:47.981 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:47.981 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:47.981 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:47.981 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:47.982 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-02-02T15:46:47.982 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:47.982 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.982 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.982 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.982 INFO:teuthology.orchestra.run.smithi001.stdout: "registry_username": { 2024-02-02T15:46:47.982 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "registry_username", 2024-02-02T15:46:47.982 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:47.982 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.982 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:47.983 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:47.983 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:47.983 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:47.983 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:47.983 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-02-02T15:46:47.983 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:47.983 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.983 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.983 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.984 INFO:teuthology.orchestra.run.smithi001.stdout: "secure_monitoring_stack": { 2024-02-02T15:46:47.984 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "secure_monitoring_stack", 2024-02-02T15:46:47.984 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:47.984 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.984 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:47.984 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:47.984 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:47.984 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:47.985 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:47.985 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-02-02T15:46:47.985 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:47.985 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.985 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.985 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.985 INFO:teuthology.orchestra.run.smithi001.stdout: "service_discovery_port": { 2024-02-02T15:46:47.985 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "service_discovery_port", 2024-02-02T15:46:47.986 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:47.986 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.986 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:47.986 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8765", 2024-02-02T15:46:47.986 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:47.986 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:47.986 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:47.986 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "cephadm service discovery port", 2024-02-02T15:46:47.987 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:47.987 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.987 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.987 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.987 INFO:teuthology.orchestra.run.smithi001.stdout: "ssh_config_file": { 2024-02-02T15:46:47.987 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ssh_config_file", 2024-02-02T15:46:47.987 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:47.987 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.987 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:47.988 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:47.988 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:47.988 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:47.988 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:47.988 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-02-02T15:46:47.988 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:47.989 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.989 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.989 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.989 INFO:teuthology.orchestra.run.smithi001.stdout: "use_agent": { 2024-02-02T15:46:47.989 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "use_agent", 2024-02-02T15:46:47.989 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:47.989 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.990 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:47.990 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:47.990 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:47.990 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:47.990 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:47.990 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-02-02T15:46:47.990 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:47.990 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.991 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.991 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.991 INFO:teuthology.orchestra.run.smithi001.stdout: "use_repo_digest": { 2024-02-02T15:46:47.991 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "use_repo_digest", 2024-02-02T15:46:47.991 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:47.991 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.991 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:47.992 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:47.992 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:47.992 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:47.992 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:47.992 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-02-02T15:46:47.992 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:47.992 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.992 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.993 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.993 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_on_failed_host_check": { 2024-02-02T15:46:47.993 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_on_failed_host_check", 2024-02-02T15:46:47.993 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:47.993 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.993 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:47.993 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:47.993 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:47.993 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:47.994 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:47.994 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "raise a health warning if the host check fails", 2024-02-02T15:46:47.994 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:47.994 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.994 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.994 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.994 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_on_stray_daemons": { 2024-02-02T15:46:47.994 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_on_stray_daemons", 2024-02-02T15:46:47.994 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:47.995 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.995 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:47.995 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:47.995 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:47.995 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:47.995 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:47.995 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-02-02T15:46:47.995 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:47.995 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.996 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.996 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.996 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_on_stray_hosts": { 2024-02-02T15:46:47.996 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_on_stray_hosts", 2024-02-02T15:46:47.996 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:47.996 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.996 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:47.997 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:47.997 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:47.997 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:47.997 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:47.997 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-02-02T15:46:47.997 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:47.997 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:47.998 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:47.998 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:47.998 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:47.998 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:47.998 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:47.998 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crash", 2024-02-02T15:46:47.998 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:47.998 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:47.999 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:47.999 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:47.999 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:47.999 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:47.999 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:47.999 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:47.999 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:47.999 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:47.999 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.000 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.000 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.000 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.000 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.000 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.000 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.000 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.000 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.000 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.001 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.001 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.001 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.001 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.001 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:48.001 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:48.001 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.001 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.001 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.001 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.002 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.002 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.002 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.002 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.002 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.002 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.002 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.002 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.002 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:48.002 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:48.003 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.003 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.003 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.003 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:48.003 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.003 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.003 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.003 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.004 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.004 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.004 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.004 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.004 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.004 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.004 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.004 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.005 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.005 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.005 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.005 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:48.005 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:48.005 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.005 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.006 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.006 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.006 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.006 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.006 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.006 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.006 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.006 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.006 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.007 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.007 INFO:teuthology.orchestra.run.smithi001.stdout: "retain_interval": { 2024-02-02T15:46:48.007 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "retain_interval", 2024-02-02T15:46:48.007 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-02-02T15:46:48.007 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.007 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.007 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "31536000", 2024-02-02T15:46:48.007 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.007 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.008 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.008 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how long to retain crashes before pruning them", 2024-02-02T15:46:48.008 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.008 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.008 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.008 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.008 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_recent_interval": { 2024-02-02T15:46:48.008 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_recent_interval", 2024-02-02T15:46:48.009 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-02-02T15:46:48.009 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.009 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.009 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "1209600", 2024-02-02T15:46:48.009 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.009 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.009 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.009 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "time interval in which to warn about recent crashes", 2024-02-02T15:46:48.009 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.010 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.010 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.010 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.010 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.010 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.010 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:48.010 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "dashboard", 2024-02-02T15:46:48.010 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:48.011 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:48.011 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:48.011 INFO:teuthology.orchestra.run.smithi001.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-02-02T15:46:48.011 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-02-02T15:46:48.011 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:48.011 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.011 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.011 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-02-02T15:46:48.011 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.011 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.012 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.012 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.012 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.012 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.012 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.012 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.012 INFO:teuthology.orchestra.run.smithi001.stdout: "ALERTMANAGER_API_HOST": { 2024-02-02T15:46:48.012 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ALERTMANAGER_API_HOST", 2024-02-02T15:46:48.012 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.013 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.013 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.013 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.013 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.013 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.013 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.013 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.013 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.014 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.014 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.014 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.014 INFO:teuthology.orchestra.run.smithi001.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-02-02T15:46:48.014 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-02-02T15:46:48.014 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.014 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.015 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.015 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:48.015 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.015 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.015 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.015 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.015 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.015 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.015 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.016 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.016 INFO:teuthology.orchestra.run.smithi001.stdout: "AUDIT_API_ENABLED": { 2024-02-02T15:46:48.016 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "AUDIT_API_ENABLED", 2024-02-02T15:46:48.016 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.016 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.016 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.016 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.017 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.017 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.017 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.017 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.017 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.017 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.017 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.017 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.017 INFO:teuthology.orchestra.run.smithi001.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-02-02T15:46:48.018 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-02-02T15:46:48.018 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.018 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.018 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.018 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:48.018 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.018 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.018 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.018 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.019 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.019 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.019 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.019 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.019 INFO:teuthology.orchestra.run.smithi001.stdout: "ENABLE_BROWSABLE_API": { 2024-02-02T15:46:48.019 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ENABLE_BROWSABLE_API", 2024-02-02T15:46:48.019 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.019 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.019 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.020 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:48.020 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.020 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.020 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.020 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.020 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.020 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.020 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.020 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.021 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-02-02T15:46:48.021 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-02-02T15:46:48.021 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.021 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.021 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.021 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:48.021 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.021 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.021 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.022 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.022 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.022 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.022 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.022 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.022 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-02-02T15:46:48.022 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-02-02T15:46:48.022 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.022 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.022 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.023 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:48.023 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.023 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.023 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.023 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.023 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.023 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.023 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.023 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.024 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-02-02T15:46:48.024 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-02-02T15:46:48.024 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.024 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.024 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.024 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:48.024 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.024 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.024 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.025 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.025 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.025 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.025 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.025 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.025 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-02-02T15:46:48.025 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-02-02T15:46:48.025 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.026 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.026 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.026 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:48.026 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.026 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.026 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.026 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.026 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.026 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.027 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.027 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.027 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_NFS": { 2024-02-02T15:46:48.027 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-02-02T15:46:48.027 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.027 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.027 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.027 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:48.028 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.028 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.028 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.028 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.028 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.028 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.028 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.028 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.029 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_RBD": { 2024-02-02T15:46:48.029 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-02-02T15:46:48.029 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.029 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.029 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.029 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:48.029 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.029 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.029 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.030 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.030 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.030 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.030 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.030 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.030 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_RGW": { 2024-02-02T15:46:48.030 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-02-02T15:46:48.030 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.030 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.031 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.031 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:48.031 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.031 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.031 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.031 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.031 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.031 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.032 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.032 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.032 INFO:teuthology.orchestra.run.smithi001.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-02-02T15:46:48.032 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-02-02T15:46:48.032 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.032 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.032 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.032 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.032 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.032 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.033 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.033 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.033 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.033 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.033 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.033 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.033 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_API_PASSWORD": { 2024-02-02T15:46:48.033 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_API_PASSWORD", 2024-02-02T15:46:48.033 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.034 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.034 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.034 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "admin", 2024-02-02T15:46:48.034 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.034 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.034 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.034 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.034 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.035 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.035 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.035 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.035 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-02-02T15:46:48.035 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-02-02T15:46:48.035 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.035 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.035 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.035 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:48.036 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.036 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.036 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.036 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.036 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.036 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.036 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.037 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.037 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_API_URL": { 2024-02-02T15:46:48.037 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_API_URL", 2024-02-02T15:46:48.037 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.037 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.037 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.037 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.037 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.037 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.038 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.038 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.038 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.038 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.038 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.038 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.038 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_API_USERNAME": { 2024-02-02T15:46:48.038 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_API_USERNAME", 2024-02-02T15:46:48.038 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.038 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.039 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.039 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "admin", 2024-02-02T15:46:48.039 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.039 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.039 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.039 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.039 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.039 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.040 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.040 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.040 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-02-02T15:46:48.040 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-02-02T15:46:48.040 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.040 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.040 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.040 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.041 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.041 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.041 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.041 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.041 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.041 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.041 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.041 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.042 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-02-02T15:46:48.042 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-02-02T15:46:48.042 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.042 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.042 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.042 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.042 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.042 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.042 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.043 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.043 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.043 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.043 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.043 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.043 INFO:teuthology.orchestra.run.smithi001.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-02-02T15:46:48.043 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-02-02T15:46:48.043 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.043 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.044 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.044 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:48.044 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.044 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.044 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.044 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.044 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.044 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.044 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.044 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.045 INFO:teuthology.orchestra.run.smithi001.stdout: "ISSUE_TRACKER_API_KEY": { 2024-02-02T15:46:48.045 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-02-02T15:46:48.045 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.045 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.045 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.045 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.045 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.045 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.045 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.045 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.046 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.046 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.046 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.046 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.046 INFO:teuthology.orchestra.run.smithi001.stdout: "PROMETHEUS_API_HOST": { 2024-02-02T15:46:48.046 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PROMETHEUS_API_HOST", 2024-02-02T15:46:48.046 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.047 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.047 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.047 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.047 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.047 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.047 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.047 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.047 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.048 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.048 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.048 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.048 INFO:teuthology.orchestra.run.smithi001.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-02-02T15:46:48.048 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-02-02T15:46:48.048 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.048 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.048 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.049 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:48.049 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.049 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.049 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.049 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.049 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.049 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.050 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.050 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.050 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-02-02T15:46:48.050 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-02-02T15:46:48.050 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.050 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.050 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.050 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.051 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.051 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.051 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.051 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.051 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.051 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.051 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.052 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.052 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-02-02T15:46:48.052 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-02-02T15:46:48.052 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.052 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.052 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.052 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.052 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.053 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.053 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.053 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.053 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.053 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.053 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.053 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.054 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-02-02T15:46:48.054 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-02-02T15:46:48.054 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.054 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.054 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.054 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:48.054 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.054 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.055 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.055 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.055 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.055 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.055 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.055 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.055 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-02-02T15:46:48.056 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-02-02T15:46:48.056 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.056 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.056 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.056 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:48.056 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.056 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.056 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.057 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.057 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.057 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.057 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.057 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.057 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-02-02T15:46:48.057 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-02-02T15:46:48.058 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.058 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.058 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.058 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.058 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.058 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.058 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.058 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.059 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.059 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.059 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.059 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.059 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-02-02T15:46:48.059 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-02-02T15:46:48.059 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.059 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.060 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.060 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.060 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.060 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.060 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.060 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.060 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.061 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.061 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.061 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.061 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-02-02T15:46:48.061 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-02-02T15:46:48.061 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.061 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.061 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.062 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.062 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.062 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.062 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.062 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.062 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.062 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.062 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.063 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.063 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_ENABLED": { 2024-02-02T15:46:48.063 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_ENABLED", 2024-02-02T15:46:48.063 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.063 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.063 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.063 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:48.063 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.063 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.064 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.064 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.064 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.064 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.064 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.064 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.064 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-02-02T15:46:48.064 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-02-02T15:46:48.065 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.065 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.065 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.065 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-02-02T15:46:48.065 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.065 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.065 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.065 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.066 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.066 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.066 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.066 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.066 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-02-02T15:46:48.066 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-02-02T15:46:48.066 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:48.066 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.066 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.067 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-02-02T15:46:48.067 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.067 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.067 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.067 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.067 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.067 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.067 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.067 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.068 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-02-02T15:46:48.068 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-02-02T15:46:48.068 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:48.068 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.068 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.068 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8", 2024-02-02T15:46:48.068 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.068 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.069 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.069 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.069 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.069 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.069 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.069 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.069 INFO:teuthology.orchestra.run.smithi001.stdout: "REST_REQUESTS_TIMEOUT": { 2024-02-02T15:46:48.069 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-02-02T15:46:48.070 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:48.070 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.070 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.070 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "45", 2024-02-02T15:46:48.070 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.070 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.070 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.070 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.070 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.071 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.071 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.071 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.071 INFO:teuthology.orchestra.run.smithi001.stdout: "RGW_API_ACCESS_KEY": { 2024-02-02T15:46:48.071 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RGW_API_ACCESS_KEY", 2024-02-02T15:46:48.071 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.071 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.071 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.071 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.072 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.072 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.072 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.072 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.072 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.072 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.072 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.073 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.073 INFO:teuthology.orchestra.run.smithi001.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-02-02T15:46:48.073 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-02-02T15:46:48.073 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.073 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.073 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.073 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "admin", 2024-02-02T15:46:48.073 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.074 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.074 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.074 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.074 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.074 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.074 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.074 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.074 INFO:teuthology.orchestra.run.smithi001.stdout: "RGW_API_SECRET_KEY": { 2024-02-02T15:46:48.075 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RGW_API_SECRET_KEY", 2024-02-02T15:46:48.075 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.075 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.075 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.075 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.075 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.075 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.075 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.076 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.076 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.076 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.076 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.076 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.076 INFO:teuthology.orchestra.run.smithi001.stdout: "RGW_API_SSL_VERIFY": { 2024-02-02T15:46:48.076 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RGW_API_SSL_VERIFY", 2024-02-02T15:46:48.076 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.077 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.077 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.077 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:48.077 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.077 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.077 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.077 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.077 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.077 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.078 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.078 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.078 INFO:teuthology.orchestra.run.smithi001.stdout: "UNSAFE_TLS_v1_2": { 2024-02-02T15:46:48.078 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "UNSAFE_TLS_v1_2", 2024-02-02T15:46:48.078 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.078 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.078 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.078 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.078 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.079 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.079 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.079 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.079 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.079 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.079 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.079 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.079 INFO:teuthology.orchestra.run.smithi001.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-02-02T15:46:48.079 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-02-02T15:46:48.079 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:48.080 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.080 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.080 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-02-02T15:46:48.080 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.080 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.080 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.080 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.080 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.081 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.081 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.081 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.081 INFO:teuthology.orchestra.run.smithi001.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-02-02T15:46:48.081 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-02-02T15:46:48.081 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:48.081 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.081 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.081 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-02-02T15:46:48.081 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.082 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.082 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.082 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.082 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.082 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.082 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.082 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.082 INFO:teuthology.orchestra.run.smithi001.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-02-02T15:46:48.083 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-02-02T15:46:48.083 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:48.083 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.083 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.083 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2024-02-02T15:46:48.083 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.083 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.083 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.084 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.084 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.084 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.084 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.084 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.084 INFO:teuthology.orchestra.run.smithi001.stdout: "cross_origin_url": { 2024-02-02T15:46:48.084 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cross_origin_url", 2024-02-02T15:46:48.085 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.085 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.085 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.085 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.085 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.085 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.085 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.085 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.086 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.086 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.086 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.086 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.086 INFO:teuthology.orchestra.run.smithi001.stdout: "crt_file": { 2024-02-02T15:46:48.086 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crt_file", 2024-02-02T15:46:48.086 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.086 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.087 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.087 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.087 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.087 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.087 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.087 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.087 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.087 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.088 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.088 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.088 INFO:teuthology.orchestra.run.smithi001.stdout: "debug": { 2024-02-02T15:46:48.088 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "debug", 2024-02-02T15:46:48.088 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.088 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.088 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.088 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.088 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.089 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.089 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.089 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Enable/disable debug options", 2024-02-02T15:46:48.089 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.089 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.089 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.089 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.089 INFO:teuthology.orchestra.run.smithi001.stdout: "jwt_token_ttl": { 2024-02-02T15:46:48.089 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "jwt_token_ttl", 2024-02-02T15:46:48.090 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:48.090 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.090 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.090 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "28800", 2024-02-02T15:46:48.090 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.090 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.090 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.090 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.091 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.091 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.091 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.091 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.091 INFO:teuthology.orchestra.run.smithi001.stdout: "key_file": { 2024-02-02T15:46:48.091 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "key_file", 2024-02-02T15:46:48.091 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.092 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.092 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.092 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.092 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.092 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.092 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.092 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.092 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.093 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.093 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.093 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.093 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:48.093 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:48.093 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.093 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.093 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.094 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.094 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.094 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.094 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.094 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.094 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.094 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.094 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.094 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.095 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.095 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.095 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.095 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.095 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.095 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.095 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.095 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:48.096 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:48.096 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.096 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.096 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.096 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.096 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.096 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.096 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.096 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.097 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.097 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.097 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.097 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.097 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:48.097 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:48.097 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.097 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.098 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.098 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:48.098 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.098 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.098 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.098 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.098 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.098 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.098 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.099 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.099 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.099 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.099 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.099 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.099 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.099 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.099 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.100 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:48.100 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:48.100 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.100 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.100 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.100 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.100 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.100 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.101 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.101 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.101 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.101 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.101 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.101 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.101 INFO:teuthology.orchestra.run.smithi001.stdout: "motd": { 2024-02-02T15:46:48.101 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "motd", 2024-02-02T15:46:48.101 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.102 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.102 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.102 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.102 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.102 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.102 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.102 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "The message of the day", 2024-02-02T15:46:48.102 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.103 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.103 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.103 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.103 INFO:teuthology.orchestra.run.smithi001.stdout: "redirect_resolve_ip_addr": { 2024-02-02T15:46:48.103 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "redirect_resolve_ip_addr", 2024-02-02T15:46:48.103 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.103 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.103 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.104 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.104 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.104 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.104 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.104 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.104 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.104 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.104 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.105 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.105 INFO:teuthology.orchestra.run.smithi001.stdout: "server_addr": { 2024-02-02T15:46:48.105 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_addr", 2024-02-02T15:46:48.105 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.105 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.105 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.105 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "::", 2024-02-02T15:46:48.106 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.106 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.106 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.106 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.106 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.106 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.106 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.106 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.106 INFO:teuthology.orchestra.run.smithi001.stdout: "server_port": { 2024-02-02T15:46:48.107 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_port", 2024-02-02T15:46:48.107 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:48.107 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.107 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.107 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8080", 2024-02-02T15:46:48.107 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.107 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.107 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.108 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.108 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.108 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.108 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.108 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.108 INFO:teuthology.orchestra.run.smithi001.stdout: "ssl": { 2024-02-02T15:46:48.108 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ssl", 2024-02-02T15:46:48.108 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.108 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.109 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.109 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:48.109 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.109 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.109 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.109 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.109 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.109 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.110 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.110 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.110 INFO:teuthology.orchestra.run.smithi001.stdout: "ssl_server_port": { 2024-02-02T15:46:48.110 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ssl_server_port", 2024-02-02T15:46:48.110 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:48.110 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.110 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.110 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8443", 2024-02-02T15:46:48.111 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.111 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.111 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.111 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.111 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.111 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.111 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.111 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.112 INFO:teuthology.orchestra.run.smithi001.stdout: "standby_behaviour": { 2024-02-02T15:46:48.112 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "standby_behaviour", 2024-02-02T15:46:48.112 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.112 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.112 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.112 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "redirect", 2024-02-02T15:46:48.112 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.112 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.113 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.113 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.113 INFO:teuthology.orchestra.run.smithi001.stdout: "redirect" 2024-02-02T15:46:48.113 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.113 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.113 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.113 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.113 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.114 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.114 INFO:teuthology.orchestra.run.smithi001.stdout: "standby_error_status_code": { 2024-02-02T15:46:48.114 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "standby_error_status_code", 2024-02-02T15:46:48.114 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:48.114 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.114 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.114 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "500", 2024-02-02T15:46:48.114 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "400", 2024-02-02T15:46:48.114 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "599", 2024-02-02T15:46:48.115 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.115 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.115 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.115 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.115 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.115 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.115 INFO:teuthology.orchestra.run.smithi001.stdout: "url_prefix": { 2024-02-02T15:46:48.115 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "url_prefix", 2024-02-02T15:46:48.116 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.116 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.116 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.116 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.116 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.116 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.116 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.116 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.116 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.117 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.117 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.117 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.117 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.117 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.117 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:48.117 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "devicehealth", 2024-02-02T15:46:48.117 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:48.117 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:48.118 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:48.118 INFO:teuthology.orchestra.run.smithi001.stdout: "enable_monitoring": { 2024-02-02T15:46:48.118 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "enable_monitoring", 2024-02-02T15:46:48.118 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.118 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.118 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.118 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:48.118 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.119 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.119 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.119 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "monitor device health metrics", 2024-02-02T15:46:48.119 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.119 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.119 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.119 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.119 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:48.120 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:48.120 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.120 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.120 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.120 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.120 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.120 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.121 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.121 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.121 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.121 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.121 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.121 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.121 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.121 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.121 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.122 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.122 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.122 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.122 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.122 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:48.122 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:48.122 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.122 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.122 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.123 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.123 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.123 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.123 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.123 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.123 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.123 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.123 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.123 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.123 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:48.124 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:48.124 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.124 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.124 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.124 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:48.124 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.124 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.125 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.125 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.125 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.125 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.125 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.125 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.125 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.125 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.125 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.126 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.126 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.126 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.126 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.126 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:48.126 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:48.126 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.126 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.127 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.127 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.127 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.127 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.127 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.127 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.127 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.127 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.127 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.127 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.128 INFO:teuthology.orchestra.run.smithi001.stdout: "mark_out_threshold": { 2024-02-02T15:46:48.128 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mark_out_threshold", 2024-02-02T15:46:48.128 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-02-02T15:46:48.128 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.128 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.128 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "2419200", 2024-02-02T15:46:48.128 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.128 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.128 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.129 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-02-02T15:46:48.129 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.129 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.129 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.129 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.129 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_name": { 2024-02-02T15:46:48.129 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "pool_name", 2024-02-02T15:46:48.129 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.130 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.130 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.130 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "device_health_metrics", 2024-02-02T15:46:48.130 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.130 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.130 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.130 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "name of pool in which to store device health metrics", 2024-02-02T15:46:48.130 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.130 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.130 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.131 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.131 INFO:teuthology.orchestra.run.smithi001.stdout: "retention_period": { 2024-02-02T15:46:48.131 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "retention_period", 2024-02-02T15:46:48.131 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-02-02T15:46:48.131 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.131 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.131 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "15552000", 2024-02-02T15:46:48.131 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.131 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.131 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.132 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how long to retain device health metrics", 2024-02-02T15:46:48.132 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.132 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.132 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.132 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.132 INFO:teuthology.orchestra.run.smithi001.stdout: "scrape_frequency": { 2024-02-02T15:46:48.132 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "scrape_frequency", 2024-02-02T15:46:48.132 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-02-02T15:46:48.132 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.132 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.133 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "86400", 2024-02-02T15:46:48.133 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.133 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.133 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.133 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to scrape device health metrics", 2024-02-02T15:46:48.133 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.133 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.133 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.133 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.134 INFO:teuthology.orchestra.run.smithi001.stdout: "self_heal": { 2024-02-02T15:46:48.134 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "self_heal", 2024-02-02T15:46:48.134 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.134 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.134 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.134 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:48.134 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.134 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.134 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.135 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-02-02T15:46:48.135 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.135 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.135 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.135 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.135 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-02-02T15:46:48.135 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-02-02T15:46:48.135 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-02-02T15:46:48.135 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.136 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.136 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-02-02T15:46:48.136 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.136 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.136 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.136 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to wake up and check device health", 2024-02-02T15:46:48.136 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.136 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.137 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.137 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.137 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_threshold": { 2024-02-02T15:46:48.137 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_threshold", 2024-02-02T15:46:48.137 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-02-02T15:46:48.137 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.137 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.137 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "7257600", 2024-02-02T15:46:48.137 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.138 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.138 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.138 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-02-02T15:46:48.138 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.138 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.138 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.138 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.138 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.139 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.139 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:48.139 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "diskprediction_local", 2024-02-02T15:46:48.139 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:48.139 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:48.139 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:48.139 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:48.139 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:48.139 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.140 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.140 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.140 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.140 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.140 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.140 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.141 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.141 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.141 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.141 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.141 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.141 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.141 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.142 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.142 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.142 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.142 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.142 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.142 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:48.143 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:48.143 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.143 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.143 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.143 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.143 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.143 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.144 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.144 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.144 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.144 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.144 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.144 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.144 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:48.145 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:48.145 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.145 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.145 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.145 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:48.145 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.145 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.146 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.146 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.146 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.146 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.146 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.146 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.146 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.146 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.147 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.147 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.147 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.147 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.147 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.147 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:48.147 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:48.148 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.148 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.148 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.148 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.148 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.148 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.148 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.148 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.148 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.149 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.149 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.149 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.149 INFO:teuthology.orchestra.run.smithi001.stdout: "predict_interval": { 2024-02-02T15:46:48.149 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "predict_interval", 2024-02-02T15:46:48.149 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.149 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.150 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.150 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "86400", 2024-02-02T15:46:48.150 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.150 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.150 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.150 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.150 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.150 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.150 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.151 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.151 INFO:teuthology.orchestra.run.smithi001.stdout: "predictor_model": { 2024-02-02T15:46:48.151 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "predictor_model", 2024-02-02T15:46:48.151 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.151 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.151 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.151 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "prophetstor", 2024-02-02T15:46:48.151 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.152 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.152 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.152 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.152 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.152 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.152 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.152 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.152 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-02-02T15:46:48.152 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-02-02T15:46:48.153 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.153 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.153 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.153 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-02-02T15:46:48.153 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.153 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.153 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.154 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.154 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.154 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.154 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.154 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.154 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.154 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.154 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:48.154 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "influx", 2024-02-02T15:46:48.155 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": false, 2024-02-02T15:46:48.155 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "influxdb python module not found", 2024-02-02T15:46:48.155 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:48.155 INFO:teuthology.orchestra.run.smithi001.stdout: "batch_size": { 2024-02-02T15:46:48.155 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "batch_size", 2024-02-02T15:46:48.155 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:48.155 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.155 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.155 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5000", 2024-02-02T15:46:48.156 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.156 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.156 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.156 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-02-02T15:46:48.156 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.156 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.156 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.156 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.156 INFO:teuthology.orchestra.run.smithi001.stdout: "database": { 2024-02-02T15:46:48.156 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "database", 2024-02-02T15:46:48.157 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.157 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.157 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.157 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "ceph", 2024-02-02T15:46:48.157 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.157 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.157 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.157 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-02-02T15:46:48.157 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.157 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.158 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.158 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.158 INFO:teuthology.orchestra.run.smithi001.stdout: "hostname": { 2024-02-02T15:46:48.158 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "hostname", 2024-02-02T15:46:48.158 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.158 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.158 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.158 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.158 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.159 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.159 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.159 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "InfluxDB server hostname", 2024-02-02T15:46:48.159 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.159 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.159 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.159 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.159 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-02-02T15:46:48.160 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-02-02T15:46:48.160 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-02-02T15:46:48.160 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.160 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.160 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "30", 2024-02-02T15:46:48.160 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "5", 2024-02-02T15:46:48.160 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.160 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.160 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-02-02T15:46:48.161 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.161 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.161 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.161 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.161 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:48.161 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:48.161 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.161 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.161 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.161 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.162 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.162 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.162 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.162 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.162 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.162 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.162 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.162 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.163 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.163 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.163 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.163 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.163 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.163 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.163 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.163 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:48.163 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:48.164 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.164 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.164 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.164 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.164 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.164 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.164 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.164 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.165 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.165 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.165 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.165 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.165 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:48.165 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:48.165 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.165 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.166 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.166 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:48.166 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.166 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.166 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.166 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.166 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.166 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.167 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.167 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.167 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.167 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.167 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.167 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.167 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.167 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.168 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.168 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:48.168 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:48.168 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.168 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.168 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.168 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.168 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.168 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.169 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.169 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.169 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.169 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.169 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.169 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.169 INFO:teuthology.orchestra.run.smithi001.stdout: "password": { 2024-02-02T15:46:48.169 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "password", 2024-02-02T15:46:48.169 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.170 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.170 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.170 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.170 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.170 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.170 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.170 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "password of InfluxDB server user", 2024-02-02T15:46:48.170 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.171 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.171 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.171 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.171 INFO:teuthology.orchestra.run.smithi001.stdout: "port": { 2024-02-02T15:46:48.171 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "port", 2024-02-02T15:46:48.171 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:48.171 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.171 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.171 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8086", 2024-02-02T15:46:48.172 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.172 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.172 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.172 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "InfluxDB server port", 2024-02-02T15:46:48.172 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.172 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.172 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.172 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.172 INFO:teuthology.orchestra.run.smithi001.stdout: "ssl": { 2024-02-02T15:46:48.173 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ssl", 2024-02-02T15:46:48.173 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.173 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.173 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.173 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "false", 2024-02-02T15:46:48.173 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.173 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.173 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.173 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-02-02T15:46:48.174 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.174 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.174 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.174 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.174 INFO:teuthology.orchestra.run.smithi001.stdout: "threads": { 2024-02-02T15:46:48.174 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "threads", 2024-02-02T15:46:48.174 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:48.174 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.174 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.175 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2024-02-02T15:46:48.175 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1", 2024-02-02T15:46:48.175 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "32", 2024-02-02T15:46:48.175 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.175 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-02-02T15:46:48.175 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.175 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.175 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.176 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.176 INFO:teuthology.orchestra.run.smithi001.stdout: "username": { 2024-02-02T15:46:48.176 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "username", 2024-02-02T15:46:48.176 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.176 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.176 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.176 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.176 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.176 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.177 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.177 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "username of InfluxDB server user", 2024-02-02T15:46:48.177 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.177 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.177 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.177 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.177 INFO:teuthology.orchestra.run.smithi001.stdout: "verify_ssl": { 2024-02-02T15:46:48.177 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "verify_ssl", 2024-02-02T15:46:48.178 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.178 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.178 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.178 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "true", 2024-02-02T15:46:48.178 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.178 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.178 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.178 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-02-02T15:46:48.178 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.179 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.179 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.179 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.179 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.179 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.179 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:48.179 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "insights", 2024-02-02T15:46:48.180 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:48.180 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:48.180 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:48.180 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:48.180 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:48.180 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.180 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.180 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.181 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.181 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.181 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.181 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.181 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.181 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.181 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.181 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.182 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.182 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.182 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.182 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.182 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.182 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.182 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.182 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.183 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:48.183 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:48.183 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.183 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.183 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.183 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.183 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.183 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.183 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.184 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.184 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.184 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.184 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.184 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.184 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:48.184 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:48.184 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.184 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.184 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.185 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:48.185 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.185 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.185 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.185 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.185 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.185 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.185 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.185 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.185 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.186 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.186 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.186 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.186 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.186 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.186 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.186 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:48.186 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:48.187 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.187 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.187 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.187 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.187 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.187 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.187 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.187 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.187 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.188 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.188 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.188 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.188 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.188 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.188 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:48.188 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "iostat", 2024-02-02T15:46:48.188 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:48.188 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:48.188 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:48.189 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:48.189 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:48.189 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.189 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.189 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.189 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.189 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.189 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.189 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.190 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.190 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.190 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.190 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.190 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.190 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.190 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.190 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.190 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.190 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.191 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.191 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.191 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:48.191 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:48.191 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.191 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.191 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.191 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.191 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.192 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.192 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.192 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.192 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.192 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.192 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.192 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.192 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:48.192 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:48.193 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.193 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.193 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.193 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:48.193 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.193 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.193 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.193 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.193 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.193 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.194 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.194 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.194 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.194 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.194 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.194 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.194 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.194 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.194 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.194 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:48.195 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:48.195 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.195 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.195 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.195 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.195 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.195 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.195 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.196 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.196 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.196 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.196 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.196 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.196 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.196 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.196 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:48.196 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "k8sevents", 2024-02-02T15:46:48.196 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:48.197 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:48.197 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:48.197 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_event_retention_days": { 2024-02-02T15:46:48.197 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ceph_event_retention_days", 2024-02-02T15:46:48.197 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:48.197 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.197 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.197 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "7", 2024-02-02T15:46:48.197 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.198 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.198 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.198 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Days to hold ceph event information within local cache", 2024-02-02T15:46:48.198 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.198 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.198 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.198 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.198 INFO:teuthology.orchestra.run.smithi001.stdout: "config_check_secs": { 2024-02-02T15:46:48.199 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "config_check_secs", 2024-02-02T15:46:48.199 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:48.199 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.199 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.199 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-02-02T15:46:48.199 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "10", 2024-02-02T15:46:48.199 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.199 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.200 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-02-02T15:46:48.200 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.200 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.200 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.200 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.200 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:48.200 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:48.200 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.200 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.201 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.201 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.201 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.201 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.201 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.201 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.201 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.202 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.202 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.202 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.202 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.202 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.202 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.202 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.202 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.202 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.203 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.203 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:48.203 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:48.203 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.203 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.203 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.203 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.203 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.203 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.204 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.204 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.204 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.204 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.204 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.204 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.204 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:48.204 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:48.205 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.205 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.205 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.205 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:48.205 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.205 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.205 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.205 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.205 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.206 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.206 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.206 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.206 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.206 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.206 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.206 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.206 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.207 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.207 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.207 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:48.207 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:48.207 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.207 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.207 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.207 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.207 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.207 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.208 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.208 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.208 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.208 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.208 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.208 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.208 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.208 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.209 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:48.209 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "localpool", 2024-02-02T15:46:48.209 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:48.209 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:48.209 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:48.209 INFO:teuthology.orchestra.run.smithi001.stdout: "failure_domain": { 2024-02-02T15:46:48.209 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "failure_domain", 2024-02-02T15:46:48.209 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.209 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.210 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.210 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "host", 2024-02-02T15:46:48.210 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.210 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.210 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.210 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "failure domain for any created local pool", 2024-02-02T15:46:48.210 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-02-02T15:46:48.210 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.211 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.211 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.211 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:48.211 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:48.211 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.211 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.211 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.211 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.211 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.212 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.212 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.212 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.212 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.212 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.212 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.212 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.212 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.213 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.213 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.213 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.213 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.213 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.213 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.213 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:48.213 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:48.213 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.214 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.214 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.214 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.214 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.214 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.214 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.214 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.214 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.215 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.215 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.215 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.215 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:48.215 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:48.215 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.215 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.215 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.216 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:48.216 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.216 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.216 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.216 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.216 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.216 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.216 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.217 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.217 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.217 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.217 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.217 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.217 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.217 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.217 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.218 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:48.218 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:48.218 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.218 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.218 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.218 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.218 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.218 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.218 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.219 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.219 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.219 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.219 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.219 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.219 INFO:teuthology.orchestra.run.smithi001.stdout: "min_size": { 2024-02-02T15:46:48.219 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "min_size", 2024-02-02T15:46:48.219 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:48.219 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.220 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.220 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.220 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.220 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.220 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.220 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "default min_size for any created local pool", 2024-02-02T15:46:48.220 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-02-02T15:46:48.220 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.220 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.221 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.221 INFO:teuthology.orchestra.run.smithi001.stdout: "num_rep": { 2024-02-02T15:46:48.221 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "num_rep", 2024-02-02T15:46:48.221 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:48.221 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.221 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.221 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "3", 2024-02-02T15:46:48.221 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.221 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.222 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.222 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "default replica count for any created local pool", 2024-02-02T15:46:48.222 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.222 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.222 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.222 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.222 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num": { 2024-02-02T15:46:48.222 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "pg_num", 2024-02-02T15:46:48.222 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:48.223 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.223 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.223 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "128", 2024-02-02T15:46:48.223 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.223 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.223 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.223 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "default pg_num for any created local pool", 2024-02-02T15:46:48.223 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.224 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.224 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.224 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.224 INFO:teuthology.orchestra.run.smithi001.stdout: "prefix": { 2024-02-02T15:46:48.224 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "prefix", 2024-02-02T15:46:48.224 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.224 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.224 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.224 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.225 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.225 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.225 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.225 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "name prefix for any created local pool", 2024-02-02T15:46:48.225 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.225 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.225 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.225 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.225 INFO:teuthology.orchestra.run.smithi001.stdout: "subtree": { 2024-02-02T15:46:48.226 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "subtree", 2024-02-02T15:46:48.226 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.226 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.226 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.226 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "rack", 2024-02-02T15:46:48.226 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.226 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.226 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.226 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "CRUSH level for which to create a local pool", 2024-02-02T15:46:48.227 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-02-02T15:46:48.227 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.227 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.227 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.227 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.227 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.227 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:48.227 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mds_autoscaler", 2024-02-02T15:46:48.227 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:48.227 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:48.228 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:48.228 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:48.228 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:48.228 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.228 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.228 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.228 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.228 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.229 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.229 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.229 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.229 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.229 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.229 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.229 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.229 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.230 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.230 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.230 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.230 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.230 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.230 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.230 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:48.230 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:48.231 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.231 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.231 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.231 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.231 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.231 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.231 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.231 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.231 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.231 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.232 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.232 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.232 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:48.232 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:48.232 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.232 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.232 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.232 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:48.232 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.232 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.233 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.233 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.233 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.233 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.233 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.233 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.233 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.233 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.233 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.234 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.234 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.234 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.234 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.234 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:48.234 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:48.234 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.234 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.234 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.234 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.235 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.235 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.235 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.235 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.235 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.235 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.235 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.235 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.235 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.236 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.236 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:48.236 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mirroring", 2024-02-02T15:46:48.236 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:48.236 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:48.236 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:48.236 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:48.236 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:48.237 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.237 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.237 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.237 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.237 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.237 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.237 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.237 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.238 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.238 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.238 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.238 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.238 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.238 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.238 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.239 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.239 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.239 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.239 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.239 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:48.239 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:48.239 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.239 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.240 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.240 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.240 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.240 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.240 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.240 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.240 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.240 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.241 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.241 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.241 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:48.241 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:48.241 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.241 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.241 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.242 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:48.242 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.242 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.242 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.242 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.242 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.242 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.242 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.243 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.243 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.243 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.243 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.243 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.243 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.243 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.244 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.244 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:48.244 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:48.244 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.244 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.244 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.244 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.244 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.245 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.245 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.245 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.245 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.245 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.245 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.245 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.246 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.246 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.246 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:48.246 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "nfs", 2024-02-02T15:46:48.246 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:48.246 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:48.246 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:48.246 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:48.246 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:48.247 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.247 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.247 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.247 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.247 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.247 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.247 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.247 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.247 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.248 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.248 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.248 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.248 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.248 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.248 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.248 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.249 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.249 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.249 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.249 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:48.249 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:48.249 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.249 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.249 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.249 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.250 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.250 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.250 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.250 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.250 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.250 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.250 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.250 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.251 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:48.251 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:48.251 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.251 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.251 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.251 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:48.251 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.251 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.251 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.252 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.252 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.252 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.252 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.252 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.252 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.252 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.252 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.253 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.253 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.253 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.253 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.253 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:48.253 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:48.253 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.253 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.253 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.254 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.254 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.254 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.254 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.254 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.254 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.254 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.254 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.255 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.255 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.255 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.255 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:48.255 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "orchestrator", 2024-02-02T15:46:48.255 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:48.255 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:48.255 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:48.256 INFO:teuthology.orchestra.run.smithi001.stdout: "fail_fs": { 2024-02-02T15:46:48.256 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "fail_fs", 2024-02-02T15:46:48.256 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.256 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.256 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.256 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.256 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.256 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.256 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.257 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-02-02T15:46:48.257 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.257 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.257 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.257 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.257 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:48.257 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:48.257 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.257 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.258 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.258 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.258 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.258 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.258 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.258 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.258 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.258 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.259 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.259 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.259 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.259 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.259 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.259 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.259 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.259 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.260 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.260 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:48.260 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:48.260 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.260 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.260 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.260 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.261 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.261 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.261 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.261 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.261 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.261 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.261 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.261 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.262 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:48.262 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:48.262 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.262 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.262 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.262 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:48.262 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.262 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.262 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.263 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.263 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.263 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.263 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.263 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.263 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.263 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.263 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.263 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.264 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.264 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.264 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.264 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:48.264 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:48.264 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.264 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.264 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.265 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.265 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.265 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.265 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.265 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.265 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.265 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.266 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.266 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.266 INFO:teuthology.orchestra.run.smithi001.stdout: "orchestrator": { 2024-02-02T15:46:48.266 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "orchestrator", 2024-02-02T15:46:48.266 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.266 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.266 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.266 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.266 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.267 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.267 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.267 INFO:teuthology.orchestra.run.smithi001.stdout: "cephadm", 2024-02-02T15:46:48.267 INFO:teuthology.orchestra.run.smithi001.stdout: "rook", 2024-02-02T15:46:48.267 INFO:teuthology.orchestra.run.smithi001.stdout: "test_orchestrator" 2024-02-02T15:46:48.267 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.267 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Orchestrator backend", 2024-02-02T15:46:48.267 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.267 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.268 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.268 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.268 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.268 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.268 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:48.268 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "osd_perf_query", 2024-02-02T15:46:48.268 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:48.268 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:48.268 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:48.269 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:48.269 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:48.269 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.269 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.269 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.269 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.269 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.269 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.269 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.270 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.270 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.270 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.270 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.270 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.270 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.270 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.270 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.270 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.271 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.271 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.271 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.271 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:48.271 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:48.271 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.271 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.271 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.272 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.272 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.272 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.272 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.272 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.272 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.272 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.272 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.273 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.273 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:48.273 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:48.273 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.273 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.273 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.273 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:48.273 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.273 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.274 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.274 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.274 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.274 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.274 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.274 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.274 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.274 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.275 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.275 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.275 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.275 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.275 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.275 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:48.275 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:48.275 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.275 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.276 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.276 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.276 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.276 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.276 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.276 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.276 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.276 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.277 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.277 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.277 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.277 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.277 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:48.277 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "osd_support", 2024-02-02T15:46:48.277 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:48.277 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:48.278 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:48.278 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:48.278 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:48.278 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.278 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.278 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.278 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.278 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.278 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.279 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.279 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.279 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.279 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.279 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.279 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.279 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.279 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.280 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.280 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.280 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.280 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.280 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.280 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:48.280 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:48.280 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.280 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.281 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.281 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.281 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.281 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.281 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.281 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.281 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.281 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.282 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.282 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.282 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:48.282 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:48.282 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.282 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.282 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.282 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:48.283 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.283 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.283 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.283 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.283 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.283 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.283 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.283 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.283 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.284 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.284 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.284 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.284 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.284 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.284 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.284 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:48.284 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:48.285 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.285 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.285 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.285 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.285 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.285 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.285 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.286 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.286 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.286 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.286 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.286 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.286 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.286 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.286 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:48.287 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "pg_autoscaler", 2024-02-02T15:46:48.287 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:48.287 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:48.287 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:48.287 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:48.287 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:48.287 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.288 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.288 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.288 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.288 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.288 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.288 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.288 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.288 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.289 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.289 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.289 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.289 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.289 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.289 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.289 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.289 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.289 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.290 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.290 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:48.290 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:48.290 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.290 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.290 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.290 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.290 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.291 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.291 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.291 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.291 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.291 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.291 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.291 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.291 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:48.291 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:48.292 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.292 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.292 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.292 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:48.292 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.292 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.292 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.292 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.292 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.293 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.293 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.293 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.293 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.293 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.293 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.293 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.293 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.294 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.294 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.294 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:48.294 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:48.294 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.294 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.294 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.294 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.295 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.295 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.295 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.295 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.295 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.295 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.295 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.295 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.295 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-02-02T15:46:48.296 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-02-02T15:46:48.296 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-02-02T15:46:48.296 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.296 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.296 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-02-02T15:46:48.296 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.296 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.296 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.297 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.297 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.297 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.297 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.297 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.297 INFO:teuthology.orchestra.run.smithi001.stdout: "threshold": { 2024-02-02T15:46:48.297 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "threshold", 2024-02-02T15:46:48.297 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-02-02T15:46:48.298 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.298 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.298 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "3.0", 2024-02-02T15:46:48.298 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1.0", 2024-02-02T15:46:48.298 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.298 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.298 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "scaling threshold", 2024-02-02T15:46:48.298 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-02-02T15:46:48.298 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.299 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.299 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.299 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.299 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.299 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:48.299 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "progress", 2024-02-02T15:46:48.299 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:48.299 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:48.300 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:48.300 INFO:teuthology.orchestra.run.smithi001.stdout: "allow_pg_recovery_event": { 2024-02-02T15:46:48.300 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "allow_pg_recovery_event", 2024-02-02T15:46:48.300 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.300 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.300 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.300 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.300 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.300 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.301 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.301 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "allow the module to show pg recovery progress", 2024-02-02T15:46:48.301 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.301 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.301 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.301 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.301 INFO:teuthology.orchestra.run.smithi001.stdout: "enabled": { 2024-02-02T15:46:48.301 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "enabled", 2024-02-02T15:46:48.301 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.302 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.302 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.302 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:48.302 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.302 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.302 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.302 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.302 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.303 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.303 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.303 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.303 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:48.303 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:48.303 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.303 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.303 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.304 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.304 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.304 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.304 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.304 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.304 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.304 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.304 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.304 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.305 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.305 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.305 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.305 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.305 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.305 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.305 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.305 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:48.306 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:48.306 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.306 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.306 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.306 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.306 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.306 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.306 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.307 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.307 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.307 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.307 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.307 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.307 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:48.307 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:48.307 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.308 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.308 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.308 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:48.308 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.308 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.308 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.308 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.308 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.308 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.309 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.309 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.309 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.309 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.309 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.309 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.309 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.309 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.309 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.310 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:48.310 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:48.310 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.310 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.310 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.310 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.310 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.310 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.310 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.311 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.311 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.311 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.311 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.311 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.311 INFO:teuthology.orchestra.run.smithi001.stdout: "max_completed_events": { 2024-02-02T15:46:48.311 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_completed_events", 2024-02-02T15:46:48.311 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:48.311 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.312 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.312 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "50", 2024-02-02T15:46:48.312 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.312 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.312 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.312 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "number of past completed events to remember", 2024-02-02T15:46:48.312 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.313 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.313 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.313 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.313 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-02-02T15:46:48.313 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-02-02T15:46:48.313 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-02-02T15:46:48.313 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.313 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.314 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2024-02-02T15:46:48.314 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.314 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.314 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.314 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how long the module is going to sleep", 2024-02-02T15:46:48.314 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.314 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.314 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.315 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.315 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.315 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.315 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:48.315 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "prometheus", 2024-02-02T15:46:48.315 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:48.315 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:48.315 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:48.315 INFO:teuthology.orchestra.run.smithi001.stdout: "cache": { 2024-02-02T15:46:48.316 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cache", 2024-02-02T15:46:48.316 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.316 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.316 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.316 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:48.316 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.316 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.316 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.317 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.317 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.317 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.317 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.317 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.317 INFO:teuthology.orchestra.run.smithi001.stdout: "exclude_perf_counters": { 2024-02-02T15:46:48.317 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "exclude_perf_counters", 2024-02-02T15:46:48.317 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.318 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.318 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.318 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:48.318 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.318 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.318 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.318 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-02-02T15:46:48.318 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-02-02T15:46:48.318 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.319 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.319 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.319 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:48.319 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:48.319 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.319 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.319 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.319 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.320 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.320 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.320 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.320 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.320 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.320 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.320 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.320 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.320 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.320 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.321 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.321 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.321 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.321 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.321 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.321 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:48.321 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:48.321 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.321 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.322 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.322 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.322 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.322 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.322 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.322 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.322 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.322 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.323 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.323 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.323 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:48.323 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:48.323 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.323 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.323 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.323 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:48.323 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.324 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.324 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.324 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.324 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.324 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.324 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.324 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.325 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.325 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.325 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.325 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.325 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.325 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.325 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.325 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:48.325 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:48.325 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.326 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.326 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.326 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.326 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.326 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.326 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.326 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.326 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.326 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.326 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.327 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.327 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_stats_pools": { 2024-02-02T15:46:48.327 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rbd_stats_pools", 2024-02-02T15:46:48.327 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.327 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.327 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.327 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.327 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.327 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.328 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.328 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.328 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.328 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.328 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.328 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.328 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_stats_pools_refresh_interval": { 2024-02-02T15:46:48.328 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-02-02T15:46:48.328 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:48.329 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.329 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.329 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "300", 2024-02-02T15:46:48.329 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.329 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.329 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.329 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.329 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.329 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.330 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.330 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.330 INFO:teuthology.orchestra.run.smithi001.stdout: "scrape_interval": { 2024-02-02T15:46:48.330 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "scrape_interval", 2024-02-02T15:46:48.330 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-02-02T15:46:48.330 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.330 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.330 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "15.0", 2024-02-02T15:46:48.331 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.331 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.331 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.331 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.331 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.331 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.331 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.331 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.332 INFO:teuthology.orchestra.run.smithi001.stdout: "server_addr": { 2024-02-02T15:46:48.332 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_addr", 2024-02-02T15:46:48.332 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.332 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.332 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.332 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "::", 2024-02-02T15:46:48.332 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.332 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.333 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.333 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-02-02T15:46:48.333 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.333 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.333 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.333 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.333 INFO:teuthology.orchestra.run.smithi001.stdout: "server_port": { 2024-02-02T15:46:48.333 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_port", 2024-02-02T15:46:48.334 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:48.334 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.334 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.334 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "9283", 2024-02-02T15:46:48.334 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.334 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.334 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.334 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-02-02T15:46:48.335 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.335 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.335 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.335 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.335 INFO:teuthology.orchestra.run.smithi001.stdout: "stale_cache_strategy": { 2024-02-02T15:46:48.335 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "stale_cache_strategy", 2024-02-02T15:46:48.335 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.335 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.336 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.336 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "log", 2024-02-02T15:46:48.336 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.336 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.336 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.336 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.336 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.336 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.337 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.337 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.337 INFO:teuthology.orchestra.run.smithi001.stdout: "standby_behaviour": { 2024-02-02T15:46:48.337 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "standby_behaviour", 2024-02-02T15:46:48.337 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.337 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.337 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.337 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "default", 2024-02-02T15:46:48.337 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.338 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.338 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.338 INFO:teuthology.orchestra.run.smithi001.stdout: "default", 2024-02-02T15:46:48.338 INFO:teuthology.orchestra.run.smithi001.stdout: "error" 2024-02-02T15:46:48.338 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.338 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.338 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.338 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.338 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.339 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.339 INFO:teuthology.orchestra.run.smithi001.stdout: "standby_error_status_code": { 2024-02-02T15:46:48.339 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "standby_error_status_code", 2024-02-02T15:46:48.339 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:48.339 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.339 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.339 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "500", 2024-02-02T15:46:48.339 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "400", 2024-02-02T15:46:48.340 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "599", 2024-02-02T15:46:48.340 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.340 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.340 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.340 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.340 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.340 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.340 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.341 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.341 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:48.341 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rbd_support", 2024-02-02T15:46:48.341 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:48.341 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:48.341 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:48.341 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:48.341 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:48.342 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.342 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.342 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.342 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.342 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.342 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.342 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.342 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.343 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.343 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.343 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.343 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.343 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.343 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.343 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.343 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.343 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.344 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.344 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.344 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:48.344 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:48.344 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.344 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.344 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.344 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.344 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.345 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.345 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.345 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.345 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.345 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.345 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.345 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.345 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:48.346 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:48.346 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.346 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.346 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.346 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:48.346 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.346 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.346 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.346 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.347 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.347 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.347 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.347 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.347 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.347 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.347 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.348 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.348 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.348 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.348 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.348 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:48.348 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:48.348 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.348 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.349 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.349 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.349 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.349 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.349 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.349 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.349 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.349 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.349 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.350 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.350 INFO:teuthology.orchestra.run.smithi001.stdout: "max_concurrent_snap_create": { 2024-02-02T15:46:48.350 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_concurrent_snap_create", 2024-02-02T15:46:48.350 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:48.350 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.350 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.350 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-02-02T15:46:48.350 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.350 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.351 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.351 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.351 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.351 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.351 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.351 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.351 INFO:teuthology.orchestra.run.smithi001.stdout: "mirror_snapshot_schedule": { 2024-02-02T15:46:48.351 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mirror_snapshot_schedule", 2024-02-02T15:46:48.351 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.352 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.352 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.352 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.352 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.352 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.352 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.352 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.352 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.352 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.353 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.353 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.353 INFO:teuthology.orchestra.run.smithi001.stdout: "trash_purge_schedule": { 2024-02-02T15:46:48.353 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "trash_purge_schedule", 2024-02-02T15:46:48.353 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.353 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.353 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.353 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.353 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.354 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.354 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.354 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.354 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.354 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.354 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.354 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.354 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.355 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.355 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:48.355 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "restful", 2024-02-02T15:46:48.355 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:48.355 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:48.355 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:48.355 INFO:teuthology.orchestra.run.smithi001.stdout: "enable_auth": { 2024-02-02T15:46:48.355 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "enable_auth", 2024-02-02T15:46:48.356 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.356 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.356 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.356 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:48.356 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.356 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.356 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.356 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.357 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.357 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.357 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.357 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.357 INFO:teuthology.orchestra.run.smithi001.stdout: "key_file": { 2024-02-02T15:46:48.357 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "key_file", 2024-02-02T15:46:48.357 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.357 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.358 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.358 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.358 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.358 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.358 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.358 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.358 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.358 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.358 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.359 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.359 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:48.359 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:48.359 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.359 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.359 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.359 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.359 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.359 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.360 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.360 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.360 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.360 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.360 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.360 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.360 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.360 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.361 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.361 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.361 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.361 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.361 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.361 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:48.361 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:48.361 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.361 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.362 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.362 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.362 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.362 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.362 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.362 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.362 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.362 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.362 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.363 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.363 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:48.363 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:48.363 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.363 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.363 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.363 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:48.363 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.363 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.364 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.364 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.364 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.364 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.364 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.364 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.364 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.364 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.365 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.365 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.365 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.365 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.365 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.365 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:48.365 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:48.366 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.366 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.366 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.366 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.366 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.366 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.366 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.366 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.366 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.367 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.367 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.367 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.367 INFO:teuthology.orchestra.run.smithi001.stdout: "server_addr": { 2024-02-02T15:46:48.367 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_addr", 2024-02-02T15:46:48.367 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.368 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.368 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.368 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.368 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.368 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.368 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.368 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.368 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.368 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.369 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.369 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.369 INFO:teuthology.orchestra.run.smithi001.stdout: "server_port": { 2024-02-02T15:46:48.369 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_port", 2024-02-02T15:46:48.369 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.369 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.369 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.369 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.369 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.370 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.370 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.370 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.370 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.370 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.370 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.370 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.370 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.370 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.371 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:48.371 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rgw", 2024-02-02T15:46:48.371 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:48.371 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:48.371 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:48.371 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:48.371 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:48.371 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.371 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.372 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.372 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.372 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.372 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.372 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.372 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.372 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.372 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.372 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.373 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.373 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.373 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.373 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.373 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.373 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.373 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.373 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.374 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:48.374 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:48.374 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.374 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.374 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.374 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.374 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.374 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.374 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.375 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.375 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.375 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.375 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.375 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.375 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:48.375 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:48.375 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.376 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.376 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.376 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:48.376 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.376 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.376 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.376 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.376 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.376 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.377 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.377 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.377 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.377 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.377 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.377 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.377 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.377 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.377 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.378 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:48.378 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:48.378 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.378 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.378 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.378 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.378 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.378 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.379 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.379 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.379 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.379 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.379 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.379 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.379 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.379 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.379 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:48.380 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rook", 2024-02-02T15:46:48.380 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:48.380 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:48.380 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:48.380 INFO:teuthology.orchestra.run.smithi001.stdout: "drive_group_interval": { 2024-02-02T15:46:48.380 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "drive_group_interval", 2024-02-02T15:46:48.380 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-02-02T15:46:48.380 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.380 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.381 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "300.0", 2024-02-02T15:46:48.381 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.381 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.381 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.381 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2024-02-02T15:46:48.381 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.381 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.381 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.382 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.382 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:48.382 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:48.382 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.382 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.382 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.382 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.382 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.383 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.383 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.383 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.383 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.383 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.383 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.383 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.383 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.384 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.384 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.384 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.384 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.384 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.384 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.384 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:48.384 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:48.385 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.385 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.385 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.385 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.385 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.385 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.385 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.385 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.385 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.386 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.386 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.386 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.386 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:48.386 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:48.386 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.386 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.386 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.387 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:48.387 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.387 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.387 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.387 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.387 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.387 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.387 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.387 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.388 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.388 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.388 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.388 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.388 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.388 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.388 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.388 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:48.389 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:48.389 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.389 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.389 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.389 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.389 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.389 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.389 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.389 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.390 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.390 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.390 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.390 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.390 INFO:teuthology.orchestra.run.smithi001.stdout: "storage_class": { 2024-02-02T15:46:48.390 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "storage_class", 2024-02-02T15:46:48.390 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.390 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.391 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.391 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "local", 2024-02-02T15:46:48.391 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.391 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.391 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.391 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-02-02T15:46:48.391 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.391 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.392 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.392 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.392 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.392 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.392 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:48.392 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "selftest", 2024-02-02T15:46:48.392 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:48.392 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:48.392 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:48.392 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:48.393 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:48.393 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.393 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.393 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.393 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.393 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.393 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.393 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.394 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.394 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.394 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.394 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.394 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.394 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.394 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.394 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.394 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.394 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.395 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.395 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.395 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:48.395 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:48.395 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.395 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.395 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.395 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.395 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.396 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.396 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.396 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.396 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.396 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.396 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.399 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.399 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:48.399 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:48.399 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.399 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.399 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.400 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:48.400 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.400 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.400 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.400 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.400 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.400 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.400 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.400 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.401 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.401 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.401 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.401 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.401 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.401 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.401 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.401 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:48.402 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:48.402 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.402 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.402 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.402 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.402 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.402 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.402 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.402 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.402 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.403 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.403 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.403 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.403 INFO:teuthology.orchestra.run.smithi001.stdout: "roption1": { 2024-02-02T15:46:48.403 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "roption1", 2024-02-02T15:46:48.403 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.403 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.403 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.403 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.404 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.404 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.404 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.404 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.404 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.404 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.404 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.404 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.404 INFO:teuthology.orchestra.run.smithi001.stdout: "roption2": { 2024-02-02T15:46:48.404 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "roption2", 2024-02-02T15:46:48.405 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.405 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.405 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.405 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "xyz", 2024-02-02T15:46:48.405 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.405 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.405 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.405 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.405 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.406 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.406 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.406 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.406 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption1": { 2024-02-02T15:46:48.406 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption1", 2024-02-02T15:46:48.406 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.406 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.406 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.407 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.407 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.407 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.407 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.407 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.407 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.407 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.407 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.408 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.408 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption2": { 2024-02-02T15:46:48.408 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption2", 2024-02-02T15:46:48.408 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:48.408 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.408 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.408 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.408 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.409 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.409 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.409 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.409 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.409 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.409 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.409 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.409 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption3": { 2024-02-02T15:46:48.410 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption3", 2024-02-02T15:46:48.410 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-02-02T15:46:48.410 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.410 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.410 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.410 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.410 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.410 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.411 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.411 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.411 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.411 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.411 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.411 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption4": { 2024-02-02T15:46:48.411 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption4", 2024-02-02T15:46:48.411 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.412 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.412 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.412 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.412 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.412 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.412 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.412 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.412 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.413 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.413 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.413 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.413 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption5": { 2024-02-02T15:46:48.413 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption5", 2024-02-02T15:46:48.413 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.413 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.413 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.414 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.414 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.414 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.414 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.414 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.414 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.414 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.414 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.415 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.415 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption6": { 2024-02-02T15:46:48.415 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption6", 2024-02-02T15:46:48.415 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.415 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.415 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.415 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:48.416 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.416 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.416 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.416 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.416 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.416 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.416 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.417 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.417 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption7": { 2024-02-02T15:46:48.417 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption7", 2024-02-02T15:46:48.417 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:48.417 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.417 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.417 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.417 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1", 2024-02-02T15:46:48.417 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "42", 2024-02-02T15:46:48.418 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.418 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.418 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.418 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.418 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.418 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.418 INFO:teuthology.orchestra.run.smithi001.stdout: "testkey": { 2024-02-02T15:46:48.418 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "testkey", 2024-02-02T15:46:48.419 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.419 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.419 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.419 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.419 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.419 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.419 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.419 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.419 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.420 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.420 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.420 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.420 INFO:teuthology.orchestra.run.smithi001.stdout: "testlkey": { 2024-02-02T15:46:48.420 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "testlkey", 2024-02-02T15:46:48.420 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.420 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.420 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.421 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.421 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.421 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.421 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.421 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.421 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.421 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.421 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.422 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.422 INFO:teuthology.orchestra.run.smithi001.stdout: "testnewline": { 2024-02-02T15:46:48.422 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "testnewline", 2024-02-02T15:46:48.422 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.422 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.422 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.422 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.422 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.423 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.423 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.423 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.423 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.423 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.423 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.423 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.424 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.424 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.424 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:48.424 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "snap_schedule", 2024-02-02T15:46:48.424 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:48.424 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:48.424 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:48.424 INFO:teuthology.orchestra.run.smithi001.stdout: "allow_m_granularity": { 2024-02-02T15:46:48.425 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "allow_m_granularity", 2024-02-02T15:46:48.425 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.425 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.425 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.425 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.425 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.425 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.426 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.426 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "allow minute scheduled snapshots", 2024-02-02T15:46:48.426 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.426 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.426 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.426 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.426 INFO:teuthology.orchestra.run.smithi001.stdout: "dump_on_update": { 2024-02-02T15:46:48.427 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "dump_on_update", 2024-02-02T15:46:48.427 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.427 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.427 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.427 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.427 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.427 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.428 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.428 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "dump database to debug log on update", 2024-02-02T15:46:48.428 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.428 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.428 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.428 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.428 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:48.428 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:48.429 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.429 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.429 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.429 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.429 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.429 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.429 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.429 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.429 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.430 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.430 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.430 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.430 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.430 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.430 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.430 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.430 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.431 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.431 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.431 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:48.431 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:48.431 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.431 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.431 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.431 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.432 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.432 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.432 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.432 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.432 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.432 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.432 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.432 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.433 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:48.433 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:48.433 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.433 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.433 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.433 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:48.433 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.433 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.434 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.434 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.434 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.434 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.434 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.434 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.434 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.434 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.435 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.435 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.435 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.435 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.435 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.435 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:48.435 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:48.435 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.435 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.436 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.436 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.436 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.436 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.436 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.436 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.437 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.437 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.437 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.437 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.437 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.437 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.437 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:48.437 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "stats", 2024-02-02T15:46:48.437 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:48.438 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:48.438 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:48.438 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:48.438 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:48.438 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.438 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.438 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.438 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.439 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.439 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.439 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.439 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.439 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.439 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.439 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.439 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.439 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.439 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.440 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.440 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.440 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.440 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.440 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.440 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:48.440 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:48.440 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.441 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.441 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.441 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.441 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.441 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.441 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.441 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.441 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.441 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.441 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.442 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.442 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:48.442 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:48.442 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.442 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.442 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.442 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:48.442 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.442 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.443 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.443 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.443 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.443 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.443 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.443 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.443 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.443 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.443 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.444 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.444 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.444 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.444 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.444 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:48.444 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:48.444 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.445 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.445 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.445 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.445 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.445 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.445 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.445 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.445 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.445 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.446 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.446 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.446 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.446 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.446 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:48.446 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "status", 2024-02-02T15:46:48.446 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:48.447 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:48.447 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:48.447 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:48.447 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:48.447 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.447 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.447 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.447 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.447 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.448 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.448 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.448 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.448 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.448 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.448 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.448 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.448 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.449 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.449 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.449 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.449 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.449 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.449 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.449 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:48.449 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:48.449 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.450 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.450 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.450 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.450 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.450 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.450 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.450 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.450 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.451 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.451 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.451 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.451 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:48.451 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:48.451 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.451 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.451 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.452 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:48.452 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.452 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.452 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.452 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.452 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.452 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.452 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.453 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.453 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.453 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.453 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.453 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.453 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.453 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.453 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.453 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:48.454 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:48.454 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.454 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.454 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.454 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.454 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.454 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.454 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.454 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.454 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.455 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.455 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.455 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.455 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.455 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.455 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:48.455 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "telegraf", 2024-02-02T15:46:48.455 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:48.455 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:48.456 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:48.456 INFO:teuthology.orchestra.run.smithi001.stdout: "address": { 2024-02-02T15:46:48.456 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "address", 2024-02-02T15:46:48.456 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.456 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.456 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.456 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-02-02T15:46:48.456 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.456 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.457 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.457 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.457 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.457 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.457 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.457 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.457 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-02-02T15:46:48.457 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-02-02T15:46:48.457 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-02-02T15:46:48.458 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.458 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.458 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "15", 2024-02-02T15:46:48.458 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.458 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.458 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.458 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.458 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.458 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.459 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.459 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.459 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:48.459 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:48.459 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.459 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.459 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.459 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.460 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.460 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.460 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.460 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.460 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.460 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.460 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.461 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.461 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.461 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.461 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.461 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.461 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.461 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.461 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.461 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:48.461 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:48.462 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.462 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.462 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.462 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.462 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.462 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.462 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.462 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.462 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.462 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.463 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.463 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.463 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:48.463 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:48.463 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.463 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.463 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.464 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:48.464 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.464 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.464 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.464 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.464 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.464 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.465 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.465 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.465 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.465 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.465 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.465 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.465 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.465 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.466 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.466 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:48.466 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:48.466 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.466 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.466 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.466 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.466 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.466 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.467 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.467 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.467 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.467 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.467 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.467 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.467 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.467 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.468 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:48.468 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "telemetry", 2024-02-02T15:46:48.468 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:48.468 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:48.468 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:48.468 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_basic": { 2024-02-02T15:46:48.468 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_basic", 2024-02-02T15:46:48.468 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.469 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.469 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.469 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:48.469 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.469 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.469 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.469 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share basic cluster information (size, version)", 2024-02-02T15:46:48.470 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.470 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.470 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.470 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.470 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_crash": { 2024-02-02T15:46:48.470 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_crash", 2024-02-02T15:46:48.470 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.470 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.471 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.471 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:48.471 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.471 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.471 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.471 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-02-02T15:46:48.471 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.471 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.471 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.472 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.472 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_device": { 2024-02-02T15:46:48.472 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_device", 2024-02-02T15:46:48.472 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.472 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.472 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.472 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:48.472 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.473 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.473 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.473 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-02-02T15:46:48.473 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.473 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.473 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.473 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.474 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_ident": { 2024-02-02T15:46:48.474 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_ident", 2024-02-02T15:46:48.474 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.474 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.474 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.474 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.474 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.474 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.475 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.475 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-02-02T15:46:48.475 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.475 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.475 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.475 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.475 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_perf": { 2024-02-02T15:46:48.475 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_perf", 2024-02-02T15:46:48.476 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.476 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.476 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.476 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.476 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.476 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.476 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.476 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share various performance metrics of a cluster", 2024-02-02T15:46:48.477 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.477 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.477 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.477 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.477 INFO:teuthology.orchestra.run.smithi001.stdout: "contact": { 2024-02-02T15:46:48.477 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "contact", 2024-02-02T15:46:48.477 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.478 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.478 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.478 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.478 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.478 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.478 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.478 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.478 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.478 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.479 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.479 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.479 INFO:teuthology.orchestra.run.smithi001.stdout: "description": { 2024-02-02T15:46:48.479 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "description", 2024-02-02T15:46:48.479 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.479 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.479 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.479 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.480 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.480 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.480 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.480 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.480 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.480 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.480 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.480 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.480 INFO:teuthology.orchestra.run.smithi001.stdout: "device_url": { 2024-02-02T15:46:48.481 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "device_url", 2024-02-02T15:46:48.481 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.481 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.481 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.481 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-02-02T15:46:48.481 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.481 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.481 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.481 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.481 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.482 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.482 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.482 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.482 INFO:teuthology.orchestra.run.smithi001.stdout: "enabled": { 2024-02-02T15:46:48.482 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "enabled", 2024-02-02T15:46:48.482 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.482 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.482 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.482 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.483 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.483 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.483 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.483 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.483 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.483 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.483 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.483 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.483 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-02-02T15:46:48.483 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-02-02T15:46:48.484 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:48.484 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.484 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.484 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "24", 2024-02-02T15:46:48.484 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "8", 2024-02-02T15:46:48.484 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.484 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.484 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.484 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.485 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.485 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.485 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.485 INFO:teuthology.orchestra.run.smithi001.stdout: "last_opt_revision": { 2024-02-02T15:46:48.485 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "last_opt_revision", 2024-02-02T15:46:48.485 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:48.485 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.485 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.486 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "1", 2024-02-02T15:46:48.486 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.486 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.486 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.486 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.486 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.486 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.487 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.487 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.487 INFO:teuthology.orchestra.run.smithi001.stdout: "leaderboard": { 2024-02-02T15:46:48.487 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "leaderboard", 2024-02-02T15:46:48.487 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.487 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.487 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.487 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.488 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.488 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.488 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.488 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.488 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.488 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.488 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.488 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.489 INFO:teuthology.orchestra.run.smithi001.stdout: "leaderboard_description": { 2024-02-02T15:46:48.489 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "leaderboard_description", 2024-02-02T15:46:48.489 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.489 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.489 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.489 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.489 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.489 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.490 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.490 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.490 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.490 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.490 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.490 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.490 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:48.490 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:48.491 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.491 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.491 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.491 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.491 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.491 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.491 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.491 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.492 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.492 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.492 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.492 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.492 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.492 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.492 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.493 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.493 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.493 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.493 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.493 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:48.493 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:48.493 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.493 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.493 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.494 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.494 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.494 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.494 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.494 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.494 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.494 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.494 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.495 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.495 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:48.495 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:48.495 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.495 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.495 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.495 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:48.495 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.495 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.495 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.496 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.496 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.496 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.496 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.496 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.496 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.496 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.496 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.497 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.497 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.497 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.497 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.497 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:48.497 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:48.497 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.497 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.497 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.498 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.498 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.498 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.498 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.498 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.498 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.498 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.498 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.498 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.499 INFO:teuthology.orchestra.run.smithi001.stdout: "organization": { 2024-02-02T15:46:48.499 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "organization", 2024-02-02T15:46:48.499 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.499 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.499 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.499 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.499 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.499 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.500 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.500 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.500 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.500 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.500 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.500 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.500 INFO:teuthology.orchestra.run.smithi001.stdout: "proxy": { 2024-02-02T15:46:48.500 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "proxy", 2024-02-02T15:46:48.500 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.501 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.501 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.501 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.501 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.501 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.501 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.501 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.501 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.502 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.502 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.502 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.502 INFO:teuthology.orchestra.run.smithi001.stdout: "url": { 2024-02-02T15:46:48.502 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "url", 2024-02-02T15:46:48.502 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.502 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.502 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.503 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-02-02T15:46:48.503 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.503 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.503 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.503 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.503 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.503 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.503 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.504 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.504 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.504 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.504 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:48.504 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "test_orchestrator", 2024-02-02T15:46:48.504 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:48.504 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:48.504 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:48.505 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:48.505 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:48.505 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.505 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.505 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.505 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.505 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.505 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.506 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.506 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.506 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.506 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.506 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.506 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.506 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.506 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.507 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.507 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.507 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.507 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.507 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.507 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:48.507 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:48.507 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.508 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.508 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.508 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.508 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.508 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.508 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.508 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.508 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.509 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.509 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.509 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.509 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:48.509 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:48.509 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.509 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.510 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.510 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:48.510 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.510 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.510 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.510 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.510 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.511 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.511 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.511 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.511 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.511 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.511 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.511 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.512 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.512 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.512 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.512 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:48.512 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:48.512 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.512 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.512 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.513 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.513 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.513 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.513 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.513 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.513 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.513 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.513 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.513 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.514 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.514 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.514 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:48.514 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "volumes", 2024-02-02T15:46:48.514 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:48.514 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:48.514 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:48.514 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:48.514 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:48.514 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.515 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.515 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.515 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.515 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.515 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.515 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.515 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.515 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.515 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.516 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.516 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.516 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.516 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.516 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.516 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.516 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.516 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.516 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.517 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:48.517 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:48.517 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.517 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.517 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.517 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.517 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.517 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.518 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.518 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.518 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.518 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.518 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.518 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.518 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:48.519 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:48.519 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.519 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.519 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.519 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:48.519 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.519 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.519 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.519 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.520 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.520 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.520 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.520 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.520 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.520 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.520 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.521 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.521 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.521 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.521 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.521 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:48.521 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:48.521 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.521 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.522 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.522 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.522 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.522 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.522 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.522 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.522 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.522 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.523 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.523 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.523 INFO:teuthology.orchestra.run.smithi001.stdout: "max_concurrent_clones": { 2024-02-02T15:46:48.523 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_concurrent_clones", 2024-02-02T15:46:48.523 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:48.523 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.523 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.523 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "4", 2024-02-02T15:46:48.524 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.524 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.524 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.524 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Number of asynchronous cloner threads", 2024-02-02T15:46:48.524 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.524 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.524 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.524 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.525 INFO:teuthology.orchestra.run.smithi001.stdout: "snapshot_clone_delay": { 2024-02-02T15:46:48.525 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "snapshot_clone_delay", 2024-02-02T15:46:48.525 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:48.525 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.525 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.525 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-02-02T15:46:48.525 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.526 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.526 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.526 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-02-02T15:46:48.526 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.526 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.526 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.526 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.526 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.527 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.527 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:48.527 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "zabbix", 2024-02-02T15:46:48.527 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:48.527 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:48.527 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:48.527 INFO:teuthology.orchestra.run.smithi001.stdout: "discovery_interval": { 2024-02-02T15:46:48.527 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "discovery_interval", 2024-02-02T15:46:48.528 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-02-02T15:46:48.528 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.528 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.528 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "100", 2024-02-02T15:46:48.528 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.528 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.528 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.528 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.529 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.529 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.529 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.529 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.529 INFO:teuthology.orchestra.run.smithi001.stdout: "identifier": { 2024-02-02T15:46:48.529 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "identifier", 2024-02-02T15:46:48.529 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.529 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.529 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.530 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.530 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.530 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.530 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.530 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.530 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.530 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.530 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.530 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.531 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-02-02T15:46:48.531 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-02-02T15:46:48.531 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-02-02T15:46:48.531 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.531 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.531 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-02-02T15:46:48.531 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.531 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.532 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.532 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.532 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.532 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.532 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.532 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.532 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:48.532 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:48.532 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.533 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.533 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.533 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.533 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.533 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.533 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.533 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.533 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.534 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.534 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.534 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.534 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.534 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.534 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.534 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.534 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.534 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.534 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.535 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:48.535 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:48.535 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.535 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.535 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.535 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.535 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.535 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.535 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.536 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.536 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.536 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.536 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.536 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.536 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:48.536 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:48.536 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.536 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.537 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.537 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:48.537 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.537 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.537 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.537 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.537 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.537 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.537 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.537 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.538 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.538 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.538 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.538 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.538 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.538 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.538 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.538 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:48.538 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:48.539 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.539 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.539 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.539 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.539 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.539 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.539 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.539 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.539 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.539 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.540 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.540 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.540 INFO:teuthology.orchestra.run.smithi001.stdout: "zabbix_host": { 2024-02-02T15:46:48.540 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "zabbix_host", 2024-02-02T15:46:48.540 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.540 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.540 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.540 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.541 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.541 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.541 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.541 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.541 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.541 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.541 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.541 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.541 INFO:teuthology.orchestra.run.smithi001.stdout: "zabbix_port": { 2024-02-02T15:46:48.542 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "zabbix_port", 2024-02-02T15:46:48.542 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:48.542 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.542 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.542 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10051", 2024-02-02T15:46:48.542 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.542 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.542 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.542 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.543 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.543 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.543 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.543 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.543 INFO:teuthology.orchestra.run.smithi001.stdout: "zabbix_sender": { 2024-02-02T15:46:48.543 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "zabbix_sender", 2024-02-02T15:46:48.543 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.544 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.544 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.544 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-02-02T15:46:48.544 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.544 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.544 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.544 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.544 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.545 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.545 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.545 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.545 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.545 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.545 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-02-02T15:46:48.545 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.546 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:48.546 INFO:teuthology.orchestra.run.smithi001.stdout: "gid": 25600, 2024-02-02T15:46:48.546 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "z", 2024-02-02T15:46:48.546 INFO:teuthology.orchestra.run.smithi001.stdout: "mgr_features": 4540138322906710015, 2024-02-02T15:46:48.546 INFO:teuthology.orchestra.run.smithi001.stdout: "available_modules": [ 2024-02-02T15:46:48.546 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:48.546 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "alerts", 2024-02-02T15:46:48.546 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:48.547 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:48.547 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:48.547 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-02-02T15:46:48.547 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-02-02T15:46:48.547 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-02-02T15:46:48.547 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.547 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.547 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-02-02T15:46:48.547 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.548 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.548 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.548 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "How frequently to reexamine health status", 2024-02-02T15:46:48.548 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.548 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.548 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.548 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.548 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:48.548 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:48.549 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.549 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.549 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.549 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.549 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.549 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.549 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.549 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.549 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.550 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.550 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.550 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.550 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.550 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.550 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.550 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.550 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.550 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.551 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.551 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:48.551 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:48.551 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.551 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.551 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.551 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.551 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.552 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.552 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.552 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.552 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.552 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.552 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.552 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.552 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:48.552 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:48.553 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.553 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.553 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.553 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:48.553 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.553 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.554 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.554 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.554 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.554 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.554 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.554 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.554 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.554 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.554 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.554 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.555 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.555 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.555 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.555 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:48.555 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:48.555 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.555 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.555 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.555 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.556 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.556 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.556 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.556 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.556 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.556 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.556 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.556 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.556 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_destination": { 2024-02-02T15:46:48.557 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_destination", 2024-02-02T15:46:48.557 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.557 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.557 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.557 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.557 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.557 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.557 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.557 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Email address to send alerts to", 2024-02-02T15:46:48.557 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.558 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.558 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.558 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.558 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_from_name": { 2024-02-02T15:46:48.558 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_from_name", 2024-02-02T15:46:48.558 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.558 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.558 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.558 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "Ceph", 2024-02-02T15:46:48.558 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.559 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.559 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.559 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Email From: name", 2024-02-02T15:46:48.559 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.559 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.559 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.559 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.559 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_host": { 2024-02-02T15:46:48.559 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_host", 2024-02-02T15:46:48.559 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.560 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.560 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.560 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.560 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.560 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.560 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.560 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "SMTP server", 2024-02-02T15:46:48.560 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.560 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.560 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.561 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.561 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_password": { 2024-02-02T15:46:48.561 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_password", 2024-02-02T15:46:48.561 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.561 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.561 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.561 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.561 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.561 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.562 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.562 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Password to authenticate with", 2024-02-02T15:46:48.562 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.562 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.562 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.562 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.562 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_port": { 2024-02-02T15:46:48.562 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_port", 2024-02-02T15:46:48.562 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:48.562 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.563 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.563 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "465", 2024-02-02T15:46:48.563 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.563 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.563 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.563 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "SMTP port", 2024-02-02T15:46:48.563 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.563 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.563 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.563 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.564 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_sender": { 2024-02-02T15:46:48.564 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_sender", 2024-02-02T15:46:48.564 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.564 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.564 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.564 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.564 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.564 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.565 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.565 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "SMTP envelope sender", 2024-02-02T15:46:48.565 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.565 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.565 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.565 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.565 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_ssl": { 2024-02-02T15:46:48.566 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_ssl", 2024-02-02T15:46:48.566 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.566 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.566 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.566 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:48.566 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.566 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.566 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.566 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Use SSL to connect to SMTP server", 2024-02-02T15:46:48.567 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.567 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.567 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.567 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.567 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_user": { 2024-02-02T15:46:48.567 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_user", 2024-02-02T15:46:48.567 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.567 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.568 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.568 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.568 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.568 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.568 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.568 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "User to authenticate as", 2024-02-02T15:46:48.568 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.568 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.568 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.569 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.569 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.569 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.569 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:48.569 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "balancer", 2024-02-02T15:46:48.569 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:48.569 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:48.569 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:48.569 INFO:teuthology.orchestra.run.smithi001.stdout: "active": { 2024-02-02T15:46:48.570 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "active", 2024-02-02T15:46:48.570 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.570 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.570 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.570 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:48.570 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.570 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.571 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.571 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "automatically balance PGs across cluster", 2024-02-02T15:46:48.571 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.571 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.571 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.571 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.571 INFO:teuthology.orchestra.run.smithi001.stdout: "begin_time": { 2024-02-02T15:46:48.571 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "begin_time", 2024-02-02T15:46:48.572 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.572 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.572 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.572 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0000", 2024-02-02T15:46:48.572 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.572 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.572 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.573 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "beginning time of day to automatically balance", 2024-02-02T15:46:48.573 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-02-02T15:46:48.573 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.573 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.573 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.573 INFO:teuthology.orchestra.run.smithi001.stdout: "begin_weekday": { 2024-02-02T15:46:48.573 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "begin_weekday", 2024-02-02T15:46:48.573 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-02-02T15:46:48.574 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.574 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.574 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-02-02T15:46:48.574 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "0", 2024-02-02T15:46:48.574 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "6", 2024-02-02T15:46:48.574 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.574 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-02-02T15:46:48.574 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-02-02T15:46:48.575 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.575 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.575 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.575 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_compat_max_iterations": { 2024-02-02T15:46:48.575 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crush_compat_max_iterations", 2024-02-02T15:46:48.575 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-02-02T15:46:48.575 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.575 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.576 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "25", 2024-02-02T15:46:48.576 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1", 2024-02-02T15:46:48.576 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "250", 2024-02-02T15:46:48.576 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.576 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-02-02T15:46:48.576 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.576 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.577 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.577 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.577 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_compat_metrics": { 2024-02-02T15:46:48.577 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crush_compat_metrics", 2024-02-02T15:46:48.577 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.577 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.577 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.577 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "pgs,objects,bytes", 2024-02-02T15:46:48.578 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.578 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.578 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.578 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-02-02T15:46:48.578 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-02-02T15:46:48.578 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.578 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.579 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.579 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_compat_step": { 2024-02-02T15:46:48.579 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crush_compat_step", 2024-02-02T15:46:48.579 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-02-02T15:46:48.579 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.579 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.579 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0.5", 2024-02-02T15:46:48.579 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "0.001", 2024-02-02T15:46:48.580 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "0.999", 2024-02-02T15:46:48.580 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.580 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "aggressiveness of optimization", 2024-02-02T15:46:48.580 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-02-02T15:46:48.580 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.580 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.580 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.580 INFO:teuthology.orchestra.run.smithi001.stdout: "end_time": { 2024-02-02T15:46:48.580 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "end_time", 2024-02-02T15:46:48.581 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.581 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.581 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.581 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "2359", 2024-02-02T15:46:48.581 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.581 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.581 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.581 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "ending time of day to automatically balance", 2024-02-02T15:46:48.582 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-02-02T15:46:48.582 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.582 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.582 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.582 INFO:teuthology.orchestra.run.smithi001.stdout: "end_weekday": { 2024-02-02T15:46:48.582 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "end_weekday", 2024-02-02T15:46:48.582 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-02-02T15:46:48.582 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.583 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.583 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-02-02T15:46:48.583 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "0", 2024-02-02T15:46:48.583 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "6", 2024-02-02T15:46:48.583 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.583 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-02-02T15:46:48.583 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-02-02T15:46:48.583 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.584 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.584 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.584 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:48.584 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:48.584 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.584 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.584 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.585 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.585 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.585 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.585 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.585 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.585 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.585 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.586 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.586 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.586 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.586 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.586 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.586 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.586 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.586 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.587 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.587 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:48.587 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:48.587 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.587 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.587 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.587 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.587 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.587 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.588 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.588 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.588 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.588 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.588 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.588 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.588 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:48.588 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:48.588 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.589 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.589 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.589 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:48.589 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.589 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.589 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.589 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.589 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.589 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.590 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.590 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.590 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.590 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.590 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.590 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.590 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.590 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.590 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.591 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:48.591 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:48.591 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.591 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.591 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.591 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.591 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.591 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.591 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.591 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.592 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.592 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.592 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.592 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.592 INFO:teuthology.orchestra.run.smithi001.stdout: "min_score": { 2024-02-02T15:46:48.592 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "min_score", 2024-02-02T15:46:48.592 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-02-02T15:46:48.592 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.593 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.593 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-02-02T15:46:48.593 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.593 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.593 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.593 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-02-02T15:46:48.593 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.593 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.593 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.594 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.594 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": { 2024-02-02T15:46:48.594 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mode", 2024-02-02T15:46:48.594 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.594 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.594 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.594 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "upmap", 2024-02-02T15:46:48.594 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.594 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.595 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.595 INFO:teuthology.orchestra.run.smithi001.stdout: "crush-compat", 2024-02-02T15:46:48.595 INFO:teuthology.orchestra.run.smithi001.stdout: "none", 2024-02-02T15:46:48.595 INFO:teuthology.orchestra.run.smithi001.stdout: "upmap" 2024-02-02T15:46:48.595 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.595 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Balancer mode", 2024-02-02T15:46:48.595 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.595 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.596 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.596 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.596 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_ids": { 2024-02-02T15:46:48.596 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "pool_ids", 2024-02-02T15:46:48.596 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.596 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.596 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.596 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.597 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.597 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.597 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.597 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-02-02T15:46:48.597 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.597 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.597 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.597 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.598 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-02-02T15:46:48.598 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-02-02T15:46:48.598 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-02-02T15:46:48.598 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.598 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.598 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-02-02T15:46:48.598 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.598 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.599 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.599 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-02-02T15:46:48.599 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.599 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.599 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.599 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.599 INFO:teuthology.orchestra.run.smithi001.stdout: "upmap_max_deviation": { 2024-02-02T15:46:48.600 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "upmap_max_deviation", 2024-02-02T15:46:48.600 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:48.600 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.600 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.600 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2024-02-02T15:46:48.603 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1", 2024-02-02T15:46:48.603 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.603 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.603 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "deviation below which no optimization is attempted", 2024-02-02T15:46:48.603 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-02-02T15:46:48.603 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.604 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.604 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.604 INFO:teuthology.orchestra.run.smithi001.stdout: "upmap_max_optimizations": { 2024-02-02T15:46:48.604 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "upmap_max_optimizations", 2024-02-02T15:46:48.604 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-02-02T15:46:48.604 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.604 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.604 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-02-02T15:46:48.605 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.605 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.605 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.605 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-02-02T15:46:48.605 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.605 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.605 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.605 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.606 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.606 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.606 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:48.606 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cephadm", 2024-02-02T15:46:48.606 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:48.606 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:48.606 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:48.607 INFO:teuthology.orchestra.run.smithi001.stdout: "agent_down_multiplier": { 2024-02-02T15:46:48.607 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "agent_down_multiplier", 2024-02-02T15:46:48.607 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-02-02T15:46:48.607 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.607 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.607 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "3.0", 2024-02-02T15:46:48.607 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.607 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.608 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.608 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-02-02T15:46:48.608 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.608 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.608 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.608 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.608 INFO:teuthology.orchestra.run.smithi001.stdout: "agent_refresh_rate": { 2024-02-02T15:46:48.608 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "agent_refresh_rate", 2024-02-02T15:46:48.609 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-02-02T15:46:48.609 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.609 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.609 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "20", 2024-02-02T15:46:48.609 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.609 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.609 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.609 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-02-02T15:46:48.609 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.610 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.610 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.610 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.610 INFO:teuthology.orchestra.run.smithi001.stdout: "agent_starting_port": { 2024-02-02T15:46:48.610 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "agent_starting_port", 2024-02-02T15:46:48.610 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:48.610 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.610 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.611 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "4721", 2024-02-02T15:46:48.611 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.611 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.611 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.611 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-02-02T15:46:48.611 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.611 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.612 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.612 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.612 INFO:teuthology.orchestra.run.smithi001.stdout: "allow_ptrace": { 2024-02-02T15:46:48.612 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "allow_ptrace", 2024-02-02T15:46:48.612 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.612 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.612 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.612 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.613 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.613 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.613 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.613 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-02-02T15:46:48.613 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-02-02T15:46:48.613 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.613 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.614 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.614 INFO:teuthology.orchestra.run.smithi001.stdout: "autotune_interval": { 2024-02-02T15:46:48.614 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "autotune_interval", 2024-02-02T15:46:48.614 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-02-02T15:46:48.614 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.614 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.614 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-02-02T15:46:48.614 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.614 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.615 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.615 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to autotune daemon memory", 2024-02-02T15:46:48.615 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.615 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.615 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.615 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.615 INFO:teuthology.orchestra.run.smithi001.stdout: "autotune_memory_target_ratio": { 2024-02-02T15:46:48.615 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "autotune_memory_target_ratio", 2024-02-02T15:46:48.616 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-02-02T15:46:48.616 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.616 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.616 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0.7", 2024-02-02T15:46:48.616 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.616 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.616 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.616 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-02-02T15:46:48.616 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.617 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.617 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.617 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.617 INFO:teuthology.orchestra.run.smithi001.stdout: "cgroups_split": { 2024-02-02T15:46:48.617 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cgroups_split", 2024-02-02T15:46:48.617 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.617 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.617 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.617 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:48.618 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.618 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.618 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.618 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-02-02T15:46:48.618 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.618 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.618 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.618 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.619 INFO:teuthology.orchestra.run.smithi001.stdout: "config_checks_enabled": { 2024-02-02T15:46:48.619 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "config_checks_enabled", 2024-02-02T15:46:48.619 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.619 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.619 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.619 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.619 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.619 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.619 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.620 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-02-02T15:46:48.620 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.620 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.620 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.620 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.620 INFO:teuthology.orchestra.run.smithi001.stdout: "config_dashboard": { 2024-02-02T15:46:48.620 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "config_dashboard", 2024-02-02T15:46:48.621 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.621 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.621 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.621 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:48.621 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.621 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.621 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.621 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-02-02T15:46:48.622 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.622 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.622 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.622 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.622 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_alertmanager": { 2024-02-02T15:46:48.622 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_alertmanager", 2024-02-02T15:46:48.622 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.623 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.623 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.623 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-02-02T15:46:48.623 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.623 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.623 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.623 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus container image", 2024-02-02T15:46:48.624 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.624 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.624 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.624 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.624 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_base": { 2024-02-02T15:46:48.624 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_base", 2024-02-02T15:46:48.624 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.624 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.625 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.625 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/ceph", 2024-02-02T15:46:48.625 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.625 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.625 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.625 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Container image name, without the tag", 2024-02-02T15:46:48.625 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.625 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.626 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.626 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.626 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_elasticsearch": { 2024-02-02T15:46:48.626 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_elasticsearch", 2024-02-02T15:46:48.626 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.626 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.626 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.626 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-02-02T15:46:48.626 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.627 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.627 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.627 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "elasticsearch container image", 2024-02-02T15:46:48.627 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.627 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.627 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.627 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.628 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_grafana": { 2024-02-02T15:46:48.628 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_grafana", 2024-02-02T15:46:48.628 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.628 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.628 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.628 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2024-02-02T15:46:48.628 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.628 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.629 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.629 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus container image", 2024-02-02T15:46:48.629 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.629 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.629 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.629 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.629 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_haproxy": { 2024-02-02T15:46:48.629 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_haproxy", 2024-02-02T15:46:48.629 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.629 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.630 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.630 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-02-02T15:46:48.630 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.630 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.630 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.630 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "HAproxy container image", 2024-02-02T15:46:48.630 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.631 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.631 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.631 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.631 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_jaeger_agent": { 2024-02-02T15:46:48.631 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_jaeger_agent", 2024-02-02T15:46:48.631 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.631 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.632 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.632 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-02-02T15:46:48.632 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.632 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.632 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.632 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Jaeger agent container image", 2024-02-02T15:46:48.632 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.632 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.633 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.633 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.633 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_jaeger_collector": { 2024-02-02T15:46:48.633 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_jaeger_collector", 2024-02-02T15:46:48.633 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.633 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.633 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.633 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-02-02T15:46:48.634 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.634 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.634 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.634 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Jaeger collector container image", 2024-02-02T15:46:48.634 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.634 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.634 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.634 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.635 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_jaeger_query": { 2024-02-02T15:46:48.635 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_jaeger_query", 2024-02-02T15:46:48.635 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.635 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.635 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.635 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-02-02T15:46:48.635 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.636 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.636 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.636 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Jaeger query container image", 2024-02-02T15:46:48.636 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.636 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.636 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.636 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.636 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_keepalived": { 2024-02-02T15:46:48.636 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_keepalived", 2024-02-02T15:46:48.637 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.637 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.637 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.637 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-02-02T15:46:48.637 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.637 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.637 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.637 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Keepalived container image", 2024-02-02T15:46:48.637 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.638 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.638 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.638 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.638 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_loki": { 2024-02-02T15:46:48.638 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_loki", 2024-02-02T15:46:48.638 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.638 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.639 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.639 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-02-02T15:46:48.639 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.639 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.639 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.639 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Loki container image", 2024-02-02T15:46:48.639 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.639 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.639 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.640 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.640 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_node_exporter": { 2024-02-02T15:46:48.640 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_node_exporter", 2024-02-02T15:46:48.640 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.640 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.640 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.640 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-02-02T15:46:48.640 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.641 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.641 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.641 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus container image", 2024-02-02T15:46:48.641 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.641 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.641 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.641 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.641 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_nvmeof": { 2024-02-02T15:46:48.641 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_nvmeof", 2024-02-02T15:46:48.642 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.642 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.642 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.642 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/nvmeof:0.0.2", 2024-02-02T15:46:48.642 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.642 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.642 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.642 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Nvme-of container image", 2024-02-02T15:46:48.642 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.643 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.643 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.643 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.643 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_prometheus": { 2024-02-02T15:46:48.643 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_prometheus", 2024-02-02T15:46:48.643 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.643 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.643 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.643 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-02-02T15:46:48.643 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.644 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.644 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.644 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus container image", 2024-02-02T15:46:48.644 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.644 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.644 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.644 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.645 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_promtail": { 2024-02-02T15:46:48.645 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_promtail", 2024-02-02T15:46:48.645 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.645 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.645 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.645 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-02-02T15:46:48.645 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.645 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.646 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.646 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Promtail container image", 2024-02-02T15:46:48.646 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.646 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.646 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.646 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.646 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_snmp_gateway": { 2024-02-02T15:46:48.646 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_snmp_gateway", 2024-02-02T15:46:48.646 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.647 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.647 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.647 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-02-02T15:46:48.647 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.647 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.647 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.647 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "SNMP Gateway container image", 2024-02-02T15:46:48.647 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.647 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.648 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.648 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.648 INFO:teuthology.orchestra.run.smithi001.stdout: "container_init": { 2024-02-02T15:46:48.648 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_init", 2024-02-02T15:46:48.648 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.648 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.648 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.648 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:48.648 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.649 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.649 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.649 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Run podman/docker with `--init`", 2024-02-02T15:46:48.649 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.649 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.649 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.649 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.649 INFO:teuthology.orchestra.run.smithi001.stdout: "daemon_cache_timeout": { 2024-02-02T15:46:48.649 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "daemon_cache_timeout", 2024-02-02T15:46:48.650 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-02-02T15:46:48.650 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.650 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.650 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-02-02T15:46:48.650 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.650 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.650 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.650 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-02-02T15:46:48.651 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.651 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.651 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.651 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.651 INFO:teuthology.orchestra.run.smithi001.stdout: "default_cephadm_command_timeout": { 2024-02-02T15:46:48.651 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "default_cephadm_command_timeout", 2024-02-02T15:46:48.651 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-02-02T15:46:48.651 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.651 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.652 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "900", 2024-02-02T15:46:48.652 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.652 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.652 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.652 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-02-02T15:46:48.652 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.652 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.652 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.652 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.653 INFO:teuthology.orchestra.run.smithi001.stdout: "default_registry": { 2024-02-02T15:46:48.653 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "default_registry", 2024-02-02T15:46:48.653 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.653 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.653 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.653 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "docker.io", 2024-02-02T15:46:48.653 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.653 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.653 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.654 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-02-02T15:46:48.654 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.654 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.654 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.654 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.654 INFO:teuthology.orchestra.run.smithi001.stdout: "device_cache_timeout": { 2024-02-02T15:46:48.654 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "device_cache_timeout", 2024-02-02T15:46:48.654 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-02-02T15:46:48.655 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.655 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.655 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "1800", 2024-02-02T15:46:48.655 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.655 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.655 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.655 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "seconds to cache device inventory", 2024-02-02T15:46:48.655 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.655 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.656 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.656 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.656 INFO:teuthology.orchestra.run.smithi001.stdout: "device_enhanced_scan": { 2024-02-02T15:46:48.656 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "device_enhanced_scan", 2024-02-02T15:46:48.656 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.656 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.656 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.656 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.656 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.657 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.657 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.657 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Use libstoragemgmt during device scans", 2024-02-02T15:46:48.657 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.657 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.657 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.657 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.657 INFO:teuthology.orchestra.run.smithi001.stdout: "facts_cache_timeout": { 2024-02-02T15:46:48.657 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "facts_cache_timeout", 2024-02-02T15:46:48.658 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-02-02T15:46:48.658 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.658 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.658 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-02-02T15:46:48.658 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.658 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.658 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.658 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "seconds to cache host facts data", 2024-02-02T15:46:48.658 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.659 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.659 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.659 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.659 INFO:teuthology.orchestra.run.smithi001.stdout: "host_check_interval": { 2024-02-02T15:46:48.659 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "host_check_interval", 2024-02-02T15:46:48.659 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-02-02T15:46:48.659 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.659 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.659 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-02-02T15:46:48.659 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.660 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.660 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.660 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to perform a host check", 2024-02-02T15:46:48.660 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.660 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.660 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.660 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.660 INFO:teuthology.orchestra.run.smithi001.stdout: "inventory_list_all": { 2024-02-02T15:46:48.660 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "inventory_list_all", 2024-02-02T15:46:48.661 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.661 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.661 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.661 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.661 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.661 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.661 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.661 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-02-02T15:46:48.661 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.661 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.662 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.662 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.662 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:48.662 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:48.662 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.662 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.662 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.662 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.663 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.663 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.663 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.663 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.663 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.663 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.663 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.663 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.663 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.663 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.664 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.664 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.664 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.664 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.664 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.664 INFO:teuthology.orchestra.run.smithi001.stdout: "log_refresh_metadata": { 2024-02-02T15:46:48.664 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_refresh_metadata", 2024-02-02T15:46:48.664 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.664 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.665 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.665 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.665 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.665 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.665 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.665 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-02-02T15:46:48.665 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.665 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.665 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.666 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.666 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:48.666 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:48.666 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.666 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.666 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.666 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:48.667 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.667 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.667 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.667 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-02-02T15:46:48.667 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.667 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.667 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.667 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.667 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:48.668 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:48.668 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.668 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.668 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.668 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:48.668 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.668 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.668 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.668 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.669 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.669 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.669 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.669 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.669 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.669 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.669 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.669 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.669 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.669 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.670 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.670 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:48.670 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:48.670 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.670 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.670 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.670 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.670 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.670 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.671 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.671 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.671 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.671 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.671 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.671 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.671 INFO:teuthology.orchestra.run.smithi001.stdout: "manage_etc_ceph_ceph_conf": { 2024-02-02T15:46:48.671 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-02-02T15:46:48.672 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.672 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.672 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.672 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.672 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.672 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.672 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.672 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-02-02T15:46:48.673 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.673 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.673 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.673 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.673 INFO:teuthology.orchestra.run.smithi001.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-02-02T15:46:48.673 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-02-02T15:46:48.673 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.673 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.673 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.674 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "*", 2024-02-02T15:46:48.674 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.674 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.674 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.674 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-02-02T15:46:48.674 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.674 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.674 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.674 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.675 INFO:teuthology.orchestra.run.smithi001.stdout: "max_count_per_host": { 2024-02-02T15:46:48.675 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_count_per_host", 2024-02-02T15:46:48.675 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:48.675 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.675 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.675 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-02-02T15:46:48.675 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.675 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.675 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.676 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "max number of daemons per service per host", 2024-02-02T15:46:48.676 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.676 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.676 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.676 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.676 INFO:teuthology.orchestra.run.smithi001.stdout: "max_osd_draining_count": { 2024-02-02T15:46:48.676 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_osd_draining_count", 2024-02-02T15:46:48.676 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:48.677 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.677 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.677 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-02-02T15:46:48.677 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.677 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.677 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.677 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-02-02T15:46:48.677 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.677 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.677 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.678 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.678 INFO:teuthology.orchestra.run.smithi001.stdout: "migration_current": { 2024-02-02T15:46:48.678 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "migration_current", 2024-02-02T15:46:48.678 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:48.678 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.678 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.678 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.678 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.678 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.679 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.679 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "internal - do not modify", 2024-02-02T15:46:48.679 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.679 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.679 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.679 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.679 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": { 2024-02-02T15:46:48.679 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mode", 2024-02-02T15:46:48.679 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.679 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.680 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.680 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "root", 2024-02-02T15:46:48.680 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.680 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.680 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.680 INFO:teuthology.orchestra.run.smithi001.stdout: "cephadm-package", 2024-02-02T15:46:48.680 INFO:teuthology.orchestra.run.smithi001.stdout: "root" 2024-02-02T15:46:48.680 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.680 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "mode for remote execution of cephadm", 2024-02-02T15:46:48.681 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.681 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.681 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.681 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.681 INFO:teuthology.orchestra.run.smithi001.stdout: "prometheus_alerts_path": { 2024-02-02T15:46:48.681 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "prometheus_alerts_path", 2024-02-02T15:46:48.681 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.681 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.682 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.682 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-02-02T15:46:48.682 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.682 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.682 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.682 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-02-02T15:46:48.682 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.683 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.683 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.683 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.683 INFO:teuthology.orchestra.run.smithi001.stdout: "registry_insecure": { 2024-02-02T15:46:48.683 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "registry_insecure", 2024-02-02T15:46:48.683 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.683 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.683 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.683 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.684 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.684 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.684 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.684 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-02-02T15:46:48.684 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.684 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.684 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.684 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.685 INFO:teuthology.orchestra.run.smithi001.stdout: "registry_password": { 2024-02-02T15:46:48.685 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "registry_password", 2024-02-02T15:46:48.685 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.685 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.685 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.685 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.685 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.685 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.685 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.686 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-02-02T15:46:48.686 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.686 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.686 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.686 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.686 INFO:teuthology.orchestra.run.smithi001.stdout: "registry_url": { 2024-02-02T15:46:48.686 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "registry_url", 2024-02-02T15:46:48.686 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.687 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.687 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.687 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.687 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.687 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.687 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.687 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-02-02T15:46:48.687 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.687 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.688 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.688 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.688 INFO:teuthology.orchestra.run.smithi001.stdout: "registry_username": { 2024-02-02T15:46:48.688 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "registry_username", 2024-02-02T15:46:48.688 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.688 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.688 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.688 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.688 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.688 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.689 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.689 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-02-02T15:46:48.689 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.689 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.689 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.689 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.689 INFO:teuthology.orchestra.run.smithi001.stdout: "secure_monitoring_stack": { 2024-02-02T15:46:48.689 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "secure_monitoring_stack", 2024-02-02T15:46:48.689 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.690 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.690 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.690 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.690 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.690 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.690 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.690 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-02-02T15:46:48.690 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.691 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.691 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.691 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.691 INFO:teuthology.orchestra.run.smithi001.stdout: "service_discovery_port": { 2024-02-02T15:46:48.691 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "service_discovery_port", 2024-02-02T15:46:48.691 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:48.691 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.691 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.692 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8765", 2024-02-02T15:46:48.692 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.692 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.692 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.692 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "cephadm service discovery port", 2024-02-02T15:46:48.692 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.692 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.692 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.693 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.693 INFO:teuthology.orchestra.run.smithi001.stdout: "ssh_config_file": { 2024-02-02T15:46:48.693 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ssh_config_file", 2024-02-02T15:46:48.693 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.693 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.693 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.693 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.693 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.693 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.694 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.694 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-02-02T15:46:48.694 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.694 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.694 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.694 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.694 INFO:teuthology.orchestra.run.smithi001.stdout: "use_agent": { 2024-02-02T15:46:48.694 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "use_agent", 2024-02-02T15:46:48.694 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.694 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.695 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.695 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.695 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.695 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.695 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.695 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-02-02T15:46:48.695 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.695 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.696 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.696 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.696 INFO:teuthology.orchestra.run.smithi001.stdout: "use_repo_digest": { 2024-02-02T15:46:48.696 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "use_repo_digest", 2024-02-02T15:46:48.696 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.696 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.696 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.696 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:48.697 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.697 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.697 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.697 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-02-02T15:46:48.697 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.697 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.697 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.698 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.698 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_on_failed_host_check": { 2024-02-02T15:46:48.698 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_on_failed_host_check", 2024-02-02T15:46:48.698 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.698 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.698 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.698 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:48.698 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.698 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.698 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.699 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "raise a health warning if the host check fails", 2024-02-02T15:46:48.699 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.699 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.699 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.699 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.699 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_on_stray_daemons": { 2024-02-02T15:46:48.699 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_on_stray_daemons", 2024-02-02T15:46:48.699 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.699 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.700 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.700 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:48.700 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.700 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.700 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.700 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-02-02T15:46:48.701 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.701 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.701 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.701 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.701 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_on_stray_hosts": { 2024-02-02T15:46:48.701 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_on_stray_hosts", 2024-02-02T15:46:48.701 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.701 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.701 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.702 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:48.702 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.702 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.702 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.702 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-02-02T15:46:48.702 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.702 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.702 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.703 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.703 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.703 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.703 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:48.703 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crash", 2024-02-02T15:46:48.703 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:48.704 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:48.704 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:48.704 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:48.704 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:48.704 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.704 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.704 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.704 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.704 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.705 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.705 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.705 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.705 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.705 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.705 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.705 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.705 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.705 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.706 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.706 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.706 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.706 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.706 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.706 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:48.706 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:48.706 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.707 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.707 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.707 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.707 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.707 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.707 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.707 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.707 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.707 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.708 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.708 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.708 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:48.708 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:48.708 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.708 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.708 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.708 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:48.709 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.709 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.709 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.709 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.709 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.709 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.709 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.710 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.710 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.710 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.710 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.710 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.710 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.710 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.710 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.711 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:48.711 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:48.711 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.711 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.711 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.711 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.711 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.711 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.712 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.712 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.712 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.712 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.712 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.712 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.712 INFO:teuthology.orchestra.run.smithi001.stdout: "retain_interval": { 2024-02-02T15:46:48.712 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "retain_interval", 2024-02-02T15:46:48.713 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-02-02T15:46:48.713 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.713 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.713 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "31536000", 2024-02-02T15:46:48.713 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.713 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.713 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.713 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how long to retain crashes before pruning them", 2024-02-02T15:46:48.714 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.714 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.714 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.714 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.714 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_recent_interval": { 2024-02-02T15:46:48.714 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_recent_interval", 2024-02-02T15:46:48.714 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-02-02T15:46:48.715 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.715 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.715 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "1209600", 2024-02-02T15:46:48.715 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.715 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.715 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.715 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "time interval in which to warn about recent crashes", 2024-02-02T15:46:48.715 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.716 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.716 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.716 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.716 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.716 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.716 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:48.716 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "dashboard", 2024-02-02T15:46:48.716 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:48.717 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:48.717 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:48.717 INFO:teuthology.orchestra.run.smithi001.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-02-02T15:46:48.717 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-02-02T15:46:48.717 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:48.717 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.717 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.718 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-02-02T15:46:48.718 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.718 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.718 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.718 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.718 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.718 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.718 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.719 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.719 INFO:teuthology.orchestra.run.smithi001.stdout: "ALERTMANAGER_API_HOST": { 2024-02-02T15:46:48.719 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ALERTMANAGER_API_HOST", 2024-02-02T15:46:48.719 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.719 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.719 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.719 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.719 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.720 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.720 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.720 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.720 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.720 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.720 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.720 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.720 INFO:teuthology.orchestra.run.smithi001.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-02-02T15:46:48.721 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-02-02T15:46:48.721 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.721 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.721 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.721 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:48.721 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.721 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.721 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.722 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.722 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.722 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.722 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.722 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.722 INFO:teuthology.orchestra.run.smithi001.stdout: "AUDIT_API_ENABLED": { 2024-02-02T15:46:48.722 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "AUDIT_API_ENABLED", 2024-02-02T15:46:48.722 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.722 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.723 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.723 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.723 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.723 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.723 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.723 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.723 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.723 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.723 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.723 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.724 INFO:teuthology.orchestra.run.smithi001.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-02-02T15:46:48.724 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-02-02T15:46:48.724 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.724 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.724 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.724 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:48.724 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.724 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.724 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.725 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.725 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.725 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.725 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.725 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.725 INFO:teuthology.orchestra.run.smithi001.stdout: "ENABLE_BROWSABLE_API": { 2024-02-02T15:46:48.725 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ENABLE_BROWSABLE_API", 2024-02-02T15:46:48.725 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.726 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.726 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.726 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:48.726 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.726 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.726 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.726 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.726 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.726 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.727 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.727 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.727 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-02-02T15:46:48.727 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-02-02T15:46:48.727 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.727 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.727 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.727 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:48.728 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.728 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.728 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.728 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.728 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.728 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.728 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.728 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.729 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-02-02T15:46:48.729 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-02-02T15:46:48.729 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.729 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.729 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.729 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:48.729 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.730 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.730 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.730 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.730 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.730 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.730 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.730 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.730 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-02-02T15:46:48.731 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-02-02T15:46:48.731 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.731 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.731 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.731 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:48.731 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.731 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.731 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.732 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.732 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.732 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.732 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.732 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.732 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-02-02T15:46:48.732 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-02-02T15:46:48.732 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.733 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.733 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.733 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:48.733 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.733 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.733 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.733 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.734 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.734 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.734 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.734 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.734 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_NFS": { 2024-02-02T15:46:48.734 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-02-02T15:46:48.734 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.734 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.735 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.735 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:48.735 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.735 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.735 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.735 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.735 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.735 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.736 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.736 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.736 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_RBD": { 2024-02-02T15:46:48.736 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-02-02T15:46:48.736 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.736 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.736 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.736 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:48.736 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.737 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.737 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.737 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.737 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.737 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.737 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.737 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.737 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_RGW": { 2024-02-02T15:46:48.737 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-02-02T15:46:48.738 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.738 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.738 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.738 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:48.738 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.738 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.738 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.738 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.738 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.739 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.739 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.739 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.739 INFO:teuthology.orchestra.run.smithi001.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-02-02T15:46:48.739 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-02-02T15:46:48.739 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.739 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.739 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.739 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.740 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.740 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.740 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.740 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.740 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.740 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.740 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.740 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.740 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_API_PASSWORD": { 2024-02-02T15:46:48.740 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_API_PASSWORD", 2024-02-02T15:46:48.741 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.741 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.741 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.741 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "admin", 2024-02-02T15:46:48.741 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.741 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.741 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.741 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.741 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.742 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.742 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.742 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.742 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-02-02T15:46:48.742 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-02-02T15:46:48.742 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.742 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.742 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.742 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:48.743 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.743 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.743 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.743 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.743 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.743 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.743 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.743 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.744 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_API_URL": { 2024-02-02T15:46:48.744 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_API_URL", 2024-02-02T15:46:48.744 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.744 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.744 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.744 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.744 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.745 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.745 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.745 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.745 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.745 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.745 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.745 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.745 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_API_USERNAME": { 2024-02-02T15:46:48.746 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_API_USERNAME", 2024-02-02T15:46:48.746 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.746 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.746 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.746 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "admin", 2024-02-02T15:46:48.746 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.746 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.746 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.747 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.747 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.747 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.747 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.747 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.747 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-02-02T15:46:48.747 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-02-02T15:46:48.747 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.747 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.748 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.748 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.748 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.748 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.748 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.748 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.748 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.749 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.749 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.749 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.749 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-02-02T15:46:48.749 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-02-02T15:46:48.749 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.749 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.749 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.750 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.750 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.750 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.750 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.750 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.750 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.750 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.750 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.750 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.751 INFO:teuthology.orchestra.run.smithi001.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-02-02T15:46:48.751 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-02-02T15:46:48.751 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.751 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.751 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.751 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:48.751 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.751 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.752 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.752 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.752 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.752 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.752 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.752 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.752 INFO:teuthology.orchestra.run.smithi001.stdout: "ISSUE_TRACKER_API_KEY": { 2024-02-02T15:46:48.752 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-02-02T15:46:48.752 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.753 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.753 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.753 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.753 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.753 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.753 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.753 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.753 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.754 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.754 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.754 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.754 INFO:teuthology.orchestra.run.smithi001.stdout: "PROMETHEUS_API_HOST": { 2024-02-02T15:46:48.754 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PROMETHEUS_API_HOST", 2024-02-02T15:46:48.754 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.754 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.754 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.754 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.755 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.755 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.755 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.755 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.755 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.755 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.756 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.756 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.756 INFO:teuthology.orchestra.run.smithi001.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-02-02T15:46:48.756 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-02-02T15:46:48.756 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.756 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.756 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.756 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:48.757 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.757 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.757 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.757 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.757 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.757 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.757 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.758 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.758 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-02-02T15:46:48.758 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-02-02T15:46:48.758 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.758 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.758 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.758 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.758 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.759 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.759 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.759 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.759 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.759 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.759 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.759 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.759 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-02-02T15:46:48.759 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-02-02T15:46:48.760 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.760 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.760 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.760 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.760 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.760 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.760 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.760 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.761 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.761 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.761 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.761 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.761 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-02-02T15:46:48.761 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-02-02T15:46:48.761 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.762 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.762 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.762 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:48.762 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.762 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.762 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.762 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.762 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.763 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.763 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.763 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.763 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-02-02T15:46:48.763 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-02-02T15:46:48.763 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.763 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.763 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.764 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:48.764 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.764 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.764 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.764 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.764 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.764 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.764 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.765 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.765 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-02-02T15:46:48.765 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-02-02T15:46:48.765 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.765 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.765 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.765 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.765 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.765 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.766 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.766 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.766 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.766 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.766 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.766 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.766 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-02-02T15:46:48.766 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-02-02T15:46:48.767 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.767 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.767 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.767 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.767 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.767 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.767 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.767 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.768 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.768 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.768 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.768 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.768 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-02-02T15:46:48.768 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-02-02T15:46:48.768 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.768 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.768 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.769 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.769 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.769 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.769 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.769 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.769 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.769 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.769 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.770 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.770 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_ENABLED": { 2024-02-02T15:46:48.770 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_ENABLED", 2024-02-02T15:46:48.770 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.770 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.770 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.770 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:48.770 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.771 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.771 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.771 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.771 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.771 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.771 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.771 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.771 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-02-02T15:46:48.772 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-02-02T15:46:48.772 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.772 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.772 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.772 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-02-02T15:46:48.772 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.772 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.772 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.772 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.773 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.773 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.773 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.773 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.773 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-02-02T15:46:48.773 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-02-02T15:46:48.773 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:48.773 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.773 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.774 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-02-02T15:46:48.774 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.774 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.774 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.774 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.774 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.774 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.774 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.775 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.775 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-02-02T15:46:48.775 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-02-02T15:46:48.775 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:48.775 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.775 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.775 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8", 2024-02-02T15:46:48.775 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.775 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.776 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.776 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.776 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.776 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.776 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.776 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.776 INFO:teuthology.orchestra.run.smithi001.stdout: "REST_REQUESTS_TIMEOUT": { 2024-02-02T15:46:48.776 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-02-02T15:46:48.776 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:48.776 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.777 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.777 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "45", 2024-02-02T15:46:48.777 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.777 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.777 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.777 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.777 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.777 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.777 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.778 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.778 INFO:teuthology.orchestra.run.smithi001.stdout: "RGW_API_ACCESS_KEY": { 2024-02-02T15:46:48.778 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RGW_API_ACCESS_KEY", 2024-02-02T15:46:48.778 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.778 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.778 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.778 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.778 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.778 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.778 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.779 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.779 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.779 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.779 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.779 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.779 INFO:teuthology.orchestra.run.smithi001.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-02-02T15:46:48.779 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-02-02T15:46:48.779 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.779 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.780 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.780 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "admin", 2024-02-02T15:46:48.780 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.780 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.780 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.780 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.780 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.780 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.780 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.781 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.781 INFO:teuthology.orchestra.run.smithi001.stdout: "RGW_API_SECRET_KEY": { 2024-02-02T15:46:48.781 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RGW_API_SECRET_KEY", 2024-02-02T15:46:48.781 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.781 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.781 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.781 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.781 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.781 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.782 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.782 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.782 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.782 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.782 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.782 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.782 INFO:teuthology.orchestra.run.smithi001.stdout: "RGW_API_SSL_VERIFY": { 2024-02-02T15:46:48.782 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RGW_API_SSL_VERIFY", 2024-02-02T15:46:48.782 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.783 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.783 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.783 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:48.783 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.783 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.783 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.783 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.783 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.783 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.783 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.784 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.784 INFO:teuthology.orchestra.run.smithi001.stdout: "UNSAFE_TLS_v1_2": { 2024-02-02T15:46:48.784 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "UNSAFE_TLS_v1_2", 2024-02-02T15:46:48.784 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.784 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.784 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.784 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.784 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.785 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.785 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.785 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.785 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.785 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.785 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.785 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.785 INFO:teuthology.orchestra.run.smithi001.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-02-02T15:46:48.786 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-02-02T15:46:48.786 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:48.786 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.786 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.786 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-02-02T15:46:48.786 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.786 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.786 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.786 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.787 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.787 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.787 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.787 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.787 INFO:teuthology.orchestra.run.smithi001.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-02-02T15:46:48.787 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-02-02T15:46:48.787 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:48.787 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.788 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.788 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-02-02T15:46:48.788 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.788 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.788 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.788 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.788 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.788 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.788 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.789 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.789 INFO:teuthology.orchestra.run.smithi001.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-02-02T15:46:48.789 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-02-02T15:46:48.789 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:48.789 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.789 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.789 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2024-02-02T15:46:48.789 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.789 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.789 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.790 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.790 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.790 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.790 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.790 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.790 INFO:teuthology.orchestra.run.smithi001.stdout: "cross_origin_url": { 2024-02-02T15:46:48.790 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cross_origin_url", 2024-02-02T15:46:48.790 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.791 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.791 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.791 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.791 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.791 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.791 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.791 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.792 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.792 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.792 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.792 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.792 INFO:teuthology.orchestra.run.smithi001.stdout: "crt_file": { 2024-02-02T15:46:48.792 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crt_file", 2024-02-02T15:46:48.792 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.792 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.792 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.793 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.793 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.793 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.793 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.793 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.793 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.793 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.793 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.793 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.794 INFO:teuthology.orchestra.run.smithi001.stdout: "debug": { 2024-02-02T15:46:48.794 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "debug", 2024-02-02T15:46:48.794 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.794 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.794 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.794 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.794 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.794 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.794 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.795 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Enable/disable debug options", 2024-02-02T15:46:48.795 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.795 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.795 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.795 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.795 INFO:teuthology.orchestra.run.smithi001.stdout: "jwt_token_ttl": { 2024-02-02T15:46:48.795 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "jwt_token_ttl", 2024-02-02T15:46:48.795 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:48.796 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.796 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.796 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "28800", 2024-02-02T15:46:48.796 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.796 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.796 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.796 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.796 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.796 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.797 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.797 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.797 INFO:teuthology.orchestra.run.smithi001.stdout: "key_file": { 2024-02-02T15:46:48.797 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "key_file", 2024-02-02T15:46:48.797 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.797 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.797 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.797 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.798 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.798 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.798 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.798 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.798 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.798 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.798 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.798 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.798 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:48.798 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:48.799 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.799 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.799 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.799 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.799 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.799 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.799 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.799 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.799 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.800 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.800 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.800 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.800 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.800 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.800 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.800 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.800 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.800 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.801 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.801 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:48.801 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:48.801 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.801 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.801 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.801 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.801 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.802 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.802 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.802 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.802 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.802 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.802 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.803 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.803 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:48.803 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:48.803 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.803 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.803 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.803 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:48.803 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.804 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.804 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.804 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.804 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.804 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.804 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.804 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.804 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.805 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.805 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.805 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.805 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.805 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.805 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.805 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:48.806 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:48.806 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.806 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.806 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.806 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.806 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.806 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.807 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.807 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.807 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.807 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.807 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.807 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.807 INFO:teuthology.orchestra.run.smithi001.stdout: "motd": { 2024-02-02T15:46:48.807 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "motd", 2024-02-02T15:46:48.808 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.808 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.808 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.808 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.808 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.808 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.808 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.809 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "The message of the day", 2024-02-02T15:46:48.809 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.809 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.809 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.809 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.809 INFO:teuthology.orchestra.run.smithi001.stdout: "redirect_resolve_ip_addr": { 2024-02-02T15:46:48.809 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "redirect_resolve_ip_addr", 2024-02-02T15:46:48.809 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.810 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.810 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.810 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.810 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.810 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.810 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.810 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.810 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.811 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.811 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.811 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.811 INFO:teuthology.orchestra.run.smithi001.stdout: "server_addr": { 2024-02-02T15:46:48.811 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_addr", 2024-02-02T15:46:48.811 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.811 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.812 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.812 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "::", 2024-02-02T15:46:48.812 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.812 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.812 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.812 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.812 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.813 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.813 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.813 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.813 INFO:teuthology.orchestra.run.smithi001.stdout: "server_port": { 2024-02-02T15:46:48.813 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_port", 2024-02-02T15:46:48.813 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:48.813 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.813 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.814 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8080", 2024-02-02T15:46:48.814 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.814 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.814 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.814 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.814 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.814 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.814 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.814 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.815 INFO:teuthology.orchestra.run.smithi001.stdout: "ssl": { 2024-02-02T15:46:48.815 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ssl", 2024-02-02T15:46:48.815 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.815 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.815 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.815 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:48.815 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.815 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.816 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.816 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.816 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.816 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.816 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.816 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.816 INFO:teuthology.orchestra.run.smithi001.stdout: "ssl_server_port": { 2024-02-02T15:46:48.816 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ssl_server_port", 2024-02-02T15:46:48.816 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:48.816 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.817 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.817 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8443", 2024-02-02T15:46:48.817 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.817 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.817 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.817 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.817 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.817 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.817 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.818 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.818 INFO:teuthology.orchestra.run.smithi001.stdout: "standby_behaviour": { 2024-02-02T15:46:48.818 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "standby_behaviour", 2024-02-02T15:46:48.818 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.818 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.818 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.818 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "redirect", 2024-02-02T15:46:48.818 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.818 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.819 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.819 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.819 INFO:teuthology.orchestra.run.smithi001.stdout: "redirect" 2024-02-02T15:46:48.819 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.819 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.819 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.819 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.819 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.819 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.820 INFO:teuthology.orchestra.run.smithi001.stdout: "standby_error_status_code": { 2024-02-02T15:46:48.820 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "standby_error_status_code", 2024-02-02T15:46:48.820 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:48.820 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.820 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.820 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "500", 2024-02-02T15:46:48.820 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "400", 2024-02-02T15:46:48.821 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "599", 2024-02-02T15:46:48.821 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.821 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.821 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.821 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.821 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.821 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.821 INFO:teuthology.orchestra.run.smithi001.stdout: "url_prefix": { 2024-02-02T15:46:48.822 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "url_prefix", 2024-02-02T15:46:48.822 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.822 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.822 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.822 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.822 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.822 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.822 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.823 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.823 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.823 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.823 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.823 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.823 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.823 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.823 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:48.824 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "devicehealth", 2024-02-02T15:46:48.824 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:48.824 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:48.824 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:48.824 INFO:teuthology.orchestra.run.smithi001.stdout: "enable_monitoring": { 2024-02-02T15:46:48.824 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "enable_monitoring", 2024-02-02T15:46:48.824 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.824 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.825 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.825 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:48.825 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.825 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.825 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.825 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "monitor device health metrics", 2024-02-02T15:46:48.825 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.826 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.826 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.826 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.826 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:48.826 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:48.826 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.826 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.827 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.827 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.827 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.827 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.827 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.827 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.827 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.828 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.828 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.828 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.828 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.828 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.828 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.828 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.828 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.829 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.829 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.829 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:48.829 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:48.829 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.829 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.829 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.829 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.829 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.830 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.830 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.830 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.830 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.830 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.830 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.830 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.830 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:48.830 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:48.831 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.831 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.831 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.831 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:48.831 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.831 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.831 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.831 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.831 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.832 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.832 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.832 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.832 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.832 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.832 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.832 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.832 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.832 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.832 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.833 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:48.833 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:48.833 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.833 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.833 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.833 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.833 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.833 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.833 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.834 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.834 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.834 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.834 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.834 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.834 INFO:teuthology.orchestra.run.smithi001.stdout: "mark_out_threshold": { 2024-02-02T15:46:48.834 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mark_out_threshold", 2024-02-02T15:46:48.834 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-02-02T15:46:48.834 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.834 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.835 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "2419200", 2024-02-02T15:46:48.835 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.835 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.835 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.835 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-02-02T15:46:48.835 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.835 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.835 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.835 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.836 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_name": { 2024-02-02T15:46:48.836 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "pool_name", 2024-02-02T15:46:48.836 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.836 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.836 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.836 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "device_health_metrics", 2024-02-02T15:46:48.836 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.836 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.836 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.837 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "name of pool in which to store device health metrics", 2024-02-02T15:46:48.837 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.837 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.837 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.837 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.837 INFO:teuthology.orchestra.run.smithi001.stdout: "retention_period": { 2024-02-02T15:46:48.837 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "retention_period", 2024-02-02T15:46:48.837 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-02-02T15:46:48.837 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.837 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.838 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "15552000", 2024-02-02T15:46:48.838 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.838 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.838 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.838 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how long to retain device health metrics", 2024-02-02T15:46:48.838 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.838 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.838 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.838 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.839 INFO:teuthology.orchestra.run.smithi001.stdout: "scrape_frequency": { 2024-02-02T15:46:48.839 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "scrape_frequency", 2024-02-02T15:46:48.839 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-02-02T15:46:48.839 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.839 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.839 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "86400", 2024-02-02T15:46:48.839 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.839 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.839 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.840 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to scrape device health metrics", 2024-02-02T15:46:48.840 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.840 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.840 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.840 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.840 INFO:teuthology.orchestra.run.smithi001.stdout: "self_heal": { 2024-02-02T15:46:48.840 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "self_heal", 2024-02-02T15:46:48.840 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.840 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.841 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.841 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:48.841 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.841 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.841 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.841 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-02-02T15:46:48.841 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.841 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.842 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.842 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.842 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-02-02T15:46:48.842 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-02-02T15:46:48.842 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-02-02T15:46:48.842 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.842 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.842 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-02-02T15:46:48.843 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.843 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.843 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.843 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to wake up and check device health", 2024-02-02T15:46:48.843 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.843 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.843 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.843 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.843 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_threshold": { 2024-02-02T15:46:48.844 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_threshold", 2024-02-02T15:46:48.844 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-02-02T15:46:48.844 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.844 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.844 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "7257600", 2024-02-02T15:46:48.844 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.844 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.844 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.845 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-02-02T15:46:48.845 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.845 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.845 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.845 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.845 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.845 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.845 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:48.845 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "diskprediction_local", 2024-02-02T15:46:48.846 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:48.846 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:48.846 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:48.846 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:48.846 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:48.846 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.846 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.846 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.847 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.847 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.847 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.847 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.847 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.847 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.847 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.847 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.848 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.848 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.848 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.848 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.848 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.848 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.848 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.848 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.849 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:48.849 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:48.849 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.849 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.849 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.849 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.849 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.849 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.850 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.850 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.850 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.850 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.850 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.850 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.850 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:48.851 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:48.851 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.851 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.851 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.851 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:48.851 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.851 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.851 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.851 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.852 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.852 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.852 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.852 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.852 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.852 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.852 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.852 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.853 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.853 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.853 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.853 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:48.853 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:48.853 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.854 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.854 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.854 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.854 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.854 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.855 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.855 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.855 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.855 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.855 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.855 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.856 INFO:teuthology.orchestra.run.smithi001.stdout: "predict_interval": { 2024-02-02T15:46:48.856 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "predict_interval", 2024-02-02T15:46:48.856 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.856 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.856 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.856 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "86400", 2024-02-02T15:46:48.857 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.857 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.857 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.857 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.857 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.857 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.858 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.858 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.858 INFO:teuthology.orchestra.run.smithi001.stdout: "predictor_model": { 2024-02-02T15:46:48.858 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "predictor_model", 2024-02-02T15:46:48.858 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.858 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.858 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.858 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "prophetstor", 2024-02-02T15:46:48.859 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.859 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.859 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.859 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.859 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.859 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.859 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.860 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.860 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-02-02T15:46:48.860 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-02-02T15:46:48.860 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.860 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.861 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.861 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-02-02T15:46:48.861 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.861 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.861 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.862 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.862 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.862 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.862 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.863 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.863 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.863 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.863 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:48.863 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "influx", 2024-02-02T15:46:48.864 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": false, 2024-02-02T15:46:48.864 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "influxdb python module not found", 2024-02-02T15:46:48.864 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:48.864 INFO:teuthology.orchestra.run.smithi001.stdout: "batch_size": { 2024-02-02T15:46:48.864 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "batch_size", 2024-02-02T15:46:48.865 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:48.865 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.865 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.865 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5000", 2024-02-02T15:46:48.865 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.865 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.866 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.866 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-02-02T15:46:48.866 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.866 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.866 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.866 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.866 INFO:teuthology.orchestra.run.smithi001.stdout: "database": { 2024-02-02T15:46:48.866 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "database", 2024-02-02T15:46:48.866 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.867 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.867 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.867 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "ceph", 2024-02-02T15:46:48.867 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.867 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.867 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.867 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-02-02T15:46:48.868 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.868 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.868 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.868 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.868 INFO:teuthology.orchestra.run.smithi001.stdout: "hostname": { 2024-02-02T15:46:48.868 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "hostname", 2024-02-02T15:46:48.868 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.869 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.869 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.869 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.869 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.869 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.869 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.870 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "InfluxDB server hostname", 2024-02-02T15:46:48.870 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.870 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.870 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.870 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.870 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-02-02T15:46:48.870 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-02-02T15:46:48.871 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-02-02T15:46:48.871 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.871 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.871 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "30", 2024-02-02T15:46:48.871 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "5", 2024-02-02T15:46:48.871 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.872 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.872 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-02-02T15:46:48.872 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.872 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.872 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.872 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.873 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:48.873 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:48.873 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.873 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.873 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.873 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.873 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.874 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.874 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.874 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.874 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.874 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.874 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.875 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.875 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.875 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.875 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.875 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.875 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.875 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.875 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.875 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:48.876 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:48.876 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.876 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.876 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.876 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.876 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.876 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.876 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.876 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.877 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.877 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.877 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.877 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.877 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:48.877 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:48.877 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.877 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.877 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.878 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:48.878 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.878 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.878 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.878 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.878 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.878 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.878 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.878 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.879 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.879 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.879 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.879 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.879 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.879 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.879 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.879 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:48.880 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:48.880 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.880 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.880 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.880 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.880 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.880 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.880 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.881 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.881 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.881 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.881 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.881 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.881 INFO:teuthology.orchestra.run.smithi001.stdout: "password": { 2024-02-02T15:46:48.881 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "password", 2024-02-02T15:46:48.881 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.881 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.882 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.882 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.882 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.882 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.882 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.882 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "password of InfluxDB server user", 2024-02-02T15:46:48.882 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.882 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.882 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.883 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.883 INFO:teuthology.orchestra.run.smithi001.stdout: "port": { 2024-02-02T15:46:48.883 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "port", 2024-02-02T15:46:48.883 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:48.883 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.883 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.883 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8086", 2024-02-02T15:46:48.883 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.884 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.884 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.884 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "InfluxDB server port", 2024-02-02T15:46:48.884 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.884 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.884 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.884 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.884 INFO:teuthology.orchestra.run.smithi001.stdout: "ssl": { 2024-02-02T15:46:48.884 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ssl", 2024-02-02T15:46:48.885 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.885 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.885 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.885 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "false", 2024-02-02T15:46:48.885 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.885 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.885 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.885 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-02-02T15:46:48.885 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.886 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.886 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.886 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.886 INFO:teuthology.orchestra.run.smithi001.stdout: "threads": { 2024-02-02T15:46:48.886 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "threads", 2024-02-02T15:46:48.886 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:48.886 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.886 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.887 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2024-02-02T15:46:48.887 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1", 2024-02-02T15:46:48.887 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "32", 2024-02-02T15:46:48.887 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.887 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-02-02T15:46:48.887 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.887 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.887 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.888 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.888 INFO:teuthology.orchestra.run.smithi001.stdout: "username": { 2024-02-02T15:46:48.888 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "username", 2024-02-02T15:46:48.888 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.888 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.888 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.888 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.888 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.888 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.889 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.889 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "username of InfluxDB server user", 2024-02-02T15:46:48.889 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.889 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.889 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.889 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.889 INFO:teuthology.orchestra.run.smithi001.stdout: "verify_ssl": { 2024-02-02T15:46:48.890 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "verify_ssl", 2024-02-02T15:46:48.890 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.890 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.890 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.890 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "true", 2024-02-02T15:46:48.890 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.890 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.890 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.890 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-02-02T15:46:48.891 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.891 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.891 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.891 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.891 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.891 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.891 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:48.891 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "insights", 2024-02-02T15:46:48.892 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:48.892 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:48.892 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:48.892 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:48.892 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:48.892 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.892 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.892 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.892 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.893 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.893 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.893 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.893 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.893 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.893 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.893 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.893 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.893 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.894 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.894 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.894 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.894 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.894 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.894 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.894 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:48.894 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:48.895 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.895 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.895 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.895 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.895 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.895 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.895 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.895 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.896 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.896 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.896 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.896 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.896 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:48.896 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:48.896 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.896 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.897 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.897 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:48.897 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.897 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.897 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.897 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.897 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.897 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.897 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.898 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.898 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.898 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.898 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.898 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.898 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.898 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.898 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.899 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:48.899 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:48.899 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.899 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.899 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.899 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.899 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.899 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.900 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.900 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.900 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.900 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.900 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.900 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.900 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.900 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.901 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:48.901 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "iostat", 2024-02-02T15:46:48.901 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:48.901 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:48.901 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:48.901 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:48.901 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:48.902 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.902 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.902 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.902 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.902 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.903 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.903 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.903 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.903 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.903 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.903 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.903 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.903 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.904 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.904 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.904 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.904 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.904 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.904 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.904 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:48.904 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:48.905 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.905 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.905 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.905 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.905 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.905 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.905 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.906 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.906 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.906 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.906 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.906 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.906 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:48.906 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:48.906 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.907 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.907 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.907 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:48.907 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.907 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.907 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.907 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.908 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.908 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.908 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.908 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.908 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.908 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.908 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.908 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.909 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.909 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.909 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.909 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:48.909 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:48.909 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.909 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.909 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.909 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.910 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.910 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.910 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.910 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.910 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.910 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.910 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.910 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.910 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.911 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.911 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:48.911 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "k8sevents", 2024-02-02T15:46:48.911 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:48.911 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:48.911 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:48.911 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_event_retention_days": { 2024-02-02T15:46:48.911 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ceph_event_retention_days", 2024-02-02T15:46:48.911 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:48.912 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.912 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.912 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "7", 2024-02-02T15:46:48.912 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.912 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.912 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.912 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Days to hold ceph event information within local cache", 2024-02-02T15:46:48.912 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.912 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.913 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.913 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.913 INFO:teuthology.orchestra.run.smithi001.stdout: "config_check_secs": { 2024-02-02T15:46:48.913 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "config_check_secs", 2024-02-02T15:46:48.913 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:48.913 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.913 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.913 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-02-02T15:46:48.913 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "10", 2024-02-02T15:46:48.914 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.914 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.914 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-02-02T15:46:48.914 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.914 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.914 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.914 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.914 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:48.914 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:48.914 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.915 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.915 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.915 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.915 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.915 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.915 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.915 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.915 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.916 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.916 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.916 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.916 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.916 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.916 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.916 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.916 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.917 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.917 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.917 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:48.917 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:48.917 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.917 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.917 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.917 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.918 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.918 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.918 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.918 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.918 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.918 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.918 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.918 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.919 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:48.919 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:48.919 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.919 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.919 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.919 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:48.919 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.919 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.919 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.920 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.920 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.920 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.920 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.920 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.920 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.920 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.920 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.921 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.921 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.921 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.921 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.921 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:48.921 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:48.921 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.921 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.922 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.922 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.922 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.922 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.922 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.922 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.922 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.922 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.923 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.923 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.923 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.923 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.923 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:48.923 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "localpool", 2024-02-02T15:46:48.923 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:48.923 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:48.923 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:48.924 INFO:teuthology.orchestra.run.smithi001.stdout: "failure_domain": { 2024-02-02T15:46:48.924 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "failure_domain", 2024-02-02T15:46:48.924 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.924 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.924 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.924 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "host", 2024-02-02T15:46:48.924 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.924 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.925 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.925 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "failure domain for any created local pool", 2024-02-02T15:46:48.925 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-02-02T15:46:48.925 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.925 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.925 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.925 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:48.926 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:48.926 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.926 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.926 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.926 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.926 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.926 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.926 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.927 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.927 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.927 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.927 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.927 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.927 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.927 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.927 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.927 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.928 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.928 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.928 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.928 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:48.928 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:48.928 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.928 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.928 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.929 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.929 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.929 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.929 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.929 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.929 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.929 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.929 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.929 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.929 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:48.930 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:48.930 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.930 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.930 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.930 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:48.930 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.930 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.930 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.931 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.931 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.931 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.931 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.931 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.931 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.931 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.931 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.932 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.932 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.932 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.932 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.932 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:48.932 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:48.932 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.932 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.932 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.933 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.933 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.933 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.933 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.933 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.933 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.933 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.933 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.933 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.934 INFO:teuthology.orchestra.run.smithi001.stdout: "min_size": { 2024-02-02T15:46:48.934 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "min_size", 2024-02-02T15:46:48.934 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:48.934 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.934 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.934 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.934 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.934 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.934 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.935 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "default min_size for any created local pool", 2024-02-02T15:46:48.935 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-02-02T15:46:48.935 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.935 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.935 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.935 INFO:teuthology.orchestra.run.smithi001.stdout: "num_rep": { 2024-02-02T15:46:48.935 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "num_rep", 2024-02-02T15:46:48.935 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:48.936 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.936 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.936 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "3", 2024-02-02T15:46:48.936 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.936 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.936 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.936 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "default replica count for any created local pool", 2024-02-02T15:46:48.936 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.936 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.937 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.937 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.937 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num": { 2024-02-02T15:46:48.937 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "pg_num", 2024-02-02T15:46:48.937 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:48.937 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.937 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.937 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "128", 2024-02-02T15:46:48.937 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.938 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.938 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.938 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "default pg_num for any created local pool", 2024-02-02T15:46:48.938 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.938 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.938 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.938 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.938 INFO:teuthology.orchestra.run.smithi001.stdout: "prefix": { 2024-02-02T15:46:48.939 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "prefix", 2024-02-02T15:46:48.939 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.939 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.939 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.939 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.939 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.939 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.939 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.939 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "name prefix for any created local pool", 2024-02-02T15:46:48.939 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.940 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.940 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.940 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.940 INFO:teuthology.orchestra.run.smithi001.stdout: "subtree": { 2024-02-02T15:46:48.940 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "subtree", 2024-02-02T15:46:48.940 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.940 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.940 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.941 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "rack", 2024-02-02T15:46:48.941 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.941 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.941 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.941 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "CRUSH level for which to create a local pool", 2024-02-02T15:46:48.941 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-02-02T15:46:48.941 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.941 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.941 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.941 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.942 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.942 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:48.942 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mds_autoscaler", 2024-02-02T15:46:48.942 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:48.942 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:48.942 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:48.942 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:48.942 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:48.942 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.943 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.943 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.943 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.943 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.943 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.943 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.943 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.943 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.943 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.944 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.944 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.944 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.944 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.944 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.944 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.944 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.945 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.945 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.945 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:48.945 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:48.945 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.945 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.945 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.945 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.946 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.946 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.946 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.946 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.946 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.946 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.946 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.946 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.947 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:48.947 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:48.947 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.947 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.947 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.947 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:48.947 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.948 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.948 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.948 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.948 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.948 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.948 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.948 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.948 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.949 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.949 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.949 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.949 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.949 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.949 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.949 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:48.949 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:48.949 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.950 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.950 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.950 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.950 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.950 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.950 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.950 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.950 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.950 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.951 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.951 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.951 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.951 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.951 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:48.951 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mirroring", 2024-02-02T15:46:48.951 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:48.951 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:48.951 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:48.952 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:48.952 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:48.952 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.952 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.952 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.952 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.952 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.952 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.952 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.953 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.953 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.953 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.953 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.953 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.953 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.953 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.953 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.954 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.954 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.954 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.954 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.954 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:48.954 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:48.954 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.955 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.955 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.955 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.955 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.955 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.955 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.955 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.955 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.956 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.956 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.956 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.956 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:48.956 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:48.956 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.956 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.957 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.957 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:48.957 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.957 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.957 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.957 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.957 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.957 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.957 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.958 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.958 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.958 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.958 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.958 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.958 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.958 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.958 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.958 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:48.959 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:48.959 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.959 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.959 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.959 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.959 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.959 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.959 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.960 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.960 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.960 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.960 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.960 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.960 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.960 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.960 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:48.961 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "nfs", 2024-02-02T15:46:48.961 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:48.961 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:48.961 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:48.961 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:48.961 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:48.961 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.961 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.961 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.962 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.962 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.962 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.962 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.962 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.962 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.962 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.962 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.963 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.963 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.963 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.963 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.963 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.963 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.963 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.963 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.963 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:48.964 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:48.964 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.964 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.964 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.964 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.964 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.964 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.964 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.964 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.965 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.965 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.965 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.965 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.965 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:48.965 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:48.965 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.965 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.965 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.966 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:48.966 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.966 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.966 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.966 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.966 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.966 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.966 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.966 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.966 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.967 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.967 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.967 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.967 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.967 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.967 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.967 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:48.967 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:48.968 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.968 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.968 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.968 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.968 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.968 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.968 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.968 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.968 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.969 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.969 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.969 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.969 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.969 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.969 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:48.969 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "orchestrator", 2024-02-02T15:46:48.969 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:48.969 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:48.970 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:48.970 INFO:teuthology.orchestra.run.smithi001.stdout: "fail_fs": { 2024-02-02T15:46:48.970 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "fail_fs", 2024-02-02T15:46:48.970 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.970 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.970 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:48.970 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.970 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.971 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.971 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.971 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-02-02T15:46:48.971 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.971 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.971 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.971 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.971 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:48.971 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:48.972 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.972 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.972 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.972 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.972 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.972 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.972 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.973 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.973 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.973 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.973 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.973 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.973 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.973 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.973 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.973 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.974 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.974 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.974 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.974 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:48.974 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:48.974 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.974 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.974 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.975 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.975 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.975 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.975 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.975 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.975 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.975 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.975 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.975 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.976 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:48.976 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:48.976 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.976 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.976 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.976 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:48.976 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.976 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.977 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.977 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.977 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.977 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.977 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.977 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.977 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.977 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.978 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.978 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.978 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.978 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.978 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.978 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:48.978 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:48.978 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.978 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.979 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.979 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.979 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.979 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.979 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.979 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.979 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.979 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.980 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.980 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.980 INFO:teuthology.orchestra.run.smithi001.stdout: "orchestrator": { 2024-02-02T15:46:48.980 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "orchestrator", 2024-02-02T15:46:48.980 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.980 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.980 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.980 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.980 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.981 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.981 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.981 INFO:teuthology.orchestra.run.smithi001.stdout: "cephadm", 2024-02-02T15:46:48.981 INFO:teuthology.orchestra.run.smithi001.stdout: "rook", 2024-02-02T15:46:48.981 INFO:teuthology.orchestra.run.smithi001.stdout: "test_orchestrator" 2024-02-02T15:46:48.981 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.981 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Orchestrator backend", 2024-02-02T15:46:48.981 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.981 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.982 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.982 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.982 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.982 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.982 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:48.982 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "osd_perf_query", 2024-02-02T15:46:48.982 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:48.982 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:48.982 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:48.982 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:48.983 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:48.983 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.983 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.983 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.983 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.983 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.983 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.983 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.983 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.984 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.984 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.984 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.984 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.984 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.984 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.984 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.984 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.985 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.985 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.985 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.985 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:48.985 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:48.985 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.985 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.985 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.986 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.986 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.986 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.986 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.986 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.986 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.986 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.986 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.986 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.987 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:48.987 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:48.987 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.987 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.987 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.987 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:48.987 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.987 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.988 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.988 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.988 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.988 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.988 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.988 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.988 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.988 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.988 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.989 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.989 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.989 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.989 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.989 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:48.989 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:48.989 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.989 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.989 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.990 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.990 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.990 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.990 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.990 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.990 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.990 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.990 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.990 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.991 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:48.991 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.991 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:48.991 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "osd_support", 2024-02-02T15:46:48.991 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:48.991 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:48.991 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:48.991 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:48.991 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:48.991 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.992 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.992 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.992 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:48.992 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.992 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.992 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.992 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.992 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.993 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.993 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.993 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.993 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.993 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.993 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.993 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.993 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.994 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.994 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.994 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:48.994 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:48.994 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.994 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.994 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.994 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.995 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.995 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.995 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.995 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.995 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.995 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.995 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.995 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.996 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:48.996 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:48.996 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:48.996 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.996 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.996 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:48.996 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.996 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.997 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:48.997 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:48.997 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:48.997 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:48.997 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:48.997 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:48.997 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:48.997 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:48.998 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:48.998 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:48.998 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:48.998 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:48.998 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:48.998 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:48.998 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:48.998 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:48.999 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:48.999 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:48.999 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:48.999 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:48.999 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:48.999 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:48.999 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.000 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.000 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.000 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.000 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.000 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.000 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.000 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:49.000 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "pg_autoscaler", 2024-02-02T15:46:49.001 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:49.001 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:49.001 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:49.001 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:49.001 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:49.001 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.001 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.002 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.002 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.002 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.002 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.002 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.002 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.002 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.002 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.003 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.003 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.003 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.003 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.003 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.003 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.003 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.003 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.004 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.004 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:49.004 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:49.004 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.004 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.004 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.004 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.004 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.005 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.005 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.005 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.005 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.005 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.005 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.005 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.005 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:49.005 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:49.006 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.006 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.006 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.006 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:49.006 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.006 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.006 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.006 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.006 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.007 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.007 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.007 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.007 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.007 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.007 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.007 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.007 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.007 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.008 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.008 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:49.008 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:49.008 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.008 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.008 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.008 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.008 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.009 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.009 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.009 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.009 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.009 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.009 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.009 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.009 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-02-02T15:46:49.010 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-02-02T15:46:49.010 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-02-02T15:46:49.010 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.010 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.010 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-02-02T15:46:49.010 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.010 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.010 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.011 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.011 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.011 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.011 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.011 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.011 INFO:teuthology.orchestra.run.smithi001.stdout: "threshold": { 2024-02-02T15:46:49.011 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "threshold", 2024-02-02T15:46:49.012 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-02-02T15:46:49.012 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.012 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.012 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "3.0", 2024-02-02T15:46:49.012 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1.0", 2024-02-02T15:46:49.012 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.012 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.012 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "scaling threshold", 2024-02-02T15:46:49.013 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-02-02T15:46:49.013 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.013 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.013 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.013 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.013 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.013 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:49.013 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "progress", 2024-02-02T15:46:49.014 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:49.014 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:49.014 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:49.014 INFO:teuthology.orchestra.run.smithi001.stdout: "allow_pg_recovery_event": { 2024-02-02T15:46:49.014 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "allow_pg_recovery_event", 2024-02-02T15:46:49.014 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.014 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.015 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.015 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.015 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.015 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.015 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.015 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "allow the module to show pg recovery progress", 2024-02-02T15:46:49.015 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.016 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.016 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.016 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.016 INFO:teuthology.orchestra.run.smithi001.stdout: "enabled": { 2024-02-02T15:46:49.016 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "enabled", 2024-02-02T15:46:49.016 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.016 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.016 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.017 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:49.017 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.017 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.017 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.017 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.017 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.017 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.017 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.018 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.018 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:49.018 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:49.018 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.018 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.018 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.018 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.018 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.018 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.019 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.019 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.019 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.019 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.019 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.019 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.019 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.019 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.019 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.020 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.020 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.020 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.020 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.020 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:49.020 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:49.020 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.020 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.020 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.021 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.021 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.021 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.021 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.021 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.021 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.021 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.021 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.021 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.022 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:49.022 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:49.022 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.022 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.022 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.022 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:49.022 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.022 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.022 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.022 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.023 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.023 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.023 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.023 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.023 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.023 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.023 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.023 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.024 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.024 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.024 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.024 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:49.024 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:49.024 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.024 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.024 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.024 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.025 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.025 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.025 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.025 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.025 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.025 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.025 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.025 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.026 INFO:teuthology.orchestra.run.smithi001.stdout: "max_completed_events": { 2024-02-02T15:46:49.026 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_completed_events", 2024-02-02T15:46:49.026 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:49.026 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.026 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.026 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "50", 2024-02-02T15:46:49.026 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.026 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.026 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.027 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "number of past completed events to remember", 2024-02-02T15:46:49.027 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.027 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.027 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.027 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.027 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-02-02T15:46:49.027 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-02-02T15:46:49.027 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-02-02T15:46:49.027 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.028 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.028 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2024-02-02T15:46:49.028 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.028 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.028 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.028 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how long the module is going to sleep", 2024-02-02T15:46:49.028 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.028 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.028 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.029 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.029 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.029 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.029 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:49.029 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "prometheus", 2024-02-02T15:46:49.029 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:49.029 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:49.029 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:49.029 INFO:teuthology.orchestra.run.smithi001.stdout: "cache": { 2024-02-02T15:46:49.030 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cache", 2024-02-02T15:46:49.030 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.030 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.030 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.030 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:49.030 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.030 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.030 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.030 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.031 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.031 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.031 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.031 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.031 INFO:teuthology.orchestra.run.smithi001.stdout: "exclude_perf_counters": { 2024-02-02T15:46:49.031 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "exclude_perf_counters", 2024-02-02T15:46:49.031 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.031 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.032 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.032 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:49.032 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.032 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.032 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.032 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-02-02T15:46:49.032 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-02-02T15:46:49.032 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.033 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.033 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.033 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:49.033 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:49.033 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.033 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.033 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.033 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.034 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.034 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.034 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.034 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.034 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.034 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.034 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.034 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.034 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.035 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.035 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.035 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.035 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.035 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.035 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.035 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:49.035 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:49.036 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.036 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.036 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.036 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.036 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.036 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.036 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.036 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.037 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.037 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.037 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.037 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.037 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:49.037 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:49.037 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.037 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.038 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.038 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:49.038 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.038 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.038 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.038 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.038 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.038 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.039 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.039 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.039 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.039 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.039 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.039 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.039 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.039 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.040 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.040 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:49.040 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:49.040 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.040 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.040 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.040 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.040 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.040 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.041 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.041 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.041 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.041 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.041 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.041 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.041 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_stats_pools": { 2024-02-02T15:46:49.041 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rbd_stats_pools", 2024-02-02T15:46:49.042 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.042 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.042 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.042 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.042 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.042 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.042 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.042 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.043 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.043 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.043 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.043 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.043 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_stats_pools_refresh_interval": { 2024-02-02T15:46:49.043 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-02-02T15:46:49.043 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:49.043 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.044 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.044 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "300", 2024-02-02T15:46:49.044 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.044 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.044 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.044 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.044 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.044 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.045 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.045 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.045 INFO:teuthology.orchestra.run.smithi001.stdout: "scrape_interval": { 2024-02-02T15:46:49.045 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "scrape_interval", 2024-02-02T15:46:49.045 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-02-02T15:46:49.045 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.045 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.045 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "15.0", 2024-02-02T15:46:49.046 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.046 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.046 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.046 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.046 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.046 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.046 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.046 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.047 INFO:teuthology.orchestra.run.smithi001.stdout: "server_addr": { 2024-02-02T15:46:49.047 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_addr", 2024-02-02T15:46:49.047 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.047 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.047 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.047 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "::", 2024-02-02T15:46:49.047 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.047 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.048 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.048 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-02-02T15:46:49.048 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.048 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.048 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.048 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.048 INFO:teuthology.orchestra.run.smithi001.stdout: "server_port": { 2024-02-02T15:46:49.048 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_port", 2024-02-02T15:46:49.049 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:49.049 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.049 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.049 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "9283", 2024-02-02T15:46:49.049 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.049 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.049 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.049 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-02-02T15:46:49.050 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.050 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.050 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.050 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.050 INFO:teuthology.orchestra.run.smithi001.stdout: "stale_cache_strategy": { 2024-02-02T15:46:49.050 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "stale_cache_strategy", 2024-02-02T15:46:49.050 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.050 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.051 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.051 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "log", 2024-02-02T15:46:49.051 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.051 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.051 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.051 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.051 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.052 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.052 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.052 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.052 INFO:teuthology.orchestra.run.smithi001.stdout: "standby_behaviour": { 2024-02-02T15:46:49.052 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "standby_behaviour", 2024-02-02T15:46:49.052 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.052 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.052 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.053 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "default", 2024-02-02T15:46:49.053 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.053 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.053 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.053 INFO:teuthology.orchestra.run.smithi001.stdout: "default", 2024-02-02T15:46:49.053 INFO:teuthology.orchestra.run.smithi001.stdout: "error" 2024-02-02T15:46:49.053 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.053 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.053 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.054 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.054 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.054 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.054 INFO:teuthology.orchestra.run.smithi001.stdout: "standby_error_status_code": { 2024-02-02T15:46:49.054 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "standby_error_status_code", 2024-02-02T15:46:49.054 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:49.054 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.054 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.055 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "500", 2024-02-02T15:46:49.055 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "400", 2024-02-02T15:46:49.055 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "599", 2024-02-02T15:46:49.055 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.055 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.055 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.055 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.055 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.056 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.056 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.056 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.056 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:49.056 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rbd_support", 2024-02-02T15:46:49.056 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:49.056 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:49.056 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:49.057 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:49.057 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:49.057 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.057 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.057 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.057 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.057 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.057 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.058 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.058 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.058 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.058 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.058 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.058 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.058 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.058 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.058 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.059 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.059 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.059 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.059 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.059 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:49.059 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:49.059 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.059 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.059 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.060 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.060 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.060 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.060 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.060 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.060 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.060 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.060 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.060 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.061 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:49.061 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:49.061 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.061 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.061 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.061 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:49.061 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.061 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.062 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.062 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.062 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.062 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.062 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.062 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.062 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.062 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.062 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.063 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.063 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.063 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.063 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.063 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:49.063 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:49.063 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.063 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.064 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.064 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.064 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.064 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.064 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.064 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.064 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.064 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.065 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.065 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.065 INFO:teuthology.orchestra.run.smithi001.stdout: "max_concurrent_snap_create": { 2024-02-02T15:46:49.065 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_concurrent_snap_create", 2024-02-02T15:46:49.065 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:49.065 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.065 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.065 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-02-02T15:46:49.065 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.066 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.066 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.066 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.066 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.066 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.066 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.066 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.066 INFO:teuthology.orchestra.run.smithi001.stdout: "mirror_snapshot_schedule": { 2024-02-02T15:46:49.067 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mirror_snapshot_schedule", 2024-02-02T15:46:49.067 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.067 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.067 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.067 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.067 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.067 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.067 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.068 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.068 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.068 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.068 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.068 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.068 INFO:teuthology.orchestra.run.smithi001.stdout: "trash_purge_schedule": { 2024-02-02T15:46:49.068 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "trash_purge_schedule", 2024-02-02T15:46:49.068 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.068 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.069 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.069 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.069 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.069 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.069 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.069 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.069 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.069 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.070 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.070 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.070 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.070 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.070 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:49.070 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "restful", 2024-02-02T15:46:49.070 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:49.070 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:49.070 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:49.071 INFO:teuthology.orchestra.run.smithi001.stdout: "enable_auth": { 2024-02-02T15:46:49.071 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "enable_auth", 2024-02-02T15:46:49.071 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.071 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.071 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.071 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:49.071 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.071 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.071 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.072 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.072 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.072 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.072 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.072 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.072 INFO:teuthology.orchestra.run.smithi001.stdout: "key_file": { 2024-02-02T15:46:49.072 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "key_file", 2024-02-02T15:46:49.072 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.072 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.073 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.073 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.073 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.073 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.073 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.073 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.073 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.073 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.073 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.074 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.074 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:49.074 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:49.074 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.074 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.074 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.074 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.074 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.074 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.075 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.075 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.075 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.075 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.075 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.075 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.075 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.075 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.076 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.076 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.076 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.076 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.076 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.076 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:49.076 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:49.076 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.076 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.077 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.077 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.077 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.077 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.077 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.077 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.077 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.077 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.078 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.078 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.078 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:49.078 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:49.078 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.078 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.078 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.078 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:49.078 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.079 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.079 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.079 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.079 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.079 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.079 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.079 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.079 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.080 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.080 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.080 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.080 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.080 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.080 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.080 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:49.080 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:49.081 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.081 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.081 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.081 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.081 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.081 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.081 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.081 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.081 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.081 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.082 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.082 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.082 INFO:teuthology.orchestra.run.smithi001.stdout: "server_addr": { 2024-02-02T15:46:49.082 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_addr", 2024-02-02T15:46:49.082 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.082 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.082 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.082 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.082 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.083 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.083 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.083 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.083 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.083 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.083 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.083 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.083 INFO:teuthology.orchestra.run.smithi001.stdout: "server_port": { 2024-02-02T15:46:49.083 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_port", 2024-02-02T15:46:49.084 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.084 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.084 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.084 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.084 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.084 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.084 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.084 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.085 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.085 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.085 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.085 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.085 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.085 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.085 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:49.085 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rgw", 2024-02-02T15:46:49.085 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:49.086 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:49.086 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:49.086 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:49.086 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:49.086 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.086 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.086 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.086 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.086 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.087 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.087 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.087 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.087 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.087 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.087 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.087 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.087 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.087 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.088 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.088 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.088 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.088 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.088 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.088 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:49.088 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:49.088 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.089 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.089 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.089 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.089 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.089 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.089 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.089 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.089 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.090 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.090 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.090 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.090 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:49.090 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:49.090 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.090 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.090 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.091 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:49.091 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.091 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.091 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.091 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.091 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.091 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.091 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.091 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.092 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.092 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.092 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.092 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.092 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.092 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.092 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.093 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:49.093 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:49.093 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.093 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.093 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.093 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.093 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.093 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.094 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.094 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.094 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.094 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.094 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.094 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.094 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.095 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.095 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:49.095 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rook", 2024-02-02T15:46:49.095 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:49.095 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:49.095 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:49.095 INFO:teuthology.orchestra.run.smithi001.stdout: "drive_group_interval": { 2024-02-02T15:46:49.095 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "drive_group_interval", 2024-02-02T15:46:49.096 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-02-02T15:46:49.096 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.096 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.096 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "300.0", 2024-02-02T15:46:49.096 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.096 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.096 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.097 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2024-02-02T15:46:49.097 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.097 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.097 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.097 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.097 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:49.097 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:49.097 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.097 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.098 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.098 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.098 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.098 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.098 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.098 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.098 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.098 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.099 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.099 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.099 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.099 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.099 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.099 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.099 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.100 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.100 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.100 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:49.100 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:49.100 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.100 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.100 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.100 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.101 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.101 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.101 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.101 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.101 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.101 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.101 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.101 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.101 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:49.102 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:49.102 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.102 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.102 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.102 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:49.102 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.102 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.102 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.102 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.103 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.103 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.103 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.103 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.103 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.103 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.103 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.103 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.103 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.103 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.104 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.104 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:49.104 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:49.104 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.104 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.104 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.104 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.104 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.105 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.105 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.105 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.105 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.105 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.105 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.105 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.105 INFO:teuthology.orchestra.run.smithi001.stdout: "storage_class": { 2024-02-02T15:46:49.106 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "storage_class", 2024-02-02T15:46:49.106 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.106 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.106 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.106 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "local", 2024-02-02T15:46:49.106 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.106 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.106 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.107 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-02-02T15:46:49.107 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.107 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.107 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.107 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.107 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.107 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.107 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:49.107 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "selftest", 2024-02-02T15:46:49.108 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:49.108 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:49.108 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:49.108 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:49.108 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:49.108 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.108 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.108 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.109 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.109 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.109 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.109 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.109 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.109 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.109 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.109 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.110 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.110 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.110 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.110 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.110 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.110 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.110 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.110 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.111 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:49.111 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:49.111 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.111 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.111 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.111 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.111 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.111 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.112 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.112 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.112 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.112 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.112 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.112 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.112 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:49.112 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:49.112 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.113 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.113 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.113 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:49.113 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.113 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.113 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.113 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.113 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.114 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.114 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.114 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.114 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.114 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.114 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.114 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.114 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.114 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.114 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.115 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:49.115 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:49.115 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.115 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.115 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.115 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.115 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.115 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.115 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.116 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.116 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.116 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.116 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.116 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.116 INFO:teuthology.orchestra.run.smithi001.stdout: "roption1": { 2024-02-02T15:46:49.116 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "roption1", 2024-02-02T15:46:49.116 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.116 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.116 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.117 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.117 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.117 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.117 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.117 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.117 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.117 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.117 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.117 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.118 INFO:teuthology.orchestra.run.smithi001.stdout: "roption2": { 2024-02-02T15:46:49.118 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "roption2", 2024-02-02T15:46:49.118 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.118 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.118 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.118 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "xyz", 2024-02-02T15:46:49.118 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.118 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.118 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.119 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.119 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.119 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.119 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.119 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.119 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption1": { 2024-02-02T15:46:49.119 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption1", 2024-02-02T15:46:49.119 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.120 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.120 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.120 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.120 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.120 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.120 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.120 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.120 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.121 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.121 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.121 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.121 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption2": { 2024-02-02T15:46:49.121 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption2", 2024-02-02T15:46:49.121 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:49.121 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.121 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.122 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.122 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.122 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.122 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.122 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.122 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.122 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.122 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.123 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.123 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption3": { 2024-02-02T15:46:49.123 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption3", 2024-02-02T15:46:49.123 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-02-02T15:46:49.123 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.123 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.123 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.123 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.123 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.124 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.124 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.124 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.124 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.124 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.124 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.124 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption4": { 2024-02-02T15:46:49.124 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption4", 2024-02-02T15:46:49.125 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.125 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.125 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.125 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.125 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.125 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.125 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.125 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.125 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.126 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.126 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.126 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.126 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption5": { 2024-02-02T15:46:49.126 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption5", 2024-02-02T15:46:49.126 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.126 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.126 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.127 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.127 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.127 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.127 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.127 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.127 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.127 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.127 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.127 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.128 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption6": { 2024-02-02T15:46:49.128 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption6", 2024-02-02T15:46:49.128 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.128 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.128 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.128 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:49.128 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.128 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.129 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.129 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.129 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.129 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.129 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.129 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.129 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption7": { 2024-02-02T15:46:49.129 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption7", 2024-02-02T15:46:49.129 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:49.130 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.130 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.130 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.130 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1", 2024-02-02T15:46:49.130 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "42", 2024-02-02T15:46:49.130 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.130 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.130 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.131 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.131 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.131 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.131 INFO:teuthology.orchestra.run.smithi001.stdout: "testkey": { 2024-02-02T15:46:49.131 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "testkey", 2024-02-02T15:46:49.131 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.131 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.131 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.131 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.132 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.132 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.132 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.132 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.132 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.132 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.132 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.132 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.133 INFO:teuthology.orchestra.run.smithi001.stdout: "testlkey": { 2024-02-02T15:46:49.133 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "testlkey", 2024-02-02T15:46:49.133 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.133 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.133 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.133 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.133 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.133 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.134 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.134 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.134 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.134 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.134 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.134 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.134 INFO:teuthology.orchestra.run.smithi001.stdout: "testnewline": { 2024-02-02T15:46:49.134 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "testnewline", 2024-02-02T15:46:49.135 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.135 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.135 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.135 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.135 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.135 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.135 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.135 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.136 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.136 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.136 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.136 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.136 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.136 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.136 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:49.136 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "snap_schedule", 2024-02-02T15:46:49.137 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:49.137 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:49.137 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:49.137 INFO:teuthology.orchestra.run.smithi001.stdout: "allow_m_granularity": { 2024-02-02T15:46:49.137 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "allow_m_granularity", 2024-02-02T15:46:49.137 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.137 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.137 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.138 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.138 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.138 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.138 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.138 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "allow minute scheduled snapshots", 2024-02-02T15:46:49.138 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.138 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.138 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.139 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.139 INFO:teuthology.orchestra.run.smithi001.stdout: "dump_on_update": { 2024-02-02T15:46:49.139 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "dump_on_update", 2024-02-02T15:46:49.139 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.139 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.139 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.139 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.139 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.140 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.140 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.140 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "dump database to debug log on update", 2024-02-02T15:46:49.140 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.140 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.140 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.140 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.141 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:49.141 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:49.141 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.141 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.141 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.141 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.141 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.141 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.141 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.142 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.142 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.142 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.142 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.142 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.142 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.142 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.142 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.143 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.143 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.143 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.143 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.143 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:49.143 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:49.143 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.144 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.144 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.144 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.144 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.144 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.144 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.144 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.144 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.144 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.145 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.145 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.145 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:49.145 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:49.145 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.145 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.145 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.145 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:49.146 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.146 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.146 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.146 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.146 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.146 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.146 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.146 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.146 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.146 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.147 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.147 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.147 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.147 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.147 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.147 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:49.147 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:49.147 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.147 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.148 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.148 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.148 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.148 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.148 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.148 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.148 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.148 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.148 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.149 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.149 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.149 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.149 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:49.149 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "stats", 2024-02-02T15:46:49.149 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:49.149 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:49.149 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:49.149 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:49.149 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:49.150 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.150 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.150 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.150 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.150 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.150 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.150 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.150 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.150 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.151 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.151 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.151 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.151 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.151 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.151 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.151 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.151 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.152 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.152 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.152 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:49.152 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:49.152 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.152 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.152 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.153 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.153 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.153 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.153 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.153 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.153 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.153 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.153 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.154 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.154 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:49.154 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:49.154 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.154 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.154 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.154 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:49.154 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.154 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.155 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.155 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.155 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.155 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.155 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.155 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.155 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.155 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.156 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.156 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.156 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.156 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.156 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.156 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:49.156 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:49.156 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.157 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.157 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.157 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.157 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.157 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.157 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.157 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.157 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.157 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.157 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.158 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.158 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.158 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.158 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:49.158 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "status", 2024-02-02T15:46:49.158 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:49.158 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:49.158 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:49.159 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:49.159 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:49.159 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.159 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.159 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.159 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.159 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.159 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.159 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.159 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.160 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.160 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.160 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.160 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.160 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.160 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.160 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.161 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.161 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.161 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.161 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.161 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:49.161 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:49.161 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.161 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.161 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.162 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.162 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.162 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.162 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.162 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.162 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.162 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.162 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.163 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.163 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:49.163 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:49.163 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.163 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.163 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.163 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:49.163 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.164 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.164 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.164 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.164 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.164 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.164 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.164 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.164 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.165 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.165 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.165 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.165 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.165 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.165 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.165 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:49.165 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:49.165 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.166 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.166 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.166 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.166 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.166 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.166 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.166 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.166 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.166 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.167 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.167 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.167 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.167 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.167 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:49.167 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "telegraf", 2024-02-02T15:46:49.167 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:49.167 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:49.167 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:49.168 INFO:teuthology.orchestra.run.smithi001.stdout: "address": { 2024-02-02T15:46:49.168 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "address", 2024-02-02T15:46:49.168 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.168 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.168 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.168 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-02-02T15:46:49.168 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.168 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.169 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.169 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.169 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.169 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.169 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.169 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.169 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-02-02T15:46:49.170 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-02-02T15:46:49.170 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-02-02T15:46:49.170 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.170 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.170 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "15", 2024-02-02T15:46:49.170 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.170 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.170 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.171 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.171 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.171 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.171 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.171 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.171 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:49.171 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:49.171 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.171 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.172 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.172 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.172 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.172 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.172 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.172 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.172 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.172 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.173 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.173 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.173 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.173 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.173 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.173 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.173 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.173 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.174 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.174 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:49.174 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:49.174 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.174 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.174 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.174 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.174 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.175 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.175 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.175 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.175 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.175 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.175 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.175 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.175 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:49.175 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:49.176 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.176 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.176 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.176 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:49.176 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.176 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.176 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.176 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.177 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.177 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.177 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.177 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.177 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.177 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.177 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.177 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.177 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.178 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.178 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.178 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:49.178 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:49.178 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.178 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.178 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.178 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.178 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.178 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.179 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.179 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.179 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.179 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.179 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.179 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.179 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.179 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.180 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:49.180 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "telemetry", 2024-02-02T15:46:49.180 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:49.180 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:49.180 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:49.180 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_basic": { 2024-02-02T15:46:49.180 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_basic", 2024-02-02T15:46:49.180 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.180 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.180 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.181 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:49.181 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.181 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.181 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.181 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share basic cluster information (size, version)", 2024-02-02T15:46:49.181 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.181 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.181 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.182 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.182 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_crash": { 2024-02-02T15:46:49.182 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_crash", 2024-02-02T15:46:49.182 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.182 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.182 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.182 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:49.182 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.182 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.183 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.183 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-02-02T15:46:49.183 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.183 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.183 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.183 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.183 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_device": { 2024-02-02T15:46:49.183 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_device", 2024-02-02T15:46:49.184 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.184 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.184 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.184 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:49.184 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.184 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.184 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.184 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-02-02T15:46:49.185 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.185 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.185 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.185 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.185 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_ident": { 2024-02-02T15:46:49.185 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_ident", 2024-02-02T15:46:49.185 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.185 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.185 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.186 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.186 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.186 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.186 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.186 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-02-02T15:46:49.186 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.186 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.186 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.187 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.187 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_perf": { 2024-02-02T15:46:49.187 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_perf", 2024-02-02T15:46:49.187 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.187 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.187 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.187 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.187 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.188 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.188 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.188 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share various performance metrics of a cluster", 2024-02-02T15:46:49.188 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.188 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.188 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.188 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.189 INFO:teuthology.orchestra.run.smithi001.stdout: "contact": { 2024-02-02T15:46:49.189 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "contact", 2024-02-02T15:46:49.189 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.189 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.189 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.189 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.189 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.189 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.190 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.190 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.190 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.190 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.190 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.190 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.190 INFO:teuthology.orchestra.run.smithi001.stdout: "description": { 2024-02-02T15:46:49.190 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "description", 2024-02-02T15:46:49.191 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.191 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.191 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.191 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.191 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.191 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.191 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.191 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.192 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.192 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.192 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.192 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.192 INFO:teuthology.orchestra.run.smithi001.stdout: "device_url": { 2024-02-02T15:46:49.192 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "device_url", 2024-02-02T15:46:49.192 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.192 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.193 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.193 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-02-02T15:46:49.193 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.193 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.193 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.193 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.193 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.194 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.194 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.194 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.194 INFO:teuthology.orchestra.run.smithi001.stdout: "enabled": { 2024-02-02T15:46:49.194 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "enabled", 2024-02-02T15:46:49.194 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.194 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.194 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.195 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.195 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.195 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.195 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.195 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.195 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.195 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.195 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.196 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.196 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-02-02T15:46:49.196 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-02-02T15:46:49.196 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:49.196 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.196 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.196 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "24", 2024-02-02T15:46:49.196 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "8", 2024-02-02T15:46:49.196 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.197 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.197 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.197 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.197 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.197 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.197 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.197 INFO:teuthology.orchestra.run.smithi001.stdout: "last_opt_revision": { 2024-02-02T15:46:49.197 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "last_opt_revision", 2024-02-02T15:46:49.197 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:49.198 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.198 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.198 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "1", 2024-02-02T15:46:49.198 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.198 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.198 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.198 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.198 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.199 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.199 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.199 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.199 INFO:teuthology.orchestra.run.smithi001.stdout: "leaderboard": { 2024-02-02T15:46:49.199 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "leaderboard", 2024-02-02T15:46:49.199 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.199 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.199 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.200 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.200 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.200 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.200 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.200 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.200 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.200 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.200 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.200 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.201 INFO:teuthology.orchestra.run.smithi001.stdout: "leaderboard_description": { 2024-02-02T15:46:49.201 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "leaderboard_description", 2024-02-02T15:46:49.201 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.201 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.201 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.201 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.201 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.201 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.201 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.202 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.202 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.202 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.202 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.202 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.202 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:49.202 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:49.202 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.202 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.202 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.203 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.203 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.203 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.203 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.203 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.203 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.203 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.203 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.204 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.204 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.204 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.204 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.204 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.204 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.205 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.205 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.205 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:49.205 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:49.205 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.205 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.205 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.205 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.206 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.206 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.206 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.206 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.206 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.206 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.206 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.207 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.207 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:49.207 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:49.207 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.207 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.207 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.207 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:49.207 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.208 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.208 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.208 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.208 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.208 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.208 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.208 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.208 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.208 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.209 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.209 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.209 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.209 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.209 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.209 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:49.209 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:49.210 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.210 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.210 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.210 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.210 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.210 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.210 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.210 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.210 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.211 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.211 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.211 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.211 INFO:teuthology.orchestra.run.smithi001.stdout: "organization": { 2024-02-02T15:46:49.211 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "organization", 2024-02-02T15:46:49.211 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.211 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.211 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.211 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.211 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.212 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.212 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.212 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.212 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.212 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.212 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.212 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.212 INFO:teuthology.orchestra.run.smithi001.stdout: "proxy": { 2024-02-02T15:46:49.212 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "proxy", 2024-02-02T15:46:49.213 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.213 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.213 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.213 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.213 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.213 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.213 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.213 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.213 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.214 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.214 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.214 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.214 INFO:teuthology.orchestra.run.smithi001.stdout: "url": { 2024-02-02T15:46:49.214 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "url", 2024-02-02T15:46:49.214 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.214 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.215 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.215 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-02-02T15:46:49.215 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.215 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.215 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.215 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.215 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.215 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.215 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.216 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.216 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.216 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.216 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:49.216 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "test_orchestrator", 2024-02-02T15:46:49.216 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:49.216 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:49.216 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:49.216 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:49.217 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:49.217 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.217 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.217 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.217 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.217 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.217 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.217 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.217 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.218 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.218 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.218 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.218 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.218 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.218 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.218 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.218 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.219 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.219 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.219 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.219 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:49.219 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:49.219 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.219 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.219 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.219 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.220 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.220 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.220 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.220 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.220 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.220 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.220 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.220 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.221 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:49.221 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:49.221 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.221 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.221 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.221 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:49.221 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.221 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.222 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.222 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.222 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.222 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.222 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.222 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.222 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.222 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.222 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.223 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.223 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.223 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.223 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.223 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:49.223 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:49.223 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.223 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.223 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.224 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.224 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.224 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.224 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.224 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.224 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.224 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.224 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.225 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.225 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.225 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.225 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:49.225 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "volumes", 2024-02-02T15:46:49.225 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:49.225 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:49.225 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:49.225 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:49.226 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:49.226 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.226 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.226 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.226 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.226 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.226 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.226 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.227 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.227 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.227 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.227 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.227 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.227 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.227 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.227 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.228 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.228 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.228 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.228 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.228 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:49.228 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:49.228 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.228 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.229 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.229 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.229 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.229 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.229 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.229 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.229 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.229 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.229 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.230 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.230 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:49.230 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:49.230 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.230 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.230 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.230 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:49.230 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.230 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.231 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.231 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.231 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.231 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.231 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.231 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.231 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.231 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.231 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.232 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.232 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.232 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.232 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.232 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:49.232 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:49.232 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.232 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.232 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.233 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.233 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.233 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.233 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.233 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.233 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.233 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.233 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.233 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.234 INFO:teuthology.orchestra.run.smithi001.stdout: "max_concurrent_clones": { 2024-02-02T15:46:49.234 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_concurrent_clones", 2024-02-02T15:46:49.234 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:49.234 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.234 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.234 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "4", 2024-02-02T15:46:49.234 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.234 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.234 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.234 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Number of asynchronous cloner threads", 2024-02-02T15:46:49.235 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.235 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.235 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.235 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.235 INFO:teuthology.orchestra.run.smithi001.stdout: "snapshot_clone_delay": { 2024-02-02T15:46:49.235 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "snapshot_clone_delay", 2024-02-02T15:46:49.235 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:49.235 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.236 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.236 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-02-02T15:46:49.236 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.236 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.236 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.236 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-02-02T15:46:49.236 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.236 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.237 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.237 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.237 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.237 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.237 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:49.237 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "zabbix", 2024-02-02T15:46:49.237 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:49.237 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:49.238 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:49.238 INFO:teuthology.orchestra.run.smithi001.stdout: "discovery_interval": { 2024-02-02T15:46:49.238 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "discovery_interval", 2024-02-02T15:46:49.238 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-02-02T15:46:49.238 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.238 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.238 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "100", 2024-02-02T15:46:49.238 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.238 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.239 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.239 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.239 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.239 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.239 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.239 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.239 INFO:teuthology.orchestra.run.smithi001.stdout: "identifier": { 2024-02-02T15:46:49.239 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "identifier", 2024-02-02T15:46:49.239 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.240 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.240 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.240 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.240 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.240 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.240 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.240 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.240 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.240 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.241 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.241 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.241 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-02-02T15:46:49.241 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-02-02T15:46:49.241 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-02-02T15:46:49.241 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.241 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.241 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-02-02T15:46:49.242 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.242 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.242 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.242 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.242 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.242 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.242 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.242 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.242 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:49.243 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:49.243 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.243 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.243 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.243 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.243 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.243 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.243 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.244 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.244 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.244 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.244 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.244 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.244 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.244 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.244 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.245 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.245 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.245 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.245 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.245 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:49.245 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:49.245 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.245 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.245 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.246 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.246 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.246 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.246 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.246 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.246 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.246 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.246 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.246 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.247 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:49.247 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:49.247 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.247 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.247 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.247 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:49.247 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.247 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.248 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.248 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.248 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.248 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.248 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.248 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.248 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.248 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.249 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.249 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.249 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.249 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.249 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.249 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:49.249 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:49.249 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.249 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.250 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.250 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.250 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.250 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.250 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.250 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.250 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.250 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.250 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.251 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.251 INFO:teuthology.orchestra.run.smithi001.stdout: "zabbix_host": { 2024-02-02T15:46:49.251 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "zabbix_host", 2024-02-02T15:46:49.251 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.251 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.251 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.251 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.251 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.252 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.252 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.252 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.252 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.252 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.252 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.253 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.253 INFO:teuthology.orchestra.run.smithi001.stdout: "zabbix_port": { 2024-02-02T15:46:49.253 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "zabbix_port", 2024-02-02T15:46:49.253 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:49.253 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.253 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.253 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10051", 2024-02-02T15:46:49.253 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.253 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.254 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.254 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.254 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.254 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.254 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.254 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.254 INFO:teuthology.orchestra.run.smithi001.stdout: "zabbix_sender": { 2024-02-02T15:46:49.254 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "zabbix_sender", 2024-02-02T15:46:49.254 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.255 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.255 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.255 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-02-02T15:46:49.255 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.255 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.255 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.255 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.255 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.256 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.256 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.256 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.256 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.256 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.256 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-02-02T15:46:49.256 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.257 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.257 INFO:teuthology.orchestra.run.smithi001.stdout: "modules": [ 2024-02-02T15:46:49.257 INFO:teuthology.orchestra.run.smithi001.stdout: "cephadm", 2024-02-02T15:46:49.257 INFO:teuthology.orchestra.run.smithi001.stdout: "dashboard", 2024-02-02T15:46:49.257 INFO:teuthology.orchestra.run.smithi001.stdout: "iostat", 2024-02-02T15:46:49.257 INFO:teuthology.orchestra.run.smithi001.stdout: "nfs", 2024-02-02T15:46:49.257 INFO:teuthology.orchestra.run.smithi001.stdout: "restful", 2024-02-02T15:46:49.257 INFO:teuthology.orchestra.run.smithi001.stdout: "snap_schedule" 2024-02-02T15:46:49.258 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.258 INFO:teuthology.orchestra.run.smithi001.stdout: "available_modules": [ 2024-02-02T15:46:49.258 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:49.258 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "alerts", 2024-02-02T15:46:49.258 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:49.258 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:49.258 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:49.258 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-02-02T15:46:49.258 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-02-02T15:46:49.259 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-02-02T15:46:49.259 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.259 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.259 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-02-02T15:46:49.259 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.259 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.259 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.259 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "How frequently to reexamine health status", 2024-02-02T15:46:49.260 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.260 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.260 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.260 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.260 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:49.260 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:49.260 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.260 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.261 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.261 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.261 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.261 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.261 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.261 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.261 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.261 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.262 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.262 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.262 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.262 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.262 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.262 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.262 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.262 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.263 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.263 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:49.263 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:49.263 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.263 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.263 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.263 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.263 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.264 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.264 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.264 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.264 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.264 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.264 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.264 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.265 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:49.265 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:49.265 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.265 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.265 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.265 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:49.265 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.265 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.266 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.266 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.266 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.266 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.266 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.266 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.266 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.267 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.267 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.267 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.267 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.267 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.267 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.267 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:49.267 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:49.268 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.268 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.268 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.268 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.268 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.268 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.268 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.269 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.269 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.269 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.269 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.269 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.269 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_destination": { 2024-02-02T15:46:49.269 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_destination", 2024-02-02T15:46:49.269 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.270 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.270 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.270 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.270 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.270 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.270 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.270 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Email address to send alerts to", 2024-02-02T15:46:49.270 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.270 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.270 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.271 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.271 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_from_name": { 2024-02-02T15:46:49.271 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_from_name", 2024-02-02T15:46:49.271 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.271 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.271 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.271 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "Ceph", 2024-02-02T15:46:49.271 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.272 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.272 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.272 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Email From: name", 2024-02-02T15:46:49.272 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.272 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.272 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.272 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.272 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_host": { 2024-02-02T15:46:49.272 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_host", 2024-02-02T15:46:49.272 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.273 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.273 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.273 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.273 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.273 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.273 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.273 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "SMTP server", 2024-02-02T15:46:49.273 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.273 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.273 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.274 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.274 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_password": { 2024-02-02T15:46:49.274 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_password", 2024-02-02T15:46:49.274 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.274 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.274 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.274 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.274 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.274 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.274 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.275 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Password to authenticate with", 2024-02-02T15:46:49.275 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.275 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.275 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.275 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.275 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_port": { 2024-02-02T15:46:49.275 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_port", 2024-02-02T15:46:49.275 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:49.276 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.276 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.276 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "465", 2024-02-02T15:46:49.276 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.276 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.276 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.276 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "SMTP port", 2024-02-02T15:46:49.276 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.276 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.277 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.277 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.277 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_sender": { 2024-02-02T15:46:49.277 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_sender", 2024-02-02T15:46:49.277 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.277 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.277 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.277 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.278 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.278 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.278 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.278 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "SMTP envelope sender", 2024-02-02T15:46:49.278 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.278 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.278 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.278 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.278 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_ssl": { 2024-02-02T15:46:49.279 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_ssl", 2024-02-02T15:46:49.279 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.279 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.279 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.279 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:49.279 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.279 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.280 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.280 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Use SSL to connect to SMTP server", 2024-02-02T15:46:49.280 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.280 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.280 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.280 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.280 INFO:teuthology.orchestra.run.smithi001.stdout: "smtp_user": { 2024-02-02T15:46:49.280 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "smtp_user", 2024-02-02T15:46:49.281 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.281 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.281 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.281 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.281 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.281 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.281 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.281 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "User to authenticate as", 2024-02-02T15:46:49.281 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.282 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.282 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.282 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.282 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.282 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.282 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:49.282 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "balancer", 2024-02-02T15:46:49.283 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:49.283 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:49.283 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:49.283 INFO:teuthology.orchestra.run.smithi001.stdout: "active": { 2024-02-02T15:46:49.283 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "active", 2024-02-02T15:46:49.283 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.283 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.283 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.284 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:49.284 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.284 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.284 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.284 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "automatically balance PGs across cluster", 2024-02-02T15:46:49.284 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.284 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.284 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.285 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.285 INFO:teuthology.orchestra.run.smithi001.stdout: "begin_time": { 2024-02-02T15:46:49.285 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "begin_time", 2024-02-02T15:46:49.285 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.285 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.285 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.285 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0000", 2024-02-02T15:46:49.285 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.286 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.286 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.286 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "beginning time of day to automatically balance", 2024-02-02T15:46:49.286 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-02-02T15:46:49.286 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.286 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.286 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.286 INFO:teuthology.orchestra.run.smithi001.stdout: "begin_weekday": { 2024-02-02T15:46:49.287 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "begin_weekday", 2024-02-02T15:46:49.287 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-02-02T15:46:49.287 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.287 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.287 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-02-02T15:46:49.287 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "0", 2024-02-02T15:46:49.287 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "6", 2024-02-02T15:46:49.288 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.288 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-02-02T15:46:49.288 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-02-02T15:46:49.288 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.288 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.288 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.288 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_compat_max_iterations": { 2024-02-02T15:46:49.288 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crush_compat_max_iterations", 2024-02-02T15:46:49.289 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-02-02T15:46:49.289 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.289 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.289 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "25", 2024-02-02T15:46:49.289 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1", 2024-02-02T15:46:49.289 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "250", 2024-02-02T15:46:49.289 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.289 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-02-02T15:46:49.290 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.290 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.290 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.290 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.290 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_compat_metrics": { 2024-02-02T15:46:49.290 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crush_compat_metrics", 2024-02-02T15:46:49.290 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.290 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.291 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.291 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "pgs,objects,bytes", 2024-02-02T15:46:49.291 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.291 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.291 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.291 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-02-02T15:46:49.291 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-02-02T15:46:49.291 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.292 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.292 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.292 INFO:teuthology.orchestra.run.smithi001.stdout: "crush_compat_step": { 2024-02-02T15:46:49.292 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crush_compat_step", 2024-02-02T15:46:49.292 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-02-02T15:46:49.292 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.292 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.292 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0.5", 2024-02-02T15:46:49.292 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "0.001", 2024-02-02T15:46:49.293 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "0.999", 2024-02-02T15:46:49.293 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.293 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "aggressiveness of optimization", 2024-02-02T15:46:49.293 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-02-02T15:46:49.293 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.293 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.293 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.293 INFO:teuthology.orchestra.run.smithi001.stdout: "end_time": { 2024-02-02T15:46:49.294 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "end_time", 2024-02-02T15:46:49.294 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.294 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.294 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.294 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "2359", 2024-02-02T15:46:49.294 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.294 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.294 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.294 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "ending time of day to automatically balance", 2024-02-02T15:46:49.295 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-02-02T15:46:49.295 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.295 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.295 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.295 INFO:teuthology.orchestra.run.smithi001.stdout: "end_weekday": { 2024-02-02T15:46:49.295 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "end_weekday", 2024-02-02T15:46:49.295 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-02-02T15:46:49.295 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.296 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.296 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-02-02T15:46:49.296 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "0", 2024-02-02T15:46:49.296 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "6", 2024-02-02T15:46:49.296 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.296 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-02-02T15:46:49.296 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-02-02T15:46:49.296 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.297 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.297 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.297 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:49.297 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:49.297 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.297 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.297 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.297 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.297 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.297 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.298 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.298 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.298 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.298 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.298 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.298 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.298 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.298 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.298 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.299 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.299 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.299 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.299 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.299 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:49.299 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:49.299 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.299 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.299 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.299 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.300 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.300 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.300 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.300 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.300 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.300 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.300 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.300 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.300 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:49.300 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:49.301 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.301 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.301 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.301 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:49.301 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.301 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.301 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.301 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.302 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.302 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.302 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.302 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.302 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.302 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.302 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.302 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.303 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.303 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.303 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.303 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:49.303 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:49.303 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.303 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.303 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.303 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.303 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.304 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.304 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.304 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.304 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.304 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.304 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.304 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.304 INFO:teuthology.orchestra.run.smithi001.stdout: "min_score": { 2024-02-02T15:46:49.304 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "min_score", 2024-02-02T15:46:49.305 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-02-02T15:46:49.305 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.305 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.305 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-02-02T15:46:49.305 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.305 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.305 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.305 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-02-02T15:46:49.305 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.306 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.306 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.306 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.306 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": { 2024-02-02T15:46:49.306 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mode", 2024-02-02T15:46:49.306 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.306 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.306 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.306 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "upmap", 2024-02-02T15:46:49.306 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.307 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.307 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.307 INFO:teuthology.orchestra.run.smithi001.stdout: "crush-compat", 2024-02-02T15:46:49.307 INFO:teuthology.orchestra.run.smithi001.stdout: "none", 2024-02-02T15:46:49.307 INFO:teuthology.orchestra.run.smithi001.stdout: "upmap" 2024-02-02T15:46:49.307 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.307 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Balancer mode", 2024-02-02T15:46:49.307 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.307 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.308 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.308 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.308 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_ids": { 2024-02-02T15:46:49.308 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "pool_ids", 2024-02-02T15:46:49.308 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.308 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.308 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.308 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.308 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.309 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.309 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.309 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-02-02T15:46:49.309 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.309 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.309 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.309 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.309 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-02-02T15:46:49.310 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-02-02T15:46:49.310 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-02-02T15:46:49.310 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.310 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.310 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-02-02T15:46:49.310 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.310 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.310 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.311 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-02-02T15:46:49.311 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.311 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.311 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.311 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.311 INFO:teuthology.orchestra.run.smithi001.stdout: "upmap_max_deviation": { 2024-02-02T15:46:49.311 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "upmap_max_deviation", 2024-02-02T15:46:49.312 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:49.312 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.312 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.312 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2024-02-02T15:46:49.312 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1", 2024-02-02T15:46:49.312 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.312 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.312 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "deviation below which no optimization is attempted", 2024-02-02T15:46:49.313 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-02-02T15:46:49.313 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.313 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.313 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.313 INFO:teuthology.orchestra.run.smithi001.stdout: "upmap_max_optimizations": { 2024-02-02T15:46:49.313 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "upmap_max_optimizations", 2024-02-02T15:46:49.313 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-02-02T15:46:49.313 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.314 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.314 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-02-02T15:46:49.314 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.314 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.314 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.314 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-02-02T15:46:49.314 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.314 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.315 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.315 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.315 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.315 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.315 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:49.315 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cephadm", 2024-02-02T15:46:49.315 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:49.315 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:49.315 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:49.316 INFO:teuthology.orchestra.run.smithi001.stdout: "agent_down_multiplier": { 2024-02-02T15:46:49.316 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "agent_down_multiplier", 2024-02-02T15:46:49.316 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-02-02T15:46:49.316 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.316 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.316 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "3.0", 2024-02-02T15:46:49.316 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.316 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.316 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.317 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-02-02T15:46:49.317 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.317 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.317 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.317 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.317 INFO:teuthology.orchestra.run.smithi001.stdout: "agent_refresh_rate": { 2024-02-02T15:46:49.317 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "agent_refresh_rate", 2024-02-02T15:46:49.317 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-02-02T15:46:49.317 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.317 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.318 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "20", 2024-02-02T15:46:49.318 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.318 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.318 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.318 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-02-02T15:46:49.318 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.318 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.318 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.318 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.318 INFO:teuthology.orchestra.run.smithi001.stdout: "agent_starting_port": { 2024-02-02T15:46:49.319 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "agent_starting_port", 2024-02-02T15:46:49.319 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:49.319 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.319 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.319 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "4721", 2024-02-02T15:46:49.319 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.319 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.319 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.319 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-02-02T15:46:49.320 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.320 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.320 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.320 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.320 INFO:teuthology.orchestra.run.smithi001.stdout: "allow_ptrace": { 2024-02-02T15:46:49.320 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "allow_ptrace", 2024-02-02T15:46:49.320 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.320 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.320 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.320 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.321 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.321 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.321 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.321 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-02-02T15:46:49.321 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-02-02T15:46:49.321 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.321 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.321 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.321 INFO:teuthology.orchestra.run.smithi001.stdout: "autotune_interval": { 2024-02-02T15:46:49.321 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "autotune_interval", 2024-02-02T15:46:49.322 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-02-02T15:46:49.322 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.322 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.322 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-02-02T15:46:49.322 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.322 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.322 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.322 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to autotune daemon memory", 2024-02-02T15:46:49.322 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.323 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.323 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.323 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.323 INFO:teuthology.orchestra.run.smithi001.stdout: "autotune_memory_target_ratio": { 2024-02-02T15:46:49.323 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "autotune_memory_target_ratio", 2024-02-02T15:46:49.323 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-02-02T15:46:49.323 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.323 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.323 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0.7", 2024-02-02T15:46:49.324 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.324 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.324 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.324 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-02-02T15:46:49.324 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.324 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.324 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.324 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.325 INFO:teuthology.orchestra.run.smithi001.stdout: "cgroups_split": { 2024-02-02T15:46:49.325 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cgroups_split", 2024-02-02T15:46:49.325 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.325 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.325 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.325 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:49.325 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.325 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.325 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.326 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-02-02T15:46:49.326 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.326 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.326 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.326 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.326 INFO:teuthology.orchestra.run.smithi001.stdout: "config_checks_enabled": { 2024-02-02T15:46:49.326 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "config_checks_enabled", 2024-02-02T15:46:49.326 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.326 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.327 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.327 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.327 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.327 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.327 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.327 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-02-02T15:46:49.327 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.327 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.328 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.328 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.328 INFO:teuthology.orchestra.run.smithi001.stdout: "config_dashboard": { 2024-02-02T15:46:49.328 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "config_dashboard", 2024-02-02T15:46:49.328 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.328 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.328 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.328 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:49.329 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.329 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.329 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.329 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-02-02T15:46:49.329 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.329 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.329 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.329 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.330 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_alertmanager": { 2024-02-02T15:46:49.330 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_alertmanager", 2024-02-02T15:46:49.330 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.330 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.330 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.330 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-02-02T15:46:49.330 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.330 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.331 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.331 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus container image", 2024-02-02T15:46:49.331 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.331 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.331 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.331 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.331 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_base": { 2024-02-02T15:46:49.331 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_base", 2024-02-02T15:46:49.332 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.332 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.332 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.332 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/ceph", 2024-02-02T15:46:49.332 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.332 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.332 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.332 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Container image name, without the tag", 2024-02-02T15:46:49.333 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.333 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.333 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.333 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.333 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_elasticsearch": { 2024-02-02T15:46:49.333 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_elasticsearch", 2024-02-02T15:46:49.333 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.333 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.334 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.334 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-02-02T15:46:49.334 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.334 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.334 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.334 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "elasticsearch container image", 2024-02-02T15:46:49.334 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.334 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.334 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.335 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.335 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_grafana": { 2024-02-02T15:46:49.335 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_grafana", 2024-02-02T15:46:49.335 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.335 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.335 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.335 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2024-02-02T15:46:49.335 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.335 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.336 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.336 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus container image", 2024-02-02T15:46:49.336 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.336 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.336 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.336 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.336 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_haproxy": { 2024-02-02T15:46:49.336 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_haproxy", 2024-02-02T15:46:49.337 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.337 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.337 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.337 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-02-02T15:46:49.337 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.337 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.337 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.338 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "HAproxy container image", 2024-02-02T15:46:49.338 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.338 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.338 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.338 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.338 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_jaeger_agent": { 2024-02-02T15:46:49.338 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_jaeger_agent", 2024-02-02T15:46:49.338 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.339 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.339 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.339 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-02-02T15:46:49.339 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.339 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.339 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.339 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Jaeger agent container image", 2024-02-02T15:46:49.339 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.339 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.340 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.340 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.340 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_jaeger_collector": { 2024-02-02T15:46:49.340 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_jaeger_collector", 2024-02-02T15:46:49.340 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.340 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.340 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.340 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-02-02T15:46:49.341 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.341 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.341 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.341 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Jaeger collector container image", 2024-02-02T15:46:49.341 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.341 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.341 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.341 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.341 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_jaeger_query": { 2024-02-02T15:46:49.342 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_jaeger_query", 2024-02-02T15:46:49.342 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.342 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.342 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.342 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-02-02T15:46:49.342 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.342 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.343 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.343 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Jaeger query container image", 2024-02-02T15:46:49.343 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.343 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.343 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.343 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.343 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_keepalived": { 2024-02-02T15:46:49.343 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_keepalived", 2024-02-02T15:46:49.344 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.344 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.344 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.344 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-02-02T15:46:49.344 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.344 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.344 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.344 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Keepalived container image", 2024-02-02T15:46:49.345 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.345 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.345 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.345 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.345 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_loki": { 2024-02-02T15:46:49.345 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_loki", 2024-02-02T15:46:49.345 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.345 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.345 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.346 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-02-02T15:46:49.346 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.346 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.346 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.346 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Loki container image", 2024-02-02T15:46:49.346 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.346 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.346 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.347 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.347 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_node_exporter": { 2024-02-02T15:46:49.347 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_node_exporter", 2024-02-02T15:46:49.347 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.347 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.347 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.347 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-02-02T15:46:49.347 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.348 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.348 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.348 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus container image", 2024-02-02T15:46:49.348 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.348 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.348 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.348 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.348 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_nvmeof": { 2024-02-02T15:46:49.349 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_nvmeof", 2024-02-02T15:46:49.349 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.349 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.349 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.349 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/ceph/nvmeof:0.0.2", 2024-02-02T15:46:49.349 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.349 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.349 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.350 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Nvme-of container image", 2024-02-02T15:46:49.350 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.350 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.350 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.350 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.350 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_prometheus": { 2024-02-02T15:46:49.350 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_prometheus", 2024-02-02T15:46:49.350 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.350 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.351 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.351 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-02-02T15:46:49.351 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.351 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.351 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.351 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Prometheus container image", 2024-02-02T15:46:49.351 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.351 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.352 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.352 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.352 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_promtail": { 2024-02-02T15:46:49.352 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_promtail", 2024-02-02T15:46:49.352 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.352 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.352 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.353 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-02-02T15:46:49.353 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.353 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.353 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.353 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Promtail container image", 2024-02-02T15:46:49.353 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.353 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.353 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.354 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.354 INFO:teuthology.orchestra.run.smithi001.stdout: "container_image_snmp_gateway": { 2024-02-02T15:46:49.354 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_image_snmp_gateway", 2024-02-02T15:46:49.354 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.354 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.354 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.354 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-02-02T15:46:49.354 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.354 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.355 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.355 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "SNMP Gateway container image", 2024-02-02T15:46:49.355 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.355 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.355 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.355 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.355 INFO:teuthology.orchestra.run.smithi001.stdout: "container_init": { 2024-02-02T15:46:49.355 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "container_init", 2024-02-02T15:46:49.355 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.356 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.356 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.356 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:49.356 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.356 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.356 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.356 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Run podman/docker with `--init`", 2024-02-02T15:46:49.356 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.357 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.357 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.357 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.357 INFO:teuthology.orchestra.run.smithi001.stdout: "daemon_cache_timeout": { 2024-02-02T15:46:49.357 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "daemon_cache_timeout", 2024-02-02T15:46:49.357 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-02-02T15:46:49.357 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.357 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.357 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-02-02T15:46:49.357 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.358 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.358 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.358 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-02-02T15:46:49.358 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.358 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.358 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.358 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.358 INFO:teuthology.orchestra.run.smithi001.stdout: "default_cephadm_command_timeout": { 2024-02-02T15:46:49.358 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "default_cephadm_command_timeout", 2024-02-02T15:46:49.359 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-02-02T15:46:49.359 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.359 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.359 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "900", 2024-02-02T15:46:49.359 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.359 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.359 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.359 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-02-02T15:46:49.359 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.360 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.360 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.360 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.360 INFO:teuthology.orchestra.run.smithi001.stdout: "default_registry": { 2024-02-02T15:46:49.360 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "default_registry", 2024-02-02T15:46:49.360 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.360 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.360 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.361 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "docker.io", 2024-02-02T15:46:49.361 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.361 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.361 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.361 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-02-02T15:46:49.361 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.361 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.361 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.362 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.362 INFO:teuthology.orchestra.run.smithi001.stdout: "device_cache_timeout": { 2024-02-02T15:46:49.362 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "device_cache_timeout", 2024-02-02T15:46:49.362 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-02-02T15:46:49.362 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.362 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.362 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "1800", 2024-02-02T15:46:49.362 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.362 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.363 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.363 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "seconds to cache device inventory", 2024-02-02T15:46:49.363 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.363 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.363 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.363 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.363 INFO:teuthology.orchestra.run.smithi001.stdout: "device_enhanced_scan": { 2024-02-02T15:46:49.363 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "device_enhanced_scan", 2024-02-02T15:46:49.363 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.364 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.364 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.364 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.364 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.364 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.364 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.364 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Use libstoragemgmt during device scans", 2024-02-02T15:46:49.364 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.365 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.365 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.365 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.365 INFO:teuthology.orchestra.run.smithi001.stdout: "facts_cache_timeout": { 2024-02-02T15:46:49.365 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "facts_cache_timeout", 2024-02-02T15:46:49.365 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-02-02T15:46:49.365 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.365 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.365 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-02-02T15:46:49.366 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.366 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.366 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.366 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "seconds to cache host facts data", 2024-02-02T15:46:49.366 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.366 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.366 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.366 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.366 INFO:teuthology.orchestra.run.smithi001.stdout: "host_check_interval": { 2024-02-02T15:46:49.367 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "host_check_interval", 2024-02-02T15:46:49.367 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-02-02T15:46:49.367 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.367 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.367 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-02-02T15:46:49.367 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.367 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.367 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.367 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to perform a host check", 2024-02-02T15:46:49.367 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.368 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.368 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.368 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.368 INFO:teuthology.orchestra.run.smithi001.stdout: "inventory_list_all": { 2024-02-02T15:46:49.368 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "inventory_list_all", 2024-02-02T15:46:49.368 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.368 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.368 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.368 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.369 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.369 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.369 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.369 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-02-02T15:46:49.369 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.369 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.369 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.369 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.369 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:49.369 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:49.370 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.370 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.370 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.370 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.370 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.370 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.370 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.370 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.370 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.370 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.371 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.371 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.371 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.371 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.371 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.371 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.371 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.371 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.371 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.372 INFO:teuthology.orchestra.run.smithi001.stdout: "log_refresh_metadata": { 2024-02-02T15:46:49.372 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_refresh_metadata", 2024-02-02T15:46:49.372 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.372 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.372 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.372 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.372 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.372 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.372 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.373 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-02-02T15:46:49.373 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.373 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.373 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.373 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.373 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:49.373 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:49.373 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.373 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.374 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.374 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:49.374 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.374 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.374 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.374 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-02-02T15:46:49.374 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.374 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.375 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.375 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.375 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:49.375 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:49.375 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.375 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.375 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.375 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:49.375 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.376 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.376 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.376 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.376 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.376 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.376 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.376 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.376 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.376 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.377 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.377 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.377 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.377 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.377 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.377 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:49.377 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:49.377 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.377 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.378 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.378 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.378 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.378 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.378 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.378 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.378 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.378 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.379 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.379 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.379 INFO:teuthology.orchestra.run.smithi001.stdout: "manage_etc_ceph_ceph_conf": { 2024-02-02T15:46:49.379 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-02-02T15:46:49.379 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.379 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.379 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.379 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.380 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.380 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.380 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.380 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-02-02T15:46:49.380 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.380 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.380 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.380 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.381 INFO:teuthology.orchestra.run.smithi001.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-02-02T15:46:49.381 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-02-02T15:46:49.381 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.381 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.381 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.381 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "*", 2024-02-02T15:46:49.381 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.381 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.382 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.382 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-02-02T15:46:49.382 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.382 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.382 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.382 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.382 INFO:teuthology.orchestra.run.smithi001.stdout: "max_count_per_host": { 2024-02-02T15:46:49.382 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_count_per_host", 2024-02-02T15:46:49.382 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:49.383 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.383 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.383 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-02-02T15:46:49.383 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.383 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.383 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.383 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "max number of daemons per service per host", 2024-02-02T15:46:49.384 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.384 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.384 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.384 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.384 INFO:teuthology.orchestra.run.smithi001.stdout: "max_osd_draining_count": { 2024-02-02T15:46:49.384 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_osd_draining_count", 2024-02-02T15:46:49.384 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:49.385 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.385 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.385 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-02-02T15:46:49.385 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.385 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.385 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.385 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-02-02T15:46:49.385 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.386 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.386 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.386 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.386 INFO:teuthology.orchestra.run.smithi001.stdout: "migration_current": { 2024-02-02T15:46:49.386 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "migration_current", 2024-02-02T15:46:49.386 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:49.386 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.386 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.387 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.387 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.387 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.387 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.387 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "internal - do not modify", 2024-02-02T15:46:49.387 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.387 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.387 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.387 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.388 INFO:teuthology.orchestra.run.smithi001.stdout: "mode": { 2024-02-02T15:46:49.388 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mode", 2024-02-02T15:46:49.388 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.388 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.388 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.388 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "root", 2024-02-02T15:46:49.388 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.388 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.389 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.389 INFO:teuthology.orchestra.run.smithi001.stdout: "cephadm-package", 2024-02-02T15:46:49.389 INFO:teuthology.orchestra.run.smithi001.stdout: "root" 2024-02-02T15:46:49.389 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.389 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "mode for remote execution of cephadm", 2024-02-02T15:46:49.389 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.389 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.389 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.390 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.390 INFO:teuthology.orchestra.run.smithi001.stdout: "prometheus_alerts_path": { 2024-02-02T15:46:49.390 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "prometheus_alerts_path", 2024-02-02T15:46:49.390 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.390 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.390 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.390 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-02-02T15:46:49.390 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.390 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.390 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.391 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-02-02T15:46:49.391 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.391 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.391 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.391 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.391 INFO:teuthology.orchestra.run.smithi001.stdout: "registry_insecure": { 2024-02-02T15:46:49.391 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "registry_insecure", 2024-02-02T15:46:49.391 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.391 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.391 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.392 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.392 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.392 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.392 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.392 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-02-02T15:46:49.392 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.392 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.392 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.393 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.393 INFO:teuthology.orchestra.run.smithi001.stdout: "registry_password": { 2024-02-02T15:46:49.393 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "registry_password", 2024-02-02T15:46:49.393 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.393 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.393 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.393 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.393 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.394 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.394 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.394 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-02-02T15:46:49.394 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.394 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.394 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.394 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.394 INFO:teuthology.orchestra.run.smithi001.stdout: "registry_url": { 2024-02-02T15:46:49.395 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "registry_url", 2024-02-02T15:46:49.395 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.395 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.395 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.395 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.395 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.395 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.395 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.395 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-02-02T15:46:49.396 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.396 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.396 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.396 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.396 INFO:teuthology.orchestra.run.smithi001.stdout: "registry_username": { 2024-02-02T15:46:49.396 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "registry_username", 2024-02-02T15:46:49.396 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.396 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.397 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.397 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.397 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.397 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.397 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.397 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-02-02T15:46:49.397 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.398 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.398 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.398 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.398 INFO:teuthology.orchestra.run.smithi001.stdout: "secure_monitoring_stack": { 2024-02-02T15:46:49.398 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "secure_monitoring_stack", 2024-02-02T15:46:49.398 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.398 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.398 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.398 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.399 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.399 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.399 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.399 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-02-02T15:46:49.399 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.399 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.399 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.399 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.399 INFO:teuthology.orchestra.run.smithi001.stdout: "service_discovery_port": { 2024-02-02T15:46:49.399 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "service_discovery_port", 2024-02-02T15:46:49.400 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:49.400 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.400 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.400 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8765", 2024-02-02T15:46:49.400 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.400 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.400 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.400 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "cephadm service discovery port", 2024-02-02T15:46:49.400 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.401 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.401 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.401 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.401 INFO:teuthology.orchestra.run.smithi001.stdout: "ssh_config_file": { 2024-02-02T15:46:49.401 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ssh_config_file", 2024-02-02T15:46:49.401 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.401 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.401 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.401 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.401 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.402 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.402 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.402 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-02-02T15:46:49.402 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.402 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.402 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.402 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.402 INFO:teuthology.orchestra.run.smithi001.stdout: "use_agent": { 2024-02-02T15:46:49.402 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "use_agent", 2024-02-02T15:46:49.403 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.403 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.403 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.403 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.403 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.403 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.403 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.403 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-02-02T15:46:49.403 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.404 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.404 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.404 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.404 INFO:teuthology.orchestra.run.smithi001.stdout: "use_repo_digest": { 2024-02-02T15:46:49.404 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "use_repo_digest", 2024-02-02T15:46:49.404 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.404 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.404 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.404 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:49.405 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.405 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.405 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.405 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-02-02T15:46:49.405 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.405 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.405 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.405 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.406 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_on_failed_host_check": { 2024-02-02T15:46:49.406 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_on_failed_host_check", 2024-02-02T15:46:49.406 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.406 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.406 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.406 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:49.406 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.406 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.407 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.407 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "raise a health warning if the host check fails", 2024-02-02T15:46:49.407 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.407 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.407 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.407 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.407 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_on_stray_daemons": { 2024-02-02T15:46:49.407 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_on_stray_daemons", 2024-02-02T15:46:49.407 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.408 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.408 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.408 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:49.408 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.408 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.408 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.408 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-02-02T15:46:49.408 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.409 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.409 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.409 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.409 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_on_stray_hosts": { 2024-02-02T15:46:49.409 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_on_stray_hosts", 2024-02-02T15:46:49.409 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.409 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.409 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.410 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:49.410 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.410 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.410 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.410 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-02-02T15:46:49.410 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.410 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.410 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.410 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.411 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.411 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.411 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:49.411 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crash", 2024-02-02T15:46:49.411 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:49.411 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:49.411 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:49.411 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:49.412 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:49.412 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.412 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.412 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.412 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.412 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.412 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.412 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.413 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.413 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.413 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.413 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.413 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.413 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.413 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.413 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.414 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.414 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.414 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.414 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.414 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:49.414 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:49.414 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.414 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.414 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.415 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.415 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.415 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.415 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.415 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.415 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.415 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.415 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.415 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.416 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:49.416 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:49.416 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.416 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.416 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.416 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:49.416 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.416 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.417 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.417 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.417 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.417 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.417 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.417 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.417 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.417 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.417 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.418 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.418 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.418 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.418 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.418 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:49.418 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:49.418 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.418 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.418 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.418 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.419 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.419 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.419 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.419 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.419 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.419 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.419 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.419 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.419 INFO:teuthology.orchestra.run.smithi001.stdout: "retain_interval": { 2024-02-02T15:46:49.419 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "retain_interval", 2024-02-02T15:46:49.420 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-02-02T15:46:49.420 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.420 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.420 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "31536000", 2024-02-02T15:46:49.420 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.420 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.420 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.420 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how long to retain crashes before pruning them", 2024-02-02T15:46:49.423 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.423 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.423 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.423 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.423 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_recent_interval": { 2024-02-02T15:46:49.423 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_recent_interval", 2024-02-02T15:46:49.423 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-02-02T15:46:49.423 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.424 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.424 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "1209600", 2024-02-02T15:46:49.424 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.424 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.424 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.424 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "time interval in which to warn about recent crashes", 2024-02-02T15:46:49.424 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.424 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.425 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.425 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.425 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.425 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.425 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:49.425 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "dashboard", 2024-02-02T15:46:49.425 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:49.425 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:49.426 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:49.426 INFO:teuthology.orchestra.run.smithi001.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-02-02T15:46:49.426 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-02-02T15:46:49.426 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:49.426 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.426 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.426 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-02-02T15:46:49.426 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.427 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.427 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.427 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.427 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.427 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.427 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.427 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.427 INFO:teuthology.orchestra.run.smithi001.stdout: "ALERTMANAGER_API_HOST": { 2024-02-02T15:46:49.428 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ALERTMANAGER_API_HOST", 2024-02-02T15:46:49.428 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.428 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.428 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.428 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.428 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.428 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.428 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.428 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.429 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.429 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.429 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.429 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.429 INFO:teuthology.orchestra.run.smithi001.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-02-02T15:46:49.429 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-02-02T15:46:49.429 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.429 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.430 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.430 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:49.430 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.430 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.430 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.430 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.430 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.430 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.430 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.430 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.431 INFO:teuthology.orchestra.run.smithi001.stdout: "AUDIT_API_ENABLED": { 2024-02-02T15:46:49.431 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "AUDIT_API_ENABLED", 2024-02-02T15:46:49.431 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.431 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.431 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.431 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.431 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.431 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.431 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.432 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.432 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.432 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.432 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.432 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.432 INFO:teuthology.orchestra.run.smithi001.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-02-02T15:46:49.432 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-02-02T15:46:49.432 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.433 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.433 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.433 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:49.433 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.433 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.433 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.433 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.433 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.434 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.434 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.434 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.434 INFO:teuthology.orchestra.run.smithi001.stdout: "ENABLE_BROWSABLE_API": { 2024-02-02T15:46:49.434 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ENABLE_BROWSABLE_API", 2024-02-02T15:46:49.434 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.434 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.435 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.435 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:49.435 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.435 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.435 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.435 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.435 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.435 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.436 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.436 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.436 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-02-02T15:46:49.436 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-02-02T15:46:49.436 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.436 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.436 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.436 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:49.437 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.437 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.437 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.437 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.437 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.437 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.437 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.437 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.437 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-02-02T15:46:49.438 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-02-02T15:46:49.438 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.438 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.438 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.438 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:49.438 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.438 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.438 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.438 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.438 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.439 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.439 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.439 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.439 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-02-02T15:46:49.439 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-02-02T15:46:49.439 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.439 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.439 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.439 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:49.440 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.440 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.440 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.440 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.440 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.440 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.440 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.440 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.441 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-02-02T15:46:49.441 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-02-02T15:46:49.441 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.441 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.441 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.441 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:49.441 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.441 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.442 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.442 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.442 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.442 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.442 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.442 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.442 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_NFS": { 2024-02-02T15:46:49.442 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-02-02T15:46:49.443 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.443 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.443 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.443 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:49.443 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.443 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.443 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.443 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.443 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.444 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.444 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.444 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.444 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_RBD": { 2024-02-02T15:46:49.444 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-02-02T15:46:49.444 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.444 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.444 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.445 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:49.445 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.445 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.445 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.445 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.445 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.445 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.445 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.445 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.445 INFO:teuthology.orchestra.run.smithi001.stdout: "FEATURE_TOGGLE_RGW": { 2024-02-02T15:46:49.446 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-02-02T15:46:49.446 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.446 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.446 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.446 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:49.446 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.446 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.446 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.446 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.447 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.447 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.447 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.447 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.447 INFO:teuthology.orchestra.run.smithi001.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-02-02T15:46:49.447 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-02-02T15:46:49.447 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.447 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.447 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.447 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.448 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.448 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.448 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.448 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.448 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.448 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.448 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.448 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.448 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_API_PASSWORD": { 2024-02-02T15:46:49.449 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_API_PASSWORD", 2024-02-02T15:46:49.449 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.449 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.449 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.449 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "admin", 2024-02-02T15:46:49.449 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.449 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.449 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.449 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.449 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.450 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.450 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.450 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.450 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-02-02T15:46:49.450 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-02-02T15:46:49.450 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.450 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.450 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.450 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:49.451 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.451 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.451 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.451 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.451 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.451 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.451 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.451 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.451 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_API_URL": { 2024-02-02T15:46:49.451 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_API_URL", 2024-02-02T15:46:49.452 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.452 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.452 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.452 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.452 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.452 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.452 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.452 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.452 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.453 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.453 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.453 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.453 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_API_USERNAME": { 2024-02-02T15:46:49.453 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_API_USERNAME", 2024-02-02T15:46:49.453 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.453 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.454 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.454 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "admin", 2024-02-02T15:46:49.454 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.454 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.454 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.454 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.454 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.454 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.454 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.455 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.455 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-02-02T15:46:49.455 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-02-02T15:46:49.455 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.455 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.455 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.455 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.456 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.456 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.456 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.456 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.456 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.456 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.456 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.456 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.457 INFO:teuthology.orchestra.run.smithi001.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-02-02T15:46:49.457 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-02-02T15:46:49.457 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.457 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.457 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.457 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.457 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.457 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.457 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.458 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.458 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.458 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.458 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.458 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.458 INFO:teuthology.orchestra.run.smithi001.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-02-02T15:46:49.458 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-02-02T15:46:49.458 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.459 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.459 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.459 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:49.459 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.459 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.459 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.459 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.460 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.460 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.460 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.460 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.460 INFO:teuthology.orchestra.run.smithi001.stdout: "ISSUE_TRACKER_API_KEY": { 2024-02-02T15:46:49.460 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-02-02T15:46:49.460 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.460 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.460 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.461 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.461 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.461 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.461 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.461 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.461 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.461 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.461 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.461 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.462 INFO:teuthology.orchestra.run.smithi001.stdout: "PROMETHEUS_API_HOST": { 2024-02-02T15:46:49.462 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PROMETHEUS_API_HOST", 2024-02-02T15:46:49.462 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.462 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.462 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.462 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.462 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.462 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.462 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.463 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.463 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.463 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.463 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.463 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.463 INFO:teuthology.orchestra.run.smithi001.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-02-02T15:46:49.463 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-02-02T15:46:49.464 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.464 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.464 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.464 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:49.464 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.464 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.464 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.464 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.465 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.465 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.465 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.465 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.465 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-02-02T15:46:49.465 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-02-02T15:46:49.465 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.465 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.465 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.466 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.466 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.466 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.466 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.466 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.466 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.466 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.466 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.466 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.467 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-02-02T15:46:49.467 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-02-02T15:46:49.467 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.467 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.467 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.467 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.467 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.467 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.468 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.468 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.468 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.468 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.468 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.468 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.468 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-02-02T15:46:49.468 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-02-02T15:46:49.469 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.469 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.469 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.469 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:49.469 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.469 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.469 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.469 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.469 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.470 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.470 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.470 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.470 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-02-02T15:46:49.470 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-02-02T15:46:49.470 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.470 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.470 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.471 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:49.471 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.471 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.471 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.471 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.471 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.471 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.471 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.472 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.472 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-02-02T15:46:49.472 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-02-02T15:46:49.472 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.472 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.472 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.472 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.472 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.473 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.473 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.473 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.473 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.473 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.473 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.473 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.473 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-02-02T15:46:49.474 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-02-02T15:46:49.474 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.474 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.474 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.474 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.474 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.474 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.474 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.475 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.475 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.475 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.475 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.475 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.475 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-02-02T15:46:49.475 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-02-02T15:46:49.475 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.476 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.476 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.476 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.476 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.476 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.476 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.476 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.476 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.477 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.477 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.477 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.477 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_ENABLED": { 2024-02-02T15:46:49.477 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_ENABLED", 2024-02-02T15:46:49.477 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.477 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.477 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.478 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:49.478 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.478 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.478 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.478 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.478 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.478 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.478 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.479 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.479 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-02-02T15:46:49.479 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-02-02T15:46:49.479 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.479 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.479 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.479 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-02-02T15:46:49.479 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.480 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.480 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.480 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.480 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.480 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.480 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.480 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.480 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-02-02T15:46:49.481 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-02-02T15:46:49.481 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:49.481 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.481 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.481 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-02-02T15:46:49.481 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.481 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.482 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.482 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.482 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.482 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.482 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.482 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.482 INFO:teuthology.orchestra.run.smithi001.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-02-02T15:46:49.482 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-02-02T15:46:49.483 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:49.483 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.483 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.483 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8", 2024-02-02T15:46:49.483 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.483 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.483 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.483 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.483 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.484 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.484 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.484 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.484 INFO:teuthology.orchestra.run.smithi001.stdout: "REST_REQUESTS_TIMEOUT": { 2024-02-02T15:46:49.484 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-02-02T15:46:49.484 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:49.484 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.484 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.484 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "45", 2024-02-02T15:46:49.485 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.485 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.485 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.485 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.485 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.485 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.485 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.485 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.486 INFO:teuthology.orchestra.run.smithi001.stdout: "RGW_API_ACCESS_KEY": { 2024-02-02T15:46:49.486 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RGW_API_ACCESS_KEY", 2024-02-02T15:46:49.486 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.486 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.486 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.486 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.486 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.486 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.487 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.487 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.487 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.487 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.487 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.487 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.487 INFO:teuthology.orchestra.run.smithi001.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-02-02T15:46:49.487 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-02-02T15:46:49.488 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.488 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.488 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.488 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "admin", 2024-02-02T15:46:49.488 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.488 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.488 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.489 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.489 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.489 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.489 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.489 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.489 INFO:teuthology.orchestra.run.smithi001.stdout: "RGW_API_SECRET_KEY": { 2024-02-02T15:46:49.489 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RGW_API_SECRET_KEY", 2024-02-02T15:46:49.489 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.489 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.490 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.490 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.490 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.490 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.490 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.490 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.490 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.490 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.491 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.491 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.491 INFO:teuthology.orchestra.run.smithi001.stdout: "RGW_API_SSL_VERIFY": { 2024-02-02T15:46:49.491 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "RGW_API_SSL_VERIFY", 2024-02-02T15:46:49.491 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.491 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.491 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.491 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:49.491 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.492 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.492 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.492 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.492 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.492 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.492 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.492 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.492 INFO:teuthology.orchestra.run.smithi001.stdout: "UNSAFE_TLS_v1_2": { 2024-02-02T15:46:49.493 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "UNSAFE_TLS_v1_2", 2024-02-02T15:46:49.493 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.493 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.493 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.493 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.493 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.493 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.493 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.493 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.494 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.494 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.494 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.494 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.494 INFO:teuthology.orchestra.run.smithi001.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-02-02T15:46:49.494 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-02-02T15:46:49.494 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:49.494 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.494 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.495 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-02-02T15:46:49.495 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.495 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.495 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.495 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.495 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.495 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.495 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.496 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.496 INFO:teuthology.orchestra.run.smithi001.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-02-02T15:46:49.496 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-02-02T15:46:49.496 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:49.496 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.496 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.496 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-02-02T15:46:49.496 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.497 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.497 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.497 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.497 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.497 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.497 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.497 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.497 INFO:teuthology.orchestra.run.smithi001.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-02-02T15:46:49.498 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-02-02T15:46:49.498 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:49.498 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.498 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.498 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2024-02-02T15:46:49.498 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.498 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.498 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.499 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.499 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.499 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.499 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.499 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.499 INFO:teuthology.orchestra.run.smithi001.stdout: "cross_origin_url": { 2024-02-02T15:46:49.499 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cross_origin_url", 2024-02-02T15:46:49.499 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.500 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.500 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.500 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.500 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.500 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.500 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.500 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.501 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.501 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.501 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.501 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.501 INFO:teuthology.orchestra.run.smithi001.stdout: "crt_file": { 2024-02-02T15:46:49.501 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "crt_file", 2024-02-02T15:46:49.501 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.501 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.501 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.501 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.502 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.502 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.502 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.502 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.502 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.502 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.502 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.503 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.503 INFO:teuthology.orchestra.run.smithi001.stdout: "debug": { 2024-02-02T15:46:49.503 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "debug", 2024-02-02T15:46:49.503 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.503 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.503 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.503 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.503 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.504 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.504 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.504 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Enable/disable debug options", 2024-02-02T15:46:49.504 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.504 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.504 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.504 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.504 INFO:teuthology.orchestra.run.smithi001.stdout: "jwt_token_ttl": { 2024-02-02T15:46:49.505 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "jwt_token_ttl", 2024-02-02T15:46:49.505 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:49.505 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.505 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.505 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "28800", 2024-02-02T15:46:49.505 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.505 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.505 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.506 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.506 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.506 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.506 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.506 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.506 INFO:teuthology.orchestra.run.smithi001.stdout: "key_file": { 2024-02-02T15:46:49.506 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "key_file", 2024-02-02T15:46:49.507 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.507 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.507 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.507 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.507 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.507 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.507 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.508 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.508 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.508 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.508 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.508 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.508 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:49.508 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:49.509 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.509 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.509 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.509 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.509 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.509 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.509 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.509 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.509 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.510 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.510 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.510 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.510 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.510 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.510 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.510 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.510 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.511 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.511 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.511 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:49.511 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:49.511 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.511 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.511 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.511 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.512 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.512 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.512 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.512 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.512 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.512 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.512 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.512 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.512 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:49.513 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:49.513 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.513 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.513 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.513 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:49.513 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.513 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.513 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.514 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.514 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.514 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.514 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.514 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.514 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.514 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.514 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.514 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.515 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.515 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.515 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.515 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:49.515 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:49.515 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.515 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.516 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.516 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.516 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.516 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.516 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.516 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.516 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.516 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.516 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.517 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.517 INFO:teuthology.orchestra.run.smithi001.stdout: "motd": { 2024-02-02T15:46:49.517 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "motd", 2024-02-02T15:46:49.517 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.517 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.517 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.517 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.517 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.517 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.518 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.518 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "The message of the day", 2024-02-02T15:46:49.518 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.518 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.518 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.518 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.518 INFO:teuthology.orchestra.run.smithi001.stdout: "redirect_resolve_ip_addr": { 2024-02-02T15:46:49.518 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "redirect_resolve_ip_addr", 2024-02-02T15:46:49.518 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.519 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.519 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.519 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.519 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.519 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.519 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.519 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.519 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.519 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.519 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.520 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.520 INFO:teuthology.orchestra.run.smithi001.stdout: "server_addr": { 2024-02-02T15:46:49.520 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_addr", 2024-02-02T15:46:49.520 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.520 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.520 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.520 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "::", 2024-02-02T15:46:49.520 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.520 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.521 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.521 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.521 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.521 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.521 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.521 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.521 INFO:teuthology.orchestra.run.smithi001.stdout: "server_port": { 2024-02-02T15:46:49.521 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_port", 2024-02-02T15:46:49.521 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:49.521 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.522 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.522 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8080", 2024-02-02T15:46:49.522 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.522 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.522 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.522 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.522 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.522 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.522 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.523 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.523 INFO:teuthology.orchestra.run.smithi001.stdout: "ssl": { 2024-02-02T15:46:49.523 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ssl", 2024-02-02T15:46:49.523 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.523 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.523 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.523 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:49.523 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.524 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.524 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.524 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.524 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.524 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.524 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.524 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.524 INFO:teuthology.orchestra.run.smithi001.stdout: "ssl_server_port": { 2024-02-02T15:46:49.525 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ssl_server_port", 2024-02-02T15:46:49.525 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:49.525 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.525 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.525 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8443", 2024-02-02T15:46:49.525 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.525 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.525 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.525 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.526 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.526 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.526 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.526 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.526 INFO:teuthology.orchestra.run.smithi001.stdout: "standby_behaviour": { 2024-02-02T15:46:49.526 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "standby_behaviour", 2024-02-02T15:46:49.526 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.526 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.526 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.527 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "redirect", 2024-02-02T15:46:49.527 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.527 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.527 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.527 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.527 INFO:teuthology.orchestra.run.smithi001.stdout: "redirect" 2024-02-02T15:46:49.527 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.527 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.527 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.528 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.528 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.528 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.528 INFO:teuthology.orchestra.run.smithi001.stdout: "standby_error_status_code": { 2024-02-02T15:46:49.528 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "standby_error_status_code", 2024-02-02T15:46:49.528 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:49.528 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.528 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.529 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "500", 2024-02-02T15:46:49.529 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "400", 2024-02-02T15:46:49.529 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "599", 2024-02-02T15:46:49.529 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.529 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.529 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.529 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.529 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.529 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.530 INFO:teuthology.orchestra.run.smithi001.stdout: "url_prefix": { 2024-02-02T15:46:49.530 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "url_prefix", 2024-02-02T15:46:49.530 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.530 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.530 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.530 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.530 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.530 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.530 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.531 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.531 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.531 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.531 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.531 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.531 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.531 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.531 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:49.531 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "devicehealth", 2024-02-02T15:46:49.532 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:49.532 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:49.532 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:49.532 INFO:teuthology.orchestra.run.smithi001.stdout: "enable_monitoring": { 2024-02-02T15:46:49.532 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "enable_monitoring", 2024-02-02T15:46:49.532 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.532 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.532 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.532 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:49.533 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.533 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.533 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.533 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "monitor device health metrics", 2024-02-02T15:46:49.533 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.533 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.533 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.533 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.533 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:49.533 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:49.534 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.534 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.534 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.534 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.534 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.534 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.534 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.534 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.534 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.535 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.535 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.535 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.535 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.535 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.535 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.535 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.535 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.535 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.536 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.536 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:49.536 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:49.536 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.536 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.536 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.536 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.536 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.536 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.537 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.537 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.537 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.537 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.537 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.537 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.537 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:49.537 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:49.537 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.538 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.538 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.538 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:49.538 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.538 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.538 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.538 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.538 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.538 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.538 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.539 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.539 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.539 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.539 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.539 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.539 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.539 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.539 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.539 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:49.540 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:49.540 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.540 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.540 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.540 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.540 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.540 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.540 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.541 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.541 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.541 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.541 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.541 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.541 INFO:teuthology.orchestra.run.smithi001.stdout: "mark_out_threshold": { 2024-02-02T15:46:49.541 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mark_out_threshold", 2024-02-02T15:46:49.541 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-02-02T15:46:49.542 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.542 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.542 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "2419200", 2024-02-02T15:46:49.542 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.542 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.542 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.542 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-02-02T15:46:49.542 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.542 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.543 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.543 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.543 INFO:teuthology.orchestra.run.smithi001.stdout: "pool_name": { 2024-02-02T15:46:49.543 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "pool_name", 2024-02-02T15:46:49.543 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.543 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.543 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.543 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "device_health_metrics", 2024-02-02T15:46:49.544 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.544 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.544 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.544 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "name of pool in which to store device health metrics", 2024-02-02T15:46:49.544 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.544 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.544 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.544 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.544 INFO:teuthology.orchestra.run.smithi001.stdout: "retention_period": { 2024-02-02T15:46:49.545 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "retention_period", 2024-02-02T15:46:49.545 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-02-02T15:46:49.545 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.545 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.545 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "15552000", 2024-02-02T15:46:49.545 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.545 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.545 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.546 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how long to retain device health metrics", 2024-02-02T15:46:49.546 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.546 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.546 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.546 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.546 INFO:teuthology.orchestra.run.smithi001.stdout: "scrape_frequency": { 2024-02-02T15:46:49.546 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "scrape_frequency", 2024-02-02T15:46:49.546 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-02-02T15:46:49.546 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.547 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.547 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "86400", 2024-02-02T15:46:49.547 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.547 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.547 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.547 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to scrape device health metrics", 2024-02-02T15:46:49.547 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.547 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.548 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.548 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.548 INFO:teuthology.orchestra.run.smithi001.stdout: "self_heal": { 2024-02-02T15:46:49.548 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "self_heal", 2024-02-02T15:46:49.548 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.548 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.548 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.548 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:49.549 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.549 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.549 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.549 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-02-02T15:46:49.549 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.549 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.549 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.549 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.550 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-02-02T15:46:49.550 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-02-02T15:46:49.550 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-02-02T15:46:49.550 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.550 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.550 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-02-02T15:46:49.550 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.550 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.551 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.551 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how frequently to wake up and check device health", 2024-02-02T15:46:49.551 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.551 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.551 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.551 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.551 INFO:teuthology.orchestra.run.smithi001.stdout: "warn_threshold": { 2024-02-02T15:46:49.551 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "warn_threshold", 2024-02-02T15:46:49.551 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-02-02T15:46:49.552 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.552 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.552 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "7257600", 2024-02-02T15:46:49.552 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.552 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.552 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.552 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-02-02T15:46:49.552 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.553 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.553 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.553 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.553 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.553 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.553 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:49.553 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "diskprediction_local", 2024-02-02T15:46:49.553 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:49.553 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:49.554 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:49.554 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:49.554 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:49.554 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.554 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.554 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.554 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.554 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.555 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.555 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.555 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.555 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.555 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.555 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.555 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.555 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.556 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.556 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.556 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.556 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.556 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.556 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.556 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:49.556 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:49.556 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.557 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.557 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.557 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.557 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.557 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.557 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.557 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.557 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.558 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.558 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.558 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.558 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:49.558 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:49.558 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.558 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.558 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.559 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:49.559 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.559 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.559 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.559 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.559 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.559 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.559 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.559 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.560 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.560 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.560 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.560 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.560 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.560 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.560 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.560 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:49.561 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:49.561 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.561 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.561 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.561 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.561 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.561 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.561 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.561 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.562 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.562 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.562 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.562 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.562 INFO:teuthology.orchestra.run.smithi001.stdout: "predict_interval": { 2024-02-02T15:46:49.562 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "predict_interval", 2024-02-02T15:46:49.562 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.562 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.563 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.563 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "86400", 2024-02-02T15:46:49.563 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.563 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.563 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.563 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.563 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.563 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.563 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.563 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.564 INFO:teuthology.orchestra.run.smithi001.stdout: "predictor_model": { 2024-02-02T15:46:49.564 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "predictor_model", 2024-02-02T15:46:49.564 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.564 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.564 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.564 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "prophetstor", 2024-02-02T15:46:49.564 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.564 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.565 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.565 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.565 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.565 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.565 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.565 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.565 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-02-02T15:46:49.566 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-02-02T15:46:49.566 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.566 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.566 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.566 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "600", 2024-02-02T15:46:49.566 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.566 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.566 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.567 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.567 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.567 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.567 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.567 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.567 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.567 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.567 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:49.568 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "influx", 2024-02-02T15:46:49.568 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": false, 2024-02-02T15:46:49.568 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "influxdb python module not found", 2024-02-02T15:46:49.568 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:49.568 INFO:teuthology.orchestra.run.smithi001.stdout: "batch_size": { 2024-02-02T15:46:49.568 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "batch_size", 2024-02-02T15:46:49.568 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:49.568 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.569 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.569 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5000", 2024-02-02T15:46:49.569 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.569 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.569 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.569 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-02-02T15:46:49.569 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.569 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.570 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.570 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.570 INFO:teuthology.orchestra.run.smithi001.stdout: "database": { 2024-02-02T15:46:49.570 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "database", 2024-02-02T15:46:49.570 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.570 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.570 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.570 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "ceph", 2024-02-02T15:46:49.571 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.571 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.571 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.571 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-02-02T15:46:49.571 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.571 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.571 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.571 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.572 INFO:teuthology.orchestra.run.smithi001.stdout: "hostname": { 2024-02-02T15:46:49.572 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "hostname", 2024-02-02T15:46:49.572 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.572 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.572 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.572 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.572 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.572 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.573 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.573 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "InfluxDB server hostname", 2024-02-02T15:46:49.573 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.573 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.573 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.573 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.573 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-02-02T15:46:49.573 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-02-02T15:46:49.573 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-02-02T15:46:49.574 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.574 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.574 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "30", 2024-02-02T15:46:49.574 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "5", 2024-02-02T15:46:49.574 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.574 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.574 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-02-02T15:46:49.574 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.575 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.575 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.575 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.575 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:49.575 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:49.575 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.575 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.575 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.576 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.576 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.576 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.576 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.576 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.576 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.576 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.576 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.577 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.577 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.577 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.577 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.577 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.577 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.577 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.577 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.577 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:49.578 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:49.578 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.578 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.578 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.578 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.578 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.578 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.578 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.579 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.579 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.579 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.579 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.579 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.579 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:49.579 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:49.579 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.579 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.580 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.580 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:49.580 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.580 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.580 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.580 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.580 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.580 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.581 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.581 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.581 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.581 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.581 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.581 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.581 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.582 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.582 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.582 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:49.582 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:49.582 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.582 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.582 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.582 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.583 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.583 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.583 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.583 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.583 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.583 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.583 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.583 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.584 INFO:teuthology.orchestra.run.smithi001.stdout: "password": { 2024-02-02T15:46:49.584 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "password", 2024-02-02T15:46:49.584 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.584 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.584 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.584 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.584 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.584 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.585 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.585 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "password of InfluxDB server user", 2024-02-02T15:46:49.585 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.585 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.585 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.585 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.585 INFO:teuthology.orchestra.run.smithi001.stdout: "port": { 2024-02-02T15:46:49.585 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "port", 2024-02-02T15:46:49.586 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:49.586 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.586 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.586 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "8086", 2024-02-02T15:46:49.586 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.586 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.586 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.587 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "InfluxDB server port", 2024-02-02T15:46:49.587 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.587 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.587 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.587 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.587 INFO:teuthology.orchestra.run.smithi001.stdout: "ssl": { 2024-02-02T15:46:49.587 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ssl", 2024-02-02T15:46:49.587 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.587 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.588 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.588 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "false", 2024-02-02T15:46:49.588 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.588 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.588 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.588 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-02-02T15:46:49.588 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.588 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.589 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.589 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.589 INFO:teuthology.orchestra.run.smithi001.stdout: "threads": { 2024-02-02T15:46:49.589 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "threads", 2024-02-02T15:46:49.589 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:49.589 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.589 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.589 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2024-02-02T15:46:49.589 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1", 2024-02-02T15:46:49.590 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "32", 2024-02-02T15:46:49.590 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.590 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-02-02T15:46:49.590 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.590 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.590 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.590 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.590 INFO:teuthology.orchestra.run.smithi001.stdout: "username": { 2024-02-02T15:46:49.590 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "username", 2024-02-02T15:46:49.591 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.591 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.591 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.591 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.591 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.591 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.591 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.591 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "username of InfluxDB server user", 2024-02-02T15:46:49.592 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.592 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.592 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.592 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.592 INFO:teuthology.orchestra.run.smithi001.stdout: "verify_ssl": { 2024-02-02T15:46:49.592 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "verify_ssl", 2024-02-02T15:46:49.592 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.592 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.592 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.593 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "true", 2024-02-02T15:46:49.593 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.593 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.593 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.593 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-02-02T15:46:49.593 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.593 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.593 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.594 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.594 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.594 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.594 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:49.594 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "insights", 2024-02-02T15:46:49.594 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:49.594 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:49.594 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:49.594 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:49.595 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:49.595 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.595 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.595 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.595 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.595 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.595 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.595 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.595 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.596 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.596 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.596 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.596 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.596 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.596 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.596 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.597 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.597 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.597 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.597 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.597 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:49.597 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:49.597 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.597 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.597 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.597 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.598 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.598 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.598 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.598 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.598 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.598 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.598 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.598 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.598 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:49.599 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:49.599 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.599 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.599 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.599 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:49.599 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.599 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.599 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.599 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.599 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.600 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.600 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.600 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.600 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.600 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.600 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.600 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.600 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.601 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.601 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.601 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:49.601 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:49.601 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.601 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.601 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.601 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.601 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.602 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.602 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.602 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.602 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.602 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.602 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.602 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.602 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.603 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.603 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:49.603 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "iostat", 2024-02-02T15:46:49.603 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:49.603 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:49.603 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:49.603 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:49.603 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:49.604 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.604 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.604 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.604 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.604 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.604 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.604 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.605 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.605 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.605 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.605 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.605 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.605 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.605 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.605 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.606 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.606 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.606 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.606 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.606 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:49.606 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:49.606 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.606 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.606 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.607 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.607 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.607 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.607 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.607 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.607 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.607 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.607 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.607 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.608 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:49.608 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:49.608 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.608 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.608 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.608 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:49.608 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.609 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.609 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.609 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.609 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.609 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.609 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.609 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.609 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.609 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.609 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.610 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.610 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.610 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.610 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.610 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:49.610 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:49.610 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.610 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.610 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.610 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.611 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.611 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.611 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.611 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.611 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.611 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.611 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.611 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.612 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.612 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.612 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:49.612 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "k8sevents", 2024-02-02T15:46:49.612 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:49.612 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:49.612 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:49.612 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_event_retention_days": { 2024-02-02T15:46:49.612 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "ceph_event_retention_days", 2024-02-02T15:46:49.613 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:49.613 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.613 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.613 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "7", 2024-02-02T15:46:49.613 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.613 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.613 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.613 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Days to hold ceph event information within local cache", 2024-02-02T15:46:49.613 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.614 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.614 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.614 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.614 INFO:teuthology.orchestra.run.smithi001.stdout: "config_check_secs": { 2024-02-02T15:46:49.614 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "config_check_secs", 2024-02-02T15:46:49.614 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:49.614 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.614 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.614 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-02-02T15:46:49.614 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "10", 2024-02-02T15:46:49.615 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.615 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.615 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-02-02T15:46:49.615 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.615 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.615 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.615 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.615 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:49.615 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:49.615 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.616 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.616 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.616 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.616 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.616 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.616 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.616 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.616 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.616 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.617 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.617 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.617 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.617 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.617 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.617 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.617 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.617 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.617 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.617 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:49.618 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:49.618 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.618 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.618 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.618 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.618 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.618 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.618 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.618 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.618 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.619 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.619 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.619 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.619 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:49.619 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:49.619 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.619 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.619 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.619 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:49.619 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.620 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.620 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.620 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.620 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.620 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.620 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.620 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.620 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.620 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.620 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.621 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.621 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.621 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.621 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.621 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:49.621 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:49.621 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.621 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.622 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.622 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.622 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.622 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.622 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.622 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.622 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.622 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.623 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.623 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.623 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.623 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.623 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:49.623 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "localpool", 2024-02-02T15:46:49.623 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:49.623 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:49.624 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:49.624 INFO:teuthology.orchestra.run.smithi001.stdout: "failure_domain": { 2024-02-02T15:46:49.624 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "failure_domain", 2024-02-02T15:46:49.624 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.624 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.624 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.624 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "host", 2024-02-02T15:46:49.624 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.624 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.625 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.625 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "failure domain for any created local pool", 2024-02-02T15:46:49.625 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-02-02T15:46:49.625 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.625 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.625 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.627 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:49.627 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:49.627 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.627 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.627 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.627 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.627 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.628 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.628 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.628 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.628 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.628 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.628 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.628 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.628 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.629 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.629 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.629 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.629 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.629 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.629 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.629 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:49.629 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:49.630 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.630 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.630 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.630 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.630 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.630 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.630 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.630 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.630 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.631 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.631 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.631 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.631 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:49.631 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:49.631 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.631 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.631 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.631 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:49.631 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.632 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.632 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.632 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.632 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.632 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.632 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.632 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.632 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.632 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.633 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.633 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.633 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.633 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.633 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.633 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:49.633 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:49.633 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.633 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.634 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.634 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.634 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.634 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.634 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.634 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.634 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.634 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.635 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.635 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.635 INFO:teuthology.orchestra.run.smithi001.stdout: "min_size": { 2024-02-02T15:46:49.635 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "min_size", 2024-02-02T15:46:49.635 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:49.635 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.635 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.635 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.636 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.636 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.636 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.636 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "default min_size for any created local pool", 2024-02-02T15:46:49.636 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-02-02T15:46:49.636 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.636 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.636 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.637 INFO:teuthology.orchestra.run.smithi001.stdout: "num_rep": { 2024-02-02T15:46:49.637 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "num_rep", 2024-02-02T15:46:49.637 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:49.637 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.637 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.637 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "3", 2024-02-02T15:46:49.637 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.638 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.638 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.638 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "default replica count for any created local pool", 2024-02-02T15:46:49.638 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.638 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.638 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.638 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.638 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_num": { 2024-02-02T15:46:49.638 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "pg_num", 2024-02-02T15:46:49.639 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:49.639 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.639 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.639 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "128", 2024-02-02T15:46:49.639 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.639 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.639 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.639 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "default pg_num for any created local pool", 2024-02-02T15:46:49.639 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.640 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.640 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.640 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.640 INFO:teuthology.orchestra.run.smithi001.stdout: "prefix": { 2024-02-02T15:46:49.640 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "prefix", 2024-02-02T15:46:49.640 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.640 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.640 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.640 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.641 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.641 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.641 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.641 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "name prefix for any created local pool", 2024-02-02T15:46:49.641 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.641 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.641 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.641 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.641 INFO:teuthology.orchestra.run.smithi001.stdout: "subtree": { 2024-02-02T15:46:49.642 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "subtree", 2024-02-02T15:46:49.642 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.642 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.642 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.642 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "rack", 2024-02-02T15:46:49.642 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.642 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.642 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.642 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "CRUSH level for which to create a local pool", 2024-02-02T15:46:49.642 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-02-02T15:46:49.643 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.643 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.643 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.643 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.643 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.643 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:49.643 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mds_autoscaler", 2024-02-02T15:46:49.643 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:49.644 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:49.644 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:49.644 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:49.644 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:49.644 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.644 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.644 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.644 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.645 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.645 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.645 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.645 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.645 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.645 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.645 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.646 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.646 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.646 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.646 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.646 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.646 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.646 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.646 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.647 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:49.647 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:49.647 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.647 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.647 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.647 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.647 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.647 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.647 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.648 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.648 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.648 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.648 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.648 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.648 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:49.648 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:49.648 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.648 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.649 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.649 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:49.649 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.649 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.649 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.649 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.649 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.649 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.650 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.650 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.650 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.650 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.650 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.650 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.650 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.650 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.651 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.651 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:49.651 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:49.651 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.651 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.651 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.651 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.652 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.652 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.652 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.652 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.652 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.652 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.652 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.652 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.653 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.653 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.653 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:49.653 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mirroring", 2024-02-02T15:46:49.653 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:49.653 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:49.653 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:49.653 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:49.654 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:49.654 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.654 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.654 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.654 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.654 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.654 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.654 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.655 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.655 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.655 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.655 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.655 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.655 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.655 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.655 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.655 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.656 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.656 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.656 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.656 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:49.656 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:49.656 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.656 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.656 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.656 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.657 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.657 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.657 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.657 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.657 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.657 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.657 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.657 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.658 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:49.658 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:49.658 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.658 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.658 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.658 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:49.658 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.658 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.658 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.659 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.659 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.659 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.659 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.659 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.659 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.659 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.660 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.660 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.660 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.660 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.660 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.660 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:49.660 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:49.661 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.661 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.661 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.661 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.661 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.661 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.661 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.661 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.662 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.662 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.662 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.662 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.662 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.662 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.662 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:49.662 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "nfs", 2024-02-02T15:46:49.663 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:49.663 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:49.663 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:49.663 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:49.663 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:49.663 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.663 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.664 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.664 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.664 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.664 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.664 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.664 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.664 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.664 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.664 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.665 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.665 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.665 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.665 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.665 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.665 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.665 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.666 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.666 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:49.666 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:49.666 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.666 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.666 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.666 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.666 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.667 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.667 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.667 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.667 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.667 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.667 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.667 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.667 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:49.668 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:49.668 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.668 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.668 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.668 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:49.668 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.668 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.668 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.668 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.669 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.669 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.669 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.669 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.669 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.669 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.669 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.669 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.670 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.670 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.670 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.670 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:49.670 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:49.670 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.670 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.670 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.670 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.671 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.671 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.671 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.671 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.671 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.671 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.671 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.671 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.672 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.672 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.672 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:49.672 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "orchestrator", 2024-02-02T15:46:49.672 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:49.672 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:49.672 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:49.672 INFO:teuthology.orchestra.run.smithi001.stdout: "fail_fs": { 2024-02-02T15:46:49.672 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "fail_fs", 2024-02-02T15:46:49.673 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.673 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.673 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.673 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.673 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.673 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.673 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.673 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-02-02T15:46:49.673 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.674 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.674 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.674 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.674 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:49.674 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:49.674 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.674 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.674 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.675 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.675 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.675 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.675 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.675 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.675 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.675 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.675 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.675 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.676 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.676 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.676 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.676 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.676 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.676 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.676 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.676 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:49.677 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:49.677 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.677 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.677 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.677 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.677 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.677 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.677 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.677 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.678 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.678 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.678 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.678 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.678 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:49.678 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:49.678 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.679 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.679 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.679 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:49.679 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.679 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.679 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.679 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.679 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.679 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.680 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.680 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.680 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.680 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.680 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.680 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.680 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.681 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.681 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.681 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:49.681 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:49.681 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.681 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.681 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.681 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.682 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.682 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.682 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.682 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.682 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.682 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.682 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.682 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.683 INFO:teuthology.orchestra.run.smithi001.stdout: "orchestrator": { 2024-02-02T15:46:49.683 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "orchestrator", 2024-02-02T15:46:49.683 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.683 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.683 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.683 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.683 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.683 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.684 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.684 INFO:teuthology.orchestra.run.smithi001.stdout: "cephadm", 2024-02-02T15:46:49.684 INFO:teuthology.orchestra.run.smithi001.stdout: "rook", 2024-02-02T15:46:49.684 INFO:teuthology.orchestra.run.smithi001.stdout: "test_orchestrator" 2024-02-02T15:46:49.684 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.684 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Orchestrator backend", 2024-02-02T15:46:49.684 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.684 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.684 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.685 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.685 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.685 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.685 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:49.685 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "osd_perf_query", 2024-02-02T15:46:49.685 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:49.685 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:49.685 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:49.686 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:49.686 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:49.686 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.686 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.686 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.686 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.686 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.686 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.687 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.687 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.687 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.687 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.687 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.687 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.687 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.687 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.688 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.688 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.688 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.688 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.688 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.688 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:49.688 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:49.688 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.689 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.689 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.689 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.689 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.689 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.689 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.689 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.689 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.690 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.690 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.690 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.690 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:49.690 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:49.690 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.690 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.690 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.691 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:49.691 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.691 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.691 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.691 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.691 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.691 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.691 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.692 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.692 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.692 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.692 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.692 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.692 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.692 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.692 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.693 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:49.693 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:49.693 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.693 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.693 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.693 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.693 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.693 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.693 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.694 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.694 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.694 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.694 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.694 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.694 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.694 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.695 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:49.695 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "osd_support", 2024-02-02T15:46:49.695 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:49.695 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:49.695 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:49.695 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:49.695 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:49.695 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.695 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.696 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.696 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.696 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.696 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.696 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.696 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.696 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.697 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.697 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.697 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.697 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.697 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.697 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.697 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.697 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.698 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.698 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.698 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:49.698 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:49.698 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.698 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.698 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.699 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.699 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.699 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.699 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.699 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.699 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.699 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.700 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.700 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.700 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:49.700 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:49.700 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.700 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.700 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.701 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:49.701 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.701 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.701 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.701 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.701 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.701 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.701 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.701 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.702 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.702 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.702 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.702 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.702 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.702 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.702 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.702 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:49.702 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:49.702 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.703 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.703 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.703 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.703 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.703 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.703 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.703 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.703 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.703 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.704 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.704 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.704 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.704 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.704 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:49.704 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "pg_autoscaler", 2024-02-02T15:46:49.704 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:49.704 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:49.705 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:49.705 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:49.705 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:49.705 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.705 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.705 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.705 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.705 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.705 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.706 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.706 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.706 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.706 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.706 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.706 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.706 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.706 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.706 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.707 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.707 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.707 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.707 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.707 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:49.707 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:49.707 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.707 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.708 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.708 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.708 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.708 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.708 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.708 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.708 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.708 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.709 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.709 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.709 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:49.709 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:49.709 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.709 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.709 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.709 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:49.709 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.710 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.710 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.710 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.710 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.710 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.710 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.710 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.710 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.711 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.711 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.711 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.711 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.711 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.711 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.711 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:49.711 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:49.712 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.712 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.712 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.712 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.712 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.712 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.712 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.713 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.713 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.713 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.713 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.713 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.713 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-02-02T15:46:49.713 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-02-02T15:46:49.713 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-02-02T15:46:49.713 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.714 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.714 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-02-02T15:46:49.714 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.714 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.714 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.714 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.714 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.714 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.714 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.715 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.715 INFO:teuthology.orchestra.run.smithi001.stdout: "threshold": { 2024-02-02T15:46:49.715 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "threshold", 2024-02-02T15:46:49.715 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-02-02T15:46:49.715 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.715 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.715 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "3.0", 2024-02-02T15:46:49.715 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1.0", 2024-02-02T15:46:49.716 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.716 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.716 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "scaling threshold", 2024-02-02T15:46:49.716 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-02-02T15:46:49.716 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.716 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.716 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.716 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.716 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.717 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:49.717 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "progress", 2024-02-02T15:46:49.717 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:49.717 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:49.717 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:49.717 INFO:teuthology.orchestra.run.smithi001.stdout: "allow_pg_recovery_event": { 2024-02-02T15:46:49.717 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "allow_pg_recovery_event", 2024-02-02T15:46:49.717 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.717 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.718 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.718 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.718 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.718 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.718 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.718 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "allow the module to show pg recovery progress", 2024-02-02T15:46:49.718 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.718 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.719 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.719 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.719 INFO:teuthology.orchestra.run.smithi001.stdout: "enabled": { 2024-02-02T15:46:49.719 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "enabled", 2024-02-02T15:46:49.719 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.719 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.719 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.719 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:49.720 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.720 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.720 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.720 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.720 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.720 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.720 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.720 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.721 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:49.721 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:49.721 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.721 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.721 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.721 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.721 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.721 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.721 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.722 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.722 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.722 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.722 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.722 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.722 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.722 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.723 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.723 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.723 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.723 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.723 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.723 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:49.723 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:49.723 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.724 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.724 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.724 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.724 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.724 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.724 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.724 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.724 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.725 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.725 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.725 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.725 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:49.725 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:49.725 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.725 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.725 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.726 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:49.726 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.726 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.726 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.726 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.726 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.726 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.726 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.726 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.727 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.727 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.727 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.727 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.727 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.727 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.727 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.727 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:49.728 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:49.728 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.728 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.728 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.728 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.728 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.728 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.728 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.728 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.729 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.729 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.729 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.729 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.729 INFO:teuthology.orchestra.run.smithi001.stdout: "max_completed_events": { 2024-02-02T15:46:49.729 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_completed_events", 2024-02-02T15:46:49.729 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:49.729 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.729 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.729 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "50", 2024-02-02T15:46:49.730 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.730 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.730 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.730 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "number of past completed events to remember", 2024-02-02T15:46:49.730 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.730 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.730 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.731 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.731 INFO:teuthology.orchestra.run.smithi001.stdout: "sleep_interval": { 2024-02-02T15:46:49.731 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "sleep_interval", 2024-02-02T15:46:49.731 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-02-02T15:46:49.731 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.731 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.731 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "5", 2024-02-02T15:46:49.731 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.732 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.732 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.732 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "how long the module is going to sleep", 2024-02-02T15:46:49.732 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.732 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.732 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.732 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.732 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.732 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.733 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:49.733 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "prometheus", 2024-02-02T15:46:49.733 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:49.733 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:49.733 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:49.733 INFO:teuthology.orchestra.run.smithi001.stdout: "cache": { 2024-02-02T15:46:49.733 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "cache", 2024-02-02T15:46:49.733 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.734 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.734 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.734 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:49.734 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.734 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.734 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.734 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.734 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.734 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.735 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.735 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.735 INFO:teuthology.orchestra.run.smithi001.stdout: "exclude_perf_counters": { 2024-02-02T15:46:49.735 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "exclude_perf_counters", 2024-02-02T15:46:49.735 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.735 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.735 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.736 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:49.736 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.736 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.736 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.736 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-02-02T15:46:49.736 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-02-02T15:46:49.736 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.737 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.737 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.737 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:49.737 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:49.737 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.737 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.737 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.737 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.738 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.738 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.738 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.738 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.738 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.738 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.738 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.738 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.738 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.739 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.739 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.739 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.739 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.739 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.739 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.739 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:49.739 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:49.740 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.740 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.740 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.740 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.740 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.740 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.740 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.741 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.741 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.741 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.741 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.741 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.741 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:49.741 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:49.741 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.741 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.742 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.742 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:49.742 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.742 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.742 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.742 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.742 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.742 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.743 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.743 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.743 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.743 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.743 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.743 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.743 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.743 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.743 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.743 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:49.744 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:49.744 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.744 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.744 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.744 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.744 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.744 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.744 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.745 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.745 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.745 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.745 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.745 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.745 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_stats_pools": { 2024-02-02T15:46:49.745 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rbd_stats_pools", 2024-02-02T15:46:49.745 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.746 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.746 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.746 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.746 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.746 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.746 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.746 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.746 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.747 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.747 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.747 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.747 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_stats_pools_refresh_interval": { 2024-02-02T15:46:49.747 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-02-02T15:46:49.747 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:49.747 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.747 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.747 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "300", 2024-02-02T15:46:49.748 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.748 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.748 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.748 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.748 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.748 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.748 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.748 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.749 INFO:teuthology.orchestra.run.smithi001.stdout: "scrape_interval": { 2024-02-02T15:46:49.749 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "scrape_interval", 2024-02-02T15:46:49.749 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-02-02T15:46:49.749 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.749 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.749 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "15.0", 2024-02-02T15:46:49.749 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.749 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.750 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.750 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.750 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.750 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.750 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.750 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.750 INFO:teuthology.orchestra.run.smithi001.stdout: "server_addr": { 2024-02-02T15:46:49.750 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_addr", 2024-02-02T15:46:49.750 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.751 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.751 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.751 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "::", 2024-02-02T15:46:49.751 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.751 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.751 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.751 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-02-02T15:46:49.751 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.752 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.752 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.752 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.752 INFO:teuthology.orchestra.run.smithi001.stdout: "server_port": { 2024-02-02T15:46:49.752 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_port", 2024-02-02T15:46:49.752 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:49.752 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.752 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.753 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "9283", 2024-02-02T15:46:49.753 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.753 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.753 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.753 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-02-02T15:46:49.753 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.753 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.753 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.753 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.754 INFO:teuthology.orchestra.run.smithi001.stdout: "stale_cache_strategy": { 2024-02-02T15:46:49.754 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "stale_cache_strategy", 2024-02-02T15:46:49.754 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.754 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.754 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.754 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "log", 2024-02-02T15:46:49.754 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.754 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.755 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.755 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.755 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.755 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.755 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.755 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.755 INFO:teuthology.orchestra.run.smithi001.stdout: "standby_behaviour": { 2024-02-02T15:46:49.755 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "standby_behaviour", 2024-02-02T15:46:49.756 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.756 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.756 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.756 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "default", 2024-02-02T15:46:49.756 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.756 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.756 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.756 INFO:teuthology.orchestra.run.smithi001.stdout: "default", 2024-02-02T15:46:49.757 INFO:teuthology.orchestra.run.smithi001.stdout: "error" 2024-02-02T15:46:49.757 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.757 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.757 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.757 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.757 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.757 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.757 INFO:teuthology.orchestra.run.smithi001.stdout: "standby_error_status_code": { 2024-02-02T15:46:49.758 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "standby_error_status_code", 2024-02-02T15:46:49.758 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:49.758 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.758 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.758 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "500", 2024-02-02T15:46:49.758 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "400", 2024-02-02T15:46:49.758 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "599", 2024-02-02T15:46:49.759 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.759 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.759 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.759 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.759 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.759 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.759 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.759 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.760 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:49.760 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rbd_support", 2024-02-02T15:46:49.760 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:49.760 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:49.760 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:49.760 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:49.760 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:49.760 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.761 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.761 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.761 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.761 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.761 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.761 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.761 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.761 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.761 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.762 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.762 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.762 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.762 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.762 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.762 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.762 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.762 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.763 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.763 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:49.763 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:49.763 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.763 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.763 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.763 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.763 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.763 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.763 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.764 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.764 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.764 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.764 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.764 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.764 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:49.764 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:49.764 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.764 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.765 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.765 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:49.765 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.765 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.765 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.765 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.765 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.765 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.766 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.766 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.766 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.766 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.766 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.766 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.766 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.766 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.766 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.767 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:49.767 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:49.767 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.767 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.767 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.767 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.767 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.767 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.767 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.768 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.768 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.768 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.768 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.768 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.768 INFO:teuthology.orchestra.run.smithi001.stdout: "max_concurrent_snap_create": { 2024-02-02T15:46:49.768 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_concurrent_snap_create", 2024-02-02T15:46:49.768 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:49.769 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.769 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.769 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10", 2024-02-02T15:46:49.769 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.769 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.769 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.769 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.769 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.770 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.770 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.770 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.770 INFO:teuthology.orchestra.run.smithi001.stdout: "mirror_snapshot_schedule": { 2024-02-02T15:46:49.770 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "mirror_snapshot_schedule", 2024-02-02T15:46:49.770 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.770 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.770 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.771 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.771 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.771 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.771 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.771 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.771 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.771 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.771 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.771 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.772 INFO:teuthology.orchestra.run.smithi001.stdout: "trash_purge_schedule": { 2024-02-02T15:46:49.772 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "trash_purge_schedule", 2024-02-02T15:46:49.772 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.772 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.772 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.772 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.772 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.772 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.773 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.773 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.773 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.773 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.773 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.773 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.773 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.773 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.773 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:49.774 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "restful", 2024-02-02T15:46:49.774 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:49.774 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:49.774 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:49.774 INFO:teuthology.orchestra.run.smithi001.stdout: "enable_auth": { 2024-02-02T15:46:49.774 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "enable_auth", 2024-02-02T15:46:49.774 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.775 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.775 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.775 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:49.775 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.775 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.775 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.775 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.775 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.775 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.776 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.776 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.776 INFO:teuthology.orchestra.run.smithi001.stdout: "key_file": { 2024-02-02T15:46:49.776 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "key_file", 2024-02-02T15:46:49.776 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.776 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.777 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.777 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.777 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.777 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.777 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.777 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.777 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.777 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.777 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.778 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.778 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:49.778 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:49.778 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.778 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.778 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.778 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.779 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.779 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.779 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.779 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.779 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.779 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.779 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.779 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.779 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.780 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.780 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.780 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.780 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.780 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.780 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.780 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:49.780 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:49.781 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.781 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.781 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.781 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.781 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.781 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.781 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.781 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.782 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.782 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.782 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.782 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.782 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:49.782 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:49.782 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.782 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.783 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.783 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:49.783 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.783 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.783 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.783 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.783 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.783 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.783 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.783 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.784 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.784 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.784 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.784 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.784 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.784 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.784 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.784 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:49.785 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:49.785 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.785 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.785 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.785 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.785 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.785 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.785 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.785 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.786 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.786 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.786 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.786 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.786 INFO:teuthology.orchestra.run.smithi001.stdout: "server_addr": { 2024-02-02T15:46:49.786 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_addr", 2024-02-02T15:46:49.786 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.786 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.786 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.787 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.787 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.787 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.787 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.787 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.787 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.787 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.787 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.788 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.789 INFO:teuthology.orchestra.run.smithi001.stdout: "server_port": { 2024-02-02T15:46:49.789 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "server_port", 2024-02-02T15:46:49.789 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.789 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.789 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.789 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.790 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.790 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.790 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.790 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.790 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.790 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.790 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.790 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.791 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.791 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.791 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:49.791 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rgw", 2024-02-02T15:46:49.791 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:49.791 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:49.791 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:49.791 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:49.792 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:49.792 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.792 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.792 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.792 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.792 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.792 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.792 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.793 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.793 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.793 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.793 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.793 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.793 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.793 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.793 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.793 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.794 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.794 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.794 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.794 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:49.794 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:49.794 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.794 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.795 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.795 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.795 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.795 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.795 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.795 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.795 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.795 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.796 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.796 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.796 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:49.796 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:49.796 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.796 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.796 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.796 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:49.797 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.797 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.797 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.797 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.797 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.797 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.797 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.797 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.797 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.798 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.798 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.798 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.798 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.798 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.798 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.798 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:49.798 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:49.798 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.799 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.799 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.799 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.799 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.799 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.799 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.799 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.799 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.799 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.800 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.800 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.800 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.800 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.800 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:49.800 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rook", 2024-02-02T15:46:49.800 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:49.800 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:49.800 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:49.801 INFO:teuthology.orchestra.run.smithi001.stdout: "drive_group_interval": { 2024-02-02T15:46:49.801 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "drive_group_interval", 2024-02-02T15:46:49.801 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-02-02T15:46:49.801 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.801 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.801 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "300.0", 2024-02-02T15:46:49.801 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.801 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.801 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.802 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2024-02-02T15:46:49.802 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.802 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.802 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.802 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.802 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:49.802 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:49.802 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.802 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.802 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.803 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.803 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.803 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.803 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.803 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.803 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.803 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.803 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.803 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.804 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.804 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.804 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.804 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.804 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.804 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.804 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.804 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:49.804 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:49.805 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.805 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.805 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.805 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.805 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.805 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.805 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.805 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.806 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.806 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.806 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.806 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.806 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:49.806 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:49.806 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.806 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.806 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.807 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:49.807 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.807 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.807 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.807 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.807 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.807 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.807 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.807 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.808 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.808 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.808 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.808 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.808 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.808 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.808 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.808 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:49.808 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:49.809 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.809 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.809 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.809 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.809 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.809 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.809 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.809 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.810 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.810 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.810 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.810 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.810 INFO:teuthology.orchestra.run.smithi001.stdout: "storage_class": { 2024-02-02T15:46:49.810 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "storage_class", 2024-02-02T15:46:49.810 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.810 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.810 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.811 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "local", 2024-02-02T15:46:49.811 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.811 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.811 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.811 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-02-02T15:46:49.811 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.811 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.811 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.811 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.812 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.812 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.812 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:49.812 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "selftest", 2024-02-02T15:46:49.812 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:49.812 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:49.812 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:49.812 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:49.813 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:49.813 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.813 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.813 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.813 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.813 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.813 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.813 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.814 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.814 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.814 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.814 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.814 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.814 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.814 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.814 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.814 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.815 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.815 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.815 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.815 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:49.815 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:49.815 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.815 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.815 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.816 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.816 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.816 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.816 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.816 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.816 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.816 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.816 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.817 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.817 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:49.817 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:49.817 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.817 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.817 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.817 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:49.817 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.817 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.818 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.818 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.818 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.818 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.818 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.818 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.818 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.818 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.819 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.819 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.819 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.819 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.819 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.819 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:49.819 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:49.819 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.819 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.820 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.820 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.820 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.820 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.820 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.820 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.820 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.820 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.821 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.821 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.821 INFO:teuthology.orchestra.run.smithi001.stdout: "roption1": { 2024-02-02T15:46:49.821 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "roption1", 2024-02-02T15:46:49.821 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.821 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.821 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.821 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.821 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.822 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.822 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.822 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.822 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.822 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.822 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.822 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.822 INFO:teuthology.orchestra.run.smithi001.stdout: "roption2": { 2024-02-02T15:46:49.823 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "roption2", 2024-02-02T15:46:49.823 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.823 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.823 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.823 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "xyz", 2024-02-02T15:46:49.823 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.823 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.823 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.824 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.824 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.824 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.824 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.824 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.824 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption1": { 2024-02-02T15:46:49.824 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption1", 2024-02-02T15:46:49.824 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.825 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.825 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.825 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.825 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.825 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.825 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.825 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.825 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.825 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.826 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.826 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.826 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption2": { 2024-02-02T15:46:49.826 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption2", 2024-02-02T15:46:49.826 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:49.826 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.826 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.826 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.827 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.827 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.827 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.827 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.827 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.827 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.827 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.827 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.828 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption3": { 2024-02-02T15:46:49.828 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption3", 2024-02-02T15:46:49.828 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "float", 2024-02-02T15:46:49.828 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.828 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.828 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.828 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.828 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.828 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.829 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.829 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.829 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.829 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.829 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.829 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption4": { 2024-02-02T15:46:49.829 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption4", 2024-02-02T15:46:49.829 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.829 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.830 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.830 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.830 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.830 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.830 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.830 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.830 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.830 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.830 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.831 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.831 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption5": { 2024-02-02T15:46:49.831 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption5", 2024-02-02T15:46:49.831 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.831 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.831 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.831 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.831 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.832 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.832 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.832 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.832 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.832 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.832 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.832 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.832 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption6": { 2024-02-02T15:46:49.833 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption6", 2024-02-02T15:46:49.833 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.833 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.833 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.833 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:49.833 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.833 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.833 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.834 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.834 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.834 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.834 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.834 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.834 INFO:teuthology.orchestra.run.smithi001.stdout: "rwoption7": { 2024-02-02T15:46:49.834 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rwoption7", 2024-02-02T15:46:49.835 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:49.835 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.835 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.835 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.835 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "1", 2024-02-02T15:46:49.835 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "42", 2024-02-02T15:46:49.835 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.835 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.836 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.836 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.836 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.836 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.836 INFO:teuthology.orchestra.run.smithi001.stdout: "testkey": { 2024-02-02T15:46:49.836 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "testkey", 2024-02-02T15:46:49.836 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.837 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.837 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.837 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.837 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.837 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.837 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.837 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.837 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.837 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.838 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.838 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.838 INFO:teuthology.orchestra.run.smithi001.stdout: "testlkey": { 2024-02-02T15:46:49.838 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "testlkey", 2024-02-02T15:46:49.838 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.838 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.838 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.838 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.838 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.839 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.839 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.839 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.839 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.839 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.839 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.839 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.839 INFO:teuthology.orchestra.run.smithi001.stdout: "testnewline": { 2024-02-02T15:46:49.839 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "testnewline", 2024-02-02T15:46:49.839 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.840 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.840 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.840 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.840 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.840 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.840 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.840 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.840 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.840 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.841 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.841 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.841 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.841 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.841 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:49.841 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "snap_schedule", 2024-02-02T15:46:49.841 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:49.841 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:49.841 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:49.842 INFO:teuthology.orchestra.run.smithi001.stdout: "allow_m_granularity": { 2024-02-02T15:46:49.842 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "allow_m_granularity", 2024-02-02T15:46:49.842 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.842 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.842 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.842 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.842 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.842 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.842 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.843 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "allow minute scheduled snapshots", 2024-02-02T15:46:49.843 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.843 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.843 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.843 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.843 INFO:teuthology.orchestra.run.smithi001.stdout: "dump_on_update": { 2024-02-02T15:46:49.843 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "dump_on_update", 2024-02-02T15:46:49.843 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.843 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.843 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.844 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.844 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.844 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.844 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.844 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "dump database to debug log on update", 2024-02-02T15:46:49.844 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.844 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.844 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.845 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.845 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:49.845 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:49.845 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.845 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.845 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.845 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.845 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.846 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.846 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.846 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.846 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.846 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.846 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.846 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.846 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.846 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.847 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.847 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.847 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.847 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.847 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.847 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:49.847 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:49.847 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.847 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.848 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.848 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.848 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.848 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.848 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.848 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.848 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.848 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.849 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.849 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.849 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:49.849 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:49.849 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.849 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.849 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.849 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:49.849 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.850 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.850 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.850 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.850 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.850 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.850 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.850 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.851 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.851 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.851 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.851 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.851 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.851 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.851 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.851 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:49.852 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:49.852 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.852 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.852 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.852 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.852 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.852 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.853 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.853 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.853 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.853 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.853 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.853 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.853 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.854 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.854 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:49.854 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "stats", 2024-02-02T15:46:49.854 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:49.854 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:49.854 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:49.854 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:49.854 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:49.855 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.855 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.855 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.855 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.855 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.855 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.855 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.855 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.856 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.856 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.856 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.856 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.856 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.856 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.856 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.856 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.857 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.857 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.857 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.857 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:49.857 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:49.857 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.857 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.857 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.858 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.858 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.858 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.858 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.858 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.858 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.858 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.858 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.858 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.859 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:49.859 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:49.859 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.859 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.859 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.859 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:49.859 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.859 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.859 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.860 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.860 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.860 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.860 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.860 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.860 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.860 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.860 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.860 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.861 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.861 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.861 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.861 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:49.861 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:49.861 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.861 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.861 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.861 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.862 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.862 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.862 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.862 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.862 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.862 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.862 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.862 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.862 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.863 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.863 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:49.863 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "status", 2024-02-02T15:46:49.863 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:49.863 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:49.863 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:49.863 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:49.863 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:49.864 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.864 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.864 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.864 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.864 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.864 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.864 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.864 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.865 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.865 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.865 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.865 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.865 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.865 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.865 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.865 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.866 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.866 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.866 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.866 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:49.866 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:49.866 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.866 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.866 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.866 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.867 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.867 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.867 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.867 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.867 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.867 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.867 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.867 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.867 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:49.868 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:49.868 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.868 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.868 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.868 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:49.868 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.868 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.868 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.869 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.869 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.869 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.869 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.869 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.869 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.869 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.869 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.869 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.870 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.870 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.870 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.870 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:49.870 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:49.870 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.870 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.870 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.871 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.871 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.871 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.871 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.871 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.871 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.871 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.871 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.872 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.872 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.872 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.872 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:49.872 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "telegraf", 2024-02-02T15:46:49.872 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:49.872 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:49.872 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:49.873 INFO:teuthology.orchestra.run.smithi001.stdout: "address": { 2024-02-02T15:46:49.873 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "address", 2024-02-02T15:46:49.873 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.873 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.873 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.873 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-02-02T15:46:49.873 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.873 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.874 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.874 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.874 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.874 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.874 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.874 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.874 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-02-02T15:46:49.874 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-02-02T15:46:49.874 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-02-02T15:46:49.875 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.875 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.875 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "15", 2024-02-02T15:46:49.875 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.875 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.875 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.875 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.875 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.876 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.876 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.876 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.876 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:49.876 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:49.876 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.876 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.876 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.876 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.876 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.877 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.877 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.877 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.877 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.877 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.877 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.877 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.877 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.877 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.878 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.878 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.878 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.878 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.878 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.878 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:49.878 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:49.878 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.878 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.879 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.879 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.879 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.879 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.879 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.879 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.879 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.879 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.880 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.880 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.880 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:49.880 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:49.880 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.880 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.880 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.880 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:49.880 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.881 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.881 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.881 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.881 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.881 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.881 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.881 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.881 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.881 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.882 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.882 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.882 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.882 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.882 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.882 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:49.882 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:49.882 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.883 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.883 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.883 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.883 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.883 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.883 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.883 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.883 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.884 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.884 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.884 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.884 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.884 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.884 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:49.884 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "telemetry", 2024-02-02T15:46:49.885 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:49.885 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:49.885 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:49.885 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_basic": { 2024-02-02T15:46:49.885 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_basic", 2024-02-02T15:46:49.885 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.885 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.885 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.886 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:49.886 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.886 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.886 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.886 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share basic cluster information (size, version)", 2024-02-02T15:46:49.886 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.886 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.886 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.887 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.887 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_crash": { 2024-02-02T15:46:49.887 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_crash", 2024-02-02T15:46:49.887 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.887 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.887 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.887 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:49.887 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.888 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.888 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.888 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-02-02T15:46:49.888 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.888 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.888 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.888 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.888 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_device": { 2024-02-02T15:46:49.889 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_device", 2024-02-02T15:46:49.889 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.889 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.889 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.889 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "True", 2024-02-02T15:46:49.889 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.889 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.890 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.890 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-02-02T15:46:49.890 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.890 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.890 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.890 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.890 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_ident": { 2024-02-02T15:46:49.890 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_ident", 2024-02-02T15:46:49.890 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.891 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.891 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.891 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.891 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.891 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.891 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.891 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-02-02T15:46:49.891 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.891 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.892 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.892 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.892 INFO:teuthology.orchestra.run.smithi001.stdout: "channel_perf": { 2024-02-02T15:46:49.892 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "channel_perf", 2024-02-02T15:46:49.892 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.892 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.892 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.892 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.892 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.893 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.893 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.893 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Share various performance metrics of a cluster", 2024-02-02T15:46:49.893 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.893 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.893 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.893 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.893 INFO:teuthology.orchestra.run.smithi001.stdout: "contact": { 2024-02-02T15:46:49.893 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "contact", 2024-02-02T15:46:49.894 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.894 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.894 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.894 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.894 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.894 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.894 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.894 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.894 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.894 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.895 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.895 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.895 INFO:teuthology.orchestra.run.smithi001.stdout: "description": { 2024-02-02T15:46:49.895 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "description", 2024-02-02T15:46:49.895 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.895 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.895 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.895 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.895 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.896 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.896 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.896 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.896 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.896 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.896 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.896 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.896 INFO:teuthology.orchestra.run.smithi001.stdout: "device_url": { 2024-02-02T15:46:49.896 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "device_url", 2024-02-02T15:46:49.897 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.897 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.897 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.897 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-02-02T15:46:49.897 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.897 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.897 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.897 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.897 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.898 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.898 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.898 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.898 INFO:teuthology.orchestra.run.smithi001.stdout: "enabled": { 2024-02-02T15:46:49.898 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "enabled", 2024-02-02T15:46:49.898 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.898 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.898 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.898 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.898 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.899 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.899 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.899 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.899 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.899 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.899 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.899 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.899 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-02-02T15:46:49.899 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-02-02T15:46:49.899 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:49.900 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.900 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.900 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "24", 2024-02-02T15:46:49.900 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "8", 2024-02-02T15:46:49.900 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.900 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.900 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.900 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.900 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.900 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.901 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.901 INFO:teuthology.orchestra.run.smithi001.stdout: "last_opt_revision": { 2024-02-02T15:46:49.901 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "last_opt_revision", 2024-02-02T15:46:49.901 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:49.901 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.901 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.901 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "1", 2024-02-02T15:46:49.901 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.901 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.901 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.902 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.902 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.902 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.902 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.902 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.902 INFO:teuthology.orchestra.run.smithi001.stdout: "leaderboard": { 2024-02-02T15:46:49.902 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "leaderboard", 2024-02-02T15:46:49.902 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.902 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.903 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.903 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.903 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.903 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.903 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.903 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.903 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.903 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.903 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.903 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.904 INFO:teuthology.orchestra.run.smithi001.stdout: "leaderboard_description": { 2024-02-02T15:46:49.904 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "leaderboard_description", 2024-02-02T15:46:49.904 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.904 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.904 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.904 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.904 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.904 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.904 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.905 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.905 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.905 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.905 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.905 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.905 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:49.905 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:49.905 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.906 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.906 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.906 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.906 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.906 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.906 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.906 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.906 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.907 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.907 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.907 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.907 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.907 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.907 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.907 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.907 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.907 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.907 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.908 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:49.908 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:49.908 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.908 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.908 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.908 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.908 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.908 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.909 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.909 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.909 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.909 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.909 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.909 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.909 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:49.909 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:49.909 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.909 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.910 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.910 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:49.910 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.910 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.910 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.910 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.910 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.910 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.910 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.911 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.911 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.911 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.911 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.911 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.911 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.911 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.911 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.912 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:49.912 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:49.912 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.912 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.912 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.912 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.912 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.912 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.912 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.913 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.913 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.913 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.913 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.913 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.913 INFO:teuthology.orchestra.run.smithi001.stdout: "organization": { 2024-02-02T15:46:49.913 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "organization", 2024-02-02T15:46:49.913 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.914 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.914 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.914 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.914 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.914 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.914 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.914 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.914 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.915 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.915 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.915 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.915 INFO:teuthology.orchestra.run.smithi001.stdout: "proxy": { 2024-02-02T15:46:49.915 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "proxy", 2024-02-02T15:46:49.915 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.915 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.915 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.915 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.915 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.916 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.916 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.916 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.916 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.916 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.916 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.916 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.916 INFO:teuthology.orchestra.run.smithi001.stdout: "url": { 2024-02-02T15:46:49.917 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "url", 2024-02-02T15:46:49.917 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.917 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.917 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.917 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-02-02T15:46:49.917 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.917 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.917 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.917 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.918 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.918 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.918 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.918 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.918 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.918 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.918 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:49.918 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "test_orchestrator", 2024-02-02T15:46:49.918 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:49.919 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:49.919 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:49.919 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:49.919 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:49.919 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.919 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.919 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.919 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.919 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.919 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.920 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.920 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.920 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.920 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.920 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.920 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.920 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.920 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.920 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.921 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.921 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.921 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.921 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.921 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:49.921 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:49.921 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.921 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.921 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.921 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.922 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.922 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.922 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.922 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.922 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.922 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.922 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.922 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.922 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:49.923 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:49.923 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.923 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.923 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.923 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:49.923 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.923 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.923 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.923 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.924 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.924 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.924 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.924 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.924 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.924 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.924 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.924 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.925 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.925 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.925 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.925 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:49.925 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:49.925 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.925 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.925 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.926 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.926 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.926 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.926 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.926 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.926 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.926 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.926 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.927 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.927 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.927 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.927 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:49.927 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "volumes", 2024-02-02T15:46:49.927 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:49.927 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:49.927 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:49.927 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:49.928 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:49.928 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.928 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.928 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.928 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.928 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.928 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.929 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.929 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.929 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.929 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.929 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.929 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.929 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.929 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.930 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.930 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.930 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.930 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.930 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.930 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:49.930 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:49.931 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.931 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.931 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.931 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.931 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.931 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.931 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.932 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.932 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.932 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.932 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.932 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.932 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:49.932 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:49.932 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.932 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.933 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.933 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:49.933 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.933 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.933 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.933 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.933 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.933 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.933 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.934 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.934 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.934 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.934 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.934 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.934 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.934 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.934 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.934 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:49.935 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:49.935 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.935 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.935 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.935 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.935 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.935 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.935 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.935 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.936 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.936 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.936 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.936 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.936 INFO:teuthology.orchestra.run.smithi001.stdout: "max_concurrent_clones": { 2024-02-02T15:46:49.936 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "max_concurrent_clones", 2024-02-02T15:46:49.936 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:49.936 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.937 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.937 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "4", 2024-02-02T15:46:49.937 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.937 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.937 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.937 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Number of asynchronous cloner threads", 2024-02-02T15:46:49.937 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.937 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.937 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.937 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.938 INFO:teuthology.orchestra.run.smithi001.stdout: "snapshot_clone_delay": { 2024-02-02T15:46:49.938 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "snapshot_clone_delay", 2024-02-02T15:46:49.938 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:49.938 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.938 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.938 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "0", 2024-02-02T15:46:49.938 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.938 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.938 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.939 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-02-02T15:46:49.939 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.939 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.939 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.939 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.939 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.939 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.939 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:49.939 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "zabbix", 2024-02-02T15:46:49.939 INFO:teuthology.orchestra.run.smithi001.stdout: "can_run": true, 2024-02-02T15:46:49.940 INFO:teuthology.orchestra.run.smithi001.stdout: "error_string": "", 2024-02-02T15:46:49.940 INFO:teuthology.orchestra.run.smithi001.stdout: "module_options": { 2024-02-02T15:46:49.940 INFO:teuthology.orchestra.run.smithi001.stdout: "discovery_interval": { 2024-02-02T15:46:49.940 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "discovery_interval", 2024-02-02T15:46:49.940 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "uint", 2024-02-02T15:46:49.940 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.940 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.940 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "100", 2024-02-02T15:46:49.940 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.940 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.941 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.941 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.941 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.941 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.941 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.941 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.941 INFO:teuthology.orchestra.run.smithi001.stdout: "identifier": { 2024-02-02T15:46:49.941 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "identifier", 2024-02-02T15:46:49.941 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.942 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.942 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.942 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.942 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.942 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.942 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.942 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.942 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.942 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.942 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.943 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.943 INFO:teuthology.orchestra.run.smithi001.stdout: "interval": { 2024-02-02T15:46:49.943 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "interval", 2024-02-02T15:46:49.943 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "secs", 2024-02-02T15:46:49.943 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.943 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.943 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "60", 2024-02-02T15:46:49.943 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.943 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.943 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.944 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.944 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.944 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.944 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.944 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.944 INFO:teuthology.orchestra.run.smithi001.stdout: "log_level": { 2024-02-02T15:46:49.944 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_level", 2024-02-02T15:46:49.944 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.944 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.945 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.945 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.945 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.945 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.945 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.945 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.945 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.945 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.946 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.946 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.946 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.946 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.946 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.946 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.946 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.946 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.947 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.947 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster": { 2024-02-02T15:46:49.947 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster", 2024-02-02T15:46:49.947 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.947 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.947 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.947 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.947 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.948 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.948 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.948 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.948 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.948 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.948 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.948 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.949 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_cluster_level": { 2024-02-02T15:46:49.949 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_cluster_level", 2024-02-02T15:46:49.949 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.949 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.949 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.949 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "info", 2024-02-02T15:46:49.949 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.949 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.950 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [ 2024-02-02T15:46:49.950 INFO:teuthology.orchestra.run.smithi001.stdout: "", 2024-02-02T15:46:49.950 INFO:teuthology.orchestra.run.smithi001.stdout: "critical", 2024-02-02T15:46:49.950 INFO:teuthology.orchestra.run.smithi001.stdout: "debug", 2024-02-02T15:46:49.950 INFO:teuthology.orchestra.run.smithi001.stdout: "error", 2024-02-02T15:46:49.950 INFO:teuthology.orchestra.run.smithi001.stdout: "info", 2024-02-02T15:46:49.950 INFO:teuthology.orchestra.run.smithi001.stdout: "warning" 2024-02-02T15:46:49.950 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.951 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.951 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.951 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.951 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.951 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.951 INFO:teuthology.orchestra.run.smithi001.stdout: "log_to_file": { 2024-02-02T15:46:49.951 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "log_to_file", 2024-02-02T15:46:49.951 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "bool", 2024-02-02T15:46:49.952 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.952 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 1, 2024-02-02T15:46:49.952 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "False", 2024-02-02T15:46:49.952 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.952 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.952 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.952 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.952 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.953 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.953 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.953 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.953 INFO:teuthology.orchestra.run.smithi001.stdout: "zabbix_host": { 2024-02-02T15:46:49.953 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "zabbix_host", 2024-02-02T15:46:49.953 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.953 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.953 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.953 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "", 2024-02-02T15:46:49.954 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.954 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.954 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.954 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.954 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.954 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.954 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.954 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.954 INFO:teuthology.orchestra.run.smithi001.stdout: "zabbix_port": { 2024-02-02T15:46:49.954 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "zabbix_port", 2024-02-02T15:46:49.955 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "int", 2024-02-02T15:46:49.955 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.955 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.955 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "10051", 2024-02-02T15:46:49.955 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.955 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.955 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.955 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.955 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.956 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.956 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.956 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.956 INFO:teuthology.orchestra.run.smithi001.stdout: "zabbix_sender": { 2024-02-02T15:46:49.956 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "zabbix_sender", 2024-02-02T15:46:49.956 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "str", 2024-02-02T15:46:49.956 INFO:teuthology.orchestra.run.smithi001.stdout: "level": "advanced", 2024-02-02T15:46:49.956 INFO:teuthology.orchestra.run.smithi001.stdout: "flags": 0, 2024-02-02T15:46:49.957 INFO:teuthology.orchestra.run.smithi001.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-02-02T15:46:49.957 INFO:teuthology.orchestra.run.smithi001.stdout: "min": "", 2024-02-02T15:46:49.957 INFO:teuthology.orchestra.run.smithi001.stdout: "max": "", 2024-02-02T15:46:49.957 INFO:teuthology.orchestra.run.smithi001.stdout: "enum_allowed": [], 2024-02-02T15:46:49.957 INFO:teuthology.orchestra.run.smithi001.stdout: "desc": "", 2024-02-02T15:46:49.957 INFO:teuthology.orchestra.run.smithi001.stdout: "long_desc": "", 2024-02-02T15:46:49.957 INFO:teuthology.orchestra.run.smithi001.stdout: "tags": [], 2024-02-02T15:46:49.957 INFO:teuthology.orchestra.run.smithi001.stdout: "see_also": [] 2024-02-02T15:46:49.957 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.958 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.958 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.958 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.958 INFO:teuthology.orchestra.run.smithi001.stdout: "services": { 2024-02-02T15:46:49.958 INFO:teuthology.orchestra.run.smithi001.stdout: "dashboard": "https://172.21.15.1:8443/" 2024-02-02T15:46:49.958 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.958 INFO:teuthology.orchestra.run.smithi001.stdout: "always_on_modules": { 2024-02-02T15:46:49.958 INFO:teuthology.orchestra.run.smithi001.stdout: "octopus": [ 2024-02-02T15:46:49.958 INFO:teuthology.orchestra.run.smithi001.stdout: "balancer", 2024-02-02T15:46:49.958 INFO:teuthology.orchestra.run.smithi001.stdout: "crash", 2024-02-02T15:46:49.959 INFO:teuthology.orchestra.run.smithi001.stdout: "devicehealth", 2024-02-02T15:46:49.959 INFO:teuthology.orchestra.run.smithi001.stdout: "orchestrator", 2024-02-02T15:46:49.959 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_autoscaler", 2024-02-02T15:46:49.959 INFO:teuthology.orchestra.run.smithi001.stdout: "progress", 2024-02-02T15:46:49.959 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_support", 2024-02-02T15:46:49.959 INFO:teuthology.orchestra.run.smithi001.stdout: "status", 2024-02-02T15:46:49.959 INFO:teuthology.orchestra.run.smithi001.stdout: "telemetry", 2024-02-02T15:46:49.959 INFO:teuthology.orchestra.run.smithi001.stdout: "volumes" 2024-02-02T15:46:49.959 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.959 INFO:teuthology.orchestra.run.smithi001.stdout: "pacific": [ 2024-02-02T15:46:49.960 INFO:teuthology.orchestra.run.smithi001.stdout: "balancer", 2024-02-02T15:46:49.960 INFO:teuthology.orchestra.run.smithi001.stdout: "crash", 2024-02-02T15:46:49.960 INFO:teuthology.orchestra.run.smithi001.stdout: "devicehealth", 2024-02-02T15:46:49.960 INFO:teuthology.orchestra.run.smithi001.stdout: "orchestrator", 2024-02-02T15:46:49.960 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_autoscaler", 2024-02-02T15:46:49.960 INFO:teuthology.orchestra.run.smithi001.stdout: "progress", 2024-02-02T15:46:49.960 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_support", 2024-02-02T15:46:49.960 INFO:teuthology.orchestra.run.smithi001.stdout: "status", 2024-02-02T15:46:49.960 INFO:teuthology.orchestra.run.smithi001.stdout: "telemetry", 2024-02-02T15:46:49.961 INFO:teuthology.orchestra.run.smithi001.stdout: "volumes" 2024-02-02T15:46:49.961 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.961 INFO:teuthology.orchestra.run.smithi001.stdout: "quincy": [ 2024-02-02T15:46:49.961 INFO:teuthology.orchestra.run.smithi001.stdout: "balancer", 2024-02-02T15:46:49.961 INFO:teuthology.orchestra.run.smithi001.stdout: "crash", 2024-02-02T15:46:49.961 INFO:teuthology.orchestra.run.smithi001.stdout: "devicehealth", 2024-02-02T15:46:49.961 INFO:teuthology.orchestra.run.smithi001.stdout: "orchestrator", 2024-02-02T15:46:49.961 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_autoscaler", 2024-02-02T15:46:49.961 INFO:teuthology.orchestra.run.smithi001.stdout: "progress", 2024-02-02T15:46:49.962 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_support", 2024-02-02T15:46:49.962 INFO:teuthology.orchestra.run.smithi001.stdout: "status", 2024-02-02T15:46:49.962 INFO:teuthology.orchestra.run.smithi001.stdout: "telemetry", 2024-02-02T15:46:49.962 INFO:teuthology.orchestra.run.smithi001.stdout: "volumes" 2024-02-02T15:46:49.962 INFO:teuthology.orchestra.run.smithi001.stdout: ], 2024-02-02T15:46:49.962 INFO:teuthology.orchestra.run.smithi001.stdout: "reef": [ 2024-02-02T15:46:49.962 INFO:teuthology.orchestra.run.smithi001.stdout: "balancer", 2024-02-02T15:46:49.962 INFO:teuthology.orchestra.run.smithi001.stdout: "crash", 2024-02-02T15:46:49.962 INFO:teuthology.orchestra.run.smithi001.stdout: "devicehealth", 2024-02-02T15:46:49.963 INFO:teuthology.orchestra.run.smithi001.stdout: "orchestrator", 2024-02-02T15:46:49.963 INFO:teuthology.orchestra.run.smithi001.stdout: "pg_autoscaler", 2024-02-02T15:46:49.963 INFO:teuthology.orchestra.run.smithi001.stdout: "progress", 2024-02-02T15:46:49.963 INFO:teuthology.orchestra.run.smithi001.stdout: "rbd_support", 2024-02-02T15:46:49.963 INFO:teuthology.orchestra.run.smithi001.stdout: "status", 2024-02-02T15:46:49.963 INFO:teuthology.orchestra.run.smithi001.stdout: "telemetry", 2024-02-02T15:46:49.963 INFO:teuthology.orchestra.run.smithi001.stdout: "volumes" 2024-02-02T15:46:49.963 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-02-02T15:46:49.963 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.964 INFO:teuthology.orchestra.run.smithi001.stdout: "last_failure_osd_epoch": 89, 2024-02-02T15:46:49.964 INFO:teuthology.orchestra.run.smithi001.stdout: "active_clients": [ 2024-02-02T15:46:49.964 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:49.964 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "devicehealth", 2024-02-02T15:46:49.964 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-02-02T15:46:49.964 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:49.964 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-02-02T15:46:49.964 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:0", 2024-02-02T15:46:49.965 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2384672839 2024-02-02T15:46:49.965 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.965 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-02-02T15:46:49.965 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.965 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:49.965 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "libcephsqlite", 2024-02-02T15:46:49.965 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-02-02T15:46:49.965 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:49.966 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-02-02T15:46:49.966 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:0", 2024-02-02T15:46:49.966 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 1279332394 2024-02-02T15:46:49.966 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.966 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-02-02T15:46:49.966 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.966 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:49.966 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "rbd_support", 2024-02-02T15:46:49.967 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-02-02T15:46:49.967 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:49.967 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-02-02T15:46:49.967 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:0", 2024-02-02T15:46:49.967 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 2956076392 2024-02-02T15:46:49.967 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.967 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-02-02T15:46:49.967 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.968 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:49.968 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "snap_schedule", 2024-02-02T15:46:49.968 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-02-02T15:46:49.968 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:49.968 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-02-02T15:46:49.968 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:0", 2024-02-02T15:46:49.968 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 3914891732 2024-02-02T15:46:49.968 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.968 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-02-02T15:46:49.969 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.969 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:49.969 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "snap_schedule", 2024-02-02T15:46:49.969 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-02-02T15:46:49.969 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:49.969 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-02-02T15:46:49.969 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:0", 2024-02-02T15:46:49.969 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 3514160608 2024-02-02T15:46:49.969 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.969 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-02-02T15:46:49.970 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:49.970 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:49.970 INFO:teuthology.orchestra.run.smithi001.stdout: "name": "volumes", 2024-02-02T15:46:49.970 INFO:teuthology.orchestra.run.smithi001.stdout: "addrvec": [ 2024-02-02T15:46:49.970 INFO:teuthology.orchestra.run.smithi001.stdout: { 2024-02-02T15:46:49.970 INFO:teuthology.orchestra.run.smithi001.stdout: "type": "v2", 2024-02-02T15:46:49.970 INFO:teuthology.orchestra.run.smithi001.stdout: "addr": "172.21.15.1:0", 2024-02-02T15:46:49.970 INFO:teuthology.orchestra.run.smithi001.stdout: "nonce": 381626412 2024-02-02T15:46:49.970 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.970 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-02-02T15:46:49.971 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:49.971 INFO:teuthology.orchestra.run.smithi001.stdout: ] 2024-02-02T15:46:49.971 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:46:49.973 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:46:47 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/2126011963' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-02T15:46:49.973 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:46:47 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/907938665' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-02T15:46:49.973 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:46:48 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/3305643017' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-02-02T15:46:49.973 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:46:48 smithi142 ceph-mon[45728]: pgmap v623: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-02-02T15:46:49.973 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:46:47 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/2126011963' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-02T15:46:49.973 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:46:47 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/907938665' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-02T15:46:49.973 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:46:48 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/3305643017' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-02-02T15:46:49.974 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:46:48 smithi071 ceph-mon[44707]: pgmap v623: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-02-02T15:46:49.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.837+0000 7f4f6b000700 1 -- 172.21.15.1:0/3305643017 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f4f440a0690 msgr2=0x7f4f440a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:49.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.837+0000 7f4f6b000700 1 --2- 172.21.15.1:0/3305643017 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f4f440a0690 0x7f4f440a2b50 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f4f54009c60 tx=0x7f4f54009260 comp rx=0 tx=0).stop 2024-02-02T15:46:49.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.837+0000 7f4f6b000700 1 -- 172.21.15.1:0/3305643017 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f64073ac0 msgr2=0x7f4f641a5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:46:49.974 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.837+0000 7f4f6b000700 1 --2- 172.21.15.1:0/3305643017 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f64073ac0 0x7f4f641a5080 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c00ef30 tx=0x7f4f5c00c3a0 comp rx=0 tx=0).stop 2024-02-02T15:46:49.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.837+0000 7f4f6b000700 1 -- 172.21.15.1:0/3305643017 shutdown_connections 2024-02-02T15:46:49.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.837+0000 7f4f6b000700 1 --2- 172.21.15.1:0/3305643017 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f4f440a0690 0x7f4f440a2b50 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:49.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.837+0000 7f4f6b000700 1 --2- 172.21.15.1:0/3305643017 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f4f64073ac0 0x7f4f641a5080 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:49.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.837+0000 7f4f6b000700 1 --2- 172.21.15.1:0/3305643017 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f4f64100f10 0x7f4f64073040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:49.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.837+0000 7f4f6b000700 1 --2- 172.21.15.1:0/3305643017 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f64106030 0x7f4f64073580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:46:49.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.837+0000 7f4f6b000700 1 -- 172.21.15.1:0/3305643017 >> 172.21.15.1:0/3305643017 conn(0x7f4f640fc790 msgr2=0x7f4f640fd260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:46:49.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.837+0000 7f4f6b000700 1 -- 172.21.15.1:0/3305643017 shutdown_connections 2024-02-02T15:46:49.975 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:46:47.837+0000 7f4f6b000700 1 -- 172.21.15.1:0/3305643017 wait complete. 2024-02-02T15:46:49.981 DEBUG:tasks.check_counter:Getting stats from a 2024-02-02T15:46:49.982 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph daemon mds.a perf dump 2024-02-02T15:46:49.982 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:46:47 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/2126011963' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-02T15:46:49.982 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:46:47 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/907938665' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-02T15:46:49.982 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:46:48 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/3305643017' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-02-02T15:46:49.982 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:46:48 smithi001 ceph-mon[35470]: pgmap v623: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 41 KiB/s wr, 12 op/s 2024-02-02T15:46:50.358 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.a/config 2024-02-02T15:46:51.562 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:46:51.562 INFO:teuthology.orchestra.run.smithi001.stdout: "AsyncMessenger::Worker-0": { 2024-02-02T15:46:51.563 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_messages": 2142, 2024-02-02T15:46:51.563 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages": 2144, 2024-02-02T15:46:51.563 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_bytes": 122285923, 2024-02-02T15:46:51.563 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_bytes": 666163, 2024-02-02T15:46:51.563 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_created_connections": 8, 2024-02-02T15:46:51.563 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_active_connections": 4, 2024-02-02T15:46:51.563 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_total_time": 0.434436242, 2024-02-02T15:46:51.564 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_send_time": 0.076609098, 2024-02-02T15:46:51.564 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_recv_time": 0.252178191, 2024-02-02T15:46:51.564 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_fast_dispatch_time": 0.078434407, 2024-02-02T15:46:51.564 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages_queue_lat": { 2024-02-02T15:46:51.564 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 2142, 2024-02-02T15:46:51.564 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.151294328, 2024-02-02T15:46:51.564 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000070632 2024-02-02T15:46:51.565 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:51.565 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_handle_ack_lat": { 2024-02-02T15:46:51.565 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:51.565 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:51.565 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:51.565 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:51.565 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_encrypted_bytes": 0, 2024-02-02T15:46:51.565 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_encrypted_bytes": 0 2024-02-02T15:46:51.565 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:51.566 INFO:teuthology.orchestra.run.smithi001.stdout: "AsyncMessenger::Worker-1": { 2024-02-02T15:46:51.566 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_messages": 1991, 2024-02-02T15:46:51.566 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages": 2258, 2024-02-02T15:46:51.566 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_bytes": 168296739, 2024-02-02T15:46:51.566 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_bytes": 1323652, 2024-02-02T15:46:51.566 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_created_connections": 10, 2024-02-02T15:46:51.566 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_active_connections": 7, 2024-02-02T15:46:51.567 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_total_time": 0.507779525, 2024-02-02T15:46:51.567 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_send_time": 0.072453431, 2024-02-02T15:46:51.567 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_recv_time": 0.348324542, 2024-02-02T15:46:51.567 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_fast_dispatch_time": 0.065391527, 2024-02-02T15:46:51.567 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages_queue_lat": { 2024-02-02T15:46:51.567 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 2256, 2024-02-02T15:46:51.567 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.149661716, 2024-02-02T15:46:51.567 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000066339 2024-02-02T15:46:51.568 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:51.568 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_handle_ack_lat": { 2024-02-02T15:46:51.568 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:51.568 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:51.568 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:51.568 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:51.568 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_encrypted_bytes": 3472, 2024-02-02T15:46:51.568 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_encrypted_bytes": 706560 2024-02-02T15:46:51.568 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:51.569 INFO:teuthology.orchestra.run.smithi001.stdout: "AsyncMessenger::Worker-2": { 2024-02-02T15:46:51.569 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_messages": 1267, 2024-02-02T15:46:51.569 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages": 1200, 2024-02-02T15:46:51.569 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_bytes": 119113151, 2024-02-02T15:46:51.569 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_bytes": 461005, 2024-02-02T15:46:51.569 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_created_connections": 5, 2024-02-02T15:46:51.569 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_active_connections": 4, 2024-02-02T15:46:51.569 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_total_time": 0.357476416, 2024-02-02T15:46:51.569 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_send_time": 0.064356771, 2024-02-02T15:46:51.570 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_recv_time": 0.255127353, 2024-02-02T15:46:51.570 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_fast_dispatch_time": 0.045940689, 2024-02-02T15:46:51.570 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages_queue_lat": { 2024-02-02T15:46:51.570 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 1199, 2024-02-02T15:46:51.570 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.089408469, 2024-02-02T15:46:51.570 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000074569 2024-02-02T15:46:51.570 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:51.570 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_handle_ack_lat": { 2024-02-02T15:46:51.570 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:51.571 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:51.571 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:51.571 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:51.571 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_encrypted_bytes": 2230480, 2024-02-02T15:46:51.571 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_encrypted_bytes": 208784 2024-02-02T15:46:51.571 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:51.571 INFO:teuthology.orchestra.run.smithi001.stdout: "cct": { 2024-02-02T15:46:51.572 INFO:teuthology.orchestra.run.smithi001.stdout: "total_workers": 1, 2024-02-02T15:46:51.572 INFO:teuthology.orchestra.run.smithi001.stdout: "unhealthy_workers": 0 2024-02-02T15:46:51.572 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:51.572 INFO:teuthology.orchestra.run.smithi001.stdout: "finisher-MDSRank": { 2024-02-02T15:46:51.572 INFO:teuthology.orchestra.run.smithi001.stdout: "queue_len": 0, 2024-02-02T15:46:51.572 INFO:teuthology.orchestra.run.smithi001.stdout: "complete_latency": { 2024-02-02T15:46:51.572 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 8150, 2024-02-02T15:46:51.572 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.268912402, 2024-02-02T15:46:51.572 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000032995 2024-02-02T15:46:51.573 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:51.573 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:51.573 INFO:teuthology.orchestra.run.smithi001.stdout: "finisher-PurgeQueue": { 2024-02-02T15:46:51.573 INFO:teuthology.orchestra.run.smithi001.stdout: "queue_len": 0, 2024-02-02T15:46:51.573 INFO:teuthology.orchestra.run.smithi001.stdout: "complete_latency": { 2024-02-02T15:46:51.573 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:51.573 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:51.573 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:51.573 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:51.574 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:51.574 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": { 2024-02-02T15:46:51.574 INFO:teuthology.orchestra.run.smithi001.stdout: "request": 0, 2024-02-02T15:46:51.574 INFO:teuthology.orchestra.run.smithi001.stdout: "reply": 0, 2024-02-02T15:46:51.574 INFO:teuthology.orchestra.run.smithi001.stdout: "reply_latency": { 2024-02-02T15:46:51.574 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:51.574 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:51.574 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:51.575 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:51.575 INFO:teuthology.orchestra.run.smithi001.stdout: "slow_reply": 0, 2024-02-02T15:46:51.575 INFO:teuthology.orchestra.run.smithi001.stdout: "forward": 0, 2024-02-02T15:46:51.575 INFO:teuthology.orchestra.run.smithi001.stdout: "dir_fetch_complete": 0, 2024-02-02T15:46:51.575 INFO:teuthology.orchestra.run.smithi001.stdout: "dir_fetch_keys": 0, 2024-02-02T15:46:51.575 INFO:teuthology.orchestra.run.smithi001.stdout: "dir_commit": 0, 2024-02-02T15:46:51.575 INFO:teuthology.orchestra.run.smithi001.stdout: "dir_split": 0, 2024-02-02T15:46:51.575 INFO:teuthology.orchestra.run.smithi001.stdout: "dir_merge": 0, 2024-02-02T15:46:51.576 INFO:teuthology.orchestra.run.smithi001.stdout: "inodes": 18019, 2024-02-02T15:46:51.576 INFO:teuthology.orchestra.run.smithi001.stdout: "inodes_top": 254, 2024-02-02T15:46:51.576 INFO:teuthology.orchestra.run.smithi001.stdout: "inodes_bottom": 17765, 2024-02-02T15:46:51.576 INFO:teuthology.orchestra.run.smithi001.stdout: "inodes_pin_tail": 0, 2024-02-02T15:46:51.576 INFO:teuthology.orchestra.run.smithi001.stdout: "inodes_pinned": 17656, 2024-02-02T15:46:51.576 INFO:teuthology.orchestra.run.smithi001.stdout: "inodes_expired": 0, 2024-02-02T15:46:51.576 INFO:teuthology.orchestra.run.smithi001.stdout: "inodes_with_caps": 0, 2024-02-02T15:46:51.576 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": 0, 2024-02-02T15:46:51.577 INFO:teuthology.orchestra.run.smithi001.stdout: "subtrees": 5, 2024-02-02T15:46:51.577 INFO:teuthology.orchestra.run.smithi001.stdout: "traverse": 0, 2024-02-02T15:46:51.577 INFO:teuthology.orchestra.run.smithi001.stdout: "traverse_hit": 0, 2024-02-02T15:46:51.577 INFO:teuthology.orchestra.run.smithi001.stdout: "traverse_forward": 0, 2024-02-02T15:46:51.577 INFO:teuthology.orchestra.run.smithi001.stdout: "traverse_discover": 0, 2024-02-02T15:46:51.577 INFO:teuthology.orchestra.run.smithi001.stdout: "traverse_dir_fetch": 0, 2024-02-02T15:46:51.577 INFO:teuthology.orchestra.run.smithi001.stdout: "traverse_remote_ino": 0, 2024-02-02T15:46:51.577 INFO:teuthology.orchestra.run.smithi001.stdout: "traverse_lock": 0, 2024-02-02T15:46:51.578 INFO:teuthology.orchestra.run.smithi001.stdout: "load_cent": 0, 2024-02-02T15:46:51.578 INFO:teuthology.orchestra.run.smithi001.stdout: "q": 0, 2024-02-02T15:46:51.578 INFO:teuthology.orchestra.run.smithi001.stdout: "exported": 0, 2024-02-02T15:46:51.578 INFO:teuthology.orchestra.run.smithi001.stdout: "exported_inodes": 0, 2024-02-02T15:46:51.578 INFO:teuthology.orchestra.run.smithi001.stdout: "imported": 0, 2024-02-02T15:46:51.578 INFO:teuthology.orchestra.run.smithi001.stdout: "imported_inodes": 0, 2024-02-02T15:46:51.578 INFO:teuthology.orchestra.run.smithi001.stdout: "openino_dir_fetch": 0, 2024-02-02T15:46:51.578 INFO:teuthology.orchestra.run.smithi001.stdout: "openino_backtrace_fetch": 0, 2024-02-02T15:46:51.578 INFO:teuthology.orchestra.run.smithi001.stdout: "openino_peer_discover": 0, 2024-02-02T15:46:51.579 INFO:teuthology.orchestra.run.smithi001.stdout: "root_rfiles": 0, 2024-02-02T15:46:51.579 INFO:teuthology.orchestra.run.smithi001.stdout: "root_rbytes": 0, 2024-02-02T15:46:51.579 INFO:teuthology.orchestra.run.smithi001.stdout: "root_rsnaps": 0, 2024-02-02T15:46:51.579 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_backtrace_fetch": 0, 2024-02-02T15:46:51.579 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_set_tag": 0, 2024-02-02T15:46:51.579 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_backtrace_repaired": 0, 2024-02-02T15:46:51.579 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_inotable_repaired": 0, 2024-02-02T15:46:51.579 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_dir_inodes": 0, 2024-02-02T15:46:51.580 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_dir_base_inodes": 0, 2024-02-02T15:46:51.580 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_dirfrag_rstats": 0, 2024-02-02T15:46:51.580 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_file_inodes": 0, 2024-02-02T15:46:51.580 INFO:teuthology.orchestra.run.smithi001.stdout: "handle_inode_file_caps": 0, 2024-02-02T15:46:51.580 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_cap_op_revoke": 0, 2024-02-02T15:46:51.580 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_cap_op_grant": 0, 2024-02-02T15:46:51.580 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_cap_op_trunc": 0, 2024-02-02T15:46:51.580 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-02-02T15:46:51.581 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_cap_op_flush_ack": 0, 2024-02-02T15:46:51.581 INFO:teuthology.orchestra.run.smithi001.stdout: "handle_client_caps": 0, 2024-02-02T15:46:51.581 INFO:teuthology.orchestra.run.smithi001.stdout: "handle_client_caps_dirty": 0, 2024-02-02T15:46:51.581 INFO:teuthology.orchestra.run.smithi001.stdout: "handle_client_cap_release": 0, 2024-02-02T15:46:51.581 INFO:teuthology.orchestra.run.smithi001.stdout: "process_request_cap_release": 0 2024-02-02T15:46:51.581 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:51.581 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_cache": { 2024-02-02T15:46:51.581 INFO:teuthology.orchestra.run.smithi001.stdout: "dir_update": 0, 2024-02-02T15:46:51.581 INFO:teuthology.orchestra.run.smithi001.stdout: "dir_update_receipt": 0, 2024-02-02T15:46:51.582 INFO:teuthology.orchestra.run.smithi001.stdout: "dir_try_discover": 0, 2024-02-02T15:46:51.582 INFO:teuthology.orchestra.run.smithi001.stdout: "dir_send_discover": 0, 2024-02-02T15:46:51.582 INFO:teuthology.orchestra.run.smithi001.stdout: "dir_handle_discover": 0, 2024-02-02T15:46:51.582 INFO:teuthology.orchestra.run.smithi001.stdout: "num_strays": 0, 2024-02-02T15:46:51.583 INFO:teuthology.orchestra.run.smithi001.stdout: "num_strays_delayed": 0, 2024-02-02T15:46:51.583 INFO:teuthology.orchestra.run.smithi001.stdout: "num_strays_enqueuing": 0, 2024-02-02T15:46:51.583 INFO:teuthology.orchestra.run.smithi001.stdout: "strays_created": 0, 2024-02-02T15:46:51.583 INFO:teuthology.orchestra.run.smithi001.stdout: "strays_enqueued": 0, 2024-02-02T15:46:51.584 INFO:teuthology.orchestra.run.smithi001.stdout: "strays_reintegrated": 0, 2024-02-02T15:46:51.584 INFO:teuthology.orchestra.run.smithi001.stdout: "strays_migrated": 0, 2024-02-02T15:46:51.584 INFO:teuthology.orchestra.run.smithi001.stdout: "num_recovering_processing": 0, 2024-02-02T15:46:51.584 INFO:teuthology.orchestra.run.smithi001.stdout: "num_recovering_enqueued": 0, 2024-02-02T15:46:51.584 INFO:teuthology.orchestra.run.smithi001.stdout: "num_recovering_prioritized": 0, 2024-02-02T15:46:51.584 INFO:teuthology.orchestra.run.smithi001.stdout: "recovery_started": 0, 2024-02-02T15:46:51.584 INFO:teuthology.orchestra.run.smithi001.stdout: "recovery_completed": 0, 2024-02-02T15:46:51.584 INFO:teuthology.orchestra.run.smithi001.stdout: "ireq_enqueue_scrub": 0, 2024-02-02T15:46:51.585 INFO:teuthology.orchestra.run.smithi001.stdout: "ireq_exportdir": 0, 2024-02-02T15:46:51.585 INFO:teuthology.orchestra.run.smithi001.stdout: "ireq_flush": 0, 2024-02-02T15:46:51.585 INFO:teuthology.orchestra.run.smithi001.stdout: "ireq_fragmentdir": 0, 2024-02-02T15:46:51.585 INFO:teuthology.orchestra.run.smithi001.stdout: "ireq_fragstats": 0, 2024-02-02T15:46:51.585 INFO:teuthology.orchestra.run.smithi001.stdout: "ireq_inodestats": 0 2024-02-02T15:46:51.585 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:51.585 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_log": { 2024-02-02T15:46:51.585 INFO:teuthology.orchestra.run.smithi001.stdout: "evadd": 0, 2024-02-02T15:46:51.586 INFO:teuthology.orchestra.run.smithi001.stdout: "evex": 0, 2024-02-02T15:46:51.586 INFO:teuthology.orchestra.run.smithi001.stdout: "evtrm": 0, 2024-02-02T15:46:51.586 INFO:teuthology.orchestra.run.smithi001.stdout: "ev": 93950, 2024-02-02T15:46:51.586 INFO:teuthology.orchestra.run.smithi001.stdout: "evexg": 0, 2024-02-02T15:46:51.586 INFO:teuthology.orchestra.run.smithi001.stdout: "evexd": 0, 2024-02-02T15:46:51.586 INFO:teuthology.orchestra.run.smithi001.stdout: "segadd": 0, 2024-02-02T15:46:51.586 INFO:teuthology.orchestra.run.smithi001.stdout: "segex": 0, 2024-02-02T15:46:51.586 INFO:teuthology.orchestra.run.smithi001.stdout: "segtrm": 0, 2024-02-02T15:46:51.586 INFO:teuthology.orchestra.run.smithi001.stdout: "seg": 129, 2024-02-02T15:46:51.587 INFO:teuthology.orchestra.run.smithi001.stdout: "segexg": 0, 2024-02-02T15:46:51.587 INFO:teuthology.orchestra.run.smithi001.stdout: "segexd": 0, 2024-02-02T15:46:51.587 INFO:teuthology.orchestra.run.smithi001.stdout: "expos": 8392644, 2024-02-02T15:46:51.587 INFO:teuthology.orchestra.run.smithi001.stdout: "wrpos": 410276908, 2024-02-02T15:46:51.587 INFO:teuthology.orchestra.run.smithi001.stdout: "rdpos": 410270454, 2024-02-02T15:46:51.587 INFO:teuthology.orchestra.run.smithi001.stdout: "jlat": { 2024-02-02T15:46:51.587 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:51.587 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:51.587 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:51.588 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:51.588 INFO:teuthology.orchestra.run.smithi001.stdout: "replayed": 93950 2024-02-02T15:46:51.588 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:51.588 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_mem": { 2024-02-02T15:46:51.588 INFO:teuthology.orchestra.run.smithi001.stdout: "ino": 787, 2024-02-02T15:46:51.588 INFO:teuthology.orchestra.run.smithi001.stdout: "ino+": 18152, 2024-02-02T15:46:51.588 INFO:teuthology.orchestra.run.smithi001.stdout: "ino-": 17365, 2024-02-02T15:46:51.588 INFO:teuthology.orchestra.run.smithi001.stdout: "dir": 55, 2024-02-02T15:46:51.589 INFO:teuthology.orchestra.run.smithi001.stdout: "dir+": 71, 2024-02-02T15:46:51.589 INFO:teuthology.orchestra.run.smithi001.stdout: "dir-": 16, 2024-02-02T15:46:51.589 INFO:teuthology.orchestra.run.smithi001.stdout: "dn": 18019, 2024-02-02T15:46:51.589 INFO:teuthology.orchestra.run.smithi001.stdout: "dn+": 18321, 2024-02-02T15:46:51.589 INFO:teuthology.orchestra.run.smithi001.stdout: "dn-": 302, 2024-02-02T15:46:51.589 INFO:teuthology.orchestra.run.smithi001.stdout: "cap": 0, 2024-02-02T15:46:51.589 INFO:teuthology.orchestra.run.smithi001.stdout: "cap+": 0, 2024-02-02T15:46:51.589 INFO:teuthology.orchestra.run.smithi001.stdout: "cap-": 0, 2024-02-02T15:46:51.589 INFO:teuthology.orchestra.run.smithi001.stdout: "rss": 70432, 2024-02-02T15:46:51.590 INFO:teuthology.orchestra.run.smithi001.stdout: "heap": 207156 2024-02-02T15:46:51.590 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:51.590 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_server": { 2024-02-02T15:46:51.590 INFO:teuthology.orchestra.run.smithi001.stdout: "dispatch_client_request": 0, 2024-02-02T15:46:51.590 INFO:teuthology.orchestra.run.smithi001.stdout: "dispatch_server_request": 0, 2024-02-02T15:46:51.590 INFO:teuthology.orchestra.run.smithi001.stdout: "handle_client_request": 0, 2024-02-02T15:46:51.590 INFO:teuthology.orchestra.run.smithi001.stdout: "handle_client_session": 0, 2024-02-02T15:46:51.590 INFO:teuthology.orchestra.run.smithi001.stdout: "handle_peer_request": 0, 2024-02-02T15:46:51.590 INFO:teuthology.orchestra.run.smithi001.stdout: "req_create_latency": { 2024-02-02T15:46:51.591 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:51.591 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:51.591 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:51.591 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:51.591 INFO:teuthology.orchestra.run.smithi001.stdout: "req_getattr_latency": { 2024-02-02T15:46:51.591 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:51.591 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:51.591 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:51.592 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:51.592 INFO:teuthology.orchestra.run.smithi001.stdout: "req_getfilelock_latency": { 2024-02-02T15:46:51.592 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:51.592 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:51.592 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:51.592 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:51.592 INFO:teuthology.orchestra.run.smithi001.stdout: "req_link_latency": { 2024-02-02T15:46:51.592 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:51.593 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:51.593 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:51.593 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:51.593 INFO:teuthology.orchestra.run.smithi001.stdout: "req_lookup_latency": { 2024-02-02T15:46:51.593 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:51.593 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:51.593 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:51.593 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:51.593 INFO:teuthology.orchestra.run.smithi001.stdout: "req_lookuphash_latency": { 2024-02-02T15:46:51.594 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:51.594 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:51.594 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:51.594 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:51.594 INFO:teuthology.orchestra.run.smithi001.stdout: "req_lookupino_latency": { 2024-02-02T15:46:51.594 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:51.594 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:51.594 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:51.594 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:51.595 INFO:teuthology.orchestra.run.smithi001.stdout: "req_lookupname_latency": { 2024-02-02T15:46:51.595 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:51.595 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:51.595 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:51.595 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:51.595 INFO:teuthology.orchestra.run.smithi001.stdout: "req_lookupparent_latency": { 2024-02-02T15:46:51.595 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:51.595 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:51.595 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:51.596 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:51.596 INFO:teuthology.orchestra.run.smithi001.stdout: "req_lookupsnap_latency": { 2024-02-02T15:46:51.596 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:51.596 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:51.596 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:51.596 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:51.596 INFO:teuthology.orchestra.run.smithi001.stdout: "req_lssnap_latency": { 2024-02-02T15:46:51.596 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:51.597 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:51.597 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:51.597 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:51.597 INFO:teuthology.orchestra.run.smithi001.stdout: "req_mkdir_latency": { 2024-02-02T15:46:51.597 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:51.597 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:51.597 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:51.597 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:51.598 INFO:teuthology.orchestra.run.smithi001.stdout: "req_mknod_latency": { 2024-02-02T15:46:51.598 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:51.598 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:51.598 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:51.598 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:51.598 INFO:teuthology.orchestra.run.smithi001.stdout: "req_mksnap_latency": { 2024-02-02T15:46:51.598 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:51.598 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:51.598 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:51.599 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:51.599 INFO:teuthology.orchestra.run.smithi001.stdout: "req_open_latency": { 2024-02-02T15:46:51.599 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:51.599 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:51.599 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:51.599 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:51.599 INFO:teuthology.orchestra.run.smithi001.stdout: "req_readdir_latency": { 2024-02-02T15:46:51.599 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:51.599 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:51.600 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:51.600 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:51.600 INFO:teuthology.orchestra.run.smithi001.stdout: "req_rename_latency": { 2024-02-02T15:46:51.600 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:51.600 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:51.600 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:51.600 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:51.600 INFO:teuthology.orchestra.run.smithi001.stdout: "req_renamesnap_latency": { 2024-02-02T15:46:51.601 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:51.601 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:51.601 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:51.601 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:51.601 INFO:teuthology.orchestra.run.smithi001.stdout: "req_snapdiff_latency": { 2024-02-02T15:46:51.601 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:51.601 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:51.601 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:51.601 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:51.602 INFO:teuthology.orchestra.run.smithi001.stdout: "req_rmdir_latency": { 2024-02-02T15:46:51.602 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:51.602 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:51.602 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:51.602 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:51.602 INFO:teuthology.orchestra.run.smithi001.stdout: "req_rmsnap_latency": { 2024-02-02T15:46:51.602 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:51.602 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:51.603 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:51.603 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:51.603 INFO:teuthology.orchestra.run.smithi001.stdout: "req_rmxattr_latency": { 2024-02-02T15:46:51.603 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:51.603 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:51.603 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:51.603 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:51.603 INFO:teuthology.orchestra.run.smithi001.stdout: "req_setattr_latency": { 2024-02-02T15:46:51.603 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:51.604 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:51.604 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:51.604 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:51.604 INFO:teuthology.orchestra.run.smithi001.stdout: "req_setdirlayout_latency": { 2024-02-02T15:46:51.604 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:51.604 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:51.604 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:51.604 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:51.604 INFO:teuthology.orchestra.run.smithi001.stdout: "req_setfilelock_latency": { 2024-02-02T15:46:51.605 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:51.605 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:51.605 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:51.605 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:51.605 INFO:teuthology.orchestra.run.smithi001.stdout: "req_setlayout_latency": { 2024-02-02T15:46:51.605 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:51.605 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:51.605 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:51.606 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:51.606 INFO:teuthology.orchestra.run.smithi001.stdout: "req_setxattr_latency": { 2024-02-02T15:46:51.606 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:51.606 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:51.606 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:51.606 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:51.606 INFO:teuthology.orchestra.run.smithi001.stdout: "req_symlink_latency": { 2024-02-02T15:46:51.606 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:51.606 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:51.607 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:51.607 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:51.607 INFO:teuthology.orchestra.run.smithi001.stdout: "req_unlink_latency": { 2024-02-02T15:46:51.607 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:51.607 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:51.607 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:51.607 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:51.607 INFO:teuthology.orchestra.run.smithi001.stdout: "cap_revoke_eviction": 0, 2024-02-02T15:46:51.607 INFO:teuthology.orchestra.run.smithi001.stdout: "cap_acquisition_throttle": 0, 2024-02-02T15:46:51.608 INFO:teuthology.orchestra.run.smithi001.stdout: "req_getvxattr_latency": { 2024-02-02T15:46:51.608 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:51.608 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:51.608 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:51.608 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:51.608 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:51.608 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_sessions": { 2024-02-02T15:46:51.608 INFO:teuthology.orchestra.run.smithi001.stdout: "session_count": 2, 2024-02-02T15:46:51.609 INFO:teuthology.orchestra.run.smithi001.stdout: "session_add": 3, 2024-02-02T15:46:51.609 INFO:teuthology.orchestra.run.smithi001.stdout: "session_remove": 1, 2024-02-02T15:46:51.609 INFO:teuthology.orchestra.run.smithi001.stdout: "sessions_open": 2, 2024-02-02T15:46:51.609 INFO:teuthology.orchestra.run.smithi001.stdout: "sessions_stale": 0, 2024-02-02T15:46:51.609 INFO:teuthology.orchestra.run.smithi001.stdout: "total_load": 0, 2024-02-02T15:46:51.609 INFO:teuthology.orchestra.run.smithi001.stdout: "average_load": 0, 2024-02-02T15:46:51.609 INFO:teuthology.orchestra.run.smithi001.stdout: "avg_session_uptime": 3830, 2024-02-02T15:46:51.609 INFO:teuthology.orchestra.run.smithi001.stdout: "mdthresh_evicted": 0 2024-02-02T15:46:51.610 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:51.610 INFO:teuthology.orchestra.run.smithi001.stdout: "mempool": { 2024-02-02T15:46:51.610 INFO:teuthology.orchestra.run.smithi001.stdout: "bloom_filter_bytes": 0, 2024-02-02T15:46:51.610 INFO:teuthology.orchestra.run.smithi001.stdout: "bloom_filter_items": 0, 2024-02-02T15:46:51.610 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_alloc_bytes": 0, 2024-02-02T15:46:51.610 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_alloc_items": 0, 2024-02-02T15:46:51.610 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_data_bytes": 0, 2024-02-02T15:46:51.610 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_data_items": 0, 2024-02-02T15:46:51.610 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-02T15:46:51.611 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_onode_items": 0, 2024-02-02T15:46:51.611 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-02T15:46:51.611 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_meta_items": 0, 2024-02-02T15:46:51.611 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_other_bytes": 0, 2024-02-02T15:46:51.611 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_other_items": 0, 2024-02-02T15:46:51.611 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-02T15:46:51.611 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_buffer_items": 0, 2024-02-02T15:46:51.611 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_extent_bytes": 0, 2024-02-02T15:46:51.612 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_extent_items": 0, 2024-02-02T15:46:51.612 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_blob_bytes": 0, 2024-02-02T15:46:51.612 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_blob_items": 0, 2024-02-02T15:46:51.612 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-02T15:46:51.612 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_shared_blob_items": 0, 2024-02-02T15:46:51.612 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-02T15:46:51.612 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_inline_bl_items": 0, 2024-02-02T15:46:51.612 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_bytes": 0, 2024-02-02T15:46:51.612 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_items": 0, 2024-02-02T15:46:51.613 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_txc_bytes": 0, 2024-02-02T15:46:51.613 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_txc_items": 0, 2024-02-02T15:46:51.613 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-02T15:46:51.613 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_deferred_items": 0, 2024-02-02T15:46:51.613 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_bytes": 0, 2024-02-02T15:46:51.613 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_items": 0, 2024-02-02T15:46:51.613 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_bytes": 0, 2024-02-02T15:46:51.613 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_items": 0, 2024-02-02T15:46:51.613 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_reader_bytes": 0, 2024-02-02T15:46:51.614 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_reader_items": 0, 2024-02-02T15:46:51.614 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_writer_bytes": 0, 2024-02-02T15:46:51.614 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_writer_items": 0, 2024-02-02T15:46:51.614 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_anon_bytes": 273122, 2024-02-02T15:46:51.614 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_anon_items": 1028, 2024-02-02T15:46:51.614 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_meta_bytes": 0, 2024-02-02T15:46:51.614 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_meta_items": 0, 2024-02-02T15:46:51.614 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_bytes": 0, 2024-02-02T15:46:51.615 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_items": 0, 2024-02-02T15:46:51.615 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mapbl_bytes": 0, 2024-02-02T15:46:51.615 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mapbl_items": 0, 2024-02-02T15:46:51.615 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pglog_bytes": 0, 2024-02-02T15:46:51.615 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pglog_items": 0, 2024-02-02T15:46:51.615 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_bytes": 9608, 2024-02-02T15:46:51.615 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_items": 184, 2024-02-02T15:46:51.615 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_mapping_bytes": 0, 2024-02-02T15:46:51.615 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_mapping_items": 0, 2024-02-02T15:46:51.616 INFO:teuthology.orchestra.run.smithi001.stdout: "pgmap_bytes": 0, 2024-02-02T15:46:51.616 INFO:teuthology.orchestra.run.smithi001.stdout: "pgmap_items": 0, 2024-02-02T15:46:51.616 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_co_bytes": 14700077, 2024-02-02T15:46:51.616 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_co_items": 94501, 2024-02-02T15:46:51.616 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_1_bytes": 0, 2024-02-02T15:46:51.616 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_1_items": 0, 2024-02-02T15:46:51.616 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_2_bytes": 0, 2024-02-02T15:46:51.616 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_2_items": 0 2024-02-02T15:46:51.616 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:51.617 INFO:teuthology.orchestra.run.smithi001.stdout: "objecter": { 2024-02-02T15:46:51.617 INFO:teuthology.orchestra.run.smithi001.stdout: "op_active": 0, 2024-02-02T15:46:51.617 INFO:teuthology.orchestra.run.smithi001.stdout: "op_laggy": 0, 2024-02-02T15:46:51.617 INFO:teuthology.orchestra.run.smithi001.stdout: "op_send": 4937, 2024-02-02T15:46:51.617 INFO:teuthology.orchestra.run.smithi001.stdout: "op_send_bytes": 21, 2024-02-02T15:46:51.617 INFO:teuthology.orchestra.run.smithi001.stdout: "op_resend": 3, 2024-02-02T15:46:51.617 INFO:teuthology.orchestra.run.smithi001.stdout: "op_reply": 4937, 2024-02-02T15:46:51.617 INFO:teuthology.orchestra.run.smithi001.stdout: "op_latency": { 2024-02-02T15:46:51.618 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 4937, 2024-02-02T15:46:51.618 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 9.629881277, 2024-02-02T15:46:51.618 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.001950553 2024-02-02T15:46:51.618 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:51.618 INFO:teuthology.orchestra.run.smithi001.stdout: "op_inflight": 0, 2024-02-02T15:46:51.618 INFO:teuthology.orchestra.run.smithi001.stdout: "oplen_avg": { 2024-02-02T15:46:51.618 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 4937, 2024-02-02T15:46:51.618 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 4938 2024-02-02T15:46:51.618 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:51.619 INFO:teuthology.orchestra.run.smithi001.stdout: "op": 4937, 2024-02-02T15:46:51.619 INFO:teuthology.orchestra.run.smithi001.stdout: "op_r": 4937, 2024-02-02T15:46:51.619 INFO:teuthology.orchestra.run.smithi001.stdout: "op_w": 0, 2024-02-02T15:46:51.619 INFO:teuthology.orchestra.run.smithi001.stdout: "op_rmw": 0, 2024-02-02T15:46:51.619 INFO:teuthology.orchestra.run.smithi001.stdout: "op_pg": 0, 2024-02-02T15:46:51.619 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_stat": 2727, 2024-02-02T15:46:51.619 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_create": 0, 2024-02-02T15:46:51.619 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_read": 2208, 2024-02-02T15:46:51.620 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_write": 0, 2024-02-02T15:46:51.620 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_writefull": 0, 2024-02-02T15:46:51.620 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_writesame": 0, 2024-02-02T15:46:51.620 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_append": 0, 2024-02-02T15:46:51.620 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_zero": 0, 2024-02-02T15:46:51.620 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_truncate": 0, 2024-02-02T15:46:51.620 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_delete": 0, 2024-02-02T15:46:51.620 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_mapext": 0, 2024-02-02T15:46:51.620 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_sparse_read": 0, 2024-02-02T15:46:51.621 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_clonerange": 0, 2024-02-02T15:46:51.621 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_getxattr": 1, 2024-02-02T15:46:51.621 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_setxattr": 0, 2024-02-02T15:46:51.621 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_cmpxattr": 0, 2024-02-02T15:46:51.621 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_rmxattr": 0, 2024-02-02T15:46:51.621 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_resetxattrs": 0, 2024-02-02T15:46:51.621 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_call": 0, 2024-02-02T15:46:51.621 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_watch": 0, 2024-02-02T15:46:51.621 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_notify": 0, 2024-02-02T15:46:51.622 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_src_cmpxattr": 0, 2024-02-02T15:46:51.622 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_pgls": 0, 2024-02-02T15:46:51.622 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_pgls_filter": 0, 2024-02-02T15:46:51.622 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_other": 0, 2024-02-02T15:46:51.622 INFO:teuthology.orchestra.run.smithi001.stdout: "linger_active": 0, 2024-02-02T15:46:51.622 INFO:teuthology.orchestra.run.smithi001.stdout: "linger_send": 0, 2024-02-02T15:46:51.622 INFO:teuthology.orchestra.run.smithi001.stdout: "linger_resend": 0, 2024-02-02T15:46:51.622 INFO:teuthology.orchestra.run.smithi001.stdout: "linger_ping": 0, 2024-02-02T15:46:51.622 INFO:teuthology.orchestra.run.smithi001.stdout: "poolop_active": 0, 2024-02-02T15:46:51.623 INFO:teuthology.orchestra.run.smithi001.stdout: "poolop_send": 0, 2024-02-02T15:46:51.623 INFO:teuthology.orchestra.run.smithi001.stdout: "poolop_resend": 0, 2024-02-02T15:46:51.623 INFO:teuthology.orchestra.run.smithi001.stdout: "poolstat_active": 0, 2024-02-02T15:46:51.623 INFO:teuthology.orchestra.run.smithi001.stdout: "poolstat_send": 0, 2024-02-02T15:46:51.623 INFO:teuthology.orchestra.run.smithi001.stdout: "poolstat_resend": 0, 2024-02-02T15:46:51.623 INFO:teuthology.orchestra.run.smithi001.stdout: "statfs_active": 0, 2024-02-02T15:46:51.623 INFO:teuthology.orchestra.run.smithi001.stdout: "statfs_send": 0, 2024-02-02T15:46:51.623 INFO:teuthology.orchestra.run.smithi001.stdout: "statfs_resend": 0, 2024-02-02T15:46:51.623 INFO:teuthology.orchestra.run.smithi001.stdout: "command_active": 0, 2024-02-02T15:46:51.624 INFO:teuthology.orchestra.run.smithi001.stdout: "command_send": 0, 2024-02-02T15:46:51.624 INFO:teuthology.orchestra.run.smithi001.stdout: "command_resend": 0, 2024-02-02T15:46:51.624 INFO:teuthology.orchestra.run.smithi001.stdout: "map_epoch": 115, 2024-02-02T15:46:51.624 INFO:teuthology.orchestra.run.smithi001.stdout: "map_full": 0, 2024-02-02T15:46:51.624 INFO:teuthology.orchestra.run.smithi001.stdout: "map_inc": 31, 2024-02-02T15:46:51.624 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_sessions": 11, 2024-02-02T15:46:51.624 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_session_open": 11, 2024-02-02T15:46:51.624 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_session_close": 0, 2024-02-02T15:46:51.624 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_laggy": 0, 2024-02-02T15:46:51.625 INFO:teuthology.orchestra.run.smithi001.stdout: "omap_wr": 0, 2024-02-02T15:46:51.625 INFO:teuthology.orchestra.run.smithi001.stdout: "omap_rd": 2, 2024-02-02T15:46:51.625 INFO:teuthology.orchestra.run.smithi001.stdout: "omap_del": 0 2024-02-02T15:46:51.625 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:51.625 INFO:teuthology.orchestra.run.smithi001.stdout: "oft": { 2024-02-02T15:46:51.625 INFO:teuthology.orchestra.run.smithi001.stdout: "omap_total_objs": 0, 2024-02-02T15:46:51.625 INFO:teuthology.orchestra.run.smithi001.stdout: "omap_total_kv_pairs": 0, 2024-02-02T15:46:51.625 INFO:teuthology.orchestra.run.smithi001.stdout: "omap_total_updates": 0, 2024-02-02T15:46:51.625 INFO:teuthology.orchestra.run.smithi001.stdout: "omap_total_removes": 0 2024-02-02T15:46:51.626 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:51.626 INFO:teuthology.orchestra.run.smithi001.stdout: "osd-slow-ops": { 2024-02-02T15:46:51.626 INFO:teuthology.orchestra.run.smithi001.stdout: "slow_ops_count": 0 2024-02-02T15:46:51.626 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:51.626 INFO:teuthology.orchestra.run.smithi001.stdout: "purge_queue": { 2024-02-02T15:46:51.626 INFO:teuthology.orchestra.run.smithi001.stdout: "pq_executing_ops": 0, 2024-02-02T15:46:51.626 INFO:teuthology.orchestra.run.smithi001.stdout: "pq_executing_ops_high_water": 0, 2024-02-02T15:46:51.626 INFO:teuthology.orchestra.run.smithi001.stdout: "pq_executing": 0, 2024-02-02T15:46:51.627 INFO:teuthology.orchestra.run.smithi001.stdout: "pq_executing_high_water": 0, 2024-02-02T15:46:51.627 INFO:teuthology.orchestra.run.smithi001.stdout: "pq_executed": 0, 2024-02-02T15:46:51.627 INFO:teuthology.orchestra.run.smithi001.stdout: "pq_item_in_journal": 0 2024-02-02T15:46:51.627 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:51.627 INFO:teuthology.orchestra.run.smithi001.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-02T15:46:51.627 INFO:teuthology.orchestra.run.smithi001.stdout: "val": 0, 2024-02-02T15:46:51.627 INFO:teuthology.orchestra.run.smithi001.stdout: "max": 104857600, 2024-02-02T15:46:51.627 INFO:teuthology.orchestra.run.smithi001.stdout: "get_started": 0, 2024-02-02T15:46:51.627 INFO:teuthology.orchestra.run.smithi001.stdout: "get": 5396, 2024-02-02T15:46:51.628 INFO:teuthology.orchestra.run.smithi001.stdout: "get_sum": 409184108, 2024-02-02T15:46:51.628 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_fail": 0, 2024-02-02T15:46:51.628 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_success": 5396, 2024-02-02T15:46:51.628 INFO:teuthology.orchestra.run.smithi001.stdout: "take": 0, 2024-02-02T15:46:51.628 INFO:teuthology.orchestra.run.smithi001.stdout: "take_sum": 0, 2024-02-02T15:46:51.628 INFO:teuthology.orchestra.run.smithi001.stdout: "put": 5396, 2024-02-02T15:46:51.628 INFO:teuthology.orchestra.run.smithi001.stdout: "put_sum": 409184108, 2024-02-02T15:46:51.628 INFO:teuthology.orchestra.run.smithi001.stdout: "wait": { 2024-02-02T15:46:51.629 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:51.629 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:51.629 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:51.629 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:51.629 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:51.629 INFO:teuthology.orchestra.run.smithi001.stdout: "throttle-objecter_bytes": { 2024-02-02T15:46:51.629 INFO:teuthology.orchestra.run.smithi001.stdout: "val": 0, 2024-02-02T15:46:51.629 INFO:teuthology.orchestra.run.smithi001.stdout: "max": 104857600, 2024-02-02T15:46:51.629 INFO:teuthology.orchestra.run.smithi001.stdout: "get_started": 0, 2024-02-02T15:46:51.630 INFO:teuthology.orchestra.run.smithi001.stdout: "get": 0, 2024-02-02T15:46:51.630 INFO:teuthology.orchestra.run.smithi001.stdout: "get_sum": 0, 2024-02-02T15:46:51.630 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_fail": 0, 2024-02-02T15:46:51.630 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_success": 0, 2024-02-02T15:46:51.630 INFO:teuthology.orchestra.run.smithi001.stdout: "take": 4937, 2024-02-02T15:46:51.630 INFO:teuthology.orchestra.run.smithi001.stdout: "take_sum": 406101459, 2024-02-02T15:46:51.630 INFO:teuthology.orchestra.run.smithi001.stdout: "put": 4937, 2024-02-02T15:46:51.630 INFO:teuthology.orchestra.run.smithi001.stdout: "put_sum": 406101459, 2024-02-02T15:46:51.630 INFO:teuthology.orchestra.run.smithi001.stdout: "wait": { 2024-02-02T15:46:51.631 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:51.631 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:51.631 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:51.631 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:51.631 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:51.631 INFO:teuthology.orchestra.run.smithi001.stdout: "throttle-objecter_ops": { 2024-02-02T15:46:51.631 INFO:teuthology.orchestra.run.smithi001.stdout: "val": 0, 2024-02-02T15:46:51.631 INFO:teuthology.orchestra.run.smithi001.stdout: "max": 1024, 2024-02-02T15:46:51.632 INFO:teuthology.orchestra.run.smithi001.stdout: "get_started": 0, 2024-02-02T15:46:51.632 INFO:teuthology.orchestra.run.smithi001.stdout: "get": 0, 2024-02-02T15:46:51.632 INFO:teuthology.orchestra.run.smithi001.stdout: "get_sum": 0, 2024-02-02T15:46:51.632 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_fail": 0, 2024-02-02T15:46:51.632 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_success": 0, 2024-02-02T15:46:51.632 INFO:teuthology.orchestra.run.smithi001.stdout: "take": 4937, 2024-02-02T15:46:51.632 INFO:teuthology.orchestra.run.smithi001.stdout: "take_sum": 4937, 2024-02-02T15:46:51.632 INFO:teuthology.orchestra.run.smithi001.stdout: "put": 4937, 2024-02-02T15:46:51.633 INFO:teuthology.orchestra.run.smithi001.stdout: "put_sum": 4937, 2024-02-02T15:46:51.633 INFO:teuthology.orchestra.run.smithi001.stdout: "wait": { 2024-02-02T15:46:51.633 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:51.633 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:51.633 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:51.633 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:51.633 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:51.633 INFO:teuthology.orchestra.run.smithi001.stdout: "throttle-write_buf_throttle": { 2024-02-02T15:46:51.633 INFO:teuthology.orchestra.run.smithi001.stdout: "val": 0, 2024-02-02T15:46:51.634 INFO:teuthology.orchestra.run.smithi001.stdout: "max": 3758096384, 2024-02-02T15:46:51.634 INFO:teuthology.orchestra.run.smithi001.stdout: "get_started": 0, 2024-02-02T15:46:51.634 INFO:teuthology.orchestra.run.smithi001.stdout: "get": 0, 2024-02-02T15:46:51.634 INFO:teuthology.orchestra.run.smithi001.stdout: "get_sum": 0, 2024-02-02T15:46:51.634 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_fail": 0, 2024-02-02T15:46:51.634 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_success": 0, 2024-02-02T15:46:51.634 INFO:teuthology.orchestra.run.smithi001.stdout: "take": 0, 2024-02-02T15:46:51.634 INFO:teuthology.orchestra.run.smithi001.stdout: "take_sum": 0, 2024-02-02T15:46:51.634 INFO:teuthology.orchestra.run.smithi001.stdout: "put": 0, 2024-02-02T15:46:51.635 INFO:teuthology.orchestra.run.smithi001.stdout: "put_sum": 0, 2024-02-02T15:46:51.635 INFO:teuthology.orchestra.run.smithi001.stdout: "wait": { 2024-02-02T15:46:51.635 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:51.635 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:51.635 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:51.635 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:51.635 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:51.635 INFO:teuthology.orchestra.run.smithi001.stdout: "throttle-write_buf_throttle-0x56363672c1e0": { 2024-02-02T15:46:51.635 INFO:teuthology.orchestra.run.smithi001.stdout: "val": 0, 2024-02-02T15:46:51.636 INFO:teuthology.orchestra.run.smithi001.stdout: "max": 3758096384, 2024-02-02T15:46:51.636 INFO:teuthology.orchestra.run.smithi001.stdout: "get_started": 0, 2024-02-02T15:46:51.636 INFO:teuthology.orchestra.run.smithi001.stdout: "get": 0, 2024-02-02T15:46:51.636 INFO:teuthology.orchestra.run.smithi001.stdout: "get_sum": 0, 2024-02-02T15:46:51.636 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_fail": 0, 2024-02-02T15:46:51.636 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_success": 0, 2024-02-02T15:46:51.636 INFO:teuthology.orchestra.run.smithi001.stdout: "take": 0, 2024-02-02T15:46:51.636 INFO:teuthology.orchestra.run.smithi001.stdout: "take_sum": 0, 2024-02-02T15:46:51.636 INFO:teuthology.orchestra.run.smithi001.stdout: "put": 0, 2024-02-02T15:46:51.637 INFO:teuthology.orchestra.run.smithi001.stdout: "put_sum": 0, 2024-02-02T15:46:51.637 INFO:teuthology.orchestra.run.smithi001.stdout: "wait": { 2024-02-02T15:46:51.637 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:51.637 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:51.637 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:51.637 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:51.637 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:51.637 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:46:51.638 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:46:51 smithi001 ceph-mon[35470]: pgmap v624: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 37 KiB/s wr, 11 op/s 2024-02-02T15:46:51.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:46:51 smithi071 ceph-mon[44707]: pgmap v624: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 37 KiB/s wr, 11 op/s 2024-02-02T15:46:51.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:46:51 smithi142 ceph-mon[45728]: pgmap v624: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 37 KiB/s wr, 11 op/s 2024-02-02T15:46:52.061 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-02-02T15:46:52.061 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-02-02T15:46:52.062 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=0 2024-02-02T15:46:52.062 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=0 2024-02-02T15:46:52.062 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=0 2024-02-02T15:46:52.062 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-02-02T15:46:52.062 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-02-02T15:46:52.062 DEBUG:tasks.check_counter:Getting stats from d 2024-02-02T15:46:52.062 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph daemon mds.d perf dump 2024-02-02T15:46:52.433 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.a/config 2024-02-02T15:46:52.884 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:46:52 smithi001 ceph-mon[35470]: pgmap v625: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 117 KiB/s wr, 16 op/s 2024-02-02T15:46:53.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:46:52 smithi071 ceph-mon[44707]: pgmap v625: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 117 KiB/s wr, 16 op/s 2024-02-02T15:46:53.169 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:46:52 smithi142 ceph-mon[45728]: pgmap v625: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 117 KiB/s wr, 16 op/s 2024-02-02T15:46:53.607 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:46:53.608 INFO:teuthology.orchestra.run.smithi001.stdout: "AsyncMessenger::Worker-0": { 2024-02-02T15:46:53.608 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_messages": 371, 2024-02-02T15:46:53.608 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages": 358, 2024-02-02T15:46:53.608 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_bytes": 1862880, 2024-02-02T15:46:53.608 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_bytes": 205456, 2024-02-02T15:46:53.608 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_created_connections": 4, 2024-02-02T15:46:53.609 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_active_connections": 0, 2024-02-02T15:46:53.609 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_total_time": 0.081537501, 2024-02-02T15:46:53.609 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_send_time": 0.030289646, 2024-02-02T15:46:53.609 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_recv_time": 0.049855699, 2024-02-02T15:46:53.609 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_fast_dispatch_time": 0.008262937, 2024-02-02T15:46:53.609 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages_queue_lat": { 2024-02-02T15:46:53.609 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 357, 2024-02-02T15:46:53.609 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.030959939, 2024-02-02T15:46:53.609 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000086722 2024-02-02T15:46:53.609 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:53.610 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_handle_ack_lat": { 2024-02-02T15:46:53.610 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:53.610 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:53.610 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:53.610 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:53.610 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_encrypted_bytes": 1862880, 2024-02-02T15:46:53.610 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_encrypted_bytes": 205456 2024-02-02T15:46:53.610 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:53.610 INFO:teuthology.orchestra.run.smithi001.stdout: "AsyncMessenger::Worker-1": { 2024-02-02T15:46:53.611 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_messages": 3, 2024-02-02T15:46:53.611 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages": 275, 2024-02-02T15:46:53.611 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_bytes": 480, 2024-02-02T15:46:53.611 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_bytes": 276832, 2024-02-02T15:46:53.611 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_created_connections": 6, 2024-02-02T15:46:53.611 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_active_connections": 3, 2024-02-02T15:46:53.611 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_total_time": 0.029217698, 2024-02-02T15:46:53.611 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_send_time": 0.020752715, 2024-02-02T15:46:53.611 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_recv_time": 0.003693166, 2024-02-02T15:46:53.612 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-02T15:46:53.612 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages_queue_lat": { 2024-02-02T15:46:53.612 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 272, 2024-02-02T15:46:53.612 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.036578124, 2024-02-02T15:46:53.612 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000134478 2024-02-02T15:46:53.612 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:53.612 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_handle_ack_lat": { 2024-02-02T15:46:53.612 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:53.613 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:53.613 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:53.613 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:53.613 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_encrypted_bytes": 480, 2024-02-02T15:46:53.613 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_encrypted_bytes": 276832 2024-02-02T15:46:53.613 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:53.613 INFO:teuthology.orchestra.run.smithi001.stdout: "AsyncMessenger::Worker-2": { 2024-02-02T15:46:53.613 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_messages": 4, 2024-02-02T15:46:53.613 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages": 4, 2024-02-02T15:46:53.614 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_bytes": 2992, 2024-02-02T15:46:53.614 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_bytes": 448, 2024-02-02T15:46:53.614 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_created_connections": 2, 2024-02-02T15:46:53.614 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_active_connections": 1, 2024-02-02T15:46:53.614 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_total_time": 0.001332996, 2024-02-02T15:46:53.614 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_send_time": 0.000067979, 2024-02-02T15:46:53.614 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_recv_time": 0.001164172, 2024-02-02T15:46:53.615 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-02T15:46:53.615 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages_queue_lat": { 2024-02-02T15:46:53.615 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 3, 2024-02-02T15:46:53.615 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.001211955, 2024-02-02T15:46:53.615 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000403985 2024-02-02T15:46:53.615 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:53.615 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_handle_ack_lat": { 2024-02-02T15:46:53.615 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:53.616 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:53.616 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:53.616 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:53.616 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-02-02T15:46:53.616 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_encrypted_bytes": 448 2024-02-02T15:46:53.616 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:53.616 INFO:teuthology.orchestra.run.smithi001.stdout: "cct": { 2024-02-02T15:46:53.617 INFO:teuthology.orchestra.run.smithi001.stdout: "total_workers": 0, 2024-02-02T15:46:53.617 INFO:teuthology.orchestra.run.smithi001.stdout: "unhealthy_workers": 0 2024-02-02T15:46:53.617 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:53.617 INFO:teuthology.orchestra.run.smithi001.stdout: "mempool": { 2024-02-02T15:46:53.617 INFO:teuthology.orchestra.run.smithi001.stdout: "bloom_filter_bytes": 0, 2024-02-02T15:46:53.617 INFO:teuthology.orchestra.run.smithi001.stdout: "bloom_filter_items": 0, 2024-02-02T15:46:53.618 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_alloc_bytes": 0, 2024-02-02T15:46:53.618 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_alloc_items": 0, 2024-02-02T15:46:53.618 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_data_bytes": 0, 2024-02-02T15:46:53.618 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_data_items": 0, 2024-02-02T15:46:53.618 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-02T15:46:53.618 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_onode_items": 0, 2024-02-02T15:46:53.618 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-02T15:46:53.618 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_meta_items": 0, 2024-02-02T15:46:53.619 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_other_bytes": 0, 2024-02-02T15:46:53.619 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_other_items": 0, 2024-02-02T15:46:53.619 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-02T15:46:53.619 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_buffer_items": 0, 2024-02-02T15:46:53.619 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_extent_bytes": 0, 2024-02-02T15:46:53.619 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_extent_items": 0, 2024-02-02T15:46:53.619 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_blob_bytes": 0, 2024-02-02T15:46:53.619 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_blob_items": 0, 2024-02-02T15:46:53.619 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-02T15:46:53.620 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_shared_blob_items": 0, 2024-02-02T15:46:53.620 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-02T15:46:53.620 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_inline_bl_items": 0, 2024-02-02T15:46:53.620 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_bytes": 0, 2024-02-02T15:46:53.620 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_items": 0, 2024-02-02T15:46:53.620 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_txc_bytes": 0, 2024-02-02T15:46:53.620 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_txc_items": 0, 2024-02-02T15:46:53.621 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-02T15:46:53.621 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_deferred_items": 0, 2024-02-02T15:46:53.621 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_bytes": 0, 2024-02-02T15:46:53.621 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_items": 0, 2024-02-02T15:46:53.621 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_bytes": 0, 2024-02-02T15:46:53.621 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_items": 0, 2024-02-02T15:46:53.621 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_reader_bytes": 0, 2024-02-02T15:46:53.621 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_reader_items": 0, 2024-02-02T15:46:53.622 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_writer_bytes": 0, 2024-02-02T15:46:53.622 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_writer_items": 0, 2024-02-02T15:46:53.622 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_anon_bytes": 148714, 2024-02-02T15:46:53.622 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_anon_items": 30, 2024-02-02T15:46:53.622 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_meta_bytes": 0, 2024-02-02T15:46:53.622 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_meta_items": 0, 2024-02-02T15:46:53.622 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_bytes": 0, 2024-02-02T15:46:53.622 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_items": 0, 2024-02-02T15:46:53.623 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mapbl_bytes": 0, 2024-02-02T15:46:53.623 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mapbl_items": 0, 2024-02-02T15:46:53.623 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pglog_bytes": 0, 2024-02-02T15:46:53.623 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pglog_items": 0, 2024-02-02T15:46:53.623 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_bytes": 0, 2024-02-02T15:46:53.623 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_items": 0, 2024-02-02T15:46:53.623 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_mapping_bytes": 0, 2024-02-02T15:46:53.623 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_mapping_items": 0, 2024-02-02T15:46:53.624 INFO:teuthology.orchestra.run.smithi001.stdout: "pgmap_bytes": 0, 2024-02-02T15:46:53.624 INFO:teuthology.orchestra.run.smithi001.stdout: "pgmap_items": 0, 2024-02-02T15:46:53.624 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_co_bytes": 928, 2024-02-02T15:46:53.624 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_co_items": 2, 2024-02-02T15:46:53.624 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_1_bytes": 0, 2024-02-02T15:46:53.624 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_1_items": 0, 2024-02-02T15:46:53.624 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_2_bytes": 0, 2024-02-02T15:46:53.624 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_2_items": 0 2024-02-02T15:46:53.625 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:53.625 INFO:teuthology.orchestra.run.smithi001.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-02T15:46:53.625 INFO:teuthology.orchestra.run.smithi001.stdout: "val": 0, 2024-02-02T15:46:53.625 INFO:teuthology.orchestra.run.smithi001.stdout: "max": 104857600, 2024-02-02T15:46:53.625 INFO:teuthology.orchestra.run.smithi001.stdout: "get_started": 0, 2024-02-02T15:46:53.625 INFO:teuthology.orchestra.run.smithi001.stdout: "get": 374, 2024-02-02T15:46:53.626 INFO:teuthology.orchestra.run.smithi001.stdout: "get_sum": 1810543, 2024-02-02T15:46:53.626 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_fail": 0, 2024-02-02T15:46:53.626 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_success": 374, 2024-02-02T15:46:53.626 INFO:teuthology.orchestra.run.smithi001.stdout: "take": 0, 2024-02-02T15:46:53.626 INFO:teuthology.orchestra.run.smithi001.stdout: "take_sum": 0, 2024-02-02T15:46:53.626 INFO:teuthology.orchestra.run.smithi001.stdout: "put": 374, 2024-02-02T15:46:53.626 INFO:teuthology.orchestra.run.smithi001.stdout: "put_sum": 1810543, 2024-02-02T15:46:53.626 INFO:teuthology.orchestra.run.smithi001.stdout: "wait": { 2024-02-02T15:46:53.627 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:53.627 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:53.627 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:53.627 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:53.627 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:53.627 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:46:54.099 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-02-02T15:46:54.099 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-02-02T15:46:54.100 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-02-02T15:46:54.100 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-02-02T15:46:54.100 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-02-02T15:46:54.100 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-02-02T15:46:54.100 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-02-02T15:46:54.100 DEBUG:tasks.check_counter:Getting stats from g 2024-02-02T15:46:54.100 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph daemon mds.g perf dump 2024-02-02T15:46:54.479 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.a/config 2024-02-02T15:46:55.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:46:55 smithi001 ceph-mon[35470]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:46:55.440 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:46:55 smithi001 ceph-mon[35470]: pgmap v626: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 95 KiB/s wr, 13 op/s 2024-02-02T15:46:55.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:46:55 smithi071 ceph-mon[44707]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:46:55.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:46:55 smithi071 ceph-mon[44707]: pgmap v626: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 95 KiB/s wr, 13 op/s 2024-02-02T15:46:55.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:46:55 smithi142 ceph-mon[45728]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:46:55.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:46:55 smithi142 ceph-mon[45728]: pgmap v626: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 95 KiB/s wr, 13 op/s 2024-02-02T15:46:56.331 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:46:56.331 INFO:teuthology.orchestra.run.smithi001.stdout: "AsyncMessenger::Worker-0": { 2024-02-02T15:46:56.331 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_messages": 2576, 2024-02-02T15:46:56.332 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages": 2506, 2024-02-02T15:46:56.332 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_bytes": 103557770, 2024-02-02T15:46:56.332 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_bytes": 867733, 2024-02-02T15:46:56.332 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_created_connections": 7, 2024-02-02T15:46:56.332 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_active_connections": 3, 2024-02-02T15:46:56.332 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_total_time": 0.537237323, 2024-02-02T15:46:56.332 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_send_time": 0.139078763, 2024-02-02T15:46:56.332 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_recv_time": 0.303624554, 2024-02-02T15:46:56.333 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_fast_dispatch_time": 0.083051560, 2024-02-02T15:46:56.333 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages_queue_lat": { 2024-02-02T15:46:56.333 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 2505, 2024-02-02T15:46:56.333 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.170655377, 2024-02-02T15:46:56.333 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000068125 2024-02-02T15:46:56.333 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:56.333 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_handle_ack_lat": { 2024-02-02T15:46:56.333 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:56.333 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:56.334 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:56.334 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:56.334 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_encrypted_bytes": 2243088, 2024-02-02T15:46:56.334 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_encrypted_bytes": 209968 2024-02-02T15:46:56.334 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:56.334 INFO:teuthology.orchestra.run.smithi001.stdout: "AsyncMessenger::Worker-1": { 2024-02-02T15:46:56.334 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_messages": 1698, 2024-02-02T15:46:56.334 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages": 1966, 2024-02-02T15:46:56.335 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_bytes": 113680481, 2024-02-02T15:46:56.335 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_bytes": 1149133, 2024-02-02T15:46:56.335 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_created_connections": 9, 2024-02-02T15:46:56.335 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_active_connections": 7, 2024-02-02T15:46:56.335 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_total_time": 0.400522384, 2024-02-02T15:46:56.335 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_send_time": 0.063453559, 2024-02-02T15:46:56.335 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_recv_time": 0.265182272, 2024-02-02T15:46:56.335 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_fast_dispatch_time": 0.054117222, 2024-02-02T15:46:56.335 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages_queue_lat": { 2024-02-02T15:46:56.336 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 1964, 2024-02-02T15:46:56.336 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.110357048, 2024-02-02T15:46:56.336 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000056189 2024-02-02T15:46:56.336 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:56.336 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_handle_ack_lat": { 2024-02-02T15:46:56.336 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:56.336 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:56.336 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:56.337 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:56.337 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-02-02T15:46:56.337 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_encrypted_bytes": 622864 2024-02-02T15:46:56.337 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:56.337 INFO:teuthology.orchestra.run.smithi001.stdout: "AsyncMessenger::Worker-2": { 2024-02-02T15:46:56.337 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_messages": 1243, 2024-02-02T15:46:56.337 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages": 1245, 2024-02-02T15:46:56.337 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_bytes": 213580038, 2024-02-02T15:46:56.337 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_bytes": 386585, 2024-02-02T15:46:56.338 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_created_connections": 7, 2024-02-02T15:46:56.338 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_active_connections": 5, 2024-02-02T15:46:56.338 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_total_time": 0.552591578, 2024-02-02T15:46:56.338 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_send_time": 0.039027065, 2024-02-02T15:46:56.338 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_recv_time": 0.445811452, 2024-02-02T15:46:56.338 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_fast_dispatch_time": 0.056171728, 2024-02-02T15:46:56.338 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages_queue_lat": { 2024-02-02T15:46:56.338 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 1243, 2024-02-02T15:46:56.338 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.088195058, 2024-02-02T15:46:56.339 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000070953 2024-02-02T15:46:56.339 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:56.339 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_handle_ack_lat": { 2024-02-02T15:46:56.339 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:56.339 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:56.339 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:56.339 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:56.339 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_encrypted_bytes": 0, 2024-02-02T15:46:56.339 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_encrypted_bytes": 0 2024-02-02T15:46:56.340 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:56.340 INFO:teuthology.orchestra.run.smithi001.stdout: "cct": { 2024-02-02T15:46:56.340 INFO:teuthology.orchestra.run.smithi001.stdout: "total_workers": 1, 2024-02-02T15:46:56.340 INFO:teuthology.orchestra.run.smithi001.stdout: "unhealthy_workers": 0 2024-02-02T15:46:56.340 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:56.340 INFO:teuthology.orchestra.run.smithi001.stdout: "finisher-MDSRank": { 2024-02-02T15:46:56.340 INFO:teuthology.orchestra.run.smithi001.stdout: "queue_len": 0, 2024-02-02T15:46:56.340 INFO:teuthology.orchestra.run.smithi001.stdout: "complete_latency": { 2024-02-02T15:46:56.341 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 8288, 2024-02-02T15:46:56.341 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.260110663, 2024-02-02T15:46:56.341 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000031384 2024-02-02T15:46:56.341 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:56.341 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:56.341 INFO:teuthology.orchestra.run.smithi001.stdout: "finisher-PurgeQueue": { 2024-02-02T15:46:56.341 INFO:teuthology.orchestra.run.smithi001.stdout: "queue_len": 0, 2024-02-02T15:46:56.341 INFO:teuthology.orchestra.run.smithi001.stdout: "complete_latency": { 2024-02-02T15:46:56.341 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:56.342 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:56.342 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:56.342 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:56.342 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:56.342 INFO:teuthology.orchestra.run.smithi001.stdout: "mds": { 2024-02-02T15:46:56.342 INFO:teuthology.orchestra.run.smithi001.stdout: "request": 0, 2024-02-02T15:46:56.342 INFO:teuthology.orchestra.run.smithi001.stdout: "reply": 0, 2024-02-02T15:46:56.342 INFO:teuthology.orchestra.run.smithi001.stdout: "reply_latency": { 2024-02-02T15:46:56.342 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:56.342 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:56.343 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:56.343 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:56.343 INFO:teuthology.orchestra.run.smithi001.stdout: "slow_reply": 0, 2024-02-02T15:46:56.343 INFO:teuthology.orchestra.run.smithi001.stdout: "forward": 0, 2024-02-02T15:46:56.343 INFO:teuthology.orchestra.run.smithi001.stdout: "dir_fetch_complete": 0, 2024-02-02T15:46:56.343 INFO:teuthology.orchestra.run.smithi001.stdout: "dir_fetch_keys": 0, 2024-02-02T15:46:56.343 INFO:teuthology.orchestra.run.smithi001.stdout: "dir_commit": 0, 2024-02-02T15:46:56.343 INFO:teuthology.orchestra.run.smithi001.stdout: "dir_split": 0, 2024-02-02T15:46:56.343 INFO:teuthology.orchestra.run.smithi001.stdout: "dir_merge": 0, 2024-02-02T15:46:56.344 INFO:teuthology.orchestra.run.smithi001.stdout: "inodes": 16156, 2024-02-02T15:46:56.344 INFO:teuthology.orchestra.run.smithi001.stdout: "inodes_top": 47, 2024-02-02T15:46:56.344 INFO:teuthology.orchestra.run.smithi001.stdout: "inodes_bottom": 16109, 2024-02-02T15:46:56.344 INFO:teuthology.orchestra.run.smithi001.stdout: "inodes_pin_tail": 0, 2024-02-02T15:46:56.344 INFO:teuthology.orchestra.run.smithi001.stdout: "inodes_pinned": 16089, 2024-02-02T15:46:56.344 INFO:teuthology.orchestra.run.smithi001.stdout: "inodes_expired": 0, 2024-02-02T15:46:56.344 INFO:teuthology.orchestra.run.smithi001.stdout: "inodes_with_caps": 0, 2024-02-02T15:46:56.344 INFO:teuthology.orchestra.run.smithi001.stdout: "caps": 0, 2024-02-02T15:46:56.344 INFO:teuthology.orchestra.run.smithi001.stdout: "subtrees": 14, 2024-02-02T15:46:56.345 INFO:teuthology.orchestra.run.smithi001.stdout: "traverse": 0, 2024-02-02T15:46:56.345 INFO:teuthology.orchestra.run.smithi001.stdout: "traverse_hit": 0, 2024-02-02T15:46:56.345 INFO:teuthology.orchestra.run.smithi001.stdout: "traverse_forward": 0, 2024-02-02T15:46:56.345 INFO:teuthology.orchestra.run.smithi001.stdout: "traverse_discover": 0, 2024-02-02T15:46:56.345 INFO:teuthology.orchestra.run.smithi001.stdout: "traverse_dir_fetch": 0, 2024-02-02T15:46:56.345 INFO:teuthology.orchestra.run.smithi001.stdout: "traverse_remote_ino": 0, 2024-02-02T15:46:56.345 INFO:teuthology.orchestra.run.smithi001.stdout: "traverse_lock": 0, 2024-02-02T15:46:56.345 INFO:teuthology.orchestra.run.smithi001.stdout: "load_cent": 0, 2024-02-02T15:46:56.345 INFO:teuthology.orchestra.run.smithi001.stdout: "q": 0, 2024-02-02T15:46:56.345 INFO:teuthology.orchestra.run.smithi001.stdout: "exported": 0, 2024-02-02T15:46:56.346 INFO:teuthology.orchestra.run.smithi001.stdout: "exported_inodes": 0, 2024-02-02T15:46:56.346 INFO:teuthology.orchestra.run.smithi001.stdout: "imported": 0, 2024-02-02T15:46:56.346 INFO:teuthology.orchestra.run.smithi001.stdout: "imported_inodes": 0, 2024-02-02T15:46:56.346 INFO:teuthology.orchestra.run.smithi001.stdout: "openino_dir_fetch": 0, 2024-02-02T15:46:56.346 INFO:teuthology.orchestra.run.smithi001.stdout: "openino_backtrace_fetch": 0, 2024-02-02T15:46:56.346 INFO:teuthology.orchestra.run.smithi001.stdout: "openino_peer_discover": 0, 2024-02-02T15:46:56.346 INFO:teuthology.orchestra.run.smithi001.stdout: "root_rfiles": 0, 2024-02-02T15:46:56.346 INFO:teuthology.orchestra.run.smithi001.stdout: "root_rbytes": 0, 2024-02-02T15:46:56.346 INFO:teuthology.orchestra.run.smithi001.stdout: "root_rsnaps": 0, 2024-02-02T15:46:56.346 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_backtrace_fetch": 0, 2024-02-02T15:46:56.347 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_set_tag": 0, 2024-02-02T15:46:56.347 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_backtrace_repaired": 0, 2024-02-02T15:46:56.347 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_inotable_repaired": 0, 2024-02-02T15:46:56.347 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_dir_inodes": 0, 2024-02-02T15:46:56.347 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_dir_base_inodes": 0, 2024-02-02T15:46:56.347 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_dirfrag_rstats": 0, 2024-02-02T15:46:56.347 INFO:teuthology.orchestra.run.smithi001.stdout: "scrub_file_inodes": 0, 2024-02-02T15:46:56.347 INFO:teuthology.orchestra.run.smithi001.stdout: "handle_inode_file_caps": 0, 2024-02-02T15:46:56.347 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_cap_op_revoke": 0, 2024-02-02T15:46:56.348 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_cap_op_grant": 0, 2024-02-02T15:46:56.348 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_cap_op_trunc": 0, 2024-02-02T15:46:56.348 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-02-02T15:46:56.348 INFO:teuthology.orchestra.run.smithi001.stdout: "ceph_cap_op_flush_ack": 0, 2024-02-02T15:46:56.348 INFO:teuthology.orchestra.run.smithi001.stdout: "handle_client_caps": 0, 2024-02-02T15:46:56.348 INFO:teuthology.orchestra.run.smithi001.stdout: "handle_client_caps_dirty": 0, 2024-02-02T15:46:56.348 INFO:teuthology.orchestra.run.smithi001.stdout: "handle_client_cap_release": 0, 2024-02-02T15:46:56.348 INFO:teuthology.orchestra.run.smithi001.stdout: "process_request_cap_release": 0 2024-02-02T15:46:56.348 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:56.349 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_cache": { 2024-02-02T15:46:56.349 INFO:teuthology.orchestra.run.smithi001.stdout: "dir_update": 0, 2024-02-02T15:46:56.349 INFO:teuthology.orchestra.run.smithi001.stdout: "dir_update_receipt": 0, 2024-02-02T15:46:56.349 INFO:teuthology.orchestra.run.smithi001.stdout: "dir_try_discover": 0, 2024-02-02T15:46:56.349 INFO:teuthology.orchestra.run.smithi001.stdout: "dir_send_discover": 0, 2024-02-02T15:46:56.349 INFO:teuthology.orchestra.run.smithi001.stdout: "dir_handle_discover": 0, 2024-02-02T15:46:56.349 INFO:teuthology.orchestra.run.smithi001.stdout: "num_strays": 0, 2024-02-02T15:46:56.349 INFO:teuthology.orchestra.run.smithi001.stdout: "num_strays_delayed": 0, 2024-02-02T15:46:56.349 INFO:teuthology.orchestra.run.smithi001.stdout: "num_strays_enqueuing": 0, 2024-02-02T15:46:56.350 INFO:teuthology.orchestra.run.smithi001.stdout: "strays_created": 0, 2024-02-02T15:46:56.350 INFO:teuthology.orchestra.run.smithi001.stdout: "strays_enqueued": 0, 2024-02-02T15:46:56.350 INFO:teuthology.orchestra.run.smithi001.stdout: "strays_reintegrated": 0, 2024-02-02T15:46:56.350 INFO:teuthology.orchestra.run.smithi001.stdout: "strays_migrated": 0, 2024-02-02T15:46:56.350 INFO:teuthology.orchestra.run.smithi001.stdout: "num_recovering_processing": 0, 2024-02-02T15:46:56.350 INFO:teuthology.orchestra.run.smithi001.stdout: "num_recovering_enqueued": 0, 2024-02-02T15:46:56.350 INFO:teuthology.orchestra.run.smithi001.stdout: "num_recovering_prioritized": 0, 2024-02-02T15:46:56.350 INFO:teuthology.orchestra.run.smithi001.stdout: "recovery_started": 0, 2024-02-02T15:46:56.350 INFO:teuthology.orchestra.run.smithi001.stdout: "recovery_completed": 0, 2024-02-02T15:46:56.350 INFO:teuthology.orchestra.run.smithi001.stdout: "ireq_enqueue_scrub": 0, 2024-02-02T15:46:56.351 INFO:teuthology.orchestra.run.smithi001.stdout: "ireq_exportdir": 0, 2024-02-02T15:46:56.351 INFO:teuthology.orchestra.run.smithi001.stdout: "ireq_flush": 0, 2024-02-02T15:46:56.351 INFO:teuthology.orchestra.run.smithi001.stdout: "ireq_fragmentdir": 0, 2024-02-02T15:46:56.351 INFO:teuthology.orchestra.run.smithi001.stdout: "ireq_fragstats": 0, 2024-02-02T15:46:56.351 INFO:teuthology.orchestra.run.smithi001.stdout: "ireq_inodestats": 0 2024-02-02T15:46:56.351 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:56.351 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_log": { 2024-02-02T15:46:56.351 INFO:teuthology.orchestra.run.smithi001.stdout: "evadd": 0, 2024-02-02T15:46:56.351 INFO:teuthology.orchestra.run.smithi001.stdout: "evex": 0, 2024-02-02T15:46:56.351 INFO:teuthology.orchestra.run.smithi001.stdout: "evtrm": 0, 2024-02-02T15:46:56.352 INFO:teuthology.orchestra.run.smithi001.stdout: "ev": 83264, 2024-02-02T15:46:56.352 INFO:teuthology.orchestra.run.smithi001.stdout: "evexg": 0, 2024-02-02T15:46:56.352 INFO:teuthology.orchestra.run.smithi001.stdout: "evexd": 0, 2024-02-02T15:46:56.352 INFO:teuthology.orchestra.run.smithi001.stdout: "segadd": 0, 2024-02-02T15:46:56.352 INFO:teuthology.orchestra.run.smithi001.stdout: "segex": 0, 2024-02-02T15:46:56.352 INFO:teuthology.orchestra.run.smithi001.stdout: "segtrm": 0, 2024-02-02T15:46:56.352 INFO:teuthology.orchestra.run.smithi001.stdout: "seg": 125, 2024-02-02T15:46:56.352 INFO:teuthology.orchestra.run.smithi001.stdout: "segexg": 0, 2024-02-02T15:46:56.352 INFO:teuthology.orchestra.run.smithi001.stdout: "segexd": 0, 2024-02-02T15:46:56.353 INFO:teuthology.orchestra.run.smithi001.stdout: "expos": 4194304, 2024-02-02T15:46:56.353 INFO:teuthology.orchestra.run.smithi001.stdout: "wrpos": 431364875, 2024-02-02T15:46:56.353 INFO:teuthology.orchestra.run.smithi001.stdout: "rdpos": 431353282, 2024-02-02T15:46:56.353 INFO:teuthology.orchestra.run.smithi001.stdout: "jlat": { 2024-02-02T15:46:56.353 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:56.353 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:56.353 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:56.353 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:56.353 INFO:teuthology.orchestra.run.smithi001.stdout: "replayed": 83264 2024-02-02T15:46:56.354 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:56.354 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_mem": { 2024-02-02T15:46:56.354 INFO:teuthology.orchestra.run.smithi001.stdout: "ino": 1353, 2024-02-02T15:46:56.354 INFO:teuthology.orchestra.run.smithi001.stdout: "ino+": 16165, 2024-02-02T15:46:56.354 INFO:teuthology.orchestra.run.smithi001.stdout: "ino-": 14812, 2024-02-02T15:46:56.354 INFO:teuthology.orchestra.run.smithi001.stdout: "dir": 72, 2024-02-02T15:46:56.354 INFO:teuthology.orchestra.run.smithi001.stdout: "dir+": 81, 2024-02-02T15:46:56.354 INFO:teuthology.orchestra.run.smithi001.stdout: "dir-": 9, 2024-02-02T15:46:56.354 INFO:teuthology.orchestra.run.smithi001.stdout: "dn": 16156, 2024-02-02T15:46:56.354 INFO:teuthology.orchestra.run.smithi001.stdout: "dn+": 16409, 2024-02-02T15:46:56.355 INFO:teuthology.orchestra.run.smithi001.stdout: "dn-": 253, 2024-02-02T15:46:56.355 INFO:teuthology.orchestra.run.smithi001.stdout: "cap": 0, 2024-02-02T15:46:56.355 INFO:teuthology.orchestra.run.smithi001.stdout: "cap+": 0, 2024-02-02T15:46:56.355 INFO:teuthology.orchestra.run.smithi001.stdout: "cap-": 0, 2024-02-02T15:46:56.355 INFO:teuthology.orchestra.run.smithi001.stdout: "rss": 71772, 2024-02-02T15:46:56.355 INFO:teuthology.orchestra.run.smithi001.stdout: "heap": 207156 2024-02-02T15:46:56.355 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:56.355 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_server": { 2024-02-02T15:46:56.355 INFO:teuthology.orchestra.run.smithi001.stdout: "dispatch_client_request": 0, 2024-02-02T15:46:56.355 INFO:teuthology.orchestra.run.smithi001.stdout: "dispatch_server_request": 0, 2024-02-02T15:46:56.356 INFO:teuthology.orchestra.run.smithi001.stdout: "handle_client_request": 0, 2024-02-02T15:46:56.356 INFO:teuthology.orchestra.run.smithi001.stdout: "handle_client_session": 0, 2024-02-02T15:46:56.356 INFO:teuthology.orchestra.run.smithi001.stdout: "handle_peer_request": 0, 2024-02-02T15:46:56.356 INFO:teuthology.orchestra.run.smithi001.stdout: "req_create_latency": { 2024-02-02T15:46:56.356 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:56.356 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:56.356 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:56.356 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:56.356 INFO:teuthology.orchestra.run.smithi001.stdout: "req_getattr_latency": { 2024-02-02T15:46:56.357 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:56.357 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:56.357 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:56.357 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:56.357 INFO:teuthology.orchestra.run.smithi001.stdout: "req_getfilelock_latency": { 2024-02-02T15:46:56.357 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:56.357 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:56.357 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:56.357 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:56.358 INFO:teuthology.orchestra.run.smithi001.stdout: "req_link_latency": { 2024-02-02T15:46:56.358 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:56.358 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:56.358 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:56.358 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:56.358 INFO:teuthology.orchestra.run.smithi001.stdout: "req_lookup_latency": { 2024-02-02T15:46:56.358 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:56.358 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:56.358 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:56.359 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:56.359 INFO:teuthology.orchestra.run.smithi001.stdout: "req_lookuphash_latency": { 2024-02-02T15:46:56.359 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:56.359 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:56.359 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:56.359 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:56.359 INFO:teuthology.orchestra.run.smithi001.stdout: "req_lookupino_latency": { 2024-02-02T15:46:56.359 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:56.359 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:56.360 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:56.360 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:56.360 INFO:teuthology.orchestra.run.smithi001.stdout: "req_lookupname_latency": { 2024-02-02T15:46:56.360 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:56.360 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:56.360 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:56.360 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:56.360 INFO:teuthology.orchestra.run.smithi001.stdout: "req_lookupparent_latency": { 2024-02-02T15:46:56.360 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:56.361 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:56.361 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:56.361 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:56.361 INFO:teuthology.orchestra.run.smithi001.stdout: "req_lookupsnap_latency": { 2024-02-02T15:46:56.361 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:56.361 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:56.361 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:56.361 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:56.361 INFO:teuthology.orchestra.run.smithi001.stdout: "req_lssnap_latency": { 2024-02-02T15:46:56.361 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:56.362 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:56.362 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:56.362 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:56.362 INFO:teuthology.orchestra.run.smithi001.stdout: "req_mkdir_latency": { 2024-02-02T15:46:56.362 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:56.362 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:56.362 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:56.362 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:56.363 INFO:teuthology.orchestra.run.smithi001.stdout: "req_mknod_latency": { 2024-02-02T15:46:56.363 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:56.363 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:56.363 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:56.363 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:56.363 INFO:teuthology.orchestra.run.smithi001.stdout: "req_mksnap_latency": { 2024-02-02T15:46:56.363 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:56.363 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:56.363 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:56.364 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:56.364 INFO:teuthology.orchestra.run.smithi001.stdout: "req_open_latency": { 2024-02-02T15:46:56.364 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:56.364 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:56.364 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:56.364 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:56.364 INFO:teuthology.orchestra.run.smithi001.stdout: "req_readdir_latency": { 2024-02-02T15:46:56.364 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:56.365 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:56.365 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:56.365 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:56.365 INFO:teuthology.orchestra.run.smithi001.stdout: "req_rename_latency": { 2024-02-02T15:46:56.365 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:56.365 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:56.365 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:56.365 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:56.366 INFO:teuthology.orchestra.run.smithi001.stdout: "req_renamesnap_latency": { 2024-02-02T15:46:56.366 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:56.366 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:56.366 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:56.366 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:56.366 INFO:teuthology.orchestra.run.smithi001.stdout: "req_snapdiff_latency": { 2024-02-02T15:46:56.366 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:56.366 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:56.366 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:56.367 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:56.367 INFO:teuthology.orchestra.run.smithi001.stdout: "req_rmdir_latency": { 2024-02-02T15:46:56.367 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:56.367 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:56.367 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:56.367 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:56.367 INFO:teuthology.orchestra.run.smithi001.stdout: "req_rmsnap_latency": { 2024-02-02T15:46:56.367 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:56.368 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:56.368 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:56.368 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:56.368 INFO:teuthology.orchestra.run.smithi001.stdout: "req_rmxattr_latency": { 2024-02-02T15:46:56.368 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:56.368 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:56.368 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:56.368 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:56.368 INFO:teuthology.orchestra.run.smithi001.stdout: "req_setattr_latency": { 2024-02-02T15:46:56.369 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:56.369 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:56.369 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:56.369 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:56.369 INFO:teuthology.orchestra.run.smithi001.stdout: "req_setdirlayout_latency": { 2024-02-02T15:46:56.369 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:56.369 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:56.369 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:56.369 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:56.370 INFO:teuthology.orchestra.run.smithi001.stdout: "req_setfilelock_latency": { 2024-02-02T15:46:56.370 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:56.370 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:56.370 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:56.370 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:56.370 INFO:teuthology.orchestra.run.smithi001.stdout: "req_setlayout_latency": { 2024-02-02T15:46:56.370 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:56.370 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:56.370 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:56.370 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:56.371 INFO:teuthology.orchestra.run.smithi001.stdout: "req_setxattr_latency": { 2024-02-02T15:46:56.371 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:56.371 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:56.371 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:56.371 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:56.371 INFO:teuthology.orchestra.run.smithi001.stdout: "req_symlink_latency": { 2024-02-02T15:46:56.371 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:56.371 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:56.371 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:56.371 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:56.372 INFO:teuthology.orchestra.run.smithi001.stdout: "req_unlink_latency": { 2024-02-02T15:46:56.372 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:56.372 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:56.372 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:56.372 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:56.375 INFO:teuthology.orchestra.run.smithi001.stdout: "cap_revoke_eviction": 0, 2024-02-02T15:46:56.375 INFO:teuthology.orchestra.run.smithi001.stdout: "cap_acquisition_throttle": 0, 2024-02-02T15:46:56.375 INFO:teuthology.orchestra.run.smithi001.stdout: "req_getvxattr_latency": { 2024-02-02T15:46:56.375 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:56.375 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:56.375 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:56.376 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:56.376 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:56.376 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_sessions": { 2024-02-02T15:46:56.376 INFO:teuthology.orchestra.run.smithi001.stdout: "session_count": 2, 2024-02-02T15:46:56.376 INFO:teuthology.orchestra.run.smithi001.stdout: "session_add": 3, 2024-02-02T15:46:56.376 INFO:teuthology.orchestra.run.smithi001.stdout: "session_remove": 1, 2024-02-02T15:46:56.376 INFO:teuthology.orchestra.run.smithi001.stdout: "sessions_open": 2, 2024-02-02T15:46:56.376 INFO:teuthology.orchestra.run.smithi001.stdout: "sessions_stale": 0, 2024-02-02T15:46:56.377 INFO:teuthology.orchestra.run.smithi001.stdout: "total_load": 0, 2024-02-02T15:46:56.377 INFO:teuthology.orchestra.run.smithi001.stdout: "average_load": 0, 2024-02-02T15:46:56.377 INFO:teuthology.orchestra.run.smithi001.stdout: "avg_session_uptime": 2512, 2024-02-02T15:46:56.377 INFO:teuthology.orchestra.run.smithi001.stdout: "mdthresh_evicted": 0 2024-02-02T15:46:56.377 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:56.377 INFO:teuthology.orchestra.run.smithi001.stdout: "mempool": { 2024-02-02T15:46:56.377 INFO:teuthology.orchestra.run.smithi001.stdout: "bloom_filter_bytes": 0, 2024-02-02T15:46:56.377 INFO:teuthology.orchestra.run.smithi001.stdout: "bloom_filter_items": 0, 2024-02-02T15:46:56.377 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_alloc_bytes": 0, 2024-02-02T15:46:56.378 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_alloc_items": 0, 2024-02-02T15:46:56.378 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_data_bytes": 0, 2024-02-02T15:46:56.378 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_data_items": 0, 2024-02-02T15:46:56.378 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-02T15:46:56.378 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_onode_items": 0, 2024-02-02T15:46:56.378 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-02T15:46:56.378 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_meta_items": 0, 2024-02-02T15:46:56.378 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_other_bytes": 0, 2024-02-02T15:46:56.379 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_other_items": 0, 2024-02-02T15:46:56.379 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-02T15:46:56.379 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_buffer_items": 0, 2024-02-02T15:46:56.379 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_extent_bytes": 0, 2024-02-02T15:46:56.379 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_extent_items": 0, 2024-02-02T15:46:56.379 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_blob_bytes": 0, 2024-02-02T15:46:56.379 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_blob_items": 0, 2024-02-02T15:46:56.379 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-02T15:46:56.379 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_shared_blob_items": 0, 2024-02-02T15:46:56.380 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-02T15:46:56.380 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_inline_bl_items": 0, 2024-02-02T15:46:56.380 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_bytes": 0, 2024-02-02T15:46:56.380 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_items": 0, 2024-02-02T15:46:56.380 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_txc_bytes": 0, 2024-02-02T15:46:56.380 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_txc_items": 0, 2024-02-02T15:46:56.380 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-02T15:46:56.380 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_deferred_items": 0, 2024-02-02T15:46:56.380 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_bytes": 0, 2024-02-02T15:46:56.381 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_items": 0, 2024-02-02T15:46:56.381 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_bytes": 0, 2024-02-02T15:46:56.381 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_items": 0, 2024-02-02T15:46:56.381 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_reader_bytes": 0, 2024-02-02T15:46:56.381 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_reader_items": 0, 2024-02-02T15:46:56.381 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_writer_bytes": 0, 2024-02-02T15:46:56.381 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_writer_items": 0, 2024-02-02T15:46:56.381 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_anon_bytes": 296062, 2024-02-02T15:46:56.381 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_anon_items": 1715, 2024-02-02T15:46:56.382 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_meta_bytes": 0, 2024-02-02T15:46:56.382 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_meta_items": 0, 2024-02-02T15:46:56.382 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_bytes": 0, 2024-02-02T15:46:56.382 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_items": 0, 2024-02-02T15:46:56.382 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mapbl_bytes": 0, 2024-02-02T15:46:56.382 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mapbl_items": 0, 2024-02-02T15:46:56.382 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pglog_bytes": 0, 2024-02-02T15:46:56.382 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pglog_items": 0, 2024-02-02T15:46:56.383 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_bytes": 9608, 2024-02-02T15:46:56.383 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_items": 184, 2024-02-02T15:46:56.383 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_mapping_bytes": 0, 2024-02-02T15:46:56.383 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_mapping_items": 0, 2024-02-02T15:46:56.383 INFO:teuthology.orchestra.run.smithi001.stdout: "pgmap_bytes": 0, 2024-02-02T15:46:56.383 INFO:teuthology.orchestra.run.smithi001.stdout: "pgmap_items": 0, 2024-02-02T15:46:56.383 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_co_bytes": 14796512, 2024-02-02T15:46:56.383 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_co_items": 102650, 2024-02-02T15:46:56.383 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_1_bytes": 0, 2024-02-02T15:46:56.384 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_1_items": 0, 2024-02-02T15:46:56.384 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_2_bytes": 0, 2024-02-02T15:46:56.384 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_2_items": 0 2024-02-02T15:46:56.384 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:56.384 INFO:teuthology.orchestra.run.smithi001.stdout: "objecter": { 2024-02-02T15:46:56.384 INFO:teuthology.orchestra.run.smithi001.stdout: "op_active": 0, 2024-02-02T15:46:56.384 INFO:teuthology.orchestra.run.smithi001.stdout: "op_laggy": 0, 2024-02-02T15:46:56.384 INFO:teuthology.orchestra.run.smithi001.stdout: "op_send": 5050, 2024-02-02T15:46:56.384 INFO:teuthology.orchestra.run.smithi001.stdout: "op_send_bytes": 21, 2024-02-02T15:46:56.385 INFO:teuthology.orchestra.run.smithi001.stdout: "op_resend": 3, 2024-02-02T15:46:56.385 INFO:teuthology.orchestra.run.smithi001.stdout: "op_reply": 5050, 2024-02-02T15:46:56.385 INFO:teuthology.orchestra.run.smithi001.stdout: "op_latency": { 2024-02-02T15:46:56.385 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 5050, 2024-02-02T15:46:56.385 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 15.224813068, 2024-02-02T15:46:56.385 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.003014814 2024-02-02T15:46:56.385 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:56.385 INFO:teuthology.orchestra.run.smithi001.stdout: "op_inflight": 0, 2024-02-02T15:46:56.385 INFO:teuthology.orchestra.run.smithi001.stdout: "oplen_avg": { 2024-02-02T15:46:56.386 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 5050, 2024-02-02T15:46:56.386 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 5051 2024-02-02T15:46:56.386 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:56.386 INFO:teuthology.orchestra.run.smithi001.stdout: "op": 5050, 2024-02-02T15:46:56.386 INFO:teuthology.orchestra.run.smithi001.stdout: "op_r": 5050, 2024-02-02T15:46:56.386 INFO:teuthology.orchestra.run.smithi001.stdout: "op_w": 0, 2024-02-02T15:46:56.386 INFO:teuthology.orchestra.run.smithi001.stdout: "op_rmw": 0, 2024-02-02T15:46:56.386 INFO:teuthology.orchestra.run.smithi001.stdout: "op_pg": 0, 2024-02-02T15:46:56.386 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_stat": 2847, 2024-02-02T15:46:56.387 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_create": 0, 2024-02-02T15:46:56.387 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_read": 2201, 2024-02-02T15:46:56.387 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_write": 0, 2024-02-02T15:46:56.387 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_writefull": 0, 2024-02-02T15:46:56.387 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_writesame": 0, 2024-02-02T15:46:56.387 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_append": 0, 2024-02-02T15:46:56.387 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_zero": 0, 2024-02-02T15:46:56.387 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_truncate": 0, 2024-02-02T15:46:56.387 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_delete": 0, 2024-02-02T15:46:56.388 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_mapext": 0, 2024-02-02T15:46:56.388 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_sparse_read": 0, 2024-02-02T15:46:56.388 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_clonerange": 0, 2024-02-02T15:46:56.388 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_getxattr": 1, 2024-02-02T15:46:56.388 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_setxattr": 0, 2024-02-02T15:46:56.388 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_cmpxattr": 0, 2024-02-02T15:46:56.388 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_rmxattr": 0, 2024-02-02T15:46:56.388 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_resetxattrs": 0, 2024-02-02T15:46:56.389 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_call": 0, 2024-02-02T15:46:56.389 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_watch": 0, 2024-02-02T15:46:56.389 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_notify": 0, 2024-02-02T15:46:56.389 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_src_cmpxattr": 0, 2024-02-02T15:46:56.389 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_pgls": 0, 2024-02-02T15:46:56.389 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_pgls_filter": 0, 2024-02-02T15:46:56.389 INFO:teuthology.orchestra.run.smithi001.stdout: "osdop_other": 0, 2024-02-02T15:46:56.389 INFO:teuthology.orchestra.run.smithi001.stdout: "linger_active": 0, 2024-02-02T15:46:56.389 INFO:teuthology.orchestra.run.smithi001.stdout: "linger_send": 0, 2024-02-02T15:46:56.389 INFO:teuthology.orchestra.run.smithi001.stdout: "linger_resend": 0, 2024-02-02T15:46:56.390 INFO:teuthology.orchestra.run.smithi001.stdout: "linger_ping": 0, 2024-02-02T15:46:56.390 INFO:teuthology.orchestra.run.smithi001.stdout: "poolop_active": 0, 2024-02-02T15:46:56.390 INFO:teuthology.orchestra.run.smithi001.stdout: "poolop_send": 0, 2024-02-02T15:46:56.390 INFO:teuthology.orchestra.run.smithi001.stdout: "poolop_resend": 0, 2024-02-02T15:46:56.390 INFO:teuthology.orchestra.run.smithi001.stdout: "poolstat_active": 0, 2024-02-02T15:46:56.390 INFO:teuthology.orchestra.run.smithi001.stdout: "poolstat_send": 0, 2024-02-02T15:46:56.390 INFO:teuthology.orchestra.run.smithi001.stdout: "poolstat_resend": 0, 2024-02-02T15:46:56.390 INFO:teuthology.orchestra.run.smithi001.stdout: "statfs_active": 0, 2024-02-02T15:46:56.390 INFO:teuthology.orchestra.run.smithi001.stdout: "statfs_send": 0, 2024-02-02T15:46:56.391 INFO:teuthology.orchestra.run.smithi001.stdout: "statfs_resend": 0, 2024-02-02T15:46:56.391 INFO:teuthology.orchestra.run.smithi001.stdout: "command_active": 0, 2024-02-02T15:46:56.391 INFO:teuthology.orchestra.run.smithi001.stdout: "command_send": 0, 2024-02-02T15:46:56.391 INFO:teuthology.orchestra.run.smithi001.stdout: "command_resend": 0, 2024-02-02T15:46:56.391 INFO:teuthology.orchestra.run.smithi001.stdout: "map_epoch": 115, 2024-02-02T15:46:56.391 INFO:teuthology.orchestra.run.smithi001.stdout: "map_full": 0, 2024-02-02T15:46:56.391 INFO:teuthology.orchestra.run.smithi001.stdout: "map_inc": 31, 2024-02-02T15:46:56.391 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_sessions": 11, 2024-02-02T15:46:56.391 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_session_open": 12, 2024-02-02T15:46:56.391 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_session_close": 1, 2024-02-02T15:46:56.392 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_laggy": 0, 2024-02-02T15:46:56.392 INFO:teuthology.orchestra.run.smithi001.stdout: "omap_wr": 0, 2024-02-02T15:46:56.392 INFO:teuthology.orchestra.run.smithi001.stdout: "omap_rd": 2, 2024-02-02T15:46:56.392 INFO:teuthology.orchestra.run.smithi001.stdout: "omap_del": 0 2024-02-02T15:46:56.392 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:56.392 INFO:teuthology.orchestra.run.smithi001.stdout: "oft": { 2024-02-02T15:46:56.392 INFO:teuthology.orchestra.run.smithi001.stdout: "omap_total_objs": 0, 2024-02-02T15:46:56.392 INFO:teuthology.orchestra.run.smithi001.stdout: "omap_total_kv_pairs": 0, 2024-02-02T15:46:56.392 INFO:teuthology.orchestra.run.smithi001.stdout: "omap_total_updates": 0, 2024-02-02T15:46:56.393 INFO:teuthology.orchestra.run.smithi001.stdout: "omap_total_removes": 0 2024-02-02T15:46:56.393 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:56.393 INFO:teuthology.orchestra.run.smithi001.stdout: "osd-slow-ops": { 2024-02-02T15:46:56.393 INFO:teuthology.orchestra.run.smithi001.stdout: "slow_ops_count": 0 2024-02-02T15:46:56.393 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:56.393 INFO:teuthology.orchestra.run.smithi001.stdout: "purge_queue": { 2024-02-02T15:46:56.393 INFO:teuthology.orchestra.run.smithi001.stdout: "pq_executing_ops": 0, 2024-02-02T15:46:56.393 INFO:teuthology.orchestra.run.smithi001.stdout: "pq_executing_ops_high_water": 0, 2024-02-02T15:46:56.393 INFO:teuthology.orchestra.run.smithi001.stdout: "pq_executing": 0, 2024-02-02T15:46:56.394 INFO:teuthology.orchestra.run.smithi001.stdout: "pq_executing_high_water": 0, 2024-02-02T15:46:56.394 INFO:teuthology.orchestra.run.smithi001.stdout: "pq_executed": 0, 2024-02-02T15:46:56.394 INFO:teuthology.orchestra.run.smithi001.stdout: "pq_item_in_journal": 0 2024-02-02T15:46:56.394 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:56.394 INFO:teuthology.orchestra.run.smithi001.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-02T15:46:56.394 INFO:teuthology.orchestra.run.smithi001.stdout: "val": 0, 2024-02-02T15:46:56.394 INFO:teuthology.orchestra.run.smithi001.stdout: "max": 104857600, 2024-02-02T15:46:56.394 INFO:teuthology.orchestra.run.smithi001.stdout: "get_started": 0, 2024-02-02T15:46:56.394 INFO:teuthology.orchestra.run.smithi001.stdout: "get": 5513, 2024-02-02T15:46:56.395 INFO:teuthology.orchestra.run.smithi001.stdout: "get_sum": 430295907, 2024-02-02T15:46:56.395 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_fail": 0, 2024-02-02T15:46:56.395 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_success": 5513, 2024-02-02T15:46:56.395 INFO:teuthology.orchestra.run.smithi001.stdout: "take": 0, 2024-02-02T15:46:56.395 INFO:teuthology.orchestra.run.smithi001.stdout: "take_sum": 0, 2024-02-02T15:46:56.395 INFO:teuthology.orchestra.run.smithi001.stdout: "put": 5513, 2024-02-02T15:46:56.395 INFO:teuthology.orchestra.run.smithi001.stdout: "put_sum": 430295907, 2024-02-02T15:46:56.395 INFO:teuthology.orchestra.run.smithi001.stdout: "wait": { 2024-02-02T15:46:56.395 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:56.395 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:56.396 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:56.396 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:56.396 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:56.396 INFO:teuthology.orchestra.run.smithi001.stdout: "throttle-objecter_bytes": { 2024-02-02T15:46:56.396 INFO:teuthology.orchestra.run.smithi001.stdout: "val": 0, 2024-02-02T15:46:56.396 INFO:teuthology.orchestra.run.smithi001.stdout: "max": 104857600, 2024-02-02T15:46:56.396 INFO:teuthology.orchestra.run.smithi001.stdout: "get_started": 0, 2024-02-02T15:46:56.396 INFO:teuthology.orchestra.run.smithi001.stdout: "get": 0, 2024-02-02T15:46:56.396 INFO:teuthology.orchestra.run.smithi001.stdout: "get_sum": 0, 2024-02-02T15:46:56.397 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_fail": 0, 2024-02-02T15:46:56.397 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_success": 0, 2024-02-02T15:46:56.397 INFO:teuthology.orchestra.run.smithi001.stdout: "take": 5050, 2024-02-02T15:46:56.397 INFO:teuthology.orchestra.run.smithi001.stdout: "take_sum": 427181527, 2024-02-02T15:46:56.397 INFO:teuthology.orchestra.run.smithi001.stdout: "put": 5050, 2024-02-02T15:46:56.397 INFO:teuthology.orchestra.run.smithi001.stdout: "put_sum": 427181527, 2024-02-02T15:46:56.397 INFO:teuthology.orchestra.run.smithi001.stdout: "wait": { 2024-02-02T15:46:56.397 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:56.398 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:56.398 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:56.398 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:56.398 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:56.398 INFO:teuthology.orchestra.run.smithi001.stdout: "throttle-objecter_ops": { 2024-02-02T15:46:56.398 INFO:teuthology.orchestra.run.smithi001.stdout: "val": 0, 2024-02-02T15:46:56.398 INFO:teuthology.orchestra.run.smithi001.stdout: "max": 1024, 2024-02-02T15:46:56.398 INFO:teuthology.orchestra.run.smithi001.stdout: "get_started": 0, 2024-02-02T15:46:56.398 INFO:teuthology.orchestra.run.smithi001.stdout: "get": 0, 2024-02-02T15:46:56.399 INFO:teuthology.orchestra.run.smithi001.stdout: "get_sum": 0, 2024-02-02T15:46:56.399 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_fail": 0, 2024-02-02T15:46:56.399 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_success": 0, 2024-02-02T15:46:56.399 INFO:teuthology.orchestra.run.smithi001.stdout: "take": 5050, 2024-02-02T15:46:56.399 INFO:teuthology.orchestra.run.smithi001.stdout: "take_sum": 5050, 2024-02-02T15:46:56.399 INFO:teuthology.orchestra.run.smithi001.stdout: "put": 5050, 2024-02-02T15:46:56.399 INFO:teuthology.orchestra.run.smithi001.stdout: "put_sum": 5050, 2024-02-02T15:46:56.399 INFO:teuthology.orchestra.run.smithi001.stdout: "wait": { 2024-02-02T15:46:56.399 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:56.400 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:56.400 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:56.400 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:56.400 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:56.400 INFO:teuthology.orchestra.run.smithi001.stdout: "throttle-write_buf_throttle": { 2024-02-02T15:46:56.400 INFO:teuthology.orchestra.run.smithi001.stdout: "val": 0, 2024-02-02T15:46:56.400 INFO:teuthology.orchestra.run.smithi001.stdout: "max": 3758096384, 2024-02-02T15:46:56.400 INFO:teuthology.orchestra.run.smithi001.stdout: "get_started": 0, 2024-02-02T15:46:56.400 INFO:teuthology.orchestra.run.smithi001.stdout: "get": 0, 2024-02-02T15:46:56.401 INFO:teuthology.orchestra.run.smithi001.stdout: "get_sum": 0, 2024-02-02T15:46:56.401 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_fail": 0, 2024-02-02T15:46:56.401 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_success": 0, 2024-02-02T15:46:56.401 INFO:teuthology.orchestra.run.smithi001.stdout: "take": 0, 2024-02-02T15:46:56.401 INFO:teuthology.orchestra.run.smithi001.stdout: "take_sum": 0, 2024-02-02T15:46:56.401 INFO:teuthology.orchestra.run.smithi001.stdout: "put": 0, 2024-02-02T15:46:56.401 INFO:teuthology.orchestra.run.smithi001.stdout: "put_sum": 0, 2024-02-02T15:46:56.401 INFO:teuthology.orchestra.run.smithi001.stdout: "wait": { 2024-02-02T15:46:56.401 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:56.401 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:56.402 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:56.402 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:56.402 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:56.402 INFO:teuthology.orchestra.run.smithi001.stdout: "throttle-write_buf_throttle-0x55a8f219a140": { 2024-02-02T15:46:56.402 INFO:teuthology.orchestra.run.smithi001.stdout: "val": 0, 2024-02-02T15:46:56.402 INFO:teuthology.orchestra.run.smithi001.stdout: "max": 3758096384, 2024-02-02T15:46:56.402 INFO:teuthology.orchestra.run.smithi001.stdout: "get_started": 0, 2024-02-02T15:46:56.402 INFO:teuthology.orchestra.run.smithi001.stdout: "get": 0, 2024-02-02T15:46:56.402 INFO:teuthology.orchestra.run.smithi001.stdout: "get_sum": 0, 2024-02-02T15:46:56.403 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_fail": 0, 2024-02-02T15:46:56.403 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_success": 0, 2024-02-02T15:46:56.403 INFO:teuthology.orchestra.run.smithi001.stdout: "take": 0, 2024-02-02T15:46:56.403 INFO:teuthology.orchestra.run.smithi001.stdout: "take_sum": 0, 2024-02-02T15:46:56.403 INFO:teuthology.orchestra.run.smithi001.stdout: "put": 0, 2024-02-02T15:46:56.403 INFO:teuthology.orchestra.run.smithi001.stdout: "put_sum": 0, 2024-02-02T15:46:56.403 INFO:teuthology.orchestra.run.smithi001.stdout: "wait": { 2024-02-02T15:46:56.403 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:56.403 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:56.404 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:56.404 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:56.404 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:56.404 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:46:56.840 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-02-02T15:46:56.840 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-02-02T15:46:56.840 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update=0 2024-02-02T15:46:56.840 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update_receipt=0 2024-02-02T15:46:56.841 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=0 2024-02-02T15:46:56.841 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-02-02T15:46:56.841 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-02-02T15:46:56.841 DEBUG:tasks.check_counter:Getting stats from j 2024-02-02T15:46:56.841 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph daemon mds.j perf dump 2024-02-02T15:46:57.214 INFO:teuthology.orchestra.run.smithi001.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.a/config 2024-02-02T15:46:57.485 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:46:57 smithi001 ceph-mon[35470]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:46:57.485 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:46:57 smithi001 ceph-mon[35470]: pgmap v627: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 160 KiB/s wr, 19 op/s 2024-02-02T15:46:57.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:46:57 smithi071 ceph-mon[44707]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:46:57.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:46:57 smithi071 ceph-mon[44707]: pgmap v627: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 160 KiB/s wr, 19 op/s 2024-02-02T15:46:57.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:46:57 smithi142 ceph-mon[45728]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-02-02T15:46:57.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:46:57 smithi142 ceph-mon[45728]: pgmap v627: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 160 KiB/s wr, 19 op/s 2024-02-02T15:46:58.633 INFO:teuthology.orchestra.run.smithi001.stdout:{ 2024-02-02T15:46:58.633 INFO:teuthology.orchestra.run.smithi001.stdout: "AsyncMessenger::Worker-0": { 2024-02-02T15:46:58.633 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_messages": 0, 2024-02-02T15:46:58.633 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages": 2, 2024-02-02T15:46:58.633 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_bytes": 0, 2024-02-02T15:46:58.633 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_bytes": 0, 2024-02-02T15:46:58.634 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_created_connections": 4, 2024-02-02T15:46:58.634 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_active_connections": 0, 2024-02-02T15:46:58.634 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_total_time": 0.003488919, 2024-02-02T15:46:58.634 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_send_time": 0.000000000, 2024-02-02T15:46:58.634 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_recv_time": 0.000782994, 2024-02-02T15:46:58.634 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-02T15:46:58.634 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages_queue_lat": { 2024-02-02T15:46:58.634 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:58.634 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:58.635 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:58.635 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:58.635 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_handle_ack_lat": { 2024-02-02T15:46:58.635 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:58.635 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:58.635 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:58.635 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:58.635 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_encrypted_bytes": 0, 2024-02-02T15:46:58.635 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_encrypted_bytes": 0 2024-02-02T15:46:58.636 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:58.636 INFO:teuthology.orchestra.run.smithi001.stdout: "AsyncMessenger::Worker-1": { 2024-02-02T15:46:58.636 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_messages": 7, 2024-02-02T15:46:58.636 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages": 278, 2024-02-02T15:46:58.636 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_bytes": 3472, 2024-02-02T15:46:58.636 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_bytes": 277456, 2024-02-02T15:46:58.636 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_created_connections": 6, 2024-02-02T15:46:58.636 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_active_connections": 3, 2024-02-02T15:46:58.637 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_total_time": 0.032394211, 2024-02-02T15:46:58.637 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_send_time": 0.023584530, 2024-02-02T15:46:58.637 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_recv_time": 0.004324916, 2024-02-02T15:46:58.637 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-02-02T15:46:58.637 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages_queue_lat": { 2024-02-02T15:46:58.637 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 276, 2024-02-02T15:46:58.637 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.027972613, 2024-02-02T15:46:58.637 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000101350 2024-02-02T15:46:58.637 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:58.638 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_handle_ack_lat": { 2024-02-02T15:46:58.638 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:58.638 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:58.638 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:58.638 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:58.638 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_encrypted_bytes": 3472, 2024-02-02T15:46:58.638 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_encrypted_bytes": 277456 2024-02-02T15:46:58.638 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:58.638 INFO:teuthology.orchestra.run.smithi001.stdout: "AsyncMessenger::Worker-2": { 2024-02-02T15:46:58.639 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_messages": 372, 2024-02-02T15:46:58.639 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages": 359, 2024-02-02T15:46:58.639 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_bytes": 1863152, 2024-02-02T15:46:58.639 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_bytes": 206048, 2024-02-02T15:46:58.639 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_created_connections": 2, 2024-02-02T15:46:58.639 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_active_connections": 1, 2024-02-02T15:46:58.639 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_total_time": 0.081616665, 2024-02-02T15:46:58.639 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_send_time": 0.032783731, 2024-02-02T15:46:58.639 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_recv_time": 0.050270593, 2024-02-02T15:46:58.640 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_running_fast_dispatch_time": 0.009299779, 2024-02-02T15:46:58.640 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_messages_queue_lat": { 2024-02-02T15:46:58.640 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 358, 2024-02-02T15:46:58.640 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.027272592, 2024-02-02T15:46:58.640 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000076180 2024-02-02T15:46:58.640 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:58.640 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_handle_ack_lat": { 2024-02-02T15:46:58.640 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:58.640 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:58.641 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:58.641 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:58.641 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_recv_encrypted_bytes": 1863152, 2024-02-02T15:46:58.641 INFO:teuthology.orchestra.run.smithi001.stdout: "msgr_send_encrypted_bytes": 206048 2024-02-02T15:46:58.641 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:58.641 INFO:teuthology.orchestra.run.smithi001.stdout: "cct": { 2024-02-02T15:46:58.641 INFO:teuthology.orchestra.run.smithi001.stdout: "total_workers": 0, 2024-02-02T15:46:58.641 INFO:teuthology.orchestra.run.smithi001.stdout: "unhealthy_workers": 0 2024-02-02T15:46:58.641 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:58.642 INFO:teuthology.orchestra.run.smithi001.stdout: "mempool": { 2024-02-02T15:46:58.642 INFO:teuthology.orchestra.run.smithi001.stdout: "bloom_filter_bytes": 0, 2024-02-02T15:46:58.642 INFO:teuthology.orchestra.run.smithi001.stdout: "bloom_filter_items": 0, 2024-02-02T15:46:58.642 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_alloc_bytes": 0, 2024-02-02T15:46:58.642 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_alloc_items": 0, 2024-02-02T15:46:58.642 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_data_bytes": 0, 2024-02-02T15:46:58.642 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_data_items": 0, 2024-02-02T15:46:58.642 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-02T15:46:58.642 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_onode_items": 0, 2024-02-02T15:46:58.643 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-02T15:46:58.643 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_meta_items": 0, 2024-02-02T15:46:58.643 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_other_bytes": 0, 2024-02-02T15:46:58.643 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_other_items": 0, 2024-02-02T15:46:58.643 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-02T15:46:58.643 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_cache_buffer_items": 0, 2024-02-02T15:46:58.643 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_extent_bytes": 0, 2024-02-02T15:46:58.643 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_extent_items": 0, 2024-02-02T15:46:58.643 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_blob_bytes": 0, 2024-02-02T15:46:58.644 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_blob_items": 0, 2024-02-02T15:46:58.644 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-02T15:46:58.644 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_shared_blob_items": 0, 2024-02-02T15:46:58.644 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-02T15:46:58.644 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_inline_bl_items": 0, 2024-02-02T15:46:58.644 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_bytes": 0, 2024-02-02T15:46:58.644 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_fsck_items": 0, 2024-02-02T15:46:58.644 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_txc_bytes": 0, 2024-02-02T15:46:58.645 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_txc_items": 0, 2024-02-02T15:46:58.645 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-02T15:46:58.645 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_deferred_items": 0, 2024-02-02T15:46:58.645 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_bytes": 0, 2024-02-02T15:46:58.645 INFO:teuthology.orchestra.run.smithi001.stdout: "bluestore_writing_items": 0, 2024-02-02T15:46:58.645 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_bytes": 0, 2024-02-02T15:46:58.645 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_items": 0, 2024-02-02T15:46:58.645 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_reader_bytes": 0, 2024-02-02T15:46:58.645 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_reader_items": 0, 2024-02-02T15:46:58.646 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_writer_bytes": 0, 2024-02-02T15:46:58.646 INFO:teuthology.orchestra.run.smithi001.stdout: "bluefs_file_writer_items": 0, 2024-02-02T15:46:58.646 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_anon_bytes": 148754, 2024-02-02T15:46:58.646 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_anon_items": 31, 2024-02-02T15:46:58.646 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_meta_bytes": 0, 2024-02-02T15:46:58.646 INFO:teuthology.orchestra.run.smithi001.stdout: "buffer_meta_items": 0, 2024-02-02T15:46:58.646 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_bytes": 0, 2024-02-02T15:46:58.646 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_items": 0, 2024-02-02T15:46:58.646 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mapbl_bytes": 0, 2024-02-02T15:46:58.647 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_mapbl_items": 0, 2024-02-02T15:46:58.647 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pglog_bytes": 0, 2024-02-02T15:46:58.647 INFO:teuthology.orchestra.run.smithi001.stdout: "osd_pglog_items": 0, 2024-02-02T15:46:58.647 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_bytes": 0, 2024-02-02T15:46:58.647 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_items": 0, 2024-02-02T15:46:58.647 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_mapping_bytes": 0, 2024-02-02T15:46:58.647 INFO:teuthology.orchestra.run.smithi001.stdout: "osdmap_mapping_items": 0, 2024-02-02T15:46:58.647 INFO:teuthology.orchestra.run.smithi001.stdout: "pgmap_bytes": 0, 2024-02-02T15:46:58.647 INFO:teuthology.orchestra.run.smithi001.stdout: "pgmap_items": 0, 2024-02-02T15:46:58.648 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_co_bytes": 928, 2024-02-02T15:46:58.648 INFO:teuthology.orchestra.run.smithi001.stdout: "mds_co_items": 2, 2024-02-02T15:46:58.648 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_1_bytes": 0, 2024-02-02T15:46:58.648 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_1_items": 0, 2024-02-02T15:46:58.648 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_2_bytes": 0, 2024-02-02T15:46:58.648 INFO:teuthology.orchestra.run.smithi001.stdout: "unittest_2_items": 0 2024-02-02T15:46:58.648 INFO:teuthology.orchestra.run.smithi001.stdout: }, 2024-02-02T15:46:58.648 INFO:teuthology.orchestra.run.smithi001.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-02T15:46:58.648 INFO:teuthology.orchestra.run.smithi001.stdout: "val": 0, 2024-02-02T15:46:58.649 INFO:teuthology.orchestra.run.smithi001.stdout: "max": 104857600, 2024-02-02T15:46:58.649 INFO:teuthology.orchestra.run.smithi001.stdout: "get_started": 0, 2024-02-02T15:46:58.649 INFO:teuthology.orchestra.run.smithi001.stdout: "get": 375, 2024-02-02T15:46:58.649 INFO:teuthology.orchestra.run.smithi001.stdout: "get_sum": 1810673, 2024-02-02T15:46:58.649 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_fail": 0, 2024-02-02T15:46:58.649 INFO:teuthology.orchestra.run.smithi001.stdout: "get_or_fail_success": 375, 2024-02-02T15:46:58.649 INFO:teuthology.orchestra.run.smithi001.stdout: "take": 0, 2024-02-02T15:46:58.649 INFO:teuthology.orchestra.run.smithi001.stdout: "take_sum": 0, 2024-02-02T15:46:58.650 INFO:teuthology.orchestra.run.smithi001.stdout: "put": 375, 2024-02-02T15:46:58.650 INFO:teuthology.orchestra.run.smithi001.stdout: "put_sum": 1810673, 2024-02-02T15:46:58.650 INFO:teuthology.orchestra.run.smithi001.stdout: "wait": { 2024-02-02T15:46:58.650 INFO:teuthology.orchestra.run.smithi001.stdout: "avgcount": 0, 2024-02-02T15:46:58.650 INFO:teuthology.orchestra.run.smithi001.stdout: "sum": 0.000000000, 2024-02-02T15:46:58.650 INFO:teuthology.orchestra.run.smithi001.stdout: "avgtime": 0.000000000 2024-02-02T15:46:58.650 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:58.650 INFO:teuthology.orchestra.run.smithi001.stdout: } 2024-02-02T15:46:58.650 INFO:teuthology.orchestra.run.smithi001.stdout:} 2024-02-02T15:46:58.972 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-02-02T15:46:58.972 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-02-02T15:46:58.972 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-02-02T15:46:58.972 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-02-02T15:46:58.972 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-02-02T15:46:58.973 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-02-02T15:46:58.973 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-02-02T15:46:58.973 DEBUG:tasks.check_counter:Getting stats from b 2024-02-02T15:46:58.973 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph daemon mds.b perf dump 2024-02-02T15:46:59.362 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.b/config 2024-02-02T15:46:59.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:46:59 smithi071 ceph-mon[44707]: pgmap v628: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 155 KiB/s wr, 17 op/s 2024-02-02T15:46:59.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:46:59 smithi142 ceph-mon[45728]: pgmap v628: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 155 KiB/s wr, 17 op/s 2024-02-02T15:46:59.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:46:59 smithi001 ceph-mon[35470]: pgmap v628: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 155 KiB/s wr, 17 op/s 2024-02-02T15:47:01.194 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-02-02T15:47:01.195 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-0": { 2024-02-02T15:47:01.195 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 345636, 2024-02-02T15:47:01.195 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 470384, 2024-02-02T15:47:01.195 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 84085586, 2024-02-02T15:47:01.195 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 308055010, 2024-02-02T15:47:01.195 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 10, 2024-02-02T15:47:01.195 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 5, 2024-02-02T15:47:01.196 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 18.179192863, 2024-02-02T15:47:01.196 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 7.437320055, 2024-02-02T15:47:01.196 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 11.555589349, 2024-02-02T15:47:01.196 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.773466571, 2024-02-02T15:47:01.196 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-02-02T15:47:01.196 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 470386, 2024-02-02T15:47:01.196 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 30.832688043, 2024-02-02T15:47:01.197 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000065547 2024-02-02T15:47:01.197 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:01.197 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-02-02T15:47:01.197 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 748529, 2024-02-02T15:47:01.197 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.115036852, 2024-02-02T15:47:01.197 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000153 2024-02-02T15:47:01.197 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:01.197 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 2297168, 2024-02-02T15:47:01.198 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 780400 2024-02-02T15:47:01.198 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:01.198 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-1": { 2024-02-02T15:47:01.198 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 218957, 2024-02-02T15:47:01.198 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 275293, 2024-02-02T15:47:01.198 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 55756292, 2024-02-02T15:47:01.198 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 409510936, 2024-02-02T15:47:01.198 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 31, 2024-02-02T15:47:01.199 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 6, 2024-02-02T15:47:01.199 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 15.207581251, 2024-02-02T15:47:01.199 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 6.666917374, 2024-02-02T15:47:01.199 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 8.053379282, 2024-02-02T15:47:01.199 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 1.526185767, 2024-02-02T15:47:01.199 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-02-02T15:47:01.199 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 275292, 2024-02-02T15:47:01.199 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 19.563167451, 2024-02-02T15:47:01.199 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000071063 2024-02-02T15:47:01.200 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:01.200 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-02-02T15:47:01.200 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 402703, 2024-02-02T15:47:01.200 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.069128249, 2024-02-02T15:47:01.200 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000171 2024-02-02T15:47:01.200 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:01.200 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 653040, 2024-02-02T15:47:01.201 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 330208 2024-02-02T15:47:01.201 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:01.201 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-2": { 2024-02-02T15:47:01.201 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 481758, 2024-02-02T15:47:01.201 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 643227, 2024-02-02T15:47:01.201 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 134776238, 2024-02-02T15:47:01.201 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 674071774, 2024-02-02T15:47:01.201 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 282, 2024-02-02T15:47:01.201 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 10, 2024-02-02T15:47:01.202 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 24.965004964, 2024-02-02T15:47:01.202 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 11.166149429, 2024-02-02T15:47:01.202 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 16.034107217, 2024-02-02T15:47:01.202 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 1.271329177, 2024-02-02T15:47:01.202 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-02-02T15:47:01.202 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 643221, 2024-02-02T15:47:01.202 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 60.578928547, 2024-02-02T15:47:01.202 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000094180 2024-02-02T15:47:01.203 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:01.203 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-02-02T15:47:01.203 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 707321, 2024-02-02T15:47:01.203 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.149737703, 2024-02-02T15:47:01.203 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000211 2024-02-02T15:47:01.203 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:01.203 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 89616, 2024-02-02T15:47:01.203 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 30320 2024-02-02T15:47:01.204 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:01.204 INFO:teuthology.orchestra.run.smithi071.stdout: "cct": { 2024-02-02T15:47:01.204 INFO:teuthology.orchestra.run.smithi071.stdout: "total_workers": 1, 2024-02-02T15:47:01.204 INFO:teuthology.orchestra.run.smithi071.stdout: "unhealthy_workers": 0 2024-02-02T15:47:01.204 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:01.204 INFO:teuthology.orchestra.run.smithi071.stdout: "finisher-MDSRank": { 2024-02-02T15:47:01.204 INFO:teuthology.orchestra.run.smithi071.stdout: "queue_len": 0, 2024-02-02T15:47:01.204 INFO:teuthology.orchestra.run.smithi071.stdout: "complete_latency": { 2024-02-02T15:47:01.205 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 113458, 2024-02-02T15:47:01.205 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 82.456930962, 2024-02-02T15:47:01.205 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000726761 2024-02-02T15:47:01.205 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-02-02T15:47:01.205 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:01.205 INFO:teuthology.orchestra.run.smithi071.stdout: "finisher-PurgeQueue": { 2024-02-02T15:47:01.205 INFO:teuthology.orchestra.run.smithi071.stdout: "queue_len": 0, 2024-02-02T15:47:01.205 INFO:teuthology.orchestra.run.smithi071.stdout: "complete_latency": { 2024-02-02T15:47:01.205 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 32982, 2024-02-02T15:47:01.206 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 4.822748422, 2024-02-02T15:47:01.206 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000146223 2024-02-02T15:47:01.206 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-02-02T15:47:01.206 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:01.206 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-02-02T15:47:01.206 INFO:teuthology.orchestra.run.smithi071.stdout: "request": 186391, 2024-02-02T15:47:01.206 INFO:teuthology.orchestra.run.smithi071.stdout: "reply": 156139, 2024-02-02T15:47:01.206 INFO:teuthology.orchestra.run.smithi071.stdout: "reply_latency": { 2024-02-02T15:47:01.206 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 156139, 2024-02-02T15:47:01.206 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 916.752949326, 2024-02-02T15:47:01.207 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.005871389 2024-02-02T15:47:01.207 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:01.207 INFO:teuthology.orchestra.run.smithi071.stdout: "slow_reply": 0, 2024-02-02T15:47:01.207 INFO:teuthology.orchestra.run.smithi071.stdout: "forward": 30252, 2024-02-02T15:47:01.207 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_fetch_complete": 3, 2024-02-02T15:47:01.207 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_fetch_keys": 0, 2024-02-02T15:47:01.207 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_commit": 84, 2024-02-02T15:47:01.207 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_split": 7, 2024-02-02T15:47:01.208 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_merge": 4, 2024-02-02T15:47:01.208 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes": 1390, 2024-02-02T15:47:01.208 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_top": 346, 2024-02-02T15:47:01.208 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_bottom": 967, 2024-02-02T15:47:01.208 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_pin_tail": 77, 2024-02-02T15:47:01.208 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_pinned": 896, 2024-02-02T15:47:01.208 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_expired": 2665, 2024-02-02T15:47:01.208 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_with_caps": 158, 2024-02-02T15:47:01.209 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": 159, 2024-02-02T15:47:01.209 INFO:teuthology.orchestra.run.smithi071.stdout: "subtrees": 17, 2024-02-02T15:47:01.209 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse": 288194, 2024-02-02T15:47:01.209 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_hit": 178288, 2024-02-02T15:47:01.209 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_forward": 30211, 2024-02-02T15:47:01.209 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_discover": 150, 2024-02-02T15:47:01.209 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_dir_fetch": 0, 2024-02-02T15:47:01.209 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_remote_ino": 0, 2024-02-02T15:47:01.209 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_lock": 1921, 2024-02-02T15:47:01.210 INFO:teuthology.orchestra.run.smithi071.stdout: "load_cent": 2076, 2024-02-02T15:47:01.210 INFO:teuthology.orchestra.run.smithi071.stdout: "q": 0, 2024-02-02T15:47:01.210 INFO:teuthology.orchestra.run.smithi071.stdout: "exported": 28, 2024-02-02T15:47:01.210 INFO:teuthology.orchestra.run.smithi071.stdout: "exported_inodes": 1973, 2024-02-02T15:47:01.210 INFO:teuthology.orchestra.run.smithi071.stdout: "imported": 15, 2024-02-02T15:47:01.210 INFO:teuthology.orchestra.run.smithi071.stdout: "imported_inodes": 660, 2024-02-02T15:47:01.210 INFO:teuthology.orchestra.run.smithi071.stdout: "openino_dir_fetch": 0, 2024-02-02T15:47:01.210 INFO:teuthology.orchestra.run.smithi071.stdout: "openino_backtrace_fetch": 2822, 2024-02-02T15:47:01.210 INFO:teuthology.orchestra.run.smithi071.stdout: "openino_peer_discover": 0, 2024-02-02T15:47:01.211 INFO:teuthology.orchestra.run.smithi071.stdout: "root_rfiles": 670, 2024-02-02T15:47:01.211 INFO:teuthology.orchestra.run.smithi071.stdout: "root_rbytes": 218701295, 2024-02-02T15:47:01.211 INFO:teuthology.orchestra.run.smithi071.stdout: "root_rsnaps": 6, 2024-02-02T15:47:01.211 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_backtrace_fetch": 2822, 2024-02-02T15:47:01.211 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_set_tag": 0, 2024-02-02T15:47:01.211 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_backtrace_repaired": 0, 2024-02-02T15:47:01.211 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_inotable_repaired": 0, 2024-02-02T15:47:01.211 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_dir_inodes": 885, 2024-02-02T15:47:01.212 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_dir_base_inodes": 35, 2024-02-02T15:47:01.212 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_dirfrag_rstats": 850, 2024-02-02T15:47:01.212 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_file_inodes": 1937, 2024-02-02T15:47:01.212 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_inode_file_caps": 33, 2024-02-02T15:47:01.212 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_revoke": 49956, 2024-02-02T15:47:01.212 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_grant": 79224, 2024-02-02T15:47:01.212 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_trunc": 0, 2024-02-02T15:47:01.212 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-02-02T15:47:01.212 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_flush_ack": 0, 2024-02-02T15:47:01.213 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_caps": 82041, 2024-02-02T15:47:01.213 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_caps_dirty": 17176, 2024-02-02T15:47:01.213 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_cap_release": 7047, 2024-02-02T15:47:01.213 INFO:teuthology.orchestra.run.smithi071.stdout: "process_request_cap_release": 88878 2024-02-02T15:47:01.213 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:01.213 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_cache": { 2024-02-02T15:47:01.213 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_update": 564, 2024-02-02T15:47:01.213 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_update_receipt": 560, 2024-02-02T15:47:01.213 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_try_discover": 298, 2024-02-02T15:47:01.214 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_send_discover": 635, 2024-02-02T15:47:01.214 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_handle_discover": 3902, 2024-02-02T15:47:01.214 INFO:teuthology.orchestra.run.smithi071.stdout: "num_strays": 0, 2024-02-02T15:47:01.214 INFO:teuthology.orchestra.run.smithi071.stdout: "num_strays_delayed": 0, 2024-02-02T15:47:01.214 INFO:teuthology.orchestra.run.smithi071.stdout: "num_strays_enqueuing": 0, 2024-02-02T15:47:01.214 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_created": 17348, 2024-02-02T15:47:01.214 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_enqueued": 17348, 2024-02-02T15:47:01.214 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_reintegrated": 0, 2024-02-02T15:47:01.214 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_migrated": 0, 2024-02-02T15:47:01.215 INFO:teuthology.orchestra.run.smithi071.stdout: "num_recovering_processing": 0, 2024-02-02T15:47:01.215 INFO:teuthology.orchestra.run.smithi071.stdout: "num_recovering_enqueued": 0, 2024-02-02T15:47:01.215 INFO:teuthology.orchestra.run.smithi071.stdout: "num_recovering_prioritized": 0, 2024-02-02T15:47:01.215 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_started": 0, 2024-02-02T15:47:01.215 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_completed": 0, 2024-02-02T15:47:01.215 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_enqueue_scrub": 35, 2024-02-02T15:47:01.215 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_exportdir": 33, 2024-02-02T15:47:01.215 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_flush": 0, 2024-02-02T15:47:01.215 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_fragmentdir": 11, 2024-02-02T15:47:01.216 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_fragstats": 0, 2024-02-02T15:47:01.216 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_inodestats": 0 2024-02-02T15:47:01.216 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:01.216 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_log": { 2024-02-02T15:47:01.216 INFO:teuthology.orchestra.run.smithi071.stdout: "evadd": 120676, 2024-02-02T15:47:01.216 INFO:teuthology.orchestra.run.smithi071.stdout: "evex": 46170, 2024-02-02T15:47:01.216 INFO:teuthology.orchestra.run.smithi071.stdout: "evtrm": 46170, 2024-02-02T15:47:01.216 INFO:teuthology.orchestra.run.smithi071.stdout: "ev": 74506, 2024-02-02T15:47:01.217 INFO:teuthology.orchestra.run.smithi071.stdout: "evexg": 0, 2024-02-02T15:47:01.217 INFO:teuthology.orchestra.run.smithi071.stdout: "evexd": 282, 2024-02-02T15:47:01.217 INFO:teuthology.orchestra.run.smithi071.stdout: "segadd": 206, 2024-02-02T15:47:01.217 INFO:teuthology.orchestra.run.smithi071.stdout: "segex": 78, 2024-02-02T15:47:01.217 INFO:teuthology.orchestra.run.smithi071.stdout: "segtrm": 78, 2024-02-02T15:47:01.217 INFO:teuthology.orchestra.run.smithi071.stdout: "seg": 129, 2024-02-02T15:47:01.217 INFO:teuthology.orchestra.run.smithi071.stdout: "segexg": 0, 2024-02-02T15:47:01.217 INFO:teuthology.orchestra.run.smithi071.stdout: "segexd": 1, 2024-02-02T15:47:01.218 INFO:teuthology.orchestra.run.smithi071.stdout: "expos": 289433354, 2024-02-02T15:47:01.218 INFO:teuthology.orchestra.run.smithi071.stdout: "wrpos": 807097914, 2024-02-02T15:47:01.218 INFO:teuthology.orchestra.run.smithi071.stdout: "rdpos": 0, 2024-02-02T15:47:01.218 INFO:teuthology.orchestra.run.smithi071.stdout: "jlat": { 2024-02-02T15:47:01.218 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 58615, 2024-02-02T15:47:01.218 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 306.274510393, 2024-02-02T15:47:01.218 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.005225189 2024-02-02T15:47:01.218 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:01.218 INFO:teuthology.orchestra.run.smithi071.stdout: "replayed": 0 2024-02-02T15:47:01.218 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:01.219 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_mem": { 2024-02-02T15:47:01.219 INFO:teuthology.orchestra.run.smithi071.stdout: "ino": 1022, 2024-02-02T15:47:01.219 INFO:teuthology.orchestra.run.smithi071.stdout: "ino+": 19868, 2024-02-02T15:47:01.219 INFO:teuthology.orchestra.run.smithi071.stdout: "ino-": 18846, 2024-02-02T15:47:01.219 INFO:teuthology.orchestra.run.smithi071.stdout: "dir": 203, 2024-02-02T15:47:01.219 INFO:teuthology.orchestra.run.smithi071.stdout: "dir+": 555, 2024-02-02T15:47:01.219 INFO:teuthology.orchestra.run.smithi071.stdout: "dir-": 352, 2024-02-02T15:47:01.219 INFO:teuthology.orchestra.run.smithi071.stdout: "dn": 1390, 2024-02-02T15:47:01.220 INFO:teuthology.orchestra.run.smithi071.stdout: "dn+": 39960, 2024-02-02T15:47:01.220 INFO:teuthology.orchestra.run.smithi071.stdout: "dn-": 38570, 2024-02-02T15:47:01.220 INFO:teuthology.orchestra.run.smithi071.stdout: "cap": 159, 2024-02-02T15:47:01.220 INFO:teuthology.orchestra.run.smithi071.stdout: "cap+": 20445, 2024-02-02T15:47:01.220 INFO:teuthology.orchestra.run.smithi071.stdout: "cap-": 20286, 2024-02-02T15:47:01.220 INFO:teuthology.orchestra.run.smithi071.stdout: "rss": 79604, 2024-02-02T15:47:01.220 INFO:teuthology.orchestra.run.smithi071.stdout: "heap": 207156 2024-02-02T15:47:01.220 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:01.220 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_server": { 2024-02-02T15:47:01.221 INFO:teuthology.orchestra.run.smithi071.stdout: "dispatch_client_request": 273354, 2024-02-02T15:47:01.221 INFO:teuthology.orchestra.run.smithi071.stdout: "dispatch_server_request": 29591, 2024-02-02T15:47:01.221 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_request": 186391, 2024-02-02T15:47:01.221 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_session": 2379, 2024-02-02T15:47:01.221 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_peer_request": 59661, 2024-02-02T15:47:01.221 INFO:teuthology.orchestra.run.smithi071.stdout: "req_create_latency": { 2024-02-02T15:47:01.221 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 18048, 2024-02-02T15:47:01.221 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 102.885952004, 2024-02-02T15:47:01.221 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.005700684 2024-02-02T15:47:01.222 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:01.222 INFO:teuthology.orchestra.run.smithi071.stdout: "req_getattr_latency": { 2024-02-02T15:47:01.222 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 35287, 2024-02-02T15:47:01.222 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 251.154884035, 2024-02-02T15:47:01.222 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.007117490 2024-02-02T15:47:01.222 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:01.222 INFO:teuthology.orchestra.run.smithi071.stdout: "req_getfilelock_latency": { 2024-02-02T15:47:01.222 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:01.222 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:01.223 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:01.223 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:01.223 INFO:teuthology.orchestra.run.smithi071.stdout: "req_link_latency": { 2024-02-02T15:47:01.223 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:01.223 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:01.223 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:01.223 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:01.223 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookup_latency": { 2024-02-02T15:47:01.223 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 39447, 2024-02-02T15:47:01.223 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 132.032615667, 2024-02-02T15:47:01.224 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.003347088 2024-02-02T15:47:01.224 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:01.224 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookuphash_latency": { 2024-02-02T15:47:01.224 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:01.224 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:01.224 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:01.224 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:01.224 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupino_latency": { 2024-02-02T15:47:01.225 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 1, 2024-02-02T15:47:01.225 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000133119, 2024-02-02T15:47:01.225 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000133119 2024-02-02T15:47:01.225 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:01.225 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupname_latency": { 2024-02-02T15:47:01.225 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:01.225 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:01.225 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:01.225 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:01.225 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupparent_latency": { 2024-02-02T15:47:01.226 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:01.226 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:01.226 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:01.226 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:01.226 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupsnap_latency": { 2024-02-02T15:47:01.226 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:01.226 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:01.226 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:01.227 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:01.227 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lssnap_latency": { 2024-02-02T15:47:01.227 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 20, 2024-02-02T15:47:01.227 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.006944781, 2024-02-02T15:47:01.227 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000347239 2024-02-02T15:47:01.227 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:01.227 INFO:teuthology.orchestra.run.smithi071.stdout: "req_mkdir_latency": { 2024-02-02T15:47:01.227 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 119, 2024-02-02T15:47:01.227 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.880648269, 2024-02-02T15:47:01.228 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.007400405 2024-02-02T15:47:01.228 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:01.228 INFO:teuthology.orchestra.run.smithi071.stdout: "req_mknod_latency": { 2024-02-02T15:47:01.228 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:01.228 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:01.228 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:01.228 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:01.228 INFO:teuthology.orchestra.run.smithi071.stdout: "req_mksnap_latency": { 2024-02-02T15:47:01.229 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 20, 2024-02-02T15:47:01.229 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 13.706344990, 2024-02-02T15:47:01.229 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.685317249 2024-02-02T15:47:01.229 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:01.229 INFO:teuthology.orchestra.run.smithi071.stdout: "req_open_latency": { 2024-02-02T15:47:01.229 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 9, 2024-02-02T15:47:01.229 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.024830977, 2024-02-02T15:47:01.229 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.002758997 2024-02-02T15:47:01.230 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:01.230 INFO:teuthology.orchestra.run.smithi071.stdout: "req_readdir_latency": { 2024-02-02T15:47:01.230 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 41723, 2024-02-02T15:47:01.230 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 210.039867232, 2024-02-02T15:47:01.230 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.005034150 2024-02-02T15:47:01.230 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:01.230 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rename_latency": { 2024-02-02T15:47:01.230 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 3509, 2024-02-02T15:47:01.230 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 97.917521008, 2024-02-02T15:47:01.231 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.027904679 2024-02-02T15:47:01.231 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:01.231 INFO:teuthology.orchestra.run.smithi071.stdout: "req_renamesnap_latency": { 2024-02-02T15:47:01.231 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:01.231 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:01.231 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:01.231 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:01.231 INFO:teuthology.orchestra.run.smithi071.stdout: "req_snapdiff_latency": { 2024-02-02T15:47:01.232 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:01.232 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:01.232 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:01.232 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:01.232 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rmdir_latency": { 2024-02-02T15:47:01.232 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 1, 2024-02-02T15:47:01.232 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.001173284, 2024-02-02T15:47:01.232 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.001173284 2024-02-02T15:47:01.233 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:01.233 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rmsnap_latency": { 2024-02-02T15:47:01.233 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 14, 2024-02-02T15:47:01.233 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 4.603200273, 2024-02-02T15:47:01.233 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.328800019 2024-02-02T15:47:01.233 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:01.233 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rmxattr_latency": { 2024-02-02T15:47:01.233 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:01.234 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:01.234 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:01.234 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:01.234 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setattr_latency": { 2024-02-02T15:47:01.234 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 66, 2024-02-02T15:47:01.234 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 1.760933400, 2024-02-02T15:47:01.234 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.026680809 2024-02-02T15:47:01.234 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:01.234 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setdirlayout_latency": { 2024-02-02T15:47:01.235 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:01.235 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:01.235 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:01.235 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:01.235 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setfilelock_latency": { 2024-02-02T15:47:01.235 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 520, 2024-02-02T15:47:01.235 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 1.205992048, 2024-02-02T15:47:01.235 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.002319215 2024-02-02T15:47:01.236 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:01.236 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setlayout_latency": { 2024-02-02T15:47:01.236 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:01.236 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:01.236 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:01.236 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:01.236 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setxattr_latency": { 2024-02-02T15:47:01.236 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 7, 2024-02-02T15:47:01.236 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.020491045, 2024-02-02T15:47:01.237 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.002927292 2024-02-02T15:47:01.237 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:01.237 INFO:teuthology.orchestra.run.smithi071.stdout: "req_symlink_latency": { 2024-02-02T15:47:01.237 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:01.237 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:01.237 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:01.237 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:01.237 INFO:teuthology.orchestra.run.smithi071.stdout: "req_unlink_latency": { 2024-02-02T15:47:01.237 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 17348, 2024-02-02T15:47:01.238 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 100.511417194, 2024-02-02T15:47:01.238 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.005793833 2024-02-02T15:47:01.238 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:01.238 INFO:teuthology.orchestra.run.smithi071.stdout: "cap_revoke_eviction": 0, 2024-02-02T15:47:01.238 INFO:teuthology.orchestra.run.smithi071.stdout: "cap_acquisition_throttle": 0, 2024-02-02T15:47:01.238 INFO:teuthology.orchestra.run.smithi071.stdout: "req_getvxattr_latency": { 2024-02-02T15:47:01.238 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:01.238 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:01.238 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:01.239 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-02-02T15:47:01.239 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:01.239 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_sessions": { 2024-02-02T15:47:01.239 INFO:teuthology.orchestra.run.smithi071.stdout: "session_count": 2, 2024-02-02T15:47:01.239 INFO:teuthology.orchestra.run.smithi071.stdout: "session_add": 7, 2024-02-02T15:47:01.239 INFO:teuthology.orchestra.run.smithi071.stdout: "session_remove": 5, 2024-02-02T15:47:01.239 INFO:teuthology.orchestra.run.smithi071.stdout: "sessions_open": 2, 2024-02-02T15:47:01.239 INFO:teuthology.orchestra.run.smithi071.stdout: "sessions_stale": 0, 2024-02-02T15:47:01.239 INFO:teuthology.orchestra.run.smithi071.stdout: "total_load": 10933, 2024-02-02T15:47:01.240 INFO:teuthology.orchestra.run.smithi071.stdout: "average_load": 5466, 2024-02-02T15:47:01.240 INFO:teuthology.orchestra.run.smithi071.stdout: "avg_session_uptime": 996, 2024-02-02T15:47:01.240 INFO:teuthology.orchestra.run.smithi071.stdout: "mdthresh_evicted": 0 2024-02-02T15:47:01.240 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:01.240 INFO:teuthology.orchestra.run.smithi071.stdout: "mempool": { 2024-02-02T15:47:01.240 INFO:teuthology.orchestra.run.smithi071.stdout: "bloom_filter_bytes": 0, 2024-02-02T15:47:01.240 INFO:teuthology.orchestra.run.smithi071.stdout: "bloom_filter_items": 0, 2024-02-02T15:47:01.240 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_alloc_bytes": 0, 2024-02-02T15:47:01.240 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_alloc_items": 0, 2024-02-02T15:47:01.241 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_data_bytes": 0, 2024-02-02T15:47:01.241 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_data_items": 0, 2024-02-02T15:47:01.241 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-02T15:47:01.241 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_onode_items": 0, 2024-02-02T15:47:01.241 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-02T15:47:01.241 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_meta_items": 0, 2024-02-02T15:47:01.241 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_other_bytes": 0, 2024-02-02T15:47:01.241 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_other_items": 0, 2024-02-02T15:47:01.241 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-02T15:47:01.242 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_buffer_items": 0, 2024-02-02T15:47:01.242 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_extent_bytes": 0, 2024-02-02T15:47:01.242 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_extent_items": 0, 2024-02-02T15:47:01.242 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_blob_bytes": 0, 2024-02-02T15:47:01.242 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_blob_items": 0, 2024-02-02T15:47:01.242 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-02T15:47:01.242 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_shared_blob_items": 0, 2024-02-02T15:47:01.242 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-02T15:47:01.242 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_inline_bl_items": 0, 2024-02-02T15:47:01.243 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_bytes": 0, 2024-02-02T15:47:01.243 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_items": 0, 2024-02-02T15:47:01.243 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_txc_bytes": 0, 2024-02-02T15:47:01.243 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_txc_items": 0, 2024-02-02T15:47:01.243 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-02T15:47:01.243 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_deferred_items": 0, 2024-02-02T15:47:01.243 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_bytes": 0, 2024-02-02T15:47:01.243 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_items": 0, 2024-02-02T15:47:01.244 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_bytes": 0, 2024-02-02T15:47:01.244 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_items": 0, 2024-02-02T15:47:01.244 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_reader_bytes": 0, 2024-02-02T15:47:01.244 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_reader_items": 0, 2024-02-02T15:47:01.244 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_writer_bytes": 0, 2024-02-02T15:47:01.244 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_writer_items": 0, 2024-02-02T15:47:01.244 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_anon_bytes": 319941, 2024-02-02T15:47:01.244 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_anon_items": 646, 2024-02-02T15:47:01.245 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_meta_bytes": 176, 2024-02-02T15:47:01.245 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_meta_items": 2, 2024-02-02T15:47:01.245 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_bytes": 0, 2024-02-02T15:47:01.245 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_items": 0, 2024-02-02T15:47:01.245 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mapbl_bytes": 0, 2024-02-02T15:47:01.245 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mapbl_items": 0, 2024-02-02T15:47:01.245 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pglog_bytes": 0, 2024-02-02T15:47:01.245 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pglog_items": 0, 2024-02-02T15:47:01.245 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_bytes": 9608, 2024-02-02T15:47:01.245 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_items": 184, 2024-02-02T15:47:01.246 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_mapping_bytes": 0, 2024-02-02T15:47:01.246 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_mapping_items": 0, 2024-02-02T15:47:01.246 INFO:teuthology.orchestra.run.smithi071.stdout: "pgmap_bytes": 0, 2024-02-02T15:47:01.246 INFO:teuthology.orchestra.run.smithi071.stdout: "pgmap_items": 0, 2024-02-02T15:47:01.246 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_co_bytes": 4292498, 2024-02-02T15:47:01.246 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_co_items": 39250, 2024-02-02T15:47:01.246 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_1_bytes": 0, 2024-02-02T15:47:01.246 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_1_items": 0, 2024-02-02T15:47:01.246 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_2_bytes": 0, 2024-02-02T15:47:01.247 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_2_items": 0 2024-02-02T15:47:01.247 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:01.247 INFO:teuthology.orchestra.run.smithi071.stdout: "objecter": { 2024-02-02T15:47:01.247 INFO:teuthology.orchestra.run.smithi071.stdout: "op_active": 0, 2024-02-02T15:47:01.247 INFO:teuthology.orchestra.run.smithi071.stdout: "op_laggy": 0, 2024-02-02T15:47:01.247 INFO:teuthology.orchestra.run.smithi071.stdout: "op_send": 99469, 2024-02-02T15:47:01.247 INFO:teuthology.orchestra.run.smithi071.stdout: "op_send_bytes": 806941887, 2024-02-02T15:47:01.247 INFO:teuthology.orchestra.run.smithi071.stdout: "op_resend": 37, 2024-02-02T15:47:01.248 INFO:teuthology.orchestra.run.smithi071.stdout: "op_reply": 99469, 2024-02-02T15:47:01.248 INFO:teuthology.orchestra.run.smithi071.stdout: "op_latency": { 2024-02-02T15:47:01.248 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 99469, 2024-02-02T15:47:01.248 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 772.735558266, 2024-02-02T15:47:01.248 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.007768606 2024-02-02T15:47:01.248 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:01.248 INFO:teuthology.orchestra.run.smithi071.stdout: "op_inflight": 0, 2024-02-02T15:47:01.248 INFO:teuthology.orchestra.run.smithi071.stdout: "oplen_avg": { 2024-02-02T15:47:01.248 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 99469, 2024-02-02T15:47:01.248 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 100052 2024-02-02T15:47:01.249 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:01.249 INFO:teuthology.orchestra.run.smithi071.stdout: "op": 99469, 2024-02-02T15:47:01.249 INFO:teuthology.orchestra.run.smithi071.stdout: "op_r": 3693, 2024-02-02T15:47:01.249 INFO:teuthology.orchestra.run.smithi071.stdout: "op_w": 95776, 2024-02-02T15:47:01.249 INFO:teuthology.orchestra.run.smithi071.stdout: "op_rmw": 0, 2024-02-02T15:47:01.249 INFO:teuthology.orchestra.run.smithi071.stdout: "op_pg": 0, 2024-02-02T15:47:01.249 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_stat": 0, 2024-02-02T15:47:01.249 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_create": 41, 2024-02-02T15:47:01.249 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_read": 830, 2024-02-02T15:47:01.250 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_write": 59410, 2024-02-02T15:47:01.250 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_writefull": 504, 2024-02-02T15:47:01.250 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_writesame": 0, 2024-02-02T15:47:01.250 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_append": 0, 2024-02-02T15:47:01.250 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_zero": 0, 2024-02-02T15:47:01.250 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_truncate": 7, 2024-02-02T15:47:01.250 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_delete": 35499, 2024-02-02T15:47:01.250 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_mapext": 0, 2024-02-02T15:47:01.250 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_sparse_read": 0, 2024-02-02T15:47:01.250 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_clonerange": 0, 2024-02-02T15:47:01.251 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_getxattr": 2857, 2024-02-02T15:47:01.251 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_setxattr": 78, 2024-02-02T15:47:01.251 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_cmpxattr": 0, 2024-02-02T15:47:01.251 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_rmxattr": 0, 2024-02-02T15:47:01.251 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_resetxattrs": 0, 2024-02-02T15:47:01.251 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_call": 0, 2024-02-02T15:47:01.251 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_watch": 0, 2024-02-02T15:47:01.251 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_notify": 0, 2024-02-02T15:47:01.252 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_src_cmpxattr": 0, 2024-02-02T15:47:01.252 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_pgls": 0, 2024-02-02T15:47:01.252 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_pgls_filter": 0, 2024-02-02T15:47:01.252 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_other": 0, 2024-02-02T15:47:01.252 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_active": 0, 2024-02-02T15:47:01.252 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_send": 0, 2024-02-02T15:47:01.252 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_resend": 0, 2024-02-02T15:47:01.252 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_ping": 0, 2024-02-02T15:47:01.252 INFO:teuthology.orchestra.run.smithi071.stdout: "poolop_active": 0, 2024-02-02T15:47:01.253 INFO:teuthology.orchestra.run.smithi071.stdout: "poolop_send": 0, 2024-02-02T15:47:01.253 INFO:teuthology.orchestra.run.smithi071.stdout: "poolop_resend": 0, 2024-02-02T15:47:01.253 INFO:teuthology.orchestra.run.smithi071.stdout: "poolstat_active": 0, 2024-02-02T15:47:01.253 INFO:teuthology.orchestra.run.smithi071.stdout: "poolstat_send": 0, 2024-02-02T15:47:01.253 INFO:teuthology.orchestra.run.smithi071.stdout: "poolstat_resend": 0, 2024-02-02T15:47:01.253 INFO:teuthology.orchestra.run.smithi071.stdout: "statfs_active": 0, 2024-02-02T15:47:01.253 INFO:teuthology.orchestra.run.smithi071.stdout: "statfs_send": 0, 2024-02-02T15:47:01.253 INFO:teuthology.orchestra.run.smithi071.stdout: "statfs_resend": 0, 2024-02-02T15:47:01.253 INFO:teuthology.orchestra.run.smithi071.stdout: "command_active": 0, 2024-02-02T15:47:01.254 INFO:teuthology.orchestra.run.smithi071.stdout: "command_send": 0, 2024-02-02T15:47:01.254 INFO:teuthology.orchestra.run.smithi071.stdout: "command_resend": 0, 2024-02-02T15:47:01.254 INFO:teuthology.orchestra.run.smithi071.stdout: "map_epoch": 115, 2024-02-02T15:47:01.254 INFO:teuthology.orchestra.run.smithi071.stdout: "map_full": 0, 2024-02-02T15:47:01.254 INFO:teuthology.orchestra.run.smithi071.stdout: "map_inc": 36, 2024-02-02T15:47:01.254 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_sessions": 12, 2024-02-02T15:47:01.254 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_session_open": 12, 2024-02-02T15:47:01.254 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_session_close": 0, 2024-02-02T15:47:01.255 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_laggy": 0, 2024-02-02T15:47:01.255 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_wr": 597, 2024-02-02T15:47:01.255 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_rd": 9, 2024-02-02T15:47:01.255 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_del": 220 2024-02-02T15:47:01.255 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:01.255 INFO:teuthology.orchestra.run.smithi071.stdout: "oft": { 2024-02-02T15:47:01.255 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_objs": 1, 2024-02-02T15:47:01.255 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_kv_pairs": 0, 2024-02-02T15:47:01.255 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_updates": 0, 2024-02-02T15:47:01.256 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_removes": 0 2024-02-02T15:47:01.256 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:01.256 INFO:teuthology.orchestra.run.smithi071.stdout: "osd-slow-ops": { 2024-02-02T15:47:01.256 INFO:teuthology.orchestra.run.smithi071.stdout: "slow_ops_count": 216300 2024-02-02T15:47:01.256 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:01.256 INFO:teuthology.orchestra.run.smithi071.stdout: "purge_queue": { 2024-02-02T15:47:01.256 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing_ops": 0, 2024-02-02T15:47:01.256 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing_ops_high_water": 10, 2024-02-02T15:47:01.256 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing": 0, 2024-02-02T15:47:01.257 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing_high_water": 5, 2024-02-02T15:47:01.257 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executed": 17348, 2024-02-02T15:47:01.257 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_item_in_journal": 0 2024-02-02T15:47:01.257 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:01.257 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-02T15:47:01.257 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-02-02T15:47:01.257 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 104857600, 2024-02-02T15:47:01.257 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-02-02T15:47:01.257 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 1046347, 2024-02-02T15:47:01.258 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 184881451, 2024-02-02T15:47:01.258 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-02-02T15:47:01.258 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 1046347, 2024-02-02T15:47:01.258 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 0, 2024-02-02T15:47:01.258 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 0, 2024-02-02T15:47:01.258 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 1046347, 2024-02-02T15:47:01.258 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 184881451, 2024-02-02T15:47:01.258 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-02-02T15:47:01.258 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:01.259 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:01.259 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:01.259 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-02-02T15:47:01.259 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:01.259 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-objecter_bytes": { 2024-02-02T15:47:01.259 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-02-02T15:47:01.259 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 104857600, 2024-02-02T15:47:01.259 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-02-02T15:47:01.259 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 0, 2024-02-02T15:47:01.260 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 0, 2024-02-02T15:47:01.260 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-02-02T15:47:01.260 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 0, 2024-02-02T15:47:01.260 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 99469, 2024-02-02T15:47:01.260 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 808835550, 2024-02-02T15:47:01.260 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 99469, 2024-02-02T15:47:01.260 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 808835550, 2024-02-02T15:47:01.260 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-02-02T15:47:01.260 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:01.261 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:01.261 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:01.261 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-02-02T15:47:01.261 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:01.261 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-objecter_ops": { 2024-02-02T15:47:01.261 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-02-02T15:47:01.261 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 1024, 2024-02-02T15:47:01.261 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-02-02T15:47:01.261 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 0, 2024-02-02T15:47:01.262 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 0, 2024-02-02T15:47:01.262 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-02-02T15:47:01.262 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 0, 2024-02-02T15:47:01.262 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 99469, 2024-02-02T15:47:01.262 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 99469, 2024-02-02T15:47:01.262 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 99469, 2024-02-02T15:47:01.262 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 99469, 2024-02-02T15:47:01.262 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-02-02T15:47:01.263 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:01.263 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:01.263 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:01.263 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-02-02T15:47:01.263 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:01.263 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-write_buf_throttle": { 2024-02-02T15:47:01.263 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-02-02T15:47:01.263 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 3758096384, 2024-02-02T15:47:01.263 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-02-02T15:47:01.264 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 17348, 2024-02-02T15:47:01.264 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 1893804, 2024-02-02T15:47:01.264 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-02-02T15:47:01.264 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 17348, 2024-02-02T15:47:01.264 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 0, 2024-02-02T15:47:01.264 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 0, 2024-02-02T15:47:01.264 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 795, 2024-02-02T15:47:01.264 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 1893804, 2024-02-02T15:47:01.265 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-02-02T15:47:01.265 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:01.265 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:01.265 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:01.265 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-02-02T15:47:01.265 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:01.265 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-write_buf_throttle-0x55cb5c2d2000": { 2024-02-02T15:47:01.265 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 84, 2024-02-02T15:47:01.265 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 3758096384, 2024-02-02T15:47:01.266 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-02-02T15:47:01.266 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 120676, 2024-02-02T15:47:01.266 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 802903610, 2024-02-02T15:47:01.266 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-02-02T15:47:01.266 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 120676, 2024-02-02T15:47:01.266 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 0, 2024-02-02T15:47:01.266 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 0, 2024-02-02T15:47:01.266 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 58615, 2024-02-02T15:47:01.267 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 802903526, 2024-02-02T15:47:01.267 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-02-02T15:47:01.267 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:01.267 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:01.267 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:01.267 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-02-02T15:47:01.267 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-02-02T15:47:01.267 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-02-02T15:47:01.546 INFO:tasks.check_counter:Daemon mds.b mds.exported=28 2024-02-02T15:47:01.546 INFO:tasks.check_counter:Daemon mds.b mds.imported=15 2024-02-02T15:47:01.546 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=564 2024-02-02T15:47:01.546 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=560 2024-02-02T15:47:01.547 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-02-02T15:47:01.547 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=20 2024-02-02T15:47:01.547 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=14 2024-02-02T15:47:01.547 DEBUG:tasks.check_counter:Getting stats from e 2024-02-02T15:47:01.547 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph daemon mds.e perf dump 2024-02-02T15:47:01.786 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.b/config 2024-02-02T15:47:01.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:47:01 smithi071 ceph-mon[44707]: pgmap v629: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 155 KiB/s wr, 15 op/s 2024-02-02T15:47:01.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:47:01 smithi142 ceph-mon[45728]: pgmap v629: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 155 KiB/s wr, 15 op/s 2024-02-02T15:47:01.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:47:01 smithi001 ceph-mon[35470]: pgmap v629: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 155 KiB/s wr, 15 op/s 2024-02-02T15:47:02.728 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-02-02T15:47:02.729 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-0": { 2024-02-02T15:47:02.729 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 2821, 2024-02-02T15:47:02.729 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 2822, 2024-02-02T15:47:02.729 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 302815213, 2024-02-02T15:47:02.729 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 877030, 2024-02-02T15:47:02.729 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 8, 2024-02-02T15:47:02.730 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 4, 2024-02-02T15:47:02.730 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 0.341463087, 2024-02-02T15:47:02.730 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 0.068871300, 2024-02-02T15:47:02.730 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 0.185691026, 2024-02-02T15:47:02.730 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.067049333, 2024-02-02T15:47:02.730 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-02-02T15:47:02.730 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 2820, 2024-02-02T15:47:02.730 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.158438582, 2024-02-02T15:47:02.730 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000056183 2024-02-02T15:47:02.731 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:02.731 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-02-02T15:47:02.731 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:02.731 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:02.731 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:02.731 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:02.731 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 0, 2024-02-02T15:47:02.731 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 0 2024-02-02T15:47:02.732 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:02.732 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-1": { 2024-02-02T15:47:02.732 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 1512, 2024-02-02T15:47:02.732 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 1783, 2024-02-02T15:47:02.732 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 308381487, 2024-02-02T15:47:02.732 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 1093392, 2024-02-02T15:47:02.732 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 9, 2024-02-02T15:47:02.732 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 7, 2024-02-02T15:47:02.733 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 0.286757716, 2024-02-02T15:47:02.733 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 0.034049415, 2024-02-02T15:47:02.733 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 0.205923106, 2024-02-02T15:47:02.733 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.035138034, 2024-02-02T15:47:02.733 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-02-02T15:47:02.733 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 1781, 2024-02-02T15:47:02.733 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.111911485, 2024-02-02T15:47:02.734 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000062836 2024-02-02T15:47:02.734 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:02.734 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-02-02T15:47:02.734 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:02.734 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:02.734 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:02.734 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:02.734 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-02-02T15:47:02.734 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 624960 2024-02-02T15:47:02.735 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:02.735 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-2": { 2024-02-02T15:47:02.735 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 1528, 2024-02-02T15:47:02.735 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 1458, 2024-02-02T15:47:02.735 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 195454503, 2024-02-02T15:47:02.735 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 542612, 2024-02-02T15:47:02.735 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 5, 2024-02-02T15:47:02.735 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 4, 2024-02-02T15:47:02.735 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 0.242114101, 2024-02-02T15:47:02.736 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 0.036603537, 2024-02-02T15:47:02.736 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 0.176390103, 2024-02-02T15:47:02.736 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.034310736, 2024-02-02T15:47:02.736 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-02-02T15:47:02.736 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 1457, 2024-02-02T15:47:02.736 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.066265668, 2024-02-02T15:47:02.736 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000045480 2024-02-02T15:47:02.736 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:02.737 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-02-02T15:47:02.737 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:02.737 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:02.737 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:02.737 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:02.737 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 2244672, 2024-02-02T15:47:02.737 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 212640 2024-02-02T15:47:02.737 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:02.737 INFO:teuthology.orchestra.run.smithi071.stdout: "cct": { 2024-02-02T15:47:02.738 INFO:teuthology.orchestra.run.smithi071.stdout: "total_workers": 1, 2024-02-02T15:47:02.738 INFO:teuthology.orchestra.run.smithi071.stdout: "unhealthy_workers": 0 2024-02-02T15:47:02.738 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:02.738 INFO:teuthology.orchestra.run.smithi071.stdout: "finisher-MDSRank": { 2024-02-02T15:47:02.738 INFO:teuthology.orchestra.run.smithi071.stdout: "queue_len": 0, 2024-02-02T15:47:02.738 INFO:teuthology.orchestra.run.smithi071.stdout: "complete_latency": { 2024-02-02T15:47:02.738 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 8865, 2024-02-02T15:47:02.738 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.191047568, 2024-02-02T15:47:02.738 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000021550 2024-02-02T15:47:02.739 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-02-02T15:47:02.739 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:02.739 INFO:teuthology.orchestra.run.smithi071.stdout: "finisher-PurgeQueue": { 2024-02-02T15:47:02.739 INFO:teuthology.orchestra.run.smithi071.stdout: "queue_len": 0, 2024-02-02T15:47:02.739 INFO:teuthology.orchestra.run.smithi071.stdout: "complete_latency": { 2024-02-02T15:47:02.739 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:02.739 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:02.739 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:02.740 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-02-02T15:47:02.740 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:02.740 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-02-02T15:47:02.740 INFO:teuthology.orchestra.run.smithi071.stdout: "request": 0, 2024-02-02T15:47:02.740 INFO:teuthology.orchestra.run.smithi071.stdout: "reply": 0, 2024-02-02T15:47:02.740 INFO:teuthology.orchestra.run.smithi071.stdout: "reply_latency": { 2024-02-02T15:47:02.740 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:02.740 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:02.741 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:02.741 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:02.741 INFO:teuthology.orchestra.run.smithi071.stdout: "slow_reply": 0, 2024-02-02T15:47:02.741 INFO:teuthology.orchestra.run.smithi071.stdout: "forward": 0, 2024-02-02T15:47:02.741 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_fetch_complete": 0, 2024-02-02T15:47:02.741 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_fetch_keys": 0, 2024-02-02T15:47:02.741 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_commit": 0, 2024-02-02T15:47:02.741 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_split": 0, 2024-02-02T15:47:02.741 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_merge": 0, 2024-02-02T15:47:02.742 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes": 18427, 2024-02-02T15:47:02.742 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_top": 2805, 2024-02-02T15:47:02.742 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_bottom": 15622, 2024-02-02T15:47:02.742 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_pin_tail": 0, 2024-02-02T15:47:02.742 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_pinned": 14420, 2024-02-02T15:47:02.742 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_expired": 0, 2024-02-02T15:47:02.742 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_with_caps": 0, 2024-02-02T15:47:02.742 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": 0, 2024-02-02T15:47:02.742 INFO:teuthology.orchestra.run.smithi071.stdout: "subtrees": 12, 2024-02-02T15:47:02.743 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse": 0, 2024-02-02T15:47:02.743 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_hit": 0, 2024-02-02T15:47:02.743 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_forward": 0, 2024-02-02T15:47:02.743 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_discover": 0, 2024-02-02T15:47:02.743 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_dir_fetch": 0, 2024-02-02T15:47:02.743 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_remote_ino": 0, 2024-02-02T15:47:02.743 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_lock": 0, 2024-02-02T15:47:02.743 INFO:teuthology.orchestra.run.smithi071.stdout: "load_cent": 0, 2024-02-02T15:47:02.744 INFO:teuthology.orchestra.run.smithi071.stdout: "q": 0, 2024-02-02T15:47:02.744 INFO:teuthology.orchestra.run.smithi071.stdout: "exported": 0, 2024-02-02T15:47:02.744 INFO:teuthology.orchestra.run.smithi071.stdout: "exported_inodes": 0, 2024-02-02T15:47:02.744 INFO:teuthology.orchestra.run.smithi071.stdout: "imported": 0, 2024-02-02T15:47:02.744 INFO:teuthology.orchestra.run.smithi071.stdout: "imported_inodes": 0, 2024-02-02T15:47:02.744 INFO:teuthology.orchestra.run.smithi071.stdout: "openino_dir_fetch": 0, 2024-02-02T15:47:02.744 INFO:teuthology.orchestra.run.smithi071.stdout: "openino_backtrace_fetch": 0, 2024-02-02T15:47:02.744 INFO:teuthology.orchestra.run.smithi071.stdout: "openino_peer_discover": 0, 2024-02-02T15:47:02.745 INFO:teuthology.orchestra.run.smithi071.stdout: "root_rfiles": 670, 2024-02-02T15:47:02.745 INFO:teuthology.orchestra.run.smithi071.stdout: "root_rbytes": 218701295, 2024-02-02T15:47:02.745 INFO:teuthology.orchestra.run.smithi071.stdout: "root_rsnaps": 6, 2024-02-02T15:47:02.745 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_backtrace_fetch": 0, 2024-02-02T15:47:02.745 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_set_tag": 0, 2024-02-02T15:47:02.745 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_backtrace_repaired": 0, 2024-02-02T15:47:02.745 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_inotable_repaired": 0, 2024-02-02T15:47:02.745 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_dir_inodes": 0, 2024-02-02T15:47:02.746 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_dir_base_inodes": 0, 2024-02-02T15:47:02.746 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_dirfrag_rstats": 0, 2024-02-02T15:47:02.746 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_file_inodes": 0, 2024-02-02T15:47:02.746 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_inode_file_caps": 0, 2024-02-02T15:47:02.746 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_revoke": 0, 2024-02-02T15:47:02.746 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_grant": 0, 2024-02-02T15:47:02.746 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_trunc": 0, 2024-02-02T15:47:02.746 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-02-02T15:47:02.747 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_flush_ack": 0, 2024-02-02T15:47:02.747 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_caps": 0, 2024-02-02T15:47:02.747 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_caps_dirty": 0, 2024-02-02T15:47:02.747 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_cap_release": 0, 2024-02-02T15:47:02.747 INFO:teuthology.orchestra.run.smithi071.stdout: "process_request_cap_release": 0 2024-02-02T15:47:02.747 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:02.747 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_cache": { 2024-02-02T15:47:02.747 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_update": 0, 2024-02-02T15:47:02.747 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_update_receipt": 0, 2024-02-02T15:47:02.748 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_try_discover": 0, 2024-02-02T15:47:02.748 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_send_discover": 0, 2024-02-02T15:47:02.748 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_handle_discover": 0, 2024-02-02T15:47:02.748 INFO:teuthology.orchestra.run.smithi071.stdout: "num_strays": 0, 2024-02-02T15:47:02.748 INFO:teuthology.orchestra.run.smithi071.stdout: "num_strays_delayed": 0, 2024-02-02T15:47:02.748 INFO:teuthology.orchestra.run.smithi071.stdout: "num_strays_enqueuing": 0, 2024-02-02T15:47:02.748 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_created": 0, 2024-02-02T15:47:02.748 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_enqueued": 0, 2024-02-02T15:47:02.749 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_reintegrated": 0, 2024-02-02T15:47:02.749 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_migrated": 0, 2024-02-02T15:47:02.749 INFO:teuthology.orchestra.run.smithi071.stdout: "num_recovering_processing": 0, 2024-02-02T15:47:02.749 INFO:teuthology.orchestra.run.smithi071.stdout: "num_recovering_enqueued": 0, 2024-02-02T15:47:02.749 INFO:teuthology.orchestra.run.smithi071.stdout: "num_recovering_prioritized": 0, 2024-02-02T15:47:02.749 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_started": 0, 2024-02-02T15:47:02.749 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_completed": 0, 2024-02-02T15:47:02.749 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_enqueue_scrub": 0, 2024-02-02T15:47:02.749 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_exportdir": 0, 2024-02-02T15:47:02.750 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_flush": 0, 2024-02-02T15:47:02.750 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_fragmentdir": 0, 2024-02-02T15:47:02.750 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_fragstats": 0, 2024-02-02T15:47:02.750 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_inodestats": 0 2024-02-02T15:47:02.750 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:02.750 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_log": { 2024-02-02T15:47:02.750 INFO:teuthology.orchestra.run.smithi071.stdout: "evadd": 0, 2024-02-02T15:47:02.751 INFO:teuthology.orchestra.run.smithi071.stdout: "evex": 0, 2024-02-02T15:47:02.751 INFO:teuthology.orchestra.run.smithi071.stdout: "evtrm": 0, 2024-02-02T15:47:02.751 INFO:teuthology.orchestra.run.smithi071.stdout: "ev": 120677, 2024-02-02T15:47:02.751 INFO:teuthology.orchestra.run.smithi071.stdout: "evexg": 0, 2024-02-02T15:47:02.751 INFO:teuthology.orchestra.run.smithi071.stdout: "evexd": 0, 2024-02-02T15:47:02.751 INFO:teuthology.orchestra.run.smithi071.stdout: "segadd": 0, 2024-02-02T15:47:02.751 INFO:teuthology.orchestra.run.smithi071.stdout: "segex": 0, 2024-02-02T15:47:02.751 INFO:teuthology.orchestra.run.smithi071.stdout: "segtrm": 0, 2024-02-02T15:47:02.751 INFO:teuthology.orchestra.run.smithi071.stdout: "seg": 129, 2024-02-02T15:47:02.752 INFO:teuthology.orchestra.run.smithi071.stdout: "segexg": 0, 2024-02-02T15:47:02.752 INFO:teuthology.orchestra.run.smithi071.stdout: "segexd": 0, 2024-02-02T15:47:02.752 INFO:teuthology.orchestra.run.smithi071.stdout: "expos": 289433354, 2024-02-02T15:47:02.752 INFO:teuthology.orchestra.run.smithi071.stdout: "wrpos": 807102841, 2024-02-02T15:47:02.752 INFO:teuthology.orchestra.run.smithi071.stdout: "rdpos": 807097914, 2024-02-02T15:47:02.752 INFO:teuthology.orchestra.run.smithi071.stdout: "jlat": { 2024-02-02T15:47:02.752 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:02.752 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:02.753 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:02.753 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:02.753 INFO:teuthology.orchestra.run.smithi071.stdout: "replayed": 120677 2024-02-02T15:47:02.753 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:02.753 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_mem": { 2024-02-02T15:47:02.753 INFO:teuthology.orchestra.run.smithi071.stdout: "ino": 914, 2024-02-02T15:47:02.753 INFO:teuthology.orchestra.run.smithi071.stdout: "ino+": 19557, 2024-02-02T15:47:02.753 INFO:teuthology.orchestra.run.smithi071.stdout: "ino-": 18643, 2024-02-02T15:47:02.753 INFO:teuthology.orchestra.run.smithi071.stdout: "dir": 67, 2024-02-02T15:47:02.754 INFO:teuthology.orchestra.run.smithi071.stdout: "dir+": 193, 2024-02-02T15:47:02.754 INFO:teuthology.orchestra.run.smithi071.stdout: "dir-": 126, 2024-02-02T15:47:02.754 INFO:teuthology.orchestra.run.smithi071.stdout: "dn": 18427, 2024-02-02T15:47:02.754 INFO:teuthology.orchestra.run.smithi071.stdout: "dn+": 20338, 2024-02-02T15:47:02.754 INFO:teuthology.orchestra.run.smithi071.stdout: "dn-": 1911, 2024-02-02T15:47:02.754 INFO:teuthology.orchestra.run.smithi071.stdout: "cap": 0, 2024-02-02T15:47:02.754 INFO:teuthology.orchestra.run.smithi071.stdout: "cap+": 0, 2024-02-02T15:47:02.754 INFO:teuthology.orchestra.run.smithi071.stdout: "cap-": 0, 2024-02-02T15:47:02.755 INFO:teuthology.orchestra.run.smithi071.stdout: "rss": 73072, 2024-02-02T15:47:02.755 INFO:teuthology.orchestra.run.smithi071.stdout: "heap": 207156 2024-02-02T15:47:02.755 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:02.755 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_server": { 2024-02-02T15:47:02.755 INFO:teuthology.orchestra.run.smithi071.stdout: "dispatch_client_request": 0, 2024-02-02T15:47:02.755 INFO:teuthology.orchestra.run.smithi071.stdout: "dispatch_server_request": 0, 2024-02-02T15:47:02.755 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_request": 0, 2024-02-02T15:47:02.755 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_session": 0, 2024-02-02T15:47:02.756 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_peer_request": 0, 2024-02-02T15:47:02.756 INFO:teuthology.orchestra.run.smithi071.stdout: "req_create_latency": { 2024-02-02T15:47:02.756 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:02.756 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:02.756 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:02.756 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:02.757 INFO:teuthology.orchestra.run.smithi071.stdout: "req_getattr_latency": { 2024-02-02T15:47:02.757 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:02.757 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:02.757 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:02.757 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:02.757 INFO:teuthology.orchestra.run.smithi071.stdout: "req_getfilelock_latency": { 2024-02-02T15:47:02.757 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:02.757 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:02.757 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:02.758 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:02.758 INFO:teuthology.orchestra.run.smithi071.stdout: "req_link_latency": { 2024-02-02T15:47:02.758 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:02.758 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:02.758 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:02.758 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:02.758 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookup_latency": { 2024-02-02T15:47:02.758 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:02.759 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:02.759 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:02.759 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:02.759 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookuphash_latency": { 2024-02-02T15:47:02.759 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:02.759 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:02.759 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:02.759 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:02.760 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupino_latency": { 2024-02-02T15:47:02.760 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:02.760 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:02.760 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:02.760 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:02.760 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupname_latency": { 2024-02-02T15:47:02.760 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:02.760 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:02.761 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:02.761 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:02.761 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupparent_latency": { 2024-02-02T15:47:02.761 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:02.761 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:02.761 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:02.761 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:02.761 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupsnap_latency": { 2024-02-02T15:47:02.762 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:02.762 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:02.762 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:02.762 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:02.762 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lssnap_latency": { 2024-02-02T15:47:02.762 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:02.762 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:02.762 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:02.763 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:02.763 INFO:teuthology.orchestra.run.smithi071.stdout: "req_mkdir_latency": { 2024-02-02T15:47:02.763 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:02.763 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:02.763 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:02.763 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:02.763 INFO:teuthology.orchestra.run.smithi071.stdout: "req_mknod_latency": { 2024-02-02T15:47:02.764 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:02.764 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:02.764 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:02.764 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:02.764 INFO:teuthology.orchestra.run.smithi071.stdout: "req_mksnap_latency": { 2024-02-02T15:47:02.764 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:02.764 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:02.764 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:02.764 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:02.765 INFO:teuthology.orchestra.run.smithi071.stdout: "req_open_latency": { 2024-02-02T15:47:02.765 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:02.765 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:02.765 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:02.765 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:02.765 INFO:teuthology.orchestra.run.smithi071.stdout: "req_readdir_latency": { 2024-02-02T15:47:02.765 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:02.765 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:02.766 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:02.766 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:02.766 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rename_latency": { 2024-02-02T15:47:02.766 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:02.766 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:02.766 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:02.766 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:02.766 INFO:teuthology.orchestra.run.smithi071.stdout: "req_renamesnap_latency": { 2024-02-02T15:47:02.766 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:02.767 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:02.767 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:02.767 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:02.767 INFO:teuthology.orchestra.run.smithi071.stdout: "req_snapdiff_latency": { 2024-02-02T15:47:02.767 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:02.767 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:02.767 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:02.767 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:02.767 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rmdir_latency": { 2024-02-02T15:47:02.768 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:02.768 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:02.768 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:02.768 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:02.768 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rmsnap_latency": { 2024-02-02T15:47:02.768 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:02.768 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:02.768 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:02.769 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:02.769 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rmxattr_latency": { 2024-02-02T15:47:02.769 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:02.769 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:02.769 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:02.769 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:02.769 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setattr_latency": { 2024-02-02T15:47:02.769 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:02.770 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:02.770 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:02.770 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:02.770 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setdirlayout_latency": { 2024-02-02T15:47:02.770 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:02.770 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:02.770 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:02.770 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:02.770 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setfilelock_latency": { 2024-02-02T15:47:02.770 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:02.771 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:02.771 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:02.771 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:02.771 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setlayout_latency": { 2024-02-02T15:47:02.771 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:02.771 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:02.771 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:02.771 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:02.771 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setxattr_latency": { 2024-02-02T15:47:02.772 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:02.772 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:02.772 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:02.772 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:02.772 INFO:teuthology.orchestra.run.smithi071.stdout: "req_symlink_latency": { 2024-02-02T15:47:02.772 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:02.772 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:02.772 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:02.772 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:02.773 INFO:teuthology.orchestra.run.smithi071.stdout: "req_unlink_latency": { 2024-02-02T15:47:02.773 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:02.773 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:02.773 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:02.773 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:02.773 INFO:teuthology.orchestra.run.smithi071.stdout: "cap_revoke_eviction": 0, 2024-02-02T15:47:02.773 INFO:teuthology.orchestra.run.smithi071.stdout: "cap_acquisition_throttle": 0, 2024-02-02T15:47:02.773 INFO:teuthology.orchestra.run.smithi071.stdout: "req_getvxattr_latency": { 2024-02-02T15:47:02.773 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:02.774 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:02.774 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:02.774 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-02-02T15:47:02.774 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:02.774 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_sessions": { 2024-02-02T15:47:02.774 INFO:teuthology.orchestra.run.smithi071.stdout: "session_count": 2, 2024-02-02T15:47:02.774 INFO:teuthology.orchestra.run.smithi071.stdout: "session_add": 7, 2024-02-02T15:47:02.774 INFO:teuthology.orchestra.run.smithi071.stdout: "session_remove": 5, 2024-02-02T15:47:02.774 INFO:teuthology.orchestra.run.smithi071.stdout: "sessions_open": 2, 2024-02-02T15:47:02.775 INFO:teuthology.orchestra.run.smithi071.stdout: "sessions_stale": 0, 2024-02-02T15:47:02.775 INFO:teuthology.orchestra.run.smithi071.stdout: "total_load": 0, 2024-02-02T15:47:02.775 INFO:teuthology.orchestra.run.smithi071.stdout: "average_load": 0, 2024-02-02T15:47:02.775 INFO:teuthology.orchestra.run.smithi071.stdout: "avg_session_uptime": 8058, 2024-02-02T15:47:02.775 INFO:teuthology.orchestra.run.smithi071.stdout: "mdthresh_evicted": 0 2024-02-02T15:47:02.775 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:02.775 INFO:teuthology.orchestra.run.smithi071.stdout: "mempool": { 2024-02-02T15:47:02.775 INFO:teuthology.orchestra.run.smithi071.stdout: "bloom_filter_bytes": 0, 2024-02-02T15:47:02.775 INFO:teuthology.orchestra.run.smithi071.stdout: "bloom_filter_items": 0, 2024-02-02T15:47:02.776 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_alloc_bytes": 0, 2024-02-02T15:47:02.776 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_alloc_items": 0, 2024-02-02T15:47:02.776 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_data_bytes": 0, 2024-02-02T15:47:02.776 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_data_items": 0, 2024-02-02T15:47:02.776 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-02T15:47:02.776 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_onode_items": 0, 2024-02-02T15:47:02.776 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-02T15:47:02.776 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_meta_items": 0, 2024-02-02T15:47:02.777 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_other_bytes": 0, 2024-02-02T15:47:02.777 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_other_items": 0, 2024-02-02T15:47:02.777 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-02T15:47:02.777 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_buffer_items": 0, 2024-02-02T15:47:02.777 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_extent_bytes": 0, 2024-02-02T15:47:02.777 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_extent_items": 0, 2024-02-02T15:47:02.777 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_blob_bytes": 0, 2024-02-02T15:47:02.777 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_blob_items": 0, 2024-02-02T15:47:02.778 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-02T15:47:02.778 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_shared_blob_items": 0, 2024-02-02T15:47:02.778 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-02T15:47:02.778 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_inline_bl_items": 0, 2024-02-02T15:47:02.778 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_bytes": 0, 2024-02-02T15:47:02.778 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_items": 0, 2024-02-02T15:47:02.778 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_txc_bytes": 0, 2024-02-02T15:47:02.778 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_txc_items": 0, 2024-02-02T15:47:02.778 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-02T15:47:02.779 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_deferred_items": 0, 2024-02-02T15:47:02.779 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_bytes": 0, 2024-02-02T15:47:02.779 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_items": 0, 2024-02-02T15:47:02.779 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_bytes": 0, 2024-02-02T15:47:02.779 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_items": 0, 2024-02-02T15:47:02.779 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_reader_bytes": 0, 2024-02-02T15:47:02.779 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_reader_items": 0, 2024-02-02T15:47:02.779 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_writer_bytes": 0, 2024-02-02T15:47:02.779 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_writer_items": 0, 2024-02-02T15:47:02.780 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_anon_bytes": 280113, 2024-02-02T15:47:02.780 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_anon_items": 1254, 2024-02-02T15:47:02.780 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_meta_bytes": 0, 2024-02-02T15:47:02.780 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_meta_items": 0, 2024-02-02T15:47:02.780 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_bytes": 0, 2024-02-02T15:47:02.780 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_items": 0, 2024-02-02T15:47:02.780 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mapbl_bytes": 0, 2024-02-02T15:47:02.780 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mapbl_items": 0, 2024-02-02T15:47:02.781 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pglog_bytes": 0, 2024-02-02T15:47:02.781 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pglog_items": 0, 2024-02-02T15:47:02.781 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_bytes": 9608, 2024-02-02T15:47:02.781 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_items": 184, 2024-02-02T15:47:02.781 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_mapping_bytes": 0, 2024-02-02T15:47:02.781 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_mapping_items": 0, 2024-02-02T15:47:02.781 INFO:teuthology.orchestra.run.smithi071.stdout: "pgmap_bytes": 0, 2024-02-02T15:47:02.781 INFO:teuthology.orchestra.run.smithi071.stdout: "pgmap_items": 0, 2024-02-02T15:47:02.781 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_co_bytes": 15464742, 2024-02-02T15:47:02.782 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_co_items": 101237, 2024-02-02T15:47:02.782 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_1_bytes": 0, 2024-02-02T15:47:02.782 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_1_items": 0, 2024-02-02T15:47:02.782 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_2_bytes": 0, 2024-02-02T15:47:02.782 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_2_items": 0 2024-02-02T15:47:02.782 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:02.782 INFO:teuthology.orchestra.run.smithi071.stdout: "objecter": { 2024-02-02T15:47:02.782 INFO:teuthology.orchestra.run.smithi071.stdout: "op_active": 0, 2024-02-02T15:47:02.783 INFO:teuthology.orchestra.run.smithi071.stdout: "op_laggy": 0, 2024-02-02T15:47:02.783 INFO:teuthology.orchestra.run.smithi071.stdout: "op_send": 5387, 2024-02-02T15:47:02.783 INFO:teuthology.orchestra.run.smithi071.stdout: "op_send_bytes": 26, 2024-02-02T15:47:02.783 INFO:teuthology.orchestra.run.smithi071.stdout: "op_resend": 4, 2024-02-02T15:47:02.783 INFO:teuthology.orchestra.run.smithi071.stdout: "op_reply": 5387, 2024-02-02T15:47:02.783 INFO:teuthology.orchestra.run.smithi071.stdout: "op_latency": { 2024-02-02T15:47:02.783 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 5387, 2024-02-02T15:47:02.783 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 16.714709386, 2024-02-02T15:47:02.783 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.003102786 2024-02-02T15:47:02.784 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:02.784 INFO:teuthology.orchestra.run.smithi071.stdout: "op_inflight": 0, 2024-02-02T15:47:02.784 INFO:teuthology.orchestra.run.smithi071.stdout: "oplen_avg": { 2024-02-02T15:47:02.784 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 5387, 2024-02-02T15:47:02.784 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 5388 2024-02-02T15:47:02.784 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:02.784 INFO:teuthology.orchestra.run.smithi071.stdout: "op": 5387, 2024-02-02T15:47:02.784 INFO:teuthology.orchestra.run.smithi071.stdout: "op_r": 5387, 2024-02-02T15:47:02.785 INFO:teuthology.orchestra.run.smithi071.stdout: "op_w": 0, 2024-02-02T15:47:02.785 INFO:teuthology.orchestra.run.smithi071.stdout: "op_rmw": 0, 2024-02-02T15:47:02.785 INFO:teuthology.orchestra.run.smithi071.stdout: "op_pg": 0, 2024-02-02T15:47:02.785 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_stat": 2915, 2024-02-02T15:47:02.785 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_create": 0, 2024-02-02T15:47:02.785 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_read": 2469, 2024-02-02T15:47:02.785 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_write": 0, 2024-02-02T15:47:02.785 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_writefull": 0, 2024-02-02T15:47:02.785 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_writesame": 0, 2024-02-02T15:47:02.785 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_append": 0, 2024-02-02T15:47:02.786 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_zero": 0, 2024-02-02T15:47:02.786 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_truncate": 0, 2024-02-02T15:47:02.786 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_delete": 0, 2024-02-02T15:47:02.786 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_mapext": 0, 2024-02-02T15:47:02.786 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_sparse_read": 0, 2024-02-02T15:47:02.786 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_clonerange": 0, 2024-02-02T15:47:02.786 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_getxattr": 2, 2024-02-02T15:47:02.786 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_setxattr": 0, 2024-02-02T15:47:02.787 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_cmpxattr": 0, 2024-02-02T15:47:02.787 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_rmxattr": 0, 2024-02-02T15:47:02.787 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_resetxattrs": 0, 2024-02-02T15:47:02.787 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_call": 0, 2024-02-02T15:47:02.787 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_watch": 0, 2024-02-02T15:47:02.787 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_notify": 0, 2024-02-02T15:47:02.787 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_src_cmpxattr": 0, 2024-02-02T15:47:02.787 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_pgls": 0, 2024-02-02T15:47:02.787 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_pgls_filter": 0, 2024-02-02T15:47:02.788 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_other": 0, 2024-02-02T15:47:02.788 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_active": 0, 2024-02-02T15:47:02.788 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_send": 0, 2024-02-02T15:47:02.788 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_resend": 0, 2024-02-02T15:47:02.788 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_ping": 0, 2024-02-02T15:47:02.788 INFO:teuthology.orchestra.run.smithi071.stdout: "poolop_active": 0, 2024-02-02T15:47:02.788 INFO:teuthology.orchestra.run.smithi071.stdout: "poolop_send": 0, 2024-02-02T15:47:02.788 INFO:teuthology.orchestra.run.smithi071.stdout: "poolop_resend": 0, 2024-02-02T15:47:02.789 INFO:teuthology.orchestra.run.smithi071.stdout: "poolstat_active": 0, 2024-02-02T15:47:02.789 INFO:teuthology.orchestra.run.smithi071.stdout: "poolstat_send": 0, 2024-02-02T15:47:02.789 INFO:teuthology.orchestra.run.smithi071.stdout: "poolstat_resend": 0, 2024-02-02T15:47:02.789 INFO:teuthology.orchestra.run.smithi071.stdout: "statfs_active": 0, 2024-02-02T15:47:02.789 INFO:teuthology.orchestra.run.smithi071.stdout: "statfs_send": 0, 2024-02-02T15:47:02.789 INFO:teuthology.orchestra.run.smithi071.stdout: "statfs_resend": 0, 2024-02-02T15:47:02.789 INFO:teuthology.orchestra.run.smithi071.stdout: "command_active": 0, 2024-02-02T15:47:02.789 INFO:teuthology.orchestra.run.smithi071.stdout: "command_send": 0, 2024-02-02T15:47:02.790 INFO:teuthology.orchestra.run.smithi071.stdout: "command_resend": 0, 2024-02-02T15:47:02.790 INFO:teuthology.orchestra.run.smithi071.stdout: "map_epoch": 115, 2024-02-02T15:47:02.790 INFO:teuthology.orchestra.run.smithi071.stdout: "map_full": 0, 2024-02-02T15:47:02.790 INFO:teuthology.orchestra.run.smithi071.stdout: "map_inc": 31, 2024-02-02T15:47:02.790 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_sessions": 11, 2024-02-02T15:47:02.790 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_session_open": 11, 2024-02-02T15:47:02.790 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_session_close": 0, 2024-02-02T15:47:02.790 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_laggy": 0, 2024-02-02T15:47:02.791 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_wr": 0, 2024-02-02T15:47:02.791 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_rd": 2, 2024-02-02T15:47:02.791 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_del": 0 2024-02-02T15:47:02.791 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:02.791 INFO:teuthology.orchestra.run.smithi071.stdout: "oft": { 2024-02-02T15:47:02.791 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_objs": 0, 2024-02-02T15:47:02.791 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_kv_pairs": 0, 2024-02-02T15:47:02.791 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_updates": 0, 2024-02-02T15:47:02.792 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_removes": 0 2024-02-02T15:47:02.792 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:02.792 INFO:teuthology.orchestra.run.smithi071.stdout: "osd-slow-ops": { 2024-02-02T15:47:02.792 INFO:teuthology.orchestra.run.smithi071.stdout: "slow_ops_count": 0 2024-02-02T15:47:02.792 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:02.792 INFO:teuthology.orchestra.run.smithi071.stdout: "purge_queue": { 2024-02-02T15:47:02.792 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing_ops": 0, 2024-02-02T15:47:02.793 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing_ops_high_water": 0, 2024-02-02T15:47:02.793 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing": 0, 2024-02-02T15:47:02.793 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing_high_water": 0, 2024-02-02T15:47:02.793 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executed": 0, 2024-02-02T15:47:02.793 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_item_in_journal": 0 2024-02-02T15:47:02.793 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:02.793 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-02T15:47:02.793 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-02-02T15:47:02.793 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 104857600, 2024-02-02T15:47:02.794 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-02-02T15:47:02.794 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 5857, 2024-02-02T15:47:02.794 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 806097555, 2024-02-02T15:47:02.794 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-02-02T15:47:02.794 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 5857, 2024-02-02T15:47:02.794 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 0, 2024-02-02T15:47:02.794 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 0, 2024-02-02T15:47:02.794 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 5857, 2024-02-02T15:47:02.794 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 806097555, 2024-02-02T15:47:02.794 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-02-02T15:47:02.795 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:02.795 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:02.795 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:02.795 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-02-02T15:47:02.795 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:02.795 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-objecter_bytes": { 2024-02-02T15:47:02.795 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-02-02T15:47:02.796 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 104857600, 2024-02-02T15:47:02.796 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-02-02T15:47:02.796 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 0, 2024-02-02T15:47:02.796 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 0, 2024-02-02T15:47:02.796 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-02-02T15:47:02.796 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 0, 2024-02-02T15:47:02.796 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 5387, 2024-02-02T15:47:02.796 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 802922797, 2024-02-02T15:47:02.797 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 5387, 2024-02-02T15:47:02.797 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 802922797, 2024-02-02T15:47:02.797 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-02-02T15:47:02.797 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:02.797 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:02.797 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:02.797 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-02-02T15:47:02.797 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:02.797 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-objecter_ops": { 2024-02-02T15:47:02.798 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-02-02T15:47:02.798 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 1024, 2024-02-02T15:47:02.798 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-02-02T15:47:02.798 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 0, 2024-02-02T15:47:02.798 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 0, 2024-02-02T15:47:02.798 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-02-02T15:47:02.798 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 0, 2024-02-02T15:47:02.798 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 5387, 2024-02-02T15:47:02.799 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 5387, 2024-02-02T15:47:02.799 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 5387, 2024-02-02T15:47:02.799 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 5387, 2024-02-02T15:47:02.799 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-02-02T15:47:02.799 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:02.799 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:02.799 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:02.799 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-02-02T15:47:02.799 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:02.800 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-write_buf_throttle": { 2024-02-02T15:47:02.800 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-02-02T15:47:02.800 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 3758096384, 2024-02-02T15:47:02.800 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-02-02T15:47:02.800 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 0, 2024-02-02T15:47:02.800 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 0, 2024-02-02T15:47:02.800 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-02-02T15:47:02.800 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 0, 2024-02-02T15:47:02.801 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 0, 2024-02-02T15:47:02.801 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 0, 2024-02-02T15:47:02.801 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 0, 2024-02-02T15:47:02.801 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 0, 2024-02-02T15:47:02.801 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-02-02T15:47:02.801 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:02.801 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:02.801 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:02.801 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-02-02T15:47:02.802 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:02.802 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-write_buf_throttle-0x55c3786f4140": { 2024-02-02T15:47:02.802 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-02-02T15:47:02.802 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 3758096384, 2024-02-02T15:47:02.802 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-02-02T15:47:02.802 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 0, 2024-02-02T15:47:02.802 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 0, 2024-02-02T15:47:02.802 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-02-02T15:47:02.803 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 0, 2024-02-02T15:47:02.803 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 0, 2024-02-02T15:47:02.803 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 0, 2024-02-02T15:47:02.803 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 0, 2024-02-02T15:47:02.803 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 0, 2024-02-02T15:47:02.803 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-02-02T15:47:02.803 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:02.803 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:02.804 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:02.804 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-02-02T15:47:02.804 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-02-02T15:47:02.804 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-02-02T15:47:02.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:47:02 smithi071 ceph-mon[44707]: pgmap v630: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 189 KiB/s wr, 21 op/s 2024-02-02T15:47:03.101 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-02-02T15:47:03.101 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-02-02T15:47:03.101 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-02-02T15:47:03.102 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-02-02T15:47:03.102 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=6 2024-02-02T15:47:03.102 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-02-02T15:47:03.102 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-02-02T15:47:03.102 DEBUG:tasks.check_counter:Getting stats from h 2024-02-02T15:47:03.102 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph daemon mds.h perf dump 2024-02-02T15:47:03.169 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:47:02 smithi142 ceph-mon[45728]: pgmap v630: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 189 KiB/s wr, 21 op/s 2024-02-02T15:47:03.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:47:02 smithi001 ceph-mon[35470]: pgmap v630: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 189 KiB/s wr, 21 op/s 2024-02-02T15:47:03.326 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.b/config 2024-02-02T15:47:04.384 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-02-02T15:47:04.384 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-0": { 2024-02-02T15:47:04.384 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 93127, 2024-02-02T15:47:04.384 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 93664, 2024-02-02T15:47:04.384 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 25341033, 2024-02-02T15:47:04.384 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 193412596, 2024-02-02T15:47:04.385 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 81, 2024-02-02T15:47:04.385 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 7, 2024-02-02T15:47:04.385 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 5.240472615, 2024-02-02T15:47:04.385 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 1.950153881, 2024-02-02T15:47:04.385 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 2.788776145, 2024-02-02T15:47:04.385 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.848154672, 2024-02-02T15:47:04.385 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-02-02T15:47:04.385 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 93663, 2024-02-02T15:47:04.385 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 14.774816242, 2024-02-02T15:47:04.386 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000157744 2024-02-02T15:47:04.386 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:04.386 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-02-02T15:47:04.386 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 120190, 2024-02-02T15:47:04.386 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.020701553, 2024-02-02T15:47:04.386 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000172 2024-02-02T15:47:04.386 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:04.386 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 4720, 2024-02-02T15:47:04.386 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 510656 2024-02-02T15:47:04.387 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:04.387 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-1": { 2024-02-02T15:47:04.387 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 302716, 2024-02-02T15:47:04.387 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 392353, 2024-02-02T15:47:04.387 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 89268169, 2024-02-02T15:47:04.387 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 412245062, 2024-02-02T15:47:04.387 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 15, 2024-02-02T15:47:04.387 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 6, 2024-02-02T15:47:04.387 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 15.018159715, 2024-02-02T15:47:04.387 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 6.724428035, 2024-02-02T15:47:04.388 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 9.687704262, 2024-02-02T15:47:04.388 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.917094246, 2024-02-02T15:47:04.388 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-02-02T15:47:04.388 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 392350, 2024-02-02T15:47:04.388 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 49.839054563, 2024-02-02T15:47:04.388 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000127027 2024-02-02T15:47:04.388 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:04.388 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-02-02T15:47:04.388 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 375456, 2024-02-02T15:47:04.389 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.090875313, 2024-02-02T15:47:04.389 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000242 2024-02-02T15:47:04.389 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:04.389 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 2288, 2024-02-02T15:47:04.389 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 123136 2024-02-02T15:47:04.389 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:04.389 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-2": { 2024-02-02T15:47:04.389 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 308516, 2024-02-02T15:47:04.389 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 252663, 2024-02-02T15:47:04.390 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 94281341, 2024-02-02T15:47:04.390 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 133590122, 2024-02-02T15:47:04.390 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 14, 2024-02-02T15:47:04.390 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 6, 2024-02-02T15:47:04.390 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 15.723986843, 2024-02-02T15:47:04.390 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 6.762772381, 2024-02-02T15:47:04.390 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 11.314813455, 2024-02-02T15:47:04.390 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.377342179, 2024-02-02T15:47:04.390 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-02-02T15:47:04.391 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 252662, 2024-02-02T15:47:04.391 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 21.143286305, 2024-02-02T15:47:04.391 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000083682 2024-02-02T15:47:04.391 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:04.391 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-02-02T15:47:04.391 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 520371, 2024-02-02T15:47:04.391 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.079530201, 2024-02-02T15:47:04.391 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000152 2024-02-02T15:47:04.391 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:04.392 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 2283424, 2024-02-02T15:47:04.392 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 281440 2024-02-02T15:47:04.392 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:04.392 INFO:teuthology.orchestra.run.smithi071.stdout: "cct": { 2024-02-02T15:47:04.392 INFO:teuthology.orchestra.run.smithi071.stdout: "total_workers": 1, 2024-02-02T15:47:04.392 INFO:teuthology.orchestra.run.smithi071.stdout: "unhealthy_workers": 0 2024-02-02T15:47:04.392 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:04.392 INFO:teuthology.orchestra.run.smithi071.stdout: "finisher-MDSRank": { 2024-02-02T15:47:04.393 INFO:teuthology.orchestra.run.smithi071.stdout: "queue_len": 0, 2024-02-02T15:47:04.393 INFO:teuthology.orchestra.run.smithi071.stdout: "complete_latency": { 2024-02-02T15:47:04.393 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 61385, 2024-02-02T15:47:04.393 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 54.540279501, 2024-02-02T15:47:04.393 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000888495 2024-02-02T15:47:04.393 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-02-02T15:47:04.393 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:04.393 INFO:teuthology.orchestra.run.smithi071.stdout: "finisher-PurgeQueue": { 2024-02-02T15:47:04.393 INFO:teuthology.orchestra.run.smithi071.stdout: "queue_len": 0, 2024-02-02T15:47:04.394 INFO:teuthology.orchestra.run.smithi071.stdout: "complete_latency": { 2024-02-02T15:47:04.394 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 34427, 2024-02-02T15:47:04.394 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 3.389554988, 2024-02-02T15:47:04.394 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000098456 2024-02-02T15:47:04.394 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-02-02T15:47:04.394 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:04.394 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-02-02T15:47:04.394 INFO:teuthology.orchestra.run.smithi071.stdout: "request": 146009, 2024-02-02T15:47:04.395 INFO:teuthology.orchestra.run.smithi071.stdout: "reply": 121020, 2024-02-02T15:47:04.395 INFO:teuthology.orchestra.run.smithi071.stdout: "reply_latency": { 2024-02-02T15:47:04.395 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 121020, 2024-02-02T15:47:04.395 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 441.551119951, 2024-02-02T15:47:04.395 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.003648579 2024-02-02T15:47:04.395 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:04.395 INFO:teuthology.orchestra.run.smithi071.stdout: "slow_reply": 0, 2024-02-02T15:47:04.395 INFO:teuthology.orchestra.run.smithi071.stdout: "forward": 24988, 2024-02-02T15:47:04.396 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_fetch_complete": 6, 2024-02-02T15:47:04.396 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_fetch_keys": 0, 2024-02-02T15:47:04.396 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_commit": 29, 2024-02-02T15:47:04.396 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_split": 2, 2024-02-02T15:47:04.396 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_merge": 0, 2024-02-02T15:47:04.396 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes": 1024, 2024-02-02T15:47:04.396 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_top": 233, 2024-02-02T15:47:04.396 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_bottom": 665, 2024-02-02T15:47:04.397 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_pin_tail": 126, 2024-02-02T15:47:04.397 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_pinned": 692, 2024-02-02T15:47:04.397 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_expired": 1063, 2024-02-02T15:47:04.397 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_with_caps": 107, 2024-02-02T15:47:04.397 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": 107, 2024-02-02T15:47:04.397 INFO:teuthology.orchestra.run.smithi071.stdout: "subtrees": 16, 2024-02-02T15:47:04.397 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse": 243288, 2024-02-02T15:47:04.397 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_hit": 126947, 2024-02-02T15:47:04.397 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_forward": 24967, 2024-02-02T15:47:04.398 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_discover": 298, 2024-02-02T15:47:04.398 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_dir_fetch": 0, 2024-02-02T15:47:04.398 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_remote_ino": 0, 2024-02-02T15:47:04.398 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_lock": 3088, 2024-02-02T15:47:04.398 INFO:teuthology.orchestra.run.smithi071.stdout: "load_cent": 1842, 2024-02-02T15:47:04.398 INFO:teuthology.orchestra.run.smithi071.stdout: "q": 0, 2024-02-02T15:47:04.398 INFO:teuthology.orchestra.run.smithi071.stdout: "exported": 4, 2024-02-02T15:47:04.398 INFO:teuthology.orchestra.run.smithi071.stdout: "exported_inodes": 760, 2024-02-02T15:47:04.398 INFO:teuthology.orchestra.run.smithi071.stdout: "imported": 8, 2024-02-02T15:47:04.399 INFO:teuthology.orchestra.run.smithi071.stdout: "imported_inodes": 882, 2024-02-02T15:47:04.399 INFO:teuthology.orchestra.run.smithi071.stdout: "openino_dir_fetch": 0, 2024-02-02T15:47:04.399 INFO:teuthology.orchestra.run.smithi071.stdout: "openino_backtrace_fetch": 1870, 2024-02-02T15:47:04.399 INFO:teuthology.orchestra.run.smithi071.stdout: "openino_peer_discover": 0, 2024-02-02T15:47:04.399 INFO:teuthology.orchestra.run.smithi071.stdout: "root_rfiles": 670, 2024-02-02T15:47:04.399 INFO:teuthology.orchestra.run.smithi071.stdout: "root_rbytes": 218701295, 2024-02-02T15:47:04.399 INFO:teuthology.orchestra.run.smithi071.stdout: "root_rsnaps": 6, 2024-02-02T15:47:04.399 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_backtrace_fetch": 1870, 2024-02-02T15:47:04.399 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_set_tag": 0, 2024-02-02T15:47:04.400 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_backtrace_repaired": 0, 2024-02-02T15:47:04.400 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_inotable_repaired": 0, 2024-02-02T15:47:04.400 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_dir_inodes": 210, 2024-02-02T15:47:04.400 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_dir_base_inodes": 0, 2024-02-02T15:47:04.400 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_dirfrag_rstats": 210, 2024-02-02T15:47:04.400 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_file_inodes": 1660, 2024-02-02T15:47:04.400 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_inode_file_caps": 19, 2024-02-02T15:47:04.400 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_revoke": 12497, 2024-02-02T15:47:04.400 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_grant": 68811, 2024-02-02T15:47:04.401 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_trunc": 0, 2024-02-02T15:47:04.401 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-02-02T15:47:04.401 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_flush_ack": 0, 2024-02-02T15:47:04.401 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_caps": 43593, 2024-02-02T15:47:04.401 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_caps_dirty": 18836, 2024-02-02T15:47:04.401 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_cap_release": 5642, 2024-02-02T15:47:04.401 INFO:teuthology.orchestra.run.smithi071.stdout: "process_request_cap_release": 95590 2024-02-02T15:47:04.401 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:04.401 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_cache": { 2024-02-02T15:47:04.401 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_update": 817, 2024-02-02T15:47:04.402 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_update_receipt": 2247, 2024-02-02T15:47:04.402 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_try_discover": 1198, 2024-02-02T15:47:04.402 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_send_discover": 2306, 2024-02-02T15:47:04.402 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_handle_discover": 1032, 2024-02-02T15:47:04.402 INFO:teuthology.orchestra.run.smithi071.stdout: "num_strays": 0, 2024-02-02T15:47:04.402 INFO:teuthology.orchestra.run.smithi071.stdout: "num_strays_delayed": 0, 2024-02-02T15:47:04.402 INFO:teuthology.orchestra.run.smithi071.stdout: "num_strays_enqueuing": 0, 2024-02-02T15:47:04.402 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_created": 18049, 2024-02-02T15:47:04.402 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_enqueued": 18049, 2024-02-02T15:47:04.403 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_reintegrated": 0, 2024-02-02T15:47:04.403 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_migrated": 0, 2024-02-02T15:47:04.403 INFO:teuthology.orchestra.run.smithi071.stdout: "num_recovering_processing": 0, 2024-02-02T15:47:04.403 INFO:teuthology.orchestra.run.smithi071.stdout: "num_recovering_enqueued": 0, 2024-02-02T15:47:04.403 INFO:teuthology.orchestra.run.smithi071.stdout: "num_recovering_prioritized": 0, 2024-02-02T15:47:04.403 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_started": 0, 2024-02-02T15:47:04.403 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_completed": 0, 2024-02-02T15:47:04.403 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_enqueue_scrub": 0, 2024-02-02T15:47:04.403 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_exportdir": 6, 2024-02-02T15:47:04.404 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_flush": 0, 2024-02-02T15:47:04.404 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_fragmentdir": 2, 2024-02-02T15:47:04.404 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_fragstats": 0, 2024-02-02T15:47:04.404 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_inodestats": 0 2024-02-02T15:47:04.404 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:04.404 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_log": { 2024-02-02T15:47:04.404 INFO:teuthology.orchestra.run.smithi071.stdout: "evadd": 100745, 2024-02-02T15:47:04.404 INFO:teuthology.orchestra.run.smithi071.stdout: "evex": 7943, 2024-02-02T15:47:04.405 INFO:teuthology.orchestra.run.smithi071.stdout: "evtrm": 7943, 2024-02-02T15:47:04.405 INFO:teuthology.orchestra.run.smithi071.stdout: "ev": 92802, 2024-02-02T15:47:04.405 INFO:teuthology.orchestra.run.smithi071.stdout: "evexg": 0, 2024-02-02T15:47:04.405 INFO:teuthology.orchestra.run.smithi071.stdout: "evexd": 1024, 2024-02-02T15:47:04.405 INFO:teuthology.orchestra.run.smithi071.stdout: "segadd": 136, 2024-02-02T15:47:04.405 INFO:teuthology.orchestra.run.smithi071.stdout: "segex": 8, 2024-02-02T15:47:04.405 INFO:teuthology.orchestra.run.smithi071.stdout: "segtrm": 8, 2024-02-02T15:47:04.405 INFO:teuthology.orchestra.run.smithi071.stdout: "seg": 129, 2024-02-02T15:47:04.405 INFO:teuthology.orchestra.run.smithi071.stdout: "segexg": 0, 2024-02-02T15:47:04.406 INFO:teuthology.orchestra.run.smithi071.stdout: "segexd": 1, 2024-02-02T15:47:04.406 INFO:teuthology.orchestra.run.smithi071.stdout: "expos": 15558850, 2024-02-02T15:47:04.406 INFO:teuthology.orchestra.run.smithi071.stdout: "wrpos": 416600648, 2024-02-02T15:47:04.406 INFO:teuthology.orchestra.run.smithi071.stdout: "rdpos": 0, 2024-02-02T15:47:04.406 INFO:teuthology.orchestra.run.smithi071.stdout: "jlat": { 2024-02-02T15:47:04.406 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 31968, 2024-02-02T15:47:04.406 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 219.668032252, 2024-02-02T15:47:04.406 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.006871497 2024-02-02T15:47:04.406 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:04.407 INFO:teuthology.orchestra.run.smithi071.stdout: "replayed": 0 2024-02-02T15:47:04.407 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:04.407 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_mem": { 2024-02-02T15:47:04.407 INFO:teuthology.orchestra.run.smithi071.stdout: "ino": 803, 2024-02-02T15:47:04.407 INFO:teuthology.orchestra.run.smithi071.stdout: "ino+": 19709, 2024-02-02T15:47:04.407 INFO:teuthology.orchestra.run.smithi071.stdout: "ino-": 18906, 2024-02-02T15:47:04.407 INFO:teuthology.orchestra.run.smithi071.stdout: "dir": 202, 2024-02-02T15:47:04.407 INFO:teuthology.orchestra.run.smithi071.stdout: "dir+": 1290, 2024-02-02T15:47:04.407 INFO:teuthology.orchestra.run.smithi071.stdout: "dir-": 1088, 2024-02-02T15:47:04.408 INFO:teuthology.orchestra.run.smithi071.stdout: "dn": 1024, 2024-02-02T15:47:04.408 INFO:teuthology.orchestra.run.smithi071.stdout: "dn+": 32521, 2024-02-02T15:47:04.408 INFO:teuthology.orchestra.run.smithi071.stdout: "dn-": 31497, 2024-02-02T15:47:04.408 INFO:teuthology.orchestra.run.smithi071.stdout: "cap": 107, 2024-02-02T15:47:04.408 INFO:teuthology.orchestra.run.smithi071.stdout: "cap+": 19835, 2024-02-02T15:47:04.408 INFO:teuthology.orchestra.run.smithi071.stdout: "cap-": 19728, 2024-02-02T15:47:04.408 INFO:teuthology.orchestra.run.smithi071.stdout: "rss": 74028, 2024-02-02T15:47:04.408 INFO:teuthology.orchestra.run.smithi071.stdout: "heap": 198964 2024-02-02T15:47:04.408 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:04.409 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_server": { 2024-02-02T15:47:04.409 INFO:teuthology.orchestra.run.smithi071.stdout: "dispatch_client_request": 213657, 2024-02-02T15:47:04.409 INFO:teuthology.orchestra.run.smithi071.stdout: "dispatch_server_request": 1930, 2024-02-02T15:47:04.409 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_request": 146009, 2024-02-02T15:47:04.409 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_session": 3844, 2024-02-02T15:47:04.409 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_peer_request": 10253, 2024-02-02T15:47:04.409 INFO:teuthology.orchestra.run.smithi071.stdout: "req_create_latency": { 2024-02-02T15:47:04.409 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 18107, 2024-02-02T15:47:04.409 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 81.730370913, 2024-02-02T15:47:04.409 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.004513744 2024-02-02T15:47:04.410 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:04.410 INFO:teuthology.orchestra.run.smithi071.stdout: "req_getattr_latency": { 2024-02-02T15:47:04.410 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 6048, 2024-02-02T15:47:04.410 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 68.319041894, 2024-02-02T15:47:04.410 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.011296137 2024-02-02T15:47:04.410 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:04.410 INFO:teuthology.orchestra.run.smithi071.stdout: "req_getfilelock_latency": { 2024-02-02T15:47:04.410 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:04.410 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:04.411 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:04.411 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:04.411 INFO:teuthology.orchestra.run.smithi071.stdout: "req_link_latency": { 2024-02-02T15:47:04.411 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:04.411 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:04.411 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:04.411 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:04.411 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookup_latency": { 2024-02-02T15:47:04.411 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 33747, 2024-02-02T15:47:04.411 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 58.834518705, 2024-02-02T15:47:04.411 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.001743399 2024-02-02T15:47:04.412 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:04.412 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookuphash_latency": { 2024-02-02T15:47:04.412 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:04.412 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:04.412 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:04.412 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:04.412 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupino_latency": { 2024-02-02T15:47:04.412 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 1, 2024-02-02T15:47:04.412 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000103146, 2024-02-02T15:47:04.413 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000103146 2024-02-02T15:47:04.413 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:04.413 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupname_latency": { 2024-02-02T15:47:04.413 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:04.413 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:04.413 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:04.413 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:04.413 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupparent_latency": { 2024-02-02T15:47:04.413 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:04.413 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:04.414 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:04.414 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:04.414 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupsnap_latency": { 2024-02-02T15:47:04.414 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:04.414 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:04.414 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:04.414 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:04.414 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lssnap_latency": { 2024-02-02T15:47:04.414 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:04.415 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:04.415 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:04.415 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:04.415 INFO:teuthology.orchestra.run.smithi071.stdout: "req_mkdir_latency": { 2024-02-02T15:47:04.415 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:04.415 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:04.415 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:04.415 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:04.415 INFO:teuthology.orchestra.run.smithi071.stdout: "req_mknod_latency": { 2024-02-02T15:47:04.415 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:04.416 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:04.416 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:04.416 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:04.416 INFO:teuthology.orchestra.run.smithi071.stdout: "req_mksnap_latency": { 2024-02-02T15:47:04.416 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:04.416 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:04.416 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:04.416 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:04.416 INFO:teuthology.orchestra.run.smithi071.stdout: "req_open_latency": { 2024-02-02T15:47:04.417 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:04.417 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:04.417 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:04.417 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:04.417 INFO:teuthology.orchestra.run.smithi071.stdout: "req_readdir_latency": { 2024-02-02T15:47:04.417 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 39579, 2024-02-02T15:47:04.417 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 157.831343505, 2024-02-02T15:47:04.417 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.003987754 2024-02-02T15:47:04.417 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:04.418 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rename_latency": { 2024-02-02T15:47:04.418 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 4276, 2024-02-02T15:47:04.418 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 19.989214974, 2024-02-02T15:47:04.418 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.004674746 2024-02-02T15:47:04.418 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:04.418 INFO:teuthology.orchestra.run.smithi071.stdout: "req_renamesnap_latency": { 2024-02-02T15:47:04.418 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:04.418 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:04.419 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:04.419 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:04.419 INFO:teuthology.orchestra.run.smithi071.stdout: "req_snapdiff_latency": { 2024-02-02T15:47:04.419 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:04.419 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:04.419 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:04.419 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:04.419 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rmdir_latency": { 2024-02-02T15:47:04.420 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:04.420 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:04.420 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:04.420 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:04.420 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rmsnap_latency": { 2024-02-02T15:47:04.420 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:04.420 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:04.420 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:04.421 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:04.421 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rmxattr_latency": { 2024-02-02T15:47:04.421 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:04.421 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:04.421 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:04.421 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:04.421 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setattr_latency": { 2024-02-02T15:47:04.421 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 33, 2024-02-02T15:47:04.421 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.720395659, 2024-02-02T15:47:04.422 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.021830171 2024-02-02T15:47:04.422 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:04.422 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setdirlayout_latency": { 2024-02-02T15:47:04.422 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:04.422 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:04.422 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:04.422 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:04.422 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setfilelock_latency": { 2024-02-02T15:47:04.423 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 1180, 2024-02-02T15:47:04.423 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.821047607, 2024-02-02T15:47:04.423 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000695803 2024-02-02T15:47:04.423 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:04.423 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setlayout_latency": { 2024-02-02T15:47:04.423 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:04.423 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:04.423 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:04.424 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:04.424 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setxattr_latency": { 2024-02-02T15:47:04.424 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:04.424 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:04.424 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:04.424 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:04.424 INFO:teuthology.orchestra.run.smithi071.stdout: "req_symlink_latency": { 2024-02-02T15:47:04.424 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:04.425 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:04.425 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:04.425 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:04.425 INFO:teuthology.orchestra.run.smithi071.stdout: "req_unlink_latency": { 2024-02-02T15:47:04.425 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 18049, 2024-02-02T15:47:04.425 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 53.305083548, 2024-02-02T15:47:04.425 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.002953353 2024-02-02T15:47:04.425 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:04.426 INFO:teuthology.orchestra.run.smithi071.stdout: "cap_revoke_eviction": 0, 2024-02-02T15:47:04.426 INFO:teuthology.orchestra.run.smithi071.stdout: "cap_acquisition_throttle": 0, 2024-02-02T15:47:04.426 INFO:teuthology.orchestra.run.smithi071.stdout: "req_getvxattr_latency": { 2024-02-02T15:47:04.426 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:04.426 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:04.426 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:04.426 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-02-02T15:47:04.426 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:04.426 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_sessions": { 2024-02-02T15:47:04.427 INFO:teuthology.orchestra.run.smithi071.stdout: "session_count": 2, 2024-02-02T15:47:04.427 INFO:teuthology.orchestra.run.smithi071.stdout: "session_add": 5, 2024-02-02T15:47:04.427 INFO:teuthology.orchestra.run.smithi071.stdout: "session_remove": 3, 2024-02-02T15:47:04.427 INFO:teuthology.orchestra.run.smithi071.stdout: "sessions_open": 2, 2024-02-02T15:47:04.427 INFO:teuthology.orchestra.run.smithi071.stdout: "sessions_stale": 0, 2024-02-02T15:47:04.427 INFO:teuthology.orchestra.run.smithi071.stdout: "total_load": 21334, 2024-02-02T15:47:04.427 INFO:teuthology.orchestra.run.smithi071.stdout: "average_load": 10667, 2024-02-02T15:47:04.427 INFO:teuthology.orchestra.run.smithi071.stdout: "avg_session_uptime": 990, 2024-02-02T15:47:04.427 INFO:teuthology.orchestra.run.smithi071.stdout: "mdthresh_evicted": 0 2024-02-02T15:47:04.427 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:04.428 INFO:teuthology.orchestra.run.smithi071.stdout: "mempool": { 2024-02-02T15:47:04.428 INFO:teuthology.orchestra.run.smithi071.stdout: "bloom_filter_bytes": 0, 2024-02-02T15:47:04.428 INFO:teuthology.orchestra.run.smithi071.stdout: "bloom_filter_items": 0, 2024-02-02T15:47:04.428 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_alloc_bytes": 0, 2024-02-02T15:47:04.428 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_alloc_items": 0, 2024-02-02T15:47:04.428 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_data_bytes": 0, 2024-02-02T15:47:04.428 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_data_items": 0, 2024-02-02T15:47:04.428 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-02T15:47:04.429 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_onode_items": 0, 2024-02-02T15:47:04.429 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-02T15:47:04.429 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_meta_items": 0, 2024-02-02T15:47:04.429 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_other_bytes": 0, 2024-02-02T15:47:04.429 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_other_items": 0, 2024-02-02T15:47:04.429 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-02T15:47:04.429 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_buffer_items": 0, 2024-02-02T15:47:04.429 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_extent_bytes": 0, 2024-02-02T15:47:04.429 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_extent_items": 0, 2024-02-02T15:47:04.429 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_blob_bytes": 0, 2024-02-02T15:47:04.430 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_blob_items": 0, 2024-02-02T15:47:04.430 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-02T15:47:04.430 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_shared_blob_items": 0, 2024-02-02T15:47:04.430 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-02T15:47:04.430 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_inline_bl_items": 0, 2024-02-02T15:47:04.430 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_bytes": 0, 2024-02-02T15:47:04.430 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_items": 0, 2024-02-02T15:47:04.430 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_txc_bytes": 0, 2024-02-02T15:47:04.430 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_txc_items": 0, 2024-02-02T15:47:04.430 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-02T15:47:04.431 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_deferred_items": 0, 2024-02-02T15:47:04.431 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_bytes": 0, 2024-02-02T15:47:04.431 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_items": 0, 2024-02-02T15:47:04.431 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_bytes": 0, 2024-02-02T15:47:04.431 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_items": 0, 2024-02-02T15:47:04.431 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_reader_bytes": 0, 2024-02-02T15:47:04.431 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_reader_items": 0, 2024-02-02T15:47:04.431 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_writer_bytes": 0, 2024-02-02T15:47:04.431 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_writer_items": 0, 2024-02-02T15:47:04.432 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_anon_bytes": 434772, 2024-02-02T15:47:04.432 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_anon_items": 720, 2024-02-02T15:47:04.432 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_meta_bytes": 88, 2024-02-02T15:47:04.432 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_meta_items": 1, 2024-02-02T15:47:04.432 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_bytes": 0, 2024-02-02T15:47:04.432 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_items": 0, 2024-02-02T15:47:04.432 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mapbl_bytes": 0, 2024-02-02T15:47:04.432 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mapbl_items": 0, 2024-02-02T15:47:04.432 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pglog_bytes": 0, 2024-02-02T15:47:04.433 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pglog_items": 0, 2024-02-02T15:47:04.433 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_bytes": 9608, 2024-02-02T15:47:04.433 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_items": 184, 2024-02-02T15:47:04.433 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_mapping_bytes": 0, 2024-02-02T15:47:04.433 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_mapping_items": 0, 2024-02-02T15:47:04.433 INFO:teuthology.orchestra.run.smithi071.stdout: "pgmap_bytes": 0, 2024-02-02T15:47:04.433 INFO:teuthology.orchestra.run.smithi071.stdout: "pgmap_items": 0, 2024-02-02T15:47:04.433 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_co_bytes": 3444841, 2024-02-02T15:47:04.433 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_co_items": 29905, 2024-02-02T15:47:04.434 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_1_bytes": 0, 2024-02-02T15:47:04.434 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_1_items": 0, 2024-02-02T15:47:04.434 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_2_bytes": 0, 2024-02-02T15:47:04.434 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_2_items": 0 2024-02-02T15:47:04.434 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:04.434 INFO:teuthology.orchestra.run.smithi071.stdout: "objecter": { 2024-02-02T15:47:04.434 INFO:teuthology.orchestra.run.smithi071.stdout: "op_active": 0, 2024-02-02T15:47:04.434 INFO:teuthology.orchestra.run.smithi071.stdout: "op_laggy": 0, 2024-02-02T15:47:04.434 INFO:teuthology.orchestra.run.smithi071.stdout: "op_send": 72612, 2024-02-02T15:47:04.435 INFO:teuthology.orchestra.run.smithi071.stdout: "op_send_bytes": 415921733, 2024-02-02T15:47:04.435 INFO:teuthology.orchestra.run.smithi071.stdout: "op_resend": 33, 2024-02-02T15:47:04.435 INFO:teuthology.orchestra.run.smithi071.stdout: "op_reply": 72612, 2024-02-02T15:47:04.435 INFO:teuthology.orchestra.run.smithi071.stdout: "op_latency": { 2024-02-02T15:47:04.435 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 72612, 2024-02-02T15:47:04.435 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 740.923147397, 2024-02-02T15:47:04.435 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.010203866 2024-02-02T15:47:04.435 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:04.436 INFO:teuthology.orchestra.run.smithi071.stdout: "op_inflight": 0, 2024-02-02T15:47:04.436 INFO:teuthology.orchestra.run.smithi071.stdout: "oplen_avg": { 2024-02-02T15:47:04.436 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 72612, 2024-02-02T15:47:04.436 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 73099 2024-02-02T15:47:04.436 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:04.436 INFO:teuthology.orchestra.run.smithi071.stdout: "op": 72612, 2024-02-02T15:47:04.436 INFO:teuthology.orchestra.run.smithi071.stdout: "op_r": 2765, 2024-02-02T15:47:04.436 INFO:teuthology.orchestra.run.smithi071.stdout: "op_w": 69847, 2024-02-02T15:47:04.437 INFO:teuthology.orchestra.run.smithi071.stdout: "op_rmw": 0, 2024-02-02T15:47:04.437 INFO:teuthology.orchestra.run.smithi071.stdout: "op_pg": 0, 2024-02-02T15:47:04.437 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_stat": 0, 2024-02-02T15:47:04.437 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_create": 19, 2024-02-02T15:47:04.437 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_read": 885, 2024-02-02T15:47:04.437 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_write": 32853, 2024-02-02T15:47:04.437 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_writefull": 513, 2024-02-02T15:47:04.437 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_writesame": 0, 2024-02-02T15:47:04.438 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_append": 0, 2024-02-02T15:47:04.438 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_zero": 0, 2024-02-02T15:47:04.438 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_truncate": 2, 2024-02-02T15:47:04.438 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_delete": 36210, 2024-02-02T15:47:04.438 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_mapext": 0, 2024-02-02T15:47:04.438 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_sparse_read": 0, 2024-02-02T15:47:04.438 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_clonerange": 0, 2024-02-02T15:47:04.438 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_getxattr": 1870, 2024-02-02T15:47:04.439 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_setxattr": 38, 2024-02-02T15:47:04.439 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_cmpxattr": 0, 2024-02-02T15:47:04.439 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_rmxattr": 0, 2024-02-02T15:47:04.439 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_resetxattrs": 0, 2024-02-02T15:47:04.439 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_call": 0, 2024-02-02T15:47:04.439 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_watch": 0, 2024-02-02T15:47:04.439 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_notify": 0, 2024-02-02T15:47:04.439 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_src_cmpxattr": 0, 2024-02-02T15:47:04.440 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_pgls": 0, 2024-02-02T15:47:04.440 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_pgls_filter": 0, 2024-02-02T15:47:04.440 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_other": 0, 2024-02-02T15:47:04.440 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_active": 0, 2024-02-02T15:47:04.440 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_send": 0, 2024-02-02T15:47:04.440 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_resend": 0, 2024-02-02T15:47:04.440 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_ping": 0, 2024-02-02T15:47:04.440 INFO:teuthology.orchestra.run.smithi071.stdout: "poolop_active": 0, 2024-02-02T15:47:04.441 INFO:teuthology.orchestra.run.smithi071.stdout: "poolop_send": 0, 2024-02-02T15:47:04.441 INFO:teuthology.orchestra.run.smithi071.stdout: "poolop_resend": 0, 2024-02-02T15:47:04.441 INFO:teuthology.orchestra.run.smithi071.stdout: "poolstat_active": 0, 2024-02-02T15:47:04.441 INFO:teuthology.orchestra.run.smithi071.stdout: "poolstat_send": 0, 2024-02-02T15:47:04.441 INFO:teuthology.orchestra.run.smithi071.stdout: "poolstat_resend": 0, 2024-02-02T15:47:04.441 INFO:teuthology.orchestra.run.smithi071.stdout: "statfs_active": 0, 2024-02-02T15:47:04.441 INFO:teuthology.orchestra.run.smithi071.stdout: "statfs_send": 0, 2024-02-02T15:47:04.441 INFO:teuthology.orchestra.run.smithi071.stdout: "statfs_resend": 0, 2024-02-02T15:47:04.442 INFO:teuthology.orchestra.run.smithi071.stdout: "command_active": 0, 2024-02-02T15:47:04.442 INFO:teuthology.orchestra.run.smithi071.stdout: "command_send": 0, 2024-02-02T15:47:04.442 INFO:teuthology.orchestra.run.smithi071.stdout: "command_resend": 0, 2024-02-02T15:47:04.442 INFO:teuthology.orchestra.run.smithi071.stdout: "map_epoch": 115, 2024-02-02T15:47:04.442 INFO:teuthology.orchestra.run.smithi071.stdout: "map_full": 0, 2024-02-02T15:47:04.442 INFO:teuthology.orchestra.run.smithi071.stdout: "map_inc": 31, 2024-02-02T15:47:04.442 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_sessions": 12, 2024-02-02T15:47:04.442 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_session_open": 12, 2024-02-02T15:47:04.442 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_session_close": 0, 2024-02-02T15:47:04.443 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_laggy": 0, 2024-02-02T15:47:04.443 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_wr": 477, 2024-02-02T15:47:04.443 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_rd": 16, 2024-02-02T15:47:04.443 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_del": 216 2024-02-02T15:47:04.443 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:04.443 INFO:teuthology.orchestra.run.smithi071.stdout: "oft": { 2024-02-02T15:47:04.443 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_objs": 1, 2024-02-02T15:47:04.443 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_kv_pairs": 0, 2024-02-02T15:47:04.443 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_updates": 0, 2024-02-02T15:47:04.443 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_removes": 0 2024-02-02T15:47:04.444 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:04.444 INFO:teuthology.orchestra.run.smithi071.stdout: "osd-slow-ops": { 2024-02-02T15:47:04.444 INFO:teuthology.orchestra.run.smithi071.stdout: "slow_ops_count": 147096 2024-02-02T15:47:04.444 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:04.444 INFO:teuthology.orchestra.run.smithi071.stdout: "purge_queue": { 2024-02-02T15:47:04.444 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing_ops": 0, 2024-02-02T15:47:04.444 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing_ops_high_water": 10, 2024-02-02T15:47:04.444 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing": 0, 2024-02-02T15:47:04.444 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing_high_water": 5, 2024-02-02T15:47:04.445 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executed": 18049, 2024-02-02T15:47:04.445 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_item_in_journal": 0 2024-02-02T15:47:04.445 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:04.445 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-02T15:47:04.445 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-02-02T15:47:04.445 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 104857600, 2024-02-02T15:47:04.445 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-02-02T15:47:04.445 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 704355, 2024-02-02T15:47:04.445 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 148727627, 2024-02-02T15:47:04.446 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-02-02T15:47:04.446 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 704355, 2024-02-02T15:47:04.446 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 0, 2024-02-02T15:47:04.446 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 0, 2024-02-02T15:47:04.446 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 704355, 2024-02-02T15:47:04.446 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 148727627, 2024-02-02T15:47:04.446 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-02-02T15:47:04.446 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:04.446 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:04.447 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:04.447 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-02-02T15:47:04.447 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:04.447 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-objecter_bytes": { 2024-02-02T15:47:04.447 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-02-02T15:47:04.447 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 104857600, 2024-02-02T15:47:04.447 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-02-02T15:47:04.447 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 0, 2024-02-02T15:47:04.448 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 0, 2024-02-02T15:47:04.448 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-02-02T15:47:04.448 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 0, 2024-02-02T15:47:04.448 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 72612, 2024-02-02T15:47:04.448 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 417891352, 2024-02-02T15:47:04.448 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 72612, 2024-02-02T15:47:04.448 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 417891352, 2024-02-02T15:47:04.448 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-02-02T15:47:04.449 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:04.449 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:04.449 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:04.449 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-02-02T15:47:04.449 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:04.449 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-objecter_ops": { 2024-02-02T15:47:04.449 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-02-02T15:47:04.449 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 1024, 2024-02-02T15:47:04.450 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-02-02T15:47:04.450 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 0, 2024-02-02T15:47:04.450 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 0, 2024-02-02T15:47:04.450 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-02-02T15:47:04.450 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 0, 2024-02-02T15:47:04.450 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 72612, 2024-02-02T15:47:04.450 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 72612, 2024-02-02T15:47:04.450 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 72612, 2024-02-02T15:47:04.451 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 72612, 2024-02-02T15:47:04.451 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-02-02T15:47:04.451 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:04.451 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:04.451 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:04.451 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-02-02T15:47:04.451 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:04.451 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-write_buf_throttle": { 2024-02-02T15:47:04.451 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-02-02T15:47:04.452 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 3758096384, 2024-02-02T15:47:04.452 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-02-02T15:47:04.452 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 18049, 2024-02-02T15:47:04.452 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 1969837, 2024-02-02T15:47:04.452 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-02-02T15:47:04.452 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 18049, 2024-02-02T15:47:04.452 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 0, 2024-02-02T15:47:04.452 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 0, 2024-02-02T15:47:04.453 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 885, 2024-02-02T15:47:04.453 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 1969837, 2024-02-02T15:47:04.453 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-02-02T15:47:04.453 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:04.453 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:04.453 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:04.453 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-02-02T15:47:04.453 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:04.453 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-write_buf_throttle-0x55a0d45ca000": { 2024-02-02T15:47:04.453 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-02-02T15:47:04.454 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 3758096384, 2024-02-02T15:47:04.454 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-02-02T15:47:04.454 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 100745, 2024-02-02T15:47:04.454 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 412406344, 2024-02-02T15:47:04.454 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-02-02T15:47:04.454 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 100745, 2024-02-02T15:47:04.454 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 0, 2024-02-02T15:47:04.454 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 0, 2024-02-02T15:47:04.454 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 31968, 2024-02-02T15:47:04.455 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 412406344, 2024-02-02T15:47:04.455 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-02-02T15:47:04.455 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:04.455 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:04.455 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:04.455 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-02-02T15:47:04.455 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-02-02T15:47:04.455 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-02-02T15:47:04.742 INFO:tasks.check_counter:Daemon mds.h mds.exported=4 2024-02-02T15:47:04.742 INFO:tasks.check_counter:Daemon mds.h mds.imported=8 2024-02-02T15:47:04.742 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=817 2024-02-02T15:47:04.742 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=2247 2024-02-02T15:47:04.742 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-02-02T15:47:04.742 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-02-02T15:47:04.742 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-02-02T15:47:04.742 DEBUG:tasks.check_counter:Getting stats from k 2024-02-02T15:47:04.743 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph daemon mds.k perf dump 2024-02-02T15:47:04.963 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.b/config 2024-02-02T15:47:05.721 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:47:05 smithi071 ceph-mon[44707]: pgmap v631: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 161 KiB/s wr, 32 op/s 2024-02-02T15:47:05.919 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-02-02T15:47:05.919 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-0": { 2024-02-02T15:47:05.919 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 2941, 2024-02-02T15:47:05.919 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 2941, 2024-02-02T15:47:05.919 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 147107953, 2024-02-02T15:47:05.920 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 913862, 2024-02-02T15:47:05.920 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 8, 2024-02-02T15:47:05.920 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 4, 2024-02-02T15:47:05.920 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 0.300559928, 2024-02-02T15:47:05.920 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 0.066106248, 2024-02-02T15:47:05.920 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 0.140308143, 2024-02-02T15:47:05.920 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.073831010, 2024-02-02T15:47:05.920 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-02-02T15:47:05.921 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 2940, 2024-02-02T15:47:05.921 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.150609190, 2024-02-02T15:47:05.921 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000051227 2024-02-02T15:47:05.921 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:05.921 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-02-02T15:47:05.921 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:05.921 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:05.921 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:05.921 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:05.922 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-02-02T15:47:05.922 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 448 2024-02-02T15:47:05.922 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:05.922 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-1": { 2024-02-02T15:47:05.922 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 1477, 2024-02-02T15:47:05.922 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 1408, 2024-02-02T15:47:05.922 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 90582134, 2024-02-02T15:47:05.922 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 528117, 2024-02-02T15:47:05.922 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 6, 2024-02-02T15:47:05.923 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 5, 2024-02-02T15:47:05.923 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 0.175561005, 2024-02-02T15:47:05.923 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 0.038604546, 2024-02-02T15:47:05.923 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 0.111811373, 2024-02-02T15:47:05.923 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.031283301, 2024-02-02T15:47:05.923 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-02-02T15:47:05.923 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 1407, 2024-02-02T15:47:05.923 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.072690873, 2024-02-02T15:47:05.923 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000051663 2024-02-02T15:47:05.924 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:05.924 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-02-02T15:47:05.924 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:05.924 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:05.924 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:05.924 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:05.924 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 2245328, 2024-02-02T15:47:05.924 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 214256 2024-02-02T15:47:05.924 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:05.925 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-2": { 2024-02-02T15:47:05.925 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 900, 2024-02-02T15:47:05.925 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 1175, 2024-02-02T15:47:05.925 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 134453663, 2024-02-02T15:47:05.925 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 905710, 2024-02-02T15:47:05.925 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 9, 2024-02-02T15:47:05.925 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 6, 2024-02-02T15:47:05.925 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 0.150774859, 2024-02-02T15:47:05.925 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 0.025212333, 2024-02-02T15:47:05.926 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 0.095586349, 2024-02-02T15:47:05.926 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.021558026, 2024-02-02T15:47:05.926 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-02-02T15:47:05.926 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 1172, 2024-02-02T15:47:05.926 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.098010895, 2024-02-02T15:47:05.926 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000083627 2024-02-02T15:47:05.926 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:05.926 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-02-02T15:47:05.926 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:05.927 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:05.927 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:05.927 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:05.927 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 320, 2024-02-02T15:47:05.927 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 626432 2024-02-02T15:47:05.927 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:05.927 INFO:teuthology.orchestra.run.smithi071.stdout: "cct": { 2024-02-02T15:47:05.927 INFO:teuthology.orchestra.run.smithi071.stdout: "total_workers": 1, 2024-02-02T15:47:05.927 INFO:teuthology.orchestra.run.smithi071.stdout: "unhealthy_workers": 0 2024-02-02T15:47:05.928 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:05.928 INFO:teuthology.orchestra.run.smithi071.stdout: "finisher-MDSRank": { 2024-02-02T15:47:05.928 INFO:teuthology.orchestra.run.smithi071.stdout: "queue_len": 0, 2024-02-02T15:47:05.928 INFO:teuthology.orchestra.run.smithi071.stdout: "complete_latency": { 2024-02-02T15:47:05.928 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 8123, 2024-02-02T15:47:05.928 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.170273296, 2024-02-02T15:47:05.928 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000020961 2024-02-02T15:47:05.928 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-02-02T15:47:05.929 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:05.929 INFO:teuthology.orchestra.run.smithi071.stdout: "finisher-PurgeQueue": { 2024-02-02T15:47:05.929 INFO:teuthology.orchestra.run.smithi071.stdout: "queue_len": 0, 2024-02-02T15:47:05.929 INFO:teuthology.orchestra.run.smithi071.stdout: "complete_latency": { 2024-02-02T15:47:05.929 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:05.929 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:05.929 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:05.930 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-02-02T15:47:05.930 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:05.930 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-02-02T15:47:05.930 INFO:teuthology.orchestra.run.smithi071.stdout: "request": 0, 2024-02-02T15:47:05.930 INFO:teuthology.orchestra.run.smithi071.stdout: "reply": 0, 2024-02-02T15:47:05.930 INFO:teuthology.orchestra.run.smithi071.stdout: "reply_latency": { 2024-02-02T15:47:05.930 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:05.930 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:05.930 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:05.930 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:05.931 INFO:teuthology.orchestra.run.smithi071.stdout: "slow_reply": 0, 2024-02-02T15:47:05.931 INFO:teuthology.orchestra.run.smithi071.stdout: "forward": 0, 2024-02-02T15:47:05.931 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_fetch_complete": 0, 2024-02-02T15:47:05.931 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_fetch_keys": 0, 2024-02-02T15:47:05.931 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_commit": 0, 2024-02-02T15:47:05.931 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_split": 0, 2024-02-02T15:47:05.931 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_merge": 0, 2024-02-02T15:47:05.932 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes": 21583, 2024-02-02T15:47:05.932 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_top": 23, 2024-02-02T15:47:05.932 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_bottom": 21560, 2024-02-02T15:47:05.932 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_pin_tail": 0, 2024-02-02T15:47:05.932 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_pinned": 21550, 2024-02-02T15:47:05.932 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_expired": 0, 2024-02-02T15:47:05.932 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_with_caps": 0, 2024-02-02T15:47:05.932 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": 0, 2024-02-02T15:47:05.932 INFO:teuthology.orchestra.run.smithi071.stdout: "subtrees": 16, 2024-02-02T15:47:05.933 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse": 0, 2024-02-02T15:47:05.933 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_hit": 0, 2024-02-02T15:47:05.933 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_forward": 0, 2024-02-02T15:47:05.933 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_discover": 0, 2024-02-02T15:47:05.933 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_dir_fetch": 0, 2024-02-02T15:47:05.933 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_remote_ino": 0, 2024-02-02T15:47:05.933 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_lock": 0, 2024-02-02T15:47:05.933 INFO:teuthology.orchestra.run.smithi071.stdout: "load_cent": 0, 2024-02-02T15:47:05.934 INFO:teuthology.orchestra.run.smithi071.stdout: "q": 0, 2024-02-02T15:47:05.934 INFO:teuthology.orchestra.run.smithi071.stdout: "exported": 0, 2024-02-02T15:47:05.934 INFO:teuthology.orchestra.run.smithi071.stdout: "exported_inodes": 0, 2024-02-02T15:47:05.934 INFO:teuthology.orchestra.run.smithi071.stdout: "imported": 0, 2024-02-02T15:47:05.934 INFO:teuthology.orchestra.run.smithi071.stdout: "imported_inodes": 0, 2024-02-02T15:47:05.934 INFO:teuthology.orchestra.run.smithi071.stdout: "openino_dir_fetch": 0, 2024-02-02T15:47:05.934 INFO:teuthology.orchestra.run.smithi071.stdout: "openino_backtrace_fetch": 0, 2024-02-02T15:47:05.934 INFO:teuthology.orchestra.run.smithi071.stdout: "openino_peer_discover": 0, 2024-02-02T15:47:05.934 INFO:teuthology.orchestra.run.smithi071.stdout: "root_rfiles": 0, 2024-02-02T15:47:05.935 INFO:teuthology.orchestra.run.smithi071.stdout: "root_rbytes": 0, 2024-02-02T15:47:05.935 INFO:teuthology.orchestra.run.smithi071.stdout: "root_rsnaps": 0, 2024-02-02T15:47:05.935 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_backtrace_fetch": 0, 2024-02-02T15:47:05.935 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_set_tag": 0, 2024-02-02T15:47:05.935 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_backtrace_repaired": 0, 2024-02-02T15:47:05.935 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_inotable_repaired": 0, 2024-02-02T15:47:05.935 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_dir_inodes": 0, 2024-02-02T15:47:05.935 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_dir_base_inodes": 0, 2024-02-02T15:47:05.935 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_dirfrag_rstats": 0, 2024-02-02T15:47:05.936 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_file_inodes": 0, 2024-02-02T15:47:05.936 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_inode_file_caps": 0, 2024-02-02T15:47:05.936 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_revoke": 0, 2024-02-02T15:47:05.936 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_grant": 0, 2024-02-02T15:47:05.936 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_trunc": 0, 2024-02-02T15:47:05.936 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-02-02T15:47:05.936 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_flush_ack": 0, 2024-02-02T15:47:05.936 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_caps": 0, 2024-02-02T15:47:05.937 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_caps_dirty": 0, 2024-02-02T15:47:05.937 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_cap_release": 0, 2024-02-02T15:47:05.937 INFO:teuthology.orchestra.run.smithi071.stdout: "process_request_cap_release": 0 2024-02-02T15:47:05.937 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:05.937 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_cache": { 2024-02-02T15:47:05.937 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_update": 0, 2024-02-02T15:47:05.937 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_update_receipt": 0, 2024-02-02T15:47:05.937 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_try_discover": 0, 2024-02-02T15:47:05.937 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_send_discover": 0, 2024-02-02T15:47:05.938 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_handle_discover": 0, 2024-02-02T15:47:05.938 INFO:teuthology.orchestra.run.smithi071.stdout: "num_strays": 0, 2024-02-02T15:47:05.938 INFO:teuthology.orchestra.run.smithi071.stdout: "num_strays_delayed": 0, 2024-02-02T15:47:05.938 INFO:teuthology.orchestra.run.smithi071.stdout: "num_strays_enqueuing": 0, 2024-02-02T15:47:05.938 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_created": 0, 2024-02-02T15:47:05.938 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_enqueued": 0, 2024-02-02T15:47:05.938 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_reintegrated": 0, 2024-02-02T15:47:05.938 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_migrated": 0, 2024-02-02T15:47:05.938 INFO:teuthology.orchestra.run.smithi071.stdout: "num_recovering_processing": 0, 2024-02-02T15:47:05.939 INFO:teuthology.orchestra.run.smithi071.stdout: "num_recovering_enqueued": 0, 2024-02-02T15:47:05.939 INFO:teuthology.orchestra.run.smithi071.stdout: "num_recovering_prioritized": 0, 2024-02-02T15:47:05.939 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_started": 0, 2024-02-02T15:47:05.939 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_completed": 0, 2024-02-02T15:47:05.939 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_enqueue_scrub": 0, 2024-02-02T15:47:05.939 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_exportdir": 0, 2024-02-02T15:47:05.939 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_flush": 0, 2024-02-02T15:47:05.939 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_fragmentdir": 0, 2024-02-02T15:47:05.939 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_fragstats": 0, 2024-02-02T15:47:05.940 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_inodestats": 0 2024-02-02T15:47:05.940 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:05.940 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_log": { 2024-02-02T15:47:05.940 INFO:teuthology.orchestra.run.smithi071.stdout: "evadd": 0, 2024-02-02T15:47:05.940 INFO:teuthology.orchestra.run.smithi071.stdout: "evex": 0, 2024-02-02T15:47:05.940 INFO:teuthology.orchestra.run.smithi071.stdout: "evtrm": 0, 2024-02-02T15:47:05.940 INFO:teuthology.orchestra.run.smithi071.stdout: "ev": 98344, 2024-02-02T15:47:05.940 INFO:teuthology.orchestra.run.smithi071.stdout: "evexg": 0, 2024-02-02T15:47:05.941 INFO:teuthology.orchestra.run.smithi071.stdout: "evexd": 0, 2024-02-02T15:47:05.941 INFO:teuthology.orchestra.run.smithi071.stdout: "segadd": 0, 2024-02-02T15:47:05.941 INFO:teuthology.orchestra.run.smithi071.stdout: "segex": 0, 2024-02-02T15:47:05.941 INFO:teuthology.orchestra.run.smithi071.stdout: "segtrm": 0, 2024-02-02T15:47:05.941 INFO:teuthology.orchestra.run.smithi071.stdout: "seg": 127, 2024-02-02T15:47:05.941 INFO:teuthology.orchestra.run.smithi071.stdout: "segexg": 0, 2024-02-02T15:47:05.941 INFO:teuthology.orchestra.run.smithi071.stdout: "segexd": 0, 2024-02-02T15:47:05.941 INFO:teuthology.orchestra.run.smithi071.stdout: "expos": 4194304, 2024-02-02T15:47:05.941 INFO:teuthology.orchestra.run.smithi071.stdout: "wrpos": 372744534, 2024-02-02T15:47:05.942 INFO:teuthology.orchestra.run.smithi071.stdout: "rdpos": 372736167, 2024-02-02T15:47:05.942 INFO:teuthology.orchestra.run.smithi071.stdout: "jlat": { 2024-02-02T15:47:05.942 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:05.942 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:05.942 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:05.942 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:05.942 INFO:teuthology.orchestra.run.smithi071.stdout: "replayed": 98344 2024-02-02T15:47:05.942 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:05.943 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_mem": { 2024-02-02T15:47:05.943 INFO:teuthology.orchestra.run.smithi071.stdout: "ino": 803, 2024-02-02T15:47:05.943 INFO:teuthology.orchestra.run.smithi071.stdout: "ino+": 21601, 2024-02-02T15:47:05.943 INFO:teuthology.orchestra.run.smithi071.stdout: "ino-": 20798, 2024-02-02T15:47:05.943 INFO:teuthology.orchestra.run.smithi071.stdout: "dir": 67, 2024-02-02T15:47:05.943 INFO:teuthology.orchestra.run.smithi071.stdout: "dir+": 88, 2024-02-02T15:47:05.943 INFO:teuthology.orchestra.run.smithi071.stdout: "dir-": 21, 2024-02-02T15:47:05.943 INFO:teuthology.orchestra.run.smithi071.stdout: "dn": 21583, 2024-02-02T15:47:05.943 INFO:teuthology.orchestra.run.smithi071.stdout: "dn+": 21906, 2024-02-02T15:47:05.944 INFO:teuthology.orchestra.run.smithi071.stdout: "dn-": 323, 2024-02-02T15:47:05.944 INFO:teuthology.orchestra.run.smithi071.stdout: "cap": 0, 2024-02-02T15:47:05.944 INFO:teuthology.orchestra.run.smithi071.stdout: "cap+": 0, 2024-02-02T15:47:05.944 INFO:teuthology.orchestra.run.smithi071.stdout: "cap-": 0, 2024-02-02T15:47:05.944 INFO:teuthology.orchestra.run.smithi071.stdout: "rss": 71304, 2024-02-02T15:47:05.944 INFO:teuthology.orchestra.run.smithi071.stdout: "heap": 207156 2024-02-02T15:47:05.944 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:05.945 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_server": { 2024-02-02T15:47:05.945 INFO:teuthology.orchestra.run.smithi071.stdout: "dispatch_client_request": 0, 2024-02-02T15:47:05.945 INFO:teuthology.orchestra.run.smithi071.stdout: "dispatch_server_request": 0, 2024-02-02T15:47:05.945 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_request": 0, 2024-02-02T15:47:05.945 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_session": 0, 2024-02-02T15:47:05.945 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_peer_request": 0, 2024-02-02T15:47:05.945 INFO:teuthology.orchestra.run.smithi071.stdout: "req_create_latency": { 2024-02-02T15:47:05.945 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:05.946 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:05.946 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:05.946 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:05.946 INFO:teuthology.orchestra.run.smithi071.stdout: "req_getattr_latency": { 2024-02-02T15:47:05.946 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:05.946 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:05.946 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:05.946 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:05.947 INFO:teuthology.orchestra.run.smithi071.stdout: "req_getfilelock_latency": { 2024-02-02T15:47:05.947 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:05.947 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:05.947 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:05.947 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:05.947 INFO:teuthology.orchestra.run.smithi071.stdout: "req_link_latency": { 2024-02-02T15:47:05.947 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:05.947 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:05.948 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:05.948 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:05.948 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookup_latency": { 2024-02-02T15:47:05.948 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:05.948 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:05.948 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:05.948 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:05.948 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookuphash_latency": { 2024-02-02T15:47:05.949 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:05.949 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:05.949 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:05.949 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:05.949 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupino_latency": { 2024-02-02T15:47:05.949 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:05.949 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:05.949 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:05.949 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:05.950 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupname_latency": { 2024-02-02T15:47:05.950 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:05.950 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:05.950 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:05.950 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:05.950 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupparent_latency": { 2024-02-02T15:47:05.950 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:05.950 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:05.951 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:05.951 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:05.951 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupsnap_latency": { 2024-02-02T15:47:05.951 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:05.951 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:05.951 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:05.951 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:05.951 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lssnap_latency": { 2024-02-02T15:47:05.951 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:05.952 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:05.952 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:05.952 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:05.952 INFO:teuthology.orchestra.run.smithi071.stdout: "req_mkdir_latency": { 2024-02-02T15:47:05.952 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:05.952 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:05.953 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:05.953 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:05.953 INFO:teuthology.orchestra.run.smithi071.stdout: "req_mknod_latency": { 2024-02-02T15:47:05.953 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:05.953 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:05.953 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:05.953 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:05.954 INFO:teuthology.orchestra.run.smithi071.stdout: "req_mksnap_latency": { 2024-02-02T15:47:05.954 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:05.954 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:05.954 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:05.954 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:05.954 INFO:teuthology.orchestra.run.smithi071.stdout: "req_open_latency": { 2024-02-02T15:47:05.954 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:05.954 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:05.954 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:05.955 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:05.955 INFO:teuthology.orchestra.run.smithi071.stdout: "req_readdir_latency": { 2024-02-02T15:47:05.955 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:05.955 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:05.955 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:05.955 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:05.955 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rename_latency": { 2024-02-02T15:47:05.955 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:05.955 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:05.956 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:05.956 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:05.956 INFO:teuthology.orchestra.run.smithi071.stdout: "req_renamesnap_latency": { 2024-02-02T15:47:05.956 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:05.956 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:05.956 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:05.956 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:05.956 INFO:teuthology.orchestra.run.smithi071.stdout: "req_snapdiff_latency": { 2024-02-02T15:47:05.957 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:05.957 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:05.957 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:05.957 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:05.957 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rmdir_latency": { 2024-02-02T15:47:05.957 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:05.957 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:05.957 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:05.957 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:05.958 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rmsnap_latency": { 2024-02-02T15:47:05.958 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:05.958 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:05.958 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:05.958 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:05.958 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rmxattr_latency": { 2024-02-02T15:47:05.958 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:05.958 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:05.959 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:05.959 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:05.959 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setattr_latency": { 2024-02-02T15:47:05.959 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:05.959 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:05.959 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:05.959 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:05.959 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setdirlayout_latency": { 2024-02-02T15:47:05.959 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:05.960 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:05.960 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:05.960 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:05.960 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setfilelock_latency": { 2024-02-02T15:47:05.960 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:05.960 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:05.960 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:05.960 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:05.961 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setlayout_latency": { 2024-02-02T15:47:05.961 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:05.961 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:05.961 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:05.961 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:05.961 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setxattr_latency": { 2024-02-02T15:47:05.961 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:05.961 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:05.961 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:05.962 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:05.962 INFO:teuthology.orchestra.run.smithi071.stdout: "req_symlink_latency": { 2024-02-02T15:47:05.962 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:05.962 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:05.962 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:05.962 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:05.962 INFO:teuthology.orchestra.run.smithi071.stdout: "req_unlink_latency": { 2024-02-02T15:47:05.963 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:05.963 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:05.963 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:05.963 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:05.963 INFO:teuthology.orchestra.run.smithi071.stdout: "cap_revoke_eviction": 0, 2024-02-02T15:47:05.963 INFO:teuthology.orchestra.run.smithi071.stdout: "cap_acquisition_throttle": 0, 2024-02-02T15:47:05.963 INFO:teuthology.orchestra.run.smithi071.stdout: "req_getvxattr_latency": { 2024-02-02T15:47:05.964 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:05.964 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:05.964 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:05.964 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-02-02T15:47:05.964 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:05.964 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_sessions": { 2024-02-02T15:47:05.965 INFO:teuthology.orchestra.run.smithi071.stdout: "session_count": 2, 2024-02-02T15:47:05.965 INFO:teuthology.orchestra.run.smithi071.stdout: "session_add": 3, 2024-02-02T15:47:05.965 INFO:teuthology.orchestra.run.smithi071.stdout: "session_remove": 1, 2024-02-02T15:47:05.965 INFO:teuthology.orchestra.run.smithi071.stdout: "sessions_open": 2, 2024-02-02T15:47:05.965 INFO:teuthology.orchestra.run.smithi071.stdout: "sessions_stale": 0, 2024-02-02T15:47:05.965 INFO:teuthology.orchestra.run.smithi071.stdout: "total_load": 0, 2024-02-02T15:47:05.966 INFO:teuthology.orchestra.run.smithi071.stdout: "average_load": 0, 2024-02-02T15:47:05.966 INFO:teuthology.orchestra.run.smithi071.stdout: "avg_session_uptime": 4663, 2024-02-02T15:47:05.966 INFO:teuthology.orchestra.run.smithi071.stdout: "mdthresh_evicted": 0 2024-02-02T15:47:05.966 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:05.966 INFO:teuthology.orchestra.run.smithi071.stdout: "mempool": { 2024-02-02T15:47:05.966 INFO:teuthology.orchestra.run.smithi071.stdout: "bloom_filter_bytes": 0, 2024-02-02T15:47:05.966 INFO:teuthology.orchestra.run.smithi071.stdout: "bloom_filter_items": 0, 2024-02-02T15:47:05.966 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_alloc_bytes": 0, 2024-02-02T15:47:05.966 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_alloc_items": 0, 2024-02-02T15:47:05.967 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_data_bytes": 0, 2024-02-02T15:47:05.967 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_data_items": 0, 2024-02-02T15:47:05.967 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-02T15:47:05.967 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_onode_items": 0, 2024-02-02T15:47:05.967 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-02T15:47:05.967 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_meta_items": 0, 2024-02-02T15:47:05.967 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_other_bytes": 0, 2024-02-02T15:47:05.968 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_other_items": 0, 2024-02-02T15:47:05.968 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-02T15:47:05.968 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_buffer_items": 0, 2024-02-02T15:47:05.968 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_extent_bytes": 0, 2024-02-02T15:47:05.968 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_extent_items": 0, 2024-02-02T15:47:05.968 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_blob_bytes": 0, 2024-02-02T15:47:05.968 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_blob_items": 0, 2024-02-02T15:47:05.968 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-02T15:47:05.968 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_shared_blob_items": 0, 2024-02-02T15:47:05.971 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-02T15:47:05.971 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_inline_bl_items": 0, 2024-02-02T15:47:05.971 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_bytes": 0, 2024-02-02T15:47:05.971 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_items": 0, 2024-02-02T15:47:05.971 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_txc_bytes": 0, 2024-02-02T15:47:05.972 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_txc_items": 0, 2024-02-02T15:47:05.972 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-02T15:47:05.972 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_deferred_items": 0, 2024-02-02T15:47:05.972 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_bytes": 0, 2024-02-02T15:47:05.972 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_items": 0, 2024-02-02T15:47:05.972 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_bytes": 0, 2024-02-02T15:47:05.972 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_items": 0, 2024-02-02T15:47:05.973 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_reader_bytes": 0, 2024-02-02T15:47:05.973 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_reader_items": 0, 2024-02-02T15:47:05.973 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_writer_bytes": 0, 2024-02-02T15:47:05.973 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_writer_items": 0, 2024-02-02T15:47:05.973 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_anon_bytes": 274513, 2024-02-02T15:47:05.973 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_anon_items": 1048, 2024-02-02T15:47:05.973 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_meta_bytes": 0, 2024-02-02T15:47:05.973 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_meta_items": 0, 2024-02-02T15:47:05.973 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_bytes": 0, 2024-02-02T15:47:05.974 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_items": 0, 2024-02-02T15:47:05.974 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mapbl_bytes": 0, 2024-02-02T15:47:05.974 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mapbl_items": 0, 2024-02-02T15:47:05.974 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pglog_bytes": 0, 2024-02-02T15:47:05.974 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pglog_items": 0, 2024-02-02T15:47:05.974 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_bytes": 9608, 2024-02-02T15:47:05.974 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_items": 184, 2024-02-02T15:47:05.974 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_mapping_bytes": 0, 2024-02-02T15:47:05.975 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_mapping_items": 0, 2024-02-02T15:47:05.975 INFO:teuthology.orchestra.run.smithi071.stdout: "pgmap_bytes": 0, 2024-02-02T15:47:05.975 INFO:teuthology.orchestra.run.smithi071.stdout: "pgmap_items": 0, 2024-02-02T15:47:05.975 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_co_bytes": 17233440, 2024-02-02T15:47:05.975 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_co_items": 109072, 2024-02-02T15:47:05.975 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_1_bytes": 0, 2024-02-02T15:47:05.975 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_1_items": 0, 2024-02-02T15:47:05.975 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_2_bytes": 0, 2024-02-02T15:47:05.976 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_2_items": 0 2024-02-02T15:47:05.976 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:05.976 INFO:teuthology.orchestra.run.smithi071.stdout: "objecter": { 2024-02-02T15:47:05.976 INFO:teuthology.orchestra.run.smithi071.stdout: "op_active": 0, 2024-02-02T15:47:05.976 INFO:teuthology.orchestra.run.smithi071.stdout: "op_laggy": 0, 2024-02-02T15:47:05.976 INFO:teuthology.orchestra.run.smithi071.stdout: "op_send": 4844, 2024-02-02T15:47:05.976 INFO:teuthology.orchestra.run.smithi071.stdout: "op_send_bytes": 21, 2024-02-02T15:47:05.976 INFO:teuthology.orchestra.run.smithi071.stdout: "op_resend": 3, 2024-02-02T15:47:05.977 INFO:teuthology.orchestra.run.smithi071.stdout: "op_reply": 4844, 2024-02-02T15:47:05.977 INFO:teuthology.orchestra.run.smithi071.stdout: "op_latency": { 2024-02-02T15:47:05.977 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 4844, 2024-02-02T15:47:05.977 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 16.388716004, 2024-02-02T15:47:05.977 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.003383302 2024-02-02T15:47:05.977 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:05.977 INFO:teuthology.orchestra.run.smithi071.stdout: "op_inflight": 0, 2024-02-02T15:47:05.977 INFO:teuthology.orchestra.run.smithi071.stdout: "oplen_avg": { 2024-02-02T15:47:05.978 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 4844, 2024-02-02T15:47:05.978 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 4845 2024-02-02T15:47:05.978 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:05.978 INFO:teuthology.orchestra.run.smithi071.stdout: "op": 4844, 2024-02-02T15:47:05.978 INFO:teuthology.orchestra.run.smithi071.stdout: "op_r": 4844, 2024-02-02T15:47:05.978 INFO:teuthology.orchestra.run.smithi071.stdout: "op_w": 0, 2024-02-02T15:47:05.978 INFO:teuthology.orchestra.run.smithi071.stdout: "op_rmw": 0, 2024-02-02T15:47:05.978 INFO:teuthology.orchestra.run.smithi071.stdout: "op_pg": 0, 2024-02-02T15:47:05.978 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_stat": 2690, 2024-02-02T15:47:05.979 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_create": 0, 2024-02-02T15:47:05.979 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_read": 2152, 2024-02-02T15:47:05.979 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_write": 0, 2024-02-02T15:47:05.979 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_writefull": 0, 2024-02-02T15:47:05.979 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_writesame": 0, 2024-02-02T15:47:05.979 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_append": 0, 2024-02-02T15:47:05.979 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_zero": 0, 2024-02-02T15:47:05.979 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_truncate": 0, 2024-02-02T15:47:05.979 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_delete": 0, 2024-02-02T15:47:05.980 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_mapext": 0, 2024-02-02T15:47:05.980 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_sparse_read": 0, 2024-02-02T15:47:05.980 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_clonerange": 0, 2024-02-02T15:47:05.980 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_getxattr": 1, 2024-02-02T15:47:05.980 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_setxattr": 0, 2024-02-02T15:47:05.980 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_cmpxattr": 0, 2024-02-02T15:47:05.980 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_rmxattr": 0, 2024-02-02T15:47:05.980 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_resetxattrs": 0, 2024-02-02T15:47:05.981 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_call": 0, 2024-02-02T15:47:05.981 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_watch": 0, 2024-02-02T15:47:05.981 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_notify": 0, 2024-02-02T15:47:05.981 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_src_cmpxattr": 0, 2024-02-02T15:47:05.981 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_pgls": 0, 2024-02-02T15:47:05.981 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_pgls_filter": 0, 2024-02-02T15:47:05.981 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_other": 0, 2024-02-02T15:47:05.981 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_active": 0, 2024-02-02T15:47:05.981 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_send": 0, 2024-02-02T15:47:05.982 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_resend": 0, 2024-02-02T15:47:05.982 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_ping": 0, 2024-02-02T15:47:05.982 INFO:teuthology.orchestra.run.smithi071.stdout: "poolop_active": 0, 2024-02-02T15:47:05.982 INFO:teuthology.orchestra.run.smithi071.stdout: "poolop_send": 0, 2024-02-02T15:47:05.982 INFO:teuthology.orchestra.run.smithi071.stdout: "poolop_resend": 0, 2024-02-02T15:47:05.982 INFO:teuthology.orchestra.run.smithi071.stdout: "poolstat_active": 0, 2024-02-02T15:47:05.982 INFO:teuthology.orchestra.run.smithi071.stdout: "poolstat_send": 0, 2024-02-02T15:47:05.982 INFO:teuthology.orchestra.run.smithi071.stdout: "poolstat_resend": 0, 2024-02-02T15:47:05.983 INFO:teuthology.orchestra.run.smithi071.stdout: "statfs_active": 0, 2024-02-02T15:47:05.983 INFO:teuthology.orchestra.run.smithi071.stdout: "statfs_send": 0, 2024-02-02T15:47:05.983 INFO:teuthology.orchestra.run.smithi071.stdout: "statfs_resend": 0, 2024-02-02T15:47:05.983 INFO:teuthology.orchestra.run.smithi071.stdout: "command_active": 0, 2024-02-02T15:47:05.983 INFO:teuthology.orchestra.run.smithi071.stdout: "command_send": 0, 2024-02-02T15:47:05.983 INFO:teuthology.orchestra.run.smithi071.stdout: "command_resend": 0, 2024-02-02T15:47:05.983 INFO:teuthology.orchestra.run.smithi071.stdout: "map_epoch": 115, 2024-02-02T15:47:05.983 INFO:teuthology.orchestra.run.smithi071.stdout: "map_full": 0, 2024-02-02T15:47:05.983 INFO:teuthology.orchestra.run.smithi071.stdout: "map_inc": 31, 2024-02-02T15:47:05.984 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_sessions": 11, 2024-02-02T15:47:05.984 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_session_open": 12, 2024-02-02T15:47:05.984 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_session_close": 1, 2024-02-02T15:47:05.984 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_laggy": 0, 2024-02-02T15:47:05.984 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_wr": 0, 2024-02-02T15:47:05.984 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_rd": 2, 2024-02-02T15:47:05.984 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_del": 0 2024-02-02T15:47:05.984 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:05.984 INFO:teuthology.orchestra.run.smithi071.stdout: "oft": { 2024-02-02T15:47:05.985 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_objs": 0, 2024-02-02T15:47:05.985 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_kv_pairs": 0, 2024-02-02T15:47:05.985 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_updates": 0, 2024-02-02T15:47:05.985 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_removes": 0 2024-02-02T15:47:05.985 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:05.985 INFO:teuthology.orchestra.run.smithi071.stdout: "osd-slow-ops": { 2024-02-02T15:47:05.985 INFO:teuthology.orchestra.run.smithi071.stdout: "slow_ops_count": 0 2024-02-02T15:47:05.986 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:05.986 INFO:teuthology.orchestra.run.smithi071.stdout: "purge_queue": { 2024-02-02T15:47:05.986 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing_ops": 0, 2024-02-02T15:47:05.986 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing_ops_high_water": 0, 2024-02-02T15:47:05.986 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing": 0, 2024-02-02T15:47:05.986 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing_high_water": 0, 2024-02-02T15:47:05.986 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executed": 0, 2024-02-02T15:47:05.987 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_item_in_journal": 0 2024-02-02T15:47:05.987 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:05.987 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-02T15:47:05.987 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-02-02T15:47:05.987 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 104857600, 2024-02-02T15:47:05.987 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-02-02T15:47:05.987 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 5314, 2024-02-02T15:47:05.987 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 371638862, 2024-02-02T15:47:05.988 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-02-02T15:47:05.988 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 5314, 2024-02-02T15:47:05.988 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 0, 2024-02-02T15:47:05.988 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 0, 2024-02-02T15:47:05.988 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 5314, 2024-02-02T15:47:05.988 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 371638862, 2024-02-02T15:47:05.988 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-02-02T15:47:05.988 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:05.988 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:05.989 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:05.989 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-02-02T15:47:05.989 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:05.989 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-objecter_bytes": { 2024-02-02T15:47:05.989 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-02-02T15:47:05.989 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 104857600, 2024-02-02T15:47:05.989 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-02-02T15:47:05.989 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 0, 2024-02-02T15:47:05.990 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 0, 2024-02-02T15:47:05.990 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-02-02T15:47:05.990 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 0, 2024-02-02T15:47:05.990 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 4844, 2024-02-02T15:47:05.990 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 368556552, 2024-02-02T15:47:05.990 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 4844, 2024-02-02T15:47:05.990 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 368556552, 2024-02-02T15:47:05.990 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-02-02T15:47:05.990 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:05.991 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:05.991 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:05.991 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-02-02T15:47:05.991 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:05.991 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-objecter_ops": { 2024-02-02T15:47:05.991 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-02-02T15:47:05.991 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 1024, 2024-02-02T15:47:05.991 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-02-02T15:47:05.991 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 0, 2024-02-02T15:47:05.992 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 0, 2024-02-02T15:47:05.992 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-02-02T15:47:05.992 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 0, 2024-02-02T15:47:05.992 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 4844, 2024-02-02T15:47:05.992 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 4844, 2024-02-02T15:47:05.992 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 4844, 2024-02-02T15:47:05.992 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 4844, 2024-02-02T15:47:05.992 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-02-02T15:47:05.993 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:05.993 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:05.993 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:05.993 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-02-02T15:47:05.993 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:05.993 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-write_buf_throttle": { 2024-02-02T15:47:05.993 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-02-02T15:47:05.993 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 3758096384, 2024-02-02T15:47:05.993 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-02-02T15:47:05.994 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 0, 2024-02-02T15:47:05.994 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 0, 2024-02-02T15:47:05.994 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-02-02T15:47:05.994 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 0, 2024-02-02T15:47:05.994 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 0, 2024-02-02T15:47:05.994 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 0, 2024-02-02T15:47:05.994 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 0, 2024-02-02T15:47:05.994 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 0, 2024-02-02T15:47:05.994 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-02-02T15:47:05.995 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:05.995 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:05.995 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:05.995 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-02-02T15:47:05.995 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-02-02T15:47:05.995 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-write_buf_throttle-0x56115a6ac140": { 2024-02-02T15:47:05.995 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-02-02T15:47:05.995 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 3758096384, 2024-02-02T15:47:05.995 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-02-02T15:47:05.995 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 0, 2024-02-02T15:47:05.996 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 0, 2024-02-02T15:47:05.996 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-02-02T15:47:05.996 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 0, 2024-02-02T15:47:05.996 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 0, 2024-02-02T15:47:05.996 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 0, 2024-02-02T15:47:05.996 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 0, 2024-02-02T15:47:05.996 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 0, 2024-02-02T15:47:05.996 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-02-02T15:47:05.997 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-02-02T15:47:05.997 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-02-02T15:47:05.997 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-02-02T15:47:05.997 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-02-02T15:47:05.997 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-02-02T15:47:05.997 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-02-02T15:47:05.998 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:47:05 smithi001 ceph-mon[35470]: pgmap v631: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 161 KiB/s wr, 32 op/s 2024-02-02T15:47:05.998 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:47:05 smithi142 ceph-mon[45728]: pgmap v631: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 161 KiB/s wr, 32 op/s 2024-02-02T15:47:06.274 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-02-02T15:47:06.274 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-02-02T15:47:06.274 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=0 2024-02-02T15:47:06.274 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=0 2024-02-02T15:47:06.275 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=0 2024-02-02T15:47:06.275 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-02-02T15:47:06.275 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-02-02T15:47:06.275 DEBUG:tasks.check_counter:Getting stats from c 2024-02-02T15:47:06.275 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph daemon mds.c perf dump 2024-02-02T15:47:06.699 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.c/config 2024-02-02T15:47:07.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:47:07 smithi071 ceph-mon[44707]: pgmap v632: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 191 KiB/s wr, 36 op/s 2024-02-02T15:47:07.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:47:07 smithi071 ceph-mon[44707]: osdmap e116: 12 total, 12 up, 12 in 2024-02-02T15:47:07.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:47:07 smithi142 ceph-mon[45728]: pgmap v632: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 191 KiB/s wr, 36 op/s 2024-02-02T15:47:07.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:47:07 smithi142 ceph-mon[45728]: osdmap e116: 12 total, 12 up, 12 in 2024-02-02T15:47:07.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:47:07 smithi001 ceph-mon[35470]: pgmap v632: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 191 KiB/s wr, 36 op/s 2024-02-02T15:47:07.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:47:07 smithi001 ceph-mon[35470]: osdmap e116: 12 total, 12 up, 12 in 2024-02-02T15:47:08.148 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-02-02T15:47:08.148 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-0": { 2024-02-02T15:47:08.148 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 77520, 2024-02-02T15:47:08.149 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 78510, 2024-02-02T15:47:08.149 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 23509109, 2024-02-02T15:47:08.149 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 135397871, 2024-02-02T15:47:08.149 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 59, 2024-02-02T15:47:08.149 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 7, 2024-02-02T15:47:08.149 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 5.409900104, 2024-02-02T15:47:08.149 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 2.252674233, 2024-02-02T15:47:08.149 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 2.695940099, 2024-02-02T15:47:08.150 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.873982097, 2024-02-02T15:47:08.150 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-02-02T15:47:08.150 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 78508, 2024-02-02T15:47:08.150 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 19.501003560, 2024-02-02T15:47:08.150 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000248395 2024-02-02T15:47:08.150 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:08.150 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-02-02T15:47:08.150 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 104659, 2024-02-02T15:47:08.151 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.021642264, 2024-02-02T15:47:08.151 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000206 2024-02-02T15:47:08.151 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:08.151 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 2276576, 2024-02-02T15:47:08.151 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 250320 2024-02-02T15:47:08.151 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:08.151 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-1": { 2024-02-02T15:47:08.151 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 144501, 2024-02-02T15:47:08.151 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 143827, 2024-02-02T15:47:08.152 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 39364676, 2024-02-02T15:47:08.152 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 216395294, 2024-02-02T15:47:08.152 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 10, 2024-02-02T15:47:08.152 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 5, 2024-02-02T15:47:08.152 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 8.246818892, 2024-02-02T15:47:08.152 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 3.321787367, 2024-02-02T15:47:08.152 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 4.955616757, 2024-02-02T15:47:08.152 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 1.010207513, 2024-02-02T15:47:08.153 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-02-02T15:47:08.153 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 143829, 2024-02-02T15:47:08.153 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 37.283907868, 2024-02-02T15:47:08.153 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000259223 2024-02-02T15:47:08.153 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:08.153 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-02-02T15:47:08.154 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 224179, 2024-02-02T15:47:08.154 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.042162082, 2024-02-02T15:47:08.154 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000188 2024-02-02T15:47:08.154 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:08.154 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 320, 2024-02-02T15:47:08.154 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 629152 2024-02-02T15:47:08.154 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:08.154 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-2": { 2024-02-02T15:47:08.155 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 443217, 2024-02-02T15:47:08.155 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 459982, 2024-02-02T15:47:08.155 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 136491420, 2024-02-02T15:47:08.155 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 360051707, 2024-02-02T15:47:08.155 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 39, 2024-02-02T15:47:08.155 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 7, 2024-02-02T15:47:08.155 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 21.260649197, 2024-02-02T15:47:08.155 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 8.936702371, 2024-02-02T15:47:08.156 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 15.570251435, 2024-02-02T15:47:08.156 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.861282187, 2024-02-02T15:47:08.156 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-02-02T15:47:08.156 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 459977, 2024-02-02T15:47:08.156 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 89.261007818, 2024-02-02T15:47:08.156 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000194055 2024-02-02T15:47:08.156 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:08.156 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-02-02T15:47:08.157 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 611015, 2024-02-02T15:47:08.157 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.132406083, 2024-02-02T15:47:08.157 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000216 2024-02-02T15:47:08.157 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:08.157 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 4032, 2024-02-02T15:47:08.157 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 12464 2024-02-02T15:47:08.157 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:08.157 INFO:teuthology.orchestra.run.smithi142.stdout: "cct": { 2024-02-02T15:47:08.158 INFO:teuthology.orchestra.run.smithi142.stdout: "total_workers": 1, 2024-02-02T15:47:08.158 INFO:teuthology.orchestra.run.smithi142.stdout: "unhealthy_workers": 0 2024-02-02T15:47:08.158 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:08.158 INFO:teuthology.orchestra.run.smithi142.stdout: "finisher-MDSRank": { 2024-02-02T15:47:08.158 INFO:teuthology.orchestra.run.smithi142.stdout: "queue_len": 0, 2024-02-02T15:47:08.158 INFO:teuthology.orchestra.run.smithi142.stdout: "complete_latency": { 2024-02-02T15:47:08.158 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 50412, 2024-02-02T15:47:08.158 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 55.263599029, 2024-02-02T15:47:08.159 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.001096238 2024-02-02T15:47:08.159 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-02-02T15:47:08.159 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:08.159 INFO:teuthology.orchestra.run.smithi142.stdout: "finisher-PurgeQueue": { 2024-02-02T15:47:08.159 INFO:teuthology.orchestra.run.smithi142.stdout: "queue_len": 0, 2024-02-02T15:47:08.159 INFO:teuthology.orchestra.run.smithi142.stdout: "complete_latency": { 2024-02-02T15:47:08.159 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 31865, 2024-02-02T15:47:08.159 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 3.853109928, 2024-02-02T15:47:08.159 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000120919 2024-02-02T15:47:08.160 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-02-02T15:47:08.160 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:08.160 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2024-02-02T15:47:08.160 INFO:teuthology.orchestra.run.smithi142.stdout: "request": 142573, 2024-02-02T15:47:08.160 INFO:teuthology.orchestra.run.smithi142.stdout: "reply": 117320, 2024-02-02T15:47:08.160 INFO:teuthology.orchestra.run.smithi142.stdout: "reply_latency": { 2024-02-02T15:47:08.160 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 117320, 2024-02-02T15:47:08.160 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 431.516955585, 2024-02-02T15:47:08.161 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.003678119 2024-02-02T15:47:08.161 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:08.161 INFO:teuthology.orchestra.run.smithi142.stdout: "slow_reply": 0, 2024-02-02T15:47:08.161 INFO:teuthology.orchestra.run.smithi142.stdout: "forward": 25253, 2024-02-02T15:47:08.161 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_fetch_complete": 1, 2024-02-02T15:47:08.161 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_fetch_keys": 0, 2024-02-02T15:47:08.161 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_commit": 29, 2024-02-02T15:47:08.161 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_split": 2, 2024-02-02T15:47:08.161 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_merge": 0, 2024-02-02T15:47:08.162 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes": 1109, 2024-02-02T15:47:08.162 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_top": 239, 2024-02-02T15:47:08.162 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_bottom": 857, 2024-02-02T15:47:08.162 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_pin_tail": 13, 2024-02-02T15:47:08.162 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_pinned": 768, 2024-02-02T15:47:08.162 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_expired": 409, 2024-02-02T15:47:08.162 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_with_caps": 92, 2024-02-02T15:47:08.162 INFO:teuthology.orchestra.run.smithi142.stdout: "caps": 92, 2024-02-02T15:47:08.163 INFO:teuthology.orchestra.run.smithi142.stdout: "subtrees": 13, 2024-02-02T15:47:08.163 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse": 242529, 2024-02-02T15:47:08.163 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_hit": 126137, 2024-02-02T15:47:08.163 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_forward": 25249, 2024-02-02T15:47:08.163 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_discover": 231, 2024-02-02T15:47:08.163 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_dir_fetch": 0, 2024-02-02T15:47:08.163 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_remote_ino": 0, 2024-02-02T15:47:08.163 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_lock": 3704, 2024-02-02T15:47:08.164 INFO:teuthology.orchestra.run.smithi142.stdout: "load_cent": 1893, 2024-02-02T15:47:08.164 INFO:teuthology.orchestra.run.smithi142.stdout: "q": 0, 2024-02-02T15:47:08.164 INFO:teuthology.orchestra.run.smithi142.stdout: "exported": 5, 2024-02-02T15:47:08.164 INFO:teuthology.orchestra.run.smithi142.stdout: "exported_inodes": 287, 2024-02-02T15:47:08.164 INFO:teuthology.orchestra.run.smithi142.stdout: "imported": 6, 2024-02-02T15:47:08.164 INFO:teuthology.orchestra.run.smithi142.stdout: "imported_inodes": 396, 2024-02-02T15:47:08.164 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_dir_fetch": 0, 2024-02-02T15:47:08.164 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_backtrace_fetch": 1788, 2024-02-02T15:47:08.165 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_peer_discover": 0, 2024-02-02T15:47:08.165 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rfiles": 670, 2024-02-02T15:47:08.165 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rbytes": 218701295, 2024-02-02T15:47:08.165 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rsnaps": 6, 2024-02-02T15:47:08.165 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_backtrace_fetch": 1788, 2024-02-02T15:47:08.165 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_set_tag": 0, 2024-02-02T15:47:08.165 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_backtrace_repaired": 0, 2024-02-02T15:47:08.165 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_inotable_repaired": 0, 2024-02-02T15:47:08.166 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dir_inodes": 200, 2024-02-02T15:47:08.166 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dir_base_inodes": 0, 2024-02-02T15:47:08.166 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dirfrag_rstats": 200, 2024-02-02T15:47:08.166 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_file_inodes": 1588, 2024-02-02T15:47:08.166 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_inode_file_caps": 11, 2024-02-02T15:47:08.166 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_revoke": 10594, 2024-02-02T15:47:08.166 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_grant": 67330, 2024-02-02T15:47:08.166 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_trunc": 0, 2024-02-02T15:47:08.166 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-02-02T15:47:08.167 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_flush_ack": 0, 2024-02-02T15:47:08.167 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_caps": 38398, 2024-02-02T15:47:08.167 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_caps_dirty": 14283, 2024-02-02T15:47:08.167 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_cap_release": 6009, 2024-02-02T15:47:08.167 INFO:teuthology.orchestra.run.smithi142.stdout: "process_request_cap_release": 92083 2024-02-02T15:47:08.167 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:08.167 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_cache": { 2024-02-02T15:47:08.167 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_update": 862, 2024-02-02T15:47:08.167 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_update_receipt": 2292, 2024-02-02T15:47:08.168 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_try_discover": 1213, 2024-02-02T15:47:08.168 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_send_discover": 2306, 2024-02-02T15:47:08.168 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_handle_discover": 1000, 2024-02-02T15:47:08.168 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays": 0, 2024-02-02T15:47:08.168 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays_delayed": 0, 2024-02-02T15:47:08.168 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays_enqueuing": 0, 2024-02-02T15:47:08.168 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_created": 17107, 2024-02-02T15:47:08.168 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_enqueued": 17107, 2024-02-02T15:47:08.169 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_reintegrated": 0, 2024-02-02T15:47:08.169 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_migrated": 0, 2024-02-02T15:47:08.169 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_processing": 0, 2024-02-02T15:47:08.169 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_enqueued": 0, 2024-02-02T15:47:08.169 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_prioritized": 0, 2024-02-02T15:47:08.169 INFO:teuthology.orchestra.run.smithi142.stdout: "recovery_started": 0, 2024-02-02T15:47:08.169 INFO:teuthology.orchestra.run.smithi142.stdout: "recovery_completed": 0, 2024-02-02T15:47:08.169 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_enqueue_scrub": 0, 2024-02-02T15:47:08.169 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_exportdir": 5, 2024-02-02T15:47:08.170 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_flush": 0, 2024-02-02T15:47:08.170 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_fragmentdir": 2, 2024-02-02T15:47:08.170 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_fragstats": 0, 2024-02-02T15:47:08.170 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_inodestats": 0 2024-02-02T15:47:08.170 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:08.170 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_log": { 2024-02-02T15:47:08.170 INFO:teuthology.orchestra.run.smithi142.stdout: "evadd": 93971, 2024-02-02T15:47:08.170 INFO:teuthology.orchestra.run.smithi142.stdout: "evex": 2671, 2024-02-02T15:47:08.171 INFO:teuthology.orchestra.run.smithi142.stdout: "evtrm": 2671, 2024-02-02T15:47:08.171 INFO:teuthology.orchestra.run.smithi142.stdout: "ev": 91300, 2024-02-02T15:47:08.171 INFO:teuthology.orchestra.run.smithi142.stdout: "evexg": 0, 2024-02-02T15:47:08.171 INFO:teuthology.orchestra.run.smithi142.stdout: "evexd": 623, 2024-02-02T15:47:08.171 INFO:teuthology.orchestra.run.smithi142.stdout: "segadd": 131, 2024-02-02T15:47:08.171 INFO:teuthology.orchestra.run.smithi142.stdout: "segex": 3, 2024-02-02T15:47:08.171 INFO:teuthology.orchestra.run.smithi142.stdout: "segtrm": 3, 2024-02-02T15:47:08.171 INFO:teuthology.orchestra.run.smithi142.stdout: "seg": 129, 2024-02-02T15:47:08.171 INFO:teuthology.orchestra.run.smithi142.stdout: "segexg": 0, 2024-02-02T15:47:08.172 INFO:teuthology.orchestra.run.smithi142.stdout: "segexd": 1, 2024-02-02T15:47:08.172 INFO:teuthology.orchestra.run.smithi142.stdout: "expos": 8392644, 2024-02-02T15:47:08.172 INFO:teuthology.orchestra.run.smithi142.stdout: "wrpos": 410475771, 2024-02-02T15:47:08.172 INFO:teuthology.orchestra.run.smithi142.stdout: "rdpos": 0, 2024-02-02T15:47:08.172 INFO:teuthology.orchestra.run.smithi142.stdout: "jlat": { 2024-02-02T15:47:08.172 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 26513, 2024-02-02T15:47:08.172 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 197.633904685, 2024-02-02T15:47:08.172 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.007454226 2024-02-02T15:47:08.173 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:08.173 INFO:teuthology.orchestra.run.smithi142.stdout: "replayed": 0 2024-02-02T15:47:08.173 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:08.173 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_mem": { 2024-02-02T15:47:08.173 INFO:teuthology.orchestra.run.smithi142.stdout: "ino": 908, 2024-02-02T15:47:08.173 INFO:teuthology.orchestra.run.smithi142.stdout: "ino+": 18350, 2024-02-02T15:47:08.173 INFO:teuthology.orchestra.run.smithi142.stdout: "ino-": 17442, 2024-02-02T15:47:08.173 INFO:teuthology.orchestra.run.smithi142.stdout: "dir": 204, 2024-02-02T15:47:08.173 INFO:teuthology.orchestra.run.smithi142.stdout: "dir+": 1271, 2024-02-02T15:47:08.174 INFO:teuthology.orchestra.run.smithi142.stdout: "dir-": 1067, 2024-02-02T15:47:08.174 INFO:teuthology.orchestra.run.smithi142.stdout: "dn": 1109, 2024-02-02T15:47:08.174 INFO:teuthology.orchestra.run.smithi142.stdout: "dn+": 32552, 2024-02-02T15:47:08.174 INFO:teuthology.orchestra.run.smithi142.stdout: "dn-": 31443, 2024-02-02T15:47:08.174 INFO:teuthology.orchestra.run.smithi142.stdout: "cap": 92, 2024-02-02T15:47:08.174 INFO:teuthology.orchestra.run.smithi142.stdout: "cap+": 19232, 2024-02-02T15:47:08.174 INFO:teuthology.orchestra.run.smithi142.stdout: "cap-": 19140, 2024-02-02T15:47:08.174 INFO:teuthology.orchestra.run.smithi142.stdout: "rss": 76204, 2024-02-02T15:47:08.174 INFO:teuthology.orchestra.run.smithi142.stdout: "heap": 198964 2024-02-02T15:47:08.175 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:08.175 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_server": { 2024-02-02T15:47:08.175 INFO:teuthology.orchestra.run.smithi142.stdout: "dispatch_client_request": 204057, 2024-02-02T15:47:08.175 INFO:teuthology.orchestra.run.smithi142.stdout: "dispatch_server_request": 244, 2024-02-02T15:47:08.175 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_request": 142573, 2024-02-02T15:47:08.175 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_session": 2382, 2024-02-02T15:47:08.175 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_peer_request": 7803, 2024-02-02T15:47:08.175 INFO:teuthology.orchestra.run.smithi142.stdout: "req_create_latency": { 2024-02-02T15:47:08.176 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 17107, 2024-02-02T15:47:08.176 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 79.077927064, 2024-02-02T15:47:08.176 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.004622547 2024-02-02T15:47:08.176 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:08.176 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getattr_latency": { 2024-02-02T15:47:08.176 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 6058, 2024-02-02T15:47:08.176 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 64.375630349, 2024-02-02T15:47:08.176 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.010626548 2024-02-02T15:47:08.176 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:08.177 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getfilelock_latency": { 2024-02-02T15:47:08.177 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:08.177 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:08.177 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:08.177 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:08.177 INFO:teuthology.orchestra.run.smithi142.stdout: "req_link_latency": { 2024-02-02T15:47:08.177 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:08.177 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:08.177 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:08.178 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:08.178 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookup_latency": { 2024-02-02T15:47:08.178 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 35147, 2024-02-02T15:47:08.178 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 60.519356771, 2024-02-02T15:47:08.178 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.001721892 2024-02-02T15:47:08.178 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:08.178 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookuphash_latency": { 2024-02-02T15:47:08.178 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:08.178 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:08.179 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:08.179 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:08.179 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupino_latency": { 2024-02-02T15:47:08.179 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:08.179 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:08.179 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:08.179 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:08.179 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupname_latency": { 2024-02-02T15:47:08.180 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:08.180 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:08.180 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:08.180 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:08.180 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupparent_latency": { 2024-02-02T15:47:08.180 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:08.180 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:08.180 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:08.180 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:08.181 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupsnap_latency": { 2024-02-02T15:47:08.181 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:08.181 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:08.181 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:08.181 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:08.181 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lssnap_latency": { 2024-02-02T15:47:08.181 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:08.181 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:08.181 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:08.182 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:08.182 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mkdir_latency": { 2024-02-02T15:47:08.182 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:08.182 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:08.182 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:08.182 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:08.182 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mknod_latency": { 2024-02-02T15:47:08.182 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:08.182 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:08.183 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:08.183 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:08.183 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mksnap_latency": { 2024-02-02T15:47:08.183 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:08.183 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:08.183 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:08.183 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:08.183 INFO:teuthology.orchestra.run.smithi142.stdout: "req_open_latency": { 2024-02-02T15:47:08.183 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:08.184 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:08.184 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:08.184 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:08.184 INFO:teuthology.orchestra.run.smithi142.stdout: "req_readdir_latency": { 2024-02-02T15:47:08.184 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 36631, 2024-02-02T15:47:08.184 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 147.827366198, 2024-02-02T15:47:08.184 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.004035580 2024-02-02T15:47:08.184 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:08.185 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rename_latency": { 2024-02-02T15:47:08.185 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 4833, 2024-02-02T15:47:08.185 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 20.477216506, 2024-02-02T15:47:08.185 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.004236957 2024-02-02T15:47:08.185 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:08.185 INFO:teuthology.orchestra.run.smithi142.stdout: "req_renamesnap_latency": { 2024-02-02T15:47:08.185 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:08.185 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:08.185 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:08.186 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:08.186 INFO:teuthology.orchestra.run.smithi142.stdout: "req_snapdiff_latency": { 2024-02-02T15:47:08.186 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:08.186 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:08.186 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:08.186 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:08.186 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmdir_latency": { 2024-02-02T15:47:08.186 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:08.186 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:08.187 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:08.187 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:08.187 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmsnap_latency": { 2024-02-02T15:47:08.187 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:08.187 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:08.187 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:08.187 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:08.187 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmxattr_latency": { 2024-02-02T15:47:08.188 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:08.188 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:08.188 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:08.188 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:08.188 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setattr_latency": { 2024-02-02T15:47:08.188 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 29, 2024-02-02T15:47:08.188 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.660025539, 2024-02-02T15:47:08.188 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.022759501 2024-02-02T15:47:08.189 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:08.189 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setdirlayout_latency": { 2024-02-02T15:47:08.189 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:08.189 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:08.189 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:08.189 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:08.189 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setfilelock_latency": { 2024-02-02T15:47:08.189 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 408, 2024-02-02T15:47:08.190 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.594724079, 2024-02-02T15:47:08.190 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.001457657 2024-02-02T15:47:08.190 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:08.190 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setlayout_latency": { 2024-02-02T15:47:08.190 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:08.190 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:08.190 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:08.190 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:08.190 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setxattr_latency": { 2024-02-02T15:47:08.191 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:08.191 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:08.191 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:08.191 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:08.191 INFO:teuthology.orchestra.run.smithi142.stdout: "req_symlink_latency": { 2024-02-02T15:47:08.191 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:08.191 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:08.191 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:08.192 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:08.192 INFO:teuthology.orchestra.run.smithi142.stdout: "req_unlink_latency": { 2024-02-02T15:47:08.192 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 17107, 2024-02-02T15:47:08.192 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 57.984709079, 2024-02-02T15:47:08.192 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.003389531 2024-02-02T15:47:08.192 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:08.192 INFO:teuthology.orchestra.run.smithi142.stdout: "cap_revoke_eviction": 0, 2024-02-02T15:47:08.192 INFO:teuthology.orchestra.run.smithi142.stdout: "cap_acquisition_throttle": 0, 2024-02-02T15:47:08.193 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getvxattr_latency": { 2024-02-02T15:47:08.193 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:08.193 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:08.193 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:08.193 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-02-02T15:47:08.193 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:08.193 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_sessions": { 2024-02-02T15:47:08.193 INFO:teuthology.orchestra.run.smithi142.stdout: "session_count": 2, 2024-02-02T15:47:08.193 INFO:teuthology.orchestra.run.smithi142.stdout: "session_add": 3, 2024-02-02T15:47:08.194 INFO:teuthology.orchestra.run.smithi142.stdout: "session_remove": 1, 2024-02-02T15:47:08.194 INFO:teuthology.orchestra.run.smithi142.stdout: "sessions_open": 2, 2024-02-02T15:47:08.194 INFO:teuthology.orchestra.run.smithi142.stdout: "sessions_stale": 0, 2024-02-02T15:47:08.194 INFO:teuthology.orchestra.run.smithi142.stdout: "total_load": 21023, 2024-02-02T15:47:08.194 INFO:teuthology.orchestra.run.smithi142.stdout: "average_load": 10511, 2024-02-02T15:47:08.194 INFO:teuthology.orchestra.run.smithi142.stdout: "avg_session_uptime": 952, 2024-02-02T15:47:08.194 INFO:teuthology.orchestra.run.smithi142.stdout: "mdthresh_evicted": 0 2024-02-02T15:47:08.194 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:08.195 INFO:teuthology.orchestra.run.smithi142.stdout: "mempool": { 2024-02-02T15:47:08.195 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_bytes": 0, 2024-02-02T15:47:08.195 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_items": 0, 2024-02-02T15:47:08.195 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_bytes": 0, 2024-02-02T15:47:08.195 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_items": 0, 2024-02-02T15:47:08.195 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_bytes": 0, 2024-02-02T15:47:08.195 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_items": 0, 2024-02-02T15:47:08.195 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-02T15:47:08.195 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_items": 0, 2024-02-02T15:47:08.196 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-02T15:47:08.196 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_items": 0, 2024-02-02T15:47:08.196 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_bytes": 0, 2024-02-02T15:47:08.196 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_items": 0, 2024-02-02T15:47:08.196 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-02T15:47:08.196 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_items": 0, 2024-02-02T15:47:08.196 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_bytes": 0, 2024-02-02T15:47:08.196 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_items": 0, 2024-02-02T15:47:08.197 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_bytes": 0, 2024-02-02T15:47:08.197 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_items": 0, 2024-02-02T15:47:08.197 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-02T15:47:08.197 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_items": 0, 2024-02-02T15:47:08.197 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-02T15:47:08.197 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_items": 0, 2024-02-02T15:47:08.197 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_bytes": 0, 2024-02-02T15:47:08.197 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_items": 0, 2024-02-02T15:47:08.198 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_bytes": 0, 2024-02-02T15:47:08.198 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_items": 0, 2024-02-02T15:47:08.198 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-02T15:47:08.198 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_items": 0, 2024-02-02T15:47:08.198 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_bytes": 0, 2024-02-02T15:47:08.198 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_items": 0, 2024-02-02T15:47:08.198 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_bytes": 0, 2024-02-02T15:47:08.198 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_items": 0, 2024-02-02T15:47:08.198 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_bytes": 0, 2024-02-02T15:47:08.199 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_items": 0, 2024-02-02T15:47:08.199 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_bytes": 0, 2024-02-02T15:47:08.199 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_items": 0, 2024-02-02T15:47:08.199 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_bytes": 415536, 2024-02-02T15:47:08.199 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_items": 693, 2024-02-02T15:47:08.199 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_bytes": 440, 2024-02-02T15:47:08.199 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_items": 5, 2024-02-02T15:47:08.199 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_bytes": 0, 2024-02-02T15:47:08.199 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_items": 0, 2024-02-02T15:47:08.200 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_bytes": 0, 2024-02-02T15:47:08.200 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_items": 0, 2024-02-02T15:47:08.200 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_bytes": 0, 2024-02-02T15:47:08.200 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_items": 0, 2024-02-02T15:47:08.200 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_bytes": 9608, 2024-02-02T15:47:08.200 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_items": 184, 2024-02-02T15:47:08.200 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_bytes": 0, 2024-02-02T15:47:08.200 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_items": 0, 2024-02-02T15:47:08.200 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_bytes": 0, 2024-02-02T15:47:08.201 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_items": 0, 2024-02-02T15:47:08.201 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_bytes": 3609383, 2024-02-02T15:47:08.201 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_items": 30500, 2024-02-02T15:47:08.201 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_bytes": 0, 2024-02-02T15:47:08.201 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_items": 0, 2024-02-02T15:47:08.201 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_bytes": 0, 2024-02-02T15:47:08.201 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_items": 0 2024-02-02T15:47:08.201 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:08.201 INFO:teuthology.orchestra.run.smithi142.stdout: "objecter": { 2024-02-02T15:47:08.201 INFO:teuthology.orchestra.run.smithi142.stdout: "op_active": 0, 2024-02-02T15:47:08.202 INFO:teuthology.orchestra.run.smithi142.stdout: "op_laggy": 0, 2024-02-02T15:47:08.202 INFO:teuthology.orchestra.run.smithi142.stdout: "op_send": 65148, 2024-02-02T15:47:08.202 INFO:teuthology.orchestra.run.smithi142.stdout: "op_send_bytes": 409671224, 2024-02-02T15:47:08.202 INFO:teuthology.orchestra.run.smithi142.stdout: "op_resend": 31, 2024-02-02T15:47:08.202 INFO:teuthology.orchestra.run.smithi142.stdout: "op_reply": 65148, 2024-02-02T15:47:08.202 INFO:teuthology.orchestra.run.smithi142.stdout: "op_latency": { 2024-02-02T15:47:08.202 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 65148, 2024-02-02T15:47:08.202 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 722.728889994, 2024-02-02T15:47:08.203 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.011093646 2024-02-02T15:47:08.203 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:08.203 INFO:teuthology.orchestra.run.smithi142.stdout: "op_inflight": 0, 2024-02-02T15:47:08.203 INFO:teuthology.orchestra.run.smithi142.stdout: "oplen_avg": { 2024-02-02T15:47:08.203 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 65148, 2024-02-02T15:47:08.203 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 65618 2024-02-02T15:47:08.203 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:08.203 INFO:teuthology.orchestra.run.smithi142.stdout: "op": 65148, 2024-02-02T15:47:08.203 INFO:teuthology.orchestra.run.smithi142.stdout: "op_r": 2665, 2024-02-02T15:47:08.204 INFO:teuthology.orchestra.run.smithi142.stdout: "op_w": 62483, 2024-02-02T15:47:08.204 INFO:teuthology.orchestra.run.smithi142.stdout: "op_rmw": 0, 2024-02-02T15:47:08.204 INFO:teuthology.orchestra.run.smithi142.stdout: "op_pg": 0, 2024-02-02T15:47:08.204 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_stat": 0, 2024-02-02T15:47:08.204 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_create": 14, 2024-02-02T15:47:08.204 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_read": 875, 2024-02-02T15:47:08.204 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_write": 27388, 2024-02-02T15:47:08.204 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_writefull": 508, 2024-02-02T15:47:08.205 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_writesame": 0, 2024-02-02T15:47:08.205 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_append": 0, 2024-02-02T15:47:08.205 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_zero": 0, 2024-02-02T15:47:08.205 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_truncate": 2, 2024-02-02T15:47:08.205 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_delete": 34323, 2024-02-02T15:47:08.205 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_mapext": 0, 2024-02-02T15:47:08.205 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_sparse_read": 0, 2024-02-02T15:47:08.205 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_clonerange": 0, 2024-02-02T15:47:08.205 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_getxattr": 1788, 2024-02-02T15:47:08.205 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_setxattr": 28, 2024-02-02T15:47:08.206 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_cmpxattr": 0, 2024-02-02T15:47:08.206 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_rmxattr": 0, 2024-02-02T15:47:08.206 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_resetxattrs": 0, 2024-02-02T15:47:08.206 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_call": 0, 2024-02-02T15:47:08.206 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_watch": 0, 2024-02-02T15:47:08.206 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_notify": 0, 2024-02-02T15:47:08.206 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_src_cmpxattr": 0, 2024-02-02T15:47:08.206 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_pgls": 0, 2024-02-02T15:47:08.207 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_pgls_filter": 0, 2024-02-02T15:47:08.207 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_other": 0, 2024-02-02T15:47:08.207 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_active": 0, 2024-02-02T15:47:08.207 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_send": 0, 2024-02-02T15:47:08.207 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_resend": 0, 2024-02-02T15:47:08.207 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_ping": 0, 2024-02-02T15:47:08.207 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_active": 0, 2024-02-02T15:47:08.207 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_send": 0, 2024-02-02T15:47:08.207 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_resend": 0, 2024-02-02T15:47:08.208 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_active": 0, 2024-02-02T15:47:08.208 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_send": 0, 2024-02-02T15:47:08.208 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_resend": 0, 2024-02-02T15:47:08.208 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_active": 0, 2024-02-02T15:47:08.208 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_send": 0, 2024-02-02T15:47:08.208 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_resend": 0, 2024-02-02T15:47:08.208 INFO:teuthology.orchestra.run.smithi142.stdout: "command_active": 0, 2024-02-02T15:47:08.208 INFO:teuthology.orchestra.run.smithi142.stdout: "command_send": 0, 2024-02-02T15:47:08.209 INFO:teuthology.orchestra.run.smithi142.stdout: "command_resend": 0, 2024-02-02T15:47:08.209 INFO:teuthology.orchestra.run.smithi142.stdout: "map_epoch": 116, 2024-02-02T15:47:08.209 INFO:teuthology.orchestra.run.smithi142.stdout: "map_full": 0, 2024-02-02T15:47:08.209 INFO:teuthology.orchestra.run.smithi142.stdout: "map_inc": 32, 2024-02-02T15:47:08.209 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_sessions": 12, 2024-02-02T15:47:08.209 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_session_open": 12, 2024-02-02T15:47:08.209 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_session_close": 0, 2024-02-02T15:47:08.209 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_laggy": 0, 2024-02-02T15:47:08.209 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_wr": 477, 2024-02-02T15:47:08.209 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_rd": 3, 2024-02-02T15:47:08.210 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_del": 212 2024-02-02T15:47:08.210 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:08.210 INFO:teuthology.orchestra.run.smithi142.stdout: "oft": { 2024-02-02T15:47:08.210 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_objs": 1, 2024-02-02T15:47:08.210 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_kv_pairs": 0, 2024-02-02T15:47:08.210 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_updates": 0, 2024-02-02T15:47:08.210 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_removes": 0 2024-02-02T15:47:08.210 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:08.211 INFO:teuthology.orchestra.run.smithi142.stdout: "osd-slow-ops": { 2024-02-02T15:47:08.211 INFO:teuthology.orchestra.run.smithi142.stdout: "slow_ops_count": 142805 2024-02-02T15:47:08.211 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:08.211 INFO:teuthology.orchestra.run.smithi142.stdout: "purge_queue": { 2024-02-02T15:47:08.211 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_ops": 0, 2024-02-02T15:47:08.211 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_ops_high_water": 10, 2024-02-02T15:47:08.211 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing": 0, 2024-02-02T15:47:08.211 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_high_water": 5, 2024-02-02T15:47:08.211 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executed": 17107, 2024-02-02T15:47:08.212 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_item_in_journal": 0 2024-02-02T15:47:08.212 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:08.212 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-02T15:47:08.212 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-02-02T15:47:08.212 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 104857600, 2024-02-02T15:47:08.212 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-02-02T15:47:08.212 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 665234, 2024-02-02T15:47:08.212 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 142572158, 2024-02-02T15:47:08.213 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-02-02T15:47:08.213 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 665234, 2024-02-02T15:47:08.213 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-02-02T15:47:08.213 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-02-02T15:47:08.213 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 665234, 2024-02-02T15:47:08.213 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 142572158, 2024-02-02T15:47:08.213 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-02-02T15:47:08.213 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:08.213 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:08.214 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:08.214 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-02-02T15:47:08.214 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:08.214 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-objecter_bytes": { 2024-02-02T15:47:08.214 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-02-02T15:47:08.214 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 104857600, 2024-02-02T15:47:08.214 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-02-02T15:47:08.214 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-02-02T15:47:08.215 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-02-02T15:47:08.215 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-02-02T15:47:08.215 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-02-02T15:47:08.215 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 65148, 2024-02-02T15:47:08.215 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 411538913, 2024-02-02T15:47:08.215 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 65148, 2024-02-02T15:47:08.215 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 411538913, 2024-02-02T15:47:08.215 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-02-02T15:47:08.216 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:08.216 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:08.216 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:08.216 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-02-02T15:47:08.216 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:08.219 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-objecter_ops": { 2024-02-02T15:47:08.219 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-02-02T15:47:08.219 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 1024, 2024-02-02T15:47:08.219 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-02-02T15:47:08.219 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-02-02T15:47:08.219 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-02-02T15:47:08.219 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-02-02T15:47:08.220 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-02-02T15:47:08.220 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 65148, 2024-02-02T15:47:08.220 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 65148, 2024-02-02T15:47:08.220 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 65148, 2024-02-02T15:47:08.220 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 65148, 2024-02-02T15:47:08.220 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-02-02T15:47:08.220 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:08.220 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:08.220 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:08.221 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-02-02T15:47:08.221 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:08.221 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-write_buf_throttle": { 2024-02-02T15:47:08.221 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-02-02T15:47:08.221 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 3758096384, 2024-02-02T15:47:08.221 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-02-02T15:47:08.221 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 17107, 2024-02-02T15:47:08.221 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 1867735, 2024-02-02T15:47:08.222 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-02-02T15:47:08.222 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 17107, 2024-02-02T15:47:08.222 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-02-02T15:47:08.222 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-02-02T15:47:08.222 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 875, 2024-02-02T15:47:08.222 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 1867735, 2024-02-02T15:47:08.222 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-02-02T15:47:08.222 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:08.222 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:08.223 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:08.223 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-02-02T15:47:08.223 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:08.223 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-write_buf_throttle-0x5649a4d1c000": { 2024-02-02T15:47:08.223 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 15573, 2024-02-02T15:47:08.223 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 3758096384, 2024-02-02T15:47:08.223 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-02-02T15:47:08.223 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 93971, 2024-02-02T15:47:08.224 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 406281467, 2024-02-02T15:47:08.224 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-02-02T15:47:08.224 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 93971, 2024-02-02T15:47:08.224 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-02-02T15:47:08.224 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-02-02T15:47:08.224 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 26513, 2024-02-02T15:47:08.224 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 406265894, 2024-02-02T15:47:08.224 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-02-02T15:47:08.225 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:08.225 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:08.225 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:08.225 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-02-02T15:47:08.225 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-02-02T15:47:08.225 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-02-02T15:47:08.508 INFO:tasks.check_counter:Daemon mds.c mds.exported=5 2024-02-02T15:47:08.509 INFO:tasks.check_counter:Daemon mds.c mds.imported=6 2024-02-02T15:47:08.509 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=862 2024-02-02T15:47:08.509 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=2292 2024-02-02T15:47:08.509 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-02-02T15:47:08.509 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-02-02T15:47:08.509 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-02-02T15:47:08.509 DEBUG:tasks.check_counter:Getting stats from f 2024-02-02T15:47:08.510 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph daemon mds.f perf dump 2024-02-02T15:47:08.757 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.c/config 2024-02-02T15:47:09.717 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-02-02T15:47:09.717 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-0": { 2024-02-02T15:47:09.717 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 295112, 2024-02-02T15:47:09.717 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 224928, 2024-02-02T15:47:09.718 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 87686544, 2024-02-02T15:47:09.718 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 187864320, 2024-02-02T15:47:09.718 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 57, 2024-02-02T15:47:09.718 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 6, 2024-02-02T15:47:09.718 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 14.706435319, 2024-02-02T15:47:09.718 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 5.834088504, 2024-02-02T15:47:09.718 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 10.696289245, 2024-02-02T15:47:09.718 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.719855326, 2024-02-02T15:47:09.718 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-02-02T15:47:09.719 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 224927, 2024-02-02T15:47:09.719 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 37.111817940, 2024-02-02T15:47:09.719 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000164994 2024-02-02T15:47:09.719 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:09.719 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-02-02T15:47:09.719 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 470110, 2024-02-02T15:47:09.719 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.079120202, 2024-02-02T15:47:09.719 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000168 2024-02-02T15:47:09.720 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:09.720 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-02-02T15:47:09.720 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 448 2024-02-02T15:47:09.720 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:09.720 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-1": { 2024-02-02T15:47:09.720 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 145473, 2024-02-02T15:47:09.720 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 148758, 2024-02-02T15:47:09.720 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 41333409, 2024-02-02T15:47:09.721 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 180565269, 2024-02-02T15:47:09.721 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 8, 2024-02-02T15:47:09.721 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 6, 2024-02-02T15:47:09.721 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 8.190316761, 2024-02-02T15:47:09.721 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 3.422216148, 2024-02-02T15:47:09.721 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 5.255834507, 2024-02-02T15:47:09.721 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.657848081, 2024-02-02T15:47:09.721 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-02-02T15:47:09.722 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 148759, 2024-02-02T15:47:09.722 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 39.230800121, 2024-02-02T15:47:09.722 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000263720 2024-02-02T15:47:09.722 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:09.722 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-02-02T15:47:09.722 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 240159, 2024-02-02T15:47:09.722 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.044214955, 2024-02-02T15:47:09.722 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000184 2024-02-02T15:47:09.722 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:09.723 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 2268416, 2024-02-02T15:47:09.723 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 254208 2024-02-02T15:47:09.723 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:09.723 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-2": { 2024-02-02T15:47:09.723 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 165351, 2024-02-02T15:47:09.723 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 233793, 2024-02-02T15:47:09.723 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 51369362, 2024-02-02T15:47:09.723 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 354490547, 2024-02-02T15:47:09.723 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 42, 2024-02-02T15:47:09.723 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 7, 2024-02-02T15:47:09.724 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 9.322708988, 2024-02-02T15:47:09.724 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 4.359022899, 2024-02-02T15:47:09.724 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 5.646834553, 2024-02-02T15:47:09.724 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.727741553, 2024-02-02T15:47:09.724 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-02-02T15:47:09.724 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 233788, 2024-02-02T15:47:09.724 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 58.427935661, 2024-02-02T15:47:09.724 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000249918 2024-02-02T15:47:09.724 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:09.725 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-02-02T15:47:09.725 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 174490, 2024-02-02T15:47:09.725 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.054585123, 2024-02-02T15:47:09.725 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000312 2024-02-02T15:47:09.725 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:09.725 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 3104, 2024-02-02T15:47:09.725 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 641312 2024-02-02T15:47:09.725 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:09.725 INFO:teuthology.orchestra.run.smithi142.stdout: "cct": { 2024-02-02T15:47:09.725 INFO:teuthology.orchestra.run.smithi142.stdout: "total_workers": 1, 2024-02-02T15:47:09.726 INFO:teuthology.orchestra.run.smithi142.stdout: "unhealthy_workers": 0 2024-02-02T15:47:09.726 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:09.726 INFO:teuthology.orchestra.run.smithi142.stdout: "finisher-MDSRank": { 2024-02-02T15:47:09.726 INFO:teuthology.orchestra.run.smithi142.stdout: "queue_len": 0, 2024-02-02T15:47:09.726 INFO:teuthology.orchestra.run.smithi142.stdout: "complete_latency": { 2024-02-02T15:47:09.726 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 42950, 2024-02-02T15:47:09.726 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 49.568445391, 2024-02-02T15:47:09.726 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.001154096 2024-02-02T15:47:09.726 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-02-02T15:47:09.727 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:09.727 INFO:teuthology.orchestra.run.smithi142.stdout: "finisher-PurgeQueue": { 2024-02-02T15:47:09.727 INFO:teuthology.orchestra.run.smithi142.stdout: "queue_len": 0, 2024-02-02T15:47:09.727 INFO:teuthology.orchestra.run.smithi142.stdout: "complete_latency": { 2024-02-02T15:47:09.727 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 27405, 2024-02-02T15:47:09.727 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 3.629801933, 2024-02-02T15:47:09.727 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000132450 2024-02-02T15:47:09.727 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-02-02T15:47:09.727 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:09.727 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2024-02-02T15:47:09.728 INFO:teuthology.orchestra.run.smithi142.stdout: "request": 105670, 2024-02-02T15:47:09.728 INFO:teuthology.orchestra.run.smithi142.stdout: "reply": 104758, 2024-02-02T15:47:09.728 INFO:teuthology.orchestra.run.smithi142.stdout: "reply_latency": { 2024-02-02T15:47:09.728 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 104758, 2024-02-02T15:47:09.728 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 461.879049339, 2024-02-02T15:47:09.728 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.004409009 2024-02-02T15:47:09.728 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:09.728 INFO:teuthology.orchestra.run.smithi142.stdout: "slow_reply": 0, 2024-02-02T15:47:09.728 INFO:teuthology.orchestra.run.smithi142.stdout: "forward": 912, 2024-02-02T15:47:09.729 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_fetch_complete": 1, 2024-02-02T15:47:09.729 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_fetch_keys": 0, 2024-02-02T15:47:09.729 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_commit": 38, 2024-02-02T15:47:09.729 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_split": 3, 2024-02-02T15:47:09.729 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_merge": 0, 2024-02-02T15:47:09.729 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes": 1674, 2024-02-02T15:47:09.729 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_top": 306, 2024-02-02T15:47:09.729 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_bottom": 1337, 2024-02-02T15:47:09.729 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_pin_tail": 31, 2024-02-02T15:47:09.730 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_pinned": 1237, 2024-02-02T15:47:09.730 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_expired": 381, 2024-02-02T15:47:09.730 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_with_caps": 59, 2024-02-02T15:47:09.730 INFO:teuthology.orchestra.run.smithi142.stdout: "caps": 59, 2024-02-02T15:47:09.730 INFO:teuthology.orchestra.run.smithi142.stdout: "subtrees": 22, 2024-02-02T15:47:09.730 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse": 185437, 2024-02-02T15:47:09.730 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_hit": 111220, 2024-02-02T15:47:09.730 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_forward": 908, 2024-02-02T15:47:09.730 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_discover": 192, 2024-02-02T15:47:09.731 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_dir_fetch": 0, 2024-02-02T15:47:09.731 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_remote_ino": 0, 2024-02-02T15:47:09.731 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_lock": 2672, 2024-02-02T15:47:09.731 INFO:teuthology.orchestra.run.smithi142.stdout: "load_cent": 2633, 2024-02-02T15:47:09.731 INFO:teuthology.orchestra.run.smithi142.stdout: "q": 0, 2024-02-02T15:47:09.731 INFO:teuthology.orchestra.run.smithi142.stdout: "exported": 6, 2024-02-02T15:47:09.731 INFO:teuthology.orchestra.run.smithi142.stdout: "exported_inodes": 270, 2024-02-02T15:47:09.731 INFO:teuthology.orchestra.run.smithi142.stdout: "imported": 9, 2024-02-02T15:47:09.731 INFO:teuthology.orchestra.run.smithi142.stdout: "imported_inodes": 1021, 2024-02-02T15:47:09.732 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_dir_fetch": 0, 2024-02-02T15:47:09.732 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_backtrace_fetch": 2140, 2024-02-02T15:47:09.732 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_peer_discover": 0, 2024-02-02T15:47:09.732 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rfiles": 670, 2024-02-02T15:47:09.732 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rbytes": 218701295, 2024-02-02T15:47:09.732 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rsnaps": 6, 2024-02-02T15:47:09.732 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_backtrace_fetch": 2140, 2024-02-02T15:47:09.732 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_set_tag": 0, 2024-02-02T15:47:09.733 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_backtrace_repaired": 0, 2024-02-02T15:47:09.733 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_inotable_repaired": 0, 2024-02-02T15:47:09.733 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dir_inodes": 190, 2024-02-02T15:47:09.733 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dir_base_inodes": 0, 2024-02-02T15:47:09.733 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dirfrag_rstats": 190, 2024-02-02T15:47:09.733 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_file_inodes": 1950, 2024-02-02T15:47:09.733 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_inode_file_caps": 20, 2024-02-02T15:47:09.734 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_revoke": 5169, 2024-02-02T15:47:09.734 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_grant": 51658, 2024-02-02T15:47:09.734 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_trunc": 0, 2024-02-02T15:47:09.734 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-02-02T15:47:09.734 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_flush_ack": 0, 2024-02-02T15:47:09.734 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_caps": 27413, 2024-02-02T15:47:09.734 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_caps_dirty": 12681, 2024-02-02T15:47:09.734 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_cap_release": 4872, 2024-02-02T15:47:09.735 INFO:teuthology.orchestra.run.smithi142.stdout: "process_request_cap_release": 87550 2024-02-02T15:47:09.735 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:09.735 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_cache": { 2024-02-02T15:47:09.735 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_update": 1143, 2024-02-02T15:47:09.735 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_update_receipt": 2102, 2024-02-02T15:47:09.735 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_try_discover": 1140, 2024-02-02T15:47:09.735 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_send_discover": 1900, 2024-02-02T15:47:09.736 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_handle_discover": 1330, 2024-02-02T15:47:09.736 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays": 0, 2024-02-02T15:47:09.736 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays_delayed": 0, 2024-02-02T15:47:09.736 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays_enqueuing": 0, 2024-02-02T15:47:09.736 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_created": 14617, 2024-02-02T15:47:09.736 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_enqueued": 14617, 2024-02-02T15:47:09.736 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_reintegrated": 0, 2024-02-02T15:47:09.736 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_migrated": 0, 2024-02-02T15:47:09.736 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_processing": 0, 2024-02-02T15:47:09.737 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_enqueued": 0, 2024-02-02T15:47:09.737 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_prioritized": 0, 2024-02-02T15:47:09.737 INFO:teuthology.orchestra.run.smithi142.stdout: "recovery_started": 0, 2024-02-02T15:47:09.737 INFO:teuthology.orchestra.run.smithi142.stdout: "recovery_completed": 0, 2024-02-02T15:47:09.737 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_enqueue_scrub": 0, 2024-02-02T15:47:09.737 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_exportdir": 6, 2024-02-02T15:47:09.737 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_flush": 0, 2024-02-02T15:47:09.737 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_fragmentdir": 3, 2024-02-02T15:47:09.737 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_fragstats": 0, 2024-02-02T15:47:09.738 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_inodestats": 0 2024-02-02T15:47:09.738 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:09.738 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_log": { 2024-02-02T15:47:09.738 INFO:teuthology.orchestra.run.smithi142.stdout: "evadd": 83277, 2024-02-02T15:47:09.738 INFO:teuthology.orchestra.run.smithi142.stdout: "evex": 0, 2024-02-02T15:47:09.738 INFO:teuthology.orchestra.run.smithi142.stdout: "evtrm": 0, 2024-02-02T15:47:09.738 INFO:teuthology.orchestra.run.smithi142.stdout: "ev": 83277, 2024-02-02T15:47:09.738 INFO:teuthology.orchestra.run.smithi142.stdout: "evexg": 0, 2024-02-02T15:47:09.738 INFO:teuthology.orchestra.run.smithi142.stdout: "evexd": 0, 2024-02-02T15:47:09.738 INFO:teuthology.orchestra.run.smithi142.stdout: "segadd": 125, 2024-02-02T15:47:09.739 INFO:teuthology.orchestra.run.smithi142.stdout: "segex": 0, 2024-02-02T15:47:09.739 INFO:teuthology.orchestra.run.smithi142.stdout: "segtrm": 0, 2024-02-02T15:47:09.739 INFO:teuthology.orchestra.run.smithi142.stdout: "seg": 125, 2024-02-02T15:47:09.739 INFO:teuthology.orchestra.run.smithi142.stdout: "segexg": 0, 2024-02-02T15:47:09.739 INFO:teuthology.orchestra.run.smithi142.stdout: "segexd": 0, 2024-02-02T15:47:09.739 INFO:teuthology.orchestra.run.smithi142.stdout: "expos": 4194304, 2024-02-02T15:47:09.739 INFO:teuthology.orchestra.run.smithi142.stdout: "wrpos": 431500639, 2024-02-02T15:47:09.739 INFO:teuthology.orchestra.run.smithi142.stdout: "rdpos": 0, 2024-02-02T15:47:09.739 INFO:teuthology.orchestra.run.smithi142.stdout: "jlat": { 2024-02-02T15:47:09.740 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 22581, 2024-02-02T15:47:09.740 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 194.449346786, 2024-02-02T15:47:09.740 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.008611192 2024-02-02T15:47:09.740 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:09.740 INFO:teuthology.orchestra.run.smithi142.stdout: "replayed": 0 2024-02-02T15:47:09.740 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:09.740 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_mem": { 2024-02-02T15:47:09.740 INFO:teuthology.orchestra.run.smithi142.stdout: "ino": 1450, 2024-02-02T15:47:09.740 INFO:teuthology.orchestra.run.smithi142.stdout: "ino+": 16372, 2024-02-02T15:47:09.741 INFO:teuthology.orchestra.run.smithi142.stdout: "ino-": 14922, 2024-02-02T15:47:09.741 INFO:teuthology.orchestra.run.smithi142.stdout: "dir": 200, 2024-02-02T15:47:09.741 INFO:teuthology.orchestra.run.smithi142.stdout: "dir+": 1111, 2024-02-02T15:47:09.741 INFO:teuthology.orchestra.run.smithi142.stdout: "dir-": 911, 2024-02-02T15:47:09.741 INFO:teuthology.orchestra.run.smithi142.stdout: "dn": 1674, 2024-02-02T15:47:09.741 INFO:teuthology.orchestra.run.smithi142.stdout: "dn+": 25069, 2024-02-02T15:47:09.741 INFO:teuthology.orchestra.run.smithi142.stdout: "dn-": 23395, 2024-02-02T15:47:09.741 INFO:teuthology.orchestra.run.smithi142.stdout: "cap": 59, 2024-02-02T15:47:09.741 INFO:teuthology.orchestra.run.smithi142.stdout: "cap+": 15880, 2024-02-02T15:47:09.742 INFO:teuthology.orchestra.run.smithi142.stdout: "cap-": 15821, 2024-02-02T15:47:09.742 INFO:teuthology.orchestra.run.smithi142.stdout: "rss": 76916, 2024-02-02T15:47:09.742 INFO:teuthology.orchestra.run.smithi142.stdout: "heap": 198964 2024-02-02T15:47:09.742 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:09.742 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_server": { 2024-02-02T15:47:09.742 INFO:teuthology.orchestra.run.smithi142.stdout: "dispatch_client_request": 163527, 2024-02-02T15:47:09.742 INFO:teuthology.orchestra.run.smithi142.stdout: "dispatch_server_request": 768, 2024-02-02T15:47:09.742 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_request": 105670, 2024-02-02T15:47:09.742 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_session": 1697, 2024-02-02T15:47:09.743 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_peer_request": 8124, 2024-02-02T15:47:09.743 INFO:teuthology.orchestra.run.smithi142.stdout: "req_create_latency": { 2024-02-02T15:47:09.743 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 14592, 2024-02-02T15:47:09.743 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 98.178020379, 2024-02-02T15:47:09.743 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.006728208 2024-02-02T15:47:09.743 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:09.743 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getattr_latency": { 2024-02-02T15:47:09.743 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 8184, 2024-02-02T15:47:09.744 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 110.779760158, 2024-02-02T15:47:09.744 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.013536138 2024-02-02T15:47:09.744 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:09.744 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getfilelock_latency": { 2024-02-02T15:47:09.744 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:09.744 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:09.744 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:09.744 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:09.745 INFO:teuthology.orchestra.run.smithi142.stdout: "req_link_latency": { 2024-02-02T15:47:09.745 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:09.745 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:09.745 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:09.745 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:09.745 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookup_latency": { 2024-02-02T15:47:09.745 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 23928, 2024-02-02T15:47:09.745 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 59.565747858, 2024-02-02T15:47:09.746 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.002489374 2024-02-02T15:47:09.746 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:09.746 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookuphash_latency": { 2024-02-02T15:47:09.746 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:09.746 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:09.746 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:09.746 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:09.746 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupino_latency": { 2024-02-02T15:47:09.746 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:09.747 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:09.747 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:09.747 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:09.747 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupname_latency": { 2024-02-02T15:47:09.747 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:09.747 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:09.747 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:09.747 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:09.747 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupparent_latency": { 2024-02-02T15:47:09.747 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:09.748 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:09.748 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:09.748 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:09.748 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupsnap_latency": { 2024-02-02T15:47:09.748 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:09.748 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:09.748 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:09.748 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:09.749 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lssnap_latency": { 2024-02-02T15:47:09.749 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:09.749 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:09.749 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:09.749 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:09.749 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mkdir_latency": { 2024-02-02T15:47:09.749 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:09.749 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:09.750 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:09.750 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:09.750 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mknod_latency": { 2024-02-02T15:47:09.750 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:09.750 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:09.750 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:09.750 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:09.750 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mksnap_latency": { 2024-02-02T15:47:09.751 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:09.751 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:09.751 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:09.751 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:09.751 INFO:teuthology.orchestra.run.smithi142.stdout: "req_open_latency": { 2024-02-02T15:47:09.751 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 4, 2024-02-02T15:47:09.751 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.005330395, 2024-02-02T15:47:09.751 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.001332598 2024-02-02T15:47:09.752 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:09.752 INFO:teuthology.orchestra.run.smithi142.stdout: "req_readdir_latency": { 2024-02-02T15:47:09.752 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 39997, 2024-02-02T15:47:09.752 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 111.898794145, 2024-02-02T15:47:09.752 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.002797679 2024-02-02T15:47:09.752 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:09.752 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rename_latency": { 2024-02-02T15:47:09.753 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 3057, 2024-02-02T15:47:09.753 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 18.950323462, 2024-02-02T15:47:09.753 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.006198993 2024-02-02T15:47:09.753 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:09.753 INFO:teuthology.orchestra.run.smithi142.stdout: "req_renamesnap_latency": { 2024-02-02T15:47:09.753 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:09.753 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:09.753 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:09.754 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:09.754 INFO:teuthology.orchestra.run.smithi142.stdout: "req_snapdiff_latency": { 2024-02-02T15:47:09.754 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:09.754 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:09.754 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:09.754 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:09.754 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmdir_latency": { 2024-02-02T15:47:09.755 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:09.755 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:09.755 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:09.755 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:09.755 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmsnap_latency": { 2024-02-02T15:47:09.755 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:09.755 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:09.755 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:09.756 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:09.756 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmxattr_latency": { 2024-02-02T15:47:09.756 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:09.756 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:09.756 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:09.756 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:09.756 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setattr_latency": { 2024-02-02T15:47:09.756 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 31, 2024-02-02T15:47:09.756 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.605241773, 2024-02-02T15:47:09.757 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.019523928 2024-02-02T15:47:09.757 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:09.757 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setdirlayout_latency": { 2024-02-02T15:47:09.757 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:09.757 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:09.757 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:09.757 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:09.757 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setfilelock_latency": { 2024-02-02T15:47:09.757 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 348, 2024-02-02T15:47:09.758 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.769801105, 2024-02-02T15:47:09.758 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.002212072 2024-02-02T15:47:09.758 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:09.758 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setlayout_latency": { 2024-02-02T15:47:09.758 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:09.758 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:09.758 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:09.758 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:09.758 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setxattr_latency": { 2024-02-02T15:47:09.759 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:09.759 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:09.759 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:09.759 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:09.759 INFO:teuthology.orchestra.run.smithi142.stdout: "req_symlink_latency": { 2024-02-02T15:47:09.759 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:09.759 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:09.759 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:09.759 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:09.760 INFO:teuthology.orchestra.run.smithi142.stdout: "req_unlink_latency": { 2024-02-02T15:47:09.760 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 14617, 2024-02-02T15:47:09.760 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 61.126030064, 2024-02-02T15:47:09.760 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.004181845 2024-02-02T15:47:09.760 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:09.760 INFO:teuthology.orchestra.run.smithi142.stdout: "cap_revoke_eviction": 0, 2024-02-02T15:47:09.760 INFO:teuthology.orchestra.run.smithi142.stdout: "cap_acquisition_throttle": 0, 2024-02-02T15:47:09.760 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getvxattr_latency": { 2024-02-02T15:47:09.760 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:09.760 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:09.761 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:09.761 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-02-02T15:47:09.761 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:09.761 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_sessions": { 2024-02-02T15:47:09.761 INFO:teuthology.orchestra.run.smithi142.stdout: "session_count": 2, 2024-02-02T15:47:09.761 INFO:teuthology.orchestra.run.smithi142.stdout: "session_add": 3, 2024-02-02T15:47:09.761 INFO:teuthology.orchestra.run.smithi142.stdout: "session_remove": 1, 2024-02-02T15:47:09.761 INFO:teuthology.orchestra.run.smithi142.stdout: "sessions_open": 2, 2024-02-02T15:47:09.761 INFO:teuthology.orchestra.run.smithi142.stdout: "sessions_stale": 0, 2024-02-02T15:47:09.762 INFO:teuthology.orchestra.run.smithi142.stdout: "total_load": 19798, 2024-02-02T15:47:09.762 INFO:teuthology.orchestra.run.smithi142.stdout: "average_load": 9899, 2024-02-02T15:47:09.762 INFO:teuthology.orchestra.run.smithi142.stdout: "avg_session_uptime": 1848, 2024-02-02T15:47:09.762 INFO:teuthology.orchestra.run.smithi142.stdout: "mdthresh_evicted": 0 2024-02-02T15:47:09.762 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:09.762 INFO:teuthology.orchestra.run.smithi142.stdout: "mempool": { 2024-02-02T15:47:09.762 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_bytes": 0, 2024-02-02T15:47:09.762 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_items": 0, 2024-02-02T15:47:09.762 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_bytes": 0, 2024-02-02T15:47:09.763 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_items": 0, 2024-02-02T15:47:09.763 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_bytes": 0, 2024-02-02T15:47:09.763 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_items": 0, 2024-02-02T15:47:09.763 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-02T15:47:09.763 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_items": 0, 2024-02-02T15:47:09.763 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-02T15:47:09.763 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_items": 0, 2024-02-02T15:47:09.763 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_bytes": 0, 2024-02-02T15:47:09.763 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_items": 0, 2024-02-02T15:47:09.764 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-02T15:47:09.764 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_items": 0, 2024-02-02T15:47:09.764 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_bytes": 0, 2024-02-02T15:47:09.764 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_items": 0, 2024-02-02T15:47:09.764 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_bytes": 0, 2024-02-02T15:47:09.764 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_items": 0, 2024-02-02T15:47:09.764 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-02T15:47:09.764 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_items": 0, 2024-02-02T15:47:09.765 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-02T15:47:09.765 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_items": 0, 2024-02-02T15:47:09.765 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_bytes": 0, 2024-02-02T15:47:09.765 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_items": 0, 2024-02-02T15:47:09.765 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_bytes": 0, 2024-02-02T15:47:09.765 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_items": 0, 2024-02-02T15:47:09.765 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-02T15:47:09.765 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_items": 0, 2024-02-02T15:47:09.765 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_bytes": 0, 2024-02-02T15:47:09.766 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_items": 0, 2024-02-02T15:47:09.766 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_bytes": 0, 2024-02-02T15:47:09.766 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_items": 0, 2024-02-02T15:47:09.766 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_bytes": 0, 2024-02-02T15:47:09.766 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_items": 0, 2024-02-02T15:47:09.766 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_bytes": 0, 2024-02-02T15:47:09.766 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_items": 0, 2024-02-02T15:47:09.766 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_bytes": 384593, 2024-02-02T15:47:09.767 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_items": 1557, 2024-02-02T15:47:09.767 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_bytes": 176, 2024-02-02T15:47:09.767 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_items": 2, 2024-02-02T15:47:09.767 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_bytes": 0, 2024-02-02T15:47:09.767 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_items": 0, 2024-02-02T15:47:09.767 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_bytes": 0, 2024-02-02T15:47:09.767 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_items": 0, 2024-02-02T15:47:09.767 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_bytes": 0, 2024-02-02T15:47:09.768 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_items": 0, 2024-02-02T15:47:09.768 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_bytes": 9848, 2024-02-02T15:47:09.768 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_items": 190, 2024-02-02T15:47:09.768 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_bytes": 0, 2024-02-02T15:47:09.768 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_items": 0, 2024-02-02T15:47:09.768 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_bytes": 0, 2024-02-02T15:47:09.768 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_items": 0, 2024-02-02T15:47:09.768 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_bytes": 5467382, 2024-02-02T15:47:09.769 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_items": 53264, 2024-02-02T15:47:09.769 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_bytes": 0, 2024-02-02T15:47:09.769 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_items": 0, 2024-02-02T15:47:09.769 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_bytes": 0, 2024-02-02T15:47:09.769 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_items": 0 2024-02-02T15:47:09.769 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:09.769 INFO:teuthology.orchestra.run.smithi142.stdout: "objecter": { 2024-02-02T15:47:09.770 INFO:teuthology.orchestra.run.smithi142.stdout: "op_active": 0, 2024-02-02T15:47:09.770 INFO:teuthology.orchestra.run.smithi142.stdout: "op_laggy": 0, 2024-02-02T15:47:09.770 INFO:teuthology.orchestra.run.smithi142.stdout: "op_send": 56417, 2024-02-02T15:47:09.770 INFO:teuthology.orchestra.run.smithi142.stdout: "op_send_bytes": 430445468, 2024-02-02T15:47:09.770 INFO:teuthology.orchestra.run.smithi142.stdout: "op_resend": 33, 2024-02-02T15:47:09.770 INFO:teuthology.orchestra.run.smithi142.stdout: "op_reply": 56417, 2024-02-02T15:47:09.770 INFO:teuthology.orchestra.run.smithi142.stdout: "op_latency": { 2024-02-02T15:47:09.770 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 56417, 2024-02-02T15:47:09.771 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 684.455590770, 2024-02-02T15:47:09.771 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.012132080 2024-02-02T15:47:09.771 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:09.771 INFO:teuthology.orchestra.run.smithi142.stdout: "op_inflight": 0, 2024-02-02T15:47:09.771 INFO:teuthology.orchestra.run.smithi142.stdout: "oplen_avg": { 2024-02-02T15:47:09.771 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 56417, 2024-02-02T15:47:09.771 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 56883 2024-02-02T15:47:09.771 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:09.772 INFO:teuthology.orchestra.run.smithi142.stdout: "op": 56417, 2024-02-02T15:47:09.772 INFO:teuthology.orchestra.run.smithi142.stdout: "op_r": 2947, 2024-02-02T15:47:09.772 INFO:teuthology.orchestra.run.smithi142.stdout: "op_w": 53470, 2024-02-02T15:47:09.772 INFO:teuthology.orchestra.run.smithi142.stdout: "op_rmw": 0, 2024-02-02T15:47:09.772 INFO:teuthology.orchestra.run.smithi142.stdout: "op_pg": 0, 2024-02-02T15:47:09.772 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_stat": 0, 2024-02-02T15:47:09.772 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_create": 10, 2024-02-02T15:47:09.772 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_read": 805, 2024-02-02T15:47:09.773 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_write": 23386, 2024-02-02T15:47:09.773 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_writefull": 469, 2024-02-02T15:47:09.773 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_writesame": 0, 2024-02-02T15:47:09.773 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_append": 0, 2024-02-02T15:47:09.773 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_zero": 0, 2024-02-02T15:47:09.773 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_truncate": 3, 2024-02-02T15:47:09.773 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_delete": 29347, 2024-02-02T15:47:09.773 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_mapext": 0, 2024-02-02T15:47:09.774 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_sparse_read": 0, 2024-02-02T15:47:09.774 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_clonerange": 0, 2024-02-02T15:47:09.774 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_getxattr": 2140, 2024-02-02T15:47:09.774 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_setxattr": 20, 2024-02-02T15:47:09.774 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_cmpxattr": 0, 2024-02-02T15:47:09.774 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_rmxattr": 0, 2024-02-02T15:47:09.774 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_resetxattrs": 0, 2024-02-02T15:47:09.774 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_call": 0, 2024-02-02T15:47:09.775 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_watch": 0, 2024-02-02T15:47:09.775 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_notify": 0, 2024-02-02T15:47:09.775 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_src_cmpxattr": 0, 2024-02-02T15:47:09.775 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_pgls": 0, 2024-02-02T15:47:09.775 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_pgls_filter": 0, 2024-02-02T15:47:09.775 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_other": 0, 2024-02-02T15:47:09.775 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_active": 0, 2024-02-02T15:47:09.775 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_send": 0, 2024-02-02T15:47:09.776 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_resend": 0, 2024-02-02T15:47:09.776 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_ping": 0, 2024-02-02T15:47:09.776 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_active": 0, 2024-02-02T15:47:09.776 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_send": 0, 2024-02-02T15:47:09.776 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_resend": 0, 2024-02-02T15:47:09.776 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_active": 0, 2024-02-02T15:47:09.776 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_send": 0, 2024-02-02T15:47:09.776 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_resend": 0, 2024-02-02T15:47:09.777 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_active": 0, 2024-02-02T15:47:09.777 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_send": 0, 2024-02-02T15:47:09.777 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_resend": 0, 2024-02-02T15:47:09.777 INFO:teuthology.orchestra.run.smithi142.stdout: "command_active": 0, 2024-02-02T15:47:09.777 INFO:teuthology.orchestra.run.smithi142.stdout: "command_send": 0, 2024-02-02T15:47:09.777 INFO:teuthology.orchestra.run.smithi142.stdout: "command_resend": 0, 2024-02-02T15:47:09.777 INFO:teuthology.orchestra.run.smithi142.stdout: "map_epoch": 116, 2024-02-02T15:47:09.777 INFO:teuthology.orchestra.run.smithi142.stdout: "map_full": 0, 2024-02-02T15:47:09.777 INFO:teuthology.orchestra.run.smithi142.stdout: "map_inc": 32, 2024-02-02T15:47:09.778 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_sessions": 12, 2024-02-02T15:47:09.778 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_session_open": 12, 2024-02-02T15:47:09.778 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_session_close": 0, 2024-02-02T15:47:09.778 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_laggy": 0, 2024-02-02T15:47:09.778 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_wr": 487, 2024-02-02T15:47:09.778 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_rd": 3, 2024-02-02T15:47:09.778 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_del": 213 2024-02-02T15:47:09.778 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:09.778 INFO:teuthology.orchestra.run.smithi142.stdout: "oft": { 2024-02-02T15:47:09.778 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_objs": 1, 2024-02-02T15:47:09.779 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_kv_pairs": 0, 2024-02-02T15:47:09.779 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_updates": 0, 2024-02-02T15:47:09.779 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_removes": 0 2024-02-02T15:47:09.779 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:09.779 INFO:teuthology.orchestra.run.smithi142.stdout: "osd-slow-ops": { 2024-02-02T15:47:09.779 INFO:teuthology.orchestra.run.smithi142.stdout: "slow_ops_count": 106446 2024-02-02T15:47:09.779 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:09.779 INFO:teuthology.orchestra.run.smithi142.stdout: "purge_queue": { 2024-02-02T15:47:09.779 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_ops": 0, 2024-02-02T15:47:09.780 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_ops_high_water": 10, 2024-02-02T15:47:09.780 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing": 0, 2024-02-02T15:47:09.780 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_high_water": 5, 2024-02-02T15:47:09.780 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executed": 14617, 2024-02-02T15:47:09.780 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_item_in_journal": 0 2024-02-02T15:47:09.780 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:09.780 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-02T15:47:09.780 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-02-02T15:47:09.781 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 104857600, 2024-02-02T15:47:09.781 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-02-02T15:47:09.781 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 605932, 2024-02-02T15:47:09.781 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 128155941, 2024-02-02T15:47:09.781 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-02-02T15:47:09.781 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 605932, 2024-02-02T15:47:09.781 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-02-02T15:47:09.781 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-02-02T15:47:09.781 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 605932, 2024-02-02T15:47:09.782 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 128155941, 2024-02-02T15:47:09.782 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-02-02T15:47:09.782 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:09.782 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:09.782 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:09.782 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-02-02T15:47:09.782 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:09.782 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-objecter_bytes": { 2024-02-02T15:47:09.782 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-02-02T15:47:09.783 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 104857600, 2024-02-02T15:47:09.783 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-02-02T15:47:09.783 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-02-02T15:47:09.783 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-02-02T15:47:09.783 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-02-02T15:47:09.783 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-02-02T15:47:09.783 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 56417, 2024-02-02T15:47:09.783 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 432041892, 2024-02-02T15:47:09.784 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 56417, 2024-02-02T15:47:09.784 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 432041892, 2024-02-02T15:47:09.784 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-02-02T15:47:09.784 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:09.784 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:09.784 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:09.784 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-02-02T15:47:09.784 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:09.785 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-objecter_ops": { 2024-02-02T15:47:09.785 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-02-02T15:47:09.785 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 1024, 2024-02-02T15:47:09.785 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-02-02T15:47:09.785 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-02-02T15:47:09.785 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-02-02T15:47:09.785 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-02-02T15:47:09.785 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-02-02T15:47:09.785 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 56417, 2024-02-02T15:47:09.786 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 56417, 2024-02-02T15:47:09.786 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 56417, 2024-02-02T15:47:09.786 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 56417, 2024-02-02T15:47:09.786 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-02-02T15:47:09.786 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:09.786 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:09.786 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:09.786 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-02-02T15:47:09.786 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:09.787 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-write_buf_throttle": { 2024-02-02T15:47:09.787 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-02-02T15:47:09.787 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 3758096384, 2024-02-02T15:47:09.787 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-02-02T15:47:09.787 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 14617, 2024-02-02T15:47:09.787 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 1596469, 2024-02-02T15:47:09.787 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-02-02T15:47:09.787 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 14617, 2024-02-02T15:47:09.788 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-02-02T15:47:09.788 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-02-02T15:47:09.788 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 805, 2024-02-02T15:47:09.788 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 1596469, 2024-02-02T15:47:09.788 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-02-02T15:47:09.788 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:09.788 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:09.789 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:09.789 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-02-02T15:47:09.789 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:09.789 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-write_buf_throttle-0x557a13f02000": { 2024-02-02T15:47:09.789 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-02-02T15:47:09.789 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 3758096384, 2024-02-02T15:47:09.789 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-02-02T15:47:09.789 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 83277, 2024-02-02T15:47:09.789 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 427306335, 2024-02-02T15:47:09.790 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-02-02T15:47:09.790 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 83277, 2024-02-02T15:47:09.790 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-02-02T15:47:09.790 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-02-02T15:47:09.790 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 22581, 2024-02-02T15:47:09.790 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 427306335, 2024-02-02T15:47:09.790 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-02-02T15:47:09.790 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:09.790 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:09.791 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:09.791 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-02-02T15:47:09.791 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-02-02T15:47:09.791 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-02-02T15:47:09.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:47:09 smithi071 ceph-mon[44707]: pgmap v634: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 151 KiB/s wr, 34 op/s 2024-02-02T15:47:09.869 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:47:09 smithi142 ceph-mon[45728]: pgmap v634: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 151 KiB/s wr, 34 op/s 2024-02-02T15:47:09.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:47:09 smithi001 ceph-mon[35470]: pgmap v634: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 151 KiB/s wr, 34 op/s 2024-02-02T15:47:10.083 INFO:tasks.check_counter:Daemon mds.f mds.exported=6 2024-02-02T15:47:10.084 INFO:tasks.check_counter:Daemon mds.f mds.imported=9 2024-02-02T15:47:10.084 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=1143 2024-02-02T15:47:10.084 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=2102 2024-02-02T15:47:10.084 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=6 2024-02-02T15:47:10.084 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-02-02T15:47:10.084 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-02-02T15:47:10.084 DEBUG:tasks.check_counter:Getting stats from i 2024-02-02T15:47:10.085 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph daemon mds.i perf dump 2024-02-02T15:47:10.312 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.c/config 2024-02-02T15:47:11.260 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-02-02T15:47:11.260 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-0": { 2024-02-02T15:47:11.260 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 312807, 2024-02-02T15:47:11.260 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 253996, 2024-02-02T15:47:11.260 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 96508890, 2024-02-02T15:47:11.260 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 172338210, 2024-02-02T15:47:11.260 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 76, 2024-02-02T15:47:11.261 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 7, 2024-02-02T15:47:11.261 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 15.880259712, 2024-02-02T15:47:11.261 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 6.357858831, 2024-02-02T15:47:11.261 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 11.465090156, 2024-02-02T15:47:11.261 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.707317113, 2024-02-02T15:47:11.261 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-02-02T15:47:11.261 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 253995, 2024-02-02T15:47:11.261 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 38.685400083, 2024-02-02T15:47:11.262 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000152307 2024-02-02T15:47:11.262 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:11.262 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-02-02T15:47:11.262 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 507557, 2024-02-02T15:47:11.262 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.085806360, 2024-02-02T15:47:11.262 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000169 2024-02-02T15:47:11.262 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:11.262 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 2281200, 2024-02-02T15:47:11.262 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 254816 2024-02-02T15:47:11.263 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:11.263 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-1": { 2024-02-02T15:47:11.263 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 75880, 2024-02-02T15:47:11.263 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 75134, 2024-02-02T15:47:11.263 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 19181937, 2024-02-02T15:47:11.263 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 112282816, 2024-02-02T15:47:11.263 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 22, 2024-02-02T15:47:11.263 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 6, 2024-02-02T15:47:11.264 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 5.055065759, 2024-02-02T15:47:11.264 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 2.105187533, 2024-02-02T15:47:11.264 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 2.652018115, 2024-02-02T15:47:11.264 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.748361880, 2024-02-02T15:47:11.264 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-02-02T15:47:11.264 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 75131, 2024-02-02T15:47:11.264 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 19.295927125, 2024-02-02T15:47:11.264 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000256830 2024-02-02T15:47:11.265 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:11.265 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-02-02T15:47:11.265 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 105089, 2024-02-02T15:47:11.265 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.020719120, 2024-02-02T15:47:11.265 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000197 2024-02-02T15:47:11.265 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:11.265 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 3584, 2024-02-02T15:47:11.265 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 644656 2024-02-02T15:47:11.265 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:11.266 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-2": { 2024-02-02T15:47:11.266 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 311376, 2024-02-02T15:47:11.266 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 419897, 2024-02-02T15:47:11.266 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 95968414, 2024-02-02T15:47:11.266 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 413931725, 2024-02-02T15:47:11.266 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 10, 2024-02-02T15:47:11.266 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 7, 2024-02-02T15:47:11.266 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 16.101985975, 2024-02-02T15:47:11.266 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 7.121417930, 2024-02-02T15:47:11.267 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 10.411068498, 2024-02-02T15:47:11.267 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 1.130837045, 2024-02-02T15:47:11.267 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-02-02T15:47:11.267 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 419896, 2024-02-02T15:47:11.267 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 89.835597912, 2024-02-02T15:47:11.267 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000213947 2024-02-02T15:47:11.267 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:11.267 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-02-02T15:47:11.267 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 390643, 2024-02-02T15:47:11.268 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.101517616, 2024-02-02T15:47:11.268 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000259 2024-02-02T15:47:11.268 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:11.268 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-02-02T15:47:11.268 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 448 2024-02-02T15:47:11.268 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:11.268 INFO:teuthology.orchestra.run.smithi142.stdout: "cct": { 2024-02-02T15:47:11.268 INFO:teuthology.orchestra.run.smithi142.stdout: "total_workers": 1, 2024-02-02T15:47:11.269 INFO:teuthology.orchestra.run.smithi142.stdout: "unhealthy_workers": 0 2024-02-02T15:47:11.269 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:11.269 INFO:teuthology.orchestra.run.smithi142.stdout: "finisher-MDSRank": { 2024-02-02T15:47:11.269 INFO:teuthology.orchestra.run.smithi142.stdout: "queue_len": 0, 2024-02-02T15:47:11.269 INFO:teuthology.orchestra.run.smithi142.stdout: "complete_latency": { 2024-02-02T15:47:11.269 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 51169, 2024-02-02T15:47:11.269 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 56.707737397, 2024-02-02T15:47:11.269 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.001108244 2024-02-02T15:47:11.269 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-02-02T15:47:11.270 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:11.270 INFO:teuthology.orchestra.run.smithi142.stdout: "finisher-PurgeQueue": { 2024-02-02T15:47:11.270 INFO:teuthology.orchestra.run.smithi142.stdout: "queue_len": 0, 2024-02-02T15:47:11.270 INFO:teuthology.orchestra.run.smithi142.stdout: "complete_latency": { 2024-02-02T15:47:11.270 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 38615, 2024-02-02T15:47:11.270 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 4.451116079, 2024-02-02T15:47:11.270 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000115269 2024-02-02T15:47:11.270 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-02-02T15:47:11.270 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:11.271 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2024-02-02T15:47:11.271 INFO:teuthology.orchestra.run.smithi142.stdout: "request": 157312, 2024-02-02T15:47:11.271 INFO:teuthology.orchestra.run.smithi142.stdout: "reply": 131092, 2024-02-02T15:47:11.271 INFO:teuthology.orchestra.run.smithi142.stdout: "reply_latency": { 2024-02-02T15:47:11.271 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 131092, 2024-02-02T15:47:11.271 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 616.426744098, 2024-02-02T15:47:11.271 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.004702245 2024-02-02T15:47:11.271 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:11.272 INFO:teuthology.orchestra.run.smithi142.stdout: "slow_reply": 0, 2024-02-02T15:47:11.272 INFO:teuthology.orchestra.run.smithi142.stdout: "forward": 26220, 2024-02-02T15:47:11.272 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_fetch_complete": 6, 2024-02-02T15:47:11.272 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_fetch_keys": 0, 2024-02-02T15:47:11.272 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_commit": 29, 2024-02-02T15:47:11.272 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_split": 2, 2024-02-02T15:47:11.272 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_merge": 1, 2024-02-02T15:47:11.272 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes": 1237, 2024-02-02T15:47:11.273 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_top": 236, 2024-02-02T15:47:11.273 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_bottom": 912, 2024-02-02T15:47:11.273 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_pin_tail": 89, 2024-02-02T15:47:11.273 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_pinned": 900, 2024-02-02T15:47:11.273 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_expired": 750, 2024-02-02T15:47:11.273 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_with_caps": 105, 2024-02-02T15:47:11.273 INFO:teuthology.orchestra.run.smithi142.stdout: "caps": 105, 2024-02-02T15:47:11.273 INFO:teuthology.orchestra.run.smithi142.stdout: "subtrees": 25, 2024-02-02T15:47:11.273 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse": 258076, 2024-02-02T15:47:11.274 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_hit": 122496, 2024-02-02T15:47:11.274 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_forward": 26202, 2024-02-02T15:47:11.274 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_discover": 274, 2024-02-02T15:47:11.274 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_dir_fetch": 0, 2024-02-02T15:47:11.274 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_remote_ino": 0, 2024-02-02T15:47:11.274 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_lock": 1821, 2024-02-02T15:47:11.274 INFO:teuthology.orchestra.run.smithi142.stdout: "load_cent": 1598, 2024-02-02T15:47:11.274 INFO:teuthology.orchestra.run.smithi142.stdout: "q": 0, 2024-02-02T15:47:11.275 INFO:teuthology.orchestra.run.smithi142.stdout: "exported": 9, 2024-02-02T15:47:11.275 INFO:teuthology.orchestra.run.smithi142.stdout: "exported_inodes": 330, 2024-02-02T15:47:11.275 INFO:teuthology.orchestra.run.smithi142.stdout: "imported": 14, 2024-02-02T15:47:11.275 INFO:teuthology.orchestra.run.smithi142.stdout: "imported_inodes": 661, 2024-02-02T15:47:11.275 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_dir_fetch": 0, 2024-02-02T15:47:11.275 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_backtrace_fetch": 1630, 2024-02-02T15:47:11.275 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_peer_discover": 0, 2024-02-02T15:47:11.275 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rfiles": 670, 2024-02-02T15:47:11.275 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rbytes": 218701295, 2024-02-02T15:47:11.276 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rsnaps": 6, 2024-02-02T15:47:11.276 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_backtrace_fetch": 1630, 2024-02-02T15:47:11.276 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_set_tag": 0, 2024-02-02T15:47:11.276 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_backtrace_repaired": 0, 2024-02-02T15:47:11.276 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_inotable_repaired": 0, 2024-02-02T15:47:11.276 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dir_inodes": 200, 2024-02-02T15:47:11.276 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dir_base_inodes": 0, 2024-02-02T15:47:11.276 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dirfrag_rstats": 200, 2024-02-02T15:47:11.277 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_file_inodes": 1430, 2024-02-02T15:47:11.277 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_inode_file_caps": 18, 2024-02-02T15:47:11.277 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_revoke": 19426, 2024-02-02T15:47:11.277 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_grant": 80307, 2024-02-02T15:47:11.277 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_trunc": 0, 2024-02-02T15:47:11.277 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-02-02T15:47:11.277 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_flush_ack": 0, 2024-02-02T15:47:11.277 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_caps": 38651, 2024-02-02T15:47:11.277 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_caps_dirty": 9790, 2024-02-02T15:47:11.278 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_cap_release": 3677, 2024-02-02T15:47:11.278 INFO:teuthology.orchestra.run.smithi142.stdout: "process_request_cap_release": 107230 2024-02-02T15:47:11.278 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:11.278 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_cache": { 2024-02-02T15:47:11.278 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_update": 740, 2024-02-02T15:47:11.278 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_update_receipt": 1710, 2024-02-02T15:47:11.278 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_try_discover": 936, 2024-02-02T15:47:11.278 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_send_discover": 1671, 2024-02-02T15:47:11.278 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_handle_discover": 1554, 2024-02-02T15:47:11.279 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays": 0, 2024-02-02T15:47:11.279 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays_delayed": 0, 2024-02-02T15:47:11.279 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays_enqueuing": 0, 2024-02-02T15:47:11.279 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_created": 20580, 2024-02-02T15:47:11.279 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_enqueued": 20580, 2024-02-02T15:47:11.279 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_reintegrated": 0, 2024-02-02T15:47:11.279 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_migrated": 0, 2024-02-02T15:47:11.279 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_processing": 0, 2024-02-02T15:47:11.279 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_enqueued": 0, 2024-02-02T15:47:11.280 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_prioritized": 0, 2024-02-02T15:47:11.280 INFO:teuthology.orchestra.run.smithi142.stdout: "recovery_started": 0, 2024-02-02T15:47:11.280 INFO:teuthology.orchestra.run.smithi142.stdout: "recovery_completed": 0, 2024-02-02T15:47:11.280 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_enqueue_scrub": 0, 2024-02-02T15:47:11.280 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_exportdir": 15, 2024-02-02T15:47:11.280 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_flush": 0, 2024-02-02T15:47:11.280 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_fragmentdir": 3, 2024-02-02T15:47:11.280 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_fragstats": 0, 2024-02-02T15:47:11.281 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_inodestats": 0 2024-02-02T15:47:11.281 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:11.281 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_log": { 2024-02-02T15:47:11.281 INFO:teuthology.orchestra.run.smithi142.stdout: "evadd": 98346, 2024-02-02T15:47:11.281 INFO:teuthology.orchestra.run.smithi142.stdout: "evex": 0, 2024-02-02T15:47:11.281 INFO:teuthology.orchestra.run.smithi142.stdout: "evtrm": 0, 2024-02-02T15:47:11.281 INFO:teuthology.orchestra.run.smithi142.stdout: "ev": 98346, 2024-02-02T15:47:11.281 INFO:teuthology.orchestra.run.smithi142.stdout: "evexg": 0, 2024-02-02T15:47:11.281 INFO:teuthology.orchestra.run.smithi142.stdout: "evexd": 0, 2024-02-02T15:47:11.282 INFO:teuthology.orchestra.run.smithi142.stdout: "segadd": 127, 2024-02-02T15:47:11.282 INFO:teuthology.orchestra.run.smithi142.stdout: "segex": 0, 2024-02-02T15:47:11.282 INFO:teuthology.orchestra.run.smithi142.stdout: "segtrm": 0, 2024-02-02T15:47:11.282 INFO:teuthology.orchestra.run.smithi142.stdout: "seg": 127, 2024-02-02T15:47:11.282 INFO:teuthology.orchestra.run.smithi142.stdout: "segexg": 0, 2024-02-02T15:47:11.282 INFO:teuthology.orchestra.run.smithi142.stdout: "segexd": 0, 2024-02-02T15:47:11.282 INFO:teuthology.orchestra.run.smithi142.stdout: "expos": 4194304, 2024-02-02T15:47:11.282 INFO:teuthology.orchestra.run.smithi142.stdout: "wrpos": 372760107, 2024-02-02T15:47:11.282 INFO:teuthology.orchestra.run.smithi142.stdout: "rdpos": 0, 2024-02-02T15:47:11.283 INFO:teuthology.orchestra.run.smithi142.stdout: "jlat": { 2024-02-02T15:47:11.283 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 26917, 2024-02-02T15:47:11.283 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 208.008036485, 2024-02-02T15:47:11.283 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.007727757 2024-02-02T15:47:11.283 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:11.283 INFO:teuthology.orchestra.run.smithi142.stdout: "replayed": 0 2024-02-02T15:47:11.283 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:11.283 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_mem": { 2024-02-02T15:47:11.283 INFO:teuthology.orchestra.run.smithi142.stdout: "ino": 907, 2024-02-02T15:47:11.284 INFO:teuthology.orchestra.run.smithi142.stdout: "ino+": 21940, 2024-02-02T15:47:11.284 INFO:teuthology.orchestra.run.smithi142.stdout: "ino-": 21033, 2024-02-02T15:47:11.284 INFO:teuthology.orchestra.run.smithi142.stdout: "dir": 204, 2024-02-02T15:47:11.284 INFO:teuthology.orchestra.run.smithi142.stdout: "dir+": 948, 2024-02-02T15:47:11.284 INFO:teuthology.orchestra.run.smithi142.stdout: "dir-": 744, 2024-02-02T15:47:11.284 INFO:teuthology.orchestra.run.smithi142.stdout: "dn": 1237, 2024-02-02T15:47:11.284 INFO:teuthology.orchestra.run.smithi142.stdout: "dn+": 36398, 2024-02-02T15:47:11.284 INFO:teuthology.orchestra.run.smithi142.stdout: "dn-": 35161, 2024-02-02T15:47:11.284 INFO:teuthology.orchestra.run.smithi142.stdout: "cap": 102, 2024-02-02T15:47:11.285 INFO:teuthology.orchestra.run.smithi142.stdout: "cap+": 22127, 2024-02-02T15:47:11.285 INFO:teuthology.orchestra.run.smithi142.stdout: "cap-": 22025, 2024-02-02T15:47:11.285 INFO:teuthology.orchestra.run.smithi142.stdout: "rss": 72328, 2024-02-02T15:47:11.285 INFO:teuthology.orchestra.run.smithi142.stdout: "heap": 198964 2024-02-02T15:47:11.285 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:11.285 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_server": { 2024-02-02T15:47:11.285 INFO:teuthology.orchestra.run.smithi142.stdout: "dispatch_client_request": 231780, 2024-02-02T15:47:11.285 INFO:teuthology.orchestra.run.smithi142.stdout: "dispatch_server_request": 1527, 2024-02-02T15:47:11.286 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_request": 157312, 2024-02-02T15:47:11.286 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_session": 1482, 2024-02-02T15:47:11.286 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_peer_request": 13866, 2024-02-02T15:47:11.286 INFO:teuthology.orchestra.run.smithi142.stdout: "req_create_latency": { 2024-02-02T15:47:11.286 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 20517, 2024-02-02T15:47:11.286 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 102.409035557, 2024-02-02T15:47:11.286 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.004991423 2024-02-02T15:47:11.286 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:11.286 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getattr_latency": { 2024-02-02T15:47:11.287 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 5988, 2024-02-02T15:47:11.287 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 87.309845255, 2024-02-02T15:47:11.287 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.014580802 2024-02-02T15:47:11.287 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:11.287 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getfilelock_latency": { 2024-02-02T15:47:11.287 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:11.287 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:11.287 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:11.288 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:11.288 INFO:teuthology.orchestra.run.smithi142.stdout: "req_link_latency": { 2024-02-02T15:47:11.288 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:11.288 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:11.288 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:11.288 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:11.288 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookup_latency": { 2024-02-02T15:47:11.288 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 33925, 2024-02-02T15:47:11.289 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 114.344658558, 2024-02-02T15:47:11.289 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.003370513 2024-02-02T15:47:11.289 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:11.289 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookuphash_latency": { 2024-02-02T15:47:11.289 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:11.289 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:11.289 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:11.289 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:11.289 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupino_latency": { 2024-02-02T15:47:11.290 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:11.290 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:11.290 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:11.290 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:11.290 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupname_latency": { 2024-02-02T15:47:11.290 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:11.290 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:11.290 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:11.291 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:11.291 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupparent_latency": { 2024-02-02T15:47:11.291 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:11.291 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:11.291 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:11.291 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:11.291 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupsnap_latency": { 2024-02-02T15:47:11.291 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:11.291 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:11.292 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:11.292 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:11.292 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lssnap_latency": { 2024-02-02T15:47:11.292 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:11.292 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:11.292 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:11.292 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:11.292 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mkdir_latency": { 2024-02-02T15:47:11.293 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:11.293 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:11.293 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:11.293 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:11.293 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mknod_latency": { 2024-02-02T15:47:11.293 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:11.293 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:11.293 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:11.293 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:11.294 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mksnap_latency": { 2024-02-02T15:47:11.294 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:11.294 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:11.294 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:11.294 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:11.294 INFO:teuthology.orchestra.run.smithi142.stdout: "req_open_latency": { 2024-02-02T15:47:11.294 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 3, 2024-02-02T15:47:11.294 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.001753508, 2024-02-02T15:47:11.295 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000584502 2024-02-02T15:47:11.295 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:11.295 INFO:teuthology.orchestra.run.smithi142.stdout: "req_readdir_latency": { 2024-02-02T15:47:11.295 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 46932, 2024-02-02T15:47:11.295 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 139.193229365, 2024-02-02T15:47:11.295 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.002965849 2024-02-02T15:47:11.295 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:11.295 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rename_latency": { 2024-02-02T15:47:11.295 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 2749, 2024-02-02T15:47:11.296 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 94.806507037, 2024-02-02T15:47:11.296 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.034487634 2024-02-02T15:47:11.296 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:11.296 INFO:teuthology.orchestra.run.smithi142.stdout: "req_renamesnap_latency": { 2024-02-02T15:47:11.296 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:11.296 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:11.296 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:11.296 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:11.297 INFO:teuthology.orchestra.run.smithi142.stdout: "req_snapdiff_latency": { 2024-02-02T15:47:11.297 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:11.297 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:11.297 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:11.297 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:11.297 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmdir_latency": { 2024-02-02T15:47:11.297 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:11.297 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:11.298 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:11.298 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:11.298 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmsnap_latency": { 2024-02-02T15:47:11.298 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:11.298 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:11.298 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:11.298 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:11.298 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmxattr_latency": { 2024-02-02T15:47:11.298 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:11.299 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:11.299 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:11.299 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:11.299 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setattr_latency": { 2024-02-02T15:47:11.299 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 26, 2024-02-02T15:47:11.299 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.672657841, 2024-02-02T15:47:11.299 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.025871455 2024-02-02T15:47:11.299 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:11.300 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setdirlayout_latency": { 2024-02-02T15:47:11.300 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:11.300 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:11.300 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:11.300 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:11.300 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setfilelock_latency": { 2024-02-02T15:47:11.300 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 372, 2024-02-02T15:47:11.300 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.500828914, 2024-02-02T15:47:11.301 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.001346314 2024-02-02T15:47:11.301 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:11.301 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setlayout_latency": { 2024-02-02T15:47:11.301 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:11.301 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:11.301 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:11.301 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:11.301 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setxattr_latency": { 2024-02-02T15:47:11.301 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:11.302 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:11.302 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:11.302 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:11.302 INFO:teuthology.orchestra.run.smithi142.stdout: "req_symlink_latency": { 2024-02-02T15:47:11.302 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:11.302 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:11.302 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:11.302 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:11.302 INFO:teuthology.orchestra.run.smithi142.stdout: "req_unlink_latency": { 2024-02-02T15:47:11.303 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 20580, 2024-02-02T15:47:11.303 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 77.188228063, 2024-02-02T15:47:11.303 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.003750642 2024-02-02T15:47:11.303 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:11.303 INFO:teuthology.orchestra.run.smithi142.stdout: "cap_revoke_eviction": 0, 2024-02-02T15:47:11.303 INFO:teuthology.orchestra.run.smithi142.stdout: "cap_acquisition_throttle": 0, 2024-02-02T15:47:11.303 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getvxattr_latency": { 2024-02-02T15:47:11.303 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:11.304 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:11.304 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:11.304 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-02-02T15:47:11.304 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:11.304 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_sessions": { 2024-02-02T15:47:11.304 INFO:teuthology.orchestra.run.smithi142.stdout: "session_count": 2, 2024-02-02T15:47:11.304 INFO:teuthology.orchestra.run.smithi142.stdout: "session_add": 3, 2024-02-02T15:47:11.304 INFO:teuthology.orchestra.run.smithi142.stdout: "session_remove": 1, 2024-02-02T15:47:11.304 INFO:teuthology.orchestra.run.smithi142.stdout: "sessions_open": 2, 2024-02-02T15:47:11.305 INFO:teuthology.orchestra.run.smithi142.stdout: "sessions_stale": 0, 2024-02-02T15:47:11.305 INFO:teuthology.orchestra.run.smithi142.stdout: "total_load": 19991, 2024-02-02T15:47:11.305 INFO:teuthology.orchestra.run.smithi142.stdout: "average_load": 9995, 2024-02-02T15:47:11.305 INFO:teuthology.orchestra.run.smithi142.stdout: "avg_session_uptime": 956, 2024-02-02T15:47:11.305 INFO:teuthology.orchestra.run.smithi142.stdout: "mdthresh_evicted": 0 2024-02-02T15:47:11.305 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:11.305 INFO:teuthology.orchestra.run.smithi142.stdout: "mempool": { 2024-02-02T15:47:11.305 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_bytes": 0, 2024-02-02T15:47:11.306 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_items": 0, 2024-02-02T15:47:11.306 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_bytes": 0, 2024-02-02T15:47:11.306 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_items": 0, 2024-02-02T15:47:11.306 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_bytes": 0, 2024-02-02T15:47:11.306 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_items": 0, 2024-02-02T15:47:11.306 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-02T15:47:11.306 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_items": 0, 2024-02-02T15:47:11.306 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-02T15:47:11.307 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_items": 0, 2024-02-02T15:47:11.307 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_bytes": 0, 2024-02-02T15:47:11.307 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_items": 0, 2024-02-02T15:47:11.307 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-02T15:47:11.307 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_items": 0, 2024-02-02T15:47:11.307 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_bytes": 0, 2024-02-02T15:47:11.307 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_items": 0, 2024-02-02T15:47:11.307 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_bytes": 0, 2024-02-02T15:47:11.307 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_items": 0, 2024-02-02T15:47:11.308 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-02T15:47:11.308 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_items": 0, 2024-02-02T15:47:11.308 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-02T15:47:11.308 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_items": 0, 2024-02-02T15:47:11.308 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_bytes": 0, 2024-02-02T15:47:11.308 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_items": 0, 2024-02-02T15:47:11.308 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_bytes": 0, 2024-02-02T15:47:11.308 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_items": 0, 2024-02-02T15:47:11.309 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-02T15:47:11.309 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_items": 0, 2024-02-02T15:47:11.309 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_bytes": 0, 2024-02-02T15:47:11.309 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_items": 0, 2024-02-02T15:47:11.309 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_bytes": 0, 2024-02-02T15:47:11.309 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_items": 0, 2024-02-02T15:47:11.309 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_bytes": 0, 2024-02-02T15:47:11.309 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_items": 0, 2024-02-02T15:47:11.310 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_bytes": 0, 2024-02-02T15:47:11.310 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_items": 0, 2024-02-02T15:47:11.310 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_bytes": 349184, 2024-02-02T15:47:11.310 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_items": 734, 2024-02-02T15:47:11.310 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_bytes": 264, 2024-02-02T15:47:11.310 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_items": 3, 2024-02-02T15:47:11.310 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_bytes": 0, 2024-02-02T15:47:11.310 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_items": 0, 2024-02-02T15:47:11.311 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_bytes": 0, 2024-02-02T15:47:11.311 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_items": 0, 2024-02-02T15:47:11.311 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_bytes": 0, 2024-02-02T15:47:11.311 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_items": 0, 2024-02-02T15:47:11.311 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_bytes": 9848, 2024-02-02T15:47:11.311 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_items": 190, 2024-02-02T15:47:11.311 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_bytes": 0, 2024-02-02T15:47:11.311 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_items": 0, 2024-02-02T15:47:11.311 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_bytes": 0, 2024-02-02T15:47:11.312 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_items": 0, 2024-02-02T15:47:11.312 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_bytes": 3685158, 2024-02-02T15:47:11.312 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_items": 32329, 2024-02-02T15:47:11.312 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_bytes": 0, 2024-02-02T15:47:11.312 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_items": 0, 2024-02-02T15:47:11.312 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_bytes": 0, 2024-02-02T15:47:11.312 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_items": 0 2024-02-02T15:47:11.312 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:11.313 INFO:teuthology.orchestra.run.smithi142.stdout: "objecter": { 2024-02-02T15:47:11.313 INFO:teuthology.orchestra.run.smithi142.stdout: "op_active": 0, 2024-02-02T15:47:11.313 INFO:teuthology.orchestra.run.smithi142.stdout: "op_laggy": 0, 2024-02-02T15:47:11.313 INFO:teuthology.orchestra.run.smithi142.stdout: "op_send": 72489, 2024-02-02T15:47:11.313 INFO:teuthology.orchestra.run.smithi142.stdout: "op_send_bytes": 372317659, 2024-02-02T15:47:11.313 INFO:teuthology.orchestra.run.smithi142.stdout: "op_resend": 27, 2024-02-02T15:47:11.313 INFO:teuthology.orchestra.run.smithi142.stdout: "op_reply": 72489, 2024-02-02T15:47:11.313 INFO:teuthology.orchestra.run.smithi142.stdout: "op_latency": { 2024-02-02T15:47:11.314 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 72489, 2024-02-02T15:47:11.314 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 721.328907456, 2024-02-02T15:47:11.314 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.009950874 2024-02-02T15:47:11.314 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:11.314 INFO:teuthology.orchestra.run.smithi142.stdout: "op_inflight": 0, 2024-02-02T15:47:11.314 INFO:teuthology.orchestra.run.smithi142.stdout: "oplen_avg": { 2024-02-02T15:47:11.314 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 72489, 2024-02-02T15:47:11.314 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 72957 2024-02-02T15:47:11.314 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:11.315 INFO:teuthology.orchestra.run.smithi142.stdout: "op": 72489, 2024-02-02T15:47:11.315 INFO:teuthology.orchestra.run.smithi142.stdout: "op_r": 2578, 2024-02-02T15:47:11.315 INFO:teuthology.orchestra.run.smithi142.stdout: "op_w": 69911, 2024-02-02T15:47:11.315 INFO:teuthology.orchestra.run.smithi142.stdout: "op_rmw": 0, 2024-02-02T15:47:11.315 INFO:teuthology.orchestra.run.smithi142.stdout: "op_pg": 0, 2024-02-02T15:47:11.315 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_stat": 0, 2024-02-02T15:47:11.315 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_create": 10, 2024-02-02T15:47:11.315 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_read": 938, 2024-02-02T15:47:11.316 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_write": 27855, 2024-02-02T15:47:11.316 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_writefull": 528, 2024-02-02T15:47:11.316 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_writesame": 0, 2024-02-02T15:47:11.316 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_append": 0, 2024-02-02T15:47:11.316 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_zero": 0, 2024-02-02T15:47:11.316 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_truncate": 2, 2024-02-02T15:47:11.316 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_delete": 41267, 2024-02-02T15:47:11.316 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_mapext": 0, 2024-02-02T15:47:11.316 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_sparse_read": 0, 2024-02-02T15:47:11.317 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_clonerange": 0, 2024-02-02T15:47:11.317 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_getxattr": 1630, 2024-02-02T15:47:11.317 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_setxattr": 20, 2024-02-02T15:47:11.317 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_cmpxattr": 0, 2024-02-02T15:47:11.317 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_rmxattr": 0, 2024-02-02T15:47:11.317 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_resetxattrs": 0, 2024-02-02T15:47:11.317 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_call": 0, 2024-02-02T15:47:11.317 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_watch": 0, 2024-02-02T15:47:11.318 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_notify": 0, 2024-02-02T15:47:11.318 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_src_cmpxattr": 0, 2024-02-02T15:47:11.318 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_pgls": 0, 2024-02-02T15:47:11.318 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_pgls_filter": 0, 2024-02-02T15:47:11.318 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_other": 0, 2024-02-02T15:47:11.318 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_active": 0, 2024-02-02T15:47:11.318 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_send": 0, 2024-02-02T15:47:11.318 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_resend": 0, 2024-02-02T15:47:11.318 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_ping": 0, 2024-02-02T15:47:11.319 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_active": 0, 2024-02-02T15:47:11.319 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_send": 0, 2024-02-02T15:47:11.319 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_resend": 0, 2024-02-02T15:47:11.319 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_active": 0, 2024-02-02T15:47:11.319 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_send": 0, 2024-02-02T15:47:11.319 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_resend": 0, 2024-02-02T15:47:11.319 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_active": 0, 2024-02-02T15:47:11.319 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_send": 0, 2024-02-02T15:47:11.319 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_resend": 0, 2024-02-02T15:47:11.320 INFO:teuthology.orchestra.run.smithi142.stdout: "command_active": 0, 2024-02-02T15:47:11.320 INFO:teuthology.orchestra.run.smithi142.stdout: "command_send": 0, 2024-02-02T15:47:11.320 INFO:teuthology.orchestra.run.smithi142.stdout: "command_resend": 0, 2024-02-02T15:47:11.320 INFO:teuthology.orchestra.run.smithi142.stdout: "map_epoch": 116, 2024-02-02T15:47:11.320 INFO:teuthology.orchestra.run.smithi142.stdout: "map_full": 0, 2024-02-02T15:47:11.320 INFO:teuthology.orchestra.run.smithi142.stdout: "map_inc": 32, 2024-02-02T15:47:11.320 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_sessions": 12, 2024-02-02T15:47:11.320 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_session_open": 12, 2024-02-02T15:47:11.321 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_session_close": 0, 2024-02-02T15:47:11.321 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_laggy": 0, 2024-02-02T15:47:11.321 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_wr": 476, 2024-02-02T15:47:11.321 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_rd": 16, 2024-02-02T15:47:11.321 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_del": 215 2024-02-02T15:47:11.321 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:11.321 INFO:teuthology.orchestra.run.smithi142.stdout: "oft": { 2024-02-02T15:47:11.321 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_objs": 1, 2024-02-02T15:47:11.321 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_kv_pairs": 0, 2024-02-02T15:47:11.322 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_updates": 0, 2024-02-02T15:47:11.322 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_removes": 0 2024-02-02T15:47:11.322 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:11.322 INFO:teuthology.orchestra.run.smithi142.stdout: "osd-slow-ops": { 2024-02-02T15:47:11.322 INFO:teuthology.orchestra.run.smithi142.stdout: "slow_ops_count": 158614 2024-02-02T15:47:11.322 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:11.322 INFO:teuthology.orchestra.run.smithi142.stdout: "purge_queue": { 2024-02-02T15:47:11.322 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_ops": 0, 2024-02-02T15:47:11.322 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_ops_high_water": 10, 2024-02-02T15:47:11.323 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing": 0, 2024-02-02T15:47:11.323 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_high_water": 5, 2024-02-02T15:47:11.323 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executed": 20580, 2024-02-02T15:47:11.323 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_item_in_journal": 0 2024-02-02T15:47:11.323 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:11.323 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-02T15:47:11.323 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-02-02T15:47:11.323 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 104857600, 2024-02-02T15:47:11.323 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-02-02T15:47:11.324 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 700059, 2024-02-02T15:47:11.324 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 151980191, 2024-02-02T15:47:11.324 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-02-02T15:47:11.324 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 700059, 2024-02-02T15:47:11.324 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-02-02T15:47:11.324 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-02-02T15:47:11.324 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 700059, 2024-02-02T15:47:11.324 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 151980191, 2024-02-02T15:47:11.324 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-02-02T15:47:11.325 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:11.325 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:11.325 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:11.325 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-02-02T15:47:11.325 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:11.325 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-objecter_bytes": { 2024-02-02T15:47:11.325 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-02-02T15:47:11.325 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 104857600, 2024-02-02T15:47:11.325 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-02-02T15:47:11.326 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-02-02T15:47:11.326 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-02-02T15:47:11.326 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-02-02T15:47:11.326 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-02-02T15:47:11.326 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 72489, 2024-02-02T15:47:11.326 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 374562322, 2024-02-02T15:47:11.326 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 72489, 2024-02-02T15:47:11.326 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 374562322, 2024-02-02T15:47:11.326 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-02-02T15:47:11.327 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:11.327 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:11.327 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:11.327 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-02-02T15:47:11.327 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:11.327 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-objecter_ops": { 2024-02-02T15:47:11.327 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-02-02T15:47:11.327 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 1024, 2024-02-02T15:47:11.327 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-02-02T15:47:11.328 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-02-02T15:47:11.328 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-02-02T15:47:11.328 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-02-02T15:47:11.328 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-02-02T15:47:11.328 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 72489, 2024-02-02T15:47:11.328 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 72489, 2024-02-02T15:47:11.328 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 72489, 2024-02-02T15:47:11.328 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 72489, 2024-02-02T15:47:11.329 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-02-02T15:47:11.329 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:11.329 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:11.329 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:11.329 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-02-02T15:47:11.329 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:11.329 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-write_buf_throttle": { 2024-02-02T15:47:11.329 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-02-02T15:47:11.329 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 3758096384, 2024-02-02T15:47:11.330 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-02-02T15:47:11.330 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 20580, 2024-02-02T15:47:11.330 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 2244884, 2024-02-02T15:47:11.330 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-02-02T15:47:11.330 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 20580, 2024-02-02T15:47:11.330 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-02-02T15:47:11.330 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-02-02T15:47:11.330 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 938, 2024-02-02T15:47:11.331 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 2244884, 2024-02-02T15:47:11.331 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-02-02T15:47:11.331 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:11.331 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:11.331 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:11.331 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-02-02T15:47:11.331 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:11.331 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-write_buf_throttle-0x55f409620000": { 2024-02-02T15:47:11.331 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-02-02T15:47:11.332 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 3758096384, 2024-02-02T15:47:11.332 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-02-02T15:47:11.332 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 98346, 2024-02-02T15:47:11.332 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 368565803, 2024-02-02T15:47:11.332 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-02-02T15:47:11.332 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 98346, 2024-02-02T15:47:11.332 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-02-02T15:47:11.332 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-02-02T15:47:11.332 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 26917, 2024-02-02T15:47:11.333 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 368565803, 2024-02-02T15:47:11.333 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-02-02T15:47:11.333 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:11.333 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:11.333 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:11.333 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-02-02T15:47:11.333 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-02-02T15:47:11.333 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-02-02T15:47:11.617 INFO:tasks.check_counter:Daemon mds.i mds.exported=9 2024-02-02T15:47:11.617 INFO:tasks.check_counter:Daemon mds.i mds.imported=14 2024-02-02T15:47:11.617 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=740 2024-02-02T15:47:11.617 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=1710 2024-02-02T15:47:11.617 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-02-02T15:47:11.617 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-02-02T15:47:11.617 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-02-02T15:47:11.618 DEBUG:tasks.check_counter:Getting stats from l 2024-02-02T15:47:11.618 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:0fb327883c124fd83155c94db0804f823b805a12 shell --fsid 371a409c-c1de-11ee-95b6-87774f69a715 -- ceph daemon mds.l perf dump 2024-02-02T15:47:11.670 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:47:11 smithi142 ceph-mon[45728]: pgmap v635: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 151 KiB/s wr, 34 op/s 2024-02-02T15:47:11.847 INFO:teuthology.orchestra.run.smithi142.stderr:Inferring config /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/mon.c/config 2024-02-02T15:47:11.864 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:47:11 smithi071 ceph-mon[44707]: pgmap v635: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 151 KiB/s wr, 34 op/s 2024-02-02T15:47:11.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:47:11 smithi001 ceph-mon[35470]: pgmap v635: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 151 KiB/s wr, 34 op/s 2024-02-02T15:47:12.821 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-02-02T15:47:12.821 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-0": { 2024-02-02T15:47:12.821 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 1200, 2024-02-02T15:47:12.821 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 1202, 2024-02-02T15:47:12.821 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 151303659, 2024-02-02T15:47:12.822 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 373264, 2024-02-02T15:47:12.822 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 9, 2024-02-02T15:47:12.822 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 4, 2024-02-02T15:47:12.822 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.189468687, 2024-02-02T15:47:12.822 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.023885351, 2024-02-02T15:47:12.822 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.123722993, 2024-02-02T15:47:12.822 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.032077236, 2024-02-02T15:47:12.822 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-02-02T15:47:12.823 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 1200, 2024-02-02T15:47:12.823 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.143565879, 2024-02-02T15:47:12.823 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000119638 2024-02-02T15:47:12.823 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:12.823 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-02-02T15:47:12.823 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:12.823 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:12.823 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:12.824 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:12.824 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 0, 2024-02-02T15:47:12.824 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 0 2024-02-02T15:47:12.824 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:12.824 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-1": { 2024-02-02T15:47:12.824 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 1727, 2024-02-02T15:47:12.824 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 2002, 2024-02-02T15:47:12.824 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 161202006, 2024-02-02T15:47:12.825 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 1164036, 2024-02-02T15:47:12.825 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 9, 2024-02-02T15:47:12.825 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 7, 2024-02-02T15:47:12.825 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.212114236, 2024-02-02T15:47:12.825 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.046435756, 2024-02-02T15:47:12.825 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.107242466, 2024-02-02T15:47:12.825 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.045270003, 2024-02-02T15:47:12.825 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-02-02T15:47:12.825 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 2000, 2024-02-02T15:47:12.826 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.129614138, 2024-02-02T15:47:12.826 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000064807 2024-02-02T15:47:12.826 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:12.826 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-02-02T15:47:12.826 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:12.826 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:12.826 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:12.826 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:12.827 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-02-02T15:47:12.827 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 628800 2024-02-02T15:47:12.827 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:12.827 INFO:teuthology.orchestra.run.smithi142.stdout: "AsyncMessenger::Worker-2": { 2024-02-02T15:47:12.827 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_messages": 2648, 2024-02-02T15:47:12.827 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages": 2579, 2024-02-02T15:47:12.827 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_bytes": 103570260, 2024-02-02T15:47:12.828 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_bytes": 892663, 2024-02-02T15:47:12.828 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_created_connections": 5, 2024-02-02T15:47:12.828 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_active_connections": 4, 2024-02-02T15:47:12.828 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_total_time": 0.301612288, 2024-02-02T15:47:12.828 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_send_time": 0.068671876, 2024-02-02T15:47:12.828 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_recv_time": 0.156816553, 2024-02-02T15:47:12.828 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_running_fast_dispatch_time": 0.071620982, 2024-02-02T15:47:12.828 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_messages_queue_lat": { 2024-02-02T15:47:12.829 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 2578, 2024-02-02T15:47:12.829 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.201703531, 2024-02-02T15:47:12.829 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000078240 2024-02-02T15:47:12.829 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:12.829 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_handle_ack_lat": { 2024-02-02T15:47:12.829 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:12.829 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:12.829 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:12.830 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:12.830 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_recv_encrypted_bytes": 2247184, 2024-02-02T15:47:12.830 INFO:teuthology.orchestra.run.smithi142.stdout: "msgr_send_encrypted_bytes": 215616 2024-02-02T15:47:12.830 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:12.830 INFO:teuthology.orchestra.run.smithi142.stdout: "cct": { 2024-02-02T15:47:12.830 INFO:teuthology.orchestra.run.smithi142.stdout: "total_workers": 1, 2024-02-02T15:47:12.830 INFO:teuthology.orchestra.run.smithi142.stdout: "unhealthy_workers": 0 2024-02-02T15:47:12.830 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:12.830 INFO:teuthology.orchestra.run.smithi142.stdout: "finisher-MDSRank": { 2024-02-02T15:47:12.831 INFO:teuthology.orchestra.run.smithi142.stdout: "queue_len": 0, 2024-02-02T15:47:12.831 INFO:teuthology.orchestra.run.smithi142.stdout: "complete_latency": { 2024-02-02T15:47:12.831 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 8462, 2024-02-02T15:47:12.831 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.195742958, 2024-02-02T15:47:12.831 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000023131 2024-02-02T15:47:12.831 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-02-02T15:47:12.831 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:12.831 INFO:teuthology.orchestra.run.smithi142.stdout: "finisher-PurgeQueue": { 2024-02-02T15:47:12.832 INFO:teuthology.orchestra.run.smithi142.stdout: "queue_len": 0, 2024-02-02T15:47:12.832 INFO:teuthology.orchestra.run.smithi142.stdout: "complete_latency": { 2024-02-02T15:47:12.832 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:12.832 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:12.832 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:12.832 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-02-02T15:47:12.832 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:12.832 INFO:teuthology.orchestra.run.smithi142.stdout: "mds": { 2024-02-02T15:47:12.832 INFO:teuthology.orchestra.run.smithi142.stdout: "request": 0, 2024-02-02T15:47:12.833 INFO:teuthology.orchestra.run.smithi142.stdout: "reply": 0, 2024-02-02T15:47:12.833 INFO:teuthology.orchestra.run.smithi142.stdout: "reply_latency": { 2024-02-02T15:47:12.833 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:12.833 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:12.833 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:12.833 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:12.833 INFO:teuthology.orchestra.run.smithi142.stdout: "slow_reply": 0, 2024-02-02T15:47:12.833 INFO:teuthology.orchestra.run.smithi142.stdout: "forward": 0, 2024-02-02T15:47:12.834 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_fetch_complete": 0, 2024-02-02T15:47:12.834 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_fetch_keys": 0, 2024-02-02T15:47:12.834 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_commit": 0, 2024-02-02T15:47:12.834 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_split": 0, 2024-02-02T15:47:12.834 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_merge": 0, 2024-02-02T15:47:12.834 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes": 18905, 2024-02-02T15:47:12.834 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_top": 774, 2024-02-02T15:47:12.834 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_bottom": 18131, 2024-02-02T15:47:12.834 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_pin_tail": 0, 2024-02-02T15:47:12.835 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_pinned": 17800, 2024-02-02T15:47:12.835 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_expired": 0, 2024-02-02T15:47:12.835 INFO:teuthology.orchestra.run.smithi142.stdout: "inodes_with_caps": 0, 2024-02-02T15:47:12.835 INFO:teuthology.orchestra.run.smithi142.stdout: "caps": 0, 2024-02-02T15:47:12.835 INFO:teuthology.orchestra.run.smithi142.stdout: "subtrees": 5, 2024-02-02T15:47:12.835 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse": 0, 2024-02-02T15:47:12.835 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_hit": 0, 2024-02-02T15:47:12.835 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_forward": 0, 2024-02-02T15:47:12.836 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_discover": 0, 2024-02-02T15:47:12.836 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_dir_fetch": 0, 2024-02-02T15:47:12.836 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_remote_ino": 0, 2024-02-02T15:47:12.836 INFO:teuthology.orchestra.run.smithi142.stdout: "traverse_lock": 0, 2024-02-02T15:47:12.836 INFO:teuthology.orchestra.run.smithi142.stdout: "load_cent": 0, 2024-02-02T15:47:12.836 INFO:teuthology.orchestra.run.smithi142.stdout: "q": 0, 2024-02-02T15:47:12.836 INFO:teuthology.orchestra.run.smithi142.stdout: "exported": 0, 2024-02-02T15:47:12.836 INFO:teuthology.orchestra.run.smithi142.stdout: "exported_inodes": 0, 2024-02-02T15:47:12.837 INFO:teuthology.orchestra.run.smithi142.stdout: "imported": 0, 2024-02-02T15:47:12.837 INFO:teuthology.orchestra.run.smithi142.stdout: "imported_inodes": 0, 2024-02-02T15:47:12.837 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_dir_fetch": 0, 2024-02-02T15:47:12.837 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_backtrace_fetch": 0, 2024-02-02T15:47:12.837 INFO:teuthology.orchestra.run.smithi142.stdout: "openino_peer_discover": 0, 2024-02-02T15:47:12.837 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rfiles": 0, 2024-02-02T15:47:12.837 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rbytes": 0, 2024-02-02T15:47:12.837 INFO:teuthology.orchestra.run.smithi142.stdout: "root_rsnaps": 0, 2024-02-02T15:47:12.838 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_backtrace_fetch": 0, 2024-02-02T15:47:12.838 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_set_tag": 0, 2024-02-02T15:47:12.838 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_backtrace_repaired": 0, 2024-02-02T15:47:12.838 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_inotable_repaired": 0, 2024-02-02T15:47:12.838 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dir_inodes": 0, 2024-02-02T15:47:12.838 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dir_base_inodes": 0, 2024-02-02T15:47:12.838 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_dirfrag_rstats": 0, 2024-02-02T15:47:12.838 INFO:teuthology.orchestra.run.smithi142.stdout: "scrub_file_inodes": 0, 2024-02-02T15:47:12.838 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_inode_file_caps": 0, 2024-02-02T15:47:12.838 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_revoke": 0, 2024-02-02T15:47:12.839 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_grant": 0, 2024-02-02T15:47:12.839 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_trunc": 0, 2024-02-02T15:47:12.839 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-02-02T15:47:12.839 INFO:teuthology.orchestra.run.smithi142.stdout: "ceph_cap_op_flush_ack": 0, 2024-02-02T15:47:12.839 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_caps": 0, 2024-02-02T15:47:12.839 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_caps_dirty": 0, 2024-02-02T15:47:12.839 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_cap_release": 0, 2024-02-02T15:47:12.839 INFO:teuthology.orchestra.run.smithi142.stdout: "process_request_cap_release": 0 2024-02-02T15:47:12.840 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:12.840 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_cache": { 2024-02-02T15:47:12.840 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_update": 0, 2024-02-02T15:47:12.840 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_update_receipt": 0, 2024-02-02T15:47:12.840 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_try_discover": 0, 2024-02-02T15:47:12.840 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_send_discover": 0, 2024-02-02T15:47:12.840 INFO:teuthology.orchestra.run.smithi142.stdout: "dir_handle_discover": 0, 2024-02-02T15:47:12.840 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays": 0, 2024-02-02T15:47:12.841 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays_delayed": 0, 2024-02-02T15:47:12.841 INFO:teuthology.orchestra.run.smithi142.stdout: "num_strays_enqueuing": 0, 2024-02-02T15:47:12.841 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_created": 0, 2024-02-02T15:47:12.841 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_enqueued": 0, 2024-02-02T15:47:12.841 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_reintegrated": 0, 2024-02-02T15:47:12.841 INFO:teuthology.orchestra.run.smithi142.stdout: "strays_migrated": 0, 2024-02-02T15:47:12.841 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_processing": 0, 2024-02-02T15:47:12.841 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_enqueued": 0, 2024-02-02T15:47:12.841 INFO:teuthology.orchestra.run.smithi142.stdout: "num_recovering_prioritized": 0, 2024-02-02T15:47:12.842 INFO:teuthology.orchestra.run.smithi142.stdout: "recovery_started": 0, 2024-02-02T15:47:12.842 INFO:teuthology.orchestra.run.smithi142.stdout: "recovery_completed": 0, 2024-02-02T15:47:12.842 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_enqueue_scrub": 0, 2024-02-02T15:47:12.842 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_exportdir": 0, 2024-02-02T15:47:12.842 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_flush": 0, 2024-02-02T15:47:12.842 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_fragmentdir": 0, 2024-02-02T15:47:12.842 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_fragstats": 0, 2024-02-02T15:47:12.842 INFO:teuthology.orchestra.run.smithi142.stdout: "ireq_inodestats": 0 2024-02-02T15:47:12.843 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:12.843 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_log": { 2024-02-02T15:47:12.843 INFO:teuthology.orchestra.run.smithi142.stdout: "evadd": 0, 2024-02-02T15:47:12.843 INFO:teuthology.orchestra.run.smithi142.stdout: "evex": 0, 2024-02-02T15:47:12.843 INFO:teuthology.orchestra.run.smithi142.stdout: "evtrm": 0, 2024-02-02T15:47:12.843 INFO:teuthology.orchestra.run.smithi142.stdout: "ev": 100747, 2024-02-02T15:47:12.843 INFO:teuthology.orchestra.run.smithi142.stdout: "evexg": 0, 2024-02-02T15:47:12.843 INFO:teuthology.orchestra.run.smithi142.stdout: "evexd": 0, 2024-02-02T15:47:12.844 INFO:teuthology.orchestra.run.smithi142.stdout: "segadd": 0, 2024-02-02T15:47:12.844 INFO:teuthology.orchestra.run.smithi142.stdout: "segex": 0, 2024-02-02T15:47:12.844 INFO:teuthology.orchestra.run.smithi142.stdout: "segtrm": 0, 2024-02-02T15:47:12.844 INFO:teuthology.orchestra.run.smithi142.stdout: "seg": 129, 2024-02-02T15:47:12.844 INFO:teuthology.orchestra.run.smithi142.stdout: "segexg": 0, 2024-02-02T15:47:12.844 INFO:teuthology.orchestra.run.smithi142.stdout: "segexd": 0, 2024-02-02T15:47:12.844 INFO:teuthology.orchestra.run.smithi142.stdout: "expos": 15558850, 2024-02-02T15:47:12.844 INFO:teuthology.orchestra.run.smithi142.stdout: "wrpos": 416614727, 2024-02-02T15:47:12.845 INFO:teuthology.orchestra.run.smithi142.stdout: "rdpos": 416607953, 2024-02-02T15:47:12.845 INFO:teuthology.orchestra.run.smithi142.stdout: "jlat": { 2024-02-02T15:47:12.845 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:12.845 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:12.845 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:12.845 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:12.845 INFO:teuthology.orchestra.run.smithi142.stdout: "replayed": 100747 2024-02-02T15:47:12.845 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:12.846 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_mem": { 2024-02-02T15:47:12.846 INFO:teuthology.orchestra.run.smithi142.stdout: "ino": 676, 2024-02-02T15:47:12.846 INFO:teuthology.orchestra.run.smithi142.stdout: "ino+": 19469, 2024-02-02T15:47:12.846 INFO:teuthology.orchestra.run.smithi142.stdout: "ino-": 18793, 2024-02-02T15:47:12.846 INFO:teuthology.orchestra.run.smithi142.stdout: "dir": 55, 2024-02-02T15:47:12.846 INFO:teuthology.orchestra.run.smithi142.stdout: "dir+": 77, 2024-02-02T15:47:12.846 INFO:teuthology.orchestra.run.smithi142.stdout: "dir-": 22, 2024-02-02T15:47:12.846 INFO:teuthology.orchestra.run.smithi142.stdout: "dn": 18905, 2024-02-02T15:47:12.846 INFO:teuthology.orchestra.run.smithi142.stdout: "dn+": 19668, 2024-02-02T15:47:12.847 INFO:teuthology.orchestra.run.smithi142.stdout: "dn-": 763, 2024-02-02T15:47:12.847 INFO:teuthology.orchestra.run.smithi142.stdout: "cap": 0, 2024-02-02T15:47:12.847 INFO:teuthology.orchestra.run.smithi142.stdout: "cap+": 0, 2024-02-02T15:47:12.847 INFO:teuthology.orchestra.run.smithi142.stdout: "cap-": 0, 2024-02-02T15:47:12.847 INFO:teuthology.orchestra.run.smithi142.stdout: "rss": 71548, 2024-02-02T15:47:12.847 INFO:teuthology.orchestra.run.smithi142.stdout: "heap": 207156 2024-02-02T15:47:12.847 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:12.847 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_server": { 2024-02-02T15:47:12.848 INFO:teuthology.orchestra.run.smithi142.stdout: "dispatch_client_request": 0, 2024-02-02T15:47:12.848 INFO:teuthology.orchestra.run.smithi142.stdout: "dispatch_server_request": 0, 2024-02-02T15:47:12.848 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_request": 0, 2024-02-02T15:47:12.848 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_client_session": 0, 2024-02-02T15:47:12.848 INFO:teuthology.orchestra.run.smithi142.stdout: "handle_peer_request": 0, 2024-02-02T15:47:12.848 INFO:teuthology.orchestra.run.smithi142.stdout: "req_create_latency": { 2024-02-02T15:47:12.848 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:12.848 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:12.849 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:12.849 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:12.849 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getattr_latency": { 2024-02-02T15:47:12.849 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:12.849 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:12.849 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:12.849 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:12.849 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getfilelock_latency": { 2024-02-02T15:47:12.849 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:12.850 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:12.850 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:12.850 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:12.850 INFO:teuthology.orchestra.run.smithi142.stdout: "req_link_latency": { 2024-02-02T15:47:12.850 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:12.850 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:12.850 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:12.850 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:12.850 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookup_latency": { 2024-02-02T15:47:12.851 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:12.851 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:12.851 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:12.851 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:12.851 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookuphash_latency": { 2024-02-02T15:47:12.851 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:12.851 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:12.851 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:12.851 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:12.852 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupino_latency": { 2024-02-02T15:47:12.852 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:12.852 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:12.852 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:12.852 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:12.852 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupname_latency": { 2024-02-02T15:47:12.852 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:12.852 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:12.853 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:12.853 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:12.853 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupparent_latency": { 2024-02-02T15:47:12.853 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:12.853 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:12.853 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:12.853 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:12.853 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lookupsnap_latency": { 2024-02-02T15:47:12.853 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:12.853 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:12.854 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:12.854 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:12.854 INFO:teuthology.orchestra.run.smithi142.stdout: "req_lssnap_latency": { 2024-02-02T15:47:12.854 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:12.854 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:12.854 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:12.854 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:12.854 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mkdir_latency": { 2024-02-02T15:47:12.854 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:12.854 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:12.855 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:12.855 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:12.855 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mknod_latency": { 2024-02-02T15:47:12.855 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:12.855 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:12.855 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:12.855 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:12.855 INFO:teuthology.orchestra.run.smithi142.stdout: "req_mksnap_latency": { 2024-02-02T15:47:12.855 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:12.856 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:12.856 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:12.856 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:12.856 INFO:teuthology.orchestra.run.smithi142.stdout: "req_open_latency": { 2024-02-02T15:47:12.856 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:12.856 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:12.856 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:12.856 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:12.856 INFO:teuthology.orchestra.run.smithi142.stdout: "req_readdir_latency": { 2024-02-02T15:47:12.857 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:12.857 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:12.857 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:12.857 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:12.857 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rename_latency": { 2024-02-02T15:47:12.857 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:12.857 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:12.858 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:12.858 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:12.858 INFO:teuthology.orchestra.run.smithi142.stdout: "req_renamesnap_latency": { 2024-02-02T15:47:12.858 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:12.858 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:12.858 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:12.858 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:12.858 INFO:teuthology.orchestra.run.smithi142.stdout: "req_snapdiff_latency": { 2024-02-02T15:47:12.858 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:12.858 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:12.859 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:12.859 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:12.859 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmdir_latency": { 2024-02-02T15:47:12.859 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:12.859 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:12.859 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:12.859 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:12.859 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmsnap_latency": { 2024-02-02T15:47:12.859 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:12.860 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:12.860 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:12.860 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:12.860 INFO:teuthology.orchestra.run.smithi142.stdout: "req_rmxattr_latency": { 2024-02-02T15:47:12.860 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:12.860 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:12.860 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:12.860 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:12.860 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setattr_latency": { 2024-02-02T15:47:12.861 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:12.861 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:12.861 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:12.861 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:12.861 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setdirlayout_latency": { 2024-02-02T15:47:12.861 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:12.861 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:12.861 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:12.861 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:12.861 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setfilelock_latency": { 2024-02-02T15:47:12.862 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:12.862 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:12.862 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:12.862 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:12.862 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setlayout_latency": { 2024-02-02T15:47:12.862 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:12.862 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:12.862 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:12.862 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:12.862 INFO:teuthology.orchestra.run.smithi142.stdout: "req_setxattr_latency": { 2024-02-02T15:47:12.863 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:12.863 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:12.863 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:12.863 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:12.863 INFO:teuthology.orchestra.run.smithi142.stdout: "req_symlink_latency": { 2024-02-02T15:47:12.863 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:12.863 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:12.863 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:12.863 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:12.863 INFO:teuthology.orchestra.run.smithi142.stdout: "req_unlink_latency": { 2024-02-02T15:47:12.864 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:12.864 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:12.864 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:12.864 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:12.864 INFO:teuthology.orchestra.run.smithi142.stdout: "cap_revoke_eviction": 0, 2024-02-02T15:47:12.864 INFO:teuthology.orchestra.run.smithi142.stdout: "cap_acquisition_throttle": 0, 2024-02-02T15:47:12.864 INFO:teuthology.orchestra.run.smithi142.stdout: "req_getvxattr_latency": { 2024-02-02T15:47:12.864 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:12.865 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:12.865 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:12.865 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-02-02T15:47:12.865 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:12.865 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_sessions": { 2024-02-02T15:47:12.865 INFO:teuthology.orchestra.run.smithi142.stdout: "session_count": 2, 2024-02-02T15:47:12.865 INFO:teuthology.orchestra.run.smithi142.stdout: "session_add": 5, 2024-02-02T15:47:12.865 INFO:teuthology.orchestra.run.smithi142.stdout: "session_remove": 3, 2024-02-02T15:47:12.865 INFO:teuthology.orchestra.run.smithi142.stdout: "sessions_open": 2, 2024-02-02T15:47:12.866 INFO:teuthology.orchestra.run.smithi142.stdout: "sessions_stale": 0, 2024-02-02T15:47:12.866 INFO:teuthology.orchestra.run.smithi142.stdout: "total_load": 0, 2024-02-02T15:47:12.866 INFO:teuthology.orchestra.run.smithi142.stdout: "average_load": 0, 2024-02-02T15:47:12.866 INFO:teuthology.orchestra.run.smithi142.stdout: "avg_session_uptime": 6372, 2024-02-02T15:47:12.866 INFO:teuthology.orchestra.run.smithi142.stdout: "mdthresh_evicted": 0 2024-02-02T15:47:12.866 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:12.866 INFO:teuthology.orchestra.run.smithi142.stdout: "mempool": { 2024-02-02T15:47:12.866 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_bytes": 0, 2024-02-02T15:47:12.866 INFO:teuthology.orchestra.run.smithi142.stdout: "bloom_filter_items": 0, 2024-02-02T15:47:12.867 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_bytes": 0, 2024-02-02T15:47:12.867 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_alloc_items": 0, 2024-02-02T15:47:12.867 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_bytes": 0, 2024-02-02T15:47:12.867 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_data_items": 0, 2024-02-02T15:47:12.867 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_bytes": 0, 2024-02-02T15:47:12.867 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_onode_items": 0, 2024-02-02T15:47:12.867 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_bytes": 0, 2024-02-02T15:47:12.868 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_meta_items": 0, 2024-02-02T15:47:12.868 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_bytes": 0, 2024-02-02T15:47:12.868 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_other_items": 0, 2024-02-02T15:47:12.868 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_bytes": 0, 2024-02-02T15:47:12.868 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_cache_buffer_items": 0, 2024-02-02T15:47:12.868 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_bytes": 0, 2024-02-02T15:47:12.868 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_extent_items": 0, 2024-02-02T15:47:12.868 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_bytes": 0, 2024-02-02T15:47:12.869 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_blob_items": 0, 2024-02-02T15:47:12.869 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_bytes": 0, 2024-02-02T15:47:12.869 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_shared_blob_items": 0, 2024-02-02T15:47:12.869 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_bytes": 0, 2024-02-02T15:47:12.869 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_inline_bl_items": 0, 2024-02-02T15:47:12.869 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_bytes": 0, 2024-02-02T15:47:12.869 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_fsck_items": 0, 2024-02-02T15:47:12.869 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_bytes": 0, 2024-02-02T15:47:12.869 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_txc_items": 0, 2024-02-02T15:47:12.869 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_bytes": 0, 2024-02-02T15:47:12.870 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_deferred_items": 0, 2024-02-02T15:47:12.870 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_bytes": 0, 2024-02-02T15:47:12.870 INFO:teuthology.orchestra.run.smithi142.stdout: "bluestore_writing_items": 0, 2024-02-02T15:47:12.870 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_bytes": 0, 2024-02-02T15:47:12.870 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_items": 0, 2024-02-02T15:47:12.870 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_bytes": 0, 2024-02-02T15:47:12.870 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_reader_items": 0, 2024-02-02T15:47:12.870 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_bytes": 0, 2024-02-02T15:47:12.871 INFO:teuthology.orchestra.run.smithi142.stdout: "bluefs_file_writer_items": 0, 2024-02-02T15:47:12.871 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_bytes": 270423, 2024-02-02T15:47:12.871 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_anon_items": 906, 2024-02-02T15:47:12.871 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_bytes": 0, 2024-02-02T15:47:12.871 INFO:teuthology.orchestra.run.smithi142.stdout: "buffer_meta_items": 0, 2024-02-02T15:47:12.871 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_bytes": 0, 2024-02-02T15:47:12.871 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_items": 0, 2024-02-02T15:47:12.871 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_bytes": 0, 2024-02-02T15:47:12.871 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_mapbl_items": 0, 2024-02-02T15:47:12.872 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_bytes": 0, 2024-02-02T15:47:12.872 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_pglog_items": 0, 2024-02-02T15:47:12.872 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_bytes": 9848, 2024-02-02T15:47:12.872 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_items": 190, 2024-02-02T15:47:12.872 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_bytes": 0, 2024-02-02T15:47:12.872 INFO:teuthology.orchestra.run.smithi142.stdout: "osdmap_mapping_items": 0, 2024-02-02T15:47:12.872 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_bytes": 0, 2024-02-02T15:47:12.872 INFO:teuthology.orchestra.run.smithi142.stdout: "pgmap_items": 0, 2024-02-02T15:47:12.873 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_bytes": 15050992, 2024-02-02T15:47:12.873 INFO:teuthology.orchestra.run.smithi142.stdout: "mds_co_items": 94956, 2024-02-02T15:47:12.873 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_bytes": 0, 2024-02-02T15:47:12.873 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_1_items": 0, 2024-02-02T15:47:12.873 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_bytes": 0, 2024-02-02T15:47:12.873 INFO:teuthology.orchestra.run.smithi142.stdout: "unittest_2_items": 0 2024-02-02T15:47:12.873 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:12.873 INFO:teuthology.orchestra.run.smithi142.stdout: "objecter": { 2024-02-02T15:47:12.874 INFO:teuthology.orchestra.run.smithi142.stdout: "op_active": 0, 2024-02-02T15:47:12.874 INFO:teuthology.orchestra.run.smithi142.stdout: "op_laggy": 0, 2024-02-02T15:47:12.874 INFO:teuthology.orchestra.run.smithi142.stdout: "op_send": 5098, 2024-02-02T15:47:12.874 INFO:teuthology.orchestra.run.smithi142.stdout: "op_send_bytes": 21, 2024-02-02T15:47:12.874 INFO:teuthology.orchestra.run.smithi142.stdout: "op_resend": 3, 2024-02-02T15:47:12.874 INFO:teuthology.orchestra.run.smithi142.stdout: "op_reply": 5098, 2024-02-02T15:47:12.874 INFO:teuthology.orchestra.run.smithi142.stdout: "op_latency": { 2024-02-02T15:47:12.874 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 5098, 2024-02-02T15:47:12.875 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 13.379756632, 2024-02-02T15:47:12.875 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.002624510 2024-02-02T15:47:12.875 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:12.875 INFO:teuthology.orchestra.run.smithi142.stdout: "op_inflight": 0, 2024-02-02T15:47:12.875 INFO:teuthology.orchestra.run.smithi142.stdout: "oplen_avg": { 2024-02-02T15:47:12.875 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 5098, 2024-02-02T15:47:12.875 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 5099 2024-02-02T15:47:12.875 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:12.875 INFO:teuthology.orchestra.run.smithi142.stdout: "op": 5098, 2024-02-02T15:47:12.876 INFO:teuthology.orchestra.run.smithi142.stdout: "op_r": 5098, 2024-02-02T15:47:12.876 INFO:teuthology.orchestra.run.smithi142.stdout: "op_w": 0, 2024-02-02T15:47:12.876 INFO:teuthology.orchestra.run.smithi142.stdout: "op_rmw": 0, 2024-02-02T15:47:12.876 INFO:teuthology.orchestra.run.smithi142.stdout: "op_pg": 0, 2024-02-02T15:47:12.876 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_stat": 2838, 2024-02-02T15:47:12.876 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_create": 0, 2024-02-02T15:47:12.876 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_read": 2258, 2024-02-02T15:47:12.876 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_write": 0, 2024-02-02T15:47:12.877 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_writefull": 0, 2024-02-02T15:47:12.877 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_writesame": 0, 2024-02-02T15:47:12.877 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_append": 0, 2024-02-02T15:47:12.877 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_zero": 0, 2024-02-02T15:47:12.877 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_truncate": 0, 2024-02-02T15:47:12.877 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_delete": 0, 2024-02-02T15:47:12.877 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_mapext": 0, 2024-02-02T15:47:12.877 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_sparse_read": 0, 2024-02-02T15:47:12.878 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_clonerange": 0, 2024-02-02T15:47:12.878 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_getxattr": 1, 2024-02-02T15:47:12.878 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_setxattr": 0, 2024-02-02T15:47:12.878 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_cmpxattr": 0, 2024-02-02T15:47:12.878 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_rmxattr": 0, 2024-02-02T15:47:12.879 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_resetxattrs": 0, 2024-02-02T15:47:12.879 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_call": 0, 2024-02-02T15:47:12.879 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_watch": 0, 2024-02-02T15:47:12.879 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_notify": 0, 2024-02-02T15:47:12.879 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_src_cmpxattr": 0, 2024-02-02T15:47:12.879 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_pgls": 0, 2024-02-02T15:47:12.879 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_pgls_filter": 0, 2024-02-02T15:47:12.879 INFO:teuthology.orchestra.run.smithi142.stdout: "osdop_other": 0, 2024-02-02T15:47:12.880 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_active": 0, 2024-02-02T15:47:12.880 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_send": 0, 2024-02-02T15:47:12.880 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_resend": 0, 2024-02-02T15:47:12.880 INFO:teuthology.orchestra.run.smithi142.stdout: "linger_ping": 0, 2024-02-02T15:47:12.880 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_active": 0, 2024-02-02T15:47:12.880 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_send": 0, 2024-02-02T15:47:12.880 INFO:teuthology.orchestra.run.smithi142.stdout: "poolop_resend": 0, 2024-02-02T15:47:12.880 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_active": 0, 2024-02-02T15:47:12.880 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_send": 0, 2024-02-02T15:47:12.881 INFO:teuthology.orchestra.run.smithi142.stdout: "poolstat_resend": 0, 2024-02-02T15:47:12.881 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_active": 0, 2024-02-02T15:47:12.881 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_send": 0, 2024-02-02T15:47:12.881 INFO:teuthology.orchestra.run.smithi142.stdout: "statfs_resend": 0, 2024-02-02T15:47:12.881 INFO:teuthology.orchestra.run.smithi142.stdout: "command_active": 0, 2024-02-02T15:47:12.881 INFO:teuthology.orchestra.run.smithi142.stdout: "command_send": 0, 2024-02-02T15:47:12.881 INFO:teuthology.orchestra.run.smithi142.stdout: "command_resend": 0, 2024-02-02T15:47:12.881 INFO:teuthology.orchestra.run.smithi142.stdout: "map_epoch": 116, 2024-02-02T15:47:12.881 INFO:teuthology.orchestra.run.smithi142.stdout: "map_full": 0, 2024-02-02T15:47:12.882 INFO:teuthology.orchestra.run.smithi142.stdout: "map_inc": 32, 2024-02-02T15:47:12.882 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_sessions": 11, 2024-02-02T15:47:12.882 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_session_open": 12, 2024-02-02T15:47:12.882 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_session_close": 1, 2024-02-02T15:47:12.882 INFO:teuthology.orchestra.run.smithi142.stdout: "osd_laggy": 0, 2024-02-02T15:47:12.882 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_wr": 0, 2024-02-02T15:47:12.882 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_rd": 2, 2024-02-02T15:47:12.882 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_del": 0 2024-02-02T15:47:12.882 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:12.883 INFO:teuthology.orchestra.run.smithi142.stdout: "oft": { 2024-02-02T15:47:12.883 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_objs": 0, 2024-02-02T15:47:12.883 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_kv_pairs": 0, 2024-02-02T15:47:12.883 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_updates": 0, 2024-02-02T15:47:12.883 INFO:teuthology.orchestra.run.smithi142.stdout: "omap_total_removes": 0 2024-02-02T15:47:12.883 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:12.883 INFO:teuthology.orchestra.run.smithi142.stdout: "osd-slow-ops": { 2024-02-02T15:47:12.883 INFO:teuthology.orchestra.run.smithi142.stdout: "slow_ops_count": 0 2024-02-02T15:47:12.883 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:12.884 INFO:teuthology.orchestra.run.smithi142.stdout: "purge_queue": { 2024-02-02T15:47:12.884 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_ops": 0, 2024-02-02T15:47:12.884 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_ops_high_water": 0, 2024-02-02T15:47:12.884 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing": 0, 2024-02-02T15:47:12.884 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executing_high_water": 0, 2024-02-02T15:47:12.884 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_executed": 0, 2024-02-02T15:47:12.884 INFO:teuthology.orchestra.run.smithi142.stdout: "pq_item_in_journal": 0 2024-02-02T15:47:12.884 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:12.885 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-02-02T15:47:12.885 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-02-02T15:47:12.885 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 104857600, 2024-02-02T15:47:12.885 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-02-02T15:47:12.885 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 5571, 2024-02-02T15:47:12.885 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 415547753, 2024-02-02T15:47:12.885 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-02-02T15:47:12.885 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 5571, 2024-02-02T15:47:12.885 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-02-02T15:47:12.885 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-02-02T15:47:12.886 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 5571, 2024-02-02T15:47:12.886 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 415547753, 2024-02-02T15:47:12.886 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-02-02T15:47:12.886 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:12.886 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:12.886 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:12.886 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-02-02T15:47:12.886 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:12.887 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-objecter_bytes": { 2024-02-02T15:47:12.887 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-02-02T15:47:12.887 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 104857600, 2024-02-02T15:47:12.887 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-02-02T15:47:12.887 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-02-02T15:47:12.887 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-02-02T15:47:12.887 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-02-02T15:47:12.887 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-02-02T15:47:12.887 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 5098, 2024-02-02T15:47:12.888 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 412422825, 2024-02-02T15:47:12.888 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 5098, 2024-02-02T15:47:12.888 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 412422825, 2024-02-02T15:47:12.888 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-02-02T15:47:12.888 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:12.888 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:12.888 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:12.888 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-02-02T15:47:12.889 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:12.889 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-objecter_ops": { 2024-02-02T15:47:12.889 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-02-02T15:47:12.889 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 1024, 2024-02-02T15:47:12.889 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-02-02T15:47:12.889 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-02-02T15:47:12.889 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-02-02T15:47:12.889 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-02-02T15:47:12.889 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-02-02T15:47:12.890 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 5098, 2024-02-02T15:47:12.890 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 5098, 2024-02-02T15:47:12.890 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 5098, 2024-02-02T15:47:12.890 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 5098, 2024-02-02T15:47:12.890 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-02-02T15:47:12.890 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:12.890 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:12.890 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:12.891 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-02-02T15:47:12.891 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:12.891 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-write_buf_throttle": { 2024-02-02T15:47:12.891 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-02-02T15:47:12.891 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 3758096384, 2024-02-02T15:47:12.891 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-02-02T15:47:12.891 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-02-02T15:47:12.891 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-02-02T15:47:12.892 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-02-02T15:47:12.892 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-02-02T15:47:12.892 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-02-02T15:47:12.892 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-02-02T15:47:12.892 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 0, 2024-02-02T15:47:12.892 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 0, 2024-02-02T15:47:12.892 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-02-02T15:47:12.892 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:12.892 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:12.893 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:12.893 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-02-02T15:47:12.893 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-02-02T15:47:12.893 INFO:teuthology.orchestra.run.smithi142.stdout: "throttle-write_buf_throttle-0x5647c13640a0": { 2024-02-02T15:47:12.893 INFO:teuthology.orchestra.run.smithi142.stdout: "val": 0, 2024-02-02T15:47:12.893 INFO:teuthology.orchestra.run.smithi142.stdout: "max": 3758096384, 2024-02-02T15:47:12.893 INFO:teuthology.orchestra.run.smithi142.stdout: "get_started": 0, 2024-02-02T15:47:12.893 INFO:teuthology.orchestra.run.smithi142.stdout: "get": 0, 2024-02-02T15:47:12.894 INFO:teuthology.orchestra.run.smithi142.stdout: "get_sum": 0, 2024-02-02T15:47:12.894 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_fail": 0, 2024-02-02T15:47:12.894 INFO:teuthology.orchestra.run.smithi142.stdout: "get_or_fail_success": 0, 2024-02-02T15:47:12.894 INFO:teuthology.orchestra.run.smithi142.stdout: "take": 0, 2024-02-02T15:47:12.894 INFO:teuthology.orchestra.run.smithi142.stdout: "take_sum": 0, 2024-02-02T15:47:12.894 INFO:teuthology.orchestra.run.smithi142.stdout: "put": 0, 2024-02-02T15:47:12.894 INFO:teuthology.orchestra.run.smithi142.stdout: "put_sum": 0, 2024-02-02T15:47:12.894 INFO:teuthology.orchestra.run.smithi142.stdout: "wait": { 2024-02-02T15:47:12.895 INFO:teuthology.orchestra.run.smithi142.stdout: "avgcount": 0, 2024-02-02T15:47:12.895 INFO:teuthology.orchestra.run.smithi142.stdout: "sum": 0.000000000, 2024-02-02T15:47:12.895 INFO:teuthology.orchestra.run.smithi142.stdout: "avgtime": 0.000000000 2024-02-02T15:47:12.895 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-02-02T15:47:12.895 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-02-02T15:47:12.895 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-02-02T15:47:12.920 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:47:12 smithi142 ceph-mon[45728]: pgmap v636: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 131 KiB/s wr, 33 op/s 2024-02-02T15:47:13.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:47:12 smithi071 ceph-mon[44707]: pgmap v636: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 131 KiB/s wr, 33 op/s 2024-02-02T15:47:13.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:47:12 smithi001 ceph-mon[35470]: pgmap v636: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 131 KiB/s wr, 33 op/s 2024-02-02T15:47:13.192 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-02-02T15:47:13.192 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-02-02T15:47:13.192 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=0 2024-02-02T15:47:13.192 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=0 2024-02-02T15:47:13.192 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2024-02-02T15:47:13.192 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-02-02T15:47:13.192 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-02-02T15:47:13.193 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-02-02T15:47:13.206 INFO:tasks.kclient:Unmounting kernel clients... 2024-02-02T15:47:13.207 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-02-02T15:47:13.207 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T15:47:13.257 DEBUG:teuthology.orchestra.run.smithi001:> set -ex 2024-02-02T15:47:13.257 DEBUG:teuthology.orchestra.run.smithi001:> dd if=/proc/self/mounts of=/dev/stdout 2024-02-02T15:47:13.301 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T15:47:13.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:13.548+0000 7f2e96200700 1 -- 172.21.15.1:0/3388216783 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e90106030 msgr2=0x7f2e9010dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:47:13.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:13.548+0000 7f2e96200700 1 --2- 172.21.15.1:0/3388216783 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e90106030 0x7f2e9010dbb0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f2e8800b0a0 tx=0x7f2e8800b3b0 comp rx=0 tx=0).stop 2024-02-02T15:47:13.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:13.548+0000 7f2e96200700 1 -- 172.21.15.1:0/3388216783 shutdown_connections 2024-02-02T15:47:13.548 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:13.548+0000 7f2e96200700 1 --2- 172.21.15.1:0/3388216783 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e90106030 0x7f2e9010dbb0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:13.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:13.548+0000 7f2e96200700 1 --2- 172.21.15.1:0/3388216783 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e90100f10 0x7f2e901012f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:13.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:13.548+0000 7f2e96200700 1 --2- 172.21.15.1:0/3388216783 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e901018c0 0x7f2e90105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:13.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:13.548+0000 7f2e96200700 1 -- 172.21.15.1:0/3388216783 >> 172.21.15.1:0/3388216783 conn(0x7f2e900fc790 msgr2=0x7f2e900febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:47:13.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:13.548+0000 7f2e96200700 1 -- 172.21.15.1:0/3388216783 shutdown_connections 2024-02-02T15:47:13.549 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:13.548+0000 7f2e96200700 1 -- 172.21.15.1:0/3388216783 wait complete. 2024-02-02T15:47:13.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:13.549+0000 7f2e96200700 1 Processor -- start 2024-02-02T15:47:13.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:13.549+0000 7f2e96200700 1 -- start start 2024-02-02T15:47:13.551 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:13.549+0000 7f2e96200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e90100f10 0x7f2e90073040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:47:13.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:13.549+0000 7f2e96200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e90106030 0x7f2e90073580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:47:13.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:13.549+0000 7f2e96200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e90073ac0 0x7f2e901a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:47:13.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:13.549+0000 7f2e96200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e901a5730 con 0x7f2e90073ac0 2024-02-02T15:47:13.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:13.549+0000 7f2e96200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e901a58a0 con 0x7f2e90106030 2024-02-02T15:47:13.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:13.549+0000 7f2e96200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e901a5a60 con 0x7f2e90100f10 2024-02-02T15:47:13.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:13.550+0000 7f2e8ea00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e90106030 0x7f2e90073580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:47:13.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:13.550+0000 7f2e8f400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e90100f10 0x7f2e90073040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:47:13.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:13.550+0000 7f2e8fe00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e90073ac0 0x7f2e901a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:47:13.552 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:13.550+0000 7f2e8ea00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e90106030 0x7f2e90073580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:56844/0 (socket says 172.21.15.1:56844) 2024-02-02T15:47:13.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:13.550+0000 7f2e8ea00700 1 -- 172.21.15.1:0/3642339990 learned_addr learned my addr 172.21.15.1:0/3642339990 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:47:13.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:13.550+0000 7f2e8ea00700 1 -- 172.21.15.1:0/3642339990 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e90100f10 msgr2=0x7f2e90073040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:47:13.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:13.550+0000 7f2e8ea00700 1 --2- 172.21.15.1:0/3642339990 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e90100f10 0x7f2e90073040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:13.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:13.550+0000 7f2e8ea00700 1 -- 172.21.15.1:0/3642339990 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e90073ac0 msgr2=0x7f2e901a50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:47:13.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:13.550+0000 7f2e8ea00700 1 --2- 172.21.15.1:0/3642339990 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e90073ac0 0x7f2e901a50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:13.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:13.550+0000 7f2e8ea00700 1 -- 172.21.15.1:0/3642339990 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e88009d00 con 0x7f2e90106030 2024-02-02T15:47:13.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:13.550+0000 7f2e8fe00700 1 --2- 172.21.15.1:0/3642339990 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e90073ac0 0x7f2e901a50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:47:13.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:13.550+0000 7f2e8ea00700 1 --2- 172.21.15.1:0/3642339990 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e90106030 0x7f2e90073580 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f2e7800e970 tx=0x7f2e7800ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:47:13.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:13.551+0000 7f2e87e00700 1 -- 172.21.15.1:0/3642339990 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e78009790 con 0x7f2e90106030 2024-02-02T15:47:13.553 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:13.551+0000 7f2e87e00700 1 -- 172.21.15.1:0/3642339990 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e7800c7e0 con 0x7f2e90106030 2024-02-02T15:47:13.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:13.551+0000 7f2e96200700 1 -- 172.21.15.1:0/3642339990 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e901a5d40 con 0x7f2e90106030 2024-02-02T15:47:13.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:13.551+0000 7f2e87e00700 1 -- 172.21.15.1:0/3642339990 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e780054e0 con 0x7f2e90106030 2024-02-02T15:47:13.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:13.551+0000 7f2e96200700 1 -- 172.21.15.1:0/3642339990 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e901a61f0 con 0x7f2e90106030 2024-02-02T15:47:13.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:13.552+0000 7f2e96200700 1 -- 172.21.15.1:0/3642339990 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e90067bc0 con 0x7f2e90106030 2024-02-02T15:47:13.554 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:13.554+0000 7f2e87e00700 1 -- 172.21.15.1:0/3642339990 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f2e7800c950 con 0x7f2e90106030 2024-02-02T15:47:13.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:13.555+0000 7f2e87e00700 1 --2- 172.21.15.1:0/3642339990 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f2e700a0690 0x7f2e700a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:47:13.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:13.555+0000 7f2e87e00700 1 -- 172.21.15.1:0/3642339990 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f2e78014070 con 0x7f2e90106030 2024-02-02T15:47:13.555 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:13.555+0000 7f2e8f400700 1 --2- 172.21.15.1:0/3642339990 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f2e700a0690 0x7f2e700a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:47:13.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:13.556+0000 7f2e8f400700 1 --2- 172.21.15.1:0/3642339990 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f2e700a0690 0x7f2e700a2b50 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f2e80009c60 tx=0x7f2e80009260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:47:13.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:13.557+0000 7f2e87e00700 1 -- 172.21.15.1:0/3642339990 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f2e78095980 con 0x7f2e90106030 2024-02-02T15:47:13.774 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:13.774+0000 7f2e96200700 1 -- 172.21.15.1:0/3642339990 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2e90074570 con 0x7f2e90106030 2024-02-02T15:47:13.775 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:13.775+0000 7f2e87e00700 1 -- 172.21.15.1:0/3642339990 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) v1 ==== 74+0+25150 (secure 0 0 0) 0x7f2e78098fa0 con 0x7f2e90106030 2024-02-02T15:47:13.776 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:47:13.776 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":116,"fsid":"371a409c-c1de-11ee-95b6-87774f69a715","created":"2024-02-02T15:18:26.746506+0000","modified":"2024-02-02T15:47:06.547340+0000","last_up_change":"2024-02-02T15:24:08.800951+0000","last_in_change":"2024-02-02T15:23:52.784280+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T15:21:22.471754+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-02T15:24:46.195779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-02T15:24:47.226403+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":35,"snap_epoch":116,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-02T15:24:50.300462+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":35,"snap_epoch":116,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"73e8a07f-3560-4c19-85ed-030d4d8f0f23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6803","nonce":2591906423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6805","nonce":2591906423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6809","nonce":2591906423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6807","nonce":2591906423}]},"public_addr":"172.21.15.1:6803/2591906423","cluster_addr":"172.21.15.1:6805/2591906423","heartbeat_back_addr":"172.21.15.1:6809/2591906423","heartbeat_front_addr":"172.21.15.1:6807/2591906423","state":["exists","up"]},{"osd":1,"uuid":"e53cffec-0ff9-4ed9-8dcf-ff47d04db2bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6801","nonce":747581719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6803","nonce":747581719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6807","nonce":747581719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6805","nonce":747581719}]},"public_addr":"172.21.15.71:6801/747581719","cluster_addr":"172.21.15.71:6803/747581719","heartbeat_back_addr":"172.21.15.71:6807/747581719","heartbeat_front_addr":"172.21.15.71:6805/747581719","state":["exists","up"]},{"osd":2,"uuid":"856c2347-294c-482b-8595-ad77f4792db4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6801","nonce":377511414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6803","nonce":377511414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6807","nonce":377511414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6805","nonce":377511414}]},"public_addr":"172.21.15.142:6801/377511414","cluster_addr":"172.21.15.142:6803/377511414","heartbeat_back_addr":"172.21.15.142:6807/377511414","heartbeat_front_addr":"172.21.15.142:6805/377511414","state":["exists","up"]},{"osd":3,"uuid":"4b946ad2-2da9-4e0b-9d61-d039d8669a42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6811","nonce":1086124477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6813","nonce":1086124477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6817","nonce":1086124477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6815","nonce":1086124477}]},"public_addr":"172.21.15.1:6811/1086124477","cluster_addr":"172.21.15.1:6813/1086124477","heartbeat_back_addr":"172.21.15.1:6817/1086124477","heartbeat_front_addr":"172.21.15.1:6815/1086124477","state":["exists","up"]},{"osd":4,"uuid":"43e36c43-47a0-4634-a45b-1da14a0e19d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6809","nonce":3095380305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6811","nonce":3095380305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6815","nonce":3095380305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6813","nonce":3095380305}]},"public_addr":"172.21.15.71:6809/3095380305","cluster_addr":"172.21.15.71:6811/3095380305","heartbeat_back_addr":"172.21.15.71:6815/3095380305","heartbeat_front_addr":"172.21.15.71:6813/3095380305","state":["exists","up"]},{"osd":5,"uuid":"eb48160e-ea7c-42f7-9477-be110bc8a34e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6809","nonce":4195149262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6811","nonce":4195149262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6815","nonce":4195149262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6813","nonce":4195149262}]},"public_addr":"172.21.15.142:6809/4195149262","cluster_addr":"172.21.15.142:6811/4195149262","heartbeat_back_addr":"172.21.15.142:6815/4195149262","heartbeat_front_addr":"172.21.15.142:6813/4195149262","state":["exists","up"]},{"osd":6,"uuid":"0ec4501a-5853-4ca6-b325-231f5b72335e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6819","nonce":1560153428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6821","nonce":1560153428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6825","nonce":1560153428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6823","nonce":1560153428}]},"public_addr":"172.21.15.1:6819/1560153428","cluster_addr":"172.21.15.1:6821/1560153428","heartbeat_back_addr":"172.21.15.1:6825/1560153428","heartbeat_front_addr":"172.21.15.1:6823/1560153428","state":["exists","up"]},{"osd":7,"uuid":"029debe8-9cdf-44f6-9aea-cabd1d185f93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6817","nonce":4080955538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6819","nonce":4080955538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6823","nonce":4080955538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6821","nonce":4080955538}]},"public_addr":"172.21.15.71:6817/4080955538","cluster_addr":"172.21.15.71:6819/4080955538","heartbeat_back_addr":"172.21.15.71:6823/4080955538","heartbeat_front_addr":"172.21.15.71:6821/4080955538","state":["exists","up"]},{"osd":8,"uuid":"7c90896c-9955-4c1b-aaca-7531dd5457f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6817","nonce":3942038209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6819","nonce":3942038209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6823","nonce":3942038209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6821","nonce":3942038209}]},"public_addr":"172.21.15.142:6817/3942038209","cluster_addr":"172.21.15.142:6819/3942038209","heartbeat_back_addr":"172.21.15.142:6823/3942038209","heartbeat_front_addr":"172.21.15.142:6821/3942038209","state":["exists","up"]},{"osd":9,"uuid":"7e165e68-31a8-4598-a577-de3e90ae5c5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6827","nonce":2716544850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6829","nonce":2716544850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6833","nonce":2716544850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6831","nonce":2716544850}]},"public_addr":"172.21.15.1:6827/2716544850","cluster_addr":"172.21.15.1:6829/2716544850","heartbeat_back_addr":"172.21.15.1:6833/2716544850","heartbeat_front_addr":"172.21.15.1:6831/2716544850","state":["exists","up"]},{"osd":10,"uuid":"48cd5fdb-0887-444a-aab8-a1de24282bb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6825","nonce":84691841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6827","nonce":84691841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6831","nonce":84691841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6829","nonce":84691841}]},"public_addr":"172.21.15.71:6825/84691841","cluster_addr":"172.21.15.71:6827/84691841","heartbeat_back_addr":"172.21.15.71:6831/84691841","heartbeat_front_addr":"172.21.15.71:6829/84691841","state":["exists","up"]},{"osd":11,"uuid":"f011dd27-11b0-4166-adfe-76756533163a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6825","nonce":2290179934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6827","nonce":2290179934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6831","nonce":2290179934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6829","nonce":2290179934}]},"public_addr":"172.21.15.142:6825/2290179934","cluster_addr":"172.21.15.142:6827/2290179934","heartbeat_back_addr":"172.21.15.142:6831/2290179934","heartbeat_front_addr":"172.21.15.142:6829/2290179934","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:20:44.060692+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:01.903817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:19.368258+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:40.028037+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:57.513361+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:22:14.805944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:22:35.188439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:22:53.014412+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:23:10.507420+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:23:31.849765+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:23:50.070612+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:24:07.388625+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":5,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/3997770901":"2024-02-03T15:27:00.115055+0000","172.21.15.1:0/3236204176":"2024-02-03T15:27:00.115055+0000","172.21.15.1:0/2256116842":"2024-02-03T15:27:00.115055+0000","172.21.15.1:0/4092532060":"2024-02-03T15:19:25.836171+0000","172.21.15.1:6801/3768989981":"2024-02-03T15:27:00.115055+0000","172.21.15.1:6800/3768989981":"2024-02-03T15:27:00.115055+0000","172.21.15.1:6801/395498296":"2024-02-03T15:19:25.836171+0000","172.21.15.1:6800/395498296":"2024-02-03T15:19:25.836171+0000","172.21.15.1:0/1574620681":"2024-02-03T15:27:00.115055+0000","172.21.15.1:0/1575373489":"2024-02-03T15:19:25.836171+0000","172.21.15.1:0/865983374":"2024-02-03T15:19:01.431596+0000","172.21.15.1:0/3165960227":"2024-02-03T15:19:01.431596+0000","172.21.15.1:0/2350830293":"2024-02-03T15:19:01.431596+0000","172.21.15.1:0/2959690250":"2024-02-03T15:19:25.836171+0000","172.21.15.1:6800/1162530732":"2024-02-03T15:19:01.431596+0000","172.21.15.1:6801/1162530732":"2024-02-03T15:19:01.431596+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":22,"length":1},{"begin":35,"length":1}]},{"pool":4,"snaps":[{"begin":22,"length":1},{"begin":35,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":22,"length":1},{"begin":35,"length":1}]},{"pool":4,"snaps":[{"begin":22,"length":1},{"begin":35,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T15:47:13.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:13.778+0000 7f2e96200700 1 -- 172.21.15.1:0/3642339990 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f2e700a0690 msgr2=0x7f2e700a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:47:13.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:13.778+0000 7f2e96200700 1 --2- 172.21.15.1:0/3642339990 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f2e700a0690 0x7f2e700a2b50 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f2e80009c60 tx=0x7f2e80009260 comp rx=0 tx=0).stop 2024-02-02T15:47:13.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:13.778+0000 7f2e96200700 1 -- 172.21.15.1:0/3642339990 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e90106030 msgr2=0x7f2e90073580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:47:13.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:13.778+0000 7f2e96200700 1 --2- 172.21.15.1:0/3642339990 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e90106030 0x7f2e90073580 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f2e7800e970 tx=0x7f2e7800ec80 comp rx=0 tx=0).stop 2024-02-02T15:47:13.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:13.778+0000 7f2e96200700 1 -- 172.21.15.1:0/3642339990 shutdown_connections 2024-02-02T15:47:13.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:13.778+0000 7f2e96200700 1 --2- 172.21.15.1:0/3642339990 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f2e700a0690 0x7f2e700a2b50 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:13.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:13.778+0000 7f2e96200700 1 --2- 172.21.15.1:0/3642339990 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2e90073ac0 0x7f2e901a50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:13.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:13.778+0000 7f2e96200700 1 --2- 172.21.15.1:0/3642339990 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2e90100f10 0x7f2e90073040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:13.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:13.778+0000 7f2e96200700 1 --2- 172.21.15.1:0/3642339990 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2e90106030 0x7f2e90073580 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:13.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:13.778+0000 7f2e96200700 1 -- 172.21.15.1:0/3642339990 >> 172.21.15.1:0/3642339990 conn(0x7f2e900fc790 msgr2=0x7f2e900fd220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:47:13.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:13.778+0000 7f2e96200700 1 -- 172.21.15.1:0/3642339990 shutdown_connections 2024-02-02T15:47:13.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:13.778+0000 7f2e96200700 1 -- 172.21.15.1:0/3642339990 wait complete. 2024-02-02T15:47:13.797 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-02T15:21:22.471754+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-02-02T15:24:46.195779+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-02-02T15:24:47.226403+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '116', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 35, 'snap_epoch': 116, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-02-02T15:24:50.300462+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '116', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 35, 'snap_epoch': 116, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-02-02T15:47:13.798 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-02-02T15:47:14.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:13.999+0000 7f57d3200700 1 -- 172.21.15.1:0/254428000 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57cc111d90 msgr2=0x7f57cc114180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:47:14.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:13.999+0000 7f57d3200700 1 --2- 172.21.15.1:0/254428000 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57cc111d90 0x7f57cc114180 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f57c4009f90 tx=0x7f57c400b330 comp rx=0 tx=0).stop 2024-02-02T15:47:14.000 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.000+0000 7f57d3200700 1 -- 172.21.15.1:0/254428000 shutdown_connections 2024-02-02T15:47:14.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.000+0000 7f57d3200700 1 --2- 172.21.15.1:0/254428000 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57cc111d90 0x7f57cc114180 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:14.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.000+0000 7f57d3200700 1 --2- 172.21.15.1:0/254428000 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57cc0730e0 0x7f57cc0734c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:14.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.000+0000 7f57d3200700 1 --2- 172.21.15.1:0/254428000 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f57cc073a00 0x7f57cc111850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:14.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.000+0000 7f57d3200700 1 -- 172.21.15.1:0/254428000 >> 172.21.15.1:0/254428000 conn(0x7f57cc0fc810 msgr2=0x7f57cc0fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:47:14.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.000+0000 7f57d3200700 1 -- 172.21.15.1:0/254428000 shutdown_connections 2024-02-02T15:47:14.001 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.000+0000 7f57d3200700 1 -- 172.21.15.1:0/254428000 wait complete. 2024-02-02T15:47:14.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.001+0000 7f57d3200700 1 Processor -- start 2024-02-02T15:47:14.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.001+0000 7f57d3200700 1 -- start start 2024-02-02T15:47:14.003 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.001+0000 7f57d3200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57cc0730e0 0x7f57cc19d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:47:14.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.001+0000 7f57d3200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f57cc073a00 0x7f57cc19dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:47:14.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.001+0000 7f57d3200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57cc111d90 0x7f57cc1a1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:47:14.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.001+0000 7f57d3200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57cc1a2580 con 0x7f57cc111d90 2024-02-02T15:47:14.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.001+0000 7f57d3200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57cc1a26f0 con 0x7f57cc0730e0 2024-02-02T15:47:14.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.001+0000 7f57d3200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57cc1a28b0 con 0x7f57cc073a00 2024-02-02T15:47:14.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.002+0000 7f57d1200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57cc111d90 0x7f57cc1a1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:47:14.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.002+0000 7f57cbe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57cc0730e0 0x7f57cc19d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:47:14.004 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.002+0000 7f57d1200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57cc111d90 0x7f57cc1a1f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43306/0 (socket says 172.21.15.1:43306) 2024-02-02T15:47:14.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.002+0000 7f57d1200700 1 -- 172.21.15.1:0/2919256403 learned_addr learned my addr 172.21.15.1:0/2919256403 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:47:14.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.002+0000 7f57cb400700 1 --2- 172.21.15.1:0/2919256403 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f57cc073a00 0x7f57cc19dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:47:14.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.002+0000 7f57d1200700 1 -- 172.21.15.1:0/2919256403 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f57cc073a00 msgr2=0x7f57cc19dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:47:14.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.002+0000 7f57d1200700 1 --2- 172.21.15.1:0/2919256403 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f57cc073a00 0x7f57cc19dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:14.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.002+0000 7f57d1200700 1 -- 172.21.15.1:0/2919256403 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57cc0730e0 msgr2=0x7f57cc19d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:47:14.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.002+0000 7f57d1200700 1 --2- 172.21.15.1:0/2919256403 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57cc0730e0 0x7f57cc19d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:14.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.002+0000 7f57d1200700 1 -- 172.21.15.1:0/2919256403 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57c4009c40 con 0x7f57cc111d90 2024-02-02T15:47:14.005 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.002+0000 7f57cb400700 1 --2- 172.21.15.1:0/2919256403 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f57cc073a00 0x7f57cc19dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-02-02T15:47:14.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.002+0000 7f57d1200700 1 --2- 172.21.15.1:0/2919256403 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57cc111d90 0x7f57cc1a1f40 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f57c40094b0 tx=0x7f57c40094e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:47:14.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.002+0000 7f57c8c00700 1 -- 172.21.15.1:0/2919256403 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c400e040 con 0x7f57cc111d90 2024-02-02T15:47:14.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.002+0000 7f57c8c00700 1 -- 172.21.15.1:0/2919256403 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f57c4004e70 con 0x7f57cc111d90 2024-02-02T15:47:14.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.002+0000 7f57d3200700 1 -- 172.21.15.1:0/2919256403 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57cc1a2b30 con 0x7f57cc111d90 2024-02-02T15:47:14.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.003+0000 7f57c8c00700 1 -- 172.21.15.1:0/2919256403 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c40052a0 con 0x7f57cc111d90 2024-02-02T15:47:14.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.003+0000 7f57d3200700 1 -- 172.21.15.1:0/2919256403 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57cc1aa3d0 con 0x7f57cc111d90 2024-02-02T15:47:14.006 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.004+0000 7f57d3200700 1 -- 172.21.15.1:0/2919256403 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57cc104270 con 0x7f57cc111d90 2024-02-02T15:47:14.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.005+0000 7f57c8c00700 1 -- 172.21.15.1:0/2919256403 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f57c401d470 con 0x7f57cc111d90 2024-02-02T15:47:14.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.007+0000 7f57c8c00700 1 --2- 172.21.15.1:0/2919256403 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f57ac0a0690 0x7f57ac0a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:47:14.007 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.007+0000 7f57c8c00700 1 -- 172.21.15.1:0/2919256403 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f57c40cd7c0 con 0x7f57cc111d90 2024-02-02T15:47:14.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.007+0000 7f57cbe00700 1 --2- 172.21.15.1:0/2919256403 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f57ac0a0690 0x7f57ac0a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:47:14.008 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.008+0000 7f57cbe00700 1 --2- 172.21.15.1:0/2919256403 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f57ac0a0690 0x7f57ac0a2b50 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f57bc009c60 tx=0x7f57bc009260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:47:14.011 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.011+0000 7f57c8c00700 1 -- 172.21.15.1:0/2919256403 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f57c4096c50 con 0x7f57cc111d90 2024-02-02T15:47:14.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:47:13 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/3642339990' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-02T15:47:14.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:47:13 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/3642339990' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-02T15:47:14.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:47:13 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/3642339990' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-02T15:47:14.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.221+0000 7f57d3200700 1 -- 172.21.15.1:0/2919256403 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f57cc103c50 con 0x7f57cc111d90 2024-02-02T15:47:14.221 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.221+0000 7f57c8c00700 1 -- 172.21.15.1:0/2919256403 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v116) v1 ==== 93+0+10 (secure 0 0 0) 0x7f57c401d720 con 0x7f57cc111d90 2024-02-02T15:47:14.221 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 1 2024-02-02T15:47:14.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.224+0000 7f57d3200700 1 -- 172.21.15.1:0/2919256403 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f57ac0a0690 msgr2=0x7f57ac0a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:47:14.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.224+0000 7f57d3200700 1 --2- 172.21.15.1:0/2919256403 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f57ac0a0690 0x7f57ac0a2b50 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f57bc009c60 tx=0x7f57bc009260 comp rx=0 tx=0).stop 2024-02-02T15:47:14.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.224+0000 7f57d3200700 1 -- 172.21.15.1:0/2919256403 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57cc111d90 msgr2=0x7f57cc1a1f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:47:14.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.224+0000 7f57d3200700 1 --2- 172.21.15.1:0/2919256403 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57cc111d90 0x7f57cc1a1f40 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f57c40094b0 tx=0x7f57c40094e0 comp rx=0 tx=0).stop 2024-02-02T15:47:14.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.224+0000 7f57d3200700 1 -- 172.21.15.1:0/2919256403 shutdown_connections 2024-02-02T15:47:14.224 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.224+0000 7f57d3200700 1 --2- 172.21.15.1:0/2919256403 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f57ac0a0690 0x7f57ac0a2b50 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:14.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.224+0000 7f57d3200700 1 --2- 172.21.15.1:0/2919256403 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f57cc111d90 0x7f57cc1a1f40 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:14.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.224+0000 7f57d3200700 1 --2- 172.21.15.1:0/2919256403 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f57cc0730e0 0x7f57cc19d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:14.225 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.224+0000 7f57d3200700 1 --2- 172.21.15.1:0/2919256403 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f57cc073a00 0x7f57cc19dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:14.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.224+0000 7f57d3200700 1 -- 172.21.15.1:0/2919256403 >> 172.21.15.1:0/2919256403 conn(0x7f57cc0fc810 msgr2=0x7f57cc0fd480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:47:14.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.224+0000 7f57d3200700 1 -- 172.21.15.1:0/2919256403 shutdown_connections 2024-02-02T15:47:14.226 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.224+0000 7f57d3200700 1 -- 172.21.15.1:0/2919256403 wait complete. 2024-02-02T15:47:14.242 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-02-02T15:47:14.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.443+0000 7fbafd800700 1 -- 172.21.15.1:0/2013605629 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbaf8100ff0 msgr2=0x7fbaf81013d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:47:14.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.443+0000 7fbafd800700 1 --2- 172.21.15.1:0/2013605629 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbaf8100ff0 0x7fbaf81013d0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fbae8009990 tx=0x7fbae8009ca0 comp rx=0 tx=0).stop 2024-02-02T15:47:14.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.443+0000 7fbafd800700 1 -- 172.21.15.1:0/2013605629 shutdown_connections 2024-02-02T15:47:14.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.443+0000 7fbafd800700 1 --2- 172.21.15.1:0/2013605629 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbaf8106110 0x7fbaf810dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:14.444 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.443+0000 7fbafd800700 1 --2- 172.21.15.1:0/2013605629 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbaf8100ff0 0x7fbaf81013d0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:14.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.443+0000 7fbafd800700 1 --2- 172.21.15.1:0/2013605629 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbaf81019a0 0x7fbaf81059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:14.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.443+0000 7fbafd800700 1 -- 172.21.15.1:0/2013605629 >> 172.21.15.1:0/2013605629 conn(0x7fbaf80fc810 msgr2=0x7fbaf80fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:47:14.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.444+0000 7fbafd800700 1 -- 172.21.15.1:0/2013605629 shutdown_connections 2024-02-02T15:47:14.445 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.444+0000 7fbafd800700 1 -- 172.21.15.1:0/2013605629 wait complete. 2024-02-02T15:47:14.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.445+0000 7fbafd800700 1 Processor -- start 2024-02-02T15:47:14.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.445+0000 7fbafd800700 1 -- start start 2024-02-02T15:47:14.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.445+0000 7fbafd800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbaf8100ff0 0x7fbaf8199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:47:14.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.445+0000 7fbafd800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbaf8106110 0x7fbaf8199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:47:14.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.445+0000 7fbafd800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbaf819dac0 0x7fbaf819df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:47:14.447 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.445+0000 7fbafd800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf819e540 con 0x7fbaf819dac0 2024-02-02T15:47:14.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.445+0000 7fbafd800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf819e6b0 con 0x7fbaf8100ff0 2024-02-02T15:47:14.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.445+0000 7fbafd800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf819e870 con 0x7fbaf8106110 2024-02-02T15:47:14.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.445+0000 7fbaf7400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbaf819dac0 0x7fbaf819df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:47:14.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.445+0000 7fbaf6a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbaf8100ff0 0x7fbaf8199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:47:14.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.445+0000 7fbaf6a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbaf8100ff0 0x7fbaf8199120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:56878/0 (socket says 172.21.15.1:56878) 2024-02-02T15:47:14.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.445+0000 7fbaf6a00700 1 -- 172.21.15.1:0/3308511258 learned_addr learned my addr 172.21.15.1:0/3308511258 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:47:14.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.446+0000 7fbaf6000700 1 --2- 172.21.15.1:0/3308511258 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbaf8106110 0x7fbaf8199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:47:14.448 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.446+0000 7fbaf6a00700 1 -- 172.21.15.1:0/3308511258 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbaf8106110 msgr2=0x7fbaf8199660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:47:14.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.446+0000 7fbaf6a00700 1 --2- 172.21.15.1:0/3308511258 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbaf8106110 0x7fbaf8199660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:14.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.446+0000 7fbaf6a00700 1 -- 172.21.15.1:0/3308511258 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbaf819dac0 msgr2=0x7fbaf819df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:47:14.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.446+0000 7fbaf6a00700 1 --2- 172.21.15.1:0/3308511258 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbaf819dac0 0x7fbaf819df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:14.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.446+0000 7fbaf6a00700 1 -- 172.21.15.1:0/3308511258 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbae80095f0 con 0x7fbaf8100ff0 2024-02-02T15:47:14.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.446+0000 7fbaf7400700 1 --2- 172.21.15.1:0/3308511258 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbaf819dac0 0x7fbaf819df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:47:14.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.446+0000 7fbaf6a00700 1 --2- 172.21.15.1:0/3308511258 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbaf8100ff0 0x7fbaf8199120 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fbae8005910 tx=0x7fbae8004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:47:14.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.446+0000 7fbaef400700 1 -- 172.21.15.1:0/3308511258 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbae801d070 con 0x7fbaf8100ff0 2024-02-02T15:47:14.449 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.446+0000 7fbafd800700 1 -- 172.21.15.1:0/3308511258 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbaf81a5d70 con 0x7fbaf8100ff0 2024-02-02T15:47:14.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.447+0000 7fbafd800700 1 -- 172.21.15.1:0/3308511258 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbaf81a61a0 con 0x7fbaf8100ff0 2024-02-02T15:47:14.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.447+0000 7fbaef400700 1 -- 172.21.15.1:0/3308511258 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbae8022470 con 0x7fbaf8100ff0 2024-02-02T15:47:14.450 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.447+0000 7fbaef400700 1 -- 172.21.15.1:0/3308511258 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbae8017650 con 0x7fbaf8100ff0 2024-02-02T15:47:14.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.448+0000 7fbafd800700 1 -- 172.21.15.1:0/3308511258 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbaf8067bc0 con 0x7fbaf8100ff0 2024-02-02T15:47:14.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.450+0000 7fbaef400700 1 -- 172.21.15.1:0/3308511258 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7fbae80225e0 con 0x7fbaf8100ff0 2024-02-02T15:47:14.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.451+0000 7fbaef400700 1 --2- 172.21.15.1:0/3308511258 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fbad80a0690 0x7fbad80a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:47:14.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.451+0000 7fbaef400700 1 -- 172.21.15.1:0/3308511258 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 9541+0+0 (secure 0 0 0) 0x7fbae80cd580 con 0x7fbaf8100ff0 2024-02-02T15:47:14.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.451+0000 7fbaf6000700 1 --2- 172.21.15.1:0/3308511258 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fbad80a0690 0x7fbad80a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:47:14.451 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.452+0000 7fbaf6000700 1 --2- 172.21.15.1:0/3308511258 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fbad80a0690 0x7fbad80a2b50 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fbaf819a730 tx=0x7fbae0009260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:47:14.453 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.453+0000 7fbaef400700 1 -- 172.21.15.1:0/3308511258 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7fbae8096a70 con 0x7fbaf8100ff0 2024-02-02T15:47:14.663 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.663+0000 7fbafd800700 1 -- 172.21.15.1:0/3308511258 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbaf8101960 con 0x7fbaf8100ff0 2024-02-02T15:47:14.664 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.664+0000 7fbaef400700 1 -- 172.21.15.1:0/3308511258 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v116) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbae809a090 con 0x7fbaf8100ff0 2024-02-02T15:47:14.664 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-02-02T15:47:14.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.667+0000 7fbafd800700 1 -- 172.21.15.1:0/3308511258 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fbad80a0690 msgr2=0x7fbad80a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:47:14.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.667+0000 7fbafd800700 1 --2- 172.21.15.1:0/3308511258 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fbad80a0690 0x7fbad80a2b50 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fbaf819a730 tx=0x7fbae0009260 comp rx=0 tx=0).stop 2024-02-02T15:47:14.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.667+0000 7fbafd800700 1 -- 172.21.15.1:0/3308511258 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbaf8100ff0 msgr2=0x7fbaf8199120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:47:14.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.667+0000 7fbafd800700 1 --2- 172.21.15.1:0/3308511258 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbaf8100ff0 0x7fbaf8199120 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fbae8005910 tx=0x7fbae8004730 comp rx=0 tx=0).stop 2024-02-02T15:47:14.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.667+0000 7fbafd800700 1 -- 172.21.15.1:0/3308511258 shutdown_connections 2024-02-02T15:47:14.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.667+0000 7fbafd800700 1 --2- 172.21.15.1:0/3308511258 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fbad80a0690 0x7fbad80a2b50 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:14.667 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.667+0000 7fbafd800700 1 --2- 172.21.15.1:0/3308511258 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbaf819dac0 0x7fbaf819df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:14.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.667+0000 7fbafd800700 1 --2- 172.21.15.1:0/3308511258 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbaf8100ff0 0x7fbaf8199120 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:14.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.667+0000 7fbafd800700 1 --2- 172.21.15.1:0/3308511258 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbaf8106110 0x7fbaf8199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:14.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.667+0000 7fbafd800700 1 -- 172.21.15.1:0/3308511258 >> 172.21.15.1:0/3308511258 conn(0x7fbaf80fc810 msgr2=0x7fbaf80fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:47:14.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.667+0000 7fbafd800700 1 -- 172.21.15.1:0/3308511258 shutdown_connections 2024-02-02T15:47:14.668 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.667+0000 7fbafd800700 1 -- 172.21.15.1:0/3308511258 wait complete. 2024-02-02T15:47:14.686 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-02-02T15:47:14.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.886+0000 7f828fa00700 1 -- 172.21.15.1:0/3338929271 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f82880730e0 msgr2=0x7f82880734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:47:14.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.886+0000 7f828fa00700 1 --2- 172.21.15.1:0/3338929271 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f82880730e0 0x7f82880734c0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f8278009990 tx=0x7f8278009ca0 comp rx=0 tx=0).stop 2024-02-02T15:47:14.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.886+0000 7f828fa00700 1 -- 172.21.15.1:0/3338929271 shutdown_connections 2024-02-02T15:47:14.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.886+0000 7f828fa00700 1 --2- 172.21.15.1:0/3338929271 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8288111d90 0x7f8288114180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:14.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.886+0000 7f828fa00700 1 --2- 172.21.15.1:0/3338929271 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f82880730e0 0x7f82880734c0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:14.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.886+0000 7f828fa00700 1 --2- 172.21.15.1:0/3338929271 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8288073a00 0x7f8288111850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:14.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.886+0000 7f828fa00700 1 -- 172.21.15.1:0/3338929271 >> 172.21.15.1:0/3338929271 conn(0x7f82880fc810 msgr2=0x7f82880fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:47:14.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.886+0000 7f828fa00700 1 -- 172.21.15.1:0/3338929271 shutdown_connections 2024-02-02T15:47:14.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.886+0000 7f828fa00700 1 -- 172.21.15.1:0/3338929271 wait complete. 2024-02-02T15:47:14.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.887+0000 7f828fa00700 1 Processor -- start 2024-02-02T15:47:14.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.888+0000 7f828fa00700 1 -- start start 2024-02-02T15:47:14.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.888+0000 7f828fa00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f82880730e0 0x7f828819d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:47:14.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.888+0000 7f828fa00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8288073a00 0x7f828819da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:47:14.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.888+0000 7f828fa00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8288111d90 0x7f82881a1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:47:14.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.888+0000 7f828fa00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82881a24e0 con 0x7f82880730e0 2024-02-02T15:47:14.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.888+0000 7f828fa00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82881a2650 con 0x7f8288073a00 2024-02-02T15:47:14.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.888+0000 7f828fa00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82881a2810 con 0x7f8288111d90 2024-02-02T15:47:14.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.888+0000 7f828d000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f82880730e0 0x7f828819d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:47:14.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.888+0000 7f828d000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f82880730e0 0x7f828819d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43336/0 (socket says 172.21.15.1:43336) 2024-02-02T15:47:14.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.888+0000 7f828d000700 1 -- 172.21.15.1:0/620029797 learned_addr learned my addr 172.21.15.1:0/620029797 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:47:14.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.888+0000 7f828da00700 1 --2- 172.21.15.1:0/620029797 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8288111d90 0x7f82881a1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:47:14.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.888+0000 7f827fe00700 1 --2- 172.21.15.1:0/620029797 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8288073a00 0x7f828819da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:47:14.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.889+0000 7f828d000700 1 -- 172.21.15.1:0/620029797 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8288111d90 msgr2=0x7f82881a1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:47:14.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.889+0000 7f828d000700 1 --2- 172.21.15.1:0/620029797 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8288111d90 0x7f82881a1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:14.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.889+0000 7f828d000700 1 -- 172.21.15.1:0/620029797 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8288073a00 msgr2=0x7f828819da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:47:14.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.889+0000 7f828d000700 1 --2- 172.21.15.1:0/620029797 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8288073a00 0x7f828819da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:14.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.889+0000 7f828d000700 1 -- 172.21.15.1:0/620029797 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8270009520 con 0x7f82880730e0 2024-02-02T15:47:14.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.889+0000 7f828d000700 1 --2- 172.21.15.1:0/620029797 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f82880730e0 0x7f828819d500 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f8278005910 tx=0x7f8278004710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:47:14.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.889+0000 7f8286000700 1 -- 172.21.15.1:0/620029797 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f827801d070 con 0x7f82880730e0 2024-02-02T15:47:14.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.889+0000 7f8286000700 1 -- 172.21.15.1:0/620029797 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8278022470 con 0x7f82880730e0 2024-02-02T15:47:14.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.889+0000 7f828fa00700 1 -- 172.21.15.1:0/620029797 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82780095f0 con 0x7f82880730e0 2024-02-02T15:47:14.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.889+0000 7f8286000700 1 -- 172.21.15.1:0/620029797 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8278017650 con 0x7f82880730e0 2024-02-02T15:47:14.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.889+0000 7f828fa00700 1 -- 172.21.15.1:0/620029797 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82881aa150 con 0x7f82880730e0 2024-02-02T15:47:14.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.890+0000 7f828fa00700 1 -- 172.21.15.1:0/620029797 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8288104270 con 0x7f82880730e0 2024-02-02T15:47:14.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.892+0000 7f8286000700 1 -- 172.21.15.1:0/620029797 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f82780225e0 con 0x7f82880730e0 2024-02-02T15:47:14.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.893+0000 7f8286000700 1 --2- 172.21.15.1:0/620029797 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f82680a0690 0x7f82680a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:47:14.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.894+0000 7f8286000700 1 -- 172.21.15.1:0/620029797 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f82780cd3d0 con 0x7f82880730e0 2024-02-02T15:47:14.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.894+0000 7f827fe00700 1 --2- 172.21.15.1:0/620029797 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f82680a0690 0x7f82680a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:47:14.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.894+0000 7f827fe00700 1 --2- 172.21.15.1:0/620029797 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f82680a0690 0x7f82680a2b50 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f828819eb10 tx=0x7f8270009260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:47:14.897 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:14.897+0000 7f8286000700 1 -- 172.21.15.1:0/620029797 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f827805d110 con 0x7f82880730e0 2024-02-02T15:47:14.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:47:14 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/2919256403' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-02T15:47:14.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:47:14 smithi001 ceph-mon[35470]: pgmap v637: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 74 KiB/s wr, 16 op/s 2024-02-02T15:47:14.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:47:14 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/3308511258' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-02T15:47:15.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.109+0000 7f828fa00700 1 -- 172.21.15.1:0/620029797 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8288103c50 con 0x7f82880730e0 2024-02-02T15:47:15.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.109+0000 7f8286000700 1 -- 172.21.15.1:0/620029797 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v116) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8278096860 con 0x7f82880730e0 2024-02-02T15:47:15.110 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 64 2024-02-02T15:47:15.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.112+0000 7f828fa00700 1 -- 172.21.15.1:0/620029797 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f82680a0690 msgr2=0x7f82680a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:47:15.112 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.112+0000 7f828fa00700 1 --2- 172.21.15.1:0/620029797 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f82680a0690 0x7f82680a2b50 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f828819eb10 tx=0x7f8270009260 comp rx=0 tx=0).stop 2024-02-02T15:47:15.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.112+0000 7f828fa00700 1 -- 172.21.15.1:0/620029797 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f82880730e0 msgr2=0x7f828819d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:47:15.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.112+0000 7f828fa00700 1 --2- 172.21.15.1:0/620029797 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f82880730e0 0x7f828819d500 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f8278005910 tx=0x7f8278004710 comp rx=0 tx=0).stop 2024-02-02T15:47:15.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.112+0000 7f828fa00700 1 -- 172.21.15.1:0/620029797 shutdown_connections 2024-02-02T15:47:15.113 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.112+0000 7f828fa00700 1 --2- 172.21.15.1:0/620029797 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f82680a0690 0x7f82680a2b50 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:15.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.112+0000 7f828fa00700 1 --2- 172.21.15.1:0/620029797 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f8288111d90 0x7f82881a1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:15.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.112+0000 7f828fa00700 1 --2- 172.21.15.1:0/620029797 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f82880730e0 0x7f828819d500 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:15.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.112+0000 7f828fa00700 1 --2- 172.21.15.1:0/620029797 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8288073a00 0x7f828819da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:15.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.112+0000 7f828fa00700 1 -- 172.21.15.1:0/620029797 >> 172.21.15.1:0/620029797 conn(0x7f82880fc810 msgr2=0x7f82880fd480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:47:15.114 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.112+0000 7f828fa00700 1 -- 172.21.15.1:0/620029797 shutdown_connections 2024-02-02T15:47:15.115 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.112+0000 7f828fa00700 1 -- 172.21.15.1:0/620029797 wait complete. 2024-02-02T15:47:15.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:47:14 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/2919256403' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-02T15:47:15.116 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:47:14 smithi071 ceph-mon[44707]: pgmap v637: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 74 KiB/s wr, 16 op/s 2024-02-02T15:47:15.116 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:47:14 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/3308511258' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-02T15:47:15.131 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-02-02T15:47:15.169 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:47:14 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/2919256403' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-02T15:47:15.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:47:14 smithi142 ceph-mon[45728]: pgmap v637: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 74 KiB/s wr, 16 op/s 2024-02-02T15:47:15.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:47:14 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/3308511258' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-02-02T15:47:15.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.333+0000 7fbe4b800700 1 -- 172.21.15.1:0/923274322 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe441019a0 msgr2=0x7fbe441059e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:47:15.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.333+0000 7fbe4b800700 1 --2- 172.21.15.1:0/923274322 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe441019a0 0x7fbe441059e0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c009990 tx=0x7fbe2c009ca0 comp rx=0 tx=0).stop 2024-02-02T15:47:15.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.334+0000 7fbe4b800700 1 -- 172.21.15.1:0/923274322 shutdown_connections 2024-02-02T15:47:15.334 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.334+0000 7fbe4b800700 1 --2- 172.21.15.1:0/923274322 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe44106110 0x7fbe4410dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:15.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.334+0000 7fbe4b800700 1 --2- 172.21.15.1:0/923274322 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe44100ff0 0x7fbe441013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:15.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.334+0000 7fbe4b800700 1 --2- 172.21.15.1:0/923274322 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe441019a0 0x7fbe441059e0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:15.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.334+0000 7fbe4b800700 1 -- 172.21.15.1:0/923274322 >> 172.21.15.1:0/923274322 conn(0x7fbe440fc810 msgr2=0x7fbe440fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:47:15.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.334+0000 7fbe4b800700 1 -- 172.21.15.1:0/923274322 shutdown_connections 2024-02-02T15:47:15.335 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.334+0000 7fbe4b800700 1 -- 172.21.15.1:0/923274322 wait complete. 2024-02-02T15:47:15.337 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.335+0000 7fbe4b800700 1 Processor -- start 2024-02-02T15:47:15.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.335+0000 7fbe4b800700 1 -- start start 2024-02-02T15:47:15.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.335+0000 7fbe4b800700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe44100ff0 0x7fbe44073040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:47:15.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.335+0000 7fbe4b800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe44106110 0x7fbe44073580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:47:15.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.335+0000 7fbe4b800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe44073ac0 0x7fbe441a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:47:15.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.335+0000 7fbe4b800700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe441a5750 con 0x7fbe44073ac0 2024-02-02T15:47:15.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.335+0000 7fbe4b800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe441a58c0 con 0x7fbe44106110 2024-02-02T15:47:15.338 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.335+0000 7fbe4b800700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe441a5a80 con 0x7fbe44100ff0 2024-02-02T15:47:15.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.335+0000 7fbe49800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe44073ac0 0x7fbe441a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:47:15.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.335+0000 7fbe49800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe44073ac0 0x7fbe441a50c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43356/0 (socket says 172.21.15.1:43356) 2024-02-02T15:47:15.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.335+0000 7fbe49800700 1 -- 172.21.15.1:0/1522415909 learned_addr learned my addr 172.21.15.1:0/1522415909 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:47:15.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.336+0000 7fbe48e00700 1 --2- 172.21.15.1:0/1522415909 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe44100ff0 0x7fbe44073040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:47:15.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.336+0000 7fbe43e00700 1 --2- 172.21.15.1:0/1522415909 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe44106110 0x7fbe44073580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:47:15.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.336+0000 7fbe49800700 1 -- 172.21.15.1:0/1522415909 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe44100ff0 msgr2=0x7fbe44073040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:47:15.339 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.336+0000 7fbe49800700 1 --2- 172.21.15.1:0/1522415909 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe44100ff0 0x7fbe44073040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:15.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.336+0000 7fbe49800700 1 -- 172.21.15.1:0/1522415909 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe44106110 msgr2=0x7fbe44073580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:47:15.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.336+0000 7fbe49800700 1 --2- 172.21.15.1:0/1522415909 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe44106110 0x7fbe44073580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:15.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.336+0000 7fbe49800700 1 -- 172.21.15.1:0/1522415909 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe2c0095f0 con 0x7fbe44073ac0 2024-02-02T15:47:15.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.336+0000 7fbe49800700 1 --2- 172.21.15.1:0/1522415909 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe44073ac0 0x7fbe441a50c0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c007ee0 tx=0x7fbe3c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:47:15.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.336+0000 7fbe41600700 1 -- 172.21.15.1:0/1522415909 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3c00dd20 con 0x7fbe44073ac0 2024-02-02T15:47:15.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.336+0000 7fbe41600700 1 -- 172.21.15.1:0/1522415909 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbe3c00f040 con 0x7fbe44073ac0 2024-02-02T15:47:15.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.336+0000 7fbe4b800700 1 -- 172.21.15.1:0/1522415909 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe441a5d60 con 0x7fbe44073ac0 2024-02-02T15:47:15.340 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.336+0000 7fbe41600700 1 -- 172.21.15.1:0/1522415909 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3c0048a0 con 0x7fbe44073ac0 2024-02-02T15:47:15.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.337+0000 7fbe4b800700 1 -- 172.21.15.1:0/1522415909 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe441a62c0 con 0x7fbe44073ac0 2024-02-02T15:47:15.341 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.338+0000 7fbe4b800700 1 -- 172.21.15.1:0/1522415909 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe44067bc0 con 0x7fbe44073ac0 2024-02-02T15:47:15.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.340+0000 7fbe41600700 1 -- 172.21.15.1:0/1522415909 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7fbe3c003a00 con 0x7fbe44073ac0 2024-02-02T15:47:15.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.341+0000 7fbe41600700 1 --2- 172.21.15.1:0/1522415909 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fbe240a0710 0x7fbe240a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:47:15.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.341+0000 7fbe41600700 1 -- 172.21.15.1:0/1522415909 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 9541+0+0 (secure 0 0 0) 0x7fbe3c0cc0a0 con 0x7fbe44073ac0 2024-02-02T15:47:15.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.341+0000 7fbe48e00700 1 --2- 172.21.15.1:0/1522415909 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fbe240a0710 0x7fbe240a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:47:15.342 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.342+0000 7fbe48e00700 1 --2- 172.21.15.1:0/1522415909 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fbe240a0710 0x7fbe240a2bd0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fbe34009c60 tx=0x7fbe34009260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:47:15.343 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.343+0000 7fbe41600700 1 -- 172.21.15.1:0/1522415909 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7fbe3c0954a0 con 0x7fbe44073ac0 2024-02-02T15:47:15.556 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.556+0000 7fbe4b800700 1 -- 172.21.15.1:0/1522415909 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fbe441019a0 con 0x7fbe44073ac0 2024-02-02T15:47:15.557 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.557+0000 7fbe41600700 1 -- 172.21.15.1:0/1522415909 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v116) v1 ==== 103+0+11 (secure 0 0 0) 0x7fbe3c023020 con 0x7fbe44073ac0 2024-02-02T15:47:15.557 INFO:teuthology.orchestra.run.smithi001.stdout:pg_num: 32 2024-02-02T15:47:15.559 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.559+0000 7fbe4b800700 1 -- 172.21.15.1:0/1522415909 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fbe240a0710 msgr2=0x7fbe240a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:47:15.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.559+0000 7fbe4b800700 1 --2- 172.21.15.1:0/1522415909 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fbe240a0710 0x7fbe240a2bd0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fbe34009c60 tx=0x7fbe34009260 comp rx=0 tx=0).stop 2024-02-02T15:47:15.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.559+0000 7fbe4b800700 1 -- 172.21.15.1:0/1522415909 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe44073ac0 msgr2=0x7fbe441a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:47:15.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.559+0000 7fbe4b800700 1 --2- 172.21.15.1:0/1522415909 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe44073ac0 0x7fbe441a50c0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c007ee0 tx=0x7fbe3c00d3b0 comp rx=0 tx=0).stop 2024-02-02T15:47:15.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.559+0000 7fbe4b800700 1 -- 172.21.15.1:0/1522415909 shutdown_connections 2024-02-02T15:47:15.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.559+0000 7fbe4b800700 1 --2- 172.21.15.1:0/1522415909 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7fbe240a0710 0x7fbe240a2bd0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:15.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.559+0000 7fbe4b800700 1 --2- 172.21.15.1:0/1522415909 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7fbe44073ac0 0x7fbe441a50c0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:15.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.559+0000 7fbe4b800700 1 --2- 172.21.15.1:0/1522415909 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7fbe44100ff0 0x7fbe44073040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:15.560 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.559+0000 7fbe4b800700 1 --2- 172.21.15.1:0/1522415909 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbe44106110 0x7fbe44073580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:15.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.559+0000 7fbe4b800700 1 -- 172.21.15.1:0/1522415909 >> 172.21.15.1:0/1522415909 conn(0x7fbe440fc810 msgr2=0x7fbe440fd320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:47:15.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.560+0000 7fbe4b800700 1 -- 172.21.15.1:0/1522415909 shutdown_connections 2024-02-02T15:47:15.561 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.560+0000 7fbe4b800700 1 -- 172.21.15.1:0/1522415909 wait complete. 2024-02-02T15:47:15.578 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-02-02T15:47:15.778 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.778+0000 7f1dd6c00700 1 -- 172.21.15.1:0/3236958677 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1dd01060f0 msgr2=0x7f1dd010dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:47:15.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.778+0000 7f1dd6c00700 1 --2- 172.21.15.1:0/3236958677 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1dd01060f0 0x7f1dd010dc70 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f1dc8009f90 tx=0x7f1dc800b330 comp rx=0 tx=0).stop 2024-02-02T15:47:15.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.778+0000 7f1dd6c00700 1 -- 172.21.15.1:0/3236958677 shutdown_connections 2024-02-02T15:47:15.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.778+0000 7f1dd6c00700 1 --2- 172.21.15.1:0/3236958677 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1dd01060f0 0x7f1dd010dc70 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:15.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.778+0000 7f1dd6c00700 1 --2- 172.21.15.1:0/3236958677 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1dd0100fd0 0x7f1dd01013b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:15.779 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.778+0000 7f1dd6c00700 1 --2- 172.21.15.1:0/3236958677 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1dd0101980 0x7f1dd01059c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:15.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.778+0000 7f1dd6c00700 1 -- 172.21.15.1:0/3236958677 >> 172.21.15.1:0/3236958677 conn(0x7f1dd00fc850 msgr2=0x7f1dd00fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:47:15.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.778+0000 7f1dd6c00700 1 -- 172.21.15.1:0/3236958677 shutdown_connections 2024-02-02T15:47:15.780 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.778+0000 7f1dd6c00700 1 -- 172.21.15.1:0/3236958677 wait complete. 2024-02-02T15:47:15.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.779+0000 7f1dd6c00700 1 Processor -- start 2024-02-02T15:47:15.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.779+0000 7f1dd6c00700 1 -- start start 2024-02-02T15:47:15.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.780+0000 7f1dd6c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1dd0100fd0 0x7f1dd0198fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:47:15.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.780+0000 7f1dd6c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1dd01060f0 0x7f1dd01994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:47:15.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.780+0000 7f1dd6c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1dd019d940 0x7f1dd019ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:47:15.783 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.780+0000 7f1dd6c00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd019e3c0 con 0x7f1dd01060f0 2024-02-02T15:47:15.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.780+0000 7f1dd6c00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd019e530 con 0x7f1dd019d940 2024-02-02T15:47:15.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.780+0000 7f1dd6c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd019e6f0 con 0x7f1dd0100fd0 2024-02-02T15:47:15.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.780+0000 7f1dcfe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1dd0100fd0 0x7f1dd0198fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:47:15.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.780+0000 7f1dc7e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1dd01060f0 0x7f1dd01994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:47:15.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.780+0000 7f1dd4c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1dd019d940 0x7f1dd019ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:47:15.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.780+0000 7f1dcfe00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1dd0100fd0 0x7f1dd0198fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.142:3300/0 says I am v2:172.21.15.1:43990/0 (socket says 172.21.15.1:43990) 2024-02-02T15:47:15.784 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.780+0000 7f1dc7e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1dd01060f0 0x7f1dd01994e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43372/0 (socket says 172.21.15.1:43372) 2024-02-02T15:47:15.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.780+0000 7f1dcfe00700 1 -- 172.21.15.1:0/3768694116 learned_addr learned my addr 172.21.15.1:0/3768694116 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:47:15.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.780+0000 7f1dd4c00700 1 -- 172.21.15.1:0/3768694116 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1dd0100fd0 msgr2=0x7f1dd0198fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:47:15.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.780+0000 7f1dd4c00700 1 --2- 172.21.15.1:0/3768694116 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1dd0100fd0 0x7f1dd0198fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:15.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.780+0000 7f1dd4c00700 1 -- 172.21.15.1:0/3768694116 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1dd01060f0 msgr2=0x7f1dd01994e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:47:15.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.780+0000 7f1dd4c00700 1 --2- 172.21.15.1:0/3768694116 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1dd01060f0 0x7f1dd01994e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:15.785 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.780+0000 7f1dd4c00700 1 -- 172.21.15.1:0/3768694116 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dc00095f0 con 0x7f1dd019d940 2024-02-02T15:47:15.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.780+0000 7f1dc7e00700 1 --2- 172.21.15.1:0/3768694116 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1dd01060f0 0x7f1dd01994e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:47:15.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.781+0000 7f1dd4c00700 1 --2- 172.21.15.1:0/3768694116 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1dd019d940 0x7f1dd019ddc0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f1dc800d010 tx=0x7f1dc8004a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:47:15.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.781+0000 7f1dcd600700 1 -- 172.21.15.1:0/3768694116 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc800e040 con 0x7f1dd019d940 2024-02-02T15:47:15.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.781+0000 7f1dd6c00700 1 -- 172.21.15.1:0/3768694116 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dc8009c40 con 0x7f1dd019d940 2024-02-02T15:47:15.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.781+0000 7f1dcd600700 1 -- 172.21.15.1:0/3768694116 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1dc8004e50 con 0x7f1dd019d940 2024-02-02T15:47:15.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.781+0000 7f1dcd600700 1 -- 172.21.15.1:0/3768694116 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc801b6b0 con 0x7f1dd019d940 2024-02-02T15:47:15.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.781+0000 7f1dd6c00700 1 -- 172.21.15.1:0/3768694116 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dd01a5f30 con 0x7f1dd019d940 2024-02-02T15:47:15.786 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.782+0000 7f1dd6c00700 1 -- 172.21.15.1:0/3768694116 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dd0067bc0 con 0x7f1dd019d940 2024-02-02T15:47:15.787 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.784+0000 7f1dcd600700 1 -- 172.21.15.1:0/3768694116 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f1dc801d480 con 0x7f1dd019d940 2024-02-02T15:47:15.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.786+0000 7f1dcd600700 1 --2- 172.21.15.1:0/3768694116 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f1db00a0710 0x7f1db00a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:47:15.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.786+0000 7f1dcd600700 1 -- 172.21.15.1:0/3768694116 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 9541+0+0 (secure 0 0 0) 0x7f1dc80cdc40 con 0x7f1dd019d940 2024-02-02T15:47:15.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.786+0000 7f1dcfe00700 1 --2- 172.21.15.1:0/3768694116 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f1db00a0710 0x7f1db00a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:47:15.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.786+0000 7f1dcfe00700 1 --2- 172.21.15.1:0/3768694116 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f1db00a0710 0x7f1db00a2bd0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0005d10 tx=0x7f1dc0009310 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:47:15.788 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:15.788+0000 7f1dcd600700 1 -- 172.21.15.1:0/3768694116 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f1dc80970d0 con 0x7f1dd019d940 2024-02-02T15:47:16.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:16.021+0000 7f1dd6c00700 1 -- 172.21.15.1:0/3768694116 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f1dd019a2f0 con 0x7f1dd019d940 2024-02-02T15:47:16.021 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:16.022+0000 7f1dcd600700 1 -- 172.21.15.1:0/3768694116 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v116) v1 ==== 81+0+927 (secure 0 0 0) 0x7f1dc8025e90 con 0x7f1dd019d940 2024-02-02T15:47:16.022 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/3997770901 2024-02-03T15:27:00.115055+0000 2024-02-02T15:47:16.022 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/3236204176 2024-02-03T15:27:00.115055+0000 2024-02-02T15:47:16.022 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/2256116842 2024-02-03T15:27:00.115055+0000 2024-02-02T15:47:16.022 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/4092532060 2024-02-03T15:19:25.836171+0000 2024-02-02T15:47:16.022 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:6801/3768989981 2024-02-03T15:27:00.115055+0000 2024-02-02T15:47:16.022 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:6800/3768989981 2024-02-03T15:27:00.115055+0000 2024-02-02T15:47:16.023 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:6801/395498296 2024-02-03T15:19:25.836171+0000 2024-02-02T15:47:16.023 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:6800/395498296 2024-02-03T15:19:25.836171+0000 2024-02-02T15:47:16.023 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/1574620681 2024-02-03T15:27:00.115055+0000 2024-02-02T15:47:16.023 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/1575373489 2024-02-03T15:19:25.836171+0000 2024-02-02T15:47:16.023 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/865983374 2024-02-03T15:19:01.431596+0000 2024-02-02T15:47:16.023 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/3165960227 2024-02-03T15:19:01.431596+0000 2024-02-02T15:47:16.023 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/2350830293 2024-02-03T15:19:01.431596+0000 2024-02-02T15:47:16.023 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:0/2959690250 2024-02-03T15:19:25.836171+0000 2024-02-02T15:47:16.024 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:6800/1162530732 2024-02-03T15:19:01.431596+0000 2024-02-02T15:47:16.024 INFO:teuthology.orchestra.run.smithi001.stdout:172.21.15.1:6801/1162530732 2024-02-03T15:19:01.431596+0000 2024-02-02T15:47:16.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:16.024+0000 7f1dd6c00700 1 -- 172.21.15.1:0/3768694116 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f1db00a0710 msgr2=0x7f1db00a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:47:16.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:16.024+0000 7f1dd6c00700 1 --2- 172.21.15.1:0/3768694116 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f1db00a0710 0x7f1db00a2bd0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0005d10 tx=0x7f1dc0009310 comp rx=0 tx=0).stop 2024-02-02T15:47:16.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:16.024+0000 7f1dd6c00700 1 -- 172.21.15.1:0/3768694116 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1dd019d940 msgr2=0x7f1dd019ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:47:16.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:16.024+0000 7f1dd6c00700 1 --2- 172.21.15.1:0/3768694116 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1dd019d940 0x7f1dd019ddc0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f1dc800d010 tx=0x7f1dc8004a50 comp rx=0 tx=0).stop 2024-02-02T15:47:16.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:16.024+0000 7f1dd6c00700 1 -- 172.21.15.1:0/3768694116 shutdown_connections 2024-02-02T15:47:16.025 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:16.024+0000 7f1dd6c00700 1 --2- 172.21.15.1:0/3768694116 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f1db00a0710 0x7f1db00a2bd0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:16.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:16.024+0000 7f1dd6c00700 1 --2- 172.21.15.1:0/3768694116 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1dd019d940 0x7f1dd019ddc0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:16.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:16.024+0000 7f1dd6c00700 1 --2- 172.21.15.1:0/3768694116 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f1dd0100fd0 0x7f1dd0198fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:16.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:16.024+0000 7f1dd6c00700 1 --2- 172.21.15.1:0/3768694116 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f1dd01060f0 0x7f1dd01994e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:16.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:16.024+0000 7f1dd6c00700 1 -- 172.21.15.1:0/3768694116 >> 172.21.15.1:0/3768694116 conn(0x7f1dd00fc850 msgr2=0x7f1dd00fd570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:47:16.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:16.025+0000 7f1dd6c00700 1 -- 172.21.15.1:0/3768694116 shutdown_connections 2024-02-02T15:47:16.026 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:16.025+0000 7f1dd6c00700 1 -- 172.21.15.1:0/3768694116 wait complete. 2024-02-02T15:47:16.026 INFO:teuthology.orchestra.run.smithi001.stderr:listed 16 entries 2024-02-02T15:47:16.043 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-02-02T15:47:16.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T15:47:16.043 DEBUG:teuthology.orchestra.run.smithi001:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-02-02T15:47:16.104 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:47:15 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/620029797' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-02T15:47:16.104 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:47:15 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/1522415909' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-02T15:47:16.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:47:15 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/620029797' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-02T15:47:16.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:47:15 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/1522415909' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-02T15:47:16.173 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:47:15 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/620029797' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-02-02T15:47:16.173 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:47:15 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/1522415909' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-02-02T15:47:16.240 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi001.front.sepia.ceph.com 2024-02-02T15:47:16.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T15:47:16.240 DEBUG:teuthology.orchestra.run.smithi001:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-02-02T15:47:16.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T15:47:16.337 DEBUG:teuthology.orchestra.run.smithi001:> ip netns list 2024-02-02T15:47:16.431 INFO:teuthology.orchestra.run.smithi001.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-02-02T15:47:16.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T15:47:16.433 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-02-02T15:47:16.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-02-02T15:47:16.572 DEBUG:teuthology.orchestra.run.smithi001:> sudo ip link delete ceph-brx 2024-02-02T15:47:16.705 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-02-02T15:47:16.717 INFO:tasks.cephadm:Teardown begin 2024-02-02T15:47:16.717 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-02-02T15:47:16.718 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T15:47:16.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:16.926+0000 7f295ac00700 1 -- 172.21.15.1:0/1873408921 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f295410d4e0 msgr2=0x7f295410f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:47:16.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:16.926+0000 7f295ac00700 1 --2- 172.21.15.1:0/1873408921 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f295410d4e0 0x7f295410f950 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f294c00b0a0 tx=0x7f294c00b3b0 comp rx=0 tx=0).stop 2024-02-02T15:47:16.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:16.926+0000 7f295ac00700 1 -- 172.21.15.1:0/1873408921 shutdown_connections 2024-02-02T15:47:16.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:16.926+0000 7f295ac00700 1 --2- 172.21.15.1:0/1873408921 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f295410d4e0 0x7f295410f950 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:16.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:16.926+0000 7f295ac00700 1 --2- 172.21.15.1:0/1873408921 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f2954073170 0x7f2954073550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:16.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:16.926+0000 7f295ac00700 1 --2- 172.21.15.1:0/1873408921 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2954073a90 0x7f295410cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:16.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:16.926+0000 7f295ac00700 1 -- 172.21.15.1:0/1873408921 >> 172.21.15.1:0/1873408921 conn(0x7f29540fc790 msgr2=0x7f29540febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:47:16.927 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:16.926+0000 7f295ac00700 1 -- 172.21.15.1:0/1873408921 shutdown_connections 2024-02-02T15:47:16.928 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:16.926+0000 7f295ac00700 1 -- 172.21.15.1:0/1873408921 wait complete. 2024-02-02T15:47:16.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:16.927+0000 7f295ac00700 1 Processor -- start 2024-02-02T15:47:16.930 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:16.927+0000 7f295ac00700 1 -- start start 2024-02-02T15:47:16.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:16.927+0000 7f295ac00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2954073170 0x7f29541990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:47:16.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:16.928+0000 7f295ac00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2954073a90 0x7f2954199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:47:16.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:16.928+0000 7f295ac00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f295410d4e0 0x7f295419da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:47:16.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:16.928+0000 7f295ac00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f295419e0d0 con 0x7f2954073170 2024-02-02T15:47:16.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:16.928+0000 7f295ac00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f295419e240 con 0x7f2954073a90 2024-02-02T15:47:16.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:16.928+0000 7f295ac00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f295419e400 con 0x7f295410d4e0 2024-02-02T15:47:16.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:16.928+0000 7f2953e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2954073170 0x7f29541990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:47:16.931 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:16.928+0000 7f2953400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2954073a90 0x7f2954199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:47:16.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:16.928+0000 7f2953400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2954073a90 0x7f2954199630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:56934/0 (socket says 172.21.15.1:56934) 2024-02-02T15:47:16.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:16.928+0000 7f2953400700 1 -- 172.21.15.1:0/1764421135 learned_addr learned my addr 172.21.15.1:0/1764421135 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:47:16.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:16.928+0000 7f2953e00700 1 --2- 172.21.15.1:0/1764421135 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2954073170 0x7f29541990f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43394/0 (socket says 172.21.15.1:43394) 2024-02-02T15:47:16.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:16.928+0000 7f2958c00700 1 --2- 172.21.15.1:0/1764421135 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f295410d4e0 0x7f295419da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:47:16.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:16.928+0000 7f2953400700 1 -- 172.21.15.1:0/1764421135 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f295410d4e0 msgr2=0x7f295419da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:47:16.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:16.928+0000 7f2953400700 1 --2- 172.21.15.1:0/1764421135 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f295410d4e0 0x7f295419da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:16.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:16.928+0000 7f2953400700 1 -- 172.21.15.1:0/1764421135 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2954073170 msgr2=0x7f29541990f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:47:16.932 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:16.928+0000 7f2953400700 1 --2- 172.21.15.1:0/1764421135 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2954073170 0x7f29541990f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:16.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:16.928+0000 7f2953400700 1 -- 172.21.15.1:0/1764421135 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f293c009520 con 0x7f2954073a90 2024-02-02T15:47:16.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:16.928+0000 7f2958c00700 1 --2- 172.21.15.1:0/1764421135 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f295410d4e0 0x7f295419da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:47:16.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:16.928+0000 7f2953e00700 1 --2- 172.21.15.1:0/1764421135 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2954073170 0x7f29541990f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:47:16.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:16.929+0000 7f2953400700 1 --2- 172.21.15.1:0/1764421135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2954073a90 0x7f2954199630 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f293c00eb30 tx=0x7f293c00ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:47:16.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:16.929+0000 7f2950c00700 1 -- 172.21.15.1:0/1764421135 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f293c009880 con 0x7f2954073a90 2024-02-02T15:47:16.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:16.929+0000 7f2950c00700 1 -- 172.21.15.1:0/1764421135 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f293c0044e0 con 0x7f2954073a90 2024-02-02T15:47:16.933 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:16.929+0000 7f2950c00700 1 -- 172.21.15.1:0/1764421135 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f293c003d80 con 0x7f2954073a90 2024-02-02T15:47:16.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:16.929+0000 7f295ac00700 1 -- 172.21.15.1:0/1764421135 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f294c009d00 con 0x7f2954073a90 2024-02-02T15:47:16.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:16.929+0000 7f295ac00700 1 -- 172.21.15.1:0/1764421135 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29541a5d40 con 0x7f2954073a90 2024-02-02T15:47:16.934 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:16.930+0000 7f295ac00700 1 -- 172.21.15.1:0/1764421135 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2954107e00 con 0x7f2954073a90 2024-02-02T15:47:16.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:16.932+0000 7f2950c00700 1 -- 172.21.15.1:0/1764421135 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f293c003f20 con 0x7f2954073a90 2024-02-02T15:47:16.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:16.933+0000 7f2950c00700 1 --2- 172.21.15.1:0/1764421135 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f29340a0710 0x7f29340a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:47:16.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:16.933+0000 7f2950c00700 1 -- 172.21.15.1:0/1764421135 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f293c014070 con 0x7f2954073a90 2024-02-02T15:47:16.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:16.933+0000 7f2953e00700 1 --2- 172.21.15.1:0/1764421135 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f29340a0710 0x7f29340a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:47:16.935 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:16.934+0000 7f2953e00700 1 --2- 172.21.15.1:0/1764421135 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f29340a0710 0x7f29340a2bd0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f2944011490 tx=0x7f2944009260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:47:16.936 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:16.937+0000 7f2950c00700 1 -- 172.21.15.1:0/1764421135 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f293c095a20 con 0x7f2954073a90 2024-02-02T15:47:17.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:47:16 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/3768694116' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-02T15:47:17.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:47:16 smithi071 ceph-mon[44707]: pgmap v638: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 110 KiB/s wr, 16 op/s 2024-02-02T15:47:17.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:47:16 smithi071 ceph-mon[44707]: osdmap e117: 12 total, 12 up, 12 in 2024-02-02T15:47:17.169 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:47:16 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/3768694116' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-02T15:47:17.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:47:16 smithi142 ceph-mon[45728]: pgmap v638: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 110 KiB/s wr, 16 op/s 2024-02-02T15:47:17.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:47:16 smithi142 ceph-mon[45728]: osdmap e117: 12 total, 12 up, 12 in 2024-02-02T15:47:17.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:47:16 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/3768694116' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-02T15:47:17.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:47:16 smithi001 ceph-mon[35470]: pgmap v638: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 110 KiB/s wr, 16 op/s 2024-02-02T15:47:17.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:47:16 smithi001 ceph-mon[35470]: osdmap e117: 12 total, 12 up, 12 in 2024-02-02T15:47:17.195 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.196+0000 7f295ac00700 1 -- 172.21.15.1:0/1764421135 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2954067bc0 con 0x7f2954073a90 2024-02-02T15:47:17.196 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.197+0000 7f2950c00700 1 -- 172.21.15.1:0/1764421135 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+10867 (secure 0 0 0) 0x7f293c099040 con 0x7f2954073a90 2024-02-02T15:47:17.197 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:47:17.197 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":83,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2922240448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2922240448},{"type":"v1","addr":"172.21.15.1:6835","nonce":2922240448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2729497431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2729497431},{"type":"v1","addr":"172.21.15.1:6837","nonce":2729497431}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":83,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-02T15:24:47.761435+0000","modified":"2024-02-02T15:46:56.206191+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24529,"mds_2":24518,"mds_3":24514,"mds_4":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14517":{"gid":14517,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/1218303683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":1218303683},{"type":"v1","addr":"172.21.15.1:6839","nonce":1218303683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14529":{"gid":14529,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/2059197496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2059197496},{"type":"v1","addr":"172.21.15.1:6841","nonce":2059197496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/7648986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":7648986},{"type":"v1","addr":"172.21.15.142:6833","nonce":7648986}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/3482746094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3482746094},{"type":"v1","addr":"172.21.15.71:6833","nonce":3482746094}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/388207814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":388207814},{"type":"v1","addr":"172.21.15.71:6835","nonce":388207814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6835/1052606547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1052606547},{"type":"v1","addr":"172.21.15.142:6835","nonce":1052606547}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.142:6837/2870289019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2870289019},{"type":"v1","addr":"172.21.15.142:6837","nonce":2870289019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.71:6837/628016199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":628016199},{"type":"v1","addr":"172.21.15.71:6837","nonce":628016199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.142:6839/321675421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":321675421},{"type":"v1","addr":"172.21.15.142:6839","nonce":321675421}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/1268086493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1268086493},{"type":"v1","addr":"172.21.15.71:6839","nonce":1268086493}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-02T15:47:17.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.199+0000 7f295ac00700 1 -- 172.21.15.1:0/1764421135 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f29340a0710 msgr2=0x7f29340a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:47:17.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.199+0000 7f295ac00700 1 --2- 172.21.15.1:0/1764421135 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f29340a0710 0x7f29340a2bd0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f2944011490 tx=0x7f2944009260 comp rx=0 tx=0).stop 2024-02-02T15:47:17.199 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.199+0000 7f295ac00700 1 -- 172.21.15.1:0/1764421135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2954073a90 msgr2=0x7f2954199630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:47:17.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.199+0000 7f295ac00700 1 --2- 172.21.15.1:0/1764421135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2954073a90 0x7f2954199630 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f293c00eb30 tx=0x7f293c00ee40 comp rx=0 tx=0).stop 2024-02-02T15:47:17.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.199+0000 7f295ac00700 1 -- 172.21.15.1:0/1764421135 shutdown_connections 2024-02-02T15:47:17.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.199+0000 7f295ac00700 1 --2- 172.21.15.1:0/1764421135 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f29340a0710 0x7f29340a2bd0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:17.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.199+0000 7f295ac00700 1 --2- 172.21.15.1:0/1764421135 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f295410d4e0 0x7f295419da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:17.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.199+0000 7f295ac00700 1 --2- 172.21.15.1:0/1764421135 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f2954073170 0x7f29541990f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:17.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.199+0000 7f295ac00700 1 --2- 172.21.15.1:0/1764421135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2954073a90 0x7f2954199630 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:17.200 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.199+0000 7f295ac00700 1 -- 172.21.15.1:0/1764421135 >> 172.21.15.1:0/1764421135 conn(0x7f29540fc790 msgr2=0x7f2954104f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:47:17.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.199+0000 7f295ac00700 1 -- 172.21.15.1:0/1764421135 shutdown_connections 2024-02-02T15:47:17.201 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.199+0000 7f295ac00700 1 -- 172.21.15.1:0/1764421135 wait complete. 2024-02-02T15:47:17.201 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 83 2024-02-02T15:47:17.218 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-02-02T15:47:17.428 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.421+0000 7f9e15200700 1 -- 172.21.15.1:0/3945378328 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e101018c0 msgr2=0x7f9e10105900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:47:17.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.421+0000 7f9e15200700 1 --2- 172.21.15.1:0/3945378328 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e101018c0 0x7f9e10105900 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f9df8009990 tx=0x7f9df8009ca0 comp rx=0 tx=0).stop 2024-02-02T15:47:17.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.421+0000 7f9e15200700 1 -- 172.21.15.1:0/3945378328 shutdown_connections 2024-02-02T15:47:17.429 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.421+0000 7f9e15200700 1 --2- 172.21.15.1:0/3945378328 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e10106030 0x7f9e1010dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:17.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.421+0000 7f9e15200700 1 --2- 172.21.15.1:0/3945378328 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e10100f10 0x7f9e101012f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:17.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.421+0000 7f9e15200700 1 --2- 172.21.15.1:0/3945378328 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e101018c0 0x7f9e10105900 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:17.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.421+0000 7f9e15200700 1 -- 172.21.15.1:0/3945378328 >> 172.21.15.1:0/3945378328 conn(0x7f9e100fc790 msgr2=0x7f9e100febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:47:17.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.421+0000 7f9e15200700 1 -- 172.21.15.1:0/3945378328 shutdown_connections 2024-02-02T15:47:17.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.421+0000 7f9e15200700 1 -- 172.21.15.1:0/3945378328 wait complete. 2024-02-02T15:47:17.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.422+0000 7f9e15200700 1 Processor -- start 2024-02-02T15:47:17.430 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.422+0000 7f9e15200700 1 -- start start 2024-02-02T15:47:17.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.423+0000 7f9e15200700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e10100f10 0x7f9e10073040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:47:17.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.423+0000 7f9e15200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e10106030 0x7f9e10073580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:47:17.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.423+0000 7f9e15200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e10073ac0 0x7f9e101a4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:47:17.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.423+0000 7f9e15200700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e101a5600 con 0x7f9e10073ac0 2024-02-02T15:47:17.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.423+0000 7f9e15200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e101a5770 con 0x7f9e10106030 2024-02-02T15:47:17.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.423+0000 7f9e15200700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e101a5930 con 0x7f9e10100f10 2024-02-02T15:47:17.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.423+0000 7f9e0e800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e10073ac0 0x7f9e101a4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:47:17.431 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.423+0000 7f9e0e800700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e10073ac0 0x7f9e101a4f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43422/0 (socket says 172.21.15.1:43422) 2024-02-02T15:47:17.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.423+0000 7f9e0d400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e10106030 0x7f9e10073580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:47:17.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.423+0000 7f9e0e800700 1 -- 172.21.15.1:0/3002360265 learned_addr learned my addr 172.21.15.1:0/3002360265 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:47:17.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.423+0000 7f9e0de00700 1 --2- 172.21.15.1:0/3002360265 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e10100f10 0x7f9e10073040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:47:17.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.423+0000 7f9e0d400700 1 -- 172.21.15.1:0/3002360265 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e10100f10 msgr2=0x7f9e10073040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:47:17.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.424+0000 7f9e0d400700 1 --2- 172.21.15.1:0/3002360265 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e10100f10 0x7f9e10073040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:17.432 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.424+0000 7f9e0d400700 1 -- 172.21.15.1:0/3002360265 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e10073ac0 msgr2=0x7f9e101a4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:47:17.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.424+0000 7f9e0d400700 1 --2- 172.21.15.1:0/3002360265 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e10073ac0 0x7f9e101a4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:17.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.424+0000 7f9e0d400700 1 -- 172.21.15.1:0/3002360265 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9df80095f0 con 0x7f9e10106030 2024-02-02T15:47:17.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.424+0000 7f9e0e800700 1 --2- 172.21.15.1:0/3002360265 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e10073ac0 0x7f9e101a4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:47:17.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.424+0000 7f9e0d400700 1 --2- 172.21.15.1:0/3002360265 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e10106030 0x7f9e10073580 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f9df8000c00 tx=0x7f9df8005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:47:17.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.424+0000 7f9e06a00700 1 -- 172.21.15.1:0/3002360265 <== mon.2 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df801d070 con 0x7f9e10106030 2024-02-02T15:47:17.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.424+0000 7f9e06a00700 1 -- 172.21.15.1:0/3002360265 <== mon.2 v2:172.21.15.71:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9df8021410 con 0x7f9e10106030 2024-02-02T15:47:17.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.424+0000 7f9e15200700 1 -- 172.21.15.1:0/3002360265 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e101a5bb0 con 0x7f9e10106030 2024-02-02T15:47:17.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.424+0000 7f9e06a00700 1 -- 172.21.15.1:0/3002360265 <== mon.2 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df8017650 con 0x7f9e10106030 2024-02-02T15:47:17.433 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.424+0000 7f9e15200700 1 -- 172.21.15.1:0/3002360265 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e101a6080 con 0x7f9e10106030 2024-02-02T15:47:17.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.425+0000 7f9e15200700 1 -- 172.21.15.1:0/3002360265 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e10067bc0 con 0x7f9e10106030 2024-02-02T15:47:17.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.427+0000 7f9e06a00700 1 -- 172.21.15.1:0/3002360265 <== mon.2 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f9df8021580 con 0x7f9e10106030 2024-02-02T15:47:17.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.428+0000 7f9e06a00700 1 --2- 172.21.15.1:0/3002360265 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f9df00a0710 0x7f9df00a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:47:17.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.429+0000 7f9e06a00700 1 -- 172.21.15.1:0/3002360265 <== mon.2 v2:172.21.15.71:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f9df80cd420 con 0x7f9e10106030 2024-02-02T15:47:17.434 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.429+0000 7f9e0de00700 1 --2- 172.21.15.1:0/3002360265 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f9df00a0710 0x7f9df00a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:47:17.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.429+0000 7f9e0de00700 1 --2- 172.21.15.1:0/3002360265 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f9df00a0710 0x7f9df00a2bd0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f9e00009c60 tx=0x7f9e00009260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:47:17.435 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.433+0000 7f9e06a00700 1 -- 172.21.15.1:0/3002360265 <== mon.2 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f9df8096910 con 0x7f9e10106030 2024-02-02T15:47:17.692 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.692+0000 7f9e15200700 1 -- 172.21.15.1:0/3002360265 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9e10074570 con 0x7f9e10106030 2024-02-02T15:47:17.693 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.693+0000 7f9e06a00700 1 -- 172.21.15.1:0/3002360265 <== mon.2 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) v1 ==== 94+0+10867 (secure 0 0 0) 0x7f9df8099f30 con 0x7f9e10106030 2024-02-02T15:47:17.693 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:47:17.693 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":83,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14457,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6835/2922240448","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6834","nonce":2922240448},{"type":"v1","addr":"172.21.15.1:6835","nonce":2922240448}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14469,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.1:6837/2729497431","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6836","nonce":2729497431},{"type":"v1","addr":"172.21.15.1:6837","nonce":2729497431}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":83,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-02-02T15:24:47.761435+0000","modified":"2024-02-02T15:46:56.206191+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24557,"mds_1":24529,"mds_2":24518,"mds_3":24514,"mds_4":24502},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14517":{"gid":14517,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6839/1218303683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6838","nonce":1218303683},{"type":"v1","addr":"172.21.15.1:6839","nonce":1218303683}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14529":{"gid":14529,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.1:6841/2059197496","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6840","nonce":2059197496},{"type":"v1","addr":"172.21.15.1:6841","nonce":2059197496}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24457":{"gid":24457,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.142:6833/7648986","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6832","nonce":7648986},{"type":"v1","addr":"172.21.15.142:6833","nonce":7648986}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24458":{"gid":24458,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6833/3482746094","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":3482746094},{"type":"v1","addr":"172.21.15.71:6833","nonce":3482746094}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24494":{"gid":24494,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.71:6835/388207814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":388207814},{"type":"v1","addr":"172.21.15.71:6835","nonce":388207814}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24502":{"gid":24502,"name":"f","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.142:6835/1052606547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6834","nonce":1052606547},{"type":"v1","addr":"172.21.15.142:6835","nonce":1052606547}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24514":{"gid":24514,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.142:6837/2870289019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6836","nonce":2870289019},{"type":"v1","addr":"172.21.15.142:6837","nonce":2870289019}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24518":{"gid":24518,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.71:6837/628016199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":628016199},{"type":"v1","addr":"172.21.15.71:6837","nonce":628016199}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24529":{"gid":24529,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.142:6839/321675421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6838","nonce":321675421},{"type":"v1","addr":"172.21.15.142:6839","nonce":321675421}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24557":{"gid":24557,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.71:6839/1268086493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1268086493},{"type":"v1","addr":"172.21.15.71:6839","nonce":1268086493}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-02-02T15:47:17.695 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.695+0000 7f9e15200700 1 -- 172.21.15.1:0/3002360265 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f9df00a0710 msgr2=0x7f9df00a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:47:17.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.695+0000 7f9e15200700 1 --2- 172.21.15.1:0/3002360265 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f9df00a0710 0x7f9df00a2bd0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f9e00009c60 tx=0x7f9e00009260 comp rx=0 tx=0).stop 2024-02-02T15:47:17.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.695+0000 7f9e15200700 1 -- 172.21.15.1:0/3002360265 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e10106030 msgr2=0x7f9e10073580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:47:17.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.695+0000 7f9e15200700 1 --2- 172.21.15.1:0/3002360265 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e10106030 0x7f9e10073580 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f9df8000c00 tx=0x7f9df8005740 comp rx=0 tx=0).stop 2024-02-02T15:47:17.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.695+0000 7f9e15200700 1 -- 172.21.15.1:0/3002360265 shutdown_connections 2024-02-02T15:47:17.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.696+0000 7f9e15200700 1 --2- 172.21.15.1:0/3002360265 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f9df00a0710 0x7f9df00a2bd0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:17.696 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.696+0000 7f9e15200700 1 --2- 172.21.15.1:0/3002360265 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9e10073ac0 0x7f9e101a4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:17.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.696+0000 7f9e15200700 1 --2- 172.21.15.1:0/3002360265 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9e10100f10 0x7f9e10073040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:17.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.696+0000 7f9e15200700 1 --2- 172.21.15.1:0/3002360265 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9e10106030 0x7f9e10073580 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:17.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.696+0000 7f9e15200700 1 -- 172.21.15.1:0/3002360265 >> 172.21.15.1:0/3002360265 conn(0x7f9e100fc790 msgr2=0x7f9e100fd260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:47:17.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.696+0000 7f9e15200700 1 -- 172.21.15.1:0/3002360265 shutdown_connections 2024-02-02T15:47:17.697 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.696+0000 7f9e15200700 1 -- 172.21.15.1:0/3002360265 wait complete. 2024-02-02T15:47:17.697 INFO:teuthology.orchestra.run.smithi001.stderr:dumped fsmap epoch 83 2024-02-02T15:47:17.714 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-02-02T15:47:17.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.913+0000 7f9c63c00700 1 -- 172.21.15.1:0/1791885248 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9c5c106e20 msgr2=0x7f9c5c107200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:47:17.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.913+0000 7f9c63c00700 1 --2- 172.21.15.1:0/1791885248 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9c5c106e20 0x7f9c5c107200 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c009990 tx=0x7f9c4c009ca0 comp rx=0 tx=0).stop 2024-02-02T15:47:17.913 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.913+0000 7f9c63c00700 1 -- 172.21.15.1:0/1791885248 shutdown_connections 2024-02-02T15:47:17.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.913+0000 7f9c63c00700 1 --2- 172.21.15.1:0/1791885248 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9c5c068a30 0x7f9c5c10dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:17.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.913+0000 7f9c63c00700 1 --2- 172.21.15.1:0/1791885248 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9c5c106e20 0x7f9c5c107200 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:17.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.913+0000 7f9c63c00700 1 --2- 172.21.15.1:0/1791885248 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9c5c068070 0x7f9c5c0684f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:17.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.913+0000 7f9c63c00700 1 -- 172.21.15.1:0/1791885248 >> 172.21.15.1:0/1791885248 conn(0x7f9c5c0769c0 msgr2=0x7f9c5c076dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:47:17.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.913+0000 7f9c63c00700 1 -- 172.21.15.1:0/1791885248 shutdown_connections 2024-02-02T15:47:17.914 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.913+0000 7f9c63c00700 1 -- 172.21.15.1:0/1791885248 wait complete. 2024-02-02T15:47:17.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.914+0000 7f9c63c00700 1 Processor -- start 2024-02-02T15:47:17.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.914+0000 7f9c63c00700 1 -- start start 2024-02-02T15:47:17.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.914+0000 7f9c63c00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9c5c068070 0x7f9c5c0ffc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:47:17.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.914+0000 7f9c63c00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9c5c068a30 0x7f9c5c100170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:47:17.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.914+0000 7f9c63c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9c5c106e20 0x7f9c5c1a4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:47:17.917 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.914+0000 7f9c63c00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5c1009a0 con 0x7f9c5c068a30 2024-02-02T15:47:17.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.914+0000 7f9c63c00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5c1a5470 con 0x7f9c5c106e20 2024-02-02T15:47:17.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.914+0000 7f9c63c00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5c1a55b0 con 0x7f9c5c068070 2024-02-02T15:47:17.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.915+0000 7f9c53e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9c5c068a30 0x7f9c5c100170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:47:17.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.915+0000 7f9c53e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9c5c068a30 0x7f9c5c100170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43436/0 (socket says 172.21.15.1:43436) 2024-02-02T15:47:17.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.915+0000 7f9c53e00700 1 -- 172.21.15.1:0/1803689724 learned_addr learned my addr 172.21.15.1:0/1803689724 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:47:17.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.915+0000 7f9c61200700 1 --2- 172.21.15.1:0/1803689724 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9c5c068070 0x7f9c5c0ffc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:47:17.918 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.915+0000 7f9c61c00700 1 --2- 172.21.15.1:0/1803689724 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9c5c106e20 0x7f9c5c1a4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:47:17.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.915+0000 7f9c53e00700 1 -- 172.21.15.1:0/1803689724 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9c5c068070 msgr2=0x7f9c5c0ffc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:47:17.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.915+0000 7f9c53e00700 1 --2- 172.21.15.1:0/1803689724 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9c5c068070 0x7f9c5c0ffc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:17.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.915+0000 7f9c53e00700 1 -- 172.21.15.1:0/1803689724 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9c5c106e20 msgr2=0x7f9c5c1a4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:47:17.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.915+0000 7f9c53e00700 1 --2- 172.21.15.1:0/1803689724 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9c5c106e20 0x7f9c5c1a4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:17.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.915+0000 7f9c53e00700 1 -- 172.21.15.1:0/1803689724 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c4c0095f0 con 0x7f9c5c068a30 2024-02-02T15:47:17.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.915+0000 7f9c53e00700 1 --2- 172.21.15.1:0/1803689724 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9c5c068a30 0x7f9c5c100170 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f9c4400e9a0 tx=0x7f9c4400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:47:17.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.916+0000 7f9c5a000700 1 -- 172.21.15.1:0/1803689724 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c44009790 con 0x7f9c5c068a30 2024-02-02T15:47:17.919 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.916+0000 7f9c5a000700 1 -- 172.21.15.1:0/1803689724 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9c44010430 con 0x7f9c5c068a30 2024-02-02T15:47:17.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.916+0000 7f9c63c00700 1 -- 172.21.15.1:0/1803689724 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c5c1a5810 con 0x7f9c5c068a30 2024-02-02T15:47:17.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.916+0000 7f9c5a000700 1 -- 172.21.15.1:0/1803689724 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c44018680 con 0x7f9c5c068a30 2024-02-02T15:47:17.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.916+0000 7f9c63c00700 1 -- 172.21.15.1:0/1803689724 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c5c1a5d40 con 0x7f9c5c068a30 2024-02-02T15:47:17.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.917+0000 7f9c63c00700 1 -- 172.21.15.1:0/1803689724 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c5c067bc0 con 0x7f9c5c068a30 2024-02-02T15:47:17.920 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.919+0000 7f9c5a000700 1 -- 172.21.15.1:0/1803689724 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f9c440105a0 con 0x7f9c5c068a30 2024-02-02T15:47:17.921 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.921+0000 7f9c5a000700 1 --2- 172.21.15.1:0/1803689724 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f9c3c0a0710 0x7f9c3c0a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:47:17.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.922+0000 7f9c5a000700 1 -- 172.21.15.1:0/1803689724 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f9c44014070 con 0x7f9c5c068a30 2024-02-02T15:47:17.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.922+0000 7f9c61200700 1 --2- 172.21.15.1:0/1803689724 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f9c3c0a0710 0x7f9c3c0a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:47:17.922 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.922+0000 7f9c61200700 1 --2- 172.21.15.1:0/1803689724 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f9c3c0a0710 0x7f9c3c0a2bd0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c00b2e0 tx=0x7f9c4c005fd0 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:47:17.926 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:17.926+0000 7f9c5a000700 1 -- 172.21.15.1:0/1803689724 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f9c44052110 con 0x7f9c5c068a30 2024-02-02T15:47:18.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:47:17 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/1764421135' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:47:18.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:47:17 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/3002360265' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:47:18.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.136+0000 7f9c63c00700 1 -- 172.21.15.1:0/1803689724 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9c5c101160 con 0x7f9c5c068a30 2024-02-02T15:47:18.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.138+0000 7f9c5a000700 1 -- 172.21.15.1:0/1803689724 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v117) v1 ==== 74+0+25013 (secure 0 0 0) 0x7f9c44095750 con 0x7f9c5c068a30 2024-02-02T15:47:18.138 INFO:teuthology.orchestra.run.smithi001.stdout: 2024-02-02T15:47:18.139 INFO:teuthology.orchestra.run.smithi001.stdout:{"epoch":117,"fsid":"371a409c-c1de-11ee-95b6-87774f69a715","created":"2024-02-02T15:18:26.746506+0000","modified":"2024-02-02T15:47:16.697363+0000","last_up_change":"2024-02-02T15:24:08.800951+0000","last_in_change":"2024-02-02T15:23:52.784280+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-02T15:21:22.471754+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-02-02T15:24:46.195779+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-02-02T15:24:47.226403+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":35,"snap_epoch":116,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-02-02T15:24:50.300462+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":35,"snap_epoch":116,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"73e8a07f-3560-4c19-85ed-030d4d8f0f23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6802","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6803","nonce":2591906423}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6804","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6805","nonce":2591906423}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6808","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6809","nonce":2591906423}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6806","nonce":2591906423},{"type":"v1","addr":"172.21.15.1:6807","nonce":2591906423}]},"public_addr":"172.21.15.1:6803/2591906423","cluster_addr":"172.21.15.1:6805/2591906423","heartbeat_back_addr":"172.21.15.1:6809/2591906423","heartbeat_front_addr":"172.21.15.1:6807/2591906423","state":["exists","up"]},{"osd":1,"uuid":"e53cffec-0ff9-4ed9-8dcf-ff47d04db2bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6801","nonce":747581719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6803","nonce":747581719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6807","nonce":747581719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":747581719},{"type":"v1","addr":"172.21.15.71:6805","nonce":747581719}]},"public_addr":"172.21.15.71:6801/747581719","cluster_addr":"172.21.15.71:6803/747581719","heartbeat_back_addr":"172.21.15.71:6807/747581719","heartbeat_front_addr":"172.21.15.71:6805/747581719","state":["exists","up"]},{"osd":2,"uuid":"856c2347-294c-482b-8595-ad77f4792db4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6801","nonce":377511414}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6803","nonce":377511414}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6806","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6807","nonce":377511414}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6804","nonce":377511414},{"type":"v1","addr":"172.21.15.142:6805","nonce":377511414}]},"public_addr":"172.21.15.142:6801/377511414","cluster_addr":"172.21.15.142:6803/377511414","heartbeat_back_addr":"172.21.15.142:6807/377511414","heartbeat_front_addr":"172.21.15.142:6805/377511414","state":["exists","up"]},{"osd":3,"uuid":"4b946ad2-2da9-4e0b-9d61-d039d8669a42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6810","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6811","nonce":1086124477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6812","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6813","nonce":1086124477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6816","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6817","nonce":1086124477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6814","nonce":1086124477},{"type":"v1","addr":"172.21.15.1:6815","nonce":1086124477}]},"public_addr":"172.21.15.1:6811/1086124477","cluster_addr":"172.21.15.1:6813/1086124477","heartbeat_back_addr":"172.21.15.1:6817/1086124477","heartbeat_front_addr":"172.21.15.1:6815/1086124477","state":["exists","up"]},{"osd":4,"uuid":"43e36c43-47a0-4634-a45b-1da14a0e19d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6809","nonce":3095380305}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6811","nonce":3095380305}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6815","nonce":3095380305}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":3095380305},{"type":"v1","addr":"172.21.15.71:6813","nonce":3095380305}]},"public_addr":"172.21.15.71:6809/3095380305","cluster_addr":"172.21.15.71:6811/3095380305","heartbeat_back_addr":"172.21.15.71:6815/3095380305","heartbeat_front_addr":"172.21.15.71:6813/3095380305","state":["exists","up"]},{"osd":5,"uuid":"eb48160e-ea7c-42f7-9477-be110bc8a34e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6809","nonce":4195149262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6810","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6811","nonce":4195149262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6814","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6815","nonce":4195149262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":4195149262},{"type":"v1","addr":"172.21.15.142:6813","nonce":4195149262}]},"public_addr":"172.21.15.142:6809/4195149262","cluster_addr":"172.21.15.142:6811/4195149262","heartbeat_back_addr":"172.21.15.142:6815/4195149262","heartbeat_front_addr":"172.21.15.142:6813/4195149262","state":["exists","up"]},{"osd":6,"uuid":"0ec4501a-5853-4ca6-b325-231f5b72335e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6818","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6819","nonce":1560153428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6820","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6821","nonce":1560153428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6824","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6825","nonce":1560153428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6822","nonce":1560153428},{"type":"v1","addr":"172.21.15.1:6823","nonce":1560153428}]},"public_addr":"172.21.15.1:6819/1560153428","cluster_addr":"172.21.15.1:6821/1560153428","heartbeat_back_addr":"172.21.15.1:6825/1560153428","heartbeat_front_addr":"172.21.15.1:6823/1560153428","state":["exists","up"]},{"osd":7,"uuid":"029debe8-9cdf-44f6-9aea-cabd1d185f93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6817","nonce":4080955538}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6819","nonce":4080955538}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6823","nonce":4080955538}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":4080955538},{"type":"v1","addr":"172.21.15.71:6821","nonce":4080955538}]},"public_addr":"172.21.15.71:6817/4080955538","cluster_addr":"172.21.15.71:6819/4080955538","heartbeat_back_addr":"172.21.15.71:6823/4080955538","heartbeat_front_addr":"172.21.15.71:6821/4080955538","state":["exists","up"]},{"osd":8,"uuid":"7c90896c-9955-4c1b-aaca-7531dd5457f6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6817","nonce":3942038209}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6819","nonce":3942038209}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6823","nonce":3942038209}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":3942038209},{"type":"v1","addr":"172.21.15.142:6821","nonce":3942038209}]},"public_addr":"172.21.15.142:6817/3942038209","cluster_addr":"172.21.15.142:6819/3942038209","heartbeat_back_addr":"172.21.15.142:6823/3942038209","heartbeat_front_addr":"172.21.15.142:6821/3942038209","state":["exists","up"]},{"osd":9,"uuid":"7e165e68-31a8-4598-a577-de3e90ae5c5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6826","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6827","nonce":2716544850}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6828","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6829","nonce":2716544850}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6832","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6833","nonce":2716544850}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.1:6830","nonce":2716544850},{"type":"v1","addr":"172.21.15.1:6831","nonce":2716544850}]},"public_addr":"172.21.15.1:6827/2716544850","cluster_addr":"172.21.15.1:6829/2716544850","heartbeat_back_addr":"172.21.15.1:6833/2716544850","heartbeat_front_addr":"172.21.15.1:6831/2716544850","state":["exists","up"]},{"osd":10,"uuid":"48cd5fdb-0887-444a-aab8-a1de24282bb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6825","nonce":84691841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6827","nonce":84691841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6831","nonce":84691841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":84691841},{"type":"v1","addr":"172.21.15.71:6829","nonce":84691841}]},"public_addr":"172.21.15.71:6825/84691841","cluster_addr":"172.21.15.71:6827/84691841","heartbeat_back_addr":"172.21.15.71:6831/84691841","heartbeat_front_addr":"172.21.15.71:6829/84691841","state":["exists","up"]},{"osd":11,"uuid":"f011dd27-11b0-4166-adfe-76756533163a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6825","nonce":2290179934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6827","nonce":2290179934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6831","nonce":2290179934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2290179934},{"type":"v1","addr":"172.21.15.142:6829","nonce":2290179934}]},"public_addr":"172.21.15.142:6825/2290179934","cluster_addr":"172.21.15.142:6827/2290179934","heartbeat_back_addr":"172.21.15.142:6831/2290179934","heartbeat_front_addr":"172.21.15.142:6829/2290179934","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:20:44.060692+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:01.903817+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:19.368258+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:40.028037+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:21:57.513361+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:22:14.805944+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:22:35.188439+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:22:53.014412+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:23:10.507420+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:23:31.849765+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:23:50.070612+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-02-02T15:24:07.388625+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":1}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.14","mappings":[{"from":4,"to":7}]},{"pgid":"3.1d","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":5,"to":6}]},{"pgid":"3.29","mappings":[{"from":3,"to":9}]},{"pgid":"4.e","mappings":[{"from":9,"to":11}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]},{"pgid":"4.12","mappings":[{"from":5,"to":10}]},{"pgid":"4.16","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.1:0/3997770901":"2024-02-03T15:27:00.115055+0000","172.21.15.1:0/3236204176":"2024-02-03T15:27:00.115055+0000","172.21.15.1:0/2256116842":"2024-02-03T15:27:00.115055+0000","172.21.15.1:0/4092532060":"2024-02-03T15:19:25.836171+0000","172.21.15.1:6801/3768989981":"2024-02-03T15:27:00.115055+0000","172.21.15.1:6800/3768989981":"2024-02-03T15:27:00.115055+0000","172.21.15.1:6801/395498296":"2024-02-03T15:19:25.836171+0000","172.21.15.1:6800/395498296":"2024-02-03T15:19:25.836171+0000","172.21.15.1:0/1574620681":"2024-02-03T15:27:00.115055+0000","172.21.15.1:0/1575373489":"2024-02-03T15:19:25.836171+0000","172.21.15.1:0/865983374":"2024-02-03T15:19:01.431596+0000","172.21.15.1:0/3165960227":"2024-02-03T15:19:01.431596+0000","172.21.15.1:0/2350830293":"2024-02-03T15:19:01.431596+0000","172.21.15.1:0/2959690250":"2024-02-03T15:19:25.836171+0000","172.21.15.1:6800/1162530732":"2024-02-03T15:19:01.431596+0000","172.21.15.1:6801/1162530732":"2024-02-03T15:19:01.431596+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":22,"length":1},{"begin":35,"length":1}]},{"pool":4,"snaps":[{"begin":22,"length":1},{"begin":35,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-02T15:47:18.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.141+0000 7f9c63c00700 1 -- 172.21.15.1:0/1803689724 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f9c3c0a0710 msgr2=0x7f9c3c0a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:47:18.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.141+0000 7f9c63c00700 1 --2- 172.21.15.1:0/1803689724 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f9c3c0a0710 0x7f9c3c0a2bd0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c00b2e0 tx=0x7f9c4c005fd0 comp rx=0 tx=0).stop 2024-02-02T15:47:18.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.141+0000 7f9c63c00700 1 -- 172.21.15.1:0/1803689724 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9c5c068a30 msgr2=0x7f9c5c100170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:47:18.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.141+0000 7f9c63c00700 1 --2- 172.21.15.1:0/1803689724 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9c5c068a30 0x7f9c5c100170 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f9c4400e9a0 tx=0x7f9c4400ecb0 comp rx=0 tx=0).stop 2024-02-02T15:47:18.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.141+0000 7f9c63c00700 1 -- 172.21.15.1:0/1803689724 shutdown_connections 2024-02-02T15:47:18.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.141+0000 7f9c63c00700 1 --2- 172.21.15.1:0/1803689724 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f9c3c0a0710 0x7f9c3c0a2bd0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:18.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.141+0000 7f9c63c00700 1 --2- 172.21.15.1:0/1803689724 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9c5c106e20 0x7f9c5c1a4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:18.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.141+0000 7f9c63c00700 1 --2- 172.21.15.1:0/1803689724 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f9c5c068070 0x7f9c5c0ffc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:18.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.141+0000 7f9c63c00700 1 --2- 172.21.15.1:0/1803689724 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f9c5c068a30 0x7f9c5c100170 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:18.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.141+0000 7f9c63c00700 1 -- 172.21.15.1:0/1803689724 >> 172.21.15.1:0/1803689724 conn(0x7f9c5c0769c0 msgr2=0x7f9c5c0fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:47:18.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.141+0000 7f9c63c00700 1 -- 172.21.15.1:0/1803689724 shutdown_connections 2024-02-02T15:47:18.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.141+0000 7f9c63c00700 1 -- 172.21.15.1:0/1803689724 wait complete. 2024-02-02T15:47:18.157 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:47:17 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/1764421135' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:47:18.158 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:47:17 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/3002360265' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:47:18.160 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-02-02T15:47:18.169 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:47:17 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/1764421135' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:47:18.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:47:17 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/3002360265' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-02-02T15:47:18.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.362+0000 7f05b6e00700 1 -- 172.21.15.1:0/1440728784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05b0101990 msgr2=0x7f05b01059d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:47:18.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.362+0000 7f05b6e00700 1 --2- 172.21.15.1:0/1440728784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05b0101990 0x7f05b01059d0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f0598009990 tx=0x7f0598009ca0 comp rx=0 tx=0).stop 2024-02-02T15:47:18.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.362+0000 7f05b6e00700 1 -- 172.21.15.1:0/1440728784 shutdown_connections 2024-02-02T15:47:18.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.362+0000 7f05b6e00700 1 --2- 172.21.15.1:0/1440728784 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f05b0106100 0x7f05b010dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:18.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.362+0000 7f05b6e00700 1 --2- 172.21.15.1:0/1440728784 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05b0100fe0 0x7f05b01013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:18.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.362+0000 7f05b6e00700 1 --2- 172.21.15.1:0/1440728784 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05b0101990 0x7f05b01059d0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:18.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.362+0000 7f05b6e00700 1 -- 172.21.15.1:0/1440728784 >> 172.21.15.1:0/1440728784 conn(0x7f05b00fc880 msgr2=0x7f05b00feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:47:18.363 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.362+0000 7f05b6e00700 1 -- 172.21.15.1:0/1440728784 shutdown_connections 2024-02-02T15:47:18.364 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.362+0000 7f05b6e00700 1 -- 172.21.15.1:0/1440728784 wait complete. 2024-02-02T15:47:18.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.363+0000 7f05b6e00700 1 Processor -- start 2024-02-02T15:47:18.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.363+0000 7f05b6e00700 1 -- start start 2024-02-02T15:47:18.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.364+0000 7f05b6e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05b0100fe0 0x7f05b0073040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:47:18.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.364+0000 7f05b6e00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05b0106100 0x7f05b0073580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:47:18.366 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.364+0000 7f05b6e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f05b0073ac0 0x7f05b01a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:47:18.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.364+0000 7f05b6e00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b01a5760 con 0x7f05b0073ac0 2024-02-02T15:47:18.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.364+0000 7f05b6e00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b01a58d0 con 0x7f05b0100fe0 2024-02-02T15:47:18.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.364+0000 7f05b6e00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b01a5a90 con 0x7f05b0106100 2024-02-02T15:47:18.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.364+0000 7f05b4e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f05b0073ac0 0x7f05b01a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:47:18.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.364+0000 7f05b4e00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f05b0073ac0 0x7f05b01a50d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43460/0 (socket says 172.21.15.1:43460) 2024-02-02T15:47:18.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.364+0000 7f05b4e00700 1 -- 172.21.15.1:0/1017746641 learned_addr learned my addr 172.21.15.1:0/1017746641 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:47:18.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.364+0000 7f05afe00700 1 --2- 172.21.15.1:0/1017746641 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05b0100fe0 0x7f05b0073040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:47:18.367 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.364+0000 7f05af400700 1 --2- 172.21.15.1:0/1017746641 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05b0106100 0x7f05b0073580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:47:18.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.364+0000 7f05b4e00700 1 -- 172.21.15.1:0/1017746641 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05b0106100 msgr2=0x7f05b0073580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:47:18.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.364+0000 7f05b4e00700 1 --2- 172.21.15.1:0/1017746641 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05b0106100 0x7f05b0073580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:18.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.364+0000 7f05b4e00700 1 -- 172.21.15.1:0/1017746641 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05b0100fe0 msgr2=0x7f05b0073040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:47:18.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.364+0000 7f05b4e00700 1 --2- 172.21.15.1:0/1017746641 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05b0100fe0 0x7f05b0073040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:18.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.364+0000 7f05b4e00700 1 -- 172.21.15.1:0/1017746641 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05a8009c40 con 0x7f05b0073ac0 2024-02-02T15:47:18.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.365+0000 7f05b4e00700 1 --2- 172.21.15.1:0/1017746641 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f05b0073ac0 0x7f05b01a50d0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f05a800c370 tx=0x7f05a800c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:47:18.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.365+0000 7f05acc00700 1 -- 172.21.15.1:0/1017746641 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a8017070 con 0x7f05b0073ac0 2024-02-02T15:47:18.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.365+0000 7f05acc00700 1 -- 172.21.15.1:0/1017746641 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f05a800f040 con 0x7f05b0073ac0 2024-02-02T15:47:18.368 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.365+0000 7f05acc00700 1 -- 172.21.15.1:0/1017746641 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a800e050 con 0x7f05b0073ac0 2024-02-02T15:47:18.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.365+0000 7f05b6e00700 1 -- 172.21.15.1:0/1017746641 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05980095f0 con 0x7f05b0073ac0 2024-02-02T15:47:18.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.365+0000 7f05b6e00700 1 -- 172.21.15.1:0/1017746641 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05b01a6060 con 0x7f05b0073ac0 2024-02-02T15:47:18.369 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.366+0000 7f05b6e00700 1 -- 172.21.15.1:0/1017746641 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05b0067bc0 con 0x7f05b0073ac0 2024-02-02T15:47:18.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.368+0000 7f05acc00700 1 -- 172.21.15.1:0/1017746641 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f05a800cb60 con 0x7f05b0073ac0 2024-02-02T15:47:18.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.369+0000 7f05acc00700 1 --2- 172.21.15.1:0/1017746641 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f05900a0710 0x7f05900a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:47:18.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.370+0000 7f05acc00700 1 -- 172.21.15.1:0/1017746641 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 9453+0+0 (secure 0 0 0) 0x7f05a80cbfc0 con 0x7f05b0073ac0 2024-02-02T15:47:18.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.370+0000 7f05afe00700 1 --2- 172.21.15.1:0/1017746641 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f05900a0710 0x7f05900a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:47:18.370 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.370+0000 7f05afe00700 1 --2- 172.21.15.1:0/1017746641 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f05900a0710 0x7f05900a2bd0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f05a0009c60 tx=0x7f05a0009260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:47:18.372 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.372+0000 7f05acc00700 1 -- 172.21.15.1:0/1017746641 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f05a8095420 con 0x7f05b0073ac0 2024-02-02T15:47:18.631 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.631+0000 7f05b6e00700 1 -- 172.21.15.1:0/1017746641 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f05b0101990 con 0x7f05b0073ac0 2024-02-02T15:47:18.643 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.643+0000 7f05acc00700 1 -- 172.21.15.1:0/1017746641 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v84) v1 ==== 161+0+0 (secure 0 0 0) 0x7f05a8098a40 con 0x7f05b0073ac0 2024-02-02T15:47:18.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.646+0000 7f05b6e00700 1 -- 172.21.15.1:0/1017746641 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f05900a0710 msgr2=0x7f05900a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:47:18.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.646+0000 7f05b6e00700 1 --2- 172.21.15.1:0/1017746641 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f05900a0710 0x7f05900a2bd0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f05a0009c60 tx=0x7f05a0009260 comp rx=0 tx=0).stop 2024-02-02T15:47:18.648 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.646+0000 7f05b6e00700 1 -- 172.21.15.1:0/1017746641 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f05b0073ac0 msgr2=0x7f05b01a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:47:18.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.646+0000 7f05b6e00700 1 --2- 172.21.15.1:0/1017746641 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f05b0073ac0 0x7f05b01a50d0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f05a800c370 tx=0x7f05a800c680 comp rx=0 tx=0).stop 2024-02-02T15:47:18.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.646+0000 7f05b6e00700 1 -- 172.21.15.1:0/1017746641 shutdown_connections 2024-02-02T15:47:18.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.646+0000 7f05b6e00700 1 --2- 172.21.15.1:0/1017746641 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f05900a0710 0x7f05900a2bd0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:18.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.646+0000 7f05b6e00700 1 --2- 172.21.15.1:0/1017746641 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f05b0073ac0 0x7f05b01a50d0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:18.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.646+0000 7f05b6e00700 1 --2- 172.21.15.1:0/1017746641 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05b0100fe0 0x7f05b0073040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:18.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.646+0000 7f05b6e00700 1 --2- 172.21.15.1:0/1017746641 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f05b0106100 0x7f05b0073580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:18.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.646+0000 7f05b6e00700 1 -- 172.21.15.1:0/1017746641 >> 172.21.15.1:0/1017746641 conn(0x7f05b00fc880 msgr2=0x7f05b00fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:47:18.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.646+0000 7f05b6e00700 1 -- 172.21.15.1:0/1017746641 shutdown_connections 2024-02-02T15:47:18.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.646+0000 7f05b6e00700 1 -- 172.21.15.1:0/1017746641 wait complete. 2024-02-02T15:47:18.650 INFO:teuthology.orchestra.run.smithi001.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-02-02T15:47:18.667 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-02-02T15:47:18.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.884+0000 7ff82bc00700 1 -- 172.21.15.1:0/2936752085 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff8240730e0 msgr2=0x7ff8240734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:47:18.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.884+0000 7ff82bc00700 1 --2- 172.21.15.1:0/2936752085 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff8240730e0 0x7ff8240734c0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7ff814009990 tx=0x7ff814009ca0 comp rx=0 tx=0).stop 2024-02-02T15:47:18.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.884+0000 7ff82bc00700 1 -- 172.21.15.1:0/2936752085 shutdown_connections 2024-02-02T15:47:18.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.884+0000 7ff82bc00700 1 --2- 172.21.15.1:0/2936752085 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff824111d90 0x7ff824114180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:18.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.884+0000 7ff82bc00700 1 --2- 172.21.15.1:0/2936752085 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff8240730e0 0x7ff8240734c0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:18.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.884+0000 7ff82bc00700 1 --2- 172.21.15.1:0/2936752085 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff824073a00 0x7ff824111850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:18.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.884+0000 7ff82bc00700 1 -- 172.21.15.1:0/2936752085 >> 172.21.15.1:0/2936752085 conn(0x7ff8240fc810 msgr2=0x7ff8240fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:47:18.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.884+0000 7ff82bc00700 1 -- 172.21.15.1:0/2936752085 shutdown_connections 2024-02-02T15:47:18.886 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.884+0000 7ff82bc00700 1 -- 172.21.15.1:0/2936752085 wait complete. 2024-02-02T15:47:18.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.885+0000 7ff82bc00700 1 Processor -- start 2024-02-02T15:47:18.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.885+0000 7ff82bc00700 1 -- start start 2024-02-02T15:47:18.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.885+0000 7ff82bc00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff8240730e0 0x7ff82419d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:47:18.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.885+0000 7ff82bc00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff824073a00 0x7ff82419da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:47:18.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.885+0000 7ff82bc00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff824111d90 0x7ff8241a1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:47:18.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.885+0000 7ff82bc00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8241a24d0 con 0x7ff8240730e0 2024-02-02T15:47:18.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.885+0000 7ff82bc00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8241a2640 con 0x7ff824111d90 2024-02-02T15:47:18.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.885+0000 7ff82bc00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8241a2800 con 0x7ff824073a00 2024-02-02T15:47:18.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.886+0000 7ff829200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff8240730e0 0x7ff82419d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:47:18.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.886+0000 7ff829200700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff8240730e0 0x7ff82419d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43508/0 (socket says 172.21.15.1:43508) 2024-02-02T15:47:18.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.886+0000 7ff829200700 1 -- 172.21.15.1:0/2419236202 learned_addr learned my addr 172.21.15.1:0/2419236202 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:47:18.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.886+0000 7ff823e00700 1 --2- 172.21.15.1:0/2419236202 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff824073a00 0x7ff82419da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:47:18.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.886+0000 7ff829c00700 1 --2- 172.21.15.1:0/2419236202 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff824111d90 0x7ff8241a1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:47:18.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.886+0000 7ff829200700 1 -- 172.21.15.1:0/2419236202 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff824073a00 msgr2=0x7ff82419da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:47:18.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.886+0000 7ff829200700 1 --2- 172.21.15.1:0/2419236202 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff824073a00 0x7ff82419da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:18.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.886+0000 7ff829200700 1 -- 172.21.15.1:0/2419236202 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff824111d90 msgr2=0x7ff8241a1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:47:18.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.886+0000 7ff829200700 1 --2- 172.21.15.1:0/2419236202 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff824111d90 0x7ff8241a1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:18.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.886+0000 7ff829200700 1 -- 172.21.15.1:0/2419236202 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8140095f0 con 0x7ff8240730e0 2024-02-02T15:47:18.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.886+0000 7ff823e00700 1 --2- 172.21.15.1:0/2419236202 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff824073a00 0x7ff82419da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:47:18.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.886+0000 7ff829200700 1 --2- 172.21.15.1:0/2419236202 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff8240730e0 0x7ff82419d4f0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7ff814005910 tx=0x7ff814004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:47:18.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.887+0000 7ff821600700 1 -- 172.21.15.1:0/2419236202 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81401d070 con 0x7ff8240730e0 2024-02-02T15:47:18.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.887+0000 7ff821600700 1 -- 172.21.15.1:0/2419236202 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff814022470 con 0x7ff8240730e0 2024-02-02T15:47:18.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.887+0000 7ff821600700 1 -- 172.21.15.1:0/2419236202 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff814017650 con 0x7ff8240730e0 2024-02-02T15:47:18.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.887+0000 7ff82bc00700 1 -- 172.21.15.1:0/2419236202 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8241a2a80 con 0x7ff8240730e0 2024-02-02T15:47:18.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.887+0000 7ff82bc00700 1 -- 172.21.15.1:0/2419236202 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8241a2d00 con 0x7ff8240730e0 2024-02-02T15:47:18.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.888+0000 7ff82bc00700 1 -- 172.21.15.1:0/2419236202 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff824104270 con 0x7ff8240730e0 2024-02-02T15:47:18.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.890+0000 7ff821600700 1 -- 172.21.15.1:0/2419236202 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7ff81400fad0 con 0x7ff8240730e0 2024-02-02T15:47:18.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.891+0000 7ff821600700 1 --2- 172.21.15.1:0/2419236202 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff8040a0710 0x7ff8040a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:47:18.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.891+0000 7ff821600700 1 -- 172.21.15.1:0/2419236202 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 9795+0+0 (secure 0 0 0) 0x7ff8140cd530 con 0x7ff8240730e0 2024-02-02T15:47:18.893 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.891+0000 7ff823e00700 1 --2- 172.21.15.1:0/2419236202 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff8040a0710 0x7ff8040a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:47:18.894 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.892+0000 7ff823e00700 1 --2- 172.21.15.1:0/2419236202 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff8040a0710 0x7ff8040a2bd0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff82419eb00 tx=0x7ff80c009260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:47:18.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:18.895+0000 7ff821600700 1 -- 172.21.15.1:0/2419236202 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7ff8140969e0 con 0x7ff8240730e0 2024-02-02T15:47:18.940 INFO:journalctl@ceph.mds.g.smithi001.stdout:Feb 02 15:47:18 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mds-g[84349]: ignoring --setuser ceph since I am not root 2024-02-02T15:47:18.940 INFO:journalctl@ceph.mds.g.smithi001.stdout:Feb 02 15:47:18 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mds-g[84349]: ignoring --setgroup ceph since I am not root 2024-02-02T15:47:18.940 INFO:journalctl@ceph.mds.g.smithi001.stdout:Feb 02 15:47:18 smithi001 ceph-mds[84372]: -- 172.21.15.1:0/376179574 <== mon.1 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55c2f507f1e0 con 0x55c2f5056800 2024-02-02T15:47:18.940 INFO:journalctl@ceph.mds.g.smithi001.stdout:Feb 02 15:47:18 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mds-g[84349]: starting mds.g at 2024-02-02T15:47:18.941 INFO:journalctl@ceph.mds.a.smithi001.stdout:Feb 02 15:47:18 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mds-a[84735]: ignoring --setuser ceph since I am not root 2024-02-02T15:47:18.941 INFO:journalctl@ceph.mds.a.smithi001.stdout:Feb 02 15:47:18 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mds-a[84735]: ignoring --setgroup ceph since I am not root 2024-02-02T15:47:18.941 INFO:journalctl@ceph.mds.a.smithi001.stdout:Feb 02 15:47:18 smithi001 ceph-mds[84758]: -- 172.21.15.1:0/2687803922 <== mon.1 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560fba7391e0 con 0x560fba711000 2024-02-02T15:47:18.941 INFO:journalctl@ceph.mds.a.smithi001.stdout:Feb 02 15:47:18 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mds-a[84735]: starting mds.a at 2024-02-02T15:47:18.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:47:18 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mon-a[35447]: 2024-02-02T15:47:18.633+0000 7fc29ce00700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-02T15:47:18.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:47:18 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/1803689724' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-02T15:47:18.942 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:47:18 smithi001 ceph-mon[35470]: pgmap v640: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 124 KiB/s wr, 16 op/s 2024-02-02T15:47:18.942 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:47:18 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/1017746641' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-02-02T15:47:18.942 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:47:18 smithi001 ceph-mon[35470]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-02-02T15:47:18.942 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:47:18 smithi001 ceph-mon[35470]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-02T15:47:18.942 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:47:18 smithi001 ceph-mon[35470]: osdmap e118: 12 total, 12 up, 12 in 2024-02-02T15:47:18.942 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:47:18 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/1017746641' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-02-02T15:47:18.942 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:47:18 smithi001 ceph-mon[35470]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-02-02T15:47:19.114 INFO:journalctl@ceph.mds.e.smithi071.stdout:Feb 02 15:47:18 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mds-e[78979]: ignoring --setuser ceph since I am not root 2024-02-02T15:47:19.114 INFO:journalctl@ceph.mds.e.smithi071.stdout:Feb 02 15:47:18 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mds-e[78979]: ignoring --setgroup ceph since I am not root 2024-02-02T15:47:19.114 INFO:journalctl@ceph.mds.e.smithi071.stdout:Feb 02 15:47:18 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mds-e[78979]: starting mds.e at 2024-02-02T15:47:19.115 INFO:journalctl@ceph.mds.b.smithi071.stdout:Feb 02 15:47:18 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mds-b[79660]: ignoring --setuser ceph since I am not root 2024-02-02T15:47:19.115 INFO:journalctl@ceph.mds.b.smithi071.stdout:Feb 02 15:47:18 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mds-b[79660]: ignoring --setgroup ceph since I am not root 2024-02-02T15:47:19.115 INFO:journalctl@ceph.mds.b.smithi071.stdout:Feb 02 15:47:18 smithi071 ceph-mds[79683]: -- 172.21.15.71:0/2552015453 <== mon.2 v2:172.21.15.71:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558cb4bad1e0 con 0x558cb4b83000 2024-02-02T15:47:19.115 INFO:journalctl@ceph.mds.b.smithi071.stdout:Feb 02 15:47:18 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mds-b[79660]: starting mds.b at 2024-02-02T15:47:19.116 INFO:journalctl@ceph.mds.h.smithi071.stdout:Feb 02 15:47:18 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mds-h[79320]: ignoring --setuser ceph since I am not root 2024-02-02T15:47:19.116 INFO:journalctl@ceph.mds.h.smithi071.stdout:Feb 02 15:47:18 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mds-h[79320]: ignoring --setgroup ceph since I am not root 2024-02-02T15:47:19.116 INFO:journalctl@ceph.mds.h.smithi071.stdout:Feb 02 15:47:18 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mds-h[79320]: starting mds.h at 2024-02-02T15:47:19.116 INFO:journalctl@ceph.mds.k.smithi071.stdout:Feb 02 15:47:18 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mds-k[78638]: ignoring --setuser ceph since I am not root 2024-02-02T15:47:19.117 INFO:journalctl@ceph.mds.k.smithi071.stdout:Feb 02 15:47:18 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mds-k[78638]: ignoring --setgroup ceph since I am not root 2024-02-02T15:47:19.117 INFO:journalctl@ceph.mds.k.smithi071.stdout:Feb 02 15:47:18 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mds-k[78638]: starting mds.k at 2024-02-02T15:47:19.117 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:47:18 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/1803689724' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-02T15:47:19.117 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:47:18 smithi071 ceph-mon[44707]: pgmap v640: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 124 KiB/s wr, 16 op/s 2024-02-02T15:47:19.117 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:47:18 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/1017746641' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-02-02T15:47:19.117 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:47:18 smithi071 ceph-mon[44707]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-02-02T15:47:19.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:47:18 smithi071 ceph-mon[44707]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-02T15:47:19.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:47:18 smithi071 ceph-mon[44707]: osdmap e118: 12 total, 12 up, 12 in 2024-02-02T15:47:19.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:47:18 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/1017746641' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-02-02T15:47:19.118 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:47:18 smithi071 ceph-mon[44707]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-02-02T15:47:19.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:19.153+0000 7ff82bc00700 1 -- 172.21.15.1:0/2419236202 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff8241a2c10 con 0x7ff8240730e0 2024-02-02T15:47:19.170 INFO:journalctl@ceph.mds.i.smithi142.stdout:Feb 02 15:47:18 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mds-i[83198]: ignoring --setuser ceph since I am not root 2024-02-02T15:47:19.170 INFO:journalctl@ceph.mds.i.smithi142.stdout:Feb 02 15:47:18 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mds-i[83198]: ignoring --setgroup ceph since I am not root 2024-02-02T15:47:19.170 INFO:journalctl@ceph.mds.i.smithi142.stdout:Feb 02 15:47:18 smithi142 ceph-mds[83221]: -- 172.21.15.142:0/3253847715 <== mon.2 v2:172.21.15.71:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559d092271e0 con 0x559d091fcc00 2024-02-02T15:47:19.170 INFO:journalctl@ceph.mds.i.smithi142.stdout:Feb 02 15:47:18 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mds-i[83198]: starting mds.i at 2024-02-02T15:47:19.171 INFO:journalctl@ceph.mds.l.smithi142.stdout:Feb 02 15:47:18 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mds-l[82172]: ignoring --setuser ceph since I am not root 2024-02-02T15:47:19.171 INFO:journalctl@ceph.mds.l.smithi142.stdout:Feb 02 15:47:18 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mds-l[82172]: ignoring --setgroup ceph since I am not root 2024-02-02T15:47:19.171 INFO:journalctl@ceph.mds.l.smithi142.stdout:Feb 02 15:47:18 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mds-l[82172]: starting mds.l at 2024-02-02T15:47:19.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:47:18 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/1803689724' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-02T15:47:19.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:47:18 smithi142 ceph-mon[45728]: pgmap v640: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 124 KiB/s wr, 16 op/s 2024-02-02T15:47:19.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:47:18 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/1017746641' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-02-02T15:47:19.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:47:18 smithi142 ceph-mon[45728]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-02-02T15:47:19.172 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:47:18 smithi142 ceph-mon[45728]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-02-02T15:47:19.172 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:47:18 smithi142 ceph-mon[45728]: osdmap e118: 12 total, 12 up, 12 in 2024-02-02T15:47:19.172 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:47:18 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/1017746641' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-02-02T15:47:19.172 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:47:18 smithi142 ceph-mon[45728]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-02-02T15:47:19.172 INFO:journalctl@ceph.mds.c.smithi142.stdout:Feb 02 15:47:18 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mds-c[82857]: ignoring --setuser ceph since I am not root 2024-02-02T15:47:19.173 INFO:journalctl@ceph.mds.c.smithi142.stdout:Feb 02 15:47:18 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mds-c[82857]: ignoring --setgroup ceph since I am not root 2024-02-02T15:47:19.173 INFO:journalctl@ceph.mds.c.smithi142.stdout:Feb 02 15:47:18 smithi142 ceph-mds[82880]: -- 172.21.15.142:0/115341298 <== mon.1 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b4b855f1e0 con 0x55b4b8534c00 2024-02-02T15:47:19.173 INFO:journalctl@ceph.mds.c.smithi142.stdout:Feb 02 15:47:18 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mds-c[82857]: starting mds.c at 2024-02-02T15:47:19.173 INFO:journalctl@ceph.mds.f.smithi142.stdout:Feb 02 15:47:18 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mds-f[82516]: ignoring --setuser ceph since I am not root 2024-02-02T15:47:19.173 INFO:journalctl@ceph.mds.f.smithi142.stdout:Feb 02 15:47:18 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mds-f[82516]: ignoring --setgroup ceph since I am not root 2024-02-02T15:47:19.173 INFO:journalctl@ceph.mds.f.smithi142.stdout:Feb 02 15:47:18 smithi142 ceph-mds[82539]: -- 172.21.15.142:0/472228276 <== mon.1 v2:172.21.15.142:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5594188131e0 con 0x5594187e9000 2024-02-02T15:47:19.173 INFO:journalctl@ceph.mds.f.smithi142.stdout:Feb 02 15:47:18 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mds-f[82516]: starting mds.f at 2024-02-02T15:47:19.645 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:19.645+0000 7ff821600700 1 -- 172.21.15.1:0/2419236202 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v85) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff81409a000 con 0x7ff8240730e0 2024-02-02T15:47:19.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:19.649+0000 7ff82bc00700 1 -- 172.21.15.1:0/2419236202 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff8040a0710 msgr2=0x7ff8040a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:47:19.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:19.649+0000 7ff82bc00700 1 --2- 172.21.15.1:0/2419236202 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff8040a0710 0x7ff8040a2bd0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff82419eb00 tx=0x7ff80c009260 comp rx=0 tx=0).stop 2024-02-02T15:47:19.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:19.649+0000 7ff82bc00700 1 -- 172.21.15.1:0/2419236202 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff8240730e0 msgr2=0x7ff82419d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:47:19.649 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:19.649+0000 7ff82bc00700 1 --2- 172.21.15.1:0/2419236202 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff8240730e0 0x7ff82419d4f0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7ff814005910 tx=0x7ff814004730 comp rx=0 tx=0).stop 2024-02-02T15:47:19.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:19.650+0000 7ff82bc00700 1 -- 172.21.15.1:0/2419236202 shutdown_connections 2024-02-02T15:47:19.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:19.650+0000 7ff82bc00700 1 --2- 172.21.15.1:0/2419236202 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7ff8040a0710 0x7ff8040a2bd0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:19.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:19.650+0000 7ff82bc00700 1 --2- 172.21.15.1:0/2419236202 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff824111d90 0x7ff8241a1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:19.650 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:19.650+0000 7ff82bc00700 1 --2- 172.21.15.1:0/2419236202 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7ff8240730e0 0x7ff82419d4f0 secure :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7ff814005910 tx=0x7ff814004730 comp rx=0 tx=0).stop 2024-02-02T15:47:19.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:19.650+0000 7ff82bc00700 1 --2- 172.21.15.1:0/2419236202 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7ff824073a00 0x7ff82419da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:19.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:19.650+0000 7ff82bc00700 1 -- 172.21.15.1:0/2419236202 >> 172.21.15.1:0/2419236202 conn(0x7ff8240fc810 msgr2=0x7ff8240fd480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:47:19.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:19.650+0000 7ff82bc00700 1 -- 172.21.15.1:0/2419236202 shutdown_connections 2024-02-02T15:47:19.651 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:19.650+0000 7ff82bc00700 1 -- 172.21.15.1:0/2419236202 wait complete. 2024-02-02T15:47:19.674 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-02-02T15:47:19.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:19.884+0000 7f97ade00700 1 -- 172.21.15.1:0/3802001990 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f97a80689b0 msgr2=0x7f97a810dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:47:19.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:19.884+0000 7f97ade00700 1 --2- 172.21.15.1:0/3802001990 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f97a80689b0 0x7f97a810dc20 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f97a000b0d0 tx=0x7f97a000b3e0 comp rx=0 tx=0).stop 2024-02-02T15:47:19.884 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:19.884+0000 7f97ade00700 1 -- 172.21.15.1:0/3802001990 shutdown_connections 2024-02-02T15:47:19.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:19.884+0000 7f97ade00700 1 --2- 172.21.15.1:0/3802001990 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f97a80689b0 0x7f97a810dc20 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:19.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:19.884+0000 7f97ade00700 1 --2- 172.21.15.1:0/3802001990 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97a8106ed0 0x7f97a81072b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:19.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:19.884+0000 7f97ade00700 1 --2- 172.21.15.1:0/3802001990 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97a8068010 0x7f97a8068470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:19.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:19.884+0000 7f97ade00700 1 -- 172.21.15.1:0/3802001990 >> 172.21.15.1:0/3802001990 conn(0x7f97a80757d0 msgr2=0x7f97a8075be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:47:19.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:19.884+0000 7f97ade00700 1 -- 172.21.15.1:0/3802001990 shutdown_connections 2024-02-02T15:47:19.885 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:19.884+0000 7f97ade00700 1 -- 172.21.15.1:0/3802001990 wait complete. 2024-02-02T15:47:19.887 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:19.885+0000 7f97ade00700 1 Processor -- start 2024-02-02T15:47:19.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:19.885+0000 7f97ade00700 1 -- start start 2024-02-02T15:47:19.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:19.885+0000 7f97ade00700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97a8068010 0x7f97a8199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:47:19.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:19.885+0000 7f97ade00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f97a80689b0 0x7f97a81997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:47:19.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:19.885+0000 7f97ade00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97a8106ed0 0x7f97a819dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:47:19.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:19.885+0000 7f97ade00700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a819e220 con 0x7f97a80689b0 2024-02-02T15:47:19.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:19.885+0000 7f97ade00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a819e390 con 0x7f97a8106ed0 2024-02-02T15:47:19.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:19.885+0000 7f97ade00700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a819e550 con 0x7f97a8068010 2024-02-02T15:47:19.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:19.886+0000 7f97a6000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f97a80689b0 0x7f97a81997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:47:19.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:19.886+0000 7f97a6000700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f97a80689b0 0x7f97a81997d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43524/0 (socket says 172.21.15.1:43524) 2024-02-02T15:47:19.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:19.886+0000 7f97a6000700 1 -- 172.21.15.1:0/3329506642 learned_addr learned my addr 172.21.15.1:0/3329506642 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:47:19.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:19.886+0000 7f97a7400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97a8106ed0 0x7f97a819dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:47:19.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:19.886+0000 7f97a6a00700 1 --2- 172.21.15.1:0/3329506642 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97a8068010 0x7f97a8199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:47:19.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:19.886+0000 7f97a6000700 1 -- 172.21.15.1:0/3329506642 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97a8068010 msgr2=0x7f97a8199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:47:19.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:19.886+0000 7f97a6000700 1 --2- 172.21.15.1:0/3329506642 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97a8068010 0x7f97a8199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:19.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:19.886+0000 7f97a6000700 1 -- 172.21.15.1:0/3329506642 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97a8106ed0 msgr2=0x7f97a819dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:47:19.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:19.886+0000 7f97a6000700 1 --2- 172.21.15.1:0/3329506642 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97a8106ed0 0x7f97a819dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:19.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:19.886+0000 7f97a6000700 1 -- 172.21.15.1:0/3329506642 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97a0009d00 con 0x7f97a80689b0 2024-02-02T15:47:19.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:19.886+0000 7f97a7400700 1 --2- 172.21.15.1:0/3329506642 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97a8106ed0 0x7f97a819dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:47:19.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:19.886+0000 7f97a6000700 1 --2- 172.21.15.1:0/3329506642 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f97a80689b0 0x7f97a81997d0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f979000e9a0 tx=0x7f979000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:47:19.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:19.886+0000 7f97aca00700 1 -- 172.21.15.1:0/3329506642 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9790009790 con 0x7f97a80689b0 2024-02-02T15:47:19.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:19.886+0000 7f97ade00700 1 -- 172.21.15.1:0/3329506642 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97a819e830 con 0x7f97a80689b0 2024-02-02T15:47:19.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:19.887+0000 7f97aca00700 1 -- 172.21.15.1:0/3329506642 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9790010430 con 0x7f97a80689b0 2024-02-02T15:47:19.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:19.887+0000 7f97ade00700 1 -- 172.21.15.1:0/3329506642 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97a81a6070 con 0x7f97a80689b0 2024-02-02T15:47:19.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:19.887+0000 7f97aca00700 1 -- 172.21.15.1:0/3329506642 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9790018620 con 0x7f97a80689b0 2024-02-02T15:47:19.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:19.888+0000 7f97ade00700 1 -- 172.21.15.1:0/3329506642 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97a8067bc0 con 0x7f97a80689b0 2024-02-02T15:47:19.891 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:19.890+0000 7f97aca00700 1 -- 172.21.15.1:0/3329506642 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f97900105a0 con 0x7f97a80689b0 2024-02-02T15:47:19.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:19.891+0000 7f97aca00700 1 --2- 172.21.15.1:0/3329506642 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f97880a0690 0x7f97880a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:47:19.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:19.892+0000 7f97aca00700 1 -- 172.21.15.1:0/3329506642 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 9795+0+0 (secure 0 0 0) 0x7f9790014070 con 0x7f97a80689b0 2024-02-02T15:47:19.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:19.892+0000 7f97a6a00700 1 --2- 172.21.15.1:0/3329506642 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f97880a0690 0x7f97880a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:47:19.892 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:19.892+0000 7f97a6a00700 1 --2- 172.21.15.1:0/3329506642 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f97880a0690 0x7f97880a2b50 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f9798009be0 tx=0x7f9798009260 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:47:19.895 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:19.895+0000 7f97aca00700 1 -- 172.21.15.1:0/3329506642 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f9790052110 con 0x7f97a80689b0 2024-02-02T15:47:19.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:47:19 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/2419236202' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-02-02T15:47:19.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:47:19 smithi001 ceph-mon[35470]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-02-02T15:47:19.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:47:19 smithi001 ceph-mon[35470]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-02-02T15:47:19.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:47:19 smithi001 ceph-mon[35470]: Cluster is now healthy 2024-02-02T15:47:19.940 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:47:19 smithi001 ceph-mon[35470]: mds.? [v2:172.21.15.1:6838/3476139961,v1:172.21.15.1:6839/3476139961] up:boot 2024-02-02T15:47:19.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:47:19 smithi001 ceph-mon[35470]: mds.? [v2:172.21.15.1:6840/939381803,v1:172.21.15.1:6841/939381803] up:boot 2024-02-02T15:47:19.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:47:19 smithi001 ceph-mon[35470]: mds.? [v2:172.21.15.71:6835/905058176,v1:172.21.15.71:6838/905058176] up:boot 2024-02-02T15:47:19.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:47:19 smithi001 ceph-mon[35470]: mds.? [v2:172.21.15.71:6834/2313133542,v1:172.21.15.71:6839/2313133542] up:boot 2024-02-02T15:47:19.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:47:19 smithi001 ceph-mon[35470]: mds.? [v2:172.21.15.71:6833/1628139120,v1:172.21.15.71:6837/1628139120] up:boot 2024-02-02T15:47:19.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:47:19 smithi001 ceph-mon[35470]: mds.? [v2:172.21.15.71:6832/2010154081,v1:172.21.15.71:6836/2010154081] up:boot 2024-02-02T15:47:19.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:47:19 smithi001 ceph-mon[35470]: mds.? [v2:172.21.15.142:6838/1188830659,v1:172.21.15.142:6839/1188830659] up:boot 2024-02-02T15:47:19.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:47:19 smithi001 ceph-mon[35470]: mds.? [v2:172.21.15.142:6834/2419081703,v1:172.21.15.142:6835/2419081703] up:boot 2024-02-02T15:47:19.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:47:19 smithi001 ceph-mon[35470]: mds.? [v2:172.21.15.142:6832/3242903098,v1:172.21.15.142:6833/3242903098] up:boot 2024-02-02T15:47:19.941 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:47:19 smithi001 ceph-mon[35470]: mds.? [v2:172.21.15.142:6836/2320424491,v1:172.21.15.142:6837/2320424491] up:boot 2024-02-02T15:47:19.942 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:47:19 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/2419236202' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-02-02T15:47:19.942 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:47:19 smithi001 ceph-mon[35470]: fsmap 12 up:standby 2024-02-02T15:47:19.942 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:47:19 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-02-02T15:47:19.942 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:47:19 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-02-02T15:47:19.942 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:47:19 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-02-02T15:47:19.942 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:47:19 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-02-02T15:47:19.942 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:47:19 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-02T15:47:19.942 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:47:19 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-02T15:47:19.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:47:19 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-02-02T15:47:19.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:47:19 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-02-02T15:47:19.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:47:19 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-02-02T15:47:19.943 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:47:19 smithi001 ceph-mon[35470]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-02-02T15:47:20.109 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:20.109+0000 7f97ade00700 1 -- 172.21.15.1:0/3329506642 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f97a81a6200 con 0x7f97a80689b0 2024-02-02T15:47:20.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:47:19 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/2419236202' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-02-02T15:47:20.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:47:19 smithi071 ceph-mon[44707]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-02-02T15:47:20.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:47:19 smithi071 ceph-mon[44707]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-02-02T15:47:20.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:47:19 smithi071 ceph-mon[44707]: Cluster is now healthy 2024-02-02T15:47:20.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:47:19 smithi071 ceph-mon[44707]: mds.? [v2:172.21.15.1:6838/3476139961,v1:172.21.15.1:6839/3476139961] up:boot 2024-02-02T15:47:20.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:47:19 smithi071 ceph-mon[44707]: mds.? [v2:172.21.15.1:6840/939381803,v1:172.21.15.1:6841/939381803] up:boot 2024-02-02T15:47:20.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:47:19 smithi071 ceph-mon[44707]: mds.? [v2:172.21.15.71:6835/905058176,v1:172.21.15.71:6838/905058176] up:boot 2024-02-02T15:47:20.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:47:19 smithi071 ceph-mon[44707]: mds.? [v2:172.21.15.71:6834/2313133542,v1:172.21.15.71:6839/2313133542] up:boot 2024-02-02T15:47:20.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:47:19 smithi071 ceph-mon[44707]: mds.? [v2:172.21.15.71:6833/1628139120,v1:172.21.15.71:6837/1628139120] up:boot 2024-02-02T15:47:20.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:47:19 smithi071 ceph-mon[44707]: mds.? [v2:172.21.15.71:6832/2010154081,v1:172.21.15.71:6836/2010154081] up:boot 2024-02-02T15:47:20.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:47:19 smithi071 ceph-mon[44707]: mds.? [v2:172.21.15.142:6838/1188830659,v1:172.21.15.142:6839/1188830659] up:boot 2024-02-02T15:47:20.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:47:19 smithi071 ceph-mon[44707]: mds.? [v2:172.21.15.142:6834/2419081703,v1:172.21.15.142:6835/2419081703] up:boot 2024-02-02T15:47:20.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:47:19 smithi071 ceph-mon[44707]: mds.? [v2:172.21.15.142:6832/3242903098,v1:172.21.15.142:6833/3242903098] up:boot 2024-02-02T15:47:20.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:47:19 smithi071 ceph-mon[44707]: mds.? [v2:172.21.15.142:6836/2320424491,v1:172.21.15.142:6837/2320424491] up:boot 2024-02-02T15:47:20.115 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:47:19 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/2419236202' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-02-02T15:47:20.116 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:47:19 smithi071 ceph-mon[44707]: fsmap 12 up:standby 2024-02-02T15:47:20.116 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:47:19 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-02-02T15:47:20.116 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:47:19 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-02-02T15:47:20.116 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:47:19 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-02-02T15:47:20.116 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:47:19 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-02-02T15:47:20.116 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:47:19 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-02T15:47:20.116 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:47:19 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-02T15:47:20.116 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:47:19 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-02-02T15:47:20.117 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:47:19 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-02-02T15:47:20.117 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:47:19 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-02-02T15:47:20.117 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:47:19 smithi071 ceph-mon[44707]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-02-02T15:47:20.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:47:19 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/2419236202' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-02-02T15:47:20.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:47:19 smithi142 ceph-mon[45728]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-02-02T15:47:20.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:47:19 smithi142 ceph-mon[45728]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-02-02T15:47:20.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:47:19 smithi142 ceph-mon[45728]: Cluster is now healthy 2024-02-02T15:47:20.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:47:19 smithi142 ceph-mon[45728]: mds.? [v2:172.21.15.1:6838/3476139961,v1:172.21.15.1:6839/3476139961] up:boot 2024-02-02T15:47:20.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:47:19 smithi142 ceph-mon[45728]: mds.? [v2:172.21.15.1:6840/939381803,v1:172.21.15.1:6841/939381803] up:boot 2024-02-02T15:47:20.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:47:19 smithi142 ceph-mon[45728]: mds.? [v2:172.21.15.71:6835/905058176,v1:172.21.15.71:6838/905058176] up:boot 2024-02-02T15:47:20.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:47:19 smithi142 ceph-mon[45728]: mds.? [v2:172.21.15.71:6834/2313133542,v1:172.21.15.71:6839/2313133542] up:boot 2024-02-02T15:47:20.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:47:19 smithi142 ceph-mon[45728]: mds.? [v2:172.21.15.71:6833/1628139120,v1:172.21.15.71:6837/1628139120] up:boot 2024-02-02T15:47:20.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:47:19 smithi142 ceph-mon[45728]: mds.? [v2:172.21.15.71:6832/2010154081,v1:172.21.15.71:6836/2010154081] up:boot 2024-02-02T15:47:20.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:47:19 smithi142 ceph-mon[45728]: mds.? [v2:172.21.15.142:6838/1188830659,v1:172.21.15.142:6839/1188830659] up:boot 2024-02-02T15:47:20.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:47:19 smithi142 ceph-mon[45728]: mds.? [v2:172.21.15.142:6834/2419081703,v1:172.21.15.142:6835/2419081703] up:boot 2024-02-02T15:47:20.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:47:19 smithi142 ceph-mon[45728]: mds.? [v2:172.21.15.142:6832/3242903098,v1:172.21.15.142:6833/3242903098] up:boot 2024-02-02T15:47:20.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:47:19 smithi142 ceph-mon[45728]: mds.? [v2:172.21.15.142:6836/2320424491,v1:172.21.15.142:6837/2320424491] up:boot 2024-02-02T15:47:20.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:47:19 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/2419236202' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-02-02T15:47:20.172 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:47:19 smithi142 ceph-mon[45728]: fsmap 12 up:standby 2024-02-02T15:47:20.172 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:47:19 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-02-02T15:47:20.172 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:47:19 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-02-02T15:47:20.172 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:47:19 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-02-02T15:47:20.172 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:47:19 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-02-02T15:47:20.172 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:47:19 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-02-02T15:47:20.172 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:47:19 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-02-02T15:47:20.172 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:47:19 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-02-02T15:47:20.173 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:47:19 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-02-02T15:47:20.173 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:47:19 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-02-02T15:47:20.173 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:47:19 smithi142 ceph-mon[45728]: from='mgr.25603 172.21.15.1:0/1430837344' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-02-02T15:47:20.863 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:20.863+0000 7f97aca00700 1 -- 172.21.15.1:0/3329506642 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v119) v1 ==== 181+0+0 (secure 0 0 0) 0x7f9790099200 con 0x7f97a80689b0 2024-02-02T15:47:20.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:20.868+0000 7f97ade00700 1 -- 172.21.15.1:0/3329506642 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f97880a0690 msgr2=0x7f97880a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:47:20.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:20.868+0000 7f97ade00700 1 --2- 172.21.15.1:0/3329506642 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f97880a0690 0x7f97880a2b50 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f9798009be0 tx=0x7f9798009260 comp rx=0 tx=0).stop 2024-02-02T15:47:20.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:20.868+0000 7f97ade00700 1 -- 172.21.15.1:0/3329506642 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f97a80689b0 msgr2=0x7f97a81997d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:47:20.868 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:20.868+0000 7f97ade00700 1 --2- 172.21.15.1:0/3329506642 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f97a80689b0 0x7f97a81997d0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f979000e9a0 tx=0x7f979000ecb0 comp rx=0 tx=0).stop 2024-02-02T15:47:20.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:20.869+0000 7f97ade00700 1 -- 172.21.15.1:0/3329506642 shutdown_connections 2024-02-02T15:47:20.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:20.869+0000 7f97ade00700 1 --2- 172.21.15.1:0/3329506642 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f97880a0690 0x7f97880a2b50 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:20.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:20.869+0000 7f97ade00700 1 --2- 172.21.15.1:0/3329506642 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f97a8106ed0 0x7f97a819dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:20.869 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:20.869+0000 7f97ade00700 1 --2- 172.21.15.1:0/3329506642 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f97a8068010 0x7f97a8199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:20.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:20.869+0000 7f97ade00700 1 --2- 172.21.15.1:0/3329506642 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f97a80689b0 0x7f97a81997d0 secure :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f979000e9a0 tx=0x7f979000ecb0 comp rx=0 tx=0).stop 2024-02-02T15:47:20.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:20.869+0000 7f97ade00700 1 -- 172.21.15.1:0/3329506642 >> 172.21.15.1:0/3329506642 conn(0x7f97a80757d0 msgr2=0x7f97a80feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:47:20.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:20.869+0000 7f97ade00700 1 -- 172.21.15.1:0/3329506642 shutdown_connections 2024-02-02T15:47:20.870 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:20.869+0000 7f97ade00700 1 -- 172.21.15.1:0/3329506642 wait complete. 2024-02-02T15:47:20.871 INFO:teuthology.orchestra.run.smithi001.stderr:pool 'cephfs_metadata' removed 2024-02-02T15:47:20.894 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-02-02T15:47:21.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:47:20 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/3329506642' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-02T15:47:21.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:47:20 smithi071 ceph-mon[44707]: pgmap v642: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 128 KiB/s wr, 13 op/s 2024-02-02T15:47:21.136 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:21.136+0000 7f0ba1600700 1 -- 172.21.15.1:0/1988546384 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b9c0ffa90 msgr2=0x7f0b9c0ffe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:47:21.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:21.136+0000 7f0ba1600700 1 --2- 172.21.15.1:0/1988546384 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b9c0ffa90 0x7f0b9c0ffe70 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c009990 tx=0x7f0b8c009ca0 comp rx=0 tx=0).stop 2024-02-02T15:47:21.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:21.136+0000 7f0ba1600700 1 -- 172.21.15.1:0/1988546384 shutdown_connections 2024-02-02T15:47:21.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:21.136+0000 7f0ba1600700 1 --2- 172.21.15.1:0/1988546384 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b9c104e90 0x7f0b9c105340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:21.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:21.136+0000 7f0ba1600700 1 --2- 172.21.15.1:0/1988546384 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b9c0ffa90 0x7f0b9c0ffe70 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:21.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:21.136+0000 7f0ba1600700 1 --2- 172.21.15.1:0/1988546384 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0b9c1003b0 0x7f0b9c104870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:21.137 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:21.136+0000 7f0ba1600700 1 -- 172.21.15.1:0/1988546384 >> 172.21.15.1:0/1988546384 conn(0x7f0b9c0fb480 msgr2=0x7f0b9c0fd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:47:21.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:21.137+0000 7f0ba1600700 1 -- 172.21.15.1:0/1988546384 shutdown_connections 2024-02-02T15:47:21.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:21.137+0000 7f0ba1600700 1 -- 172.21.15.1:0/1988546384 wait complete. 2024-02-02T15:47:21.138 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:21.138+0000 7f0ba1600700 1 Processor -- start 2024-02-02T15:47:21.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:21.138+0000 7f0ba1600700 1 -- start start 2024-02-02T15:47:21.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:21.139+0000 7f0ba1600700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0b9c0ffa90 0x7f0b9c19d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:47:21.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:21.139+0000 7f0ba1600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b9c1003b0 0x7f0b9c19d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:47:21.139 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:21.139+0000 7f0ba1600700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b9c104e90 0x7f0b9c1a1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:47:21.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:21.139+0000 7f0ba1600700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9c1a21c0 con 0x7f0b9c0ffa90 2024-02-02T15:47:21.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:21.139+0000 7f0ba1600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9c1a2330 con 0x7f0b9c1003b0 2024-02-02T15:47:21.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:21.139+0000 7f0ba1600700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9c1a24f0 con 0x7f0b9c104e90 2024-02-02T15:47:21.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:21.139+0000 7f0b9b400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b9c1003b0 0x7f0b9c19d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:47:21.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:21.139+0000 7f0b9be00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0b9c0ffa90 0x7f0b9c19d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:47:21.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:21.139+0000 7f0b9b400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b9c1003b0 0x7f0b9c19d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.1:57068/0 (socket says 172.21.15.1:57068) 2024-02-02T15:47:21.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:21.139+0000 7f0b9b400700 1 -- 172.21.15.1:0/4113015959 learned_addr learned my addr 172.21.15.1:0/4113015959 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:47:21.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:21.140+0000 7f0ba0c00700 1 --2- 172.21.15.1:0/4113015959 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b9c104e90 0x7f0b9c1a1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:47:21.141 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:21.140+0000 7f0b9be00700 1 -- 172.21.15.1:0/4113015959 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b9c104e90 msgr2=0x7f0b9c1a1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:47:21.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:21.140+0000 7f0b9be00700 1 --2- 172.21.15.1:0/4113015959 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b9c104e90 0x7f0b9c1a1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:21.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:21.140+0000 7f0b9be00700 1 -- 172.21.15.1:0/4113015959 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b9c1003b0 msgr2=0x7f0b9c19d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:47:21.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:21.140+0000 7f0b9be00700 1 --2- 172.21.15.1:0/4113015959 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b9c1003b0 0x7f0b9c19d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:21.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:21.140+0000 7f0b9be00700 1 -- 172.21.15.1:0/4113015959 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b8c0095f0 con 0x7f0b9c0ffa90 2024-02-02T15:47:21.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:21.140+0000 7f0b9b400700 1 --2- 172.21.15.1:0/4113015959 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b9c1003b0 0x7f0b9c19d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:47:21.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:21.140+0000 7f0b9be00700 1 --2- 172.21.15.1:0/4113015959 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0b9c0ffa90 0x7f0b9c19d1e0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c000c00 tx=0x7f0b8c005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:47:21.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:21.140+0000 7f0b98c00700 1 -- 172.21.15.1:0/4113015959 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b8c01d070 con 0x7f0b9c0ffa90 2024-02-02T15:47:21.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:21.140+0000 7f0ba1600700 1 -- 172.21.15.1:0/4113015959 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b9c1a2770 con 0x7f0b9c0ffa90 2024-02-02T15:47:21.142 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:21.140+0000 7f0b98c00700 1 -- 172.21.15.1:0/4113015959 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0b8c021410 con 0x7f0b9c0ffa90 2024-02-02T15:47:21.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:21.140+0000 7f0b98c00700 1 -- 172.21.15.1:0/4113015959 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b8c017650 con 0x7f0b9c0ffa90 2024-02-02T15:47:21.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:21.140+0000 7f0ba1600700 1 -- 172.21.15.1:0/4113015959 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b9c1aa010 con 0x7f0b9c0ffa90 2024-02-02T15:47:21.143 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:21.142+0000 7f0ba1600700 1 -- 172.21.15.1:0/4113015959 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b9c04ea40 con 0x7f0b9c0ffa90 2024-02-02T15:47:21.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:21.144+0000 7f0b98c00700 1 -- 172.21.15.1:0/4113015959 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f0b8c021580 con 0x7f0b9c0ffa90 2024-02-02T15:47:21.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:21.147+0000 7f0b98c00700 1 --2- 172.21.15.1:0/4113015959 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f0b7c0a0690 0x7f0b7c0a2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:47:21.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:21.147+0000 7f0b98c00700 1 -- 172.21.15.1:0/4113015959 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 9270+0+0 (secure 0 0 0) 0x7f0b8c0cdb30 con 0x7f0b9c0ffa90 2024-02-02T15:47:21.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:21.147+0000 7f0b9b400700 1 --2- 172.21.15.1:0/4113015959 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f0b7c0a0690 0x7f0b7c0a2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:47:21.147 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:21.148+0000 7f0b98c00700 1 -- 172.21.15.1:0/4113015959 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f0b8c0971f0 con 0x7f0b9c0ffa90 2024-02-02T15:47:21.148 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:21.148+0000 7f0b9b400700 1 --2- 172.21.15.1:0/4113015959 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f0b7c0a0690 0x7f0b7c0a2b50 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c19e7f0 tx=0x7f0b8400b450 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:47:21.169 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:47:20 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/3329506642' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-02T15:47:21.169 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:47:20 smithi142 ceph-mon[45728]: pgmap v642: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 128 KiB/s wr, 13 op/s 2024-02-02T15:47:21.189 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:47:20 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/3329506642' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-02T15:47:21.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:47:20 smithi001 ceph-mon[35470]: pgmap v642: 129 pgs: 129 active+clean; 2.3 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 128 KiB/s wr, 13 op/s 2024-02-02T15:47:21.362 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:21.362+0000 7f0ba1600700 1 -- 172.21.15.1:0/4113015959 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0b9c1aa1d0 con 0x7f0b9c0ffa90 2024-02-02T15:47:21.865 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:21.865+0000 7f0b98c00700 1 -- 172.21.15.1:0/4113015959 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v120) v1 ==== 169+0+0 (secure 0 0 0) 0x7f0b8c026020 con 0x7f0b9c0ffa90 2024-02-02T15:47:21.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:21.871+0000 7f0ba1600700 1 -- 172.21.15.1:0/4113015959 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f0b7c0a0690 msgr2=0x7f0b7c0a2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:47:21.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:21.871+0000 7f0ba1600700 1 --2- 172.21.15.1:0/4113015959 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f0b7c0a0690 0x7f0b7c0a2b50 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c19e7f0 tx=0x7f0b8400b450 comp rx=0 tx=0).stop 2024-02-02T15:47:21.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:21.871+0000 7f0ba1600700 1 -- 172.21.15.1:0/4113015959 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0b9c0ffa90 msgr2=0x7f0b9c19d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:47:21.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:21.871+0000 7f0ba1600700 1 --2- 172.21.15.1:0/4113015959 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0b9c0ffa90 0x7f0b9c19d1e0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c000c00 tx=0x7f0b8c005740 comp rx=0 tx=0).stop 2024-02-02T15:47:21.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:21.871+0000 7f0ba1600700 1 -- 172.21.15.1:0/4113015959 shutdown_connections 2024-02-02T15:47:21.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:21.871+0000 7f0ba1600700 1 --2- 172.21.15.1:0/4113015959 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f0b7c0a0690 0x7f0b7c0a2b50 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:21.871 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:21.871+0000 7f0ba1600700 1 --2- 172.21.15.1:0/4113015959 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f0b9c104e90 0x7f0b9c1a1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:21.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:21.871+0000 7f0ba1600700 1 --2- 172.21.15.1:0/4113015959 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f0b9c0ffa90 0x7f0b9c19d1e0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:21.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:21.871+0000 7f0ba1600700 1 --2- 172.21.15.1:0/4113015959 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0b9c1003b0 0x7f0b9c19d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:21.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:21.871+0000 7f0ba1600700 1 -- 172.21.15.1:0/4113015959 >> 172.21.15.1:0/4113015959 conn(0x7f0b9c0fb480 msgr2=0x7f0b9c102990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:47:21.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:21.871+0000 7f0ba1600700 1 -- 172.21.15.1:0/4113015959 shutdown_connections 2024-02-02T15:47:21.872 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:21.871+0000 7f0ba1600700 1 -- 172.21.15.1:0/4113015959 wait complete. 2024-02-02T15:47:21.877 INFO:teuthology.orchestra.run.smithi001.stderr:pool 'cephfs_data' removed 2024-02-02T15:47:21.902 DEBUG:teuthology.orchestra.run.smithi001:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-02-02T15:47:22.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:22.143+0000 7f6073400700 1 -- 172.21.15.1:0/2493251938 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f606c0ffe20 msgr2=0x7f606c100200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:47:22.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:22.143+0000 7f6073400700 1 --2- 172.21.15.1:0/2493251938 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f606c0ffe20 0x7f606c100200 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f605c009990 tx=0x7f605c009ca0 comp rx=0 tx=0).stop 2024-02-02T15:47:22.144 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:22.143+0000 7f6073400700 1 -- 172.21.15.1:0/2493251938 shutdown_connections 2024-02-02T15:47:22.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:22.143+0000 7f6073400700 1 --2- 172.21.15.1:0/2493251938 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f606c105220 0x7f606c1056d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:22.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:22.143+0000 7f6073400700 1 --2- 172.21.15.1:0/2493251938 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f606c0ffe20 0x7f606c100200 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:22.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:22.143+0000 7f6073400700 1 --2- 172.21.15.1:0/2493251938 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f606c100740 0x7f606c104c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:22.145 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:22.143+0000 7f6073400700 1 -- 172.21.15.1:0/2493251938 >> 172.21.15.1:0/2493251938 conn(0x7f606c076d70 msgr2=0x7f606c077180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:47:22.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:22.143+0000 7f6073400700 1 -- 172.21.15.1:0/2493251938 shutdown_connections 2024-02-02T15:47:22.146 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:22.143+0000 7f6073400700 1 -- 172.21.15.1:0/2493251938 wait complete. 2024-02-02T15:47:22.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:22.144+0000 7f6073400700 1 Processor -- start 2024-02-02T15:47:22.150 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:22.144+0000 7f6073400700 1 -- start start 2024-02-02T15:47:22.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:22.145+0000 7f6073400700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f606c0ffe20 0x7f606c19d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:47:22.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:22.145+0000 7f6073400700 1 --2- >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f606c100740 0x7f606c19da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:47:22.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:22.145+0000 7f6073400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f606c105220 0x7f606c1a1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:47:22.151 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:22.145+0000 7f6073400700 1 -- --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f606c1a2510 con 0x7f606c0ffe20 2024-02-02T15:47:22.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:22.145+0000 7f6073400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f606c1a2680 con 0x7f606c105220 2024-02-02T15:47:22.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:22.145+0000 7f6073400700 1 -- --> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f606c1a2840 con 0x7f606c100740 2024-02-02T15:47:22.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:22.145+0000 7f6070a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f606c0ffe20 0x7f606c19d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:47:22.152 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:22.145+0000 7f6070a00700 1 --2- >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f606c0ffe20 0x7f606c19d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.1:3300/0 says I am v2:172.21.15.1:43548/0 (socket says 172.21.15.1:43548) 2024-02-02T15:47:22.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:22.145+0000 7f6070a00700 1 -- 172.21.15.1:0/3037981485 learned_addr learned my addr 172.21.15.1:0/3037981485 (peer_addr_for_me v2:172.21.15.1:0/0) 2024-02-02T15:47:22.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:22.145+0000 7f6071400700 1 --2- 172.21.15.1:0/3037981485 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f606c105220 0x7f606c1a1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:47:22.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:22.145+0000 7f606be00700 1 --2- 172.21.15.1:0/3037981485 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f606c100740 0x7f606c19da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:47:22.153 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:22.146+0000 7f6070a00700 1 -- 172.21.15.1:0/3037981485 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f606c100740 msgr2=0x7f606c19da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:47:22.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:22.146+0000 7f6070a00700 1 --2- 172.21.15.1:0/3037981485 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f606c100740 0x7f606c19da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:22.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:22.146+0000 7f6070a00700 1 -- 172.21.15.1:0/3037981485 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f606c105220 msgr2=0x7f606c1a1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:47:22.154 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:22.146+0000 7f6070a00700 1 --2- 172.21.15.1:0/3037981485 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f606c105220 0x7f606c1a1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:22.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:22.146+0000 7f6070a00700 1 -- 172.21.15.1:0/3037981485 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f605c0095f0 con 0x7f606c0ffe20 2024-02-02T15:47:22.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:22.146+0000 7f6071400700 1 --2- 172.21.15.1:0/3037981485 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f606c105220 0x7f606c1a1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-02-02T15:47:22.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:22.146+0000 7f6070a00700 1 --2- 172.21.15.1:0/3037981485 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f606c0ffe20 0x7f606c19d530 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f605c004730 tx=0x7f605c004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:47:22.155 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:22.146+0000 7f6069600700 1 -- 172.21.15.1:0/3037981485 <== mon.0 v2:172.21.15.1:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f605c01d070 con 0x7f606c0ffe20 2024-02-02T15:47:22.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:22.146+0000 7f6073400700 1 -- 172.21.15.1:0/3037981485 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f606c1a2ac0 con 0x7f606c0ffe20 2024-02-02T15:47:22.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:22.146+0000 7f6069600700 1 -- 172.21.15.1:0/3037981485 <== mon.0 v2:172.21.15.1:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f605c022470 con 0x7f606c0ffe20 2024-02-02T15:47:22.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:22.146+0000 7f6069600700 1 -- 172.21.15.1:0/3037981485 <== mon.0 v2:172.21.15.1:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f605c017650 con 0x7f606c0ffe20 2024-02-02T15:47:22.156 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:22.147+0000 7f6073400700 1 -- 172.21.15.1:0/3037981485 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f606c1aa360 con 0x7f606c0ffe20 2024-02-02T15:47:22.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:22.148+0000 7f6073400700 1 -- 172.21.15.1:0/3037981485 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f606c067330 con 0x7f606c0ffe20 2024-02-02T15:47:22.157 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:22.150+0000 7f6069600700 1 -- 172.21.15.1:0/3037981485 <== mon.0 v2:172.21.15.1:3300/0 4 ==== mgrmap(e 28) v1 ==== 135200+0+0 (secure 0 0 0) 0x7f605c0225e0 con 0x7f606c0ffe20 2024-02-02T15:47:22.158 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:22.153+0000 7f6069600700 1 --2- 172.21.15.1:0/3037981485 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f604c0a0710 0x7f604c0a2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-02-02T15:47:22.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:22.153+0000 7f6069600700 1 -- 172.21.15.1:0/3037981485 <== mon.0 v2:172.21.15.1:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 8682+0+0 (secure 0 0 0) 0x7f605c0cd220 con 0x7f606c0ffe20 2024-02-02T15:47:22.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:22.154+0000 7f606be00700 1 --2- 172.21.15.1:0/3037981485 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f604c0a0710 0x7f604c0a2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-02-02T15:47:22.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:22.155+0000 7f606be00700 1 --2- 172.21.15.1:0/3037981485 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f604c0a0710 0x7f604c0a2bd0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f606c19eb40 tx=0x7f605400b410 comp rx=0 tx=0).ready entity=mgr.25603 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-02-02T15:47:22.159 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:22.156+0000 7f6069600700 1 -- 172.21.15.1:0/3037981485 <== mon.0 v2:172.21.15.1:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+180467 (secure 0 0 0) 0x7f605c096b90 con 0x7f606c0ffe20 2024-02-02T15:47:22.169 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:47:21 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/3329506642' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-02-02T15:47:22.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:47:21 smithi142 ceph-mon[45728]: osdmap e119: 12 total, 12 up, 12 in 2024-02-02T15:47:22.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:47:21 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/4113015959' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-02T15:47:22.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:47:21 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/3329506642' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-02-02T15:47:22.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:47:21 smithi001 ceph-mon[35470]: osdmap e119: 12 total, 12 up, 12 in 2024-02-02T15:47:22.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:47:21 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/4113015959' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-02T15:47:22.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:47:21 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/3329506642' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-02-02T15:47:22.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:47:21 smithi071 ceph-mon[44707]: osdmap e119: 12 total, 12 up, 12 in 2024-02-02T15:47:22.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:47:21 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/4113015959' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-02T15:47:22.390 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:22.391+0000 7f6073400700 1 -- 172.21.15.1:0/3037981485 --> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f606c19e880 con 0x7f606c0ffe20 2024-02-02T15:47:22.883 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:22.883+0000 7f6069600700 1 -- 172.21.15.1:0/3037981485 <== mon.0 v2:172.21.15.1:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v121) v1 ==== 178+0+0 (secure 0 0 0) 0x7f605c09a1b0 con 0x7f606c0ffe20 2024-02-02T15:47:22.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:22.888+0000 7f6073400700 1 -- 172.21.15.1:0/3037981485 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f604c0a0710 msgr2=0x7f604c0a2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:47:22.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:22.888+0000 7f6073400700 1 --2- 172.21.15.1:0/3037981485 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f604c0a0710 0x7f604c0a2bd0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f606c19eb40 tx=0x7f605400b410 comp rx=0 tx=0).stop 2024-02-02T15:47:22.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:22.888+0000 7f6073400700 1 -- 172.21.15.1:0/3037981485 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f606c0ffe20 msgr2=0x7f606c19d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-02T15:47:22.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:22.888+0000 7f6073400700 1 --2- 172.21.15.1:0/3037981485 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f606c0ffe20 0x7f606c19d530 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f605c004730 tx=0x7f605c004760 comp rx=0 tx=0).stop 2024-02-02T15:47:22.888 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:22.888+0000 7f6073400700 1 -- 172.21.15.1:0/3037981485 shutdown_connections 2024-02-02T15:47:22.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:22.888+0000 7f6073400700 1 --2- 172.21.15.1:0/3037981485 >> [v2:172.21.15.1:6800/167617173,v1:172.21.15.1:6801/167617173] conn(0x7f604c0a0710 0x7f604c0a2bd0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:22.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:22.888+0000 7f6073400700 1 --2- 172.21.15.1:0/3037981485 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f606c105220 0x7f606c1a1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:22.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:22.888+0000 7f6073400700 1 --2- 172.21.15.1:0/3037981485 >> [v2:172.21.15.1:3300/0,v1:172.21.15.1:6789/0] conn(0x7f606c0ffe20 0x7f606c19d530 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:22.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:22.888+0000 7f6073400700 1 --2- 172.21.15.1:0/3037981485 >> [v2:172.21.15.142:3300/0,v1:172.21.15.142:6789/0] conn(0x7f606c100740 0x7f606c19da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-02-02T15:47:22.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:22.888+0000 7f6073400700 1 -- 172.21.15.1:0/3037981485 >> 172.21.15.1:0/3037981485 conn(0x7f606c076d70 msgr2=0x7f606c102d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-02T15:47:22.889 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:22.888+0000 7f6073400700 1 -- 172.21.15.1:0/3037981485 shutdown_connections 2024-02-02T15:47:22.890 INFO:teuthology.orchestra.run.smithi001.stderr:2024-02-02T15:47:22.889+0000 7f6073400700 1 -- 172.21.15.1:0/3037981485 wait complete. 2024-02-02T15:47:22.893 INFO:teuthology.orchestra.run.smithi001.stderr:pool 'cephfs_data_ec' removed 2024-02-02T15:47:22.927 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-02T15:47:23.053 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-02T15:47:23.129 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-02T15:47:23.170 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:47:22 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/4113015959' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-02-02T15:47:23.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:47:22 smithi142 ceph-mon[45728]: osdmap e120: 12 total, 12 up, 12 in 2024-02-02T15:47:23.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:47:22 smithi142 ceph-mon[45728]: from='client.? 172.21.15.1:0/3037981485' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-02T15:47:23.171 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:47:22 smithi142 ceph-mon[45728]: pgmap v645: 33 pgs: 33 active+clean; 318 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail 2024-02-02T15:47:23.183 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-02-02T15:47:23.183 DEBUG:teuthology.orchestra.run.smithi001:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-02T15:47:23.190 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:47:22 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/4113015959' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-02-02T15:47:23.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:47:22 smithi001 ceph-mon[35470]: osdmap e120: 12 total, 12 up, 12 in 2024-02-02T15:47:23.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:47:22 smithi001 ceph-mon[35470]: from='client.? 172.21.15.1:0/3037981485' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-02T15:47:23.191 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:47:22 smithi001 ceph-mon[35470]: pgmap v645: 33 pgs: 33 active+clean; 318 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail 2024-02-02T15:47:23.240 DEBUG:teuthology.orchestra.run.smithi071:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-02T15:47:23.282 DEBUG:teuthology.orchestra.run.smithi142:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-02T15:47:23.301 INFO:tasks.cephadm:Stopping all daemons... 2024-02-02T15:47:23.301 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-02-02T15:47:23.301 DEBUG:teuthology.orchestra.run.smithi001:> sudo systemctl stop ceph-371a409c-c1de-11ee-95b6-87774f69a715@mon.a 2024-02-02T15:47:23.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:47:22 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/4113015959' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-02-02T15:47:23.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:47:22 smithi071 ceph-mon[44707]: osdmap e120: 12 total, 12 up, 12 in 2024-02-02T15:47:23.364 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:47:22 smithi071 ceph-mon[44707]: from='client.? 172.21.15.1:0/3037981485' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-02T15:47:23.365 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:47:22 smithi071 ceph-mon[44707]: pgmap v645: 33 pgs: 33 active+clean; 318 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail 2024-02-02T15:47:23.690 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:47:23 smithi001 systemd[1]: Stopping Ceph mon.a for 371a409c-c1de-11ee-95b6-87774f69a715... 2024-02-02T15:47:24.105 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:47:23 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mon-a[35447]: 2024-02-02T15:47:23.736+0000 7fc2a3c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-02T15:47:24.105 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:47:23 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mon-a[35447]: 2024-02-02T15:47:23.736+0000 7fc2a3c00700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-02-02T15:47:24.441 INFO:journalctl@ceph.mon.a.smithi001.stdout:Feb 02 15:47:24 smithi001 bash[131027]: ceph-371a409c-c1de-11ee-95b6-87774f69a715-mon-a 2024-02-02T15:47:24.608 DEBUG:teuthology.orchestra.run.smithi001:> sudo pkill -f 'journalctl -f -n 0 -u ceph-371a409c-c1de-11ee-95b6-87774f69a715@mon.a.service' 2024-02-02T15:47:24.709 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-02T15:47:24.710 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-02-02T15:47:24.710 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-02-02T15:47:24.710 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-371a409c-c1de-11ee-95b6-87774f69a715@mon.b 2024-02-02T15:47:25.114 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:47:24 smithi071 systemd[1]: Stopping Ceph mon.b for 371a409c-c1de-11ee-95b6-87774f69a715... 2024-02-02T15:47:26.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:47:26 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mon-b[44684]: 2024-02-02T15:47:26.132+0000 7f8947fe9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-02T15:47:26.614 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:47:26 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mon-b[44684]: 2024-02-02T15:47:26.132+0000 7f8947fe9700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-02-02T15:47:27.767 INFO:journalctl@ceph.mon.b.smithi071.stdout:Feb 02 15:47:27 smithi071 bash[88772]: ceph-371a409c-c1de-11ee-95b6-87774f69a715-mon-b 2024-02-02T15:47:27.774 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-371a409c-c1de-11ee-95b6-87774f69a715@mon.b.service' 2024-02-02T15:47:27.822 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-02T15:47:27.822 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-02-02T15:47:27.822 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-02-02T15:47:27.822 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-371a409c-c1de-11ee-95b6-87774f69a715@mon.c 2024-02-02T15:47:28.112 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:47:27 smithi142 systemd[1]: Stopping Ceph mon.c for 371a409c-c1de-11ee-95b6-87774f69a715... 2024-02-02T15:47:28.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:47:28 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mon-c[45705]: 2024-02-02T15:47:28.110+0000 7f132799a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-02T15:47:28.420 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:47:28 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mon-c[45705]: 2024-02-02T15:47:28.110+0000 7f132799a700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-02-02T15:47:28.818 INFO:journalctl@ceph.mon.c.smithi142.stdout:Feb 02 15:47:28 smithi142 bash[91373]: ceph-371a409c-c1de-11ee-95b6-87774f69a715-mon-c 2024-02-02T15:47:28.825 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-371a409c-c1de-11ee-95b6-87774f69a715@mon.c.service' 2024-02-02T15:47:28.874 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-02T15:47:28.874 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-02-02T15:47:28.874 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-02-02T15:47:28.874 DEBUG:teuthology.orchestra.run.smithi001:> sudo systemctl stop ceph-371a409c-c1de-11ee-95b6-87774f69a715@mgr.x 2024-02-02T15:47:29.190 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:47:28 smithi001 systemd[1]: Stopping Ceph mgr.x for 371a409c-c1de-11ee-95b6-87774f69a715... 2024-02-02T15:47:29.584 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:47:29 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x[35723]: 2024-02-02T15:47:29.253+0000 7f1a24000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-02T15:47:29.940 INFO:journalctl@ceph.mgr.x.smithi001.stdout:Feb 02 15:47:29 smithi001 bash[131527]: ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-x 2024-02-02T15:47:30.009 DEBUG:teuthology.orchestra.run.smithi001:> sudo pkill -f 'journalctl -f -n 0 -u ceph-371a409c-c1de-11ee-95b6-87774f69a715@mgr.x.service' 2024-02-02T15:47:30.095 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-02T15:47:30.095 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-02-02T15:47:30.096 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-02-02T15:47:30.099 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-371a409c-c1de-11ee-95b6-87774f69a715@mgr.y 2024-02-02T15:47:30.548 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Feb 02 15:47:30 smithi071 systemd[1]: Stopping Ceph mgr.y for 371a409c-c1de-11ee-95b6-87774f69a715... 2024-02-02T15:47:30.864 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Feb 02 15:47:30 smithi071 bash[88913]: ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-y 2024-02-02T15:47:30.864 INFO:journalctl@ceph.mgr.y.smithi071.stdout:Feb 02 15:47:30 smithi071 systemd[1]: ceph-371a409c-c1de-11ee-95b6-87774f69a715@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-02-02T15:47:30.873 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-371a409c-c1de-11ee-95b6-87774f69a715@mgr.y.service' 2024-02-02T15:47:30.909 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-02T15:47:30.909 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-02-02T15:47:30.910 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-02-02T15:47:30.910 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-371a409c-c1de-11ee-95b6-87774f69a715@mgr.z 2024-02-02T15:47:31.275 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Feb 02 15:47:30 smithi142 systemd[1]: Stopping Ceph mgr.z for 371a409c-c1de-11ee-95b6-87774f69a715... 2024-02-02T15:47:31.585 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Feb 02 15:47:31 smithi142 bash[91514]: ceph-371a409c-c1de-11ee-95b6-87774f69a715-mgr-z 2024-02-02T15:47:31.586 INFO:journalctl@ceph.mgr.z.smithi142.stdout:Feb 02 15:47:31 smithi142 systemd[1]: ceph-371a409c-c1de-11ee-95b6-87774f69a715@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-02-02T15:47:31.592 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-371a409c-c1de-11ee-95b6-87774f69a715@mgr.z.service' 2024-02-02T15:47:31.628 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-02T15:47:31.628 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-02-02T15:47:31.628 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-02-02T15:47:31.628 DEBUG:teuthology.orchestra.run.smithi001:> sudo systemctl stop ceph-371a409c-c1de-11ee-95b6-87774f69a715@osd.0 2024-02-02T15:47:32.040 INFO:journalctl@ceph.osd.0.smithi001.stdout:Feb 02 15:47:31 smithi001 systemd[1]: Stopping Ceph osd.0 for 371a409c-c1de-11ee-95b6-87774f69a715... 2024-02-02T15:47:32.440 INFO:journalctl@ceph.osd.0.smithi001.stdout:Feb 02 15:47:32 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-0[52951]: 2024-02-02T15:47:32.039+0000 7f1174000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-02T15:47:32.440 INFO:journalctl@ceph.osd.0.smithi001.stdout:Feb 02 15:47:32 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-0[52951]: 2024-02-02T15:47:32.039+0000 7f1174000700 -1 osd.0 121 *** Got signal Terminated *** 2024-02-02T15:47:32.440 INFO:journalctl@ceph.osd.0.smithi001.stdout:Feb 02 15:47:32 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-0[52951]: 2024-02-02T15:47:32.039+0000 7f1174000700 -1 osd.0 121 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-02T15:47:38.190 INFO:journalctl@ceph.osd.0.smithi001.stdout:Feb 02 15:47:37 smithi001 bash[131667]: ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-0 2024-02-02T15:47:39.647 DEBUG:teuthology.orchestra.run.smithi001:> sudo pkill -f 'journalctl -f -n 0 -u ceph-371a409c-c1de-11ee-95b6-87774f69a715@osd.0.service' 2024-02-02T15:47:39.732 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-02T15:47:39.732 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-02-02T15:47:39.733 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-02-02T15:47:39.733 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-371a409c-c1de-11ee-95b6-87774f69a715@osd.1 2024-02-02T15:47:40.114 INFO:journalctl@ceph.osd.1.smithi071.stdout:Feb 02 15:47:39 smithi071 systemd[1]: Stopping Ceph osd.1 for 371a409c-c1de-11ee-95b6-87774f69a715... 2024-02-02T15:47:40.115 INFO:journalctl@ceph.osd.1.smithi071.stdout:Feb 02 15:47:39 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-1[49653]: 2024-02-02T15:47:39.973+0000 7fb5f1232700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-02T15:47:40.115 INFO:journalctl@ceph.osd.1.smithi071.stdout:Feb 02 15:47:39 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-1[49653]: 2024-02-02T15:47:39.973+0000 7fb5f1232700 -1 osd.1 121 *** Got signal Terminated *** 2024-02-02T15:47:40.115 INFO:journalctl@ceph.osd.1.smithi071.stdout:Feb 02 15:47:39 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-1[49653]: 2024-02-02T15:47:39.974+0000 7fb5f1232700 -1 osd.1 121 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-02T15:47:46.114 INFO:journalctl@ceph.osd.1.smithi071.stdout:Feb 02 15:47:45 smithi071 bash[89058]: ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-1 2024-02-02T15:47:47.086 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-371a409c-c1de-11ee-95b6-87774f69a715@osd.1.service' 2024-02-02T15:47:47.127 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-02T15:47:47.127 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-02-02T15:47:47.128 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-02-02T15:47:47.128 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-371a409c-c1de-11ee-95b6-87774f69a715@osd.2 2024-02-02T15:47:47.420 INFO:journalctl@ceph.osd.2.smithi142.stdout:Feb 02 15:47:47 smithi142 systemd[1]: Stopping Ceph osd.2 for 371a409c-c1de-11ee-95b6-87774f69a715... 2024-02-02T15:47:47.420 INFO:journalctl@ceph.osd.2.smithi142.stdout:Feb 02 15:47:47 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-2[51513]: 2024-02-02T15:47:47.368+0000 7f3d30211700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-02T15:47:47.420 INFO:journalctl@ceph.osd.2.smithi142.stdout:Feb 02 15:47:47 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-2[51513]: 2024-02-02T15:47:47.368+0000 7f3d30211700 -1 osd.2 121 *** Got signal Terminated *** 2024-02-02T15:47:47.420 INFO:journalctl@ceph.osd.2.smithi142.stdout:Feb 02 15:47:47 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-2[51513]: 2024-02-02T15:47:47.368+0000 7f3d30211700 -1 osd.2 121 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-02T15:47:53.420 INFO:journalctl@ceph.osd.2.smithi142.stdout:Feb 02 15:47:53 smithi142 bash[91659]: ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-2 2024-02-02T15:47:54.504 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-371a409c-c1de-11ee-95b6-87774f69a715@osd.2.service' 2024-02-02T15:47:54.545 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-02T15:47:54.546 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-02-02T15:47:54.546 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-02-02T15:47:54.546 DEBUG:teuthology.orchestra.run.smithi001:> sudo systemctl stop ceph-371a409c-c1de-11ee-95b6-87774f69a715@osd.3 2024-02-02T15:47:54.917 INFO:journalctl@ceph.osd.3.smithi001.stdout:Feb 02 15:47:54 smithi001 systemd[1]: Stopping Ceph osd.3 for 371a409c-c1de-11ee-95b6-87774f69a715... 2024-02-02T15:47:55.190 INFO:journalctl@ceph.osd.3.smithi001.stdout:Feb 02 15:47:54 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-3[61288]: 2024-02-02T15:47:54.916+0000 7f53be000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-02T15:47:55.190 INFO:journalctl@ceph.osd.3.smithi001.stdout:Feb 02 15:47:54 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-3[61288]: 2024-02-02T15:47:54.916+0000 7f53be000700 -1 osd.3 121 *** Got signal Terminated *** 2024-02-02T15:47:55.191 INFO:journalctl@ceph.osd.3.smithi001.stdout:Feb 02 15:47:54 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-3[61288]: 2024-02-02T15:47:54.916+0000 7f53be000700 -1 osd.3 121 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-02T15:47:58.190 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:47:57 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:47:57.888+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.102474+0000 front 2024-02-02T15:47:33.102427+0000 (oldest deadline 2024-02-02T15:47:57.802602+0000) 2024-02-02T15:47:59.170 INFO:journalctl@ceph.osd.5.smithi142.stdout:Feb 02 15:47:58 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-5[59288]: 2024-02-02T15:47:58.838+0000 7f761ec92700 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:36.187466+0000 front 2024-02-02T15:47:36.187507+0000 (oldest deadline 2024-02-02T15:47:58.487340+0000) 2024-02-02T15:47:59.190 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:47:58 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:47:58.929+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.102474+0000 front 2024-02-02T15:47:33.102427+0000 (oldest deadline 2024-02-02T15:47:57.802602+0000) 2024-02-02T15:48:00.170 INFO:journalctl@ceph.osd.5.smithi142.stdout:Feb 02 15:47:59 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-5[59288]: 2024-02-02T15:47:59.805+0000 7f761ec92700 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:36.187466+0000 front 2024-02-02T15:47:36.187507+0000 (oldest deadline 2024-02-02T15:47:58.487340+0000) 2024-02-02T15:48:00.190 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:47:59 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:47:59.936+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.102474+0000 front 2024-02-02T15:47:33.102427+0000 (oldest deadline 2024-02-02T15:47:57.802602+0000) 2024-02-02T15:48:00.940 INFO:journalctl@ceph.osd.3.smithi001.stdout:Feb 02 15:48:00 smithi001 bash[131928]: ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-3 2024-02-02T15:48:01.170 INFO:journalctl@ceph.osd.5.smithi142.stdout:Feb 02 15:48:00 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-5[59288]: 2024-02-02T15:48:00.802+0000 7f761ec92700 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:36.187466+0000 front 2024-02-02T15:47:36.187507+0000 (oldest deadline 2024-02-02T15:47:58.487340+0000) 2024-02-02T15:48:01.364 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:00 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:00.896+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.558440+0000 front 2024-02-02T15:47:33.558460+0000 (oldest deadline 2024-02-02T15:47:59.958896+0000) 2024-02-02T15:48:01.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:00 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:00.958+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.102474+0000 front 2024-02-02T15:47:33.102427+0000 (oldest deadline 2024-02-02T15:47:57.802602+0000) 2024-02-02T15:48:02.078 INFO:journalctl@ceph.osd.5.smithi142.stdout:Feb 02 15:48:01 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-5[59288]: 2024-02-02T15:48:01.813+0000 7f761ec92700 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:36.187466+0000 front 2024-02-02T15:47:36.187507+0000 (oldest deadline 2024-02-02T15:47:58.487340+0000) 2024-02-02T15:48:02.335 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:01 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:01.993+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.102474+0000 front 2024-02-02T15:47:33.102427+0000 (oldest deadline 2024-02-02T15:47:57.802602+0000) 2024-02-02T15:48:02.362 DEBUG:teuthology.orchestra.run.smithi001:> sudo pkill -f 'journalctl -f -n 0 -u ceph-371a409c-c1de-11ee-95b6-87774f69a715@osd.3.service' 2024-02-02T15:48:02.364 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:01 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:01.919+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.558440+0000 front 2024-02-02T15:47:33.558460+0000 (oldest deadline 2024-02-02T15:47:59.958896+0000) 2024-02-02T15:48:02.420 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:02 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:02.079+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:35.342656+0000 front 2024-02-02T15:47:35.342649+0000 (oldest deadline 2024-02-02T15:48:01.143219+0000) 2024-02-02T15:48:02.449 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-02T15:48:02.449 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-02-02T15:48:02.450 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-02-02T15:48:02.450 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-371a409c-c1de-11ee-95b6-87774f69a715@osd.4 2024-02-02T15:48:02.864 INFO:journalctl@ceph.osd.4.smithi071.stdout:Feb 02 15:48:02 smithi071 systemd[1]: Stopping Ceph osd.4 for 371a409c-c1de-11ee-95b6-87774f69a715... 2024-02-02T15:48:02.864 INFO:journalctl@ceph.osd.4.smithi071.stdout:Feb 02 15:48:02 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-4[57502]: 2024-02-02T15:48:02.726+0000 7f6371e70700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-02T15:48:02.865 INFO:journalctl@ceph.osd.4.smithi071.stdout:Feb 02 15:48:02 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-4[57502]: 2024-02-02T15:48:02.726+0000 7f6371e70700 -1 osd.4 121 *** Got signal Terminated *** 2024-02-02T15:48:02.865 INFO:journalctl@ceph.osd.4.smithi071.stdout:Feb 02 15:48:02 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-4[57502]: 2024-02-02T15:48:02.726+0000 7f6371e70700 -1 osd.4 121 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-02T15:48:03.170 INFO:journalctl@ceph.osd.5.smithi142.stdout:Feb 02 15:48:02 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-5[59288]: 2024-02-02T15:48:02.838+0000 7f761ec92700 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:36.187466+0000 front 2024-02-02T15:47:36.187507+0000 (oldest deadline 2024-02-02T15:47:58.487340+0000) 2024-02-02T15:48:03.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:03 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:03.052+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:35.342656+0000 front 2024-02-02T15:47:35.342649+0000 (oldest deadline 2024-02-02T15:48:01.143219+0000) 2024-02-02T15:48:03.364 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:02 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:02.895+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.558440+0000 front 2024-02-02T15:47:33.558460+0000 (oldest deadline 2024-02-02T15:47:59.958896+0000) 2024-02-02T15:48:03.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:02 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:02.990+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.102474+0000 front 2024-02-02T15:47:33.102427+0000 (oldest deadline 2024-02-02T15:47:57.802602+0000) 2024-02-02T15:48:04.170 INFO:journalctl@ceph.osd.5.smithi142.stdout:Feb 02 15:48:03 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-5[59288]: 2024-02-02T15:48:03.792+0000 7f761ec92700 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:36.187466+0000 front 2024-02-02T15:47:36.187507+0000 (oldest deadline 2024-02-02T15:47:58.487340+0000) 2024-02-02T15:48:04.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:04 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:04.006+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:35.342656+0000 front 2024-02-02T15:47:35.342649+0000 (oldest deadline 2024-02-02T15:48:01.143219+0000) 2024-02-02T15:48:04.364 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:03 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:03.920+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.558440+0000 front 2024-02-02T15:47:33.558460+0000 (oldest deadline 2024-02-02T15:47:59.958896+0000) 2024-02-02T15:48:04.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:04 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:04.016+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.102474+0000 front 2024-02-02T15:47:33.102427+0000 (oldest deadline 2024-02-02T15:47:57.802602+0000) 2024-02-02T15:48:05.170 INFO:journalctl@ceph.osd.5.smithi142.stdout:Feb 02 15:48:04 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-5[59288]: 2024-02-02T15:48:04.794+0000 7f761ec92700 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:36.187466+0000 front 2024-02-02T15:47:36.187507+0000 (oldest deadline 2024-02-02T15:47:58.487340+0000) 2024-02-02T15:48:05.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:05 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:05.024+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:35.342656+0000 front 2024-02-02T15:47:35.342649+0000 (oldest deadline 2024-02-02T15:48:01.143219+0000) 2024-02-02T15:48:05.364 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:04 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:04.884+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.558440+0000 front 2024-02-02T15:47:33.558460+0000 (oldest deadline 2024-02-02T15:47:59.958896+0000) 2024-02-02T15:48:05.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:05 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:05.042+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.102474+0000 front 2024-02-02T15:47:33.102427+0000 (oldest deadline 2024-02-02T15:47:57.802602+0000) 2024-02-02T15:48:06.170 INFO:journalctl@ceph.osd.5.smithi142.stdout:Feb 02 15:48:05 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-5[59288]: 2024-02-02T15:48:05.828+0000 7f761ec92700 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:36.187466+0000 front 2024-02-02T15:47:36.187507+0000 (oldest deadline 2024-02-02T15:47:58.487340+0000) 2024-02-02T15:48:06.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:06 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:06.055+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:35.342656+0000 front 2024-02-02T15:47:35.342649+0000 (oldest deadline 2024-02-02T15:48:01.143219+0000) 2024-02-02T15:48:06.364 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:05 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:05.869+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.558440+0000 front 2024-02-02T15:47:33.558460+0000 (oldest deadline 2024-02-02T15:47:59.958896+0000) 2024-02-02T15:48:06.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:05 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:05.998+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.102474+0000 front 2024-02-02T15:47:33.102427+0000 (oldest deadline 2024-02-02T15:47:57.802602+0000) 2024-02-02T15:48:06.839 INFO:journalctl@ceph.osd.7.smithi071.stdout:Feb 02 15:48:06 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-7[65443]: 2024-02-02T15:48:06.569+0000 7f66be2a3700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:40.134496+0000 front 2024-02-02T15:47:40.134524+0000 (oldest deadline 2024-02-02T15:48:06.034624+0000) 2024-02-02T15:48:07.114 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:06 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:06.839+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.558440+0000 front 2024-02-02T15:47:33.558460+0000 (oldest deadline 2024-02-02T15:47:59.958896+0000) 2024-02-02T15:48:07.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:07 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:07.012+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:35.342656+0000 front 2024-02-02T15:47:35.342649+0000 (oldest deadline 2024-02-02T15:48:01.143219+0000) 2024-02-02T15:48:07.170 INFO:journalctl@ceph.osd.5.smithi142.stdout:Feb 02 15:48:06 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-5[59288]: 2024-02-02T15:48:06.805+0000 7f761ec92700 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:36.187466+0000 front 2024-02-02T15:47:36.187507+0000 (oldest deadline 2024-02-02T15:47:58.487340+0000) 2024-02-02T15:48:07.170 INFO:journalctl@ceph.osd.5.smithi142.stdout:Feb 02 15:48:06 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-5[59288]: 2024-02-02T15:48:06.805+0000 7f761ec92700 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:41.988425+0000 front 2024-02-02T15:47:41.988334+0000 (oldest deadline 2024-02-02T15:48:06.088345+0000) 2024-02-02T15:48:07.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:06 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:06.958+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.102474+0000 front 2024-02-02T15:47:33.102427+0000 (oldest deadline 2024-02-02T15:47:57.802602+0000) 2024-02-02T15:48:07.864 INFO:journalctl@ceph.osd.7.smithi071.stdout:Feb 02 15:48:07 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-7[65443]: 2024-02-02T15:48:07.607+0000 7f66be2a3700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:40.134496+0000 front 2024-02-02T15:47:40.134524+0000 (oldest deadline 2024-02-02T15:48:06.034624+0000) 2024-02-02T15:48:07.864 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:07 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:07.838+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.558440+0000 front 2024-02-02T15:47:33.558460+0000 (oldest deadline 2024-02-02T15:47:59.958896+0000) 2024-02-02T15:48:08.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:08 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:08.001+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:35.342656+0000 front 2024-02-02T15:47:35.342649+0000 (oldest deadline 2024-02-02T15:48:01.143219+0000) 2024-02-02T15:48:08.170 INFO:journalctl@ceph.osd.5.smithi142.stdout:Feb 02 15:48:07 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-5[59288]: 2024-02-02T15:48:07.838+0000 7f761ec92700 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:36.187466+0000 front 2024-02-02T15:47:36.187507+0000 (oldest deadline 2024-02-02T15:47:58.487340+0000) 2024-02-02T15:48:08.170 INFO:journalctl@ceph.osd.5.smithi142.stdout:Feb 02 15:48:07 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-5[59288]: 2024-02-02T15:48:07.838+0000 7f761ec92700 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:41.988425+0000 front 2024-02-02T15:47:41.988334+0000 (oldest deadline 2024-02-02T15:48:06.088345+0000) 2024-02-02T15:48:08.190 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:07 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:07.914+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.102474+0000 front 2024-02-02T15:47:33.102427+0000 (oldest deadline 2024-02-02T15:47:57.802602+0000) 2024-02-02T15:48:08.190 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:07 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:07.914+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:44.106107+0000 front 2024-02-02T15:47:44.106141+0000 (oldest deadline 2024-02-02T15:48:07.006384+0000) 2024-02-02T15:48:08.580 INFO:journalctl@ceph.osd.4.smithi071.stdout:Feb 02 15:48:08 smithi071 bash[89320]: ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-4 2024-02-02T15:48:08.848 INFO:journalctl@ceph.osd.7.smithi071.stdout:Feb 02 15:48:08 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-7[65443]: 2024-02-02T15:48:08.579+0000 7f66be2a3700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:40.134496+0000 front 2024-02-02T15:47:40.134524+0000 (oldest deadline 2024-02-02T15:48:06.034624+0000) 2024-02-02T15:48:08.849 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:08 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:08.797+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.558440+0000 front 2024-02-02T15:47:33.558460+0000 (oldest deadline 2024-02-02T15:47:59.958896+0000) 2024-02-02T15:48:08.868 INFO:journalctl@ceph.osd.8.smithi142.stdout:Feb 02 15:48:08 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-8[67933]: 2024-02-02T15:48:08.460+0000 7f525db65700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:41.946745+0000 front 2024-02-02T15:47:41.946768+0000 (oldest deadline 2024-02-02T15:48:07.846739+0000) 2024-02-02T15:48:09.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:08 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:08.985+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:35.342656+0000 front 2024-02-02T15:47:35.342649+0000 (oldest deadline 2024-02-02T15:48:01.143219+0000) 2024-02-02T15:48:09.204 INFO:journalctl@ceph.osd.5.smithi142.stdout:Feb 02 15:48:08 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-5[59288]: 2024-02-02T15:48:08.868+0000 7f761ec92700 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:36.187466+0000 front 2024-02-02T15:47:36.187507+0000 (oldest deadline 2024-02-02T15:47:58.487340+0000) 2024-02-02T15:48:09.204 INFO:journalctl@ceph.osd.5.smithi142.stdout:Feb 02 15:48:08 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-5[59288]: 2024-02-02T15:48:08.868+0000 7f761ec92700 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:41.988425+0000 front 2024-02-02T15:47:41.988334+0000 (oldest deadline 2024-02-02T15:48:06.088345+0000) 2024-02-02T15:48:09.204 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:08 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:08.935+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.102474+0000 front 2024-02-02T15:47:33.102427+0000 (oldest deadline 2024-02-02T15:47:57.802602+0000) 2024-02-02T15:48:09.358 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:08 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:08.935+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:44.106107+0000 front 2024-02-02T15:47:44.106141+0000 (oldest deadline 2024-02-02T15:48:07.006384+0000) 2024-02-02T15:48:09.548 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-371a409c-c1de-11ee-95b6-87774f69a715@osd.4.service' 2024-02-02T15:48:09.588 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-02T15:48:09.589 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-02-02T15:48:09.589 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-02-02T15:48:09.589 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-371a409c-c1de-11ee-95b6-87774f69a715@osd.5 2024-02-02T15:48:09.830 INFO:journalctl@ceph.osd.8.smithi142.stdout:Feb 02 15:48:09 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-8[67933]: 2024-02-02T15:48:09.505+0000 7f525db65700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:41.946745+0000 front 2024-02-02T15:47:41.946768+0000 (oldest deadline 2024-02-02T15:48:07.846739+0000) 2024-02-02T15:48:09.830 INFO:journalctl@ceph.osd.5.smithi142.stdout:Feb 02 15:48:09 smithi142 systemd[1]: Stopping Ceph osd.5 for 371a409c-c1de-11ee-95b6-87774f69a715... 2024-02-02T15:48:09.831 INFO:journalctl@ceph.osd.5.smithi142.stdout:Feb 02 15:48:09 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-5[59288]: 2024-02-02T15:48:09.830+0000 7f76246a0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-02T15:48:09.864 INFO:journalctl@ceph.osd.7.smithi071.stdout:Feb 02 15:48:09 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-7[65443]: 2024-02-02T15:48:09.564+0000 7f66be2a3700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:40.134496+0000 front 2024-02-02T15:47:40.134524+0000 (oldest deadline 2024-02-02T15:48:06.034624+0000) 2024-02-02T15:48:09.864 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:09 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:09.779+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.558440+0000 front 2024-02-02T15:47:33.558460+0000 (oldest deadline 2024-02-02T15:47:59.958896+0000) 2024-02-02T15:48:09.864 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:09 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:09.779+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:43.861259+0000 front 2024-02-02T15:47:43.861243+0000 (oldest deadline 2024-02-02T15:48:09.161042+0000) 2024-02-02T15:48:10.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:09 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:09.962+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:35.342656+0000 front 2024-02-02T15:47:35.342649+0000 (oldest deadline 2024-02-02T15:48:01.143219+0000) 2024-02-02T15:48:10.170 INFO:journalctl@ceph.osd.5.smithi142.stdout:Feb 02 15:48:09 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-5[59288]: 2024-02-02T15:48:09.830+0000 7f76246a0700 -1 osd.5 121 *** Got signal Terminated *** 2024-02-02T15:48:10.170 INFO:journalctl@ceph.osd.5.smithi142.stdout:Feb 02 15:48:09 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-5[59288]: 2024-02-02T15:48:09.830+0000 7f76246a0700 -1 osd.5 121 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-02T15:48:10.170 INFO:journalctl@ceph.osd.5.smithi142.stdout:Feb 02 15:48:09 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-5[59288]: 2024-02-02T15:48:09.859+0000 7f761ec92700 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:36.187466+0000 front 2024-02-02T15:47:36.187507+0000 (oldest deadline 2024-02-02T15:47:58.487340+0000) 2024-02-02T15:48:10.171 INFO:journalctl@ceph.osd.5.smithi142.stdout:Feb 02 15:48:09 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-5[59288]: 2024-02-02T15:48:09.859+0000 7f761ec92700 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:41.988425+0000 front 2024-02-02T15:47:41.988334+0000 (oldest deadline 2024-02-02T15:48:06.088345+0000) 2024-02-02T15:48:10.189 INFO:journalctl@ceph.osd.6.smithi001.stdout:Feb 02 15:48:09 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-6[69227]: 2024-02-02T15:48:09.934+0000 7f4fe8e00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:43.493158+0000 front 2024-02-02T15:47:43.493194+0000 (oldest deadline 2024-02-02T15:48:09.393396+0000) 2024-02-02T15:48:10.190 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:09 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:09.925+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.102474+0000 front 2024-02-02T15:47:33.102427+0000 (oldest deadline 2024-02-02T15:47:57.802602+0000) 2024-02-02T15:48:10.190 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:09 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:09.925+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:44.106107+0000 front 2024-02-02T15:47:44.106141+0000 (oldest deadline 2024-02-02T15:48:07.006384+0000) 2024-02-02T15:48:10.864 INFO:journalctl@ceph.osd.7.smithi071.stdout:Feb 02 15:48:10 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-7[65443]: 2024-02-02T15:48:10.573+0000 7f66be2a3700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:40.134496+0000 front 2024-02-02T15:47:40.134524+0000 (oldest deadline 2024-02-02T15:48:06.034624+0000) 2024-02-02T15:48:10.864 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:10 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:10.747+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.558440+0000 front 2024-02-02T15:47:33.558460+0000 (oldest deadline 2024-02-02T15:47:59.958896+0000) 2024-02-02T15:48:10.864 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:10 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:10.747+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:43.861259+0000 front 2024-02-02T15:47:43.861243+0000 (oldest deadline 2024-02-02T15:48:09.161042+0000) 2024-02-02T15:48:10.909 INFO:journalctl@ceph.osd.8.smithi142.stdout:Feb 02 15:48:10 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-8[67933]: 2024-02-02T15:48:10.465+0000 7f525db65700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:41.946745+0000 front 2024-02-02T15:47:41.946768+0000 (oldest deadline 2024-02-02T15:48:07.846739+0000) 2024-02-02T15:48:11.169 INFO:journalctl@ceph.osd.5.smithi142.stdout:Feb 02 15:48:10 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-5[59288]: 2024-02-02T15:48:10.909+0000 7f761ec92700 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:36.187466+0000 front 2024-02-02T15:47:36.187507+0000 (oldest deadline 2024-02-02T15:47:58.487340+0000) 2024-02-02T15:48:11.170 INFO:journalctl@ceph.osd.5.smithi142.stdout:Feb 02 15:48:10 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-5[59288]: 2024-02-02T15:48:10.909+0000 7f761ec92700 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:41.988425+0000 front 2024-02-02T15:47:41.988334+0000 (oldest deadline 2024-02-02T15:48:06.088345+0000) 2024-02-02T15:48:11.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:10 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:10.979+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:35.342656+0000 front 2024-02-02T15:47:35.342649+0000 (oldest deadline 2024-02-02T15:48:01.143219+0000) 2024-02-02T15:48:11.189 INFO:journalctl@ceph.osd.6.smithi001.stdout:Feb 02 15:48:10 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-6[69227]: 2024-02-02T15:48:10.914+0000 7f4fe8e00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:43.493158+0000 front 2024-02-02T15:47:43.493194+0000 (oldest deadline 2024-02-02T15:48:09.393396+0000) 2024-02-02T15:48:11.190 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:10 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:10.958+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.102474+0000 front 2024-02-02T15:47:33.102427+0000 (oldest deadline 2024-02-02T15:47:57.802602+0000) 2024-02-02T15:48:11.190 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:10 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:10.958+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:44.106107+0000 front 2024-02-02T15:47:44.106141+0000 (oldest deadline 2024-02-02T15:48:07.006384+0000) 2024-02-02T15:48:11.862 INFO:journalctl@ceph.osd.8.smithi142.stdout:Feb 02 15:48:11 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-8[67933]: 2024-02-02T15:48:11.457+0000 7f525db65700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:41.946745+0000 front 2024-02-02T15:47:41.946768+0000 (oldest deadline 2024-02-02T15:48:07.846739+0000) 2024-02-02T15:48:11.864 INFO:journalctl@ceph.osd.7.smithi071.stdout:Feb 02 15:48:11 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-7[65443]: 2024-02-02T15:48:11.565+0000 7f66be2a3700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:40.134496+0000 front 2024-02-02T15:47:40.134524+0000 (oldest deadline 2024-02-02T15:48:06.034624+0000) 2024-02-02T15:48:11.864 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:11 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:11.747+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.558440+0000 front 2024-02-02T15:47:33.558460+0000 (oldest deadline 2024-02-02T15:47:59.958896+0000) 2024-02-02T15:48:11.864 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:11 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:11.747+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:43.861259+0000 front 2024-02-02T15:47:43.861243+0000 (oldest deadline 2024-02-02T15:48:09.161042+0000) 2024-02-02T15:48:12.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:11 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:11.946+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:35.342656+0000 front 2024-02-02T15:47:35.342649+0000 (oldest deadline 2024-02-02T15:48:01.143219+0000) 2024-02-02T15:48:12.170 INFO:journalctl@ceph.osd.5.smithi142.stdout:Feb 02 15:48:11 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-5[59288]: 2024-02-02T15:48:11.863+0000 7f761ec92700 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:36.187466+0000 front 2024-02-02T15:47:36.187507+0000 (oldest deadline 2024-02-02T15:47:58.487340+0000) 2024-02-02T15:48:12.170 INFO:journalctl@ceph.osd.5.smithi142.stdout:Feb 02 15:48:11 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-5[59288]: 2024-02-02T15:48:11.863+0000 7f761ec92700 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:41.988425+0000 front 2024-02-02T15:47:41.988334+0000 (oldest deadline 2024-02-02T15:48:06.088345+0000) 2024-02-02T15:48:12.190 INFO:journalctl@ceph.osd.6.smithi001.stdout:Feb 02 15:48:11 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-6[69227]: 2024-02-02T15:48:11.875+0000 7f4fe8e00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:43.493158+0000 front 2024-02-02T15:47:43.493194+0000 (oldest deadline 2024-02-02T15:48:09.393396+0000) 2024-02-02T15:48:12.190 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:11 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:11.990+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.102474+0000 front 2024-02-02T15:47:33.102427+0000 (oldest deadline 2024-02-02T15:47:57.802602+0000) 2024-02-02T15:48:12.190 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:11 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:11.990+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:44.106107+0000 front 2024-02-02T15:47:44.106141+0000 (oldest deadline 2024-02-02T15:48:07.006384+0000) 2024-02-02T15:48:12.813 INFO:journalctl@ceph.osd.8.smithi142.stdout:Feb 02 15:48:12 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-8[67933]: 2024-02-02T15:48:12.436+0000 7f525db65700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:41.946745+0000 front 2024-02-02T15:47:41.946768+0000 (oldest deadline 2024-02-02T15:48:07.846739+0000) 2024-02-02T15:48:12.815 INFO:journalctl@ceph.osd.5.smithi142.stdout:Feb 02 15:48:12 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-5[59288]: 2024-02-02T15:48:12.814+0000 7f761ec92700 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:36.187466+0000 front 2024-02-02T15:47:36.187507+0000 (oldest deadline 2024-02-02T15:47:58.487340+0000) 2024-02-02T15:48:12.815 INFO:journalctl@ceph.osd.5.smithi142.stdout:Feb 02 15:48:12 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-5[59288]: 2024-02-02T15:48:12.814+0000 7f761ec92700 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:41.988425+0000 front 2024-02-02T15:47:41.988334+0000 (oldest deadline 2024-02-02T15:48:06.088345+0000) 2024-02-02T15:48:12.864 INFO:journalctl@ceph.osd.7.smithi071.stdout:Feb 02 15:48:12 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-7[65443]: 2024-02-02T15:48:12.536+0000 7f66be2a3700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:40.134496+0000 front 2024-02-02T15:47:40.134524+0000 (oldest deadline 2024-02-02T15:48:06.034624+0000) 2024-02-02T15:48:12.864 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:12 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:12.777+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.558440+0000 front 2024-02-02T15:47:33.558460+0000 (oldest deadline 2024-02-02T15:47:59.958896+0000) 2024-02-02T15:48:12.864 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:12 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:12.777+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:43.861259+0000 front 2024-02-02T15:47:43.861243+0000 (oldest deadline 2024-02-02T15:48:09.161042+0000) 2024-02-02T15:48:13.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:12 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:12.940+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:35.342656+0000 front 2024-02-02T15:47:35.342649+0000 (oldest deadline 2024-02-02T15:48:01.143219+0000) 2024-02-02T15:48:13.189 INFO:journalctl@ceph.osd.6.smithi001.stdout:Feb 02 15:48:12 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-6[69227]: 2024-02-02T15:48:12.883+0000 7f4fe8e00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:43.493158+0000 front 2024-02-02T15:47:43.493194+0000 (oldest deadline 2024-02-02T15:48:09.393396+0000) 2024-02-02T15:48:13.190 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:12 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:12.979+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.102474+0000 front 2024-02-02T15:47:33.102427+0000 (oldest deadline 2024-02-02T15:47:57.802602+0000) 2024-02-02T15:48:13.190 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:12 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:12.979+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:44.106107+0000 front 2024-02-02T15:47:44.106141+0000 (oldest deadline 2024-02-02T15:48:07.006384+0000) 2024-02-02T15:48:13.798 INFO:journalctl@ceph.osd.8.smithi142.stdout:Feb 02 15:48:13 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-8[67933]: 2024-02-02T15:48:13.445+0000 7f525db65700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:41.946745+0000 front 2024-02-02T15:47:41.946768+0000 (oldest deadline 2024-02-02T15:48:07.846739+0000) 2024-02-02T15:48:13.864 INFO:journalctl@ceph.osd.7.smithi071.stdout:Feb 02 15:48:13 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-7[65443]: 2024-02-02T15:48:13.525+0000 7f66be2a3700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:40.134496+0000 front 2024-02-02T15:47:40.134524+0000 (oldest deadline 2024-02-02T15:48:06.034624+0000) 2024-02-02T15:48:13.864 INFO:journalctl@ceph.osd.7.smithi071.stdout:Feb 02 15:48:13 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-7[65443]: 2024-02-02T15:48:13.525+0000 7f66be2a3700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:47.135707+0000 front 2024-02-02T15:47:47.135621+0000 (oldest deadline 2024-02-02T15:48:13.035562+0000) 2024-02-02T15:48:13.864 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:13 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:13.738+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.558440+0000 front 2024-02-02T15:47:33.558460+0000 (oldest deadline 2024-02-02T15:47:59.958896+0000) 2024-02-02T15:48:13.865 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:13 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:13.738+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:43.861259+0000 front 2024-02-02T15:47:43.861243+0000 (oldest deadline 2024-02-02T15:48:09.161042+0000) 2024-02-02T15:48:13.865 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:13 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:13.738+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:49.161607+0000 front 2024-02-02T15:47:49.161481+0000 (oldest deadline 2024-02-02T15:48:13.261576+0000) 2024-02-02T15:48:14.170 INFO:journalctl@ceph.osd.5.smithi142.stdout:Feb 02 15:48:13 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-5[59288]: 2024-02-02T15:48:13.798+0000 7f761ec92700 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:36.187466+0000 front 2024-02-02T15:47:36.187507+0000 (oldest deadline 2024-02-02T15:47:58.487340+0000) 2024-02-02T15:48:14.170 INFO:journalctl@ceph.osd.5.smithi142.stdout:Feb 02 15:48:13 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-5[59288]: 2024-02-02T15:48:13.798+0000 7f761ec92700 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:41.988425+0000 front 2024-02-02T15:47:41.988334+0000 (oldest deadline 2024-02-02T15:48:06.088345+0000) 2024-02-02T15:48:14.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:13 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:13.915+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:35.342656+0000 front 2024-02-02T15:47:35.342649+0000 (oldest deadline 2024-02-02T15:48:01.143219+0000) 2024-02-02T15:48:14.171 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:13 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:13.915+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:47.544669+0000 front 2024-02-02T15:47:47.544689+0000 (oldest deadline 2024-02-02T15:48:13.444898+0000) 2024-02-02T15:48:14.190 INFO:journalctl@ceph.osd.6.smithi001.stdout:Feb 02 15:48:13 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-6[69227]: 2024-02-02T15:48:13.892+0000 7f4fe8e00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:43.493158+0000 front 2024-02-02T15:47:43.493194+0000 (oldest deadline 2024-02-02T15:48:09.393396+0000) 2024-02-02T15:48:14.190 INFO:journalctl@ceph.osd.6.smithi001.stdout:Feb 02 15:48:13 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-6[69227]: 2024-02-02T15:48:13.892+0000 7f4fe8e00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:49.394343+0000 front 2024-02-02T15:47:49.394309+0000 (oldest deadline 2024-02-02T15:48:13.494579+0000) 2024-02-02T15:48:14.190 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:13 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:13.970+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.102474+0000 front 2024-02-02T15:47:33.102427+0000 (oldest deadline 2024-02-02T15:47:57.802602+0000) 2024-02-02T15:48:14.190 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:13 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:13.970+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:44.106107+0000 front 2024-02-02T15:47:44.106141+0000 (oldest deadline 2024-02-02T15:48:07.006384+0000) 2024-02-02T15:48:14.772 INFO:journalctl@ceph.osd.7.smithi071.stdout:Feb 02 15:48:14 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-7[65443]: 2024-02-02T15:48:14.487+0000 7f66be2a3700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:40.134496+0000 front 2024-02-02T15:47:40.134524+0000 (oldest deadline 2024-02-02T15:48:06.034624+0000) 2024-02-02T15:48:14.772 INFO:journalctl@ceph.osd.7.smithi071.stdout:Feb 02 15:48:14 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-7[65443]: 2024-02-02T15:48:14.487+0000 7f66be2a3700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:47.135707+0000 front 2024-02-02T15:47:47.135621+0000 (oldest deadline 2024-02-02T15:48:13.035562+0000) 2024-02-02T15:48:14.827 INFO:journalctl@ceph.osd.8.smithi142.stdout:Feb 02 15:48:14 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-8[67933]: 2024-02-02T15:48:14.430+0000 7f525db65700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:41.946745+0000 front 2024-02-02T15:47:41.946768+0000 (oldest deadline 2024-02-02T15:48:07.846739+0000) 2024-02-02T15:48:15.114 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:14 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:14.773+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.558440+0000 front 2024-02-02T15:47:33.558460+0000 (oldest deadline 2024-02-02T15:47:59.958896+0000) 2024-02-02T15:48:15.114 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:14 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:14.773+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:43.861259+0000 front 2024-02-02T15:47:43.861243+0000 (oldest deadline 2024-02-02T15:48:09.161042+0000) 2024-02-02T15:48:15.114 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:14 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:14.773+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:49.161607+0000 front 2024-02-02T15:47:49.161481+0000 (oldest deadline 2024-02-02T15:48:13.261576+0000) 2024-02-02T15:48:15.170 INFO:journalctl@ceph.osd.5.smithi142.stdout:Feb 02 15:48:14 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-5[59288]: 2024-02-02T15:48:14.825+0000 7f761ec92700 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:36.187466+0000 front 2024-02-02T15:47:36.187507+0000 (oldest deadline 2024-02-02T15:47:58.487340+0000) 2024-02-02T15:48:15.170 INFO:journalctl@ceph.osd.5.smithi142.stdout:Feb 02 15:48:14 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-5[59288]: 2024-02-02T15:48:14.825+0000 7f761ec92700 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:41.988425+0000 front 2024-02-02T15:47:41.988334+0000 (oldest deadline 2024-02-02T15:48:06.088345+0000) 2024-02-02T15:48:15.170 INFO:journalctl@ceph.osd.5.smithi142.stdout:Feb 02 15:48:14 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-5[59288]: 2024-02-02T15:48:14.825+0000 7f761ec92700 -1 osd.5 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:51.790747+0000 front 2024-02-02T15:47:51.790728+0000 (oldest deadline 2024-02-02T15:48:14.691039+0000) 2024-02-02T15:48:15.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:14 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:14.931+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:35.342656+0000 front 2024-02-02T15:47:35.342649+0000 (oldest deadline 2024-02-02T15:48:01.143219+0000) 2024-02-02T15:48:15.171 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:14 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:14.931+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:47.544669+0000 front 2024-02-02T15:47:47.544689+0000 (oldest deadline 2024-02-02T15:48:13.444898+0000) 2024-02-02T15:48:15.190 INFO:journalctl@ceph.osd.6.smithi001.stdout:Feb 02 15:48:14 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-6[69227]: 2024-02-02T15:48:14.904+0000 7f4fe8e00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:43.493158+0000 front 2024-02-02T15:47:43.493194+0000 (oldest deadline 2024-02-02T15:48:09.393396+0000) 2024-02-02T15:48:15.190 INFO:journalctl@ceph.osd.6.smithi001.stdout:Feb 02 15:48:14 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-6[69227]: 2024-02-02T15:48:14.904+0000 7f4fe8e00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:49.394343+0000 front 2024-02-02T15:47:49.394309+0000 (oldest deadline 2024-02-02T15:48:13.494579+0000) 2024-02-02T15:48:15.190 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:14 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:14.933+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.102474+0000 front 2024-02-02T15:47:33.102427+0000 (oldest deadline 2024-02-02T15:47:57.802602+0000) 2024-02-02T15:48:15.190 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:14 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:14.933+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:44.106107+0000 front 2024-02-02T15:47:44.106141+0000 (oldest deadline 2024-02-02T15:48:07.006384+0000) 2024-02-02T15:48:15.670 INFO:journalctl@ceph.osd.5.smithi142.stdout:Feb 02 15:48:15 smithi142 bash[91920]: ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-5 2024-02-02T15:48:15.670 INFO:journalctl@ceph.osd.8.smithi142.stdout:Feb 02 15:48:15 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-8[67933]: 2024-02-02T15:48:15.410+0000 7f525db65700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:41.946745+0000 front 2024-02-02T15:47:41.946768+0000 (oldest deadline 2024-02-02T15:48:07.846739+0000) 2024-02-02T15:48:15.864 INFO:journalctl@ceph.osd.7.smithi071.stdout:Feb 02 15:48:15 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-7[65443]: 2024-02-02T15:48:15.530+0000 7f66be2a3700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:40.134496+0000 front 2024-02-02T15:47:40.134524+0000 (oldest deadline 2024-02-02T15:48:06.034624+0000) 2024-02-02T15:48:15.864 INFO:journalctl@ceph.osd.7.smithi071.stdout:Feb 02 15:48:15 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-7[65443]: 2024-02-02T15:48:15.530+0000 7f66be2a3700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:47.135707+0000 front 2024-02-02T15:47:47.135621+0000 (oldest deadline 2024-02-02T15:48:13.035562+0000) 2024-02-02T15:48:15.865 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:15 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:15.774+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.558440+0000 front 2024-02-02T15:47:33.558460+0000 (oldest deadline 2024-02-02T15:47:59.958896+0000) 2024-02-02T15:48:15.865 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:15 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:15.774+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:43.861259+0000 front 2024-02-02T15:47:43.861243+0000 (oldest deadline 2024-02-02T15:48:09.161042+0000) 2024-02-02T15:48:15.865 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:15 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:15.774+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:49.161607+0000 front 2024-02-02T15:47:49.161481+0000 (oldest deadline 2024-02-02T15:48:13.261576+0000) 2024-02-02T15:48:16.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:15 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:15.902+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:35.342656+0000 front 2024-02-02T15:47:35.342649+0000 (oldest deadline 2024-02-02T15:48:01.143219+0000) 2024-02-02T15:48:16.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:15 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:15.902+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:47.544669+0000 front 2024-02-02T15:47:47.544689+0000 (oldest deadline 2024-02-02T15:48:13.444898+0000) 2024-02-02T15:48:16.190 INFO:journalctl@ceph.osd.6.smithi001.stdout:Feb 02 15:48:15 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-6[69227]: 2024-02-02T15:48:15.932+0000 7f4fe8e00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:43.493158+0000 front 2024-02-02T15:47:43.493194+0000 (oldest deadline 2024-02-02T15:48:09.393396+0000) 2024-02-02T15:48:16.190 INFO:journalctl@ceph.osd.6.smithi001.stdout:Feb 02 15:48:15 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-6[69227]: 2024-02-02T15:48:15.932+0000 7f4fe8e00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:49.394343+0000 front 2024-02-02T15:47:49.394309+0000 (oldest deadline 2024-02-02T15:48:13.494579+0000) 2024-02-02T15:48:16.190 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:15 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:15.922+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.102474+0000 front 2024-02-02T15:47:33.102427+0000 (oldest deadline 2024-02-02T15:47:57.802602+0000) 2024-02-02T15:48:16.191 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:15 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:15.922+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:44.106107+0000 front 2024-02-02T15:47:44.106141+0000 (oldest deadline 2024-02-02T15:48:07.006384+0000) 2024-02-02T15:48:16.191 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:15 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:15.922+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:51.707992+0000 front 2024-02-02T15:47:51.707872+0000 (oldest deadline 2024-02-02T15:48:15.808055+0000) 2024-02-02T15:48:16.610 INFO:journalctl@ceph.osd.8.smithi142.stdout:Feb 02 15:48:16 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-8[67933]: 2024-02-02T15:48:16.420+0000 7f525db65700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:41.946745+0000 front 2024-02-02T15:47:41.946768+0000 (oldest deadline 2024-02-02T15:48:07.846739+0000) 2024-02-02T15:48:16.610 INFO:journalctl@ceph.osd.8.smithi142.stdout:Feb 02 15:48:16 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-8[67933]: 2024-02-02T15:48:16.420+0000 7f525db65700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:50.147681+0000 front 2024-02-02T15:47:50.147810+0000 (oldest deadline 2024-02-02T15:48:16.047900+0000) 2024-02-02T15:48:16.617 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-371a409c-c1de-11ee-95b6-87774f69a715@osd.5.service' 2024-02-02T15:48:16.657 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-02T15:48:16.658 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-02-02T15:48:16.658 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-02-02T15:48:16.658 DEBUG:teuthology.orchestra.run.smithi001:> sudo systemctl stop ceph-371a409c-c1de-11ee-95b6-87774f69a715@osd.6 2024-02-02T15:48:16.760 INFO:journalctl@ceph.osd.7.smithi071.stdout:Feb 02 15:48:16 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-7[65443]: 2024-02-02T15:48:16.504+0000 7f66be2a3700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:40.134496+0000 front 2024-02-02T15:47:40.134524+0000 (oldest deadline 2024-02-02T15:48:06.034624+0000) 2024-02-02T15:48:16.761 INFO:journalctl@ceph.osd.7.smithi071.stdout:Feb 02 15:48:16 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-7[65443]: 2024-02-02T15:48:16.504+0000 7f66be2a3700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:47.135707+0000 front 2024-02-02T15:47:47.135621+0000 (oldest deadline 2024-02-02T15:48:13.035562+0000) 2024-02-02T15:48:17.030 INFO:journalctl@ceph.osd.6.smithi001.stdout:Feb 02 15:48:16 smithi001 systemd[1]: Stopping Ceph osd.6 for 371a409c-c1de-11ee-95b6-87774f69a715... 2024-02-02T15:48:17.030 INFO:journalctl@ceph.osd.6.smithi001.stdout:Feb 02 15:48:16 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-6[69227]: 2024-02-02T15:48:16.952+0000 7f4fe8e00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:43.493158+0000 front 2024-02-02T15:47:43.493194+0000 (oldest deadline 2024-02-02T15:48:09.393396+0000) 2024-02-02T15:48:17.030 INFO:journalctl@ceph.osd.6.smithi001.stdout:Feb 02 15:48:16 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-6[69227]: 2024-02-02T15:48:16.952+0000 7f4fe8e00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:49.394343+0000 front 2024-02-02T15:47:49.394309+0000 (oldest deadline 2024-02-02T15:48:13.494579+0000) 2024-02-02T15:48:17.030 INFO:journalctl@ceph.osd.6.smithi001.stdout:Feb 02 15:48:17 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-6[69227]: 2024-02-02T15:48:17.030+0000 7f4ff0800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-02T15:48:17.030 INFO:journalctl@ceph.osd.6.smithi001.stdout:Feb 02 15:48:17 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-6[69227]: 2024-02-02T15:48:17.030+0000 7f4ff0800700 -1 osd.6 121 *** Got signal Terminated *** 2024-02-02T15:48:17.031 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:16 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:16.884+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.102474+0000 front 2024-02-02T15:47:33.102427+0000 (oldest deadline 2024-02-02T15:47:57.802602+0000) 2024-02-02T15:48:17.031 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:16 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:16.884+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:44.106107+0000 front 2024-02-02T15:47:44.106141+0000 (oldest deadline 2024-02-02T15:48:07.006384+0000) 2024-02-02T15:48:17.031 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:16 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:16.884+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:51.707992+0000 front 2024-02-02T15:47:51.707872+0000 (oldest deadline 2024-02-02T15:48:15.808055+0000) 2024-02-02T15:48:17.114 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:16 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:16.761+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.558440+0000 front 2024-02-02T15:47:33.558460+0000 (oldest deadline 2024-02-02T15:47:59.958896+0000) 2024-02-02T15:48:17.114 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:16 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:16.761+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:43.861259+0000 front 2024-02-02T15:47:43.861243+0000 (oldest deadline 2024-02-02T15:48:09.161042+0000) 2024-02-02T15:48:17.114 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:16 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:16.761+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:49.161607+0000 front 2024-02-02T15:47:49.161481+0000 (oldest deadline 2024-02-02T15:48:13.261576+0000) 2024-02-02T15:48:17.169 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:16 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:16.866+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:35.342656+0000 front 2024-02-02T15:47:35.342649+0000 (oldest deadline 2024-02-02T15:48:01.143219+0000) 2024-02-02T15:48:17.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:16 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:16.866+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:47.544669+0000 front 2024-02-02T15:47:47.544689+0000 (oldest deadline 2024-02-02T15:48:13.444898+0000) 2024-02-02T15:48:17.440 INFO:journalctl@ceph.osd.6.smithi001.stdout:Feb 02 15:48:17 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-6[69227]: 2024-02-02T15:48:17.030+0000 7f4ff0800700 -1 osd.6 121 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-02T15:48:17.786 INFO:journalctl@ceph.osd.7.smithi071.stdout:Feb 02 15:48:17 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-7[65443]: 2024-02-02T15:48:17.532+0000 7f66be2a3700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:40.134496+0000 front 2024-02-02T15:47:40.134524+0000 (oldest deadline 2024-02-02T15:48:06.034624+0000) 2024-02-02T15:48:17.786 INFO:journalctl@ceph.osd.7.smithi071.stdout:Feb 02 15:48:17 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-7[65443]: 2024-02-02T15:48:17.532+0000 7f66be2a3700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:47.135707+0000 front 2024-02-02T15:47:47.135621+0000 (oldest deadline 2024-02-02T15:48:13.035562+0000) 2024-02-02T15:48:17.913 INFO:journalctl@ceph.osd.8.smithi142.stdout:Feb 02 15:48:17 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-8[67933]: 2024-02-02T15:48:17.443+0000 7f525db65700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:41.946745+0000 front 2024-02-02T15:47:41.946768+0000 (oldest deadline 2024-02-02T15:48:07.846739+0000) 2024-02-02T15:48:17.913 INFO:journalctl@ceph.osd.8.smithi142.stdout:Feb 02 15:48:17 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-8[67933]: 2024-02-02T15:48:17.443+0000 7f525db65700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:50.147681+0000 front 2024-02-02T15:47:50.147810+0000 (oldest deadline 2024-02-02T15:48:16.047900+0000) 2024-02-02T15:48:18.114 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:17 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:17.787+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.558440+0000 front 2024-02-02T15:47:33.558460+0000 (oldest deadline 2024-02-02T15:47:59.958896+0000) 2024-02-02T15:48:18.114 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:17 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:17.787+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:43.861259+0000 front 2024-02-02T15:47:43.861243+0000 (oldest deadline 2024-02-02T15:48:09.161042+0000) 2024-02-02T15:48:18.114 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:17 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:17.787+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:49.161607+0000 front 2024-02-02T15:47:49.161481+0000 (oldest deadline 2024-02-02T15:48:13.261576+0000) 2024-02-02T15:48:18.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:17 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:17.913+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:35.342656+0000 front 2024-02-02T15:47:35.342649+0000 (oldest deadline 2024-02-02T15:48:01.143219+0000) 2024-02-02T15:48:18.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:17 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:17.913+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:47.544669+0000 front 2024-02-02T15:47:47.544689+0000 (oldest deadline 2024-02-02T15:48:13.444898+0000) 2024-02-02T15:48:18.190 INFO:journalctl@ceph.osd.6.smithi001.stdout:Feb 02 15:48:17 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-6[69227]: 2024-02-02T15:48:17.965+0000 7f4fe8e00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:43.493158+0000 front 2024-02-02T15:47:43.493194+0000 (oldest deadline 2024-02-02T15:48:09.393396+0000) 2024-02-02T15:48:18.190 INFO:journalctl@ceph.osd.6.smithi001.stdout:Feb 02 15:48:17 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-6[69227]: 2024-02-02T15:48:17.965+0000 7f4fe8e00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:49.394343+0000 front 2024-02-02T15:47:49.394309+0000 (oldest deadline 2024-02-02T15:48:13.494579+0000) 2024-02-02T15:48:18.190 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:17 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:17.922+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.102474+0000 front 2024-02-02T15:47:33.102427+0000 (oldest deadline 2024-02-02T15:47:57.802602+0000) 2024-02-02T15:48:18.190 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:17 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:17.922+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:44.106107+0000 front 2024-02-02T15:47:44.106141+0000 (oldest deadline 2024-02-02T15:48:07.006384+0000) 2024-02-02T15:48:18.190 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:17 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:17.922+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:51.707992+0000 front 2024-02-02T15:47:51.707872+0000 (oldest deadline 2024-02-02T15:48:15.808055+0000) 2024-02-02T15:48:18.818 INFO:journalctl@ceph.osd.7.smithi071.stdout:Feb 02 15:48:18 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-7[65443]: 2024-02-02T15:48:18.490+0000 7f66be2a3700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:40.134496+0000 front 2024-02-02T15:47:40.134524+0000 (oldest deadline 2024-02-02T15:48:06.034624+0000) 2024-02-02T15:48:18.819 INFO:journalctl@ceph.osd.7.smithi071.stdout:Feb 02 15:48:18 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-7[65443]: 2024-02-02T15:48:18.490+0000 7f66be2a3700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:47.135707+0000 front 2024-02-02T15:47:47.135621+0000 (oldest deadline 2024-02-02T15:48:13.035562+0000) 2024-02-02T15:48:18.907 INFO:journalctl@ceph.osd.8.smithi142.stdout:Feb 02 15:48:18 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-8[67933]: 2024-02-02T15:48:18.423+0000 7f525db65700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:41.946745+0000 front 2024-02-02T15:47:41.946768+0000 (oldest deadline 2024-02-02T15:48:07.846739+0000) 2024-02-02T15:48:18.907 INFO:journalctl@ceph.osd.8.smithi142.stdout:Feb 02 15:48:18 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-8[67933]: 2024-02-02T15:48:18.423+0000 7f525db65700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:50.147681+0000 front 2024-02-02T15:47:50.147810+0000 (oldest deadline 2024-02-02T15:48:16.047900+0000) 2024-02-02T15:48:19.115 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:18 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:18.819+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.558440+0000 front 2024-02-02T15:47:33.558460+0000 (oldest deadline 2024-02-02T15:47:59.958896+0000) 2024-02-02T15:48:19.115 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:18 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:18.819+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:43.861259+0000 front 2024-02-02T15:47:43.861243+0000 (oldest deadline 2024-02-02T15:48:09.161042+0000) 2024-02-02T15:48:19.115 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:18 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:18.819+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:49.161607+0000 front 2024-02-02T15:47:49.161481+0000 (oldest deadline 2024-02-02T15:48:13.261576+0000) 2024-02-02T15:48:19.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:18 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:18.907+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:35.342656+0000 front 2024-02-02T15:47:35.342649+0000 (oldest deadline 2024-02-02T15:48:01.143219+0000) 2024-02-02T15:48:19.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:18 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:18.907+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:47.544669+0000 front 2024-02-02T15:47:47.544689+0000 (oldest deadline 2024-02-02T15:48:13.444898+0000) 2024-02-02T15:48:19.190 INFO:journalctl@ceph.osd.6.smithi001.stdout:Feb 02 15:48:18 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-6[69227]: 2024-02-02T15:48:18.920+0000 7f4fe8e00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:43.493158+0000 front 2024-02-02T15:47:43.493194+0000 (oldest deadline 2024-02-02T15:48:09.393396+0000) 2024-02-02T15:48:19.190 INFO:journalctl@ceph.osd.6.smithi001.stdout:Feb 02 15:48:18 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-6[69227]: 2024-02-02T15:48:18.920+0000 7f4fe8e00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:49.394343+0000 front 2024-02-02T15:47:49.394309+0000 (oldest deadline 2024-02-02T15:48:13.494579+0000) 2024-02-02T15:48:19.190 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:18 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:18.940+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.102474+0000 front 2024-02-02T15:47:33.102427+0000 (oldest deadline 2024-02-02T15:47:57.802602+0000) 2024-02-02T15:48:19.190 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:18 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:18.940+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:44.106107+0000 front 2024-02-02T15:47:44.106141+0000 (oldest deadline 2024-02-02T15:48:07.006384+0000) 2024-02-02T15:48:19.190 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:18 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:18.940+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:51.707992+0000 front 2024-02-02T15:47:51.707872+0000 (oldest deadline 2024-02-02T15:48:15.808055+0000) 2024-02-02T15:48:19.864 INFO:journalctl@ceph.osd.7.smithi071.stdout:Feb 02 15:48:19 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-7[65443]: 2024-02-02T15:48:19.527+0000 7f66be2a3700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:40.134496+0000 front 2024-02-02T15:47:40.134524+0000 (oldest deadline 2024-02-02T15:48:06.034624+0000) 2024-02-02T15:48:19.864 INFO:journalctl@ceph.osd.7.smithi071.stdout:Feb 02 15:48:19 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-7[65443]: 2024-02-02T15:48:19.527+0000 7f66be2a3700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:47.135707+0000 front 2024-02-02T15:47:47.135621+0000 (oldest deadline 2024-02-02T15:48:13.035562+0000) 2024-02-02T15:48:19.873 INFO:journalctl@ceph.osd.8.smithi142.stdout:Feb 02 15:48:19 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-8[67933]: 2024-02-02T15:48:19.440+0000 7f525db65700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:41.946745+0000 front 2024-02-02T15:47:41.946768+0000 (oldest deadline 2024-02-02T15:48:07.846739+0000) 2024-02-02T15:48:19.873 INFO:journalctl@ceph.osd.8.smithi142.stdout:Feb 02 15:48:19 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-8[67933]: 2024-02-02T15:48:19.440+0000 7f525db65700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:50.147681+0000 front 2024-02-02T15:47:50.147810+0000 (oldest deadline 2024-02-02T15:48:16.047900+0000) 2024-02-02T15:48:20.169 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:19 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:19.873+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:35.342656+0000 front 2024-02-02T15:47:35.342649+0000 (oldest deadline 2024-02-02T15:48:01.143219+0000) 2024-02-02T15:48:20.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:19 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:19.873+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:47.544669+0000 front 2024-02-02T15:47:47.544689+0000 (oldest deadline 2024-02-02T15:48:13.444898+0000) 2024-02-02T15:48:20.190 INFO:journalctl@ceph.osd.6.smithi001.stdout:Feb 02 15:48:19 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-6[69227]: 2024-02-02T15:48:19.878+0000 7f4fe8e00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:43.493158+0000 front 2024-02-02T15:47:43.493194+0000 (oldest deadline 2024-02-02T15:48:09.393396+0000) 2024-02-02T15:48:20.190 INFO:journalctl@ceph.osd.6.smithi001.stdout:Feb 02 15:48:19 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-6[69227]: 2024-02-02T15:48:19.878+0000 7f4fe8e00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:49.394343+0000 front 2024-02-02T15:47:49.394309+0000 (oldest deadline 2024-02-02T15:48:13.494579+0000) 2024-02-02T15:48:20.190 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:19 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:19.976+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.102474+0000 front 2024-02-02T15:47:33.102427+0000 (oldest deadline 2024-02-02T15:47:57.802602+0000) 2024-02-02T15:48:20.190 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:19 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:19.976+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:44.106107+0000 front 2024-02-02T15:47:44.106141+0000 (oldest deadline 2024-02-02T15:48:07.006384+0000) 2024-02-02T15:48:20.190 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:19 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:19.976+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:51.707992+0000 front 2024-02-02T15:47:51.707872+0000 (oldest deadline 2024-02-02T15:48:15.808055+0000) 2024-02-02T15:48:20.364 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:19 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:19.868+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.558440+0000 front 2024-02-02T15:47:33.558460+0000 (oldest deadline 2024-02-02T15:47:59.958896+0000) 2024-02-02T15:48:20.364 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:19 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:19.868+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:43.861259+0000 front 2024-02-02T15:47:43.861243+0000 (oldest deadline 2024-02-02T15:48:09.161042+0000) 2024-02-02T15:48:20.364 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:19 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:19.868+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:49.161607+0000 front 2024-02-02T15:47:49.161481+0000 (oldest deadline 2024-02-02T15:48:13.261576+0000) 2024-02-02T15:48:20.861 INFO:journalctl@ceph.osd.8.smithi142.stdout:Feb 02 15:48:20 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-8[67933]: 2024-02-02T15:48:20.471+0000 7f525db65700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:41.946745+0000 front 2024-02-02T15:47:41.946768+0000 (oldest deadline 2024-02-02T15:48:07.846739+0000) 2024-02-02T15:48:20.861 INFO:journalctl@ceph.osd.8.smithi142.stdout:Feb 02 15:48:20 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-8[67933]: 2024-02-02T15:48:20.471+0000 7f525db65700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:50.147681+0000 front 2024-02-02T15:47:50.147810+0000 (oldest deadline 2024-02-02T15:48:16.047900+0000) 2024-02-02T15:48:20.864 INFO:journalctl@ceph.osd.7.smithi071.stdout:Feb 02 15:48:20 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-7[65443]: 2024-02-02T15:48:20.577+0000 7f66be2a3700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:40.134496+0000 front 2024-02-02T15:47:40.134524+0000 (oldest deadline 2024-02-02T15:48:06.034624+0000) 2024-02-02T15:48:20.864 INFO:journalctl@ceph.osd.7.smithi071.stdout:Feb 02 15:48:20 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-7[65443]: 2024-02-02T15:48:20.577+0000 7f66be2a3700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:47.135707+0000 front 2024-02-02T15:47:47.135621+0000 (oldest deadline 2024-02-02T15:48:13.035562+0000) 2024-02-02T15:48:20.864 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:20 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:20.823+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.558440+0000 front 2024-02-02T15:47:33.558460+0000 (oldest deadline 2024-02-02T15:47:59.958896+0000) 2024-02-02T15:48:20.864 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:20 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:20.823+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:43.861259+0000 front 2024-02-02T15:47:43.861243+0000 (oldest deadline 2024-02-02T15:48:09.161042+0000) 2024-02-02T15:48:20.864 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:20 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:20.823+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:49.161607+0000 front 2024-02-02T15:47:49.161481+0000 (oldest deadline 2024-02-02T15:48:13.261576+0000) 2024-02-02T15:48:21.169 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:20 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:20.861+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:35.342656+0000 front 2024-02-02T15:47:35.342649+0000 (oldest deadline 2024-02-02T15:48:01.143219+0000) 2024-02-02T15:48:21.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:20 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:20.861+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:47.544669+0000 front 2024-02-02T15:47:47.544689+0000 (oldest deadline 2024-02-02T15:48:13.444898+0000) 2024-02-02T15:48:21.190 INFO:journalctl@ceph.osd.6.smithi001.stdout:Feb 02 15:48:20 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-6[69227]: 2024-02-02T15:48:20.859+0000 7f4fe8e00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:43.493158+0000 front 2024-02-02T15:47:43.493194+0000 (oldest deadline 2024-02-02T15:48:09.393396+0000) 2024-02-02T15:48:21.190 INFO:journalctl@ceph.osd.6.smithi001.stdout:Feb 02 15:48:20 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-6[69227]: 2024-02-02T15:48:20.859+0000 7f4fe8e00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:49.394343+0000 front 2024-02-02T15:47:49.394309+0000 (oldest deadline 2024-02-02T15:48:13.494579+0000) 2024-02-02T15:48:21.190 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:21 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:21.015+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.102474+0000 front 2024-02-02T15:47:33.102427+0000 (oldest deadline 2024-02-02T15:47:57.802602+0000) 2024-02-02T15:48:21.190 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:21 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:21.015+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:44.106107+0000 front 2024-02-02T15:47:44.106141+0000 (oldest deadline 2024-02-02T15:48:07.006384+0000) 2024-02-02T15:48:21.190 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:21 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:21.015+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:51.707992+0000 front 2024-02-02T15:47:51.707872+0000 (oldest deadline 2024-02-02T15:48:15.808055+0000) 2024-02-02T15:48:21.812 INFO:journalctl@ceph.osd.8.smithi142.stdout:Feb 02 15:48:21 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-8[67933]: 2024-02-02T15:48:21.510+0000 7f525db65700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:41.946745+0000 front 2024-02-02T15:47:41.946768+0000 (oldest deadline 2024-02-02T15:48:07.846739+0000) 2024-02-02T15:48:21.813 INFO:journalctl@ceph.osd.8.smithi142.stdout:Feb 02 15:48:21 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-8[67933]: 2024-02-02T15:48:21.510+0000 7f525db65700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:50.147681+0000 front 2024-02-02T15:47:50.147810+0000 (oldest deadline 2024-02-02T15:48:16.047900+0000) 2024-02-02T15:48:21.820 INFO:journalctl@ceph.osd.7.smithi071.stdout:Feb 02 15:48:21 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-7[65443]: 2024-02-02T15:48:21.548+0000 7f66be2a3700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:40.134496+0000 front 2024-02-02T15:47:40.134524+0000 (oldest deadline 2024-02-02T15:48:06.034624+0000) 2024-02-02T15:48:21.820 INFO:journalctl@ceph.osd.7.smithi071.stdout:Feb 02 15:48:21 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-7[65443]: 2024-02-02T15:48:21.548+0000 7f66be2a3700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:47.135707+0000 front 2024-02-02T15:47:47.135621+0000 (oldest deadline 2024-02-02T15:48:13.035562+0000) 2024-02-02T15:48:22.114 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:21 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:21.821+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.558440+0000 front 2024-02-02T15:47:33.558460+0000 (oldest deadline 2024-02-02T15:47:59.958896+0000) 2024-02-02T15:48:22.114 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:21 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:21.821+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:43.861259+0000 front 2024-02-02T15:47:43.861243+0000 (oldest deadline 2024-02-02T15:48:09.161042+0000) 2024-02-02T15:48:22.114 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:21 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:21.821+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:49.161607+0000 front 2024-02-02T15:47:49.161481+0000 (oldest deadline 2024-02-02T15:48:13.261576+0000) 2024-02-02T15:48:22.114 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:21 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:21.821+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:56.063095+0000 front 2024-02-02T15:47:56.063158+0000 (oldest deadline 2024-02-02T15:48:21.363242+0000) 2024-02-02T15:48:22.171 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:21 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:21.812+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:35.342656+0000 front 2024-02-02T15:47:35.342649+0000 (oldest deadline 2024-02-02T15:48:01.143219+0000) 2024-02-02T15:48:22.172 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:21 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:21.812+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:47.544669+0000 front 2024-02-02T15:47:47.544689+0000 (oldest deadline 2024-02-02T15:48:13.444898+0000) 2024-02-02T15:48:22.190 INFO:journalctl@ceph.osd.6.smithi001.stdout:Feb 02 15:48:21 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-6[69227]: 2024-02-02T15:48:21.905+0000 7f4fe8e00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:43.493158+0000 front 2024-02-02T15:47:43.493194+0000 (oldest deadline 2024-02-02T15:48:09.393396+0000) 2024-02-02T15:48:22.190 INFO:journalctl@ceph.osd.6.smithi001.stdout:Feb 02 15:48:21 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-6[69227]: 2024-02-02T15:48:21.905+0000 7f4fe8e00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:49.394343+0000 front 2024-02-02T15:47:49.394309+0000 (oldest deadline 2024-02-02T15:48:13.494579+0000) 2024-02-02T15:48:22.190 INFO:journalctl@ceph.osd.6.smithi001.stdout:Feb 02 15:48:21 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-6[69227]: 2024-02-02T15:48:21.905+0000 7f4fe8e00700 -1 osd.6 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:59.099051+0000 front 2024-02-02T15:47:59.099021+0000 (oldest deadline 2024-02-02T15:48:21.399191+0000) 2024-02-02T15:48:22.190 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:21 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:21.989+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.102474+0000 front 2024-02-02T15:47:33.102427+0000 (oldest deadline 2024-02-02T15:47:57.802602+0000) 2024-02-02T15:48:22.191 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:21 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:21.989+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:44.106107+0000 front 2024-02-02T15:47:44.106141+0000 (oldest deadline 2024-02-02T15:48:07.006384+0000) 2024-02-02T15:48:22.191 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:21 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:21.989+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:51.707992+0000 front 2024-02-02T15:47:51.707872+0000 (oldest deadline 2024-02-02T15:48:15.808055+0000) 2024-02-02T15:48:22.820 INFO:journalctl@ceph.osd.7.smithi071.stdout:Feb 02 15:48:22 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-7[65443]: 2024-02-02T15:48:22.565+0000 7f66be2a3700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:40.134496+0000 front 2024-02-02T15:47:40.134524+0000 (oldest deadline 2024-02-02T15:48:06.034624+0000) 2024-02-02T15:48:22.820 INFO:journalctl@ceph.osd.7.smithi071.stdout:Feb 02 15:48:22 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-7[65443]: 2024-02-02T15:48:22.565+0000 7f66be2a3700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:47.135707+0000 front 2024-02-02T15:47:47.135621+0000 (oldest deadline 2024-02-02T15:48:13.035562+0000) 2024-02-02T15:48:22.838 INFO:journalctl@ceph.osd.8.smithi142.stdout:Feb 02 15:48:22 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-8[67933]: 2024-02-02T15:48:22.476+0000 7f525db65700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:41.946745+0000 front 2024-02-02T15:47:41.946768+0000 (oldest deadline 2024-02-02T15:48:07.846739+0000) 2024-02-02T15:48:22.838 INFO:journalctl@ceph.osd.8.smithi142.stdout:Feb 02 15:48:22 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-8[67933]: 2024-02-02T15:48:22.476+0000 7f525db65700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:50.147681+0000 front 2024-02-02T15:47:50.147810+0000 (oldest deadline 2024-02-02T15:48:16.047900+0000) 2024-02-02T15:48:23.114 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:22 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:22.821+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.558440+0000 front 2024-02-02T15:47:33.558460+0000 (oldest deadline 2024-02-02T15:47:59.958896+0000) 2024-02-02T15:48:23.114 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:22 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:22.821+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:43.861259+0000 front 2024-02-02T15:47:43.861243+0000 (oldest deadline 2024-02-02T15:48:09.161042+0000) 2024-02-02T15:48:23.114 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:22 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:22.821+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:49.161607+0000 front 2024-02-02T15:47:49.161481+0000 (oldest deadline 2024-02-02T15:48:13.261576+0000) 2024-02-02T15:48:23.114 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:22 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:22.821+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:56.063095+0000 front 2024-02-02T15:47:56.063158+0000 (oldest deadline 2024-02-02T15:48:21.363242+0000) 2024-02-02T15:48:23.169 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:22 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:22.838+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:35.342656+0000 front 2024-02-02T15:47:35.342649+0000 (oldest deadline 2024-02-02T15:48:01.143219+0000) 2024-02-02T15:48:23.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:22 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:22.838+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:47.544669+0000 front 2024-02-02T15:47:47.544689+0000 (oldest deadline 2024-02-02T15:48:13.444898+0000) 2024-02-02T15:48:23.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:22 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:22.838+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:59.846650+0000 front 2024-02-02T15:47:59.846736+0000 (oldest deadline 2024-02-02T15:48:22.746483+0000) 2024-02-02T15:48:23.440 INFO:journalctl@ceph.osd.6.smithi001.stdout:Feb 02 15:48:23 smithi001 bash[132192]: ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-6 2024-02-02T15:48:23.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:23 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:23.002+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.102474+0000 front 2024-02-02T15:47:33.102427+0000 (oldest deadline 2024-02-02T15:47:57.802602+0000) 2024-02-02T15:48:23.441 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:23 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:23.002+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:44.106107+0000 front 2024-02-02T15:47:44.106141+0000 (oldest deadline 2024-02-02T15:48:07.006384+0000) 2024-02-02T15:48:23.441 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:23 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:23.002+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:51.707992+0000 front 2024-02-02T15:47:51.707872+0000 (oldest deadline 2024-02-02T15:48:15.808055+0000) 2024-02-02T15:48:23.441 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:23 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:23.002+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:59.309660+0000 front 2024-02-02T15:47:59.309683+0000 (oldest deadline 2024-02-02T15:48:22.809935+0000) 2024-02-02T15:48:23.532 INFO:journalctl@ceph.osd.7.smithi071.stdout:Feb 02 15:48:23 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-7[65443]: 2024-02-02T15:48:23.532+0000 7f66be2a3700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:40.134496+0000 front 2024-02-02T15:47:40.134524+0000 (oldest deadline 2024-02-02T15:48:06.034624+0000) 2024-02-02T15:48:23.532 INFO:journalctl@ceph.osd.7.smithi071.stdout:Feb 02 15:48:23 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-7[65443]: 2024-02-02T15:48:23.532+0000 7f66be2a3700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:47.135707+0000 front 2024-02-02T15:47:47.135621+0000 (oldest deadline 2024-02-02T15:48:13.035562+0000) 2024-02-02T15:48:23.533 INFO:journalctl@ceph.osd.7.smithi071.stdout:Feb 02 15:48:23 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-7[65443]: 2024-02-02T15:48:23.532+0000 7f66be2a3700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:57.637778+0000 front 2024-02-02T15:47:57.637675+0000 (oldest deadline 2024-02-02T15:48:22.937585+0000) 2024-02-02T15:48:23.841 INFO:journalctl@ceph.osd.8.smithi142.stdout:Feb 02 15:48:23 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-8[67933]: 2024-02-02T15:48:23.443+0000 7f525db65700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:41.946745+0000 front 2024-02-02T15:47:41.946768+0000 (oldest deadline 2024-02-02T15:48:07.846739+0000) 2024-02-02T15:48:23.841 INFO:journalctl@ceph.osd.8.smithi142.stdout:Feb 02 15:48:23 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-8[67933]: 2024-02-02T15:48:23.443+0000 7f525db65700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:50.147681+0000 front 2024-02-02T15:47:50.147810+0000 (oldest deadline 2024-02-02T15:48:16.047900+0000) 2024-02-02T15:48:23.841 INFO:journalctl@ceph.osd.8.smithi142.stdout:Feb 02 15:48:23 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-8[67933]: 2024-02-02T15:48:23.443+0000 7f525db65700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:59.350523+0000 front 2024-02-02T15:47:59.350409+0000 (oldest deadline 2024-02-02T15:48:22.850391+0000) 2024-02-02T15:48:24.114 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:23 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:23.785+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.558440+0000 front 2024-02-02T15:47:33.558460+0000 (oldest deadline 2024-02-02T15:47:59.958896+0000) 2024-02-02T15:48:24.114 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:23 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:23.785+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:43.861259+0000 front 2024-02-02T15:47:43.861243+0000 (oldest deadline 2024-02-02T15:48:09.161042+0000) 2024-02-02T15:48:24.114 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:23 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:23.785+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:49.161607+0000 front 2024-02-02T15:47:49.161481+0000 (oldest deadline 2024-02-02T15:48:13.261576+0000) 2024-02-02T15:48:24.114 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:23 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:23.785+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:56.063095+0000 front 2024-02-02T15:47:56.063158+0000 (oldest deadline 2024-02-02T15:48:21.363242+0000) 2024-02-02T15:48:24.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:23 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:23.841+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:35.342656+0000 front 2024-02-02T15:47:35.342649+0000 (oldest deadline 2024-02-02T15:48:01.143219+0000) 2024-02-02T15:48:24.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:23 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:23.841+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:47.544669+0000 front 2024-02-02T15:47:47.544689+0000 (oldest deadline 2024-02-02T15:48:13.444898+0000) 2024-02-02T15:48:24.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:23 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:23.841+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:59.846650+0000 front 2024-02-02T15:47:59.846736+0000 (oldest deadline 2024-02-02T15:48:22.746483+0000) 2024-02-02T15:48:24.397 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:24 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:24.005+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.102474+0000 front 2024-02-02T15:47:33.102427+0000 (oldest deadline 2024-02-02T15:47:57.802602+0000) 2024-02-02T15:48:24.397 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:24 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:24.005+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:44.106107+0000 front 2024-02-02T15:47:44.106141+0000 (oldest deadline 2024-02-02T15:48:07.006384+0000) 2024-02-02T15:48:24.397 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:24 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:24.005+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:51.707992+0000 front 2024-02-02T15:47:51.707872+0000 (oldest deadline 2024-02-02T15:48:15.808055+0000) 2024-02-02T15:48:24.397 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:24 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:24.005+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:59.309660+0000 front 2024-02-02T15:47:59.309683+0000 (oldest deadline 2024-02-02T15:48:22.809935+0000) 2024-02-02T15:48:24.625 DEBUG:teuthology.orchestra.run.smithi001:> sudo pkill -f 'journalctl -f -n 0 -u ceph-371a409c-c1de-11ee-95b6-87774f69a715@osd.6.service' 2024-02-02T15:48:24.669 INFO:journalctl@ceph.osd.8.smithi142.stdout:Feb 02 15:48:24 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-8[67933]: 2024-02-02T15:48:24.419+0000 7f525db65700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:41.946745+0000 front 2024-02-02T15:47:41.946768+0000 (oldest deadline 2024-02-02T15:48:07.846739+0000) 2024-02-02T15:48:24.670 INFO:journalctl@ceph.osd.8.smithi142.stdout:Feb 02 15:48:24 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-8[67933]: 2024-02-02T15:48:24.419+0000 7f525db65700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:50.147681+0000 front 2024-02-02T15:47:50.147810+0000 (oldest deadline 2024-02-02T15:48:16.047900+0000) 2024-02-02T15:48:24.670 INFO:journalctl@ceph.osd.8.smithi142.stdout:Feb 02 15:48:24 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-8[67933]: 2024-02-02T15:48:24.419+0000 7f525db65700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:59.350523+0000 front 2024-02-02T15:47:59.350409+0000 (oldest deadline 2024-02-02T15:48:22.850391+0000) 2024-02-02T15:48:24.684 INFO:journalctl@ceph.osd.6.smithi001.stdout:Feb 02 15:48:24 smithi001 systemd[1]: ceph-371a409c-c1de-11ee-95b6-87774f69a715@osd.6.service: Succeeded. 2024-02-02T15:48:24.684 INFO:journalctl@ceph.osd.6.smithi001.stdout:Feb 02 15:48:24 smithi001 systemd[1]: Stopped Ceph osd.6 for 371a409c-c1de-11ee-95b6-87774f69a715. 2024-02-02T15:48:24.710 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-02T15:48:24.710 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-02-02T15:48:24.710 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-02-02T15:48:24.711 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-371a409c-c1de-11ee-95b6-87774f69a715@osd.7 2024-02-02T15:48:24.740 INFO:journalctl@ceph.osd.7.smithi071.stdout:Feb 02 15:48:24 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-7[65443]: 2024-02-02T15:48:24.486+0000 7f66be2a3700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:40.134496+0000 front 2024-02-02T15:47:40.134524+0000 (oldest deadline 2024-02-02T15:48:06.034624+0000) 2024-02-02T15:48:24.741 INFO:journalctl@ceph.osd.7.smithi071.stdout:Feb 02 15:48:24 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-7[65443]: 2024-02-02T15:48:24.486+0000 7f66be2a3700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:47.135707+0000 front 2024-02-02T15:47:47.135621+0000 (oldest deadline 2024-02-02T15:48:13.035562+0000) 2024-02-02T15:48:24.741 INFO:journalctl@ceph.osd.7.smithi071.stdout:Feb 02 15:48:24 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-7[65443]: 2024-02-02T15:48:24.486+0000 7f66be2a3700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:57.637778+0000 front 2024-02-02T15:47:57.637675+0000 (oldest deadline 2024-02-02T15:48:22.937585+0000) 2024-02-02T15:48:25.114 INFO:journalctl@ceph.osd.7.smithi071.stdout:Feb 02 15:48:24 smithi071 systemd[1]: Stopping Ceph osd.7 for 371a409c-c1de-11ee-95b6-87774f69a715... 2024-02-02T15:48:25.114 INFO:journalctl@ceph.osd.7.smithi071.stdout:Feb 02 15:48:24 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-7[65443]: 2024-02-02T15:48:24.957+0000 7f66c1cad700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-02T15:48:25.114 INFO:journalctl@ceph.osd.7.smithi071.stdout:Feb 02 15:48:24 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-7[65443]: 2024-02-02T15:48:24.957+0000 7f66c1cad700 -1 osd.7 121 *** Got signal Terminated *** 2024-02-02T15:48:25.115 INFO:journalctl@ceph.osd.7.smithi071.stdout:Feb 02 15:48:24 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-7[65443]: 2024-02-02T15:48:24.957+0000 7f66c1cad700 -1 osd.7 121 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-02T15:48:25.115 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:24 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:24.794+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.558440+0000 front 2024-02-02T15:47:33.558460+0000 (oldest deadline 2024-02-02T15:47:59.958896+0000) 2024-02-02T15:48:25.115 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:24 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:24.794+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:43.861259+0000 front 2024-02-02T15:47:43.861243+0000 (oldest deadline 2024-02-02T15:48:09.161042+0000) 2024-02-02T15:48:25.115 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:24 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:24.794+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:49.161607+0000 front 2024-02-02T15:47:49.161481+0000 (oldest deadline 2024-02-02T15:48:13.261576+0000) 2024-02-02T15:48:25.115 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:24 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:24.794+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:56.063095+0000 front 2024-02-02T15:47:56.063158+0000 (oldest deadline 2024-02-02T15:48:21.363242+0000) 2024-02-02T15:48:25.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:24 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:24.794+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:35.342656+0000 front 2024-02-02T15:47:35.342649+0000 (oldest deadline 2024-02-02T15:48:01.143219+0000) 2024-02-02T15:48:25.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:24 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:24.794+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:47.544669+0000 front 2024-02-02T15:47:47.544689+0000 (oldest deadline 2024-02-02T15:48:13.444898+0000) 2024-02-02T15:48:25.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:24 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:24.794+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:59.846650+0000 front 2024-02-02T15:47:59.846736+0000 (oldest deadline 2024-02-02T15:48:22.746483+0000) 2024-02-02T15:48:25.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:25 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:25.023+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.102474+0000 front 2024-02-02T15:47:33.102427+0000 (oldest deadline 2024-02-02T15:47:57.802602+0000) 2024-02-02T15:48:25.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:25 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:25.023+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:44.106107+0000 front 2024-02-02T15:47:44.106141+0000 (oldest deadline 2024-02-02T15:48:07.006384+0000) 2024-02-02T15:48:25.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:25 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:25.023+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:51.707992+0000 front 2024-02-02T15:47:51.707872+0000 (oldest deadline 2024-02-02T15:48:15.808055+0000) 2024-02-02T15:48:25.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:25 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:25.023+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:59.309660+0000 front 2024-02-02T15:47:59.309683+0000 (oldest deadline 2024-02-02T15:48:22.809935+0000) 2024-02-02T15:48:25.670 INFO:journalctl@ceph.osd.8.smithi142.stdout:Feb 02 15:48:25 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-8[67933]: 2024-02-02T15:48:25.413+0000 7f525db65700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:41.946745+0000 front 2024-02-02T15:47:41.946768+0000 (oldest deadline 2024-02-02T15:48:07.846739+0000) 2024-02-02T15:48:25.670 INFO:journalctl@ceph.osd.8.smithi142.stdout:Feb 02 15:48:25 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-8[67933]: 2024-02-02T15:48:25.413+0000 7f525db65700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:50.147681+0000 front 2024-02-02T15:47:50.147810+0000 (oldest deadline 2024-02-02T15:48:16.047900+0000) 2024-02-02T15:48:25.670 INFO:journalctl@ceph.osd.8.smithi142.stdout:Feb 02 15:48:25 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-8[67933]: 2024-02-02T15:48:25.413+0000 7f525db65700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:59.350523+0000 front 2024-02-02T15:47:59.350409+0000 (oldest deadline 2024-02-02T15:48:22.850391+0000) 2024-02-02T15:48:25.864 INFO:journalctl@ceph.osd.7.smithi071.stdout:Feb 02 15:48:25 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-7[65443]: 2024-02-02T15:48:25.503+0000 7f66be2a3700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:40.134496+0000 front 2024-02-02T15:47:40.134524+0000 (oldest deadline 2024-02-02T15:48:06.034624+0000) 2024-02-02T15:48:25.864 INFO:journalctl@ceph.osd.7.smithi071.stdout:Feb 02 15:48:25 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-7[65443]: 2024-02-02T15:48:25.503+0000 7f66be2a3700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:47.135707+0000 front 2024-02-02T15:47:47.135621+0000 (oldest deadline 2024-02-02T15:48:13.035562+0000) 2024-02-02T15:48:25.864 INFO:journalctl@ceph.osd.7.smithi071.stdout:Feb 02 15:48:25 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-7[65443]: 2024-02-02T15:48:25.503+0000 7f66be2a3700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:57.637778+0000 front 2024-02-02T15:47:57.637675+0000 (oldest deadline 2024-02-02T15:48:22.937585+0000) 2024-02-02T15:48:25.865 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:25 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:25.748+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.558440+0000 front 2024-02-02T15:47:33.558460+0000 (oldest deadline 2024-02-02T15:47:59.958896+0000) 2024-02-02T15:48:25.865 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:25 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:25.748+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:43.861259+0000 front 2024-02-02T15:47:43.861243+0000 (oldest deadline 2024-02-02T15:48:09.161042+0000) 2024-02-02T15:48:25.865 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:25 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:25.748+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:49.161607+0000 front 2024-02-02T15:47:49.161481+0000 (oldest deadline 2024-02-02T15:48:13.261576+0000) 2024-02-02T15:48:25.865 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:25 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:25.748+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:56.063095+0000 front 2024-02-02T15:47:56.063158+0000 (oldest deadline 2024-02-02T15:48:21.363242+0000) 2024-02-02T15:48:26.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:25 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:25.757+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:35.342656+0000 front 2024-02-02T15:47:35.342649+0000 (oldest deadline 2024-02-02T15:48:01.143219+0000) 2024-02-02T15:48:26.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:25 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:25.757+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:47.544669+0000 front 2024-02-02T15:47:47.544689+0000 (oldest deadline 2024-02-02T15:48:13.444898+0000) 2024-02-02T15:48:26.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:25 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:25.757+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:59.846650+0000 front 2024-02-02T15:47:59.846736+0000 (oldest deadline 2024-02-02T15:48:22.746483+0000) 2024-02-02T15:48:26.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:25 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:25.974+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.102474+0000 front 2024-02-02T15:47:33.102427+0000 (oldest deadline 2024-02-02T15:47:57.802602+0000) 2024-02-02T15:48:26.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:25 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:25.974+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:44.106107+0000 front 2024-02-02T15:47:44.106141+0000 (oldest deadline 2024-02-02T15:48:07.006384+0000) 2024-02-02T15:48:26.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:25 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:25.974+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:51.707992+0000 front 2024-02-02T15:47:51.707872+0000 (oldest deadline 2024-02-02T15:48:15.808055+0000) 2024-02-02T15:48:26.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:25 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:25.974+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:59.309660+0000 front 2024-02-02T15:47:59.309683+0000 (oldest deadline 2024-02-02T15:48:22.809935+0000) 2024-02-02T15:48:26.765 INFO:journalctl@ceph.osd.8.smithi142.stdout:Feb 02 15:48:26 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-8[67933]: 2024-02-02T15:48:26.434+0000 7f525db65700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:41.946745+0000 front 2024-02-02T15:47:41.946768+0000 (oldest deadline 2024-02-02T15:48:07.846739+0000) 2024-02-02T15:48:26.765 INFO:journalctl@ceph.osd.8.smithi142.stdout:Feb 02 15:48:26 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-8[67933]: 2024-02-02T15:48:26.434+0000 7f525db65700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:50.147681+0000 front 2024-02-02T15:47:50.147810+0000 (oldest deadline 2024-02-02T15:48:16.047900+0000) 2024-02-02T15:48:26.765 INFO:journalctl@ceph.osd.8.smithi142.stdout:Feb 02 15:48:26 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-8[67933]: 2024-02-02T15:48:26.434+0000 7f525db65700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:59.350523+0000 front 2024-02-02T15:47:59.350409+0000 (oldest deadline 2024-02-02T15:48:22.850391+0000) 2024-02-02T15:48:26.864 INFO:journalctl@ceph.osd.7.smithi071.stdout:Feb 02 15:48:26 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-7[65443]: 2024-02-02T15:48:26.543+0000 7f66be2a3700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:40.134496+0000 front 2024-02-02T15:47:40.134524+0000 (oldest deadline 2024-02-02T15:48:06.034624+0000) 2024-02-02T15:48:26.864 INFO:journalctl@ceph.osd.7.smithi071.stdout:Feb 02 15:48:26 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-7[65443]: 2024-02-02T15:48:26.543+0000 7f66be2a3700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:47.135707+0000 front 2024-02-02T15:47:47.135621+0000 (oldest deadline 2024-02-02T15:48:13.035562+0000) 2024-02-02T15:48:26.864 INFO:journalctl@ceph.osd.7.smithi071.stdout:Feb 02 15:48:26 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-7[65443]: 2024-02-02T15:48:26.543+0000 7f66be2a3700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:57.637778+0000 front 2024-02-02T15:47:57.637675+0000 (oldest deadline 2024-02-02T15:48:22.937585+0000) 2024-02-02T15:48:26.868 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:26 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:26.770+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.558440+0000 front 2024-02-02T15:47:33.558460+0000 (oldest deadline 2024-02-02T15:47:59.958896+0000) 2024-02-02T15:48:26.868 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:26 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:26.770+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:43.861259+0000 front 2024-02-02T15:47:43.861243+0000 (oldest deadline 2024-02-02T15:48:09.161042+0000) 2024-02-02T15:48:26.868 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:26 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:26.770+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:49.161607+0000 front 2024-02-02T15:47:49.161481+0000 (oldest deadline 2024-02-02T15:48:13.261576+0000) 2024-02-02T15:48:26.868 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:26 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:26.770+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:56.063095+0000 front 2024-02-02T15:47:56.063158+0000 (oldest deadline 2024-02-02T15:48:21.363242+0000) 2024-02-02T15:48:27.169 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:26 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:26.765+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:35.342656+0000 front 2024-02-02T15:47:35.342649+0000 (oldest deadline 2024-02-02T15:48:01.143219+0000) 2024-02-02T15:48:27.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:26 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:26.765+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:47.544669+0000 front 2024-02-02T15:47:47.544689+0000 (oldest deadline 2024-02-02T15:48:13.444898+0000) 2024-02-02T15:48:27.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:26 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:26.765+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:59.846650+0000 front 2024-02-02T15:47:59.846736+0000 (oldest deadline 2024-02-02T15:48:22.746483+0000) 2024-02-02T15:48:27.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:26 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:26.959+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.102474+0000 front 2024-02-02T15:47:33.102427+0000 (oldest deadline 2024-02-02T15:47:57.802602+0000) 2024-02-02T15:48:27.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:26 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:26.959+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:44.106107+0000 front 2024-02-02T15:47:44.106141+0000 (oldest deadline 2024-02-02T15:48:07.006384+0000) 2024-02-02T15:48:27.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:26 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:26.959+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:51.707992+0000 front 2024-02-02T15:47:51.707872+0000 (oldest deadline 2024-02-02T15:48:15.808055+0000) 2024-02-02T15:48:27.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:26 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:26.959+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:59.309660+0000 front 2024-02-02T15:47:59.309683+0000 (oldest deadline 2024-02-02T15:48:22.809935+0000) 2024-02-02T15:48:27.669 INFO:journalctl@ceph.osd.8.smithi142.stdout:Feb 02 15:48:27 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-8[67933]: 2024-02-02T15:48:27.395+0000 7f525db65700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:41.946745+0000 front 2024-02-02T15:47:41.946768+0000 (oldest deadline 2024-02-02T15:48:07.846739+0000) 2024-02-02T15:48:27.670 INFO:journalctl@ceph.osd.8.smithi142.stdout:Feb 02 15:48:27 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-8[67933]: 2024-02-02T15:48:27.395+0000 7f525db65700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:50.147681+0000 front 2024-02-02T15:47:50.147810+0000 (oldest deadline 2024-02-02T15:48:16.047900+0000) 2024-02-02T15:48:27.670 INFO:journalctl@ceph.osd.8.smithi142.stdout:Feb 02 15:48:27 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-8[67933]: 2024-02-02T15:48:27.395+0000 7f525db65700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:59.350523+0000 front 2024-02-02T15:47:59.350409+0000 (oldest deadline 2024-02-02T15:48:22.850391+0000) 2024-02-02T15:48:27.803 INFO:journalctl@ceph.osd.7.smithi071.stdout:Feb 02 15:48:27 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-7[65443]: 2024-02-02T15:48:27.544+0000 7f66be2a3700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:40.134496+0000 front 2024-02-02T15:47:40.134524+0000 (oldest deadline 2024-02-02T15:48:06.034624+0000) 2024-02-02T15:48:27.804 INFO:journalctl@ceph.osd.7.smithi071.stdout:Feb 02 15:48:27 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-7[65443]: 2024-02-02T15:48:27.544+0000 7f66be2a3700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:47.135707+0000 front 2024-02-02T15:47:47.135621+0000 (oldest deadline 2024-02-02T15:48:13.035562+0000) 2024-02-02T15:48:27.804 INFO:journalctl@ceph.osd.7.smithi071.stdout:Feb 02 15:48:27 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-7[65443]: 2024-02-02T15:48:27.544+0000 7f66be2a3700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:57.637778+0000 front 2024-02-02T15:47:57.637675+0000 (oldest deadline 2024-02-02T15:48:22.937585+0000) 2024-02-02T15:48:28.114 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:27 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:27.805+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.558440+0000 front 2024-02-02T15:47:33.558460+0000 (oldest deadline 2024-02-02T15:47:59.958896+0000) 2024-02-02T15:48:28.114 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:27 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:27.805+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:43.861259+0000 front 2024-02-02T15:47:43.861243+0000 (oldest deadline 2024-02-02T15:48:09.161042+0000) 2024-02-02T15:48:28.114 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:27 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:27.805+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:49.161607+0000 front 2024-02-02T15:47:49.161481+0000 (oldest deadline 2024-02-02T15:48:13.261576+0000) 2024-02-02T15:48:28.114 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:27 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:27.805+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:56.063095+0000 front 2024-02-02T15:47:56.063158+0000 (oldest deadline 2024-02-02T15:48:21.363242+0000) 2024-02-02T15:48:28.169 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:27 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:27.779+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:35.342656+0000 front 2024-02-02T15:47:35.342649+0000 (oldest deadline 2024-02-02T15:48:01.143219+0000) 2024-02-02T15:48:28.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:27 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:27.779+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:47.544669+0000 front 2024-02-02T15:47:47.544689+0000 (oldest deadline 2024-02-02T15:48:13.444898+0000) 2024-02-02T15:48:28.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:27 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:27.779+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:59.846650+0000 front 2024-02-02T15:47:59.846736+0000 (oldest deadline 2024-02-02T15:48:22.746483+0000) 2024-02-02T15:48:28.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:27 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:27.980+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.102474+0000 front 2024-02-02T15:47:33.102427+0000 (oldest deadline 2024-02-02T15:47:57.802602+0000) 2024-02-02T15:48:28.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:27 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:27.980+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:44.106107+0000 front 2024-02-02T15:47:44.106141+0000 (oldest deadline 2024-02-02T15:48:07.006384+0000) 2024-02-02T15:48:28.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:27 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:27.980+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:51.707992+0000 front 2024-02-02T15:47:51.707872+0000 (oldest deadline 2024-02-02T15:48:15.808055+0000) 2024-02-02T15:48:28.441 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:27 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:27.980+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:59.309660+0000 front 2024-02-02T15:47:59.309683+0000 (oldest deadline 2024-02-02T15:48:22.809935+0000) 2024-02-02T15:48:28.670 INFO:journalctl@ceph.osd.8.smithi142.stdout:Feb 02 15:48:28 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-8[67933]: 2024-02-02T15:48:28.354+0000 7f525db65700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:41.946745+0000 front 2024-02-02T15:47:41.946768+0000 (oldest deadline 2024-02-02T15:48:07.846739+0000) 2024-02-02T15:48:28.670 INFO:journalctl@ceph.osd.8.smithi142.stdout:Feb 02 15:48:28 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-8[67933]: 2024-02-02T15:48:28.354+0000 7f525db65700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:50.147681+0000 front 2024-02-02T15:47:50.147810+0000 (oldest deadline 2024-02-02T15:48:16.047900+0000) 2024-02-02T15:48:28.670 INFO:journalctl@ceph.osd.8.smithi142.stdout:Feb 02 15:48:28 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-8[67933]: 2024-02-02T15:48:28.354+0000 7f525db65700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:59.350523+0000 front 2024-02-02T15:47:59.350409+0000 (oldest deadline 2024-02-02T15:48:22.850391+0000) 2024-02-02T15:48:28.864 INFO:journalctl@ceph.osd.7.smithi071.stdout:Feb 02 15:48:28 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-7[65443]: 2024-02-02T15:48:28.589+0000 7f66be2a3700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:40.134496+0000 front 2024-02-02T15:47:40.134524+0000 (oldest deadline 2024-02-02T15:48:06.034624+0000) 2024-02-02T15:48:28.864 INFO:journalctl@ceph.osd.7.smithi071.stdout:Feb 02 15:48:28 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-7[65443]: 2024-02-02T15:48:28.589+0000 7f66be2a3700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:47.135707+0000 front 2024-02-02T15:47:47.135621+0000 (oldest deadline 2024-02-02T15:48:13.035562+0000) 2024-02-02T15:48:28.864 INFO:journalctl@ceph.osd.7.smithi071.stdout:Feb 02 15:48:28 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-7[65443]: 2024-02-02T15:48:28.589+0000 7f66be2a3700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:57.637778+0000 front 2024-02-02T15:47:57.637675+0000 (oldest deadline 2024-02-02T15:48:22.937585+0000) 2024-02-02T15:48:28.864 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:28 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:28.835+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.558440+0000 front 2024-02-02T15:47:33.558460+0000 (oldest deadline 2024-02-02T15:47:59.958896+0000) 2024-02-02T15:48:28.865 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:28 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:28.835+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:43.861259+0000 front 2024-02-02T15:47:43.861243+0000 (oldest deadline 2024-02-02T15:48:09.161042+0000) 2024-02-02T15:48:28.865 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:28 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:28.835+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:49.161607+0000 front 2024-02-02T15:47:49.161481+0000 (oldest deadline 2024-02-02T15:48:13.261576+0000) 2024-02-02T15:48:28.865 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:28 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:28.835+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:56.063095+0000 front 2024-02-02T15:47:56.063158+0000 (oldest deadline 2024-02-02T15:48:21.363242+0000) 2024-02-02T15:48:29.169 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:28 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:28.803+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:35.342656+0000 front 2024-02-02T15:47:35.342649+0000 (oldest deadline 2024-02-02T15:48:01.143219+0000) 2024-02-02T15:48:29.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:28 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:28.803+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:47.544669+0000 front 2024-02-02T15:47:47.544689+0000 (oldest deadline 2024-02-02T15:48:13.444898+0000) 2024-02-02T15:48:29.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:28 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:28.803+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:59.846650+0000 front 2024-02-02T15:47:59.846736+0000 (oldest deadline 2024-02-02T15:48:22.746483+0000) 2024-02-02T15:48:29.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:28 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:28.803+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-02-02T15:48:03.247880+0000 front 2024-02-02T15:48:03.247853+0000 (oldest deadline 2024-02-02T15:48:28.547857+0000) 2024-02-02T15:48:29.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:29 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:29.012+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.102474+0000 front 2024-02-02T15:47:33.102427+0000 (oldest deadline 2024-02-02T15:47:57.802602+0000) 2024-02-02T15:48:29.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:29 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:29.012+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:44.106107+0000 front 2024-02-02T15:47:44.106141+0000 (oldest deadline 2024-02-02T15:48:07.006384+0000) 2024-02-02T15:48:29.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:29 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:29.012+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:51.707992+0000 front 2024-02-02T15:47:51.707872+0000 (oldest deadline 2024-02-02T15:48:15.808055+0000) 2024-02-02T15:48:29.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:29 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:29.012+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:59.309660+0000 front 2024-02-02T15:47:59.309683+0000 (oldest deadline 2024-02-02T15:48:22.809935+0000) 2024-02-02T15:48:29.670 INFO:journalctl@ceph.osd.8.smithi142.stdout:Feb 02 15:48:29 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-8[67933]: 2024-02-02T15:48:29.344+0000 7f525db65700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:41.946745+0000 front 2024-02-02T15:47:41.946768+0000 (oldest deadline 2024-02-02T15:48:07.846739+0000) 2024-02-02T15:48:29.670 INFO:journalctl@ceph.osd.8.smithi142.stdout:Feb 02 15:48:29 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-8[67933]: 2024-02-02T15:48:29.344+0000 7f525db65700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:50.147681+0000 front 2024-02-02T15:47:50.147810+0000 (oldest deadline 2024-02-02T15:48:16.047900+0000) 2024-02-02T15:48:29.670 INFO:journalctl@ceph.osd.8.smithi142.stdout:Feb 02 15:48:29 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-8[67933]: 2024-02-02T15:48:29.344+0000 7f525db65700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:59.350523+0000 front 2024-02-02T15:47:59.350409+0000 (oldest deadline 2024-02-02T15:48:22.850391+0000) 2024-02-02T15:48:29.850 INFO:journalctl@ceph.osd.7.smithi071.stdout:Feb 02 15:48:29 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-7[65443]: 2024-02-02T15:48:29.540+0000 7f66be2a3700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:40.134496+0000 front 2024-02-02T15:47:40.134524+0000 (oldest deadline 2024-02-02T15:48:06.034624+0000) 2024-02-02T15:48:29.850 INFO:journalctl@ceph.osd.7.smithi071.stdout:Feb 02 15:48:29 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-7[65443]: 2024-02-02T15:48:29.540+0000 7f66be2a3700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:47.135707+0000 front 2024-02-02T15:47:47.135621+0000 (oldest deadline 2024-02-02T15:48:13.035562+0000) 2024-02-02T15:48:29.850 INFO:journalctl@ceph.osd.7.smithi071.stdout:Feb 02 15:48:29 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-7[65443]: 2024-02-02T15:48:29.540+0000 7f66be2a3700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:57.637778+0000 front 2024-02-02T15:47:57.637675+0000 (oldest deadline 2024-02-02T15:48:22.937585+0000) 2024-02-02T15:48:29.850 INFO:journalctl@ceph.osd.7.smithi071.stdout:Feb 02 15:48:29 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-7[65443]: 2024-02-02T15:48:29.540+0000 7f66be2a3700 -1 osd.7 121 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-02-02T15:48:04.638552+0000 front 2024-02-02T15:48:04.638573+0000 (oldest deadline 2024-02-02T15:48:28.738714+0000) 2024-02-02T15:48:30.114 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:29 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:29.851+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.558440+0000 front 2024-02-02T15:47:33.558460+0000 (oldest deadline 2024-02-02T15:47:59.958896+0000) 2024-02-02T15:48:30.114 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:29 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:29.851+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:43.861259+0000 front 2024-02-02T15:47:43.861243+0000 (oldest deadline 2024-02-02T15:48:09.161042+0000) 2024-02-02T15:48:30.114 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:29 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:29.851+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:49.161607+0000 front 2024-02-02T15:47:49.161481+0000 (oldest deadline 2024-02-02T15:48:13.261576+0000) 2024-02-02T15:48:30.115 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:29 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:29.851+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:56.063095+0000 front 2024-02-02T15:47:56.063158+0000 (oldest deadline 2024-02-02T15:48:21.363242+0000) 2024-02-02T15:48:30.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:29 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:29.755+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:35.342656+0000 front 2024-02-02T15:47:35.342649+0000 (oldest deadline 2024-02-02T15:48:01.143219+0000) 2024-02-02T15:48:30.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:29 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:29.755+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:47.544669+0000 front 2024-02-02T15:47:47.544689+0000 (oldest deadline 2024-02-02T15:48:13.444898+0000) 2024-02-02T15:48:30.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:29 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:29.756+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:59.846650+0000 front 2024-02-02T15:47:59.846736+0000 (oldest deadline 2024-02-02T15:48:22.746483+0000) 2024-02-02T15:48:30.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:29 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:29.756+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-02-02T15:48:03.247880+0000 front 2024-02-02T15:48:03.247853+0000 (oldest deadline 2024-02-02T15:48:28.547857+0000) 2024-02-02T15:48:30.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:30 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:30.053+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.102474+0000 front 2024-02-02T15:47:33.102427+0000 (oldest deadline 2024-02-02T15:47:57.802602+0000) 2024-02-02T15:48:30.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:30 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:30.053+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:44.106107+0000 front 2024-02-02T15:47:44.106141+0000 (oldest deadline 2024-02-02T15:48:07.006384+0000) 2024-02-02T15:48:30.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:30 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:30.053+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:51.707992+0000 front 2024-02-02T15:47:51.707872+0000 (oldest deadline 2024-02-02T15:48:15.808055+0000) 2024-02-02T15:48:30.441 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:30 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:30.053+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:59.309660+0000 front 2024-02-02T15:47:59.309683+0000 (oldest deadline 2024-02-02T15:48:22.809935+0000) 2024-02-02T15:48:30.441 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:30 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:30.053+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-02-02T15:48:03.911403+0000 front 2024-02-02T15:48:03.911524+0000 (oldest deadline 2024-02-02T15:48:29.211532+0000) 2024-02-02T15:48:30.670 INFO:journalctl@ceph.osd.8.smithi142.stdout:Feb 02 15:48:30 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-8[67933]: 2024-02-02T15:48:30.306+0000 7f525db65700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:41.946745+0000 front 2024-02-02T15:47:41.946768+0000 (oldest deadline 2024-02-02T15:48:07.846739+0000) 2024-02-02T15:48:30.670 INFO:journalctl@ceph.osd.8.smithi142.stdout:Feb 02 15:48:30 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-8[67933]: 2024-02-02T15:48:30.306+0000 7f525db65700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:50.147681+0000 front 2024-02-02T15:47:50.147810+0000 (oldest deadline 2024-02-02T15:48:16.047900+0000) 2024-02-02T15:48:30.670 INFO:journalctl@ceph.osd.8.smithi142.stdout:Feb 02 15:48:30 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-8[67933]: 2024-02-02T15:48:30.306+0000 7f525db65700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:59.350523+0000 front 2024-02-02T15:47:59.350409+0000 (oldest deadline 2024-02-02T15:48:22.850391+0000) 2024-02-02T15:48:31.114 INFO:journalctl@ceph.osd.7.smithi071.stdout:Feb 02 15:48:30 smithi071 bash[89575]: ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-7 2024-02-02T15:48:31.114 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:30 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:30.823+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.558440+0000 front 2024-02-02T15:47:33.558460+0000 (oldest deadline 2024-02-02T15:47:59.958896+0000) 2024-02-02T15:48:31.114 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:30 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:30.823+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:43.861259+0000 front 2024-02-02T15:47:43.861243+0000 (oldest deadline 2024-02-02T15:48:09.161042+0000) 2024-02-02T15:48:31.114 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:30 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:30.823+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:49.161607+0000 front 2024-02-02T15:47:49.161481+0000 (oldest deadline 2024-02-02T15:48:13.261576+0000) 2024-02-02T15:48:31.115 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:30 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:30.823+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:56.063095+0000 front 2024-02-02T15:47:56.063158+0000 (oldest deadline 2024-02-02T15:48:21.363242+0000) 2024-02-02T15:48:31.115 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:30 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:30.823+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-02-02T15:48:07.264171+0000 front 2024-02-02T15:48:07.264186+0000 (oldest deadline 2024-02-02T15:48:30.164422+0000) 2024-02-02T15:48:31.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:30 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:30.802+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:35.342656+0000 front 2024-02-02T15:47:35.342649+0000 (oldest deadline 2024-02-02T15:48:01.143219+0000) 2024-02-02T15:48:31.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:30 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:30.802+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:47.544669+0000 front 2024-02-02T15:47:47.544689+0000 (oldest deadline 2024-02-02T15:48:13.444898+0000) 2024-02-02T15:48:31.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:30 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:30.802+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:59.846650+0000 front 2024-02-02T15:47:59.846736+0000 (oldest deadline 2024-02-02T15:48:22.746483+0000) 2024-02-02T15:48:31.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:30 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:30.802+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-02-02T15:48:03.247880+0000 front 2024-02-02T15:48:03.247853+0000 (oldest deadline 2024-02-02T15:48:28.547857+0000) 2024-02-02T15:48:31.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:31 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:31.043+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.102474+0000 front 2024-02-02T15:47:33.102427+0000 (oldest deadline 2024-02-02T15:47:57.802602+0000) 2024-02-02T15:48:31.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:31 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:31.043+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:44.106107+0000 front 2024-02-02T15:47:44.106141+0000 (oldest deadline 2024-02-02T15:48:07.006384+0000) 2024-02-02T15:48:31.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:31 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:31.043+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:51.707992+0000 front 2024-02-02T15:47:51.707872+0000 (oldest deadline 2024-02-02T15:48:15.808055+0000) 2024-02-02T15:48:31.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:31 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:31.043+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:59.309660+0000 front 2024-02-02T15:47:59.309683+0000 (oldest deadline 2024-02-02T15:48:22.809935+0000) 2024-02-02T15:48:31.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:31 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:31.043+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-02-02T15:48:03.911403+0000 front 2024-02-02T15:48:03.911524+0000 (oldest deadline 2024-02-02T15:48:29.211532+0000) 2024-02-02T15:48:31.670 INFO:journalctl@ceph.osd.8.smithi142.stdout:Feb 02 15:48:31 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-8[67933]: 2024-02-02T15:48:31.314+0000 7f525db65700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:41.946745+0000 front 2024-02-02T15:47:41.946768+0000 (oldest deadline 2024-02-02T15:48:07.846739+0000) 2024-02-02T15:48:31.670 INFO:journalctl@ceph.osd.8.smithi142.stdout:Feb 02 15:48:31 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-8[67933]: 2024-02-02T15:48:31.314+0000 7f525db65700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:50.147681+0000 front 2024-02-02T15:47:50.147810+0000 (oldest deadline 2024-02-02T15:48:16.047900+0000) 2024-02-02T15:48:31.670 INFO:journalctl@ceph.osd.8.smithi142.stdout:Feb 02 15:48:31 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-8[67933]: 2024-02-02T15:48:31.314+0000 7f525db65700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:59.350523+0000 front 2024-02-02T15:47:59.350409+0000 (oldest deadline 2024-02-02T15:48:22.850391+0000) 2024-02-02T15:48:31.864 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:31 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:31.857+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.558440+0000 front 2024-02-02T15:47:33.558460+0000 (oldest deadline 2024-02-02T15:47:59.958896+0000) 2024-02-02T15:48:31.864 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:31 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:31.857+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:43.861259+0000 front 2024-02-02T15:47:43.861243+0000 (oldest deadline 2024-02-02T15:48:09.161042+0000) 2024-02-02T15:48:31.865 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:31 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:31.857+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:49.161607+0000 front 2024-02-02T15:47:49.161481+0000 (oldest deadline 2024-02-02T15:48:13.261576+0000) 2024-02-02T15:48:31.865 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:31 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:31.857+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:56.063095+0000 front 2024-02-02T15:47:56.063158+0000 (oldest deadline 2024-02-02T15:48:21.363242+0000) 2024-02-02T15:48:31.865 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:31 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:31.857+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-02-02T15:48:07.264171+0000 front 2024-02-02T15:48:07.264186+0000 (oldest deadline 2024-02-02T15:48:30.164422+0000) 2024-02-02T15:48:31.934 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-371a409c-c1de-11ee-95b6-87774f69a715@osd.7.service' 2024-02-02T15:48:32.016 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-02T15:48:32.016 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-02-02T15:48:32.016 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-02-02T15:48:32.017 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-371a409c-c1de-11ee-95b6-87774f69a715@osd.8 2024-02-02T15:48:32.046 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:31 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:31.791+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:35.342656+0000 front 2024-02-02T15:47:35.342649+0000 (oldest deadline 2024-02-02T15:48:01.143219+0000) 2024-02-02T15:48:32.047 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:31 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:31.791+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:47.544669+0000 front 2024-02-02T15:47:47.544689+0000 (oldest deadline 2024-02-02T15:48:13.444898+0000) 2024-02-02T15:48:32.047 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:31 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:31.791+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:59.846650+0000 front 2024-02-02T15:47:59.846736+0000 (oldest deadline 2024-02-02T15:48:22.746483+0000) 2024-02-02T15:48:32.047 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:31 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:31.791+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-02-02T15:48:03.247880+0000 front 2024-02-02T15:48:03.247853+0000 (oldest deadline 2024-02-02T15:48:28.547857+0000) 2024-02-02T15:48:32.361 INFO:journalctl@ceph.osd.8.smithi142.stdout:Feb 02 15:48:32 smithi142 systemd[1]: Stopping Ceph osd.8 for 371a409c-c1de-11ee-95b6-87774f69a715... 2024-02-02T15:48:32.361 INFO:journalctl@ceph.osd.8.smithi142.stdout:Feb 02 15:48:32 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-8[67933]: 2024-02-02T15:48:32.251+0000 7f526156f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-02T15:48:32.361 INFO:journalctl@ceph.osd.8.smithi142.stdout:Feb 02 15:48:32 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-8[67933]: 2024-02-02T15:48:32.251+0000 7f526156f700 -1 osd.8 121 *** Got signal Terminated *** 2024-02-02T15:48:32.361 INFO:journalctl@ceph.osd.8.smithi142.stdout:Feb 02 15:48:32 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-8[67933]: 2024-02-02T15:48:32.251+0000 7f526156f700 -1 osd.8 121 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-02T15:48:32.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:32 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:32.018+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.102474+0000 front 2024-02-02T15:47:33.102427+0000 (oldest deadline 2024-02-02T15:47:57.802602+0000) 2024-02-02T15:48:32.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:32 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:32.018+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:44.106107+0000 front 2024-02-02T15:47:44.106141+0000 (oldest deadline 2024-02-02T15:48:07.006384+0000) 2024-02-02T15:48:32.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:32 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:32.018+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:51.707992+0000 front 2024-02-02T15:47:51.707872+0000 (oldest deadline 2024-02-02T15:48:15.808055+0000) 2024-02-02T15:48:32.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:32 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:32.018+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:59.309660+0000 front 2024-02-02T15:47:59.309683+0000 (oldest deadline 2024-02-02T15:48:22.809935+0000) 2024-02-02T15:48:32.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:32 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:32.018+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-02-02T15:48:03.911403+0000 front 2024-02-02T15:48:03.911524+0000 (oldest deadline 2024-02-02T15:48:29.211532+0000) 2024-02-02T15:48:32.670 INFO:journalctl@ceph.osd.8.smithi142.stdout:Feb 02 15:48:32 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-8[67933]: 2024-02-02T15:48:32.362+0000 7f525db65700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:41.946745+0000 front 2024-02-02T15:47:41.946768+0000 (oldest deadline 2024-02-02T15:48:07.846739+0000) 2024-02-02T15:48:32.670 INFO:journalctl@ceph.osd.8.smithi142.stdout:Feb 02 15:48:32 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-8[67933]: 2024-02-02T15:48:32.362+0000 7f525db65700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:50.147681+0000 front 2024-02-02T15:47:50.147810+0000 (oldest deadline 2024-02-02T15:48:16.047900+0000) 2024-02-02T15:48:32.670 INFO:journalctl@ceph.osd.8.smithi142.stdout:Feb 02 15:48:32 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-8[67933]: 2024-02-02T15:48:32.362+0000 7f525db65700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:59.350523+0000 front 2024-02-02T15:47:59.350409+0000 (oldest deadline 2024-02-02T15:48:22.850391+0000) 2024-02-02T15:48:32.670 INFO:journalctl@ceph.osd.8.smithi142.stdout:Feb 02 15:48:32 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-8[67933]: 2024-02-02T15:48:32.362+0000 7f525db65700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-02-02T15:48:03.951551+0000 front 2024-02-02T15:48:03.951559+0000 (oldest deadline 2024-02-02T15:48:32.151913+0000) 2024-02-02T15:48:33.169 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:32 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:32.753+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:35.342656+0000 front 2024-02-02T15:47:35.342649+0000 (oldest deadline 2024-02-02T15:48:01.143219+0000) 2024-02-02T15:48:33.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:32 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:32.753+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:47.544669+0000 front 2024-02-02T15:47:47.544689+0000 (oldest deadline 2024-02-02T15:48:13.444898+0000) 2024-02-02T15:48:33.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:32 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:32.753+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:59.846650+0000 front 2024-02-02T15:47:59.846736+0000 (oldest deadline 2024-02-02T15:48:22.746483+0000) 2024-02-02T15:48:33.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:32 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:32.753+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-02-02T15:48:03.247880+0000 front 2024-02-02T15:48:03.247853+0000 (oldest deadline 2024-02-02T15:48:28.547857+0000) 2024-02-02T15:48:33.364 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:32 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:32.884+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.558440+0000 front 2024-02-02T15:47:33.558460+0000 (oldest deadline 2024-02-02T15:47:59.958896+0000) 2024-02-02T15:48:33.364 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:32 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:32.884+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:43.861259+0000 front 2024-02-02T15:47:43.861243+0000 (oldest deadline 2024-02-02T15:48:09.161042+0000) 2024-02-02T15:48:33.364 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:32 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:32.884+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:49.161607+0000 front 2024-02-02T15:47:49.161481+0000 (oldest deadline 2024-02-02T15:48:13.261576+0000) 2024-02-02T15:48:33.365 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:32 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:32.884+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:56.063095+0000 front 2024-02-02T15:47:56.063158+0000 (oldest deadline 2024-02-02T15:48:21.363242+0000) 2024-02-02T15:48:33.365 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:32 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:32.884+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-02-02T15:48:07.264171+0000 front 2024-02-02T15:48:07.264186+0000 (oldest deadline 2024-02-02T15:48:30.164422+0000) 2024-02-02T15:48:33.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:32 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:32.988+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.102474+0000 front 2024-02-02T15:47:33.102427+0000 (oldest deadline 2024-02-02T15:47:57.802602+0000) 2024-02-02T15:48:33.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:32 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:32.988+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:44.106107+0000 front 2024-02-02T15:47:44.106141+0000 (oldest deadline 2024-02-02T15:48:07.006384+0000) 2024-02-02T15:48:33.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:32 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:32.988+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:51.707992+0000 front 2024-02-02T15:47:51.707872+0000 (oldest deadline 2024-02-02T15:48:15.808055+0000) 2024-02-02T15:48:33.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:32 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:32.988+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:59.309660+0000 front 2024-02-02T15:47:59.309683+0000 (oldest deadline 2024-02-02T15:48:22.809935+0000) 2024-02-02T15:48:33.441 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:32 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:32.988+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-02-02T15:48:03.911403+0000 front 2024-02-02T15:48:03.911524+0000 (oldest deadline 2024-02-02T15:48:29.211532+0000) 2024-02-02T15:48:33.670 INFO:journalctl@ceph.osd.8.smithi142.stdout:Feb 02 15:48:33 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-8[67933]: 2024-02-02T15:48:33.356+0000 7f525db65700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:41.946745+0000 front 2024-02-02T15:47:41.946768+0000 (oldest deadline 2024-02-02T15:48:07.846739+0000) 2024-02-02T15:48:33.670 INFO:journalctl@ceph.osd.8.smithi142.stdout:Feb 02 15:48:33 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-8[67933]: 2024-02-02T15:48:33.356+0000 7f525db65700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:50.147681+0000 front 2024-02-02T15:47:50.147810+0000 (oldest deadline 2024-02-02T15:48:16.047900+0000) 2024-02-02T15:48:33.670 INFO:journalctl@ceph.osd.8.smithi142.stdout:Feb 02 15:48:33 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-8[67933]: 2024-02-02T15:48:33.356+0000 7f525db65700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:59.350523+0000 front 2024-02-02T15:47:59.350409+0000 (oldest deadline 2024-02-02T15:48:22.850391+0000) 2024-02-02T15:48:33.670 INFO:journalctl@ceph.osd.8.smithi142.stdout:Feb 02 15:48:33 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-8[67933]: 2024-02-02T15:48:33.356+0000 7f525db65700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-02-02T15:48:03.951551+0000 front 2024-02-02T15:48:03.951559+0000 (oldest deadline 2024-02-02T15:48:32.151913+0000) 2024-02-02T15:48:34.114 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:33 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:33.847+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.558440+0000 front 2024-02-02T15:47:33.558460+0000 (oldest deadline 2024-02-02T15:47:59.958896+0000) 2024-02-02T15:48:34.114 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:33 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:33.847+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:43.861259+0000 front 2024-02-02T15:47:43.861243+0000 (oldest deadline 2024-02-02T15:48:09.161042+0000) 2024-02-02T15:48:34.114 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:33 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:33.847+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:49.161607+0000 front 2024-02-02T15:47:49.161481+0000 (oldest deadline 2024-02-02T15:48:13.261576+0000) 2024-02-02T15:48:34.114 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:33 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:33.847+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:56.063095+0000 front 2024-02-02T15:47:56.063158+0000 (oldest deadline 2024-02-02T15:48:21.363242+0000) 2024-02-02T15:48:34.115 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:33 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:33.847+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-02-02T15:48:07.264171+0000 front 2024-02-02T15:48:07.264186+0000 (oldest deadline 2024-02-02T15:48:30.164422+0000) 2024-02-02T15:48:34.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:33 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:33.723+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:35.342656+0000 front 2024-02-02T15:47:35.342649+0000 (oldest deadline 2024-02-02T15:48:01.143219+0000) 2024-02-02T15:48:34.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:33 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:33.723+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:47.544669+0000 front 2024-02-02T15:47:47.544689+0000 (oldest deadline 2024-02-02T15:48:13.444898+0000) 2024-02-02T15:48:34.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:33 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:33.723+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:59.846650+0000 front 2024-02-02T15:47:59.846736+0000 (oldest deadline 2024-02-02T15:48:22.746483+0000) 2024-02-02T15:48:34.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:33 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:33.723+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-02-02T15:48:03.247880+0000 front 2024-02-02T15:48:03.247853+0000 (oldest deadline 2024-02-02T15:48:28.547857+0000) 2024-02-02T15:48:34.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:34 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:34.030+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.102474+0000 front 2024-02-02T15:47:33.102427+0000 (oldest deadline 2024-02-02T15:47:57.802602+0000) 2024-02-02T15:48:34.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:34 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:34.030+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:44.106107+0000 front 2024-02-02T15:47:44.106141+0000 (oldest deadline 2024-02-02T15:48:07.006384+0000) 2024-02-02T15:48:34.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:34 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:34.030+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:51.707992+0000 front 2024-02-02T15:47:51.707872+0000 (oldest deadline 2024-02-02T15:48:15.808055+0000) 2024-02-02T15:48:34.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:34 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:34.030+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:59.309660+0000 front 2024-02-02T15:47:59.309683+0000 (oldest deadline 2024-02-02T15:48:22.809935+0000) 2024-02-02T15:48:34.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:34 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:34.030+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-02-02T15:48:03.911403+0000 front 2024-02-02T15:48:03.911524+0000 (oldest deadline 2024-02-02T15:48:29.211532+0000) 2024-02-02T15:48:34.670 INFO:journalctl@ceph.osd.8.smithi142.stdout:Feb 02 15:48:34 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-8[67933]: 2024-02-02T15:48:34.341+0000 7f525db65700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:41.946745+0000 front 2024-02-02T15:47:41.946768+0000 (oldest deadline 2024-02-02T15:48:07.846739+0000) 2024-02-02T15:48:34.670 INFO:journalctl@ceph.osd.8.smithi142.stdout:Feb 02 15:48:34 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-8[67933]: 2024-02-02T15:48:34.341+0000 7f525db65700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:50.147681+0000 front 2024-02-02T15:47:50.147810+0000 (oldest deadline 2024-02-02T15:48:16.047900+0000) 2024-02-02T15:48:34.670 INFO:journalctl@ceph.osd.8.smithi142.stdout:Feb 02 15:48:34 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-8[67933]: 2024-02-02T15:48:34.341+0000 7f525db65700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:59.350523+0000 front 2024-02-02T15:47:59.350409+0000 (oldest deadline 2024-02-02T15:48:22.850391+0000) 2024-02-02T15:48:34.670 INFO:journalctl@ceph.osd.8.smithi142.stdout:Feb 02 15:48:34 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-8[67933]: 2024-02-02T15:48:34.341+0000 7f525db65700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-02-02T15:48:03.951551+0000 front 2024-02-02T15:48:03.951559+0000 (oldest deadline 2024-02-02T15:48:32.151913+0000) 2024-02-02T15:48:35.114 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:34 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:34.814+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.558440+0000 front 2024-02-02T15:47:33.558460+0000 (oldest deadline 2024-02-02T15:47:59.958896+0000) 2024-02-02T15:48:35.114 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:34 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:34.814+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:43.861259+0000 front 2024-02-02T15:47:43.861243+0000 (oldest deadline 2024-02-02T15:48:09.161042+0000) 2024-02-02T15:48:35.114 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:34 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:34.814+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:49.161607+0000 front 2024-02-02T15:47:49.161481+0000 (oldest deadline 2024-02-02T15:48:13.261576+0000) 2024-02-02T15:48:35.114 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:34 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:34.814+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:56.063095+0000 front 2024-02-02T15:47:56.063158+0000 (oldest deadline 2024-02-02T15:48:21.363242+0000) 2024-02-02T15:48:35.114 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:34 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:34.814+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-02-02T15:48:07.264171+0000 front 2024-02-02T15:48:07.264186+0000 (oldest deadline 2024-02-02T15:48:30.164422+0000) 2024-02-02T15:48:35.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:34 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:34.737+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:35.342656+0000 front 2024-02-02T15:47:35.342649+0000 (oldest deadline 2024-02-02T15:48:01.143219+0000) 2024-02-02T15:48:35.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:34 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:34.737+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:47.544669+0000 front 2024-02-02T15:47:47.544689+0000 (oldest deadline 2024-02-02T15:48:13.444898+0000) 2024-02-02T15:48:35.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:34 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:34.737+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:59.846650+0000 front 2024-02-02T15:47:59.846736+0000 (oldest deadline 2024-02-02T15:48:22.746483+0000) 2024-02-02T15:48:35.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:34 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:34.737+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-02-02T15:48:03.247880+0000 front 2024-02-02T15:48:03.247853+0000 (oldest deadline 2024-02-02T15:48:28.547857+0000) 2024-02-02T15:48:35.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:34 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:34.986+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.102474+0000 front 2024-02-02T15:47:33.102427+0000 (oldest deadline 2024-02-02T15:47:57.802602+0000) 2024-02-02T15:48:35.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:34 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:34.986+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:44.106107+0000 front 2024-02-02T15:47:44.106141+0000 (oldest deadline 2024-02-02T15:48:07.006384+0000) 2024-02-02T15:48:35.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:34 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:34.986+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:51.707992+0000 front 2024-02-02T15:47:51.707872+0000 (oldest deadline 2024-02-02T15:48:15.808055+0000) 2024-02-02T15:48:35.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:34 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:34.986+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:59.309660+0000 front 2024-02-02T15:47:59.309683+0000 (oldest deadline 2024-02-02T15:48:22.809935+0000) 2024-02-02T15:48:35.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:34 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:34.986+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-02-02T15:48:03.911403+0000 front 2024-02-02T15:48:03.911524+0000 (oldest deadline 2024-02-02T15:48:29.211532+0000) 2024-02-02T15:48:35.669 INFO:journalctl@ceph.osd.8.smithi142.stdout:Feb 02 15:48:35 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-8[67933]: 2024-02-02T15:48:35.378+0000 7f525db65700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:41.946745+0000 front 2024-02-02T15:47:41.946768+0000 (oldest deadline 2024-02-02T15:48:07.846739+0000) 2024-02-02T15:48:35.670 INFO:journalctl@ceph.osd.8.smithi142.stdout:Feb 02 15:48:35 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-8[67933]: 2024-02-02T15:48:35.378+0000 7f525db65700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:50.147681+0000 front 2024-02-02T15:47:50.147810+0000 (oldest deadline 2024-02-02T15:48:16.047900+0000) 2024-02-02T15:48:35.670 INFO:journalctl@ceph.osd.8.smithi142.stdout:Feb 02 15:48:35 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-8[67933]: 2024-02-02T15:48:35.378+0000 7f525db65700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:59.350523+0000 front 2024-02-02T15:47:59.350409+0000 (oldest deadline 2024-02-02T15:48:22.850391+0000) 2024-02-02T15:48:35.670 INFO:journalctl@ceph.osd.8.smithi142.stdout:Feb 02 15:48:35 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-8[67933]: 2024-02-02T15:48:35.378+0000 7f525db65700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-02-02T15:48:03.951551+0000 front 2024-02-02T15:48:03.951559+0000 (oldest deadline 2024-02-02T15:48:32.151913+0000) 2024-02-02T15:48:36.114 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:35 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:35.771+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.558440+0000 front 2024-02-02T15:47:33.558460+0000 (oldest deadline 2024-02-02T15:47:59.958896+0000) 2024-02-02T15:48:36.114 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:35 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:35.771+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:43.861259+0000 front 2024-02-02T15:47:43.861243+0000 (oldest deadline 2024-02-02T15:48:09.161042+0000) 2024-02-02T15:48:36.114 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:35 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:35.771+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:49.161607+0000 front 2024-02-02T15:47:49.161481+0000 (oldest deadline 2024-02-02T15:48:13.261576+0000) 2024-02-02T15:48:36.115 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:35 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:35.771+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:56.063095+0000 front 2024-02-02T15:47:56.063158+0000 (oldest deadline 2024-02-02T15:48:21.363242+0000) 2024-02-02T15:48:36.115 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:35 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:35.771+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-02-02T15:48:07.264171+0000 front 2024-02-02T15:48:07.264186+0000 (oldest deadline 2024-02-02T15:48:30.164422+0000) 2024-02-02T15:48:36.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:35 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:35.687+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:35.342656+0000 front 2024-02-02T15:47:35.342649+0000 (oldest deadline 2024-02-02T15:48:01.143219+0000) 2024-02-02T15:48:36.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:35 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:35.687+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:47.544669+0000 front 2024-02-02T15:47:47.544689+0000 (oldest deadline 2024-02-02T15:48:13.444898+0000) 2024-02-02T15:48:36.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:35 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:35.687+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:59.846650+0000 front 2024-02-02T15:47:59.846736+0000 (oldest deadline 2024-02-02T15:48:22.746483+0000) 2024-02-02T15:48:36.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:35 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:35.687+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-02-02T15:48:03.247880+0000 front 2024-02-02T15:48:03.247853+0000 (oldest deadline 2024-02-02T15:48:28.547857+0000) 2024-02-02T15:48:36.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:35 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:35.954+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.102474+0000 front 2024-02-02T15:47:33.102427+0000 (oldest deadline 2024-02-02T15:47:57.802602+0000) 2024-02-02T15:48:36.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:35 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:35.954+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:44.106107+0000 front 2024-02-02T15:47:44.106141+0000 (oldest deadline 2024-02-02T15:48:07.006384+0000) 2024-02-02T15:48:36.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:35 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:35.954+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:51.707992+0000 front 2024-02-02T15:47:51.707872+0000 (oldest deadline 2024-02-02T15:48:15.808055+0000) 2024-02-02T15:48:36.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:35 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:35.954+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:59.309660+0000 front 2024-02-02T15:47:59.309683+0000 (oldest deadline 2024-02-02T15:48:22.809935+0000) 2024-02-02T15:48:36.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:35 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:35.954+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-02-02T15:48:03.911403+0000 front 2024-02-02T15:48:03.911524+0000 (oldest deadline 2024-02-02T15:48:29.211532+0000) 2024-02-02T15:48:36.670 INFO:journalctl@ceph.osd.8.smithi142.stdout:Feb 02 15:48:36 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-8[67933]: 2024-02-02T15:48:36.361+0000 7f525db65700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:41.946745+0000 front 2024-02-02T15:47:41.946768+0000 (oldest deadline 2024-02-02T15:48:07.846739+0000) 2024-02-02T15:48:36.670 INFO:journalctl@ceph.osd.8.smithi142.stdout:Feb 02 15:48:36 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-8[67933]: 2024-02-02T15:48:36.361+0000 7f525db65700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:50.147681+0000 front 2024-02-02T15:47:50.147810+0000 (oldest deadline 2024-02-02T15:48:16.047900+0000) 2024-02-02T15:48:36.670 INFO:journalctl@ceph.osd.8.smithi142.stdout:Feb 02 15:48:36 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-8[67933]: 2024-02-02T15:48:36.361+0000 7f525db65700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:59.350523+0000 front 2024-02-02T15:47:59.350409+0000 (oldest deadline 2024-02-02T15:48:22.850391+0000) 2024-02-02T15:48:36.670 INFO:journalctl@ceph.osd.8.smithi142.stdout:Feb 02 15:48:36 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-8[67933]: 2024-02-02T15:48:36.361+0000 7f525db65700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-02-02T15:48:03.951551+0000 front 2024-02-02T15:48:03.951559+0000 (oldest deadline 2024-02-02T15:48:32.151913+0000) 2024-02-02T15:48:36.670 INFO:journalctl@ceph.osd.8.smithi142.stdout:Feb 02 15:48:36 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-8[67933]: 2024-02-02T15:48:36.361+0000 7f525db65700 -1 osd.8 121 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-02-02T15:48:12.152318+0000 front 2024-02-02T15:48:12.152338+0000 (oldest deadline 2024-02-02T15:48:36.252447+0000) 2024-02-02T15:48:37.114 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:36 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:36.810+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.558440+0000 front 2024-02-02T15:47:33.558460+0000 (oldest deadline 2024-02-02T15:47:59.958896+0000) 2024-02-02T15:48:37.114 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:36 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:36.810+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:43.861259+0000 front 2024-02-02T15:47:43.861243+0000 (oldest deadline 2024-02-02T15:48:09.161042+0000) 2024-02-02T15:48:37.114 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:36 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:36.810+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:49.161607+0000 front 2024-02-02T15:47:49.161481+0000 (oldest deadline 2024-02-02T15:48:13.261576+0000) 2024-02-02T15:48:37.114 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:36 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:36.810+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:56.063095+0000 front 2024-02-02T15:47:56.063158+0000 (oldest deadline 2024-02-02T15:48:21.363242+0000) 2024-02-02T15:48:37.115 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:36 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:36.810+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-02-02T15:48:07.264171+0000 front 2024-02-02T15:48:07.264186+0000 (oldest deadline 2024-02-02T15:48:30.164422+0000) 2024-02-02T15:48:37.115 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:36 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:36.810+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-02-02T15:48:10.165002+0000 front 2024-02-02T15:48:10.164971+0000 (oldest deadline 2024-02-02T15:48:36.064922+0000) 2024-02-02T15:48:37.169 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:36 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:36.699+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:35.342656+0000 front 2024-02-02T15:47:35.342649+0000 (oldest deadline 2024-02-02T15:48:01.143219+0000) 2024-02-02T15:48:37.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:36 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:36.699+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:47.544669+0000 front 2024-02-02T15:47:47.544689+0000 (oldest deadline 2024-02-02T15:48:13.444898+0000) 2024-02-02T15:48:37.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:36 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:36.699+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:59.846650+0000 front 2024-02-02T15:47:59.846736+0000 (oldest deadline 2024-02-02T15:48:22.746483+0000) 2024-02-02T15:48:37.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:36 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:36.699+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-02-02T15:48:03.247880+0000 front 2024-02-02T15:48:03.247853+0000 (oldest deadline 2024-02-02T15:48:28.547857+0000) 2024-02-02T15:48:37.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:36 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:36.972+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.102474+0000 front 2024-02-02T15:47:33.102427+0000 (oldest deadline 2024-02-02T15:47:57.802602+0000) 2024-02-02T15:48:37.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:36 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:36.972+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:44.106107+0000 front 2024-02-02T15:47:44.106141+0000 (oldest deadline 2024-02-02T15:48:07.006384+0000) 2024-02-02T15:48:37.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:36 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:36.972+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:51.707992+0000 front 2024-02-02T15:47:51.707872+0000 (oldest deadline 2024-02-02T15:48:15.808055+0000) 2024-02-02T15:48:37.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:36 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:36.972+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:59.309660+0000 front 2024-02-02T15:47:59.309683+0000 (oldest deadline 2024-02-02T15:48:22.809935+0000) 2024-02-02T15:48:37.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:36 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:36.972+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-02-02T15:48:03.911403+0000 front 2024-02-02T15:48:03.911524+0000 (oldest deadline 2024-02-02T15:48:29.211532+0000) 2024-02-02T15:48:37.920 INFO:journalctl@ceph.osd.8.smithi142.stdout:Feb 02 15:48:37 smithi142 bash[92175]: ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-8 2024-02-02T15:48:37.920 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:37 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:37.665+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:35.342656+0000 front 2024-02-02T15:47:35.342649+0000 (oldest deadline 2024-02-02T15:48:01.143219+0000) 2024-02-02T15:48:37.920 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:37 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:37.665+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:47.544669+0000 front 2024-02-02T15:47:47.544689+0000 (oldest deadline 2024-02-02T15:48:13.444898+0000) 2024-02-02T15:48:37.921 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:37 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:37.665+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:59.846650+0000 front 2024-02-02T15:47:59.846736+0000 (oldest deadline 2024-02-02T15:48:22.746483+0000) 2024-02-02T15:48:37.921 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:37 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:37.665+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-02-02T15:48:03.247880+0000 front 2024-02-02T15:48:03.247853+0000 (oldest deadline 2024-02-02T15:48:28.547857+0000) 2024-02-02T15:48:38.114 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:37 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:37.797+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.558440+0000 front 2024-02-02T15:47:33.558460+0000 (oldest deadline 2024-02-02T15:47:59.958896+0000) 2024-02-02T15:48:38.114 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:37 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:37.797+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:43.861259+0000 front 2024-02-02T15:47:43.861243+0000 (oldest deadline 2024-02-02T15:48:09.161042+0000) 2024-02-02T15:48:38.114 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:37 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:37.797+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:49.161607+0000 front 2024-02-02T15:47:49.161481+0000 (oldest deadline 2024-02-02T15:48:13.261576+0000) 2024-02-02T15:48:38.114 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:37 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:37.797+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:56.063095+0000 front 2024-02-02T15:47:56.063158+0000 (oldest deadline 2024-02-02T15:48:21.363242+0000) 2024-02-02T15:48:38.115 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:37 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:37.797+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-02-02T15:48:07.264171+0000 front 2024-02-02T15:48:07.264186+0000 (oldest deadline 2024-02-02T15:48:30.164422+0000) 2024-02-02T15:48:38.115 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:37 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:37.797+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-02-02T15:48:10.165002+0000 front 2024-02-02T15:48:10.164971+0000 (oldest deadline 2024-02-02T15:48:36.064922+0000) 2024-02-02T15:48:38.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:37 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:37.981+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.102474+0000 front 2024-02-02T15:47:33.102427+0000 (oldest deadline 2024-02-02T15:47:57.802602+0000) 2024-02-02T15:48:38.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:37 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:37.981+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:44.106107+0000 front 2024-02-02T15:47:44.106141+0000 (oldest deadline 2024-02-02T15:48:07.006384+0000) 2024-02-02T15:48:38.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:37 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:37.981+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:51.707992+0000 front 2024-02-02T15:47:51.707872+0000 (oldest deadline 2024-02-02T15:48:15.808055+0000) 2024-02-02T15:48:38.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:37 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:37.981+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:59.309660+0000 front 2024-02-02T15:47:59.309683+0000 (oldest deadline 2024-02-02T15:48:22.809935+0000) 2024-02-02T15:48:38.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:37 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:37.981+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-02-02T15:48:03.911403+0000 front 2024-02-02T15:48:03.911524+0000 (oldest deadline 2024-02-02T15:48:29.211532+0000) 2024-02-02T15:48:38.920 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:38 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:38.632+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:35.342656+0000 front 2024-02-02T15:47:35.342649+0000 (oldest deadline 2024-02-02T15:48:01.143219+0000) 2024-02-02T15:48:38.920 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:38 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:38.632+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:47.544669+0000 front 2024-02-02T15:47:47.544689+0000 (oldest deadline 2024-02-02T15:48:13.444898+0000) 2024-02-02T15:48:38.921 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:38 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:38.632+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:59.846650+0000 front 2024-02-02T15:47:59.846736+0000 (oldest deadline 2024-02-02T15:48:22.746483+0000) 2024-02-02T15:48:38.921 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:38 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:38.632+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-02-02T15:48:03.247880+0000 front 2024-02-02T15:48:03.247853+0000 (oldest deadline 2024-02-02T15:48:28.547857+0000) 2024-02-02T15:48:38.921 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:38 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:38.632+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-02-02T15:48:12.048751+0000 front 2024-02-02T15:48:12.048729+0000 (oldest deadline 2024-02-02T15:48:37.849453+0000) 2024-02-02T15:48:39.021 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-371a409c-c1de-11ee-95b6-87774f69a715@osd.8.service' 2024-02-02T15:48:39.102 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-02T15:48:39.102 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-02-02T15:48:39.102 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-02-02T15:48:39.103 DEBUG:teuthology.orchestra.run.smithi001:> sudo systemctl stop ceph-371a409c-c1de-11ee-95b6-87774f69a715@osd.9 2024-02-02T15:48:39.114 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:38 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:38.756+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.558440+0000 front 2024-02-02T15:47:33.558460+0000 (oldest deadline 2024-02-02T15:47:59.958896+0000) 2024-02-02T15:48:39.114 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:38 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:38.756+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:43.861259+0000 front 2024-02-02T15:47:43.861243+0000 (oldest deadline 2024-02-02T15:48:09.161042+0000) 2024-02-02T15:48:39.114 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:38 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:38.756+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:49.161607+0000 front 2024-02-02T15:47:49.161481+0000 (oldest deadline 2024-02-02T15:48:13.261576+0000) 2024-02-02T15:48:39.114 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:38 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:38.756+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:56.063095+0000 front 2024-02-02T15:47:56.063158+0000 (oldest deadline 2024-02-02T15:48:21.363242+0000) 2024-02-02T15:48:39.114 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:38 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:38.756+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-02-02T15:48:07.264171+0000 front 2024-02-02T15:48:07.264186+0000 (oldest deadline 2024-02-02T15:48:30.164422+0000) 2024-02-02T15:48:39.115 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:38 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:38.756+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-02-02T15:48:10.165002+0000 front 2024-02-02T15:48:10.164971+0000 (oldest deadline 2024-02-02T15:48:36.064922+0000) 2024-02-02T15:48:39.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:38 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:38.992+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.102474+0000 front 2024-02-02T15:47:33.102427+0000 (oldest deadline 2024-02-02T15:47:57.802602+0000) 2024-02-02T15:48:39.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:38 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:38.992+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:44.106107+0000 front 2024-02-02T15:47:44.106141+0000 (oldest deadline 2024-02-02T15:48:07.006384+0000) 2024-02-02T15:48:39.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:38 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:38.992+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:51.707992+0000 front 2024-02-02T15:47:51.707872+0000 (oldest deadline 2024-02-02T15:48:15.808055+0000) 2024-02-02T15:48:39.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:38 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:38.992+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:59.309660+0000 front 2024-02-02T15:47:59.309683+0000 (oldest deadline 2024-02-02T15:48:22.809935+0000) 2024-02-02T15:48:39.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:38 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:38.992+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-02-02T15:48:03.911403+0000 front 2024-02-02T15:48:03.911524+0000 (oldest deadline 2024-02-02T15:48:29.211532+0000) 2024-02-02T15:48:39.441 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:38 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:38.992+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-02-02T15:48:13.912773+0000 front 2024-02-02T15:48:13.912878+0000 (oldest deadline 2024-02-02T15:48:38.612968+0000) 2024-02-02T15:48:39.441 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:39 smithi001 systemd[1]: Stopping Ceph osd.9 for 371a409c-c1de-11ee-95b6-87774f69a715... 2024-02-02T15:48:39.940 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:39 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:39.493+0000 7fb98fe00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-02T15:48:39.940 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:39 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:39.493+0000 7fb98fe00700 -1 osd.9 121 *** Got signal Terminated *** 2024-02-02T15:48:39.940 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:39 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:39.493+0000 7fb98fe00700 -1 osd.9 121 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-02T15:48:40.114 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:39 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:39.793+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.558440+0000 front 2024-02-02T15:47:33.558460+0000 (oldest deadline 2024-02-02T15:47:59.958896+0000) 2024-02-02T15:48:40.114 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:39 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:39.793+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:43.861259+0000 front 2024-02-02T15:47:43.861243+0000 (oldest deadline 2024-02-02T15:48:09.161042+0000) 2024-02-02T15:48:40.114 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:39 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:39.793+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:49.161607+0000 front 2024-02-02T15:47:49.161481+0000 (oldest deadline 2024-02-02T15:48:13.261576+0000) 2024-02-02T15:48:40.114 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:39 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:39.793+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:56.063095+0000 front 2024-02-02T15:47:56.063158+0000 (oldest deadline 2024-02-02T15:48:21.363242+0000) 2024-02-02T15:48:40.114 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:39 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:39.793+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-02-02T15:48:07.264171+0000 front 2024-02-02T15:48:07.264186+0000 (oldest deadline 2024-02-02T15:48:30.164422+0000) 2024-02-02T15:48:40.115 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:39 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:39.793+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-02-02T15:48:10.165002+0000 front 2024-02-02T15:48:10.164971+0000 (oldest deadline 2024-02-02T15:48:36.064922+0000) 2024-02-02T15:48:40.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:39 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:39.675+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:35.342656+0000 front 2024-02-02T15:47:35.342649+0000 (oldest deadline 2024-02-02T15:48:01.143219+0000) 2024-02-02T15:48:40.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:39 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:39.675+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:47.544669+0000 front 2024-02-02T15:47:47.544689+0000 (oldest deadline 2024-02-02T15:48:13.444898+0000) 2024-02-02T15:48:40.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:39 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:39.675+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:59.846650+0000 front 2024-02-02T15:47:59.846736+0000 (oldest deadline 2024-02-02T15:48:22.746483+0000) 2024-02-02T15:48:40.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:39 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:39.675+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-02-02T15:48:03.247880+0000 front 2024-02-02T15:48:03.247853+0000 (oldest deadline 2024-02-02T15:48:28.547857+0000) 2024-02-02T15:48:40.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:39 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:39.675+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-02-02T15:48:12.048751+0000 front 2024-02-02T15:48:12.048729+0000 (oldest deadline 2024-02-02T15:48:37.849453+0000) 2024-02-02T15:48:40.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:39 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:39.979+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.102474+0000 front 2024-02-02T15:47:33.102427+0000 (oldest deadline 2024-02-02T15:47:57.802602+0000) 2024-02-02T15:48:40.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:39 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:39.979+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:44.106107+0000 front 2024-02-02T15:47:44.106141+0000 (oldest deadline 2024-02-02T15:48:07.006384+0000) 2024-02-02T15:48:40.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:39 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:39.979+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:51.707992+0000 front 2024-02-02T15:47:51.707872+0000 (oldest deadline 2024-02-02T15:48:15.808055+0000) 2024-02-02T15:48:40.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:39 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:39.979+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:59.309660+0000 front 2024-02-02T15:47:59.309683+0000 (oldest deadline 2024-02-02T15:48:22.809935+0000) 2024-02-02T15:48:40.441 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:39 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:39.979+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-02-02T15:48:03.911403+0000 front 2024-02-02T15:48:03.911524+0000 (oldest deadline 2024-02-02T15:48:29.211532+0000) 2024-02-02T15:48:40.441 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:39 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:39.979+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-02-02T15:48:13.912773+0000 front 2024-02-02T15:48:13.912878+0000 (oldest deadline 2024-02-02T15:48:38.612968+0000) 2024-02-02T15:48:41.114 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:40 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:40.790+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.558440+0000 front 2024-02-02T15:47:33.558460+0000 (oldest deadline 2024-02-02T15:47:59.958896+0000) 2024-02-02T15:48:41.114 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:40 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:40.790+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:43.861259+0000 front 2024-02-02T15:47:43.861243+0000 (oldest deadline 2024-02-02T15:48:09.161042+0000) 2024-02-02T15:48:41.114 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:40 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:40.790+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:49.161607+0000 front 2024-02-02T15:47:49.161481+0000 (oldest deadline 2024-02-02T15:48:13.261576+0000) 2024-02-02T15:48:41.114 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:40 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:40.790+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:56.063095+0000 front 2024-02-02T15:47:56.063158+0000 (oldest deadline 2024-02-02T15:48:21.363242+0000) 2024-02-02T15:48:41.115 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:40 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:40.790+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-02-02T15:48:07.264171+0000 front 2024-02-02T15:48:07.264186+0000 (oldest deadline 2024-02-02T15:48:30.164422+0000) 2024-02-02T15:48:41.115 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:40 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:40.790+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-02-02T15:48:10.165002+0000 front 2024-02-02T15:48:10.164971+0000 (oldest deadline 2024-02-02T15:48:36.064922+0000) 2024-02-02T15:48:41.169 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:40 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:40.688+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:35.342656+0000 front 2024-02-02T15:47:35.342649+0000 (oldest deadline 2024-02-02T15:48:01.143219+0000) 2024-02-02T15:48:41.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:40 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:40.688+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:47.544669+0000 front 2024-02-02T15:47:47.544689+0000 (oldest deadline 2024-02-02T15:48:13.444898+0000) 2024-02-02T15:48:41.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:40 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:40.688+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:59.846650+0000 front 2024-02-02T15:47:59.846736+0000 (oldest deadline 2024-02-02T15:48:22.746483+0000) 2024-02-02T15:48:41.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:40 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:40.688+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-02-02T15:48:03.247880+0000 front 2024-02-02T15:48:03.247853+0000 (oldest deadline 2024-02-02T15:48:28.547857+0000) 2024-02-02T15:48:41.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:40 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:40.688+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-02-02T15:48:12.048751+0000 front 2024-02-02T15:48:12.048729+0000 (oldest deadline 2024-02-02T15:48:37.849453+0000) 2024-02-02T15:48:41.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:40 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:40.956+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.102474+0000 front 2024-02-02T15:47:33.102427+0000 (oldest deadline 2024-02-02T15:47:57.802602+0000) 2024-02-02T15:48:41.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:40 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:40.956+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:44.106107+0000 front 2024-02-02T15:47:44.106141+0000 (oldest deadline 2024-02-02T15:48:07.006384+0000) 2024-02-02T15:48:41.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:40 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:40.956+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:51.707992+0000 front 2024-02-02T15:47:51.707872+0000 (oldest deadline 2024-02-02T15:48:15.808055+0000) 2024-02-02T15:48:41.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:40 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:40.956+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:59.309660+0000 front 2024-02-02T15:47:59.309683+0000 (oldest deadline 2024-02-02T15:48:22.809935+0000) 2024-02-02T15:48:41.441 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:40 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:40.956+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-02-02T15:48:03.911403+0000 front 2024-02-02T15:48:03.911524+0000 (oldest deadline 2024-02-02T15:48:29.211532+0000) 2024-02-02T15:48:41.441 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:40 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:40.956+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-02-02T15:48:13.912773+0000 front 2024-02-02T15:48:13.912878+0000 (oldest deadline 2024-02-02T15:48:38.612968+0000) 2024-02-02T15:48:42.114 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:41 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:41.830+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.558440+0000 front 2024-02-02T15:47:33.558460+0000 (oldest deadline 2024-02-02T15:47:59.958896+0000) 2024-02-02T15:48:42.114 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:41 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:41.830+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:43.861259+0000 front 2024-02-02T15:47:43.861243+0000 (oldest deadline 2024-02-02T15:48:09.161042+0000) 2024-02-02T15:48:42.114 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:41 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:41.830+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:49.161607+0000 front 2024-02-02T15:47:49.161481+0000 (oldest deadline 2024-02-02T15:48:13.261576+0000) 2024-02-02T15:48:42.114 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:41 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:41.830+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:56.063095+0000 front 2024-02-02T15:47:56.063158+0000 (oldest deadline 2024-02-02T15:48:21.363242+0000) 2024-02-02T15:48:42.115 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:41 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:41.830+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-02-02T15:48:07.264171+0000 front 2024-02-02T15:48:07.264186+0000 (oldest deadline 2024-02-02T15:48:30.164422+0000) 2024-02-02T15:48:42.115 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:41 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:41.830+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-02-02T15:48:10.165002+0000 front 2024-02-02T15:48:10.164971+0000 (oldest deadline 2024-02-02T15:48:36.064922+0000) 2024-02-02T15:48:42.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:41 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:41.678+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:35.342656+0000 front 2024-02-02T15:47:35.342649+0000 (oldest deadline 2024-02-02T15:48:01.143219+0000) 2024-02-02T15:48:42.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:41 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:41.678+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:47.544669+0000 front 2024-02-02T15:47:47.544689+0000 (oldest deadline 2024-02-02T15:48:13.444898+0000) 2024-02-02T15:48:42.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:41 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:41.678+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:59.846650+0000 front 2024-02-02T15:47:59.846736+0000 (oldest deadline 2024-02-02T15:48:22.746483+0000) 2024-02-02T15:48:42.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:41 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:41.678+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-02-02T15:48:03.247880+0000 front 2024-02-02T15:48:03.247853+0000 (oldest deadline 2024-02-02T15:48:28.547857+0000) 2024-02-02T15:48:42.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:41 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:41.678+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-02-02T15:48:12.048751+0000 front 2024-02-02T15:48:12.048729+0000 (oldest deadline 2024-02-02T15:48:37.849453+0000) 2024-02-02T15:48:42.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:41 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:41.963+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.102474+0000 front 2024-02-02T15:47:33.102427+0000 (oldest deadline 2024-02-02T15:47:57.802602+0000) 2024-02-02T15:48:42.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:41 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:41.963+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:44.106107+0000 front 2024-02-02T15:47:44.106141+0000 (oldest deadline 2024-02-02T15:48:07.006384+0000) 2024-02-02T15:48:42.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:41 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:41.963+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:51.707992+0000 front 2024-02-02T15:47:51.707872+0000 (oldest deadline 2024-02-02T15:48:15.808055+0000) 2024-02-02T15:48:42.441 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:41 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:41.963+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:59.309660+0000 front 2024-02-02T15:47:59.309683+0000 (oldest deadline 2024-02-02T15:48:22.809935+0000) 2024-02-02T15:48:42.441 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:41 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:41.963+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-02-02T15:48:03.911403+0000 front 2024-02-02T15:48:03.911524+0000 (oldest deadline 2024-02-02T15:48:29.211532+0000) 2024-02-02T15:48:42.441 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:41 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:41.963+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-02-02T15:48:13.912773+0000 front 2024-02-02T15:48:13.912878+0000 (oldest deadline 2024-02-02T15:48:38.612968+0000) 2024-02-02T15:48:43.114 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:42 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:42.842+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.558440+0000 front 2024-02-02T15:47:33.558460+0000 (oldest deadline 2024-02-02T15:47:59.958896+0000) 2024-02-02T15:48:43.114 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:42 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:42.842+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:43.861259+0000 front 2024-02-02T15:47:43.861243+0000 (oldest deadline 2024-02-02T15:48:09.161042+0000) 2024-02-02T15:48:43.114 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:42 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:42.842+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:49.161607+0000 front 2024-02-02T15:47:49.161481+0000 (oldest deadline 2024-02-02T15:48:13.261576+0000) 2024-02-02T15:48:43.114 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:42 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:42.842+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:56.063095+0000 front 2024-02-02T15:47:56.063158+0000 (oldest deadline 2024-02-02T15:48:21.363242+0000) 2024-02-02T15:48:43.114 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:42 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:42.842+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-02-02T15:48:07.264171+0000 front 2024-02-02T15:48:07.264186+0000 (oldest deadline 2024-02-02T15:48:30.164422+0000) 2024-02-02T15:48:43.115 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:42 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:42.842+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-02-02T15:48:10.165002+0000 front 2024-02-02T15:48:10.164971+0000 (oldest deadline 2024-02-02T15:48:36.064922+0000) 2024-02-02T15:48:43.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:42 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:42.706+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:35.342656+0000 front 2024-02-02T15:47:35.342649+0000 (oldest deadline 2024-02-02T15:48:01.143219+0000) 2024-02-02T15:48:43.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:42 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:42.706+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:47.544669+0000 front 2024-02-02T15:47:47.544689+0000 (oldest deadline 2024-02-02T15:48:13.444898+0000) 2024-02-02T15:48:43.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:42 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:42.706+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:59.846650+0000 front 2024-02-02T15:47:59.846736+0000 (oldest deadline 2024-02-02T15:48:22.746483+0000) 2024-02-02T15:48:43.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:42 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:42.706+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-02-02T15:48:03.247880+0000 front 2024-02-02T15:48:03.247853+0000 (oldest deadline 2024-02-02T15:48:28.547857+0000) 2024-02-02T15:48:43.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:42 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:42.706+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-02-02T15:48:12.048751+0000 front 2024-02-02T15:48:12.048729+0000 (oldest deadline 2024-02-02T15:48:37.849453+0000) 2024-02-02T15:48:43.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:42 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:42.986+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.102474+0000 front 2024-02-02T15:47:33.102427+0000 (oldest deadline 2024-02-02T15:47:57.802602+0000) 2024-02-02T15:48:43.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:42 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:42.986+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:44.106107+0000 front 2024-02-02T15:47:44.106141+0000 (oldest deadline 2024-02-02T15:48:07.006384+0000) 2024-02-02T15:48:43.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:42 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:42.986+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:51.707992+0000 front 2024-02-02T15:47:51.707872+0000 (oldest deadline 2024-02-02T15:48:15.808055+0000) 2024-02-02T15:48:43.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:42 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:42.986+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:59.309660+0000 front 2024-02-02T15:47:59.309683+0000 (oldest deadline 2024-02-02T15:48:22.809935+0000) 2024-02-02T15:48:43.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:42 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:42.986+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-02-02T15:48:03.911403+0000 front 2024-02-02T15:48:03.911524+0000 (oldest deadline 2024-02-02T15:48:29.211532+0000) 2024-02-02T15:48:43.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:42 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:42.986+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-02-02T15:48:13.912773+0000 front 2024-02-02T15:48:13.912878+0000 (oldest deadline 2024-02-02T15:48:38.612968+0000) 2024-02-02T15:48:43.920 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:43 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:43.658+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:35.342656+0000 front 2024-02-02T15:47:35.342649+0000 (oldest deadline 2024-02-02T15:48:01.143219+0000) 2024-02-02T15:48:43.920 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:43 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:43.658+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:47.544669+0000 front 2024-02-02T15:47:47.544689+0000 (oldest deadline 2024-02-02T15:48:13.444898+0000) 2024-02-02T15:48:43.920 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:43 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:43.658+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:59.846650+0000 front 2024-02-02T15:47:59.846736+0000 (oldest deadline 2024-02-02T15:48:22.746483+0000) 2024-02-02T15:48:43.920 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:43 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:43.658+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-02-02T15:48:03.247880+0000 front 2024-02-02T15:48:03.247853+0000 (oldest deadline 2024-02-02T15:48:28.547857+0000) 2024-02-02T15:48:43.921 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:43 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:43.658+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-02-02T15:48:12.048751+0000 front 2024-02-02T15:48:12.048729+0000 (oldest deadline 2024-02-02T15:48:37.849453+0000) 2024-02-02T15:48:44.364 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:43 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:43.864+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.558440+0000 front 2024-02-02T15:47:33.558460+0000 (oldest deadline 2024-02-02T15:47:59.958896+0000) 2024-02-02T15:48:44.364 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:43 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:43.864+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:43.861259+0000 front 2024-02-02T15:47:43.861243+0000 (oldest deadline 2024-02-02T15:48:09.161042+0000) 2024-02-02T15:48:44.364 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:43 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:43.864+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:49.161607+0000 front 2024-02-02T15:47:49.161481+0000 (oldest deadline 2024-02-02T15:48:13.261576+0000) 2024-02-02T15:48:44.364 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:43 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:43.864+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:56.063095+0000 front 2024-02-02T15:47:56.063158+0000 (oldest deadline 2024-02-02T15:48:21.363242+0000) 2024-02-02T15:48:44.365 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:43 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:43.864+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-02-02T15:48:07.264171+0000 front 2024-02-02T15:48:07.264186+0000 (oldest deadline 2024-02-02T15:48:30.164422+0000) 2024-02-02T15:48:44.365 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:43 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:43.864+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-02-02T15:48:10.165002+0000 front 2024-02-02T15:48:10.164971+0000 (oldest deadline 2024-02-02T15:48:36.064922+0000) 2024-02-02T15:48:44.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:43 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:43.972+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.102474+0000 front 2024-02-02T15:47:33.102427+0000 (oldest deadline 2024-02-02T15:47:57.802602+0000) 2024-02-02T15:48:44.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:43 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:43.972+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:44.106107+0000 front 2024-02-02T15:47:44.106141+0000 (oldest deadline 2024-02-02T15:48:07.006384+0000) 2024-02-02T15:48:44.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:43 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:43.972+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:51.707992+0000 front 2024-02-02T15:47:51.707872+0000 (oldest deadline 2024-02-02T15:48:15.808055+0000) 2024-02-02T15:48:44.440 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:43 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:43.972+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:59.309660+0000 front 2024-02-02T15:47:59.309683+0000 (oldest deadline 2024-02-02T15:48:22.809935+0000) 2024-02-02T15:48:44.441 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:43 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:43.972+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-02-02T15:48:03.911403+0000 front 2024-02-02T15:48:03.911524+0000 (oldest deadline 2024-02-02T15:48:29.211532+0000) 2024-02-02T15:48:44.441 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:43 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9[77079]: 2024-02-02T15:48:43.972+0000 7fb988e00700 -1 osd.9 121 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-02-02T15:48:13.912773+0000 front 2024-02-02T15:48:13.912878+0000 (oldest deadline 2024-02-02T15:48:38.612968+0000) 2024-02-02T15:48:45.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:44 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:44.689+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:35.342656+0000 front 2024-02-02T15:47:35.342649+0000 (oldest deadline 2024-02-02T15:48:01.143219+0000) 2024-02-02T15:48:45.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:44 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:44.689+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:47.544669+0000 front 2024-02-02T15:47:47.544689+0000 (oldest deadline 2024-02-02T15:48:13.444898+0000) 2024-02-02T15:48:45.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:44 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:44.689+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:59.846650+0000 front 2024-02-02T15:47:59.846736+0000 (oldest deadline 2024-02-02T15:48:22.746483+0000) 2024-02-02T15:48:45.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:44 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:44.689+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-02-02T15:48:03.247880+0000 front 2024-02-02T15:48:03.247853+0000 (oldest deadline 2024-02-02T15:48:28.547857+0000) 2024-02-02T15:48:45.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:44 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:44.689+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-02-02T15:48:12.048751+0000 front 2024-02-02T15:48:12.048729+0000 (oldest deadline 2024-02-02T15:48:37.849453+0000) 2024-02-02T15:48:45.364 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:44 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:44.886+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.558440+0000 front 2024-02-02T15:47:33.558460+0000 (oldest deadline 2024-02-02T15:47:59.958896+0000) 2024-02-02T15:48:45.364 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:44 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:44.886+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:43.861259+0000 front 2024-02-02T15:47:43.861243+0000 (oldest deadline 2024-02-02T15:48:09.161042+0000) 2024-02-02T15:48:45.364 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:44 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:44.886+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:49.161607+0000 front 2024-02-02T15:47:49.161481+0000 (oldest deadline 2024-02-02T15:48:13.261576+0000) 2024-02-02T15:48:45.364 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:44 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:44.886+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:56.063095+0000 front 2024-02-02T15:47:56.063158+0000 (oldest deadline 2024-02-02T15:48:21.363242+0000) 2024-02-02T15:48:45.364 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:44 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:44.886+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-02-02T15:48:07.264171+0000 front 2024-02-02T15:48:07.264186+0000 (oldest deadline 2024-02-02T15:48:30.164422+0000) 2024-02-02T15:48:45.365 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:44 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:44.886+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-02-02T15:48:10.165002+0000 front 2024-02-02T15:48:10.164971+0000 (oldest deadline 2024-02-02T15:48:36.064922+0000) 2024-02-02T15:48:45.690 INFO:journalctl@ceph.osd.9.smithi001.stdout:Feb 02 15:48:45 smithi001 bash[132448]: ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-9 2024-02-02T15:48:45.920 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:45 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:45.643+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:35.342656+0000 front 2024-02-02T15:47:35.342649+0000 (oldest deadline 2024-02-02T15:48:01.143219+0000) 2024-02-02T15:48:45.920 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:45 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:45.643+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:47.544669+0000 front 2024-02-02T15:47:47.544689+0000 (oldest deadline 2024-02-02T15:48:13.444898+0000) 2024-02-02T15:48:45.920 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:45 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:45.643+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:59.846650+0000 front 2024-02-02T15:47:59.846736+0000 (oldest deadline 2024-02-02T15:48:22.746483+0000) 2024-02-02T15:48:45.920 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:45 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:45.643+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-02-02T15:48:03.247880+0000 front 2024-02-02T15:48:03.247853+0000 (oldest deadline 2024-02-02T15:48:28.547857+0000) 2024-02-02T15:48:45.920 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:45 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:45.643+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-02-02T15:48:12.048751+0000 front 2024-02-02T15:48:12.048729+0000 (oldest deadline 2024-02-02T15:48:37.849453+0000) 2024-02-02T15:48:46.364 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:45 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:45.900+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.558440+0000 front 2024-02-02T15:47:33.558460+0000 (oldest deadline 2024-02-02T15:47:59.958896+0000) 2024-02-02T15:48:46.364 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:45 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:45.900+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:43.861259+0000 front 2024-02-02T15:47:43.861243+0000 (oldest deadline 2024-02-02T15:48:09.161042+0000) 2024-02-02T15:48:46.364 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:45 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:45.900+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:49.161607+0000 front 2024-02-02T15:47:49.161481+0000 (oldest deadline 2024-02-02T15:48:13.261576+0000) 2024-02-02T15:48:46.364 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:45 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:45.900+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:56.063095+0000 front 2024-02-02T15:47:56.063158+0000 (oldest deadline 2024-02-02T15:48:21.363242+0000) 2024-02-02T15:48:46.365 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:45 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:45.900+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-02-02T15:48:07.264171+0000 front 2024-02-02T15:48:07.264186+0000 (oldest deadline 2024-02-02T15:48:30.164422+0000) 2024-02-02T15:48:46.365 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:45 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:45.900+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-02-02T15:48:10.165002+0000 front 2024-02-02T15:48:10.164971+0000 (oldest deadline 2024-02-02T15:48:36.064922+0000) 2024-02-02T15:48:46.365 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:45 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:45.900+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-02-02T15:48:21.966108+0000 front 2024-02-02T15:48:21.966150+0000 (oldest deadline 2024-02-02T15:48:45.465975+0000) 2024-02-02T15:48:46.867 DEBUG:teuthology.orchestra.run.smithi001:> sudo pkill -f 'journalctl -f -n 0 -u ceph-371a409c-c1de-11ee-95b6-87774f69a715@osd.9.service' 2024-02-02T15:48:46.919 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:46 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:46.648+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:35.342656+0000 front 2024-02-02T15:47:35.342649+0000 (oldest deadline 2024-02-02T15:48:01.143219+0000) 2024-02-02T15:48:46.920 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:46 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:46.648+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:47.544669+0000 front 2024-02-02T15:47:47.544689+0000 (oldest deadline 2024-02-02T15:48:13.444898+0000) 2024-02-02T15:48:46.920 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:46 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:46.648+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:59.846650+0000 front 2024-02-02T15:47:59.846736+0000 (oldest deadline 2024-02-02T15:48:22.746483+0000) 2024-02-02T15:48:46.920 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:46 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:46.648+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-02-02T15:48:03.247880+0000 front 2024-02-02T15:48:03.247853+0000 (oldest deadline 2024-02-02T15:48:28.547857+0000) 2024-02-02T15:48:46.920 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:46 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:46.648+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-02-02T15:48:12.048751+0000 front 2024-02-02T15:48:12.048729+0000 (oldest deadline 2024-02-02T15:48:37.849453+0000) 2024-02-02T15:48:46.952 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-02T15:48:46.952 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-02-02T15:48:46.952 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-02-02T15:48:46.952 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-371a409c-c1de-11ee-95b6-87774f69a715@osd.10 2024-02-02T15:48:47.209 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:46 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:46.939+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.558440+0000 front 2024-02-02T15:47:33.558460+0000 (oldest deadline 2024-02-02T15:47:59.958896+0000) 2024-02-02T15:48:47.210 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:46 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:46.939+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:43.861259+0000 front 2024-02-02T15:47:43.861243+0000 (oldest deadline 2024-02-02T15:48:09.161042+0000) 2024-02-02T15:48:47.210 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:46 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:46.939+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:49.161607+0000 front 2024-02-02T15:47:49.161481+0000 (oldest deadline 2024-02-02T15:48:13.261576+0000) 2024-02-02T15:48:47.210 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:46 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:46.939+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:56.063095+0000 front 2024-02-02T15:47:56.063158+0000 (oldest deadline 2024-02-02T15:48:21.363242+0000) 2024-02-02T15:48:47.210 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:46 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:46.939+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-02-02T15:48:07.264171+0000 front 2024-02-02T15:48:07.264186+0000 (oldest deadline 2024-02-02T15:48:30.164422+0000) 2024-02-02T15:48:47.210 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:46 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:46.939+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-02-02T15:48:10.165002+0000 front 2024-02-02T15:48:10.164971+0000 (oldest deadline 2024-02-02T15:48:36.064922+0000) 2024-02-02T15:48:47.210 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:46 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:46.939+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-02-02T15:48:21.966108+0000 front 2024-02-02T15:48:21.966150+0000 (oldest deadline 2024-02-02T15:48:45.465975+0000) 2024-02-02T15:48:47.210 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:46 smithi071 systemd[1]: Stopping Ceph osd.10 for 371a409c-c1de-11ee-95b6-87774f69a715... 2024-02-02T15:48:47.210 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:47 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:47.209+0000 7f4069cda700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-02T15:48:47.211 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:47 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:47.209+0000 7f4069cda700 -1 osd.10 121 *** Got signal Terminated *** 2024-02-02T15:48:47.211 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:47 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:47.209+0000 7f4069cda700 -1 osd.10 121 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-02T15:48:47.920 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:47 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:47.650+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:35.342656+0000 front 2024-02-02T15:47:35.342649+0000 (oldest deadline 2024-02-02T15:48:01.143219+0000) 2024-02-02T15:48:47.920 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:47 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:47.650+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:47.544669+0000 front 2024-02-02T15:47:47.544689+0000 (oldest deadline 2024-02-02T15:48:13.444898+0000) 2024-02-02T15:48:47.920 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:47 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:47.650+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:59.846650+0000 front 2024-02-02T15:47:59.846736+0000 (oldest deadline 2024-02-02T15:48:22.746483+0000) 2024-02-02T15:48:47.921 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:47 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:47.650+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-02-02T15:48:03.247880+0000 front 2024-02-02T15:48:03.247853+0000 (oldest deadline 2024-02-02T15:48:28.547857+0000) 2024-02-02T15:48:47.921 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:47 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:47.650+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-02-02T15:48:12.048751+0000 front 2024-02-02T15:48:12.048729+0000 (oldest deadline 2024-02-02T15:48:37.849453+0000) 2024-02-02T15:48:47.921 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:47 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:47.650+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-02-02T15:48:18.950663+0000 front 2024-02-02T15:48:18.950635+0000 (oldest deadline 2024-02-02T15:48:47.150912+0000) 2024-02-02T15:48:48.364 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:47 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:47.982+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.558440+0000 front 2024-02-02T15:47:33.558460+0000 (oldest deadline 2024-02-02T15:47:59.958896+0000) 2024-02-02T15:48:48.365 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:47 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:47.982+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:43.861259+0000 front 2024-02-02T15:47:43.861243+0000 (oldest deadline 2024-02-02T15:48:09.161042+0000) 2024-02-02T15:48:48.365 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:47 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:47.982+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:49.161607+0000 front 2024-02-02T15:47:49.161481+0000 (oldest deadline 2024-02-02T15:48:13.261576+0000) 2024-02-02T15:48:48.365 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:47 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:47.982+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:56.063095+0000 front 2024-02-02T15:47:56.063158+0000 (oldest deadline 2024-02-02T15:48:21.363242+0000) 2024-02-02T15:48:48.366 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:47 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:47.982+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-02-02T15:48:07.264171+0000 front 2024-02-02T15:48:07.264186+0000 (oldest deadline 2024-02-02T15:48:30.164422+0000) 2024-02-02T15:48:48.366 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:47 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:47.982+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-02-02T15:48:10.165002+0000 front 2024-02-02T15:48:10.164971+0000 (oldest deadline 2024-02-02T15:48:36.064922+0000) 2024-02-02T15:48:48.366 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:47 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:47.982+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-02-02T15:48:21.966108+0000 front 2024-02-02T15:48:21.966150+0000 (oldest deadline 2024-02-02T15:48:45.465975+0000) 2024-02-02T15:48:49.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:48 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:48.699+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:35.342656+0000 front 2024-02-02T15:47:35.342649+0000 (oldest deadline 2024-02-02T15:48:01.143219+0000) 2024-02-02T15:48:49.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:48 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:48.699+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:47.544669+0000 front 2024-02-02T15:47:47.544689+0000 (oldest deadline 2024-02-02T15:48:13.444898+0000) 2024-02-02T15:48:49.171 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:48 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:48.699+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:59.846650+0000 front 2024-02-02T15:47:59.846736+0000 (oldest deadline 2024-02-02T15:48:22.746483+0000) 2024-02-02T15:48:49.171 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:48 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:48.699+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-02-02T15:48:03.247880+0000 front 2024-02-02T15:48:03.247853+0000 (oldest deadline 2024-02-02T15:48:28.547857+0000) 2024-02-02T15:48:49.171 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:48 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:48.699+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-02-02T15:48:12.048751+0000 front 2024-02-02T15:48:12.048729+0000 (oldest deadline 2024-02-02T15:48:37.849453+0000) 2024-02-02T15:48:49.171 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:48 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:48.699+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-02-02T15:48:18.950663+0000 front 2024-02-02T15:48:18.950635+0000 (oldest deadline 2024-02-02T15:48:47.150912+0000) 2024-02-02T15:48:49.364 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:49 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:49.003+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.558440+0000 front 2024-02-02T15:47:33.558460+0000 (oldest deadline 2024-02-02T15:47:59.958896+0000) 2024-02-02T15:48:49.364 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:49 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:49.003+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:43.861259+0000 front 2024-02-02T15:47:43.861243+0000 (oldest deadline 2024-02-02T15:48:09.161042+0000) 2024-02-02T15:48:49.365 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:49 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:49.003+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:49.161607+0000 front 2024-02-02T15:47:49.161481+0000 (oldest deadline 2024-02-02T15:48:13.261576+0000) 2024-02-02T15:48:49.365 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:49 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:49.003+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:56.063095+0000 front 2024-02-02T15:47:56.063158+0000 (oldest deadline 2024-02-02T15:48:21.363242+0000) 2024-02-02T15:48:49.365 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:49 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:49.003+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-02-02T15:48:07.264171+0000 front 2024-02-02T15:48:07.264186+0000 (oldest deadline 2024-02-02T15:48:30.164422+0000) 2024-02-02T15:48:49.365 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:49 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:49.003+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-02-02T15:48:10.165002+0000 front 2024-02-02T15:48:10.164971+0000 (oldest deadline 2024-02-02T15:48:36.064922+0000) 2024-02-02T15:48:49.366 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:49 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:49.003+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-02-02T15:48:21.966108+0000 front 2024-02-02T15:48:21.966150+0000 (oldest deadline 2024-02-02T15:48:45.465975+0000) 2024-02-02T15:48:50.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:49 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:49.677+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:35.342656+0000 front 2024-02-02T15:47:35.342649+0000 (oldest deadline 2024-02-02T15:48:01.143219+0000) 2024-02-02T15:48:50.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:49 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:49.677+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:47.544669+0000 front 2024-02-02T15:47:47.544689+0000 (oldest deadline 2024-02-02T15:48:13.444898+0000) 2024-02-02T15:48:50.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:49 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:49.677+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:59.846650+0000 front 2024-02-02T15:47:59.846736+0000 (oldest deadline 2024-02-02T15:48:22.746483+0000) 2024-02-02T15:48:50.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:49 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:49.677+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-02-02T15:48:03.247880+0000 front 2024-02-02T15:48:03.247853+0000 (oldest deadline 2024-02-02T15:48:28.547857+0000) 2024-02-02T15:48:50.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:49 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:49.677+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-02-02T15:48:12.048751+0000 front 2024-02-02T15:48:12.048729+0000 (oldest deadline 2024-02-02T15:48:37.849453+0000) 2024-02-02T15:48:50.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:49 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:49.677+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-02-02T15:48:18.950663+0000 front 2024-02-02T15:48:18.950635+0000 (oldest deadline 2024-02-02T15:48:47.150912+0000) 2024-02-02T15:48:50.364 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:49 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:49.998+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.558440+0000 front 2024-02-02T15:47:33.558460+0000 (oldest deadline 2024-02-02T15:47:59.958896+0000) 2024-02-02T15:48:50.364 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:49 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:49.998+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:43.861259+0000 front 2024-02-02T15:47:43.861243+0000 (oldest deadline 2024-02-02T15:48:09.161042+0000) 2024-02-02T15:48:50.364 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:49 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:49.998+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:49.161607+0000 front 2024-02-02T15:47:49.161481+0000 (oldest deadline 2024-02-02T15:48:13.261576+0000) 2024-02-02T15:48:50.365 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:49 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:49.998+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:56.063095+0000 front 2024-02-02T15:47:56.063158+0000 (oldest deadline 2024-02-02T15:48:21.363242+0000) 2024-02-02T15:48:50.365 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:49 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:49.998+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-02-02T15:48:07.264171+0000 front 2024-02-02T15:48:07.264186+0000 (oldest deadline 2024-02-02T15:48:30.164422+0000) 2024-02-02T15:48:50.365 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:49 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:49.998+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-02-02T15:48:10.165002+0000 front 2024-02-02T15:48:10.164971+0000 (oldest deadline 2024-02-02T15:48:36.064922+0000) 2024-02-02T15:48:50.365 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:49 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:49.998+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-02-02T15:48:21.966108+0000 front 2024-02-02T15:48:21.966150+0000 (oldest deadline 2024-02-02T15:48:45.465975+0000) 2024-02-02T15:48:51.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:50 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:50.692+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:35.342656+0000 front 2024-02-02T15:47:35.342649+0000 (oldest deadline 2024-02-02T15:48:01.143219+0000) 2024-02-02T15:48:51.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:50 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:50.692+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:47.544669+0000 front 2024-02-02T15:47:47.544689+0000 (oldest deadline 2024-02-02T15:48:13.444898+0000) 2024-02-02T15:48:51.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:50 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:50.692+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:59.846650+0000 front 2024-02-02T15:47:59.846736+0000 (oldest deadline 2024-02-02T15:48:22.746483+0000) 2024-02-02T15:48:51.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:50 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:50.692+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-02-02T15:48:03.247880+0000 front 2024-02-02T15:48:03.247853+0000 (oldest deadline 2024-02-02T15:48:28.547857+0000) 2024-02-02T15:48:51.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:50 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:50.692+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-02-02T15:48:12.048751+0000 front 2024-02-02T15:48:12.048729+0000 (oldest deadline 2024-02-02T15:48:37.849453+0000) 2024-02-02T15:48:51.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:50 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:50.692+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-02-02T15:48:18.950663+0000 front 2024-02-02T15:48:18.950635+0000 (oldest deadline 2024-02-02T15:48:47.150912+0000) 2024-02-02T15:48:51.364 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:51 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:51.019+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.558440+0000 front 2024-02-02T15:47:33.558460+0000 (oldest deadline 2024-02-02T15:47:59.958896+0000) 2024-02-02T15:48:51.364 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:51 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:51.019+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:43.861259+0000 front 2024-02-02T15:47:43.861243+0000 (oldest deadline 2024-02-02T15:48:09.161042+0000) 2024-02-02T15:48:51.365 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:51 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:51.019+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:49.161607+0000 front 2024-02-02T15:47:49.161481+0000 (oldest deadline 2024-02-02T15:48:13.261576+0000) 2024-02-02T15:48:51.365 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:51 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:51.019+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:56.063095+0000 front 2024-02-02T15:47:56.063158+0000 (oldest deadline 2024-02-02T15:48:21.363242+0000) 2024-02-02T15:48:51.365 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:51 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:51.019+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-02-02T15:48:07.264171+0000 front 2024-02-02T15:48:07.264186+0000 (oldest deadline 2024-02-02T15:48:30.164422+0000) 2024-02-02T15:48:51.365 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:51 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:51.019+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-02-02T15:48:10.165002+0000 front 2024-02-02T15:48:10.164971+0000 (oldest deadline 2024-02-02T15:48:36.064922+0000) 2024-02-02T15:48:51.365 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:51 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:51.019+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-02-02T15:48:21.966108+0000 front 2024-02-02T15:48:21.966150+0000 (oldest deadline 2024-02-02T15:48:45.465975+0000) 2024-02-02T15:48:52.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:51 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:51.725+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:35.342656+0000 front 2024-02-02T15:47:35.342649+0000 (oldest deadline 2024-02-02T15:48:01.143219+0000) 2024-02-02T15:48:52.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:51 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:51.725+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:47.544669+0000 front 2024-02-02T15:47:47.544689+0000 (oldest deadline 2024-02-02T15:48:13.444898+0000) 2024-02-02T15:48:52.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:51 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:51.725+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:59.846650+0000 front 2024-02-02T15:47:59.846736+0000 (oldest deadline 2024-02-02T15:48:22.746483+0000) 2024-02-02T15:48:52.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:51 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:51.725+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-02-02T15:48:03.247880+0000 front 2024-02-02T15:48:03.247853+0000 (oldest deadline 2024-02-02T15:48:28.547857+0000) 2024-02-02T15:48:52.171 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:51 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:51.725+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-02-02T15:48:12.048751+0000 front 2024-02-02T15:48:12.048729+0000 (oldest deadline 2024-02-02T15:48:37.849453+0000) 2024-02-02T15:48:52.171 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:51 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:51.725+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-02-02T15:48:18.950663+0000 front 2024-02-02T15:48:18.950635+0000 (oldest deadline 2024-02-02T15:48:47.150912+0000) 2024-02-02T15:48:52.364 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:51 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:51.975+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:33.558440+0000 front 2024-02-02T15:47:33.558460+0000 (oldest deadline 2024-02-02T15:47:59.958896+0000) 2024-02-02T15:48:52.364 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:51 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:51.975+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.71:6804 osd.1 since back 2024-02-02T15:47:43.861259+0000 front 2024-02-02T15:47:43.861243+0000 (oldest deadline 2024-02-02T15:48:09.161042+0000) 2024-02-02T15:48:52.365 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:51 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:51.975+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:49.161607+0000 front 2024-02-02T15:47:49.161481+0000 (oldest deadline 2024-02-02T15:48:13.261576+0000) 2024-02-02T15:48:52.365 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:51 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:51.975+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:56.063095+0000 front 2024-02-02T15:47:56.063158+0000 (oldest deadline 2024-02-02T15:48:21.363242+0000) 2024-02-02T15:48:52.365 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:51 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:51.975+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-02-02T15:48:07.264171+0000 front 2024-02-02T15:48:07.264186+0000 (oldest deadline 2024-02-02T15:48:30.164422+0000) 2024-02-02T15:48:52.365 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:51 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:51.975+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-02-02T15:48:10.165002+0000 front 2024-02-02T15:48:10.164971+0000 (oldest deadline 2024-02-02T15:48:36.064922+0000) 2024-02-02T15:48:52.365 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:51 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10[73419]: 2024-02-02T15:48:51.975+0000 7f40622c8700 -1 osd.10 121 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-02-02T15:48:21.966108+0000 front 2024-02-02T15:48:21.966150+0000 (oldest deadline 2024-02-02T15:48:45.465975+0000) 2024-02-02T15:48:53.114 INFO:journalctl@ceph.osd.10.smithi071.stdout:Feb 02 15:48:52 smithi071 bash[89839]: ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-10 2024-02-02T15:48:53.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:52 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:52.728+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:35.342656+0000 front 2024-02-02T15:47:35.342649+0000 (oldest deadline 2024-02-02T15:48:01.143219+0000) 2024-02-02T15:48:53.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:52 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:52.728+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:47.544669+0000 front 2024-02-02T15:47:47.544689+0000 (oldest deadline 2024-02-02T15:48:13.444898+0000) 2024-02-02T15:48:53.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:52 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:52.728+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:59.846650+0000 front 2024-02-02T15:47:59.846736+0000 (oldest deadline 2024-02-02T15:48:22.746483+0000) 2024-02-02T15:48:53.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:52 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:52.728+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-02-02T15:48:03.247880+0000 front 2024-02-02T15:48:03.247853+0000 (oldest deadline 2024-02-02T15:48:28.547857+0000) 2024-02-02T15:48:53.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:52 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:52.728+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-02-02T15:48:12.048751+0000 front 2024-02-02T15:48:12.048729+0000 (oldest deadline 2024-02-02T15:48:37.849453+0000) 2024-02-02T15:48:53.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:52 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:52.728+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-02-02T15:48:18.950663+0000 front 2024-02-02T15:48:18.950635+0000 (oldest deadline 2024-02-02T15:48:47.150912+0000) 2024-02-02T15:48:54.038 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-371a409c-c1de-11ee-95b6-87774f69a715@osd.10.service' 2024-02-02T15:48:54.078 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-02T15:48:54.079 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-02-02T15:48:54.079 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-02-02T15:48:54.079 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-371a409c-c1de-11ee-95b6-87774f69a715@osd.11 2024-02-02T15:48:54.110 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:53 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:53.778+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:35.342656+0000 front 2024-02-02T15:47:35.342649+0000 (oldest deadline 2024-02-02T15:48:01.143219+0000) 2024-02-02T15:48:54.110 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:53 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:53.778+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:47.544669+0000 front 2024-02-02T15:47:47.544689+0000 (oldest deadline 2024-02-02T15:48:13.444898+0000) 2024-02-02T15:48:54.110 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:53 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:53.778+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:59.846650+0000 front 2024-02-02T15:47:59.846736+0000 (oldest deadline 2024-02-02T15:48:22.746483+0000) 2024-02-02T15:48:54.110 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:53 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:53.778+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-02-02T15:48:03.247880+0000 front 2024-02-02T15:48:03.247853+0000 (oldest deadline 2024-02-02T15:48:28.547857+0000) 2024-02-02T15:48:54.110 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:53 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:53.778+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-02-02T15:48:12.048751+0000 front 2024-02-02T15:48:12.048729+0000 (oldest deadline 2024-02-02T15:48:37.849453+0000) 2024-02-02T15:48:54.110 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:53 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:53.778+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-02-02T15:48:18.950663+0000 front 2024-02-02T15:48:18.950635+0000 (oldest deadline 2024-02-02T15:48:47.150912+0000) 2024-02-02T15:48:54.111 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:53 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:53.778+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.71:6820 osd.7 since back 2024-02-02T15:48:27.151434+0000 front 2024-02-02T15:48:27.151389+0000 (oldest deadline 2024-02-02T15:48:53.051363+0000) 2024-02-02T15:48:54.420 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:54 smithi142 systemd[1]: Stopping Ceph osd.11 for 371a409c-c1de-11ee-95b6-87774f69a715... 2024-02-02T15:48:54.420 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:54 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:54.331+0000 7f9aa0559700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-02T15:48:54.420 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:54 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:54.331+0000 7f9aa0559700 -1 osd.11 121 *** Got signal Terminated *** 2024-02-02T15:48:54.420 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:54 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:54.331+0000 7f9aa0559700 -1 osd.11 121 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-02T15:48:55.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:54 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:54.770+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:35.342656+0000 front 2024-02-02T15:47:35.342649+0000 (oldest deadline 2024-02-02T15:48:01.143219+0000) 2024-02-02T15:48:55.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:54 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:54.770+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:47.544669+0000 front 2024-02-02T15:47:47.544689+0000 (oldest deadline 2024-02-02T15:48:13.444898+0000) 2024-02-02T15:48:55.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:54 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:54.770+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:59.846650+0000 front 2024-02-02T15:47:59.846736+0000 (oldest deadline 2024-02-02T15:48:22.746483+0000) 2024-02-02T15:48:55.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:54 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:54.770+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-02-02T15:48:03.247880+0000 front 2024-02-02T15:48:03.247853+0000 (oldest deadline 2024-02-02T15:48:28.547857+0000) 2024-02-02T15:48:55.171 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:54 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:54.770+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-02-02T15:48:12.048751+0000 front 2024-02-02T15:48:12.048729+0000 (oldest deadline 2024-02-02T15:48:37.849453+0000) 2024-02-02T15:48:55.171 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:54 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:54.770+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-02-02T15:48:18.950663+0000 front 2024-02-02T15:48:18.950635+0000 (oldest deadline 2024-02-02T15:48:47.150912+0000) 2024-02-02T15:48:55.171 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:54 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:54.770+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.71:6820 osd.7 since back 2024-02-02T15:48:27.151434+0000 front 2024-02-02T15:48:27.151389+0000 (oldest deadline 2024-02-02T15:48:53.051363+0000) 2024-02-02T15:48:56.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:55 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:55.755+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:35.342656+0000 front 2024-02-02T15:47:35.342649+0000 (oldest deadline 2024-02-02T15:48:01.143219+0000) 2024-02-02T15:48:56.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:55 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:55.755+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:47.544669+0000 front 2024-02-02T15:47:47.544689+0000 (oldest deadline 2024-02-02T15:48:13.444898+0000) 2024-02-02T15:48:56.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:55 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:55.755+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:59.846650+0000 front 2024-02-02T15:47:59.846736+0000 (oldest deadline 2024-02-02T15:48:22.746483+0000) 2024-02-02T15:48:56.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:55 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:55.755+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-02-02T15:48:03.247880+0000 front 2024-02-02T15:48:03.247853+0000 (oldest deadline 2024-02-02T15:48:28.547857+0000) 2024-02-02T15:48:56.171 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:55 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:55.755+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-02-02T15:48:12.048751+0000 front 2024-02-02T15:48:12.048729+0000 (oldest deadline 2024-02-02T15:48:37.849453+0000) 2024-02-02T15:48:56.171 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:55 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:55.755+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-02-02T15:48:18.950663+0000 front 2024-02-02T15:48:18.950635+0000 (oldest deadline 2024-02-02T15:48:47.150912+0000) 2024-02-02T15:48:56.171 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:55 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:55.755+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.71:6820 osd.7 since back 2024-02-02T15:48:27.151434+0000 front 2024-02-02T15:48:27.151389+0000 (oldest deadline 2024-02-02T15:48:53.051363+0000) 2024-02-02T15:48:57.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:56 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:56.753+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:35.342656+0000 front 2024-02-02T15:47:35.342649+0000 (oldest deadline 2024-02-02T15:48:01.143219+0000) 2024-02-02T15:48:57.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:56 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:56.753+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:47.544669+0000 front 2024-02-02T15:47:47.544689+0000 (oldest deadline 2024-02-02T15:48:13.444898+0000) 2024-02-02T15:48:57.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:56 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:56.753+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:59.846650+0000 front 2024-02-02T15:47:59.846736+0000 (oldest deadline 2024-02-02T15:48:22.746483+0000) 2024-02-02T15:48:57.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:56 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:56.753+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-02-02T15:48:03.247880+0000 front 2024-02-02T15:48:03.247853+0000 (oldest deadline 2024-02-02T15:48:28.547857+0000) 2024-02-02T15:48:57.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:56 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:56.753+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-02-02T15:48:12.048751+0000 front 2024-02-02T15:48:12.048729+0000 (oldest deadline 2024-02-02T15:48:37.849453+0000) 2024-02-02T15:48:57.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:56 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:56.753+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-02-02T15:48:18.950663+0000 front 2024-02-02T15:48:18.950635+0000 (oldest deadline 2024-02-02T15:48:47.150912+0000) 2024-02-02T15:48:57.171 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:56 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:56.753+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.71:6820 osd.7 since back 2024-02-02T15:48:27.151434+0000 front 2024-02-02T15:48:27.151389+0000 (oldest deadline 2024-02-02T15:48:53.051363+0000) 2024-02-02T15:48:58.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:57 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:57.799+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:35.342656+0000 front 2024-02-02T15:47:35.342649+0000 (oldest deadline 2024-02-02T15:48:01.143219+0000) 2024-02-02T15:48:58.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:57 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:57.799+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:47.544669+0000 front 2024-02-02T15:47:47.544689+0000 (oldest deadline 2024-02-02T15:48:13.444898+0000) 2024-02-02T15:48:58.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:57 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:57.799+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:59.846650+0000 front 2024-02-02T15:47:59.846736+0000 (oldest deadline 2024-02-02T15:48:22.746483+0000) 2024-02-02T15:48:58.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:57 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:57.799+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-02-02T15:48:03.247880+0000 front 2024-02-02T15:48:03.247853+0000 (oldest deadline 2024-02-02T15:48:28.547857+0000) 2024-02-02T15:48:58.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:57 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:57.799+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-02-02T15:48:12.048751+0000 front 2024-02-02T15:48:12.048729+0000 (oldest deadline 2024-02-02T15:48:37.849453+0000) 2024-02-02T15:48:58.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:57 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:57.799+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-02-02T15:48:18.950663+0000 front 2024-02-02T15:48:18.950635+0000 (oldest deadline 2024-02-02T15:48:47.150912+0000) 2024-02-02T15:48:58.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:57 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:57.799+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.71:6820 osd.7 since back 2024-02-02T15:48:27.151434+0000 front 2024-02-02T15:48:27.151389+0000 (oldest deadline 2024-02-02T15:48:53.051363+0000) 2024-02-02T15:48:59.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:58 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:58.806+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6806 osd.0 since back 2024-02-02T15:47:35.342656+0000 front 2024-02-02T15:47:35.342649+0000 (oldest deadline 2024-02-02T15:48:01.143219+0000) 2024-02-02T15:48:59.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:58 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:58.806+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.142:6804 osd.2 since back 2024-02-02T15:47:47.544669+0000 front 2024-02-02T15:47:47.544689+0000 (oldest deadline 2024-02-02T15:48:13.444898+0000) 2024-02-02T15:48:59.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:58 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:58.806+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6814 osd.3 since back 2024-02-02T15:47:59.846650+0000 front 2024-02-02T15:47:59.846736+0000 (oldest deadline 2024-02-02T15:48:22.746483+0000) 2024-02-02T15:48:59.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:58 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:58.806+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.71:6812 osd.4 since back 2024-02-02T15:48:03.247880+0000 front 2024-02-02T15:48:03.247853+0000 (oldest deadline 2024-02-02T15:48:28.547857+0000) 2024-02-02T15:48:59.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:58 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:58.806+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.142:6812 osd.5 since back 2024-02-02T15:48:12.048751+0000 front 2024-02-02T15:48:12.048729+0000 (oldest deadline 2024-02-02T15:48:37.849453+0000) 2024-02-02T15:48:59.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:58 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:58.806+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.1:6822 osd.6 since back 2024-02-02T15:48:18.950663+0000 front 2024-02-02T15:48:18.950635+0000 (oldest deadline 2024-02-02T15:48:47.150912+0000) 2024-02-02T15:48:59.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:58 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:58.806+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.71:6820 osd.7 since back 2024-02-02T15:48:27.151434+0000 front 2024-02-02T15:48:27.151389+0000 (oldest deadline 2024-02-02T15:48:53.051363+0000) 2024-02-02T15:48:59.170 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:48:58 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11[76667]: 2024-02-02T15:48:58.806+0000 7f9a9c34e700 -1 osd.11 121 heartbeat_check: no reply from 172.21.15.142:6820 osd.8 since back 2024-02-02T15:48:37.152296+0000 front 2024-02-02T15:48:37.152238+0000 (oldest deadline 2024-02-02T15:48:58.252312+0000) 2024-02-02T15:49:00.419 INFO:journalctl@ceph.osd.11.smithi142.stdout:Feb 02 15:49:00 smithi142 bash[92439]: ceph-371a409c-c1de-11ee-95b6-87774f69a715-osd-11 2024-02-02T15:49:01.201 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-371a409c-c1de-11ee-95b6-87774f69a715@osd.11.service' 2024-02-02T15:49:01.282 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-02T15:49:01.282 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-02-02T15:49:01.283 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-02-02T15:49:01.283 DEBUG:teuthology.orchestra.run.smithi001:> sudo systemctl stop ceph-371a409c-c1de-11ee-95b6-87774f69a715@mds.a 2024-02-02T15:49:01.682 INFO:journalctl@ceph.mds.a.smithi001.stdout:Feb 02 15:49:01 smithi001 systemd[1]: Stopping Ceph mds.a for 371a409c-c1de-11ee-95b6-87774f69a715... 2024-02-02T15:49:01.939 INFO:journalctl@ceph.mds.a.smithi001.stdout:Feb 02 15:49:01 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mds-a[84735]: 2024-02-02T15:49:01.682+0000 7f272f400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-02T15:49:01.940 INFO:journalctl@ceph.mds.a.smithi001.stdout:Feb 02 15:49:01 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mds-a[84735]: 2024-02-02T15:49:01.682+0000 7f272f400700 -1 mds.a *** got signal Terminated *** 2024-02-02T15:49:03.190 INFO:journalctl@ceph.mds.a.smithi001.stdout:Feb 02 15:49:02 smithi001 bash[132715]: ceph-371a409c-c1de-11ee-95b6-87774f69a715-mds-a 2024-02-02T15:49:03.317 DEBUG:teuthology.orchestra.run.smithi001:> sudo pkill -f 'journalctl -f -n 0 -u ceph-371a409c-c1de-11ee-95b6-87774f69a715@mds.a.service' 2024-02-02T15:49:03.400 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-02T15:49:03.401 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-02-02T15:49:03.401 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-02-02T15:49:03.401 DEBUG:teuthology.orchestra.run.smithi001:> sudo systemctl stop ceph-371a409c-c1de-11ee-95b6-87774f69a715@mds.d 2024-02-02T15:49:03.690 INFO:journalctl@ceph.mds.d.smithi001.stdout:Feb 02 15:49:03 smithi001 systemd[1]: Stopping Ceph mds.d for 371a409c-c1de-11ee-95b6-87774f69a715... 2024-02-02T15:49:04.190 INFO:journalctl@ceph.mds.d.smithi001.stdout:Feb 02 15:49:03 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mds-d[83781]: 2024-02-02T15:49:03.757+0000 7f8f73800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-02T15:49:04.190 INFO:journalctl@ceph.mds.d.smithi001.stdout:Feb 02 15:49:03 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mds-d[83781]: 2024-02-02T15:49:03.757+0000 7f8f73800700 -1 mds.d *** got signal Terminated *** 2024-02-02T15:49:05.939 INFO:journalctl@ceph.mds.d.smithi001.stdout:Feb 02 15:49:05 smithi001 bash[132853]: ceph-371a409c-c1de-11ee-95b6-87774f69a715-mds-d 2024-02-02T15:49:06.070 DEBUG:teuthology.orchestra.run.smithi001:> sudo pkill -f 'journalctl -f -n 0 -u ceph-371a409c-c1de-11ee-95b6-87774f69a715@mds.d.service' 2024-02-02T15:49:06.151 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-02T15:49:06.151 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-02-02T15:49:06.152 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-02-02T15:49:06.152 DEBUG:teuthology.orchestra.run.smithi001:> sudo systemctl stop ceph-371a409c-c1de-11ee-95b6-87774f69a715@mds.g 2024-02-02T15:49:06.440 INFO:journalctl@ceph.mds.g.smithi001.stdout:Feb 02 15:49:06 smithi001 systemd[1]: Stopping Ceph mds.g for 371a409c-c1de-11ee-95b6-87774f69a715... 2024-02-02T15:49:06.940 INFO:journalctl@ceph.mds.g.smithi001.stdout:Feb 02 15:49:06 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mds-g[84349]: 2024-02-02T15:49:06.514+0000 7f961a400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-02T15:49:06.943 INFO:journalctl@ceph.mds.g.smithi001.stdout:Feb 02 15:49:06 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mds-g[84349]: 2024-02-02T15:49:06.514+0000 7f961a400700 -1 mds.g *** got signal Terminated *** 2024-02-02T15:49:10.940 INFO:journalctl@ceph.mds.g.smithi001.stdout:Feb 02 15:49:10 smithi001 bash[132992]: ceph-371a409c-c1de-11ee-95b6-87774f69a715-mds-g 2024-02-02T15:49:11.195 DEBUG:teuthology.orchestra.run.smithi001:> sudo pkill -f 'journalctl -f -n 0 -u ceph-371a409c-c1de-11ee-95b6-87774f69a715@mds.g.service' 2024-02-02T15:49:11.276 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-02T15:49:11.276 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-02-02T15:49:11.276 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-02-02T15:49:11.277 DEBUG:teuthology.orchestra.run.smithi001:> sudo systemctl stop ceph-371a409c-c1de-11ee-95b6-87774f69a715@mds.j 2024-02-02T15:49:11.440 INFO:journalctl@ceph.mds.j.smithi001.stdout:Feb 02 15:49:11 smithi001 systemd[1]: Stopping Ceph mds.j for 371a409c-c1de-11ee-95b6-87774f69a715... 2024-02-02T15:49:11.940 INFO:journalctl@ceph.mds.j.smithi001.stdout:Feb 02 15:49:11 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mds-j[83394]: 2024-02-02T15:49:11.680+0000 7fd556a00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-02T15:49:11.940 INFO:journalctl@ceph.mds.j.smithi001.stdout:Feb 02 15:49:11 smithi001 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mds-j[83394]: 2024-02-02T15:49:11.680+0000 7fd556a00700 -1 mds.j *** got signal Terminated *** 2024-02-02T15:49:15.440 INFO:journalctl@ceph.mds.j.smithi001.stdout:Feb 02 15:49:15 smithi001 bash[133130]: ceph-371a409c-c1de-11ee-95b6-87774f69a715-mds-j 2024-02-02T15:49:15.517 DEBUG:teuthology.orchestra.run.smithi001:> sudo pkill -f 'journalctl -f -n 0 -u ceph-371a409c-c1de-11ee-95b6-87774f69a715@mds.j.service' 2024-02-02T15:49:15.596 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-02T15:49:15.596 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-02-02T15:49:15.596 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-02-02T15:49:15.597 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-371a409c-c1de-11ee-95b6-87774f69a715@mds.b 2024-02-02T15:49:16.114 INFO:journalctl@ceph.mds.b.smithi071.stdout:Feb 02 15:49:15 smithi071 systemd[1]: Stopping Ceph mds.b for 371a409c-c1de-11ee-95b6-87774f69a715... 2024-02-02T15:49:16.115 INFO:journalctl@ceph.mds.b.smithi071.stdout:Feb 02 15:49:15 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mds-b[79660]: 2024-02-02T15:49:15.829+0000 7f7f08ded700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-02T15:49:16.115 INFO:journalctl@ceph.mds.b.smithi071.stdout:Feb 02 15:49:15 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mds-b[79660]: 2024-02-02T15:49:15.829+0000 7f7f08ded700 -1 mds.b *** got signal Terminated *** 2024-02-02T15:49:19.229 INFO:journalctl@ceph.mds.b.smithi071.stdout:Feb 02 15:49:18 smithi071 bash[90094]: ceph-371a409c-c1de-11ee-95b6-87774f69a715-mds-b 2024-02-02T15:49:19.236 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-371a409c-c1de-11ee-95b6-87774f69a715@mds.b.service' 2024-02-02T15:49:19.272 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-02T15:49:19.272 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-02-02T15:49:19.272 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-02-02T15:49:19.272 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-371a409c-c1de-11ee-95b6-87774f69a715@mds.e 2024-02-02T15:49:19.536 INFO:journalctl@ceph.mds.e.smithi071.stdout:Feb 02 15:49:19 smithi071 systemd[1]: Stopping Ceph mds.e for 371a409c-c1de-11ee-95b6-87774f69a715... 2024-02-02T15:49:19.864 INFO:journalctl@ceph.mds.e.smithi071.stdout:Feb 02 15:49:19 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mds-e[78979]: 2024-02-02T15:49:19.537+0000 7f5241d94700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-02T15:49:19.864 INFO:journalctl@ceph.mds.e.smithi071.stdout:Feb 02 15:49:19 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mds-e[78979]: 2024-02-02T15:49:19.537+0000 7f5241d94700 -1 mds.e *** got signal Terminated *** 2024-02-02T15:49:23.200 INFO:journalctl@ceph.mds.e.smithi071.stdout:Feb 02 15:49:22 smithi071 bash[90233]: ceph-371a409c-c1de-11ee-95b6-87774f69a715-mds-e 2024-02-02T15:49:23.205 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-371a409c-c1de-11ee-95b6-87774f69a715@mds.e.service' 2024-02-02T15:49:23.239 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-02T15:49:23.239 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-02-02T15:49:23.239 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-02-02T15:49:23.240 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-371a409c-c1de-11ee-95b6-87774f69a715@mds.h 2024-02-02T15:49:23.493 INFO:journalctl@ceph.mds.h.smithi071.stdout:Feb 02 15:49:23 smithi071 systemd[1]: Stopping Ceph mds.h for 371a409c-c1de-11ee-95b6-87774f69a715... 2024-02-02T15:49:23.864 INFO:journalctl@ceph.mds.h.smithi071.stdout:Feb 02 15:49:23 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mds-h[79320]: 2024-02-02T15:49:23.494+0000 7ffa4faea700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-02T15:49:23.864 INFO:journalctl@ceph.mds.h.smithi071.stdout:Feb 02 15:49:23 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mds-h[79320]: 2024-02-02T15:49:23.494+0000 7ffa4faea700 -1 mds.h *** got signal Terminated *** 2024-02-02T15:49:27.302 INFO:journalctl@ceph.mds.h.smithi071.stdout:Feb 02 15:49:26 smithi071 bash[90371]: ceph-371a409c-c1de-11ee-95b6-87774f69a715-mds-h 2024-02-02T15:49:27.307 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-371a409c-c1de-11ee-95b6-87774f69a715@mds.h.service' 2024-02-02T15:49:27.341 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-02T15:49:27.341 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-02-02T15:49:27.342 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-02-02T15:49:27.342 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-371a409c-c1de-11ee-95b6-87774f69a715@mds.k 2024-02-02T15:49:27.609 INFO:journalctl@ceph.mds.k.smithi071.stdout:Feb 02 15:49:27 smithi071 systemd[1]: Stopping Ceph mds.k for 371a409c-c1de-11ee-95b6-87774f69a715... 2024-02-02T15:49:27.864 INFO:journalctl@ceph.mds.k.smithi071.stdout:Feb 02 15:49:27 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mds-k[78638]: 2024-02-02T15:49:27.609+0000 7f233d639700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-02T15:49:27.864 INFO:journalctl@ceph.mds.k.smithi071.stdout:Feb 02 15:49:27 smithi071 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mds-k[78638]: 2024-02-02T15:49:27.609+0000 7f233d639700 -1 mds.k *** got signal Terminated *** 2024-02-02T15:49:31.250 INFO:journalctl@ceph.mds.k.smithi071.stdout:Feb 02 15:49:30 smithi071 bash[90510]: ceph-371a409c-c1de-11ee-95b6-87774f69a715-mds-k 2024-02-02T15:49:31.256 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-371a409c-c1de-11ee-95b6-87774f69a715@mds.k.service' 2024-02-02T15:49:31.290 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-02T15:49:31.291 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-02-02T15:49:31.291 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-02-02T15:49:31.291 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-371a409c-c1de-11ee-95b6-87774f69a715@mds.c 2024-02-02T15:49:31.670 INFO:journalctl@ceph.mds.c.smithi142.stdout:Feb 02 15:49:31 smithi142 systemd[1]: Stopping Ceph mds.c for 371a409c-c1de-11ee-95b6-87774f69a715... 2024-02-02T15:49:31.670 INFO:journalctl@ceph.mds.c.smithi142.stdout:Feb 02 15:49:31 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mds-c[82857]: 2024-02-02T15:49:31.525+0000 7f1355a7e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-02T15:49:31.670 INFO:journalctl@ceph.mds.c.smithi142.stdout:Feb 02 15:49:31 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mds-c[82857]: 2024-02-02T15:49:31.525+0000 7f1355a7e700 -1 mds.c *** got signal Terminated *** 2024-02-02T15:49:35.241 INFO:journalctl@ceph.mds.c.smithi142.stdout:Feb 02 15:49:35 smithi142 bash[92693]: ceph-371a409c-c1de-11ee-95b6-87774f69a715-mds-c 2024-02-02T15:49:35.247 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-371a409c-c1de-11ee-95b6-87774f69a715@mds.c.service' 2024-02-02T15:49:35.283 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-02T15:49:35.284 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-02-02T15:49:35.284 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-02-02T15:49:35.284 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-371a409c-c1de-11ee-95b6-87774f69a715@mds.f 2024-02-02T15:49:35.541 INFO:journalctl@ceph.mds.f.smithi142.stdout:Feb 02 15:49:35 smithi142 systemd[1]: Stopping Ceph mds.f for 371a409c-c1de-11ee-95b6-87774f69a715... 2024-02-02T15:49:35.920 INFO:journalctl@ceph.mds.f.smithi142.stdout:Feb 02 15:49:35 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mds-f[82516]: 2024-02-02T15:49:35.541+0000 7f2d4c2e6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-02T15:49:35.920 INFO:journalctl@ceph.mds.f.smithi142.stdout:Feb 02 15:49:35 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mds-f[82516]: 2024-02-02T15:49:35.541+0000 7f2d4c2e6700 -1 mds.f *** got signal Terminated *** 2024-02-02T15:49:39.476 INFO:journalctl@ceph.mds.f.smithi142.stdout:Feb 02 15:49:39 smithi142 bash[92833]: ceph-371a409c-c1de-11ee-95b6-87774f69a715-mds-f 2024-02-02T15:49:39.482 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-371a409c-c1de-11ee-95b6-87774f69a715@mds.f.service' 2024-02-02T15:49:39.516 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-02T15:49:39.517 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-02-02T15:49:39.517 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-02-02T15:49:39.517 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-371a409c-c1de-11ee-95b6-87774f69a715@mds.i 2024-02-02T15:49:39.781 INFO:journalctl@ceph.mds.i.smithi142.stdout:Feb 02 15:49:39 smithi142 systemd[1]: Stopping Ceph mds.i for 371a409c-c1de-11ee-95b6-87774f69a715... 2024-02-02T15:49:40.169 INFO:journalctl@ceph.mds.i.smithi142.stdout:Feb 02 15:49:39 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mds-i[83198]: 2024-02-02T15:49:39.782+0000 7fd5bd39c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-02T15:49:40.170 INFO:journalctl@ceph.mds.i.smithi142.stdout:Feb 02 15:49:39 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mds-i[83198]: 2024-02-02T15:49:39.782+0000 7fd5bd39c700 -1 mds.i *** got signal Terminated *** 2024-02-02T15:49:43.334 INFO:journalctl@ceph.mds.i.smithi142.stdout:Feb 02 15:49:43 smithi142 bash[92970]: ceph-371a409c-c1de-11ee-95b6-87774f69a715-mds-i 2024-02-02T15:49:43.339 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-371a409c-c1de-11ee-95b6-87774f69a715@mds.i.service' 2024-02-02T15:49:43.373 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-02T15:49:43.373 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-02-02T15:49:43.373 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-02-02T15:49:43.374 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ceph-371a409c-c1de-11ee-95b6-87774f69a715@mds.l 2024-02-02T15:49:43.622 INFO:journalctl@ceph.mds.l.smithi142.stdout:Feb 02 15:49:43 smithi142 systemd[1]: Stopping Ceph mds.l for 371a409c-c1de-11ee-95b6-87774f69a715... 2024-02-02T15:49:43.919 INFO:journalctl@ceph.mds.l.smithi142.stdout:Feb 02 15:49:43 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mds-l[82172]: 2024-02-02T15:49:43.622+0000 7f1803ce0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-02T15:49:43.920 INFO:journalctl@ceph.mds.l.smithi142.stdout:Feb 02 15:49:43 smithi142 ceph-371a409c-c1de-11ee-95b6-87774f69a715-mds-l[82172]: 2024-02-02T15:49:43.622+0000 7f1803ce0700 -1 mds.l *** got signal Terminated *** 2024-02-02T15:49:47.389 INFO:journalctl@ceph.mds.l.smithi142.stdout:Feb 02 15:49:47 smithi142 bash[93110]: ceph-371a409c-c1de-11ee-95b6-87774f69a715-mds-l 2024-02-02T15:49:47.394 DEBUG:teuthology.orchestra.run.smithi142:> sudo pkill -f 'journalctl -f -n 0 -u ceph-371a409c-c1de-11ee-95b6-87774f69a715@mds.l.service' 2024-02-02T15:49:47.427 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-02T15:49:47.428 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-02-02T15:49:47.428 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 371a409c-c1de-11ee-95b6-87774f69a715 --force --keep-logs 2024-02-02T15:49:47.696 INFO:teuthology.orchestra.run.smithi001.stdout:Deleting cluster with fsid: 371a409c-c1de-11ee-95b6-87774f69a715 2024-02-02T15:49:50.699 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 371a409c-c1de-11ee-95b6-87774f69a715 --force --keep-logs 2024-02-02T15:49:50.931 INFO:teuthology.orchestra.run.smithi071.stdout:Deleting cluster with fsid: 371a409c-c1de-11ee-95b6-87774f69a715 2024-02-02T15:49:52.419 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 371a409c-c1de-11ee-95b6-87774f69a715 --force --keep-logs 2024-02-02T15:49:52.694 INFO:teuthology.orchestra.run.smithi142.stdout:Deleting cluster with fsid: 371a409c-c1de-11ee-95b6-87774f69a715 2024-02-02T15:49:54.111 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-02T15:49:54.186 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-02T15:49:54.223 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-02T15:49:54.254 INFO:tasks.cephadm:Archiving crash dumps... 2024-02-02T15:49:54.256 DEBUG:teuthology.misc:Transferring archived files from smithi001:/var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/crash to /home/teuthworker/archive/vshankar-2024-02-02_11:17:23-fs-wip-vshankar-testing1-reef-2024-02-02-1152-testing-default-smithi/7543409/remote/smithi001/crash 2024-02-02T15:49:54.257 DEBUG:teuthology.orchestra.run.smithi001:> sudo tar c -f - -C /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/crash -- . 2024-02-02T15:49:54.337 INFO:teuthology.orchestra.run.smithi001.stderr:tar: /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/crash: Cannot open: No such file or directory 2024-02-02T15:49:54.337 INFO:teuthology.orchestra.run.smithi001.stderr:tar: Error is not recoverable: exiting now 2024-02-02T15:49:54.340 DEBUG:teuthology.misc:Transferring archived files from smithi071:/var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/crash to /home/teuthworker/archive/vshankar-2024-02-02_11:17:23-fs-wip-vshankar-testing1-reef-2024-02-02-1152-testing-default-smithi/7543409/remote/smithi071/crash 2024-02-02T15:49:54.341 DEBUG:teuthology.orchestra.run.smithi071:> sudo tar c -f - -C /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/crash -- . 2024-02-02T15:49:54.421 INFO:teuthology.orchestra.run.smithi071.stderr:tar: /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/crash: Cannot open: No such file or directory 2024-02-02T15:49:54.421 INFO:teuthology.orchestra.run.smithi071.stderr:tar: Error is not recoverable: exiting now 2024-02-02T15:49:54.422 DEBUG:teuthology.misc:Transferring archived files from smithi142:/var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/crash to /home/teuthworker/archive/vshankar-2024-02-02_11:17:23-fs-wip-vshankar-testing1-reef-2024-02-02-1152-testing-default-smithi/7543409/remote/smithi142/crash 2024-02-02T15:49:54.423 DEBUG:teuthology.orchestra.run.smithi142:> sudo tar c -f - -C /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/crash -- . 2024-02-02T15:49:54.469 INFO:teuthology.orchestra.run.smithi142.stderr:tar: /var/lib/ceph/371a409c-c1de-11ee-95b6-87774f69a715/crash: Cannot open: No such file or directory 2024-02-02T15:49:54.469 INFO:teuthology.orchestra.run.smithi142.stderr:tar: Error is not recoverable: exiting now 2024-02-02T15:49:54.470 INFO:tasks.cephadm:Checking cluster log for badness... 2024-02-02T15:49:54.470 DEBUG:teuthology.orchestra.run.smithi001:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/371a409c-c1de-11ee-95b6-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-02-02T15:49:54.548 INFO:teuthology.orchestra.run.smithi001.stderr:grep: /var/log/ceph/371a409c-c1de-11ee-95b6-87774f69a715/ceph.log: No such file or directory 2024-02-02T15:49:54.553 INFO:tasks.cephadm:Compressing logs... 2024-02-02T15:49:54.553 DEBUG:teuthology.orchestra.run.smithi001:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-02T15:49:54.558 DEBUG:teuthology.orchestra.run.smithi071:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-02T15:49:54.560 DEBUG:teuthology.orchestra.run.smithi142:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-02T15:49:54.596 INFO:teuthology.orchestra.run.smithi142.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-02-02T15:49:54.605 INFO:teuthology.orchestra.run.smithi071.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-02-02T15:49:54.619 INFO:teuthology.orchestra.run.smithi001.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-02-02T15:59:09.821 INFO:tasks.cephadm:Archiving logs... 2024-02-02T15:59:09.821 DEBUG:teuthology.misc:Transferring archived files from smithi001:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-02-02_11:17:23-fs-wip-vshankar-testing1-reef-2024-02-02-1152-testing-default-smithi/7543409/remote/smithi001/log 2024-02-02T15:59:09.823 DEBUG:teuthology.orchestra.run.smithi001:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-02T15:59:38.813 DEBUG:teuthology.misc:Transferring archived files from smithi071:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-02-02_11:17:23-fs-wip-vshankar-testing1-reef-2024-02-02-1152-testing-default-smithi/7543409/remote/smithi071/log 2024-02-02T15:59:38.818 DEBUG:teuthology.orchestra.run.smithi071:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-02T16:00:41.272 DEBUG:teuthology.misc:Transferring archived files from smithi142:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-02-02_11:17:23-fs-wip-vshankar-testing1-reef-2024-02-02-1152-testing-default-smithi/7543409/remote/smithi142/log 2024-02-02T16:00:41.365 DEBUG:teuthology.orchestra.run.smithi142:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-02T16:01:52.713 INFO:tasks.cephadm:Removing cluster... 2024-02-02T16:01:52.713 DEBUG:teuthology.orchestra.run.smithi001:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 371a409c-c1de-11ee-95b6-87774f69a715 --force 2024-02-02T16:01:52.982 INFO:teuthology.orchestra.run.smithi001.stdout:Deleting cluster with fsid: 371a409c-c1de-11ee-95b6-87774f69a715 2024-02-02T16:01:54.092 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 371a409c-c1de-11ee-95b6-87774f69a715 --force 2024-02-02T16:01:54.281 INFO:teuthology.orchestra.run.smithi071.stdout:Deleting cluster with fsid: 371a409c-c1de-11ee-95b6-87774f69a715 2024-02-02T16:01:55.018 DEBUG:teuthology.orchestra.run.smithi142:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 371a409c-c1de-11ee-95b6-87774f69a715 --force 2024-02-02T16:01:55.583 INFO:teuthology.orchestra.run.smithi142.stdout:Deleting cluster with fsid: 371a409c-c1de-11ee-95b6-87774f69a715 2024-02-02T16:01:56.407 INFO:tasks.cephadm:Removing cephadm ... 2024-02-02T16:01:56.408 DEBUG:teuthology.orchestra.run.smithi001:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-02T16:01:56.454 DEBUG:teuthology.orchestra.run.smithi071:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-02T16:01:56.547 DEBUG:teuthology.orchestra.run.smithi142:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-02T16:01:56.583 INFO:tasks.cephadm:Teardown complete 2024-02-02T16:01:56.583 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-02-02T16:01:56.594 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-02-02T16:01:56.594 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-02-02T16:01:56.598 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-02-02T16:01:56.601 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-02-02T16:01:56.739 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-02-02T16:01:56.750 INFO:teuthology.task.clock:Checking final clock skew... 2024-02-02T16:01:56.750 DEBUG:teuthology.orchestra.run.smithi001:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-02T16:01:56.755 DEBUG:teuthology.orchestra.run.smithi071:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-02T16:01:56.758 DEBUG:teuthology.orchestra.run.smithi142:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-02T16:01:56.774 INFO:teuthology.orchestra.run.smithi142.stderr:bash: ntpq: command not found 2024-02-02T16:01:56.775 INFO:teuthology.orchestra.run.smithi071.stderr:bash: ntpq: command not found 2024-02-02T16:01:56.792 INFO:teuthology.orchestra.run.smithi001.stderr:bash: ntpq: command not found 2024-02-02T16:01:56.799 INFO:teuthology.orchestra.run.smithi001.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-02T16:01:56.800 INFO:teuthology.orchestra.run.smithi001.stdout:=============================================================================== 2024-02-02T16:01:56.800 INFO:teuthology.orchestra.run.smithi001.stdout:^- hv01.front.sepia.ceph.com 2 6 377 13 +1910us[+1923us] +/- 1016ms 2024-02-02T16:01:56.800 INFO:teuthology.orchestra.run.smithi001.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 11 +537us[ +550us] +/- 77ms 2024-02-02T16:01:56.800 INFO:teuthology.orchestra.run.smithi001.stdout:^* hv03.front.sepia.ceph.com 3 6 377 11 -425us[ -412us] +/- 69ms 2024-02-02T16:01:56.801 INFO:teuthology.orchestra.run.smithi001.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-02-02T16:01:56.812 INFO:teuthology.orchestra.run.smithi071.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-02T16:01:56.812 INFO:teuthology.orchestra.run.smithi071.stdout:=============================================================================== 2024-02-02T16:01:56.812 INFO:teuthology.orchestra.run.smithi071.stdout:^* hv01.front.sepia.ceph.com 2 6 377 10 +1187us[+1203us] +/- 1016ms 2024-02-02T16:01:56.813 INFO:teuthology.orchestra.run.smithi071.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 12 -205us[ -189us] +/- 77ms 2024-02-02T16:01:56.813 INFO:teuthology.orchestra.run.smithi071.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 12 -1146us[-1130us] +/- 69ms 2024-02-02T16:01:56.813 INFO:teuthology.orchestra.run.smithi071.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-02-02T16:01:56.814 INFO:teuthology.orchestra.run.smithi142.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-02T16:01:56.814 INFO:teuthology.orchestra.run.smithi142.stdout:=============================================================================== 2024-02-02T16:01:56.815 INFO:teuthology.orchestra.run.smithi142.stdout:^- hv01.front.sepia.ceph.com 2 6 377 8 +1887us[+1887us] +/- 1016ms 2024-02-02T16:01:56.815 INFO:teuthology.orchestra.run.smithi142.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 10 +489us[ +514us] +/- 77ms 2024-02-02T16:01:56.815 INFO:teuthology.orchestra.run.smithi142.stdout:^* hv03.front.sepia.ceph.com 3 6 377 9 -431us[ -407us] +/- 69ms 2024-02-02T16:01:56.815 INFO:teuthology.orchestra.run.smithi142.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-02-02T16:01:56.816 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-02-02T16:01:56.827 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-02-02T16:01:56.828 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-02-02T16:01:56.850 DEBUG:teuthology.orchestra.run.smithi001:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-02T16:01:57.000 DEBUG:teuthology.orchestra.run.smithi071:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-02T16:01:57.113 DEBUG:teuthology.orchestra.run.smithi142:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-02T16:01:57.189 DEBUG:teuthology.orchestra.run.smithi001:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-02T16:01:57.260 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-02T16:01:57.260 DEBUG:teuthology.orchestra.run.smithi071:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-02T16:01:57.294 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-02T16:01:57.294 DEBUG:teuthology.orchestra.run.smithi142:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-02T16:01:57.324 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-02T16:01:57.327 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-02-02T16:01:57.338 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-02-02T16:01:57.383 INFO:teuthology.task.internal:Duration was 3391.691789 seconds 2024-02-02T16:01:57.384 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-02-02T16:01:57.417 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-02-02T16:01:57.417 DEBUG:teuthology.orchestra.run.smithi001:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-02T16:01:57.421 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-02T16:01:57.424 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-02T16:01:57.502 INFO:teuthology.orchestra.run.smithi071.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-02T16:01:57.504 INFO:teuthology.orchestra.run.smithi142.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-02T16:01:57.531 INFO:teuthology.orchestra.run.smithi001.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-02T16:01:57.970 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-02-02T16:01:57.970 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi001.front.sepia.ceph.com 2024-02-02T16:01:57.971 DEBUG:teuthology.orchestra.run.smithi001:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-02T16:01:58.028 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi071.front.sepia.ceph.com 2024-02-02T16:01:58.028 DEBUG:teuthology.orchestra.run.smithi071:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-02T16:01:58.058 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi142.front.sepia.ceph.com 2024-02-02T16:01:58.058 DEBUG:teuthology.orchestra.run.smithi142:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-02T16:01:58.089 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-02-02T16:01:58.089 DEBUG:teuthology.orchestra.run.smithi001:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-02T16:01:58.094 DEBUG:teuthology.orchestra.run.smithi071:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-02T16:01:58.101 DEBUG:teuthology.orchestra.run.smithi142:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-02T16:01:58.196 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-02-02T16:01:58.196 DEBUG:teuthology.orchestra.run.smithi001:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-02T16:01:58.200 DEBUG:teuthology.orchestra.run.smithi071:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-02T16:01:58.202 DEBUG:teuthology.orchestra.run.smithi142:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-02T16:01:58.380 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-02-02T16:01:58.393 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-02-02T16:01:58.394 DEBUG:teuthology.orchestra.run.smithi001:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-02T16:01:58.460 DEBUG:teuthology.orchestra.run.smithi071:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-02T16:01:58.512 DEBUG:teuthology.orchestra.run.smithi142:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-02T16:01:58.542 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-02-02T16:01:58.558 DEBUG:teuthology.orchestra.run.smithi001:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-02T16:01:58.563 DEBUG:teuthology.orchestra.run.smithi071:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-02T16:01:58.565 DEBUG:teuthology.orchestra.run.smithi142:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-02T16:01:58.596 INFO:teuthology.orchestra.run.smithi071.stdout:kernel.core_pattern = core 2024-02-02T16:01:58.623 INFO:teuthology.orchestra.run.smithi001.stdout:kernel.core_pattern = core 2024-02-02T16:01:58.642 INFO:teuthology.orchestra.run.smithi142.stdout:kernel.core_pattern = core 2024-02-02T16:01:58.658 DEBUG:teuthology.orchestra.run.smithi001:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-02T16:01:58.697 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-02T16:01:58.697 DEBUG:teuthology.orchestra.run.smithi071:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-02T16:01:58.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-02T16:01:58.714 DEBUG:teuthology.orchestra.run.smithi142:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-02T16:01:58.730 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-02T16:01:58.731 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-02-02T16:01:58.743 INFO:teuthology.task.internal:Transferring archived files... 2024-02-02T16:01:58.744 DEBUG:teuthology.misc:Transferring archived files from smithi001:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-02-02_11:17:23-fs-wip-vshankar-testing1-reef-2024-02-02-1152-testing-default-smithi/7543409/remote/smithi001 2024-02-02T16:01:58.744 DEBUG:teuthology.orchestra.run.smithi001:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-02T16:01:58.829 DEBUG:teuthology.misc:Transferring archived files from smithi071:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-02-02_11:17:23-fs-wip-vshankar-testing1-reef-2024-02-02-1152-testing-default-smithi/7543409/remote/smithi071 2024-02-02T16:01:58.829 DEBUG:teuthology.orchestra.run.smithi071:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-02T16:01:58.870 DEBUG:teuthology.misc:Transferring archived files from smithi142:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-02-02_11:17:23-fs-wip-vshankar-testing1-reef-2024-02-02-1152-testing-default-smithi/7543409/remote/smithi142 2024-02-02T16:01:58.870 DEBUG:teuthology.orchestra.run.smithi142:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-02T16:01:58.912 INFO:teuthology.task.internal:Removing archive directory... 2024-02-02T16:01:58.912 DEBUG:teuthology.orchestra.run.smithi001:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-02T16:01:58.917 DEBUG:teuthology.orchestra.run.smithi071:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-02T16:01:58.920 DEBUG:teuthology.orchestra.run.smithi142:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-02T16:01:58.960 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-02-02T16:01:58.978 INFO:teuthology.task.internal:Not uploading archives. 2024-02-02T16:01:58.978 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-02-02T16:01:59.018 INFO:teuthology.task.internal:Tidying up after the test... 2024-02-02T16:01:59.018 DEBUG:teuthology.orchestra.run.smithi001:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-02T16:01:59.023 DEBUG:teuthology.orchestra.run.smithi071:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-02T16:01:59.025 DEBUG:teuthology.orchestra.run.smithi142:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-02T16:01:59.039 INFO:teuthology.orchestra.run.smithi071.stdout: 262157 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 2 16:01 /home/ubuntu/cephtest 2024-02-02T16:01:59.042 INFO:teuthology.orchestra.run.smithi142.stdout: 262157 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 2 16:01 /home/ubuntu/cephtest 2024-02-02T16:01:59.060 INFO:teuthology.orchestra.run.smithi001.stdout: 524301 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 2 16:01 /home/ubuntu/cephtest 2024-02-02T16:01:59.063 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-02-02T16:01:59.077 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-02-02T16:01:59.135 INFO:teuthology.nuke:Checking targets against current locks 2024-02-02T16:01:59.161 DEBUG:teuthology.nuke:shortname: smithi001 2024-02-02T16:01:59.162 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-02T16:01:59.187 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi001.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-02-02_11:17:23-fs-wip-vshankar-testing1-reef-2024-02-02-1152-testing-default-smithi/7543409', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-02 14:54:37.324227', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBH1rn2VC4T8VnaJuVwz+S1DdI4E7WVLu7YZZWz0pzYDe25xLhAUK2WtqwrPTA3U3fIB0taj0HtHpwttu1J9yK+U='} 2024-02-02T16:01:59.192 DEBUG:teuthology.nuke:shortname: smithi071 2024-02-02T16:01:59.192 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-02T16:01:59.215 DEBUG:teuthology.nuke:shortname: smithi142 2024-02-02T16:01:59.215 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-02T16:01:59.221 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi071.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-02-02_11:17:23-fs-wip-vshankar-testing1-reef-2024-02-02-1152-testing-default-smithi/7543409', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-02 14:54:37.326251', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBA4b+FN1PwkTwL2SUBH4cuQNZlKGLXLMASayfQMdZpKqmE9KMIwLXafmB8FDCeE6DOZEP1cBujIXfptse6F6jj0='} 2024-02-02T16:01:59.248 INFO:teuthology.orchestra.console.smithi001:Power off 2024-02-02T16:01:59.248 DEBUG:teuthology.orchestra.console.smithi001:pexpect command: ipmitool -H smithi001.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-02T16:01:59.262 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi142.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-02-02_11:17:23-fs-wip-vshankar-testing1-reef-2024-02-02-1152-testing-default-smithi/7543409', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-02 14:54:37.327759', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJ+9kvY0NLYoBysi0hGw7rDPk0gJW8sqe6LiuU2miCRPy1oChyg3cTXZufOoR0OzCMCGWpCi60qpdFaEulx7JFw='} 2024-02-02T16:01:59.292 INFO:teuthology.orchestra.console.smithi071:Power off 2024-02-02T16:01:59.293 DEBUG:teuthology.orchestra.console.smithi071:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-02T16:01:59.307 DEBUG:teuthology.orchestra.console.smithi001:power off output: Chassis Power Control: Down/Off 2024-02-02T16:01:59.308 DEBUG:teuthology.orchestra.console.smithi001:pexpect command: ipmitool -H smithi001.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-02T16:01:59.319 DEBUG:teuthology.orchestra.console.smithi071:power off output: Chassis Power Control: Down/Off 2024-02-02T16:01:59.319 DEBUG:teuthology.orchestra.console.smithi071:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-02T16:01:59.346 INFO:teuthology.orchestra.console.smithi142:Power off 2024-02-02T16:01:59.347 DEBUG:teuthology.orchestra.console.smithi142:pexpect command: ipmitool -H smithi142.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-02T16:01:59.356 DEBUG:teuthology.orchestra.console.smithi001:check power output: Chassis Power is on 2024-02-02T16:01:59.356 DEBUG:teuthology.orchestra.console.smithi071:check power output: Chassis Power is on 2024-02-02T16:01:59.369 DEBUG:teuthology.orchestra.console.smithi142:power off output: Chassis Power Control: Down/Off 2024-02-02T16:01:59.369 DEBUG:teuthology.orchestra.console.smithi142:pexpect command: ipmitool -H smithi142.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-02T16:01:59.394 DEBUG:teuthology.orchestra.console.smithi142:check power output: Chassis Power is on 2024-02-02T16:02:03.357 DEBUG:teuthology.orchestra.console.smithi001:pexpect command: ipmitool -H smithi001.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-02T16:02:03.367 DEBUG:teuthology.orchestra.console.smithi071:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-02T16:02:03.395 DEBUG:teuthology.orchestra.console.smithi142:pexpect command: ipmitool -H smithi142.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-02T16:02:03.470 DEBUG:teuthology.orchestra.console.smithi001:check power output: Chassis Power is on 2024-02-02T16:02:03.479 DEBUG:teuthology.orchestra.console.smithi071:check power output: Chassis Power is on 2024-02-02T16:02:03.507 DEBUG:teuthology.orchestra.console.smithi142:check power output: Chassis Power is on 2024-02-02T16:02:07.472 DEBUG:teuthology.orchestra.console.smithi001:pexpect command: ipmitool -H smithi001.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-02T16:02:07.481 DEBUG:teuthology.orchestra.console.smithi071:pexpect command: ipmitool -H smithi071.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-02T16:02:07.508 DEBUG:teuthology.orchestra.console.smithi142:pexpect command: ipmitool -H smithi142.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-02T16:02:07.583 DEBUG:teuthology.orchestra.console.smithi001:check power output: Chassis Power is off 2024-02-02T16:02:07.591 DEBUG:teuthology.orchestra.console.smithi071:check power output: Chassis Power is off 2024-02-02T16:02:07.619 DEBUG:teuthology.orchestra.console.smithi142:check power output: Chassis Power is off 2024-02-02T16:02:07.684 INFO:teuthology.orchestra.console.smithi001:Power off completed 2024-02-02T16:02:07.692 INFO:teuthology.orchestra.console.smithi071:Power off completed 2024-02-02T16:02:07.719 INFO:teuthology.orchestra.console.smithi142:Power off completed 2024-02-02T16:02:07.827 INFO:teuthology.run:Summary data: client.0-kernel-sha1: cd42140e628558a450ba1d39a9d0835a2fa62cda description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-workunit/suites/dbench}} duration: 3391.6917893886566 failure_reason: 'Command failed (workunit test suites/dbench.sh) on smithi001 with status 1: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=0fb327883c124fd83155c94db0804f823b805a12 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh''' flavor: default owner: scheduled_vshankar@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=6e95807c1d0f4617a7189c92498fb1ac status: fail success: false 2024-02-02T16:02:07.827 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-02T16:02:07.908 INFO:teuthology.run:FAIL